2024-09-15T23:57:19.904 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T23:57:19.927 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T23:57:20.012 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906374 branch: squid description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906374' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub - but it is still running - objects unfound and apparently lost - MDS_SLOW_METADATA_IO - MDS_TRIM sha1: 8921c8ef947807b717db041d85f0761b7914d5fc ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi139.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF80ej5qAloOHRRNdDb5hB76D13JReZBWyWPm8uPZnzBQQCLb9RA3lm0ujaOHUpSVuyB3zts7RTwolkrrPznXvY= smithi164.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDhiPGctwJicHDFmkt2NbbZfku1RX+7WSVleApUhlwR0NVYMt+aa0ZeF91TCgfpO7JKhV13Yez3x+rGQgU3V4Jw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - thrashosds: null - workunit: clients: all: - suites/fsstress.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T23:57:20.012 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-15T23:57:20.013 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-15T23:57:20.013 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T23:57:20.015 INFO:teuthology.task.internal:Checking packages... 2024-09-15T23:57:20.036 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-15T23:57:20.036 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T23:57:20.037 INFO:teuthology.packaging:ref: None 2024-09-15T23:57:20.037 INFO:teuthology.packaging:tag: None 2024-09-15T23:57:20.037 INFO:teuthology.packaging:branch: squid 2024-09-15T23:57:20.037 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T23:57:20.037 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-15T23:57:20.233 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-328.gad35db06 2024-09-15T23:57:20.235 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T23:57:20.243 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T23:57:20.243 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T23:57:20.251 INFO:teuthology.task.internal:Saving configuration 2024-09-15T23:57:20.268 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T23:57:20.275 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T23:57:20.305 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906374', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 23:52:07.697396', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF80ej5qAloOHRRNdDb5hB76D13JReZBWyWPm8uPZnzBQQCLb9RA3lm0ujaOHUpSVuyB3zts7RTwolkrrPznXvY='} 2024-09-15T23:57:20.329 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906374', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 23:52:07.698485', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDhiPGctwJicHDFmkt2NbbZfku1RX+7WSVleApUhlwR0NVYMt+aa0ZeF91TCgfpO7JKhV13Yez3x+rGQgU3V4Jw='} 2024-09-15T23:57:20.329 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T23:57:20.337 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-09-15T23:57:20.338 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-09-15T23:57:20.338 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T23:57:20.398 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f76ae74f490>, signals=[15]) 2024-09-15T23:57:20.399 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T23:57:20.405 INFO:teuthology.task.internal:Opening connections... 2024-09-15T23:57:20.405 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi139.front.sepia.ceph.com 2024-09-15T23:57:20.406 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:57:20.484 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2024-09-15T23:57:20.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:57:20.560 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T23:57:20.567 DEBUG:teuthology.orchestra.run.smithi139:> uname -m 2024-09-15T23:57:20.585 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2024-09-15T23:57:20.585 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2024-09-15T23:57:20.642 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="CentOS Stream" 2024-09-15T23:57:20.642 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="9" 2024-09-15T23:57:20.643 INFO:teuthology.orchestra.run.smithi139.stdout:ID="centos" 2024-09-15T23:57:20.643 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="rhel fedora" 2024-09-15T23:57:20.643 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="9" 2024-09-15T23:57:20.643 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el9" 2024-09-15T23:57:20.643 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T23:57:20.643 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2024-09-15T23:57:20.643 INFO:teuthology.orchestra.run.smithi139.stdout:LOGO="fedora-logo-icon" 2024-09-15T23:57:20.643 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T23:57:20.643 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://centos.org/" 2024-09-15T23:57:20.643 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T23:57:20.643 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T23:57:20.643 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T23:57:20.644 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2024-09-15T23:57:20.669 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2024-09-15T23:57:20.687 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2024-09-15T23:57:20.688 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2024-09-15T23:57:20.746 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="CentOS Stream" 2024-09-15T23:57:20.746 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="9" 2024-09-15T23:57:20.747 INFO:teuthology.orchestra.run.smithi164.stdout:ID="centos" 2024-09-15T23:57:20.747 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="rhel fedora" 2024-09-15T23:57:20.747 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="9" 2024-09-15T23:57:20.747 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el9" 2024-09-15T23:57:20.747 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T23:57:20.747 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2024-09-15T23:57:20.747 INFO:teuthology.orchestra.run.smithi164.stdout:LOGO="fedora-logo-icon" 2024-09-15T23:57:20.747 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T23:57:20.747 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://centos.org/" 2024-09-15T23:57:20.747 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T23:57:20.747 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T23:57:20.747 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T23:57:20.748 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2024-09-15T23:57:20.769 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T23:57:20.778 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T23:57:20.783 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T23:57:20.784 DEBUG:teuthology.orchestra.run.smithi139:> test '!' -e /home/ubuntu/cephtest 2024-09-15T23:57:20.786 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2024-09-15T23:57:20.803 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T23:57:20.809 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T23:57:20.809 DEBUG:teuthology.orchestra.run.smithi139:> test -z $(ls -A /var/lib/ceph) 2024-09-15T23:57:20.844 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2024-09-15T23:57:21.107 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T23:57:21.160 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T23:57:21.174 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-15T23:57:21.175 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-15T23:57:21.175 DEBUG:teuthology.orchestra.run.smithi139:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T23:57:21.175 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T23:57:21.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:57:21.192 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-09-15T23:57:21.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:57:21.193 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-09-15T23:57:21.250 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-503.el9.x86_64 2024-09-15T23:57:21.250 INFO:teuthology.task.kernel:Running kernel on smithi139: 5.14.0-503.el9.x86_64 2024-09-15T23:57:21.250 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum install -y kernel 2024-09-15T23:57:21.252 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-503.el9.x86_64 2024-09-15T23:57:21.253 INFO:teuthology.task.kernel:Running kernel on smithi164: 5.14.0-503.el9.x86_64 2024-09-15T23:57:21.253 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum install -y kernel 2024-09-15T23:57:23.731 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-15T23:57:23.978 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 8.4 MB/s | 8.2 MB 00:00 2024-09-15T23:57:25.482 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 32 MB/s | 20 MB 00:00 2024-09-15T23:57:26.312 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 17 MB/s | 20 MB 00:01 2024-09-15T23:57:30.207 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 6.5 MB 00:00 2024-09-15T23:57:31.537 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.5 MB 00:01 2024-09-15T23:57:31.641 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 85 kB/s | 19 kB 00:00 2024-09-15T23:57:32.512 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-15T23:57:33.029 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 19 kB 00:00 2024-09-15T23:57:34.050 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-15T23:57:37.952 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-15T23:57:39.481 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-15T23:57:39.975 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T23:57:39.975 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T23:57:40.078 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-15T23:57:40.086 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-15T23:57:40.086 INFO:teuthology.orchestra.run.smithi164.stdout: Package Architecture Version Repository Size 2024-09-15T23:57:40.086 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-15T23:57:40.086 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-15T23:57:40.086 INFO:teuthology.orchestra.run.smithi164.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T23:57:40.086 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-15T23:57:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T23:57:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T23:57:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T23:57:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-15T23:57:40.087 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-15T23:57:40.087 INFO:teuthology.orchestra.run.smithi164.stdout:================================================================================ 2024-09-15T23:57:40.087 INFO:teuthology.orchestra.run.smithi164.stdout:Install 4 Packages 2024-09-15T23:57:40.087 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-15T23:57:40.087 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 79 M 2024-09-15T23:57:40.087 INFO:teuthology.orchestra.run.smithi164.stdout:Installed size: 126 M 2024-09-15T23:57:40.087 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-15T23:57:40.294 INFO:teuthology.orchestra.run.smithi164.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 285 kB/s | 23 kB 00:00 2024-09-15T23:57:41.011 INFO:teuthology.orchestra.run.smithi164.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-15T23:57:41.403 INFO:teuthology.orchestra.run.smithi164.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 29 MB/s | 35 MB 00:01 2024-09-15T23:57:41.437 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T23:57:41.437 INFO:teuthology.orchestra.run.smithi139.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T23:57:41.522 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-15T23:57:41.531 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-15T23:57:41.531 INFO:teuthology.orchestra.run.smithi139.stdout: Package Architecture Version Repository Size 2024-09-15T23:57:41.531 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-15T23:57:41.531 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-15T23:57:41.531 INFO:teuthology.orchestra.run.smithi139.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T23:57:41.531 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-15T23:57:41.531 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T23:57:41.531 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T23:57:41.532 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T23:57:41.532 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-15T23:57:41.532 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-15T23:57:41.532 INFO:teuthology.orchestra.run.smithi139.stdout:================================================================================ 2024-09-15T23:57:41.532 INFO:teuthology.orchestra.run.smithi139.stdout:Install 4 Packages 2024-09-15T23:57:41.532 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-15T23:57:41.532 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 79 M 2024-09-15T23:57:41.533 INFO:teuthology.orchestra.run.smithi139.stdout:Installed size: 126 M 2024-09-15T23:57:41.533 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-15T23:57:41.807 INFO:teuthology.orchestra.run.smithi139.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 151 kB/s | 23 kB 00:00 2024-09-15T23:57:42.928 INFO:teuthology.orchestra.run.smithi164.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 11 MB/s | 29 MB 00:02 2024-09-15T23:57:42.929 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-15T23:57:42.929 INFO:teuthology.orchestra.run.smithi164.stdout:Total 28 MB/s | 79 MB 00:02 2024-09-15T23:57:43.007 INFO:teuthology.orchestra.run.smithi139.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-09-15T23:57:43.574 INFO:teuthology.orchestra.run.smithi139.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 16 MB/s | 29 MB 00:01 2024-09-15T23:57:43.611 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-15T23:57:43.960 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-15T23:57:43.960 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-15T23:57:43.983 INFO:teuthology.orchestra.run.smithi139.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 15 MB/s | 35 MB 00:02 2024-09-15T23:57:43.984 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-15T23:57:43.984 INFO:teuthology.orchestra.run.smithi139.stdout:Total 32 MB/s | 79 MB 00:02 2024-09-15T23:57:44.664 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-15T23:57:44.909 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-15T23:57:44.909 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-15T23:57:45.006 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-15T23:57:45.006 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-15T23:57:45.968 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-15T23:57:45.969 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-15T23:57:47.366 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-15T23:57:48.097 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T23:57:48.195 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:57:48.432 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-15T23:57:49.067 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T23:57:49.149 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:57:50.993 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:57:51.131 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:57:51.940 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:57:52.094 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:57:55.970 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:57:56.098 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:57:56.951 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:57:56.994 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:58:00.950 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:58:01.822 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:58:39.098 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:58:39.138 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:58:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:58:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T23:58:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:58:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:58:39.852 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:58:39.892 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:58:40.261 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:58:40.262 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T23:58:40.262 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T23:58:40.262 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T23:58:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:58:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-15T23:58:40.367 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-15T23:58:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T23:58:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T23:58:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T23:58:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T23:58:40.368 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-15T23:58:40.368 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-15T23:58:40.737 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T23:58:40.831 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T23:58:40.832 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T23:58:40.832 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T23:58:40.832 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T23:58:40.832 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T23:58:40.832 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-15T23:58:40.832 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi164.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T23:58:40.832 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T23:58:40.891 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T23:58:40.891 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T23:58:40.892 DEBUG:teuthology.orchestra.run.smithi164:> sudo rpm -qi grub2-tools 2024-09-15T23:58:40.927 INFO:teuthology.orchestra.run.smithi164.stdout:Name : grub2-tools 2024-09-15T23:58:40.927 INFO:teuthology.orchestra.run.smithi164.stdout:Epoch : 1 2024-09-15T23:58:40.927 INFO:teuthology.orchestra.run.smithi164.stdout:Version : 2.06 2024-09-15T23:58:40.928 INFO:teuthology.orchestra.run.smithi164.stdout:Release : 82.el9 2024-09-15T23:58:40.928 INFO:teuthology.orchestra.run.smithi164.stdout:Architecture: x86_64 2024-09-15T23:58:40.928 INFO:teuthology.orchestra.run.smithi164.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T23:58:40.928 INFO:teuthology.orchestra.run.smithi164.stdout:Group : Unspecified 2024-09-15T23:58:40.928 INFO:teuthology.orchestra.run.smithi164.stdout:Size : 8274670 2024-09-15T23:58:40.928 INFO:teuthology.orchestra.run.smithi164.stdout:License : GPLv3+ 2024-09-15T23:58:40.928 INFO:teuthology.orchestra.run.smithi164.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T23:58:40.928 INFO:teuthology.orchestra.run.smithi164.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T23:58:40.928 INFO:teuthology.orchestra.run.smithi164.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T23:58:40.928 INFO:teuthology.orchestra.run.smithi164.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T23:58:40.928 INFO:teuthology.orchestra.run.smithi164.stdout:Packager : builder@centos.org 2024-09-15T23:58:40.928 INFO:teuthology.orchestra.run.smithi164.stdout:Vendor : CentOS 2024-09-15T23:58:40.928 INFO:teuthology.orchestra.run.smithi164.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T23:58:40.929 INFO:teuthology.orchestra.run.smithi164.stdout:Summary : Support tools for GRUB. 2024-09-15T23:58:40.929 INFO:teuthology.orchestra.run.smithi164.stdout:Description : 2024-09-15T23:58:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-15T23:58:40.929 INFO:teuthology.orchestra.run.smithi164.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T23:58:40.929 INFO:teuthology.orchestra.run.smithi164.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T23:58:40.929 INFO:teuthology.orchestra.run.smithi164.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T23:58:40.929 INFO:teuthology.orchestra.run.smithi164.stdout:hardware devices. 2024-09-15T23:58:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-15T23:58:40.929 INFO:teuthology.orchestra.run.smithi164.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T23:58:40.930 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T23:58:40.930 INFO:teuthology.task.kernel:Updating grub on smithi164 to boot 5.14.0-505.el9.x86_64 2024-09-15T23:58:40.930 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T23:58:41.099 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T23:58:41.099 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-15T23:58:41.099 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-15T23:58:41.099 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T23:58:41.099 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T23:58:41.099 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T23:58:41.099 INFO:teuthology.orchestra.run.smithi139.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T23:58:41.099 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-15T23:58:41.099 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-15T23:58:41.482 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T23:58:41.571 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T23:58:41.571 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T23:58:41.571 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T23:58:41.572 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T23:58:41.572 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T23:58:41.572 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-15T23:58:41.572 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi139.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T23:58:41.572 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T23:58:41.579 INFO:teuthology.orchestra.run.smithi164.stderr:Generating grub configuration file ... 2024-09-15T23:58:41.629 INFO:teuthology.orchestra.run.smithi139.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T23:58:41.630 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T23:58:41.631 DEBUG:teuthology.orchestra.run.smithi139:> sudo rpm -qi grub2-tools 2024-09-15T23:58:41.662 INFO:teuthology.orchestra.run.smithi139.stdout:Name : grub2-tools 2024-09-15T23:58:41.663 INFO:teuthology.orchestra.run.smithi139.stdout:Epoch : 1 2024-09-15T23:58:41.663 INFO:teuthology.orchestra.run.smithi139.stdout:Version : 2.06 2024-09-15T23:58:41.663 INFO:teuthology.orchestra.run.smithi139.stdout:Release : 82.el9 2024-09-15T23:58:41.663 INFO:teuthology.orchestra.run.smithi139.stdout:Architecture: x86_64 2024-09-15T23:58:41.663 INFO:teuthology.orchestra.run.smithi139.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T23:58:41.663 INFO:teuthology.orchestra.run.smithi139.stdout:Group : Unspecified 2024-09-15T23:58:41.663 INFO:teuthology.orchestra.run.smithi139.stdout:Size : 8274670 2024-09-15T23:58:41.663 INFO:teuthology.orchestra.run.smithi139.stdout:License : GPLv3+ 2024-09-15T23:58:41.663 INFO:teuthology.orchestra.run.smithi139.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T23:58:41.663 INFO:teuthology.orchestra.run.smithi139.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T23:58:41.663 INFO:teuthology.orchestra.run.smithi139.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T23:58:41.663 INFO:teuthology.orchestra.run.smithi139.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T23:58:41.664 INFO:teuthology.orchestra.run.smithi139.stdout:Packager : builder@centos.org 2024-09-15T23:58:41.664 INFO:teuthology.orchestra.run.smithi139.stdout:Vendor : CentOS 2024-09-15T23:58:41.664 INFO:teuthology.orchestra.run.smithi139.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T23:58:41.664 INFO:teuthology.orchestra.run.smithi139.stdout:Summary : Support tools for GRUB. 2024-09-15T23:58:41.664 INFO:teuthology.orchestra.run.smithi139.stdout:Description : 2024-09-15T23:58:41.664 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-15T23:58:41.664 INFO:teuthology.orchestra.run.smithi139.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T23:58:41.664 INFO:teuthology.orchestra.run.smithi139.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T23:58:41.664 INFO:teuthology.orchestra.run.smithi139.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T23:58:41.664 INFO:teuthology.orchestra.run.smithi139.stdout:hardware devices. 2024-09-15T23:58:41.664 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-15T23:58:41.664 INFO:teuthology.orchestra.run.smithi139.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T23:58:41.665 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T23:58:41.665 INFO:teuthology.task.kernel:Updating grub on smithi139 to boot 5.14.0-505.el9.x86_64 2024-09-15T23:58:41.665 DEBUG:teuthology.orchestra.run.smithi139:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T23:58:42.306 INFO:teuthology.orchestra.run.smithi139.stderr:Generating grub configuration file ... 2024-09-15T23:58:42.889 INFO:teuthology.orchestra.run.smithi164.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T23:58:42.923 INFO:teuthology.orchestra.run.smithi164.stderr:done 2024-09-15T23:58:42.925 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-09-15T23:58:42.941 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.bxB6i4hHep 2024-09-15T23:58:42.942 DEBUG:teuthology.orchestra.run.smithi164:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bxB6i4hHep 2024-09-15T23:58:43.008 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.bxB6i4hHep 2024-09-15T23:58:43.143 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.bxB6i4hHep is 6KB 2024-09-15T23:58:43.191 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.bxB6i4hHep 2024-09-15T23:58:43.206 DEBUG:teuthology.orchestra.run.smithi164:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T23:58:43.269 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T23:58:43.269 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T23:58:43.269 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T23:58:43.269 INFO:teuthology.orchestra.run.smithi164.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T23:58:43.270 DEBUG:teuthology.orchestra.run.smithi164:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T23:58:43.431 DEBUG:teuthology.orchestra.run.smithi164:> sudo shutdown -r now 2024-09-15T23:58:43.662 INFO:teuthology.orchestra.run.smithi139.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T23:58:43.693 INFO:teuthology.orchestra.run.smithi139.stderr:done 2024-09-15T23:58:43.695 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2024-09-15T23:58:43.712 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.D9XIDyCsIw 2024-09-15T23:58:43.712 DEBUG:teuthology.orchestra.run.smithi139:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.D9XIDyCsIw 2024-09-15T23:58:43.782 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.D9XIDyCsIw 2024-09-15T23:58:43.924 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.D9XIDyCsIw is 6KB 2024-09-15T23:58:43.973 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.D9XIDyCsIw 2024-09-15T23:58:43.988 DEBUG:teuthology.orchestra.run.smithi139:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T23:58:44.052 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T23:58:44.052 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T23:58:44.052 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T23:58:44.052 INFO:teuthology.orchestra.run.smithi139.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T23:58:44.053 DEBUG:teuthology.orchestra.run.smithi139:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T23:58:44.226 DEBUG:teuthology.orchestra.run.smithi139:> sudo shutdown -r now 2024-09-15T23:59:13.461 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-15T23:59:13.461 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-15T23:59:13.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:59:14.229 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-15T23:59:14.229 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-15T23:59:14.229 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T23:59:48.029 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.164 2024-09-15T23:59:57.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-15T23:59:57.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:00:14.246 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T00:00:23.253 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T00:00:23.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:00:25.026 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-09-16T00:00:25.360 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T00:00:25.360 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-16T00:00:25.360 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-09-16T00:00:25.377 INFO:teuthology.orchestra.run.smithi139.stdout:5.14.0-505.el9.x86_64 2024-09-16T00:00:25.378 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T00:00:25.378 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T00:00:25.378 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T00:00:26.378 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T00:00:26.379 DEBUG:teuthology.orchestra.run.smithi139:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T00:00:26.484 INFO:teuthology.orchestra.run.smithi139.stdout:ttyS1 2024-09-16T00:00:26.514 DEBUG:teuthology.parallel:result is None 2024-09-16T00:00:57.037 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T00:01:09.038 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T00:01:09.039 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:01:09.389 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-16T00:01:09.780 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T00:01:09.780 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-16T00:01:09.781 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2024-09-16T00:01:09.797 INFO:teuthology.orchestra.run.smithi164.stdout:5.14.0-505.el9.x86_64 2024-09-16T00:01:09.798 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T00:01:09.798 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T00:01:09.798 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T00:01:10.799 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-16T00:01:10.799 DEBUG:teuthology.orchestra.run.smithi164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T00:01:10.901 INFO:teuthology.orchestra.run.smithi164.stdout:ttyS1 2024-09-16T00:01:10.922 DEBUG:teuthology.parallel:result is None 2024-09-16T00:01:10.923 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T00:01:10.951 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T00:01:10.951 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T00:01:10.955 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T00:01:10.973 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T00:01:10.984 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T00:01:10.991 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T00:01:10.991 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T00:01:11.015 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T00:01:11.133 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T00:01:11.144 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T00:01:11.145 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T00:01:11.147 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T00:01:11.181 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:01:11.190 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:01:11.192 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:01:11.202 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:01:11.204 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T00:01:11.232 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T00:01:11.232 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T00:01:11.235 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T00:01:11.263 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T00:01:11.279 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T00:01:11.279 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T00:01:11.303 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T00:01:11.320 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:01:11.405 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:01:11.478 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:01:11.523 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:01:11.562 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:01:11.562 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T00:01:11.628 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:01:11.676 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:01:11.759 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:01:11.804 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:01:11.847 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:01:11.847 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T00:01:11.915 DEBUG:teuthology.orchestra.run.smithi139:> sudo service rsyslog restart 2024-09-16T00:01:11.917 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2024-09-16T00:01:11.956 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:01:12.005 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:01:12.444 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T00:01:12.472 INFO:teuthology.task.internal:Starting timer... 2024-09-16T00:01:12.472 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T00:01:12.487 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T00:01:12.494 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-16T00:01:12.494 DEBUG:teuthology.orchestra.run.smithi139:> sudo service auditd rotate 2024-09-16T00:01:12.562 INFO:teuthology.orchestra.run.smithi139.stdout:Rotating logs: 2024-09-16T00:01:12.564 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2024-09-16T00:01:12.644 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2024-09-16T00:01:12.646 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T00:01:12.646 DEBUG:teuthology.orchestra.run.smithi139:> /usr/sbin/getenforce 2024-09-16T00:01:12.676 INFO:teuthology.orchestra.run.smithi139.stdout:Permissive 2024-09-16T00:01:12.677 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2024-09-16T00:01:12.709 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2024-09-16T00:01:12.709 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi139.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi164.front.sepia.ceph.com': 'permissive'} 2024-09-16T00:01:12.710 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:01:12.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:01:12.743 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:01:12.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:01:12.778 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T00:01:12.778 DEBUG:teuthology.orchestra.run.smithi139:> sudo /usr/sbin/setenforce permissive 2024-09-16T00:01:12.813 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2024-09-16T00:01:12.860 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T00:01:12.919 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-16T00:01:12.922 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-16T00:01:13.066 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T00:01:13.077 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T00:01:13.080 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi139.front.sepia.ceph.com,smithi164.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T00:04:55.248 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi139.front.sepia.ceph.com'), Remote(name='ubuntu@smithi164.front.sepia.ceph.com')] 2024-09-16T00:04:55.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T00:04:55.251 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:04:55.329 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-09-16T00:04:55.411 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-16T00:04:55.412 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T00:04:55.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:04:55.492 DEBUG:teuthology.orchestra.run.smithi164:> true 2024-09-16T00:04:55.578 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi164.front.sepia.ceph.com' 2024-09-16T00:04:55.579 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T00:04:55.775 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T00:04:55.776 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T00:04:55.777 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:04:55.780 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T00:04:55.780 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:04:55.807 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T00:04:55.809 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T00:04:55.821 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T00:04:55.823 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T00:04:55.845 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: ntpd: command not found 2024-09-16T00:04:55.848 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2024-09-16T00:04:55.859 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2024-09-16T00:04:55.861 INFO:teuthology.orchestra.run.smithi139.stdout:506 Cannot talk to daemon 2024-09-16T00:04:55.871 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T00:04:55.873 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T00:04:55.887 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T00:04:55.888 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T00:04:55.945 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:04:55.946 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:04:55.946 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-09-16T00:04:55.946 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:04:55.946 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:04:55.946 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:04:55.946 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:04:55.947 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:04:55.951 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:04:55.951 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-09-16T00:04:55.951 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:04:55.951 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:04:55.951 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:04:55.951 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:04:55.951 INFO:teuthology.run_tasks:Running task install... 2024-09-16T00:04:56.759 DEBUG:teuthology.task.install:project ceph 2024-09-16T00:04:56.759 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-16T00:04:56.759 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-16T00:04:56.759 INFO:teuthology.task.install:Using flavor: default 2024-09-16T00:04:56.765 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T00:04:56.765 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-16T00:04:56.765 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T00:04:56.766 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:04:56.768 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T00:04:56.768 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:04:56.916 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T00:04:56.916 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T00:04:56.921 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T00:04:56.921 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T00:04:57.028 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T00:04:57.028 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:04:57.028 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T00:04:57.032 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T00:04:57.032 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:04:57.032 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T00:04:57.059 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T00:04:57.060 DEBUG:teuthology.orchestra.run.smithi139:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T00:04:57.063 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T00:04:57.063 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T00:04:57.129 DEBUG:teuthology.orchestra.run.smithi139:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T00:04:57.132 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T00:04:57.209 DEBUG:teuthology.orchestra.run.smithi139:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T00:04:57.210 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T00:04:57.276 INFO:teuthology.orchestra.run.smithi139.stdout:check_obsoletes = 1 2024-09-16T00:04:57.276 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2024-09-16T00:04:57.277 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum clean all 2024-09-16T00:04:57.278 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2024-09-16T00:04:57.509 INFO:teuthology.orchestra.run.smithi139.stdout:45 files removed 2024-09-16T00:04:57.519 INFO:teuthology.orchestra.run.smithi164.stdout:45 files removed 2024-09-16T00:04:57.537 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T00:04:57.546 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T00:04:58.081 INFO:teuthology.orchestra.run.smithi139.stdout:ceph packages for x86_64 320 kB/s | 83 kB 00:00 2024-09-16T00:04:58.101 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 316 kB/s | 83 kB 00:00 2024-09-16T00:04:58.348 INFO:teuthology.orchestra.run.smithi139.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-16T00:04:58.387 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-16T00:04:58.616 INFO:teuthology.orchestra.run.smithi139.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-16T00:04:58.662 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-16T00:04:59.140 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-16T00:04:59.546 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - BaseOS 9.7 MB/s | 8.2 MB 00:00 2024-09-16T00:05:01.223 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - AppStream 36 MB/s | 20 MB 00:00 2024-09-16T00:05:01.267 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-16T00:05:05.648 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-16T00:05:06.518 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.5 MB 00:01 2024-09-16T00:05:07.182 INFO:teuthology.orchestra.run.smithi164.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 19 kB 00:00 2024-09-16T00:05:08.087 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-16T00:05:08.093 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 91 kB/s | 19 kB 00:00 2024-09-16T00:05:08.981 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-16T00:05:13.576 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T00:05:14.437 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-16T00:05:15.929 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:05:15.929 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:05:15.941 INFO:teuthology.orchestra.run.smithi164.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T00:05:15.941 INFO:teuthology.orchestra.run.smithi164.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T00:05:15.942 INFO:teuthology.orchestra.run.smithi164.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T00:05:15.943 INFO:teuthology.orchestra.run.smithi164.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T00:05:15.945 INFO:teuthology.orchestra.run.smithi164.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T00:05:15.947 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T00:05:15.948 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T00:05:16.012 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2024-09-16T00:05:16.024 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-09-16T00:05:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2024-09-16T00:05:16.025 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-09-16T00:05:16.025 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2024-09-16T00:05:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T00:05:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T00:05:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T00:05:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T00:05:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T00:05:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T00:05:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T00:05:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T00:05:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T00:05:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T00:05:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T00:05:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T00:05:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T00:05:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T00:05:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T00:05:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T00:05:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T00:05:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T00:05:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T00:05:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T00:05:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T00:05:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T00:05:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T00:05:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T00:05:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T00:05:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T00:05:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T00:05:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T00:05:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T00:05:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T00:05:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T00:05:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T00:05:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T00:05:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T00:05:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T00:05:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T00:05:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T00:05:16.028 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2024-09-16T00:05:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T00:05:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T00:05:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T00:05:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T00:05:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T00:05:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T00:05:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T00:05:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T00:05:16.028 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2024-09-16T00:05:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T00:05:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T00:05:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T00:05:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T00:05:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T00:05:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T00:05:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T00:05:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T00:05:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T00:05:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T00:05:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T00:05:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T00:05:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T00:05:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T00:05:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T00:05:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T00:05:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T00:05:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T00:05:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T00:05:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T00:05:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T00:05:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T00:05:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T00:05:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T00:05:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T00:05:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T00:05:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T00:05:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T00:05:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T00:05:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T00:05:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T00:05:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T00:05:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T00:05:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T00:05:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T00:05:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T00:05:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T00:05:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T00:05:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T00:05:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T00:05:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T00:05:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T00:05:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T00:05:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T00:05:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T00:05:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T00:05:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T00:05:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T00:05:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T00:05:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T00:05:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T00:05:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T00:05:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T00:05:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T00:05:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T00:05:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T00:05:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T00:05:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T00:05:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T00:05:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T00:05:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T00:05:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T00:05:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T00:05:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T00:05:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T00:05:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T00:05:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T00:05:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T00:05:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T00:05:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T00:05:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T00:05:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T00:05:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T00:05:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T00:05:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T00:05:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T00:05:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T00:05:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T00:05:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T00:05:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T00:05:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T00:05:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T00:05:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T00:05:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T00:05:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T00:05:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T00:05:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T00:05:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T00:05:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T00:05:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T00:05:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T00:05:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T00:05:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T00:05:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T00:05:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T00:05:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T00:05:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T00:05:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T00:05:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T00:05:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T00:05:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T00:05:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T00:05:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T00:05:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T00:05:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T00:05:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T00:05:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T00:05:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T00:05:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T00:05:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T00:05:16.036 INFO:teuthology.orchestra.run.smithi164.stdout:Installing weak dependencies: 2024-09-16T00:05:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T00:05:16.037 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:05:16.037 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2024-09-16T00:05:16.037 INFO:teuthology.orchestra.run.smithi164.stdout:====================================================================================== 2024-09-16T00:05:16.037 INFO:teuthology.orchestra.run.smithi164.stdout:Install 148 Packages 2024-09-16T00:05:16.037 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 8 Packages 2024-09-16T00:05:16.037 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:05:16.037 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 226 M 2024-09-16T00:05:16.037 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2024-09-16T00:05:16.652 INFO:teuthology.orchestra.run.smithi164.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 59 kB/s | 6.5 kB 00:00 2024-09-16T00:05:16.765 INFO:teuthology.orchestra.run.smithi139.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:05:16.766 INFO:teuthology.orchestra.run.smithi139.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:05:16.779 INFO:teuthology.orchestra.run.smithi139.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T00:05:16.781 INFO:teuthology.orchestra.run.smithi139.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T00:05:16.782 INFO:teuthology.orchestra.run.smithi139.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T00:05:16.783 INFO:teuthology.orchestra.run.smithi139.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T00:05:16.784 INFO:teuthology.orchestra.run.smithi139.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T00:05:16.787 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T00:05:16.788 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T00:05:16.836 INFO:teuthology.orchestra.run.smithi164.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 4.8 MB/s | 896 kB 00:00 2024-09-16T00:05:16.850 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-09-16T00:05:16.870 INFO:teuthology.orchestra.run.smithi139.stdout:====================================================================================== 2024-09-16T00:05:16.871 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-09-16T00:05:16.871 INFO:teuthology.orchestra.run.smithi139.stdout:====================================================================================== 2024-09-16T00:05:16.871 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-09-16T00:05:16.871 INFO:teuthology.orchestra.run.smithi139.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T00:05:16.871 INFO:teuthology.orchestra.run.smithi139.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T00:05:16.871 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T00:05:16.871 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T00:05:16.871 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T00:05:16.871 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T00:05:16.871 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T00:05:16.871 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T00:05:16.872 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T00:05:16.872 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T00:05:16.872 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T00:05:16.872 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T00:05:16.872 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T00:05:16.872 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T00:05:16.872 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T00:05:16.872 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T00:05:16.872 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T00:05:16.872 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T00:05:16.872 INFO:teuthology.orchestra.run.smithi139.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T00:05:16.872 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T00:05:16.873 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T00:05:16.873 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T00:05:16.873 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T00:05:16.873 INFO:teuthology.orchestra.run.smithi139.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T00:05:16.873 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T00:05:16.873 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T00:05:16.873 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T00:05:16.873 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T00:05:16.873 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T00:05:16.873 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T00:05:16.873 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T00:05:16.873 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T00:05:16.873 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T00:05:16.873 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T00:05:16.874 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T00:05:16.874 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T00:05:16.874 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T00:05:16.874 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrading: 2024-09-16T00:05:16.874 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T00:05:16.874 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T00:05:16.874 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T00:05:16.874 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T00:05:16.874 INFO:teuthology.orchestra.run.smithi139.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T00:05:16.874 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T00:05:16.874 INFO:teuthology.orchestra.run.smithi139.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T00:05:16.874 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T00:05:16.875 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-09-16T00:05:16.875 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T00:05:16.875 INFO:teuthology.orchestra.run.smithi139.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T00:05:16.875 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T00:05:16.875 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T00:05:16.875 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T00:05:16.875 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T00:05:16.875 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T00:05:16.875 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T00:05:16.875 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T00:05:16.875 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T00:05:16.875 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T00:05:16.875 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T00:05:16.876 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T00:05:16.876 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T00:05:16.876 INFO:teuthology.orchestra.run.smithi139.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T00:05:16.876 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T00:05:16.876 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T00:05:16.876 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T00:05:16.876 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T00:05:16.876 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T00:05:16.876 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T00:05:16.876 INFO:teuthology.orchestra.run.smithi139.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T00:05:16.876 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T00:05:16.876 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T00:05:16.876 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T00:05:16.877 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T00:05:16.877 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T00:05:16.877 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T00:05:16.877 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T00:05:16.877 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T00:05:16.877 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T00:05:16.877 INFO:teuthology.orchestra.run.smithi139.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T00:05:16.877 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T00:05:16.877 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T00:05:16.877 INFO:teuthology.orchestra.run.smithi139.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T00:05:16.877 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T00:05:16.877 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T00:05:16.877 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T00:05:16.878 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T00:05:16.878 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T00:05:16.878 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T00:05:16.878 INFO:teuthology.orchestra.run.smithi139.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T00:05:16.878 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T00:05:16.878 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T00:05:16.878 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T00:05:16.878 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T00:05:16.878 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T00:05:16.878 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T00:05:16.878 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T00:05:16.878 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T00:05:16.878 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T00:05:16.879 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T00:05:16.879 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T00:05:16.879 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T00:05:16.879 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T00:05:16.879 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T00:05:16.879 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T00:05:16.879 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T00:05:16.879 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T00:05:16.879 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T00:05:16.879 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T00:05:16.879 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T00:05:16.879 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T00:05:16.879 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T00:05:16.879 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T00:05:16.880 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T00:05:16.880 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T00:05:16.880 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T00:05:16.880 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T00:05:16.880 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T00:05:16.880 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T00:05:16.880 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T00:05:16.880 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T00:05:16.880 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T00:05:16.880 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T00:05:16.880 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T00:05:16.880 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T00:05:16.880 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T00:05:16.881 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T00:05:16.881 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T00:05:16.881 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T00:05:16.881 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T00:05:16.881 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T00:05:16.881 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T00:05:16.881 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T00:05:16.881 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T00:05:16.881 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T00:05:16.881 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T00:05:16.881 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T00:05:16.881 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T00:05:16.881 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T00:05:16.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T00:05:16.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T00:05:16.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T00:05:16.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T00:05:16.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T00:05:16.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T00:05:16.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T00:05:16.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T00:05:16.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T00:05:16.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T00:05:16.882 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T00:05:16.882 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T00:05:16.882 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T00:05:16.882 INFO:teuthology.orchestra.run.smithi139.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T00:05:16.883 INFO:teuthology.orchestra.run.smithi139.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T00:05:16.883 INFO:teuthology.orchestra.run.smithi139.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T00:05:16.883 INFO:teuthology.orchestra.run.smithi139.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T00:05:16.883 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T00:05:16.883 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T00:05:16.883 INFO:teuthology.orchestra.run.smithi139.stdout:Installing weak dependencies: 2024-09-16T00:05:16.883 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T00:05:16.883 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:05:16.883 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-09-16T00:05:16.883 INFO:teuthology.orchestra.run.smithi139.stdout:====================================================================================== 2024-09-16T00:05:16.883 INFO:teuthology.orchestra.run.smithi139.stdout:Install 148 Packages 2024-09-16T00:05:16.883 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrade 8 Packages 2024-09-16T00:05:16.883 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:05:16.884 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 226 M 2024-09-16T00:05:16.884 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-09-16T00:05:16.928 INFO:teuthology.orchestra.run.smithi164.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 1.5 MB/s | 145 kB 00:00 2024-09-16T00:05:17.011 INFO:teuthology.orchestra.run.smithi164.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-16T00:05:17.104 INFO:teuthology.orchestra.run.smithi164.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 17 MB/s | 1.5 MB 00:00 2024-09-16T00:05:17.154 INFO:teuthology.orchestra.run.smithi164.stdout:(6/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 2.4 MB 00:00 2024-09-16T00:05:17.387 INFO:teuthology.orchestra.run.smithi164.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 17 MB/s | 4.7 MB 00:00 2024-09-16T00:05:17.571 INFO:teuthology.orchestra.run.smithi139.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 34 kB/s | 6.5 kB 00:00 2024-09-16T00:05:17.704 INFO:teuthology.orchestra.run.smithi164.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 18 MB/s | 21 MB 00:01 2024-09-16T00:05:17.788 INFO:teuthology.orchestra.run.smithi164.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 303 kB/s | 25 kB 00:00 2024-09-16T00:05:17.830 INFO:teuthology.orchestra.run.smithi139.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 3.4 MB/s | 896 kB 00:00 2024-09-16T00:05:17.913 INFO:teuthology.orchestra.run.smithi139.stdout:(3/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 10 MB/s | 5.4 MB 00:00 2024-09-16T00:05:17.939 INFO:teuthology.orchestra.run.smithi139.stdout:(4/156): ceph-immutable-object-cache-19.1.1-326 1.3 MB/s | 145 kB 00:00 2024-09-16T00:05:18.139 INFO:teuthology.orchestra.run.smithi139.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 2.4 MB 00:00 2024-09-16T00:05:18.181 INFO:teuthology.orchestra.run.smithi139.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 6.3 MB/s | 1.5 MB 00:00 2024-09-16T00:05:18.355 INFO:teuthology.orchestra.run.smithi164.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 14 MB/s | 17 MB 00:01 2024-09-16T00:05:18.447 INFO:teuthology.orchestra.run.smithi164.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 2.4 MB/s | 228 kB 00:00 2024-09-16T00:05:18.522 INFO:teuthology.orchestra.run.smithi164.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 361 kB/s | 27 kB 00:00 2024-09-16T00:05:18.564 INFO:teuthology.orchestra.run.smithi139.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 4.7 MB 00:00 2024-09-16T00:05:18.639 INFO:teuthology.orchestra.run.smithi164.stdout:(13/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 8.5 MB/s | 11 MB 00:01 2024-09-16T00:05:18.664 INFO:teuthology.orchestra.run.smithi164.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 223 kB/s | 32 kB 00:00 2024-09-16T00:05:18.881 INFO:teuthology.orchestra.run.smithi164.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 753 kB/s | 163 kB 00:00 2024-09-16T00:05:18.915 INFO:teuthology.orchestra.run.smithi164.stdout:(16/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 2.7 MB/s | 751 kB 00:00 2024-09-16T00:05:18.956 INFO:teuthology.orchestra.run.smithi164.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 1.6 MB/s | 127 kB 00:00 2024-09-16T00:05:19.073 INFO:teuthology.orchestra.run.smithi164.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 3.1 MB/s | 506 kB 00:00 2024-09-16T00:05:19.174 INFO:teuthology.orchestra.run.smithi164.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 451 kB/s | 45 kB 00:00 2024-09-16T00:05:19.257 INFO:teuthology.orchestra.run.smithi164.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.6 MB/s | 136 kB 00:00 2024-09-16T00:05:19.341 INFO:teuthology.orchestra.run.smithi164.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.9 MB/s | 161 kB 00:00 2024-09-16T00:05:19.390 INFO:teuthology.orchestra.run.smithi139.stdout:(8/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86_ 14 MB/s | 17 MB 00:01 2024-09-16T00:05:19.425 INFO:teuthology.orchestra.run.smithi164.stdout:(22/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 5.3 MB 00:00 2024-09-16T00:05:19.458 INFO:teuthology.orchestra.run.smithi164.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 2.7 MB/s | 320 kB 00:00 2024-09-16T00:05:19.492 INFO:teuthology.orchestra.run.smithi164.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 4.4 MB/s | 300 kB 00:00 2024-09-16T00:05:19.509 INFO:teuthology.orchestra.run.smithi164.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.9 MB/s | 99 kB 00:00 2024-09-16T00:05:19.515 INFO:teuthology.orchestra.run.smithi139.stdout:(9/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9. 11 MB/s | 11 MB 00:00 2024-09-16T00:05:19.540 INFO:teuthology.orchestra.run.smithi139.stdout:(10/156): ceph-selinux-19.1.1-326.g8921c8ef.el9 169 kB/s | 25 kB 00:00 2024-09-16T00:05:19.542 INFO:teuthology.orchestra.run.smithi164.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-16T00:05:19.590 INFO:teuthology.orchestra.run.smithi139.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 4.5 MB/s | 228 kB 00:00 2024-09-16T00:05:19.626 INFO:teuthology.orchestra.run.smithi164.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.0 MB/s | 170 kB 00:00 2024-09-16T00:05:19.657 INFO:teuthology.orchestra.run.smithi139.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 404 kB/s | 27 kB 00:00 2024-09-16T00:05:19.702 INFO:teuthology.orchestra.run.smithi164.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 425 kB/s | 32 kB 00:00 2024-09-16T00:05:19.724 INFO:teuthology.orchestra.run.smithi139.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 475 kB/s | 32 kB 00:00 2024-09-16T00:05:19.777 INFO:teuthology.orchestra.run.smithi164.stdout:(29/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.9 MB/s | 146 kB 00:00 2024-09-16T00:05:19.883 INFO:teuthology.orchestra.run.smithi139.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 4.6 MB/s | 751 kB 00:00 2024-09-16T00:05:19.969 INFO:teuthology.orchestra.run.smithi164.stdout:(30/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 6.8 MB/s | 3.1 MB 00:00 2024-09-16T00:05:19.975 INFO:teuthology.orchestra.run.smithi139.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 1.7 MB/s | 163 kB 00:00 2024-09-16T00:05:20.075 INFO:teuthology.orchestra.run.smithi139.stdout:(16/156): librados-devel-19.1.1-326.g8921c8ef.e 1.2 MB/s | 127 kB 00:00 2024-09-16T00:05:20.152 INFO:teuthology.orchestra.run.smithi164.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 9.5 MB/s | 3.6 MB 00:00 2024-09-16T00:05:20.175 INFO:teuthology.orchestra.run.smithi139.stdout:(17/156): libradosstriper1-19.1.1-326.g8921c8ef 4.9 MB/s | 506 kB 00:00 2024-09-16T00:05:20.245 INFO:teuthology.orchestra.run.smithi164.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 2.7 MB/s | 249 kB 00:00 2024-09-16T00:05:20.337 INFO:teuthology.orchestra.run.smithi164.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 538 kB/s | 49 kB 00:00 2024-09-16T00:05:20.420 INFO:teuthology.orchestra.run.smithi164.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 197 kB/s | 16 kB 00:00 2024-09-16T00:05:20.512 INFO:teuthology.orchestra.run.smithi164.stdout:(35/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 3.0 MB/s | 285 kB 00:00 2024-09-16T00:05:20.576 INFO:teuthology.orchestra.run.smithi139.stdout:(18/156): ceph-common-19.1.1-326.g8921c8ef.el9. 6.6 MB/s | 21 MB 00:03 2024-09-16T00:05:20.612 INFO:teuthology.orchestra.run.smithi164.stdout:(36/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 7.5 MB/s | 765 kB 00:00 2024-09-16T00:05:20.662 INFO:teuthology.orchestra.run.smithi164.stdout:(37/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 365 kB/s | 18 kB 00:00 2024-09-16T00:05:20.676 INFO:teuthology.orchestra.run.smithi139.stdout:(19/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 5.3 MB 00:00 2024-09-16T00:05:20.701 INFO:teuthology.orchestra.run.smithi139.stdout:(20/156): python3-ceph-argparse-19.1.1-326.g892 361 kB/s | 45 kB 00:00 2024-09-16T00:05:20.760 INFO:teuthology.orchestra.run.smithi139.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 2.7 MB/s | 161 kB 00:00 2024-09-16T00:05:20.818 INFO:teuthology.orchestra.run.smithi139.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 5.4 MB/s | 320 kB 00:00 2024-09-16T00:05:20.844 INFO:teuthology.orchestra.run.smithi139.stdout:(23/156): python3-ceph-common-19.1.1-326.g8921c 813 kB/s | 136 kB 00:00 2024-09-16T00:05:20.877 INFO:teuthology.orchestra.run.smithi139.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.0 MB/s | 300 kB 00:00 2024-09-16T00:05:20.902 INFO:teuthology.orchestra.run.smithi139.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.7 MB/s | 99 kB 00:00 2024-09-16T00:05:20.928 INFO:teuthology.orchestra.run.smithi139.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-16T00:05:21.012 INFO:teuthology.orchestra.run.smithi139.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.0 MB/s | 170 kB 00:00 2024-09-16T00:05:21.079 INFO:teuthology.orchestra.run.smithi139.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 477 kB/s | 32 kB 00:00 2024-09-16T00:05:21.096 INFO:teuthology.orchestra.run.smithi164.stdout:(38/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 15 MB/s | 49 MB 00:03 2024-09-16T00:05:21.137 INFO:teuthology.orchestra.run.smithi139.stdout:(29/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 13 MB/s | 3.1 MB 00:00 2024-09-16T00:05:21.162 INFO:teuthology.orchestra.run.smithi139.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.7 MB/s | 146 kB 00:00 2024-09-16T00:05:21.205 INFO:teuthology.orchestra.run.smithi164.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 6.0 MB/s | 7.4 MB 00:01 2024-09-16T00:05:21.380 INFO:teuthology.orchestra.run.smithi164.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 111 kB/s | 80 kB 00:00 2024-09-16T00:05:21.413 INFO:teuthology.orchestra.run.smithi139.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 13 MB/s | 3.6 MB 00:00 2024-09-16T00:05:21.488 INFO:teuthology.orchestra.run.smithi139.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 3.3 MB/s | 249 kB 00:00 2024-09-16T00:05:21.613 INFO:teuthology.orchestra.run.smithi139.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 16 MB/s | 7.4 MB 00:00 2024-09-16T00:05:21.638 INFO:teuthology.orchestra.run.smithi139.stdout:(34/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 328 kB/s | 49 kB 00:00 2024-09-16T00:05:21.664 INFO:teuthology.orchestra.run.smithi164.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 894 kB/s | 253 kB 00:00 2024-09-16T00:05:21.664 INFO:teuthology.orchestra.run.smithi139.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 328 kB/s | 16 kB 00:00 2024-09-16T00:05:21.698 INFO:teuthology.orchestra.run.smithi139.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 4.7 MB/s | 285 kB 00:00 2024-09-16T00:05:21.731 INFO:teuthology.orchestra.run.smithi139.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 11 MB/s | 765 kB 00:00 2024-09-16T00:05:21.756 INFO:teuthology.orchestra.run.smithi164.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 347 kB/s | 191 kB 00:00 2024-09-16T00:05:21.756 INFO:teuthology.orchestra.run.smithi139.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 312 kB/s | 18 kB 00:00 2024-09-16T00:05:21.872 INFO:teuthology.orchestra.run.smithi164.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 915 kB/s | 106 kB 00:00 2024-09-16T00:05:21.939 INFO:teuthology.orchestra.run.smithi164.stdout:(44/156): libgfortran-11.5.0-2.el9.x86_64.rpm 951 kB/s | 801 kB 00:00 2024-09-16T00:05:21.981 INFO:teuthology.orchestra.run.smithi164.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-16T00:05:22.048 INFO:teuthology.orchestra.run.smithi164.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-16T00:05:22.182 INFO:teuthology.orchestra.run.smithi139.stdout:(39/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 18 MB/s | 49 MB 00:02 2024-09-16T00:05:22.223 INFO:teuthology.orchestra.run.smithi164.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 2.2 MB/s | 1.2 MB 00:00 2024-09-16T00:05:22.249 INFO:teuthology.orchestra.run.smithi164.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 803 kB/s | 215 kB 00:00 2024-09-16T00:05:22.324 INFO:teuthology.orchestra.run.smithi139.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 135 kB/s | 80 kB 00:00 2024-09-16T00:05:22.341 INFO:teuthology.orchestra.run.smithi164.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-09-16T00:05:22.383 INFO:teuthology.orchestra.run.smithi164.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 523 kB/s | 22 kB 00:00 2024-09-16T00:05:22.407 INFO:teuthology.orchestra.run.smithi139.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 850 kB/s | 191 kB 00:00 2024-09-16T00:05:22.408 INFO:teuthology.orchestra.run.smithi164.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 673 kB/s | 107 kB 00:00 2024-09-16T00:05:22.441 INFO:teuthology.orchestra.run.smithi139.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-16T00:05:22.450 INFO:teuthology.orchestra.run.smithi164.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 786 kB/s | 33 kB 00:00 2024-09-16T00:05:22.466 INFO:teuthology.orchestra.run.smithi139.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-09-16T00:05:22.491 INFO:teuthology.orchestra.run.smithi139.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-16T00:05:22.492 INFO:teuthology.orchestra.run.smithi164.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-16T00:05:22.525 INFO:teuthology.orchestra.run.smithi139.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-16T00:05:22.534 INFO:teuthology.orchestra.run.smithi164.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-09-16T00:05:22.567 INFO:teuthology.orchestra.run.smithi139.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 7.7 MB/s | 1.2 MB 00:00 2024-09-16T00:05:22.584 INFO:teuthology.orchestra.run.smithi164.stdout:(55/156): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-09-16T00:05:22.592 INFO:teuthology.orchestra.run.smithi139.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-09-16T00:05:22.601 INFO:teuthology.orchestra.run.smithi164.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-09-16T00:05:22.618 INFO:teuthology.orchestra.run.smithi139.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-16T00:05:22.643 INFO:teuthology.orchestra.run.smithi164.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-16T00:05:22.668 INFO:teuthology.orchestra.run.smithi164.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-16T00:05:22.751 INFO:teuthology.orchestra.run.smithi139.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-09-16T00:05:22.793 INFO:teuthology.orchestra.run.smithi139.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 610 kB/s | 107 kB 00:00 2024-09-16T00:05:22.827 INFO:teuthology.orchestra.run.smithi139.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 3.9 MB/s | 943 kB 00:00 2024-09-16T00:05:22.835 INFO:teuthology.orchestra.run.smithi164.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-09-16T00:05:22.860 INFO:teuthology.orchestra.run.smithi139.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 199 kB/s | 22 kB 00:00 2024-09-16T00:05:22.861 INFO:teuthology.orchestra.run.smithi164.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 3.0 MB/s | 581 kB 00:00 2024-09-16T00:05:22.878 INFO:teuthology.orchestra.run.smithi164.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 584 kB/s | 24 kB 00:00 2024-09-16T00:05:22.888 INFO:teuthology.orchestra.run.smithi139.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-09-16T00:05:22.903 INFO:teuthology.orchestra.run.smithi164.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-16T00:05:22.911 INFO:teuthology.orchestra.run.smithi139.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-09-16T00:05:22.961 INFO:teuthology.orchestra.run.smithi139.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 242 kB/s | 18 kB 00:00 2024-09-16T00:05:22.986 INFO:teuthology.orchestra.run.smithi139.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-09-16T00:05:22.987 INFO:teuthology.orchestra.run.smithi164.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-16T00:05:23.012 INFO:teuthology.orchestra.run.smithi164.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-16T00:05:23.029 INFO:teuthology.orchestra.run.smithi164.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 918 kB/s | 38 kB 00:00 2024-09-16T00:05:23.045 INFO:teuthology.orchestra.run.smithi139.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-09-16T00:05:23.073 INFO:teuthology.orchestra.run.smithi139.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-09-16T00:05:23.104 INFO:teuthology.orchestra.run.smithi139.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-09-16T00:05:23.154 INFO:teuthology.orchestra.run.smithi139.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-09-16T00:05:23.179 INFO:teuthology.orchestra.run.smithi164.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.1 MB/s | 3.0 MB 00:00 2024-09-16T00:05:23.179 INFO:teuthology.orchestra.run.smithi139.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 325 kB/s | 24 kB 00:00 2024-09-16T00:05:23.205 INFO:teuthology.orchestra.run.smithi139.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 4.2 MB/s | 581 kB 00:00 2024-09-16T00:05:23.247 INFO:teuthology.orchestra.run.smithi139.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 515 kB/s | 47 kB 00:00 2024-09-16T00:05:23.272 INFO:teuthology.orchestra.run.smithi139.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-16T00:05:23.297 INFO:teuthology.orchestra.run.smithi139.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-09-16T00:05:23.331 INFO:teuthology.orchestra.run.smithi139.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 458 kB/s | 38 kB 00:00 2024-09-16T00:05:23.448 INFO:teuthology.orchestra.run.smithi139.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 28 MB/s | 4.9 MB 00:00 2024-09-16T00:05:23.521 INFO:teuthology.orchestra.run.smithi164.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-09-16T00:05:23.532 INFO:teuthology.orchestra.run.smithi139.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-09-16T00:05:23.573 INFO:teuthology.orchestra.run.smithi139.stdout:(69/156): postgresql-13.16-1.el9.x86_64.rpm 6.6 MB/s | 1.6 MB 00:00 2024-09-16T00:05:23.615 INFO:teuthology.orchestra.run.smithi139.stdout:(70/156): postgresql-contrib-13.16-1.el9.x86_64 5.3 MB/s | 910 kB 00:00 2024-09-16T00:05:23.641 INFO:teuthology.orchestra.run.smithi139.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-16T00:05:23.691 INFO:teuthology.orchestra.run.smithi139.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-16T00:05:23.713 INFO:teuthology.orchestra.run.smithi164.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 4.6 MB/s | 910 kB 00:00 2024-09-16T00:05:23.724 INFO:teuthology.orchestra.run.smithi139.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-16T00:05:23.771 INFO:teuthology.orchestra.run.smithi164.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-16T00:05:23.825 INFO:teuthology.orchestra.run.smithi139.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-16T00:05:23.850 INFO:teuthology.orchestra.run.smithi139.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 489 kB/s | 61 kB 00:00 2024-09-16T00:05:23.942 INFO:teuthology.orchestra.run.smithi139.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 6.0 MB 00:00 2024-09-16T00:05:23.967 INFO:teuthology.orchestra.run.smithi139.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-16T00:05:23.971 INFO:teuthology.orchestra.run.smithi164.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.7 MB/s | 4.4 MB 00:00 2024-09-16T00:05:23.992 INFO:teuthology.orchestra.run.smithi139.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-09-16T00:05:24.009 INFO:teuthology.orchestra.run.smithi139.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 384 kB/s | 26 kB 00:00 2024-09-16T00:05:24.085 INFO:teuthology.orchestra.run.smithi139.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-09-16T00:05:24.101 INFO:teuthology.orchestra.run.smithi139.stdout:(81/156): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-09-16T00:05:24.118 INFO:teuthology.orchestra.run.smithi139.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 319 kB/s | 35 kB 00:00 2024-09-16T00:05:24.164 INFO:teuthology.orchestra.run.smithi164.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-09-16T00:05:24.210 INFO:teuthology.orchestra.run.smithi139.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 5.1 MB 00:00 2024-09-16T00:05:24.214 INFO:teuthology.orchestra.run.smithi164.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-16T00:05:24.235 INFO:teuthology.orchestra.run.smithi139.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-09-16T00:05:24.261 INFO:teuthology.orchestra.run.smithi139.stdout:(85/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-09-16T00:05:24.294 INFO:teuthology.orchestra.run.smithi139.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 923 kB/s | 77 kB 00:00 2024-09-16T00:05:24.320 INFO:teuthology.orchestra.run.smithi139.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 504 kB/s | 42 kB 00:00 2024-09-16T00:05:24.353 INFO:teuthology.orchestra.run.smithi139.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-09-16T00:05:24.379 INFO:teuthology.orchestra.run.smithi139.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-09-16T00:05:24.389 INFO:teuthology.orchestra.run.smithi164.stdout:(73/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.6 MB/s | 4.9 MB 00:01 2024-09-16T00:05:24.404 INFO:teuthology.orchestra.run.smithi139.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.2 MB/s | 279 kB 00:00 2024-09-16T00:05:24.429 INFO:teuthology.orchestra.run.smithi139.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-16T00:05:24.439 INFO:teuthology.orchestra.run.smithi164.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-16T00:05:24.471 INFO:teuthology.orchestra.run.smithi139.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 582 kB/s | 54 kB 00:00 2024-09-16T00:05:24.556 INFO:teuthology.orchestra.run.smithi164.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-16T00:05:24.563 INFO:teuthology.orchestra.run.smithi139.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 311 kB/s | 42 kB 00:00 2024-09-16T00:05:24.588 INFO:teuthology.orchestra.run.smithi139.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 391 kB/s | 44 kB 00:00 2024-09-16T00:05:24.647 INFO:teuthology.orchestra.run.smithi139.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-16T00:05:24.665 INFO:teuthology.orchestra.run.smithi164.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-16T00:05:24.689 INFO:teuthology.orchestra.run.smithi139.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 358 kB/s | 36 kB 00:00 2024-09-16T00:05:24.715 INFO:teuthology.orchestra.run.smithi164.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-16T00:05:24.731 INFO:teuthology.orchestra.run.smithi139.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-09-16T00:05:24.766 INFO:teuthology.orchestra.run.smithi164.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-16T00:05:24.883 INFO:teuthology.orchestra.run.smithi164.stdout:(79/156): postgresql-server-13.16-1.el9.x86_64. 5.2 MB/s | 5.8 MB 00:01 2024-09-16T00:05:24.890 INFO:teuthology.orchestra.run.smithi139.stdout:(98/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-16T00:05:24.908 INFO:teuthology.orchestra.run.smithi164.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-16T00:05:24.915 INFO:teuthology.orchestra.run.smithi139.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 315 kB/s | 58 kB 00:00 2024-09-16T00:05:24.933 INFO:teuthology.orchestra.run.smithi164.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-16T00:05:24.940 INFO:teuthology.orchestra.run.smithi139.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-09-16T00:05:24.990 INFO:teuthology.orchestra.run.smithi139.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 956 kB/s | 96 kB 00:00 2024-09-16T00:05:25.015 INFO:teuthology.orchestra.run.smithi139.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 596 kB/s | 45 kB 00:00 2024-09-16T00:05:25.025 INFO:teuthology.orchestra.run.smithi164.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-09-16T00:05:25.041 INFO:teuthology.orchestra.run.smithi139.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 507 kB/s | 64 kB 00:00 2024-09-16T00:05:25.066 INFO:teuthology.orchestra.run.smithi139.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 295 kB/s | 22 kB 00:00 2024-09-16T00:05:25.109 INFO:teuthology.orchestra.run.smithi164.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-16T00:05:25.116 INFO:teuthology.orchestra.run.smithi139.stdout:(105/156): protobuf-compiler-3.14.0-13.el9.x86_ 8.4 MB/s | 863 kB 00:00 2024-09-16T00:05:25.142 INFO:teuthology.orchestra.run.smithi139.stdout:(106/156): gperftools-libs-2.9.1-3.el9.x86_64.r 4.0 MB/s | 308 kB 00:00 2024-09-16T00:05:25.159 INFO:teuthology.orchestra.run.smithi164.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-16T00:05:25.200 INFO:teuthology.orchestra.run.smithi139.stdout:(107/156): abseil-cpp-20211102.0-3.el9.x86_64.r 3.4 MB/s | 548 kB 00:00 2024-09-16T00:05:25.201 INFO:teuthology.orchestra.run.smithi164.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-16T00:05:25.225 INFO:teuthology.orchestra.run.smithi139.stdout:(108/156): grpc-data-1.46.7-10.el9.noarch.rpm 182 kB/s | 19 kB 00:00 2024-09-16T00:05:25.251 INFO:teuthology.orchestra.run.smithi139.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 517 kB/s | 26 kB 00:00 2024-09-16T00:05:25.276 INFO:teuthology.orchestra.run.smithi139.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 968 kB/s | 49 kB 00:00 2024-09-16T00:05:25.301 INFO:teuthology.orchestra.run.smithi139.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-16T00:05:25.302 INFO:teuthology.orchestra.run.smithi164.stdout:(86/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.5 MB/s | 6.0 MB 00:01 2024-09-16T00:05:25.326 INFO:teuthology.orchestra.run.smithi139.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-16T00:05:25.327 INFO:teuthology.orchestra.run.smithi164.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-09-16T00:05:25.352 INFO:teuthology.orchestra.run.smithi164.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-16T00:05:25.368 INFO:teuthology.orchestra.run.smithi139.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-16T00:05:25.394 INFO:teuthology.orchestra.run.smithi164.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-16T00:05:25.402 INFO:teuthology.orchestra.run.smithi139.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-09-16T00:05:25.419 INFO:teuthology.orchestra.run.smithi164.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-09-16T00:05:25.428 INFO:teuthology.orchestra.run.smithi139.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-09-16T00:05:25.436 INFO:teuthology.orchestra.run.smithi164.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-16T00:05:25.453 INFO:teuthology.orchestra.run.smithi139.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-09-16T00:05:25.478 INFO:teuthology.orchestra.run.smithi139.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 283 kB/s | 14 kB 00:00 2024-09-16T00:05:25.478 INFO:teuthology.orchestra.run.smithi164.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-16T00:05:25.503 INFO:teuthology.orchestra.run.smithi139.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-16T00:05:25.528 INFO:teuthology.orchestra.run.smithi164.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 886 kB/s | 44 kB 00:00 2024-09-16T00:05:25.537 INFO:teuthology.orchestra.run.smithi139.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-16T00:05:25.562 INFO:teuthology.orchestra.run.smithi139.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-16T00:05:25.579 INFO:teuthology.orchestra.run.smithi164.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-16T00:05:25.587 INFO:teuthology.orchestra.run.smithi139.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-16T00:05:25.621 INFO:teuthology.orchestra.run.smithi139.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-16T00:05:25.629 INFO:teuthology.orchestra.run.smithi164.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 721 kB/s | 36 kB 00:00 2024-09-16T00:05:25.646 INFO:teuthology.orchestra.run.smithi139.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-16T00:05:25.663 INFO:teuthology.orchestra.run.smithi139.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-16T00:05:25.671 INFO:teuthology.orchestra.run.smithi164.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-16T00:05:25.722 INFO:teuthology.orchestra.run.smithi139.stdout:(125/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-09-16T00:05:25.738 INFO:teuthology.orchestra.run.smithi139.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2024-09-16T00:05:25.754 INFO:teuthology.orchestra.run.smithi164.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.6 MB/s | 304 kB 00:00 2024-09-16T00:05:25.764 INFO:teuthology.orchestra.run.smithi139.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 465 kB/s | 19 kB 00:00 2024-09-16T00:05:25.789 INFO:teuthology.orchestra.run.smithi139.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 389 kB/s | 20 kB 00:00 2024-09-16T00:05:25.805 INFO:teuthology.orchestra.run.smithi164.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T00:05:25.815 INFO:teuthology.orchestra.run.smithi139.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T00:05:25.855 INFO:teuthology.orchestra.run.smithi164.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.9 MB/s | 96 kB 00:00 2024-09-16T00:05:25.865 INFO:teuthology.orchestra.run.smithi139.stdout:(130/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.7 MB/s | 2.0 MB 00:00 2024-09-16T00:05:25.890 INFO:teuthology.orchestra.run.smithi139.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-09-16T00:05:25.897 INFO:teuthology.orchestra.run.smithi164.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-16T00:05:25.915 INFO:teuthology.orchestra.run.smithi139.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-16T00:05:25.939 INFO:teuthology.orchestra.run.smithi164.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-09-16T00:05:25.940 INFO:teuthology.orchestra.run.smithi139.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-16T00:05:25.966 INFO:teuthology.orchestra.run.smithi139.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-16T00:05:25.991 INFO:teuthology.orchestra.run.smithi139.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-16T00:05:26.016 INFO:teuthology.orchestra.run.smithi139.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 564 kB/s | 28 kB 00:00 2024-09-16T00:05:26.039 INFO:teuthology.orchestra.run.smithi164.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-09-16T00:05:26.042 INFO:teuthology.orchestra.run.smithi139.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-16T00:05:26.067 INFO:teuthology.orchestra.run.smithi139.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-16T00:05:26.092 INFO:teuthology.orchestra.run.smithi139.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-16T00:05:26.126 INFO:teuthology.orchestra.run.smithi139.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-09-16T00:05:26.160 INFO:teuthology.orchestra.run.smithi139.stdout:(141/156): python3-kubernetes-26.1.0-2.el9.noar 2.6 MB/s | 1.0 MB 00:00 2024-09-16T00:05:26.185 INFO:teuthology.orchestra.run.smithi139.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 807 kB/s | 74 kB 00:00 2024-09-16T00:05:26.206 INFO:teuthology.orchestra.run.smithi164.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-09-16T00:05:26.210 INFO:teuthology.orchestra.run.smithi139.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-16T00:05:26.235 INFO:teuthology.orchestra.run.smithi139.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-16T00:05:26.256 INFO:teuthology.orchestra.run.smithi164.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-16T00:05:26.261 INFO:teuthology.orchestra.run.smithi139.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-16T00:05:26.281 INFO:teuthology.orchestra.run.smithi164.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-16T00:05:26.294 INFO:teuthology.orchestra.run.smithi139.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-16T00:05:26.299 INFO:teuthology.orchestra.run.smithi164.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-16T00:05:26.328 INFO:teuthology.orchestra.run.smithi139.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-16T00:05:26.386 INFO:teuthology.orchestra.run.smithi139.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-16T00:05:26.424 INFO:teuthology.orchestra.run.smithi164.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-09-16T00:05:26.450 INFO:teuthology.orchestra.run.smithi164.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-16T00:05:26.516 INFO:teuthology.orchestra.run.smithi164.stdout:(109/156): python3-numpy-1.20.1-5.el9.x86_64.rp 3.2 MB/s | 5.1 MB 00:01 2024-09-16T00:05:26.564 INFO:teuthology.orchestra.run.smithi164.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-09-16T00:05:26.575 INFO:teuthology.orchestra.run.smithi164.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.7 MB/s | 151 kB 00:00 2024-09-16T00:05:26.586 INFO:teuthology.orchestra.run.smithi139.stdout:(149/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 12 MB/s | 3.2 MB 00:00 2024-09-16T00:05:26.609 INFO:teuthology.orchestra.run.smithi164.stdout:(112/156): libunwind-1.6.2-1.el9.x86_64.rpm 730 kB/s | 67 kB 00:00 2024-09-16T00:05:26.634 INFO:teuthology.orchestra.run.smithi164.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-16T00:05:26.637 INFO:teuthology.orchestra.run.smithi139.stdout:(150/156): librados2-19.1.1-326.g8921c8ef.el9.x 10 MB/s | 3.4 MB 00:00 2024-09-16T00:05:26.659 INFO:teuthology.orchestra.run.smithi164.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T00:05:26.685 INFO:teuthology.orchestra.run.smithi164.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-16T00:05:26.710 INFO:teuthology.orchestra.run.smithi164.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-16T00:05:26.735 INFO:teuthology.orchestra.run.smithi164.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-16T00:05:26.737 INFO:teuthology.orchestra.run.smithi139.stdout:(151/156): NetworkManager-1.49.90-1.el9.x86_64. 6.6 MB/s | 2.3 MB 00:00 2024-09-16T00:05:26.762 INFO:teuthology.orchestra.run.smithi139.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 325 kB/s | 41 kB 00:00 2024-09-16T00:05:26.769 INFO:teuthology.orchestra.run.smithi164.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-09-16T00:05:26.803 INFO:teuthology.orchestra.run.smithi164.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-16T00:05:26.821 INFO:teuthology.orchestra.run.smithi139.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 2.9 MB/s | 249 kB 00:00 2024-09-16T00:05:26.828 INFO:teuthology.orchestra.run.smithi164.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-16T00:05:26.853 INFO:teuthology.orchestra.run.smithi164.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-16T00:05:26.878 INFO:teuthology.orchestra.run.smithi164.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-16T00:05:26.895 INFO:teuthology.orchestra.run.smithi164.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 642 kB/s | 11 kB 00:00 2024-09-16T00:05:26.912 INFO:teuthology.orchestra.run.smithi164.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-16T00:05:26.913 INFO:teuthology.orchestra.run.smithi139.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 5.5 MB/s | 1.8 MB 00:00 2024-09-16T00:05:26.929 INFO:teuthology.orchestra.run.smithi164.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-16T00:05:26.946 INFO:teuthology.orchestra.run.smithi164.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-16T00:05:26.963 INFO:teuthology.orchestra.run.smithi139.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-09-16T00:05:26.963 INFO:teuthology.orchestra.run.smithi164.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-16T00:05:27.005 INFO:teuthology.orchestra.run.smithi139.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 5.6 MB/s | 1.4 MB 00:00 2024-09-16T00:05:27.011 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:05:27.011 INFO:teuthology.orchestra.run.smithi139.stdout:Total 22 MB/s | 226 MB 00:10 2024-09-16T00:05:27.022 INFO:teuthology.orchestra.run.smithi164.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-09-16T00:05:27.039 INFO:teuthology.orchestra.run.smithi164.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 2.8 MB/s | 46 kB 00:00 2024-09-16T00:05:27.064 INFO:teuthology.orchestra.run.smithi164.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-16T00:05:27.081 INFO:teuthology.orchestra.run.smithi164.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-16T00:05:27.115 INFO:teuthology.orchestra.run.smithi164.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.7 MB/s | 1.0 MB 00:00 2024-09-16T00:05:27.132 INFO:teuthology.orchestra.run.smithi164.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 988 kB/s | 16 kB 00:00 2024-09-16T00:05:27.157 INFO:teuthology.orchestra.run.smithi164.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-16T00:05:27.183 INFO:teuthology.orchestra.run.smithi164.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-16T00:05:27.199 INFO:teuthology.orchestra.run.smithi164.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 669 kB/s | 28 kB 00:00 2024-09-16T00:05:27.216 INFO:teuthology.orchestra.run.smithi164.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 922 kB/s | 31 kB 00:00 2024-09-16T00:05:27.233 INFO:teuthology.orchestra.run.smithi164.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-16T00:05:27.251 INFO:teuthology.orchestra.run.smithi164.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 2.1 MB/s | 36 kB 00:00 2024-09-16T00:05:27.276 INFO:teuthology.orchestra.run.smithi164.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-16T00:05:27.293 INFO:teuthology.orchestra.run.smithi164.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-16T00:05:27.326 INFO:teuthology.orchestra.run.smithi164.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2024-09-16T00:05:27.352 INFO:teuthology.orchestra.run.smithi164.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-16T00:05:27.377 INFO:teuthology.orchestra.run.smithi164.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 800 kB/s | 20 kB 00:00 2024-09-16T00:05:27.402 INFO:teuthology.orchestra.run.smithi164.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-16T00:05:27.436 INFO:teuthology.orchestra.run.smithi164.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-09-16T00:05:27.595 INFO:teuthology.orchestra.run.smithi164.stdout:(147/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.5 MB/s | 16 MB 00:02 2024-09-16T00:05:27.645 INFO:teuthology.orchestra.run.smithi164.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 6.5 MB/s | 1.6 MB 00:00 2024-09-16T00:05:27.787 INFO:teuthology.orchestra.run.smithi164.stdout:(149/156): librados2-19.1.1-326.g8921c8ef.el9.x 9.8 MB/s | 3.4 MB 00:00 2024-09-16T00:05:27.920 INFO:teuthology.orchestra.run.smithi164.stdout:(150/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 9.7 MB/s | 3.2 MB 00:00 2024-09-16T00:05:28.212 INFO:teuthology.orchestra.run.smithi164.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 140 kB/s | 41 kB 00:00 2024-09-16T00:05:28.245 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-09-16T00:05:28.396 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-09-16T00:05:28.396 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-09-16T00:05:28.588 INFO:teuthology.orchestra.run.smithi164.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 663 kB/s | 249 kB 00:00 2024-09-16T00:05:28.763 INFO:teuthology.orchestra.run.smithi164.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 2.1 MB/s | 2.3 MB 00:01 2024-09-16T00:05:29.063 INFO:teuthology.orchestra.run.smithi164.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 2.8 MB/s | 1.4 MB 00:00 2024-09-16T00:05:29.105 INFO:teuthology.orchestra.run.smithi164.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.0 MB/s | 2.4 MB 00:00 2024-09-16T00:05:29.188 INFO:teuthology.orchestra.run.smithi164.stdout:(156/156): NetworkManager-libnm-1.49.90-1.el9.x 1.3 MB/s | 1.8 MB 00:01 2024-09-16T00:05:29.194 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:05:29.194 INFO:teuthology.orchestra.run.smithi164.stdout:Total 17 MB/s | 226 MB 00:13 2024-09-16T00:05:30.250 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-09-16T00:05:30.252 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-09-16T00:05:30.439 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2024-09-16T00:05:30.591 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2024-09-16T00:05:30.591 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2024-09-16T00:05:32.429 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2024-09-16T00:05:32.430 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2024-09-16T00:05:36.547 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-09-16T00:05:36.641 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T00:05:36.751 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T00:05:36.984 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T00:05:37.018 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T00:05:37.051 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T00:05:37.116 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T00:05:37.497 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T00:05:37.552 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:05:37.673 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:05:37.702 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:05:37.794 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:05:37.824 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:05:37.988 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T00:05:38.046 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T00:05:38.073 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:05:38.177 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:05:38.200 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:05:38.263 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:05:38.292 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:05:38.317 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:05:38.477 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:05:38.639 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:05:38.749 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T00:05:38.856 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:05:38.921 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2024-09-16T00:05:38.967 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:05:39.045 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T00:05:39.084 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T00:05:39.138 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T00:05:39.176 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T00:05:39.202 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T00:05:39.206 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:05:39.267 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T00:05:39.315 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T00:05:39.410 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T00:05:39.447 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T00:05:39.490 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T00:05:39.554 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T00:05:39.598 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T00:05:39.599 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T00:05:39.631 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T00:05:39.674 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T00:05:39.707 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T00:05:39.724 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:05:39.735 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T00:05:39.803 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T00:05:39.853 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:05:39.857 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T00:05:39.881 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:05:39.897 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T00:05:39.940 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T00:05:39.971 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:05:40.006 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:05:40.016 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T00:05:40.050 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T00:05:40.094 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T00:05:40.122 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T00:05:40.187 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T00:05:40.191 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T00:05:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T00:05:40.237 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T00:05:40.260 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:05:40.311 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T00:05:40.363 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:05:40.387 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:05:40.441 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:05:40.453 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T00:05:40.454 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:05:40.475 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:05:40.523 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T00:05:40.665 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:05:40.822 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:05:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T00:05:41.044 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T00:05:41.073 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:05:41.129 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:05:41.135 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T00:05:41.173 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T00:05:41.215 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T00:05:41.253 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T00:05:41.295 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T00:05:41.341 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:05:41.454 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T00:05:41.464 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T00:05:41.514 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T00:05:41.556 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T00:05:41.569 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T00:05:41.593 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T00:05:41.627 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T00:05:41.642 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T00:05:41.682 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T00:05:41.693 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T00:05:41.726 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T00:05:41.745 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T00:05:41.773 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T00:05:41.786 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T00:05:41.809 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T00:05:41.830 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T00:05:41.852 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T00:05:41.863 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T00:05:41.889 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T00:05:41.894 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T00:05:41.959 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T00:05:42.012 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T00:05:42.043 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T00:05:42.086 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T00:05:42.105 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T00:05:42.154 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T00:05:42.162 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T00:05:42.195 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T00:05:42.196 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T00:05:42.240 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T00:05:42.268 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T00:05:42.306 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T00:05:42.339 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T00:05:42.346 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T00:05:42.384 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T00:05:42.386 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T00:05:42.420 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T00:05:42.444 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T00:05:42.458 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T00:05:42.540 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T00:05:42.592 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T00:05:42.644 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T00:05:42.672 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T00:05:42.713 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T00:05:42.741 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T00:05:42.778 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T00:05:42.814 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T00:05:42.846 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T00:05:42.876 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T00:05:42.910 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T00:05:42.952 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T00:05:42.994 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T00:05:43.028 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T00:05:43.114 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T00:05:43.158 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T00:05:43.183 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T00:05:43.189 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T00:05:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T00:05:43.291 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T00:05:43.294 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T00:05:43.342 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T00:05:43.348 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T00:05:43.383 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T00:05:43.426 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T00:05:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T00:05:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T00:05:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T00:05:43.797 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T00:05:43.829 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T00:05:43.865 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T00:05:43.912 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T00:05:43.948 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T00:05:43.991 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T00:05:44.034 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T00:05:44.063 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T00:05:44.104 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T00:05:44.137 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T00:05:44.171 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T00:05:44.218 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T00:05:44.236 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T00:05:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T00:05:44.319 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T00:05:44.363 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T00:05:44.371 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T00:05:44.465 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T00:05:44.498 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T00:05:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T00:05:44.682 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T00:05:44.708 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T00:05:44.712 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T00:05:44.742 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T00:05:44.768 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T00:05:44.790 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T00:05:44.921 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T00:05:45.295 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T00:05:45.336 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T00:05:45.378 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T00:05:45.414 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T00:05:45.439 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T00:05:45.451 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T00:05:45.491 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T00:05:45.520 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T00:05:45.547 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T00:05:45.552 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T00:05:45.575 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T00:05:45.617 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T00:05:45.651 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T00:05:45.739 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T00:05:45.790 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T00:05:45.821 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T00:05:45.937 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T00:05:45.989 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T00:05:46.720 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T00:05:46.769 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T00:05:46.789 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T00:05:46.803 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T00:05:46.874 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T00:05:46.901 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T00:05:46.903 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T00:05:46.934 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T00:05:46.950 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T00:05:46.995 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T00:05:47.083 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T00:05:47.123 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T00:05:47.247 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T00:05:47.289 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:05:47.307 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T00:05:47.337 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T00:05:47.363 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:05:47.394 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T00:05:47.414 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T00:05:47.437 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T00:05:47.918 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T00:05:47.950 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:05:48.013 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:05:48.044 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T00:05:48.064 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T00:05:48.188 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T00:05:49.453 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T00:05:49.552 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T00:05:49.579 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T00:05:49.613 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T00:05:49.923 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T00:05:49.963 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:05:50.066 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:05:50.098 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T00:05:50.134 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T00:05:50.363 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:05:50.548 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:05:50.614 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T00:05:50.646 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:05:50.716 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:05:50.739 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T00:05:51.109 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:05:51.152 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:05:51.227 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:05:51.227 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T00:05:51.227 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T00:05:51.228 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:05:51.261 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:05:53.059 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:05:53.096 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:05:53.669 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:05:53.708 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:05:53.802 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:05:53.802 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T00:05:53.802 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T00:05:53.802 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:05:53.834 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:06:03.028 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:06:03.028 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /sys 2024-09-16T00:06:03.028 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /proc 2024-09-16T00:06:03.028 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /mnt 2024-09-16T00:06:03.028 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /var/tmp 2024-09-16T00:06:03.028 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /home 2024-09-16T00:06:03.029 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /root 2024-09-16T00:06:03.029 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /tmp 2024-09-16T00:06:03.029 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:03.206 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:06:03.223 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:06:03.223 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:03.224 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T00:06:03.224 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:04.676 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:06:04.703 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:06:04.704 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:04.704 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T00:06:04.704 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:06:04.704 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:06:04.704 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:04.927 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:06:04.951 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:06:04.951 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:04.951 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T00:06:04.951 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:06:04.952 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:06:04.952 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:05.394 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:06:05.418 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:06:05.418 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:05.418 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T00:06:05.418 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:06:05.418 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:06:05.418 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:05.525 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T00:06:05.581 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T00:06:05.636 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T00:06:05.751 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:06:05.751 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2024-09-16T00:06:05.751 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2024-09-16T00:06:05.752 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2024-09-16T00:06:05.752 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2024-09-16T00:06:05.752 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2024-09-16T00:06:05.752 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /root 2024-09-16T00:06:05.752 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2024-09-16T00:06:05.752 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:05.820 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T00:06:05.869 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T00:06:05.939 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T00:06:05.947 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:06:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:06:05.963 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:05.963 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T00:06:05.963 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:06.027 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T00:06:06.167 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T00:06:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T00:06:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T00:06:06.279 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T00:06:06.691 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:06:06.720 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:06:06.751 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T00:06:06.813 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T00:06:06.851 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:06:07.433 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:06:07.460 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:06:07.460 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:07.461 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T00:06:07.461 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:06:07.461 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:06:07.461 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:07.684 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:06:07.707 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:06:07.707 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:07.708 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T00:06:07.708 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:06:07.708 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:06:07.708 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:06:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:06:08.143 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:06:08.166 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:06:08.166 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:08.166 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T00:06:08.166 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:06:08.167 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:06:08.167 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:08.273 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T00:06:08.331 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T00:06:08.386 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T00:06:08.574 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T00:06:08.627 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T00:06:08.707 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T00:06:08.796 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T00:06:08.895 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:06:08.936 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T00:06:08.937 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:06:08.982 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T00:06:09.018 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T00:06:09.040 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:06:09.045 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T00:06:09.208 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T00:06:09.238 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:06:09.262 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:06:09.262 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:09.262 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T00:06:09.263 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:06:09.263 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:06:09.263 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:09.324 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:06:09.334 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:06:09.432 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T00:06:09.482 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T00:06:09.508 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:06:09.536 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:06:09.559 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T00:06:09.614 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T00:06:09.642 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:06:10.401 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T00:06:10.433 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:06:10.456 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:06:10.457 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:10.457 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T00:06:10.457 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:06:10.457 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:06:10.457 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:10.500 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:06:10.524 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:06:10.524 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:10.524 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T00:06:10.524 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:10.584 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:06:10.608 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:06:10.608 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:10.608 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T00:06:10.608 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:06:10.609 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:06:10.609 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:10.684 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:06:10.719 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:06:10.910 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:06:10.933 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:06:10.934 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:10.934 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T00:06:10.934 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:06:10.934 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:06:10.934 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:11.681 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:06:11.711 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:06:11.815 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:06:11.994 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T00:06:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:06:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:06:12.046 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:12.046 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T00:06:12.046 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:06:12.046 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:06:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:12.097 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:06:12.109 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:06:12.206 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T00:06:12.258 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T00:06:13.182 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T00:06:13.208 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:06:13.233 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:06:13.233 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:13.233 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T00:06:13.233 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:06:13.233 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:06:13.233 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:13.275 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:06:13.298 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:06:13.298 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:13.299 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T00:06:13.299 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:13.343 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:06:13.366 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:06:13.367 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:13.367 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T00:06:13.367 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:06:13.367 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:06:13.367 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:13.669 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:06:13.692 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:06:13.692 INFO:teuthology.orchestra.run.smithi164.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:06:13.692 INFO:teuthology.orchestra.run.smithi164.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T00:06:13.692 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:06:13.692 INFO:teuthology.orchestra.run.smithi164.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:06:13.693 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:15.496 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T00:06:15.538 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T00:06:15.579 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T00:06:15.611 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T00:06:15.638 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T00:06:15.827 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T00:06:15.876 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T00:06:15.907 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T00:06:15.936 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T00:06:15.982 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T00:06:16.631 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:06:16.672 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:06:16.693 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:06:17.538 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T00:06:17.667 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T00:06:17.712 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T00:06:17.801 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T00:06:17.837 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T00:06:17.862 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T00:06:17.888 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T00:06:17.889 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:06:17.972 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:06:17.973 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:06:18.057 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:06:18.081 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T00:06:18.087 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:06:18.087 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:06:18.256 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T00:06:18.266 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:06:18.266 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:06:18.296 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T00:06:18.347 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T00:06:18.359 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:06:18.378 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T00:06:18.405 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T00:06:18.433 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:06:18.602 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T00:06:18.652 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T00:06:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T00:06:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T00:06:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T00:06:19.309 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:06:19.348 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:06:19.369 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:06:20.234 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T00:06:20.311 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T00:06:20.362 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T00:06:20.453 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T00:06:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T00:06:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T00:06:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T00:06:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:06:20.617 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:06:20.617 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:06:20.701 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:06:20.726 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T00:06:20.732 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:06:20.732 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:06:20.919 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:06:20.920 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:06:21.004 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:06:21.068 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:06:24.007 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:06:24.007 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T00:06:24.007 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T00:06:24.007 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T00:06:24.008 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T00:06:24.008 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T00:06:24.008 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T00:06:24.008 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T00:06:24.008 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:06:24.008 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:06:24.008 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:06:24.008 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T00:06:24.008 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T00:06:24.008 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:06:24.008 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T00:06:24.009 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T00:06:24.009 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T00:06:24.009 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T00:06:24.009 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T00:06:24.009 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T00:06:24.009 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T00:06:24.009 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T00:06:24.009 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T00:06:24.009 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T00:06:24.009 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T00:06:24.009 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T00:06:24.009 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T00:06:24.009 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T00:06:24.010 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T00:06:24.010 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T00:06:24.010 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T00:06:24.021 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T00:06:24.021 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T00:06:24.021 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T00:06:24.021 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T00:06:24.021 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T00:06:24.021 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T00:06:24.021 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T00:06:24.021 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T00:06:24.022 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T00:06:24.022 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T00:06:24.022 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T00:06:24.022 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T00:06:24.022 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T00:06:24.022 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T00:06:24.022 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T00:06:24.022 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T00:06:24.022 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T00:06:24.022 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T00:06:24.022 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T00:06:24.022 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T00:06:24.022 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T00:06:24.023 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T00:06:24.023 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T00:06:24.023 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T00:06:24.023 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T00:06:24.023 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T00:06:24.023 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T00:06:24.023 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T00:06:24.023 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T00:06:24.023 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T00:06:24.023 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T00:06:24.023 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T00:06:24.023 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T00:06:24.023 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T00:06:24.024 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T00:06:24.024 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T00:06:24.024 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T00:06:24.024 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T00:06:24.024 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T00:06:24.024 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T00:06:24.024 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T00:06:24.024 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T00:06:24.024 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T00:06:24.024 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T00:06:24.024 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T00:06:24.024 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T00:06:24.025 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T00:06:24.025 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T00:06:24.025 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T00:06:24.025 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T00:06:24.025 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T00:06:24.025 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T00:06:24.025 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T00:06:24.025 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T00:06:24.025 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T00:06:24.025 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T00:06:24.025 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T00:06:24.025 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T00:06:24.025 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T00:06:24.026 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T00:06:24.026 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T00:06:24.026 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T00:06:24.026 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T00:06:24.026 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T00:06:24.026 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T00:06:24.026 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T00:06:24.026 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T00:06:24.026 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T00:06:24.026 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T00:06:24.026 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T00:06:24.026 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T00:06:24.026 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T00:06:24.027 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T00:06:24.027 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T00:06:24.027 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T00:06:24.027 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T00:06:24.027 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T00:06:24.027 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T00:06:24.027 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T00:06:24.027 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T00:06:24.027 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T00:06:24.027 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T00:06:24.028 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T00:06:24.028 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T00:06:24.028 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T00:06:24.028 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T00:06:24.028 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T00:06:24.028 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T00:06:24.028 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T00:06:24.028 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T00:06:24.028 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T00:06:24.028 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T00:06:24.028 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T00:06:24.028 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T00:06:24.028 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T00:06:24.029 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T00:06:24.029 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T00:06:24.029 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T00:06:24.029 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T00:06:24.029 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T00:06:24.029 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T00:06:24.029 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T00:06:24.029 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T00:06:24.029 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T00:06:24.029 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T00:06:24.029 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T00:06:24.029 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T00:06:24.030 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T00:06:24.030 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T00:06:24.030 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T00:06:24.030 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T00:06:24.030 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T00:06:24.030 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T00:06:24.030 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T00:06:24.030 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T00:06:24.030 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T00:06:24.030 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T00:06:24.030 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T00:06:24.030 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T00:06:24.030 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T00:06:24.031 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T00:06:24.031 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T00:06:24.031 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T00:06:24.031 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T00:06:24.031 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T00:06:24.031 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T00:06:24.031 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T00:06:24.031 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T00:06:24.031 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T00:06:24.031 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T00:06:24.031 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T00:06:24.031 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T00:06:24.031 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T00:06:26.815 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:06:26.816 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:26.816 INFO:teuthology.orchestra.run.smithi139.stdout:Upgraded: 2024-09-16T00:06:26.816 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T00:06:26.816 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T00:06:26.816 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T00:06:26.816 INFO:teuthology.orchestra.run.smithi139.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T00:06:26.816 INFO:teuthology.orchestra.run.smithi139.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.816 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.816 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T00:06:26.816 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T00:06:26.816 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-09-16T00:06:26.816 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T00:06:26.817 INFO:teuthology.orchestra.run.smithi139.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T00:06:26.817 INFO:teuthology.orchestra.run.smithi139.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T00:06:26.817 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T00:06:26.817 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.817 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.817 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.817 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.817 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:26.817 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.817 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.817 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.817 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:26.817 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:26.818 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:26.818 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:26.818 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:26.818 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.818 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.818 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:26.818 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.818 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.818 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.818 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:26.818 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:26.818 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.818 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.819 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:26.819 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T00:06:26.819 INFO:teuthology.orchestra.run.smithi139.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T00:06:26.819 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T00:06:26.819 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T00:06:26.819 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T00:06:26.819 INFO:teuthology.orchestra.run.smithi139.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T00:06:26.819 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T00:06:26.819 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T00:06:26.819 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T00:06:26.819 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T00:06:26.819 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T00:06:26.819 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.820 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.820 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.820 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T00:06:26.820 INFO:teuthology.orchestra.run.smithi139.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T00:06:26.820 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T00:06:26.820 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T00:06:26.820 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T00:06:26.820 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.820 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.820 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T00:06:26.820 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.820 INFO:teuthology.orchestra.run.smithi139.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T00:06:26.820 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T00:06:26.821 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T00:06:26.821 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T00:06:26.821 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T00:06:26.821 INFO:teuthology.orchestra.run.smithi139.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T00:06:26.821 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T00:06:26.821 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T00:06:26.821 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T00:06:26.821 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T00:06:26.821 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T00:06:26.821 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T00:06:26.821 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T00:06:26.821 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T00:06:26.821 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T00:06:26.822 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T00:06:26.822 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T00:06:26.822 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T00:06:26.822 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T00:06:26.822 INFO:teuthology.orchestra.run.smithi139.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T00:06:26.822 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T00:06:26.822 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T00:06:26.822 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T00:06:26.822 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.822 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.822 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.822 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T00:06:26.822 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T00:06:26.823 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T00:06:26.823 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T00:06:26.823 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T00:06:26.823 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T00:06:26.823 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T00:06:26.823 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T00:06:26.823 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T00:06:26.823 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T00:06:26.823 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T00:06:26.823 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T00:06:26.823 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T00:06:26.823 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T00:06:26.823 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T00:06:26.824 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T00:06:26.824 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T00:06:26.824 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T00:06:26.824 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T00:06:26.824 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T00:06:26.824 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T00:06:26.824 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T00:06:26.824 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T00:06:26.824 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T00:06:26.824 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T00:06:26.824 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T00:06:26.824 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T00:06:26.824 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T00:06:26.825 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T00:06:26.825 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T00:06:26.825 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T00:06:26.825 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T00:06:26.825 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T00:06:26.825 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T00:06:26.825 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T00:06:26.825 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T00:06:26.825 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T00:06:26.825 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T00:06:26.825 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T00:06:26.825 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T00:06:26.825 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.826 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.826 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T00:06:26.826 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T00:06:26.826 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T00:06:26.826 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.826 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T00:06:26.826 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T00:06:26.826 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T00:06:26.826 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T00:06:26.826 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T00:06:26.826 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T00:06:26.826 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T00:06:26.826 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T00:06:26.827 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T00:06:26.827 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T00:06:26.827 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T00:06:26.827 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T00:06:26.827 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T00:06:26.827 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T00:06:26.827 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T00:06:26.827 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.827 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.827 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:26.827 INFO:teuthology.orchestra.run.smithi139.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T00:06:26.827 INFO:teuthology.orchestra.run.smithi139.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T00:06:26.827 INFO:teuthology.orchestra.run.smithi139.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T00:06:26.828 INFO:teuthology.orchestra.run.smithi139.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T00:06:26.828 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T00:06:26.828 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T00:06:26.828 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T00:06:26.828 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:26.828 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-09-16T00:06:26.937 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:06:26.937 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T00:06:26.937 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T00:06:26.937 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T00:06:26.937 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T00:06:26.938 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T00:06:26.938 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T00:06:26.938 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T00:06:26.938 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:06:26.938 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:06:26.938 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:06:26.938 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T00:06:26.938 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T00:06:26.938 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:06:26.938 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T00:06:26.938 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T00:06:26.938 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T00:06:26.938 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T00:06:26.939 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T00:06:26.939 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T00:06:26.939 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T00:06:26.939 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T00:06:26.939 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T00:06:26.939 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T00:06:26.939 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T00:06:26.939 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T00:06:26.939 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T00:06:26.939 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T00:06:26.939 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T00:06:26.939 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T00:06:26.940 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T00:06:26.940 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T00:06:26.940 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T00:06:26.940 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T00:06:26.940 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T00:06:26.940 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T00:06:26.940 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T00:06:26.940 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T00:06:26.940 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T00:06:26.940 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T00:06:26.940 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T00:06:26.941 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T00:06:26.941 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T00:06:26.941 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T00:06:26.941 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T00:06:26.941 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T00:06:26.941 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T00:06:26.941 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T00:06:26.941 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T00:06:26.941 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T00:06:26.941 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T00:06:26.941 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T00:06:26.941 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T00:06:26.941 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T00:06:26.942 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T00:06:26.942 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T00:06:26.942 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T00:06:26.942 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T00:06:26.942 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T00:06:26.942 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T00:06:26.942 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T00:06:26.942 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T00:06:26.942 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T00:06:26.942 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T00:06:26.942 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T00:06:26.942 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T00:06:26.942 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T00:06:26.943 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T00:06:26.943 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T00:06:26.943 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T00:06:26.943 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T00:06:26.943 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T00:06:26.943 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T00:06:26.943 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T00:06:26.943 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T00:06:26.943 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T00:06:26.943 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T00:06:26.943 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T00:06:26.943 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T00:06:26.943 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T00:06:26.943 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T00:06:26.944 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T00:06:26.944 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T00:06:26.944 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T00:06:26.944 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T00:06:26.944 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T00:06:26.944 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T00:06:26.944 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T00:06:26.944 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T00:06:26.944 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T00:06:26.944 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T00:06:26.944 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T00:06:26.944 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T00:06:26.945 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T00:06:26.945 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T00:06:26.945 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T00:06:26.945 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T00:06:26.945 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T00:06:26.945 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T00:06:26.945 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T00:06:26.945 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T00:06:26.945 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T00:06:26.945 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T00:06:26.945 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T00:06:26.945 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T00:06:26.945 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T00:06:26.945 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T00:06:26.946 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T00:06:26.946 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T00:06:26.946 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T00:06:26.946 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T00:06:26.946 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T00:06:26.946 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T00:06:26.946 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T00:06:26.946 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T00:06:26.946 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T00:06:26.946 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T00:06:26.946 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T00:06:26.946 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T00:06:26.946 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T00:06:26.947 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T00:06:26.947 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T00:06:26.947 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T00:06:26.947 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T00:06:26.947 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T00:06:26.947 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T00:06:26.947 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T00:06:26.947 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T00:06:26.947 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T00:06:26.947 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T00:06:26.947 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T00:06:26.947 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T00:06:26.947 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T00:06:26.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T00:06:26.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T00:06:26.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T00:06:26.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T00:06:26.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T00:06:26.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T00:06:26.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T00:06:26.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T00:06:26.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T00:06:26.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T00:06:26.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T00:06:26.948 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T00:06:26.949 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T00:06:26.949 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T00:06:26.949 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T00:06:26.949 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T00:06:26.949 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T00:06:26.949 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T00:06:26.949 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T00:06:26.949 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T00:06:26.949 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T00:06:26.949 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T00:06:26.949 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T00:06:26.949 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T00:06:26.949 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T00:06:26.950 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T00:06:26.950 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T00:06:26.950 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T00:06:26.950 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T00:06:26.950 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T00:06:26.950 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T00:06:27.271 DEBUG:teuthology.parallel:result is None 2024-09-16T00:06:29.467 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:06:29.467 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:29.467 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2024-09-16T00:06:29.467 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T00:06:29.467 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T00:06:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T00:06:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T00:06:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T00:06:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T00:06:29.468 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2024-09-16T00:06:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T00:06:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T00:06:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T00:06:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T00:06:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.470 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.470 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:29.470 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.470 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.470 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.470 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:29.470 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:29.470 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.470 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.470 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:06:29.470 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T00:06:29.470 INFO:teuthology.orchestra.run.smithi164.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T00:06:29.470 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T00:06:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T00:06:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T00:06:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T00:06:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T00:06:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T00:06:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T00:06:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T00:06:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T00:06:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T00:06:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T00:06:29.472 INFO:teuthology.orchestra.run.smithi164.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T00:06:29.472 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T00:06:29.472 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T00:06:29.472 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.472 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.472 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T00:06:29.472 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.472 INFO:teuthology.orchestra.run.smithi164.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T00:06:29.473 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T00:06:29.473 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T00:06:29.473 INFO:teuthology.orchestra.run.smithi164.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T00:06:29.473 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T00:06:29.473 INFO:teuthology.orchestra.run.smithi164.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T00:06:29.473 INFO:teuthology.orchestra.run.smithi164.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T00:06:29.473 INFO:teuthology.orchestra.run.smithi164.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T00:06:29.473 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T00:06:29.473 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T00:06:29.473 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T00:06:29.473 INFO:teuthology.orchestra.run.smithi164.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T00:06:29.473 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T00:06:29.473 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T00:06:29.474 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T00:06:29.474 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T00:06:29.474 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T00:06:29.474 INFO:teuthology.orchestra.run.smithi164.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T00:06:29.474 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T00:06:29.474 INFO:teuthology.orchestra.run.smithi164.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T00:06:29.474 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T00:06:29.474 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T00:06:29.474 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T00:06:29.474 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.474 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.474 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.474 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T00:06:29.474 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T00:06:29.475 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T00:06:29.475 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T00:06:29.475 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T00:06:29.475 INFO:teuthology.orchestra.run.smithi164.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T00:06:29.475 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T00:06:29.475 INFO:teuthology.orchestra.run.smithi164.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T00:06:29.475 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T00:06:29.475 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T00:06:29.475 INFO:teuthology.orchestra.run.smithi164.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T00:06:29.475 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T00:06:29.475 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T00:06:29.475 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T00:06:29.475 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T00:06:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T00:06:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T00:06:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T00:06:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T00:06:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T00:06:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T00:06:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T00:06:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T00:06:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T00:06:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T00:06:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T00:06:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T00:06:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T00:06:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T00:06:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T00:06:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T00:06:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T00:06:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T00:06:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T00:06:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T00:06:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T00:06:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T00:06:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T00:06:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T00:06:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T00:06:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T00:06:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T00:06:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T00:06:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T00:06:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T00:06:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T00:06:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T00:06:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T00:06:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T00:06:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T00:06:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T00:06:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T00:06:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T00:06:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T00:06:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T00:06:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T00:06:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T00:06:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T00:06:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:06:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T00:06:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T00:06:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T00:06:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T00:06:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T00:06:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T00:06:29.481 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T00:06:29.481 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:29.481 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2024-09-16T00:06:29.902 DEBUG:teuthology.parallel:result is None 2024-09-16T00:06:29.903 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:06:30.047 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T00:06:30.073 INFO:teuthology.orchestra.run.smithi139.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T00:06:30.074 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T00:06:30.074 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T00:06:30.076 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:06:30.213 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T00:06:30.236 INFO:teuthology.orchestra.run.smithi164.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T00:06:30.236 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T00:06:30.236 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T00:06:30.239 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T00:06:30.239 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:06:30.239 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T00:06:30.272 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:06:30.272 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T00:06:30.303 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T00:06:30.303 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:06:30.304 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T00:06:30.342 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T00:06:30.410 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:06:30.410 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T00:06:30.436 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T00:06:30.501 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T00:06:30.501 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:06:30.501 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T00:06:30.528 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T00:06:30.597 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:06:30.597 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T00:06:30.621 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T00:06:30.686 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T00:06:30.686 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:06:30.686 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T00:06:30.712 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T00:06:30.777 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:06:30.777 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T00:06:30.802 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T00:06:30.868 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T00:06:31.011 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T00:06:31.011 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 777 /var/log/ceph 2024-09-16T00:06:31.014 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /var/log/ceph 2024-09-16T00:06:31.039 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T00:06:31.040 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T00:06:31.079 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T00:06:31.104 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T00:06:31.105 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T00:06:31.144 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T00:06:31.213 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-16T00:06:31.213 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-16T00:06:31.213 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-16T00:06:31.214 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:06:31.214 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-16T00:06:31.229 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-16T00:06:31.337 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-16T00:06:31.379 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:06:31.379 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-16T00:06:31.395 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-16T00:06:31.499 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-16T00:06:31.530 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T00:06:31.530 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-16T00:06:31.530 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906374', 'branch': 'squid', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/fsstress}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7906374', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ad35db06e8924912f01e32b35576297c0f1352b4'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 3393, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'sleep_before_teardown': 0, 'subset': '294/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'ad35db06e8924912f01e32b35576297c0f1352b4', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi139.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF80ej5qAloOHRRNdDb5hB76D13JReZBWyWPm8uPZnzBQQCLb9RA3lm0ujaOHUpSVuyB3zts7RTwolkrrPznXvY=', 'smithi164.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDhiPGctwJicHDFmkt2NbbZfku1RX+7WSVleApUhlwR0NVYMt+aa0ZeF91TCgfpO7JKhV13Yez3x+rGQgU3V4Jw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'thrashosds': None}, {'workunit': {'clients': {'all': ['suites/fsstress.sh']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_21:24:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T00:06:31.531 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T00:06:31.534 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T00:06:31.579 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T00:06:31.589 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T00:06:31.644 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:06:31.644 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T00:06:31.666 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T00:06:31.666 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_1 2024-09-16T00:06:31.720 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T00:06:31.720 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:06:31.721 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-16T00:06:31.721 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:06:31.721 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:06:31.721 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-16 00:06:18.804920866 +0000 2024-09-16T00:06:31.721 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-16 00:04:35.884993136 +0000 2024-09-16T00:06:31.721 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-16 00:04:35.884993136 +0000 2024-09-16T00:06:31.721 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-16 00:04:35.884993136 +0000 2024-09-16T00:06:31.721 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T00:06:31.783 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-16T00:06:31.784 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-16T00:06:31.784 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000184924 s, 2.8 MB/s 2024-09-16T00:06:31.785 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T00:06:31.842 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_2 2024-09-16T00:06:31.897 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T00:06:31.897 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:06:31.897 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T00:06:31.897 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:06:31.897 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:06:31.897 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-16 00:06:18.805920845 +0000 2024-09-16T00:06:31.897 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-16 00:04:36.209986448 +0000 2024-09-16T00:06:31.897 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-16 00:04:36.209986448 +0000 2024-09-16T00:06:31.897 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-16 00:04:36.209986448 +0000 2024-09-16T00:06:31.897 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T00:06:31.959 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-16T00:06:31.959 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-16T00:06:31.959 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000170665 s, 3.0 MB/s 2024-09-16T00:06:31.960 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T00:06:32.016 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_3 2024-09-16T00:06:32.071 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T00:06:32.072 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:06:32.072 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-16T00:06:32.072 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:06:32.072 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:06:32.072 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-16 00:06:18.805920845 +0000 2024-09-16T00:06:32.072 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-16 00:04:36.537979699 +0000 2024-09-16T00:06:32.072 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-16 00:04:36.537979699 +0000 2024-09-16T00:06:32.072 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-16 00:04:36.537979699 +0000 2024-09-16T00:06:32.072 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T00:06:32.134 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-16T00:06:32.134 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-16T00:06:32.135 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000203936 s, 2.5 MB/s 2024-09-16T00:06:32.136 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T00:06:32.192 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_4 2024-09-16T00:06:32.246 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T00:06:32.246 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:06:32.247 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-16T00:06:32.247 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:06:32.247 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:06:32.247 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-16 00:06:18.805920845 +0000 2024-09-16T00:06:32.247 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-16 00:04:36.872972806 +0000 2024-09-16T00:06:32.247 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-16 00:04:36.872972806 +0000 2024-09-16T00:06:32.247 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-09-16 00:04:36.872972806 +0000 2024-09-16T00:06:32.247 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T00:06:32.310 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-16T00:06:32.310 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-16T00:06:32.310 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000230433 s, 2.2 MB/s 2024-09-16T00:06:32.311 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T00:06:32.368 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T00:06:32.368 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:06:32.368 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T00:06:32.385 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T00:06:32.386 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2024-09-16T00:06:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T00:06:32.442 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:06:32.442 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-16T00:06:32.442 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:06:32.442 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:06:32.442 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 00:06:21.471835747 +0000 2024-09-16T00:06:32.443 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 00:04:36.226366091 +0000 2024-09-16T00:06:32.443 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 00:04:36.226366091 +0000 2024-09-16T00:06:32.443 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-16 00:04:36.226366091 +0000 2024-09-16T00:06:32.443 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T00:06:32.507 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T00:06:32.507 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T00:06:32.508 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000273687 s, 1.9 MB/s 2024-09-16T00:06:32.509 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T00:06:32.568 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2024-09-16T00:06:32.624 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T00:06:32.624 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:06:32.624 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T00:06:32.625 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:06:32.625 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:06:32.625 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 00:06:21.472835731 +0000 2024-09-16T00:06:32.625 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 00:04:36.558360846 +0000 2024-09-16T00:06:32.625 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 00:04:36.558360846 +0000 2024-09-16T00:06:32.625 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-16 00:04:36.558360846 +0000 2024-09-16T00:06:32.625 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T00:06:32.689 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T00:06:32.689 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T00:06:32.689 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000225745 s, 2.3 MB/s 2024-09-16T00:06:32.691 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T00:06:32.748 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2024-09-16T00:06:32.804 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T00:06:32.804 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:06:32.804 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-16T00:06:32.804 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:06:32.805 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:06:32.805 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 00:06:21.472835731 +0000 2024-09-16T00:06:32.805 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 00:04:36.873355869 +0000 2024-09-16T00:06:32.805 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 00:04:36.873355869 +0000 2024-09-16T00:06:32.805 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-16 00:04:36.873355869 +0000 2024-09-16T00:06:32.805 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T00:06:32.868 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T00:06:32.868 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T00:06:32.868 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000203488 s, 2.5 MB/s 2024-09-16T00:06:32.869 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T00:06:32.926 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2024-09-16T00:06:32.982 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T00:06:32.982 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:06:32.982 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-16T00:06:32.982 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:06:32.982 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:06:32.982 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2024-09-16 00:06:21.472835731 +0000 2024-09-16T00:06:32.982 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2024-09-16 00:04:37.191350844 +0000 2024-09-16T00:06:32.982 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2024-09-16 00:04:37.191350844 +0000 2024-09-16T00:06:32.982 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: 2024-09-16 00:04:37.191350844 +0000 2024-09-16T00:06:32.983 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T00:06:33.045 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2024-09-16T00:06:33.095 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2024-09-16T00:06:33.095 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000246066 s, 2.1 MB/s 2024-09-16T00:06:33.096 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T00:06:33.153 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T00:06:33.153 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi139.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T00:06:33.153 INFO:tasks.ceph:Generating config... 2024-09-16T00:06:33.154 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-16T00:06:33.154 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-09-16T00:06:33.154 INFO:tasks.ceph:[client] debug client = 20 2024-09-16T00:06:33.154 INFO:tasks.ceph:[client] debug ms = 1 2024-09-16T00:06:33.155 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-16T00:06:33.155 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-16T00:06:33.155 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-16T00:06:33.155 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-16T00:06:33.155 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-16T00:06:33.155 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-09-16T00:06:33.155 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-09-16T00:06:33.155 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-09-16T00:06:33.155 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-09-16T00:06:33.155 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-16T00:06:33.156 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-16T00:06:33.156 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-09-16T00:06:33.156 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-16T00:06:33.156 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-16T00:06:33.156 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-09-16T00:06:33.156 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-09-16T00:06:33.156 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-16T00:06:33.156 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-16T00:06:33.156 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-16T00:06:33.156 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-16T00:06:33.156 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T00:06:33.156 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T00:06:33.157 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T00:06:33.157 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T00:06:33.157 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T00:06:33.157 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-16T00:06:33.157 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-16T00:06:33.157 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-16T00:06:33.157 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-16T00:06:33.157 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-16T00:06:33.157 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-16T00:06:33.157 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-16T00:06:33.157 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-16T00:06:33.157 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T00:06:33.157 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T00:06:33.157 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-16T00:06:33.158 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-16T00:06:33.158 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-16T00:06:33.158 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-16T00:06:33.158 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-16T00:06:33.158 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-16T00:06:33.158 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-16T00:06:33.158 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-16T00:06:33.158 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-16T00:06:33.158 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T00:06:33.158 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T00:06:33.210 INFO:teuthology.orchestra.run.smithi139.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T00:06:33.213 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T00:06:33.260 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T00:06:33.286 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.139'), ('mon.b', '172.21.15.164'), ('mon.c', '[v2:172.21.15.164:3301,v1:172.21.15.164:6790]')] 2024-09-16T00:06:33.287 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.139,172.21.15.164,[v2:172.21.15.164:3301,v1:172.21.15.164:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-16T00:06:33.288 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:06:33.288 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T00:06:33.343 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.139 --add b 172.21.15.164 --addv c '[v2:172.21.15.164:3301,v1:172.21.15.164:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T00:06:33.433 INFO:teuthology.orchestra.run.smithi139.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T00:06:33.433 INFO:teuthology.orchestra.run.smithi139.stdout:monmaptool: generated fsid 2d3366e5-6a59-4b59-93b2-615e72ac7015 2024-09-16T00:06:33.433 INFO:teuthology.orchestra.run.smithi139.stdout:setting min_mon_release = quincy 2024-09-16T00:06:33.433 INFO:teuthology.orchestra.run.smithi139.stdout:epoch 0 2024-09-16T00:06:33.433 INFO:teuthology.orchestra.run.smithi139.stdout:fsid 2d3366e5-6a59-4b59-93b2-615e72ac7015 2024-09-16T00:06:33.433 INFO:teuthology.orchestra.run.smithi139.stdout:last_changed 2024-09-16T00:06:33.432904+0000 2024-09-16T00:06:33.433 INFO:teuthology.orchestra.run.smithi139.stdout:created 2024-09-16T00:06:33.432904+0000 2024-09-16T00:06:33.433 INFO:teuthology.orchestra.run.smithi139.stdout:min_mon_release 17 (quincy) 2024-09-16T00:06:33.434 INFO:teuthology.orchestra.run.smithi139.stdout:election_strategy: 1 2024-09-16T00:06:33.434 INFO:teuthology.orchestra.run.smithi139.stdout:0: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.a 2024-09-16T00:06:33.434 INFO:teuthology.orchestra.run.smithi139.stdout:1: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.b 2024-09-16T00:06:33.434 INFO:teuthology.orchestra.run.smithi139.stdout:2: [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] mon.c 2024-09-16T00:06:33.434 INFO:teuthology.orchestra.run.smithi139.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-16T00:06:33.435 DEBUG:teuthology.orchestra.run.smithi139:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T00:06:33.452 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2d3366e5-6a59-4b59-93b2-615e72ac7015... 2024-09-16T00:06:33.453 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T00:06:33.495 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T00:06:33.535 INFO:teuthology.orchestra.run.smithi139.stdout:[global] 2024-09-16T00:06:33.536 INFO:teuthology.orchestra.run.smithi139.stdout: chdir = "" 2024-09-16T00:06:33.536 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T00:06:33.536 INFO:teuthology.orchestra.run.smithi139.stdout: auth supported = cephx 2024-09-16T00:06:33.536 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.536 INFO:teuthology.orchestra.run.smithi139.stdout: filestore xattr use omap = true 2024-09-16T00:06:33.536 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.536 INFO:teuthology.orchestra.run.smithi139.stdout: mon clock drift allowed = 1.000 2024-09-16T00:06:33.537 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.537 INFO:teuthology.orchestra.run.smithi139.stdout: osd crush chooseleaf type = 0 2024-09-16T00:06:33.537 INFO:teuthology.orchestra.run.smithi139.stdout: auth debug = true 2024-09-16T00:06:33.537 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.537 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on old message = true 2024-09-16T00:06:33.538 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on bug = true 2024-09-16T00:06:33.538 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.538 INFO:teuthology.orchestra.run.smithi139.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T00:06:33.538 INFO:teuthology.orchestra.run.smithi139.stdout: mon pg warn max object skew = 0 2024-09-16T00:06:33.538 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.538 INFO:teuthology.orchestra.run.smithi139.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T00:06:33.538 INFO:teuthology.orchestra.run.smithi139.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T00:06:33.538 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.538 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default size = 2 2024-09-16T00:06:33.538 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.538 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow primary affinity = true 2024-09-16T00:06:33.539 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow pg remap = true 2024-09-16T00:06:33.539 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on legacy crush tunables = false 2024-09-16T00:06:33.539 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on crush straw calc version zero = false 2024-09-16T00:06:33.539 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on no sortbitwise = false 2024-09-16T00:06:33.539 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on osd down out interval zero = false 2024-09-16T00:06:33.539 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on too few osds = false 2024-09-16T00:06:33.539 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T00:06:33.539 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T00:06:33.539 INFO:teuthology.orchestra.run.smithi139.stdout: mon_allow_pool_size_one = true 2024-09-16T00:06:33.539 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.539 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T00:06:33.539 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.539 INFO:teuthology.orchestra.run.smithi139.stdout: osd default data pool replay window = 5 2024-09-16T00:06:33.539 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.540 INFO:teuthology.orchestra.run.smithi139.stdout: mon allow pool delete = true 2024-09-16T00:06:33.540 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.540 INFO:teuthology.orchestra.run.smithi139.stdout: mon cluster log file level = debug 2024-09-16T00:06:33.540 INFO:teuthology.orchestra.run.smithi139.stdout: debug asserts on shutdown = true 2024-09-16T00:06:33.540 INFO:teuthology.orchestra.run.smithi139.stdout: mon health detail to clog = false 2024-09-16T00:06:33.540 INFO:teuthology.orchestra.run.smithi139.stdout: mon host = "172.21.15.139,172.21.15.164,[v2:172.21.15.164:3301,v1:172.21.15.164:6790]" 2024-09-16T00:06:33.540 INFO:teuthology.orchestra.run.smithi139.stdout: fsid = 2d3366e5-6a59-4b59-93b2-615e72ac7015 2024-09-16T00:06:33.540 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.540 INFO:teuthology.orchestra.run.smithi139.stdout:[osd] 2024-09-16T00:06:33.540 INFO:teuthology.orchestra.run.smithi139.stdout: osd journal size = 100 2024-09-16T00:06:33.541 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.541 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub load threshold = 5.0 2024-09-16T00:06:33.541 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub max interval = 600 2024-09-16T00:06:33.541 INFO:teuthology.orchestra.run.smithi139.stdout: osd mclock profile = high_recovery_ops 2024-09-16T00:06:33.541 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.541 INFO:teuthology.orchestra.run.smithi139.stdout: osd recover clone overlap = true 2024-09-16T00:06:33.541 INFO:teuthology.orchestra.run.smithi139.stdout: osd recovery max chunk = 1048576 2024-09-16T00:06:33.541 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.542 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug shutdown = true 2024-09-16T00:06:33.542 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug op order = true 2024-09-16T00:06:33.542 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug verify stray on activate = true 2024-09-16T00:06:33.542 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug trim objects = true 2024-09-16T00:06:33.542 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.542 INFO:teuthology.orchestra.run.smithi139.stdout: osd open classes on start = true 2024-09-16T00:06:33.542 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug pg log writeout = true 2024-09-16T00:06:33.542 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.542 INFO:teuthology.orchestra.run.smithi139.stdout: osd deep scrub update digest min age = 30 2024-09-16T00:06:33.543 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.543 INFO:teuthology.orchestra.run.smithi139.stdout: osd map max advance = 10 2024-09-16T00:06:33.543 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.543 INFO:teuthology.orchestra.run.smithi139.stdout: journal zero on create = true 2024-09-16T00:06:33.543 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.543 INFO:teuthology.orchestra.run.smithi139.stdout: filestore ondisk finisher threads = 3 2024-09-16T00:06:33.543 INFO:teuthology.orchestra.run.smithi139.stdout: filestore apply finisher threads = 3 2024-09-16T00:06:33.543 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.543 INFO:teuthology.orchestra.run.smithi139.stdout: bdev debug aio = true 2024-09-16T00:06:33.543 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug misdirected ops = true 2024-09-16T00:06:33.544 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore block size = 96636764160 2024-09-16T00:06:33.544 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore compression mode = aggressive 2024-09-16T00:06:33.544 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore fsck on mount = True 2024-09-16T00:06:33.544 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluefs = 20 2024-09-16T00:06:33.544 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluestore = 20 2024-09-16T00:06:33.544 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-09-16T00:06:33.544 INFO:teuthology.orchestra.run.smithi139.stdout: debug osd = 20 2024-09-16T00:06:33.544 INFO:teuthology.orchestra.run.smithi139.stdout: debug rocksdb = 10 2024-09-16T00:06:33.544 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T00:06:33.544 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd full ratio = 0.9 2024-09-16T00:06:33.544 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd nearfull ratio = 0.8 2024-09-16T00:06:33.544 INFO:teuthology.orchestra.run.smithi139.stdout: osd blocked scrub grace period = 3600 2024-09-16T00:06:33.545 INFO:teuthology.orchestra.run.smithi139.stdout: osd failsafe full ratio = 0.95 2024-09-16T00:06:33.545 INFO:teuthology.orchestra.run.smithi139.stdout: osd max markdown count = 1000 2024-09-16T00:06:33.545 INFO:teuthology.orchestra.run.smithi139.stdout: osd objectstore = bluestore 2024-09-16T00:06:33.545 INFO:teuthology.orchestra.run.smithi139.stdout: osd op complaint time = 180 2024-09-16T00:06:33.545 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.545 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr] 2024-09-16T00:06:33.545 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-09-16T00:06:33.545 INFO:teuthology.orchestra.run.smithi139.stdout: debug mgr = 20 2024-09-16T00:06:33.545 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-09-16T00:06:33.545 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-09-16T00:06:33.545 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-09-16T00:06:33.545 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-09-16T00:06:33.545 INFO:teuthology.orchestra.run.smithi139.stdout: mgr/telemetry/nag = false 2024-09-16T00:06:33.546 INFO:teuthology.orchestra.run.smithi139.stdout: debug client = 20 2024-09-16T00:06:33.546 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.546 INFO:teuthology.orchestra.run.smithi139.stdout:[mon] 2024-09-16T00:06:33.546 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-09-16T00:06:33.546 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-09-16T00:06:33.546 INFO:teuthology.orchestra.run.smithi139.stdout: debug paxos = 20 2024-09-16T00:06:33.546 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-09-16T00:06:33.546 INFO:teuthology.orchestra.run.smithi139.stdout: mon data avail warn = 5 2024-09-16T00:06:33.546 INFO:teuthology.orchestra.run.smithi139.stdout: mon mgr mkfs grace = 240 2024-09-16T00:06:33.546 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-09-16T00:06:33.546 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd reporter subtree level = osd 2024-09-16T00:06:33.546 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd prime pg temp = true 2024-09-16T00:06:33.547 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-09-16T00:06:33.547 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.547 INFO:teuthology.orchestra.run.smithi139.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T00:06:33.547 INFO:teuthology.orchestra.run.smithi139.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T00:06:33.547 INFO:teuthology.orchestra.run.smithi139.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T00:06:33.547 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.547 INFO:teuthology.orchestra.run.smithi139.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T00:06:33.547 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T00:06:33.547 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T00:06:33.547 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.547 INFO:teuthology.orchestra.run.smithi139.stdout: # 1m isn't quite enough 2024-09-16T00:06:33.548 INFO:teuthology.orchestra.run.smithi139.stdout: mon_down_mkfs_grace = 2m 2024-09-16T00:06:33.548 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.548 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_filestore_osds = false 2024-09-16T00:06:33.548 INFO:teuthology.orchestra.run.smithi139.stdout: mon down mkfs grace = 300 2024-09-16T00:06:33.548 INFO:teuthology.orchestra.run.smithi139.stdout: mon op complaint time = 120 2024-09-16T00:06:33.548 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on pool no app = False 2024-09-16T00:06:33.549 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:06:33.549 INFO:teuthology.orchestra.run.smithi139.stdout:[client] 2024-09-16T00:06:33.549 INFO:teuthology.orchestra.run.smithi139.stdout: rgw cache enabled = true 2024-09-16T00:06:33.549 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable ops log = true 2024-09-16T00:06:33.549 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable usage log = true 2024-09-16T00:06:33.549 INFO:teuthology.orchestra.run.smithi139.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T00:06:33.550 INFO:teuthology.orchestra.run.smithi139.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T00:06:33.550 INFO:teuthology.orchestra.run.smithi139.stdout: client mount timeout = 600 2024-09-16T00:06:33.550 INFO:teuthology.orchestra.run.smithi139.stdout: client_shutdown_timeout = 120 2024-09-16T00:06:33.550 INFO:teuthology.orchestra.run.smithi139.stdout: debug client = 20 2024-09-16T00:06:33.550 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-09-16T00:06:33.550 INFO:teuthology.orchestra.run.smithi139.stdout: rados mon op timeout = 900 2024-09-16T00:06:33.550 INFO:teuthology.orchestra.run.smithi139.stdout: rados osd op timeout = 900 2024-09-16T00:06:33.551 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.a] 2024-09-16T00:06:33.551 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.b] 2024-09-16T00:06:33.551 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.c] 2024-09-16T00:06:33.551 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.a] 2024-09-16T00:06:33.551 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.c] 2024-09-16T00:06:33.551 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.e] 2024-09-16T00:06:33.551 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.b] 2024-09-16T00:06:33.551 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.d] 2024-09-16T00:06:33.552 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.f] 2024-09-16T00:06:33.552 INFO:teuthology.orchestra.run.smithi139.stdout:[mds] 2024-09-16T00:06:33.552 INFO:teuthology.orchestra.run.smithi139.stdout: debug mds = 20 2024-09-16T00:06:33.552 INFO:teuthology.orchestra.run.smithi139.stdout: debug mds balancer = 20 2024-09-16T00:06:33.552 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-09-16T00:06:33.552 INFO:teuthology.orchestra.run.smithi139.stdout: mds bal fragment size max = 10000 2024-09-16T00:06:33.552 INFO:teuthology.orchestra.run.smithi139.stdout: mds bal merge size = 5 2024-09-16T00:06:33.552 INFO:teuthology.orchestra.run.smithi139.stdout: mds bal split bits = 3 2024-09-16T00:06:33.553 INFO:teuthology.orchestra.run.smithi139.stdout: mds bal split size = 100 2024-09-16T00:06:33.553 INFO:teuthology.orchestra.run.smithi139.stdout: mds debug frag = True 2024-09-16T00:06:33.553 INFO:teuthology.orchestra.run.smithi139.stdout: mds debug scatterstat = True 2024-09-16T00:06:33.553 INFO:teuthology.orchestra.run.smithi139.stdout: mds oft prefetch dirfrags = True 2024-09-16T00:06:33.553 INFO:teuthology.orchestra.run.smithi139.stdout: mds op complaint time = 180 2024-09-16T00:06:33.553 INFO:teuthology.orchestra.run.smithi139.stdout: mds verify scatter = True 2024-09-16T00:06:33.554 INFO:teuthology.orchestra.run.smithi139.stdout: mds_dir_prefetch = True 2024-09-16T00:06:33.554 INFO:teuthology.orchestra.run.smithi139.stdout: mds_sleep_rank_change = 5000000.0 2024-09-16T00:06:33.554 INFO:teuthology.orchestra.run.smithi139.stdout: osd op complaint time = 180 2024-09-16T00:06:33.554 INFO:teuthology.orchestra.run.smithi139.stdout: rados mon op timeout = 900 2024-09-16T00:06:33.554 INFO:teuthology.orchestra.run.smithi139.stdout: rados osd op timeout = 900 2024-09-16T00:06:33.557 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2024-09-16T00:06:33.557 INFO:teuthology.orchestra.run.smithi164.stdout: chdir = "" 2024-09-16T00:06:33.557 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T00:06:33.557 INFO:teuthology.orchestra.run.smithi164.stdout: auth supported = cephx 2024-09-16T00:06:33.557 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.557 INFO:teuthology.orchestra.run.smithi164.stdout: filestore xattr use omap = true 2024-09-16T00:06:33.557 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.557 INFO:teuthology.orchestra.run.smithi164.stdout: mon clock drift allowed = 1.000 2024-09-16T00:06:33.557 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.557 INFO:teuthology.orchestra.run.smithi164.stdout: osd crush chooseleaf type = 0 2024-09-16T00:06:33.558 INFO:teuthology.orchestra.run.smithi164.stdout: auth debug = true 2024-09-16T00:06:33.558 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.558 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on old message = true 2024-09-16T00:06:33.558 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on bug = true 2024-09-16T00:06:33.559 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.559 INFO:teuthology.orchestra.run.smithi164.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T00:06:33.559 INFO:teuthology.orchestra.run.smithi164.stdout: mon pg warn max object skew = 0 2024-09-16T00:06:33.559 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.559 INFO:teuthology.orchestra.run.smithi164.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T00:06:33.559 INFO:teuthology.orchestra.run.smithi164.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T00:06:33.559 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.559 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default size = 2 2024-09-16T00:06:33.559 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.560 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow primary affinity = true 2024-09-16T00:06:33.560 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow pg remap = true 2024-09-16T00:06:33.560 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on legacy crush tunables = false 2024-09-16T00:06:33.560 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on crush straw calc version zero = false 2024-09-16T00:06:33.560 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on no sortbitwise = false 2024-09-16T00:06:33.560 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on osd down out interval zero = false 2024-09-16T00:06:33.560 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on too few osds = false 2024-09-16T00:06:33.560 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T00:06:33.560 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T00:06:33.561 INFO:teuthology.orchestra.run.smithi164.stdout: mon_allow_pool_size_one = true 2024-09-16T00:06:33.561 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.561 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T00:06:33.561 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.561 INFO:teuthology.orchestra.run.smithi164.stdout: osd default data pool replay window = 5 2024-09-16T00:06:33.561 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.561 INFO:teuthology.orchestra.run.smithi164.stdout: mon allow pool delete = true 2024-09-16T00:06:33.561 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.561 INFO:teuthology.orchestra.run.smithi164.stdout: mon cluster log file level = debug 2024-09-16T00:06:33.561 INFO:teuthology.orchestra.run.smithi164.stdout: debug asserts on shutdown = true 2024-09-16T00:06:33.562 INFO:teuthology.orchestra.run.smithi164.stdout: mon health detail to clog = false 2024-09-16T00:06:33.562 INFO:teuthology.orchestra.run.smithi164.stdout: mon host = "172.21.15.139,172.21.15.164,[v2:172.21.15.164:3301,v1:172.21.15.164:6790]" 2024-09-16T00:06:33.562 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = 2d3366e5-6a59-4b59-93b2-615e72ac7015 2024-09-16T00:06:33.562 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.562 INFO:teuthology.orchestra.run.smithi164.stdout:[osd] 2024-09-16T00:06:33.562 INFO:teuthology.orchestra.run.smithi164.stdout: osd journal size = 100 2024-09-16T00:06:33.562 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.562 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub load threshold = 5.0 2024-09-16T00:06:33.562 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub max interval = 600 2024-09-16T00:06:33.562 INFO:teuthology.orchestra.run.smithi164.stdout: osd mclock profile = high_recovery_ops 2024-09-16T00:06:33.562 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.562 INFO:teuthology.orchestra.run.smithi164.stdout: osd recover clone overlap = true 2024-09-16T00:06:33.563 INFO:teuthology.orchestra.run.smithi164.stdout: osd recovery max chunk = 1048576 2024-09-16T00:06:33.563 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.563 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug shutdown = true 2024-09-16T00:06:33.563 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug op order = true 2024-09-16T00:06:33.563 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug verify stray on activate = true 2024-09-16T00:06:33.563 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug trim objects = true 2024-09-16T00:06:33.563 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.563 INFO:teuthology.orchestra.run.smithi164.stdout: osd open classes on start = true 2024-09-16T00:06:33.563 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug pg log writeout = true 2024-09-16T00:06:33.563 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.563 INFO:teuthology.orchestra.run.smithi164.stdout: osd deep scrub update digest min age = 30 2024-09-16T00:06:33.563 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.564 INFO:teuthology.orchestra.run.smithi164.stdout: osd map max advance = 10 2024-09-16T00:06:33.564 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.564 INFO:teuthology.orchestra.run.smithi164.stdout: journal zero on create = true 2024-09-16T00:06:33.564 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.564 INFO:teuthology.orchestra.run.smithi164.stdout: filestore ondisk finisher threads = 3 2024-09-16T00:06:33.564 INFO:teuthology.orchestra.run.smithi164.stdout: filestore apply finisher threads = 3 2024-09-16T00:06:33.564 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.564 INFO:teuthology.orchestra.run.smithi164.stdout: bdev debug aio = true 2024-09-16T00:06:33.564 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug misdirected ops = true 2024-09-16T00:06:33.564 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore block size = 96636764160 2024-09-16T00:06:33.564 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore compression mode = aggressive 2024-09-16T00:06:33.564 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore fsck on mount = True 2024-09-16T00:06:33.564 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluefs = 20 2024-09-16T00:06:33.565 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluestore = 20 2024-09-16T00:06:33.565 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-16T00:06:33.565 INFO:teuthology.orchestra.run.smithi164.stdout: debug osd = 20 2024-09-16T00:06:33.565 INFO:teuthology.orchestra.run.smithi164.stdout: debug rocksdb = 10 2024-09-16T00:06:33.565 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T00:06:33.565 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd full ratio = 0.9 2024-09-16T00:06:33.565 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd nearfull ratio = 0.8 2024-09-16T00:06:33.565 INFO:teuthology.orchestra.run.smithi164.stdout: osd blocked scrub grace period = 3600 2024-09-16T00:06:33.565 INFO:teuthology.orchestra.run.smithi164.stdout: osd failsafe full ratio = 0.95 2024-09-16T00:06:33.565 INFO:teuthology.orchestra.run.smithi164.stdout: osd max markdown count = 1000 2024-09-16T00:06:33.565 INFO:teuthology.orchestra.run.smithi164.stdout: osd objectstore = bluestore 2024-09-16T00:06:33.565 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2024-09-16T00:06:33.565 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.565 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr] 2024-09-16T00:06:33.566 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-16T00:06:33.566 INFO:teuthology.orchestra.run.smithi164.stdout: debug mgr = 20 2024-09-16T00:06:33.566 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-09-16T00:06:33.566 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-09-16T00:06:33.566 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-09-16T00:06:33.566 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-09-16T00:06:33.566 INFO:teuthology.orchestra.run.smithi164.stdout: mgr/telemetry/nag = false 2024-09-16T00:06:33.566 INFO:teuthology.orchestra.run.smithi164.stdout: debug client = 20 2024-09-16T00:06:33.566 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.566 INFO:teuthology.orchestra.run.smithi164.stdout:[mon] 2024-09-16T00:06:33.566 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-16T00:06:33.566 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2024-09-16T00:06:33.566 INFO:teuthology.orchestra.run.smithi164.stdout: debug paxos = 20 2024-09-16T00:06:33.566 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2024-09-16T00:06:33.567 INFO:teuthology.orchestra.run.smithi164.stdout: mon data avail warn = 5 2024-09-16T00:06:33.567 INFO:teuthology.orchestra.run.smithi164.stdout: mon mgr mkfs grace = 240 2024-09-16T00:06:33.567 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2024-09-16T00:06:33.567 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd reporter subtree level = osd 2024-09-16T00:06:33.567 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd prime pg temp = true 2024-09-16T00:06:33.567 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2024-09-16T00:06:33.567 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.567 INFO:teuthology.orchestra.run.smithi164.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T00:06:33.567 INFO:teuthology.orchestra.run.smithi164.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T00:06:33.567 INFO:teuthology.orchestra.run.smithi164.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T00:06:33.567 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.567 INFO:teuthology.orchestra.run.smithi164.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T00:06:33.567 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T00:06:33.567 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T00:06:33.568 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.568 INFO:teuthology.orchestra.run.smithi164.stdout: # 1m isn't quite enough 2024-09-16T00:06:33.568 INFO:teuthology.orchestra.run.smithi164.stdout: mon_down_mkfs_grace = 2m 2024-09-16T00:06:33.568 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.568 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_filestore_osds = false 2024-09-16T00:06:33.568 INFO:teuthology.orchestra.run.smithi164.stdout: mon down mkfs grace = 300 2024-09-16T00:06:33.568 INFO:teuthology.orchestra.run.smithi164.stdout: mon op complaint time = 120 2024-09-16T00:06:33.568 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on pool no app = False 2024-09-16T00:06:33.568 INFO:teuthology.orchestra.run.smithi164.stdout: 2024-09-16T00:06:33.568 INFO:teuthology.orchestra.run.smithi164.stdout:[client] 2024-09-16T00:06:33.568 INFO:teuthology.orchestra.run.smithi164.stdout: rgw cache enabled = true 2024-09-16T00:06:33.568 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable ops log = true 2024-09-16T00:06:33.568 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable usage log = true 2024-09-16T00:06:33.568 INFO:teuthology.orchestra.run.smithi164.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T00:06:33.568 INFO:teuthology.orchestra.run.smithi164.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T00:06:33.569 INFO:teuthology.orchestra.run.smithi164.stdout: client mount timeout = 600 2024-09-16T00:06:33.569 INFO:teuthology.orchestra.run.smithi164.stdout: client_shutdown_timeout = 120 2024-09-16T00:06:33.569 INFO:teuthology.orchestra.run.smithi164.stdout: debug client = 20 2024-09-16T00:06:33.569 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-16T00:06:33.569 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 900 2024-09-16T00:06:33.569 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 900 2024-09-16T00:06:33.569 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.a] 2024-09-16T00:06:33.569 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.b] 2024-09-16T00:06:33.569 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.c] 2024-09-16T00:06:33.569 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.a] 2024-09-16T00:06:33.569 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.c] 2024-09-16T00:06:33.569 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.e] 2024-09-16T00:06:33.569 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.b] 2024-09-16T00:06:33.569 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.d] 2024-09-16T00:06:33.569 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.f] 2024-09-16T00:06:33.570 INFO:teuthology.orchestra.run.smithi164.stdout:[mds] 2024-09-16T00:06:33.570 INFO:teuthology.orchestra.run.smithi164.stdout: debug mds = 20 2024-09-16T00:06:33.570 INFO:teuthology.orchestra.run.smithi164.stdout: debug mds balancer = 20 2024-09-16T00:06:33.570 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2024-09-16T00:06:33.570 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal fragment size max = 10000 2024-09-16T00:06:33.570 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal merge size = 5 2024-09-16T00:06:33.570 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal split bits = 3 2024-09-16T00:06:33.570 INFO:teuthology.orchestra.run.smithi164.stdout: mds bal split size = 100 2024-09-16T00:06:33.570 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug frag = True 2024-09-16T00:06:33.570 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug scatterstat = True 2024-09-16T00:06:33.570 INFO:teuthology.orchestra.run.smithi164.stdout: mds oft prefetch dirfrags = True 2024-09-16T00:06:33.570 INFO:teuthology.orchestra.run.smithi164.stdout: mds op complaint time = 180 2024-09-16T00:06:33.570 INFO:teuthology.orchestra.run.smithi164.stdout: mds verify scatter = True 2024-09-16T00:06:33.571 INFO:teuthology.orchestra.run.smithi164.stdout: mds_dir_prefetch = True 2024-09-16T00:06:33.571 INFO:teuthology.orchestra.run.smithi164.stdout: mds_sleep_rank_change = 5000000.0 2024-09-16T00:06:33.571 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2024-09-16T00:06:33.571 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 900 2024-09-16T00:06:33.571 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 900 2024-09-16T00:06:33.571 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T00:06:33.571 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T00:06:33.651 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T00:06:33.651 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:06:33.651 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T00:06:33.666 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:06:33.667 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T00:06:33.721 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T00:06:33.722 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:06:33.722 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T00:06:33.722 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T00:06:33.797 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:06:33.797 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T00:06:33.851 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T00:06:33.852 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:06:33.852 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T00:06:33.852 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T00:06:33.889 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:06:33.889 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T00:06:33.945 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T00:06:33.945 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T00:06:33.945 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T00:06:33.996 INFO:teuthology.orchestra.run.smithi139.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-16T00:06:33.999 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T00:06:34.069 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T00:06:34.071 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T00:06:34.071 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-16T00:06:34.122 INFO:teuthology.orchestra.run.smithi139.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-16T00:06:34.125 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-16T00:06:34.149 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-16T00:06:34.240 INFO:teuthology.orchestra.run.smithi139.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-16T00:06:34.242 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-16T00:06:34.266 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-09-16T00:06:34.357 INFO:teuthology.orchestra.run.smithi139.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-09-16T00:06:34.360 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-09-16T00:06:34.384 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-16T00:06:34.437 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-16T00:06:34.440 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-16T00:06:34.465 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-16T00:06:34.557 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-16T00:06:34.560 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-16T00:06:34.584 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-09-16T00:06:34.675 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-09-16T00:06:34.678 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-09-16T00:06:34.702 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T00:06:34.703 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T00:06:34.745 INFO:teuthology.orchestra.run.smithi139.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-16T00:06:34.757 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T00:06:34.758 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi139.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-16T00:06:34.758 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T00:06:34.822 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T00:06:34.822 INFO:tasks.ceph:role: osd.0 2024-09-16T00:06:34.822 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T00:06:34.822 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T00:06:34.894 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T00:06:34.894 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T00:06:34.894 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T00:06:34.894 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T00:06:34.894 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T00:06:34.894 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2024-09-16T00:06:34.894 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T00:06:34.894 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T00:06:34.895 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T00:06:34.895 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T00:06:35.396 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2024-09-16T00:06:35.398 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2024-09-16T00:06:35.398 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T00:06:35.522 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T00:06:35.554 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T00:06:35.619 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T00:06:35.619 INFO:tasks.ceph:role: osd.1 2024-09-16T00:06:35.619 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T00:06:35.619 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T00:06:35.692 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T00:06:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T00:06:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T00:06:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T00:06:35.693 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T00:06:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2024-09-16T00:06:35.693 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T00:06:35.693 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T00:06:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T00:06:35.693 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T00:06:36.142 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2024-09-16T00:06:36.145 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2024-09-16T00:06:36.145 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-16T00:06:36.239 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T00:06:36.267 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T00:06:36.332 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T00:06:36.333 INFO:tasks.ceph:role: osd.2 2024-09-16T00:06:36.333 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T00:06:36.333 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T00:06:36.405 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T00:06:36.406 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T00:06:36.406 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T00:06:36.406 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T00:06:36.406 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T00:06:36.406 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2024-09-16T00:06:36.406 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T00:06:36.406 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T00:06:36.406 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T00:06:36.406 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T00:06:36.900 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2024-09-16T00:06:36.902 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2024-09-16T00:06:36.903 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-16T00:06:37.043 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T00:06:37.074 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-16T00:06:37.140 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-16T00:06:37.140 INFO:tasks.ceph:role: osd.3 2024-09-16T00:06:37.140 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T00:06:37.140 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T00:06:37.213 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T00:06:37.213 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T00:06:37.213 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T00:06:37.213 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T00:06:37.213 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T00:06:37.213 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2024-09-16T00:06:37.213 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T00:06:37.213 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T00:06:37.213 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T00:06:37.213 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T00:06:37.734 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2024-09-16T00:06:37.736 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2024-09-16T00:06:37.736 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-16T00:06:37.885 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-16T00:06:37.917 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T00:06:38.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:06:38.017+0000 7f9c8bf45640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-16T00:06:38.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:06:38.018+0000 7f9c8bf45640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-16T00:06:38.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:06:38.018+0000 7f9c8bf45640 -1 bdev(0x564b8dc09800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-16T00:06:38.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:06:38.018+0000 7f9c8bf45640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-16T00:06:41.173 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T00:06:41.203 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T00:06:41.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:06:41.291+0000 7fedcaf41640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-16T00:06:41.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:06:41.292+0000 7fedcaf41640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-16T00:06:41.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:06:41.292+0000 7fedcaf41640 -1 bdev(0x56368ca3d800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-16T00:06:41.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:06:41.292+0000 7fedcaf41640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-16T00:06:44.949 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T00:06:44.979 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T00:06:45.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:06:45.068+0000 7f73bb6a5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-16T00:06:45.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:06:45.068+0000 7f73bb6a5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-16T00:06:45.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:06:45.068+0000 7f73bb6a5640 -1 bdev(0x5648fdff7800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-16T00:06:45.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:06:45.068+0000 7f73bb6a5640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-16T00:06:48.489 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T00:06:48.519 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T00:06:48.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:06:48.611+0000 7fdaf3816640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-16T00:06:48.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:06:48.612+0000 7fdaf3816640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-16T00:06:48.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:06:48.612+0000 7fdaf3816640 -1 bdev(0x55a87d95b800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-16T00:06:48.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:06:48.612+0000 7fdaf3816640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-16T00:06:51.526 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T00:06:51.556 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-16T00:06:51.592 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T00:06:51.593 INFO:tasks.ceph:role: osd.4 2024-09-16T00:06:51.595 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T00:06:51.596 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T00:06:51.668 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T00:06:51.668 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T00:06:51.668 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T00:06:51.668 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T00:06:51.669 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T00:06:51.669 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-09-16T00:06:51.669 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T00:06:51.669 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T00:06:51.669 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T00:06:51.669 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T00:06:52.143 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-09-16T00:06:52.145 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-09-16T00:06:52.145 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-16T00:06:52.243 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-16T00:06:52.273 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-16T00:06:52.341 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T00:06:52.341 INFO:tasks.ceph:role: osd.5 2024-09-16T00:06:52.341 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T00:06:52.341 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T00:06:52.415 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T00:06:52.415 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T00:06:52.416 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T00:06:52.416 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T00:06:52.416 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T00:06:52.416 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-09-16T00:06:52.416 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T00:06:52.416 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T00:06:52.416 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T00:06:52.416 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T00:06:52.871 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-09-16T00:06:52.873 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-09-16T00:06:52.873 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-16T00:06:52.947 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-16T00:06:53.017 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-16T00:06:53.082 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T00:06:53.082 INFO:tasks.ceph:role: osd.6 2024-09-16T00:06:53.082 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T00:06:53.082 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T00:06:53.154 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T00:06:53.155 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T00:06:53.155 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T00:06:53.155 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T00:06:53.155 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T00:06:53.155 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-09-16T00:06:53.155 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T00:06:53.155 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T00:06:53.155 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T00:06:53.155 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T00:06:53.638 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-09-16T00:06:53.645 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-09-16T00:06:53.646 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-16T00:06:53.744 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-16T00:06:53.815 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-16T00:06:53.880 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-16T00:06:53.880 INFO:tasks.ceph:role: osd.7 2024-09-16T00:06:53.880 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T00:06:53.880 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-16T00:06:53.953 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T00:06:53.953 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T00:06:53.954 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T00:06:53.954 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T00:06:53.954 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T00:06:53.954 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2024-09-16T00:06:53.954 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T00:06:53.954 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T00:06:53.954 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T00:06:53.954 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T00:06:54.474 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2024-09-16T00:06:54.476 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2024-09-16T00:06:54.476 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-16T00:06:54.618 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-16T00:06:54.650 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T00:06:54.750 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:06:54.750+0000 7ff541e87640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-16T00:06:54.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:06:54.754+0000 7ff541e87640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-16T00:06:54.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:06:54.754+0000 7ff541e87640 -1 bdev(0x55a30db2b800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-16T00:06:54.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:06:54.754+0000 7ff541e87640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-16T00:06:57.670 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T00:06:57.700 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T00:06:57.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:06:57.790+0000 7eff8addd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-16T00:06:57.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:06:57.790+0000 7eff8addd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-16T00:06:57.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:06:57.790+0000 7eff8addd640 -1 bdev(0x5618ae575800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-16T00:06:57.791 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:06:57.790+0000 7eff8addd640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-16T00:07:00.714 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T00:07:00.745 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T00:07:00.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:00.835+0000 7fdea580b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-16T00:07:00.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:00.835+0000 7fdea580b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-16T00:07:00.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:00.835+0000 7fdea580b640 -1 bdev(0x56340bd39800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-16T00:07:00.836 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:00.835+0000 7fdea580b640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-16T00:07:01.532 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:07:01.536 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:07:03.744 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-16T00:07:03.774 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T00:07:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:03.864+0000 7fa7c1619640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-16T00:07:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:03.864+0000 7fa7c1619640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-16T00:07:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:03.864+0000 7fa7c1619640 -1 bdev(0x5575859bf800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-16T00:07:03.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:03.864+0000 7fa7c1619640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-16T00:07:07.032 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-16T00:07:07.062 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T00:07:07.063 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:07:07.063 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-16T00:07:07.095 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:07:07.095 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-16T00:07:07.164 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:07:07.164 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-16T00:07:07.231 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:07:07.232 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-09-16T00:07:07.298 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:07:07.298 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T00:07:07.365 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:07:07.365 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T00:07:07.431 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:07:07.432 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T00:07:07.498 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:07:07.498 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-16T00:07:07.563 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:07:07.563 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T00:07:07.593 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:07:07.593 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-16T00:07:07.662 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:07:07.662 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-16T00:07:07.730 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:07:07.731 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-09-16T00:07:07.798 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:07:07.798 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-16T00:07:07.865 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:07:07.865 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-16T00:07:07.931 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:07:07.931 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-16T00:07:07.996 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:07:07.996 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-16T00:07:08.061 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:07:08.062 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-16T00:07:08.079 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T00:07:08.079 DEBUG:teuthology.orchestra.run.smithi139:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T00:07:08.122 DEBUG:teuthology.orchestra.run.smithi164:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T00:07:08.145 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr.y] 2024-09-16T00:07:08.145 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCJdudmdNJeOxAApx6vD2KpXdC6Cc9IicCciA== 2024-09-16T00:07:08.145 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.a] 2024-09-16T00:07:08.146 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCKdudmiAVMBxAAOXvCg/h3FE+36/Y8G1aBTA== 2024-09-16T00:07:08.146 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.c] 2024-09-16T00:07:08.146 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCKdudmPqdRDhAAOyx3AarXEadqnqN0mZYb1g== 2024-09-16T00:07:08.146 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.e] 2024-09-16T00:07:08.146 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCKdudmHKRVFRAAvirMaO477ktUrN2DKUWtWQ== 2024-09-16T00:07:08.147 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.0] 2024-09-16T00:07:08.147 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCOdudm620bARAAV+HJqj1kpY+9G9AtXfAieg== 2024-09-16T00:07:08.147 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.1] 2024-09-16T00:07:08.148 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCRdudmTAxwERAApP0zvhAGatBUIV7P0JhagQ== 2024-09-16T00:07:08.148 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.2] 2024-09-16T00:07:08.148 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCVdudmunYdBBAAFNQVpEDxQ54PKW64xdMHrg== 2024-09-16T00:07:08.149 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.3] 2024-09-16T00:07:08.149 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCYdudmcueCJBAAVfkzlp8ItnIgdnjeHvwdbg== 2024-09-16T00:07:08.149 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr.x] 2024-09-16T00:07:08.150 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCKdudmCCYVBBAAjFbHiZjB5iBgZnPTLOxLKw== 2024-09-16T00:07:08.150 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.b] 2024-09-16T00:07:08.150 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCKdudmQ04ZGhAArFJsH3FKY4F/TEMzXjfsAQ== 2024-09-16T00:07:08.151 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.d] 2024-09-16T00:07:08.151 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCKdudm+xo7IRAA6HiU3px/aFyYsxeP+2obxw== 2024-09-16T00:07:08.151 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.f] 2024-09-16T00:07:08.152 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCKdudm6W5IKBAA6Q6JkGb0e5/4J2dIk4PEGQ== 2024-09-16T00:07:08.152 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.4] 2024-09-16T00:07:08.152 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCedudmsXLALBAAxOkcS3Kul47W/dccdUpD5w== 2024-09-16T00:07:08.153 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.5] 2024-09-16T00:07:08.153 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQChdudm4D0hLxAA40ulpk7lTRYUAQ4RVke8HA== 2024-09-16T00:07:08.153 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.6] 2024-09-16T00:07:08.154 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCkdudmUNPQMRAAQjuIJ0q2CThv5f5d6sSg3w== 2024-09-16T00:07:08.154 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.7] 2024-09-16T00:07:08.154 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCndudmZ0yLMxAAyAELnuc327k6J6nLrMlxsw== 2024-09-16T00:07:08.155 INFO:teuthology.orchestra.run.smithi139.stdout:[client.0] 2024-09-16T00:07:08.155 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQCKdudmeUVuLBAAMfY2Zd9yWTAe51k7mtx6Qw== 2024-09-16T00:07:08.160 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.y] 2024-09-16T00:07:08.160 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCJdudmdNJeOxAApx6vD2KpXdC6Cc9IicCciA== 2024-09-16T00:07:08.161 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.a] 2024-09-16T00:07:08.161 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKdudmiAVMBxAAOXvCg/h3FE+36/Y8G1aBTA== 2024-09-16T00:07:08.162 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.c] 2024-09-16T00:07:08.162 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKdudmPqdRDhAAOyx3AarXEadqnqN0mZYb1g== 2024-09-16T00:07:08.162 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.e] 2024-09-16T00:07:08.162 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKdudmHKRVFRAAvirMaO477ktUrN2DKUWtWQ== 2024-09-16T00:07:08.163 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.0] 2024-09-16T00:07:08.163 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCOdudm620bARAAV+HJqj1kpY+9G9AtXfAieg== 2024-09-16T00:07:08.163 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.1] 2024-09-16T00:07:08.164 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCRdudmTAxwERAApP0zvhAGatBUIV7P0JhagQ== 2024-09-16T00:07:08.164 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.2] 2024-09-16T00:07:08.164 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCVdudmunYdBBAAFNQVpEDxQ54PKW64xdMHrg== 2024-09-16T00:07:08.165 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.3] 2024-09-16T00:07:08.165 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCYdudmcueCJBAAVfkzlp8ItnIgdnjeHvwdbg== 2024-09-16T00:07:08.165 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.x] 2024-09-16T00:07:08.165 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKdudmCCYVBBAAjFbHiZjB5iBgZnPTLOxLKw== 2024-09-16T00:07:08.165 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.b] 2024-09-16T00:07:08.166 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKdudmQ04ZGhAArFJsH3FKY4F/TEMzXjfsAQ== 2024-09-16T00:07:08.166 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.d] 2024-09-16T00:07:08.166 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKdudm+xo7IRAA6HiU3px/aFyYsxeP+2obxw== 2024-09-16T00:07:08.166 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.f] 2024-09-16T00:07:08.166 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKdudm6W5IKBAA6Q6JkGb0e5/4J2dIk4PEGQ== 2024-09-16T00:07:08.166 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.4] 2024-09-16T00:07:08.167 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCedudmsXLALBAAxOkcS3Kul47W/dccdUpD5w== 2024-09-16T00:07:08.167 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.5] 2024-09-16T00:07:08.167 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQChdudm4D0hLxAA40ulpk7lTRYUAQ4RVke8HA== 2024-09-16T00:07:08.167 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.6] 2024-09-16T00:07:08.167 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCkdudmUNPQMRAAQjuIJ0q2CThv5f5d6sSg3w== 2024-09-16T00:07:08.167 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.7] 2024-09-16T00:07:08.167 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCndudmZ0yLMxAAyAELnuc327k6J6nLrMlxsw== 2024-09-16T00:07:08.167 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2024-09-16T00:07:08.167 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQCKdudmeUVuLBAAMfY2Zd9yWTAe51k7mtx6Qw== 2024-09-16T00:07:08.168 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T00:07:08.203 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T00:07:08.255 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T00:07:08.257 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T00:07:08.310 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T00:07:08.313 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T00:07:08.366 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T00:07:08.369 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T00:07:08.423 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T00:07:08.425 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T00:07:08.476 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T00:07:08.478 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T00:07:08.529 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T00:07:08.532 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T00:07:08.583 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T00:07:08.585 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T00:07:08.636 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T00:07:08.638 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T00:07:08.688 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T00:07:08.691 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T00:07:08.746 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T00:07:08.749 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T00:07:08.802 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T00:07:08.805 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-16T00:07:08.859 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T00:07:08.861 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T00:07:08.911 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T00:07:08.914 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T00:07:08.965 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T00:07:08.968 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T00:07:09.019 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T00:07:09.022 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T00:07:09.074 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T00:07:09.076 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T00:07:09.130 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T00:07:09.130 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T00:07:09.155 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T00:07:09.251 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T00:07:09.276 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-16T00:07:09.301 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T00:07:09.397 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-16T00:07:09.422 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-16T00:07:09.488 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T00:07:09.584 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-16T00:07:09.609 DEBUG:teuthology.orchestra.run.smithi139:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T00:07:09.612 DEBUG:teuthology.orchestra.run.smithi164:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T00:07:09.665 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T00:07:09.666 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T00:07:09.666 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T00:07:09.669 INFO:tasks.ceph.mon.a:Started 2024-09-16T00:07:09.670 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-16T00:07:09.670 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-16T00:07:09.708 INFO:tasks.ceph.mon.b:Started 2024-09-16T00:07:09.708 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-16T00:07:09.708 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-16T00:07:09.710 INFO:tasks.ceph.mon.c:Started 2024-09-16T00:07:09.710 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T00:07:09.711 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-16T00:07:09.711 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-16T00:07:09.714 INFO:tasks.ceph.mgr.y:Started 2024-09-16T00:07:09.714 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T00:07:09.714 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T00:07:09.716 INFO:tasks.ceph.mgr.x:Started 2024-09-16T00:07:09.717 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T00:07:09.717 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T00:07:09.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.816+0000 7fdadc623640 1 Processor -- start 2024-09-16T00:07:09.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.816+0000 7fdadc623640 1 -- start start 2024-09-16T00:07:09.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdadc623640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad40c6b20 0x7fdad406eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:09.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdadc623640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdad406c680 0x7fdad41524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:09.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdadc623640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdad40c98a0 con 0x7fdad40c6f00 2024-09-16T00:07:09.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdadc623640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdad406f020 con 0x7fdad406c680 2024-09-16T00:07:09.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdadc623640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdad40c9f80 con 0x7fdad40c6b20 2024-09-16T00:07:09.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdadabc0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdad406c680 0x7fdad41524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:09.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdadabc0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdad406c680 0x7fdad41524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50758/0 (socket says 172.21.15.139:50758) 2024-09-16T00:07:09.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdadabc0640 1 -- 172.21.15.139:0/2080432217 learned_addr learned my addr 172.21.15.139:0/2080432217 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:09.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdadabc0640 1 -- 172.21.15.139:0/2080432217 >> v2:172.21.15.139:3300/0 conn(0x7fdad406c680 msgr2=0x7fdad41524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-16T00:07:09.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdadabc0640 1 -- 172.21.15.139:0/2080432217 >> v2:172.21.15.139:3300/0 conn(0x7fdad406c680 msgr2=0x7fdad41524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T00:07:09.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdadabc0640 1 --2- 172.21.15.139:0/2080432217 >> v2:172.21.15.139:3300/0 conn(0x7fdad406c680 0x7fdad41524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T00:07:09.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdadabc0640 1 --2- 172.21.15.139:0/2080432217 >> v2:172.21.15.139:3300/0 conn(0x7fdad406c680 0x7fdad41524a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:07:09.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdada3bf640 1 --2- 172.21.15.139:0/2080432217 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad40c6b20 0x7fdad406eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:09.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdada3bf640 1 -- 172.21.15.139:0/2080432217 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad40c6b20 msgr2=0x7fdad406eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-16T00:07:09.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdada3bf640 1 -- 172.21.15.139:0/2080432217 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad40c6b20 msgr2=0x7fdad406eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-16T00:07:09.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdada3bf640 1 --2- 172.21.15.139:0/2080432217 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad40c6b20 0x7fdad406eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T00:07:09.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.817+0000 7fdada3bf640 1 --2- 172.21.15.139:0/2080432217 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad40c6b20 0x7fdad406eae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:07:09.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad93bd640 1 -- 172.21.15.139:0/2080432217 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1869477158 0 0) 0x7fdad40c98a0 con 0x7fdad40c6f00 2024-09-16T00:07:09.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad93bd640 1 -- 172.21.15.139:0/2080432217 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdac0003520 con 0x7fdad40c6f00 2024-09-16T00:07:09.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad93bd640 1 -- 172.21.15.139:0/2080432217 <== mon.1 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3029524789 0 0) 0x7fdac0003520 con 0x7fdad40c6f00 2024-09-16T00:07:09.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad93bd640 1 -- 172.21.15.139:0/2080432217 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad40c6b20 msgr2=0x7fdad406eae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:07:09.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad93bd640 1 --2- 172.21.15.139:0/2080432217 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad40c6b20 0x7fdad406eae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:09.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad93bd640 1 -- 172.21.15.139:0/2080432217 >> v2:172.21.15.139:3300/0 conn(0x7fdad406c680 msgr2=0x7fdad41524a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:07:09.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad93bd640 1 --2- 172.21.15.139:0/2080432217 >> v2:172.21.15.139:3300/0 conn(0x7fdad406c680 0x7fdad41524a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:09.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad93bd640 1 -- 172.21.15.139:0/2080432217 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad40c9b80 con 0x7fdad40c6f00 2024-09-16T00:07:09.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad93bd640 1 -- 172.21.15.139:0/2080432217 <== mon.1 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7fdac4001670 con 0x7fdad40c6f00 2024-09-16T00:07:09.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad93bd640 1 -- 172.21.15.139:0/2080432217 >> v1:172.21.15.164:6789/0 conn(0x7fdad40c6f00 legacy=0x7fdad4158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:09.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad93bd640 1 --2- 172.21.15.139:0/2080432217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdac00039b0 0x7fdac0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:09.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad93bd640 1 --2- 172.21.15.139:0/2080432217 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad40c6b20 0x7fdac0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:09.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad93bd640 1 --2- 172.21.15.139:0/2080432217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0004d40 0x7fdac00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:09.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad93bd640 1 -- 172.21.15.139:0/2080432217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdad406f020 con 0x7fdac00039b0 2024-09-16T00:07:09.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad93bd640 1 -- 172.21.15.139:0/2080432217 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdad40c9f80 con 0x7fdac0004d40 2024-09-16T00:07:09.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad93bd640 1 -- 172.21.15.139:0/2080432217 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdac000ba70 con 0x7fdad40c6b20 2024-09-16T00:07:09.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdada3bf640 1 --2- 172.21.15.139:0/2080432217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdac00039b0 0x7fdac0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:09.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdadabc0640 1 --2- 172.21.15.139:0/2080432217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0004d40 0x7fdac00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:09.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdad9bbe640 1 --2- 172.21.15.139:0/2080432217 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad40c6b20 0x7fdac0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:09.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdada3bf640 1 -- 172.21.15.139:0/2080432217 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad40c6b20 msgr2=0x7fdac0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:09.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdada3bf640 1 --2- 172.21.15.139:0/2080432217 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad40c6b20 0x7fdac0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:09.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdada3bf640 1 -- 172.21.15.139:0/2080432217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0004d40 msgr2=0x7fdac00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:09.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdada3bf640 1 --2- 172.21.15.139:0/2080432217 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0004d40 0x7fdac00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:09.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.976+0000 7fdada3bf640 1 -- 172.21.15.139:0/2080432217 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac0003520 con 0x7fdac00039b0 2024-09-16T00:07:09.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.977+0000 7fdada3bf640 1 --2- 172.21.15.139:0/2080432217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdac00039b0 0x7fdac0003dd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdad000ba10 tx=0x7fdac0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d88545a4ca68ef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:09.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.977+0000 7fdad93bd640 1 -- 172.21.15.139:0/2080432217 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0030070 con 0x7fdac00039b0 2024-09-16T00:07:09.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.977+0000 7fdad93bd640 1 -- 172.21.15.139:0/2080432217 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad002ade0 con 0x7fdac00039b0 2024-09-16T00:07:09.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.977+0000 7fdad93bd640 1 -- 172.21.15.139:0/2080432217 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad000b5e0 con 0x7fdac00039b0 2024-09-16T00:07:09.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.978+0000 7fdadc623640 1 -- 172.21.15.139:0/2080432217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdac00039b0 msgr2=0x7fdac0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:09.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.978+0000 7fdadc623640 1 --2- 172.21.15.139:0/2080432217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdac00039b0 0x7fdac0003dd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdad000ba10 tx=0x7fdac0005830 comp rx=0 tx=0).stop 2024-09-16T00:07:09.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.978+0000 7fdadabc0640 1 -- 172.21.15.139:0/2080432217 reap_dead start 2024-09-16T00:07:09.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.978+0000 7fdadc623640 1 -- 172.21.15.139:0/2080432217 shutdown_connections 2024-09-16T00:07:09.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.978+0000 7fdadc623640 1 -- 172.21.15.139:0/2080432217 >> 172.21.15.139:0/2080432217 conn(0x7fdad4072b30 msgr2=0x7fdad4070510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:09.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.978+0000 7fdadc623640 1 -- 172.21.15.139:0/2080432217 shutdown_connections 2024-09-16T00:07:09.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.978+0000 7fdadc623640 1 -- 172.21.15.139:0/2080432217 wait complete. 2024-09-16T00:07:09.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.979+0000 7fdadc623640 1 Processor -- start 2024-09-16T00:07:09.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.980+0000 7fdadc623640 1 -- start start 2024-09-16T00:07:09.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.980+0000 7fdadc623640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac00039b0 0x7fdad4158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:09.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.980+0000 7fdadc623640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad41e42b0 0x7fdad41eea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:09.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.980+0000 7fdadc623640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdad41eefb0 0x7fdad4157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:09.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.980+0000 7fdadc623640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdad40ca9d0 con 0x7fdad41eefb0 2024-09-16T00:07:09.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.980+0000 7fdadc623640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdad40ca4b0 con 0x7fdac00039b0 2024-09-16T00:07:09.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.980+0000 7fdadc623640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdad41ef710 con 0x7fdad41e42b0 2024-09-16T00:07:09.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.980+0000 7fdadabc0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdad41eefb0 0x7fdad4157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:09.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.980+0000 7fdadabc0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdad41eefb0 0x7fdad4157480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50774/0 (socket says 172.21.15.139:50774) 2024-09-16T00:07:09.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.980+0000 7fdadabc0640 1 -- 172.21.15.139:0/2881869434 learned_addr learned my addr 172.21.15.139:0/2881869434 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:09.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.980+0000 7fdada3bf640 1 --2- 172.21.15.139:0/2881869434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac00039b0 0x7fdad4158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:09.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.980+0000 7fdad9bbe640 1 --2- 172.21.15.139:0/2881869434 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad41e42b0 0x7fdad41eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:09.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.981+0000 7fdadabc0640 1 -- 172.21.15.139:0/2881869434 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad41e42b0 msgr2=0x7fdad41eea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:09.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.981+0000 7fdadabc0640 1 --2- 172.21.15.139:0/2881869434 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad41e42b0 0x7fdad41eea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:09.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.981+0000 7fdadabc0640 1 -- 172.21.15.139:0/2881869434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac00039b0 msgr2=0x7fdad4158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:09.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.981+0000 7fdadabc0640 1 --2- 172.21.15.139:0/2881869434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac00039b0 0x7fdad4158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:09.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.981+0000 7fdadabc0640 1 -- 172.21.15.139:0/2881869434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad41ef430 con 0x7fdad41eefb0 2024-09-16T00:07:09.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.981+0000 7fdad9bbe640 1 --2- 172.21.15.139:0/2881869434 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad41e42b0 0x7fdad41eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:09.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.981+0000 7fdadabc0640 1 --2- 172.21.15.139:0/2881869434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdad41eefb0 0x7fdad4157480 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdad40c7390 tx=0x7fdacc004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:09.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.982+0000 7fdacaffd640 1 -- 172.21.15.139:0/2881869434 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc007cd0 con 0x7fdad41eefb0 2024-09-16T00:07:09.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.982+0000 7fdadc623640 1 -- 172.21.15.139:0/2881869434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad41df1d0 con 0x7fdad41eefb0 2024-09-16T00:07:09.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.982+0000 7fdacaffd640 1 -- 172.21.15.139:0/2881869434 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdacc019070 con 0x7fdad41eefb0 2024-09-16T00:07:09.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.982+0000 7fdacaffd640 1 -- 172.21.15.139:0/2881869434 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc014650 con 0x7fdad41eefb0 2024-09-16T00:07:09.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.982+0000 7fdadc623640 1 -- 172.21.15.139:0/2881869434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad41df790 con 0x7fdad41eefb0 2024-09-16T00:07:09.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.982+0000 7fdacaffd640 1 -- 172.21.15.139:0/2881869434 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdacc014b80 con 0x7fdad41eefb0 2024-09-16T00:07:09.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.982+0000 7fdacaffd640 1 -- 172.21.15.139:0/2881869434 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdacc01dcd0 con 0x7fdad41eefb0 2024-09-16T00:07:09.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.983+0000 7fdadc623640 1 -- 172.21.15.139:0/2881869434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad40c7ba0 con 0x7fdad41eefb0 2024-09-16T00:07:09.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:09.985+0000 7fdacaffd640 1 -- 172.21.15.139:0/2881869434 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fdacc01d410 con 0x7fdad41eefb0 2024-09-16T00:07:10.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:10.049+0000 7fdadc623640 1 -- 172.21.15.139:0/2881869434 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fdad41df960 con 0x7fdad41eefb0 2024-09-16T00:07:10.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:10.050+0000 7fdacaffd640 1 -- 172.21.15.139:0/2881869434 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fdacc01d5f0 con 0x7fdad41eefb0 2024-09-16T00:07:10.052 INFO:teuthology.orchestra.run.smithi139.stderr:adjusted tunables profile to default 2024-09-16T00:07:10.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:10.051+0000 7fdadc623640 1 -- 172.21.15.139:0/2881869434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdad41eefb0 msgr2=0x7fdad4157480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:10.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:10.051+0000 7fdadc623640 1 --2- 172.21.15.139:0/2881869434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdad41eefb0 0x7fdad4157480 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdad40c7390 tx=0x7fdacc004700 comp rx=0 tx=0).stop 2024-09-16T00:07:10.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:10.051+0000 7fdadc623640 1 -- 172.21.15.139:0/2881869434 shutdown_connections 2024-09-16T00:07:10.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:10.051+0000 7fdadc623640 1 --2- 172.21.15.139:0/2881869434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdad41eefb0 0x7fdad4157480 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:10.051+0000 7fdadc623640 1 --2- 172.21.15.139:0/2881869434 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdad41e42b0 0x7fdad41eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:10.051+0000 7fdadc623640 1 --2- 172.21.15.139:0/2881869434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac00039b0 0x7fdad4158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:10.051+0000 7fdadc623640 1 -- 172.21.15.139:0/2881869434 >> 172.21.15.139:0/2881869434 conn(0x7fdad4072b30 msgr2=0x7fdad406d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:10.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:10.051+0000 7fdadc623640 1 -- 172.21.15.139:0/2881869434 shutdown_connections 2024-09-16T00:07:10.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:10.051+0000 7fdadc623640 1 -- 172.21.15.139:0/2881869434 wait complete. 2024-09-16T00:07:10.064 INFO:tasks.ceph:check_enable_crimson: False 2024-09-16T00:07:10.064 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T00:07:10.064 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:07:10.064 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T00:07:10.088 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:10.087+0000 7f7123673040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:07:10.089 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:07:10.089 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T00:07:10.097 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:10.095+0000 7f399452b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:07:10.120 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:07:10.120 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T00:07:10.128 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:10.128+0000 7f7123673040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:07:10.140 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:10.138+0000 7f399452b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:07:10.164 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:07:10.164 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-16T00:07:10.229 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:07:10.230 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-16T00:07:10.255 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:07:10.255 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-16T00:07:10.271 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:10.270+0000 7f7123673040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:07:10.289 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:10.288+0000 7f399452b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:07:10.295 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:07:10.295 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-16T00:07:10.324 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:10.324+0000 7f7123673040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:07:10.339 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:10.338+0000 7f399452b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:07:10.350 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2024-09-16T00:07:10.350 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-16T00:07:10.416 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 83699a05-4937-43c3-aefc-b669bed9bb6f 0 2024-09-16T00:07:10.491 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:10.491+0000 7f7123673040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:07:10.505 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:10.503+0000 7f399452b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:07:10.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.554+0000 7f95cadf1640 1 Processor -- start 2024-09-16T00:07:10.555 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.554+0000 7f95cadf1640 1 -- start start 2024-09-16T00:07:10.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.555+0000 7f95cadf1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f95c4151850 0x7f95c40c6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:10.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.555+0000 7f95cadf1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f95c4151c30 0x7f95c415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:10.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.555+0000 7f95cadf1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c40c7390 0x7f95c40c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:10.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.555+0000 7f95cadf1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f95c40c88a0 con 0x7f95c4151c30 2024-09-16T00:07:10.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.555+0000 7f95cadf1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f95c40c8d30 con 0x7f95c4151850 2024-09-16T00:07:10.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.555+0000 7f95cadf1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f95c40c6a90 con 0x7f95c40c7390 2024-09-16T00:07:10.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.555+0000 7f95c8b8d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f95c4151850 0x7f95c40c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:10.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.555+0000 7f95c938e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c40c7390 0x7f95c40c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:10.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.555+0000 7f95c8b8d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f95c4151850 0x7f95c40c6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35130/0 (socket says 172.21.15.164:35130) 2024-09-16T00:07:10.557 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.555+0000 7f95c8b8d640 1 -- 172.21.15.164:0/3256169991 learned_addr learned my addr 172.21.15.164:0/3256169991 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T00:07:10.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.555+0000 7f95c938e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c40c7390 0x7f95c40c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:39594/0 (socket says 172.21.15.164:39594) 2024-09-16T00:07:10.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.555+0000 7f95c3fff640 1 --2- 172.21.15.164:0/3256169991 >> v2:172.21.15.139:3300/0 conn(0x7f95c4151c30 0x7f95c415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:10.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.556+0000 7f95c8b8d640 1 -- 172.21.15.164:0/3256169991 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c40c7390 msgr2=0x7f95c40c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:10.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.556+0000 7f95c8b8d640 1 --2- 172.21.15.164:0/3256169991 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c40c7390 0x7f95c40c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.556+0000 7f95c8b8d640 1 -- 172.21.15.164:0/3256169991 >> v2:172.21.15.139:3300/0 conn(0x7f95c4151c30 msgr2=0x7f95c415ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:10.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.556+0000 7f95c8b8d640 1 --2- 172.21.15.164:0/3256169991 >> v2:172.21.15.139:3300/0 conn(0x7f95c4151c30 0x7f95c415ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.556+0000 7f95c8b8d640 1 -- 172.21.15.164:0/3256169991 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c40c67b0 con 0x7f95c4151850 2024-09-16T00:07:10.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.556+0000 7f95c938e640 1 --2- 172.21.15.164:0/3256169991 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c40c7390 0x7f95c40c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:10.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.556+0000 7f95c3fff640 1 --2- 172.21.15.164:0/3256169991 >> v2:172.21.15.139:3300/0 conn(0x7f95c4151c30 0x7f95c415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:10.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.556+0000 7f95c8b8d640 1 --2- 172.21.15.164:0/3256169991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95c4151850 0x7f95c40c6100 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95c4075c60 tx=0x7f95b802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3a02f1d046cce89 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:10.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.556+0000 7f95c37fe640 1 -- 172.21.15.164:0/3256169991 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8037070 con 0x7f95c4151850 2024-09-16T00:07:10.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.556+0000 7f95c37fe640 1 -- 172.21.15.164:0/3256169991 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b8032440 con 0x7f95c4151850 2024-09-16T00:07:10.558 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.556+0000 7f95c37fe640 1 -- 172.21.15.164:0/3256169991 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8032810 con 0x7f95c4151850 2024-09-16T00:07:10.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.557+0000 7f95cadf1640 1 -- 172.21.15.164:0/3256169991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95c4151850 msgr2=0x7f95c40c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:10.561 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.557+0000 7f95cadf1640 1 --2- 172.21.15.164:0/3256169991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95c4151850 0x7f95c40c6100 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95c4075c60 tx=0x7f95b802a270 comp rx=0 tx=0).stop 2024-09-16T00:07:10.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.557+0000 7f95cadf1640 1 -- 172.21.15.164:0/3256169991 shutdown_connections 2024-09-16T00:07:10.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.557+0000 7f95cadf1640 1 --2- 172.21.15.164:0/3256169991 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c40c7390 0x7f95c40c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.557+0000 7f95cadf1640 1 --2- 172.21.15.164:0/3256169991 >> v2:172.21.15.139:3300/0 conn(0x7f95c4151c30 0x7f95c415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.557+0000 7f95cadf1640 1 --2- 172.21.15.164:0/3256169991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95c4151850 0x7f95c40c6100 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.557+0000 7f95cadf1640 1 -- 172.21.15.164:0/3256169991 >> 172.21.15.164:0/3256169991 conn(0x7f95c406cc30 msgr2=0x7f95c406d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:10.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.557+0000 7f95cadf1640 1 -- 172.21.15.164:0/3256169991 shutdown_connections 2024-09-16T00:07:10.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.557+0000 7f95cadf1640 1 -- 172.21.15.164:0/3256169991 wait complete. 2024-09-16T00:07:10.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.558+0000 7f95cadf1640 1 Processor -- start 2024-09-16T00:07:10.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.558+0000 7f95cadf1640 1 -- start start 2024-09-16T00:07:10.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.559+0000 7f95cadf1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c40c7390 0x7f95c41544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:10.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.559+0000 7f95cadf1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95c4151850 0x7f95c4153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:10.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.559+0000 7f95cadf1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95c4151c30 0x7f95c41eee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:10.562 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.559+0000 7f95cadf1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f95c40c8530 con 0x7f95c4151c30 2024-09-16T00:07:10.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.559+0000 7f95cadf1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f95c40c8250 con 0x7f95c4151850 2024-09-16T00:07:10.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.559+0000 7f95cadf1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f95c415b150 con 0x7f95c40c7390 2024-09-16T00:07:10.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.559+0000 7f95c8b8d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c40c7390 0x7f95c41544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:10.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.559+0000 7f95c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95c4151850 0x7f95c4153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:10.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.559+0000 7f95c3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95c4151850 0x7f95c4153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35132/0 (socket says 172.21.15.164:35132) 2024-09-16T00:07:10.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.559+0000 7f95c3fff640 1 -- 172.21.15.164:0/1942479442 learned_addr learned my addr 172.21.15.164:0/1942479442 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T00:07:10.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.559+0000 7f95c938e640 1 --2- 172.21.15.164:0/1942479442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95c4151c30 0x7f95c41eee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:10.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.560+0000 7f95c8b8d640 1 -- 172.21.15.164:0/1942479442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95c4151850 msgr2=0x7f95c4153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:10.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.560+0000 7f95c8b8d640 1 --2- 172.21.15.164:0/1942479442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95c4151850 0x7f95c4153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.560+0000 7f95c8b8d640 1 -- 172.21.15.164:0/1942479442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95c4151c30 msgr2=0x7f95c41eee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:10.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.560+0000 7f95c8b8d640 1 --2- 172.21.15.164:0/1942479442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95c4151c30 0x7f95c41eee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.560+0000 7f95c8b8d640 1 -- 172.21.15.164:0/1942479442 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c41ef360 con 0x7f95c40c7390 2024-09-16T00:07:10.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.560+0000 7f95c3fff640 1 --2- 172.21.15.164:0/1942479442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95c4151850 0x7f95c4153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:10.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.560+0000 7f95c8b8d640 1 --2- 172.21.15.164:0/1942479442 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c40c7390 0x7f95c41544a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95b8031000 tx=0x7f95b80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:10.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.560+0000 7f95c17fa640 1 -- 172.21.15.164:0/1942479442 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8041030 con 0x7f95c40c7390 2024-09-16T00:07:10.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.560+0000 7f95c17fa640 1 -- 172.21.15.164:0/1942479442 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95b8030440 con 0x7f95c40c7390 2024-09-16T00:07:10.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.560+0000 7f95cadf1640 1 -- 172.21.15.164:0/1942479442 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c414fdd0 con 0x7f95c40c7390 2024-09-16T00:07:10.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.560+0000 7f95c17fa640 1 -- 172.21.15.164:0/1942479442 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8037040 con 0x7f95c40c7390 2024-09-16T00:07:10.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.560+0000 7f95cadf1640 1 -- 172.21.15.164:0/1942479442 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c4154e10 con 0x7f95c40c7390 2024-09-16T00:07:10.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.560+0000 7f95c17fa640 1 -- 172.21.15.164:0/1942479442 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f95b8044070 con 0x7f95c40c7390 2024-09-16T00:07:10.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.560+0000 7f95c17fa640 1 -- 172.21.15.164:0/1942479442 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f95b803d8d0 con 0x7f95c40c7390 2024-09-16T00:07:10.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.561+0000 7f95cadf1640 1 -- 172.21.15.164:0/1942479442 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c40c45a0 con 0x7f95c40c7390 2024-09-16T00:07:10.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.563+0000 7f95c17fa640 1 -- 172.21.15.164:0/1942479442 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f95c41ef360 con 0x7f95c40c7390 2024-09-16T00:07:10.607 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:10.606+0000 7f7123673040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:07:10.613 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:10.612+0000 7f399452b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:07:10.631 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.631+0000 7f95cadf1640 1 -- 172.21.15.164:0/1942479442 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd new", "uuid": "83699a05-4937-43c3-aefc-b669bed9bb6f", "id": 0} v 0) -- 0x7f95c414ec20 con 0x7f95c40c7390 2024-09-16T00:07:10.634 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.634+0000 7f95c17fa640 1 -- 172.21.15.164:0/1942479442 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "83699a05-4937-43c3-aefc-b669bed9bb6f", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f95b803db20 con 0x7f95c40c7390 2024-09-16T00:07:10.634 INFO:teuthology.orchestra.run.smithi164.stdout:0 2024-09-16T00:07:10.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.635+0000 7f95cadf1640 1 -- 172.21.15.164:0/1942479442 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c40c7390 msgr2=0x7f95c41544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:10.635 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.635+0000 7f95cadf1640 1 --2- 172.21.15.164:0/1942479442 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c40c7390 0x7f95c41544a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f95b8031000 tx=0x7f95b80046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.635+0000 7f95cadf1640 1 -- 172.21.15.164:0/1942479442 shutdown_connections 2024-09-16T00:07:10.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.635+0000 7f95cadf1640 1 --2- 172.21.15.164:0/1942479442 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95c4151c30 0x7f95c41eee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.635+0000 7f95cadf1640 1 --2- 172.21.15.164:0/1942479442 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f95c4151850 0x7f95c4153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.635+0000 7f95cadf1640 1 --2- 172.21.15.164:0/1942479442 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f95c40c7390 0x7f95c41544a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.635+0000 7f95cadf1640 1 -- 172.21.15.164:0/1942479442 >> 172.21.15.164:0/1942479442 conn(0x7f95c406cc30 msgr2=0x7f95c4070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:10.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.635+0000 7f95cadf1640 1 -- 172.21.15.164:0/1942479442 shutdown_connections 2024-09-16T00:07:10.636 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.635+0000 7f95cadf1640 1 -- 172.21.15.164:0/1942479442 wait complete. 2024-09-16T00:07:10.646 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 9c04991e-9695-4bbb-8011-bf613bf404b3 1 2024-09-16T00:07:10.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.788+0000 7f6e8d2e4640 1 Processor -- start 2024-09-16T00:07:10.792 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.788+0000 7f6e8d2e4640 1 -- start start 2024-09-16T00:07:10.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.789+0000 7f6e8d2e4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6e88068060 0x7f6e8815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:10.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.789+0000 7f6e8d2e4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e880cc4b0 con 0x7f6e880ccc90 2024-09-16T00:07:10.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.789+0000 7f6e8d2e4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e880cc790 con 0x7f6e88154c20 2024-09-16T00:07:10.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.789+0000 7f6e8d2e4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f6e880c5830 con 0x7f6e88068060 2024-09-16T00:07:10.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.789+0000 7f6e7ffff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6e88068060 0x7f6e8815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:10.793 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.789+0000 7f6e86ffd640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f6e88154c20 0x7f6e880c5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58648/0 (socket says 172.21.15.164:58648) 2024-09-16T00:07:10.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.789+0000 7f6e7ffff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6e88068060 0x7f6e8815b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35140/0 (socket says 172.21.15.164:35140) 2024-09-16T00:07:10.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.789+0000 7f6e86ffd640 1 -- 172.21.15.164:0/1794899887 learned_addr learned my addr 172.21.15.164:0/1794899887 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-09-16T00:07:10.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.789+0000 7f6e7ffff640 1 -- 172.21.15.164:0/1794899887 >> v1:172.21.15.164:6789/0 conn(0x7f6e88154c20 legacy=0x7f6e880c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:10.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.789+0000 7f6e7ffff640 1 -- 172.21.15.164:0/1794899887 >> v1:172.21.15.139:6789/0 conn(0x7f6e880ccc90 legacy=0x7f6e88165180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:10.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.789+0000 7f6e7ffff640 1 -- 172.21.15.164:0/1794899887 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8815ddf0 con 0x7f6e88068060 2024-09-16T00:07:10.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.790+0000 7f6e7ffff640 1 --2- 172.21.15.164:0/1794899887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e88068060 0x7f6e8815b280 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009790 tx=0x7f6e7002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f35e21b08476da4a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:10.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.790+0000 7f6e867fc640 1 -- 172.21.15.164:0/1794899887 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70037070 con 0x7f6e88068060 2024-09-16T00:07:10.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.790+0000 7f6e867fc640 1 -- 172.21.15.164:0/1794899887 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e70032440 con 0x7f6e88068060 2024-09-16T00:07:10.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.790+0000 7f6e867fc640 1 -- 172.21.15.164:0/1794899887 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70032810 con 0x7f6e88068060 2024-09-16T00:07:10.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.791+0000 7f6e8d2e4640 1 -- 172.21.15.164:0/1794899887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e88068060 msgr2=0x7f6e8815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:10.794 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.791+0000 7f6e8d2e4640 1 --2- 172.21.15.164:0/1794899887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e88068060 0x7f6e8815b280 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009790 tx=0x7f6e7002a210 comp rx=0 tx=0).stop 2024-09-16T00:07:10.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.791+0000 7f6e8d2e4640 1 -- 172.21.15.164:0/1794899887 shutdown_connections 2024-09-16T00:07:10.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.791+0000 7f6e8d2e4640 1 --2- 172.21.15.164:0/1794899887 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e88068060 0x7f6e8815b280 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.791+0000 7f6e8d2e4640 1 -- 172.21.15.164:0/1794899887 >> 172.21.15.164:0/1794899887 conn(0x7f6e8806ca40 msgr2=0x7f6e88067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:10.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.791+0000 7f6e8d2e4640 1 -- 172.21.15.164:0/1794899887 shutdown_connections 2024-09-16T00:07:10.795 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.791+0000 7f6e8d2e4640 1 -- 172.21.15.164:0/1794899887 wait complete. 2024-09-16T00:07:10.797 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.792+0000 7f6e8d2e4640 1 Processor -- start 2024-09-16T00:07:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.792+0000 7f6e8d2e4640 1 -- start start 2024-09-16T00:07:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e8d2e4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e88068060 0x7f6e88156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e8d2e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e880ccc90 0x7f6e8815d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e8d2e4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e88154c20 0x7f6e8815d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e8d2e4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6e880cc260 con 0x7f6e880ccc90 2024-09-16T00:07:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e8d2e4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8815e3b0 con 0x7f6e88068060 2024-09-16T00:07:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e8d2e4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6e8815e690 con 0x7f6e88154c20 2024-09-16T00:07:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e86ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e88068060 0x7f6e88156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e877fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e88154c20 0x7f6e8815d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e7ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e880ccc90 0x7f6e8815d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e877fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e88154c20 0x7f6e8815d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:39602/0 (socket says 172.21.15.164:39602) 2024-09-16T00:07:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e86ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e88068060 0x7f6e88156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35148/0 (socket says 172.21.15.164:35148) 2024-09-16T00:07:10.798 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e877fe640 1 -- 172.21.15.164:0/1816748019 learned_addr learned my addr 172.21.15.164:0/1816748019 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T00:07:10.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e7ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e880ccc90 0x7f6e8815d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.164:58584/0 (socket says 172.21.15.164:58584) 2024-09-16T00:07:10.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e7ffff640 1 -- 172.21.15.164:0/1816748019 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e88154c20 msgr2=0x7f6e8815d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:10.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e7ffff640 1 --2- 172.21.15.164:0/1816748019 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e88154c20 0x7f6e8815d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e7ffff640 1 -- 172.21.15.164:0/1816748019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e88068060 msgr2=0x7f6e88156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:10.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e7ffff640 1 --2- 172.21.15.164:0/1816748019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e88068060 0x7f6e88156e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e7ffff640 1 -- 172.21.15.164:0/1816748019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8815cc10 con 0x7f6e880ccc90 2024-09-16T00:07:10.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e86ffd640 1 --2- 172.21.15.164:0/1816748019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e88068060 0x7f6e88156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:10.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.793+0000 7f6e877fe640 1 --2- 172.21.15.164:0/1816748019 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e88154c20 0x7f6e8815d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:10.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.794+0000 7f6e7ffff640 1 --2- 172.21.15.164:0/1816748019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e880ccc90 0x7f6e8815d2d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6e88074130 tx=0x7f6e700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:10.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.794+0000 7f6e7f7fe640 1 -- 172.21.15.164:0/1816748019 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70032e60 con 0x7f6e880ccc90 2024-09-16T00:07:10.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.794+0000 7f6e7f7fe640 1 -- 172.21.15.164:0/1816748019 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e70005c50 con 0x7f6e880ccc90 2024-09-16T00:07:10.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.794+0000 7f6e8d2e4640 1 -- 172.21.15.164:0/1816748019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e88157be0 con 0x7f6e880ccc90 2024-09-16T00:07:10.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.794+0000 7f6e7f7fe640 1 -- 172.21.15.164:0/1816748019 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70037040 con 0x7f6e880ccc90 2024-09-16T00:07:10.799 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.794+0000 7f6e8d2e4640 1 -- 172.21.15.164:0/1816748019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e881f47e0 con 0x7f6e880ccc90 2024-09-16T00:07:10.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.794+0000 7f6e7f7fe640 1 -- 172.21.15.164:0/1816748019 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6e700306f0 con 0x7f6e880ccc90 2024-09-16T00:07:10.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.794+0000 7f6e7f7fe640 1 -- 172.21.15.164:0/1816748019 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f6e7003b430 con 0x7f6e880ccc90 2024-09-16T00:07:10.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.795+0000 7f6e8d2e4640 1 -- 172.21.15.164:0/1816748019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e880686b0 con 0x7f6e880ccc90 2024-09-16T00:07:10.800 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.797+0000 7f6e7f7fe640 1 -- 172.21.15.164:0/1816748019 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6e70030b20 con 0x7f6e880ccc90 2024-09-16T00:07:10.865 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.864+0000 7f6e8d2e4640 1 -- 172.21.15.164:0/1816748019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9c04991e-9695-4bbb-8011-bf613bf404b3", "id": 1} v 0) -- 0x7f6e880ca640 con 0x7f6e880ccc90 2024-09-16T00:07:10.866 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.866+0000 7f6e7f7fe640 1 -- 172.21.15.164:0/1816748019 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9c04991e-9695-4bbb-8011-bf613bf404b3", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f6e70030d00 con 0x7f6e880ccc90 2024-09-16T00:07:10.866 INFO:teuthology.orchestra.run.smithi164.stdout:1 2024-09-16T00:07:10.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.867+0000 7f6e8d2e4640 1 -- 172.21.15.164:0/1816748019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e880ccc90 msgr2=0x7f6e8815d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:10.867 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.867+0000 7f6e8d2e4640 1 --2- 172.21.15.164:0/1816748019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e880ccc90 0x7f6e8815d2d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6e88074130 tx=0x7f6e700046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.867+0000 7f6e8d2e4640 1 -- 172.21.15.164:0/1816748019 shutdown_connections 2024-09-16T00:07:10.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.867+0000 7f6e8d2e4640 1 --2- 172.21.15.164:0/1816748019 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6e88154c20 0x7f6e8815d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.867+0000 7f6e8d2e4640 1 --2- 172.21.15.164:0/1816748019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e880ccc90 0x7f6e8815d2d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.867+0000 7f6e8d2e4640 1 --2- 172.21.15.164:0/1816748019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e88068060 0x7f6e88156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.867+0000 7f6e8d2e4640 1 -- 172.21.15.164:0/1816748019 >> 172.21.15.164:0/1816748019 conn(0x7f6e8806ca40 msgr2=0x7f6e880c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:10.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.867+0000 7f6e8d2e4640 1 -- 172.21.15.164:0/1816748019 shutdown_connections 2024-09-16T00:07:10.868 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.867+0000 7f6e8d2e4640 1 -- 172.21.15.164:0/1816748019 wait complete. 2024-09-16T00:07:10.878 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 8048fc64-72b3-4866-ac62-6a8b2bf6f5c8 2 2024-09-16T00:07:10.924 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:10.922+0000 7f399452b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:07:10.938 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:10.938+0000 7f7123673040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:07:10.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.976+0000 7f8a66fa6640 1 Processor -- start 2024-09-16T00:07:10.977 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.976+0000 7f8a66fa6640 1 -- start start 2024-09-16T00:07:10.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.978+0000 7f8a66fa6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a60151b30 0x7f8a600c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:10.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.978+0000 7f8a66fa6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8a600cd680 0x7f8a60157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:10.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.978+0000 7f8a66fa6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a6006f990 con 0x7f8a600cd2a0 2024-09-16T00:07:10.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.978+0000 7f8a66fa6640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f8a6006ee10 con 0x7f8a600cd680 2024-09-16T00:07:10.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.978+0000 7f8a66fa6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8a6006f0f0 con 0x7f8a60151b30 2024-09-16T00:07:10.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.978+0000 7f8a65543640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8a600cd680 0x7f8a60157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:10.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.978+0000 7f8a5ffff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a60151b30 0x7f8a600c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:10.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.978+0000 7f8a64d42640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8a600cd2a0 0x7f8a60151420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58658/0 (socket says 172.21.15.164:58658) 2024-09-16T00:07:10.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.978+0000 7f8a64d42640 1 -- 172.21.15.164:0/2099437939 learned_addr learned my addr 172.21.15.164:0/2099437939 (peer_addr_for_me v1:172.21.15.164:0/0) 2024-09-16T00:07:10.979 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.978+0000 7f8a65543640 1 -- 172.21.15.164:0/2099437939 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a60151b30 msgr2=0x7f8a600c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:10.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.978+0000 7f8a65543640 1 --2- 172.21.15.164:0/2099437939 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a60151b30 0x7f8a600c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.978+0000 7f8a65543640 1 -- 172.21.15.164:0/2099437939 >> v1:172.21.15.164:6789/0 conn(0x7f8a600cd2a0 legacy=0x7f8a60151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:10.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.978+0000 7f8a65543640 1 -- 172.21.15.164:0/2099437939 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6006f3d0 con 0x7f8a600cd680 2024-09-16T00:07:10.980 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.978+0000 7f8a5ffff640 1 --2- 172.21.15.164:0/2099437939 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a60151b30 0x7f8a600c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:10.980 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:10.977+0000 7f399452b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:07:10.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.979+0000 7f8a65543640 1 --2- 172.21.15.164:0/2099437939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a600cd680 0x7f8a60157bd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009d90 tx=0x7f8a5802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f537ca586ebbe3e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:10.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.979+0000 7f8a5f7fe640 1 -- 172.21.15.164:0/2099437939 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58007740 con 0x7f8a600cd680 2024-09-16T00:07:10.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.979+0000 7f8a5f7fe640 1 -- 172.21.15.164:0/2099437939 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a580078e0 con 0x7f8a600cd680 2024-09-16T00:07:10.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.979+0000 7f8a5f7fe640 1 -- 172.21.15.164:0/2099437939 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58007cb0 con 0x7f8a600cd680 2024-09-16T00:07:10.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.980+0000 7f8a66fa6640 1 -- 172.21.15.164:0/2099437939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a600cd680 msgr2=0x7f8a60157bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:10.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.980+0000 7f8a66fa6640 1 --2- 172.21.15.164:0/2099437939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a600cd680 0x7f8a60157bd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009d90 tx=0x7f8a5802a6f0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.980+0000 7f8a66fa6640 1 -- 172.21.15.164:0/2099437939 shutdown_connections 2024-09-16T00:07:10.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.980+0000 7f8a66fa6640 1 --2- 172.21.15.164:0/2099437939 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a600cd680 0x7f8a60157bd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.980+0000 7f8a66fa6640 1 --2- 172.21.15.164:0/2099437939 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a60151b30 0x7f8a600c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.980+0000 7f8a66fa6640 1 -- 172.21.15.164:0/2099437939 >> 172.21.15.164:0/2099437939 conn(0x7f8a6006cc30 msgr2=0x7f8a6006d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:10.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.980+0000 7f8a66fa6640 1 -- 172.21.15.164:0/2099437939 shutdown_connections 2024-09-16T00:07:10.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.980+0000 7f8a66fa6640 1 -- 172.21.15.164:0/2099437939 wait complete. 2024-09-16T00:07:10.982 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.981+0000 7f8a66fa6640 1 Processor -- start 2024-09-16T00:07:10.983 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.981+0000 7f8a66fa6640 1 -- start start 2024-09-16T00:07:10.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.982+0000 7f8a66fa6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a600cd2a0 0x7f8a6013fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:10.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.982+0000 7f8a66fa6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a600cd680 0x7f8a601f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:10.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.982+0000 7f8a66fa6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a60151b30 0x7f8a6013b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:10.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.982+0000 7f8a66fa6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8a600c72b0 con 0x7f8a600cd680 2024-09-16T00:07:10.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.982+0000 7f8a66fa6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601576b0 con 0x7f8a60151b30 2024-09-16T00:07:10.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.982+0000 7f8a66fa6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8a600c7430 con 0x7f8a600cd2a0 2024-09-16T00:07:10.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.982+0000 7f8a65543640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a60151b30 0x7f8a6013b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:10.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.982+0000 7f8a64d42640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a600cd2a0 0x7f8a6013fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:10.985 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.982+0000 7f8a65543640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a60151b30 0x7f8a6013b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35168/0 (socket says 172.21.15.164:35168) 2024-09-16T00:07:10.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.982+0000 7f8a65543640 1 -- 172.21.15.164:0/128324068 learned_addr learned my addr 172.21.15.164:0/128324068 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T00:07:10.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.982+0000 7f8a5ffff640 1 --2- 172.21.15.164:0/128324068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a600cd680 0x7f8a601f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:10.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.982+0000 7f8a64d42640 1 -- 172.21.15.164:0/128324068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a60151b30 msgr2=0x7f8a6013b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:10.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.982+0000 7f8a64d42640 1 --2- 172.21.15.164:0/128324068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a60151b30 0x7f8a6013b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.983+0000 7f8a64d42640 1 -- 172.21.15.164:0/128324068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a600cd680 msgr2=0x7f8a601f3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:10.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.983+0000 7f8a64d42640 1 --2- 172.21.15.164:0/128324068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a600cd680 0x7f8a601f3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:10.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.983+0000 7f8a64d42640 1 -- 172.21.15.164:0/128324068 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a60156f60 con 0x7f8a600cd2a0 2024-09-16T00:07:10.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.983+0000 7f8a65543640 1 --2- 172.21.15.164:0/128324068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a60151b30 0x7f8a6013b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:10.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.983+0000 7f8a64d42640 1 --2- 172.21.15.164:0/128324068 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a600cd2a0 0x7f8a6013fda0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8a54000f60 tx=0x7f8a5400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:10.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.983+0000 7f8a5d7fa640 1 -- 172.21.15.164:0/128324068 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54017070 con 0x7f8a600cd2a0 2024-09-16T00:07:10.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.983+0000 7f8a5d7fa640 1 -- 172.21.15.164:0/128324068 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a54005ce0 con 0x7f8a600cd2a0 2024-09-16T00:07:10.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.983+0000 7f8a5d7fa640 1 -- 172.21.15.164:0/128324068 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54007310 con 0x7f8a600cd2a0 2024-09-16T00:07:10.986 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.983+0000 7f8a66fa6640 1 -- 172.21.15.164:0/128324068 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a60164710 con 0x7f8a600cd2a0 2024-09-16T00:07:10.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.983+0000 7f8a66fa6640 1 -- 172.21.15.164:0/128324068 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a60164cd0 con 0x7f8a600cd2a0 2024-09-16T00:07:10.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.983+0000 7f8a5d7fa640 1 -- 172.21.15.164:0/128324068 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8a5401a070 con 0x7f8a600cd2a0 2024-09-16T00:07:10.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.983+0000 7f8a5d7fa640 1 -- 172.21.15.164:0/128324068 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f8a54012820 con 0x7f8a600cd2a0 2024-09-16T00:07:10.987 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.984+0000 7f8a66fa6640 1 -- 172.21.15.164:0/128324068 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a600c45a0 con 0x7f8a600cd2a0 2024-09-16T00:07:10.988 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:10.987+0000 7f8a5d7fa640 1 -- 172.21.15.164:0/128324068 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f8a54021070 con 0x7f8a600cd2a0 2024-09-16T00:07:10.993 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:10.992+0000 7f7123673040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:07:11.056 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.056+0000 7f8a66fa6640 1 -- 172.21.15.164:0/128324068 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8048fc64-72b3-4866-ac62-6a8b2bf6f5c8", "id": 2} v 0) -- 0x7f8a600c6150 con 0x7f8a600cd2a0 2024-09-16T00:07:11.062 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.062+0000 7f8a5d7fa640 1 -- 172.21.15.164:0/128324068 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8048fc64-72b3-4866-ac62-6a8b2bf6f5c8", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f8a540076b0 con 0x7f8a600cd2a0 2024-09-16T00:07:11.062 INFO:teuthology.orchestra.run.smithi164.stdout:2 2024-09-16T00:07:11.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.063+0000 7f8a66fa6640 1 -- 172.21.15.164:0/128324068 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a600cd2a0 msgr2=0x7f8a6013fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.063+0000 7f8a66fa6640 1 --2- 172.21.15.164:0/128324068 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a600cd2a0 0x7f8a6013fda0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8a54000f60 tx=0x7f8a5400c800 comp rx=0 tx=0).stop 2024-09-16T00:07:11.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.063+0000 7f8a66fa6640 1 -- 172.21.15.164:0/128324068 shutdown_connections 2024-09-16T00:07:11.063 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.063+0000 7f8a66fa6640 1 --2- 172.21.15.164:0/128324068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a60151b30 0x7f8a6013b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.063+0000 7f8a66fa6640 1 --2- 172.21.15.164:0/128324068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8a600cd680 0x7f8a601f3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.063+0000 7f8a66fa6640 1 --2- 172.21.15.164:0/128324068 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8a600cd2a0 0x7f8a6013fda0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.063+0000 7f8a66fa6640 1 -- 172.21.15.164:0/128324068 >> 172.21.15.164:0/128324068 conn(0x7f8a6006cc30 msgr2=0x7f8a600cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:11.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.063+0000 7f8a66fa6640 1 -- 172.21.15.164:0/128324068 shutdown_connections 2024-09-16T00:07:11.064 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.063+0000 7f8a66fa6640 1 -- 172.21.15.164:0/128324068 wait complete. 2024-09-16T00:07:11.074 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new c34ac77b-354c-4884-902d-3e2a84f5eb27 3 2024-09-16T00:07:11.129 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:11.127+0000 7f399452b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:07:11.152 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:11.152+0000 7f7123673040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:07:11.172 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.172+0000 7f2197be4640 1 Processor -- start 2024-09-16T00:07:11.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.172+0000 7f2197be4640 1 -- start start 2024-09-16T00:07:11.173 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.173+0000 7f2197be4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21900cd2a0 0x7f21900c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.173+0000 7f2197be4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f21900cd680 0x7f2190151420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.173+0000 7f2197be4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f219006f990 con 0x7f21900c72b0 2024-09-16T00:07:11.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.173+0000 7f2197be4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f219006ee10 con 0x7f21900cd680 2024-09-16T00:07:11.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.173+0000 7f2197be4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f219006f0f0 con 0x7f21900cd2a0 2024-09-16T00:07:11.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.173+0000 7f2195980640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21900cd2a0 0x7f21900c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.173+0000 7f219517f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f21900cd680 0x7f2190151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.173+0000 7f2195980640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21900cd2a0 0x7f21900c7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:39622/0 (socket says 172.21.15.164:39622) 2024-09-16T00:07:11.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.173+0000 7f2195980640 1 -- 172.21.15.164:0/2156903186 learned_addr learned my addr 172.21.15.164:0/2156903186 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T00:07:11.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.173+0000 7f219517f640 1 -- 172.21.15.164:0/2156903186 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21900cd2a0 msgr2=0x7f21900c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:11.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.173+0000 7f219517f640 1 --2- 172.21.15.164:0/2156903186 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21900cd2a0 0x7f21900c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.173+0000 7f219517f640 1 -- 172.21.15.164:0/2156903186 >> v1:172.21.15.139:6789/0 conn(0x7f21900c72b0 legacy=0x7f2190157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.174 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.173+0000 7f219517f640 1 -- 172.21.15.164:0/2156903186 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219006f3d0 con 0x7f21900cd680 2024-09-16T00:07:11.175 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.174+0000 7f219517f640 1 --2- 172.21.15.164:0/2156903186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21900cd680 0x7f2190151420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f21800097d0 tx=0x7f218002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14168febda2bb47a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:11.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.174+0000 7f219497e640 1 -- 172.21.15.164:0/2156903186 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180037070 con 0x7f21900cd680 2024-09-16T00:07:11.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.174+0000 7f219497e640 1 -- 172.21.15.164:0/2156903186 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2180032440 con 0x7f21900cd680 2024-09-16T00:07:11.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.174+0000 7f219497e640 1 -- 172.21.15.164:0/2156903186 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180032810 con 0x7f21900cd680 2024-09-16T00:07:11.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.175+0000 7f2197be4640 1 -- 172.21.15.164:0/2156903186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21900cd680 msgr2=0x7f2190151420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.175+0000 7f2197be4640 1 --2- 172.21.15.164:0/2156903186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21900cd680 0x7f2190151420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f21800097d0 tx=0x7f218002a250 comp rx=0 tx=0).stop 2024-09-16T00:07:11.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.175+0000 7f2197be4640 1 -- 172.21.15.164:0/2156903186 shutdown_connections 2024-09-16T00:07:11.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.175+0000 7f2197be4640 1 --2- 172.21.15.164:0/2156903186 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21900cd680 0x7f2190151420 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.175+0000 7f2197be4640 1 --2- 172.21.15.164:0/2156903186 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21900cd2a0 0x7f21900c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.175+0000 7f2197be4640 1 -- 172.21.15.164:0/2156903186 >> 172.21.15.164:0/2156903186 conn(0x7f219006cc30 msgr2=0x7f219006d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:11.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.175+0000 7f2197be4640 1 -- 172.21.15.164:0/2156903186 shutdown_connections 2024-09-16T00:07:11.176 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.175+0000 7f2197be4640 1 -- 172.21.15.164:0/2156903186 wait complete. 2024-09-16T00:07:11.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.176+0000 7f2197be4640 1 Processor -- start 2024-09-16T00:07:11.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.176+0000 7f2197be4640 1 -- start start 2024-09-16T00:07:11.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f2197be4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21900c72b0 0x7f21901472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f2197be4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21900cd2a0 0x7f21901ed580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f2197be4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21900cd680 0x7f21901edac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f2197be4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f21901582e0 con 0x7f21900cd680 2024-09-16T00:07:11.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f2197be4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f21901576b0 con 0x7f21900cd2a0 2024-09-16T00:07:11.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f2197be4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2190158460 con 0x7f21900c72b0 2024-09-16T00:07:11.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f219517f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21900cd2a0 0x7f21901ed580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f2195980640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21900c72b0 0x7f21901472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f219517f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21900cd2a0 0x7f21901ed580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35184/0 (socket says 172.21.15.164:35184) 2024-09-16T00:07:11.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f219517f640 1 -- 172.21.15.164:0/477920709 learned_addr learned my addr 172.21.15.164:0/477920709 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T00:07:11.179 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f2196181640 1 --2- 172.21.15.164:0/477920709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21900cd680 0x7f21901edac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f219517f640 1 -- 172.21.15.164:0/477920709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21900c72b0 msgr2=0x7f21901472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f219517f640 1 --2- 172.21.15.164:0/477920709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21900c72b0 0x7f21901472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f219517f640 1 -- 172.21.15.164:0/477920709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21900cd680 msgr2=0x7f21901edac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f219517f640 1 --2- 172.21.15.164:0/477920709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21900cd680 0x7f21901edac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f219517f640 1 -- 172.21.15.164:0/477920709 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219014d970 con 0x7f21900cd2a0 2024-09-16T00:07:11.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f2195980640 1 --2- 172.21.15.164:0/477920709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21900c72b0 0x7f21901472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:11.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f219517f640 1 --2- 172.21.15.164:0/477920709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21900cd2a0 0x7f21901ed580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2180000c00 tx=0x7f21800046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:11.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f21867fc640 1 -- 172.21.15.164:0/477920709 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180037040 con 0x7f21900cd2a0 2024-09-16T00:07:11.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f21867fc640 1 -- 172.21.15.164:0/477920709 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2180042070 con 0x7f21900cd2a0 2024-09-16T00:07:11.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f2197be4640 1 -- 172.21.15.164:0/477920709 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219014e1f0 con 0x7f21900cd2a0 2024-09-16T00:07:11.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.177+0000 7f21867fc640 1 -- 172.21.15.164:0/477920709 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180030920 con 0x7f21900cd2a0 2024-09-16T00:07:11.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.178+0000 7f2197be4640 1 -- 172.21.15.164:0/477920709 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f219014df10 con 0x7f21900cd2a0 2024-09-16T00:07:11.180 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.178+0000 7f21867fc640 1 -- 172.21.15.164:0/477920709 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2180030ac0 con 0x7f21900cd2a0 2024-09-16T00:07:11.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.178+0000 7f21867fc640 1 -- 172.21.15.164:0/477920709 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f218003b430 con 0x7f21900cd2a0 2024-09-16T00:07:11.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.178+0000 7f2197be4640 1 -- 172.21.15.164:0/477920709 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21900c45a0 con 0x7f21900cd2a0 2024-09-16T00:07:11.181 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.181+0000 7f21867fc640 1 -- 172.21.15.164:0/477920709 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f2180047460 con 0x7f21900cd2a0 2024-09-16T00:07:11.249 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.249+0000 7f2197be4640 1 -- 172.21.15.164:0/477920709 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c34ac77b-354c-4884-902d-3e2a84f5eb27", "id": 3} v 0) -- 0x7f2190070b00 con 0x7f21900cd2a0 2024-09-16T00:07:11.255 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.254+0000 7f21867fc640 1 -- 172.21.15.164:0/477920709 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c34ac77b-354c-4884-902d-3e2a84f5eb27", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f21800476c0 con 0x7f21900cd2a0 2024-09-16T00:07:11.255 INFO:teuthology.orchestra.run.smithi164.stdout:3 2024-09-16T00:07:11.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.255+0000 7f2197be4640 1 -- 172.21.15.164:0/477920709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21900cd2a0 msgr2=0x7f21901ed580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.255+0000 7f2197be4640 1 --2- 172.21.15.164:0/477920709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21900cd2a0 0x7f21901ed580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2180000c00 tx=0x7f21800046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.255+0000 7f2197be4640 1 -- 172.21.15.164:0/477920709 shutdown_connections 2024-09-16T00:07:11.256 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.255+0000 7f2197be4640 1 --2- 172.21.15.164:0/477920709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21900cd680 0x7f21901edac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.255+0000 7f2197be4640 1 --2- 172.21.15.164:0/477920709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f21900cd2a0 0x7f21901ed580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.255+0000 7f2197be4640 1 --2- 172.21.15.164:0/477920709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f21900c72b0 0x7f21901472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.255+0000 7f2197be4640 1 -- 172.21.15.164:0/477920709 >> 172.21.15.164:0/477920709 conn(0x7f219006cc30 msgr2=0x7f21900cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:11.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.255+0000 7f2197be4640 1 -- 172.21.15.164:0/477920709 shutdown_connections 2024-09-16T00:07:11.257 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.255+0000 7f2197be4640 1 -- 172.21.15.164:0/477920709 wait complete. 2024-09-16T00:07:11.266 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new ba9a7c93-9672-43e1-8c98-2878cdf7f548 4 2024-09-16T00:07:11.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.364+0000 7efe8157f640 1 Processor -- start 2024-09-16T00:07:11.365 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.364+0000 7efe8157f640 1 -- start start 2024-09-16T00:07:11.366 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.365+0000 7efe8157f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efe7c06f1b0 0x7efe7c151100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.365+0000 7efe8157f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efe7c15b490 0x7efe7c151640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.365+0000 7efe8157f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe7c0c7670 con 0x7efe7c06edd0 2024-09-16T00:07:11.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.365+0000 7efe8157f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7efe7c0c5f00 con 0x7efe7c15b490 2024-09-16T00:07:11.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.365+0000 7efe8157f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7efe7c157800 con 0x7efe7c06f1b0 2024-09-16T00:07:11.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.365+0000 7efe7a7fc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efe7c06f1b0 0x7efe7c151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.365+0000 7efe7a7fc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efe7c06f1b0 0x7efe7c151100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35192/0 (socket says 172.21.15.164:35192) 2024-09-16T00:07:11.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.365+0000 7efe7a7fc640 1 -- 172.21.15.164:0/2207225558 learned_addr learned my addr 172.21.15.164:0/2207225558 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T00:07:11.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.365+0000 7efe7b7fe640 1 --2- 172.21.15.164:0/2207225558 >> v2:172.21.15.139:3300/0 conn(0x7efe7c15b490 0x7efe7c151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.365+0000 7efe7a7fc640 1 -- 172.21.15.164:0/2207225558 >> v2:172.21.15.139:3300/0 conn(0x7efe7c15b490 msgr2=0x7efe7c151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:11.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.365+0000 7efe7a7fc640 1 --2- 172.21.15.164:0/2207225558 >> v2:172.21.15.139:3300/0 conn(0x7efe7c15b490 0x7efe7c151640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.365+0000 7efe7a7fc640 1 -- 172.21.15.164:0/2207225558 >> v1:172.21.15.139:6789/0 conn(0x7efe7c06edd0 legacy=0x7efe7c15ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.365+0000 7efe7a7fc640 1 -- 172.21.15.164:0/2207225558 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7c157ae0 con 0x7efe7c06f1b0 2024-09-16T00:07:11.367 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.366+0000 7efe7a7fc640 1 --2- 172.21.15.164:0/2207225558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c06f1b0 0x7efe7c151100 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0c61d0 tx=0x7efe7002c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e77a9864a589e59 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:11.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.366+0000 7efe79ffb640 1 -- 172.21.15.164:0/2207225558 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70033640 con 0x7efe7c06f1b0 2024-09-16T00:07:11.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.366+0000 7efe79ffb640 1 -- 172.21.15.164:0/2207225558 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe70038070 con 0x7efe7c06f1b0 2024-09-16T00:07:11.368 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.366+0000 7efe79ffb640 1 -- 172.21.15.164:0/2207225558 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70033a00 con 0x7efe7c06f1b0 2024-09-16T00:07:11.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.367+0000 7efe8157f640 1 -- 172.21.15.164:0/2207225558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c06f1b0 msgr2=0x7efe7c151100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.367+0000 7efe8157f640 1 --2- 172.21.15.164:0/2207225558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c06f1b0 0x7efe7c151100 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0c61d0 tx=0x7efe7002c8b0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.367+0000 7efe8157f640 1 -- 172.21.15.164:0/2207225558 shutdown_connections 2024-09-16T00:07:11.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.367+0000 7efe8157f640 1 --2- 172.21.15.164:0/2207225558 >> v2:172.21.15.139:3300/0 conn(0x7efe7c15b490 0x7efe7c151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.367+0000 7efe8157f640 1 --2- 172.21.15.164:0/2207225558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c06f1b0 0x7efe7c151100 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.367+0000 7efe8157f640 1 -- 172.21.15.164:0/2207225558 >> 172.21.15.164:0/2207225558 conn(0x7efe7c067950 msgr2=0x7efe7c067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:11.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.367+0000 7efe8157f640 1 -- 172.21.15.164:0/2207225558 shutdown_connections 2024-09-16T00:07:11.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.367+0000 7efe8157f640 1 -- 172.21.15.164:0/2207225558 wait complete. 2024-09-16T00:07:11.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.368+0000 7efe8157f640 1 Processor -- start 2024-09-16T00:07:11.369 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.368+0000 7efe8157f640 1 -- start start 2024-09-16T00:07:11.371 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe8157f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe7c06edd0 0x7efe7c1f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe8157f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe7c06f1b0 0x7efe7c156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe8157f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c15b490 0x7efe7c14ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe8157f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c158380 con 0x7efe7c06edd0 2024-09-16T00:07:11.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe8157f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c157e80 con 0x7efe7c15b490 2024-09-16T00:07:11.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe8157f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7efe7c151b80 con 0x7efe7c06f1b0 2024-09-16T00:07:11.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe7a7fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe7c06f1b0 0x7efe7c156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe7b7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c15b490 0x7efe7c14ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe7a7fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe7c06f1b0 0x7efe7c156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:39628/0 (socket says 172.21.15.164:39628) 2024-09-16T00:07:11.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe7a7fc640 1 -- 172.21.15.164:0/1380587245 learned_addr learned my addr 172.21.15.164:0/1380587245 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T00:07:11.372 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe7b7fe640 1 --2- 172.21.15.164:0/1380587245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c15b490 0x7efe7c14ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35208/0 (socket says 172.21.15.164:35208) 2024-09-16T00:07:11.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe7affd640 1 --2- 172.21.15.164:0/1380587245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe7c06edd0 0x7efe7c1f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe7b7fe640 1 -- 172.21.15.164:0/1380587245 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe7c06f1b0 msgr2=0x7efe7c156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe7b7fe640 1 --2- 172.21.15.164:0/1380587245 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe7c06f1b0 0x7efe7c156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe7b7fe640 1 -- 172.21.15.164:0/1380587245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe7c06edd0 msgr2=0x7efe7c1f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe7b7fe640 1 --2- 172.21.15.164:0/1380587245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe7c06edd0 0x7efe7c1f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe7b7fe640 1 -- 172.21.15.164:0/1380587245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7c06c710 con 0x7efe7c15b490 2024-09-16T00:07:11.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.369+0000 7efe7a7fc640 1 --2- 172.21.15.164:0/1380587245 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe7c06f1b0 0x7efe7c156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:11.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.370+0000 7efe7b7fe640 1 --2- 172.21.15.164:0/1380587245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c15b490 0x7efe7c14ab40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efe7400ebd0 tx=0x7efe7400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:11.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.370+0000 7efe637fe640 1 -- 172.21.15.164:0/1380587245 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe74018020 con 0x7efe7c15b490 2024-09-16T00:07:11.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.370+0000 7efe637fe640 1 -- 172.21.15.164:0/1380587245 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe7400cd20 con 0x7efe7c15b490 2024-09-16T00:07:11.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.370+0000 7efe8157f640 1 -- 172.21.15.164:0/1380587245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7c06c9d0 con 0x7efe7c15b490 2024-09-16T00:07:11.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.370+0000 7efe637fe640 1 -- 172.21.15.164:0/1380587245 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe740146c0 con 0x7efe7c15b490 2024-09-16T00:07:11.373 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.370+0000 7efe8157f640 1 -- 172.21.15.164:0/1380587245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe7c0c9240 con 0x7efe7c15b490 2024-09-16T00:07:11.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.370+0000 7efe637fe640 1 -- 172.21.15.164:0/1380587245 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efe7401b020 con 0x7efe7c15b490 2024-09-16T00:07:11.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.370+0000 7efe637fe640 1 -- 172.21.15.164:0/1380587245 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7efe7401dd10 con 0x7efe7c15b490 2024-09-16T00:07:11.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.371+0000 7efe8157f640 1 -- 172.21.15.164:0/1380587245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe7c1574b0 con 0x7efe7c15b490 2024-09-16T00:07:11.374 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.373+0000 7efe637fe640 1 -- 172.21.15.164:0/1380587245 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7efe74022020 con 0x7efe7c15b490 2024-09-16T00:07:11.443 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.442+0000 7efe8157f640 1 -- 172.21.15.164:0/1380587245 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ba9a7c93-9672-43e1-8c98-2878cdf7f548", "id": 4} v 0) -- 0x7efe7c0c5ad0 con 0x7efe7c15b490 2024-09-16T00:07:11.451 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.450+0000 7efe637fe640 1 -- 172.21.15.164:0/1380587245 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ba9a7c93-9672-43e1-8c98-2878cdf7f548", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7efe74010040 con 0x7efe7c15b490 2024-09-16T00:07:11.451 INFO:teuthology.orchestra.run.smithi164.stdout:4 2024-09-16T00:07:11.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.451+0000 7efe8157f640 1 -- 172.21.15.164:0/1380587245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c15b490 msgr2=0x7efe7c14ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.451+0000 7efe8157f640 1 --2- 172.21.15.164:0/1380587245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c15b490 0x7efe7c14ab40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efe7400ebd0 tx=0x7efe7400c490 comp rx=0 tx=0).stop 2024-09-16T00:07:11.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.451+0000 7efe8157f640 1 -- 172.21.15.164:0/1380587245 shutdown_connections 2024-09-16T00:07:11.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.451+0000 7efe8157f640 1 --2- 172.21.15.164:0/1380587245 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe7c15b490 0x7efe7c14ab40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.451+0000 7efe8157f640 1 --2- 172.21.15.164:0/1380587245 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe7c06f1b0 0x7efe7c156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.451+0000 7efe8157f640 1 --2- 172.21.15.164:0/1380587245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe7c06edd0 0x7efe7c1f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.452 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.451+0000 7efe8157f640 1 -- 172.21.15.164:0/1380587245 >> 172.21.15.164:0/1380587245 conn(0x7efe7c067950 msgr2=0x7efe7c074d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:11.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.451+0000 7efe8157f640 1 -- 172.21.15.164:0/1380587245 shutdown_connections 2024-09-16T00:07:11.453 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.451+0000 7efe8157f640 1 -- 172.21.15.164:0/1380587245 wait complete. 2024-09-16T00:07:11.463 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 46a0916b-474c-411b-84ad-86f27d4cbf53 5 2024-09-16T00:07:11.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.562+0000 7fef54682640 1 Processor -- start 2024-09-16T00:07:11.563 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.562+0000 7fef54682640 1 -- start start 2024-09-16T00:07:11.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.563+0000 7fef54682640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fef4c06cda0 0x7fef4c067b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.564 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.563+0000 7fef54682640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fef4c1550a0 0x7fef4c15ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.563+0000 7fef54682640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef4c070680 con 0x7fef4c06d180 2024-09-16T00:07:11.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.563+0000 7fef54682640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fef4c15e160 con 0x7fef4c06cda0 2024-09-16T00:07:11.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.563+0000 7fef54682640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fef4c15f0f0 con 0x7fef4c1550a0 2024-09-16T00:07:11.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.563+0000 7fef52c1f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fef4c1550a0 0x7fef4c15ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.563+0000 7fef52c1f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fef4c1550a0 0x7fef4c15ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35218/0 (socket says 172.21.15.164:35218) 2024-09-16T00:07:11.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.563+0000 7fef51c1d640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fef4c06d180 0x7fef4c154990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:58674/0 (socket says 172.21.15.164:58674) 2024-09-16T00:07:11.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.563+0000 7fef52c1f640 1 -- 172.21.15.164:0/436359386 learned_addr learned my addr 172.21.15.164:0/436359386 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T00:07:11.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.563+0000 7fef5241e640 1 --2- 172.21.15.164:0/436359386 >> v2:172.21.15.139:3300/0 conn(0x7fef4c06cda0 0x7fef4c067b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.564+0000 7fef52c1f640 1 -- 172.21.15.164:0/436359386 >> v2:172.21.15.139:3300/0 conn(0x7fef4c06cda0 msgr2=0x7fef4c067b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:11.565 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.564+0000 7fef52c1f640 1 --2- 172.21.15.164:0/436359386 >> v2:172.21.15.139:3300/0 conn(0x7fef4c06cda0 0x7fef4c067b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.564+0000 7fef5141c640 1 -- 172.21.15.164:0/436359386 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3437025844 0 0) 0x7fef4c070680 con 0x7fef4c06d180 2024-09-16T00:07:11.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.564+0000 7fef52c1f640 1 -- 172.21.15.164:0/436359386 >> v1:172.21.15.164:6789/0 conn(0x7fef4c06d180 legacy=0x7fef4c154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.566 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.564+0000 7fef52c1f640 1 -- 172.21.15.164:0/436359386 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c070960 con 0x7fef4c1550a0 2024-09-16T00:07:11.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.565+0000 7fef52c1f640 1 --2- 172.21.15.164:0/436359386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c1550a0 0x7fef4c15ebb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0713b0 tx=0x7fef4402a7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=644a8624a031232b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:11.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.565+0000 7fef5141c640 1 -- 172.21.15.164:0/436359386 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44036070 con 0x7fef4c1550a0 2024-09-16T00:07:11.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.565+0000 7fef5141c640 1 -- 172.21.15.164:0/436359386 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef44031600 con 0x7fef4c1550a0 2024-09-16T00:07:11.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.565+0000 7fef5141c640 1 -- 172.21.15.164:0/436359386 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef440319f0 con 0x7fef4c1550a0 2024-09-16T00:07:11.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.565+0000 7fef54682640 1 -- 172.21.15.164:0/436359386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c1550a0 msgr2=0x7fef4c15ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.565+0000 7fef54682640 1 --2- 172.21.15.164:0/436359386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c1550a0 0x7fef4c15ebb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0713b0 tx=0x7fef4402a7c0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.566+0000 7fef54682640 1 -- 172.21.15.164:0/436359386 shutdown_connections 2024-09-16T00:07:11.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.566+0000 7fef54682640 1 --2- 172.21.15.164:0/436359386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c1550a0 0x7fef4c15ebb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.566+0000 7fef54682640 1 --2- 172.21.15.164:0/436359386 >> v2:172.21.15.139:3300/0 conn(0x7fef4c06cda0 0x7fef4c067b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.566+0000 7fef54682640 1 -- 172.21.15.164:0/436359386 >> 172.21.15.164:0/436359386 conn(0x7fef4c06f800 msgr2=0x7fef4c06fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:11.571 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.566+0000 7fef54682640 1 -- 172.21.15.164:0/436359386 shutdown_connections 2024-09-16T00:07:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.566+0000 7fef54682640 1 -- 172.21.15.164:0/436359386 wait complete. 2024-09-16T00:07:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.567+0000 7fef54682640 1 Processor -- start 2024-09-16T00:07:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.567+0000 7fef54682640 1 -- start start 2024-09-16T00:07:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.567+0000 7fef54682640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fef4c06cda0 0x7fef4c0c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.567+0000 7fef54682640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c06d180 0x7fef4c155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.567+0000 7fef54682640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef4c1550a0 0x7fef4c15cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.567+0000 7fef54682640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c0c8990 con 0x7fef4c1550a0 2024-09-16T00:07:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.567+0000 7fef54682640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c15e690 con 0x7fef4c06d180 2024-09-16T00:07:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.567+0000 7fef54682640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fef4c0c8b10 con 0x7fef4c06cda0 2024-09-16T00:07:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef51c1d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c06d180 0x7fef4c155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef5241e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fef4c06cda0 0x7fef4c0c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef51c1d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c06d180 0x7fef4c155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35228/0 (socket says 172.21.15.164:35228) 2024-09-16T00:07:11.572 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef51c1d640 1 -- 172.21.15.164:0/3593449384 learned_addr learned my addr 172.21.15.164:0/3593449384 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T00:07:11.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef52c1f640 1 --2- 172.21.15.164:0/3593449384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef4c1550a0 0x7fef4c15cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef51c1d640 1 -- 172.21.15.164:0/3593449384 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fef4c06cda0 msgr2=0x7fef4c0c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef51c1d640 1 --2- 172.21.15.164:0/3593449384 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fef4c06cda0 0x7fef4c0c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef51c1d640 1 -- 172.21.15.164:0/3593449384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef4c1550a0 msgr2=0x7fef4c15cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef51c1d640 1 --2- 172.21.15.164:0/3593449384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef4c1550a0 0x7fef4c15cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef51c1d640 1 -- 172.21.15.164:0/3593449384 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c15bd30 con 0x7fef4c06d180 2024-09-16T00:07:11.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef51c1d640 1 --2- 172.21.15.164:0/3593449384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c06d180 0x7fef4c155730 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fef4800c4d0 tx=0x7fef4800c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:11.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef42ffd640 1 -- 172.21.15.164:0/3593449384 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48018070 con 0x7fef4c06d180 2024-09-16T00:07:11.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef42ffd640 1 -- 172.21.15.164:0/3593449384 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef48007500 con 0x7fef4c06d180 2024-09-16T00:07:11.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef42ffd640 1 -- 172.21.15.164:0/3593449384 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48004a20 con 0x7fef4c06d180 2024-09-16T00:07:11.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef54682640 1 -- 172.21.15.164:0/3593449384 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4c1f11e0 con 0x7fef4c06d180 2024-09-16T00:07:11.573 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef54682640 1 -- 172.21.15.164:0/3593449384 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef4c15c2d0 con 0x7fef4c06d180 2024-09-16T00:07:11.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef42ffd640 1 -- 172.21.15.164:0/3593449384 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fef4801b070 con 0x7fef4c06d180 2024-09-16T00:07:11.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.568+0000 7fef42ffd640 1 -- 172.21.15.164:0/3593449384 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fef48004bc0 con 0x7fef4c06d180 2024-09-16T00:07:11.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.569+0000 7fef54682640 1 -- 172.21.15.164:0/3593449384 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef4c15b050 con 0x7fef4c06d180 2024-09-16T00:07:11.574 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.571+0000 7fef42ffd640 1 -- 172.21.15.164:0/3593449384 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fef4c15b050 con 0x7fef4c06d180 2024-09-16T00:07:11.640 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.639+0000 7fef54682640 1 -- 172.21.15.164:0/3593449384 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd new", "uuid": "46a0916b-474c-411b-84ad-86f27d4cbf53", "id": 5} v 0) -- 0x7fef4c0720d0 con 0x7fef4c06d180 2024-09-16T00:07:11.645 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.645+0000 7fef42ffd640 1 -- 172.21.15.164:0/3593449384 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "46a0916b-474c-411b-84ad-86f27d4cbf53", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fef4801dbc0 con 0x7fef4c06d180 2024-09-16T00:07:11.645 INFO:teuthology.orchestra.run.smithi164.stdout:5 2024-09-16T00:07:11.646 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.646+0000 7fef54682640 1 -- 172.21.15.164:0/3593449384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c06d180 msgr2=0x7fef4c155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.646+0000 7fef54682640 1 --2- 172.21.15.164:0/3593449384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c06d180 0x7fef4c155730 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fef4800c4d0 tx=0x7fef4800c990 comp rx=0 tx=0).stop 2024-09-16T00:07:11.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.646+0000 7fef54682640 1 -- 172.21.15.164:0/3593449384 shutdown_connections 2024-09-16T00:07:11.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.646+0000 7fef54682640 1 --2- 172.21.15.164:0/3593449384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef4c1550a0 0x7fef4c15cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.646+0000 7fef54682640 1 --2- 172.21.15.164:0/3593449384 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef4c06d180 0x7fef4c155730 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.646+0000 7fef54682640 1 --2- 172.21.15.164:0/3593449384 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fef4c06cda0 0x7fef4c0c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.646+0000 7fef54682640 1 -- 172.21.15.164:0/3593449384 >> 172.21.15.164:0/3593449384 conn(0x7fef4c06f800 msgr2=0x7fef4c159b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:11.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.646+0000 7fef54682640 1 -- 172.21.15.164:0/3593449384 shutdown_connections 2024-09-16T00:07:11.647 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.646+0000 7fef54682640 1 -- 172.21.15.164:0/3593449384 wait complete. 2024-09-16T00:07:11.657 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 9b6b33cc-d6d9-4a0a-8143-526073a58b93 6 2024-09-16T00:07:11.755 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.755+0000 7f5dbc391640 1 Processor -- start 2024-09-16T00:07:11.756 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.755+0000 7f5dbc391640 1 -- start start 2024-09-16T00:07:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.756+0000 7f5dbc391640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5db4151850 0x7f5db40c6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.756+0000 7f5dbc391640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5db4151c30 0x7f5db415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.757 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.756+0000 7f5dbc391640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5db40c8d30 con 0x7f5db40c7390 2024-09-16T00:07:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.756+0000 7f5dbc391640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5db40c88a0 con 0x7f5db4151c30 2024-09-16T00:07:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.756+0000 7f5dbc391640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5db40c6a90 con 0x7f5db4151850 2024-09-16T00:07:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.756+0000 7f5dba12d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5db4151850 0x7f5db40c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.756+0000 7f5dba12d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5db4151850 0x7f5db40c6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:35244/0 (socket says 172.21.15.164:35244) 2024-09-16T00:07:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.756+0000 7f5dba12d640 1 -- 172.21.15.164:0/2961644600 learned_addr learned my addr 172.21.15.164:0/2961644600 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T00:07:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.756+0000 7f5db992c640 1 --2- 172.21.15.164:0/2961644600 >> v2:172.21.15.139:3300/0 conn(0x7f5db4151c30 0x7f5db415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.756+0000 7f5dba12d640 1 -- 172.21.15.164:0/2961644600 >> v2:172.21.15.139:3300/0 conn(0x7f5db4151c30 msgr2=0x7f5db415ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.756+0000 7f5dba12d640 1 --2- 172.21.15.164:0/2961644600 >> v2:172.21.15.139:3300/0 conn(0x7f5db4151c30 0x7f5db415ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.756+0000 7f5dba12d640 1 -- 172.21.15.164:0/2961644600 >> v1:172.21.15.139:6789/0 conn(0x7f5db40c7390 legacy=0x7f5db40c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.756+0000 7f5dba12d640 1 -- 172.21.15.164:0/2961644600 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db40c67b0 con 0x7f5db4151850 2024-09-16T00:07:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.756+0000 7f5db992c640 1 --2- 172.21.15.164:0/2961644600 >> v2:172.21.15.139:3300/0 conn(0x7f5db4151c30 0x7f5db415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.757+0000 7f5dba12d640 1 --2- 172.21.15.164:0/2961644600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db4151850 0x7f5db40c6100 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5db4075c60 tx=0x7f5db002a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d603672e1acbc94f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:11.758 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.757+0000 7f5db912b640 1 -- 172.21.15.164:0/2961644600 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0037070 con 0x7f5db4151850 2024-09-16T00:07:11.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.757+0000 7f5db912b640 1 -- 172.21.15.164:0/2961644600 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db002ae50 con 0x7f5db4151850 2024-09-16T00:07:11.759 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.757+0000 7f5db912b640 1 -- 172.21.15.164:0/2961644600 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0032860 con 0x7f5db4151850 2024-09-16T00:07:11.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.758+0000 7f5dbc391640 1 -- 172.21.15.164:0/2961644600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db4151850 msgr2=0x7f5db40c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.761 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.758+0000 7f5dbc391640 1 --2- 172.21.15.164:0/2961644600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db4151850 0x7f5db40c6100 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5db4075c60 tx=0x7f5db002a300 comp rx=0 tx=0).stop 2024-09-16T00:07:11.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.758+0000 7f5dbc391640 1 -- 172.21.15.164:0/2961644600 shutdown_connections 2024-09-16T00:07:11.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.758+0000 7f5dbc391640 1 --2- 172.21.15.164:0/2961644600 >> v2:172.21.15.139:3300/0 conn(0x7f5db4151c30 0x7f5db415ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.758+0000 7f5dbc391640 1 --2- 172.21.15.164:0/2961644600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db4151850 0x7f5db40c6100 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.758+0000 7f5dbc391640 1 -- 172.21.15.164:0/2961644600 >> 172.21.15.164:0/2961644600 conn(0x7f5db406cc30 msgr2=0x7f5db406d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:11.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.758+0000 7f5dbc391640 1 -- 172.21.15.164:0/2961644600 shutdown_connections 2024-09-16T00:07:11.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.758+0000 7f5dbc391640 1 -- 172.21.15.164:0/2961644600 wait complete. 2024-09-16T00:07:11.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.759+0000 7f5dbc391640 1 Processor -- start 2024-09-16T00:07:11.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.759+0000 7f5dbc391640 1 -- start start 2024-09-16T00:07:11.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5dbc391640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5db40c7390 0x7f5db4154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5dbc391640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5db4151850 0x7f5db41f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5dbc391640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db4151c30 0x7f5db4142c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.762 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5dbc391640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5db40c8530 con 0x7f5db40c7390 2024-09-16T00:07:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5dbc391640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5db40c8250 con 0x7f5db4151c30 2024-09-16T00:07:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5dbc391640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5db40c96c0 con 0x7f5db4151850 2024-09-16T00:07:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5db992c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5db4151850 0x7f5db41f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5dba92e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db4151c30 0x7f5db4142c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5db992c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5db4151850 0x7f5db41f3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:39650/0 (socket says 172.21.15.164:39650) 2024-09-16T00:07:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5db992c640 1 -- 172.21.15.164:0/1881411446 learned_addr learned my addr 172.21.15.164:0/1881411446 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T00:07:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5dba12d640 1 --2- 172.21.15.164:0/1881411446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5db40c7390 0x7f5db4154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5dba92e640 1 -- 172.21.15.164:0/1881411446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5db4151850 msgr2=0x7f5db41f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5dba92e640 1 --2- 172.21.15.164:0/1881411446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5db4151850 0x7f5db41f3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5dba92e640 1 -- 172.21.15.164:0/1881411446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5db40c7390 msgr2=0x7f5db4154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5dba92e640 1 --2- 172.21.15.164:0/1881411446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5db40c7390 0x7f5db4154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5dba92e640 1 -- 172.21.15.164:0/1881411446 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db41f42b0 con 0x7f5db4151c30 2024-09-16T00:07:11.763 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5db992c640 1 --2- 172.21.15.164:0/1881411446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5db4151850 0x7f5db41f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:11.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.760+0000 7f5dba92e640 1 --2- 172.21.15.164:0/1881411446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db4151c30 0x7f5db4142c90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00c700 tx=0x7f5dac00cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:11.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.761+0000 7f5daaffd640 1 -- 172.21.15.164:0/1881411446 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac003b70 con 0x7f5db4151c30 2024-09-16T00:07:11.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.761+0000 7f5daaffd640 1 -- 172.21.15.164:0/1881411446 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dac003d10 con 0x7f5db4151c30 2024-09-16T00:07:11.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.761+0000 7f5daaffd640 1 -- 172.21.15.164:0/1881411446 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac005310 con 0x7f5db4151c30 2024-09-16T00:07:11.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.761+0000 7f5dbc391640 1 -- 172.21.15.164:0/1881411446 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db4153e80 con 0x7f5db4151c30 2024-09-16T00:07:11.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.761+0000 7f5dbc391640 1 -- 172.21.15.164:0/1881411446 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db41f09a0 con 0x7f5db4151c30 2024-09-16T00:07:11.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.761+0000 7f5daaffd640 1 -- 172.21.15.164:0/1881411446 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5dac0054b0 con 0x7f5db4151c30 2024-09-16T00:07:11.764 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.761+0000 7f5daaffd640 1 -- 172.21.15.164:0/1881411446 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f5dac012a10 con 0x7f5db4151c30 2024-09-16T00:07:11.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.762+0000 7f5dbc391640 1 -- 172.21.15.164:0/1881411446 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db414fd60 con 0x7f5db4151c30 2024-09-16T00:07:11.765 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.764+0000 7f5daaffd640 1 -- 172.21.15.164:0/1881411446 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5db414fd60 con 0x7f5db4151c30 2024-09-16T00:07:11.832 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.832+0000 7f5dbc391640 1 -- 172.21.15.164:0/1881411446 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9b6b33cc-d6d9-4a0a-8143-526073a58b93", "id": 6} v 0) -- 0x7f5db40c45a0 con 0x7f5db4151c30 2024-09-16T00:07:11.838 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.837+0000 7f5daaffd640 1 -- 172.21.15.164:0/1881411446 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9b6b33cc-d6d9-4a0a-8143-526073a58b93", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f5dac012d30 con 0x7f5db4151c30 2024-09-16T00:07:11.838 INFO:teuthology.orchestra.run.smithi164.stdout:6 2024-09-16T00:07:11.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.838+0000 7f5dbc391640 1 -- 172.21.15.164:0/1881411446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db4151c30 msgr2=0x7f5db4142c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.838+0000 7f5dbc391640 1 --2- 172.21.15.164:0/1881411446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db4151c30 0x7f5db4142c90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00c700 tx=0x7f5dac00cbc0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.838+0000 7f5dbc391640 1 -- 172.21.15.164:0/1881411446 shutdown_connections 2024-09-16T00:07:11.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.838+0000 7f5dbc391640 1 --2- 172.21.15.164:0/1881411446 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5db4151c30 0x7f5db4142c90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.838+0000 7f5dbc391640 1 --2- 172.21.15.164:0/1881411446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5db4151850 0x7f5db41f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.838+0000 7f5dbc391640 1 --2- 172.21.15.164:0/1881411446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5db40c7390 0x7f5db4154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.839 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.838+0000 7f5dbc391640 1 -- 172.21.15.164:0/1881411446 >> 172.21.15.164:0/1881411446 conn(0x7f5db406cc30 msgr2=0x7f5db41462f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.838+0000 7f5dbc391640 1 -- 172.21.15.164:0/1881411446 shutdown_connections 2024-09-16T00:07:11.840 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.838+0000 7f5dbc391640 1 -- 172.21.15.164:0/1881411446 wait complete. 2024-09-16T00:07:11.849 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd new 50bcd808-3d3e-4a72-88e4-30decb53cfea 7 2024-09-16T00:07:11.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.945+0000 7f2d7af2a640 1 Processor -- start 2024-09-16T00:07:11.947 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.945+0000 7f2d7af2a640 1 -- start start 2024-09-16T00:07:11.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.946+0000 7f2d7af2a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d740cd2a0 0x7f2d7406bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.946+0000 7f2d7af2a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2d7406c260 0x7f2d74158890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.946+0000 7f2d7af2a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d7406db20 con 0x7f2d740cd680 2024-09-16T00:07:11.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.946+0000 7f2d7af2a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2d7406c760 con 0x7f2d7406c260 2024-09-16T00:07:11.948 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.946+0000 7f2d7af2a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2d740c9b20 con 0x7f2d740cd2a0 2024-09-16T00:07:11.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.946+0000 7f2d78cc6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d740cd2a0 0x7f2d7406bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.946+0000 7f2d78cc6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d740cd2a0 0x7f2d7406bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:39658/0 (socket says 172.21.15.164:39658) 2024-09-16T00:07:11.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.946+0000 7f2d78cc6640 1 -- 172.21.15.164:0/3834633004 learned_addr learned my addr 172.21.15.164:0/3834633004 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T00:07:11.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.946+0000 7f2d794c7640 1 --2- 172.21.15.164:0/3834633004 >> v2:172.21.15.139:3300/0 conn(0x7f2d7406c260 0x7f2d74158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.946+0000 7f2d78cc6640 1 -- 172.21.15.164:0/3834633004 >> v2:172.21.15.139:3300/0 conn(0x7f2d7406c260 msgr2=0x7f2d74158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:11.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.946+0000 7f2d78cc6640 1 --2- 172.21.15.164:0/3834633004 >> v2:172.21.15.139:3300/0 conn(0x7f2d7406c260 0x7f2d74158890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.946+0000 7f2d78cc6640 1 -- 172.21.15.164:0/3834633004 >> v1:172.21.15.139:6789/0 conn(0x7f2d740cd680 legacy=0x7f2d74162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.949 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.946+0000 7f2d78cc6640 1 -- 172.21.15.164:0/3834633004 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d740c9e00 con 0x7f2d740cd2a0 2024-09-16T00:07:11.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.947+0000 7f2d78cc6640 1 --2- 172.21.15.164:0/3834633004 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d740cd2a0 0x7f2d7406bd20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d680098a0 tx=0x7f2d6802a320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dcb115586932a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:11.950 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.947+0000 7f2d737fe640 1 -- 172.21.15.164:0/3834633004 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68037070 con 0x7f2d740cd2a0 2024-09-16T00:07:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.947+0000 7f2d737fe640 1 -- 172.21.15.164:0/3834633004 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d68032640 con 0x7f2d740cd2a0 2024-09-16T00:07:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.947+0000 7f2d737fe640 1 -- 172.21.15.164:0/3834633004 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68032a10 con 0x7f2d740cd2a0 2024-09-16T00:07:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.948+0000 7f2d7af2a640 1 -- 172.21.15.164:0/3834633004 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d740cd2a0 msgr2=0x7f2d7406bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.948+0000 7f2d7af2a640 1 --2- 172.21.15.164:0/3834633004 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d740cd2a0 0x7f2d7406bd20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2d680098a0 tx=0x7f2d6802a320 comp rx=0 tx=0).stop 2024-09-16T00:07:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.948+0000 7f2d7af2a640 1 -- 172.21.15.164:0/3834633004 shutdown_connections 2024-09-16T00:07:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.948+0000 7f2d7af2a640 1 --2- 172.21.15.164:0/3834633004 >> v2:172.21.15.139:3300/0 conn(0x7f2d7406c260 0x7f2d74158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.948+0000 7f2d7af2a640 1 --2- 172.21.15.164:0/3834633004 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d740cd2a0 0x7f2d7406bd20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.948+0000 7f2d7af2a640 1 -- 172.21.15.164:0/3834633004 >> 172.21.15.164:0/3834633004 conn(0x7f2d7406cc30 msgr2=0x7f2d7406d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.948+0000 7f2d7af2a640 1 -- 172.21.15.164:0/3834633004 shutdown_connections 2024-09-16T00:07:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.948+0000 7f2d7af2a640 1 -- 172.21.15.164:0/3834633004 wait complete. 2024-09-16T00:07:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.949+0000 7f2d7af2a640 1 Processor -- start 2024-09-16T00:07:11.951 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.949+0000 7f2d7af2a640 1 -- start start 2024-09-16T00:07:11.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.950+0000 7f2d7af2a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d7406c260 0x7f2d741f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.950+0000 7f2d7af2a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d740cd2a0 0x7f2d740caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.953 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.950+0000 7f2d7af2a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d740cd680 0x7f2d741e79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.950+0000 7f2d7af2a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2d74162860 con 0x7f2d740cd2a0 2024-09-16T00:07:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.950+0000 7f2d7af2a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2d740ca3c0 con 0x7f2d7406c260 2024-09-16T00:07:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.950+0000 7f2d7af2a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2d741629e0 con 0x7f2d740cd680 2024-09-16T00:07:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.950+0000 7f2d794c7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d740cd680 0x7f2d741e79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.950+0000 7f2d794c7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d740cd680 0x7f2d741e79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.164:39674/0 (socket says 172.21.15.164:39674) 2024-09-16T00:07:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.950+0000 7f2d78cc6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d7406c260 0x7f2d741f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.950+0000 7f2d794c7640 1 -- 172.21.15.164:0/853502316 learned_addr learned my addr 172.21.15.164:0/853502316 (peer_addr_for_me v2:172.21.15.164:0/0) 2024-09-16T00:07:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.950+0000 7f2d73fff640 1 --2- 172.21.15.164:0/853502316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d740cd2a0 0x7f2d740caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:11.954 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.950+0000 7f2d794c7640 1 -- 172.21.15.164:0/853502316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d7406c260 msgr2=0x7f2d741f7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.950+0000 7f2d794c7640 1 --2- 172.21.15.164:0/853502316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d7406c260 0x7f2d741f7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.950+0000 7f2d794c7640 1 -- 172.21.15.164:0/853502316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d740cd2a0 msgr2=0x7f2d740caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.950+0000 7f2d794c7640 1 --2- 172.21.15.164:0/853502316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d740cd2a0 0x7f2d740caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.951+0000 7f2d794c7640 1 -- 172.21.15.164:0/853502316 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d741f5170 con 0x7f2d740cd680 2024-09-16T00:07:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.951+0000 7f2d794c7640 1 --2- 172.21.15.164:0/853502316 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d740cd680 0x7f2d741e79e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00efc0 tx=0x7f2d6c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.951+0000 7f2d717fa640 1 -- 172.21.15.164:0/853502316 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c018020 con 0x7f2d740cd680 2024-09-16T00:07:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.951+0000 7f2d717fa640 1 -- 172.21.15.164:0/853502316 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6c014480 con 0x7f2d740cd680 2024-09-16T00:07:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.951+0000 7f2d7af2a640 1 -- 172.21.15.164:0/853502316 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d741f5450 con 0x7f2d740cd680 2024-09-16T00:07:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.951+0000 7f2d717fa640 1 -- 172.21.15.164:0/853502316 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c014870 con 0x7f2d740cd680 2024-09-16T00:07:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.951+0000 7f2d7af2a640 1 -- 172.21.15.164:0/853502316 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d74161ba0 con 0x7f2d740cd680 2024-09-16T00:07:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.951+0000 7f2d717fa640 1 -- 172.21.15.164:0/853502316 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2d6c005d10 con 0x7f2d740cd680 2024-09-16T00:07:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.951+0000 7f2d717fa640 1 -- 172.21.15.164:0/853502316 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f2d6c013a40 con 0x7f2d740cd680 2024-09-16T00:07:11.955 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.952+0000 7f2d7af2a640 1 -- 172.21.15.164:0/853502316 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d740cc780 con 0x7f2d740cd680 2024-09-16T00:07:11.956 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:11.954+0000 7f2d717fa640 1 -- 172.21.15.164:0/853502316 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f2d6c022030 con 0x7f2d740cd680 2024-09-16T00:07:12.024 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:12.023+0000 7f2d7af2a640 1 -- 172.21.15.164:0/853502316 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd new", "uuid": "50bcd808-3d3e-4a72-88e4-30decb53cfea", "id": 7} v 0) -- 0x7f2d740c7cb0 con 0x7f2d740cd680 2024-09-16T00:07:12.033 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:12.031+0000 7f2d717fa640 1 -- 172.21.15.164:0/853502316 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "50bcd808-3d3e-4a72-88e4-30decb53cfea", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f2d6c010040 con 0x7f2d740cd680 2024-09-16T00:07:12.033 INFO:teuthology.orchestra.run.smithi164.stdout:7 2024-09-16T00:07:12.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:12.032+0000 7f2d7af2a640 1 -- 172.21.15.164:0/853502316 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d740cd680 msgr2=0x7f2d741e79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:12.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:12.032+0000 7f2d7af2a640 1 --2- 172.21.15.164:0/853502316 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d740cd680 0x7f2d741e79e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00efc0 tx=0x7f2d6c00c490 comp rx=0 tx=0).stop 2024-09-16T00:07:12.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:12.033+0000 7f2d7af2a640 1 -- 172.21.15.164:0/853502316 shutdown_connections 2024-09-16T00:07:12.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:12.033+0000 7f2d7af2a640 1 --2- 172.21.15.164:0/853502316 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2d740cd680 0x7f2d741e79e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:12.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:12.033+0000 7f2d7af2a640 1 --2- 172.21.15.164:0/853502316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2d740cd2a0 0x7f2d740caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:12.034 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:12.033+0000 7f2d7af2a640 1 --2- 172.21.15.164:0/853502316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2d7406c260 0x7f2d741f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:12.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:12.033+0000 7f2d7af2a640 1 -- 172.21.15.164:0/853502316 >> 172.21.15.164:0/853502316 conn(0x7f2d7406cc30 msgr2=0x7f2d7406f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:12.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:12.033+0000 7f2d7af2a640 1 -- 172.21.15.164:0/853502316 shutdown_connections 2024-09-16T00:07:12.035 INFO:teuthology.orchestra.run.smithi164.stderr:2024-09-16T00:07:12.033+0000 7f2d7af2a640 1 -- 172.21.15.164:0/853502316 wait complete. 2024-09-16T00:07:12.044 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T00:07:12.044 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T00:07:12.047 INFO:tasks.ceph.osd.0:Started 2024-09-16T00:07:12.047 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T00:07:12.047 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T00:07:12.047 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:12.045+0000 7f399452b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:07:12.049 INFO:tasks.ceph.osd.1:Started 2024-09-16T00:07:12.049 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T00:07:12.049 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T00:07:12.051 INFO:tasks.ceph.osd.2:Started 2024-09-16T00:07:12.051 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-16T00:07:12.051 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-16T00:07:12.053 INFO:tasks.ceph.osd.3:Started 2024-09-16T00:07:12.053 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-16T00:07:12.054 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-16T00:07:12.056 INFO:tasks.ceph.osd.4:Started 2024-09-16T00:07:12.056 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-16T00:07:12.056 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-16T00:07:12.058 INFO:tasks.ceph.osd.5:Started 2024-09-16T00:07:12.058 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-16T00:07:12.058 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-16T00:07:12.060 INFO:tasks.ceph.osd.6:Started 2024-09-16T00:07:12.060 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-16T00:07:12.061 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-16T00:07:12.063 INFO:tasks.ceph.osd.7:Started 2024-09-16T00:07:12.063 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:07:12.132 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:12.130+0000 7f399452b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:07:12.147 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:12.145+0000 7f7123673040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:07:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.177+0000 7fde3aa8d640 1 Processor -- start 2024-09-16T00:07:12.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.177+0000 7fde3aa8d640 1 -- start start 2024-09-16T00:07:12.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.178+0000 7fde3aa8d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fde340c67e0 0x7fde340c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:12.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.178+0000 7fde3aa8d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fde3406c110 0x7fde3415a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:12.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.178+0000 7fde3aa8d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde3406ca30 con 0x7fde3406c560 2024-09-16T00:07:12.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.178+0000 7fde3aa8d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fde3406cd10 con 0x7fde340c67e0 2024-09-16T00:07:12.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.178+0000 7fde3aa8d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fde340c6bc0 con 0x7fde3406c110 2024-09-16T00:07:12.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.178+0000 7fde38829640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fde340c67e0 0x7fde340c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:12.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.178+0000 7fde3902a640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fde3406c560 0x7fde340cbe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:55610/0 (socket says 172.21.15.139:55610) 2024-09-16T00:07:12.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.178+0000 7fde3902a640 1 -- 172.21.15.139:0/3833804543 learned_addr learned my addr 172.21.15.139:0/3833804543 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:12.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.178+0000 7fde33fff640 1 --2- 172.21.15.139:0/3833804543 >> v2:172.21.15.164:3300/0 conn(0x7fde3406c110 0x7fde3415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:12.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.178+0000 7fde337fe640 1 -- 172.21.15.139:0/3833804543 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1228485339 0 0) 0x7fde3406ca30 con 0x7fde3406c560 2024-09-16T00:07:12.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.178+0000 7fde337fe640 1 -- 172.21.15.139:0/3833804543 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde24003520 con 0x7fde3406c560 2024-09-16T00:07:12.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.178+0000 7fde33fff640 1 -- 172.21.15.139:0/3833804543 >> v2:172.21.15.139:3300/0 conn(0x7fde340c67e0 msgr2=0x7fde340c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:12.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.178+0000 7fde33fff640 1 --2- 172.21.15.139:0/3833804543 >> v2:172.21.15.139:3300/0 conn(0x7fde340c67e0 0x7fde340c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:12.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.178+0000 7fde33fff640 1 -- 172.21.15.139:0/3833804543 >> v1:172.21.15.139:6789/0 conn(0x7fde3406c560 legacy=0x7fde340cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:12.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.178+0000 7fde33fff640 1 -- 172.21.15.139:0/3833804543 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3415df20 con 0x7fde3406c110 2024-09-16T00:07:12.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.178+0000 7fde38829640 1 --2- 172.21.15.139:0/3833804543 >> v2:172.21.15.139:3300/0 conn(0x7fde340c67e0 0x7fde340c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:12.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.179+0000 7fde33fff640 1 --2- 172.21.15.139:0/3833804543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde3406c110 0x7fde3415a810 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fde34158440 tx=0x7fde2002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6cb07025b4968fa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:12.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.179+0000 7fde337fe640 1 -- 172.21.15.139:0/3833804543 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20037070 con 0x7fde3406c110 2024-09-16T00:07:12.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.179+0000 7fde337fe640 1 -- 172.21.15.139:0/3833804543 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde20032440 con 0x7fde3406c110 2024-09-16T00:07:12.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.179+0000 7fde337fe640 1 -- 172.21.15.139:0/3833804543 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde200327f0 con 0x7fde3406c110 2024-09-16T00:07:12.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.180+0000 7fde3aa8d640 1 -- 172.21.15.139:0/3833804543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde3406c110 msgr2=0x7fde3415a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:12.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.180+0000 7fde3aa8d640 1 --2- 172.21.15.139:0/3833804543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde3406c110 0x7fde3415a810 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fde34158440 tx=0x7fde2002a230 comp rx=0 tx=0).stop 2024-09-16T00:07:12.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.180+0000 7fde3aa8d640 1 -- 172.21.15.139:0/3833804543 shutdown_connections 2024-09-16T00:07:12.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.180+0000 7fde3aa8d640 1 --2- 172.21.15.139:0/3833804543 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde3406c110 0x7fde3415a810 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:12.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.180+0000 7fde3aa8d640 1 --2- 172.21.15.139:0/3833804543 >> v2:172.21.15.139:3300/0 conn(0x7fde340c67e0 0x7fde340c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:12.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.180+0000 7fde3aa8d640 1 -- 172.21.15.139:0/3833804543 >> 172.21.15.139:0/3833804543 conn(0x7fde340d05f0 msgr2=0x7fde340d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:12.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.180+0000 7fde3aa8d640 1 -- 172.21.15.139:0/3833804543 shutdown_connections 2024-09-16T00:07:12.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.180+0000 7fde3aa8d640 1 -- 172.21.15.139:0/3833804543 wait complete. 2024-09-16T00:07:12.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.181+0000 7fde3aa8d640 1 Processor -- start 2024-09-16T00:07:12.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.181+0000 7fde3aa8d640 1 -- start start 2024-09-16T00:07:12.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.182+0000 7fde3aa8d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde3406c110 0x7fde340c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:12.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.182+0000 7fde3aa8d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde3406c560 0x7fde3414e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:12.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.182+0000 7fde3aa8d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde340c67e0 0x7fde3414ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:12.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.182+0000 7fde3aa8d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fde3415e4e0 con 0x7fde3406c110 2024-09-16T00:07:12.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.182+0000 7fde3aa8d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fde3415e850 con 0x7fde3406c560 2024-09-16T00:07:12.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.182+0000 7fde3aa8d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fde340cc560 con 0x7fde340c67e0 2024-09-16T00:07:12.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.182+0000 7fde38829640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde3406c110 0x7fde340c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:12.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.182+0000 7fde38829640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde3406c110 0x7fde340c1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50830/0 (socket says 172.21.15.139:50830) 2024-09-16T00:07:12.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.182+0000 7fde38829640 1 -- 172.21.15.139:0/4083342547 learned_addr learned my addr 172.21.15.139:0/4083342547 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:12.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.182+0000 7fde33fff640 1 --2- 172.21.15.139:0/4083342547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde3406c560 0x7fde3414e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:12.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.182+0000 7fde3902a640 1 --2- 172.21.15.139:0/4083342547 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde340c67e0 0x7fde3414ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:12.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.183+0000 7fde38829640 1 -- 172.21.15.139:0/4083342547 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde340c67e0 msgr2=0x7fde3414ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:12.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.183+0000 7fde38829640 1 --2- 172.21.15.139:0/4083342547 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde340c67e0 0x7fde3414ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:12.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.183+0000 7fde38829640 1 -- 172.21.15.139:0/4083342547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde3406c560 msgr2=0x7fde3414e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:12.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.183+0000 7fde38829640 1 --2- 172.21.15.139:0/4083342547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde3406c560 0x7fde3414e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:12.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.183+0000 7fde38829640 1 -- 172.21.15.139:0/4083342547 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3415dcd0 con 0x7fde3406c110 2024-09-16T00:07:12.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.183+0000 7fde38829640 1 --2- 172.21.15.139:0/4083342547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde3406c110 0x7fde340c1cd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde2800e990 tx=0x7fde2800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:12.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.183+0000 7fde317fa640 1 -- 172.21.15.139:0/4083342547 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28017070 con 0x7fde3406c110 2024-09-16T00:07:12.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.183+0000 7fde317fa640 1 -- 172.21.15.139:0/4083342547 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde2800c9e0 con 0x7fde3406c110 2024-09-16T00:07:12.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.183+0000 7fde317fa640 1 -- 172.21.15.139:0/4083342547 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2800cdb0 con 0x7fde3406c110 2024-09-16T00:07:12.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.183+0000 7fde3aa8d640 1 -- 172.21.15.139:0/4083342547 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde340c2210 con 0x7fde3406c110 2024-09-16T00:07:12.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.183+0000 7fde3aa8d640 1 -- 172.21.15.139:0/4083342547 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde341f4100 con 0x7fde3406c110 2024-09-16T00:07:12.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.183+0000 7fde317fa640 1 -- 172.21.15.139:0/4083342547 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fde2801a070 con 0x7fde3406c110 2024-09-16T00:07:12.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.183+0000 7fde317fa640 1 -- 172.21.15.139:0/4083342547 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fde28009400 con 0x7fde3406c110 2024-09-16T00:07:12.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.184+0000 7fde3aa8d640 1 -- 172.21.15.139:0/4083342547 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde340cc130 con 0x7fde3406c110 2024-09-16T00:07:12.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.186+0000 7fde317fa640 1 -- 172.21.15.139:0/4083342547 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fde280064d0 con 0x7fde3406c110 2024-09-16T00:07:12.208 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:12.207+0000 7f7123673040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:07:12.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.243+0000 7fde3aa8d640 1 -- 172.21.15.139:0/4083342547 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde34075530 con 0x7fde3406c110 2024-09-16T00:07:12.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.243+0000 7fde317fa640 1 -- 172.21.15.139:0/4083342547 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fde280066b0 con 0x7fde3406c110 2024-09-16T00:07:12.245 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:12.246 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":10,"fsid":"2d3366e5-6a59-4b59-93b2-615e72ac7015","created":"2024-09-16T00:07:09.967833+0000","modified":"2024-09-16T00:07:12.025620+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T00:07:12.025620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"83699a05-4937-43c3-aefc-b669bed9bb6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9c04991e-9695-4bbb-8011-bf613bf404b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8048fc64-72b3-4866-ac62-6a8b2bf6f5c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c34ac77b-354c-4884-902d-3e2a84f5eb27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ba9a7c93-9672-43e1-8c98-2878cdf7f548","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"46a0916b-474c-411b-84ad-86f27d4cbf53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9b6b33cc-d6d9-4a0a-8143-526073a58b93","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"50bcd808-3d3e-4a72-88e4-30decb53cfea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:07:12.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.244+0000 7fde3aa8d640 1 -- 172.21.15.139:0/4083342547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde3406c110 msgr2=0x7fde340c1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:12.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.244+0000 7fde3aa8d640 1 --2- 172.21.15.139:0/4083342547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde3406c110 0x7fde340c1cd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde2800e990 tx=0x7fde2800ee50 comp rx=0 tx=0).stop 2024-09-16T00:07:12.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.244+0000 7fde3aa8d640 1 -- 172.21.15.139:0/4083342547 shutdown_connections 2024-09-16T00:07:12.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.244+0000 7fde3aa8d640 1 --2- 172.21.15.139:0/4083342547 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fde340c67e0 0x7fde3414ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:12.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.244+0000 7fde3aa8d640 1 --2- 172.21.15.139:0/4083342547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fde3406c560 0x7fde3414e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:12.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.244+0000 7fde3aa8d640 1 --2- 172.21.15.139:0/4083342547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde3406c110 0x7fde340c1cd0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:12.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.244+0000 7fde3aa8d640 1 -- 172.21.15.139:0/4083342547 >> 172.21.15.139:0/4083342547 conn(0x7fde340d05f0 msgr2=0x7fde340c45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:12.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.244+0000 7fde3aa8d640 1 -- 172.21.15.139:0/4083342547 shutdown_connections 2024-09-16T00:07:12.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:12.244+0000 7fde3aa8d640 1 -- 172.21.15.139:0/4083342547 wait complete. 2024-09-16T00:07:12.257 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T00:07:12.257 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T00:07:12.509 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:12.507+0000 7f399452b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:07:12.577 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:12.575+0000 7f7123673040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:07:12.592 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:12.590+0000 7f399452b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:07:12.661 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:12.659+0000 7f7123673040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:07:12.698 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:12.696+0000 7f399452b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:07:12.745 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:12.743+0000 7f399452b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:07:12.768 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:12.767+0000 7f7123673040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:07:12.814 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:12.813+0000 7f7123673040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:07:12.856 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:12.854+0000 7f399452b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:07:12.908 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:12.906+0000 7f399452b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:07:12.926 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:12.925+0000 7f7123673040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:07:12.953 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:12.951+0000 7f399452b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:07:12.981 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:12.980+0000 7f7123673040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:07:13.028 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:13.026+0000 7f7123673040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:07:13.037 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:07:13.035+0000 7fe16df89640 -1 Falling back to public interface 2024-09-16T00:07:13.064 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T00:07:13.067 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:13.065+0000 7f399452b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:07:13.120 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:13.118+0000 7f399452b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:07:13.141 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:13.139+0000 7f7123673040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:07:13.156 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.154+0000 7f55c4f31640 1 Processor -- start 2024-09-16T00:07:13.156 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.154+0000 7f55c4f31640 1 -- start start 2024-09-16T00:07:13.158 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.156+0000 7f55c4f31640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55c00b2f40 0x7f55c00c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:13.160 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.156+0000 7f55c4f31640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f55c00cb860 0x7f55c00d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:13.160 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.156+0000 7f55c4f31640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55c0072ff0 con 0x7f55c01549f0 2024-09-16T00:07:13.160 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.156+0000 7f55c4f31640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f55c00cbf90 con 0x7f55c00cb860 2024-09-16T00:07:13.160 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.156+0000 7f55c4f31640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f55c00732d0 con 0x7f55c00b2f40 2024-09-16T00:07:13.161 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.156+0000 7f55bed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f55c00cb860 0x7f55c00d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:13.161 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.156+0000 7f55be59c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f55c01549f0 0x7f55c00cc420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:55620/0 (socket says 172.21.15.139:55620) 2024-09-16T00:07:13.161 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.156+0000 7f55be59c640 1 -- 172.21.15.139:0/1477530878 learned_addr learned my addr 172.21.15.139:0/1477530878 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:13.161 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.156+0000 7f55bdd9b640 1 --2- 172.21.15.139:0/1477530878 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55c00b2f40 0x7f55c00c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:13.161 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.156+0000 7f55bed9d640 1 -- 172.21.15.139:0/1477530878 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55c00b2f40 msgr2=0x7f55c00c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:13.161 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.156+0000 7f55bed9d640 1 --2- 172.21.15.139:0/1477530878 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55c00b2f40 0x7f55c00c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:13.161 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.156+0000 7f55bed9d640 1 -- 172.21.15.139:0/1477530878 >> v1:172.21.15.139:6789/0 conn(0x7f55c01549f0 legacy=0x7f55c00cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:13.161 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.156+0000 7f55bed9d640 1 -- 172.21.15.139:0/1477530878 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c00cafa0 con 0x7f55c00cb860 2024-09-16T00:07:13.161 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.156+0000 7f55bd59a640 1 -- 172.21.15.139:0/1477530878 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1555177786 0 0) 0x7f55c0072ff0 con 0x7f55c01549f0 2024-09-16T00:07:13.162 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.157+0000 7f55bed9d640 1 --2- 172.21.15.139:0/1477530878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55c00cb860 0x7f55c00d07a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55c00cc110 tx=0x7f55b802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6979ca114f9cd059 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:13.163 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.157+0000 7f55bd59a640 1 -- 172.21.15.139:0/1477530878 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8007740 con 0x7f55c00cb860 2024-09-16T00:07:13.163 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.157+0000 7f55bd59a640 1 -- 172.21.15.139:0/1477530878 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b80078e0 con 0x7f55c00cb860 2024-09-16T00:07:13.163 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.157+0000 7f55bd59a640 1 -- 172.21.15.139:0/1477530878 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8007cb0 con 0x7f55c00cb860 2024-09-16T00:07:13.163 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.158+0000 7f55c4f31640 1 -- 172.21.15.139:0/1477530878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55c00cb860 msgr2=0x7f55c00d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:13.163 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.158+0000 7f55c4f31640 1 --2- 172.21.15.139:0/1477530878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55c00cb860 0x7f55c00d07a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f55c00cc110 tx=0x7f55b802a6d0 comp rx=0 tx=0).stop 2024-09-16T00:07:13.163 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.158+0000 7f55c4f31640 1 -- 172.21.15.139:0/1477530878 shutdown_connections 2024-09-16T00:07:13.163 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.158+0000 7f55c4f31640 1 --2- 172.21.15.139:0/1477530878 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55c00cb860 0x7f55c00d07a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:13.163 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.158+0000 7f55c4f31640 1 --2- 172.21.15.139:0/1477530878 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55c00b2f40 0x7f55c00c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:13.163 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.158+0000 7f55c4f31640 1 -- 172.21.15.139:0/1477530878 >> 172.21.15.139:0/1477530878 conn(0x7f55c006d2b0 msgr2=0x7f55c006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:13.163 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.159+0000 7f55c4f31640 1 -- 172.21.15.139:0/1477530878 shutdown_connections 2024-09-16T00:07:13.163 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.159+0000 7f55c4f31640 1 -- 172.21.15.139:0/1477530878 wait complete. 2024-09-16T00:07:13.163 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.160+0000 7f55c4f31640 1 Processor -- start 2024-09-16T00:07:13.164 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.161+0000 7f55c4f31640 1 -- start start 2024-09-16T00:07:13.166 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55c4f31640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55c00b2f40 0x7f55c0150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:13.166 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55c4f31640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55c00cb860 0x7f55c01f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:13.166 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55c4f31640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55c01549f0 0x7f55c0153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:13.166 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55c4f31640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f55c0154dd0 con 0x7f55c00cb860 2024-09-16T00:07:13.166 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55c4f31640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f55c00cb560 con 0x7f55c00b2f40 2024-09-16T00:07:13.166 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55c4f31640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f55c0154f50 con 0x7f55c01549f0 2024-09-16T00:07:13.166 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55bdd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55c00cb860 0x7f55c01f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:13.166 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55bdd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55c00cb860 0x7f55c01f4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50854/0 (socket says 172.21.15.139:50854) 2024-09-16T00:07:13.166 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55bdd9b640 1 -- 172.21.15.139:0/2939250161 learned_addr learned my addr 172.21.15.139:0/2939250161 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:13.166 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55be59c640 1 --2- 172.21.15.139:0/2939250161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55c00b2f40 0x7f55c0150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:13.166 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55bed9d640 1 --2- 172.21.15.139:0/2939250161 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55c01549f0 0x7f55c0153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:13.167 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55bdd9b640 1 -- 172.21.15.139:0/2939250161 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55c01549f0 msgr2=0x7f55c0153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:13.167 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55bdd9b640 1 --2- 172.21.15.139:0/2939250161 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55c01549f0 0x7f55c0153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:13.167 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55bdd9b640 1 -- 172.21.15.139:0/2939250161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55c00b2f40 msgr2=0x7f55c0150080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:13.167 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55bdd9b640 1 --2- 172.21.15.139:0/2939250161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55c00b2f40 0x7f55c0150080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:13.167 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55bdd9b640 1 -- 172.21.15.139:0/2939250161 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c00b52c0 con 0x7f55c00cb860 2024-09-16T00:07:13.167 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55bdd9b640 1 --2- 172.21.15.139:0/2939250161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55c00cb860 0x7f55c01f4080 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f55b400e9b0 tx=0x7f55b400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:13.167 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.162+0000 7f55aeffd640 1 -- 172.21.15.139:0/2939250161 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4017070 con 0x7f55c00cb860 2024-09-16T00:07:13.167 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.163+0000 7f55aeffd640 1 -- 172.21.15.139:0/2939250161 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b400c9a0 con 0x7f55c00cb860 2024-09-16T00:07:13.167 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.163+0000 7f55c4f31640 1 -- 172.21.15.139:0/2939250161 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c01f45c0 con 0x7f55c00cb860 2024-09-16T00:07:13.167 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.163+0000 7f55aeffd640 1 -- 172.21.15.139:0/2939250161 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b400cd70 con 0x7f55c00cb860 2024-09-16T00:07:13.167 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.163+0000 7f55c4f31640 1 -- 172.21.15.139:0/2939250161 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c014ef20 con 0x7f55c00cb860 2024-09-16T00:07:13.167 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.163+0000 7f55aeffd640 1 -- 172.21.15.139:0/2939250161 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f55b401a070 con 0x7f55c00cb860 2024-09-16T00:07:13.167 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.163+0000 7f55aeffd640 1 -- 172.21.15.139:0/2939250161 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f55b4006610 con 0x7f55c00cb860 2024-09-16T00:07:13.168 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.164+0000 7f55c4f31640 1 -- 172.21.15.139:0/2939250161 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c015dd40 con 0x7f55c00cb860 2024-09-16T00:07:13.168 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.166+0000 7f55aeffd640 1 -- 172.21.15.139:0/2939250161 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f55b4005990 con 0x7f55c00cb860 2024-09-16T00:07:13.176 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:13.174+0000 7f399452b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:07:13.191 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:13.190+0000 7f7123673040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:07:13.226 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.224+0000 7f55c4f31640 1 -- 172.21.15.139:0/2939250161 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55c00c6200 con 0x7f55c00cb860 2024-09-16T00:07:13.227 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.225+0000 7f55aeffd640 1 -- 172.21.15.139:0/2939250161 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f55b4005b70 con 0x7f55c00cb860 2024-09-16T00:07:13.227 INFO:teuthology.misc.health.smithi139.stdout: 2024-09-16T00:07:13.227 INFO:teuthology.misc.health.smithi139.stdout:{"epoch":10,"fsid":"2d3366e5-6a59-4b59-93b2-615e72ac7015","created":"2024-09-16T00:07:09.967833+0000","modified":"2024-09-16T00:07:12.025620+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T00:07:12.025620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"83699a05-4937-43c3-aefc-b669bed9bb6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9c04991e-9695-4bbb-8011-bf613bf404b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8048fc64-72b3-4866-ac62-6a8b2bf6f5c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c34ac77b-354c-4884-902d-3e2a84f5eb27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ba9a7c93-9672-43e1-8c98-2878cdf7f548","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"46a0916b-474c-411b-84ad-86f27d4cbf53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9b6b33cc-d6d9-4a0a-8143-526073a58b93","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"50bcd808-3d3e-4a72-88e4-30decb53cfea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:07:13.228 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.226+0000 7f55c4f31640 1 -- 172.21.15.139:0/2939250161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55c00cb860 msgr2=0x7f55c01f4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:13.228 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.226+0000 7f55c4f31640 1 --2- 172.21.15.139:0/2939250161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55c00cb860 0x7f55c01f4080 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f55b400e9b0 tx=0x7f55b400ee70 comp rx=0 tx=0).stop 2024-09-16T00:07:13.228 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.226+0000 7f55c4f31640 1 -- 172.21.15.139:0/2939250161 shutdown_connections 2024-09-16T00:07:13.228 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.226+0000 7f55c4f31640 1 --2- 172.21.15.139:0/2939250161 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55c01549f0 0x7f55c0153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:13.228 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.226+0000 7f55c4f31640 1 --2- 172.21.15.139:0/2939250161 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55c00cb860 0x7f55c01f4080 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:13.228 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.226+0000 7f55c4f31640 1 --2- 172.21.15.139:0/2939250161 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55c00b2f40 0x7f55c0150080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:13.228 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.226+0000 7f55c4f31640 1 -- 172.21.15.139:0/2939250161 >> 172.21.15.139:0/2939250161 conn(0x7f55c006d2b0 msgr2=0x7f55c00c5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:13.228 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.226+0000 7f55c4f31640 1 -- 172.21.15.139:0/2939250161 shutdown_connections 2024-09-16T00:07:13.228 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:13.226+0000 7f55c4f31640 1 -- 172.21.15.139:0/2939250161 wait complete. 2024-09-16T00:07:13.237 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-16T00:07:13.243 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:13.242+0000 7f7123673040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:07:13.247 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:13.245+0000 7f399452b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:07:13.281 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:07:13.279+0000 7fa4cae32640 -1 Falling back to public interface 2024-09-16T00:07:13.294 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:13.293+0000 7f399452b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:07:13.312 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:13.311+0000 7f7123673040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:07:13.344 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:13.342+0000 7f399452b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:07:13.357 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:13.356+0000 7f7123673040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:07:13.401 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:13.399+0000 7f7123673040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:07:13.524 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:07:13.523+0000 7fe8f3aef640 -1 Falling back to public interface 2024-09-16T00:07:13.525 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:07:13.523+0000 7fd696bbf640 -1 Falling back to public interface 2024-09-16T00:07:13.529 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:07:13.528+0000 7f2660b45640 -1 Falling back to public interface 2024-09-16T00:07:13.769 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:13.767+0000 7f343549c640 -1 Falling back to public interface 2024-09-16T00:07:13.782 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:07:13.780+0000 7f04ef816640 -1 Falling back to public interface 2024-09-16T00:07:14.016 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:14.014+0000 7f399452b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:07:14.045 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:07:14.043+0000 7fe1d1945640 -1 Falling back to public interface 2024-09-16T00:07:14.063 INFO:tasks.ceph.mgr.x.smithi164.stderr:2024-09-16T00:07:14.061+0000 7f7123673040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:07:15.029 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:15.027+0000 7f3934396640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-16T00:07:15.031 INFO:tasks.ceph.mgr.y.smithi139.stderr:2024-09-16T00:07:15.030+0000 7f3934396640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-16T00:07:16.872 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:07:16.871+0000 7fa4cae32640 -1 osd.2 0 log_to_monitors true 2024-09-16T00:07:17.117 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:07:17.115+0000 7fe16df89640 -1 osd.5 0 log_to_monitors true 2024-09-16T00:07:17.340 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:07:17.339+0000 7fd696bbf640 -1 osd.7 0 log_to_monitors true 2024-09-16T00:07:17.589 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:07:17.587+0000 7f04ef816640 -1 osd.6 0 log_to_monitors true 2024-09-16T00:07:17.610 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:07:17.610+0000 7fe8f3aef640 -1 osd.1 0 log_to_monitors true 2024-09-16T00:07:17.808 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:17.807+0000 7f343549c640 -1 osd.4 0 log_to_monitors true 2024-09-16T00:07:17.853 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:07:17.852+0000 7f2660b45640 -1 osd.0 0 log_to_monitors true 2024-09-16T00:07:18.313 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:07:18.313+0000 7fe1d1945640 -1 osd.3 0 log_to_monitors true 2024-09-16T00:07:18.598 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:07:18.597+0000 7fa4c6b86640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T00:07:18.607 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:07:18.607+0000 7fa4be192640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:07:19.786 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:07:19.785+0000 7f265cad4640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T00:07:19.797 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:07:19.796+0000 7fe8efa7e640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T00:07:19.801 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:07:19.801+0000 7f26548f4640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:07:19.812 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:07:19.810+0000 7fe8e789e640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:07:20.030 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:20.028+0000 7f343142b640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T00:07:20.036 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:07:20.035+0000 7fd692b4e640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T00:07:20.045 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T00:07:20.048 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:20.046+0000 7f342924b640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:07:20.049 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:07:20.048+0000 7fd68a96e640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:07:20.058 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:07:20.056+0000 7fe16a72c640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T00:07:20.074 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:07:20.072+0000 7f04eb17a640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T00:07:20.078 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:07:20.077+0000 7fe161d38640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:07:20.086 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:07:20.085+0000 7f04e2f9a640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:07:20.171 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.170+0000 7fd426978640 1 Processor -- start 2024-09-16T00:07:20.171 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.170+0000 7fd426978640 1 -- start start 2024-09-16T00:07:20.174 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.170+0000 7fd426978640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd42015aa10 0x7fd4200c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:20.174 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.170+0000 7fd426978640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd420072370 con 0x7fd42006be80 2024-09-16T00:07:20.174 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.170+0000 7fd426978640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4200734f0 con 0x7fd42006c260 2024-09-16T00:07:20.174 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.170+0000 7fd426978640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd4200732a0 con 0x7fd42015aa10 2024-09-16T00:07:20.174 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.171+0000 7fd41ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd42006be80 0x7fd420072b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56038/0 (socket says 172.21.15.139:56038) 2024-09-16T00:07:20.174 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.171+0000 7fd41ffff640 1 -- 172.21.15.139:0/603176847 learned_addr learned my addr 172.21.15.139:0/603176847 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:20.174 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.171+0000 7fd424f15640 1 --2- 172.21.15.139:0/603176847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd42015aa10 0x7fd4200c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:20.175 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.171+0000 7fd41effd640 1 -- 172.21.15.139:0/603176847 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1292377752 0 0) 0x7fd420072370 con 0x7fd42006be80 2024-09-16T00:07:20.175 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.171+0000 7fd41effd640 1 -- 172.21.15.139:0/603176847 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd408003520 con 0x7fd42006be80 2024-09-16T00:07:20.175 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.171+0000 7fd41effd640 1 -- 172.21.15.139:0/603176847 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4231938431 0 0) 0x7fd4200734f0 con 0x7fd42006c260 2024-09-16T00:07:20.175 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.171+0000 7fd41effd640 1 -- 172.21.15.139:0/603176847 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd420072370 con 0x7fd42006c260 2024-09-16T00:07:20.175 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.171+0000 7fd424f15640 1 -- 172.21.15.139:0/603176847 >> v1:172.21.15.164:6789/0 conn(0x7fd42006c260 legacy=0x7fd420151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:20.175 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.171+0000 7fd424f15640 1 -- 172.21.15.139:0/603176847 >> v1:172.21.15.139:6789/0 conn(0x7fd42006be80 legacy=0x7fd420072b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:20.175 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.171+0000 7fd424f15640 1 -- 172.21.15.139:0/603176847 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4200737d0 con 0x7fd42015aa10 2024-09-16T00:07:20.175 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.172+0000 7fd424f15640 1 --2- 172.21.15.139:0/603176847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd42015aa10 0x7fd4200c35f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd420073420 tx=0x7fd41802a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f464ea72493eba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:20.175 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.172+0000 7fd41effd640 1 -- 172.21.15.139:0/603176847 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418007740 con 0x7fd42015aa10 2024-09-16T00:07:20.175 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.172+0000 7fd41effd640 1 -- 172.21.15.139:0/603176847 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4180078e0 con 0x7fd42015aa10 2024-09-16T00:07:20.175 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.172+0000 7fd41effd640 1 -- 172.21.15.139:0/603176847 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418007cb0 con 0x7fd42015aa10 2024-09-16T00:07:20.175 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.173+0000 7fd426978640 1 -- 172.21.15.139:0/603176847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd42015aa10 msgr2=0x7fd4200c35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:20.175 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.173+0000 7fd426978640 1 --2- 172.21.15.139:0/603176847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd42015aa10 0x7fd4200c35f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd420073420 tx=0x7fd41802a750 comp rx=0 tx=0).stop 2024-09-16T00:07:20.176 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.173+0000 7fd426978640 1 -- 172.21.15.139:0/603176847 shutdown_connections 2024-09-16T00:07:20.176 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.173+0000 7fd426978640 1 --2- 172.21.15.139:0/603176847 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd42015aa10 0x7fd4200c35f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:20.176 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.173+0000 7fd426978640 1 -- 172.21.15.139:0/603176847 >> 172.21.15.139:0/603176847 conn(0x7fd4200cc0f0 msgr2=0x7fd4200cc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:20.176 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.173+0000 7fd426978640 1 -- 172.21.15.139:0/603176847 shutdown_connections 2024-09-16T00:07:20.176 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.173+0000 7fd426978640 1 -- 172.21.15.139:0/603176847 wait complete. 2024-09-16T00:07:20.179 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.174+0000 7fd426978640 1 Processor -- start 2024-09-16T00:07:20.179 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.175+0000 7fd426978640 1 -- start start 2024-09-16T00:07:20.179 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.175+0000 7fd426978640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd42006be80 0x7fd4201476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:20.180 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.175+0000 7fd426978640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd42006c260 0x7fd420147c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:20.180 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.175+0000 7fd426978640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd42015aa10 0x7fd42014bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:20.180 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.175+0000 7fd426978640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd420072630 con 0x7fd42015aa10 2024-09-16T00:07:20.180 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.175+0000 7fd426978640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd420073d90 con 0x7fd42006c260 2024-09-16T00:07:20.180 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.175+0000 7fd426978640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd420071b60 con 0x7fd42006be80 2024-09-16T00:07:20.180 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.175+0000 7fd424f15640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd42015aa10 0x7fd42014bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:20.180 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.175+0000 7fd424f15640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd42015aa10 0x7fd42014bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40082/0 (socket says 172.21.15.139:40082) 2024-09-16T00:07:20.180 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.175+0000 7fd424f15640 1 -- 172.21.15.139:0/2233367215 learned_addr learned my addr 172.21.15.139:0/2233367215 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:20.180 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.175+0000 7fd41ffff640 1 --2- 172.21.15.139:0/2233367215 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd42006be80 0x7fd4201476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:20.180 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.176+0000 7fd41f7fe640 1 --2- 172.21.15.139:0/2233367215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd42006c260 0x7fd420147c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:20.180 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.176+0000 7fd424f15640 1 -- 172.21.15.139:0/2233367215 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd42006be80 msgr2=0x7fd4201476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:20.181 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.176+0000 7fd424f15640 1 --2- 172.21.15.139:0/2233367215 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd42006be80 0x7fd4201476e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:20.181 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.176+0000 7fd424f15640 1 -- 172.21.15.139:0/2233367215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd42006c260 msgr2=0x7fd420147c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:20.181 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.176+0000 7fd424f15640 1 --2- 172.21.15.139:0/2233367215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd42006c260 0x7fd420147c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:20.181 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.176+0000 7fd424f15640 1 -- 172.21.15.139:0/2233367215 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd420148190 con 0x7fd42015aa10 2024-09-16T00:07:20.181 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.176+0000 7fd41ffff640 1 --2- 172.21.15.139:0/2233367215 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd42006be80 0x7fd4201476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:20.181 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.176+0000 7fd424f15640 1 --2- 172.21.15.139:0/2233367215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd42015aa10 0x7fd42014bd40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd41802ad40 tx=0x7fd418004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:20.181 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.176+0000 7fd41cff9640 1 -- 172.21.15.139:0/2233367215 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41803f070 con 0x7fd42015aa10 2024-09-16T00:07:20.181 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.176+0000 7fd41cff9640 1 -- 172.21.15.139:0/2233367215 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd418031bc0 con 0x7fd42015aa10 2024-09-16T00:07:20.181 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.176+0000 7fd426978640 1 -- 172.21.15.139:0/2233367215 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd420148450 con 0x7fd42015aa10 2024-09-16T00:07:20.181 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.176+0000 7fd41cff9640 1 -- 172.21.15.139:0/2233367215 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418009450 con 0x7fd42015aa10 2024-09-16T00:07:20.181 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.176+0000 7fd426978640 1 -- 172.21.15.139:0/2233367215 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd420146ee0 con 0x7fd42015aa10 2024-09-16T00:07:20.182 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.177+0000 7fd426978640 1 -- 172.21.15.139:0/2233367215 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4200c6170 con 0x7fd42015aa10 2024-09-16T00:07:20.182 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.178+0000 7fd41cff9640 1 -- 172.21.15.139:0/2233367215 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 4) ==== 90790+0+0 (secure 0 0 0) 0x7fd418044460 con 0x7fd42015aa10 2024-09-16T00:07:20.182 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.178+0000 7fd41cff9640 1 --2- 172.21.15.139:0/2233367215 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd40c06c250 0x7fd40c06e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:20.182 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.178+0000 7fd41cff9640 1 -- 172.21.15.139:0/2233367215 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3071+0+0 (secure 0 0 0) 0x7fd4180abd40 con 0x7fd42015aa10 2024-09-16T00:07:20.182 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.178+0000 7fd41ffff640 1 --2- 172.21.15.139:0/2233367215 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd40c06c250 0x7fd40c06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:20.182 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.179+0000 7fd41ffff640 1 --2- 172.21.15.139:0/2233367215 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd40c06c250 0x7fd40c06e710 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd410000f60 tx=0x7fd410007d70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:20.184 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.184+0000 7fd41cff9640 1 -- 172.21.15.139:0/2233367215 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd418049060 con 0x7fd42015aa10 2024-09-16T00:07:20.378 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.377+0000 7fd426978640 1 -- 172.21.15.139:0/2233367215 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4200c32e0 con 0x7fd42015aa10 2024-09-16T00:07:20.379 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.378+0000 7fd41cff9640 1 -- 172.21.15.139:0/2233367215 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+8562 (secure 0 0 0) 0x7fd41807aa20 con 0x7fd42015aa10 2024-09-16T00:07:20.379 INFO:teuthology.misc.health.smithi139.stdout: 2024-09-16T00:07:20.379 INFO:teuthology.misc.health.smithi139.stdout:{"epoch":14,"fsid":"2d3366e5-6a59-4b59-93b2-615e72ac7015","created":"2024-09-16T00:07:09.967833+0000","modified":"2024-09-16T00:07:20.040336+0000","last_up_change":"2024-09-16T00:07:20.040336+0000","last_in_change":"2024-09-16T00:07:12.025620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"83699a05-4937-43c3-aefc-b669bed9bb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6817","nonce":1974901438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6819","nonce":1974901438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6823","nonce":1974901438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6821","nonce":1974901438}]},"public_addr":"172.21.15.139:6817/1974901438","cluster_addr":"172.21.15.139:6819/1974901438","heartbeat_back_addr":"172.21.15.139:6823/1974901438","heartbeat_front_addr":"172.21.15.139:6821/1974901438","state":["exists","up"]},{"osd":1,"uuid":"9c04991e-9695-4bbb-8011-bf613bf404b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6809","nonce":2232380998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6811","nonce":2232380998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6815","nonce":2232380998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6813","nonce":2232380998}]},"public_addr":"172.21.15.139:6809/2232380998","cluster_addr":"172.21.15.139:6811/2232380998","heartbeat_back_addr":"172.21.15.139:6815/2232380998","heartbeat_front_addr":"172.21.15.139:6813/2232380998","state":["exists","up"]},{"osd":2,"uuid":"8048fc64-72b3-4866-ac62-6a8b2bf6f5c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6801","nonce":4180588905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6803","nonce":4180588905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6807","nonce":4180588905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6805","nonce":4180588905}]},"public_addr":"172.21.15.139:6801/4180588905","cluster_addr":"172.21.15.139:6803/4180588905","heartbeat_back_addr":"172.21.15.139:6807/4180588905","heartbeat_front_addr":"172.21.15.139:6805/4180588905","state":["exists","up"]},{"osd":3,"uuid":"c34ac77b-354c-4884-902d-3e2a84f5eb27","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ba9a7c93-9672-43e1-8c98-2878cdf7f548","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"46a0916b-474c-411b-84ad-86f27d4cbf53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9b6b33cc-d6d9-4a0a-8143-526073a58b93","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"50bcd808-3d3e-4a72-88e4-30decb53cfea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:17.919457+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:07:20.382 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.381+0000 7fd426978640 1 -- 172.21.15.139:0/2233367215 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd40c06c250 msgr2=0x7fd40c06e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:20.382 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.381+0000 7fd426978640 1 --2- 172.21.15.139:0/2233367215 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd40c06c250 0x7fd40c06e710 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd410000f60 tx=0x7fd410007d70 comp rx=0 tx=0).stop 2024-09-16T00:07:20.382 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.381+0000 7fd426978640 1 -- 172.21.15.139:0/2233367215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd42015aa10 msgr2=0x7fd42014bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:20.382 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.381+0000 7fd426978640 1 --2- 172.21.15.139:0/2233367215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd42015aa10 0x7fd42014bd40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd41802ad40 tx=0x7fd418004a40 comp rx=0 tx=0).stop 2024-09-16T00:07:20.382 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.381+0000 7fd426978640 1 -- 172.21.15.139:0/2233367215 shutdown_connections 2024-09-16T00:07:20.382 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.381+0000 7fd426978640 1 --2- 172.21.15.139:0/2233367215 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd40c06c250 0x7fd40c06e710 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:20.382 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.381+0000 7fd426978640 1 --2- 172.21.15.139:0/2233367215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd42015aa10 0x7fd42014bd40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:20.382 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.381+0000 7fd426978640 1 --2- 172.21.15.139:0/2233367215 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd42006c260 0x7fd420147c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:20.382 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.381+0000 7fd426978640 1 --2- 172.21.15.139:0/2233367215 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd42006be80 0x7fd4201476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:20.383 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.381+0000 7fd426978640 1 -- 172.21.15.139:0/2233367215 >> 172.21.15.139:0/2233367215 conn(0x7fd4200cc0f0 msgr2=0x7fd4200cc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:20.383 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.381+0000 7fd426978640 1 -- 172.21.15.139:0/2233367215 shutdown_connections 2024-09-16T00:07:20.383 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:20.381+0000 7fd426978640 1 -- 172.21.15.139:0/2233367215 wait complete. 2024-09-16T00:07:20.393 DEBUG:teuthology.misc:3 of 8 OSDs are up 2024-09-16T00:07:20.674 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:07:20.673+0000 7fe1cd8d4640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T00:07:20.683 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:07:20.683+0000 7fe1c56f4640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:07:27.201 DEBUG:teuthology.orchestra.run.smithi139:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T00:07:27.304 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.303+0000 7f397e92a640 1 Processor -- start 2024-09-16T00:07:27.304 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.303+0000 7f397e92a640 1 -- start start 2024-09-16T00:07:27.307 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.304+0000 7f397e92a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f397806be80 0x7f39780c35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:27.307 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.304+0000 7f397e92a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f397806c260 0x7f3978072b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:27.307 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.304+0000 7f397e92a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f397815aa10 0x7f3978151310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:27.307 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.304+0000 7f397e92a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f39780732a0 con 0x7f397806c260 2024-09-16T00:07:27.307 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.304+0000 7f397e92a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f3978073090 con 0x7f397815aa10 2024-09-16T00:07:27.308 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.304+0000 7f397e92a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3978072370 con 0x7f397806be80 2024-09-16T00:07:27.308 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.304+0000 7f39777fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f397806c260 0x7f3978072b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:27.308 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.304+0000 7f39777fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f397806c260 0x7f3978072b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46316/0 (socket says 172.21.15.139:46316) 2024-09-16T00:07:27.308 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.304+0000 7f39777fe640 1 -- 172.21.15.139:0/917993110 learned_addr learned my addr 172.21.15.139:0/917993110 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:27.308 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.304+0000 7f39777fe640 1 -- 172.21.15.139:0/917993110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f397806be80 msgr2=0x7f39780c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:27.308 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.304+0000 7f39777fe640 1 --2- 172.21.15.139:0/917993110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f397806be80 0x7f39780c35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.308 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.304+0000 7f39777fe640 1 -- 172.21.15.139:0/917993110 >> v2:172.21.15.164:3300/0 conn(0x7f397815aa10 msgr2=0x7f3978151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:27.308 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.304+0000 7f397cec7640 1 --2- 172.21.15.139:0/917993110 >> v2:172.21.15.164:3300/0 conn(0x7f397815aa10 0x7f3978151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:27.308 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.304+0000 7f39777fe640 1 --2- 172.21.15.139:0/917993110 >> v2:172.21.15.164:3300/0 conn(0x7f397815aa10 0x7f3978151310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.308 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.304+0000 7f39777fe640 1 -- 172.21.15.139:0/917993110 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39780734f0 con 0x7f397806c260 2024-09-16T00:07:27.308 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.305+0000 7f39777fe640 1 --2- 172.21.15.139:0/917993110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397806c260 0x7f3978072b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f397806c640 tx=0x7f396c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481ef2b6af26dcd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:27.308 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.305+0000 7f3976ffd640 1 -- 172.21.15.139:0/917993110 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c037070 con 0x7f397806c260 2024-09-16T00:07:27.309 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.305+0000 7f3976ffd640 1 -- 172.21.15.139:0/917993110 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f396c032440 con 0x7f397806c260 2024-09-16T00:07:27.309 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.305+0000 7f3976ffd640 1 -- 172.21.15.139:0/917993110 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c032810 con 0x7f397806c260 2024-09-16T00:07:27.309 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.306+0000 7f397e92a640 1 -- 172.21.15.139:0/917993110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397806c260 msgr2=0x7f3978072b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:27.309 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.306+0000 7f397e92a640 1 --2- 172.21.15.139:0/917993110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397806c260 0x7f3978072b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f397806c640 tx=0x7f396c02a270 comp rx=0 tx=0).stop 2024-09-16T00:07:27.309 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.306+0000 7f397e92a640 1 -- 172.21.15.139:0/917993110 shutdown_connections 2024-09-16T00:07:27.309 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.306+0000 7f397e92a640 1 --2- 172.21.15.139:0/917993110 >> v2:172.21.15.164:3300/0 conn(0x7f397815aa10 0x7f3978151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.310 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.306+0000 7f397e92a640 1 --2- 172.21.15.139:0/917993110 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397806c260 0x7f3978072b50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.310 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.306+0000 7f397e92a640 1 --2- 172.21.15.139:0/917993110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f397806be80 0x7f39780c35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.310 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.306+0000 7f397e92a640 1 -- 172.21.15.139:0/917993110 >> 172.21.15.139:0/917993110 conn(0x7f39780cc0f0 msgr2=0x7f39780cc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:27.310 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.306+0000 7f397e92a640 1 -- 172.21.15.139:0/917993110 shutdown_connections 2024-09-16T00:07:27.310 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.306+0000 7f397e92a640 1 -- 172.21.15.139:0/917993110 wait complete. 2024-09-16T00:07:27.312 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.307+0000 7f397e92a640 1 Processor -- start 2024-09-16T00:07:27.313 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.307+0000 7f397e92a640 1 -- start start 2024-09-16T00:07:27.313 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f397e92a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f397806be80 0x7f3978141570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:27.313 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f397e92a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397806c260 0x7f397806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:27.313 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f397e92a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f397815aa10 0x7f3978145e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:27.313 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f397e92a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3978073d90 con 0x7f397806c260 2024-09-16T00:07:27.313 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f397e92a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3978073ab0 con 0x7f397815aa10 2024-09-16T00:07:27.313 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f397e92a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3978072630 con 0x7f397806be80 2024-09-16T00:07:27.313 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f39777fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397806c260 0x7f397806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:27.313 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f39777fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397806c260 0x7f397806f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46318/0 (socket says 172.21.15.139:46318) 2024-09-16T00:07:27.313 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f39777fe640 1 -- 172.21.15.139:0/2454917272 learned_addr learned my addr 172.21.15.139:0/2454917272 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:27.313 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f3977fff640 1 --2- 172.21.15.139:0/2454917272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f397806be80 0x7f3978141570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:27.314 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f397cec7640 1 --2- 172.21.15.139:0/2454917272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f397815aa10 0x7f3978145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:27.314 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f39777fe640 1 -- 172.21.15.139:0/2454917272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f397806be80 msgr2=0x7f3978141570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:27.314 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f39777fe640 1 --2- 172.21.15.139:0/2454917272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f397806be80 0x7f3978141570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.314 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f39777fe640 1 -- 172.21.15.139:0/2454917272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f397815aa10 msgr2=0x7f3978145e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:27.314 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f39777fe640 1 --2- 172.21.15.139:0/2454917272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f397815aa10 0x7f3978145e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.314 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f39777fe640 1 -- 172.21.15.139:0/2454917272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397815a3b0 con 0x7f397806c260 2024-09-16T00:07:27.314 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f397cec7640 1 --2- 172.21.15.139:0/2454917272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f397815aa10 0x7f3978145e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:27.314 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.308+0000 7f39777fe640 1 --2- 172.21.15.139:0/2454917272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397806c260 0x7f397806f590 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f396c02a2b0 tx=0x7f396c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:27.315 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.309+0000 7f3974ff9640 1 -- 172.21.15.139:0/2454917272 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c037040 con 0x7f397806c260 2024-09-16T00:07:27.315 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.309+0000 7f3974ff9640 1 -- 172.21.15.139:0/2454917272 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f396c042070 con 0x7f397806c260 2024-09-16T00:07:27.315 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.309+0000 7f397e92a640 1 -- 172.21.15.139:0/2454917272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39780cb9f0 con 0x7f397806c260 2024-09-16T00:07:27.315 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.309+0000 7f3974ff9640 1 -- 172.21.15.139:0/2454917272 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c030bb0 con 0x7f397806c260 2024-09-16T00:07:27.315 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.309+0000 7f397e92a640 1 -- 172.21.15.139:0/2454917272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3978145320 con 0x7f397806c260 2024-09-16T00:07:27.315 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.310+0000 7f397e92a640 1 -- 172.21.15.139:0/2454917272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39780c4860 con 0x7f397806c260 2024-09-16T00:07:27.315 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.310+0000 7f3974ff9640 1 -- 172.21.15.139:0/2454917272 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f396c047460 con 0x7f397806c260 2024-09-16T00:07:27.315 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.311+0000 7f3974ff9640 1 --2- 172.21.15.139:0/2454917272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f396406c2b0 0x7f396406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:27.315 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.311+0000 7f3977fff640 1 --2- 172.21.15.139:0/2454917272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f396406c2b0 0x7f396406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:27.315 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.311+0000 7f3974ff9640 1 -- 172.21.15.139:0/2454917272 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f396c0abd20 con 0x7f397806c260 2024-09-16T00:07:27.315 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.311+0000 7f3977fff640 1 --2- 172.21.15.139:0/2454917272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f396406c2b0 0x7f396406e770 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3968009990 tx=0x7f3968009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:27.316 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.315+0000 7f3974ff9640 1 -- 172.21.15.139:0/2454917272 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f39780c4860 con 0x7f397806c260 2024-09-16T00:07:27.483 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.482+0000 7f397e92a640 1 -- 172.21.15.139:0/2454917272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3978071b60 con 0x7f397806c260 2024-09-16T00:07:27.483 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.483+0000 7f3974ff9640 1 -- 172.21.15.139:0/2454917272 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12956 (secure 0 0 0) 0x7f396c0b0070 con 0x7f397806c260 2024-09-16T00:07:27.484 INFO:teuthology.misc.health.smithi139.stdout: 2024-09-16T00:07:27.484 INFO:teuthology.misc.health.smithi139.stdout:{"epoch":18,"fsid":"2d3366e5-6a59-4b59-93b2-615e72ac7015","created":"2024-09-16T00:07:09.967833+0000","modified":"2024-09-16T00:07:24.081531+0000","last_up_change":"2024-09-16T00:07:21.049678+0000","last_in_change":"2024-09-16T00:07:12.025620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:07:22.056303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83699a05-4937-43c3-aefc-b669bed9bb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6817","nonce":1974901438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6819","nonce":1974901438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6823","nonce":1974901438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6821","nonce":1974901438}]},"public_addr":"172.21.15.139:6817/1974901438","cluster_addr":"172.21.15.139:6819/1974901438","heartbeat_back_addr":"172.21.15.139:6823/1974901438","heartbeat_front_addr":"172.21.15.139:6821/1974901438","state":["exists","up"]},{"osd":1,"uuid":"9c04991e-9695-4bbb-8011-bf613bf404b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6809","nonce":2232380998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6811","nonce":2232380998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6815","nonce":2232380998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6813","nonce":2232380998}]},"public_addr":"172.21.15.139:6809/2232380998","cluster_addr":"172.21.15.139:6811/2232380998","heartbeat_back_addr":"172.21.15.139:6815/2232380998","heartbeat_front_addr":"172.21.15.139:6813/2232380998","state":["exists","up"]},{"osd":2,"uuid":"8048fc64-72b3-4866-ac62-6a8b2bf6f5c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6801","nonce":4180588905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6803","nonce":4180588905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6807","nonce":4180588905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6805","nonce":4180588905}]},"public_addr":"172.21.15.139:6801/4180588905","cluster_addr":"172.21.15.139:6803/4180588905","heartbeat_back_addr":"172.21.15.139:6807/4180588905","heartbeat_front_addr":"172.21.15.139:6805/4180588905","state":["exists","up"]},{"osd":3,"uuid":"c34ac77b-354c-4884-902d-3e2a84f5eb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6827","nonce":3947768492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6829","nonce":3947768492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6833","nonce":3947768492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6831","nonce":3947768492}]},"public_addr":"172.21.15.139:6827/3947768492","cluster_addr":"172.21.15.139:6829/3947768492","heartbeat_back_addr":"172.21.15.139:6833/3947768492","heartbeat_front_addr":"172.21.15.139:6831/3947768492","state":["exists","up"]},{"osd":4,"uuid":"ba9a7c93-9672-43e1-8c98-2878cdf7f548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6817","nonce":642632916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6819","nonce":642632916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6823","nonce":642632916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6821","nonce":642632916}]},"public_addr":"172.21.15.164:6817/642632916","cluster_addr":"172.21.15.164:6819/642632916","heartbeat_back_addr":"172.21.15.164:6823/642632916","heartbeat_front_addr":"172.21.15.164:6821/642632916","state":["exists","up"]},{"osd":5,"uuid":"46a0916b-474c-411b-84ad-86f27d4cbf53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6801","nonce":1051778383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6803","nonce":1051778383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6807","nonce":1051778383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6805","nonce":1051778383}]},"public_addr":"172.21.15.164:6801/1051778383","cluster_addr":"172.21.15.164:6803/1051778383","heartbeat_back_addr":"172.21.15.164:6807/1051778383","heartbeat_front_addr":"172.21.15.164:6805/1051778383","state":["exists","up"]},{"osd":6,"uuid":"9b6b33cc-d6d9-4a0a-8143-526073a58b93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6825","nonce":1908495807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6827","nonce":1908495807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6831","nonce":1908495807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6829","nonce":1908495807}]},"public_addr":"172.21.15.164:6825/1908495807","cluster_addr":"172.21.15.164:6827/1908495807","heartbeat_back_addr":"172.21.15.164:6831/1908495807","heartbeat_front_addr":"172.21.15.164:6829/1908495807","state":["exists","up"]},{"osd":7,"uuid":"50bcd808-3d3e-4a72-88e4-30decb53cfea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6809","nonce":1158838808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6811","nonce":1158838808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6815","nonce":1158838808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6813","nonce":1158838808}]},"public_addr":"172.21.15.164:6809/1158838808","cluster_addr":"172.21.15.164:6811/1158838808","heartbeat_back_addr":"172.21.15.164:6815/1158838808","heartbeat_front_addr":"172.21.15.164:6813/1158838808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.843117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.622789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:17.919457+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:19.330341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.815640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.083046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.551499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.377179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:07:27.489 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.485+0000 7f397e92a640 1 -- 172.21.15.139:0/2454917272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f396406c2b0 msgr2=0x7f396406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:27.489 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.485+0000 7f397e92a640 1 --2- 172.21.15.139:0/2454917272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f396406c2b0 0x7f396406e770 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3968009990 tx=0x7f3968009110 comp rx=0 tx=0).stop 2024-09-16T00:07:27.489 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.485+0000 7f397e92a640 1 -- 172.21.15.139:0/2454917272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397806c260 msgr2=0x7f397806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:27.489 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.485+0000 7f397e92a640 1 --2- 172.21.15.139:0/2454917272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397806c260 0x7f397806f590 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f396c02a2b0 tx=0x7f396c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.489 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.485+0000 7f397e92a640 1 -- 172.21.15.139:0/2454917272 shutdown_connections 2024-09-16T00:07:27.489 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.485+0000 7f397e92a640 1 --2- 172.21.15.139:0/2454917272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f396406c2b0 0x7f396406e770 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.489 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.485+0000 7f397e92a640 1 --2- 172.21.15.139:0/2454917272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f397815aa10 0x7f3978145e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.490 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.485+0000 7f397e92a640 1 --2- 172.21.15.139:0/2454917272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f397806c260 0x7f397806f590 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.490 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.485+0000 7f397e92a640 1 --2- 172.21.15.139:0/2454917272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f397806be80 0x7f3978141570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.490 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.485+0000 7f397e92a640 1 -- 172.21.15.139:0/2454917272 >> 172.21.15.139:0/2454917272 conn(0x7f39780cc0f0 msgr2=0x7f397814c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:27.490 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.485+0000 7f397e92a640 1 -- 172.21.15.139:0/2454917272 shutdown_connections 2024-09-16T00:07:27.490 INFO:teuthology.misc.health.smithi139.stderr:2024-09-16T00:07:27.485+0000 7f397e92a640 1 -- 172.21.15.139:0/2454917272 wait complete. 2024-09-16T00:07:27.495 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-16T00:07:27.495 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T00:07:27.496 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-16T00:07:27.496 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-16T00:07:27.498 INFO:tasks.ceph.mds.a:Started 2024-09-16T00:07:27.498 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-16T00:07:27.498 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-16T00:07:27.500 INFO:tasks.ceph.mds.c:Started 2024-09-16T00:07:27.500 INFO:tasks.ceph.mds.e:Restarting daemon 2024-09-16T00:07:27.500 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-09-16T00:07:27.502 INFO:tasks.ceph.mds.e:Started 2024-09-16T00:07:27.502 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-16T00:07:27.502 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-16T00:07:27.505 INFO:tasks.ceph.mds.b:Started 2024-09-16T00:07:27.505 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-16T00:07:27.505 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-16T00:07:27.507 INFO:tasks.ceph.mds.d:Started 2024-09-16T00:07:27.507 INFO:tasks.ceph.mds.f:Restarting daemon 2024-09-16T00:07:27.508 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-09-16T00:07:27.510 INFO:tasks.ceph.mds.f:Started 2024-09-16T00:07:27.510 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-16T00:07:27.511 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T00:07:27.575 INFO:tasks.ceph.mds.a.smithi139.stdout:starting mds.a at 2024-09-16T00:07:27.577 INFO:tasks.ceph.mds.c.smithi139.stdout:starting mds.c at 2024-09-16T00:07:27.580 INFO:tasks.ceph.mds.e.smithi139.stdout:starting mds.e at 2024-09-16T00:07:27.594 INFO:tasks.ceph.mds.b.smithi164.stdout:starting mds.b at 2024-09-16T00:07:27.594 INFO:tasks.ceph.mds.f.smithi164.stdout:starting mds.f at 2024-09-16T00:07:27.594 INFO:tasks.ceph.mds.d.smithi164.stdout:starting mds.d at 2024-09-16T00:07:27.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.617+0000 7f504353a640 1 Processor -- start 2024-09-16T00:07:27.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.617+0000 7f504353a640 1 -- start start 2024-09-16T00:07:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.618+0000 7f504353a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f503c154a80 0x7f503c0ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.618+0000 7f504353a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f503c06c6f0 0x7f503c067d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.618+0000 7f504353a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f503c15a7d0 con 0x7f503c067950 2024-09-16T00:07:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.618+0000 7f504353a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f503c15aa90 con 0x7f503c06c6f0 2024-09-16T00:07:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.618+0000 7f504353a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f503c15ad70 con 0x7f503c154a80 2024-09-16T00:07:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.618+0000 7f5041ad7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f503c06c6f0 0x7f503c067d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.618+0000 7f5040ad5640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f503c067950 0x7f503c06bfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38106/0 (socket says 172.21.15.139:38106) 2024-09-16T00:07:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.618+0000 7f5040ad5640 1 -- 172.21.15.139:0/3502201973 learned_addr learned my addr 172.21.15.139:0/3502201973 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.619+0000 7f50412d6640 1 --2- 172.21.15.139:0/3502201973 >> v2:172.21.15.164:3300/0 conn(0x7f503c154a80 0x7f503c0ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:27.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.619+0000 7f5041ad7640 1 -- 172.21.15.139:0/3502201973 >> v2:172.21.15.164:3300/0 conn(0x7f503c154a80 msgr2=0x7f503c0ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:27.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.619+0000 7f5041ad7640 1 --2- 172.21.15.139:0/3502201973 >> v2:172.21.15.164:3300/0 conn(0x7f503c154a80 0x7f503c0ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.619+0000 7f5033fff640 1 -- 172.21.15.139:0/3502201973 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3289184115 0 0) 0x7f503c15a7d0 con 0x7f503c067950 2024-09-16T00:07:27.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.619+0000 7f5041ad7640 1 -- 172.21.15.139:0/3502201973 >> v1:172.21.15.139:6789/0 conn(0x7f503c067950 legacy=0x7f503c06bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:27.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.619+0000 7f5041ad7640 1 -- 172.21.15.139:0/3502201973 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503c15b050 con 0x7f503c06c6f0 2024-09-16T00:07:27.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.619+0000 7f50412d6640 1 --2- 172.21.15.139:0/3502201973 >> v2:172.21.15.164:3300/0 conn(0x7f503c154a80 0x7f503c0ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:27.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.619+0000 7f5041ad7640 1 --2- 172.21.15.139:0/3502201973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f503c06c6f0 0x7f503c067d50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f503c0c7860 tx=0x7f503402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a1603d7ee8a683 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:27.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.620+0000 7f5033fff640 1 -- 172.21.15.139:0/3502201973 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034007740 con 0x7f503c06c6f0 2024-09-16T00:07:27.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.620+0000 7f5033fff640 1 -- 172.21.15.139:0/3502201973 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50340078e0 con 0x7f503c06c6f0 2024-09-16T00:07:27.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.620+0000 7f5033fff640 1 -- 172.21.15.139:0/3502201973 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034007cb0 con 0x7f503c06c6f0 2024-09-16T00:07:27.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.620+0000 7f504353a640 1 -- 172.21.15.139:0/3502201973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f503c06c6f0 msgr2=0x7f503c067d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:27.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.620+0000 7f504353a640 1 --2- 172.21.15.139:0/3502201973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f503c06c6f0 0x7f503c067d50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f503c0c7860 tx=0x7f503402a680 comp rx=0 tx=0).stop 2024-09-16T00:07:27.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.620+0000 7f504353a640 1 -- 172.21.15.139:0/3502201973 shutdown_connections 2024-09-16T00:07:27.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.620+0000 7f504353a640 1 --2- 172.21.15.139:0/3502201973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f503c06c6f0 0x7f503c067d50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.620+0000 7f504353a640 1 --2- 172.21.15.139:0/3502201973 >> v2:172.21.15.164:3300/0 conn(0x7f503c154a80 0x7f503c0ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.620+0000 7f504353a640 1 -- 172.21.15.139:0/3502201973 >> 172.21.15.139:0/3502201973 conn(0x7f503c06ea70 msgr2=0x7f503c06ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:27.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.620+0000 7f504353a640 1 -- 172.21.15.139:0/3502201973 shutdown_connections 2024-09-16T00:07:27.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.620+0000 7f504353a640 1 -- 172.21.15.139:0/3502201973 wait complete. 2024-09-16T00:07:27.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.621+0000 7f504353a640 1 Processor -- start 2024-09-16T00:07:27.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.621+0000 7f504353a640 1 -- start start 2024-09-16T00:07:27.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.622+0000 7f504353a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f503c067950 0x7f503c0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:27.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.622+0000 7f504353a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f503c06c6f0 0x7f503c1f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:27.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.622+0000 7f504353a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f503c154a80 0x7f503c157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:27.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.622+0000 7f504353a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f503c154e60 con 0x7f503c06c6f0 2024-09-16T00:07:27.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.622+0000 7f504353a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f503c15b610 con 0x7f503c154a80 2024-09-16T00:07:27.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.622+0000 7f504353a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f503c0704e0 con 0x7f503c067950 2024-09-16T00:07:27.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.622+0000 7f5040ad5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f503c06c6f0 0x7f503c1f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:27.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.622+0000 7f5040ad5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f503c06c6f0 0x7f503c1f16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46376/0 (socket says 172.21.15.139:46376) 2024-09-16T00:07:27.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.622+0000 7f5040ad5640 1 -- 172.21.15.139:0/4056784561 learned_addr learned my addr 172.21.15.139:0/4056784561 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:27.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.622+0000 7f50412d6640 1 --2- 172.21.15.139:0/4056784561 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f503c067950 0x7f503c0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:27.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.622+0000 7f5041ad7640 1 --2- 172.21.15.139:0/4056784561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f503c154a80 0x7f503c157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:27.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.623+0000 7f5040ad5640 1 -- 172.21.15.139:0/4056784561 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f503c067950 msgr2=0x7f503c0c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:27.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.623+0000 7f5040ad5640 1 --2- 172.21.15.139:0/4056784561 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f503c067950 0x7f503c0c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.623+0000 7f5040ad5640 1 -- 172.21.15.139:0/4056784561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f503c154a80 msgr2=0x7f503c157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:27.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.623+0000 7f5040ad5640 1 --2- 172.21.15.139:0/4056784561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f503c154a80 0x7f503c157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.623+0000 7f5040ad5640 1 -- 172.21.15.139:0/4056784561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503c1f11c0 con 0x7f503c06c6f0 2024-09-16T00:07:27.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.623+0000 7f50412d6640 1 --2- 172.21.15.139:0/4056784561 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f503c067950 0x7f503c0c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:27.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.623+0000 7f5040ad5640 1 --2- 172.21.15.139:0/4056784561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f503c06c6f0 0x7f503c1f16d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f502c00c4e0 tx=0x7f502c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:27.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.623+0000 7f5031ffb640 1 -- 172.21.15.139:0/4056784561 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c018070 con 0x7f503c06c6f0 2024-09-16T00:07:27.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.623+0000 7f5031ffb640 1 -- 172.21.15.139:0/4056784561 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f502c007500 con 0x7f503c06c6f0 2024-09-16T00:07:27.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.623+0000 7f504353a640 1 -- 172.21.15.139:0/4056784561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f503c157d90 con 0x7f503c06c6f0 2024-09-16T00:07:27.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.623+0000 7f5031ffb640 1 -- 172.21.15.139:0/4056784561 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c004a20 con 0x7f503c06c6f0 2024-09-16T00:07:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.623+0000 7f504353a640 1 -- 172.21.15.139:0/4056784561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f503c1f06e0 con 0x7f503c06c6f0 2024-09-16T00:07:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.624+0000 7f504353a640 1 -- 172.21.15.139:0/4056784561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f503c0ca4b0 con 0x7f503c06c6f0 2024-09-16T00:07:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.624+0000 7f5031ffb640 1 -- 172.21.15.139:0/4056784561 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f502c01b070 con 0x7f503c06c6f0 2024-09-16T00:07:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.625+0000 7f5031ffb640 1 --2- 172.21.15.139:0/4056784561 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f502006c2b0 0x7f502006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.625+0000 7f5031ffb640 1 -- 172.21.15.139:0/4056784561 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f502c08a3e0 con 0x7f503c06c6f0 2024-09-16T00:07:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.625+0000 7f50412d6640 1 --2- 172.21.15.139:0/4056784561 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f502006c2b0 0x7f502006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:27.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.625+0000 7f50412d6640 1 --2- 172.21.15.139:0/4056784561 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f502006c2b0 0x7f502006e770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5038006fd0 tx=0x7f5038008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:27.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.629+0000 7f5031ffb640 1 -- 172.21.15.139:0/4056784561 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f502c058900 con 0x7f503c06c6f0 2024-09-16T00:07:27.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.829+0000 7f504353a640 1 -- 172.21.15.139:0/4056784561 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f503c0709a0 con 0x7f503c06c6f0 2024-09-16T00:07:27.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.829+0000 7f5031ffb640 1 -- 172.21.15.139:0/4056784561 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1437 (secure 0 0 0) 0x7f502c05cc00 con 0x7f503c06c6f0 2024-09-16T00:07:27.830 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:27.831 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":2,"btime":"2024-09-16T00:07:27:580308+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2773358731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2773358731},{"type":"v1","addr":"172.21.15.139:6835","nonce":2773358731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-16T00:07:27.831 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 2 2024-09-16T00:07:27.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.832+0000 7f504353a640 1 -- 172.21.15.139:0/4056784561 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f502006c2b0 msgr2=0x7f502006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:27.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.832+0000 7f504353a640 1 --2- 172.21.15.139:0/4056784561 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f502006c2b0 0x7f502006e770 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5038006fd0 tx=0x7f5038008040 comp rx=0 tx=0).stop 2024-09-16T00:07:27.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.832+0000 7f504353a640 1 -- 172.21.15.139:0/4056784561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f503c06c6f0 msgr2=0x7f503c1f16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:27.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.832+0000 7f504353a640 1 --2- 172.21.15.139:0/4056784561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f503c06c6f0 0x7f503c1f16d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f502c00c4e0 tx=0x7f502c00c9a0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.832+0000 7f504353a640 1 -- 172.21.15.139:0/4056784561 shutdown_connections 2024-09-16T00:07:27.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.832+0000 7f504353a640 1 --2- 172.21.15.139:0/4056784561 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f502006c2b0 0x7f502006e770 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.832+0000 7f504353a640 1 --2- 172.21.15.139:0/4056784561 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f503c154a80 0x7f503c157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.832+0000 7f504353a640 1 --2- 172.21.15.139:0/4056784561 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f503c06c6f0 0x7f503c1f16d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.832+0000 7f504353a640 1 --2- 172.21.15.139:0/4056784561 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f503c067950 0x7f503c0c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:27.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.832+0000 7f504353a640 1 -- 172.21.15.139:0/4056784561 >> 172.21.15.139:0/4056784561 conn(0x7f503c06ea70 msgr2=0x7f503c0c9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:27.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.832+0000 7f504353a640 1 -- 172.21.15.139:0/4056784561 shutdown_connections 2024-09-16T00:07:27.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:27.832+0000 7f504353a640 1 -- 172.21.15.139:0/4056784561 wait complete. 2024-09-16T00:07:29.844 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T00:07:29.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.956+0000 7f1833a53640 1 Processor -- start 2024-09-16T00:07:29.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.956+0000 7f1833a53640 1 -- start start 2024-09-16T00:07:29.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.957+0000 7f1833a53640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f182c152a20 0x7f182c0c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:29.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.957+0000 7f1833a53640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f182c0c4ef0 0x7f182c158550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:29.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.957+0000 7f1833a53640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f182c15bbe0 con 0x7f182c0714d0 2024-09-16T00:07:29.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.957+0000 7f1833a53640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f182c0c8d80 con 0x7f182c0c4ef0 2024-09-16T00:07:29.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.957+0000 7f1833a53640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f182c15c4d0 con 0x7f182c152a20 2024-09-16T00:07:29.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.957+0000 7f1830fee640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f182c0c4ef0 0x7f182c158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:29.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.957+0000 7f1830fee640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f182c0c4ef0 0x7f182c158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46386/0 (socket says 172.21.15.139:46386) 2024-09-16T00:07:29.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.957+0000 7f1830fee640 1 -- 172.21.15.139:0/3768393533 learned_addr learned my addr 172.21.15.139:0/3768393533 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:29.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.958+0000 7f18317ef640 1 --2- 172.21.15.139:0/3768393533 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f182c152a20 0x7f182c0c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:29.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.958+0000 7f1830fee640 1 -- 172.21.15.139:0/3768393533 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f182c152a20 msgr2=0x7f182c0c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:29.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.958+0000 7f1830fee640 1 --2- 172.21.15.139:0/3768393533 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f182c152a20 0x7f182c0c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:29.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.958+0000 7f1830fee640 1 -- 172.21.15.139:0/3768393533 >> v1:172.21.15.164:6789/0 conn(0x7f182c0714d0 legacy=0x7f182c0ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:29.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.958+0000 7f1830fee640 1 -- 172.21.15.139:0/3768393533 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f182c0c35b0 con 0x7f182c0c4ef0 2024-09-16T00:07:29.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.958+0000 7f1830fee640 1 --2- 172.21.15.139:0/3768393533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f182c0c4ef0 0x7f182c158550 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f181c009870 tx=0x7f181c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a5fd403c94f818 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:29.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.959+0000 7f1823fff640 1 -- 172.21.15.139:0/3768393533 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c037070 con 0x7f182c0c4ef0 2024-09-16T00:07:29.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.959+0000 7f1823fff640 1 -- 172.21.15.139:0/3768393533 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f181c032440 con 0x7f182c0c4ef0 2024-09-16T00:07:29.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.959+0000 7f1823fff640 1 -- 172.21.15.139:0/3768393533 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181c032810 con 0x7f182c0c4ef0 2024-09-16T00:07:29.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.959+0000 7f1833a53640 1 -- 172.21.15.139:0/3768393533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f182c0c4ef0 msgr2=0x7f182c158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:29.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.959+0000 7f1833a53640 1 --2- 172.21.15.139:0/3768393533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f182c0c4ef0 0x7f182c158550 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f181c009870 tx=0x7f181c02a270 comp rx=0 tx=0).stop 2024-09-16T00:07:29.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.959+0000 7f1833a53640 1 -- 172.21.15.139:0/3768393533 shutdown_connections 2024-09-16T00:07:29.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.959+0000 7f1833a53640 1 --2- 172.21.15.139:0/3768393533 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f182c0c4ef0 0x7f182c158550 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:29.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.959+0000 7f1833a53640 1 --2- 172.21.15.139:0/3768393533 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f182c152a20 0x7f182c0c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:29.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.959+0000 7f1833a53640 1 -- 172.21.15.139:0/3768393533 >> 172.21.15.139:0/3768393533 conn(0x7f182c073e70 msgr2=0x7f182c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:29.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.959+0000 7f1833a53640 1 -- 172.21.15.139:0/3768393533 shutdown_connections 2024-09-16T00:07:29.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.959+0000 7f1833a53640 1 -- 172.21.15.139:0/3768393533 wait complete. 2024-09-16T00:07:29.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.960+0000 7f1833a53640 1 Processor -- start 2024-09-16T00:07:29.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.961+0000 7f1833a53640 1 -- start start 2024-09-16T00:07:29.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.961+0000 7f1833a53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f182c0714d0 0x7f182c1563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:29.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.961+0000 7f1833a53640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f182c0c4ef0 0x7f182c15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:29.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.961+0000 7f1833a53640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f182c152a20 0x7f182c14faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:29.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.961+0000 7f1833a53640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f182c0c3890 con 0x7f182c0714d0 2024-09-16T00:07:29.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.961+0000 7f1833a53640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f182c15c160 con 0x7f182c0c4ef0 2024-09-16T00:07:29.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.961+0000 7f1833a53640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f182c0c3a10 con 0x7f182c152a20 2024-09-16T00:07:29.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.961+0000 7f18317ef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f182c0714d0 0x7f182c1563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:29.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.961+0000 7f18317ef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f182c0714d0 0x7f182c1563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46396/0 (socket says 172.21.15.139:46396) 2024-09-16T00:07:29.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.961+0000 7f18317ef640 1 -- 172.21.15.139:0/2895851710 learned_addr learned my addr 172.21.15.139:0/2895851710 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:29.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.961+0000 7f1831ff0640 1 --2- 172.21.15.139:0/2895851710 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f182c152a20 0x7f182c14faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:29.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.961+0000 7f1830fee640 1 --2- 172.21.15.139:0/2895851710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f182c0c4ef0 0x7f182c15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:29.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.962+0000 7f18317ef640 1 -- 172.21.15.139:0/2895851710 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f182c152a20 msgr2=0x7f182c14faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:29.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.962+0000 7f18317ef640 1 --2- 172.21.15.139:0/2895851710 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f182c152a20 0x7f182c14faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:29.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.962+0000 7f18317ef640 1 -- 172.21.15.139:0/2895851710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f182c0c4ef0 msgr2=0x7f182c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:29.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.962+0000 7f18317ef640 1 --2- 172.21.15.139:0/2895851710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f182c0c4ef0 0x7f182c15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:29.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.962+0000 7f18317ef640 1 -- 172.21.15.139:0/2895851710 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f182c157b70 con 0x7f182c0714d0 2024-09-16T00:07:29.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.962+0000 7f18317ef640 1 --2- 172.21.15.139:0/2895851710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f182c0714d0 0x7f182c1563c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f182800e870 tx=0x7f182800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:29.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.962+0000 7f1821ffb640 1 -- 172.21.15.139:0/2895851710 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828017070 con 0x7f182c0714d0 2024-09-16T00:07:29.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.962+0000 7f1821ffb640 1 -- 172.21.15.139:0/2895851710 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f182800c8e0 con 0x7f182c0714d0 2024-09-16T00:07:29.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.962+0000 7f1833a53640 1 -- 172.21.15.139:0/2895851710 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f182c156900 con 0x7f182c0714d0 2024-09-16T00:07:29.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.962+0000 7f1821ffb640 1 -- 172.21.15.139:0/2895851710 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182800ccd0 con 0x7f182c0714d0 2024-09-16T00:07:29.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.962+0000 7f1833a53640 1 -- 172.21.15.139:0/2895851710 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f182c1e45f0 con 0x7f182c0714d0 2024-09-16T00:07:29.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.963+0000 7f1833a53640 1 -- 172.21.15.139:0/2895851710 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f182c0cefd0 con 0x7f182c0714d0 2024-09-16T00:07:29.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.964+0000 7f1821ffb640 1 -- 172.21.15.139:0/2895851710 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f182801a070 con 0x7f182c0714d0 2024-09-16T00:07:29.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.964+0000 7f1821ffb640 1 --2- 172.21.15.139:0/2895851710 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f181006c2b0 0x7f181006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:29.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.964+0000 7f1830fee640 1 --2- 172.21.15.139:0/2895851710 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f181006c2b0 0x7f181006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:29.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.964+0000 7f1821ffb640 1 -- 172.21.15.139:0/2895851710 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f182808ae40 con 0x7f182c0714d0 2024-09-16T00:07:29.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.965+0000 7f1830fee640 1 --2- 172.21.15.139:0/2895851710 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f181006c2b0 0x7f181006e770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f181c034000 tx=0x7f181c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:29.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:29.968+0000 7f1821ffb640 1 -- 172.21.15.139:0/2895851710 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f182c0cefd0 con 0x7f182c0714d0 2024-09-16T00:07:30.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.168+0000 7f1833a53640 1 -- 172.21.15.139:0/2895851710 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f182c15c960 con 0x7f182c0714d0 2024-09-16T00:07:30.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.168+0000 7f1821ffb640 1 -- 172.21.15.139:0/2895851710 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5569 (secure 0 0 0) 0x7f182c157b70 con 0x7f182c0714d0 2024-09-16T00:07:30.169 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:30.169 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":3,"btime":"2024-09-16T00:07:28:582370+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2773358731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2773358731},{"type":"v1","addr":"172.21.15.139:6835","nonce":2773358731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4268,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/2919602297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":2919602297},{"type":"v1","addr":"172.21.15.164:6837","nonce":2919602297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3575043661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":3575043661},{"type":"v1","addr":"172.21.15.139:6837","nonce":3575043661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/4113980181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":4113980181},{"type":"v1","addr":"172.21.15.139:6839","nonce":4113980181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/374774889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":374774889},{"type":"v1","addr":"172.21.15.164:6834","nonce":374774889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4300,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/4279698936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":4279698936},{"type":"v1","addr":"172.21.15.164:6836","nonce":4279698936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-16T00:07:30.170 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 3 2024-09-16T00:07:30.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.170+0000 7f1833a53640 1 -- 172.21.15.139:0/2895851710 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f181006c2b0 msgr2=0x7f181006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:30.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.170+0000 7f1833a53640 1 --2- 172.21.15.139:0/2895851710 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f181006c2b0 0x7f181006e770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f181c034000 tx=0x7f181c02e000 comp rx=0 tx=0).stop 2024-09-16T00:07:30.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.170+0000 7f1833a53640 1 -- 172.21.15.139:0/2895851710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f182c0714d0 msgr2=0x7f182c1563c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:30.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.170+0000 7f1833a53640 1 --2- 172.21.15.139:0/2895851710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f182c0714d0 0x7f182c1563c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f182800e870 tx=0x7f182800ed30 comp rx=0 tx=0).stop 2024-09-16T00:07:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.170+0000 7f1833a53640 1 -- 172.21.15.139:0/2895851710 shutdown_connections 2024-09-16T00:07:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.170+0000 7f1833a53640 1 --2- 172.21.15.139:0/2895851710 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f181006c2b0 0x7f181006e770 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.170+0000 7f1833a53640 1 --2- 172.21.15.139:0/2895851710 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f182c152a20 0x7f182c14faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.170+0000 7f1833a53640 1 --2- 172.21.15.139:0/2895851710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f182c0c4ef0 0x7f182c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.170+0000 7f1833a53640 1 --2- 172.21.15.139:0/2895851710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f182c0714d0 0x7f182c1563c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.170+0000 7f1833a53640 1 -- 172.21.15.139:0/2895851710 >> 172.21.15.139:0/2895851710 conn(0x7f182c073e70 msgr2=0x7f182c0ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.170+0000 7f1833a53640 1 -- 172.21.15.139:0/2895851710 shutdown_connections 2024-09-16T00:07:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.170+0000 7f1833a53640 1 -- 172.21.15.139:0/2895851710 wait complete. 2024-09-16T00:07:30.182 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-16T00:07:30.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.279+0000 7f86a9f52640 1 Processor -- start 2024-09-16T00:07:30.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.279+0000 7f86a9f52640 1 -- start start 2024-09-16T00:07:30.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.280+0000 7f86a9f52640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f86a4154bd0 0x7f86a415b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:30.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.280+0000 7f86a9f52640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f86a406ea20 0x7f86a40c3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:30.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.280+0000 7f86a9f52640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86a406cd40 con 0x7f86a40c8b50 2024-09-16T00:07:30.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.280+0000 7f86a9f52640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f86a415a9d0 con 0x7f86a406ea20 2024-09-16T00:07:30.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.280+0000 7f86a9f52640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f86a406cfc0 con 0x7f86a4154bd0 2024-09-16T00:07:30.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.280+0000 7f86a2ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f86a406ea20 0x7f86a40c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:30.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.280+0000 7f86a2ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f86a406ea20 0x7f86a40c3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46398/0 (socket says 172.21.15.139:46398) 2024-09-16T00:07:30.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.280+0000 7f86a2ffd640 1 -- 172.21.15.139:0/1771756512 learned_addr learned my addr 172.21.15.139:0/1771756512 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:30.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.280+0000 7f86a37fe640 1 --2- 172.21.15.139:0/1771756512 >> v2:172.21.15.164:3300/0 conn(0x7f86a4154bd0 0x7f86a415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:30.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.280+0000 7f86a2ffd640 1 -- 172.21.15.139:0/1771756512 >> v2:172.21.15.164:3300/0 conn(0x7f86a4154bd0 msgr2=0x7f86a415b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:30.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.280+0000 7f86a2ffd640 1 --2- 172.21.15.139:0/1771756512 >> v2:172.21.15.164:3300/0 conn(0x7f86a4154bd0 0x7f86a415b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.280+0000 7f86a2ffd640 1 -- 172.21.15.139:0/1771756512 >> v1:172.21.15.164:6789/0 conn(0x7f86a40c8b50 legacy=0x7f86a40ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:30.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.280+0000 7f86a2ffd640 1 -- 172.21.15.139:0/1771756512 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a415ac70 con 0x7f86a406ea20 2024-09-16T00:07:30.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.280+0000 7f86a37fe640 1 --2- 172.21.15.139:0/1771756512 >> v2:172.21.15.164:3300/0 conn(0x7f86a4154bd0 0x7f86a415b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:30.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.281+0000 7f86a2ffd640 1 --2- 172.21.15.139:0/1771756512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86a406ea20 0x7f86a40c3170 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f86a40cbc90 tx=0x7f869802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddac3ac035dcf104 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:30.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.281+0000 7f86a27fc640 1 -- 172.21.15.139:0/1771756512 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698037070 con 0x7f86a406ea20 2024-09-16T00:07:30.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.281+0000 7f86a27fc640 1 -- 172.21.15.139:0/1771756512 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8698032440 con 0x7f86a406ea20 2024-09-16T00:07:30.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.281+0000 7f86a27fc640 1 -- 172.21.15.139:0/1771756512 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698032810 con 0x7f86a406ea20 2024-09-16T00:07:30.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.282+0000 7f86a9f52640 1 -- 172.21.15.139:0/1771756512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86a406ea20 msgr2=0x7f86a40c3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:30.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.282+0000 7f86a9f52640 1 --2- 172.21.15.139:0/1771756512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86a406ea20 0x7f86a40c3170 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f86a40cbc90 tx=0x7f869802a230 comp rx=0 tx=0).stop 2024-09-16T00:07:30.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.282+0000 7f86a9f52640 1 -- 172.21.15.139:0/1771756512 shutdown_connections 2024-09-16T00:07:30.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.282+0000 7f86a9f52640 1 --2- 172.21.15.139:0/1771756512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86a406ea20 0x7f86a40c3170 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.282+0000 7f86a9f52640 1 --2- 172.21.15.139:0/1771756512 >> v2:172.21.15.164:3300/0 conn(0x7f86a4154bd0 0x7f86a415b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.282+0000 7f86a9f52640 1 -- 172.21.15.139:0/1771756512 >> 172.21.15.139:0/1771756512 conn(0x7f86a406c150 msgr2=0x7f86a4067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:30.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.282+0000 7f86a9f52640 1 -- 172.21.15.139:0/1771756512 shutdown_connections 2024-09-16T00:07:30.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.282+0000 7f86a9f52640 1 -- 172.21.15.139:0/1771756512 wait complete. 2024-09-16T00:07:30.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.283+0000 7f86a9f52640 1 Processor -- start 2024-09-16T00:07:30.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.283+0000 7f86a9f52640 1 -- start start 2024-09-16T00:07:30.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a9f52640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86a406ea20 0x7f86a4071870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:30.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a9f52640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86a40c8b50 0x7f86a40c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:30.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a9f52640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86a4154bd0 0x7f86a41f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:30.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a9f52640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f86a406d2a0 con 0x7f86a40c8b50 2024-09-16T00:07:30.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a9f52640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f86a4154fb0 con 0x7f86a4154bd0 2024-09-16T00:07:30.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a9f52640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f86a406d420 con 0x7f86a406ea20 2024-09-16T00:07:30.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86a40c8b50 0x7f86a40c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:30.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86a40c8b50 0x7f86a40c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46402/0 (socket says 172.21.15.139:46402) 2024-09-16T00:07:30.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a2ffd640 1 -- 172.21.15.139:0/3298299055 learned_addr learned my addr 172.21.15.139:0/3298299055 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:30.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a3fff640 1 --2- 172.21.15.139:0/3298299055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86a4154bd0 0x7f86a41f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:30.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a37fe640 1 --2- 172.21.15.139:0/3298299055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86a406ea20 0x7f86a4071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:30.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a2ffd640 1 -- 172.21.15.139:0/3298299055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86a406ea20 msgr2=0x7f86a4071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:30.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a2ffd640 1 --2- 172.21.15.139:0/3298299055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86a406ea20 0x7f86a4071870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a2ffd640 1 -- 172.21.15.139:0/3298299055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86a4154bd0 msgr2=0x7f86a41f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:30.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a2ffd640 1 --2- 172.21.15.139:0/3298299055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86a4154bd0 0x7f86a41f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a2ffd640 1 -- 172.21.15.139:0/3298299055 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a40c9010 con 0x7f86a40c8b50 2024-09-16T00:07:30.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a3fff640 1 --2- 172.21.15.139:0/3298299055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86a4154bd0 0x7f86a41f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:30.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a2ffd640 1 --2- 172.21.15.139:0/3298299055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86a40c8b50 0x7f86a40c9390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f869802a270 tx=0x7f86980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:30.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f868bfff640 1 -- 172.21.15.139:0/3298299055 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698037040 con 0x7f86a40c8b50 2024-09-16T00:07:30.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f868bfff640 1 -- 172.21.15.139:0/3298299055 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8698042070 con 0x7f86a40c8b50 2024-09-16T00:07:30.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.284+0000 7f86a9f52640 1 -- 172.21.15.139:0/3298299055 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a40cacb0 con 0x7f86a40c8b50 2024-09-16T00:07:30.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.285+0000 7f868bfff640 1 -- 172.21.15.139:0/3298299055 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698030bb0 con 0x7f86a40c8b50 2024-09-16T00:07:30.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.285+0000 7f86a9f52640 1 -- 172.21.15.139:0/3298299055 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a41e5010 con 0x7f86a40c8b50 2024-09-16T00:07:30.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.286+0000 7f86a9f52640 1 -- 172.21.15.139:0/3298299055 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86a40c3170 con 0x7f86a40c8b50 2024-09-16T00:07:30.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.286+0000 7f868bfff640 1 -- 172.21.15.139:0/3298299055 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8698047460 con 0x7f86a40c8b50 2024-09-16T00:07:30.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.286+0000 7f868bfff640 1 --2- 172.21.15.139:0/3298299055 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f869006c2b0 0x7f869006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:30.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.286+0000 7f868bfff640 1 -- 172.21.15.139:0/3298299055 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f86980abd50 con 0x7f86a40c8b50 2024-09-16T00:07:30.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.286+0000 7f86a37fe640 1 --2- 172.21.15.139:0/3298299055 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f869006c2b0 0x7f869006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:30.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.287+0000 7f86a37fe640 1 --2- 172.21.15.139:0/3298299055 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f869006c2b0 0x7f869006e770 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f869400a950 tx=0x7f8694008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:30.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.291+0000 7f868bfff640 1 -- 172.21.15.139:0/3298299055 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f86980af030 con 0x7f86a40c8b50 2024-09-16T00:07:30.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.456+0000 7f86a9f52640 1 -- 172.21.15.139:0/3298299055 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f86a406e420 con 0x7f86a40c8b50 2024-09-16T00:07:30.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.456+0000 7f868bfff640 1 -- 172.21.15.139:0/3298299055 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7f86a406cd40 con 0x7f86a40c8b50 2024-09-16T00:07:30.457 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:30.458 INFO:teuthology.orchestra.run.smithi139.stdout:[ 2024-09-16T00:07:30.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:30.458 INFO:teuthology.orchestra.run.smithi139.stdout: "poolnum": 1, 2024-09-16T00:07:30.458 INFO:teuthology.orchestra.run.smithi139.stdout: "poolname": ".mgr" 2024-09-16T00:07:30.458 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:30.458 INFO:teuthology.orchestra.run.smithi139.stdout:] 2024-09-16T00:07:30.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.458+0000 7f86a9f52640 1 -- 172.21.15.139:0/3298299055 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f869006c2b0 msgr2=0x7f869006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:30.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.458+0000 7f86a9f52640 1 --2- 172.21.15.139:0/3298299055 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f869006c2b0 0x7f869006e770 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f869400a950 tx=0x7f8694008040 comp rx=0 tx=0).stop 2024-09-16T00:07:30.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.458+0000 7f86a9f52640 1 -- 172.21.15.139:0/3298299055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86a40c8b50 msgr2=0x7f86a40c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:30.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.458+0000 7f86a9f52640 1 --2- 172.21.15.139:0/3298299055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86a40c8b50 0x7f86a40c9390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f869802a270 tx=0x7f86980046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.459+0000 7f86a9f52640 1 -- 172.21.15.139:0/3298299055 shutdown_connections 2024-09-16T00:07:30.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.459+0000 7f86a9f52640 1 --2- 172.21.15.139:0/3298299055 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f869006c2b0 0x7f869006e770 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.459+0000 7f86a9f52640 1 --2- 172.21.15.139:0/3298299055 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86a4154bd0 0x7f86a41f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.459+0000 7f86a9f52640 1 --2- 172.21.15.139:0/3298299055 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86a40c8b50 0x7f86a40c9390 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.459+0000 7f86a9f52640 1 --2- 172.21.15.139:0/3298299055 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86a406ea20 0x7f86a4071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.459+0000 7f86a9f52640 1 -- 172.21.15.139:0/3298299055 >> 172.21.15.139:0/3298299055 conn(0x7f86a406c150 msgr2=0x7f86a4153e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:30.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.459+0000 7f86a9f52640 1 -- 172.21.15.139:0/3298299055 shutdown_connections 2024-09-16T00:07:30.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.459+0000 7f86a9f52640 1 -- 172.21.15.139:0/3298299055 wait complete. 2024-09-16T00:07:30.470 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T00:07:30.470 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T00:07:30.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.572+0000 7fe8e9196640 1 Processor -- start 2024-09-16T00:07:30.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.572+0000 7fe8e9196640 1 -- start start 2024-09-16T00:07:30.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.573+0000 7fe8e9196640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe8e406d540 0x7fe8e40b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:30.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.573+0000 7fe8e9196640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8e415dcc0 0x7fe8e415e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:30.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.573+0000 7fe8e9196640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e41581e0 con 0x7fe8e40cb4b0 2024-09-16T00:07:30.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.573+0000 7fe8e9196640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe8e40b61f0 con 0x7fe8e406d540 2024-09-16T00:07:30.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.573+0000 7fe8e9196640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e40cca30 con 0x7fe8e415dcc0 2024-09-16T00:07:30.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.573+0000 7fe8e2d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe8e406d540 0x7fe8e40b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:30.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.573+0000 7fe8da59c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe8e40cb4b0 0x7fe8e40cb8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38118/0 (socket says 172.21.15.139:38118) 2024-09-16T00:07:30.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.573+0000 7fe8da59c640 1 -- 172.21.15.139:0/2037262498 learned_addr learned my addr 172.21.15.139:0/2037262498 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:30.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.573+0000 7fe8e359e640 1 --2- 172.21.15.139:0/2037262498 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8e415dcc0 0x7fe8e415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:30.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.573+0000 7fe8e2d9d640 1 -- 172.21.15.139:0/2037262498 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8e415dcc0 msgr2=0x7fe8e415e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:30.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.573+0000 7fe8e2d9d640 1 --2- 172.21.15.139:0/2037262498 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8e415dcc0 0x7fe8e415e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.573+0000 7fe8e2d9d640 1 -- 172.21.15.139:0/2037262498 >> v1:172.21.15.139:6789/0 conn(0x7fe8e40cb4b0 legacy=0x7fe8e40cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:30.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.573+0000 7fe8e2d9d640 1 -- 172.21.15.139:0/2037262498 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e40c8430 con 0x7fe8e406d540 2024-09-16T00:07:30.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.574+0000 7fe8e2d9d640 1 --2- 172.21.15.139:0/2037262498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e406d540 0x7fe8e40b5cb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40097a0 tx=0x7fe8d402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df16083742963bdd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:30.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.574+0000 7fe8e259c640 1 -- 172.21.15.139:0/2037262498 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4037070 con 0x7fe8e406d540 2024-09-16T00:07:30.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.574+0000 7fe8e259c640 1 -- 172.21.15.139:0/2037262498 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d4032440 con 0x7fe8e406d540 2024-09-16T00:07:30.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.574+0000 7fe8e259c640 1 -- 172.21.15.139:0/2037262498 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4032810 con 0x7fe8e406d540 2024-09-16T00:07:30.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.574+0000 7fe8e9196640 1 -- 172.21.15.139:0/2037262498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e406d540 msgr2=0x7fe8e40b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:30.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.574+0000 7fe8e9196640 1 --2- 172.21.15.139:0/2037262498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e406d540 0x7fe8e40b5cb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40097a0 tx=0x7fe8d402a180 comp rx=0 tx=0).stop 2024-09-16T00:07:30.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.574+0000 7fe8e9196640 1 -- 172.21.15.139:0/2037262498 shutdown_connections 2024-09-16T00:07:30.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.574+0000 7fe8e9196640 1 --2- 172.21.15.139:0/2037262498 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8e415dcc0 0x7fe8e415e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.574+0000 7fe8e9196640 1 --2- 172.21.15.139:0/2037262498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e406d540 0x7fe8e40b5cb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.574+0000 7fe8e9196640 1 -- 172.21.15.139:0/2037262498 >> 172.21.15.139:0/2037262498 conn(0x7fe8e40d0210 msgr2=0x7fe8e40d0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:30.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.574+0000 7fe8e9196640 1 -- 172.21.15.139:0/2037262498 shutdown_connections 2024-09-16T00:07:30.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.574+0000 7fe8e9196640 1 -- 172.21.15.139:0/2037262498 wait complete. 2024-09-16T00:07:30.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.576+0000 7fe8e9196640 1 Processor -- start 2024-09-16T00:07:30.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.576+0000 7fe8e9196640 1 -- start start 2024-09-16T00:07:30.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.576+0000 7fe8e9196640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8e406d540 0x7fe8e41f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:30.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.576+0000 7fe8e9196640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e40cb4b0 0x7fe8e41f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:30.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8e9196640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8e415dcc0 0x7fe8e41f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:30.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8e9196640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e40c8cd0 con 0x7fe8e40cb4b0 2024-09-16T00:07:30.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8e9196640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e40c89f0 con 0x7fe8e415dcc0 2024-09-16T00:07:30.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8e9196640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e40c8fb0 con 0x7fe8e406d540 2024-09-16T00:07:30.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8da59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e40cb4b0 0x7fe8e41f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:30.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8da59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e40cb4b0 0x7fe8e41f2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46418/0 (socket says 172.21.15.139:46418) 2024-09-16T00:07:30.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8da59c640 1 -- 172.21.15.139:0/3750089488 learned_addr learned my addr 172.21.15.139:0/3750089488 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:30.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8e359e640 1 --2- 172.21.15.139:0/3750089488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8e415dcc0 0x7fe8e41f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:30.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8da59c640 1 -- 172.21.15.139:0/3750089488 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8e406d540 msgr2=0x7fe8e41f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:30.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8da59c640 1 --2- 172.21.15.139:0/3750089488 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8e406d540 0x7fe8e41f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8da59c640 1 -- 172.21.15.139:0/3750089488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8e415dcc0 msgr2=0x7fe8e41f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:30.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8da59c640 1 --2- 172.21.15.139:0/3750089488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8e415dcc0 0x7fe8e41f3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:30.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8da59c640 1 -- 172.21.15.139:0/3750089488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e415b0c0 con 0x7fe8e40cb4b0 2024-09-16T00:07:30.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8da59c640 1 --2- 172.21.15.139:0/3750089488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e40cb4b0 0x7fe8e41f2a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c430 tx=0x7fe8cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:30.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8dbfff640 1 -- 172.21.15.139:0/3750089488 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc017070 con 0x7fe8e40cb4b0 2024-09-16T00:07:30.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8dbfff640 1 -- 172.21.15.139:0/3750089488 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8cc005ce0 con 0x7fe8e40cb4b0 2024-09-16T00:07:30.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8e9196640 1 -- 172.21.15.139:0/3750089488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e415b3a0 con 0x7fe8e40cb4b0 2024-09-16T00:07:30.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8dbfff640 1 -- 172.21.15.139:0/3750089488 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0072f0 con 0x7fe8e40cb4b0 2024-09-16T00:07:30.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.577+0000 7fe8e9196640 1 -- 172.21.15.139:0/3750089488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e41f1fd0 con 0x7fe8e40cb4b0 2024-09-16T00:07:30.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.578+0000 7fe8e9196640 1 -- 172.21.15.139:0/3750089488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e41f21d0 con 0x7fe8e40cb4b0 2024-09-16T00:07:30.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.579+0000 7fe8dbfff640 1 -- 172.21.15.139:0/3750089488 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe8cc01a070 con 0x7fe8e40cb4b0 2024-09-16T00:07:30.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.579+0000 7fe8dbfff640 1 --2- 172.21.15.139:0/3750089488 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe8c406c2b0 0x7fe8c406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:30.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.579+0000 7fe8e2d9d640 1 --2- 172.21.15.139:0/3750089488 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe8c406c2b0 0x7fe8c406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:30.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.579+0000 7fe8dbfff640 1 -- 172.21.15.139:0/3750089488 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fe8cc08a560 con 0x7fe8e40cb4b0 2024-09-16T00:07:30.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.580+0000 7fe8e2d9d640 1 --2- 172.21.15.139:0/3750089488 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe8c406c2b0 0x7fe8c406e770 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4034000 tx=0x7fe8d402c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:30.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.584+0000 7fe8dbfff640 1 -- 172.21.15.139:0/3750089488 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe8e41f21d0 con 0x7fe8e40cb4b0 2024-09-16T00:07:30.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:30.749+0000 7fe8e9196640 1 -- 172.21.15.139:0/3750089488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe8e40c6f70 con 0x7fe8e40cb4b0 2024-09-16T00:07:31.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.601+0000 7fe8dbfff640 1 -- 172.21.15.139:0/3750089488 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7fe8cc058fa0 con 0x7fe8e40cb4b0 2024-09-16T00:07:31.602 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' created 2024-09-16T00:07:31.603 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:07:31.606 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:07:31.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.607+0000 7fe8e9196640 1 -- 172.21.15.139:0/3750089488 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe8c406c2b0 msgr2=0x7fe8c406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:31.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.607+0000 7fe8e9196640 1 --2- 172.21.15.139:0/3750089488 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe8c406c2b0 0x7fe8c406e770 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4034000 tx=0x7fe8d402c000 comp rx=0 tx=0).stop 2024-09-16T00:07:31.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.608+0000 7fe8e9196640 1 -- 172.21.15.139:0/3750089488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e40cb4b0 msgr2=0x7fe8e41f2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:31.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.608+0000 7fe8e9196640 1 --2- 172.21.15.139:0/3750089488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e40cb4b0 0x7fe8e41f2a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00c430 tx=0x7fe8cc00c8f0 comp rx=0 tx=0).stop 2024-09-16T00:07:31.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.608+0000 7fe8e9196640 1 -- 172.21.15.139:0/3750089488 shutdown_connections 2024-09-16T00:07:31.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.608+0000 7fe8e9196640 1 --2- 172.21.15.139:0/3750089488 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe8c406c2b0 0x7fe8c406e770 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:31.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.609+0000 7fe8e9196640 1 --2- 172.21.15.139:0/3750089488 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8e415dcc0 0x7fe8e41f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:31.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.609+0000 7fe8e9196640 1 --2- 172.21.15.139:0/3750089488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8e40cb4b0 0x7fe8e41f2a30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:31.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.609+0000 7fe8e9196640 1 --2- 172.21.15.139:0/3750089488 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8e406d540 0x7fe8e41f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:31.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.609+0000 7fe8e9196640 1 -- 172.21.15.139:0/3750089488 >> 172.21.15.139:0/3750089488 conn(0x7fe8e40d0210 msgr2=0x7fe8e40d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:31.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.609+0000 7fe8e9196640 1 -- 172.21.15.139:0/3750089488 shutdown_connections 2024-09-16T00:07:31.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.609+0000 7fe8e9196640 1 -- 172.21.15.139:0/3750089488 wait complete. 2024-09-16T00:07:31.623 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T00:07:31.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.811+0000 7f4ef960a640 1 Processor -- start 2024-09-16T00:07:31.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.811+0000 7f4ef960a640 1 -- start start 2024-09-16T00:07:31.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.812+0000 7f4ef960a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4ef406ea70 0x7f4ef415e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:31.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.812+0000 7f4ef960a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4ef406bd20 0x7f4ef415e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:31.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.812+0000 7f4ef960a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ef415a950 con 0x7f4ef4154a10 2024-09-16T00:07:31.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.812+0000 7f4ef960a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4ef415ac30 con 0x7f4ef406bd20 2024-09-16T00:07:31.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.812+0000 7f4ef960a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f4ef415af10 con 0x7f4ef406ea70 2024-09-16T00:07:31.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.812+0000 7f4ef37fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4ef406bd20 0x7f4ef415e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:31.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.812+0000 7f4ef37fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4ef406bd20 0x7f4ef415e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46420/0 (socket says 172.21.15.139:46420) 2024-09-16T00:07:31.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.812+0000 7f4ef37fe640 1 -- 172.21.15.139:0/2693190488 learned_addr learned my addr 172.21.15.139:0/2693190488 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:31.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.813+0000 7f4ef27fc640 1 --2- 172.21.15.139:0/2693190488 >> v2:172.21.15.164:3300/0 conn(0x7f4ef406ea70 0x7f4ef415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:31.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.813+0000 7f4ef37fe640 1 -- 172.21.15.139:0/2693190488 >> v2:172.21.15.164:3300/0 conn(0x7f4ef406ea70 msgr2=0x7f4ef415e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:31.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.813+0000 7f4ef37fe640 1 --2- 172.21.15.139:0/2693190488 >> v2:172.21.15.164:3300/0 conn(0x7f4ef406ea70 0x7f4ef415e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:31.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.813+0000 7f4ef37fe640 1 -- 172.21.15.139:0/2693190488 >> v1:172.21.15.164:6789/0 conn(0x7f4ef4154a10 legacy=0x7f4ef40cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:31.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.813+0000 7f4ef37fe640 1 -- 172.21.15.139:0/2693190488 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef415b1f0 con 0x7f4ef406bd20 2024-09-16T00:07:31.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.813+0000 7f4ef37fe640 1 --2- 172.21.15.139:0/2693190488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef406bd20 0x7f4ef415e670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40722b0 tx=0x7f4eec02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5190ba6023365a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:31.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.813+0000 7f4ef1ffb640 1 -- 172.21.15.139:0/2693190488 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec037070 con 0x7f4ef406bd20 2024-09-16T00:07:31.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.814+0000 7f4ef1ffb640 1 -- 172.21.15.139:0/2693190488 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eec032440 con 0x7f4ef406bd20 2024-09-16T00:07:31.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.814+0000 7f4ef1ffb640 1 -- 172.21.15.139:0/2693190488 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec032810 con 0x7f4ef406bd20 2024-09-16T00:07:31.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.814+0000 7f4ef960a640 1 -- 172.21.15.139:0/2693190488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef406bd20 msgr2=0x7f4ef415e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:31.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.814+0000 7f4ef960a640 1 --2- 172.21.15.139:0/2693190488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef406bd20 0x7f4ef415e670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40722b0 tx=0x7f4eec02a2c0 comp rx=0 tx=0).stop 2024-09-16T00:07:31.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.814+0000 7f4ef960a640 1 -- 172.21.15.139:0/2693190488 shutdown_connections 2024-09-16T00:07:31.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.814+0000 7f4ef960a640 1 --2- 172.21.15.139:0/2693190488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef406bd20 0x7f4ef415e670 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:31.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.814+0000 7f4ef960a640 1 --2- 172.21.15.139:0/2693190488 >> v2:172.21.15.164:3300/0 conn(0x7f4ef406ea70 0x7f4ef415e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:31.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.814+0000 7f4ef960a640 1 -- 172.21.15.139:0/2693190488 >> 172.21.15.139:0/2693190488 conn(0x7f4ef406cc30 msgr2=0x7f4ef406d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:31.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.814+0000 7f4ef960a640 1 -- 172.21.15.139:0/2693190488 shutdown_connections 2024-09-16T00:07:31.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.814+0000 7f4ef960a640 1 -- 172.21.15.139:0/2693190488 wait complete. 2024-09-16T00:07:31.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.815+0000 7f4ef960a640 1 Processor -- start 2024-09-16T00:07:31.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.815+0000 7f4ef960a640 1 -- start start 2024-09-16T00:07:31.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.816+0000 7f4ef960a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4ef406bd20 0x7f4ef41f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:31.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.816+0000 7f4ef960a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef406ea70 0x7f4ef415cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:31.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.816+0000 7f4ef960a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef4154a10 0x7f4ef414ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:31.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.816+0000 7f4ef960a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef4154df0 con 0x7f4ef406ea70 2024-09-16T00:07:31.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.816+0000 7f4ef960a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef406da90 con 0x7f4ef4154a10 2024-09-16T00:07:31.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.816+0000 7f4ef960a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4ef406c460 con 0x7f4ef406bd20 2024-09-16T00:07:31.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.816+0000 7f4ef27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef406ea70 0x7f4ef415cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:31.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.816+0000 7f4ef27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef406ea70 0x7f4ef415cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46424/0 (socket says 172.21.15.139:46424) 2024-09-16T00:07:31.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.816+0000 7f4ef27fc640 1 -- 172.21.15.139:0/2423653305 learned_addr learned my addr 172.21.15.139:0/2423653305 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:31.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.816+0000 7f4ef37fe640 1 --2- 172.21.15.139:0/2423653305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef4154a10 0x7f4ef414ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:31.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.816+0000 7f4ef2ffd640 1 --2- 172.21.15.139:0/2423653305 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4ef406bd20 0x7f4ef41f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:31.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.816+0000 7f4ef27fc640 1 -- 172.21.15.139:0/2423653305 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4ef406bd20 msgr2=0x7f4ef41f1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:31.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.816+0000 7f4ef27fc640 1 --2- 172.21.15.139:0/2423653305 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4ef406bd20 0x7f4ef41f1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:31.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.816+0000 7f4ef27fc640 1 -- 172.21.15.139:0/2423653305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef4154a10 msgr2=0x7f4ef414ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:31.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.816+0000 7f4ef27fc640 1 --2- 172.21.15.139:0/2423653305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef4154a10 0x7f4ef414ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:31.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.816+0000 7f4ef27fc640 1 -- 172.21.15.139:0/2423653305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef41553d0 con 0x7f4ef406ea70 2024-09-16T00:07:31.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.817+0000 7f4ef27fc640 1 --2- 172.21.15.139:0/2423653305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef406ea70 0x7f4ef415cf60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00d800 tx=0x7f4edc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:31.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.817+0000 7f4eeb7fe640 1 -- 172.21.15.139:0/2423653305 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00b6c0 con 0x7f4ef406ea70 2024-09-16T00:07:31.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.817+0000 7f4eeb7fe640 1 -- 172.21.15.139:0/2423653305 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4edc019070 con 0x7f4ef406ea70 2024-09-16T00:07:31.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.817+0000 7f4eeb7fe640 1 -- 172.21.15.139:0/2423653305 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00bac0 con 0x7f4ef406ea70 2024-09-16T00:07:31.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.817+0000 7f4ef960a640 1 -- 172.21.15.139:0/2423653305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef415b9c0 con 0x7f4ef406ea70 2024-09-16T00:07:31.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.817+0000 7f4ef960a640 1 -- 172.21.15.139:0/2423653305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef415bf80 con 0x7f4ef406ea70 2024-09-16T00:07:31.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.818+0000 7f4ef960a640 1 -- 172.21.15.139:0/2423653305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef40cd030 con 0x7f4ef406ea70 2024-09-16T00:07:31.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.818+0000 7f4eeb7fe640 1 -- 172.21.15.139:0/2423653305 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4edc00bc60 con 0x7f4ef406ea70 2024-09-16T00:07:31.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.819+0000 7f4eeb7fe640 1 --2- 172.21.15.139:0/2423653305 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4ed406c2b0 0x7f4ed406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:31.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.819+0000 7f4eeb7fe640 1 -- 172.21.15.139:0/2423653305 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7f4edc08b5c0 con 0x7f4ef406ea70 2024-09-16T00:07:31.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.819+0000 7f4ef2ffd640 1 --2- 172.21.15.139:0/2423653305 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4ed406c2b0 0x7f4ed406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:31.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.819+0000 7f4ef2ffd640 1 --2- 172.21.15.139:0/2423653305 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4ed406c2b0 0x7f4ed406e770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4005660 tx=0x7f4ee4003d90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:31.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.823+0000 7f4eeb7fe640 1 -- 172.21.15.139:0/2423653305 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4edc059950 con 0x7f4ef406ea70 2024-09-16T00:07:31.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:31.989+0000 7f4ef960a640 1 -- 172.21.15.139:0/2423653305 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4ef41e6c40 con 0x7f4ef406ea70 2024-09-16T00:07:32.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.607+0000 7f4eeb7fe640 1 -- 172.21.15.139:0/2423653305 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) ==== 170+0+0 (secure 0 0 0) 0x7f4edc05dc50 con 0x7f4ef406ea70 2024-09-16T00:07:32.608 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' created 2024-09-16T00:07:32.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.610+0000 7f4ef960a640 1 -- 172.21.15.139:0/2423653305 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4ed406c2b0 msgr2=0x7f4ed406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:32.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.610+0000 7f4ef960a640 1 --2- 172.21.15.139:0/2423653305 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4ed406c2b0 0x7f4ed406e770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4005660 tx=0x7f4ee4003d90 comp rx=0 tx=0).stop 2024-09-16T00:07:32.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.610+0000 7f4ef960a640 1 -- 172.21.15.139:0/2423653305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef406ea70 msgr2=0x7f4ef415cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:32.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.610+0000 7f4ef960a640 1 --2- 172.21.15.139:0/2423653305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef406ea70 0x7f4ef415cf60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00d800 tx=0x7f4edc00dcc0 comp rx=0 tx=0).stop 2024-09-16T00:07:32.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.610+0000 7f4ef960a640 1 -- 172.21.15.139:0/2423653305 shutdown_connections 2024-09-16T00:07:32.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.610+0000 7f4ef960a640 1 --2- 172.21.15.139:0/2423653305 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4ed406c2b0 0x7f4ed406e770 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:32.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.610+0000 7f4ef960a640 1 --2- 172.21.15.139:0/2423653305 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4ef4154a10 0x7f4ef414ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:32.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.610+0000 7f4ef960a640 1 --2- 172.21.15.139:0/2423653305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ef406ea70 0x7f4ef415cf60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:32.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.610+0000 7f4ef960a640 1 --2- 172.21.15.139:0/2423653305 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4ef406bd20 0x7f4ef41f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:32.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.610+0000 7f4ef960a640 1 -- 172.21.15.139:0/2423653305 >> 172.21.15.139:0/2423653305 conn(0x7f4ef406cc30 msgr2=0x7f4ef4152230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:32.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.610+0000 7f4ef960a640 1 -- 172.21.15.139:0/2423653305 shutdown_connections 2024-09-16T00:07:32.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.610+0000 7f4ef960a640 1 -- 172.21.15.139:0/2423653305 wait complete. 2024-09-16T00:07:32.634 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T00:07:32.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.735+0000 7f94956d2640 1 Processor -- start 2024-09-16T00:07:32.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.735+0000 7f94956d2640 1 -- start start 2024-09-16T00:07:32.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.736+0000 7f94956d2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9490154ab0 0x7f949015b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:32.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.736+0000 7f94956d2640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f94900d07b0 0x7f94900c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:32.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.736+0000 7f94956d2640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f949015b740 con 0x7f94900cb100 2024-09-16T00:07:32.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.736+0000 7f94956d2640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f94900cd510 con 0x7f94900d07b0 2024-09-16T00:07:32.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.736+0000 7f94956d2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9490154e90 con 0x7f9490154ab0 2024-09-16T00:07:32.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.736+0000 7f948e7fc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f94900d07b0 0x7f94900c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:32.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.736+0000 7f948effd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9490154ab0 0x7f949015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:32.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.736+0000 7f948e7fc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f94900d07b0 0x7f94900c5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:43024/0 (socket says 172.21.15.139:43024) 2024-09-16T00:07:32.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.736+0000 7f948effd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9490154ab0 0x7f949015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:55516/0 (socket says 172.21.15.139:55516) 2024-09-16T00:07:32.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.736+0000 7f948e7fc640 1 -- 172.21.15.139:0/2566188067 learned_addr learned my addr 172.21.15.139:0/2566188067 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:32.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.736+0000 7f948f7fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f94900cb100 0x7f949015dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:48646/0 (socket says 172.21.15.139:48646) 2024-09-16T00:07:32.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.737+0000 7f948e7fc640 1 -- 172.21.15.139:0/2566188067 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9490154ab0 msgr2=0x7f949015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:32.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.737+0000 7f948e7fc640 1 --2- 172.21.15.139:0/2566188067 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9490154ab0 0x7f949015b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:32.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.737+0000 7f948e7fc640 1 -- 172.21.15.139:0/2566188067 >> v1:172.21.15.164:6789/0 conn(0x7f94900cb100 legacy=0x7f949015dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:32.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.737+0000 7f948dffb640 1 -- 172.21.15.139:0/2566188067 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3346544315 0 0) 0x7f9488001110 con 0x7f94900cb100 2024-09-16T00:07:32.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.737+0000 7f948e7fc640 1 -- 172.21.15.139:0/2566188067 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94900cc620 con 0x7f94900d07b0 2024-09-16T00:07:32.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.737+0000 7f948e7fc640 1 --2- 172.21.15.139:0/2566188067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94900d07b0 0x7f94900c5140 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9484009850 tx=0x7f948402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8222a02a55fa1b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:32.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.738+0000 7f948dffb640 1 -- 172.21.15.139:0/2566188067 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484004230 con 0x7f94900d07b0 2024-09-16T00:07:32.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.738+0000 7f948dffb640 1 -- 172.21.15.139:0/2566188067 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94840043d0 con 0x7f94900d07b0 2024-09-16T00:07:32.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.738+0000 7f948dffb640 1 -- 172.21.15.139:0/2566188067 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484003810 con 0x7f94900d07b0 2024-09-16T00:07:32.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.738+0000 7f94956d2640 1 -- 172.21.15.139:0/2566188067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94900d07b0 msgr2=0x7f94900c5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:32.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.738+0000 7f94956d2640 1 --2- 172.21.15.139:0/2566188067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94900d07b0 0x7f94900c5140 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9484009850 tx=0x7f948402a2c0 comp rx=0 tx=0).stop 2024-09-16T00:07:32.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.738+0000 7f94956d2640 1 -- 172.21.15.139:0/2566188067 shutdown_connections 2024-09-16T00:07:32.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.738+0000 7f94956d2640 1 --2- 172.21.15.139:0/2566188067 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f94900d07b0 0x7f94900c5140 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:32.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.738+0000 7f94956d2640 1 --2- 172.21.15.139:0/2566188067 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9490154ab0 0x7f949015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:32.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.738+0000 7f94956d2640 1 -- 172.21.15.139:0/2566188067 >> 172.21.15.139:0/2566188067 conn(0x7f949006c830 msgr2=0x7f949006cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:32.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.738+0000 7f94956d2640 1 -- 172.21.15.139:0/2566188067 shutdown_connections 2024-09-16T00:07:32.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.738+0000 7f94956d2640 1 -- 172.21.15.139:0/2566188067 wait complete. 2024-09-16T00:07:32.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.739+0000 7f94956d2640 1 Processor -- start 2024-09-16T00:07:32.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.739+0000 7f94956d2640 1 -- start start 2024-09-16T00:07:32.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.740+0000 7f94956d2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94900cb100 0x7f94900b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:32.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.740+0000 7f94956d2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94900d07b0 0x7f949014cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:32.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.740+0000 7f94956d2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9490154ab0 0x7f94901f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:32.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.740+0000 7f94956d2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f94900ccec0 con 0x7f94900d07b0 2024-09-16T00:07:32.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.740+0000 7f94956d2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f94900ccbe0 con 0x7f9490154ab0 2024-09-16T00:07:32.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.740+0000 7f94956d2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f94900cd1a0 con 0x7f94900cb100 2024-09-16T00:07:32.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.740+0000 7f948e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94900d07b0 0x7f949014cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:32.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.740+0000 7f948e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94900d07b0 0x7f949014cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46438/0 (socket says 172.21.15.139:46438) 2024-09-16T00:07:32.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.740+0000 7f948e7fc640 1 -- 172.21.15.139:0/4057342358 learned_addr learned my addr 172.21.15.139:0/4057342358 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:32.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.740+0000 7f948f7fe640 1 --2- 172.21.15.139:0/4057342358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9490154ab0 0x7f94901f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:32.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.740+0000 7f948effd640 1 --2- 172.21.15.139:0/4057342358 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94900cb100 0x7f94900b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:32.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.741+0000 7f948e7fc640 1 -- 172.21.15.139:0/4057342358 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94900cb100 msgr2=0x7f94900b5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:32.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.741+0000 7f948e7fc640 1 --2- 172.21.15.139:0/4057342358 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94900cb100 0x7f94900b5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:32.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.741+0000 7f948e7fc640 1 -- 172.21.15.139:0/4057342358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9490154ab0 msgr2=0x7f94901f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:32.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.741+0000 7f948e7fc640 1 --2- 172.21.15.139:0/4057342358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9490154ab0 0x7f94901f1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:32.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.741+0000 7f948e7fc640 1 -- 172.21.15.139:0/4057342358 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94901f1370 con 0x7f94900d07b0 2024-09-16T00:07:32.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.741+0000 7f948e7fc640 1 --2- 172.21.15.139:0/4057342358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94900d07b0 0x7f949014cd10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9484034040 tx=0x7f94840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:32.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.741+0000 7f94777fe640 1 -- 172.21.15.139:0/4057342358 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484041030 con 0x7f94900d07b0 2024-09-16T00:07:32.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.741+0000 7f94777fe640 1 -- 172.21.15.139:0/4057342358 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9484003c50 con 0x7f94900d07b0 2024-09-16T00:07:32.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.741+0000 7f94956d2640 1 -- 172.21.15.139:0/4057342358 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94901f1650 con 0x7f94900d07b0 2024-09-16T00:07:32.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.741+0000 7f94777fe640 1 -- 172.21.15.139:0/4057342358 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484003e20 con 0x7f94900d07b0 2024-09-16T00:07:32.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.741+0000 7f94956d2640 1 -- 172.21.15.139:0/4057342358 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f949014be00 con 0x7f94900d07b0 2024-09-16T00:07:32.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.742+0000 7f94956d2640 1 -- 172.21.15.139:0/4057342358 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94900cc220 con 0x7f94900d07b0 2024-09-16T00:07:32.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.743+0000 7f94777fe640 1 -- 172.21.15.139:0/4057342358 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9484044070 con 0x7f94900d07b0 2024-09-16T00:07:32.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.743+0000 7f94777fe640 1 --2- 172.21.15.139:0/4057342358 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f947c06c2b0 0x7f947c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:32.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.743+0000 7f94777fe640 1 -- 172.21.15.139:0/4057342358 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f94840abcd0 con 0x7f94900d07b0 2024-09-16T00:07:32.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.743+0000 7f948effd640 1 --2- 172.21.15.139:0/4057342358 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f947c06c2b0 0x7f947c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:32.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.744+0000 7f948effd640 1 --2- 172.21.15.139:0/4057342358 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f947c06c2b0 0x7f947c06e770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9480011340 tx=0x7f948000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:32.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.748+0000 7f94777fe640 1 -- 172.21.15.139:0/4057342358 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9484079e50 con 0x7f94900d07b0 2024-09-16T00:07:32.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:32.948+0000 7f94956d2640 1 -- 172.21.15.139:0/4057342358 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f94900b6430 con 0x7f94900d07b0 2024-09-16T00:07:33.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.609+0000 7f94777fe640 1 -- 172.21.15.139:0/4057342358 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f948407e150 con 0x7f94900d07b0 2024-09-16T00:07:33.610 INFO:teuthology.orchestra.run.smithi139.stderr:new fs with metadata pool 2 and data pool 3 2024-09-16T00:07:33.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.612+0000 7f94956d2640 1 -- 172.21.15.139:0/4057342358 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f947c06c2b0 msgr2=0x7f947c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:33.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.612+0000 7f94956d2640 1 --2- 172.21.15.139:0/4057342358 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f947c06c2b0 0x7f947c06e770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9480011340 tx=0x7f948000b040 comp rx=0 tx=0).stop 2024-09-16T00:07:33.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.612+0000 7f94956d2640 1 -- 172.21.15.139:0/4057342358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94900d07b0 msgr2=0x7f949014cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:33.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.612+0000 7f94956d2640 1 --2- 172.21.15.139:0/4057342358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94900d07b0 0x7f949014cd10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9484034040 tx=0x7f94840046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.613+0000 7f94956d2640 1 -- 172.21.15.139:0/4057342358 shutdown_connections 2024-09-16T00:07:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.613+0000 7f94956d2640 1 --2- 172.21.15.139:0/4057342358 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f947c06c2b0 0x7f947c06e770 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.613+0000 7f94956d2640 1 --2- 172.21.15.139:0/4057342358 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9490154ab0 0x7f94901f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.613+0000 7f94956d2640 1 --2- 172.21.15.139:0/4057342358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94900d07b0 0x7f949014cd10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.613+0000 7f94956d2640 1 --2- 172.21.15.139:0/4057342358 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f94900cb100 0x7f94900b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:33.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.613+0000 7f94956d2640 1 -- 172.21.15.139:0/4057342358 >> 172.21.15.139:0/4057342358 conn(0x7f949006c830 msgr2=0x7f94900cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:33.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.615+0000 7f94956d2640 1 -- 172.21.15.139:0/4057342358 shutdown_connections 2024-09-16T00:07:33.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.615+0000 7f94956d2640 1 -- 172.21.15.139:0/4057342358 wait complete. 2024-09-16T00:07:33.642 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:07:33.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.740+0000 7fcdc432a640 1 Processor -- start 2024-09-16T00:07:33.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.740+0000 7fcdc432a640 1 -- start start 2024-09-16T00:07:33.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.741+0000 7fcdc432a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcdbc1527f0 0x7fcdbc0c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:33.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.741+0000 7fcdc432a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdbc0707f0 con 0x7fcdbc06f7f0 2024-09-16T00:07:33.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.741+0000 7fcdc432a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdbc070170 con 0x7fcdbc0c8250 2024-09-16T00:07:33.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.741+0000 7fcdc432a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcdbc06ff20 con 0x7fcdbc1527f0 2024-09-16T00:07:33.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.741+0000 7fcdc18c5640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fcdbc06f7f0 0x7fcdbc0ce0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38134/0 (socket says 172.21.15.139:38134) 2024-09-16T00:07:33.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.741+0000 7fcdc18c5640 1 -- 172.21.15.139:0/3999764774 learned_addr learned my addr 172.21.15.139:0/3999764774 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:33.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.741+0000 7fcdc20c6640 1 --2- 172.21.15.139:0/3999764774 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcdbc1527f0 0x7fcdbc0c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:33.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.741+0000 7fcdc10c4640 1 -- 172.21.15.139:0/3999764774 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3248745577 0 0) 0x7fcdbc0707f0 con 0x7fcdbc06f7f0 2024-09-16T00:07:33.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.741+0000 7fcdc10c4640 1 -- 172.21.15.139:0/3999764774 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcda8003520 con 0x7fcdbc06f7f0 2024-09-16T00:07:33.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.742+0000 7fcdc10c4640 1 -- 172.21.15.139:0/3999764774 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1271896707 0 0) 0x7fcdbc070170 con 0x7fcdbc0c8250 2024-09-16T00:07:33.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.742+0000 7fcdc10c4640 1 -- 172.21.15.139:0/3999764774 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdbc0707f0 con 0x7fcdbc0c8250 2024-09-16T00:07:33.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.742+0000 7fcdc10c4640 1 -- 172.21.15.139:0/3999764774 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3847214801 0 0) 0x7fcda8003520 con 0x7fcdbc06f7f0 2024-09-16T00:07:33.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.742+0000 7fcdc20c6640 1 -- 172.21.15.139:0/3999764774 >> v1:172.21.15.164:6789/0 conn(0x7fcdbc0c8250 legacy=0x7fcdbc162ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:33.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.742+0000 7fcdc20c6640 1 -- 172.21.15.139:0/3999764774 >> v1:172.21.15.139:6789/0 conn(0x7fcdbc06f7f0 legacy=0x7fcdbc0ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:33.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.742+0000 7fcdc20c6640 1 -- 172.21.15.139:0/3999764774 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc070a90 con 0x7fcdbc1527f0 2024-09-16T00:07:33.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.742+0000 7fcdc20c6640 1 --2- 172.21.15.139:0/3999764774 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcdbc1527f0 0x7fcdbc0c4e80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80097d0 tx=0x7fcdb802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a726d743317a48d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:33.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.742+0000 7fcdc10c4640 1 -- 172.21.15.139:0/3999764774 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8037070 con 0x7fcdbc1527f0 2024-09-16T00:07:33.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.743+0000 7fcdc10c4640 1 -- 172.21.15.139:0/3999764774 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdb802ac30 con 0x7fcdbc1527f0 2024-09-16T00:07:33.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.743+0000 7fcdc10c4640 1 -- 172.21.15.139:0/3999764774 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac001670 con 0x7fcdbc1527f0 2024-09-16T00:07:33.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.743+0000 7fcdc432a640 1 -- 172.21.15.139:0/3999764774 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcdbc1527f0 msgr2=0x7fcdbc0c4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:33.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.743+0000 7fcdc432a640 1 --2- 172.21.15.139:0/3999764774 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcdbc1527f0 0x7fcdbc0c4e80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80097d0 tx=0x7fcdb802a1a0 comp rx=0 tx=0).stop 2024-09-16T00:07:33.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.743+0000 7fcdc432a640 1 -- 172.21.15.139:0/3999764774 shutdown_connections 2024-09-16T00:07:33.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.743+0000 7fcdc432a640 1 --2- 172.21.15.139:0/3999764774 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcdbc1527f0 0x7fcdbc0c4e80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:33.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.743+0000 7fcdc432a640 1 -- 172.21.15.139:0/3999764774 >> 172.21.15.139:0/3999764774 conn(0x7fcdbc06c150 msgr2=0x7fcdbc067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:33.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.743+0000 7fcdc432a640 1 -- 172.21.15.139:0/3999764774 shutdown_connections 2024-09-16T00:07:33.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.743+0000 7fcdc432a640 1 -- 172.21.15.139:0/3999764774 wait complete. 2024-09-16T00:07:33.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.744+0000 7fcdc432a640 1 Processor -- start 2024-09-16T00:07:33.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.744+0000 7fcdc432a640 1 -- start start 2024-09-16T00:07:33.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.745+0000 7fcdc432a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdbc06f7f0 0x7fcdbc14e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:33.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.745+0000 7fcdc432a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc0c8250 0x7fcdbc15adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:33.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.745+0000 7fcdc432a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcdbc1527f0 0x7fcdbc156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:33.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.745+0000 7fcdc432a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc0ce7d0 con 0x7fcdbc0c8250 2024-09-16T00:07:33.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.745+0000 7fcdc432a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc152bd0 con 0x7fcdbc06f7f0 2024-09-16T00:07:33.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.745+0000 7fcdc432a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcdbc0ce950 con 0x7fcdbc1527f0 2024-09-16T00:07:33.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.745+0000 7fcdc18c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc0c8250 0x7fcdbc15adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:33.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.745+0000 7fcdc18c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc0c8250 0x7fcdbc15adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46448/0 (socket says 172.21.15.139:46448) 2024-09-16T00:07:33.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.745+0000 7fcdc18c5640 1 -- 172.21.15.139:0/3592587898 learned_addr learned my addr 172.21.15.139:0/3592587898 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:33.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.745+0000 7fcdc20c6640 1 --2- 172.21.15.139:0/3592587898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdbc06f7f0 0x7fcdbc14e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:33.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.745+0000 7fcdc28c7640 1 --2- 172.21.15.139:0/3592587898 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcdbc1527f0 0x7fcdbc156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:33.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.745+0000 7fcdc18c5640 1 -- 172.21.15.139:0/3592587898 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcdbc1527f0 msgr2=0x7fcdbc156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:33.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.745+0000 7fcdc18c5640 1 --2- 172.21.15.139:0/3592587898 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcdbc1527f0 0x7fcdbc156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:33.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.745+0000 7fcdc18c5640 1 -- 172.21.15.139:0/3592587898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdbc06f7f0 msgr2=0x7fcdbc14e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:33.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.745+0000 7fcdc18c5640 1 --2- 172.21.15.139:0/3592587898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdbc06f7f0 0x7fcdbc14e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:33.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.745+0000 7fcdc18c5640 1 -- 172.21.15.139:0/3592587898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc158020 con 0x7fcdbc0c8250 2024-09-16T00:07:33.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.746+0000 7fcdc18c5640 1 --2- 172.21.15.139:0/3592587898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc0c8250 0x7fcdbc15adf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcdac000fd0 tx=0x7fcdac00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:33.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.746+0000 7fcdb2ffd640 1 -- 172.21.15.139:0/3592587898 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac017070 con 0x7fcdbc0c8250 2024-09-16T00:07:33.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.746+0000 7fcdb2ffd640 1 -- 172.21.15.139:0/3592587898 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdac005ce0 con 0x7fcdbc0c8250 2024-09-16T00:07:33.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.746+0000 7fcdb2ffd640 1 -- 172.21.15.139:0/3592587898 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac0072b0 con 0x7fcdbc0c8250 2024-09-16T00:07:33.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.746+0000 7fcdc432a640 1 -- 172.21.15.139:0/3592587898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdbc158300 con 0x7fcdbc0c8250 2024-09-16T00:07:33.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.746+0000 7fcdc432a640 1 -- 172.21.15.139:0/3592587898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdbc15c2f0 con 0x7fcdbc0c8250 2024-09-16T00:07:33.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.747+0000 7fcdc432a640 1 -- 172.21.15.139:0/3592587898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdbc0c4e80 con 0x7fcdbc0c8250 2024-09-16T00:07:33.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.747+0000 7fcdb2ffd640 1 -- 172.21.15.139:0/3592587898 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcdac01a070 con 0x7fcdbc0c8250 2024-09-16T00:07:33.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.748+0000 7fcdb2ffd640 1 --2- 172.21.15.139:0/3592587898 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcda006c2b0 0x7fcda006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:33.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.748+0000 7fcdb2ffd640 1 -- 172.21.15.139:0/3592587898 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcdac08a690 con 0x7fcdbc0c8250 2024-09-16T00:07:33.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.748+0000 7fcdc20c6640 1 --2- 172.21.15.139:0/3592587898 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcda006c2b0 0x7fcda006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:33.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.748+0000 7fcdc20c6640 1 --2- 172.21.15.139:0/3592587898 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcda006c2b0 0x7fcda006e770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8034000 tx=0x7fcdb802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:33.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.752+0000 7fcdb2ffd640 1 -- 172.21.15.139:0/3592587898 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcdac0587b0 con 0x7fcdbc0c8250 2024-09-16T00:07:33.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.921+0000 7fcdc432a640 1 -- 172.21.15.139:0/3592587898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcdbc0755e0 con 0x7fcdbc0c8250 2024-09-16T00:07:33.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.922+0000 7fcdb2ffd640 1 -- 172.21.15.139:0/3592587898 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16738 (secure 0 0 0) 0x7fcdac05cab0 con 0x7fcdbc0c8250 2024-09-16T00:07:33.923 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:33.924 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":21,"fsid":"2d3366e5-6a59-4b59-93b2-615e72ac7015","created":"2024-09-16T00:07:09.967833+0000","modified":"2024-09-16T00:07:32.949533+0000","last_up_change":"2024-09-16T00:07:21.049678+0000","last_in_change":"2024-09-16T00:07:12.025620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:07:22.056303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:07:30.768649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:07:32.008209+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83699a05-4937-43c3-aefc-b669bed9bb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6817","nonce":1974901438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6819","nonce":1974901438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6823","nonce":1974901438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6821","nonce":1974901438}]},"public_addr":"172.21.15.139:6817/1974901438","cluster_addr":"172.21.15.139:6819/1974901438","heartbeat_back_addr":"172.21.15.139:6823/1974901438","heartbeat_front_addr":"172.21.15.139:6821/1974901438","state":["exists","up"]},{"osd":1,"uuid":"9c04991e-9695-4bbb-8011-bf613bf404b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6809","nonce":2232380998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6811","nonce":2232380998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6815","nonce":2232380998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6813","nonce":2232380998}]},"public_addr":"172.21.15.139:6809/2232380998","cluster_addr":"172.21.15.139:6811/2232380998","heartbeat_back_addr":"172.21.15.139:6815/2232380998","heartbeat_front_addr":"172.21.15.139:6813/2232380998","state":["exists","up"]},{"osd":2,"uuid":"8048fc64-72b3-4866-ac62-6a8b2bf6f5c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6801","nonce":4180588905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6803","nonce":4180588905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6807","nonce":4180588905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6805","nonce":4180588905}]},"public_addr":"172.21.15.139:6801/4180588905","cluster_addr":"172.21.15.139:6803/4180588905","heartbeat_back_addr":"172.21.15.139:6807/4180588905","heartbeat_front_addr":"172.21.15.139:6805/4180588905","state":["exists","up"]},{"osd":3,"uuid":"c34ac77b-354c-4884-902d-3e2a84f5eb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6827","nonce":3947768492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6829","nonce":3947768492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6833","nonce":3947768492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6831","nonce":3947768492}]},"public_addr":"172.21.15.139:6827/3947768492","cluster_addr":"172.21.15.139:6829/3947768492","heartbeat_back_addr":"172.21.15.139:6833/3947768492","heartbeat_front_addr":"172.21.15.139:6831/3947768492","state":["exists","up"]},{"osd":4,"uuid":"ba9a7c93-9672-43e1-8c98-2878cdf7f548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6817","nonce":642632916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6819","nonce":642632916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6823","nonce":642632916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6821","nonce":642632916}]},"public_addr":"172.21.15.164:6817/642632916","cluster_addr":"172.21.15.164:6819/642632916","heartbeat_back_addr":"172.21.15.164:6823/642632916","heartbeat_front_addr":"172.21.15.164:6821/642632916","state":["exists","up"]},{"osd":5,"uuid":"46a0916b-474c-411b-84ad-86f27d4cbf53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6801","nonce":1051778383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6803","nonce":1051778383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6807","nonce":1051778383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6805","nonce":1051778383}]},"public_addr":"172.21.15.164:6801/1051778383","cluster_addr":"172.21.15.164:6803/1051778383","heartbeat_back_addr":"172.21.15.164:6807/1051778383","heartbeat_front_addr":"172.21.15.164:6805/1051778383","state":["exists","up"]},{"osd":6,"uuid":"9b6b33cc-d6d9-4a0a-8143-526073a58b93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6825","nonce":1908495807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6827","nonce":1908495807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6831","nonce":1908495807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6829","nonce":1908495807}]},"public_addr":"172.21.15.164:6825/1908495807","cluster_addr":"172.21.15.164:6827/1908495807","heartbeat_back_addr":"172.21.15.164:6831/1908495807","heartbeat_front_addr":"172.21.15.164:6829/1908495807","state":["exists","up"]},{"osd":7,"uuid":"50bcd808-3d3e-4a72-88e4-30decb53cfea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6809","nonce":1158838808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6811","nonce":1158838808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6815","nonce":1158838808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6813","nonce":1158838808}]},"public_addr":"172.21.15.164:6809/1158838808","cluster_addr":"172.21.15.164:6811/1158838808","heartbeat_back_addr":"172.21.15.164:6815/1158838808","heartbeat_front_addr":"172.21.15.164:6813/1158838808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.843117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.622789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:17.919457+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:19.330341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.815640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.083046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.551499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.377179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:07:33.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.924+0000 7fcdc432a640 1 -- 172.21.15.139:0/3592587898 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcda006c2b0 msgr2=0x7fcda006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:33.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.924+0000 7fcdc432a640 1 --2- 172.21.15.139:0/3592587898 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcda006c2b0 0x7fcda006e770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8034000 tx=0x7fcdb802e000 comp rx=0 tx=0).stop 2024-09-16T00:07:33.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.924+0000 7fcdc432a640 1 -- 172.21.15.139:0/3592587898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc0c8250 msgr2=0x7fcdbc15adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:33.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.924+0000 7fcdc432a640 1 --2- 172.21.15.139:0/3592587898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc0c8250 0x7fcdbc15adf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcdac000fd0 tx=0x7fcdac00c960 comp rx=0 tx=0).stop 2024-09-16T00:07:33.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.924+0000 7fcdc432a640 1 -- 172.21.15.139:0/3592587898 shutdown_connections 2024-09-16T00:07:33.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.924+0000 7fcdc432a640 1 --2- 172.21.15.139:0/3592587898 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcda006c2b0 0x7fcda006e770 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:33.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.924+0000 7fcdc432a640 1 --2- 172.21.15.139:0/3592587898 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcdbc1527f0 0x7fcdbc156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:33.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.924+0000 7fcdc432a640 1 --2- 172.21.15.139:0/3592587898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdbc0c8250 0x7fcdbc15adf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:33.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.924+0000 7fcdc432a640 1 --2- 172.21.15.139:0/3592587898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcdbc06f7f0 0x7fcdbc14e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:33.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.924+0000 7fcdc432a640 1 -- 172.21.15.139:0/3592587898 >> 172.21.15.139:0/3592587898 conn(0x7fcdbc06c150 msgr2=0x7fcdbc0ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:33.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.924+0000 7fcdc432a640 1 -- 172.21.15.139:0/3592587898 shutdown_connections 2024-09-16T00:07:33.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:33.924+0000 7fcdc432a640 1 -- 172.21.15.139:0/3592587898 wait complete. 2024-09-16T00:07:33.936 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:07:34.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.033+0000 7fc0f994b640 1 Processor -- start 2024-09-16T00:07:34.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.033+0000 7fc0f994b640 1 -- start start 2024-09-16T00:07:34.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.034+0000 7fc0f994b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc0f4152740 0x7fc0f40b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.034+0000 7fc0f994b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc0f4152b20 0x7fc0f40ca690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.034+0000 7fc0f994b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0f40c99e0 con 0x7fc0f4070650 2024-09-16T00:07:34.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.034+0000 7fc0f994b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fc0f415bbc0 con 0x7fc0f4152740 2024-09-16T00:07:34.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.034+0000 7fc0f994b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f40c9420 con 0x7fc0f4152b20 2024-09-16T00:07:34.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.034+0000 7fc0f37fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc0f4070650 0x7fc0f406fb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38150/0 (socket says 172.21.15.139:38150) 2024-09-16T00:07:34.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.034+0000 7fc0f37fe640 1 -- 172.21.15.139:0/1615693181 learned_addr learned my addr 172.21.15.139:0/1615693181 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:34.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.035+0000 7fc0f2ffd640 1 --2- 172.21.15.139:0/1615693181 >> v2:172.21.15.164:3300/0 conn(0x7fc0f4152740 0x7fc0f40b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.035+0000 7fc0f1ffb640 1 -- 172.21.15.139:0/1615693181 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2062063379 0 0) 0x7fc0f40c99e0 con 0x7fc0f4070650 2024-09-16T00:07:34.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.035+0000 7fc0f27fc640 1 --2- 172.21.15.139:0/1615693181 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc0f4152b20 0x7fc0f40ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.035+0000 7fc0f1ffb640 1 -- 172.21.15.139:0/1615693181 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0dc003520 con 0x7fc0f4070650 2024-09-16T00:07:34.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.035+0000 7fc0f2ffd640 1 -- 172.21.15.139:0/1615693181 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc0f4152b20 msgr2=0x7fc0f40ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:34.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.035+0000 7fc0f2ffd640 1 --2- 172.21.15.139:0/1615693181 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc0f4152b20 0x7fc0f40ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.035+0000 7fc0f1ffb640 1 -- 172.21.15.139:0/1615693181 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2657134863 0 0) 0x7fc0dc003520 con 0x7fc0f4070650 2024-09-16T00:07:34.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.035+0000 7fc0f2ffd640 1 -- 172.21.15.139:0/1615693181 >> v1:172.21.15.139:6789/0 conn(0x7fc0f4070650 legacy=0x7fc0f406fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:34.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.035+0000 7fc0f2ffd640 1 -- 172.21.15.139:0/1615693181 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f40c9700 con 0x7fc0f4152740 2024-09-16T00:07:34.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.036+0000 7fc0f2ffd640 1 --2- 172.21.15.139:0/1615693181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0f4152740 0x7fc0f40b0fd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009d90 tx=0x7fc0e402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fd6ad280eab301d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:34.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.036+0000 7fc0f1ffb640 1 -- 172.21.15.139:0/1615693181 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec001670 con 0x7fc0f4152740 2024-09-16T00:07:34.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.036+0000 7fc0f1ffb640 1 -- 172.21.15.139:0/1615693181 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e4007540 con 0x7fc0f4152740 2024-09-16T00:07:34.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.036+0000 7fc0f1ffb640 1 -- 172.21.15.139:0/1615693181 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4007910 con 0x7fc0f4152740 2024-09-16T00:07:34.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.036+0000 7fc0f994b640 1 -- 172.21.15.139:0/1615693181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0f4152740 msgr2=0x7fc0f40b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:34.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.036+0000 7fc0f994b640 1 --2- 172.21.15.139:0/1615693181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0f4152740 0x7fc0f40b0fd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009d90 tx=0x7fc0e402a680 comp rx=0 tx=0).stop 2024-09-16T00:07:34.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.036+0000 7fc0f994b640 1 -- 172.21.15.139:0/1615693181 shutdown_connections 2024-09-16T00:07:34.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.036+0000 7fc0f994b640 1 --2- 172.21.15.139:0/1615693181 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc0f4152b20 0x7fc0f40ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.036+0000 7fc0f994b640 1 --2- 172.21.15.139:0/1615693181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0f4152740 0x7fc0f40b0fd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.036+0000 7fc0f994b640 1 -- 172.21.15.139:0/1615693181 >> 172.21.15.139:0/1615693181 conn(0x7fc0f4067d70 msgr2=0x7fc0f4068180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:34.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.036+0000 7fc0f994b640 1 -- 172.21.15.139:0/1615693181 shutdown_connections 2024-09-16T00:07:34.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.036+0000 7fc0f994b640 1 -- 172.21.15.139:0/1615693181 wait complete. 2024-09-16T00:07:34.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.037+0000 7fc0f994b640 1 Processor -- start 2024-09-16T00:07:34.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.038+0000 7fc0f994b640 1 -- start start 2024-09-16T00:07:34.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.038+0000 7fc0f994b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0f4070650 0x7fc0f414a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.038+0000 7fc0f994b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc0f4152740 0x7fc0f414c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.038+0000 7fc0f994b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0f4152b20 0x7fc0f41489a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.038+0000 7fc0f994b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f4070260 con 0x7fc0f4070650 2024-09-16T00:07:34.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.038+0000 7fc0f994b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f415c140 con 0x7fc0f4152b20 2024-09-16T00:07:34.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.038+0000 7fc0f994b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f40703e0 con 0x7fc0f4152740 2024-09-16T00:07:34.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.038+0000 7fc0f2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0f4070650 0x7fc0f414a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.038+0000 7fc0f2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0f4070650 0x7fc0f414a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46462/0 (socket says 172.21.15.139:46462) 2024-09-16T00:07:34.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.038+0000 7fc0f2ffd640 1 -- 172.21.15.139:0/185998925 learned_addr learned my addr 172.21.15.139:0/185998925 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.038+0000 7fc0f37fe640 1 --2- 172.21.15.139:0/185998925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0f4152b20 0x7fc0f41489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.038+0000 7fc0f27fc640 1 --2- 172.21.15.139:0/185998925 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc0f4152740 0x7fc0f414c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.039+0000 7fc0f2ffd640 1 -- 172.21.15.139:0/185998925 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc0f4152740 msgr2=0x7fc0f414c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.039+0000 7fc0f2ffd640 1 --2- 172.21.15.139:0/185998925 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc0f4152740 0x7fc0f414c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.039+0000 7fc0f2ffd640 1 -- 172.21.15.139:0/185998925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0f4152b20 msgr2=0x7fc0f41489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.039+0000 7fc0f2ffd640 1 --2- 172.21.15.139:0/185998925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0f4152b20 0x7fc0f41489a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.039+0000 7fc0f2ffd640 1 -- 172.21.15.139:0/185998925 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f414e2d0 con 0x7fc0f4070650 2024-09-16T00:07:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.039+0000 7fc0f2ffd640 1 --2- 172.21.15.139:0/185998925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0f4070650 0x7fc0f414a130 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009eb0 tx=0x7fc0e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.039+0000 7fc0dbfff640 1 -- 172.21.15.139:0/185998925 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4040070 con 0x7fc0f4070650 2024-09-16T00:07:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.039+0000 7fc0dbfff640 1 -- 172.21.15.139:0/185998925 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e4031c40 con 0x7fc0f4070650 2024-09-16T00:07:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.039+0000 7fc0f994b640 1 -- 172.21.15.139:0/185998925 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f406cfe0 con 0x7fc0f4070650 2024-09-16T00:07:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.039+0000 7fc0dbfff640 1 -- 172.21.15.139:0/185998925 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4009450 con 0x7fc0f4070650 2024-09-16T00:07:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.039+0000 7fc0f994b640 1 -- 172.21.15.139:0/185998925 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f414e570 con 0x7fc0f4070650 2024-09-16T00:07:34.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.040+0000 7fc0f994b640 1 -- 172.21.15.139:0/185998925 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f40c3980 con 0x7fc0f4070650 2024-09-16T00:07:34.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.040+0000 7fc0dbfff640 1 -- 172.21.15.139:0/185998925 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc0e4039040 con 0x7fc0f4070650 2024-09-16T00:07:34.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.041+0000 7fc0dbfff640 1 --2- 172.21.15.139:0/185998925 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc0e006c2b0 0x7fc0e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.041+0000 7fc0dbfff640 1 -- 172.21.15.139:0/185998925 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc0e40acab0 con 0x7fc0f4070650 2024-09-16T00:07:34.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.041+0000 7fc0f27fc640 1 --2- 172.21.15.139:0/185998925 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc0e006c2b0 0x7fc0e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.041+0000 7fc0f27fc640 1 --2- 172.21.15.139:0/185998925 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc0e006c2b0 0x7fc0e006e770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc0f41eda90 tx=0x7fc0e8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:34.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.045+0000 7fc0dbfff640 1 -- 172.21.15.139:0/185998925 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc0f40c3980 con 0x7fc0f4070650 2024-09-16T00:07:34.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.212+0000 7fc0f994b640 1 -- 172.21.15.139:0/185998925 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0f40ceed0 con 0x7fc0f4070650 2024-09-16T00:07:34.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.213+0000 7fc0dbfff640 1 -- 172.21.15.139:0/185998925 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16738 (secure 0 0 0) 0x7fc0e407abd0 con 0x7fc0f4070650 2024-09-16T00:07:34.214 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:34.215 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":21,"fsid":"2d3366e5-6a59-4b59-93b2-615e72ac7015","created":"2024-09-16T00:07:09.967833+0000","modified":"2024-09-16T00:07:32.949533+0000","last_up_change":"2024-09-16T00:07:21.049678+0000","last_in_change":"2024-09-16T00:07:12.025620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:07:22.056303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:07:30.768649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:07:32.008209+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83699a05-4937-43c3-aefc-b669bed9bb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6817","nonce":1974901438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6819","nonce":1974901438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6823","nonce":1974901438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6821","nonce":1974901438}]},"public_addr":"172.21.15.139:6817/1974901438","cluster_addr":"172.21.15.139:6819/1974901438","heartbeat_back_addr":"172.21.15.139:6823/1974901438","heartbeat_front_addr":"172.21.15.139:6821/1974901438","state":["exists","up"]},{"osd":1,"uuid":"9c04991e-9695-4bbb-8011-bf613bf404b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6809","nonce":2232380998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6811","nonce":2232380998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6815","nonce":2232380998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6813","nonce":2232380998}]},"public_addr":"172.21.15.139:6809/2232380998","cluster_addr":"172.21.15.139:6811/2232380998","heartbeat_back_addr":"172.21.15.139:6815/2232380998","heartbeat_front_addr":"172.21.15.139:6813/2232380998","state":["exists","up"]},{"osd":2,"uuid":"8048fc64-72b3-4866-ac62-6a8b2bf6f5c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6801","nonce":4180588905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6803","nonce":4180588905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6807","nonce":4180588905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6805","nonce":4180588905}]},"public_addr":"172.21.15.139:6801/4180588905","cluster_addr":"172.21.15.139:6803/4180588905","heartbeat_back_addr":"172.21.15.139:6807/4180588905","heartbeat_front_addr":"172.21.15.139:6805/4180588905","state":["exists","up"]},{"osd":3,"uuid":"c34ac77b-354c-4884-902d-3e2a84f5eb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6827","nonce":3947768492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6829","nonce":3947768492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6833","nonce":3947768492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6831","nonce":3947768492}]},"public_addr":"172.21.15.139:6827/3947768492","cluster_addr":"172.21.15.139:6829/3947768492","heartbeat_back_addr":"172.21.15.139:6833/3947768492","heartbeat_front_addr":"172.21.15.139:6831/3947768492","state":["exists","up"]},{"osd":4,"uuid":"ba9a7c93-9672-43e1-8c98-2878cdf7f548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6817","nonce":642632916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6819","nonce":642632916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6823","nonce":642632916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6821","nonce":642632916}]},"public_addr":"172.21.15.164:6817/642632916","cluster_addr":"172.21.15.164:6819/642632916","heartbeat_back_addr":"172.21.15.164:6823/642632916","heartbeat_front_addr":"172.21.15.164:6821/642632916","state":["exists","up"]},{"osd":5,"uuid":"46a0916b-474c-411b-84ad-86f27d4cbf53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6801","nonce":1051778383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6803","nonce":1051778383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6807","nonce":1051778383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6805","nonce":1051778383}]},"public_addr":"172.21.15.164:6801/1051778383","cluster_addr":"172.21.15.164:6803/1051778383","heartbeat_back_addr":"172.21.15.164:6807/1051778383","heartbeat_front_addr":"172.21.15.164:6805/1051778383","state":["exists","up"]},{"osd":6,"uuid":"9b6b33cc-d6d9-4a0a-8143-526073a58b93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6825","nonce":1908495807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6827","nonce":1908495807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6831","nonce":1908495807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6829","nonce":1908495807}]},"public_addr":"172.21.15.164:6825/1908495807","cluster_addr":"172.21.15.164:6827/1908495807","heartbeat_back_addr":"172.21.15.164:6831/1908495807","heartbeat_front_addr":"172.21.15.164:6829/1908495807","state":["exists","up"]},{"osd":7,"uuid":"50bcd808-3d3e-4a72-88e4-30decb53cfea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6809","nonce":1158838808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6811","nonce":1158838808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6815","nonce":1158838808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6813","nonce":1158838808}]},"public_addr":"172.21.15.164:6809/1158838808","cluster_addr":"172.21.15.164:6811/1158838808","heartbeat_back_addr":"172.21.15.164:6815/1158838808","heartbeat_front_addr":"172.21.15.164:6813/1158838808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.843117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.622789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:17.919457+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:19.330341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.815640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.083046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.551499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.377179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:07:34.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.215+0000 7fc0f994b640 1 -- 172.21.15.139:0/185998925 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc0e006c2b0 msgr2=0x7fc0e006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:34.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.215+0000 7fc0f994b640 1 --2- 172.21.15.139:0/185998925 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc0e006c2b0 0x7fc0e006e770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc0f41eda90 tx=0x7fc0e8009190 comp rx=0 tx=0).stop 2024-09-16T00:07:34.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.215+0000 7fc0f994b640 1 -- 172.21.15.139:0/185998925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0f4070650 msgr2=0x7fc0f414a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:34.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.215+0000 7fc0f994b640 1 --2- 172.21.15.139:0/185998925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0f4070650 0x7fc0f414a130 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009eb0 tx=0x7fc0e4004a40 comp rx=0 tx=0).stop 2024-09-16T00:07:34.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.215+0000 7fc0f994b640 1 -- 172.21.15.139:0/185998925 shutdown_connections 2024-09-16T00:07:34.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.215+0000 7fc0f994b640 1 --2- 172.21.15.139:0/185998925 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc0e006c2b0 0x7fc0e006e770 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.215+0000 7fc0f994b640 1 --2- 172.21.15.139:0/185998925 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc0f4152b20 0x7fc0f41489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.215+0000 7fc0f994b640 1 --2- 172.21.15.139:0/185998925 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc0f4152740 0x7fc0f414c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.215+0000 7fc0f994b640 1 --2- 172.21.15.139:0/185998925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0f4070650 0x7fc0f414a130 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.215+0000 7fc0f994b640 1 -- 172.21.15.139:0/185998925 >> 172.21.15.139:0/185998925 conn(0x7fc0f4067d70 msgr2=0x7fc0f40c3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:34.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.216+0000 7fc0f994b640 1 -- 172.21.15.139:0/185998925 shutdown_connections 2024-09-16T00:07:34.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.216+0000 7fc0f994b640 1 -- 172.21.15.139:0/185998925 wait complete. 2024-09-16T00:07:34.231 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-16T00:07:34.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.331+0000 7fcf7b55f640 1 Processor -- start 2024-09-16T00:07:34.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.332+0000 7fcf7b55f640 1 -- start start 2024-09-16T00:07:34.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.332+0000 7fcf7b55f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf74154bd0 0x7fcf7415b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.332+0000 7fcf7b55f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcf7406ea20 0x7fcf740c3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.332+0000 7fcf7b55f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcf740c8b50 0x7fcf7415e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.332+0000 7fcf7b55f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcf7415ed80 con 0x7fcf740c8b50 2024-09-16T00:07:34.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.332+0000 7fcf7b55f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fcf7406cd40 con 0x7fcf7406ea20 2024-09-16T00:07:34.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.332+0000 7fcf7b55f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7415a9d0 con 0x7fcf74154bd0 2024-09-16T00:07:34.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.333+0000 7fcf79afc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcf740c8b50 0x7fcf7415e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.333+0000 7fcf79afc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcf740c8b50 0x7fcf7415e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46468/0 (socket says 172.21.15.139:46468) 2024-09-16T00:07:34.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.333+0000 7fcf79afc640 1 -- 172.21.15.139:0/2747148821 learned_addr learned my addr 172.21.15.139:0/2747148821 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:34.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.333+0000 7fcf78afa640 1 --2- 172.21.15.139:0/2747148821 >> v2:172.21.15.164:3300/0 conn(0x7fcf7406ea20 0x7fcf740c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.333+0000 7fcf79afc640 1 -- 172.21.15.139:0/2747148821 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf74154bd0 msgr2=0x7fcf7415b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:34.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.333+0000 7fcf79afc640 1 --2- 172.21.15.139:0/2747148821 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf74154bd0 0x7fcf7415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.333+0000 7fcf79afc640 1 -- 172.21.15.139:0/2747148821 >> v2:172.21.15.164:3300/0 conn(0x7fcf7406ea20 msgr2=0x7fcf740c3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:34.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.333+0000 7fcf79afc640 1 --2- 172.21.15.139:0/2747148821 >> v2:172.21.15.164:3300/0 conn(0x7fcf7406ea20 0x7fcf740c3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.333+0000 7fcf79afc640 1 -- 172.21.15.139:0/2747148821 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7406cfc0 con 0x7fcf740c8b50 2024-09-16T00:07:34.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.333+0000 7fcf79afc640 1 --2- 172.21.15.139:0/2747148821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf740c8b50 0x7fcf7415e840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcf74068820 tx=0x7fcf6402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1210924bb3a94dc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:34.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.334+0000 7fcf73fff640 1 -- 172.21.15.139:0/2747148821 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64036070 con 0x7fcf740c8b50 2024-09-16T00:07:34.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.334+0000 7fcf73fff640 1 -- 172.21.15.139:0/2747148821 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf64031600 con 0x7fcf740c8b50 2024-09-16T00:07:34.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.334+0000 7fcf73fff640 1 -- 172.21.15.139:0/2747148821 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf640319d0 con 0x7fcf740c8b50 2024-09-16T00:07:34.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.334+0000 7fcf7b55f640 1 -- 172.21.15.139:0/2747148821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf740c8b50 msgr2=0x7fcf7415e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:34.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.334+0000 7fcf7b55f640 1 --2- 172.21.15.139:0/2747148821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf740c8b50 0x7fcf7415e840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcf74068820 tx=0x7fcf6402a6a0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.334+0000 7fcf7b55f640 1 -- 172.21.15.139:0/2747148821 shutdown_connections 2024-09-16T00:07:34.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.334+0000 7fcf7b55f640 1 --2- 172.21.15.139:0/2747148821 >> v2:172.21.15.164:3300/0 conn(0x7fcf7406ea20 0x7fcf740c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.334+0000 7fcf7b55f640 1 --2- 172.21.15.139:0/2747148821 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf740c8b50 0x7fcf7415e840 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.334+0000 7fcf7b55f640 1 --2- 172.21.15.139:0/2747148821 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf74154bd0 0x7fcf7415b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.334+0000 7fcf7b55f640 1 -- 172.21.15.139:0/2747148821 >> 172.21.15.139:0/2747148821 conn(0x7fcf7406c150 msgr2=0x7fcf74067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:34.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.334+0000 7fcf7b55f640 1 -- 172.21.15.139:0/2747148821 shutdown_connections 2024-09-16T00:07:34.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.334+0000 7fcf7b55f640 1 -- 172.21.15.139:0/2747148821 wait complete. 2024-09-16T00:07:34.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.335+0000 7fcf7b55f640 1 Processor -- start 2024-09-16T00:07:34.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.335+0000 7fcf7b55f640 1 -- start start 2024-09-16T00:07:34.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.336+0000 7fcf7b55f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf7406ea20 0x7fcf741f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.336+0000 7fcf7b55f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf740c8b50 0x7fcf740c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.336+0000 7fcf7b55f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf74154bd0 0x7fcf740c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.336+0000 7fcf7b55f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74154fb0 con 0x7fcf74154bd0 2024-09-16T00:07:34.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.336+0000 7fcf7b55f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7415af50 con 0x7fcf7406ea20 2024-09-16T00:07:34.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.336+0000 7fcf7b55f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7406d2a0 con 0x7fcf740c8b50 2024-09-16T00:07:34.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.336+0000 7fcf79afc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf74154bd0 0x7fcf740c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.336+0000 7fcf79afc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf74154bd0 0x7fcf740c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46482/0 (socket says 172.21.15.139:46482) 2024-09-16T00:07:34.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.336+0000 7fcf79afc640 1 -- 172.21.15.139:0/1975095501 learned_addr learned my addr 172.21.15.139:0/1975095501 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:34.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.336+0000 7fcf79afc640 1 -- 172.21.15.139:0/1975095501 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf740c8b50 msgr2=0x7fcf740c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:34.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.336+0000 7fcf78afa640 1 --2- 172.21.15.139:0/1975095501 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf740c8b50 0x7fcf740c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.336+0000 7fcf792fb640 1 --2- 172.21.15.139:0/1975095501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf7406ea20 0x7fcf741f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.336+0000 7fcf79afc640 1 --2- 172.21.15.139:0/1975095501 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf740c8b50 0x7fcf740c39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.336+0000 7fcf79afc640 1 -- 172.21.15.139:0/1975095501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf7406ea20 msgr2=0x7fcf741f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:34.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.336+0000 7fcf79afc640 1 --2- 172.21.15.139:0/1975095501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf7406ea20 0x7fcf741f45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.336+0000 7fcf79afc640 1 -- 172.21.15.139:0/1975095501 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf740caff0 con 0x7fcf74154bd0 2024-09-16T00:07:34.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.337+0000 7fcf79afc640 1 --2- 172.21.15.139:0/1975095501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf74154bd0 0x7fcf740c9460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcf6402ac90 tx=0x7fcf64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:34.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.337+0000 7fcf71ffb640 1 -- 172.21.15.139:0/1975095501 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64041020 con 0x7fcf74154bd0 2024-09-16T00:07:34.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.337+0000 7fcf71ffb640 1 -- 172.21.15.139:0/1975095501 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf64007800 con 0x7fcf74154bd0 2024-09-16T00:07:34.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.337+0000 7fcf7b55f640 1 -- 172.21.15.139:0/1975095501 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7415a5e0 con 0x7fcf74154bd0 2024-09-16T00:07:34.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.337+0000 7fcf71ffb640 1 -- 172.21.15.139:0/1975095501 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64036040 con 0x7fcf74154bd0 2024-09-16T00:07:34.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.337+0000 7fcf7b55f640 1 -- 172.21.15.139:0/1975095501 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf740cad10 con 0x7fcf74154bd0 2024-09-16T00:07:34.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.338+0000 7fcf7b55f640 1 -- 172.21.15.139:0/1975095501 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf740c3170 con 0x7fcf74154bd0 2024-09-16T00:07:34.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.338+0000 7fcf71ffb640 1 -- 172.21.15.139:0/1975095501 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcf64044020 con 0x7fcf74154bd0 2024-09-16T00:07:34.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.339+0000 7fcf71ffb640 1 --2- 172.21.15.139:0/1975095501 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcf6006c2b0 0x7fcf6006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.339+0000 7fcf71ffb640 1 -- 172.21.15.139:0/1975095501 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcf640ac370 con 0x7fcf74154bd0 2024-09-16T00:07:34.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.339+0000 7fcf792fb640 1 --2- 172.21.15.139:0/1975095501 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcf6006c2b0 0x7fcf6006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.339+0000 7fcf792fb640 1 --2- 172.21.15.139:0/1975095501 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcf6006c2b0 0x7fcf6006e770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00a770 tx=0x7fcf6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:34.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.343+0000 7fcf71ffb640 1 -- 172.21.15.139:0/1975095501 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcf6404b030 con 0x7fcf74154bd0 2024-09-16T00:07:34.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.545+0000 7fcf7b55f640 1 -- 172.21.15.139:0/1975095501 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcf740cb1c0 con 0x7fcf74154bd0 2024-09-16T00:07:34.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.623+0000 7fcf71ffb640 1 -- 172.21.15.139:0/1975095501 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7fcf7415a5e0 con 0x7fcf74154bd0 2024-09-16T00:07:34.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.626+0000 7fcf7b55f640 1 -- 172.21.15.139:0/1975095501 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcf6006c2b0 msgr2=0x7fcf6006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:34.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.626+0000 7fcf7b55f640 1 --2- 172.21.15.139:0/1975095501 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcf6006c2b0 0x7fcf6006e770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00a770 tx=0x7fcf6c008040 comp rx=0 tx=0).stop 2024-09-16T00:07:34.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.626+0000 7fcf7b55f640 1 -- 172.21.15.139:0/1975095501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf74154bd0 msgr2=0x7fcf740c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:34.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.626+0000 7fcf7b55f640 1 --2- 172.21.15.139:0/1975095501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf74154bd0 0x7fcf740c9460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcf6402ac90 tx=0x7fcf64004a40 comp rx=0 tx=0).stop 2024-09-16T00:07:34.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.626+0000 7fcf7b55f640 1 -- 172.21.15.139:0/1975095501 shutdown_connections 2024-09-16T00:07:34.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.626+0000 7fcf7b55f640 1 --2- 172.21.15.139:0/1975095501 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcf6006c2b0 0x7fcf6006e770 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.626+0000 7fcf7b55f640 1 --2- 172.21.15.139:0/1975095501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf74154bd0 0x7fcf740c9460 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.626+0000 7fcf7b55f640 1 --2- 172.21.15.139:0/1975095501 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcf740c8b50 0x7fcf740c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.626+0000 7fcf7b55f640 1 --2- 172.21.15.139:0/1975095501 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcf7406ea20 0x7fcf741f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.626+0000 7fcf7b55f640 1 -- 172.21.15.139:0/1975095501 >> 172.21.15.139:0/1975095501 conn(0x7fcf7406c150 msgr2=0x7fcf74067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:34.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.626+0000 7fcf7b55f640 1 -- 172.21.15.139:0/1975095501 shutdown_connections 2024-09-16T00:07:34.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.626+0000 7fcf7b55f640 1 -- 172.21.15.139:0/1975095501 wait complete. 2024-09-16T00:07:34.638 DEBUG:tasks.cephfs.filesystem:fs_config: {'session_timeout': 300} 2024-09-16T00:07:34.639 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-16T00:07:34.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.739+0000 7fcda0fd8640 1 Processor -- start 2024-09-16T00:07:34.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.739+0000 7fcda0fd8640 1 -- start start 2024-09-16T00:07:34.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.739+0000 7fcda0fd8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcd9c0c6a20 0x7fcd9c06f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.739+0000 7fcda0fd8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd9c0c6e00 0x7fcd9c15bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.739+0000 7fcda0fd8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcd9c06ea70 0x7fcd9c0c9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.739+0000 7fcda0fd8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9c06f520 con 0x7fcd9c06ea70 2024-09-16T00:07:34.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.739+0000 7fcda0fd8640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9c06f210 con 0x7fcd9c0c6a20 2024-09-16T00:07:34.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.739+0000 7fcda0fd8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9c06ef20 con 0x7fcd9c0c6e00 2024-09-16T00:07:34.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.740+0000 7fcd9ad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcd9c06ea70 0x7fcd9c0c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.740+0000 7fcd9ad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcd9c06ea70 0x7fcd9c0c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46490/0 (socket says 172.21.15.139:46490) 2024-09-16T00:07:34.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.740+0000 7fcd9ad9d640 1 -- 172.21.15.139:0/2084467306 learned_addr learned my addr 172.21.15.139:0/2084467306 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:34.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.740+0000 7fcd9a59c640 1 --2- 172.21.15.139:0/2084467306 >> v2:172.21.15.164:3300/0 conn(0x7fcd9c0c6a20 0x7fcd9c06f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.740+0000 7fcd9ad9d640 1 -- 172.21.15.139:0/2084467306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd9c0c6e00 msgr2=0x7fcd9c15bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:34.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.740+0000 7fcd9ad9d640 1 --2- 172.21.15.139:0/2084467306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd9c0c6e00 0x7fcd9c15bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.740+0000 7fcd9ad9d640 1 -- 172.21.15.139:0/2084467306 >> v2:172.21.15.164:3300/0 conn(0x7fcd9c0c6a20 msgr2=0x7fcd9c06f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:34.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.740+0000 7fcd9ad9d640 1 --2- 172.21.15.139:0/2084467306 >> v2:172.21.15.164:3300/0 conn(0x7fcd9c0c6a20 0x7fcd9c06f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.740+0000 7fcd9ad9d640 1 -- 172.21.15.139:0/2084467306 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c0ca080 con 0x7fcd9c06ea70 2024-09-16T00:07:34.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.740+0000 7fcd9a59c640 1 --2- 172.21.15.139:0/2084467306 >> v2:172.21.15.164:3300/0 conn(0x7fcd9c0c6a20 0x7fcd9c06f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:34.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.740+0000 7fcd9ad9d640 1 --2- 172.21.15.139:0/2084467306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd9c06ea70 0x7fcd9c0c9530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c071490 tx=0x7fcd9402a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e85888947788744 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:34.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.741+0000 7fcd99d9b640 1 -- 172.21.15.139:0/2084467306 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94036070 con 0x7fcd9c06ea70 2024-09-16T00:07:34.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.741+0000 7fcd99d9b640 1 -- 172.21.15.139:0/2084467306 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd94031600 con 0x7fcd9c06ea70 2024-09-16T00:07:34.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.741+0000 7fcd99d9b640 1 -- 172.21.15.139:0/2084467306 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd940319d0 con 0x7fcd9c06ea70 2024-09-16T00:07:34.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.741+0000 7fcda0fd8640 1 -- 172.21.15.139:0/2084467306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd9c06ea70 msgr2=0x7fcd9c0c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:34.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.741+0000 7fcda0fd8640 1 --2- 172.21.15.139:0/2084467306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd9c06ea70 0x7fcd9c0c9530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c071490 tx=0x7fcd9402a750 comp rx=0 tx=0).stop 2024-09-16T00:07:34.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.741+0000 7fcda0fd8640 1 -- 172.21.15.139:0/2084467306 shutdown_connections 2024-09-16T00:07:34.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.741+0000 7fcda0fd8640 1 --2- 172.21.15.139:0/2084467306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd9c06ea70 0x7fcd9c0c9530 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.741+0000 7fcda0fd8640 1 --2- 172.21.15.139:0/2084467306 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd9c0c6e00 0x7fcd9c15bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.741+0000 7fcda0fd8640 1 --2- 172.21.15.139:0/2084467306 >> v2:172.21.15.164:3300/0 conn(0x7fcd9c0c6a20 0x7fcd9c06f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.741+0000 7fcda0fd8640 1 -- 172.21.15.139:0/2084467306 >> 172.21.15.139:0/2084467306 conn(0x7fcd9c06cc30 msgr2=0x7fcd9c06d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:34.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.741+0000 7fcda0fd8640 1 -- 172.21.15.139:0/2084467306 shutdown_connections 2024-09-16T00:07:34.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.741+0000 7fcda0fd8640 1 -- 172.21.15.139:0/2084467306 wait complete. 2024-09-16T00:07:34.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.743+0000 7fcda0fd8640 1 Processor -- start 2024-09-16T00:07:34.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.743+0000 7fcda0fd8640 1 -- start start 2024-09-16T00:07:34.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.743+0000 7fcda0fd8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd9c06ea70 0x7fcd9c156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.743+0000 7fcda0fd8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd9c0c6a20 0x7fcd9c1ef480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.743+0000 7fcda0fd8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd9c0c6e00 0x7fcd9c0cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.743+0000 7fcda0fd8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c0ca600 con 0x7fcd9c0c6e00 2024-09-16T00:07:34.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.743+0000 7fcda0fd8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c0ca320 con 0x7fcd9c06ea70 2024-09-16T00:07:34.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.743+0000 7fcda0fd8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9c0ca8e0 con 0x7fcd9c0c6a20 2024-09-16T00:07:34.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.743+0000 7fcd9ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd9c0c6e00 0x7fcd9c0cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.743+0000 7fcd9ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd9c0c6e00 0x7fcd9c0cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46502/0 (socket says 172.21.15.139:46502) 2024-09-16T00:07:34.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.743+0000 7fcd9ad9d640 1 -- 172.21.15.139:0/3959814417 learned_addr learned my addr 172.21.15.139:0/3959814417 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:34.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.744+0000 7fcd9a59c640 1 --2- 172.21.15.139:0/3959814417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd9c06ea70 0x7fcd9c156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.744+0000 7fcd91d9b640 1 --2- 172.21.15.139:0/3959814417 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd9c0c6a20 0x7fcd9c1ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.744+0000 7fcd9ad9d640 1 -- 172.21.15.139:0/3959814417 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd9c0c6a20 msgr2=0x7fcd9c1ef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:34.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.744+0000 7fcd9ad9d640 1 --2- 172.21.15.139:0/3959814417 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd9c0c6a20 0x7fcd9c1ef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.744+0000 7fcd9ad9d640 1 -- 172.21.15.139:0/3959814417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd9c06ea70 msgr2=0x7fcd9c156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:34.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.744+0000 7fcd9ad9d640 1 --2- 172.21.15.139:0/3959814417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd9c06ea70 0x7fcd9c156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:34.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.744+0000 7fcd9ad9d640 1 -- 172.21.15.139:0/3959814417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c15ae60 con 0x7fcd9c0c6e00 2024-09-16T00:07:34.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.744+0000 7fcd9ad9d640 1 --2- 172.21.15.139:0/3959814417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd9c0c6e00 0x7fcd9c0cd250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcd94034040 tx=0x7fcd94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:34.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.744+0000 7fcd937fe640 1 -- 172.21.15.139:0/3959814417 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94036050 con 0x7fcd9c0c6e00 2024-09-16T00:07:34.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.744+0000 7fcd937fe640 1 -- 172.21.15.139:0/3959814417 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd94042070 con 0x7fcd9c0c6e00 2024-09-16T00:07:34.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.744+0000 7fcd937fe640 1 -- 172.21.15.139:0/3959814417 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94009510 con 0x7fcd9c0c6e00 2024-09-16T00:07:34.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.744+0000 7fcda0fd8640 1 -- 172.21.15.139:0/3959814417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9c0cbac0 con 0x7fcd9c0c6e00 2024-09-16T00:07:34.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.744+0000 7fcda0fd8640 1 -- 172.21.15.139:0/3959814417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9c0cc080 con 0x7fcd9c0c6e00 2024-09-16T00:07:34.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.745+0000 7fcda0fd8640 1 -- 172.21.15.139:0/3959814417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9c06f9b0 con 0x7fcd9c0c6e00 2024-09-16T00:07:34.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.745+0000 7fcd937fe640 1 -- 172.21.15.139:0/3959814417 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcd94007bf0 con 0x7fcd9c0c6e00 2024-09-16T00:07:34.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.746+0000 7fcd937fe640 1 --2- 172.21.15.139:0/3959814417 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcd7c06c2e0 0x7fcd7c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:34.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.746+0000 7fcd937fe640 1 -- 172.21.15.139:0/3959814417 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcd940abcf0 con 0x7fcd9c0c6e00 2024-09-16T00:07:34.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.746+0000 7fcd9a59c640 1 --2- 172.21.15.139:0/3959814417 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcd7c06c2e0 0x7fcd7c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:34.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.746+0000 7fcd9a59c640 1 --2- 172.21.15.139:0/3959814417 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcd7c06c2e0 0x7fcd7c06e7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00a990 tx=0x7fcd8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:34.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.750+0000 7fcd937fe640 1 -- 172.21.15.139:0/3959814417 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcd940b0070 con 0x7fcd9c0c6e00 2024-09-16T00:07:34.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:34.949+0000 7fcda0fd8640 1 -- 172.21.15.139:0/3959814417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fcd9c06c8c0 con 0x7fcd9c0c6e00 2024-09-16T00:07:35.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.628+0000 7fcd937fe640 1 -- 172.21.15.139:0/3959814417 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7fcd9404b020 con 0x7fcd9c0c6e00 2024-09-16T00:07:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.631+0000 7fcda0fd8640 1 -- 172.21.15.139:0/3959814417 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcd7c06c2e0 msgr2=0x7fcd7c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.631+0000 7fcda0fd8640 1 --2- 172.21.15.139:0/3959814417 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcd7c06c2e0 0x7fcd7c06e7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00a990 tx=0x7fcd8c008040 comp rx=0 tx=0).stop 2024-09-16T00:07:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.631+0000 7fcda0fd8640 1 -- 172.21.15.139:0/3959814417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd9c0c6e00 msgr2=0x7fcd9c0cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.631+0000 7fcda0fd8640 1 --2- 172.21.15.139:0/3959814417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd9c0c6e00 0x7fcd9c0cd250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcd94034040 tx=0x7fcd94004a40 comp rx=0 tx=0).stop 2024-09-16T00:07:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.631+0000 7fcda0fd8640 1 -- 172.21.15.139:0/3959814417 shutdown_connections 2024-09-16T00:07:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.631+0000 7fcda0fd8640 1 --2- 172.21.15.139:0/3959814417 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcd7c06c2e0 0x7fcd7c06e7a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.631+0000 7fcda0fd8640 1 --2- 172.21.15.139:0/3959814417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd9c0c6e00 0x7fcd9c0cd250 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.631+0000 7fcda0fd8640 1 --2- 172.21.15.139:0/3959814417 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd9c0c6a20 0x7fcd9c1ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:35.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.631+0000 7fcda0fd8640 1 --2- 172.21.15.139:0/3959814417 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd9c06ea70 0x7fcd9c156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:35.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.632+0000 7fcda0fd8640 1 -- 172.21.15.139:0/3959814417 >> 172.21.15.139:0/3959814417 conn(0x7fcd9c06cc30 msgr2=0x7fcd9c1e39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:35.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.632+0000 7fcda0fd8640 1 -- 172.21.15.139:0/3959814417 shutdown_connections 2024-09-16T00:07:35.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.632+0000 7fcda0fd8640 1 -- 172.21.15.139:0/3959814417 wait complete. 2024-09-16T00:07:35.646 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-09-16T00:07:35.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.752+0000 7fb424cc3640 1 Processor -- start 2024-09-16T00:07:35.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.752+0000 7fb424cc3640 1 -- start start 2024-09-16T00:07:35.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb424cc3640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb420154bd0 0x7fb42015b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:35.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb424cc3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb42006ea20 0x7fb4200c3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:35.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb424cc3640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4200c8b50 0x7fb42015e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb424cc3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb42015ed80 con 0x7fb42006ea20 2024-09-16T00:07:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb424cc3640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb42006cd40 con 0x7fb420154bd0 2024-09-16T00:07:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb424cc3640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb42015a9d0 con 0x7fb4200c8b50 2024-09-16T00:07:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb41dd9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb42006ea20 0x7fb4200c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb41dd9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb42006ea20 0x7fb4200c3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46514/0 (socket says 172.21.15.139:46514) 2024-09-16T00:07:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb41dd9b640 1 -- 172.21.15.139:0/3634506992 learned_addr learned my addr 172.21.15.139:0/3634506992 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb41e59c640 1 --2- 172.21.15.139:0/3634506992 >> v2:172.21.15.164:3300/0 conn(0x7fb420154bd0 0x7fb42015b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb41ed9d640 1 --2- 172.21.15.139:0/3634506992 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4200c8b50 0x7fb42015e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb41dd9b640 1 -- 172.21.15.139:0/3634506992 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4200c8b50 msgr2=0x7fb42015e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb41dd9b640 1 --2- 172.21.15.139:0/3634506992 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4200c8b50 0x7fb42015e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb41dd9b640 1 -- 172.21.15.139:0/3634506992 >> v2:172.21.15.164:3300/0 conn(0x7fb420154bd0 msgr2=0x7fb42015b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:35.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb41dd9b640 1 --2- 172.21.15.139:0/3634506992 >> v2:172.21.15.164:3300/0 conn(0x7fb420154bd0 0x7fb42015b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:35.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb41dd9b640 1 -- 172.21.15.139:0/3634506992 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42006cfc0 con 0x7fb42006ea20 2024-09-16T00:07:35.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.753+0000 7fb41ed9d640 1 --2- 172.21.15.139:0/3634506992 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4200c8b50 0x7fb42015e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:35.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.754+0000 7fb41dd9b640 1 --2- 172.21.15.139:0/3634506992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42006ea20 0x7fb4200c3170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb4200cbc90 tx=0x7fb40802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9733a65726e880 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.754+0000 7fb41d59a640 1 -- 172.21.15.139:0/3634506992 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408037070 con 0x7fb42006ea20 2024-09-16T00:07:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.754+0000 7fb41d59a640 1 -- 172.21.15.139:0/3634506992 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb408032440 con 0x7fb42006ea20 2024-09-16T00:07:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.754+0000 7fb41d59a640 1 -- 172.21.15.139:0/3634506992 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408032810 con 0x7fb42006ea20 2024-09-16T00:07:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.754+0000 7fb424cc3640 1 -- 172.21.15.139:0/3634506992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42006ea20 msgr2=0x7fb4200c3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.754+0000 7fb424cc3640 1 --2- 172.21.15.139:0/3634506992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42006ea20 0x7fb4200c3170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb4200cbc90 tx=0x7fb40802a1e0 comp rx=0 tx=0).stop 2024-09-16T00:07:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.754+0000 7fb424cc3640 1 -- 172.21.15.139:0/3634506992 shutdown_connections 2024-09-16T00:07:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.754+0000 7fb424cc3640 1 --2- 172.21.15.139:0/3634506992 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4200c8b50 0x7fb42015e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.754+0000 7fb424cc3640 1 --2- 172.21.15.139:0/3634506992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42006ea20 0x7fb4200c3170 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.754+0000 7fb424cc3640 1 --2- 172.21.15.139:0/3634506992 >> v2:172.21.15.164:3300/0 conn(0x7fb420154bd0 0x7fb42015b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:35.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.755+0000 7fb424cc3640 1 -- 172.21.15.139:0/3634506992 >> 172.21.15.139:0/3634506992 conn(0x7fb42006c150 msgr2=0x7fb420067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:35.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.755+0000 7fb424cc3640 1 -- 172.21.15.139:0/3634506992 shutdown_connections 2024-09-16T00:07:35.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.755+0000 7fb424cc3640 1 -- 172.21.15.139:0/3634506992 wait complete. 2024-09-16T00:07:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.756+0000 7fb424cc3640 1 Processor -- start 2024-09-16T00:07:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.756+0000 7fb424cc3640 1 -- start start 2024-09-16T00:07:35.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.756+0000 7fb424cc3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb42006ea20 0x7fb42014c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb424cc3640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4200c8b50 0x7fb4201f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb424cc3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb420154bd0 0x7fb420071790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb424cc3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb420154fb0 con 0x7fb420154bd0 2024-09-16T00:07:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb424cc3640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb42015af50 con 0x7fb42006ea20 2024-09-16T00:07:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb424cc3640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb42006d2a0 con 0x7fb4200c8b50 2024-09-16T00:07:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb41ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb420154bd0 0x7fb420071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb41ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb420154bd0 0x7fb420071790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46528/0 (socket says 172.21.15.139:46528) 2024-09-16T00:07:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb41ed9d640 1 -- 172.21.15.139:0/1332676595 learned_addr learned my addr 172.21.15.139:0/1332676595 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb41e59c640 1 --2- 172.21.15.139:0/1332676595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb42006ea20 0x7fb42014c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb41dd9b640 1 --2- 172.21.15.139:0/1332676595 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4200c8b50 0x7fb4201f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb41ed9d640 1 -- 172.21.15.139:0/1332676595 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4200c8b50 msgr2=0x7fb4201f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:35.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb41ed9d640 1 --2- 172.21.15.139:0/1332676595 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4200c8b50 0x7fb4201f42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb41ed9d640 1 -- 172.21.15.139:0/1332676595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb42006ea20 msgr2=0x7fb42014c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb41ed9d640 1 --2- 172.21.15.139:0/1332676595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb42006ea20 0x7fb42014c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb41ed9d640 1 -- 172.21.15.139:0/1332676595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4201f4800 con 0x7fb420154bd0 2024-09-16T00:07:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb41dd9b640 1 --2- 172.21.15.139:0/1332676595 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4200c8b50 0x7fb4201f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb41ed9d640 1 --2- 172.21.15.139:0/1332676595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb420154bd0 0x7fb420071790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb41800cef0 tx=0x7fb41800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb416ffd640 1 -- 172.21.15.139:0/1332676595 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41801a070 con 0x7fb420154bd0 2024-09-16T00:07:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb416ffd640 1 -- 172.21.15.139:0/1332676595 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb41800d840 con 0x7fb420154bd0 2024-09-16T00:07:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb424cc3640 1 -- 172.21.15.139:0/1332676595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4200ca870 con 0x7fb420154bd0 2024-09-16T00:07:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb416ffd640 1 -- 172.21.15.139:0/1332676595 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41800dc30 con 0x7fb420154bd0 2024-09-16T00:07:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.757+0000 7fb424cc3640 1 -- 172.21.15.139:0/1332676595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4201f3ac0 con 0x7fb420154bd0 2024-09-16T00:07:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.758+0000 7fb424cc3640 1 -- 172.21.15.139:0/1332676595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb420154380 con 0x7fb420154bd0 2024-09-16T00:07:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.759+0000 7fb416ffd640 1 -- 172.21.15.139:0/1332676595 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb41801d070 con 0x7fb420154bd0 2024-09-16T00:07:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.759+0000 7fb416ffd640 1 --2- 172.21.15.139:0/1332676595 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb40006c2b0 0x7fb40006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:35.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.759+0000 7fb416ffd640 1 -- 172.21.15.139:0/1332676595 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb41808bba0 con 0x7fb420154bd0 2024-09-16T00:07:35.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.759+0000 7fb41e59c640 1 --2- 172.21.15.139:0/1332676595 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb40006c2b0 0x7fb40006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:35.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.760+0000 7fb41e59c640 1 --2- 172.21.15.139:0/1332676595 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb40006c2b0 0x7fb40006e770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb4100079a0 tx=0x7fb410008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:35.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.764+0000 7fb416ffd640 1 -- 172.21.15.139:0/1332676595 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb420154380 con 0x7fb420154bd0 2024-09-16T00:07:35.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:35.963+0000 7fb424cc3640 1 -- 172.21.15.139:0/1332676595 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb42015e840 con 0x7fb420154bd0 2024-09-16T00:07:36.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.633+0000 7fb416ffd640 1 -- 172.21.15.139:0/1332676595 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v7) ==== 115+0+0 (secure 0 0 0) 0x7fb418059cc0 con 0x7fb420154bd0 2024-09-16T00:07:36.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.636+0000 7fb424cc3640 1 -- 172.21.15.139:0/1332676595 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb40006c2b0 msgr2=0x7fb40006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:36.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.636+0000 7fb424cc3640 1 --2- 172.21.15.139:0/1332676595 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb40006c2b0 0x7fb40006e770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb4100079a0 tx=0x7fb410008040 comp rx=0 tx=0).stop 2024-09-16T00:07:36.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.636+0000 7fb424cc3640 1 -- 172.21.15.139:0/1332676595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb420154bd0 msgr2=0x7fb420071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:36.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.636+0000 7fb424cc3640 1 --2- 172.21.15.139:0/1332676595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb420154bd0 0x7fb420071790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb41800cef0 tx=0x7fb41800a710 comp rx=0 tx=0).stop 2024-09-16T00:07:36.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.636+0000 7fb424cc3640 1 -- 172.21.15.139:0/1332676595 shutdown_connections 2024-09-16T00:07:36.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.636+0000 7fb424cc3640 1 --2- 172.21.15.139:0/1332676595 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb40006c2b0 0x7fb40006e770 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.636+0000 7fb424cc3640 1 --2- 172.21.15.139:0/1332676595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb420154bd0 0x7fb420071790 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.636+0000 7fb424cc3640 1 --2- 172.21.15.139:0/1332676595 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4200c8b50 0x7fb4201f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.636+0000 7fb424cc3640 1 --2- 172.21.15.139:0/1332676595 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb42006ea20 0x7fb42014c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.636+0000 7fb424cc3640 1 -- 172.21.15.139:0/1332676595 >> 172.21.15.139:0/1332676595 conn(0x7fb42006c150 msgr2=0x7fb420153b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:36.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.636+0000 7fb424cc3640 1 -- 172.21.15.139:0/1332676595 shutdown_connections 2024-09-16T00:07:36.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.636+0000 7fb424cc3640 1 -- 172.21.15.139:0/1332676595 wait complete. 2024-09-16T00:07:36.650 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-16T00:07:36.650 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T00:07:36.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.753+0000 7f5ec0dfd640 1 Processor -- start 2024-09-16T00:07:36.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.753+0000 7f5ec0dfd640 1 -- start start 2024-09-16T00:07:36.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5ec0dfd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5ebc0cba00 0x7f5ebc0c8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:36.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5ec0dfd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ebc0cbf40 con 0x7f5ebc15e0c0 2024-09-16T00:07:36.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5ec0dfd640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ebc0cc200 con 0x7f5ebc0c76f0 2024-09-16T00:07:36.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5ec0dfd640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc0c7cb0 con 0x7f5ebc0cba00 2024-09-16T00:07:36.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eba59c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5ebc15e0c0 0x7f5ebc15e4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38154/0 (socket says 172.21.15.139:38154) 2024-09-16T00:07:36.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eba59c640 1 -- 172.21.15.139:0/1141419773 learned_addr learned my addr 172.21.15.139:0/1141419773 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:36.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb9d9b640 1 --2- 172.21.15.139:0/1141419773 >> v2:172.21.15.164:3300/0 conn(0x7f5ebc0cba00 0x7f5ebc0c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:36.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb959a640 1 -- 172.21.15.139:0/1141419773 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3544543009 0 0) 0x7f5ebc0cbf40 con 0x7f5ebc15e0c0 2024-09-16T00:07:36.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb959a640 1 -- 172.21.15.139:0/1141419773 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ea8003520 con 0x7f5ebc15e0c0 2024-09-16T00:07:36.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb959a640 1 -- 172.21.15.139:0/1141419773 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3418432523 0 0) 0x7f5ea8003520 con 0x7f5ebc15e0c0 2024-09-16T00:07:36.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb959a640 1 -- 172.21.15.139:0/1141419773 >> v2:172.21.15.164:3300/0 conn(0x7f5ebc0cba00 msgr2=0x7f5ebc0c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:36.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb959a640 1 --2- 172.21.15.139:0/1141419773 >> v2:172.21.15.164:3300/0 conn(0x7f5ebc0cba00 0x7f5ebc0c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb959a640 1 -- 172.21.15.139:0/1141419773 >> v1:172.21.15.164:6789/0 conn(0x7f5ebc0c76f0 legacy=0x7f5ebc06cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:36.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb959a640 1 -- 172.21.15.139:0/1141419773 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc0cc4e0 con 0x7f5ebc15e0c0 2024-09-16T00:07:36.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb959a640 1 -- 172.21.15.139:0/1141419773 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7f5eac001670 con 0x7f5ebc15e0c0 2024-09-16T00:07:36.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb959a640 1 -- 172.21.15.139:0/1141419773 >> v1:172.21.15.139:6789/0 conn(0x7f5ebc15e0c0 legacy=0x7f5ebc15e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:36.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb959a640 1 --2- 172.21.15.139:0/1141419773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ea8003a00 0x7f5ea8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:36.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb959a640 1 --2- 172.21.15.139:0/1141419773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea8004800 0x7f5ea8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:36.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb959a640 1 --2- 172.21.15.139:0/1141419773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ea80051c0 0x7f5ea800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:36.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb959a640 1 -- 172.21.15.139:0/1141419773 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc0c7cb0 con 0x7f5ea8004800 2024-09-16T00:07:36.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb959a640 1 -- 172.21.15.139:0/1141419773 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea800c0a0 con 0x7f5ea80051c0 2024-09-16T00:07:36.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb959a640 1 -- 172.21.15.139:0/1141419773 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea800c2a0 con 0x7f5ea8003a00 2024-09-16T00:07:36.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.754+0000 7f5eb9d9b640 1 --2- 172.21.15.139:0/1141419773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea8004800 0x7f5ea8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:36.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.755+0000 7f5ebad9d640 1 --2- 172.21.15.139:0/1141419773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ea80051c0 0x7f5ea800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:36.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.755+0000 7f5eba59c640 1 --2- 172.21.15.139:0/1141419773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ea8003a00 0x7f5ea8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:36.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.755+0000 7f5eb9d9b640 1 -- 172.21.15.139:0/1141419773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ea8003a00 msgr2=0x7f5ea8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:36.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.755+0000 7f5eb9d9b640 1 --2- 172.21.15.139:0/1141419773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ea8003a00 0x7f5ea8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.755+0000 7f5eb9d9b640 1 -- 172.21.15.139:0/1141419773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ea80051c0 msgr2=0x7f5ea800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:36.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.755+0000 7f5eb9d9b640 1 --2- 172.21.15.139:0/1141419773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ea80051c0 0x7f5ea800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.755+0000 7f5eb9d9b640 1 -- 172.21.15.139:0/1141419773 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea8003520 con 0x7f5ea8004800 2024-09-16T00:07:36.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.755+0000 7f5ebad9d640 1 -- 172.21.15.139:0/1141419773 reap_dead start 2024-09-16T00:07:36.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.755+0000 7f5eb9d9b640 1 --2- 172.21.15.139:0/1141419773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea8004800 0x7f5ea8004c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009ac0 tx=0x7f5ea8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2985ca327724f7fd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:36.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.756+0000 7f5eb959a640 1 -- 172.21.15.139:0/1141419773 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4036070 con 0x7f5ea8004800 2024-09-16T00:07:36.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.756+0000 7f5eb959a640 1 -- 172.21.15.139:0/1141419773 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea4031400 con 0x7f5ea8004800 2024-09-16T00:07:36.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.756+0000 7f5eb959a640 1 -- 172.21.15.139:0/1141419773 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4009690 con 0x7f5ea8004800 2024-09-16T00:07:36.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.756+0000 7f5ec0dfd640 1 -- 172.21.15.139:0/1141419773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea8004800 msgr2=0x7f5ea8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:36.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.756+0000 7f5ec0dfd640 1 --2- 172.21.15.139:0/1141419773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea8004800 0x7f5ea8004c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009ac0 tx=0x7f5ea8005830 comp rx=0 tx=0).stop 2024-09-16T00:07:36.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.756+0000 7f5ec0dfd640 1 -- 172.21.15.139:0/1141419773 shutdown_connections 2024-09-16T00:07:36.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.756+0000 7f5ec0dfd640 1 --2- 172.21.15.139:0/1141419773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea8004800 0x7f5ea8004c80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.756+0000 7f5ec0dfd640 1 -- 172.21.15.139:0/1141419773 >> 172.21.15.139:0/1141419773 conn(0x7f5ebc0d03b0 msgr2=0x7f5ebc0d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:36.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.756+0000 7f5ec0dfd640 1 -- 172.21.15.139:0/1141419773 shutdown_connections 2024-09-16T00:07:36.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.756+0000 7f5ec0dfd640 1 -- 172.21.15.139:0/1141419773 wait complete. 2024-09-16T00:07:36.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.758+0000 7f5ec0dfd640 1 Processor -- start 2024-09-16T00:07:36.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.758+0000 7f5ec0dfd640 1 -- start start 2024-09-16T00:07:36.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.758+0000 7f5ec0dfd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea8004800 0x7f5ebc153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:36.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.758+0000 7f5ec0dfd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ebc152d10 0x7f5ebc14d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:36.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.758+0000 7f5ec0dfd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ebc156450 0x7f5ebc0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:36.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.758+0000 7f5ec0dfd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc15e4a0 con 0x7f5ea8004800 2024-09-16T00:07:36.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.758+0000 7f5ec0dfd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc0c8270 con 0x7f5ebc152d10 2024-09-16T00:07:36.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.758+0000 7f5ec0dfd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc15e620 con 0x7f5ebc156450 2024-09-16T00:07:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.758+0000 7f5eba59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea8004800 0x7f5ebc153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.758+0000 7f5eba59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea8004800 0x7f5ebc153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46558/0 (socket says 172.21.15.139:46558) 2024-09-16T00:07:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.758+0000 7f5eba59c640 1 -- 172.21.15.139:0/1153196130 learned_addr learned my addr 172.21.15.139:0/1153196130 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.758+0000 7f5eb9d9b640 1 --2- 172.21.15.139:0/1153196130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ebc152d10 0x7f5ebc14d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.759+0000 7f5ebad9d640 1 --2- 172.21.15.139:0/1153196130 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ebc156450 0x7f5ebc0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.759+0000 7f5eba59c640 1 -- 172.21.15.139:0/1153196130 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ebc156450 msgr2=0x7f5ebc0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.759+0000 7f5eba59c640 1 --2- 172.21.15.139:0/1153196130 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ebc156450 0x7f5ebc0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.759+0000 7f5eba59c640 1 -- 172.21.15.139:0/1153196130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ebc152d10 msgr2=0x7f5ebc14d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.759+0000 7f5eba59c640 1 --2- 172.21.15.139:0/1153196130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ebc152d10 0x7f5ebc14d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.759+0000 7f5eba59c640 1 -- 172.21.15.139:0/1153196130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc155350 con 0x7f5ea8004800 2024-09-16T00:07:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.759+0000 7f5ebad9d640 1 --2- 172.21.15.139:0/1153196130 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ebc156450 0x7f5ebc0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.759+0000 7f5eba59c640 1 --2- 172.21.15.139:0/1153196130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea8004800 0x7f5ebc153160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5eac007340 tx=0x7f5eac007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:36.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.759+0000 7f5eb2ffd640 1 -- 172.21.15.139:0/1153196130 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac01a070 con 0x7f5ea8004800 2024-09-16T00:07:36.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.759+0000 7f5eb2ffd640 1 -- 172.21.15.139:0/1153196130 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eac00f8c0 con 0x7f5ea8004800 2024-09-16T00:07:36.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.759+0000 7f5ec0dfd640 1 -- 172.21.15.139:0/1153196130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc155630 con 0x7f5ea8004800 2024-09-16T00:07:36.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.759+0000 7f5eb2ffd640 1 -- 172.21.15.139:0/1153196130 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac001430 con 0x7f5ea8004800 2024-09-16T00:07:36.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.759+0000 7f5ec0dfd640 1 -- 172.21.15.139:0/1153196130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc0c41f0 con 0x7f5ea8004800 2024-09-16T00:07:36.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.760+0000 7f5ec0dfd640 1 -- 172.21.15.139:0/1153196130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc0c9ca0 con 0x7f5ea8004800 2024-09-16T00:07:36.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.760+0000 7f5eb2ffd640 1 -- 172.21.15.139:0/1153196130 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5eac01d070 con 0x7f5ea8004800 2024-09-16T00:07:36.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.761+0000 7f5eb2ffd640 1 --2- 172.21.15.139:0/1153196130 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e9c06c2e0 0x7f5e9c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:36.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.761+0000 7f5eb2ffd640 1 -- 172.21.15.139:0/1153196130 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5eac08aca0 con 0x7f5ea8004800 2024-09-16T00:07:36.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.761+0000 7f5eb9d9b640 1 --2- 172.21.15.139:0/1153196130 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e9c06c2e0 0x7f5e9c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:36.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.761+0000 7f5eb9d9b640 1 --2- 172.21.15.139:0/1153196130 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e9c06c2e0 0x7f5e9c06e7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4033000 tx=0x7f5ea4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:36.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.765+0000 7f5eb2ffd640 1 -- 172.21.15.139:0/1153196130 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5ebc0c9ca0 con 0x7f5ea8004800 2024-09-16T00:07:36.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.962+0000 7f5ec0dfd640 1 -- 172.21.15.139:0/1153196130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ebc0cacc0 con 0x7f5ea8004800 2024-09-16T00:07:36.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.963+0000 7f5eb2ffd640 1 -- 172.21.15.139:0/1153196130 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+6876 (secure 0 0 0) 0x7f5ebc0cacc0 con 0x7f5ea8004800 2024-09-16T00:07:36.965 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:36.965 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":7,"btime":"2024-09-16T00:07:36:629927+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2773358731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2773358731},{"type":"v1","addr":"172.21.15.139:6835","nonce":2773358731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4268,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/2919602297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":2919602297},{"type":"v1","addr":"172.21.15.164:6837","nonce":2919602297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3575043661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":3575043661},{"type":"v1","addr":"172.21.15.139:6837","nonce":3575043661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/4113980181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":4113980181},{"type":"v1","addr":"172.21.15.139:6839","nonce":4113980181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/374774889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":374774889},{"type":"v1","addr":"172.21.15.164:6834","nonce":374774889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T00:07:32.949865+0000","modified":"2024-09-16T00:07:35.964876+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"f","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.164:6836/4279698936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":4279698936},{"type":"v1","addr":"172.21.15.164:6836","nonce":4279698936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T00:07:36.965 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 7 2024-09-16T00:07:36.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.965+0000 7f5ec0dfd640 1 -- 172.21.15.139:0/1153196130 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e9c06c2e0 msgr2=0x7f5e9c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:36.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.965+0000 7f5ec0dfd640 1 --2- 172.21.15.139:0/1153196130 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e9c06c2e0 0x7f5e9c06e7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4033000 tx=0x7f5ea4008030 comp rx=0 tx=0).stop 2024-09-16T00:07:36.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.965+0000 7f5ec0dfd640 1 -- 172.21.15.139:0/1153196130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea8004800 msgr2=0x7f5ebc153160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:36.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.965+0000 7f5ec0dfd640 1 --2- 172.21.15.139:0/1153196130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea8004800 0x7f5ebc153160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5eac007340 tx=0x7f5eac007800 comp rx=0 tx=0).stop 2024-09-16T00:07:36.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.965+0000 7f5ec0dfd640 1 -- 172.21.15.139:0/1153196130 shutdown_connections 2024-09-16T00:07:36.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.965+0000 7f5ec0dfd640 1 --2- 172.21.15.139:0/1153196130 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e9c06c2e0 0x7f5e9c06e7a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.965+0000 7f5ec0dfd640 1 --2- 172.21.15.139:0/1153196130 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5ebc156450 0x7f5ebc0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.965+0000 7f5ec0dfd640 1 --2- 172.21.15.139:0/1153196130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5ebc152d10 0x7f5ebc14d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.965+0000 7f5ec0dfd640 1 --2- 172.21.15.139:0/1153196130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ea8004800 0x7f5ebc153160 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:36.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.965+0000 7f5ec0dfd640 1 -- 172.21.15.139:0/1153196130 >> 172.21.15.139:0/1153196130 conn(0x7f5ebc0d03b0 msgr2=0x7f5ebc0cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:36.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.965+0000 7f5ec0dfd640 1 -- 172.21.15.139:0/1153196130 shutdown_connections 2024-09-16T00:07:36.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:36.965+0000 7f5ec0dfd640 1 -- 172.21.15.139:0/1153196130 wait complete. 2024-09-16T00:07:36.978 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:07:37.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.075+0000 7fb109b49640 1 Processor -- start 2024-09-16T00:07:37.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.075+0000 7fb109b49640 1 -- start start 2024-09-16T00:07:37.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb109b49640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb104154bd0 0x7fb10415b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb109b49640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb1040c8b50 0x7fb1040c3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb109b49640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb10406cd40 con 0x7fb10406ea20 2024-09-16T00:07:37.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb109b49640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb10415a9d0 con 0x7fb1040c8b50 2024-09-16T00:07:37.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb109b49640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb10406cfc0 con 0x7fb104154bd0 2024-09-16T00:07:37.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb102ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb10406ea20 0x7fb1040ce0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38168/0 (socket says 172.21.15.139:38168) 2024-09-16T00:07:37.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb102ffd640 1 -- 172.21.15.139:0/3018894313 learned_addr learned my addr 172.21.15.139:0/3018894313 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:37.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb1037fe640 1 --2- 172.21.15.139:0/3018894313 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb104154bd0 0x7fb10415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb103fff640 1 --2- 172.21.15.139:0/3018894313 >> v2:172.21.15.164:3300/0 conn(0x7fb1040c8b50 0x7fb1040c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb1027fc640 1 -- 172.21.15.139:0/3018894313 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1039192744 0 0) 0x7fb0f8001110 con 0x7fb10406ea20 2024-09-16T00:07:37.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb1027fc640 1 -- 172.21.15.139:0/3018894313 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0ec003520 con 0x7fb10406ea20 2024-09-16T00:07:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb1027fc640 1 -- 172.21.15.139:0/3018894313 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3215097057 0 0) 0x7fb0ec003520 con 0x7fb10406ea20 2024-09-16T00:07:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb1027fc640 1 -- 172.21.15.139:0/3018894313 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb104154bd0 msgr2=0x7fb10415b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb1027fc640 1 --2- 172.21.15.139:0/3018894313 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb104154bd0 0x7fb10415b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb1027fc640 1 -- 172.21.15.139:0/3018894313 >> v2:172.21.15.164:3300/0 conn(0x7fb1040c8b50 msgr2=0x7fb1040c3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb1027fc640 1 --2- 172.21.15.139:0/3018894313 >> v2:172.21.15.164:3300/0 conn(0x7fb1040c8b50 0x7fb1040c3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb1027fc640 1 -- 172.21.15.139:0/3018894313 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10415ac70 con 0x7fb10406ea20 2024-09-16T00:07:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb1027fc640 1 -- 172.21.15.139:0/3018894313 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7fb0f8001840 con 0x7fb10406ea20 2024-09-16T00:07:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb1027fc640 1 -- 172.21.15.139:0/3018894313 >> v1:172.21.15.139:6789/0 conn(0x7fb10406ea20 legacy=0x7fb1040ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb1027fc640 1 --2- 172.21.15.139:0/3018894313 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb104154bd0 0x7fb0ec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.076+0000 7fb1027fc640 1 --2- 172.21.15.139:0/3018894313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0ec003f30 0x7fb0ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.077+0000 7fb1027fc640 1 --2- 172.21.15.139:0/3018894313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ec004d40 0x7fb0ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.077+0000 7fb1027fc640 1 -- 172.21.15.139:0/3018894313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb10415a9d0 con 0x7fb0ec004d40 2024-09-16T00:07:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.077+0000 7fb1027fc640 1 -- 172.21.15.139:0/3018894313 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb10406cfc0 con 0x7fb0ec003f30 2024-09-16T00:07:37.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.077+0000 7fb1027fc640 1 -- 172.21.15.139:0/3018894313 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb0ec00b9c0 con 0x7fb104154bd0 2024-09-16T00:07:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.077+0000 7fb103fff640 1 --2- 172.21.15.139:0/3018894313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ec004d40 0x7fb0ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.077+0000 7fb1037fe640 1 --2- 172.21.15.139:0/3018894313 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb104154bd0 0x7fb0ec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.077+0000 7fb102ffd640 1 --2- 172.21.15.139:0/3018894313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0ec003f30 0x7fb0ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.077+0000 7fb103fff640 1 -- 172.21.15.139:0/3018894313 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb104154bd0 msgr2=0x7fb0ec0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.077+0000 7fb103fff640 1 --2- 172.21.15.139:0/3018894313 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb104154bd0 0x7fb0ec0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.077+0000 7fb103fff640 1 -- 172.21.15.139:0/3018894313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0ec003f30 msgr2=0x7fb0ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.077+0000 7fb103fff640 1 --2- 172.21.15.139:0/3018894313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0ec003f30 0x7fb0ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.077+0000 7fb103fff640 1 -- 172.21.15.139:0/3018894313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ec003520 con 0x7fb0ec004d40 2024-09-16T00:07:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.077+0000 7fb103fff640 1 --2- 172.21.15.139:0/3018894313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ec004d40 0x7fb0ec0051c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009ac0 tx=0x7fb0ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a186e0d6b60d7461 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.078+0000 7fb1027fc640 1 -- 172.21.15.139:0/3018894313 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc036070 con 0x7fb0ec004d40 2024-09-16T00:07:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.078+0000 7fb1027fc640 1 -- 172.21.15.139:0/3018894313 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0fc031400 con 0x7fb0ec004d40 2024-09-16T00:07:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.078+0000 7fb1027fc640 1 -- 172.21.15.139:0/3018894313 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc009690 con 0x7fb0ec004d40 2024-09-16T00:07:37.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.078+0000 7fb109b49640 1 -- 172.21.15.139:0/3018894313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ec004d40 msgr2=0x7fb0ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.078+0000 7fb109b49640 1 --2- 172.21.15.139:0/3018894313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ec004d40 0x7fb0ec0051c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009ac0 tx=0x7fb0ec005830 comp rx=0 tx=0).stop 2024-09-16T00:07:37.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.078+0000 7fb103fff640 1 -- 172.21.15.139:0/3018894313 reap_dead start 2024-09-16T00:07:37.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.078+0000 7fb109b49640 1 -- 172.21.15.139:0/3018894313 shutdown_connections 2024-09-16T00:07:37.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.078+0000 7fb109b49640 1 -- 172.21.15.139:0/3018894313 >> 172.21.15.139:0/3018894313 conn(0x7fb10406c150 msgr2=0x7fb104067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:37.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.078+0000 7fb109b49640 1 -- 172.21.15.139:0/3018894313 shutdown_connections 2024-09-16T00:07:37.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.078+0000 7fb109b49640 1 -- 172.21.15.139:0/3018894313 wait complete. 2024-09-16T00:07:37.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.079+0000 7fb109b49640 1 Processor -- start 2024-09-16T00:07:37.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.080+0000 7fb109b49640 1 -- start start 2024-09-16T00:07:37.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.080+0000 7fb109b49640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ec004d40 0x7fb1041f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.080+0000 7fb109b49640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1041f0220 0x7fb104153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.080+0000 7fb109b49640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1040c7bc0 0x7fb10414a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.080+0000 7fb109b49640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb10406d2a0 con 0x7fb0ec004d40 2024-09-16T00:07:37.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.080+0000 7fb109b49640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb104154fb0 con 0x7fb1041f0220 2024-09-16T00:07:37.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.080+0000 7fb109b49640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb10406d420 con 0x7fb1040c7bc0 2024-09-16T00:07:37.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.080+0000 7fb1037fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ec004d40 0x7fb1041f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.080+0000 7fb1037fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ec004d40 0x7fb1041f4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:46572/0 (socket says 172.21.15.139:46572) 2024-09-16T00:07:37.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.080+0000 7fb1037fe640 1 -- 172.21.15.139:0/3678722502 learned_addr learned my addr 172.21.15.139:0/3678722502 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:37.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.080+0000 7fb102ffd640 1 --2- 172.21.15.139:0/3678722502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1041f0220 0x7fb104153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.080+0000 7fb103fff640 1 --2- 172.21.15.139:0/3678722502 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1040c7bc0 0x7fb10414a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.081+0000 7fb1037fe640 1 -- 172.21.15.139:0/3678722502 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1040c7bc0 msgr2=0x7fb10414a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.081+0000 7fb1037fe640 1 --2- 172.21.15.139:0/3678722502 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1040c7bc0 0x7fb10414a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.081+0000 7fb1037fe640 1 -- 172.21.15.139:0/3678722502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1041f0220 msgr2=0x7fb104153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.081+0000 7fb1037fe640 1 --2- 172.21.15.139:0/3678722502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1041f0220 0x7fb104153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.081+0000 7fb1037fe640 1 -- 172.21.15.139:0/3678722502 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb104150f00 con 0x7fb0ec004d40 2024-09-16T00:07:37.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.081+0000 7fb1037fe640 1 --2- 172.21.15.139:0/3678722502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ec004d40 0x7fb1041f4080 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4011e30 tx=0x7fb0f400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:37.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.081+0000 7fb0ebfff640 1 -- 172.21.15.139:0/3678722502 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f400fea0 con 0x7fb0ec004d40 2024-09-16T00:07:37.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.081+0000 7fb0ebfff640 1 -- 172.21.15.139:0/3678722502 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f4015630 con 0x7fb0ec004d40 2024-09-16T00:07:37.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.081+0000 7fb109b49640 1 -- 172.21.15.139:0/3678722502 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1041f38c0 con 0x7fb0ec004d40 2024-09-16T00:07:37.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.081+0000 7fb0ebfff640 1 -- 172.21.15.139:0/3678722502 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4015a00 con 0x7fb0ec004d40 2024-09-16T00:07:37.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.081+0000 7fb109b49640 1 -- 172.21.15.139:0/3678722502 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb104150940 con 0x7fb0ec004d40 2024-09-16T00:07:37.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.082+0000 7fb109b49640 1 -- 172.21.15.139:0/3678722502 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1040c3170 con 0x7fb0ec004d40 2024-09-16T00:07:37.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.082+0000 7fb0ebfff640 1 -- 172.21.15.139:0/3678722502 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb0f4015ba0 con 0x7fb0ec004d40 2024-09-16T00:07:37.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.083+0000 7fb0ebfff640 1 --2- 172.21.15.139:0/3678722502 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb0f006c2e0 0x7fb0f006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.083+0000 7fb102ffd640 1 --2- 172.21.15.139:0/3678722502 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb0f006c2e0 0x7fb0f006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.083+0000 7fb0ebfff640 1 -- 172.21.15.139:0/3678722502 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb0f408c3f0 con 0x7fb0ec004d40 2024-09-16T00:07:37.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.083+0000 7fb102ffd640 1 --2- 172.21.15.139:0/3678722502 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb0f006c2e0 0x7fb0f006e7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb1041f0600 tx=0x7fb0f8007b00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:37.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.087+0000 7fb0ebfff640 1 -- 172.21.15.139:0/3678722502 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb0f4019030 con 0x7fb0ec004d40 2024-09-16T00:07:37.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.253+0000 7fb109b49640 1 -- 172.21.15.139:0/3678722502 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb10406e420 con 0x7fb0ec004d40 2024-09-16T00:07:37.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.254+0000 7fb0ebfff640 1 -- 172.21.15.139:0/3678722502 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16725 (secure 0 0 0) 0x7fb0f4010040 con 0x7fb0ec004d40 2024-09-16T00:07:37.255 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:37.255 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":22,"fsid":"2d3366e5-6a59-4b59-93b2-615e72ac7015","created":"2024-09-16T00:07:09.967833+0000","modified":"2024-09-16T00:07:34.609954+0000","last_up_change":"2024-09-16T00:07:21.049678+0000","last_in_change":"2024-09-16T00:07:12.025620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:07:22.056303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:07:30.768649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:07:32.008209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83699a05-4937-43c3-aefc-b669bed9bb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6817","nonce":1974901438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6819","nonce":1974901438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6823","nonce":1974901438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6821","nonce":1974901438}]},"public_addr":"172.21.15.139:6817/1974901438","cluster_addr":"172.21.15.139:6819/1974901438","heartbeat_back_addr":"172.21.15.139:6823/1974901438","heartbeat_front_addr":"172.21.15.139:6821/1974901438","state":["exists","up"]},{"osd":1,"uuid":"9c04991e-9695-4bbb-8011-bf613bf404b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6809","nonce":2232380998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6811","nonce":2232380998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6815","nonce":2232380998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6813","nonce":2232380998}]},"public_addr":"172.21.15.139:6809/2232380998","cluster_addr":"172.21.15.139:6811/2232380998","heartbeat_back_addr":"172.21.15.139:6815/2232380998","heartbeat_front_addr":"172.21.15.139:6813/2232380998","state":["exists","up"]},{"osd":2,"uuid":"8048fc64-72b3-4866-ac62-6a8b2bf6f5c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6801","nonce":4180588905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6803","nonce":4180588905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6807","nonce":4180588905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6805","nonce":4180588905}]},"public_addr":"172.21.15.139:6801/4180588905","cluster_addr":"172.21.15.139:6803/4180588905","heartbeat_back_addr":"172.21.15.139:6807/4180588905","heartbeat_front_addr":"172.21.15.139:6805/4180588905","state":["exists","up"]},{"osd":3,"uuid":"c34ac77b-354c-4884-902d-3e2a84f5eb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6827","nonce":3947768492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6829","nonce":3947768492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6833","nonce":3947768492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6831","nonce":3947768492}]},"public_addr":"172.21.15.139:6827/3947768492","cluster_addr":"172.21.15.139:6829/3947768492","heartbeat_back_addr":"172.21.15.139:6833/3947768492","heartbeat_front_addr":"172.21.15.139:6831/3947768492","state":["exists","up"]},{"osd":4,"uuid":"ba9a7c93-9672-43e1-8c98-2878cdf7f548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6817","nonce":642632916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6819","nonce":642632916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6823","nonce":642632916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6821","nonce":642632916}]},"public_addr":"172.21.15.164:6817/642632916","cluster_addr":"172.21.15.164:6819/642632916","heartbeat_back_addr":"172.21.15.164:6823/642632916","heartbeat_front_addr":"172.21.15.164:6821/642632916","state":["exists","up"]},{"osd":5,"uuid":"46a0916b-474c-411b-84ad-86f27d4cbf53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6801","nonce":1051778383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6803","nonce":1051778383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6807","nonce":1051778383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6805","nonce":1051778383}]},"public_addr":"172.21.15.164:6801/1051778383","cluster_addr":"172.21.15.164:6803/1051778383","heartbeat_back_addr":"172.21.15.164:6807/1051778383","heartbeat_front_addr":"172.21.15.164:6805/1051778383","state":["exists","up"]},{"osd":6,"uuid":"9b6b33cc-d6d9-4a0a-8143-526073a58b93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6825","nonce":1908495807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6827","nonce":1908495807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6831","nonce":1908495807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6829","nonce":1908495807}]},"public_addr":"172.21.15.164:6825/1908495807","cluster_addr":"172.21.15.164:6827/1908495807","heartbeat_back_addr":"172.21.15.164:6831/1908495807","heartbeat_front_addr":"172.21.15.164:6829/1908495807","state":["exists","up"]},{"osd":7,"uuid":"50bcd808-3d3e-4a72-88e4-30decb53cfea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6809","nonce":1158838808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6811","nonce":1158838808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6815","nonce":1158838808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6813","nonce":1158838808}]},"public_addr":"172.21.15.164:6809/1158838808","cluster_addr":"172.21.15.164:6811/1158838808","heartbeat_back_addr":"172.21.15.164:6815/1158838808","heartbeat_front_addr":"172.21.15.164:6813/1158838808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.843117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.622789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:17.919457+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:19.330341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.815640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.083046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.551499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.377179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:07:37.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.256+0000 7fb109b49640 1 -- 172.21.15.139:0/3678722502 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb0f006c2e0 msgr2=0x7fb0f006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.256+0000 7fb109b49640 1 --2- 172.21.15.139:0/3678722502 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb0f006c2e0 0x7fb0f006e7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb1041f0600 tx=0x7fb0f8007b00 comp rx=0 tx=0).stop 2024-09-16T00:07:37.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.256+0000 7fb109b49640 1 -- 172.21.15.139:0/3678722502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ec004d40 msgr2=0x7fb1041f4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.256+0000 7fb109b49640 1 --2- 172.21.15.139:0/3678722502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ec004d40 0x7fb1041f4080 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4011e30 tx=0x7fb0f400f580 comp rx=0 tx=0).stop 2024-09-16T00:07:37.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.256+0000 7fb109b49640 1 -- 172.21.15.139:0/3678722502 shutdown_connections 2024-09-16T00:07:37.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.256+0000 7fb109b49640 1 --2- 172.21.15.139:0/3678722502 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb0f006c2e0 0x7fb0f006e7a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.256+0000 7fb109b49640 1 --2- 172.21.15.139:0/3678722502 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb1040c7bc0 0x7fb10414a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.256+0000 7fb109b49640 1 --2- 172.21.15.139:0/3678722502 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb1041f0220 0x7fb104153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.256+0000 7fb109b49640 1 --2- 172.21.15.139:0/3678722502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ec004d40 0x7fb1041f4080 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.256+0000 7fb109b49640 1 -- 172.21.15.139:0/3678722502 >> 172.21.15.139:0/3678722502 conn(0x7fb10406c150 msgr2=0x7fb10414ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:37.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.256+0000 7fb109b49640 1 -- 172.21.15.139:0/3678722502 shutdown_connections 2024-09-16T00:07:37.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.256+0000 7fb109b49640 1 -- 172.21.15.139:0/3678722502 wait complete. 2024-09-16T00:07:37.268 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T00:07:37.268 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:07:37.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.365+0000 7fabd003c640 1 Processor -- start 2024-09-16T00:07:37.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.365+0000 7fabd003c640 1 -- start start 2024-09-16T00:07:37.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.366+0000 7fabd003c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fabc8154e10 0x7fabc80c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.366+0000 7fabd003c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc80cc4e0 con 0x7fabc815b600 2024-09-16T00:07:37.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.366+0000 7fabd003c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc806d770 con 0x7fabc806d360 2024-09-16T00:07:37.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.366+0000 7fabd003c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fabc806bfe0 con 0x7fabc8154e10 2024-09-16T00:07:37.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.366+0000 7fabcddd8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fabc8154e10 0x7fabc80c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.366+0000 7fabcd5d7640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fabc815b600 0x7fabc80cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45096/0 (socket says 172.21.15.139:45096) 2024-09-16T00:07:37.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.366+0000 7fabcddd8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fabc8154e10 0x7fabc80c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40244/0 (socket says 172.21.15.139:40244) 2024-09-16T00:07:37.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.366+0000 7fabcd5d7640 1 -- 172.21.15.139:0/2875548098 learned_addr learned my addr 172.21.15.139:0/2875548098 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:37.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.366+0000 7fabcddd8640 1 -- 172.21.15.139:0/2875548098 >> v1:172.21.15.164:6789/0 conn(0x7fabc806d360 legacy=0x7fabc80cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.366+0000 7fabcddd8640 1 -- 172.21.15.139:0/2875548098 >> v1:172.21.15.139:6789/0 conn(0x7fabc815b600 legacy=0x7fabc80cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.367+0000 7fabcddd8640 1 -- 172.21.15.139:0/2875548098 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc806c2c0 con 0x7fabc8154e10 2024-09-16T00:07:37.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.366+0000 7fabccdd6640 1 -- 172.21.15.139:0/2875548098 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4093183021 0 0) 0x7fabc80cc4e0 con 0x7fabc815b600 2024-09-16T00:07:37.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.367+0000 7fabcddd8640 1 --2- 172.21.15.139:0/2875548098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc8154e10 0x7fabc80c75d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fabc40097a0 tx=0x7fabc402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02c4d42f85b14cd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:37.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.367+0000 7fabccdd6640 1 -- 172.21.15.139:0/2875548098 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4037070 con 0x7fabc8154e10 2024-09-16T00:07:37.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.367+0000 7fabccdd6640 1 -- 172.21.15.139:0/2875548098 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabc4032640 con 0x7fabc8154e10 2024-09-16T00:07:37.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.367+0000 7fabccdd6640 1 -- 172.21.15.139:0/2875548098 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4032a10 con 0x7fabc8154e10 2024-09-16T00:07:37.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.368+0000 7fabd003c640 1 -- 172.21.15.139:0/2875548098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc8154e10 msgr2=0x7fabc80c75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.368+0000 7fabd003c640 1 --2- 172.21.15.139:0/2875548098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc8154e10 0x7fabc80c75d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fabc40097a0 tx=0x7fabc402a2c0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.368+0000 7fabd003c640 1 -- 172.21.15.139:0/2875548098 shutdown_connections 2024-09-16T00:07:37.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.368+0000 7fabd003c640 1 --2- 172.21.15.139:0/2875548098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc8154e10 0x7fabc80c75d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.368+0000 7fabd003c640 1 -- 172.21.15.139:0/2875548098 >> 172.21.15.139:0/2875548098 conn(0x7fabc80d03b0 msgr2=0x7fabc80d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:37.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.368+0000 7fabd003c640 1 -- 172.21.15.139:0/2875548098 shutdown_connections 2024-09-16T00:07:37.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.368+0000 7fabd003c640 1 -- 172.21.15.139:0/2875548098 wait complete. 2024-09-16T00:07:37.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.369+0000 7fabd003c640 1 Processor -- start 2024-09-16T00:07:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.369+0000 7fabd003c640 1 -- start start 2024-09-16T00:07:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabd003c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fabc806d360 0x7fabc814f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabd003c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabc8154e10 0x7fabc814ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabd003c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc815b600 0x7fabc814da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabd003c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fabc806cb60 con 0x7fabc815b600 2024-09-16T00:07:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabd003c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fabc806c880 con 0x7fabc8154e10 2024-09-16T00:07:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabd003c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fabc806ce40 con 0x7fabc806d360 2024-09-16T00:07:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabce5d9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc815b600 0x7fabc814da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabce5d9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc815b600 0x7fabc814da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40260/0 (socket says 172.21.15.139:40260) 2024-09-16T00:07:37.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabce5d9640 1 -- 172.21.15.139:0/4266877839 learned_addr learned my addr 172.21.15.139:0/4266877839 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:37.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabcd5d7640 1 --2- 172.21.15.139:0/4266877839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabc8154e10 0x7fabc814ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabcddd8640 1 --2- 172.21.15.139:0/4266877839 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fabc806d360 0x7fabc814f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabce5d9640 1 -- 172.21.15.139:0/4266877839 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fabc806d360 msgr2=0x7fabc814f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabce5d9640 1 --2- 172.21.15.139:0/4266877839 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fabc806d360 0x7fabc814f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabce5d9640 1 -- 172.21.15.139:0/4266877839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabc8154e10 msgr2=0x7fabc814ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabce5d9640 1 --2- 172.21.15.139:0/4266877839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabc8154e10 0x7fabc814ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabce5d9640 1 -- 172.21.15.139:0/4266877839 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc81f3f50 con 0x7fabc815b600 2024-09-16T00:07:37.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabcd5d7640 1 --2- 172.21.15.139:0/4266877839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabc8154e10 0x7fabc814ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:37.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabcddd8640 1 --2- 172.21.15.139:0/4266877839 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fabc806d360 0x7fabc814f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:37.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.370+0000 7fabce5d9640 1 --2- 172.21.15.139:0/4266877839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc815b600 0x7fabc814da50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fabc000c7d0 tx=0x7fabc000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:37.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.371+0000 7fabbe7fc640 1 -- 172.21.15.139:0/4266877839 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0003b70 con 0x7fabc815b600 2024-09-16T00:07:37.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.371+0000 7fabbe7fc640 1 -- 172.21.15.139:0/4266877839 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabc0003d30 con 0x7fabc815b600 2024-09-16T00:07:37.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.371+0000 7fabbe7fc640 1 -- 172.21.15.139:0/4266877839 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc000aa90 con 0x7fabc815b600 2024-09-16T00:07:37.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.371+0000 7fabd003c640 1 -- 172.21.15.139:0/4266877839 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc81f4230 con 0x7fabc815b600 2024-09-16T00:07:37.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.371+0000 7fabd003c640 1 -- 172.21.15.139:0/4266877839 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc81f47f0 con 0x7fabc815b600 2024-09-16T00:07:37.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.372+0000 7fabd003c640 1 -- 172.21.15.139:0/4266877839 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc806bcc0 con 0x7fabc815b600 2024-09-16T00:07:37.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.372+0000 7fabbe7fc640 1 -- 172.21.15.139:0/4266877839 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fabc000ac30 con 0x7fabc815b600 2024-09-16T00:07:37.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.372+0000 7fabbe7fc640 1 --2- 172.21.15.139:0/4266877839 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fabac06c2e0 0x7fabac06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.373+0000 7fabbe7fc640 1 -- 172.21.15.139:0/4266877839 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fabc008b3d0 con 0x7fabc815b600 2024-09-16T00:07:37.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.373+0000 7fabcddd8640 1 --2- 172.21.15.139:0/4266877839 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fabac06c2e0 0x7fabac06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.373+0000 7fabcddd8640 1 --2- 172.21.15.139:0/4266877839 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fabac06c2e0 0x7fabac06e7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fabc4034000 tx=0x7fabc402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:37.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.377+0000 7fabbe7fc640 1 -- 172.21.15.139:0/4266877839 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fabc0059ab0 con 0x7fabc815b600 2024-09-16T00:07:37.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.540+0000 7fabd003c640 1 -- 172.21.15.139:0/4266877839 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fabc80c3ab0 con 0x7fabac06c2e0 2024-09-16T00:07:37.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.546+0000 7fabbe7fc640 1 -- 172.21.15.139:0/4266877839 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239826 (secure 0 0 0) 0x7fabc80c3ab0 con 0x7fabac06c2e0 2024-09-16T00:07:37.549 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:37.551 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:07:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.549+0000 7fabd003c640 1 -- 172.21.15.139:0/4266877839 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fabac06c2e0 msgr2=0x7fabac06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.549+0000 7fabd003c640 1 --2- 172.21.15.139:0/4266877839 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fabac06c2e0 0x7fabac06e7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fabc4034000 tx=0x7fabc402e000 comp rx=0 tx=0).stop 2024-09-16T00:07:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.549+0000 7fabd003c640 1 -- 172.21.15.139:0/4266877839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc815b600 msgr2=0x7fabc814da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.549+0000 7fabd003c640 1 --2- 172.21.15.139:0/4266877839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc815b600 0x7fabc814da50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fabc000c7d0 tx=0x7fabc000cc90 comp rx=0 tx=0).stop 2024-09-16T00:07:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.549+0000 7fabd003c640 1 -- 172.21.15.139:0/4266877839 shutdown_connections 2024-09-16T00:07:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.549+0000 7fabd003c640 1 --2- 172.21.15.139:0/4266877839 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fabac06c2e0 0x7fabac06e7a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.549+0000 7fabd003c640 1 --2- 172.21.15.139:0/4266877839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc815b600 0x7fabc814da50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.549+0000 7fabd003c640 1 --2- 172.21.15.139:0/4266877839 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabc8154e10 0x7fabc814ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.549+0000 7fabd003c640 1 --2- 172.21.15.139:0/4266877839 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fabc806d360 0x7fabc814f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.549+0000 7fabd003c640 1 -- 172.21.15.139:0/4266877839 >> 172.21.15.139:0/4266877839 conn(0x7fabc80d03b0 msgr2=0x7fabc81e6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.549+0000 7fabd003c640 1 -- 172.21.15.139:0/4266877839 shutdown_connections 2024-09-16T00:07:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.549+0000 7fabd003c640 1 -- 172.21.15.139:0/4266877839 wait complete. 2024-09-16T00:07:37.561 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-16T00:07:36.026088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":217800,"kb_used_data":1532,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756920,"statfs":{"total":773094113280,"available":772871086080,"internally_reserved":0,"allocated":1568768,"data_stored":1634352,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.423436"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613100+0000","last_change":"2024-09-16T00:07:33.630797+0000","last_active":"2024-09-16T00:07:34.613100+0000","last_peered":"2024-09-16T00:07:34.613100+0000","last_clean":"2024-09-16T00:07:34.613100+0000","last_became_active":"2024-09-16T00:07:33.630491+0000","last_became_peered":"2024-09-16T00:07:33.630491+0000","last_unstale":"2024-09-16T00:07:34.613100+0000","last_undegraded":"2024-09-16T00:07:34.613100+0000","last_fullsized":"2024-09-16T00:07:34.613100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:20.097626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613050+0000","last_change":"2024-09-16T00:07:33.621107+0000","last_active":"2024-09-16T00:07:34.613050+0000","last_peered":"2024-09-16T00:07:34.613050+0000","last_clean":"2024-09-16T00:07:34.613050+0000","last_became_active":"2024-09-16T00:07:33.620858+0000","last_became_peered":"2024-09-16T00:07:33.620858+0000","last_unstale":"2024-09-16T00:07:34.613050+0000","last_undegraded":"2024-09-16T00:07:34.613050+0000","last_fullsized":"2024-09-16T00:07:34.613050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:30.176079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.607160+0000","last_change":"2024-09-16T00:07:31.607160+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.607160+0000","last_undegraded":"2024-09-16T00:07:31.607160+0000","last_fullsized":"2024-09-16T00:07:31.607160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.604908+0000","last_change":"2024-09-16T00:07:31.604908+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.604908+0000","last_undegraded":"2024-09-16T00:07:31.604908+0000","last_fullsized":"2024-09-16T00:07:31.604908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.607987+0000","last_change":"2024-09-16T00:07:31.607987+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.607987+0000","last_undegraded":"2024-09-16T00:07:31.607987+0000","last_fullsized":"2024-09-16T00:07:31.607987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.605023+0000","last_change":"2024-09-16T00:07:31.605023+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.605023+0000","last_undegraded":"2024-09-16T00:07:31.605023+0000","last_fullsized":"2024-09-16T00:07:31.605023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.608809+0000","last_change":"2024-09-16T00:07:31.608809+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.608809+0000","last_undegraded":"2024-09-16T00:07:31.608809+0000","last_fullsized":"2024-09-16T00:07:31.608809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.605567+0000","last_change":"2024-09-16T00:07:31.605567+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.605567+0000","last_undegraded":"2024-09-16T00:07:31.605567+0000","last_fullsized":"2024-09-16T00:07:31.605567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.607098+0000","last_change":"2024-09-16T00:07:31.607098+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.607098+0000","last_undegraded":"2024-09-16T00:07:31.607098+0000","last_fullsized":"2024-09-16T00:07:31.607098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.612904+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:07:34.612904+0000","last_peered":"2024-09-16T00:07:34.612904+0000","last_clean":"2024-09-16T00:07:34.612904+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:07:34.612904+0000","last_undegraded":"2024-09-16T00:07:34.612904+0000","last_fullsized":"2024-09-16T00:07:34.612904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.604873+0000","last_change":"2024-09-16T00:07:31.604873+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.604873+0000","last_undegraded":"2024-09-16T00:07:31.604873+0000","last_fullsized":"2024-09-16T00:07:31.604873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613114+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:07:34.613114+0000","last_peered":"2024-09-16T00:07:34.613114+0000","last_clean":"2024-09-16T00:07:34.613114+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:07:34.613114+0000","last_undegraded":"2024-09-16T00:07:34.613114+0000","last_fullsized":"2024-09-16T00:07:34.613114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613057+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:07:34.613057+0000","last_peered":"2024-09-16T00:07:34.613057+0000","last_clean":"2024-09-16T00:07:34.613057+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:07:34.613057+0000","last_undegraded":"2024-09-16T00:07:34.613057+0000","last_fullsized":"2024-09-16T00:07:34.613057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T00:07:31.610883+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:07:31.610883+0000","last_peered":"2024-09-16T00:07:31.610883+0000","last_clean":"2024-09-16T00:07:31.610883+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:07:31.610883+0000","last_undegraded":"2024-09-16T00:07:31.610883+0000","last_fullsized":"2024-09-16T00:07:31.610883+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.604878+0000","last_change":"2024-09-16T00:07:31.604878+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.604878+0000","last_undegraded":"2024-09-16T00:07:31.604878+0000","last_fullsized":"2024-09-16T00:07:31.604878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.608596+0000","last_change":"2024-09-16T00:07:31.608596+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.608596+0000","last_undegraded":"2024-09-16T00:07:31.608596+0000","last_fullsized":"2024-09-16T00:07:31.608596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.612974+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:07:34.612974+0000","last_peered":"2024-09-16T00:07:34.612974+0000","last_clean":"2024-09-16T00:07:34.612974+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:07:34.612974+0000","last_undegraded":"2024-09-16T00:07:34.612974+0000","last_fullsized":"2024-09-16T00:07:34.612974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.604882+0000","last_change":"2024-09-16T00:07:31.604882+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.604882+0000","last_undegraded":"2024-09-16T00:07:31.604882+0000","last_fullsized":"2024-09-16T00:07:31.604882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.608272+0000","last_change":"2024-09-16T00:07:31.608272+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.608272+0000","last_undegraded":"2024-09-16T00:07:31.608272+0000","last_fullsized":"2024-09-16T00:07:31.608272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.607264+0000","last_change":"2024-09-16T00:07:31.607264+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.607264+0000","last_undegraded":"2024-09-16T00:07:31.607264+0000","last_fullsized":"2024-09-16T00:07:31.607264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.609991+0000","last_change":"2024-09-16T00:07:31.609991+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.609991+0000","last_undegraded":"2024-09-16T00:07:31.609991+0000","last_fullsized":"2024-09-16T00:07:31.609991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.608465+0000","last_change":"2024-09-16T00:07:31.608465+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.608465+0000","last_undegraded":"2024-09-16T00:07:31.608465+0000","last_fullsized":"2024-09-16T00:07:31.608465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.608642+0000","last_change":"2024-09-16T00:07:31.608642+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.608642+0000","last_undegraded":"2024-09-16T00:07:31.608642+0000","last_fullsized":"2024-09-16T00:07:31.608642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.609156+0000","last_change":"2024-09-16T00:07:31.609156+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.609156+0000","last_undegraded":"2024-09-16T00:07:31.609156+0000","last_fullsized":"2024-09-16T00:07:31.609156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":4,"acting":4,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":42,"acting":42,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":237568,"data_stored":643197,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":52829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":52829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":58646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":52829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":52829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":643197,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:07:37.565 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:07:37.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.698+0000 7fbed11c4640 1 Processor -- start 2024-09-16T00:07:37.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.698+0000 7fbed11c4640 1 -- start start 2024-09-16T00:07:37.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.699+0000 7fbed11c4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbecc154bd0 0x7fbecc15b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.699+0000 7fbed11c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbecc06ea20 0x7fbecc0c3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.699+0000 7fbed11c4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbecc06cd40 con 0x7fbecc0c8b50 2024-09-16T00:07:37.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.699+0000 7fbed11c4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbecc15a9d0 con 0x7fbecc154bd0 2024-09-16T00:07:37.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.699+0000 7fbed11c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbecc06cfc0 con 0x7fbecc06ea20 2024-09-16T00:07:37.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.699+0000 7fbecad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbecc154bd0 0x7fbecc15b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.699+0000 7fbecb59e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fbecc0c8b50 0x7fbecc0ce0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45112/0 (socket says 172.21.15.139:45112) 2024-09-16T00:07:37.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.699+0000 7fbecb59e640 1 -- 172.21.15.139:0/130864922 learned_addr learned my addr 172.21.15.139:0/130864922 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:37.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.699+0000 7fbeca59c640 1 --2- 172.21.15.139:0/130864922 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbecc06ea20 0x7fbecc0c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.699+0000 7fbecad9d640 1 -- 172.21.15.139:0/130864922 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbecc06ea20 msgr2=0x7fbecc0c3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:37.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.699+0000 7fbecad9d640 1 --2- 172.21.15.139:0/130864922 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbecc06ea20 0x7fbecc0c3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.699+0000 7fbecad9d640 1 -- 172.21.15.139:0/130864922 >> v1:172.21.15.139:6789/0 conn(0x7fbecc0c8b50 legacy=0x7fbecc0ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.699+0000 7fbecad9d640 1 -- 172.21.15.139:0/130864922 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbecc15ac70 con 0x7fbecc154bd0 2024-09-16T00:07:37.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.699+0000 7fbec9d9b640 1 -- 172.21.15.139:0/130864922 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2738171834 0 0) 0x7fbec4001110 con 0x7fbecc0c8b50 2024-09-16T00:07:37.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.699+0000 7fbeca59c640 1 --2- 172.21.15.139:0/130864922 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbecc06ea20 0x7fbecc0c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:37.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.700+0000 7fbecad9d640 1 --2- 172.21.15.139:0/130864922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbecc154bd0 0x7fbecc15b470 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009d90 tx=0x7fbebc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590b709597ec6da6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:37.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.700+0000 7fbec9d9b640 1 -- 172.21.15.139:0/130864922 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc007740 con 0x7fbecc154bd0 2024-09-16T00:07:37.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.700+0000 7fbec9d9b640 1 -- 172.21.15.139:0/130864922 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbebc0078e0 con 0x7fbecc154bd0 2024-09-16T00:07:37.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.700+0000 7fbec9d9b640 1 -- 172.21.15.139:0/130864922 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc007cb0 con 0x7fbecc154bd0 2024-09-16T00:07:37.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.700+0000 7fbed11c4640 1 -- 172.21.15.139:0/130864922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbecc154bd0 msgr2=0x7fbecc15b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.700+0000 7fbed11c4640 1 --2- 172.21.15.139:0/130864922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbecc154bd0 0x7fbecc15b470 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009d90 tx=0x7fbebc02a660 comp rx=0 tx=0).stop 2024-09-16T00:07:37.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.700+0000 7fbed11c4640 1 -- 172.21.15.139:0/130864922 shutdown_connections 2024-09-16T00:07:37.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.700+0000 7fbed11c4640 1 --2- 172.21.15.139:0/130864922 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbecc06ea20 0x7fbecc0c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.700+0000 7fbed11c4640 1 --2- 172.21.15.139:0/130864922 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbecc154bd0 0x7fbecc15b470 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.701+0000 7fbed11c4640 1 -- 172.21.15.139:0/130864922 >> 172.21.15.139:0/130864922 conn(0x7fbecc06c150 msgr2=0x7fbecc067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:37.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.701+0000 7fbed11c4640 1 -- 172.21.15.139:0/130864922 shutdown_connections 2024-09-16T00:07:37.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.701+0000 7fbed11c4640 1 -- 172.21.15.139:0/130864922 wait complete. 2024-09-16T00:07:37.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.702+0000 7fbed11c4640 1 Processor -- start 2024-09-16T00:07:37.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.702+0000 7fbed11c4640 1 -- start start 2024-09-16T00:07:37.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbed11c4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbecc06ea20 0x7fbecc0c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbed11c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbecc0c8b50 0x7fbecc0c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbed11c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbecc154bd0 0x7fbecc0c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbed11c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc06d2a0 con 0x7fbecc0c8b50 2024-09-16T00:07:37.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbed11c4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc154fb0 con 0x7fbecc06ea20 2024-09-16T00:07:37.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbed11c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbecc06d420 con 0x7fbecc154bd0 2024-09-16T00:07:37.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbeca59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbecc0c8b50 0x7fbecc0c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbeca59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbecc0c8b50 0x7fbecc0c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40272/0 (socket says 172.21.15.139:40272) 2024-09-16T00:07:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbeca59c640 1 -- 172.21.15.139:0/87246363 learned_addr learned my addr 172.21.15.139:0/87246363 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbecad9d640 1 --2- 172.21.15.139:0/87246363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbecc06ea20 0x7fbecc0c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbecb59e640 1 --2- 172.21.15.139:0/87246363 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbecc154bd0 0x7fbecc0c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbeca59c640 1 -- 172.21.15.139:0/87246363 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbecc154bd0 msgr2=0x7fbecc0c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbeca59c640 1 --2- 172.21.15.139:0/87246363 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbecc154bd0 0x7fbecc0c65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbeca59c640 1 -- 172.21.15.139:0/87246363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbecc06ea20 msgr2=0x7fbecc0c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbeca59c640 1 --2- 172.21.15.139:0/87246363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbecc06ea20 0x7fbecc0c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbeca59c640 1 -- 172.21.15.139:0/87246363 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbecc0c8540 con 0x7fbecc0c8b50 2024-09-16T00:07:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbecad9d640 1 --2- 172.21.15.139:0/87246363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbecc06ea20 0x7fbecc0c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbecb59e640 1 --2- 172.21.15.139:0/87246363 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbecc154bd0 0x7fbecc0c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbeca59c640 1 --2- 172.21.15.139:0/87246363 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbecc0c8b50 0x7fbecc0c9f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbec000c820 tx=0x7fbec000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbeb37fe640 1 -- 172.21.15.139:0/87246363 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0015070 con 0x7fbecc0c8b50 2024-09-16T00:07:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbeb37fe640 1 -- 172.21.15.139:0/87246363 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbec0007a30 con 0x7fbecc0c8b50 2024-09-16T00:07:37.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbed11c4640 1 -- 172.21.15.139:0/87246363 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbecc0cde90 con 0x7fbecc0c8b50 2024-09-16T00:07:37.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbeb37fe640 1 -- 172.21.15.139:0/87246363 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0007e00 con 0x7fbecc0c8b50 2024-09-16T00:07:37.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.703+0000 7fbed11c4640 1 -- 172.21.15.139:0/87246363 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbecc0ccc00 con 0x7fbecc0c8b50 2024-09-16T00:07:37.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.704+0000 7fbed11c4640 1 -- 172.21.15.139:0/87246363 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbecc0c3170 con 0x7fbecc0c8b50 2024-09-16T00:07:37.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.705+0000 7fbeb37fe640 1 -- 172.21.15.139:0/87246363 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbec0018070 con 0x7fbecc0c8b50 2024-09-16T00:07:37.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.705+0000 7fbeb37fe640 1 --2- 172.21.15.139:0/87246363 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbeb806c2e0 0x7fbeb806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:37.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.706+0000 7fbeb37fe640 1 -- 172.21.15.139:0/87246363 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbec008b2f0 con 0x7fbecc0c8b50 2024-09-16T00:07:37.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.706+0000 7fbecad9d640 1 --2- 172.21.15.139:0/87246363 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbeb806c2e0 0x7fbeb806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:37.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.706+0000 7fbecad9d640 1 --2- 172.21.15.139:0/87246363 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbeb806c2e0 0x7fbeb806e7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbebc034000 tx=0x7fbebc02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:37.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.710+0000 7fbeb37fe640 1 -- 172.21.15.139:0/87246363 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbec0059af0 con 0x7fbecc0c8b50 2024-09-16T00:07:37.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.873+0000 7fbed11c4640 1 -- 172.21.15.139:0/87246363 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbecc0c4fe0 con 0x7fbeb806c2e0 2024-09-16T00:07:37.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.879+0000 7fbeb37fe640 1 -- 172.21.15.139:0/87246363 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239826 (secure 0 0 0) 0x7fbecc0c4fe0 con 0x7fbeb806c2e0 2024-09-16T00:07:37.882 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:37.883 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:07:37.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.882+0000 7fbed11c4640 1 -- 172.21.15.139:0/87246363 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbeb806c2e0 msgr2=0x7fbeb806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.882+0000 7fbed11c4640 1 --2- 172.21.15.139:0/87246363 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbeb806c2e0 0x7fbeb806e7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbebc034000 tx=0x7fbebc02f000 comp rx=0 tx=0).stop 2024-09-16T00:07:37.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.882+0000 7fbed11c4640 1 -- 172.21.15.139:0/87246363 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbecc0c8b50 msgr2=0x7fbecc0c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:37.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.882+0000 7fbed11c4640 1 --2- 172.21.15.139:0/87246363 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbecc0c8b50 0x7fbecc0c9f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbec000c820 tx=0x7fbec000cc20 comp rx=0 tx=0).stop 2024-09-16T00:07:37.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.882+0000 7fbed11c4640 1 -- 172.21.15.139:0/87246363 shutdown_connections 2024-09-16T00:07:37.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.882+0000 7fbed11c4640 1 --2- 172.21.15.139:0/87246363 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbeb806c2e0 0x7fbeb806e7a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.882+0000 7fbed11c4640 1 --2- 172.21.15.139:0/87246363 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbecc154bd0 0x7fbecc0c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.882+0000 7fbed11c4640 1 --2- 172.21.15.139:0/87246363 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbecc0c8b50 0x7fbecc0c9f60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.882+0000 7fbed11c4640 1 --2- 172.21.15.139:0/87246363 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbecc06ea20 0x7fbecc0c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:37.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.882+0000 7fbed11c4640 1 -- 172.21.15.139:0/87246363 >> 172.21.15.139:0/87246363 conn(0x7fbecc06c150 msgr2=0x7fbecc067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:37.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.882+0000 7fbed11c4640 1 -- 172.21.15.139:0/87246363 shutdown_connections 2024-09-16T00:07:37.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:37.882+0000 7fbed11c4640 1 -- 172.21.15.139:0/87246363 wait complete. 2024-09-16T00:07:37.894 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-16T00:07:36.026088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":217800,"kb_used_data":1532,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756920,"statfs":{"total":773094113280,"available":772871086080,"internally_reserved":0,"allocated":1568768,"data_stored":1634352,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.423436"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613100+0000","last_change":"2024-09-16T00:07:33.630797+0000","last_active":"2024-09-16T00:07:34.613100+0000","last_peered":"2024-09-16T00:07:34.613100+0000","last_clean":"2024-09-16T00:07:34.613100+0000","last_became_active":"2024-09-16T00:07:33.630491+0000","last_became_peered":"2024-09-16T00:07:33.630491+0000","last_unstale":"2024-09-16T00:07:34.613100+0000","last_undegraded":"2024-09-16T00:07:34.613100+0000","last_fullsized":"2024-09-16T00:07:34.613100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:20.097626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613050+0000","last_change":"2024-09-16T00:07:33.621107+0000","last_active":"2024-09-16T00:07:34.613050+0000","last_peered":"2024-09-16T00:07:34.613050+0000","last_clean":"2024-09-16T00:07:34.613050+0000","last_became_active":"2024-09-16T00:07:33.620858+0000","last_became_peered":"2024-09-16T00:07:33.620858+0000","last_unstale":"2024-09-16T00:07:34.613050+0000","last_undegraded":"2024-09-16T00:07:34.613050+0000","last_fullsized":"2024-09-16T00:07:34.613050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:30.176079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.607160+0000","last_change":"2024-09-16T00:07:31.607160+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.607160+0000","last_undegraded":"2024-09-16T00:07:31.607160+0000","last_fullsized":"2024-09-16T00:07:31.607160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.604908+0000","last_change":"2024-09-16T00:07:31.604908+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.604908+0000","last_undegraded":"2024-09-16T00:07:31.604908+0000","last_fullsized":"2024-09-16T00:07:31.604908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.607987+0000","last_change":"2024-09-16T00:07:31.607987+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.607987+0000","last_undegraded":"2024-09-16T00:07:31.607987+0000","last_fullsized":"2024-09-16T00:07:31.607987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.605023+0000","last_change":"2024-09-16T00:07:31.605023+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.605023+0000","last_undegraded":"2024-09-16T00:07:31.605023+0000","last_fullsized":"2024-09-16T00:07:31.605023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.608809+0000","last_change":"2024-09-16T00:07:31.608809+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.608809+0000","last_undegraded":"2024-09-16T00:07:31.608809+0000","last_fullsized":"2024-09-16T00:07:31.608809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.605567+0000","last_change":"2024-09-16T00:07:31.605567+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.605567+0000","last_undegraded":"2024-09-16T00:07:31.605567+0000","last_fullsized":"2024-09-16T00:07:31.605567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.607098+0000","last_change":"2024-09-16T00:07:31.607098+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.607098+0000","last_undegraded":"2024-09-16T00:07:31.607098+0000","last_fullsized":"2024-09-16T00:07:31.607098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.612904+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:07:34.612904+0000","last_peered":"2024-09-16T00:07:34.612904+0000","last_clean":"2024-09-16T00:07:34.612904+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:07:34.612904+0000","last_undegraded":"2024-09-16T00:07:34.612904+0000","last_fullsized":"2024-09-16T00:07:34.612904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.604873+0000","last_change":"2024-09-16T00:07:31.604873+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.604873+0000","last_undegraded":"2024-09-16T00:07:31.604873+0000","last_fullsized":"2024-09-16T00:07:31.604873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613114+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:07:34.613114+0000","last_peered":"2024-09-16T00:07:34.613114+0000","last_clean":"2024-09-16T00:07:34.613114+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:07:34.613114+0000","last_undegraded":"2024-09-16T00:07:34.613114+0000","last_fullsized":"2024-09-16T00:07:34.613114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613057+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:07:34.613057+0000","last_peered":"2024-09-16T00:07:34.613057+0000","last_clean":"2024-09-16T00:07:34.613057+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:07:34.613057+0000","last_undegraded":"2024-09-16T00:07:34.613057+0000","last_fullsized":"2024-09-16T00:07:34.613057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-16T00:07:31.610883+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:07:31.610883+0000","last_peered":"2024-09-16T00:07:31.610883+0000","last_clean":"2024-09-16T00:07:31.610883+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:07:31.610883+0000","last_undegraded":"2024-09-16T00:07:31.610883+0000","last_fullsized":"2024-09-16T00:07:31.610883+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.604878+0000","last_change":"2024-09-16T00:07:31.604878+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.604878+0000","last_undegraded":"2024-09-16T00:07:31.604878+0000","last_fullsized":"2024-09-16T00:07:31.604878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.608596+0000","last_change":"2024-09-16T00:07:31.608596+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.608596+0000","last_undegraded":"2024-09-16T00:07:31.608596+0000","last_fullsized":"2024-09-16T00:07:31.608596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.612974+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:07:34.612974+0000","last_peered":"2024-09-16T00:07:34.612974+0000","last_clean":"2024-09-16T00:07:34.612974+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:07:34.612974+0000","last_undegraded":"2024-09-16T00:07:34.612974+0000","last_fullsized":"2024-09-16T00:07:34.612974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.604882+0000","last_change":"2024-09-16T00:07:31.604882+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.604882+0000","last_undegraded":"2024-09-16T00:07:31.604882+0000","last_fullsized":"2024-09-16T00:07:31.604882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.608272+0000","last_change":"2024-09-16T00:07:31.608272+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.608272+0000","last_undegraded":"2024-09-16T00:07:31.608272+0000","last_fullsized":"2024-09-16T00:07:31.608272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.607264+0000","last_change":"2024-09-16T00:07:31.607264+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.607264+0000","last_undegraded":"2024-09-16T00:07:31.607264+0000","last_fullsized":"2024-09-16T00:07:31.607264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.609991+0000","last_change":"2024-09-16T00:07:31.609991+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.609991+0000","last_undegraded":"2024-09-16T00:07:31.609991+0000","last_fullsized":"2024-09-16T00:07:31.609991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.608465+0000","last_change":"2024-09-16T00:07:31.608465+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.608465+0000","last_undegraded":"2024-09-16T00:07:31.608465+0000","last_fullsized":"2024-09-16T00:07:31.608465+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.608642+0000","last_change":"2024-09-16T00:07:31.608642+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.608642+0000","last_undegraded":"2024-09-16T00:07:31.608642+0000","last_fullsized":"2024-09-16T00:07:31.608642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-16T00:07:31.609156+0000","last_change":"2024-09-16T00:07:31.609156+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.609156+0000","last_undegraded":"2024-09-16T00:07:31.609156+0000","last_fullsized":"2024-09-16T00:07:31.609156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:32.602688+0000","last_change":"2024-09-16T00:07:32.602688+0000","last_active":"2024-09-16T00:07:32.602688+0000","last_peered":"2024-09-16T00:07:32.602688+0000","last_clean":"2024-09-16T00:07:32.602688+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:32.602688+0000","last_undegraded":"2024-09-16T00:07:32.602688+0000","last_fullsized":"2024-09-16T00:07:32.602688+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-16T00:07:31.595159+0000","last_change":"2024-09-16T00:07:31.595159+0000","last_active":"2024-09-16T00:07:31.595159+0000","last_peered":"2024-09-16T00:07:31.595159+0000","last_clean":"2024-09-16T00:07:31.595159+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-16T00:07:31.595159+0000","last_undegraded":"2024-09-16T00:07:31.595159+0000","last_fullsized":"2024-09-16T00:07:31.595159+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":4,"acting":4,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":42,"acting":42,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":237568,"data_stored":643197,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":52829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":52829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":58646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":52829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":52829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":643197,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:07:37.897 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:07:38.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.031+0000 7fa2196e6640 1 Processor -- start 2024-09-16T00:07:38.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.031+0000 7fa2196e6640 1 -- start start 2024-09-16T00:07:38.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.032+0000 7fa2196e6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa2140b3ee0 0x7fa2140c9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.032+0000 7fa2196e6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa214158610 0x7fa2140c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.032+0000 7fa2196e6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa214158ba0 con 0x7fa214070d40 2024-09-16T00:07:38.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.032+0000 7fa2196e6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa2140c3510 con 0x7fa2140b3ee0 2024-09-16T00:07:38.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.032+0000 7fa2196e6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa214158e80 con 0x7fa214158610 2024-09-16T00:07:38.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.032+0000 7fa212ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa2140b3ee0 0x7fa2140c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.032+0000 7fa212ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa2140b3ee0 0x7fa2140c9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40286/0 (socket says 172.21.15.139:40286) 2024-09-16T00:07:38.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.032+0000 7fa212ffd640 1 -- 172.21.15.139:0/3203097154 learned_addr learned my addr 172.21.15.139:0/3203097154 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:38.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.032+0000 7fa2127fc640 1 --1- 172.21.15.139:0/3203097154 >> v1:172.21.15.139:6789/0 conn(0x7fa214070d40 0x7fa2140ce7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45118/0 (socket says 172.21.15.139:45118) 2024-09-16T00:07:38.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.032+0000 7fa2137fe640 1 --2- 172.21.15.139:0/3203097154 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa214158610 0x7fa2140c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.033+0000 7fa212ffd640 1 -- 172.21.15.139:0/3203097154 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa214158610 msgr2=0x7fa2140c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:38.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.033+0000 7fa212ffd640 1 --2- 172.21.15.139:0/3203097154 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa214158610 0x7fa2140c39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.033+0000 7fa212ffd640 1 -- 172.21.15.139:0/3203097154 >> v1:172.21.15.139:6789/0 conn(0x7fa214070d40 legacy=0x7fa2140ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.033+0000 7fa211ffb640 1 -- 172.21.15.139:0/3203097154 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2329352293 0 0) 0x7fa214158ba0 con 0x7fa214070d40 2024-09-16T00:07:38.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.033+0000 7fa212ffd640 1 -- 172.21.15.139:0/3203097154 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa214159160 con 0x7fa2140b3ee0 2024-09-16T00:07:38.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.033+0000 7fa212ffd640 1 --2- 172.21.15.139:0/3203097154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2140b3ee0 0x7fa2140c9660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa204009e50 tx=0x7fa20402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834ef6d6dd256fbc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:38.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.033+0000 7fa211ffb640 1 -- 172.21.15.139:0/3203097154 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204036070 con 0x7fa2140b3ee0 2024-09-16T00:07:38.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.033+0000 7fa211ffb640 1 -- 172.21.15.139:0/3203097154 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa204031600 con 0x7fa2140b3ee0 2024-09-16T00:07:38.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.033+0000 7fa211ffb640 1 -- 172.21.15.139:0/3203097154 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2040319d0 con 0x7fa2140b3ee0 2024-09-16T00:07:38.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.034+0000 7fa2196e6640 1 -- 172.21.15.139:0/3203097154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2140b3ee0 msgr2=0x7fa2140c9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.034+0000 7fa2196e6640 1 --2- 172.21.15.139:0/3203097154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2140b3ee0 0x7fa2140c9660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa204009e50 tx=0x7fa20402a7a0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.034+0000 7fa2196e6640 1 -- 172.21.15.139:0/3203097154 shutdown_connections 2024-09-16T00:07:38.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.034+0000 7fa2196e6640 1 --2- 172.21.15.139:0/3203097154 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa214158610 0x7fa2140c39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.034+0000 7fa2196e6640 1 --2- 172.21.15.139:0/3203097154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2140b3ee0 0x7fa2140c9660 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.034+0000 7fa2196e6640 1 -- 172.21.15.139:0/3203097154 >> 172.21.15.139:0/3203097154 conn(0x7fa214067d70 msgr2=0x7fa214068180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:38.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.034+0000 7fa2196e6640 1 -- 172.21.15.139:0/3203097154 shutdown_connections 2024-09-16T00:07:38.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.034+0000 7fa2196e6640 1 -- 172.21.15.139:0/3203097154 wait complete. 2024-09-16T00:07:38.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.035+0000 7fa2196e6640 1 Processor -- start 2024-09-16T00:07:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.035+0000 7fa2196e6640 1 -- start start 2024-09-16T00:07:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa2196e6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa214070d40 0x7fa214150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa2196e6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2140b3ee0 0x7fa214152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa2196e6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa214158610 0x7fa214157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa2196e6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa2140ceed0 con 0x7fa214070d40 2024-09-16T00:07:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa2196e6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2140b42c0 con 0x7fa2140b3ee0 2024-09-16T00:07:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa2196e6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa2140cf050 con 0x7fa214158610 2024-09-16T00:07:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa212ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa214070d40 0x7fa214150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa212ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa214070d40 0x7fa214150430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40288/0 (socket says 172.21.15.139:40288) 2024-09-16T00:07:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa212ffd640 1 -- 172.21.15.139:0/1580728638 learned_addr learned my addr 172.21.15.139:0/1580728638 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa2127fc640 1 --2- 172.21.15.139:0/1580728638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2140b3ee0 0x7fa214152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa2137fe640 1 --2- 172.21.15.139:0/1580728638 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa214158610 0x7fa214157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa212ffd640 1 -- 172.21.15.139:0/1580728638 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa214158610 msgr2=0x7fa214157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa212ffd640 1 --2- 172.21.15.139:0/1580728638 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa214158610 0x7fa214157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa212ffd640 1 -- 172.21.15.139:0/1580728638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2140b3ee0 msgr2=0x7fa214152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa212ffd640 1 --2- 172.21.15.139:0/1580728638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2140b3ee0 0x7fa214152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa212ffd640 1 -- 172.21.15.139:0/1580728638 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa214152a60 con 0x7fa214070d40 2024-09-16T00:07:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa2127fc640 1 --2- 172.21.15.139:0/1580728638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2140b3ee0 0x7fa214152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.036+0000 7fa212ffd640 1 --2- 172.21.15.139:0/1580728638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa214070d40 0x7fa214150430 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa204034040 tx=0x7fa204004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.037+0000 7fa20b7fe640 1 -- 172.21.15.139:0/1580728638 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204041020 con 0x7fa214070d40 2024-09-16T00:07:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.037+0000 7fa20b7fe640 1 -- 172.21.15.139:0/1580728638 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa204037040 con 0x7fa214070d40 2024-09-16T00:07:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.037+0000 7fa2196e6640 1 -- 172.21.15.139:0/1580728638 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21406ce60 con 0x7fa214070d40 2024-09-16T00:07:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.037+0000 7fa20b7fe640 1 -- 172.21.15.139:0/1580728638 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204036050 con 0x7fa214070d40 2024-09-16T00:07:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.037+0000 7fa2196e6640 1 -- 172.21.15.139:0/1580728638 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa21415c950 con 0x7fa214070d40 2024-09-16T00:07:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.038+0000 7fa2196e6640 1 -- 172.21.15.139:0/1580728638 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2140c9660 con 0x7fa214070d40 2024-09-16T00:07:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.038+0000 7fa20b7fe640 1 -- 172.21.15.139:0/1580728638 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa204007c70 con 0x7fa214070d40 2024-09-16T00:07:38.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.039+0000 7fa20b7fe640 1 --2- 172.21.15.139:0/1580728638 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa1f406c2b0 0x7fa1f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.039+0000 7fa2127fc640 1 --2- 172.21.15.139:0/1580728638 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa1f406c2b0 0x7fa1f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.039+0000 7fa20b7fe640 1 -- 172.21.15.139:0/1580728638 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa2040abad0 con 0x7fa214070d40 2024-09-16T00:07:38.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.039+0000 7fa2127fc640 1 --2- 172.21.15.139:0/1580728638 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa1f406c2b0 0x7fa1f406e770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa2140b2dc0 tx=0x7fa1fc007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:38.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.043+0000 7fa20b7fe640 1 -- 172.21.15.139:0/1580728638 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa20404b060 con 0x7fa214070d40 2024-09-16T00:07:38.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.206+0000 7fa2196e6640 1 -- 172.21.15.139:0/1580728638 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa2140cb170 con 0x7fa1f406c2b0 2024-09-16T00:07:38.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.212+0000 7fa20b7fe640 1 -- 172.21.15.139:0/1580728638 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251926 (secure 0 0 0) 0x7fa2140cb170 con 0x7fa1f406c2b0 2024-09-16T00:07:38.213 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:38.217 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:07:38.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.215+0000 7fa2196e6640 1 -- 172.21.15.139:0/1580728638 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa1f406c2b0 msgr2=0x7fa1f406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.215+0000 7fa2196e6640 1 --2- 172.21.15.139:0/1580728638 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa1f406c2b0 0x7fa1f406e770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa2140b2dc0 tx=0x7fa1fc007c10 comp rx=0 tx=0).stop 2024-09-16T00:07:38.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.215+0000 7fa2196e6640 1 -- 172.21.15.139:0/1580728638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa214070d40 msgr2=0x7fa214150430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.215+0000 7fa2196e6640 1 --2- 172.21.15.139:0/1580728638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa214070d40 0x7fa214150430 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa204034040 tx=0x7fa204004a40 comp rx=0 tx=0).stop 2024-09-16T00:07:38.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.215+0000 7fa2196e6640 1 -- 172.21.15.139:0/1580728638 shutdown_connections 2024-09-16T00:07:38.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.215+0000 7fa2196e6640 1 --2- 172.21.15.139:0/1580728638 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa1f406c2b0 0x7fa1f406e770 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.215+0000 7fa2196e6640 1 --2- 172.21.15.139:0/1580728638 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa214158610 0x7fa214157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.215+0000 7fa2196e6640 1 --2- 172.21.15.139:0/1580728638 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2140b3ee0 0x7fa214152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.215+0000 7fa2196e6640 1 --2- 172.21.15.139:0/1580728638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa214070d40 0x7fa214150430 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.215+0000 7fa2196e6640 1 -- 172.21.15.139:0/1580728638 >> 172.21.15.139:0/1580728638 conn(0x7fa214067d70 msgr2=0x7fa214071350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:38.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.215+0000 7fa2196e6640 1 -- 172.21.15.139:0/1580728638 shutdown_connections 2024-09-16T00:07:38.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.215+0000 7fa2196e6640 1 -- 172.21.15.139:0/1580728638 wait complete. 2024-09-16T00:07:38.227 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-16T00:07:38.026369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218104,"kb_used_data":1856,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756616,"statfs":{"total":773094113280,"available":772870774784,"internally_reserved":0,"allocated":1900544,"data_stored":1804704,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000886"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115528+0000","last_change":"2024-09-16T00:07:33.635096+0000","last_active":"2024-09-16T00:07:35.115528+0000","last_peered":"2024-09-16T00:07:35.115528+0000","last_clean":"2024-09-16T00:07:35.115528+0000","last_became_active":"2024-09-16T00:07:33.634750+0000","last_became_peered":"2024-09-16T00:07:33.634750+0000","last_unstale":"2024-09-16T00:07:35.115528+0000","last_undegraded":"2024-09-16T00:07:35.115528+0000","last_fullsized":"2024-09-16T00:07:35.115528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115558+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:07:35.115558+0000","last_peered":"2024-09-16T00:07:35.115558+0000","last_clean":"2024-09-16T00:07:35.115558+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:07:35.115558+0000","last_undegraded":"2024-09-16T00:07:35.115558+0000","last_fullsized":"2024-09-16T00:07:35.115558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115392+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:07:35.115392+0000","last_peered":"2024-09-16T00:07:35.115392+0000","last_clean":"2024-09-16T00:07:35.115392+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:07:35.115392+0000","last_undegraded":"2024-09-16T00:07:35.115392+0000","last_fullsized":"2024-09-16T00:07:35.115392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615167+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:07:34.615167+0000","last_peered":"2024-09-16T00:07:34.615167+0000","last_clean":"2024-09-16T00:07:34.615167+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:07:34.615167+0000","last_undegraded":"2024-09-16T00:07:34.615167+0000","last_fullsized":"2024-09-16T00:07:34.615167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115724+0000","last_change":"2024-09-16T00:07:33.630029+0000","last_active":"2024-09-16T00:07:35.115724+0000","last_peered":"2024-09-16T00:07:35.115724+0000","last_clean":"2024-09-16T00:07:35.115724+0000","last_became_active":"2024-09-16T00:07:33.629712+0000","last_became_peered":"2024-09-16T00:07:33.629712+0000","last_unstale":"2024-09-16T00:07:35.115724+0000","last_undegraded":"2024-09-16T00:07:35.115724+0000","last_fullsized":"2024-09-16T00:07:35.115724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614973+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:07:34.614973+0000","last_peered":"2024-09-16T00:07:34.614973+0000","last_clean":"2024-09-16T00:07:34.614973+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:07:34.614973+0000","last_undegraded":"2024-09-16T00:07:34.614973+0000","last_fullsized":"2024-09-16T00:07:34.614973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115522+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:07:35.115522+0000","last_peered":"2024-09-16T00:07:35.115522+0000","last_clean":"2024-09-16T00:07:35.115522+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:07:35.115522+0000","last_undegraded":"2024-09-16T00:07:35.115522+0000","last_fullsized":"2024-09-16T00:07:35.115522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114492+0000","last_change":"2024-09-16T00:07:33.624708+0000","last_active":"2024-09-16T00:07:35.114492+0000","last_peered":"2024-09-16T00:07:35.114492+0000","last_clean":"2024-09-16T00:07:35.114492+0000","last_became_active":"2024-09-16T00:07:33.624458+0000","last_became_peered":"2024-09-16T00:07:33.624458+0000","last_unstale":"2024-09-16T00:07:35.114492+0000","last_undegraded":"2024-09-16T00:07:35.114492+0000","last_fullsized":"2024-09-16T00:07:35.114492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:48.052488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115751+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:07:35.115751+0000","last_peered":"2024-09-16T00:07:35.115751+0000","last_clean":"2024-09-16T00:07:35.115751+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:07:35.115751+0000","last_undegraded":"2024-09-16T00:07:35.115751+0000","last_fullsized":"2024-09-16T00:07:35.115751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115614+0000","last_change":"2024-09-16T00:07:33.635973+0000","last_active":"2024-09-16T00:07:35.115614+0000","last_peered":"2024-09-16T00:07:35.115614+0000","last_clean":"2024-09-16T00:07:35.115614+0000","last_became_active":"2024-09-16T00:07:33.635486+0000","last_became_peered":"2024-09-16T00:07:33.635486+0000","last_unstale":"2024-09-16T00:07:35.115614+0000","last_undegraded":"2024-09-16T00:07:35.115614+0000","last_fullsized":"2024-09-16T00:07:35.115614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115027+0000","last_change":"2024-09-16T00:07:33.636971+0000","last_active":"2024-09-16T00:07:35.115027+0000","last_peered":"2024-09-16T00:07:35.115027+0000","last_clean":"2024-09-16T00:07:35.115027+0000","last_became_active":"2024-09-16T00:07:33.635760+0000","last_became_peered":"2024-09-16T00:07:33.635760+0000","last_unstale":"2024-09-16T00:07:35.115027+0000","last_undegraded":"2024-09-16T00:07:35.115027+0000","last_fullsized":"2024-09-16T00:07:35.115027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614631+0000","last_change":"2024-09-16T00:07:33.632261+0000","last_active":"2024-09-16T00:07:34.614631+0000","last_peered":"2024-09-16T00:07:34.614631+0000","last_clean":"2024-09-16T00:07:34.614631+0000","last_became_active":"2024-09-16T00:07:33.631944+0000","last_became_peered":"2024-09-16T00:07:33.631944+0000","last_unstale":"2024-09-16T00:07:34.614631+0000","last_undegraded":"2024-09-16T00:07:34.614631+0000","last_fullsized":"2024-09-16T00:07:34.614631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:35.052417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114641+0000","last_change":"2024-09-16T00:07:33.628273+0000","last_active":"2024-09-16T00:07:35.114641+0000","last_peered":"2024-09-16T00:07:35.114641+0000","last_clean":"2024-09-16T00:07:35.114641+0000","last_became_active":"2024-09-16T00:07:33.627960+0000","last_became_peered":"2024-09-16T00:07:33.627960+0000","last_unstale":"2024-09-16T00:07:35.114641+0000","last_undegraded":"2024-09-16T00:07:35.114641+0000","last_fullsized":"2024-09-16T00:07:35.114641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:36.451688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114782+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:07:35.114782+0000","last_peered":"2024-09-16T00:07:35.114782+0000","last_clean":"2024-09-16T00:07:35.114782+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:07:35.114782+0000","last_undegraded":"2024-09-16T00:07:35.114782+0000","last_fullsized":"2024-09-16T00:07:35.114782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114664+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:07:35.114664+0000","last_peered":"2024-09-16T00:07:35.114664+0000","last_clean":"2024-09-16T00:07:35.114664+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:07:35.114664+0000","last_undegraded":"2024-09-16T00:07:35.114664+0000","last_fullsized":"2024-09-16T00:07:35.114664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614471+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:07:34.614471+0000","last_peered":"2024-09-16T00:07:34.614471+0000","last_clean":"2024-09-16T00:07:34.614471+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:07:34.614471+0000","last_undegraded":"2024-09-16T00:07:34.614471+0000","last_fullsized":"2024-09-16T00:07:34.614471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615697+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:07:34.615697+0000","last_peered":"2024-09-16T00:07:34.615697+0000","last_clean":"2024-09-16T00:07:34.615697+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:07:34.615697+0000","last_undegraded":"2024-09-16T00:07:34.615697+0000","last_fullsized":"2024-09-16T00:07:34.615697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613100+0000","last_change":"2024-09-16T00:07:33.630797+0000","last_active":"2024-09-16T00:07:34.613100+0000","last_peered":"2024-09-16T00:07:34.613100+0000","last_clean":"2024-09-16T00:07:34.613100+0000","last_became_active":"2024-09-16T00:07:33.630491+0000","last_became_peered":"2024-09-16T00:07:33.630491+0000","last_unstale":"2024-09-16T00:07:34.613100+0000","last_undegraded":"2024-09-16T00:07:34.613100+0000","last_fullsized":"2024-09-16T00:07:34.613100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:20.097626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115840+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:07:35.115840+0000","last_peered":"2024-09-16T00:07:35.115840+0000","last_clean":"2024-09-16T00:07:35.115840+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:07:35.115840+0000","last_undegraded":"2024-09-16T00:07:35.115840+0000","last_fullsized":"2024-09-16T00:07:35.115840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114960+0000","last_change":"2024-09-16T00:07:33.638554+0000","last_active":"2024-09-16T00:07:35.114960+0000","last_peered":"2024-09-16T00:07:35.114960+0000","last_clean":"2024-09-16T00:07:35.114960+0000","last_became_active":"2024-09-16T00:07:33.638263+0000","last_became_peered":"2024-09-16T00:07:33.638263+0000","last_unstale":"2024-09-16T00:07:35.114960+0000","last_undegraded":"2024-09-16T00:07:35.114960+0000","last_fullsized":"2024-09-16T00:07:35.114960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115286+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:07:35.115286+0000","last_peered":"2024-09-16T00:07:35.115286+0000","last_clean":"2024-09-16T00:07:35.115286+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:07:35.115286+0000","last_undegraded":"2024-09-16T00:07:35.115286+0000","last_fullsized":"2024-09-16T00:07:35.115286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615450+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:07:34.615450+0000","last_peered":"2024-09-16T00:07:34.615450+0000","last_clean":"2024-09-16T00:07:34.615450+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:07:34.615450+0000","last_undegraded":"2024-09-16T00:07:34.615450+0000","last_fullsized":"2024-09-16T00:07:34.615450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115015+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:07:35.115015+0000","last_peered":"2024-09-16T00:07:35.115015+0000","last_clean":"2024-09-16T00:07:35.115015+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:07:35.115015+0000","last_undegraded":"2024-09-16T00:07:35.115015+0000","last_fullsized":"2024-09-16T00:07:35.115015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614616+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:07:34.614616+0000","last_peered":"2024-09-16T00:07:34.614616+0000","last_clean":"2024-09-16T00:07:34.614616+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:07:34.614616+0000","last_undegraded":"2024-09-16T00:07:34.614616+0000","last_fullsized":"2024-09-16T00:07:34.614616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114694+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:07:35.114694+0000","last_peered":"2024-09-16T00:07:35.114694+0000","last_clean":"2024-09-16T00:07:35.114694+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:07:35.114694+0000","last_undegraded":"2024-09-16T00:07:35.114694+0000","last_fullsized":"2024-09-16T00:07:35.114694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115306+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:07:35.115306+0000","last_peered":"2024-09-16T00:07:35.115306+0000","last_clean":"2024-09-16T00:07:35.115306+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:07:35.115306+0000","last_undegraded":"2024-09-16T00:07:35.115306+0000","last_fullsized":"2024-09-16T00:07:35.115306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115543+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:07:35.115543+0000","last_peered":"2024-09-16T00:07:35.115543+0000","last_clean":"2024-09-16T00:07:35.115543+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:07:35.115543+0000","last_undegraded":"2024-09-16T00:07:35.115543+0000","last_fullsized":"2024-09-16T00:07:35.115543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114985+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:07:35.114985+0000","last_peered":"2024-09-16T00:07:35.114985+0000","last_clean":"2024-09-16T00:07:35.114985+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:07:35.114985+0000","last_undegraded":"2024-09-16T00:07:35.114985+0000","last_fullsized":"2024-09-16T00:07:35.114985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115448+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:07:35.115448+0000","last_peered":"2024-09-16T00:07:35.115448+0000","last_clean":"2024-09-16T00:07:35.115448+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:07:35.115448+0000","last_undegraded":"2024-09-16T00:07:35.115448+0000","last_fullsized":"2024-09-16T00:07:35.115448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615338+0000","last_change":"2024-09-16T00:07:33.627789+0000","last_active":"2024-09-16T00:07:34.615338+0000","last_peered":"2024-09-16T00:07:34.615338+0000","last_clean":"2024-09-16T00:07:34.615338+0000","last_became_active":"2024-09-16T00:07:33.627500+0000","last_became_peered":"2024-09-16T00:07:33.627500+0000","last_unstale":"2024-09-16T00:07:34.615338+0000","last_undegraded":"2024-09-16T00:07:34.615338+0000","last_fullsized":"2024-09-16T00:07:34.615338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613050+0000","last_change":"2024-09-16T00:07:33.621107+0000","last_active":"2024-09-16T00:07:34.613050+0000","last_peered":"2024-09-16T00:07:34.613050+0000","last_clean":"2024-09-16T00:07:34.613050+0000","last_became_active":"2024-09-16T00:07:33.620858+0000","last_became_peered":"2024-09-16T00:07:33.620858+0000","last_unstale":"2024-09-16T00:07:34.613050+0000","last_undegraded":"2024-09-16T00:07:34.613050+0000","last_fullsized":"2024-09-16T00:07:34.613050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:30.176079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115031+0000","last_change":"2024-09-16T00:07:33.633578+0000","last_active":"2024-09-16T00:07:35.115031+0000","last_peered":"2024-09-16T00:07:35.115031+0000","last_clean":"2024-09-16T00:07:35.115031+0000","last_became_active":"2024-09-16T00:07:33.633310+0000","last_became_peered":"2024-09-16T00:07:33.633310+0000","last_unstale":"2024-09-16T00:07:35.115031+0000","last_undegraded":"2024-09-16T00:07:35.115031+0000","last_fullsized":"2024-09-16T00:07:35.115031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614556+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:07:34.614556+0000","last_peered":"2024-09-16T00:07:34.614556+0000","last_clean":"2024-09-16T00:07:34.614556+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:07:34.614556+0000","last_undegraded":"2024-09-16T00:07:34.614556+0000","last_fullsized":"2024-09-16T00:07:34.614556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115057+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:07:35.115057+0000","last_peered":"2024-09-16T00:07:35.115057+0000","last_clean":"2024-09-16T00:07:35.115057+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:07:35.115057+0000","last_undegraded":"2024-09-16T00:07:35.115057+0000","last_fullsized":"2024-09-16T00:07:35.115057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115509+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:07:35.115509+0000","last_peered":"2024-09-16T00:07:35.115509+0000","last_clean":"2024-09-16T00:07:35.115509+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:07:35.115509+0000","last_undegraded":"2024-09-16T00:07:35.115509+0000","last_fullsized":"2024-09-16T00:07:35.115509+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114784+0000","last_change":"2024-09-16T00:07:32.626800+0000","last_active":"2024-09-16T00:07:35.114784+0000","last_peered":"2024-09-16T00:07:35.114784+0000","last_clean":"2024-09-16T00:07:35.114784+0000","last_became_active":"2024-09-16T00:07:32.626511+0000","last_became_peered":"2024-09-16T00:07:32.626511+0000","last_unstale":"2024-09-16T00:07:35.114784+0000","last_undegraded":"2024-09-16T00:07:35.114784+0000","last_fullsized":"2024-09-16T00:07:35.114784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115156+0000","last_change":"2024-09-16T00:07:32.629539+0000","last_active":"2024-09-16T00:07:35.115156+0000","last_peered":"2024-09-16T00:07:35.115156+0000","last_clean":"2024-09-16T00:07:35.115156+0000","last_became_active":"2024-09-16T00:07:32.629186+0000","last_became_peered":"2024-09-16T00:07:32.629186+0000","last_unstale":"2024-09-16T00:07:35.115156+0000","last_undegraded":"2024-09-16T00:07:35.115156+0000","last_fullsized":"2024-09-16T00:07:35.115156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:59:32.860208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114535+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:07:35.114535+0000","last_peered":"2024-09-16T00:07:35.114535+0000","last_clean":"2024-09-16T00:07:35.114535+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:07:35.114535+0000","last_undegraded":"2024-09-16T00:07:35.114535+0000","last_fullsized":"2024-09-16T00:07:35.114535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614668+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:07:34.614668+0000","last_peered":"2024-09-16T00:07:34.614668+0000","last_clean":"2024-09-16T00:07:34.614668+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:07:34.614668+0000","last_undegraded":"2024-09-16T00:07:34.614668+0000","last_fullsized":"2024-09-16T00:07:34.614668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114527+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:07:35.114527+0000","last_peered":"2024-09-16T00:07:35.114527+0000","last_clean":"2024-09-16T00:07:35.114527+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:07:35.114527+0000","last_undegraded":"2024-09-16T00:07:35.114527+0000","last_fullsized":"2024-09-16T00:07:35.114527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115138+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:07:35.115138+0000","last_peered":"2024-09-16T00:07:35.115138+0000","last_clean":"2024-09-16T00:07:35.115138+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:07:35.115138+0000","last_undegraded":"2024-09-16T00:07:35.115138+0000","last_fullsized":"2024-09-16T00:07:35.115138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615408+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:07:34.615408+0000","last_peered":"2024-09-16T00:07:34.615408+0000","last_clean":"2024-09-16T00:07:34.615408+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:07:34.615408+0000","last_undegraded":"2024-09-16T00:07:34.615408+0000","last_fullsized":"2024-09-16T00:07:34.615408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615356+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:07:34.615356+0000","last_peered":"2024-09-16T00:07:34.615356+0000","last_clean":"2024-09-16T00:07:34.615356+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:07:34.615356+0000","last_undegraded":"2024-09-16T00:07:34.615356+0000","last_fullsized":"2024-09-16T00:07:34.615356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115277+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:07:35.115277+0000","last_peered":"2024-09-16T00:07:35.115277+0000","last_clean":"2024-09-16T00:07:35.115277+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:07:35.115277+0000","last_undegraded":"2024-09-16T00:07:35.115277+0000","last_fullsized":"2024-09-16T00:07:35.115277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115634+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:07:35.115634+0000","last_peered":"2024-09-16T00:07:35.115634+0000","last_clean":"2024-09-16T00:07:35.115634+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:07:35.115634+0000","last_undegraded":"2024-09-16T00:07:35.115634+0000","last_fullsized":"2024-09-16T00:07:35.115634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614648+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:07:34.614648+0000","last_peered":"2024-09-16T00:07:34.614648+0000","last_clean":"2024-09-16T00:07:34.614648+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:07:34.614648+0000","last_undegraded":"2024-09-16T00:07:34.614648+0000","last_fullsized":"2024-09-16T00:07:34.614648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115117+0000","last_change":"2024-09-16T00:07:32.629132+0000","last_active":"2024-09-16T00:07:35.115117+0000","last_peered":"2024-09-16T00:07:35.115117+0000","last_clean":"2024-09-16T00:07:35.115117+0000","last_became_active":"2024-09-16T00:07:32.628803+0000","last_became_peered":"2024-09-16T00:07:32.628803+0000","last_unstale":"2024-09-16T00:07:35.115117+0000","last_undegraded":"2024-09-16T00:07:35.115117+0000","last_fullsized":"2024-09-16T00:07:35.115117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115230+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:07:35.115230+0000","last_peered":"2024-09-16T00:07:35.115230+0000","last_clean":"2024-09-16T00:07:35.115230+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:07:35.115230+0000","last_undegraded":"2024-09-16T00:07:35.115230+0000","last_fullsized":"2024-09-16T00:07:35.115230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115538+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:07:35.115538+0000","last_peered":"2024-09-16T00:07:35.115538+0000","last_clean":"2024-09-16T00:07:35.115538+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:07:35.115538+0000","last_undegraded":"2024-09-16T00:07:35.115538+0000","last_fullsized":"2024-09-16T00:07:35.115538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115121+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:07:35.115121+0000","last_peered":"2024-09-16T00:07:35.115121+0000","last_clean":"2024-09-16T00:07:35.115121+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:07:35.115121+0000","last_undegraded":"2024-09-16T00:07:35.115121+0000","last_fullsized":"2024-09-16T00:07:35.115121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615462+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:07:34.615462+0000","last_peered":"2024-09-16T00:07:34.615462+0000","last_clean":"2024-09-16T00:07:34.615462+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:07:34.615462+0000","last_undegraded":"2024-09-16T00:07:34.615462+0000","last_fullsized":"2024-09-16T00:07:34.615462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115766+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:07:35.115766+0000","last_peered":"2024-09-16T00:07:35.115766+0000","last_clean":"2024-09-16T00:07:35.115766+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:07:35.115766+0000","last_undegraded":"2024-09-16T00:07:35.115766+0000","last_fullsized":"2024-09-16T00:07:35.115766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114802+0000","last_change":"2024-09-16T00:07:32.626856+0000","last_active":"2024-09-16T00:07:35.114802+0000","last_peered":"2024-09-16T00:07:35.114802+0000","last_clean":"2024-09-16T00:07:35.114802+0000","last_became_active":"2024-09-16T00:07:32.626532+0000","last_became_peered":"2024-09-16T00:07:32.626532+0000","last_unstale":"2024-09-16T00:07:35.114802+0000","last_undegraded":"2024-09-16T00:07:35.114802+0000","last_fullsized":"2024-09-16T00:07:35.114802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114886+0000","last_change":"2024-09-16T00:07:33.632870+0000","last_active":"2024-09-16T00:07:35.114886+0000","last_peered":"2024-09-16T00:07:35.114886+0000","last_clean":"2024-09-16T00:07:35.114886+0000","last_became_active":"2024-09-16T00:07:33.632608+0000","last_became_peered":"2024-09-16T00:07:33.632608+0000","last_unstale":"2024-09-16T00:07:35.114886+0000","last_undegraded":"2024-09-16T00:07:35.114886+0000","last_fullsized":"2024-09-16T00:07:35.114886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.612904+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:07:34.612904+0000","last_peered":"2024-09-16T00:07:34.612904+0000","last_clean":"2024-09-16T00:07:34.612904+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:07:34.612904+0000","last_undegraded":"2024-09-16T00:07:34.612904+0000","last_fullsized":"2024-09-16T00:07:34.612904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115409+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:07:35.115409+0000","last_peered":"2024-09-16T00:07:35.115409+0000","last_clean":"2024-09-16T00:07:35.115409+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:07:35.115409+0000","last_undegraded":"2024-09-16T00:07:35.115409+0000","last_fullsized":"2024-09-16T00:07:35.115409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614717+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:07:34.614717+0000","last_peered":"2024-09-16T00:07:34.614717+0000","last_clean":"2024-09-16T00:07:34.614717+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:07:34.614717+0000","last_undegraded":"2024-09-16T00:07:34.614717+0000","last_fullsized":"2024-09-16T00:07:34.614717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115480+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:07:35.115480+0000","last_peered":"2024-09-16T00:07:35.115480+0000","last_clean":"2024-09-16T00:07:35.115480+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:07:35.115480+0000","last_undegraded":"2024-09-16T00:07:35.115480+0000","last_fullsized":"2024-09-16T00:07:35.115480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613114+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:07:34.613114+0000","last_peered":"2024-09-16T00:07:34.613114+0000","last_clean":"2024-09-16T00:07:34.613114+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:07:34.613114+0000","last_undegraded":"2024-09-16T00:07:34.613114+0000","last_fullsized":"2024-09-16T00:07:34.613114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115888+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:07:35.115888+0000","last_peered":"2024-09-16T00:07:35.115888+0000","last_clean":"2024-09-16T00:07:35.115888+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:07:35.115888+0000","last_undegraded":"2024-09-16T00:07:35.115888+0000","last_fullsized":"2024-09-16T00:07:35.115888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115613+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:07:35.115613+0000","last_peered":"2024-09-16T00:07:35.115613+0000","last_clean":"2024-09-16T00:07:35.115613+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:07:35.115613+0000","last_undegraded":"2024-09-16T00:07:35.115613+0000","last_fullsized":"2024-09-16T00:07:35.115613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615597+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:07:34.615597+0000","last_peered":"2024-09-16T00:07:34.615597+0000","last_clean":"2024-09-16T00:07:34.615597+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:07:34.615597+0000","last_undegraded":"2024-09-16T00:07:34.615597+0000","last_fullsized":"2024-09-16T00:07:34.615597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613057+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:07:34.613057+0000","last_peered":"2024-09-16T00:07:34.613057+0000","last_clean":"2024-09-16T00:07:34.613057+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:07:34.613057+0000","last_undegraded":"2024-09-16T00:07:34.613057+0000","last_fullsized":"2024-09-16T00:07:34.613057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115577+0000","last_change":"2024-09-16T00:07:33.638312+0000","last_active":"2024-09-16T00:07:35.115577+0000","last_peered":"2024-09-16T00:07:35.115577+0000","last_clean":"2024-09-16T00:07:35.115577+0000","last_became_active":"2024-09-16T00:07:33.638049+0000","last_became_peered":"2024-09-16T00:07:33.638049+0000","last_unstale":"2024-09-16T00:07:35.115577+0000","last_undegraded":"2024-09-16T00:07:35.115577+0000","last_fullsized":"2024-09-16T00:07:35.115577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115169+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:07:35.115169+0000","last_peered":"2024-09-16T00:07:35.115169+0000","last_clean":"2024-09-16T00:07:35.115169+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:07:35.115169+0000","last_undegraded":"2024-09-16T00:07:35.115169+0000","last_fullsized":"2024-09-16T00:07:35.115169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614834+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:07:34.614834+0000","last_peered":"2024-09-16T00:07:34.614834+0000","last_clean":"2024-09-16T00:07:34.614834+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:07:34.614834+0000","last_undegraded":"2024-09-16T00:07:34.614834+0000","last_fullsized":"2024-09-16T00:07:34.614834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114892+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:07:35.114892+0000","last_peered":"2024-09-16T00:07:35.114892+0000","last_clean":"2024-09-16T00:07:35.114892+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:07:35.114892+0000","last_undegraded":"2024-09-16T00:07:35.114892+0000","last_fullsized":"2024-09-16T00:07:35.114892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115065+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:07:35.115065+0000","last_peered":"2024-09-16T00:07:35.115065+0000","last_clean":"2024-09-16T00:07:35.115065+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:07:35.115065+0000","last_undegraded":"2024-09-16T00:07:35.115065+0000","last_fullsized":"2024-09-16T00:07:35.115065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115288+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:07:35.115288+0000","last_peered":"2024-09-16T00:07:35.115288+0000","last_clean":"2024-09-16T00:07:35.115288+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:07:35.115288+0000","last_undegraded":"2024-09-16T00:07:35.115288+0000","last_fullsized":"2024-09-16T00:07:35.115288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.612974+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:07:34.612974+0000","last_peered":"2024-09-16T00:07:34.612974+0000","last_clean":"2024-09-16T00:07:34.612974+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:07:34.612974+0000","last_undegraded":"2024-09-16T00:07:34.612974+0000","last_fullsized":"2024-09-16T00:07:34.612974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114905+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:07:35.114905+0000","last_peered":"2024-09-16T00:07:35.114905+0000","last_clean":"2024-09-16T00:07:35.114905+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:07:35.114905+0000","last_undegraded":"2024-09-16T00:07:35.114905+0000","last_fullsized":"2024-09-16T00:07:35.114905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615621+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:07:34.615621+0000","last_peered":"2024-09-16T00:07:34.615621+0000","last_clean":"2024-09-16T00:07:34.615621+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:07:34.615621+0000","last_undegraded":"2024-09-16T00:07:34.615621+0000","last_fullsized":"2024-09-16T00:07:34.615621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615008+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:07:34.615008+0000","last_peered":"2024-09-16T00:07:34.615008+0000","last_clean":"2024-09-16T00:07:34.615008+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:07:34.615008+0000","last_undegraded":"2024-09-16T00:07:34.615008+0000","last_fullsized":"2024-09-16T00:07:34.615008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115398+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:07:35.115398+0000","last_peered":"2024-09-16T00:07:35.115398+0000","last_clean":"2024-09-16T00:07:35.115398+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:07:35.115398+0000","last_undegraded":"2024-09-16T00:07:35.115398+0000","last_fullsized":"2024-09-16T00:07:35.115398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615631+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:07:34.615631+0000","last_peered":"2024-09-16T00:07:34.615631+0000","last_clean":"2024-09-16T00:07:34.615631+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:07:34.615631+0000","last_undegraded":"2024-09-16T00:07:34.615631+0000","last_fullsized":"2024-09-16T00:07:34.615631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115814+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:07:35.115814+0000","last_peered":"2024-09-16T00:07:35.115814+0000","last_clean":"2024-09-16T00:07:35.115814+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:07:35.115814+0000","last_undegraded":"2024-09-16T00:07:35.115814+0000","last_fullsized":"2024-09-16T00:07:35.115814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115305+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:07:35.115305+0000","last_peered":"2024-09-16T00:07:35.115305+0000","last_clean":"2024-09-16T00:07:35.115305+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:07:35.115305+0000","last_undegraded":"2024-09-16T00:07:35.115305+0000","last_fullsized":"2024-09-16T00:07:35.115305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115644+0000","last_change":"2024-09-16T00:07:32.627236+0000","last_active":"2024-09-16T00:07:35.115644+0000","last_peered":"2024-09-16T00:07:35.115644+0000","last_clean":"2024-09-16T00:07:35.115644+0000","last_became_active":"2024-09-16T00:07:32.626743+0000","last_became_peered":"2024-09-16T00:07:32.626743+0000","last_unstale":"2024-09-16T00:07:35.115644+0000","last_undegraded":"2024-09-16T00:07:35.115644+0000","last_fullsized":"2024-09-16T00:07:35.115644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114835+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:07:35.114835+0000","last_peered":"2024-09-16T00:07:35.114835+0000","last_clean":"2024-09-16T00:07:35.114835+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:07:35.114835+0000","last_undegraded":"2024-09-16T00:07:35.114835+0000","last_fullsized":"2024-09-16T00:07:35.114835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115252+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:07:35.115252+0000","last_peered":"2024-09-16T00:07:35.115252+0000","last_clean":"2024-09-16T00:07:35.115252+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:07:35.115252+0000","last_undegraded":"2024-09-16T00:07:35.115252+0000","last_fullsized":"2024-09-16T00:07:35.115252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114804+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:07:35.114804+0000","last_peered":"2024-09-16T00:07:35.114804+0000","last_clean":"2024-09-16T00:07:35.114804+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:07:35.114804+0000","last_undegraded":"2024-09-16T00:07:35.114804+0000","last_fullsized":"2024-09-16T00:07:35.114804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114947+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:07:35.114947+0000","last_peered":"2024-09-16T00:07:35.114947+0000","last_clean":"2024-09-16T00:07:35.114947+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:07:35.114947+0000","last_undegraded":"2024-09-16T00:07:35.114947+0000","last_fullsized":"2024-09-16T00:07:35.114947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614501+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:07:34.614501+0000","last_peered":"2024-09-16T00:07:34.614501+0000","last_clean":"2024-09-16T00:07:34.614501+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:07:34.614501+0000","last_undegraded":"2024-09-16T00:07:34.614501+0000","last_fullsized":"2024-09-16T00:07:34.614501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114765+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:07:35.114765+0000","last_peered":"2024-09-16T00:07:35.114765+0000","last_clean":"2024-09-16T00:07:35.114765+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:07:35.114765+0000","last_undegraded":"2024-09-16T00:07:35.114765+0000","last_fullsized":"2024-09-16T00:07:35.114765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114640+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:07:35.114640+0000","last_peered":"2024-09-16T00:07:35.114640+0000","last_clean":"2024-09-16T00:07:35.114640+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:07:35.114640+0000","last_undegraded":"2024-09-16T00:07:35.114640+0000","last_fullsized":"2024-09-16T00:07:35.114640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115333+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:07:35.115333+0000","last_peered":"2024-09-16T00:07:35.115333+0000","last_clean":"2024-09-16T00:07:35.115333+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:07:35.115333+0000","last_undegraded":"2024-09-16T00:07:35.115333+0000","last_fullsized":"2024-09-16T00:07:35.115333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114803+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:07:35.114803+0000","last_peered":"2024-09-16T00:07:35.114803+0000","last_clean":"2024-09-16T00:07:35.114803+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:07:35.114803+0000","last_undegraded":"2024-09-16T00:07:35.114803+0000","last_fullsized":"2024-09-16T00:07:35.114803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115800+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:07:35.115800+0000","last_peered":"2024-09-16T00:07:35.115800+0000","last_clean":"2024-09-16T00:07:35.115800+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:07:35.115800+0000","last_undegraded":"2024-09-16T00:07:35.115800+0000","last_fullsized":"2024-09-16T00:07:35.115800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114993+0000","last_change":"2024-09-16T00:07:33.637979+0000","last_active":"2024-09-16T00:07:35.114993+0000","last_peered":"2024-09-16T00:07:35.114993+0000","last_clean":"2024-09-16T00:07:35.114993+0000","last_became_active":"2024-09-16T00:07:33.637713+0000","last_became_peered":"2024-09-16T00:07:33.637713+0000","last_unstale":"2024-09-16T00:07:35.114993+0000","last_undegraded":"2024-09-16T00:07:35.114993+0000","last_fullsized":"2024-09-16T00:07:35.114993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615387+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:07:34.615387+0000","last_peered":"2024-09-16T00:07:34.615387+0000","last_clean":"2024-09-16T00:07:34.615387+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:07:34.615387+0000","last_undegraded":"2024-09-16T00:07:34.615387+0000","last_fullsized":"2024-09-16T00:07:34.615387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115228+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:07:35.115228+0000","last_peered":"2024-09-16T00:07:35.115228+0000","last_clean":"2024-09-16T00:07:35.115228+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:07:35.115228+0000","last_undegraded":"2024-09-16T00:07:35.115228+0000","last_fullsized":"2024-09-16T00:07:35.115228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615266+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:07:34.615266+0000","last_peered":"2024-09-16T00:07:34.615266+0000","last_clean":"2024-09-16T00:07:34.615266+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:07:34.615266+0000","last_undegraded":"2024-09-16T00:07:34.615266+0000","last_fullsized":"2024-09-16T00:07:34.615266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115696+0000","last_change":"2024-09-16T00:07:32.632481+0000","last_active":"2024-09-16T00:07:35.115696+0000","last_peered":"2024-09-16T00:07:35.115696+0000","last_clean":"2024-09-16T00:07:35.115696+0000","last_became_active":"2024-09-16T00:07:32.632152+0000","last_became_peered":"2024-09-16T00:07:32.632152+0000","last_unstale":"2024-09-16T00:07:35.115696+0000","last_undegraded":"2024-09-16T00:07:35.115696+0000","last_fullsized":"2024-09-16T00:07:35.115696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114783+0000","last_change":"2024-09-16T00:07:33.629777+0000","last_active":"2024-09-16T00:07:35.114783+0000","last_peered":"2024-09-16T00:07:35.114783+0000","last_clean":"2024-09-16T00:07:35.114783+0000","last_became_active":"2024-09-16T00:07:33.627917+0000","last_became_peered":"2024-09-16T00:07:33.627917+0000","last_unstale":"2024-09-16T00:07:35.114783+0000","last_undegraded":"2024-09-16T00:07:35.114783+0000","last_fullsized":"2024-09-16T00:07:35.114783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:38.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114933+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:07:35.114933+0000","last_peered":"2024-09-16T00:07:35.114933+0000","last_clean":"2024-09-16T00:07:35.114933+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:07:35.114933+0000","last_undegraded":"2024-09-16T00:07:35.114933+0000","last_fullsized":"2024-09-16T00:07:35.114933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114899+0000","last_change":"2024-09-16T00:07:33.630746+0000","last_active":"2024-09-16T00:07:35.114899+0000","last_peered":"2024-09-16T00:07:35.114899+0000","last_clean":"2024-09-16T00:07:35.114899+0000","last_became_active":"2024-09-16T00:07:33.630480+0000","last_became_peered":"2024-09-16T00:07:33.630480+0000","last_unstale":"2024-09-16T00:07:35.114899+0000","last_undegraded":"2024-09-16T00:07:35.114899+0000","last_fullsized":"2024-09-16T00:07:35.114899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:50.931998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115042+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:07:35.115042+0000","last_peered":"2024-09-16T00:07:35.115042+0000","last_clean":"2024-09-16T00:07:35.115042+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:07:35.115042+0000","last_undegraded":"2024-09-16T00:07:35.115042+0000","last_fullsized":"2024-09-16T00:07:35.115042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":668364,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":668364,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:07:38.233 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-16T00:07:38.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.364+0000 7f080807b640 1 Processor -- start 2024-09-16T00:07:38.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.365+0000 7f080807b640 1 -- start start 2024-09-16T00:07:38.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.365+0000 7f080807b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f08001548c0 0x7f080006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.365+0000 7f080807b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08000c8a30 0x7f080006f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.365+0000 7f080807b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08000c7f90 con 0x7f080006bb20 2024-09-16T00:07:38.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.365+0000 7f080807b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f08000c9820 con 0x7f08001548c0 2024-09-16T00:07:38.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.365+0000 7f080807b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f08000cabd0 con 0x7f08000c8a30 2024-09-16T00:07:38.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.365+0000 7f0805616640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f080006bb20 0x7f080015a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45124/0 (socket says 172.21.15.139:45124) 2024-09-16T00:07:38.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.366+0000 7f0805616640 1 -- 172.21.15.139:0/3885602339 learned_addr learned my addr 172.21.15.139:0/3885602339 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:38.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.366+0000 7f0805e17640 1 --2- 172.21.15.139:0/3885602339 >> v2:172.21.15.164:3300/0 conn(0x7f08001548c0 0x7f080006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.366+0000 7f0806618640 1 --2- 172.21.15.139:0/3885602339 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08000c8a30 0x7f080006f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.366+0000 7f0804e15640 1 -- 172.21.15.139:0/3885602339 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 377942448 0 0) 0x7f08000c7f90 con 0x7f080006bb20 2024-09-16T00:07:38.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.366+0000 7f0804e15640 1 -- 172.21.15.139:0/3885602339 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07ec003520 con 0x7f080006bb20 2024-09-16T00:07:38.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.366+0000 7f0805e17640 1 -- 172.21.15.139:0/3885602339 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08000c8a30 msgr2=0x7f080006f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:38.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.366+0000 7f0805e17640 1 --2- 172.21.15.139:0/3885602339 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08000c8a30 0x7f080006f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.366+0000 7f0805e17640 1 -- 172.21.15.139:0/3885602339 >> v1:172.21.15.139:6789/0 conn(0x7f080006bb20 legacy=0x7f080015a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.366+0000 7f0805e17640 1 -- 172.21.15.139:0/3885602339 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08000c8250 con 0x7f08001548c0 2024-09-16T00:07:38.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.366+0000 7f0806618640 1 --2- 172.21.15.139:0/3885602339 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08000c8a30 0x7f080006f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:38.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.367+0000 7f0805e17640 1 --2- 172.21.15.139:0/3885602339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08001548c0 0x7f080006eac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f07f40097a0 tx=0x7f07f402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8481bcbeb1185e07 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:38.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.367+0000 7f0804e15640 1 -- 172.21.15.139:0/3885602339 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f4037070 con 0x7f08001548c0 2024-09-16T00:07:38.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.367+0000 7f0804e15640 1 -- 172.21.15.139:0/3885602339 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07f4032440 con 0x7f08001548c0 2024-09-16T00:07:38.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.367+0000 7f0804e15640 1 -- 172.21.15.139:0/3885602339 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f40327f0 con 0x7f08001548c0 2024-09-16T00:07:38.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.367+0000 7f080807b640 1 -- 172.21.15.139:0/3885602339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08001548c0 msgr2=0x7f080006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.367+0000 7f080807b640 1 --2- 172.21.15.139:0/3885602339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08001548c0 0x7f080006eac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f07f40097a0 tx=0x7f07f402a210 comp rx=0 tx=0).stop 2024-09-16T00:07:38.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.367+0000 7f080807b640 1 -- 172.21.15.139:0/3885602339 shutdown_connections 2024-09-16T00:07:38.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.367+0000 7f080807b640 1 --2- 172.21.15.139:0/3885602339 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08000c8a30 0x7f080006f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.367+0000 7f080807b640 1 --2- 172.21.15.139:0/3885602339 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08001548c0 0x7f080006eac0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.367+0000 7f080807b640 1 -- 172.21.15.139:0/3885602339 >> 172.21.15.139:0/3885602339 conn(0x7f0800067950 msgr2=0x7f0800067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:38.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.368+0000 7f080807b640 1 -- 172.21.15.139:0/3885602339 shutdown_connections 2024-09-16T00:07:38.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.368+0000 7f080807b640 1 -- 172.21.15.139:0/3885602339 wait complete. 2024-09-16T00:07:38.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.369+0000 7f080807b640 1 Processor -- start 2024-09-16T00:07:38.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.369+0000 7f080807b640 1 -- start start 2024-09-16T00:07:38.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.369+0000 7f080807b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f080006bb20 0x7f0800149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.369+0000 7f080807b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08000c8a30 0x7f0800156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.369+0000 7f080807b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08001548c0 0x7f080014fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.369+0000 7f080807b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f08000c91d0 con 0x7f08001548c0 2024-09-16T00:07:38.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.369+0000 7f080807b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f08000c8ef0 con 0x7f08000c8a30 2024-09-16T00:07:38.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.369+0000 7f080807b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f08000c94b0 con 0x7f080006bb20 2024-09-16T00:07:38.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f0806618640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08001548c0 0x7f080014fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f0806618640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08001548c0 0x7f080014fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40294/0 (socket says 172.21.15.139:40294) 2024-09-16T00:07:38.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f0806618640 1 -- 172.21.15.139:0/2451867960 learned_addr learned my addr 172.21.15.139:0/2451867960 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:38.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f0805616640 1 --2- 172.21.15.139:0/2451867960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08000c8a30 0x7f0800156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f0805e17640 1 --2- 172.21.15.139:0/2451867960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f080006bb20 0x7f0800149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f0806618640 1 -- 172.21.15.139:0/2451867960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f080006bb20 msgr2=0x7f0800149ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f0806618640 1 --2- 172.21.15.139:0/2451867960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f080006bb20 0x7f0800149ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f0806618640 1 -- 172.21.15.139:0/2451867960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08000c8a30 msgr2=0x7f0800156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f0806618640 1 --2- 172.21.15.139:0/2451867960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08000c8a30 0x7f0800156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f0806618640 1 -- 172.21.15.139:0/2451867960 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08001567c0 con 0x7f08001548c0 2024-09-16T00:07:38.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f0805616640 1 --2- 172.21.15.139:0/2451867960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08000c8a30 0x7f0800156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:38.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f0805e17640 1 --2- 172.21.15.139:0/2451867960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f080006bb20 0x7f0800149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:38.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f0806618640 1 --2- 172.21.15.139:0/2451867960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08001548c0 0x7f080014fe20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009f00 tx=0x7f07f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:38.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f07f27fc640 1 -- 172.21.15.139:0/2451867960 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8018030 con 0x7f08001548c0 2024-09-16T00:07:38.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f07f27fc640 1 -- 172.21.15.139:0/2451867960 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07f800ce50 con 0x7f08001548c0 2024-09-16T00:07:38.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f07f27fc640 1 -- 172.21.15.139:0/2451867960 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8003bb0 con 0x7f08001548c0 2024-09-16T00:07:38.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f080807b640 1 -- 172.21.15.139:0/2451867960 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08001f0d30 con 0x7f08001548c0 2024-09-16T00:07:38.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.370+0000 7f080807b640 1 -- 172.21.15.139:0/2451867960 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0800155290 con 0x7f08001548c0 2024-09-16T00:07:38.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.371+0000 7f080807b640 1 -- 172.21.15.139:0/2451867960 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08000c7c10 con 0x7f08001548c0 2024-09-16T00:07:38.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.372+0000 7f07f27fc640 1 -- 172.21.15.139:0/2451867960 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f07f801b070 con 0x7f08001548c0 2024-09-16T00:07:38.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.372+0000 7f07f27fc640 1 --2- 172.21.15.139:0/2451867960 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07e406c2e0 0x7f07e406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.372+0000 7f07f27fc640 1 -- 172.21.15.139:0/2451867960 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f07f808bdf0 con 0x7f08001548c0 2024-09-16T00:07:38.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.372+0000 7f0805e17640 1 --2- 172.21.15.139:0/2451867960 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07e406c2e0 0x7f07e406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.373+0000 7f0805e17640 1 --2- 172.21.15.139:0/2451867960 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07e406c2e0 0x7f07e406e7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f07f4034000 tx=0x7f07f402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:38.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.376+0000 7f07f27fc640 1 -- 172.21.15.139:0/2451867960 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f07f8010040 con 0x7f08001548c0 2024-09-16T00:07:38.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.604+0000 7f080807b640 1 -- 172.21.15.139:0/2451867960 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f0800151e80 con 0x7f08001548c0 2024-09-16T00:07:38.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.604+0000 7f07f27fc640 1 -- 172.21.15.139:0/2451867960 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1151 (secure 0 0 0) 0x7f07f8012340 con 0x7f08001548c0 2024-09-16T00:07:38.605 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:38.605 INFO:teuthology.orchestra.run.smithi139.stdout:{"fsid":"2d3366e5-6a59-4b59-93b2-615e72ac7015","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":28,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":22,"num_osds":8,"num_up_osds":8,"osd_up_since":1726445241,"num_in_osds":8,"osd_in_since":1726445232,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":73},{"state_name":"creating+peering","count":17},{"state_name":"active+clean","count":7}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":223027200,"bytes_avail":772871086080,"bytes_total":773094113280,"unknown_pgs_ratio":0.75257730484008789,"inactive_pgs_ratio":0.17525772750377655},"fsmap":{"epoch":7,"btime":"2024-09-16T00:07:36:629927+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"f","status":"up:creating","gid":4300}],"up:standby":5},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-16T00:07:09.968855+0000","services":{}},"progress_events":{}} 2024-09-16T00:07:38.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.606+0000 7f080807b640 1 -- 172.21.15.139:0/2451867960 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07e406c2e0 msgr2=0x7f07e406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.606+0000 7f080807b640 1 --2- 172.21.15.139:0/2451867960 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07e406c2e0 0x7f07e406e7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f07f4034000 tx=0x7f07f402e000 comp rx=0 tx=0).stop 2024-09-16T00:07:38.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.606+0000 7f080807b640 1 -- 172.21.15.139:0/2451867960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08001548c0 msgr2=0x7f080014fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.606+0000 7f080807b640 1 --2- 172.21.15.139:0/2451867960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08001548c0 0x7f080014fe20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009f00 tx=0x7f07f800c490 comp rx=0 tx=0).stop 2024-09-16T00:07:38.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.606+0000 7f080807b640 1 -- 172.21.15.139:0/2451867960 shutdown_connections 2024-09-16T00:07:38.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.606+0000 7f080807b640 1 --2- 172.21.15.139:0/2451867960 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07e406c2e0 0x7f07e406e7a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.606+0000 7f080807b640 1 --2- 172.21.15.139:0/2451867960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08001548c0 0x7f080014fe20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.606+0000 7f080807b640 1 --2- 172.21.15.139:0/2451867960 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08000c8a30 0x7f0800156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.606+0000 7f080807b640 1 --2- 172.21.15.139:0/2451867960 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f080006bb20 0x7f0800149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.606+0000 7f080807b640 1 -- 172.21.15.139:0/2451867960 >> 172.21.15.139:0/2451867960 conn(0x7f0800067950 msgr2=0x7f0800151660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:38.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.606+0000 7f080807b640 1 -- 172.21.15.139:0/2451867960 shutdown_connections 2024-09-16T00:07:38.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.606+0000 7f080807b640 1 -- 172.21.15.139:0/2451867960 wait complete. 2024-09-16T00:07:38.620 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-16T00:07:38.621 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:07:38.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.717+0000 7f49cfda6640 1 Processor -- start 2024-09-16T00:07:38.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.718+0000 7f49cfda6640 1 -- start start 2024-09-16T00:07:38.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.718+0000 7f49cfda6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f49c8151810 0x7f49c8073b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.718+0000 7f49cfda6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49c80c3910 0x7f49c80c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.718+0000 7f49cfda6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49c815ab60 con 0x7f49c80afbf0 2024-09-16T00:07:38.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.718+0000 7f49cfda6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f49c8157940 con 0x7f49c8151810 2024-09-16T00:07:38.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.718+0000 7f49cfda6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f49c81575f0 con 0x7f49c80c3910 2024-09-16T00:07:38.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.718+0000 7f49cdb42640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f49c8151810 0x7f49c8073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.718+0000 7f49cd341640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f49c80afbf0 0x7f49c8157dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45140/0 (socket says 172.21.15.139:45140) 2024-09-16T00:07:38.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.718+0000 7f49cdb42640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f49c8151810 0x7f49c8073b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40304/0 (socket says 172.21.15.139:40304) 2024-09-16T00:07:38.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.718+0000 7f49cd341640 1 -- 172.21.15.139:0/4062136305 learned_addr learned my addr 172.21.15.139:0/4062136305 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:38.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.719+0000 7f49ce343640 1 --2- 172.21.15.139:0/4062136305 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49c80c3910 0x7f49c80c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.719+0000 7f49cdb42640 1 -- 172.21.15.139:0/4062136305 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49c80c3910 msgr2=0x7f49c80c3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:38.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.719+0000 7f49cdb42640 1 --2- 172.21.15.139:0/4062136305 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49c80c3910 0x7f49c80c3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.719+0000 7f49cdb42640 1 -- 172.21.15.139:0/4062136305 >> v1:172.21.15.139:6789/0 conn(0x7f49c80afbf0 legacy=0x7f49c8157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.719+0000 7f49cdb42640 1 -- 172.21.15.139:0/4062136305 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c80c7ed0 con 0x7f49c8151810 2024-09-16T00:07:38.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.719+0000 7f49ccb40640 1 -- 172.21.15.139:0/4062136305 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 31316498 0 0) 0x7f49c815ab60 con 0x7f49c80afbf0 2024-09-16T00:07:38.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.719+0000 7f49ce343640 1 --2- 172.21.15.139:0/4062136305 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49c80c3910 0x7f49c80c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:38.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.719+0000 7f49cdb42640 1 --2- 172.21.15.139:0/4062136305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49c8151810 0x7f49c8073b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0097a0 tx=0x7f49bc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6852ac14762dd61 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:38.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.720+0000 7f49ccb40640 1 -- 172.21.15.139:0/4062136305 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc037070 con 0x7f49c8151810 2024-09-16T00:07:38.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.720+0000 7f49ccb40640 1 -- 172.21.15.139:0/4062136305 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49bc032440 con 0x7f49c8151810 2024-09-16T00:07:38.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.720+0000 7f49ccb40640 1 -- 172.21.15.139:0/4062136305 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc032810 con 0x7f49c8151810 2024-09-16T00:07:38.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.720+0000 7f49cfda6640 1 -- 172.21.15.139:0/4062136305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49c8151810 msgr2=0x7f49c8073b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.720+0000 7f49cfda6640 1 --2- 172.21.15.139:0/4062136305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49c8151810 0x7f49c8073b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0097a0 tx=0x7f49bc02a210 comp rx=0 tx=0).stop 2024-09-16T00:07:38.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.720+0000 7f49cfda6640 1 -- 172.21.15.139:0/4062136305 shutdown_connections 2024-09-16T00:07:38.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.720+0000 7f49cfda6640 1 --2- 172.21.15.139:0/4062136305 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49c80c3910 0x7f49c80c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.720+0000 7f49cfda6640 1 --2- 172.21.15.139:0/4062136305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49c8151810 0x7f49c8073b00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.720+0000 7f49cfda6640 1 -- 172.21.15.139:0/4062136305 >> 172.21.15.139:0/4062136305 conn(0x7f49c8070440 msgr2=0x7f49c8070820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:38.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.720+0000 7f49cfda6640 1 -- 172.21.15.139:0/4062136305 shutdown_connections 2024-09-16T00:07:38.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.720+0000 7f49cfda6640 1 -- 172.21.15.139:0/4062136305 wait complete. 2024-09-16T00:07:38.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.721+0000 7f49cfda6640 1 Processor -- start 2024-09-16T00:07:38.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.721+0000 7f49cfda6640 1 -- start start 2024-09-16T00:07:38.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.722+0000 7f49cfda6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49c80afbf0 0x7f49c814d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.722+0000 7f49cfda6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49c80c3910 0x7f49c80cc380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.722+0000 7f49cfda6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49c8151810 0x7f49c814c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.722+0000 7f49cfda6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f49c80c8770 con 0x7f49c8151810 2024-09-16T00:07:38.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.722+0000 7f49cfda6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f49c80c8490 con 0x7f49c80c3910 2024-09-16T00:07:38.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.722+0000 7f49cfda6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f49c81584e0 con 0x7f49c80afbf0 2024-09-16T00:07:38.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.722+0000 7f49ce343640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49c8151810 0x7f49c814c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.722+0000 7f49ce343640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49c8151810 0x7f49c814c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40308/0 (socket says 172.21.15.139:40308) 2024-09-16T00:07:38.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.722+0000 7f49ce343640 1 -- 172.21.15.139:0/489643928 learned_addr learned my addr 172.21.15.139:0/489643928 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:38.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.722+0000 7f49cdb42640 1 --2- 172.21.15.139:0/489643928 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49c80afbf0 0x7f49c814d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.722+0000 7f49cd341640 1 --2- 172.21.15.139:0/489643928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49c80c3910 0x7f49c80cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.722+0000 7f49ce343640 1 -- 172.21.15.139:0/489643928 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49c80afbf0 msgr2=0x7f49c814d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.722+0000 7f49ce343640 1 --2- 172.21.15.139:0/489643928 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49c80afbf0 0x7f49c814d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.722+0000 7f49ce343640 1 -- 172.21.15.139:0/489643928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49c80c3910 msgr2=0x7f49c80cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.722+0000 7f49ce343640 1 --2- 172.21.15.139:0/489643928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49c80c3910 0x7f49c80cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.722+0000 7f49ce343640 1 -- 172.21.15.139:0/489643928 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c814dc20 con 0x7f49c8151810 2024-09-16T00:07:38.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.723+0000 7f49ce343640 1 --2- 172.21.15.139:0/489643928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49c8151810 0x7f49c814c390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f49c000cef0 tx=0x7f49c000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:38.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.723+0000 7f49ba7fc640 1 -- 172.21.15.139:0/489643928 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c001a070 con 0x7f49c8151810 2024-09-16T00:07:38.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.723+0000 7f49ba7fc640 1 -- 172.21.15.139:0/489643928 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c000d860 con 0x7f49c8151810 2024-09-16T00:07:38.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.723+0000 7f49ba7fc640 1 -- 172.21.15.139:0/489643928 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c000dc50 con 0x7f49c8151810 2024-09-16T00:07:38.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.723+0000 7f49cfda6640 1 -- 172.21.15.139:0/489643928 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c80b3450 con 0x7f49c8151810 2024-09-16T00:07:38.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.723+0000 7f49cfda6640 1 -- 172.21.15.139:0/489643928 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c814be30 con 0x7f49c8151810 2024-09-16T00:07:38.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.724+0000 7f49cfda6640 1 -- 172.21.15.139:0/489643928 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c8069000 con 0x7f49c8151810 2024-09-16T00:07:38.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.724+0000 7f49ba7fc640 1 -- 172.21.15.139:0/489643928 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f49c001d070 con 0x7f49c8151810 2024-09-16T00:07:38.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.725+0000 7f49ba7fc640 1 --2- 172.21.15.139:0/489643928 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f49ac06c2b0 0x7f49ac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:38.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.725+0000 7f49ba7fc640 1 -- 172.21.15.139:0/489643928 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f49c008bd60 con 0x7f49c8151810 2024-09-16T00:07:38.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.725+0000 7f49cdb42640 1 --2- 172.21.15.139:0/489643928 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f49ac06c2b0 0x7f49ac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:38.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.725+0000 7f49cdb42640 1 --2- 172.21.15.139:0/489643928 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f49ac06c2b0 0x7f49ac06e770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f49bc034000 tx=0x7f49bc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:38.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.729+0000 7f49ba7fc640 1 -- 172.21.15.139:0/489643928 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f49c8069000 con 0x7f49c8151810 2024-09-16T00:07:38.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.892+0000 7f49cfda6640 1 -- 172.21.15.139:0/489643928 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f49c80c7660 con 0x7f49ac06c2b0 2024-09-16T00:07:38.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.897+0000 7f49ba7fc640 1 -- 172.21.15.139:0/489643928 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251926 (secure 0 0 0) 0x7f49c80c7660 con 0x7f49ac06c2b0 2024-09-16T00:07:38.901 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:38.902 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:07:38.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.900+0000 7f49cfda6640 1 -- 172.21.15.139:0/489643928 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f49ac06c2b0 msgr2=0x7f49ac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.900+0000 7f49cfda6640 1 --2- 172.21.15.139:0/489643928 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f49ac06c2b0 0x7f49ac06e770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f49bc034000 tx=0x7f49bc02e000 comp rx=0 tx=0).stop 2024-09-16T00:07:38.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.900+0000 7f49cfda6640 1 -- 172.21.15.139:0/489643928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49c8151810 msgr2=0x7f49c814c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:38.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.900+0000 7f49cfda6640 1 --2- 172.21.15.139:0/489643928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49c8151810 0x7f49c814c390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f49c000cef0 tx=0x7f49c000a710 comp rx=0 tx=0).stop 2024-09-16T00:07:38.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.900+0000 7f49cfda6640 1 -- 172.21.15.139:0/489643928 shutdown_connections 2024-09-16T00:07:38.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.900+0000 7f49cfda6640 1 --2- 172.21.15.139:0/489643928 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f49ac06c2b0 0x7f49ac06e770 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.900+0000 7f49cfda6640 1 --2- 172.21.15.139:0/489643928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49c8151810 0x7f49c814c390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.900+0000 7f49cfda6640 1 --2- 172.21.15.139:0/489643928 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49c80c3910 0x7f49c80cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.900+0000 7f49cfda6640 1 --2- 172.21.15.139:0/489643928 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49c80afbf0 0x7f49c814d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:38.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.900+0000 7f49cfda6640 1 -- 172.21.15.139:0/489643928 >> 172.21.15.139:0/489643928 conn(0x7f49c8070440 msgr2=0x7f49c80aebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:38.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.900+0000 7f49cfda6640 1 -- 172.21.15.139:0/489643928 shutdown_connections 2024-09-16T00:07:38.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:38.900+0000 7f49cfda6640 1 -- 172.21.15.139:0/489643928 wait complete. 2024-09-16T00:07:38.912 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-16T00:07:38.026369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218104,"kb_used_data":1856,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756616,"statfs":{"total":773094113280,"available":772870774784,"internally_reserved":0,"allocated":1900544,"data_stored":1804704,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000886"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115528+0000","last_change":"2024-09-16T00:07:33.635096+0000","last_active":"2024-09-16T00:07:35.115528+0000","last_peered":"2024-09-16T00:07:35.115528+0000","last_clean":"2024-09-16T00:07:35.115528+0000","last_became_active":"2024-09-16T00:07:33.634750+0000","last_became_peered":"2024-09-16T00:07:33.634750+0000","last_unstale":"2024-09-16T00:07:35.115528+0000","last_undegraded":"2024-09-16T00:07:35.115528+0000","last_fullsized":"2024-09-16T00:07:35.115528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115558+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:07:35.115558+0000","last_peered":"2024-09-16T00:07:35.115558+0000","last_clean":"2024-09-16T00:07:35.115558+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:07:35.115558+0000","last_undegraded":"2024-09-16T00:07:35.115558+0000","last_fullsized":"2024-09-16T00:07:35.115558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115392+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:07:35.115392+0000","last_peered":"2024-09-16T00:07:35.115392+0000","last_clean":"2024-09-16T00:07:35.115392+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:07:35.115392+0000","last_undegraded":"2024-09-16T00:07:35.115392+0000","last_fullsized":"2024-09-16T00:07:35.115392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615167+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:07:34.615167+0000","last_peered":"2024-09-16T00:07:34.615167+0000","last_clean":"2024-09-16T00:07:34.615167+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:07:34.615167+0000","last_undegraded":"2024-09-16T00:07:34.615167+0000","last_fullsized":"2024-09-16T00:07:34.615167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115724+0000","last_change":"2024-09-16T00:07:33.630029+0000","last_active":"2024-09-16T00:07:35.115724+0000","last_peered":"2024-09-16T00:07:35.115724+0000","last_clean":"2024-09-16T00:07:35.115724+0000","last_became_active":"2024-09-16T00:07:33.629712+0000","last_became_peered":"2024-09-16T00:07:33.629712+0000","last_unstale":"2024-09-16T00:07:35.115724+0000","last_undegraded":"2024-09-16T00:07:35.115724+0000","last_fullsized":"2024-09-16T00:07:35.115724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614973+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:07:34.614973+0000","last_peered":"2024-09-16T00:07:34.614973+0000","last_clean":"2024-09-16T00:07:34.614973+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:07:34.614973+0000","last_undegraded":"2024-09-16T00:07:34.614973+0000","last_fullsized":"2024-09-16T00:07:34.614973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115522+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:07:35.115522+0000","last_peered":"2024-09-16T00:07:35.115522+0000","last_clean":"2024-09-16T00:07:35.115522+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:07:35.115522+0000","last_undegraded":"2024-09-16T00:07:35.115522+0000","last_fullsized":"2024-09-16T00:07:35.115522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114492+0000","last_change":"2024-09-16T00:07:33.624708+0000","last_active":"2024-09-16T00:07:35.114492+0000","last_peered":"2024-09-16T00:07:35.114492+0000","last_clean":"2024-09-16T00:07:35.114492+0000","last_became_active":"2024-09-16T00:07:33.624458+0000","last_became_peered":"2024-09-16T00:07:33.624458+0000","last_unstale":"2024-09-16T00:07:35.114492+0000","last_undegraded":"2024-09-16T00:07:35.114492+0000","last_fullsized":"2024-09-16T00:07:35.114492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:48.052488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115751+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:07:35.115751+0000","last_peered":"2024-09-16T00:07:35.115751+0000","last_clean":"2024-09-16T00:07:35.115751+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:07:35.115751+0000","last_undegraded":"2024-09-16T00:07:35.115751+0000","last_fullsized":"2024-09-16T00:07:35.115751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115614+0000","last_change":"2024-09-16T00:07:33.635973+0000","last_active":"2024-09-16T00:07:35.115614+0000","last_peered":"2024-09-16T00:07:35.115614+0000","last_clean":"2024-09-16T00:07:35.115614+0000","last_became_active":"2024-09-16T00:07:33.635486+0000","last_became_peered":"2024-09-16T00:07:33.635486+0000","last_unstale":"2024-09-16T00:07:35.115614+0000","last_undegraded":"2024-09-16T00:07:35.115614+0000","last_fullsized":"2024-09-16T00:07:35.115614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115027+0000","last_change":"2024-09-16T00:07:33.636971+0000","last_active":"2024-09-16T00:07:35.115027+0000","last_peered":"2024-09-16T00:07:35.115027+0000","last_clean":"2024-09-16T00:07:35.115027+0000","last_became_active":"2024-09-16T00:07:33.635760+0000","last_became_peered":"2024-09-16T00:07:33.635760+0000","last_unstale":"2024-09-16T00:07:35.115027+0000","last_undegraded":"2024-09-16T00:07:35.115027+0000","last_fullsized":"2024-09-16T00:07:35.115027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614631+0000","last_change":"2024-09-16T00:07:33.632261+0000","last_active":"2024-09-16T00:07:34.614631+0000","last_peered":"2024-09-16T00:07:34.614631+0000","last_clean":"2024-09-16T00:07:34.614631+0000","last_became_active":"2024-09-16T00:07:33.631944+0000","last_became_peered":"2024-09-16T00:07:33.631944+0000","last_unstale":"2024-09-16T00:07:34.614631+0000","last_undegraded":"2024-09-16T00:07:34.614631+0000","last_fullsized":"2024-09-16T00:07:34.614631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:35.052417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114641+0000","last_change":"2024-09-16T00:07:33.628273+0000","last_active":"2024-09-16T00:07:35.114641+0000","last_peered":"2024-09-16T00:07:35.114641+0000","last_clean":"2024-09-16T00:07:35.114641+0000","last_became_active":"2024-09-16T00:07:33.627960+0000","last_became_peered":"2024-09-16T00:07:33.627960+0000","last_unstale":"2024-09-16T00:07:35.114641+0000","last_undegraded":"2024-09-16T00:07:35.114641+0000","last_fullsized":"2024-09-16T00:07:35.114641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:36.451688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114782+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:07:35.114782+0000","last_peered":"2024-09-16T00:07:35.114782+0000","last_clean":"2024-09-16T00:07:35.114782+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:07:35.114782+0000","last_undegraded":"2024-09-16T00:07:35.114782+0000","last_fullsized":"2024-09-16T00:07:35.114782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114664+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:07:35.114664+0000","last_peered":"2024-09-16T00:07:35.114664+0000","last_clean":"2024-09-16T00:07:35.114664+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:07:35.114664+0000","last_undegraded":"2024-09-16T00:07:35.114664+0000","last_fullsized":"2024-09-16T00:07:35.114664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614471+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:07:34.614471+0000","last_peered":"2024-09-16T00:07:34.614471+0000","last_clean":"2024-09-16T00:07:34.614471+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:07:34.614471+0000","last_undegraded":"2024-09-16T00:07:34.614471+0000","last_fullsized":"2024-09-16T00:07:34.614471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615697+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:07:34.615697+0000","last_peered":"2024-09-16T00:07:34.615697+0000","last_clean":"2024-09-16T00:07:34.615697+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:07:34.615697+0000","last_undegraded":"2024-09-16T00:07:34.615697+0000","last_fullsized":"2024-09-16T00:07:34.615697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613100+0000","last_change":"2024-09-16T00:07:33.630797+0000","last_active":"2024-09-16T00:07:34.613100+0000","last_peered":"2024-09-16T00:07:34.613100+0000","last_clean":"2024-09-16T00:07:34.613100+0000","last_became_active":"2024-09-16T00:07:33.630491+0000","last_became_peered":"2024-09-16T00:07:33.630491+0000","last_unstale":"2024-09-16T00:07:34.613100+0000","last_undegraded":"2024-09-16T00:07:34.613100+0000","last_fullsized":"2024-09-16T00:07:34.613100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:20.097626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115840+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:07:35.115840+0000","last_peered":"2024-09-16T00:07:35.115840+0000","last_clean":"2024-09-16T00:07:35.115840+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:07:35.115840+0000","last_undegraded":"2024-09-16T00:07:35.115840+0000","last_fullsized":"2024-09-16T00:07:35.115840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114960+0000","last_change":"2024-09-16T00:07:33.638554+0000","last_active":"2024-09-16T00:07:35.114960+0000","last_peered":"2024-09-16T00:07:35.114960+0000","last_clean":"2024-09-16T00:07:35.114960+0000","last_became_active":"2024-09-16T00:07:33.638263+0000","last_became_peered":"2024-09-16T00:07:33.638263+0000","last_unstale":"2024-09-16T00:07:35.114960+0000","last_undegraded":"2024-09-16T00:07:35.114960+0000","last_fullsized":"2024-09-16T00:07:35.114960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115286+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:07:35.115286+0000","last_peered":"2024-09-16T00:07:35.115286+0000","last_clean":"2024-09-16T00:07:35.115286+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:07:35.115286+0000","last_undegraded":"2024-09-16T00:07:35.115286+0000","last_fullsized":"2024-09-16T00:07:35.115286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615450+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:07:34.615450+0000","last_peered":"2024-09-16T00:07:34.615450+0000","last_clean":"2024-09-16T00:07:34.615450+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:07:34.615450+0000","last_undegraded":"2024-09-16T00:07:34.615450+0000","last_fullsized":"2024-09-16T00:07:34.615450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115015+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:07:35.115015+0000","last_peered":"2024-09-16T00:07:35.115015+0000","last_clean":"2024-09-16T00:07:35.115015+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:07:35.115015+0000","last_undegraded":"2024-09-16T00:07:35.115015+0000","last_fullsized":"2024-09-16T00:07:35.115015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614616+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:07:34.614616+0000","last_peered":"2024-09-16T00:07:34.614616+0000","last_clean":"2024-09-16T00:07:34.614616+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:07:34.614616+0000","last_undegraded":"2024-09-16T00:07:34.614616+0000","last_fullsized":"2024-09-16T00:07:34.614616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114694+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:07:35.114694+0000","last_peered":"2024-09-16T00:07:35.114694+0000","last_clean":"2024-09-16T00:07:35.114694+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:07:35.114694+0000","last_undegraded":"2024-09-16T00:07:35.114694+0000","last_fullsized":"2024-09-16T00:07:35.114694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115306+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:07:35.115306+0000","last_peered":"2024-09-16T00:07:35.115306+0000","last_clean":"2024-09-16T00:07:35.115306+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:07:35.115306+0000","last_undegraded":"2024-09-16T00:07:35.115306+0000","last_fullsized":"2024-09-16T00:07:35.115306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115543+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:07:35.115543+0000","last_peered":"2024-09-16T00:07:35.115543+0000","last_clean":"2024-09-16T00:07:35.115543+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:07:35.115543+0000","last_undegraded":"2024-09-16T00:07:35.115543+0000","last_fullsized":"2024-09-16T00:07:35.115543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114985+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:07:35.114985+0000","last_peered":"2024-09-16T00:07:35.114985+0000","last_clean":"2024-09-16T00:07:35.114985+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:07:35.114985+0000","last_undegraded":"2024-09-16T00:07:35.114985+0000","last_fullsized":"2024-09-16T00:07:35.114985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115448+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:07:35.115448+0000","last_peered":"2024-09-16T00:07:35.115448+0000","last_clean":"2024-09-16T00:07:35.115448+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:07:35.115448+0000","last_undegraded":"2024-09-16T00:07:35.115448+0000","last_fullsized":"2024-09-16T00:07:35.115448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615338+0000","last_change":"2024-09-16T00:07:33.627789+0000","last_active":"2024-09-16T00:07:34.615338+0000","last_peered":"2024-09-16T00:07:34.615338+0000","last_clean":"2024-09-16T00:07:34.615338+0000","last_became_active":"2024-09-16T00:07:33.627500+0000","last_became_peered":"2024-09-16T00:07:33.627500+0000","last_unstale":"2024-09-16T00:07:34.615338+0000","last_undegraded":"2024-09-16T00:07:34.615338+0000","last_fullsized":"2024-09-16T00:07:34.615338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613050+0000","last_change":"2024-09-16T00:07:33.621107+0000","last_active":"2024-09-16T00:07:34.613050+0000","last_peered":"2024-09-16T00:07:34.613050+0000","last_clean":"2024-09-16T00:07:34.613050+0000","last_became_active":"2024-09-16T00:07:33.620858+0000","last_became_peered":"2024-09-16T00:07:33.620858+0000","last_unstale":"2024-09-16T00:07:34.613050+0000","last_undegraded":"2024-09-16T00:07:34.613050+0000","last_fullsized":"2024-09-16T00:07:34.613050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:30.176079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115031+0000","last_change":"2024-09-16T00:07:33.633578+0000","last_active":"2024-09-16T00:07:35.115031+0000","last_peered":"2024-09-16T00:07:35.115031+0000","last_clean":"2024-09-16T00:07:35.115031+0000","last_became_active":"2024-09-16T00:07:33.633310+0000","last_became_peered":"2024-09-16T00:07:33.633310+0000","last_unstale":"2024-09-16T00:07:35.115031+0000","last_undegraded":"2024-09-16T00:07:35.115031+0000","last_fullsized":"2024-09-16T00:07:35.115031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614556+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:07:34.614556+0000","last_peered":"2024-09-16T00:07:34.614556+0000","last_clean":"2024-09-16T00:07:34.614556+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:07:34.614556+0000","last_undegraded":"2024-09-16T00:07:34.614556+0000","last_fullsized":"2024-09-16T00:07:34.614556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115057+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:07:35.115057+0000","last_peered":"2024-09-16T00:07:35.115057+0000","last_clean":"2024-09-16T00:07:35.115057+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:07:35.115057+0000","last_undegraded":"2024-09-16T00:07:35.115057+0000","last_fullsized":"2024-09-16T00:07:35.115057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115509+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:07:35.115509+0000","last_peered":"2024-09-16T00:07:35.115509+0000","last_clean":"2024-09-16T00:07:35.115509+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:07:35.115509+0000","last_undegraded":"2024-09-16T00:07:35.115509+0000","last_fullsized":"2024-09-16T00:07:35.115509+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114784+0000","last_change":"2024-09-16T00:07:32.626800+0000","last_active":"2024-09-16T00:07:35.114784+0000","last_peered":"2024-09-16T00:07:35.114784+0000","last_clean":"2024-09-16T00:07:35.114784+0000","last_became_active":"2024-09-16T00:07:32.626511+0000","last_became_peered":"2024-09-16T00:07:32.626511+0000","last_unstale":"2024-09-16T00:07:35.114784+0000","last_undegraded":"2024-09-16T00:07:35.114784+0000","last_fullsized":"2024-09-16T00:07:35.114784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115156+0000","last_change":"2024-09-16T00:07:32.629539+0000","last_active":"2024-09-16T00:07:35.115156+0000","last_peered":"2024-09-16T00:07:35.115156+0000","last_clean":"2024-09-16T00:07:35.115156+0000","last_became_active":"2024-09-16T00:07:32.629186+0000","last_became_peered":"2024-09-16T00:07:32.629186+0000","last_unstale":"2024-09-16T00:07:35.115156+0000","last_undegraded":"2024-09-16T00:07:35.115156+0000","last_fullsized":"2024-09-16T00:07:35.115156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:59:32.860208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114535+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:07:35.114535+0000","last_peered":"2024-09-16T00:07:35.114535+0000","last_clean":"2024-09-16T00:07:35.114535+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:07:35.114535+0000","last_undegraded":"2024-09-16T00:07:35.114535+0000","last_fullsized":"2024-09-16T00:07:35.114535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614668+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:07:34.614668+0000","last_peered":"2024-09-16T00:07:34.614668+0000","last_clean":"2024-09-16T00:07:34.614668+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:07:34.614668+0000","last_undegraded":"2024-09-16T00:07:34.614668+0000","last_fullsized":"2024-09-16T00:07:34.614668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114527+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:07:35.114527+0000","last_peered":"2024-09-16T00:07:35.114527+0000","last_clean":"2024-09-16T00:07:35.114527+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:07:35.114527+0000","last_undegraded":"2024-09-16T00:07:35.114527+0000","last_fullsized":"2024-09-16T00:07:35.114527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115138+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:07:35.115138+0000","last_peered":"2024-09-16T00:07:35.115138+0000","last_clean":"2024-09-16T00:07:35.115138+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:07:35.115138+0000","last_undegraded":"2024-09-16T00:07:35.115138+0000","last_fullsized":"2024-09-16T00:07:35.115138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615408+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:07:34.615408+0000","last_peered":"2024-09-16T00:07:34.615408+0000","last_clean":"2024-09-16T00:07:34.615408+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:07:34.615408+0000","last_undegraded":"2024-09-16T00:07:34.615408+0000","last_fullsized":"2024-09-16T00:07:34.615408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615356+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:07:34.615356+0000","last_peered":"2024-09-16T00:07:34.615356+0000","last_clean":"2024-09-16T00:07:34.615356+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:07:34.615356+0000","last_undegraded":"2024-09-16T00:07:34.615356+0000","last_fullsized":"2024-09-16T00:07:34.615356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115277+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:07:35.115277+0000","last_peered":"2024-09-16T00:07:35.115277+0000","last_clean":"2024-09-16T00:07:35.115277+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:07:35.115277+0000","last_undegraded":"2024-09-16T00:07:35.115277+0000","last_fullsized":"2024-09-16T00:07:35.115277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115634+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:07:35.115634+0000","last_peered":"2024-09-16T00:07:35.115634+0000","last_clean":"2024-09-16T00:07:35.115634+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:07:35.115634+0000","last_undegraded":"2024-09-16T00:07:35.115634+0000","last_fullsized":"2024-09-16T00:07:35.115634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614648+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:07:34.614648+0000","last_peered":"2024-09-16T00:07:34.614648+0000","last_clean":"2024-09-16T00:07:34.614648+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:07:34.614648+0000","last_undegraded":"2024-09-16T00:07:34.614648+0000","last_fullsized":"2024-09-16T00:07:34.614648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115117+0000","last_change":"2024-09-16T00:07:32.629132+0000","last_active":"2024-09-16T00:07:35.115117+0000","last_peered":"2024-09-16T00:07:35.115117+0000","last_clean":"2024-09-16T00:07:35.115117+0000","last_became_active":"2024-09-16T00:07:32.628803+0000","last_became_peered":"2024-09-16T00:07:32.628803+0000","last_unstale":"2024-09-16T00:07:35.115117+0000","last_undegraded":"2024-09-16T00:07:35.115117+0000","last_fullsized":"2024-09-16T00:07:35.115117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115230+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:07:35.115230+0000","last_peered":"2024-09-16T00:07:35.115230+0000","last_clean":"2024-09-16T00:07:35.115230+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:07:35.115230+0000","last_undegraded":"2024-09-16T00:07:35.115230+0000","last_fullsized":"2024-09-16T00:07:35.115230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115538+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:07:35.115538+0000","last_peered":"2024-09-16T00:07:35.115538+0000","last_clean":"2024-09-16T00:07:35.115538+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:07:35.115538+0000","last_undegraded":"2024-09-16T00:07:35.115538+0000","last_fullsized":"2024-09-16T00:07:35.115538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115121+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:07:35.115121+0000","last_peered":"2024-09-16T00:07:35.115121+0000","last_clean":"2024-09-16T00:07:35.115121+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:07:35.115121+0000","last_undegraded":"2024-09-16T00:07:35.115121+0000","last_fullsized":"2024-09-16T00:07:35.115121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615462+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:07:34.615462+0000","last_peered":"2024-09-16T00:07:34.615462+0000","last_clean":"2024-09-16T00:07:34.615462+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:07:34.615462+0000","last_undegraded":"2024-09-16T00:07:34.615462+0000","last_fullsized":"2024-09-16T00:07:34.615462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115766+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:07:35.115766+0000","last_peered":"2024-09-16T00:07:35.115766+0000","last_clean":"2024-09-16T00:07:35.115766+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:07:35.115766+0000","last_undegraded":"2024-09-16T00:07:35.115766+0000","last_fullsized":"2024-09-16T00:07:35.115766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114802+0000","last_change":"2024-09-16T00:07:32.626856+0000","last_active":"2024-09-16T00:07:35.114802+0000","last_peered":"2024-09-16T00:07:35.114802+0000","last_clean":"2024-09-16T00:07:35.114802+0000","last_became_active":"2024-09-16T00:07:32.626532+0000","last_became_peered":"2024-09-16T00:07:32.626532+0000","last_unstale":"2024-09-16T00:07:35.114802+0000","last_undegraded":"2024-09-16T00:07:35.114802+0000","last_fullsized":"2024-09-16T00:07:35.114802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114886+0000","last_change":"2024-09-16T00:07:33.632870+0000","last_active":"2024-09-16T00:07:35.114886+0000","last_peered":"2024-09-16T00:07:35.114886+0000","last_clean":"2024-09-16T00:07:35.114886+0000","last_became_active":"2024-09-16T00:07:33.632608+0000","last_became_peered":"2024-09-16T00:07:33.632608+0000","last_unstale":"2024-09-16T00:07:35.114886+0000","last_undegraded":"2024-09-16T00:07:35.114886+0000","last_fullsized":"2024-09-16T00:07:35.114886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.612904+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:07:34.612904+0000","last_peered":"2024-09-16T00:07:34.612904+0000","last_clean":"2024-09-16T00:07:34.612904+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:07:34.612904+0000","last_undegraded":"2024-09-16T00:07:34.612904+0000","last_fullsized":"2024-09-16T00:07:34.612904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115409+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:07:35.115409+0000","last_peered":"2024-09-16T00:07:35.115409+0000","last_clean":"2024-09-16T00:07:35.115409+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:07:35.115409+0000","last_undegraded":"2024-09-16T00:07:35.115409+0000","last_fullsized":"2024-09-16T00:07:35.115409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614717+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:07:34.614717+0000","last_peered":"2024-09-16T00:07:34.614717+0000","last_clean":"2024-09-16T00:07:34.614717+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:07:34.614717+0000","last_undegraded":"2024-09-16T00:07:34.614717+0000","last_fullsized":"2024-09-16T00:07:34.614717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115480+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:07:35.115480+0000","last_peered":"2024-09-16T00:07:35.115480+0000","last_clean":"2024-09-16T00:07:35.115480+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:07:35.115480+0000","last_undegraded":"2024-09-16T00:07:35.115480+0000","last_fullsized":"2024-09-16T00:07:35.115480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613114+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:07:34.613114+0000","last_peered":"2024-09-16T00:07:34.613114+0000","last_clean":"2024-09-16T00:07:34.613114+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:07:34.613114+0000","last_undegraded":"2024-09-16T00:07:34.613114+0000","last_fullsized":"2024-09-16T00:07:34.613114+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115888+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:07:35.115888+0000","last_peered":"2024-09-16T00:07:35.115888+0000","last_clean":"2024-09-16T00:07:35.115888+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:07:35.115888+0000","last_undegraded":"2024-09-16T00:07:35.115888+0000","last_fullsized":"2024-09-16T00:07:35.115888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115613+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:07:35.115613+0000","last_peered":"2024-09-16T00:07:35.115613+0000","last_clean":"2024-09-16T00:07:35.115613+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:07:35.115613+0000","last_undegraded":"2024-09-16T00:07:35.115613+0000","last_fullsized":"2024-09-16T00:07:35.115613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615597+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:07:34.615597+0000","last_peered":"2024-09-16T00:07:34.615597+0000","last_clean":"2024-09-16T00:07:34.615597+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:07:34.615597+0000","last_undegraded":"2024-09-16T00:07:34.615597+0000","last_fullsized":"2024-09-16T00:07:34.615597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613057+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:07:34.613057+0000","last_peered":"2024-09-16T00:07:34.613057+0000","last_clean":"2024-09-16T00:07:34.613057+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:07:34.613057+0000","last_undegraded":"2024-09-16T00:07:34.613057+0000","last_fullsized":"2024-09-16T00:07:34.613057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115577+0000","last_change":"2024-09-16T00:07:33.638312+0000","last_active":"2024-09-16T00:07:35.115577+0000","last_peered":"2024-09-16T00:07:35.115577+0000","last_clean":"2024-09-16T00:07:35.115577+0000","last_became_active":"2024-09-16T00:07:33.638049+0000","last_became_peered":"2024-09-16T00:07:33.638049+0000","last_unstale":"2024-09-16T00:07:35.115577+0000","last_undegraded":"2024-09-16T00:07:35.115577+0000","last_fullsized":"2024-09-16T00:07:35.115577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115169+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:07:35.115169+0000","last_peered":"2024-09-16T00:07:35.115169+0000","last_clean":"2024-09-16T00:07:35.115169+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:07:35.115169+0000","last_undegraded":"2024-09-16T00:07:35.115169+0000","last_fullsized":"2024-09-16T00:07:35.115169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614834+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:07:34.614834+0000","last_peered":"2024-09-16T00:07:34.614834+0000","last_clean":"2024-09-16T00:07:34.614834+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:07:34.614834+0000","last_undegraded":"2024-09-16T00:07:34.614834+0000","last_fullsized":"2024-09-16T00:07:34.614834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114892+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:07:35.114892+0000","last_peered":"2024-09-16T00:07:35.114892+0000","last_clean":"2024-09-16T00:07:35.114892+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:07:35.114892+0000","last_undegraded":"2024-09-16T00:07:35.114892+0000","last_fullsized":"2024-09-16T00:07:35.114892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115065+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:07:35.115065+0000","last_peered":"2024-09-16T00:07:35.115065+0000","last_clean":"2024-09-16T00:07:35.115065+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:07:35.115065+0000","last_undegraded":"2024-09-16T00:07:35.115065+0000","last_fullsized":"2024-09-16T00:07:35.115065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115288+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:07:35.115288+0000","last_peered":"2024-09-16T00:07:35.115288+0000","last_clean":"2024-09-16T00:07:35.115288+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:07:35.115288+0000","last_undegraded":"2024-09-16T00:07:35.115288+0000","last_fullsized":"2024-09-16T00:07:35.115288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.612974+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:07:34.612974+0000","last_peered":"2024-09-16T00:07:34.612974+0000","last_clean":"2024-09-16T00:07:34.612974+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:07:34.612974+0000","last_undegraded":"2024-09-16T00:07:34.612974+0000","last_fullsized":"2024-09-16T00:07:34.612974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114905+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:07:35.114905+0000","last_peered":"2024-09-16T00:07:35.114905+0000","last_clean":"2024-09-16T00:07:35.114905+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:07:35.114905+0000","last_undegraded":"2024-09-16T00:07:35.114905+0000","last_fullsized":"2024-09-16T00:07:35.114905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615621+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:07:34.615621+0000","last_peered":"2024-09-16T00:07:34.615621+0000","last_clean":"2024-09-16T00:07:34.615621+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:07:34.615621+0000","last_undegraded":"2024-09-16T00:07:34.615621+0000","last_fullsized":"2024-09-16T00:07:34.615621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615008+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:07:34.615008+0000","last_peered":"2024-09-16T00:07:34.615008+0000","last_clean":"2024-09-16T00:07:34.615008+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:07:34.615008+0000","last_undegraded":"2024-09-16T00:07:34.615008+0000","last_fullsized":"2024-09-16T00:07:34.615008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115398+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:07:35.115398+0000","last_peered":"2024-09-16T00:07:35.115398+0000","last_clean":"2024-09-16T00:07:35.115398+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:07:35.115398+0000","last_undegraded":"2024-09-16T00:07:35.115398+0000","last_fullsized":"2024-09-16T00:07:35.115398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615631+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:07:34.615631+0000","last_peered":"2024-09-16T00:07:34.615631+0000","last_clean":"2024-09-16T00:07:34.615631+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:07:34.615631+0000","last_undegraded":"2024-09-16T00:07:34.615631+0000","last_fullsized":"2024-09-16T00:07:34.615631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115814+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:07:35.115814+0000","last_peered":"2024-09-16T00:07:35.115814+0000","last_clean":"2024-09-16T00:07:35.115814+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:07:35.115814+0000","last_undegraded":"2024-09-16T00:07:35.115814+0000","last_fullsized":"2024-09-16T00:07:35.115814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115305+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:07:35.115305+0000","last_peered":"2024-09-16T00:07:35.115305+0000","last_clean":"2024-09-16T00:07:35.115305+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:07:35.115305+0000","last_undegraded":"2024-09-16T00:07:35.115305+0000","last_fullsized":"2024-09-16T00:07:35.115305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115644+0000","last_change":"2024-09-16T00:07:32.627236+0000","last_active":"2024-09-16T00:07:35.115644+0000","last_peered":"2024-09-16T00:07:35.115644+0000","last_clean":"2024-09-16T00:07:35.115644+0000","last_became_active":"2024-09-16T00:07:32.626743+0000","last_became_peered":"2024-09-16T00:07:32.626743+0000","last_unstale":"2024-09-16T00:07:35.115644+0000","last_undegraded":"2024-09-16T00:07:35.115644+0000","last_fullsized":"2024-09-16T00:07:35.115644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114835+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:07:35.114835+0000","last_peered":"2024-09-16T00:07:35.114835+0000","last_clean":"2024-09-16T00:07:35.114835+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:07:35.114835+0000","last_undegraded":"2024-09-16T00:07:35.114835+0000","last_fullsized":"2024-09-16T00:07:35.114835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115252+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:07:35.115252+0000","last_peered":"2024-09-16T00:07:35.115252+0000","last_clean":"2024-09-16T00:07:35.115252+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:07:35.115252+0000","last_undegraded":"2024-09-16T00:07:35.115252+0000","last_fullsized":"2024-09-16T00:07:35.115252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114804+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:07:35.114804+0000","last_peered":"2024-09-16T00:07:35.114804+0000","last_clean":"2024-09-16T00:07:35.114804+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:07:35.114804+0000","last_undegraded":"2024-09-16T00:07:35.114804+0000","last_fullsized":"2024-09-16T00:07:35.114804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114947+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:07:35.114947+0000","last_peered":"2024-09-16T00:07:35.114947+0000","last_clean":"2024-09-16T00:07:35.114947+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:07:35.114947+0000","last_undegraded":"2024-09-16T00:07:35.114947+0000","last_fullsized":"2024-09-16T00:07:35.114947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614501+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:07:34.614501+0000","last_peered":"2024-09-16T00:07:34.614501+0000","last_clean":"2024-09-16T00:07:34.614501+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:07:34.614501+0000","last_undegraded":"2024-09-16T00:07:34.614501+0000","last_fullsized":"2024-09-16T00:07:34.614501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114765+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:07:35.114765+0000","last_peered":"2024-09-16T00:07:35.114765+0000","last_clean":"2024-09-16T00:07:35.114765+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:07:35.114765+0000","last_undegraded":"2024-09-16T00:07:35.114765+0000","last_fullsized":"2024-09-16T00:07:35.114765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114640+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:07:35.114640+0000","last_peered":"2024-09-16T00:07:35.114640+0000","last_clean":"2024-09-16T00:07:35.114640+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:07:35.114640+0000","last_undegraded":"2024-09-16T00:07:35.114640+0000","last_fullsized":"2024-09-16T00:07:35.114640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115333+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:07:35.115333+0000","last_peered":"2024-09-16T00:07:35.115333+0000","last_clean":"2024-09-16T00:07:35.115333+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:07:35.115333+0000","last_undegraded":"2024-09-16T00:07:35.115333+0000","last_fullsized":"2024-09-16T00:07:35.115333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114803+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:07:35.114803+0000","last_peered":"2024-09-16T00:07:35.114803+0000","last_clean":"2024-09-16T00:07:35.114803+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:07:35.114803+0000","last_undegraded":"2024-09-16T00:07:35.114803+0000","last_fullsized":"2024-09-16T00:07:35.114803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115800+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:07:35.115800+0000","last_peered":"2024-09-16T00:07:35.115800+0000","last_clean":"2024-09-16T00:07:35.115800+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:07:35.115800+0000","last_undegraded":"2024-09-16T00:07:35.115800+0000","last_fullsized":"2024-09-16T00:07:35.115800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114993+0000","last_change":"2024-09-16T00:07:33.637979+0000","last_active":"2024-09-16T00:07:35.114993+0000","last_peered":"2024-09-16T00:07:35.114993+0000","last_clean":"2024-09-16T00:07:35.114993+0000","last_became_active":"2024-09-16T00:07:33.637713+0000","last_became_peered":"2024-09-16T00:07:33.637713+0000","last_unstale":"2024-09-16T00:07:35.114993+0000","last_undegraded":"2024-09-16T00:07:35.114993+0000","last_fullsized":"2024-09-16T00:07:35.114993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615387+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:07:34.615387+0000","last_peered":"2024-09-16T00:07:34.615387+0000","last_clean":"2024-09-16T00:07:34.615387+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:07:34.615387+0000","last_undegraded":"2024-09-16T00:07:34.615387+0000","last_fullsized":"2024-09-16T00:07:34.615387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115228+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:07:35.115228+0000","last_peered":"2024-09-16T00:07:35.115228+0000","last_clean":"2024-09-16T00:07:35.115228+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:07:35.115228+0000","last_undegraded":"2024-09-16T00:07:35.115228+0000","last_fullsized":"2024-09-16T00:07:35.115228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615266+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:07:34.615266+0000","last_peered":"2024-09-16T00:07:34.615266+0000","last_clean":"2024-09-16T00:07:34.615266+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:07:34.615266+0000","last_undegraded":"2024-09-16T00:07:34.615266+0000","last_fullsized":"2024-09-16T00:07:34.615266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115696+0000","last_change":"2024-09-16T00:07:32.632481+0000","last_active":"2024-09-16T00:07:35.115696+0000","last_peered":"2024-09-16T00:07:35.115696+0000","last_clean":"2024-09-16T00:07:35.115696+0000","last_became_active":"2024-09-16T00:07:32.632152+0000","last_became_peered":"2024-09-16T00:07:32.632152+0000","last_unstale":"2024-09-16T00:07:35.115696+0000","last_undegraded":"2024-09-16T00:07:35.115696+0000","last_fullsized":"2024-09-16T00:07:35.115696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114783+0000","last_change":"2024-09-16T00:07:33.629777+0000","last_active":"2024-09-16T00:07:35.114783+0000","last_peered":"2024-09-16T00:07:35.114783+0000","last_clean":"2024-09-16T00:07:35.114783+0000","last_became_active":"2024-09-16T00:07:33.627917+0000","last_became_peered":"2024-09-16T00:07:33.627917+0000","last_unstale":"2024-09-16T00:07:35.114783+0000","last_undegraded":"2024-09-16T00:07:35.114783+0000","last_fullsized":"2024-09-16T00:07:35.114783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:38.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114933+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:07:35.114933+0000","last_peered":"2024-09-16T00:07:35.114933+0000","last_clean":"2024-09-16T00:07:35.114933+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:07:35.114933+0000","last_undegraded":"2024-09-16T00:07:35.114933+0000","last_fullsized":"2024-09-16T00:07:35.114933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114899+0000","last_change":"2024-09-16T00:07:33.630746+0000","last_active":"2024-09-16T00:07:35.114899+0000","last_peered":"2024-09-16T00:07:35.114899+0000","last_clean":"2024-09-16T00:07:35.114899+0000","last_became_active":"2024-09-16T00:07:33.630480+0000","last_became_peered":"2024-09-16T00:07:33.630480+0000","last_unstale":"2024-09-16T00:07:35.114899+0000","last_undegraded":"2024-09-16T00:07:35.114899+0000","last_fullsized":"2024-09-16T00:07:35.114899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:50.931998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115042+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:07:35.115042+0000","last_peered":"2024-09-16T00:07:35.115042+0000","last_clean":"2024-09-16T00:07:35.115042+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:07:35.115042+0000","last_undegraded":"2024-09-16T00:07:35.115042+0000","last_fullsized":"2024-09-16T00:07:35.115042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":668364,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":668364,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:07:41.917 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:07:42.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.027+0000 7f2b05ca5640 1 Processor -- start 2024-09-16T00:07:42.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.027+0000 7f2b05ca5640 1 -- start start 2024-09-16T00:07:42.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.028+0000 7f2b05ca5640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b00151810 0x7f2b00073b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.028+0000 7f2b05ca5640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2b000afbf0 0x7f2b00157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.028+0000 7f2b05ca5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b0015ab60 con 0x7f2b00158310 2024-09-16T00:07:42.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.028+0000 7f2b05ca5640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f2b00157940 con 0x7f2b000afbf0 2024-09-16T00:07:42.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.028+0000 7f2b05ca5640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2b001575f0 con 0x7f2b00151810 2024-09-16T00:07:42.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.028+0000 7f2afffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2b00158310 0x7f2b000c3910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45144/0 (socket says 172.21.15.139:45144) 2024-09-16T00:07:42.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.028+0000 7f2afffff640 1 -- 172.21.15.139:0/412110301 learned_addr learned my addr 172.21.15.139:0/412110301 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:42.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.029+0000 7f2afeffd640 1 --2- 172.21.15.139:0/412110301 >> v2:172.21.15.164:3300/0 conn(0x7f2b000afbf0 0x7f2b00157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.029+0000 7f2aff7fe640 1 --2- 172.21.15.139:0/412110301 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b00151810 0x7f2b00073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.029+0000 7f2afe7fc640 1 -- 172.21.15.139:0/412110301 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3668318411 0 0) 0x7f2b0015ab60 con 0x7f2b00158310 2024-09-16T00:07:42.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.029+0000 7f2afe7fc640 1 -- 172.21.15.139:0/412110301 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ae8003520 con 0x7f2b00158310 2024-09-16T00:07:42.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.029+0000 7f2afeffd640 1 -- 172.21.15.139:0/412110301 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b00151810 msgr2=0x7f2b00073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:42.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.029+0000 7f2afe7fc640 1 -- 172.21.15.139:0/412110301 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2320622460 0 0) 0x7f2ae8003520 con 0x7f2b00158310 2024-09-16T00:07:42.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.029+0000 7f2afeffd640 1 --2- 172.21.15.139:0/412110301 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b00151810 0x7f2b00073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.029+0000 7f2afeffd640 1 -- 172.21.15.139:0/412110301 >> v1:172.21.15.139:6789/0 conn(0x7f2b00158310 legacy=0x7f2b000c3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.029+0000 7f2afeffd640 1 -- 172.21.15.139:0/412110301 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b000c7ed0 con 0x7f2b000afbf0 2024-09-16T00:07:42.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.029+0000 7f2aff7fe640 1 --2- 172.21.15.139:0/412110301 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b00151810 0x7f2b00073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:42.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.030+0000 7f2afeffd640 1 --2- 172.21.15.139:0/412110301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b000afbf0 0x7f2b00157dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2af40097a0 tx=0x7f2af402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2baea6147f792f88 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:42.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.030+0000 7f2afe7fc640 1 -- 172.21.15.139:0/412110301 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4037070 con 0x7f2b000afbf0 2024-09-16T00:07:42.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.030+0000 7f2afe7fc640 1 -- 172.21.15.139:0/412110301 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af402ac30 con 0x7f2b000afbf0 2024-09-16T00:07:42.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.030+0000 7f2afe7fc640 1 -- 172.21.15.139:0/412110301 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af80019f0 con 0x7f2b000afbf0 2024-09-16T00:07:42.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.030+0000 7f2b05ca5640 1 -- 172.21.15.139:0/412110301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b000afbf0 msgr2=0x7f2b00157dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.030+0000 7f2b05ca5640 1 --2- 172.21.15.139:0/412110301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b000afbf0 0x7f2b00157dd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2af40097a0 tx=0x7f2af402a1a0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.030+0000 7f2b05ca5640 1 -- 172.21.15.139:0/412110301 shutdown_connections 2024-09-16T00:07:42.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.030+0000 7f2b05ca5640 1 --2- 172.21.15.139:0/412110301 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b000afbf0 0x7f2b00157dd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.030+0000 7f2b05ca5640 1 --2- 172.21.15.139:0/412110301 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b00151810 0x7f2b00073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.030+0000 7f2b05ca5640 1 -- 172.21.15.139:0/412110301 >> 172.21.15.139:0/412110301 conn(0x7f2b00070440 msgr2=0x7f2b00070820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:42.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.030+0000 7f2b05ca5640 1 -- 172.21.15.139:0/412110301 shutdown_connections 2024-09-16T00:07:42.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.030+0000 7f2b05ca5640 1 -- 172.21.15.139:0/412110301 wait complete. 2024-09-16T00:07:42.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.032+0000 7f2b05ca5640 1 Processor -- start 2024-09-16T00:07:42.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.032+0000 7f2b05ca5640 1 -- start start 2024-09-16T00:07:42.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2b05ca5640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b000afbf0 0x7f2b0013e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2b05ca5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b00151810 0x7f2b000c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2b05ca5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b00158310 0x7f2b000c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2b05ca5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2b000c8770 con 0x7f2b00158310 2024-09-16T00:07:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2b05ca5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2b000c8490 con 0x7f2b00151810 2024-09-16T00:07:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2b05ca5640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2b000c7b90 con 0x7f2b000afbf0 2024-09-16T00:07:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2afffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b00158310 0x7f2b000c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2afffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b00158310 0x7f2b000c6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40324/0 (socket says 172.21.15.139:40324) 2024-09-16T00:07:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2afffff640 1 -- 172.21.15.139:0/3188112009 learned_addr learned my addr 172.21.15.139:0/3188112009 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2afeffd640 1 --2- 172.21.15.139:0/3188112009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b00151810 0x7f2b000c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2aff7fe640 1 --2- 172.21.15.139:0/3188112009 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b000afbf0 0x7f2b0013e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2afffff640 1 -- 172.21.15.139:0/3188112009 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b000afbf0 msgr2=0x7f2b0013e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2afffff640 1 --2- 172.21.15.139:0/3188112009 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b000afbf0 0x7f2b0013e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2afffff640 1 -- 172.21.15.139:0/3188112009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b00151810 msgr2=0x7f2b000c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2afffff640 1 --2- 172.21.15.139:0/3188112009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b00151810 0x7f2b000c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2afffff640 1 -- 172.21.15.139:0/3188112009 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b00159e30 con 0x7f2b00158310 2024-09-16T00:07:42.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2aff7fe640 1 --2- 172.21.15.139:0/3188112009 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b000afbf0 0x7f2b0013e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:42.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2afffff640 1 --2- 172.21.15.139:0/3188112009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b00158310 0x7f2b000c6120 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2af800cc30 tx=0x7f2af800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:42.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2ae7fff640 1 -- 172.21.15.139:0/3188112009 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8017070 con 0x7f2b00158310 2024-09-16T00:07:42.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2ae7fff640 1 -- 172.21.15.139:0/3188112009 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af800ad40 con 0x7f2b00158310 2024-09-16T00:07:42.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2b05ca5640 1 -- 172.21.15.139:0/3188112009 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0015a730 con 0x7f2b00158310 2024-09-16T00:07:42.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2ae7fff640 1 -- 172.21.15.139:0/3188112009 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8012640 con 0x7f2b00158310 2024-09-16T00:07:42.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.033+0000 7f2b05ca5640 1 -- 172.21.15.139:0/3188112009 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b000c4bf0 con 0x7f2b00158310 2024-09-16T00:07:42.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.035+0000 7f2b05ca5640 1 -- 172.21.15.139:0/3188112009 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b000c2240 con 0x7f2b00158310 2024-09-16T00:07:42.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.035+0000 7f2ae7fff640 1 -- 172.21.15.139:0/3188112009 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2af801a070 con 0x7f2b00158310 2024-09-16T00:07:42.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.035+0000 7f2ae7fff640 1 --2- 172.21.15.139:0/3188112009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2aec06c2b0 0x7f2aec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.035+0000 7f2ae7fff640 1 -- 172.21.15.139:0/3188112009 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2af808b400 con 0x7f2b00158310 2024-09-16T00:07:42.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.035+0000 7f2aff7fe640 1 --2- 172.21.15.139:0/3188112009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2aec06c2b0 0x7f2aec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.036+0000 7f2aff7fe640 1 --2- 172.21.15.139:0/3188112009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2aec06c2b0 0x7f2aec06e770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009ba0 tx=0x7f2af0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:42.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.040+0000 7f2ae7fff640 1 -- 172.21.15.139:0/3188112009 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2af80595b0 con 0x7f2b00158310 2024-09-16T00:07:42.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.204+0000 7f2b05ca5640 1 -- 172.21.15.139:0/3188112009 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b000caa90 con 0x7f2aec06c2b0 2024-09-16T00:07:42.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.210+0000 7f2ae7fff640 1 -- 172.21.15.139:0/3188112009 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251985 (secure 0 0 0) 0x7f2b000caa90 con 0x7f2aec06c2b0 2024-09-16T00:07:42.213 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:42.214 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:07:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.213+0000 7f2b05ca5640 1 -- 172.21.15.139:0/3188112009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2aec06c2b0 msgr2=0x7f2aec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.213+0000 7f2b05ca5640 1 --2- 172.21.15.139:0/3188112009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2aec06c2b0 0x7f2aec06e770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2af0009ba0 tx=0x7f2af0008040 comp rx=0 tx=0).stop 2024-09-16T00:07:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.213+0000 7f2b05ca5640 1 -- 172.21.15.139:0/3188112009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b00158310 msgr2=0x7f2b000c6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.213+0000 7f2b05ca5640 1 --2- 172.21.15.139:0/3188112009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b00158310 0x7f2b000c6120 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2af800cc30 tx=0x7f2af800a430 comp rx=0 tx=0).stop 2024-09-16T00:07:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.213+0000 7f2b05ca5640 1 -- 172.21.15.139:0/3188112009 shutdown_connections 2024-09-16T00:07:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.213+0000 7f2b05ca5640 1 --2- 172.21.15.139:0/3188112009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2aec06c2b0 0x7f2aec06e770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.213+0000 7f2b05ca5640 1 --2- 172.21.15.139:0/3188112009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b00158310 0x7f2b000c6120 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.213+0000 7f2b05ca5640 1 --2- 172.21.15.139:0/3188112009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2b00151810 0x7f2b000c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.213+0000 7f2b05ca5640 1 --2- 172.21.15.139:0/3188112009 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2b000afbf0 0x7f2b0013e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.213+0000 7f2b05ca5640 1 -- 172.21.15.139:0/3188112009 >> 172.21.15.139:0/3188112009 conn(0x7f2b00070440 msgr2=0x7f2b0013fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:42.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.213+0000 7f2b05ca5640 1 -- 172.21.15.139:0/3188112009 shutdown_connections 2024-09-16T00:07:42.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.213+0000 7f2b05ca5640 1 -- 172.21.15.139:0/3188112009 wait complete. 2024-09-16T00:07:42.225 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-16T00:07:42.027194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591144,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":186,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218112,"kb_used_data":1888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756608,"statfs":{"total":773094113280,"available":772870766592,"internally_reserved":0,"allocated":1933312,"data_stored":1807114,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":776,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.416505"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115528+0000","last_change":"2024-09-16T00:07:33.635096+0000","last_active":"2024-09-16T00:07:35.115528+0000","last_peered":"2024-09-16T00:07:35.115528+0000","last_clean":"2024-09-16T00:07:35.115528+0000","last_became_active":"2024-09-16T00:07:33.634750+0000","last_became_peered":"2024-09-16T00:07:33.634750+0000","last_unstale":"2024-09-16T00:07:35.115528+0000","last_undegraded":"2024-09-16T00:07:35.115528+0000","last_fullsized":"2024-09-16T00:07:35.115528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115558+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:07:35.115558+0000","last_peered":"2024-09-16T00:07:35.115558+0000","last_clean":"2024-09-16T00:07:35.115558+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:07:35.115558+0000","last_undegraded":"2024-09-16T00:07:35.115558+0000","last_fullsized":"2024-09-16T00:07:35.115558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115392+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:07:35.115392+0000","last_peered":"2024-09-16T00:07:35.115392+0000","last_clean":"2024-09-16T00:07:35.115392+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:07:35.115392+0000","last_undegraded":"2024-09-16T00:07:35.115392+0000","last_fullsized":"2024-09-16T00:07:35.115392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615167+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:07:34.615167+0000","last_peered":"2024-09-16T00:07:34.615167+0000","last_clean":"2024-09-16T00:07:34.615167+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:07:34.615167+0000","last_undegraded":"2024-09-16T00:07:34.615167+0000","last_fullsized":"2024-09-16T00:07:34.615167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115724+0000","last_change":"2024-09-16T00:07:33.630029+0000","last_active":"2024-09-16T00:07:35.115724+0000","last_peered":"2024-09-16T00:07:35.115724+0000","last_clean":"2024-09-16T00:07:35.115724+0000","last_became_active":"2024-09-16T00:07:33.629712+0000","last_became_peered":"2024-09-16T00:07:33.629712+0000","last_unstale":"2024-09-16T00:07:35.115724+0000","last_undegraded":"2024-09-16T00:07:35.115724+0000","last_fullsized":"2024-09-16T00:07:35.115724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614973+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:07:34.614973+0000","last_peered":"2024-09-16T00:07:34.614973+0000","last_clean":"2024-09-16T00:07:34.614973+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:07:34.614973+0000","last_undegraded":"2024-09-16T00:07:34.614973+0000","last_fullsized":"2024-09-16T00:07:34.614973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115522+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:07:35.115522+0000","last_peered":"2024-09-16T00:07:35.115522+0000","last_clean":"2024-09-16T00:07:35.115522+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:07:35.115522+0000","last_undegraded":"2024-09-16T00:07:35.115522+0000","last_fullsized":"2024-09-16T00:07:35.115522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114492+0000","last_change":"2024-09-16T00:07:33.624708+0000","last_active":"2024-09-16T00:07:35.114492+0000","last_peered":"2024-09-16T00:07:35.114492+0000","last_clean":"2024-09-16T00:07:35.114492+0000","last_became_active":"2024-09-16T00:07:33.624458+0000","last_became_peered":"2024-09-16T00:07:33.624458+0000","last_unstale":"2024-09-16T00:07:35.114492+0000","last_undegraded":"2024-09-16T00:07:35.114492+0000","last_fullsized":"2024-09-16T00:07:35.114492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:48.052488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115751+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:07:35.115751+0000","last_peered":"2024-09-16T00:07:35.115751+0000","last_clean":"2024-09-16T00:07:35.115751+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:07:35.115751+0000","last_undegraded":"2024-09-16T00:07:35.115751+0000","last_fullsized":"2024-09-16T00:07:35.115751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115614+0000","last_change":"2024-09-16T00:07:33.635973+0000","last_active":"2024-09-16T00:07:35.115614+0000","last_peered":"2024-09-16T00:07:35.115614+0000","last_clean":"2024-09-16T00:07:35.115614+0000","last_became_active":"2024-09-16T00:07:33.635486+0000","last_became_peered":"2024-09-16T00:07:33.635486+0000","last_unstale":"2024-09-16T00:07:35.115614+0000","last_undegraded":"2024-09-16T00:07:35.115614+0000","last_fullsized":"2024-09-16T00:07:35.115614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115027+0000","last_change":"2024-09-16T00:07:33.636971+0000","last_active":"2024-09-16T00:07:35.115027+0000","last_peered":"2024-09-16T00:07:35.115027+0000","last_clean":"2024-09-16T00:07:35.115027+0000","last_became_active":"2024-09-16T00:07:33.635760+0000","last_became_peered":"2024-09-16T00:07:33.635760+0000","last_unstale":"2024-09-16T00:07:35.115027+0000","last_undegraded":"2024-09-16T00:07:35.115027+0000","last_fullsized":"2024-09-16T00:07:35.115027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614631+0000","last_change":"2024-09-16T00:07:33.632261+0000","last_active":"2024-09-16T00:07:34.614631+0000","last_peered":"2024-09-16T00:07:34.614631+0000","last_clean":"2024-09-16T00:07:34.614631+0000","last_became_active":"2024-09-16T00:07:33.631944+0000","last_became_peered":"2024-09-16T00:07:33.631944+0000","last_unstale":"2024-09-16T00:07:34.614631+0000","last_undegraded":"2024-09-16T00:07:34.614631+0000","last_fullsized":"2024-09-16T00:07:34.614631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:35.052417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114641+0000","last_change":"2024-09-16T00:07:33.628273+0000","last_active":"2024-09-16T00:07:35.114641+0000","last_peered":"2024-09-16T00:07:35.114641+0000","last_clean":"2024-09-16T00:07:35.114641+0000","last_became_active":"2024-09-16T00:07:33.627960+0000","last_became_peered":"2024-09-16T00:07:33.627960+0000","last_unstale":"2024-09-16T00:07:35.114641+0000","last_undegraded":"2024-09-16T00:07:35.114641+0000","last_fullsized":"2024-09-16T00:07:35.114641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:36.451688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114782+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:07:35.114782+0000","last_peered":"2024-09-16T00:07:35.114782+0000","last_clean":"2024-09-16T00:07:35.114782+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:07:35.114782+0000","last_undegraded":"2024-09-16T00:07:35.114782+0000","last_fullsized":"2024-09-16T00:07:35.114782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114664+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:07:35.114664+0000","last_peered":"2024-09-16T00:07:35.114664+0000","last_clean":"2024-09-16T00:07:35.114664+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:07:35.114664+0000","last_undegraded":"2024-09-16T00:07:35.114664+0000","last_fullsized":"2024-09-16T00:07:35.114664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614471+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:07:34.614471+0000","last_peered":"2024-09-16T00:07:34.614471+0000","last_clean":"2024-09-16T00:07:34.614471+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:07:34.614471+0000","last_undegraded":"2024-09-16T00:07:34.614471+0000","last_fullsized":"2024-09-16T00:07:34.614471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615697+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:07:34.615697+0000","last_peered":"2024-09-16T00:07:34.615697+0000","last_clean":"2024-09-16T00:07:34.615697+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:07:34.615697+0000","last_undegraded":"2024-09-16T00:07:34.615697+0000","last_fullsized":"2024-09-16T00:07:34.615697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613100+0000","last_change":"2024-09-16T00:07:33.630797+0000","last_active":"2024-09-16T00:07:34.613100+0000","last_peered":"2024-09-16T00:07:34.613100+0000","last_clean":"2024-09-16T00:07:34.613100+0000","last_became_active":"2024-09-16T00:07:33.630491+0000","last_became_peered":"2024-09-16T00:07:33.630491+0000","last_unstale":"2024-09-16T00:07:34.613100+0000","last_undegraded":"2024-09-16T00:07:34.613100+0000","last_fullsized":"2024-09-16T00:07:34.613100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:20.097626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115840+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:07:35.115840+0000","last_peered":"2024-09-16T00:07:35.115840+0000","last_clean":"2024-09-16T00:07:35.115840+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:07:35.115840+0000","last_undegraded":"2024-09-16T00:07:35.115840+0000","last_fullsized":"2024-09-16T00:07:35.115840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114960+0000","last_change":"2024-09-16T00:07:33.638554+0000","last_active":"2024-09-16T00:07:35.114960+0000","last_peered":"2024-09-16T00:07:35.114960+0000","last_clean":"2024-09-16T00:07:35.114960+0000","last_became_active":"2024-09-16T00:07:33.638263+0000","last_became_peered":"2024-09-16T00:07:33.638263+0000","last_unstale":"2024-09-16T00:07:35.114960+0000","last_undegraded":"2024-09-16T00:07:35.114960+0000","last_fullsized":"2024-09-16T00:07:35.114960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115286+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:07:35.115286+0000","last_peered":"2024-09-16T00:07:35.115286+0000","last_clean":"2024-09-16T00:07:35.115286+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:07:35.115286+0000","last_undegraded":"2024-09-16T00:07:35.115286+0000","last_fullsized":"2024-09-16T00:07:35.115286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615450+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:07:34.615450+0000","last_peered":"2024-09-16T00:07:34.615450+0000","last_clean":"2024-09-16T00:07:34.615450+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:07:34.615450+0000","last_undegraded":"2024-09-16T00:07:34.615450+0000","last_fullsized":"2024-09-16T00:07:34.615450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115015+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:07:35.115015+0000","last_peered":"2024-09-16T00:07:35.115015+0000","last_clean":"2024-09-16T00:07:35.115015+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:07:35.115015+0000","last_undegraded":"2024-09-16T00:07:35.115015+0000","last_fullsized":"2024-09-16T00:07:35.115015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614616+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:07:34.614616+0000","last_peered":"2024-09-16T00:07:34.614616+0000","last_clean":"2024-09-16T00:07:34.614616+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:07:34.614616+0000","last_undegraded":"2024-09-16T00:07:34.614616+0000","last_fullsized":"2024-09-16T00:07:34.614616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114694+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:07:35.114694+0000","last_peered":"2024-09-16T00:07:35.114694+0000","last_clean":"2024-09-16T00:07:35.114694+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:07:35.114694+0000","last_undegraded":"2024-09-16T00:07:35.114694+0000","last_fullsized":"2024-09-16T00:07:35.114694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115306+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:07:35.115306+0000","last_peered":"2024-09-16T00:07:35.115306+0000","last_clean":"2024-09-16T00:07:35.115306+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:07:35.115306+0000","last_undegraded":"2024-09-16T00:07:35.115306+0000","last_fullsized":"2024-09-16T00:07:35.115306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115543+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:07:35.115543+0000","last_peered":"2024-09-16T00:07:35.115543+0000","last_clean":"2024-09-16T00:07:35.115543+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:07:35.115543+0000","last_undegraded":"2024-09-16T00:07:35.115543+0000","last_fullsized":"2024-09-16T00:07:35.115543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114985+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:07:35.114985+0000","last_peered":"2024-09-16T00:07:35.114985+0000","last_clean":"2024-09-16T00:07:35.114985+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:07:35.114985+0000","last_undegraded":"2024-09-16T00:07:35.114985+0000","last_fullsized":"2024-09-16T00:07:35.114985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115448+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:07:35.115448+0000","last_peered":"2024-09-16T00:07:35.115448+0000","last_clean":"2024-09-16T00:07:35.115448+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:07:35.115448+0000","last_undegraded":"2024-09-16T00:07:35.115448+0000","last_fullsized":"2024-09-16T00:07:35.115448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615338+0000","last_change":"2024-09-16T00:07:33.627789+0000","last_active":"2024-09-16T00:07:34.615338+0000","last_peered":"2024-09-16T00:07:34.615338+0000","last_clean":"2024-09-16T00:07:34.615338+0000","last_became_active":"2024-09-16T00:07:33.627500+0000","last_became_peered":"2024-09-16T00:07:33.627500+0000","last_unstale":"2024-09-16T00:07:34.615338+0000","last_undegraded":"2024-09-16T00:07:34.615338+0000","last_fullsized":"2024-09-16T00:07:34.615338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613050+0000","last_change":"2024-09-16T00:07:33.621107+0000","last_active":"2024-09-16T00:07:34.613050+0000","last_peered":"2024-09-16T00:07:34.613050+0000","last_clean":"2024-09-16T00:07:34.613050+0000","last_became_active":"2024-09-16T00:07:33.620858+0000","last_became_peered":"2024-09-16T00:07:33.620858+0000","last_unstale":"2024-09-16T00:07:34.613050+0000","last_undegraded":"2024-09-16T00:07:34.613050+0000","last_fullsized":"2024-09-16T00:07:34.613050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:30.176079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115031+0000","last_change":"2024-09-16T00:07:33.633578+0000","last_active":"2024-09-16T00:07:35.115031+0000","last_peered":"2024-09-16T00:07:35.115031+0000","last_clean":"2024-09-16T00:07:35.115031+0000","last_became_active":"2024-09-16T00:07:33.633310+0000","last_became_peered":"2024-09-16T00:07:33.633310+0000","last_unstale":"2024-09-16T00:07:35.115031+0000","last_undegraded":"2024-09-16T00:07:35.115031+0000","last_fullsized":"2024-09-16T00:07:35.115031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614556+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:07:34.614556+0000","last_peered":"2024-09-16T00:07:34.614556+0000","last_clean":"2024-09-16T00:07:34.614556+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:07:34.614556+0000","last_undegraded":"2024-09-16T00:07:34.614556+0000","last_fullsized":"2024-09-16T00:07:34.614556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.059131+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:07:36.059131+0000","last_peered":"2024-09-16T00:07:36.059131+0000","last_clean":"2024-09-16T00:07:36.059131+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:07:36.059131+0000","last_undegraded":"2024-09-16T00:07:36.059131+0000","last_fullsized":"2024-09-16T00:07:36.059131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115509+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:07:35.115509+0000","last_peered":"2024-09-16T00:07:35.115509+0000","last_clean":"2024-09-16T00:07:35.115509+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:07:35.115509+0000","last_undegraded":"2024-09-16T00:07:35.115509+0000","last_fullsized":"2024-09-16T00:07:35.115509+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114784+0000","last_change":"2024-09-16T00:07:32.626800+0000","last_active":"2024-09-16T00:07:35.114784+0000","last_peered":"2024-09-16T00:07:35.114784+0000","last_clean":"2024-09-16T00:07:35.114784+0000","last_became_active":"2024-09-16T00:07:32.626511+0000","last_became_peered":"2024-09-16T00:07:32.626511+0000","last_unstale":"2024-09-16T00:07:35.114784+0000","last_undegraded":"2024-09-16T00:07:35.114784+0000","last_fullsized":"2024-09-16T00:07:35.114784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115156+0000","last_change":"2024-09-16T00:07:32.629539+0000","last_active":"2024-09-16T00:07:35.115156+0000","last_peered":"2024-09-16T00:07:35.115156+0000","last_clean":"2024-09-16T00:07:35.115156+0000","last_became_active":"2024-09-16T00:07:32.629186+0000","last_became_peered":"2024-09-16T00:07:32.629186+0000","last_unstale":"2024-09-16T00:07:35.115156+0000","last_undegraded":"2024-09-16T00:07:35.115156+0000","last_fullsized":"2024-09-16T00:07:35.115156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:59:32.860208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114535+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:07:35.114535+0000","last_peered":"2024-09-16T00:07:35.114535+0000","last_clean":"2024-09-16T00:07:35.114535+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:07:35.114535+0000","last_undegraded":"2024-09-16T00:07:35.114535+0000","last_fullsized":"2024-09-16T00:07:35.114535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614668+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:07:34.614668+0000","last_peered":"2024-09-16T00:07:34.614668+0000","last_clean":"2024-09-16T00:07:34.614668+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:07:34.614668+0000","last_undegraded":"2024-09-16T00:07:34.614668+0000","last_fullsized":"2024-09-16T00:07:34.614668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114527+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:07:35.114527+0000","last_peered":"2024-09-16T00:07:35.114527+0000","last_clean":"2024-09-16T00:07:35.114527+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:07:35.114527+0000","last_undegraded":"2024-09-16T00:07:35.114527+0000","last_fullsized":"2024-09-16T00:07:35.114527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115138+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:07:35.115138+0000","last_peered":"2024-09-16T00:07:35.115138+0000","last_clean":"2024-09-16T00:07:35.115138+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:07:35.115138+0000","last_undegraded":"2024-09-16T00:07:35.115138+0000","last_fullsized":"2024-09-16T00:07:35.115138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615408+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:07:34.615408+0000","last_peered":"2024-09-16T00:07:34.615408+0000","last_clean":"2024-09-16T00:07:34.615408+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:07:34.615408+0000","last_undegraded":"2024-09-16T00:07:34.615408+0000","last_fullsized":"2024-09-16T00:07:34.615408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615356+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:07:34.615356+0000","last_peered":"2024-09-16T00:07:34.615356+0000","last_clean":"2024-09-16T00:07:34.615356+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:07:34.615356+0000","last_undegraded":"2024-09-16T00:07:34.615356+0000","last_fullsized":"2024-09-16T00:07:34.615356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115277+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:07:35.115277+0000","last_peered":"2024-09-16T00:07:35.115277+0000","last_clean":"2024-09-16T00:07:35.115277+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:07:35.115277+0000","last_undegraded":"2024-09-16T00:07:35.115277+0000","last_fullsized":"2024-09-16T00:07:35.115277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.619745+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:07:38.619745+0000","last_peered":"2024-09-16T00:07:38.619745+0000","last_clean":"2024-09-16T00:07:38.619745+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:07:38.619745+0000","last_undegraded":"2024-09-16T00:07:38.619745+0000","last_fullsized":"2024-09-16T00:07:38.619745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614648+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:07:34.614648+0000","last_peered":"2024-09-16T00:07:34.614648+0000","last_clean":"2024-09-16T00:07:34.614648+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:07:34.614648+0000","last_undegraded":"2024-09-16T00:07:34.614648+0000","last_fullsized":"2024-09-16T00:07:34.614648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115117+0000","last_change":"2024-09-16T00:07:32.629132+0000","last_active":"2024-09-16T00:07:35.115117+0000","last_peered":"2024-09-16T00:07:35.115117+0000","last_clean":"2024-09-16T00:07:35.115117+0000","last_became_active":"2024-09-16T00:07:32.628803+0000","last_became_peered":"2024-09-16T00:07:32.628803+0000","last_unstale":"2024-09-16T00:07:35.115117+0000","last_undegraded":"2024-09-16T00:07:35.115117+0000","last_fullsized":"2024-09-16T00:07:35.115117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115230+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:07:35.115230+0000","last_peered":"2024-09-16T00:07:35.115230+0000","last_clean":"2024-09-16T00:07:35.115230+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:07:35.115230+0000","last_undegraded":"2024-09-16T00:07:35.115230+0000","last_fullsized":"2024-09-16T00:07:35.115230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620127+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:07:38.620127+0000","last_peered":"2024-09-16T00:07:38.620127+0000","last_clean":"2024-09-16T00:07:38.620127+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:07:38.620127+0000","last_undegraded":"2024-09-16T00:07:38.620127+0000","last_fullsized":"2024-09-16T00:07:38.620127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115121+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:07:35.115121+0000","last_peered":"2024-09-16T00:07:35.115121+0000","last_clean":"2024-09-16T00:07:35.115121+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:07:35.115121+0000","last_undegraded":"2024-09-16T00:07:35.115121+0000","last_fullsized":"2024-09-16T00:07:35.115121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615462+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:07:34.615462+0000","last_peered":"2024-09-16T00:07:34.615462+0000","last_clean":"2024-09-16T00:07:34.615462+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:07:34.615462+0000","last_undegraded":"2024-09-16T00:07:34.615462+0000","last_fullsized":"2024-09-16T00:07:34.615462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115766+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:07:35.115766+0000","last_peered":"2024-09-16T00:07:35.115766+0000","last_clean":"2024-09-16T00:07:35.115766+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:07:35.115766+0000","last_undegraded":"2024-09-16T00:07:35.115766+0000","last_fullsized":"2024-09-16T00:07:35.115766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114802+0000","last_change":"2024-09-16T00:07:32.626856+0000","last_active":"2024-09-16T00:07:35.114802+0000","last_peered":"2024-09-16T00:07:35.114802+0000","last_clean":"2024-09-16T00:07:35.114802+0000","last_became_active":"2024-09-16T00:07:32.626532+0000","last_became_peered":"2024-09-16T00:07:32.626532+0000","last_unstale":"2024-09-16T00:07:35.114802+0000","last_undegraded":"2024-09-16T00:07:35.114802+0000","last_fullsized":"2024-09-16T00:07:35.114802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114886+0000","last_change":"2024-09-16T00:07:33.632870+0000","last_active":"2024-09-16T00:07:35.114886+0000","last_peered":"2024-09-16T00:07:35.114886+0000","last_clean":"2024-09-16T00:07:35.114886+0000","last_became_active":"2024-09-16T00:07:33.632608+0000","last_became_peered":"2024-09-16T00:07:33.632608+0000","last_unstale":"2024-09-16T00:07:35.114886+0000","last_undegraded":"2024-09-16T00:07:35.114886+0000","last_fullsized":"2024-09-16T00:07:35.114886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.621550+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:07:38.621550+0000","last_peered":"2024-09-16T00:07:38.621550+0000","last_clean":"2024-09-16T00:07:38.621550+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:07:38.621550+0000","last_undegraded":"2024-09-16T00:07:38.621550+0000","last_fullsized":"2024-09-16T00:07:38.621550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115409+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:07:35.115409+0000","last_peered":"2024-09-16T00:07:35.115409+0000","last_clean":"2024-09-16T00:07:35.115409+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:07:35.115409+0000","last_undegraded":"2024-09-16T00:07:35.115409+0000","last_fullsized":"2024-09-16T00:07:35.115409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614717+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:07:34.614717+0000","last_peered":"2024-09-16T00:07:34.614717+0000","last_clean":"2024-09-16T00:07:34.614717+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:07:34.614717+0000","last_undegraded":"2024-09-16T00:07:34.614717+0000","last_fullsized":"2024-09-16T00:07:34.614717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115480+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:07:35.115480+0000","last_peered":"2024-09-16T00:07:35.115480+0000","last_clean":"2024-09-16T00:07:35.115480+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:07:35.115480+0000","last_undegraded":"2024-09-16T00:07:35.115480+0000","last_fullsized":"2024-09-16T00:07:35.115480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620942+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:07:38.620942+0000","last_peered":"2024-09-16T00:07:38.620942+0000","last_clean":"2024-09-16T00:07:38.620942+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:07:38.620942+0000","last_undegraded":"2024-09-16T00:07:38.620942+0000","last_fullsized":"2024-09-16T00:07:38.620942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115888+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:07:35.115888+0000","last_peered":"2024-09-16T00:07:35.115888+0000","last_clean":"2024-09-16T00:07:35.115888+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:07:35.115888+0000","last_undegraded":"2024-09-16T00:07:35.115888+0000","last_fullsized":"2024-09-16T00:07:35.115888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.048699+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:07:36.048699+0000","last_peered":"2024-09-16T00:07:36.048699+0000","last_clean":"2024-09-16T00:07:36.048699+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:07:36.048699+0000","last_undegraded":"2024-09-16T00:07:36.048699+0000","last_fullsized":"2024-09-16T00:07:36.048699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615597+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:07:34.615597+0000","last_peered":"2024-09-16T00:07:34.615597+0000","last_clean":"2024-09-16T00:07:34.615597+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:07:34.615597+0000","last_undegraded":"2024-09-16T00:07:34.615597+0000","last_fullsized":"2024-09-16T00:07:34.615597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613057+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:07:34.613057+0000","last_peered":"2024-09-16T00:07:34.613057+0000","last_clean":"2024-09-16T00:07:34.613057+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:07:34.613057+0000","last_undegraded":"2024-09-16T00:07:34.613057+0000","last_fullsized":"2024-09-16T00:07:34.613057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115577+0000","last_change":"2024-09-16T00:07:33.638312+0000","last_active":"2024-09-16T00:07:35.115577+0000","last_peered":"2024-09-16T00:07:35.115577+0000","last_clean":"2024-09-16T00:07:35.115577+0000","last_became_active":"2024-09-16T00:07:33.638049+0000","last_became_peered":"2024-09-16T00:07:33.638049+0000","last_unstale":"2024-09-16T00:07:35.115577+0000","last_undegraded":"2024-09-16T00:07:35.115577+0000","last_fullsized":"2024-09-16T00:07:35.115577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115169+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:07:35.115169+0000","last_peered":"2024-09-16T00:07:35.115169+0000","last_clean":"2024-09-16T00:07:35.115169+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:07:35.115169+0000","last_undegraded":"2024-09-16T00:07:35.115169+0000","last_fullsized":"2024-09-16T00:07:35.115169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614834+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:07:34.614834+0000","last_peered":"2024-09-16T00:07:34.614834+0000","last_clean":"2024-09-16T00:07:34.614834+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:07:34.614834+0000","last_undegraded":"2024-09-16T00:07:34.614834+0000","last_fullsized":"2024-09-16T00:07:34.614834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114892+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:07:35.114892+0000","last_peered":"2024-09-16T00:07:35.114892+0000","last_clean":"2024-09-16T00:07:35.114892+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:07:35.114892+0000","last_undegraded":"2024-09-16T00:07:35.114892+0000","last_fullsized":"2024-09-16T00:07:35.114892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115065+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:07:35.115065+0000","last_peered":"2024-09-16T00:07:35.115065+0000","last_clean":"2024-09-16T00:07:35.115065+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:07:35.115065+0000","last_undegraded":"2024-09-16T00:07:35.115065+0000","last_fullsized":"2024-09-16T00:07:35.115065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115288+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:07:35.115288+0000","last_peered":"2024-09-16T00:07:35.115288+0000","last_clean":"2024-09-16T00:07:35.115288+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:07:35.115288+0000","last_undegraded":"2024-09-16T00:07:35.115288+0000","last_fullsized":"2024-09-16T00:07:35.115288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.612974+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:07:34.612974+0000","last_peered":"2024-09-16T00:07:34.612974+0000","last_clean":"2024-09-16T00:07:34.612974+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:07:34.612974+0000","last_undegraded":"2024-09-16T00:07:34.612974+0000","last_fullsized":"2024-09-16T00:07:34.612974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114905+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:07:35.114905+0000","last_peered":"2024-09-16T00:07:35.114905+0000","last_clean":"2024-09-16T00:07:35.114905+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:07:35.114905+0000","last_undegraded":"2024-09-16T00:07:35.114905+0000","last_fullsized":"2024-09-16T00:07:35.114905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615621+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:07:34.615621+0000","last_peered":"2024-09-16T00:07:34.615621+0000","last_clean":"2024-09-16T00:07:34.615621+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:07:34.615621+0000","last_undegraded":"2024-09-16T00:07:34.615621+0000","last_fullsized":"2024-09-16T00:07:34.615621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615008+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:07:34.615008+0000","last_peered":"2024-09-16T00:07:34.615008+0000","last_clean":"2024-09-16T00:07:34.615008+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:07:34.615008+0000","last_undegraded":"2024-09-16T00:07:34.615008+0000","last_fullsized":"2024-09-16T00:07:34.615008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115398+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:07:35.115398+0000","last_peered":"2024-09-16T00:07:35.115398+0000","last_clean":"2024-09-16T00:07:35.115398+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:07:35.115398+0000","last_undegraded":"2024-09-16T00:07:35.115398+0000","last_fullsized":"2024-09-16T00:07:35.115398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615631+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:07:34.615631+0000","last_peered":"2024-09-16T00:07:34.615631+0000","last_clean":"2024-09-16T00:07:34.615631+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:07:34.615631+0000","last_undegraded":"2024-09-16T00:07:34.615631+0000","last_fullsized":"2024-09-16T00:07:34.615631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115814+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:07:35.115814+0000","last_peered":"2024-09-16T00:07:35.115814+0000","last_clean":"2024-09-16T00:07:35.115814+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:07:35.115814+0000","last_undegraded":"2024-09-16T00:07:35.115814+0000","last_fullsized":"2024-09-16T00:07:35.115814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115305+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:07:35.115305+0000","last_peered":"2024-09-16T00:07:35.115305+0000","last_clean":"2024-09-16T00:07:35.115305+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:07:35.115305+0000","last_undegraded":"2024-09-16T00:07:35.115305+0000","last_fullsized":"2024-09-16T00:07:35.115305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115644+0000","last_change":"2024-09-16T00:07:32.627236+0000","last_active":"2024-09-16T00:07:35.115644+0000","last_peered":"2024-09-16T00:07:35.115644+0000","last_clean":"2024-09-16T00:07:35.115644+0000","last_became_active":"2024-09-16T00:07:32.626743+0000","last_became_peered":"2024-09-16T00:07:32.626743+0000","last_unstale":"2024-09-16T00:07:35.115644+0000","last_undegraded":"2024-09-16T00:07:35.115644+0000","last_fullsized":"2024-09-16T00:07:35.115644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114835+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:07:35.114835+0000","last_peered":"2024-09-16T00:07:35.114835+0000","last_clean":"2024-09-16T00:07:35.114835+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:07:35.114835+0000","last_undegraded":"2024-09-16T00:07:35.114835+0000","last_fullsized":"2024-09-16T00:07:35.114835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115252+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:07:35.115252+0000","last_peered":"2024-09-16T00:07:35.115252+0000","last_clean":"2024-09-16T00:07:35.115252+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:07:35.115252+0000","last_undegraded":"2024-09-16T00:07:35.115252+0000","last_fullsized":"2024-09-16T00:07:35.115252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114804+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:07:35.114804+0000","last_peered":"2024-09-16T00:07:35.114804+0000","last_clean":"2024-09-16T00:07:35.114804+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:07:35.114804+0000","last_undegraded":"2024-09-16T00:07:35.114804+0000","last_fullsized":"2024-09-16T00:07:35.114804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114947+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:07:35.114947+0000","last_peered":"2024-09-16T00:07:35.114947+0000","last_clean":"2024-09-16T00:07:35.114947+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:07:35.114947+0000","last_undegraded":"2024-09-16T00:07:35.114947+0000","last_fullsized":"2024-09-16T00:07:35.114947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614501+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:07:34.614501+0000","last_peered":"2024-09-16T00:07:34.614501+0000","last_clean":"2024-09-16T00:07:34.614501+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:07:34.614501+0000","last_undegraded":"2024-09-16T00:07:34.614501+0000","last_fullsized":"2024-09-16T00:07:34.614501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114765+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:07:35.114765+0000","last_peered":"2024-09-16T00:07:35.114765+0000","last_clean":"2024-09-16T00:07:35.114765+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:07:35.114765+0000","last_undegraded":"2024-09-16T00:07:35.114765+0000","last_fullsized":"2024-09-16T00:07:35.114765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114640+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:07:35.114640+0000","last_peered":"2024-09-16T00:07:35.114640+0000","last_clean":"2024-09-16T00:07:35.114640+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:07:35.114640+0000","last_undegraded":"2024-09-16T00:07:35.114640+0000","last_fullsized":"2024-09-16T00:07:35.114640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115333+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:07:35.115333+0000","last_peered":"2024-09-16T00:07:35.115333+0000","last_clean":"2024-09-16T00:07:35.115333+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:07:35.115333+0000","last_undegraded":"2024-09-16T00:07:35.115333+0000","last_fullsized":"2024-09-16T00:07:35.115333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114803+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:07:35.114803+0000","last_peered":"2024-09-16T00:07:35.114803+0000","last_clean":"2024-09-16T00:07:35.114803+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:07:35.114803+0000","last_undegraded":"2024-09-16T00:07:35.114803+0000","last_fullsized":"2024-09-16T00:07:35.114803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115800+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:07:35.115800+0000","last_peered":"2024-09-16T00:07:35.115800+0000","last_clean":"2024-09-16T00:07:35.115800+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:07:35.115800+0000","last_undegraded":"2024-09-16T00:07:35.115800+0000","last_fullsized":"2024-09-16T00:07:35.115800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114993+0000","last_change":"2024-09-16T00:07:33.637979+0000","last_active":"2024-09-16T00:07:35.114993+0000","last_peered":"2024-09-16T00:07:35.114993+0000","last_clean":"2024-09-16T00:07:35.114993+0000","last_became_active":"2024-09-16T00:07:33.637713+0000","last_became_peered":"2024-09-16T00:07:33.637713+0000","last_unstale":"2024-09-16T00:07:35.114993+0000","last_undegraded":"2024-09-16T00:07:35.114993+0000","last_fullsized":"2024-09-16T00:07:35.114993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615387+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:07:34.615387+0000","last_peered":"2024-09-16T00:07:34.615387+0000","last_clean":"2024-09-16T00:07:34.615387+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:07:34.615387+0000","last_undegraded":"2024-09-16T00:07:34.615387+0000","last_fullsized":"2024-09-16T00:07:34.615387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115228+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:07:35.115228+0000","last_peered":"2024-09-16T00:07:35.115228+0000","last_clean":"2024-09-16T00:07:35.115228+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:07:35.115228+0000","last_undegraded":"2024-09-16T00:07:35.115228+0000","last_fullsized":"2024-09-16T00:07:35.115228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615266+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:07:34.615266+0000","last_peered":"2024-09-16T00:07:34.615266+0000","last_clean":"2024-09-16T00:07:34.615266+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:07:34.615266+0000","last_undegraded":"2024-09-16T00:07:34.615266+0000","last_fullsized":"2024-09-16T00:07:34.615266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115696+0000","last_change":"2024-09-16T00:07:32.632481+0000","last_active":"2024-09-16T00:07:35.115696+0000","last_peered":"2024-09-16T00:07:35.115696+0000","last_clean":"2024-09-16T00:07:35.115696+0000","last_became_active":"2024-09-16T00:07:32.632152+0000","last_became_peered":"2024-09-16T00:07:32.632152+0000","last_unstale":"2024-09-16T00:07:35.115696+0000","last_undegraded":"2024-09-16T00:07:35.115696+0000","last_fullsized":"2024-09-16T00:07:35.115696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114783+0000","last_change":"2024-09-16T00:07:33.629777+0000","last_active":"2024-09-16T00:07:35.114783+0000","last_peered":"2024-09-16T00:07:35.114783+0000","last_clean":"2024-09-16T00:07:35.114783+0000","last_became_active":"2024-09-16T00:07:33.627917+0000","last_became_peered":"2024-09-16T00:07:33.627917+0000","last_unstale":"2024-09-16T00:07:35.114783+0000","last_undegraded":"2024-09-16T00:07:35.114783+0000","last_fullsized":"2024-09-16T00:07:35.114783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:38.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.059155+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:07:36.059155+0000","last_peered":"2024-09-16T00:07:36.059155+0000","last_clean":"2024-09-16T00:07:36.059155+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:07:36.059155+0000","last_undegraded":"2024-09-16T00:07:36.059155+0000","last_fullsized":"2024-09-16T00:07:36.059155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114899+0000","last_change":"2024-09-16T00:07:33.630746+0000","last_active":"2024-09-16T00:07:35.114899+0000","last_peered":"2024-09-16T00:07:35.114899+0000","last_clean":"2024-09-16T00:07:35.114899+0000","last_became_active":"2024-09-16T00:07:33.630480+0000","last_became_peered":"2024-09-16T00:07:33.630480+0000","last_unstale":"2024-09-16T00:07:35.114899+0000","last_undegraded":"2024-09-16T00:07:35.114899+0000","last_fullsized":"2024-09-16T00:07:35.114899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:50.931998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.622293+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:07:38.622293+0000","last_peered":"2024-09-16T00:07:38.622293+0000","last_clean":"2024-09-16T00:07:38.622293+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:07:38.622293+0000","last_undegraded":"2024-09-16T00:07:38.622293+0000","last_fullsized":"2024-09-16T00:07:38.622293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":776,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22,"ondisk_log_size":22,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":668364,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":78156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":669002,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:07:42.229 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T00:07:42.229 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T00:07:42.230 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T00:07:42.230 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T00:07:42.230 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T00:07:42.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.364+0000 7f1380c0e640 1 Processor -- start 2024-09-16T00:07:42.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.364+0000 7f1380c0e640 1 -- start start 2024-09-16T00:07:42.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.365+0000 7f1380c0e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f137c15e0c0 0x7f137c15e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.365+0000 7f1380c0e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f137c0cba00 0x7f137c0c8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.365+0000 7f1380c0e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f137c0cbf40 con 0x7f137c15e9e0 2024-09-16T00:07:42.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.365+0000 7f1380c0e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f137c0c7cb0 con 0x7f137c15e0c0 2024-09-16T00:07:42.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.365+0000 7f1380c0e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f137c0c8270 con 0x7f137c0cba00 2024-09-16T00:07:42.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.365+0000 7f137a59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f137c15e0c0 0x7f137c15e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.365+0000 7f137ad9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f137c15e9e0 0x7f137c06cf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45156/0 (socket says 172.21.15.139:45156) 2024-09-16T00:07:42.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.365+0000 7f137a59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f137c15e0c0 0x7f137c15e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40330/0 (socket says 172.21.15.139:40330) 2024-09-16T00:07:42.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.365+0000 7f137ad9d640 1 -- 172.21.15.139:0/3305093621 learned_addr learned my addr 172.21.15.139:0/3305093621 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:42.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.365+0000 7f1379d9b640 1 --2- 172.21.15.139:0/3305093621 >> v2:172.21.15.164:3300/0 conn(0x7f137c0cba00 0x7f137c0c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.366+0000 7f137a59c640 1 -- 172.21.15.139:0/3305093621 >> v2:172.21.15.164:3300/0 conn(0x7f137c0cba00 msgr2=0x7f137c0c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:42.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.366+0000 7f137a59c640 1 --2- 172.21.15.139:0/3305093621 >> v2:172.21.15.164:3300/0 conn(0x7f137c0cba00 0x7f137c0c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.366+0000 7f137a59c640 1 -- 172.21.15.139:0/3305093621 >> v1:172.21.15.139:6789/0 conn(0x7f137c15e9e0 legacy=0x7f137c06cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.366+0000 7f137a59c640 1 -- 172.21.15.139:0/3305093621 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137c0cc200 con 0x7f137c15e0c0 2024-09-16T00:07:42.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.366+0000 7f137959a640 1 -- 172.21.15.139:0/3305093621 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2837138891 0 0) 0x7f137c0cbf40 con 0x7f137c15e9e0 2024-09-16T00:07:42.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.366+0000 7f137a59c640 1 --2- 172.21.15.139:0/3305093621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f137c15e0c0 0x7f137c15e4a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f137c1584b0 tx=0x7f136c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bfebbe472020d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:42.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.366+0000 7f137959a640 1 -- 172.21.15.139:0/3305093621 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c037070 con 0x7f137c15e0c0 2024-09-16T00:07:42.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.366+0000 7f137959a640 1 -- 172.21.15.139:0/3305093621 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136c032440 con 0x7f137c15e0c0 2024-09-16T00:07:42.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.366+0000 7f137959a640 1 -- 172.21.15.139:0/3305093621 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c032810 con 0x7f137c15e0c0 2024-09-16T00:07:42.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.367+0000 7f1380c0e640 1 -- 172.21.15.139:0/3305093621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f137c15e0c0 msgr2=0x7f137c15e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.367+0000 7f1380c0e640 1 --2- 172.21.15.139:0/3305093621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f137c15e0c0 0x7f137c15e4a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f137c1584b0 tx=0x7f136c02a230 comp rx=0 tx=0).stop 2024-09-16T00:07:42.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.367+0000 7f1380c0e640 1 -- 172.21.15.139:0/3305093621 shutdown_connections 2024-09-16T00:07:42.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.367+0000 7f1380c0e640 1 --2- 172.21.15.139:0/3305093621 >> v2:172.21.15.164:3300/0 conn(0x7f137c0cba00 0x7f137c0c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.367+0000 7f1380c0e640 1 --2- 172.21.15.139:0/3305093621 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f137c15e0c0 0x7f137c15e4a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.367+0000 7f1380c0e640 1 -- 172.21.15.139:0/3305093621 >> 172.21.15.139:0/3305093621 conn(0x7f137c0d03b0 msgr2=0x7f137c0d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:42.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.367+0000 7f1380c0e640 1 -- 172.21.15.139:0/3305093621 shutdown_connections 2024-09-16T00:07:42.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.367+0000 7f1380c0e640 1 -- 172.21.15.139:0/3305093621 wait complete. 2024-09-16T00:07:42.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.368+0000 7f1380c0e640 1 Processor -- start 2024-09-16T00:07:42.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.368+0000 7f1380c0e640 1 -- start start 2024-09-16T00:07:42.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.369+0000 7f1380c0e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f137c0cba00 0x7f137c157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.369+0000 7f1380c0e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f137c15e0c0 0x7f137c1f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.369+0000 7f1380c0e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f137c15e9e0 0x7f137c1ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.369+0000 7f1380c0e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f137c0c76f0 con 0x7f137c15e9e0 2024-09-16T00:07:42.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.369+0000 7f1380c0e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f137c0c7f00 con 0x7f137c0cba00 2024-09-16T00:07:42.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.369+0000 7f1380c0e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f137c0c7870 con 0x7f137c15e0c0 2024-09-16T00:07:42.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.369+0000 7f137ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f137c15e9e0 0x7f137c1ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.369+0000 7f137ad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f137c15e9e0 0x7f137c1ef2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40332/0 (socket says 172.21.15.139:40332) 2024-09-16T00:07:42.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.369+0000 7f137ad9d640 1 -- 172.21.15.139:0/2247071507 learned_addr learned my addr 172.21.15.139:0/2247071507 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:42.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.369+0000 7f137a59c640 1 --2- 172.21.15.139:0/2247071507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f137c0cba00 0x7f137c157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.369+0000 7f137ad9d640 1 -- 172.21.15.139:0/2247071507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f137c15e0c0 msgr2=0x7f137c1f37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.369+0000 7f137ad9d640 1 --2- 172.21.15.139:0/2247071507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f137c15e0c0 0x7f137c1f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.369+0000 7f137ad9d640 1 -- 172.21.15.139:0/2247071507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f137c0cba00 msgr2=0x7f137c157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.369+0000 7f137ad9d640 1 --2- 172.21.15.139:0/2247071507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f137c0cba00 0x7f137c157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.369+0000 7f137ad9d640 1 -- 172.21.15.139:0/2247071507 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137c0c2e40 con 0x7f137c15e9e0 2024-09-16T00:07:42.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.370+0000 7f137ad9d640 1 --2- 172.21.15.139:0/2247071507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f137c15e9e0 0x7f137c1ef2e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f137400c700 tx=0x7f137400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:42.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.370+0000 7f136affd640 1 -- 172.21.15.139:0/2247071507 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374018070 con 0x7f137c15e9e0 2024-09-16T00:07:42.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.370+0000 7f136affd640 1 -- 172.21.15.139:0/2247071507 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137400ce10 con 0x7f137c15e9e0 2024-09-16T00:07:42.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.370+0000 7f1380c0e640 1 -- 172.21.15.139:0/2247071507 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137c1f4140 con 0x7f137c15e9e0 2024-09-16T00:07:42.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.370+0000 7f136affd640 1 -- 172.21.15.139:0/2247071507 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137400a880 con 0x7f137c15e9e0 2024-09-16T00:07:42.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.370+0000 7f1380c0e640 1 -- 172.21.15.139:0/2247071507 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f137c1eedc0 con 0x7f137c15e9e0 2024-09-16T00:07:42.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.371+0000 7f1380c0e640 1 -- 172.21.15.139:0/2247071507 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f137c1eefc0 con 0x7f137c15e9e0 2024-09-16T00:07:42.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.371+0000 7f136affd640 1 -- 172.21.15.139:0/2247071507 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f137401b070 con 0x7f137c15e9e0 2024-09-16T00:07:42.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.372+0000 7f136affd640 1 --2- 172.21.15.139:0/2247071507 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f135c06c2b0 0x7f135c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.372+0000 7f136affd640 1 -- 172.21.15.139:0/2247071507 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f137408a8e0 con 0x7f137c15e9e0 2024-09-16T00:07:42.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.372+0000 7f137a59c640 1 --2- 172.21.15.139:0/2247071507 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f135c06c2b0 0x7f135c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.372+0000 7f137a59c640 1 --2- 172.21.15.139:0/2247071507 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f135c06c2b0 0x7f135c06e770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f136c034000 tx=0x7f136c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:42.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.376+0000 7f136affd640 1 -- 172.21.15.139:0/2247071507 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f137c1eefc0 con 0x7f137c15e9e0 2024-09-16T00:07:42.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.584+0000 7f1380c0e640 1 -- 172.21.15.139:0/2247071507 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f137c15e4a0 con 0x7f137c15e9e0 2024-09-16T00:07:42.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.587+0000 7f136affd640 1 -- 172.21.15.139:0/2247071507 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+173511 (secure 0 0 0) 0x7f13740590e0 con 0x7f137c15e9e0 2024-09-16T00:07:42.590 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:42.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.589+0000 7f1380c0e640 1 -- 172.21.15.139:0/2247071507 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f135c06c2b0 msgr2=0x7f135c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.589+0000 7f1380c0e640 1 --2- 172.21.15.139:0/2247071507 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f135c06c2b0 0x7f135c06e770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f136c034000 tx=0x7f136c02e000 comp rx=0 tx=0).stop 2024-09-16T00:07:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.589+0000 7f1380c0e640 1 -- 172.21.15.139:0/2247071507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f137c15e9e0 msgr2=0x7f137c1ef2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.589+0000 7f1380c0e640 1 --2- 172.21.15.139:0/2247071507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f137c15e9e0 0x7f137c1ef2e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f137400c700 tx=0x7f137400cbc0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.589+0000 7f1380c0e640 1 -- 172.21.15.139:0/2247071507 shutdown_connections 2024-09-16T00:07:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.589+0000 7f1380c0e640 1 --2- 172.21.15.139:0/2247071507 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f135c06c2b0 0x7f135c06e770 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.589+0000 7f1380c0e640 1 --2- 172.21.15.139:0/2247071507 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f137c15e9e0 0x7f137c1ef2e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.589+0000 7f1380c0e640 1 --2- 172.21.15.139:0/2247071507 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f137c15e0c0 0x7f137c1f37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.589+0000 7f1380c0e640 1 --2- 172.21.15.139:0/2247071507 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f137c0cba00 0x7f137c157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.589+0000 7f1380c0e640 1 -- 172.21.15.139:0/2247071507 >> 172.21.15.139:0/2247071507 conn(0x7f137c0d03b0 msgr2=0x7f137c0ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.590+0000 7f1380c0e640 1 -- 172.21.15.139:0/2247071507 shutdown_connections 2024-09-16T00:07:42.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.590+0000 7f1380c0e640 1 -- 172.21.15.139:0/2247071507 wait complete. 2024-09-16T00:07:42.601 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":2998946291},{"type":"v1","addr":"172.21.15.139:6825","nonce":2998946291}]},"active_addr":"172.21.15.139:6825/2998946291","active_change":"2024-09-16T00:07:14.017725+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":1593217842}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":2049479133}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":628464217}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.139:0","nonce":3478940121}]}]} 2024-09-16T00:07:42.606 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T00:07:42.606 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T00:07:42.606 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:07:42.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.705+0000 7f24347a6640 1 Processor -- start 2024-09-16T00:07:42.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.705+0000 7f24347a6640 1 -- start start 2024-09-16T00:07:42.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.706+0000 7f24347a6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f242c1527d0 0x7f242c163f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.706+0000 7f24347a6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f242c0c7f00 con 0x7f242c06d670 2024-09-16T00:07:42.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.706+0000 7f24347a6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f242c06bcc0 con 0x7f242c1523f0 2024-09-16T00:07:42.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.706+0000 7f24347a6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f242c06d2c0 con 0x7f242c1527d0 2024-09-16T00:07:42.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.706+0000 7f2431d41640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f242c06d670 0x7f242c06eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45164/0 (socket says 172.21.15.139:45164) 2024-09-16T00:07:42.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.706+0000 7f2432d43640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f242c1527d0 0x7f242c163f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.706+0000 7f2431d41640 1 -- 172.21.15.139:0/3048279942 learned_addr learned my addr 172.21.15.139:0/3048279942 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:42.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.706+0000 7f2431540640 1 -- 172.21.15.139:0/3048279942 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3614326638 0 0) 0x7f242c0c7f00 con 0x7f242c06d670 2024-09-16T00:07:42.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.706+0000 7f2431540640 1 -- 172.21.15.139:0/3048279942 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2418003520 con 0x7f242c06d670 2024-09-16T00:07:42.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.706+0000 7f2432d43640 1 -- 172.21.15.139:0/3048279942 >> v1:172.21.15.164:6789/0 conn(0x7f242c1523f0 legacy=0x7f242c158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.706+0000 7f2432d43640 1 -- 172.21.15.139:0/3048279942 >> v1:172.21.15.139:6789/0 conn(0x7f242c06d670 legacy=0x7f242c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.706+0000 7f2432d43640 1 -- 172.21.15.139:0/3048279942 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c06bfa0 con 0x7f242c1527d0 2024-09-16T00:07:42.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.707+0000 7f2432d43640 1 --2- 172.21.15.139:0/3048279942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f242c1527d0 0x7f242c163f10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2424009d90 tx=0x7f242402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b8e5270cb8a217 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:42.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.707+0000 7f2431540640 1 -- 172.21.15.139:0/3048279942 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424037070 con 0x7f242c1527d0 2024-09-16T00:07:42.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.707+0000 7f2431540640 1 -- 172.21.15.139:0/3048279942 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2424007740 con 0x7f242c1527d0 2024-09-16T00:07:42.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.707+0000 7f2431540640 1 -- 172.21.15.139:0/3048279942 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2424007b10 con 0x7f242c1527d0 2024-09-16T00:07:42.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.707+0000 7f24347a6640 1 -- 172.21.15.139:0/3048279942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f242c1527d0 msgr2=0x7f242c163f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.707+0000 7f24347a6640 1 --2- 172.21.15.139:0/3048279942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f242c1527d0 0x7f242c163f10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2424009d90 tx=0x7f242402a780 comp rx=0 tx=0).stop 2024-09-16T00:07:42.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.708+0000 7f24347a6640 1 -- 172.21.15.139:0/3048279942 shutdown_connections 2024-09-16T00:07:42.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.708+0000 7f24347a6640 1 --2- 172.21.15.139:0/3048279942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f242c1527d0 0x7f242c163f10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.708+0000 7f24347a6640 1 -- 172.21.15.139:0/3048279942 >> 172.21.15.139:0/3048279942 conn(0x7f242c067950 msgr2=0x7f242c067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:42.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.708+0000 7f24347a6640 1 -- 172.21.15.139:0/3048279942 shutdown_connections 2024-09-16T00:07:42.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.708+0000 7f24347a6640 1 -- 172.21.15.139:0/3048279942 wait complete. 2024-09-16T00:07:42.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.709+0000 7f24347a6640 1 Processor -- start 2024-09-16T00:07:42.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.709+0000 7f24347a6640 1 -- start start 2024-09-16T00:07:42.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.709+0000 7f24347a6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f242c06d670 0x7f242c1515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.709+0000 7f24347a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f242c1523f0 0x7f242c1505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.709+0000 7f24347a6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f242c1527d0 0x7f242c1f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.709+0000 7f24347a6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f242c0c76e0 con 0x7f242c1523f0 2024-09-16T00:07:42.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.709+0000 7f24347a6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f242c0c7cb0 con 0x7f242c1527d0 2024-09-16T00:07:42.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.709+0000 7f24347a6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f242c158580 con 0x7f242c06d670 2024-09-16T00:07:42.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.709+0000 7f2431d41640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f242c1523f0 0x7f242c1505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.710+0000 7f2431d41640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f242c1523f0 0x7f242c1505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40344/0 (socket says 172.21.15.139:40344) 2024-09-16T00:07:42.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.710+0000 7f2431d41640 1 -- 172.21.15.139:0/342553662 learned_addr learned my addr 172.21.15.139:0/342553662 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:42.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.710+0000 7f2432d43640 1 --2- 172.21.15.139:0/342553662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f242c1527d0 0x7f242c1f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.710+0000 7f2432542640 1 --2- 172.21.15.139:0/342553662 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f242c06d670 0x7f242c1515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.710+0000 7f2431d41640 1 -- 172.21.15.139:0/342553662 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f242c06d670 msgr2=0x7f242c1515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.710+0000 7f2431d41640 1 --2- 172.21.15.139:0/342553662 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f242c06d670 0x7f242c1515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.710+0000 7f2431d41640 1 -- 172.21.15.139:0/342553662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f242c1527d0 msgr2=0x7f242c1f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.710+0000 7f2431d41640 1 --2- 172.21.15.139:0/342553662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f242c1527d0 0x7f242c1f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.710+0000 7f2431d41640 1 -- 172.21.15.139:0/342553662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c1f3d10 con 0x7f242c1523f0 2024-09-16T00:07:42.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.710+0000 7f2431d41640 1 --2- 172.21.15.139:0/342553662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f242c1523f0 0x7f242c1505f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f241c00c500 tx=0x7f241c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:42.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.710+0000 7f2422ffd640 1 -- 172.21.15.139:0/342553662 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c017070 con 0x7f242c1523f0 2024-09-16T00:07:42.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.710+0000 7f2422ffd640 1 -- 172.21.15.139:0/342553662 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f241c005ce0 con 0x7f242c1523f0 2024-09-16T00:07:42.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.710+0000 7f24347a6640 1 -- 172.21.15.139:0/342553662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242c0ccb80 con 0x7f242c1523f0 2024-09-16T00:07:42.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.710+0000 7f2422ffd640 1 -- 172.21.15.139:0/342553662 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c007330 con 0x7f242c1523f0 2024-09-16T00:07:42.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.710+0000 7f24347a6640 1 -- 172.21.15.139:0/342553662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f242c1f3fd0 con 0x7f242c1523f0 2024-09-16T00:07:42.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.711+0000 7f24347a6640 1 -- 172.21.15.139:0/342553662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f242c157690 con 0x7f242c1523f0 2024-09-16T00:07:42.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.711+0000 7f2422ffd640 1 -- 172.21.15.139:0/342553662 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f241c01a070 con 0x7f242c1523f0 2024-09-16T00:07:42.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.712+0000 7f2422ffd640 1 --2- 172.21.15.139:0/342553662 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f241006c2b0 0x7f241006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.712+0000 7f2432542640 1 --2- 172.21.15.139:0/342553662 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f241006c2b0 0x7f241006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.712+0000 7f2422ffd640 1 -- 172.21.15.139:0/342553662 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f241c08a750 con 0x7f242c1523f0 2024-09-16T00:07:42.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.712+0000 7f2432542640 1 --2- 172.21.15.139:0/342553662 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f241006c2b0 0x7f241006e770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2428001780 tx=0x7f2428009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:42.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.716+0000 7f2422ffd640 1 -- 172.21.15.139:0/342553662 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f242c157690 con 0x7f242c1523f0 2024-09-16T00:07:42.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.882+0000 7f24347a6640 1 -- 172.21.15.139:0/342553662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f242c0cd1e0 con 0x7f242c1523f0 2024-09-16T00:07:42.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.883+0000 7f2422ffd640 1 -- 172.21.15.139:0/342553662 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16725 (secure 0 0 0) 0x7f241c0589c0 con 0x7f242c1523f0 2024-09-16T00:07:42.884 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:42.884 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":22,"fsid":"2d3366e5-6a59-4b59-93b2-615e72ac7015","created":"2024-09-16T00:07:09.967833+0000","modified":"2024-09-16T00:07:34.609954+0000","last_up_change":"2024-09-16T00:07:21.049678+0000","last_in_change":"2024-09-16T00:07:12.025620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:07:22.056303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:07:30.768649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:07:32.008209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83699a05-4937-43c3-aefc-b669bed9bb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6817","nonce":1974901438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6819","nonce":1974901438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6823","nonce":1974901438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6821","nonce":1974901438}]},"public_addr":"172.21.15.139:6817/1974901438","cluster_addr":"172.21.15.139:6819/1974901438","heartbeat_back_addr":"172.21.15.139:6823/1974901438","heartbeat_front_addr":"172.21.15.139:6821/1974901438","state":["exists","up"]},{"osd":1,"uuid":"9c04991e-9695-4bbb-8011-bf613bf404b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6809","nonce":2232380998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6811","nonce":2232380998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6815","nonce":2232380998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6813","nonce":2232380998}]},"public_addr":"172.21.15.139:6809/2232380998","cluster_addr":"172.21.15.139:6811/2232380998","heartbeat_back_addr":"172.21.15.139:6815/2232380998","heartbeat_front_addr":"172.21.15.139:6813/2232380998","state":["exists","up"]},{"osd":2,"uuid":"8048fc64-72b3-4866-ac62-6a8b2bf6f5c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6801","nonce":4180588905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6803","nonce":4180588905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6807","nonce":4180588905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6805","nonce":4180588905}]},"public_addr":"172.21.15.139:6801/4180588905","cluster_addr":"172.21.15.139:6803/4180588905","heartbeat_back_addr":"172.21.15.139:6807/4180588905","heartbeat_front_addr":"172.21.15.139:6805/4180588905","state":["exists","up"]},{"osd":3,"uuid":"c34ac77b-354c-4884-902d-3e2a84f5eb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6827","nonce":3947768492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6829","nonce":3947768492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6833","nonce":3947768492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6831","nonce":3947768492}]},"public_addr":"172.21.15.139:6827/3947768492","cluster_addr":"172.21.15.139:6829/3947768492","heartbeat_back_addr":"172.21.15.139:6833/3947768492","heartbeat_front_addr":"172.21.15.139:6831/3947768492","state":["exists","up"]},{"osd":4,"uuid":"ba9a7c93-9672-43e1-8c98-2878cdf7f548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6817","nonce":642632916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6819","nonce":642632916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6823","nonce":642632916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6821","nonce":642632916}]},"public_addr":"172.21.15.164:6817/642632916","cluster_addr":"172.21.15.164:6819/642632916","heartbeat_back_addr":"172.21.15.164:6823/642632916","heartbeat_front_addr":"172.21.15.164:6821/642632916","state":["exists","up"]},{"osd":5,"uuid":"46a0916b-474c-411b-84ad-86f27d4cbf53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6801","nonce":1051778383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6803","nonce":1051778383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6807","nonce":1051778383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6805","nonce":1051778383}]},"public_addr":"172.21.15.164:6801/1051778383","cluster_addr":"172.21.15.164:6803/1051778383","heartbeat_back_addr":"172.21.15.164:6807/1051778383","heartbeat_front_addr":"172.21.15.164:6805/1051778383","state":["exists","up"]},{"osd":6,"uuid":"9b6b33cc-d6d9-4a0a-8143-526073a58b93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6825","nonce":1908495807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6827","nonce":1908495807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6831","nonce":1908495807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6829","nonce":1908495807}]},"public_addr":"172.21.15.164:6825/1908495807","cluster_addr":"172.21.15.164:6827/1908495807","heartbeat_back_addr":"172.21.15.164:6831/1908495807","heartbeat_front_addr":"172.21.15.164:6829/1908495807","state":["exists","up"]},{"osd":7,"uuid":"50bcd808-3d3e-4a72-88e4-30decb53cfea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6809","nonce":1158838808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6811","nonce":1158838808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6815","nonce":1158838808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6813","nonce":1158838808}]},"public_addr":"172.21.15.164:6809/1158838808","cluster_addr":"172.21.15.164:6811/1158838808","heartbeat_back_addr":"172.21.15.164:6815/1158838808","heartbeat_front_addr":"172.21.15.164:6813/1158838808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.843117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.622789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:17.919457+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:19.330341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.815640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.083046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.551499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.377179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:07:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.885+0000 7f24347a6640 1 -- 172.21.15.139:0/342553662 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f241006c2b0 msgr2=0x7f241006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.885+0000 7f24347a6640 1 --2- 172.21.15.139:0/342553662 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f241006c2b0 0x7f241006e770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2428001780 tx=0x7f2428009040 comp rx=0 tx=0).stop 2024-09-16T00:07:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.885+0000 7f24347a6640 1 -- 172.21.15.139:0/342553662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f242c1523f0 msgr2=0x7f242c1505f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.885+0000 7f24347a6640 1 --2- 172.21.15.139:0/342553662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f242c1523f0 0x7f242c1505f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f241c00c500 tx=0x7f241c00c9c0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.885+0000 7f24347a6640 1 -- 172.21.15.139:0/342553662 shutdown_connections 2024-09-16T00:07:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.885+0000 7f24347a6640 1 --2- 172.21.15.139:0/342553662 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f241006c2b0 0x7f241006e770 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.885+0000 7f24347a6640 1 --2- 172.21.15.139:0/342553662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f242c1527d0 0x7f242c1f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.885+0000 7f24347a6640 1 --2- 172.21.15.139:0/342553662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f242c1523f0 0x7f242c1505f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.885+0000 7f24347a6640 1 --2- 172.21.15.139:0/342553662 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f242c06d670 0x7f242c1515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:42.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.885+0000 7f24347a6640 1 -- 172.21.15.139:0/342553662 >> 172.21.15.139:0/342553662 conn(0x7f242c067950 msgr2=0x7f242c0c9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:42.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.885+0000 7f24347a6640 1 -- 172.21.15.139:0/342553662 shutdown_connections 2024-09-16T00:07:42.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.885+0000 7f24347a6640 1 -- 172.21.15.139:0/342553662 wait complete. 2024-09-16T00:07:42.897 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T00:07:42.897 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:07:42.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.995+0000 7f8b7d3a6640 1 Processor -- start 2024-09-16T00:07:42.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.996+0000 7f8b7d3a6640 1 -- start start 2024-09-16T00:07:42.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.996+0000 7f8b7d3a6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8b780c75d0 0x7f8b7806d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:42.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.996+0000 7f8b7d3a6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b780cc4e0 con 0x7f8b78154e10 2024-09-16T00:07:42.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.996+0000 7f8b7d3a6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b7806bfe0 con 0x7f8b7815b600 2024-09-16T00:07:42.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.996+0000 7f8b7d3a6640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f8b780c79b0 con 0x7f8b780c75d0 2024-09-16T00:07:42.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.996+0000 7f8b76ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8b78154e10 0x7f8b780cbdd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45170/0 (socket says 172.21.15.139:45170) 2024-09-16T00:07:42.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.996+0000 7f8b76ffd640 1 -- 172.21.15.139:0/2378211687 learned_addr learned my addr 172.21.15.139:0/2378211687 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:42.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.997+0000 7f8b777fe640 1 --2- 172.21.15.139:0/2378211687 >> v2:172.21.15.164:3300/0 conn(0x7f8b780c75d0 0x7f8b7806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:42.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.997+0000 7f8b767fc640 1 -- 172.21.15.139:0/2378211687 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 901243539 0 0) 0x7f8b780cc4e0 con 0x7f8b78154e10 2024-09-16T00:07:42.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.997+0000 7f8b767fc640 1 -- 172.21.15.139:0/2378211687 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b64003520 con 0x7f8b78154e10 2024-09-16T00:07:42.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.997+0000 7f8b777fe640 1 -- 172.21.15.139:0/2378211687 >> v1:172.21.15.164:6789/0 conn(0x7f8b7815b600 legacy=0x7f8b780cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.997+0000 7f8b767fc640 1 -- 172.21.15.139:0/2378211687 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3112823535 0 0) 0x7f8b64003520 con 0x7f8b78154e10 2024-09-16T00:07:42.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.997+0000 7f8b777fe640 1 -- 172.21.15.139:0/2378211687 >> v1:172.21.15.139:6789/0 conn(0x7f8b78154e10 legacy=0x7f8b780cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:42.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.997+0000 7f8b777fe640 1 -- 172.21.15.139:0/2378211687 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7806c2c0 con 0x7f8b780c75d0 2024-09-16T00:07:43.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.998+0000 7f8b777fe640 1 --2- 172.21.15.139:0/2378211687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b780c75d0 0x7f8b7806d360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009d90 tx=0x7f8b7002a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8a0c3c602a9a365 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.998+0000 7f8b767fc640 1 -- 172.21.15.139:0/2378211687 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70035020 con 0x7f8b780c75d0 2024-09-16T00:07:43.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.998+0000 7f8b767fc640 1 -- 172.21.15.139:0/2378211687 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b70007540 con 0x7f8b780c75d0 2024-09-16T00:07:43.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.998+0000 7f8b767fc640 1 -- 172.21.15.139:0/2378211687 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68001670 con 0x7f8b780c75d0 2024-09-16T00:07:43.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.998+0000 7f8b7d3a6640 1 -- 172.21.15.139:0/2378211687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b780c75d0 msgr2=0x7f8b7806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.998+0000 7f8b7d3a6640 1 --2- 172.21.15.139:0/2378211687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b780c75d0 0x7f8b7806d360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009d90 tx=0x7f8b7002a630 comp rx=0 tx=0).stop 2024-09-16T00:07:43.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.998+0000 7f8b7d3a6640 1 -- 172.21.15.139:0/2378211687 shutdown_connections 2024-09-16T00:07:43.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.998+0000 7f8b7d3a6640 1 --2- 172.21.15.139:0/2378211687 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b780c75d0 0x7f8b7806d360 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.998+0000 7f8b7d3a6640 1 -- 172.21.15.139:0/2378211687 >> 172.21.15.139:0/2378211687 conn(0x7f8b780d03b0 msgr2=0x7f8b780d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.998+0000 7f8b7d3a6640 1 -- 172.21.15.139:0/2378211687 shutdown_connections 2024-09-16T00:07:43.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.998+0000 7f8b7d3a6640 1 -- 172.21.15.139:0/2378211687 wait complete. 2024-09-16T00:07:43.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:42.999+0000 7f8b7d3a6640 1 Processor -- start 2024-09-16T00:07:43.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.000+0000 7f8b7d3a6640 1 -- start start 2024-09-16T00:07:43.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.000+0000 7f8b7d3a6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8b780c75d0 0x7f8b781e23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.000+0000 7f8b7d3a6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b78154e10 0x7f8b78159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.000+0000 7f8b7d3a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b7815b600 0x7f8b781f0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.000+0000 7f8b7d3a6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7806cb60 con 0x7f8b7815b600 2024-09-16T00:07:43.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.000+0000 7f8b7d3a6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7806c880 con 0x7f8b78154e10 2024-09-16T00:07:43.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.000+0000 7f8b7d3a6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7806ce40 con 0x7f8b780c75d0 2024-09-16T00:07:43.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.000+0000 7f8b777fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b7815b600 0x7f8b781f0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.000+0000 7f8b777fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b7815b600 0x7f8b781f0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40348/0 (socket says 172.21.15.139:40348) 2024-09-16T00:07:43.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.000+0000 7f8b777fe640 1 -- 172.21.15.139:0/1485994198 learned_addr learned my addr 172.21.15.139:0/1485994198 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.001+0000 7f8b6e7fc640 1 --2- 172.21.15.139:0/1485994198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b78154e10 0x7f8b78159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.001+0000 7f8b76ffd640 1 --2- 172.21.15.139:0/1485994198 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8b780c75d0 0x7f8b781e23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.001+0000 7f8b777fe640 1 -- 172.21.15.139:0/1485994198 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8b780c75d0 msgr2=0x7f8b781e23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.001+0000 7f8b777fe640 1 --2- 172.21.15.139:0/1485994198 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8b780c75d0 0x7f8b781e23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.001+0000 7f8b777fe640 1 -- 172.21.15.139:0/1485994198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b78154e10 msgr2=0x7f8b78159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.001+0000 7f8b777fe640 1 --2- 172.21.15.139:0/1485994198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b78154e10 0x7f8b78159680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.001+0000 7f8b777fe640 1 -- 172.21.15.139:0/1485994198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b781e09e0 con 0x7f8b7815b600 2024-09-16T00:07:43.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.001+0000 7f8b777fe640 1 --2- 172.21.15.139:0/1485994198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b7815b600 0x7f8b781f0790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009eb0 tx=0x7f8b70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.001+0000 7f8b6ffff640 1 -- 172.21.15.139:0/1485994198 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70035070 con 0x7f8b7815b600 2024-09-16T00:07:43.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.001+0000 7f8b6ffff640 1 -- 172.21.15.139:0/1485994198 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b70031c40 con 0x7f8b7815b600 2024-09-16T00:07:43.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.001+0000 7f8b7d3a6640 1 -- 172.21.15.139:0/1485994198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b781e0cb0 con 0x7f8b7815b600 2024-09-16T00:07:43.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.001+0000 7f8b6ffff640 1 -- 172.21.15.139:0/1485994198 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70009450 con 0x7f8b7815b600 2024-09-16T00:07:43.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.001+0000 7f8b7d3a6640 1 -- 172.21.15.139:0/1485994198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b781f3f50 con 0x7f8b7815b600 2024-09-16T00:07:43.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.002+0000 7f8b7d3a6640 1 -- 172.21.15.139:0/1485994198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b7806bcc0 con 0x7f8b7815b600 2024-09-16T00:07:43.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.003+0000 7f8b6ffff640 1 -- 172.21.15.139:0/1485994198 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8b70003990 con 0x7f8b7815b600 2024-09-16T00:07:43.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.003+0000 7f8b6ffff640 1 --2- 172.21.15.139:0/1485994198 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8b5806c2b0 0x7f8b5806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.003+0000 7f8b76ffd640 1 --2- 172.21.15.139:0/1485994198 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8b5806c2b0 0x7f8b5806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.003+0000 7f8b6ffff640 1 -- 172.21.15.139:0/1485994198 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8b700ac800 con 0x7f8b7815b600 2024-09-16T00:07:43.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.003+0000 7f8b76ffd640 1 --2- 172.21.15.139:0/1485994198 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8b5806c2b0 0x7f8b5806e770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8b68001eb0 tx=0x7f8b68001e20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.007+0000 7f8b6ffff640 1 -- 172.21.15.139:0/1485994198 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8b7007a9b0 con 0x7f8b7815b600 2024-09-16T00:07:43.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.175+0000 7f8b7d3a6640 1 -- 172.21.15.139:0/1485994198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b7806bcc0 con 0x7f8b7815b600 2024-09-16T00:07:43.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.176+0000 7f8b6ffff640 1 -- 172.21.15.139:0/1485994198 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16725 (secure 0 0 0) 0x7f8b7007ecb0 con 0x7f8b7815b600 2024-09-16T00:07:43.177 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:43.177 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":22,"fsid":"2d3366e5-6a59-4b59-93b2-615e72ac7015","created":"2024-09-16T00:07:09.967833+0000","modified":"2024-09-16T00:07:34.609954+0000","last_up_change":"2024-09-16T00:07:21.049678+0000","last_in_change":"2024-09-16T00:07:12.025620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:07:22.056303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:07:30.768649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:07:32.008209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83699a05-4937-43c3-aefc-b669bed9bb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6817","nonce":1974901438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6819","nonce":1974901438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6823","nonce":1974901438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6821","nonce":1974901438}]},"public_addr":"172.21.15.139:6817/1974901438","cluster_addr":"172.21.15.139:6819/1974901438","heartbeat_back_addr":"172.21.15.139:6823/1974901438","heartbeat_front_addr":"172.21.15.139:6821/1974901438","state":["exists","up"]},{"osd":1,"uuid":"9c04991e-9695-4bbb-8011-bf613bf404b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6809","nonce":2232380998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6811","nonce":2232380998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6815","nonce":2232380998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6813","nonce":2232380998}]},"public_addr":"172.21.15.139:6809/2232380998","cluster_addr":"172.21.15.139:6811/2232380998","heartbeat_back_addr":"172.21.15.139:6815/2232380998","heartbeat_front_addr":"172.21.15.139:6813/2232380998","state":["exists","up"]},{"osd":2,"uuid":"8048fc64-72b3-4866-ac62-6a8b2bf6f5c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6801","nonce":4180588905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6803","nonce":4180588905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6807","nonce":4180588905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6805","nonce":4180588905}]},"public_addr":"172.21.15.139:6801/4180588905","cluster_addr":"172.21.15.139:6803/4180588905","heartbeat_back_addr":"172.21.15.139:6807/4180588905","heartbeat_front_addr":"172.21.15.139:6805/4180588905","state":["exists","up"]},{"osd":3,"uuid":"c34ac77b-354c-4884-902d-3e2a84f5eb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6827","nonce":3947768492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6829","nonce":3947768492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6833","nonce":3947768492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6831","nonce":3947768492}]},"public_addr":"172.21.15.139:6827/3947768492","cluster_addr":"172.21.15.139:6829/3947768492","heartbeat_back_addr":"172.21.15.139:6833/3947768492","heartbeat_front_addr":"172.21.15.139:6831/3947768492","state":["exists","up"]},{"osd":4,"uuid":"ba9a7c93-9672-43e1-8c98-2878cdf7f548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6817","nonce":642632916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6819","nonce":642632916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6823","nonce":642632916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6821","nonce":642632916}]},"public_addr":"172.21.15.164:6817/642632916","cluster_addr":"172.21.15.164:6819/642632916","heartbeat_back_addr":"172.21.15.164:6823/642632916","heartbeat_front_addr":"172.21.15.164:6821/642632916","state":["exists","up"]},{"osd":5,"uuid":"46a0916b-474c-411b-84ad-86f27d4cbf53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6801","nonce":1051778383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6803","nonce":1051778383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6807","nonce":1051778383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6805","nonce":1051778383}]},"public_addr":"172.21.15.164:6801/1051778383","cluster_addr":"172.21.15.164:6803/1051778383","heartbeat_back_addr":"172.21.15.164:6807/1051778383","heartbeat_front_addr":"172.21.15.164:6805/1051778383","state":["exists","up"]},{"osd":6,"uuid":"9b6b33cc-d6d9-4a0a-8143-526073a58b93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6825","nonce":1908495807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6827","nonce":1908495807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6831","nonce":1908495807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6829","nonce":1908495807}]},"public_addr":"172.21.15.164:6825/1908495807","cluster_addr":"172.21.15.164:6827/1908495807","heartbeat_back_addr":"172.21.15.164:6831/1908495807","heartbeat_front_addr":"172.21.15.164:6829/1908495807","state":["exists","up"]},{"osd":7,"uuid":"50bcd808-3d3e-4a72-88e4-30decb53cfea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6809","nonce":1158838808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6811","nonce":1158838808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6815","nonce":1158838808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6813","nonce":1158838808}]},"public_addr":"172.21.15.164:6809/1158838808","cluster_addr":"172.21.15.164:6811/1158838808","heartbeat_back_addr":"172.21.15.164:6815/1158838808","heartbeat_front_addr":"172.21.15.164:6813/1158838808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.843117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.622789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:17.919457+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:19.330341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.815640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.083046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.551499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.377179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:07:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.178+0000 7f8b7d3a6640 1 -- 172.21.15.139:0/1485994198 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8b5806c2b0 msgr2=0x7f8b5806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.178+0000 7f8b7d3a6640 1 --2- 172.21.15.139:0/1485994198 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8b5806c2b0 0x7f8b5806e770 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8b68001eb0 tx=0x7f8b68001e20 comp rx=0 tx=0).stop 2024-09-16T00:07:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.178+0000 7f8b7d3a6640 1 -- 172.21.15.139:0/1485994198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b7815b600 msgr2=0x7f8b781f0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.178+0000 7f8b7d3a6640 1 --2- 172.21.15.139:0/1485994198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b7815b600 0x7f8b781f0790 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009eb0 tx=0x7f8b70004a40 comp rx=0 tx=0).stop 2024-09-16T00:07:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.178+0000 7f8b7d3a6640 1 -- 172.21.15.139:0/1485994198 shutdown_connections 2024-09-16T00:07:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.178+0000 7f8b7d3a6640 1 --2- 172.21.15.139:0/1485994198 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8b5806c2b0 0x7f8b5806e770 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.178+0000 7f8b7d3a6640 1 --2- 172.21.15.139:0/1485994198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b7815b600 0x7f8b781f0790 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.178+0000 7f8b7d3a6640 1 --2- 172.21.15.139:0/1485994198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8b78154e10 0x7f8b78159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.178+0000 7f8b7d3a6640 1 --2- 172.21.15.139:0/1485994198 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8b780c75d0 0x7f8b781e23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.178+0000 7f8b7d3a6640 1 -- 172.21.15.139:0/1485994198 >> 172.21.15.139:0/1485994198 conn(0x7f8b780d03b0 msgr2=0x7f8b780d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.178+0000 7f8b7d3a6640 1 -- 172.21.15.139:0/1485994198 shutdown_connections 2024-09-16T00:07:43.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.178+0000 7f8b7d3a6640 1 -- 172.21.15.139:0/1485994198 wait complete. 2024-09-16T00:07:43.191 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T00:07:43.191 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T00:07:43.192 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T00:07:43.193 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T00:07:43.194 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T00:07:43.195 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T00:07:43.195 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T00:07:43.196 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T00:07:43.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.362+0000 7f97374c9640 1 Processor -- start 2024-09-16T00:07:43.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.362+0000 7f97374c9640 1 -- start start 2024-09-16T00:07:43.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.362+0000 7f58eb59e640 1 Processor -- start 2024-09-16T00:07:43.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.362+0000 7f58eb59e640 1 -- start start 2024-09-16T00:07:43.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.363+0000 7f068cbc4640 1 Processor -- start 2024-09-16T00:07:43.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.363+0000 7f068cbc4640 1 -- start start 2024-09-16T00:07:43.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.363+0000 7f97374c9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9730071700 0x7f97301566f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.363+0000 7f97374c9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f97300c1000 0x7f97300c23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.363+0000 7f97374c9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9730157210 con 0x7f97300c2900 2024-09-16T00:07:43.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.363+0000 7f97374c9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f97300c1f30 con 0x7f97300c1000 2024-09-16T00:07:43.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.363+0000 7f97374c9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f97300c1970 con 0x7f9730071700 2024-09-16T00:07:43.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.363+0000 7f7623fff640 1 Processor -- start 2024-09-16T00:07:43.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7f7623fff640 1 -- start start 2024-09-16T00:07:43.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7fc6f054b640 1 Processor -- start 2024-09-16T00:07:43.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7f9734a64640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f97300c1000 0x7f97300c23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7f9734a64640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f97300c1000 0x7f97300c23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40356/0 (socket says 172.21.15.139:40356) 2024-09-16T00:07:43.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7f9734a64640 1 -- 172.21.15.139:0/1293119401 learned_addr learned my addr 172.21.15.139:0/1293119401 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7f9735265640 1 --2- 172.21.15.139:0/1293119401 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9730071700 0x7f97301566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7f58eb59e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ec0b1d80 0x7f58ec06cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7f58eb59e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f58ec15eb60 0x7f58ec15f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7f58eb59e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58ec073040 con 0x7f58ec06d070 2024-09-16T00:07:43.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7f58eb59e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f58ec1590f0 con 0x7f58ec15eb60 2024-09-16T00:07:43.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7f58eb59e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f58ec071cc0 con 0x7f58ec0b1d80 2024-09-16T00:07:43.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7fb875cd0640 1 Processor -- start 2024-09-16T00:07:43.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7fb875cd0640 1 -- start start 2024-09-16T00:07:43.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7f58ea59c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ec0b1d80 0x7f58ec06cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7f58ea59c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ec0b1d80 0x7f58ec06cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:54950/0 (socket says 172.21.15.139:54950) 2024-09-16T00:07:43.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7f58ea59c640 1 -- 172.21.15.139:0/3494607290 learned_addr learned my addr 172.21.15.139:0/3494607290 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.364+0000 7f9735265640 1 -- 172.21.15.139:0/1293119401 >> v2:172.21.15.139:3300/0 conn(0x7f97300c1000 msgr2=0x7f97300c23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:43.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f58ea59c640 1 -- 172.21.15.139:0/3494607290 >> v2:172.21.15.139:3300/0 conn(0x7f58ec15eb60 msgr2=0x7f58ec15f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:43.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f58ea59c640 1 --2- 172.21.15.139:0/3494607290 >> v2:172.21.15.139:3300/0 conn(0x7f58ec15eb60 0x7f58ec15f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f58ea59c640 1 -- 172.21.15.139:0/3494607290 >> v1:172.21.15.139:6789/0 conn(0x7f58ec06d070 legacy=0x7f58ec0c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:43.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f58ea59c640 1 -- 172.21.15.139:0/3494607290 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ec071fa0 con 0x7f58ec0b1d80 2024-09-16T00:07:43.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f068cbc4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f06800d4970 0x7f0680005a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f068cbc4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0680045120 0x7f06800ddc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f068cbc4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f068004c0c0 0x7f06800de1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f0685d9b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0680045120 0x7f06800ddc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f0685d9b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0680045120 0x7f06800ddc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39410/0 (socket says 172.21.15.139:39410) 2024-09-16T00:07:43.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f0685d9b640 1 -- 172.21.15.139:0/3592172292 learned_addr learned my addr 172.21.15.139:0/3592172292 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f068cbc4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0680044c90 con 0x7f06800d4970 2024-09-16T00:07:43.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f068cbc4640 1 -- 172.21.15.139:0/3592172292 --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f068004aa70 con 0x7f0680045120 2024-09-16T00:07:43.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f068cbc4640 1 -- 172.21.15.139:0/3592172292 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f068004ad10 con 0x7f068004c0c0 2024-09-16T00:07:43.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f068659c640 1 --2- 172.21.15.139:0/3592172292 >> v2:172.21.15.139:3300/0 conn(0x7f06800d4970 0x7f0680005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7f0686d9d640 1 --2- 172.21.15.139:0/3592172292 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f068004c0c0 0x7f06800de1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7fb875cd0640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb87015fa70 0x7fb8700b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7fc6f054b640 1 -- start start 2024-09-16T00:07:43.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f7623fff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f761c0d5850 0x7f761c007730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f7623fff640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f761c0dc3a0 con 0x7f761c047a80 2024-09-16T00:07:43.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f7623fff640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f761c0dc0c0 con 0x7f761c04d310 2024-09-16T00:07:43.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f7623fff640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f761c0dbe90 con 0x7f761c0d5850 2024-09-16T00:07:43.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f7622ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f761c0d5850 0x7f761c007730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f76227fc640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f761c04d310 0x7f761c045d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:35802/0 (socket says 172.21.15.139:35802) 2024-09-16T00:07:43.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f76227fc640 1 -- 172.21.15.139:0/2221925479 learned_addr learned my addr 172.21.15.139:0/2221925479 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:43.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7f7621ffb640 1 -- 172.21.15.139:0/2221925479 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1267814831 0 0) 0x7f761c0dc0c0 con 0x7f761c04d310 2024-09-16T00:07:43.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7f7621ffb640 1 -- 172.21.15.139:0/2221925479 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f760c003520 con 0x7f761c04d310 2024-09-16T00:07:43.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7fb875cd0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb8700b5280 0x7fb870162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7f7622ffd640 1 -- 172.21.15.139:0/2221925479 >> v1:172.21.15.164:6789/0 conn(0x7f761c04d310 legacy=0x7f761c045d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7f7622ffd640 1 -- 172.21.15.139:0/2221925479 >> v1:172.21.15.139:6789/0 conn(0x7f761c047a80 legacy=0x7f761c0e5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7f7622ffd640 1 -- 172.21.15.139:0/2221925479 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c046cc0 con 0x7f761c0d5850 2024-09-16T00:07:43.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7fd32ecd7640 1 Processor -- start 2024-09-16T00:07:43.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7fb875cd0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb8700738c0 0x7fb8701628c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7f0685d9b640 1 -- 172.21.15.139:0/3592172292 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f068004c0c0 msgr2=0x7f06800de1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:43.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7f0685d9b640 1 --2- 172.21.15.139:0/3592172292 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f068004c0c0 0x7f06800de1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7f0685d9b640 1 -- 172.21.15.139:0/3592172292 >> v2:172.21.15.139:3300/0 conn(0x7f06800d4970 msgr2=0x7f0680005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:43.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7f0685d9b640 1 --2- 172.21.15.139:0/3592172292 >> v2:172.21.15.139:3300/0 conn(0x7f06800d4970 0x7f0680005a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fb875cd0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb8700b5b90 con 0x7fb8700738c0 2024-09-16T00:07:43.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7f0685d9b640 1 -- 172.21.15.139:0/3592172292 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068004afd0 con 0x7f0680045120 2024-09-16T00:07:43.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fb86effd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb8700b5280 0x7fb870162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fb86effd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb8700b5280 0x7fb870162380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:54970/0 (socket says 172.21.15.139:54970) 2024-09-16T00:07:43.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fb86effd640 1 -- 172.21.15.139:0/2282560182 learned_addr learned my addr 172.21.15.139:0/2282560182 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fb875cd0640 1 -- 172.21.15.139:0/2282560182 --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb8700caf30 con 0x7fb87015fa70 2024-09-16T00:07:43.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7fd32ecd7640 1 -- start start 2024-09-16T00:07:43.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fb875cd0640 1 -- 172.21.15.139:0/2282560182 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb8700c52a0 con 0x7fb8700b5280 2024-09-16T00:07:43.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7fc6f054b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6e80cc330 0x7fc6e81572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7fc6f054b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc6e806de20 0x7fc6e8072550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fc6f054b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6e80c9b40 con 0x7fc6e80c8ad0 2024-09-16T00:07:43.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fc6f054b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fc6e80720c0 con 0x7fc6e806de20 2024-09-16T00:07:43.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fc6f054b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc6e80c8fc0 con 0x7fc6e80cc330 2024-09-16T00:07:43.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.365+0000 7f9727fff640 1 -- 172.21.15.139:0/1293119401 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2122878850 0 0) 0x7f9730157210 con 0x7f97300c2900 2024-09-16T00:07:43.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7f9735265640 1 --2- 172.21.15.139:0/1293119401 >> v2:172.21.15.139:3300/0 conn(0x7f97300c1000 0x7f97300c23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7f9735265640 1 -- 172.21.15.139:0/1293119401 >> v1:172.21.15.139:6789/0 conn(0x7f97300c2900 legacy=0x7f97300c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.366+0000 7f9735265640 1 -- 172.21.15.139:0/1293119401 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97301574f0 con 0x7f9730071700 2024-09-16T00:07:43.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7f9734a64640 1 --2- 172.21.15.139:0/1293119401 >> v2:172.21.15.139:3300/0 conn(0x7f97300c1000 0x7f97300c23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:43.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7f9735265640 1 --2- 172.21.15.139:0/1293119401 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9730071700 0x7f97301566f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9728009d90 tx=0x7f972802a610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f42f3782bf22096 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fc6ee2e7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6e80cc330 0x7fc6e81572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fc6ee2e7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6e80cc330 0x7fc6e81572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:54982/0 (socket says 172.21.15.139:54982) 2024-09-16T00:07:43.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fc6ee2e7640 1 -- 172.21.15.139:0/2511155148 learned_addr learned my addr 172.21.15.139:0/2511155148 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fc6edae6640 1 --2- 172.21.15.139:0/2511155148 >> v2:172.21.15.164:3300/0 conn(0x7fc6e806de20 0x7fc6e8072550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7f53bd5bd640 1 Processor -- start 2024-09-16T00:07:43.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7f0685d9b640 1 --2- 172.21.15.139:0/3592172292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0680045120 0x7f06800ddc80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f06800360c0 tx=0x7f067c00abe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf235eb0d9b0c36a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fb86effd640 1 -- 172.21.15.139:0/2282560182 >> v2:172.21.15.164:3300/0 conn(0x7fb87015fa70 msgr2=0x7fb8700b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:43.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fb86effd640 1 --2- 172.21.15.139:0/2282560182 >> v2:172.21.15.164:3300/0 conn(0x7fb87015fa70 0x7fb8700b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fb86effd640 1 -- 172.21.15.139:0/2282560182 >> v2:172.21.15.139:3300/0 conn(0x7fb8700738c0 msgr2=0x7fb8701628c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:43.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fb86effd640 1 --2- 172.21.15.139:0/2282560182 >> v2:172.21.15.139:3300/0 conn(0x7fb8700738c0 0x7fb8701628c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fb86effd640 1 -- 172.21.15.139:0/2282560182 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8700c5580 con 0x7fb8700b5280 2024-09-16T00:07:43.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fc6ee2e7640 1 -- 172.21.15.139:0/2511155148 >> v2:172.21.15.164:3300/0 conn(0x7fc6e806de20 msgr2=0x7fc6e8072550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:43.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fc6ee2e7640 1 --2- 172.21.15.139:0/2511155148 >> v2:172.21.15.164:3300/0 conn(0x7fc6e806de20 0x7fc6e8072550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fc6ee2e7640 1 -- 172.21.15.139:0/2511155148 >> v1:172.21.15.139:6789/0 conn(0x7fc6e80c8ad0 legacy=0x7fc6e8150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7fc6ee2e7640 1 -- 172.21.15.139:0/2511155148 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e80c92a0 con 0x7fc6e80cc330 2024-09-16T00:07:43.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7f9727fff640 1 -- 172.21.15.139:0/1293119401 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728007740 con 0x7f9730071700 2024-09-16T00:07:43.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7f9727fff640 1 -- 172.21.15.139:0/1293119401 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97280078e0 con 0x7f9730071700 2024-09-16T00:07:43.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7f53bd5bd640 1 -- start start 2024-09-16T00:07:43.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.367+0000 7f58ea59c640 1 --2- 172.21.15.139:0/3494607290 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ec0b1d80 0x7f58ec06cb30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009e50 tx=0x7f58e402a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca840e451d7ccf2a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7f068559a640 1 -- 172.21.15.139:0/3592172292 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c03a070 con 0x7f0680045120 2024-09-16T00:07:43.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7f7622ffd640 1 --2- 172.21.15.139:0/2221925479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f761c0d5850 0x7f761c007730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f76180097a0 tx=0x7f761802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e60a3ddfbcbc20c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7f068559a640 1 -- 172.21.15.139:0/3592172292 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f067c008df0 con 0x7f0680045120 2024-09-16T00:07:43.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7f068559a640 1 -- 172.21.15.139:0/3592172292 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c00a2f0 con 0x7f0680045120 2024-09-16T00:07:43.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7fd32ecd7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3280c8b00 0x7fd3281572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7fd32ecd7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd328072c60 0x7fd328150630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7fd32ecd7640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3280c9b40 con 0x7fd3280cc330 2024-09-16T00:07:43.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7fd32ecd7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd3280720c0 con 0x7fd328072c60 2024-09-16T00:07:43.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7fd32ecd7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd3280c8fc0 con 0x7fd3280c8b00 2024-09-16T00:07:43.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7f7621ffb640 1 -- 172.21.15.139:0/2221925479 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618037070 con 0x7f761c0d5850 2024-09-16T00:07:43.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7f7621ffb640 1 -- 172.21.15.139:0/2221925479 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7618032440 con 0x7f761c0d5850 2024-09-16T00:07:43.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7fd32ca73640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd3280cc330 0x7fd328072550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:35810/0 (socket says 172.21.15.139:35810) 2024-09-16T00:07:43.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7fd327fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3280c8b00 0x7fd3281572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7fd32ca73640 1 -- 172.21.15.139:0/3277216975 learned_addr learned my addr 172.21.15.139:0/3277216975 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:43.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7f7621ffb640 1 -- 172.21.15.139:0/2221925479 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618032810 con 0x7f761c0d5850 2024-09-16T00:07:43.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7f58e959a640 1 -- 172.21.15.139:0/3494607290 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e4036070 con 0x7f58ec0b1d80 2024-09-16T00:07:43.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7f58e959a640 1 -- 172.21.15.139:0/3494607290 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e4031600 con 0x7f58ec0b1d80 2024-09-16T00:07:43.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7f58e959a640 1 -- 172.21.15.139:0/3494607290 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e40319d0 con 0x7f58ec0b1d80 2024-09-16T00:07:43.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7fd32d274640 1 --2- 172.21.15.139:0/3277216975 >> v2:172.21.15.139:3300/0 conn(0x7fd328072c60 0x7fd328150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7fc6ee2e7640 1 --2- 172.21.15.139:0/2511155148 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6e80cc330 0x7fc6e81572f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009870 tx=0x7fc6e402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcb56a0561b444d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7fd327fff640 1 -- 172.21.15.139:0/3277216975 >> v2:172.21.15.139:3300/0 conn(0x7fd328072c60 msgr2=0x7fd328150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:43.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7fd327fff640 1 --2- 172.21.15.139:0/3277216975 >> v2:172.21.15.139:3300/0 conn(0x7fd328072c60 0x7fd328150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7f9727fff640 1 -- 172.21.15.139:0/1293119401 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728007cd0 con 0x7f9730071700 2024-09-16T00:07:43.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7fd3277fe640 1 -- 172.21.15.139:0/3277216975 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 572842712 0 0) 0x7fd3280c9b40 con 0x7fd3280cc330 2024-09-16T00:07:43.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7fd327fff640 1 -- 172.21.15.139:0/3277216975 >> v1:172.21.15.164:6789/0 conn(0x7fd3280cc330 legacy=0x7fd328072550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f53bd5bd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53b8154960 0x7f53b80c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.368+0000 7fb86effd640 1 --2- 172.21.15.139:0/2282560182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb8700b5280 0x7fb870162380 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb87006dee0 tx=0x7fb86402a260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add0c25b91cfc06e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7fb86e7fc640 1 -- 172.21.15.139:0/2282560182 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864037070 con 0x7fb8700b5280 2024-09-16T00:07:43.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7fb86e7fc640 1 -- 172.21.15.139:0/2282560182 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb864032440 con 0x7fb8700b5280 2024-09-16T00:07:43.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7fb86e7fc640 1 -- 172.21.15.139:0/2282560182 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864032810 con 0x7fb8700b5280 2024-09-16T00:07:43.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f068cbc4640 1 -- 172.21.15.139:0/3592172292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0680045120 msgr2=0x7f06800ddc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f068cbc4640 1 --2- 172.21.15.139:0/3592172292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0680045120 0x7f06800ddc80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f06800360c0 tx=0x7f067c00abe0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f53bd5bd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f53b80cd8f0 0x7f53b815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f53bd5bd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53b80c5660 con 0x7f53b80d0790 2024-09-16T00:07:43.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f53bd5bd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f53b80cd460 con 0x7f53b80cd8f0 2024-09-16T00:07:43.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f53bd5bd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f53b815b700 con 0x7f53b8154960 2024-09-16T00:07:43.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7fd327fff640 1 -- 172.21.15.139:0/3277216975 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3280c92a0 con 0x7fd3280c8b00 2024-09-16T00:07:43.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f53b6ffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53b8154960 0x7f53b80c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f97374c9640 1 -- 172.21.15.139:0/1293119401 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9730071700 msgr2=0x7f97301566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f97374c9640 1 --2- 172.21.15.139:0/1293119401 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9730071700 0x7f97301566f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9728009d90 tx=0x7f972802a610 comp rx=0 tx=0).stop 2024-09-16T00:07:43.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f53b77fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f53b80d0790 0x7f53b80cb0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45216/0 (socket says 172.21.15.139:45216) 2024-09-16T00:07:43.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f53b77fe640 1 -- 172.21.15.139:0/747591302 learned_addr learned my addr 172.21.15.139:0/747591302 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:43.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f53b67fc640 1 --2- 172.21.15.139:0/747591302 >> v2:172.21.15.139:3300/0 conn(0x7f53b80cd8f0 0x7f53b815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f7623fff640 1 -- 172.21.15.139:0/2221925479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f761c0d5850 msgr2=0x7f761c007730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f7623fff640 1 --2- 172.21.15.139:0/2221925479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f761c0d5850 0x7f761c007730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f76180097a0 tx=0x7f761802a230 comp rx=0 tx=0).stop 2024-09-16T00:07:43.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7fd32d274640 1 --2- 172.21.15.139:0/3277216975 >> v2:172.21.15.139:3300/0 conn(0x7fd328072c60 0x7fd328150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:43.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f97374c9640 1 -- 172.21.15.139:0/1293119401 shutdown_connections 2024-09-16T00:07:43.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f7623fff640 1 -- 172.21.15.139:0/2221925479 shutdown_connections 2024-09-16T00:07:43.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f7623fff640 1 --2- 172.21.15.139:0/2221925479 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f761c0d5850 0x7f761c007730 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f76180097a0 tx=0x7f761802a230 comp rx=0 tx=0).stop 2024-09-16T00:07:43.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f7623fff640 1 -- 172.21.15.139:0/2221925479 >> 172.21.15.139:0/2221925479 conn(0x7f761c04d830 msgr2=0x7f761c04dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f97374c9640 1 --2- 172.21.15.139:0/1293119401 >> v2:172.21.15.139:3300/0 conn(0x7f97300c1000 0x7f97300c23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f58eb59e640 1 -- 172.21.15.139:0/3494607290 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ec0b1d80 msgr2=0x7f58ec06cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f58eb59e640 1 --2- 172.21.15.139:0/3494607290 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ec0b1d80 0x7f58ec06cb30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009e50 tx=0x7f58e402a750 comp rx=0 tx=0).stop 2024-09-16T00:07:43.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f97374c9640 1 --2- 172.21.15.139:0/1293119401 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9730071700 0x7f97301566f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f53b5ffb640 1 -- 172.21.15.139:0/747591302 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2794743716 0 0) 0x7f53b80c5660 con 0x7f53b80d0790 2024-09-16T00:07:43.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f97374c9640 1 -- 172.21.15.139:0/1293119401 >> 172.21.15.139:0/1293119401 conn(0x7f97300c8ae0 msgr2=0x7f97300c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f97374c9640 1 -- 172.21.15.139:0/1293119401 shutdown_connections 2024-09-16T00:07:43.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f97374c9640 1 -- 172.21.15.139:0/1293119401 wait complete. 2024-09-16T00:07:43.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f58eb59e640 1 -- 172.21.15.139:0/3494607290 shutdown_connections 2024-09-16T00:07:43.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f58eb59e640 1 --2- 172.21.15.139:0/3494607290 >> v2:172.21.15.139:3300/0 conn(0x7f58ec15eb60 0x7f58ec15f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f58eb59e640 1 --2- 172.21.15.139:0/3494607290 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ec0b1d80 0x7f58ec06cb30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f58eb59e640 1 -- 172.21.15.139:0/3494607290 >> 172.21.15.139:0/3494607290 conn(0x7f58ec0c8a00 msgr2=0x7f58ec0c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7fc6ed2e5640 1 -- 172.21.15.139:0/2511155148 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4037070 con 0x7fc6e80cc330 2024-09-16T00:07:43.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7fc6ed2e5640 1 -- 172.21.15.139:0/2511155148 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e402ae50 con 0x7fc6e80cc330 2024-09-16T00:07:43.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7fc6ed2e5640 1 -- 172.21.15.139:0/2511155148 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4032860 con 0x7fc6e80cc330 2024-09-16T00:07:43.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f58eb59e640 1 -- 172.21.15.139:0/3494607290 shutdown_connections 2024-09-16T00:07:43.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f7623fff640 1 -- 172.21.15.139:0/2221925479 shutdown_connections 2024-09-16T00:07:43.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7fd327fff640 1 --2- 172.21.15.139:0/3277216975 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3280c8b00 0x7fd3281572f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd3100097a0 tx=0x7fd31002a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7942ce4e3ce392d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f068cbc4640 1 -- 172.21.15.139:0/3592172292 shutdown_connections 2024-09-16T00:07:43.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f068cbc4640 1 --2- 172.21.15.139:0/3592172292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0680045120 0x7f06800ddc80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f068cbc4640 1 --2- 172.21.15.139:0/3592172292 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f068004c0c0 0x7f06800de1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f068cbc4640 1 --2- 172.21.15.139:0/3592172292 >> v2:172.21.15.139:3300/0 conn(0x7f06800d4970 0x7f0680005a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f068cbc4640 1 -- 172.21.15.139:0/3592172292 >> 172.21.15.139:0/3592172292 conn(0x7f068004fc80 msgr2=0x7f0680050090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7fb875cd0640 1 -- 172.21.15.139:0/2282560182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb8700b5280 msgr2=0x7fb870162380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7fb875cd0640 1 --2- 172.21.15.139:0/2282560182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb8700b5280 0x7fb870162380 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb87006dee0 tx=0x7fb86402a260 comp rx=0 tx=0).stop 2024-09-16T00:07:43.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f58eb59e640 1 -- 172.21.15.139:0/3494607290 wait complete. 2024-09-16T00:07:43.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.369+0000 7f53b5ffb640 1 -- 172.21.15.139:0/747591302 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53a0003520 con 0x7f53b80d0790 2024-09-16T00:07:43.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f068cbc4640 1 -- 172.21.15.139:0/3592172292 shutdown_connections 2024-09-16T00:07:43.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f068cbc4640 1 -- 172.21.15.139:0/3592172292 wait complete. 2024-09-16T00:07:43.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7fb875cd0640 1 -- 172.21.15.139:0/2282560182 shutdown_connections 2024-09-16T00:07:43.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7fb875cd0640 1 --2- 172.21.15.139:0/2282560182 >> v2:172.21.15.139:3300/0 conn(0x7fb8700738c0 0x7fb8701628c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7fb875cd0640 1 --2- 172.21.15.139:0/2282560182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb8700b5280 0x7fb870162380 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7fb875cd0640 1 --2- 172.21.15.139:0/2282560182 >> v2:172.21.15.164:3300/0 conn(0x7fb87015fa70 0x7fb8700b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7fb875cd0640 1 -- 172.21.15.139:0/2282560182 >> 172.21.15.139:0/2282560182 conn(0x7fb8700d06a0 msgr2=0x7fb8700d0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7fd3277fe640 1 -- 172.21.15.139:0/3277216975 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310037070 con 0x7fd3280c8b00 2024-09-16T00:07:43.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7fd3277fe640 1 -- 172.21.15.139:0/3277216975 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd310032440 con 0x7fd3280c8b00 2024-09-16T00:07:43.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f53b6ffd640 1 -- 172.21.15.139:0/747591302 >> v2:172.21.15.139:3300/0 conn(0x7f53b80cd8f0 msgr2=0x7f53b815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:43.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f53b6ffd640 1 --2- 172.21.15.139:0/747591302 >> v2:172.21.15.139:3300/0 conn(0x7f53b80cd8f0 0x7f53b815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f53b6ffd640 1 -- 172.21.15.139:0/747591302 >> v1:172.21.15.139:6789/0 conn(0x7f53b80d0790 legacy=0x7f53b80cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f53b6ffd640 1 -- 172.21.15.139:0/747591302 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b80cc570 con 0x7f53b8154960 2024-09-16T00:07:43.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7fd3277fe640 1 -- 172.21.15.139:0/3277216975 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310037070 con 0x7fd3280c8b00 2024-09-16T00:07:43.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7f53b6ffd640 1 --2- 172.21.15.139:0/747591302 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53b8154960 0x7f53b80c5120 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0097d0 tx=0x7f53ac02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e815a5bc2a1f602 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.370+0000 7f7623fff640 1 -- 172.21.15.139:0/2221925479 wait complete. 2024-09-16T00:07:43.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7f53b5ffb640 1 -- 172.21.15.139:0/747591302 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac037070 con 0x7f53b8154960 2024-09-16T00:07:43.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7fc6f054b640 1 -- 172.21.15.139:0/2511155148 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6e80cc330 msgr2=0x7fc6e81572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7fc6f054b640 1 --2- 172.21.15.139:0/2511155148 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6e80cc330 0x7fc6e81572f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009870 tx=0x7fc6e402a300 comp rx=0 tx=0).stop 2024-09-16T00:07:43.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7f53b5ffb640 1 -- 172.21.15.139:0/747591302 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ac032440 con 0x7f53b8154960 2024-09-16T00:07:43.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7f53b5ffb640 1 -- 172.21.15.139:0/747591302 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac032810 con 0x7f53b8154960 2024-09-16T00:07:43.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7fc6f054b640 1 -- 172.21.15.139:0/2511155148 shutdown_connections 2024-09-16T00:07:43.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7fc6f054b640 1 --2- 172.21.15.139:0/2511155148 >> v2:172.21.15.164:3300/0 conn(0x7fc6e806de20 0x7fc6e8072550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7fc6f054b640 1 --2- 172.21.15.139:0/2511155148 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6e80cc330 0x7fc6e81572f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7fc6f054b640 1 -- 172.21.15.139:0/2511155148 >> 172.21.15.139:0/2511155148 conn(0x7fc6e806da40 msgr2=0x7fc6e806f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7fc6f054b640 1 -- 172.21.15.139:0/2511155148 shutdown_connections 2024-09-16T00:07:43.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7fc6f054b640 1 -- 172.21.15.139:0/2511155148 wait complete. 2024-09-16T00:07:43.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7fd32ecd7640 1 -- 172.21.15.139:0/3277216975 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3280c8b00 msgr2=0x7fd3281572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7fd32ecd7640 1 --2- 172.21.15.139:0/3277216975 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3280c8b00 0x7fd3281572f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd3100097a0 tx=0x7fd31002a230 comp rx=0 tx=0).stop 2024-09-16T00:07:43.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7f97374c9640 1 Processor -- start 2024-09-16T00:07:43.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7f97374c9640 1 -- start start 2024-09-16T00:07:43.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f97374c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9730071700 0x7f9730138bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.372+0000 7f7623fff640 1 Processor -- start 2024-09-16T00:07:43.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.372+0000 7f7623fff640 1 -- start start 2024-09-16T00:07:43.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7fb875cd0640 1 -- 172.21.15.139:0/2282560182 shutdown_connections 2024-09-16T00:07:43.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.372+0000 7f58eb59e640 1 Processor -- start 2024-09-16T00:07:43.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.372+0000 7f58eb59e640 1 -- start start 2024-09-16T00:07:43.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.372+0000 7f53bd5bd640 1 -- 172.21.15.139:0/747591302 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53b8154960 msgr2=0x7f53b80c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.372+0000 7f53bd5bd640 1 --2- 172.21.15.139:0/747591302 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53b8154960 0x7f53b80c5120 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0097d0 tx=0x7f53ac02a1c0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.372+0000 7f53bd5bd640 1 -- 172.21.15.139:0/747591302 shutdown_connections 2024-09-16T00:07:43.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.372+0000 7f53bd5bd640 1 --2- 172.21.15.139:0/747591302 >> v2:172.21.15.139:3300/0 conn(0x7f53b80cd8f0 0x7f53b815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.372+0000 7f53bd5bd640 1 --2- 172.21.15.139:0/747591302 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53b8154960 0x7f53b80c5120 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.372+0000 7f53bd5bd640 1 -- 172.21.15.139:0/747591302 >> 172.21.15.139:0/747591302 conn(0x7f53b806c850 msgr2=0x7f53b806cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7fd32ecd7640 1 -- 172.21.15.139:0/3277216975 shutdown_connections 2024-09-16T00:07:43.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7fd32ecd7640 1 --2- 172.21.15.139:0/3277216975 >> v2:172.21.15.139:3300/0 conn(0x7fd328072c60 0x7fd328150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7fd32ecd7640 1 --2- 172.21.15.139:0/3277216975 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3280c8b00 0x7fd3281572f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.371+0000 7fd32ecd7640 1 -- 172.21.15.139:0/3277216975 >> 172.21.15.139:0/3277216975 conn(0x7fd32806da40 msgr2=0x7fd32806f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.372+0000 7fd32ecd7640 1 -- 172.21.15.139:0/3277216975 shutdown_connections 2024-09-16T00:07:43.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.372+0000 7f068cbc4640 1 Processor -- start 2024-09-16T00:07:43.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.372+0000 7f068cbc4640 1 -- start start 2024-09-16T00:07:43.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f58eb59e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ec06d070 0x7f58ec1f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f58eb59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58ec0b1d80 0x7f58ec1e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f58eb59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ec15eb60 0x7f58ec1e8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f58eb59e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec072840 con 0x7f58ec15eb60 2024-09-16T00:07:43.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f58eb59e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec072560 con 0x7f58ec0b1d80 2024-09-16T00:07:43.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f58eb59e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f58ec072b20 con 0x7f58ec06d070 2024-09-16T00:07:43.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f53bd5bd640 1 -- 172.21.15.139:0/747591302 shutdown_connections 2024-09-16T00:07:43.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.372+0000 7fb875cd0640 1 -- 172.21.15.139:0/2282560182 wait complete. 2024-09-16T00:07:43.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7fb875cd0640 1 Processor -- start 2024-09-16T00:07:43.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f9735265640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9730071700 0x7f9730138bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f9735265640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9730071700 0x7f9730138bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40430/0 (socket says 172.21.15.139:40430) 2024-09-16T00:07:43.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.372+0000 7fd32ecd7640 1 -- 172.21.15.139:0/3277216975 wait complete. 2024-09-16T00:07:43.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f58ead9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ec15eb60 0x7f58ec1e8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f58ead9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ec15eb60 0x7f58ec1e8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40440/0 (socket says 172.21.15.139:40440) 2024-09-16T00:07:43.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f58ead9d640 1 -- 172.21.15.139:0/2611075467 learned_addr learned my addr 172.21.15.139:0/2611075467 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7fd32ecd7640 1 Processor -- start 2024-09-16T00:07:43.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f58ea59c640 1 --2- 172.21.15.139:0/2611075467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ec06d070 0x7f58ec1f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f9735265640 1 -- 172.21.15.139:0/682029708 learned_addr learned my addr 172.21.15.139:0/682029708 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f97374c9640 1 --2- 172.21.15.139:0/682029708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97300c1000 0x7f97301510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f97374c9640 1 --2- 172.21.15.139:0/682029708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9730158b20 0x7f9730151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f068cbc4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0680045120 0x7f06800cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f068cbc4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068004c0c0 0x7f06800da490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f068cbc4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06800d4970 0x7f06800da9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f068cbc4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f06800467a0 con 0x7f068004c0c0 2024-09-16T00:07:43.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f068cbc4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f06800464c0 con 0x7f06800d4970 2024-09-16T00:07:43.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f068cbc4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0680005f70 con 0x7f0680045120 2024-09-16T00:07:43.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f068659c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0680045120 0x7f06800cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f068659c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0680045120 0x7f06800cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:55022/0 (socket says 172.21.15.139:55022) 2024-09-16T00:07:43.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f068659c640 1 -- 172.21.15.139:0/3790464205 learned_addr learned my addr 172.21.15.139:0/3790464205 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f9734a64640 1 --2- 172.21.15.139:0/682029708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97300c1000 0x7f97301510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f58ead9d640 1 -- 172.21.15.139:0/2611075467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ec06d070 msgr2=0x7f58ec1f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f58ead9d640 1 --2- 172.21.15.139:0/2611075467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ec06d070 0x7f58ec1f4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7fd32ecd7640 1 -- start start 2024-09-16T00:07:43.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f97374c9640 1 -- 172.21.15.139:0/682029708 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f97300c7660 con 0x7f9730071700 2024-09-16T00:07:43.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f0685d9b640 1 --2- 172.21.15.139:0/3790464205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068004c0c0 0x7f06800da490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f97374c9640 1 -- 172.21.15.139:0/682029708 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9730156c30 con 0x7f97300c1000 2024-09-16T00:07:43.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f0686d9d640 1 --2- 172.21.15.139:0/3790464205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06800d4970 0x7f06800da9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f97374c9640 1 -- 172.21.15.139:0/682029708 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f97300727f0 con 0x7f9730158b20 2024-09-16T00:07:43.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f068659c640 1 -- 172.21.15.139:0/3790464205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06800d4970 msgr2=0x7f06800da9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f58ead9d640 1 -- 172.21.15.139:0/2611075467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58ec0b1d80 msgr2=0x7f58ec1e81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f58ead9d640 1 --2- 172.21.15.139:0/2611075467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58ec0b1d80 0x7f58ec1e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f58ead9d640 1 -- 172.21.15.139:0/2611075467 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ec1554d0 con 0x7f58ec15eb60 2024-09-16T00:07:43.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f58ea59c640 1 --2- 172.21.15.139:0/2611075467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ec06d070 0x7f58ec1f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:43.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f58ead9d640 1 --2- 172.21.15.139:0/2611075467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ec15eb60 0x7f58ec1e8710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00b410 tx=0x7f58dc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7fc6f054b640 1 Processor -- start 2024-09-16T00:07:43.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7fc6f054b640 1 -- start start 2024-09-16T00:07:43.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7fc6f054b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6e806de20 0x7fc6e80cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7fc6f054b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6e80c8ad0 0x7fc6e81473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7fc6f054b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6e80cc330 0x7fc6e81478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7fc6f054b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e8150d40 con 0x7fc6e806de20 2024-09-16T00:07:43.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7fc6f054b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e80c9860 con 0x7fc6e80c8ad0 2024-09-16T00:07:43.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7fc6f054b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc6e8150ec0 con 0x7fc6e80cc330 2024-09-16T00:07:43.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f068659c640 1 --2- 172.21.15.139:0/3790464205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f06800d4970 0x7f06800da9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f068659c640 1 -- 172.21.15.139:0/3790464205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068004c0c0 msgr2=0x7f06800da490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f068659c640 1 --2- 172.21.15.139:0/3790464205 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f068004c0c0 0x7f06800da490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7fc6eeae8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6e80cc330 0x7fc6e81478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f9735a66640 1 --2- 172.21.15.139:0/682029708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9730158b20 0x7f9730151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7fc6eeae8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6e80cc330 0x7fc6e81478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:55050/0 (socket says 172.21.15.139:55050) 2024-09-16T00:07:43.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7fc6eeae8640 1 -- 172.21.15.139:0/3524909242 learned_addr learned my addr 172.21.15.139:0/3524909242 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7f068659c640 1 -- 172.21.15.139:0/3790464205 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068004ed60 con 0x7f0680045120 2024-09-16T00:07:43.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fc6edae6640 1 --2- 172.21.15.139:0/3524909242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6e80c8ad0 0x7fc6e81473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.373+0000 7f53bd5bd640 1 -- 172.21.15.139:0/747591302 wait complete. 2024-09-16T00:07:43.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.374+0000 7fb875cd0640 1 -- start start 2024-09-16T00:07:43.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fb875cd0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb8700738c0 0x7fb8701ea690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fb875cd0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8700b5280 0x7fb8701fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fb875cd0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb87015fa70 0x7fb8701fdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f7623fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f761c047a80 0x7f761c0d5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f7623fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f761c04d310 0x7f761c185140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f7623fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f761c0514f0 0x7f761c185680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f7623fff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f761c047560 con 0x7f761c0514f0 2024-09-16T00:07:43.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f7623fff640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f761c047280 con 0x7f761c047a80 2024-09-16T00:07:43.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fb875cd0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb8700b4340 con 0x7fb8700b5280 2024-09-16T00:07:43.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fb875cd0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb8700b60e0 con 0x7fb87015fa70 2024-09-16T00:07:43.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fb875cd0640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb8700b44c0 con 0x7fb8700738c0 2024-09-16T00:07:43.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f7623fff640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f761c0468c0 con 0x7f761c04d310 2024-09-16T00:07:43.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f9735265640 1 -- 172.21.15.139:0/682029708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9730158b20 msgr2=0x7f9730151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f9735265640 1 --2- 172.21.15.139:0/682029708 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9730158b20 0x7f9730151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f9735265640 1 -- 172.21.15.139:0/682029708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97300c1000 msgr2=0x7f97301510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f9735265640 1 --2- 172.21.15.139:0/682029708 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f97300c1000 0x7f97301510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f7622ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f761c047a80 0x7f761c0d5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f58daffd640 1 -- 172.21.15.139:0/2611075467 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc015020 con 0x7f58ec15eb60 2024-09-16T00:07:43.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f7622ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f761c047a80 0x7f761c0d5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39476/0 (socket says 172.21.15.139:39476) 2024-09-16T00:07:43.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f7622ffd640 1 -- 172.21.15.139:0/4073981168 learned_addr learned my addr 172.21.15.139:0/4073981168 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f9735265640 1 -- 172.21.15.139:0/682029708 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9730151ef0 con 0x7f9730071700 2024-09-16T00:07:43.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f58eb59e640 1 -- 172.21.15.139:0/2611075467 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ec1547e0 con 0x7f58ec15eb60 2024-09-16T00:07:43.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fb86effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8700b5280 0x7fb8701fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fb86effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8700b5280 0x7fb8701fd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40456/0 (socket says 172.21.15.139:40456) 2024-09-16T00:07:43.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fb86effd640 1 -- 172.21.15.139:0/512074014 learned_addr learned my addr 172.21.15.139:0/512074014 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fd32ecd7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd328072c60 0x7fd32814c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fd32ecd7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3280c8b00 0x7fd3281558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fd32ecd7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3280cc330 0x7fd3281fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fd32ecd7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd32806de20 con 0x7fd3280cc330 2024-09-16T00:07:43.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fd32ecd7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd3280c9860 con 0x7fd328072c60 2024-09-16T00:07:43.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fd32ecd7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd32806dfa0 con 0x7fd3280c8b00 2024-09-16T00:07:43.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fc6eeae8640 1 -- 172.21.15.139:0/3524909242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6e80c8ad0 msgr2=0x7fc6e81473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fc6eeae8640 1 --2- 172.21.15.139:0/3524909242 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc6e80c8ad0 0x7fc6e81473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fc6eeae8640 1 -- 172.21.15.139:0/3524909242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6e806de20 msgr2=0x7fc6e80cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fc6eeae8640 1 --2- 172.21.15.139:0/3524909242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6e806de20 0x7fc6e80cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fc6eeae8640 1 -- 172.21.15.139:0/3524909242 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e808cbc0 con 0x7fc6e80cc330 2024-09-16T00:07:43.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f58eb59e640 1 -- 172.21.15.139:0/2611075467 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ec1ea210 con 0x7f58ec15eb60 2024-09-16T00:07:43.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f76227fc640 1 --2- 172.21.15.139:0/4073981168 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f761c04d310 0x7f761c185140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fd32d274640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3280cc330 0x7fd3281fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f068659c640 1 --2- 172.21.15.139:0/3790464205 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0680045120 0x7f06800cd550 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f067800eb40 tx=0x7f067800d350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f58daffd640 1 -- 172.21.15.139:0/2611075467 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58dc011830 con 0x7f58ec15eb60 2024-09-16T00:07:43.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7f58daffd640 1 -- 172.21.15.139:0/2611075467 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc015020 con 0x7f58ec15eb60 2024-09-16T00:07:43.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fb86effd640 1 -- 172.21.15.139:0/512074014 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb8700738c0 msgr2=0x7fb8701ea690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fb86effd640 1 --2- 172.21.15.139:0/512074014 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb8700738c0 0x7fb8701ea690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fb86effd640 1 -- 172.21.15.139:0/512074014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb87015fa70 msgr2=0x7fb8701fdd90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:07:43.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fb86effd640 1 --2- 172.21.15.139:0/512074014 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb87015fa70 0x7fb8701fdd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fb86effd640 1 -- 172.21.15.139:0/512074014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb870201490 con 0x7fb8700b5280 2024-09-16T00:07:43.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fd32ca73640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd328072c60 0x7fd32814c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fd32d274640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3280cc330 0x7fd3281fcdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40458/0 (socket says 172.21.15.139:40458) 2024-09-16T00:07:43.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.375+0000 7fd32d274640 1 -- 172.21.15.139:0/3144167841 learned_addr learned my addr 172.21.15.139:0/3144167841 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7fd327fff640 1 --2- 172.21.15.139:0/3144167841 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3280c8b00 0x7fd3281558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7f0676ffd640 1 -- 172.21.15.139:0/3790464205 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678010440 con 0x7f0680045120 2024-09-16T00:07:43.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7f7622ffd640 1 -- 172.21.15.139:0/4073981168 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f761c04d310 msgr2=0x7f761c185140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7f7622ffd640 1 --2- 172.21.15.139:0/4073981168 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f761c04d310 0x7f761c185140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7f7622ffd640 1 -- 172.21.15.139:0/4073981168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f761c0514f0 msgr2=0x7f761c185680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:07:43.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7f0676ffd640 1 -- 172.21.15.139:0/3790464205 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06780105e0 con 0x7f0680045120 2024-09-16T00:07:43.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7f7622ffd640 1 --2- 172.21.15.139:0/4073981168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f761c0514f0 0x7f761c185680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7f7622ffd640 1 -- 172.21.15.139:0/4073981168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c0d8210 con 0x7f761c047a80 2024-09-16T00:07:43.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7fb86effd640 1 --2- 172.21.15.139:0/512074014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8700b5280 0x7fb8701fd850 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb864009960 tx=0x7fb8640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7f0676ffd640 1 -- 172.21.15.139:0/3790464205 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06780109d0 con 0x7f0680045120 2024-09-16T00:07:43.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7f068cbc4640 1 -- 172.21.15.139:0/3790464205 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0680184e40 con 0x7f0680045120 2024-09-16T00:07:43.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7f068cbc4640 1 -- 172.21.15.139:0/3790464205 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f06801850e0 con 0x7f0680045120 2024-09-16T00:07:43.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7fd32ca73640 1 -- 172.21.15.139:0/3144167841 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3280c8b00 msgr2=0x7fd3281558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7fd32ca73640 1 --2- 172.21.15.139:0/3144167841 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd3280c8b00 0x7fd3281558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7fd32ca73640 1 -- 172.21.15.139:0/3144167841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3280cc330 msgr2=0x7fd3281fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7fd32ca73640 1 --2- 172.21.15.139:0/3144167841 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3280cc330 0x7fd3281fcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7fd32ca73640 1 -- 172.21.15.139:0/3144167841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32814b5c0 con 0x7fd328072c60 2024-09-16T00:07:43.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7fc6eeae8640 1 --2- 172.21.15.139:0/3524909242 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6e80cc330 0x7fc6e81478e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000c740 tx=0x7fc6e000cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7f53bd5bd640 1 Processor -- start 2024-09-16T00:07:43.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7f9735265640 1 --2- 172.21.15.139:0/682029708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9730071700 0x7f9730138bf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9728009d90 tx=0x7f9728004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7f7622ffd640 1 --2- 172.21.15.139:0/4073981168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f761c047a80 0x7f761c0d5d60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f761802a7a0 tx=0x7f76180046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.376+0000 7f58eb59e640 1 -- 172.21.15.139:0/2611075467 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f58d0001030 con 0x7f58ec15eb60 2024-09-16T00:07:43.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7fb857fff640 1 -- 172.21.15.139:0/512074014 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864041070 con 0x7fb8700b5280 2024-09-16T00:07:43.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7fc6deffd640 1 -- 172.21.15.139:0/3524909242 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0016020 con 0x7fc6e80cc330 2024-09-16T00:07:43.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7fc6deffd640 1 -- 172.21.15.139:0/3524909242 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e000ce50 con 0x7fc6e80cc330 2024-09-16T00:07:43.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7fc6deffd640 1 -- 172.21.15.139:0/3524909242 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0003970 con 0x7fc6e80cc330 2024-09-16T00:07:43.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7f53bd5bd640 1 -- start start 2024-09-16T00:07:43.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7fb875cd0640 1 -- 172.21.15.139:0/512074014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb870162270 con 0x7fb8700b5280 2024-09-16T00:07:43.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7fd32ca73640 1 --2- 172.21.15.139:0/3144167841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd328072c60 0x7fd32814c060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd31800c7b0 tx=0x7fd31800cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7f760b7fe640 1 -- 172.21.15.139:0/4073981168 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618037040 con 0x7f761c047a80 2024-09-16T00:07:43.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7f760b7fe640 1 -- 172.21.15.139:0/4073981168 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76180304c0 con 0x7f761c047a80 2024-09-16T00:07:43.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7fb875cd0640 1 -- 172.21.15.139:0/512074014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb870162830 con 0x7fb8700b5280 2024-09-16T00:07:43.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7fc6f054b640 1 -- 172.21.15.139:0/3524909242 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e81488f0 con 0x7fc6e80cc330 2024-09-16T00:07:43.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7fc6f054b640 1 -- 172.21.15.139:0/3524909242 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e8148eb0 con 0x7fc6e80cc330 2024-09-16T00:07:43.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7f7623fff640 1 -- 172.21.15.139:0/4073981168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761c0d84f0 con 0x7f761c047a80 2024-09-16T00:07:43.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7fb857fff640 1 -- 172.21.15.139:0/512074014 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb86403c930 con 0x7fb8700b5280 2024-09-16T00:07:43.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7f760b7fe640 1 -- 172.21.15.139:0/4073981168 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618030890 con 0x7f761c047a80 2024-09-16T00:07:43.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7f7623fff640 1 -- 172.21.15.139:0/4073981168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761c0d62a0 con 0x7f761c047a80 2024-09-16T00:07:43.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.377+0000 7fb857fff640 1 -- 172.21.15.139:0/512074014 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864037040 con 0x7fb8700b5280 2024-09-16T00:07:43.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.378+0000 7fd3257fa640 1 -- 172.21.15.139:0/3144167841 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318018070 con 0x7fd328072c60 2024-09-16T00:07:43.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.378+0000 7fd3257fa640 1 -- 172.21.15.139:0/3144167841 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31800a640 con 0x7fd328072c60 2024-09-16T00:07:43.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.378+0000 7f53bd5bd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b80cd8f0 0x7f53b81f0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.378+0000 7f9725ffb640 1 -- 172.21.15.139:0/682029708 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728004240 con 0x7f9730071700 2024-09-16T00:07:43.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.378+0000 7f97374c9640 1 -- 172.21.15.139:0/682029708 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9730152590 con 0x7f9730071700 2024-09-16T00:07:43.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.378+0000 7f97374c9640 1 -- 172.21.15.139:0/682029708 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9730152b50 con 0x7f9730071700 2024-09-16T00:07:43.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.378+0000 7f58daffd640 1 -- 172.21.15.139:0/2611075467 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f58dc018020 con 0x7f58ec15eb60 2024-09-16T00:07:43.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f58daffd640 1 --2- 172.21.15.139:0/2611075467 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f58cc06c4b0 0x7f58cc06e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f53bd5bd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53b80d0790 0x7f53b81e0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f53bd5bd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53b8154960 0x7f53b81585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f53bd5bd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53b80cce10 con 0x7f53b80d0790 2024-09-16T00:07:43.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f53bd5bd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f53b80ccb30 con 0x7f53b80cd8f0 2024-09-16T00:07:43.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f53bd5bd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f53b80cd0f0 con 0x7f53b8154960 2024-09-16T00:07:43.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.378+0000 7fd3257fa640 1 -- 172.21.15.139:0/3144167841 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31800ce60 con 0x7fd328072c60 2024-09-16T00:07:43.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7fd32ecd7640 1 -- 172.21.15.139:0/3144167841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3281fd710 con 0x7fd328072c60 2024-09-16T00:07:43.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7fd32ecd7640 1 -- 172.21.15.139:0/3144167841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32814b860 con 0x7fd328072c60 2024-09-16T00:07:43.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f53b6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b80cd8f0 0x7f53b81f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f53b77fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53b8154960 0x7f53b81585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f53b6ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b80cd8f0 0x7f53b81f0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39502/0 (socket says 172.21.15.139:39502) 2024-09-16T00:07:43.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f53b6ffd640 1 -- 172.21.15.139:0/3902979575 learned_addr learned my addr 172.21.15.139:0/3902979575 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f068cbc4640 1 -- 172.21.15.139:0/3790464205 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f06800b1100 con 0x7f0680045120 2024-09-16T00:07:43.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7fc6deffd640 1 -- 172.21.15.139:0/3524909242 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc6e0003b10 con 0x7fc6e80cc330 2024-09-16T00:07:43.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f0676ffd640 1 -- 172.21.15.139:0/3790464205 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0678010b70 con 0x7f0680045120 2024-09-16T00:07:43.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f9725ffb640 1 -- 172.21.15.139:0/682029708 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9728004400 con 0x7f9730071700 2024-09-16T00:07:43.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f9725ffb640 1 -- 172.21.15.139:0/682029708 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97280047f0 con 0x7f9730071700 2024-09-16T00:07:43.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f53b6ffd640 1 -- 172.21.15.139:0/3902979575 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53b8154960 msgr2=0x7f53b81585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f53b6ffd640 1 --2- 172.21.15.139:0/3902979575 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53b8154960 0x7f53b81585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f53b6ffd640 1 -- 172.21.15.139:0/3902979575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53b80d0790 msgr2=0x7f53b81e0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.380+0000 7f971b7fe640 1 -- 172.21.15.139:0/682029708 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9730152340 con 0x7f9730071700 2024-09-16T00:07:43.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.380+0000 7fb875cd0640 1 -- 172.21.15.139:0/512074014 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb850001030 con 0x7fb8700b5280 2024-09-16T00:07:43.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f53b6ffd640 1 --2- 172.21.15.139:0/3902979575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53b80d0790 0x7f53b81e0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.379+0000 7f53b6ffd640 1 -- 172.21.15.139:0/3902979575 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b81e0eb0 con 0x7f53b80cd8f0 2024-09-16T00:07:43.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.380+0000 7fd32ecd7640 1 -- 172.21.15.139:0/3144167841 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd3280cc710 con 0x7fd328072c60 2024-09-16T00:07:43.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.380+0000 7f53b6ffd640 1 --2- 172.21.15.139:0/3902979575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b80cd8f0 0x7f53b81f0450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0098f0 tx=0x7f53ac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.380+0000 7f0676ffd640 1 --2- 172.21.15.139:0/3790464205 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f066806c380 0x7f066806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.380+0000 7fc6deffd640 1 --2- 172.21.15.139:0/3524909242 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc6cc06c380 0x7fc6cc06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.380+0000 7f7623fff640 1 -- 172.21.15.139:0/4073981168 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f761c0d8d60 con 0x7f761c047a80 2024-09-16T00:07:43.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.380+0000 7fc6deffd640 1 -- 172.21.15.139:0/3524909242 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc6e008afd0 con 0x7fc6e80cc330 2024-09-16T00:07:43.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.380+0000 7f539f7fe640 1 -- 172.21.15.139:0/3902979575 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac040020 con 0x7f53b80cd8f0 2024-09-16T00:07:43.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.380+0000 7f53bd5bd640 1 -- 172.21.15.139:0/3902979575 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b81e1190 con 0x7f53b80cd8f0 2024-09-16T00:07:43.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.380+0000 7f53bd5bd640 1 -- 172.21.15.139:0/3902979575 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b81e1750 con 0x7f53b80cd8f0 2024-09-16T00:07:43.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.380+0000 7f539f7fe640 1 -- 172.21.15.139:0/3902979575 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ac038040 con 0x7f53b80cd8f0 2024-09-16T00:07:43.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.381+0000 7f539f7fe640 1 -- 172.21.15.139:0/3902979575 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac037070 con 0x7f53b80cd8f0 2024-09-16T00:07:43.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.381+0000 7fc6ee2e7640 1 --2- 172.21.15.139:0/3524909242 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc6cc06c380 0x7fc6cc06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.381+0000 7f760b7fe640 1 -- 172.21.15.139:0/4073981168 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7618044020 con 0x7f761c047a80 2024-09-16T00:07:43.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.381+0000 7fd3257fa640 1 -- 172.21.15.139:0/3144167841 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd31800aa30 con 0x7fd328072c60 2024-09-16T00:07:43.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.381+0000 7fc6f054b640 1 --2- 172.21.15.139:0/3524909242 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fc6e8153f90 0x7fc6e8152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.381+0000 7fc6f054b640 1 -- 172.21.15.139:0/3524909242 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc6e80cc710 con 0x7fc6e8153f90 2024-09-16T00:07:43.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.381+0000 7f53bd5bd640 1 -- 172.21.15.139:0/3902979575 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f53b81e1e10 con 0x7f53b80cd8f0 2024-09-16T00:07:43.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7fc6ee2e7640 1 --2- 172.21.15.139:0/3524909242 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc6cc06c380 0x7fc6cc06e840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4034000 tx=0x7fc6e402c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7fd3257fa640 1 --2- 172.21.15.139:0/3144167841 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd30806c2b0 0x7fd30806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7f760b7fe640 1 --2- 172.21.15.139:0/4073981168 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f761006c380 0x7f761006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7fd3257fa640 1 -- 172.21.15.139:0/3144167841 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd31808ab60 con 0x7fd328072c60 2024-09-16T00:07:43.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7fb857fff640 1 -- 172.21.15.139:0/512074014 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb864044070 con 0x7fb8700b5280 2024-09-16T00:07:43.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7fd327fff640 1 --2- 172.21.15.139:0/3144167841 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd30806c2b0 0x7fd30806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7f9725ffb640 1 -- 172.21.15.139:0/682029708 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f972804c420 con 0x7f9730071700 2024-09-16T00:07:43.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7f58daffd640 1 -- 172.21.15.139:0/2611075467 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f58dc0576d0 con 0x7f58ec15eb60 2024-09-16T00:07:43.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7f58ea59c640 1 --2- 172.21.15.139:0/2611075467 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f58cc06c4b0 0x7f58cc06e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7f58daffd640 1 --2- 172.21.15.139:0/2611075467 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f58cc075320 0x7f58cc0777c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7f58daffd640 1 -- 172.21.15.139:0/2611075467 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f58cc077e90 con 0x7f58cc075320 2024-09-16T00:07:43.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7f58daffd640 1 -- 172.21.15.139:0/2611075467 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f58dc056060 con 0x7f58ec15eb60 2024-09-16T00:07:43.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7fd3257fa640 1 --2- 172.21.15.139:0/3144167841 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd308074530 0x7fd308076950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7fc6edae6640 1 --2- 172.21.15.139:0/3524909242 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fc6e8153f90 0x7fc6e8152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7fd3257fa640 1 -- 172.21.15.139:0/3144167841 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd308077000 con 0x7fd308074530 2024-09-16T00:07:43.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7fd3257fa640 1 -- 172.21.15.139:0/3144167841 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd31808af50 con 0x7fd328072c60 2024-09-16T00:07:43.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7f539f7fe640 1 -- 172.21.15.139:0/3902979575 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f53ac043020 con 0x7f53b80cd8f0 2024-09-16T00:07:43.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7fb857fff640 1 --2- 172.21.15.139:0/512074014 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb85c06c2e0 0x7fb85c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7f76227fc640 1 --2- 172.21.15.139:0/4073981168 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f761006c380 0x7f761006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7fb857fff640 1 -- 172.21.15.139:0/512074014 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb8640ac1d0 con 0x7fb8700b5280 2024-09-16T00:07:43.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f58e9d9b640 1 --2- 172.21.15.139:0/2611075467 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f58cc075320 0x7f58cc0777c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.382+0000 7f0676ffd640 1 -- 172.21.15.139:0/3790464205 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f067808bd50 con 0x7f0680045120 2024-09-16T00:07:43.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f0676ffd640 1 --2- 172.21.15.139:0/3790464205 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0668074600 0x7f0668076a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f0676ffd640 1 -- 172.21.15.139:0/3790464205 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f06680770d0 con 0x7f0668074600 2024-09-16T00:07:43.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f0676ffd640 1 -- 172.21.15.139:0/3790464205 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f067808e3b0 con 0x7f0680045120 2024-09-16T00:07:43.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7fd327fff640 1 --2- 172.21.15.139:0/3144167841 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd30806c2b0 0x7fd30806e770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd310034000 tx=0x7fd31002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f9725ffb640 1 --2- 172.21.15.139:0/682029708 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f971406c3b0 0x7f971406e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f76227fc640 1 --2- 172.21.15.139:0/4073981168 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f761006c380 0x7f761006e840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7624000b60 tx=0x7f762406e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f760b7fe640 1 -- 172.21.15.139:0/4073981168 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f76180ab3a0 con 0x7f761c047a80 2024-09-16T00:07:43.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7fb857fff640 1 --2- 172.21.15.139:0/512074014 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fb85c074560 0x7fb85c076980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7fb857fff640 1 -- 172.21.15.139:0/512074014 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb864034ec0 con 0x7fb85c074560 2024-09-16T00:07:43.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7fd32d274640 1 --2- 172.21.15.139:0/3144167841 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd308074530 0x7fd308076950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7fb857fff640 1 -- 172.21.15.139:0/512074014 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fb8640b3050 con 0x7fb8700b5280 2024-09-16T00:07:43.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f0685d9b640 1 --2- 172.21.15.139:0/3790464205 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f066806c380 0x7f066806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f0686d9d640 1 --2- 172.21.15.139:0/3790464205 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0668074600 0x7f0668076a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f760b7fe640 1 --2- 172.21.15.139:0/4073981168 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f7610074580 0x7f76100769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7fb86ffff640 1 --2- 172.21.15.139:0/512074014 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fb85c074560 0x7fb85c076980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f539f7fe640 1 --2- 172.21.15.139:0/3902979575 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f53a406c3b0 0x7f53a406e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f760b7fe640 1 -- 172.21.15.139:0/4073981168 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7618032440 con 0x7f7610074580 2024-09-16T00:07:43.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f58e9d9b640 1 --2- 172.21.15.139:0/2611075467 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f58cc075320 0x7f58cc0777c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f760b7fe640 1 -- 172.21.15.139:0/4073981168 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f76180b2050 con 0x7f761c047a80 2024-09-16T00:07:43.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f76237fe640 1 --2- 172.21.15.139:0/4073981168 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f7610074580 0x7f76100769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7fb86f7fe640 1 --2- 172.21.15.139:0/512074014 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb85c06c2e0 0x7fb85c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f539f7fe640 1 -- 172.21.15.139:0/3902979575 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f53ac0ab9c0 con 0x7f53b80cd8f0 2024-09-16T00:07:43.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.384+0000 7f539f7fe640 1 --2- 172.21.15.139:0/3902979575 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f53a4074630 0x7f53a4076a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.384+0000 7f539f7fe640 1 -- 172.21.15.139:0/3902979575 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53ac02be50 con 0x7f53a4074630 2024-09-16T00:07:43.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.384+0000 7f539f7fe640 1 -- 172.21.15.139:0/3902979575 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f53ac0b02a0 con 0x7f53b80cd8f0 2024-09-16T00:07:43.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f9725ffb640 1 -- 172.21.15.139:0/682029708 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f97280ac650 con 0x7f9730071700 2024-09-16T00:07:43.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f9725ffb640 1 --2- 172.21.15.139:0/682029708 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f97140745d0 0x7f97140769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f9725ffb640 1 -- 172.21.15.139:0/682029708 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f97140770a0 con 0x7f97140745d0 2024-09-16T00:07:43.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f9725ffb640 1 -- 172.21.15.139:0/682029708 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f97280aca40 con 0x7f9730071700 2024-09-16T00:07:43.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.383+0000 7f9734a64640 1 --2- 172.21.15.139:0/682029708 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f971406c3b0 0x7f971406e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.384+0000 7fd32d274640 1 --2- 172.21.15.139:0/3144167841 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd308074530 0x7fd308076950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.384+0000 7f9735a66640 1 --2- 172.21.15.139:0/682029708 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f97140745d0 0x7f97140769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.384+0000 7fb86ffff640 1 --2- 172.21.15.139:0/512074014 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fb85c074560 0x7fb85c076980 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.384+0000 7f53b67fc640 1 --2- 172.21.15.139:0/3902979575 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f53a406c3b0 0x7f53a406e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.384+0000 7f9734a64640 1 --2- 172.21.15.139:0/682029708 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f971406c3b0 0x7f971406e870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f972c009630 tx=0x7f972c009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.384+0000 7f0685d9b640 1 --2- 172.21.15.139:0/3790464205 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f066806c380 0x7f066806e840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f067c037000 tx=0x7f067c00d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.384+0000 7f53b77fe640 1 --2- 172.21.15.139:0/3902979575 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f53a4074630 0x7f53a4076a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.384+0000 7f58ea59c640 1 --2- 172.21.15.139:0/2611075467 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f58cc06c4b0 0x7f58cc06e970 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f58e4033000 tx=0x7f58e4009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.384+0000 7f76237fe640 1 --2- 172.21.15.139:0/4073981168 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f7610074580 0x7f76100769a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.384+0000 7fc6edae6640 1 --2- 172.21.15.139:0/3524909242 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fc6e8153f90 0x7fc6e8152af0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.384+0000 7f53b67fc640 1 --2- 172.21.15.139:0/3902979575 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f53a406c3b0 0x7f53a406e870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f53b81503f0 tx=0x7f53a80095a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.384+0000 7f0686d9d640 1 --2- 172.21.15.139:0/3790464205 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0668074600 0x7f0668076a20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.384+0000 7fb86f7fe640 1 --2- 172.21.15.139:0/512074014 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb85c06c2e0 0x7fb85c06e7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb86800ab90 tx=0x7fb868009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.385+0000 7f53b77fe640 1 --2- 172.21.15.139:0/3902979575 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f53a4074630 0x7f53a4076a50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.385+0000 7f9735a66640 1 --2- 172.21.15.139:0/682029708 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f97140745d0 0x7f97140769f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.385+0000 7f58daffd640 1 -- 172.21.15.139:0/2611075467 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f58cc077e90 con 0x7f58cc075320 2024-09-16T00:07:43.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.385+0000 7fb857fff640 1 -- 172.21.15.139:0/512074014 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb864034ec0 con 0x7fb85c074560 2024-09-16T00:07:43.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.385+0000 7fd3257fa640 1 -- 172.21.15.139:0/3144167841 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd308077000 con 0x7fd308074530 2024-09-16T00:07:43.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.385+0000 7fc6deffd640 1 -- 172.21.15.139:0/3524909242 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc6e80cc710 con 0x7fc6e8153f90 2024-09-16T00:07:43.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.385+0000 7f760b7fe640 1 -- 172.21.15.139:0/4073981168 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7618032440 con 0x7f7610074580 2024-09-16T00:07:43.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.385+0000 7f0676ffd640 1 -- 172.21.15.139:0/3790464205 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f06680770d0 con 0x7f0668074600 2024-09-16T00:07:43.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.385+0000 7f539f7fe640 1 -- 172.21.15.139:0/3902979575 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f53ac02be50 con 0x7f53a4074630 2024-09-16T00:07:43.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.386+0000 7f9725ffb640 1 -- 172.21.15.139:0/682029708 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f97140770a0 con 0x7f97140745d0 2024-09-16T00:07:43.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.415+0000 7f58eb59e640 1 -- 172.21.15.139:0/2611075467 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f58d0002dd0 con 0x7f58cc075320 2024-09-16T00:07:43.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.416+0000 7f58daffd640 1 -- 172.21.15.139:0/2611075467 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f58d0002dd0 con 0x7f58cc075320 2024-09-16T00:07:43.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.416+0000 7fb875cd0640 1 -- 172.21.15.139:0/512074014 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb850002dd0 con 0x7fb85c074560 2024-09-16T00:07:43.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.416+0000 7f971b7fe640 1 -- 172.21.15.139:0/682029708 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9730152340 con 0x7f97140745d0 2024-09-16T00:07:43.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.416+0000 7fd32ecd7640 1 -- 172.21.15.139:0/3144167841 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd328165a40 con 0x7fd308074530 2024-09-16T00:07:43.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.416+0000 7f7623fff640 1 -- 172.21.15.139:0/4073981168 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f761c04c5f0 con 0x7f7610074580 2024-09-16T00:07:43.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.416+0000 7f068cbc4640 1 -- 172.21.15.139:0/3790464205 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0680007a20 con 0x7f0668074600 2024-09-16T00:07:43.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.416+0000 7f53bd5bd640 1 -- 172.21.15.139:0/3902979575 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f53b80d13d0 con 0x7f53a4074630 2024-09-16T00:07:43.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.416+0000 7fb857fff640 1 -- 172.21.15.139:0/512074014 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb850002dd0 con 0x7fb85c074560 2024-09-16T00:07:43.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.416+0000 7f58eb59e640 1 -- 172.21.15.139:0/2611075467 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f58cc075320 msgr2=0x7f58cc0777c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.416+0000 7f58eb59e640 1 --2- 172.21.15.139:0/2611075467 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f58cc075320 0x7f58cc0777c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.416+0000 7fc6f054b640 1 -- 172.21.15.139:0/3524909242 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc6e81492f0 con 0x7fc6e8153f90 2024-09-16T00:07:43.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.416+0000 7f9725ffb640 1 -- 172.21.15.139:0/682029708 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9730152340 con 0x7f97140745d0 2024-09-16T00:07:43.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7f58eb59e640 1 -- 172.21.15.139:0/2611075467 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f58cc06c4b0 msgr2=0x7f58cc06e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7f58eb59e640 1 --2- 172.21.15.139:0/2611075467 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f58cc06c4b0 0x7f58cc06e970 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f58e4033000 tx=0x7f58e4009000 comp rx=0 tx=0).stop 2024-09-16T00:07:43.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7f58eb59e640 1 -- 172.21.15.139:0/2611075467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ec15eb60 msgr2=0x7f58ec1e8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7f58eb59e640 1 --2- 172.21.15.139:0/2611075467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ec15eb60 0x7f58ec1e8710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00b410 tx=0x7f58dc00b8d0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7f58ead9d640 1 -- 172.21.15.139:0/2611075467 reap_dead start 2024-09-16T00:07:43.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7f760b7fe640 1 -- 172.21.15.139:0/4073981168 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f761c04c5f0 con 0x7f7610074580 2024-09-16T00:07:43.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7f539f7fe640 1 -- 172.21.15.139:0/3902979575 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f53b80d13d0 con 0x7f53a4074630 2024-09-16T00:07:43.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7fd3257fa640 1 -- 172.21.15.139:0/3144167841 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd328165a40 con 0x7fd308074530 2024-09-16T00:07:43.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7f0676ffd640 1 -- 172.21.15.139:0/3790464205 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0680007a20 con 0x7f0668074600 2024-09-16T00:07:43.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7fc6deffd640 1 -- 172.21.15.139:0/3524909242 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc6e81492f0 con 0x7fc6e8153f90 2024-09-16T00:07:43.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7f58eb59e640 1 -- 172.21.15.139:0/2611075467 shutdown_connections 2024-09-16T00:07:43.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7f58eb59e640 1 -- 172.21.15.139:0/2611075467 >> 172.21.15.139:0/2611075467 conn(0x7f58ec0c8a00 msgr2=0x7f58ec0c6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7f971b7fe640 1 -- 172.21.15.139:0/682029708 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f97140745d0 msgr2=0x7f97140769f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7f971b7fe640 1 --2- 172.21.15.139:0/682029708 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f97140745d0 0x7f97140769f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7fb875cd0640 1 -- 172.21.15.139:0/512074014 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fb85c074560 msgr2=0x7fb85c076980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7fd32ecd7640 1 -- 172.21.15.139:0/3144167841 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd308074530 msgr2=0x7fd308076950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7fd32ecd7640 1 --2- 172.21.15.139:0/3144167841 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd308074530 0x7fd308076950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7fb875cd0640 1 --2- 172.21.15.139:0/512074014 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fb85c074560 0x7fb85c076980 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7f7623fff640 1 -- 172.21.15.139:0/4073981168 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f7610074580 msgr2=0x7f76100769a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7f7623fff640 1 --2- 172.21.15.139:0/4073981168 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f7610074580 0x7f76100769a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7fd32ecd7640 1 -- 172.21.15.139:0/3144167841 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd30806c2b0 msgr2=0x7fd30806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.417+0000 7fd32ecd7640 1 --2- 172.21.15.139:0/3144167841 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd30806c2b0 0x7fd30806e770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd310034000 tx=0x7fd31002e000 comp rx=0 tx=0).stop 2024-09-16T00:07:43.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f971b7fe640 1 -- 172.21.15.139:0/682029708 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f971406c3b0 msgr2=0x7f971406e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f971b7fe640 1 --2- 172.21.15.139:0/682029708 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f971406c3b0 0x7f971406e870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f972c009630 tx=0x7f972c009240 comp rx=0 tx=0).stop 2024-09-16T00:07:43.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f971b7fe640 1 -- 172.21.15.139:0/682029708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9730071700 msgr2=0x7f9730138bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f971b7fe640 1 --2- 172.21.15.139:0/682029708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9730071700 0x7f9730138bf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9728009d90 tx=0x7f9728004a40 comp rx=0 tx=0).stop 2024-09-16T00:07:43.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f9735a66640 1 -- 172.21.15.139:0/682029708 reap_dead start 2024-09-16T00:07:43.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f7623fff640 1 -- 172.21.15.139:0/4073981168 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f761006c380 msgr2=0x7f761006e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f7623fff640 1 --2- 172.21.15.139:0/4073981168 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f761006c380 0x7f761006e840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7624000b60 tx=0x7f762406e040 comp rx=0 tx=0).stop 2024-09-16T00:07:43.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f7623fff640 1 -- 172.21.15.139:0/4073981168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f761c047a80 msgr2=0x7f761c0d5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f7623fff640 1 --2- 172.21.15.139:0/4073981168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f761c047a80 0x7f761c0d5d60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f761802a7a0 tx=0x7f76180046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fb875cd0640 1 -- 172.21.15.139:0/512074014 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb85c06c2e0 msgr2=0x7fb85c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fb875cd0640 1 --2- 172.21.15.139:0/512074014 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb85c06c2e0 0x7fb85c06e7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb86800ab90 tx=0x7fb868009250 comp rx=0 tx=0).stop 2024-09-16T00:07:43.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fb875cd0640 1 -- 172.21.15.139:0/512074014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8700b5280 msgr2=0x7fb8701fd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fb875cd0640 1 --2- 172.21.15.139:0/512074014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8700b5280 0x7fb8701fd850 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb864009960 tx=0x7fb8640046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f58eb59e640 1 -- 172.21.15.139:0/2611075467 shutdown_connections 2024-09-16T00:07:43.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fc6f054b640 1 -- 172.21.15.139:0/3524909242 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fc6e8153f90 msgr2=0x7fc6e8152af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fc6f054b640 1 --2- 172.21.15.139:0/3524909242 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fc6e8153f90 0x7fc6e8152af0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fd32ecd7640 1 -- 172.21.15.139:0/3144167841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd328072c60 msgr2=0x7fd32814c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fd32ecd7640 1 --2- 172.21.15.139:0/3144167841 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd328072c60 0x7fd32814c060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd31800c7b0 tx=0x7fd31800cc70 comp rx=0 tx=0).stop 2024-09-16T00:07:43.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fd32d274640 1 -- 172.21.15.139:0/3144167841 reap_dead start 2024-09-16T00:07:43.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f76237fe640 1 -- 172.21.15.139:0/4073981168 reap_dead start 2024-09-16T00:07:43.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f7623fff640 1 -- 172.21.15.139:0/4073981168 shutdown_connections 2024-09-16T00:07:43.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fb86ffff640 1 -- 172.21.15.139:0/512074014 reap_dead start 2024-09-16T00:07:43.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f068cbc4640 1 -- 172.21.15.139:0/3790464205 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0668074600 msgr2=0x7f0668076a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f068cbc4640 1 --2- 172.21.15.139:0/3790464205 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0668074600 0x7f0668076a20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f068cbc4640 1 -- 172.21.15.139:0/3790464205 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f066806c380 msgr2=0x7f066806e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f068cbc4640 1 --2- 172.21.15.139:0/3790464205 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f066806c380 0x7f066806e840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f067c037000 tx=0x7f067c00d040 comp rx=0 tx=0).stop 2024-09-16T00:07:43.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f068cbc4640 1 -- 172.21.15.139:0/3790464205 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0680045120 msgr2=0x7f06800cd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f068cbc4640 1 --2- 172.21.15.139:0/3790464205 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0680045120 0x7f06800cd550 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f067800eb40 tx=0x7f067800d350 comp rx=0 tx=0).stop 2024-09-16T00:07:43.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f0686d9d640 1 -- 172.21.15.139:0/3790464205 reap_dead start 2024-09-16T00:07:43.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fd32ecd7640 1 -- 172.21.15.139:0/3144167841 shutdown_connections 2024-09-16T00:07:43.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fd32ecd7640 1 -- 172.21.15.139:0/3144167841 >> 172.21.15.139:0/3144167841 conn(0x7fd32806da40 msgr2=0x7fd3280ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f7623fff640 1 -- 172.21.15.139:0/4073981168 >> 172.21.15.139:0/4073981168 conn(0x7f761c04d830 msgr2=0x7f761c0d1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fb875cd0640 1 -- 172.21.15.139:0/512074014 shutdown_connections 2024-09-16T00:07:43.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fb875cd0640 1 -- 172.21.15.139:0/512074014 >> 172.21.15.139:0/512074014 conn(0x7fb8700d06a0 msgr2=0x7fb87015e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f971b7fe640 1 -- 172.21.15.139:0/682029708 shutdown_connections 2024-09-16T00:07:43.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f971b7fe640 1 -- 172.21.15.139:0/682029708 >> 172.21.15.139:0/682029708 conn(0x7f97300c8ae0 msgr2=0x7f973014fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f7623fff640 1 -- 172.21.15.139:0/4073981168 shutdown_connections 2024-09-16T00:07:43.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f7623fff640 1 -- 172.21.15.139:0/4073981168 wait complete. 2024-09-16T00:07:43.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fb875cd0640 1 -- 172.21.15.139:0/512074014 shutdown_connections 2024-09-16T00:07:43.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fb875cd0640 1 -- 172.21.15.139:0/512074014 wait complete. 2024-09-16T00:07:43.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f068cbc4640 1 -- 172.21.15.139:0/3790464205 shutdown_connections 2024-09-16T00:07:43.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f068cbc4640 1 -- 172.21.15.139:0/3790464205 >> 172.21.15.139:0/3790464205 conn(0x7f068004fc80 msgr2=0x7f0680034cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f068cbc4640 1 -- 172.21.15.139:0/3790464205 shutdown_connections 2024-09-16T00:07:43.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f068cbc4640 1 -- 172.21.15.139:0/3790464205 wait complete. 2024-09-16T00:07:43.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f58eb59e640 1 -- 172.21.15.139:0/2611075467 wait complete. 2024-09-16T00:07:43.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fc6f054b640 1 -- 172.21.15.139:0/3524909242 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc6cc06c380 msgr2=0x7fc6cc06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fc6f054b640 1 --2- 172.21.15.139:0/3524909242 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc6cc06c380 0x7fc6cc06e840 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4034000 tx=0x7fc6e402c000 comp rx=0 tx=0).stop 2024-09-16T00:07:43.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fc6f054b640 1 -- 172.21.15.139:0/3524909242 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6e80cc330 msgr2=0x7fc6e81478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fc6f054b640 1 --2- 172.21.15.139:0/3524909242 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc6e80cc330 0x7fc6e81478e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000c740 tx=0x7fc6e000cc00 comp rx=0 tx=0).stop 2024-09-16T00:07:43.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fc6eeae8640 1 -- 172.21.15.139:0/3524909242 reap_dead start 2024-09-16T00:07:43.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fc6f054b640 1 -- 172.21.15.139:0/3524909242 shutdown_connections 2024-09-16T00:07:43.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fc6f054b640 1 -- 172.21.15.139:0/3524909242 >> 172.21.15.139:0/3524909242 conn(0x7fc6e806da40 msgr2=0x7fc6e8154b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f53bd5bd640 1 -- 172.21.15.139:0/3902979575 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f53a4074630 msgr2=0x7f53a4076a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f53bd5bd640 1 --2- 172.21.15.139:0/3902979575 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f53a4074630 0x7f53a4076a50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f53bd5bd640 1 -- 172.21.15.139:0/3902979575 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f53a406c3b0 msgr2=0x7f53a406e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f53bd5bd640 1 --2- 172.21.15.139:0/3902979575 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f53a406c3b0 0x7f53a406e870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f53b81503f0 tx=0x7f53a80095a0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f53bd5bd640 1 -- 172.21.15.139:0/3902979575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b80cd8f0 msgr2=0x7f53b81f0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f53bd5bd640 1 --2- 172.21.15.139:0/3902979575 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53b80cd8f0 0x7f53b81f0450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0098f0 tx=0x7f53ac0046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f53b77fe640 1 -- 172.21.15.139:0/3902979575 reap_dead start 2024-09-16T00:07:43.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fd32ecd7640 1 -- 172.21.15.139:0/3144167841 shutdown_connections 2024-09-16T00:07:43.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fd32ecd7640 1 -- 172.21.15.139:0/3144167841 wait complete. 2024-09-16T00:07:43.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f971b7fe640 1 -- 172.21.15.139:0/682029708 shutdown_connections 2024-09-16T00:07:43.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f53bd5bd640 1 -- 172.21.15.139:0/3902979575 shutdown_connections 2024-09-16T00:07:43.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f53bd5bd640 1 -- 172.21.15.139:0/3902979575 >> 172.21.15.139:0/3902979575 conn(0x7f53b806c850 msgr2=0x7f53b8151650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7f971b7fe640 1 -- 172.21.15.139:0/682029708 wait complete. 2024-09-16T00:07:43.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fc6f054b640 1 -- 172.21.15.139:0/3524909242 shutdown_connections 2024-09-16T00:07:43.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.418+0000 7fc6f054b640 1 -- 172.21.15.139:0/3524909242 wait complete. 2024-09-16T00:07:43.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.419+0000 7f53bd5bd640 1 -- 172.21.15.139:0/3902979575 shutdown_connections 2024-09-16T00:07:43.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.419+0000 7f53bd5bd640 1 -- 172.21.15.139:0/3902979575 wait complete. 2024-09-16T00:07:43.489 INFO:teuthology.orchestra.run.smithi139.stdout:64424509448 2024-09-16T00:07:43.490 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T00:07:43.491 INFO:teuthology.orchestra.run.smithi139.stdout:55834574855 2024-09-16T00:07:43.491 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T00:07:43.491 INFO:teuthology.orchestra.run.smithi139.stdout:60129542151 2024-09-16T00:07:43.492 INFO:teuthology.orchestra.run.smithi139.stdout:64424509447 2024-09-16T00:07:43.492 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T00:07:43.493 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T00:07:43.494 INFO:teuthology.orchestra.run.smithi139.stdout:64424509446 2024-09-16T00:07:43.494 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T00:07:43.495 INFO:teuthology.orchestra.run.smithi139.stdout:64424509448 2024-09-16T00:07:43.495 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T00:07:43.496 INFO:teuthology.orchestra.run.smithi139.stdout:64424509448 2024-09-16T00:07:43.496 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T00:07:43.497 INFO:teuthology.orchestra.run.smithi139.stdout:60129542151 2024-09-16T00:07:43.497 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T00:07:43.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.698+0000 7f18e8800640 1 Processor -- start 2024-09-16T00:07:43.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.698+0000 7efcdf8f1640 1 Processor -- start 2024-09-16T00:07:43.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.698+0000 7f18e8800640 1 -- start start 2024-09-16T00:07:43.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.699+0000 7f28b25fd640 1 Processor -- start 2024-09-16T00:07:43.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.699+0000 7f77f544c640 1 Processor -- start 2024-09-16T00:07:43.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.699+0000 7f77f544c640 1 -- start start 2024-09-16T00:07:43.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.699+0000 7f28b25fd640 1 -- start start 2024-09-16T00:07:43.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.699+0000 7f18e8800640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f18e406ef10 0x7f18e406f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.699+0000 7f18e8800640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18e40c5e00 con 0x7f18e40c6fc0 2024-09-16T00:07:43.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.699+0000 7f18e8800640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18e40c60e0 con 0x7f18e4073e20 2024-09-16T00:07:43.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.699+0000 7f18e8800640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f18e406dc00 con 0x7f18e406ef10 2024-09-16T00:07:43.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f18e359e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f18e40c6fc0 0x7f18e4169420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45224/0 (socket says 172.21.15.139:45224) 2024-09-16T00:07:43.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f18e359e640 1 -- 172.21.15.139:0/3689540167 learned_addr learned my addr 172.21.15.139:0/3689540167 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:43.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f18e2d9d640 1 --2- 172.21.15.139:0/3689540167 >> v2:172.21.15.139:3300/0 conn(0x7f18e406ef10 0x7f18e406f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.699+0000 7efcdf8f1640 1 -- start start 2024-09-16T00:07:43.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7efcdf8f1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efcd806fc20 0x7efcd80cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f77f544c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f77f0154a80 0x7f77f0072af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7efcdf8f1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd80727d0 con 0x7efcd8070040 2024-09-16T00:07:43.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7efcdf8f1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd8072ab0 con 0x7efcd8158f00 2024-09-16T00:07:43.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7efcdf8f1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7efcd80c8c90 con 0x7efcd806fc20 2024-09-16T00:07:43.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7efcdce8c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efcd806fc20 0x7efcd80cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7efcdce8c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efcd806fc20 0x7efcd80cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39514/0 (socket says 172.21.15.139:39514) 2024-09-16T00:07:43.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7efcdce8c640 1 -- 172.21.15.139:0/3743306953 learned_addr learned my addr 172.21.15.139:0/3743306953 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f77f544c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77f0073030 0x7f77f0165260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f77eeffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f77f0154a80 0x7f77f0072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f77eeffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f77f0154a80 0x7f77f0072af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40486/0 (socket says 172.21.15.139:40486) 2024-09-16T00:07:43.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f77ee7fc640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f77f00c8020 0x7f77f00cc150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:35850/0 (socket says 172.21.15.139:35850) 2024-09-16T00:07:43.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7efcdce8c640 1 -- 172.21.15.139:0/3743306953 >> v1:172.21.15.164:6789/0 conn(0x7efcd8158f00 legacy=0x7efcd80c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:43.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7efcdce8c640 1 -- 172.21.15.139:0/3743306953 >> v1:172.21.15.139:6789/0 conn(0x7efcd8070040 legacy=0x7efcd80c9120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:07:43.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f28b25fd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28ac154960 0x7f28ac0c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f28b25fd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f28ac0cd8f0 0x7f28ac15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f28b25fd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f28ac0d0790 0x7f28ac0cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f28b25fd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f28ac0cd460 con 0x7f28ac0d0790 2024-09-16T00:07:43.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f28b25fd640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f28ac0c5660 con 0x7f28ac0cd8f0 2024-09-16T00:07:43.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f28b25fd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f28ac15b700 con 0x7f28ac154960 2024-09-16T00:07:43.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f28ab7fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f28ac0cd8f0 0x7f28ac15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f28ab7fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f28ac0cd8f0 0x7f28ac15b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39522/0 (socket says 172.21.15.139:39522) 2024-09-16T00:07:43.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f28ab7fe640 1 -- 172.21.15.139:0/1950841741 learned_addr learned my addr 172.21.15.139:0/1950841741 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f28b0b9a640 1 --2- 172.21.15.139:0/1950841741 >> v2:172.21.15.139:3300/0 conn(0x7f28ac0d0790 0x7f28ac0cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f77eeffd640 1 -- 172.21.15.139:0/3729868191 learned_addr learned my addr 172.21.15.139:0/3729868191 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f77f544c640 1 -- 172.21.15.139:0/3729868191 --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77f00c6f80 con 0x7f77f00c8020 2024-09-16T00:07:43.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f77ef7fe640 1 --2- 172.21.15.139:0/3729868191 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77f0073030 0x7f77f0165260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f28ab7fe640 1 -- 172.21.15.139:0/1950841741 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28ac154960 msgr2=0x7f28ac0c5120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:07:43.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f28ab7fe640 1 --2- 172.21.15.139:0/1950841741 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28ac154960 0x7f28ac0c5120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f28ab7fe640 1 -- 172.21.15.139:0/1950841741 >> v2:172.21.15.139:3300/0 conn(0x7f28ac0d0790 msgr2=0x7f28ac0cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:43.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f28ab7fe640 1 --2- 172.21.15.139:0/1950841741 >> v2:172.21.15.139:3300/0 conn(0x7f28ac0d0790 0x7f28ac0cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f28ab7fe640 1 -- 172.21.15.139:0/1950841741 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ac0cc570 con 0x7f28ac0cd8f0 2024-09-16T00:07:43.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f77f544c640 1 -- 172.21.15.139:0/3729868191 --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f77f00702d0 con 0x7f77f0154a80 2024-09-16T00:07:43.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f7c7759e640 1 Processor -- start 2024-09-16T00:07:43.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f77f544c640 1 -- 172.21.15.139:0/3729868191 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f77f006fd50 con 0x7f77f0073030 2024-09-16T00:07:43.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f7c7759e640 1 -- start start 2024-09-16T00:07:43.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.702+0000 7f77edffb640 1 -- 172.21.15.139:0/3729868191 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2667286722 0 0) 0x7f77f00c6f80 con 0x7f77f00c8020 2024-09-16T00:07:43.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.702+0000 7f28b0b9a640 1 --2- 172.21.15.139:0/1950841741 >> v2:172.21.15.139:3300/0 conn(0x7f28ac0d0790 0x7f28ac0cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:43.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f18e2d9d640 1 -- 172.21.15.139:0/3689540167 >> v1:172.21.15.164:6789/0 conn(0x7f18e4073e20 legacy=0x7f18e415f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f18e2d9d640 1 -- 172.21.15.139:0/3689540167 >> v1:172.21.15.139:6789/0 conn(0x7f18e40c6fc0 legacy=0x7f18e4169420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7efcdce8c640 1 -- 172.21.15.139:0/3743306953 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd8072d90 con 0x7efcd806fc20 2024-09-16T00:07:43.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.700+0000 7f18e2d9d640 1 -- 172.21.15.139:0/3689540167 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e415ec90 con 0x7f18e406ef10 2024-09-16T00:07:43.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.701+0000 7f18e2d9d640 1 --2- 172.21.15.139:0/3689540167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18e406ef10 0x7f18e406f2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009d90 tx=0x7f18d402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70ea213672d9d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.702+0000 7f77eeffd640 1 -- 172.21.15.139:0/3729868191 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77f0073030 msgr2=0x7f77f0165260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:43.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.702+0000 7fcc3268d640 1 Processor -- start 2024-09-16T00:07:43.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.702+0000 7f77eeffd640 1 --2- 172.21.15.139:0/3729868191 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77f0073030 0x7f77f0165260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.702+0000 7efcdce8c640 1 --2- 172.21.15.139:0/3743306953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcd806fc20 0x7efcd80cc150 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009870 tx=0x7efcd002a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd612556943371a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.702+0000 7fcc3268d640 1 -- start start 2024-09-16T00:07:43.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.702+0000 7f18e1d9b640 1 -- 172.21.15.139:0/3689540167 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4007740 con 0x7f18e406ef10 2024-09-16T00:07:43.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.702+0000 7f18e1d9b640 1 -- 172.21.15.139:0/3689540167 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d40078e0 con 0x7f18e406ef10 2024-09-16T00:07:43.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.702+0000 7f77eeffd640 1 -- 172.21.15.139:0/3729868191 >> v1:172.21.15.164:6789/0 conn(0x7f77f00c8020 legacy=0x7f77f00cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.702+0000 7f77eeffd640 1 -- 172.21.15.139:0/3729868191 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f00c6ca0 con 0x7f77f0154a80 2024-09-16T00:07:43.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.702+0000 7efccffff640 1 -- 172.21.15.139:0/3743306953 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0037070 con 0x7efcd806fc20 2024-09-16T00:07:43.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.702+0000 7efccffff640 1 -- 172.21.15.139:0/3743306953 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcd0032640 con 0x7efcd806fc20 2024-09-16T00:07:43.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.702+0000 7efccffff640 1 -- 172.21.15.139:0/3743306953 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0032a10 con 0x7efcd806fc20 2024-09-16T00:07:43.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.702+0000 7f18e1d9b640 1 -- 172.21.15.139:0/3689540167 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d4007cb0 con 0x7f18e406ef10 2024-09-16T00:07:43.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f28ab7fe640 1 --2- 172.21.15.139:0/1950841741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ac0cd8f0 0x7f28ac15b1c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009e50 tx=0x7f28a402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da863119d5edf028 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f28aaffd640 1 -- 172.21.15.139:0/1950841741 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4036070 con 0x7f28ac0cd8f0 2024-09-16T00:07:43.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f28aaffd640 1 -- 172.21.15.139:0/1950841741 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a4031600 con 0x7f28ac0cd8f0 2024-09-16T00:07:43.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f28aaffd640 1 -- 172.21.15.139:0/1950841741 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a40319d0 con 0x7f28ac0cd8f0 2024-09-16T00:07:43.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f18e8800640 1 -- 172.21.15.139:0/3689540167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18e406ef10 msgr2=0x7f18e406f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f18e8800640 1 --2- 172.21.15.139:0/3689540167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18e406ef10 0x7f18e406f2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009d90 tx=0x7f18d402a7a0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f7c7759e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7c7815a6b0 0x7f7c780c6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f77ef7fe640 1 --2- 172.21.15.139:0/3729868191 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77f0073030 0x7f77f0165260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:43.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f7c7759e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7c7806fbc0 0x7f7c7806ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f7c7759e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c7806cb90 con 0x7f7c780c5dc0 2024-09-16T00:07:43.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f7c7759e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7c7815aba0 con 0x7f7c7815a6b0 2024-09-16T00:07:43.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f37ba4e7640 1 Processor -- start 2024-09-16T00:07:43.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f7c76d9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7c7806fbc0 0x7f7c7806ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f7c76d9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7c7806fbc0 0x7f7c7806ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:55122/0 (socket says 172.21.15.139:55122) 2024-09-16T00:07:43.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f7c76d9d640 1 -- 172.21.15.139:0/2277371721 learned_addr learned my addr 172.21.15.139:0/2277371721 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f7c7759e640 1 -- 172.21.15.139:0/2277371721 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7c7806ce70 con 0x7f7c7806fbc0 2024-09-16T00:07:43.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7efcdf8f1640 1 -- 172.21.15.139:0/3743306953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcd806fc20 msgr2=0x7efcd80cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7efcdf8f1640 1 --2- 172.21.15.139:0/3743306953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcd806fc20 0x7efcd80cc150 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009870 tx=0x7efcd002a300 comp rx=0 tx=0).stop 2024-09-16T00:07:43.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f18e8800640 1 -- 172.21.15.139:0/3689540167 shutdown_connections 2024-09-16T00:07:43.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f18e8800640 1 --2- 172.21.15.139:0/3689540167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18e406ef10 0x7f18e406f2f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7f18e8800640 1 -- 172.21.15.139:0/3689540167 >> 172.21.15.139:0/3689540167 conn(0x7f18e40cbe90 msgr2=0x7f18e40cc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7f77eeffd640 1 --2- 172.21.15.139:0/3729868191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77f0154a80 0x7f77f0072af0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009e50 tx=0x7f77e802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683b0d4c58e16f20 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7f37ba4e7640 1 -- start start 2024-09-16T00:07:43.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.703+0000 7fcc3268d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcc2c0ccb30 0x7fcc2c15b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7fcc3268d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc2c0cfff0 con 0x7fcc2c0cba80 2024-09-16T00:07:43.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7efcdf8f1640 1 -- 172.21.15.139:0/3743306953 shutdown_connections 2024-09-16T00:07:43.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7efcdf8f1640 1 --2- 172.21.15.139:0/3743306953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcd806fc20 0x7efcd80cc150 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7fcc3268d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc2c0d02d0 con 0x7fcc2c0cab80 2024-09-16T00:07:43.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7f7c75d9b640 1 --2- 172.21.15.139:0/2277371721 >> v2:172.21.15.164:3300/0 conn(0x7f7c7815a6b0 0x7f7c780c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7efcdf8f1640 1 -- 172.21.15.139:0/3743306953 >> 172.21.15.139:0/3743306953 conn(0x7efcd806da40 msgr2=0x7efcd806f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7f77edffb640 1 -- 172.21.15.139:0/3729868191 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8036070 con 0x7f77f0154a80 2024-09-16T00:07:43.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7f77edffb640 1 -- 172.21.15.139:0/3729868191 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e8031600 con 0x7f77f0154a80 2024-09-16T00:07:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7f77edffb640 1 -- 172.21.15.139:0/3729868191 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8031a10 con 0x7f77f0154a80 2024-09-16T00:07:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7fcc3268d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcc2c0d0ee0 con 0x7fcc2c0ccb30 2024-09-16T00:07:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7fcc2b7fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fcc2c0cab80 0x7fcc2c0cb370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:35866/0 (socket says 172.21.15.139:35866) 2024-09-16T00:07:43.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7efcdf8f1640 1 -- 172.21.15.139:0/3743306953 shutdown_connections 2024-09-16T00:07:43.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7fcc2b7fe640 1 -- 172.21.15.139:0/3039950623 learned_addr learned my addr 172.21.15.139:0/3039950623 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:43.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7f18e8800640 1 -- 172.21.15.139:0/3689540167 shutdown_connections 2024-09-16T00:07:43.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7f18e8800640 1 -- 172.21.15.139:0/3689540167 wait complete. 2024-09-16T00:07:43.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7f7c7559a640 1 -- 172.21.15.139:0/2277371721 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3838029302 0 0) 0x7f7c7806cb90 con 0x7f7c780c5dc0 2024-09-16T00:07:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7f7c75d9b640 1 -- 172.21.15.139:0/2277371721 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7c7806fbc0 msgr2=0x7f7c7806ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7f7c75d9b640 1 --2- 172.21.15.139:0/2277371721 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7c7806fbc0 0x7f7c7806ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7f7c75d9b640 1 -- 172.21.15.139:0/2277371721 >> v1:172.21.15.164:6789/0 conn(0x7f7c780c5dc0 legacy=0x7f7c780c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7f7c75d9b640 1 -- 172.21.15.139:0/2277371721 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7806d150 con 0x7f7c7815a6b0 2024-09-16T00:07:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7fcc2bfff640 1 --2- 172.21.15.139:0/3039950623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcc2c0ccb30 0x7fcc2c15b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7fbee45fd640 1 Processor -- start 2024-09-16T00:07:43.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7fbee45fd640 1 -- start start 2024-09-16T00:07:43.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7efcdf8f1640 1 -- 172.21.15.139:0/3743306953 wait complete. 2024-09-16T00:07:43.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7fcc2affd640 1 -- 172.21.15.139:0/3039950623 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2752730179 0 0) 0x7fcc2c0d02d0 con 0x7fcc2c0cab80 2024-09-16T00:07:43.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7fcc2affd640 1 -- 172.21.15.139:0/3039950623 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc14003520 con 0x7fcc2c0cab80 2024-09-16T00:07:43.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7fcc2affd640 1 -- 172.21.15.139:0/3039950623 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 603753055 0 0) 0x7fcc2c0cfff0 con 0x7fcc2c0cba80 2024-09-16T00:07:43.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7fcc2affd640 1 -- 172.21.15.139:0/3039950623 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc2c0d02d0 con 0x7fcc2c0cba80 2024-09-16T00:07:43.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7f77f544c640 1 -- 172.21.15.139:0/3729868191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77f0154a80 msgr2=0x7f77f0072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7f77f544c640 1 --2- 172.21.15.139:0/3729868191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77f0154a80 0x7f77f0072af0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009e50 tx=0x7f77e802a6a0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7f77f544c640 1 -- 172.21.15.139:0/3729868191 shutdown_connections 2024-09-16T00:07:43.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7f77f544c640 1 --2- 172.21.15.139:0/3729868191 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77f0073030 0x7f77f0165260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7f28b25fd640 1 -- 172.21.15.139:0/1950841741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ac0cd8f0 msgr2=0x7f28ac15b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.704+0000 7f28b25fd640 1 --2- 172.21.15.139:0/1950841741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ac0cd8f0 0x7f28ac15b1c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f28a4009e50 tx=0x7f28a402a6f0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7f77f544c640 1 --2- 172.21.15.139:0/3729868191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77f0154a80 0x7f77f0072af0 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009e50 tx=0x7f77e802a6a0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7f77f544c640 1 -- 172.21.15.139:0/3729868191 >> 172.21.15.139:0/3729868191 conn(0x7f77f006da40 msgr2=0x7f77f006f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7f77f544c640 1 -- 172.21.15.139:0/3729868191 shutdown_connections 2024-09-16T00:07:43.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7f37ba4e7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37ac034f20 0x7f37ac0d4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7f37ba4e7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f37ac0db410 0x7f37ac0df330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7f37ba4e7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37ac0331d0 con 0x7f37ac007290 2024-09-16T00:07:43.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7f37ba4e7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f37ac04ba20 con 0x7f37ac0db410 2024-09-16T00:07:43.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7f37ba4e7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f37ac04ab30 con 0x7f37ac034f20 2024-09-16T00:07:43.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7f37b9ce6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f37ac0db410 0x7f37ac0df330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7f37b9ce6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f37ac0db410 0x7f37ac0df330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40492/0 (socket says 172.21.15.139:40492) 2024-09-16T00:07:43.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7f37b9ce6640 1 -- 172.21.15.139:0/770228908 learned_addr learned my addr 172.21.15.139:0/770228908 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7f77f544c640 1 -- 172.21.15.139:0/3729868191 wait complete. 2024-09-16T00:07:43.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7f37b9ce6640 1 -- 172.21.15.139:0/770228908 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37ac034f20 msgr2=0x7f37ac0d4bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:07:43.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7f37b9ce6640 1 --2- 172.21.15.139:0/770228908 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37ac034f20 0x7f37ac0d4bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7f37b9ce6640 1 -- 172.21.15.139:0/770228908 >> v1:172.21.15.139:6789/0 conn(0x7f37ac007290 legacy=0x7f37ac034810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:07:43.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7f18e8800640 1 Processor -- start 2024-09-16T00:07:43.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7f18e8800640 1 -- start start 2024-09-16T00:07:43.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7fcc2bfff640 1 -- 172.21.15.139:0/3039950623 >> v1:172.21.15.164:6789/0 conn(0x7fcc2c0cab80 legacy=0x7fcc2c0cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7fbee45fd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbed40d62e0 0x7fbed40dc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7fbee45fd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbed40d66c0 0x7fbed404bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7f7c75d9b640 1 --2- 172.21.15.139:0/2277371721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c7815a6b0 0x7f7c780c6790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c00ba40 tx=0x7f7c6c02c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4e5d7a4c7cc3962 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7f7c7559a640 1 -- 172.21.15.139:0/2277371721 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c032750 con 0x7f7c7815a6b0 2024-09-16T00:07:43.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7f7c7559a640 1 -- 172.21.15.139:0/2277371721 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c6c030050 con 0x7f7c7815a6b0 2024-09-16T00:07:43.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7f7c7559a640 1 -- 172.21.15.139:0/2277371721 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c032b10 con 0x7f7c7815a6b0 2024-09-16T00:07:43.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7efcdf8f1640 1 Processor -- start 2024-09-16T00:07:43.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7fcc2bfff640 1 -- 172.21.15.139:0/3039950623 >> v1:172.21.15.139:6789/0 conn(0x7fcc2c0cba80 legacy=0x7fcc2c164f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.705+0000 7fcc2bfff640 1 -- 172.21.15.139:0/3039950623 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2c0d05b0 con 0x7fcc2c0ccb30 2024-09-16T00:07:43.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7fbee45fd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbed4044b30 0x7fbed404c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7efcdf8f1640 1 -- start start 2024-09-16T00:07:43.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7fbee2399640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbed40d62e0 0x7fbed40dc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7fcc2bfff640 1 --2- 172.21.15.139:0/3039950623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcc2c0ccb30 0x7fcc2c15b0f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcc200097d0 tx=0x7fcc2002a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4653b18a280c3ea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7f28b25fd640 1 -- 172.21.15.139:0/1950841741 shutdown_connections 2024-09-16T00:07:43.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7f28b25fd640 1 --2- 172.21.15.139:0/1950841741 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ac0cd8f0 0x7f28ac15b1c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7f28b25fd640 1 --2- 172.21.15.139:0/1950841741 >> v2:172.21.15.139:3300/0 conn(0x7f28ac0d0790 0x7f28ac0cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7f28b25fd640 1 --2- 172.21.15.139:0/1950841741 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28ac154960 0x7f28ac0c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7f28b25fd640 1 -- 172.21.15.139:0/1950841741 >> 172.21.15.139:0/1950841741 conn(0x7f28ac06c850 msgr2=0x7f28ac06cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7fbee2399640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbed40d62e0 0x7fbed40dc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40508/0 (socket says 172.21.15.139:40508) 2024-09-16T00:07:43.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7fbee1b98640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbed40d66c0 0x7fbed404bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7fbee1b98640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbed40d66c0 0x7fbed404bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39536/0 (socket says 172.21.15.139:39536) 2024-09-16T00:07:43.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7fbee2b9a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbed4044b30 0x7fbed404c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7fbee2b9a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbed4044b30 0x7fbed404c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:55134/0 (socket says 172.21.15.139:55134) 2024-09-16T00:07:43.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7fbee45fd640 1 -- 172.21.15.139:0/232617737 --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbed404b200 con 0x7fbed40d62e0 2024-09-16T00:07:43.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7fbee2399640 1 -- 172.21.15.139:0/232617737 learned_addr learned my addr 172.21.15.139:0/232617737 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7fcc2affd640 1 -- 172.21.15.139:0/3039950623 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20037070 con 0x7fcc2c0ccb30 2024-09-16T00:07:43.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7fcc2affd640 1 -- 172.21.15.139:0/3039950623 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc20032640 con 0x7fcc2c0ccb30 2024-09-16T00:07:43.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7fcc2affd640 1 -- 172.21.15.139:0/3039950623 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc200329f0 con 0x7fcc2c0ccb30 2024-09-16T00:07:43.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f28b25fd640 1 -- 172.21.15.139:0/1950841741 shutdown_connections 2024-09-16T00:07:43.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7fbee45fd640 1 -- 172.21.15.139:0/232617737 --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fbed40dcc30 con 0x7fbed40d66c0 2024-09-16T00:07:43.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f18e8800640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f18e406ef10 0x7f18e41f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f18e8800640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18e4073e20 0x7f18e415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7fbee45fd640 1 -- 172.21.15.139:0/232617737 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbed40dcf10 con 0x7fbed4044b30 2024-09-16T00:07:43.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7fbee2399640 1 -- 172.21.15.139:0/232617737 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbed4044b30 msgr2=0x7fbed404c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:43.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7fbee2399640 1 --2- 172.21.15.139:0/232617737 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbed4044b30 0x7fbed404c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7fbee2399640 1 -- 172.21.15.139:0/232617737 >> v2:172.21.15.164:3300/0 conn(0x7fbed40d66c0 msgr2=0x7fbed404bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:43.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7fbee2399640 1 --2- 172.21.15.139:0/232617737 >> v2:172.21.15.164:3300/0 conn(0x7fbed40d66c0 0x7fbed404bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7fbee2399640 1 -- 172.21.15.139:0/232617737 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed40dd1f0 con 0x7fbed40d62e0 2024-09-16T00:07:43.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f18e8800640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18e40c6fc0 0x7f18e4154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f18e8800640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f18e40c6ad0 con 0x7f18e40c6fc0 2024-09-16T00:07:43.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f18e8800640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f18e40c63c0 con 0x7f18e4073e20 2024-09-16T00:07:43.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f18e8800640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f18e4159160 con 0x7f18e406ef10 2024-09-16T00:07:43.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f7c7759e640 1 -- 172.21.15.139:0/2277371721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c7815a6b0 msgr2=0x7f7c780c6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f7c7759e640 1 --2- 172.21.15.139:0/2277371721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c7815a6b0 0x7f7c780c6790 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c00ba40 tx=0x7f7c6c02c850 comp rx=0 tx=0).stop 2024-09-16T00:07:43.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f18e2d9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f18e406ef10 0x7f18e41f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f18e259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18e4073e20 0x7f18e415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f18e259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18e4073e20 0x7f18e415a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39550/0 (socket says 172.21.15.139:39550) 2024-09-16T00:07:43.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f18e259c640 1 -- 172.21.15.139:0/219719612 learned_addr learned my addr 172.21.15.139:0/219719612 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7efcdf8f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efcd806fc20 0x7efcd814c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f28b25fd640 1 -- 172.21.15.139:0/1950841741 wait complete. 2024-09-16T00:07:43.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.706+0000 7f37b9ce6640 1 -- 172.21.15.139:0/770228908 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac04ae10 con 0x7f37ac0db410 2024-09-16T00:07:43.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f37b9ce6640 1 --2- 172.21.15.139:0/770228908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37ac0db410 0x7f37ac0df330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f37b40518b0 tx=0x7f37b40a6950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5bf0d7fcd7159 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f37abfff640 1 -- 172.21.15.139:0/770228908 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b40b0020 con 0x7f37ac0db410 2024-09-16T00:07:43.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7efcdf8f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcd8070040 0x7efcd80c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f37abfff640 1 -- 172.21.15.139:0/770228908 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b40ac720 con 0x7f37ac0db410 2024-09-16T00:07:43.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.707+0000 7f37abfff640 1 -- 172.21.15.139:0/770228908 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b40acbd0 con 0x7f37ac0db410 2024-09-16T00:07:43.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7fcc3268d640 1 -- 172.21.15.139:0/3039950623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcc2c0ccb30 msgr2=0x7fcc2c15b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7efcdf8f1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd8158f00 0x7efcd814a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7efcdd68d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efcd806fc20 0x7efcd814c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7efcdd68d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efcd806fc20 0x7efcd814c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:55148/0 (socket says 172.21.15.139:55148) 2024-09-16T00:07:43.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7f18e259c640 1 -- 172.21.15.139:0/219719612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f18e406ef10 msgr2=0x7f18e41f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7efcdd68d640 1 -- 172.21.15.139:0/1686338288 learned_addr learned my addr 172.21.15.139:0/1686338288 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7fcc3268d640 1 --2- 172.21.15.139:0/3039950623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcc2c0ccb30 0x7fcc2c15b0f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcc200097d0 tx=0x7fcc2002a2c0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7efcdce8c640 1 --2- 172.21.15.139:0/1686338288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcd8070040 0x7efcd80c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7efcdf8f1640 1 -- 172.21.15.139:0/1686338288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efcd8159460 con 0x7efcd8158f00 2024-09-16T00:07:43.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7efcdf8f1640 1 -- 172.21.15.139:0/1686338288 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efcd81592e0 con 0x7efcd8070040 2024-09-16T00:07:43.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7efcdf8f1640 1 -- 172.21.15.139:0/1686338288 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7efcd80c9830 con 0x7efcd806fc20 2024-09-16T00:07:43.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7f18e259c640 1 --2- 172.21.15.139:0/219719612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f18e406ef10 0x7f18e41f8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7f18e259c640 1 -- 172.21.15.139:0/219719612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18e40c6fc0 msgr2=0x7f18e4154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7f18e259c640 1 --2- 172.21.15.139:0/219719612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18e40c6fc0 0x7f18e4154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7f18e259c640 1 -- 172.21.15.139:0/219719612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e40c4e30 con 0x7f18e4073e20 2024-09-16T00:07:43.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7f7c7759e640 1 -- 172.21.15.139:0/2277371721 shutdown_connections 2024-09-16T00:07:43.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7f7c7759e640 1 --2- 172.21.15.139:0/2277371721 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7c7806fbc0 0x7f7c7806ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7f7c7759e640 1 --2- 172.21.15.139:0/2277371721 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c7815a6b0 0x7f7c780c6790 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7f7c7759e640 1 -- 172.21.15.139:0/2277371721 >> 172.21.15.139:0/2277371721 conn(0x7f7c780cbe90 msgr2=0x7f7c780cc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7fcc3268d640 1 -- 172.21.15.139:0/3039950623 shutdown_connections 2024-09-16T00:07:43.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7fcc3268d640 1 --2- 172.21.15.139:0/3039950623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcc2c0ccb30 0x7fcc2c15b0f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7fcc3268d640 1 -- 172.21.15.139:0/3039950623 >> 172.21.15.139:0/3039950623 conn(0x7fcc2c06c310 msgr2=0x7fcc2c06c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7fbee2399640 1 --2- 172.21.15.139:0/232617737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed40d62e0 0x7fbed40dc3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbed40e0aa0 tx=0x7fbed802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c6cdaf7eb9efc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7f37ba4e7640 1 -- 172.21.15.139:0/770228908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37ac0db410 msgr2=0x7f37ac0df330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7f37ba4e7640 1 --2- 172.21.15.139:0/770228908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37ac0db410 0x7f37ac0df330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f37b40518b0 tx=0x7f37b40a6950 comp rx=0 tx=0).stop 2024-09-16T00:07:43.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7fcc3268d640 1 -- 172.21.15.139:0/3039950623 shutdown_connections 2024-09-16T00:07:43.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7fcc3268d640 1 -- 172.21.15.139:0/3039950623 wait complete. 2024-09-16T00:07:43.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7efcdde8e640 1 --2- 172.21.15.139:0/1686338288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd8158f00 0x7efcd814a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7efcdce8c640 1 -- 172.21.15.139:0/1686338288 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efcd806fc20 msgr2=0x7efcd814c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7efcdce8c640 1 --2- 172.21.15.139:0/1686338288 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efcd806fc20 0x7efcd814c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7efcdce8c640 1 -- 172.21.15.139:0/1686338288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd8158f00 msgr2=0x7efcd814a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.708+0000 7f18e259c640 1 --2- 172.21.15.139:0/219719612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18e4073e20 0x7f18e415a070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00c450 tx=0x7f18cc00c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7f37ba4e7640 1 -- 172.21.15.139:0/770228908 shutdown_connections 2024-09-16T00:07:43.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7f37ba4e7640 1 --2- 172.21.15.139:0/770228908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37ac0db410 0x7f37ac0df330 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7f37ba4e7640 1 --2- 172.21.15.139:0/770228908 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37ac034f20 0x7f37ac0d4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7f77f544c640 1 Processor -- start 2024-09-16T00:07:43.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7f37ba4e7640 1 -- 172.21.15.139:0/770228908 >> 172.21.15.139:0/770228908 conn(0x7f37ac0500a0 msgr2=0x7f37ac0504b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7f77f544c640 1 -- start start 2024-09-16T00:07:43.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7efcdce8c640 1 --2- 172.21.15.139:0/1686338288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd8158f00 0x7efcd814a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7efcdce8c640 1 -- 172.21.15.139:0/1686338288 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd814d280 con 0x7efcd8070040 2024-09-16T00:07:43.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7efcdde8e640 1 --2- 172.21.15.139:0/1686338288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd8158f00 0x7efcd814a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:43.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7f7c7759e640 1 -- 172.21.15.139:0/2277371721 shutdown_connections 2024-09-16T00:07:43.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7f37ba4e7640 1 -- 172.21.15.139:0/770228908 shutdown_connections 2024-09-16T00:07:43.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7f37ba4e7640 1 -- 172.21.15.139:0/770228908 wait complete. 2024-09-16T00:07:43.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.710+0000 7f18db7fe640 1 -- 172.21.15.139:0/219719612 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc0052d0 con 0x7f18e4073e20 2024-09-16T00:07:43.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.710+0000 7f18e8800640 1 -- 172.21.15.139:0/219719612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e40c5110 con 0x7f18e4073e20 2024-09-16T00:07:43.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.710+0000 7f18e8800640 1 -- 172.21.15.139:0/219719612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e4159870 con 0x7f18e4073e20 2024-09-16T00:07:43.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.710+0000 7f18db7fe640 1 -- 172.21.15.139:0/219719612 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18cc005470 con 0x7f18e4073e20 2024-09-16T00:07:43.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.710+0000 7f18db7fe640 1 -- 172.21.15.139:0/219719612 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc012870 con 0x7f18e4073e20 2024-09-16T00:07:43.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.710+0000 7f77f544c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77f0073030 0x7f77f01e13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.710+0000 7f77f544c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77f0153550 0x7f77f00c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7f7c7759e640 1 -- 172.21.15.139:0/2277371721 wait complete. 2024-09-16T00:07:43.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7efcdce8c640 1 --2- 172.21.15.139:0/1686338288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcd8070040 0x7efcd80c5070 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efcd8073430 tx=0x7efcd00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7efccdffb640 1 -- 172.21.15.139:0/1686338288 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0037040 con 0x7efcd8070040 2024-09-16T00:07:43.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7efccdffb640 1 -- 172.21.15.139:0/1686338288 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcd0030b80 con 0x7efcd8070040 2024-09-16T00:07:43.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7efcdf8f1640 1 -- 172.21.15.139:0/1686338288 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd814f3a0 con 0x7efcd8070040 2024-09-16T00:07:43.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.710+0000 7efcdf8f1640 1 -- 172.21.15.139:0/1686338288 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd814f960 con 0x7efcd8070040 2024-09-16T00:07:43.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.710+0000 7efccdffb640 1 -- 172.21.15.139:0/1686338288 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0004030 con 0x7efcd8070040 2024-09-16T00:07:43.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7f28b25fd640 1 Processor -- start 2024-09-16T00:07:43.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7f28b25fd640 1 -- start start 2024-09-16T00:07:43.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.710+0000 7f77eeffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77f0073030 0x7f77f01e13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.710+0000 7f77eeffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77f0073030 0x7f77f01e13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39564/0 (socket says 172.21.15.139:39564) 2024-09-16T00:07:43.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f28b25fd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28ac0cd8f0 0x7f28ac06fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7fcc3268d640 1 Processor -- start 2024-09-16T00:07:43.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f37ba4e7640 1 Processor -- start 2024-09-16T00:07:43.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f28b25fd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28ac0d0790 0x7f28ac0c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f28b25fd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ac154960 0x7f28ac149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f28b25fd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac0cce10 con 0x7f28ac0cd8f0 2024-09-16T00:07:43.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f28b25fd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac0ccb30 con 0x7f28ac154960 2024-09-16T00:07:43.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f28b25fd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f28ac0cd0f0 con 0x7f28ac0d0790 2024-09-16T00:07:43.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f37ba4e7640 1 -- start start 2024-09-16T00:07:43.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f28abfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28ac0cd8f0 0x7f28ac06fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f28abfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28ac0cd8f0 0x7f28ac06fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40538/0 (socket says 172.21.15.139:40538) 2024-09-16T00:07:43.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f28abfff640 1 -- 172.21.15.139:0/4183654677 learned_addr learned my addr 172.21.15.139:0/4183654677 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7efccdffb640 1 -- 172.21.15.139:0/1686338288 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efcd003a070 con 0x7efcd8070040 2024-09-16T00:07:43.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f7c7759e640 1 Processor -- start 2024-09-16T00:07:43.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.710+0000 7f77eeffd640 1 -- 172.21.15.139:0/2512889859 learned_addr learned my addr 172.21.15.139:0/2512889859 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f28b0b9a640 1 --2- 172.21.15.139:0/4183654677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ac154960 0x7f28ac149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f28ab7fe640 1 --2- 172.21.15.139:0/4183654677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28ac0d0790 0x7f28ac0c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f28abfff640 1 -- 172.21.15.139:0/4183654677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28ac0d0790 msgr2=0x7f28ac0c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f28abfff640 1 --2- 172.21.15.139:0/4183654677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28ac0d0790 0x7f28ac0c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f28abfff640 1 -- 172.21.15.139:0/4183654677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ac154960 msgr2=0x7f28ac149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7f28abfff640 1 --2- 172.21.15.139:0/4183654677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ac154960 0x7f28ac149bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f28abfff640 1 -- 172.21.15.139:0/4183654677 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ac1f27f0 con 0x7f28ac0cd8f0 2024-09-16T00:07:43.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f77ee7fc640 1 --2- 172.21.15.139:0/2512889859 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77f0153550 0x7f77f00c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f7c7759e640 1 -- start start 2024-09-16T00:07:43.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f77f544c640 1 --2- 172.21.15.139:0/2512889859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77f01e50a0 0x7f77f01e5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7fbee1397640 1 -- 172.21.15.139:0/232617737 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8037070 con 0x7fbed40d62e0 2024-09-16T00:07:43.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7fbee1397640 1 -- 172.21.15.139:0/232617737 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed8032440 con 0x7fbed40d62e0 2024-09-16T00:07:43.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.709+0000 7fbee1397640 1 -- 172.21.15.139:0/232617737 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8032810 con 0x7fbed40d62e0 2024-09-16T00:07:43.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.710+0000 7fbee45fd640 1 -- 172.21.15.139:0/232617737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed40d62e0 msgr2=0x7fbed40dc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.710+0000 7fbee45fd640 1 --2- 172.21.15.139:0/232617737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed40d62e0 0x7fbed40dc3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbed40e0aa0 tx=0x7fbed802a1e0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7fbee45fd640 1 -- 172.21.15.139:0/232617737 shutdown_connections 2024-09-16T00:07:43.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7fbee45fd640 1 --2- 172.21.15.139:0/232617737 >> v2:172.21.15.164:3300/0 conn(0x7fbed40d66c0 0x7fbed404bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7fbee45fd640 1 --2- 172.21.15.139:0/232617737 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbed4044b30 0x7fbed404c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7fbee45fd640 1 --2- 172.21.15.139:0/232617737 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed40d62e0 0x7fbed40dc3f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.711+0000 7fbee45fd640 1 -- 172.21.15.139:0/232617737 >> 172.21.15.139:0/232617737 conn(0x7fbed404fc50 msgr2=0x7fbed4050060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:43.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f77f544c640 1 -- 172.21.15.139:0/2512889859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f77f00718b0 con 0x7f77f01e50a0 2024-09-16T00:07:43.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f18e8800640 1 -- 172.21.15.139:0/219719612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c8005290 con 0x7f18e4073e20 2024-09-16T00:07:43.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f77f544c640 1 -- 172.21.15.139:0/2512889859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f77f00725d0 con 0x7f77f0073030 2024-09-16T00:07:43.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f77f544c640 1 -- 172.21.15.139:0/2512889859 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f77f015a920 con 0x7f77f0153550 2024-09-16T00:07:43.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f28abfff640 1 --2- 172.21.15.139:0/4183654677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28ac0cd8f0 0x7f28ac06fc30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f28a000b480 tx=0x7f28a000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f77ef7fe640 1 --2- 172.21.15.139:0/2512889859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77f01e50a0 0x7f77f01e5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7fbee45fd640 1 -- 172.21.15.139:0/232617737 shutdown_connections 2024-09-16T00:07:43.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7efccdffb640 1 --2- 172.21.15.139:0/1686338288 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efcbc06c300 0x7efcbc06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7fcc3268d640 1 -- start start 2024-09-16T00:07:43.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f77eeffd640 1 -- 172.21.15.139:0/2512889859 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77f0153550 msgr2=0x7f77f00c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f77eeffd640 1 --2- 172.21.15.139:0/2512889859 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77f0153550 0x7f77f00c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f77eeffd640 1 -- 172.21.15.139:0/2512889859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77f01e50a0 msgr2=0x7f77f01e5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f77eeffd640 1 --2- 172.21.15.139:0/2512889859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77f01e50a0 0x7f77f01e5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f77eeffd640 1 -- 172.21.15.139:0/2512889859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f01f3780 con 0x7f77f0073030 2024-09-16T00:07:43.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f77ef7fe640 1 --2- 172.21.15.139:0/2512889859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77f01e50a0 0x7f77f01e5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:43.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7efccdffb640 1 -- 172.21.15.139:0/1686338288 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7efcd00ac3e0 con 0x7efcd8070040 2024-09-16T00:07:43.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7efcdd68d640 1 --2- 172.21.15.139:0/1686338288 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efcbc06c300 0x7efcbc06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f28a8ff9640 1 -- 172.21.15.139:0/4183654677 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a0014070 con 0x7f28ac0cd8f0 2024-09-16T00:07:43.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f7c7759e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c7806fbc0 0x7f7c78154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f28a8ff9640 1 -- 172.21.15.139:0/4183654677 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a0010890 con 0x7f28ac0cd8f0 2024-09-16T00:07:43.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f28a8ff9640 1 -- 172.21.15.139:0/4183654677 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a0010c80 con 0x7f28ac0cd8f0 2024-09-16T00:07:43.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f7c7759e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c780c5dc0 0x7f7c78155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f37ba4e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37ac007290 0x7f37ac049cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f7c7759e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7c7815a6b0 0x7f7c78156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f7c7659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c7806fbc0 0x7f7c78154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f7c7659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c7806fbc0 0x7f7c78154bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39588/0 (socket says 172.21.15.139:39588) 2024-09-16T00:07:43.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f7c7659c640 1 -- 172.21.15.139:0/3730708142 learned_addr learned my addr 172.21.15.139:0/3730708142 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f7c7759e640 1 -- 172.21.15.139:0/3730708142 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7c780c7cb0 con 0x7f7c780c5dc0 2024-09-16T00:07:43.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f7c7759e640 1 -- 172.21.15.139:0/3730708142 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7806d710 con 0x7f7c7806fbc0 2024-09-16T00:07:43.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f7c7759e640 1 -- 172.21.15.139:0/3730708142 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7c780c7e30 con 0x7f7c7815a6b0 2024-09-16T00:07:43.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f37ba4e7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37ac034f20 0x7f37ac047450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f37ba4e7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37ac0db410 0x7f37ac0dcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f37b94e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37ac007290 0x7f37ac049cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f37ba4e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac04b6b0 con 0x7f37ac007290 2024-09-16T00:07:43.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f37ba4e7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac04b3d0 con 0x7f37ac034f20 2024-09-16T00:07:43.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f37ba4e7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f37ac0db8a0 con 0x7f37ac0db410 2024-09-16T00:07:43.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f37b94e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37ac007290 0x7f37ac049cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40552/0 (socket says 172.21.15.139:40552) 2024-09-16T00:07:43.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f37b94e5640 1 -- 172.21.15.139:0/1320260272 learned_addr learned my addr 172.21.15.139:0/1320260272 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7fbee45fd640 1 -- 172.21.15.139:0/232617737 wait complete. 2024-09-16T00:07:43.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7efcdd68d640 1 --2- 172.21.15.139:0/1686338288 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efcbc06c300 0x7efcbc06e7c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efcd400e250 tx=0x7efcd400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7fcc3268d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc2c0cab80 0x7fcc2c15a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7fcc3268d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc2c0cba80 0x7fcc2c157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7fcc3268d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcc2c0ccb30 0x7fcc2c157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7fcc2b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc2c0cba80 0x7fcc2c157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7fcc2bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc2c0cab80 0x7fcc2c15a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7fcc3268d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c151770 con 0x7fcc2c0cba80 2024-09-16T00:07:43.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7fcc2b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc2c0cba80 0x7fcc2c157640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40574/0 (socket says 172.21.15.139:40574) 2024-09-16T00:07:43.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7fcc2b7fe640 1 -- 172.21.15.139:0/2056087510 learned_addr learned my addr 172.21.15.139:0/2056087510 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7fcc3268d640 1 -- 172.21.15.139:0/2056087510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c0d0b70 con 0x7fcc2c0cab80 2024-09-16T00:07:43.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7fcc3268d640 1 -- 172.21.15.139:0/2056087510 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcc2c1518f0 con 0x7fcc2c0ccb30 2024-09-16T00:07:43.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f28b25fd640 1 -- 172.21.15.139:0/4183654677 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ac0c8770 con 0x7f28ac0cd8f0 2024-09-16T00:07:43.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f28b25fd640 1 -- 172.21.15.139:0/4183654677 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ac0c8d30 con 0x7f28ac0cd8f0 2024-09-16T00:07:43.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.712+0000 7f18db7fe640 1 -- 172.21.15.139:0/219719612 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f18cc012a10 con 0x7f18e4073e20 2024-09-16T00:07:43.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7efcdf8f1640 1 -- 172.21.15.139:0/1686338288 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb8005290 con 0x7efcd8070040 2024-09-16T00:07:43.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f18db7fe640 1 --2- 172.21.15.139:0/219719612 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f18c406c3b0 0x7f18c406e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f77eeffd640 1 --2- 172.21.15.139:0/2512889859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77f0073030 0x7f77f01e13d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009f70 tx=0x7f77e8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.713+0000 7f18db7fe640 1 -- 172.21.15.139:0/219719612 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f18cc08af80 con 0x7f18e4073e20 2024-09-16T00:07:43.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f7c75d9b640 1 --2- 172.21.15.139:0/3730708142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c780c5dc0 0x7f7c78155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f7c76d9d640 1 --2- 172.21.15.139:0/3730708142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7c7815a6b0 0x7f7c78156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f7c7659c640 1 -- 172.21.15.139:0/3730708142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7c7815a6b0 msgr2=0x7f7c78156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f7c7659c640 1 --2- 172.21.15.139:0/3730708142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7c7815a6b0 0x7f7c78156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f7c7659c640 1 -- 172.21.15.139:0/3730708142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c780c5dc0 msgr2=0x7f7c78155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f7c7659c640 1 --2- 172.21.15.139:0/3730708142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c780c5dc0 0x7f7c78155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f7c7659c640 1 -- 172.21.15.139:0/3730708142 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c781f5280 con 0x7f7c7806fbc0 2024-09-16T00:07:43.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f37b9ce6640 1 --2- 172.21.15.139:0/1320260272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37ac0db410 0x7f37ac0dcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f37b8ce4640 1 --2- 172.21.15.139:0/1320260272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37ac034f20 0x7f37ac047450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7fcc30c2a640 1 --2- 172.21.15.139:0/2056087510 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcc2c0ccb30 0x7fcc2c157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f18e2d9d640 1 --2- 172.21.15.139:0/219719612 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f18c406c3b0 0x7f18c406e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7fcc2bfff640 1 -- 172.21.15.139:0/2056087510 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcc2c0ccb30 msgr2=0x7fcc2c157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7fcc2bfff640 1 --2- 172.21.15.139:0/2056087510 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcc2c0ccb30 0x7fcc2c157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7fcc2bfff640 1 -- 172.21.15.139:0/2056087510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc2c0cba80 msgr2=0x7fcc2c157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7fcc2bfff640 1 --2- 172.21.15.139:0/2056087510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc2c0cba80 0x7fcc2c157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7fcc2bfff640 1 -- 172.21.15.139:0/2056087510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2c0c7200 con 0x7fcc2c0cab80 2024-09-16T00:07:43.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f7c7659c640 1 --2- 172.21.15.139:0/3730708142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c7806fbc0 0x7f7c78154bc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800c4c0 tx=0x7f7c6800c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f37b94e5640 1 -- 172.21.15.139:0/1320260272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37ac0db410 msgr2=0x7f37ac0dcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f37b94e5640 1 --2- 172.21.15.139:0/1320260272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37ac0db410 0x7f37ac0dcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f37b94e5640 1 -- 172.21.15.139:0/1320260272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37ac034f20 msgr2=0x7f37ac047450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f37b94e5640 1 --2- 172.21.15.139:0/1320260272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37ac034f20 0x7f37ac047450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.714+0000 7f37b94e5640 1 -- 172.21.15.139:0/1320260272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac0dbd10 con 0x7f37ac007290 2024-09-16T00:07:43.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.715+0000 7f37b9ce6640 1 --2- 172.21.15.139:0/1320260272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37ac0db410 0x7f37ac0dcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:43.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.715+0000 7fcc2bfff640 1 --2- 172.21.15.139:0/2056087510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc2c0cab80 0x7fcc2c15a240 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcc20031000 tx=0x7fcc200046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.715+0000 7f18e2d9d640 1 --2- 172.21.15.139:0/219719612 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f18c406c3b0 0x7f18c406e870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f18d4034000 tx=0x7f18d402f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.715+0000 7f37b94e5640 1 --2- 172.21.15.139:0/1320260272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37ac007290 0x7f37ac049cb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f37b000b3c0 tx=0x7f37b000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.715+0000 7f37a9ffb640 1 -- 172.21.15.139:0/1320260272 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0015020 con 0x7f37ac007290 2024-09-16T00:07:43.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.715+0000 7f37a9ffb640 1 -- 172.21.15.139:0/1320260272 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b0011830 con 0x7f37ac007290 2024-09-16T00:07:43.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.715+0000 7f37a9ffb640 1 -- 172.21.15.139:0/1320260272 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0011c20 con 0x7f37ac007290 2024-09-16T00:07:43.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.715+0000 7f28b25fd640 1 -- 172.21.15.139:0/4183654677 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28ac0cc1e0 con 0x7f28ac0cd8f0 2024-09-16T00:07:43.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.715+0000 7fbee45fd640 1 Processor -- start 2024-09-16T00:07:43.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.715+0000 7fbee45fd640 1 -- start start 2024-09-16T00:07:43.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.716+0000 7f28a8ff9640 1 -- 172.21.15.139:0/4183654677 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f28a0017070 con 0x7f28ac0cd8f0 2024-09-16T00:07:43.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.716+0000 7f37ba4e7640 1 -- 172.21.15.139:0/1320260272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ac0dbff0 con 0x7f37ac007290 2024-09-16T00:07:43.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.716+0000 7f37ba4e7640 1 -- 172.21.15.139:0/1320260272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ac0d9790 con 0x7f37ac007290 2024-09-16T00:07:43.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.716+0000 7f28a8ff9640 1 --2- 172.21.15.139:0/4183654677 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f289806c380 0x7f289806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.716+0000 7fbee45fd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed4044b30 0x7fbed40db2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.716+0000 7fbee45fd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbed40d62e0 0x7fbed4048d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee2399640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed4044b30 0x7fbed40db2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee45fd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed4049260 0x7fbed40d8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee2399640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed4044b30 0x7fbed40db2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40580/0 (socket says 172.21.15.139:40580) 2024-09-16T00:07:43.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.716+0000 7f28a8ff9640 1 -- 172.21.15.139:0/4183654677 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f28a008b290 con 0x7f28ac0cd8f0 2024-09-16T00:07:43.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.716+0000 7f28ab7fe640 1 --2- 172.21.15.139:0/4183654677 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f289806c380 0x7f289806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee45fd640 1 -- 172.21.15.139:0/769999980 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbed40dc930 con 0x7fbed4044b30 2024-09-16T00:07:43.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee2399640 1 -- 172.21.15.139:0/769999980 learned_addr learned my addr 172.21.15.139:0/769999980 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:43.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee45fd640 1 -- 172.21.15.139:0/769999980 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbed404b730 con 0x7fbed4049260 2024-09-16T00:07:43.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee45fd640 1 -- 172.21.15.139:0/769999980 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbed4035cf0 con 0x7fbed40d62e0 2024-09-16T00:07:43.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee1b98640 1 --2- 172.21.15.139:0/769999980 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbed40d62e0 0x7fbed4048d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee2b9a640 1 --2- 172.21.15.139:0/769999980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed4049260 0x7fbed40d8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7f28ab7fe640 1 --2- 172.21.15.139:0/4183654677 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f289806c380 0x7f289806e840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f28a4033000 tx=0x7f28a4009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee2399640 1 -- 172.21.15.139:0/769999980 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbed40d62e0 msgr2=0x7fbed4048d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee2399640 1 --2- 172.21.15.139:0/769999980 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbed40d62e0 0x7fbed4048d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee2399640 1 -- 172.21.15.139:0/769999980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed4049260 msgr2=0x7fbed40d8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:43.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee2399640 1 --2- 172.21.15.139:0/769999980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed4049260 0x7fbed40d8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:43.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee2399640 1 -- 172.21.15.139:0/769999980 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed40d7800 con 0x7fbed4044b30 2024-09-16T00:07:43.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee2b9a640 1 --2- 172.21.15.139:0/769999980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed4049260 0x7fbed40d8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:43.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee1b98640 1 --2- 172.21.15.139:0/769999980 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbed40d62e0 0x7fbed4048d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:43.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7f37ba4e7640 1 -- 172.21.15.139:0/1320260272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ac04bdc0 con 0x7f37ac007290 2024-09-16T00:07:43.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.717+0000 7fbee2399640 1 --2- 172.21.15.139:0/769999980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed4044b30 0x7fbed40db2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009960 tx=0x7fbed80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.718+0000 7f18db7fe640 1 -- 172.21.15.139:0/219719612 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f18cc0597a0 con 0x7f18e4073e20 2024-09-16T00:07:43.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.721+0000 7f28a8ff9640 1 -- 172.21.15.139:0/4183654677 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f28a00599e0 con 0x7f28ac0cd8f0 2024-09-16T00:07:43.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.721+0000 7fbed2ffd640 1 -- 172.21.15.139:0/769999980 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8037040 con 0x7fbed4044b30 2024-09-16T00:07:43.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.722+0000 7fbed2ffd640 1 -- 172.21.15.139:0/769999980 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed8042070 con 0x7fbed4044b30 2024-09-16T00:07:43.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.722+0000 7fbed2ffd640 1 -- 172.21.15.139:0/769999980 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8030bd0 con 0x7fbed4044b30 2024-09-16T00:07:43.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.722+0000 7fbee45fd640 1 -- 172.21.15.139:0/769999980 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed40d7ae0 con 0x7fbed4044b30 2024-09-16T00:07:43.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.722+0000 7fbee45fd640 1 -- 172.21.15.139:0/769999980 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed40d20a0 con 0x7fbed4044b30 2024-09-16T00:07:43.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.722+0000 7f77e77fe640 1 -- 172.21.15.139:0/2512889859 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8041020 con 0x7f77f0073030 2024-09-16T00:07:43.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.722+0000 7f77e77fe640 1 -- 172.21.15.139:0/2512889859 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77e80087b0 con 0x7f77f0073030 2024-09-16T00:07:43.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.722+0000 7f7c66ffd640 1 -- 172.21.15.139:0/3730708142 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68005480 con 0x7f7c7806fbc0 2024-09-16T00:07:43.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.723+0000 7f7c66ffd640 1 -- 172.21.15.139:0/3730708142 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c680035f0 con 0x7f7c7806fbc0 2024-09-16T00:07:43.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.723+0000 7fcc28ff9640 1 -- 172.21.15.139:0/2056087510 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20040020 con 0x7fcc2c0cab80 2024-09-16T00:07:43.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.723+0000 7fcc28ff9640 1 -- 172.21.15.139:0/2056087510 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc20038040 con 0x7fcc2c0cab80 2024-09-16T00:07:43.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.723+0000 7fcc28ff9640 1 -- 172.21.15.139:0/2056087510 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc20037070 con 0x7fcc2c0cab80 2024-09-16T00:07:43.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.723+0000 7fcc3268d640 1 -- 172.21.15.139:0/2056087510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2c154a20 con 0x7fcc2c0cab80 2024-09-16T00:07:43.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.723+0000 7f7c7759e640 1 -- 172.21.15.139:0/3730708142 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c781560f0 con 0x7f7c7806fbc0 2024-09-16T00:07:43.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.722+0000 7f37a9ffb640 1 -- 172.21.15.139:0/1320260272 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f37b0018020 con 0x7f37ac007290 2024-09-16T00:07:43.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.723+0000 7fbee45fd640 1 -- 172.21.15.139:0/769999980 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed4050da0 con 0x7fbed4044b30 2024-09-16T00:07:43.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.723+0000 7f77e77fe640 1 -- 172.21.15.139:0/2512889859 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8036040 con 0x7f77f0073030 2024-09-16T00:07:43.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.724+0000 7f77f544c640 1 -- 172.21.15.139:0/2512889859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f01f3a20 con 0x7f77f0073030 2024-09-16T00:07:43.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.724+0000 7f77f544c640 1 -- 172.21.15.139:0/2512889859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f01f3fe0 con 0x7f77f0073030 2024-09-16T00:07:43.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.723+0000 7f7c66ffd640 1 -- 172.21.15.139:0/3730708142 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c680039c0 con 0x7f7c7806fbc0 2024-09-16T00:07:43.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.723+0000 7f7c7759e640 1 -- 172.21.15.139:0/3730708142 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c78155c90 con 0x7f7c7806fbc0 2024-09-16T00:07:43.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.723+0000 7f37a9ffb640 1 --2- 172.21.15.139:0/1320260272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f379806c3b0 0x7f379806e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.723+0000 7f37a9ffb640 1 -- 172.21.15.139:0/1320260272 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f37b008b860 con 0x7f37ac007290 2024-09-16T00:07:43.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.723+0000 7efccdffb640 1 -- 172.21.15.139:0/1686338288 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efcd0041020 con 0x7efcd8070040 2024-09-16T00:07:43.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.723+0000 7fcc3268d640 1 -- 172.21.15.139:0/2056087510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc2c1f4070 con 0x7fcc2c0cab80 2024-09-16T00:07:43.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.724+0000 7f37b8ce4640 1 --2- 172.21.15.139:0/1320260272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f379806c3b0 0x7f379806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.725+0000 7fcc127fc640 1 -- 172.21.15.139:0/2056087510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc2c15b0f0 con 0x7fcc2c0cab80 2024-09-16T00:07:43.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.725+0000 7f37b8ce4640 1 --2- 172.21.15.139:0/1320260272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f379806c3b0 0x7f379806e870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0da2c0 tx=0x7f37a40090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.726+0000 7f7c7759e640 1 -- 172.21.15.139:0/3730708142 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c78155ef0 con 0x7f7c7806fbc0 2024-09-16T00:07:43.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.726+0000 7f77e77fe640 1 -- 172.21.15.139:0/2512889859 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f77e8044020 con 0x7f77f0073030 2024-09-16T00:07:43.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.726+0000 7f77f544c640 1 -- 172.21.15.139:0/2512889859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f0154e60 con 0x7f77f0073030 2024-09-16T00:07:43.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.726+0000 7f7c66ffd640 1 -- 172.21.15.139:0/3730708142 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7c68003b60 con 0x7f7c7806fbc0 2024-09-16T00:07:43.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.726+0000 7f77e77fe640 1 --2- 172.21.15.139:0/2512889859 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f77d006c300 0x7f77d006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.726+0000 7f7c66ffd640 1 --2- 172.21.15.139:0/3730708142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7c5806c4a0 0x7f7c5806e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.727+0000 7f77ee7fc640 1 --2- 172.21.15.139:0/2512889859 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f77d006c300 0x7f77d006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.727+0000 7f7c66ffd640 1 -- 172.21.15.139:0/3730708142 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7c6808af50 con 0x7f7c7806fbc0 2024-09-16T00:07:43.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.727+0000 7f77e77fe640 1 -- 172.21.15.139:0/2512889859 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f77e80abd30 con 0x7f77f0073030 2024-09-16T00:07:43.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.727+0000 7f7c75d9b640 1 --2- 172.21.15.139:0/3730708142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7c5806c4a0 0x7f7c5806e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.727+0000 7f7c75d9b640 1 --2- 172.21.15.139:0/3730708142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7c5806c4a0 0x7f7c5806e960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c037000 tx=0x7f7c6c036040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.727+0000 7f77ee7fc640 1 --2- 172.21.15.139:0/2512889859 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f77d006c300 0x7f77d006e7c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f77d8000f60 tx=0x7f77d8007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.727+0000 7fcc28ff9640 1 -- 172.21.15.139:0/2056087510 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcc20043020 con 0x7fcc2c0cab80 2024-09-16T00:07:43.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.727+0000 7fcc28ff9640 1 --2- 172.21.15.139:0/2056087510 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcc1806c3b0 0x7fcc1806e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.727+0000 7fcc28ff9640 1 -- 172.21.15.139:0/2056087510 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcc200ab600 con 0x7fcc2c0cab80 2024-09-16T00:07:43.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.728+0000 7fcc2b7fe640 1 --2- 172.21.15.139:0/2056087510 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcc1806c3b0 0x7fcc1806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.728+0000 7fcc2b7fe640 1 --2- 172.21.15.139:0/2056087510 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcc1806c3b0 0x7fcc1806e870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c06cb20 tx=0x7fcc1c007e10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.729+0000 7f37a9ffb640 1 -- 172.21.15.139:0/1320260272 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f37b001f020 con 0x7f37ac007290 2024-09-16T00:07:43.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.730+0000 7fbed2ffd640 1 -- 172.21.15.139:0/769999980 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbed8047460 con 0x7fbed4044b30 2024-09-16T00:07:43.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.731+0000 7fbed2ffd640 1 --2- 172.21.15.139:0/769999980 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbec006c300 0x7fbec006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:43.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.731+0000 7fbed2ffd640 1 -- 172.21.15.139:0/769999980 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbed80ac980 con 0x7fbed4044b30 2024-09-16T00:07:43.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.731+0000 7f77e77fe640 1 -- 172.21.15.139:0/2512889859 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f77e804b030 con 0x7f77f0073030 2024-09-16T00:07:43.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.731+0000 7fbee1b98640 1 --2- 172.21.15.139:0/769999980 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbec006c300 0x7fbec006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:43.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.732+0000 7fbee1b98640 1 --2- 172.21.15.139:0/769999980 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbec006c300 0x7fbec006e7c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbed40e0aa0 tx=0x7fbecc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:43.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.735+0000 7f7c66ffd640 1 -- 172.21.15.139:0/3730708142 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7c680182b0 con 0x7f7c7806fbc0 2024-09-16T00:07:43.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.739+0000 7fbed2ffd640 1 -- 172.21.15.139:0/769999980 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbed807aa80 con 0x7fbed4044b30 2024-09-16T00:07:43.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:43.740+0000 7fcc28ff9640 1 -- 172.21.15.139:0/2056087510 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcc2004a030 con 0x7fcc2c0cab80 2024-09-16T00:07:44.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.101+0000 7f18e8800640 1 -- 172.21.15.139:0/219719612 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f18c8005580 con 0x7f18e4073e20 2024-09-16T00:07:44.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.102+0000 7f18db7fe640 1 -- 172.21.15.139:0/219719612 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f18cc0590a0 con 0x7f18e4073e20 2024-09-16T00:07:44.103 INFO:teuthology.orchestra.run.smithi139.stdout:64424509446 2024-09-16T00:07:44.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.105+0000 7f18d97fa640 1 -- 172.21.15.139:0/219719612 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f18c406c3b0 msgr2=0x7f18c406e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:44.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.105+0000 7f18d97fa640 1 --2- 172.21.15.139:0/219719612 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f18c406c3b0 0x7f18c406e870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f18d4034000 tx=0x7f18d402f000 comp rx=0 tx=0).stop 2024-09-16T00:07:44.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.105+0000 7f18d97fa640 1 -- 172.21.15.139:0/219719612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18e4073e20 msgr2=0x7f18e415a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:44.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.105+0000 7f18d97fa640 1 --2- 172.21.15.139:0/219719612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18e4073e20 0x7f18e415a070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f18cc00c450 tx=0x7f18cc00c910 comp rx=0 tx=0).stop 2024-09-16T00:07:44.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.106+0000 7f18d97fa640 1 -- 172.21.15.139:0/219719612 shutdown_connections 2024-09-16T00:07:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.106+0000 7f18d97fa640 1 --2- 172.21.15.139:0/219719612 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f18c406c3b0 0x7f18c406e870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.106+0000 7f18d97fa640 1 --2- 172.21.15.139:0/219719612 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18e40c6fc0 0x7f18e4154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.106+0000 7f18d97fa640 1 --2- 172.21.15.139:0/219719612 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18e4073e20 0x7f18e415a070 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.106+0000 7f18d97fa640 1 --2- 172.21.15.139:0/219719612 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f18e406ef10 0x7f18e41f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.106+0000 7f18d97fa640 1 -- 172.21.15.139:0/219719612 >> 172.21.15.139:0/219719612 conn(0x7f18e40cbe90 msgr2=0x7f18e415d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.106+0000 7f18d97fa640 1 -- 172.21.15.139:0/219719612 shutdown_connections 2024-09-16T00:07:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.106+0000 7f18d97fa640 1 -- 172.21.15.139:0/219719612 wait complete. 2024-09-16T00:07:44.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.120+0000 7f28b25fd640 1 -- 172.21.15.139:0/4183654677 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f28ac0cbc00 con 0x7f28ac0cd8f0 2024-09-16T00:07:44.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.121+0000 7f28a8ff9640 1 -- 172.21.15.139:0/4183654677 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f28a0010e20 con 0x7f28ac0cd8f0 2024-09-16T00:07:44.122 INFO:teuthology.orchestra.run.smithi139.stdout:64424509446 2024-09-16T00:07:44.123 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509446 for osd.6 2024-09-16T00:07:44.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.123+0000 7efcdf8f1640 1 -- 172.21.15.139:0/1686338288 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7efcb8005580 con 0x7efcd8070040 2024-09-16T00:07:44.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.124+0000 7efccdffb640 1 -- 172.21.15.139:0/1686338288 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efcd007a570 con 0x7efcd8070040 2024-09-16T00:07:44.125 INFO:teuthology.orchestra.run.smithi139.stdout:64424509446 2024-09-16T00:07:44.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.124+0000 7f28b25fd640 1 -- 172.21.15.139:0/4183654677 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f289806c380 msgr2=0x7f289806e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:44.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.124+0000 7f28b25fd640 1 --2- 172.21.15.139:0/4183654677 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f289806c380 0x7f289806e840 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f28a4033000 tx=0x7f28a4009000 comp rx=0 tx=0).stop 2024-09-16T00:07:44.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.124+0000 7f28b25fd640 1 -- 172.21.15.139:0/4183654677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28ac0cd8f0 msgr2=0x7f28ac06fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:44.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.125+0000 7f28b25fd640 1 --2- 172.21.15.139:0/4183654677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28ac0cd8f0 0x7f28ac06fc30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f28a000b480 tx=0x7f28a000b940 comp rx=0 tx=0).stop 2024-09-16T00:07:44.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.125+0000 7f28b25fd640 1 -- 172.21.15.139:0/4183654677 shutdown_connections 2024-09-16T00:07:44.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.125+0000 7f28b25fd640 1 --2- 172.21.15.139:0/4183654677 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f289806c380 0x7f289806e840 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.125+0000 7f28b25fd640 1 --2- 172.21.15.139:0/4183654677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28ac154960 0x7f28ac149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.125+0000 7f28b25fd640 1 --2- 172.21.15.139:0/4183654677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28ac0d0790 0x7f28ac0c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.125+0000 7f28b25fd640 1 --2- 172.21.15.139:0/4183654677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28ac0cd8f0 0x7f28ac06fc30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.125+0000 7f28b25fd640 1 -- 172.21.15.139:0/4183654677 >> 172.21.15.139:0/4183654677 conn(0x7f28ac06c850 msgr2=0x7f28ac0cb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:44.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.125+0000 7f28b25fd640 1 -- 172.21.15.139:0/4183654677 shutdown_connections 2024-09-16T00:07:44.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.125+0000 7f28b25fd640 1 -- 172.21.15.139:0/4183654677 wait complete. 2024-09-16T00:07:44.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.127+0000 7efcdf8f1640 1 -- 172.21.15.139:0/1686338288 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efcbc06c300 msgr2=0x7efcbc06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:44.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.127+0000 7efcdf8f1640 1 --2- 172.21.15.139:0/1686338288 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efcbc06c300 0x7efcbc06e7c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efcd400e250 tx=0x7efcd400b040 comp rx=0 tx=0).stop 2024-09-16T00:07:44.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.127+0000 7efcdf8f1640 1 -- 172.21.15.139:0/1686338288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcd8070040 msgr2=0x7efcd80c5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:44.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.127+0000 7efcdf8f1640 1 --2- 172.21.15.139:0/1686338288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcd8070040 0x7efcd80c5070 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efcd8073430 tx=0x7efcd00046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.127+0000 7efcdf8f1640 1 -- 172.21.15.139:0/1686338288 shutdown_connections 2024-09-16T00:07:44.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.127+0000 7efcdf8f1640 1 --2- 172.21.15.139:0/1686338288 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efcbc06c300 0x7efcbc06e7c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.127+0000 7efcdf8f1640 1 --2- 172.21.15.139:0/1686338288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcd8158f00 0x7efcd814a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.127+0000 7efcdf8f1640 1 --2- 172.21.15.139:0/1686338288 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efcd8070040 0x7efcd80c5070 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efcd8073430 tx=0x7efcd00046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.127+0000 7efcdf8f1640 1 --2- 172.21.15.139:0/1686338288 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efcd806fc20 0x7efcd814c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.127+0000 7efcdf8f1640 1 -- 172.21.15.139:0/1686338288 >> 172.21.15.139:0/1686338288 conn(0x7efcd806da40 msgr2=0x7efcd8071890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:44.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.127+0000 7efcdf8f1640 1 -- 172.21.15.139:0/1686338288 shutdown_connections 2024-09-16T00:07:44.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.127+0000 7efcdf8f1640 1 -- 172.21.15.139:0/1686338288 wait complete. 2024-09-16T00:07:44.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.131+0000 7f37ba4e7640 1 -- 172.21.15.139:0/1320260272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f37ac04d690 con 0x7f37ac007290 2024-09-16T00:07:44.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.131+0000 7f77f544c640 1 -- 172.21.15.139:0/2512889859 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f77f0157030 con 0x7f77f0073030 2024-09-16T00:07:44.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.131+0000 7f37a9ffb640 1 -- 172.21.15.139:0/1320260272 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f37b0059960 con 0x7f37ac007290 2024-09-16T00:07:44.133 INFO:teuthology.orchestra.run.smithi139.stdout:64424509445 2024-09-16T00:07:44.133 INFO:teuthology.orchestra.run.smithi139.stdout:64424509445 2024-09-16T00:07:44.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.132+0000 7f77e77fe640 1 -- 172.21.15.139:0/2512889859 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f77e8079ec0 con 0x7f77f0073030 2024-09-16T00:07:44.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.134+0000 7fcc127fc640 1 -- 172.21.15.139:0/2056087510 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fcc2c154dc0 con 0x7fcc2c0cab80 2024-09-16T00:07:44.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.135+0000 7f37ba4e7640 1 -- 172.21.15.139:0/1320260272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f379806c3b0 msgr2=0x7f379806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:44.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.135+0000 7f37ba4e7640 1 --2- 172.21.15.139:0/1320260272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f379806c3b0 0x7f379806e870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f37ac0da2c0 tx=0x7f37a40090e0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.135+0000 7f37ba4e7640 1 -- 172.21.15.139:0/1320260272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37ac007290 msgr2=0x7f37ac049cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:44.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.135+0000 7f37ba4e7640 1 --2- 172.21.15.139:0/1320260272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37ac007290 0x7f37ac049cb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f37b000b3c0 tx=0x7f37b000b880 comp rx=0 tx=0).stop 2024-09-16T00:07:44.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.135+0000 7f37ba4e7640 1 -- 172.21.15.139:0/1320260272 shutdown_connections 2024-09-16T00:07:44.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.135+0000 7f37ba4e7640 1 --2- 172.21.15.139:0/1320260272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f379806c3b0 0x7f379806e870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.135+0000 7f37ba4e7640 1 --2- 172.21.15.139:0/1320260272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37ac0db410 0x7f37ac0dcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.135+0000 7f37ba4e7640 1 --2- 172.21.15.139:0/1320260272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f37ac034f20 0x7f37ac047450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.135+0000 7f37ba4e7640 1 --2- 172.21.15.139:0/1320260272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f37ac007290 0x7f37ac049cb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.135+0000 7f37ba4e7640 1 -- 172.21.15.139:0/1320260272 >> 172.21.15.139:0/1320260272 conn(0x7f37ac0500a0 msgr2=0x7f37ac0cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:44.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.135+0000 7f37ba4e7640 1 -- 172.21.15.139:0/1320260272 shutdown_connections 2024-09-16T00:07:44.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.135+0000 7fcc28ff9640 1 -- 172.21.15.139:0/2056087510 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcc20079cc0 con 0x7fcc2c0cab80 2024-09-16T00:07:44.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.136+0000 7f37ba4e7640 1 -- 172.21.15.139:0/1320260272 wait complete. 2024-09-16T00:07:44.141 INFO:teuthology.orchestra.run.smithi139.stdout:60129542150 2024-09-16T00:07:44.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.136+0000 7f77f544c640 1 -- 172.21.15.139:0/2512889859 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f77d006c300 msgr2=0x7f77d006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:44.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.136+0000 7f77f544c640 1 --2- 172.21.15.139:0/2512889859 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f77d006c300 0x7f77d006e7c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f77d8000f60 tx=0x7f77d8007e40 comp rx=0 tx=0).stop 2024-09-16T00:07:44.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.136+0000 7f77f544c640 1 -- 172.21.15.139:0/2512889859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77f0073030 msgr2=0x7f77f01e13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:44.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.136+0000 7f77f544c640 1 --2- 172.21.15.139:0/2512889859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77f0073030 0x7f77f01e13d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009f70 tx=0x7f77e8004a40 comp rx=0 tx=0).stop 2024-09-16T00:07:44.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.137+0000 7f77f544c640 1 -- 172.21.15.139:0/2512889859 shutdown_connections 2024-09-16T00:07:44.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.137+0000 7f77f544c640 1 --2- 172.21.15.139:0/2512889859 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f77d006c300 0x7f77d006e7c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.137+0000 7f77f544c640 1 --2- 172.21.15.139:0/2512889859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f77f01e50a0 0x7f77f01e5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.137+0000 7f77f544c640 1 --2- 172.21.15.139:0/2512889859 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f77f0153550 0x7f77f00c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.137+0000 7f77f544c640 1 --2- 172.21.15.139:0/2512889859 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f77f0073030 0x7f77f01e13d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.137+0000 7f77f544c640 1 -- 172.21.15.139:0/2512889859 >> 172.21.15.139:0/2512889859 conn(0x7f77f006da40 msgr2=0x7f77f00c73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:44.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.137+0000 7f77f544c640 1 -- 172.21.15.139:0/2512889859 shutdown_connections 2024-09-16T00:07:44.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.137+0000 7f77f544c640 1 -- 172.21.15.139:0/2512889859 wait complete. 2024-09-16T00:07:44.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.138+0000 7f7c7759e640 1 -- 172.21.15.139:0/3730708142 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7c780c6da0 con 0x7f7c7806fbc0 2024-09-16T00:07:44.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.138+0000 7f7c66ffd640 1 -- 172.21.15.139:0/3730708142 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7c68059830 con 0x7f7c7806fbc0 2024-09-16T00:07:44.147 INFO:teuthology.orchestra.run.smithi139.stdout:60129542150 2024-09-16T00:07:44.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.139+0000 7fcc127fc640 1 -- 172.21.15.139:0/2056087510 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcc1806c3b0 msgr2=0x7fcc1806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:44.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.139+0000 7fcc127fc640 1 --2- 172.21.15.139:0/2056087510 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcc1806c3b0 0x7fcc1806e870 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c06cb20 tx=0x7fcc1c007e10 comp rx=0 tx=0).stop 2024-09-16T00:07:44.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.139+0000 7fcc127fc640 1 -- 172.21.15.139:0/2056087510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc2c0cab80 msgr2=0x7fcc2c15a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:44.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.139+0000 7fcc127fc640 1 --2- 172.21.15.139:0/2056087510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc2c0cab80 0x7fcc2c15a240 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcc20031000 tx=0x7fcc200046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.139+0000 7fbee45fd640 1 -- 172.21.15.139:0/769999980 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbed40d5230 con 0x7fbed4044b30 2024-09-16T00:07:44.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.140+0000 7fcc127fc640 1 -- 172.21.15.139:0/2056087510 shutdown_connections 2024-09-16T00:07:44.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.140+0000 7fcc127fc640 1 --2- 172.21.15.139:0/2056087510 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcc1806c3b0 0x7fcc1806e870 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.140+0000 7fcc127fc640 1 --2- 172.21.15.139:0/2056087510 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcc2c0ccb30 0x7fcc2c157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.140+0000 7fcc127fc640 1 --2- 172.21.15.139:0/2056087510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc2c0cba80 0x7fcc2c157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.140+0000 7fcc127fc640 1 --2- 172.21.15.139:0/2056087510 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcc2c0cab80 0x7fcc2c15a240 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.140+0000 7fbed2ffd640 1 -- 172.21.15.139:0/769999980 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbed807ed80 con 0x7fbed4044b30 2024-09-16T00:07:44.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.140+0000 7fcc127fc640 1 -- 172.21.15.139:0/2056087510 >> 172.21.15.139:0/2056087510 conn(0x7fcc2c06c310 msgr2=0x7fcc2c0c5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:44.152 INFO:teuthology.orchestra.run.smithi139.stdout:55834574854 2024-09-16T00:07:44.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.140+0000 7fcc127fc640 1 -- 172.21.15.139:0/2056087510 shutdown_connections 2024-09-16T00:07:44.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.140+0000 7fcc127fc640 1 -- 172.21.15.139:0/2056087510 wait complete. 2024-09-16T00:07:44.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.142+0000 7f7c7759e640 1 -- 172.21.15.139:0/3730708142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7c5806c4a0 msgr2=0x7f7c5806e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:44.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.142+0000 7f7c7759e640 1 --2- 172.21.15.139:0/3730708142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7c5806c4a0 0x7f7c5806e960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c037000 tx=0x7f7c6c036040 comp rx=0 tx=0).stop 2024-09-16T00:07:44.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7f7c7759e640 1 -- 172.21.15.139:0/3730708142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c7806fbc0 msgr2=0x7f7c78154bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:44.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7f7c7759e640 1 --2- 172.21.15.139:0/3730708142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c7806fbc0 0x7f7c78154bc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800c4c0 tx=0x7f7c6800c980 comp rx=0 tx=0).stop 2024-09-16T00:07:44.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7f7c7759e640 1 -- 172.21.15.139:0/3730708142 shutdown_connections 2024-09-16T00:07:44.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7f7c7759e640 1 --2- 172.21.15.139:0/3730708142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7c5806c4a0 0x7f7c5806e960 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c037000 tx=0x7f7c6c036040 comp rx=0 tx=0).stop 2024-09-16T00:07:44.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7f7c7759e640 1 --2- 172.21.15.139:0/3730708142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7c7815a6b0 0x7f7c78156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7f7c7759e640 1 --2- 172.21.15.139:0/3730708142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c780c5dc0 0x7f7c78155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7f7c7759e640 1 --2- 172.21.15.139:0/3730708142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7c7806fbc0 0x7f7c78154bc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7f7c7759e640 1 -- 172.21.15.139:0/3730708142 >> 172.21.15.139:0/3730708142 conn(0x7f7c780cbe90 msgr2=0x7f7c781530b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:44.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7fbee45fd640 1 -- 172.21.15.139:0/769999980 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbec006c300 msgr2=0x7fbec006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:44.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7f7c7759e640 1 -- 172.21.15.139:0/3730708142 shutdown_connections 2024-09-16T00:07:44.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7fbee45fd640 1 --2- 172.21.15.139:0/769999980 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbec006c300 0x7fbec006e7c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbed40e0aa0 tx=0x7fbecc008040 comp rx=0 tx=0).stop 2024-09-16T00:07:44.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7fbee45fd640 1 -- 172.21.15.139:0/769999980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed4044b30 msgr2=0x7fbed40db2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:44.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7fbee45fd640 1 --2- 172.21.15.139:0/769999980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed4044b30 0x7fbed40db2f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009960 tx=0x7fbed80046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7f7c7759e640 1 -- 172.21.15.139:0/3730708142 wait complete. 2024-09-16T00:07:44.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7fbee45fd640 1 -- 172.21.15.139:0/769999980 shutdown_connections 2024-09-16T00:07:44.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7fbee45fd640 1 --2- 172.21.15.139:0/769999980 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbec006c300 0x7fbec006e7c0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbed40e0aa0 tx=0x7fbecc008040 comp rx=0 tx=0).stop 2024-09-16T00:07:44.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7fbee45fd640 1 --2- 172.21.15.139:0/769999980 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbed4049260 0x7fbed40d8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7fbee45fd640 1 --2- 172.21.15.139:0/769999980 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbed40d62e0 0x7fbed4048d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7fbee45fd640 1 --2- 172.21.15.139:0/769999980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbed4044b30 0x7fbed40db2f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:44.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7fbee45fd640 1 -- 172.21.15.139:0/769999980 >> 172.21.15.139:0/769999980 conn(0x7fbed404fc50 msgr2=0x7fbed404dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:44.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.143+0000 7fbee45fd640 1 -- 172.21.15.139:0/769999980 shutdown_connections 2024-09-16T00:07:44.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:44.144+0000 7fbee45fd640 1 -- 172.21.15.139:0/769999980 wait complete. 2024-09-16T00:07:44.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509446 for osd.4 2024-09-16T00:07:44.163 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509446 for osd.7 2024-09-16T00:07:44.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.3 2024-09-16T00:07:44.165 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.0 2024-09-16T00:07:44.166 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-09-16T00:07:44.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.5 2024-09-16T00:07:44.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.1 2024-09-16T00:07:45.124 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T00:07:45.163 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T00:07:45.164 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T00:07:45.164 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T00:07:45.166 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T00:07:45.166 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T00:07:45.168 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T00:07:45.168 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T00:07:45.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.278+0000 7f7bac35b640 1 Processor -- start 2024-09-16T00:07:45.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.278+0000 7f7bac35b640 1 -- start start 2024-09-16T00:07:45.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.280+0000 7f7bac35b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ba40d0700 0x7f7ba41602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.280+0000 7f7bac35b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7ba40d0ae0 0x7f7ba4160820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.280+0000 7f7bac35b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ba40cc880 con 0x7f7ba40cbdf0 2024-09-16T00:07:45.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.280+0000 7f7bac35b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7ba40b6350 con 0x7f7ba40d0ae0 2024-09-16T00:07:45.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.280+0000 7f7bac35b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba40ccb60 con 0x7f7ba40d0700 2024-09-16T00:07:45.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.280+0000 7f7ba98f6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7ba40d0ae0 0x7f7ba4160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.280+0000 7f7ba98f6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7ba40d0ae0 0x7f7ba4160820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40588/0 (socket says 172.21.15.139:40588) 2024-09-16T00:07:45.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.280+0000 7f7ba98f6640 1 -- 172.21.15.139:0/3775976812 learned_addr learned my addr 172.21.15.139:0/3775976812 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:45.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.280+0000 7f7baa0f7640 1 --2- 172.21.15.139:0/3775976812 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ba40d0700 0x7f7ba41602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.281+0000 7f7ba98f6640 1 -- 172.21.15.139:0/3775976812 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ba40d0700 msgr2=0x7f7ba41602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:45.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.281+0000 7f7ba98f6640 1 --2- 172.21.15.139:0/3775976812 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ba40d0700 0x7f7ba41602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.281+0000 7f7ba98f6640 1 -- 172.21.15.139:0/3775976812 >> v1:172.21.15.164:6789/0 conn(0x7f7ba40cbdf0 legacy=0x7f7ba415f2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:07:45.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.281+0000 7f7ba98f6640 1 -- 172.21.15.139:0/3775976812 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba40cce40 con 0x7f7ba40d0ae0 2024-09-16T00:07:45.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.282+0000 7f7ba98f6640 1 --2- 172.21.15.139:0/3775976812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ba40d0ae0 0x7f7ba4160820 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009e50 tx=0x7f7b9c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d1d57b58e7fc63 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.282+0000 7f7ba90f5640 1 -- 172.21.15.139:0/3775976812 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c036070 con 0x7f7ba40d0ae0 2024-09-16T00:07:45.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.283+0000 7f7ba90f5640 1 -- 172.21.15.139:0/3775976812 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b9c031600 con 0x7f7ba40d0ae0 2024-09-16T00:07:45.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.284+0000 7f7bac35b640 1 -- 172.21.15.139:0/3775976812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ba40d0ae0 msgr2=0x7f7ba4160820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.284+0000 7f7bac35b640 1 --2- 172.21.15.139:0/3775976812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ba40d0ae0 0x7f7ba4160820 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009e50 tx=0x7f7b9c02a6f0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.288+0000 7f7bac35b640 1 -- 172.21.15.139:0/3775976812 shutdown_connections 2024-09-16T00:07:45.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.288+0000 7f7bac35b640 1 --2- 172.21.15.139:0/3775976812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ba40d0ae0 0x7f7ba4160820 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.288+0000 7f7bac35b640 1 --2- 172.21.15.139:0/3775976812 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ba40d0700 0x7f7ba41602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.288+0000 7f7bac35b640 1 -- 172.21.15.139:0/3775976812 >> 172.21.15.139:0/3775976812 conn(0x7f7ba406c500 msgr2=0x7f7ba406c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.288+0000 7f7bac35b640 1 -- 172.21.15.139:0/3775976812 shutdown_connections 2024-09-16T00:07:45.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.288+0000 7f7bac35b640 1 -- 172.21.15.139:0/3775976812 wait complete. 2024-09-16T00:07:45.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.290+0000 7f7bac35b640 1 Processor -- start 2024-09-16T00:07:45.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.291+0000 7f7bac35b640 1 -- start start 2024-09-16T00:07:45.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.292+0000 7f7bac35b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ba40cbdf0 0x7f7ba40cf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.292+0000 7f7baa0f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ba40cbdf0 0x7f7ba40cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.292+0000 7f7baa0f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ba40cbdf0 0x7f7ba40cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40592/0 (socket says 172.21.15.139:40592) 2024-09-16T00:07:45.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.292+0000 7f7baa0f7640 1 -- 172.21.15.139:0/1165115653 learned_addr learned my addr 172.21.15.139:0/1165115653 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:45.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.293+0000 7f7bac35b640 1 --2- 172.21.15.139:0/1165115653 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ba40d0700 0x7f7ba41f8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.293+0000 7f7bac35b640 1 --2- 172.21.15.139:0/1165115653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ba40d0ae0 0x7f7ba41f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.293+0000 7f7bac35b640 1 -- 172.21.15.139:0/1165115653 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba40caf00 con 0x7f7ba40cbdf0 2024-09-16T00:07:45.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.293+0000 7f7bac35b640 1 -- 172.21.15.139:0/1165115653 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba40cd400 con 0x7f7ba40d0ae0 2024-09-16T00:07:45.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.293+0000 7f7ba98f6640 1 --2- 172.21.15.139:0/1165115653 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ba40d0700 0x7f7ba41f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.293+0000 7f7bac35b640 1 -- 172.21.15.139:0/1165115653 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba40cb080 con 0x7f7ba40d0700 2024-09-16T00:07:45.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.294+0000 7f7baa0f7640 1 -- 172.21.15.139:0/1165115653 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ba40d0700 msgr2=0x7f7ba41f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.294+0000 7f7baa0f7640 1 --2- 172.21.15.139:0/1165115653 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ba40d0700 0x7f7ba41f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.294+0000 7f7baa0f7640 1 -- 172.21.15.139:0/1165115653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ba40d0ae0 msgr2=0x7f7ba41f86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.294+0000 7f7baa0f7640 1 --2- 172.21.15.139:0/1165115653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ba40d0ae0 0x7f7ba41f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.294+0000 7f7baa0f7640 1 -- 172.21.15.139:0/1165115653 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba4160f90 con 0x7f7ba40cbdf0 2024-09-16T00:07:45.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.294+0000 7f7baa0f7640 1 --2- 172.21.15.139:0/1165115653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ba40cbdf0 0x7f7ba40cf610 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000d800 tx=0x7f7ba000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.294+0000 7f7b9affd640 1 -- 172.21.15.139:0/1165115653 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0018020 con 0x7f7ba40cbdf0 2024-09-16T00:07:45.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.294+0000 7f7bac35b640 1 -- 172.21.15.139:0/1165115653 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba41f4c70 con 0x7f7ba40cbdf0 2024-09-16T00:07:45.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.295+0000 7f7bac35b640 1 -- 172.21.15.139:0/1165115653 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba41f5230 con 0x7f7ba40cbdf0 2024-09-16T00:07:45.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.296+0000 7f7b9affd640 1 -- 172.21.15.139:0/1165115653 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba00096a0 con 0x7f7ba40cbdf0 2024-09-16T00:07:45.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.296+0000 7f7b9affd640 1 -- 172.21.15.139:0/1165115653 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0009a50 con 0x7f7ba40cbdf0 2024-09-16T00:07:45.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.297+0000 7f7bac35b640 1 -- 172.21.15.139:0/1165115653 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b8c005290 con 0x7f7ba40cbdf0 2024-09-16T00:07:45.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.308+0000 7f7b9affd640 1 -- 172.21.15.139:0/1165115653 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7ba001b020 con 0x7f7ba40cbdf0 2024-09-16T00:07:45.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.308+0000 7f7b9affd640 1 --2- 172.21.15.139:0/1165115653 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b8806c2b0 0x7f7b8806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.309+0000 7f7ba98f6640 1 --2- 172.21.15.139:0/1165115653 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b8806c2b0 0x7f7b8806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.310+0000 7f7b9affd640 1 -- 172.21.15.139:0/1165115653 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7ba008b600 con 0x7f7ba40cbdf0 2024-09-16T00:07:45.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.310+0000 7f7ba98f6640 1 --2- 172.21.15.139:0/1165115653 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b8806c2b0 0x7f7b8806e770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c033000 tx=0x7f7b9c009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.310+0000 7f7b9affd640 1 -- 172.21.15.139:0/1165115653 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7ba0022030 con 0x7f7ba40cbdf0 2024-09-16T00:07:45.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.337+0000 7ffa3613b640 1 Processor -- start 2024-09-16T00:07:45.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.337+0000 7ffa3613b640 1 -- start start 2024-09-16T00:07:45.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.338+0000 7f49acadd640 1 Processor -- start 2024-09-16T00:07:45.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.338+0000 7ffa3613b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffa30158020 0x7ffa30162e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7ffa3613b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa300c8170 con 0x7ffa300c71e0 2024-09-16T00:07:45.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7ffa2f7fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ffa30151800 0x7ffa3006eaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:35868/0 (socket says 172.21.15.139:35868) 2024-09-16T00:07:45.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7ffa2f7fe640 1 -- 172.21.15.139:0/1417701659 learned_addr learned my addr 172.21.15.139:0/1417701659 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:45.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7ffa3613b640 1 -- 172.21.15.139:0/1417701659 --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa300c8450 con 0x7ffa30151800 2024-09-16T00:07:45.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7ffa2ffff640 1 --2- 172.21.15.139:0/1417701659 >> v2:172.21.15.139:3300/0 conn(0x7ffa30158020 0x7ffa30162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7ffa3613b640 1 -- 172.21.15.139:0/1417701659 --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ffa300c7f20 con 0x7ffa30158020 2024-09-16T00:07:45.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7f732599f640 1 Processor -- start 2024-09-16T00:07:45.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7ffa2e7fc640 1 -- 172.21.15.139:0/1417701659 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1745173118 0 0) 0x7ffa300c8450 con 0x7ffa30151800 2024-09-16T00:07:45.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7f732599f640 1 -- start start 2024-09-16T00:07:45.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7ffa2e7fc640 1 -- 172.21.15.139:0/1417701659 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa18003520 con 0x7ffa30151800 2024-09-16T00:07:45.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.337+0000 7fd64283c640 1 Processor -- start 2024-09-16T00:07:45.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7ffa2e7fc640 1 -- 172.21.15.139:0/1417701659 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 855800152 0 0) 0x7ffa300c8170 con 0x7ffa300c71e0 2024-09-16T00:07:45.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7ffa2e7fc640 1 -- 172.21.15.139:0/1417701659 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa300c8450 con 0x7ffa300c71e0 2024-09-16T00:07:45.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.338+0000 7f49acadd640 1 -- start start 2024-09-16T00:07:45.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7f49acadd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f49a8150f10 0x7f49a80c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7f49acadd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f49a80c3520 0x7f49a80c8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7f49acadd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49a815a2b0 0x7f49a815a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7f49acadd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f49a80cbcf0 con 0x7f49a80c3520 2024-09-16T00:07:45.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7f49acadd640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f49a80c74f0 con 0x7f49a8150f10 2024-09-16T00:07:45.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7f49acadd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f49a80cb730 con 0x7f49a815a2b0 2024-09-16T00:07:45.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.337+0000 7fd64283c640 1 -- start start 2024-09-16T00:07:45.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7ffa2e7fc640 1 -- 172.21.15.139:0/1417701659 <== mon.1 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3515672334 0 0) 0x7ffa18003520 con 0x7ffa30151800 2024-09-16T00:07:45.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7fd64283c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd63c0c8510 0x7fd63c15a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7ffa2e7fc640 1 -- 172.21.15.139:0/1417701659 >> v2:172.21.15.139:3300/0 conn(0x7ffa30158020 msgr2=0x7ffa30162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:45.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7ffa2e7fc640 1 --2- 172.21.15.139:0/1417701659 >> v2:172.21.15.139:3300/0 conn(0x7ffa30158020 0x7ffa30162e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7ffa2e7fc640 1 -- 172.21.15.139:0/1417701659 >> v1:172.21.15.139:6789/0 conn(0x7ffa300c71e0 legacy=0x7ffa30157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7fd64283c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd63c0c3520 0x7fd63c15a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7fd64283c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63c0cbcf0 con 0x7fd63c150f10 2024-09-16T00:07:45.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7fd64283c640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fd63c0c74f0 con 0x7fd63c0c8510 2024-09-16T00:07:45.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7fd64283c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd63c0cb730 con 0x7fd63c0c3520 2024-09-16T00:07:45.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7ffa2e7fc640 1 -- 172.21.15.139:0/1417701659 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa300c8730 con 0x7ffa30151800 2024-09-16T00:07:45.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7ffa2e7fc640 1 -- 172.21.15.139:0/1417701659 <== mon.1 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7ffa20001670 con 0x7ffa30151800 2024-09-16T00:07:45.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7ffa2e7fc640 1 -- 172.21.15.139:0/1417701659 >> v1:172.21.15.164:6789/0 conn(0x7ffa30151800 legacy=0x7ffa3006eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7fd640dd9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd63c0c3520 0x7fd63c15a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7fd6337fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd63c0c8510 0x7fd63c15a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7ffa2e7fc640 1 --2- 172.21.15.139:0/1417701659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa18003b40 0x7ffa18003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7ffa2e7fc640 1 --2- 172.21.15.139:0/1417701659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa18004800 0x7ffa18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7ffa2e7fc640 1 --2- 172.21.15.139:0/1417701659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa180051c0 0x7ffa1800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7ffa2e7fc640 1 -- 172.21.15.139:0/1417701659 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffa300c7f20 con 0x7ffa18003b40 2024-09-16T00:07:45.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7ffa2e7fc640 1 -- 172.21.15.139:0/1417701659 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1800e300 con 0x7ffa180051c0 2024-09-16T00:07:45.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7ffa2e7fc640 1 -- 172.21.15.139:0/1417701659 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1800e500 con 0x7ffa18004800 2024-09-16T00:07:45.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7fd640dd9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd63c0c3520 0x7fd63c15a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:55236/0 (socket says 172.21.15.139:55236) 2024-09-16T00:07:45.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7ffa2f7fe640 1 --2- 172.21.15.139:0/1417701659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa18003b40 0x7ffa18003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.339+0000 7fd640dd9640 1 -- 172.21.15.139:0/2061072472 learned_addr learned my addr 172.21.15.139:0/2061072472 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:45.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7ffa2ffff640 1 --2- 172.21.15.139:0/1417701659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa180051c0 0x7ffa1800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7ffa2effd640 1 --2- 172.21.15.139:0/1417701659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa18004800 0x7ffa18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7fd640dd9640 1 -- 172.21.15.139:0/2061072472 >> v2:172.21.15.164:3300/0 conn(0x7fd63c0c8510 msgr2=0x7fd63c15a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:45.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7f732599f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7320073a80 0x7f73200c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7fd640dd9640 1 --2- 172.21.15.139:0/2061072472 >> v2:172.21.15.164:3300/0 conn(0x7fd63c0c8510 0x7fd63c15a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7fd640dd9640 1 -- 172.21.15.139:0/2061072472 >> v1:172.21.15.164:6789/0 conn(0x7fd63c150f10 legacy=0x7fd63c0c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:45.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7fd640dd9640 1 -- 172.21.15.139:0/2061072472 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c0cba10 con 0x7fd63c0c3520 2024-09-16T00:07:45.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7f732499d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7320073a80 0x7f73200c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7f732499d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7320073a80 0x7f73200c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:55268/0 (socket says 172.21.15.139:55268) 2024-09-16T00:07:45.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7f49a6d9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49a815a2b0 0x7f49a815a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7f49a659c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f49a8150f10 0x7f49a80c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7f49a659c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f49a8150f10 0x7f49a80c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39648/0 (socket says 172.21.15.139:39648) 2024-09-16T00:07:45.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7f49a659c640 1 -- 172.21.15.139:0/31141625 learned_addr learned my addr 172.21.15.139:0/31141625 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:45.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7f49a659c640 1 -- 172.21.15.139:0/31141625 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49a815a2b0 msgr2=0x7f49a815a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:45.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7f49a659c640 1 --2- 172.21.15.139:0/31141625 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49a815a2b0 0x7f49a815a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.340+0000 7f49a659c640 1 -- 172.21.15.139:0/31141625 >> v2:172.21.15.139:3300/0 conn(0x7f49a80c3520 msgr2=0x7f49a80c8340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:07:45.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7ffa2f7fe640 1 -- 172.21.15.139:0/1417701659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa18004800 msgr2=0x7ffa18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:45.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7ffa2f7fe640 1 --2- 172.21.15.139:0/1417701659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa18004800 0x7ffa18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7ffa2f7fe640 1 -- 172.21.15.139:0/1417701659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa180051c0 msgr2=0x7ffa1800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:45.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7ffa2f7fe640 1 --2- 172.21.15.139:0/1417701659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa180051c0 0x7ffa1800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7ffa2f7fe640 1 -- 172.21.15.139:0/1417701659 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa300c8730 con 0x7ffa18003b40 2024-09-16T00:07:45.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7f732599f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f73200c76e0 0x7f73201587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7f732499d640 1 -- 172.21.15.139:0/1301872108 learned_addr learned my addr 172.21.15.139:0/1301872108 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:45.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7f732599f640 1 -- 172.21.15.139:0/1301872108 --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73200c7dd0 con 0x7f7320073e60 2024-09-16T00:07:45.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7f732599f640 1 -- 172.21.15.139:0/1301872108 --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f73200c3ad0 con 0x7f73200c76e0 2024-09-16T00:07:45.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7f732599f640 1 -- 172.21.15.139:0/1301872108 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f73200c70a0 con 0x7f7320073a80 2024-09-16T00:07:45.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7ffa2effd640 1 --2- 172.21.15.139:0/1417701659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa18004800 0x7ffa18004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:45.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7ffa2ffff640 1 -- 172.21.15.139:0/1417701659 reap_dead start 2024-09-16T00:07:45.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7f80081fd640 1 Processor -- start 2024-09-16T00:07:45.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7f732519e640 1 --2- 172.21.15.139:0/1301872108 >> v2:172.21.15.164:3300/0 conn(0x7f73200c76e0 0x7f73201587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7f49a5d9b640 1 --2- 172.21.15.139:0/31141625 >> v2:172.21.15.139:3300/0 conn(0x7f49a80c3520 0x7f49a80c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7f49a659c640 1 --2- 172.21.15.139:0/31141625 >> v2:172.21.15.139:3300/0 conn(0x7f49a80c3520 0x7f49a80c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7f49a659c640 1 -- 172.21.15.139:0/31141625 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a80cba10 con 0x7f49a8150f10 2024-09-16T00:07:45.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7f49a5d9b640 1 --2- 172.21.15.139:0/31141625 >> v2:172.21.15.139:3300/0 conn(0x7f49a80c3520 0x7f49a80c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:45.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7f731f7fe640 1 -- 172.21.15.139:0/1301872108 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1925883367 0 0) 0x7f73200c7dd0 con 0x7f7320073e60 2024-09-16T00:07:45.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7f731f7fe640 1 -- 172.21.15.139:0/1301872108 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7308003520 con 0x7f7320073e60 2024-09-16T00:07:45.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f5553533640 1 Processor -- start 2024-09-16T00:07:45.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7fd640dd9640 1 --2- 172.21.15.139:0/2061072472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd63c0c3520 0x7fd63c15a7f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd634009e50 tx=0x7fd63402c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=198b44f2268a6174 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731f7fe640 1 -- 172.21.15.139:0/1301872108 <== mon.1 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3015156527 0 0) 0x7f7308003520 con 0x7f7320073e60 2024-09-16T00:07:45.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f5553533640 1 -- start start 2024-09-16T00:07:45.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731f7fe640 1 -- 172.21.15.139:0/1301872108 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7320073a80 msgr2=0x7f73200c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:45.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731f7fe640 1 --2- 172.21.15.139:0/1301872108 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7320073a80 0x7f73200c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731f7fe640 1 -- 172.21.15.139:0/1301872108 >> v2:172.21.15.164:3300/0 conn(0x7f73200c76e0 msgr2=0x7f73201587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:45.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731f7fe640 1 --2- 172.21.15.139:0/1301872108 >> v2:172.21.15.164:3300/0 conn(0x7f73200c76e0 0x7f73201587c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731f7fe640 1 -- 172.21.15.139:0/1301872108 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73200c3510 con 0x7f7320073e60 2024-09-16T00:07:45.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731f7fe640 1 -- 172.21.15.139:0/1301872108 <== mon.1 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7f7310001670 con 0x7f7320073e60 2024-09-16T00:07:45.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731f7fe640 1 -- 172.21.15.139:0/1301872108 >> v1:172.21.15.164:6789/0 conn(0x7f7320073e60 legacy=0x7f73201524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731f7fe640 1 --2- 172.21.15.139:0/1301872108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73080039b0 0x7f7308003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.341+0000 7f80081fd640 1 -- start start 2024-09-16T00:07:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f80081fd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f800006fc20 0x7f80000cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f80081fd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8000070040 0x7f80000cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f80081fd640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80000727d0 con 0x7f8000158f00 2024-09-16T00:07:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f80081fd640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f80000c8c90 con 0x7f800006fc20 2024-09-16T00:07:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f80081fd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8000072ab0 con 0x7f8000070040 2024-09-16T00:07:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f800679a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8000070040 0x7f80000cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f8005798640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f800006fc20 0x7f80000cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f800679a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8000070040 0x7f80000cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:55286/0 (socket says 172.21.15.139:55286) 2024-09-16T00:07:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f800679a640 1 -- 172.21.15.139:0/1984277798 learned_addr learned my addr 172.21.15.139:0/1984277798 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f49a659c640 1 --2- 172.21.15.139:0/31141625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49a8150f10 0x7f49a80c7e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f49a815b370 tx=0x7f499802e880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23d01b1d5808556a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f5553533640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55440dcdc0 0x7f5544045e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f5553533640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f55440e3100 0x7f554404aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f5553533640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55440e3570 con 0x7f554404a6b0 2024-09-16T00:07:45.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f5553533640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5544045980 con 0x7f55440e3100 2024-09-16T00:07:45.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7fd63b7fe640 1 -- 172.21.15.139:0/2061072472 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634033610 con 0x7fd63c0c3520 2024-09-16T00:07:45.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7fd63b7fe640 1 -- 172.21.15.139:0/2061072472 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6340337b0 con 0x7fd63c0c3520 2024-09-16T00:07:45.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7fd63b7fe640 1 -- 172.21.15.139:0/2061072472 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634033bc0 con 0x7fd63c0c3520 2024-09-16T00:07:45.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f5553533640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f55440dd1a0 con 0x7f55440dcdc0 2024-09-16T00:07:45.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f144cae6640 1 Processor -- start 2024-09-16T00:07:45.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f49a559a640 1 -- 172.21.15.139:0/31141625 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499803d070 con 0x7f49a8150f10 2024-09-16T00:07:45.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7ffa2f7fe640 1 --2- 172.21.15.139:0/1417701659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa18003b40 0x7ffa18003f40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000c240 tx=0x7ffa18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95a13ce0c2ede73 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7ffa2e7fc640 1 -- 172.21.15.139:0/1417701659 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20007d30 con 0x7ffa18003b40 2024-09-16T00:07:45.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f49a559a640 1 -- 172.21.15.139:0/31141625 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49980356f0 con 0x7f49a8150f10 2024-09-16T00:07:45.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f49a559a640 1 -- 172.21.15.139:0/31141625 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998035ac0 con 0x7f49a8150f10 2024-09-16T00:07:45.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f800679a640 1 -- 172.21.15.139:0/1984277798 >> v2:172.21.15.164:3300/0 conn(0x7f800006fc20 msgr2=0x7f80000cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:45.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f800679a640 1 --2- 172.21.15.139:0/1984277798 >> v2:172.21.15.164:3300/0 conn(0x7f800006fc20 0x7f80000cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f800679a640 1 -- 172.21.15.139:0/1984277798 >> v1:172.21.15.164:6789/0 conn(0x7f8000158f00 legacy=0x7f80000c6040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:07:45.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f800679a640 1 -- 172.21.15.139:0/1984277798 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8000072d90 con 0x7f8000070040 2024-09-16T00:07:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7fd64283c640 1 -- 172.21.15.139:0/2061072472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd63c0c3520 msgr2=0x7fd63c15a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7fd64283c640 1 --2- 172.21.15.139:0/2061072472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd63c0c3520 0x7fd63c15a7f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd634009e50 tx=0x7fd63402c900 comp rx=0 tx=0).stop 2024-09-16T00:07:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7fd64283c640 1 -- 172.21.15.139:0/2061072472 shutdown_connections 2024-09-16T00:07:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7fd64283c640 1 --2- 172.21.15.139:0/2061072472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd63c0c3520 0x7fd63c15a7f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7fd64283c640 1 --2- 172.21.15.139:0/2061072472 >> v2:172.21.15.164:3300/0 conn(0x7fd63c0c8510 0x7fd63c15a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7fd64283c640 1 -- 172.21.15.139:0/2061072472 >> 172.21.15.139:0/2061072472 conn(0x7fd63c0c91d0 msgr2=0x7fd63c0c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7ffa2e7fc640 1 -- 172.21.15.139:0/1417701659 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa20005af0 con 0x7ffa18003b40 2024-09-16T00:07:45.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7ffa2e7fc640 1 -- 172.21.15.139:0/1417701659 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20007070 con 0x7ffa18003b40 2024-09-16T00:07:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f55512cf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55440dcdc0 0x7f5544045e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731f7fe640 1 --2- 172.21.15.139:0/1301872108 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7308004800 0x7f7308004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731f7fe640 1 --2- 172.21.15.139:0/1301872108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73080051c0 0x7f730800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731f7fe640 1 -- 172.21.15.139:0/1301872108 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73200c3ad0 con 0x7f73080051c0 2024-09-16T00:07:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731f7fe640 1 -- 172.21.15.139:0/1301872108 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f73200c70a0 con 0x7f73080039b0 2024-09-16T00:07:45.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731f7fe640 1 -- 172.21.15.139:0/1301872108 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f730800c010 con 0x7f7308004800 2024-09-16T00:07:45.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f732519e640 1 --2- 172.21.15.139:0/1301872108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73080051c0 0x7f730800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731ffff640 1 --2- 172.21.15.139:0/1301872108 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7308004800 0x7f7308004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f55512cf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55440dcdc0 0x7f5544045e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:55300/0 (socket says 172.21.15.139:55300) 2024-09-16T00:07:45.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f55512cf640 1 -- 172.21.15.139:0/3852324103 learned_addr learned my addr 172.21.15.139:0/3852324103 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:45.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f732499d640 1 --2- 172.21.15.139:0/1301872108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73080039b0 0x7f7308003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731ffff640 1 -- 172.21.15.139:0/1301872108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73080039b0 msgr2=0x7f7308003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:45.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731ffff640 1 --2- 172.21.15.139:0/1301872108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73080039b0 0x7f7308003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731ffff640 1 -- 172.21.15.139:0/1301872108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73080051c0 msgr2=0x7f730800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:45.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731ffff640 1 --2- 172.21.15.139:0/1301872108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73080051c0 0x7f730800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.342+0000 7f731ffff640 1 -- 172.21.15.139:0/1301872108 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73200c3510 con 0x7f7308004800 2024-09-16T00:07:45.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.343+0000 7f732499d640 1 --2- 172.21.15.139:0/1301872108 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f73080039b0 0x7f7308003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:45.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.344+0000 7fd64283c640 1 -- 172.21.15.139:0/2061072472 shutdown_connections 2024-09-16T00:07:45.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.344+0000 7f731ffff640 1 --2- 172.21.15.139:0/1301872108 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7308004800 0x7f7308004c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f731000e570 tx=0x7f7308005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=100cf1e4c3866d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.344+0000 7f144cae6640 1 -- start start 2024-09-16T00:07:45.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.344+0000 7f731f7fe640 1 -- 172.21.15.139:0/1301872108 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310036070 con 0x7f7308004800 2024-09-16T00:07:45.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.344+0000 7f731f7fe640 1 -- 172.21.15.139:0/1301872108 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731000c450 con 0x7f7308004800 2024-09-16T00:07:45.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.344+0000 7f731f7fe640 1 -- 172.21.15.139:0/1301872108 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310007d70 con 0x7f7308004800 2024-09-16T00:07:45.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.344+0000 7f5551ad0640 1 --2- 172.21.15.139:0/3852324103 >> v2:172.21.15.164:3300/0 conn(0x7f55440e3100 0x7f554404aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f800679a640 1 --2- 172.21.15.139:0/1984277798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8000070040 0x7f80000cc690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009870 tx=0x7f7ffc02a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4897deccb321802 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.344+0000 7f55512cf640 1 -- 172.21.15.139:0/3852324103 >> v2:172.21.15.164:3300/0 conn(0x7f55440e3100 msgr2=0x7f554404aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:45.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.344+0000 7f55512cf640 1 --2- 172.21.15.139:0/3852324103 >> v2:172.21.15.164:3300/0 conn(0x7f55440e3100 0x7f554404aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.344+0000 7f55512cf640 1 -- 172.21.15.139:0/3852324103 >> v1:172.21.15.139:6789/0 conn(0x7f554404a6b0 legacy=0x7f55440e29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.344+0000 7f55512cf640 1 -- 172.21.15.139:0/3852324103 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55440e3820 con 0x7f55440dcdc0 2024-09-16T00:07:45.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7ffa3613b640 1 -- 172.21.15.139:0/1417701659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa18003b40 msgr2=0x7ffa18003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7ffa3613b640 1 --2- 172.21.15.139:0/1417701659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa18003b40 0x7ffa18003f40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000c240 tx=0x7ffa18005830 comp rx=0 tx=0).stop 2024-09-16T00:07:45.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f49acadd640 1 -- 172.21.15.139:0/31141625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49a8150f10 msgr2=0x7f49a80c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7ffa3613b640 1 -- 172.21.15.139:0/1417701659 shutdown_connections 2024-09-16T00:07:45.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7ffa3613b640 1 --2- 172.21.15.139:0/1417701659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa18003b40 0x7ffa18003f40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7ffa3613b640 1 -- 172.21.15.139:0/1417701659 >> 172.21.15.139:0/1417701659 conn(0x7ffa30070460 msgr2=0x7ffa30070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f8004f97640 1 -- 172.21.15.139:0/1984277798 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc037070 con 0x7f8000070040 2024-09-16T00:07:45.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f8004f97640 1 -- 172.21.15.139:0/1984277798 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ffc032440 con 0x7f8000070040 2024-09-16T00:07:45.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f49acadd640 1 --2- 172.21.15.139:0/31141625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49a8150f10 0x7f49a80c7e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f49a815b370 tx=0x7f499802e880 comp rx=0 tx=0).stop 2024-09-16T00:07:45.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f732599f640 1 -- 172.21.15.139:0/1301872108 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7308004800 msgr2=0x7f7308004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f49acadd640 1 -- 172.21.15.139:0/31141625 shutdown_connections 2024-09-16T00:07:45.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f49acadd640 1 --2- 172.21.15.139:0/31141625 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49a815a2b0 0x7f49a815a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7fd64283c640 1 -- 172.21.15.139:0/2061072472 wait complete. 2024-09-16T00:07:45.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f49acadd640 1 --2- 172.21.15.139:0/31141625 >> v2:172.21.15.139:3300/0 conn(0x7f49a80c3520 0x7f49a80c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f49acadd640 1 --2- 172.21.15.139:0/31141625 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49a8150f10 0x7f49a80c7e00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f49acadd640 1 -- 172.21.15.139:0/31141625 >> 172.21.15.139:0/31141625 conn(0x7f49a80c91d0 msgr2=0x7f49a80c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f144cae6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f14480c56d0 0x7f1448157500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f144cae6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14480c70d0 0x7f144815b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f144cae6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1448151a60 con 0x7f1448151680 2024-09-16T00:07:45.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f144cae6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f144815aa70 con 0x7f14480c56d0 2024-09-16T00:07:45.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f144cae6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f14480c5be0 con 0x7f14480c70d0 2024-09-16T00:07:45.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f732599f640 1 --2- 172.21.15.139:0/1301872108 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7308004800 0x7f7308004c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f731000e570 tx=0x7f7308005830 comp rx=0 tx=0).stop 2024-09-16T00:07:45.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f144659c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f1448151680 0x7f14480c3bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45286/0 (socket says 172.21.15.139:45286) 2024-09-16T00:07:45.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f144659c640 1 -- 172.21.15.139:0/4245264135 learned_addr learned my addr 172.21.15.139:0/4245264135 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:45.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f8004f97640 1 -- 172.21.15.139:0/1984277798 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc037070 con 0x7f8000070040 2024-09-16T00:07:45.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f55512cf640 1 --2- 172.21.15.139:0/3852324103 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55440dcdc0 0x7f5544045e10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5548009870 tx=0x7f554802a2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc16cd3e09a0dc0b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f1446d9d640 1 --2- 172.21.15.139:0/4245264135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14480c70d0 0x7f144815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f143dd9b640 1 --2- 172.21.15.139:0/4245264135 >> v2:172.21.15.139:3300/0 conn(0x7f14480c56d0 0x7f1448157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7f80081fd640 1 -- 172.21.15.139:0/1984277798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8000070040 msgr2=0x7f80000cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7f80081fd640 1 --2- 172.21.15.139:0/1984277798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8000070040 0x7f80000cc690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009870 tx=0x7f7ffc02a2a0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7ffa3613b640 1 -- 172.21.15.139:0/1417701659 shutdown_connections 2024-09-16T00:07:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7ffa3613b640 1 -- 172.21.15.139:0/1417701659 wait complete. 2024-09-16T00:07:45.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.345+0000 7f732519e640 1 -- 172.21.15.139:0/1301872108 reap_dead start 2024-09-16T00:07:45.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7f732599f640 1 -- 172.21.15.139:0/1301872108 shutdown_connections 2024-09-16T00:07:45.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7f732599f640 1 -- 172.21.15.139:0/1301872108 >> 172.21.15.139:0/1301872108 conn(0x7f73200cbe90 msgr2=0x7f73200cc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7f732599f640 1 -- 172.21.15.139:0/1301872108 shutdown_connections 2024-09-16T00:07:45.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7f49acadd640 1 -- 172.21.15.139:0/31141625 shutdown_connections 2024-09-16T00:07:45.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7f80081fd640 1 -- 172.21.15.139:0/1984277798 shutdown_connections 2024-09-16T00:07:45.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7f80081fd640 1 --2- 172.21.15.139:0/1984277798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8000070040 0x7f80000cc690 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7f80081fd640 1 --2- 172.21.15.139:0/1984277798 >> v2:172.21.15.164:3300/0 conn(0x7f800006fc20 0x7f80000cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7f80081fd640 1 -- 172.21.15.139:0/1984277798 >> 172.21.15.139:0/1984277798 conn(0x7f800006da40 msgr2=0x7f800006f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7f80081fd640 1 -- 172.21.15.139:0/1984277798 shutdown_connections 2024-09-16T00:07:45.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7f5543fff640 1 -- 172.21.15.139:0/3852324103 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548037070 con 0x7f55440dcdc0 2024-09-16T00:07:45.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7f80081fd640 1 -- 172.21.15.139:0/1984277798 wait complete. 2024-09-16T00:07:45.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7f5543fff640 1 -- 172.21.15.139:0/3852324103 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5548032640 con 0x7f55440dcdc0 2024-09-16T00:07:45.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7f732599f640 1 -- 172.21.15.139:0/1301872108 wait complete. 2024-09-16T00:07:45.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.346+0000 7f49acadd640 1 -- 172.21.15.139:0/31141625 wait complete. 2024-09-16T00:07:45.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.347+0000 7f5553533640 1 -- 172.21.15.139:0/3852324103 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55440dcdc0 msgr2=0x7f5544045e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.347+0000 7f5553533640 1 --2- 172.21.15.139:0/3852324103 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55440dcdc0 0x7f5544045e10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5548009870 tx=0x7f554802a2f0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.348+0000 7ffa3613b640 1 Processor -- start 2024-09-16T00:07:45.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.348+0000 7f49acadd640 1 Processor -- start 2024-09-16T00:07:45.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.348+0000 7f49acadd640 1 -- start start 2024-09-16T00:07:45.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.347+0000 7f143dd9b640 1 -- 172.21.15.139:0/4245264135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14480c70d0 msgr2=0x7f144815b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:45.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.348+0000 7ffa3613b640 1 -- start start 2024-09-16T00:07:45.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.348+0000 7f80081fd640 1 Processor -- start 2024-09-16T00:07:45.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.348+0000 7f80081fd640 1 -- start start 2024-09-16T00:07:45.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.347+0000 7f143dd9b640 1 --2- 172.21.15.139:0/4245264135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14480c70d0 0x7f144815b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.347+0000 7f143dd9b640 1 -- 172.21.15.139:0/4245264135 >> v1:172.21.15.139:6789/0 conn(0x7f1448151680 legacy=0x7f14480c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.347+0000 7f143dd9b640 1 -- 172.21.15.139:0/4245264135 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14480c5ec0 con 0x7f14480c56d0 2024-09-16T00:07:45.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.347+0000 7f732599f640 1 Processor -- start 2024-09-16T00:07:45.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.348+0000 7f732599f640 1 -- start start 2024-09-16T00:07:45.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.348+0000 7f5553533640 1 -- 172.21.15.139:0/3852324103 shutdown_connections 2024-09-16T00:07:45.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.348+0000 7f5553533640 1 --2- 172.21.15.139:0/3852324103 >> v2:172.21.15.164:3300/0 conn(0x7f55440e3100 0x7f554404aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.348+0000 7f5553533640 1 --2- 172.21.15.139:0/3852324103 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55440dcdc0 0x7f5544045e10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.348+0000 7f5553533640 1 -- 172.21.15.139:0/3852324103 >> 172.21.15.139:0/3852324103 conn(0x7f554404fc80 msgr2=0x7f5544050090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.347+0000 7fd64283c640 1 Processor -- start 2024-09-16T00:07:45.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f5553533640 1 -- 172.21.15.139:0/3852324103 shutdown_connections 2024-09-16T00:07:45.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.347+0000 7fd64283c640 1 -- start start 2024-09-16T00:07:45.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.348+0000 7fd64283c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd63c0c3520 0x7fd63c147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.348+0000 7fd64283c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd63c0c8510 0x7fd63c2062d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7fd64283c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd63c150f10 0x7fd63c206810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7fd64283c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c0cb420 con 0x7fd63c0c8510 2024-09-16T00:07:45.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7fd64283c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c06ea70 con 0x7fd63c150f10 2024-09-16T00:07:45.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f732599f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7308004800 0x7f7320150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f732599f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f732006ed90 0x7f7320200c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f732599f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73202011b0 0x7f73201fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f732599f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73200c69a0 con 0x7f73202011b0 2024-09-16T00:07:45.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f732599f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f73200c6d70 con 0x7f7308004800 2024-09-16T00:07:45.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f732599f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7320152c00 con 0x7f732006ed90 2024-09-16T00:07:45.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.348+0000 7f143dd9b640 1 --2- 172.21.15.139:0/4245264135 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14480c56d0 0x7f1448157500 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1438009d90 tx=0x7f143802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d748687e488443e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7fd64283c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd63c0c70d0 con 0x7fd63c0c3520 2024-09-16T00:07:45.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f732499d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7308004800 0x7f7320150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f732499d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7308004800 0x7f7320150470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39712/0 (socket says 172.21.15.139:39712) 2024-09-16T00:07:45.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f732499d640 1 -- 172.21.15.139:0/189913068 learned_addr learned my addr 172.21.15.139:0/189913068 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:45.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f1445d9b640 1 -- 172.21.15.139:0/4245264135 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1438007740 con 0x7f14480c56d0 2024-09-16T00:07:45.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7fd6337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd63c0c8510 0x7fd63c2062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7fd6337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd63c0c8510 0x7fd63c2062d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40638/0 (socket says 172.21.15.139:40638) 2024-09-16T00:07:45.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7fd6337fe640 1 -- 172.21.15.139:0/227998112 learned_addr learned my addr 172.21.15.139:0/227998112 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:45.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7fd63bfff640 1 --2- 172.21.15.139:0/227998112 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd63c0c3520 0x7fd63c147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f731ffff640 1 --2- 172.21.15.139:0/189913068 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f732006ed90 0x7f7320200c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7fd640dd9640 1 --2- 172.21.15.139:0/227998112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd63c150f10 0x7fd63c206810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f1445d9b640 1 -- 172.21.15.139:0/4245264135 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14380078e0 con 0x7f14480c56d0 2024-09-16T00:07:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f1445d9b640 1 -- 172.21.15.139:0/4245264135 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1438007cd0 con 0x7f14480c56d0 2024-09-16T00:07:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f5553533640 1 -- 172.21.15.139:0/3852324103 wait complete. 2024-09-16T00:07:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f80081fd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800006fc20 0x7f80001466b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f80081fd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8000070040 0x7f80000caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f80081fd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8000158f00 0x7f80000cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f80081fd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8000159460 con 0x7f800006fc20 2024-09-16T00:07:45.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f80081fd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80001592e0 con 0x7f8000070040 2024-09-16T00:07:45.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f80081fd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f80000c9120 con 0x7f8000158f00 2024-09-16T00:07:45.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f49acadd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49a80c3520 0x7f49a81453d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f49acadd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49a8150f10 0x7f49a814a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f49acadd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49a815a2b0 0x7f49a814a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f49acadd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f49a80cb420 con 0x7f49a8150f10 2024-09-16T00:07:45.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f49acadd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f49a806ea70 con 0x7f49a815a2b0 2024-09-16T00:07:45.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f49acadd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f49a80c70d0 con 0x7f49a80c3520 2024-09-16T00:07:45.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7ffa3613b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa18003b40 0x7ffa301e1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f49a5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49a8150f10 0x7f49a814a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f49a5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49a8150f10 0x7f49a814a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40674/0 (socket says 172.21.15.139:40674) 2024-09-16T00:07:45.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f49a5d9b640 1 -- 172.21.15.139:0/2176104007 learned_addr learned my addr 172.21.15.139:0/2176104007 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:45.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7f49a659c640 1 --2- 172.21.15.139:0/2176104007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49a80c3520 0x7f49a81453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7fd640dd9640 1 -- 172.21.15.139:0/227998112 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd63c0c3520 msgr2=0x7fd63c147ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7fd640dd9640 1 --2- 172.21.15.139:0/227998112 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd63c0c3520 0x7fd63c147ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f8005f99640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800006fc20 0x7f80001466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7fd640dd9640 1 -- 172.21.15.139:0/227998112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd63c0c8510 msgr2=0x7fd63c2062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f8005f99640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800006fc20 0x7f80001466b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40644/0 (socket says 172.21.15.139:40644) 2024-09-16T00:07:45.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f8005f99640 1 -- 172.21.15.139:0/127500133 learned_addr learned my addr 172.21.15.139:0/127500133 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:45.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7ffa3613b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa3014f890 0x7ffa30156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7ffa3613b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa3014cc20 0x7ffa301456d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7ffa3613b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffa300c44c0 con 0x7ffa3014cc20 2024-09-16T00:07:45.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7ffa3613b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffa30151be0 con 0x7ffa18003b40 2024-09-16T00:07:45.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7ffa3613b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ffa300c4640 con 0x7ffa3014f890 2024-09-16T00:07:45.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7fd640dd9640 1 --2- 172.21.15.139:0/227998112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd63c0c8510 0x7fd63c2062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.349+0000 7fd640dd9640 1 -- 172.21.15.139:0/227998112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c157f40 con 0x7fd63c150f10 2024-09-16T00:07:45.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7fd640dd9640 1 --2- 172.21.15.139:0/227998112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd63c150f10 0x7fd63c206810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd634033000 tx=0x7fd634004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f731ffff640 1 -- 172.21.15.139:0/189913068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7308004800 msgr2=0x7f7320150470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7ffa2f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa18003b40 0x7ffa301e1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7ffa2f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa18003b40 0x7ffa301e1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39730/0 (socket says 172.21.15.139:39730) 2024-09-16T00:07:45.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7ffa2f7fe640 1 -- 172.21.15.139:0/625983316 learned_addr learned my addr 172.21.15.139:0/625983316 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:45.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7ffa2ffff640 1 --2- 172.21.15.139:0/625983316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa3014cc20 0x7ffa301456d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f8005798640 1 --2- 172.21.15.139:0/127500133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8000070040 0x7f80000caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f731ffff640 1 --2- 172.21.15.139:0/189913068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7308004800 0x7f7320150470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7fd6397fa640 1 -- 172.21.15.139:0/227998112 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd634042020 con 0x7fd63c150f10 2024-09-16T00:07:45.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7fd6397fa640 1 -- 172.21.15.139:0/227998112 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd63403e680 con 0x7fd63c150f10 2024-09-16T00:07:45.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f731ffff640 1 -- 172.21.15.139:0/189913068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73202011b0 msgr2=0x7f73201fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f144cae6640 1 -- 172.21.15.139:0/4245264135 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14480c56d0 msgr2=0x7f1448157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7fd6397fa640 1 -- 172.21.15.139:0/227998112 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63403ea90 con 0x7fd63c150f10 2024-09-16T00:07:45.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f144cae6640 1 --2- 172.21.15.139:0/4245264135 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14480c56d0 0x7f1448157500 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1438009d90 tx=0x7f143802a610 comp rx=0 tx=0).stop 2024-09-16T00:07:45.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f8005798640 1 -- 172.21.15.139:0/127500133 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8000158f00 msgr2=0x7f80000cb020 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:07:45.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7fd64283c640 1 -- 172.21.15.139:0/227998112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c158220 con 0x7fd63c150f10 2024-09-16T00:07:45.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7fd64283c640 1 -- 172.21.15.139:0/227998112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63c158500 con 0x7fd63c150f10 2024-09-16T00:07:45.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7ffa2effd640 1 --2- 172.21.15.139:0/625983316 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa3014f890 0x7ffa30156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f49a659c640 1 -- 172.21.15.139:0/2176104007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49a815a2b0 msgr2=0x7f49a814a750 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:07:45.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f49a659c640 1 --2- 172.21.15.139:0/2176104007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49a815a2b0 0x7f49a814a750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f49a659c640 1 -- 172.21.15.139:0/2176104007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49a8150f10 msgr2=0x7f49a814a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f49a659c640 1 --2- 172.21.15.139:0/2176104007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49a8150f10 0x7f49a814a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f49a659c640 1 -- 172.21.15.139:0/2176104007 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a814c490 con 0x7f49a80c3520 2024-09-16T00:07:45.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f49a659c640 1 --2- 172.21.15.139:0/2176104007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49a80c3520 0x7f49a81453d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4998037000 tx=0x7f49980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7ffa2f7fe640 1 -- 172.21.15.139:0/625983316 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa3014f890 msgr2=0x7ffa30156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7ffa2f7fe640 1 --2- 172.21.15.139:0/625983316 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa3014f890 0x7ffa30156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7ffa2f7fe640 1 -- 172.21.15.139:0/625983316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa3014cc20 msgr2=0x7ffa301456d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7ffa2f7fe640 1 --2- 172.21.15.139:0/625983316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa3014cc20 0x7ffa301456d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7ffa2f7fe640 1 -- 172.21.15.139:0/625983316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3014e790 con 0x7ffa18003b40 2024-09-16T00:07:45.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7f5553533640 1 Processor -- start 2024-09-16T00:07:45.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7ffa2f7fe640 1 --2- 172.21.15.139:0/625983316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa18003b40 0x7ffa301e1b10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000c360 tx=0x7ffa20001970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f731ffff640 1 --2- 172.21.15.139:0/189913068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73202011b0 0x7f73201fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f731ffff640 1 -- 172.21.15.139:0/189913068 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732006f350 con 0x7f732006ed90 2024-09-16T00:07:45.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f731ffff640 1 --2- 172.21.15.139:0/189913068 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f732006ed90 0x7f7320200c70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f730800c1e0 tx=0x7f7310001970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7f731d7fa640 1 -- 172.21.15.139:0/189913068 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310036040 con 0x7f732006ed90 2024-09-16T00:07:45.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7f731d7fa640 1 -- 172.21.15.139:0/189913068 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7310005070 con 0x7f732006ed90 2024-09-16T00:07:45.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7f731d7fa640 1 -- 172.21.15.139:0/189913068 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310007d70 con 0x7f732006ed90 2024-09-16T00:07:45.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7f732599f640 1 -- 172.21.15.139:0/189913068 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73200c2ea0 con 0x7f732006ed90 2024-09-16T00:07:45.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7f732599f640 1 -- 172.21.15.139:0/189913068 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f732006f8b0 con 0x7f732006ed90 2024-09-16T00:07:45.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7f5553533640 1 -- start start 2024-09-16T00:07:45.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f8005798640 1 --2- 172.21.15.139:0/127500133 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8000158f00 0x7f80000cb020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f8005798640 1 -- 172.21.15.139:0/127500133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800006fc20 msgr2=0x7f80001466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.350+0000 7f8005798640 1 --2- 172.21.15.139:0/127500133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800006fc20 0x7f80001466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7f8005798640 1 -- 172.21.15.139:0/127500133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8000145b10 con 0x7f8000070040 2024-09-16T00:07:45.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7f144cae6640 1 -- 172.21.15.139:0/4245264135 shutdown_connections 2024-09-16T00:07:45.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7f144cae6640 1 --2- 172.21.15.139:0/4245264135 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14480c70d0 0x7f144815b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7f144cae6640 1 --2- 172.21.15.139:0/4245264135 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14480c56d0 0x7f1448157500 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7f144cae6640 1 -- 172.21.15.139:0/4245264135 >> 172.21.15.139:0/4245264135 conn(0x7f14480c91d0 msgr2=0x7f14480c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.352+0000 7f144cae6640 1 -- 172.21.15.139:0/4245264135 shutdown_connections 2024-09-16T00:07:45.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7ffa17fff640 1 -- 172.21.15.139:0/625983316 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20041070 con 0x7ffa18003b40 2024-09-16T00:07:45.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7ffa17fff640 1 -- 172.21.15.139:0/625983316 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa20007070 con 0x7ffa18003b40 2024-09-16T00:07:45.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7ffa17fff640 1 -- 172.21.15.139:0/625983316 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20007d30 con 0x7ffa18003b40 2024-09-16T00:07:45.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.352+0000 7ffa3613b640 1 -- 172.21.15.139:0/625983316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3014e270 con 0x7ffa18003b40 2024-09-16T00:07:45.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.352+0000 7f4996ffd640 1 -- 172.21.15.139:0/2176104007 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499803d070 con 0x7f49a80c3520 2024-09-16T00:07:45.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.352+0000 7f4996ffd640 1 -- 172.21.15.139:0/2176104007 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4998010640 con 0x7f49a80c3520 2024-09-16T00:07:45.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.352+0000 7ffa3613b640 1 -- 172.21.15.139:0/625983316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa300cae60 con 0x7ffa18003b40 2024-09-16T00:07:45.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.352+0000 7fd64283c640 1 -- 172.21.15.139:0/227998112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63c20a300 con 0x7fd63c150f10 2024-09-16T00:07:45.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7f5553533640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554404a6b0 0x7f554416fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7f144cae6640 1 -- 172.21.15.139:0/4245264135 wait complete. 2024-09-16T00:07:45.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7f732599f640 1 -- 172.21.15.139:0/189913068 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7300005290 con 0x7f732006ed90 2024-09-16T00:07:45.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.351+0000 7f8005798640 1 --2- 172.21.15.139:0/127500133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8000070040 0x7f80000caae0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000d620 tx=0x7f7ff000dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.352+0000 7f7ff67fc640 1 -- 172.21.15.139:0/127500133 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff000b620 con 0x7f8000070040 2024-09-16T00:07:45.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.352+0000 7f7ff67fc640 1 -- 172.21.15.139:0/127500133 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff0019070 con 0x7f8000070040 2024-09-16T00:07:45.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.352+0000 7f7ff67fc640 1 -- 172.21.15.139:0/127500133 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff000b9e0 con 0x7f8000070040 2024-09-16T00:07:45.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.352+0000 7f80081fd640 1 -- 172.21.15.139:0/127500133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8000144c60 con 0x7f8000070040 2024-09-16T00:07:45.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.352+0000 7f80081fd640 1 -- 172.21.15.139:0/127500133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80001451e0 con 0x7f8000070040 2024-09-16T00:07:45.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7fd6397fa640 1 -- 172.21.15.139:0/227998112 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd634045020 con 0x7fd63c150f10 2024-09-16T00:07:45.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7f5553533640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55440dcdc0 0x7f554417bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7f5553533640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55440e3100 0x7f554417c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7f55512cf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554404a6b0 0x7f554416fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7f55512cf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554404a6b0 0x7f554416fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40688/0 (socket says 172.21.15.139:40688) 2024-09-16T00:07:45.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7f55512cf640 1 -- 172.21.15.139:0/1579149589 learned_addr learned my addr 172.21.15.139:0/1579149589 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:45.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7f5553533640 1 -- 172.21.15.139:0/1579149589 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5544045330 con 0x7f554404a6b0 2024-09-16T00:07:45.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7ffa3613b640 1 -- 172.21.15.139:0/625983316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa300c4970 con 0x7ffa18003b40 2024-09-16T00:07:45.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7f5553533640 1 -- 172.21.15.139:0/1579149589 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5544045050 con 0x7f55440e3100 2024-09-16T00:07:45.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7f5553533640 1 -- 172.21.15.139:0/1579149589 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5544045610 con 0x7f55440dcdc0 2024-09-16T00:07:45.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7f49acadd640 1 -- 172.21.15.139:0/2176104007 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a814ad30 con 0x7f49a80c3520 2024-09-16T00:07:45.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7f49acadd640 1 -- 172.21.15.139:0/2176104007 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a814c9e0 con 0x7f49a80c3520 2024-09-16T00:07:45.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7f4996ffd640 1 -- 172.21.15.139:0/2176104007 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49980386c0 con 0x7f49a80c3520 2024-09-16T00:07:45.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7f5551ad0640 1 --2- 172.21.15.139:0/1579149589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55440e3100 0x7f554417c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.353+0000 7f5550ace640 1 --2- 172.21.15.139:0/1579149589 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55440dcdc0 0x7f554417bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.354+0000 7f731d7fa640 1 -- 172.21.15.139:0/189913068 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7310007030 con 0x7f732006ed90 2024-09-16T00:07:45.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.354+0000 7fd6397fa640 1 --2- 172.21.15.139:0/227998112 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd61c06c380 0x7fd61c06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.354+0000 7f55512cf640 1 -- 172.21.15.139:0/1579149589 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55440dcdc0 msgr2=0x7f554417bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.354+0000 7f55512cf640 1 --2- 172.21.15.139:0/1579149589 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55440dcdc0 0x7f554417bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.354+0000 7f55512cf640 1 -- 172.21.15.139:0/1579149589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55440e3100 msgr2=0x7f554417c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.354+0000 7f55512cf640 1 --2- 172.21.15.139:0/1579149589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55440e3100 0x7f554417c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.354+0000 7f55512cf640 1 -- 172.21.15.139:0/1579149589 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554417c960 con 0x7f554404a6b0 2024-09-16T00:07:45.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.354+0000 7fd63bfff640 1 --2- 172.21.15.139:0/227998112 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd61c06c380 0x7fd61c06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.354+0000 7fd6397fa640 1 -- 172.21.15.139:0/227998112 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd6340abc20 con 0x7fd63c150f10 2024-09-16T00:07:45.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.355+0000 7f731d7fa640 1 --2- 172.21.15.139:0/189913068 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f730c06c380 0x7f730c06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.355+0000 7f55512cf640 1 --2- 172.21.15.139:0/1579149589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554404a6b0 0x7f554416fa90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5548009990 tx=0x7f55480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.355+0000 7f144cae6640 1 Processor -- start 2024-09-16T00:07:45.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.355+0000 7fd63bfff640 1 --2- 172.21.15.139:0/227998112 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd61c06c380 0x7fd61c06e840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd624004f60 tx=0x7fd624007e30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.355+0000 7f731d7fa640 1 -- 172.21.15.139:0/189913068 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f73100ab4b0 con 0x7f732006ed90 2024-09-16T00:07:45.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.355+0000 7f732499d640 1 --2- 172.21.15.139:0/189913068 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f730c06c380 0x7f730c06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.355+0000 7ffa17fff640 1 -- 172.21.15.139:0/625983316 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ffa20044070 con 0x7ffa18003b40 2024-09-16T00:07:45.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.355+0000 7f732499d640 1 --2- 172.21.15.139:0/189913068 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f730c06c380 0x7f730c06e840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7318009440 tx=0x7f731800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.355+0000 7f5541ffb640 1 -- 172.21.15.139:0/1579149589 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548041020 con 0x7f554404a6b0 2024-09-16T00:07:45.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.355+0000 7f5541ffb640 1 -- 172.21.15.139:0/1579149589 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f554803d420 con 0x7f554404a6b0 2024-09-16T00:07:45.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.356+0000 7f5553533640 1 -- 172.21.15.139:0/1579149589 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554417c680 con 0x7f554404a6b0 2024-09-16T00:07:45.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.356+0000 7f144cae6640 1 -- start start 2024-09-16T00:07:45.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.356+0000 7f49acadd640 1 -- 172.21.15.139:0/2176104007 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a81512f0 con 0x7f49a80c3520 2024-09-16T00:07:45.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.356+0000 7ffa17fff640 1 --2- 172.21.15.139:0/625983316 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffa1c06c3b0 0x7ffa1c06e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.356+0000 7f4996ffd640 1 -- 172.21.15.139:0/2176104007 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4998038860 con 0x7f49a80c3520 2024-09-16T00:07:45.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.356+0000 7ffa17fff640 1 -- 172.21.15.139:0/625983316 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ffa200ab9a0 con 0x7ffa18003b40 2024-09-16T00:07:45.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.356+0000 7f5541ffb640 1 -- 172.21.15.139:0/1579149589 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548037040 con 0x7f554404a6b0 2024-09-16T00:07:45.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.356+0000 7f5553533640 1 -- 172.21.15.139:0/1579149589 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55440dd6a0 con 0x7f554404a6b0 2024-09-16T00:07:45.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.357+0000 7f4996ffd640 1 --2- 172.21.15.139:0/2176104007 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f498806c300 0x7f498806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.356+0000 7f80081fd640 1 -- 172.21.15.139:0/127500133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fe8005290 con 0x7f8000070040 2024-09-16T00:07:45.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.357+0000 7f49a5d9b640 1 --2- 172.21.15.139:0/2176104007 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f498806c300 0x7f498806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.357+0000 7f4996ffd640 1 -- 172.21.15.139:0/2176104007 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f49980ab800 con 0x7f49a80c3520 2024-09-16T00:07:45.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.357+0000 7ffa2effd640 1 --2- 172.21.15.139:0/625983316 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffa1c06c3b0 0x7ffa1c06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.357+0000 7f144cae6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14480c56d0 0x7f14481f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.357+0000 7f144cae6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14480c70d0 0x7f144814c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.357+0000 7f144cae6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1448151680 0x7f144814c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.357+0000 7f144cae6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f14480c7640 con 0x7f1448151680 2024-09-16T00:07:45.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.357+0000 7f144cae6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f144815ad10 con 0x7f14480c70d0 2024-09-16T00:07:45.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.357+0000 7f144cae6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f14480c77c0 con 0x7f14480c56d0 2024-09-16T00:07:45.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.358+0000 7f49a5d9b640 1 --2- 172.21.15.139:0/2176104007 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f498806c300 0x7f498806e7c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f49a0007fd0 tx=0x7f49a00078d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.358+0000 7ffa2effd640 1 --2- 172.21.15.139:0/625983316 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffa1c06c3b0 0x7ffa1c06e870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffa3014f510 tx=0x7ffa24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.358+0000 7f5553533640 1 -- 172.21.15.139:0/1579149589 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5544035c50 con 0x7f554404a6b0 2024-09-16T00:07:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.358+0000 7f144659c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14480c56d0 0x7f14481f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.358+0000 7f143dd9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14480c70d0 0x7f144814c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.358+0000 7f144659c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14480c56d0 0x7f14481f4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:55376/0 (socket says 172.21.15.139:55376) 2024-09-16T00:07:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.358+0000 7f144659c640 1 -- 172.21.15.139:0/1491824948 learned_addr learned my addr 172.21.15.139:0/1491824948 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.358+0000 7f1446d9d640 1 --2- 172.21.15.139:0/1491824948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1448151680 0x7f144814c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.358+0000 7f144659c640 1 -- 172.21.15.139:0/1491824948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14480c70d0 msgr2=0x7f144814c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.358+0000 7f144659c640 1 --2- 172.21.15.139:0/1491824948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14480c70d0 0x7f144814c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.358+0000 7f144659c640 1 -- 172.21.15.139:0/1491824948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1448151680 msgr2=0x7f144814c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.358+0000 7f144659c640 1 --2- 172.21.15.139:0/1491824948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1448151680 0x7f144814c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.358+0000 7f144659c640 1 -- 172.21.15.139:0/1491824948 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144814b880 con 0x7f14480c56d0 2024-09-16T00:07:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.358+0000 7f144659c640 1 --2- 172.21.15.139:0/1491824948 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14480c56d0 0x7f14481f4400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f143000c480 tx=0x7f143000c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.359+0000 7fd6397fa640 1 -- 172.21.15.139:0/227998112 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd63404c070 con 0x7fd63c150f10 2024-09-16T00:07:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.359+0000 7f5541ffb640 1 -- 172.21.15.139:0/1579149589 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5548044020 con 0x7f554404a6b0 2024-09-16T00:07:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.359+0000 7f731d7fa640 1 -- 172.21.15.139:0/189913068 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f732006f350 con 0x7f732006ed90 2024-09-16T00:07:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.360+0000 7f5541ffb640 1 --2- 172.21.15.139:0/1579149589 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f553006c380 0x7f553006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.360+0000 7f5550ace640 1 --2- 172.21.15.139:0/1579149589 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f553006c380 0x7f553006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.360+0000 7f5550ace640 1 --2- 172.21.15.139:0/1579149589 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f553006c380 0x7f553006e840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f554404dfd0 tx=0x7f553c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.360+0000 7f5541ffb640 1 -- 172.21.15.139:0/1579149589 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f55480ac410 con 0x7f554404a6b0 2024-09-16T00:07:45.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.363+0000 7ffa17fff640 1 -- 172.21.15.139:0/625983316 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffa300c4970 con 0x7ffa18003b40 2024-09-16T00:07:45.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.364+0000 7f143f7fe640 1 -- 172.21.15.139:0/1491824948 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1430017070 con 0x7f14480c56d0 2024-09-16T00:07:45.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.364+0000 7f143f7fe640 1 -- 172.21.15.139:0/1491824948 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1430005ce0 con 0x7f14480c56d0 2024-09-16T00:07:45.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.364+0000 7f144cae6640 1 -- 172.21.15.139:0/1491824948 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1448155200 con 0x7f14480c56d0 2024-09-16T00:07:45.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.364+0000 7f143f7fe640 1 -- 172.21.15.139:0/1491824948 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14300072f0 con 0x7f14480c56d0 2024-09-16T00:07:45.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.364+0000 7f144cae6640 1 -- 172.21.15.139:0/1491824948 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f14481f04d0 con 0x7f14480c56d0 2024-09-16T00:07:45.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.365+0000 7f7ff67fc640 1 -- 172.21.15.139:0/127500133 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7ff000bb80 con 0x7f8000070040 2024-09-16T00:07:45.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.365+0000 7f5541ffb640 1 -- 172.21.15.139:0/1579149589 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5544035c50 con 0x7f554404a6b0 2024-09-16T00:07:45.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.366+0000 7f7ff67fc640 1 --2- 172.21.15.139:0/127500133 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7fe406c380 0x7f7fe406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.366+0000 7f7ff67fc640 1 -- 172.21.15.139:0/127500133 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7ff008c310 con 0x7f8000070040 2024-09-16T00:07:45.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.365+0000 7f144cae6640 1 -- 172.21.15.139:0/1491824948 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f142c005290 con 0x7f14480c56d0 2024-09-16T00:07:45.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.366+0000 7f8005f99640 1 --2- 172.21.15.139:0/127500133 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7fe406c380 0x7f7fe406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.366+0000 7f4996ffd640 1 -- 172.21.15.139:0/2176104007 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4998032030 con 0x7f49a80c3520 2024-09-16T00:07:45.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.367+0000 7f8005f99640 1 --2- 172.21.15.139:0/127500133 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7fe406c380 0x7f7fe406e840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009f80 tx=0x7f7ff8009600 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.367+0000 7f143f7fe640 1 -- 172.21.15.139:0/1491824948 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f143001a070 con 0x7f14480c56d0 2024-09-16T00:07:45.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.368+0000 7f143f7fe640 1 --2- 172.21.15.139:0/1491824948 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f142806c2b0 0x7f142806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.368+0000 7f7ff67fc640 1 -- 172.21.15.139:0/127500133 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7ff005aaf0 con 0x7f8000070040 2024-09-16T00:07:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.368+0000 7f143f7fe640 1 -- 172.21.15.139:0/1491824948 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f143008a5f0 con 0x7f14480c56d0 2024-09-16T00:07:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.368+0000 7f143dd9b640 1 --2- 172.21.15.139:0/1491824948 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f142806c2b0 0x7f142806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.369+0000 7f143dd9b640 1 --2- 172.21.15.139:0/1491824948 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f142806c2b0 0x7f142806e770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1438034000 tx=0x7f143802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.372+0000 7f143f7fe640 1 -- 172.21.15.139:0/1491824948 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f144814b880 con 0x7f14480c56d0 2024-09-16T00:07:45.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.703+0000 7f7bac35b640 1 -- 172.21.15.139:0/1165115653 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7b8c005580 con 0x7f7ba40cbdf0 2024-09-16T00:07:45.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.704+0000 7f7b9affd640 1 -- 172.21.15.139:0/1165115653 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7ba0059cd0 con 0x7f7ba40cbdf0 2024-09-16T00:07:45.705 INFO:teuthology.orchestra.run.smithi139.stdout:64424509448 2024-09-16T00:07:45.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.707+0000 7f7b98ff9640 1 -- 172.21.15.139:0/1165115653 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b8806c2b0 msgr2=0x7f7b8806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.707+0000 7f7b98ff9640 1 --2- 172.21.15.139:0/1165115653 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b8806c2b0 0x7f7b8806e770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c033000 tx=0x7f7b9c009000 comp rx=0 tx=0).stop 2024-09-16T00:07:45.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.707+0000 7f7b98ff9640 1 -- 172.21.15.139:0/1165115653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ba40cbdf0 msgr2=0x7f7ba40cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.708+0000 7f7b98ff9640 1 --2- 172.21.15.139:0/1165115653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ba40cbdf0 0x7f7ba40cf610 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000d800 tx=0x7f7ba000dcc0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.708+0000 7f7b98ff9640 1 -- 172.21.15.139:0/1165115653 shutdown_connections 2024-09-16T00:07:45.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.708+0000 7f7b98ff9640 1 --2- 172.21.15.139:0/1165115653 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b8806c2b0 0x7f7b8806e770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.708+0000 7f7b98ff9640 1 --2- 172.21.15.139:0/1165115653 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ba40d0ae0 0x7f7ba41f86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.708+0000 7f7b98ff9640 1 --2- 172.21.15.139:0/1165115653 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ba40d0700 0x7f7ba41f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.708+0000 7f7b98ff9640 1 --2- 172.21.15.139:0/1165115653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ba40cbdf0 0x7f7ba40cf610 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.708+0000 7f7b98ff9640 1 -- 172.21.15.139:0/1165115653 >> 172.21.15.139:0/1165115653 conn(0x7f7ba406c500 msgr2=0x7f7ba40cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.708+0000 7f7b98ff9640 1 -- 172.21.15.139:0/1165115653 shutdown_connections 2024-09-16T00:07:45.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.709+0000 7f7b98ff9640 1 -- 172.21.15.139:0/1165115653 wait complete. 2024-09-16T00:07:45.726 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.6 2024-09-16T00:07:45.726 DEBUG:teuthology.parallel:result is None 2024-09-16T00:07:45.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.741+0000 7f80081fd640 1 -- 172.21.15.139:0/127500133 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7fe8005850 con 0x7f8000070040 2024-09-16T00:07:45.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.741+0000 7f7ff67fc640 1 -- 172.21.15.139:0/127500133 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7ff005a3f0 con 0x7f8000070040 2024-09-16T00:07:45.742 INFO:teuthology.orchestra.run.smithi139.stdout:64424509448 2024-09-16T00:07:45.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.744+0000 7f80081fd640 1 -- 172.21.15.139:0/127500133 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7fe406c380 msgr2=0x7f7fe406e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.744+0000 7f80081fd640 1 --2- 172.21.15.139:0/127500133 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7fe406c380 0x7f7fe406e840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009f80 tx=0x7f7ff8009600 comp rx=0 tx=0).stop 2024-09-16T00:07:45.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.744+0000 7f80081fd640 1 -- 172.21.15.139:0/127500133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8000070040 msgr2=0x7f80000caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.744+0000 7f80081fd640 1 --2- 172.21.15.139:0/127500133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8000070040 0x7f80000caae0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000d620 tx=0x7f7ff000dae0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.744+0000 7f80081fd640 1 -- 172.21.15.139:0/127500133 shutdown_connections 2024-09-16T00:07:45.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.744+0000 7f80081fd640 1 --2- 172.21.15.139:0/127500133 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7fe406c380 0x7f7fe406e840 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.744+0000 7f80081fd640 1 --2- 172.21.15.139:0/127500133 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8000158f00 0x7f80000cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.744+0000 7f80081fd640 1 --2- 172.21.15.139:0/127500133 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8000070040 0x7f80000caae0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.744+0000 7f80081fd640 1 --2- 172.21.15.139:0/127500133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800006fc20 0x7f80001466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.744+0000 7f80081fd640 1 -- 172.21.15.139:0/127500133 >> 172.21.15.139:0/127500133 conn(0x7f800006da40 msgr2=0x7f80000cbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.744+0000 7f80081fd640 1 -- 172.21.15.139:0/127500133 shutdown_connections 2024-09-16T00:07:45.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.744+0000 7f80081fd640 1 -- 172.21.15.139:0/127500133 wait complete. 2024-09-16T00:07:45.756 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.4 2024-09-16T00:07:45.756 DEBUG:teuthology.parallel:result is None 2024-09-16T00:07:45.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.757+0000 7f732599f640 1 -- 172.21.15.139:0/189913068 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7300005580 con 0x7f732006ed90 2024-09-16T00:07:45.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.757+0000 7fd64283c640 1 -- 172.21.15.139:0/227998112 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd63c1512f0 con 0x7fd63c150f10 2024-09-16T00:07:45.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.758+0000 7f731d7fa640 1 -- 172.21.15.139:0/189913068 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7310079640 con 0x7f732006ed90 2024-09-16T00:07:45.759 INFO:teuthology.orchestra.run.smithi139.stdout:64424509447 2024-09-16T00:07:45.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.758+0000 7fd6397fa640 1 -- 172.21.15.139:0/227998112 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd634080020 con 0x7fd63c150f10 2024-09-16T00:07:45.760 INFO:teuthology.orchestra.run.smithi139.stdout:64424509446 2024-09-16T00:07:45.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7fd64283c640 1 -- 172.21.15.139:0/227998112 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd61c06c380 msgr2=0x7fd61c06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7fd64283c640 1 --2- 172.21.15.139:0/227998112 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd61c06c380 0x7fd61c06e840 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd624004f60 tx=0x7fd624007e30 comp rx=0 tx=0).stop 2024-09-16T00:07:45.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7fd64283c640 1 -- 172.21.15.139:0/227998112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd63c150f10 msgr2=0x7fd63c206810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7fd64283c640 1 --2- 172.21.15.139:0/227998112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd63c150f10 0x7fd63c206810 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd634033000 tx=0x7fd634004a40 comp rx=0 tx=0).stop 2024-09-16T00:07:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7fd64283c640 1 -- 172.21.15.139:0/227998112 shutdown_connections 2024-09-16T00:07:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7fd64283c640 1 --2- 172.21.15.139:0/227998112 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd61c06c380 0x7fd61c06e840 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7fd64283c640 1 --2- 172.21.15.139:0/227998112 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd63c150f10 0x7fd63c206810 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7fd64283c640 1 --2- 172.21.15.139:0/227998112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd63c0c8510 0x7fd63c2062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7fd64283c640 1 --2- 172.21.15.139:0/227998112 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd63c0c3520 0x7fd63c147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7fd64283c640 1 -- 172.21.15.139:0/227998112 >> 172.21.15.139:0/227998112 conn(0x7fd63c0c91d0 msgr2=0x7fd63c0c95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7f732599f640 1 -- 172.21.15.139:0/189913068 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f730c06c380 msgr2=0x7f730c06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7fd64283c640 1 -- 172.21.15.139:0/227998112 shutdown_connections 2024-09-16T00:07:45.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7fd64283c640 1 -- 172.21.15.139:0/227998112 wait complete. 2024-09-16T00:07:45.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7f732599f640 1 --2- 172.21.15.139:0/189913068 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f730c06c380 0x7f730c06e840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7318009440 tx=0x7f731800b040 comp rx=0 tx=0).stop 2024-09-16T00:07:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7f732599f640 1 -- 172.21.15.139:0/189913068 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f732006ed90 msgr2=0x7f7320200c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7f732599f640 1 --2- 172.21.15.139:0/189913068 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f732006ed90 0x7f7320200c70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f730800c1e0 tx=0x7f7310001970 comp rx=0 tx=0).stop 2024-09-16T00:07:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7f732599f640 1 -- 172.21.15.139:0/189913068 shutdown_connections 2024-09-16T00:07:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7f732599f640 1 --2- 172.21.15.139:0/189913068 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f730c06c380 0x7f730c06e840 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7f732599f640 1 --2- 172.21.15.139:0/189913068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73202011b0 0x7f73201fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7f732599f640 1 --2- 172.21.15.139:0/189913068 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f732006ed90 0x7f7320200c70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7f732599f640 1 --2- 172.21.15.139:0/189913068 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7308004800 0x7f7320150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.761+0000 7f732599f640 1 -- 172.21.15.139:0/189913068 >> 172.21.15.139:0/189913068 conn(0x7f73200cbe90 msgr2=0x7f7320158ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.762+0000 7f732599f640 1 -- 172.21.15.139:0/189913068 shutdown_connections 2024-09-16T00:07:45.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.762+0000 7f732599f640 1 -- 172.21.15.139:0/189913068 wait complete. 2024-09-16T00:07:45.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.762+0000 7f5553533640 1 -- 172.21.15.139:0/1579149589 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f55440e6180 con 0x7f554404a6b0 2024-09-16T00:07:45.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.762+0000 7f5541ffb640 1 -- 172.21.15.139:0/1579149589 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f554804b030 con 0x7f554404a6b0 2024-09-16T00:07:45.767 INFO:teuthology.orchestra.run.smithi139.stdout:60129542151 2024-09-16T00:07:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.765+0000 7f5553533640 1 -- 172.21.15.139:0/1579149589 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f553006c380 msgr2=0x7f553006e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.765+0000 7f5553533640 1 --2- 172.21.15.139:0/1579149589 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f553006c380 0x7f553006e840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f554404dfd0 tx=0x7f553c009040 comp rx=0 tx=0).stop 2024-09-16T00:07:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.765+0000 7f5553533640 1 -- 172.21.15.139:0/1579149589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554404a6b0 msgr2=0x7f554416fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.765+0000 7f5553533640 1 --2- 172.21.15.139:0/1579149589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554404a6b0 0x7f554416fa90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5548009990 tx=0x7f55480046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.765+0000 7f5553533640 1 -- 172.21.15.139:0/1579149589 shutdown_connections 2024-09-16T00:07:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.765+0000 7f5553533640 1 --2- 172.21.15.139:0/1579149589 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f553006c380 0x7f553006e840 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.765+0000 7f5553533640 1 --2- 172.21.15.139:0/1579149589 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f55440e3100 0x7f554417c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.765+0000 7f5553533640 1 --2- 172.21.15.139:0/1579149589 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f55440dcdc0 0x7f554417bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.765+0000 7f5553533640 1 --2- 172.21.15.139:0/1579149589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f554404a6b0 0x7f554416fa90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.765+0000 7f5553533640 1 -- 172.21.15.139:0/1579149589 >> 172.21.15.139:0/1579149589 conn(0x7f554404fc80 msgr2=0x7f554404b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.765+0000 7f5553533640 1 -- 172.21.15.139:0/1579149589 shutdown_connections 2024-09-16T00:07:45.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.765+0000 7f5553533640 1 -- 172.21.15.139:0/1579149589 wait complete. 2024-09-16T00:07:45.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.767+0000 7f144cae6640 1 -- 172.21.15.139:0/1491824948 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f142c005850 con 0x7f14480c56d0 2024-09-16T00:07:45.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.768+0000 7f143f7fe640 1 -- 172.21.15.139:0/1491824948 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f14300587a0 con 0x7f14480c56d0 2024-09-16T00:07:45.771 INFO:teuthology.orchestra.run.smithi139.stdout:64424509448 2024-09-16T00:07:45.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.769+0000 7f49acadd640 1 -- 172.21.15.139:0/2176104007 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f49a81e2120 con 0x7f49a80c3520 2024-09-16T00:07:45.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.770+0000 7ffa3613b640 1 -- 172.21.15.139:0/625983316 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ffa3006f750 con 0x7ffa18003b40 2024-09-16T00:07:45.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.771+0000 7ffa17fff640 1 -- 172.21.15.139:0/625983316 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffa3014e790 con 0x7ffa18003b40 2024-09-16T00:07:45.773 INFO:teuthology.orchestra.run.smithi139.stdout:55834574855 2024-09-16T00:07:45.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.771+0000 7f144cae6640 1 -- 172.21.15.139:0/1491824948 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f142806c2b0 msgr2=0x7f142806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.771+0000 7f144cae6640 1 --2- 172.21.15.139:0/1491824948 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f142806c2b0 0x7f142806e770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1438034000 tx=0x7f143802f000 comp rx=0 tx=0).stop 2024-09-16T00:07:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.771+0000 7f144cae6640 1 -- 172.21.15.139:0/1491824948 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14480c56d0 msgr2=0x7f14481f4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.771+0000 7f144cae6640 1 --2- 172.21.15.139:0/1491824948 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14480c56d0 0x7f14481f4400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f143000c480 tx=0x7f143000c940 comp rx=0 tx=0).stop 2024-09-16T00:07:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.772+0000 7f144cae6640 1 -- 172.21.15.139:0/1491824948 shutdown_connections 2024-09-16T00:07:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.772+0000 7f144cae6640 1 --2- 172.21.15.139:0/1491824948 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f142806c2b0 0x7f142806e770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.772+0000 7f144cae6640 1 --2- 172.21.15.139:0/1491824948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1448151680 0x7f144814c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.772+0000 7f144cae6640 1 --2- 172.21.15.139:0/1491824948 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14480c70d0 0x7f144814c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.772+0000 7f144cae6640 1 --2- 172.21.15.139:0/1491824948 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14480c56d0 0x7f14481f4400 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f143000c480 tx=0x7f143000c940 comp rx=0 tx=0).stop 2024-09-16T00:07:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.772+0000 7f144cae6640 1 -- 172.21.15.139:0/1491824948 >> 172.21.15.139:0/1491824948 conn(0x7f14480c91d0 msgr2=0x7f144806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.772+0000 7f144cae6640 1 -- 172.21.15.139:0/1491824948 shutdown_connections 2024-09-16T00:07:45.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.772+0000 7f144cae6640 1 -- 172.21.15.139:0/1491824948 wait complete. 2024-09-16T00:07:45.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.772+0000 7f4996ffd640 1 -- 172.21.15.139:0/2176104007 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f49980a9e20 con 0x7f49a80c3520 2024-09-16T00:07:45.775 INFO:teuthology.orchestra.run.smithi139.stdout:60129542151 2024-09-16T00:07:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.774+0000 7ffa3613b640 1 -- 172.21.15.139:0/625983316 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffa1c06c3b0 msgr2=0x7ffa1c06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.774+0000 7ffa3613b640 1 --2- 172.21.15.139:0/625983316 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffa1c06c3b0 0x7ffa1c06e870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffa3014f510 tx=0x7ffa24009040 comp rx=0 tx=0).stop 2024-09-16T00:07:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.774+0000 7ffa3613b640 1 -- 172.21.15.139:0/625983316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa18003b40 msgr2=0x7ffa301e1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.774+0000 7ffa3613b640 1 --2- 172.21.15.139:0/625983316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa18003b40 0x7ffa301e1b10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000c360 tx=0x7ffa20001970 comp rx=0 tx=0).stop 2024-09-16T00:07:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.774+0000 7ffa3613b640 1 -- 172.21.15.139:0/625983316 shutdown_connections 2024-09-16T00:07:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.774+0000 7ffa3613b640 1 --2- 172.21.15.139:0/625983316 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffa1c06c3b0 0x7ffa1c06e870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.774+0000 7ffa3613b640 1 --2- 172.21.15.139:0/625983316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa3014cc20 0x7ffa301456d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.774+0000 7ffa3613b640 1 --2- 172.21.15.139:0/625983316 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffa3014f890 0x7ffa30156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.774+0000 7ffa3613b640 1 --2- 172.21.15.139:0/625983316 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffa18003b40 0x7ffa301e1b10 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000c360 tx=0x7ffa20001970 comp rx=0 tx=0).stop 2024-09-16T00:07:45.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.774+0000 7ffa3613b640 1 -- 172.21.15.139:0/625983316 >> 172.21.15.139:0/625983316 conn(0x7ffa30070460 msgr2=0x7ffa30147a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.775+0000 7ffa3613b640 1 -- 172.21.15.139:0/625983316 shutdown_connections 2024-09-16T00:07:45.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.775+0000 7ffa3613b640 1 -- 172.21.15.139:0/625983316 wait complete. 2024-09-16T00:07:45.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.775+0000 7f49acadd640 1 -- 172.21.15.139:0/2176104007 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f498806c300 msgr2=0x7f498806e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.775+0000 7f49acadd640 1 --2- 172.21.15.139:0/2176104007 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f498806c300 0x7f498806e7c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f49a0007fd0 tx=0x7f49a00078d0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.775+0000 7f49acadd640 1 -- 172.21.15.139:0/2176104007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49a80c3520 msgr2=0x7f49a81453d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.775+0000 7f49acadd640 1 --2- 172.21.15.139:0/2176104007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49a80c3520 0x7f49a81453d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4998037000 tx=0x7f49980046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.776+0000 7f49acadd640 1 -- 172.21.15.139:0/2176104007 shutdown_connections 2024-09-16T00:07:45.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.776+0000 7f49acadd640 1 --2- 172.21.15.139:0/2176104007 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f498806c300 0x7f498806e7c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.776+0000 7f49acadd640 1 --2- 172.21.15.139:0/2176104007 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f49a815a2b0 0x7f49a814a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.776+0000 7f49acadd640 1 --2- 172.21.15.139:0/2176104007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49a8150f10 0x7f49a814a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.776+0000 7f49acadd640 1 --2- 172.21.15.139:0/2176104007 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f49a80c3520 0x7f49a81453d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.776+0000 7f49acadd640 1 -- 172.21.15.139:0/2176104007 >> 172.21.15.139:0/2176104007 conn(0x7f49a80c91d0 msgr2=0x7f49a80cabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.776+0000 7f49acadd640 1 -- 172.21.15.139:0/2176104007 shutdown_connections 2024-09-16T00:07:45.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.776+0000 7f49acadd640 1 -- 172.21.15.139:0/2176104007 wait complete. 2024-09-16T00:07:45.780 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.5 2024-09-16T00:07:45.781 DEBUG:teuthology.parallel:result is None 2024-09-16T00:07:45.781 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.3 2024-09-16T00:07:45.781 DEBUG:teuthology.parallel:result is None 2024-09-16T00:07:45.784 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.0 2024-09-16T00:07:45.784 DEBUG:teuthology.parallel:result is None 2024-09-16T00:07:45.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.7 2024-09-16T00:07:45.790 DEBUG:teuthology.parallel:result is None 2024-09-16T00:07:45.793 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-09-16T00:07:45.793 DEBUG:teuthology.parallel:result is None 2024-09-16T00:07:45.795 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.1 2024-09-16T00:07:45.796 DEBUG:teuthology.parallel:result is None 2024-09-16T00:07:45.796 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T00:07:45.796 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:07:45.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.932+0000 7f4cdce77640 1 Processor -- start 2024-09-16T00:07:45.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.932+0000 7f4cdce77640 1 -- start start 2024-09-16T00:07:45.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.933+0000 7f4cdce77640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4cd815e0c0 0x7f4cd815e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.933+0000 7f4cdce77640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cd80cba00 0x7f4cd80c8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.933+0000 7f4cdce77640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cd80cbf40 con 0x7f4cd815e9e0 2024-09-16T00:07:45.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.933+0000 7f4cdce77640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f4cd80c7cb0 con 0x7f4cd815e0c0 2024-09-16T00:07:45.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.933+0000 7f4cdce77640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd80c8270 con 0x7f4cd80cba00 2024-09-16T00:07:45.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.933+0000 7f4cd5d9b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cd80cba00 0x7f4cd80c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.933+0000 7f4cd659c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4cd815e0c0 0x7f4cd815e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.933+0000 7f4cd6d9d640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f4cd815e9e0 0x7f4cd806cf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:35898/0 (socket says 172.21.15.139:35898) 2024-09-16T00:07:45.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.933+0000 7f4cd5d9b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cd80cba00 0x7f4cd80c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:55392/0 (socket says 172.21.15.139:55392) 2024-09-16T00:07:45.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.933+0000 7f4cd6d9d640 1 -- 172.21.15.139:0/2551843160 learned_addr learned my addr 172.21.15.139:0/2551843160 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:45.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.933+0000 7f4cd659c640 1 --2- 172.21.15.139:0/2551843160 >> v2:172.21.15.164:3300/0 conn(0x7f4cd815e0c0 0x7f4cd815e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39762/0 (socket says 172.21.15.139:39762) 2024-09-16T00:07:45.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.934+0000 7f4cd659c640 1 -- 172.21.15.139:0/2551843160 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cd80cba00 msgr2=0x7f4cd80c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:45.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.934+0000 7f4cd659c640 1 --2- 172.21.15.139:0/2551843160 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cd80cba00 0x7f4cd80c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.934+0000 7f4cd659c640 1 -- 172.21.15.139:0/2551843160 >> v1:172.21.15.164:6789/0 conn(0x7f4cd815e9e0 legacy=0x7f4cd806cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.934+0000 7f4cd659c640 1 -- 172.21.15.139:0/2551843160 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd80cc200 con 0x7f4cd815e0c0 2024-09-16T00:07:45.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.934+0000 7f4cd5d9b640 1 --2- 172.21.15.139:0/2551843160 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cd80cba00 0x7f4cd80c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:45.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.935+0000 7f4cd659c640 1 --2- 172.21.15.139:0/2551843160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cd815e0c0 0x7f4cd815e4a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80c7e30 tx=0x7f4cc802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c1868cdbcb12b06 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.935+0000 7f4cd559a640 1 -- 172.21.15.139:0/2551843160 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8037070 con 0x7f4cd815e0c0 2024-09-16T00:07:45.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.935+0000 7f4cd559a640 1 -- 172.21.15.139:0/2551843160 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc8032440 con 0x7f4cd815e0c0 2024-09-16T00:07:45.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.935+0000 7f4cd559a640 1 -- 172.21.15.139:0/2551843160 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8032810 con 0x7f4cd815e0c0 2024-09-16T00:07:45.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.935+0000 7f4cdce77640 1 -- 172.21.15.139:0/2551843160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cd815e0c0 msgr2=0x7f4cd815e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.935+0000 7f4cdce77640 1 --2- 172.21.15.139:0/2551843160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cd815e0c0 0x7f4cd815e4a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80c7e30 tx=0x7f4cc802a210 comp rx=0 tx=0).stop 2024-09-16T00:07:45.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.935+0000 7f4cdce77640 1 -- 172.21.15.139:0/2551843160 shutdown_connections 2024-09-16T00:07:45.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.935+0000 7f4cdce77640 1 --2- 172.21.15.139:0/2551843160 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cd80cba00 0x7f4cd80c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.935+0000 7f4cdce77640 1 --2- 172.21.15.139:0/2551843160 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cd815e0c0 0x7f4cd815e4a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.935+0000 7f4cdce77640 1 -- 172.21.15.139:0/2551843160 >> 172.21.15.139:0/2551843160 conn(0x7f4cd80d03b0 msgr2=0x7f4cd80d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:45.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.935+0000 7f4cdce77640 1 -- 172.21.15.139:0/2551843160 shutdown_connections 2024-09-16T00:07:45.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.935+0000 7f4cdce77640 1 -- 172.21.15.139:0/2551843160 wait complete. 2024-09-16T00:07:45.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.937+0000 7f4cdce77640 1 Processor -- start 2024-09-16T00:07:45.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.937+0000 7f4cdce77640 1 -- start start 2024-09-16T00:07:45.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.937+0000 7f4cdce77640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cd80cba00 0x7f4cd815cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.937+0000 7f4cdce77640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cd815e0c0 0x7f4cd81f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.937+0000 7f4cdce77640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cd815e9e0 0x7f4cd81f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.937+0000 7f4cdce77640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd80c76f0 con 0x7f4cd815e0c0 2024-09-16T00:07:45.943 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.937+0000 7f4cdce77640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd80c7f00 con 0x7f4cd815e9e0 2024-09-16T00:07:45.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.937+0000 7f4cdce77640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd80c7870 con 0x7f4cd80cba00 2024-09-16T00:07:45.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.937+0000 7f4cd5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cd815e0c0 0x7f4cd81f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.937+0000 7f4cd5d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cd815e0c0 0x7f4cd81f0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40706/0 (socket says 172.21.15.139:40706) 2024-09-16T00:07:45.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.937+0000 7f4cd5d9b640 1 -- 172.21.15.139:0/3418323848 learned_addr learned my addr 172.21.15.139:0/3418323848 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:45.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.938+0000 7f4cd6d9d640 1 --2- 172.21.15.139:0/3418323848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cd815e9e0 0x7f4cd81f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.938+0000 7f4cd659c640 1 --2- 172.21.15.139:0/3418323848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cd80cba00 0x7f4cd815cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.938+0000 7f4cd5d9b640 1 -- 172.21.15.139:0/3418323848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cd80cba00 msgr2=0x7f4cd815cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.938+0000 7f4cd5d9b640 1 --2- 172.21.15.139:0/3418323848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cd80cba00 0x7f4cd815cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.938+0000 7f4cd5d9b640 1 -- 172.21.15.139:0/3418323848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cd815e9e0 msgr2=0x7f4cd81f11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:45.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.938+0000 7f4cd5d9b640 1 --2- 172.21.15.139:0/3418323848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cd815e9e0 0x7f4cd81f11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:45.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.938+0000 7f4cd5d9b640 1 -- 172.21.15.139:0/3418323848 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd815b460 con 0x7f4cd815e0c0 2024-09-16T00:07:45.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.938+0000 7f4cd6d9d640 1 --2- 172.21.15.139:0/3418323848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cd815e9e0 0x7f4cd81f11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:45.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.938+0000 7f4cd5d9b640 1 --2- 172.21.15.139:0/3418323848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cd815e0c0 0x7f4cd81f0c70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00ebe0 tx=0x7f4ccc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.938+0000 7f4cc6ffd640 1 -- 172.21.15.139:0/3418323848 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc017070 con 0x7f4cd815e0c0 2024-09-16T00:07:45.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.938+0000 7f4cc6ffd640 1 -- 172.21.15.139:0/3418323848 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ccc00cbd0 con 0x7f4cd815e0c0 2024-09-16T00:07:45.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.938+0000 7f4cdce77640 1 -- 172.21.15.139:0/3418323848 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd81506b0 con 0x7f4cd815e0c0 2024-09-16T00:07:45.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.938+0000 7f4cc6ffd640 1 -- 172.21.15.139:0/3418323848 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc013470 con 0x7f4cd815e0c0 2024-09-16T00:07:45.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.938+0000 7f4cdce77640 1 -- 172.21.15.139:0/3418323848 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd8150c70 con 0x7f4cd815e0c0 2024-09-16T00:07:45.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.939+0000 7f4cdce77640 1 -- 172.21.15.139:0/3418323848 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd80ca820 con 0x7f4cd815e0c0 2024-09-16T00:07:45.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.940+0000 7f4cc6ffd640 1 -- 172.21.15.139:0/3418323848 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4ccc01a070 con 0x7f4cd815e0c0 2024-09-16T00:07:45.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.940+0000 7f4cc6ffd640 1 --2- 172.21.15.139:0/3418323848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4cb806c2e0 0x7f4cb806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:45.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.940+0000 7f4cc6ffd640 1 -- 172.21.15.139:0/3418323848 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4ccc08b330 con 0x7f4cd815e0c0 2024-09-16T00:07:45.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.940+0000 7f4cd659c640 1 --2- 172.21.15.139:0/3418323848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4cb806c2e0 0x7f4cb806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:45.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.940+0000 7f4cd659c640 1 --2- 172.21.15.139:0/3418323848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4cb806c2e0 0x7f4cb806e7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8034000 tx=0x7f4cc802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:45.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:45.944+0000 7f4cc6ffd640 1 -- 172.21.15.139:0/3418323848 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4cd80ca820 con 0x7f4cd815e0c0 2024-09-16T00:07:46.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.107+0000 7f4cdce77640 1 -- 172.21.15.139:0/3418323848 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4cd8150460 con 0x7f4cb806c2e0 2024-09-16T00:07:46.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.113+0000 7f4cc6ffd640 1 -- 172.21.15.139:0/3418323848 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251913 (secure 0 0 0) 0x7f4cd8150460 con 0x7f4cb806c2e0 2024-09-16T00:07:46.119 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:46.120 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:07:46.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.116+0000 7f4cdce77640 1 -- 172.21.15.139:0/3418323848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4cb806c2e0 msgr2=0x7f4cb806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.116+0000 7f4cdce77640 1 --2- 172.21.15.139:0/3418323848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4cb806c2e0 0x7f4cb806e7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8034000 tx=0x7f4cc802e000 comp rx=0 tx=0).stop 2024-09-16T00:07:46.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.116+0000 7f4cdce77640 1 -- 172.21.15.139:0/3418323848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cd815e0c0 msgr2=0x7f4cd81f0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.116+0000 7f4cdce77640 1 --2- 172.21.15.139:0/3418323848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cd815e0c0 0x7f4cd81f0c70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00ebe0 tx=0x7f4ccc00c520 comp rx=0 tx=0).stop 2024-09-16T00:07:46.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.116+0000 7f4cdce77640 1 -- 172.21.15.139:0/3418323848 shutdown_connections 2024-09-16T00:07:46.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.116+0000 7f4cdce77640 1 --2- 172.21.15.139:0/3418323848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4cb806c2e0 0x7f4cb806e7a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.116+0000 7f4cdce77640 1 --2- 172.21.15.139:0/3418323848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4cd815e9e0 0x7f4cd81f11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.116+0000 7f4cdce77640 1 --2- 172.21.15.139:0/3418323848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cd815e0c0 0x7f4cd81f0c70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.116+0000 7f4cdce77640 1 --2- 172.21.15.139:0/3418323848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4cd80cba00 0x7f4cd815cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.116+0000 7f4cdce77640 1 -- 172.21.15.139:0/3418323848 >> 172.21.15.139:0/3418323848 conn(0x7f4cd80d03b0 msgr2=0x7f4cd81538b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:46.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.117+0000 7f4cdce77640 1 -- 172.21.15.139:0/3418323848 shutdown_connections 2024-09-16T00:07:46.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.117+0000 7f4cdce77640 1 -- 172.21.15.139:0/3418323848 wait complete. 2024-09-16T00:07:46.128 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-16T00:07:46.028091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218112,"kb_used_data":1920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756608,"statfs":{"total":773094113280,"available":772870766592,"internally_reserved":0,"allocated":1966080,"data_stored":1809524,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.416234"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115528+0000","last_change":"2024-09-16T00:07:33.635096+0000","last_active":"2024-09-16T00:07:35.115528+0000","last_peered":"2024-09-16T00:07:35.115528+0000","last_clean":"2024-09-16T00:07:35.115528+0000","last_became_active":"2024-09-16T00:07:33.634750+0000","last_became_peered":"2024-09-16T00:07:33.634750+0000","last_unstale":"2024-09-16T00:07:35.115528+0000","last_undegraded":"2024-09-16T00:07:35.115528+0000","last_fullsized":"2024-09-16T00:07:35.115528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115558+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:07:35.115558+0000","last_peered":"2024-09-16T00:07:35.115558+0000","last_clean":"2024-09-16T00:07:35.115558+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:07:35.115558+0000","last_undegraded":"2024-09-16T00:07:35.115558+0000","last_fullsized":"2024-09-16T00:07:35.115558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115392+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:07:35.115392+0000","last_peered":"2024-09-16T00:07:35.115392+0000","last_clean":"2024-09-16T00:07:35.115392+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:07:35.115392+0000","last_undegraded":"2024-09-16T00:07:35.115392+0000","last_fullsized":"2024-09-16T00:07:35.115392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615167+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:07:34.615167+0000","last_peered":"2024-09-16T00:07:34.615167+0000","last_clean":"2024-09-16T00:07:34.615167+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:07:34.615167+0000","last_undegraded":"2024-09-16T00:07:34.615167+0000","last_fullsized":"2024-09-16T00:07:34.615167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115724+0000","last_change":"2024-09-16T00:07:33.630029+0000","last_active":"2024-09-16T00:07:35.115724+0000","last_peered":"2024-09-16T00:07:35.115724+0000","last_clean":"2024-09-16T00:07:35.115724+0000","last_became_active":"2024-09-16T00:07:33.629712+0000","last_became_peered":"2024-09-16T00:07:33.629712+0000","last_unstale":"2024-09-16T00:07:35.115724+0000","last_undegraded":"2024-09-16T00:07:35.115724+0000","last_fullsized":"2024-09-16T00:07:35.115724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614973+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:07:34.614973+0000","last_peered":"2024-09-16T00:07:34.614973+0000","last_clean":"2024-09-16T00:07:34.614973+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:07:34.614973+0000","last_undegraded":"2024-09-16T00:07:34.614973+0000","last_fullsized":"2024-09-16T00:07:34.614973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115522+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:07:35.115522+0000","last_peered":"2024-09-16T00:07:35.115522+0000","last_clean":"2024-09-16T00:07:35.115522+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:07:35.115522+0000","last_undegraded":"2024-09-16T00:07:35.115522+0000","last_fullsized":"2024-09-16T00:07:35.115522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114492+0000","last_change":"2024-09-16T00:07:33.624708+0000","last_active":"2024-09-16T00:07:35.114492+0000","last_peered":"2024-09-16T00:07:35.114492+0000","last_clean":"2024-09-16T00:07:35.114492+0000","last_became_active":"2024-09-16T00:07:33.624458+0000","last_became_peered":"2024-09-16T00:07:33.624458+0000","last_unstale":"2024-09-16T00:07:35.114492+0000","last_undegraded":"2024-09-16T00:07:35.114492+0000","last_fullsized":"2024-09-16T00:07:35.114492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:48.052488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115751+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:07:35.115751+0000","last_peered":"2024-09-16T00:07:35.115751+0000","last_clean":"2024-09-16T00:07:35.115751+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:07:35.115751+0000","last_undegraded":"2024-09-16T00:07:35.115751+0000","last_fullsized":"2024-09-16T00:07:35.115751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115614+0000","last_change":"2024-09-16T00:07:33.635973+0000","last_active":"2024-09-16T00:07:35.115614+0000","last_peered":"2024-09-16T00:07:35.115614+0000","last_clean":"2024-09-16T00:07:35.115614+0000","last_became_active":"2024-09-16T00:07:33.635486+0000","last_became_peered":"2024-09-16T00:07:33.635486+0000","last_unstale":"2024-09-16T00:07:35.115614+0000","last_undegraded":"2024-09-16T00:07:35.115614+0000","last_fullsized":"2024-09-16T00:07:35.115614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115027+0000","last_change":"2024-09-16T00:07:33.636971+0000","last_active":"2024-09-16T00:07:35.115027+0000","last_peered":"2024-09-16T00:07:35.115027+0000","last_clean":"2024-09-16T00:07:35.115027+0000","last_became_active":"2024-09-16T00:07:33.635760+0000","last_became_peered":"2024-09-16T00:07:33.635760+0000","last_unstale":"2024-09-16T00:07:35.115027+0000","last_undegraded":"2024-09-16T00:07:35.115027+0000","last_fullsized":"2024-09-16T00:07:35.115027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614631+0000","last_change":"2024-09-16T00:07:33.632261+0000","last_active":"2024-09-16T00:07:34.614631+0000","last_peered":"2024-09-16T00:07:34.614631+0000","last_clean":"2024-09-16T00:07:34.614631+0000","last_became_active":"2024-09-16T00:07:33.631944+0000","last_became_peered":"2024-09-16T00:07:33.631944+0000","last_unstale":"2024-09-16T00:07:34.614631+0000","last_undegraded":"2024-09-16T00:07:34.614631+0000","last_fullsized":"2024-09-16T00:07:34.614631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:35.052417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114641+0000","last_change":"2024-09-16T00:07:33.628273+0000","last_active":"2024-09-16T00:07:35.114641+0000","last_peered":"2024-09-16T00:07:35.114641+0000","last_clean":"2024-09-16T00:07:35.114641+0000","last_became_active":"2024-09-16T00:07:33.627960+0000","last_became_peered":"2024-09-16T00:07:33.627960+0000","last_unstale":"2024-09-16T00:07:35.114641+0000","last_undegraded":"2024-09-16T00:07:35.114641+0000","last_fullsized":"2024-09-16T00:07:35.114641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:36.451688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114782+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:07:35.114782+0000","last_peered":"2024-09-16T00:07:35.114782+0000","last_clean":"2024-09-16T00:07:35.114782+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:07:35.114782+0000","last_undegraded":"2024-09-16T00:07:35.114782+0000","last_fullsized":"2024-09-16T00:07:35.114782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114664+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:07:35.114664+0000","last_peered":"2024-09-16T00:07:35.114664+0000","last_clean":"2024-09-16T00:07:35.114664+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:07:35.114664+0000","last_undegraded":"2024-09-16T00:07:35.114664+0000","last_fullsized":"2024-09-16T00:07:35.114664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614471+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:07:34.614471+0000","last_peered":"2024-09-16T00:07:34.614471+0000","last_clean":"2024-09-16T00:07:34.614471+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:07:34.614471+0000","last_undegraded":"2024-09-16T00:07:34.614471+0000","last_fullsized":"2024-09-16T00:07:34.614471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615697+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:07:34.615697+0000","last_peered":"2024-09-16T00:07:34.615697+0000","last_clean":"2024-09-16T00:07:34.615697+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:07:34.615697+0000","last_undegraded":"2024-09-16T00:07:34.615697+0000","last_fullsized":"2024-09-16T00:07:34.615697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613100+0000","last_change":"2024-09-16T00:07:33.630797+0000","last_active":"2024-09-16T00:07:34.613100+0000","last_peered":"2024-09-16T00:07:34.613100+0000","last_clean":"2024-09-16T00:07:34.613100+0000","last_became_active":"2024-09-16T00:07:33.630491+0000","last_became_peered":"2024-09-16T00:07:33.630491+0000","last_unstale":"2024-09-16T00:07:34.613100+0000","last_undegraded":"2024-09-16T00:07:34.613100+0000","last_fullsized":"2024-09-16T00:07:34.613100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:20.097626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115840+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:07:35.115840+0000","last_peered":"2024-09-16T00:07:35.115840+0000","last_clean":"2024-09-16T00:07:35.115840+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:07:35.115840+0000","last_undegraded":"2024-09-16T00:07:35.115840+0000","last_fullsized":"2024-09-16T00:07:35.115840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114960+0000","last_change":"2024-09-16T00:07:33.638554+0000","last_active":"2024-09-16T00:07:35.114960+0000","last_peered":"2024-09-16T00:07:35.114960+0000","last_clean":"2024-09-16T00:07:35.114960+0000","last_became_active":"2024-09-16T00:07:33.638263+0000","last_became_peered":"2024-09-16T00:07:33.638263+0000","last_unstale":"2024-09-16T00:07:35.114960+0000","last_undegraded":"2024-09-16T00:07:35.114960+0000","last_fullsized":"2024-09-16T00:07:35.114960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115286+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:07:35.115286+0000","last_peered":"2024-09-16T00:07:35.115286+0000","last_clean":"2024-09-16T00:07:35.115286+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:07:35.115286+0000","last_undegraded":"2024-09-16T00:07:35.115286+0000","last_fullsized":"2024-09-16T00:07:35.115286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615450+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:07:34.615450+0000","last_peered":"2024-09-16T00:07:34.615450+0000","last_clean":"2024-09-16T00:07:34.615450+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:07:34.615450+0000","last_undegraded":"2024-09-16T00:07:34.615450+0000","last_fullsized":"2024-09-16T00:07:34.615450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115015+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:07:35.115015+0000","last_peered":"2024-09-16T00:07:35.115015+0000","last_clean":"2024-09-16T00:07:35.115015+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:07:35.115015+0000","last_undegraded":"2024-09-16T00:07:35.115015+0000","last_fullsized":"2024-09-16T00:07:35.115015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614616+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:07:34.614616+0000","last_peered":"2024-09-16T00:07:34.614616+0000","last_clean":"2024-09-16T00:07:34.614616+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:07:34.614616+0000","last_undegraded":"2024-09-16T00:07:34.614616+0000","last_fullsized":"2024-09-16T00:07:34.614616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114694+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:07:35.114694+0000","last_peered":"2024-09-16T00:07:35.114694+0000","last_clean":"2024-09-16T00:07:35.114694+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:07:35.114694+0000","last_undegraded":"2024-09-16T00:07:35.114694+0000","last_fullsized":"2024-09-16T00:07:35.114694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115306+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:07:35.115306+0000","last_peered":"2024-09-16T00:07:35.115306+0000","last_clean":"2024-09-16T00:07:35.115306+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:07:35.115306+0000","last_undegraded":"2024-09-16T00:07:35.115306+0000","last_fullsized":"2024-09-16T00:07:35.115306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115543+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:07:35.115543+0000","last_peered":"2024-09-16T00:07:35.115543+0000","last_clean":"2024-09-16T00:07:35.115543+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:07:35.115543+0000","last_undegraded":"2024-09-16T00:07:35.115543+0000","last_fullsized":"2024-09-16T00:07:35.115543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114985+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:07:35.114985+0000","last_peered":"2024-09-16T00:07:35.114985+0000","last_clean":"2024-09-16T00:07:35.114985+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:07:35.114985+0000","last_undegraded":"2024-09-16T00:07:35.114985+0000","last_fullsized":"2024-09-16T00:07:35.114985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115448+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:07:35.115448+0000","last_peered":"2024-09-16T00:07:35.115448+0000","last_clean":"2024-09-16T00:07:35.115448+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:07:35.115448+0000","last_undegraded":"2024-09-16T00:07:35.115448+0000","last_fullsized":"2024-09-16T00:07:35.115448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615338+0000","last_change":"2024-09-16T00:07:33.627789+0000","last_active":"2024-09-16T00:07:34.615338+0000","last_peered":"2024-09-16T00:07:34.615338+0000","last_clean":"2024-09-16T00:07:34.615338+0000","last_became_active":"2024-09-16T00:07:33.627500+0000","last_became_peered":"2024-09-16T00:07:33.627500+0000","last_unstale":"2024-09-16T00:07:34.615338+0000","last_undegraded":"2024-09-16T00:07:34.615338+0000","last_fullsized":"2024-09-16T00:07:34.615338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613050+0000","last_change":"2024-09-16T00:07:33.621107+0000","last_active":"2024-09-16T00:07:34.613050+0000","last_peered":"2024-09-16T00:07:34.613050+0000","last_clean":"2024-09-16T00:07:34.613050+0000","last_became_active":"2024-09-16T00:07:33.620858+0000","last_became_peered":"2024-09-16T00:07:33.620858+0000","last_unstale":"2024-09-16T00:07:34.613050+0000","last_undegraded":"2024-09-16T00:07:34.613050+0000","last_fullsized":"2024-09-16T00:07:34.613050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:30.176079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115031+0000","last_change":"2024-09-16T00:07:33.633578+0000","last_active":"2024-09-16T00:07:35.115031+0000","last_peered":"2024-09-16T00:07:35.115031+0000","last_clean":"2024-09-16T00:07:35.115031+0000","last_became_active":"2024-09-16T00:07:33.633310+0000","last_became_peered":"2024-09-16T00:07:33.633310+0000","last_unstale":"2024-09-16T00:07:35.115031+0000","last_undegraded":"2024-09-16T00:07:35.115031+0000","last_fullsized":"2024-09-16T00:07:35.115031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614556+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:07:34.614556+0000","last_peered":"2024-09-16T00:07:34.614556+0000","last_clean":"2024-09-16T00:07:34.614556+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:07:34.614556+0000","last_undegraded":"2024-09-16T00:07:34.614556+0000","last_fullsized":"2024-09-16T00:07:34.614556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.059131+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:07:36.059131+0000","last_peered":"2024-09-16T00:07:36.059131+0000","last_clean":"2024-09-16T00:07:36.059131+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:07:36.059131+0000","last_undegraded":"2024-09-16T00:07:36.059131+0000","last_fullsized":"2024-09-16T00:07:36.059131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115509+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:07:35.115509+0000","last_peered":"2024-09-16T00:07:35.115509+0000","last_clean":"2024-09-16T00:07:35.115509+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:07:35.115509+0000","last_undegraded":"2024-09-16T00:07:35.115509+0000","last_fullsized":"2024-09-16T00:07:35.115509+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114845+0000","last_change":"2024-09-16T00:07:32.626800+0000","last_active":"2024-09-16T00:07:35.114845+0000","last_peered":"2024-09-16T00:07:35.114845+0000","last_clean":"2024-09-16T00:07:35.114845+0000","last_became_active":"2024-09-16T00:07:32.626511+0000","last_became_peered":"2024-09-16T00:07:32.626511+0000","last_unstale":"2024-09-16T00:07:35.114845+0000","last_undegraded":"2024-09-16T00:07:35.114845+0000","last_fullsized":"2024-09-16T00:07:35.114845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115156+0000","last_change":"2024-09-16T00:07:32.629539+0000","last_active":"2024-09-16T00:07:35.115156+0000","last_peered":"2024-09-16T00:07:35.115156+0000","last_clean":"2024-09-16T00:07:35.115156+0000","last_became_active":"2024-09-16T00:07:32.629186+0000","last_became_peered":"2024-09-16T00:07:32.629186+0000","last_unstale":"2024-09-16T00:07:35.115156+0000","last_undegraded":"2024-09-16T00:07:35.115156+0000","last_fullsized":"2024-09-16T00:07:35.115156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:59:32.860208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114535+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:07:35.114535+0000","last_peered":"2024-09-16T00:07:35.114535+0000","last_clean":"2024-09-16T00:07:35.114535+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:07:35.114535+0000","last_undegraded":"2024-09-16T00:07:35.114535+0000","last_fullsized":"2024-09-16T00:07:35.114535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614668+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:07:34.614668+0000","last_peered":"2024-09-16T00:07:34.614668+0000","last_clean":"2024-09-16T00:07:34.614668+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:07:34.614668+0000","last_undegraded":"2024-09-16T00:07:34.614668+0000","last_fullsized":"2024-09-16T00:07:34.614668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114527+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:07:35.114527+0000","last_peered":"2024-09-16T00:07:35.114527+0000","last_clean":"2024-09-16T00:07:35.114527+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:07:35.114527+0000","last_undegraded":"2024-09-16T00:07:35.114527+0000","last_fullsized":"2024-09-16T00:07:35.114527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115138+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:07:35.115138+0000","last_peered":"2024-09-16T00:07:35.115138+0000","last_clean":"2024-09-16T00:07:35.115138+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:07:35.115138+0000","last_undegraded":"2024-09-16T00:07:35.115138+0000","last_fullsized":"2024-09-16T00:07:35.115138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615408+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:07:34.615408+0000","last_peered":"2024-09-16T00:07:34.615408+0000","last_clean":"2024-09-16T00:07:34.615408+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:07:34.615408+0000","last_undegraded":"2024-09-16T00:07:34.615408+0000","last_fullsized":"2024-09-16T00:07:34.615408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615356+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:07:34.615356+0000","last_peered":"2024-09-16T00:07:34.615356+0000","last_clean":"2024-09-16T00:07:34.615356+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:07:34.615356+0000","last_undegraded":"2024-09-16T00:07:34.615356+0000","last_fullsized":"2024-09-16T00:07:34.615356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115277+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:07:35.115277+0000","last_peered":"2024-09-16T00:07:35.115277+0000","last_clean":"2024-09-16T00:07:35.115277+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:07:35.115277+0000","last_undegraded":"2024-09-16T00:07:35.115277+0000","last_fullsized":"2024-09-16T00:07:35.115277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.619745+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:07:38.619745+0000","last_peered":"2024-09-16T00:07:38.619745+0000","last_clean":"2024-09-16T00:07:38.619745+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:07:38.619745+0000","last_undegraded":"2024-09-16T00:07:38.619745+0000","last_fullsized":"2024-09-16T00:07:38.619745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614648+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:07:34.614648+0000","last_peered":"2024-09-16T00:07:34.614648+0000","last_clean":"2024-09-16T00:07:34.614648+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:07:34.614648+0000","last_undegraded":"2024-09-16T00:07:34.614648+0000","last_fullsized":"2024-09-16T00:07:34.614648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.622263+0000","last_change":"2024-09-16T00:07:32.629132+0000","last_active":"2024-09-16T00:07:38.622263+0000","last_peered":"2024-09-16T00:07:38.622263+0000","last_clean":"2024-09-16T00:07:38.622263+0000","last_became_active":"2024-09-16T00:07:32.628803+0000","last_became_peered":"2024-09-16T00:07:32.628803+0000","last_unstale":"2024-09-16T00:07:38.622263+0000","last_undegraded":"2024-09-16T00:07:38.622263+0000","last_fullsized":"2024-09-16T00:07:38.622263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115230+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:07:35.115230+0000","last_peered":"2024-09-16T00:07:35.115230+0000","last_clean":"2024-09-16T00:07:35.115230+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:07:35.115230+0000","last_undegraded":"2024-09-16T00:07:35.115230+0000","last_fullsized":"2024-09-16T00:07:35.115230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620127+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:07:38.620127+0000","last_peered":"2024-09-16T00:07:38.620127+0000","last_clean":"2024-09-16T00:07:38.620127+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:07:38.620127+0000","last_undegraded":"2024-09-16T00:07:38.620127+0000","last_fullsized":"2024-09-16T00:07:38.620127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115121+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:07:35.115121+0000","last_peered":"2024-09-16T00:07:35.115121+0000","last_clean":"2024-09-16T00:07:35.115121+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:07:35.115121+0000","last_undegraded":"2024-09-16T00:07:35.115121+0000","last_fullsized":"2024-09-16T00:07:35.115121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615462+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:07:34.615462+0000","last_peered":"2024-09-16T00:07:34.615462+0000","last_clean":"2024-09-16T00:07:34.615462+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:07:34.615462+0000","last_undegraded":"2024-09-16T00:07:34.615462+0000","last_fullsized":"2024-09-16T00:07:34.615462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115766+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:07:35.115766+0000","last_peered":"2024-09-16T00:07:35.115766+0000","last_clean":"2024-09-16T00:07:35.115766+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:07:35.115766+0000","last_undegraded":"2024-09-16T00:07:35.115766+0000","last_fullsized":"2024-09-16T00:07:35.115766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114802+0000","last_change":"2024-09-16T00:07:32.626856+0000","last_active":"2024-09-16T00:07:35.114802+0000","last_peered":"2024-09-16T00:07:35.114802+0000","last_clean":"2024-09-16T00:07:35.114802+0000","last_became_active":"2024-09-16T00:07:32.626532+0000","last_became_peered":"2024-09-16T00:07:32.626532+0000","last_unstale":"2024-09-16T00:07:35.114802+0000","last_undegraded":"2024-09-16T00:07:35.114802+0000","last_fullsized":"2024-09-16T00:07:35.114802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114886+0000","last_change":"2024-09-16T00:07:33.632870+0000","last_active":"2024-09-16T00:07:35.114886+0000","last_peered":"2024-09-16T00:07:35.114886+0000","last_clean":"2024-09-16T00:07:35.114886+0000","last_became_active":"2024-09-16T00:07:33.632608+0000","last_became_peered":"2024-09-16T00:07:33.632608+0000","last_unstale":"2024-09-16T00:07:35.114886+0000","last_undegraded":"2024-09-16T00:07:35.114886+0000","last_fullsized":"2024-09-16T00:07:35.114886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.621550+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:07:38.621550+0000","last_peered":"2024-09-16T00:07:38.621550+0000","last_clean":"2024-09-16T00:07:38.621550+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:07:38.621550+0000","last_undegraded":"2024-09-16T00:07:38.621550+0000","last_fullsized":"2024-09-16T00:07:38.621550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115409+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:07:35.115409+0000","last_peered":"2024-09-16T00:07:35.115409+0000","last_clean":"2024-09-16T00:07:35.115409+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:07:35.115409+0000","last_undegraded":"2024-09-16T00:07:35.115409+0000","last_fullsized":"2024-09-16T00:07:35.115409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614761+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:07:34.614761+0000","last_peered":"2024-09-16T00:07:34.614761+0000","last_clean":"2024-09-16T00:07:34.614761+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:07:34.614761+0000","last_undegraded":"2024-09-16T00:07:34.614761+0000","last_fullsized":"2024-09-16T00:07:34.614761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115480+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:07:35.115480+0000","last_peered":"2024-09-16T00:07:35.115480+0000","last_clean":"2024-09-16T00:07:35.115480+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:07:35.115480+0000","last_undegraded":"2024-09-16T00:07:35.115480+0000","last_fullsized":"2024-09-16T00:07:35.115480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620942+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:07:38.620942+0000","last_peered":"2024-09-16T00:07:38.620942+0000","last_clean":"2024-09-16T00:07:38.620942+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:07:38.620942+0000","last_undegraded":"2024-09-16T00:07:38.620942+0000","last_fullsized":"2024-09-16T00:07:38.620942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115888+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:07:35.115888+0000","last_peered":"2024-09-16T00:07:35.115888+0000","last_clean":"2024-09-16T00:07:35.115888+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:07:35.115888+0000","last_undegraded":"2024-09-16T00:07:35.115888+0000","last_fullsized":"2024-09-16T00:07:35.115888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.048699+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:07:36.048699+0000","last_peered":"2024-09-16T00:07:36.048699+0000","last_clean":"2024-09-16T00:07:36.048699+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:07:36.048699+0000","last_undegraded":"2024-09-16T00:07:36.048699+0000","last_fullsized":"2024-09-16T00:07:36.048699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615597+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:07:34.615597+0000","last_peered":"2024-09-16T00:07:34.615597+0000","last_clean":"2024-09-16T00:07:34.615597+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:07:34.615597+0000","last_undegraded":"2024-09-16T00:07:34.615597+0000","last_fullsized":"2024-09-16T00:07:34.615597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613057+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:07:34.613057+0000","last_peered":"2024-09-16T00:07:34.613057+0000","last_clean":"2024-09-16T00:07:34.613057+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:07:34.613057+0000","last_undegraded":"2024-09-16T00:07:34.613057+0000","last_fullsized":"2024-09-16T00:07:34.613057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115577+0000","last_change":"2024-09-16T00:07:33.638312+0000","last_active":"2024-09-16T00:07:35.115577+0000","last_peered":"2024-09-16T00:07:35.115577+0000","last_clean":"2024-09-16T00:07:35.115577+0000","last_became_active":"2024-09-16T00:07:33.638049+0000","last_became_peered":"2024-09-16T00:07:33.638049+0000","last_unstale":"2024-09-16T00:07:35.115577+0000","last_undegraded":"2024-09-16T00:07:35.115577+0000","last_fullsized":"2024-09-16T00:07:35.115577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115169+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:07:35.115169+0000","last_peered":"2024-09-16T00:07:35.115169+0000","last_clean":"2024-09-16T00:07:35.115169+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:07:35.115169+0000","last_undegraded":"2024-09-16T00:07:35.115169+0000","last_fullsized":"2024-09-16T00:07:35.115169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614834+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:07:34.614834+0000","last_peered":"2024-09-16T00:07:34.614834+0000","last_clean":"2024-09-16T00:07:34.614834+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:07:34.614834+0000","last_undegraded":"2024-09-16T00:07:34.614834+0000","last_fullsized":"2024-09-16T00:07:34.614834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114892+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:07:35.114892+0000","last_peered":"2024-09-16T00:07:35.114892+0000","last_clean":"2024-09-16T00:07:35.114892+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:07:35.114892+0000","last_undegraded":"2024-09-16T00:07:35.114892+0000","last_fullsized":"2024-09-16T00:07:35.114892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115065+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:07:35.115065+0000","last_peered":"2024-09-16T00:07:35.115065+0000","last_clean":"2024-09-16T00:07:35.115065+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:07:35.115065+0000","last_undegraded":"2024-09-16T00:07:35.115065+0000","last_fullsized":"2024-09-16T00:07:35.115065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115288+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:07:35.115288+0000","last_peered":"2024-09-16T00:07:35.115288+0000","last_clean":"2024-09-16T00:07:35.115288+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:07:35.115288+0000","last_undegraded":"2024-09-16T00:07:35.115288+0000","last_fullsized":"2024-09-16T00:07:35.115288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.612974+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:07:34.612974+0000","last_peered":"2024-09-16T00:07:34.612974+0000","last_clean":"2024-09-16T00:07:34.612974+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:07:34.612974+0000","last_undegraded":"2024-09-16T00:07:34.612974+0000","last_fullsized":"2024-09-16T00:07:34.612974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114905+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:07:35.114905+0000","last_peered":"2024-09-16T00:07:35.114905+0000","last_clean":"2024-09-16T00:07:35.114905+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:07:35.114905+0000","last_undegraded":"2024-09-16T00:07:35.114905+0000","last_fullsized":"2024-09-16T00:07:35.114905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615621+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:07:34.615621+0000","last_peered":"2024-09-16T00:07:34.615621+0000","last_clean":"2024-09-16T00:07:34.615621+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:07:34.615621+0000","last_undegraded":"2024-09-16T00:07:34.615621+0000","last_fullsized":"2024-09-16T00:07:34.615621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615008+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:07:34.615008+0000","last_peered":"2024-09-16T00:07:34.615008+0000","last_clean":"2024-09-16T00:07:34.615008+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:07:34.615008+0000","last_undegraded":"2024-09-16T00:07:34.615008+0000","last_fullsized":"2024-09-16T00:07:34.615008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115398+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:07:35.115398+0000","last_peered":"2024-09-16T00:07:35.115398+0000","last_clean":"2024-09-16T00:07:35.115398+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:07:35.115398+0000","last_undegraded":"2024-09-16T00:07:35.115398+0000","last_fullsized":"2024-09-16T00:07:35.115398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615631+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:07:34.615631+0000","last_peered":"2024-09-16T00:07:34.615631+0000","last_clean":"2024-09-16T00:07:34.615631+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:07:34.615631+0000","last_undegraded":"2024-09-16T00:07:34.615631+0000","last_fullsized":"2024-09-16T00:07:34.615631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115814+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:07:35.115814+0000","last_peered":"2024-09-16T00:07:35.115814+0000","last_clean":"2024-09-16T00:07:35.115814+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:07:35.115814+0000","last_undegraded":"2024-09-16T00:07:35.115814+0000","last_fullsized":"2024-09-16T00:07:35.115814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115305+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:07:35.115305+0000","last_peered":"2024-09-16T00:07:35.115305+0000","last_clean":"2024-09-16T00:07:35.115305+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:07:35.115305+0000","last_undegraded":"2024-09-16T00:07:35.115305+0000","last_fullsized":"2024-09-16T00:07:35.115305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115644+0000","last_change":"2024-09-16T00:07:32.627236+0000","last_active":"2024-09-16T00:07:35.115644+0000","last_peered":"2024-09-16T00:07:35.115644+0000","last_clean":"2024-09-16T00:07:35.115644+0000","last_became_active":"2024-09-16T00:07:32.626743+0000","last_became_peered":"2024-09-16T00:07:32.626743+0000","last_unstale":"2024-09-16T00:07:35.115644+0000","last_undegraded":"2024-09-16T00:07:35.115644+0000","last_fullsized":"2024-09-16T00:07:35.115644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114835+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:07:35.114835+0000","last_peered":"2024-09-16T00:07:35.114835+0000","last_clean":"2024-09-16T00:07:35.114835+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:07:35.114835+0000","last_undegraded":"2024-09-16T00:07:35.114835+0000","last_fullsized":"2024-09-16T00:07:35.114835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620513+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:07:38.620513+0000","last_peered":"2024-09-16T00:07:38.620513+0000","last_clean":"2024-09-16T00:07:38.620513+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:07:38.620513+0000","last_undegraded":"2024-09-16T00:07:38.620513+0000","last_fullsized":"2024-09-16T00:07:38.620513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114804+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:07:35.114804+0000","last_peered":"2024-09-16T00:07:35.114804+0000","last_clean":"2024-09-16T00:07:35.114804+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:07:35.114804+0000","last_undegraded":"2024-09-16T00:07:35.114804+0000","last_fullsized":"2024-09-16T00:07:35.114804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114947+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:07:35.114947+0000","last_peered":"2024-09-16T00:07:35.114947+0000","last_clean":"2024-09-16T00:07:35.114947+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:07:35.114947+0000","last_undegraded":"2024-09-16T00:07:35.114947+0000","last_fullsized":"2024-09-16T00:07:35.114947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614501+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:07:34.614501+0000","last_peered":"2024-09-16T00:07:34.614501+0000","last_clean":"2024-09-16T00:07:34.614501+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:07:34.614501+0000","last_undegraded":"2024-09-16T00:07:34.614501+0000","last_fullsized":"2024-09-16T00:07:34.614501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114765+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:07:35.114765+0000","last_peered":"2024-09-16T00:07:35.114765+0000","last_clean":"2024-09-16T00:07:35.114765+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:07:35.114765+0000","last_undegraded":"2024-09-16T00:07:35.114765+0000","last_fullsized":"2024-09-16T00:07:35.114765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114640+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:07:35.114640+0000","last_peered":"2024-09-16T00:07:35.114640+0000","last_clean":"2024-09-16T00:07:35.114640+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:07:35.114640+0000","last_undegraded":"2024-09-16T00:07:35.114640+0000","last_fullsized":"2024-09-16T00:07:35.114640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115333+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:07:35.115333+0000","last_peered":"2024-09-16T00:07:35.115333+0000","last_clean":"2024-09-16T00:07:35.115333+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:07:35.115333+0000","last_undegraded":"2024-09-16T00:07:35.115333+0000","last_fullsized":"2024-09-16T00:07:35.115333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114803+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:07:35.114803+0000","last_peered":"2024-09-16T00:07:35.114803+0000","last_clean":"2024-09-16T00:07:35.114803+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:07:35.114803+0000","last_undegraded":"2024-09-16T00:07:35.114803+0000","last_fullsized":"2024-09-16T00:07:35.114803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.621871+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:07:38.621871+0000","last_peered":"2024-09-16T00:07:38.621871+0000","last_clean":"2024-09-16T00:07:38.621871+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:07:38.621871+0000","last_undegraded":"2024-09-16T00:07:38.621871+0000","last_fullsized":"2024-09-16T00:07:38.621871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114993+0000","last_change":"2024-09-16T00:07:33.637979+0000","last_active":"2024-09-16T00:07:35.114993+0000","last_peered":"2024-09-16T00:07:35.114993+0000","last_clean":"2024-09-16T00:07:35.114993+0000","last_became_active":"2024-09-16T00:07:33.637713+0000","last_became_peered":"2024-09-16T00:07:33.637713+0000","last_unstale":"2024-09-16T00:07:35.114993+0000","last_undegraded":"2024-09-16T00:07:35.114993+0000","last_fullsized":"2024-09-16T00:07:35.114993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615387+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:07:34.615387+0000","last_peered":"2024-09-16T00:07:34.615387+0000","last_clean":"2024-09-16T00:07:34.615387+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:07:34.615387+0000","last_undegraded":"2024-09-16T00:07:34.615387+0000","last_fullsized":"2024-09-16T00:07:34.615387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.621868+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:07:38.621868+0000","last_peered":"2024-09-16T00:07:38.621868+0000","last_clean":"2024-09-16T00:07:38.621868+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:07:38.621868+0000","last_undegraded":"2024-09-16T00:07:38.621868+0000","last_fullsized":"2024-09-16T00:07:38.621868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615266+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:07:34.615266+0000","last_peered":"2024-09-16T00:07:34.615266+0000","last_clean":"2024-09-16T00:07:34.615266+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:07:34.615266+0000","last_undegraded":"2024-09-16T00:07:34.615266+0000","last_fullsized":"2024-09-16T00:07:34.615266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.624540+0000","last_change":"2024-09-16T00:07:32.632481+0000","last_active":"2024-09-16T00:07:38.624540+0000","last_peered":"2024-09-16T00:07:38.624540+0000","last_clean":"2024-09-16T00:07:38.624540+0000","last_became_active":"2024-09-16T00:07:32.632152+0000","last_became_peered":"2024-09-16T00:07:32.632152+0000","last_unstale":"2024-09-16T00:07:38.624540+0000","last_undegraded":"2024-09-16T00:07:38.624540+0000","last_fullsized":"2024-09-16T00:07:38.624540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114783+0000","last_change":"2024-09-16T00:07:33.629777+0000","last_active":"2024-09-16T00:07:35.114783+0000","last_peered":"2024-09-16T00:07:35.114783+0000","last_clean":"2024-09-16T00:07:35.114783+0000","last_became_active":"2024-09-16T00:07:33.627917+0000","last_became_peered":"2024-09-16T00:07:33.627917+0000","last_unstale":"2024-09-16T00:07:35.114783+0000","last_undegraded":"2024-09-16T00:07:35.114783+0000","last_fullsized":"2024-09-16T00:07:35.114783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:38.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.059155+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:07:36.059155+0000","last_peered":"2024-09-16T00:07:36.059155+0000","last_clean":"2024-09-16T00:07:36.059155+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:07:36.059155+0000","last_undegraded":"2024-09-16T00:07:36.059155+0000","last_fullsized":"2024-09-16T00:07:36.059155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114899+0000","last_change":"2024-09-16T00:07:33.630746+0000","last_active":"2024-09-16T00:07:35.114899+0000","last_peered":"2024-09-16T00:07:35.114899+0000","last_clean":"2024-09-16T00:07:35.114899+0000","last_became_active":"2024-09-16T00:07:33.630480+0000","last_became_peered":"2024-09-16T00:07:33.630480+0000","last_unstale":"2024-09-16T00:07:35.114899+0000","last_undegraded":"2024-09-16T00:07:35.114899+0000","last_fullsized":"2024-09-16T00:07:35.114899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:50.931998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.622293+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:07:38.622293+0000","last_peered":"2024-09-16T00:07:38.622293+0000","last_clean":"2024-09-16T00:07:38.622293+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:07:38.622293+0000","last_undegraded":"2024-09-16T00:07:38.622293+0000","last_fullsized":"2024-09-16T00:07:38.622293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":668946,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":78122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":78156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":669002,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:07:46.132 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:07:46.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.265+0000 7f1e1dc77640 1 Processor -- start 2024-09-16T00:07:46.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.265+0000 7f1e1dc77640 1 -- start start 2024-09-16T00:07:46.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.266+0000 7f1e1dc77640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1e180c7cb0 0x7f1e18154580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.266+0000 7f1e1dc77640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1e1806f150 0x7f1e18154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.266+0000 7f1e1dc77640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e180c9890 con 0x7f1e1806ed70 2024-09-16T00:07:46.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.266+0000 7f1e1dc77640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1e1806c540 con 0x7f1e1806f150 2024-09-16T00:07:46.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.266+0000 7f1e1dc77640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f1e1806c820 con 0x7f1e180c7cb0 2024-09-16T00:07:46.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.266+0000 7f1e17fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1e1806f150 0x7f1e18154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.266+0000 7f1e17fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1e1806f150 0x7f1e18154ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40722/0 (socket says 172.21.15.139:40722) 2024-09-16T00:07:46.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.266+0000 7f1e17fff640 1 -- 172.21.15.139:0/1005693266 learned_addr learned my addr 172.21.15.139:0/1005693266 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:46.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.267+0000 7f1e16ffd640 1 --2- 172.21.15.139:0/1005693266 >> v2:172.21.15.164:3300/0 conn(0x7f1e180c7cb0 0x7f1e18154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.267+0000 7f1e17fff640 1 -- 172.21.15.139:0/1005693266 >> v2:172.21.15.164:3300/0 conn(0x7f1e180c7cb0 msgr2=0x7f1e18154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:46.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.267+0000 7f1e17fff640 1 --2- 172.21.15.139:0/1005693266 >> v2:172.21.15.164:3300/0 conn(0x7f1e180c7cb0 0x7f1e18154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.267+0000 7f1e17fff640 1 -- 172.21.15.139:0/1005693266 >> v1:172.21.15.164:6789/0 conn(0x7f1e1806ed70 legacy=0x7f1e1815a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:46.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.267+0000 7f1e17fff640 1 -- 172.21.15.139:0/1005693266 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1806cb00 con 0x7f1e1806f150 2024-09-16T00:07:46.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.267+0000 7f1e17fff640 1 --2- 172.21.15.139:0/1005693266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e1806f150 0x7f1e18154ac0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009e50 tx=0x7f1e1002a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64ea1f40f9e086f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:46.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.267+0000 7f1e167fc640 1 -- 172.21.15.139:0/1005693266 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10036070 con 0x7f1e1806f150 2024-09-16T00:07:46.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.268+0000 7f1e167fc640 1 -- 172.21.15.139:0/1005693266 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e10031600 con 0x7f1e1806f150 2024-09-16T00:07:46.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.268+0000 7f1e167fc640 1 -- 172.21.15.139:0/1005693266 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e100319d0 con 0x7f1e1806f150 2024-09-16T00:07:46.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.268+0000 7f1e1dc77640 1 -- 172.21.15.139:0/1005693266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e1806f150 msgr2=0x7f1e18154ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.268+0000 7f1e1dc77640 1 --2- 172.21.15.139:0/1005693266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e1806f150 0x7f1e18154ac0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009e50 tx=0x7f1e1002a7f0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.268+0000 7f1e1dc77640 1 -- 172.21.15.139:0/1005693266 shutdown_connections 2024-09-16T00:07:46.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.268+0000 7f1e1dc77640 1 --2- 172.21.15.139:0/1005693266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e1806f150 0x7f1e18154ac0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.268+0000 7f1e1dc77640 1 --2- 172.21.15.139:0/1005693266 >> v2:172.21.15.164:3300/0 conn(0x7f1e180c7cb0 0x7f1e18154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.268+0000 7f1e1dc77640 1 -- 172.21.15.139:0/1005693266 >> 172.21.15.139:0/1005693266 conn(0x7f1e18067950 msgr2=0x7f1e18067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:46.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.268+0000 7f1e1dc77640 1 -- 172.21.15.139:0/1005693266 shutdown_connections 2024-09-16T00:07:46.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.268+0000 7f1e1dc77640 1 -- 172.21.15.139:0/1005693266 wait complete. 2024-09-16T00:07:46.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.269+0000 7f1e1dc77640 1 Processor -- start 2024-09-16T00:07:46.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.269+0000 7f1e1dc77640 1 -- start start 2024-09-16T00:07:46.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.270+0000 7f1e1dc77640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1806ed70 0x7f1e1814d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.270+0000 7f1e1dc77640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e1806f150 0x7f1e181e0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.270+0000 7f1e1dc77640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e180c7cb0 0x7f1e180c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.270+0000 7f1e1dc77640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1815b060 con 0x7f1e1806f150 2024-09-16T00:07:46.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.270+0000 7f1e1dc77640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1e180c9b70 con 0x7f1e1806ed70 2024-09-16T00:07:46.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.270+0000 7f1e1dc77640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1e1815b1e0 con 0x7f1e180c7cb0 2024-09-16T00:07:46.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.270+0000 7f1e16ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e1806f150 0x7f1e181e0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.270+0000 7f1e16ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e1806f150 0x7f1e181e0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40726/0 (socket says 172.21.15.139:40726) 2024-09-16T00:07:46.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.270+0000 7f1e16ffd640 1 -- 172.21.15.139:0/1566056831 learned_addr learned my addr 172.21.15.139:0/1566056831 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:46.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.270+0000 7f1e177fe640 1 --2- 172.21.15.139:0/1566056831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1806ed70 0x7f1e1814d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.270+0000 7f1e17fff640 1 --2- 172.21.15.139:0/1566056831 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e180c7cb0 0x7f1e180c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.271+0000 7f1e16ffd640 1 -- 172.21.15.139:0/1566056831 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e180c7cb0 msgr2=0x7f1e180c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.271+0000 7f1e16ffd640 1 --2- 172.21.15.139:0/1566056831 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e180c7cb0 0x7f1e180c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.271+0000 7f1e16ffd640 1 -- 172.21.15.139:0/1566056831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1806ed70 msgr2=0x7f1e1814d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.271+0000 7f1e16ffd640 1 --2- 172.21.15.139:0/1566056831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1806ed70 0x7f1e1814d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.271+0000 7f1e16ffd640 1 -- 172.21.15.139:0/1566056831 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e181e3840 con 0x7f1e1806f150 2024-09-16T00:07:46.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.271+0000 7f1e17fff640 1 --2- 172.21.15.139:0/1566056831 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e180c7cb0 0x7f1e180c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:46.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.271+0000 7f1e16ffd640 1 --2- 172.21.15.139:0/1566056831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e1806f150 0x7f1e181e0860 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00d620 tx=0x7f1e0c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:46.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.271+0000 7f1dfffff640 1 -- 172.21.15.139:0/1566056831 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c00b620 con 0x7f1e1806f150 2024-09-16T00:07:46.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.271+0000 7f1dfffff640 1 -- 172.21.15.139:0/1566056831 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e0c019070 con 0x7f1e1806f150 2024-09-16T00:07:46.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.271+0000 7f1dfffff640 1 -- 172.21.15.139:0/1566056831 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c00ba00 con 0x7f1e1806f150 2024-09-16T00:07:46.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.271+0000 7f1e1dc77640 1 -- 172.21.15.139:0/1566056831 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e180c51f0 con 0x7f1e1806f150 2024-09-16T00:07:46.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.271+0000 7f1e1dc77640 1 -- 172.21.15.139:0/1566056831 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e180c57b0 con 0x7f1e1806f150 2024-09-16T00:07:46.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.272+0000 7f1e1dc77640 1 -- 172.21.15.139:0/1566056831 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e1806bb20 con 0x7f1e1806f150 2024-09-16T00:07:46.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.272+0000 7f1dfffff640 1 -- 172.21.15.139:0/1566056831 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1e0c00bba0 con 0x7f1e1806f150 2024-09-16T00:07:46.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.273+0000 7f1dfffff640 1 --2- 172.21.15.139:0/1566056831 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1e0406c2e0 0x7f1e0406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.273+0000 7f1e177fe640 1 --2- 172.21.15.139:0/1566056831 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1e0406c2e0 0x7f1e0406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.273+0000 7f1dfffff640 1 -- 172.21.15.139:0/1566056831 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1e0c08b6c0 con 0x7f1e1806f150 2024-09-16T00:07:46.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.273+0000 7f1e177fe640 1 --2- 172.21.15.139:0/1566056831 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1e0406c2e0 0x7f1e0406e7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e08001780 tx=0x7f1e08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:46.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.277+0000 7f1dfffff640 1 -- 172.21.15.139:0/1566056831 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1e0c059ec0 con 0x7f1e1806f150 2024-09-16T00:07:46.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.437+0000 7f1e1dc77640 1 -- 172.21.15.139:0/1566056831 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e180c45f0 con 0x7f1e0406c2e0 2024-09-16T00:07:46.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.442+0000 7f1dfffff640 1 -- 172.21.15.139:0/1566056831 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251913 (secure 0 0 0) 0x7f1e180c45f0 con 0x7f1e0406c2e0 2024-09-16T00:07:46.446 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:46.447 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:07:46.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.446+0000 7f1e1dc77640 1 -- 172.21.15.139:0/1566056831 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1e0406c2e0 msgr2=0x7f1e0406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.446+0000 7f1e1dc77640 1 --2- 172.21.15.139:0/1566056831 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1e0406c2e0 0x7f1e0406e7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e08001780 tx=0x7f1e08009040 comp rx=0 tx=0).stop 2024-09-16T00:07:46.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.446+0000 7f1e1dc77640 1 -- 172.21.15.139:0/1566056831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e1806f150 msgr2=0x7f1e181e0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.446+0000 7f1e1dc77640 1 --2- 172.21.15.139:0/1566056831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e1806f150 0x7f1e181e0860 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00d620 tx=0x7f1e0c00dae0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.446+0000 7f1e1dc77640 1 -- 172.21.15.139:0/1566056831 shutdown_connections 2024-09-16T00:07:46.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.446+0000 7f1e1dc77640 1 --2- 172.21.15.139:0/1566056831 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1e0406c2e0 0x7f1e0406e7a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.446+0000 7f1e1dc77640 1 --2- 172.21.15.139:0/1566056831 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1e180c7cb0 0x7f1e180c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.446+0000 7f1e1dc77640 1 --2- 172.21.15.139:0/1566056831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e1806f150 0x7f1e181e0860 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.446+0000 7f1e1dc77640 1 --2- 172.21.15.139:0/1566056831 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1e1806ed70 0x7f1e1814d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.446+0000 7f1e1dc77640 1 -- 172.21.15.139:0/1566056831 >> 172.21.15.139:0/1566056831 conn(0x7f1e18067950 msgr2=0x7f1e181530c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:46.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.446+0000 7f1e1dc77640 1 -- 172.21.15.139:0/1566056831 shutdown_connections 2024-09-16T00:07:46.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.446+0000 7f1e1dc77640 1 -- 172.21.15.139:0/1566056831 wait complete. 2024-09-16T00:07:46.458 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-16T00:07:46.028091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218112,"kb_used_data":1920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756608,"statfs":{"total":773094113280,"available":772870766592,"internally_reserved":0,"allocated":1966080,"data_stored":1809524,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.416234"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115528+0000","last_change":"2024-09-16T00:07:33.635096+0000","last_active":"2024-09-16T00:07:35.115528+0000","last_peered":"2024-09-16T00:07:35.115528+0000","last_clean":"2024-09-16T00:07:35.115528+0000","last_became_active":"2024-09-16T00:07:33.634750+0000","last_became_peered":"2024-09-16T00:07:33.634750+0000","last_unstale":"2024-09-16T00:07:35.115528+0000","last_undegraded":"2024-09-16T00:07:35.115528+0000","last_fullsized":"2024-09-16T00:07:35.115528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115558+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:07:35.115558+0000","last_peered":"2024-09-16T00:07:35.115558+0000","last_clean":"2024-09-16T00:07:35.115558+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:07:35.115558+0000","last_undegraded":"2024-09-16T00:07:35.115558+0000","last_fullsized":"2024-09-16T00:07:35.115558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115392+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:07:35.115392+0000","last_peered":"2024-09-16T00:07:35.115392+0000","last_clean":"2024-09-16T00:07:35.115392+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:07:35.115392+0000","last_undegraded":"2024-09-16T00:07:35.115392+0000","last_fullsized":"2024-09-16T00:07:35.115392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615167+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:07:34.615167+0000","last_peered":"2024-09-16T00:07:34.615167+0000","last_clean":"2024-09-16T00:07:34.615167+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:07:34.615167+0000","last_undegraded":"2024-09-16T00:07:34.615167+0000","last_fullsized":"2024-09-16T00:07:34.615167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115724+0000","last_change":"2024-09-16T00:07:33.630029+0000","last_active":"2024-09-16T00:07:35.115724+0000","last_peered":"2024-09-16T00:07:35.115724+0000","last_clean":"2024-09-16T00:07:35.115724+0000","last_became_active":"2024-09-16T00:07:33.629712+0000","last_became_peered":"2024-09-16T00:07:33.629712+0000","last_unstale":"2024-09-16T00:07:35.115724+0000","last_undegraded":"2024-09-16T00:07:35.115724+0000","last_fullsized":"2024-09-16T00:07:35.115724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614973+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:07:34.614973+0000","last_peered":"2024-09-16T00:07:34.614973+0000","last_clean":"2024-09-16T00:07:34.614973+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:07:34.614973+0000","last_undegraded":"2024-09-16T00:07:34.614973+0000","last_fullsized":"2024-09-16T00:07:34.614973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115522+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:07:35.115522+0000","last_peered":"2024-09-16T00:07:35.115522+0000","last_clean":"2024-09-16T00:07:35.115522+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:07:35.115522+0000","last_undegraded":"2024-09-16T00:07:35.115522+0000","last_fullsized":"2024-09-16T00:07:35.115522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114492+0000","last_change":"2024-09-16T00:07:33.624708+0000","last_active":"2024-09-16T00:07:35.114492+0000","last_peered":"2024-09-16T00:07:35.114492+0000","last_clean":"2024-09-16T00:07:35.114492+0000","last_became_active":"2024-09-16T00:07:33.624458+0000","last_became_peered":"2024-09-16T00:07:33.624458+0000","last_unstale":"2024-09-16T00:07:35.114492+0000","last_undegraded":"2024-09-16T00:07:35.114492+0000","last_fullsized":"2024-09-16T00:07:35.114492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:48.052488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115751+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:07:35.115751+0000","last_peered":"2024-09-16T00:07:35.115751+0000","last_clean":"2024-09-16T00:07:35.115751+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:07:35.115751+0000","last_undegraded":"2024-09-16T00:07:35.115751+0000","last_fullsized":"2024-09-16T00:07:35.115751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115614+0000","last_change":"2024-09-16T00:07:33.635973+0000","last_active":"2024-09-16T00:07:35.115614+0000","last_peered":"2024-09-16T00:07:35.115614+0000","last_clean":"2024-09-16T00:07:35.115614+0000","last_became_active":"2024-09-16T00:07:33.635486+0000","last_became_peered":"2024-09-16T00:07:33.635486+0000","last_unstale":"2024-09-16T00:07:35.115614+0000","last_undegraded":"2024-09-16T00:07:35.115614+0000","last_fullsized":"2024-09-16T00:07:35.115614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115027+0000","last_change":"2024-09-16T00:07:33.636971+0000","last_active":"2024-09-16T00:07:35.115027+0000","last_peered":"2024-09-16T00:07:35.115027+0000","last_clean":"2024-09-16T00:07:35.115027+0000","last_became_active":"2024-09-16T00:07:33.635760+0000","last_became_peered":"2024-09-16T00:07:33.635760+0000","last_unstale":"2024-09-16T00:07:35.115027+0000","last_undegraded":"2024-09-16T00:07:35.115027+0000","last_fullsized":"2024-09-16T00:07:35.115027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614631+0000","last_change":"2024-09-16T00:07:33.632261+0000","last_active":"2024-09-16T00:07:34.614631+0000","last_peered":"2024-09-16T00:07:34.614631+0000","last_clean":"2024-09-16T00:07:34.614631+0000","last_became_active":"2024-09-16T00:07:33.631944+0000","last_became_peered":"2024-09-16T00:07:33.631944+0000","last_unstale":"2024-09-16T00:07:34.614631+0000","last_undegraded":"2024-09-16T00:07:34.614631+0000","last_fullsized":"2024-09-16T00:07:34.614631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:35.052417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114641+0000","last_change":"2024-09-16T00:07:33.628273+0000","last_active":"2024-09-16T00:07:35.114641+0000","last_peered":"2024-09-16T00:07:35.114641+0000","last_clean":"2024-09-16T00:07:35.114641+0000","last_became_active":"2024-09-16T00:07:33.627960+0000","last_became_peered":"2024-09-16T00:07:33.627960+0000","last_unstale":"2024-09-16T00:07:35.114641+0000","last_undegraded":"2024-09-16T00:07:35.114641+0000","last_fullsized":"2024-09-16T00:07:35.114641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:36.451688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114782+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:07:35.114782+0000","last_peered":"2024-09-16T00:07:35.114782+0000","last_clean":"2024-09-16T00:07:35.114782+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:07:35.114782+0000","last_undegraded":"2024-09-16T00:07:35.114782+0000","last_fullsized":"2024-09-16T00:07:35.114782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114664+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:07:35.114664+0000","last_peered":"2024-09-16T00:07:35.114664+0000","last_clean":"2024-09-16T00:07:35.114664+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:07:35.114664+0000","last_undegraded":"2024-09-16T00:07:35.114664+0000","last_fullsized":"2024-09-16T00:07:35.114664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614471+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:07:34.614471+0000","last_peered":"2024-09-16T00:07:34.614471+0000","last_clean":"2024-09-16T00:07:34.614471+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:07:34.614471+0000","last_undegraded":"2024-09-16T00:07:34.614471+0000","last_fullsized":"2024-09-16T00:07:34.614471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615697+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:07:34.615697+0000","last_peered":"2024-09-16T00:07:34.615697+0000","last_clean":"2024-09-16T00:07:34.615697+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:07:34.615697+0000","last_undegraded":"2024-09-16T00:07:34.615697+0000","last_fullsized":"2024-09-16T00:07:34.615697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613100+0000","last_change":"2024-09-16T00:07:33.630797+0000","last_active":"2024-09-16T00:07:34.613100+0000","last_peered":"2024-09-16T00:07:34.613100+0000","last_clean":"2024-09-16T00:07:34.613100+0000","last_became_active":"2024-09-16T00:07:33.630491+0000","last_became_peered":"2024-09-16T00:07:33.630491+0000","last_unstale":"2024-09-16T00:07:34.613100+0000","last_undegraded":"2024-09-16T00:07:34.613100+0000","last_fullsized":"2024-09-16T00:07:34.613100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:20.097626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115840+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:07:35.115840+0000","last_peered":"2024-09-16T00:07:35.115840+0000","last_clean":"2024-09-16T00:07:35.115840+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:07:35.115840+0000","last_undegraded":"2024-09-16T00:07:35.115840+0000","last_fullsized":"2024-09-16T00:07:35.115840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114960+0000","last_change":"2024-09-16T00:07:33.638554+0000","last_active":"2024-09-16T00:07:35.114960+0000","last_peered":"2024-09-16T00:07:35.114960+0000","last_clean":"2024-09-16T00:07:35.114960+0000","last_became_active":"2024-09-16T00:07:33.638263+0000","last_became_peered":"2024-09-16T00:07:33.638263+0000","last_unstale":"2024-09-16T00:07:35.114960+0000","last_undegraded":"2024-09-16T00:07:35.114960+0000","last_fullsized":"2024-09-16T00:07:35.114960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115286+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:07:35.115286+0000","last_peered":"2024-09-16T00:07:35.115286+0000","last_clean":"2024-09-16T00:07:35.115286+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:07:35.115286+0000","last_undegraded":"2024-09-16T00:07:35.115286+0000","last_fullsized":"2024-09-16T00:07:35.115286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615450+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:07:34.615450+0000","last_peered":"2024-09-16T00:07:34.615450+0000","last_clean":"2024-09-16T00:07:34.615450+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:07:34.615450+0000","last_undegraded":"2024-09-16T00:07:34.615450+0000","last_fullsized":"2024-09-16T00:07:34.615450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115015+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:07:35.115015+0000","last_peered":"2024-09-16T00:07:35.115015+0000","last_clean":"2024-09-16T00:07:35.115015+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:07:35.115015+0000","last_undegraded":"2024-09-16T00:07:35.115015+0000","last_fullsized":"2024-09-16T00:07:35.115015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614616+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:07:34.614616+0000","last_peered":"2024-09-16T00:07:34.614616+0000","last_clean":"2024-09-16T00:07:34.614616+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:07:34.614616+0000","last_undegraded":"2024-09-16T00:07:34.614616+0000","last_fullsized":"2024-09-16T00:07:34.614616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114694+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:07:35.114694+0000","last_peered":"2024-09-16T00:07:35.114694+0000","last_clean":"2024-09-16T00:07:35.114694+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:07:35.114694+0000","last_undegraded":"2024-09-16T00:07:35.114694+0000","last_fullsized":"2024-09-16T00:07:35.114694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115306+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:07:35.115306+0000","last_peered":"2024-09-16T00:07:35.115306+0000","last_clean":"2024-09-16T00:07:35.115306+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:07:35.115306+0000","last_undegraded":"2024-09-16T00:07:35.115306+0000","last_fullsized":"2024-09-16T00:07:35.115306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115543+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:07:35.115543+0000","last_peered":"2024-09-16T00:07:35.115543+0000","last_clean":"2024-09-16T00:07:35.115543+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:07:35.115543+0000","last_undegraded":"2024-09-16T00:07:35.115543+0000","last_fullsized":"2024-09-16T00:07:35.115543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114985+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:07:35.114985+0000","last_peered":"2024-09-16T00:07:35.114985+0000","last_clean":"2024-09-16T00:07:35.114985+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:07:35.114985+0000","last_undegraded":"2024-09-16T00:07:35.114985+0000","last_fullsized":"2024-09-16T00:07:35.114985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115448+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:07:35.115448+0000","last_peered":"2024-09-16T00:07:35.115448+0000","last_clean":"2024-09-16T00:07:35.115448+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:07:35.115448+0000","last_undegraded":"2024-09-16T00:07:35.115448+0000","last_fullsized":"2024-09-16T00:07:35.115448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615338+0000","last_change":"2024-09-16T00:07:33.627789+0000","last_active":"2024-09-16T00:07:34.615338+0000","last_peered":"2024-09-16T00:07:34.615338+0000","last_clean":"2024-09-16T00:07:34.615338+0000","last_became_active":"2024-09-16T00:07:33.627500+0000","last_became_peered":"2024-09-16T00:07:33.627500+0000","last_unstale":"2024-09-16T00:07:34.615338+0000","last_undegraded":"2024-09-16T00:07:34.615338+0000","last_fullsized":"2024-09-16T00:07:34.615338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613050+0000","last_change":"2024-09-16T00:07:33.621107+0000","last_active":"2024-09-16T00:07:34.613050+0000","last_peered":"2024-09-16T00:07:34.613050+0000","last_clean":"2024-09-16T00:07:34.613050+0000","last_became_active":"2024-09-16T00:07:33.620858+0000","last_became_peered":"2024-09-16T00:07:33.620858+0000","last_unstale":"2024-09-16T00:07:34.613050+0000","last_undegraded":"2024-09-16T00:07:34.613050+0000","last_fullsized":"2024-09-16T00:07:34.613050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:30.176079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115031+0000","last_change":"2024-09-16T00:07:33.633578+0000","last_active":"2024-09-16T00:07:35.115031+0000","last_peered":"2024-09-16T00:07:35.115031+0000","last_clean":"2024-09-16T00:07:35.115031+0000","last_became_active":"2024-09-16T00:07:33.633310+0000","last_became_peered":"2024-09-16T00:07:33.633310+0000","last_unstale":"2024-09-16T00:07:35.115031+0000","last_undegraded":"2024-09-16T00:07:35.115031+0000","last_fullsized":"2024-09-16T00:07:35.115031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614556+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:07:34.614556+0000","last_peered":"2024-09-16T00:07:34.614556+0000","last_clean":"2024-09-16T00:07:34.614556+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:07:34.614556+0000","last_undegraded":"2024-09-16T00:07:34.614556+0000","last_fullsized":"2024-09-16T00:07:34.614556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.059131+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:07:36.059131+0000","last_peered":"2024-09-16T00:07:36.059131+0000","last_clean":"2024-09-16T00:07:36.059131+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:07:36.059131+0000","last_undegraded":"2024-09-16T00:07:36.059131+0000","last_fullsized":"2024-09-16T00:07:36.059131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115509+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:07:35.115509+0000","last_peered":"2024-09-16T00:07:35.115509+0000","last_clean":"2024-09-16T00:07:35.115509+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:07:35.115509+0000","last_undegraded":"2024-09-16T00:07:35.115509+0000","last_fullsized":"2024-09-16T00:07:35.115509+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114845+0000","last_change":"2024-09-16T00:07:32.626800+0000","last_active":"2024-09-16T00:07:35.114845+0000","last_peered":"2024-09-16T00:07:35.114845+0000","last_clean":"2024-09-16T00:07:35.114845+0000","last_became_active":"2024-09-16T00:07:32.626511+0000","last_became_peered":"2024-09-16T00:07:32.626511+0000","last_unstale":"2024-09-16T00:07:35.114845+0000","last_undegraded":"2024-09-16T00:07:35.114845+0000","last_fullsized":"2024-09-16T00:07:35.114845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115156+0000","last_change":"2024-09-16T00:07:32.629539+0000","last_active":"2024-09-16T00:07:35.115156+0000","last_peered":"2024-09-16T00:07:35.115156+0000","last_clean":"2024-09-16T00:07:35.115156+0000","last_became_active":"2024-09-16T00:07:32.629186+0000","last_became_peered":"2024-09-16T00:07:32.629186+0000","last_unstale":"2024-09-16T00:07:35.115156+0000","last_undegraded":"2024-09-16T00:07:35.115156+0000","last_fullsized":"2024-09-16T00:07:35.115156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:59:32.860208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114535+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:07:35.114535+0000","last_peered":"2024-09-16T00:07:35.114535+0000","last_clean":"2024-09-16T00:07:35.114535+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:07:35.114535+0000","last_undegraded":"2024-09-16T00:07:35.114535+0000","last_fullsized":"2024-09-16T00:07:35.114535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614668+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:07:34.614668+0000","last_peered":"2024-09-16T00:07:34.614668+0000","last_clean":"2024-09-16T00:07:34.614668+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:07:34.614668+0000","last_undegraded":"2024-09-16T00:07:34.614668+0000","last_fullsized":"2024-09-16T00:07:34.614668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114527+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:07:35.114527+0000","last_peered":"2024-09-16T00:07:35.114527+0000","last_clean":"2024-09-16T00:07:35.114527+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:07:35.114527+0000","last_undegraded":"2024-09-16T00:07:35.114527+0000","last_fullsized":"2024-09-16T00:07:35.114527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115138+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:07:35.115138+0000","last_peered":"2024-09-16T00:07:35.115138+0000","last_clean":"2024-09-16T00:07:35.115138+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:07:35.115138+0000","last_undegraded":"2024-09-16T00:07:35.115138+0000","last_fullsized":"2024-09-16T00:07:35.115138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615408+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:07:34.615408+0000","last_peered":"2024-09-16T00:07:34.615408+0000","last_clean":"2024-09-16T00:07:34.615408+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:07:34.615408+0000","last_undegraded":"2024-09-16T00:07:34.615408+0000","last_fullsized":"2024-09-16T00:07:34.615408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615356+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:07:34.615356+0000","last_peered":"2024-09-16T00:07:34.615356+0000","last_clean":"2024-09-16T00:07:34.615356+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:07:34.615356+0000","last_undegraded":"2024-09-16T00:07:34.615356+0000","last_fullsized":"2024-09-16T00:07:34.615356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115277+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:07:35.115277+0000","last_peered":"2024-09-16T00:07:35.115277+0000","last_clean":"2024-09-16T00:07:35.115277+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:07:35.115277+0000","last_undegraded":"2024-09-16T00:07:35.115277+0000","last_fullsized":"2024-09-16T00:07:35.115277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.619745+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:07:38.619745+0000","last_peered":"2024-09-16T00:07:38.619745+0000","last_clean":"2024-09-16T00:07:38.619745+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:07:38.619745+0000","last_undegraded":"2024-09-16T00:07:38.619745+0000","last_fullsized":"2024-09-16T00:07:38.619745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614648+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:07:34.614648+0000","last_peered":"2024-09-16T00:07:34.614648+0000","last_clean":"2024-09-16T00:07:34.614648+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:07:34.614648+0000","last_undegraded":"2024-09-16T00:07:34.614648+0000","last_fullsized":"2024-09-16T00:07:34.614648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.622263+0000","last_change":"2024-09-16T00:07:32.629132+0000","last_active":"2024-09-16T00:07:38.622263+0000","last_peered":"2024-09-16T00:07:38.622263+0000","last_clean":"2024-09-16T00:07:38.622263+0000","last_became_active":"2024-09-16T00:07:32.628803+0000","last_became_peered":"2024-09-16T00:07:32.628803+0000","last_unstale":"2024-09-16T00:07:38.622263+0000","last_undegraded":"2024-09-16T00:07:38.622263+0000","last_fullsized":"2024-09-16T00:07:38.622263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115230+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:07:35.115230+0000","last_peered":"2024-09-16T00:07:35.115230+0000","last_clean":"2024-09-16T00:07:35.115230+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:07:35.115230+0000","last_undegraded":"2024-09-16T00:07:35.115230+0000","last_fullsized":"2024-09-16T00:07:35.115230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620127+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:07:38.620127+0000","last_peered":"2024-09-16T00:07:38.620127+0000","last_clean":"2024-09-16T00:07:38.620127+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:07:38.620127+0000","last_undegraded":"2024-09-16T00:07:38.620127+0000","last_fullsized":"2024-09-16T00:07:38.620127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115121+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:07:35.115121+0000","last_peered":"2024-09-16T00:07:35.115121+0000","last_clean":"2024-09-16T00:07:35.115121+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:07:35.115121+0000","last_undegraded":"2024-09-16T00:07:35.115121+0000","last_fullsized":"2024-09-16T00:07:35.115121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615462+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:07:34.615462+0000","last_peered":"2024-09-16T00:07:34.615462+0000","last_clean":"2024-09-16T00:07:34.615462+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:07:34.615462+0000","last_undegraded":"2024-09-16T00:07:34.615462+0000","last_fullsized":"2024-09-16T00:07:34.615462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115766+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:07:35.115766+0000","last_peered":"2024-09-16T00:07:35.115766+0000","last_clean":"2024-09-16T00:07:35.115766+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:07:35.115766+0000","last_undegraded":"2024-09-16T00:07:35.115766+0000","last_fullsized":"2024-09-16T00:07:35.115766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114802+0000","last_change":"2024-09-16T00:07:32.626856+0000","last_active":"2024-09-16T00:07:35.114802+0000","last_peered":"2024-09-16T00:07:35.114802+0000","last_clean":"2024-09-16T00:07:35.114802+0000","last_became_active":"2024-09-16T00:07:32.626532+0000","last_became_peered":"2024-09-16T00:07:32.626532+0000","last_unstale":"2024-09-16T00:07:35.114802+0000","last_undegraded":"2024-09-16T00:07:35.114802+0000","last_fullsized":"2024-09-16T00:07:35.114802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114886+0000","last_change":"2024-09-16T00:07:33.632870+0000","last_active":"2024-09-16T00:07:35.114886+0000","last_peered":"2024-09-16T00:07:35.114886+0000","last_clean":"2024-09-16T00:07:35.114886+0000","last_became_active":"2024-09-16T00:07:33.632608+0000","last_became_peered":"2024-09-16T00:07:33.632608+0000","last_unstale":"2024-09-16T00:07:35.114886+0000","last_undegraded":"2024-09-16T00:07:35.114886+0000","last_fullsized":"2024-09-16T00:07:35.114886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.621550+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:07:38.621550+0000","last_peered":"2024-09-16T00:07:38.621550+0000","last_clean":"2024-09-16T00:07:38.621550+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:07:38.621550+0000","last_undegraded":"2024-09-16T00:07:38.621550+0000","last_fullsized":"2024-09-16T00:07:38.621550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115409+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:07:35.115409+0000","last_peered":"2024-09-16T00:07:35.115409+0000","last_clean":"2024-09-16T00:07:35.115409+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:07:35.115409+0000","last_undegraded":"2024-09-16T00:07:35.115409+0000","last_fullsized":"2024-09-16T00:07:35.115409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614761+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:07:34.614761+0000","last_peered":"2024-09-16T00:07:34.614761+0000","last_clean":"2024-09-16T00:07:34.614761+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:07:34.614761+0000","last_undegraded":"2024-09-16T00:07:34.614761+0000","last_fullsized":"2024-09-16T00:07:34.614761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115480+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:07:35.115480+0000","last_peered":"2024-09-16T00:07:35.115480+0000","last_clean":"2024-09-16T00:07:35.115480+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:07:35.115480+0000","last_undegraded":"2024-09-16T00:07:35.115480+0000","last_fullsized":"2024-09-16T00:07:35.115480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620942+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:07:38.620942+0000","last_peered":"2024-09-16T00:07:38.620942+0000","last_clean":"2024-09-16T00:07:38.620942+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:07:38.620942+0000","last_undegraded":"2024-09-16T00:07:38.620942+0000","last_fullsized":"2024-09-16T00:07:38.620942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115888+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:07:35.115888+0000","last_peered":"2024-09-16T00:07:35.115888+0000","last_clean":"2024-09-16T00:07:35.115888+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:07:35.115888+0000","last_undegraded":"2024-09-16T00:07:35.115888+0000","last_fullsized":"2024-09-16T00:07:35.115888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.048699+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:07:36.048699+0000","last_peered":"2024-09-16T00:07:36.048699+0000","last_clean":"2024-09-16T00:07:36.048699+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:07:36.048699+0000","last_undegraded":"2024-09-16T00:07:36.048699+0000","last_fullsized":"2024-09-16T00:07:36.048699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615597+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:07:34.615597+0000","last_peered":"2024-09-16T00:07:34.615597+0000","last_clean":"2024-09-16T00:07:34.615597+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:07:34.615597+0000","last_undegraded":"2024-09-16T00:07:34.615597+0000","last_fullsized":"2024-09-16T00:07:34.615597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613057+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:07:34.613057+0000","last_peered":"2024-09-16T00:07:34.613057+0000","last_clean":"2024-09-16T00:07:34.613057+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:07:34.613057+0000","last_undegraded":"2024-09-16T00:07:34.613057+0000","last_fullsized":"2024-09-16T00:07:34.613057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115577+0000","last_change":"2024-09-16T00:07:33.638312+0000","last_active":"2024-09-16T00:07:35.115577+0000","last_peered":"2024-09-16T00:07:35.115577+0000","last_clean":"2024-09-16T00:07:35.115577+0000","last_became_active":"2024-09-16T00:07:33.638049+0000","last_became_peered":"2024-09-16T00:07:33.638049+0000","last_unstale":"2024-09-16T00:07:35.115577+0000","last_undegraded":"2024-09-16T00:07:35.115577+0000","last_fullsized":"2024-09-16T00:07:35.115577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115169+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:07:35.115169+0000","last_peered":"2024-09-16T00:07:35.115169+0000","last_clean":"2024-09-16T00:07:35.115169+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:07:35.115169+0000","last_undegraded":"2024-09-16T00:07:35.115169+0000","last_fullsized":"2024-09-16T00:07:35.115169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614834+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:07:34.614834+0000","last_peered":"2024-09-16T00:07:34.614834+0000","last_clean":"2024-09-16T00:07:34.614834+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:07:34.614834+0000","last_undegraded":"2024-09-16T00:07:34.614834+0000","last_fullsized":"2024-09-16T00:07:34.614834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114892+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:07:35.114892+0000","last_peered":"2024-09-16T00:07:35.114892+0000","last_clean":"2024-09-16T00:07:35.114892+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:07:35.114892+0000","last_undegraded":"2024-09-16T00:07:35.114892+0000","last_fullsized":"2024-09-16T00:07:35.114892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115065+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:07:35.115065+0000","last_peered":"2024-09-16T00:07:35.115065+0000","last_clean":"2024-09-16T00:07:35.115065+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:07:35.115065+0000","last_undegraded":"2024-09-16T00:07:35.115065+0000","last_fullsized":"2024-09-16T00:07:35.115065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115288+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:07:35.115288+0000","last_peered":"2024-09-16T00:07:35.115288+0000","last_clean":"2024-09-16T00:07:35.115288+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:07:35.115288+0000","last_undegraded":"2024-09-16T00:07:35.115288+0000","last_fullsized":"2024-09-16T00:07:35.115288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.612974+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:07:34.612974+0000","last_peered":"2024-09-16T00:07:34.612974+0000","last_clean":"2024-09-16T00:07:34.612974+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:07:34.612974+0000","last_undegraded":"2024-09-16T00:07:34.612974+0000","last_fullsized":"2024-09-16T00:07:34.612974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114905+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:07:35.114905+0000","last_peered":"2024-09-16T00:07:35.114905+0000","last_clean":"2024-09-16T00:07:35.114905+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:07:35.114905+0000","last_undegraded":"2024-09-16T00:07:35.114905+0000","last_fullsized":"2024-09-16T00:07:35.114905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615621+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:07:34.615621+0000","last_peered":"2024-09-16T00:07:34.615621+0000","last_clean":"2024-09-16T00:07:34.615621+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:07:34.615621+0000","last_undegraded":"2024-09-16T00:07:34.615621+0000","last_fullsized":"2024-09-16T00:07:34.615621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615008+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:07:34.615008+0000","last_peered":"2024-09-16T00:07:34.615008+0000","last_clean":"2024-09-16T00:07:34.615008+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:07:34.615008+0000","last_undegraded":"2024-09-16T00:07:34.615008+0000","last_fullsized":"2024-09-16T00:07:34.615008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115398+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:07:35.115398+0000","last_peered":"2024-09-16T00:07:35.115398+0000","last_clean":"2024-09-16T00:07:35.115398+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:07:35.115398+0000","last_undegraded":"2024-09-16T00:07:35.115398+0000","last_fullsized":"2024-09-16T00:07:35.115398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615631+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:07:34.615631+0000","last_peered":"2024-09-16T00:07:34.615631+0000","last_clean":"2024-09-16T00:07:34.615631+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:07:34.615631+0000","last_undegraded":"2024-09-16T00:07:34.615631+0000","last_fullsized":"2024-09-16T00:07:34.615631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115814+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:07:35.115814+0000","last_peered":"2024-09-16T00:07:35.115814+0000","last_clean":"2024-09-16T00:07:35.115814+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:07:35.115814+0000","last_undegraded":"2024-09-16T00:07:35.115814+0000","last_fullsized":"2024-09-16T00:07:35.115814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115305+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:07:35.115305+0000","last_peered":"2024-09-16T00:07:35.115305+0000","last_clean":"2024-09-16T00:07:35.115305+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:07:35.115305+0000","last_undegraded":"2024-09-16T00:07:35.115305+0000","last_fullsized":"2024-09-16T00:07:35.115305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115644+0000","last_change":"2024-09-16T00:07:32.627236+0000","last_active":"2024-09-16T00:07:35.115644+0000","last_peered":"2024-09-16T00:07:35.115644+0000","last_clean":"2024-09-16T00:07:35.115644+0000","last_became_active":"2024-09-16T00:07:32.626743+0000","last_became_peered":"2024-09-16T00:07:32.626743+0000","last_unstale":"2024-09-16T00:07:35.115644+0000","last_undegraded":"2024-09-16T00:07:35.115644+0000","last_fullsized":"2024-09-16T00:07:35.115644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114835+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:07:35.114835+0000","last_peered":"2024-09-16T00:07:35.114835+0000","last_clean":"2024-09-16T00:07:35.114835+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:07:35.114835+0000","last_undegraded":"2024-09-16T00:07:35.114835+0000","last_fullsized":"2024-09-16T00:07:35.114835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620513+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:07:38.620513+0000","last_peered":"2024-09-16T00:07:38.620513+0000","last_clean":"2024-09-16T00:07:38.620513+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:07:38.620513+0000","last_undegraded":"2024-09-16T00:07:38.620513+0000","last_fullsized":"2024-09-16T00:07:38.620513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114804+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:07:35.114804+0000","last_peered":"2024-09-16T00:07:35.114804+0000","last_clean":"2024-09-16T00:07:35.114804+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:07:35.114804+0000","last_undegraded":"2024-09-16T00:07:35.114804+0000","last_fullsized":"2024-09-16T00:07:35.114804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114947+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:07:35.114947+0000","last_peered":"2024-09-16T00:07:35.114947+0000","last_clean":"2024-09-16T00:07:35.114947+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:07:35.114947+0000","last_undegraded":"2024-09-16T00:07:35.114947+0000","last_fullsized":"2024-09-16T00:07:35.114947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614501+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:07:34.614501+0000","last_peered":"2024-09-16T00:07:34.614501+0000","last_clean":"2024-09-16T00:07:34.614501+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:07:34.614501+0000","last_undegraded":"2024-09-16T00:07:34.614501+0000","last_fullsized":"2024-09-16T00:07:34.614501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114765+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:07:35.114765+0000","last_peered":"2024-09-16T00:07:35.114765+0000","last_clean":"2024-09-16T00:07:35.114765+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:07:35.114765+0000","last_undegraded":"2024-09-16T00:07:35.114765+0000","last_fullsized":"2024-09-16T00:07:35.114765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114640+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:07:35.114640+0000","last_peered":"2024-09-16T00:07:35.114640+0000","last_clean":"2024-09-16T00:07:35.114640+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:07:35.114640+0000","last_undegraded":"2024-09-16T00:07:35.114640+0000","last_fullsized":"2024-09-16T00:07:35.114640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115333+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:07:35.115333+0000","last_peered":"2024-09-16T00:07:35.115333+0000","last_clean":"2024-09-16T00:07:35.115333+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:07:35.115333+0000","last_undegraded":"2024-09-16T00:07:35.115333+0000","last_fullsized":"2024-09-16T00:07:35.115333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114803+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:07:35.114803+0000","last_peered":"2024-09-16T00:07:35.114803+0000","last_clean":"2024-09-16T00:07:35.114803+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:07:35.114803+0000","last_undegraded":"2024-09-16T00:07:35.114803+0000","last_fullsized":"2024-09-16T00:07:35.114803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.621871+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:07:38.621871+0000","last_peered":"2024-09-16T00:07:38.621871+0000","last_clean":"2024-09-16T00:07:38.621871+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:07:38.621871+0000","last_undegraded":"2024-09-16T00:07:38.621871+0000","last_fullsized":"2024-09-16T00:07:38.621871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114993+0000","last_change":"2024-09-16T00:07:33.637979+0000","last_active":"2024-09-16T00:07:35.114993+0000","last_peered":"2024-09-16T00:07:35.114993+0000","last_clean":"2024-09-16T00:07:35.114993+0000","last_became_active":"2024-09-16T00:07:33.637713+0000","last_became_peered":"2024-09-16T00:07:33.637713+0000","last_unstale":"2024-09-16T00:07:35.114993+0000","last_undegraded":"2024-09-16T00:07:35.114993+0000","last_fullsized":"2024-09-16T00:07:35.114993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615387+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:07:34.615387+0000","last_peered":"2024-09-16T00:07:34.615387+0000","last_clean":"2024-09-16T00:07:34.615387+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:07:34.615387+0000","last_undegraded":"2024-09-16T00:07:34.615387+0000","last_fullsized":"2024-09-16T00:07:34.615387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.621868+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:07:38.621868+0000","last_peered":"2024-09-16T00:07:38.621868+0000","last_clean":"2024-09-16T00:07:38.621868+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:07:38.621868+0000","last_undegraded":"2024-09-16T00:07:38.621868+0000","last_fullsized":"2024-09-16T00:07:38.621868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615266+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:07:34.615266+0000","last_peered":"2024-09-16T00:07:34.615266+0000","last_clean":"2024-09-16T00:07:34.615266+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:07:34.615266+0000","last_undegraded":"2024-09-16T00:07:34.615266+0000","last_fullsized":"2024-09-16T00:07:34.615266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.624540+0000","last_change":"2024-09-16T00:07:32.632481+0000","last_active":"2024-09-16T00:07:38.624540+0000","last_peered":"2024-09-16T00:07:38.624540+0000","last_clean":"2024-09-16T00:07:38.624540+0000","last_became_active":"2024-09-16T00:07:32.632152+0000","last_became_peered":"2024-09-16T00:07:32.632152+0000","last_unstale":"2024-09-16T00:07:38.624540+0000","last_undegraded":"2024-09-16T00:07:38.624540+0000","last_fullsized":"2024-09-16T00:07:38.624540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114783+0000","last_change":"2024-09-16T00:07:33.629777+0000","last_active":"2024-09-16T00:07:35.114783+0000","last_peered":"2024-09-16T00:07:35.114783+0000","last_clean":"2024-09-16T00:07:35.114783+0000","last_became_active":"2024-09-16T00:07:33.627917+0000","last_became_peered":"2024-09-16T00:07:33.627917+0000","last_unstale":"2024-09-16T00:07:35.114783+0000","last_undegraded":"2024-09-16T00:07:35.114783+0000","last_fullsized":"2024-09-16T00:07:35.114783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:38.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.059155+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:07:36.059155+0000","last_peered":"2024-09-16T00:07:36.059155+0000","last_clean":"2024-09-16T00:07:36.059155+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:07:36.059155+0000","last_undegraded":"2024-09-16T00:07:36.059155+0000","last_fullsized":"2024-09-16T00:07:36.059155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114899+0000","last_change":"2024-09-16T00:07:33.630746+0000","last_active":"2024-09-16T00:07:35.114899+0000","last_peered":"2024-09-16T00:07:35.114899+0000","last_clean":"2024-09-16T00:07:35.114899+0000","last_became_active":"2024-09-16T00:07:33.630480+0000","last_became_peered":"2024-09-16T00:07:33.630480+0000","last_unstale":"2024-09-16T00:07:35.114899+0000","last_undegraded":"2024-09-16T00:07:35.114899+0000","last_fullsized":"2024-09-16T00:07:35.114899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:50.931998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.622293+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:07:38.622293+0000","last_peered":"2024-09-16T00:07:38.622293+0000","last_clean":"2024-09-16T00:07:38.622293+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:07:38.622293+0000","last_undegraded":"2024-09-16T00:07:38.622293+0000","last_fullsized":"2024-09-16T00:07:38.622293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":668946,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":78122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":78156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":669002,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:07:46.462 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T00:07:46.462 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T00:07:46.462 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T00:07:46.462 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T00:07:46.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.594+0000 7f04905b7640 1 Processor -- start 2024-09-16T00:07:46.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.594+0000 7f04905b7640 1 -- start start 2024-09-16T00:07:46.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.595+0000 7f04905b7640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0488072c30 0x7f04880c6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.595+0000 7f04905b7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0488158550 con 0x7f0488073010 2024-09-16T00:07:46.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.595+0000 7f04905b7640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04881576f0 con 0x7f048815aeb0 2024-09-16T00:07:46.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.595+0000 7f04905b7640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f04881579d0 con 0x7f0488072c30 2024-09-16T00:07:46.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.595+0000 7f048db52640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f0488073010 0x7f04880c3590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45292/0 (socket says 172.21.15.139:45292) 2024-09-16T00:07:46.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.595+0000 7f048db52640 1 -- 172.21.15.139:0/2492183735 learned_addr learned my addr 172.21.15.139:0/2492183735 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:46.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.595+0000 7f048e353640 1 --2- 172.21.15.139:0/2492183735 >> v2:172.21.15.164:3300/0 conn(0x7f0488072c30 0x7f04880c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.596+0000 7f048d351640 1 -- 172.21.15.139:0/2492183735 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2685950278 0 0) 0x7f0488158550 con 0x7f0488073010 2024-09-16T00:07:46.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.596+0000 7f048d351640 1 -- 172.21.15.139:0/2492183735 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0474003520 con 0x7f0488073010 2024-09-16T00:07:46.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.596+0000 7f048d351640 1 -- 172.21.15.139:0/2492183735 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2343038063 0 0) 0x7f04881576f0 con 0x7f048815aeb0 2024-09-16T00:07:46.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.596+0000 7f048d351640 1 -- 172.21.15.139:0/2492183735 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0488158550 con 0x7f048815aeb0 2024-09-16T00:07:46.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.596+0000 7f048e353640 1 -- 172.21.15.139:0/2492183735 >> v1:172.21.15.164:6789/0 conn(0x7f048815aeb0 legacy=0x7f048815b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.596+0000 7f048e353640 1 -- 172.21.15.139:0/2492183735 >> v1:172.21.15.139:6789/0 conn(0x7f0488073010 legacy=0x7f04880c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.596+0000 7f048e353640 1 -- 172.21.15.139:0/2492183735 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0488157cb0 con 0x7f0488072c30 2024-09-16T00:07:46.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.596+0000 7f048e353640 1 --2- 172.21.15.139:0/2492183735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0488072c30 0x7f04880c6210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f04840097a0 tx=0x7f048402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b603a6727b43627 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:46.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.597+0000 7f048d351640 1 -- 172.21.15.139:0/2492183735 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484037070 con 0x7f0488072c30 2024-09-16T00:07:46.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.597+0000 7f048d351640 1 -- 172.21.15.139:0/2492183735 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0484032640 con 0x7f0488072c30 2024-09-16T00:07:46.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.597+0000 7f048d351640 1 -- 172.21.15.139:0/2492183735 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484032a10 con 0x7f0488072c30 2024-09-16T00:07:46.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.597+0000 7f04905b7640 1 -- 172.21.15.139:0/2492183735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0488072c30 msgr2=0x7f04880c6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.597+0000 7f04905b7640 1 --2- 172.21.15.139:0/2492183735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0488072c30 0x7f04880c6210 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f04840097a0 tx=0x7f048402a2c0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.597+0000 7f04905b7640 1 -- 172.21.15.139:0/2492183735 shutdown_connections 2024-09-16T00:07:46.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.597+0000 7f04905b7640 1 --2- 172.21.15.139:0/2492183735 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0488072c30 0x7f04880c6210 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.597+0000 7f04905b7640 1 -- 172.21.15.139:0/2492183735 >> 172.21.15.139:0/2492183735 conn(0x7f0488067950 msgr2=0x7f0488067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:46.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.597+0000 7f04905b7640 1 -- 172.21.15.139:0/2492183735 shutdown_connections 2024-09-16T00:07:46.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.597+0000 7f04905b7640 1 -- 172.21.15.139:0/2492183735 wait complete. 2024-09-16T00:07:46.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.598+0000 7f04905b7640 1 Processor -- start 2024-09-16T00:07:46.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.598+0000 7f04905b7640 1 -- start start 2024-09-16T00:07:46.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f04905b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0488072c30 0x7f04881f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f04905b7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0488073010 0x7f048806f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f04905b7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f048815aeb0 0x7f04881e2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f04905b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f04880c6b20 con 0x7f0488072c30 2024-09-16T00:07:46.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f04905b7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0488158270 con 0x7f0488073010 2024-09-16T00:07:46.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f04905b7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0488151020 con 0x7f048815aeb0 2024-09-16T00:07:46.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f048e353640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0488072c30 0x7f04881f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f048e353640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0488072c30 0x7f04881f00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40740/0 (socket says 172.21.15.139:40740) 2024-09-16T00:07:46.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f048e353640 1 -- 172.21.15.139:0/3264742896 learned_addr learned my addr 172.21.15.139:0/3264742896 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:46.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f048db52640 1 --2- 172.21.15.139:0/3264742896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0488073010 0x7f048806f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f048eb54640 1 --2- 172.21.15.139:0/3264742896 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f048815aeb0 0x7f04881e2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f048e353640 1 -- 172.21.15.139:0/3264742896 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f048815aeb0 msgr2=0x7f04881e2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f048e353640 1 --2- 172.21.15.139:0/3264742896 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f048815aeb0 0x7f04881e2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f048e353640 1 -- 172.21.15.139:0/3264742896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0488073010 msgr2=0x7f048806f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f048e353640 1 --2- 172.21.15.139:0/3264742896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0488073010 0x7f048806f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f048e353640 1 -- 172.21.15.139:0/3264742896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0488158c10 con 0x7f0488072c30 2024-09-16T00:07:46.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.599+0000 7f048db52640 1 --2- 172.21.15.139:0/3264742896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0488073010 0x7f048806f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:46.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.600+0000 7f048e353640 1 --2- 172.21.15.139:0/3264742896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0488072c30 0x7f04881f00d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0484035040 tx=0x7f04840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:46.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.600+0000 7f047effd640 1 -- 172.21.15.139:0/3264742896 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484037040 con 0x7f0488072c30 2024-09-16T00:07:46.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.600+0000 7f047effd640 1 -- 172.21.15.139:0/3264742896 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0484032440 con 0x7f0488072c30 2024-09-16T00:07:46.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.600+0000 7f047effd640 1 -- 172.21.15.139:0/3264742896 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04840309d0 con 0x7f0488072c30 2024-09-16T00:07:46.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.600+0000 7f04905b7640 1 -- 172.21.15.139:0/3264742896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0488158ed0 con 0x7f0488072c30 2024-09-16T00:07:46.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.600+0000 7f04905b7640 1 -- 172.21.15.139:0/3264742896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0488159940 con 0x7f0488072c30 2024-09-16T00:07:46.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.601+0000 7f04905b7640 1 -- 172.21.15.139:0/3264742896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04880c6210 con 0x7f0488072c30 2024-09-16T00:07:46.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.601+0000 7f047effd640 1 -- 172.21.15.139:0/3264742896 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0484044020 con 0x7f0488072c30 2024-09-16T00:07:46.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.601+0000 7f047effd640 1 --2- 172.21.15.139:0/3264742896 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f046c06c2e0 0x7f046c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.602+0000 7f047effd640 1 -- 172.21.15.139:0/3264742896 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f04840ab6d0 con 0x7f0488072c30 2024-09-16T00:07:46.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.602+0000 7f048db52640 1 --2- 172.21.15.139:0/3264742896 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f046c06c2e0 0x7f046c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.602+0000 7f048db52640 1 --2- 172.21.15.139:0/3264742896 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f046c06c2e0 0x7f046c06e7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0478007fd0 tx=0x7f0478009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:46.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.606+0000 7f047effd640 1 -- 172.21.15.139:0/3264742896 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0484079ed0 con 0x7f0488072c30 2024-09-16T00:07:46.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.807+0000 7f04905b7640 1 -- 172.21.15.139:0/3264742896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f04881e4420 con 0x7f0488072c30 2024-09-16T00:07:46.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.807+0000 7f047effd640 1 -- 172.21.15.139:0/3264742896 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f04840797d0 con 0x7f0488072c30 2024-09-16T00:07:46.809 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:46.809 INFO:teuthology.orchestra.run.smithi139.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T00:07:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.809+0000 7f04905b7640 1 -- 172.21.15.139:0/3264742896 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f046c06c2e0 msgr2=0x7f046c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.809+0000 7f04905b7640 1 --2- 172.21.15.139:0/3264742896 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f046c06c2e0 0x7f046c06e7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0478007fd0 tx=0x7f0478009040 comp rx=0 tx=0).stop 2024-09-16T00:07:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.809+0000 7f04905b7640 1 -- 172.21.15.139:0/3264742896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0488072c30 msgr2=0x7f04881f00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.809+0000 7f04905b7640 1 --2- 172.21.15.139:0/3264742896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0488072c30 0x7f04881f00d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0484035040 tx=0x7f04840046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.809+0000 7f04905b7640 1 -- 172.21.15.139:0/3264742896 shutdown_connections 2024-09-16T00:07:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.809+0000 7f04905b7640 1 --2- 172.21.15.139:0/3264742896 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f046c06c2e0 0x7f046c06e7a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.809+0000 7f04905b7640 1 --2- 172.21.15.139:0/3264742896 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f048815aeb0 0x7f04881e2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.809+0000 7f04905b7640 1 --2- 172.21.15.139:0/3264742896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0488073010 0x7f048806f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.809+0000 7f04905b7640 1 --2- 172.21.15.139:0/3264742896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0488072c30 0x7f04881f00d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.809+0000 7f04905b7640 1 -- 172.21.15.139:0/3264742896 >> 172.21.15.139:0/3264742896 conn(0x7f0488067950 msgr2=0x7f048814e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.809+0000 7f04905b7640 1 -- 172.21.15.139:0/3264742896 shutdown_connections 2024-09-16T00:07:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.809+0000 7f04905b7640 1 -- 172.21.15.139:0/3264742896 wait complete. 2024-09-16T00:07:46.821 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T00:07:46.821 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T00:07:46.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.918+0000 7f2a8d56f640 1 Processor -- start 2024-09-16T00:07:46.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.918+0000 7f2a8d56f640 1 -- start start 2024-09-16T00:07:46.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.919+0000 7f2a8d56f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2a88154db0 0x7f2a8806cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.919+0000 7f2a8d56f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2a88074030 0x7f2a880ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.919+0000 7f2a8d56f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a8815dfd0 con 0x7f2a88072850 2024-09-16T00:07:46.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.919+0000 7f2a8d56f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2a8806d4a0 con 0x7f2a88154db0 2024-09-16T00:07:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.919+0000 7f2a8d56f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f2a8815e240 con 0x7f2a88074030 2024-09-16T00:07:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.919+0000 7f2a86ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2a88154db0 0x7f2a8806cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.919+0000 7f2a86ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2a88154db0 0x7f2a8806cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40742/0 (socket says 172.21.15.139:40742) 2024-09-16T00:07:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.919+0000 7f2a86ffd640 1 -- 172.21.15.139:0/2180867108 learned_addr learned my addr 172.21.15.139:0/2180867108 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.920+0000 7f2a7e7fc640 1 --2- 172.21.15.139:0/2180867108 >> v2:172.21.15.164:3300/0 conn(0x7f2a88074030 0x7f2a880ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.920+0000 7f2a86ffd640 1 -- 172.21.15.139:0/2180867108 >> v2:172.21.15.164:3300/0 conn(0x7f2a88074030 msgr2=0x7f2a880ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.920+0000 7f2a86ffd640 1 --2- 172.21.15.139:0/2180867108 >> v2:172.21.15.164:3300/0 conn(0x7f2a88074030 0x7f2a880ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.920+0000 7f2a86ffd640 1 -- 172.21.15.139:0/2180867108 >> v1:172.21.15.164:6789/0 conn(0x7f2a88072850 legacy=0x7f2a88073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.920+0000 7f2a86ffd640 1 -- 172.21.15.139:0/2180867108 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8815e520 con 0x7f2a88154db0 2024-09-16T00:07:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.920+0000 7f2a86ffd640 1 --2- 172.21.15.139:0/2180867108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a88154db0 0x7f2a8806cc40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2a88075870 tx=0x7f2a7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a39f9e53fa51695 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.920+0000 7f2a867fc640 1 -- 172.21.15.139:0/2180867108 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a78037070 con 0x7f2a88154db0 2024-09-16T00:07:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.921+0000 7f2a867fc640 1 -- 172.21.15.139:0/2180867108 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a78032440 con 0x7f2a88154db0 2024-09-16T00:07:46.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.921+0000 7f2a867fc640 1 -- 172.21.15.139:0/2180867108 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a78032810 con 0x7f2a88154db0 2024-09-16T00:07:46.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.921+0000 7f2a8d56f640 1 -- 172.21.15.139:0/2180867108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a88154db0 msgr2=0x7f2a8806cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.921+0000 7f2a8d56f640 1 --2- 172.21.15.139:0/2180867108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a88154db0 0x7f2a8806cc40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2a88075870 tx=0x7f2a7802a270 comp rx=0 tx=0).stop 2024-09-16T00:07:46.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.921+0000 7f2a8d56f640 1 -- 172.21.15.139:0/2180867108 shutdown_connections 2024-09-16T00:07:46.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.921+0000 7f2a8d56f640 1 --2- 172.21.15.139:0/2180867108 >> v2:172.21.15.164:3300/0 conn(0x7f2a88074030 0x7f2a880ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.921+0000 7f2a8d56f640 1 --2- 172.21.15.139:0/2180867108 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a88154db0 0x7f2a8806cc40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.921+0000 7f2a8d56f640 1 -- 172.21.15.139:0/2180867108 >> 172.21.15.139:0/2180867108 conn(0x7f2a88070440 msgr2=0x7f2a88070820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:46.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.921+0000 7f2a8d56f640 1 -- 172.21.15.139:0/2180867108 shutdown_connections 2024-09-16T00:07:46.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.921+0000 7f2a8d56f640 1 -- 172.21.15.139:0/2180867108 wait complete. 2024-09-16T00:07:46.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.922+0000 7f2a8d56f640 1 Processor -- start 2024-09-16T00:07:46.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.922+0000 7f2a8d56f640 1 -- start start 2024-09-16T00:07:46.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.923+0000 7f2a8d56f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a88072850 0x7f2a881f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.923+0000 7f2a8d56f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a88154db0 0x7f2a881f0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.923+0000 7f2a8d56f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a8815b560 0x7f2a881537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.923+0000 7f2a8d56f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8815ed20 con 0x7f2a8815b560 2024-09-16T00:07:46.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.923+0000 7f2a8d56f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8806d780 con 0x7f2a88154db0 2024-09-16T00:07:46.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.923+0000 7f2a8d56f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2a880c3700 con 0x7f2a88072850 2024-09-16T00:07:46.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.923+0000 7f2a877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a8815b560 0x7f2a881537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.923+0000 7f2a877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a8815b560 0x7f2a881537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:40748/0 (socket says 172.21.15.139:40748) 2024-09-16T00:07:46.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.923+0000 7f2a877fe640 1 -- 172.21.15.139:0/2071365069 learned_addr learned my addr 172.21.15.139:0/2071365069 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:46.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.923+0000 7f2a7e7fc640 1 --2- 172.21.15.139:0/2071365069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a88154db0 0x7f2a881f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.923+0000 7f2a86ffd640 1 --2- 172.21.15.139:0/2071365069 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a88072850 0x7f2a881f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.924+0000 7f2a877fe640 1 -- 172.21.15.139:0/2071365069 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a88072850 msgr2=0x7f2a881f4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.924+0000 7f2a877fe640 1 --2- 172.21.15.139:0/2071365069 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a88072850 0x7f2a881f4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.924+0000 7f2a877fe640 1 -- 172.21.15.139:0/2071365069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a88154db0 msgr2=0x7f2a881f0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:46.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.924+0000 7f2a877fe640 1 --2- 172.21.15.139:0/2071365069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a88154db0 0x7f2a881f0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:46.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.924+0000 7f2a877fe640 1 -- 172.21.15.139:0/2071365069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a880c39c0 con 0x7f2a8815b560 2024-09-16T00:07:46.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.924+0000 7f2a877fe640 1 --2- 172.21.15.139:0/2071365069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a8815b560 0x7f2a881537c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000c730 tx=0x7f2a8000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:46.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.924+0000 7f2a7ffff640 1 -- 172.21.15.139:0/2071365069 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80016020 con 0x7f2a8815b560 2024-09-16T00:07:46.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.924+0000 7f2a7ffff640 1 -- 172.21.15.139:0/2071365069 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a80006d50 con 0x7f2a8815b560 2024-09-16T00:07:46.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.924+0000 7f2a7ffff640 1 -- 172.21.15.139:0/2071365069 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80003bb0 con 0x7f2a8815b560 2024-09-16T00:07:46.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.924+0000 7f2a8d56f640 1 -- 172.21.15.139:0/2071365069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a881f3a50 con 0x7f2a8815b560 2024-09-16T00:07:46.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.924+0000 7f2a8d56f640 1 -- 172.21.15.139:0/2071365069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a881f4010 con 0x7f2a8815b560 2024-09-16T00:07:46.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.925+0000 7f2a8d56f640 1 -- 172.21.15.139:0/2071365069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a881f4210 con 0x7f2a8815b560 2024-09-16T00:07:46.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.925+0000 7f2a7ffff640 1 -- 172.21.15.139:0/2071365069 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2a800050d0 con 0x7f2a8815b560 2024-09-16T00:07:46.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.926+0000 7f2a7ffff640 1 --2- 172.21.15.139:0/2071365069 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2a6806c2e0 0x7f2a6806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:46.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.926+0000 7f2a7ffff640 1 -- 172.21.15.139:0/2071365069 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2a8008af10 con 0x7f2a8815b560 2024-09-16T00:07:46.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.926+0000 7f2a86ffd640 1 --2- 172.21.15.139:0/2071365069 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2a6806c2e0 0x7f2a6806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:46.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.926+0000 7f2a86ffd640 1 --2- 172.21.15.139:0/2071365069 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2a6806c2e0 0x7f2a6806e7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2a78034000 tx=0x7f2a7802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:46.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:46.930+0000 7f2a7ffff640 1 -- 172.21.15.139:0/2071365069 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2a881f4210 con 0x7f2a8815b560 2024-09-16T00:07:47.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.130+0000 7f2a8d56f640 1 -- 172.21.15.139:0/2071365069 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a8806cc40 con 0x7f2a8815b560 2024-09-16T00:07:47.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.131+0000 7f2a7ffff640 1 -- 172.21.15.139:0/2071365069 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6881 (secure 0 0 0) 0x7f2a80020020 con 0x7f2a8815b560 2024-09-16T00:07:47.132 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:47.132 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":8,"btime":"2024-09-16T00:07:38:635384+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2773358731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2773358731},{"type":"v1","addr":"172.21.15.139:6835","nonce":2773358731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4268,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/2919602297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":2919602297},{"type":"v1","addr":"172.21.15.164:6837","nonce":2919602297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3575043661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":3575043661},{"type":"v1","addr":"172.21.15.139:6837","nonce":3575043661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/4113980181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":4113980181},{"type":"v1","addr":"172.21.15.139:6839","nonce":4113980181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/374774889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":374774889},{"type":"v1","addr":"172.21.15.164:6834","nonce":374774889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T00:07:32.949865+0000","modified":"2024-09-16T00:07:38.635380+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.164:6836/4279698936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":4279698936},{"type":"v1","addr":"172.21.15.164:6836","nonce":4279698936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-09-16T00:07:47.132 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 8 2024-09-16T00:07:47.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.133+0000 7f2a8d56f640 1 -- 172.21.15.139:0/2071365069 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2a6806c2e0 msgr2=0x7f2a6806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.133+0000 7f2a8d56f640 1 --2- 172.21.15.139:0/2071365069 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2a6806c2e0 0x7f2a6806e7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2a78034000 tx=0x7f2a7802e000 comp rx=0 tx=0).stop 2024-09-16T00:07:47.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.133+0000 7f2a8d56f640 1 -- 172.21.15.139:0/2071365069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a8815b560 msgr2=0x7f2a881537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.133+0000 7f2a8d56f640 1 --2- 172.21.15.139:0/2071365069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a8815b560 0x7f2a881537c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000c730 tx=0x7f2a8000cb30 comp rx=0 tx=0).stop 2024-09-16T00:07:47.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.133+0000 7f2a8d56f640 1 -- 172.21.15.139:0/2071365069 shutdown_connections 2024-09-16T00:07:47.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.133+0000 7f2a8d56f640 1 --2- 172.21.15.139:0/2071365069 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2a6806c2e0 0x7f2a6806e7a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.133+0000 7f2a8d56f640 1 --2- 172.21.15.139:0/2071365069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a8815b560 0x7f2a881537c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.133+0000 7f2a8d56f640 1 --2- 172.21.15.139:0/2071365069 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2a88154db0 0x7f2a881f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.133+0000 7f2a8d56f640 1 --2- 172.21.15.139:0/2071365069 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2a88072850 0x7f2a881f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.133+0000 7f2a8d56f640 1 -- 172.21.15.139:0/2071365069 >> 172.21.15.139:0/2071365069 conn(0x7f2a88070440 msgr2=0x7f2a8815d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:47.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.133+0000 7f2a8d56f640 1 -- 172.21.15.139:0/2071365069 shutdown_connections 2024-09-16T00:07:47.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.133+0000 7f2a8d56f640 1 -- 172.21.15.139:0/2071365069 wait complete. 2024-09-16T00:07:47.145 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T00:07:47.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.242+0000 7f7b960a5640 1 Processor -- start 2024-09-16T00:07:47.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.242+0000 7f7b960a5640 1 -- start start 2024-09-16T00:07:47.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.243+0000 7f7b960a5640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b90154bd0 0x7f7b9015b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.243+0000 7f7b960a5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b9006cd40 con 0x7f7b900c8b50 2024-09-16T00:07:47.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.243+0000 7f7b960a5640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b9015a9d0 con 0x7f7b9006ea20 2024-09-16T00:07:47.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.243+0000 7f7b960a5640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9006cfc0 con 0x7f7b90154bd0 2024-09-16T00:07:47.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.243+0000 7f7b8ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f7b900c8b50 0x7f7b90165160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57790/0 (socket says 172.21.15.139:57790) 2024-09-16T00:07:47.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.243+0000 7f7b8ffff640 1 -- 172.21.15.139:0/1477002366 learned_addr learned my addr 172.21.15.139:0/1477002366 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:47.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.243+0000 7f7b8f7fe640 1 --2- 172.21.15.139:0/1477002366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b90154bd0 0x7f7b9015b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.243+0000 7f7b8e7fc640 1 -- 172.21.15.139:0/1477002366 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2084048149 0 0) 0x7f7b88001110 con 0x7f7b900c8b50 2024-09-16T00:07:47.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8e7fc640 1 -- 172.21.15.139:0/1477002366 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b78003520 con 0x7f7b900c8b50 2024-09-16T00:07:47.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8e7fc640 1 -- 172.21.15.139:0/1477002366 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3751054627 0 0) 0x7f7b78003520 con 0x7f7b900c8b50 2024-09-16T00:07:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8e7fc640 1 -- 172.21.15.139:0/1477002366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b90154bd0 msgr2=0x7f7b9015b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8e7fc640 1 --2- 172.21.15.139:0/1477002366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b90154bd0 0x7f7b9015b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8e7fc640 1 -- 172.21.15.139:0/1477002366 >> v1:172.21.15.164:6789/0 conn(0x7f7b9006ea20 legacy=0x7f7b900ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8e7fc640 1 -- 172.21.15.139:0/1477002366 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9015ac70 con 0x7f7b900c8b50 2024-09-16T00:07:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8e7fc640 1 -- 172.21.15.139:0/1477002366 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7f7b88001840 con 0x7f7b900c8b50 2024-09-16T00:07:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8e7fc640 1 -- 172.21.15.139:0/1477002366 >> v1:172.21.15.139:6789/0 conn(0x7f7b900c8b50 legacy=0x7f7b90165160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8e7fc640 1 --2- 172.21.15.139:0/1477002366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b780039e0 0x7f7b78003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8e7fc640 1 --2- 172.21.15.139:0/1477002366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b78004800 0x7f7b78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8e7fc640 1 --2- 172.21.15.139:0/1477002366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b90154bd0 0x7f7b780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8e7fc640 1 -- 172.21.15.139:0/1477002366 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9006cfc0 con 0x7f7b780039e0 2024-09-16T00:07:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8e7fc640 1 -- 172.21.15.139:0/1477002366 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7800ba70 con 0x7f7b78004800 2024-09-16T00:07:47.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8e7fc640 1 -- 172.21.15.139:0/1477002366 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7b7800bc70 con 0x7f7b90154bd0 2024-09-16T00:07:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8f7fe640 1 --2- 172.21.15.139:0/1477002366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b780039e0 0x7f7b78003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8ffff640 1 --2- 172.21.15.139:0/1477002366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b90154bd0 0x7f7b780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8effd640 1 --2- 172.21.15.139:0/1477002366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b78004800 0x7f7b78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8f7fe640 1 -- 172.21.15.139:0/1477002366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b90154bd0 msgr2=0x7f7b780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8f7fe640 1 --2- 172.21.15.139:0/1477002366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b90154bd0 0x7f7b780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8f7fe640 1 -- 172.21.15.139:0/1477002366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b78004800 msgr2=0x7f7b78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8f7fe640 1 --2- 172.21.15.139:0/1477002366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b78004800 0x7f7b78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8f7fe640 1 -- 172.21.15.139:0/1477002366 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b78003520 con 0x7f7b780039e0 2024-09-16T00:07:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.244+0000 7f7b8effd640 1 --2- 172.21.15.139:0/1477002366 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b78004800 0x7f7b78004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.245+0000 7f7b8f7fe640 1 --2- 172.21.15.139:0/1477002366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b780039e0 0x7f7b78003e00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009f90 tx=0x7f7b78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8405a34a8c32a600 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.245+0000 7f7b8e7fc640 1 -- 172.21.15.139:0/1477002366 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80009c10 con 0x7f7b780039e0 2024-09-16T00:07:47.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.245+0000 7f7b8e7fc640 1 -- 172.21.15.139:0/1477002366 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b80036070 con 0x7f7b780039e0 2024-09-16T00:07:47.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.245+0000 7f7b8e7fc640 1 -- 172.21.15.139:0/1477002366 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80031830 con 0x7f7b780039e0 2024-09-16T00:07:47.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.245+0000 7f7b960a5640 1 -- 172.21.15.139:0/1477002366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b780039e0 msgr2=0x7f7b78003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.245+0000 7f7b960a5640 1 --2- 172.21.15.139:0/1477002366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b780039e0 0x7f7b78003e00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009f90 tx=0x7f7b78005830 comp rx=0 tx=0).stop 2024-09-16T00:07:47.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.245+0000 7f7b8ffff640 1 -- 172.21.15.139:0/1477002366 reap_dead start 2024-09-16T00:07:47.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.246+0000 7f7b960a5640 1 -- 172.21.15.139:0/1477002366 shutdown_connections 2024-09-16T00:07:47.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.246+0000 7f7b960a5640 1 -- 172.21.15.139:0/1477002366 >> 172.21.15.139:0/1477002366 conn(0x7f7b9006c150 msgr2=0x7f7b90067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:47.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.246+0000 7f7b960a5640 1 -- 172.21.15.139:0/1477002366 shutdown_connections 2024-09-16T00:07:47.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.246+0000 7f7b960a5640 1 -- 172.21.15.139:0/1477002366 wait complete. 2024-09-16T00:07:47.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.247+0000 7f7b960a5640 1 Processor -- start 2024-09-16T00:07:47.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.247+0000 7f7b960a5640 1 -- start start 2024-09-16T00:07:47.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b960a5640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b780039e0 0x7f7b9014a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b960a5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b900cadf0 0x7f7b901f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b960a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b900c9380 0x7f7b900cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b960a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9006d2a0 con 0x7f7b900c9380 2024-09-16T00:07:47.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b960a5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90154fb0 con 0x7f7b900cadf0 2024-09-16T00:07:47.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b960a5640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9006d420 con 0x7f7b780039e0 2024-09-16T00:07:47.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b8ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b900c9380 0x7f7b900cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b8ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b900c9380 0x7f7b900cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47606/0 (socket says 172.21.15.139:47606) 2024-09-16T00:07:47.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b8ffff640 1 -- 172.21.15.139:0/285713569 learned_addr learned my addr 172.21.15.139:0/285713569 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:47.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b8effd640 1 --2- 172.21.15.139:0/285713569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b900cadf0 0x7f7b901f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b8f7fe640 1 --2- 172.21.15.139:0/285713569 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b780039e0 0x7f7b9014a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b8ffff640 1 -- 172.21.15.139:0/285713569 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b780039e0 msgr2=0x7f7b9014a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b8ffff640 1 --2- 172.21.15.139:0/285713569 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b780039e0 0x7f7b9014a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b8ffff640 1 -- 172.21.15.139:0/285713569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b900cadf0 msgr2=0x7f7b901f18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b8ffff640 1 --2- 172.21.15.139:0/285713569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b900cadf0 0x7f7b901f18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b8ffff640 1 -- 172.21.15.139:0/285713569 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b901f0ff0 con 0x7f7b900c9380 2024-09-16T00:07:47.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b8ffff640 1 --2- 172.21.15.139:0/285713569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b900c9380 0x7f7b900cbf50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7b88001eb0 tx=0x7f7b88007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:47.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b77fff640 1 -- 172.21.15.139:0/285713569 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8801a070 con 0x7f7b900c9380 2024-09-16T00:07:47.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b77fff640 1 -- 172.21.15.139:0/285713569 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b88001230 con 0x7f7b900c9380 2024-09-16T00:07:47.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b960a5640 1 -- 172.21.15.139:0/285713569 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b900ca030 con 0x7f7b900c9380 2024-09-16T00:07:47.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b77fff640 1 -- 172.21.15.139:0/285713569 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8800fad0 con 0x7f7b900c9380 2024-09-16T00:07:47.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.248+0000 7f7b960a5640 1 -- 172.21.15.139:0/285713569 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b900ca5f0 con 0x7f7b900c9380 2024-09-16T00:07:47.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.249+0000 7f7b960a5640 1 -- 172.21.15.139:0/285713569 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b9015b470 con 0x7f7b900c9380 2024-09-16T00:07:47.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.250+0000 7f7b77fff640 1 -- 172.21.15.139:0/285713569 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7b8800fc70 con 0x7f7b900c9380 2024-09-16T00:07:47.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.250+0000 7f7b77fff640 1 --2- 172.21.15.139:0/285713569 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b7c06c2e0 0x7f7b7c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.250+0000 7f7b77fff640 1 -- 172.21.15.139:0/285713569 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7b8808a9a0 con 0x7f7b900c9380 2024-09-16T00:07:47.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.250+0000 7f7b8f7fe640 1 --2- 172.21.15.139:0/285713569 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b7c06c2e0 0x7f7b7c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.251+0000 7f7b8f7fe640 1 --2- 172.21.15.139:0/285713569 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b7c06c2e0 0x7f7b7c06e7a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7b80035000 tx=0x7f7b8002f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:47.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.254+0000 7f7b77fff640 1 -- 172.21.15.139:0/285713569 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7b9006cd40 con 0x7f7b900c9380 2024-09-16T00:07:47.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.453+0000 7f7b960a5640 1 -- 172.21.15.139:0/285713569 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b9014b460 con 0x7f7b900c9380 2024-09-16T00:07:47.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.454+0000 7f7b77fff640 1 -- 172.21.15.139:0/285713569 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6881 (secure 0 0 0) 0x7f7b88058b50 con 0x7f7b900c9380 2024-09-16T00:07:47.455 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:47.455 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":8,"btime":"2024-09-16T00:07:38:635384+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2773358731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2773358731},{"type":"v1","addr":"172.21.15.139:6835","nonce":2773358731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4268,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/2919602297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":2919602297},{"type":"v1","addr":"172.21.15.164:6837","nonce":2919602297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3575043661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":3575043661},{"type":"v1","addr":"172.21.15.139:6837","nonce":3575043661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/4113980181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":4113980181},{"type":"v1","addr":"172.21.15.139:6839","nonce":4113980181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/374774889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":374774889},{"type":"v1","addr":"172.21.15.164:6834","nonce":374774889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T00:07:32.949865+0000","modified":"2024-09-16T00:07:38.635380+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.164:6836/4279698936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":4279698936},{"type":"v1","addr":"172.21.15.164:6836","nonce":4279698936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-09-16T00:07:47.455 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 8 2024-09-16T00:07:47.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.456+0000 7f7b960a5640 1 -- 172.21.15.139:0/285713569 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b7c06c2e0 msgr2=0x7f7b7c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.456+0000 7f7b960a5640 1 --2- 172.21.15.139:0/285713569 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b7c06c2e0 0x7f7b7c06e7a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7b80035000 tx=0x7f7b8002f000 comp rx=0 tx=0).stop 2024-09-16T00:07:47.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.456+0000 7f7b960a5640 1 -- 172.21.15.139:0/285713569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b900c9380 msgr2=0x7f7b900cbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.456+0000 7f7b960a5640 1 --2- 172.21.15.139:0/285713569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b900c9380 0x7f7b900cbf50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7b88001eb0 tx=0x7f7b88007720 comp rx=0 tx=0).stop 2024-09-16T00:07:47.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.456+0000 7f7b960a5640 1 -- 172.21.15.139:0/285713569 shutdown_connections 2024-09-16T00:07:47.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.456+0000 7f7b960a5640 1 --2- 172.21.15.139:0/285713569 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b7c06c2e0 0x7f7b7c06e7a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.456+0000 7f7b960a5640 1 --2- 172.21.15.139:0/285713569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b900c9380 0x7f7b900cbf50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.456+0000 7f7b960a5640 1 --2- 172.21.15.139:0/285713569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b900cadf0 0x7f7b901f18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.456+0000 7f7b960a5640 1 --2- 172.21.15.139:0/285713569 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b780039e0 0x7f7b9014a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.456+0000 7f7b960a5640 1 -- 172.21.15.139:0/285713569 >> 172.21.15.139:0/285713569 conn(0x7f7b9006c150 msgr2=0x7f7b900c3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:47.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.456+0000 7f7b960a5640 1 -- 172.21.15.139:0/285713569 shutdown_connections 2024-09-16T00:07:47.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.456+0000 7f7b960a5640 1 -- 172.21.15.139:0/285713569 wait complete. 2024-09-16T00:07:47.468 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:07:47.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.565+0000 7f2fb48e6640 1 Processor -- start 2024-09-16T00:07:47.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.566+0000 7f2fb48e6640 1 -- start start 2024-09-16T00:07:47.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.566+0000 7f2fb48e6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2fb01527f0 0x7f2fb00c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.566+0000 7f2fb48e6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2fb006f7f0 0x7f2fb0158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.567+0000 7f2fb48e6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fb00707f0 con 0x7f2fb00c8250 2024-09-16T00:07:47.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.567+0000 7f2fb48e6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2fb006ff20 con 0x7f2fb01527f0 2024-09-16T00:07:47.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.567+0000 7f2fb48e6640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f2fb0070170 con 0x7f2fb006f7f0 2024-09-16T00:07:47.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.567+0000 7f2fae59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2fb01527f0 0x7f2fb00c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.567+0000 7f2faed9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2fb00c8250 0x7f2fb00ce0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57806/0 (socket says 172.21.15.139:57806) 2024-09-16T00:07:47.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.567+0000 7f2fae59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2fb01527f0 0x7f2fb00c4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47620/0 (socket says 172.21.15.139:47620) 2024-09-16T00:07:47.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.567+0000 7f2faed9d640 1 -- 172.21.15.139:0/4094836539 learned_addr learned my addr 172.21.15.139:0/4094836539 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:47.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.567+0000 7f2fadd9b640 1 --2- 172.21.15.139:0/4094836539 >> v2:172.21.15.164:3300/0 conn(0x7f2fb006f7f0 0x7f2fb0158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.567+0000 7f2fae59c640 1 -- 172.21.15.139:0/4094836539 >> v2:172.21.15.164:3300/0 conn(0x7f2fb006f7f0 msgr2=0x7f2fb0158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:47.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.567+0000 7f2fae59c640 1 --2- 172.21.15.139:0/4094836539 >> v2:172.21.15.164:3300/0 conn(0x7f2fb006f7f0 0x7f2fb0158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.567+0000 7f2fae59c640 1 -- 172.21.15.139:0/4094836539 >> v1:172.21.15.139:6789/0 conn(0x7f2fb00c8250 legacy=0x7f2fb00ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.567+0000 7f2fae59c640 1 -- 172.21.15.139:0/4094836539 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb0070a90 con 0x7f2fb01527f0 2024-09-16T00:07:47.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.567+0000 7f2fad59a640 1 -- 172.21.15.139:0/4094836539 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1740434100 0 0) 0x7f2fb00707f0 con 0x7f2fb00c8250 2024-09-16T00:07:47.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.568+0000 7f2fae59c640 1 --2- 172.21.15.139:0/4094836539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb01527f0 0x7f2fb00c4e80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00ced20 tx=0x7f2fa002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744505b3ede0818a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:47.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.568+0000 7f2fad59a640 1 -- 172.21.15.139:0/4094836539 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0037070 con 0x7f2fb01527f0 2024-09-16T00:07:47.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.568+0000 7f2fad59a640 1 -- 172.21.15.139:0/4094836539 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fa0032440 con 0x7f2fb01527f0 2024-09-16T00:07:47.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.568+0000 7f2fad59a640 1 -- 172.21.15.139:0/4094836539 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa002ae60 con 0x7f2fb01527f0 2024-09-16T00:07:47.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.568+0000 7f2fb48e6640 1 -- 172.21.15.139:0/4094836539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb01527f0 msgr2=0x7f2fb00c4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.568+0000 7f2fb48e6640 1 --2- 172.21.15.139:0/4094836539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb01527f0 0x7f2fb00c4e80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00ced20 tx=0x7f2fa002a110 comp rx=0 tx=0).stop 2024-09-16T00:07:47.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.568+0000 7f2fb48e6640 1 -- 172.21.15.139:0/4094836539 shutdown_connections 2024-09-16T00:07:47.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.568+0000 7f2fb48e6640 1 --2- 172.21.15.139:0/4094836539 >> v2:172.21.15.164:3300/0 conn(0x7f2fb006f7f0 0x7f2fb0158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.568+0000 7f2fb48e6640 1 --2- 172.21.15.139:0/4094836539 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb01527f0 0x7f2fb00c4e80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.568+0000 7f2fb48e6640 1 -- 172.21.15.139:0/4094836539 >> 172.21.15.139:0/4094836539 conn(0x7f2fb006c150 msgr2=0x7f2fb0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:47.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.568+0000 7f2fb48e6640 1 -- 172.21.15.139:0/4094836539 shutdown_connections 2024-09-16T00:07:47.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.568+0000 7f2fb48e6640 1 -- 172.21.15.139:0/4094836539 wait complete. 2024-09-16T00:07:47.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.570+0000 7f2fb48e6640 1 Processor -- start 2024-09-16T00:07:47.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.570+0000 7f2fb48e6640 1 -- start start 2024-09-16T00:07:47.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.570+0000 7f2fb48e6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fb006f7f0 0x7f2fb00c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.570+0000 7f2fb48e6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb00c8250 0x7f2fb01ef3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.570+0000 7f2fb48e6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2fb01527f0 0x7f2fb01ef920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.570+0000 7f2fb48e6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb00ce7d0 con 0x7f2fb00c8250 2024-09-16T00:07:47.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.570+0000 7f2fb48e6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb0152bd0 con 0x7f2fb006f7f0 2024-09-16T00:07:47.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.570+0000 7f2fb48e6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb00ce950 con 0x7f2fb01527f0 2024-09-16T00:07:47.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2fadd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb00c8250 0x7f2fb01ef3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2fadd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb00c8250 0x7f2fb01ef3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47626/0 (socket says 172.21.15.139:47626) 2024-09-16T00:07:47.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2fadd9b640 1 -- 172.21.15.139:0/3980270353 learned_addr learned my addr 172.21.15.139:0/3980270353 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:47.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2fae59c640 1 --2- 172.21.15.139:0/3980270353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fb006f7f0 0x7f2fb00c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2faed9d640 1 --2- 172.21.15.139:0/3980270353 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2fb01527f0 0x7f2fb01ef920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2fadd9b640 1 -- 172.21.15.139:0/3980270353 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2fb01527f0 msgr2=0x7f2fb01ef920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2fadd9b640 1 --2- 172.21.15.139:0/3980270353 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2fb01527f0 0x7f2fb01ef920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2fadd9b640 1 -- 172.21.15.139:0/3980270353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fb006f7f0 msgr2=0x7f2fb00c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2fadd9b640 1 --2- 172.21.15.139:0/3980270353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fb006f7f0 0x7f2fb00c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2fadd9b640 1 -- 172.21.15.139:0/3980270353 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb01f2080 con 0x7f2fb00c8250 2024-09-16T00:07:47.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2fae59c640 1 --2- 172.21.15.139:0/3980270353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fb006f7f0 0x7f2fb00c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:47.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2faed9d640 1 --2- 172.21.15.139:0/3980270353 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2fb01527f0 0x7f2fb01ef920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:47.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2fadd9b640 1 --2- 172.21.15.139:0/3980270353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb00c8250 0x7f2fb01ef3e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800d5f0 tx=0x7f2f9800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:47.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2fa6ffd640 1 -- 172.21.15.139:0/3980270353 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f980125f0 con 0x7f2fb00c8250 2024-09-16T00:07:47.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2fa6ffd640 1 -- 172.21.15.139:0/3980270353 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f98019070 con 0x7f2fb00c8250 2024-09-16T00:07:47.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2fb48e6640 1 -- 172.21.15.139:0/3980270353 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb01f1190 con 0x7f2fb00c8250 2024-09-16T00:07:47.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2fa6ffd640 1 -- 172.21.15.139:0/3980270353 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f980129d0 con 0x7f2fb00c8250 2024-09-16T00:07:47.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.571+0000 7f2fb48e6640 1 -- 172.21.15.139:0/3980270353 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb01f1750 con 0x7f2fb00c8250 2024-09-16T00:07:47.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.572+0000 7f2fb48e6640 1 -- 172.21.15.139:0/3980270353 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb0158580 con 0x7f2fb00c8250 2024-09-16T00:07:47.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.573+0000 7f2fa6ffd640 1 -- 172.21.15.139:0/3980270353 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2f98012b70 con 0x7f2fb00c8250 2024-09-16T00:07:47.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.573+0000 7f2fa6ffd640 1 --2- 172.21.15.139:0/3980270353 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2f9006c2e0 0x7f2f9006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.573+0000 7f2fa6ffd640 1 -- 172.21.15.139:0/3980270353 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2f9808b7e0 con 0x7f2fb00c8250 2024-09-16T00:07:47.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.573+0000 7f2fae59c640 1 --2- 172.21.15.139:0/3980270353 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2f9006c2e0 0x7f2f9006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.574+0000 7f2fae59c640 1 --2- 172.21.15.139:0/3980270353 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2f9006c2e0 0x7f2f9006e7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0034000 tx=0x7f2fa002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:47.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.577+0000 7f2fa6ffd640 1 -- 172.21.15.139:0/3980270353 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2f98059f50 con 0x7f2fb00c8250 2024-09-16T00:07:47.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.745+0000 7f2fb48e6640 1 -- 172.21.15.139:0/3980270353 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fb00c4fb0 con 0x7f2fb00c8250 2024-09-16T00:07:47.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.746+0000 7f2fa6ffd640 1 -- 172.21.15.139:0/3980270353 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16725 (secure 0 0 0) 0x7f2f98059850 con 0x7f2fb00c8250 2024-09-16T00:07:47.749 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:47.749 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":22,"fsid":"2d3366e5-6a59-4b59-93b2-615e72ac7015","created":"2024-09-16T00:07:09.967833+0000","modified":"2024-09-16T00:07:34.609954+0000","last_up_change":"2024-09-16T00:07:21.049678+0000","last_in_change":"2024-09-16T00:07:12.025620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:07:22.056303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:07:30.768649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:07:32.008209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83699a05-4937-43c3-aefc-b669bed9bb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6817","nonce":1974901438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6819","nonce":1974901438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6823","nonce":1974901438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6821","nonce":1974901438}]},"public_addr":"172.21.15.139:6817/1974901438","cluster_addr":"172.21.15.139:6819/1974901438","heartbeat_back_addr":"172.21.15.139:6823/1974901438","heartbeat_front_addr":"172.21.15.139:6821/1974901438","state":["exists","up"]},{"osd":1,"uuid":"9c04991e-9695-4bbb-8011-bf613bf404b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6809","nonce":2232380998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6811","nonce":2232380998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6815","nonce":2232380998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6813","nonce":2232380998}]},"public_addr":"172.21.15.139:6809/2232380998","cluster_addr":"172.21.15.139:6811/2232380998","heartbeat_back_addr":"172.21.15.139:6815/2232380998","heartbeat_front_addr":"172.21.15.139:6813/2232380998","state":["exists","up"]},{"osd":2,"uuid":"8048fc64-72b3-4866-ac62-6a8b2bf6f5c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6801","nonce":4180588905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6803","nonce":4180588905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6807","nonce":4180588905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6805","nonce":4180588905}]},"public_addr":"172.21.15.139:6801/4180588905","cluster_addr":"172.21.15.139:6803/4180588905","heartbeat_back_addr":"172.21.15.139:6807/4180588905","heartbeat_front_addr":"172.21.15.139:6805/4180588905","state":["exists","up"]},{"osd":3,"uuid":"c34ac77b-354c-4884-902d-3e2a84f5eb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6827","nonce":3947768492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6829","nonce":3947768492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6833","nonce":3947768492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6831","nonce":3947768492}]},"public_addr":"172.21.15.139:6827/3947768492","cluster_addr":"172.21.15.139:6829/3947768492","heartbeat_back_addr":"172.21.15.139:6833/3947768492","heartbeat_front_addr":"172.21.15.139:6831/3947768492","state":["exists","up"]},{"osd":4,"uuid":"ba9a7c93-9672-43e1-8c98-2878cdf7f548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6817","nonce":642632916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6819","nonce":642632916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6823","nonce":642632916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6821","nonce":642632916}]},"public_addr":"172.21.15.164:6817/642632916","cluster_addr":"172.21.15.164:6819/642632916","heartbeat_back_addr":"172.21.15.164:6823/642632916","heartbeat_front_addr":"172.21.15.164:6821/642632916","state":["exists","up"]},{"osd":5,"uuid":"46a0916b-474c-411b-84ad-86f27d4cbf53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6801","nonce":1051778383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6803","nonce":1051778383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6807","nonce":1051778383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6805","nonce":1051778383}]},"public_addr":"172.21.15.164:6801/1051778383","cluster_addr":"172.21.15.164:6803/1051778383","heartbeat_back_addr":"172.21.15.164:6807/1051778383","heartbeat_front_addr":"172.21.15.164:6805/1051778383","state":["exists","up"]},{"osd":6,"uuid":"9b6b33cc-d6d9-4a0a-8143-526073a58b93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6825","nonce":1908495807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6827","nonce":1908495807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6831","nonce":1908495807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":1908495807},{"type":"v1","addr":"172.21.15.164:6829","nonce":1908495807}]},"public_addr":"172.21.15.164:6825/1908495807","cluster_addr":"172.21.15.164:6827/1908495807","heartbeat_back_addr":"172.21.15.164:6831/1908495807","heartbeat_front_addr":"172.21.15.164:6829/1908495807","state":["exists","up"]},{"osd":7,"uuid":"50bcd808-3d3e-4a72-88e4-30decb53cfea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6809","nonce":1158838808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6811","nonce":1158838808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6815","nonce":1158838808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":1158838808},{"type":"v1","addr":"172.21.15.164:6813","nonce":1158838808}]},"public_addr":"172.21.15.164:6809/1158838808","cluster_addr":"172.21.15.164:6811/1158838808","heartbeat_back_addr":"172.21.15.164:6815/1158838808","heartbeat_front_addr":"172.21.15.164:6813/1158838808","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.843117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.622789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:17.919457+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:19.330341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.815640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.083046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.551499+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.377179+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:07:47.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.748+0000 7f2fb48e6640 1 -- 172.21.15.139:0/3980270353 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2f9006c2e0 msgr2=0x7f2f9006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.748+0000 7f2fb48e6640 1 --2- 172.21.15.139:0/3980270353 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2f9006c2e0 0x7f2f9006e7a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0034000 tx=0x7f2fa002e000 comp rx=0 tx=0).stop 2024-09-16T00:07:47.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.748+0000 7f2fb48e6640 1 -- 172.21.15.139:0/3980270353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb00c8250 msgr2=0x7f2fb01ef3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.748+0000 7f2fb48e6640 1 --2- 172.21.15.139:0/3980270353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb00c8250 0x7f2fb01ef3e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800d5f0 tx=0x7f2f9800dab0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.748+0000 7f2fb48e6640 1 -- 172.21.15.139:0/3980270353 shutdown_connections 2024-09-16T00:07:47.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.748+0000 7f2fb48e6640 1 --2- 172.21.15.139:0/3980270353 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2f9006c2e0 0x7f2f9006e7a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.748+0000 7f2fb48e6640 1 --2- 172.21.15.139:0/3980270353 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2fb01527f0 0x7f2fb01ef920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.748+0000 7f2fb48e6640 1 --2- 172.21.15.139:0/3980270353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb00c8250 0x7f2fb01ef3e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.748+0000 7f2fb48e6640 1 --2- 172.21.15.139:0/3980270353 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2fb006f7f0 0x7f2fb00c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.748+0000 7f2fb48e6640 1 -- 172.21.15.139:0/3980270353 >> 172.21.15.139:0/3980270353 conn(0x7f2fb006c150 msgr2=0x7f2fb0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:47.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.749+0000 7f2fb48e6640 1 -- 172.21.15.139:0/3980270353 shutdown_connections 2024-09-16T00:07:47.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.749+0000 7f2fb48e6640 1 -- 172.21.15.139:0/3980270353 wait complete. 2024-09-16T00:07:47.763 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T00:07:47.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.861+0000 7ff5f1de4640 1 Processor -- start 2024-09-16T00:07:47.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.861+0000 7ff5f1de4640 1 -- start start 2024-09-16T00:07:47.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.862+0000 7ff5f1de4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff5ec154cd0 0x7ff5ec15b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.862+0000 7ff5f1de4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5ec15dd40 0x7ff5ec0714d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.862+0000 7ff5f1de4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5ec06ce40 con 0x7ff5ec06e9b0 2024-09-16T00:07:47.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.862+0000 7ff5f1de4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff5ec15acd0 con 0x7ff5ec154cd0 2024-09-16T00:07:47.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.862+0000 7ff5f1de4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ec06d120 con 0x7ff5ec15dd40 2024-09-16T00:07:47.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.862+0000 7ff5eb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff5ec154cd0 0x7ff5ec15b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.862+0000 7ff5eb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff5ec154cd0 0x7ff5ec15b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47628/0 (socket says 172.21.15.139:47628) 2024-09-16T00:07:47.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.862+0000 7ff5eb7fe640 1 -- 172.21.15.139:0/2339384493 learned_addr learned my addr 172.21.15.139:0/2339384493 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:47.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.862+0000 7ff5eaffd640 1 --2- 172.21.15.139:0/2339384493 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5ec15dd40 0x7ff5ec0714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.862+0000 7ff5eb7fe640 1 -- 172.21.15.139:0/2339384493 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5ec15dd40 msgr2=0x7ff5ec0714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:47.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.863+0000 7ff5eb7fe640 1 --2- 172.21.15.139:0/2339384493 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5ec15dd40 0x7ff5ec0714d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.863+0000 7ff5eb7fe640 1 -- 172.21.15.139:0/2339384493 >> v1:172.21.15.164:6789/0 conn(0x7ff5ec06e9b0 legacy=0x7ff5ec0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.863+0000 7ff5eb7fe640 1 -- 172.21.15.139:0/2339384493 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec15a960 con 0x7ff5ec154cd0 2024-09-16T00:07:47.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.863+0000 7ff5eaffd640 1 --2- 172.21.15.139:0/2339384493 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5ec15dd40 0x7ff5ec0714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:47.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.863+0000 7ff5eb7fe640 1 --2- 172.21.15.139:0/2339384493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5ec154cd0 0x7ff5ec15b160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec15ae50 tx=0x7ff5dc02a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc03d9767167e183 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:47.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.863+0000 7ff5ea7fc640 1 -- 172.21.15.139:0/2339384493 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc037070 con 0x7ff5ec154cd0 2024-09-16T00:07:47.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.863+0000 7ff5ea7fc640 1 -- 172.21.15.139:0/2339384493 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5dc032640 con 0x7ff5ec154cd0 2024-09-16T00:07:47.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.863+0000 7ff5ea7fc640 1 -- 172.21.15.139:0/2339384493 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc032a10 con 0x7ff5ec154cd0 2024-09-16T00:07:47.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.864+0000 7ff5f1de4640 1 -- 172.21.15.139:0/2339384493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5ec154cd0 msgr2=0x7ff5ec15b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.864+0000 7ff5f1de4640 1 --2- 172.21.15.139:0/2339384493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5ec154cd0 0x7ff5ec15b160 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec15ae50 tx=0x7ff5dc02a2e0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.864+0000 7ff5f1de4640 1 -- 172.21.15.139:0/2339384493 shutdown_connections 2024-09-16T00:07:47.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.864+0000 7ff5f1de4640 1 --2- 172.21.15.139:0/2339384493 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5ec15dd40 0x7ff5ec0714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.864+0000 7ff5f1de4640 1 --2- 172.21.15.139:0/2339384493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5ec154cd0 0x7ff5ec15b160 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.864+0000 7ff5f1de4640 1 -- 172.21.15.139:0/2339384493 >> 172.21.15.139:0/2339384493 conn(0x7ff5ec073e70 msgr2=0x7ff5ec06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:47.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.864+0000 7ff5f1de4640 1 -- 172.21.15.139:0/2339384493 shutdown_connections 2024-09-16T00:07:47.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.864+0000 7ff5f1de4640 1 -- 172.21.15.139:0/2339384493 wait complete. 2024-09-16T00:07:47.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.865+0000 7ff5f1de4640 1 Processor -- start 2024-09-16T00:07:47.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.865+0000 7ff5f1de4640 1 -- start start 2024-09-16T00:07:47.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5f1de4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5ec06e9b0 0x7ff5ec0c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5f1de4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5ec154cd0 0x7ff5ec152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5f1de4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5ec15a2c0 0x7ff5ec1f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5f1de4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec0cb9d0 con 0x7ff5ec06e9b0 2024-09-16T00:07:47.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5f1de4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec15b6a0 con 0x7ff5ec15a2c0 2024-09-16T00:07:47.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5f1de4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ec0c8b50 con 0x7ff5ec154cd0 2024-09-16T00:07:47.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5eb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5ec06e9b0 0x7ff5ec0c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5eb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5ec06e9b0 0x7ff5ec0c7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47642/0 (socket says 172.21.15.139:47642) 2024-09-16T00:07:47.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5eb7fe640 1 -- 172.21.15.139:0/557639033 learned_addr learned my addr 172.21.15.139:0/557639033 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:47.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5eaffd640 1 --2- 172.21.15.139:0/557639033 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5ec154cd0 0x7ff5ec152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5ebfff640 1 --2- 172.21.15.139:0/557639033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5ec15a2c0 0x7ff5ec1f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5eb7fe640 1 -- 172.21.15.139:0/557639033 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5ec154cd0 msgr2=0x7ff5ec152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5eb7fe640 1 --2- 172.21.15.139:0/557639033 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5ec154cd0 0x7ff5ec152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5eb7fe640 1 -- 172.21.15.139:0/557639033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5ec15a2c0 msgr2=0x7ff5ec1f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:47.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5eb7fe640 1 --2- 172.21.15.139:0/557639033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5ec15a2c0 0x7ff5ec1f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:47.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5eb7fe640 1 -- 172.21.15.139:0/557639033 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec1f0180 con 0x7ff5ec06e9b0 2024-09-16T00:07:47.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.866+0000 7ff5ebfff640 1 --2- 172.21.15.139:0/557639033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5ec15a2c0 0x7ff5ec1f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:47.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.867+0000 7ff5eb7fe640 1 --2- 172.21.15.139:0/557639033 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5ec06e9b0 0x7ff5ec0c7280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc02a320 tx=0x7ff5dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:47.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.867+0000 7ff5d3fff640 1 -- 172.21.15.139:0/557639033 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc037040 con 0x7ff5ec06e9b0 2024-09-16T00:07:47.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.867+0000 7ff5d3fff640 1 -- 172.21.15.139:0/557639033 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5dc042070 con 0x7ff5ec06e9b0 2024-09-16T00:07:47.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.867+0000 7ff5d3fff640 1 -- 172.21.15.139:0/557639033 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc030990 con 0x7ff5ec06e9b0 2024-09-16T00:07:47.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.867+0000 7ff5f1de4640 1 -- 172.21.15.139:0/557639033 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ec1f0440 con 0x7ff5ec06e9b0 2024-09-16T00:07:47.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.867+0000 7ff5f1de4640 1 -- 172.21.15.139:0/557639033 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ec1f0a00 con 0x7ff5ec06e9b0 2024-09-16T00:07:47.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.868+0000 7ff5f1de4640 1 -- 172.21.15.139:0/557639033 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5ec0c9ce0 con 0x7ff5ec06e9b0 2024-09-16T00:07:47.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.868+0000 7ff5d3fff640 1 -- 172.21.15.139:0/557639033 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff5dc030b30 con 0x7ff5ec06e9b0 2024-09-16T00:07:47.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.869+0000 7ff5d3fff640 1 --2- 172.21.15.139:0/557639033 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff5d806c2b0 0x7ff5d806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:47.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.869+0000 7ff5d3fff640 1 -- 172.21.15.139:0/557639033 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff5dc0abd10 con 0x7ff5ec06e9b0 2024-09-16T00:07:47.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.869+0000 7ff5eaffd640 1 --2- 172.21.15.139:0/557639033 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff5d806c2b0 0x7ff5d806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:47.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.869+0000 7ff5eaffd640 1 --2- 172.21.15.139:0/557639033 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff5d806c2b0 0x7ff5d806e770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec050250 tx=0x7ff5e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:47.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:47.873+0000 7ff5d3fff640 1 -- 172.21.15.139:0/557639033 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff5dc030e20 con 0x7ff5ec06e9b0 2024-09-16T00:07:48.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.073+0000 7ff5f1de4640 1 -- 172.21.15.139:0/557639033 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5ec0cb7b0 con 0x7ff5ec06e9b0 2024-09-16T00:07:48.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.074+0000 7ff5d3fff640 1 -- 172.21.15.139:0/557639033 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6881 (secure 0 0 0) 0x7ff5dc07a490 con 0x7ff5ec06e9b0 2024-09-16T00:07:48.076 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:48.076 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":8,"btime":"2024-09-16T00:07:38:635384+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2773358731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2773358731},{"type":"v1","addr":"172.21.15.139:6835","nonce":2773358731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4268,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/2919602297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":2919602297},{"type":"v1","addr":"172.21.15.164:6837","nonce":2919602297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3575043661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":3575043661},{"type":"v1","addr":"172.21.15.139:6837","nonce":3575043661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/4113980181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":4113980181},{"type":"v1","addr":"172.21.15.139:6839","nonce":4113980181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/374774889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":374774889},{"type":"v1","addr":"172.21.15.164:6834","nonce":374774889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T00:07:32.949865+0000","modified":"2024-09-16T00:07:38.635380+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.164:6836/4279698936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":4279698936},{"type":"v1","addr":"172.21.15.164:6836","nonce":4279698936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-09-16T00:07:48.078 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 8 2024-09-16T00:07:48.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.076+0000 7ff5f1de4640 1 -- 172.21.15.139:0/557639033 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff5d806c2b0 msgr2=0x7ff5d806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:48.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.076+0000 7ff5f1de4640 1 --2- 172.21.15.139:0/557639033 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff5d806c2b0 0x7ff5d806e770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec050250 tx=0x7ff5e0008040 comp rx=0 tx=0).stop 2024-09-16T00:07:48.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.076+0000 7ff5f1de4640 1 -- 172.21.15.139:0/557639033 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5ec06e9b0 msgr2=0x7ff5ec0c7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:48.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.076+0000 7ff5f1de4640 1 --2- 172.21.15.139:0/557639033 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5ec06e9b0 0x7ff5ec0c7280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc02a320 tx=0x7ff5dc0046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.076+0000 7ff5f1de4640 1 -- 172.21.15.139:0/557639033 shutdown_connections 2024-09-16T00:07:48.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.076+0000 7ff5f1de4640 1 --2- 172.21.15.139:0/557639033 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff5d806c2b0 0x7ff5d806e770 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.076+0000 7ff5f1de4640 1 --2- 172.21.15.139:0/557639033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff5ec15a2c0 0x7ff5ec1f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.076+0000 7ff5f1de4640 1 --2- 172.21.15.139:0/557639033 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff5ec154cd0 0x7ff5ec152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.076+0000 7ff5f1de4640 1 --2- 172.21.15.139:0/557639033 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5ec06e9b0 0x7ff5ec0c7280 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.076+0000 7ff5f1de4640 1 -- 172.21.15.139:0/557639033 >> 172.21.15.139:0/557639033 conn(0x7ff5ec073e70 msgr2=0x7ff5ec0714d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:48.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.076+0000 7ff5f1de4640 1 -- 172.21.15.139:0/557639033 shutdown_connections 2024-09-16T00:07:48.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.076+0000 7ff5f1de4640 1 -- 172.21.15.139:0/557639033 wait complete. 2024-09-16T00:07:48.593 INFO:tasks.ceph.mds.f.smithi164.stderr:2024-09-16T00:07:48.592+0000 7fc3798a4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T00:07:48.692 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T00:07:32.949865+0000', 'modified': '2024-09-16T00:07:38.635380+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4300}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4300': {'gid': 4300, 'name': 'f', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.164:6836/4279698936', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6833', 'nonce': 4279698936}, {'type': 'v1', 'addr': '172.21.15.164:6836', 'nonce': 4279698936}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4300, 'qdb_cluster': [4300]} 2024-09-16T00:07:48.693 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T00:07:48.693 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-09-16T00:07:48.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.796+0000 7f2303853640 1 Processor -- start 2024-09-16T00:07:48.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.796+0000 7f2303853640 1 -- start start 2024-09-16T00:07:48.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.797+0000 7f2303853640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22fc06ea20 0x7f22fc15b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.797+0000 7f2303853640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f22fc0c8b50 0x7f22fc0c3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.797+0000 7f2303853640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22fc06cd40 con 0x7f22fc154bd0 2024-09-16T00:07:48.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.797+0000 7f2303853640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f22fc15a9d0 con 0x7f22fc0c8b50 2024-09-16T00:07:48.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.797+0000 7f2303853640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f22fc06cfc0 con 0x7f22fc06ea20 2024-09-16T00:07:48.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.797+0000 7f2301df0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f22fc0c8b50 0x7f22fc0c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:48.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.797+0000 7f23015ef640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f22fc154bd0 0x7f22fc0ce0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57818/0 (socket says 172.21.15.139:57818) 2024-09-16T00:07:48.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.797+0000 7f2301df0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f22fc0c8b50 0x7f22fc0c3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47644/0 (socket says 172.21.15.139:47644) 2024-09-16T00:07:48.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.797+0000 7f23015ef640 1 -- 172.21.15.139:0/1534831893 learned_addr learned my addr 172.21.15.139:0/1534831893 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:48.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.797+0000 7f2300dee640 1 --2- 172.21.15.139:0/1534831893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22fc06ea20 0x7f22fc15b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:48.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.797+0000 7f2301df0640 1 -- 172.21.15.139:0/1534831893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22fc06ea20 msgr2=0x7f22fc15b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:48.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.797+0000 7f2301df0640 1 --2- 172.21.15.139:0/1534831893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22fc06ea20 0x7f22fc15b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.797+0000 7f2301df0640 1 -- 172.21.15.139:0/1534831893 >> v1:172.21.15.139:6789/0 conn(0x7f22fc154bd0 legacy=0x7f22fc0ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:48.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.797+0000 7f22f3fff640 1 -- 172.21.15.139:0/1534831893 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3927645532 0 0) 0x7f22f8001110 con 0x7f22fc154bd0 2024-09-16T00:07:48.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.797+0000 7f2301df0640 1 -- 172.21.15.139:0/1534831893 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22fc15ac70 con 0x7f22fc0c8b50 2024-09-16T00:07:48.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.797+0000 7f2300dee640 1 --2- 172.21.15.139:0/1534831893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22fc06ea20 0x7f22fc15b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:48.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.798+0000 7f2301df0640 1 --2- 172.21.15.139:0/1534831893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22fc0c8b50 0x7f22fc0c3170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009d90 tx=0x7f22f402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bad320980adae10 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:48.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.798+0000 7f22f3fff640 1 -- 172.21.15.139:0/1534831893 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f4007740 con 0x7f22fc0c8b50 2024-09-16T00:07:48.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.798+0000 7f22f3fff640 1 -- 172.21.15.139:0/1534831893 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f40078e0 con 0x7f22fc0c8b50 2024-09-16T00:07:48.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.798+0000 7f22f3fff640 1 -- 172.21.15.139:0/1534831893 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f4007cb0 con 0x7f22fc0c8b50 2024-09-16T00:07:48.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.799+0000 7f2303853640 1 -- 172.21.15.139:0/1534831893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22fc0c8b50 msgr2=0x7f22fc0c3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:48.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.799+0000 7f2303853640 1 --2- 172.21.15.139:0/1534831893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22fc0c8b50 0x7f22fc0c3170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009d90 tx=0x7f22f402a660 comp rx=0 tx=0).stop 2024-09-16T00:07:48.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.799+0000 7f2303853640 1 -- 172.21.15.139:0/1534831893 shutdown_connections 2024-09-16T00:07:48.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.799+0000 7f2303853640 1 --2- 172.21.15.139:0/1534831893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22fc0c8b50 0x7f22fc0c3170 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.799+0000 7f2303853640 1 --2- 172.21.15.139:0/1534831893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22fc06ea20 0x7f22fc15b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.799+0000 7f2303853640 1 -- 172.21.15.139:0/1534831893 >> 172.21.15.139:0/1534831893 conn(0x7f22fc06c150 msgr2=0x7f22fc067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:48.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.799+0000 7f2303853640 1 -- 172.21.15.139:0/1534831893 shutdown_connections 2024-09-16T00:07:48.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.799+0000 7f2303853640 1 -- 172.21.15.139:0/1534831893 wait complete. 2024-09-16T00:07:48.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.800+0000 7f2303853640 1 Processor -- start 2024-09-16T00:07:48.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.800+0000 7f2303853640 1 -- start start 2024-09-16T00:07:48.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f2303853640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22fc06ea20 0x7f22fc0ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f2303853640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22fc0c8b50 0x7f22fc0cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f2303853640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22fc154bd0 0x7f22fc0ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f2303853640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc06d2a0 con 0x7f22fc06ea20 2024-09-16T00:07:48.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f2303853640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc154fb0 con 0x7f22fc154bd0 2024-09-16T00:07:48.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f2303853640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f22fc06d420 con 0x7f22fc0c8b50 2024-09-16T00:07:48.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f23015ef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22fc06ea20 0x7f22fc0ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:48.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f23015ef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22fc06ea20 0x7f22fc0ca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47658/0 (socket says 172.21.15.139:47658) 2024-09-16T00:07:48.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f23015ef640 1 -- 172.21.15.139:0/143792896 learned_addr learned my addr 172.21.15.139:0/143792896 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:48.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f23015ef640 1 -- 172.21.15.139:0/143792896 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22fc0c8b50 msgr2=0x7f22fc0cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:48.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f23015ef640 1 --2- 172.21.15.139:0/143792896 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22fc0c8b50 0x7f22fc0cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f23015ef640 1 -- 172.21.15.139:0/143792896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22fc154bd0 msgr2=0x7f22fc0ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:48.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f23015ef640 1 --2- 172.21.15.139:0/143792896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22fc154bd0 0x7f22fc0ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f23015ef640 1 -- 172.21.15.139:0/143792896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22fc0cd750 con 0x7f22fc06ea20 2024-09-16T00:07:48.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f23015ef640 1 --2- 172.21.15.139:0/143792896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22fc06ea20 0x7f22fc0ca390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f22f800c430 tx=0x7f22f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:48.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f22f1ffb640 1 -- 172.21.15.139:0/143792896 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f80074e0 con 0x7f22fc06ea20 2024-09-16T00:07:48.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f22f1ffb640 1 -- 172.21.15.139:0/143792896 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f80047e0 con 0x7f22fc06ea20 2024-09-16T00:07:48.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f2303853640 1 -- 172.21.15.139:0/143792896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22fc0cda30 con 0x7f22fc06ea20 2024-09-16T00:07:48.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.801+0000 7f22f1ffb640 1 -- 172.21.15.139:0/143792896 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8004bd0 con 0x7f22fc06ea20 2024-09-16T00:07:48.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.802+0000 7f2303853640 1 -- 172.21.15.139:0/143792896 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22fc0c3cb0 con 0x7f22fc06ea20 2024-09-16T00:07:48.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.803+0000 7f22f1ffb640 1 -- 172.21.15.139:0/143792896 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f22f8011070 con 0x7f22fc06ea20 2024-09-16T00:07:48.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.803+0000 7f22f1ffb640 1 --2- 172.21.15.139:0/143792896 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f22e006c2b0 0x7f22e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.803+0000 7f22f1ffb640 1 -- 172.21.15.139:0/143792896 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f22f808b010 con 0x7f22fc06ea20 2024-09-16T00:07:48.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.803+0000 7f2300dee640 1 --2- 172.21.15.139:0/143792896 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f22e006c2b0 0x7f22e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:48.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.804+0000 7f2300dee640 1 --2- 172.21.15.139:0/143792896 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f22e006c2b0 0x7f22e006e770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f22ec006fd0 tx=0x7f22ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:48.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.804+0000 7f23065e9740 1 Processor -- start 2024-09-16T00:07:48.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.804+0000 7f23065e9740 1 -- start start 2024-09-16T00:07:48.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.805+0000 7f23065e9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8f750 0x55cf94fc79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.805+0000 7f23065e9740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8cf80 0x55cf94fc7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.805+0000 7f23065e9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8fb30 0x55cf94f8ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.805+0000 7f23065e9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cf94ef0520 con 0x55cf94f8f750 2024-09-16T00:07:48.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.805+0000 7f23065e9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cf94e983e0 con 0x55cf94f8fb30 2024-09-16T00:07:48.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.805+0000 7f23065e9740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x55cf94eec4e0 con 0x55cf94f8cf80 2024-09-16T00:07:48.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.805+0000 7f23015ef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8f750 0x55cf94fc79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:48.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.805+0000 7f23015ef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8f750 0x55cf94fc79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47670/0 (socket says 172.21.15.139:47670) 2024-09-16T00:07:48.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.805+0000 7f23015ef640 1 -- 172.21.15.139:0/3188440369 learned_addr learned my addr 172.21.15.139:0/3188440369 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:48.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.805+0000 7f23015ef640 1 -- 172.21.15.139:0/3188440369 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8cf80 msgr2=0x55cf94fc7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:48.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.805+0000 7f23015ef640 1 --2- 172.21.15.139:0/3188440369 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8cf80 0x55cf94fc7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.805+0000 7f23015ef640 1 -- 172.21.15.139:0/3188440369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8fb30 msgr2=0x55cf94f8ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:48.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.805+0000 7f23015ef640 1 --2- 172.21.15.139:0/3188440369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8fb30 0x55cf94f8ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.805+0000 7f23015ef640 1 -- 172.21.15.139:0/3188440369 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf94eb32f0 con 0x55cf94f8f750 2024-09-16T00:07:48.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.806+0000 7f23015ef640 1 --2- 172.21.15.139:0/3188440369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8f750 0x55cf94fc79e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f22f8089540 tx=0x7f22f8058b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1bce549c9f28d05 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:48.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.806+0000 7f22e6ffd640 1 -- 172.21.15.139:0/3188440369 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f805d4c0 con 0x55cf94f8f750 2024-09-16T00:07:48.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.806+0000 7f22e6ffd640 1 -- 172.21.15.139:0/3188440369 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f8061cf0 con 0x55cf94f8f750 2024-09-16T00:07:48.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.806+0000 7f22e6ffd640 1 -- 172.21.15.139:0/3188440369 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8004d70 con 0x55cf94f8f750 2024-09-16T00:07:48.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.806+0000 7f23065e9740 1 -- 172.21.15.139:0/3188440369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8f750 msgr2=0x55cf94fc79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:48.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.806+0000 7f23065e9740 1 --2- 172.21.15.139:0/3188440369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8f750 0x55cf94fc79e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f22f8089540 tx=0x7f22f8058b70 comp rx=0 tx=0).stop 2024-09-16T00:07:48.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.806+0000 7f23065e9740 1 -- 172.21.15.139:0/3188440369 shutdown_connections 2024-09-16T00:07:48.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.806+0000 7f23065e9740 1 --2- 172.21.15.139:0/3188440369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8fb30 0x55cf94f8ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.806+0000 7f23065e9740 1 --2- 172.21.15.139:0/3188440369 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8cf80 0x55cf94fc7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.806+0000 7f23065e9740 1 --2- 172.21.15.139:0/3188440369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8f750 0x55cf94fc79e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.806+0000 7f23065e9740 1 -- 172.21.15.139:0/3188440369 >> 172.21.15.139:0/3188440369 conn(0x55cf94f8de70 msgr2=0x55cf94f8e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:48.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.806+0000 7f23065e9740 1 -- 172.21.15.139:0/3188440369 shutdown_connections 2024-09-16T00:07:48.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.806+0000 7f23065e9740 1 -- 172.21.15.139:0/3188440369 wait complete. 2024-09-16T00:07:48.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.807+0000 7f23065e9740 1 -- start start 2024-09-16T00:07:48.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.807+0000 7f23065e9740 1 Processor -- start 2024-09-16T00:07:48.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23065e9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8cf80 0x55cf94fdf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23065e9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8f750 0x55cf94fbf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23065e9740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8fb30 0x55cf94fac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23065e9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cf94f8efe0 con 0x55cf94f8cf80 2024-09-16T00:07:48.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23065e9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cf94e79450 con 0x55cf94f8f750 2024-09-16T00:07:48.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23065e9740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x55cf94fc7550 con 0x55cf94f8fb30 2024-09-16T00:07:48.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23065e9740 10 client.0 _ll_register_callbacks cb 0x55cf94c02ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:07:48.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23015ef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8cf80 0x55cf94fdf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:48.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23015ef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8cf80 0x55cf94fdf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47672/0 (socket says 172.21.15.139:47672) 2024-09-16T00:07:48.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23015ef640 1 -- 172.21.15.139:0/2209027058 learned_addr learned my addr 172.21.15.139:0/2209027058 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:48.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f2300dee640 1 --2- 172.21.15.139:0/2209027058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8f750 0x55cf94fbf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:48.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f2301df0640 1 --2- 172.21.15.139:0/2209027058 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8fb30 0x55cf94fac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:48.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23015ef640 1 -- 172.21.15.139:0/2209027058 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8fb30 msgr2=0x55cf94fac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:48.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23015ef640 1 --2- 172.21.15.139:0/2209027058 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8fb30 0x55cf94fac9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23015ef640 1 -- 172.21.15.139:0/2209027058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8f750 msgr2=0x55cf94fbf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:48.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23015ef640 1 --2- 172.21.15.139:0/2209027058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8f750 0x55cf94fbf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23015ef640 1 -- 172.21.15.139:0/2209027058 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf94c032d0 con 0x55cf94f8cf80 2024-09-16T00:07:48.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23015ef640 1 --2- 172.21.15.139:0/2209027058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8cf80 0x55cf94fdf220 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f22f8058e10 tx=0x7f22f8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58881c053ddca8b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:48.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f22dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T00:07:48.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f22dbfff640 1 -- 172.21.15.139:0/2209027058 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f805d4c0 con 0x55cf94f8cf80 2024-09-16T00:07:48.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f22dbfff640 1 -- 172.21.15.139:0/2209027058 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f22f8004d70 con 0x55cf94f8cf80 2024-09-16T00:07:48.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23065e9740 1 -- 172.21.15.139:0/2209027058 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22f8061cf0 con 0x55cf94f8cf80 2024-09-16T00:07:48.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f22dbfff640 1 -- 172.21.15.139:0/2209027058 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f22f807f050 con 0x55cf94f8cf80 2024-09-16T00:07:48.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23065e9740 10 client.4620 fetch_fsmap learned FSMap version 8 2024-09-16T00:07:48.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.808+0000 7f23065e9740 1 -- 172.21.15.139:0/2209027058 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55cf94ccd6b0 con 0x55cf94f8cf80 2024-09-16T00:07:48.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.809+0000 7f22dbfff640 1 -- 172.21.15.139:0/2209027058 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22f8061cf0 con 0x55cf94f8cf80 2024-09-16T00:07:48.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.809+0000 7f22dbfff640 1 -- 172.21.15.139:0/2209027058 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 8) ==== 4712+0+0 (secure 0 0 0) 0x7f22f8082020 con 0x55cf94f8cf80 2024-09-16T00:07:48.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.809+0000 7f23065e9740 10 client.4620 fetch_fsmap finished waiting for FSMap version 8 2024-09-16T00:07:48.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.809+0000 7f23065e9740 10 client.4620 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T00:07:48.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.809+0000 7f23065e9740 20 client.4620 populate_metadata read hostname 'smithi139' 2024-09-16T00:07:48.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.809+0000 7f23065e9740 1 --2- 172.21.15.139:0/2209027058 >> [v2:172.21.15.164:6833/4279698936,v1:172.21.15.164:6836/4279698936] conn(0x55cf94fe24f0 0x55cf94fed3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.809+0000 7f23065e9740 4 client.4620 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:07:48.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.809+0000 7f23065e9740 1 -- 172.21.15.139:0/2209027058 --> [v2:172.21.15.164:6833/4279698936,v1:172.21.15.164:6836/4279698936] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf94efdc50 con 0x55cf94fe24f0 2024-09-16T00:07:48.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.809+0000 7f2300dee640 1 --2- 172.21.15.139:0/2209027058 >> [v2:172.21.15.164:6833/4279698936,v1:172.21.15.164:6836/4279698936] conn(0x55cf94fe24f0 0x55cf94fed3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:48.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.809+0000 7f2300dee640 1 --2- 172.21.15.139:0/2209027058 >> [v2:172.21.15.164:6833/4279698936,v1:172.21.15.164:6836/4279698936] conn(0x55cf94fe24f0 0x55cf94fed3f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efb3dff9559f32a2 server_cookie=9015dab4d53df51 in_seq=0 out_seq=0 2024-09-16T00:07:48.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.809+0000 7f22dbfff640 10 client.4620 ms_handle_connect on v2:172.21.15.164:6833/4279698936 2024-09-16T00:07:48.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.810+0000 7f22dbfff640 1 -- 172.21.15.139:0/2209027058 <== mds.0 v2:172.21.15.164:6833/4279698936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf94efdc50 con 0x55cf94fe24f0 2024-09-16T00:07:48.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.810+0000 7f22dbfff640 10 client.4620 handle_command_reply: tid=0 2024-09-16T00:07:48.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.810+0000 7f23065e9740 1 client.4620 shutdown 2024-09-16T00:07:48.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.810+0000 7f23065e9740 1 -- 172.21.15.139:0/2209027058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8cf80 msgr2=0x55cf94fdf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:48.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.810+0000 7f23065e9740 1 --2- 172.21.15.139:0/2209027058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8cf80 0x55cf94fdf220 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f22f8058e10 tx=0x7f22f8000ed0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.810+0000 7f23065e9740 1 -- 172.21.15.139:0/2209027058 shutdown_connections 2024-09-16T00:07:48.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.810+0000 7f23065e9740 1 --2- 172.21.15.139:0/2209027058 >> [v2:172.21.15.164:6833/4279698936,v1:172.21.15.164:6836/4279698936] conn(0x55cf94fe24f0 0x55cf94fed3f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.810+0000 7f23065e9740 1 --2- 172.21.15.139:0/2209027058 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8fb30 0x55cf94fac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.810+0000 7f22dbfff640 0 client.4620 ms_handle_reset on v2:172.21.15.164:6833/4279698936 2024-09-16T00:07:48.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.810+0000 7f23065e9740 1 --2- 172.21.15.139:0/2209027058 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8f750 0x55cf94fbf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.810+0000 7f23065e9740 1 --2- 172.21.15.139:0/2209027058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8cf80 0x55cf94fdf220 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.810+0000 7f23065e9740 1 -- 172.21.15.139:0/2209027058 >> 172.21.15.139:0/2209027058 conn(0x55cf94f8de70 msgr2=0x55cf94fc79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:48.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.810+0000 7f23065e9740 1 -- 172.21.15.139:0/2209027058 shutdown_connections 2024-09-16T00:07:48.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.810+0000 7f23065e9740 1 -- 172.21.15.139:0/2209027058 wait complete. 2024-09-16T00:07:48.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.811+0000 7f23065e9740 20 client.4620 trim_cache size 0 max 16384 2024-09-16T00:07:48.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.828+0000 7f23065e9740 1 Processor -- start 2024-09-16T00:07:48.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.828+0000 7f23065e9740 1 -- start start 2024-09-16T00:07:48.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f23065e9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8cf80 0x55cf94f8d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f23065e9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8f750 0x55cf94f8d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f23065e9740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8fb30 0x55cf94fb4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f23065e9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22fc1f4780 con 0x55cf94f8f750 2024-09-16T00:07:48.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f23065e9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cf94e79450 con 0x55cf94f8cf80 2024-09-16T00:07:48.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f23065e9740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x55cf94f8efe0 con 0x55cf94f8fb30 2024-09-16T00:07:48.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f2300dee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8f750 0x55cf94f8d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:48.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f2300dee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8f750 0x55cf94f8d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47676/0 (socket says 172.21.15.139:47676) 2024-09-16T00:07:48.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f2300dee640 1 -- 172.21.15.139:0/2832878485 learned_addr learned my addr 172.21.15.139:0/2832878485 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:48.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f2300dee640 1 -- 172.21.15.139:0/2832878485 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8fb30 msgr2=0x55cf94fb4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:48.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f23015ef640 1 --2- 172.21.15.139:0/2832878485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8cf80 0x55cf94f8d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:48.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f2301df0640 1 --2- 172.21.15.139:0/2832878485 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8fb30 0x55cf94fb4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:48.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f2300dee640 1 --2- 172.21.15.139:0/2832878485 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8fb30 0x55cf94fb4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f2300dee640 1 -- 172.21.15.139:0/2832878485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8cf80 msgr2=0x55cf94f8d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:48.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f2300dee640 1 --2- 172.21.15.139:0/2832878485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8cf80 0x55cf94f8d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f2300dee640 1 -- 172.21.15.139:0/2832878485 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf94fb10f0 con 0x55cf94f8f750 2024-09-16T00:07:48.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f23015ef640 1 --2- 172.21.15.139:0/2832878485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8cf80 0x55cf94f8d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:48.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f2300dee640 1 --2- 172.21.15.139:0/2832878485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8f750 0x55cf94f8d8c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0cb020 tx=0x7f22ec00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32cfb010f757a64 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:48.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f22e67fc640 1 -- 172.21.15.139:0/2832878485 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec013e20 con 0x55cf94f8f750 2024-09-16T00:07:48.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f22e67fc640 1 -- 172.21.15.139:0/2832878485 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ec029470 con 0x55cf94f8f750 2024-09-16T00:07:48.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.829+0000 7f22e67fc640 1 -- 172.21.15.139:0/2832878485 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec029840 con 0x55cf94f8f750 2024-09-16T00:07:48.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.830+0000 7f23065e9740 1 -- 172.21.15.139:0/2832878485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8f750 msgr2=0x55cf94f8d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:48.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.830+0000 7f23065e9740 1 --2- 172.21.15.139:0/2832878485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8f750 0x55cf94f8d8c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0cb020 tx=0x7f22ec00f000 comp rx=0 tx=0).stop 2024-09-16T00:07:48.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.830+0000 7f23065e9740 1 -- 172.21.15.139:0/2832878485 shutdown_connections 2024-09-16T00:07:48.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.830+0000 7f23065e9740 1 --2- 172.21.15.139:0/2832878485 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8fb30 0x55cf94fb4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.830+0000 7f23065e9740 1 --2- 172.21.15.139:0/2832878485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8f750 0x55cf94f8d8c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.830+0000 7f23065e9740 1 --2- 172.21.15.139:0/2832878485 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8cf80 0x55cf94f8d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.830+0000 7f23065e9740 1 -- 172.21.15.139:0/2832878485 >> 172.21.15.139:0/2832878485 conn(0x55cf94f8de70 msgr2=0x55cf94fc85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:48.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.830+0000 7f23065e9740 1 -- 172.21.15.139:0/2832878485 shutdown_connections 2024-09-16T00:07:48.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.830+0000 7f23065e9740 1 -- 172.21.15.139:0/2832878485 wait complete. 2024-09-16T00:07:48.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.830+0000 7f23065e9740 1 -- start start 2024-09-16T00:07:48.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.830+0000 7f23065e9740 1 Processor -- start 2024-09-16T00:07:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.831+0000 7f23065e9740 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8cf80 0x55cf94f886c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.831+0000 7f23065e9740 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8f750 0x55cf94fb4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.831+0000 7f23065e9740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8fb30 0x55cf94fc5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.831+0000 7f23065e9740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cf94fb4a20 con 0x55cf94f8fb30 2024-09-16T00:07:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.831+0000 7f23065e9740 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x55cf94fc0360 con 0x55cf94f8f750 2024-09-16T00:07:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.831+0000 7f23065e9740 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x55cf94fad070 con 0x55cf94f8cf80 2024-09-16T00:07:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.831+0000 7f23065e9740 10 client.0 _ll_register_callbacks cb 0x7f22f8058cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:07:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.831+0000 7f2301df0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8fb30 0x55cf94fc5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.831+0000 7f2301df0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8fb30 0x55cf94fc5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47692/0 (socket says 172.21.15.139:47692) 2024-09-16T00:07:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.831+0000 7f2301df0640 1 -- 172.21.15.139:0/2428775317 learned_addr learned my addr 172.21.15.139:0/2428775317 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:48.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.831+0000 7f2300dee640 1 --2- 172.21.15.139:0/2428775317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8f750 0x55cf94fb4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.831+0000 7f23015ef640 1 --2- 172.21.15.139:0/2428775317 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8cf80 0x55cf94f886c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f2301df0640 1 -- 172.21.15.139:0/2428775317 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8cf80 msgr2=0x55cf94f886c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f2301df0640 1 --2- 172.21.15.139:0/2428775317 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8cf80 0x55cf94f886c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f2301df0640 1 -- 172.21.15.139:0/2428775317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8f750 msgr2=0x55cf94fb4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f2301df0640 1 --2- 172.21.15.139:0/2428775317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8f750 0x55cf94fb4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f2301df0640 1 -- 172.21.15.139:0/2428775317 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf94f88c00 con 0x55cf94f8fb30 2024-09-16T00:07:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f2301df0640 1 --2- 172.21.15.139:0/2428775317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8fb30 0x55cf94fc5cb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009eb0 tx=0x7f22f4007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c24106b16fc62f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f22dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-09-16T00:07:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f22dbfff640 1 -- 172.21.15.139:0/2428775317 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f4009450 con 0x55cf94f8fb30 2024-09-16T00:07:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f22dbfff640 1 -- 172.21.15.139:0/2428775317 <== mon.0 v2:172.21.15.139:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f22f4047cf0 con 0x55cf94f8fb30 2024-09-16T00:07:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f23065e9740 1 -- 172.21.15.139:0/2428775317 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22ec029470 con 0x55cf94f8fb30 2024-09-16T00:07:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f22dbfff640 1 -- 172.21.15.139:0/2428775317 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f22f404e050 con 0x55cf94f8fb30 2024-09-16T00:07:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f23065e9740 10 client.4626 fetch_fsmap learned FSMap version 8 2024-09-16T00:07:48.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f23065e9740 1 -- 172.21.15.139:0/2428775317 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55cf94f8cbd0 con 0x55cf94f8fb30 2024-09-16T00:07:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f22dbfff640 1 -- 172.21.15.139:0/2428775317 <== mon.0 v2:172.21.15.139:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ec029470 con 0x55cf94f8fb30 2024-09-16T00:07:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f22dbfff640 1 -- 172.21.15.139:0/2428775317 <== mon.0 v2:172.21.15.139:3300/0 5 ==== fsmap(e 8) ==== 4712+0+0 (secure 0 0 0) 0x7f22f4051020 con 0x55cf94f8fb30 2024-09-16T00:07:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f23065e9740 10 client.4626 fetch_fsmap finished waiting for FSMap version 8 2024-09-16T00:07:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f23065e9740 10 client.4626 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-16T00:07:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f23065e9740 20 client.4626 populate_metadata read hostname 'smithi139' 2024-09-16T00:07:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f23065e9740 1 --2- 172.21.15.139:0/2428775317 >> [v2:172.21.15.164:6833/4279698936,v1:172.21.15.164:6836/4279698936] conn(0x55cf9504afa0 0x55cf9504b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f23065e9740 4 client.4626 mds_command: new command op to 4300 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:07:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.832+0000 7f23065e9740 1 -- 172.21.15.139:0/2428775317 --> [v2:172.21.15.164:6833/4279698936,v1:172.21.15.164:6836/4279698936] -- command(tid 0: {"prefix": "status"}) -- 0x55cf94ef7fa0 con 0x55cf9504afa0 2024-09-16T00:07:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f23015ef640 1 --2- 172.21.15.139:0/2428775317 >> [v2:172.21.15.164:6833/4279698936,v1:172.21.15.164:6836/4279698936] conn(0x55cf9504afa0 0x55cf9504b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f22dbfff640 10 client.4626 ms_handle_connect on v2:172.21.15.164:6833/4279698936 2024-09-16T00:07:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f23015ef640 1 --2- 172.21.15.139:0/2428775317 >> [v2:172.21.15.164:6833/4279698936,v1:172.21.15.164:6836/4279698936] conn(0x55cf9504afa0 0x55cf9504b380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1814b8f190d5b50 server_cookie=597e12ba62083bce in_seq=0 out_seq=0 2024-09-16T00:07:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f22dbfff640 1 -- 172.21.15.139:0/2428775317 <== mds.0 v2:172.21.15.164:6833/4279698936 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55cf94ef7fa0 con 0x55cf9504afa0 2024-09-16T00:07:48.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f22dbfff640 10 client.4626 handle_command_reply: tid=0 2024-09-16T00:07:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f23065e9740 1 client.4626 shutdown 2024-09-16T00:07:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f23065e9740 1 -- 172.21.15.139:0/2428775317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8fb30 msgr2=0x55cf94fc5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f23065e9740 1 --2- 172.21.15.139:0/2428775317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8fb30 0x55cf94fc5cb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f22f4009eb0 tx=0x7f22f4007cb0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f23065e9740 1 -- 172.21.15.139:0/2428775317 shutdown_connections 2024-09-16T00:07:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f23065e9740 1 --2- 172.21.15.139:0/2428775317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf94f8fb30 0x55cf94fc5cb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f23065e9740 1 --2- 172.21.15.139:0/2428775317 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x55cf94f8f750 0x55cf94fb4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f23065e9740 1 --2- 172.21.15.139:0/2428775317 >> [v2:172.21.15.164:6833/4279698936,v1:172.21.15.164:6836/4279698936] conn(0x55cf9504afa0 0x55cf9504b380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f23065e9740 1 --2- 172.21.15.139:0/2428775317 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x55cf94f8cf80 0x55cf94f886c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f22dbfff640 0 client.4626 ms_handle_reset on v2:172.21.15.164:6833/4279698936 2024-09-16T00:07:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f23065e9740 1 -- 172.21.15.139:0/2428775317 >> 172.21.15.139:0/2428775317 conn(0x55cf94f8de70 msgr2=0x55cf94fe8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f23065e9740 1 -- 172.21.15.139:0/2428775317 shutdown_connections 2024-09-16T00:07:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.833+0000 7f23065e9740 1 -- 172.21.15.139:0/2428775317 wait complete. 2024-09-16T00:07:48.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.834+0000 7f23065e9740 20 client.4626 trim_cache size 0 max 16384 2024-09-16T00:07:48.842 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:07:48.842 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_fsid": "2d3366e5-6a59-4b59-93b2-615e72ac7015", 2024-09-16T00:07:48.842 INFO:teuthology.orchestra.run.smithi139.stdout: "whoami": 0, 2024-09-16T00:07:48.842 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 4300, 2024-09-16T00:07:48.842 INFO:teuthology.orchestra.run.smithi139.stdout: "want_state": "up:active", 2024-09-16T00:07:48.842 INFO:teuthology.orchestra.run.smithi139.stdout: "state": "up:active", 2024-09-16T00:07:48.842 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs", 2024-09-16T00:07:48.842 INFO:teuthology.orchestra.run.smithi139.stdout: "rank_uptime": 15.223114142, 2024-09-16T00:07:48.842 INFO:teuthology.orchestra.run.smithi139.stdout: "mdsmap_epoch": 8, 2024-09-16T00:07:48.843 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch": 22, 2024-09-16T00:07:48.843 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_epoch_barrier": 22, 2024-09-16T00:07:48.843 INFO:teuthology.orchestra.run.smithi139.stdout: "uptime": 21.238087514 2024-09-16T00:07:48.843 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:07:48.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.834+0000 7f2303853640 1 -- 172.21.15.139:0/143792896 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f22e006c2b0 msgr2=0x7f22e006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:48.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.834+0000 7f2303853640 1 --2- 172.21.15.139:0/143792896 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f22e006c2b0 0x7f22e006e770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f22ec006fd0 tx=0x7f22ec008040 comp rx=0 tx=0).stop 2024-09-16T00:07:48.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.834+0000 7f2303853640 1 -- 172.21.15.139:0/143792896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22fc06ea20 msgr2=0x7f22fc0ca390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:48.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.834+0000 7f2303853640 1 --2- 172.21.15.139:0/143792896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22fc06ea20 0x7f22fc0ca390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f22f800c430 tx=0x7f22f800c8f0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.834+0000 7f2303853640 1 -- 172.21.15.139:0/143792896 shutdown_connections 2024-09-16T00:07:48.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.834+0000 7f2303853640 1 --2- 172.21.15.139:0/143792896 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f22e006c2b0 0x7f22e006e770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.834+0000 7f2303853640 1 --2- 172.21.15.139:0/143792896 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22fc154bd0 0x7f22fc0ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.834+0000 7f2303853640 1 --2- 172.21.15.139:0/143792896 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22fc0c8b50 0x7f22fc0cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.834+0000 7f2303853640 1 --2- 172.21.15.139:0/143792896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22fc06ea20 0x7f22fc0ca390 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:48.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.834+0000 7f2303853640 1 -- 172.21.15.139:0/143792896 >> 172.21.15.139:0/143792896 conn(0x7f22fc06c150 msgr2=0x7f22fc1517e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:48.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.834+0000 7f2303853640 1 -- 172.21.15.139:0/143792896 shutdown_connections 2024-09-16T00:07:48.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:48.834+0000 7f2303853640 1 -- 172.21.15.139:0/143792896 wait complete. 2024-09-16T00:07:48.846 INFO:teuthology.run_tasks:Running task sequential... 2024-09-16T00:07:48.934 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-16T00:07:48.935 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-16T00:07:48.935 INFO:teuthology.task.print:Enabling mgr modules 2024-09-16T00:07:48.935 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-16T00:07:48.980 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-16T00:07:48.980 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-16T00:07:48.980 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-16T00:07:48.981 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T00:07:48.981 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T00:07:48.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:07:48.981 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-16T00:07:48.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:07:48.998 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link delete ceph-brx 2024-09-16T00:07:49.064 INFO:teuthology.orchestra.run.smithi139.stderr:Cannot find device "ceph-brx" 2024-09-16T00:07:49.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:07:49.066 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-16T00:07:49.066 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-16T00:07:49.066 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-16T00:07:49.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.203+0000 7f53fed59640 1 Processor -- start 2024-09-16T00:07:49.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.203+0000 7f53fed59640 1 -- start start 2024-09-16T00:07:49.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.204+0000 7f53fed59640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53f8072c30 0x7f53f80c6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:49.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.204+0000 7f53fed59640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f53f8073010 0x7f53f80c3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:49.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.204+0000 7f53fed59640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f8158550 con 0x7f53f815aeb0 2024-09-16T00:07:49.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.204+0000 7f53fed59640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f53f81576f0 con 0x7f53f8073010 2024-09-16T00:07:49.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.204+0000 7f53fed59640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f53f81579d0 con 0x7f53f8072c30 2024-09-16T00:07:49.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.204+0000 7f53f7fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f53f8073010 0x7f53f80c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:49.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.204+0000 7f53f7fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f53f8073010 0x7f53f80c3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47696/0 (socket says 172.21.15.139:47696) 2024-09-16T00:07:49.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.204+0000 7f53f7fff640 1 -- 172.21.15.139:0/428062840 learned_addr learned my addr 172.21.15.139:0/428062840 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:49.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.204+0000 7f53f7fff640 1 -- 172.21.15.139:0/428062840 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53f8072c30 msgr2=0x7f53f80c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:49.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.204+0000 7f53fcaf5640 1 --2- 172.21.15.139:0/428062840 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53f8072c30 0x7f53f80c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:49.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.204+0000 7f53f7fff640 1 --2- 172.21.15.139:0/428062840 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53f8072c30 0x7f53f80c6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.204+0000 7f53f7fff640 1 -- 172.21.15.139:0/428062840 >> v1:172.21.15.164:6789/0 conn(0x7f53f815aeb0 legacy=0x7f53f815b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.204+0000 7f53f7fff640 1 -- 172.21.15.139:0/428062840 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f8157cb0 con 0x7f53f8073010 2024-09-16T00:07:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.205+0000 7f53f7fff640 1 --2- 172.21.15.139:0/428062840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f8073010 0x7f53f80c3590 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009870 tx=0x7f53e402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d7b4179d857f14 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.205+0000 7f53f77fe640 1 -- 172.21.15.139:0/428062840 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4037070 con 0x7f53f8073010 2024-09-16T00:07:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.205+0000 7f53f77fe640 1 -- 172.21.15.139:0/428062840 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e4032440 con 0x7f53f8073010 2024-09-16T00:07:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.205+0000 7f53f77fe640 1 -- 172.21.15.139:0/428062840 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4032810 con 0x7f53f8073010 2024-09-16T00:07:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.205+0000 7f53fed59640 1 -- 172.21.15.139:0/428062840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f8073010 msgr2=0x7f53f80c3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.205+0000 7f53fed59640 1 --2- 172.21.15.139:0/428062840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f8073010 0x7f53f80c3590 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009870 tx=0x7f53e402a2a0 comp rx=0 tx=0).stop 2024-09-16T00:07:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.205+0000 7f53fed59640 1 -- 172.21.15.139:0/428062840 shutdown_connections 2024-09-16T00:07:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.206+0000 7f53fed59640 1 --2- 172.21.15.139:0/428062840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f8073010 0x7f53f80c3590 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.206+0000 7f53fed59640 1 --2- 172.21.15.139:0/428062840 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53f8072c30 0x7f53f80c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:49.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.206+0000 7f53fed59640 1 -- 172.21.15.139:0/428062840 >> 172.21.15.139:0/428062840 conn(0x7f53f8067950 msgr2=0x7f53f8067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:49.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.206+0000 7f53fed59640 1 -- 172.21.15.139:0/428062840 shutdown_connections 2024-09-16T00:07:49.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.206+0000 7f53fed59640 1 -- 172.21.15.139:0/428062840 wait complete. 2024-09-16T00:07:49.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.207+0000 7f53fed59640 1 Processor -- start 2024-09-16T00:07:49.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.207+0000 7f53fed59640 1 -- start start 2024-09-16T00:07:49.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.207+0000 7f53fed59640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53f8072c30 0x7f53f8146c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:49.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.207+0000 7f53fed59640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f8073010 0x7f53f80cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:49.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.207+0000 7f53fed59640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f815aeb0 0x7f53f81f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:49.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.207+0000 7f53fed59640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53f80c3ad0 con 0x7f53f8073010 2024-09-16T00:07:49.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.207+0000 7f53fed59640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8158270 con 0x7f53f815aeb0 2024-09-16T00:07:49.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.207+0000 7f53fed59640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f53f80c6b20 con 0x7f53f8072c30 2024-09-16T00:07:49.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.208+0000 7f53f7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f8073010 0x7f53f80cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:49.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.208+0000 7f53f7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f8073010 0x7f53f80cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47706/0 (socket says 172.21.15.139:47706) 2024-09-16T00:07:49.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.208+0000 7f53f7fff640 1 -- 172.21.15.139:0/1249551449 learned_addr learned my addr 172.21.15.139:0/1249551449 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:49.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.208+0000 7f53fd2f6640 1 --2- 172.21.15.139:0/1249551449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f815aeb0 0x7f53f81f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:49.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.208+0000 7f53fcaf5640 1 --2- 172.21.15.139:0/1249551449 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53f8072c30 0x7f53f8146c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:49.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.208+0000 7f53f7fff640 1 -- 172.21.15.139:0/1249551449 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53f8072c30 msgr2=0x7f53f8146c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:49.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.208+0000 7f53f7fff640 1 --2- 172.21.15.139:0/1249551449 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53f8072c30 0x7f53f8146c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:49.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.208+0000 7f53f7fff640 1 -- 172.21.15.139:0/1249551449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f815aeb0 msgr2=0x7f53f81f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:49.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.208+0000 7f53f7fff640 1 --2- 172.21.15.139:0/1249551449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f815aeb0 0x7f53f81f4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:49.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.208+0000 7f53f7fff640 1 -- 172.21.15.139:0/1249551449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f81471b0 con 0x7f53f8073010 2024-09-16T00:07:49.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.208+0000 7f53f7fff640 1 --2- 172.21.15.139:0/1249551449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f8073010 0x7f53f80cc600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f53e4000c00 tx=0x7f53e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:49.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.208+0000 7f53f57fa640 1 -- 172.21.15.139:0/1249551449 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4037040 con 0x7f53f8073010 2024-09-16T00:07:49.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.208+0000 7f53f57fa640 1 -- 172.21.15.139:0/1249551449 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e4042070 con 0x7f53f8073010 2024-09-16T00:07:49.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.208+0000 7f53f57fa640 1 -- 172.21.15.139:0/1249551449 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4030bd0 con 0x7f53f8073010 2024-09-16T00:07:49.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.208+0000 7f53fed59640 1 -- 172.21.15.139:0/1249551449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f80ca980 con 0x7f53f8073010 2024-09-16T00:07:49.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.208+0000 7f53fed59640 1 -- 172.21.15.139:0/1249551449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f80ca3c0 con 0x7f53f8073010 2024-09-16T00:07:49.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.209+0000 7f53fed59640 1 -- 172.21.15.139:0/1249551449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f80c6210 con 0x7f53f8073010 2024-09-16T00:07:49.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.210+0000 7f53f57fa640 1 -- 172.21.15.139:0/1249551449 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f53e4047460 con 0x7f53f8073010 2024-09-16T00:07:49.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.210+0000 7f53f57fa640 1 --2- 172.21.15.139:0/1249551449 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f53dc06c2e0 0x7f53dc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:49.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.210+0000 7f53fcaf5640 1 --2- 172.21.15.139:0/1249551449 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f53dc06c2e0 0x7f53dc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:49.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.210+0000 7f53f57fa640 1 -- 172.21.15.139:0/1249551449 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f53e40abfd0 con 0x7f53f8073010 2024-09-16T00:07:49.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.210+0000 7f53fcaf5640 1 --2- 172.21.15.139:0/1249551449 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f53dc06c2e0 0x7f53dc06e7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f53ec006fd0 tx=0x7f53ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:49.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.214+0000 7f53f57fa640 1 -- 172.21.15.139:0/1249551449 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f53e4045360 con 0x7f53f8073010 2024-09-16T00:07:49.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.415+0000 7f53fed59640 1 -- 172.21.15.139:0/1249551449 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f53f80702f0 con 0x7f53f8073010 2024-09-16T00:07:49.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.415+0000 7f53f57fa640 1 -- 172.21.15.139:0/1249551449 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f53e407a7d0 con 0x7f53f8073010 2024-09-16T00:07:49.417 INFO:teuthology.orchestra.run.smithi139.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-16T00:07:49.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.417+0000 7f53fed59640 1 -- 172.21.15.139:0/1249551449 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f53dc06c2e0 msgr2=0x7f53dc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:49.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.417+0000 7f53fed59640 1 --2- 172.21.15.139:0/1249551449 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f53dc06c2e0 0x7f53dc06e7a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f53ec006fd0 tx=0x7f53ec008040 comp rx=0 tx=0).stop 2024-09-16T00:07:49.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.417+0000 7f53fed59640 1 -- 172.21.15.139:0/1249551449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f8073010 msgr2=0x7f53f80cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:49.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.417+0000 7f53fed59640 1 --2- 172.21.15.139:0/1249551449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f8073010 0x7f53f80cc600 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f53e4000c00 tx=0x7f53e40046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:49.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.418+0000 7f53fed59640 1 -- 172.21.15.139:0/1249551449 shutdown_connections 2024-09-16T00:07:49.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.418+0000 7f53fed59640 1 --2- 172.21.15.139:0/1249551449 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f53dc06c2e0 0x7f53dc06e7a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:49.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.418+0000 7f53fed59640 1 --2- 172.21.15.139:0/1249551449 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53f815aeb0 0x7f53f81f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:49.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.418+0000 7f53fed59640 1 --2- 172.21.15.139:0/1249551449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53f8073010 0x7f53f80cc600 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:49.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.418+0000 7f53fed59640 1 --2- 172.21.15.139:0/1249551449 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53f8072c30 0x7f53f8146c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:49.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.418+0000 7f53fed59640 1 -- 172.21.15.139:0/1249551449 >> 172.21.15.139:0/1249551449 conn(0x7f53f8067950 msgr2=0x7f53f806fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:49.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.418+0000 7f53fed59640 1 -- 172.21.15.139:0/1249551449 shutdown_connections 2024-09-16T00:07:49.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49.418+0000 7f53fed59640 1 -- 172.21.15.139:0/1249551449 wait complete. 2024-09-16T00:07:49.429 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T00:07:49.429 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T00:07:49.429 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-09-16T00:07:49.430 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T00:07:49.430 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T00:07:49.430 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T00:07:49.430 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T00:07:49.430 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T00:07:49.430 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T00:07:49.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:07:49.430 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-09-16T00:07:49.448 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T00:07:49.448 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T00:07:49.448 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T00:07:49.448 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:07:49.448 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-16T00:07:49.449 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:07:49.449 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:07:49.449 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:07:49.449 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-09-16T00:07:49.449 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:07:49.449 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:07:49.449 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-09-16T00:07:49.449 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:07:49.449 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:07:49.449 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T00:07:49.449 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:07:49.449 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T00:07:49.449 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42755sec preferred_lft 42755sec 2024-09-16T00:07:49.449 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link noprefixroute 2024-09-16T00:07:49.450 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:07:49.450 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T00:07:49.450 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:07:49.450 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-16T00:07:49.450 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add name ceph-brx type bridge 2024-09-16T00:07:49.450 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip addr flush dev ceph-brx 2024-09-16T00:07:49.450 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set ceph-brx up 2024-09-16T00:07:49.450 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T00:07:49.450 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-16T00:07:49.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:07:49.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:07:49.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:07:49.644 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T00:07:49.669 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-16T00:07:49.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:07:49.670 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-09-16T00:07:49.736 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-09-16T00:07:49.736 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T00:07:49.736 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T00:07:49.736 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T00:07:49.736 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T00:07:49.737 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:07:49.737 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-16T00:07:49.737 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T00:07:49.737 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T00:07:49.737 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T00:07:49.737 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-16T00:07:49.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:07:50.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:07:50.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:07:50.043 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-16T00:07:50.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:07:50.059 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list-id 2024-09-16T00:07:50.115 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:07:50.116 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-16T00:07:50.116 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:07:50.116 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-16T00:07:50.116 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-16T00:07:50.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:07:50.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:07:50.233 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-16T00:07:50.233 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:07:50.233 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-16T00:07:50.233 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-16T00:07:50.233 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T00:07:50.233 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-16T00:07:50.234 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-16T00:07:50.234 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-16T00:07:50.234 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-16T00:07:50.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:07:50.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:07:50.358 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:07:50.358 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-16T00:07:50.358 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-09-16T00:07:50.358 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T00:07:50.358 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-16T00:07:50.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:07:50.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:07:50.434 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-16T00:07:50.434 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:07:50.434 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T00:07:50.450 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T00:07:50.451 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:07:50.451 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T00:07:50.506 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-16T00:07:50.572 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:07:50.629 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-16T00:07:50.629 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-16T00:07:50.629 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-16T00:07:50.629 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-16T00:07:50.629 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-16T00:07:50.629 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-16T00:07:50.629 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-16T00:07:50.629 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-16T00:07:50.629 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-16T00:07:50.629 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-16T00:07:50.630 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-16T00:07:50.630 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/selinux 2024-09-16T00:07:50.630 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:07:50.630 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-16T00:07:50.630 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-16T00:07:50.630 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-16T00:07:50.630 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-16T00:07:50.630 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-16T00:07:50.630 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-16T00:07:50.630 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:07:50.630 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:07:50.630 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:07:50.630 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:07:50.631 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-16T00:07:50.631 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-16T00:07:50.631 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-0 2024-09-16T00:07:50.631 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-1 2024-09-16T00:07:50.631 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-2 2024-09-16T00:07:50.631 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-3 2024-09-16T00:07:50.631 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-16T00:07:50.631 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:07:50.631 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:07:50.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:07:50.631 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-16T00:07:50.684 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-16T00:07:50.684 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-16T00:07:50.726 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-16T00:07:50.750 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:07:50.808 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-16T00:07:50.808 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-16T00:07:50.808 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-16T00:07:50.808 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-16T00:07:50.808 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-16T00:07:50.809 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-16T00:07:50.809 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-16T00:07:50.809 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-16T00:07:50.809 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-16T00:07:50.809 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-16T00:07:50.809 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-16T00:07:50.809 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/selinux 2024-09-16T00:07:50.809 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:07:50.809 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-16T00:07:50.809 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-16T00:07:50.809 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-16T00:07:50.809 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-16T00:07:50.809 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-16T00:07:50.809 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-16T00:07:50.810 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:07:50.810 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:07:50.810 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:07:50.810 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:07:50.810 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-16T00:07:50.810 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-16T00:07:50.810 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-0 2024-09-16T00:07:50.810 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-1 2024-09-16T00:07:50.810 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-2 2024-09-16T00:07:50.810 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-3 2024-09-16T00:07:50.810 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-16T00:07:50.810 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:07:50.810 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:07:50.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:07:50.811 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-16T00:07:50.837 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi139.stderr:2024-09-16T00:07:50.836+0000 7fbe62f6f480 -1 init, newargv = 0x55ff0782aff0 newargc=15 2024-09-16T00:07:50.837 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi139.stderr:ceph-fuse[43418]: starting ceph client 2024-09-16T00:07:50.841 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi139.stderr:ceph-fuse[43418]: starting fuse 2024-09-16T00:07:50.853 INFO:teuthology.orchestra.run.smithi139.stdout:42 2024-09-16T00:07:50.853 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-16T00:07:50.853 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2024-09-16T00:07:50.854 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2024-09-16T00:07:50.854 DEBUG:teuthology.orchestra.run.smithi139:> import re 2024-09-16T00:07:50.854 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-09-16T00:07:50.854 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2024-09-16T00:07:50.854 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-16T00:07:50.854 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2024-09-16T00:07:50.854 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-16T00:07:50.854 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2024-09-16T00:07:50.854 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-16T00:07:50.854 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-16T00:07:50.854 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2024-09-16T00:07:50.854 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2024-09-16T00:07:50.854 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2024-09-16T00:07:50.854 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2024-09-16T00:07:50.854 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-16T00:07:50.855 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2024-09-16T00:07:50.855 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-16T00:07:50.855 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-16T00:07:50.855 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-16T00:07:50.855 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2024-09-16T00:07:50.855 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2024-09-16T00:07:50.855 DEBUG:teuthology.orchestra.run.smithi139:> return f 2024-09-16T00:07:50.855 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-16T00:07:50.855 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-16T00:07:50.855 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.0")) 2024-09-16T00:07:50.855 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-09-16T00:07:50.971 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.0.43418.asok 2024-09-16T00:07:50.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:07:50.982 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43418.asok 2024-09-16T00:07:50.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:07:50.982 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43418.asok status 2024-09-16T00:07:51.117 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:07:51.117 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2024-09-16T00:07:51.117 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "8921c8ef947807b717db041d85f0761b7914d5fc", 2024-09-16T00:07:51.117 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:07:51.117 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "0", 2024-09-16T00:07:51.117 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2024-09-16T00:07:51.117 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-16T00:07:51.117 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "43418", 2024-09-16T00:07:51.117 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2024-09-16T00:07:51.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:51.117 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2024-09-16T00:07:51.118 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2024-09-16T00:07:51.118 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 4638, 2024-09-16T00:07:51.118 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2024-09-16T00:07:51.118 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2024-09-16T00:07:51.118 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2024-09-16T00:07:51.118 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 4638 2024-09-16T00:07:51.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:51.118 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-16T00:07:51.118 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:51.118 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-16T00:07:51.118 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3876595971 2024-09-16T00:07:51.118 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:51.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:51.118 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-16T00:07:51.119 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:51.119 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-16T00:07:51.119 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3876595971 2024-09-16T00:07:51.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:51.119 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.4638 192.168.0.1:0/3876595971", 2024-09-16T00:07:51.119 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.1:0/3876595971", 2024-09-16T00:07:51.119 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2024-09-16T00:07:51.119 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 8, 2024-09-16T00:07:51.119 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 22, 2024-09-16T00:07:51.119 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2024-09-16T00:07:51.119 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2024-09-16T00:07:51.119 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2024-09-16T00:07:51.119 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:07:51.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:07:51.125 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2024-09-16T00:07:51.125 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-16T00:07:51.141 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2024-09-16T00:07:51.141 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-16T00:07:51.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:07:51.142 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T00:07:51.207 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-16T00:07:51.244 INFO:tasks.thrashosds:config is {'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-16T00:07:51.244 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ad35db06e8924912f01e32b35576297c0f1352b4'}} 2024-09-16T00:07:51.244 INFO:tasks.thrashosds:config is {'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-16T00:07:51.245 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-16T00:07:51.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.348+0000 7f0a5d49b640 1 Processor -- start 2024-09-16T00:07:51.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.348+0000 7f0a5d49b640 1 -- start start 2024-09-16T00:07:51.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.349+0000 7f0a5d49b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0a58154c20 0x7f0a5815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.349+0000 7f0a5d49b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a58068060 0x7f0a580c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.349+0000 7f0a5d49b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a580cc4b0 con 0x7f0a580ccc90 2024-09-16T00:07:51.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.349+0000 7f0a5d49b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f0a5815b7c0 con 0x7f0a58154c20 2024-09-16T00:07:51.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.349+0000 7f0a5d49b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0a580cc260 con 0x7f0a58068060 2024-09-16T00:07:51.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.349+0000 7f0a567fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a58068060 0x7f0a580c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.349+0000 7f0a56ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0a58154c20 0x7f0a5815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.349+0000 7f0a567fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a58068060 0x7f0a580c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:60762/0 (socket says 172.21.15.139:60762) 2024-09-16T00:07:51.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.349+0000 7f0a567fc640 1 -- 172.21.15.139:0/296774799 learned_addr learned my addr 172.21.15.139:0/296774799 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:51.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.349+0000 7f0a577fe640 1 --1- 172.21.15.139:0/296774799 >> v1:172.21.15.164:6789/0 conn(0x7f0a580ccc90 0x7f0a580cac00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:60550/0 (socket says 172.21.15.139:60550) 2024-09-16T00:07:51.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.350+0000 7f0a567fc640 1 -- 172.21.15.139:0/296774799 >> v2:172.21.15.164:3300/0 conn(0x7f0a58154c20 msgr2=0x7f0a5815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:51.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.350+0000 7f0a567fc640 1 --2- 172.21.15.139:0/296774799 >> v2:172.21.15.164:3300/0 conn(0x7f0a58154c20 0x7f0a5815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.350+0000 7f0a567fc640 1 -- 172.21.15.139:0/296774799 >> v1:172.21.15.164:6789/0 conn(0x7f0a580ccc90 legacy=0x7f0a580cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:51.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.350+0000 7f0a567fc640 1 -- 172.21.15.139:0/296774799 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a580cc790 con 0x7f0a58068060 2024-09-16T00:07:51.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.350+0000 7f0a567fc640 1 --2- 172.21.15.139:0/296774799 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a58068060 0x7f0a580c5120 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c009850 tx=0x7f0a4c02a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fb3dbfe1392f05e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:51.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.351+0000 7f0a55ffb640 1 -- 172.21.15.139:0/296774799 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c004230 con 0x7f0a58068060 2024-09-16T00:07:51.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.351+0000 7f0a55ffb640 1 -- 172.21.15.139:0/296774799 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a4c0043d0 con 0x7f0a58068060 2024-09-16T00:07:51.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.351+0000 7f0a55ffb640 1 -- 172.21.15.139:0/296774799 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c003810 con 0x7f0a58068060 2024-09-16T00:07:51.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.351+0000 7f0a5d49b640 1 -- 172.21.15.139:0/296774799 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a58068060 msgr2=0x7f0a580c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:51.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.351+0000 7f0a5d49b640 1 --2- 172.21.15.139:0/296774799 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a58068060 0x7f0a580c5120 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c009850 tx=0x7f0a4c02a2c0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.351+0000 7f0a5d49b640 1 -- 172.21.15.139:0/296774799 shutdown_connections 2024-09-16T00:07:51.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.351+0000 7f0a5d49b640 1 --2- 172.21.15.139:0/296774799 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a58068060 0x7f0a580c5120 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.351+0000 7f0a5d49b640 1 --2- 172.21.15.139:0/296774799 >> v2:172.21.15.164:3300/0 conn(0x7f0a58154c20 0x7f0a5815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.351+0000 7f0a5d49b640 1 -- 172.21.15.139:0/296774799 >> 172.21.15.139:0/296774799 conn(0x7f0a5806ca40 msgr2=0x7f0a58067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:51.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.351+0000 7f0a5d49b640 1 -- 172.21.15.139:0/296774799 shutdown_connections 2024-09-16T00:07:51.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.351+0000 7f0a5d49b640 1 -- 172.21.15.139:0/296774799 wait complete. 2024-09-16T00:07:51.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.352+0000 7f0a5d49b640 1 Processor -- start 2024-09-16T00:07:51.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.353+0000 7f0a5d49b640 1 -- start start 2024-09-16T00:07:51.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.353+0000 7f0a5d49b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a58068060 0x7f0a5814d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.353+0000 7f0a5d49b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a580ccc90 0x7f0a5814c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.353+0000 7f0a5d49b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a58154c20 0x7f0a5815cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.353+0000 7f0a5d49b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5815e3b0 con 0x7f0a580ccc90 2024-09-16T00:07:51.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.353+0000 7f0a5d49b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5815e0d0 con 0x7f0a58068060 2024-09-16T00:07:51.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.353+0000 7f0a5d49b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0a5815e690 con 0x7f0a58154c20 2024-09-16T00:07:51.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.353+0000 7f0a567fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a580ccc90 0x7f0a5814c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.353+0000 7f0a567fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a580ccc90 0x7f0a5814c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47708/0 (socket says 172.21.15.139:47708) 2024-09-16T00:07:51.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.353+0000 7f0a567fc640 1 -- 172.21.15.139:0/2376299286 learned_addr learned my addr 172.21.15.139:0/2376299286 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:51.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.353+0000 7f0a56ffd640 1 --2- 172.21.15.139:0/2376299286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a58068060 0x7f0a5814d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.353+0000 7f0a577fe640 1 --2- 172.21.15.139:0/2376299286 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a58154c20 0x7f0a5815cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.354+0000 7f0a567fc640 1 -- 172.21.15.139:0/2376299286 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a58154c20 msgr2=0x7f0a5815cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:51.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.354+0000 7f0a567fc640 1 --2- 172.21.15.139:0/2376299286 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a58154c20 0x7f0a5815cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.354+0000 7f0a567fc640 1 -- 172.21.15.139:0/2376299286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a58068060 msgr2=0x7f0a5814d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:51.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.354+0000 7f0a567fc640 1 --2- 172.21.15.139:0/2376299286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a58068060 0x7f0a5814d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.354+0000 7f0a567fc640 1 -- 172.21.15.139:0/2376299286 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5815a030 con 0x7f0a580ccc90 2024-09-16T00:07:51.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.354+0000 7f0a567fc640 1 --2- 172.21.15.139:0/2376299286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a580ccc90 0x7f0a5814c3e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c034040 tx=0x7f0a4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:51.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.354+0000 7f0a3f7fe640 1 -- 172.21.15.139:0/2376299286 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c041030 con 0x7f0a580ccc90 2024-09-16T00:07:51.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.354+0000 7f0a3f7fe640 1 -- 172.21.15.139:0/2376299286 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a4c003c50 con 0x7f0a580ccc90 2024-09-16T00:07:51.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.354+0000 7f0a5d49b640 1 -- 172.21.15.139:0/2376299286 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5815a810 con 0x7f0a580ccc90 2024-09-16T00:07:51.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.354+0000 7f0a3f7fe640 1 -- 172.21.15.139:0/2376299286 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c003e20 con 0x7f0a580ccc90 2024-09-16T00:07:51.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.354+0000 7f0a5d49b640 1 -- 172.21.15.139:0/2376299286 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a5814d930 con 0x7f0a580ccc90 2024-09-16T00:07:51.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.355+0000 7f0a5d49b640 1 -- 172.21.15.139:0/2376299286 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a580686b0 con 0x7f0a580ccc90 2024-09-16T00:07:51.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.356+0000 7f0a3f7fe640 1 -- 172.21.15.139:0/2376299286 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0a4c044070 con 0x7f0a580ccc90 2024-09-16T00:07:51.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.356+0000 7f0a3f7fe640 1 --2- 172.21.15.139:0/2376299286 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a4406c2e0 0x7f0a4406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.356+0000 7f0a3f7fe640 1 -- 172.21.15.139:0/2376299286 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0a4c0abc90 con 0x7f0a580ccc90 2024-09-16T00:07:51.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.356+0000 7f0a56ffd640 1 --2- 172.21.15.139:0/2376299286 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a4406c2e0 0x7f0a4406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.357+0000 7f0a56ffd640 1 --2- 172.21.15.139:0/2376299286 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a4406c2e0 0x7f0a4406e7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009aa0 tx=0x7f0a48009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:51.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.360+0000 7f0a3f7fe640 1 -- 172.21.15.139:0/2376299286 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0a4c079dc0 con 0x7f0a580ccc90 2024-09-16T00:07:51.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.526+0000 7f0a5d49b640 1 -- 172.21.15.139:0/2376299286 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mgr_debug_aggressive_pg_num_changes}] v 0) -- 0x7f0a58159080 con 0x7f0a580ccc90 2024-09-16T00:07:51.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.529+0000 7f0a3f7fe640 1 -- 172.21.15.139:0/2376299286 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr_debug_aggressive_pg_num_changes}]=0 v10)=0 v10) ==== 136+0+0 (secure 0 0 0) 0x7f0a4c07e0c0 con 0x7f0a580ccc90 2024-09-16T00:07:51.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.531+0000 7f0a5d49b640 1 -- 172.21.15.139:0/2376299286 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a4406c2e0 msgr2=0x7f0a4406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:51.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.531+0000 7f0a5d49b640 1 --2- 172.21.15.139:0/2376299286 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a4406c2e0 0x7f0a4406e7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009aa0 tx=0x7f0a48009190 comp rx=0 tx=0).stop 2024-09-16T00:07:51.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.531+0000 7f0a5d49b640 1 -- 172.21.15.139:0/2376299286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a580ccc90 msgr2=0x7f0a5814c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:51.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.531+0000 7f0a5d49b640 1 --2- 172.21.15.139:0/2376299286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a580ccc90 0x7f0a5814c3e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c034040 tx=0x7f0a4c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.531+0000 7f0a5d49b640 1 -- 172.21.15.139:0/2376299286 shutdown_connections 2024-09-16T00:07:51.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.531+0000 7f0a5d49b640 1 --2- 172.21.15.139:0/2376299286 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a4406c2e0 0x7f0a4406e7a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.531+0000 7f0a5d49b640 1 --2- 172.21.15.139:0/2376299286 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a58154c20 0x7f0a5815cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.531+0000 7f0a5d49b640 1 --2- 172.21.15.139:0/2376299286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a580ccc90 0x7f0a5814c3e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.531+0000 7f0a5d49b640 1 --2- 172.21.15.139:0/2376299286 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a58068060 0x7f0a5814d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.531+0000 7f0a5d49b640 1 -- 172.21.15.139:0/2376299286 >> 172.21.15.139:0/2376299286 conn(0x7f0a5806ca40 msgr2=0x7f0a580c5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:51.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.532+0000 7f0a5d49b640 1 -- 172.21.15.139:0/2376299286 shutdown_connections 2024-09-16T00:07:51.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.532+0000 7f0a5d49b640 1 -- 172.21.15.139:0/2376299286 wait complete. 2024-09-16T00:07:51.544 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-16T00:07:51.544 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T00:07:51.544 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:07:51.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.643+0000 7fbaf073b640 1 Processor -- start 2024-09-16T00:07:51.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.643+0000 7fbaf073b640 1 -- start start 2024-09-16T00:07:51.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.644+0000 7fbaf073b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbae8154cd0 0x7fbae815b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.644+0000 7fbaf073b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbae806e9b0 0x7fbae8165120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.644+0000 7fbaf073b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbae806ce40 con 0x7fbae815dd40 2024-09-16T00:07:51.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.644+0000 7fbaf073b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbae815acd0 con 0x7fbae8154cd0 2024-09-16T00:07:51.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.644+0000 7fbaf073b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fbae806d120 con 0x7fbae806e9b0 2024-09-16T00:07:51.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.644+0000 7fbaee4d7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbae8154cd0 0x7fbae815b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.644+0000 7fbaedcd6640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fbae815dd40 0x7fbae80714d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57826/0 (socket says 172.21.15.139:57826) 2024-09-16T00:07:51.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.644+0000 7fbaedcd6640 1 -- 172.21.15.139:0/2068878923 learned_addr learned my addr 172.21.15.139:0/2068878923 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:51.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.644+0000 7fbaeecd8640 1 --2- 172.21.15.139:0/2068878923 >> v2:172.21.15.164:3300/0 conn(0x7fbae806e9b0 0x7fbae8165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.645+0000 7fbaee4d7640 1 -- 172.21.15.139:0/2068878923 >> v2:172.21.15.164:3300/0 conn(0x7fbae806e9b0 msgr2=0x7fbae8165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:51.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.645+0000 7fbaee4d7640 1 --2- 172.21.15.139:0/2068878923 >> v2:172.21.15.164:3300/0 conn(0x7fbae806e9b0 0x7fbae8165120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.645+0000 7fbaee4d7640 1 -- 172.21.15.139:0/2068878923 >> v1:172.21.15.139:6789/0 conn(0x7fbae815dd40 legacy=0x7fbae80714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:51.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.645+0000 7fbaee4d7640 1 -- 172.21.15.139:0/2068878923 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae815a960 con 0x7fbae8154cd0 2024-09-16T00:07:51.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.645+0000 7fbaed4d5640 1 -- 172.21.15.139:0/2068878923 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 840067588 0 0) 0x7fbae806ce40 con 0x7fbae815dd40 2024-09-16T00:07:51.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.645+0000 7fbaee4d7640 1 --2- 172.21.15.139:0/2068878923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae8154cd0 0x7fbae815b160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbae8071130 tx=0x7fbae002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9ebe239f323427 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:51.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.645+0000 7fbaed4d5640 1 -- 172.21.15.139:0/2068878923 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0007740 con 0x7fbae8154cd0 2024-09-16T00:07:51.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.645+0000 7fbaed4d5640 1 -- 172.21.15.139:0/2068878923 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae00078e0 con 0x7fbae8154cd0 2024-09-16T00:07:51.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.645+0000 7fbaed4d5640 1 -- 172.21.15.139:0/2068878923 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0007cb0 con 0x7fbae8154cd0 2024-09-16T00:07:51.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.646+0000 7fbaf073b640 1 -- 172.21.15.139:0/2068878923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae8154cd0 msgr2=0x7fbae815b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:51.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.646+0000 7fbaf073b640 1 --2- 172.21.15.139:0/2068878923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae8154cd0 0x7fbae815b160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbae8071130 tx=0x7fbae002a710 comp rx=0 tx=0).stop 2024-09-16T00:07:51.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.646+0000 7fbaf073b640 1 -- 172.21.15.139:0/2068878923 shutdown_connections 2024-09-16T00:07:51.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.646+0000 7fbaf073b640 1 --2- 172.21.15.139:0/2068878923 >> v2:172.21.15.164:3300/0 conn(0x7fbae806e9b0 0x7fbae8165120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.646+0000 7fbaf073b640 1 --2- 172.21.15.139:0/2068878923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae8154cd0 0x7fbae815b160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.646+0000 7fbaf073b640 1 -- 172.21.15.139:0/2068878923 >> 172.21.15.139:0/2068878923 conn(0x7fbae8073e70 msgr2=0x7fbae806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:51.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.646+0000 7fbaf073b640 1 -- 172.21.15.139:0/2068878923 shutdown_connections 2024-09-16T00:07:51.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.646+0000 7fbaf073b640 1 -- 172.21.15.139:0/2068878923 wait complete. 2024-09-16T00:07:51.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.647+0000 7fbaf073b640 1 Processor -- start 2024-09-16T00:07:51.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.647+0000 7fbaf073b640 1 -- start start 2024-09-16T00:07:51.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaf073b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae806e9b0 0x7fbae80c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaf073b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbae8154cd0 0x7fbae81500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaf073b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbae80c75b0 0x7fbae81f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaf073b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbae80cb9d0 con 0x7fbae806e9b0 2024-09-16T00:07:51.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaf073b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbae815b6a0 con 0x7fbae80c75b0 2024-09-16T00:07:51.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaf073b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbae8071be0 con 0x7fbae8154cd0 2024-09-16T00:07:51.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaee4d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae806e9b0 0x7fbae80c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaee4d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae806e9b0 0x7fbae80c4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47724/0 (socket says 172.21.15.139:47724) 2024-09-16T00:07:51.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaee4d7640 1 -- 172.21.15.139:0/1326292998 learned_addr learned my addr 172.21.15.139:0/1326292998 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaeecd8640 1 --2- 172.21.15.139:0/1326292998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbae80c75b0 0x7fbae81f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaedcd6640 1 --2- 172.21.15.139:0/1326292998 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbae8154cd0 0x7fbae81500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaee4d7640 1 -- 172.21.15.139:0/1326292998 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbae8154cd0 msgr2=0x7fbae81500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaee4d7640 1 --2- 172.21.15.139:0/1326292998 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbae8154cd0 0x7fbae81500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaee4d7640 1 -- 172.21.15.139:0/1326292998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbae80c75b0 msgr2=0x7fbae81f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaee4d7640 1 --2- 172.21.15.139:0/1326292998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbae80c75b0 0x7fbae81f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaee4d7640 1 -- 172.21.15.139:0/1326292998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae81f0b80 con 0x7fbae806e9b0 2024-09-16T00:07:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaeecd8640 1 --2- 172.21.15.139:0/1326292998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbae80c75b0 0x7fbae81f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaedcd6640 1 --2- 172.21.15.139:0/1326292998 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbae8154cd0 0x7fbae81500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.648+0000 7fbaee4d7640 1 --2- 172.21.15.139:0/1326292998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae806e9b0 0x7fbae80c4ae0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009e80 tx=0x7fbae0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.649+0000 7fbadeffd640 1 -- 172.21.15.139:0/1326292998 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0040070 con 0x7fbae806e9b0 2024-09-16T00:07:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.649+0000 7fbadeffd640 1 -- 172.21.15.139:0/1326292998 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae0031c40 con 0x7fbae806e9b0 2024-09-16T00:07:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.649+0000 7fbaf073b640 1 -- 172.21.15.139:0/1326292998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae80c5220 con 0x7fbae806e9b0 2024-09-16T00:07:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.649+0000 7fbadeffd640 1 -- 172.21.15.139:0/1326292998 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae0009450 con 0x7fbae806e9b0 2024-09-16T00:07:51.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.649+0000 7fbaf073b640 1 -- 172.21.15.139:0/1326292998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae81f10d0 con 0x7fbae806e9b0 2024-09-16T00:07:51.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.650+0000 7fbaf073b640 1 -- 172.21.15.139:0/1326292998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae806e1d0 con 0x7fbae806e9b0 2024-09-16T00:07:51.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.650+0000 7fbadeffd640 1 -- 172.21.15.139:0/1326292998 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbae0039040 con 0x7fbae806e9b0 2024-09-16T00:07:51.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.650+0000 7fbadeffd640 1 --2- 172.21.15.139:0/1326292998 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbacc06c2b0 0x7fbacc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.651+0000 7fbadeffd640 1 -- 172.21.15.139:0/1326292998 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbae00ac610 con 0x7fbae806e9b0 2024-09-16T00:07:51.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.651+0000 7fbaedcd6640 1 --2- 172.21.15.139:0/1326292998 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbacc06c2b0 0x7fbacc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.651+0000 7fbaedcd6640 1 --2- 172.21.15.139:0/1326292998 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbacc06c2b0 0x7fbacc06e770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbae8150d40 tx=0x7fbad8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:51.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.655+0000 7fbadeffd640 1 -- 172.21.15.139:0/1326292998 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbae806e1d0 con 0x7fbae806e9b0 2024-09-16T00:07:51.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.818+0000 7fbaf073b640 1 -- 172.21.15.139:0/1326292998 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbae815e360 con 0x7fbacc06c2b0 2024-09-16T00:07:51.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.824+0000 7fbadeffd640 1 -- 172.21.15.139:0/1326292998 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251938 (secure 0 0 0) 0x7fbae815e360 con 0x7fbacc06c2b0 2024-09-16T00:07:51.827 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:51.829 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:07:51.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.827+0000 7fbaf073b640 1 -- 172.21.15.139:0/1326292998 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbacc06c2b0 msgr2=0x7fbacc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:51.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.827+0000 7fbaf073b640 1 --2- 172.21.15.139:0/1326292998 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbacc06c2b0 0x7fbacc06e770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbae8150d40 tx=0x7fbad8009040 comp rx=0 tx=0).stop 2024-09-16T00:07:51.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.827+0000 7fbaf073b640 1 -- 172.21.15.139:0/1326292998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae806e9b0 msgr2=0x7fbae80c4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:51.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.827+0000 7fbaf073b640 1 --2- 172.21.15.139:0/1326292998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae806e9b0 0x7fbae80c4ae0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009e80 tx=0x7fbae0004a40 comp rx=0 tx=0).stop 2024-09-16T00:07:51.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.827+0000 7fbaf073b640 1 -- 172.21.15.139:0/1326292998 shutdown_connections 2024-09-16T00:07:51.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.827+0000 7fbaf073b640 1 --2- 172.21.15.139:0/1326292998 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbacc06c2b0 0x7fbacc06e770 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.827+0000 7fbaf073b640 1 --2- 172.21.15.139:0/1326292998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbae80c75b0 0x7fbae81f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.827+0000 7fbaf073b640 1 --2- 172.21.15.139:0/1326292998 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbae8154cd0 0x7fbae81500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.827+0000 7fbaf073b640 1 --2- 172.21.15.139:0/1326292998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae806e9b0 0x7fbae80c4ae0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.827+0000 7fbaf073b640 1 -- 172.21.15.139:0/1326292998 >> 172.21.15.139:0/1326292998 conn(0x7fbae8073e70 msgr2=0x7fbae80ca250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:51.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.827+0000 7fbaf073b640 1 -- 172.21.15.139:0/1326292998 shutdown_connections 2024-09-16T00:07:51.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.827+0000 7fbaf073b640 1 -- 172.21.15.139:0/1326292998 wait complete. 2024-09-16T00:07:51.839 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-16T00:07:50.028820+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218112,"kb_used_data":1920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756608,"statfs":{"total":773094113280,"available":772870766592,"internally_reserved":0,"allocated":1966080,"data_stored":1809524,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002451"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115528+0000","last_change":"2024-09-16T00:07:33.635096+0000","last_active":"2024-09-16T00:07:35.115528+0000","last_peered":"2024-09-16T00:07:35.115528+0000","last_clean":"2024-09-16T00:07:35.115528+0000","last_became_active":"2024-09-16T00:07:33.634750+0000","last_became_peered":"2024-09-16T00:07:33.634750+0000","last_unstale":"2024-09-16T00:07:35.115528+0000","last_undegraded":"2024-09-16T00:07:35.115528+0000","last_fullsized":"2024-09-16T00:07:35.115528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115558+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:07:35.115558+0000","last_peered":"2024-09-16T00:07:35.115558+0000","last_clean":"2024-09-16T00:07:35.115558+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:07:35.115558+0000","last_undegraded":"2024-09-16T00:07:35.115558+0000","last_fullsized":"2024-09-16T00:07:35.115558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115392+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:07:35.115392+0000","last_peered":"2024-09-16T00:07:35.115392+0000","last_clean":"2024-09-16T00:07:35.115392+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:07:35.115392+0000","last_undegraded":"2024-09-16T00:07:35.115392+0000","last_fullsized":"2024-09-16T00:07:35.115392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615167+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:07:34.615167+0000","last_peered":"2024-09-16T00:07:34.615167+0000","last_clean":"2024-09-16T00:07:34.615167+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:07:34.615167+0000","last_undegraded":"2024-09-16T00:07:34.615167+0000","last_fullsized":"2024-09-16T00:07:34.615167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115724+0000","last_change":"2024-09-16T00:07:33.630029+0000","last_active":"2024-09-16T00:07:35.115724+0000","last_peered":"2024-09-16T00:07:35.115724+0000","last_clean":"2024-09-16T00:07:35.115724+0000","last_became_active":"2024-09-16T00:07:33.629712+0000","last_became_peered":"2024-09-16T00:07:33.629712+0000","last_unstale":"2024-09-16T00:07:35.115724+0000","last_undegraded":"2024-09-16T00:07:35.115724+0000","last_fullsized":"2024-09-16T00:07:35.115724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614973+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:07:34.614973+0000","last_peered":"2024-09-16T00:07:34.614973+0000","last_clean":"2024-09-16T00:07:34.614973+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:07:34.614973+0000","last_undegraded":"2024-09-16T00:07:34.614973+0000","last_fullsized":"2024-09-16T00:07:34.614973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115522+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:07:35.115522+0000","last_peered":"2024-09-16T00:07:35.115522+0000","last_clean":"2024-09-16T00:07:35.115522+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:07:35.115522+0000","last_undegraded":"2024-09-16T00:07:35.115522+0000","last_fullsized":"2024-09-16T00:07:35.115522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114492+0000","last_change":"2024-09-16T00:07:33.624708+0000","last_active":"2024-09-16T00:07:35.114492+0000","last_peered":"2024-09-16T00:07:35.114492+0000","last_clean":"2024-09-16T00:07:35.114492+0000","last_became_active":"2024-09-16T00:07:33.624458+0000","last_became_peered":"2024-09-16T00:07:33.624458+0000","last_unstale":"2024-09-16T00:07:35.114492+0000","last_undegraded":"2024-09-16T00:07:35.114492+0000","last_fullsized":"2024-09-16T00:07:35.114492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:48.052488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115751+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:07:35.115751+0000","last_peered":"2024-09-16T00:07:35.115751+0000","last_clean":"2024-09-16T00:07:35.115751+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:07:35.115751+0000","last_undegraded":"2024-09-16T00:07:35.115751+0000","last_fullsized":"2024-09-16T00:07:35.115751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115614+0000","last_change":"2024-09-16T00:07:33.635973+0000","last_active":"2024-09-16T00:07:35.115614+0000","last_peered":"2024-09-16T00:07:35.115614+0000","last_clean":"2024-09-16T00:07:35.115614+0000","last_became_active":"2024-09-16T00:07:33.635486+0000","last_became_peered":"2024-09-16T00:07:33.635486+0000","last_unstale":"2024-09-16T00:07:35.115614+0000","last_undegraded":"2024-09-16T00:07:35.115614+0000","last_fullsized":"2024-09-16T00:07:35.115614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115027+0000","last_change":"2024-09-16T00:07:33.636971+0000","last_active":"2024-09-16T00:07:35.115027+0000","last_peered":"2024-09-16T00:07:35.115027+0000","last_clean":"2024-09-16T00:07:35.115027+0000","last_became_active":"2024-09-16T00:07:33.635760+0000","last_became_peered":"2024-09-16T00:07:33.635760+0000","last_unstale":"2024-09-16T00:07:35.115027+0000","last_undegraded":"2024-09-16T00:07:35.115027+0000","last_fullsized":"2024-09-16T00:07:35.115027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614631+0000","last_change":"2024-09-16T00:07:33.632261+0000","last_active":"2024-09-16T00:07:34.614631+0000","last_peered":"2024-09-16T00:07:34.614631+0000","last_clean":"2024-09-16T00:07:34.614631+0000","last_became_active":"2024-09-16T00:07:33.631944+0000","last_became_peered":"2024-09-16T00:07:33.631944+0000","last_unstale":"2024-09-16T00:07:34.614631+0000","last_undegraded":"2024-09-16T00:07:34.614631+0000","last_fullsized":"2024-09-16T00:07:34.614631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:35.052417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114641+0000","last_change":"2024-09-16T00:07:33.628273+0000","last_active":"2024-09-16T00:07:35.114641+0000","last_peered":"2024-09-16T00:07:35.114641+0000","last_clean":"2024-09-16T00:07:35.114641+0000","last_became_active":"2024-09-16T00:07:33.627960+0000","last_became_peered":"2024-09-16T00:07:33.627960+0000","last_unstale":"2024-09-16T00:07:35.114641+0000","last_undegraded":"2024-09-16T00:07:35.114641+0000","last_fullsized":"2024-09-16T00:07:35.114641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:36.451688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114782+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:07:35.114782+0000","last_peered":"2024-09-16T00:07:35.114782+0000","last_clean":"2024-09-16T00:07:35.114782+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:07:35.114782+0000","last_undegraded":"2024-09-16T00:07:35.114782+0000","last_fullsized":"2024-09-16T00:07:35.114782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114664+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:07:35.114664+0000","last_peered":"2024-09-16T00:07:35.114664+0000","last_clean":"2024-09-16T00:07:35.114664+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:07:35.114664+0000","last_undegraded":"2024-09-16T00:07:35.114664+0000","last_fullsized":"2024-09-16T00:07:35.114664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614471+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:07:34.614471+0000","last_peered":"2024-09-16T00:07:34.614471+0000","last_clean":"2024-09-16T00:07:34.614471+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:07:34.614471+0000","last_undegraded":"2024-09-16T00:07:34.614471+0000","last_fullsized":"2024-09-16T00:07:34.614471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615697+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:07:34.615697+0000","last_peered":"2024-09-16T00:07:34.615697+0000","last_clean":"2024-09-16T00:07:34.615697+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:07:34.615697+0000","last_undegraded":"2024-09-16T00:07:34.615697+0000","last_fullsized":"2024-09-16T00:07:34.615697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613100+0000","last_change":"2024-09-16T00:07:33.630797+0000","last_active":"2024-09-16T00:07:34.613100+0000","last_peered":"2024-09-16T00:07:34.613100+0000","last_clean":"2024-09-16T00:07:34.613100+0000","last_became_active":"2024-09-16T00:07:33.630491+0000","last_became_peered":"2024-09-16T00:07:33.630491+0000","last_unstale":"2024-09-16T00:07:34.613100+0000","last_undegraded":"2024-09-16T00:07:34.613100+0000","last_fullsized":"2024-09-16T00:07:34.613100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:20.097626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115840+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:07:35.115840+0000","last_peered":"2024-09-16T00:07:35.115840+0000","last_clean":"2024-09-16T00:07:35.115840+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:07:35.115840+0000","last_undegraded":"2024-09-16T00:07:35.115840+0000","last_fullsized":"2024-09-16T00:07:35.115840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114960+0000","last_change":"2024-09-16T00:07:33.638554+0000","last_active":"2024-09-16T00:07:35.114960+0000","last_peered":"2024-09-16T00:07:35.114960+0000","last_clean":"2024-09-16T00:07:35.114960+0000","last_became_active":"2024-09-16T00:07:33.638263+0000","last_became_peered":"2024-09-16T00:07:33.638263+0000","last_unstale":"2024-09-16T00:07:35.114960+0000","last_undegraded":"2024-09-16T00:07:35.114960+0000","last_fullsized":"2024-09-16T00:07:35.114960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115286+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:07:35.115286+0000","last_peered":"2024-09-16T00:07:35.115286+0000","last_clean":"2024-09-16T00:07:35.115286+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:07:35.115286+0000","last_undegraded":"2024-09-16T00:07:35.115286+0000","last_fullsized":"2024-09-16T00:07:35.115286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615450+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:07:34.615450+0000","last_peered":"2024-09-16T00:07:34.615450+0000","last_clean":"2024-09-16T00:07:34.615450+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:07:34.615450+0000","last_undegraded":"2024-09-16T00:07:34.615450+0000","last_fullsized":"2024-09-16T00:07:34.615450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115015+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:07:35.115015+0000","last_peered":"2024-09-16T00:07:35.115015+0000","last_clean":"2024-09-16T00:07:35.115015+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:07:35.115015+0000","last_undegraded":"2024-09-16T00:07:35.115015+0000","last_fullsized":"2024-09-16T00:07:35.115015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614616+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:07:34.614616+0000","last_peered":"2024-09-16T00:07:34.614616+0000","last_clean":"2024-09-16T00:07:34.614616+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:07:34.614616+0000","last_undegraded":"2024-09-16T00:07:34.614616+0000","last_fullsized":"2024-09-16T00:07:34.614616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114694+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:07:35.114694+0000","last_peered":"2024-09-16T00:07:35.114694+0000","last_clean":"2024-09-16T00:07:35.114694+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:07:35.114694+0000","last_undegraded":"2024-09-16T00:07:35.114694+0000","last_fullsized":"2024-09-16T00:07:35.114694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115306+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:07:35.115306+0000","last_peered":"2024-09-16T00:07:35.115306+0000","last_clean":"2024-09-16T00:07:35.115306+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:07:35.115306+0000","last_undegraded":"2024-09-16T00:07:35.115306+0000","last_fullsized":"2024-09-16T00:07:35.115306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115543+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:07:35.115543+0000","last_peered":"2024-09-16T00:07:35.115543+0000","last_clean":"2024-09-16T00:07:35.115543+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:07:35.115543+0000","last_undegraded":"2024-09-16T00:07:35.115543+0000","last_fullsized":"2024-09-16T00:07:35.115543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114985+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:07:35.114985+0000","last_peered":"2024-09-16T00:07:35.114985+0000","last_clean":"2024-09-16T00:07:35.114985+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:07:35.114985+0000","last_undegraded":"2024-09-16T00:07:35.114985+0000","last_fullsized":"2024-09-16T00:07:35.114985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115448+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:07:35.115448+0000","last_peered":"2024-09-16T00:07:35.115448+0000","last_clean":"2024-09-16T00:07:35.115448+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:07:35.115448+0000","last_undegraded":"2024-09-16T00:07:35.115448+0000","last_fullsized":"2024-09-16T00:07:35.115448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615338+0000","last_change":"2024-09-16T00:07:33.627789+0000","last_active":"2024-09-16T00:07:34.615338+0000","last_peered":"2024-09-16T00:07:34.615338+0000","last_clean":"2024-09-16T00:07:34.615338+0000","last_became_active":"2024-09-16T00:07:33.627500+0000","last_became_peered":"2024-09-16T00:07:33.627500+0000","last_unstale":"2024-09-16T00:07:34.615338+0000","last_undegraded":"2024-09-16T00:07:34.615338+0000","last_fullsized":"2024-09-16T00:07:34.615338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613050+0000","last_change":"2024-09-16T00:07:33.621107+0000","last_active":"2024-09-16T00:07:34.613050+0000","last_peered":"2024-09-16T00:07:34.613050+0000","last_clean":"2024-09-16T00:07:34.613050+0000","last_became_active":"2024-09-16T00:07:33.620858+0000","last_became_peered":"2024-09-16T00:07:33.620858+0000","last_unstale":"2024-09-16T00:07:34.613050+0000","last_undegraded":"2024-09-16T00:07:34.613050+0000","last_fullsized":"2024-09-16T00:07:34.613050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:30.176079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115031+0000","last_change":"2024-09-16T00:07:33.633578+0000","last_active":"2024-09-16T00:07:35.115031+0000","last_peered":"2024-09-16T00:07:35.115031+0000","last_clean":"2024-09-16T00:07:35.115031+0000","last_became_active":"2024-09-16T00:07:33.633310+0000","last_became_peered":"2024-09-16T00:07:33.633310+0000","last_unstale":"2024-09-16T00:07:35.115031+0000","last_undegraded":"2024-09-16T00:07:35.115031+0000","last_fullsized":"2024-09-16T00:07:35.115031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614556+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:07:34.614556+0000","last_peered":"2024-09-16T00:07:34.614556+0000","last_clean":"2024-09-16T00:07:34.614556+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:07:34.614556+0000","last_undegraded":"2024-09-16T00:07:34.614556+0000","last_fullsized":"2024-09-16T00:07:34.614556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.059131+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:07:36.059131+0000","last_peered":"2024-09-16T00:07:36.059131+0000","last_clean":"2024-09-16T00:07:36.059131+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:07:36.059131+0000","last_undegraded":"2024-09-16T00:07:36.059131+0000","last_fullsized":"2024-09-16T00:07:36.059131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115509+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:07:35.115509+0000","last_peered":"2024-09-16T00:07:35.115509+0000","last_clean":"2024-09-16T00:07:35.115509+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:07:35.115509+0000","last_undegraded":"2024-09-16T00:07:35.115509+0000","last_fullsized":"2024-09-16T00:07:35.115509+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114845+0000","last_change":"2024-09-16T00:07:32.626800+0000","last_active":"2024-09-16T00:07:35.114845+0000","last_peered":"2024-09-16T00:07:35.114845+0000","last_clean":"2024-09-16T00:07:35.114845+0000","last_became_active":"2024-09-16T00:07:32.626511+0000","last_became_peered":"2024-09-16T00:07:32.626511+0000","last_unstale":"2024-09-16T00:07:35.114845+0000","last_undegraded":"2024-09-16T00:07:35.114845+0000","last_fullsized":"2024-09-16T00:07:35.114845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115156+0000","last_change":"2024-09-16T00:07:32.629539+0000","last_active":"2024-09-16T00:07:35.115156+0000","last_peered":"2024-09-16T00:07:35.115156+0000","last_clean":"2024-09-16T00:07:35.115156+0000","last_became_active":"2024-09-16T00:07:32.629186+0000","last_became_peered":"2024-09-16T00:07:32.629186+0000","last_unstale":"2024-09-16T00:07:35.115156+0000","last_undegraded":"2024-09-16T00:07:35.115156+0000","last_fullsized":"2024-09-16T00:07:35.115156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:59:32.860208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114535+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:07:35.114535+0000","last_peered":"2024-09-16T00:07:35.114535+0000","last_clean":"2024-09-16T00:07:35.114535+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:07:35.114535+0000","last_undegraded":"2024-09-16T00:07:35.114535+0000","last_fullsized":"2024-09-16T00:07:35.114535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614668+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:07:34.614668+0000","last_peered":"2024-09-16T00:07:34.614668+0000","last_clean":"2024-09-16T00:07:34.614668+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:07:34.614668+0000","last_undegraded":"2024-09-16T00:07:34.614668+0000","last_fullsized":"2024-09-16T00:07:34.614668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114527+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:07:35.114527+0000","last_peered":"2024-09-16T00:07:35.114527+0000","last_clean":"2024-09-16T00:07:35.114527+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:07:35.114527+0000","last_undegraded":"2024-09-16T00:07:35.114527+0000","last_fullsized":"2024-09-16T00:07:35.114527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115138+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:07:35.115138+0000","last_peered":"2024-09-16T00:07:35.115138+0000","last_clean":"2024-09-16T00:07:35.115138+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:07:35.115138+0000","last_undegraded":"2024-09-16T00:07:35.115138+0000","last_fullsized":"2024-09-16T00:07:35.115138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615408+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:07:34.615408+0000","last_peered":"2024-09-16T00:07:34.615408+0000","last_clean":"2024-09-16T00:07:34.615408+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:07:34.615408+0000","last_undegraded":"2024-09-16T00:07:34.615408+0000","last_fullsized":"2024-09-16T00:07:34.615408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615356+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:07:34.615356+0000","last_peered":"2024-09-16T00:07:34.615356+0000","last_clean":"2024-09-16T00:07:34.615356+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:07:34.615356+0000","last_undegraded":"2024-09-16T00:07:34.615356+0000","last_fullsized":"2024-09-16T00:07:34.615356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115277+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:07:35.115277+0000","last_peered":"2024-09-16T00:07:35.115277+0000","last_clean":"2024-09-16T00:07:35.115277+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:07:35.115277+0000","last_undegraded":"2024-09-16T00:07:35.115277+0000","last_fullsized":"2024-09-16T00:07:35.115277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.619745+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:07:38.619745+0000","last_peered":"2024-09-16T00:07:38.619745+0000","last_clean":"2024-09-16T00:07:38.619745+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:07:38.619745+0000","last_undegraded":"2024-09-16T00:07:38.619745+0000","last_fullsized":"2024-09-16T00:07:38.619745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614648+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:07:34.614648+0000","last_peered":"2024-09-16T00:07:34.614648+0000","last_clean":"2024-09-16T00:07:34.614648+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:07:34.614648+0000","last_undegraded":"2024-09-16T00:07:34.614648+0000","last_fullsized":"2024-09-16T00:07:34.614648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.622263+0000","last_change":"2024-09-16T00:07:32.629132+0000","last_active":"2024-09-16T00:07:38.622263+0000","last_peered":"2024-09-16T00:07:38.622263+0000","last_clean":"2024-09-16T00:07:38.622263+0000","last_became_active":"2024-09-16T00:07:32.628803+0000","last_became_peered":"2024-09-16T00:07:32.628803+0000","last_unstale":"2024-09-16T00:07:38.622263+0000","last_undegraded":"2024-09-16T00:07:38.622263+0000","last_fullsized":"2024-09-16T00:07:38.622263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115230+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:07:35.115230+0000","last_peered":"2024-09-16T00:07:35.115230+0000","last_clean":"2024-09-16T00:07:35.115230+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:07:35.115230+0000","last_undegraded":"2024-09-16T00:07:35.115230+0000","last_fullsized":"2024-09-16T00:07:35.115230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620127+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:07:38.620127+0000","last_peered":"2024-09-16T00:07:38.620127+0000","last_clean":"2024-09-16T00:07:38.620127+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:07:38.620127+0000","last_undegraded":"2024-09-16T00:07:38.620127+0000","last_fullsized":"2024-09-16T00:07:38.620127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115121+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:07:35.115121+0000","last_peered":"2024-09-16T00:07:35.115121+0000","last_clean":"2024-09-16T00:07:35.115121+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:07:35.115121+0000","last_undegraded":"2024-09-16T00:07:35.115121+0000","last_fullsized":"2024-09-16T00:07:35.115121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615462+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:07:34.615462+0000","last_peered":"2024-09-16T00:07:34.615462+0000","last_clean":"2024-09-16T00:07:34.615462+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:07:34.615462+0000","last_undegraded":"2024-09-16T00:07:34.615462+0000","last_fullsized":"2024-09-16T00:07:34.615462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115766+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:07:35.115766+0000","last_peered":"2024-09-16T00:07:35.115766+0000","last_clean":"2024-09-16T00:07:35.115766+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:07:35.115766+0000","last_undegraded":"2024-09-16T00:07:35.115766+0000","last_fullsized":"2024-09-16T00:07:35.115766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114802+0000","last_change":"2024-09-16T00:07:32.626856+0000","last_active":"2024-09-16T00:07:35.114802+0000","last_peered":"2024-09-16T00:07:35.114802+0000","last_clean":"2024-09-16T00:07:35.114802+0000","last_became_active":"2024-09-16T00:07:32.626532+0000","last_became_peered":"2024-09-16T00:07:32.626532+0000","last_unstale":"2024-09-16T00:07:35.114802+0000","last_undegraded":"2024-09-16T00:07:35.114802+0000","last_fullsized":"2024-09-16T00:07:35.114802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114886+0000","last_change":"2024-09-16T00:07:33.632870+0000","last_active":"2024-09-16T00:07:35.114886+0000","last_peered":"2024-09-16T00:07:35.114886+0000","last_clean":"2024-09-16T00:07:35.114886+0000","last_became_active":"2024-09-16T00:07:33.632608+0000","last_became_peered":"2024-09-16T00:07:33.632608+0000","last_unstale":"2024-09-16T00:07:35.114886+0000","last_undegraded":"2024-09-16T00:07:35.114886+0000","last_fullsized":"2024-09-16T00:07:35.114886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.621550+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:07:38.621550+0000","last_peered":"2024-09-16T00:07:38.621550+0000","last_clean":"2024-09-16T00:07:38.621550+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:07:38.621550+0000","last_undegraded":"2024-09-16T00:07:38.621550+0000","last_fullsized":"2024-09-16T00:07:38.621550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115409+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:07:35.115409+0000","last_peered":"2024-09-16T00:07:35.115409+0000","last_clean":"2024-09-16T00:07:35.115409+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:07:35.115409+0000","last_undegraded":"2024-09-16T00:07:35.115409+0000","last_fullsized":"2024-09-16T00:07:35.115409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614761+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:07:34.614761+0000","last_peered":"2024-09-16T00:07:34.614761+0000","last_clean":"2024-09-16T00:07:34.614761+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:07:34.614761+0000","last_undegraded":"2024-09-16T00:07:34.614761+0000","last_fullsized":"2024-09-16T00:07:34.614761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115480+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:07:35.115480+0000","last_peered":"2024-09-16T00:07:35.115480+0000","last_clean":"2024-09-16T00:07:35.115480+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:07:35.115480+0000","last_undegraded":"2024-09-16T00:07:35.115480+0000","last_fullsized":"2024-09-16T00:07:35.115480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620942+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:07:38.620942+0000","last_peered":"2024-09-16T00:07:38.620942+0000","last_clean":"2024-09-16T00:07:38.620942+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:07:38.620942+0000","last_undegraded":"2024-09-16T00:07:38.620942+0000","last_fullsized":"2024-09-16T00:07:38.620942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115888+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:07:35.115888+0000","last_peered":"2024-09-16T00:07:35.115888+0000","last_clean":"2024-09-16T00:07:35.115888+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:07:35.115888+0000","last_undegraded":"2024-09-16T00:07:35.115888+0000","last_fullsized":"2024-09-16T00:07:35.115888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.048699+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:07:36.048699+0000","last_peered":"2024-09-16T00:07:36.048699+0000","last_clean":"2024-09-16T00:07:36.048699+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:07:36.048699+0000","last_undegraded":"2024-09-16T00:07:36.048699+0000","last_fullsized":"2024-09-16T00:07:36.048699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615597+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:07:34.615597+0000","last_peered":"2024-09-16T00:07:34.615597+0000","last_clean":"2024-09-16T00:07:34.615597+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:07:34.615597+0000","last_undegraded":"2024-09-16T00:07:34.615597+0000","last_fullsized":"2024-09-16T00:07:34.615597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613057+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:07:34.613057+0000","last_peered":"2024-09-16T00:07:34.613057+0000","last_clean":"2024-09-16T00:07:34.613057+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:07:34.613057+0000","last_undegraded":"2024-09-16T00:07:34.613057+0000","last_fullsized":"2024-09-16T00:07:34.613057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115577+0000","last_change":"2024-09-16T00:07:33.638312+0000","last_active":"2024-09-16T00:07:35.115577+0000","last_peered":"2024-09-16T00:07:35.115577+0000","last_clean":"2024-09-16T00:07:35.115577+0000","last_became_active":"2024-09-16T00:07:33.638049+0000","last_became_peered":"2024-09-16T00:07:33.638049+0000","last_unstale":"2024-09-16T00:07:35.115577+0000","last_undegraded":"2024-09-16T00:07:35.115577+0000","last_fullsized":"2024-09-16T00:07:35.115577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115169+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:07:35.115169+0000","last_peered":"2024-09-16T00:07:35.115169+0000","last_clean":"2024-09-16T00:07:35.115169+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:07:35.115169+0000","last_undegraded":"2024-09-16T00:07:35.115169+0000","last_fullsized":"2024-09-16T00:07:35.115169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614834+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:07:34.614834+0000","last_peered":"2024-09-16T00:07:34.614834+0000","last_clean":"2024-09-16T00:07:34.614834+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:07:34.614834+0000","last_undegraded":"2024-09-16T00:07:34.614834+0000","last_fullsized":"2024-09-16T00:07:34.614834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114892+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:07:35.114892+0000","last_peered":"2024-09-16T00:07:35.114892+0000","last_clean":"2024-09-16T00:07:35.114892+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:07:35.114892+0000","last_undegraded":"2024-09-16T00:07:35.114892+0000","last_fullsized":"2024-09-16T00:07:35.114892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115065+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:07:35.115065+0000","last_peered":"2024-09-16T00:07:35.115065+0000","last_clean":"2024-09-16T00:07:35.115065+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:07:35.115065+0000","last_undegraded":"2024-09-16T00:07:35.115065+0000","last_fullsized":"2024-09-16T00:07:35.115065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115288+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:07:35.115288+0000","last_peered":"2024-09-16T00:07:35.115288+0000","last_clean":"2024-09-16T00:07:35.115288+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:07:35.115288+0000","last_undegraded":"2024-09-16T00:07:35.115288+0000","last_fullsized":"2024-09-16T00:07:35.115288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.612974+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:07:34.612974+0000","last_peered":"2024-09-16T00:07:34.612974+0000","last_clean":"2024-09-16T00:07:34.612974+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:07:34.612974+0000","last_undegraded":"2024-09-16T00:07:34.612974+0000","last_fullsized":"2024-09-16T00:07:34.612974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114905+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:07:35.114905+0000","last_peered":"2024-09-16T00:07:35.114905+0000","last_clean":"2024-09-16T00:07:35.114905+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:07:35.114905+0000","last_undegraded":"2024-09-16T00:07:35.114905+0000","last_fullsized":"2024-09-16T00:07:35.114905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615621+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:07:34.615621+0000","last_peered":"2024-09-16T00:07:34.615621+0000","last_clean":"2024-09-16T00:07:34.615621+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:07:34.615621+0000","last_undegraded":"2024-09-16T00:07:34.615621+0000","last_fullsized":"2024-09-16T00:07:34.615621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615008+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:07:34.615008+0000","last_peered":"2024-09-16T00:07:34.615008+0000","last_clean":"2024-09-16T00:07:34.615008+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:07:34.615008+0000","last_undegraded":"2024-09-16T00:07:34.615008+0000","last_fullsized":"2024-09-16T00:07:34.615008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115398+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:07:35.115398+0000","last_peered":"2024-09-16T00:07:35.115398+0000","last_clean":"2024-09-16T00:07:35.115398+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:07:35.115398+0000","last_undegraded":"2024-09-16T00:07:35.115398+0000","last_fullsized":"2024-09-16T00:07:35.115398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615631+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:07:34.615631+0000","last_peered":"2024-09-16T00:07:34.615631+0000","last_clean":"2024-09-16T00:07:34.615631+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:07:34.615631+0000","last_undegraded":"2024-09-16T00:07:34.615631+0000","last_fullsized":"2024-09-16T00:07:34.615631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115814+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:07:35.115814+0000","last_peered":"2024-09-16T00:07:35.115814+0000","last_clean":"2024-09-16T00:07:35.115814+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:07:35.115814+0000","last_undegraded":"2024-09-16T00:07:35.115814+0000","last_fullsized":"2024-09-16T00:07:35.115814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115305+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:07:35.115305+0000","last_peered":"2024-09-16T00:07:35.115305+0000","last_clean":"2024-09-16T00:07:35.115305+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:07:35.115305+0000","last_undegraded":"2024-09-16T00:07:35.115305+0000","last_fullsized":"2024-09-16T00:07:35.115305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.048722+0000","last_change":"2024-09-16T00:07:32.627236+0000","last_active":"2024-09-16T00:07:36.048722+0000","last_peered":"2024-09-16T00:07:36.048722+0000","last_clean":"2024-09-16T00:07:36.048722+0000","last_became_active":"2024-09-16T00:07:32.626743+0000","last_became_peered":"2024-09-16T00:07:32.626743+0000","last_unstale":"2024-09-16T00:07:36.048722+0000","last_undegraded":"2024-09-16T00:07:36.048722+0000","last_fullsized":"2024-09-16T00:07:36.048722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114835+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:07:35.114835+0000","last_peered":"2024-09-16T00:07:35.114835+0000","last_clean":"2024-09-16T00:07:35.114835+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:07:35.114835+0000","last_undegraded":"2024-09-16T00:07:35.114835+0000","last_fullsized":"2024-09-16T00:07:35.114835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620513+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:07:38.620513+0000","last_peered":"2024-09-16T00:07:38.620513+0000","last_clean":"2024-09-16T00:07:38.620513+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:07:38.620513+0000","last_undegraded":"2024-09-16T00:07:38.620513+0000","last_fullsized":"2024-09-16T00:07:38.620513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114804+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:07:35.114804+0000","last_peered":"2024-09-16T00:07:35.114804+0000","last_clean":"2024-09-16T00:07:35.114804+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:07:35.114804+0000","last_undegraded":"2024-09-16T00:07:35.114804+0000","last_fullsized":"2024-09-16T00:07:35.114804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114947+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:07:35.114947+0000","last_peered":"2024-09-16T00:07:35.114947+0000","last_clean":"2024-09-16T00:07:35.114947+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:07:35.114947+0000","last_undegraded":"2024-09-16T00:07:35.114947+0000","last_fullsized":"2024-09-16T00:07:35.114947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614501+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:07:34.614501+0000","last_peered":"2024-09-16T00:07:34.614501+0000","last_clean":"2024-09-16T00:07:34.614501+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:07:34.614501+0000","last_undegraded":"2024-09-16T00:07:34.614501+0000","last_fullsized":"2024-09-16T00:07:34.614501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114765+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:07:35.114765+0000","last_peered":"2024-09-16T00:07:35.114765+0000","last_clean":"2024-09-16T00:07:35.114765+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:07:35.114765+0000","last_undegraded":"2024-09-16T00:07:35.114765+0000","last_fullsized":"2024-09-16T00:07:35.114765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114640+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:07:35.114640+0000","last_peered":"2024-09-16T00:07:35.114640+0000","last_clean":"2024-09-16T00:07:35.114640+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:07:35.114640+0000","last_undegraded":"2024-09-16T00:07:35.114640+0000","last_fullsized":"2024-09-16T00:07:35.114640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115333+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:07:35.115333+0000","last_peered":"2024-09-16T00:07:35.115333+0000","last_clean":"2024-09-16T00:07:35.115333+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:07:35.115333+0000","last_undegraded":"2024-09-16T00:07:35.115333+0000","last_fullsized":"2024-09-16T00:07:35.115333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114803+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:07:35.114803+0000","last_peered":"2024-09-16T00:07:35.114803+0000","last_clean":"2024-09-16T00:07:35.114803+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:07:35.114803+0000","last_undegraded":"2024-09-16T00:07:35.114803+0000","last_fullsized":"2024-09-16T00:07:35.114803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.621871+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:07:38.621871+0000","last_peered":"2024-09-16T00:07:38.621871+0000","last_clean":"2024-09-16T00:07:38.621871+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:07:38.621871+0000","last_undegraded":"2024-09-16T00:07:38.621871+0000","last_fullsized":"2024-09-16T00:07:38.621871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114993+0000","last_change":"2024-09-16T00:07:33.637979+0000","last_active":"2024-09-16T00:07:35.114993+0000","last_peered":"2024-09-16T00:07:35.114993+0000","last_clean":"2024-09-16T00:07:35.114993+0000","last_became_active":"2024-09-16T00:07:33.637713+0000","last_became_peered":"2024-09-16T00:07:33.637713+0000","last_unstale":"2024-09-16T00:07:35.114993+0000","last_undegraded":"2024-09-16T00:07:35.114993+0000","last_fullsized":"2024-09-16T00:07:35.114993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615387+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:07:34.615387+0000","last_peered":"2024-09-16T00:07:34.615387+0000","last_clean":"2024-09-16T00:07:34.615387+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:07:34.615387+0000","last_undegraded":"2024-09-16T00:07:34.615387+0000","last_fullsized":"2024-09-16T00:07:34.615387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.621868+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:07:38.621868+0000","last_peered":"2024-09-16T00:07:38.621868+0000","last_clean":"2024-09-16T00:07:38.621868+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:07:38.621868+0000","last_undegraded":"2024-09-16T00:07:38.621868+0000","last_fullsized":"2024-09-16T00:07:38.621868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615266+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:07:34.615266+0000","last_peered":"2024-09-16T00:07:34.615266+0000","last_clean":"2024-09-16T00:07:34.615266+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:07:34.615266+0000","last_undegraded":"2024-09-16T00:07:34.615266+0000","last_fullsized":"2024-09-16T00:07:34.615266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.624540+0000","last_change":"2024-09-16T00:07:32.632481+0000","last_active":"2024-09-16T00:07:38.624540+0000","last_peered":"2024-09-16T00:07:38.624540+0000","last_clean":"2024-09-16T00:07:38.624540+0000","last_became_active":"2024-09-16T00:07:32.632152+0000","last_became_peered":"2024-09-16T00:07:32.632152+0000","last_unstale":"2024-09-16T00:07:38.624540+0000","last_undegraded":"2024-09-16T00:07:38.624540+0000","last_fullsized":"2024-09-16T00:07:38.624540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114783+0000","last_change":"2024-09-16T00:07:33.629777+0000","last_active":"2024-09-16T00:07:35.114783+0000","last_peered":"2024-09-16T00:07:35.114783+0000","last_clean":"2024-09-16T00:07:35.114783+0000","last_became_active":"2024-09-16T00:07:33.627917+0000","last_became_peered":"2024-09-16T00:07:33.627917+0000","last_unstale":"2024-09-16T00:07:35.114783+0000","last_undegraded":"2024-09-16T00:07:35.114783+0000","last_fullsized":"2024-09-16T00:07:35.114783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:38.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.059155+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:07:36.059155+0000","last_peered":"2024-09-16T00:07:36.059155+0000","last_clean":"2024-09-16T00:07:36.059155+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:07:36.059155+0000","last_undegraded":"2024-09-16T00:07:36.059155+0000","last_fullsized":"2024-09-16T00:07:36.059155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114899+0000","last_change":"2024-09-16T00:07:33.630746+0000","last_active":"2024-09-16T00:07:35.114899+0000","last_peered":"2024-09-16T00:07:35.114899+0000","last_clean":"2024-09-16T00:07:35.114899+0000","last_became_active":"2024-09-16T00:07:33.630480+0000","last_became_peered":"2024-09-16T00:07:33.630480+0000","last_unstale":"2024-09-16T00:07:35.114899+0000","last_undegraded":"2024-09-16T00:07:35.114899+0000","last_fullsized":"2024-09-16T00:07:35.114899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:50.931998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.622293+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:07:38.622293+0000","last_peered":"2024-09-16T00:07:38.622293+0000","last_clean":"2024-09-16T00:07:38.622293+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:07:38.622293+0000","last_undegraded":"2024-09-16T00:07:38.622293+0000","last_fullsized":"2024-09-16T00:07:38.622293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":668946,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":78122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":78156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":669002,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:07:51.843 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:07:51.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.977+0000 7f5eb8131640 1 Processor -- start 2024-09-16T00:07:51.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.977+0000 7f5eb8131640 1 -- start start 2024-09-16T00:07:51.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.978+0000 7f5eb8131640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5eb015e0c0 0x7f5eb015e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.978+0000 7f5eb8131640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5eb00c8e10 0x7f5eb006cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.978+0000 7f5eb8131640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eb00cbf40 con 0x7f5eb00cba00 2024-09-16T00:07:51.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.978+0000 7f5eb8131640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb00c7cb0 con 0x7f5eb015e0c0 2024-09-16T00:07:51.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.978+0000 7f5eb8131640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb00c8270 con 0x7f5eb00c8e10 2024-09-16T00:07:51.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.978+0000 7f5eb5ecd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5eb015e0c0 0x7f5eb015e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.978+0000 7f5eb56cc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5eb00cba00 0x7f5eb00c8700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57836/0 (socket says 172.21.15.139:57836) 2024-09-16T00:07:51.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.978+0000 7f5eb56cc640 1 -- 172.21.15.139:0/899126067 learned_addr learned my addr 172.21.15.139:0/899126067 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:51.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.978+0000 7f5eb66ce640 1 --2- 172.21.15.139:0/899126067 >> v2:172.21.15.164:3300/0 conn(0x7f5eb00c8e10 0x7f5eb006cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.978+0000 7f5eb5ecd640 1 -- 172.21.15.139:0/899126067 >> v2:172.21.15.164:3300/0 conn(0x7f5eb00c8e10 msgr2=0x7f5eb006cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:51.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.978+0000 7f5eb5ecd640 1 --2- 172.21.15.139:0/899126067 >> v2:172.21.15.164:3300/0 conn(0x7f5eb00c8e10 0x7f5eb006cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.978+0000 7f5eb5ecd640 1 -- 172.21.15.139:0/899126067 >> v1:172.21.15.139:6789/0 conn(0x7f5eb00cba00 legacy=0x7f5eb00c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:51.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.978+0000 7f5eb5ecd640 1 -- 172.21.15.139:0/899126067 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb00cc200 con 0x7f5eb015e0c0 2024-09-16T00:07:51.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.978+0000 7f5eb4ecb640 1 -- 172.21.15.139:0/899126067 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2919305149 0 0) 0x7f5eb00cbf40 con 0x7f5eb00cba00 2024-09-16T00:07:51.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.979+0000 7f5eb5ecd640 1 --2- 172.21.15.139:0/899126067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eb015e0c0 0x7f5eb015e4a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00ca260 tx=0x7f5eac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd7c8685ff1c3a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:51.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.979+0000 7f5eb4ecb640 1 -- 172.21.15.139:0/899126067 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac037070 con 0x7f5eb015e0c0 2024-09-16T00:07:51.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.979+0000 7f5eb4ecb640 1 -- 172.21.15.139:0/899126067 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eac032440 con 0x7f5eb015e0c0 2024-09-16T00:07:51.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.979+0000 7f5eb4ecb640 1 -- 172.21.15.139:0/899126067 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac032810 con 0x7f5eb015e0c0 2024-09-16T00:07:51.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.980+0000 7f5eb8131640 1 -- 172.21.15.139:0/899126067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eb015e0c0 msgr2=0x7f5eb015e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:51.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.980+0000 7f5eb8131640 1 --2- 172.21.15.139:0/899126067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eb015e0c0 0x7f5eb015e4a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00ca260 tx=0x7f5eac02a230 comp rx=0 tx=0).stop 2024-09-16T00:07:51.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.980+0000 7f5eb8131640 1 -- 172.21.15.139:0/899126067 shutdown_connections 2024-09-16T00:07:51.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.980+0000 7f5eb8131640 1 --2- 172.21.15.139:0/899126067 >> v2:172.21.15.164:3300/0 conn(0x7f5eb00c8e10 0x7f5eb006cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.980+0000 7f5eb8131640 1 --2- 172.21.15.139:0/899126067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eb015e0c0 0x7f5eb015e4a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.980+0000 7f5eb8131640 1 -- 172.21.15.139:0/899126067 >> 172.21.15.139:0/899126067 conn(0x7f5eb00d03b0 msgr2=0x7f5eb00d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:51.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.980+0000 7f5eb8131640 1 -- 172.21.15.139:0/899126067 shutdown_connections 2024-09-16T00:07:51.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.980+0000 7f5eb8131640 1 -- 172.21.15.139:0/899126067 wait complete. 2024-09-16T00:07:51.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.981+0000 7f5eb8131640 1 Processor -- start 2024-09-16T00:07:51.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.981+0000 7f5eb8131640 1 -- start start 2024-09-16T00:07:51.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb8131640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb00c8e10 0x7f5eb0150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb8131640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5eb00cba00 0x7f5eb014e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb8131640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eb015e0c0 0x7f5eb00c2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb8131640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb015e9e0 con 0x7f5eb015e0c0 2024-09-16T00:07:51.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb8131640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb00c7f00 con 0x7f5eb00c8e10 2024-09-16T00:07:51.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb8131640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb015eb60 con 0x7f5eb00cba00 2024-09-16T00:07:51.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb66ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eb015e0c0 0x7f5eb00c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb66ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eb015e0c0 0x7f5eb00c2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47734/0 (socket says 172.21.15.139:47734) 2024-09-16T00:07:51.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb66ce640 1 -- 172.21.15.139:0/34050716 learned_addr learned my addr 172.21.15.139:0/34050716 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:51.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb5ecd640 1 --2- 172.21.15.139:0/34050716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb00c8e10 0x7f5eb0150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb56cc640 1 --2- 172.21.15.139:0/34050716 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5eb00cba00 0x7f5eb014e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb66ce640 1 -- 172.21.15.139:0/34050716 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5eb00cba00 msgr2=0x7f5eb014e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:51.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb66ce640 1 --2- 172.21.15.139:0/34050716 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5eb00cba00 0x7f5eb014e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb66ce640 1 -- 172.21.15.139:0/34050716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb00c8e10 msgr2=0x7f5eb0150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:51.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb66ce640 1 --2- 172.21.15.139:0/34050716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb00c8e10 0x7f5eb0150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:51.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb66ce640 1 -- 172.21.15.139:0/34050716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb014e850 con 0x7f5eb015e0c0 2024-09-16T00:07:51.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5eb66ce640 1 --2- 172.21.15.139:0/34050716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eb015e0c0 0x7f5eb00c2650 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8007e60 tx=0x7f5ea800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:51.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5ea67fc640 1 -- 172.21.15.139:0/34050716 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8014420 con 0x7f5eb015e0c0 2024-09-16T00:07:51.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.982+0000 7f5ea67fc640 1 -- 172.21.15.139:0/34050716 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea8019070 con 0x7f5eb015e0c0 2024-09-16T00:07:51.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.983+0000 7f5eb8131640 1 -- 172.21.15.139:0/34050716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb01f4500 con 0x7f5eb015e0c0 2024-09-16T00:07:51.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.983+0000 7f5ea67fc640 1 -- 172.21.15.139:0/34050716 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8014820 con 0x7f5eb015e0c0 2024-09-16T00:07:51.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.983+0000 7f5eb8131640 1 -- 172.21.15.139:0/34050716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb014f970 con 0x7f5eb015e0c0 2024-09-16T00:07:51.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.984+0000 7f5eb8131640 1 -- 172.21.15.139:0/34050716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb015dbd0 con 0x7f5eb015e0c0 2024-09-16T00:07:51.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.984+0000 7f5ea67fc640 1 -- 172.21.15.139:0/34050716 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5ea80149c0 con 0x7f5eb015e0c0 2024-09-16T00:07:51.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.984+0000 7f5ea67fc640 1 --2- 172.21.15.139:0/34050716 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e9406c2b0 0x7f5e9406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:51.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.984+0000 7f5ea67fc640 1 -- 172.21.15.139:0/34050716 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5ea808bcd0 con 0x7f5eb015e0c0 2024-09-16T00:07:51.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.984+0000 7f5eb5ecd640 1 --2- 172.21.15.139:0/34050716 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e9406c2b0 0x7f5e9406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:51.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.985+0000 7f5eb5ecd640 1 --2- 172.21.15.139:0/34050716 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e9406c2b0 0x7f5e9406e770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5eac034000 tx=0x7f5eac02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:51.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:51.989+0000 7f5ea67fc640 1 -- 172.21.15.139:0/34050716 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5ea805a3b0 con 0x7f5eb015e0c0 2024-09-16T00:07:52.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.151+0000 7f5eb8131640 1 -- 172.21.15.139:0/34050716 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5eb01541c0 con 0x7f5e9406c2b0 2024-09-16T00:07:52.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.157+0000 7f5ea67fc640 1 -- 172.21.15.139:0/34050716 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251914 (secure 0 0 0) 0x7f5eb01541c0 con 0x7f5e9406c2b0 2024-09-16T00:07:52.162 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:52.163 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:07:52.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.160+0000 7f5eb8131640 1 -- 172.21.15.139:0/34050716 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e9406c2b0 msgr2=0x7f5e9406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.160+0000 7f5eb8131640 1 --2- 172.21.15.139:0/34050716 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e9406c2b0 0x7f5e9406e770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5eac034000 tx=0x7f5eac02e000 comp rx=0 tx=0).stop 2024-09-16T00:07:52.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.160+0000 7f5eb8131640 1 -- 172.21.15.139:0/34050716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eb015e0c0 msgr2=0x7f5eb00c2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.160+0000 7f5eb8131640 1 --2- 172.21.15.139:0/34050716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eb015e0c0 0x7f5eb00c2650 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8007e60 tx=0x7f5ea800e4b0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.160+0000 7f5eb8131640 1 -- 172.21.15.139:0/34050716 shutdown_connections 2024-09-16T00:07:52.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.160+0000 7f5eb8131640 1 --2- 172.21.15.139:0/34050716 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e9406c2b0 0x7f5e9406e770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.160+0000 7f5eb8131640 1 --2- 172.21.15.139:0/34050716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5eb015e0c0 0x7f5eb00c2650 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.160+0000 7f5eb8131640 1 --2- 172.21.15.139:0/34050716 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5eb00cba00 0x7f5eb014e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.160+0000 7f5eb8131640 1 --2- 172.21.15.139:0/34050716 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5eb00c8e10 0x7f5eb0150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.160+0000 7f5eb8131640 1 -- 172.21.15.139:0/34050716 >> 172.21.15.139:0/34050716 conn(0x7f5eb00d03b0 msgr2=0x7f5eb006bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:52.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.160+0000 7f5eb8131640 1 -- 172.21.15.139:0/34050716 shutdown_connections 2024-09-16T00:07:52.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.160+0000 7f5eb8131640 1 -- 172.21.15.139:0/34050716 wait complete. 2024-09-16T00:07:52.173 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-16T00:07:52.029094+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218112,"kb_used_data":1920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756608,"statfs":{"total":773094113280,"available":772870766592,"internally_reserved":0,"allocated":1966080,"data_stored":1809524,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002217"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115528+0000","last_change":"2024-09-16T00:07:33.635096+0000","last_active":"2024-09-16T00:07:35.115528+0000","last_peered":"2024-09-16T00:07:35.115528+0000","last_clean":"2024-09-16T00:07:35.115528+0000","last_became_active":"2024-09-16T00:07:33.634750+0000","last_became_peered":"2024-09-16T00:07:33.634750+0000","last_unstale":"2024-09-16T00:07:35.115528+0000","last_undegraded":"2024-09-16T00:07:35.115528+0000","last_fullsized":"2024-09-16T00:07:35.115528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115558+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:07:35.115558+0000","last_peered":"2024-09-16T00:07:35.115558+0000","last_clean":"2024-09-16T00:07:35.115558+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:07:35.115558+0000","last_undegraded":"2024-09-16T00:07:35.115558+0000","last_fullsized":"2024-09-16T00:07:35.115558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115392+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:07:35.115392+0000","last_peered":"2024-09-16T00:07:35.115392+0000","last_clean":"2024-09-16T00:07:35.115392+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:07:35.115392+0000","last_undegraded":"2024-09-16T00:07:35.115392+0000","last_fullsized":"2024-09-16T00:07:35.115392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615167+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:07:34.615167+0000","last_peered":"2024-09-16T00:07:34.615167+0000","last_clean":"2024-09-16T00:07:34.615167+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:07:34.615167+0000","last_undegraded":"2024-09-16T00:07:34.615167+0000","last_fullsized":"2024-09-16T00:07:34.615167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115724+0000","last_change":"2024-09-16T00:07:33.630029+0000","last_active":"2024-09-16T00:07:35.115724+0000","last_peered":"2024-09-16T00:07:35.115724+0000","last_clean":"2024-09-16T00:07:35.115724+0000","last_became_active":"2024-09-16T00:07:33.629712+0000","last_became_peered":"2024-09-16T00:07:33.629712+0000","last_unstale":"2024-09-16T00:07:35.115724+0000","last_undegraded":"2024-09-16T00:07:35.115724+0000","last_fullsized":"2024-09-16T00:07:35.115724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614973+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:07:34.614973+0000","last_peered":"2024-09-16T00:07:34.614973+0000","last_clean":"2024-09-16T00:07:34.614973+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:07:34.614973+0000","last_undegraded":"2024-09-16T00:07:34.614973+0000","last_fullsized":"2024-09-16T00:07:34.614973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115522+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:07:35.115522+0000","last_peered":"2024-09-16T00:07:35.115522+0000","last_clean":"2024-09-16T00:07:35.115522+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:07:35.115522+0000","last_undegraded":"2024-09-16T00:07:35.115522+0000","last_fullsized":"2024-09-16T00:07:35.115522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114492+0000","last_change":"2024-09-16T00:07:33.624708+0000","last_active":"2024-09-16T00:07:35.114492+0000","last_peered":"2024-09-16T00:07:35.114492+0000","last_clean":"2024-09-16T00:07:35.114492+0000","last_became_active":"2024-09-16T00:07:33.624458+0000","last_became_peered":"2024-09-16T00:07:33.624458+0000","last_unstale":"2024-09-16T00:07:35.114492+0000","last_undegraded":"2024-09-16T00:07:35.114492+0000","last_fullsized":"2024-09-16T00:07:35.114492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:48.052488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115751+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:07:35.115751+0000","last_peered":"2024-09-16T00:07:35.115751+0000","last_clean":"2024-09-16T00:07:35.115751+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:07:35.115751+0000","last_undegraded":"2024-09-16T00:07:35.115751+0000","last_fullsized":"2024-09-16T00:07:35.115751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115614+0000","last_change":"2024-09-16T00:07:33.635973+0000","last_active":"2024-09-16T00:07:35.115614+0000","last_peered":"2024-09-16T00:07:35.115614+0000","last_clean":"2024-09-16T00:07:35.115614+0000","last_became_active":"2024-09-16T00:07:33.635486+0000","last_became_peered":"2024-09-16T00:07:33.635486+0000","last_unstale":"2024-09-16T00:07:35.115614+0000","last_undegraded":"2024-09-16T00:07:35.115614+0000","last_fullsized":"2024-09-16T00:07:35.115614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115027+0000","last_change":"2024-09-16T00:07:33.636971+0000","last_active":"2024-09-16T00:07:35.115027+0000","last_peered":"2024-09-16T00:07:35.115027+0000","last_clean":"2024-09-16T00:07:35.115027+0000","last_became_active":"2024-09-16T00:07:33.635760+0000","last_became_peered":"2024-09-16T00:07:33.635760+0000","last_unstale":"2024-09-16T00:07:35.115027+0000","last_undegraded":"2024-09-16T00:07:35.115027+0000","last_fullsized":"2024-09-16T00:07:35.115027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614631+0000","last_change":"2024-09-16T00:07:33.632261+0000","last_active":"2024-09-16T00:07:34.614631+0000","last_peered":"2024-09-16T00:07:34.614631+0000","last_clean":"2024-09-16T00:07:34.614631+0000","last_became_active":"2024-09-16T00:07:33.631944+0000","last_became_peered":"2024-09-16T00:07:33.631944+0000","last_unstale":"2024-09-16T00:07:34.614631+0000","last_undegraded":"2024-09-16T00:07:34.614631+0000","last_fullsized":"2024-09-16T00:07:34.614631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:35.052417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114641+0000","last_change":"2024-09-16T00:07:33.628273+0000","last_active":"2024-09-16T00:07:35.114641+0000","last_peered":"2024-09-16T00:07:35.114641+0000","last_clean":"2024-09-16T00:07:35.114641+0000","last_became_active":"2024-09-16T00:07:33.627960+0000","last_became_peered":"2024-09-16T00:07:33.627960+0000","last_unstale":"2024-09-16T00:07:35.114641+0000","last_undegraded":"2024-09-16T00:07:35.114641+0000","last_fullsized":"2024-09-16T00:07:35.114641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:36.451688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114782+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:07:35.114782+0000","last_peered":"2024-09-16T00:07:35.114782+0000","last_clean":"2024-09-16T00:07:35.114782+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:07:35.114782+0000","last_undegraded":"2024-09-16T00:07:35.114782+0000","last_fullsized":"2024-09-16T00:07:35.114782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114664+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:07:35.114664+0000","last_peered":"2024-09-16T00:07:35.114664+0000","last_clean":"2024-09-16T00:07:35.114664+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:07:35.114664+0000","last_undegraded":"2024-09-16T00:07:35.114664+0000","last_fullsized":"2024-09-16T00:07:35.114664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614471+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:07:34.614471+0000","last_peered":"2024-09-16T00:07:34.614471+0000","last_clean":"2024-09-16T00:07:34.614471+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:07:34.614471+0000","last_undegraded":"2024-09-16T00:07:34.614471+0000","last_fullsized":"2024-09-16T00:07:34.614471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615697+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:07:34.615697+0000","last_peered":"2024-09-16T00:07:34.615697+0000","last_clean":"2024-09-16T00:07:34.615697+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:07:34.615697+0000","last_undegraded":"2024-09-16T00:07:34.615697+0000","last_fullsized":"2024-09-16T00:07:34.615697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613100+0000","last_change":"2024-09-16T00:07:33.630797+0000","last_active":"2024-09-16T00:07:34.613100+0000","last_peered":"2024-09-16T00:07:34.613100+0000","last_clean":"2024-09-16T00:07:34.613100+0000","last_became_active":"2024-09-16T00:07:33.630491+0000","last_became_peered":"2024-09-16T00:07:33.630491+0000","last_unstale":"2024-09-16T00:07:34.613100+0000","last_undegraded":"2024-09-16T00:07:34.613100+0000","last_fullsized":"2024-09-16T00:07:34.613100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:20.097626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115840+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:07:35.115840+0000","last_peered":"2024-09-16T00:07:35.115840+0000","last_clean":"2024-09-16T00:07:35.115840+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:07:35.115840+0000","last_undegraded":"2024-09-16T00:07:35.115840+0000","last_fullsized":"2024-09-16T00:07:35.115840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114960+0000","last_change":"2024-09-16T00:07:33.638554+0000","last_active":"2024-09-16T00:07:35.114960+0000","last_peered":"2024-09-16T00:07:35.114960+0000","last_clean":"2024-09-16T00:07:35.114960+0000","last_became_active":"2024-09-16T00:07:33.638263+0000","last_became_peered":"2024-09-16T00:07:33.638263+0000","last_unstale":"2024-09-16T00:07:35.114960+0000","last_undegraded":"2024-09-16T00:07:35.114960+0000","last_fullsized":"2024-09-16T00:07:35.114960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115286+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:07:35.115286+0000","last_peered":"2024-09-16T00:07:35.115286+0000","last_clean":"2024-09-16T00:07:35.115286+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:07:35.115286+0000","last_undegraded":"2024-09-16T00:07:35.115286+0000","last_fullsized":"2024-09-16T00:07:35.115286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615450+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:07:34.615450+0000","last_peered":"2024-09-16T00:07:34.615450+0000","last_clean":"2024-09-16T00:07:34.615450+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:07:34.615450+0000","last_undegraded":"2024-09-16T00:07:34.615450+0000","last_fullsized":"2024-09-16T00:07:34.615450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115015+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:07:35.115015+0000","last_peered":"2024-09-16T00:07:35.115015+0000","last_clean":"2024-09-16T00:07:35.115015+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:07:35.115015+0000","last_undegraded":"2024-09-16T00:07:35.115015+0000","last_fullsized":"2024-09-16T00:07:35.115015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614616+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:07:34.614616+0000","last_peered":"2024-09-16T00:07:34.614616+0000","last_clean":"2024-09-16T00:07:34.614616+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:07:34.614616+0000","last_undegraded":"2024-09-16T00:07:34.614616+0000","last_fullsized":"2024-09-16T00:07:34.614616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114694+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:07:35.114694+0000","last_peered":"2024-09-16T00:07:35.114694+0000","last_clean":"2024-09-16T00:07:35.114694+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:07:35.114694+0000","last_undegraded":"2024-09-16T00:07:35.114694+0000","last_fullsized":"2024-09-16T00:07:35.114694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115306+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:07:35.115306+0000","last_peered":"2024-09-16T00:07:35.115306+0000","last_clean":"2024-09-16T00:07:35.115306+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:07:35.115306+0000","last_undegraded":"2024-09-16T00:07:35.115306+0000","last_fullsized":"2024-09-16T00:07:35.115306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115543+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:07:35.115543+0000","last_peered":"2024-09-16T00:07:35.115543+0000","last_clean":"2024-09-16T00:07:35.115543+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:07:35.115543+0000","last_undegraded":"2024-09-16T00:07:35.115543+0000","last_fullsized":"2024-09-16T00:07:35.115543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114985+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:07:35.114985+0000","last_peered":"2024-09-16T00:07:35.114985+0000","last_clean":"2024-09-16T00:07:35.114985+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:07:35.114985+0000","last_undegraded":"2024-09-16T00:07:35.114985+0000","last_fullsized":"2024-09-16T00:07:35.114985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115448+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:07:35.115448+0000","last_peered":"2024-09-16T00:07:35.115448+0000","last_clean":"2024-09-16T00:07:35.115448+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:07:35.115448+0000","last_undegraded":"2024-09-16T00:07:35.115448+0000","last_fullsized":"2024-09-16T00:07:35.115448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615338+0000","last_change":"2024-09-16T00:07:33.627789+0000","last_active":"2024-09-16T00:07:34.615338+0000","last_peered":"2024-09-16T00:07:34.615338+0000","last_clean":"2024-09-16T00:07:34.615338+0000","last_became_active":"2024-09-16T00:07:33.627500+0000","last_became_peered":"2024-09-16T00:07:33.627500+0000","last_unstale":"2024-09-16T00:07:34.615338+0000","last_undegraded":"2024-09-16T00:07:34.615338+0000","last_fullsized":"2024-09-16T00:07:34.615338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613050+0000","last_change":"2024-09-16T00:07:33.621107+0000","last_active":"2024-09-16T00:07:34.613050+0000","last_peered":"2024-09-16T00:07:34.613050+0000","last_clean":"2024-09-16T00:07:34.613050+0000","last_became_active":"2024-09-16T00:07:33.620858+0000","last_became_peered":"2024-09-16T00:07:33.620858+0000","last_unstale":"2024-09-16T00:07:34.613050+0000","last_undegraded":"2024-09-16T00:07:34.613050+0000","last_fullsized":"2024-09-16T00:07:34.613050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:30.176079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115031+0000","last_change":"2024-09-16T00:07:33.633578+0000","last_active":"2024-09-16T00:07:35.115031+0000","last_peered":"2024-09-16T00:07:35.115031+0000","last_clean":"2024-09-16T00:07:35.115031+0000","last_became_active":"2024-09-16T00:07:33.633310+0000","last_became_peered":"2024-09-16T00:07:33.633310+0000","last_unstale":"2024-09-16T00:07:35.115031+0000","last_undegraded":"2024-09-16T00:07:35.115031+0000","last_fullsized":"2024-09-16T00:07:35.115031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614556+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:07:34.614556+0000","last_peered":"2024-09-16T00:07:34.614556+0000","last_clean":"2024-09-16T00:07:34.614556+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:07:34.614556+0000","last_undegraded":"2024-09-16T00:07:34.614556+0000","last_fullsized":"2024-09-16T00:07:34.614556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.059131+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:07:36.059131+0000","last_peered":"2024-09-16T00:07:36.059131+0000","last_clean":"2024-09-16T00:07:36.059131+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:07:36.059131+0000","last_undegraded":"2024-09-16T00:07:36.059131+0000","last_fullsized":"2024-09-16T00:07:36.059131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115509+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:07:35.115509+0000","last_peered":"2024-09-16T00:07:35.115509+0000","last_clean":"2024-09-16T00:07:35.115509+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:07:35.115509+0000","last_undegraded":"2024-09-16T00:07:35.115509+0000","last_fullsized":"2024-09-16T00:07:35.115509+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114845+0000","last_change":"2024-09-16T00:07:32.626800+0000","last_active":"2024-09-16T00:07:35.114845+0000","last_peered":"2024-09-16T00:07:35.114845+0000","last_clean":"2024-09-16T00:07:35.114845+0000","last_became_active":"2024-09-16T00:07:32.626511+0000","last_became_peered":"2024-09-16T00:07:32.626511+0000","last_unstale":"2024-09-16T00:07:35.114845+0000","last_undegraded":"2024-09-16T00:07:35.114845+0000","last_fullsized":"2024-09-16T00:07:35.114845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115156+0000","last_change":"2024-09-16T00:07:32.629539+0000","last_active":"2024-09-16T00:07:35.115156+0000","last_peered":"2024-09-16T00:07:35.115156+0000","last_clean":"2024-09-16T00:07:35.115156+0000","last_became_active":"2024-09-16T00:07:32.629186+0000","last_became_peered":"2024-09-16T00:07:32.629186+0000","last_unstale":"2024-09-16T00:07:35.115156+0000","last_undegraded":"2024-09-16T00:07:35.115156+0000","last_fullsized":"2024-09-16T00:07:35.115156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:59:32.860208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114535+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:07:35.114535+0000","last_peered":"2024-09-16T00:07:35.114535+0000","last_clean":"2024-09-16T00:07:35.114535+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:07:35.114535+0000","last_undegraded":"2024-09-16T00:07:35.114535+0000","last_fullsized":"2024-09-16T00:07:35.114535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614668+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:07:34.614668+0000","last_peered":"2024-09-16T00:07:34.614668+0000","last_clean":"2024-09-16T00:07:34.614668+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:07:34.614668+0000","last_undegraded":"2024-09-16T00:07:34.614668+0000","last_fullsized":"2024-09-16T00:07:34.614668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114527+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:07:35.114527+0000","last_peered":"2024-09-16T00:07:35.114527+0000","last_clean":"2024-09-16T00:07:35.114527+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:07:35.114527+0000","last_undegraded":"2024-09-16T00:07:35.114527+0000","last_fullsized":"2024-09-16T00:07:35.114527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115138+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:07:35.115138+0000","last_peered":"2024-09-16T00:07:35.115138+0000","last_clean":"2024-09-16T00:07:35.115138+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:07:35.115138+0000","last_undegraded":"2024-09-16T00:07:35.115138+0000","last_fullsized":"2024-09-16T00:07:35.115138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615408+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:07:34.615408+0000","last_peered":"2024-09-16T00:07:34.615408+0000","last_clean":"2024-09-16T00:07:34.615408+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:07:34.615408+0000","last_undegraded":"2024-09-16T00:07:34.615408+0000","last_fullsized":"2024-09-16T00:07:34.615408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615356+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:07:34.615356+0000","last_peered":"2024-09-16T00:07:34.615356+0000","last_clean":"2024-09-16T00:07:34.615356+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:07:34.615356+0000","last_undegraded":"2024-09-16T00:07:34.615356+0000","last_fullsized":"2024-09-16T00:07:34.615356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115277+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:07:35.115277+0000","last_peered":"2024-09-16T00:07:35.115277+0000","last_clean":"2024-09-16T00:07:35.115277+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:07:35.115277+0000","last_undegraded":"2024-09-16T00:07:35.115277+0000","last_fullsized":"2024-09-16T00:07:35.115277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.619745+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:07:38.619745+0000","last_peered":"2024-09-16T00:07:38.619745+0000","last_clean":"2024-09-16T00:07:38.619745+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:07:38.619745+0000","last_undegraded":"2024-09-16T00:07:38.619745+0000","last_fullsized":"2024-09-16T00:07:38.619745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614648+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:07:34.614648+0000","last_peered":"2024-09-16T00:07:34.614648+0000","last_clean":"2024-09-16T00:07:34.614648+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:07:34.614648+0000","last_undegraded":"2024-09-16T00:07:34.614648+0000","last_fullsized":"2024-09-16T00:07:34.614648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.622263+0000","last_change":"2024-09-16T00:07:32.629132+0000","last_active":"2024-09-16T00:07:38.622263+0000","last_peered":"2024-09-16T00:07:38.622263+0000","last_clean":"2024-09-16T00:07:38.622263+0000","last_became_active":"2024-09-16T00:07:32.628803+0000","last_became_peered":"2024-09-16T00:07:32.628803+0000","last_unstale":"2024-09-16T00:07:38.622263+0000","last_undegraded":"2024-09-16T00:07:38.622263+0000","last_fullsized":"2024-09-16T00:07:38.622263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115230+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:07:35.115230+0000","last_peered":"2024-09-16T00:07:35.115230+0000","last_clean":"2024-09-16T00:07:35.115230+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:07:35.115230+0000","last_undegraded":"2024-09-16T00:07:35.115230+0000","last_fullsized":"2024-09-16T00:07:35.115230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620127+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:07:38.620127+0000","last_peered":"2024-09-16T00:07:38.620127+0000","last_clean":"2024-09-16T00:07:38.620127+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:07:38.620127+0000","last_undegraded":"2024-09-16T00:07:38.620127+0000","last_fullsized":"2024-09-16T00:07:38.620127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115121+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:07:35.115121+0000","last_peered":"2024-09-16T00:07:35.115121+0000","last_clean":"2024-09-16T00:07:35.115121+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:07:35.115121+0000","last_undegraded":"2024-09-16T00:07:35.115121+0000","last_fullsized":"2024-09-16T00:07:35.115121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615462+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:07:34.615462+0000","last_peered":"2024-09-16T00:07:34.615462+0000","last_clean":"2024-09-16T00:07:34.615462+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:07:34.615462+0000","last_undegraded":"2024-09-16T00:07:34.615462+0000","last_fullsized":"2024-09-16T00:07:34.615462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115766+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:07:35.115766+0000","last_peered":"2024-09-16T00:07:35.115766+0000","last_clean":"2024-09-16T00:07:35.115766+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:07:35.115766+0000","last_undegraded":"2024-09-16T00:07:35.115766+0000","last_fullsized":"2024-09-16T00:07:35.115766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114802+0000","last_change":"2024-09-16T00:07:32.626856+0000","last_active":"2024-09-16T00:07:35.114802+0000","last_peered":"2024-09-16T00:07:35.114802+0000","last_clean":"2024-09-16T00:07:35.114802+0000","last_became_active":"2024-09-16T00:07:32.626532+0000","last_became_peered":"2024-09-16T00:07:32.626532+0000","last_unstale":"2024-09-16T00:07:35.114802+0000","last_undegraded":"2024-09-16T00:07:35.114802+0000","last_fullsized":"2024-09-16T00:07:35.114802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114886+0000","last_change":"2024-09-16T00:07:33.632870+0000","last_active":"2024-09-16T00:07:35.114886+0000","last_peered":"2024-09-16T00:07:35.114886+0000","last_clean":"2024-09-16T00:07:35.114886+0000","last_became_active":"2024-09-16T00:07:33.632608+0000","last_became_peered":"2024-09-16T00:07:33.632608+0000","last_unstale":"2024-09-16T00:07:35.114886+0000","last_undegraded":"2024-09-16T00:07:35.114886+0000","last_fullsized":"2024-09-16T00:07:35.114886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.621550+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:07:38.621550+0000","last_peered":"2024-09-16T00:07:38.621550+0000","last_clean":"2024-09-16T00:07:38.621550+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:07:38.621550+0000","last_undegraded":"2024-09-16T00:07:38.621550+0000","last_fullsized":"2024-09-16T00:07:38.621550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115409+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:07:35.115409+0000","last_peered":"2024-09-16T00:07:35.115409+0000","last_clean":"2024-09-16T00:07:35.115409+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:07:35.115409+0000","last_undegraded":"2024-09-16T00:07:35.115409+0000","last_fullsized":"2024-09-16T00:07:35.115409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614761+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:07:34.614761+0000","last_peered":"2024-09-16T00:07:34.614761+0000","last_clean":"2024-09-16T00:07:34.614761+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:07:34.614761+0000","last_undegraded":"2024-09-16T00:07:34.614761+0000","last_fullsized":"2024-09-16T00:07:34.614761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115480+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:07:35.115480+0000","last_peered":"2024-09-16T00:07:35.115480+0000","last_clean":"2024-09-16T00:07:35.115480+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:07:35.115480+0000","last_undegraded":"2024-09-16T00:07:35.115480+0000","last_fullsized":"2024-09-16T00:07:35.115480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620942+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:07:38.620942+0000","last_peered":"2024-09-16T00:07:38.620942+0000","last_clean":"2024-09-16T00:07:38.620942+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:07:38.620942+0000","last_undegraded":"2024-09-16T00:07:38.620942+0000","last_fullsized":"2024-09-16T00:07:38.620942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115888+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:07:35.115888+0000","last_peered":"2024-09-16T00:07:35.115888+0000","last_clean":"2024-09-16T00:07:35.115888+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:07:35.115888+0000","last_undegraded":"2024-09-16T00:07:35.115888+0000","last_fullsized":"2024-09-16T00:07:35.115888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.048699+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:07:36.048699+0000","last_peered":"2024-09-16T00:07:36.048699+0000","last_clean":"2024-09-16T00:07:36.048699+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:07:36.048699+0000","last_undegraded":"2024-09-16T00:07:36.048699+0000","last_fullsized":"2024-09-16T00:07:36.048699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615597+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:07:34.615597+0000","last_peered":"2024-09-16T00:07:34.615597+0000","last_clean":"2024-09-16T00:07:34.615597+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:07:34.615597+0000","last_undegraded":"2024-09-16T00:07:34.615597+0000","last_fullsized":"2024-09-16T00:07:34.615597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613057+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:07:34.613057+0000","last_peered":"2024-09-16T00:07:34.613057+0000","last_clean":"2024-09-16T00:07:34.613057+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:07:34.613057+0000","last_undegraded":"2024-09-16T00:07:34.613057+0000","last_fullsized":"2024-09-16T00:07:34.613057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115577+0000","last_change":"2024-09-16T00:07:33.638312+0000","last_active":"2024-09-16T00:07:35.115577+0000","last_peered":"2024-09-16T00:07:35.115577+0000","last_clean":"2024-09-16T00:07:35.115577+0000","last_became_active":"2024-09-16T00:07:33.638049+0000","last_became_peered":"2024-09-16T00:07:33.638049+0000","last_unstale":"2024-09-16T00:07:35.115577+0000","last_undegraded":"2024-09-16T00:07:35.115577+0000","last_fullsized":"2024-09-16T00:07:35.115577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115169+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:07:35.115169+0000","last_peered":"2024-09-16T00:07:35.115169+0000","last_clean":"2024-09-16T00:07:35.115169+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:07:35.115169+0000","last_undegraded":"2024-09-16T00:07:35.115169+0000","last_fullsized":"2024-09-16T00:07:35.115169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614834+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:07:34.614834+0000","last_peered":"2024-09-16T00:07:34.614834+0000","last_clean":"2024-09-16T00:07:34.614834+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:07:34.614834+0000","last_undegraded":"2024-09-16T00:07:34.614834+0000","last_fullsized":"2024-09-16T00:07:34.614834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114892+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:07:35.114892+0000","last_peered":"2024-09-16T00:07:35.114892+0000","last_clean":"2024-09-16T00:07:35.114892+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:07:35.114892+0000","last_undegraded":"2024-09-16T00:07:35.114892+0000","last_fullsized":"2024-09-16T00:07:35.114892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115065+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:07:35.115065+0000","last_peered":"2024-09-16T00:07:35.115065+0000","last_clean":"2024-09-16T00:07:35.115065+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:07:35.115065+0000","last_undegraded":"2024-09-16T00:07:35.115065+0000","last_fullsized":"2024-09-16T00:07:35.115065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115288+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:07:35.115288+0000","last_peered":"2024-09-16T00:07:35.115288+0000","last_clean":"2024-09-16T00:07:35.115288+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:07:35.115288+0000","last_undegraded":"2024-09-16T00:07:35.115288+0000","last_fullsized":"2024-09-16T00:07:35.115288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.612974+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:07:34.612974+0000","last_peered":"2024-09-16T00:07:34.612974+0000","last_clean":"2024-09-16T00:07:34.612974+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:07:34.612974+0000","last_undegraded":"2024-09-16T00:07:34.612974+0000","last_fullsized":"2024-09-16T00:07:34.612974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114905+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:07:35.114905+0000","last_peered":"2024-09-16T00:07:35.114905+0000","last_clean":"2024-09-16T00:07:35.114905+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:07:35.114905+0000","last_undegraded":"2024-09-16T00:07:35.114905+0000","last_fullsized":"2024-09-16T00:07:35.114905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615621+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:07:34.615621+0000","last_peered":"2024-09-16T00:07:34.615621+0000","last_clean":"2024-09-16T00:07:34.615621+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:07:34.615621+0000","last_undegraded":"2024-09-16T00:07:34.615621+0000","last_fullsized":"2024-09-16T00:07:34.615621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615008+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:07:34.615008+0000","last_peered":"2024-09-16T00:07:34.615008+0000","last_clean":"2024-09-16T00:07:34.615008+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:07:34.615008+0000","last_undegraded":"2024-09-16T00:07:34.615008+0000","last_fullsized":"2024-09-16T00:07:34.615008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115398+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:07:35.115398+0000","last_peered":"2024-09-16T00:07:35.115398+0000","last_clean":"2024-09-16T00:07:35.115398+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:07:35.115398+0000","last_undegraded":"2024-09-16T00:07:35.115398+0000","last_fullsized":"2024-09-16T00:07:35.115398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615631+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:07:34.615631+0000","last_peered":"2024-09-16T00:07:34.615631+0000","last_clean":"2024-09-16T00:07:34.615631+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:07:34.615631+0000","last_undegraded":"2024-09-16T00:07:34.615631+0000","last_fullsized":"2024-09-16T00:07:34.615631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115814+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:07:35.115814+0000","last_peered":"2024-09-16T00:07:35.115814+0000","last_clean":"2024-09-16T00:07:35.115814+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:07:35.115814+0000","last_undegraded":"2024-09-16T00:07:35.115814+0000","last_fullsized":"2024-09-16T00:07:35.115814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115305+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:07:35.115305+0000","last_peered":"2024-09-16T00:07:35.115305+0000","last_clean":"2024-09-16T00:07:35.115305+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:07:35.115305+0000","last_undegraded":"2024-09-16T00:07:35.115305+0000","last_fullsized":"2024-09-16T00:07:35.115305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.048722+0000","last_change":"2024-09-16T00:07:32.627236+0000","last_active":"2024-09-16T00:07:36.048722+0000","last_peered":"2024-09-16T00:07:36.048722+0000","last_clean":"2024-09-16T00:07:36.048722+0000","last_became_active":"2024-09-16T00:07:32.626743+0000","last_became_peered":"2024-09-16T00:07:32.626743+0000","last_unstale":"2024-09-16T00:07:36.048722+0000","last_undegraded":"2024-09-16T00:07:36.048722+0000","last_fullsized":"2024-09-16T00:07:36.048722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114835+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:07:35.114835+0000","last_peered":"2024-09-16T00:07:35.114835+0000","last_clean":"2024-09-16T00:07:35.114835+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:07:35.114835+0000","last_undegraded":"2024-09-16T00:07:35.114835+0000","last_fullsized":"2024-09-16T00:07:35.114835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620513+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:07:38.620513+0000","last_peered":"2024-09-16T00:07:38.620513+0000","last_clean":"2024-09-16T00:07:38.620513+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:07:38.620513+0000","last_undegraded":"2024-09-16T00:07:38.620513+0000","last_fullsized":"2024-09-16T00:07:38.620513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114804+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:07:35.114804+0000","last_peered":"2024-09-16T00:07:35.114804+0000","last_clean":"2024-09-16T00:07:35.114804+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:07:35.114804+0000","last_undegraded":"2024-09-16T00:07:35.114804+0000","last_fullsized":"2024-09-16T00:07:35.114804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114947+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:07:35.114947+0000","last_peered":"2024-09-16T00:07:35.114947+0000","last_clean":"2024-09-16T00:07:35.114947+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:07:35.114947+0000","last_undegraded":"2024-09-16T00:07:35.114947+0000","last_fullsized":"2024-09-16T00:07:35.114947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614501+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:07:34.614501+0000","last_peered":"2024-09-16T00:07:34.614501+0000","last_clean":"2024-09-16T00:07:34.614501+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:07:34.614501+0000","last_undegraded":"2024-09-16T00:07:34.614501+0000","last_fullsized":"2024-09-16T00:07:34.614501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114765+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:07:35.114765+0000","last_peered":"2024-09-16T00:07:35.114765+0000","last_clean":"2024-09-16T00:07:35.114765+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:07:35.114765+0000","last_undegraded":"2024-09-16T00:07:35.114765+0000","last_fullsized":"2024-09-16T00:07:35.114765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114640+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:07:35.114640+0000","last_peered":"2024-09-16T00:07:35.114640+0000","last_clean":"2024-09-16T00:07:35.114640+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:07:35.114640+0000","last_undegraded":"2024-09-16T00:07:35.114640+0000","last_fullsized":"2024-09-16T00:07:35.114640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115333+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:07:35.115333+0000","last_peered":"2024-09-16T00:07:35.115333+0000","last_clean":"2024-09-16T00:07:35.115333+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:07:35.115333+0000","last_undegraded":"2024-09-16T00:07:35.115333+0000","last_fullsized":"2024-09-16T00:07:35.115333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114803+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:07:35.114803+0000","last_peered":"2024-09-16T00:07:35.114803+0000","last_clean":"2024-09-16T00:07:35.114803+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:07:35.114803+0000","last_undegraded":"2024-09-16T00:07:35.114803+0000","last_fullsized":"2024-09-16T00:07:35.114803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.621871+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:07:38.621871+0000","last_peered":"2024-09-16T00:07:38.621871+0000","last_clean":"2024-09-16T00:07:38.621871+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:07:38.621871+0000","last_undegraded":"2024-09-16T00:07:38.621871+0000","last_fullsized":"2024-09-16T00:07:38.621871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114993+0000","last_change":"2024-09-16T00:07:33.637979+0000","last_active":"2024-09-16T00:07:35.114993+0000","last_peered":"2024-09-16T00:07:35.114993+0000","last_clean":"2024-09-16T00:07:35.114993+0000","last_became_active":"2024-09-16T00:07:33.637713+0000","last_became_peered":"2024-09-16T00:07:33.637713+0000","last_unstale":"2024-09-16T00:07:35.114993+0000","last_undegraded":"2024-09-16T00:07:35.114993+0000","last_fullsized":"2024-09-16T00:07:35.114993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615387+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:07:34.615387+0000","last_peered":"2024-09-16T00:07:34.615387+0000","last_clean":"2024-09-16T00:07:34.615387+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:07:34.615387+0000","last_undegraded":"2024-09-16T00:07:34.615387+0000","last_fullsized":"2024-09-16T00:07:34.615387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.621868+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:07:38.621868+0000","last_peered":"2024-09-16T00:07:38.621868+0000","last_clean":"2024-09-16T00:07:38.621868+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:07:38.621868+0000","last_undegraded":"2024-09-16T00:07:38.621868+0000","last_fullsized":"2024-09-16T00:07:38.621868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615266+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:07:34.615266+0000","last_peered":"2024-09-16T00:07:34.615266+0000","last_clean":"2024-09-16T00:07:34.615266+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:07:34.615266+0000","last_undegraded":"2024-09-16T00:07:34.615266+0000","last_fullsized":"2024-09-16T00:07:34.615266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.624540+0000","last_change":"2024-09-16T00:07:32.632481+0000","last_active":"2024-09-16T00:07:38.624540+0000","last_peered":"2024-09-16T00:07:38.624540+0000","last_clean":"2024-09-16T00:07:38.624540+0000","last_became_active":"2024-09-16T00:07:32.632152+0000","last_became_peered":"2024-09-16T00:07:32.632152+0000","last_unstale":"2024-09-16T00:07:38.624540+0000","last_undegraded":"2024-09-16T00:07:38.624540+0000","last_fullsized":"2024-09-16T00:07:38.624540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114783+0000","last_change":"2024-09-16T00:07:33.629777+0000","last_active":"2024-09-16T00:07:35.114783+0000","last_peered":"2024-09-16T00:07:35.114783+0000","last_clean":"2024-09-16T00:07:35.114783+0000","last_became_active":"2024-09-16T00:07:33.627917+0000","last_became_peered":"2024-09-16T00:07:33.627917+0000","last_unstale":"2024-09-16T00:07:35.114783+0000","last_undegraded":"2024-09-16T00:07:35.114783+0000","last_fullsized":"2024-09-16T00:07:35.114783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:38.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.059155+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:07:36.059155+0000","last_peered":"2024-09-16T00:07:36.059155+0000","last_clean":"2024-09-16T00:07:36.059155+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:07:36.059155+0000","last_undegraded":"2024-09-16T00:07:36.059155+0000","last_fullsized":"2024-09-16T00:07:36.059155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114899+0000","last_change":"2024-09-16T00:07:33.630746+0000","last_active":"2024-09-16T00:07:35.114899+0000","last_peered":"2024-09-16T00:07:35.114899+0000","last_clean":"2024-09-16T00:07:35.114899+0000","last_became_active":"2024-09-16T00:07:33.630480+0000","last_became_peered":"2024-09-16T00:07:33.630480+0000","last_unstale":"2024-09-16T00:07:35.114899+0000","last_undegraded":"2024-09-16T00:07:35.114899+0000","last_fullsized":"2024-09-16T00:07:35.114899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:50.931998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.622293+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:07:38.622293+0000","last_peered":"2024-09-16T00:07:38.622293+0000","last_clean":"2024-09-16T00:07:38.622293+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:07:38.622293+0000","last_undegraded":"2024-09-16T00:07:38.622293+0000","last_fullsized":"2024-09-16T00:07:38.622293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":668946,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":78122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":78156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":669002,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:07:52.177 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T00:07:52.177 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-16T00:07:52.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.310+0000 7fe1162af640 1 Processor -- start 2024-09-16T00:07:52.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.310+0000 7fe1162af640 1 -- start start 2024-09-16T00:07:52.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.311+0000 7fe1162af640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe11006f150 0x7fe110166110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.311+0000 7fe1162af640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1100c9890 con 0x7fe1100c7cb0 2024-09-16T00:07:52.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.311+0000 7fe1162af640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe11006c540 con 0x7fe11006ed70 2024-09-16T00:07:52.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.311+0000 7fe1162af640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe11006c820 con 0x7fe11006f150 2024-09-16T00:07:52.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.311+0000 7fe10f7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe1100c7cb0 0x7fe110154580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57846/0 (socket says 172.21.15.139:57846) 2024-09-16T00:07:52.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.311+0000 7fe10f7fe640 1 -- 172.21.15.139:0/2681655433 learned_addr learned my addr 172.21.15.139:0/2681655433 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:52.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.311+0000 7fe11484c640 1 --2- 172.21.15.139:0/2681655433 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe11006f150 0x7fe110166110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.311+0000 7fe10effd640 1 -- 172.21.15.139:0/2681655433 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4077240564 0 0) 0x7fe1100c9890 con 0x7fe1100c7cb0 2024-09-16T00:07:52.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.311+0000 7fe10effd640 1 -- 172.21.15.139:0/2681655433 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0f8003520 con 0x7fe1100c7cb0 2024-09-16T00:07:52.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.312+0000 7fe10effd640 1 -- 172.21.15.139:0/2681655433 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1264416136 0 0) 0x7fe11006c540 con 0x7fe11006ed70 2024-09-16T00:07:52.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.312+0000 7fe10effd640 1 -- 172.21.15.139:0/2681655433 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1100c9890 con 0x7fe11006ed70 2024-09-16T00:07:52.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.312+0000 7fe11484c640 1 -- 172.21.15.139:0/2681655433 >> v1:172.21.15.164:6789/0 conn(0x7fe11006ed70 legacy=0x7fe11015a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.312+0000 7fe11484c640 1 -- 172.21.15.139:0/2681655433 >> v1:172.21.15.139:6789/0 conn(0x7fe1100c7cb0 legacy=0x7fe110154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.312+0000 7fe10effd640 1 -- 172.21.15.139:0/2681655433 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2243938980 0 0) 0x7fe0f8003520 con 0x7fe1100c7cb0 2024-09-16T00:07:52.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.312+0000 7fe11484c640 1 -- 172.21.15.139:0/2681655433 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11006cb00 con 0x7fe11006f150 2024-09-16T00:07:52.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.312+0000 7fe11484c640 1 --2- 172.21.15.139:0/2681655433 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe11006f150 0x7fe110166110 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe108009df0 tx=0x7fe10802a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1312058a3c0bcec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:52.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.313+0000 7fe10effd640 1 -- 172.21.15.139:0/2681655433 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104001670 con 0x7fe11006f150 2024-09-16T00:07:52.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.313+0000 7fe10effd640 1 -- 172.21.15.139:0/2681655433 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe108007540 con 0x7fe11006f150 2024-09-16T00:07:52.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.313+0000 7fe10effd640 1 -- 172.21.15.139:0/2681655433 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1080078f0 con 0x7fe11006f150 2024-09-16T00:07:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.313+0000 7fe1162af640 1 -- 172.21.15.139:0/2681655433 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe11006f150 msgr2=0x7fe110166110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.313+0000 7fe1162af640 1 --2- 172.21.15.139:0/2681655433 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe11006f150 0x7fe110166110 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe108009df0 tx=0x7fe10802a730 comp rx=0 tx=0).stop 2024-09-16T00:07:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.313+0000 7fe1162af640 1 -- 172.21.15.139:0/2681655433 shutdown_connections 2024-09-16T00:07:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.313+0000 7fe1162af640 1 --2- 172.21.15.139:0/2681655433 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe11006f150 0x7fe110166110 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.313+0000 7fe1162af640 1 -- 172.21.15.139:0/2681655433 >> 172.21.15.139:0/2681655433 conn(0x7fe110067950 msgr2=0x7fe110067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.313+0000 7fe1162af640 1 -- 172.21.15.139:0/2681655433 shutdown_connections 2024-09-16T00:07:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.313+0000 7fe1162af640 1 -- 172.21.15.139:0/2681655433 wait complete. 2024-09-16T00:07:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.314+0000 7fe1162af640 1 Processor -- start 2024-09-16T00:07:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.314+0000 7fe1162af640 1 -- start start 2024-09-16T00:07:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.315+0000 7fe1162af640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe11006ed70 0x7fe11014d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.315+0000 7fe1162af640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe11006f150 0x7fe1100cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.315+0000 7fe1162af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1100c7cb0 0x7fe1101e4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.315+0000 7fe1162af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe11015b060 con 0x7fe1100c7cb0 2024-09-16T00:07:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.315+0000 7fe1162af640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe1100c9b70 con 0x7fe11006ed70 2024-09-16T00:07:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.315+0000 7fe1162af640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe11015b1e0 con 0x7fe11006f150 2024-09-16T00:07:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.315+0000 7fe11484c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1100c7cb0 0x7fe1101e4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.315+0000 7fe11484c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1100c7cb0 0x7fe1101e4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47744/0 (socket says 172.21.15.139:47744) 2024-09-16T00:07:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.315+0000 7fe11484c640 1 -- 172.21.15.139:0/2431902273 learned_addr learned my addr 172.21.15.139:0/2431902273 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.315+0000 7fe10f7fe640 1 --2- 172.21.15.139:0/2431902273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe11006f150 0x7fe1100cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.315+0000 7fe10ffff640 1 --2- 172.21.15.139:0/2431902273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe11006ed70 0x7fe11014d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.316+0000 7fe11484c640 1 -- 172.21.15.139:0/2431902273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe11006f150 msgr2=0x7fe1100cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.316+0000 7fe11484c640 1 --2- 172.21.15.139:0/2431902273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe11006f150 0x7fe1100cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.316+0000 7fe11484c640 1 -- 172.21.15.139:0/2431902273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe11006ed70 msgr2=0x7fe11014d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.316+0000 7fe11484c640 1 --2- 172.21.15.139:0/2431902273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe11006ed70 0x7fe11014d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.316+0000 7fe11484c640 1 -- 172.21.15.139:0/2431902273 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1101f0390 con 0x7fe1100c7cb0 2024-09-16T00:07:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.316+0000 7fe11484c640 1 --2- 172.21.15.139:0/2431902273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1100c7cb0 0x7fe1101e4370 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe108009d90 tx=0x7fe108004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:52.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.316+0000 7fe10cff9640 1 -- 172.21.15.139:0/2431902273 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108009450 con 0x7fe1100c7cb0 2024-09-16T00:07:52.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.316+0000 7fe10cff9640 1 -- 172.21.15.139:0/2431902273 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe108004240 con 0x7fe1100c7cb0 2024-09-16T00:07:52.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.316+0000 7fe1162af640 1 -- 172.21.15.139:0/2431902273 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe11014ddc0 con 0x7fe1100c7cb0 2024-09-16T00:07:52.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.316+0000 7fe10cff9640 1 -- 172.21.15.139:0/2431902273 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1080045f0 con 0x7fe1100c7cb0 2024-09-16T00:07:52.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.316+0000 7fe1162af640 1 -- 172.21.15.139:0/2431902273 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1101f0910 con 0x7fe1100c7cb0 2024-09-16T00:07:52.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.317+0000 7fe1162af640 1 -- 172.21.15.139:0/2431902273 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe11006bb20 con 0x7fe1100c7cb0 2024-09-16T00:07:52.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.317+0000 7fe10cff9640 1 -- 172.21.15.139:0/2431902273 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe108039040 con 0x7fe1100c7cb0 2024-09-16T00:07:52.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.318+0000 7fe10cff9640 1 --2- 172.21.15.139:0/2431902273 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe0fc06c2b0 0x7fe0fc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.318+0000 7fe10cff9640 1 -- 172.21.15.139:0/2431902273 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe1080ac8c0 con 0x7fe1100c7cb0 2024-09-16T00:07:52.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.318+0000 7fe10ffff640 1 --2- 172.21.15.139:0/2431902273 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe0fc06c2b0 0x7fe0fc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.318+0000 7fe10ffff640 1 --2- 172.21.15.139:0/2431902273 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe0fc06c2b0 0x7fe0fc06e770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe1000011e0 tx=0x7fe100009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:52.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.322+0000 7fe10cff9640 1 -- 172.21.15.139:0/2431902273 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe108040030 con 0x7fe1100c7cb0 2024-09-16T00:07:52.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.489+0000 7fe1162af640 1 -- 172.21.15.139:0/2431902273 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fe11014e520 con 0x7fe1100c7cb0 2024-09-16T00:07:52.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.490+0000 7fe10cff9640 1 -- 172.21.15.139:0/2431902273 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v22) ==== 56+0+3263 (secure 0 0 0) 0x7fe10807ab00 con 0x7fe1100c7cb0 2024-09-16T00:07:52.492 INFO:teuthology.orchestra.run.smithi139.stdout:epoch 22 2024-09-16T00:07:52.492 INFO:teuthology.orchestra.run.smithi139.stdout:fsid 2d3366e5-6a59-4b59-93b2-615e72ac7015 2024-09-16T00:07:52.492 INFO:teuthology.orchestra.run.smithi139.stdout:created 2024-09-16T00:07:09.967833+0000 2024-09-16T00:07:52.492 INFO:teuthology.orchestra.run.smithi139.stdout:modified 2024-09-16T00:07:34.609954+0000 2024-09-16T00:07:52.492 INFO:teuthology.orchestra.run.smithi139.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T00:07:52.492 INFO:teuthology.orchestra.run.smithi139.stdout:crush_version 6 2024-09-16T00:07:52.492 INFO:teuthology.orchestra.run.smithi139.stdout:full_ratio 0.95 2024-09-16T00:07:52.492 INFO:teuthology.orchestra.run.smithi139.stdout:backfillfull_ratio 0.9 2024-09-16T00:07:52.492 INFO:teuthology.orchestra.run.smithi139.stdout:nearfull_ratio 0.85 2024-09-16T00:07:52.493 INFO:teuthology.orchestra.run.smithi139.stdout:require_min_compat_client luminous 2024-09-16T00:07:52.493 INFO:teuthology.orchestra.run.smithi139.stdout:min_compat_client jewel 2024-09-16T00:07:52.493 INFO:teuthology.orchestra.run.smithi139.stdout:require_osd_release squid 2024-09-16T00:07:52.493 INFO:teuthology.orchestra.run.smithi139.stdout:stretch_mode_enabled false 2024-09-16T00:07:52.493 INFO:teuthology.orchestra.run.smithi139.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-16T00:07:52.493 INFO:teuthology.orchestra.run.smithi139.stdout:pool 2 'cephfs_metadata' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.50 2024-09-16T00:07:52.493 INFO:teuthology.orchestra.run.smithi139.stdout:pool 3 'cephfs_data' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-16T00:07:52.493 INFO:teuthology.orchestra.run.smithi139.stdout:max_osd 8 2024-09-16T00:07:52.493 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] [v2:172.21.15.139:6818/1974901438,v1:172.21.15.139:6819/1974901438] exists,up 83699a05-4937-43c3-aefc-b669bed9bb6f 2024-09-16T00:07:52.493 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] [v2:172.21.15.139:6810/2232380998,v1:172.21.15.139:6811/2232380998] exists,up 9c04991e-9695-4bbb-8011-bf613bf404b3 2024-09-16T00:07:52.493 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2 up in weight 1 up_from 13 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] [v2:172.21.15.139:6802/4180588905,v1:172.21.15.139:6803/4180588905] exists,up 8048fc64-72b3-4866-ac62-6a8b2bf6f5c8 2024-09-16T00:07:52.493 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] [v2:172.21.15.139:6828/3947768492,v1:172.21.15.139:6829/3947768492] exists,up c34ac77b-354c-4884-902d-3e2a84f5eb27 2024-09-16T00:07:52.493 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] [v2:172.21.15.164:6818/642632916,v1:172.21.15.164:6819/642632916] exists,up ba9a7c93-9672-43e1-8c98-2878cdf7f548 2024-09-16T00:07:52.493 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] [v2:172.21.15.164:6802/1051778383,v1:172.21.15.164:6803/1051778383] exists,up 46a0916b-474c-411b-84ad-86f27d4cbf53 2024-09-16T00:07:52.494 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] [v2:172.21.15.164:6826/1908495807,v1:172.21.15.164:6827/1908495807] exists,up 9b6b33cc-d6d9-4a0a-8143-526073a58b93 2024-09-16T00:07:52.494 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] [v2:172.21.15.164:6810/1158838808,v1:172.21.15.164:6811/1158838808] exists,up 50bcd808-3d3e-4a72-88e4-30decb53cfea 2024-09-16T00:07:52.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.492+0000 7fe1162af640 1 -- 172.21.15.139:0/2431902273 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe0fc06c2b0 msgr2=0x7fe0fc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.492+0000 7fe1162af640 1 --2- 172.21.15.139:0/2431902273 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe0fc06c2b0 0x7fe0fc06e770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe1000011e0 tx=0x7fe100009040 comp rx=0 tx=0).stop 2024-09-16T00:07:52.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.492+0000 7fe1162af640 1 -- 172.21.15.139:0/2431902273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1100c7cb0 msgr2=0x7fe1101e4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.492+0000 7fe1162af640 1 --2- 172.21.15.139:0/2431902273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1100c7cb0 0x7fe1101e4370 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe108009d90 tx=0x7fe108004a40 comp rx=0 tx=0).stop 2024-09-16T00:07:52.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.492+0000 7fe1162af640 1 -- 172.21.15.139:0/2431902273 shutdown_connections 2024-09-16T00:07:52.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.492+0000 7fe1162af640 1 --2- 172.21.15.139:0/2431902273 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe0fc06c2b0 0x7fe0fc06e770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.492+0000 7fe1162af640 1 --2- 172.21.15.139:0/2431902273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1100c7cb0 0x7fe1101e4370 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.492+0000 7fe1162af640 1 --2- 172.21.15.139:0/2431902273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe11006f150 0x7fe1100cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.492+0000 7fe1162af640 1 --2- 172.21.15.139:0/2431902273 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe11006ed70 0x7fe11014d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.492+0000 7fe1162af640 1 -- 172.21.15.139:0/2431902273 >> 172.21.15.139:0/2431902273 conn(0x7fe110067950 msgr2=0x7fe11014faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:52.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.492+0000 7fe1162af640 1 -- 172.21.15.139:0/2431902273 shutdown_connections 2024-09-16T00:07:52.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.492+0000 7fe1162af640 1 -- 172.21.15.139:0/2431902273 wait complete. 2024-09-16T00:07:52.505 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] [v2:172.21.15.139:6818/1974901438,v1:172.21.15.139:6819/1974901438] exists,up 83699a05-4937-43c3-aefc-b669bed9bb6f', 'osd.1 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] [v2:172.21.15.139:6810/2232380998,v1:172.21.15.139:6811/2232380998] exists,up 9c04991e-9695-4bbb-8011-bf613bf404b3', 'osd.2 up in weight 1 up_from 13 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] [v2:172.21.15.139:6802/4180588905,v1:172.21.15.139:6803/4180588905] exists,up 8048fc64-72b3-4866-ac62-6a8b2bf6f5c8', 'osd.3 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] [v2:172.21.15.139:6828/3947768492,v1:172.21.15.139:6829/3947768492] exists,up c34ac77b-354c-4884-902d-3e2a84f5eb27', 'osd.4 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] [v2:172.21.15.164:6818/642632916,v1:172.21.15.164:6819/642632916] exists,up ba9a7c93-9672-43e1-8c98-2878cdf7f548', 'osd.5 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] [v2:172.21.15.164:6802/1051778383,v1:172.21.15.164:6803/1051778383] exists,up 46a0916b-474c-411b-84ad-86f27d4cbf53', 'osd.6 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] [v2:172.21.15.164:6826/1908495807,v1:172.21.15.164:6827/1908495807] exists,up 9b6b33cc-d6d9-4a0a-8143-526073a58b93', 'osd.7 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] [v2:172.21.15.164:6810/1158838808,v1:172.21.15.164:6811/1158838808] exists,up 50bcd808-3d3e-4a72-88e4-30decb53cfea'] 2024-09-16T00:07:52.506 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-16T00:07:52.605 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:07:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: "name": "mon.a", 2024-09-16T00:07:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster": "ceph", 2024-09-16T00:07:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-16T00:07:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: "admin_socket_mode": "", 2024-09-16T00:07:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-16T00:07:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: "auth_client_required": "cephx, none", 2024-09-16T00:07:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "auth_cluster_required": "cephx", 2024-09-16T00:07:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "auth_debug": "true", 2024-09-16T00:07:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-16T00:07:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-16T00:07:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "auth_service_required": "cephx", 2024-09-16T00:07:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-16T00:07:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "auth_supported": "cephx", 2024-09-16T00:07:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_aio": "true", 2024-09-16T00:07:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-16T00:07:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_aio_poll_ms": "250", 2024-09-16T00:07:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_aio_reap_max": "16", 2024-09-16T00:07:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_async_discard": "false", 2024-09-16T00:07:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_block_size": "4096", 2024-09-16T00:07:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_debug_aio": "false", 2024-09-16T00:07:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-16T00:07:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-16T00:07:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_debug_inflight_ios": "false", 2024-09-16T00:07:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_enable_discard": "false", 2024-09-16T00:07:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_flock_retry": "3", 2024-09-16T00:07:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-16T00:07:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_inject_crash": "0", 2024-09-16T00:07:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-16T00:07:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_ioring": "false", 2024-09-16T00:07:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_ioring_hipri": "false", 2024-09-16T00:07:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-16T00:07:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-16T00:07:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-16T00:07:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-16T00:07:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-16T00:07:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-16T00:07:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "bdev_type": "", 2024-09-16T00:07:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_alloc_size": "1048576", 2024-09-16T00:07:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_allocator": "hybrid", 2024-09-16T00:07:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_buffered_io": "true", 2024-09-16T00:07:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_check_for_zeros": "false", 2024-09-16T00:07:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-16T00:07:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-16T00:07:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_compact_log_sync": "false", 2024-09-16T00:07:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-16T00:07:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-16T00:07:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-16T00:07:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-16T00:07:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_max_log_runway": "4194304", 2024-09-16T00:07:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_max_prefetch": "1048576", 2024-09-16T00:07:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_min_flush_size": "524288", 2024-09-16T00:07:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_min_log_runway": "1048576", 2024-09-16T00:07:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_replay_recovery": "false", 2024-09-16T00:07:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-16T00:07:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-16T00:07:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_sync_write": "false", 2024-09-16T00:07:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-16T00:07:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-16T00:07:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-16T00:07:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_allocation_from_file": "true", 2024-09-16T00:07:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_allocator": "hybrid", 2024-09-16T00:07:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-16T00:07:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-16T00:07:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-16T00:07:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-16T00:07:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_bdev_label_multi": "true", 2024-09-16T00:07:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-16T00:07:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-16T00:07:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-16T00:07:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-16T00:07:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-16T00:07:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_block_create": "true", 2024-09-16T00:07:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_block_db_create": "false", 2024-09-16T00:07:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_block_db_path": "", 2024-09-16T00:07:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_block_db_size": "0", 2024-09-16T00:07:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_block_path": "", 2024-09-16T00:07:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_block_preallocate_file": "false", 2024-09-16T00:07:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_block_size": "107374182400", 2024-09-16T00:07:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_block_wal_create": "false", 2024-09-16T00:07:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_block_wal_path": "", 2024-09-16T00:07:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_block_wal_size": "100663296", 2024-09-16T00:07:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_bluefs": "true", 2024-09-16T00:07:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-16T00:07:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-16T00:07:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-16T00:07:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-16T00:07:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T00:07:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T00:07:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-16T00:07:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T00:07:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_autotune": "true", 2024-09-16T00:07:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-16T00:07:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-16T00:07:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-16T00:07:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-16T00:07:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_size": "0", 2024-09-16T00:07:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-16T00:07:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-16T00:07:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-16T00:07:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-16T00:07:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_type": "2q", 2024-09-16T00:07:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-16T00:07:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_clone_cow": "true", 2024-09-16T00:07:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-16T00:07:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-16T00:07:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-16T00:07:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-16T00:07:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-16T00:07:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-16T00:07:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-16T00:07:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_compression_mode": "none", 2024-09-16T00:07:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-16T00:07:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_csum_type": "crc32c", 2024-09-16T00:07:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-16T00:07:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_debug_freelist": "false", 2024-09-16T00:07:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-16T00:07:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-16T00:07:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-16T00:07:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-16T00:07:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-16T00:07:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-16T00:07:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-16T00:07:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-16T00:07:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-16T00:07:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-16T00:07:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-16T00:07:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-16T00:07:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_debug_small_allocations": "0", 2024-09-16T00:07:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-16T00:07:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_default_buffered_read": "true", 2024-09-16T00:07:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_default_buffered_write": "false", 2024-09-16T00:07:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-16T00:07:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-16T00:07:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-16T00:07:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-16T00:07:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-16T00:07:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-16T00:07:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-16T00:07:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-16T00:07:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-16T00:07:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fail_eio": "false", 2024-09-16T00:07:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-16T00:07:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-16T00:07:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-16T00:07:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-16T00:07:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-16T00:07:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-16T00:07:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_on_mount": "false", 2024-09-16T00:07:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-16T00:07:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_on_umount": "false", 2024-09-16T00:07:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-16T00:07:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-16T00:07:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-16T00:07:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-16T00:07:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-16T00:07:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-16T00:07:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-16T00:07:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-16T00:07:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_ignore_data_csum": "false", 2024-09-16T00:07:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-16T00:07:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-16T00:07:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-16T00:07:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-16T00:07:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_log_op_age": "5.000000", 2024-09-16T00:07:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_max_alloc_size": "0", 2024-09-16T00:07:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_max_blob_size": "0", 2024-09-16T00:07:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-16T00:07:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-16T00:07:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-16T00:07:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_max_deferred_txc": "32", 2024-09-16T00:07:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_min_alloc_size": "0", 2024-09-16T00:07:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-16T00:07:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-16T00:07:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_nid_prealloc": "1024", 2024-09-16T00:07:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-16T00:07:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-16T00:07:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-16T00:07:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-16T00:07:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_retry_disk_reads": "3", 2024-09-16T00:07:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_rocksdb_cf": "true", 2024-09-16T00:07:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-16T00:07:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-16T00:07:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-16T00:07:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-16T00:07:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-16T00:07:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-16T00:07:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-16T00:07:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-16T00:07:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_spdk_mem": "512", 2024-09-16T00:07:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-16T00:07:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-16T00:07:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-16T00:07:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-16T00:07:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-16T00:07:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-16T00:07:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-16T00:07:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_tracing": "false", 2024-09-16T00:07:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-16T00:07:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-16T00:07:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-16T00:07:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-16T00:07:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-16T00:07:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-16T00:07:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-16T00:07:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-16T00:07:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-16T00:07:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_zero_block_detection": "false", 2024-09-16T00:07:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-16T00:07:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-16T00:07:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-16T00:07:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-16T00:07:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-16T00:07:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-16T00:07:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-16T00:07:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-16T00:07:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-16T00:07:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-16T00:07:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-16T00:07:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-16T00:07:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-16T00:07:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-16T00:07:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "cephx_cluster_require_signatures": "false", 2024-09-16T00:07:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "cephx_cluster_require_version": "2", 2024-09-16T00:07:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "cephx_require_signatures": "false", 2024-09-16T00:07:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "cephx_require_version": "2", 2024-09-16T00:07:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "cephx_service_require_signatures": "false", 2024-09-16T00:07:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "cephx_service_require_version": "2", 2024-09-16T00:07:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "cephx_sign_messages": "true", 2024-09-16T00:07:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "chdir": "", 2024-09-16T00:07:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "client_acl_type": "", 2024-09-16T00:07:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "client_asio_thread_count": "2", 2024-09-16T00:07:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "client_cache_mid": "0.750000", 2024-09-16T00:07:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "client_cache_size": "16384", 2024-09-16T00:07:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "client_caps_release_delay": "5", 2024-09-16T00:07:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "client_check_pool_perm": "true", 2024-09-16T00:07:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-16T00:07:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "client_debug_force_sync_read": "false", 2024-09-16T00:07:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "client_debug_getattr_caps": "false", 2024-09-16T00:07:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "client_debug_inject_features": "", 2024-09-16T00:07:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "client_debug_inject_tick_delay": "0", 2024-09-16T00:07:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-16T00:07:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "client_die_on_failed_remount": "false", 2024-09-16T00:07:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "client_dirsize_rbytes": "true", 2024-09-16T00:07:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "client_force_lazyio": "false", 2024-09-16T00:07:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "client_fs": "", 2024-09-16T00:07:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-16T00:07:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "client_inject_release_failure": "false", 2024-09-16T00:07:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "client_max_inline_size": "4096", 2024-09-16T00:07:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-16T00:07:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "client_mds_namespace": "", 2024-09-16T00:07:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "client_metadata": "", 2024-09-16T00:07:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "client_mount_gid": "-1", 2024-09-16T00:07:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "client_mount_timeout": "300", 2024-09-16T00:07:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "client_mount_uid": "-1", 2024-09-16T00:07:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "client_mountpoint": "/", 2024-09-16T00:07:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "client_notify_timeout": "10", 2024-09-16T00:07:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "client_oc": "true", 2024-09-16T00:07:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "client_oc_max_dirty": "104857600", 2024-09-16T00:07:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-16T00:07:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "client_oc_max_objects": "1000", 2024-09-16T00:07:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "client_oc_size": "209715200", 2024-09-16T00:07:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "client_oc_target_dirty": "8388608", 2024-09-16T00:07:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "client_permissions": "true", 2024-09-16T00:07:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "client_quota": "true", 2024-09-16T00:07:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "client_quota_df": "true", 2024-09-16T00:07:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "client_readahead_max_bytes": "0", 2024-09-16T00:07:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "client_readahead_max_periods": "4", 2024-09-16T00:07:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "client_readahead_min": "131072", 2024-09-16T00:07:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "client_reconnect_stale": "false", 2024-09-16T00:07:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "client_shutdown_timeout": "30", 2024-09-16T00:07:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "client_snapdir": ".snap", 2024-09-16T00:07:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "client_tick_interval": "1", 2024-09-16T00:07:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "client_trace": "", 2024-09-16T00:07:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "client_try_dentry_invalidate": "false", 2024-09-16T00:07:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "client_use_faked_inos": "false", 2024-09-16T00:07:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "client_use_random_mds": "false", 2024-09-16T00:07:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "clog_to_graylog": "false", 2024-09-16T00:07:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-16T00:07:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "clog_to_graylog_port": "12201", 2024-09-16T00:07:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "clog_to_monitors": "default=true", 2024-09-16T00:07:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "clog_to_syslog": "false", 2024-09-16T00:07:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-16T00:07:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "clog_to_syslog_level": "info", 2024-09-16T00:07:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "-", 2024-09-16T00:07:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_network": "", 2024-09-16T00:07:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_network_interface": "", 2024-09-16T00:07:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "compressor_zlib_isal": "false", 2024-09-16T00:07:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "compressor_zlib_level": "5", 2024-09-16T00:07:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "compressor_zlib_winsize": "-15", 2024-09-16T00:07:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "compressor_zstd_level": "1", 2024-09-16T00:07:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-16T00:07:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-16T00:07:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "crimson_alien_op_num_threads": "6", 2024-09-16T00:07:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-16T00:07:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-16T00:07:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-16T00:07:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "crimson_osd_stat_interval": "0", 2024-09-16T00:07:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "crimson_seastar_cpu_cores": "", 2024-09-16T00:07:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "crimson_seastar_num_threads": "0", 2024-09-16T00:07:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_location": "", 2024-09-16T00:07:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_location_hook": "", 2024-09-16T00:07:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_location_hook_timeout": "10", 2024-09-16T00:07:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "daemonize": "false", 2024-09-16T00:07:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "daos_pool": "tank", 2024-09-16T00:07:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-16T00:07:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-16T00:07:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-16T00:07:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_alienstore": "0/5", 2024-09-16T00:07:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_allow_any_pool_priority": "false", 2024-09-16T00:07:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_asok": "1/5", 2024-09-16T00:07:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_asok_assert_abort": "false", 2024-09-16T00:07:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_asserts_on_shutdown": "true", 2024-09-16T00:07:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_auth": "20/20", 2024-09-16T00:07:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_bdev": "1/3", 2024-09-16T00:07:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_bluefs": "1/5", 2024-09-16T00:07:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_bluestore": "1/5", 2024-09-16T00:07:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_buffer": "0/1", 2024-09-16T00:07:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_ceph_exporter": "1/5", 2024-09-16T00:07:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_cephfs_mirror": "0/5", 2024-09-16T00:07:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_cephsqlite": "0/5", 2024-09-16T00:07:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_client": "0/5", 2024-09-16T00:07:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_compressor": "1/5", 2024-09-16T00:07:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_context": "0/1", 2024-09-16T00:07:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_crimson_interrupt": "0/5", 2024-09-16T00:07:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_crush": "1/1", 2024-09-16T00:07:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_crypto": "1/5", 2024-09-16T00:07:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_cyanstore": "0/5", 2024-09-16T00:07:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_deliberately_leak_memory": "false", 2024-09-16T00:07:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_disable_randomized_ping": "false", 2024-09-16T00:07:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_dpdk": "1/5", 2024-09-16T00:07:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_eventtrace": "1/5", 2024-09-16T00:07:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_filer": "0/1", 2024-09-16T00:07:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_filestore": "1/3", 2024-09-16T00:07:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_finisher": "1/1", 2024-09-16T00:07:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_fuse": "1/5", 2024-09-16T00:07:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_heartbeat_testing_span": "0", 2024-09-16T00:07:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_heartbeatmap": "1/5", 2024-09-16T00:07:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-16T00:07:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_javaclient": "1/5", 2024-09-16T00:07:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_journal": "1/3", 2024-09-16T00:07:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_journaler": "0/5", 2024-09-16T00:07:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_kstore": "1/5", 2024-09-16T00:07:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_lockdep": "0/1", 2024-09-16T00:07:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_mclock": "1/5", 2024-09-16T00:07:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_mds": "1/5", 2024-09-16T00:07:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_mds_balancer": "1/5", 2024-09-16T00:07:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_mds_locker": "1/5", 2024-09-16T00:07:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_mds_log": "1/5", 2024-09-16T00:07:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_mds_log_expire": "1/5", 2024-09-16T00:07:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_mds_migrator": "1/5", 2024-09-16T00:07:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_mds_quiesce": "3/5", 2024-09-16T00:07:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_memstore": "1/5", 2024-09-16T00:07:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_mgr": "2/5", 2024-09-16T00:07:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_mgrc": "1/5", 2024-09-16T00:07:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_mon": "20/20", 2024-09-16T00:07:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_monc": "0/10", 2024-09-16T00:07:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_ms": "1/1", 2024-09-16T00:07:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_none": "0/5", 2024-09-16T00:07:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_objclass": "0/5", 2024-09-16T00:07:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_objectcacher": "0/5", 2024-09-16T00:07:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_objecter": "0/1", 2024-09-16T00:07:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_optracker": "0/5", 2024-09-16T00:07:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_osd": "1/5", 2024-09-16T00:07:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_paxos": "20/20", 2024-09-16T00:07:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_perfcounter": "1/5", 2024-09-16T00:07:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_prioritycache": "1/5", 2024-09-16T00:07:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_rados": "0/5", 2024-09-16T00:07:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_rbd": "0/5", 2024-09-16T00:07:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_rbd_mirror": "0/5", 2024-09-16T00:07:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_rbd_pwl": "0/5", 2024-09-16T00:07:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_rbd_replay": "0/5", 2024-09-16T00:07:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_refs": "0/0", 2024-09-16T00:07:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_reserver": "1/1", 2024-09-16T00:07:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_rgw": "1/5", 2024-09-16T00:07:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_rgw_access": "1/5", 2024-09-16T00:07:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_rgw_datacache": "1/5", 2024-09-16T00:07:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_rgw_dbstore": "1/5", 2024-09-16T00:07:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_rgw_flight": "1/5", 2024-09-16T00:07:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-16T00:07:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_rgw_sync": "1/5", 2024-09-16T00:07:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_rocksdb": "4/5", 2024-09-16T00:07:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_seastore": "0/5", 2024-09-16T00:07:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_seastore_backref": "0/5", 2024-09-16T00:07:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_seastore_cache": "0/5", 2024-09-16T00:07:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_seastore_cleaner": "0/5", 2024-09-16T00:07:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_seastore_device": "0/5", 2024-09-16T00:07:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_seastore_epm": "0/5", 2024-09-16T00:07:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-16T00:07:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_seastore_journal": "0/5", 2024-09-16T00:07:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_seastore_lba": "0/5", 2024-09-16T00:07:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_seastore_odata": "0/5", 2024-09-16T00:07:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_seastore_omap": "0/5", 2024-09-16T00:07:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_seastore_onode": "0/5", 2024-09-16T00:07:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_seastore_t": "0/5", 2024-09-16T00:07:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_seastore_tm": "0/5", 2024-09-16T00:07:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_striper": "0/1", 2024-09-16T00:07:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_test": "0/5", 2024-09-16T00:07:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_throttle": "1/1", 2024-09-16T00:07:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_timer": "0/1", 2024-09-16T00:07:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_tp": "0/5", 2024-09-16T00:07:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: "debug_trace": "1/5", 2024-09-16T00:07:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-16T00:07:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "device_failure_prediction_mode": "none", 2024-09-16T00:07:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-16T00:07:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-16T00:07:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "err_to_graylog": "false", 2024-09-16T00:07:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "err_to_journald": "false", 2024-09-16T00:07:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "err_to_stderr": "true", 2024-09-16T00:07:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "err_to_syslog": "false", 2024-09-16T00:07:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event_tracing": "false", 2024-09-16T00:07:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "exporter_addr": "0.0.0.0", 2024-09-16T00:07:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "exporter_http_port": "9926", 2024-09-16T00:07:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "exporter_prio_limit": "5", 2024-09-16T00:07:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-16T00:07:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "exporter_sort_metrics": "true", 2024-09-16T00:07:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "exporter_stats_period": "5", 2024-09-16T00:07:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "fake_statfs_for_testing": "0", 2024-09-16T00:07:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "fatal_signal_handlers": "true", 2024-09-16T00:07:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "filer_max_purge_ops": "10", 2024-09-16T00:07:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "filer_max_truncate_ops": "128", 2024-09-16T00:07:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_apply_finisher_threads": "1", 2024-09-16T00:07:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_blackhole": "false", 2024-09-16T00:07:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_btrfs_clone_range": "true", 2024-09-16T00:07:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_btrfs_snap": "true", 2024-09-16T00:07:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_caller_concurrency": "10", 2024-09-16T00:07:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_collect_device_partition_information": "true", 2024-09-16T00:07:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_commit_timeout": "600.000000", 2024-09-16T00:07:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_debug_inject_read_err": "false", 2024-09-16T00:07:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_debug_omap_check": "false", 2024-09-16T00:07:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-16T00:07:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_debug_verify_split": "false", 2024-09-16T00:07:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_dump_file": "", 2024-09-16T00:07:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-16T00:07:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-16T00:07:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_fadvise": "true", 2024-09-16T00:07:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_fail_eio": "true", 2024-09-16T00:07:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_fd_cache_shards": "16", 2024-09-16T00:07:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_fd_cache_size": "128", 2024-09-16T00:07:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_fiemap": "false", 2024-09-16T00:07:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_fiemap_threshold": "4096", 2024-09-16T00:07:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-16T00:07:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-16T00:07:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_inject_stall": "0", 2024-09-16T00:07:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_journal_parallel": "false", 2024-09-16T00:07:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_journal_trailing": "false", 2024-09-16T00:07:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_journal_writeahead": "false", 2024-09-16T00:07:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_kill_at": "0", 2024-09-16T00:07:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-16T00:07:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-16T00:07:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-16T00:07:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-16T00:07:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-16T00:07:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_max_inline_xattrs": "0", 2024-09-16T00:07:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-16T00:07:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-16T00:07:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-16T00:07:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-16T00:07:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_max_xattr_value_size": "0", 2024-09-16T00:07:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-16T00:07:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-16T00:07:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-16T00:07:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_merge_threshold": "-10", 2024-09-16T00:07:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-16T00:07:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_odsync_write": "false", 2024-09-16T00:07:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_omap_backend": "rocksdb", 2024-09-16T00:07:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_omap_backend_path": "", 2024-09-16T00:07:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-16T00:07:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-16T00:07:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-16T00:07:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_op_thread_timeout": "60", 2024-09-16T00:07:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_op_threads": "2", 2024-09-16T00:07:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_punch_hole": "false", 2024-09-16T00:07:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-16T00:07:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-16T00:07:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-16T00:07:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-16T00:07:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-16T00:07:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-16T00:07:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-16T00:07:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-16T00:07:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-16T00:07:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_queue_max_ops": "50", 2024-09-16T00:07:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-16T00:07:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_seek_data_hole": "false", 2024-09-16T00:07:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_sloppy_crc": "false", 2024-09-16T00:07:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-16T00:07:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_splice": "false", 2024-09-16T00:07:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_split_multiple": "2", 2024-09-16T00:07:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_split_rand_factor": "20", 2024-09-16T00:07:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_update_to": "1000", 2024-09-16T00:07:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-16T00:07:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-16T00:07:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-16T00:07:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-16T00:07:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-16T00:07:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-16T00:07:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_wbthrottle_enable": "true", 2024-09-16T00:07:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-16T00:07:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-16T00:07:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-16T00:07:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-16T00:07:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-16T00:07:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-16T00:07:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_xfs_extsize": "false", 2024-09-16T00:07:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "filestore_zfs_snap": "false", 2024-09-16T00:07:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "fio_dir": "/tmp/fio", 2024-09-16T00:07:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "fsid": "2d3366e5-6a59-4b59-93b2-615e72ac7015", 2024-09-16T00:07:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "fuse_allow_other": "true", 2024-09-16T00:07:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "fuse_atomic_o_trunc": "true", 2024-09-16T00:07:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "fuse_big_writes": "true", 2024-09-16T00:07:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "fuse_debug": "false", 2024-09-16T00:07:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "fuse_default_permissions": "false", 2024-09-16T00:07:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "fuse_disable_pagecache": "false", 2024-09-16T00:07:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "fuse_max_write": "0", 2024-09-16T00:07:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "fuse_multithreaded": "true", 2024-09-16T00:07:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "fuse_require_active_mds": "true", 2024-09-16T00:07:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "fuse_set_user_groups": "true", 2024-09-16T00:07:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "fuse_splice_move": "true", 2024-09-16T00:07:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "fuse_splice_read": "true", 2024-09-16T00:07:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "fuse_splice_write": "true", 2024-09-16T00:07:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-16T00:07:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "fuse_use_invalidate_cb": "true", 2024-09-16T00:07:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-16T00:07:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "gss_target_name": "ceph", 2024-09-16T00:07:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_file": "", 2024-09-16T00:07:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_inject_failure": "0", 2024-09-16T00:07:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_interval": "5", 2024-09-16T00:07:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "host": "smithi139", 2024-09-16T00:07:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-16T00:07:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-16T00:07:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-16T00:07:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-16T00:07:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-16T00:07:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-16T00:07:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-16T00:07:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-16T00:07:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-16T00:07:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-16T00:07:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-16T00:07:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-16T00:07:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-16T00:07:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "inject_early_sigterm": "false", 2024-09-16T00:07:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "jaeger_agent_port": "6799", 2024-09-16T00:07:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "jaeger_tracing_enable": "false", 2024-09-16T00:07:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_aio": "true", 2024-09-16T00:07:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_align_min_size": "65536", 2024-09-16T00:07:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_block_align": "true", 2024-09-16T00:07:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_block_size": "4096", 2024-09-16T00:07:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_dio": "true", 2024-09-16T00:07:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_discard": "false", 2024-09-16T00:07:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_force_aio": "false", 2024-09-16T00:07:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_ignore_corruption": "false", 2024-09-16T00:07:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_max_write_bytes": "10485760", 2024-09-16T00:07:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_max_write_entries": "100", 2024-09-16T00:07:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_replay_from": "0", 2024-09-16T00:07:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-16T00:07:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-16T00:07:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-16T00:07:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-16T00:07:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_write_header_frequency": "0", 2024-09-16T00:07:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "journal_zero_on_create": "false", 2024-09-16T00:07:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "journaler_prefetch_periods": "10", 2024-09-16T00:07:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "journaler_prezero_periods": "5", 2024-09-16T00:07:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "journaler_write_head_interval": "15", 2024-09-16T00:07:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "key": "", 2024-09-16T00:07:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "keyfile": "", 2024-09-16T00:07:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-16T00:07:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "kstore_backend": "rocksdb", 2024-09-16T00:07:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "kstore_default_stripe_size": "65536", 2024-09-16T00:07:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "kstore_fsck_on_mount": "false", 2024-09-16T00:07:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-16T00:07:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "kstore_max_bytes": "67108864", 2024-09-16T00:07:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "kstore_max_ops": "512", 2024-09-16T00:07:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "kstore_nid_prealloc": "1024", 2024-09-16T00:07:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "kstore_onode_map_size": "1024", 2024-09-16T00:07:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-16T00:07:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "kstore_sync_submit_transaction": "false", 2024-09-16T00:07:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "kstore_sync_transaction": "false", 2024-09-16T00:07:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "librados_thread_count": "2", 2024-09-16T00:07:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "lockdep": "false", 2024-09-16T00:07:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "lockdep_force_backtrace": "false", 2024-09-16T00:07:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "log_coarse_timestamps": "true", 2024-09-16T00:07:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-16T00:07:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "log_flush_on_exit": "false", 2024-09-16T00:07:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "log_graylog_host": "127.0.0.1", 2024-09-16T00:07:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "log_graylog_port": "12201", 2024-09-16T00:07:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "log_max_new": "1000", 2024-09-16T00:07:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "log_max_recent": "10000", 2024-09-16T00:07:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "log_stderr_prefix": "", 2024-09-16T00:07:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "log_stop_at_utilization": "0.970000", 2024-09-16T00:07:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_file": "true", 2024-09-16T00:07:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_graylog": "false", 2024-09-16T00:07:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_journald": "false", 2024-09-16T00:07:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_stderr": "false", 2024-09-16T00:07:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "log_to_syslog": "false", 2024-09-16T00:07:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "max_rotating_auth_attempts": "10", 2024-09-16T00:07:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-16T00:07:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_action_on_write_error": "1", 2024-09-16T00:07:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_alternate_name_max": "8192", 2024-09-16T00:07:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_asio_thread_count": "2", 2024-09-16T00:07:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_export_pin": "true", 2024-09-16T00:07:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_fragment_dirs": "true", 2024-09-16T00:07:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-16T00:07:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_fragment_interval": "5", 2024-09-16T00:07:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-16T00:07:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-16T00:07:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_interval": "10", 2024-09-16T00:07:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_max": "-1", 2024-09-16T00:07:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_max_until": "-1", 2024-09-16T00:07:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_merge_size": "50", 2024-09-16T00:07:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_midchunk": "0.300000", 2024-09-16T00:07:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-16T00:07:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_min_start": "0.200000", 2024-09-16T00:07:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_minchunk": "0.001000", 2024-09-16T00:07:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_mode": "0", 2024-09-16T00:07:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_need_max": "1.200000", 2024-09-16T00:07:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_need_min": "0.800000", 2024-09-16T00:07:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_overload_epochs": "2", 2024-09-16T00:07:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-16T00:07:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-16T00:07:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_split_bits": "3", 2024-09-16T00:07:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-16T00:07:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_split_size": "10000", 2024-09-16T00:07:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-16T00:07:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_target_decay": "10.000000", 2024-09-16T00:07:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-16T00:07:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_beacon_grace": "15.000000", 2024-09-16T00:07:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_beacon_interval": "4.000000", 2024-09-16T00:07:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-16T00:07:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-16T00:07:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache_mid": "0.700000", 2024-09-16T00:07:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-16T00:07:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache_quiesce_delay": "0", 2024-09-16T00:07:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-16T00:07:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-16T00:07:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-16T00:07:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache_release_free_interval": "10", 2024-09-16T00:07:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache_reservation": "0.050000", 2024-09-16T00:07:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-16T00:07:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache_trim_interval": "1", 2024-09-16T00:07:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache_trim_threshold": "262144", 2024-09-16T00:07:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-16T00:07:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-16T00:07:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-16T00:07:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_client_prealloc_inos": "1000", 2024-09-16T00:07:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-16T00:07:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_connect_bootstrapping": "false", 2024-09-16T00:07:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_damage_table_max_entries": "10000", 2024-09-16T00:07:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-16T00:07:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_debug_auth_pins": "false", 2024-09-16T00:07:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_debug_frag": "false", 2024-09-16T00:07:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_debug_scatterstat": "false", 2024-09-16T00:07:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_debug_subtrees": "false", 2024-09-16T00:07:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_decay_halflife": "5.000000", 2024-09-16T00:07:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_default_dir_hash": "2", 2024-09-16T00:07:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_defer_session_stale": "true", 2024-09-16T00:07:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_deny_all_reconnect": "false", 2024-09-16T00:07:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_dir_keys_per_op": "16384", 2024-09-16T00:07:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_dir_max_commit_size": "10", 2024-09-16T00:07:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_dir_max_entries": "0", 2024-09-16T00:07:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_dir_prefetch": "true", 2024-09-16T00:07:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-16T00:07:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-16T00:07:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_dump_cache_on_map": "false", 2024-09-16T00:07:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-16T00:07:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-16T00:07:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_early_reply": "true", 2024-09-16T00:07:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_enable_op_tracker": "true", 2024-09-16T00:07:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_enforce_unique_name": "true", 2024-09-16T00:07:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-16T00:07:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-16T00:07:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_export_ephemeral_random": "true", 2024-09-16T00:07:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-16T00:07:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-16T00:07:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-16T00:07:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-16T00:07:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-16T00:07:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-16T00:07:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-16T00:07:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-16T00:07:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_health_summarize_threshold": "10", 2024-09-16T00:07:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-16T00:07:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-16T00:07:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_inject_health_dummy": "false", 2024-09-16T00:07:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-16T00:07:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_inject_migrator_session_race": "false", 2024-09-16T00:07:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-16T00:07:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-16T00:07:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-16T00:07:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_join_fs": "", 2024-09-16T00:07:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_journal_format": "1", 2024-09-16T00:07:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-16T00:07:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_kill_create_at": "0", 2024-09-16T00:07:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_kill_export_at": "0", 2024-09-16T00:07:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_kill_import_at": "0", 2024-09-16T00:07:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_kill_journal_at": "0", 2024-09-16T00:07:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_kill_journal_expire_at": "0", 2024-09-16T00:07:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_kill_journal_replay_at": "0", 2024-09-16T00:07:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_kill_link_at": "0", 2024-09-16T00:07:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_kill_mdstable_at": "0", 2024-09-16T00:07:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_kill_openc_at": "0", 2024-09-16T00:07:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_kill_rename_at": "0", 2024-09-16T00:07:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_kill_shutdown_at": "0", 2024-09-16T00:07:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log_event_large_threshold": "524288", 2024-09-16T00:07:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log_events_per_segment": "1024", 2024-09-16T00:07:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-16T00:07:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log_max_events": "-1", 2024-09-16T00:07:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log_max_segments": "128", 2024-09-16T00:07:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log_pause": "false", 2024-09-16T00:07:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log_segment_size": "0", 2024-09-16T00:07:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-16T00:07:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-16T00:07:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-16T00:07:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log_trim_threshold": "128", 2024-09-16T00:07:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-16T00:07:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log_warn_factor": "2.000000", 2024-09-16T00:07:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_max_caps_per_client": "1048576", 2024-09-16T00:07:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_max_completed_flushes": "100000", 2024-09-16T00:07:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_max_completed_requests": "100000", 2024-09-16T00:07:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_max_export_size": "20971520", 2024-09-16T00:07:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_max_file_recover": "32", 2024-09-16T00:07:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_max_purge_files": "64", 2024-09-16T00:07:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_max_purge_ops": "8192", 2024-09-16T00:07:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-16T00:07:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-16T00:07:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_max_snaps_per_dir": "100", 2024-09-16T00:07:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_metrics_update_interval": "2", 2024-09-16T00:07:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_min_caps_per_client": "100", 2024-09-16T00:07:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_min_caps_working_set": "10000", 2024-09-16T00:07:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-16T00:07:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_numa_node": "-1", 2024-09-16T00:07:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-16T00:07:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_op_complaint_time": "30.000000", 2024-09-16T00:07:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_op_history_duration": "600", 2024-09-16T00:07:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_op_history_size": "20", 2024-09-16T00:07:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_op_history_slow_op_size": "20", 2024-09-16T00:07:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-16T00:07:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_op_log_threshold": "5", 2024-09-16T00:07:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_ping_grace": "15", 2024-09-16T00:07:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_ping_interval": "5", 2024-09-16T00:07:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-16T00:07:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-16T00:07:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_recall_max_caps": "30000", 2024-09-16T00:07:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-16T00:07:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-16T00:07:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-16T00:07:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_recall_warning_threshold": "262144", 2024-09-16T00:07:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-16T00:07:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_replay_interval": "1.000000", 2024-09-16T00:07:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-16T00:07:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-16T00:07:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_root_ino_gid": "0", 2024-09-16T00:07:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_root_ino_uid": "0", 2024-09-16T00:07:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-16T00:07:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-16T00:07:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-16T00:07:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-16T00:07:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-16T00:07:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-16T00:07:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-16T00:07:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-16T00:07:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-16T00:07:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-16T00:07:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_shutdown_check": "0", 2024-09-16T00:07:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_skip_ino": "0", 2024-09-16T00:07:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-16T00:07:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_snap_max_uid": "4294967294", 2024-09-16T00:07:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_snap_min_uid": "0", 2024-09-16T00:07:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_snap_rstat": "false", 2024-09-16T00:07:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_standby_replay_damaged": "false", 2024-09-16T00:07:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_symlink_recovery": "true", 2024-09-16T00:07:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-16T00:07:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_thrash_exports": "0", 2024-09-16T00:07:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_thrash_fragments": "0", 2024-09-16T00:07:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_tick_interval": "5.000000", 2024-09-16T00:07:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_valgrind_exit": "false", 2024-09-16T00:07:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_verify_backtrace": "1", 2024-09-16T00:07:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_verify_scatter": "false", 2024-09-16T00:07:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-16T00:07:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_wipe_sessions": "false", 2024-09-16T00:07:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool_debug": "false", 2024-09-16T00:07:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-16T00:07:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "memstore_device_bytes": "1073741824", 2024-09-16T00:07:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "memstore_page_set": "false", 2024-09-16T00:07:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "memstore_page_size": "65536", 2024-09-16T00:07:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_client_bytes": "134217728", 2024-09-16T00:07:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_client_messages": "512", 2024-09-16T00:07:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-16T00:07:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-16T00:07:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-16T00:07:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-16T00:07:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_disabled_modules": "", 2024-09-16T00:07:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_enable_op_tracker": "true", 2024-09-16T00:07:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-16T00:07:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_max_pg_num_change": "128", 2024-09-16T00:07:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_mds_bytes": "134217728", 2024-09-16T00:07:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_mds_messages": "128", 2024-09-16T00:07:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-16T00:07:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_mon_bytes": "134217728", 2024-09-16T00:07:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_mon_messages": "128", 2024-09-16T00:07:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-16T00:07:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-16T00:07:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_op_history_duration": "600", 2024-09-16T00:07:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_op_history_size": "20", 2024-09-16T00:07:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-16T00:07:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-16T00:07:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_op_log_threshold": "5", 2024-09-16T00:07:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_osd_bytes": "536870912", 2024-09-16T00:07:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_osd_messages": "8192", 2024-09-16T00:07:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_pool": "true", 2024-09-16T00:07:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-16T00:07:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_standby_modules": "true", 2024-09-16T00:07:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_stats_period": "5", 2024-09-16T00:07:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_stats_threshold": "5", 2024-09-16T00:07:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_tick_period": "2", 2024-09-16T00:07:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-16T00:07:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-16T00:07:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_allow_pool_delete": "true", 2024-09-16T00:07:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_allow_pool_size_one": "true", 2024-09-16T00:07:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_auth_validate_all_caps": "true", 2024-09-16T00:07:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-16T00:07:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-16T00:07:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_client_bytes": "104857600", 2024-09-16T00:07:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_client_directed_command_retry": "2", 2024-09-16T00:07:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-16T00:07:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-16T00:07:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-16T00:07:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-16T00:07:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_client_hunt_parallel": "3", 2024-09-16T00:07:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_client_log_interval": "1.000000", 2024-09-16T00:07:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-16T00:07:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_client_ping_interval": "10.000000", 2024-09-16T00:07:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-16T00:07:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_client_target_rank": "-1", 2024-09-16T00:07:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-16T00:07:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-16T00:07:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-16T00:07:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_cluster_log_file_level": "debug", 2024-09-16T00:07:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_cluster_log_to_file": "true", 2024-09-16T00:07:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-16T00:07:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-16T00:07:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-16T00:07:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_cluster_log_to_journald": "false", 2024-09-16T00:07:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-16T00:07:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-16T00:07:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-16T00:07:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-16T00:07:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_compact_on_bootstrap": "false", 2024-09-16T00:07:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_compact_on_start": "false", 2024-09-16T00:07:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_compact_on_trim": "true", 2024-09-16T00:07:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-16T00:07:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-16T00:07:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-16T00:07:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_cpu_threads": "4", 2024-09-16T00:07:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_crush_min_required_version": "hammer", 2024-09-16T00:07:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_daemon_bytes": "419430400", 2024-09-16T00:07:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-16T00:07:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_data_avail_crit": "5", 2024-09-16T00:07:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_data_avail_warn": "5", 2024-09-16T00:07:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_data_size_warn": "16106127360", 2024-09-16T00:07:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-16T00:07:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-16T00:07:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_debug_dump_json": "false", 2024-09-16T00:07:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-16T00:07:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_debug_dump_transactions": "false", 2024-09-16T00:07:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_debug_extra_checks": "false", 2024-09-16T00:07:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-16T00:07:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-16T00:07:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_debug_no_require_reef": "false", 2024-09-16T00:07:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_debug_no_require_squid": "false", 2024-09-16T00:07:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-16T00:07:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-16T00:07:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-16T00:07:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_down_mkfs_grace": "300", 2024-09-16T00:07:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_down_uptime_grace": "60", 2024-09-16T00:07:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_election_default_strategy": "1", 2024-09-16T00:07:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_election_timeout": "5.000000", 2024-09-16T00:07:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-16T00:07:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_elector_ping_divisor": "2", 2024-09-16T00:07:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-16T00:07:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_enable_op_tracker": "true", 2024-09-16T00:07:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_fake_pool_delete": "false", 2024-09-16T00:07:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_force_quorum_join": "false", 2024-09-16T00:07:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-16T00:07:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_globalid_prealloc": "10000", 2024-09-16T00:07:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_health_detail_to_clog": "false", 2024-09-16T00:07:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_health_log_update_period": "5", 2024-09-16T00:07:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_health_max_detail": "50", 2024-09-16T00:07:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_health_to_clog": "true", 2024-09-16T00:07:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_health_to_clog_interval": "600", 2024-09-16T00:07:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-16T00:07:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_host": "172.21.15.139,172.21.15.164,[v2:172.21.15.164:3301,v1:172.21.15.164:6790]", 2024-09-16T00:07:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_host_override": "", 2024-09-16T00:07:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_initial_members": "", 2024-09-16T00:07:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-16T00:07:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-16T00:07:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-16T00:07:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-16T00:07:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-16T00:07:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_lease": "5.000000", 2024-09-16T00:07:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-16T00:07:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-16T00:07:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_log_full_interval": "50", 2024-09-16T00:07:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_log_max": "10000", 2024-09-16T00:07:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_log_max_summary": "50", 2024-09-16T00:07:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-16T00:07:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_max_log_epochs": "500", 2024-09-16T00:07:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-16T00:07:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-16T00:07:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_max_osd": "10000", 2024-09-16T00:07:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_max_pg_per_osd": "10000", 2024-09-16T00:07:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_max_pool_pg_num": "65536", 2024-09-16T00:07:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-16T00:07:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-16T00:07:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_mds_force_trim_to": "0", 2024-09-16T00:07:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_mds_skip_sanity": "false", 2024-09-16T00:07:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_memory_autotune": "true", 2024-09-16T00:07:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_memory_target": "2147483648", 2024-09-16T00:07:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_mgr_beacon_grace": "30", 2024-09-16T00:07:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-16T00:07:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_mgr_digest_period": "5", 2024-09-16T00:07:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_mgr_inactive_grace": "60", 2024-09-16T00:07:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-16T00:07:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-16T00:07:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_min_osdmap_epochs": "500", 2024-09-16T00:07:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_op_complaint_time": "120", 2024-09-16T00:07:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_op_history_duration": "600", 2024-09-16T00:07:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_op_history_size": "20", 2024-09-16T00:07:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_op_history_slow_op_size": "20", 2024-09-16T00:07:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-16T00:07:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_op_log_threshold": "5", 2024-09-16T00:07:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-16T00:07:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-16T00:07:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-16T00:07:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_auto_mark_in": "false", 2024-09-16T00:07:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-16T00:07:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-16T00:07:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-16T00:07:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_cache_size": "500", 2024-09-16T00:07:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-16T00:07:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-16T00:07:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-16T00:07:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_down_out_interval": "600", 2024-09-16T00:07:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-16T00:07:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-16T00:07:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_force_trim_to": "0", 2024-09-16T00:07:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-16T00:07:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-16T00:07:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-16T00:07:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-16T00:07:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-16T00:07:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-16T00:07:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-16T00:07:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-16T00:07:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_min_down_reporters": "2", 2024-09-16T00:07:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-16T00:07:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-16T00:07:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-16T00:07:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-16T00:07:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-16T00:07:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-16T00:07:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_report_timeout": "900", 2024-09-16T00:07:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-16T00:07:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-16T00:07:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-16T00:07:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-16T00:07:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-16T00:07:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-16T00:07:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-16T00:07:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-16T00:07:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-16T00:07:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_pg_stuck_threshold": "60", 2024-09-16T00:07:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-16T00:07:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-16T00:07:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-16T00:07:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-16T00:07:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-16T00:07:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-16T00:07:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_probe_timeout": "2.000000", 2024-09-16T00:07:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_reweight_max_change": "0.050000", 2024-09-16T00:07:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_reweight_max_osds": "4", 2024-09-16T00:07:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-16T00:07:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-16T00:07:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-16T00:07:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-16T00:07:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-16T00:07:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_scrub_interval": "86400", 2024-09-16T00:07:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_scrub_max_keys": "100", 2024-09-16T00:07:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_scrub_timeout": "300", 2024-09-16T00:07:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_session_timeout": "300", 2024-09-16T00:07:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_smart_report_timeout": "5", 2024-09-16T00:07:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_stat_smooth_intervals": "6", 2024-09-16T00:07:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-16T00:07:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_stretch_pool_min_size": "2", 2024-09-16T00:07:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_stretch_pool_size": "4", 2024-09-16T00:07:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-16T00:07:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-16T00:07:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_sync_debug": "false", 2024-09-16T00:07:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-16T00:07:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-16T00:07:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_sync_provider_kill_at": "0", 2024-09-16T00:07:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_sync_requester_kill_at": "0", 2024-09-16T00:07:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_sync_timeout": "60.000000", 2024-09-16T00:07:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_target_pg_per_osd": "100", 2024-09-16T00:07:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_tick_interval": "5", 2024-09-16T00:07:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_timecheck_interval": "300.000000", 2024-09-16T00:07:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-16T00:07:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_use_min_delay_socket": "false", 2024-09-16T00:07:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_older_version_delay": "604800", 2024-09-16T00:07:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-16T00:07:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-16T00:07:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-16T00:07:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-16T00:07:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-16T00:07:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-16T00:07:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-16T00:07:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_misplaced": "false", 2024-09-16T00:07:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-16T00:07:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_older_version": "true", 2024-09-16T00:07:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-16T00:07:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-16T00:07:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-16T00:07:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-16T00:07:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-16T00:07:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-16T00:07:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-16T00:07:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-16T00:07:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-16T00:07:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "monmap": "", 2024-09-16T00:07:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-16T00:07:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-16T00:07:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-16T00:07:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-16T00:07:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-16T00:07:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-16T00:07:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "motr_tracing_enabled": "false", 2024-09-16T00:07:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_op_threads": "3", 2024-09-16T00:07:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-16T00:07:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_rdma_cm": "false", 2024-09-16T00:07:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_rdma_device_name": "", 2024-09-16T00:07:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_rdma_dscp": "96", 2024-09-16T00:07:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-16T00:07:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-16T00:07:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_rdma_local_gid": "", 2024-09-16T00:07:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-16T00:07:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_rdma_port_num": "1", 2024-09-16T00:07:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-16T00:07:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-16T00:07:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-16T00:07:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-16T00:07:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_rdma_sl": "3", 2024-09-16T00:07:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_rdma_support_srq": "true", 2024-09-16T00:07:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_rdma_type": "ib", 2024-09-16T00:07:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_async_reap_threshold": "5", 2024-09-16T00:07:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_bind_before_connect": "false", 2024-09-16T00:07:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_bind_ipv4": "true", 2024-09-16T00:07:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_bind_ipv6": "false", 2024-09-16T00:07:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_bind_msgr1": "true", 2024-09-16T00:07:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_bind_msgr2": "true", 2024-09-16T00:07:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_bind_port_max": "7568", 2024-09-16T00:07:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_bind_port_min": "6800", 2024-09-16T00:07:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-16T00:07:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_bind_retry_count": "3", 2024-09-16T00:07:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_bind_retry_delay": "5", 2024-09-16T00:07:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_blackhole_client": "false", 2024-09-16T00:07:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_blackhole_mds": "false", 2024-09-16T00:07:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_blackhole_mgr": "false", 2024-09-16T00:07:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_blackhole_mon": "false", 2024-09-16T00:07:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_blackhole_osd": "false", 2024-09-16T00:07:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_client_mode": "crc secure", 2024-09-16T00:07:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_cluster_mode": "crc secure", 2024-09-16T00:07:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_cluster_type": "", 2024-09-16T00:07:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_compress_secure": "false", 2024-09-16T00:07:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_connection_idle_timeout": "900", 2024-09-16T00:07:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_connection_ready_timeout": "10", 2024-09-16T00:07:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_crc_data": "true", 2024-09-16T00:07:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_crc_header": "true", 2024-09-16T00:07:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_die_on_bad_msg": "false", 2024-09-16T00:07:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_die_on_bug": "true", 2024-09-16T00:07:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_die_on_old_message": "true", 2024-09-16T00:07:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_die_on_skipped_message": "false", 2024-09-16T00:07:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-16T00:07:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-16T00:07:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dpdk_coremask": "0xF", 2024-09-16T00:07:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-16T00:07:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-16T00:07:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dpdk_enable_tso": "true", 2024-09-16T00:07:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-16T00:07:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-16T00:07:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dpdk_hugepages": "", 2024-09-16T00:07:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-16T00:07:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-16T00:07:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dpdk_lro": "true", 2024-09-16T00:07:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dpdk_memory_channel": "4", 2024-09-16T00:07:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-16T00:07:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dpdk_pmd": "", 2024-09-16T00:07:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dpdk_port_id": "0", 2024-09-16T00:07:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-16T00:07:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-16T00:07:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_dump_on_send": "false", 2024-09-16T00:07:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_initial_backoff": "0.200000", 2024-09-16T00:07:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_inject_delay_max": "1.000000", 2024-09-16T00:07:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-16T00:07:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_inject_delay_type": "", 2024-09-16T00:07:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-16T00:07:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_inject_network_congestion": "0", 2024-09-16T00:07:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_inject_socket_failures": "0", 2024-09-16T00:07:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_learn_addr_from_peer": "true", 2024-09-16T00:07:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_max_accept_failures": "4", 2024-09-16T00:07:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_max_backoff": "15.000000", 2024-09-16T00:07:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_mon_client_mode": "secure crc", 2024-09-16T00:07:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-16T00:07:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_mon_service_mode": "secure crc", 2024-09-16T00:07:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_osd_compress_min_size": "1024", 2024-09-16T00:07:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_osd_compress_mode": "none", 2024-09-16T00:07:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-16T00:07:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-16T00:07:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_pq_min_cost": "65536", 2024-09-16T00:07:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_public_type": "", 2024-09-16T00:07:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_service_mode": "crc secure", 2024-09-16T00:07:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_tcp_listen_backlog": "512", 2024-09-16T00:07:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_tcp_nodelay": "true", 2024-09-16T00:07:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-16T00:07:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_tcp_rcvbuf": "0", 2024-09-16T00:07:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "ms_type": "async+posix", 2024-09-16T00:07:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "no_config_file": "false", 2024-09-16T00:07:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter_completion_locks_per_session": "32", 2024-09-16T00:07:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-16T00:07:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-16T00:07:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter_inflight_ops": "1024", 2024-09-16T00:07:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-16T00:07:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-16T00:07:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter_tick_interval": "5.000000", 2024-09-16T00:07:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter_timeout": "10.000000", 2024-09-16T00:07:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "objectstore_blackhole": "false", 2024-09-16T00:07:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-16T00:07:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "openssl_engine_opts": "", 2024-09-16T00:07:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_agent_delay_time": "5.000000", 2024-09-16T00:07:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_agent_hist_halflife": "1000", 2024-09-16T00:07:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_agent_max_low_ops": "2", 2024-09-16T00:07:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_agent_max_ops": "4", 2024-09-16T00:07:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-16T00:07:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-16T00:07:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_agent_slop": "0.020000", 2024-09-16T00:07:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-16T00:07:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-16T00:07:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_asio_thread_count": "2", 2024-09-16T00:07:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_async_recovery_min_cost": "100", 2024-09-16T00:07:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-16T00:07:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-16T00:07:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_backfill_scan_max": "512", 2024-09-16T00:07:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_backfill_scan_min": "64", 2024-09-16T00:07:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_backoff_on_degraded": "false", 2024-09-16T00:07:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_backoff_on_peering": "false", 2024-09-16T00:07:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_backoff_on_unfound": "true", 2024-09-16T00:07:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_beacon_report_interval": "300", 2024-09-16T00:07:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bench_duration": "30", 2024-09-16T00:07:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-16T00:07:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bench_max_block_size": "67108864", 2024-09-16T00:07:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-16T00:07:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_blkin_trace_all": "false", 2024-09-16T00:07:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-16T00:07:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-16T00:07:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-16T00:07:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-16T00:07:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_check_for_log_corruption": "false", 2024-09-16T00:07:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-16T00:07:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T00:07:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-16T00:07:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T00:07:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_class_update_on_start": "true", 2024-09-16T00:07:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_client_message_cap": "256", 2024-09-16T00:07:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_client_message_size_cap": "524288000", 2024-09-16T00:07:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_client_op_priority": "63", 2024-09-16T00:07:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_client_watch_timeout": "30", 2024-09-16T00:07:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_command_max_records": "256", 2024-09-16T00:07:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_compact_on_start": "false", 2024-09-16T00:07:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-16T00:07:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-16T00:07:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-16T00:07:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_crush_update_on_start": "true", 2024-09-16T00:07:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_crush_update_weight_set": "true", 2024-09-16T00:07:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-16T00:07:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-16T00:07:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-16T00:07:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-16T00:07:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-16T00:07:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_feed_pullee": "-1", 2024-09-16T00:07:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-16T00:07:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-16T00:07:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-16T00:07:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_misdirected_ops": "false", 2024-09-16T00:07:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_no_acting_change": "false", 2024-09-16T00:07:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_no_purge_strays": "false", 2024-09-16T00:07:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_op_order": "false", 2024-09-16T00:07:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-16T00:07:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-16T00:07:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-16T00:07:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-16T00:07:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_shutdown": "false", 2024-09-16T00:07:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-16T00:07:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-16T00:07:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_trim_objects": "false", 2024-09-16T00:07:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-16T00:07:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-16T00:07:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_verify_snaps": "false", 2024-09-16T00:07:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-16T00:07:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-16T00:07:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_deep_scrub_keys": "1024", 2024-09-16T00:07:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-16T00:07:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-16T00:07:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-16T00:07:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_deep_scrub_stride": "524288", 2024-09-16T00:07:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-16T00:07:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-16T00:07:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_default_notify_timeout": "30", 2024-09-16T00:07:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_delete_sleep": "0.000000", 2024-09-16T00:07:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-16T00:07:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-16T00:07:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-16T00:07:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_discard_disconnected_ops": "true", 2024-09-16T00:07:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_enable_op_tracker": "true", 2024-09-16T00:07:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-16T00:07:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-16T00:07:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-16T00:07:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-16T00:07:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_fast_info": "true", 2024-09-16T00:07:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_fast_shutdown": "true", 2024-09-16T00:07:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-16T00:07:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-16T00:07:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-16T00:07:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-16T00:07:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-16T00:07:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_function_tracing": "false", 2024-09-16T00:07:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_heartbeat_grace": "20", 2024-09-16T00:07:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_heartbeat_interval": "6", 2024-09-16T00:07:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-16T00:07:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_heartbeat_min_peers": "10", 2024-09-16T00:07:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_heartbeat_min_size": "2000", 2024-09-16T00:07:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_heartbeat_stale": "600", 2024-09-16T00:07:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-16T00:07:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_hit_set_max_size": "100000", 2024-09-16T00:07:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_hit_set_min_size": "1000", 2024-09-16T00:07:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-16T00:07:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-16T00:07:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-16T00:07:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-16T00:07:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-16T00:07:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-16T00:07:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_journal_size": "5120", 2024-09-16T00:07:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_kill_backfill_at": "0", 2024-09-16T00:07:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-16T00:07:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_map_cache_size": "50", 2024-09-16T00:07:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_map_dedup": "true", 2024-09-16T00:07:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_map_message_max": "40", 2024-09-16T00:07:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-16T00:07:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_map_share_max_epochs": "40", 2024-09-16T00:07:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_attr_name_len": "100", 2024-09-16T00:07:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_attr_size": "0", 2024-09-16T00:07:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_backfills": "1", 2024-09-16T00:07:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_markdown_count": "5", 2024-09-16T00:07:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_markdown_period": "600", 2024-09-16T00:07:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_object_name_len": "2048", 2024-09-16T00:07:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_object_namespace_len": "256", 2024-09-16T00:07:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_object_size": "134217728", 2024-09-16T00:07:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-16T00:07:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-16T00:07:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_pg_blocked_by": "16", 2024-09-16T00:07:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_pg_log_entries": "10000", 2024-09-16T00:07:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-16T00:07:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_pgls": "1024", 2024-09-16T00:07:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_push_cost": "8388608", 2024-09-16T00:07:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_push_objects": "10", 2024-09-16T00:07:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_scrubs": "3", 2024-09-16T00:07:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-16T00:07:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_trimming_pgs": "2", 2024-09-16T00:07:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_write_op_reply_len": "64", 2024-09-16T00:07:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_max_write_size": "90", 2024-09-16T00:07:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-16T00:07:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-16T00:07:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-16T00:07:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-16T00:07:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-16T00:07:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-16T00:07:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-16T00:07:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-16T00:07:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_profile": "balanced", 2024-09-16T00:07:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-16T00:07:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-16T00:07:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-16T00:07:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-16T00:07:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-16T00:07:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-16T00:07:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-16T00:07:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-16T00:07:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-16T00:07:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-16T00:07:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-16T00:07:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_memory_base": "805306368", 2024-09-16T00:07:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_memory_cache_min": "134217728", 2024-09-16T00:07:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-16T00:07:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-16T00:07:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_memory_target": "4294967296", 2024-09-16T00:07:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_memory_target_autotune": "false", 2024-09-16T00:07:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-16T00:07:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_min_pg_log_entries": "250", 2024-09-16T00:07:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_min_recovery_priority": "0", 2024-09-16T00:07:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-16T00:07:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-16T00:07:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mon_report_interval": "5", 2024-09-16T00:07:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-16T00:07:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-16T00:07:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_num_cache_shards": "32", 2024-09-16T00:07:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_num_op_tracker_shard": "32", 2024-09-16T00:07:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_numa_auto_affinity": "true", 2024-09-16T00:07:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_numa_node": "-1", 2024-09-16T00:07:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_numa_prefer_iface": "true", 2024-09-16T00:07:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-16T00:07:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_objecter_finishers": "1", 2024-09-16T00:07:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_objectstore": "bluestore", 2024-09-16T00:07:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_objectstore_fuse": "false", 2024-09-16T00:07:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_objectstore_tracing": "false", 2024-09-16T00:07:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_complaint_time": "30.000000", 2024-09-16T00:07:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_history_duration": "600", 2024-09-16T00:07:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_history_size": "20", 2024-09-16T00:07:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_history_slow_op_size": "20", 2024-09-16T00:07:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-16T00:07:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_log_threshold": "5", 2024-09-16T00:07:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_num_shards": "0", 2024-09-16T00:07:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_num_shards_hdd": "5", 2024-09-16T00:07:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_num_shards_ssd": "8", 2024-09-16T00:07:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-16T00:07:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-16T00:07:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-16T00:07:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-16T00:07:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_pq_min_cost": "65536", 2024-09-16T00:07:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-16T00:07:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_queue_cut_off": "high", 2024-09-16T00:07:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-16T00:07:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_op_thread_timeout": "15", 2024-09-16T00:07:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_open_classes_on_start": "true", 2024-09-16T00:07:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_os_flags": "0", 2024-09-16T00:07:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_peering_op_priority": "255", 2024-09-16T00:07:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pg_delete_cost": "1048576", 2024-09-16T00:07:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pg_delete_priority": "5", 2024-09-16T00:07:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-16T00:07:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-16T00:07:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-16T00:07:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pg_log_trim_max": "10000", 2024-09-16T00:07:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pg_log_trim_min": "100", 2024-09-16T00:07:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-16T00:07:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-16T00:07:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-16T00:07:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-16T00:07:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-16T00:07:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-16T00:07:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-16T00:07:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-16T00:07:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-16T00:07:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-16T00:07:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_crimson": "false", 2024-09-16T00:07:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-16T00:07:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-16T00:07:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-16T00:07:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-16T00:07:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-16T00:07:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-16T00:07:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-16T00:07:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-16T00:07:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_flags": "0", 2024-09-16T00:07:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-16T00:07:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_min_size": "0", 2024-09-16T00:07:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-16T00:07:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_pg_num": "32", 2024-09-16T00:07:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_pgp_num": "0", 2024-09-16T00:07:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-16T00:07:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_read_ratio": "70", 2024-09-16T00:07:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_size": "2", 2024-09-16T00:07:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_default_type": "replicated", 2024-09-16T00:07:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-16T00:07:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-16T00:07:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_push_per_object_cost": "1000", 2024-09-16T00:07:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-16T00:07:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recover_clone_overlap": "true", 2024-09-16T00:07:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-16T00:07:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recovery_cost": "20971520", 2024-09-16T00:07:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-16T00:07:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recovery_max_active": "0", 2024-09-16T00:07:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-16T00:07:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-16T00:07:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-16T00:07:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-16T00:07:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recovery_max_single_start": "1", 2024-09-16T00:07:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recovery_op_priority": "3", 2024-09-16T00:07:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-16T00:07:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recovery_priority": "5", 2024-09-16T00:07:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-16T00:07:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recovery_sleep": "0.000000", 2024-09-16T00:07:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-16T00:07:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-16T00:07:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-16T00:07:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_repair_during_recovery": "false", 2024-09-16T00:07:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_requested_scrub_priority": "120", 2024-09-16T00:07:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-16T00:07:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-16T00:07:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_auto_repair": "false", 2024-09-16T00:07:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-16T00:07:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-16T00:07:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_begin_hour": "0", 2024-09-16T00:07:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_begin_week_day": "0", 2024-09-16T00:07:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_chunk_max": "25", 2024-09-16T00:07:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_chunk_min": "5", 2024-09-16T00:07:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_cost": "52428800", 2024-09-16T00:07:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-16T00:07:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_during_recovery": "false", 2024-09-16T00:07:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_end_hour": "0", 2024-09-16T00:07:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_end_week_day": "0", 2024-09-16T00:07:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_event_cost": "4096", 2024-09-16T00:07:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-16T00:07:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-16T00:07:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_invalid_stats": "true", 2024-09-16T00:07:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-16T00:07:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-16T00:07:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_max_preemptions": "5", 2024-09-16T00:07:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-16T00:07:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_priority": "5", 2024-09-16T00:07:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_scrub_sleep": "0.000000", 2024-09-16T00:07:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-16T00:07:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-16T00:07:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-16T00:07:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_skip_data_digest": "false", 2024-09-16T00:07:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_smart_report_timeout": "5", 2024-09-16T00:07:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_snap_trim_cost": "1048576", 2024-09-16T00:07:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_snap_trim_priority": "5", 2024-09-16T00:07:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-16T00:07:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-16T00:07:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-16T00:07:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-16T00:07:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-16T00:07:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-16T00:07:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-16T00:07:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_target_transaction_size": "30", 2024-09-16T00:07:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-16T00:07:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-16T00:07:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-16T00:07:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-16T00:07:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-16T00:07:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-16T00:07:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-16T00:07:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-16T00:07:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-16T00:07:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-16T00:07:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_tracing": "false", 2024-09-16T00:07:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_use_stale_snap": "false", 2024-09-16T00:07:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-16T00:07:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "osdc_blkin_trace_all": "false", 2024-09-16T00:07:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "paxos_kill_at": "0", 2024-09-16T00:07:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "paxos_max_join_drift": "10", 2024-09-16T00:07:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "paxos_min": "500", 2024-09-16T00:07:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "paxos_min_wait": "0.050000", 2024-09-16T00:07:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "paxos_propose_interval": "1.000000", 2024-09-16T00:07:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "paxos_service_trim_max": "500", 2024-09-16T00:07:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-16T00:07:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "paxos_service_trim_min": "250", 2024-09-16T00:07:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "paxos_stash_full_interval": "25", 2024-09-16T00:07:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "paxos_trim_max": "500", 2024-09-16T00:07:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "paxos_trim_min": "250", 2024-09-16T00:07:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "perf": "true", 2024-09-16T00:07:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-16T00:07:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-16T00:07:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-16T00:07:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "-", 2024-09-16T00:07:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrv": "", 2024-09-16T00:07:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: "public_bind_addr": "-", 2024-09-16T00:07:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: "public_network": "", 2024-09-16T00:07:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: "public_network_interface": "", 2024-09-16T00:07:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: "qat_compressor_busy_polling": "false", 2024-09-16T00:07:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: "qat_compressor_enabled": "false", 2024-09-16T00:07:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: "qat_compressor_session_max_number": "256", 2024-09-16T00:07:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: "rados_mon_op_timeout": "0", 2024-09-16T00:07:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: "rados_osd_op_timeout": "0", 2024-09-16T00:07:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: "rados_tracing": "false", 2024-09-16T00:07:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_atime_update_interval": "60", 2024-09-16T00:07:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-16T00:07:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_balance_parent_reads": "false", 2024-09-16T00:07:52.723 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_balance_snap_reads": "false", 2024-09-16T00:07:52.723 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_blkin_trace_all": "false", 2024-09-16T00:07:52.723 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-16T00:07:52.723 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-16T00:07:52.723 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_cache": "true", 2024-09-16T00:07:52.723 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-16T00:07:52.723 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-16T00:07:52.723 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-16T00:07:52.723 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-16T00:07:52.723 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_cache_policy": "writearound", 2024-09-16T00:07:52.723 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_cache_size": "33554432", 2024-09-16T00:07:52.723 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-16T00:07:52.723 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-16T00:07:52.723 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_clone_copy_on_read": "false", 2024-09-16T00:07:52.724 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_compression_hint": "none", 2024-09-16T00:07:52.724 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_concurrent_management_ops": "10", 2024-09-16T00:07:52.724 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-16T00:07:52.724 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_default_clone_format": "auto", 2024-09-16T00:07:52.724 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_default_data_pool": "", 2024-09-16T00:07:52.724 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_default_features": "61", 2024-09-16T00:07:52.724 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_default_format": "2", 2024-09-16T00:07:52.724 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_default_map_options": "", 2024-09-16T00:07:52.724 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_default_order": "22", 2024-09-16T00:07:52.724 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_default_pool": "rbd", 2024-09-16T00:07:52.724 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-16T00:07:52.724 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_default_stripe_count": "0", 2024-09-16T00:07:52.725 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_default_stripe_unit": "0", 2024-09-16T00:07:52.725 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-16T00:07:52.725 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-16T00:07:52.725 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-16T00:07:52.725 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_enable_alloc_hint": "true", 2024-09-16T00:07:52.725 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-16T00:07:52.725 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_io_scheduler": "simple", 2024-09-16T00:07:52.725 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-16T00:07:52.725 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-16T00:07:52.725 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-16T00:07:52.725 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-16T00:07:52.725 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-16T00:07:52.725 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-16T00:07:52.725 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-16T00:07:52.726 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-16T00:07:52.726 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-16T00:07:52.726 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_journal_order": "24", 2024-09-16T00:07:52.726 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_journal_pool": "", 2024-09-16T00:07:52.726 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_journal_splay_width": "4", 2024-09-16T00:07:52.726 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_localize_parent_reads": "false", 2024-09-16T00:07:52.726 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_localize_snap_reads": "false", 2024-09-16T00:07:52.726 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-16T00:07:52.726 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-16T00:07:52.726 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-16T00:07:52.726 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-16T00:07:52.726 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-16T00:07:52.726 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-16T00:07:52.726 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-16T00:07:52.727 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-16T00:07:52.727 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-16T00:07:52.727 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-16T00:07:52.727 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-16T00:07:52.727 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-16T00:07:52.727 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-16T00:07:52.727 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-16T00:07:52.727 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-16T00:07:52.727 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-16T00:07:52.727 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-16T00:07:52.727 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-16T00:07:52.727 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-16T00:07:52.727 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-16T00:07:52.728 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-16T00:07:52.728 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-16T00:07:52.728 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-16T00:07:52.728 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-16T00:07:52.728 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-16T00:07:52.728 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-16T00:07:52.728 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-16T00:07:52.728 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-16T00:07:52.728 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-16T00:07:52.728 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-16T00:07:52.728 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-16T00:07:52.728 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_mtime_update_interval": "60", 2024-09-16T00:07:52.728 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_non_blocking_aio": "true", 2024-09-16T00:07:52.729 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_op_thread_timeout": "60", 2024-09-16T00:07:52.729 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_op_threads": "1", 2024-09-16T00:07:52.729 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_parent_cache_enabled": "false", 2024-09-16T00:07:52.729 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-16T00:07:52.729 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-16T00:07:52.729 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-16T00:07:52.729 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_plugins": "", 2024-09-16T00:07:52.729 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_bps_burst": "0", 2024-09-16T00:07:52.729 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-16T00:07:52.729 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_bps_limit": "0", 2024-09-16T00:07:52.729 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_exclude_ops": "0", 2024-09-16T00:07:52.729 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_iops_burst": "0", 2024-09-16T00:07:52.729 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-16T00:07:52.730 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_iops_limit": "0", 2024-09-16T00:07:52.730 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-16T00:07:52.730 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-16T00:07:52.730 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-16T00:07:52.730 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-16T00:07:52.730 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-16T00:07:52.730 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-16T00:07:52.730 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-16T00:07:52.730 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-16T00:07:52.730 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-16T00:07:52.730 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-16T00:07:52.730 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-16T00:07:52.730 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-16T00:07:52.730 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-16T00:07:52.731 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-16T00:07:52.731 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_read_from_replica_policy": "default", 2024-09-16T00:07:52.731 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-16T00:07:52.731 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-16T00:07:52.731 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-16T00:07:52.731 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-16T00:07:52.731 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_skip_partial_discard": "true", 2024-09-16T00:07:52.731 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-16T00:07:52.731 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_tracing": "false", 2024-09-16T00:07:52.731 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_validate_names": "true", 2024-09-16T00:07:52.731 INFO:teuthology.orchestra.run.smithi139.stdout: "rbd_validate_pool": "true", 2024-09-16T00:07:52.731 INFO:teuthology.orchestra.run.smithi139.stdout: "restapi_base_url": "", 2024-09-16T00:07:52.731 INFO:teuthology.orchestra.run.smithi139.stdout: "restapi_log_level": "", 2024-09-16T00:07:52.732 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-16T00:07:52.732 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-16T00:07:52.732 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_acl_grants_max_num": "100", 2024-09-16T00:07:52.732 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_admin_entry": "admin", 2024-09-16T00:07:52.732 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-16T00:07:52.732 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_backend_store": "rados", 2024-09-16T00:07:52.732 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_barbican_url": "", 2024-09-16T00:07:52.732 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_beast_enable_async": "true", 2024-09-16T00:07:52.732 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_bucket_counters_cache": "false", 2024-09-16T00:07:52.732 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-16T00:07:52.732 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-16T00:07:52.732 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-16T00:07:52.732 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-16T00:07:52.732 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-16T00:07:52.733 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-16T00:07:52.733 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-16T00:07:52.733 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-16T00:07:52.733 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_cache_enabled": "true", 2024-09-16T00:07:52.733 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_cache_expiry_interval": "900", 2024-09-16T00:07:52.733 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_cache_lru_size": "25000", 2024-09-16T00:07:52.733 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_config_store": "rados", 2024-09-16T00:07:52.733 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_content_length_compat": "false", 2024-09-16T00:07:52.733 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_copy_obj_progress": "true", 2024-09-16T00:07:52.733 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-16T00:07:52.733 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_cors_rules_max_num": "100", 2024-09-16T00:07:52.733 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_cross_domain_policy": "", 2024-09-16T00:07:52.733 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-16T00:07:52.734 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_kmip_addr": "", 2024-09-16T00:07:52.734 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-16T00:07:52.734 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-16T00:07:52.734 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-16T00:07:52.734 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-16T00:07:52.734 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_kmip_password": "", 2024-09-16T00:07:52.734 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-16T00:07:52.734 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_kmip_username": "", 2024-09-16T00:07:52.734 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_require_ssl": "true", 2024-09-16T00:07:52.734 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-16T00:07:52.734 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-16T00:07:52.734 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-16T00:07:52.734 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-16T00:07:52.734 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-16T00:07:52.735 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-16T00:07:52.735 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-16T00:07:52.735 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-16T00:07:52.735 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-16T00:07:52.735 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-16T00:07:52.735 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-16T00:07:52.735 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-16T00:07:52.735 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-16T00:07:52.735 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-16T00:07:52.735 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-16T00:07:52.735 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_vault_addr": "", 2024-09-16T00:07:52.735 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_vault_auth": "token", 2024-09-16T00:07:52.735 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_vault_namespace": "", 2024-09-16T00:07:52.735 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_vault_prefix": "", 2024-09-16T00:07:52.736 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-16T00:07:52.736 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-16T00:07:52.736 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-16T00:07:52.736 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-16T00:07:52.736 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_vault_token_file": "", 2024-09-16T00:07:52.736 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-16T00:07:52.736 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_curl_buffersize": "524288", 2024-09-16T00:07:52.736 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-16T00:07:52.736 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_curl_low_speed_time": "300", 2024-09-16T00:07:52.736 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-16T00:07:52.736 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-16T00:07:52.736 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-16T00:07:52.737 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-16T00:07:52.737 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-16T00:07:52.737 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-16T00:07:52.737 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-16T00:07:52.737 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-16T00:07:52.737 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-16T00:07:52.737 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-16T00:07:52.737 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-16T00:07:52.737 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_d4n_port": "6379", 2024-09-16T00:07:52.737 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-16T00:07:52.737 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_data_log_changes_size": "1000", 2024-09-16T00:07:52.737 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_data_log_num_shards": "128", 2024-09-16T00:07:52.737 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-16T00:07:52.737 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_data_log_window": "30", 2024-09-16T00:07:52.738 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-16T00:07:52.738 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-16T00:07:52.738 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-16T00:07:52.738 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-16T00:07:52.738 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-16T00:07:52.738 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-16T00:07:52.738 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-16T00:07:52.738 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-16T00:07:52.738 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-16T00:07:52.738 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-16T00:07:52.738 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-16T00:07:52.738 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-16T00:07:52.738 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-16T00:07:52.739 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-16T00:07:52.739 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-16T00:07:52.739 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-16T00:07:52.739 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-16T00:07:52.739 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-16T00:07:52.739 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-16T00:07:52.739 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-16T00:07:52.739 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-16T00:07:52.739 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-16T00:07:52.739 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-16T00:07:52.739 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-16T00:07:52.739 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-16T00:07:52.739 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_dns_name": "", 2024-09-16T00:07:52.739 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_dns_s3website_name": "", 2024-09-16T00:07:52.740 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_dynamic_resharding": "true", 2024-09-16T00:07:52.740 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-16T00:07:52.740 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_enable_gc_threads": "true", 2024-09-16T00:07:52.740 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_enable_lc_threads": "true", 2024-09-16T00:07:52.740 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_enable_ops_log": "false", 2024-09-16T00:07:52.740 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_enable_quota_threads": "true", 2024-09-16T00:07:52.740 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_enable_static_website": "false", 2024-09-16T00:07:52.740 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_enable_usage_log": "false", 2024-09-16T00:07:52.740 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_enforce_swift_acls": "true", 2024-09-16T00:07:52.740 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_exit_timeout_secs": "120", 2024-09-16T00:07:52.740 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_expose_bucket": "false", 2024-09-16T00:07:52.740 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_extended_http_attrs": "", 2024-09-16T00:07:52.740 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_filter": "none", 2024-09-16T00:07:52.740 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-16T00:07:52.741 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_frontends": "beast port=7480", 2024-09-16T00:07:52.741 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-16T00:07:52.741 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_gc_max_deferred": "50", 2024-09-16T00:07:52.741 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-16T00:07:52.741 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_gc_max_objs": "32", 2024-09-16T00:07:52.741 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-16T00:07:52.741 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-16T00:07:52.741 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-16T00:07:52.741 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-16T00:07:52.741 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_gc_processor_period": "3600", 2024-09-16T00:07:52.741 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-16T00:07:52.741 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-16T00:07:52.741 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-16T00:07:52.741 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-16T00:07:52.742 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_init_timeout": "300", 2024-09-16T00:07:52.742 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-16T00:07:52.742 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-16T00:07:52.742 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_kafka_connection_idle": "300", 2024-09-16T00:07:52.742 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-16T00:07:52.742 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-16T00:07:52.742 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-16T00:07:52.742 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-16T00:07:52.742 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-16T00:07:52.742 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_admin_domain": "", 2024-09-16T00:07:52.742 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_admin_password": "", 2024-09-16T00:07:52.742 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_admin_password_path": "", 2024-09-16T00:07:52.742 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_admin_project": "", 2024-09-16T00:07:52.743 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_admin_tenant": "", 2024-09-16T00:07:52.743 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_admin_token": "", 2024-09-16T00:07:52.743 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_admin_token_path": "", 2024-09-16T00:07:52.743 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_admin_user": "", 2024-09-16T00:07:52.743 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_api_version": "2", 2024-09-16T00:07:52.743 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_barbican_domain": "", 2024-09-16T00:07:52.743 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_barbican_password": "", 2024-09-16T00:07:52.743 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_barbican_project": "", 2024-09-16T00:07:52.743 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-16T00:07:52.743 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_barbican_user": "", 2024-09-16T00:07:52.743 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-16T00:07:52.743 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-16T00:07:52.743 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-16T00:07:52.744 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-16T00:07:52.744 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-16T00:07:52.744 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_url": "", 2024-09-16T00:07:52.744 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-16T00:07:52.744 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_lc_debug_interval": "-1", 2024-09-16T00:07:52.744 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_lc_lock_max_time": "90", 2024-09-16T00:07:52.744 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_lc_max_objs": "32", 2024-09-16T00:07:52.744 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_lc_max_rules": "1000", 2024-09-16T00:07:52.744 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_lc_max_worker": "3", 2024-09-16T00:07:52.744 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-16T00:07:52.744 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_lc_thread_delay": "0", 2024-09-16T00:07:52.744 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-16T00:07:52.744 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_ldap_dnattr": "uid", 2024-09-16T00:07:52.744 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-16T00:07:52.745 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_ldap_searchfilter": "", 2024-09-16T00:07:52.745 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-16T00:07:52.745 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_ldap_uri": "", 2024-09-16T00:07:52.745 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-16T00:07:52.745 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-16T00:07:52.745 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-16T00:07:52.745 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_log_http_headers": "", 2024-09-16T00:07:52.745 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-16T00:07:52.745 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-16T00:07:52.745 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_log_object_name_utc": "false", 2024-09-16T00:07:52.745 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-16T00:07:52.745 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-16T00:07:52.745 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_max_attr_name_len": "0", 2024-09-16T00:07:52.746 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_max_attr_size": "0", 2024-09-16T00:07:52.746 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-16T00:07:52.746 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_max_chunk_size": "4194304", 2024-09-16T00:07:52.746 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-16T00:07:52.746 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-16T00:07:52.746 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-16T00:07:52.746 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_max_listing_results": "1000", 2024-09-16T00:07:52.746 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_max_notify_retries": "10", 2024-09-16T00:07:52.746 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-16T00:07:52.746 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_max_put_param_size": "1048576", 2024-09-16T00:07:52.746 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_max_put_size": "5368709120", 2024-09-16T00:07:52.746 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_max_slo_entries": "1000", 2024-09-16T00:07:52.746 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_md_log_max_shards": "64", 2024-09-16T00:07:52.746 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-16T00:07:52.747 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-16T00:07:52.747 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-16T00:07:52.747 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-16T00:07:52.747 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_mp_lock_max_time": "600", 2024-09-16T00:07:52.747 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-16T00:07:52.747 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-16T00:07:52.747 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-16T00:07:52.747 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-16T00:07:52.747 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-16T00:07:52.747 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-16T00:07:52.747 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-16T00:07:52.747 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-16T00:07:52.747 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_nfs_max_gc": "300", 2024-09-16T00:07:52.748 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-16T00:07:52.748 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-16T00:07:52.748 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-16T00:07:52.748 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-16T00:07:52.748 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-16T00:07:52.748 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-16T00:07:52.748 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-16T00:07:52.748 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_num_async_rados_threads": "32", 2024-09-16T00:07:52.748 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_num_control_oids": "8", 2024-09-16T00:07:52.748 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_numa_node": "-1", 2024-09-16T00:07:52.748 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-16T00:07:52.748 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-16T00:07:52.748 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_objexp_chunk_size": "100", 2024-09-16T00:07:52.749 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_objexp_gc_interval": "600", 2024-09-16T00:07:52.749 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-16T00:07:52.749 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-16T00:07:52.749 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-16T00:07:52.749 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_op_thread_timeout": "600", 2024-09-16T00:07:52.749 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_op_tracing": "false", 2024-09-16T00:07:52.749 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_opa_token": "", 2024-09-16T00:07:52.749 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_opa_url": "", 2024-09-16T00:07:52.749 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_opa_verify_ssl": "true", 2024-09-16T00:07:52.749 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-16T00:07:52.749 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-16T00:07:52.749 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_ops_log_rados": "false", 2024-09-16T00:07:52.749 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_ops_log_socket_path": "", 2024-09-16T00:07:52.750 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-16T00:07:52.750 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-16T00:07:52.750 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-16T00:07:52.750 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_period_push_interval": "2.000000", 2024-09-16T00:07:52.750 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-16T00:07:52.750 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-16T00:07:52.750 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-16T00:07:52.750 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-16T00:07:52.750 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_posix_cache_lanes": "3", 2024-09-16T00:07:52.750 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-16T00:07:52.750 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-16T00:07:52.750 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_posix_cache_partitions": "3", 2024-09-16T00:07:52.750 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-16T00:07:52.751 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_print_continue": "true", 2024-09-16T00:07:52.751 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-16T00:07:52.751 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-16T00:07:52.751 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-16T00:07:52.751 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-16T00:07:52.751 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-16T00:07:52.751 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_rados_tracing": "false", 2024-09-16T00:07:52.751 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_realm": "", 2024-09-16T00:07:52.751 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_realm_id": "", 2024-09-16T00:07:52.751 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-16T00:07:52.751 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_region": "", 2024-09-16T00:07:52.751 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-16T00:07:52.751 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-16T00:07:52.751 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-16T00:07:52.752 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-16T00:07:52.752 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_request_uri": "", 2024-09-16T00:07:52.752 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_reshard_batch_size": "64", 2024-09-16T00:07:52.752 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-16T00:07:52.752 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_reshard_max_aio": "128", 2024-09-16T00:07:52.752 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_reshard_num_logs": "16", 2024-09-16T00:07:52.752 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_reshard_thread_interval": "600", 2024-09-16T00:07:52.752 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_resolve_cname": "false", 2024-09-16T00:07:52.752 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-16T00:07:52.752 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_run_sync_thread": "true", 2024-09-16T00:07:52.752 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-16T00:07:52.752 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-16T00:07:52.752 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-16T00:07:52.753 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-16T00:07:52.753 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-16T00:07:52.753 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-16T00:07:52.753 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-16T00:07:52.753 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-16T00:07:52.753 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-16T00:07:52.753 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_scheduler_type": "throttler", 2024-09-16T00:07:52.753 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_script_uri": "", 2024-09-16T00:07:52.753 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_service_provider_name": "", 2024-09-16T00:07:52.753 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-16T00:07:52.753 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sts_client_id": "", 2024-09-16T00:07:52.753 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sts_client_secret": "", 2024-09-16T00:07:52.753 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sts_entry": "sts", 2024-09-16T00:07:52.754 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sts_key": "sts", 2024-09-16T00:07:52.754 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-16T00:07:52.754 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sts_min_session_duration": "900", 2024-09-16T00:07:52.754 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sts_token_introspection_url": "", 2024-09-16T00:07:52.754 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_swift_account_in_url": "false", 2024-09-16T00:07:52.754 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_swift_auth_entry": "auth", 2024-09-16T00:07:52.754 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_swift_auth_url": "", 2024-09-16T00:07:52.754 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_swift_custom_header": "", 2024-09-16T00:07:52.754 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-16T00:07:52.754 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_swift_need_stats": "true", 2024-09-16T00:07:52.754 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_swift_tenant_name": "", 2024-09-16T00:07:52.754 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_swift_token_expiration": "86400", 2024-09-16T00:07:52.754 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_swift_url": "", 2024-09-16T00:07:52.754 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_swift_url_prefix": "swift", 2024-09-16T00:07:52.755 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-16T00:07:52.755 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-16T00:07:52.755 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-16T00:07:52.755 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sync_lease_period": "120", 2024-09-16T00:07:52.755 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-16T00:07:52.755 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-16T00:07:52.755 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-16T00:07:52.755 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-16T00:07:52.755 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-16T00:07:52.755 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-16T00:07:52.755 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-16T00:07:52.755 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-16T00:07:52.755 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-16T00:07:52.755 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_thread_pool_size": "512", 2024-09-16T00:07:52.756 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-16T00:07:52.756 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-16T00:07:52.756 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-16T00:07:52.756 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-16T00:07:52.756 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_torrent_comment": "", 2024-09-16T00:07:52.756 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_torrent_createby": "", 2024-09-16T00:07:52.756 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_torrent_encoding": "", 2024-09-16T00:07:52.756 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_torrent_flag": "false", 2024-09-16T00:07:52.756 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-16T00:07:52.756 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_torrent_origin": "", 2024-09-16T00:07:52.756 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-16T00:07:52.756 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_torrent_tracker": "", 2024-09-16T00:07:52.756 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_trust_forwarded_https": "false", 2024-09-16T00:07:52.757 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-16T00:07:52.757 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-16T00:07:52.757 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_usage_max_shards": "32", 2024-09-16T00:07:52.757 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_usage_max_user_shards": "1", 2024-09-16T00:07:52.757 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_use_opa_authz": "false", 2024-09-16T00:07:52.757 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_user_counters_cache": "false", 2024-09-16T00:07:52.757 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-16T00:07:52.757 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-16T00:07:52.757 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-16T00:07:52.757 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_user_max_buckets": "1000", 2024-09-16T00:07:52.757 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_user_policies_max_num": "100", 2024-09-16T00:07:52.757 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-16T00:07:52.757 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-16T00:07:52.757 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-16T00:07:52.758 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-16T00:07:52.758 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_user_unique_email": "true", 2024-09-16T00:07:52.758 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_verify_ssl": "true", 2024-09-16T00:07:52.758 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-16T00:07:52.758 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_zone": "", 2024-09-16T00:07:52.758 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_zone_id": "", 2024-09-16T00:07:52.758 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-16T00:07:52.758 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_zonegroup": "", 2024-09-16T00:07:52.758 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_zonegroup_id": "", 2024-09-16T00:07:52.758 INFO:teuthology.orchestra.run.smithi139.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-16T00:07:52.758 INFO:teuthology.orchestra.run.smithi139.stdout: "rgwlc_auto_session_clear": "true", 2024-09-16T00:07:52.758 INFO:teuthology.orchestra.run.smithi139.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-16T00:07:52.758 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_block_size": "4096", 2024-09-16T00:07:52.759 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-16T00:07:52.759 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-16T00:07:52.759 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-16T00:07:52.759 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-16T00:07:52.759 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-16T00:07:52.759 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_cache_size": "536870912", 2024-09-16T00:07:52.759 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-16T00:07:52.759 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-16T00:07:52.759 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-16T00:07:52.759 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-16T00:07:52.759 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-16T00:07:52.759 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-16T00:07:52.759 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-16T00:07:52.759 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-16T00:07:52.760 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_index_type": "binary_search", 2024-09-16T00:07:52.760 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-16T00:07:52.760 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-16T00:07:52.760 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_partition_filters": "false", 2024-09-16T00:07:52.760 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_perf": "false", 2024-09-16T00:07:52.760 INFO:teuthology.orchestra.run.smithi139.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-16T00:07:52.760 INFO:teuthology.orchestra.run.smithi139.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-16T00:07:52.760 INFO:teuthology.orchestra.run.smithi139.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-16T00:07:52.760 INFO:teuthology.orchestra.run.smithi139.stdout: "run_dir": "/var/run/ceph", 2024-09-16T00:07:52.760 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_block_create": "true", 2024-09-16T00:07:52.760 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_cache_lru_size": "67108864", 2024-09-16T00:07:52.760 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-16T00:07:52.760 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-16T00:07:52.761 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_default_max_object_size": "16777216", 2024-09-16T00:07:52.761 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-16T00:07:52.761 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_device_size": "53687091200", 2024-09-16T00:07:52.761 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-16T00:07:52.761 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_full_integrity_check": "false", 2024-09-16T00:07:52.761 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_journal_batch_capacity": "16", 2024-09-16T00:07:52.761 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-16T00:07:52.761 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-16T00:07:52.761 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-16T00:07:52.761 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_main_device_type": "SSD", 2024-09-16T00:07:52.761 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-16T00:07:52.761 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-16T00:07:52.761 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-16T00:07:52.761 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-16T00:07:52.762 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-16T00:07:52.762 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-16T00:07:52.762 INFO:teuthology.orchestra.run.smithi139.stdout: "seastore_segment_size": "67108864", 2024-09-16T00:07:52.762 INFO:teuthology.orchestra.run.smithi139.stdout: "set_keepcaps": "false", 2024-09-16T00:07:52.762 INFO:teuthology.orchestra.run.smithi139.stdout: "setgroup": "", 2024-09-16T00:07:52.762 INFO:teuthology.orchestra.run.smithi139.stdout: "setuser": "", 2024-09-16T00:07:52.762 INFO:teuthology.orchestra.run.smithi139.stdout: "setuser_match_path": "", 2024-09-16T00:07:52.762 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-16T00:07:52.762 INFO:teuthology.orchestra.run.smithi139.stdout: "thp": "false", 2024-09-16T00:07:52.762 INFO:teuthology.orchestra.run.smithi139.stdout: "threadpool_default_timeout": "60", 2024-09-16T00:07:52.762 INFO:teuthology.orchestra.run.smithi139.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-16T00:07:52.762 INFO:teuthology.orchestra.run.smithi139.stdout: "throttler_perf_counter": "true" 2024-09-16T00:07:52.762 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:07:52.764 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-16T00:07:52.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.859+0000 7f418d827640 1 Processor -- start 2024-09-16T00:07:52.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.859+0000 7f418d827640 1 -- start start 2024-09-16T00:07:52.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.860+0000 7f418d827640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f41880d0870 0x7f41880b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.860+0000 7f418d827640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f41880b3430 0x7f41880b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.860+0000 7f418d827640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f418815adc0 con 0x7f41880d0490 2024-09-16T00:07:52.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.860+0000 7f418d827640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f418815b940 con 0x7f41880d0870 2024-09-16T00:07:52.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.860+0000 7f418d827640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f41880b3bb0 con 0x7f41880b3430 2024-09-16T00:07:52.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.860+0000 7f417ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f41880d0870 0x7f41880b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.860+0000 7f4186ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f41880d0490 0x7f41880b2c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57858/0 (socket says 172.21.15.139:57858) 2024-09-16T00:07:52.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.860+0000 7f4186ffd640 1 -- 172.21.15.139:0/266110093 learned_addr learned my addr 172.21.15.139:0/266110093 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:52.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.860+0000 7f41877fe640 1 --2- 172.21.15.139:0/266110093 >> v2:172.21.15.164:3300/0 conn(0x7f41880b3430 0x7f41880b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.860+0000 7f417ffff640 1 -- 172.21.15.139:0/266110093 >> v2:172.21.15.164:3300/0 conn(0x7f41880b3430 msgr2=0x7f41880b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:52.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.860+0000 7f417ffff640 1 --2- 172.21.15.139:0/266110093 >> v2:172.21.15.164:3300/0 conn(0x7f41880b3430 0x7f41880b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.860+0000 7f417ffff640 1 -- 172.21.15.139:0/266110093 >> v1:172.21.15.139:6789/0 conn(0x7f41880d0490 legacy=0x7f41880b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.860+0000 7f417ffff640 1 -- 172.21.15.139:0/266110093 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418815b0a0 con 0x7f41880d0870 2024-09-16T00:07:52.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.861+0000 7f417ffff640 1 --2- 172.21.15.139:0/266110093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41880d0870 0x7f41880b4040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f41880c72c0 tx=0x7f417002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342d6540cff0409c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:52.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.861+0000 7f41867fc640 1 -- 172.21.15.139:0/266110093 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170037070 con 0x7f41880d0870 2024-09-16T00:07:52.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.861+0000 7f41867fc640 1 -- 172.21.15.139:0/266110093 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4170032440 con 0x7f41880d0870 2024-09-16T00:07:52.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.861+0000 7f41867fc640 1 -- 172.21.15.139:0/266110093 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170032810 con 0x7f41880d0870 2024-09-16T00:07:52.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.862+0000 7f418d827640 1 -- 172.21.15.139:0/266110093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41880d0870 msgr2=0x7f41880b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.862+0000 7f418d827640 1 --2- 172.21.15.139:0/266110093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41880d0870 0x7f41880b4040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f41880c72c0 tx=0x7f417002a230 comp rx=0 tx=0).stop 2024-09-16T00:07:52.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.862+0000 7f418d827640 1 -- 172.21.15.139:0/266110093 shutdown_connections 2024-09-16T00:07:52.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.862+0000 7f418d827640 1 --2- 172.21.15.139:0/266110093 >> v2:172.21.15.164:3300/0 conn(0x7f41880b3430 0x7f41880b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.862+0000 7f418d827640 1 --2- 172.21.15.139:0/266110093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41880d0870 0x7f41880b4040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.862+0000 7f418d827640 1 -- 172.21.15.139:0/266110093 >> 172.21.15.139:0/266110093 conn(0x7f418806c520 msgr2=0x7f418806c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:52.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.862+0000 7f418d827640 1 -- 172.21.15.139:0/266110093 shutdown_connections 2024-09-16T00:07:52.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.862+0000 7f418d827640 1 -- 172.21.15.139:0/266110093 wait complete. 2024-09-16T00:07:52.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.863+0000 7f418d827640 1 Processor -- start 2024-09-16T00:07:52.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.863+0000 7f418d827640 1 -- start start 2024-09-16T00:07:52.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f418d827640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880b3430 0x7f41880cf370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f418d827640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f41880d0490 0x7f41881f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f418d827640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41880d0870 0x7f41881f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f418d827640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f418815a980 con 0x7f41880d0870 2024-09-16T00:07:52.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f418d827640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f418815b660 con 0x7f41880b3430 2024-09-16T00:07:52.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f418d827640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f418815ab00 con 0x7f41880d0490 2024-09-16T00:07:52.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f41877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41880d0870 0x7f41881f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f41877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41880d0870 0x7f41881f2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47762/0 (socket says 172.21.15.139:47762) 2024-09-16T00:07:52.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f41877fe640 1 -- 172.21.15.139:0/2730263481 learned_addr learned my addr 172.21.15.139:0/2730263481 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:52.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f4186ffd640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880b3430 0x7f41880cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f417ffff640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f41880d0490 0x7f41881f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f41877fe640 1 -- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f41880d0490 msgr2=0x7f41881f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f41877fe640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f41880d0490 0x7f41881f24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f41877fe640 1 -- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880b3430 msgr2=0x7f41880cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f41877fe640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880b3430 0x7f41880cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f41877fe640 1 -- 172.21.15.139:0/2730263481 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4188157ec0 con 0x7f41880d0870 2024-09-16T00:07:52.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f4186ffd640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880b3430 0x7f41880cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:52.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f41877fe640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41880d0870 0x7f41881f2a30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4180007e60 tx=0x7f418000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:52.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180014420 con 0x7f41880d0870 2024-09-16T00:07:52.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4180019070 con 0x7f41880d0870 2024-09-16T00:07:52.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f418d827640 1 -- 172.21.15.139:0/2730263481 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4188155ea0 con 0x7f41880d0870 2024-09-16T00:07:52.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4180014820 con 0x7f41880d0870 2024-09-16T00:07:52.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.864+0000 7f418d827640 1 -- 172.21.15.139:0/2730263481 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41880cf8b0 con 0x7f41880d0870 2024-09-16T00:07:52.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.865+0000 7f418d827640 1 -- 172.21.15.139:0/2730263481 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4188151c90 con 0x7f41880d0870 2024-09-16T00:07:52.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.866+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f41800149c0 con 0x7f41880d0870 2024-09-16T00:07:52.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.866+0000 7f4184ff9640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f416806c2b0 0x7f416806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.866+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f418008bbc0 con 0x7f41880d0870 2024-09-16T00:07:52.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.866+0000 7f4186ffd640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f416806c2b0 0x7f416806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.866+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1289 (secure 0 0 0) 0x7f418008e300 con 0x7f41880d0870 2024-09-16T00:07:52.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.866+0000 7f418d827640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4188156d70 0x7f41881f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.866+0000 7f418d827640 1 -- 172.21.15.139:0/2730263481 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f41880d1200 con 0x7f4188156d70 2024-09-16T00:07:52.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.866+0000 7f4186ffd640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f416806c2b0 0x7f416806e770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4178000f60 tx=0x7f4178009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:52.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.867+0000 7f417ffff640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4188156d70 0x7f41881f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.867+0000 7f417ffff640 1 -- 172.21.15.139:0/2730263481 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- command(tid 2: {"prefix": "get_command_descriptions"}) -- 0x7f41880cba70 con 0x7f4188156d70 2024-09-16T00:07:52.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.867+0000 7f417ffff640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4188156d70 0x7f41881f32c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f41880cddf0 tx=0x7f41700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:52.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.867+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170042070 con 0x7f4188156d70 2024-09-16T00:07:52.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.867+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.0 v2:172.21.15.139:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4170032e40 con 0x7f4188156d70 2024-09-16T00:07:52.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.867+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4188156d70 msgr2=0x7f41881f32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.867+0000 7f4184ff9640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4188156d70 0x7f41881f32c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f41880cddf0 tx=0x7f41700046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.867+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4188156d70 msgr2=0x7f41881f32c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T00:07:52.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.867+0000 7f4184ff9640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4188156d70 0x7f41881f32c0 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f41880cddf0 tx=0x7f41700046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.873+0000 7f418d827640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41881572c0 0x7f41881f3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.873+0000 7f418d827640 1 -- 172.21.15.139:0/2730263481 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f418815bdc0 con 0x7f41881572c0 2024-09-16T00:07:52.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.873+0000 7f417ffff640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41881572c0 0x7f41881f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.874+0000 7f417ffff640 1 -- 172.21.15.139:0/2730263481 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- command(tid 3: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7f41881593f0 con 0x7f41881572c0 2024-09-16T00:07:52.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.874+0000 7f417ffff640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41881572c0 0x7f41881f3800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f41880b6020 tx=0x7f4170009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:52.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.874+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41700560a0 con 0x7f41881572c0 2024-09-16T00:07:52.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.875+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.0 v2:172.21.15.139:3300/0 2 ==== command_reply(tid 3: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f4170004030 con 0x7f41881572c0 2024-09-16T00:07:52.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.875+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41881572c0 msgr2=0x7f41881f3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.875+0000 7f4184ff9640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41881572c0 0x7f41881f3800 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f41880b6020 tx=0x7f4170009190 comp rx=0 tx=0).stop 2024-09-16T00:07:52.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.875+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41881572c0 msgr2=0x7f41881f3800 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T00:07:52.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.875+0000 7f4184ff9640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41881572c0 0x7f41881f3800 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.878 INFO:teuthology.orchestra.run.smithi139.stdout:mon.a: {} 2024-09-16T00:07:52.879 INFO:teuthology.orchestra.run.smithi139.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T00:07:52.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.875+0000 7f418d827640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41881f3fa0 0x7f41881f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.875+0000 7f418d827640 1 -- 172.21.15.139:0/2730263481 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f41881f3d40 con 0x7f41881f3fa0 2024-09-16T00:07:52.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.876+0000 7f417ffff640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41881f3fa0 0x7f41881f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.876+0000 7f417ffff640 1 -- 172.21.15.139:0/2730263481 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- command(tid 4: {"prefix": "get_command_descriptions"}) -- 0x7f41880cb790 con 0x7f41881f3fa0 2024-09-16T00:07:52.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.876+0000 7f417ffff640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41881f3fa0 0x7f41881f4380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4170000c00 tx=0x7f4170008030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:52.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.877+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170037040 con 0x7f41881f3fa0 2024-09-16T00:07:52.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.877+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.1 v2:172.21.15.164:3300/0 2 ==== command_reply(tid 4: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f417005e430 con 0x7f41881f3fa0 2024-09-16T00:07:52.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.877+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41881f3fa0 msgr2=0x7f41881f4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.877+0000 7f4184ff9640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41881f3fa0 0x7f41881f4380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4170000c00 tx=0x7f4170008030 comp rx=0 tx=0).stop 2024-09-16T00:07:52.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.877+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41881f3fa0 msgr2=0x7f41881f4380 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T00:07:52.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.877+0000 7f4184ff9640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41881f3fa0 0x7f41881f4380 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.877+0000 7f41877fe640 1 -- 172.21.15.139:0/2730263481 reap_dead start 2024-09-16T00:07:52.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.883+0000 7f418d827640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880cf370 0x7f41881f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.883+0000 7f418d827640 1 -- 172.21.15.139:0/2730263481 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f41880cb4b0 con 0x7f41880cf370 2024-09-16T00:07:52.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.883+0000 7f417ffff640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880cf370 0x7f41881f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.884+0000 7f417ffff640 1 -- 172.21.15.139:0/2730263481 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- command(tid 5: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7f41880b2c10 con 0x7f41880cf370 2024-09-16T00:07:52.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.884+0000 7f417ffff640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880cf370 0x7f41881f24d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f417002a270 tx=0x7f4170054000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:52.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.884+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170037050 con 0x7f41880cf370 2024-09-16T00:07:52.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.885+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.1 v2:172.21.15.164:3300/0 2 ==== command_reply(tid 5: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f4170046b10 con 0x7f41880cf370 2024-09-16T00:07:52.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.885+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880cf370 msgr2=0x7f41881f24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.885+0000 7f4184ff9640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880cf370 0x7f41881f24d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f417002a270 tx=0x7f4170054000 comp rx=0 tx=0).stop 2024-09-16T00:07:52.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.885+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880cf370 msgr2=0x7f41881f24d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T00:07:52.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.885+0000 7f4184ff9640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f41880cf370 0x7f41881f24d0 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f417002a270 tx=0x7f4170054000 comp rx=0 tx=0).stop 2024-09-16T00:07:52.888 INFO:teuthology.orchestra.run.smithi139.stdout:mon.b: {} 2024-09-16T00:07:52.889 INFO:teuthology.orchestra.run.smithi139.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T00:07:52.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.885+0000 7f418d827640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f41881f48c0 0x7f418814e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.886+0000 7f418d827640 1 -- 172.21.15.139:0/2730263481 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f41881563c0 con 0x7f41881f48c0 2024-09-16T00:07:52.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.886+0000 7f417ffff640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f41881f48c0 0x7f418814e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.886+0000 7f417ffff640 1 -- 172.21.15.139:0/2730263481 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f41880691c0 con 0x7f41881f48c0 2024-09-16T00:07:52.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.886+0000 7f417ffff640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f41881f48c0 0x7f418814e760 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4170000c00 tx=0x7f417005d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:52.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.886+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170076070 con 0x7f41881f48c0 2024-09-16T00:07:52.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.887+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.2 v2:172.21.15.164:3301/0 2 ==== command_reply(tid 6: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f41700039c0 con 0x7f41881f48c0 2024-09-16T00:07:52.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.887+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f41881f48c0 msgr2=0x7f418814e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.887+0000 7f4184ff9640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f41881f48c0 0x7f418814e760 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4170000c00 tx=0x7f417005d000 comp rx=0 tx=0).stop 2024-09-16T00:07:52.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.887+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f41881f48c0 msgr2=0x7f418814e760 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T00:07:52.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.887+0000 7f4184ff9640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f41881f48c0 0x7f418814e760 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4170000c00 tx=0x7f417005d000 comp rx=0 tx=0).stop 2024-09-16T00:07:52.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.892+0000 7f418d827640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f418814eca0 0x7f418814f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:52.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.892+0000 7f418d827640 1 -- 172.21.15.139:0/2730263481 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4188157c00 con 0x7f418814eca0 2024-09-16T00:07:52.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.893+0000 7f417ffff640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f418814eca0 0x7f418814f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:52.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.893+0000 7f417ffff640 1 -- 172.21.15.139:0/2730263481 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7f41881f4ca0 con 0x7f418814eca0 2024-09-16T00:07:52.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.893+0000 7f417ffff640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f418814eca0 0x7f418814f080 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f41880b6020 tx=0x7f4170008030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:52.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.894+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41700800a0 con 0x7f418814eca0 2024-09-16T00:07:52.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.895+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 <== mon.2 v2:172.21.15.164:3301/0 2 ==== command_reply(tid 7: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f4170046cc0 con 0x7f418814eca0 2024-09-16T00:07:52.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.895+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f418814eca0 msgr2=0x7f418814f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.895+0000 7f4184ff9640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f418814eca0 0x7f418814f080 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f41880b6020 tx=0x7f4170008030 comp rx=0 tx=0).stop 2024-09-16T00:07:52.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.895+0000 7f4184ff9640 1 -- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f418814eca0 msgr2=0x7f418814f080 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T00:07:52.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.895+0000 7f4184ff9640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f418814eca0 0x7f418814f080 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.898 INFO:teuthology.orchestra.run.smithi139.stdout:mon.c: {} 2024-09-16T00:07:52.899 INFO:teuthology.orchestra.run.smithi139.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T00:07:52.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.895+0000 7f418d827640 1 -- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f416806c2b0 msgr2=0x7f416806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.895+0000 7f418d827640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f416806c2b0 0x7f416806e770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4178000f60 tx=0x7f4178009040 comp rx=0 tx=0).stop 2024-09-16T00:07:52.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.895+0000 7f418d827640 1 -- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41880d0870 msgr2=0x7f41881f2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:52.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.895+0000 7f418d827640 1 --2- 172.21.15.139:0/2730263481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41880d0870 0x7f41881f2a30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4180007e60 tx=0x7f418000e4b0 comp rx=0 tx=0).stop 2024-09-16T00:07:52.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.895+0000 7f41877fe640 1 -- 172.21.15.139:0/2730263481 reap_dead start 2024-09-16T00:07:52.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.896+0000 7f418d827640 1 -- 172.21.15.139:0/2730263481 shutdown_connections 2024-09-16T00:07:52.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.896+0000 7f418d827640 1 -- 172.21.15.139:0/2730263481 >> 172.21.15.139:0/2730263481 conn(0x7f418806c520 msgr2=0x7f4188151160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:52.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.896+0000 7f418d827640 1 -- 172.21.15.139:0/2730263481 shutdown_connections 2024-09-16T00:07:52.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:52.896+0000 7f418d827640 1 -- 172.21.15.139:0/2730263481 wait complete. 2024-09-16T00:07:52.908 DEBUG:teuthology.orchestra.run.smithi164:> type ceph-objectstore-tool 2024-09-16T00:07:52.926 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T00:07:52.926 DEBUG:teuthology.orchestra.run.smithi139:> type ceph-objectstore-tool 2024-09-16T00:07:52.941 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T00:07:52.941 INFO:teuthology.run_tasks:Running task workunit... 2024-09-16T00:07:53.008 INFO:tasks.workunit:Pulling workunits from ref ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T00:07:53.009 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-16T00:07:53.009 INFO:tasks.workunit:timeout=6h 2024-09-16T00:07:53.009 INFO:tasks.workunit:cleanup=True 2024-09-16T00:07:53.010 DEBUG:teuthology.orchestra.run.smithi139:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-16T00:07:53.010 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-16T00:07:53.010 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T00:07:53.010 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:07:53.011 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T00:07:53.011 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T00:07:53.011 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-16T00:07:53.012 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-16T00:07:53.012 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:07:53.012 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-16T00:07:53.012 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:07:53.013 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-16T00:07:53.013 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T00:07:53.014 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:53.012+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:07:53.032 INFO:teuthology.orchestra.run.smithi139.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-16T00:07:53.033 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-16T00:07:53.033 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 2ah/42d Inode: 1 Links: 2 2024-09-16T00:07:53.033 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:07:53.033 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-16T00:07:53.033 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-16T00:07:53.033 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-16 00:07:38.612151510 +0000 2024-09-16T00:07:53.033 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-16 00:07:51.205303996 +0000 2024-09-16T00:07:53.033 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: - 2024-09-16T00:07:53.033 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-16T00:07:53.033 DEBUG:teuthology.orchestra.run.smithi139:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-16T00:07:53.113 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:07:53.113+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:07:53.120 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T00:07:53.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.122+0000 7f63080e4640 1 Processor -- start 2024-09-16T00:07:53.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.122+0000 7f63080e4640 1 -- start start 2024-09-16T00:07:53.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.123+0000 7f63080e4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f63000afbf0 0x7f6300073b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.123+0000 7f63080e4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f63000c3910 0x7f63000c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.123+0000 7f63080e4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f630015ab60 con 0x7f6300151810 2024-09-16T00:07:53.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.123+0000 7f63080e4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6300157940 con 0x7f63000c3910 2024-09-16T00:07:53.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.123+0000 7f63080e4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f63001575f0 con 0x7f63000afbf0 2024-09-16T00:07:53.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.123+0000 7f1b16aaf640 1 Processor -- start 2024-09-16T00:07:53.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.123+0000 7f1b16aaf640 1 -- start start 2024-09-16T00:07:53.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.123+0000 7f6306681640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f63000c3910 0x7f63000c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.123+0000 7f6306681640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f63000c3910 0x7f63000c3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47794/0 (socket says 172.21.15.139:47794) 2024-09-16T00:07:53.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.123+0000 7f6306681640 1 -- 172.21.15.139:0/596619881 learned_addr learned my addr 172.21.15.139:0/596619881 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:53.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.123+0000 7f630567f640 1 --2- 172.21.15.139:0/596619881 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f63000afbf0 0x7f6300073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.123+0000 7f6306681640 1 -- 172.21.15.139:0/596619881 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f63000afbf0 msgr2=0x7f6300073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:53.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.123+0000 7f6306681640 1 --2- 172.21.15.139:0/596619881 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f63000afbf0 0x7f6300073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.123+0000 7f6306681640 1 -- 172.21.15.139:0/596619881 >> v1:172.21.15.164:6789/0 conn(0x7f6300151810 legacy=0x7f6300157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.123+0000 7f6306681640 1 -- 172.21.15.139:0/596619881 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63000c7ed0 con 0x7f63000c3910 2024-09-16T00:07:53.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.123+0000 7f630567f640 1 --2- 172.21.15.139:0/596619881 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f63000afbf0 0x7f6300073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:07:53.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.124+0000 7f1b16aaf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1b10151730 0x7f1b10072830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.124+0000 7f1b16aaf640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1b10073b70 0x7f1b10073030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.124+0000 7f1b16aaf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1b10073570 0x7f1b101573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.124+0000 7f1b16aaf640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1b100c3ad0 con 0x7f1b10151730 2024-09-16T00:07:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.124+0000 7f1b16aaf640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f1b100cd8c0 con 0x7f1b10073b70 2024-09-16T00:07:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.124+0000 7f1b16aaf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1b10151b10 con 0x7f1b10073570 2024-09-16T00:07:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.124+0000 7f1b0ffff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1b10073b70 0x7f1b10073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.124+0000 7f1b1484b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1b10151730 0x7f1b10072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.124+0000 7f1b0ffff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1b10073b70 0x7f1b10073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:46218/0 (socket says 172.21.15.139:46218) 2024-09-16T00:07:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.124+0000 7f1b0ffff640 1 -- 172.21.15.139:0/3141213263 learned_addr learned my addr 172.21.15.139:0/3141213263 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.124+0000 7f1b1504c640 1 --2- 172.21.15.139:0/3141213263 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1b10073570 0x7f1b101573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.125+0000 7f1b1484b640 1 -- 172.21.15.139:0/3141213263 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1b10073570 msgr2=0x7f1b101573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.125+0000 7f1b1484b640 1 --2- 172.21.15.139:0/3141213263 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1b10073570 0x7f1b101573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.125+0000 7f1b1484b640 1 -- 172.21.15.139:0/3141213263 >> v2:172.21.15.164:3300/0 conn(0x7f1b10073b70 msgr2=0x7f1b10073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.125+0000 7f6306681640 1 --2- 172.21.15.139:0/596619881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63000c3910 0x7f63000c3dc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009e80 tx=0x7f62f802a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c7355e1de74bbe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.125+0000 7f1b1484b640 1 --2- 172.21.15.139:0/3141213263 >> v2:172.21.15.164:3300/0 conn(0x7f1b10073b70 0x7f1b10073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.125+0000 7f1b1484b640 1 -- 172.21.15.139:0/3141213263 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b100c3d20 con 0x7f1b10151730 2024-09-16T00:07:53.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.125+0000 7f6304e7e640 1 -- 172.21.15.139:0/596619881 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8036070 con 0x7f63000c3910 2024-09-16T00:07:53.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.125+0000 7f6304e7e640 1 -- 172.21.15.139:0/596619881 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62f8031600 con 0x7f63000c3910 2024-09-16T00:07:53.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.125+0000 7f6304e7e640 1 -- 172.21.15.139:0/596619881 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f80319d0 con 0x7f63000c3910 2024-09-16T00:07:53.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.126+0000 7f1b1484b640 1 --2- 172.21.15.139:0/3141213263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b10151730 0x7f1b10072830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1b1006ff10 tx=0x7f1b0402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7cd38dd0599138 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.126+0000 7f63080e4640 1 -- 172.21.15.139:0/596619881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63000c3910 msgr2=0x7f63000c3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.126+0000 7f63080e4640 1 --2- 172.21.15.139:0/596619881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63000c3910 0x7f63000c3dc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f62f8009e80 tx=0x7f62f802a7e0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.126+0000 7f63080e4640 1 -- 172.21.15.139:0/596619881 shutdown_connections 2024-09-16T00:07:53.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.126+0000 7f1b0f7fe640 1 -- 172.21.15.139:0/3141213263 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04037070 con 0x7f1b10151730 2024-09-16T00:07:53.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.126+0000 7f1b0f7fe640 1 -- 172.21.15.139:0/3141213263 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b04032440 con 0x7f1b10151730 2024-09-16T00:07:53.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.126+0000 7f1b0f7fe640 1 -- 172.21.15.139:0/3141213263 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b04032810 con 0x7f1b10151730 2024-09-16T00:07:53.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.126+0000 7f63080e4640 1 --2- 172.21.15.139:0/596619881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63000c3910 0x7f63000c3dc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.126+0000 7f63080e4640 1 --2- 172.21.15.139:0/596619881 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f63000afbf0 0x7f6300073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.126+0000 7f63080e4640 1 -- 172.21.15.139:0/596619881 >> 172.21.15.139:0/596619881 conn(0x7f6300070440 msgr2=0x7f6300070820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:53.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.126+0000 7fe4a34af640 1 Processor -- start 2024-09-16T00:07:53.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.126+0000 7f63080e4640 1 -- 172.21.15.139:0/596619881 shutdown_connections 2024-09-16T00:07:53.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.126+0000 7f63080e4640 1 -- 172.21.15.139:0/596619881 wait complete. 2024-09-16T00:07:53.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.126+0000 7fe4a34af640 1 -- start start 2024-09-16T00:07:53.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.127+0000 7f1b16aaf640 1 -- 172.21.15.139:0/3141213263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b10151730 msgr2=0x7f1b10072830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.127+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/3141213263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b10151730 0x7f1b10072830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1b1006ff10 tx=0x7f1b0402a1a0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.127+0000 7f1b16aaf640 1 -- 172.21.15.139:0/3141213263 shutdown_connections 2024-09-16T00:07:53.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.127+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/3141213263 >> v2:172.21.15.164:3300/0 conn(0x7f1b10073b70 0x7f1b10073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.127+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/3141213263 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1b10073570 0x7f1b101573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.127+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/3141213263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b10151730 0x7f1b10072830 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.127+0000 7f1b16aaf640 1 -- 172.21.15.139:0/3141213263 >> 172.21.15.139:0/3141213263 conn(0x7f1b10067950 msgr2=0x7f1b10067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:53.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.127+0000 7f1b16aaf640 1 -- 172.21.15.139:0/3141213263 shutdown_connections 2024-09-16T00:07:53.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.127+0000 7f1b16aaf640 1 -- 172.21.15.139:0/3141213263 wait complete. 2024-09-16T00:07:53.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.128+0000 7fe4a34af640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe49c151080 0x7fe49c070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.128+0000 7fe4a124b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe49c151080 0x7fe49c070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.128+0000 7f63080e4640 1 Processor -- start 2024-09-16T00:07:53.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.128+0000 7fe4a124b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe49c151080 0x7fe49c070520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47806/0 (socket says 172.21.15.139:47806) 2024-09-16T00:07:53.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.128+0000 7f63080e4640 1 -- start start 2024-09-16T00:07:53.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.128+0000 7fe4a124b640 1 -- 172.21.15.139:0/752418824 learned_addr learned my addr 172.21.15.139:0/752418824 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:53.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.128+0000 7fe4a34af640 1 --2- 172.21.15.139:0/752418824 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe49c151460 0x7fe49c156f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.128+0000 7fe4a0a4a640 1 --2- 172.21.15.139:0/752418824 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe49c151460 0x7fe49c156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7fe4a34af640 1 -- 172.21.15.139:0/752418824 --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe49c1577d0 con 0x7fe49c0c7a20 2024-09-16T00:07:53.144 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:07:53.144 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:07:53.144 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:07:53.144 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:07:53.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7fe4a34af640 1 -- 172.21.15.139:0/752418824 --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe49c0c8f30 con 0x7fe49c151080 2024-09-16T00:07:53.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7fe4a34af640 1 -- 172.21.15.139:0/752418824 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe49c0c6fd0 con 0x7fe49c151460 2024-09-16T00:07:53.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7fe493fff640 1 -- 172.21.15.139:0/752418824 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 860721224 0 0) 0x7fe49c1577d0 con 0x7fe49c0c7a20 2024-09-16T00:07:53.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7f1b16aaf640 1 Processor -- start 2024-09-16T00:07:53.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7fe493fff640 1 -- 172.21.15.139:0/752418824 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe488003520 con 0x7fe49c0c7a20 2024-09-16T00:07:53.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7fe4a0a4a640 1 -- 172.21.15.139:0/752418824 >> v2:172.21.15.139:3300/0 conn(0x7fe49c151080 msgr2=0x7fe49c070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:53.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7fe4a0a4a640 1 --2- 172.21.15.139:0/752418824 >> v2:172.21.15.139:3300/0 conn(0x7fe49c151080 0x7fe49c070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7fe4a0a4a640 1 -- 172.21.15.139:0/752418824 >> v1:172.21.15.139:6789/0 conn(0x7fe49c0c7a20 legacy=0x7fe49c06cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7fe4a0a4a640 1 -- 172.21.15.139:0/752418824 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c157ab0 con 0x7fe49c151460 2024-09-16T00:07:53.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7f63080e4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f63000afbf0 0x7f6300159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7f63080e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63000c3910 0x7f63001e0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7fe4a124b640 1 --2- 172.21.15.139:0/752418824 >> v2:172.21.15.139:3300/0 conn(0x7fe49c151080 0x7fe49c070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:53.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7f1b16aaf640 1 -- start start 2024-09-16T00:07:53.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7f6305e80640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f63000afbf0 0x7f6300159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7f6305e80640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f63000afbf0 0x7f6300159060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:60856/0 (socket says 172.21.15.139:60856) 2024-09-16T00:07:53.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7f630567f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63000c3910 0x7f63001e0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7f630567f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63000c3910 0x7f63001e0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47812/0 (socket says 172.21.15.139:47812) 2024-09-16T00:07:53.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.129+0000 7f6305e80640 1 -- 172.21.15.139:0/1835914187 learned_addr learned my addr 172.21.15.139:0/1835914187 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:53.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f63080e4640 1 --2- 172.21.15.139:0/1835914187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6300151810 0x7f630014aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f63080e4640 1 -- 172.21.15.139:0/1835914187 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f63000c8770 con 0x7f63000c3910 2024-09-16T00:07:53.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f63080e4640 1 -- 172.21.15.139:0/1835914187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f63000c8490 con 0x7f6300151810 2024-09-16T00:07:53.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f63080e4640 1 -- 172.21.15.139:0/1835914187 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f63001584e0 con 0x7f63000afbf0 2024-09-16T00:07:53.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f6306681640 1 --2- 172.21.15.139:0/1835914187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6300151810 0x7f630014aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f1b16aaf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b10073570 0x7f1b100c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f1b16aaf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b10073b70 0x7f1b100cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7fe4a0a4a640 1 --2- 172.21.15.139:0/752418824 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe49c151460 0x7fe49c156f60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0097a0 tx=0x7fe48c02a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f19f7edd7609ce82 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f1b16aaf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1b10151730 0x7f1b1014e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f1b16aaf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b100cd550 con 0x7f1b10073b70 2024-09-16T00:07:53.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f1b16aaf640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1b100cd270 con 0x7f1b10073570 2024-09-16T00:07:53.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f1b16aaf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1b10073fa0 con 0x7f1b10151730 2024-09-16T00:07:53.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f630567f640 1 -- 172.21.15.139:0/1835914187 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f63000afbf0 msgr2=0x7f6300159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f630567f640 1 --2- 172.21.15.139:0/1835914187 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f63000afbf0 0x7f6300159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f630567f640 1 -- 172.21.15.139:0/1835914187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6300151810 msgr2=0x7f630014aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f630567f640 1 --2- 172.21.15.139:0/1835914187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6300151810 0x7f630014aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f630567f640 1 -- 172.21.15.139:0/1835914187 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63001e2fd0 con 0x7f63000c3910 2024-09-16T00:07:53.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f1b0ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b10073b70 0x7f1b100cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f1b0ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b10073b70 0x7f1b100cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47814/0 (socket says 172.21.15.139:47814) 2024-09-16T00:07:53.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f1b0ffff640 1 -- 172.21.15.139:0/346208734 learned_addr learned my addr 172.21.15.139:0/346208734 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:53.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f1b1504c640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1b10151730 0x7f1b1014e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7f1b1484b640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b10073570 0x7f1b100c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7fe493fff640 1 -- 172.21.15.139:0/752418824 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c037070 con 0x7fe49c151460 2024-09-16T00:07:53.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7fe493fff640 1 -- 172.21.15.139:0/752418824 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe48c02ae40 con 0x7fe49c151460 2024-09-16T00:07:53.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.130+0000 7fe493fff640 1 -- 172.21.15.139:0/752418824 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c032640 con 0x7fe49c151460 2024-09-16T00:07:53.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.131+0000 7f630567f640 1 --2- 172.21.15.139:0/1835914187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63000c3910 0x7f63001e0cf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f62f000c740 tx=0x7f62f000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.131+0000 7f1b0ffff640 1 -- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1b10151730 msgr2=0x7f1b1014e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.131+0000 7f1b0ffff640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1b10151730 0x7f1b1014e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.131+0000 7f1b0ffff640 1 -- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b10073570 msgr2=0x7f1b100c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.131+0000 7f1b0ffff640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1b10073570 0x7f1b100c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.131+0000 7f1b0ffff640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b100cc1a0 con 0x7f1b10073b70 2024-09-16T00:07:53.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.131+0000 7f62f67fc640 1 -- 172.21.15.139:0/1835914187 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f00119e0 con 0x7f63000c3910 2024-09-16T00:07:53.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.131+0000 7f62f67fc640 1 -- 172.21.15.139:0/1835914187 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62f000ce50 con 0x7f63000c3910 2024-09-16T00:07:53.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.131+0000 7f1b0ffff640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b10073b70 0x7f1b100cc7b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00e8b0 tx=0x7f1afc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.131+0000 7f62f67fc640 1 -- 172.21.15.139:0/1835914187 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0011da0 con 0x7f63000c3910 2024-09-16T00:07:53.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.131+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc017070 con 0x7f1b10073b70 2024-09-16T00:07:53.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.131+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1afc00c940 con 0x7f1b10073b70 2024-09-16T00:07:53.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.131+0000 7fe4a34af640 1 -- 172.21.15.139:0/752418824 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe49c151460 msgr2=0x7fe49c156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.131+0000 7fe4a34af640 1 --2- 172.21.15.139:0/752418824 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe49c151460 0x7fe49c156f60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0097a0 tx=0x7fe48c02a0f0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.131+0000 7f63080e4640 1 -- 172.21.15.139:0/1835914187 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63001e2a30 con 0x7f63000c3910 2024-09-16T00:07:53.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.132+0000 7f63080e4640 1 -- 172.21.15.139:0/1835914187 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63001e2cf0 con 0x7f63000c3910 2024-09-16T00:07:53.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.132+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b101f51e0 con 0x7f1b10073b70 2024-09-16T00:07:53.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.132+0000 7fe4a34af640 1 -- 172.21.15.139:0/752418824 shutdown_connections 2024-09-16T00:07:53.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.132+0000 7fe4a34af640 1 --2- 172.21.15.139:0/752418824 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe49c151460 0x7fe49c156f60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.132+0000 7fe4a34af640 1 --2- 172.21.15.139:0/752418824 >> v2:172.21.15.139:3300/0 conn(0x7fe49c151080 0x7fe49c070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.132+0000 7fe4a34af640 1 -- 172.21.15.139:0/752418824 >> 172.21.15.139:0/752418824 conn(0x7fe49c06f800 msgr2=0x7fe49c06fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:53.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.132+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1014ece0 con 0x7f1b10073b70 2024-09-16T00:07:53.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.132+0000 7fe4a34af640 1 -- 172.21.15.139:0/752418824 shutdown_connections 2024-09-16T00:07:53.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.132+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1afc00cd10 con 0x7f1b10073b70 2024-09-16T00:07:53.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.132+0000 7fe4a34af640 1 -- 172.21.15.139:0/752418824 wait complete. 2024-09-16T00:07:53.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.133+0000 7f63080e4640 1 -- 172.21.15.139:0/1835914187 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6300155790 con 0x7f63000c3910 2024-09-16T00:07:53.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.133+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f1b1006d9a0 con 0x7f1b10073b70 2024-09-16T00:07:53.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.134+0000 7f62f67fc640 1 -- 172.21.15.139:0/1835914187 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f62f00075c0 con 0x7f63000c3910 2024-09-16T00:07:53.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.134+0000 7fe4a34af640 1 Processor -- start 2024-09-16T00:07:53.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.134+0000 7fe4a34af640 1 -- start start 2024-09-16T00:07:53.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.134+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1afc01a070 con 0x7f1b10073b70 2024-09-16T00:07:53.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.135+0000 7f1b0d7fa640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1af406c2e0 0x7f1af406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.135+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1afc08b680 con 0x7f1b10073b70 2024-09-16T00:07:53.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.135+0000 7f62f67fc640 1 --2- 172.21.15.139:0/1835914187 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f62e406c330 0x7f62e406e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.135+0000 7f1b1484b640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1af406c2e0 0x7f1af406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.135+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v22) ==== 54+0+16 (secure 0 0 0) 0x7f1afc091170 con 0x7f1b10073b70 2024-09-16T00:07:53.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.135+0000 7fe4a34af640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c0c7a20 0x7fe49c14c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.135+0000 7fe4a34af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49c151080 0x7fe49c1f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.135+0000 7fe4a34af640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe49c147730 0x7fe49c161b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.135+0000 7f62f67fc640 1 -- 172.21.15.139:0/1835914187 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f62f008f290 con 0x7f63000c3910 2024-09-16T00:07:53.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.136+0000 7fe4a34af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c0c7500 con 0x7fe49c151080 2024-09-16T00:07:53.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.136+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f1b101f7f10 0x7f1b101f82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.136+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b100c2800 con 0x7f1b101f7f10 2024-09-16T00:07:53.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.136+0000 7f6305e80640 1 --2- 172.21.15.139:0/1835914187 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f62e406c330 0x7f62e406e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.136+0000 7f1b1484b640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1af406c2e0 0x7f1af406e7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b04034000 tx=0x7f1b0402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.136+0000 7fe4a34af640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c0c7220 con 0x7fe49c0c7a20 2024-09-16T00:07:53.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.136+0000 7fe4a124b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c0c7a20 0x7fe49c14c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.136+0000 7fe4a34af640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe49c070a60 con 0x7fe49c147730 2024-09-16T00:07:53.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.136+0000 7fe4a0a4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49c151080 0x7fe49c1f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.136+0000 7fe4a0a4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49c151080 0x7fe49c1f3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47818/0 (socket says 172.21.15.139:47818) 2024-09-16T00:07:53.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.136+0000 7fe4a124b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c0c7a20 0x7fe49c14c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:46238/0 (socket says 172.21.15.139:46238) 2024-09-16T00:07:53.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.136+0000 7fe4a0a4a640 1 -- 172.21.15.139:0/287075834 learned_addr learned my addr 172.21.15.139:0/287075834 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:53.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.136+0000 7fe4a1a4c640 1 --2- 172.21.15.139:0/287075834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe49c147730 0x7fe49c161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.136+0000 7f6305e80640 1 --2- 172.21.15.139:0/1835914187 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f62e406c330 0x7f62e406e7f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f62fc007fd0 tx=0x7f62fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.136+0000 7f1b1504c640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f1b101f7f10 0x7f1b101f82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.137+0000 7fe4a0a4a640 1 -- 172.21.15.139:0/287075834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe49c147730 msgr2=0x7fe49c161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.137+0000 7fe4a0a4a640 1 --2- 172.21.15.139:0/287075834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe49c147730 0x7fe49c161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.137+0000 7fe4a0a4a640 1 -- 172.21.15.139:0/287075834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c0c7a20 msgr2=0x7fe49c14c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.137+0000 7fe4a0a4a640 1 --2- 172.21.15.139:0/287075834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c0c7a20 0x7fe49c14c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.137+0000 7fe4a0a4a640 1 -- 172.21.15.139:0/287075834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c14bc80 con 0x7fe49c151080 2024-09-16T00:07:53.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.137+0000 7fe4a124b640 1 --2- 172.21.15.139:0/287075834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c0c7a20 0x7fe49c14c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:53.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.137+0000 7f1b1504c640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f1b101f7f10 0x7f1b101f82f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.137+0000 7fe4a0a4a640 1 --2- 172.21.15.139:0/287075834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49c151080 0x7fe49c1f3d70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe48c02a130 tx=0x7fe48c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.137+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b100c2800 con 0x7f1b101f7f10 2024-09-16T00:07:53.177 INFO:tasks.workunit.client.0.smithi139.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-16T00:07:53.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.141+0000 7fe491ffb640 1 -- 172.21.15.139:0/287075834 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c041070 con 0x7fe49c151080 2024-09-16T00:07:53.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.141+0000 7fe4a34af640 1 -- 172.21.15.139:0/287075834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49c14c0a0 con 0x7fe49c151080 2024-09-16T00:07:53.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.141+0000 7fe491ffb640 1 -- 172.21.15.139:0/287075834 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe48c03c670 con 0x7fe49c151080 2024-09-16T00:07:53.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.141+0000 7fe491ffb640 1 -- 172.21.15.139:0/287075834 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c041070 con 0x7fe49c151080 2024-09-16T00:07:53.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.141+0000 7f62f67fc640 1 -- 172.21.15.139:0/1835914187 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6300155790 con 0x7f63000c3910 2024-09-16T00:07:53.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.141+0000 7fe4a34af640 1 -- 172.21.15.139:0/287075834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe49c14b9a0 con 0x7fe49c151080 2024-09-16T00:07:53.181 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:07:53.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.142+0000 7fe4a34af640 1 -- 172.21.15.139:0/287075834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49c14f4e0 con 0x7fe49c151080 2024-09-16T00:07:53.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.143+0000 7fe491ffb640 1 -- 172.21.15.139:0/287075834 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe48c03a070 con 0x7fe49c151080 2024-09-16T00:07:53.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.144+0000 7fe491ffb640 1 --2- 172.21.15.139:0/287075834 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe48006c4b0 0x7fe48006e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.144+0000 7fe491ffb640 1 -- 172.21.15.139:0/287075834 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe48c0ab920 con 0x7fe49c151080 2024-09-16T00:07:53.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.144+0000 7fe4a124b640 1 --2- 172.21.15.139:0/287075834 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe48006c4b0 0x7fe48006e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.144+0000 7fe4a124b640 1 --2- 172.21.15.139:0/287075834 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe48006c4b0 0x7fe48006e970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe498009ee0 tx=0x7fe498008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.150+0000 7fe491ffb640 1 -- 172.21.15.139:0/287075834 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe49c14f4e0 con 0x7fe49c151080 2024-09-16T00:07:53.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.163+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1b10072e10 con 0x7f1b101f7f10 2024-09-16T00:07:53.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.166+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1b10072e10 con 0x7f1b101f7f10 2024-09-16T00:07:53.186 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:07:53.187 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:07:53.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.166+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f1b10143ba0 0x7f1b1014ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.166+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f1b100c79f0 con 0x7f1b10143ba0 2024-09-16T00:07:53.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.166+0000 7f1b1484b640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f1b10143ba0 0x7f1b1014ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.166+0000 7f1b1484b640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f1b10143ba0 0x7f1b1014ca20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.167+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b100c79f0 con 0x7f1b10143ba0 2024-09-16T00:07:53.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.196+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1b1014e760 con 0x7f1b10143ba0 2024-09-16T00:07:53.201 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:07:53.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.198+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1b1014e760 con 0x7f1b10143ba0 2024-09-16T00:07:53.202 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:07:53.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.198+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f1b1014d470 0x7f1b10062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.198+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f1b1014d850 con 0x7f1b1014d470 2024-09-16T00:07:53.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.199+0000 7f1b0ffff640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f1b1014d470 0x7f1b10062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.199+0000 7f1b0ffff640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f1b1014d470 0x7f1b10062630 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.200+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b1014d850 con 0x7f1b1014d470 2024-09-16T00:07:53.214 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:53.214+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:07:53.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.228+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1b10062b70 con 0x7f1b1014d470 2024-09-16T00:07:53.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.230+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1b10062b70 con 0x7f1b1014d470 2024-09-16T00:07:53.232 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:07:53.232 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:07:53.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.230+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f1b100cae60 0x7f1b100cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.230+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f1b100cb7c0 con 0x7f1b100cae60 2024-09-16T00:07:53.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.230+0000 7f1b1504c640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f1b100cae60 0x7f1b100cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.231+0000 7f1b1504c640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f1b100cae60 0x7f1b100cb280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.231+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b100cb7c0 con 0x7f1b100cae60 2024-09-16T00:07:53.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.245+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1b100cb7c0 con 0x7f1b100cae60 2024-09-16T00:07:53.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.248+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1b100cb7c0 con 0x7f1b100cae60 2024-09-16T00:07:53.250 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:07:53.250 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:07:53.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.248+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f1b1002cb50 0x7f1b1002cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.248+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f1b100cb7c0 con 0x7f1b1002cb50 2024-09-16T00:07:53.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.248+0000 7f1b1484b640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f1b1002cb50 0x7f1b1002cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.249+0000 7f1b1484b640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f1b1002cb50 0x7f1b1002cfb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.250+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b100cb7c0 con 0x7f1b1002cb50 2024-09-16T00:07:53.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.263+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1b100cb7c0 con 0x7f1b1002cb50 2024-09-16T00:07:53.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.265+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1b100cb7c0 con 0x7f1b1002cb50 2024-09-16T00:07:53.268 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:07:53.269 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:07:53.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.265+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f1b10208ef0 0x7f1b1020b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.265+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f1b1002da00 con 0x7f1b10208ef0 2024-09-16T00:07:53.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.266+0000 7f1b0ffff640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f1b10208ef0 0x7f1b1020b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.266+0000 7f1b0ffff640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f1b10208ef0 0x7f1b1020b2e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.267+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b1002da00 con 0x7f1b10208ef0 2024-09-16T00:07:53.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.297+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1b1002da00 con 0x7f1b10208ef0 2024-09-16T00:07:53.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.300+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1b1002da00 con 0x7f1b10208ef0 2024-09-16T00:07:53.302 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:07:53.302 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:07:53.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.300+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f1b1020bd30 0x7f1b1020e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.300+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f1b100cbd10 con 0x7f1b1020bd30 2024-09-16T00:07:53.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.300+0000 7f1b1504c640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f1b1020bd30 0x7f1b1020e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.301+0000 7f1b1504c640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f1b1020bd30 0x7f1b1020e120 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.301+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b100cbd10 con 0x7f1b1020bd30 2024-09-16T00:07:53.312 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:53.313 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:53.315 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:07:53.313+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:07:53.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.319+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1b1020e660 con 0x7f1b1020bd30 2024-09-16T00:07:53.323 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:07:53.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.321+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1b1020e660 con 0x7f1b1020bd30 2024-09-16T00:07:53.324 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:07:53.325 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:07:53.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.322+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f1b1020ef30 0x7f1b10211320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.322+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f1b102141c0 con 0x7f1b1020ef30 2024-09-16T00:07:53.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.322+0000 7f1b1484b640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f1b1020ef30 0x7f1b10211320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.322+0000 7f1b1484b640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f1b1020ef30 0x7f1b10211320 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.323+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b102141c0 con 0x7f1b1020ef30 2024-09-16T00:07:53.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.342+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f1b10000950 con 0x7f1b1020ef30 2024-09-16T00:07:53.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.344+0000 7f1b0d7fa640 1 -- 172.21.15.139:0/346208734 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1b10000950 con 0x7f1b1020ef30 2024-09-16T00:07:53.346 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:07:53.347 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:07:53.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.344+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f1b101f7f10 msgr2=0x7f1b101f82f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.344+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f1b101f7f10 0x7f1b101f82f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.344+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f1b10143ba0 msgr2=0x7f1b1014ca20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.344+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f1b10143ba0 0x7f1b1014ca20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.344+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f1b1014d470 msgr2=0x7f1b10062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.344+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f1b1014d470 0x7f1b10062630 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.344+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f1b100cae60 msgr2=0x7f1b100cb280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.344+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f1b100cae60 0x7f1b100cb280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.344+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f1b1002cb50 msgr2=0x7f1b1002cfb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f1b1002cb50 0x7f1b1002cfb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f1b10208ef0 msgr2=0x7f1b1020b2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f1b10208ef0 0x7f1b1020b2e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b1504c640 1 -- 172.21.15.139:0/346208734 reap_dead start 2024-09-16T00:07:53.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f1b1020bd30 msgr2=0x7f1b1020e120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f1b1020bd30 0x7f1b1020e120 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b1504c640 1 -- 172.21.15.139:0/346208734 reap_dead start 2024-09-16T00:07:53.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b1504c640 1 -- 172.21.15.139:0/346208734 reap_dead start 2024-09-16T00:07:53.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b1504c640 1 -- 172.21.15.139:0/346208734 reap_dead start 2024-09-16T00:07:53.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f1b1020ef30 msgr2=0x7f1b10211320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f1b1020ef30 0x7f1b10211320 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1af406c2e0 msgr2=0x7f1af406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1af406c2e0 0x7f1af406e7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b04034000 tx=0x7f1b0402e000 comp rx=0 tx=0).stop 2024-09-16T00:07:53.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b10073b70 msgr2=0x7f1b100cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b10073b70 0x7f1b100cc7b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00e8b0 tx=0x7f1afc00ed70 comp rx=0 tx=0).stop 2024-09-16T00:07:53.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 shutdown_connections 2024-09-16T00:07:53.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f1b1020bd30 0x7f1b1020e120 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f1b1020ef30 0x7f1b10211320 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1af406c2e0 0x7f1af406e7a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 --2- 172.21.15.139:0/346208734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b10073b70 0x7f1b100cc7b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 >> 172.21.15.139:0/346208734 conn(0x7f1b10067950 msgr2=0x7f1b100c84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:53.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 shutdown_connections 2024-09-16T00:07:53.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.345+0000 7f1b16aaf640 1 -- 172.21.15.139:0/346208734 wait complete. 2024-09-16T00:07:53.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.352+0000 7fe4a34af640 1 -- 172.21.15.139:0/287075834 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7fe49c0c6060 con 0x7fe49c151080 2024-09-16T00:07:53.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.388+0000 7f63080e4640 1 -- 172.21.15.139:0/1835914187 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f63000c3130 con 0x7f63000c3910 2024-09-16T00:07:53.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.389+0000 7f62f67fc640 1 -- 172.21.15.139:0/1835914187 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v22) ==== 81+0+34256 (secure 0 0 0) 0x7f62f005d410 con 0x7f63000c3910 2024-09-16T00:07:53.392 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:53.392 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:07:53.392 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 22, 2024-09-16T00:07:53.392 INFO:teuthology.orchestra.run.smithi139.stdout: "fsid": "2d3366e5-6a59-4b59-93b2-615e72ac7015", 2024-09-16T00:07:53.392 INFO:teuthology.orchestra.run.smithi139.stdout: "created": "2024-09-16T00:07:09.967833+0000", 2024-09-16T00:07:53.392 INFO:teuthology.orchestra.run.smithi139.stdout: "modified": "2024-09-16T00:07:34.609954+0000", 2024-09-16T00:07:53.393 INFO:teuthology.orchestra.run.smithi139.stdout: "last_up_change": "2024-09-16T00:07:21.049678+0000", 2024-09-16T00:07:53.393 INFO:teuthology.orchestra.run.smithi139.stdout: "last_in_change": "2024-09-16T00:07:12.025620+0000", 2024-09-16T00:07:53.393 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T00:07:53.393 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_num": 5799936, 2024-09-16T00:07:53.393 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_set": [ 2024-09-16T00:07:53.393 INFO:teuthology.orchestra.run.smithi139.stdout: "pglog_hardlimit", 2024-09-16T00:07:53.393 INFO:teuthology.orchestra.run.smithi139.stdout: "purged_snapdirs", 2024-09-16T00:07:53.393 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_deletes", 2024-09-16T00:07:53.393 INFO:teuthology.orchestra.run.smithi139.stdout: "sortbitwise" 2024-09-16T00:07:53.393 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:07:53.393 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_version": 6, 2024-09-16T00:07:53.393 INFO:teuthology.orchestra.run.smithi139.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T00:07:53.393 INFO:teuthology.orchestra.run.smithi139.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T00:07:53.394 INFO:teuthology.orchestra.run.smithi139.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T00:07:53.394 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_snapshot": "", 2024-09-16T00:07:53.394 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_max": 3, 2024-09-16T00:07:53.394 INFO:teuthology.orchestra.run.smithi139.stdout: "max_osd": 8, 2024-09-16T00:07:53.394 INFO:teuthology.orchestra.run.smithi139.stdout: "require_min_compat_client": "luminous", 2024-09-16T00:07:53.394 INFO:teuthology.orchestra.run.smithi139.stdout: "min_compat_client": "jewel", 2024-09-16T00:07:53.394 INFO:teuthology.orchestra.run.smithi139.stdout: "require_osd_release": "squid", 2024-09-16T00:07:53.394 INFO:teuthology.orchestra.run.smithi139.stdout: "allow_crimson": false, 2024-09-16T00:07:53.394 INFO:teuthology.orchestra.run.smithi139.stdout: "pools": [ 2024-09-16T00:07:53.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.394 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 1, 2024-09-16T00:07:53.394 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": ".mgr", 2024-09-16T00:07:53.394 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:22.056303+0000", 2024-09-16T00:07:53.395 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:07:53.395 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:07:53.395 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:07:53.395 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:07:53.395 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:07:53.395 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:07:53.396 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:07:53.396 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:07:53.396 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:07:53.396 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:07:53.396 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:07:53.396 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:07:53.396 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:07:53.396 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 1, 2024-09-16T00:07:53.396 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 1, 2024-09-16T00:07:53.397 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 1, 2024-09-16T00:07:53.397 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 1, 2024-09-16T00:07:53.397 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 1, 2024-09-16T00:07:53.397 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:07:53.397 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:07:53.397 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:07:53.397 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:07:53.397 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:07:53.397 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:07:53.398 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:07:53.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.398 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "18", 2024-09-16T00:07:53.398 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:07:53.398 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:07:53.398 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:07:53.399 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:07:53.399 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:07:53.399 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:07:53.399 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:07:53.399 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:07:53.399 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:07:53.399 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:07:53.399 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:07:53.399 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:07:53.399 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:07:53.399 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:07:53.400 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:07:53.400 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:07:53.400 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:07:53.400 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:07:53.400 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:07:53.400 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:07:53.400 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:07:53.400 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:07:53.400 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:07:53.400 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:07:53.400 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:07:53.400 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:07:53.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.401 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:07:53.401 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:07:53.401 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:07:53.401 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:07:53.401 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:07:53.401 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:07:53.401 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:07:53.401 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:07:53.401 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:07:53.401 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:07:53.401 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:07:53.401 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:07:53.402 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_max": 32, 2024-09-16T00:07:53.402 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 1 2024-09-16T00:07:53.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.402 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:07:53.402 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": {} 2024-09-16T00:07:53.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.402 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:07:53.402 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:07:53.402 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 8, 2024-09-16T00:07:53.402 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 8, 2024-09-16T00:07:53.402 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 0.25, 2024-09-16T00:07:53.402 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 2, 2024-09-16T00:07:53.402 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 2, 2024-09-16T00:07:53.402 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:07:53.403 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:07:53.403 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:07:53.403 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.404 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 2, 2024-09-16T00:07:53.404 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": "cephfs_metadata", 2024-09-16T00:07:53.404 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:30.768649+0000", 2024-09-16T00:07:53.404 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:07:53.404 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:07:53.404 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:07:53.404 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:07:53.404 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:07:53.405 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:07:53.405 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:07:53.405 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:07:53.405 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:07:53.405 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:07:53.405 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:07:53.405 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:07:53.405 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:07:53.405 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 32, 2024-09-16T00:07:53.405 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 32, 2024-09-16T00:07:53.405 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 32, 2024-09-16T00:07:53.405 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 32, 2024-09-16T00:07:53.405 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 32, 2024-09-16T00:07:53.405 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:07:53.406 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:07:53.406 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:07:53.406 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:07:53.406 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:07:53.406 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:07:53.406 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:07:53.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.406 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "21", 2024-09-16T00:07:53.406 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:07:53.406 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:07:53.407 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:07:53.407 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:07:53.407 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:07:53.407 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:07:53.407 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:07:53.407 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:07:53.408 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:07:53.408 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:07:53.408 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:07:53.408 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:07:53.408 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:07:53.408 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:07:53.409 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:07:53.409 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:07:53.409 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:07:53.409 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:07:53.409 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:07:53.409 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:07:53.410 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:07:53.410 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:07:53.410 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:07:53.410 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:07:53.410 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:07:53.410 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:07:53.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.410 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:07:53.411 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:07:53.411 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:07:53.411 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:07:53.411 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:07:53.411 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:07:53.411 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:07:53.412 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:07:53.412 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:07:53.412 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:07:53.412 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:07:53.412 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:07:53.412 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_bias": 4, 2024-09-16T00:07:53.412 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 16, 2024-09-16T00:07:53.413 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_priority": 5 2024-09-16T00:07:53.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.413 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:07:53.413 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs": { 2024-09-16T00:07:53.413 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": "cephfs" 2024-09-16T00:07:53.413 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.414 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:07:53.414 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:07:53.414 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 1.5, 2024-09-16T00:07:53.414 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 1.5, 2024-09-16T00:07:53.414 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 1, 2024-09-16T00:07:53.414 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 1.5, 2024-09-16T00:07:53.414 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 1.5, 2024-09-16T00:07:53.415 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:07:53.415 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:07:53.415 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:07:53.415 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.416 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 3, 2024-09-16T00:07:53.416 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": "cephfs_data", 2024-09-16T00:07:53.416 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:32.008209+0000", 2024-09-16T00:07:53.416 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:07:53.416 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:07:53.416 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:07:53.417 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:07:53.417 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:07:53.417 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:07:53.417 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:07:53.417 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:07:53.417 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:07:53.417 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:07:53.417 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:07:53.417 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:07:53.417 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:07:53.418 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 64, 2024-09-16T00:07:53.418 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 64, 2024-09-16T00:07:53.418 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 64, 2024-09-16T00:07:53.418 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 64, 2024-09-16T00:07:53.418 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 64, 2024-09-16T00:07:53.418 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:07:53.418 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:07:53.418 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:07:53.418 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:07:53.418 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:07:53.418 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:07:53.418 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:07:53.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.419 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "22", 2024-09-16T00:07:53.419 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:07:53.419 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:07:53.419 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:07:53.419 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:07:53.419 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:07:53.419 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:07:53.419 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:07:53.419 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:07:53.419 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:07:53.419 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:07:53.419 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:07:53.419 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:07:53.419 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:07:53.420 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:07:53.420 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:07:53.420 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:07:53.420 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:07:53.420 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:07:53.420 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:07:53.420 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:07:53.420 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:07:53.420 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:07:53.420 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:07:53.420 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:07:53.420 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:07:53.420 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:07:53.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.421 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:07:53.421 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:07:53.421 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:07:53.421 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:07:53.421 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:07:53.421 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:07:53.421 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:07:53.421 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:07:53.421 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:07:53.421 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:07:53.421 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:07:53.421 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:07:53.421 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 64, 2024-09-16T00:07:53.422 INFO:teuthology.orchestra.run.smithi139.stdout: "target_size_ratio": 0.89999997615814209 2024-09-16T00:07:53.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.422 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:07:53.422 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs": { 2024-09-16T00:07:53.422 INFO:teuthology.orchestra.run.smithi139.stdout: "data": "cephfs" 2024-09-16T00:07:53.422 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.422 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:07:53.422 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:07:53.422 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 1.5, 2024-09-16T00:07:53.422 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 1.5, 2024-09-16T00:07:53.422 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 1, 2024-09-16T00:07:53.422 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 1.5, 2024-09-16T00:07:53.422 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 1.5, 2024-09-16T00:07:53.423 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:07:53.423 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:07:53.423 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:07:53.423 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.423 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.423 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:07:53.423 INFO:teuthology.orchestra.run.smithi139.stdout: "osds": [ 2024-09-16T00:07:53.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.423 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 0, 2024-09-16T00:07:53.423 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "83699a05-4937-43c3-aefc-b669bed9bb6f", 2024-09-16T00:07:53.423 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:07:53.423 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:07:53.423 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:07:53.424 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:07:53.424 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:07:53.424 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:07:53.424 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 14, 2024-09-16T00:07:53.424 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 20, 2024-09-16T00:07:53.424 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:07:53.424 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:07:53.424 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:07:53.424 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.424 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.424 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6816", 2024-09-16T00:07:53.424 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:07:53.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.425 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.425 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6817", 2024-09-16T00:07:53.425 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:07:53.425 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.425 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.425 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:07:53.425 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.425 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.425 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6818", 2024-09-16T00:07:53.425 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:07:53.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.426 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.426 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6819", 2024-09-16T00:07:53.426 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:07:53.426 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.426 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.426 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:07:53.426 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.426 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.426 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6822", 2024-09-16T00:07:53.426 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:07:53.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.427 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.427 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6823", 2024-09-16T00:07:53.427 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:07:53.427 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.427 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.427 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:07:53.427 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.427 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.427 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6820", 2024-09-16T00:07:53.427 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:07:53.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.428 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.428 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6821", 2024-09-16T00:07:53.428 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:07:53.428 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.428 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.428 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6817/1974901438", 2024-09-16T00:07:53.428 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6819/1974901438", 2024-09-16T00:07:53.428 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6823/1974901438", 2024-09-16T00:07:53.428 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6821/1974901438", 2024-09-16T00:07:53.428 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:07:53.428 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:07:53.429 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:07:53.429 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.429 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 1, 2024-09-16T00:07:53.429 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "9c04991e-9695-4bbb-8011-bf613bf404b3", 2024-09-16T00:07:53.429 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:07:53.429 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:07:53.429 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:07:53.429 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:07:53.429 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:07:53.429 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:07:53.429 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 14, 2024-09-16T00:07:53.430 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 20, 2024-09-16T00:07:53.430 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:07:53.430 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:07:53.430 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:07:53.430 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.430 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.430 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6808", 2024-09-16T00:07:53.430 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:07:53.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.430 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.430 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6809", 2024-09-16T00:07:53.430 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:07:53.431 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.431 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.431 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:07:53.431 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.431 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.431 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6810", 2024-09-16T00:07:53.431 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:07:53.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.431 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.431 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6811", 2024-09-16T00:07:53.431 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:07:53.432 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.432 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.432 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:07:53.432 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.432 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.432 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6814", 2024-09-16T00:07:53.432 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:07:53.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.432 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.432 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6815", 2024-09-16T00:07:53.433 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:07:53.433 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.433 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.433 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:07:53.433 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.433 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.433 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6812", 2024-09-16T00:07:53.433 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:07:53.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.433 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.433 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6813", 2024-09-16T00:07:53.434 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:07:53.434 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.434 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.434 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6809/2232380998", 2024-09-16T00:07:53.434 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6811/2232380998", 2024-09-16T00:07:53.434 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6815/2232380998", 2024-09-16T00:07:53.434 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6813/2232380998", 2024-09-16T00:07:53.434 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:07:53.434 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:07:53.435 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:07:53.435 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.435 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 2, 2024-09-16T00:07:53.435 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "8048fc64-72b3-4866-ac62-6a8b2bf6f5c8", 2024-09-16T00:07:53.436 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:07:53.436 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:07:53.436 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:07:53.436 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:07:53.436 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:07:53.436 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:07:53.436 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 13, 2024-09-16T00:07:53.436 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 20, 2024-09-16T00:07:53.436 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:07:53.436 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:07:53.436 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:07:53.436 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.436 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.437 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6800", 2024-09-16T00:07:53.437 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:07:53.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.437 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.437 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6801", 2024-09-16T00:07:53.437 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:07:53.438 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.438 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.438 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:07:53.438 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.439 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.439 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6802", 2024-09-16T00:07:53.439 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:07:53.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.439 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.439 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6803", 2024-09-16T00:07:53.440 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:07:53.440 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.440 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.440 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:07:53.441 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.441 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.441 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6806", 2024-09-16T00:07:53.441 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:07:53.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.442 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.442 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6807", 2024-09-16T00:07:53.442 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:07:53.442 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.443 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.443 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:07:53.443 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.443 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.443 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6804", 2024-09-16T00:07:53.444 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:07:53.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.444 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.444 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6805", 2024-09-16T00:07:53.444 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:07:53.444 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.444 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.444 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6801/4180588905", 2024-09-16T00:07:53.444 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6803/4180588905", 2024-09-16T00:07:53.446 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6807/4180588905", 2024-09-16T00:07:53.446 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6805/4180588905", 2024-09-16T00:07:53.446 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:07:53.446 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:07:53.447 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:07:53.447 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.447 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 3, 2024-09-16T00:07:53.447 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "c34ac77b-354c-4884-902d-3e2a84f5eb27", 2024-09-16T00:07:53.448 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:07:53.448 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:07:53.448 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:07:53.448 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:07:53.448 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:07:53.448 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:07:53.449 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:07:53.449 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 20, 2024-09-16T00:07:53.449 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:07:53.449 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:07:53.449 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:07:53.449 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.450 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.450 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6826", 2024-09-16T00:07:53.450 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:07:53.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.451 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.451 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6827", 2024-09-16T00:07:53.451 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:07:53.452 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.452 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.452 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:07:53.452 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.453 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.453 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6828", 2024-09-16T00:07:53.453 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:07:53.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.454 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.454 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6829", 2024-09-16T00:07:53.454 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:07:53.454 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.454 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.455 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:07:53.455 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.455 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.455 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6832", 2024-09-16T00:07:53.456 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:07:53.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.456 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.456 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6833", 2024-09-16T00:07:53.456 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:07:53.457 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.457 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.457 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:07:53.457 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.458 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.458 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6830", 2024-09-16T00:07:53.458 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:07:53.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.458 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.459 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6831", 2024-09-16T00:07:53.459 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:07:53.459 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.459 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.459 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6827/3947768492", 2024-09-16T00:07:53.460 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6829/3947768492", 2024-09-16T00:07:53.460 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6833/3947768492", 2024-09-16T00:07:53.460 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6831/3947768492", 2024-09-16T00:07:53.460 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:07:53.460 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:07:53.460 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:07:53.461 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.461 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 4, 2024-09-16T00:07:53.461 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "ba9a7c93-9672-43e1-8c98-2878cdf7f548", 2024-09-16T00:07:53.461 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:07:53.462 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:07:53.462 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:07:53.463 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:07:53.463 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:07:53.463 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:07:53.463 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:07:53.464 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 20, 2024-09-16T00:07:53.464 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:07:53.464 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:07:53.464 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:07:53.464 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.465 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.465 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6816", 2024-09-16T00:07:53.465 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:07:53.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.466 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.466 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6817", 2024-09-16T00:07:53.466 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:07:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:07:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6818", 2024-09-16T00:07:53.468 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:07:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6819", 2024-09-16T00:07:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:07:53.469 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:07:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6822", 2024-09-16T00:07:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:07:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6823", 2024-09-16T00:07:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:07:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:07:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.473 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6820", 2024-09-16T00:07:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:07:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6821", 2024-09-16T00:07:53.474 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:07:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6817/642632916", 2024-09-16T00:07:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6819/642632916", 2024-09-16T00:07:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6823/642632916", 2024-09-16T00:07:53.475 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6821/642632916", 2024-09-16T00:07:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:07:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:07:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:07:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 5, 2024-09-16T00:07:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "46a0916b-474c-411b-84ad-86f27d4cbf53", 2024-09-16T00:07:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:07:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:07:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:07:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:07:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:07:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:07:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:07:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 20, 2024-09-16T00:07:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:07:53.477 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:07:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:07:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6800", 2024-09-16T00:07:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:07:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6801", 2024-09-16T00:07:53.478 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:07:53.479 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.479 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.479 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:07:53.479 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.479 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.479 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6802", 2024-09-16T00:07:53.479 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:07:53.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.480 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.480 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6803", 2024-09-16T00:07:53.480 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:07:53.480 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.480 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.480 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:07:53.480 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.480 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.480 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6806", 2024-09-16T00:07:53.481 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:07:53.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.481 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.481 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6807", 2024-09-16T00:07:53.481 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:07:53.481 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.481 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.481 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:07:53.481 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.481 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.482 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6804", 2024-09-16T00:07:53.482 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:07:53.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.482 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.482 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6805", 2024-09-16T00:07:53.482 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:07:53.482 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.482 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.482 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6801/1051778383", 2024-09-16T00:07:53.482 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6803/1051778383", 2024-09-16T00:07:53.482 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6807/1051778383", 2024-09-16T00:07:53.483 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6805/1051778383", 2024-09-16T00:07:53.483 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:07:53.483 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:07:53.483 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:07:53.483 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.483 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 6, 2024-09-16T00:07:53.483 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "9b6b33cc-d6d9-4a0a-8143-526073a58b93", 2024-09-16T00:07:53.483 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:07:53.483 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:07:53.483 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:07:53.483 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:07:53.484 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:07:53.484 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:07:53.484 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:07:53.484 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 20, 2024-09-16T00:07:53.484 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:07:53.484 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:07:53.484 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:07:53.484 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.484 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.484 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6824", 2024-09-16T00:07:53.484 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:07:53.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.485 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.485 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6825", 2024-09-16T00:07:53.485 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:07:53.485 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.485 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.485 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:07:53.485 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.485 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.485 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6826", 2024-09-16T00:07:53.485 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:07:53.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.486 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.486 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6827", 2024-09-16T00:07:53.486 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:07:53.486 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.486 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.486 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:07:53.486 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.486 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.486 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6830", 2024-09-16T00:07:53.487 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:07:53.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.487 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.487 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6831", 2024-09-16T00:07:53.487 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:07:53.487 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.487 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.487 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:07:53.487 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.487 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.487 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6828", 2024-09-16T00:07:53.488 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:07:53.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.488 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.488 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6829", 2024-09-16T00:07:53.488 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:07:53.488 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.488 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.488 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6825/1908495807", 2024-09-16T00:07:53.488 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6827/1908495807", 2024-09-16T00:07:53.488 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6831/1908495807", 2024-09-16T00:07:53.489 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6829/1908495807", 2024-09-16T00:07:53.489 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:07:53.489 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:07:53.489 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:07:53.489 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.489 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 7, 2024-09-16T00:07:53.489 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "50bcd808-3d3e-4a72-88e4-30decb53cfea", 2024-09-16T00:07:53.489 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:07:53.489 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:07:53.489 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:07:53.490 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:07:53.490 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:07:53.490 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:07:53.490 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:07:53.490 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 20, 2024-09-16T00:07:53.490 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:07:53.490 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:07:53.490 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:07:53.490 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.491 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.491 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6808", 2024-09-16T00:07:53.491 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:07:53.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.491 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.491 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6809", 2024-09-16T00:07:53.491 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:07:53.491 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.491 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.491 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:07:53.491 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.492 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.492 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6810", 2024-09-16T00:07:53.492 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:07:53.492 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.492 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.492 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6811", 2024-09-16T00:07:53.493 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:07:53.493 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.493 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.493 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:07:53.493 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.493 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.493 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6814", 2024-09-16T00:07:53.493 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:07:53.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.494 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.494 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6815", 2024-09-16T00:07:53.494 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:07:53.494 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.494 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.494 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:07:53.494 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:07:53.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.495 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:07:53.495 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6812", 2024-09-16T00:07:53.495 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:07:53.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.495 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:07:53.495 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6813", 2024-09-16T00:07:53.495 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:07:53.495 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.495 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.495 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6809/1158838808", 2024-09-16T00:07:53.495 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6811/1158838808", 2024-09-16T00:07:53.495 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6815/1158838808", 2024-09-16T00:07:53.496 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6813/1158838808", 2024-09-16T00:07:53.496 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:07:53.496 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:07:53.496 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:07:53.496 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:53.496 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.496 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:07:53.496 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_xinfo": [ 2024-09-16T00:07:53.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.497 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 0, 2024-09-16T00:07:53.497 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:07:53.497 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:07:53.497 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:07:53.497 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:07:53.497 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:07:53.497 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.843117+0000", 2024-09-16T00:07:53.497 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:07:53.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.497 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 1, 2024-09-16T00:07:53.497 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:07:53.498 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:07:53.498 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:07:53.498 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:07:53.498 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:07:53.498 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.622789+0000", 2024-09-16T00:07:53.498 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:07:53.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.498 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.498 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 2, 2024-09-16T00:07:53.498 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:07:53.499 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:07:53.499 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:07:53.499 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:07:53.499 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:07:53.499 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:17.919457+0000", 2024-09-16T00:07:53.499 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:07:53.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.499 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 3, 2024-09-16T00:07:53.499 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:07:53.500 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:07:53.500 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:07:53.500 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:07:53.500 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:07:53.500 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:19.330341+0000", 2024-09-16T00:07:53.500 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:07:53.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.501 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 4, 2024-09-16T00:07:53.501 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:07:53.501 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:07:53.501 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:07:53.501 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:07:53.501 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:07:53.501 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.815640+0000", 2024-09-16T00:07:53.501 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:07:53.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.502 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 5, 2024-09-16T00:07:53.502 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:07:53.502 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:07:53.502 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:07:53.502 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:07:53.502 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:07:53.502 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.083046+0000", 2024-09-16T00:07:53.502 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:07:53.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.502 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 6, 2024-09-16T00:07:53.503 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:07:53.503 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:07:53.503 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:07:53.503 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:07:53.503 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:07:53.503 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.551499+0000", 2024-09-16T00:07:53.503 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:07:53.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:53.503 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 7, 2024-09-16T00:07:53.504 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:07:53.504 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:07:53.504 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:07:53.504 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:07:53.504 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:07:53.504 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.377179+0000", 2024-09-16T00:07:53.504 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:07:53.504 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.504 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:07:53.504 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap": [], 2024-09-16T00:07:53.504 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap_items": [], 2024-09-16T00:07:53.504 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap_primaries": [], 2024-09-16T00:07:53.504 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_temp": [], 2024-09-16T00:07:53.505 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_temp": [], 2024-09-16T00:07:53.505 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklist": {}, 2024-09-16T00:07:53.505 INFO:teuthology.orchestra.run.smithi139.stdout: "range_blocklist": {}, 2024-09-16T00:07:53.505 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profiles": { 2024-09-16T00:07:53.505 INFO:teuthology.orchestra.run.smithi139.stdout: "default": { 2024-09-16T00:07:53.505 INFO:teuthology.orchestra.run.smithi139.stdout: "crush-failure-domain": "osd", 2024-09-16T00:07:53.505 INFO:teuthology.orchestra.run.smithi139.stdout: "k": "2", 2024-09-16T00:07:53.505 INFO:teuthology.orchestra.run.smithi139.stdout: "m": "1", 2024-09-16T00:07:53.505 INFO:teuthology.orchestra.run.smithi139.stdout: "plugin": "jerasure", 2024-09-16T00:07:53.505 INFO:teuthology.orchestra.run.smithi139.stdout: "technique": "reed_sol_van" 2024-09-16T00:07:53.505 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:53.505 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps_queue": [], 2024-09-16T00:07:53.506 INFO:teuthology.orchestra.run.smithi139.stdout: "new_removed_snaps": [], 2024-09-16T00:07:53.506 INFO:teuthology.orchestra.run.smithi139.stdout: "new_purged_snaps": [], 2024-09-16T00:07:53.506 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_node_flags": {}, 2024-09-16T00:07:53.506 INFO:teuthology.orchestra.run.smithi139.stdout: "device_class_flags": {}, 2024-09-16T00:07:53.506 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode": { 2024-09-16T00:07:53.506 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode_enabled": false, 2024-09-16T00:07:53.506 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_bucket_count": 0, 2024-09-16T00:07:53.506 INFO:teuthology.orchestra.run.smithi139.stdout: "degraded_stretch_mode": 0, 2024-09-16T00:07:53.506 INFO:teuthology.orchestra.run.smithi139.stdout: "recovering_stretch_mode": 0, 2024-09-16T00:07:53.506 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode_bucket": 0 2024-09-16T00:07:53.506 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:53.506 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:07:53.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.391+0000 7f63080e4640 1 -- 172.21.15.139:0/1835914187 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f62e406c330 msgr2=0x7f62e406e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.391+0000 7f63080e4640 1 --2- 172.21.15.139:0/1835914187 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f62e406c330 0x7f62e406e7f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f62fc007fd0 tx=0x7f62fc009040 comp rx=0 tx=0).stop 2024-09-16T00:07:53.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.391+0000 7f63080e4640 1 -- 172.21.15.139:0/1835914187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63000c3910 msgr2=0x7f63001e0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.391+0000 7f63080e4640 1 --2- 172.21.15.139:0/1835914187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63000c3910 0x7f63001e0cf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f62f000c740 tx=0x7f62f000cc00 comp rx=0 tx=0).stop 2024-09-16T00:07:53.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.391+0000 7f63080e4640 1 -- 172.21.15.139:0/1835914187 shutdown_connections 2024-09-16T00:07:53.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.391+0000 7f63080e4640 1 --2- 172.21.15.139:0/1835914187 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f62e406c330 0x7f62e406e7f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.391+0000 7f63080e4640 1 --2- 172.21.15.139:0/1835914187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6300151810 0x7f630014aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.391+0000 7f63080e4640 1 --2- 172.21.15.139:0/1835914187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63000c3910 0x7f63001e0cf0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.391+0000 7f63080e4640 1 --2- 172.21.15.139:0/1835914187 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f63000afbf0 0x7f6300159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.391+0000 7f63080e4640 1 -- 172.21.15.139:0/1835914187 >> 172.21.15.139:0/1835914187 conn(0x7f6300070440 msgr2=0x7f63000cd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:53.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.392+0000 7f63080e4640 1 -- 172.21.15.139:0/1835914187 shutdown_connections 2024-09-16T00:07:53.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.392+0000 7f63080e4640 1 -- 172.21.15.139:0/1835914187 wait complete. 2024-09-16T00:07:53.510 INFO:tasks.thrashosds.thrasher:j is {'epoch': 22, 'fsid': '2d3366e5-6a59-4b59-93b2-615e72ac7015', 'created': '2024-09-16T00:07:09.967833+0000', 'modified': '2024-09-16T00:07:34.609954+0000', 'last_up_change': '2024-09-16T00:07:21.049678+0000', 'last_in_change': '2024-09-16T00:07:12.025620+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:07:22.056303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T00:07:30.768649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T00:07:32.008209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '83699a05-4937-43c3-aefc-b669bed9bb6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6816', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6817', 'nonce': 1974901438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6818', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6819', 'nonce': 1974901438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6822', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6823', 'nonce': 1974901438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6820', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6821', 'nonce': 1974901438}]}, 'public_addr': '172.21.15.139:6817/1974901438', 'cluster_addr': '172.21.15.139:6819/1974901438', 'heartbeat_back_addr': '172.21.15.139:6823/1974901438', 'heartbeat_front_addr': '172.21.15.139:6821/1974901438', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9c04991e-9695-4bbb-8011-bf613bf404b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6808', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6809', 'nonce': 2232380998}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6810', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6811', 'nonce': 2232380998}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6814', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6815', 'nonce': 2232380998}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6812', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6813', 'nonce': 2232380998}]}, 'public_addr': '172.21.15.139:6809/2232380998', 'cluster_addr': '172.21.15.139:6811/2232380998', 'heartbeat_back_addr': '172.21.15.139:6815/2232380998', 'heartbeat_front_addr': '172.21.15.139:6813/2232380998', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8048fc64-72b3-4866-ac62-6a8b2bf6f5c8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6800', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6801', 'nonce': 4180588905}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6802', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6803', 'nonce': 4180588905}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6806', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6807', 'nonce': 4180588905}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6804', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6805', 'nonce': 4180588905}]}, 'public_addr': '172.21.15.139:6801/4180588905', 'cluster_addr': '172.21.15.139:6803/4180588905', 'heartbeat_back_addr': '172.21.15.139:6807/4180588905', 'heartbeat_front_addr': '172.21.15.139:6805/4180588905', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c34ac77b-354c-4884-902d-3e2a84f5eb27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6826', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6827', 'nonce': 3947768492}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6828', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6829', 'nonce': 3947768492}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6832', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6833', 'nonce': 3947768492}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6830', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6831', 'nonce': 3947768492}]}, 'public_addr': '172.21.15.139:6827/3947768492', 'cluster_addr': '172.21.15.139:6829/3947768492', 'heartbeat_back_addr': '172.21.15.139:6833/3947768492', 'heartbeat_front_addr': '172.21.15.139:6831/3947768492', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ba9a7c93-9672-43e1-8c98-2878cdf7f548', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6816', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6817', 'nonce': 642632916}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6818', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 642632916}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 642632916}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6821', 'nonce': 642632916}]}, 'public_addr': '172.21.15.164:6817/642632916', 'cluster_addr': '172.21.15.164:6819/642632916', 'heartbeat_back_addr': '172.21.15.164:6823/642632916', 'heartbeat_front_addr': '172.21.15.164:6821/642632916', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '46a0916b-474c-411b-84ad-86f27d4cbf53', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 1051778383}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 1051778383}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 1051778383}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 1051778383}]}, 'public_addr': '172.21.15.164:6801/1051778383', 'cluster_addr': '172.21.15.164:6803/1051778383', 'heartbeat_back_addr': '172.21.15.164:6807/1051778383', 'heartbeat_front_addr': '172.21.15.164:6805/1051778383', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b6b33cc-d6d9-4a0a-8143-526073a58b93', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6824', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 1908495807}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6827', 'nonce': 1908495807}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6830', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 1908495807}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 1908495807}]}, 'public_addr': '172.21.15.164:6825/1908495807', 'cluster_addr': '172.21.15.164:6827/1908495807', 'heartbeat_back_addr': '172.21.15.164:6831/1908495807', 'heartbeat_front_addr': '172.21.15.164:6829/1908495807', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '50bcd808-3d3e-4a72-88e4-30decb53cfea', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6809', 'nonce': 1158838808}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 1158838808}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6815', 'nonce': 1158838808}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6812', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 1158838808}]}, 'public_addr': '172.21.15.164:6809/1158838808', 'cluster_addr': '172.21.15.164:6811/1158838808', 'heartbeat_back_addr': '172.21.15.164:6815/1158838808', 'heartbeat_front_addr': '172.21.15.164:6813/1158838808', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.843117+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.622789+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:17.919457+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:19.330341+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.815640+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.083046+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.551499+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.377179+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T00:07:53.510 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:07:53.511 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:53.511 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:53.511 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:07:53.512 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:53.507+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:07:53.608 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:53.608+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:07:53.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.650+0000 7f6bc459f640 1 Processor -- start 2024-09-16T00:07:53.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.650+0000 7f6bc459f640 1 -- start start 2024-09-16T00:07:53.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc459f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6bbc1528b0 0x7f6bbc0caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc459f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bbc0c8cc0 0x7f6bbc06fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc459f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6bbc070120 0x7f6bbc0705d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc459f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6bbc152c90 con 0x7f6bbc070120 2024-09-16T00:07:53.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc459f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f6bbc0c9990 con 0x7f6bbc1528b0 2024-09-16T00:07:53.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc459f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6bbc0caaa0 con 0x7f6bbc0c8cc0 2024-09-16T00:07:53.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc2b3c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6bbc070120 0x7f6bbc0705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc2b3c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6bbc070120 0x7f6bbc0705d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47828/0 (socket says 172.21.15.139:47828) 2024-09-16T00:07:53.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc2b3c640 1 -- 172.21.15.139:0/340969051 learned_addr learned my addr 172.21.15.139:0/340969051 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:53.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc1b3a640 1 --2- 172.21.15.139:0/340969051 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bbc0c8cc0 0x7f6bbc06fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc233b640 1 --2- 172.21.15.139:0/340969051 >> v2:172.21.15.164:3300/0 conn(0x7f6bbc1528b0 0x7f6bbc0caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc2b3c640 1 -- 172.21.15.139:0/340969051 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bbc0c8cc0 msgr2=0x7f6bbc06fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:53.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc2b3c640 1 --2- 172.21.15.139:0/340969051 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bbc0c8cc0 0x7f6bbc06fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc2b3c640 1 -- 172.21.15.139:0/340969051 >> v2:172.21.15.164:3300/0 conn(0x7f6bbc1528b0 msgr2=0x7f6bbc0caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:53.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc2b3c640 1 --2- 172.21.15.139:0/340969051 >> v2:172.21.15.164:3300/0 conn(0x7f6bbc1528b0 0x7f6bbc0caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc2b3c640 1 -- 172.21.15.139:0/340969051 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc0c32e0 con 0x7f6bbc070120 2024-09-16T00:07:53.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc1b3a640 1 --2- 172.21.15.139:0/340969051 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bbc0c8cc0 0x7f6bbc06fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:53.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.651+0000 7f6bc233b640 1 --2- 172.21.15.139:0/340969051 >> v2:172.21.15.164:3300/0 conn(0x7f6bbc1528b0 0x7f6bbc0caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:53.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.652+0000 7f6bc2b3c640 1 --2- 172.21.15.139:0/340969051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bbc070120 0x7f6bbc0705d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0ce750 tx=0x7f6bb402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ced172fef680c43 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.652+0000 7f6bc1339640 1 -- 172.21.15.139:0/340969051 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4036070 con 0x7f6bbc070120 2024-09-16T00:07:53.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.652+0000 7f6bc1339640 1 -- 172.21.15.139:0/340969051 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb4031600 con 0x7f6bbc070120 2024-09-16T00:07:53.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.652+0000 7f6bc1339640 1 -- 172.21.15.139:0/340969051 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40319d0 con 0x7f6bbc070120 2024-09-16T00:07:53.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.653+0000 7f6bc459f640 1 -- 172.21.15.139:0/340969051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bbc070120 msgr2=0x7f6bbc0705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.653+0000 7f6bc459f640 1 --2- 172.21.15.139:0/340969051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bbc070120 0x7f6bbc0705d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0ce750 tx=0x7f6bb402a6a0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.653+0000 7f6bc459f640 1 -- 172.21.15.139:0/340969051 shutdown_connections 2024-09-16T00:07:53.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.653+0000 7f6bc459f640 1 --2- 172.21.15.139:0/340969051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bbc070120 0x7f6bbc0705d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.653+0000 7f6bc459f640 1 --2- 172.21.15.139:0/340969051 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bbc0c8cc0 0x7f6bbc06fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.653+0000 7f6bc459f640 1 --2- 172.21.15.139:0/340969051 >> v2:172.21.15.164:3300/0 conn(0x7f6bbc1528b0 0x7f6bbc0caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.653+0000 7f6bc459f640 1 -- 172.21.15.139:0/340969051 >> 172.21.15.139:0/340969051 conn(0x7f6bbc067d70 msgr2=0x7f6bbc068180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:53.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.653+0000 7f6bc459f640 1 -- 172.21.15.139:0/340969051 shutdown_connections 2024-09-16T00:07:53.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.653+0000 7f6bc459f640 1 -- 172.21.15.139:0/340969051 wait complete. 2024-09-16T00:07:53.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.654+0000 7f6bc459f640 1 Processor -- start 2024-09-16T00:07:53.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.654+0000 7f6bc459f640 1 -- start start 2024-09-16T00:07:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.655+0000 7f6bc459f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bbc070120 0x7f6bbc1f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.655+0000 7f6bc459f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bbc0c8cc0 0x7f6bbc0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.655+0000 7f6bc459f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bbc1528b0 0x7f6bbc0cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.655+0000 7f6bc459f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc0c96b0 con 0x7f6bbc1528b0 2024-09-16T00:07:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.655+0000 7f6bc459f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc15c860 con 0x7f6bbc070120 2024-09-16T00:07:53.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.655+0000 7f6bc459f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6bbc0ca730 con 0x7f6bbc0c8cc0 2024-09-16T00:07:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.655+0000 7f6bc2b3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bbc1528b0 0x7f6bbc0cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.655+0000 7f6bc2b3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bbc1528b0 0x7f6bbc0cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47838/0 (socket says 172.21.15.139:47838) 2024-09-16T00:07:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.655+0000 7f6bc2b3c640 1 -- 172.21.15.139:0/2326447421 learned_addr learned my addr 172.21.15.139:0/2326447421 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.655+0000 7f6bc1b3a640 1 --2- 172.21.15.139:0/2326447421 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bbc0c8cc0 0x7f6bbc0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.655+0000 7f6bc233b640 1 --2- 172.21.15.139:0/2326447421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bbc070120 0x7f6bbc1f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.655+0000 7f6bc2b3c640 1 -- 172.21.15.139:0/2326447421 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bbc0c8cc0 msgr2=0x7f6bbc0cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.655+0000 7f6bc2b3c640 1 --2- 172.21.15.139:0/2326447421 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bbc0c8cc0 0x7f6bbc0cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.655+0000 7f6bc2b3c640 1 -- 172.21.15.139:0/2326447421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bbc070120 msgr2=0x7f6bbc1f1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.655+0000 7f6bc2b3c640 1 --2- 172.21.15.139:0/2326447421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bbc070120 0x7f6bbc1f1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.655+0000 7f6bc2b3c640 1 -- 172.21.15.139:0/2326447421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc1f2430 con 0x7f6bbc1528b0 2024-09-16T00:07:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.656+0000 7f6bc2b3c640 1 --2- 172.21.15.139:0/2326447421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bbc1528b0 0x7f6bbc0cc970 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009f40 tx=0x7f6bb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.656+0000 7f6bb2ffd640 1 -- 172.21.15.139:0/2326447421 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4041020 con 0x7f6bbc1528b0 2024-09-16T00:07:53.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.656+0000 7f6bb2ffd640 1 -- 172.21.15.139:0/2326447421 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb4007780 con 0x7f6bbc1528b0 2024-09-16T00:07:53.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.656+0000 7f6bc459f640 1 -- 172.21.15.139:0/2326447421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bbc1f26d0 con 0x7f6bbc1528b0 2024-09-16T00:07:53.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.656+0000 7f6bb2ffd640 1 -- 172.21.15.139:0/2326447421 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4036040 con 0x7f6bbc1528b0 2024-09-16T00:07:53.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.656+0000 7f6bc459f640 1 -- 172.21.15.139:0/2326447421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bbc153f70 con 0x7f6bbc1528b0 2024-09-16T00:07:53.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.657+0000 7f6bc459f640 1 -- 172.21.15.139:0/2326447421 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bbc0cf100 con 0x7f6bbc1528b0 2024-09-16T00:07:53.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.657+0000 7f6bb2ffd640 1 -- 172.21.15.139:0/2326447421 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6bb4044020 con 0x7f6bbc1528b0 2024-09-16T00:07:53.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.658+0000 7f6bb2ffd640 1 --2- 172.21.15.139:0/2326447421 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6ba006c2b0 0x7f6ba006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:53.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.658+0000 7f6bb2ffd640 1 -- 172.21.15.139:0/2326447421 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6bb40ac0c0 con 0x7f6bbc1528b0 2024-09-16T00:07:53.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.658+0000 7f6bc233b640 1 --2- 172.21.15.139:0/2326447421 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6ba006c2b0 0x7f6ba006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:53.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.658+0000 7f6bc233b640 1 --2- 172.21.15.139:0/2326447421 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6ba006c2b0 0x7f6ba006e770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8006fd0 tx=0x7f6bb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:53.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.666+0000 7f6bb2ffd640 1 -- 172.21.15.139:0/2326447421 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6bb404b030 con 0x7f6bbc1528b0 2024-09-16T00:07:53.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.678+0000 7fe491ffb640 1 -- 172.21.15.139:0/287075834 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v23) ==== 87+0+0 (secure 0 0 0) 0x7fe48c079ad0 con 0x7fe49c151080 2024-09-16T00:07:53.681 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is set 2024-09-16T00:07:53.681 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:53.682 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.681+0000 7fe4a34af640 1 -- 172.21.15.139:0/287075834 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe48006c4b0 msgr2=0x7fe48006e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.681+0000 7fe4a34af640 1 --2- 172.21.15.139:0/287075834 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe48006c4b0 0x7fe48006e970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe498009ee0 tx=0x7fe498008040 comp rx=0 tx=0).stop 2024-09-16T00:07:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.681+0000 7fe4a34af640 1 -- 172.21.15.139:0/287075834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49c151080 msgr2=0x7fe49c1f3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.681+0000 7fe4a34af640 1 --2- 172.21.15.139:0/287075834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49c151080 0x7fe49c1f3d70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe48c02a130 tx=0x7fe48c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.681+0000 7fe4a34af640 1 -- 172.21.15.139:0/287075834 shutdown_connections 2024-09-16T00:07:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.681+0000 7fe4a34af640 1 --2- 172.21.15.139:0/287075834 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe48006c4b0 0x7fe48006e970 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.681+0000 7fe4a34af640 1 --2- 172.21.15.139:0/287075834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe49c147730 0x7fe49c161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.681+0000 7fe4a34af640 1 --2- 172.21.15.139:0/287075834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe49c151080 0x7fe49c1f3d70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.681+0000 7fe4a34af640 1 --2- 172.21.15.139:0/287075834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe49c0c7a20 0x7fe49c14c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.681+0000 7fe4a34af640 1 -- 172.21.15.139:0/287075834 >> 172.21.15.139:0/287075834 conn(0x7fe49c06f800 msgr2=0x7fe49c06ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:53.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.682+0000 7fe4a34af640 1 -- 172.21.15.139:0/287075834 shutdown_connections 2024-09-16T00:07:53.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.682+0000 7fe4a34af640 1 -- 172.21.15.139:0/287075834 wait complete. 2024-09-16T00:07:53.693 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:07:53.709 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:53.709+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:07:53.802 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:53.802 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:53.811 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:07:53.811 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:07:53.810+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:07:53.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.846+0000 7f6bc459f640 1 -- 172.21.15.139:0/2326447421 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6bbc0c47a0 con 0x7f6ba006c2b0 2024-09-16T00:07:53.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.852+0000 7f6bb2ffd640 1 -- 172.21.15.139:0/2326447421 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251912 (secure 0 0 0) 0x7f6bbc0c47a0 con 0x7f6ba006c2b0 2024-09-16T00:07:53.856 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:07:53.857 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:07:53.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.855+0000 7f6bc459f640 1 -- 172.21.15.139:0/2326447421 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6ba006c2b0 msgr2=0x7f6ba006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.855+0000 7f6bc459f640 1 --2- 172.21.15.139:0/2326447421 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6ba006c2b0 0x7f6ba006e770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8006fd0 tx=0x7f6bb8008040 comp rx=0 tx=0).stop 2024-09-16T00:07:53.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.855+0000 7f6bc459f640 1 -- 172.21.15.139:0/2326447421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bbc1528b0 msgr2=0x7f6bbc0cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:53.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.855+0000 7f6bc459f640 1 --2- 172.21.15.139:0/2326447421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bbc1528b0 0x7f6bbc0cc970 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009f40 tx=0x7f6bb4004a40 comp rx=0 tx=0).stop 2024-09-16T00:07:53.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.855+0000 7f6bc459f640 1 -- 172.21.15.139:0/2326447421 shutdown_connections 2024-09-16T00:07:53.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.855+0000 7f6bc459f640 1 --2- 172.21.15.139:0/2326447421 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6ba006c2b0 0x7f6ba006e770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.855+0000 7f6bc459f640 1 --2- 172.21.15.139:0/2326447421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bbc1528b0 0x7f6bbc0cc970 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.855+0000 7f6bc459f640 1 --2- 172.21.15.139:0/2326447421 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6bbc0c8cc0 0x7f6bbc0cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.855+0000 7f6bc459f640 1 --2- 172.21.15.139:0/2326447421 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6bbc070120 0x7f6bbc1f1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:53.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.855+0000 7f6bc459f640 1 -- 172.21.15.139:0/2326447421 >> 172.21.15.139:0/2326447421 conn(0x7f6bbc067d70 msgr2=0x7f6bbc071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:53.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.855+0000 7f6bc459f640 1 -- 172.21.15.139:0/2326447421 shutdown_connections 2024-09-16T00:07:53.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:53.855+0000 7f6bc459f640 1 -- 172.21.15.139:0/2326447421 wait complete. 2024-09-16T00:07:53.869 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-16T00:07:53.680912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593221,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":205,"num_write_kb":2431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218112,"kb_used_data":1920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756608,"statfs":{"total":773094113280,"available":772870766592,"internally_reserved":0,"allocated":1966080,"data_stored":1809967,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2077,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.653718"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115528+0000","last_change":"2024-09-16T00:07:33.635096+0000","last_active":"2024-09-16T00:07:35.115528+0000","last_peered":"2024-09-16T00:07:35.115528+0000","last_clean":"2024-09-16T00:07:35.115528+0000","last_became_active":"2024-09-16T00:07:33.634750+0000","last_became_peered":"2024-09-16T00:07:33.634750+0000","last_unstale":"2024-09-16T00:07:35.115528+0000","last_undegraded":"2024-09-16T00:07:35.115528+0000","last_fullsized":"2024-09-16T00:07:35.115528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115558+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:07:35.115558+0000","last_peered":"2024-09-16T00:07:35.115558+0000","last_clean":"2024-09-16T00:07:35.115558+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:07:35.115558+0000","last_undegraded":"2024-09-16T00:07:35.115558+0000","last_fullsized":"2024-09-16T00:07:35.115558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115392+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:07:35.115392+0000","last_peered":"2024-09-16T00:07:35.115392+0000","last_clean":"2024-09-16T00:07:35.115392+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:07:35.115392+0000","last_undegraded":"2024-09-16T00:07:35.115392+0000","last_fullsized":"2024-09-16T00:07:35.115392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615167+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:07:34.615167+0000","last_peered":"2024-09-16T00:07:34.615167+0000","last_clean":"2024-09-16T00:07:34.615167+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:07:34.615167+0000","last_undegraded":"2024-09-16T00:07:34.615167+0000","last_fullsized":"2024-09-16T00:07:34.615167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115724+0000","last_change":"2024-09-16T00:07:33.630029+0000","last_active":"2024-09-16T00:07:35.115724+0000","last_peered":"2024-09-16T00:07:35.115724+0000","last_clean":"2024-09-16T00:07:35.115724+0000","last_became_active":"2024-09-16T00:07:33.629712+0000","last_became_peered":"2024-09-16T00:07:33.629712+0000","last_unstale":"2024-09-16T00:07:35.115724+0000","last_undegraded":"2024-09-16T00:07:35.115724+0000","last_fullsized":"2024-09-16T00:07:35.115724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614973+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:07:34.614973+0000","last_peered":"2024-09-16T00:07:34.614973+0000","last_clean":"2024-09-16T00:07:34.614973+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:07:34.614973+0000","last_undegraded":"2024-09-16T00:07:34.614973+0000","last_fullsized":"2024-09-16T00:07:34.614973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115522+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:07:35.115522+0000","last_peered":"2024-09-16T00:07:35.115522+0000","last_clean":"2024-09-16T00:07:35.115522+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:07:35.115522+0000","last_undegraded":"2024-09-16T00:07:35.115522+0000","last_fullsized":"2024-09-16T00:07:35.115522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114492+0000","last_change":"2024-09-16T00:07:33.624708+0000","last_active":"2024-09-16T00:07:35.114492+0000","last_peered":"2024-09-16T00:07:35.114492+0000","last_clean":"2024-09-16T00:07:35.114492+0000","last_became_active":"2024-09-16T00:07:33.624458+0000","last_became_peered":"2024-09-16T00:07:33.624458+0000","last_unstale":"2024-09-16T00:07:35.114492+0000","last_undegraded":"2024-09-16T00:07:35.114492+0000","last_fullsized":"2024-09-16T00:07:35.114492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:48.052488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115751+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:07:35.115751+0000","last_peered":"2024-09-16T00:07:35.115751+0000","last_clean":"2024-09-16T00:07:35.115751+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:07:35.115751+0000","last_undegraded":"2024-09-16T00:07:35.115751+0000","last_fullsized":"2024-09-16T00:07:35.115751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115614+0000","last_change":"2024-09-16T00:07:33.635973+0000","last_active":"2024-09-16T00:07:35.115614+0000","last_peered":"2024-09-16T00:07:35.115614+0000","last_clean":"2024-09-16T00:07:35.115614+0000","last_became_active":"2024-09-16T00:07:33.635486+0000","last_became_peered":"2024-09-16T00:07:33.635486+0000","last_unstale":"2024-09-16T00:07:35.115614+0000","last_undegraded":"2024-09-16T00:07:35.115614+0000","last_fullsized":"2024-09-16T00:07:35.115614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115027+0000","last_change":"2024-09-16T00:07:33.636971+0000","last_active":"2024-09-16T00:07:35.115027+0000","last_peered":"2024-09-16T00:07:35.115027+0000","last_clean":"2024-09-16T00:07:35.115027+0000","last_became_active":"2024-09-16T00:07:33.635760+0000","last_became_peered":"2024-09-16T00:07:33.635760+0000","last_unstale":"2024-09-16T00:07:35.115027+0000","last_undegraded":"2024-09-16T00:07:35.115027+0000","last_fullsized":"2024-09-16T00:07:35.115027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614631+0000","last_change":"2024-09-16T00:07:33.632261+0000","last_active":"2024-09-16T00:07:34.614631+0000","last_peered":"2024-09-16T00:07:34.614631+0000","last_clean":"2024-09-16T00:07:34.614631+0000","last_became_active":"2024-09-16T00:07:33.631944+0000","last_became_peered":"2024-09-16T00:07:33.631944+0000","last_unstale":"2024-09-16T00:07:34.614631+0000","last_undegraded":"2024-09-16T00:07:34.614631+0000","last_fullsized":"2024-09-16T00:07:34.614631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:35.052417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114641+0000","last_change":"2024-09-16T00:07:33.628273+0000","last_active":"2024-09-16T00:07:35.114641+0000","last_peered":"2024-09-16T00:07:35.114641+0000","last_clean":"2024-09-16T00:07:35.114641+0000","last_became_active":"2024-09-16T00:07:33.627960+0000","last_became_peered":"2024-09-16T00:07:33.627960+0000","last_unstale":"2024-09-16T00:07:35.114641+0000","last_undegraded":"2024-09-16T00:07:35.114641+0000","last_fullsized":"2024-09-16T00:07:35.114641+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:36.451688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114782+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:07:35.114782+0000","last_peered":"2024-09-16T00:07:35.114782+0000","last_clean":"2024-09-16T00:07:35.114782+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:07:35.114782+0000","last_undegraded":"2024-09-16T00:07:35.114782+0000","last_fullsized":"2024-09-16T00:07:35.114782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114664+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:07:35.114664+0000","last_peered":"2024-09-16T00:07:35.114664+0000","last_clean":"2024-09-16T00:07:35.114664+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:07:35.114664+0000","last_undegraded":"2024-09-16T00:07:35.114664+0000","last_fullsized":"2024-09-16T00:07:35.114664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614471+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:07:34.614471+0000","last_peered":"2024-09-16T00:07:34.614471+0000","last_clean":"2024-09-16T00:07:34.614471+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:07:34.614471+0000","last_undegraded":"2024-09-16T00:07:34.614471+0000","last_fullsized":"2024-09-16T00:07:34.614471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615697+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:07:34.615697+0000","last_peered":"2024-09-16T00:07:34.615697+0000","last_clean":"2024-09-16T00:07:34.615697+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:07:34.615697+0000","last_undegraded":"2024-09-16T00:07:34.615697+0000","last_fullsized":"2024-09-16T00:07:34.615697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613100+0000","last_change":"2024-09-16T00:07:33.630797+0000","last_active":"2024-09-16T00:07:34.613100+0000","last_peered":"2024-09-16T00:07:34.613100+0000","last_clean":"2024-09-16T00:07:34.613100+0000","last_became_active":"2024-09-16T00:07:33.630491+0000","last_became_peered":"2024-09-16T00:07:33.630491+0000","last_unstale":"2024-09-16T00:07:34.613100+0000","last_undegraded":"2024-09-16T00:07:34.613100+0000","last_fullsized":"2024-09-16T00:07:34.613100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:16:20.097626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115840+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:07:35.115840+0000","last_peered":"2024-09-16T00:07:35.115840+0000","last_clean":"2024-09-16T00:07:35.115840+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:07:35.115840+0000","last_undegraded":"2024-09-16T00:07:35.115840+0000","last_fullsized":"2024-09-16T00:07:35.115840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114960+0000","last_change":"2024-09-16T00:07:33.638554+0000","last_active":"2024-09-16T00:07:35.114960+0000","last_peered":"2024-09-16T00:07:35.114960+0000","last_clean":"2024-09-16T00:07:35.114960+0000","last_became_active":"2024-09-16T00:07:33.638263+0000","last_became_peered":"2024-09-16T00:07:33.638263+0000","last_unstale":"2024-09-16T00:07:35.114960+0000","last_undegraded":"2024-09-16T00:07:35.114960+0000","last_fullsized":"2024-09-16T00:07:35.114960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115286+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:07:35.115286+0000","last_peered":"2024-09-16T00:07:35.115286+0000","last_clean":"2024-09-16T00:07:35.115286+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:07:35.115286+0000","last_undegraded":"2024-09-16T00:07:35.115286+0000","last_fullsized":"2024-09-16T00:07:35.115286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615450+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:07:34.615450+0000","last_peered":"2024-09-16T00:07:34.615450+0000","last_clean":"2024-09-16T00:07:34.615450+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:07:34.615450+0000","last_undegraded":"2024-09-16T00:07:34.615450+0000","last_fullsized":"2024-09-16T00:07:34.615450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115015+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:07:35.115015+0000","last_peered":"2024-09-16T00:07:35.115015+0000","last_clean":"2024-09-16T00:07:35.115015+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:07:35.115015+0000","last_undegraded":"2024-09-16T00:07:35.115015+0000","last_fullsized":"2024-09-16T00:07:35.115015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614616+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:07:34.614616+0000","last_peered":"2024-09-16T00:07:34.614616+0000","last_clean":"2024-09-16T00:07:34.614616+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:07:34.614616+0000","last_undegraded":"2024-09-16T00:07:34.614616+0000","last_fullsized":"2024-09-16T00:07:34.614616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114694+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:07:35.114694+0000","last_peered":"2024-09-16T00:07:35.114694+0000","last_clean":"2024-09-16T00:07:35.114694+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:07:35.114694+0000","last_undegraded":"2024-09-16T00:07:35.114694+0000","last_fullsized":"2024-09-16T00:07:35.114694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115306+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:07:35.115306+0000","last_peered":"2024-09-16T00:07:35.115306+0000","last_clean":"2024-09-16T00:07:35.115306+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:07:35.115306+0000","last_undegraded":"2024-09-16T00:07:35.115306+0000","last_fullsized":"2024-09-16T00:07:35.115306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115543+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:07:35.115543+0000","last_peered":"2024-09-16T00:07:35.115543+0000","last_clean":"2024-09-16T00:07:35.115543+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:07:35.115543+0000","last_undegraded":"2024-09-16T00:07:35.115543+0000","last_fullsized":"2024-09-16T00:07:35.115543+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114985+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:07:35.114985+0000","last_peered":"2024-09-16T00:07:35.114985+0000","last_clean":"2024-09-16T00:07:35.114985+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:07:35.114985+0000","last_undegraded":"2024-09-16T00:07:35.114985+0000","last_fullsized":"2024-09-16T00:07:35.114985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115448+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:07:35.115448+0000","last_peered":"2024-09-16T00:07:35.115448+0000","last_clean":"2024-09-16T00:07:35.115448+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:07:35.115448+0000","last_undegraded":"2024-09-16T00:07:35.115448+0000","last_fullsized":"2024-09-16T00:07:35.115448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615338+0000","last_change":"2024-09-16T00:07:33.627789+0000","last_active":"2024-09-16T00:07:34.615338+0000","last_peered":"2024-09-16T00:07:34.615338+0000","last_clean":"2024-09-16T00:07:34.615338+0000","last_became_active":"2024-09-16T00:07:33.627500+0000","last_became_peered":"2024-09-16T00:07:33.627500+0000","last_unstale":"2024-09-16T00:07:34.615338+0000","last_undegraded":"2024-09-16T00:07:34.615338+0000","last_fullsized":"2024-09-16T00:07:34.615338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613050+0000","last_change":"2024-09-16T00:07:33.621107+0000","last_active":"2024-09-16T00:07:34.613050+0000","last_peered":"2024-09-16T00:07:34.613050+0000","last_clean":"2024-09-16T00:07:34.613050+0000","last_became_active":"2024-09-16T00:07:33.620858+0000","last_became_peered":"2024-09-16T00:07:33.620858+0000","last_unstale":"2024-09-16T00:07:34.613050+0000","last_undegraded":"2024-09-16T00:07:34.613050+0000","last_fullsized":"2024-09-16T00:07:34.613050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:12:30.176079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115031+0000","last_change":"2024-09-16T00:07:33.633578+0000","last_active":"2024-09-16T00:07:35.115031+0000","last_peered":"2024-09-16T00:07:35.115031+0000","last_clean":"2024-09-16T00:07:35.115031+0000","last_became_active":"2024-09-16T00:07:33.633310+0000","last_became_peered":"2024-09-16T00:07:33.633310+0000","last_unstale":"2024-09-16T00:07:35.115031+0000","last_undegraded":"2024-09-16T00:07:35.115031+0000","last_fullsized":"2024-09-16T00:07:35.115031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614556+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:07:34.614556+0000","last_peered":"2024-09-16T00:07:34.614556+0000","last_clean":"2024-09-16T00:07:34.614556+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:07:34.614556+0000","last_undegraded":"2024-09-16T00:07:34.614556+0000","last_fullsized":"2024-09-16T00:07:34.614556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.059131+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:07:36.059131+0000","last_peered":"2024-09-16T00:07:36.059131+0000","last_clean":"2024-09-16T00:07:36.059131+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:07:36.059131+0000","last_undegraded":"2024-09-16T00:07:36.059131+0000","last_fullsized":"2024-09-16T00:07:36.059131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115509+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:07:35.115509+0000","last_peered":"2024-09-16T00:07:35.115509+0000","last_clean":"2024-09-16T00:07:35.115509+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:07:35.115509+0000","last_undegraded":"2024-09-16T00:07:35.115509+0000","last_fullsized":"2024-09-16T00:07:35.115509+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114845+0000","last_change":"2024-09-16T00:07:32.626800+0000","last_active":"2024-09-16T00:07:35.114845+0000","last_peered":"2024-09-16T00:07:35.114845+0000","last_clean":"2024-09-16T00:07:35.114845+0000","last_became_active":"2024-09-16T00:07:32.626511+0000","last_became_peered":"2024-09-16T00:07:32.626511+0000","last_unstale":"2024-09-16T00:07:35.114845+0000","last_undegraded":"2024-09-16T00:07:35.114845+0000","last_fullsized":"2024-09-16T00:07:35.114845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115156+0000","last_change":"2024-09-16T00:07:32.629539+0000","last_active":"2024-09-16T00:07:35.115156+0000","last_peered":"2024-09-16T00:07:35.115156+0000","last_clean":"2024-09-16T00:07:35.115156+0000","last_became_active":"2024-09-16T00:07:32.629186+0000","last_became_peered":"2024-09-16T00:07:32.629186+0000","last_unstale":"2024-09-16T00:07:35.115156+0000","last_undegraded":"2024-09-16T00:07:35.115156+0000","last_fullsized":"2024-09-16T00:07:35.115156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:59:32.860208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114535+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:07:35.114535+0000","last_peered":"2024-09-16T00:07:35.114535+0000","last_clean":"2024-09-16T00:07:35.114535+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:07:35.114535+0000","last_undegraded":"2024-09-16T00:07:35.114535+0000","last_fullsized":"2024-09-16T00:07:35.114535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614668+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:07:34.614668+0000","last_peered":"2024-09-16T00:07:34.614668+0000","last_clean":"2024-09-16T00:07:34.614668+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:07:34.614668+0000","last_undegraded":"2024-09-16T00:07:34.614668+0000","last_fullsized":"2024-09-16T00:07:34.614668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114527+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:07:35.114527+0000","last_peered":"2024-09-16T00:07:35.114527+0000","last_clean":"2024-09-16T00:07:35.114527+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:07:35.114527+0000","last_undegraded":"2024-09-16T00:07:35.114527+0000","last_fullsized":"2024-09-16T00:07:35.114527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115138+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:07:35.115138+0000","last_peered":"2024-09-16T00:07:35.115138+0000","last_clean":"2024-09-16T00:07:35.115138+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:07:35.115138+0000","last_undegraded":"2024-09-16T00:07:35.115138+0000","last_fullsized":"2024-09-16T00:07:35.115138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615408+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:07:34.615408+0000","last_peered":"2024-09-16T00:07:34.615408+0000","last_clean":"2024-09-16T00:07:34.615408+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:07:34.615408+0000","last_undegraded":"2024-09-16T00:07:34.615408+0000","last_fullsized":"2024-09-16T00:07:34.615408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615356+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:07:34.615356+0000","last_peered":"2024-09-16T00:07:34.615356+0000","last_clean":"2024-09-16T00:07:34.615356+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:07:34.615356+0000","last_undegraded":"2024-09-16T00:07:34.615356+0000","last_fullsized":"2024-09-16T00:07:34.615356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115277+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:07:35.115277+0000","last_peered":"2024-09-16T00:07:35.115277+0000","last_clean":"2024-09-16T00:07:35.115277+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:07:35.115277+0000","last_undegraded":"2024-09-16T00:07:35.115277+0000","last_fullsized":"2024-09-16T00:07:35.115277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.619745+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:07:38.619745+0000","last_peered":"2024-09-16T00:07:38.619745+0000","last_clean":"2024-09-16T00:07:38.619745+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:07:38.619745+0000","last_undegraded":"2024-09-16T00:07:38.619745+0000","last_fullsized":"2024-09-16T00:07:38.619745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614648+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:07:34.614648+0000","last_peered":"2024-09-16T00:07:34.614648+0000","last_clean":"2024-09-16T00:07:34.614648+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:07:34.614648+0000","last_undegraded":"2024-09-16T00:07:34.614648+0000","last_fullsized":"2024-09-16T00:07:34.614648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'7","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.626903+0000","last_change":"2024-09-16T00:07:32.629132+0000","last_active":"2024-09-16T00:07:38.626903+0000","last_peered":"2024-09-16T00:07:38.626903+0000","last_clean":"2024-09-16T00:07:38.626903+0000","last_became_active":"2024-09-16T00:07:32.628803+0000","last_became_peered":"2024-09-16T00:07:32.628803+0000","last_unstale":"2024-09-16T00:07:38.626903+0000","last_undegraded":"2024-09-16T00:07:38.626903+0000","last_fullsized":"2024-09-16T00:07:38.626903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2055,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115230+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:07:35.115230+0000","last_peered":"2024-09-16T00:07:35.115230+0000","last_clean":"2024-09-16T00:07:35.115230+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:07:35.115230+0000","last_undegraded":"2024-09-16T00:07:35.115230+0000","last_fullsized":"2024-09-16T00:07:35.115230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620127+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:07:38.620127+0000","last_peered":"2024-09-16T00:07:38.620127+0000","last_clean":"2024-09-16T00:07:38.620127+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:07:38.620127+0000","last_undegraded":"2024-09-16T00:07:38.620127+0000","last_fullsized":"2024-09-16T00:07:38.620127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115121+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:07:35.115121+0000","last_peered":"2024-09-16T00:07:35.115121+0000","last_clean":"2024-09-16T00:07:35.115121+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:07:35.115121+0000","last_undegraded":"2024-09-16T00:07:35.115121+0000","last_fullsized":"2024-09-16T00:07:35.115121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615462+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:07:34.615462+0000","last_peered":"2024-09-16T00:07:34.615462+0000","last_clean":"2024-09-16T00:07:34.615462+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:07:34.615462+0000","last_undegraded":"2024-09-16T00:07:34.615462+0000","last_fullsized":"2024-09-16T00:07:34.615462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115766+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:07:35.115766+0000","last_peered":"2024-09-16T00:07:35.115766+0000","last_clean":"2024-09-16T00:07:35.115766+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:07:35.115766+0000","last_undegraded":"2024-09-16T00:07:35.115766+0000","last_fullsized":"2024-09-16T00:07:35.115766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114802+0000","last_change":"2024-09-16T00:07:32.626856+0000","last_active":"2024-09-16T00:07:35.114802+0000","last_peered":"2024-09-16T00:07:35.114802+0000","last_clean":"2024-09-16T00:07:35.114802+0000","last_became_active":"2024-09-16T00:07:32.626532+0000","last_became_peered":"2024-09-16T00:07:32.626532+0000","last_unstale":"2024-09-16T00:07:35.114802+0000","last_undegraded":"2024-09-16T00:07:35.114802+0000","last_fullsized":"2024-09-16T00:07:35.114802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114886+0000","last_change":"2024-09-16T00:07:33.632870+0000","last_active":"2024-09-16T00:07:35.114886+0000","last_peered":"2024-09-16T00:07:35.114886+0000","last_clean":"2024-09-16T00:07:35.114886+0000","last_became_active":"2024-09-16T00:07:33.632608+0000","last_became_peered":"2024-09-16T00:07:33.632608+0000","last_unstale":"2024-09-16T00:07:35.114886+0000","last_undegraded":"2024-09-16T00:07:35.114886+0000","last_fullsized":"2024-09-16T00:07:35.114886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.621550+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:07:38.621550+0000","last_peered":"2024-09-16T00:07:38.621550+0000","last_clean":"2024-09-16T00:07:38.621550+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:07:38.621550+0000","last_undegraded":"2024-09-16T00:07:38.621550+0000","last_fullsized":"2024-09-16T00:07:38.621550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115409+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:07:35.115409+0000","last_peered":"2024-09-16T00:07:35.115409+0000","last_clean":"2024-09-16T00:07:35.115409+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:07:35.115409+0000","last_undegraded":"2024-09-16T00:07:35.115409+0000","last_fullsized":"2024-09-16T00:07:35.115409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614761+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:07:34.614761+0000","last_peered":"2024-09-16T00:07:34.614761+0000","last_clean":"2024-09-16T00:07:34.614761+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:07:34.614761+0000","last_undegraded":"2024-09-16T00:07:34.614761+0000","last_fullsized":"2024-09-16T00:07:34.614761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115480+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:07:35.115480+0000","last_peered":"2024-09-16T00:07:35.115480+0000","last_clean":"2024-09-16T00:07:35.115480+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:07:35.115480+0000","last_undegraded":"2024-09-16T00:07:35.115480+0000","last_fullsized":"2024-09-16T00:07:35.115480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620942+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:07:38.620942+0000","last_peered":"2024-09-16T00:07:38.620942+0000","last_clean":"2024-09-16T00:07:38.620942+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:07:38.620942+0000","last_undegraded":"2024-09-16T00:07:38.620942+0000","last_fullsized":"2024-09-16T00:07:38.620942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115888+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:07:35.115888+0000","last_peered":"2024-09-16T00:07:35.115888+0000","last_clean":"2024-09-16T00:07:35.115888+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:07:35.115888+0000","last_undegraded":"2024-09-16T00:07:35.115888+0000","last_fullsized":"2024-09-16T00:07:35.115888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.048699+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:07:36.048699+0000","last_peered":"2024-09-16T00:07:36.048699+0000","last_clean":"2024-09-16T00:07:36.048699+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:07:36.048699+0000","last_undegraded":"2024-09-16T00:07:36.048699+0000","last_fullsized":"2024-09-16T00:07:36.048699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615597+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:07:34.615597+0000","last_peered":"2024-09-16T00:07:34.615597+0000","last_clean":"2024-09-16T00:07:34.615597+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:07:34.615597+0000","last_undegraded":"2024-09-16T00:07:34.615597+0000","last_fullsized":"2024-09-16T00:07:34.615597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.613057+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:07:34.613057+0000","last_peered":"2024-09-16T00:07:34.613057+0000","last_clean":"2024-09-16T00:07:34.613057+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:07:34.613057+0000","last_undegraded":"2024-09-16T00:07:34.613057+0000","last_fullsized":"2024-09-16T00:07:34.613057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115577+0000","last_change":"2024-09-16T00:07:33.638312+0000","last_active":"2024-09-16T00:07:35.115577+0000","last_peered":"2024-09-16T00:07:35.115577+0000","last_clean":"2024-09-16T00:07:35.115577+0000","last_became_active":"2024-09-16T00:07:33.638049+0000","last_became_peered":"2024-09-16T00:07:33.638049+0000","last_unstale":"2024-09-16T00:07:35.115577+0000","last_undegraded":"2024-09-16T00:07:35.115577+0000","last_fullsized":"2024-09-16T00:07:35.115577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":170,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115169+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:07:35.115169+0000","last_peered":"2024-09-16T00:07:35.115169+0000","last_clean":"2024-09-16T00:07:35.115169+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:07:35.115169+0000","last_undegraded":"2024-09-16T00:07:35.115169+0000","last_fullsized":"2024-09-16T00:07:35.115169+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614834+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:07:34.614834+0000","last_peered":"2024-09-16T00:07:34.614834+0000","last_clean":"2024-09-16T00:07:34.614834+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:07:34.614834+0000","last_undegraded":"2024-09-16T00:07:34.614834+0000","last_fullsized":"2024-09-16T00:07:34.614834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114892+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:07:35.114892+0000","last_peered":"2024-09-16T00:07:35.114892+0000","last_clean":"2024-09-16T00:07:35.114892+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:07:35.114892+0000","last_undegraded":"2024-09-16T00:07:35.114892+0000","last_fullsized":"2024-09-16T00:07:35.114892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115065+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:07:35.115065+0000","last_peered":"2024-09-16T00:07:35.115065+0000","last_clean":"2024-09-16T00:07:35.115065+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:07:35.115065+0000","last_undegraded":"2024-09-16T00:07:35.115065+0000","last_fullsized":"2024-09-16T00:07:35.115065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115288+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:07:35.115288+0000","last_peered":"2024-09-16T00:07:35.115288+0000","last_clean":"2024-09-16T00:07:35.115288+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:07:35.115288+0000","last_undegraded":"2024-09-16T00:07:35.115288+0000","last_fullsized":"2024-09-16T00:07:35.115288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.612974+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:07:34.612974+0000","last_peered":"2024-09-16T00:07:34.612974+0000","last_clean":"2024-09-16T00:07:34.612974+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:07:34.612974+0000","last_undegraded":"2024-09-16T00:07:34.612974+0000","last_fullsized":"2024-09-16T00:07:34.612974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114905+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:07:35.114905+0000","last_peered":"2024-09-16T00:07:35.114905+0000","last_clean":"2024-09-16T00:07:35.114905+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:07:35.114905+0000","last_undegraded":"2024-09-16T00:07:35.114905+0000","last_fullsized":"2024-09-16T00:07:35.114905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615621+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:07:34.615621+0000","last_peered":"2024-09-16T00:07:34.615621+0000","last_clean":"2024-09-16T00:07:34.615621+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:07:34.615621+0000","last_undegraded":"2024-09-16T00:07:34.615621+0000","last_fullsized":"2024-09-16T00:07:34.615621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615008+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:07:34.615008+0000","last_peered":"2024-09-16T00:07:34.615008+0000","last_clean":"2024-09-16T00:07:34.615008+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:07:34.615008+0000","last_undegraded":"2024-09-16T00:07:34.615008+0000","last_fullsized":"2024-09-16T00:07:34.615008+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115398+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:07:35.115398+0000","last_peered":"2024-09-16T00:07:35.115398+0000","last_clean":"2024-09-16T00:07:35.115398+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:07:35.115398+0000","last_undegraded":"2024-09-16T00:07:35.115398+0000","last_fullsized":"2024-09-16T00:07:35.115398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615631+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:07:34.615631+0000","last_peered":"2024-09-16T00:07:34.615631+0000","last_clean":"2024-09-16T00:07:34.615631+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:07:34.615631+0000","last_undegraded":"2024-09-16T00:07:34.615631+0000","last_fullsized":"2024-09-16T00:07:34.615631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115814+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:07:35.115814+0000","last_peered":"2024-09-16T00:07:35.115814+0000","last_clean":"2024-09-16T00:07:35.115814+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:07:35.115814+0000","last_undegraded":"2024-09-16T00:07:35.115814+0000","last_fullsized":"2024-09-16T00:07:35.115814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115305+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:07:35.115305+0000","last_peered":"2024-09-16T00:07:35.115305+0000","last_clean":"2024-09-16T00:07:35.115305+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:07:35.115305+0000","last_undegraded":"2024-09-16T00:07:35.115305+0000","last_fullsized":"2024-09-16T00:07:35.115305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.048722+0000","last_change":"2024-09-16T00:07:32.627236+0000","last_active":"2024-09-16T00:07:36.048722+0000","last_peered":"2024-09-16T00:07:36.048722+0000","last_clean":"2024-09-16T00:07:36.048722+0000","last_became_active":"2024-09-16T00:07:32.626743+0000","last_became_peered":"2024-09-16T00:07:32.626743+0000","last_unstale":"2024-09-16T00:07:36.048722+0000","last_undegraded":"2024-09-16T00:07:36.048722+0000","last_fullsized":"2024-09-16T00:07:36.048722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114835+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:07:35.114835+0000","last_peered":"2024-09-16T00:07:35.114835+0000","last_clean":"2024-09-16T00:07:35.114835+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:07:35.114835+0000","last_undegraded":"2024-09-16T00:07:35.114835+0000","last_fullsized":"2024-09-16T00:07:35.114835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.620513+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:07:38.620513+0000","last_peered":"2024-09-16T00:07:38.620513+0000","last_clean":"2024-09-16T00:07:38.620513+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:07:38.620513+0000","last_undegraded":"2024-09-16T00:07:38.620513+0000","last_fullsized":"2024-09-16T00:07:38.620513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114804+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:07:35.114804+0000","last_peered":"2024-09-16T00:07:35.114804+0000","last_clean":"2024-09-16T00:07:35.114804+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:07:35.114804+0000","last_undegraded":"2024-09-16T00:07:35.114804+0000","last_fullsized":"2024-09-16T00:07:35.114804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114947+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:07:35.114947+0000","last_peered":"2024-09-16T00:07:35.114947+0000","last_clean":"2024-09-16T00:07:35.114947+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:07:35.114947+0000","last_undegraded":"2024-09-16T00:07:35.114947+0000","last_fullsized":"2024-09-16T00:07:35.114947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.614501+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:07:34.614501+0000","last_peered":"2024-09-16T00:07:34.614501+0000","last_clean":"2024-09-16T00:07:34.614501+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:07:34.614501+0000","last_undegraded":"2024-09-16T00:07:34.614501+0000","last_fullsized":"2024-09-16T00:07:34.614501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114765+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:07:35.114765+0000","last_peered":"2024-09-16T00:07:35.114765+0000","last_clean":"2024-09-16T00:07:35.114765+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:07:35.114765+0000","last_undegraded":"2024-09-16T00:07:35.114765+0000","last_fullsized":"2024-09-16T00:07:35.114765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114640+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:07:35.114640+0000","last_peered":"2024-09-16T00:07:35.114640+0000","last_clean":"2024-09-16T00:07:35.114640+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:07:35.114640+0000","last_undegraded":"2024-09-16T00:07:35.114640+0000","last_fullsized":"2024-09-16T00:07:35.114640+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.115333+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:07:35.115333+0000","last_peered":"2024-09-16T00:07:35.115333+0000","last_clean":"2024-09-16T00:07:35.115333+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:07:35.115333+0000","last_undegraded":"2024-09-16T00:07:35.115333+0000","last_fullsized":"2024-09-16T00:07:35.115333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114803+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:07:35.114803+0000","last_peered":"2024-09-16T00:07:35.114803+0000","last_clean":"2024-09-16T00:07:35.114803+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:07:35.114803+0000","last_undegraded":"2024-09-16T00:07:35.114803+0000","last_fullsized":"2024-09-16T00:07:35.114803+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.621871+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:07:38.621871+0000","last_peered":"2024-09-16T00:07:38.621871+0000","last_clean":"2024-09-16T00:07:38.621871+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:07:38.621871+0000","last_undegraded":"2024-09-16T00:07:38.621871+0000","last_fullsized":"2024-09-16T00:07:38.621871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114993+0000","last_change":"2024-09-16T00:07:33.637979+0000","last_active":"2024-09-16T00:07:35.114993+0000","last_peered":"2024-09-16T00:07:35.114993+0000","last_clean":"2024-09-16T00:07:35.114993+0000","last_became_active":"2024-09-16T00:07:33.637713+0000","last_became_peered":"2024-09-16T00:07:33.637713+0000","last_unstale":"2024-09-16T00:07:35.114993+0000","last_undegraded":"2024-09-16T00:07:35.114993+0000","last_fullsized":"2024-09-16T00:07:35.114993+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615387+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:07:34.615387+0000","last_peered":"2024-09-16T00:07:34.615387+0000","last_clean":"2024-09-16T00:07:34.615387+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:07:34.615387+0000","last_undegraded":"2024-09-16T00:07:34.615387+0000","last_fullsized":"2024-09-16T00:07:34.615387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.621868+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:07:38.621868+0000","last_peered":"2024-09-16T00:07:38.621868+0000","last_clean":"2024-09-16T00:07:38.621868+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:07:38.621868+0000","last_undegraded":"2024-09-16T00:07:38.621868+0000","last_fullsized":"2024-09-16T00:07:38.621868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:34.615266+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:07:34.615266+0000","last_peered":"2024-09-16T00:07:34.615266+0000","last_clean":"2024-09-16T00:07:34.615266+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:07:34.615266+0000","last_undegraded":"2024-09-16T00:07:34.615266+0000","last_fullsized":"2024-09-16T00:07:34.615266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.624540+0000","last_change":"2024-09-16T00:07:32.632481+0000","last_active":"2024-09-16T00:07:38.624540+0000","last_peered":"2024-09-16T00:07:38.624540+0000","last_clean":"2024-09-16T00:07:38.624540+0000","last_became_active":"2024-09-16T00:07:32.632152+0000","last_became_peered":"2024-09-16T00:07:32.632152+0000","last_unstale":"2024-09-16T00:07:38.624540+0000","last_undegraded":"2024-09-16T00:07:38.624540+0000","last_fullsized":"2024-09-16T00:07:38.624540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114783+0000","last_change":"2024-09-16T00:07:33.629777+0000","last_active":"2024-09-16T00:07:35.114783+0000","last_peered":"2024-09-16T00:07:35.114783+0000","last_clean":"2024-09-16T00:07:35.114783+0000","last_became_active":"2024-09-16T00:07:33.627917+0000","last_became_peered":"2024-09-16T00:07:33.627917+0000","last_unstale":"2024-09-16T00:07:35.114783+0000","last_undegraded":"2024-09-16T00:07:35.114783+0000","last_fullsized":"2024-09-16T00:07:35.114783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:38.845679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:36.059155+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:07:36.059155+0000","last_peered":"2024-09-16T00:07:36.059155+0000","last_clean":"2024-09-16T00:07:36.059155+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:07:36.059155+0000","last_undegraded":"2024-09-16T00:07:36.059155+0000","last_fullsized":"2024-09-16T00:07:36.059155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:35.114899+0000","last_change":"2024-09-16T00:07:33.630746+0000","last_active":"2024-09-16T00:07:35.114899+0000","last_peered":"2024-09-16T00:07:35.114899+0000","last_clean":"2024-09-16T00:07:35.114899+0000","last_became_active":"2024-09-16T00:07:33.630480+0000","last_became_peered":"2024-09-16T00:07:33.630480+0000","last_unstale":"2024-09-16T00:07:35.114899+0000","last_undegraded":"2024-09-16T00:07:35.114899+0000","last_fullsized":"2024-09-16T00:07:35.114899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:55:50.931998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-16T00:07:38.622293+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:07:38.622293+0000","last_peered":"2024-09-16T00:07:38.622293+0000","last_clean":"2024-09-16T00:07:38.622293+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:07:38.622293+0000","last_undegraded":"2024-09-16T00:07:38.622293+0000","last_fullsized":"2024-09-16T00:07:38.622293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2853,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":5706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":668946,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":80051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":78086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":78122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":78156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":299008,"data_stored":669002,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":77996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:07:53.873 INFO:tasks.thrashosds.thrasher:Setting 3.2e to [6, 4, 3, 0] 2024-09-16T00:07:53.873 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.2e', '6', '4', '3', '0'] 2024-09-16T00:07:53.873 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.2e 6 4 3 0 2024-09-16T00:07:53.911 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:07:53.911+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:07:53.918 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:53.918 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:53.928 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:07:54.013 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:07:54.011+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:07:54.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.016+0000 7f185db7f640 1 Processor -- start 2024-09-16T00:07:54.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.016+0000 7f185db7f640 1 -- start start 2024-09-16T00:07:54.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f185db7f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f18580c8cc0 0x7f18580c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:54.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f185db7f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18580ced90 con 0x7f1858154cd0 2024-09-16T00:07:54.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f185db7f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18580ce7d0 con 0x7f185815b540 2024-09-16T00:07:54.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f185db7f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f18581550b0 con 0x7f18580c8cc0 2024-09-16T00:07:54.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18577fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f1858154cd0 0x7f18580c9530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57868/0 (socket says 172.21.15.139:57868) 2024-09-16T00:07:54.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18577fe640 1 -- 172.21.15.139:0/3288238959 learned_addr learned my addr 172.21.15.139:0/3288238959 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:54.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f1856ffd640 1 --2- 172.21.15.139:0/3288238959 >> v2:172.21.15.164:3300/0 conn(0x7f18580c8cc0 0x7f18580c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:54.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18567fc640 1 -- 172.21.15.139:0/3288238959 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 827532749 0 0) 0x7f18580ced90 con 0x7f1858154cd0 2024-09-16T00:07:54.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18567fc640 1 -- 172.21.15.139:0/3288238959 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1840003520 con 0x7f1858154cd0 2024-09-16T00:07:54.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18567fc640 1 -- 172.21.15.139:0/3288238959 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4128848837 0 0) 0x7f1840003520 con 0x7f1858154cd0 2024-09-16T00:07:54.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18567fc640 1 -- 172.21.15.139:0/3288238959 >> v2:172.21.15.164:3300/0 conn(0x7f18580c8cc0 msgr2=0x7f18580c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:54.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18567fc640 1 --2- 172.21.15.139:0/3288238959 >> v2:172.21.15.164:3300/0 conn(0x7f18580c8cc0 0x7f18580c39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:54.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18567fc640 1 -- 172.21.15.139:0/3288238959 >> v1:172.21.15.164:6789/0 conn(0x7f185815b540 legacy=0x7f18581651c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:54.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18567fc640 1 -- 172.21.15.139:0/3288238959 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18580ceab0 con 0x7f1858154cd0 2024-09-16T00:07:54.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18567fc640 1 -- 172.21.15.139:0/3288238959 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7f1848001670 con 0x7f1858154cd0 2024-09-16T00:07:54.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f1856ffd640 1 --2- 172.21.15.139:0/3288238959 >> v2:172.21.15.164:3300/0 conn(0x7f18580c8cc0 0x7f18580c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:54.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18567fc640 1 -- 172.21.15.139:0/3288238959 >> v1:172.21.15.139:6789/0 conn(0x7f1858154cd0 legacy=0x7f18580c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:54.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18567fc640 1 --2- 172.21.15.139:0/3288238959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18400039b0 0x7f1840003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:54.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18567fc640 1 --2- 172.21.15.139:0/3288238959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1840004800 0x7f1840004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:54.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18567fc640 1 --2- 172.21.15.139:0/3288238959 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f18400051c0 0x7f184000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:54.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18567fc640 1 -- 172.21.15.139:0/3288238959 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f18581550b0 con 0x7f18400039b0 2024-09-16T00:07:54.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18567fc640 1 -- 172.21.15.139:0/3288238959 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f184000c070 con 0x7f1840004800 2024-09-16T00:07:54.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18567fc640 1 -- 172.21.15.139:0/3288238959 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f184000c270 con 0x7f18400051c0 2024-09-16T00:07:54.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.017+0000 7f18577fe640 1 --2- 172.21.15.139:0/3288238959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18400039b0 0x7f1840003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:54.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.018+0000 7f1856ffd640 1 --2- 172.21.15.139:0/3288238959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1840004800 0x7f1840004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:54.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.018+0000 7f1857fff640 1 --2- 172.21.15.139:0/3288238959 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f18400051c0 0x7f184000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:54.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.018+0000 7f18577fe640 1 -- 172.21.15.139:0/3288238959 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f18400051c0 msgr2=0x7f184000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:54.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.018+0000 7f18577fe640 1 --2- 172.21.15.139:0/3288238959 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f18400051c0 0x7f184000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:54.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.018+0000 7f18577fe640 1 -- 172.21.15.139:0/3288238959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1840004800 msgr2=0x7f1840004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:54.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.018+0000 7f18577fe640 1 --2- 172.21.15.139:0/3288238959 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1840004800 0x7f1840004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:54.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.018+0000 7f18577fe640 1 -- 172.21.15.139:0/3288238959 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1840003520 con 0x7f18400039b0 2024-09-16T00:07:54.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.018+0000 7f1857fff640 1 -- 172.21.15.139:0/3288238959 reap_dead start 2024-09-16T00:07:54.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.019+0000 7f18577fe640 1 --2- 172.21.15.139:0/3288238959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18400039b0 0x7f1840003dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f184800c240 tx=0x7f1840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668c706c8221b64c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:54.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.019+0000 7f18567fc640 1 -- 172.21.15.139:0/3288238959 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848007d10 con 0x7f18400039b0 2024-09-16T00:07:54.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.019+0000 7f18567fc640 1 -- 172.21.15.139:0/3288238959 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18480058d0 con 0x7f18400039b0 2024-09-16T00:07:54.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.019+0000 7f18567fc640 1 -- 172.21.15.139:0/3288238959 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848005d20 con 0x7f18400039b0 2024-09-16T00:07:54.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.019+0000 7f185db7f640 1 -- 172.21.15.139:0/3288238959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18400039b0 msgr2=0x7f1840003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:54.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.019+0000 7f185db7f640 1 --2- 172.21.15.139:0/3288238959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18400039b0 0x7f1840003dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f184800c240 tx=0x7f1840005830 comp rx=0 tx=0).stop 2024-09-16T00:07:54.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.019+0000 7f185db7f640 1 -- 172.21.15.139:0/3288238959 shutdown_connections 2024-09-16T00:07:54.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.019+0000 7f185db7f640 1 --2- 172.21.15.139:0/3288238959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18400039b0 0x7f1840003dd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:54.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.019+0000 7f185db7f640 1 -- 172.21.15.139:0/3288238959 >> 172.21.15.139:0/3288238959 conn(0x7f1858067950 msgr2=0x7f1858067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:54.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.019+0000 7f185db7f640 1 -- 172.21.15.139:0/3288238959 shutdown_connections 2024-09-16T00:07:54.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.019+0000 7f185db7f640 1 -- 172.21.15.139:0/3288238959 wait complete. 2024-09-16T00:07:54.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.021+0000 7f185db7f640 1 Processor -- start 2024-09-16T00:07:54.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.021+0000 7f185db7f640 1 -- start start 2024-09-16T00:07:54.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.021+0000 7f185db7f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f18400039b0 0x7f18581f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:54.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.021+0000 7f185db7f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18580cda20 0x7f18580af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:54.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.021+0000 7f185db7f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18580cde60 0x7f18580afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:54.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.021+0000 7f185db7f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f18580ce390 con 0x7f18580cde60 2024-09-16T00:07:54.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.021+0000 7f185db7f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f185815ad40 con 0x7f18580cda20 2024-09-16T00:07:54.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.021+0000 7f185db7f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f18580ce510 con 0x7f18400039b0 2024-09-16T00:07:54.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.022+0000 7f1857fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18580cde60 0x7f18580afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:54.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.022+0000 7f1857fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18580cde60 0x7f18580afde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47862/0 (socket says 172.21.15.139:47862) 2024-09-16T00:07:54.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.022+0000 7f1857fff640 1 -- 172.21.15.139:0/3822989906 learned_addr learned my addr 172.21.15.139:0/3822989906 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:54.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.022+0000 7f18577fe640 1 --2- 172.21.15.139:0/3822989906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f18400039b0 0x7f18581f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:54.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.022+0000 7f1856ffd640 1 --2- 172.21.15.139:0/3822989906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18580cda20 0x7f18580af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:54.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.022+0000 7f1857fff640 1 -- 172.21.15.139:0/3822989906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f18400039b0 msgr2=0x7f18581f11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:54.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.022+0000 7f1857fff640 1 --2- 172.21.15.139:0/3822989906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f18400039b0 0x7f18581f11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:54.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.022+0000 7f1857fff640 1 -- 172.21.15.139:0/3822989906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18580cda20 msgr2=0x7f18580af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:54.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.022+0000 7f1857fff640 1 --2- 172.21.15.139:0/3822989906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18580cda20 0x7f18580af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:54.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.022+0000 7f1857fff640 1 -- 172.21.15.139:0/3822989906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18580cc260 con 0x7f18580cde60 2024-09-16T00:07:54.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.022+0000 7f1857fff640 1 --2- 172.21.15.139:0/3822989906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18580cde60 0x7f18580afde0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f18580c92c0 tx=0x7f185000fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:54.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.022+0000 7f183ffff640 1 -- 172.21.15.139:0/3822989906 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850003b80 con 0x7f18580cde60 2024-09-16T00:07:54.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.022+0000 7f183ffff640 1 -- 172.21.15.139:0/3822989906 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1850018070 con 0x7f18580cde60 2024-09-16T00:07:54.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.022+0000 7f185db7f640 1 -- 172.21.15.139:0/3822989906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18580cc540 con 0x7f18580cde60 2024-09-16T00:07:54.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.022+0000 7f183ffff640 1 -- 172.21.15.139:0/3822989906 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18500050d0 con 0x7f18580cde60 2024-09-16T00:07:54.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.022+0000 7f185db7f640 1 -- 172.21.15.139:0/3822989906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18580ccb00 con 0x7f18580cde60 2024-09-16T00:07:54.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.024+0000 7f185db7f640 1 -- 172.21.15.139:0/3822989906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18580c39c0 con 0x7f18580cde60 2024-09-16T00:07:54.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.024+0000 7f183ffff640 1 -- 172.21.15.139:0/3822989906 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1850005270 con 0x7f18580cde60 2024-09-16T00:07:54.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.024+0000 7f183ffff640 1 --2- 172.21.15.139:0/3822989906 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f184406c2b0 0x7f184406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:54.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.024+0000 7f18577fe640 1 --2- 172.21.15.139:0/3822989906 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f184406c2b0 0x7f184406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:54.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.024+0000 7f183ffff640 1 -- 172.21.15.139:0/3822989906 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f185008b140 con 0x7f18580cde60 2024-09-16T00:07:54.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.025+0000 7f18577fe640 1 --2- 172.21.15.139:0/3822989906 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f184406c2b0 0x7f184406e770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1848033000 tx=0x7f184802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:54.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.028+0000 7f183ffff640 1 -- 172.21.15.139:0/3822989906 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1850059260 con 0x7f18580cde60 2024-09-16T00:07:54.094 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:54.094 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:54.103 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:07:54.113 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:07:54.113+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:07:54.215 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:07:54.212+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:07:54.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.240+0000 7f185db7f640 1 -- 172.21.15.139:0/3822989906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pg-upmap-items", "pgid": "3.2e", "id": [6, 4, 3, 0]} v 0) -- 0x7f185806f920 con 0x7f18580cde60 2024-09-16T00:07:54.250 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:54.250 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:54.259 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:07:54.344 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:07:54.342+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:07:54.404 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:54.405 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:54.413 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:07:54.416 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:07:54.416+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:07:54.517 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:07:54.515+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:07:54.557 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:54.557 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:54.566 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:07:54.618 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:07:54.618+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:07:54.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.681+0000 7f183ffff640 1 -- 172.21.15.139:0/3822989906 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap-items", "pgid": "3.2e", "id": [6, 4, 3, 0]}]=0 set 3.2e pg_upmap_items mapping to [6->4,3->0] v24) ==== 148+0+0 (secure 0 0 0) 0x7f185005d560 con 0x7f18580cde60 2024-09-16T00:07:54.684 INFO:teuthology.orchestra.run.smithi139.stderr:set 3.2e pg_upmap_items mapping to [6->4,3->0] 2024-09-16T00:07:54.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.684+0000 7f185db7f640 1 -- 172.21.15.139:0/3822989906 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f184406c2b0 msgr2=0x7f184406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:54.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.684+0000 7f185db7f640 1 --2- 172.21.15.139:0/3822989906 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f184406c2b0 0x7f184406e770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1848033000 tx=0x7f184802e000 comp rx=0 tx=0).stop 2024-09-16T00:07:54.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.684+0000 7f185db7f640 1 -- 172.21.15.139:0/3822989906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18580cde60 msgr2=0x7f18580afde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:54.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.684+0000 7f185db7f640 1 --2- 172.21.15.139:0/3822989906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18580cde60 0x7f18580afde0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f18580c92c0 tx=0x7f185000fd40 comp rx=0 tx=0).stop 2024-09-16T00:07:54.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.684+0000 7f185db7f640 1 -- 172.21.15.139:0/3822989906 shutdown_connections 2024-09-16T00:07:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.684+0000 7f185db7f640 1 --2- 172.21.15.139:0/3822989906 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f184406c2b0 0x7f184406e770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.684+0000 7f185db7f640 1 --2- 172.21.15.139:0/3822989906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18580cde60 0x7f18580afde0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.684+0000 7f185db7f640 1 --2- 172.21.15.139:0/3822989906 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f18580cda20 0x7f18580af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.684+0000 7f185db7f640 1 --2- 172.21.15.139:0/3822989906 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f18400039b0 0x7f18581f11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:54.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.684+0000 7f185db7f640 1 -- 172.21.15.139:0/3822989906 >> 172.21.15.139:0/3822989906 conn(0x7f1858067950 msgr2=0x7f18580cb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:54.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.684+0000 7f185db7f640 1 -- 172.21.15.139:0/3822989906 shutdown_connections 2024-09-16T00:07:54.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:54.684+0000 7f185db7f640 1 -- 172.21.15.139:0/3822989906 wait complete. 2024-09-16T00:07:54.714 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:54.714 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:54.718 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:07:54.717+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:07:54.723 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:07:54.820 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:54.819+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:07:54.866 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:54.866 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:54.874 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:07:54.920 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:07:54.920+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:07:54.989 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:54.989 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:54.998 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:07:55.022 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:07:55.021+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:07:55.123 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:55.122+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:07:55.145 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:55.145 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:55.154 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:07:55.223 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:07:55.222+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:07:55.298 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:55.298 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:55.306 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:07:55.349 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:55.349+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:07:55.361 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:07:55.410 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:55.410 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:55.419 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:07:55.461 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:07:55.460+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:07:55.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.459+0000 7f5bdce27640 1 Processor -- start 2024-09-16T00:07:55.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.459+0000 7f5bdce27640 1 -- start start 2024-09-16T00:07:55.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.460+0000 7f5bdce27640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5bd80c8070 0x7f5bd815c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.460+0000 7f5bdce27640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd8071830 con 0x7f5bd80c72a0 2024-09-16T00:07:55.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.460+0000 7f5bdce27640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd8071b10 con 0x7f5bd80c9220 2024-09-16T00:07:55.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.460+0000 7f5bdce27640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5bd815cc40 con 0x7f5bd80c8070 2024-09-16T00:07:55.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.460+0000 7f5bd5d9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5bd80c72a0 0x7f5bd80c5500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:57870/0 (socket says 172.21.15.139:57870) 2024-09-16T00:07:55.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.460+0000 7f5bd5d9b640 1 -- 172.21.15.139:0/3038752656 learned_addr learned my addr 172.21.15.139:0/3038752656 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:55.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.460+0000 7f5bd659c640 1 --2- 172.21.15.139:0/3038752656 >> v2:172.21.15.164:3300/0 conn(0x7f5bd80c8070 0x7f5bd815c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.461+0000 7f5bd559a640 1 -- 172.21.15.139:0/3038752656 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1347371805 0 0) 0x7f5bd8071830 con 0x7f5bd80c72a0 2024-09-16T00:07:55.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.461+0000 7f5bd559a640 1 -- 172.21.15.139:0/3038752656 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bc0003520 con 0x7f5bd80c72a0 2024-09-16T00:07:55.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.461+0000 7f5bd659c640 1 -- 172.21.15.139:0/3038752656 >> v1:172.21.15.164:6789/0 conn(0x7f5bd80c9220 legacy=0x7f5bd8163040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.461+0000 7f5bd659c640 1 -- 172.21.15.139:0/3038752656 >> v1:172.21.15.139:6789/0 conn(0x7f5bd80c72a0 legacy=0x7f5bd80c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.461+0000 7f5bd659c640 1 -- 172.21.15.139:0/3038752656 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd8071df0 con 0x7f5bd80c8070 2024-09-16T00:07:55.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.461+0000 7f5bd659c640 1 --2- 172.21.15.139:0/3038752656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd80c8070 0x7f5bd815c700 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009d90 tx=0x7f5bc802a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73c3c496a4352d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:55.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.462+0000 7f5bd559a640 1 -- 172.21.15.139:0/3038752656 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8037070 con 0x7f5bd80c8070 2024-09-16T00:07:55.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.462+0000 7f5bd559a640 1 -- 172.21.15.139:0/3038752656 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc8007740 con 0x7f5bd80c8070 2024-09-16T00:07:55.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.462+0000 7f5bd559a640 1 -- 172.21.15.139:0/3038752656 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8007af0 con 0x7f5bd80c8070 2024-09-16T00:07:55.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.462+0000 7f5bdce27640 1 -- 172.21.15.139:0/3038752656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd80c8070 msgr2=0x7f5bd815c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.462+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3038752656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd80c8070 0x7f5bd815c700 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009d90 tx=0x7f5bc802a780 comp rx=0 tx=0).stop 2024-09-16T00:07:55.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.462+0000 7f5bdce27640 1 -- 172.21.15.139:0/3038752656 shutdown_connections 2024-09-16T00:07:55.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.462+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3038752656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd80c8070 0x7f5bd815c700 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.462+0000 7f5bdce27640 1 -- 172.21.15.139:0/3038752656 >> 172.21.15.139:0/3038752656 conn(0x7f5bd806fb00 msgr2=0x7f5bd80738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:55.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.462+0000 7f5bdce27640 1 -- 172.21.15.139:0/3038752656 shutdown_connections 2024-09-16T00:07:55.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.462+0000 7f5bdce27640 1 -- 172.21.15.139:0/3038752656 wait complete. 2024-09-16T00:07:55.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.464+0000 7f5bdce27640 1 Processor -- start 2024-09-16T00:07:55.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.464+0000 7f5bdce27640 1 -- start start 2024-09-16T00:07:55.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.464+0000 7f5bdce27640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd80c72a0 0x7f5bd8158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.464+0000 7f5bdce27640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5bd80c8070 0x7f5bd81f12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.464+0000 7f5bdce27640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd80c9220 0x7f5bd81f1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.464+0000 7f5bdce27640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd815c1e0 con 0x7f5bd80c9220 2024-09-16T00:07:55.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.464+0000 7f5bdce27640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd815bcb0 con 0x7f5bd80c72a0 2024-09-16T00:07:55.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.464+0000 7f5bdce27640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd80c8450 con 0x7f5bd80c8070 2024-09-16T00:07:55.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.464+0000 7f5bd6d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd80c9220 0x7f5bd81f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.464+0000 7f5bd659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd80c72a0 0x7f5bd8158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.464+0000 7f5bd6d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd80c9220 0x7f5bd81f1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47870/0 (socket says 172.21.15.139:47870) 2024-09-16T00:07:55.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.464+0000 7f5bd659c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd80c72a0 0x7f5bd8158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:46316/0 (socket says 172.21.15.139:46316) 2024-09-16T00:07:55.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.464+0000 7f5bd6d9d640 1 -- 172.21.15.139:0/3544373051 learned_addr learned my addr 172.21.15.139:0/3544373051 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:55.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.464+0000 7f5bd5d9b640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5bd80c8070 0x7f5bd81f12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.465+0000 7f5bd6d9d640 1 -- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5bd80c8070 msgr2=0x7f5bd81f12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.465+0000 7f5bd6d9d640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5bd80c8070 0x7f5bd81f12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.465+0000 7f5bd6d9d640 1 -- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd80c72a0 msgr2=0x7f5bd8158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.465+0000 7f5bd6d9d640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd80c72a0 0x7f5bd8158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.465+0000 7f5bd6d9d640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd80c5ee0 con 0x7f5bd80c9220 2024-09-16T00:07:55.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.465+0000 7f5bd6d9d640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd80c9220 0x7f5bd81f1820 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c400 tx=0x7f5bd000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:55.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.465+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0017070 con 0x7f5bd80c9220 2024-09-16T00:07:55.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.465+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd00052d0 con 0x7f5bd80c9220 2024-09-16T00:07:55.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.465+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd81eeb30 con 0x7f5bd80c9220 2024-09-16T00:07:55.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.465+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0003810 con 0x7f5bd80c9220 2024-09-16T00:07:55.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.465+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd814c520 con 0x7f5bd80c9220 2024-09-16T00:07:55.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.466+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f5bd80cedc0 con 0x7f5bd80c9220 2024-09-16T00:07:55.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.467+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5bd001a070 con 0x7f5bd80c9220 2024-09-16T00:07:55.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.467+0000 7f5bc6ffd640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5bb806c2b0 0x7f5bb806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.467+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5738+0+0 (secure 0 0 0) 0x7f5bd008a790 con 0x7f5bd80c9220 2024-09-16T00:07:55.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.467+0000 7f5bd659c640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5bb806c2b0 0x7f5bb806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.467+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v24) ==== 54+0+16 (secure 0 0 0) 0x7f5bd008ab70 con 0x7f5bd80c9220 2024-09-16T00:07:55.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.467+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5bd8155d30 0x7f5bd81f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.467+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5bd814bff0 con 0x7f5bd8155d30 2024-09-16T00:07:55.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.467+0000 7f5bd659c640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5bb806c2b0 0x7f5bb806e770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8034000 tx=0x7f5bc802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:55.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.468+0000 7f5bd5d9b640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5bd8155d30 0x7f5bd81f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.468+0000 7f5bd5d9b640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5bd8155d30 0x7f5bd81f1d60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:55.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.468+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd814bff0 con 0x7f5bd8155d30 2024-09-16T00:07:55.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.482+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5bd814c760 con 0x7f5bd8155d30 2024-09-16T00:07:55.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.484+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5bd814c760 con 0x7f5bd8155d30 2024-09-16T00:07:55.486 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:07:55.487 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:07:55.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.484+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5bd8062630 0x7f5bd8062a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.484+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f5bd8159a40 con 0x7f5bd8062630 2024-09-16T00:07:55.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.484+0000 7f5bd659c640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5bd8062630 0x7f5bd8062a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.484+0000 7f5bd659c640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5bd8062630 0x7f5bd8062a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:55.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.485+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd8159a40 con 0x7f5bd8062630 2024-09-16T00:07:55.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.499+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5bd8156320 con 0x7f5bd8062630 2024-09-16T00:07:55.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.500+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5bd8156320 con 0x7f5bd8062630 2024-09-16T00:07:55.502 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:07:55.503 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:07:55.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.500+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5bd8063370 0x7f5bd815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.500+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f5bd815af70 con 0x7f5bd8063370 2024-09-16T00:07:55.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.501+0000 7f5bd5d9b640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5bd8063370 0x7f5bd815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.501+0000 7f5bd5d9b640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5bd8063370 0x7f5bd815aa30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:55.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.501+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd815af70 con 0x7f5bd8063370 2024-09-16T00:07:55.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.514+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5bd815b400 con 0x7f5bd8063370 2024-09-16T00:07:55.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.516+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5bd815b400 con 0x7f5bd8063370 2024-09-16T00:07:55.524 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:07:55.526 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:07:55.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.516+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f5bd802cf10 0x7f5bd802d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.516+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f5bd814eed0 con 0x7f5bd802cf10 2024-09-16T00:07:55.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.516+0000 7f5bd6d9d640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f5bd802cf10 0x7f5bd802d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.517+0000 7f5bd6d9d640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f5bd802cf10 0x7f5bd802d310 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:55.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.517+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd814eed0 con 0x7f5bd802cf10 2024-09-16T00:07:55.528 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:07:55.527+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:07:55.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.530+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5bd802db20 con 0x7f5bd802cf10 2024-09-16T00:07:55.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.532+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5bd802db20 con 0x7f5bd802cf10 2024-09-16T00:07:55.534 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:07:55.535 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:07:55.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.532+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5bd814dcd0 0x7f5bd814e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.532+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f5bd814e610 con 0x7f5bd814dcd0 2024-09-16T00:07:55.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.532+0000 7f5bd659c640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5bd814dcd0 0x7f5bd814e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.533+0000 7f5bd659c640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5bd814dcd0 0x7f5bd814e0d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:55.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.533+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd814e610 con 0x7f5bd814dcd0 2024-09-16T00:07:55.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.546+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5bd814e610 con 0x7f5bd814dcd0 2024-09-16T00:07:55.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.548+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5bd814e610 con 0x7f5bd814dcd0 2024-09-16T00:07:55.550 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:07:55.551 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:07:55.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.548+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5bd814ec90 0x7f5bd82046b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.548+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f5bd8207480 con 0x7f5bd814ec90 2024-09-16T00:07:55.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.549+0000 7f5bd5d9b640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5bd814ec90 0x7f5bd82046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.549+0000 7f5bd5d9b640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5bd814ec90 0x7f5bd82046b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:55.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.550+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd8207480 con 0x7f5bd814ec90 2024-09-16T00:07:55.564 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:55.564 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:55.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.562+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5bd8204bf0 con 0x7f5bd814ec90 2024-09-16T00:07:55.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.564+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5bd8204bf0 con 0x7f5bd814ec90 2024-09-16T00:07:55.566 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:07:55.567 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:07:55.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.564+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5bd8205270 0x7f5bd8207660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.564+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f5bd820a590 con 0x7f5bd8205270 2024-09-16T00:07:55.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.565+0000 7f5bd6d9d640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5bd8205270 0x7f5bd8207660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.565+0000 7f5bd6d9d640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5bd8205270 0x7f5bd8207660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:55.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.566+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd820a590 con 0x7f5bd8205270 2024-09-16T00:07:55.573 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:07:55.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.579+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5bd82092e0 con 0x7f5bd8205270 2024-09-16T00:07:55.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.581+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5bd82092e0 con 0x7f5bd8205270 2024-09-16T00:07:55.583 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:07:55.584 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:07:55.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.581+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5bd8209a00 0x7f5bd8209e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.581+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f5bd820d960 con 0x7f5bd8209a00 2024-09-16T00:07:55.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.582+0000 7f5bd659c640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5bd8209a00 0x7f5bd8209e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.582+0000 7f5bd659c640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5bd8209a00 0x7f5bd8209e60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:55.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.583+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd820d960 con 0x7f5bd8209a00 2024-09-16T00:07:55.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.596+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5bd820c7e0 con 0x7f5bd8209a00 2024-09-16T00:07:55.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bc6ffd640 1 -- 172.21.15.139:0/3544373051 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5bd820c7e0 con 0x7f5bd8209a00 2024-09-16T00:07:55.600 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:07:55.601 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:07:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5bd8155d30 msgr2=0x7f5bd81f1d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5bd8155d30 0x7f5bd81f1d60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5bd8062630 msgr2=0x7f5bd8062a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5bd8062630 0x7f5bd8062a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5bd8063370 msgr2=0x7f5bd815aa30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5bd8063370 0x7f5bd815aa30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f5bd802cf10 msgr2=0x7f5bd802d310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f5bd802cf10 0x7f5bd802d310 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5bd814dcd0 msgr2=0x7f5bd814e0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bd6d9d640 1 -- 172.21.15.139:0/3544373051 reap_dead start 2024-09-16T00:07:55.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5bd814dcd0 0x7f5bd814e0d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5bd814ec90 msgr2=0x7f5bd82046b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bd6d9d640 1 -- 172.21.15.139:0/3544373051 reap_dead start 2024-09-16T00:07:55.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5bd814ec90 0x7f5bd82046b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5bd8205270 msgr2=0x7f5bd8207660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5bd8205270 0x7f5bd8207660 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5bd8209a00 msgr2=0x7f5bd8209e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5bd8209a00 0x7f5bd8209e60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5bb806c2b0 msgr2=0x7f5bb806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5bb806c2b0 0x7f5bb806e770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8034000 tx=0x7f5bc802e000 comp rx=0 tx=0).stop 2024-09-16T00:07:55.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd80c9220 msgr2=0x7f5bd81f1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 --2- 172.21.15.139:0/3544373051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd80c9220 0x7f5bd81f1820 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c400 tx=0x7f5bd000c8c0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bd6d9d640 1 -- 172.21.15.139:0/3544373051 reap_dead start 2024-09-16T00:07:55.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 shutdown_connections 2024-09-16T00:07:55.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 >> 172.21.15.139:0/3544373051 conn(0x7f5bd806fb00 msgr2=0x7f5bd8151700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:55.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 shutdown_connections 2024-09-16T00:07:55.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.598+0000 7f5bdce27640 1 -- 172.21.15.139:0/3544373051 wait complete. 2024-09-16T00:07:55.630 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:07:55.627+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:07:55.698 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T00:07:55.727 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:07:55.728 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:07:55.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:07:55.728 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:07:55.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.728 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:14 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:07:55.728 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:07:55.728 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.109202366000002, 2024-09-16T00:07:55.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029190470000000001, 2024-09-16T00:07:55.728 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:55.728 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:55.729 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:55.729 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:55.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:55.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:07:55.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:55.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:55.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:07:55.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:55.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:55.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615982+0000", 2024-09-16T00:07:55.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7439999999999999e-06 2024-09-16T00:07:55.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:55.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:07:55.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3743999999999999e-05 2024-09-16T00:07:55.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:55.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:07:55.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2255e-05 2024-09-16T00:07:55.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:55.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616029+0000", 2024-09-16T00:07:55.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1175999999999999e-05 2024-09-16T00:07:55.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:55.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616098+0000", 2024-09-16T00:07:55.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9153999999999996e-05 2024-09-16T00:07:55.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616132+0000", 2024-09-16T00:07:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3924000000000002e-05 2024-09-16T00:07:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:07:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027226189999999999 2024-09-16T00:07:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:55.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618899+0000", 2024-09-16T00:07:55.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4431000000000002e-05 2024-09-16T00:07:55.733 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:55.733 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:55.733 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:55.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.733 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:15 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:07:55.733 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:07:55.733 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.106975546000001, 2024-09-16T00:07:55.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095618099999999996, 2024-09-16T00:07:55.733 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:55.734 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:55.734 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:55.734 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:55.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:55.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:07:55.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:55.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:55.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:07:55.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:55.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:55.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618208+0000", 2024-09-16T00:07:55.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1850000000000001e-06 2024-09-16T00:07:55.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:55.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618217+0000", 2024-09-16T00:07:55.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439999999999992e-06 2024-09-16T00:07:55.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:55.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618230+0000", 2024-09-16T00:07:55.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2435999999999999e-05 2024-09-16T00:07:55.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:55.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618238+0000", 2024-09-16T00:07:55.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1319999999999994e-06 2024-09-16T00:07:55.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:55.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618331+0000", 2024-09-16T00:07:55.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3453000000000001e-05 2024-09-16T00:07:55.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:55.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618352+0000", 2024-09-16T00:07:55.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0693999999999999e-05 2024-09-16T00:07:55.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:55.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619112+0000", 2024-09-16T00:07:55.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00076007099999999997 2024-09-16T00:07:55.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:55.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619163+0000", 2024-09-16T00:07:55.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1066000000000002e-05 2024-09-16T00:07:55.737 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:55.737 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:55.737 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:55.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.738 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:30 2.15 e22/19 2:aa448500:::500.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:07:55.738 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:07:55.738 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.106918060999998, 2024-09-16T00:07:55.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058031250000000001, 2024-09-16T00:07:55.738 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:55.738 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:55.738 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:55.738 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:55.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:55.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:07:55.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:55.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:55.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:07:55.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:55.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:55.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618265+0000", 2024-09-16T00:07:55.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9200000000000001e-07 2024-09-16T00:07:55.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:55.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618271+0000", 2024-09-16T00:07:55.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9580000000000004e-06 2024-09-16T00:07:55.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:55.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618279+0000", 2024-09-16T00:07:55.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2239999999999995e-06 2024-09-16T00:07:55.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:55.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618285+0000", 2024-09-16T00:07:55.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-06 2024-09-16T00:07:55.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:55.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618567+0000", 2024-09-16T00:07:55.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028204400000000001 2024-09-16T00:07:55.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:55.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:07:55.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9678999999999999e-05 2024-09-16T00:07:55.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:55.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624024+0000", 2024-09-16T00:07:55.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054376720000000002 2024-09-16T00:07:55.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:55.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624068+0000", 2024-09-16T00:07:55.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:07:55.742 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:55.743 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:55.743 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:55.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.743 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:20 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:07:55.743 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:07:55.743 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.106561697, 2024-09-16T00:07:55.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028808739999999998, 2024-09-16T00:07:55.743 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:55.743 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:55.743 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:55.743 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:55.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:55.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:07:55.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:55.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:55.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:07:55.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:55.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:55.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618622+0000", 2024-09-16T00:07:55.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1260000000000001e-06 2024-09-16T00:07:55.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618631+0000", 2024-09-16T00:07:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4700000000000008e-06 2024-09-16T00:07:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618645+0000", 2024-09-16T00:07:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3984e-05 2024-09-16T00:07:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:55.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:07:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2101e-05 2024-09-16T00:07:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618720+0000", 2024-09-16T00:07:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2761000000000004e-05 2024-09-16T00:07:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618740+0000", 2024-09-16T00:07:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9409e-05 2024-09-16T00:07:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621436+0000", 2024-09-16T00:07:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002696211 2024-09-16T00:07:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621502+0000", 2024-09-16T00:07:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5811999999999999e-05 2024-09-16T00:07:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.747 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:21 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:07:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:07:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.105738409000001, 2024-09-16T00:07:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046318540000000004, 2024-09-16T00:07:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:07:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:55.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:07:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619445+0000", 2024-09-16T00:07:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.083e-06 2024-09-16T00:07:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619455+0000", 2024-09-16T00:07:55.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5920000000000007e-06 2024-09-16T00:07:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619467+0000", 2024-09-16T00:07:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2711000000000001e-05 2024-09-16T00:07:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619476+0000", 2024-09-16T00:07:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0139999999999998e-06 2024-09-16T00:07:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:55.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619529+0000", 2024-09-16T00:07:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3078999999999998e-05 2024-09-16T00:07:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619548+0000", 2024-09-16T00:07:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8952000000000001e-05 2024-09-16T00:07:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624037+0000", 2024-09-16T00:07:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044895730000000002 2024-09-16T00:07:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:55.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624076+0000", 2024-09-16T00:07:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8850000000000002e-05 2024-09-16T00:07:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:55.752 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:55.752 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:07:55.753 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:07:55.763 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:07:55.758+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:07:55.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.803+0000 7f8e364f1640 1 Processor -- start 2024-09-16T00:07:55.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.803+0000 7f8e364f1640 1 -- start start 2024-09-16T00:07:55.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.804+0000 7f8e364f1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8e30151800 0x7f8e300c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.804+0000 7f8e364f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8e30151be0 0x7f8e3015b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.804+0000 7f8e364f1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8e300c63c0 0x7f8e300c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.804+0000 7f8e364f1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8e300c5010 con 0x7f8e300c63c0 2024-09-16T00:07:55.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.804+0000 7f8e364f1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f8e300c9c40 con 0x7f8e30151800 2024-09-16T00:07:55.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.804+0000 7f8e364f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8e3015ad00 con 0x7f8e30151be0 2024-09-16T00:07:55.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.804+0000 7f8e34a8e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8e300c63c0 0x7f8e300c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.804+0000 7f8e34a8e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8e300c63c0 0x7f8e300c3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47880/0 (socket says 172.21.15.139:47880) 2024-09-16T00:07:55.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.804+0000 7f8e34a8e640 1 -- 172.21.15.139:0/3213454909 learned_addr learned my addr 172.21.15.139:0/3213454909 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:55.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.804+0000 7f8e2f7fe640 1 --2- 172.21.15.139:0/3213454909 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8e30151be0 0x7f8e3015b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.804+0000 7f8e2ffff640 1 --2- 172.21.15.139:0/3213454909 >> v2:172.21.15.164:3300/0 conn(0x7f8e30151800 0x7f8e300c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.804+0000 7f8e34a8e640 1 -- 172.21.15.139:0/3213454909 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8e30151be0 msgr2=0x7f8e3015b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:55.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.804+0000 7f8e34a8e640 1 --2- 172.21.15.139:0/3213454909 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8e30151be0 0x7f8e3015b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.804+0000 7f8e34a8e640 1 -- 172.21.15.139:0/3213454909 >> v2:172.21.15.164:3300/0 conn(0x7f8e30151800 msgr2=0x7f8e300c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:55.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.804+0000 7f8e34a8e640 1 --2- 172.21.15.139:0/3213454909 >> v2:172.21.15.164:3300/0 conn(0x7f8e30151800 0x7f8e300c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.804+0000 7f8e34a8e640 1 -- 172.21.15.139:0/3213454909 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e300c5650 con 0x7f8e300c63c0 2024-09-16T00:07:55.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.805+0000 7f8e34a8e640 1 --2- 172.21.15.139:0/3213454909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e300c63c0 0x7f8e300c3700 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8e3006cbb0 tx=0x7f8e2802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c612d4d2e528fa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:55.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.805+0000 7f8e2effd640 1 -- 172.21.15.139:0/3213454909 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e28036070 con 0x7f8e300c63c0 2024-09-16T00:07:55.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.805+0000 7f8e2effd640 1 -- 172.21.15.139:0/3213454909 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e28031600 con 0x7f8e300c63c0 2024-09-16T00:07:55.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.805+0000 7f8e2effd640 1 -- 172.21.15.139:0/3213454909 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e280319d0 con 0x7f8e300c63c0 2024-09-16T00:07:55.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.806+0000 7f8e364f1640 1 -- 172.21.15.139:0/3213454909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e300c63c0 msgr2=0x7f8e300c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.806+0000 7f8e364f1640 1 --2- 172.21.15.139:0/3213454909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e300c63c0 0x7f8e300c3700 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8e3006cbb0 tx=0x7f8e2802a6a0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.806+0000 7f8e364f1640 1 -- 172.21.15.139:0/3213454909 shutdown_connections 2024-09-16T00:07:55.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.806+0000 7f8e364f1640 1 --2- 172.21.15.139:0/3213454909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e300c63c0 0x7f8e300c3700 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.806+0000 7f8e364f1640 1 --2- 172.21.15.139:0/3213454909 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8e30151be0 0x7f8e3015b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.806+0000 7f8e364f1640 1 --2- 172.21.15.139:0/3213454909 >> v2:172.21.15.164:3300/0 conn(0x7f8e30151800 0x7f8e300c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.806+0000 7f8e364f1640 1 -- 172.21.15.139:0/3213454909 >> 172.21.15.139:0/3213454909 conn(0x7f8e3006da40 msgr2=0x7f8e3006f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:55.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.806+0000 7f8e364f1640 1 -- 172.21.15.139:0/3213454909 shutdown_connections 2024-09-16T00:07:55.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.806+0000 7f8e364f1640 1 -- 172.21.15.139:0/3213454909 wait complete. 2024-09-16T00:07:55.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.807+0000 7f8e364f1640 1 Processor -- start 2024-09-16T00:07:55.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.807+0000 7f8e364f1640 1 -- start start 2024-09-16T00:07:55.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e364f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8e300c63c0 0x7f8e300cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e364f1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e30151800 0x7f8e3014dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e364f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e30151be0 0x7f8e30146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e364f1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8e300c5c10 con 0x7f8e30151800 2024-09-16T00:07:55.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e364f1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3015afa0 con 0x7f8e30151be0 2024-09-16T00:07:55.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e364f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8e300c52d0 con 0x7f8e300c63c0 2024-09-16T00:07:55.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e2f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e30151800 0x7f8e3014dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e2f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e30151800 0x7f8e3014dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:47892/0 (socket says 172.21.15.139:47892) 2024-09-16T00:07:55.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e2f7fe640 1 -- 172.21.15.139:0/683120904 learned_addr learned my addr 172.21.15.139:0/683120904 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:55.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e34a8e640 1 --2- 172.21.15.139:0/683120904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e30151be0 0x7f8e30146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e2ffff640 1 --2- 172.21.15.139:0/683120904 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8e300c63c0 0x7f8e300cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e2f7fe640 1 -- 172.21.15.139:0/683120904 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8e300c63c0 msgr2=0x7f8e300cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e2f7fe640 1 --2- 172.21.15.139:0/683120904 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8e300c63c0 0x7f8e300cb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e2f7fe640 1 -- 172.21.15.139:0/683120904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e30151be0 msgr2=0x7f8e30146d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:55.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e2f7fe640 1 --2- 172.21.15.139:0/683120904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e30151be0 0x7f8e30146d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:55.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e2f7fe640 1 -- 172.21.15.139:0/683120904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3015b6a0 con 0x7f8e30151800 2024-09-16T00:07:55.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.808+0000 7f8e2f7fe640 1 --2- 172.21.15.139:0/683120904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e30151800 0x7f8e3014dda0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400e840 tx=0x7f8e2400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:55.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.809+0000 7f8e2cff9640 1 -- 172.21.15.139:0/683120904 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24017070 con 0x7f8e30151800 2024-09-16T00:07:55.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.809+0000 7f8e2cff9640 1 -- 172.21.15.139:0/683120904 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e2400c6d0 con 0x7f8e30151800 2024-09-16T00:07:55.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.809+0000 7f8e2cff9640 1 -- 172.21.15.139:0/683120904 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2400cac0 con 0x7f8e30151800 2024-09-16T00:07:55.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.809+0000 7f8e364f1640 1 -- 172.21.15.139:0/683120904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e301f28b0 con 0x7f8e30151800 2024-09-16T00:07:55.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.809+0000 7f8e364f1640 1 -- 172.21.15.139:0/683120904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e301eeb60 con 0x7f8e30151800 2024-09-16T00:07:55.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.810+0000 7f8e364f1640 1 -- 172.21.15.139:0/683120904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e301ee670 con 0x7f8e30151800 2024-09-16T00:07:55.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.810+0000 7f8e2cff9640 1 -- 172.21.15.139:0/683120904 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8e2401a070 con 0x7f8e30151800 2024-09-16T00:07:55.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.811+0000 7f8e2cff9640 1 --2- 172.21.15.139:0/683120904 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8e1c06c2e0 0x7f8e1c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:55.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.811+0000 7f8e2cff9640 1 -- 172.21.15.139:0/683120904 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5738+0+0 (secure 0 0 0) 0x7f8e2408b4d0 con 0x7f8e30151800 2024-09-16T00:07:55.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.811+0000 7f8e2ffff640 1 --2- 172.21.15.139:0/683120904 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8e1c06c2e0 0x7f8e1c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:55.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.811+0000 7f8e2ffff640 1 --2- 172.21.15.139:0/683120904 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8e1c06c2e0 0x7f8e1c06e7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009ac0 tx=0x7f8e20009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:55.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.815+0000 7f8e2cff9640 1 -- 172.21.15.139:0/683120904 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8e240595d0 con 0x7f8e30151800 2024-09-16T00:07:55.857 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:07:55.855+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:07:55.869 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:07:55.869 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:07:55.869 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:07:55.869 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:07:55.878 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:07:55.958 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:55.957+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:07:55.987 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:07:55.988 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:07:55.988 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:07:55.988 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:07:55.988 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:07:55.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:55.990+0000 7f8e364f1640 1 -- 172.21.15.139:0/683120904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f8e300c6c00 con 0x7f8e30151800 2024-09-16T00:07:55.996 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:07:56.058 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:07:56.056+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:07:56.148 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:07:56.148 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:07:56.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:07:56.148 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:07:56.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.149 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:56.149 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:07:56.149 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.531326225000001, 2024-09-16T00:07:56.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056100969999999997, 2024-09-16T00:07:56.149 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.149 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.149 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.149 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:07:56.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:07:56.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615352+0000", 2024-09-16T00:07:56.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9859999999999999e-06 2024-09-16T00:07:56.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615363+0000", 2024-09-16T00:07:56.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1168999999999999e-05 2024-09-16T00:07:56.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615382+0000", 2024-09-16T00:07:56.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8224e-05 2024-09-16T00:07:56.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615393+0000", 2024-09-16T00:07:56.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0925e-05 2024-09-16T00:07:56.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615624+0000", 2024-09-16T00:07:56.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023159600000000001 2024-09-16T00:07:56.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615721+0000", 2024-09-16T00:07:56.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7040000000000006e-05 2024-09-16T00:07:56.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:07:56.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615885+0000", 2024-09-16T00:07:56.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163456 2024-09-16T00:07:56.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:07:56.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002191368 2024-09-16T00:07:56.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620870+0000", 2024-09-16T00:07:56.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941839999999999 2024-09-16T00:07:56.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620922+0000", 2024-09-16T00:07:56.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1641999999999998e-05 2024-09-16T00:07:56.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620956+0000", 2024-09-16T00:07:56.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4507e-05 2024-09-16T00:07:56.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.154 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.154 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:56.154 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:07:56.155 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.531235865999999, 2024-09-16T00:07:56.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005695739, 2024-09-16T00:07:56.155 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.155 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.155 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.155 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:07:56.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:07:56.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:07:56.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4900000000000005e-07 2024-09-16T00:07:56.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615442+0000", 2024-09-16T00:07:56.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.994e-06 2024-09-16T00:07:56.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:07:56.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1081e-05 2024-09-16T00:07:56.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615473+0000", 2024-09-16T00:07:56.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1910000000000006e-06 2024-09-16T00:07:56.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616230+0000", 2024-09-16T00:07:56.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075700400000000003 2024-09-16T00:07:56.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616305+0000", 2024-09-16T00:07:56.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4759999999999996e-05 2024-09-16T00:07:56.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:07:56.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616466+0000", 2024-09-16T00:07:56.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000161714 2024-09-16T00:07:56.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619821+0000", 2024-09-16T00:07:56.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033547339999999998 2024-09-16T00:07:56.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621028+0000", 2024-09-16T00:07:56.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012067530000000001 2024-09-16T00:07:56.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621087+0000", 2024-09-16T00:07:56.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9364999999999998e-05 2024-09-16T00:07:56.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621132+0000", 2024-09-16T00:07:56.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5293999999999997e-05 2024-09-16T00:07:56.160 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.160 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.160 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.160 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:56.160 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:07:56.161 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.531173633999998, 2024-09-16T00:07:56.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050565760000000001, 2024-09-16T00:07:56.161 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.161 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.161 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.161 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:07:56.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:07:56.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615500+0000", 2024-09-16T00:07:56.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8199999999999999e-07 2024-09-16T00:07:56.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615505+0000", 2024-09-16T00:07:56.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5280000000000003e-06 2024-09-16T00:07:56.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:07:56.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0317e-05 2024-09-16T00:07:56.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615523+0000", 2024-09-16T00:07:56.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4580000000000004e-06 2024-09-16T00:07:56.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615607+0000", 2024-09-16T00:07:56.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4153999999999994e-05 2024-09-16T00:07:56.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615748+0000", 2024-09-16T00:07:56.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140533 2024-09-16T00:07:56.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:07:56.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615911+0000", 2024-09-16T00:07:56.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163678 2024-09-16T00:07:56.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619944+0000", 2024-09-16T00:07:56.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040330909999999999 2024-09-16T00:07:56.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620471+0000", 2024-09-16T00:07:56.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052645600000000002 2024-09-16T00:07:56.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620512+0000", 2024-09-16T00:07:56.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0738000000000002e-05 2024-09-16T00:07:56.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620555+0000", 2024-09-16T00:07:56.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3841000000000003e-05 2024-09-16T00:07:56.166 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.166 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.166 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.166 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:56.166 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:07:56.166 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.531121527, 2024-09-16T00:07:56.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089162479999999999, 2024-09-16T00:07:56.166 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.167 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.167 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.167 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:07:56.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:07:56.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615552+0000", 2024-09-16T00:07:56.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:07:56.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615556+0000", 2024-09-16T00:07:56.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3610000000000003e-06 2024-09-16T00:07:56.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615566+0000", 2024-09-16T00:07:56.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6099999999999995e-06 2024-09-16T00:07:56.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615574+0000", 2024-09-16T00:07:56.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9379999999999992e-06 2024-09-16T00:07:56.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616777+0000", 2024-09-16T00:07:56.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012031640000000001 2024-09-16T00:07:56.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616882+0000", 2024-09-16T00:07:56.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105146 2024-09-16T00:07:56.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:07:56.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619696+0000", 2024-09-16T00:07:56.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028142179999999998 2024-09-16T00:07:56.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624234+0000", 2024-09-16T00:07:56.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045381889999999998 2024-09-16T00:07:56.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624393+0000", 2024-09-16T00:07:56.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015871200000000001 2024-09-16T00:07:56.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624434+0000", 2024-09-16T00:07:56.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0908999999999997e-05 2024-09-16T00:07:56.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624467+0000", 2024-09-16T00:07:56.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3217999999999999e-05 2024-09-16T00:07:56.171 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.172 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.172 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:56.172 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:07:56.172 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.531051730000001, 2024-09-16T00:07:56.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064898359999999997, 2024-09-16T00:07:56.172 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.172 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.172 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.172 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:07:56.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:07:56.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615622+0000", 2024-09-16T00:07:56.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.181e-06 2024-09-16T00:07:56.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615628+0000", 2024-09-16T00:07:56.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2450000000000003e-06 2024-09-16T00:07:56.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:07:56.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2554999999999999e-05 2024-09-16T00:07:56.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615650+0000", 2024-09-16T00:07:56.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4350000000000003e-06 2024-09-16T00:07:56.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616772+0000", 2024-09-16T00:07:56.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001122058 2024-09-16T00:07:56.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616889+0000", 2024-09-16T00:07:56.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116623 2024-09-16T00:07:56.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:07:56.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617035+0000", 2024-09-16T00:07:56.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014579799999999999 2024-09-16T00:07:56.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619829+0000", 2024-09-16T00:07:56.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941379999999998 2024-09-16T00:07:56.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622024+0000", 2024-09-16T00:07:56.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021949920000000002 2024-09-16T00:07:56.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622087+0000", 2024-09-16T00:07:56.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3048999999999999e-05 2024-09-16T00:07:56.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622111+0000", 2024-09-16T00:07:56.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3762000000000001e-05 2024-09-16T00:07:56.177 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.177 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.177 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.177 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:56.177 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:07:56.178 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.530989169000001, 2024-09-16T00:07:56.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065192160000000004, 2024-09-16T00:07:56.178 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.178 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.178 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.178 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:07:56.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:07:56.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615684+0000", 2024-09-16T00:07:56.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8299999999999995e-07 2024-09-16T00:07:56.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615691+0000", 2024-09-16T00:07:56.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.866e-06 2024-09-16T00:07:56.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615707+0000", 2024-09-16T00:07:56.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6073999999999999e-05 2024-09-16T00:07:56.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615714+0000", 2024-09-16T00:07:56.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-16T00:07:56.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617403+0000", 2024-09-16T00:07:56.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016881629999999999 2024-09-16T00:07:56.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617559+0000", 2024-09-16T00:07:56.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015669699999999999 2024-09-16T00:07:56.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:07:56.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:07:56.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000144165 2024-09-16T00:07:56.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621413+0000", 2024-09-16T00:07:56.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037092200000000001 2024-09-16T00:07:56.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622147+0000", 2024-09-16T00:07:56.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073468800000000005 2024-09-16T00:07:56.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622180+0000", 2024-09-16T00:07:56.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2901999999999998e-05 2024-09-16T00:07:56.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622203+0000", 2024-09-16T00:07:56.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2211000000000001e-05 2024-09-16T00:07:56.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.183 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.183 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:56.183 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:07:56.183 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.530946074999999, 2024-09-16T00:07:56.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065825270000000003, 2024-09-16T00:07:56.183 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.183 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.183 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:07:56.183 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:07:56.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:07:56.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:07:56.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7299999999999996e-07 2024-09-16T00:07:56.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615730+0000", 2024-09-16T00:07:56.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.004e-06 2024-09-16T00:07:56.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615737+0000", 2024-09-16T00:07:56.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4789999999999999e-06 2024-09-16T00:07:56.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615745+0000", 2024-09-16T00:07:56.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6699999999999994e-06 2024-09-16T00:07:56.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618115+0000", 2024-09-16T00:07:56.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023701569999999999 2024-09-16T00:07:56.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618247+0000", 2024-09-16T00:07:56.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013185000000000001 2024-09-16T00:07:56.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622309+0000", 2024-09-16T00:07:56.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040617939999999996 2024-09-16T00:07:56.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.187 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.187 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:56.187 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:07:56.187 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.530918826000001, 2024-09-16T00:07:56.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036685950000000002, 2024-09-16T00:07:56.187 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.188 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.188 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:07:56.188 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:07:56.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:07:56.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:07:56.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T00:07:56.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615763+0000", 2024-09-16T00:07:56.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4360000000000002e-06 2024-09-16T00:07:56.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615770+0000", 2024-09-16T00:07:56.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7540000000000006e-06 2024-09-16T00:07:56.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615803+0000", 2024-09-16T00:07:56.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3008e-05 2024-09-16T00:07:56.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615888+0000", 2024-09-16T00:07:56.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5056999999999996e-05 2024-09-16T00:07:56.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:07:56.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013059700000000001 2024-09-16T00:07:56.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619422+0000", 2024-09-16T00:07:56.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034033230000000002 2024-09-16T00:07:56.191 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.191 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.191 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.191 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing_reply(2.1a epoch 22/19 rep_tid 2 lcod 22'1)", 2024-09-16T00:07:56.191 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:07:56.191 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.527477513000001, 2024-09-16T00:07:56.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024304200000000001, 2024-09-16T00:07:56.192 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.192 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.192 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:07:56.192 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:07:56.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:07:56.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619196+0000", 2024-09-16T00:07:56.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2759999999999999e-06 2024-09-16T00:07:56.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619203+0000", 2024-09-16T00:07:56.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1779999999999997e-06 2024-09-16T00:07:56.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619223+0000", 2024-09-16T00:07:56.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9159000000000001e-05 2024-09-16T00:07:56.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619231+0000", 2024-09-16T00:07:56.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7830000000000004e-06 2024-09-16T00:07:56.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619320+0000", 2024-09-16T00:07:56.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8805000000000001e-05 2024-09-16T00:07:56.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619438+0000", 2024-09-16T00:07:56.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117841 2024-09-16T00:07:56.195 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.195 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.195 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.195 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:16 2.1d e22/19 2:b871830b:::605.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:07:56.195 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:07:56.195 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.526930235999998, 2024-09-16T00:07:56.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045616900000000002, 2024-09-16T00:07:56.195 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.195 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.196 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.196 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:07:56.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:07:56.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:07:56.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:07:56.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619755+0000", 2024-09-16T00:07:56.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3599999999999996e-06 2024-09-16T00:07:56.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619767+0000", 2024-09-16T00:07:56.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2805e-05 2024-09-16T00:07:56.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619787+0000", 2024-09-16T00:07:56.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9264000000000001e-05 2024-09-16T00:07:56.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619869+0000", 2024-09-16T00:07:56.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2639999999999995e-05 2024-09-16T00:07:56.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619895+0000", 2024-09-16T00:07:56.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5890000000000001e-05 2024-09-16T00:07:56.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624264+0000", 2024-09-16T00:07:56.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043690049999999996 2024-09-16T00:07:56.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624304+0000", 2024-09-16T00:07:56.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9807e-05 2024-09-16T00:07:56.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.204 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.204 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.204 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:17 2.1d e22/19 2:b871830b:::605.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:07:56.204 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:07:56.204 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.526577527000001, 2024-09-16T00:07:56.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042370910000000001, 2024-09-16T00:07:56.204 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.204 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.204 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.204 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:07:56.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:07:56.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620096+0000", 2024-09-16T00:07:56.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:07:56.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620102+0000", 2024-09-16T00:07:56.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7220000000000002e-06 2024-09-16T00:07:56.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620112+0000", 2024-09-16T00:07:56.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0156999999999999e-05 2024-09-16T00:07:56.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620120+0000", 2024-09-16T00:07:56.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2880000000000004e-06 2024-09-16T00:07:56.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620171+0000", 2024-09-16T00:07:56.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.168e-05 2024-09-16T00:07:56.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620189+0000", 2024-09-16T00:07:56.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7419e-05 2024-09-16T00:07:56.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624306+0000", 2024-09-16T00:07:56.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041175430000000004 2024-09-16T00:07:56.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624332+0000", 2024-09-16T00:07:56.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5717000000000001e-05 2024-09-16T00:07:56.208 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.208 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.208 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.209 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:24 2.7 e22/19 ondisk, result = 0)", 2024-09-16T00:07:56.209 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:07:56.209 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.52631182, 2024-09-16T00:07:56.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000208018, 2024-09-16T00:07:56.209 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.209 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.209 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:07:56.209 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:07:56.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:07:56.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:07:56.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8500000000000001e-07 2024-09-16T00:07:56.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620366+0000", 2024-09-16T00:07:56.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5379999999999996e-06 2024-09-16T00:07:56.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620378+0000", 2024-09-16T00:07:56.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.204e-05 2024-09-16T00:07:56.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620385+0000", 2024-09-16T00:07:56.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.413e-06 2024-09-16T00:07:56.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620455+0000", 2024-09-16T00:07:56.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9886000000000001e-05 2024-09-16T00:07:56.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620463+0000", 2024-09-16T00:07:56.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1559999999999995e-06 2024-09-16T00:07:56.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620569+0000", 2024-09-16T00:07:56.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001054 2024-09-16T00:07:56.213 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.213 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.214 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.214 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:18 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:07:56.214 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:07:56.214 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.525926355999999, 2024-09-16T00:07:56.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021768100000000001, 2024-09-16T00:07:56.214 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.214 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.214 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:07:56.214 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:07:56.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:07:56.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620747+0000", 2024-09-16T00:07:56.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.02e-07 2024-09-16T00:07:56.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620752+0000", 2024-09-16T00:07:56.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4439999999999999e-06 2024-09-16T00:07:56.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620765+0000", 2024-09-16T00:07:56.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2714000000000001e-05 2024-09-16T00:07:56.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:07:56.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1160000000000004e-06 2024-09-16T00:07:56.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:07:56.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3374999999999994e-05 2024-09-16T00:07:56.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620864+0000", 2024-09-16T00:07:56.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4320000000000005e-06 2024-09-16T00:07:56.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620964+0000", 2024-09-16T00:07:56.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9698000000000007e-05 2024-09-16T00:07:56.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.218 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.218 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.218 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:19 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:07:56.218 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:07:56.218 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.525864812000002, 2024-09-16T00:07:56.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00033551400000000003, 2024-09-16T00:07:56.218 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.218 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.218 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:07:56.218 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:07:56.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:07:56.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:07:56.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0800000000000005e-07 2024-09-16T00:07:56.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620812+0000", 2024-09-16T00:07:56.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.427e-06 2024-09-16T00:07:56.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620820+0000", 2024-09-16T00:07:56.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6649999999999992e-06 2024-09-16T00:07:56.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620828+0000", 2024-09-16T00:07:56.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0010000000000001e-06 2024-09-16T00:07:56.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620995+0000", 2024-09-16T00:07:56.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016709400000000001 2024-09-16T00:07:56.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621005+0000", 2024-09-16T00:07:56.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7170000000000003e-06 2024-09-16T00:07:56.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621143+0000", 2024-09-16T00:07:56.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000138102 2024-09-16T00:07:56.222 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.222 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.222 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.222 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:3 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:07:56.222 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:07:56.222 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.524687540999999, 2024-09-16T00:07:56.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022247799999999999, 2024-09-16T00:07:56.222 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.222 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.222 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:07:56.223 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:07:56.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:07:56.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:07:56.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3300000000000003e-07 2024-09-16T00:07:56.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621989+0000", 2024-09-16T00:07:56.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2969999999999999e-06 2024-09-16T00:07:56.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621997+0000", 2024-09-16T00:07:56.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8990000000000001e-06 2024-09-16T00:07:56.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622002+0000", 2024-09-16T00:07:56.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5439999999999998e-06 2024-09-16T00:07:56.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622137+0000", 2024-09-16T00:07:56.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000135103 2024-09-16T00:07:56.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622143+0000", 2024-09-16T00:07:56.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9200000000000001e-06 2024-09-16T00:07:56.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622207+0000", 2024-09-16T00:07:56.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4282000000000006e-05 2024-09-16T00:07:56.226 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.226 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.226 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.226 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing_reply(2.14 epoch 22/19 rep_tid 6 lcod 22'5)", 2024-09-16T00:07:56.226 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:07:56.226 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.524656204999999, 2024-09-16T00:07:56.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029688000000000002, 2024-09-16T00:07:56.227 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.227 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.227 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:07:56.227 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:07:56.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:07:56.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622017+0000", 2024-09-16T00:07:56.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3200000000000001e-07 2024-09-16T00:07:56.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:07:56.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6240000000000002e-06 2024-09-16T00:07:56.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622026+0000", 2024-09-16T00:07:56.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.708e-06 2024-09-16T00:07:56.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622032+0000", 2024-09-16T00:07:56.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4699999999999999e-06 2024-09-16T00:07:56.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622230+0000", 2024-09-16T00:07:56.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019783800000000001 2024-09-16T00:07:56.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622313+0000", 2024-09-16T00:07:56.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2908e-05 2024-09-16T00:07:56.230 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.230 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.230 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.230 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:07:56.231 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:07:56.231 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.524210709999998, 2024-09-16T00:07:56.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044651079999999998, 2024-09-16T00:07:56.231 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.231 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.231 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.231 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:07:56.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:07:56.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622463+0000", 2024-09-16T00:07:56.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5349999999999999e-06 2024-09-16T00:07:56.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622470+0000", 2024-09-16T00:07:56.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7050000000000002e-06 2024-09-16T00:07:56.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622485+0000", 2024-09-16T00:07:56.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4978e-05 2024-09-16T00:07:56.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622493+0000", 2024-09-16T00:07:56.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2139999999999996e-06 2024-09-16T00:07:56.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622546+0000", 2024-09-16T00:07:56.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:07:56.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622595+0000", 2024-09-16T00:07:56.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9112e-05 2024-09-16T00:07:56.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:07:56.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622677+0000", 2024-09-16T00:07:56.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2332999999999999e-05 2024-09-16T00:07:56.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.623515+0000", 2024-09-16T00:07:56.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000837615 2024-09-16T00:07:56.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626867+0000", 2024-09-16T00:07:56.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033524819999999999 2024-09-16T00:07:56.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626902+0000", 2024-09-16T00:07:56.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4310000000000002e-05 2024-09-16T00:07:56.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626927+0000", 2024-09-16T00:07:56.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5128999999999999e-05 2024-09-16T00:07:56.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.236 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.236 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.236 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:39 2.6 e22/19 2:654134d2:::mds0_openfiles.0:head v 22'1, mlcod=0'0)", 2024-09-16T00:07:56.236 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:07:56.236 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.505581433, 2024-09-16T00:07:56.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082120900000000004, 2024-09-16T00:07:56.237 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.237 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.237 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.237 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:07:56.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:07:56.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641092+0000", 2024-09-16T00:07:56.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.141e-06 2024-09-16T00:07:56.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641102+0000", 2024-09-16T00:07:56.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5990000000000008e-06 2024-09-16T00:07:56.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641116+0000", 2024-09-16T00:07:56.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4667999999999999e-05 2024-09-16T00:07:56.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641125+0000", 2024-09-16T00:07:56.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0849999999999997e-06 2024-09-16T00:07:56.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641188+0000", 2024-09-16T00:07:56.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3119000000000003e-05 2024-09-16T00:07:56.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641209+0000", 2024-09-16T00:07:56.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0897e-05 2024-09-16T00:07:56.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641874+0000", 2024-09-16T00:07:56.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00066517200000000001 2024-09-16T00:07:56.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641912+0000", 2024-09-16T00:07:56.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8528000000000001e-05 2024-09-16T00:07:56.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.240 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.241 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:40 2.14 2:2e2fa760:::200.00000001:head [write 940~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:07:56.241 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:07:56.241 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3073453170000002, 2024-09-16T00:07:56.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016643039999999999, 2024-09-16T00:07:56.241 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.241 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.241 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.241 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:07:56.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:07:56.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839329+0000", 2024-09-16T00:07:56.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2330000000000001e-06 2024-09-16T00:07:56.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839343+0000", 2024-09-16T00:07:56.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3634e-05 2024-09-16T00:07:56.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839362+0000", 2024-09-16T00:07:56.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9321999999999999e-05 2024-09-16T00:07:56.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839375+0000", 2024-09-16T00:07:56.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2945000000000001e-05 2024-09-16T00:07:56.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839462+0000", 2024-09-16T00:07:56.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6371000000000004e-05 2024-09-16T00:07:56.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839547+0000", 2024-09-16T00:07:56.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4988999999999996e-05 2024-09-16T00:07:56.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:07:56.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839657+0000", 2024-09-16T00:07:56.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110501 2024-09-16T00:07:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840598+0000", 2024-09-16T00:07:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00094087100000000003 2024-09-16T00:07:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840918+0000", 2024-09-16T00:07:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031968799999999999 2024-09-16T00:07:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840955+0000", 2024-09-16T00:07:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7360999999999997e-05 2024-09-16T00:07:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840991+0000", 2024-09-16T00:07:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6389e-05 2024-09-16T00:07:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:41 2.14 2:2e2fa760:::200.00000001:head [write 1383~2831 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:07:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:07:56.246 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.0336918719999999, 2024-09-16T00:07:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017075809999999999, 2024-09-16T00:07:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:07:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:07:56.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112983+0000", 2024-09-16T00:07:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2560000000000001e-06 2024-09-16T00:07:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112993+0000", 2024-09-16T00:07:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0423e-05 2024-09-16T00:07:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113010+0000", 2024-09-16T00:07:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7169000000000001e-05 2024-09-16T00:07:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113023+0000", 2024-09-16T00:07:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2111e-05 2024-09-16T00:07:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113099+0000", 2024-09-16T00:07:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6798000000000006e-05 2024-09-16T00:07:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113174+0000", 2024-09-16T00:07:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.47e-05 2024-09-16T00:07:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:07:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113297+0000", 2024-09-16T00:07:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012250799999999999 2024-09-16T00:07:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114284+0000", 2024-09-16T00:07:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00098720399999999999 2024-09-16T00:07:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114604+0000", 2024-09-16T00:07:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031988699999999998 2024-09-16T00:07:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114644+0000", 2024-09-16T00:07:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9919999999999997e-05 2024-09-16T00:07:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114688+0000", 2024-09-16T00:07:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4604999999999997e-05 2024-09-16T00:07:56.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.252 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.252 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:07:56.253 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:07:56.254 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:56.253+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:07:56.354 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:07:56.353+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:07:56.397 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:07:56.397 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:07:56.397 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:07:56.397 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:07:56.406 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:07:56.455 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:07:56.454+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:07:56.511 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:07:56.511 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:07:56.512 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:07:56.512 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:07:56.512 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:07:56.521 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:07:56.556 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:07:56.555+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:07:56.628 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:07:56.628 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:07:56.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:07:56.628 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:07:56.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.628 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:07:56.628 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:07:56.628 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.544812563000001, 2024-09-16T00:07:56.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:07:56.628 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.628 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.629 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.629 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:07:56.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.629 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:07:56.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:07:56.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:07:56.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:07:56.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:07:56.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:07:56.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:07:56.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:07:56.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:07:56.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:07:56.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:07:56.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:07:56.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:07:56.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:07:56.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:07:56.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:07:56.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:07:56.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:07:56.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:07:56.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:07:56.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:07:56.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:07:56.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:07:56.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:07:56.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:07:56.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:07:56.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:07:56.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:07:56.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:07:56.635 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.635 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.635 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.636 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:07:56.636 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:07:56.636 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.465280933000003, 2024-09-16T00:07:56.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093071190000000008, 2024-09-16T00:07:56.636 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.636 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.636 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.636 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.636 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.636 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.636 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 106 2024-09-16T00:07:56.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.637 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:07:56.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:07:56.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160362+0000", 2024-09-16T00:07:56.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:07:56.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160438+0000", 2024-09-16T00:07:56.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6711000000000005e-05 2024-09-16T00:07:56.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160444+0000", 2024-09-16T00:07:56.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0530000000000003e-06 2024-09-16T00:07:56.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160449+0000", 2024-09-16T00:07:56.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2189999999999998e-06 2024-09-16T00:07:56.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161270+0000", 2024-09-16T00:07:56.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082099199999999999 2024-09-16T00:07:56.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161348+0000", 2024-09-16T00:07:56.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8361000000000004e-05 2024-09-16T00:07:56.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161472+0000", 2024-09-16T00:07:56.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123523 2024-09-16T00:07:56.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163461+0000", 2024-09-16T00:07:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019891560000000002 2024-09-16T00:07:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169418+0000", 2024-09-16T00:07:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059568490000000002 2024-09-16T00:07:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169570+0000", 2024-09-16T00:07:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152461 2024-09-16T00:07:56.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169668+0000", 2024-09-16T00:07:56.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T00:07:56.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.642 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:07:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:07:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.465185161999997, 2024-09-16T00:07:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092931679999999992, 2024-09-16T00:07:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107 2024-09-16T00:07:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:07:56.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:07:56.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:07:56.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0699999999999998e-07 2024-09-16T00:07:56.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160459+0000", 2024-09-16T00:07:56.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0370000000000001e-06 2024-09-16T00:07:56.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160463+0000", 2024-09-16T00:07:56.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999999e-06 2024-09-16T00:07:56.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160466+0000", 2024-09-16T00:07:56.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.644e-06 2024-09-16T00:07:56.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161873+0000", 2024-09-16T00:07:56.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014074840000000001 2024-09-16T00:07:56.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161949+0000", 2024-09-16T00:07:56.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5749999999999998e-05 2024-09-16T00:07:56.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162069+0000", 2024-09-16T00:07:56.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120257 2024-09-16T00:07:56.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163506+0000", 2024-09-16T00:07:56.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001436344 2024-09-16T00:07:56.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169704+0000", 2024-09-16T00:07:56.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061982829999999997 2024-09-16T00:07:56.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169730+0000", 2024-09-16T00:07:56.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6060000000000001e-05 2024-09-16T00:07:56.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169750+0000", 2024-09-16T00:07:56.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0092e-05 2024-09-16T00:07:56.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.651 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.651 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:07:56.651 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:07:56.651 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.465075849999998, 2024-09-16T00:07:56.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092280039999999997, 2024-09-16T00:07:56.651 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.651 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.652 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.652 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.652 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 108 2024-09-16T00:07:56.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.652 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:07:56.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:07:56.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160567+0000", 2024-09-16T00:07:56.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7300000000000001e-07 2024-09-16T00:07:56.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160569+0000", 2024-09-16T00:07:56.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5299999999999999e-06 2024-09-16T00:07:56.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160574+0000", 2024-09-16T00:07:56.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-16T00:07:56.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160577+0000", 2024-09-16T00:07:56.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.01e-06 2024-09-16T00:07:56.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162518+0000", 2024-09-16T00:07:56.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001940703 2024-09-16T00:07:56.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162571+0000", 2024-09-16T00:07:56.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2670000000000002e-05 2024-09-16T00:07:56.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162647+0000", 2024-09-16T00:07:56.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6778000000000002e-05 2024-09-16T00:07:56.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163544+0000", 2024-09-16T00:07:56.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089702599999999999 2024-09-16T00:07:56.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169754+0000", 2024-09-16T00:07:56.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062091739999999996 2024-09-16T00:07:56.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169776+0000", 2024-09-16T00:07:56.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2391999999999998e-05 2024-09-16T00:07:56.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169794+0000", 2024-09-16T00:07:56.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8318999999999999e-05 2024-09-16T00:07:56.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.657 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.658 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:07:56.658 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:07:56.658 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.465027300000003, 2024-09-16T00:07:56.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092220679999999999, 2024-09-16T00:07:56.658 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.658 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.658 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.658 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.658 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 109 2024-09-16T00:07:56.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.658 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:07:56.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:07:56.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:07:56.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9900000000000001e-07 2024-09-16T00:07:56.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160707+0000", 2024-09-16T00:07:56.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1621999999999998e-05 2024-09-16T00:07:56.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160713+0000", 2024-09-16T00:07:56.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.596e-06 2024-09-16T00:07:56.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160717+0000", 2024-09-16T00:07:56.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5829999999999998e-06 2024-09-16T00:07:56.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162869+0000", 2024-09-16T00:07:56.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021516310000000002 2024-09-16T00:07:56.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162946+0000", 2024-09-16T00:07:56.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7696999999999998e-05 2024-09-16T00:07:56.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163065+0000", 2024-09-16T00:07:56.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118581 2024-09-16T00:07:56.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164431+0000", 2024-09-16T00:07:56.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001366003 2024-09-16T00:07:56.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169797+0000", 2024-09-16T00:07:56.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005366244 2024-09-16T00:07:56.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169819+0000", 2024-09-16T00:07:56.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2036999999999999e-05 2024-09-16T00:07:56.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169837+0000", 2024-09-16T00:07:56.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7575e-05 2024-09-16T00:07:56.664 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.664 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.664 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.664 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:07:56.664 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:07:56.664 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.464917950999997, 2024-09-16T00:07:56.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092871919999999997, 2024-09-16T00:07:56.664 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.664 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.664 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.665 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 110 2024-09-16T00:07:56.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.665 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:07:56.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:07:56.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:07:56.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:07:56.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160728+0000", 2024-09-16T00:07:56.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0620000000000001e-06 2024-09-16T00:07:56.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160731+0000", 2024-09-16T00:07:56.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6500000000000002e-06 2024-09-16T00:07:56.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160734+0000", 2024-09-16T00:07:56.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.762e-06 2024-09-16T00:07:56.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163580+0000", 2024-09-16T00:07:56.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028464549999999999 2024-09-16T00:07:56.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163657+0000", 2024-09-16T00:07:56.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6695999999999999e-05 2024-09-16T00:07:56.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163798+0000", 2024-09-16T00:07:56.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014055 2024-09-16T00:07:56.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165483+0000", 2024-09-16T00:07:56.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016853090000000001 2024-09-16T00:07:56.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169970+0000", 2024-09-16T00:07:56.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00448709 2024-09-16T00:07:56.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169994+0000", 2024-09-16T00:07:56.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3575999999999999e-05 2024-09-16T00:07:56.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170011+0000", 2024-09-16T00:07:56.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7702000000000001e-05 2024-09-16T00:07:56.670 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.670 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.670 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.670 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:07:56.670 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:07:56.671 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.464904281999999, 2024-09-16T00:07:56.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093174859999999998, 2024-09-16T00:07:56.671 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.671 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.671 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.671 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 111 2024-09-16T00:07:56.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.671 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:07:56.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:07:56.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:07:56.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1e-07 2024-09-16T00:07:56.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160741+0000", 2024-09-16T00:07:56.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6110000000000001e-06 2024-09-16T00:07:56.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160744+0000", 2024-09-16T00:07:56.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3299999999999999e-06 2024-09-16T00:07:56.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160746+0000", 2024-09-16T00:07:56.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5469999999999999e-06 2024-09-16T00:07:56.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164069+0000", 2024-09-16T00:07:56.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033222709999999999 2024-09-16T00:07:56.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164113+0000", 2024-09-16T00:07:56.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4212999999999998e-05 2024-09-16T00:07:56.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164187+0000", 2024-09-16T00:07:56.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4136999999999998e-05 2024-09-16T00:07:56.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166429+0000", 2024-09-16T00:07:56.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022423180000000001 2024-09-16T00:07:56.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170016+0000", 2024-09-16T00:07:56.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035869449999999998 2024-09-16T00:07:56.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170038+0000", 2024-09-16T00:07:56.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2053e-05 2024-09-16T00:07:56.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170055+0000", 2024-09-16T00:07:56.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6850999999999998e-05 2024-09-16T00:07:56.676 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.676 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.676 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.676 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:07:56.676 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:07:56.676 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.464890566999998, 2024-09-16T00:07:56.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093480279999999995, 2024-09-16T00:07:56.677 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.677 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.677 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.677 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.677 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.677 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.677 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 112 2024-09-16T00:07:56.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.677 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:07:56.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:07:56.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160752+0000", 2024-09-16T00:07:56.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2700000000000001e-07 2024-09-16T00:07:56.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160808+0000", 2024-09-16T00:07:56.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5797999999999997e-05 2024-09-16T00:07:56.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160814+0000", 2024-09-16T00:07:56.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5980000000000002e-06 2024-09-16T00:07:56.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160819+0000", 2024-09-16T00:07:56.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8960000000000002e-06 2024-09-16T00:07:56.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164468+0000", 2024-09-16T00:07:56.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003649476 2024-09-16T00:07:56.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164525+0000", 2024-09-16T00:07:56.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6227999999999998e-05 2024-09-16T00:07:56.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164595+0000", 2024-09-16T00:07:56.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759000000000005e-05 2024-09-16T00:07:56.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167237+0000", 2024-09-16T00:07:56.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026410470000000001 2024-09-16T00:07:56.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170060+0000", 2024-09-16T00:07:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028234969999999999 2024-09-16T00:07:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170081+0000", 2024-09-16T00:07:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1242000000000001e-05 2024-09-16T00:07:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170100+0000", 2024-09-16T00:07:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8260000000000001e-05 2024-09-16T00:07:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:07:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:07:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.464813472000003, 2024-09-16T00:07:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093139409999999992, 2024-09-16T00:07:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.683 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 113 2024-09-16T00:07:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:07:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:07:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:07:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-07 2024-09-16T00:07:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160833+0000", 2024-09-16T00:07:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.557e-06 2024-09-16T00:07:56.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160839+0000", 2024-09-16T00:07:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3829999999999997e-06 2024-09-16T00:07:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160844+0000", 2024-09-16T00:07:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6909999999999996e-06 2024-09-16T00:07:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165008+0000", 2024-09-16T00:07:56.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041642129999999999 2024-09-16T00:07:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165085+0000", 2024-09-16T00:07:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6651999999999997e-05 2024-09-16T00:07:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165205+0000", 2024-09-16T00:07:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012045900000000001 2024-09-16T00:07:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167283+0000", 2024-09-16T00:07:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020782909999999999 2024-09-16T00:07:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170104+0000", 2024-09-16T00:07:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028207950000000001 2024-09-16T00:07:56.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170125+0000", 2024-09-16T00:07:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0852e-05 2024-09-16T00:07:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170143+0000", 2024-09-16T00:07:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7467000000000001e-05 2024-09-16T00:07:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.689 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:07:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:07:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.464766990000001, 2024-09-16T00:07:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093126960000000005, 2024-09-16T00:07:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.690 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-16T00:07:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:07:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:07:56.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160876+0000", 2024-09-16T00:07:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5700000000000002e-07 2024-09-16T00:07:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160880+0000", 2024-09-16T00:07:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5539999999999999e-06 2024-09-16T00:07:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160888+0000", 2024-09-16T00:07:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.157e-06 2024-09-16T00:07:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160894+0000", 2024-09-16T00:07:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7690000000000002e-06 2024-09-16T00:07:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165519+0000", 2024-09-16T00:07:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046249539999999997 2024-09-16T00:07:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165593+0000", 2024-09-16T00:07:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4039e-05 2024-09-16T00:07:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165715+0000", 2024-09-16T00:07:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012131899999999999 2024-09-16T00:07:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167309+0000", 2024-09-16T00:07:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001594011 2024-09-16T00:07:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170147+0000", 2024-09-16T00:07:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028387130000000001 2024-09-16T00:07:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170170+0000", 2024-09-16T00:07:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2517000000000001e-05 2024-09-16T00:07:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170188+0000", 2024-09-16T00:07:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8006e-05 2024-09-16T00:07:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:07:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:07:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.464725962000003, 2024-09-16T00:07:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093140040000000007, 2024-09-16T00:07:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.697 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-16T00:07:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:07:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:07:56.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160917+0000", 2024-09-16T00:07:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7599999999999996e-07 2024-09-16T00:07:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160992+0000", 2024-09-16T00:07:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4760999999999998e-05 2024-09-16T00:07:56.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161000+0000", 2024-09-16T00:07:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9829999999999996e-06 2024-09-16T00:07:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161007+0000", 2024-09-16T00:07:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4909999999999999e-06 2024-09-16T00:07:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165983+0000", 2024-09-16T00:07:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049755399999999997 2024-09-16T00:07:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166027+0000", 2024-09-16T00:07:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4109000000000001e-05 2024-09-16T00:07:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166100+0000", 2024-09-16T00:07:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3360000000000002e-05 2024-09-16T00:07:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169698+0000", 2024-09-16T00:07:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597795 2024-09-16T00:07:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170192+0000", 2024-09-16T00:07:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049447300000000005 2024-09-16T00:07:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170214+0000", 2024-09-16T00:07:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1546e-05 2024-09-16T00:07:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170230+0000", 2024-09-16T00:07:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6169999999999999e-05 2024-09-16T00:07:56.703 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:07:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:07:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.464616233000001, 2024-09-16T00:07:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092457029999999992, 2024-09-16T00:07:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.704 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-16T00:07:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:07:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:07:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:07:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7899999999999999e-07 2024-09-16T00:07:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161029+0000", 2024-09-16T00:07:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6630000000000002e-06 2024-09-16T00:07:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161036+0000", 2024-09-16T00:07:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T00:07:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161041+0000", 2024-09-16T00:07:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:07:56.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166468+0000", 2024-09-16T00:07:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054266050000000001 2024-09-16T00:07:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166511+0000", 2024-09-16T00:07:56.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3869999999999998e-05 2024-09-16T00:07:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166606+0000", 2024-09-16T00:07:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4437000000000004e-05 2024-09-16T00:07:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169955+0000", 2024-09-16T00:07:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033494810000000001 2024-09-16T00:07:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170234+0000", 2024-09-16T00:07:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027900899999999998 2024-09-16T00:07:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170255+0000", 2024-09-16T00:07:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0537999999999999e-05 2024-09-16T00:07:56.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170272+0000", 2024-09-16T00:07:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6611000000000001e-05 2024-09-16T00:07:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:07:56.711 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:07:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.464591087999999, 2024-09-16T00:07:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093511010000000006, 2024-09-16T00:07:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.712 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-16T00:07:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:07:56.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:07:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:07:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.63e-07 2024-09-16T00:07:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161055+0000", 2024-09-16T00:07:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6770000000000001e-06 2024-09-16T00:07:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161063+0000", 2024-09-16T00:07:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6469999999999998e-06 2024-09-16T00:07:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161068+0000", 2024-09-16T00:07:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.746e-06 2024-09-16T00:07:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166817+0000", 2024-09-16T00:07:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057494959999999998 2024-09-16T00:07:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166893+0000", 2024-09-16T00:07:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6210000000000004e-05 2024-09-16T00:07:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167019+0000", 2024-09-16T00:07:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125585 2024-09-16T00:07:56.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170275+0000", 2024-09-16T00:07:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032563620000000001 2024-09-16T00:07:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170309+0000", 2024-09-16T00:07:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3865999999999997e-05 2024-09-16T00:07:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170362+0000", 2024-09-16T00:07:56.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3094999999999999e-05 2024-09-16T00:07:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170402+0000", 2024-09-16T00:07:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9953999999999997e-05 2024-09-16T00:07:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:07:56.718 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:07:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.015859003000003, 2024-09-16T00:07:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097364389999999995, 2024-09-16T00:07:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138 2024-09-16T00:07:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.719 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:07:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:07:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609784+0000", 2024-09-16T00:07:56.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T00:07:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609787+0000", 2024-09-16T00:07:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280000000000001e-06 2024-09-16T00:07:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609794+0000", 2024-09-16T00:07:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6409999999999996e-06 2024-09-16T00:07:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609799+0000", 2024-09-16T00:07:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9359999999999998e-06 2024-09-16T00:07:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615759+0000", 2024-09-16T00:07:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059599910000000004 2024-09-16T00:07:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615806+0000", 2024-09-16T00:07:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7290000000000003e-05 2024-09-16T00:07:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615877+0000", 2024-09-16T00:07:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1000000000000005e-05 2024-09-16T00:07:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618961+0000", 2024-09-16T00:07:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030841390000000001 2024-09-16T00:07:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619436+0000", 2024-09-16T00:07:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047452799999999999 2024-09-16T00:07:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619482+0000", 2024-09-16T00:07:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5689000000000003e-05 2024-09-16T00:07:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619519+0000", 2024-09-16T00:07:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7938000000000002e-05 2024-09-16T00:07:56.724 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:07:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:07:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.015835289000002, 2024-09-16T00:07:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011812954000000001, 2024-09-16T00:07:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.725 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139 2024-09-16T00:07:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:07:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:07:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:07:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999998e-07 2024-09-16T00:07:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609810+0000", 2024-09-16T00:07:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2100000000000002e-06 2024-09-16T00:07:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609817+0000", 2024-09-16T00:07:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.28e-06 2024-09-16T00:07:56.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609825+0000", 2024-09-16T00:07:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:07:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616116+0000", 2024-09-16T00:07:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006290543 2024-09-16T00:07:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616187+0000", 2024-09-16T00:07:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1830999999999997e-05 2024-09-16T00:07:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616298+0000", 2024-09-16T00:07:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011044699999999999 2024-09-16T00:07:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619559+0000", 2024-09-16T00:07:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032614710000000002 2024-09-16T00:07:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621562+0000", 2024-09-16T00:07:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020028680000000001 2024-09-16T00:07:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621597+0000", 2024-09-16T00:07:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4625000000000001e-05 2024-09-16T00:07:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621620+0000", 2024-09-16T00:07:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2864e-05 2024-09-16T00:07:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.731 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:07:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:07:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.015807492999997, 2024-09-16T00:07:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011828211999999999, 2024-09-16T00:07:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 140 2024-09-16T00:07:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:07:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:07:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:07:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T00:07:56.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:07:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961000000000003e-05 2024-09-16T00:07:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609892+0000", 2024-09-16T00:07:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0516000000000001e-05 2024-09-16T00:07:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609898+0000", 2024-09-16T00:07:56.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0449999999999997e-06 2024-09-16T00:07:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616797+0000", 2024-09-16T00:07:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068998840000000002 2024-09-16T00:07:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616844+0000", 2024-09-16T00:07:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6501000000000001e-05 2024-09-16T00:07:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616935+0000", 2024-09-16T00:07:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1222000000000002e-05 2024-09-16T00:07:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619808+0000", 2024-09-16T00:07:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873198 2024-09-16T00:07:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621624+0000", 2024-09-16T00:07:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001815933 2024-09-16T00:07:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621646+0000", 2024-09-16T00:07:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1622999999999999e-05 2024-09-16T00:07:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621663+0000", 2024-09-16T00:07:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6854000000000002e-05 2024-09-16T00:07:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.737 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:07:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:07:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.015721683999999, 2024-09-16T00:07:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014331501999999999, 2024-09-16T00:07:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-16T00:07:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:07:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:07:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609921+0000", 2024-09-16T00:07:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9599999999999999e-07 2024-09-16T00:07:56.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609925+0000", 2024-09-16T00:07:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T00:07:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609932+0000", 2024-09-16T00:07:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8240000000000002e-06 2024-09-16T00:07:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609937+0000", 2024-09-16T00:07:56.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1070000000000004e-06 2024-09-16T00:07:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618987+0000", 2024-09-16T00:07:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090502719999999998 2024-09-16T00:07:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619064+0000", 2024-09-16T00:07:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6868999999999999e-05 2024-09-16T00:07:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619183+0000", 2024-09-16T00:07:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011961 2024-09-16T00:07:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.620052+0000", 2024-09-16T00:07:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086822799999999997 2024-09-16T00:07:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624197+0000", 2024-09-16T00:07:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041454930000000001 2024-09-16T00:07:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624229+0000", 2024-09-16T00:07:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1758999999999999e-05 2024-09-16T00:07:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624252+0000", 2024-09-16T00:07:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:07:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.743 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:07:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:07:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.542955876000001, 2024-09-16T00:07:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010093276999999999, 2024-09-16T00:07:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.744 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 157 2024-09-16T00:07:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:07:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:07:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:07:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4699999999999998e-07 2024-09-16T00:07:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082688+0000", 2024-09-16T00:07:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0209999999999998e-06 2024-09-16T00:07:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082692+0000", 2024-09-16T00:07:56.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6890000000000002e-06 2024-09-16T00:07:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082695+0000", 2024-09-16T00:07:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5339999999999998e-06 2024-09-16T00:07:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085303+0000", 2024-09-16T00:07:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026084789999999999 2024-09-16T00:07:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085374+0000", 2024-09-16T00:07:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1317999999999997e-05 2024-09-16T00:07:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085506+0000", 2024-09-16T00:07:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013118 2024-09-16T00:07:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086928+0000", 2024-09-16T00:07:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014224559999999999 2024-09-16T00:07:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092665+0000", 2024-09-16T00:07:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057369830000000002 2024-09-16T00:07:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092727+0000", 2024-09-16T00:07:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1810000000000006e-05 2024-09-16T00:07:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092779+0000", 2024-09-16T00:07:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2559999999999998e-05 2024-09-16T00:07:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.749 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:07:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:07:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.542943707999999, 2024-09-16T00:07:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010176388, 2024-09-16T00:07:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.750 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 158 2024-09-16T00:07:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:07:56.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:07:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082699+0000", 2024-09-16T00:07:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.11e-07 2024-09-16T00:07:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082742+0000", 2024-09-16T00:07:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3699000000000003e-05 2024-09-16T00:07:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082748+0000", 2024-09-16T00:07:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8690000000000001e-06 2024-09-16T00:07:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082751+0000", 2024-09-16T00:07:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.942e-06 2024-09-16T00:07:56.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085881+0000", 2024-09-16T00:07:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031303450000000001 2024-09-16T00:07:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085959+0000", 2024-09-16T00:07:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7874999999999996e-05 2024-09-16T00:07:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086086+0000", 2024-09-16T00:07:56.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012685499999999999 2024-09-16T00:07:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087127+0000", 2024-09-16T00:07:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001041298 2024-09-16T00:07:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092790+0000", 2024-09-16T00:07:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056628 2024-09-16T00:07:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092832+0000", 2024-09-16T00:07:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2227000000000001e-05 2024-09-16T00:07:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092875+0000", 2024-09-16T00:07:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2267e-05 2024-09-16T00:07:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:07:56.757 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:07:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.54288567, 2024-09-16T00:07:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010199547, 2024-09-16T00:07:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:07:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:07:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:07:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:07:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:07:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159 2024-09-16T00:07:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:07:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:07:56.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:07:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:07:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:07:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:07:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:07:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082757+0000", 2024-09-16T00:07:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2500000000000001e-07 2024-09-16T00:07:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:07:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082759+0000", 2024-09-16T00:07:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-16T00:07:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:07:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082763+0000", 2024-09-16T00:07:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4889999999999997e-06 2024-09-16T00:07:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:07:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082766+0000", 2024-09-16T00:07:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:07:56.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:07:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086571+0000", 2024-09-16T00:07:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038053269999999998 2024-09-16T00:07:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:07:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086619+0000", 2024-09-16T00:07:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7924999999999999e-05 2024-09-16T00:07:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:07:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086693+0000", 2024-09-16T00:07:56.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000005e-05 2024-09-16T00:07:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087383+0000", 2024-09-16T00:07:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069008299999999997 2024-09-16T00:07:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:07:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092882+0000", 2024-09-16T00:07:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054987450000000002 2024-09-16T00:07:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:07:56.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092921+0000", 2024-09-16T00:07:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9388999999999998e-05 2024-09-16T00:07:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:07:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:07:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:07:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092956+0000", 2024-09-16T00:07:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4647999999999998e-05 2024-09-16T00:07:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:07:56.763 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:07:56.763 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:07:56.765 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:56.764+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:07:56.766 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:07:56.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:56.693+0000 7f8e2cff9640 1 -- 172.21.15.139:0/683120904 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v26) ==== 97+0+0 (secure 0 0 0) 0x7f8e2405d8d0 con 0x7f8e30151800 2024-09-16T00:07:56.767 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is set 2024-09-16T00:07:56.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:56.696+0000 7f8e364f1640 1 -- 172.21.15.139:0/683120904 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8e1c06c2e0 msgr2=0x7f8e1c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:56.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:56.696+0000 7f8e364f1640 1 --2- 172.21.15.139:0/683120904 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8e1c06c2e0 0x7f8e1c06e7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009ac0 tx=0x7f8e20009190 comp rx=0 tx=0).stop 2024-09-16T00:07:56.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:56.696+0000 7f8e364f1640 1 -- 172.21.15.139:0/683120904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e30151800 msgr2=0x7f8e3014dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:56.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:56.696+0000 7f8e364f1640 1 --2- 172.21.15.139:0/683120904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e30151800 0x7f8e3014dda0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400e840 tx=0x7f8e2400ed00 comp rx=0 tx=0).stop 2024-09-16T00:07:56.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:56.696+0000 7f8e364f1640 1 -- 172.21.15.139:0/683120904 shutdown_connections 2024-09-16T00:07:56.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:56.696+0000 7f8e364f1640 1 --2- 172.21.15.139:0/683120904 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8e1c06c2e0 0x7f8e1c06e7a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:56.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:56.696+0000 7f8e364f1640 1 --2- 172.21.15.139:0/683120904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8e30151be0 0x7f8e30146d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:56.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:56.696+0000 7f8e364f1640 1 --2- 172.21.15.139:0/683120904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e30151800 0x7f8e3014dda0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:56.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:56.696+0000 7f8e364f1640 1 --2- 172.21.15.139:0/683120904 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8e300c63c0 0x7f8e300cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:56.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:56.696+0000 7f8e364f1640 1 -- 172.21.15.139:0/683120904 >> 172.21.15.139:0/683120904 conn(0x7f8e3006da40 msgr2=0x7f8e300729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:56.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:56.696+0000 7f8e364f1640 1 -- 172.21.15.139:0/683120904 shutdown_connections 2024-09-16T00:07:56.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:56.697+0000 7f8e364f1640 1 -- 172.21.15.139:0/683120904 wait complete. 2024-09-16T00:07:56.866 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:07:56.866+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:07:56.911 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:07:56.911 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:07:56.911 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:07:56.911 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:07:56.921 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:07:56.966 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:07:56.965+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:07:57.024 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:07:57.025 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:07:57.025 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:07:57.025 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:07:57.025 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:07:57.033 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:07:57.074 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:07:57.072+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:07:57.168 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:07:57.167+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:07:57.179 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:07:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:07:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:07:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:07:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'41, mlcod=17'39)", 2024-09-16T00:07:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:07:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.154871131999997, 2024-09-16T00:07:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001808488, 2024-09-16T00:07:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:07:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:07:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021032+0000", 2024-09-16T00:07:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3e-07 2024-09-16T00:07:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021037+0000", 2024-09-16T00:07:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0640000000000001e-06 2024-09-16T00:07:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021047+0000", 2024-09-16T00:07:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.023e-05 2024-09-16T00:07:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021053+0000", 2024-09-16T00:07:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3160000000000002e-06 2024-09-16T00:07:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021092+0000", 2024-09-16T00:07:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9033999999999999e-05 2024-09-16T00:07:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021105+0000", 2024-09-16T00:07:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2768e-05 2024-09-16T00:07:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022811+0000", 2024-09-16T00:07:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001705788 2024-09-16T00:07:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022840+0000", 2024-09-16T00:07:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8558e-05 2024-09-16T00:07:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:85 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'55, mlcod=17'50)", 2024-09-16T00:07:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:07:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.141842130999997, 2024-09-16T00:07:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017909530000000001, 2024-09-16T00:07:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:07:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:07:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034061+0000", 2024-09-16T00:07:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0399999999999996e-07 2024-09-16T00:07:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034065+0000", 2024-09-16T00:07:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8840000000000002e-06 2024-09-16T00:07:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034074+0000", 2024-09-16T00:07:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:07:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034079+0000", 2024-09-16T00:07:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0000000000000004e-06 2024-09-16T00:07:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034125+0000", 2024-09-16T00:07:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5905000000000002e-05 2024-09-16T00:07:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034144+0000", 2024-09-16T00:07:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9593e-05 2024-09-16T00:07:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035817+0000", 2024-09-16T00:07:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016728240000000001 2024-09-16T00:07:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035851+0000", 2024-09-16T00:07:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3809000000000001e-05 2024-09-16T00:07:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:112 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'74, mlcod=17'65)", 2024-09-16T00:07:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:07:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.011230269000002, 2024-09-16T00:07:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018457020000000001, 2024-09-16T00:07:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:07:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:07:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164673+0000", 2024-09-16T00:07:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1200000000000003e-07 2024-09-16T00:07:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164710+0000", 2024-09-16T00:07:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7756999999999998e-05 2024-09-16T00:07:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164719+0000", 2024-09-16T00:07:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2849999999999995e-06 2024-09-16T00:07:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164723+0000", 2024-09-16T00:07:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3220000000000003e-06 2024-09-16T00:07:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164760+0000", 2024-09-16T00:07:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7311999999999998e-05 2024-09-16T00:07:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164771+0000", 2024-09-16T00:07:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0869e-05 2024-09-16T00:07:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166495+0000", 2024-09-16T00:07:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001723863 2024-09-16T00:07:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166518+0000", 2024-09-16T00:07:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2682000000000001e-05 2024-09-16T00:07:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'66)", 2024-09-16T00:07:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:07:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.009726290000003, 2024-09-16T00:07:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032925509999999999, 2024-09-16T00:07:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:07:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:07:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166177+0000", 2024-09-16T00:07:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:07:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166216+0000", 2024-09-16T00:07:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9008000000000003e-05 2024-09-16T00:07:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166226+0000", 2024-09-16T00:07:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0081999999999999e-05 2024-09-16T00:07:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166230+0000", 2024-09-16T00:07:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5079999999999999e-06 2024-09-16T00:07:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166259+0000", 2024-09-16T00:07:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8928000000000002e-05 2024-09-16T00:07:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166271+0000", 2024-09-16T00:07:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1531e-05 2024-09-16T00:07:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169449+0000", 2024-09-16T00:07:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003178143 2024-09-16T00:07:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169468+0000", 2024-09-16T00:07:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9700000000000001e-05 2024-09-16T00:07:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'66)", 2024-09-16T00:07:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:07:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.009264831000003, 2024-09-16T00:07:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003179374, 2024-09-16T00:07:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:07:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:07:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166638+0000", 2024-09-16T00:07:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2300000000000002e-07 2024-09-16T00:07:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166641+0000", 2024-09-16T00:07:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0079999999999998e-06 2024-09-16T00:07:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166646+0000", 2024-09-16T00:07:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4970000000000001e-06 2024-09-16T00:07:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166650+0000", 2024-09-16T00:07:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5729999999999999e-06 2024-09-16T00:07:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166682+0000", 2024-09-16T00:07:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2218999999999998e-05 2024-09-16T00:07:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166694+0000", 2024-09-16T00:07:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1902999999999999e-05 2024-09-16T00:07:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169784+0000", 2024-09-16T00:07:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003089545 2024-09-16T00:07:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169817+0000", 2024-09-16T00:07:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3206e-05 2024-09-16T00:07:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:117 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'66)", 2024-09-16T00:07:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:07:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.008820771000003, 2024-09-16T00:07:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00275326, 2024-09-16T00:07:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:07:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:07:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167082+0000", 2024-09-16T00:07:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:07:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167086+0000", 2024-09-16T00:07:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1559999999999997e-06 2024-09-16T00:07:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167096+0000", 2024-09-16T00:07:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8390000000000002e-06 2024-09-16T00:07:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167101+0000", 2024-09-16T00:07:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:07:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167130+0000", 2024-09-16T00:07:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9649e-05 2024-09-16T00:07:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167142+0000", 2024-09-16T00:07:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1435e-05 2024-09-16T00:07:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169818+0000", 2024-09-16T00:07:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002676056 2024-09-16T00:07:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169835+0000", 2024-09-16T00:07:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6980999999999999e-05 2024-09-16T00:07:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:130 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'88, mlcod=17'85)", 2024-09-16T00:07:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:07:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.566021442, 2024-09-16T00:07:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002880456, 2024-09-16T00:07:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:07:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:07:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:07:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5699999999999998e-07 2024-09-16T00:07:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609885+0000", 2024-09-16T00:07:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.608e-06 2024-09-16T00:07:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609891+0000", 2024-09-16T00:07:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:07:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609895+0000", 2024-09-16T00:07:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.54e-06 2024-09-16T00:07:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609931+0000", 2024-09-16T00:07:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5893999999999999e-05 2024-09-16T00:07:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609945+0000", 2024-09-16T00:07:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4259e-05 2024-09-16T00:07:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612721+0000", 2024-09-16T00:07:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027757849999999998 2024-09-16T00:07:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612761+0000", 2024-09-16T00:07:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0638e-05 2024-09-16T00:07:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-16T00:07:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:07:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.565637469000002, 2024-09-16T00:07:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028016529999999999, 2024-09-16T00:07:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:07:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:07:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:07:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.63e-07 2024-09-16T00:07:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610344+0000", 2024-09-16T00:07:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8996000000000001e-05 2024-09-16T00:07:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610352+0000", 2024-09-16T00:07:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:07:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610356+0000", 2024-09-16T00:07:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0790000000000002e-06 2024-09-16T00:07:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610388+0000", 2024-09-16T00:07:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1541000000000002e-05 2024-09-16T00:07:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610400+0000", 2024-09-16T00:07:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2075e-05 2024-09-16T00:07:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613045+0000", 2024-09-16T00:07:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002645213 2024-09-16T00:07:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613066+0000", 2024-09-16T00:07:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1353e-05 2024-09-16T00:07:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-16T00:07:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:07:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.562377969000003, 2024-09-16T00:07:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027254850000000001, 2024-09-16T00:07:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:07:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:07:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613525+0000", 2024-09-16T00:07:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2800000000000002e-07 2024-09-16T00:07:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613528+0000", 2024-09-16T00:07:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9730000000000002e-06 2024-09-16T00:07:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613534+0000", 2024-09-16T00:07:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4339999999999997e-06 2024-09-16T00:07:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613538+0000", 2024-09-16T00:07:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5149999999999998e-06 2024-09-16T00:07:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613566+0000", 2024-09-16T00:07:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8104000000000001e-05 2024-09-16T00:07:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613578+0000", 2024-09-16T00:07:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2404e-05 2024-09-16T00:07:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616217+0000", 2024-09-16T00:07:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026389830000000001 2024-09-16T00:07:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616250+0000", 2024-09-16T00:07:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2644000000000001e-05 2024-09-16T00:07:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:134 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'92, mlcod=18'87)", 2024-09-16T00:07:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:07:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.561839829999997, 2024-09-16T00:07:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025322399999999998, 2024-09-16T00:07:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:07:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:07:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614063+0000", 2024-09-16T00:07:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8700000000000005e-07 2024-09-16T00:07:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614096+0000", 2024-09-16T00:07:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3136000000000003e-05 2024-09-16T00:07:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614106+0000", 2024-09-16T00:07:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9170000000000001e-06 2024-09-16T00:07:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614111+0000", 2024-09-16T00:07:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4390000000000002e-06 2024-09-16T00:07:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614141+0000", 2024-09-16T00:07:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.04e-05 2024-09-16T00:07:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614152+0000", 2024-09-16T00:07:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0987e-05 2024-09-16T00:07:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616573+0000", 2024-09-16T00:07:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002421351 2024-09-16T00:07:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616595+0000", 2024-09-16T00:07:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1322999999999999e-05 2024-09-16T00:07:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:135 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'93, mlcod=18'87)", 2024-09-16T00:07:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:07:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.561319791999999, 2024-09-16T00:07:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020304659999999999, 2024-09-16T00:07:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:07:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:07:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614583+0000", 2024-09-16T00:07:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:07:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614587+0000", 2024-09-16T00:07:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3100000000000002e-06 2024-09-16T00:07:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614597+0000", 2024-09-16T00:07:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7450000000000007e-06 2024-09-16T00:07:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614602+0000", 2024-09-16T00:07:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6099999999999999e-06 2024-09-16T00:07:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614641+0000", 2024-09-16T00:07:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9767000000000001e-05 2024-09-16T00:07:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614658+0000", 2024-09-16T00:07:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6205000000000002e-05 2024-09-16T00:07:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616596+0000", 2024-09-16T00:07:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019386659999999999 2024-09-16T00:07:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616613+0000", 2024-09-16T00:07:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6555999999999999e-05 2024-09-16T00:07:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:139 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'97, mlcod=18'87)", 2024-09-16T00:07:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:07:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.559559120999999, 2024-09-16T00:07:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030908139999999999, 2024-09-16T00:07:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:07:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:07:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616344+0000", 2024-09-16T00:07:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3000000000000001e-07 2024-09-16T00:07:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616347+0000", 2024-09-16T00:07:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7740000000000002e-06 2024-09-16T00:07:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616355+0000", 2024-09-16T00:07:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4699999999999996e-06 2024-09-16T00:07:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616358+0000", 2024-09-16T00:07:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5899999999999999e-06 2024-09-16T00:07:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616393+0000", 2024-09-16T00:07:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4529000000000001e-05 2024-09-16T00:07:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616404+0000", 2024-09-16T00:07:57.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0726e-05 2024-09-16T00:07:57.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619411+0000", 2024-09-16T00:07:57.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030076249999999999 2024-09-16T00:07:57.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619434+0000", 2024-09-16T00:07:57.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-05 2024-09-16T00:07:57.247 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.248 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.248 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.248 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:140 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'98, mlcod=18'89)", 2024-09-16T00:07:57.248 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:07:57.248 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.558917727999997, 2024-09-16T00:07:57.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027276150000000001, 2024-09-16T00:07:57.248 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.248 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.248 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.248 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:07:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:07:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616985+0000", 2024-09-16T00:07:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4099999999999999e-07 2024-09-16T00:07:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617055+0000", 2024-09-16T00:07:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9923000000000001e-05 2024-09-16T00:07:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617063+0000", 2024-09-16T00:07:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:07:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617070+0000", 2024-09-16T00:07:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9090000000000003e-06 2024-09-16T00:07:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617102+0000", 2024-09-16T00:07:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2620999999999998e-05 2024-09-16T00:07:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617114+0000", 2024-09-16T00:07:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1228e-05 2024-09-16T00:07:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619690+0000", 2024-09-16T00:07:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025763890000000001 2024-09-16T00:07:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619712+0000", 2024-09-16T00:07:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2171000000000002e-05 2024-09-16T00:07:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:07:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:07:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.560088627999999, 2024-09-16T00:07:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027355320000000002, 2024-09-16T00:07:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:07:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:07:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615815+0000", 2024-09-16T00:07:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1769999999999999e-06 2024-09-16T00:07:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615825+0000", 2024-09-16T00:07:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9000000000000001e-06 2024-09-16T00:07:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615840+0000", 2024-09-16T00:07:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4854e-05 2024-09-16T00:07:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615848+0000", 2024-09-16T00:07:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0940000000000008e-06 2024-09-16T00:07:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615926+0000", 2024-09-16T00:07:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8176999999999994e-05 2024-09-16T00:07:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615945+0000", 2024-09-16T00:07:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9663000000000001e-05 2024-09-16T00:07:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618509+0000", 2024-09-16T00:07:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002563802 2024-09-16T00:07:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618549+0000", 2024-09-16T00:07:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9864999999999999e-05 2024-09-16T00:07:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:07:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.559999487999999, 2024-09-16T00:07:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061593860000000002, 2024-09-16T00:07:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:07:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:07:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615907+0000", 2024-09-16T00:07:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9389999999999999e-06 2024-09-16T00:07:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615920+0000", 2024-09-16T00:07:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2989999999999999e-05 2024-09-16T00:07:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615931+0000", 2024-09-16T00:07:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.164e-05 2024-09-16T00:07:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615939+0000", 2024-09-16T00:07:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8029999999999997e-06 2024-09-16T00:07:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616310+0000", 2024-09-16T00:07:57.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00037111400000000003 2024-09-16T00:07:57.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616406+0000", 2024-09-16T00:07:57.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5372999999999996e-05 2024-09-16T00:07:57.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:07:57.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616514+0000", 2024-09-16T00:07:57.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108564 2024-09-16T00:07:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:07:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618158+0000", 2024-09-16T00:07:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016442449999999999 2024-09-16T00:07:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621972+0000", 2024-09-16T00:07:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038132299999999999 2024-09-16T00:07:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622018+0000", 2024-09-16T00:07:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6239000000000002e-05 2024-09-16T00:07:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622062+0000", 2024-09-16T00:07:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4249000000000003e-05 2024-09-16T00:07:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:07:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.559946020000002, 2024-09-16T00:07:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006258914, 2024-09-16T00:07:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:07:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:07:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615957+0000", 2024-09-16T00:07:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1399999999999997e-07 2024-09-16T00:07:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615960+0000", 2024-09-16T00:07:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0860000000000002e-06 2024-09-16T00:07:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615967+0000", 2024-09-16T00:07:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9059999999999996e-06 2024-09-16T00:07:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615972+0000", 2024-09-16T00:07:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5679999999999999e-06 2024-09-16T00:07:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616717+0000", 2024-09-16T00:07:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074438 2024-09-16T00:07:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616781+0000", 2024-09-16T00:07:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4808999999999996e-05 2024-09-16T00:07:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:07:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616885+0000", 2024-09-16T00:07:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010404900000000001 2024-09-16T00:07:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:07:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:07:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012773870000000001 2024-09-16T00:07:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622116+0000", 2024-09-16T00:07:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039533470000000003 2024-09-16T00:07:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622173+0000", 2024-09-16T00:07:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6347000000000001e-05 2024-09-16T00:07:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622215+0000", 2024-09-16T00:07:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2521000000000002e-05 2024-09-16T00:07:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:07:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.559922282999999, 2024-09-16T00:07:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041725529999999999, 2024-09-16T00:07:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:07:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:07:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:07:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8799999999999998e-07 2024-09-16T00:07:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615990+0000", 2024-09-16T00:07:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2150000000000007e-06 2024-09-16T00:07:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:07:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7159999999999997e-06 2024-09-16T00:07:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616001+0000", 2024-09-16T00:07:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9339999999999997e-06 2024-09-16T00:07:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616293+0000", 2024-09-16T00:07:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029210899999999998 2024-09-16T00:07:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616401+0000", 2024-09-16T00:07:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108087 2024-09-16T00:07:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:07:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616561+0000", 2024-09-16T00:07:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015940600000000001 2024-09-16T00:07:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:07:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618146+0000", 2024-09-16T00:07:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015857040000000001 2024-09-16T00:07:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620091+0000", 2024-09-16T00:07:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001944837 2024-09-16T00:07:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620126+0000", 2024-09-16T00:07:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4242999999999997e-05 2024-09-16T00:07:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620152+0000", 2024-09-16T00:07:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6914e-05 2024-09-16T00:07:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:07:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.559884058000002, 2024-09-16T00:07:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00630628, 2024-09-16T00:07:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:07:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:07:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:07:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5999999999999999e-07 2024-09-16T00:07:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616022+0000", 2024-09-16T00:07:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0149999999999999e-06 2024-09-16T00:07:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616031+0000", 2024-09-16T00:07:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9770000000000006e-06 2024-09-16T00:07:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616036+0000", 2024-09-16T00:07:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2639999999999999e-06 2024-09-16T00:07:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617057+0000", 2024-09-16T00:07:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010212999999999999 2024-09-16T00:07:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617146+0000", 2024-09-16T00:07:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9234000000000007e-05 2024-09-16T00:07:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:07:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617232+0000", 2024-09-16T00:07:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5218999999999998e-05 2024-09-16T00:07:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:07:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619107+0000", 2024-09-16T00:07:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001875016 2024-09-16T00:07:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622259+0000", 2024-09-16T00:07:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031527930000000001 2024-09-16T00:07:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622290+0000", 2024-09-16T00:07:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1022999999999998e-05 2024-09-16T00:07:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622324+0000", 2024-09-16T00:07:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3979000000000001e-05 2024-09-16T00:07:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:07:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.559858756000001, 2024-09-16T00:07:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042412739999999997, 2024-09-16T00:07:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:07:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:07:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616044+0000", 2024-09-16T00:07:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:07:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616052+0000", 2024-09-16T00:07:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.439e-06 2024-09-16T00:07:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616059+0000", 2024-09-16T00:07:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1389999999999998e-06 2024-09-16T00:07:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616065+0000", 2024-09-16T00:07:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:07:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616823+0000", 2024-09-16T00:07:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075781199999999996 2024-09-16T00:07:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616918+0000", 2024-09-16T00:07:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5174999999999996e-05 2024-09-16T00:07:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:07:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617019+0000", 2024-09-16T00:07:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101161 2024-09-16T00:07:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:07:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618503+0000", 2024-09-16T00:07:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014836750000000001 2024-09-16T00:07:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620201+0000", 2024-09-16T00:07:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00169805 2024-09-16T00:07:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620255+0000", 2024-09-16T00:07:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4499000000000001e-05 2024-09-16T00:07:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620285+0000", 2024-09-16T00:07:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9349e-05 2024-09-16T00:07:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.298 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:57.298 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:07:57.298 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.559829765, 2024-09-16T00:07:57.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063682799999999996, 2024-09-16T00:07:57.298 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.298 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.298 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:07:57.298 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:07:57.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:07:57.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:07:57.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9900000000000001e-07 2024-09-16T00:07:57.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.300 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.300 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616079+0000", 2024-09-16T00:07:57.300 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9009999999999999e-06 2024-09-16T00:07:57.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.300 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.300 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616086+0000", 2024-09-16T00:07:57.300 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.776e-06 2024-09-16T00:07:57.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616091+0000", 2024-09-16T00:07:57.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5600000000000001e-06 2024-09-16T00:07:57.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618339+0000", 2024-09-16T00:07:57.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022476179999999998 2024-09-16T00:07:57.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:07:57.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9025999999999997e-05 2024-09-16T00:07:57.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622441+0000", 2024-09-16T00:07:57.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040029999999999996 2024-09-16T00:07:57.302 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.302 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.302 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.302 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.302 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.302 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:07:57.304 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:07:57.304 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:07:57.303+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:07:57.404 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:07:57.404+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:07:57.449 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:07:57.449 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:07:57.449 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:07:57.449 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:07:57.458 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:07:57.505 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:57.505+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:07:57.561 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:07:57.561 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:07:57.561 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:07:57.561 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:07:57.561 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:07:57.570 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:07:57.612 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:07:57.613 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:07:57.612+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:07:57.678 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:07:57.678 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:07:57.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:07:57.678 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:07:57.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.679 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:57.679 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:07:57.679 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.061401615000001, 2024-09-16T00:07:57.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037514929999999998, 2024-09-16T00:07:57.679 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.679 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.679 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.679 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:07:57.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:07:57.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615364+0000", 2024-09-16T00:07:57.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8430000000000003e-06 2024-09-16T00:07:57.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615371+0000", 2024-09-16T00:07:57.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.148e-06 2024-09-16T00:07:57.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.681 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.681 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615380+0000", 2024-09-16T00:07:57.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5889999999999992e-06 2024-09-16T00:07:57.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615389+0000", 2024-09-16T00:07:57.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1620000000000008e-06 2024-09-16T00:07:57.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615525+0000", 2024-09-16T00:07:57.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000136083 2024-09-16T00:07:57.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615662+0000", 2024-09-16T00:07:57.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013696100000000001 2024-09-16T00:07:57.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:07:57.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615799+0000", 2024-09-16T00:07:57.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013781399999999999 2024-09-16T00:07:57.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:07:57.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:07:57.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002363502 2024-09-16T00:07:57.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:57.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619018+0000", 2024-09-16T00:07:57.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085522199999999999 2024-09-16T00:07:57.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619068+0000", 2024-09-16T00:07:57.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9545000000000001e-05 2024-09-16T00:07:57.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619111+0000", 2024-09-16T00:07:57.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3624000000000001e-05 2024-09-16T00:07:57.684 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.685 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.685 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.685 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:57.685 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:07:57.685 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.061349543999999, 2024-09-16T00:07:57.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043656110000000001, 2024-09-16T00:07:57.685 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.685 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.685 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.685 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:07:57.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:07:57.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:07:57.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2199999999999999e-07 2024-09-16T00:07:57.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615416+0000", 2024-09-16T00:07:57.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5590000000000001e-06 2024-09-16T00:07:57.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615423+0000", 2024-09-16T00:07:57.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6250000000000001e-06 2024-09-16T00:07:57.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615426+0000", 2024-09-16T00:07:57.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2540000000000001e-06 2024-09-16T00:07:57.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616075+0000", 2024-09-16T00:07:57.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064968800000000004 2024-09-16T00:07:57.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616142+0000", 2024-09-16T00:07:57.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6203000000000003e-05 2024-09-16T00:07:57.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:07:57.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616247+0000", 2024-09-16T00:07:57.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010572499999999999 2024-09-16T00:07:57.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:07:57.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618913+0000", 2024-09-16T00:07:57.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026654339999999999 2024-09-16T00:07:57.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:57.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619712+0000", 2024-09-16T00:07:57.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079885199999999996 2024-09-16T00:07:57.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619744+0000", 2024-09-16T00:07:57.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2790000000000003e-05 2024-09-16T00:07:57.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619778+0000", 2024-09-16T00:07:57.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3059000000000003e-05 2024-09-16T00:07:57.690 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.690 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.690 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.690 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:57.691 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:07:57.691 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.061313465000001, 2024-09-16T00:07:57.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087673219999999993, 2024-09-16T00:07:57.691 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.691 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.691 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.691 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:07:57.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:07:57.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:07:57.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:07:57.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:07:57.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5392000000000001e-05 2024-09-16T00:07:57.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615470+0000", 2024-09-16T00:07:57.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1999999999999999e-06 2024-09-16T00:07:57.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615474+0000", 2024-09-16T00:07:57.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7359999999999999e-06 2024-09-16T00:07:57.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617614+0000", 2024-09-16T00:07:57.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021398189999999998 2024-09-16T00:07:57.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617724+0000", 2024-09-16T00:07:57.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110808 2024-09-16T00:07:57.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:07:57.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617856+0000", 2024-09-16T00:07:57.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131676 2024-09-16T00:07:57.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:07:57.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:07:57.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029629280000000001 2024-09-16T00:07:57.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:57.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624158+0000", 2024-09-16T00:07:57.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033386549999999998 2024-09-16T00:07:57.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624189+0000", 2024-09-16T00:07:57.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1508999999999999e-05 2024-09-16T00:07:57.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624215+0000", 2024-09-16T00:07:57.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6228999999999999e-05 2024-09-16T00:07:57.696 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.696 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.697 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.697 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:07:57.697 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:07:57.697 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.061279450000001, 2024-09-16T00:07:57.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053285279999999999, 2024-09-16T00:07:57.697 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.697 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.697 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.698 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:07:57.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:07:57.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:07:57.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5600000000000002e-07 2024-09-16T00:07:57.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615484+0000", 2024-09-16T00:07:57.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.688e-06 2024-09-16T00:07:57.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615488+0000", 2024-09-16T00:07:57.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.658e-06 2024-09-16T00:07:57.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615490+0000", 2024-09-16T00:07:57.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.728e-06 2024-09-16T00:07:57.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616441+0000", 2024-09-16T00:07:57.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00095065500000000001 2024-09-16T00:07:57.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616539+0000", 2024-09-16T00:07:57.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8335e-05 2024-09-16T00:07:57.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:07:57.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616631+0000", 2024-09-16T00:07:57.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1260000000000004e-05 2024-09-16T00:07:57.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:57.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619837+0000", 2024-09-16T00:07:57.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003206899 2024-09-16T00:07:57.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:07:57.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620725+0000", 2024-09-16T00:07:57.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088713599999999996 2024-09-16T00:07:57.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620772+0000", 2024-09-16T00:07:57.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7639999999999998e-05 2024-09-16T00:07:57.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620811+0000", 2024-09-16T00:07:57.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8272999999999998e-05 2024-09-16T00:07:57.702 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.702 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.702 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.703 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:07:57.703 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:07:57.703 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.060864362, 2024-09-16T00:07:57.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029886169999999998, 2024-09-16T00:07:57.703 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.703 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.703 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.703 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:07:57.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:07:57.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615898+0000", 2024-09-16T00:07:57.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.237e-06 2024-09-16T00:07:57.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615904+0000", 2024-09-16T00:07:57.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:07:57.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615915+0000", 2024-09-16T00:07:57.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1003e-05 2024-09-16T00:07:57.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615922+0000", 2024-09-16T00:07:57.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9959999999999996e-06 2024-09-16T00:07:57.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615988+0000", 2024-09-16T00:07:57.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5806e-05 2024-09-16T00:07:57.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616009+0000", 2024-09-16T00:07:57.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0729000000000001e-05 2024-09-16T00:07:57.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:07:57.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028462130000000002 2024-09-16T00:07:57.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618886+0000", 2024-09-16T00:07:57.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.082e-05 2024-09-16T00:07:57.709 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.709 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.709 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.711 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:07:57.711 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:07:57.711 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.05925289, 2024-09-16T00:07:57.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032610299999999998, 2024-09-16T00:07:57.711 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.711 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.711 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.711 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:07:57.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:07:57.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617556+0000", 2024-09-16T00:07:57.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7431e-05 2024-09-16T00:07:57.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617847+0000", 2024-09-16T00:07:57.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029133400000000003 2024-09-16T00:07:57.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617858+0000", 2024-09-16T00:07:57.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0222e-05 2024-09-16T00:07:57.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617864+0000", 2024-09-16T00:07:57.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7619999999999998e-06 2024-09-16T00:07:57.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617909+0000", 2024-09-16T00:07:57.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4747999999999999e-05 2024-09-16T00:07:57.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617927+0000", 2024-09-16T00:07:57.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7785999999999999e-05 2024-09-16T00:07:57.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620734+0000", 2024-09-16T00:07:57.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002807305 2024-09-16T00:07:57.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620770+0000", 2024-09-16T00:07:57.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5441999999999997e-05 2024-09-16T00:07:57.716 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.716 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.716 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.716 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:07:57.716 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:07:57.716 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.058832206000002, 2024-09-16T00:07:57.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028641809999999999, 2024-09-16T00:07:57.717 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.717 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.717 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.717 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:07:57.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:07:57.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617930+0000", 2024-09-16T00:07:57.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0200000000000002e-07 2024-09-16T00:07:57.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617934+0000", 2024-09-16T00:07:57.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:07:57.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617943+0000", 2024-09-16T00:07:57.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5820000000000002e-06 2024-09-16T00:07:57.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617949+0000", 2024-09-16T00:07:57.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5069999999999997e-06 2024-09-16T00:07:57.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618094+0000", 2024-09-16T00:07:57.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014515599999999999 2024-09-16T00:07:57.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618111+0000", 2024-09-16T00:07:57.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6705999999999999e-05 2024-09-16T00:07:57.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:07:57.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026603070000000002 2024-09-16T00:07:57.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620793+0000", 2024-09-16T00:07:57.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2214000000000001e-05 2024-09-16T00:07:57.721 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.721 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.721 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.722 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-16T00:07:57.722 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:07:57.722 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.058801417000002, 2024-09-16T00:07:57.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028624549999999999, 2024-09-16T00:07:57.722 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.722 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.722 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.722 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:07:57.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:07:57.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:07:57.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:07:57.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:07:57.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0570000000000001e-06 2024-09-16T00:07:57.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617970+0000", 2024-09-16T00:07:57.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.669e-06 2024-09-16T00:07:57.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617976+0000", 2024-09-16T00:07:57.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1369999999999999e-06 2024-09-16T00:07:57.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618268+0000", 2024-09-16T00:07:57.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029164600000000001 2024-09-16T00:07:57.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618284+0000", 2024-09-16T00:07:57.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5824e-05 2024-09-16T00:07:57.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:07:57.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025127890000000001 2024-09-16T00:07:57.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620822+0000", 2024-09-16T00:07:57.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6009000000000001e-05 2024-09-16T00:07:57.726 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.726 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.727 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.727 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-16T00:07:57.728 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:07:57.728 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.058773836, 2024-09-16T00:07:57.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028650350000000002, 2024-09-16T00:07:57.728 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.728 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.728 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.728 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:07:57.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:07:57.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:07:57.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7599999999999998e-07 2024-09-16T00:07:57.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617991+0000", 2024-09-16T00:07:57.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1860000000000001e-06 2024-09-16T00:07:57.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617999+0000", 2024-09-16T00:07:57.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3250000000000002e-06 2024-09-16T00:07:57.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618004+0000", 2024-09-16T00:07:57.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7300000000000002e-06 2024-09-16T00:07:57.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618422+0000", 2024-09-16T00:07:57.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00041763199999999998 2024-09-16T00:07:57.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:07:57.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6112000000000001e-05 2024-09-16T00:07:57.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620824+0000", 2024-09-16T00:07:57.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002385954 2024-09-16T00:07:57.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.732 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620853+0000", 2024-09-16T00:07:57.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8719999999999999e-05 2024-09-16T00:07:57.732 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.732 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.732 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.732 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-16T00:07:57.732 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:07:57.733 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.058748013999999, 2024-09-16T00:07:57.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032703020000000001, 2024-09-16T00:07:57.733 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.733 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.733 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.733 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:07:57.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:07:57.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618014+0000", 2024-09-16T00:07:57.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7799999999999997e-07 2024-09-16T00:07:57.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618017+0000", 2024-09-16T00:07:57.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4309999999999998e-06 2024-09-16T00:07:57.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618023+0000", 2024-09-16T00:07:57.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8490000000000002e-06 2024-09-16T00:07:57.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618028+0000", 2024-09-16T00:07:57.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3489999999999999e-06 2024-09-16T00:07:57.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:07:57.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055724099999999999 2024-09-16T00:07:57.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618603+0000", 2024-09-16T00:07:57.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7486000000000002e-05 2024-09-16T00:07:57.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621235+0000", 2024-09-16T00:07:57.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002632087 2024-09-16T00:07:57.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621284+0000", 2024-09-16T00:07:57.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8581000000000001e-05 2024-09-16T00:07:57.737 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.737 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.737 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.737 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:14 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:07:57.737 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:07:57.737 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.057811034, 2024-09-16T00:07:57.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016807599999999999, 2024-09-16T00:07:57.737 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.738 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.738 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:07:57.738 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:07:57.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:07:57.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618951+0000", 2024-09-16T00:07:57.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:07:57.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618954+0000", 2024-09-16T00:07:57.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.145e-06 2024-09-16T00:07:57.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618963+0000", 2024-09-16T00:07:57.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5720000000000003e-06 2024-09-16T00:07:57.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618969+0000", 2024-09-16T00:07:57.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3319999999999997e-06 2024-09-16T00:07:57.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619008+0000", 2024-09-16T00:07:57.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9096e-05 2024-09-16T00:07:57.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619013+0000", 2024-09-16T00:07:57.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4710000000000001e-06 2024-09-16T00:07:57.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619118+0000", 2024-09-16T00:07:57.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000105809 2024-09-16T00:07:57.741 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.741 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.741 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.742 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-16T00:07:57.742 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:07:57.742 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.057772926999998, 2024-09-16T00:07:57.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002325307, 2024-09-16T00:07:57.742 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.742 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.742 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.742 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:07:57.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:07:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:07:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:07:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618993+0000", 2024-09-16T00:07:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.687e-06 2024-09-16T00:07:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619000+0000", 2024-09-16T00:07:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3590000000000002e-06 2024-09-16T00:07:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619005+0000", 2024-09-16T00:07:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.95e-06 2024-09-16T00:07:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619039+0000", 2024-09-16T00:07:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3942999999999997e-05 2024-09-16T00:07:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619055+0000", 2024-09-16T00:07:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6107e-05 2024-09-16T00:07:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621287+0000", 2024-09-16T00:07:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022318070000000001 2024-09-16T00:07:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:07:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7129999999999999e-05 2024-09-16T00:07:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-16T00:07:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:07:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.057734930999999, 2024-09-16T00:07:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002314328, 2024-09-16T00:07:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:07:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:07:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:07:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6399999999999998e-07 2024-09-16T00:07:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619030+0000", 2024-09-16T00:07:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.219e-06 2024-09-16T00:07:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619038+0000", 2024-09-16T00:07:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9899999999999997e-06 2024-09-16T00:07:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619043+0000", 2024-09-16T00:07:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5580000000000002e-06 2024-09-16T00:07:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619220+0000", 2024-09-16T00:07:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000176937 2024-09-16T00:07:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619237+0000", 2024-09-16T00:07:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7691999999999999e-05 2024-09-16T00:07:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621315+0000", 2024-09-16T00:07:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020778900000000002 2024-09-16T00:07:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621341+0000", 2024-09-16T00:07:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5678e-05 2024-09-16T00:07:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:15 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:07:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:07:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.057119962000002, 2024-09-16T00:07:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143416, 2024-09-16T00:07:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:07:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:07:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:07:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619642+0000", 2024-09-16T00:07:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5199999999999995e-07 2024-09-16T00:07:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619645+0000", 2024-09-16T00:07:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.134e-06 2024-09-16T00:07:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619655+0000", 2024-09-16T00:07:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3330000000000003e-06 2024-09-16T00:07:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619662+0000", 2024-09-16T00:07:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0500000000000003e-06 2024-09-16T00:07:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619704+0000", 2024-09-16T00:07:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1766e-05 2024-09-16T00:07:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619708+0000", 2024-09-16T00:07:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3440000000000003e-06 2024-09-16T00:07:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619785+0000", 2024-09-16T00:07:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6936999999999999e-05 2024-09-16T00:07:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:31 2.d e22/19 ondisk, result = 0)", 2024-09-16T00:07:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:07:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.057021153000001, 2024-09-16T00:07:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101826, 2024-09-16T00:07:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:07:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:07:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:07:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619741+0000", 2024-09-16T00:07:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3099999999999997e-07 2024-09-16T00:07:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:07:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1719999999999998e-06 2024-09-16T00:07:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619771+0000", 2024-09-16T00:07:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4754000000000001e-05 2024-09-16T00:07:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619790+0000", 2024-09-16T00:07:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9516000000000001e-05 2024-09-16T00:07:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619830+0000", 2024-09-16T00:07:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9848000000000002e-05 2024-09-16T00:07:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619834+0000", 2024-09-16T00:07:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1150000000000004e-06 2024-09-16T00:07:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619842+0000", 2024-09-16T00:07:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.79e-06 2024-09-16T00:07:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:30 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:07:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:07:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.052670165999999, 2024-09-16T00:07:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013076400000000001, 2024-09-16T00:07:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:07:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:07:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:07:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624092+0000", 2024-09-16T00:07:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5799999999999999e-07 2024-09-16T00:07:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624096+0000", 2024-09-16T00:07:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:07:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624106+0000", 2024-09-16T00:07:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7629999999999995e-06 2024-09-16T00:07:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624112+0000", 2024-09-16T00:07:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.626e-06 2024-09-16T00:07:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624149+0000", 2024-09-16T00:07:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7094000000000001e-05 2024-09-16T00:07:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624154+0000", 2024-09-16T00:07:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7359999999999998e-06 2024-09-16T00:07:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624222+0000", 2024-09-16T00:07:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7680000000000003e-05 2024-09-16T00:07:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:39 2.6 2:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:07:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:07:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.036483607999999, 2024-09-16T00:07:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050072119999999996, 2024-09-16T00:07:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:07:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:07:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:07:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640280+0000", 2024-09-16T00:07:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5380000000000001e-06 2024-09-16T00:07:57.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640296+0000", 2024-09-16T00:07:57.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5950000000000001e-05 2024-09-16T00:07:57.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640327+0000", 2024-09-16T00:07:57.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0309999999999999e-05 2024-09-16T00:07:57.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640340+0000", 2024-09-16T00:07:57.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3773e-05 2024-09-16T00:07:57.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640422+0000", 2024-09-16T00:07:57.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1122e-05 2024-09-16T00:07:57.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640572+0000", 2024-09-16T00:07:57.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015044099999999999 2024-09-16T00:07:57.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:07:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640691+0000", 2024-09-16T00:07:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000119393 2024-09-16T00:07:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:07:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645006+0000", 2024-09-16T00:07:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043143779999999998 2024-09-16T00:07:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:07:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645169+0000", 2024-09-16T00:07:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016323299999999999 2024-09-16T00:07:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:07:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645227+0000", 2024-09-16T00:07:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8e-05 2024-09-16T00:07:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645285+0000", 2024-09-16T00:07:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8074e-05 2024-09-16T00:07:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:39 2.6 e22/19 ondisk, result = 0)", 2024-09-16T00:07:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:07:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.031911496999999, 2024-09-16T00:07:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016541200000000001, 2024-09-16T00:07:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:07:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:07:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:07:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:07:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:07:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:07:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:07:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:07:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:07:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:07:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644852+0000", 2024-09-16T00:07:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1670000000000002e-06 2024-09-16T00:07:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:07:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644865+0000", 2024-09-16T00:07:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2826000000000001e-05 2024-09-16T00:07:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:07:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644894+0000", 2024-09-16T00:07:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9264999999999998e-05 2024-09-16T00:07:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:07:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644911+0000", 2024-09-16T00:07:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6827999999999999e-05 2024-09-16T00:07:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:07:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644989+0000", 2024-09-16T00:07:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7849999999999995e-05 2024-09-16T00:07:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:07:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644999+0000", 2024-09-16T00:07:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7289999999999995e-06 2024-09-16T00:07:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:07:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:07:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:07:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645015+0000", 2024-09-16T00:07:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6747000000000001e-05 2024-09-16T00:07:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.774 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:07:57.774 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:07:57.774 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:07:57.777 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:07:57.782 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:57.777+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:07:57.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.711+0000 7fbb006e4640 1 Processor -- start 2024-09-16T00:07:57.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.711+0000 7fbb006e4640 1 -- start start 2024-09-16T00:07:57.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.712+0000 7fbb006e4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbaf80d0490 0x7fbaf80b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.712+0000 7fbb006e4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbaf80b4750 0x7fbaf81546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.712+0000 7fbb006e4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaf80b3180 con 0x7fbaf80d0870 2024-09-16T00:07:57.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.712+0000 7fbb006e4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fbaf815b940 con 0x7fbaf80d0490 2024-09-16T00:07:57.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.712+0000 7fbb006e4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf80b3bb0 con 0x7fbaf80b4750 2024-09-16T00:07:57.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.712+0000 7fbafdc7f640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fbaf80d0870 0x7fbaf80b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37376/0 (socket says 172.21.15.139:37376) 2024-09-16T00:07:57.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.712+0000 7fbafdc7f640 1 -- 172.21.15.139:0/3883405456 learned_addr learned my addr 172.21.15.139:0/3883405456 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:57.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafe480640 1 --2- 172.21.15.139:0/3883405456 >> v2:172.21.15.164:3300/0 conn(0x7fbaf80d0490 0x7fbaf80b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafec81640 1 --2- 172.21.15.139:0/3883405456 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbaf80b4750 0x7fbaf81546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafd47e640 1 -- 172.21.15.139:0/3883405456 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2640671203 0 0) 0x7fbaf80b3180 con 0x7fbaf80d0870 2024-09-16T00:07:57.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafd47e640 1 -- 172.21.15.139:0/3883405456 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbae4003520 con 0x7fbaf80d0870 2024-09-16T00:07:57.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafd47e640 1 -- 172.21.15.139:0/3883405456 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1195278257 0 0) 0x7fbae4003520 con 0x7fbaf80d0870 2024-09-16T00:07:57.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafd47e640 1 -- 172.21.15.139:0/3883405456 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbaf80b4750 msgr2=0x7fbaf81546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:57.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafd47e640 1 --2- 172.21.15.139:0/3883405456 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbaf80b4750 0x7fbaf81546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafd47e640 1 -- 172.21.15.139:0/3883405456 >> v2:172.21.15.164:3300/0 conn(0x7fbaf80d0490 msgr2=0x7fbaf80b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:57.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafd47e640 1 --2- 172.21.15.139:0/3883405456 >> v2:172.21.15.164:3300/0 conn(0x7fbaf80d0490 0x7fbaf80b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafd47e640 1 -- 172.21.15.139:0/3883405456 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf815adc0 con 0x7fbaf80d0870 2024-09-16T00:07:57.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafd47e640 1 -- 172.21.15.139:0/3883405456 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7fbae8001670 con 0x7fbaf80d0870 2024-09-16T00:07:57.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafd47e640 1 -- 172.21.15.139:0/3883405456 >> v1:172.21.15.139:6789/0 conn(0x7fbaf80d0870 legacy=0x7fbaf80b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:57.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafd47e640 1 --2- 172.21.15.139:0/3883405456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbae40039b0 0x7fbae4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafd47e640 1 --2- 172.21.15.139:0/3883405456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae4004800 0x7fbae4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafd47e640 1 --2- 172.21.15.139:0/3883405456 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbaf80b4750 0x7fbae40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafd47e640 1 -- 172.21.15.139:0/3883405456 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf815b940 con 0x7fbae4004800 2024-09-16T00:07:57.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafd47e640 1 -- 172.21.15.139:0/3883405456 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf80b3bb0 con 0x7fbae40039b0 2024-09-16T00:07:57.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafd47e640 1 -- 172.21.15.139:0/3883405456 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbae400b9c0 con 0x7fbaf80b4750 2024-09-16T00:07:57.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafdc7f640 1 --2- 172.21.15.139:0/3883405456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae4004800 0x7fbae4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafec81640 1 --2- 172.21.15.139:0/3883405456 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbaf80b4750 0x7fbae40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafe480640 1 --2- 172.21.15.139:0/3883405456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbae40039b0 0x7fbae4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafdc7f640 1 -- 172.21.15.139:0/3883405456 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbaf80b4750 msgr2=0x7fbae40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:57.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafdc7f640 1 --2- 172.21.15.139:0/3883405456 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbaf80b4750 0x7fbae40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafdc7f640 1 -- 172.21.15.139:0/3883405456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbae40039b0 msgr2=0x7fbae4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:57.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafdc7f640 1 --2- 172.21.15.139:0/3883405456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbae40039b0 0x7fbae4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafdc7f640 1 -- 172.21.15.139:0/3883405456 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae4003520 con 0x7fbae4004800 2024-09-16T00:07:57.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.713+0000 7fbafe480640 1 --2- 172.21.15.139:0/3883405456 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbae40039b0 0x7fbae4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:57.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.714+0000 7fbafdc7f640 1 --2- 172.21.15.139:0/3883405456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae4004800 0x7fbae4004c80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbae800c290 tx=0x7fbae4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4393f86c41ae65 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:57.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.714+0000 7fbafd47e640 1 -- 172.21.15.139:0/3883405456 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8007d50 con 0x7fbae4004800 2024-09-16T00:07:57.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.714+0000 7fbafd47e640 1 -- 172.21.15.139:0/3883405456 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae8005ad0 con 0x7fbae4004800 2024-09-16T00:07:57.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.714+0000 7fbafd47e640 1 -- 172.21.15.139:0/3883405456 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8007070 con 0x7fbae4004800 2024-09-16T00:07:57.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.715+0000 7fbb006e4640 1 -- 172.21.15.139:0/3883405456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae4004800 msgr2=0x7fbae4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:57.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.715+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3883405456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae4004800 0x7fbae4004c80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbae800c290 tx=0x7fbae4005830 comp rx=0 tx=0).stop 2024-09-16T00:07:57.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.715+0000 7fbafec81640 1 -- 172.21.15.139:0/3883405456 reap_dead start 2024-09-16T00:07:57.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.715+0000 7fbb006e4640 1 -- 172.21.15.139:0/3883405456 shutdown_connections 2024-09-16T00:07:57.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.715+0000 7fbb006e4640 1 -- 172.21.15.139:0/3883405456 >> 172.21.15.139:0/3883405456 conn(0x7fbaf806c520 msgr2=0x7fbaf806c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:57.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.715+0000 7fbb006e4640 1 -- 172.21.15.139:0/3883405456 shutdown_connections 2024-09-16T00:07:57.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.715+0000 7fbb006e4640 1 -- 172.21.15.139:0/3883405456 wait complete. 2024-09-16T00:07:57.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.716+0000 7fbb006e4640 1 Processor -- start 2024-09-16T00:07:57.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.716+0000 7fbb006e4640 1 -- start start 2024-09-16T00:07:57.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbb006e4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbae4004800 0x7fbaf814dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbb006e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbaf8157a80 0x7fbaf81563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbb006e4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbaf81580a0 0x7fbaf814fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbb006e4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf815b660 con 0x7fbaf8157a80 2024-09-16T00:07:57.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbb006e4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf815b380 con 0x7fbae4004800 2024-09-16T00:07:57.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbb006e4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf815a980 con 0x7fbaf81580a0 2024-09-16T00:07:57.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbafdc7f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbaf8157a80 0x7fbaf81563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbafdc7f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbaf8157a80 0x7fbaf81563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58884/0 (socket says 172.21.15.139:58884) 2024-09-16T00:07:57.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbafdc7f640 1 -- 172.21.15.139:0/3709919876 learned_addr learned my addr 172.21.15.139:0/3709919876 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:57.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbafe480640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbae4004800 0x7fbaf814dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbafec81640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbaf81580a0 0x7fbaf814fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbafdc7f640 1 -- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbaf81580a0 msgr2=0x7fbaf814fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:57.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbafdc7f640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbaf81580a0 0x7fbaf814fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbafdc7f640 1 -- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbae4004800 msgr2=0x7fbaf814dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:57.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbafdc7f640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbae4004800 0x7fbaf814dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbafdc7f640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf81f3b30 con 0x7fbaf8157a80 2024-09-16T00:07:57.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbafec81640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbaf81580a0 0x7fbaf814fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:57.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbafe480640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbae4004800 0x7fbaf814dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:57.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbafdc7f640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbaf8157a80 0x7fbaf81563d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbae8034040 tx=0x7fbaf80c7780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:57.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8042070 con 0x7fbaf8157a80 2024-09-16T00:07:57.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae8007480 con 0x7fbaf8157a80 2024-09-16T00:07:57.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8001c80 con 0x7fbaf8157a80 2024-09-16T00:07:57.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf81f0730 con 0x7fbaf8157a80 2024-09-16T00:07:57.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.717+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf81f0cf0 con 0x7fbaf8157a80 2024-09-16T00:07:57.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.718+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fbaf814e990 con 0x7fbaf8157a80 2024-09-16T00:07:57.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.719+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbae8045070 con 0x7fbaf8157a80 2024-09-16T00:07:57.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.719+0000 7fbaeeffd640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbadc06c2b0 0x7fbadc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.719+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5738+0+0 (secure 0 0 0) 0x7fbae80ac030 con 0x7fbaf8157a80 2024-09-16T00:07:57.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.719+0000 7fbafe480640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbadc06c2b0 0x7fbadc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.719+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v26) ==== 54+0+16 (secure 0 0 0) 0x7fbae804c030 con 0x7fbaf8157a80 2024-09-16T00:07:57.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.719+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbaf80b3430 0x7fbaf8155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.719+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbaf80cc3e0 con 0x7fbaf80b3430 2024-09-16T00:07:57.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.720+0000 7fbafe480640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbadc06c2b0 0x7fbadc06e770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0007520 tx=0x7fbaf000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:57.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.720+0000 7fbafec81640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbaf80b3430 0x7fbaf8155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.720+0000 7fbafec81640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbaf80b3430 0x7fbaf8155200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:57.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.721+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf80cc3e0 con 0x7fbaf80b3430 2024-09-16T00:07:57.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.734+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbaf80cc100 con 0x7fbaf80b3430 2024-09-16T00:07:57.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.736+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbaf80cc100 con 0x7fbaf80b3430 2024-09-16T00:07:57.790 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:07:57.791 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:07:57.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.736+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbaf814e4a0 0x7fbaf8155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.736+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fbaf80cbe20 con 0x7fbaf814e4a0 2024-09-16T00:07:57.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.736+0000 7fbafe480640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbaf814e4a0 0x7fbaf8155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.737+0000 7fbafe480640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbaf814e4a0 0x7fbaf8155c50 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:57.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.737+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf80cbe20 con 0x7fbaf814e4a0 2024-09-16T00:07:57.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.751+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbaf8156910 con 0x7fbaf814e4a0 2024-09-16T00:07:57.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.753+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbaf8156910 con 0x7fbaf814e4a0 2024-09-16T00:07:57.792 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:07:57.793 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:07:57.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.753+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbaf8062b40 0x7fbaf8062f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.753+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fbaf8063490 con 0x7fbaf8062b40 2024-09-16T00:07:57.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.753+0000 7fbafdc7f640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbaf8062b40 0x7fbaf8062f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.753+0000 7fbafdc7f640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbaf8062b40 0x7fbaf8062f50 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:57.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.754+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf8063490 con 0x7fbaf8062b40 2024-09-16T00:07:57.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.767+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbaf802c890 con 0x7fbaf8062b40 2024-09-16T00:07:57.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.769+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbaf802c890 con 0x7fbaf8062b40 2024-09-16T00:07:57.794 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:07:57.795 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:07:57.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.769+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbaf802d370 0x7fbaf802d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.769+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fbaf802c890 con 0x7fbaf802d370 2024-09-16T00:07:57.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.769+0000 7fbafec81640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbaf802d370 0x7fbaf802d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.769+0000 7fbafec81640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbaf802d370 0x7fbaf802d770 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:57.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.770+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf802c890 con 0x7fbaf802d370 2024-09-16T00:07:57.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.783+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbaf802c890 con 0x7fbaf802d370 2024-09-16T00:07:57.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.785+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbaf802c890 con 0x7fbaf802d370 2024-09-16T00:07:57.796 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:07:57.797 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:07:57.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.785+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbaf80cebd0 0x7fbaf80cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.785+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fbaf80cf570 con 0x7fbaf80cebd0 2024-09-16T00:07:57.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.785+0000 7fbafe480640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbaf80cebd0 0x7fbaf80cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.786+0000 7fbafe480640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbaf80cebd0 0x7fbaf80cf030 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:57.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.787+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf80cf570 con 0x7fbaf80cebd0 2024-09-16T00:07:57.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.800+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbaf80cf570 con 0x7fbaf80cebd0 2024-09-16T00:07:57.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.802+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbaf80cf570 con 0x7fbaf80cebd0 2024-09-16T00:07:57.804 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:07:57.804 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:07:57.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.802+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbaf8204e10 0x7fbaf8207200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.802+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fbaf8209fd0 con 0x7fbaf8204e10 2024-09-16T00:07:57.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.802+0000 7fbafdc7f640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbaf8204e10 0x7fbaf8207200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.803+0000 7fbafdc7f640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbaf8204e10 0x7fbaf8207200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:57.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.804+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf8209fd0 con 0x7fbaf8204e10 2024-09-16T00:07:57.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.818+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbaf8207740 con 0x7fbaf8204e10 2024-09-16T00:07:57.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.820+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbaf8207740 con 0x7fbaf8204e10 2024-09-16T00:07:57.822 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:07:57.823 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:07:57.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.820+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbaf8207dc0 0x7fbaf820a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.820+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fbaf820cf80 con 0x7fbaf8207dc0 2024-09-16T00:07:57.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.820+0000 7fbafec81640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbaf8207dc0 0x7fbaf820a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.821+0000 7fbafec81640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbaf8207dc0 0x7fbaf820a1b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:57.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.821+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf820cf80 con 0x7fbaf8207dc0 2024-09-16T00:07:57.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.850+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbaf820a6f0 con 0x7fbaf8207dc0 2024-09-16T00:07:57.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.852+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbaf820a6f0 con 0x7fbaf8207dc0 2024-09-16T00:07:57.854 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:07:57.854 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:07:57.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.852+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbaf820ad70 0x7fbaf820d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:57.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.852+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fbaf82100e0 con 0x7fbaf820ad70 2024-09-16T00:07:57.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.852+0000 7fbafe480640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbaf820ad70 0x7fbaf820d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:57.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.853+0000 7fbafe480640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbaf820ad70 0x7fbaf820d160 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:57.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.854+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf82100e0 con 0x7fbaf820ad70 2024-09-16T00:07:57.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.870+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbaf820d6f0 con 0x7fbaf820ad70 2024-09-16T00:07:57.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.872+0000 7fbaeeffd640 1 -- 172.21.15.139:0/3709919876 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbaf820d6f0 con 0x7fbaf820ad70 2024-09-16T00:07:57.874 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:07:57.876 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:07:57.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbaf80b3430 msgr2=0x7fbaf8155200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:57.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbaf80b3430 0x7fbaf8155200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbaf814e4a0 msgr2=0x7fbaf8155c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:57.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbaf814e4a0 0x7fbaf8155c50 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbaf8062b40 msgr2=0x7fbaf8062f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:57.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbaf8062b40 0x7fbaf8062f50 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbaf802d370 msgr2=0x7fbaf802d770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:57.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbaf802d370 0x7fbaf802d770 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbaf80cebd0 msgr2=0x7fbaf80cf030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:57.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbaf80cebd0 0x7fbaf80cf030 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbaf8204e10 msgr2=0x7fbaf8207200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:57.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbaf8204e10 0x7fbaf8207200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbaf8207dc0 msgr2=0x7fbaf820a1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:57.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbafec81640 1 -- 172.21.15.139:0/3709919876 reap_dead start 2024-09-16T00:07:57.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbaf8207dc0 0x7fbaf820a1b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbafec81640 1 -- 172.21.15.139:0/3709919876 reap_dead start 2024-09-16T00:07:57.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbafec81640 1 -- 172.21.15.139:0/3709919876 reap_dead start 2024-09-16T00:07:57.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbaf820ad70 msgr2=0x7fbaf820d160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:57.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbaf820ad70 0x7fbaf820d160 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbadc06c2b0 msgr2=0x7fbadc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:57.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbadc06c2b0 0x7fbadc06e770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0007520 tx=0x7fbaf000b040 comp rx=0 tx=0).stop 2024-09-16T00:07:57.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbaf8157a80 msgr2=0x7fbaf81563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:57.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbaf8157a80 0x7fbaf81563d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbae8034040 tx=0x7fbaf80c7780 comp rx=0 tx=0).stop 2024-09-16T00:07:57.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 shutdown_connections 2024-09-16T00:07:57.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbaf8207dc0 0x7fbaf820a1b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbaf820ad70 0x7fbaf820d160 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbadc06c2b0 0x7fbadc06e770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 --2- 172.21.15.139:0/3709919876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbaf8157a80 0x7fbaf81563d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:57.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 >> 172.21.15.139:0/3709919876 conn(0x7fbaf806c520 msgr2=0x7fbaf8154470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:57.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 shutdown_connections 2024-09-16T00:07:57.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:57.873+0000 7fbb006e4640 1 -- 172.21.15.139:0/3709919876 wait complete. 2024-09-16T00:07:57.879 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:07:57.879+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:07:57.914 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:57.914 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:57.923 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:07:57.981 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:07:57.980+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:07:58.067 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:58.067 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:58.076 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:07:58.119 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:07:58.118+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:07:58.178 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:58.178 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:58.183 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:07:58.182+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:07:58.188 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:07:58.283 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:07:58.283+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:07:58.330 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:58.331 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:58.339 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:07:58.384 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:07:58.384+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:07:58.443 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:58.443 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:58.452 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:07:58.486 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:07:58.485+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:07:58.553 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:58.553 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:58.562 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:07:58.587 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:07:58.585+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:07:58.680 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:58.680 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:58.688 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:58.687+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:07:58.689 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:07:58.768 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T00:07:58.814 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:07:58.812+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:07:58.834 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:58.834 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:58.843 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:07:58.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.866+0000 7f9a53c9b640 1 Processor -- start 2024-09-16T00:07:58.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.866+0000 7f9a53c9b640 1 -- start start 2024-09-16T00:07:58.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.867+0000 7f9a53c9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9a4c06ea10 0x7f9a4c071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:58.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.867+0000 7f9a53c9b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9a4c0c93d0 0x7f9a4c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:58.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.867+0000 7f9a53c9b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a4c0ca000 con 0x7f9a4c154d40 2024-09-16T00:07:58.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.867+0000 7f9a53c9b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9a4c06d340 con 0x7f9a4c06ea10 2024-09-16T00:07:58.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.867+0000 7f9a53c9b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f9a4c0c9db0 con 0x7f9a4c0c93d0 2024-09-16T00:07:58.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.867+0000 7f9a51236640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9a4c06ea10 0x7f9a4c071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:58.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.867+0000 7f9a51a37640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f9a4c154d40 0x7f9a4c06cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37390/0 (socket says 172.21.15.139:37390) 2024-09-16T00:07:58.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.867+0000 7f9a51a37640 1 -- 172.21.15.139:0/1701675252 learned_addr learned my addr 172.21.15.139:0/1701675252 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:07:58.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.867+0000 7f9a52238640 1 --2- 172.21.15.139:0/1701675252 >> v2:172.21.15.164:3300/0 conn(0x7f9a4c0c93d0 0x7f9a4c071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:58.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.867+0000 7f9a51236640 1 -- 172.21.15.139:0/1701675252 >> v2:172.21.15.164:3300/0 conn(0x7f9a4c0c93d0 msgr2=0x7f9a4c071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:58.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.867+0000 7f9a51236640 1 --2- 172.21.15.139:0/1701675252 >> v2:172.21.15.164:3300/0 conn(0x7f9a4c0c93d0 0x7f9a4c071a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:58.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.867+0000 7f9a51236640 1 -- 172.21.15.139:0/1701675252 >> v1:172.21.15.139:6789/0 conn(0x7f9a4c154d40 legacy=0x7f9a4c06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:58.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.867+0000 7f9a50a35640 1 -- 172.21.15.139:0/1701675252 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3367556131 0 0) 0x7f9a4c0ca000 con 0x7f9a4c154d40 2024-09-16T00:07:58.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.867+0000 7f9a51236640 1 -- 172.21.15.139:0/1701675252 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c0c9900 con 0x7f9a4c06ea10 2024-09-16T00:07:58.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.868+0000 7f9a51236640 1 --2- 172.21.15.139:0/1701675252 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a4c06ea10 0x7f9a4c071520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0b3910 tx=0x7f9a3c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4f7d651025a0ff server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:58.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.868+0000 7f9a50a35640 1 -- 172.21.15.139:0/1701675252 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c037070 con 0x7f9a4c06ea10 2024-09-16T00:07:58.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.868+0000 7f9a50a35640 1 -- 172.21.15.139:0/1701675252 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a3c032440 con 0x7f9a4c06ea10 2024-09-16T00:07:58.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.868+0000 7f9a50a35640 1 -- 172.21.15.139:0/1701675252 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c032810 con 0x7f9a4c06ea10 2024-09-16T00:07:58.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.869+0000 7f9a53c9b640 1 -- 172.21.15.139:0/1701675252 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a4c06ea10 msgr2=0x7f9a4c071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:58.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.869+0000 7f9a53c9b640 1 --2- 172.21.15.139:0/1701675252 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a4c06ea10 0x7f9a4c071520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0b3910 tx=0x7f9a3c02a230 comp rx=0 tx=0).stop 2024-09-16T00:07:58.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.869+0000 7f9a53c9b640 1 -- 172.21.15.139:0/1701675252 shutdown_connections 2024-09-16T00:07:58.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.869+0000 7f9a53c9b640 1 --2- 172.21.15.139:0/1701675252 >> v2:172.21.15.164:3300/0 conn(0x7f9a4c0c93d0 0x7f9a4c071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:58.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.869+0000 7f9a53c9b640 1 --2- 172.21.15.139:0/1701675252 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a4c06ea10 0x7f9a4c071520 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:58.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.869+0000 7f9a53c9b640 1 -- 172.21.15.139:0/1701675252 >> 172.21.15.139:0/1701675252 conn(0x7f9a4c06bb20 msgr2=0x7f9a4c06f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:58.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.869+0000 7f9a53c9b640 1 -- 172.21.15.139:0/1701675252 shutdown_connections 2024-09-16T00:07:58.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.869+0000 7f9a53c9b640 1 -- 172.21.15.139:0/1701675252 wait complete. 2024-09-16T00:07:58.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.870+0000 7f9a53c9b640 1 Processor -- start 2024-09-16T00:07:58.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.870+0000 7f9a53c9b640 1 -- start start 2024-09-16T00:07:58.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a53c9b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a4c0c93d0 0x7f9a4c14ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:58.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a53c9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a4c154d40 0x7f9a4c1553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:58.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a53c9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a4c1558e0 0x7f9a4c1f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:58.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a53c9b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c0ca390 con 0x7f9a4c1558e0 2024-09-16T00:07:58.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a53c9b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c0c9040 con 0x7f9a4c154d40 2024-09-16T00:07:58.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a53c9b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9a4c0c52d0 con 0x7f9a4c0c93d0 2024-09-16T00:07:58.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a52238640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a4c1558e0 0x7f9a4c1f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:58.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a52238640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a4c1558e0 0x7f9a4c1f3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58902/0 (socket says 172.21.15.139:58902) 2024-09-16T00:07:58.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a52238640 1 -- 172.21.15.139:0/1556269649 learned_addr learned my addr 172.21.15.139:0/1556269649 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:58.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a51236640 1 --2- 172.21.15.139:0/1556269649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a4c154d40 0x7f9a4c1553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:58.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a51a37640 1 --2- 172.21.15.139:0/1556269649 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a4c0c93d0 0x7f9a4c14ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:58.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a52238640 1 -- 172.21.15.139:0/1556269649 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a4c0c93d0 msgr2=0x7f9a4c14ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:58.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a52238640 1 --2- 172.21.15.139:0/1556269649 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a4c0c93d0 0x7f9a4c14ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:58.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a52238640 1 -- 172.21.15.139:0/1556269649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a4c154d40 msgr2=0x7f9a4c1553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:58.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a52238640 1 --2- 172.21.15.139:0/1556269649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a4c154d40 0x7f9a4c1553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:58.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a52238640 1 -- 172.21.15.139:0/1556269649 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c14dd40 con 0x7f9a4c1558e0 2024-09-16T00:07:58.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a51236640 1 --2- 172.21.15.139:0/1556269649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a4c154d40 0x7f9a4c1553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:07:58.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a52238640 1 --2- 172.21.15.139:0/1556269649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a4c1558e0 0x7f9a4c1f3bb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009e30 tx=0x7f9a4400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:58.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a427fc640 1 -- 172.21.15.139:0/1556269649 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44018020 con 0x7f9a4c1558e0 2024-09-16T00:07:58.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a427fc640 1 -- 172.21.15.139:0/1556269649 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a44014480 con 0x7f9a4c1558e0 2024-09-16T00:07:58.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a53c9b640 1 -- 172.21.15.139:0/1556269649 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4c0c7eb0 con 0x7f9a4c1558e0 2024-09-16T00:07:58.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a427fc640 1 -- 172.21.15.139:0/1556269649 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44014870 con 0x7f9a4c1558e0 2024-09-16T00:07:58.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.871+0000 7f9a53c9b640 1 -- 172.21.15.139:0/1556269649 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4c0c8570 con 0x7f9a4c1558e0 2024-09-16T00:07:58.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.872+0000 7f9a53c9b640 1 -- 172.21.15.139:0/1556269649 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a4c0c8080 con 0x7f9a4c1558e0 2024-09-16T00:07:58.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.873+0000 7f9a427fc640 1 -- 172.21.15.139:0/1556269649 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9a44005d10 con 0x7f9a4c1558e0 2024-09-16T00:07:58.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.873+0000 7f9a427fc640 1 --2- 172.21.15.139:0/1556269649 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9a3006c2e0 0x7f9a3006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:58.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.873+0000 7f9a51a37640 1 --2- 172.21.15.139:0/1556269649 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9a3006c2e0 0x7f9a3006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:58.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.873+0000 7f9a427fc640 1 -- 172.21.15.139:0/1556269649 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5738+0+0 (secure 0 0 0) 0x7f9a4408c010 con 0x7f9a4c1558e0 2024-09-16T00:07:58.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.874+0000 7f9a51a37640 1 --2- 172.21.15.139:0/1556269649 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9a3006c2e0 0x7f9a3006e7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9a48000f60 tx=0x7f9a48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:58.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:58.877+0000 7f9a427fc640 1 -- 172.21.15.139:0/1556269649 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9a44022030 con 0x7f9a4c1558e0 2024-09-16T00:07:58.889 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:07:58.887+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:07:58.947 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:58.947 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:58.955 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:07:58.990 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:07:58.988+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:07:59.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.052+0000 7f9a53c9b640 1 -- 172.21.15.139:0/1556269649 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f9a4c0c4ea0 con 0x7f9a4c1558e0 2024-09-16T00:07:59.061 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:59.061 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:59.070 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:07:59.114 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:07:59.113+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:07:59.174 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:59.174 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:59.183 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:07:59.225 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:07:59.224+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:07:59.293 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:07:59.292+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:07:59.327 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:59.327 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:59.336 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:07:59.394 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:07:59.394+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:07:59.442 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:59.442 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:59.450 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:07:59.495 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:07:59.494+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:07:59.594 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:59.595 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:59.596 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:07:59.596+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:07:59.604 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:07:59.697 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:07:59.696+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:07:59.699 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T00:07:59.699 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:07:59.699 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-16T00:07:59.699 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T00:07:59.700 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T00:07:59.700 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-16T00:07:59.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.700+0000 7f9a427fc640 1 -- 172.21.15.139:0/1556269649 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v27) ==== 91+0+0 (secure 0 0 0) 0x7f9a4c14dd40 con 0x7f9a4c1558e0 2024-09-16T00:07:59.703 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is unset 2024-09-16T00:07:59.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.703+0000 7f9a53c9b640 1 -- 172.21.15.139:0/1556269649 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9a3006c2e0 msgr2=0x7f9a3006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:59.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.703+0000 7f9a53c9b640 1 --2- 172.21.15.139:0/1556269649 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9a3006c2e0 0x7f9a3006e7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9a48000f60 tx=0x7f9a48009040 comp rx=0 tx=0).stop 2024-09-16T00:07:59.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.703+0000 7f9a53c9b640 1 -- 172.21.15.139:0/1556269649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a4c1558e0 msgr2=0x7f9a4c1f3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:59.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.703+0000 7f9a53c9b640 1 --2- 172.21.15.139:0/1556269649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a4c1558e0 0x7f9a4c1f3bb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009e30 tx=0x7f9a4400c490 comp rx=0 tx=0).stop 2024-09-16T00:07:59.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.703+0000 7f9a53c9b640 1 -- 172.21.15.139:0/1556269649 shutdown_connections 2024-09-16T00:07:59.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.703+0000 7f9a53c9b640 1 --2- 172.21.15.139:0/1556269649 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9a3006c2e0 0x7f9a3006e7a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:59.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.703+0000 7f9a53c9b640 1 --2- 172.21.15.139:0/1556269649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a4c1558e0 0x7f9a4c1f3bb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:59.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.703+0000 7f9a53c9b640 1 --2- 172.21.15.139:0/1556269649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9a4c154d40 0x7f9a4c1553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:59.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.703+0000 7f9a53c9b640 1 --2- 172.21.15.139:0/1556269649 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9a4c0c93d0 0x7f9a4c14ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:59.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.703+0000 7f9a53c9b640 1 -- 172.21.15.139:0/1556269649 >> 172.21.15.139:0/1556269649 conn(0x7f9a4c06bb20 msgr2=0x7f9a4c0c3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:59.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.703+0000 7f9a53c9b640 1 -- 172.21.15.139:0/1556269649 shutdown_connections 2024-09-16T00:07:59.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.703+0000 7f9a53c9b640 1 -- 172.21.15.139:0/1556269649 wait complete. 2024-09-16T00:07:59.751 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:59.751 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:59.760 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:07:59.804 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:07:59.804+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:07:59.811 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:07:59.811 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T00:07:59.811 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:07:59.864 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:07:59.864 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:07:59.873 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:07:59.887 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:07:59.931 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:07:59.930+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:07:59.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.982+0000 7fedd9398640 1 Processor -- start 2024-09-16T00:07:59.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.982+0000 7fedd9398640 1 -- start start 2024-09-16T00:07:59.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.983+0000 7fedd9398640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fedd40d0490 0x7fedd40b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:59.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.983+0000 7fedd9398640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fedd40d0870 0x7fedd40b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:59.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.983+0000 7fedd9398640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd40b3180 con 0x7fedd40b3430 2024-09-16T00:07:59.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.983+0000 7fedd9398640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fedd415b940 con 0x7fedd40d0870 2024-09-16T00:07:59.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.983+0000 7fedd9398640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fedd40b3bb0 con 0x7fedd40d0490 2024-09-16T00:07:59.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.983+0000 7fedd27fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fedd40d0870 0x7fedd40b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:59.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.983+0000 7fedd27fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fedd40d0870 0x7fedd40b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58912/0 (socket says 172.21.15.139:58912) 2024-09-16T00:07:59.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.983+0000 7fedd27fc640 1 -- 172.21.15.139:0/2480596129 learned_addr learned my addr 172.21.15.139:0/2480596129 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:59.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.983+0000 7fedd2ffd640 1 --2- 172.21.15.139:0/2480596129 >> v2:172.21.15.164:3300/0 conn(0x7fedd40d0490 0x7fedd40b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:59.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.983+0000 7fedd27fc640 1 -- 172.21.15.139:0/2480596129 >> v2:172.21.15.164:3300/0 conn(0x7fedd40d0490 msgr2=0x7fedd40b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:07:59.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.983+0000 7fedd27fc640 1 --2- 172.21.15.139:0/2480596129 >> v2:172.21.15.164:3300/0 conn(0x7fedd40d0490 0x7fedd40b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:59.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.983+0000 7fedd27fc640 1 -- 172.21.15.139:0/2480596129 >> v1:172.21.15.164:6789/0 conn(0x7fedd40b3430 legacy=0x7fedd41546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:59.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.983+0000 7fedd27fc640 1 -- 172.21.15.139:0/2480596129 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd415adc0 con 0x7fedd40d0870 2024-09-16T00:07:59.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.984+0000 7fedd27fc640 1 --2- 172.21.15.139:0/2480596129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd40d0870 0x7fedd40b4040 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fedd40cddf0 tx=0x7fedc802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2310dc3dc3c2b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:59.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.984+0000 7fedd1ffb640 1 -- 172.21.15.139:0/2480596129 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8037070 con 0x7fedd40d0870 2024-09-16T00:07:59.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.984+0000 7fedd1ffb640 1 -- 172.21.15.139:0/2480596129 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc802ae50 con 0x7fedd40d0870 2024-09-16T00:07:59.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.984+0000 7fedd1ffb640 1 -- 172.21.15.139:0/2480596129 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8032860 con 0x7fedd40d0870 2024-09-16T00:07:59.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.985+0000 7fedd9398640 1 -- 172.21.15.139:0/2480596129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd40d0870 msgr2=0x7fedd40b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:59.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.985+0000 7fedd9398640 1 --2- 172.21.15.139:0/2480596129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd40d0870 0x7fedd40b4040 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fedd40cddf0 tx=0x7fedc802a300 comp rx=0 tx=0).stop 2024-09-16T00:07:59.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.985+0000 7fedd9398640 1 -- 172.21.15.139:0/2480596129 shutdown_connections 2024-09-16T00:07:59.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.985+0000 7fedd9398640 1 --2- 172.21.15.139:0/2480596129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd40d0870 0x7fedd40b4040 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:59.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.985+0000 7fedd9398640 1 --2- 172.21.15.139:0/2480596129 >> v2:172.21.15.164:3300/0 conn(0x7fedd40d0490 0x7fedd40b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:59.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.985+0000 7fedd9398640 1 -- 172.21.15.139:0/2480596129 >> 172.21.15.139:0/2480596129 conn(0x7fedd406c520 msgr2=0x7fedd406c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:07:59.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.985+0000 7fedd9398640 1 -- 172.21.15.139:0/2480596129 shutdown_connections 2024-09-16T00:07:59.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.985+0000 7fedd9398640 1 -- 172.21.15.139:0/2480596129 wait complete. 2024-09-16T00:07:59.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.986+0000 7fedd9398640 1 Processor -- start 2024-09-16T00:07:59.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.986+0000 7fedd9398640 1 -- start start 2024-09-16T00:07:59.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd9398640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fedd40b3430 0x7fedd41e4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:59.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd9398640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd40d0490 0x7fedd4155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:59.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd9398640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd40d0870 0x7fedd4155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:59.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd9398640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fedd415b660 con 0x7fedd40d0490 2024-09-16T00:07:59.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd9398640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fedd415b380 con 0x7fedd40d0870 2024-09-16T00:07:59.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd9398640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fedd415a980 con 0x7fedd40b3430 2024-09-16T00:07:59.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd40d0490 0x7fedd4155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:59.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd27fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd40d0490 0x7fedd4155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58918/0 (socket says 172.21.15.139:58918) 2024-09-16T00:07:59.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd27fc640 1 -- 172.21.15.139:0/3721039386 learned_addr learned my addr 172.21.15.139:0/3721039386 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:07:59.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd37fe640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd40d0870 0x7fedd4155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:59.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd2ffd640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fedd40b3430 0x7fedd41e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:59.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd27fc640 1 -- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fedd40b3430 msgr2=0x7fedd41e4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:59.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd27fc640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fedd40b3430 0x7fedd41e4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:59.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd27fc640 1 -- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd40d0870 msgr2=0x7fedd4155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:07:59.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd27fc640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fedd40d0870 0x7fedd4155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:07:59.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd27fc640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd41e5d30 con 0x7fedd40d0490 2024-09-16T00:07:59.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd2ffd640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fedd40b3430 0x7fedd41e4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:07:59.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedd27fc640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd40d0490 0x7fedd4155980 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fedc802a340 tx=0x7fedc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:59.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.987+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8041070 con 0x7fedd40d0490 2024-09-16T00:07:59.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.988+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc803c660 con 0x7fedd40d0490 2024-09-16T00:07:59.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.988+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd4157230 con 0x7fedd40d0490 2024-09-16T00:07:59.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.988+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8037040 con 0x7fedd40d0490 2024-09-16T00:07:59.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.988+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd41577f0 con 0x7fedd40d0490 2024-09-16T00:07:59.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.988+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fedd40b48f0 con 0x7fedd40d0490 2024-09-16T00:07:59.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.989+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fedc8044070 con 0x7fedd40d0490 2024-09-16T00:07:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.989+0000 7fedbb7fe640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fedc006c2b0 0x7fedc006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.989+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5738+0+0 (secure 0 0 0) 0x7fedc80ab9a0 con 0x7fedd40d0490 2024-09-16T00:07:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.989+0000 7fedd2ffd640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fedc006c2b0 0x7fedc006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.990+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v27) ==== 54+0+16 (secure 0 0 0) 0x7fedd40b48f0 con 0x7fedd40d0490 2024-09-16T00:07:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.990+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fedd4154450 0x7fedd41e62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:07:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.990+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fedd40b2c40 con 0x7fedd4154450 2024-09-16T00:07:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.990+0000 7fedd2ffd640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fedc006c2b0 0x7fedc006e770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fedc4009b90 tx=0x7fedc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.990+0000 7fedd37fe640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fedd4154450 0x7fedd41e62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:07:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.990+0000 7fedd37fe640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fedd4154450 0x7fedd41e62a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:07:59.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:07:59.991+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fedd40b2c40 con 0x7fedd4154450 2024-09-16T00:08:00.001 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:07:59.999+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:00.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.005+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fedd4159850 con 0x7fedd4154450 2024-09-16T00:08:00.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.007+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fedd4159850 con 0x7fedd4154450 2024-09-16T00:08:00.008 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:00.009 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:00.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.007+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fedd4154830 0x7fedd41f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:00.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.007+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fedd4159570 con 0x7fedd4154830 2024-09-16T00:08:00.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.007+0000 7fedd2ffd640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fedd4154830 0x7fedd41f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:00.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.007+0000 7fedd2ffd640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fedd4154830 0x7fedd41f3ec0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:00.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.008+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fedd4159570 con 0x7fedd4154830 2024-09-16T00:08:00.018 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:00.018 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:00.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.022+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fedd41f4650 con 0x7fedd4154830 2024-09-16T00:08:00.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.023+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fedd41f4650 con 0x7fedd4154830 2024-09-16T00:08:00.025 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:00.029 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:00.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.023+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fedd414a540 0x7fedd414a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:00.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.023+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fedd40b45a0 con 0x7fedd414a540 2024-09-16T00:08:00.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.024+0000 7fedd27fc640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fedd414a540 0x7fedd414a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:00.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.024+0000 7fedd27fc640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fedd414a540 0x7fedd414a940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:00.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.024+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fedd40b45a0 con 0x7fedd414a540 2024-09-16T00:08:00.032 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:08:00.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.038+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fedd40b42c0 con 0x7fedd414a540 2024-09-16T00:08:00.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.040+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fedd40b42c0 con 0x7fedd414a540 2024-09-16T00:08:00.042 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:00.043 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:00.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.040+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fedd4062b40 0x7fedd4062f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:00.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.040+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fedd40634a0 con 0x7fedd4062b40 2024-09-16T00:08:00.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.040+0000 7fedd37fe640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fedd4062b40 0x7fedd4062f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:00.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.040+0000 7fedd37fe640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fedd4062b40 0x7fedd4062f60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:00.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.041+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fedd40634a0 con 0x7fedd4062b40 2024-09-16T00:08:00.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.054+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fedd40634a0 con 0x7fedd4062b40 2024-09-16T00:08:00.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.056+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fedd40634a0 con 0x7fedd4062b40 2024-09-16T00:08:00.058 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:00.059 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:00.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.056+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fedd414f3b0 0x7fedd402c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:00.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.056+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fedd414f7f0 con 0x7fedd414f3b0 2024-09-16T00:08:00.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.056+0000 7fedd2ffd640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fedd414f3b0 0x7fedd402c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:00.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.057+0000 7fedd2ffd640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fedd414f3b0 0x7fedd402c640 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:00.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.057+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fedd414f7f0 con 0x7fedd414f3b0 2024-09-16T00:08:00.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.071+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fedd414f7f0 con 0x7fedd414f3b0 2024-09-16T00:08:00.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.073+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fedd414f7f0 con 0x7fedd414f3b0 2024-09-16T00:08:00.074 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:00.075 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:00.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.073+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fedd402d090 0x7fedd402d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:00.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.073+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fedd402da30 con 0x7fedd402d090 2024-09-16T00:08:00.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.073+0000 7fedd27fc640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fedd402d090 0x7fedd402d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:00.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.073+0000 7fedd27fc640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fedd402d090 0x7fedd402d4f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:00.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.074+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fedd402da30 con 0x7fedd402d090 2024-09-16T00:08:00.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.088+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fedd402da30 con 0x7fedd402d090 2024-09-16T00:08:00.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.090+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fedd402da30 con 0x7fedd402d090 2024-09-16T00:08:00.092 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:00.093 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:00.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.091+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fedd42077c0 0x7fedd4209bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:00.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.091+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fedd420ca50 con 0x7fedd42077c0 2024-09-16T00:08:00.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.091+0000 7fedd37fe640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fedd42077c0 0x7fedd4209bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:00.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.091+0000 7fedd37fe640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fedd42077c0 0x7fedd4209bb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:00.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.092+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fedd420ca50 con 0x7fedd42077c0 2024-09-16T00:08:00.101 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:00.099+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:00.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.106+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fedd4000950 con 0x7fedd42077c0 2024-09-16T00:08:00.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.108+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fedd4000950 con 0x7fedd42077c0 2024-09-16T00:08:00.110 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:00.111 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:00.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.108+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fedd420a600 0x7fedd420aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:00.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.109+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fedd420fe70 con 0x7fedd420a600 2024-09-16T00:08:00.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.109+0000 7fedd2ffd640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fedd420a600 0x7fedd420aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:00.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.109+0000 7fedd2ffd640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fedd420a600 0x7fedd420aa60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:00.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.110+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fedd420fe70 con 0x7fedd420a600 2024-09-16T00:08:00.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.123+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fedd420b0b0 con 0x7fedd420a600 2024-09-16T00:08:00.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedbb7fe640 1 -- 172.21.15.139:0/3721039386 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fedd420b0b0 con 0x7fedd420a600 2024-09-16T00:08:00.127 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:00.128 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:00.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fedd4154450 msgr2=0x7fedd41e62a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:00.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fedd4154450 0x7fedd41e62a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:00.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fedd4154830 msgr2=0x7fedd41f3ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:00.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fedd4154830 0x7fedd41f3ec0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fedd414a540 msgr2=0x7fedd414a940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fedd414a540 0x7fedd414a940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fedd4062b40 msgr2=0x7fedd4062f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fedd4062b40 0x7fedd4062f60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fedd414f3b0 msgr2=0x7fedd402c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fedd414f3b0 0x7fedd402c640 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fedd402d090 msgr2=0x7fedd402d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fedd402d090 0x7fedd402d4f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fedd42077c0 msgr2=0x7fedd4209bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fedd42077c0 0x7fedd4209bb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fedd420a600 msgr2=0x7fedd420aa60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fedd420a600 0x7fedd420aa60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:00.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd37fe640 1 -- 172.21.15.139:0/3721039386 reap_dead start 2024-09-16T00:08:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd37fe640 1 -- 172.21.15.139:0/3721039386 reap_dead start 2024-09-16T00:08:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd37fe640 1 -- 172.21.15.139:0/3721039386 reap_dead start 2024-09-16T00:08:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd37fe640 1 -- 172.21.15.139:0/3721039386 reap_dead start 2024-09-16T00:08:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd37fe640 1 -- 172.21.15.139:0/3721039386 reap_dead start 2024-09-16T00:08:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fedc006c2b0 msgr2=0x7fedc006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fedc006c2b0 0x7fedc006e770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fedc4009b90 tx=0x7fedc4008040 comp rx=0 tx=0).stop 2024-09-16T00:08:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd40d0490 msgr2=0x7fedd4155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd40d0490 0x7fedd4155980 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fedc802a340 tx=0x7fedc80046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 shutdown_connections 2024-09-16T00:08:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fedc006c2b0 0x7fedc006e770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 --2- 172.21.15.139:0/3721039386 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedd40d0490 0x7fedd4155980 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 >> 172.21.15.139:0/3721039386 conn(0x7fedd406c520 msgr2=0x7fedd40b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 shutdown_connections 2024-09-16T00:08:00.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:00.126+0000 7fedd9398640 1 -- 172.21.15.139:0/3721039386 wait complete. 2024-09-16T00:08:00.142 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.142 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.059570762, 2024-09-16T00:08:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:08:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:08:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:08:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:08:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:08:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:08:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:08:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:08:00.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:08:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:08:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:08:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:08:00.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:08:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:08:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:08:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:08:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:08:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:08:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:08:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:08:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:08:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:08:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:08:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:08:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:08:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:08:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:08:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:08:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:08:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.980039132000002, 2024-09-16T00:08:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093071190000000008, 2024-09-16T00:08:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.153 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 106 2024-09-16T00:08:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:00.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160362+0000", 2024-09-16T00:08:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160438+0000", 2024-09-16T00:08:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6711000000000005e-05 2024-09-16T00:08:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160444+0000", 2024-09-16T00:08:00.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0530000000000003e-06 2024-09-16T00:08:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160449+0000", 2024-09-16T00:08:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2189999999999998e-06 2024-09-16T00:08:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161270+0000", 2024-09-16T00:08:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082099199999999999 2024-09-16T00:08:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161348+0000", 2024-09-16T00:08:00.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8361000000000004e-05 2024-09-16T00:08:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161472+0000", 2024-09-16T00:08:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123523 2024-09-16T00:08:00.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163461+0000", 2024-09-16T00:08:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019891560000000002 2024-09-16T00:08:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169418+0000", 2024-09-16T00:08:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059568490000000002 2024-09-16T00:08:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169570+0000", 2024-09-16T00:08:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152461 2024-09-16T00:08:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169668+0000", 2024-09-16T00:08:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T00:08:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.979943360999997, 2024-09-16T00:08:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092931679999999992, 2024-09-16T00:08:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.163 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107 2024-09-16T00:08:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0699999999999998e-07 2024-09-16T00:08:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160459+0000", 2024-09-16T00:08:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0370000000000001e-06 2024-09-16T00:08:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160463+0000", 2024-09-16T00:08:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999999e-06 2024-09-16T00:08:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160466+0000", 2024-09-16T00:08:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.644e-06 2024-09-16T00:08:00.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161873+0000", 2024-09-16T00:08:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014074840000000001 2024-09-16T00:08:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161949+0000", 2024-09-16T00:08:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5749999999999998e-05 2024-09-16T00:08:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162069+0000", 2024-09-16T00:08:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120257 2024-09-16T00:08:00.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163506+0000", 2024-09-16T00:08:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001436344 2024-09-16T00:08:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169704+0000", 2024-09-16T00:08:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061982829999999997 2024-09-16T00:08:00.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169730+0000", 2024-09-16T00:08:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6060000000000001e-05 2024-09-16T00:08:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169750+0000", 2024-09-16T00:08:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0092e-05 2024-09-16T00:08:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.979834048999997, 2024-09-16T00:08:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092280039999999997, 2024-09-16T00:08:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.170 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 108 2024-09-16T00:08:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160567+0000", 2024-09-16T00:08:00.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7300000000000001e-07 2024-09-16T00:08:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160569+0000", 2024-09-16T00:08:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5299999999999999e-06 2024-09-16T00:08:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160574+0000", 2024-09-16T00:08:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-16T00:08:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160577+0000", 2024-09-16T00:08:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.01e-06 2024-09-16T00:08:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162518+0000", 2024-09-16T00:08:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001940703 2024-09-16T00:08:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162571+0000", 2024-09-16T00:08:00.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2670000000000002e-05 2024-09-16T00:08:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162647+0000", 2024-09-16T00:08:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6778000000000002e-05 2024-09-16T00:08:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163544+0000", 2024-09-16T00:08:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089702599999999999 2024-09-16T00:08:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169754+0000", 2024-09-16T00:08:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062091739999999996 2024-09-16T00:08:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169776+0000", 2024-09-16T00:08:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2391999999999998e-05 2024-09-16T00:08:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169794+0000", 2024-09-16T00:08:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8318999999999999e-05 2024-09-16T00:08:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.176 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.979785499000002, 2024-09-16T00:08:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092220679999999999, 2024-09-16T00:08:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.177 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 109 2024-09-16T00:08:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:00.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9900000000000001e-07 2024-09-16T00:08:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160707+0000", 2024-09-16T00:08:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1621999999999998e-05 2024-09-16T00:08:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160713+0000", 2024-09-16T00:08:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.596e-06 2024-09-16T00:08:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160717+0000", 2024-09-16T00:08:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5829999999999998e-06 2024-09-16T00:08:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162869+0000", 2024-09-16T00:08:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021516310000000002 2024-09-16T00:08:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162946+0000", 2024-09-16T00:08:00.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7696999999999998e-05 2024-09-16T00:08:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163065+0000", 2024-09-16T00:08:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118581 2024-09-16T00:08:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164431+0000", 2024-09-16T00:08:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001366003 2024-09-16T00:08:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169797+0000", 2024-09-16T00:08:00.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005366244 2024-09-16T00:08:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169819+0000", 2024-09-16T00:08:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2036999999999999e-05 2024-09-16T00:08:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169837+0000", 2024-09-16T00:08:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7575e-05 2024-09-16T00:08:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.979676150000003, 2024-09-16T00:08:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092871919999999997, 2024-09-16T00:08:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 110 2024-09-16T00:08:00.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:08:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160728+0000", 2024-09-16T00:08:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0620000000000001e-06 2024-09-16T00:08:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160731+0000", 2024-09-16T00:08:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6500000000000002e-06 2024-09-16T00:08:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160734+0000", 2024-09-16T00:08:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.762e-06 2024-09-16T00:08:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163580+0000", 2024-09-16T00:08:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028464549999999999 2024-09-16T00:08:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163657+0000", 2024-09-16T00:08:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6695999999999999e-05 2024-09-16T00:08:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163798+0000", 2024-09-16T00:08:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014055 2024-09-16T00:08:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165483+0000", 2024-09-16T00:08:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016853090000000001 2024-09-16T00:08:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169970+0000", 2024-09-16T00:08:00.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00448709 2024-09-16T00:08:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169994+0000", 2024-09-16T00:08:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3575999999999999e-05 2024-09-16T00:08:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170011+0000", 2024-09-16T00:08:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7702000000000001e-05 2024-09-16T00:08:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.189 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.979662480999998, 2024-09-16T00:08:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093174859999999998, 2024-09-16T00:08:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.190 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 111 2024-09-16T00:08:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1e-07 2024-09-16T00:08:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160741+0000", 2024-09-16T00:08:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6110000000000001e-06 2024-09-16T00:08:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160744+0000", 2024-09-16T00:08:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3299999999999999e-06 2024-09-16T00:08:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160746+0000", 2024-09-16T00:08:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5469999999999999e-06 2024-09-16T00:08:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164069+0000", 2024-09-16T00:08:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033222709999999999 2024-09-16T00:08:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164113+0000", 2024-09-16T00:08:00.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4212999999999998e-05 2024-09-16T00:08:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164187+0000", 2024-09-16T00:08:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4136999999999998e-05 2024-09-16T00:08:00.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166429+0000", 2024-09-16T00:08:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022423180000000001 2024-09-16T00:08:00.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170016+0000", 2024-09-16T00:08:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035869449999999998 2024-09-16T00:08:00.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170038+0000", 2024-09-16T00:08:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2053e-05 2024-09-16T00:08:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170055+0000", 2024-09-16T00:08:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6850999999999998e-05 2024-09-16T00:08:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.203 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.979648765999997, 2024-09-16T00:08:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093480279999999995, 2024-09-16T00:08:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.204 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 112 2024-09-16T00:08:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160752+0000", 2024-09-16T00:08:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2700000000000001e-07 2024-09-16T00:08:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160808+0000", 2024-09-16T00:08:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5797999999999997e-05 2024-09-16T00:08:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160814+0000", 2024-09-16T00:08:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5980000000000002e-06 2024-09-16T00:08:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160819+0000", 2024-09-16T00:08:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8960000000000002e-06 2024-09-16T00:08:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164468+0000", 2024-09-16T00:08:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003649476 2024-09-16T00:08:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164525+0000", 2024-09-16T00:08:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6227999999999998e-05 2024-09-16T00:08:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164595+0000", 2024-09-16T00:08:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759000000000005e-05 2024-09-16T00:08:00.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167237+0000", 2024-09-16T00:08:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026410470000000001 2024-09-16T00:08:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170060+0000", 2024-09-16T00:08:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028234969999999999 2024-09-16T00:08:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170081+0000", 2024-09-16T00:08:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1242000000000001e-05 2024-09-16T00:08:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170100+0000", 2024-09-16T00:08:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8260000000000001e-05 2024-09-16T00:08:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.213 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.979571671000002, 2024-09-16T00:08:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093139409999999992, 2024-09-16T00:08:00.214 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.215 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.215 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.215 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 113 2024-09-16T00:08:00.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.215 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:00.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:00.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:00.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-07 2024-09-16T00:08:00.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160833+0000", 2024-09-16T00:08:00.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.557e-06 2024-09-16T00:08:00.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160839+0000", 2024-09-16T00:08:00.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3829999999999997e-06 2024-09-16T00:08:00.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160844+0000", 2024-09-16T00:08:00.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6909999999999996e-06 2024-09-16T00:08:00.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165008+0000", 2024-09-16T00:08:00.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041642129999999999 2024-09-16T00:08:00.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165085+0000", 2024-09-16T00:08:00.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6651999999999997e-05 2024-09-16T00:08:00.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165205+0000", 2024-09-16T00:08:00.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012045900000000001 2024-09-16T00:08:00.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167283+0000", 2024-09-16T00:08:00.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020782909999999999 2024-09-16T00:08:00.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170104+0000", 2024-09-16T00:08:00.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028207950000000001 2024-09-16T00:08:00.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170125+0000", 2024-09-16T00:08:00.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0852e-05 2024-09-16T00:08:00.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170143+0000", 2024-09-16T00:08:00.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7467000000000001e-05 2024-09-16T00:08:00.223 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.223 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.223 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.223 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:00.223 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:00.223 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.979525189, 2024-09-16T00:08:00.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093126960000000005, 2024-09-16T00:08:00.224 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.224 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.224 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.224 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.224 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.225 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.225 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-16T00:08:00.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.225 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160876+0000", 2024-09-16T00:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5700000000000002e-07 2024-09-16T00:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160880+0000", 2024-09-16T00:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5539999999999999e-06 2024-09-16T00:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160888+0000", 2024-09-16T00:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.157e-06 2024-09-16T00:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160894+0000", 2024-09-16T00:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7690000000000002e-06 2024-09-16T00:08:00.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165519+0000", 2024-09-16T00:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046249539999999997 2024-09-16T00:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165593+0000", 2024-09-16T00:08:00.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4039e-05 2024-09-16T00:08:00.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165715+0000", 2024-09-16T00:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012131899999999999 2024-09-16T00:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167309+0000", 2024-09-16T00:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001594011 2024-09-16T00:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170147+0000", 2024-09-16T00:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028387130000000001 2024-09-16T00:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170170+0000", 2024-09-16T00:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2517000000000001e-05 2024-09-16T00:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170188+0000", 2024-09-16T00:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8006e-05 2024-09-16T00:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.234 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.979484161000002, 2024-09-16T00:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093140040000000007, 2024-09-16T00:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-16T00:08:00.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160917+0000", 2024-09-16T00:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7599999999999996e-07 2024-09-16T00:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160992+0000", 2024-09-16T00:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4760999999999998e-05 2024-09-16T00:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161000+0000", 2024-09-16T00:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9829999999999996e-06 2024-09-16T00:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161007+0000", 2024-09-16T00:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4909999999999999e-06 2024-09-16T00:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165983+0000", 2024-09-16T00:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049755399999999997 2024-09-16T00:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166027+0000", 2024-09-16T00:08:00.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4109000000000001e-05 2024-09-16T00:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166100+0000", 2024-09-16T00:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3360000000000002e-05 2024-09-16T00:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169698+0000", 2024-09-16T00:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597795 2024-09-16T00:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170192+0000", 2024-09-16T00:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049447300000000005 2024-09-16T00:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170214+0000", 2024-09-16T00:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1546e-05 2024-09-16T00:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170230+0000", 2024-09-16T00:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6169999999999999e-05 2024-09-16T00:08:00.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.241 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.979374432, 2024-09-16T00:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092457029999999992, 2024-09-16T00:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-16T00:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7899999999999999e-07 2024-09-16T00:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161029+0000", 2024-09-16T00:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6630000000000002e-06 2024-09-16T00:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161036+0000", 2024-09-16T00:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T00:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161041+0000", 2024-09-16T00:08:00.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166468+0000", 2024-09-16T00:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054266050000000001 2024-09-16T00:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166511+0000", 2024-09-16T00:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3869999999999998e-05 2024-09-16T00:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166606+0000", 2024-09-16T00:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4437000000000004e-05 2024-09-16T00:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169955+0000", 2024-09-16T00:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033494810000000001 2024-09-16T00:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170234+0000", 2024-09-16T00:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027900899999999998 2024-09-16T00:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170255+0000", 2024-09-16T00:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0537999999999999e-05 2024-09-16T00:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170272+0000", 2024-09-16T00:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6611000000000001e-05 2024-09-16T00:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.247 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.979349286999998, 2024-09-16T00:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093511010000000006, 2024-09-16T00:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-16T00:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.63e-07 2024-09-16T00:08:00.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161055+0000", 2024-09-16T00:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6770000000000001e-06 2024-09-16T00:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161063+0000", 2024-09-16T00:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6469999999999998e-06 2024-09-16T00:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161068+0000", 2024-09-16T00:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.746e-06 2024-09-16T00:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166817+0000", 2024-09-16T00:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057494959999999998 2024-09-16T00:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166893+0000", 2024-09-16T00:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6210000000000004e-05 2024-09-16T00:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167019+0000", 2024-09-16T00:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125585 2024-09-16T00:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170275+0000", 2024-09-16T00:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032563620000000001 2024-09-16T00:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170309+0000", 2024-09-16T00:08:00.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3865999999999997e-05 2024-09-16T00:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170362+0000", 2024-09-16T00:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3094999999999999e-05 2024-09-16T00:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170402+0000", 2024-09-16T00:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9953999999999997e-05 2024-09-16T00:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.530617202000002, 2024-09-16T00:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097364389999999995, 2024-09-16T00:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138 2024-09-16T00:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.254 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609784+0000", 2024-09-16T00:08:00.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T00:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609787+0000", 2024-09-16T00:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280000000000001e-06 2024-09-16T00:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609794+0000", 2024-09-16T00:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6409999999999996e-06 2024-09-16T00:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609799+0000", 2024-09-16T00:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9359999999999998e-06 2024-09-16T00:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615759+0000", 2024-09-16T00:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059599910000000004 2024-09-16T00:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615806+0000", 2024-09-16T00:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7290000000000003e-05 2024-09-16T00:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615877+0000", 2024-09-16T00:08:00.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1000000000000005e-05 2024-09-16T00:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618961+0000", 2024-09-16T00:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030841390000000001 2024-09-16T00:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619436+0000", 2024-09-16T00:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047452799999999999 2024-09-16T00:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619482+0000", 2024-09-16T00:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5689000000000003e-05 2024-09-16T00:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619519+0000", 2024-09-16T00:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7938000000000002e-05 2024-09-16T00:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.260 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.530593488000001, 2024-09-16T00:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011812954000000001, 2024-09-16T00:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.261 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139 2024-09-16T00:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999998e-07 2024-09-16T00:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609810+0000", 2024-09-16T00:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2100000000000002e-06 2024-09-16T00:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609817+0000", 2024-09-16T00:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.28e-06 2024-09-16T00:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609825+0000", 2024-09-16T00:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616116+0000", 2024-09-16T00:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006290543 2024-09-16T00:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616187+0000", 2024-09-16T00:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1830999999999997e-05 2024-09-16T00:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616298+0000", 2024-09-16T00:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011044699999999999 2024-09-16T00:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619559+0000", 2024-09-16T00:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032614710000000002 2024-09-16T00:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621562+0000", 2024-09-16T00:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020028680000000001 2024-09-16T00:08:00.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621597+0000", 2024-09-16T00:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4625000000000001e-05 2024-09-16T00:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621620+0000", 2024-09-16T00:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2864e-05 2024-09-16T00:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.530565692000003, 2024-09-16T00:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011828211999999999, 2024-09-16T00:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 140 2024-09-16T00:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.268 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:00.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T00:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961000000000003e-05 2024-09-16T00:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609892+0000", 2024-09-16T00:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0516000000000001e-05 2024-09-16T00:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609898+0000", 2024-09-16T00:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0449999999999997e-06 2024-09-16T00:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616797+0000", 2024-09-16T00:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068998840000000002 2024-09-16T00:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616844+0000", 2024-09-16T00:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6501000000000001e-05 2024-09-16T00:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616935+0000", 2024-09-16T00:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1222000000000002e-05 2024-09-16T00:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619808+0000", 2024-09-16T00:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873198 2024-09-16T00:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621624+0000", 2024-09-16T00:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001815933 2024-09-16T00:08:00.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621646+0000", 2024-09-16T00:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1622999999999999e-05 2024-09-16T00:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621663+0000", 2024-09-16T00:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6854000000000002e-05 2024-09-16T00:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.530479882999998, 2024-09-16T00:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014331501999999999, 2024-09-16T00:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-16T00:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.274 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609921+0000", 2024-09-16T00:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9599999999999999e-07 2024-09-16T00:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609925+0000", 2024-09-16T00:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T00:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609932+0000", 2024-09-16T00:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8240000000000002e-06 2024-09-16T00:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609937+0000", 2024-09-16T00:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1070000000000004e-06 2024-09-16T00:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618987+0000", 2024-09-16T00:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090502719999999998 2024-09-16T00:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619064+0000", 2024-09-16T00:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6868999999999999e-05 2024-09-16T00:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619183+0000", 2024-09-16T00:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011961 2024-09-16T00:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.620052+0000", 2024-09-16T00:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086822799999999997 2024-09-16T00:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624197+0000", 2024-09-16T00:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041454930000000001 2024-09-16T00:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624229+0000", 2024-09-16T00:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1758999999999999e-05 2024-09-16T00:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624252+0000", 2024-09-16T00:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:08:00.279 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.057714075, 2024-09-16T00:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010093276999999999, 2024-09-16T00:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.280 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 157 2024-09-16T00:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4699999999999998e-07 2024-09-16T00:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082688+0000", 2024-09-16T00:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0209999999999998e-06 2024-09-16T00:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082692+0000", 2024-09-16T00:08:00.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6890000000000002e-06 2024-09-16T00:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082695+0000", 2024-09-16T00:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5339999999999998e-06 2024-09-16T00:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085303+0000", 2024-09-16T00:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026084789999999999 2024-09-16T00:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085374+0000", 2024-09-16T00:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1317999999999997e-05 2024-09-16T00:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085506+0000", 2024-09-16T00:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013118 2024-09-16T00:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086928+0000", 2024-09-16T00:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014224559999999999 2024-09-16T00:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092665+0000", 2024-09-16T00:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057369830000000002 2024-09-16T00:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092727+0000", 2024-09-16T00:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1810000000000006e-05 2024-09-16T00:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092779+0000", 2024-09-16T00:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2559999999999998e-05 2024-09-16T00:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.057701907000002, 2024-09-16T00:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010176388, 2024-09-16T00:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.286 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 158 2024-09-16T00:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082699+0000", 2024-09-16T00:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.11e-07 2024-09-16T00:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082742+0000", 2024-09-16T00:08:00.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3699000000000003e-05 2024-09-16T00:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082748+0000", 2024-09-16T00:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8690000000000001e-06 2024-09-16T00:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082751+0000", 2024-09-16T00:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.942e-06 2024-09-16T00:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085881+0000", 2024-09-16T00:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031303450000000001 2024-09-16T00:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085959+0000", 2024-09-16T00:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7874999999999996e-05 2024-09-16T00:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086086+0000", 2024-09-16T00:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012685499999999999 2024-09-16T00:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087127+0000", 2024-09-16T00:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001041298 2024-09-16T00:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092790+0000", 2024-09-16T00:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056628 2024-09-16T00:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092832+0000", 2024-09-16T00:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2227000000000001e-05 2024-09-16T00:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092875+0000", 2024-09-16T00:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2267e-05 2024-09-16T00:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.057643869000003, 2024-09-16T00:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010199547, 2024-09-16T00:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:00.292 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159 2024-09-16T00:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082757+0000", 2024-09-16T00:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2500000000000001e-07 2024-09-16T00:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082759+0000", 2024-09-16T00:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-16T00:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082763+0000", 2024-09-16T00:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4889999999999997e-06 2024-09-16T00:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082766+0000", 2024-09-16T00:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086571+0000", 2024-09-16T00:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038053269999999998 2024-09-16T00:08:00.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086619+0000", 2024-09-16T00:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7924999999999999e-05 2024-09-16T00:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086693+0000", 2024-09-16T00:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000005e-05 2024-09-16T00:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087383+0000", 2024-09-16T00:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069008299999999997 2024-09-16T00:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092882+0000", 2024-09-16T00:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054987450000000002 2024-09-16T00:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092921+0000", 2024-09-16T00:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9388999999999998e-05 2024-09-16T00:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:00.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092956+0000", 2024-09-16T00:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4647999999999998e-05 2024-09-16T00:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:00.298 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:00.299 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:08:00.300 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:00.298+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:00.400 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:00.398+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:00.403 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:00.403 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:00.403 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:00.403 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:00.412 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:08:00.501 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:00.500+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:00.556 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:00.556 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:00.556 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:00.556 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:00.556 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:00.565 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:08:00.607 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:00.605+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:00.703 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:00.703+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:00.711 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:00.712 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:00.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:00.712 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:00.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.712 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'41, mlcod=17'39)", 2024-09-16T00:08:00.712 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:00.713 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.688855552, 2024-09-16T00:08:00.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001808488, 2024-09-16T00:08:00.713 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.713 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.713 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.713 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:00.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:00.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021032+0000", 2024-09-16T00:08:00.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3e-07 2024-09-16T00:08:00.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021037+0000", 2024-09-16T00:08:00.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0640000000000001e-06 2024-09-16T00:08:00.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021047+0000", 2024-09-16T00:08:00.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.023e-05 2024-09-16T00:08:00.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021053+0000", 2024-09-16T00:08:00.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3160000000000002e-06 2024-09-16T00:08:00.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021092+0000", 2024-09-16T00:08:00.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9033999999999999e-05 2024-09-16T00:08:00.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021105+0000", 2024-09-16T00:08:00.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2768e-05 2024-09-16T00:08:00.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022811+0000", 2024-09-16T00:08:00.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001705788 2024-09-16T00:08:00.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022840+0000", 2024-09-16T00:08:00.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8558e-05 2024-09-16T00:08:00.718 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.718 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.718 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.718 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:85 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'55, mlcod=17'50)", 2024-09-16T00:08:00.718 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:00.719 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.675826551, 2024-09-16T00:08:00.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017909530000000001, 2024-09-16T00:08:00.719 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.719 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.719 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.719 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:00.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:00.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034061+0000", 2024-09-16T00:08:00.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0399999999999996e-07 2024-09-16T00:08:00.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034065+0000", 2024-09-16T00:08:00.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8840000000000002e-06 2024-09-16T00:08:00.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034074+0000", 2024-09-16T00:08:00.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:00.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034079+0000", 2024-09-16T00:08:00.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0000000000000004e-06 2024-09-16T00:08:00.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034125+0000", 2024-09-16T00:08:00.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5905000000000002e-05 2024-09-16T00:08:00.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034144+0000", 2024-09-16T00:08:00.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9593e-05 2024-09-16T00:08:00.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035817+0000", 2024-09-16T00:08:00.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016728240000000001 2024-09-16T00:08:00.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035851+0000", 2024-09-16T00:08:00.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3809000000000001e-05 2024-09-16T00:08:00.722 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.722 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.723 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.723 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:112 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'74, mlcod=17'65)", 2024-09-16T00:08:00.723 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:00.723 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.545214688999998, 2024-09-16T00:08:00.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018457020000000001, 2024-09-16T00:08:00.723 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.723 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.723 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.723 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:00.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:00.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164673+0000", 2024-09-16T00:08:00.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1200000000000003e-07 2024-09-16T00:08:00.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164710+0000", 2024-09-16T00:08:00.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7756999999999998e-05 2024-09-16T00:08:00.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164719+0000", 2024-09-16T00:08:00.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2849999999999995e-06 2024-09-16T00:08:00.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164723+0000", 2024-09-16T00:08:00.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3220000000000003e-06 2024-09-16T00:08:00.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164760+0000", 2024-09-16T00:08:00.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7311999999999998e-05 2024-09-16T00:08:00.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164771+0000", 2024-09-16T00:08:00.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0869e-05 2024-09-16T00:08:00.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166495+0000", 2024-09-16T00:08:00.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001723863 2024-09-16T00:08:00.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166518+0000", 2024-09-16T00:08:00.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2682000000000001e-05 2024-09-16T00:08:00.729 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.729 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.729 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.730 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'66)", 2024-09-16T00:08:00.730 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:00.730 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.543710709999999, 2024-09-16T00:08:00.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032925509999999999, 2024-09-16T00:08:00.730 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.730 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.730 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.730 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:00.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:00.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166177+0000", 2024-09-16T00:08:00.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:00.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166216+0000", 2024-09-16T00:08:00.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9008000000000003e-05 2024-09-16T00:08:00.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.732 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166226+0000", 2024-09-16T00:08:00.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0081999999999999e-05 2024-09-16T00:08:00.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.732 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166230+0000", 2024-09-16T00:08:00.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5079999999999999e-06 2024-09-16T00:08:00.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166259+0000", 2024-09-16T00:08:00.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8928000000000002e-05 2024-09-16T00:08:00.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166271+0000", 2024-09-16T00:08:00.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1531e-05 2024-09-16T00:08:00.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169449+0000", 2024-09-16T00:08:00.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003178143 2024-09-16T00:08:00.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169468+0000", 2024-09-16T00:08:00.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9700000000000001e-05 2024-09-16T00:08:00.734 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.734 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.734 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.734 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'66)", 2024-09-16T00:08:00.734 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:00.734 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.543249250999999, 2024-09-16T00:08:00.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003179374, 2024-09-16T00:08:00.734 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.735 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.735 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.735 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:00.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:00.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166638+0000", 2024-09-16T00:08:00.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2300000000000002e-07 2024-09-16T00:08:00.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166641+0000", 2024-09-16T00:08:00.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0079999999999998e-06 2024-09-16T00:08:00.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166646+0000", 2024-09-16T00:08:00.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4970000000000001e-06 2024-09-16T00:08:00.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166650+0000", 2024-09-16T00:08:00.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5729999999999999e-06 2024-09-16T00:08:00.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166682+0000", 2024-09-16T00:08:00.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2218999999999998e-05 2024-09-16T00:08:00.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166694+0000", 2024-09-16T00:08:00.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1902999999999999e-05 2024-09-16T00:08:00.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169784+0000", 2024-09-16T00:08:00.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003089545 2024-09-16T00:08:00.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169817+0000", 2024-09-16T00:08:00.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3206e-05 2024-09-16T00:08:00.738 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.738 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.739 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.739 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:117 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'66)", 2024-09-16T00:08:00.739 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:00.739 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.542805190999999, 2024-09-16T00:08:00.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00275326, 2024-09-16T00:08:00.739 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.739 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.739 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.739 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:00.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:00.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167082+0000", 2024-09-16T00:08:00.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:08:00.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167086+0000", 2024-09-16T00:08:00.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1559999999999997e-06 2024-09-16T00:08:00.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167096+0000", 2024-09-16T00:08:00.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8390000000000002e-06 2024-09-16T00:08:00.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167101+0000", 2024-09-16T00:08:00.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:08:00.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167130+0000", 2024-09-16T00:08:00.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9649e-05 2024-09-16T00:08:00.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167142+0000", 2024-09-16T00:08:00.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1435e-05 2024-09-16T00:08:00.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169818+0000", 2024-09-16T00:08:00.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002676056 2024-09-16T00:08:00.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169835+0000", 2024-09-16T00:08:00.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6980999999999999e-05 2024-09-16T00:08:00.743 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.743 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.743 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.744 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:130 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'88, mlcod=17'85)", 2024-09-16T00:08:00.744 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:00.744 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.100005862000003, 2024-09-16T00:08:00.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002880456, 2024-09-16T00:08:00.744 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.744 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.744 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.744 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:00.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:00.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:00.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5699999999999998e-07 2024-09-16T00:08:00.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609885+0000", 2024-09-16T00:08:00.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.608e-06 2024-09-16T00:08:00.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609891+0000", 2024-09-16T00:08:00.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:00.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609895+0000", 2024-09-16T00:08:00.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.54e-06 2024-09-16T00:08:00.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609931+0000", 2024-09-16T00:08:00.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5893999999999999e-05 2024-09-16T00:08:00.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609945+0000", 2024-09-16T00:08:00.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4259e-05 2024-09-16T00:08:00.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612721+0000", 2024-09-16T00:08:00.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027757849999999998 2024-09-16T00:08:00.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612761+0000", 2024-09-16T00:08:00.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0638e-05 2024-09-16T00:08:00.748 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.749 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.749 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.749 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-16T00:08:00.749 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:00.749 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.099621888999998, 2024-09-16T00:08:00.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028016529999999999, 2024-09-16T00:08:00.749 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.749 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.749 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.749 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:00.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:00.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:00.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.63e-07 2024-09-16T00:08:00.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610344+0000", 2024-09-16T00:08:00.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8996000000000001e-05 2024-09-16T00:08:00.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610352+0000", 2024-09-16T00:08:00.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:00.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610356+0000", 2024-09-16T00:08:00.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0790000000000002e-06 2024-09-16T00:08:00.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610388+0000", 2024-09-16T00:08:00.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1541000000000002e-05 2024-09-16T00:08:00.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610400+0000", 2024-09-16T00:08:00.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2075e-05 2024-09-16T00:08:00.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613045+0000", 2024-09-16T00:08:00.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002645213 2024-09-16T00:08:00.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613066+0000", 2024-09-16T00:08:00.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1353e-05 2024-09-16T00:08:00.753 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.753 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.753 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.753 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-16T00:08:00.753 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:00.754 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.096362388999999, 2024-09-16T00:08:00.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027254850000000001, 2024-09-16T00:08:00.754 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.754 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.754 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.754 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:00.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:00.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613525+0000", 2024-09-16T00:08:00.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2800000000000002e-07 2024-09-16T00:08:00.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613528+0000", 2024-09-16T00:08:00.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9730000000000002e-06 2024-09-16T00:08:00.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613534+0000", 2024-09-16T00:08:00.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4339999999999997e-06 2024-09-16T00:08:00.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613538+0000", 2024-09-16T00:08:00.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5149999999999998e-06 2024-09-16T00:08:00.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613566+0000", 2024-09-16T00:08:00.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8104000000000001e-05 2024-09-16T00:08:00.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613578+0000", 2024-09-16T00:08:00.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2404e-05 2024-09-16T00:08:00.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616217+0000", 2024-09-16T00:08:00.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026389830000000001 2024-09-16T00:08:00.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616250+0000", 2024-09-16T00:08:00.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2644000000000001e-05 2024-09-16T00:08:00.758 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.758 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.758 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.758 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:134 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'92, mlcod=18'87)", 2024-09-16T00:08:00.758 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:00.758 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.09582425, 2024-09-16T00:08:00.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025322399999999998, 2024-09-16T00:08:00.758 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.758 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.758 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.759 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:00.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:00.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614063+0000", 2024-09-16T00:08:00.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8700000000000005e-07 2024-09-16T00:08:00.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614096+0000", 2024-09-16T00:08:00.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3136000000000003e-05 2024-09-16T00:08:00.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614106+0000", 2024-09-16T00:08:00.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9170000000000001e-06 2024-09-16T00:08:00.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614111+0000", 2024-09-16T00:08:00.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4390000000000002e-06 2024-09-16T00:08:00.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614141+0000", 2024-09-16T00:08:00.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.04e-05 2024-09-16T00:08:00.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614152+0000", 2024-09-16T00:08:00.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0987e-05 2024-09-16T00:08:00.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616573+0000", 2024-09-16T00:08:00.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002421351 2024-09-16T00:08:00.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616595+0000", 2024-09-16T00:08:00.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1322999999999999e-05 2024-09-16T00:08:00.762 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.762 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.762 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.763 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:135 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'93, mlcod=18'87)", 2024-09-16T00:08:00.763 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:00.763 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.095304212000002, 2024-09-16T00:08:00.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020304659999999999, 2024-09-16T00:08:00.763 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.763 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.763 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.763 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.763 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.763 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:00.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.764 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.764 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:00.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.764 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.764 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.764 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614583+0000", 2024-09-16T00:08:00.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:00.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614587+0000", 2024-09-16T00:08:00.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3100000000000002e-06 2024-09-16T00:08:00.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614597+0000", 2024-09-16T00:08:00.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7450000000000007e-06 2024-09-16T00:08:00.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614602+0000", 2024-09-16T00:08:00.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6099999999999999e-06 2024-09-16T00:08:00.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614641+0000", 2024-09-16T00:08:00.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9767000000000001e-05 2024-09-16T00:08:00.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614658+0000", 2024-09-16T00:08:00.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6205000000000002e-05 2024-09-16T00:08:00.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616596+0000", 2024-09-16T00:08:00.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019386659999999999 2024-09-16T00:08:00.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616613+0000", 2024-09-16T00:08:00.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6555999999999999e-05 2024-09-16T00:08:00.767 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.767 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.767 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.767 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:139 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'97, mlcod=18'87)", 2024-09-16T00:08:00.767 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:00.768 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.093543541000003, 2024-09-16T00:08:00.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030908139999999999, 2024-09-16T00:08:00.768 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.768 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.768 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.768 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:00.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:00.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616344+0000", 2024-09-16T00:08:00.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3000000000000001e-07 2024-09-16T00:08:00.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616347+0000", 2024-09-16T00:08:00.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7740000000000002e-06 2024-09-16T00:08:00.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.770 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616355+0000", 2024-09-16T00:08:00.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4699999999999996e-06 2024-09-16T00:08:00.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.770 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.770 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616358+0000", 2024-09-16T00:08:00.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5899999999999999e-06 2024-09-16T00:08:00.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.770 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.770 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616393+0000", 2024-09-16T00:08:00.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4529000000000001e-05 2024-09-16T00:08:00.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616404+0000", 2024-09-16T00:08:00.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0726e-05 2024-09-16T00:08:00.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619411+0000", 2024-09-16T00:08:00.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030076249999999999 2024-09-16T00:08:00.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619434+0000", 2024-09-16T00:08:00.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-05 2024-09-16T00:08:00.772 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.772 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.772 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.772 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:140 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'98, mlcod=18'89)", 2024-09-16T00:08:00.772 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:00.772 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.092902148, 2024-09-16T00:08:00.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027276150000000001, 2024-09-16T00:08:00.772 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.772 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.772 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.772 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:00.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:00.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616985+0000", 2024-09-16T00:08:00.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4099999999999999e-07 2024-09-16T00:08:00.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617055+0000", 2024-09-16T00:08:00.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9923000000000001e-05 2024-09-16T00:08:00.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617063+0000", 2024-09-16T00:08:00.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:00.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617070+0000", 2024-09-16T00:08:00.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9090000000000003e-06 2024-09-16T00:08:00.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.775 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.775 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617102+0000", 2024-09-16T00:08:00.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2620999999999998e-05 2024-09-16T00:08:00.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.775 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.775 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617114+0000", 2024-09-16T00:08:00.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1228e-05 2024-09-16T00:08:00.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.775 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.776 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619690+0000", 2024-09-16T00:08:00.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025763890000000001 2024-09-16T00:08:00.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.776 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.776 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619712+0000", 2024-09-16T00:08:00.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2171000000000002e-05 2024-09-16T00:08:00.776 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.776 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.776 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.776 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:00.776 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:00.777 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.094073047999998, 2024-09-16T00:08:00.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027355320000000002, 2024-09-16T00:08:00.777 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.777 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.777 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.777 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:00.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:00.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615815+0000", 2024-09-16T00:08:00.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1769999999999999e-06 2024-09-16T00:08:00.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615825+0000", 2024-09-16T00:08:00.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9000000000000001e-06 2024-09-16T00:08:00.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615840+0000", 2024-09-16T00:08:00.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4854e-05 2024-09-16T00:08:00.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615848+0000", 2024-09-16T00:08:00.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0940000000000008e-06 2024-09-16T00:08:00.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615926+0000", 2024-09-16T00:08:00.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8176999999999994e-05 2024-09-16T00:08:00.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615945+0000", 2024-09-16T00:08:00.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9663000000000001e-05 2024-09-16T00:08:00.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618509+0000", 2024-09-16T00:08:00.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002563802 2024-09-16T00:08:00.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618549+0000", 2024-09-16T00:08:00.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9864999999999999e-05 2024-09-16T00:08:00.781 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.781 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.781 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.781 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:00.781 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:00.781 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.093983907999998, 2024-09-16T00:08:00.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061593860000000002, 2024-09-16T00:08:00.781 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.781 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.781 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.782 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.782 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.782 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.782 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:00.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.782 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.782 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.782 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:00.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.782 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.782 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.782 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615907+0000", 2024-09-16T00:08:00.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9389999999999999e-06 2024-09-16T00:08:00.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615920+0000", 2024-09-16T00:08:00.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2989999999999999e-05 2024-09-16T00:08:00.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615931+0000", 2024-09-16T00:08:00.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.164e-05 2024-09-16T00:08:00.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615939+0000", 2024-09-16T00:08:00.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8029999999999997e-06 2024-09-16T00:08:00.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616310+0000", 2024-09-16T00:08:00.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00037111400000000003 2024-09-16T00:08:00.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616406+0000", 2024-09-16T00:08:00.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5372999999999996e-05 2024-09-16T00:08:00.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:00.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616514+0000", 2024-09-16T00:08:00.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108564 2024-09-16T00:08:00.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:00.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618158+0000", 2024-09-16T00:08:00.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016442449999999999 2024-09-16T00:08:00.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621972+0000", 2024-09-16T00:08:00.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038132299999999999 2024-09-16T00:08:00.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.786 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.786 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622018+0000", 2024-09-16T00:08:00.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6239000000000002e-05 2024-09-16T00:08:00.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.786 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.786 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622062+0000", 2024-09-16T00:08:00.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4249000000000003e-05 2024-09-16T00:08:00.786 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.786 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.786 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.787 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:00.787 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:00.787 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.093930440000001, 2024-09-16T00:08:00.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006258914, 2024-09-16T00:08:00.787 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.787 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.787 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.787 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:00.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.788 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.788 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.788 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:00.788 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.788 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.788 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.788 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615957+0000", 2024-09-16T00:08:00.788 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1399999999999997e-07 2024-09-16T00:08:00.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.788 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.788 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.789 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615960+0000", 2024-09-16T00:08:00.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0860000000000002e-06 2024-09-16T00:08:00.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.789 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.789 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615967+0000", 2024-09-16T00:08:00.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9059999999999996e-06 2024-09-16T00:08:00.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.789 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.789 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615972+0000", 2024-09-16T00:08:00.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5679999999999999e-06 2024-09-16T00:08:00.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616717+0000", 2024-09-16T00:08:00.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074438 2024-09-16T00:08:00.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616781+0000", 2024-09-16T00:08:00.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4808999999999996e-05 2024-09-16T00:08:00.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:00.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616885+0000", 2024-09-16T00:08:00.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010404900000000001 2024-09-16T00:08:00.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:00.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:00.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012773870000000001 2024-09-16T00:08:00.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622116+0000", 2024-09-16T00:08:00.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039533470000000003 2024-09-16T00:08:00.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622173+0000", 2024-09-16T00:08:00.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6347000000000001e-05 2024-09-16T00:08:00.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622215+0000", 2024-09-16T00:08:00.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2521000000000002e-05 2024-09-16T00:08:00.792 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.792 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.792 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.792 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:00.792 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:00.793 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.093906702999998, 2024-09-16T00:08:00.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041725529999999999, 2024-09-16T00:08:00.793 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.793 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.793 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.793 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:00.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:00.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:00.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8799999999999998e-07 2024-09-16T00:08:00.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615990+0000", 2024-09-16T00:08:00.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2150000000000007e-06 2024-09-16T00:08:00.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.795 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.795 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:08:00.795 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7159999999999997e-06 2024-09-16T00:08:00.795 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.795 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.795 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.795 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616001+0000", 2024-09-16T00:08:00.795 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9339999999999997e-06 2024-09-16T00:08:00.795 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.795 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.795 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.795 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616293+0000", 2024-09-16T00:08:00.795 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029210899999999998 2024-09-16T00:08:00.795 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616401+0000", 2024-09-16T00:08:00.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108087 2024-09-16T00:08:00.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:00.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616561+0000", 2024-09-16T00:08:00.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015940600000000001 2024-09-16T00:08:00.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:00.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618146+0000", 2024-09-16T00:08:00.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015857040000000001 2024-09-16T00:08:00.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620091+0000", 2024-09-16T00:08:00.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001944837 2024-09-16T00:08:00.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620126+0000", 2024-09-16T00:08:00.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4242999999999997e-05 2024-09-16T00:08:00.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620152+0000", 2024-09-16T00:08:00.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6914e-05 2024-09-16T00:08:00.798 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.798 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.798 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.798 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:00.798 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:00.798 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.093868478000001, 2024-09-16T00:08:00.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00630628, 2024-09-16T00:08:00.798 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.798 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.798 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.799 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:00.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:00.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:08:00.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5999999999999999e-07 2024-09-16T00:08:00.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.800 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616022+0000", 2024-09-16T00:08:00.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0149999999999999e-06 2024-09-16T00:08:00.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.800 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616031+0000", 2024-09-16T00:08:00.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9770000000000006e-06 2024-09-16T00:08:00.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.801 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.801 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616036+0000", 2024-09-16T00:08:00.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2639999999999999e-06 2024-09-16T00:08:00.801 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.801 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.801 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.801 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617057+0000", 2024-09-16T00:08:00.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010212999999999999 2024-09-16T00:08:00.801 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.801 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.801 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.801 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617146+0000", 2024-09-16T00:08:00.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9234000000000007e-05 2024-09-16T00:08:00.801 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:00.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617232+0000", 2024-09-16T00:08:00.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5218999999999998e-05 2024-09-16T00:08:00.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:00.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619107+0000", 2024-09-16T00:08:00.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001875016 2024-09-16T00:08:00.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622259+0000", 2024-09-16T00:08:00.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031527930000000001 2024-09-16T00:08:00.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622290+0000", 2024-09-16T00:08:00.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1022999999999998e-05 2024-09-16T00:08:00.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622324+0000", 2024-09-16T00:08:00.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3979000000000001e-05 2024-09-16T00:08:00.803 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.803 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.803 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.804 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:00.804 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:00.804 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.093843176, 2024-09-16T00:08:00.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042412739999999997, 2024-09-16T00:08:00.804 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.804 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.804 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:00.804 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:00.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:00.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616044+0000", 2024-09-16T00:08:00.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:00.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616052+0000", 2024-09-16T00:08:00.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.439e-06 2024-09-16T00:08:00.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616059+0000", 2024-09-16T00:08:00.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1389999999999998e-06 2024-09-16T00:08:00.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.807 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616065+0000", 2024-09-16T00:08:00.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:08:00.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.807 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616823+0000", 2024-09-16T00:08:00.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075781199999999996 2024-09-16T00:08:00.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.807 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616918+0000", 2024-09-16T00:08:00.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5174999999999996e-05 2024-09-16T00:08:00.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:00.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617019+0000", 2024-09-16T00:08:00.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101161 2024-09-16T00:08:00.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:00.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618503+0000", 2024-09-16T00:08:00.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014836750000000001 2024-09-16T00:08:00.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:00.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620201+0000", 2024-09-16T00:08:00.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00169805 2024-09-16T00:08:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620255+0000", 2024-09-16T00:08:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4499000000000001e-05 2024-09-16T00:08:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620285+0000", 2024-09-16T00:08:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9349e-05 2024-09-16T00:08:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.810 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:00.810 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:00.810 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.093814184999999, 2024-09-16T00:08:00.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063682799999999996, 2024-09-16T00:08:00.810 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:00.810 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:00.810 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:00.810 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:00.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:00.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:00.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:00.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:00.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:00.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:00.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:00.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9900000000000001e-07 2024-09-16T00:08:00.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:00.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616079+0000", 2024-09-16T00:08:00.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9009999999999999e-06 2024-09-16T00:08:00.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:00.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616086+0000", 2024-09-16T00:08:00.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.776e-06 2024-09-16T00:08:00.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:00.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616091+0000", 2024-09-16T00:08:00.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5600000000000001e-06 2024-09-16T00:08:00.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:00.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618339+0000", 2024-09-16T00:08:00.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022476179999999998 2024-09-16T00:08:00.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.813 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:00.813 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:00.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9025999999999997e-05 2024-09-16T00:08:00.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:00.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:00.813 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:00.813 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622441+0000", 2024-09-16T00:08:00.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040029999999999996 2024-09-16T00:08:00.813 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.813 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.814 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.814 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:00.814 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:00.814 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:00.815 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:08:00.816 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:00.815+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:00.916 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:00.914+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:00.920 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:00.921 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:00.921 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:00.921 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:00.930 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:08:01.017 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:01.016+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:01.073 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:01.074 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:01.074 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:01.074 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:01.074 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:01.084 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:08:01.118 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:01.118+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:01.220 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:01.219+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:01.229 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:01.229 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:01.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:01.229 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:01.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.229 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:01.229 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:01.229 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.612193198, 2024-09-16T00:08:01.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037514929999999998, 2024-09-16T00:08:01.229 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.229 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.230 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.230 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:01.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:01.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615364+0000", 2024-09-16T00:08:01.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8430000000000003e-06 2024-09-16T00:08:01.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615371+0000", 2024-09-16T00:08:01.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.148e-06 2024-09-16T00:08:01.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615380+0000", 2024-09-16T00:08:01.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5889999999999992e-06 2024-09-16T00:08:01.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615389+0000", 2024-09-16T00:08:01.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1620000000000008e-06 2024-09-16T00:08:01.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615525+0000", 2024-09-16T00:08:01.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000136083 2024-09-16T00:08:01.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615662+0000", 2024-09-16T00:08:01.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013696100000000001 2024-09-16T00:08:01.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:01.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615799+0000", 2024-09-16T00:08:01.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013781399999999999 2024-09-16T00:08:01.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:01.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:01.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002363502 2024-09-16T00:08:01.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:01.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619018+0000", 2024-09-16T00:08:01.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085522199999999999 2024-09-16T00:08:01.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619068+0000", 2024-09-16T00:08:01.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9545000000000001e-05 2024-09-16T00:08:01.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619111+0000", 2024-09-16T00:08:01.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3624000000000001e-05 2024-09-16T00:08:01.235 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.235 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.235 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.235 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:01.235 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:01.235 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.612141127000001, 2024-09-16T00:08:01.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043656110000000001, 2024-09-16T00:08:01.235 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.235 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.235 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.236 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.236 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:01.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.236 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:01.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.236 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:01.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2199999999999999e-07 2024-09-16T00:08:01.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615416+0000", 2024-09-16T00:08:01.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5590000000000001e-06 2024-09-16T00:08:01.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615423+0000", 2024-09-16T00:08:01.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6250000000000001e-06 2024-09-16T00:08:01.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615426+0000", 2024-09-16T00:08:01.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2540000000000001e-06 2024-09-16T00:08:01.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616075+0000", 2024-09-16T00:08:01.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064968800000000004 2024-09-16T00:08:01.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616142+0000", 2024-09-16T00:08:01.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6203000000000003e-05 2024-09-16T00:08:01.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:01.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616247+0000", 2024-09-16T00:08:01.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010572499999999999 2024-09-16T00:08:01.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:01.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618913+0000", 2024-09-16T00:08:01.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026654339999999999 2024-09-16T00:08:01.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:01.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619712+0000", 2024-09-16T00:08:01.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079885199999999996 2024-09-16T00:08:01.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619744+0000", 2024-09-16T00:08:01.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2790000000000003e-05 2024-09-16T00:08:01.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619778+0000", 2024-09-16T00:08:01.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3059000000000003e-05 2024-09-16T00:08:01.241 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.241 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.241 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.241 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:01.241 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:01.241 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.612105048, 2024-09-16T00:08:01.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087673219999999993, 2024-09-16T00:08:01.241 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.241 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.241 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.241 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.242 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.242 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:01.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.242 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.242 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:01.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.242 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.242 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:01.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:01.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:08:01.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5392000000000001e-05 2024-09-16T00:08:01.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615470+0000", 2024-09-16T00:08:01.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1999999999999999e-06 2024-09-16T00:08:01.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615474+0000", 2024-09-16T00:08:01.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7359999999999999e-06 2024-09-16T00:08:01.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617614+0000", 2024-09-16T00:08:01.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021398189999999998 2024-09-16T00:08:01.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617724+0000", 2024-09-16T00:08:01.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110808 2024-09-16T00:08:01.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:01.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617856+0000", 2024-09-16T00:08:01.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131676 2024-09-16T00:08:01.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:01.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:01.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029629280000000001 2024-09-16T00:08:01.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:01.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624158+0000", 2024-09-16T00:08:01.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033386549999999998 2024-09-16T00:08:01.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624189+0000", 2024-09-16T00:08:01.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1508999999999999e-05 2024-09-16T00:08:01.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624215+0000", 2024-09-16T00:08:01.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6228999999999999e-05 2024-09-16T00:08:01.246 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.246 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.246 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.246 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:01.247 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:01.247 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.612071032999999, 2024-09-16T00:08:01.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053285279999999999, 2024-09-16T00:08:01.247 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.247 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.247 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.247 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:01.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:01.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:01.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5600000000000002e-07 2024-09-16T00:08:01.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615484+0000", 2024-09-16T00:08:01.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.688e-06 2024-09-16T00:08:01.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615488+0000", 2024-09-16T00:08:01.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.658e-06 2024-09-16T00:08:01.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615490+0000", 2024-09-16T00:08:01.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.728e-06 2024-09-16T00:08:01.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616441+0000", 2024-09-16T00:08:01.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00095065500000000001 2024-09-16T00:08:01.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616539+0000", 2024-09-16T00:08:01.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8335e-05 2024-09-16T00:08:01.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:01.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616631+0000", 2024-09-16T00:08:01.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1260000000000004e-05 2024-09-16T00:08:01.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:01.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619837+0000", 2024-09-16T00:08:01.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003206899 2024-09-16T00:08:01.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:01.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620725+0000", 2024-09-16T00:08:01.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088713599999999996 2024-09-16T00:08:01.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620772+0000", 2024-09-16T00:08:01.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7639999999999998e-05 2024-09-16T00:08:01.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620811+0000", 2024-09-16T00:08:01.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8272999999999998e-05 2024-09-16T00:08:01.252 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.252 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.252 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.252 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:01.252 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:01.252 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.611655944999999, 2024-09-16T00:08:01.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029886169999999998, 2024-09-16T00:08:01.252 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.252 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.252 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.253 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:01.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:01.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615898+0000", 2024-09-16T00:08:01.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.237e-06 2024-09-16T00:08:01.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615904+0000", 2024-09-16T00:08:01.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:01.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615915+0000", 2024-09-16T00:08:01.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1003e-05 2024-09-16T00:08:01.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615922+0000", 2024-09-16T00:08:01.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9959999999999996e-06 2024-09-16T00:08:01.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615988+0000", 2024-09-16T00:08:01.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5806e-05 2024-09-16T00:08:01.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616009+0000", 2024-09-16T00:08:01.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0729000000000001e-05 2024-09-16T00:08:01.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:01.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028462130000000002 2024-09-16T00:08:01.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618886+0000", 2024-09-16T00:08:01.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.082e-05 2024-09-16T00:08:01.256 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.256 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.256 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.256 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:01.257 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:01.257 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.610044472999999, 2024-09-16T00:08:01.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032610299999999998, 2024-09-16T00:08:01.257 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.257 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.257 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.257 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:01.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:01.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617556+0000", 2024-09-16T00:08:01.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7431e-05 2024-09-16T00:08:01.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617847+0000", 2024-09-16T00:08:01.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029133400000000003 2024-09-16T00:08:01.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617858+0000", 2024-09-16T00:08:01.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0222e-05 2024-09-16T00:08:01.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617864+0000", 2024-09-16T00:08:01.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7619999999999998e-06 2024-09-16T00:08:01.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617909+0000", 2024-09-16T00:08:01.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4747999999999999e-05 2024-09-16T00:08:01.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617927+0000", 2024-09-16T00:08:01.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7785999999999999e-05 2024-09-16T00:08:01.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620734+0000", 2024-09-16T00:08:01.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002807305 2024-09-16T00:08:01.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620770+0000", 2024-09-16T00:08:01.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5441999999999997e-05 2024-09-16T00:08:01.261 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.261 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.261 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.261 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:01.261 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:01.261 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.609623789, 2024-09-16T00:08:01.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028641809999999999, 2024-09-16T00:08:01.261 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.262 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.262 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.262 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:01.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:01.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617930+0000", 2024-09-16T00:08:01.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0200000000000002e-07 2024-09-16T00:08:01.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617934+0000", 2024-09-16T00:08:01.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:01.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617943+0000", 2024-09-16T00:08:01.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5820000000000002e-06 2024-09-16T00:08:01.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617949+0000", 2024-09-16T00:08:01.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5069999999999997e-06 2024-09-16T00:08:01.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618094+0000", 2024-09-16T00:08:01.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014515599999999999 2024-09-16T00:08:01.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618111+0000", 2024-09-16T00:08:01.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6705999999999999e-05 2024-09-16T00:08:01.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:08:01.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026603070000000002 2024-09-16T00:08:01.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620793+0000", 2024-09-16T00:08:01.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2214000000000001e-05 2024-09-16T00:08:01.265 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.265 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.265 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.266 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:01.266 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:01.266 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.609593, 2024-09-16T00:08:01.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028624549999999999, 2024-09-16T00:08:01.266 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.266 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.266 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.266 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:01.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:01.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:01.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:01.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:01.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0570000000000001e-06 2024-09-16T00:08:01.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617970+0000", 2024-09-16T00:08:01.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.669e-06 2024-09-16T00:08:01.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617976+0000", 2024-09-16T00:08:01.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1369999999999999e-06 2024-09-16T00:08:01.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618268+0000", 2024-09-16T00:08:01.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029164600000000001 2024-09-16T00:08:01.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618284+0000", 2024-09-16T00:08:01.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5824e-05 2024-09-16T00:08:01.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:01.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025127890000000001 2024-09-16T00:08:01.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620822+0000", 2024-09-16T00:08:01.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6009000000000001e-05 2024-09-16T00:08:01.270 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.270 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.271 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.271 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-16T00:08:01.271 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:01.271 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.609565418999999, 2024-09-16T00:08:01.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028650350000000002, 2024-09-16T00:08:01.271 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.271 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.271 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.271 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:01.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:01.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:01.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7599999999999998e-07 2024-09-16T00:08:01.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617991+0000", 2024-09-16T00:08:01.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1860000000000001e-06 2024-09-16T00:08:01.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617999+0000", 2024-09-16T00:08:01.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3250000000000002e-06 2024-09-16T00:08:01.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618004+0000", 2024-09-16T00:08:01.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7300000000000002e-06 2024-09-16T00:08:01.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618422+0000", 2024-09-16T00:08:01.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00041763199999999998 2024-09-16T00:08:01.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:01.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6112000000000001e-05 2024-09-16T00:08:01.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620824+0000", 2024-09-16T00:08:01.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002385954 2024-09-16T00:08:01.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620853+0000", 2024-09-16T00:08:01.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8719999999999999e-05 2024-09-16T00:08:01.275 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.275 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.275 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.275 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-16T00:08:01.275 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:01.275 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.609539597000001, 2024-09-16T00:08:01.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032703020000000001, 2024-09-16T00:08:01.276 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.276 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.276 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.276 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.276 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.276 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:01.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.276 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.276 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:01.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618014+0000", 2024-09-16T00:08:01.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7799999999999997e-07 2024-09-16T00:08:01.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618017+0000", 2024-09-16T00:08:01.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4309999999999998e-06 2024-09-16T00:08:01.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618023+0000", 2024-09-16T00:08:01.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8490000000000002e-06 2024-09-16T00:08:01.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618028+0000", 2024-09-16T00:08:01.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3489999999999999e-06 2024-09-16T00:08:01.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:08:01.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055724099999999999 2024-09-16T00:08:01.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618603+0000", 2024-09-16T00:08:01.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7486000000000002e-05 2024-09-16T00:08:01.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621235+0000", 2024-09-16T00:08:01.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002632087 2024-09-16T00:08:01.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621284+0000", 2024-09-16T00:08:01.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8581000000000001e-05 2024-09-16T00:08:01.279 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.280 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.280 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.280 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:14 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:01.280 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:01.280 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.608602616999999, 2024-09-16T00:08:01.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016807599999999999, 2024-09-16T00:08:01.280 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.280 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.280 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:01.280 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:01.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:01.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618951+0000", 2024-09-16T00:08:01.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:01.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.282 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.282 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618954+0000", 2024-09-16T00:08:01.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.145e-06 2024-09-16T00:08:01.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.282 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.282 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.282 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618963+0000", 2024-09-16T00:08:01.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5720000000000003e-06 2024-09-16T00:08:01.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.282 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.282 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.282 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618969+0000", 2024-09-16T00:08:01.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3319999999999997e-06 2024-09-16T00:08:01.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619008+0000", 2024-09-16T00:08:01.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9096e-05 2024-09-16T00:08:01.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619013+0000", 2024-09-16T00:08:01.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4710000000000001e-06 2024-09-16T00:08:01.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619118+0000", 2024-09-16T00:08:01.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000105809 2024-09-16T00:08:01.284 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.284 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.284 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.285 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-16T00:08:01.285 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:01.285 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.608564510000001, 2024-09-16T00:08:01.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002325307, 2024-09-16T00:08:01.285 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.285 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.285 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.285 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:01.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:01.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:01.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:01.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618993+0000", 2024-09-16T00:08:01.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.687e-06 2024-09-16T00:08:01.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619000+0000", 2024-09-16T00:08:01.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3590000000000002e-06 2024-09-16T00:08:01.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.288 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619005+0000", 2024-09-16T00:08:01.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.95e-06 2024-09-16T00:08:01.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.288 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619039+0000", 2024-09-16T00:08:01.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3942999999999997e-05 2024-09-16T00:08:01.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619055+0000", 2024-09-16T00:08:01.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6107e-05 2024-09-16T00:08:01.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621287+0000", 2024-09-16T00:08:01.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022318070000000001 2024-09-16T00:08:01.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:01.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7129999999999999e-05 2024-09-16T00:08:01.290 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.290 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.290 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.290 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-16T00:08:01.290 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:01.290 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.608526514000001, 2024-09-16T00:08:01.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002314328, 2024-09-16T00:08:01.290 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.290 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.291 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.291 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:01.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:01.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:01.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6399999999999998e-07 2024-09-16T00:08:01.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619030+0000", 2024-09-16T00:08:01.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.219e-06 2024-09-16T00:08:01.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619038+0000", 2024-09-16T00:08:01.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9899999999999997e-06 2024-09-16T00:08:01.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619043+0000", 2024-09-16T00:08:01.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5580000000000002e-06 2024-09-16T00:08:01.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619220+0000", 2024-09-16T00:08:01.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000176937 2024-09-16T00:08:01.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619237+0000", 2024-09-16T00:08:01.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7691999999999999e-05 2024-09-16T00:08:01.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621315+0000", 2024-09-16T00:08:01.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020778900000000002 2024-09-16T00:08:01.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621341+0000", 2024-09-16T00:08:01.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5678e-05 2024-09-16T00:08:01.294 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.294 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.294 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.295 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:15 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:01.295 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:01.295 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.607911545, 2024-09-16T00:08:01.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143416, 2024-09-16T00:08:01.295 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.295 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.295 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:01.295 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:01.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:01.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619642+0000", 2024-09-16T00:08:01.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5199999999999995e-07 2024-09-16T00:08:01.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619645+0000", 2024-09-16T00:08:01.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.134e-06 2024-09-16T00:08:01.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619655+0000", 2024-09-16T00:08:01.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3330000000000003e-06 2024-09-16T00:08:01.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619662+0000", 2024-09-16T00:08:01.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0500000000000003e-06 2024-09-16T00:08:01.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619704+0000", 2024-09-16T00:08:01.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1766e-05 2024-09-16T00:08:01.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619708+0000", 2024-09-16T00:08:01.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3440000000000003e-06 2024-09-16T00:08:01.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619785+0000", 2024-09-16T00:08:01.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6936999999999999e-05 2024-09-16T00:08:01.298 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.298 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.298 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.299 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:31 2.d e22/19 ondisk, result = 0)", 2024-09-16T00:08:01.299 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:01.299 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.607812736, 2024-09-16T00:08:01.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101826, 2024-09-16T00:08:01.299 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.299 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.299 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:01.299 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:01.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.300 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.300 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:01.300 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.300 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.300 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619741+0000", 2024-09-16T00:08:01.300 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3099999999999997e-07 2024-09-16T00:08:01.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.300 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:08:01.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1719999999999998e-06 2024-09-16T00:08:01.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619771+0000", 2024-09-16T00:08:01.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4754000000000001e-05 2024-09-16T00:08:01.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619790+0000", 2024-09-16T00:08:01.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9516000000000001e-05 2024-09-16T00:08:01.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619830+0000", 2024-09-16T00:08:01.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9848000000000002e-05 2024-09-16T00:08:01.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619834+0000", 2024-09-16T00:08:01.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1150000000000004e-06 2024-09-16T00:08:01.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619842+0000", 2024-09-16T00:08:01.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.79e-06 2024-09-16T00:08:01.303 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.303 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.303 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.303 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:30 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:01.304 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:01.304 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.603461749000001, 2024-09-16T00:08:01.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013076400000000001, 2024-09-16T00:08:01.304 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.304 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.305 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:01.305 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:01.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.306 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.306 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:01.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.306 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.306 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624092+0000", 2024-09-16T00:08:01.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5799999999999999e-07 2024-09-16T00:08:01.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.306 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.306 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624096+0000", 2024-09-16T00:08:01.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:01.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624106+0000", 2024-09-16T00:08:01.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7629999999999995e-06 2024-09-16T00:08:01.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624112+0000", 2024-09-16T00:08:01.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.626e-06 2024-09-16T00:08:01.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624149+0000", 2024-09-16T00:08:01.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7094000000000001e-05 2024-09-16T00:08:01.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624154+0000", 2024-09-16T00:08:01.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7359999999999998e-06 2024-09-16T00:08:01.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624222+0000", 2024-09-16T00:08:01.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7680000000000003e-05 2024-09-16T00:08:01.309 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.309 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.309 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.309 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:39 2.6 2:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:01.309 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:01.309 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 17.587275191, 2024-09-16T00:08:01.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050072119999999996, 2024-09-16T00:08:01.309 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.310 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.310 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.310 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:01.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:01.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640280+0000", 2024-09-16T00:08:01.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5380000000000001e-06 2024-09-16T00:08:01.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640296+0000", 2024-09-16T00:08:01.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5950000000000001e-05 2024-09-16T00:08:01.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640327+0000", 2024-09-16T00:08:01.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0309999999999999e-05 2024-09-16T00:08:01.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640340+0000", 2024-09-16T00:08:01.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3773e-05 2024-09-16T00:08:01.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640422+0000", 2024-09-16T00:08:01.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1122e-05 2024-09-16T00:08:01.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640572+0000", 2024-09-16T00:08:01.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015044099999999999 2024-09-16T00:08:01.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:08:01.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640691+0000", 2024-09-16T00:08:01.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000119393 2024-09-16T00:08:01.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:01.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645006+0000", 2024-09-16T00:08:01.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043143779999999998 2024-09-16T00:08:01.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:01.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645169+0000", 2024-09-16T00:08:01.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016323299999999999 2024-09-16T00:08:01.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645227+0000", 2024-09-16T00:08:01.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8e-05 2024-09-16T00:08:01.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645285+0000", 2024-09-16T00:08:01.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8074e-05 2024-09-16T00:08:01.316 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.316 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.316 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.316 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:39 2.6 e22/19 ondisk, result = 0)", 2024-09-16T00:08:01.316 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:01.317 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 17.582703080000002, 2024-09-16T00:08:01.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016541200000000001, 2024-09-16T00:08:01.317 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.317 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.317 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:01.317 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:01.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:01.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644852+0000", 2024-09-16T00:08:01.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1670000000000002e-06 2024-09-16T00:08:01.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644865+0000", 2024-09-16T00:08:01.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2826000000000001e-05 2024-09-16T00:08:01.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644894+0000", 2024-09-16T00:08:01.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9264999999999998e-05 2024-09-16T00:08:01.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644911+0000", 2024-09-16T00:08:01.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6827999999999999e-05 2024-09-16T00:08:01.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644989+0000", 2024-09-16T00:08:01.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7849999999999995e-05 2024-09-16T00:08:01.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644999+0000", 2024-09-16T00:08:01.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7289999999999995e-06 2024-09-16T00:08:01.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645015+0000", 2024-09-16T00:08:01.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6747000000000001e-05 2024-09-16T00:08:01.320 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.320 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.320 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.321 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.321 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.321 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:01.322 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:08:01.323 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:01.322+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:01.423 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:01.423+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:01.426 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:01.426 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:01.426 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:01.426 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:01.435 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:08:01.524 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:01.523+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:01.538 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:01.538 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:01.538 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:01.538 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:01.539 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:01.547 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:08:01.630 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:01.629+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:01.666 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:08:01.669 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:08:01.693 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:01.693 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:01.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:01.694 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:01.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.694 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:01.694 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:01.694 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.076274403999999, 2024-09-16T00:08:01.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038263220000000001, 2024-09-16T00:08:01.694 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.694 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.694 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.694 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:01.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:01.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615465+0000", 2024-09-16T00:08:01.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2769999999999999e-06 2024-09-16T00:08:01.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615471+0000", 2024-09-16T00:08:01.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5729999999999996e-06 2024-09-16T00:08:01.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615483+0000", 2024-09-16T00:08:01.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1838000000000001e-05 2024-09-16T00:08:01.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615492+0000", 2024-09-16T00:08:01.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2410000000000001e-06 2024-09-16T00:08:01.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615596+0000", 2024-09-16T00:08:01.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104097 2024-09-16T00:08:01.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615688+0000", 2024-09-16T00:08:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1225999999999997e-05 2024-09-16T00:08:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615804+0000", 2024-09-16T00:08:01.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000116601 2024-09-16T00:08:01.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:01.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618165+0000", 2024-09-16T00:08:01.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023609429999999999 2024-09-16T00:08:01.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:01.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:08:01.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010258509999999999 2024-09-16T00:08:01.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619243+0000", 2024-09-16T00:08:01.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2327999999999998e-05 2024-09-16T00:08:01.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619287+0000", 2024-09-16T00:08:01.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3346999999999997e-05 2024-09-16T00:08:01.701 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.701 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.701 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.701 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:01.701 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:01.701 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.076219328000001, 2024-09-16T00:08:01.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0054735510000000001, 2024-09-16T00:08:01.702 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.702 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.702 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.702 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:01.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:01.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615516+0000", 2024-09-16T00:08:01.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6899999999999998e-07 2024-09-16T00:08:01.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615519+0000", 2024-09-16T00:08:01.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1760000000000002e-06 2024-09-16T00:08:01.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615526+0000", 2024-09-16T00:08:01.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8399999999999997e-06 2024-09-16T00:08:01.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615531+0000", 2024-09-16T00:08:01.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4450000000000004e-06 2024-09-16T00:08:01.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:01.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054127099999999996 2024-09-16T00:08:01.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616140+0000", 2024-09-16T00:08:01.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7230999999999994e-05 2024-09-16T00:08:01.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:01.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616249+0000", 2024-09-16T00:08:01.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109122 2024-09-16T00:08:01.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:01.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618646+0000", 2024-09-16T00:08:01.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023972279999999999 2024-09-16T00:08:01.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:01.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:08:01.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022396220000000001 2024-09-16T00:08:01.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620939+0000", 2024-09-16T00:08:01.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2735999999999998e-05 2024-09-16T00:08:01.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620989+0000", 2024-09-16T00:08:01.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0411000000000002e-05 2024-09-16T00:08:01.707 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.707 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.707 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.707 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:01.707 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:01.707 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.076193848999999, 2024-09-16T00:08:01.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055872789999999997, 2024-09-16T00:08:01.708 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.708 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.708 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.708 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:01.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:01.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:01.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.01e-07 2024-09-16T00:08:01.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615547+0000", 2024-09-16T00:08:01.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7899999999999996e-06 2024-09-16T00:08:01.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:08:01.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1119999999999998e-06 2024-09-16T00:08:01.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615560+0000", 2024-09-16T00:08:01.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:08:01.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616424+0000", 2024-09-16T00:08:01.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086350800000000003 2024-09-16T00:08:01.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:08:01.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9675999999999999e-05 2024-09-16T00:08:01.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:01.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616621+0000", 2024-09-16T00:08:01.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7345999999999999e-05 2024-09-16T00:08:01.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:01.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618654+0000", 2024-09-16T00:08:01.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020335269999999998 2024-09-16T00:08:01.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:01.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621055+0000", 2024-09-16T00:08:01.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024009729999999998 2024-09-16T00:08:01.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621100+0000", 2024-09-16T00:08:01.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5402999999999999e-05 2024-09-16T00:08:01.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621128+0000", 2024-09-16T00:08:01.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7736e-05 2024-09-16T00:08:01.713 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.713 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.713 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.713 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:01.713 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:01.713 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.076168199000001, 2024-09-16T00:08:01.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057024509999999999, 2024-09-16T00:08:01.714 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.714 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.714 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.714 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:01.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:01.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:01.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6300000000000001e-07 2024-09-16T00:08:01.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615569+0000", 2024-09-16T00:08:01.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3709999999999998e-06 2024-09-16T00:08:01.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615580+0000", 2024-09-16T00:08:01.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0448000000000001e-05 2024-09-16T00:08:01.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615584+0000", 2024-09-16T00:08:01.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6020000000000002e-06 2024-09-16T00:08:01.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616850+0000", 2024-09-16T00:08:01.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012661110000000001 2024-09-16T00:08:01.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616915+0000", 2024-09-16T00:08:01.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4820000000000006e-05 2024-09-16T00:08:01.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:01.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617031+0000", 2024-09-16T00:08:01.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115258 2024-09-16T00:08:01.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:01.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618659+0000", 2024-09-16T00:08:01.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001627975 2024-09-16T00:08:01.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:01.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621179+0000", 2024-09-16T00:08:01.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025203410000000002 2024-09-16T00:08:01.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621238+0000", 2024-09-16T00:08:01.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9323000000000001e-05 2024-09-16T00:08:01.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621269+0000", 2024-09-16T00:08:01.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0839000000000001e-05 2024-09-16T00:08:01.721 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.721 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.721 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.722 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:01.722 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:01.722 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.076144450000001, 2024-09-16T00:08:01.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058012769999999996, 2024-09-16T00:08:01.722 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.722 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.723 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.723 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:01.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:01.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615591+0000", 2024-09-16T00:08:01.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6199999999999999e-07 2024-09-16T00:08:01.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615594+0000", 2024-09-16T00:08:01.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8739999999999999e-06 2024-09-16T00:08:01.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615599+0000", 2024-09-16T00:08:01.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:08:01.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615603+0000", 2024-09-16T00:08:01.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8489999999999999e-06 2024-09-16T00:08:01.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617235+0000", 2024-09-16T00:08:01.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001632037 2024-09-16T00:08:01.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617348+0000", 2024-09-16T00:08:01.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113387 2024-09-16T00:08:01.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:01.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617458+0000", 2024-09-16T00:08:01.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109765 2024-09-16T00:08:01.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:01.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618664+0000", 2024-09-16T00:08:01.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012059950000000001 2024-09-16T00:08:01.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:01.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621311+0000", 2024-09-16T00:08:01.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026468469999999999 2024-09-16T00:08:01.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621357+0000", 2024-09-16T00:08:01.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6360999999999999e-05 2024-09-16T00:08:01.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.732 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621392+0000", 2024-09-16T00:08:01.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4190999999999998e-05 2024-09-16T00:08:01.733 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.733 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.733 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.734 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:01.734 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:01.734 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.076125825999998, 2024-09-16T00:08:01.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058737279999999999, 2024-09-16T00:08:01.734 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.735 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.735 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.735 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:01.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:01.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:01.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3799999999999999e-07 2024-09-16T00:08:01.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615611+0000", 2024-09-16T00:08:01.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-06 2024-09-16T00:08:01.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615626+0000", 2024-09-16T00:08:01.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:08:01.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615634+0000", 2024-09-16T00:08:01.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-06 2024-09-16T00:08:01.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617692+0000", 2024-09-16T00:08:01.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002057718 2024-09-16T00:08:01.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617768+0000", 2024-09-16T00:08:01.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6878999999999994e-05 2024-09-16T00:08:01.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:01.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:01.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019501699999999999 2024-09-16T00:08:01.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:01.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:08:01.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034357200000000002 2024-09-16T00:08:01.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:01.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621429+0000", 2024-09-16T00:08:01.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0012999999999999e-05 2024-09-16T00:08:01.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621459+0000", 2024-09-16T00:08:01.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9498000000000001e-05 2024-09-16T00:08:01.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621483+0000", 2024-09-16T00:08:01.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3957000000000001e-05 2024-09-16T00:08:01.744 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.744 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.745 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.745 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:01.745 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:01.745 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.076094546, 2024-09-16T00:08:01.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059371789999999999, 2024-09-16T00:08:01.746 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.746 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.746 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.746 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:01.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:01.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:08:01.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4000000000000002e-07 2024-09-16T00:08:01.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615643+0000", 2024-09-16T00:08:01.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6860000000000002e-06 2024-09-16T00:08:01.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615649+0000", 2024-09-16T00:08:01.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7610000000000004e-06 2024-09-16T00:08:01.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615653+0000", 2024-09-16T00:08:01.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0380000000000001e-06 2024-09-16T00:08:01.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618197+0000", 2024-09-16T00:08:01.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025440079999999999 2024-09-16T00:08:01.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618312+0000", 2024-09-16T00:08:01.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115083 2024-09-16T00:08:01.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:01.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618412+0000", 2024-09-16T00:08:01.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010016799999999999 2024-09-16T00:08:01.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:01.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621518+0000", 2024-09-16T00:08:01.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031058679999999999 2024-09-16T00:08:01.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:01.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621526+0000", 2024-09-16T00:08:01.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2490000000000004e-06 2024-09-16T00:08:01.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621549+0000", 2024-09-16T00:08:01.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3686999999999999e-05 2024-09-16T00:08:01.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621577+0000", 2024-09-16T00:08:01.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8191000000000002e-05 2024-09-16T00:08:01.754 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.754 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.754 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.755 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:01.755 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:01.755 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.076076879999999, 2024-09-16T00:08:01.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005989622, 2024-09-16T00:08:01.755 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.755 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.755 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.755 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:01.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:01.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:01.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3799999999999998e-07 2024-09-16T00:08:01.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615660+0000", 2024-09-16T00:08:01.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9740000000000001e-06 2024-09-16T00:08:01.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615666+0000", 2024-09-16T00:08:01.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3499999999999996e-06 2024-09-16T00:08:01.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615671+0000", 2024-09-16T00:08:01.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.13e-06 2024-09-16T00:08:01.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618759+0000", 2024-09-16T00:08:01.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030880500000000002 2024-09-16T00:08:01.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618832+0000", 2024-09-16T00:08:01.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3725000000000003e-05 2024-09-16T00:08:01.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:01.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618958+0000", 2024-09-16T00:08:01.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012587899999999999 2024-09-16T00:08:01.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:01.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621583+0000", 2024-09-16T00:08:01.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026246469999999999 2024-09-16T00:08:01.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:01.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621602+0000", 2024-09-16T00:08:01.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9137e-05 2024-09-16T00:08:01.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.763 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.763 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621629+0000", 2024-09-16T00:08:01.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6451000000000001e-05 2024-09-16T00:08:01.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.763 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.764 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621648+0000", 2024-09-16T00:08:01.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8941e-05 2024-09-16T00:08:01.764 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.764 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.764 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.764 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.765 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:01.765 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:01.765 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.076059261000001, 2024-09-16T00:08:01.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053141380000000004, 2024-09-16T00:08:01.765 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.765 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.765 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.765 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:01.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:01.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:01.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-07 2024-09-16T00:08:01.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615678+0000", 2024-09-16T00:08:01.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:08:01.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615682+0000", 2024-09-16T00:08:01.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6659999999999999e-06 2024-09-16T00:08:01.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615686+0000", 2024-09-16T00:08:01.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6990000000000001e-06 2024-09-16T00:08:01.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615736+0000", 2024-09-16T00:08:01.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9656e-05 2024-09-16T00:08:01.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.770 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.770 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615864+0000", 2024-09-16T00:08:01.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012836100000000001 2024-09-16T00:08:01.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:01.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615989+0000", 2024-09-16T00:08:01.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000125525 2024-09-16T00:08:01.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:01.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617355+0000", 2024-09-16T00:08:01.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001365982 2024-09-16T00:08:01.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:01.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620897+0000", 2024-09-16T00:08:01.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035419610000000002 2024-09-16T00:08:01.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620941+0000", 2024-09-16T00:08:01.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3998999999999997e-05 2024-09-16T00:08:01.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620990+0000", 2024-09-16T00:08:01.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8199999999999999e-05 2024-09-16T00:08:01.775 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.775 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.775 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.775 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:01.775 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:01.776 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.076032003000002, 2024-09-16T00:08:01.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006316449, 2024-09-16T00:08:01.776 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.776 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.776 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:01.776 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:01.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:01.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:01.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3299999999999998e-07 2024-09-16T00:08:01.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615705+0000", 2024-09-16T00:08:01.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.278e-06 2024-09-16T00:08:01.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615711+0000", 2024-09-16T00:08:01.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8119999999999996e-06 2024-09-16T00:08:01.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615716+0000", 2024-09-16T00:08:01.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4909999999999999e-06 2024-09-16T00:08:01.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617551+0000", 2024-09-16T00:08:01.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00183549 2024-09-16T00:08:01.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.782 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617627+0000", 2024-09-16T00:08:01.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6163999999999998e-05 2024-09-16T00:08:01.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.782 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.782 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:01.782 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617776+0000", 2024-09-16T00:08:01.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014909499999999999 2024-09-16T00:08:01.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:01.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621072+0000", 2024-09-16T00:08:01.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032955150000000002 2024-09-16T00:08:01.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:01.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621957+0000", 2024-09-16T00:08:01.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088553199999999996 2024-09-16T00:08:01.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:01.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621998+0000", 2024-09-16T00:08:01.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0815000000000001e-05 2024-09-16T00:08:01.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:08:01.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0924000000000002e-05 2024-09-16T00:08:01.785 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.785 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.785 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.786 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:6 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:01.786 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:01.787 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.072630035, 2024-09-16T00:08:01.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000188111, 2024-09-16T00:08:01.787 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.787 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.787 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:01.787 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.788 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.788 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:01.788 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.788 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.788 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.788 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:01.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.789 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.789 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619106+0000", 2024-09-16T00:08:01.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4399999999999998e-07 2024-09-16T00:08:01.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619110+0000", 2024-09-16T00:08:01.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1110000000000001e-06 2024-09-16T00:08:01.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619120+0000", 2024-09-16T00:08:01.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7909999999999999e-06 2024-09-16T00:08:01.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619133+0000", 2024-09-16T00:08:01.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.359e-05 2024-09-16T00:08:01.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619179+0000", 2024-09-16T00:08:01.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5459e-05 2024-09-16T00:08:01.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619186+0000", 2024-09-16T00:08:01.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0480000000000001e-06 2024-09-16T00:08:01.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619293+0000", 2024-09-16T00:08:01.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000107168 2024-09-16T00:08:01.794 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.794 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.794 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.794 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:22 2.e e22/19 ondisk, result = 0)", 2024-09-16T00:08:01.794 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:01.794 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.070939333999998, 2024-09-16T00:08:01.795 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020058200000000001, 2024-09-16T00:08:01.795 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.795 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.795 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:01.795 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:01.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:01.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:01.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0109999999999999e-06 2024-09-16T00:08:01.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620801+0000", 2024-09-16T00:08:01.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6779999999999999e-06 2024-09-16T00:08:01.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620814+0000", 2024-09-16T00:08:01.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3131e-05 2024-09-16T00:08:01.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620821+0000", 2024-09-16T00:08:01.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0430000000000002e-06 2024-09-16T00:08:01.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.800 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:01.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3010999999999997e-05 2024-09-16T00:08:01.801 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.801 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.801 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.801 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620891+0000", 2024-09-16T00:08:01.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1760000000000004e-06 2024-09-16T00:08:01.801 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620996+0000", 2024-09-16T00:08:01.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010453199999999999 2024-09-16T00:08:01.802 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.802 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.802 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.803 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:7 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:01.803 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:01.803 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.070931349999999, 2024-09-16T00:08:01.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000195725, 2024-09-16T00:08:01.803 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.804 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.804 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:01.804 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:01.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:01.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620804+0000", 2024-09-16T00:08:01.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6099999999999999e-07 2024-09-16T00:08:01.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.807 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620809+0000", 2024-09-16T00:08:01.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4950000000000002e-06 2024-09-16T00:08:01.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.807 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:01.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0553e-05 2024-09-16T00:08:01.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620826+0000", 2024-09-16T00:08:01.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5579999999999997e-06 2024-09-16T00:08:01.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620874+0000", 2024-09-16T00:08:01.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7648999999999997e-05 2024-09-16T00:08:01.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620880+0000", 2024-09-16T00:08:01.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8290000000000002e-06 2024-09-16T00:08:01.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620999+0000", 2024-09-16T00:08:01.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011878 2024-09-16T00:08:01.810 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.810 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.810 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.811 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:8 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:01.811 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:01.811 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.070879290000001, 2024-09-16T00:08:01.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00027991099999999998, 2024-09-16T00:08:01.811 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.811 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.812 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:01.812 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:01.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.813 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.813 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:01.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.813 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.813 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620856+0000", 2024-09-16T00:08:01.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6800000000000001e-07 2024-09-16T00:08:01.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620859+0000", 2024-09-16T00:08:01.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6879999999999999e-06 2024-09-16T00:08:01.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620867+0000", 2024-09-16T00:08:01.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2819999999999996e-06 2024-09-16T00:08:01.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620872+0000", 2024-09-16T00:08:01.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5369999999999997e-06 2024-09-16T00:08:01.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621043+0000", 2024-09-16T00:08:01.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170791 2024-09-16T00:08:01.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621050+0000", 2024-09-16T00:08:01.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8580000000000002e-06 2024-09-16T00:08:01.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621135+0000", 2024-09-16T00:08:01.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5387000000000001e-05 2024-09-16T00:08:01.818 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.818 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.818 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.818 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.818 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:9 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:01.819 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:01.819 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.070850711999999, 2024-09-16T00:08:01.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00039143799999999997, 2024-09-16T00:08:01.819 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.819 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.819 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:01.819 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:01.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:01.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:01.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6800000000000002e-07 2024-09-16T00:08:01.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:08:01.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0310000000000001e-06 2024-09-16T00:08:01.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620892+0000", 2024-09-16T00:08:01.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9390000000000002e-06 2024-09-16T00:08:01.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620898+0000", 2024-09-16T00:08:01.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4480000000000002e-06 2024-09-16T00:08:01.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.824 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621167+0000", 2024-09-16T00:08:01.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026960599999999998 2024-09-16T00:08:01.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.824 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621174+0000", 2024-09-16T00:08:01.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.567e-06 2024-09-16T00:08:01.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621275+0000", 2024-09-16T00:08:01.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101579 2024-09-16T00:08:01.826 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.826 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.826 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.826 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:10 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:01.826 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:01.827 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.070828263999999, 2024-09-16T00:08:01.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00049416200000000003, 2024-09-16T00:08:01.827 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.827 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.827 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:01.827 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:01.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:01.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620907+0000", 2024-09-16T00:08:01.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.42e-07 2024-09-16T00:08:01.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620908+0000", 2024-09-16T00:08:01.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4300000000000001e-06 2024-09-16T00:08:01.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620911+0000", 2024-09-16T00:08:01.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3890000000000001e-06 2024-09-16T00:08:01.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620914+0000", 2024-09-16T00:08:01.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3690000000000001e-06 2024-09-16T00:08:01.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621297+0000", 2024-09-16T00:08:01.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038319200000000003 2024-09-16T00:08:01.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621305+0000", 2024-09-16T00:08:01.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4780000000000008e-06 2024-09-16T00:08:01.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621401+0000", 2024-09-16T00:08:01.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5161999999999995e-05 2024-09-16T00:08:01.834 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.834 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.834 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.835 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:11 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:01.835 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:01.835 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.070425515, 2024-09-16T00:08:01.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017943899999999999, 2024-09-16T00:08:01.835 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.835 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.835 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:01.836 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:01.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:01.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621310+0000", 2024-09-16T00:08:01.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5199999999999996e-07 2024-09-16T00:08:01.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:01.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1740000000000002e-06 2024-09-16T00:08:01.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621324+0000", 2024-09-16T00:08:01.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7170000000000003e-06 2024-09-16T00:08:01.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621331+0000", 2024-09-16T00:08:01.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2280000000000001e-06 2024-09-16T00:08:01.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621422+0000", 2024-09-16T00:08:01.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1052999999999997e-05 2024-09-16T00:08:01.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621426+0000", 2024-09-16T00:08:01.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.036e-06 2024-09-16T00:08:01.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621489+0000", 2024-09-16T00:08:01.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2478999999999996e-05 2024-09-16T00:08:01.843 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.843 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.843 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.844 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:12 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:01.844 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:01.844 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.070374659999999, 2024-09-16T00:08:01.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022422300000000001, 2024-09-16T00:08:01.844 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.845 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.845 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:01.845 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:01.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:01.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:01.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6699999999999999e-07 2024-09-16T00:08:01.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621363+0000", 2024-09-16T00:08:01.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7020000000000001e-06 2024-09-16T00:08:01.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621371+0000", 2024-09-16T00:08:01.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7209999999999994e-06 2024-09-16T00:08:01.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621376+0000", 2024-09-16T00:08:01.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7940000000000002e-06 2024-09-16T00:08:01.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621511+0000", 2024-09-16T00:08:01.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013518100000000001 2024-09-16T00:08:01.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621514+0000", 2024-09-16T00:08:01.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5599999999999998e-06 2024-09-16T00:08:01.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621584+0000", 2024-09-16T00:08:01.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9898000000000001e-05 2024-09-16T00:08:01.851 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.851 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.851 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.852 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:13 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:01.852 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:01.852 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.070348396, 2024-09-16T00:08:01.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026534099999999999, 2024-09-16T00:08:01.852 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.852 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.852 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:01.853 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:01.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:01.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621387+0000", 2024-09-16T00:08:01.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4299999999999999e-07 2024-09-16T00:08:01.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621389+0000", 2024-09-16T00:08:01.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.593e-06 2024-09-16T00:08:01.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621395+0000", 2024-09-16T00:08:01.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.553e-06 2024-09-16T00:08:01.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:08:01.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8410000000000002e-06 2024-09-16T00:08:01.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621597+0000", 2024-09-16T00:08:01.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019794100000000001 2024-09-16T00:08:01.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621600+0000", 2024-09-16T00:08:01.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.185e-06 2024-09-16T00:08:01.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621652+0000", 2024-09-16T00:08:01.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1984999999999998e-05 2024-09-16T00:08:01.858 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.859 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.859 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.859 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:23 2.e e22/19 ondisk, result = 0)", 2024-09-16T00:08:01.859 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:01.859 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.069865447000002, 2024-09-16T00:08:01.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015895699999999999, 2024-09-16T00:08:01.859 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:01.860 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:01.860 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:01.860 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:01.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:01.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:01.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:01.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:01.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:01.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:01.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621870+0000", 2024-09-16T00:08:01.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1600000000000001e-07 2024-09-16T00:08:01.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:01.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621873+0000", 2024-09-16T00:08:01.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3079999999999999e-06 2024-09-16T00:08:01.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:01.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621882+0000", 2024-09-16T00:08:01.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8429999999999999e-06 2024-09-16T00:08:01.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:01.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621896+0000", 2024-09-16T00:08:01.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:08:01.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:01.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621944+0000", 2024-09-16T00:08:01.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8062999999999997e-05 2024-09-16T00:08:01.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.864 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:01.864 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621951+0000", 2024-09-16T00:08:01.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7689999999999999e-06 2024-09-16T00:08:01.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:01.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:01.864 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:01.864 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622028+0000", 2024-09-16T00:08:01.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7243999999999994e-05 2024-09-16T00:08:01.865 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.865 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.865 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.865 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:01.865 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:01.865 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:01.866 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T00:08:01.869 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:01.867+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:01.869 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:08:01.969 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:01.967+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:02.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.008+0000 7f3d78080640 1 Processor -- start 2024-09-16T00:08:02.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.008+0000 7f3d78080640 1 -- start start 2024-09-16T00:08:02.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.009+0000 7f3d78080640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3d70154a90 0x7f3d700b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.009+0000 7f3d78080640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3d700cbb60 0x7f3d7015ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.009+0000 7f3d78080640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d7015df00 con 0x7f3d70154e70 2024-09-16T00:08:02.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.009+0000 7f3d78080640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3d700c9260 con 0x7f3d700cbb60 2024-09-16T00:08:02.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.009+0000 7f3d78080640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f3d7015edb0 con 0x7f3d70154a90 2024-09-16T00:08:02.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.009+0000 7f3d7661d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3d700cbb60 0x7f3d7015ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.009+0000 7f3d7561b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f3d70154e70 0x7f3d70071610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37394/0 (socket says 172.21.15.139:37394) 2024-09-16T00:08:02.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.009+0000 7f3d7561b640 1 -- 172.21.15.139:0/4011931011 learned_addr learned my addr 172.21.15.139:0/4011931011 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:02.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.009+0000 7f3d7661d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3d700cbb60 0x7f3d7015ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58930/0 (socket says 172.21.15.139:58930) 2024-09-16T00:08:02.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.009+0000 7f3d75e1c640 1 --2- 172.21.15.139:0/4011931011 >> v2:172.21.15.164:3300/0 conn(0x7f3d70154a90 0x7f3d700b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.009+0000 7f3d7661d640 1 -- 172.21.15.139:0/4011931011 >> v2:172.21.15.164:3300/0 conn(0x7f3d70154a90 msgr2=0x7f3d700b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:02.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.009+0000 7f3d7661d640 1 --2- 172.21.15.139:0/4011931011 >> v2:172.21.15.164:3300/0 conn(0x7f3d70154a90 0x7f3d700b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.009+0000 7f3d7661d640 1 -- 172.21.15.139:0/4011931011 >> v1:172.21.15.139:6789/0 conn(0x7f3d70154e70 legacy=0x7f3d70071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.009+0000 7f3d7661d640 1 -- 172.21.15.139:0/4011931011 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7015e1a0 con 0x7f3d700cbb60 2024-09-16T00:08:02.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.009+0000 7f3d74e1a640 1 -- 172.21.15.139:0/4011931011 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1263917630 0 0) 0x7f3d7015df00 con 0x7f3d70154e70 2024-09-16T00:08:02.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.010+0000 7f3d7661d640 1 --2- 172.21.15.139:0/4011931011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d700cbb60 0x7f3d7015ac40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3d700721e0 tx=0x7f3d6802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba045ebe01f5b94 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:02.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.010+0000 7f3d74e1a640 1 -- 172.21.15.139:0/4011931011 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d68007740 con 0x7f3d700cbb60 2024-09-16T00:08:02.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.010+0000 7f3d74e1a640 1 -- 172.21.15.139:0/4011931011 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d680078e0 con 0x7f3d700cbb60 2024-09-16T00:08:02.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.010+0000 7f3d74e1a640 1 -- 172.21.15.139:0/4011931011 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d68007cb0 con 0x7f3d700cbb60 2024-09-16T00:08:02.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.011+0000 7f3d78080640 1 -- 172.21.15.139:0/4011931011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d700cbb60 msgr2=0x7f3d7015ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.011+0000 7f3d78080640 1 --2- 172.21.15.139:0/4011931011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d700cbb60 0x7f3d7015ac40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3d700721e0 tx=0x7f3d6802a710 comp rx=0 tx=0).stop 2024-09-16T00:08:02.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.011+0000 7f3d78080640 1 -- 172.21.15.139:0/4011931011 shutdown_connections 2024-09-16T00:08:02.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.011+0000 7f3d78080640 1 --2- 172.21.15.139:0/4011931011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d700cbb60 0x7f3d7015ac40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.011+0000 7f3d78080640 1 --2- 172.21.15.139:0/4011931011 >> v2:172.21.15.164:3300/0 conn(0x7f3d70154a90 0x7f3d700b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.011+0000 7f3d78080640 1 -- 172.21.15.139:0/4011931011 >> 172.21.15.139:0/4011931011 conn(0x7f3d70070720 msgr2=0x7f3d70070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:02.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.011+0000 7f3d78080640 1 -- 172.21.15.139:0/4011931011 shutdown_connections 2024-09-16T00:08:02.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.011+0000 7f3d78080640 1 -- 172.21.15.139:0/4011931011 wait complete. 2024-09-16T00:08:02.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.012+0000 7f3d78080640 1 Processor -- start 2024-09-16T00:08:02.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.012+0000 7f3d78080640 1 -- start start 2024-09-16T00:08:02.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d78080640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d700cbb60 0x7f3d70152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d78080640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d70154a90 0x7f3d700af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d78080640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d700af010 0x7f3d700afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d78080640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7015ea40 con 0x7f3d700af010 2024-09-16T00:08:02.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d78080640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7015e760 con 0x7f3d700cbb60 2024-09-16T00:08:02.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d78080640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3d700b1050 con 0x7f3d70154a90 2024-09-16T00:08:02.017 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:02.017 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:02.017 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:02.017 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:02.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d7661d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d700af010 0x7f3d700afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d7661d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d700af010 0x7f3d700afc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58942/0 (socket says 172.21.15.139:58942) 2024-09-16T00:08:02.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d7661d640 1 -- 172.21.15.139:0/1984456204 learned_addr learned my addr 172.21.15.139:0/1984456204 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:02.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d75e1c640 1 --2- 172.21.15.139:0/1984456204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d700cbb60 0x7f3d70152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d7561b640 1 --2- 172.21.15.139:0/1984456204 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d70154a90 0x7f3d700af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d7661d640 1 -- 172.21.15.139:0/1984456204 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d70154a90 msgr2=0x7f3d700af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d7661d640 1 --2- 172.21.15.139:0/1984456204 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d70154a90 0x7f3d700af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d7661d640 1 -- 172.21.15.139:0/1984456204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d700cbb60 msgr2=0x7f3d70152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d7661d640 1 --2- 172.21.15.139:0/1984456204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d700cbb60 0x7f3d70152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d7661d640 1 -- 172.21.15.139:0/1984456204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d701f4780 con 0x7f3d700af010 2024-09-16T00:08:02.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.013+0000 7f3d7661d640 1 --2- 172.21.15.139:0/1984456204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d700af010 0x7f3d700afc90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3d6802ad00 tx=0x7f3d68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:02.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.014+0000 7f3d667fc640 1 -- 172.21.15.139:0/1984456204 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d68040070 con 0x7f3d700af010 2024-09-16T00:08:02.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.014+0000 7f3d667fc640 1 -- 172.21.15.139:0/1984456204 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d68031bc0 con 0x7f3d700af010 2024-09-16T00:08:02.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.014+0000 7f3d78080640 1 -- 172.21.15.139:0/1984456204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d701f0450 con 0x7f3d700af010 2024-09-16T00:08:02.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.014+0000 7f3d667fc640 1 -- 172.21.15.139:0/1984456204 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d68009450 con 0x7f3d700af010 2024-09-16T00:08:02.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.014+0000 7f3d78080640 1 -- 172.21.15.139:0/1984456204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d701f09f0 con 0x7f3d700af010 2024-09-16T00:08:02.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.015+0000 7f3d78080640 1 -- 172.21.15.139:0/1984456204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d70070d10 con 0x7f3d700af010 2024-09-16T00:08:02.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.015+0000 7f3d667fc640 1 -- 172.21.15.139:0/1984456204 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3d68039040 con 0x7f3d700af010 2024-09-16T00:08:02.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.016+0000 7f3d667fc640 1 --2- 172.21.15.139:0/1984456204 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3d5406c2b0 0x7f3d5406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.016+0000 7f3d667fc640 1 -- 172.21.15.139:0/1984456204 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5738+0+0 (secure 0 0 0) 0x7f3d680acd60 con 0x7f3d700af010 2024-09-16T00:08:02.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.016+0000 7f3d75e1c640 1 --2- 172.21.15.139:0/1984456204 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3d5406c2b0 0x7f3d5406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.016+0000 7f3d75e1c640 1 --2- 172.21.15.139:0/1984456204 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3d5406c2b0 0x7f3d5406e770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009ad0 tx=0x7f3d6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:02.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.020+0000 7f3d667fc640 1 -- 172.21.15.139:0/1984456204 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3d70070d10 con 0x7f3d700af010 2024-09-16T00:08:02.024 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:08:02.070 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:02.069+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:02.141 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:08:02.170 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:02.171 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:02.171 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:02.171 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:02.171 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:02.172 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:02.169+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:02.181 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:08:02.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.203+0000 7f3d78080640 1 -- 172.21.15.139:0/1984456204 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f3d700c8dd0 con 0x7f3d700af010 2024-09-16T00:08:02.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.238+0000 7fe3334e4640 1 Processor -- start 2024-09-16T00:08:02.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.238+0000 7fe3334e4640 1 -- start start 2024-09-16T00:08:02.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe3334e4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe32c06cea0 0x7fe32c06dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe3334e4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe32c151cf0 0x7fe32c06d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe3334e4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe32c158520 con 0x7fe32c06d280 2024-09-16T00:08:02.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe3334e4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fe32c0caa30 con 0x7fe32c06cea0 2024-09-16T00:08:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe3334e4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe32c072a70 con 0x7fe32c151cf0 2024-09-16T00:08:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe330a7f640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe32c06d280 0x7fe32c1515e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37402/0 (socket says 172.21.15.139:37402) 2024-09-16T00:08:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe330a7f640 1 -- 172.21.15.139:0/86298773 learned_addr learned my addr 172.21.15.139:0/86298773 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe331280640 1 --2- 172.21.15.139:0/86298773 >> v2:172.21.15.164:3300/0 conn(0x7fe32c06cea0 0x7fe32c06dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe331a81640 1 --2- 172.21.15.139:0/86298773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe32c151cf0 0x7fe32c06d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe323fff640 1 -- 172.21.15.139:0/86298773 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 888799835 0 0) 0x7fe32c158520 con 0x7fe32c06d280 2024-09-16T00:08:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe323fff640 1 -- 172.21.15.139:0/86298773 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe318003520 con 0x7fe32c06d280 2024-09-16T00:08:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe323fff640 1 -- 172.21.15.139:0/86298773 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1983904092 0 0) 0x7fe318003520 con 0x7fe32c06d280 2024-09-16T00:08:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe323fff640 1 -- 172.21.15.139:0/86298773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe32c151cf0 msgr2=0x7fe32c06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe323fff640 1 --2- 172.21.15.139:0/86298773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe32c151cf0 0x7fe32c06d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe323fff640 1 -- 172.21.15.139:0/86298773 >> v2:172.21.15.164:3300/0 conn(0x7fe32c06cea0 msgr2=0x7fe32c06dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe323fff640 1 --2- 172.21.15.139:0/86298773 >> v2:172.21.15.164:3300/0 conn(0x7fe32c06cea0 0x7fe32c06dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.239+0000 7fe323fff640 1 -- 172.21.15.139:0/86298773 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32c073dc0 con 0x7fe32c06d280 2024-09-16T00:08:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe323fff640 1 -- 172.21.15.139:0/86298773 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7fe31c001670 con 0x7fe32c06d280 2024-09-16T00:08:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe323fff640 1 -- 172.21.15.139:0/86298773 >> v1:172.21.15.139:6789/0 conn(0x7fe32c06d280 legacy=0x7fe32c1515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe323fff640 1 --2- 172.21.15.139:0/86298773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3180039b0 0x7fe318003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe323fff640 1 --2- 172.21.15.139:0/86298773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe318004800 0x7fe318004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe323fff640 1 --2- 172.21.15.139:0/86298773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe32c151cf0 0x7fe3180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe323fff640 1 -- 172.21.15.139:0/86298773 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c0caa30 con 0x7fe318004800 2024-09-16T00:08:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe323fff640 1 -- 172.21.15.139:0/86298773 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c072a70 con 0x7fe3180039b0 2024-09-16T00:08:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe323fff640 1 -- 172.21.15.139:0/86298773 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe31800b9c0 con 0x7fe32c151cf0 2024-09-16T00:08:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe330a7f640 1 --2- 172.21.15.139:0/86298773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe318004800 0x7fe318004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe331280640 1 --2- 172.21.15.139:0/86298773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3180039b0 0x7fe318003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe331a81640 1 --2- 172.21.15.139:0/86298773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe32c151cf0 0x7fe3180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe330a7f640 1 -- 172.21.15.139:0/86298773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe32c151cf0 msgr2=0x7fe3180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:02.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe330a7f640 1 --2- 172.21.15.139:0/86298773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe32c151cf0 0x7fe3180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe330a7f640 1 -- 172.21.15.139:0/86298773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3180039b0 msgr2=0x7fe318003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:02.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe330a7f640 1 --2- 172.21.15.139:0/86298773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3180039b0 0x7fe318003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe330a7f640 1 -- 172.21.15.139:0/86298773 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe318003520 con 0x7fe318004800 2024-09-16T00:08:02.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.240+0000 7fe331280640 1 --2- 172.21.15.139:0/86298773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3180039b0 0x7fe318003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:02.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.241+0000 7fe330a7f640 1 --2- 172.21.15.139:0/86298773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe318004800 0x7fe318004c80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe31c007fd0 tx=0x7fe318005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257af06a3e906662 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:02.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.241+0000 7fe323fff640 1 -- 172.21.15.139:0/86298773 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c007d50 con 0x7fe318004800 2024-09-16T00:08:02.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.241+0000 7fe323fff640 1 -- 172.21.15.139:0/86298773 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe31c005ad0 con 0x7fe318004800 2024-09-16T00:08:02.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.241+0000 7fe323fff640 1 -- 172.21.15.139:0/86298773 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c007070 con 0x7fe318004800 2024-09-16T00:08:02.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.242+0000 7fe3334e4640 1 -- 172.21.15.139:0/86298773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe318004800 msgr2=0x7fe318004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.242+0000 7fe3334e4640 1 --2- 172.21.15.139:0/86298773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe318004800 0x7fe318004c80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe31c007fd0 tx=0x7fe318005830 comp rx=0 tx=0).stop 2024-09-16T00:08:02.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.242+0000 7fe331a81640 1 -- 172.21.15.139:0/86298773 reap_dead start 2024-09-16T00:08:02.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.242+0000 7fe3334e4640 1 -- 172.21.15.139:0/86298773 shutdown_connections 2024-09-16T00:08:02.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.242+0000 7fe3334e4640 1 -- 172.21.15.139:0/86298773 >> 172.21.15.139:0/86298773 conn(0x7fe32c0c9af0 msgr2=0x7fe32c0c9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:02.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.242+0000 7fe3334e4640 1 -- 172.21.15.139:0/86298773 shutdown_connections 2024-09-16T00:08:02.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.242+0000 7fe3334e4640 1 -- 172.21.15.139:0/86298773 wait complete. 2024-09-16T00:08:02.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.243+0000 7fe3334e4640 1 Processor -- start 2024-09-16T00:08:02.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.243+0000 7fe3334e4640 1 -- start start 2024-09-16T00:08:02.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.244+0000 7fe3334e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe318004800 0x7fe32c0c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.244+0000 7fe3334e4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe32c153270 0x7fe32c0c3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.244+0000 7fe3334e4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe32c06be00 0x7fe32c156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.244+0000 7fe3334e4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c1581e0 con 0x7fe318004800 2024-09-16T00:08:02.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.244+0000 7fe3334e4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c072700 con 0x7fe32c06be00 2024-09-16T00:08:02.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.244+0000 7fe3334e4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe32c06d6b0 con 0x7fe32c153270 2024-09-16T00:08:02.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.244+0000 7fe331280640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe318004800 0x7fe32c0c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.244+0000 7fe331280640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe318004800 0x7fe32c0c14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58952/0 (socket says 172.21.15.139:58952) 2024-09-16T00:08:02.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.244+0000 7fe331280640 1 -- 172.21.15.139:0/3434312814 learned_addr learned my addr 172.21.15.139:0/3434312814 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:02.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.244+0000 7fe331280640 1 -- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe32c153270 msgr2=0x7fe32c0c3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.244+0000 7fe331280640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe32c153270 0x7fe32c0c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.244+0000 7fe331280640 1 -- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe32c06be00 msgr2=0x7fe32c156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.244+0000 7fe331280640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe32c06be00 0x7fe32c156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.244+0000 7fe331280640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32c0c0840 con 0x7fe318004800 2024-09-16T00:08:02.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.244+0000 7fe331280640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe318004800 0x7fe32c0c14e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe32800f4c0 tx=0x7fe32800f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:02.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.244+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328003b90 con 0x7fe318004800 2024-09-16T00:08:02.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.245+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe328003d30 con 0x7fe318004800 2024-09-16T00:08:02.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.245+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe32c154340 con 0x7fe318004800 2024-09-16T00:08:02.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.245+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328004240 con 0x7fe318004800 2024-09-16T00:08:02.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.245+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe32c152450 con 0x7fe318004800 2024-09-16T00:08:02.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.246+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fe32c06dfe0 con 0x7fe318004800 2024-09-16T00:08:02.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.246+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe3280043e0 con 0x7fe318004800 2024-09-16T00:08:02.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.247+0000 7fe321ffb640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe31006c2b0 0x7fe31006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.247+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5738+0+0 (secure 0 0 0) 0x7fe32808cb30 con 0x7fe318004800 2024-09-16T00:08:02.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.247+0000 7fe330a7f640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe31006c2b0 0x7fe31006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.247+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v27) ==== 54+0+16 (secure 0 0 0) 0x7fe328019030 con 0x7fe318004800 2024-09-16T00:08:02.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.247+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe32c0c3fb0 0x7fe32c14e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.247+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe32c0c1a20 con 0x7fe32c0c3fb0 2024-09-16T00:08:02.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.247+0000 7fe330a7f640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe31006c2b0 0x7fe31006e770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe31c033000 tx=0x7fe31c007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:02.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.247+0000 7fe331a81640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe32c0c3fb0 0x7fe32c14e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.248+0000 7fe331a81640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe32c0c3fb0 0x7fe32c14e2d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:02.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.248+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe32c0c1a20 con 0x7fe32c0c3fb0 2024-09-16T00:08:02.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.262+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe32c1518a0 con 0x7fe32c0c3fb0 2024-09-16T00:08:02.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.264+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe32c1518a0 con 0x7fe32c0c3fb0 2024-09-16T00:08:02.266 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:02.266 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:02.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.264+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe32c152620 0x7fe32c156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.264+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fe32c0cc820 con 0x7fe32c152620 2024-09-16T00:08:02.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.264+0000 7fe331280640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe32c152620 0x7fe32c156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.265+0000 7fe331280640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe32c152620 0x7fe32c156f90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:02.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.265+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe32c0cc820 con 0x7fe32c152620 2024-09-16T00:08:02.272 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:02.270+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:02.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.280+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe32c0c3650 con 0x7fe32c152620 2024-09-16T00:08:02.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.281+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe32c0c3650 con 0x7fe32c152620 2024-09-16T00:08:02.285 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:02.286 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:02.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.282+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fe32c14c290 0x7fe32c14c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.282+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fe32c1553c0 con 0x7fe32c14c290 2024-09-16T00:08:02.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.282+0000 7fe330a7f640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fe32c14c290 0x7fe32c14c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.282+0000 7fe330a7f640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fe32c14c290 0x7fe32c14c6b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:02.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.283+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe32c1553c0 con 0x7fe32c14c290 2024-09-16T00:08:02.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.296+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe32c155870 con 0x7fe32c14c290 2024-09-16T00:08:02.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.298+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe32c155870 con 0x7fe32c14c290 2024-09-16T00:08:02.300 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:02.301 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:02.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.298+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fe32c062630 0x7fe32c062a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.298+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fe32c156350 con 0x7fe32c062630 2024-09-16T00:08:02.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.298+0000 7fe331a81640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fe32c062630 0x7fe32c062a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.298+0000 7fe331a81640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fe32c062630 0x7fe32c062a50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:02.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.299+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe32c156350 con 0x7fe32c062630 2024-09-16T00:08:02.314 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:02.314 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:02.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:02.315 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:02.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.315 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:22 2.e e22/19 2:717a0223:::608.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:02.315 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:02.315 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.696961797, 2024-09-16T00:08:02.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042195870000000003, 2024-09-16T00:08:02.315 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:02.315 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:02.315 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:02.315 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:02.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:02.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:02.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:02.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:02.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:02.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616426+0000", 2024-09-16T00:08:02.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.037e-06 2024-09-16T00:08:02.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:02.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616435+0000", 2024-09-16T00:08:02.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6440000000000006e-06 2024-09-16T00:08:02.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:02.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616447+0000", 2024-09-16T00:08:02.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2466000000000001e-05 2024-09-16T00:08:02.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:02.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616456+0000", 2024-09-16T00:08:02.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8799999999999997e-06 2024-09-16T00:08:02.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:02.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616517+0000", 2024-09-16T00:08:02.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1171000000000001e-05 2024-09-16T00:08:02.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:02.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616540+0000", 2024-09-16T00:08:02.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2991000000000001e-05 2024-09-16T00:08:02.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:02.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620599+0000", 2024-09-16T00:08:02.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040586140000000003 2024-09-16T00:08:02.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:02.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620645+0000", 2024-09-16T00:08:02.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5784000000000001e-05 2024-09-16T00:08:02.320 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.320 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:02.320 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.320 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:02.320 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:02.321 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.696872067000001, 2024-09-16T00:08:02.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044836249999999998, 2024-09-16T00:08:02.321 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:02.321 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:02.321 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:02.321 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:02.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:02.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:02.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:02.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:02.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:02.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616518+0000", 2024-09-16T00:08:02.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3699999999999999e-06 2024-09-16T00:08:02.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:02.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:08:02.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1479999999999997e-06 2024-09-16T00:08:02.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:02.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616534+0000", 2024-09-16T00:08:02.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0641e-05 2024-09-16T00:08:02.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:02.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616541+0000", 2024-09-16T00:08:02.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3339999999999996e-06 2024-09-16T00:08:02.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:02.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617318+0000", 2024-09-16T00:08:02.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00077694799999999996 2024-09-16T00:08:02.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:02.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617409+0000", 2024-09-16T00:08:02.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0740000000000002e-05 2024-09-16T00:08:02.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:02.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620998+0000", 2024-09-16T00:08:02.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035894439999999998 2024-09-16T00:08:02.325 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.325 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:02.325 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.325 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:02.325 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:02.325 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.696824163999999, 2024-09-16T00:08:02.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041597320000000002, 2024-09-16T00:08:02.325 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:02.325 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:02.326 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:02.326 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:02.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:02.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:02.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:02.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:02.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:02.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:02.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:08:02.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:02.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616566+0000", 2024-09-16T00:08:02.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6979999999999998e-06 2024-09-16T00:08:02.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:02.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616574+0000", 2024-09-16T00:08:02.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7759999999999997e-06 2024-09-16T00:08:02.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:02.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616580+0000", 2024-09-16T00:08:02.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:02.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:02.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616800+0000", 2024-09-16T00:08:02.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022016700000000001 2024-09-16T00:08:02.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:02.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616904+0000", 2024-09-16T00:08:02.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104579 2024-09-16T00:08:02.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:02.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620722+0000", 2024-09-16T00:08:02.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003818133 2024-09-16T00:08:02.330 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.330 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:02.330 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.330 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:02.330 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:02.330 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.696573479000001, 2024-09-16T00:08:02.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038451570000000001, 2024-09-16T00:08:02.331 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:02.331 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:02.331 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:02.331 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:02.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:02.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:02.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:02.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:02.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:02.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616814+0000", 2024-09-16T00:08:02.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9299999999999997e-07 2024-09-16T00:08:02.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:02.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616824+0000", 2024-09-16T00:08:02.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6600000000000007e-06 2024-09-16T00:08:02.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.333 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:02.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616835+0000", 2024-09-16T00:08:02.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0713000000000001e-05 2024-09-16T00:08:02.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:02.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616841+0000", 2024-09-16T00:08:02.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.934e-06 2024-09-16T00:08:02.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:02.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616948+0000", 2024-09-16T00:08:02.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010631100000000001 2024-09-16T00:08:02.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:02.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616967+0000", 2024-09-16T00:08:02.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9598000000000001e-05 2024-09-16T00:08:02.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:02.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620619+0000", 2024-09-16T00:08:02.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036515810000000001 2024-09-16T00:08:02.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:02.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620659+0000", 2024-09-16T00:08:02.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9666999999999998e-05 2024-09-16T00:08:02.335 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.335 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:02.335 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.336 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:02.336 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:02.336 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.696526527, 2024-09-16T00:08:02.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038240129999999998, 2024-09-16T00:08:02.336 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:02.336 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:02.336 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:02.336 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:02.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:02.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:02.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:02.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:02.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:02.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616861+0000", 2024-09-16T00:08:02.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0200000000000003e-07 2024-09-16T00:08:02.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:02.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616864+0000", 2024-09-16T00:08:02.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5090000000000002e-06 2024-09-16T00:08:02.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:02.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616871+0000", 2024-09-16T00:08:02.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9249999999999998e-06 2024-09-16T00:08:02.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:02.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616877+0000", 2024-09-16T00:08:02.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.981e-06 2024-09-16T00:08:02.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:02.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617168+0000", 2024-09-16T00:08:02.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029035699999999998 2024-09-16T00:08:02.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:02.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617184+0000", 2024-09-16T00:08:02.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6005999999999999e-05 2024-09-16T00:08:02.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:02.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620661+0000", 2024-09-16T00:08:02.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034775330000000001 2024-09-16T00:08:02.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:02.340 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620684+0000", 2024-09-16T00:08:02.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3300000000000001e-05 2024-09-16T00:08:02.340 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.340 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:02.340 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.340 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:02.340 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:02.340 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.696071084, 2024-09-16T00:08:02.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045415610000000004, 2024-09-16T00:08:02.341 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:02.341 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:02.341 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:02.341 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:02.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:02.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:02.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:02.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:02.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:02.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:02.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8800000000000002e-07 2024-09-16T00:08:02.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:02.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617321+0000", 2024-09-16T00:08:02.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6979999999999997e-06 2024-09-16T00:08:02.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:02.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617330+0000", 2024-09-16T00:08:02.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6729999999999998e-06 2024-09-16T00:08:02.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:02.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617336+0000", 2024-09-16T00:08:02.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2110000000000003e-06 2024-09-16T00:08:02.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:02.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617394+0000", 2024-09-16T00:08:02.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8087999999999997e-05 2024-09-16T00:08:02.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:02.344 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617412+0000", 2024-09-16T00:08:02.344 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8099999999999999e-05 2024-09-16T00:08:02.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.344 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.344 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:02.344 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621818+0000", 2024-09-16T00:08:02.344 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044061209999999998 2024-09-16T00:08:02.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.344 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.344 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:02.344 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621857+0000", 2024-09-16T00:08:02.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9082000000000003e-05 2024-09-16T00:08:02.345 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.345 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:02.345 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.346 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:23 2.e e22/19 2:717a0223:::608.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:02.346 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:02.346 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.695175063000001, 2024-09-16T00:08:02.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036198279999999998, 2024-09-16T00:08:02.346 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:02.346 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:02.346 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:02.346 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:02.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:02.346 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:02.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:02.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:02.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:02.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618213+0000", 2024-09-16T00:08:02.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:08:02.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:02.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618218+0000", 2024-09-16T00:08:02.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4099999999999999e-06 2024-09-16T00:08:02.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:02.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618227+0000", 2024-09-16T00:08:02.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5009999999999993e-06 2024-09-16T00:08:02.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:02.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618234+0000", 2024-09-16T00:08:02.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:02.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:02.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618307+0000", 2024-09-16T00:08:02.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3421000000000001e-05 2024-09-16T00:08:02.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:02.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618328+0000", 2024-09-16T00:08:02.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0919000000000001e-05 2024-09-16T00:08:02.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.350 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:02.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621804+0000", 2024-09-16T00:08:02.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034756560000000001 2024-09-16T00:08:02.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.350 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:02.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621832+0000", 2024-09-16T00:08:02.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7966999999999998e-05 2024-09-16T00:08:02.350 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.350 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:02.350 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.351 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.351 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing_reply(2.1e epoch 22/19 rep_tid 0 lcod 22'1)", 2024-09-16T00:08:02.351 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:02.351 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.694729895000002, 2024-09-16T00:08:02.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1371000000000003e-05, 2024-09-16T00:08:02.351 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:02.351 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:02.351 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:02.352 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:02.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.352 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:02.352 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:02.352 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.352 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.352 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:02.352 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:02.352 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.352 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.352 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:02.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:08:02.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:08:02.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:02.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618662+0000", 2024-09-16T00:08:02.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1189999999999999e-06 2024-09-16T00:08:02.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:02.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618674+0000", 2024-09-16T00:08:02.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2317000000000001e-05 2024-09-16T00:08:02.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:02.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618681+0000", 2024-09-16T00:08:02.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7170000000000002e-06 2024-09-16T00:08:02.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:02.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:08:02.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6745000000000001e-05 2024-09-16T00:08:02.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:02.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618748+0000", 2024-09-16T00:08:02.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0794000000000001e-05 2024-09-16T00:08:02.355 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.355 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:02.355 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.355 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:02.355 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:02.355 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.694216561000001, 2024-09-16T00:08:02.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027183849999999998, 2024-09-16T00:08:02.355 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:02.355 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:02.356 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:02.356 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:02.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:02.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:02.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:02.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:02.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:02.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619171+0000", 2024-09-16T00:08:02.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8999999999999995e-07 2024-09-16T00:08:02.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:02.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619179+0000", 2024-09-16T00:08:02.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2679999999999994e-06 2024-09-16T00:08:02.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:02.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:08:02.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1246e-05 2024-09-16T00:08:02.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:02.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619197+0000", 2024-09-16T00:08:02.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2820000000000002e-06 2024-09-16T00:08:02.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:02.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619308+0000", 2024-09-16T00:08:02.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011141000000000001 2024-09-16T00:08:02.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:02.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619325+0000", 2024-09-16T00:08:02.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6469000000000001e-05 2024-09-16T00:08:02.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:02.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621860+0000", 2024-09-16T00:08:02.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025353590000000001 2024-09-16T00:08:02.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:02.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621889+0000", 2024-09-16T00:08:02.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8561e-05 2024-09-16T00:08:02.360 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.360 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:02.360 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.360 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 6 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:02.360 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:02.360 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.694173862, 2024-09-16T00:08:02.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026903809999999999, 2024-09-16T00:08:02.360 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:02.360 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:02.361 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:02.361 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:02.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:02.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:02.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:02.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:02.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:02.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:02.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2000000000000001e-07 2024-09-16T00:08:02.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:02.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619219+0000", 2024-09-16T00:08:02.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:08:02.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:02.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619234+0000", 2024-09-16T00:08:02.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5075000000000001e-05 2024-09-16T00:08:02.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:02.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619251+0000", 2024-09-16T00:08:02.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6863000000000001e-05 2024-09-16T00:08:02.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:02.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619556+0000", 2024-09-16T00:08:02.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030551599999999998 2024-09-16T00:08:02.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.364 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:02.364 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621903+0000", 2024-09-16T00:08:02.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023469979999999999 2024-09-16T00:08:02.364 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.364 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:02.364 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.364 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing_reply(2.1b epoch 22/19 rep_tid 1 lcod 22'1)", 2024-09-16T00:08:02.364 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:02.365 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.693095465999999, 2024-09-16T00:08:02.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6989000000000003e-05, 2024-09-16T00:08:02.365 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:02.365 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:02.365 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:02.365 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:02.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:02.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:02.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:02.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:02.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:02.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620292+0000", 2024-09-16T00:08:02.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.37e-07 2024-09-16T00:08:02.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:02.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620296+0000", 2024-09-16T00:08:02.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2420000000000002e-06 2024-09-16T00:08:02.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:02.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620308+0000", 2024-09-16T00:08:02.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1623e-05 2024-09-16T00:08:02.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:02.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620313+0000", 2024-09-16T00:08:02.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6290000000000001e-06 2024-09-16T00:08:02.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:02.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620359+0000", 2024-09-16T00:08:02.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6201e-05 2024-09-16T00:08:02.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:02.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620378+0000", 2024-09-16T00:08:02.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9556999999999999e-05 2024-09-16T00:08:02.368 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.368 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:02.368 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.368 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:38 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'6, mlcod=22'4)", 2024-09-16T00:08:02.368 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:02.368 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.690653217000001, 2024-09-16T00:08:02.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064551599999999995, 2024-09-16T00:08:02.369 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:02.369 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:02.369 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:02.369 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:02.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:02.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:02.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:02.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:02.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:02.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:02.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4399999999999999e-07 2024-09-16T00:08:02.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:02.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622739+0000", 2024-09-16T00:08:02.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0250000000000002e-06 2024-09-16T00:08:02.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:02.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622749+0000", 2024-09-16T00:08:02.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1220000000000001e-06 2024-09-16T00:08:02.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:02.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622760+0000", 2024-09-16T00:08:02.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1640999999999999e-05 2024-09-16T00:08:02.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:02.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622799+0000", 2024-09-16T00:08:02.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8537000000000001e-05 2024-09-16T00:08:02.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:02.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622810+0000", 2024-09-16T00:08:02.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1562e-05 2024-09-16T00:08:02.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:02.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.623356+0000", 2024-09-16T00:08:02.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054522299999999995 2024-09-16T00:08:02.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:02.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.623379+0000", 2024-09-16T00:08:02.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3662000000000001e-05 2024-09-16T00:08:02.373 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.373 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:02.374 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.374 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:40 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'7, mlcod=22'5)", 2024-09-16T00:08:02.374 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:02.374 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 11.473632315, 2024-09-16T00:08:02.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001017773, 2024-09-16T00:08:02.374 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:02.374 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:02.374 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:02.374 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:02.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:02.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:02.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:02.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:02.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:02.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839756+0000", 2024-09-16T00:08:02.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.491e-06 2024-09-16T00:08:02.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:02.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839768+0000", 2024-09-16T00:08:02.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1649e-05 2024-09-16T00:08:02.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:02.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839783+0000", 2024-09-16T00:08:02.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5023e-05 2024-09-16T00:08:02.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:02.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839791+0000", 2024-09-16T00:08:02.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5620000000000004e-06 2024-09-16T00:08:02.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:02.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839848+0000", 2024-09-16T00:08:02.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6563e-05 2024-09-16T00:08:02.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:02.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839868+0000", 2024-09-16T00:08:02.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9652000000000001e-05 2024-09-16T00:08:02.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:02.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.840729+0000", 2024-09-16T00:08:02.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086159500000000005 2024-09-16T00:08:02.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:02.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.840772+0000", 2024-09-16T00:08:02.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3238000000000002e-05 2024-09-16T00:08:02.378 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.378 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:02.378 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.379 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:41 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'8, mlcod=22'6)", 2024-09-16T00:08:02.379 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:02.379 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.1999754039999999, 2024-09-16T00:08:02.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001040568, 2024-09-16T00:08:02.379 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:02.379 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:02.379 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:02.379 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:02.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:02.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:02.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:02.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:02.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:02.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:02.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113413+0000", 2024-09-16T00:08:02.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7409999999999999e-06 2024-09-16T00:08:02.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:02.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113428+0000", 2024-09-16T00:08:02.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5007999999999999e-05 2024-09-16T00:08:02.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:02.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113447+0000", 2024-09-16T00:08:02.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8731000000000001e-05 2024-09-16T00:08:02.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:02.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113457+0000", 2024-09-16T00:08:02.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0095000000000001e-05 2024-09-16T00:08:02.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:02.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113526+0000", 2024-09-16T00:08:02.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8552000000000004e-05 2024-09-16T00:08:02.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:02.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113552+0000", 2024-09-16T00:08:02.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6795999999999999e-05 2024-09-16T00:08:02.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:02.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.114417+0000", 2024-09-16T00:08:02.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086476299999999995 2024-09-16T00:08:02.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:02.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:02.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:02.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.114452+0000", 2024-09-16T00:08:02.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4882000000000002e-05 2024-09-16T00:08:02.383 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.383 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:02.383 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.383 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:02.383 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:02.383 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:02.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.312+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe32c063260 con 0x7fe32c062630 2024-09-16T00:08:02.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.314+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe32c063260 con 0x7fe32c062630 2024-09-16T00:08:02.384 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:02.385 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:02.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.314+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fe32c02cb50 0x7fe32c02cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.314+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fe32c0631e0 con 0x7fe32c02cb50 2024-09-16T00:08:02.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.315+0000 7fe331280640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fe32c02cb50 0x7fe32c02cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.315+0000 7fe331280640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fe32c02cb50 0x7fe32c02cfb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:02.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.316+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe32c0631e0 con 0x7fe32c02cb50 2024-09-16T00:08:02.386 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:08:02.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.330+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe32c062f90 con 0x7fe32c02cb50 2024-09-16T00:08:02.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.333+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe32c062f90 con 0x7fe32c02cb50 2024-09-16T00:08:02.387 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:02.389 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:02.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.333+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fe32c205150 0x7fe32c207540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.333+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fe32c0631a0 con 0x7fe32c205150 2024-09-16T00:08:02.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.333+0000 7fe330a7f640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fe32c205150 0x7fe32c207540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.333+0000 7fe330a7f640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fe32c205150 0x7fe32c207540 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:02.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.334+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe32c0631a0 con 0x7fe32c205150 2024-09-16T00:08:02.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.347+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe32c0631a0 con 0x7fe32c205150 2024-09-16T00:08:02.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.350+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe32c0631a0 con 0x7fe32c205150 2024-09-16T00:08:02.390 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.350+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fe32c207f90 0x7fe32c20a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.350+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fe32c02dba0 con 0x7fe32c207f90 2024-09-16T00:08:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.350+0000 7fe331a81640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fe32c207f90 0x7fe32c20a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.350+0000 7fe331a81640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fe32c207f90 0x7fe32c20a380 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.351+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe32c02dba0 con 0x7fe32c207f90 2024-09-16T00:08:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.364+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe32c20a8c0 con 0x7fe32c207f90 2024-09-16T00:08:02.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.366+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe32c20a8c0 con 0x7fe32c207f90 2024-09-16T00:08:02.391 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:02.392 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:02.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.366+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fe32c20afe0 0x7fe32c20b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:02.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.366+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fe32c210370 con 0x7fe32c20afe0 2024-09-16T00:08:02.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.366+0000 7fe331280640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fe32c20afe0 0x7fe32c20b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:02.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.367+0000 7fe331280640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fe32c20afe0 0x7fe32c20b440 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:02.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.368+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe32c210370 con 0x7fe32c20afe0 2024-09-16T00:08:02.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.381+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe32c20f1f0 con 0x7fe32c20afe0 2024-09-16T00:08:02.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe321ffb640 1 -- 172.21.15.139:0/3434312814 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe32c20f1f0 con 0x7fe32c20afe0 2024-09-16T00:08:02.393 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:02.394 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe32c0c3fb0 msgr2=0x7fe32c14e2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe32c0c3fb0 0x7fe32c14e2d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe32c152620 msgr2=0x7fe32c156f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe32c152620 0x7fe32c156f90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fe32c14c290 msgr2=0x7fe32c14c6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fe32c14c290 0x7fe32c14c6b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fe32c062630 msgr2=0x7fe32c062a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fe32c062630 0x7fe32c062a50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fe32c02cb50 msgr2=0x7fe32c02cfb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fe32c02cb50 0x7fe32c02cfb0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fe32c205150 msgr2=0x7fe32c207540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fe32c205150 0x7fe32c207540 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe331a81640 1 -- 172.21.15.139:0/3434312814 reap_dead start 2024-09-16T00:08:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fe32c207f90 msgr2=0x7fe32c20a380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fe32c207f90 0x7fe32c20a380 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe331a81640 1 -- 172.21.15.139:0/3434312814 reap_dead start 2024-09-16T00:08:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fe32c20afe0 msgr2=0x7fe32c20b440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fe32c20afe0 0x7fe32c20b440 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe331a81640 1 -- 172.21.15.139:0/3434312814 reap_dead start 2024-09-16T00:08:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe331a81640 1 -- 172.21.15.139:0/3434312814 reap_dead start 2024-09-16T00:08:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe31006c2b0 msgr2=0x7fe31006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe31006c2b0 0x7fe31006e770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe31c033000 tx=0x7fe31c007000 comp rx=0 tx=0).stop 2024-09-16T00:08:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe318004800 msgr2=0x7fe32c0c14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe318004800 0x7fe32c0c14e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe32800f4c0 tx=0x7fe32800f980 comp rx=0 tx=0).stop 2024-09-16T00:08:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 shutdown_connections 2024-09-16T00:08:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe31006c2b0 0x7fe31006e770 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 --2- 172.21.15.139:0/3434312814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe318004800 0x7fe32c0c14e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 >> 172.21.15.139:0/3434312814 conn(0x7fe32c0c9af0 msgr2=0x7fe32c14f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 shutdown_connections 2024-09-16T00:08:02.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.383+0000 7fe3334e4640 1 -- 172.21.15.139:0/3434312814 wait complete. 2024-09-16T00:08:02.398 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:02.394+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:02.490 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:02.490+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:02.504 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:02.504 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:02.513 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:08:02.597 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:02.597+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:02.658 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:02.659 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:02.667 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:08:02.708 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:02.708+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:02.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.721+0000 7f3d667fc640 1 -- 172.21.15.139:0/1984456204 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v28) ==== 101+0+0 (secure 0 0 0) 0x7f3d68043360 con 0x7f3d700af010 2024-09-16T00:08:02.723 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is unset 2024-09-16T00:08:02.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.723+0000 7f3d78080640 1 -- 172.21.15.139:0/1984456204 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3d5406c2b0 msgr2=0x7f3d5406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.723+0000 7f3d78080640 1 --2- 172.21.15.139:0/1984456204 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3d5406c2b0 0x7f3d5406e770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009ad0 tx=0x7f3d6c008040 comp rx=0 tx=0).stop 2024-09-16T00:08:02.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.723+0000 7f3d78080640 1 -- 172.21.15.139:0/1984456204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d700af010 msgr2=0x7f3d700afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:02.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.723+0000 7f3d78080640 1 --2- 172.21.15.139:0/1984456204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d700af010 0x7f3d700afc90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3d6802ad00 tx=0x7f3d68004a40 comp rx=0 tx=0).stop 2024-09-16T00:08:02.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.723+0000 7f3d78080640 1 -- 172.21.15.139:0/1984456204 shutdown_connections 2024-09-16T00:08:02.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.723+0000 7f3d78080640 1 --2- 172.21.15.139:0/1984456204 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3d5406c2b0 0x7f3d5406e770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.723+0000 7f3d78080640 1 --2- 172.21.15.139:0/1984456204 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d700af010 0x7f3d700afc90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.723+0000 7f3d78080640 1 --2- 172.21.15.139:0/1984456204 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d70154a90 0x7f3d700af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.723+0000 7f3d78080640 1 --2- 172.21.15.139:0/1984456204 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d700cbb60 0x7f3d70152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:02.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.723+0000 7f3d78080640 1 -- 172.21.15.139:0/1984456204 >> 172.21.15.139:0/1984456204 conn(0x7f3d70070720 msgr2=0x7f3d70159d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:02.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.723+0000 7f3d78080640 1 -- 172.21.15.139:0/1984456204 shutdown_connections 2024-09-16T00:08:02.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:02.723+0000 7f3d78080640 1 -- 172.21.15.139:0/1984456204 wait complete. 2024-09-16T00:08:02.794 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:02.794+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:02.815 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:02.815 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:02.822 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:08:02.906 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:02.906+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:02.967 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:02.967 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:02.976 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:08:03.017 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:03.017+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:03.099 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:03.098+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:03.122 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:03.122 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:03.131 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:08:03.215 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:03.215+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:03.275 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:03.275 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:03.284 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:08:03.325 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:03.325+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:03.400 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:03.400+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:03.429 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:03.429 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:03.438 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:08:03.502 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:03.501+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:03.583 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:03.583 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:03.592 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:08:03.636 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:03.636+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:03.699 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:03.700 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:03.703 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:03.702+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:03.708 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:08:03.804 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:03.803+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:03.853 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:03.853 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:03.862 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:08:03.906 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:03.906+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:03.967 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:03.967 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:03.976 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:08:04.006 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:04.005+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:04.107 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:04.107+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:04.123 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:04.123 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:04.133 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:08:04.208 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:04.207+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:04.250 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:04.250 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:04.259 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:08:04.343 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:04.342+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:04.399 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:08:04.402 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:04.402 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:04.412 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:08:04.413 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:04.412+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:04.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.498+0000 7fe3e5f3b640 1 Processor -- start 2024-09-16T00:08:04.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.498+0000 7fe3e5f3b640 1 -- start start 2024-09-16T00:08:04.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.498+0000 7fe3e5f3b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe3e00d0870 0x7fe3e00b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.498+0000 7fe3e5f3b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe3e00b3430 0x7fe3e00b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.499+0000 7fe3e5f3b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3e015adc0 con 0x7fe3e00d0490 2024-09-16T00:08:04.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.499+0000 7fe3e5f3b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe3e015b940 con 0x7fe3e00d0870 2024-09-16T00:08:04.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.499+0000 7fe3e5f3b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fe3e00b3bb0 con 0x7fe3e00b3430 2024-09-16T00:08:04.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.499+0000 7fe3deffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe3e00d0870 0x7fe3e00b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.499+0000 7fe3deffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe3e00d0870 0x7fe3e00b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58968/0 (socket says 172.21.15.139:58968) 2024-09-16T00:08:04.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.499+0000 7fe3deffd640 1 -- 172.21.15.139:0/1745306443 learned_addr learned my addr 172.21.15.139:0/1745306443 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:04.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.499+0000 7fe3dffff640 1 --2- 172.21.15.139:0/1745306443 >> v2:172.21.15.164:3300/0 conn(0x7fe3e00b3430 0x7fe3e00b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.499+0000 7fe3deffd640 1 -- 172.21.15.139:0/1745306443 >> v2:172.21.15.164:3300/0 conn(0x7fe3e00b3430 msgr2=0x7fe3e00b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:04.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.499+0000 7fe3deffd640 1 --2- 172.21.15.139:0/1745306443 >> v2:172.21.15.164:3300/0 conn(0x7fe3e00b3430 0x7fe3e00b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.499+0000 7fe3deffd640 1 -- 172.21.15.139:0/1745306443 >> v1:172.21.15.164:6789/0 conn(0x7fe3e00d0490 legacy=0x7fe3e00b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:04.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.499+0000 7fe3deffd640 1 -- 172.21.15.139:0/1745306443 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e015b0a0 con 0x7fe3e00d0870 2024-09-16T00:08:04.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.499+0000 7fe3dffff640 1 --2- 172.21.15.139:0/1745306443 >> v2:172.21.15.164:3300/0 conn(0x7fe3e00b3430 0x7fe3e00b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:08:04.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.500+0000 7fe3deffd640 1 --2- 172.21.15.139:0/1745306443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e00d0870 0x7fe3e00b4040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00c72c0 tx=0x7fe3d402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ab6b43877447ae server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.500+0000 7fe3de7fc640 1 -- 172.21.15.139:0/1745306443 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4037070 con 0x7fe3e00d0870 2024-09-16T00:08:04.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.500+0000 7fe3de7fc640 1 -- 172.21.15.139:0/1745306443 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3d4032440 con 0x7fe3e00d0870 2024-09-16T00:08:04.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.500+0000 7fe3de7fc640 1 -- 172.21.15.139:0/1745306443 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4032810 con 0x7fe3e00d0870 2024-09-16T00:08:04.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.500+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1745306443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e00d0870 msgr2=0x7fe3e00b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.500+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1745306443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e00d0870 0x7fe3e00b4040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00c72c0 tx=0x7fe3d402a2c0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.500+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1745306443 shutdown_connections 2024-09-16T00:08:04.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.500+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1745306443 >> v2:172.21.15.164:3300/0 conn(0x7fe3e00b3430 0x7fe3e00b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.500+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1745306443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e00d0870 0x7fe3e00b4040 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.500+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1745306443 >> 172.21.15.139:0/1745306443 conn(0x7fe3e006c520 msgr2=0x7fe3e006c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:04.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.501+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1745306443 shutdown_connections 2024-09-16T00:08:04.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.501+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1745306443 wait complete. 2024-09-16T00:08:04.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.502+0000 7fe3e5f3b640 1 Processor -- start 2024-09-16T00:08:04.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.502+0000 7fe3e5f3b640 1 -- start start 2024-09-16T00:08:04.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.502+0000 7fe3e5f3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e00b3430 0x7fe3e00cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3e5f3b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe3e00d0490 0x7fe3e0149800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3e5f3b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e00d0870 0x7fe3e0149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3e5f3b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e015a980 con 0x7fe3e00b3430 2024-09-16T00:08:04.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3e5f3b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e015b660 con 0x7fe3e00d0870 2024-09-16T00:08:04.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3e5f3b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e015ab00 con 0x7fe3e00d0490 2024-09-16T00:08:04.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3df7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e00b3430 0x7fe3e00cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3df7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e00b3430 0x7fe3e00cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58970/0 (socket says 172.21.15.139:58970) 2024-09-16T00:08:04.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3df7fe640 1 -- 172.21.15.139:0/1462160954 learned_addr learned my addr 172.21.15.139:0/1462160954 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:04.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3dffff640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e00d0870 0x7fe3e0149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3deffd640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe3e00d0490 0x7fe3e0149800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3df7fe640 1 -- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe3e00d0490 msgr2=0x7fe3e0149800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3df7fe640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe3e00d0490 0x7fe3e0149800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3df7fe640 1 -- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e00d0870 msgr2=0x7fe3e0149d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3df7fe640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe3e00d0870 0x7fe3e0149d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3df7fe640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e014b0e0 con 0x7fe3e00b3430 2024-09-16T00:08:04.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3df7fe640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e00b3430 0x7fe3e00cf8c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000c460 tx=0x7fe3d000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0016020 con 0x7fe3e00b3430 2024-09-16T00:08:04.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3d0012870 con 0x7fe3e00b3430 2024-09-16T00:08:04.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e01f3f70 con 0x7fe3e00b3430 2024-09-16T00:08:04.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0012c60 con 0x7fe3e00b3430 2024-09-16T00:08:04.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.503+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e01f4530 con 0x7fe3e00b3430 2024-09-16T00:08:04.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.504+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fe3e00b29a0 con 0x7fe3e00b3430 2024-09-16T00:08:04.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.505+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe3d0019020 con 0x7fe3e00b3430 2024-09-16T00:08:04.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.505+0000 7fe3c7fff640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe3cc06c2b0 0x7fe3cc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.505+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5738+0+0 (secure 0 0 0) 0x7fe3d008b040 con 0x7fe3e00b3430 2024-09-16T00:08:04.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.505+0000 7fe3deffd640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe3cc06c2b0 0x7fe3cc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.506+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v28) ==== 54+0+16 (secure 0 0 0) 0x7fe3e00b29a0 con 0x7fe3e00b3430 2024-09-16T00:08:04.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.506+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe3e014f940 0x7fe3e01f4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.506+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe3e00b2c40 con 0x7fe3e014f940 2024-09-16T00:08:04.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.506+0000 7fe3deffd640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe3cc06c2b0 0x7fe3cc06e770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4034000 tx=0x7fe3d402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.506+0000 7fe3dffff640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe3e014f940 0x7fe3e01f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.506+0000 7fe3dffff640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe3e014f940 0x7fe3e01f4770 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.507+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3e00b2c40 con 0x7fe3e014f940 2024-09-16T00:08:04.511 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:04.511+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:04.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.521+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe3e01e6700 con 0x7fe3e014f940 2024-09-16T00:08:04.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.523+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe3e01e6700 con 0x7fe3e014f940 2024-09-16T00:08:04.523 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:04.524 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:04.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.523+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe3e0147c60 0x7fe3e0148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.523+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fe3e01e6420 con 0x7fe3e0147c60 2024-09-16T00:08:04.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.523+0000 7fe3df7fe640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe3e0147c60 0x7fe3e0148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.523+0000 7fe3df7fe640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe3e0147c60 0x7fe3e0148040 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.524+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3e01e6420 con 0x7fe3e0147c60 2024-09-16T00:08:04.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.538+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe3e01e6140 con 0x7fe3e0147c60 2024-09-16T00:08:04.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.541+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe3e01e6140 con 0x7fe3e0147c60 2024-09-16T00:08:04.543 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:04.544 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:04.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.541+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fe3e0153690 0x7fe3e0153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.541+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fe3e0153fb0 con 0x7fe3e0153690 2024-09-16T00:08:04.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.542+0000 7fe3deffd640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fe3e0153690 0x7fe3e0153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.542+0000 7fe3deffd640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fe3e0153690 0x7fe3e0153a70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.543+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3e0153fb0 con 0x7fe3e0153690 2024-09-16T00:08:04.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.556+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe3e00cbff0 con 0x7fe3e0153690 2024-09-16T00:08:04.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.558+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe3e00cbff0 con 0x7fe3e0153690 2024-09-16T00:08:04.558 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:04.559 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:04.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.558+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fe3e0062f00 0x7fe3e00ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.558+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fe3e0063340 con 0x7fe3e0062f00 2024-09-16T00:08:04.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.558+0000 7fe3dffff640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fe3e0062f00 0x7fe3e00ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.558+0000 7fe3dffff640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fe3e0062f00 0x7fe3e00ce0b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.559+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3e0063340 con 0x7fe3e0062f00 2024-09-16T00:08:04.563 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'41, mlcod=17'39)", 2024-09-16T00:08:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.540689000999997, 2024-09-16T00:08:04.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001808488, 2024-09-16T00:08:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021032+0000", 2024-09-16T00:08:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3e-07 2024-09-16T00:08:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021037+0000", 2024-09-16T00:08:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0640000000000001e-06 2024-09-16T00:08:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021047+0000", 2024-09-16T00:08:04.565 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.023e-05 2024-09-16T00:08:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021053+0000", 2024-09-16T00:08:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3160000000000002e-06 2024-09-16T00:08:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021092+0000", 2024-09-16T00:08:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9033999999999999e-05 2024-09-16T00:08:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.566 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021105+0000", 2024-09-16T00:08:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2768e-05 2024-09-16T00:08:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022811+0000", 2024-09-16T00:08:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001705788 2024-09-16T00:08:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022840+0000", 2024-09-16T00:08:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8558e-05 2024-09-16T00:08:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.567 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:85 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'55, mlcod=17'50)", 2024-09-16T00:08:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.527659999999997, 2024-09-16T00:08:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017909530000000001, 2024-09-16T00:08:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.568 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034061+0000", 2024-09-16T00:08:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0399999999999996e-07 2024-09-16T00:08:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034065+0000", 2024-09-16T00:08:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8840000000000002e-06 2024-09-16T00:08:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034074+0000", 2024-09-16T00:08:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034079+0000", 2024-09-16T00:08:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0000000000000004e-06 2024-09-16T00:08:04.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034125+0000", 2024-09-16T00:08:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5905000000000002e-05 2024-09-16T00:08:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034144+0000", 2024-09-16T00:08:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9593e-05 2024-09-16T00:08:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035817+0000", 2024-09-16T00:08:04.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016728240000000001 2024-09-16T00:08:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035851+0000", 2024-09-16T00:08:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3809000000000001e-05 2024-09-16T00:08:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:112 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'74, mlcod=17'65)", 2024-09-16T00:08:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.397048138000002, 2024-09-16T00:08:04.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018457020000000001, 2024-09-16T00:08:04.574 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.574 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164673+0000", 2024-09-16T00:08:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1200000000000003e-07 2024-09-16T00:08:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164710+0000", 2024-09-16T00:08:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7756999999999998e-05 2024-09-16T00:08:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164719+0000", 2024-09-16T00:08:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2849999999999995e-06 2024-09-16T00:08:04.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164723+0000", 2024-09-16T00:08:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3220000000000003e-06 2024-09-16T00:08:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164760+0000", 2024-09-16T00:08:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7311999999999998e-05 2024-09-16T00:08:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164771+0000", 2024-09-16T00:08:04.577 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0869e-05 2024-09-16T00:08:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166495+0000", 2024-09-16T00:08:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001723863 2024-09-16T00:08:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166518+0000", 2024-09-16T00:08:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2682000000000001e-05 2024-09-16T00:08:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'66)", 2024-09-16T00:08:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.395544159000004, 2024-09-16T00:08:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032925509999999999, 2024-09-16T00:08:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:04.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166177+0000", 2024-09-16T00:08:04.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:04.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166216+0000", 2024-09-16T00:08:04.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9008000000000003e-05 2024-09-16T00:08:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166226+0000", 2024-09-16T00:08:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0081999999999999e-05 2024-09-16T00:08:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166230+0000", 2024-09-16T00:08:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5079999999999999e-06 2024-09-16T00:08:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.581 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166259+0000", 2024-09-16T00:08:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8928000000000002e-05 2024-09-16T00:08:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166271+0000", 2024-09-16T00:08:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1531e-05 2024-09-16T00:08:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169449+0000", 2024-09-16T00:08:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003178143 2024-09-16T00:08:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169468+0000", 2024-09-16T00:08:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9700000000000001e-05 2024-09-16T00:08:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'66)", 2024-09-16T00:08:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.395082700000003, 2024-09-16T00:08:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003179374, 2024-09-16T00:08:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.583 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166638+0000", 2024-09-16T00:08:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2300000000000002e-07 2024-09-16T00:08:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166641+0000", 2024-09-16T00:08:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0079999999999998e-06 2024-09-16T00:08:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166646+0000", 2024-09-16T00:08:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4970000000000001e-06 2024-09-16T00:08:04.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166650+0000", 2024-09-16T00:08:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5729999999999999e-06 2024-09-16T00:08:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166682+0000", 2024-09-16T00:08:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2218999999999998e-05 2024-09-16T00:08:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166694+0000", 2024-09-16T00:08:04.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1902999999999999e-05 2024-09-16T00:08:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169784+0000", 2024-09-16T00:08:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003089545 2024-09-16T00:08:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169817+0000", 2024-09-16T00:08:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3206e-05 2024-09-16T00:08:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:117 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'66)", 2024-09-16T00:08:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.394638639999997, 2024-09-16T00:08:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00275326, 2024-09-16T00:08:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167082+0000", 2024-09-16T00:08:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:08:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167086+0000", 2024-09-16T00:08:04.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1559999999999997e-06 2024-09-16T00:08:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167096+0000", 2024-09-16T00:08:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8390000000000002e-06 2024-09-16T00:08:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167101+0000", 2024-09-16T00:08:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:08:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167130+0000", 2024-09-16T00:08:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9649e-05 2024-09-16T00:08:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167142+0000", 2024-09-16T00:08:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1435e-05 2024-09-16T00:08:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169818+0000", 2024-09-16T00:08:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002676056 2024-09-16T00:08:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169835+0000", 2024-09-16T00:08:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6980999999999999e-05 2024-09-16T00:08:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:130 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'88, mlcod=17'85)", 2024-09-16T00:08:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 39.951839311000001, 2024-09-16T00:08:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002880456, 2024-09-16T00:08:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.592 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.593 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5699999999999998e-07 2024-09-16T00:08:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609885+0000", 2024-09-16T00:08:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.608e-06 2024-09-16T00:08:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609891+0000", 2024-09-16T00:08:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.594 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609895+0000", 2024-09-16T00:08:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.54e-06 2024-09-16T00:08:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609931+0000", 2024-09-16T00:08:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5893999999999999e-05 2024-09-16T00:08:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609945+0000", 2024-09-16T00:08:04.595 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4259e-05 2024-09-16T00:08:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612721+0000", 2024-09-16T00:08:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027757849999999998 2024-09-16T00:08:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612761+0000", 2024-09-16T00:08:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0638e-05 2024-09-16T00:08:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.596 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-16T00:08:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 39.951455338000002, 2024-09-16T00:08:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028016529999999999, 2024-09-16T00:08:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.597 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.63e-07 2024-09-16T00:08:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.598 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610344+0000", 2024-09-16T00:08:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8996000000000001e-05 2024-09-16T00:08:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610352+0000", 2024-09-16T00:08:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610356+0000", 2024-09-16T00:08:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0790000000000002e-06 2024-09-16T00:08:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.599 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610388+0000", 2024-09-16T00:08:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1541000000000002e-05 2024-09-16T00:08:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610400+0000", 2024-09-16T00:08:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2075e-05 2024-09-16T00:08:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613045+0000", 2024-09-16T00:08:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002645213 2024-09-16T00:08:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613066+0000", 2024-09-16T00:08:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1353e-05 2024-09-16T00:08:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-16T00:08:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 39.948195837999997, 2024-09-16T00:08:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027254850000000001, 2024-09-16T00:08:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.601 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613525+0000", 2024-09-16T00:08:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2800000000000002e-07 2024-09-16T00:08:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613528+0000", 2024-09-16T00:08:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9730000000000002e-06 2024-09-16T00:08:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613534+0000", 2024-09-16T00:08:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4339999999999997e-06 2024-09-16T00:08:04.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613538+0000", 2024-09-16T00:08:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5149999999999998e-06 2024-09-16T00:08:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613566+0000", 2024-09-16T00:08:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8104000000000001e-05 2024-09-16T00:08:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613578+0000", 2024-09-16T00:08:04.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2404e-05 2024-09-16T00:08:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616217+0000", 2024-09-16T00:08:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026389830000000001 2024-09-16T00:08:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616250+0000", 2024-09-16T00:08:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2644000000000001e-05 2024-09-16T00:08:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.605 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:134 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'92, mlcod=18'87)", 2024-09-16T00:08:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 39.947657698999997, 2024-09-16T00:08:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025322399999999998, 2024-09-16T00:08:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:04.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614063+0000", 2024-09-16T00:08:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8700000000000005e-07 2024-09-16T00:08:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.607 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614096+0000", 2024-09-16T00:08:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3136000000000003e-05 2024-09-16T00:08:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614106+0000", 2024-09-16T00:08:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9170000000000001e-06 2024-09-16T00:08:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614111+0000", 2024-09-16T00:08:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4390000000000002e-06 2024-09-16T00:08:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614141+0000", 2024-09-16T00:08:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.04e-05 2024-09-16T00:08:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614152+0000", 2024-09-16T00:08:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0987e-05 2024-09-16T00:08:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616573+0000", 2024-09-16T00:08:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002421351 2024-09-16T00:08:04.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616595+0000", 2024-09-16T00:08:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1322999999999999e-05 2024-09-16T00:08:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:135 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'93, mlcod=18'87)", 2024-09-16T00:08:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 39.947137660999999, 2024-09-16T00:08:04.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020304659999999999, 2024-09-16T00:08:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614583+0000", 2024-09-16T00:08:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614587+0000", 2024-09-16T00:08:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3100000000000002e-06 2024-09-16T00:08:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.612 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614597+0000", 2024-09-16T00:08:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7450000000000007e-06 2024-09-16T00:08:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614602+0000", 2024-09-16T00:08:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6099999999999999e-06 2024-09-16T00:08:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614641+0000", 2024-09-16T00:08:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9767000000000001e-05 2024-09-16T00:08:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.613 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614658+0000", 2024-09-16T00:08:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6205000000000002e-05 2024-09-16T00:08:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616596+0000", 2024-09-16T00:08:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019386659999999999 2024-09-16T00:08:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616613+0000", 2024-09-16T00:08:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6555999999999999e-05 2024-09-16T00:08:04.614 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:139 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'97, mlcod=18'87)", 2024-09-16T00:08:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 39.94537699, 2024-09-16T00:08:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030908139999999999, 2024-09-16T00:08:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.615 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616344+0000", 2024-09-16T00:08:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3000000000000001e-07 2024-09-16T00:08:04.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616347+0000", 2024-09-16T00:08:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7740000000000002e-06 2024-09-16T00:08:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616355+0000", 2024-09-16T00:08:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4699999999999996e-06 2024-09-16T00:08:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616358+0000", 2024-09-16T00:08:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5899999999999999e-06 2024-09-16T00:08:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616393+0000", 2024-09-16T00:08:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4529000000000001e-05 2024-09-16T00:08:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616404+0000", 2024-09-16T00:08:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0726e-05 2024-09-16T00:08:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619411+0000", 2024-09-16T00:08:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030076249999999999 2024-09-16T00:08:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619434+0000", 2024-09-16T00:08:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-05 2024-09-16T00:08:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.619 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:140 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'98, mlcod=18'89)", 2024-09-16T00:08:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 39.944735596999998, 2024-09-16T00:08:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027276150000000001, 2024-09-16T00:08:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616985+0000", 2024-09-16T00:08:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4099999999999999e-07 2024-09-16T00:08:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617055+0000", 2024-09-16T00:08:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9923000000000001e-05 2024-09-16T00:08:04.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617063+0000", 2024-09-16T00:08:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617070+0000", 2024-09-16T00:08:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9090000000000003e-06 2024-09-16T00:08:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.622 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617102+0000", 2024-09-16T00:08:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2620999999999998e-05 2024-09-16T00:08:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617114+0000", 2024-09-16T00:08:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1228e-05 2024-09-16T00:08:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619690+0000", 2024-09-16T00:08:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025763890000000001 2024-09-16T00:08:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619712+0000", 2024-09-16T00:08:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2171000000000002e-05 2024-09-16T00:08:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 25.945906496999999, 2024-09-16T00:08:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027355320000000002, 2024-09-16T00:08:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.624 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615815+0000", 2024-09-16T00:08:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1769999999999999e-06 2024-09-16T00:08:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615825+0000", 2024-09-16T00:08:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9000000000000001e-06 2024-09-16T00:08:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615840+0000", 2024-09-16T00:08:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4854e-05 2024-09-16T00:08:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615848+0000", 2024-09-16T00:08:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0940000000000008e-06 2024-09-16T00:08:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615926+0000", 2024-09-16T00:08:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8176999999999994e-05 2024-09-16T00:08:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615945+0000", 2024-09-16T00:08:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9663000000000001e-05 2024-09-16T00:08:04.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618509+0000", 2024-09-16T00:08:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002563802 2024-09-16T00:08:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618549+0000", 2024-09-16T00:08:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9864999999999999e-05 2024-09-16T00:08:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 25.945817356999999, 2024-09-16T00:08:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061593860000000002, 2024-09-16T00:08:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615907+0000", 2024-09-16T00:08:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9389999999999999e-06 2024-09-16T00:08:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615920+0000", 2024-09-16T00:08:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2989999999999999e-05 2024-09-16T00:08:04.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615931+0000", 2024-09-16T00:08:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.164e-05 2024-09-16T00:08:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615939+0000", 2024-09-16T00:08:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8029999999999997e-06 2024-09-16T00:08:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616310+0000", 2024-09-16T00:08:04.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00037111400000000003 2024-09-16T00:08:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616406+0000", 2024-09-16T00:08:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5372999999999996e-05 2024-09-16T00:08:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616514+0000", 2024-09-16T00:08:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108564 2024-09-16T00:08:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.632 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618158+0000", 2024-09-16T00:08:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016442449999999999 2024-09-16T00:08:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621972+0000", 2024-09-16T00:08:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038132299999999999 2024-09-16T00:08:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622018+0000", 2024-09-16T00:08:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6239000000000002e-05 2024-09-16T00:08:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622062+0000", 2024-09-16T00:08:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4249000000000003e-05 2024-09-16T00:08:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 25.945763888999998, 2024-09-16T00:08:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006258914, 2024-09-16T00:08:04.634 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615957+0000", 2024-09-16T00:08:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1399999999999997e-07 2024-09-16T00:08:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615960+0000", 2024-09-16T00:08:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0860000000000002e-06 2024-09-16T00:08:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615967+0000", 2024-09-16T00:08:04.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9059999999999996e-06 2024-09-16T00:08:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615972+0000", 2024-09-16T00:08:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5679999999999999e-06 2024-09-16T00:08:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616717+0000", 2024-09-16T00:08:04.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074438 2024-09-16T00:08:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616781+0000", 2024-09-16T00:08:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4808999999999996e-05 2024-09-16T00:08:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616885+0000", 2024-09-16T00:08:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010404900000000001 2024-09-16T00:08:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012773870000000001 2024-09-16T00:08:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622116+0000", 2024-09-16T00:08:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039533470000000003 2024-09-16T00:08:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622173+0000", 2024-09-16T00:08:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6347000000000001e-05 2024-09-16T00:08:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622215+0000", 2024-09-16T00:08:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2521000000000002e-05 2024-09-16T00:08:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 25.945740151999999, 2024-09-16T00:08:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041725529999999999, 2024-09-16T00:08:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.640 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8799999999999998e-07 2024-09-16T00:08:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615990+0000", 2024-09-16T00:08:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2150000000000007e-06 2024-09-16T00:08:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:08:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7159999999999997e-06 2024-09-16T00:08:04.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616001+0000", 2024-09-16T00:08:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9339999999999997e-06 2024-09-16T00:08:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616293+0000", 2024-09-16T00:08:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029210899999999998 2024-09-16T00:08:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616401+0000", 2024-09-16T00:08:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108087 2024-09-16T00:08:04.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616561+0000", 2024-09-16T00:08:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015940600000000001 2024-09-16T00:08:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618146+0000", 2024-09-16T00:08:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015857040000000001 2024-09-16T00:08:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:04.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620091+0000", 2024-09-16T00:08:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001944837 2024-09-16T00:08:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620126+0000", 2024-09-16T00:08:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4242999999999997e-05 2024-09-16T00:08:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620152+0000", 2024-09-16T00:08:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6914e-05 2024-09-16T00:08:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.645 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 25.945701927000002, 2024-09-16T00:08:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00630628, 2024-09-16T00:08:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.646 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:08:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5999999999999999e-07 2024-09-16T00:08:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616022+0000", 2024-09-16T00:08:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0149999999999999e-06 2024-09-16T00:08:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616031+0000", 2024-09-16T00:08:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9770000000000006e-06 2024-09-16T00:08:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616036+0000", 2024-09-16T00:08:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2639999999999999e-06 2024-09-16T00:08:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617057+0000", 2024-09-16T00:08:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010212999999999999 2024-09-16T00:08:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617146+0000", 2024-09-16T00:08:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9234000000000007e-05 2024-09-16T00:08:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617232+0000", 2024-09-16T00:08:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5218999999999998e-05 2024-09-16T00:08:04.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619107+0000", 2024-09-16T00:08:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001875016 2024-09-16T00:08:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622259+0000", 2024-09-16T00:08:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031527930000000001 2024-09-16T00:08:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.650 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622290+0000", 2024-09-16T00:08:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1022999999999998e-05 2024-09-16T00:08:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622324+0000", 2024-09-16T00:08:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3979000000000001e-05 2024-09-16T00:08:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:04.651 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 25.945676625000001, 2024-09-16T00:08:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042412739999999997, 2024-09-16T00:08:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.652 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616044+0000", 2024-09-16T00:08:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616052+0000", 2024-09-16T00:08:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.439e-06 2024-09-16T00:08:04.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616059+0000", 2024-09-16T00:08:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1389999999999998e-06 2024-09-16T00:08:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616065+0000", 2024-09-16T00:08:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:08:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616823+0000", 2024-09-16T00:08:04.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075781199999999996 2024-09-16T00:08:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616918+0000", 2024-09-16T00:08:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5174999999999996e-05 2024-09-16T00:08:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617019+0000", 2024-09-16T00:08:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101161 2024-09-16T00:08:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:04.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618503+0000", 2024-09-16T00:08:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014836750000000001 2024-09-16T00:08:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620201+0000", 2024-09-16T00:08:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00169805 2024-09-16T00:08:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620255+0000", 2024-09-16T00:08:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4499000000000001e-05 2024-09-16T00:08:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.656 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620285+0000", 2024-09-16T00:08:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9349e-05 2024-09-16T00:08:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 25.945647634, 2024-09-16T00:08:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063682799999999996, 2024-09-16T00:08:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:04.657 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9900000000000001e-07 2024-09-16T00:08:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616079+0000", 2024-09-16T00:08:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9009999999999999e-06 2024-09-16T00:08:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616086+0000", 2024-09-16T00:08:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.776e-06 2024-09-16T00:08:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616091+0000", 2024-09-16T00:08:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5600000000000001e-06 2024-09-16T00:08:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618339+0000", 2024-09-16T00:08:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022476179999999998 2024-09-16T00:08:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:04.660 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9025999999999997e-05 2024-09-16T00:08:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622441+0000", 2024-09-16T00:08:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040029999999999996 2024-09-16T00:08:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:04.661 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:04.661 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:04.663 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:08:04.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.572+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe3e00ce5f0 con 0x7fe3e0062f00 2024-09-16T00:08:04.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.574+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe3e00ce5f0 con 0x7fe3e0062f00 2024-09-16T00:08:04.664 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:04.665 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:04.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.574+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fe3e002c640 0x7fe3e002caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.574+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fe3e00ce5f0 con 0x7fe3e002c640 2024-09-16T00:08:04.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.575+0000 7fe3df7fe640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fe3e002c640 0x7fe3e002caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.575+0000 7fe3df7fe640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fe3e002c640 0x7fe3e002caa0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.576+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3e00ce5f0 con 0x7fe3e002c640 2024-09-16T00:08:04.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.589+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe3e00ce5f0 con 0x7fe3e002c640 2024-09-16T00:08:04.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.591+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe3e00ce5f0 con 0x7fe3e002c640 2024-09-16T00:08:04.665 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:04.666 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:04.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.591+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fe3e002d4f0 0x7fe3e0207260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.591+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fe3e002d930 con 0x7fe3e002d4f0 2024-09-16T00:08:04.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.592+0000 7fe3deffd640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fe3e002d4f0 0x7fe3e0207260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.592+0000 7fe3deffd640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fe3e002d4f0 0x7fe3e0207260 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.593+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3e002d930 con 0x7fe3e002d4f0 2024-09-16T00:08:04.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.606+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe3e002d930 con 0x7fe3e002d4f0 2024-09-16T00:08:04.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.608+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe3e002d930 con 0x7fe3e002d4f0 2024-09-16T00:08:04.667 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:04.668 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:04.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.608+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fe3e0207cb0 0x7fe3e020a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.608+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fe3e002db20 con 0x7fe3e0207cb0 2024-09-16T00:08:04.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.609+0000 7fe3dffff640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fe3e0207cb0 0x7fe3e020a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.609+0000 7fe3dffff640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fe3e0207cb0 0x7fe3e020a0a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.610+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3e002db20 con 0x7fe3e0207cb0 2024-09-16T00:08:04.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.623+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe3e002db20 con 0x7fe3e0207cb0 2024-09-16T00:08:04.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.625+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe3e002db20 con 0x7fe3e0207cb0 2024-09-16T00:08:04.669 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:04.670 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:04.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.625+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fe3e020aaf0 0x7fe3e020af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.625+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fe3e0210080 con 0x7fe3e020aaf0 2024-09-16T00:08:04.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.625+0000 7fe3df7fe640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fe3e020aaf0 0x7fe3e020af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.626+0000 7fe3df7fe640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fe3e020aaf0 0x7fe3e020af30 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.626+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe3e0210080 con 0x7fe3e020aaf0 2024-09-16T00:08:04.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.640+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fe3e020b4c0 con 0x7fe3e020aaf0 2024-09-16T00:08:04.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3c7fff640 1 -- 172.21.15.139:0/1462160954 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe3e020b4c0 con 0x7fe3e020aaf0 2024-09-16T00:08:04.671 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:04.673 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:04.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe3e014f940 msgr2=0x7fe3e01f4770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe3e014f940 0x7fe3e01f4770 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe3e0147c60 msgr2=0x7fe3e0148040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe3e0147c60 0x7fe3e0148040 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fe3e0153690 msgr2=0x7fe3e0153a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fe3e0153690 0x7fe3e0153a70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fe3e0062f00 msgr2=0x7fe3e00ce0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fe3e0062f00 0x7fe3e00ce0b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fe3e002c640 msgr2=0x7fe3e002caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fe3e002c640 0x7fe3e002caa0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fe3e002d4f0 msgr2=0x7fe3e0207260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fe3e002d4f0 0x7fe3e0207260 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3dffff640 1 -- 172.21.15.139:0/1462160954 reap_dead start 2024-09-16T00:08:04.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fe3e0207cb0 msgr2=0x7fe3e020a0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fe3e0207cb0 0x7fe3e020a0a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3dffff640 1 -- 172.21.15.139:0/1462160954 reap_dead start 2024-09-16T00:08:04.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3dffff640 1 -- 172.21.15.139:0/1462160954 reap_dead start 2024-09-16T00:08:04.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3dffff640 1 -- 172.21.15.139:0/1462160954 reap_dead start 2024-09-16T00:08:04.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fe3e020aaf0 msgr2=0x7fe3e020af30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fe3e020aaf0 0x7fe3e020af30 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe3cc06c2b0 msgr2=0x7fe3cc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe3cc06c2b0 0x7fe3cc06e770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4034000 tx=0x7fe3d402e000 comp rx=0 tx=0).stop 2024-09-16T00:08:04.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e00b3430 msgr2=0x7fe3e00cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e00b3430 0x7fe3e00cf8c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000c460 tx=0x7fe3d000c920 comp rx=0 tx=0).stop 2024-09-16T00:08:04.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 shutdown_connections 2024-09-16T00:08:04.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fe3e020aaf0 0x7fe3e020af30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe3cc06c2b0 0x7fe3cc06e770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 --2- 172.21.15.139:0/1462160954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3e00b3430 0x7fe3e00cf8c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.642+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 >> 172.21.15.139:0/1462160954 conn(0x7fe3e006c520 msgr2=0x7fe3e00cb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:04.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.643+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 shutdown_connections 2024-09-16T00:08:04.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.643+0000 7fe3e5f3b640 1 -- 172.21.15.139:0/1462160954 wait complete. 2024-09-16T00:08:04.676 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:04.662+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:04.738 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T00:08:04.791 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:04.790+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:04.809 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:04.809 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:04.810 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:04.810 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:04.819 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:08:04.821 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T00:08:04.821 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:08:04.822 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-16T00:08:04.822 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-16T00:08:04.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.847+0000 7fb4c428d640 1 Processor -- start 2024-09-16T00:08:04.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.847+0000 7fb4c428d640 1 -- start start 2024-09-16T00:08:04.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.848+0000 7fb4c428d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb4bc0cc720 0x7fb4bc0c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.848+0000 7fb4c428d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4bc0c7920 0x7fb4bc154490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.848+0000 7fb4c428d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4bc0c7da0 con 0x7fb4bc0ccb00 2024-09-16T00:08:04.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.848+0000 7fb4c428d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb4bc072430 con 0x7fb4bc0cc720 2024-09-16T00:08:04.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.848+0000 7fb4c428d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb4bc0726d0 con 0x7fb4bc0c7920 2024-09-16T00:08:04.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.848+0000 7fb4c282a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4bc0c7920 0x7fb4bc154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.848+0000 7fb4c2029640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb4bc0cc720 0x7fb4bc0c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.848+0000 7fb4c282a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4bc0c7920 0x7fb4bc154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:52804/0 (socket says 172.21.15.139:52804) 2024-09-16T00:08:04.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.848+0000 7fb4c2029640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb4bc0cc720 0x7fb4bc0c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:34454/0 (socket says 172.21.15.139:34454) 2024-09-16T00:08:04.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.848+0000 7fb4c282a640 1 -- 172.21.15.139:0/1223546833 learned_addr learned my addr 172.21.15.139:0/1223546833 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:04.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.848+0000 7fb4c1828640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb4bc0ccb00 0x7fb4bc0c7210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:59142/0 (socket says 172.21.15.139:59142) 2024-09-16T00:08:04.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.849+0000 7fb4c2029640 1 -- 172.21.15.139:0/1223546833 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4bc0c7920 msgr2=0x7fb4bc154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:04.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.849+0000 7fb4c2029640 1 --2- 172.21.15.139:0/1223546833 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4bc0c7920 0x7fb4bc154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.849+0000 7fb4c2029640 1 -- 172.21.15.139:0/1223546833 >> v1:172.21.15.164:6789/0 conn(0x7fb4bc0ccb00 legacy=0x7fb4bc0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.849+0000 7fb4c1027640 1 -- 172.21.15.139:0/1223546833 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3494895162 0 0) 0x7fb4bc0c7da0 con 0x7fb4bc0ccb00 2024-09-16T00:08:04.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.849+0000 7fb4c2029640 1 -- 172.21.15.139:0/1223546833 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc0729b0 con 0x7fb4bc0cc720 2024-09-16T00:08:04.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.849+0000 7fb4c2029640 1 --2- 172.21.15.139:0/1223546833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4bc0cc720 0x7fb4bc0c81a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80097a0 tx=0x7fb4b802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=580cd82586530c41 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.849+0000 7fb4c1027640 1 -- 172.21.15.139:0/1223546833 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8037070 con 0x7fb4bc0cc720 2024-09-16T00:08:04.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.850+0000 7fb4c1027640 1 -- 172.21.15.139:0/1223546833 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b8032440 con 0x7fb4bc0cc720 2024-09-16T00:08:04.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.850+0000 7fb4c1027640 1 -- 172.21.15.139:0/1223546833 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8032810 con 0x7fb4bc0cc720 2024-09-16T00:08:04.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.850+0000 7fb4c428d640 1 -- 172.21.15.139:0/1223546833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4bc0cc720 msgr2=0x7fb4bc0c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.850+0000 7fb4c428d640 1 --2- 172.21.15.139:0/1223546833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4bc0cc720 0x7fb4bc0c81a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80097a0 tx=0x7fb4b802a180 comp rx=0 tx=0).stop 2024-09-16T00:08:04.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.850+0000 7fb4c428d640 1 -- 172.21.15.139:0/1223546833 shutdown_connections 2024-09-16T00:08:04.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.850+0000 7fb4c428d640 1 --2- 172.21.15.139:0/1223546833 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4bc0c7920 0x7fb4bc154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.850+0000 7fb4c428d640 1 --2- 172.21.15.139:0/1223546833 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4bc0cc720 0x7fb4bc0c81a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.850+0000 7fb4c428d640 1 -- 172.21.15.139:0/1223546833 >> 172.21.15.139:0/1223546833 conn(0x7fb4bc067da0 msgr2=0x7fb4bc06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:04.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.850+0000 7fb4c428d640 1 -- 172.21.15.139:0/1223546833 shutdown_connections 2024-09-16T00:08:04.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.850+0000 7fb4c428d640 1 -- 172.21.15.139:0/1223546833 wait complete. 2024-09-16T00:08:04.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.851+0000 7fb4c428d640 1 Processor -- start 2024-09-16T00:08:04.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.852+0000 7fb4c428d640 1 -- start start 2024-09-16T00:08:04.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.852+0000 7fb4c428d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4bc0c7920 0x7fb4bc0c05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.852+0000 7fb4c428d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4bc0cc720 0x7fb4bc1f0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.852+0000 7fb4c428d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4bc0ccb00 0x7fb4bc0bdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.852+0000 7fb4c428d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc073490 con 0x7fb4bc0c7920 2024-09-16T00:08:04.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.852+0000 7fb4c428d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc072f70 con 0x7fb4bc0cc720 2024-09-16T00:08:04.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.852+0000 7fb4c428d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb4bc073610 con 0x7fb4bc0ccb00 2024-09-16T00:08:04.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.852+0000 7fb4c2029640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4bc0c7920 0x7fb4bc0c05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.852+0000 7fb4c1828640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4bc0cc720 0x7fb4bc1f0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.852+0000 7fb4c2029640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4bc0c7920 0x7fb4bc0c05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58978/0 (socket says 172.21.15.139:58978) 2024-09-16T00:08:04.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.852+0000 7fb4c282a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4bc0ccb00 0x7fb4bc0bdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.852+0000 7fb4c2029640 1 -- 172.21.15.139:0/888251408 learned_addr learned my addr 172.21.15.139:0/888251408 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:04.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.853+0000 7fb4c282a640 1 -- 172.21.15.139:0/888251408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4bc0cc720 msgr2=0x7fb4bc1f0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.853+0000 7fb4c282a640 1 --2- 172.21.15.139:0/888251408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4bc0cc720 0x7fb4bc1f0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.853+0000 7fb4c282a640 1 -- 172.21.15.139:0/888251408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4bc0c7920 msgr2=0x7fb4bc0c05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.853+0000 7fb4c282a640 1 --2- 172.21.15.139:0/888251408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4bc0c7920 0x7fb4bc0c05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.853+0000 7fb4c282a640 1 -- 172.21.15.139:0/888251408 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc1e5b10 con 0x7fb4bc0ccb00 2024-09-16T00:08:04.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.853+0000 7fb4c2029640 1 --2- 172.21.15.139:0/888251408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4bc0c7920 0x7fb4bc0c05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:04.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.853+0000 7fb4c282a640 1 --2- 172.21.15.139:0/888251408 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4bc0ccb00 0x7fb4bc0bdb00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400ebd0 tx=0x7fb4b400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.853+0000 7fb4b2ffd640 1 -- 172.21.15.139:0/888251408 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400cb20 con 0x7fb4bc0ccb00 2024-09-16T00:08:04.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.853+0000 7fb4b2ffd640 1 -- 172.21.15.139:0/888251408 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b400ccc0 con 0x7fb4bc0ccb00 2024-09-16T00:08:04.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.853+0000 7fb4c428d640 1 -- 172.21.15.139:0/888251408 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4bc1e5df0 con 0x7fb4bc0ccb00 2024-09-16T00:08:04.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.853+0000 7fb4b2ffd640 1 -- 172.21.15.139:0/888251408 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4003bb0 con 0x7fb4bc0ccb00 2024-09-16T00:08:04.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.853+0000 7fb4c428d640 1 -- 172.21.15.139:0/888251408 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4bc0be730 con 0x7fb4bc0ccb00 2024-09-16T00:08:04.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.854+0000 7fb4c428d640 1 -- 172.21.15.139:0/888251408 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4bc15e930 con 0x7fb4bc0ccb00 2024-09-16T00:08:04.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.855+0000 7fb4b2ffd640 1 -- 172.21.15.139:0/888251408 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb4b4012050 con 0x7fb4bc0ccb00 2024-09-16T00:08:04.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.855+0000 7fb4b2ffd640 1 --2- 172.21.15.139:0/888251408 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb4a006c2b0 0x7fb4a006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.856+0000 7fb4b2ffd640 1 -- 172.21.15.139:0/888251408 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(28..28 src has 1..28) ==== 5738+0+0 (secure 0 0 0) 0x7fb4b408b6a0 con 0x7fb4bc0ccb00 2024-09-16T00:08:04.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.856+0000 7fb4c2029640 1 --2- 172.21.15.139:0/888251408 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb4a006c2b0 0x7fb4a006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.856+0000 7fb4c2029640 1 --2- 172.21.15.139:0/888251408 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb4a006c2b0 0x7fb4a006e770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb4b802a1c0 tx=0x7fb4b802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.860+0000 7fb4b2ffd640 1 -- 172.21.15.139:0/888251408 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb4b4010040 con 0x7fb4bc0ccb00 2024-09-16T00:08:04.864 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:04.864+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:04.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.951+0000 7f50be359640 1 Processor -- start 2024-09-16T00:08:04.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.951+0000 7f50be359640 1 -- start start 2024-09-16T00:08:04.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.952+0000 7f50be359640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f50b8068060 0x7f50b815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.952+0000 7f50be359640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50b80ccc90 0x7f50b80cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.952+0000 7f50be359640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50b80cc4b0 con 0x7f50b8154c20 2024-09-16T00:08:04.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.952+0000 7f50be359640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f50b80c5830 con 0x7f50b8068060 2024-09-16T00:08:04.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.952+0000 7f50be359640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f50b815b7c0 con 0x7f50b80ccc90 2024-09-16T00:08:04.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.952+0000 7f50b77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f50b8068060 0x7f50b815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.952+0000 7f50b77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f50b8068060 0x7f50b815b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58994/0 (socket says 172.21.15.139:58994) 2024-09-16T00:08:04.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.952+0000 7f50b77fe640 1 -- 172.21.15.139:0/2802804149 learned_addr learned my addr 172.21.15.139:0/2802804149 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:04.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.952+0000 7f50bc8f6640 1 --2- 172.21.15.139:0/2802804149 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50b80ccc90 0x7f50b80cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.952+0000 7f50b77fe640 1 -- 172.21.15.139:0/2802804149 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50b80ccc90 msgr2=0x7f50b80cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:04.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.952+0000 7f50b77fe640 1 --2- 172.21.15.139:0/2802804149 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50b80ccc90 0x7f50b80cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.952+0000 7f50b77fe640 1 -- 172.21.15.139:0/2802804149 >> v1:172.21.15.164:6789/0 conn(0x7f50b8154c20 legacy=0x7f50b80c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.952+0000 7f50b77fe640 1 -- 172.21.15.139:0/2802804149 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b80cc790 con 0x7f50b8068060 2024-09-16T00:08:04.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.952+0000 7f50b6ffd640 1 -- 172.21.15.139:0/2802804149 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1654849129 0 0) 0x7f50b80cc4b0 con 0x7f50b8154c20 2024-09-16T00:08:04.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.953+0000 7f50b77fe640 1 --2- 172.21.15.139:0/2802804149 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b8068060 0x7f50b815b280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009850 tx=0x7f50ac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348cd95b43f16027 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.953+0000 7f50b6ffd640 1 -- 172.21.15.139:0/2802804149 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac004030 con 0x7f50b8068060 2024-09-16T00:08:04.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.953+0000 7f50b6ffd640 1 -- 172.21.15.139:0/2802804149 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ac0041d0 con 0x7f50b8068060 2024-09-16T00:08:04.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.953+0000 7f50b6ffd640 1 -- 172.21.15.139:0/2802804149 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac003610 con 0x7f50b8068060 2024-09-16T00:08:04.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.954+0000 7f50be359640 1 -- 172.21.15.139:0/2802804149 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b8068060 msgr2=0x7f50b815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.954+0000 7f50be359640 1 --2- 172.21.15.139:0/2802804149 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b8068060 0x7f50b815b280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009850 tx=0x7f50ac02a230 comp rx=0 tx=0).stop 2024-09-16T00:08:04.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.954+0000 7f50be359640 1 -- 172.21.15.139:0/2802804149 shutdown_connections 2024-09-16T00:08:04.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.954+0000 7f50be359640 1 --2- 172.21.15.139:0/2802804149 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50b80ccc90 0x7f50b80cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.954+0000 7f50be359640 1 --2- 172.21.15.139:0/2802804149 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b8068060 0x7f50b815b280 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.954+0000 7f50be359640 1 -- 172.21.15.139:0/2802804149 >> 172.21.15.139:0/2802804149 conn(0x7f50b806ca40 msgr2=0x7f50b8067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:04.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.954+0000 7f50be359640 1 -- 172.21.15.139:0/2802804149 shutdown_connections 2024-09-16T00:08:04.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.954+0000 7f50be359640 1 -- 172.21.15.139:0/2802804149 wait complete. 2024-09-16T00:08:04.957 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:04.957 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:04.957 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:04.957 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:04.957 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:04.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.956+0000 7f50be359640 1 Processor -- start 2024-09-16T00:08:04.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.956+0000 7f50be359640 1 -- start start 2024-09-16T00:08:04.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50be359640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50b8068060 0x7f50b8158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50be359640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b80ccc90 0x7f50b80c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50be359640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50b8154c20 0x7f50b80cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50be359640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f50b815e0d0 con 0x7f50b80ccc90 2024-09-16T00:08:04.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50be359640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f50b80cc260 con 0x7f50b8154c20 2024-09-16T00:08:04.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50be359640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f50b815e3b0 con 0x7f50b8068060 2024-09-16T00:08:04.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50b77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b80ccc90 0x7f50b80c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50b77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b80ccc90 0x7f50b80c5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59000/0 (socket says 172.21.15.139:59000) 2024-09-16T00:08:04.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50b77fe640 1 -- 172.21.15.139:0/1872163800 learned_addr learned my addr 172.21.15.139:0/1872163800 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:04.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50b7fff640 1 --2- 172.21.15.139:0/1872163800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50b8068060 0x7f50b8158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50bc8f6640 1 --2- 172.21.15.139:0/1872163800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50b8154c20 0x7f50b80cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50b77fe640 1 -- 172.21.15.139:0/1872163800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50b8068060 msgr2=0x7f50b8158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50b77fe640 1 --2- 172.21.15.139:0/1872163800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50b8068060 0x7f50b8158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50b77fe640 1 -- 172.21.15.139:0/1872163800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50b8154c20 msgr2=0x7f50b80cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:04.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50b77fe640 1 --2- 172.21.15.139:0/1872163800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50b8154c20 0x7f50b80cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:04.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50b77fe640 1 -- 172.21.15.139:0/1872163800 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b8158d00 con 0x7f50b80ccc90 2024-09-16T00:08:04.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50b7fff640 1 --2- 172.21.15.139:0/1872163800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50b8068060 0x7f50b8158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:04.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.957+0000 7f50bc8f6640 1 --2- 172.21.15.139:0/1872163800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50b8154c20 0x7f50b80cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:04.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.958+0000 7f50b77fe640 1 --2- 172.21.15.139:0/1872163800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b80ccc90 0x7f50b80c5c20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009970 tx=0x7f50ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.958+0000 7f50b4ff9640 1 -- 172.21.15.139:0/1872163800 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac041030 con 0x7f50b80ccc90 2024-09-16T00:08:04.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.958+0000 7f50b4ff9640 1 -- 172.21.15.139:0/1872163800 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ac003c50 con 0x7f50b80ccc90 2024-09-16T00:08:04.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.958+0000 7f50be359640 1 -- 172.21.15.139:0/1872163800 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b80ce100 con 0x7f50b80ccc90 2024-09-16T00:08:04.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.958+0000 7f50b4ff9640 1 -- 172.21.15.139:0/1872163800 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac004030 con 0x7f50b80ccc90 2024-09-16T00:08:04.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.958+0000 7f50be359640 1 -- 172.21.15.139:0/1872163800 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b80ce6c0 con 0x7f50b80ccc90 2024-09-16T00:08:04.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.959+0000 7f50be359640 1 -- 172.21.15.139:0/1872163800 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b80ced80 con 0x7f50b80ccc90 2024-09-16T00:08:04.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.960+0000 7f50b4ff9640 1 -- 172.21.15.139:0/1872163800 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f50ac044070 con 0x7f50b80ccc90 2024-09-16T00:08:04.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.960+0000 7f50b4ff9640 1 --2- 172.21.15.139:0/1872163800 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f50a406c2b0 0x7f50a406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:04.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.961+0000 7f50b4ff9640 1 -- 172.21.15.139:0/1872163800 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5738+0+0 (secure 0 0 0) 0x7f50ac0abcb0 con 0x7f50b80ccc90 2024-09-16T00:08:04.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.961+0000 7f50b7fff640 1 --2- 172.21.15.139:0/1872163800 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f50a406c2b0 0x7f50a406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:04.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.961+0000 7f50b7fff640 1 --2- 172.21.15.139:0/1872163800 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f50a406c2b0 0x7f50a406e770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f50a8000f60 tx=0x7f50a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:04.964 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:04.964+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:04.966 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:08:04.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:04.965+0000 7f50b4ff9640 1 -- 172.21.15.139:0/1872163800 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f50b80ced80 con 0x7f50b80ccc90 2024-09-16T00:08:05.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.046+0000 7fb4c428d640 1 -- 172.21.15.139:0/888251408 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7fb4bc0be4e0 con 0x7fb4bc0ccb00 2024-09-16T00:08:05.066 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:05.066+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:05.112 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:05.112 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:05.112 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:05.112 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:05.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.113 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:05.113 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:05.113 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.495105915, 2024-09-16T00:08:05.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037514929999999998, 2024-09-16T00:08:05.113 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.113 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.113 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.113 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.113 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:05.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.114 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:05.114 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.114 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615364+0000", 2024-09-16T00:08:05.114 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8430000000000003e-06 2024-09-16T00:08:05.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.114 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615371+0000", 2024-09-16T00:08:05.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.148e-06 2024-09-16T00:08:05.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615380+0000", 2024-09-16T00:08:05.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5889999999999992e-06 2024-09-16T00:08:05.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615389+0000", 2024-09-16T00:08:05.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1620000000000008e-06 2024-09-16T00:08:05.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615525+0000", 2024-09-16T00:08:05.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000136083 2024-09-16T00:08:05.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615662+0000", 2024-09-16T00:08:05.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013696100000000001 2024-09-16T00:08:05.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:05.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615799+0000", 2024-09-16T00:08:05.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013781399999999999 2024-09-16T00:08:05.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:05.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:05.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002363502 2024-09-16T00:08:05.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:05.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619018+0000", 2024-09-16T00:08:05.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085522199999999999 2024-09-16T00:08:05.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619068+0000", 2024-09-16T00:08:05.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9545000000000001e-05 2024-09-16T00:08:05.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619111+0000", 2024-09-16T00:08:05.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3624000000000001e-05 2024-09-16T00:08:05.118 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.118 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.118 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.118 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:05.119 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:05.119 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.495053844000001, 2024-09-16T00:08:05.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043656110000000001, 2024-09-16T00:08:05.119 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.119 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.119 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.119 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.119 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.119 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:05.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.119 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:05.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:05.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2199999999999999e-07 2024-09-16T00:08:05.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615416+0000", 2024-09-16T00:08:05.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5590000000000001e-06 2024-09-16T00:08:05.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615423+0000", 2024-09-16T00:08:05.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6250000000000001e-06 2024-09-16T00:08:05.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615426+0000", 2024-09-16T00:08:05.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2540000000000001e-06 2024-09-16T00:08:05.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616075+0000", 2024-09-16T00:08:05.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064968800000000004 2024-09-16T00:08:05.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616142+0000", 2024-09-16T00:08:05.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6203000000000003e-05 2024-09-16T00:08:05.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.123 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:05.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616247+0000", 2024-09-16T00:08:05.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010572499999999999 2024-09-16T00:08:05.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.123 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:05.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618913+0000", 2024-09-16T00:08:05.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026654339999999999 2024-09-16T00:08:05.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.124 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:05.124 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619712+0000", 2024-09-16T00:08:05.124 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079885199999999996 2024-09-16T00:08:05.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.125 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619744+0000", 2024-09-16T00:08:05.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2790000000000003e-05 2024-09-16T00:08:05.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.125 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619778+0000", 2024-09-16T00:08:05.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3059000000000003e-05 2024-09-16T00:08:05.125 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.125 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.126 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.126 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:05.126 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:05.126 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.495017765, 2024-09-16T00:08:05.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087673219999999993, 2024-09-16T00:08:05.126 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.126 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.126 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.126 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.127 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:05.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.127 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.127 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:05.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.127 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.127 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:05.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:05.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:08:05.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5392000000000001e-05 2024-09-16T00:08:05.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615470+0000", 2024-09-16T00:08:05.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1999999999999999e-06 2024-09-16T00:08:05.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615474+0000", 2024-09-16T00:08:05.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7359999999999999e-06 2024-09-16T00:08:05.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.129 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.129 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617614+0000", 2024-09-16T00:08:05.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021398189999999998 2024-09-16T00:08:05.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.129 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.129 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617724+0000", 2024-09-16T00:08:05.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110808 2024-09-16T00:08:05.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.129 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:05.129 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617856+0000", 2024-09-16T00:08:05.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131676 2024-09-16T00:08:05.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:05.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:05.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029629280000000001 2024-09-16T00:08:05.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:05.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624158+0000", 2024-09-16T00:08:05.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033386549999999998 2024-09-16T00:08:05.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624189+0000", 2024-09-16T00:08:05.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1508999999999999e-05 2024-09-16T00:08:05.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624215+0000", 2024-09-16T00:08:05.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6228999999999999e-05 2024-09-16T00:08:05.131 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.131 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.131 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.132 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:05.132 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:05.132 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.494983749999999, 2024-09-16T00:08:05.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053285279999999999, 2024-09-16T00:08:05.132 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.132 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.132 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.132 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:05.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.133 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:05.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.133 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:05.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5600000000000002e-07 2024-09-16T00:08:05.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.133 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615484+0000", 2024-09-16T00:08:05.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.688e-06 2024-09-16T00:08:05.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615488+0000", 2024-09-16T00:08:05.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.658e-06 2024-09-16T00:08:05.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615490+0000", 2024-09-16T00:08:05.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.728e-06 2024-09-16T00:08:05.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.135 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616441+0000", 2024-09-16T00:08:05.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00095065500000000001 2024-09-16T00:08:05.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.135 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.135 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616539+0000", 2024-09-16T00:08:05.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8335e-05 2024-09-16T00:08:05.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.135 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:05.135 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616631+0000", 2024-09-16T00:08:05.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1260000000000004e-05 2024-09-16T00:08:05.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.136 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:05.136 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619837+0000", 2024-09-16T00:08:05.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003206899 2024-09-16T00:08:05.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.136 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:05.136 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620725+0000", 2024-09-16T00:08:05.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088713599999999996 2024-09-16T00:08:05.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620772+0000", 2024-09-16T00:08:05.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7639999999999998e-05 2024-09-16T00:08:05.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620811+0000", 2024-09-16T00:08:05.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8272999999999998e-05 2024-09-16T00:08:05.137 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.137 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.137 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.138 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:05.138 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:05.138 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.494568661999999, 2024-09-16T00:08:05.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029886169999999998, 2024-09-16T00:08:05.138 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.138 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.138 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.138 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:05.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:05.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615898+0000", 2024-09-16T00:08:05.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.237e-06 2024-09-16T00:08:05.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.140 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.140 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.140 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615904+0000", 2024-09-16T00:08:05.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:05.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.140 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.140 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.140 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615915+0000", 2024-09-16T00:08:05.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1003e-05 2024-09-16T00:08:05.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.140 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.140 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.140 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615922+0000", 2024-09-16T00:08:05.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9959999999999996e-06 2024-09-16T00:08:05.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615988+0000", 2024-09-16T00:08:05.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5806e-05 2024-09-16T00:08:05.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616009+0000", 2024-09-16T00:08:05.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0729000000000001e-05 2024-09-16T00:08:05.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:05.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028462130000000002 2024-09-16T00:08:05.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618886+0000", 2024-09-16T00:08:05.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.082e-05 2024-09-16T00:08:05.142 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.142 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.142 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.142 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:05.142 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:05.142 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.492957189999998, 2024-09-16T00:08:05.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032610299999999998, 2024-09-16T00:08:05.143 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.143 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.143 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.143 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:05.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:05.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617556+0000", 2024-09-16T00:08:05.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7431e-05 2024-09-16T00:08:05.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617847+0000", 2024-09-16T00:08:05.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029133400000000003 2024-09-16T00:08:05.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617858+0000", 2024-09-16T00:08:05.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0222e-05 2024-09-16T00:08:05.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.145 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.145 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617864+0000", 2024-09-16T00:08:05.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7619999999999998e-06 2024-09-16T00:08:05.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.145 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.145 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617909+0000", 2024-09-16T00:08:05.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4747999999999999e-05 2024-09-16T00:08:05.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.145 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617927+0000", 2024-09-16T00:08:05.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7785999999999999e-05 2024-09-16T00:08:05.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620734+0000", 2024-09-16T00:08:05.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002807305 2024-09-16T00:08:05.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620770+0000", 2024-09-16T00:08:05.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5441999999999997e-05 2024-09-16T00:08:05.146 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.146 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.147 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.147 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:05.147 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:05.147 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.492536506, 2024-09-16T00:08:05.147 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028641809999999999, 2024-09-16T00:08:05.147 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.147 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.147 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.148 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:05.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:05.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617930+0000", 2024-09-16T00:08:05.149 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0200000000000002e-07 2024-09-16T00:08:05.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.149 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617934+0000", 2024-09-16T00:08:05.149 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:05.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.149 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617943+0000", 2024-09-16T00:08:05.149 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5820000000000002e-06 2024-09-16T00:08:05.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.150 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617949+0000", 2024-09-16T00:08:05.150 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5069999999999997e-06 2024-09-16T00:08:05.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.150 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.150 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.150 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618094+0000", 2024-09-16T00:08:05.150 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014515599999999999 2024-09-16T00:08:05.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.150 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.150 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.150 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618111+0000", 2024-09-16T00:08:05.150 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6705999999999999e-05 2024-09-16T00:08:05.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.151 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.151 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.151 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:08:05.151 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026603070000000002 2024-09-16T00:08:05.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.151 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.151 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.151 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620793+0000", 2024-09-16T00:08:05.151 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2214000000000001e-05 2024-09-16T00:08:05.151 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.151 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.151 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.152 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.152 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:05.152 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:05.152 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.492505717, 2024-09-16T00:08:05.152 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028624549999999999, 2024-09-16T00:08:05.152 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.152 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.152 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.152 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.152 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.152 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.152 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:05.152 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.152 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.153 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.153 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.153 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:05.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.153 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.153 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.153 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:05.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:05.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.153 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.153 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.153 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:05.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0570000000000001e-06 2024-09-16T00:08:05.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.154 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.154 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.154 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617970+0000", 2024-09-16T00:08:05.154 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.669e-06 2024-09-16T00:08:05.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.154 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.154 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.154 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617976+0000", 2024-09-16T00:08:05.154 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1369999999999999e-06 2024-09-16T00:08:05.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.154 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.154 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.154 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618268+0000", 2024-09-16T00:08:05.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029164600000000001 2024-09-16T00:08:05.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618284+0000", 2024-09-16T00:08:05.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5824e-05 2024-09-16T00:08:05.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:05.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025127890000000001 2024-09-16T00:08:05.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.156 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620822+0000", 2024-09-16T00:08:05.156 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6009000000000001e-05 2024-09-16T00:08:05.156 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.156 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.156 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.156 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.156 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-16T00:08:05.156 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:05.156 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.492478135999999, 2024-09-16T00:08:05.156 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028650350000000002, 2024-09-16T00:08:05.156 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.156 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.157 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.157 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:05.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:05.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:05.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7599999999999998e-07 2024-09-16T00:08:05.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.158 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.158 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617991+0000", 2024-09-16T00:08:05.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1860000000000001e-06 2024-09-16T00:08:05.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.158 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.158 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617999+0000", 2024-09-16T00:08:05.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3250000000000002e-06 2024-09-16T00:08:05.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.159 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.159 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618004+0000", 2024-09-16T00:08:05.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7300000000000002e-06 2024-09-16T00:08:05.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.159 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.159 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.159 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618422+0000", 2024-09-16T00:08:05.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00041763199999999998 2024-09-16T00:08:05.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.159 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.159 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.159 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:05.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6112000000000001e-05 2024-09-16T00:08:05.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.160 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.160 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620824+0000", 2024-09-16T00:08:05.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002385954 2024-09-16T00:08:05.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.160 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.160 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620853+0000", 2024-09-16T00:08:05.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8719999999999999e-05 2024-09-16T00:08:05.160 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.160 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.160 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.161 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-16T00:08:05.161 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:05.161 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.492452314000001, 2024-09-16T00:08:05.161 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032703020000000001, 2024-09-16T00:08:05.161 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.161 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.161 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.161 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.161 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.161 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.161 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:05.161 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.161 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:05.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618014+0000", 2024-09-16T00:08:05.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7799999999999997e-07 2024-09-16T00:08:05.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618017+0000", 2024-09-16T00:08:05.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4309999999999998e-06 2024-09-16T00:08:05.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618023+0000", 2024-09-16T00:08:05.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8490000000000002e-06 2024-09-16T00:08:05.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618028+0000", 2024-09-16T00:08:05.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3489999999999999e-06 2024-09-16T00:08:05.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:08:05.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055724099999999999 2024-09-16T00:08:05.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.164 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.164 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.164 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618603+0000", 2024-09-16T00:08:05.164 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7486000000000002e-05 2024-09-16T00:08:05.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.164 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.164 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.164 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621235+0000", 2024-09-16T00:08:05.164 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002632087 2024-09-16T00:08:05.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.164 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.164 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.164 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621284+0000", 2024-09-16T00:08:05.165 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8581000000000001e-05 2024-09-16T00:08:05.165 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.165 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.165 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.165 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:14 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:05.165 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:05.165 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.491515333999999, 2024-09-16T00:08:05.165 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016807599999999999, 2024-09-16T00:08:05.165 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.165 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.165 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:05.165 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:05.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:05.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618951+0000", 2024-09-16T00:08:05.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:05.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618954+0000", 2024-09-16T00:08:05.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.145e-06 2024-09-16T00:08:05.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618963+0000", 2024-09-16T00:08:05.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5720000000000003e-06 2024-09-16T00:08:05.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618969+0000", 2024-09-16T00:08:05.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3319999999999997e-06 2024-09-16T00:08:05.168 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.168 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.168 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.168 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619008+0000", 2024-09-16T00:08:05.168 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9096e-05 2024-09-16T00:08:05.168 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.168 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.168 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.168 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619013+0000", 2024-09-16T00:08:05.168 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4710000000000001e-06 2024-09-16T00:08:05.168 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.168 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.168 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.168 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619118+0000", 2024-09-16T00:08:05.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000105809 2024-09-16T00:08:05.169 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.169 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.169 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.169 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-16T00:08:05.169 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:05.169 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.491477227000001, 2024-09-16T00:08:05.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002325307, 2024-09-16T00:08:05.169 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.169 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.169 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.169 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.170 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.170 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:05.170 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.170 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.170 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.170 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:05.170 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.170 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.170 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.170 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:05.170 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:05.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618993+0000", 2024-09-16T00:08:05.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.687e-06 2024-09-16T00:08:05.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619000+0000", 2024-09-16T00:08:05.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3590000000000002e-06 2024-09-16T00:08:05.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619005+0000", 2024-09-16T00:08:05.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.95e-06 2024-09-16T00:08:05.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.172 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.172 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619039+0000", 2024-09-16T00:08:05.172 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3942999999999997e-05 2024-09-16T00:08:05.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.172 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.172 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619055+0000", 2024-09-16T00:08:05.172 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6107e-05 2024-09-16T00:08:05.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.172 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.172 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621287+0000", 2024-09-16T00:08:05.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022318070000000001 2024-09-16T00:08:05.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.173 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.173 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.173 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:05.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7129999999999999e-05 2024-09-16T00:08:05.173 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.173 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.173 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.173 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.173 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-16T00:08:05.173 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:05.174 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.491439231000001, 2024-09-16T00:08:05.174 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002314328, 2024-09-16T00:08:05.174 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.174 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.174 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.174 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.174 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.174 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.174 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:05.174 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.174 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.174 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.174 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.174 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:05.174 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:05.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6399999999999998e-07 2024-09-16T00:08:05.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619030+0000", 2024-09-16T00:08:05.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.219e-06 2024-09-16T00:08:05.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619038+0000", 2024-09-16T00:08:05.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9899999999999997e-06 2024-09-16T00:08:05.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619043+0000", 2024-09-16T00:08:05.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5580000000000002e-06 2024-09-16T00:08:05.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619220+0000", 2024-09-16T00:08:05.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000176937 2024-09-16T00:08:05.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619237+0000", 2024-09-16T00:08:05.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7691999999999999e-05 2024-09-16T00:08:05.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621315+0000", 2024-09-16T00:08:05.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020778900000000002 2024-09-16T00:08:05.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621341+0000", 2024-09-16T00:08:05.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5678e-05 2024-09-16T00:08:05.177 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.177 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.178 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.178 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:15 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:05.178 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:05.178 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.490824262, 2024-09-16T00:08:05.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143416, 2024-09-16T00:08:05.178 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.178 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.178 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:05.178 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:05.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.179 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:05.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.179 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619642+0000", 2024-09-16T00:08:05.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5199999999999995e-07 2024-09-16T00:08:05.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.179 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.180 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619645+0000", 2024-09-16T00:08:05.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.134e-06 2024-09-16T00:08:05.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.180 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619655+0000", 2024-09-16T00:08:05.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3330000000000003e-06 2024-09-16T00:08:05.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.180 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619662+0000", 2024-09-16T00:08:05.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0500000000000003e-06 2024-09-16T00:08:05.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619704+0000", 2024-09-16T00:08:05.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1766e-05 2024-09-16T00:08:05.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619708+0000", 2024-09-16T00:08:05.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3440000000000003e-06 2024-09-16T00:08:05.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619785+0000", 2024-09-16T00:08:05.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6936999999999999e-05 2024-09-16T00:08:05.181 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.182 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.182 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.182 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:31 2.d e22/19 ondisk, result = 0)", 2024-09-16T00:08:05.182 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:05.182 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.490725453, 2024-09-16T00:08:05.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101826, 2024-09-16T00:08:05.182 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.182 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.182 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:05.182 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:05.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:05.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619741+0000", 2024-09-16T00:08:05.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3099999999999997e-07 2024-09-16T00:08:05.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:08:05.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1719999999999998e-06 2024-09-16T00:08:05.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619771+0000", 2024-09-16T00:08:05.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4754000000000001e-05 2024-09-16T00:08:05.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619790+0000", 2024-09-16T00:08:05.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9516000000000001e-05 2024-09-16T00:08:05.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619830+0000", 2024-09-16T00:08:05.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9848000000000002e-05 2024-09-16T00:08:05.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619834+0000", 2024-09-16T00:08:05.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1150000000000004e-06 2024-09-16T00:08:05.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619842+0000", 2024-09-16T00:08:05.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.79e-06 2024-09-16T00:08:05.186 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.186 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.186 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.186 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.186 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:30 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:05.186 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:05.186 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.486374466000001, 2024-09-16T00:08:05.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013076400000000001, 2024-09-16T00:08:05.186 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.186 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.186 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:05.186 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:05.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:05.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624092+0000", 2024-09-16T00:08:05.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5799999999999999e-07 2024-09-16T00:08:05.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624096+0000", 2024-09-16T00:08:05.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:05.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624106+0000", 2024-09-16T00:08:05.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7629999999999995e-06 2024-09-16T00:08:05.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624112+0000", 2024-09-16T00:08:05.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.626e-06 2024-09-16T00:08:05.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624149+0000", 2024-09-16T00:08:05.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7094000000000001e-05 2024-09-16T00:08:05.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624154+0000", 2024-09-16T00:08:05.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7359999999999998e-06 2024-09-16T00:08:05.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624222+0000", 2024-09-16T00:08:05.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7680000000000003e-05 2024-09-16T00:08:05.190 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.190 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.190 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.190 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:39 2.6 2:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:05.190 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:05.190 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 21.470187908, 2024-09-16T00:08:05.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050072119999999996, 2024-09-16T00:08:05.190 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.191 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.191 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.191 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:05.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:05.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.192 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640280+0000", 2024-09-16T00:08:05.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5380000000000001e-06 2024-09-16T00:08:05.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.192 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640296+0000", 2024-09-16T00:08:05.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5950000000000001e-05 2024-09-16T00:08:05.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.192 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640327+0000", 2024-09-16T00:08:05.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0309999999999999e-05 2024-09-16T00:08:05.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640340+0000", 2024-09-16T00:08:05.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3773e-05 2024-09-16T00:08:05.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640422+0000", 2024-09-16T00:08:05.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1122e-05 2024-09-16T00:08:05.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640572+0000", 2024-09-16T00:08:05.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015044099999999999 2024-09-16T00:08:05.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:08:05.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640691+0000", 2024-09-16T00:08:05.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000119393 2024-09-16T00:08:05.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:05.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645006+0000", 2024-09-16T00:08:05.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043143779999999998 2024-09-16T00:08:05.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:05.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645169+0000", 2024-09-16T00:08:05.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016323299999999999 2024-09-16T00:08:05.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645227+0000", 2024-09-16T00:08:05.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8e-05 2024-09-16T00:08:05.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645285+0000", 2024-09-16T00:08:05.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8074e-05 2024-09-16T00:08:05.195 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.195 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.195 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.196 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:39 2.6 e22/19 ondisk, result = 0)", 2024-09-16T00:08:05.196 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:05.196 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 21.465615797000002, 2024-09-16T00:08:05.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016541200000000001, 2024-09-16T00:08:05.196 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.196 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.196 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:05.196 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:05.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:05.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644852+0000", 2024-09-16T00:08:05.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1670000000000002e-06 2024-09-16T00:08:05.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644865+0000", 2024-09-16T00:08:05.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2826000000000001e-05 2024-09-16T00:08:05.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644894+0000", 2024-09-16T00:08:05.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9264999999999998e-05 2024-09-16T00:08:05.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644911+0000", 2024-09-16T00:08:05.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6827999999999999e-05 2024-09-16T00:08:05.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644989+0000", 2024-09-16T00:08:05.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7849999999999995e-05 2024-09-16T00:08:05.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644999+0000", 2024-09-16T00:08:05.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7289999999999995e-06 2024-09-16T00:08:05.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645015+0000", 2024-09-16T00:08:05.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6747000000000001e-05 2024-09-16T00:08:05.199 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.199 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.199 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.199 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.199 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.200 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:05.201 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:08:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.140+0000 7f50be359640 1 -- 172.21.15.139:0/1872163800 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}) -- 0x7f50b815e690 con 0x7f50a406c2b0 2024-09-16T00:08:05.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.140+0000 7f50b4ff9640 1 -- 172.21.15.139:0/1872163800 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 no change) ==== 17+0+295 (secure 0 0 0) 0x7f50b815e690 con 0x7f50a406c2b0 2024-09-16T00:08:05.202 INFO:teuthology.orchestra.run.smithi139.stdout:moved 0 / 194 (0%) 2024-09-16T00:08:05.202 INFO:teuthology.orchestra.run.smithi139.stdout:avg 24.25 2024-09-16T00:08:05.202 INFO:teuthology.orchestra.run.smithi139.stdout:stddev 5.78252 -> 5.78252 (expected baseline 4.60638) 2024-09-16T00:08:05.202 INFO:teuthology.orchestra.run.smithi139.stdout:min osd.2 with 13 -> 13 pgs (0.536082 -> 0.536082 * mean) 2024-09-16T00:08:05.202 INFO:teuthology.orchestra.run.smithi139.stdout:max osd.5 with 30 -> 30 pgs (1.23711 -> 1.23711 * mean) 2024-09-16T00:08:05.202 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:08:05.202 INFO:teuthology.orchestra.run.smithi139.stdout:oload 1000 2024-09-16T00:08:05.203 INFO:teuthology.orchestra.run.smithi139.stdout:max_change 3 2024-09-16T00:08:05.203 INFO:teuthology.orchestra.run.smithi139.stdout:max_change_osds 4 2024-09-16T00:08:05.203 INFO:teuthology.orchestra.run.smithi139.stdout:average_utilization 0.0003 2024-09-16T00:08:05.203 INFO:teuthology.orchestra.run.smithi139.stdout:overload_utilization 0.0029 2024-09-16T00:08:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:no change 2024-09-16T00:08:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.142+0000 7f50be359640 1 -- 172.21.15.139:0/1872163800 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f50a406c2b0 msgr2=0x7f50a406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.142+0000 7f50be359640 1 --2- 172.21.15.139:0/1872163800 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f50a406c2b0 0x7f50a406e770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f50a8000f60 tx=0x7f50a8009040 comp rx=0 tx=0).stop 2024-09-16T00:08:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.143+0000 7f50be359640 1 -- 172.21.15.139:0/1872163800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b80ccc90 msgr2=0x7f50b80c5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.143+0000 7f50be359640 1 --2- 172.21.15.139:0/1872163800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b80ccc90 0x7f50b80c5c20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009970 tx=0x7f50ac0046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.143+0000 7f50be359640 1 -- 172.21.15.139:0/1872163800 shutdown_connections 2024-09-16T00:08:05.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.143+0000 7f50be359640 1 --2- 172.21.15.139:0/1872163800 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f50a406c2b0 0x7f50a406e770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.143+0000 7f50be359640 1 --2- 172.21.15.139:0/1872163800 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50b8154c20 0x7f50b80cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.143+0000 7f50be359640 1 --2- 172.21.15.139:0/1872163800 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50b80ccc90 0x7f50b80c5c20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.143+0000 7f50be359640 1 --2- 172.21.15.139:0/1872163800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50b8068060 0x7f50b8158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.143+0000 7f50be359640 1 -- 172.21.15.139:0/1872163800 >> 172.21.15.139:0/1872163800 conn(0x7f50b806ca40 msgr2=0x7f50b8067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:05.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.143+0000 7f50be359640 1 -- 172.21.15.139:0/1872163800 shutdown_connections 2024-09-16T00:08:05.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.143+0000 7f50be359640 1 -- 172.21.15.139:0/1872163800 wait complete. 2024-09-16T00:08:05.205 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-16T00:08:05.205 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-16T00:08:05.206 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:05.200+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:05.301 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:05.301+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:05.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.344+0000 7f08d4df8640 1 Processor -- start 2024-09-16T00:08:05.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.344+0000 7f08d4df8640 1 -- start start 2024-09-16T00:08:05.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.344+0000 7f08d4df8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f08d01549c0 0x7f08d00c8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:05.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.345+0000 7f08d4df8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08d006bb20 0x7f08d015e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:05.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.345+0000 7f08d4df8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d00c73f0 con 0x7f08d00729c0 2024-09-16T00:08:05.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.345+0000 7f08d4df8640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f08d00c8ec0 con 0x7f08d01549c0 2024-09-16T00:08:05.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.345+0000 7f08d4df8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f08d00c3fa0 con 0x7f08d006bb20 2024-09-16T00:08:05.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.345+0000 7f08ced9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f08d00729c0 0x7f08d0072e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37408/0 (socket says 172.21.15.139:37408) 2024-09-16T00:08:05.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.345+0000 7f08ced9d640 1 -- 172.21.15.139:0/3856327437 learned_addr learned my addr 172.21.15.139:0/3856327437 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:05.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.345+0000 7f08ce59c640 1 --2- 172.21.15.139:0/3856327437 >> v2:172.21.15.164:3300/0 conn(0x7f08d01549c0 0x7f08d00c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:05.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.345+0000 7f08c5d9b640 1 --2- 172.21.15.139:0/3856327437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08d006bb20 0x7f08d015e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:05.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.345+0000 7f08cdd9b640 1 -- 172.21.15.139:0/3856327437 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3907406940 0 0) 0x7f08d00c73f0 con 0x7f08d00729c0 2024-09-16T00:08:05.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.345+0000 7f08cdd9b640 1 -- 172.21.15.139:0/3856327437 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08bc003520 con 0x7f08d00729c0 2024-09-16T00:08:05.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.345+0000 7f08ce59c640 1 -- 172.21.15.139:0/3856327437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08d006bb20 msgr2=0x7f08d015e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:05.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.345+0000 7f08ce59c640 1 --2- 172.21.15.139:0/3856327437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08d006bb20 0x7f08d015e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.345+0000 7f08ce59c640 1 -- 172.21.15.139:0/3856327437 >> v1:172.21.15.139:6789/0 conn(0x7f08d00729c0 legacy=0x7f08d0072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:05.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.345+0000 7f08ce59c640 1 -- 172.21.15.139:0/3856327437 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d015ebb0 con 0x7f08d01549c0 2024-09-16T00:08:05.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.345+0000 7f08c5d9b640 1 --2- 172.21.15.139:0/3856327437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08d006bb20 0x7f08d015e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:05.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.346+0000 7f08ce59c640 1 --2- 172.21.15.139:0/3856327437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08d01549c0 0x7f08d00c8980 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f08c00097a0 tx=0x7f08c002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4014cb3129fe177e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:05.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.346+0000 7f08cdd9b640 1 -- 172.21.15.139:0/3856327437 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0037070 con 0x7f08d01549c0 2024-09-16T00:08:05.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.346+0000 7f08cdd9b640 1 -- 172.21.15.139:0/3856327437 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c0032440 con 0x7f08d01549c0 2024-09-16T00:08:05.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.346+0000 7f08cdd9b640 1 -- 172.21.15.139:0/3856327437 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c00327f0 con 0x7f08d01549c0 2024-09-16T00:08:05.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.347+0000 7f08d4df8640 1 -- 172.21.15.139:0/3856327437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08d01549c0 msgr2=0x7f08d00c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:05.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.347+0000 7f08d4df8640 1 --2- 172.21.15.139:0/3856327437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08d01549c0 0x7f08d00c8980 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f08c00097a0 tx=0x7f08c002a210 comp rx=0 tx=0).stop 2024-09-16T00:08:05.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.347+0000 7f08d4df8640 1 -- 172.21.15.139:0/3856327437 shutdown_connections 2024-09-16T00:08:05.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.347+0000 7f08d4df8640 1 --2- 172.21.15.139:0/3856327437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08d006bb20 0x7f08d015e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.347+0000 7f08d4df8640 1 --2- 172.21.15.139:0/3856327437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08d01549c0 0x7f08d00c8980 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.347+0000 7f08d4df8640 1 -- 172.21.15.139:0/3856327437 >> 172.21.15.139:0/3856327437 conn(0x7f08d00c90d0 msgr2=0x7f08d00c94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:05.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.347+0000 7f08d4df8640 1 -- 172.21.15.139:0/3856327437 shutdown_connections 2024-09-16T00:08:05.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.347+0000 7f08d4df8640 1 -- 172.21.15.139:0/3856327437 wait complete. 2024-09-16T00:08:05.350 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:05.350 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:05.350 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:05.350 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:05.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.348+0000 7f08d4df8640 1 Processor -- start 2024-09-16T00:08:05.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.348+0000 7f08d4df8640 1 -- start start 2024-09-16T00:08:05.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08d4df8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08d006bb20 0x7f08d01f4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:05.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08d4df8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08d00729c0 0x7f08d00c5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:05.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08d4df8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08d01549c0 0x7f08d01f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:05.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08d4df8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f08d00c7bc0 con 0x7f08d01549c0 2024-09-16T00:08:05.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08d4df8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f08d00c84f0 con 0x7f08d006bb20 2024-09-16T00:08:05.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08d4df8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f08d00c7ea0 con 0x7f08d00729c0 2024-09-16T00:08:05.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08ced9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08d01549c0 0x7f08d01f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:05.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08ced9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08d01549c0 0x7f08d01f1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59016/0 (socket says 172.21.15.139:59016) 2024-09-16T00:08:05.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08ced9d640 1 -- 172.21.15.139:0/2555187428 learned_addr learned my addr 172.21.15.139:0/2555187428 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:05.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08ce59c640 1 --2- 172.21.15.139:0/2555187428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08d006bb20 0x7f08d01f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:05.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08c5d9b640 1 --2- 172.21.15.139:0/2555187428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08d00729c0 0x7f08d00c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:05.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08ced9d640 1 -- 172.21.15.139:0/2555187428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08d00729c0 msgr2=0x7f08d00c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:05.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08ced9d640 1 --2- 172.21.15.139:0/2555187428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08d00729c0 0x7f08d00c5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08ced9d640 1 -- 172.21.15.139:0/2555187428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08d006bb20 msgr2=0x7f08d01f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:05.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08ced9d640 1 --2- 172.21.15.139:0/2555187428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08d006bb20 0x7f08d01f4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08ced9d640 1 -- 172.21.15.139:0/2555187428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d01e65e0 con 0x7f08d01549c0 2024-09-16T00:08:05.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.349+0000 7f08ced9d640 1 --2- 172.21.15.139:0/2555187428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08d01549c0 0x7f08d01f1ff0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f08c800cb40 tx=0x7f08c800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:05.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.350+0000 7f08c77fe640 1 -- 172.21.15.139:0/2555187428 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c8017070 con 0x7f08d01549c0 2024-09-16T00:08:05.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.350+0000 7f08c77fe640 1 -- 172.21.15.139:0/2555187428 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c800ac40 con 0x7f08d01549c0 2024-09-16T00:08:05.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.350+0000 7f08d4df8640 1 -- 172.21.15.139:0/2555187428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d01e68c0 con 0x7f08d01549c0 2024-09-16T00:08:05.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.350+0000 7f08c77fe640 1 -- 172.21.15.139:0/2555187428 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c8012620 con 0x7f08d01549c0 2024-09-16T00:08:05.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.350+0000 7f08d4df8640 1 -- 172.21.15.139:0/2555187428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d0157f40 con 0x7f08d01549c0 2024-09-16T00:08:05.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.351+0000 7f08d4df8640 1 -- 172.21.15.139:0/2555187428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d0154da0 con 0x7f08d01549c0 2024-09-16T00:08:05.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.351+0000 7f08c77fe640 1 -- 172.21.15.139:0/2555187428 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f08c801a070 con 0x7f08d01549c0 2024-09-16T00:08:05.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.352+0000 7f08c77fe640 1 --2- 172.21.15.139:0/2555187428 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08b006c2e0 0x7f08b006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:05.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.352+0000 7f08c77fe640 1 -- 172.21.15.139:0/2555187428 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5738+0+0 (secure 0 0 0) 0x7f08c808b210 con 0x7f08d01549c0 2024-09-16T00:08:05.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.352+0000 7f08ce59c640 1 --2- 172.21.15.139:0/2555187428 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08b006c2e0 0x7f08b006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:05.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.352+0000 7f08ce59c640 1 --2- 172.21.15.139:0/2555187428 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08b006c2e0 0x7f08b006e7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f08c0034000 tx=0x7f08c002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:05.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.356+0000 7f08c77fe640 1 -- 172.21.15.139:0/2555187428 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f08c8059960 con 0x7f08d01549c0 2024-09-16T00:08:05.358 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:08:05.443 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:05.442+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:05.502 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:05.502 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:05.503 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:05.503 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:05.503 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:05.503 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:05.502+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:05.511 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:08:05.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.526+0000 7f08d4df8640 1 -- 172.21.15.139:0/2555187428 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 1.0, "target": ["mon-mgr", ""]}) -- 0x7f08d00c69e0 con 0x7f08b006c2e0 2024-09-16T00:08:05.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.526+0000 7f08c77fe640 1 -- 172.21.15.139:0/2555187428 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 no change) ==== 17+0+295 (secure 0 0 0) 0x7f08d00c69e0 con 0x7f08b006c2e0 2024-09-16T00:08:05.528 INFO:teuthology.orchestra.run.smithi139.stdout:moved 0 / 194 (0%) 2024-09-16T00:08:05.528 INFO:teuthology.orchestra.run.smithi139.stdout:avg 24.25 2024-09-16T00:08:05.528 INFO:teuthology.orchestra.run.smithi139.stdout:stddev 5.78252 -> 5.78252 (expected baseline 4.60638) 2024-09-16T00:08:05.528 INFO:teuthology.orchestra.run.smithi139.stdout:min osd.2 with 13 -> 13 pgs (0.536082 -> 0.536082 * mean) 2024-09-16T00:08:05.528 INFO:teuthology.orchestra.run.smithi139.stdout:max osd.5 with 30 -> 30 pgs (1.23711 -> 1.23711 * mean) 2024-09-16T00:08:05.528 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:08:05.528 INFO:teuthology.orchestra.run.smithi139.stdout:oload 1000 2024-09-16T00:08:05.528 INFO:teuthology.orchestra.run.smithi139.stdout:max_change 1 2024-09-16T00:08:05.528 INFO:teuthology.orchestra.run.smithi139.stdout:max_change_osds 4 2024-09-16T00:08:05.528 INFO:teuthology.orchestra.run.smithi139.stdout:average_utilization 0.0003 2024-09-16T00:08:05.529 INFO:teuthology.orchestra.run.smithi139.stdout:overload_utilization 0.0029 2024-09-16T00:08:05.529 INFO:teuthology.orchestra.run.smithi139.stderr:no change 2024-09-16T00:08:05.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.528+0000 7f08d4df8640 1 -- 172.21.15.139:0/2555187428 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08b006c2e0 msgr2=0x7f08b006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:05.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.528+0000 7f08d4df8640 1 --2- 172.21.15.139:0/2555187428 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08b006c2e0 0x7f08b006e7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f08c0034000 tx=0x7f08c002e000 comp rx=0 tx=0).stop 2024-09-16T00:08:05.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.528+0000 7f08d4df8640 1 -- 172.21.15.139:0/2555187428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08d01549c0 msgr2=0x7f08d01f1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:05.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.528+0000 7f08d4df8640 1 --2- 172.21.15.139:0/2555187428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08d01549c0 0x7f08d01f1ff0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f08c800cb40 tx=0x7f08c800a060 comp rx=0 tx=0).stop 2024-09-16T00:08:05.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.529+0000 7f08d4df8640 1 -- 172.21.15.139:0/2555187428 shutdown_connections 2024-09-16T00:08:05.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.529+0000 7f08d4df8640 1 --2- 172.21.15.139:0/2555187428 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08b006c2e0 0x7f08b006e7a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.529+0000 7f08d4df8640 1 --2- 172.21.15.139:0/2555187428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08d01549c0 0x7f08d01f1ff0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.529+0000 7f08d4df8640 1 --2- 172.21.15.139:0/2555187428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08d00729c0 0x7f08d00c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.529+0000 7f08d4df8640 1 --2- 172.21.15.139:0/2555187428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08d006bb20 0x7f08d01f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.529+0000 7f08d4df8640 1 -- 172.21.15.139:0/2555187428 >> 172.21.15.139:0/2555187428 conn(0x7f08d00c90d0 msgr2=0x7f08d0072e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:05.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.529+0000 7f08d4df8640 1 -- 172.21.15.139:0/2555187428 shutdown_connections 2024-09-16T00:08:05.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.529+0000 7f08d4df8640 1 -- 172.21.15.139:0/2555187428 wait complete. 2024-09-16T00:08:05.542 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-16T00:08:05.542 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-16T00:08:05.603 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:05.603+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:05.637 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:05.638 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:05.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:05.638 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:05.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.638 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:05.638 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:05.638 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.020316907000002, 2024-09-16T00:08:05.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038263220000000001, 2024-09-16T00:08:05.638 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.638 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.638 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.638 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:05.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:05.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615465+0000", 2024-09-16T00:08:05.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2769999999999999e-06 2024-09-16T00:08:05.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615471+0000", 2024-09-16T00:08:05.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5729999999999996e-06 2024-09-16T00:08:05.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615483+0000", 2024-09-16T00:08:05.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1838000000000001e-05 2024-09-16T00:08:05.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615492+0000", 2024-09-16T00:08:05.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2410000000000001e-06 2024-09-16T00:08:05.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615596+0000", 2024-09-16T00:08:05.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104097 2024-09-16T00:08:05.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615688+0000", 2024-09-16T00:08:05.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1225999999999997e-05 2024-09-16T00:08:05.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:05.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615804+0000", 2024-09-16T00:08:05.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000116601 2024-09-16T00:08:05.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:05.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618165+0000", 2024-09-16T00:08:05.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023609429999999999 2024-09-16T00:08:05.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:05.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:08:05.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010258509999999999 2024-09-16T00:08:05.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619243+0000", 2024-09-16T00:08:05.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2327999999999998e-05 2024-09-16T00:08:05.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619287+0000", 2024-09-16T00:08:05.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3346999999999997e-05 2024-09-16T00:08:05.644 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.644 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.644 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.644 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:05.645 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:05.645 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.020261830999999, 2024-09-16T00:08:05.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0054735510000000001, 2024-09-16T00:08:05.645 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.645 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.645 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.645 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.645 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:05.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.646 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:05.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.646 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.646 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615516+0000", 2024-09-16T00:08:05.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6899999999999998e-07 2024-09-16T00:08:05.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.646 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.646 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615519+0000", 2024-09-16T00:08:05.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1760000000000002e-06 2024-09-16T00:08:05.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615526+0000", 2024-09-16T00:08:05.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8399999999999997e-06 2024-09-16T00:08:05.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615531+0000", 2024-09-16T00:08:05.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4450000000000004e-06 2024-09-16T00:08:05.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:05.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054127099999999996 2024-09-16T00:08:05.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616140+0000", 2024-09-16T00:08:05.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7230999999999994e-05 2024-09-16T00:08:05.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:05.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616249+0000", 2024-09-16T00:08:05.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109122 2024-09-16T00:08:05.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:05.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618646+0000", 2024-09-16T00:08:05.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023972279999999999 2024-09-16T00:08:05.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:05.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:08:05.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022396220000000001 2024-09-16T00:08:05.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620939+0000", 2024-09-16T00:08:05.650 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2735999999999998e-05 2024-09-16T00:08:05.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.650 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.650 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620989+0000", 2024-09-16T00:08:05.650 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0411000000000002e-05 2024-09-16T00:08:05.650 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.650 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.650 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.650 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:05.650 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:05.651 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.020236352000001, 2024-09-16T00:08:05.651 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055872789999999997, 2024-09-16T00:08:05.651 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.651 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.651 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.651 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.651 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.651 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.651 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:05.651 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.651 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.651 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.651 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.652 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:05.652 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.652 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.652 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.652 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:05.652 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.01e-07 2024-09-16T00:08:05.652 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.652 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.652 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615547+0000", 2024-09-16T00:08:05.652 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7899999999999996e-06 2024-09-16T00:08:05.652 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.653 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:08:05.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1119999999999998e-06 2024-09-16T00:08:05.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.653 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.653 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615560+0000", 2024-09-16T00:08:05.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:08:05.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.654 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616424+0000", 2024-09-16T00:08:05.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086350800000000003 2024-09-16T00:08:05.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.654 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:08:05.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9675999999999999e-05 2024-09-16T00:08:05.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:05.654 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616621+0000", 2024-09-16T00:08:05.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7345999999999999e-05 2024-09-16T00:08:05.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.655 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:05.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618654+0000", 2024-09-16T00:08:05.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020335269999999998 2024-09-16T00:08:05.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.655 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:05.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621055+0000", 2024-09-16T00:08:05.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024009729999999998 2024-09-16T00:08:05.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.656 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.656 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621100+0000", 2024-09-16T00:08:05.656 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5402999999999999e-05 2024-09-16T00:08:05.656 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.656 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.656 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.656 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621128+0000", 2024-09-16T00:08:05.656 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7736e-05 2024-09-16T00:08:05.656 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.656 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.656 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.656 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.656 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.656 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:05.657 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:05.657 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.020210702, 2024-09-16T00:08:05.657 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057024509999999999, 2024-09-16T00:08:05.657 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.657 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.657 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.657 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.657 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.657 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.657 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:05.658 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.658 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:05.658 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.658 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:05.658 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6300000000000001e-07 2024-09-16T00:08:05.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615569+0000", 2024-09-16T00:08:05.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3709999999999998e-06 2024-09-16T00:08:05.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.659 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615580+0000", 2024-09-16T00:08:05.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0448000000000001e-05 2024-09-16T00:08:05.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.659 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615584+0000", 2024-09-16T00:08:05.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6020000000000002e-06 2024-09-16T00:08:05.660 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.660 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.660 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.660 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616850+0000", 2024-09-16T00:08:05.660 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012661110000000001 2024-09-16T00:08:05.660 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.660 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.660 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.660 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616915+0000", 2024-09-16T00:08:05.660 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4820000000000006e-05 2024-09-16T00:08:05.660 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.660 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.660 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:05.661 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617031+0000", 2024-09-16T00:08:05.661 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115258 2024-09-16T00:08:05.661 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.661 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.661 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:05.661 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618659+0000", 2024-09-16T00:08:05.661 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001627975 2024-09-16T00:08:05.661 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.661 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.661 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:05.661 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621179+0000", 2024-09-16T00:08:05.661 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025203410000000002 2024-09-16T00:08:05.661 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.661 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.662 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.662 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621238+0000", 2024-09-16T00:08:05.662 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9323000000000001e-05 2024-09-16T00:08:05.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.662 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.662 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.662 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621269+0000", 2024-09-16T00:08:05.662 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0839000000000001e-05 2024-09-16T00:08:05.662 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.662 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.662 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.662 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.662 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:05.663 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:05.663 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.020186953, 2024-09-16T00:08:05.663 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058012769999999996, 2024-09-16T00:08:05.663 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.663 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.663 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.663 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.663 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.663 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.663 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:05.663 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.663 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.663 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.664 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.664 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:05.664 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.664 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.664 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.664 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615591+0000", 2024-09-16T00:08:05.664 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6199999999999999e-07 2024-09-16T00:08:05.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.664 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.665 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.665 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615594+0000", 2024-09-16T00:08:05.665 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8739999999999999e-06 2024-09-16T00:08:05.665 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.665 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.665 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.665 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615599+0000", 2024-09-16T00:08:05.665 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:08:05.665 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.665 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.665 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.665 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615603+0000", 2024-09-16T00:08:05.665 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8489999999999999e-06 2024-09-16T00:08:05.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617235+0000", 2024-09-16T00:08:05.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001632037 2024-09-16T00:08:05.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617348+0000", 2024-09-16T00:08:05.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113387 2024-09-16T00:08:05.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:05.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617458+0000", 2024-09-16T00:08:05.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109765 2024-09-16T00:08:05.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:05.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618664+0000", 2024-09-16T00:08:05.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012059950000000001 2024-09-16T00:08:05.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:05.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621311+0000", 2024-09-16T00:08:05.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026468469999999999 2024-09-16T00:08:05.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621357+0000", 2024-09-16T00:08:05.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6360999999999999e-05 2024-09-16T00:08:05.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.668 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621392+0000", 2024-09-16T00:08:05.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4190999999999998e-05 2024-09-16T00:08:05.668 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.668 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.668 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.669 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:05.669 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:05.669 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.020168329000001, 2024-09-16T00:08:05.669 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058737279999999999, 2024-09-16T00:08:05.669 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.669 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.669 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.669 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:05.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:05.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:05.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3799999999999999e-07 2024-09-16T00:08:05.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615611+0000", 2024-09-16T00:08:05.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-06 2024-09-16T00:08:05.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615626+0000", 2024-09-16T00:08:05.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:08:05.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615634+0000", 2024-09-16T00:08:05.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-06 2024-09-16T00:08:05.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617692+0000", 2024-09-16T00:08:05.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002057718 2024-09-16T00:08:05.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617768+0000", 2024-09-16T00:08:05.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6878999999999994e-05 2024-09-16T00:08:05.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:05.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:05.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019501699999999999 2024-09-16T00:08:05.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.674 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:05.674 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:08:05.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034357200000000002 2024-09-16T00:08:05.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.674 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:05.674 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621429+0000", 2024-09-16T00:08:05.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0012999999999999e-05 2024-09-16T00:08:05.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.675 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.675 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621459+0000", 2024-09-16T00:08:05.675 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9498000000000001e-05 2024-09-16T00:08:05.675 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.675 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.675 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.675 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621483+0000", 2024-09-16T00:08:05.675 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3957000000000001e-05 2024-09-16T00:08:05.675 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.676 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.676 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.676 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.676 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:05.676 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:05.676 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.020137048999999, 2024-09-16T00:08:05.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059371789999999999, 2024-09-16T00:08:05.676 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.676 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.676 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.676 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:05.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:05.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:08:05.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4000000000000002e-07 2024-09-16T00:08:05.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615643+0000", 2024-09-16T00:08:05.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6860000000000002e-06 2024-09-16T00:08:05.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615649+0000", 2024-09-16T00:08:05.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7610000000000004e-06 2024-09-16T00:08:05.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615653+0000", 2024-09-16T00:08:05.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0380000000000001e-06 2024-09-16T00:08:05.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618197+0000", 2024-09-16T00:08:05.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025440079999999999 2024-09-16T00:08:05.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618312+0000", 2024-09-16T00:08:05.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115083 2024-09-16T00:08:05.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:05.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618412+0000", 2024-09-16T00:08:05.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010016799999999999 2024-09-16T00:08:05.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.681 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:05.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621518+0000", 2024-09-16T00:08:05.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031058679999999999 2024-09-16T00:08:05.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.681 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.681 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:05.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621526+0000", 2024-09-16T00:08:05.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2490000000000004e-06 2024-09-16T00:08:05.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.681 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.681 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621549+0000", 2024-09-16T00:08:05.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3686999999999999e-05 2024-09-16T00:08:05.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621577+0000", 2024-09-16T00:08:05.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8191000000000002e-05 2024-09-16T00:08:05.682 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.682 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.682 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.682 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:05.682 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:05.682 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.020119383000001, 2024-09-16T00:08:05.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005989622, 2024-09-16T00:08:05.683 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.683 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.683 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.683 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:05.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:05.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:05.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3799999999999998e-07 2024-09-16T00:08:05.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615660+0000", 2024-09-16T00:08:05.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9740000000000001e-06 2024-09-16T00:08:05.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615666+0000", 2024-09-16T00:08:05.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3499999999999996e-06 2024-09-16T00:08:05.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615671+0000", 2024-09-16T00:08:05.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.13e-06 2024-09-16T00:08:05.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618759+0000", 2024-09-16T00:08:05.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030880500000000002 2024-09-16T00:08:05.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618832+0000", 2024-09-16T00:08:05.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3725000000000003e-05 2024-09-16T00:08:05.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:05.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618958+0000", 2024-09-16T00:08:05.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012587899999999999 2024-09-16T00:08:05.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:05.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621583+0000", 2024-09-16T00:08:05.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026246469999999999 2024-09-16T00:08:05.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:05.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621602+0000", 2024-09-16T00:08:05.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9137e-05 2024-09-16T00:08:05.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621629+0000", 2024-09-16T00:08:05.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6451000000000001e-05 2024-09-16T00:08:05.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621648+0000", 2024-09-16T00:08:05.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8941e-05 2024-09-16T00:08:05.688 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.688 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.688 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.688 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:05.688 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:05.688 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.020101764, 2024-09-16T00:08:05.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053141380000000004, 2024-09-16T00:08:05.689 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.689 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.689 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.689 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:05.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:05.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:05.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-07 2024-09-16T00:08:05.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615678+0000", 2024-09-16T00:08:05.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:08:05.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615682+0000", 2024-09-16T00:08:05.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6659999999999999e-06 2024-09-16T00:08:05.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615686+0000", 2024-09-16T00:08:05.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6990000000000001e-06 2024-09-16T00:08:05.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615736+0000", 2024-09-16T00:08:05.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9656e-05 2024-09-16T00:08:05.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615864+0000", 2024-09-16T00:08:05.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012836100000000001 2024-09-16T00:08:05.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:05.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615989+0000", 2024-09-16T00:08:05.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000125525 2024-09-16T00:08:05.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:05.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617355+0000", 2024-09-16T00:08:05.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001365982 2024-09-16T00:08:05.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:05.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620897+0000", 2024-09-16T00:08:05.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035419610000000002 2024-09-16T00:08:05.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620941+0000", 2024-09-16T00:08:05.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3998999999999997e-05 2024-09-16T00:08:05.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620990+0000", 2024-09-16T00:08:05.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8199999999999999e-05 2024-09-16T00:08:05.694 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.694 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.694 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.694 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:05.694 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:05.694 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.020074506, 2024-09-16T00:08:05.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006316449, 2024-09-16T00:08:05.694 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.695 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.695 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:05.695 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:05.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:05.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:05.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3299999999999998e-07 2024-09-16T00:08:05.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615705+0000", 2024-09-16T00:08:05.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.278e-06 2024-09-16T00:08:05.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615711+0000", 2024-09-16T00:08:05.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8119999999999996e-06 2024-09-16T00:08:05.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615716+0000", 2024-09-16T00:08:05.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4909999999999999e-06 2024-09-16T00:08:05.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617551+0000", 2024-09-16T00:08:05.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00183549 2024-09-16T00:08:05.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617627+0000", 2024-09-16T00:08:05.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6163999999999998e-05 2024-09-16T00:08:05.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:05.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617776+0000", 2024-09-16T00:08:05.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014909499999999999 2024-09-16T00:08:05.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:05.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621072+0000", 2024-09-16T00:08:05.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032955150000000002 2024-09-16T00:08:05.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:05.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621957+0000", 2024-09-16T00:08:05.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088553199999999996 2024-09-16T00:08:05.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:05.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621998+0000", 2024-09-16T00:08:05.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0815000000000001e-05 2024-09-16T00:08:05.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:08:05.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0924000000000002e-05 2024-09-16T00:08:05.700 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.700 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.700 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.700 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:6 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:05.700 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:05.700 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.016672538000002, 2024-09-16T00:08:05.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000188111, 2024-09-16T00:08:05.700 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.700 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.700 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:05.701 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:05.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:05.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619106+0000", 2024-09-16T00:08:05.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4399999999999998e-07 2024-09-16T00:08:05.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619110+0000", 2024-09-16T00:08:05.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1110000000000001e-06 2024-09-16T00:08:05.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619120+0000", 2024-09-16T00:08:05.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7909999999999999e-06 2024-09-16T00:08:05.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619133+0000", 2024-09-16T00:08:05.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.359e-05 2024-09-16T00:08:05.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619179+0000", 2024-09-16T00:08:05.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5459e-05 2024-09-16T00:08:05.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619186+0000", 2024-09-16T00:08:05.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0480000000000001e-06 2024-09-16T00:08:05.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619293+0000", 2024-09-16T00:08:05.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000107168 2024-09-16T00:08:05.704 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.704 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.704 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.704 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:22 2.e e22/19 ondisk, result = 0)", 2024-09-16T00:08:05.704 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:05.704 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.014981837000001, 2024-09-16T00:08:05.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020058200000000001, 2024-09-16T00:08:05.705 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.705 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.705 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:05.705 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:05.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:05.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:05.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0109999999999999e-06 2024-09-16T00:08:05.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620801+0000", 2024-09-16T00:08:05.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6779999999999999e-06 2024-09-16T00:08:05.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620814+0000", 2024-09-16T00:08:05.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3131e-05 2024-09-16T00:08:05.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620821+0000", 2024-09-16T00:08:05.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0430000000000002e-06 2024-09-16T00:08:05.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:05.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3010999999999997e-05 2024-09-16T00:08:05.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620891+0000", 2024-09-16T00:08:05.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1760000000000004e-06 2024-09-16T00:08:05.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620996+0000", 2024-09-16T00:08:05.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010453199999999999 2024-09-16T00:08:05.708 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.708 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.708 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.709 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:7 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:05.709 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:05.709 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.014973853000001, 2024-09-16T00:08:05.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000195725, 2024-09-16T00:08:05.709 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.709 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.709 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:05.709 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:05.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:05.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620804+0000", 2024-09-16T00:08:05.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6099999999999999e-07 2024-09-16T00:08:05.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620809+0000", 2024-09-16T00:08:05.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4950000000000002e-06 2024-09-16T00:08:05.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:05.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0553e-05 2024-09-16T00:08:05.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620826+0000", 2024-09-16T00:08:05.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5579999999999997e-06 2024-09-16T00:08:05.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620874+0000", 2024-09-16T00:08:05.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7648999999999997e-05 2024-09-16T00:08:05.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620880+0000", 2024-09-16T00:08:05.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8290000000000002e-06 2024-09-16T00:08:05.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620999+0000", 2024-09-16T00:08:05.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011878 2024-09-16T00:08:05.712 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.713 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.713 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.713 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:8 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:05.713 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:05.713 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.014921792999999, 2024-09-16T00:08:05.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00027991099999999998, 2024-09-16T00:08:05.713 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.713 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.713 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:05.713 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:05.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:05.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620856+0000", 2024-09-16T00:08:05.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6800000000000001e-07 2024-09-16T00:08:05.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620859+0000", 2024-09-16T00:08:05.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6879999999999999e-06 2024-09-16T00:08:05.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620867+0000", 2024-09-16T00:08:05.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2819999999999996e-06 2024-09-16T00:08:05.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620872+0000", 2024-09-16T00:08:05.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5369999999999997e-06 2024-09-16T00:08:05.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621043+0000", 2024-09-16T00:08:05.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170791 2024-09-16T00:08:05.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621050+0000", 2024-09-16T00:08:05.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8580000000000002e-06 2024-09-16T00:08:05.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621135+0000", 2024-09-16T00:08:05.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5387000000000001e-05 2024-09-16T00:08:05.717 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.717 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.717 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.717 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:9 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:05.717 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:05.717 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.014893215000001, 2024-09-16T00:08:05.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00039143799999999997, 2024-09-16T00:08:05.717 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.718 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.718 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:05.718 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:05.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:05.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:05.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6800000000000002e-07 2024-09-16T00:08:05.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:08:05.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0310000000000001e-06 2024-09-16T00:08:05.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620892+0000", 2024-09-16T00:08:05.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9390000000000002e-06 2024-09-16T00:08:05.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620898+0000", 2024-09-16T00:08:05.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4480000000000002e-06 2024-09-16T00:08:05.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621167+0000", 2024-09-16T00:08:05.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026960599999999998 2024-09-16T00:08:05.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621174+0000", 2024-09-16T00:08:05.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.567e-06 2024-09-16T00:08:05.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621275+0000", 2024-09-16T00:08:05.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101579 2024-09-16T00:08:05.721 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.721 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.721 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.721 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:10 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:05.721 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:05.721 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.014870767000001, 2024-09-16T00:08:05.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00049416200000000003, 2024-09-16T00:08:05.722 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.722 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.722 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:05.722 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:05.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:05.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620907+0000", 2024-09-16T00:08:05.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.42e-07 2024-09-16T00:08:05.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620908+0000", 2024-09-16T00:08:05.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4300000000000001e-06 2024-09-16T00:08:05.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620911+0000", 2024-09-16T00:08:05.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3890000000000001e-06 2024-09-16T00:08:05.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620914+0000", 2024-09-16T00:08:05.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3690000000000001e-06 2024-09-16T00:08:05.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621297+0000", 2024-09-16T00:08:05.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038319200000000003 2024-09-16T00:08:05.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621305+0000", 2024-09-16T00:08:05.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4780000000000008e-06 2024-09-16T00:08:05.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621401+0000", 2024-09-16T00:08:05.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5161999999999995e-05 2024-09-16T00:08:05.725 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.725 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.725 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.726 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:11 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:05.726 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:05.726 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.014468017999999, 2024-09-16T00:08:05.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017943899999999999, 2024-09-16T00:08:05.726 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.726 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.726 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:05.726 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:05.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:05.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621310+0000", 2024-09-16T00:08:05.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5199999999999996e-07 2024-09-16T00:08:05.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:05.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1740000000000002e-06 2024-09-16T00:08:05.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621324+0000", 2024-09-16T00:08:05.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7170000000000003e-06 2024-09-16T00:08:05.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621331+0000", 2024-09-16T00:08:05.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2280000000000001e-06 2024-09-16T00:08:05.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621422+0000", 2024-09-16T00:08:05.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1052999999999997e-05 2024-09-16T00:08:05.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621426+0000", 2024-09-16T00:08:05.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.036e-06 2024-09-16T00:08:05.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621489+0000", 2024-09-16T00:08:05.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2478999999999996e-05 2024-09-16T00:08:05.730 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.730 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.730 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.730 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:12 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:05.730 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:05.730 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.014417163000001, 2024-09-16T00:08:05.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022422300000000001, 2024-09-16T00:08:05.730 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.730 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.730 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:05.730 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:05.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:05.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:05.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6699999999999999e-07 2024-09-16T00:08:05.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.732 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621363+0000", 2024-09-16T00:08:05.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7020000000000001e-06 2024-09-16T00:08:05.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621371+0000", 2024-09-16T00:08:05.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7209999999999994e-06 2024-09-16T00:08:05.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621376+0000", 2024-09-16T00:08:05.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7940000000000002e-06 2024-09-16T00:08:05.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621511+0000", 2024-09-16T00:08:05.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013518100000000001 2024-09-16T00:08:05.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621514+0000", 2024-09-16T00:08:05.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5599999999999998e-06 2024-09-16T00:08:05.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621584+0000", 2024-09-16T00:08:05.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9898000000000001e-05 2024-09-16T00:08:05.734 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.734 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.734 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.734 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:13 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:05.735 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:05.735 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.014390898999999, 2024-09-16T00:08:05.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026534099999999999, 2024-09-16T00:08:05.735 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.735 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.735 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:05.735 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:05.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:05.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621387+0000", 2024-09-16T00:08:05.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4299999999999999e-07 2024-09-16T00:08:05.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621389+0000", 2024-09-16T00:08:05.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.593e-06 2024-09-16T00:08:05.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621395+0000", 2024-09-16T00:08:05.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.553e-06 2024-09-16T00:08:05.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:08:05.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8410000000000002e-06 2024-09-16T00:08:05.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621597+0000", 2024-09-16T00:08:05.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019794100000000001 2024-09-16T00:08:05.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621600+0000", 2024-09-16T00:08:05.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.185e-06 2024-09-16T00:08:05.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621652+0000", 2024-09-16T00:08:05.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1984999999999998e-05 2024-09-16T00:08:05.738 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.738 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.738 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.739 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:23 2.e e22/19 ondisk, result = 0)", 2024-09-16T00:08:05.739 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:05.739 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.01390795, 2024-09-16T00:08:05.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015895699999999999, 2024-09-16T00:08:05.739 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:05.739 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:05.739 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:05.739 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:05.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:05.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:05.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:05.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:05.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:05.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:05.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621870+0000", 2024-09-16T00:08:05.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1600000000000001e-07 2024-09-16T00:08:05.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:05.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621873+0000", 2024-09-16T00:08:05.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3079999999999999e-06 2024-09-16T00:08:05.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:05.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621882+0000", 2024-09-16T00:08:05.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8429999999999999e-06 2024-09-16T00:08:05.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:05.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621896+0000", 2024-09-16T00:08:05.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:08:05.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:05.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621944+0000", 2024-09-16T00:08:05.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8062999999999997e-05 2024-09-16T00:08:05.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:05.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621951+0000", 2024-09-16T00:08:05.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7689999999999999e-06 2024-09-16T00:08:05.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:05.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:05.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:05.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622028+0000", 2024-09-16T00:08:05.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7243999999999994e-05 2024-09-16T00:08:05.743 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.743 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.743 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.743 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:05.743 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:05.743 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:05.745 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:08:05.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.687+0000 7f7ed6477640 1 Processor -- start 2024-09-16T00:08:05.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.687+0000 7f7ed6477640 1 -- start start 2024-09-16T00:08:05.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.688+0000 7f7ed6477640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7ed0154c20 0x7f7ed015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:05.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.688+0000 7f7ed6477640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ed0068060 0x7f7ed00c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:05.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.688+0000 7f7ed6477640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7ed00ccc90 0x7f7ed00cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:05.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.688+0000 7f7ed6477640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed015b7c0 con 0x7f7ed00ccc90 2024-09-16T00:08:05.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.688+0000 7f7ed6477640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed00cc260 con 0x7f7ed0154c20 2024-09-16T00:08:05.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.688+0000 7f7ed6477640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed00cc4b0 con 0x7f7ed0068060 2024-09-16T00:08:05.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.688+0000 7f7ed4a14640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7ed00ccc90 0x7f7ed00cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:05.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.688+0000 7f7ed4a14640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7ed00ccc90 0x7f7ed00cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59030/0 (socket says 172.21.15.139:59030) 2024-09-16T00:08:05.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.688+0000 7f7ed4a14640 1 -- 172.21.15.139:0/2165492125 learned_addr learned my addr 172.21.15.139:0/2165492125 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:05.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.688+0000 7f7ecffff640 1 --2- 172.21.15.139:0/2165492125 >> v2:172.21.15.164:3300/0 conn(0x7f7ed0154c20 0x7f7ed015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:05.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.688+0000 7f7ecf7fe640 1 --2- 172.21.15.139:0/2165492125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ed0068060 0x7f7ed00c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:05.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.688+0000 7f7ed4a14640 1 -- 172.21.15.139:0/2165492125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ed0068060 msgr2=0x7f7ed00c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:05.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.688+0000 7f7ed4a14640 1 --2- 172.21.15.139:0/2165492125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ed0068060 0x7f7ed00c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.688+0000 7f7ed4a14640 1 -- 172.21.15.139:0/2165492125 >> v2:172.21.15.164:3300/0 conn(0x7f7ed0154c20 msgr2=0x7f7ed015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:05.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.688+0000 7f7ed4a14640 1 --2- 172.21.15.139:0/2165492125 >> v2:172.21.15.164:3300/0 conn(0x7f7ed0154c20 0x7f7ed015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.688+0000 7f7ed4a14640 1 -- 172.21.15.139:0/2165492125 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed00cc790 con 0x7f7ed00ccc90 2024-09-16T00:08:05.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.689+0000 7f7ed4a14640 1 --2- 172.21.15.139:0/2165492125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ed00ccc90 0x7f7ed00cd0c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00d1290 tx=0x7f7ec802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ecafb2d256d1be server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:05.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.689+0000 7f7eceffd640 1 -- 172.21.15.139:0/2165492125 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8036070 con 0x7f7ed00ccc90 2024-09-16T00:08:05.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.689+0000 7f7eceffd640 1 -- 172.21.15.139:0/2165492125 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec8031600 con 0x7f7ed00ccc90 2024-09-16T00:08:05.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.689+0000 7f7eceffd640 1 -- 172.21.15.139:0/2165492125 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80319d0 con 0x7f7ed00ccc90 2024-09-16T00:08:05.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.690+0000 7f7ed6477640 1 -- 172.21.15.139:0/2165492125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ed00ccc90 msgr2=0x7f7ed00cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:05.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.690+0000 7f7ed6477640 1 --2- 172.21.15.139:0/2165492125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ed00ccc90 0x7f7ed00cd0c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00d1290 tx=0x7f7ec802a730 comp rx=0 tx=0).stop 2024-09-16T00:08:05.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.690+0000 7f7ed6477640 1 -- 172.21.15.139:0/2165492125 shutdown_connections 2024-09-16T00:08:05.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.690+0000 7f7ed6477640 1 --2- 172.21.15.139:0/2165492125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ed00ccc90 0x7f7ed00cd0c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.690+0000 7f7ed6477640 1 --2- 172.21.15.139:0/2165492125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ed0068060 0x7f7ed00c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.690+0000 7f7ed6477640 1 --2- 172.21.15.139:0/2165492125 >> v2:172.21.15.164:3300/0 conn(0x7f7ed0154c20 0x7f7ed015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.690+0000 7f7ed6477640 1 -- 172.21.15.139:0/2165492125 >> 172.21.15.139:0/2165492125 conn(0x7f7ed006ca40 msgr2=0x7f7ed0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:05.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.690+0000 7f7ed6477640 1 -- 172.21.15.139:0/2165492125 shutdown_connections 2024-09-16T00:08:05.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.690+0000 7f7ed6477640 1 -- 172.21.15.139:0/2165492125 wait complete. 2024-09-16T00:08:05.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.691+0000 7f7ed6477640 1 Processor -- start 2024-09-16T00:08:05.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.691+0000 7f7ed6477640 1 -- start start 2024-09-16T00:08:05.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ed6477640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ed0068060 0x7f7ed0147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:05.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ed6477640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ed00ccc90 0x7f7ed00ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:05.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ed6477640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ed0154c20 0x7f7ed00cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:05.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ed6477640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed015e3b0 con 0x7f7ed0068060 2024-09-16T00:08:05.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ed6477640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed015e0d0 con 0x7f7ed0154c20 2024-09-16T00:08:05.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ed6477640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed015e690 con 0x7f7ed00ccc90 2024-09-16T00:08:05.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ecffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ed0068060 0x7f7ed0147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:05.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ecffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ed0068060 0x7f7ed0147ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59044/0 (socket says 172.21.15.139:59044) 2024-09-16T00:08:05.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ecffff640 1 -- 172.21.15.139:0/4031802641 learned_addr learned my addr 172.21.15.139:0/4031802641 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:05.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ecf7fe640 1 --2- 172.21.15.139:0/4031802641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ed00ccc90 0x7f7ed00ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:05.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ed4a14640 1 --2- 172.21.15.139:0/4031802641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ed0154c20 0x7f7ed00cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:05.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ecffff640 1 -- 172.21.15.139:0/4031802641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ed00ccc90 msgr2=0x7f7ed00ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:05.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ecffff640 1 --2- 172.21.15.139:0/4031802641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ed00ccc90 0x7f7ed00ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ecffff640 1 -- 172.21.15.139:0/4031802641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ed0154c20 msgr2=0x7f7ed00cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:05.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ecffff640 1 --2- 172.21.15.139:0/4031802641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ed0154c20 0x7f7ed00cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ecffff640 1 -- 172.21.15.139:0/4031802641 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed00b5f50 con 0x7f7ed0068060 2024-09-16T00:08:05.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ecffff640 1 --2- 172.21.15.139:0/4031802641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ed0068060 0x7f7ed0147ae0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000d620 tx=0x7f7ec000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:05.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7eccff9640 1 -- 172.21.15.139:0/4031802641 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec000b620 con 0x7f7ed0068060 2024-09-16T00:08:05.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7eccff9640 1 -- 172.21.15.139:0/4031802641 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec0019070 con 0x7f7ed0068060 2024-09-16T00:08:05.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ed6477640 1 -- 172.21.15.139:0/4031802641 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed00b6230 con 0x7f7ed0068060 2024-09-16T00:08:05.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7eccff9640 1 -- 172.21.15.139:0/4031802641 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec000ba00 con 0x7f7ed0068060 2024-09-16T00:08:05.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.692+0000 7f7ed6477640 1 -- 172.21.15.139:0/4031802641 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed01554d0 con 0x7f7ed0068060 2024-09-16T00:08:05.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.694+0000 7f7ed6477640 1 -- 172.21.15.139:0/4031802641 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed00686b0 con 0x7f7ed0068060 2024-09-16T00:08:05.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.694+0000 7f7eccff9640 1 -- 172.21.15.139:0/4031802641 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7ec000bba0 con 0x7f7ed0068060 2024-09-16T00:08:05.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.694+0000 7f7eccff9640 1 --2- 172.21.15.139:0/4031802641 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7ebc06c2b0 0x7f7ebc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:05.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.694+0000 7f7eccff9640 1 -- 172.21.15.139:0/4031802641 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5738+0+0 (secure 0 0 0) 0x7f7ec008b790 con 0x7f7ed0068060 2024-09-16T00:08:05.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.695+0000 7f7ecf7fe640 1 --2- 172.21.15.139:0/4031802641 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7ebc06c2b0 0x7f7ebc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:05.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.695+0000 7f7ecf7fe640 1 --2- 172.21.15.139:0/4031802641 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7ebc06c2b0 0x7f7ebc06e770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7ed01eaf80 tx=0x7f7ec4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:05.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.698+0000 7f7eccff9640 1 -- 172.21.15.139:0/4031802641 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7ec0059890 con 0x7f7ed0068060 2024-09-16T00:08:05.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.734+0000 7fb4b2ffd640 1 -- 172.21.15.139:0/888251408 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v29) ==== 87+0+0 (secure 0 0 0) 0x7fb4b4059790 con 0x7fb4bc0ccb00 2024-09-16T00:08:05.756 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is set 2024-09-16T00:08:05.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.737+0000 7fb4c428d640 1 -- 172.21.15.139:0/888251408 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb4a006c2b0 msgr2=0x7fb4a006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:05.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.737+0000 7fb4c428d640 1 --2- 172.21.15.139:0/888251408 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb4a006c2b0 0x7fb4a006e770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb4b802a1c0 tx=0x7fb4b802e000 comp rx=0 tx=0).stop 2024-09-16T00:08:05.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.737+0000 7fb4c428d640 1 -- 172.21.15.139:0/888251408 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4bc0ccb00 msgr2=0x7fb4bc0bdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:05.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.737+0000 7fb4c428d640 1 --2- 172.21.15.139:0/888251408 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4bc0ccb00 0x7fb4bc0bdb00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400ebd0 tx=0x7fb4b400c490 comp rx=0 tx=0).stop 2024-09-16T00:08:05.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.737+0000 7fb4c428d640 1 -- 172.21.15.139:0/888251408 shutdown_connections 2024-09-16T00:08:05.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.737+0000 7fb4c428d640 1 --2- 172.21.15.139:0/888251408 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb4a006c2b0 0x7fb4a006e770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.737+0000 7fb4c428d640 1 --2- 172.21.15.139:0/888251408 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4bc0ccb00 0x7fb4bc0bdb00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.737+0000 7fb4c428d640 1 --2- 172.21.15.139:0/888251408 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4bc0cc720 0x7fb4bc1f0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.737+0000 7fb4c428d640 1 --2- 172.21.15.139:0/888251408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4bc0c7920 0x7fb4bc0c05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.737+0000 7fb4c428d640 1 -- 172.21.15.139:0/888251408 >> 172.21.15.139:0/888251408 conn(0x7fb4bc067da0 msgr2=0x7fb4bc0c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:05.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.737+0000 7fb4c428d640 1 -- 172.21.15.139:0/888251408 shutdown_connections 2024-09-16T00:08:05.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.737+0000 7fb4c428d640 1 -- 172.21.15.139:0/888251408 wait complete. 2024-09-16T00:08:05.757 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:05.743+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:05.845 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:05.844+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:05.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.872+0000 7f7ed6477640 1 -- 172.21.15.139:0/4031802641 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}) -- 0x7f7ed00cca70 con 0x7f7ebc06c2b0 2024-09-16T00:08:05.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.873+0000 7f7eccff9640 1 -- 172.21.15.139:0/4031802641 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 no change) ==== 17+0+297 (secure 0 0 0) 0x7f7ed00cca70 con 0x7f7ebc06c2b0 2024-09-16T00:08:05.874 INFO:teuthology.orchestra.run.smithi139.stdout:moved 0 / 194 (0%) 2024-09-16T00:08:05.875 INFO:teuthology.orchestra.run.smithi139.stdout:avg 24.25 2024-09-16T00:08:05.875 INFO:teuthology.orchestra.run.smithi139.stdout:stddev 5.78252 -> 5.78252 (expected baseline 4.60638) 2024-09-16T00:08:05.875 INFO:teuthology.orchestra.run.smithi139.stdout:min osd.2 with 13 -> 13 pgs (0.536082 -> 0.536082 * mean) 2024-09-16T00:08:05.875 INFO:teuthology.orchestra.run.smithi139.stdout:max osd.5 with 30 -> 30 pgs (1.23711 -> 1.23711 * mean) 2024-09-16T00:08:05.875 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:08:05.876 INFO:teuthology.orchestra.run.smithi139.stdout:oload 110 2024-09-16T00:08:05.876 INFO:teuthology.orchestra.run.smithi139.stdout:max_change 0.05 2024-09-16T00:08:05.876 INFO:teuthology.orchestra.run.smithi139.stdout:max_change_osds 4 2024-09-16T00:08:05.876 INFO:teuthology.orchestra.run.smithi139.stdout:average_utilization 0.0003 2024-09-16T00:08:05.876 INFO:teuthology.orchestra.run.smithi139.stdout:overload_utilization 0.0003 2024-09-16T00:08:05.878 INFO:teuthology.orchestra.run.smithi139.stderr:no change 2024-09-16T00:08:05.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.875+0000 7f7ed6477640 1 -- 172.21.15.139:0/4031802641 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7ebc06c2b0 msgr2=0x7f7ebc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:05.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.875+0000 7f7ed6477640 1 --2- 172.21.15.139:0/4031802641 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7ebc06c2b0 0x7f7ebc06e770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7ed01eaf80 tx=0x7f7ec4008040 comp rx=0 tx=0).stop 2024-09-16T00:08:05.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.875+0000 7f7ed6477640 1 -- 172.21.15.139:0/4031802641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ed0068060 msgr2=0x7f7ed0147ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:05.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.875+0000 7f7ed6477640 1 --2- 172.21.15.139:0/4031802641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ed0068060 0x7f7ed0147ae0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000d620 tx=0x7f7ec000dae0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.875+0000 7f7ed6477640 1 -- 172.21.15.139:0/4031802641 shutdown_connections 2024-09-16T00:08:05.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.875+0000 7f7ed6477640 1 --2- 172.21.15.139:0/4031802641 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7ebc06c2b0 0x7f7ebc06e770 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.875+0000 7f7ed6477640 1 --2- 172.21.15.139:0/4031802641 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7ed0154c20 0x7f7ed00cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.875+0000 7f7ed6477640 1 --2- 172.21.15.139:0/4031802641 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7ed00ccc90 0x7f7ed00ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.875+0000 7f7ed6477640 1 --2- 172.21.15.139:0/4031802641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ed0068060 0x7f7ed0147ae0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:05.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.875+0000 7f7ed6477640 1 -- 172.21.15.139:0/4031802641 >> 172.21.15.139:0/4031802641 conn(0x7f7ed006ca40 msgr2=0x7f7ed00cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:05.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.875+0000 7f7ed6477640 1 -- 172.21.15.139:0/4031802641 shutdown_connections 2024-09-16T00:08:05.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:05.875+0000 7f7ed6477640 1 -- 172.21.15.139:0/4031802641 wait complete. 2024-09-16T00:08:05.889 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-16T00:08:05.889 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-16T00:08:05.905 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:05.905 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:05.905 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:05.905 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:05.914 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:08:05.955 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:05.954+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:06.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.006+0000 7fc7e7518640 1 Processor -- start 2024-09-16T00:08:06.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.007+0000 7fc7e7518640 1 -- start start 2024-09-16T00:08:06.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.007+0000 7fc7e7518640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc7e01549c0 0x7fc7e00c8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.007+0000 7fc7e7518640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7e00729c0 0x7fc7e0072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.007+0000 7fc7e7518640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7e00c73f0 con 0x7fc7e006bb20 2024-09-16T00:08:06.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.007+0000 7fc7e7518640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fc7e00c8ec0 con 0x7fc7e01549c0 2024-09-16T00:08:06.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.007+0000 7fc7e7518640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e00c3fa0 con 0x7fc7e00729c0 2024-09-16T00:08:06.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.008+0000 7fc7e4ab3640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc7e006bb20 0x7fc7e015e670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37418/0 (socket says 172.21.15.139:37418) 2024-09-16T00:08:06.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.008+0000 7fc7e4ab3640 1 -- 172.21.15.139:0/3258074182 learned_addr learned my addr 172.21.15.139:0/3258074182 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:06.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.008+0000 7fc7e52b4640 1 --2- 172.21.15.139:0/3258074182 >> v2:172.21.15.164:3300/0 conn(0x7fc7e01549c0 0x7fc7e00c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.008+0000 7fc7e5ab5640 1 --2- 172.21.15.139:0/3258074182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7e00729c0 0x7fc7e0072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.008+0000 7fc7d3fff640 1 -- 172.21.15.139:0/3258074182 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2550512360 0 0) 0x7fc7e00c73f0 con 0x7fc7e006bb20 2024-09-16T00:08:06.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.008+0000 7fc7d3fff640 1 -- 172.21.15.139:0/3258074182 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7cc003520 con 0x7fc7e006bb20 2024-09-16T00:08:06.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.008+0000 7fc7e52b4640 1 -- 172.21.15.139:0/3258074182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7e00729c0 msgr2=0x7fc7e0072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:06.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.008+0000 7fc7e52b4640 1 --2- 172.21.15.139:0/3258074182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7e00729c0 0x7fc7e0072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.008+0000 7fc7e52b4640 1 -- 172.21.15.139:0/3258074182 >> v1:172.21.15.139:6789/0 conn(0x7fc7e006bb20 legacy=0x7fc7e015e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.008+0000 7fc7e52b4640 1 -- 172.21.15.139:0/3258074182 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e00c7690 con 0x7fc7e01549c0 2024-09-16T00:08:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.009+0000 7fc7e52b4640 1 --2- 172.21.15.139:0/3258074182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7e01549c0 0x7fc7e00c8980 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40097a0 tx=0x7fc7d402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52e2aa05728751b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.009+0000 7fc7d3fff640 1 -- 172.21.15.139:0/3258074182 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4037070 con 0x7fc7e01549c0 2024-09-16T00:08:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.009+0000 7fc7d3fff640 1 -- 172.21.15.139:0/3258074182 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d4032440 con 0x7fc7e01549c0 2024-09-16T00:08:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.009+0000 7fc7d3fff640 1 -- 172.21.15.139:0/3258074182 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40327f0 con 0x7fc7e01549c0 2024-09-16T00:08:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.010+0000 7fc7e7518640 1 -- 172.21.15.139:0/3258074182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7e01549c0 msgr2=0x7fc7e00c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.010+0000 7fc7e7518640 1 --2- 172.21.15.139:0/3258074182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7e01549c0 0x7fc7e00c8980 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40097a0 tx=0x7fc7d402a210 comp rx=0 tx=0).stop 2024-09-16T00:08:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.010+0000 7fc7e7518640 1 -- 172.21.15.139:0/3258074182 shutdown_connections 2024-09-16T00:08:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.010+0000 7fc7e7518640 1 --2- 172.21.15.139:0/3258074182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7e00729c0 0x7fc7e0072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.010+0000 7fc7e7518640 1 --2- 172.21.15.139:0/3258074182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7e01549c0 0x7fc7e00c8980 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.010+0000 7fc7e7518640 1 -- 172.21.15.139:0/3258074182 >> 172.21.15.139:0/3258074182 conn(0x7fc7e00c90d0 msgr2=0x7fc7e00c94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.010+0000 7fc7e7518640 1 -- 172.21.15.139:0/3258074182 shutdown_connections 2024-09-16T00:08:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.010+0000 7fc7e7518640 1 -- 172.21.15.139:0/3258074182 wait complete. 2024-09-16T00:08:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.011+0000 7fc7e7518640 1 Processor -- start 2024-09-16T00:08:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.011+0000 7fc7e7518640 1 -- start start 2024-09-16T00:08:06.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e7518640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7e006bb20 0x7fc7e01e6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e7518640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7e00729c0 0x7fc7e0157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e7518640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e01549c0 0x7fc7e014d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e7518640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e00c7ea0 con 0x7fc7e01549c0 2024-09-16T00:08:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e7518640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e00c84f0 con 0x7fc7e006bb20 2024-09-16T00:08:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e7518640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e00c8180 con 0x7fc7e00729c0 2024-09-16T00:08:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e5ab5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e01549c0 0x7fc7e014d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e5ab5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e01549c0 0x7fc7e014d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59048/0 (socket says 172.21.15.139:59048) 2024-09-16T00:08:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e5ab5640 1 -- 172.21.15.139:0/2057644662 learned_addr learned my addr 172.21.15.139:0/2057644662 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e52b4640 1 --2- 172.21.15.139:0/2057644662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7e006bb20 0x7fc7e01e6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e4ab3640 1 --2- 172.21.15.139:0/2057644662 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7e00729c0 0x7fc7e0157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e5ab5640 1 -- 172.21.15.139:0/2057644662 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7e00729c0 msgr2=0x7fc7e0157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e5ab5640 1 --2- 172.21.15.139:0/2057644662 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7e00729c0 0x7fc7e0157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e5ab5640 1 -- 172.21.15.139:0/2057644662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7e006bb20 msgr2=0x7fc7e01e6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e5ab5640 1 --2- 172.21.15.139:0/2057644662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7e006bb20 0x7fc7e01e6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e5ab5640 1 -- 172.21.15.139:0/2057644662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e0157b50 con 0x7fc7e01549c0 2024-09-16T00:08:06.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.012+0000 7fc7e5ab5640 1 --2- 172.21.15.139:0/2057644662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e01549c0 0x7fc7e014d340 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009dd0 tx=0x7fc7d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.013+0000 7fc7d1ffb640 1 -- 172.21.15.139:0/2057644662 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8018020 con 0x7fc7e01549c0 2024-09-16T00:08:06.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.013+0000 7fc7d1ffb640 1 -- 172.21.15.139:0/2057644662 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7d8014480 con 0x7fc7e01549c0 2024-09-16T00:08:06.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.013+0000 7fc7e7518640 1 -- 172.21.15.139:0/2057644662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e0157df0 con 0x7fc7e01549c0 2024-09-16T00:08:06.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.013+0000 7fc7d1ffb640 1 -- 172.21.15.139:0/2057644662 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8014870 con 0x7fc7e01549c0 2024-09-16T00:08:06.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.013+0000 7fc7e7518640 1 -- 172.21.15.139:0/2057644662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e0158090 con 0x7fc7e01549c0 2024-09-16T00:08:06.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.014+0000 7fc7e7518640 1 -- 172.21.15.139:0/2057644662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e0154e30 con 0x7fc7e01549c0 2024-09-16T00:08:06.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.014+0000 7fc7d1ffb640 1 -- 172.21.15.139:0/2057644662 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc7d8005d10 con 0x7fc7e01549c0 2024-09-16T00:08:06.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.015+0000 7fc7d1ffb640 1 --2- 172.21.15.139:0/2057644662 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc7c406c2b0 0x7fc7c406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.015+0000 7fc7d1ffb640 1 -- 172.21.15.139:0/2057644662 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5738+0+0 (secure 0 0 0) 0x7fc7d808be00 con 0x7fc7e01549c0 2024-09-16T00:08:06.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.015+0000 7fc7e52b4640 1 --2- 172.21.15.139:0/2057644662 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc7c406c2b0 0x7fc7c406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.015+0000 7fc7e52b4640 1 --2- 172.21.15.139:0/2057644662 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc7c406c2b0 0x7fc7c406e770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4034000 tx=0x7fc7d402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.019+0000 7fc7d1ffb640 1 -- 172.21.15.139:0/2057644662 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc7d8022070 con 0x7fc7e01549c0 2024-09-16T00:08:06.047 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:06.046+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:06.077 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:06.077 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:06.077 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:06.078 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:06.078 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:06.087 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:08:06.148 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:06.147+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:06.192 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:22 2.e e22/19 2:717a0223:::608.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.574544625000001, 2024-09-16T00:08:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042195870000000003, 2024-09-16T00:08:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616426+0000", 2024-09-16T00:08:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.037e-06 2024-09-16T00:08:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616435+0000", 2024-09-16T00:08:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6440000000000006e-06 2024-09-16T00:08:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616447+0000", 2024-09-16T00:08:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2466000000000001e-05 2024-09-16T00:08:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616456+0000", 2024-09-16T00:08:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8799999999999997e-06 2024-09-16T00:08:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616517+0000", 2024-09-16T00:08:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1171000000000001e-05 2024-09-16T00:08:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616540+0000", 2024-09-16T00:08:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2991000000000001e-05 2024-09-16T00:08:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620599+0000", 2024-09-16T00:08:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040586140000000003 2024-09-16T00:08:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620645+0000", 2024-09-16T00:08:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5784000000000001e-05 2024-09-16T00:08:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.574454894999999, 2024-09-16T00:08:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044836249999999998, 2024-09-16T00:08:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616518+0000", 2024-09-16T00:08:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3699999999999999e-06 2024-09-16T00:08:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:08:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1479999999999997e-06 2024-09-16T00:08:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616534+0000", 2024-09-16T00:08:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0641e-05 2024-09-16T00:08:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616541+0000", 2024-09-16T00:08:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3339999999999996e-06 2024-09-16T00:08:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617318+0000", 2024-09-16T00:08:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00077694799999999996 2024-09-16T00:08:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617409+0000", 2024-09-16T00:08:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0740000000000002e-05 2024-09-16T00:08:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620998+0000", 2024-09-16T00:08:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035894439999999998 2024-09-16T00:08:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.574406992, 2024-09-16T00:08:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041597320000000002, 2024-09-16T00:08:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:08:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616566+0000", 2024-09-16T00:08:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6979999999999998e-06 2024-09-16T00:08:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616574+0000", 2024-09-16T00:08:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7759999999999997e-06 2024-09-16T00:08:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616580+0000", 2024-09-16T00:08:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616800+0000", 2024-09-16T00:08:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022016700000000001 2024-09-16T00:08:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616904+0000", 2024-09-16T00:08:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104579 2024-09-16T00:08:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620722+0000", 2024-09-16T00:08:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003818133 2024-09-16T00:08:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.574156306999999, 2024-09-16T00:08:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038451570000000001, 2024-09-16T00:08:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616814+0000", 2024-09-16T00:08:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9299999999999997e-07 2024-09-16T00:08:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616824+0000", 2024-09-16T00:08:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6600000000000007e-06 2024-09-16T00:08:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616835+0000", 2024-09-16T00:08:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0713000000000001e-05 2024-09-16T00:08:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616841+0000", 2024-09-16T00:08:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.934e-06 2024-09-16T00:08:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616948+0000", 2024-09-16T00:08:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010631100000000001 2024-09-16T00:08:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616967+0000", 2024-09-16T00:08:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9598000000000001e-05 2024-09-16T00:08:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620619+0000", 2024-09-16T00:08:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036515810000000001 2024-09-16T00:08:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620659+0000", 2024-09-16T00:08:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9666999999999998e-05 2024-09-16T00:08:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.574109355000001, 2024-09-16T00:08:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038240129999999998, 2024-09-16T00:08:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616861+0000", 2024-09-16T00:08:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0200000000000003e-07 2024-09-16T00:08:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616864+0000", 2024-09-16T00:08:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5090000000000002e-06 2024-09-16T00:08:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616871+0000", 2024-09-16T00:08:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9249999999999998e-06 2024-09-16T00:08:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616877+0000", 2024-09-16T00:08:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.981e-06 2024-09-16T00:08:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617168+0000", 2024-09-16T00:08:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029035699999999998 2024-09-16T00:08:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617184+0000", 2024-09-16T00:08:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6005999999999999e-05 2024-09-16T00:08:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:06.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620661+0000", 2024-09-16T00:08:06.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034775330000000001 2024-09-16T00:08:06.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.218 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:06.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620684+0000", 2024-09-16T00:08:06.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3300000000000001e-05 2024-09-16T00:08:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:06.219 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.573653912000001, 2024-09-16T00:08:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045415610000000004, 2024-09-16T00:08:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:06.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8800000000000002e-07 2024-09-16T00:08:06.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617321+0000", 2024-09-16T00:08:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6979999999999997e-06 2024-09-16T00:08:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:06.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617330+0000", 2024-09-16T00:08:06.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6729999999999998e-06 2024-09-16T00:08:06.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617336+0000", 2024-09-16T00:08:06.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2110000000000003e-06 2024-09-16T00:08:06.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:06.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617394+0000", 2024-09-16T00:08:06.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8087999999999997e-05 2024-09-16T00:08:06.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617412+0000", 2024-09-16T00:08:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8099999999999999e-05 2024-09-16T00:08:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621818+0000", 2024-09-16T00:08:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044061209999999998 2024-09-16T00:08:06.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621857+0000", 2024-09-16T00:08:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9082000000000003e-05 2024-09-16T00:08:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:23 2.e e22/19 2:717a0223:::608.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.572757890999998, 2024-09-16T00:08:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036198279999999998, 2024-09-16T00:08:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:06.228 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.229 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618213+0000", 2024-09-16T00:08:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:08:06.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618218+0000", 2024-09-16T00:08:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4099999999999999e-06 2024-09-16T00:08:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:06.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618227+0000", 2024-09-16T00:08:06.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5009999999999993e-06 2024-09-16T00:08:06.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618234+0000", 2024-09-16T00:08:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618307+0000", 2024-09-16T00:08:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3421000000000001e-05 2024-09-16T00:08:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618328+0000", 2024-09-16T00:08:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0919000000000001e-05 2024-09-16T00:08:06.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621804+0000", 2024-09-16T00:08:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034756560000000001 2024-09-16T00:08:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621832+0000", 2024-09-16T00:08:06.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7966999999999998e-05 2024-09-16T00:08:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing_reply(2.1e epoch 22/19 rep_tid 0 lcod 22'1)", 2024-09-16T00:08:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.572312723, 2024-09-16T00:08:06.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1371000000000003e-05, 2024-09-16T00:08:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:06.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:08:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:08:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:06.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618662+0000", 2024-09-16T00:08:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1189999999999999e-06 2024-09-16T00:08:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618674+0000", 2024-09-16T00:08:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2317000000000001e-05 2024-09-16T00:08:06.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618681+0000", 2024-09-16T00:08:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7170000000000002e-06 2024-09-16T00:08:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:06.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:08:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6745000000000001e-05 2024-09-16T00:08:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618748+0000", 2024-09-16T00:08:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0794000000000001e-05 2024-09-16T00:08:06.242 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:06.243 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.571799388999999, 2024-09-16T00:08:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027183849999999998, 2024-09-16T00:08:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:06.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619171+0000", 2024-09-16T00:08:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8999999999999995e-07 2024-09-16T00:08:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:06.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619179+0000", 2024-09-16T00:08:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2679999999999994e-06 2024-09-16T00:08:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:08:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1246e-05 2024-09-16T00:08:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619197+0000", 2024-09-16T00:08:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2820000000000002e-06 2024-09-16T00:08:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:06.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619308+0000", 2024-09-16T00:08:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011141000000000001 2024-09-16T00:08:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619325+0000", 2024-09-16T00:08:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6469000000000001e-05 2024-09-16T00:08:06.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621860+0000", 2024-09-16T00:08:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025353590000000001 2024-09-16T00:08:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621889+0000", 2024-09-16T00:08:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8561e-05 2024-09-16T00:08:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 6 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.571756690000001, 2024-09-16T00:08:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026903809999999999, 2024-09-16T00:08:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:06.252 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:06.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2000000000000001e-07 2024-09-16T00:08:06.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:06.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619219+0000", 2024-09-16T00:08:06.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:08:06.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:06.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619234+0000", 2024-09-16T00:08:06.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5075000000000001e-05 2024-09-16T00:08:06.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619251+0000", 2024-09-16T00:08:06.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6863000000000001e-05 2024-09-16T00:08:06.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:06.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619556+0000", 2024-09-16T00:08:06.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030551599999999998 2024-09-16T00:08:06.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:06.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621903+0000", 2024-09-16T00:08:06.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023469979999999999 2024-09-16T00:08:06.258 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.258 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:06.258 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.258 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing_reply(2.1b epoch 22/19 rep_tid 1 lcod 22'1)", 2024-09-16T00:08:06.259 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:06.259 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.570678294, 2024-09-16T00:08:06.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6989000000000003e-05, 2024-09-16T00:08:06.259 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:06.259 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:06.259 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:06.259 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:06.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:06.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:06.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:06.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:06.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:06.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620292+0000", 2024-09-16T00:08:06.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.37e-07 2024-09-16T00:08:06.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:06.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620296+0000", 2024-09-16T00:08:06.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2420000000000002e-06 2024-09-16T00:08:06.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:06.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620308+0000", 2024-09-16T00:08:06.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1623e-05 2024-09-16T00:08:06.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620313+0000", 2024-09-16T00:08:06.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6290000000000001e-06 2024-09-16T00:08:06.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:06.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620359+0000", 2024-09-16T00:08:06.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6201e-05 2024-09-16T00:08:06.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:06.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620378+0000", 2024-09-16T00:08:06.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9556999999999999e-05 2024-09-16T00:08:06.265 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.265 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:06.265 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.265 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:38 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'6, mlcod=22'4)", 2024-09-16T00:08:06.266 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:06.266 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.568236044999999, 2024-09-16T00:08:06.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064551599999999995, 2024-09-16T00:08:06.266 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:06.266 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:06.266 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:06.266 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:06.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:06.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:06.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:06.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:06.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:06.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:06.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4399999999999999e-07 2024-09-16T00:08:06.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:06.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622739+0000", 2024-09-16T00:08:06.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0250000000000002e-06 2024-09-16T00:08:06.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:06.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622749+0000", 2024-09-16T00:08:06.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1220000000000001e-06 2024-09-16T00:08:06.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622760+0000", 2024-09-16T00:08:06.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1640999999999999e-05 2024-09-16T00:08:06.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:06.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622799+0000", 2024-09-16T00:08:06.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8537000000000001e-05 2024-09-16T00:08:06.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:06.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622810+0000", 2024-09-16T00:08:06.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1562e-05 2024-09-16T00:08:06.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:06.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.623356+0000", 2024-09-16T00:08:06.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054522299999999995 2024-09-16T00:08:06.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:06.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.623379+0000", 2024-09-16T00:08:06.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3662000000000001e-05 2024-09-16T00:08:06.273 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.274 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:06.274 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.274 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:40 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'7, mlcod=22'5)", 2024-09-16T00:08:06.274 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:06.275 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 15.351215142999999, 2024-09-16T00:08:06.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001017773, 2024-09-16T00:08:06.275 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:06.275 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:06.275 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:06.275 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:06.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:06.276 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:06.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.276 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:06.276 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:06.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:06.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839756+0000", 2024-09-16T00:08:06.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.491e-06 2024-09-16T00:08:06.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:06.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839768+0000", 2024-09-16T00:08:06.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1649e-05 2024-09-16T00:08:06.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:06.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839783+0000", 2024-09-16T00:08:06.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5023e-05 2024-09-16T00:08:06.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839791+0000", 2024-09-16T00:08:06.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5620000000000004e-06 2024-09-16T00:08:06.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:06.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839848+0000", 2024-09-16T00:08:06.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6563e-05 2024-09-16T00:08:06.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:06.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839868+0000", 2024-09-16T00:08:06.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9652000000000001e-05 2024-09-16T00:08:06.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:06.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.840729+0000", 2024-09-16T00:08:06.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086159500000000005 2024-09-16T00:08:06.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:06.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.840772+0000", 2024-09-16T00:08:06.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3238000000000002e-05 2024-09-16T00:08:06.282 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.282 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:06.282 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.283 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:41 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'8, mlcod=22'6)", 2024-09-16T00:08:06.283 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:06.283 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 13.077558231999999, 2024-09-16T00:08:06.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001040568, 2024-09-16T00:08:06.283 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:06.283 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:06.283 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:06.284 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:06.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:06.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:06.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:06.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:06.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:06.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:06.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113413+0000", 2024-09-16T00:08:06.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7409999999999999e-06 2024-09-16T00:08:06.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:06.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113428+0000", 2024-09-16T00:08:06.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5007999999999999e-05 2024-09-16T00:08:06.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:06.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113447+0000", 2024-09-16T00:08:06.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8731000000000001e-05 2024-09-16T00:08:06.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.288 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113457+0000", 2024-09-16T00:08:06.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0095000000000001e-05 2024-09-16T00:08:06.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:06.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113526+0000", 2024-09-16T00:08:06.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8552000000000004e-05 2024-09-16T00:08:06.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:06.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113552+0000", 2024-09-16T00:08:06.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6795999999999999e-05 2024-09-16T00:08:06.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:06.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.114417+0000", 2024-09-16T00:08:06.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086476299999999995 2024-09-16T00:08:06.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:06.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:06.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:06.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.114452+0000", 2024-09-16T00:08:06.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4882000000000002e-05 2024-09-16T00:08:06.291 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.292 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:06.292 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.292 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:06.292 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:06.292 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:06.295 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:08:06.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.213+0000 7fc7e7518640 1 -- 172.21.15.139:0/2057644662 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}) -- 0x7fc7e00c79a0 con 0x7fc7c406c2b0 2024-09-16T00:08:06.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.213+0000 7fc7d1ffb640 1 -- 172.21.15.139:0/2057644662 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 no change) ==== 17+0+295 (secure 0 0 0) 0x7fc7e00c79a0 con 0x7fc7c406c2b0 2024-09-16T00:08:06.296 INFO:teuthology.orchestra.run.smithi139.stdout:moved 0 / 194 (0%) 2024-09-16T00:08:06.297 INFO:teuthology.orchestra.run.smithi139.stdout:avg 24.25 2024-09-16T00:08:06.297 INFO:teuthology.orchestra.run.smithi139.stdout:stddev 5.78252 -> 5.78252 (expected baseline 4.60638) 2024-09-16T00:08:06.297 INFO:teuthology.orchestra.run.smithi139.stdout:min osd.2 with 13 -> 13 pgs (0.536082 -> 0.536082 * mean) 2024-09-16T00:08:06.297 INFO:teuthology.orchestra.run.smithi139.stdout:max osd.5 with 30 -> 30 pgs (1.23711 -> 1.23711 * mean) 2024-09-16T00:08:06.297 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:08:06.297 INFO:teuthology.orchestra.run.smithi139.stdout:oload 1000 2024-09-16T00:08:06.298 INFO:teuthology.orchestra.run.smithi139.stdout:max_change 3 2024-09-16T00:08:06.298 INFO:teuthology.orchestra.run.smithi139.stdout:max_change_osds 4 2024-09-16T00:08:06.298 INFO:teuthology.orchestra.run.smithi139.stdout:average_utilization 0.0003 2024-09-16T00:08:06.298 INFO:teuthology.orchestra.run.smithi139.stdout:overload_utilization 0.0029 2024-09-16T00:08:06.301 INFO:teuthology.orchestra.run.smithi139.stderr:no change 2024-09-16T00:08:06.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.215+0000 7fc7e7518640 1 -- 172.21.15.139:0/2057644662 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc7c406c2b0 msgr2=0x7fc7c406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.215+0000 7fc7e7518640 1 --2- 172.21.15.139:0/2057644662 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc7c406c2b0 0x7fc7c406e770 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4034000 tx=0x7fc7d402e000 comp rx=0 tx=0).stop 2024-09-16T00:08:06.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.215+0000 7fc7e7518640 1 -- 172.21.15.139:0/2057644662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e01549c0 msgr2=0x7fc7e014d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.215+0000 7fc7e7518640 1 --2- 172.21.15.139:0/2057644662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e01549c0 0x7fc7e014d340 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009dd0 tx=0x7fc7d800c490 comp rx=0 tx=0).stop 2024-09-16T00:08:06.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.215+0000 7fc7e7518640 1 -- 172.21.15.139:0/2057644662 shutdown_connections 2024-09-16T00:08:06.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.215+0000 7fc7e7518640 1 --2- 172.21.15.139:0/2057644662 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc7c406c2b0 0x7fc7c406e770 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.215+0000 7fc7e7518640 1 --2- 172.21.15.139:0/2057644662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7e01549c0 0x7fc7e014d340 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.215+0000 7fc7e7518640 1 --2- 172.21.15.139:0/2057644662 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7e00729c0 0x7fc7e0157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.215+0000 7fc7e7518640 1 --2- 172.21.15.139:0/2057644662 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7e006bb20 0x7fc7e01e6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.216+0000 7fc7e7518640 1 -- 172.21.15.139:0/2057644662 >> 172.21.15.139:0/2057644662 conn(0x7fc7e00c90d0 msgr2=0x7fc7e014f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:06.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.216+0000 7fc7e7518640 1 -- 172.21.15.139:0/2057644662 shutdown_connections 2024-09-16T00:08:06.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.216+0000 7fc7e7518640 1 -- 172.21.15.139:0/2057644662 wait complete. 2024-09-16T00:08:06.303 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-16T00:08:06.303 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-16T00:08:06.304 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:06.293+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:06.395 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:06.394+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:06.421 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:06.421 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:06.421 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:06.421 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:06.430 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:08:06.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.435+0000 7f375cc7b640 1 Processor -- start 2024-09-16T00:08:06.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.435+0000 7f375cc7b640 1 -- start start 2024-09-16T00:08:06.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.436+0000 7f375cc7b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f375806ea10 0x7f3758071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.436+0000 7f375cc7b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37580ca000 con 0x7f3758154d40 2024-09-16T00:08:06.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.436+0000 7f375cc7b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37580c9900 con 0x7f37580c93d0 2024-09-16T00:08:06.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.436+0000 7f375cc7b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f375806d340 con 0x7f375806ea10 2024-09-16T00:08:06.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.436+0000 7f374dd9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f375806ea10 0x7f3758071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.436+0000 7f375659c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f3758154d40 0x7f375806cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37426/0 (socket says 172.21.15.139:37426) 2024-09-16T00:08:06.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.436+0000 7f375659c640 1 -- 172.21.15.139:0/255061851 learned_addr learned my addr 172.21.15.139:0/255061851 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:06.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.436+0000 7f374dd9b640 1 -- 172.21.15.139:0/255061851 >> v1:172.21.15.164:6789/0 conn(0x7f37580c93d0 legacy=0x7f37581650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.436+0000 7f374dd9b640 1 -- 172.21.15.139:0/255061851 >> v1:172.21.15.139:6789/0 conn(0x7f3758154d40 legacy=0x7f375806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.436+0000 7f374dd9b640 1 -- 172.21.15.139:0/255061851 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37580c8d60 con 0x7f375806ea10 2024-09-16T00:08:06.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.437+0000 7f374dd9b640 1 --2- 172.21.15.139:0/255061851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f375806ea10 0x7f3758071520 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f37400097a0 tx=0x7f374002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcef7e1faac9f32 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.437+0000 7f3755d9b640 1 -- 172.21.15.139:0/255061851 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740037070 con 0x7f375806ea10 2024-09-16T00:08:06.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.437+0000 7f3755d9b640 1 -- 172.21.15.139:0/255061851 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3740032640 con 0x7f375806ea10 2024-09-16T00:08:06.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.437+0000 7f3755d9b640 1 -- 172.21.15.139:0/255061851 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740032a10 con 0x7f375806ea10 2024-09-16T00:08:06.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.437+0000 7f375cc7b640 1 -- 172.21.15.139:0/255061851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f375806ea10 msgr2=0x7f3758071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.437+0000 7f375cc7b640 1 --2- 172.21.15.139:0/255061851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f375806ea10 0x7f3758071520 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f37400097a0 tx=0x7f374002a2c0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.437+0000 7f375cc7b640 1 -- 172.21.15.139:0/255061851 shutdown_connections 2024-09-16T00:08:06.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.437+0000 7f375cc7b640 1 --2- 172.21.15.139:0/255061851 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f375806ea10 0x7f3758071520 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.437+0000 7f375cc7b640 1 -- 172.21.15.139:0/255061851 >> 172.21.15.139:0/255061851 conn(0x7f375806bb20 msgr2=0x7f375806f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:06.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.437+0000 7f375cc7b640 1 -- 172.21.15.139:0/255061851 shutdown_connections 2024-09-16T00:08:06.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.437+0000 7f375cc7b640 1 -- 172.21.15.139:0/255061851 wait complete. 2024-09-16T00:08:06.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.439+0000 7f375cc7b640 1 Processor -- start 2024-09-16T00:08:06.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.439+0000 7f375cc7b640 1 -- start start 2024-09-16T00:08:06.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.439+0000 7f375cc7b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37580c93d0 0x7f375806fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.439+0000 7f375cc7b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3758154d40 0x7f37580c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f375cc7b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3758157f70 0x7f375814cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f375cc7b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f37580ca390 con 0x7f3758154d40 2024-09-16T00:08:06.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f375cc7b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f37580c9db0 con 0x7f3758157f70 2024-09-16T00:08:06.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f375cc7b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f37580c52d0 con 0x7f37580c93d0 2024-09-16T00:08:06.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f374dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3758154d40 0x7f37580c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f374dd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3758154d40 0x7f37580c5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59056/0 (socket says 172.21.15.139:59056) 2024-09-16T00:08:06.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f374dd9b640 1 -- 172.21.15.139:0/701131836 learned_addr learned my addr 172.21.15.139:0/701131836 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:06.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f375659c640 1 --2- 172.21.15.139:0/701131836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37580c93d0 0x7f375806fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f3756d9d640 1 --2- 172.21.15.139:0/701131836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3758157f70 0x7f375814cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f374dd9b640 1 -- 172.21.15.139:0/701131836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37580c93d0 msgr2=0x7f375806fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f374dd9b640 1 --2- 172.21.15.139:0/701131836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37580c93d0 0x7f375806fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f374dd9b640 1 -- 172.21.15.139:0/701131836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3758157f70 msgr2=0x7f375814cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f374dd9b640 1 --2- 172.21.15.139:0/701131836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3758157f70 0x7f375814cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f374dd9b640 1 -- 172.21.15.139:0/701131836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375814bec0 con 0x7f3758154d40 2024-09-16T00:08:06.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f374dd9b640 1 --2- 172.21.15.139:0/701131836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3758154d40 0x7f37580c5c10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f37400098c0 tx=0x7f37400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f374f7fe640 1 -- 172.21.15.139:0/701131836 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740041030 con 0x7f3758154d40 2024-09-16T00:08:06.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f374f7fe640 1 -- 172.21.15.139:0/701131836 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3740030750 con 0x7f3758154d40 2024-09-16T00:08:06.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f375cc7b640 1 -- 172.21.15.139:0/701131836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375814c1a0 con 0x7f3758154d40 2024-09-16T00:08:06.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f374f7fe640 1 -- 172.21.15.139:0/701131836 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740037040 con 0x7f3758154d40 2024-09-16T00:08:06.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.440+0000 7f375cc7b640 1 -- 172.21.15.139:0/701131836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f375814c760 con 0x7f3758154d40 2024-09-16T00:08:06.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.441+0000 7f375cc7b640 1 -- 172.21.15.139:0/701131836 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37580c4ea0 con 0x7f3758154d40 2024-09-16T00:08:06.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.442+0000 7f374f7fe640 1 -- 172.21.15.139:0/701131836 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3740044070 con 0x7f3758154d40 2024-09-16T00:08:06.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.442+0000 7f374f7fe640 1 --2- 172.21.15.139:0/701131836 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f373806c2b0 0x7f373806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.442+0000 7f374f7fe640 1 -- 172.21.15.139:0/701131836 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5738+0+0 (secure 0 0 0) 0x7f37400abbb0 con 0x7f3758154d40 2024-09-16T00:08:06.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.442+0000 7f375659c640 1 --2- 172.21.15.139:0/701131836 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f373806c2b0 0x7f373806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.443+0000 7f375659c640 1 --2- 172.21.15.139:0/701131836 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f373806c2b0 0x7f373806e770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3748000f60 tx=0x7f3748009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.446+0000 7f374f7fe640 1 -- 172.21.15.139:0/701131836 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3740079cb0 con 0x7f3758154d40 2024-09-16T00:08:06.496 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:06.496+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:06.534 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:06.534 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:06.534 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:06.534 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:06.535 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:06.543 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:08:06.597 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:06.597+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:06.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.610+0000 7f375cc7b640 1 -- 172.21.15.139:0/701131836 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}) -- 0x7f37581570f0 con 0x7f373806c2b0 2024-09-16T00:08:06.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.611+0000 7f374f7fe640 1 -- 172.21.15.139:0/701131836 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 no change) ==== 17+0+294 (secure 0 0 0) 0x7f37581570f0 con 0x7f373806c2b0 2024-09-16T00:08:06.612 INFO:teuthology.orchestra.run.smithi139.stdout:moved 0 / 194 (0%) 2024-09-16T00:08:06.612 INFO:teuthology.orchestra.run.smithi139.stdout:avg 24.25 2024-09-16T00:08:06.612 INFO:teuthology.orchestra.run.smithi139.stdout:stddev 5.78252 -> 5.78252 (expected baseline 4.60638) 2024-09-16T00:08:06.613 INFO:teuthology.orchestra.run.smithi139.stdout:min osd.2 with 13 -> 13 pgs (0.536082 -> 0.536082 * mean) 2024-09-16T00:08:06.613 INFO:teuthology.orchestra.run.smithi139.stdout:max osd.5 with 30 -> 30 pgs (1.23711 -> 1.23711 * mean) 2024-09-16T00:08:06.613 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:08:06.613 INFO:teuthology.orchestra.run.smithi139.stdout:oload 110 2024-09-16T00:08:06.613 INFO:teuthology.orchestra.run.smithi139.stdout:max_change 3 2024-09-16T00:08:06.613 INFO:teuthology.orchestra.run.smithi139.stdout:max_change_osds 4 2024-09-16T00:08:06.613 INFO:teuthology.orchestra.run.smithi139.stdout:average_utilization 0.0003 2024-09-16T00:08:06.613 INFO:teuthology.orchestra.run.smithi139.stdout:overload_utilization 0.0003 2024-09-16T00:08:06.613 INFO:teuthology.orchestra.run.smithi139.stderr:no change 2024-09-16T00:08:06.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.613+0000 7f375cc7b640 1 -- 172.21.15.139:0/701131836 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f373806c2b0 msgr2=0x7f373806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.613+0000 7f375cc7b640 1 --2- 172.21.15.139:0/701131836 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f373806c2b0 0x7f373806e770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3748000f60 tx=0x7f3748009040 comp rx=0 tx=0).stop 2024-09-16T00:08:06.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.613+0000 7f375cc7b640 1 -- 172.21.15.139:0/701131836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3758154d40 msgr2=0x7f37580c5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.613+0000 7f375cc7b640 1 --2- 172.21.15.139:0/701131836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3758154d40 0x7f37580c5c10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f37400098c0 tx=0x7f37400046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.613+0000 7f375cc7b640 1 -- 172.21.15.139:0/701131836 shutdown_connections 2024-09-16T00:08:06.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.613+0000 7f375cc7b640 1 --2- 172.21.15.139:0/701131836 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f373806c2b0 0x7f373806e770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.613+0000 7f375cc7b640 1 --2- 172.21.15.139:0/701131836 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3758157f70 0x7f375814cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.613+0000 7f375cc7b640 1 --2- 172.21.15.139:0/701131836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3758154d40 0x7f37580c5c10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.613+0000 7f375cc7b640 1 --2- 172.21.15.139:0/701131836 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f37580c93d0 0x7f375806fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.613+0000 7f375cc7b640 1 -- 172.21.15.139:0/701131836 >> 172.21.15.139:0/701131836 conn(0x7f375806bb20 msgr2=0x7f375806d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:06.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.613+0000 7f375cc7b640 1 -- 172.21.15.139:0/701131836 shutdown_connections 2024-09-16T00:08:06.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.613+0000 7f375cc7b640 1 -- 172.21.15.139:0/701131836 wait complete. 2024-09-16T00:08:06.649 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:06.649 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.033868366, 2024-09-16T00:08:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036770370000000002, 2024-09-16T00:08:06.650 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:06.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615154+0000", 2024-09-16T00:08:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1510000000000003e-06 2024-09-16T00:08:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615163+0000", 2024-09-16T00:08:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4490000000000005e-06 2024-09-16T00:08:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615182+0000", 2024-09-16T00:08:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8876e-05 2024-09-16T00:08:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615197+0000", 2024-09-16T00:08:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4671000000000001e-05 2024-09-16T00:08:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615349+0000", 2024-09-16T00:08:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015258700000000001 2024-09-16T00:08:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:06.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:08:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000204716 2024-09-16T00:08:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:08:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017250800000000001 2024-09-16T00:08:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616919+0000", 2024-09-16T00:08:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011921799999999999 2024-09-16T00:08:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:08:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018087769999999999 2024-09-16T00:08:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618774+0000", 2024-09-16T00:08:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6075999999999997e-05 2024-09-16T00:08:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618826+0000", 2024-09-16T00:08:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2046000000000003e-05 2024-09-16T00:08:06.656 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.032512577999999, 2024-09-16T00:08:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023501059999999998, 2024-09-16T00:08:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:06.657 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616506+0000", 2024-09-16T00:08:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4100000000000001e-06 2024-09-16T00:08:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617073+0000", 2024-09-16T00:08:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056725899999999997 2024-09-16T00:08:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:06.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617099+0000", 2024-09-16T00:08:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5860999999999999e-05 2024-09-16T00:08:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617114+0000", 2024-09-16T00:08:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5180999999999999e-05 2024-09-16T00:08:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617236+0000", 2024-09-16T00:08:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121662 2024-09-16T00:08:06.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001618733 2024-09-16T00:08:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:31 2.d e22/19 2:b50e409b:::mds_snaptable:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.031618430000002, 2024-09-16T00:08:06.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021851240000000001, 2024-09-16T00:08:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:06.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617400+0000", 2024-09-16T00:08:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.28e-06 2024-09-16T00:08:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617410+0000", 2024-09-16T00:08:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0179e-05 2024-09-16T00:08:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:06.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617435+0000", 2024-09-16T00:08:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5221000000000001e-05 2024-09-16T00:08:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617444+0000", 2024-09-16T00:08:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1989999999999995e-06 2024-09-16T00:08:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617514+0000", 2024-09-16T00:08:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9104999999999997e-05 2024-09-16T00:08:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617538+0000", 2024-09-16T00:08:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4383999999999999e-05 2024-09-16T00:08:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619536+0000", 2024-09-16T00:08:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019982789999999999 2024-09-16T00:08:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619584+0000", 2024-09-16T00:08:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7477e-05 2024-09-16T00:08:06.665 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:27 2.13 e22/19 2:cd5a64a3:::100.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.031313248, 2024-09-16T00:08:06.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018775459999999999, 2024-09-16T00:08:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:06.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617705+0000", 2024-09-16T00:08:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9900000000000009e-07 2024-09-16T00:08:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617718+0000", 2024-09-16T00:08:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3453e-05 2024-09-16T00:08:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617730+0000", 2024-09-16T00:08:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2221e-05 2024-09-16T00:08:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617739+0000", 2024-09-16T00:08:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0660000000000004e-06 2024-09-16T00:08:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617806+0000", 2024-09-16T00:08:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7397999999999994e-05 2024-09-16T00:08:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617823+0000", 2024-09-16T00:08:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7569999999999999e-05 2024-09-16T00:08:06.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619528+0000", 2024-09-16T00:08:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017045859999999999 2024-09-16T00:08:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619581+0000", 2024-09-16T00:08:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3253e-05 2024-09-16T00:08:06.671 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:28 2.13 e22/19 2:c91d4a1d:::mds0_inotable:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.030965978000001, 2024-09-16T00:08:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001558051, 2024-09-16T00:08:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:06.672 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:06.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618052+0000", 2024-09-16T00:08:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1399999999999997e-07 2024-09-16T00:08:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618065+0000", 2024-09-16T00:08:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3495e-05 2024-09-16T00:08:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:06.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:08:06.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0560999999999999e-05 2024-09-16T00:08:06.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618083+0000", 2024-09-16T00:08:06.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:08:06.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:06.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618147+0000", 2024-09-16T00:08:06.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4525999999999999e-05 2024-09-16T00:08:06.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:06.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618166+0000", 2024-09-16T00:08:06.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9045000000000001e-05 2024-09-16T00:08:06.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:06.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619585+0000", 2024-09-16T00:08:06.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014182660000000001 2024-09-16T00:08:06.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:06.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619609+0000", 2024-09-16T00:08:06.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4468e-05 2024-09-16T00:08:06.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.677 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:06.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.677 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:29 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:06.677 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:06.678 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.030615474000001, 2024-09-16T00:08:06.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012347219999999999, 2024-09-16T00:08:06.678 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:06.678 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:06.678 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:06.678 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:06.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:06.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:06.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:06.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:06.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:06.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:06.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:06.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:06.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3099999999999997e-07 2024-09-16T00:08:06.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:06.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618409+0000", 2024-09-16T00:08:06.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6239999999999996e-06 2024-09-16T00:08:06.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:06.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618420+0000", 2024-09-16T00:08:06.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1222e-05 2024-09-16T00:08:06.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618441+0000", 2024-09-16T00:08:06.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0564000000000002e-05 2024-09-16T00:08:06.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:06.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618519+0000", 2024-09-16T00:08:06.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7917999999999995e-05 2024-09-16T00:08:06.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:06.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618535+0000", 2024-09-16T00:08:06.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6266e-05 2024-09-16T00:08:06.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:06.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619612+0000", 2024-09-16T00:08:06.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010767039999999999 2024-09-16T00:08:06.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:06.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619636+0000", 2024-09-16T00:08:06.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4692999999999999e-05 2024-09-16T00:08:06.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.682 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:06.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.682 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:2 2.f e22/19 ondisk, result = 0)", 2024-09-16T00:08:06.682 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:06.683 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.030379447000001, 2024-09-16T00:08:06.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019782099999999999, 2024-09-16T00:08:06.683 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:06.683 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:06.683 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:06.683 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:06.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:06.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:06.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:06.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:06.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:06.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:06.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:06.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618639+0000", 2024-09-16T00:08:06.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5099999999999998e-07 2024-09-16T00:08:06.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:06.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618644+0000", 2024-09-16T00:08:06.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7440000000000004e-06 2024-09-16T00:08:06.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:06.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618655+0000", 2024-09-16T00:08:06.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0798999999999999e-05 2024-09-16T00:08:06.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618662+0000", 2024-09-16T00:08:06.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8700000000000003e-06 2024-09-16T00:08:06.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:06.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618717+0000", 2024-09-16T00:08:06.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5223999999999998e-05 2024-09-16T00:08:06.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:06.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618724+0000", 2024-09-16T00:08:06.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.046e-06 2024-09-16T00:08:06.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:06.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618835+0000", 2024-09-16T00:08:06.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111187 2024-09-16T00:08:06.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.687 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:06.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.687 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.b epoch 22/19 rep_tid 122 entries 22'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:06.687 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:06.687 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.029275053999999, 2024-09-16T00:08:06.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018692890000000001, 2024-09-16T00:08:06.687 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:06.687 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:06.688 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:06.688 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:06.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:06.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:06.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:06.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:06.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:06.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:06.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619743+0000", 2024-09-16T00:08:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1069999999999999e-06 2024-09-16T00:08:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619754+0000", 2024-09-16T00:08:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0789e-05 2024-09-16T00:08:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:06.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619777+0000", 2024-09-16T00:08:06.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2844e-05 2024-09-16T00:08:06.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:06.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619793+0000", 2024-09-16T00:08:06.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6231000000000001e-05 2024-09-16T00:08:06.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:06.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619875+0000", 2024-09-16T00:08:06.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2286000000000005e-05 2024-09-16T00:08:06.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:06.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:06.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:06.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621611+0000", 2024-09-16T00:08:06.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001736032 2024-09-16T00:08:06.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.691 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:06.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:06.691 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:06.691 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:06.692 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:08:06.693 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:08:06.737 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:06.736+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:06.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.789+0000 7fa313f3b640 1 Processor -- start 2024-09-16T00:08:06.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.789+0000 7fa313f3b640 1 -- start start 2024-09-16T00:08:06.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.790+0000 7fa313f3b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa30c0c8070 0x7fa30c15c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.790+0000 7fa313f3b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa30c0c9220 0x7fa30c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.790+0000 7fa313f3b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa30c071830 con 0x7fa30c0c72a0 2024-09-16T00:08:06.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.790+0000 7fa313f3b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa30c15cc40 con 0x7fa30c0c8070 2024-09-16T00:08:06.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.790+0000 7fa313f3b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fa30c15bcb0 con 0x7fa30c0c9220 2024-09-16T00:08:06.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.790+0000 7fa311cd7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa30c0c8070 0x7fa30c15c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.790+0000 7fa3114d6640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa30c0c72a0 0x7fa30c0c5500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37440/0 (socket says 172.21.15.139:37440) 2024-09-16T00:08:06.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.790+0000 7fa3114d6640 1 -- 172.21.15.139:0/704129806 learned_addr learned my addr 172.21.15.139:0/704129806 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:06.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.790+0000 7fa3124d8640 1 --2- 172.21.15.139:0/704129806 >> v2:172.21.15.164:3300/0 conn(0x7fa30c0c9220 0x7fa30c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.791+0000 7fa311cd7640 1 -- 172.21.15.139:0/704129806 >> v2:172.21.15.164:3300/0 conn(0x7fa30c0c9220 msgr2=0x7fa30c0c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:06.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.791+0000 7fa311cd7640 1 --2- 172.21.15.139:0/704129806 >> v2:172.21.15.164:3300/0 conn(0x7fa30c0c9220 0x7fa30c0c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.791+0000 7fa311cd7640 1 -- 172.21.15.139:0/704129806 >> v1:172.21.15.139:6789/0 conn(0x7fa30c0c72a0 legacy=0x7fa30c0c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.791+0000 7fa311cd7640 1 -- 172.21.15.139:0/704129806 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c071b10 con 0x7fa30c0c8070 2024-09-16T00:08:06.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.791+0000 7fa311cd7640 1 --2- 172.21.15.139:0/704129806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa30c0c8070 0x7fa30c15c700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa30c071050 tx=0x7fa30802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25340b846323501 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.791+0000 7fa310cd5640 1 -- 172.21.15.139:0/704129806 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308037070 con 0x7fa30c0c8070 2024-09-16T00:08:06.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.792+0000 7fa310cd5640 1 -- 172.21.15.139:0/704129806 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa308032440 con 0x7fa30c0c8070 2024-09-16T00:08:06.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.792+0000 7fa310cd5640 1 -- 172.21.15.139:0/704129806 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308032810 con 0x7fa30c0c8070 2024-09-16T00:08:06.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.792+0000 7fa313f3b640 1 -- 172.21.15.139:0/704129806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa30c0c8070 msgr2=0x7fa30c15c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.792+0000 7fa313f3b640 1 --2- 172.21.15.139:0/704129806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa30c0c8070 0x7fa30c15c700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa30c071050 tx=0x7fa30802a230 comp rx=0 tx=0).stop 2024-09-16T00:08:06.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.792+0000 7fa313f3b640 1 -- 172.21.15.139:0/704129806 shutdown_connections 2024-09-16T00:08:06.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.792+0000 7fa313f3b640 1 --2- 172.21.15.139:0/704129806 >> v2:172.21.15.164:3300/0 conn(0x7fa30c0c9220 0x7fa30c0c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.792+0000 7fa313f3b640 1 --2- 172.21.15.139:0/704129806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa30c0c8070 0x7fa30c15c700 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.792+0000 7fa313f3b640 1 -- 172.21.15.139:0/704129806 >> 172.21.15.139:0/704129806 conn(0x7fa30c06fb00 msgr2=0x7fa30c0738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:06.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.792+0000 7fa313f3b640 1 -- 172.21.15.139:0/704129806 shutdown_connections 2024-09-16T00:08:06.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.792+0000 7fa313f3b640 1 -- 172.21.15.139:0/704129806 wait complete. 2024-09-16T00:08:06.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.793+0000 7fa313f3b640 1 Processor -- start 2024-09-16T00:08:06.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.793+0000 7fa313f3b640 1 -- start start 2024-09-16T00:08:06.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.794+0000 7fa313f3b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa30c0c72a0 0x7fa30c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.794+0000 7fa313f3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa30c0c8070 0x7fa30c0ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.794+0000 7fa313f3b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa30c0c9220 0x7fa30c0c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.794+0000 7fa313f3b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c15c1e0 con 0x7fa30c0c8070 2024-09-16T00:08:06.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.794+0000 7fa313f3b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c15bf00 con 0x7fa30c0c9220 2024-09-16T00:08:06.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.794+0000 7fa313f3b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa30c0c8450 con 0x7fa30c0c72a0 2024-09-16T00:08:06.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.794+0000 7fa3114d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa30c0c8070 0x7fa30c0ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.794+0000 7fa3114d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa30c0c8070 0x7fa30c0ce150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59082/0 (socket says 172.21.15.139:59082) 2024-09-16T00:08:06.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.794+0000 7fa3114d6640 1 -- 172.21.15.139:0/4272245682 learned_addr learned my addr 172.21.15.139:0/4272245682 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:06.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.794+0000 7fa3124d8640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa30c0c9220 0x7fa30c0c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.794+0000 7fa311cd7640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa30c0c72a0 0x7fa30c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.794+0000 7fa3114d6640 1 -- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa30c0c72a0 msgr2=0x7fa30c06d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.794+0000 7fa3114d6640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa30c0c72a0 0x7fa30c06d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.794+0000 7fa3114d6640 1 -- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa30c0c9220 msgr2=0x7fa30c0c22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.794+0000 7fa3114d6640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa30c0c9220 0x7fa30c0c22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.794+0000 7fa3114d6640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c06ec50 con 0x7fa30c0c8070 2024-09-16T00:08:06.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.795+0000 7fa3114d6640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa30c0c8070 0x7fa30c0ce150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00c4d0 tx=0x7fa2fc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.795+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc017070 con 0x7fa30c0c8070 2024-09-16T00:08:06.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.795+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2fc005ce0 con 0x7fa30c0c8070 2024-09-16T00:08:06.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.795+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30c06cdf0 con 0x7fa30c0c8070 2024-09-16T00:08:06.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.795+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc007270 con 0x7fa30c0c8070 2024-09-16T00:08:06.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.795+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa30c14d0d0 con 0x7fa30c0c8070 2024-09-16T00:08:06.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.796+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fa30c0cca60 con 0x7fa30c0c8070 2024-09-16T00:08:06.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.796+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa2fc01a070 con 0x7fa30c0c8070 2024-09-16T00:08:06.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.797+0000 7fa3027fc640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa2f006c2b0 0x7fa2f006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.797+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5738+0+0 (secure 0 0 0) 0x7fa2fc08a750 con 0x7fa30c0c8070 2024-09-16T00:08:06.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.797+0000 7fa311cd7640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa2f006c2b0 0x7fa2f006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.797+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v29) ==== 54+0+16 (secure 0 0 0) 0x7fa30c0cca60 con 0x7fa30c0c8070 2024-09-16T00:08:06.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.797+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fa30c14c080 0x7fa30c0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.797+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa30c06e720 con 0x7fa30c14c080 2024-09-16T00:08:06.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.797+0000 7fa311cd7640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa2f006c2b0 0x7fa2f006e770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa308034000 tx=0x7fa30802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.797+0000 7fa3124d8640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fa30c14c080 0x7fa30c0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.798+0000 7fa3124d8640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fa30c14c080 0x7fa30c0c9670 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.798+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa30c06e720 con 0x7fa30c14c080 2024-09-16T00:08:06.807 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:06.803+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:06.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.812+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa30c0cedc0 con 0x7fa30c14c080 2024-09-16T00:08:06.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.814+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa30c0cedc0 con 0x7fa30c14c080 2024-09-16T00:08:06.815 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:06.815 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:06.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.814+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fa30c1f4890 0x7fa30c147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.814+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fa30c0cad20 con 0x7fa30c1f4890 2024-09-16T00:08:06.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.814+0000 7fa311cd7640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fa30c1f4890 0x7fa30c147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.814+0000 7fa311cd7640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fa30c1f4890 0x7fa30c147850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.815+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa30c0cad20 con 0x7fa30c1f4890 2024-09-16T00:08:06.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.829+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa30c0caa40 con 0x7fa30c1f4890 2024-09-16T00:08:06.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.831+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa30c0caa40 con 0x7fa30c1f4890 2024-09-16T00:08:06.831 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:06.832 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:06.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.831+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fa30c1f4c70 0x7fa30c1482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.831+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fa30c0ca480 con 0x7fa30c1f4c70 2024-09-16T00:08:06.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.831+0000 7fa3114d6640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fa30c1f4c70 0x7fa30c1482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.831+0000 7fa3114d6640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fa30c1f4c70 0x7fa30c1482a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.832+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa30c0ca480 con 0x7fa30c1f4c70 2024-09-16T00:08:06.840 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:06.840 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:06.841 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:06.841 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:06.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.845+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa30c0ca1a0 con 0x7fa30c1f4c70 2024-09-16T00:08:06.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.847+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa30c0ca1a0 con 0x7fa30c1f4c70 2024-09-16T00:08:06.848 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:06.849 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:06.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.847+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fa30c062f00 0x7fa30c15a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.847+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fa30c063340 con 0x7fa30c062f00 2024-09-16T00:08:06.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.847+0000 7fa3124d8640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fa30c062f00 0x7fa30c15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.847+0000 7fa3124d8640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fa30c062f00 0x7fa30c15a670 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.848+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa30c063340 con 0x7fa30c062f00 2024-09-16T00:08:06.850 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:08:06.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.861+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa30c15abb0 con 0x7fa30c062f00 2024-09-16T00:08:06.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.862+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa30c15abb0 con 0x7fa30c062f00 2024-09-16T00:08:06.863 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:06.864 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:06.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.863+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fa30c15b2d0 0x7fa30c02c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.863+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fa30c15abb0 con 0x7fa30c15b2d0 2024-09-16T00:08:06.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.863+0000 7fa311cd7640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fa30c15b2d0 0x7fa30c02c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.863+0000 7fa311cd7640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fa30c15b2d0 0x7fa30c02c640 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.864+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa30c15abb0 con 0x7fa30c15b2d0 2024-09-16T00:08:06.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.877+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa30c15abb0 con 0x7fa30c15b2d0 2024-09-16T00:08:06.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.879+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa30c15abb0 con 0x7fa30c15b2d0 2024-09-16T00:08:06.880 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:06.880 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:06.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.879+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fa30c02d090 0x7fa30c02d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.879+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fa30c02da50 con 0x7fa30c02d090 2024-09-16T00:08:06.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.879+0000 7fa3114d6640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fa30c02d090 0x7fa30c02d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.880+0000 7fa3114d6640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fa30c02d090 0x7fa30c02d510 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.880+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa30c02da50 con 0x7fa30c02d090 2024-09-16T00:08:06.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.893+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa30c02da50 con 0x7fa30c02d090 2024-09-16T00:08:06.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.895+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa30c02da50 con 0x7fa30c02d090 2024-09-16T00:08:06.896 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:06.896 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:06.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.895+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fa30c207860 0x7fa30c209c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.895+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fa30c20ca20 con 0x7fa30c207860 2024-09-16T00:08:06.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.895+0000 7fa3124d8640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fa30c207860 0x7fa30c209c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.896+0000 7fa3124d8640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fa30c207860 0x7fa30c209c50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.896+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa30c20ca20 con 0x7fa30c207860 2024-09-16T00:08:06.903 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:06.903+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:06.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.910+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa30c20a210 con 0x7fa30c207860 2024-09-16T00:08:06.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.912+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa30c20a210 con 0x7fa30c207860 2024-09-16T00:08:06.913 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.912+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fa30c20a930 0x7fa30c20ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.912+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fa30c20fdf0 con 0x7fa30c20a930 2024-09-16T00:08:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.913+0000 7fa311cd7640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fa30c20a930 0x7fa30c20ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.913+0000 7fa311cd7640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fa30c20a930 0x7fa30c20ad50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:06.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.914+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa30c20fdf0 con 0x7fa30c20a930 2024-09-16T00:08:06.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.927+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa30c20b3a0 con 0x7fa30c20a930 2024-09-16T00:08:06.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa3027fc640 1 -- 172.21.15.139:0/4272245682 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa30c20b3a0 con 0x7fa30c20a930 2024-09-16T00:08:06.929 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:06.931 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:06.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fa30c14c080 msgr2=0x7fa30c0c9670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fa30c14c080 0x7fa30c0c9670 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fa30c1f4890 msgr2=0x7fa30c147850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fa30c1f4890 0x7fa30c147850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fa30c1f4c70 msgr2=0x7fa30c1482a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fa30c1f4c70 0x7fa30c1482a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fa30c062f00 msgr2=0x7fa30c15a670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fa30c062f00 0x7fa30c15a670 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fa30c15b2d0 msgr2=0x7fa30c02c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fa30c15b2d0 0x7fa30c02c640 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa3124d8640 1 -- 172.21.15.139:0/4272245682 reap_dead start 2024-09-16T00:08:06.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fa30c02d090 msgr2=0x7fa30c02d510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fa30c02d090 0x7fa30c02d510 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa3124d8640 1 -- 172.21.15.139:0/4272245682 reap_dead start 2024-09-16T00:08:06.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fa30c207860 msgr2=0x7fa30c209c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fa30c207860 0x7fa30c209c50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fa30c20a930 msgr2=0x7fa30c20ad50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fa30c20a930 0x7fa30c20ad50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa3124d8640 1 -- 172.21.15.139:0/4272245682 reap_dead start 2024-09-16T00:08:06.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa2f006c2b0 msgr2=0x7fa2f006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa2f006c2b0 0x7fa2f006e770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa308034000 tx=0x7fa30802e000 comp rx=0 tx=0).stop 2024-09-16T00:08:06.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa30c0c8070 msgr2=0x7fa30c0ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:06.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa30c0c8070 0x7fa30c0ce150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00c4d0 tx=0x7fa2fc00c990 comp rx=0 tx=0).stop 2024-09-16T00:08:06.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 shutdown_connections 2024-09-16T00:08:06.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa2f006c2b0 0x7fa2f006e770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 --2- 172.21.15.139:0/4272245682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa30c0c8070 0x7fa30c0ce150 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:06.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 >> 172.21.15.139:0/4272245682 conn(0x7fa30c06fb00 msgr2=0x7fa30c0cc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:06.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 shutdown_connections 2024-09-16T00:08:06.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:06.929+0000 7fa313f3b640 1 -- 172.21.15.139:0/4272245682 wait complete. 2024-09-16T00:08:06.995 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:06.995 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:07.006 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:08:07.006 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:07.005+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:07.106 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:07.105+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:07.150 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:07.151 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:07.159 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:08:07.208 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:07.206+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:07.264 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:07.264 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:07.273 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:08:07.317 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:07.317+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:07.377 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:07.377 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:07.386 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:08:07.409 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:07.408+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:07.510 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:07.509+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:07.531 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:07.531 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:07.540 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:08:07.624 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:07.624+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:07.685 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:07.686 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:07.694 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:08:07.736 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:07.735+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:07.759 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T00:08:07.812 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:07.811+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:07.839 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:07.839 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:07.848 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:08:07.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.866+0000 7fa9ddde8640 1 Processor -- start 2024-09-16T00:08:07.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.866+0000 7fa9ddde8640 1 -- start start 2024-09-16T00:08:07.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.867+0000 7fa9ddde8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9d80c93d0 0x7fa9d8166100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:07.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.867+0000 7fa9ddde8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9d80ca000 con 0x7fa9d806ea10 2024-09-16T00:08:07.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.867+0000 7fa9ddde8640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9d80c9900 con 0x7fa9d8154d40 2024-09-16T00:08:07.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.867+0000 7fa9ddde8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d806d340 con 0x7fa9d80c93d0 2024-09-16T00:08:07.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.867+0000 7fa9d6ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa9d806ea10 0x7fa9d8071520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56302/0 (socket says 172.21.15.139:56302) 2024-09-16T00:08:07.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.867+0000 7fa9d6ffd640 1 -- 172.21.15.139:0/2895329798 learned_addr learned my addr 172.21.15.139:0/2895329798 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:07.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.867+0000 7fa9d7fff640 1 --2- 172.21.15.139:0/2895329798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9d80c93d0 0x7fa9d8166100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:07.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.867+0000 7fa9d67fc640 1 -- 172.21.15.139:0/2895329798 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 565780642 0 0) 0x7fa9d80ca000 con 0x7fa9d806ea10 2024-09-16T00:08:07.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.868+0000 7fa9d67fc640 1 -- 172.21.15.139:0/2895329798 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9c0003520 con 0x7fa9d806ea10 2024-09-16T00:08:07.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.868+0000 7fa9d67fc640 1 -- 172.21.15.139:0/2895329798 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3822858030 0 0) 0x7fa9d80c9900 con 0x7fa9d8154d40 2024-09-16T00:08:07.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.868+0000 7fa9d67fc640 1 -- 172.21.15.139:0/2895329798 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9d80ca000 con 0x7fa9d8154d40 2024-09-16T00:08:07.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.868+0000 7fa9d7fff640 1 -- 172.21.15.139:0/2895329798 >> v1:172.21.15.164:6789/0 conn(0x7fa9d8154d40 legacy=0x7fa9d806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:07.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.868+0000 7fa9d7fff640 1 -- 172.21.15.139:0/2895329798 >> v1:172.21.15.139:6789/0 conn(0x7fa9d806ea10 legacy=0x7fa9d8071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:07.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.868+0000 7fa9d7fff640 1 -- 172.21.15.139:0/2895329798 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d80c8d60 con 0x7fa9d80c93d0 2024-09-16T00:08:07.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.868+0000 7fa9d7fff640 1 --2- 172.21.15.139:0/2895329798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9d80c93d0 0x7fa9d8166100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009dc0 tx=0x7fa9d002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b73500d714c30c43 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:07.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.869+0000 7fa9d67fc640 1 -- 172.21.15.139:0/2895329798 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0035020 con 0x7fa9d80c93d0 2024-09-16T00:08:07.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.869+0000 7fa9d67fc640 1 -- 172.21.15.139:0/2895329798 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d0007540 con 0x7fa9d80c93d0 2024-09-16T00:08:07.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.869+0000 7fa9d67fc640 1 -- 172.21.15.139:0/2895329798 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc001670 con 0x7fa9d80c93d0 2024-09-16T00:08:07.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.869+0000 7fa9ddde8640 1 -- 172.21.15.139:0/2895329798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9d80c93d0 msgr2=0x7fa9d8166100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:07.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.869+0000 7fa9ddde8640 1 --2- 172.21.15.139:0/2895329798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9d80c93d0 0x7fa9d8166100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009dc0 tx=0x7fa9d002a6d0 comp rx=0 tx=0).stop 2024-09-16T00:08:07.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.869+0000 7fa9ddde8640 1 -- 172.21.15.139:0/2895329798 shutdown_connections 2024-09-16T00:08:07.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.869+0000 7fa9ddde8640 1 --2- 172.21.15.139:0/2895329798 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9d80c93d0 0x7fa9d8166100 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:07.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.869+0000 7fa9ddde8640 1 -- 172.21.15.139:0/2895329798 >> 172.21.15.139:0/2895329798 conn(0x7fa9d806bb20 msgr2=0x7fa9d806f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:07.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.869+0000 7fa9ddde8640 1 -- 172.21.15.139:0/2895329798 shutdown_connections 2024-09-16T00:08:07.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.869+0000 7fa9ddde8640 1 -- 172.21.15.139:0/2895329798 wait complete. 2024-09-16T00:08:07.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.870+0000 7fa9ddde8640 1 Processor -- start 2024-09-16T00:08:07.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.871+0000 7fa9ddde8640 1 -- start start 2024-09-16T00:08:07.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.871+0000 7fa9ddde8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9d80c93d0 0x7fa9d81e4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:07.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.871+0000 7fa9ddde8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9d8154d40 0x7fa9d81e5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:07.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.871+0000 7fa9ddde8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d8140400 0x7fa9d8145690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:07.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.871+0000 7fa9ddde8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d80ca390 con 0x7fa9d8140400 2024-09-16T00:08:07.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.871+0000 7fa9ddde8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d80c9db0 con 0x7fa9d8154d40 2024-09-16T00:08:07.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.871+0000 7fa9ddde8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d80c52d0 con 0x7fa9d80c93d0 2024-09-16T00:08:07.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.871+0000 7fa9d7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d8140400 0x7fa9d8145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:07.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.871+0000 7fa9d7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d8140400 0x7fa9d8145690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35840/0 (socket says 172.21.15.139:35840) 2024-09-16T00:08:07.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.871+0000 7fa9d7fff640 1 -- 172.21.15.139:0/3938115415 learned_addr learned my addr 172.21.15.139:0/3938115415 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:07.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.871+0000 7fa9d77fe640 1 --2- 172.21.15.139:0/3938115415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9d80c93d0 0x7fa9d81e4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:07.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.872+0000 7fa9d6ffd640 1 --2- 172.21.15.139:0/3938115415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9d8154d40 0x7fa9d81e5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:07.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.872+0000 7fa9d7fff640 1 -- 172.21.15.139:0/3938115415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9d80c93d0 msgr2=0x7fa9d81e4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:07.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.872+0000 7fa9d7fff640 1 --2- 172.21.15.139:0/3938115415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9d80c93d0 0x7fa9d81e4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:07.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.872+0000 7fa9d7fff640 1 -- 172.21.15.139:0/3938115415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9d8154d40 msgr2=0x7fa9d81e5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:07.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.872+0000 7fa9d7fff640 1 --2- 172.21.15.139:0/3938115415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9d8154d40 0x7fa9d81e5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:07.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.872+0000 7fa9d7fff640 1 -- 172.21.15.139:0/3938115415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d8140ac0 con 0x7fa9d8140400 2024-09-16T00:08:07.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.872+0000 7fa9d7fff640 1 --2- 172.21.15.139:0/3938115415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d8140400 0x7fa9d8145690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009ee0 tx=0x7fa9d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:07.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.872+0000 7fa9bffff640 1 -- 172.21.15.139:0/3938115415 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d003f030 con 0x7fa9d8140400 2024-09-16T00:08:07.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.872+0000 7fa9bffff640 1 -- 172.21.15.139:0/3938115415 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d0007e70 con 0x7fa9d8140400 2024-09-16T00:08:07.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.872+0000 7fa9bffff640 1 -- 172.21.15.139:0/3938115415 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d0035070 con 0x7fa9d8140400 2024-09-16T00:08:07.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.872+0000 7fa9ddde8640 1 -- 172.21.15.139:0/3938115415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d8140d30 con 0x7fa9d8140400 2024-09-16T00:08:07.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.872+0000 7fa9ddde8640 1 -- 172.21.15.139:0/3938115415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d81412f0 con 0x7fa9d8140400 2024-09-16T00:08:07.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.873+0000 7fa9ddde8640 1 -- 172.21.15.139:0/3938115415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d80c4ea0 con 0x7fa9d8140400 2024-09-16T00:08:07.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.874+0000 7fa9bffff640 1 -- 172.21.15.139:0/3938115415 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa9d0042030 con 0x7fa9d8140400 2024-09-16T00:08:07.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.874+0000 7fa9bffff640 1 --2- 172.21.15.139:0/3938115415 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa9c406c2e0 0x7fa9c406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:07.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.874+0000 7fa9bffff640 1 -- 172.21.15.139:0/3938115415 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5738+0+0 (secure 0 0 0) 0x7fa9d00ac070 con 0x7fa9d8140400 2024-09-16T00:08:07.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.874+0000 7fa9d77fe640 1 --2- 172.21.15.139:0/3938115415 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa9c406c2e0 0x7fa9c406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:07.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.875+0000 7fa9d77fe640 1 --2- 172.21.15.139:0/3938115415 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa9c406c2e0 0x7fa9c406e7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8007fd0 tx=0x7fa9c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:07.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:07.878+0000 7fa9bffff640 1 -- 172.21.15.139:0/3938115415 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa9d007a170 con 0x7fa9d8140400 2024-09-16T00:08:07.913 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:07.913+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:07.952 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:07.952 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:07.961 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:08:08.014 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:08.014+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:08.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:08.057+0000 7fa9ddde8640 1 -- 172.21.15.139:0/3938115415 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fa9d81475e0 con 0x7fa9d8140400 2024-09-16T00:08:08.069 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:08.069 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:08.078 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:08:08.119 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:08.118+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:08.216 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:08.216+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:08.220 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:08.221 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:08.229 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:08:08.317 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:08.317+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:08.334 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:08.334 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:08.344 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:08:08.418 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:08.417+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:08.447 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:08.447 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:08.456 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:08:08.539 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:08.539+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:08.598 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:08.599 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:08.608 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:08:08.619 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:08.619+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:08.721 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:08.720+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:08.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:08.754+0000 7fa9bffff640 1 -- 172.21.15.139:0/3938115415 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v30) ==== 97+0+0 (secure 0 0 0) 0x7fa9d007e470 con 0x7fa9d8140400 2024-09-16T00:08:08.756 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is set 2024-09-16T00:08:08.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:08.757+0000 7fa9ddde8640 1 -- 172.21.15.139:0/3938115415 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa9c406c2e0 msgr2=0x7fa9c406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:08.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:08.757+0000 7fa9ddde8640 1 --2- 172.21.15.139:0/3938115415 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa9c406c2e0 0x7fa9c406e7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8007fd0 tx=0x7fa9c8009040 comp rx=0 tx=0).stop 2024-09-16T00:08:08.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:08.757+0000 7fa9ddde8640 1 -- 172.21.15.139:0/3938115415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d8140400 msgr2=0x7fa9d8145690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:08.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:08.757+0000 7fa9ddde8640 1 --2- 172.21.15.139:0/3938115415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d8140400 0x7fa9d8145690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009ee0 tx=0x7fa9d0004a40 comp rx=0 tx=0).stop 2024-09-16T00:08:08.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:08.758+0000 7fa9ddde8640 1 -- 172.21.15.139:0/3938115415 shutdown_connections 2024-09-16T00:08:08.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:08.758+0000 7fa9ddde8640 1 --2- 172.21.15.139:0/3938115415 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa9c406c2e0 0x7fa9c406e7a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:08.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:08.758+0000 7fa9ddde8640 1 --2- 172.21.15.139:0/3938115415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa9d8140400 0x7fa9d8145690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:08.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:08.758+0000 7fa9ddde8640 1 --2- 172.21.15.139:0/3938115415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa9d8154d40 0x7fa9d81e5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:08.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:08.758+0000 7fa9ddde8640 1 --2- 172.21.15.139:0/3938115415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa9d80c93d0 0x7fa9d81e4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:08.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:08.758+0000 7fa9ddde8640 1 -- 172.21.15.139:0/3938115415 >> 172.21.15.139:0/3938115415 conn(0x7fa9d806bb20 msgr2=0x7fa9d806d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:08.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:08.758+0000 7fa9ddde8640 1 -- 172.21.15.139:0/3938115415 shutdown_connections 2024-09-16T00:08:08.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:08.758+0000 7fa9ddde8640 1 -- 172.21.15.139:0/3938115415 wait complete. 2024-09-16T00:08:08.762 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:08.762 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:08.770 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:08:08.854 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:08.854+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:08.918 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:08.918 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:08.922 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:08.922+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:08.927 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:08:08.944 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:08:09.024 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:09.023+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:09.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.065+0000 7efed20af640 1 Processor -- start 2024-09-16T00:08:09.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.065+0000 7efed20af640 1 -- start start 2024-09-16T00:08:09.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.066+0000 7efed20af640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efecc0d0870 0x7efecc0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:09.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.066+0000 7efed20af640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efecc15adc0 con 0x7efecc0b3430 2024-09-16T00:08:09.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.066+0000 7efed20af640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efecc15b0a0 con 0x7efecc0d0490 2024-09-16T00:08:09.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.066+0000 7efed20af640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7efecc15b940 con 0x7efecc0d0870 2024-09-16T00:08:09.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.066+0000 7efecbfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7efecc0b3430 0x7efecc1546a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56306/0 (socket says 172.21.15.139:56306) 2024-09-16T00:08:09.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.066+0000 7efecbfff640 1 -- 172.21.15.139:0/3570697179 learned_addr learned my addr 172.21.15.139:0/3570697179 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:09.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.066+0000 7efecaffd640 1 --2- 172.21.15.139:0/3570697179 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efecc0d0870 0x7efecc0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:09.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.067+0000 7efeca7fc640 1 -- 172.21.15.139:0/3570697179 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1442807618 0 0) 0x7efecc15adc0 con 0x7efecc0b3430 2024-09-16T00:08:09.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.067+0000 7efeca7fc640 1 -- 172.21.15.139:0/3570697179 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efeb4003520 con 0x7efecc0b3430 2024-09-16T00:08:09.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.067+0000 7efeca7fc640 1 -- 172.21.15.139:0/3570697179 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4143759169 0 0) 0x7efecc15b0a0 con 0x7efecc0d0490 2024-09-16T00:08:09.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.067+0000 7efeca7fc640 1 -- 172.21.15.139:0/3570697179 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efecc15adc0 con 0x7efecc0d0490 2024-09-16T00:08:09.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.067+0000 7efecaffd640 1 -- 172.21.15.139:0/3570697179 >> v1:172.21.15.164:6789/0 conn(0x7efecc0d0490 legacy=0x7efecc0b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:09.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.067+0000 7efecaffd640 1 -- 172.21.15.139:0/3570697179 >> v1:172.21.15.139:6789/0 conn(0x7efecc0b3430 legacy=0x7efecc1546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:09.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.067+0000 7efecaffd640 1 -- 172.21.15.139:0/3570697179 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efecc15b380 con 0x7efecc0d0870 2024-09-16T00:08:09.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.068+0000 7efecaffd640 1 --2- 172.21.15.139:0/3570697179 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efecc0d0870 0x7efecc0b4040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efec0009800 tx=0x7efec002a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee71b02d1a07baba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:09.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.068+0000 7efeca7fc640 1 -- 172.21.15.139:0/3570697179 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec0037070 con 0x7efecc0d0870 2024-09-16T00:08:09.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.068+0000 7efeca7fc640 1 -- 172.21.15.139:0/3570697179 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec0032440 con 0x7efecc0d0870 2024-09-16T00:08:09.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.068+0000 7efeca7fc640 1 -- 172.21.15.139:0/3570697179 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec00327f0 con 0x7efecc0d0870 2024-09-16T00:08:09.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.069+0000 7efed20af640 1 -- 172.21.15.139:0/3570697179 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efecc0d0870 msgr2=0x7efecc0b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:09.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.069+0000 7efed20af640 1 --2- 172.21.15.139:0/3570697179 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efecc0d0870 0x7efecc0b4040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efec0009800 tx=0x7efec002a2c0 comp rx=0 tx=0).stop 2024-09-16T00:08:09.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.069+0000 7efed20af640 1 -- 172.21.15.139:0/3570697179 shutdown_connections 2024-09-16T00:08:09.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.069+0000 7efed20af640 1 --2- 172.21.15.139:0/3570697179 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efecc0d0870 0x7efecc0b4040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:09.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.069+0000 7efed20af640 1 -- 172.21.15.139:0/3570697179 >> 172.21.15.139:0/3570697179 conn(0x7efecc06c520 msgr2=0x7efecc06c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:09.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.069+0000 7efed20af640 1 -- 172.21.15.139:0/3570697179 shutdown_connections 2024-09-16T00:08:09.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.069+0000 7efed20af640 1 -- 172.21.15.139:0/3570697179 wait complete. 2024-09-16T00:08:09.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.070+0000 7efed20af640 1 Processor -- start 2024-09-16T00:08:09.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.070+0000 7efed20af640 1 -- start start 2024-09-16T00:08:09.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.071+0000 7efed20af640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efecc0b3430 0x7efecc1f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:09.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.071+0000 7efed20af640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efecc0d0490 0x7efecc150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:09.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.071+0000 7efed20af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efecc0d0870 0x7efecc1f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:09.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.071+0000 7efed20af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efecc15a980 con 0x7efecc0d0870 2024-09-16T00:08:09.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.071+0000 7efed20af640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efecc0b3bb0 con 0x7efecc0d0490 2024-09-16T00:08:09.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.071+0000 7efed20af640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7efecc15ab00 con 0x7efecc0b3430 2024-09-16T00:08:09.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.071+0000 7efecbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efecc0d0870 0x7efecc1f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:09.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.071+0000 7efecbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efecc0d0870 0x7efecc1f04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35854/0 (socket says 172.21.15.139:35854) 2024-09-16T00:08:09.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.071+0000 7efecbfff640 1 -- 172.21.15.139:0/2287309322 learned_addr learned my addr 172.21.15.139:0/2287309322 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:09.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.071+0000 7efecaffd640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efecc0d0490 0x7efecc150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:09.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.071+0000 7efecb7fe640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efecc0b3430 0x7efecc1f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:09.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.071+0000 7efecbfff640 1 -- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efecc0b3430 msgr2=0x7efecc1f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:09.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.071+0000 7efecbfff640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efecc0b3430 0x7efecc1f3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:09.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.071+0000 7efecbfff640 1 -- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efecc0d0490 msgr2=0x7efecc150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:09.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.071+0000 7efecbfff640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efecc0d0490 0x7efecc150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:09.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.071+0000 7efecbfff640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efecc151a90 con 0x7efecc0d0870 2024-09-16T00:08:09.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.072+0000 7efecbfff640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efecc0d0870 0x7efecc1f04b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efec400caf0 tx=0x7efec400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:09.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.072+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4017070 con 0x7efecc0d0870 2024-09-16T00:08:09.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.072+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec400aca0 con 0x7efecc0d0870 2024-09-16T00:08:09.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.072+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec4012620 con 0x7efecc0d0870 2024-09-16T00:08:09.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.072+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efecc0cba20 con 0x7efecc0d0870 2024-09-16T00:08:09.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.072+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efecc0cbfe0 con 0x7efecc0d0870 2024-09-16T00:08:09.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.073+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7efecc0b2c40 con 0x7efecc0d0870 2024-09-16T00:08:09.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.073+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efec401a070 con 0x7efecc0d0870 2024-09-16T00:08:09.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.074+0000 7efeb3fff640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efeb806c2e0 0x7efeb806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:09.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.074+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5738+0+0 (secure 0 0 0) 0x7efec408b210 con 0x7efecc0d0870 2024-09-16T00:08:09.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.074+0000 7efecb7fe640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efeb806c2e0 0x7efeb806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:09.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.074+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v30) ==== 54+0+16 (secure 0 0 0) 0x7efec408d890 con 0x7efecc0d0870 2024-09-16T00:08:09.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.074+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7efecc158560 0x7efecc148360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:09.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.074+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efecc0cb7d0 con 0x7efecc158560 2024-09-16T00:08:09.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.074+0000 7efecaffd640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7efecc158560 0x7efecc148360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:09.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.074+0000 7efecb7fe640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efeb806c2e0 0x7efeb806e7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efebc001650 tx=0x7efebc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:09.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.075+0000 7efecaffd640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7efecc158560 0x7efecc148360 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:09.082 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:09.082 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:09.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.076+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efecc0cb7d0 con 0x7efecc158560 2024-09-16T00:08:09.089 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:08:09.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.095+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7efecc155200 con 0x7efecc158560 2024-09-16T00:08:09.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.097+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efec00041c0 con 0x7efecc158560 2024-09-16T00:08:09.098 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:09.099 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:09.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.097+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7efecc158940 0x7efecc0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:09.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.097+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7efecc15a760 con 0x7efecc158940 2024-09-16T00:08:09.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.097+0000 7efecb7fe640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7efecc158940 0x7efecc0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:09.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.098+0000 7efecb7fe640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7efecc158940 0x7efecc0ccca0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:09.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.098+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efecc15a760 con 0x7efecc158940 2024-09-16T00:08:09.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.114+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7efecc150be0 con 0x7efecc158940 2024-09-16T00:08:09.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.115+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efecc150be0 con 0x7efecc158940 2024-09-16T00:08:09.116 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:09.117 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:09.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.116+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7efecc0cd6f0 0x7efecc062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:09.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.116+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7efecc1f0090 con 0x7efecc0cd6f0 2024-09-16T00:08:09.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.116+0000 7efecaffd640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7efecc0cd6f0 0x7efecc062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:09.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.116+0000 7efecaffd640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7efecc0cd6f0 0x7efecc062630 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:09.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.117+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efecc1f0090 con 0x7efecc0cd6f0 2024-09-16T00:08:09.125 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:09.125+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:09.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.131+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7efecc062e40 con 0x7efecc0cd6f0 2024-09-16T00:08:09.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.132+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efecc062e40 con 0x7efecc0cd6f0 2024-09-16T00:08:09.133 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:09.134 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:09.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.133+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7efecc0ce0b0 0x7efecc0ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:09.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.133+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7efecc0cea10 con 0x7efecc0ce0b0 2024-09-16T00:08:09.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.133+0000 7efecbfff640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7efecc0ce0b0 0x7efecc0ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:09.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.133+0000 7efecbfff640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7efecc0ce0b0 0x7efecc0ce4d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:09.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.134+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efecc0cea10 con 0x7efecc0ce0b0 2024-09-16T00:08:09.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.148+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7efecc0cef20 con 0x7efecc0ce0b0 2024-09-16T00:08:09.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.150+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efecc0cef20 con 0x7efecc0ce0b0 2024-09-16T00:08:09.151 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:09.152 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:09.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.150+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7efecc02cb50 0x7efecc02cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:09.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.150+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7efecc0ceea0 con 0x7efecc02cb50 2024-09-16T00:08:09.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.150+0000 7efecb7fe640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7efecc02cb50 0x7efecc02cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:09.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.151+0000 7efecb7fe640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7efecc02cb50 0x7efecc02cf90 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:09.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.152+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efecc0ceea0 con 0x7efecc02cb50 2024-09-16T00:08:09.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.166+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7efecc02d4d0 con 0x7efecc02cb50 2024-09-16T00:08:09.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.168+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efecc02d4d0 con 0x7efecc02cb50 2024-09-16T00:08:09.169 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:09.170 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:09.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.168+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7efecc2046e0 0x7efecc206ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:09.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.168+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7efecc02daf0 con 0x7efecc2046e0 2024-09-16T00:08:09.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.169+0000 7efecaffd640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7efecc2046e0 0x7efecc206ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:09.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.169+0000 7efecaffd640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7efecc2046e0 0x7efecc206ad0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:09.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.170+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efecc02daf0 con 0x7efecc2046e0 2024-09-16T00:08:09.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.185+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7efecc207010 con 0x7efecc2046e0 2024-09-16T00:08:09.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.187+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efecc207010 con 0x7efecc2046e0 2024-09-16T00:08:09.188 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:09.188 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:09.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.187+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7efecc207730 0x7efecc209b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:09.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.187+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7efecc20ca00 con 0x7efecc207730 2024-09-16T00:08:09.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.188+0000 7efecbfff640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7efecc207730 0x7efecc209b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:09.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.188+0000 7efecbfff640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7efecc207730 0x7efecc209b20 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:09.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.189+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efecc20ca00 con 0x7efecc207730 2024-09-16T00:08:09.199 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:09.199 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:09.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:09.199 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:09.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.200 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:09.200 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:09.200 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.5815065, 2024-09-16T00:08:09.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038263220000000001, 2024-09-16T00:08:09.200 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.200 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.200 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.200 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:09.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:09.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615465+0000", 2024-09-16T00:08:09.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2769999999999999e-06 2024-09-16T00:08:09.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615471+0000", 2024-09-16T00:08:09.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5729999999999996e-06 2024-09-16T00:08:09.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615483+0000", 2024-09-16T00:08:09.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1838000000000001e-05 2024-09-16T00:08:09.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615492+0000", 2024-09-16T00:08:09.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2410000000000001e-06 2024-09-16T00:08:09.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615596+0000", 2024-09-16T00:08:09.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104097 2024-09-16T00:08:09.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615688+0000", 2024-09-16T00:08:09.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1225999999999997e-05 2024-09-16T00:08:09.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:09.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615804+0000", 2024-09-16T00:08:09.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000116601 2024-09-16T00:08:09.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:09.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618165+0000", 2024-09-16T00:08:09.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023609429999999999 2024-09-16T00:08:09.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:09.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:08:09.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010258509999999999 2024-09-16T00:08:09.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619243+0000", 2024-09-16T00:08:09.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2327999999999998e-05 2024-09-16T00:08:09.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.205 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619287+0000", 2024-09-16T00:08:09.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3346999999999997e-05 2024-09-16T00:08:09.205 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.205 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.205 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.205 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:09.205 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:09.206 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.581451424000001, 2024-09-16T00:08:09.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0054735510000000001, 2024-09-16T00:08:09.206 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.206 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.206 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.206 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:09.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:09.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615516+0000", 2024-09-16T00:08:09.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6899999999999998e-07 2024-09-16T00:08:09.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615519+0000", 2024-09-16T00:08:09.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1760000000000002e-06 2024-09-16T00:08:09.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615526+0000", 2024-09-16T00:08:09.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8399999999999997e-06 2024-09-16T00:08:09.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615531+0000", 2024-09-16T00:08:09.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4450000000000004e-06 2024-09-16T00:08:09.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:09.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054127099999999996 2024-09-16T00:08:09.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616140+0000", 2024-09-16T00:08:09.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7230999999999994e-05 2024-09-16T00:08:09.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:09.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616249+0000", 2024-09-16T00:08:09.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109122 2024-09-16T00:08:09.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:09.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618646+0000", 2024-09-16T00:08:09.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023972279999999999 2024-09-16T00:08:09.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:09.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:08:09.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022396220000000001 2024-09-16T00:08:09.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620939+0000", 2024-09-16T00:08:09.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2735999999999998e-05 2024-09-16T00:08:09.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.212 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.212 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620989+0000", 2024-09-16T00:08:09.212 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0411000000000002e-05 2024-09-16T00:08:09.212 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.212 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.212 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.212 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:09.212 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:09.212 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.581425944999999, 2024-09-16T00:08:09.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055872789999999997, 2024-09-16T00:08:09.213 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.213 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.213 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.213 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:09.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:09.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:09.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.01e-07 2024-09-16T00:08:09.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615547+0000", 2024-09-16T00:08:09.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7899999999999996e-06 2024-09-16T00:08:09.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:08:09.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1119999999999998e-06 2024-09-16T00:08:09.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615560+0000", 2024-09-16T00:08:09.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:08:09.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616424+0000", 2024-09-16T00:08:09.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086350800000000003 2024-09-16T00:08:09.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:08:09.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9675999999999999e-05 2024-09-16T00:08:09.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:09.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616621+0000", 2024-09-16T00:08:09.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7345999999999999e-05 2024-09-16T00:08:09.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:09.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618654+0000", 2024-09-16T00:08:09.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020335269999999998 2024-09-16T00:08:09.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:09.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621055+0000", 2024-09-16T00:08:09.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024009729999999998 2024-09-16T00:08:09.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.218 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621100+0000", 2024-09-16T00:08:09.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5402999999999999e-05 2024-09-16T00:08:09.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.218 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621128+0000", 2024-09-16T00:08:09.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7736e-05 2024-09-16T00:08:09.218 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.218 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.218 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.219 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.219 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:09.219 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:09.219 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.581400295000002, 2024-09-16T00:08:09.219 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057024509999999999, 2024-09-16T00:08:09.219 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.219 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.219 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.219 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.219 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.219 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.219 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:09.219 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:09.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:09.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6300000000000001e-07 2024-09-16T00:08:09.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615569+0000", 2024-09-16T00:08:09.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3709999999999998e-06 2024-09-16T00:08:09.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615580+0000", 2024-09-16T00:08:09.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0448000000000001e-05 2024-09-16T00:08:09.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615584+0000", 2024-09-16T00:08:09.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6020000000000002e-06 2024-09-16T00:08:09.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616850+0000", 2024-09-16T00:08:09.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012661110000000001 2024-09-16T00:08:09.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616915+0000", 2024-09-16T00:08:09.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4820000000000006e-05 2024-09-16T00:08:09.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:09.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617031+0000", 2024-09-16T00:08:09.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115258 2024-09-16T00:08:09.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:09.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618659+0000", 2024-09-16T00:08:09.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001627975 2024-09-16T00:08:09.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:09.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621179+0000", 2024-09-16T00:08:09.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025203410000000002 2024-09-16T00:08:09.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621238+0000", 2024-09-16T00:08:09.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9323000000000001e-05 2024-09-16T00:08:09.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621269+0000", 2024-09-16T00:08:09.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0839000000000001e-05 2024-09-16T00:08:09.229 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.229 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.229 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.229 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:09.230 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:09.230 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.581376546000001, 2024-09-16T00:08:09.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058012769999999996, 2024-09-16T00:08:09.230 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.230 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.230 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.231 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:09.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:09.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615591+0000", 2024-09-16T00:08:09.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6199999999999999e-07 2024-09-16T00:08:09.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615594+0000", 2024-09-16T00:08:09.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8739999999999999e-06 2024-09-16T00:08:09.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615599+0000", 2024-09-16T00:08:09.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:08:09.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615603+0000", 2024-09-16T00:08:09.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8489999999999999e-06 2024-09-16T00:08:09.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617235+0000", 2024-09-16T00:08:09.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001632037 2024-09-16T00:08:09.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.235 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617348+0000", 2024-09-16T00:08:09.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113387 2024-09-16T00:08:09.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.235 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:09.236 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617458+0000", 2024-09-16T00:08:09.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109765 2024-09-16T00:08:09.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:09.236 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618664+0000", 2024-09-16T00:08:09.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012059950000000001 2024-09-16T00:08:09.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:09.236 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621311+0000", 2024-09-16T00:08:09.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026468469999999999 2024-09-16T00:08:09.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621357+0000", 2024-09-16T00:08:09.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6360999999999999e-05 2024-09-16T00:08:09.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621392+0000", 2024-09-16T00:08:09.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4190999999999998e-05 2024-09-16T00:08:09.237 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.237 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.237 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.238 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:09.238 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:09.238 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.581357921999999, 2024-09-16T00:08:09.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058737279999999999, 2024-09-16T00:08:09.238 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.238 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.238 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.239 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:09.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:09.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:09.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3799999999999999e-07 2024-09-16T00:08:09.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615611+0000", 2024-09-16T00:08:09.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-06 2024-09-16T00:08:09.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615626+0000", 2024-09-16T00:08:09.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:08:09.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615634+0000", 2024-09-16T00:08:09.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-06 2024-09-16T00:08:09.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.242 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.242 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617692+0000", 2024-09-16T00:08:09.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002057718 2024-09-16T00:08:09.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.242 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.242 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617768+0000", 2024-09-16T00:08:09.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6878999999999994e-05 2024-09-16T00:08:09.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:09.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:09.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019501699999999999 2024-09-16T00:08:09.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:09.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:08:09.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034357200000000002 2024-09-16T00:08:09.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:09.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621429+0000", 2024-09-16T00:08:09.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0012999999999999e-05 2024-09-16T00:08:09.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621459+0000", 2024-09-16T00:08:09.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9498000000000001e-05 2024-09-16T00:08:09.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621483+0000", 2024-09-16T00:08:09.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3957000000000001e-05 2024-09-16T00:08:09.245 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.245 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.245 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.245 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:09.245 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:09.245 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.581326642000001, 2024-09-16T00:08:09.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059371789999999999, 2024-09-16T00:08:09.245 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.245 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.246 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.246 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:09.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:09.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:08:09.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4000000000000002e-07 2024-09-16T00:08:09.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615643+0000", 2024-09-16T00:08:09.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6860000000000002e-06 2024-09-16T00:08:09.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615649+0000", 2024-09-16T00:08:09.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7610000000000004e-06 2024-09-16T00:08:09.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615653+0000", 2024-09-16T00:08:09.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0380000000000001e-06 2024-09-16T00:08:09.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618197+0000", 2024-09-16T00:08:09.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025440079999999999 2024-09-16T00:08:09.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618312+0000", 2024-09-16T00:08:09.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115083 2024-09-16T00:08:09.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:09.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618412+0000", 2024-09-16T00:08:09.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010016799999999999 2024-09-16T00:08:09.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:09.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621518+0000", 2024-09-16T00:08:09.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031058679999999999 2024-09-16T00:08:09.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:09.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621526+0000", 2024-09-16T00:08:09.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2490000000000004e-06 2024-09-16T00:08:09.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621549+0000", 2024-09-16T00:08:09.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3686999999999999e-05 2024-09-16T00:08:09.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621577+0000", 2024-09-16T00:08:09.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8191000000000002e-05 2024-09-16T00:08:09.251 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.251 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.251 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.252 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:09.252 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:09.252 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.581308975999999, 2024-09-16T00:08:09.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005989622, 2024-09-16T00:08:09.252 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.252 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.252 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.252 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:09.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:09.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:09.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3799999999999998e-07 2024-09-16T00:08:09.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615660+0000", 2024-09-16T00:08:09.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9740000000000001e-06 2024-09-16T00:08:09.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615666+0000", 2024-09-16T00:08:09.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3499999999999996e-06 2024-09-16T00:08:09.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615671+0000", 2024-09-16T00:08:09.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.13e-06 2024-09-16T00:08:09.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618759+0000", 2024-09-16T00:08:09.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030880500000000002 2024-09-16T00:08:09.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618832+0000", 2024-09-16T00:08:09.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3725000000000003e-05 2024-09-16T00:08:09.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:09.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618958+0000", 2024-09-16T00:08:09.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012587899999999999 2024-09-16T00:08:09.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:09.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621583+0000", 2024-09-16T00:08:09.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026246469999999999 2024-09-16T00:08:09.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:09.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621602+0000", 2024-09-16T00:08:09.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9137e-05 2024-09-16T00:08:09.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621629+0000", 2024-09-16T00:08:09.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6451000000000001e-05 2024-09-16T00:08:09.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621648+0000", 2024-09-16T00:08:09.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8941e-05 2024-09-16T00:08:09.260 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.260 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.260 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.260 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:09.260 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:09.260 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.581291357000001, 2024-09-16T00:08:09.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053141380000000004, 2024-09-16T00:08:09.261 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.261 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.261 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.261 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:09.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:09.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:09.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-07 2024-09-16T00:08:09.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615678+0000", 2024-09-16T00:08:09.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:08:09.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615682+0000", 2024-09-16T00:08:09.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6659999999999999e-06 2024-09-16T00:08:09.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615686+0000", 2024-09-16T00:08:09.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6990000000000001e-06 2024-09-16T00:08:09.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615736+0000", 2024-09-16T00:08:09.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9656e-05 2024-09-16T00:08:09.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615864+0000", 2024-09-16T00:08:09.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012836100000000001 2024-09-16T00:08:09.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:09.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615989+0000", 2024-09-16T00:08:09.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000125525 2024-09-16T00:08:09.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:09.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617355+0000", 2024-09-16T00:08:09.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001365982 2024-09-16T00:08:09.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:09.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620897+0000", 2024-09-16T00:08:09.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035419610000000002 2024-09-16T00:08:09.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620941+0000", 2024-09-16T00:08:09.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3998999999999997e-05 2024-09-16T00:08:09.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620990+0000", 2024-09-16T00:08:09.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8199999999999999e-05 2024-09-16T00:08:09.268 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.268 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.268 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.268 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:09.269 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:09.269 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.581264098999998, 2024-09-16T00:08:09.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006316449, 2024-09-16T00:08:09.269 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.269 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.269 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.270 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:09.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:09.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:09.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3299999999999998e-07 2024-09-16T00:08:09.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615705+0000", 2024-09-16T00:08:09.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.278e-06 2024-09-16T00:08:09.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615711+0000", 2024-09-16T00:08:09.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8119999999999996e-06 2024-09-16T00:08:09.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615716+0000", 2024-09-16T00:08:09.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4909999999999999e-06 2024-09-16T00:08:09.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617551+0000", 2024-09-16T00:08:09.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00183549 2024-09-16T00:08:09.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617627+0000", 2024-09-16T00:08:09.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6163999999999998e-05 2024-09-16T00:08:09.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:09.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617776+0000", 2024-09-16T00:08:09.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014909499999999999 2024-09-16T00:08:09.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.276 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:09.276 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621072+0000", 2024-09-16T00:08:09.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032955150000000002 2024-09-16T00:08:09.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.276 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:09.276 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621957+0000", 2024-09-16T00:08:09.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088553199999999996 2024-09-16T00:08:09.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621998+0000", 2024-09-16T00:08:09.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0815000000000001e-05 2024-09-16T00:08:09.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:08:09.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0924000000000002e-05 2024-09-16T00:08:09.278 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.278 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.278 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.278 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:6 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:09.278 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:09.278 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.577862131, 2024-09-16T00:08:09.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000188111, 2024-09-16T00:08:09.278 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.278 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.278 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:09.278 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:09.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:09.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619106+0000", 2024-09-16T00:08:09.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4399999999999998e-07 2024-09-16T00:08:09.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619110+0000", 2024-09-16T00:08:09.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1110000000000001e-06 2024-09-16T00:08:09.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619120+0000", 2024-09-16T00:08:09.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7909999999999999e-06 2024-09-16T00:08:09.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619133+0000", 2024-09-16T00:08:09.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.359e-05 2024-09-16T00:08:09.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619179+0000", 2024-09-16T00:08:09.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5459e-05 2024-09-16T00:08:09.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.282 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.282 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.282 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619186+0000", 2024-09-16T00:08:09.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0480000000000001e-06 2024-09-16T00:08:09.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619293+0000", 2024-09-16T00:08:09.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000107168 2024-09-16T00:08:09.283 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.283 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.283 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.284 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:22 2.e e22/19 ondisk, result = 0)", 2024-09-16T00:08:09.284 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:09.284 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.576171429999999, 2024-09-16T00:08:09.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020058200000000001, 2024-09-16T00:08:09.284 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.284 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.284 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:09.284 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:09.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:09.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:09.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0109999999999999e-06 2024-09-16T00:08:09.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620801+0000", 2024-09-16T00:08:09.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6779999999999999e-06 2024-09-16T00:08:09.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620814+0000", 2024-09-16T00:08:09.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3131e-05 2024-09-16T00:08:09.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620821+0000", 2024-09-16T00:08:09.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0430000000000002e-06 2024-09-16T00:08:09.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:09.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3010999999999997e-05 2024-09-16T00:08:09.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620891+0000", 2024-09-16T00:08:09.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1760000000000004e-06 2024-09-16T00:08:09.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620996+0000", 2024-09-16T00:08:09.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010453199999999999 2024-09-16T00:08:09.288 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.288 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.288 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.288 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:7 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:09.288 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:09.288 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.576163445999999, 2024-09-16T00:08:09.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000195725, 2024-09-16T00:08:09.288 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.288 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.288 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:09.289 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:09.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:09.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620804+0000", 2024-09-16T00:08:09.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6099999999999999e-07 2024-09-16T00:08:09.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620809+0000", 2024-09-16T00:08:09.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4950000000000002e-06 2024-09-16T00:08:09.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:09.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0553e-05 2024-09-16T00:08:09.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620826+0000", 2024-09-16T00:08:09.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5579999999999997e-06 2024-09-16T00:08:09.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620874+0000", 2024-09-16T00:08:09.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7648999999999997e-05 2024-09-16T00:08:09.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620880+0000", 2024-09-16T00:08:09.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8290000000000002e-06 2024-09-16T00:08:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620999+0000", 2024-09-16T00:08:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011878 2024-09-16T00:08:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:8 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:09.292 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.576111386000001, 2024-09-16T00:08:09.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00027991099999999998, 2024-09-16T00:08:09.293 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.293 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.293 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:09.293 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:09.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:09.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620856+0000", 2024-09-16T00:08:09.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6800000000000001e-07 2024-09-16T00:08:09.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620859+0000", 2024-09-16T00:08:09.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6879999999999999e-06 2024-09-16T00:08:09.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620867+0000", 2024-09-16T00:08:09.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2819999999999996e-06 2024-09-16T00:08:09.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620872+0000", 2024-09-16T00:08:09.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5369999999999997e-06 2024-09-16T00:08:09.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621043+0000", 2024-09-16T00:08:09.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170791 2024-09-16T00:08:09.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621050+0000", 2024-09-16T00:08:09.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8580000000000002e-06 2024-09-16T00:08:09.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621135+0000", 2024-09-16T00:08:09.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5387000000000001e-05 2024-09-16T00:08:09.296 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.296 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.296 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.297 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:9 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:09.297 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:09.297 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.576082807999999, 2024-09-16T00:08:09.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00039143799999999997, 2024-09-16T00:08:09.297 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.297 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.297 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:09.297 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:09.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:09.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:09.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6800000000000002e-07 2024-09-16T00:08:09.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:08:09.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0310000000000001e-06 2024-09-16T00:08:09.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620892+0000", 2024-09-16T00:08:09.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9390000000000002e-06 2024-09-16T00:08:09.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620898+0000", 2024-09-16T00:08:09.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4480000000000002e-06 2024-09-16T00:08:09.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.300 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.300 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621167+0000", 2024-09-16T00:08:09.300 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026960599999999998 2024-09-16T00:08:09.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.300 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.300 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621174+0000", 2024-09-16T00:08:09.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.567e-06 2024-09-16T00:08:09.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621275+0000", 2024-09-16T00:08:09.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101579 2024-09-16T00:08:09.301 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.301 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.301 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.301 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:10 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:09.301 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:09.302 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.57606036, 2024-09-16T00:08:09.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00049416200000000003, 2024-09-16T00:08:09.302 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.302 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.302 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:09.302 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:09.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.303 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:09.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.303 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620907+0000", 2024-09-16T00:08:09.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.42e-07 2024-09-16T00:08:09.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620908+0000", 2024-09-16T00:08:09.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4300000000000001e-06 2024-09-16T00:08:09.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.304 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.304 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620911+0000", 2024-09-16T00:08:09.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3890000000000001e-06 2024-09-16T00:08:09.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.304 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.304 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620914+0000", 2024-09-16T00:08:09.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3690000000000001e-06 2024-09-16T00:08:09.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621297+0000", 2024-09-16T00:08:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038319200000000003 2024-09-16T00:08:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621305+0000", 2024-09-16T00:08:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4780000000000008e-06 2024-09-16T00:08:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621401+0000", 2024-09-16T00:08:09.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5161999999999995e-05 2024-09-16T00:08:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:11 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.575657611, 2024-09-16T00:08:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017943899999999999, 2024-09-16T00:08:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:09.306 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621310+0000", 2024-09-16T00:08:09.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5199999999999996e-07 2024-09-16T00:08:09.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:09.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1740000000000002e-06 2024-09-16T00:08:09.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621324+0000", 2024-09-16T00:08:09.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7170000000000003e-06 2024-09-16T00:08:09.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621331+0000", 2024-09-16T00:08:09.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2280000000000001e-06 2024-09-16T00:08:09.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621422+0000", 2024-09-16T00:08:09.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1052999999999997e-05 2024-09-16T00:08:09.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621426+0000", 2024-09-16T00:08:09.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.036e-06 2024-09-16T00:08:09.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621489+0000", 2024-09-16T00:08:09.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2478999999999996e-05 2024-09-16T00:08:09.310 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.310 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.311 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.311 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:12 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:09.311 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:09.311 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.575606755999999, 2024-09-16T00:08:09.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022422300000000001, 2024-09-16T00:08:09.311 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.311 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.311 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:09.312 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:09.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:09.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:09.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6699999999999999e-07 2024-09-16T00:08:09.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621363+0000", 2024-09-16T00:08:09.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7020000000000001e-06 2024-09-16T00:08:09.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621371+0000", 2024-09-16T00:08:09.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7209999999999994e-06 2024-09-16T00:08:09.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621376+0000", 2024-09-16T00:08:09.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7940000000000002e-06 2024-09-16T00:08:09.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621511+0000", 2024-09-16T00:08:09.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013518100000000001 2024-09-16T00:08:09.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621514+0000", 2024-09-16T00:08:09.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5599999999999998e-06 2024-09-16T00:08:09.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621584+0000", 2024-09-16T00:08:09.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9898000000000001e-05 2024-09-16T00:08:09.315 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.315 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.315 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.315 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:13 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:09.315 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:09.315 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.575580492, 2024-09-16T00:08:09.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026534099999999999, 2024-09-16T00:08:09.316 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.316 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.316 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:09.316 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:09.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:09.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621387+0000", 2024-09-16T00:08:09.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4299999999999999e-07 2024-09-16T00:08:09.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621389+0000", 2024-09-16T00:08:09.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.593e-06 2024-09-16T00:08:09.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621395+0000", 2024-09-16T00:08:09.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.553e-06 2024-09-16T00:08:09.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:08:09.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8410000000000002e-06 2024-09-16T00:08:09.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621597+0000", 2024-09-16T00:08:09.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019794100000000001 2024-09-16T00:08:09.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621600+0000", 2024-09-16T00:08:09.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.185e-06 2024-09-16T00:08:09.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621652+0000", 2024-09-16T00:08:09.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1984999999999998e-05 2024-09-16T00:08:09.319 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.319 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.319 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.319 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:23 2.e e22/19 ondisk, result = 0)", 2024-09-16T00:08:09.319 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:09.320 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.575097542999998, 2024-09-16T00:08:09.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015895699999999999, 2024-09-16T00:08:09.320 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.320 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.320 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:09.320 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:09.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:09.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621870+0000", 2024-09-16T00:08:09.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1600000000000001e-07 2024-09-16T00:08:09.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621873+0000", 2024-09-16T00:08:09.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3079999999999999e-06 2024-09-16T00:08:09.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621882+0000", 2024-09-16T00:08:09.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8429999999999999e-06 2024-09-16T00:08:09.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621896+0000", 2024-09-16T00:08:09.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:08:09.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621944+0000", 2024-09-16T00:08:09.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8062999999999997e-05 2024-09-16T00:08:09.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621951+0000", 2024-09-16T00:08:09.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7689999999999999e-06 2024-09-16T00:08:09.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622028+0000", 2024-09-16T00:08:09.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7243999999999994e-05 2024-09-16T00:08:09.323 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.323 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.324 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.324 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.324 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.324 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:09.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.204+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7efecc02da70 con 0x7efecc207730 2024-09-16T00:08:09.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.206+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efecc02da70 con 0x7efecc207730 2024-09-16T00:08:09.325 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:09.326 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:09.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.206+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7efecc20a570 0x7efecc20a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:09.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.206+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7efecc20fdd0 con 0x7efecc20a570 2024-09-16T00:08:09.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.207+0000 7efecb7fe640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7efecc20a570 0x7efecc20a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:09.327 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:08:09.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.207+0000 7efecb7fe640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7efecc20a570 0x7efecc20a9f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:09.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.208+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efecc20fdd0 con 0x7efecc20a570 2024-09-16T00:08:09.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.221+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7efecc20b040 con 0x7efecc20a570 2024-09-16T00:08:09.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efeb3fff640 1 -- 172.21.15.139:0/2287309322 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7efecc20b040 con 0x7efecc20a570 2024-09-16T00:08:09.329 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:09.330 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:09.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7efecc158560 msgr2=0x7efecc148360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:09.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7efecc158560 0x7efecc148360 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:09.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7efecc158940 msgr2=0x7efecc0ccca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:09.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7efecc158940 0x7efecc0ccca0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:09.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7efecc0cd6f0 msgr2=0x7efecc062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:09.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7efecc0cd6f0 0x7efecc062630 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:09.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7efecc0ce0b0 msgr2=0x7efecc0ce4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:09.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7efecc0ce0b0 0x7efecc0ce4d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:09.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7efecc02cb50 msgr2=0x7efecc02cf90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:09.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7efecc02cb50 0x7efecc02cf90 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:09.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efecbfff640 1 -- 172.21.15.139:0/2287309322 reap_dead start 2024-09-16T00:08:09.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7efecc2046e0 msgr2=0x7efecc206ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:09.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7efecc2046e0 0x7efecc206ad0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:09.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efecbfff640 1 -- 172.21.15.139:0/2287309322 reap_dead start 2024-09-16T00:08:09.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7efecc207730 msgr2=0x7efecc209b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:09.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7efecc207730 0x7efecc209b20 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:09.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7efecc20a570 msgr2=0x7efecc20a9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:09.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.223+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7efecc20a570 0x7efecc20a9f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:09.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.224+0000 7efecbfff640 1 -- 172.21.15.139:0/2287309322 reap_dead start 2024-09-16T00:08:09.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.224+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efeb806c2e0 msgr2=0x7efeb806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:09.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.224+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efeb806c2e0 0x7efeb806e7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efebc001650 tx=0x7efebc009040 comp rx=0 tx=0).stop 2024-09-16T00:08:09.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.224+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efecc0d0870 msgr2=0x7efecc1f04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:09.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.224+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efecc0d0870 0x7efecc1f04b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efec400caf0 tx=0x7efec400a060 comp rx=0 tx=0).stop 2024-09-16T00:08:09.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.224+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 shutdown_connections 2024-09-16T00:08:09.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.224+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efeb806c2e0 0x7efeb806e7a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:09.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.224+0000 7efed20af640 1 --2- 172.21.15.139:0/2287309322 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efecc0d0870 0x7efecc1f04b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:09.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.224+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 >> 172.21.15.139:0/2287309322 conn(0x7efecc06c520 msgr2=0x7efecc14a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:09.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.224+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 shutdown_connections 2024-09-16T00:08:09.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:09.224+0000 7efed20af640 1 -- 172.21.15.139:0/2287309322 wait complete. 2024-09-16T00:08:09.333 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:09.324+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:09.426 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:09.426+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:09.473 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:09.473 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:09.473 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:09.474 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:09.482 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:08:09.527 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:09.527+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:09.586 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:09.586 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:09.586 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:09.586 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:09.586 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:09.595 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:08:09.639 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:09.639+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:09.701 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:09.702 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:09.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:09.702 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:09.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.702 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:22 2.e e22/19 2:717a0223:::608.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:09.702 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:09.702 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.084308143000001, 2024-09-16T00:08:09.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042195870000000003, 2024-09-16T00:08:09.702 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.702 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.702 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.702 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:09.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:09.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616426+0000", 2024-09-16T00:08:09.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.037e-06 2024-09-16T00:08:09.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616435+0000", 2024-09-16T00:08:09.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6440000000000006e-06 2024-09-16T00:08:09.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616447+0000", 2024-09-16T00:08:09.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2466000000000001e-05 2024-09-16T00:08:09.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616456+0000", 2024-09-16T00:08:09.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8799999999999997e-06 2024-09-16T00:08:09.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616517+0000", 2024-09-16T00:08:09.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1171000000000001e-05 2024-09-16T00:08:09.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616540+0000", 2024-09-16T00:08:09.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2991000000000001e-05 2024-09-16T00:08:09.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620599+0000", 2024-09-16T00:08:09.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040586140000000003 2024-09-16T00:08:09.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620645+0000", 2024-09-16T00:08:09.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5784000000000001e-05 2024-09-16T00:08:09.706 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.706 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.706 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.706 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:09.706 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:09.706 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.084218412999999, 2024-09-16T00:08:09.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044836249999999998, 2024-09-16T00:08:09.707 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.707 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.707 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:09.707 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:09.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:09.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616518+0000", 2024-09-16T00:08:09.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3699999999999999e-06 2024-09-16T00:08:09.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:08:09.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1479999999999997e-06 2024-09-16T00:08:09.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616534+0000", 2024-09-16T00:08:09.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0641e-05 2024-09-16T00:08:09.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616541+0000", 2024-09-16T00:08:09.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3339999999999996e-06 2024-09-16T00:08:09.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617318+0000", 2024-09-16T00:08:09.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00077694799999999996 2024-09-16T00:08:09.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617409+0000", 2024-09-16T00:08:09.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0740000000000002e-05 2024-09-16T00:08:09.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620998+0000", 2024-09-16T00:08:09.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035894439999999998 2024-09-16T00:08:09.710 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.710 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.710 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.710 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:09.711 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:09.711 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.08417051, 2024-09-16T00:08:09.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041597320000000002, 2024-09-16T00:08:09.711 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.711 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.711 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:09.711 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:09.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:09.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:09.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:08:09.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616566+0000", 2024-09-16T00:08:09.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6979999999999998e-06 2024-09-16T00:08:09.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616574+0000", 2024-09-16T00:08:09.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7759999999999997e-06 2024-09-16T00:08:09.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616580+0000", 2024-09-16T00:08:09.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:09.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616800+0000", 2024-09-16T00:08:09.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022016700000000001 2024-09-16T00:08:09.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616904+0000", 2024-09-16T00:08:09.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104579 2024-09-16T00:08:09.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620722+0000", 2024-09-16T00:08:09.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003818133 2024-09-16T00:08:09.714 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.714 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.714 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.714 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:09.715 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:09.715 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.083919824999999, 2024-09-16T00:08:09.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038451570000000001, 2024-09-16T00:08:09.715 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.715 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.715 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.715 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:09.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:09.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616814+0000", 2024-09-16T00:08:09.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9299999999999997e-07 2024-09-16T00:08:09.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616824+0000", 2024-09-16T00:08:09.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6600000000000007e-06 2024-09-16T00:08:09.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616835+0000", 2024-09-16T00:08:09.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0713000000000001e-05 2024-09-16T00:08:09.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616841+0000", 2024-09-16T00:08:09.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.934e-06 2024-09-16T00:08:09.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616948+0000", 2024-09-16T00:08:09.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010631100000000001 2024-09-16T00:08:09.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616967+0000", 2024-09-16T00:08:09.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9598000000000001e-05 2024-09-16T00:08:09.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620619+0000", 2024-09-16T00:08:09.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036515810000000001 2024-09-16T00:08:09.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620659+0000", 2024-09-16T00:08:09.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9666999999999998e-05 2024-09-16T00:08:09.719 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.719 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.719 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.719 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:09.719 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:09.719 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.083872873000001, 2024-09-16T00:08:09.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038240129999999998, 2024-09-16T00:08:09.719 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.719 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.719 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.719 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:09.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:09.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616861+0000", 2024-09-16T00:08:09.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0200000000000003e-07 2024-09-16T00:08:09.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616864+0000", 2024-09-16T00:08:09.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5090000000000002e-06 2024-09-16T00:08:09.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616871+0000", 2024-09-16T00:08:09.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9249999999999998e-06 2024-09-16T00:08:09.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616877+0000", 2024-09-16T00:08:09.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.981e-06 2024-09-16T00:08:09.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617168+0000", 2024-09-16T00:08:09.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029035699999999998 2024-09-16T00:08:09.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617184+0000", 2024-09-16T00:08:09.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6005999999999999e-05 2024-09-16T00:08:09.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620661+0000", 2024-09-16T00:08:09.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034775330000000001 2024-09-16T00:08:09.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620684+0000", 2024-09-16T00:08:09.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3300000000000001e-05 2024-09-16T00:08:09.723 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.723 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.723 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.724 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:09.724 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:09.724 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.083417430000001, 2024-09-16T00:08:09.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045415610000000004, 2024-09-16T00:08:09.724 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.724 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.724 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.724 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:09.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:09.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:09.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8800000000000002e-07 2024-09-16T00:08:09.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617321+0000", 2024-09-16T00:08:09.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6979999999999997e-06 2024-09-16T00:08:09.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617330+0000", 2024-09-16T00:08:09.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6729999999999998e-06 2024-09-16T00:08:09.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617336+0000", 2024-09-16T00:08:09.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2110000000000003e-06 2024-09-16T00:08:09.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617394+0000", 2024-09-16T00:08:09.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8087999999999997e-05 2024-09-16T00:08:09.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617412+0000", 2024-09-16T00:08:09.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8099999999999999e-05 2024-09-16T00:08:09.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621818+0000", 2024-09-16T00:08:09.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044061209999999998 2024-09-16T00:08:09.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621857+0000", 2024-09-16T00:08:09.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9082000000000003e-05 2024-09-16T00:08:09.728 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.728 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.728 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.728 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:23 2.e e22/19 2:717a0223:::608.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:09.728 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:09.728 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.082521409000002, 2024-09-16T00:08:09.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036198279999999998, 2024-09-16T00:08:09.728 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.728 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.728 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.728 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:09.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:09.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618213+0000", 2024-09-16T00:08:09.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:08:09.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618218+0000", 2024-09-16T00:08:09.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4099999999999999e-06 2024-09-16T00:08:09.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618227+0000", 2024-09-16T00:08:09.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5009999999999993e-06 2024-09-16T00:08:09.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618234+0000", 2024-09-16T00:08:09.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:09.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618307+0000", 2024-09-16T00:08:09.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3421000000000001e-05 2024-09-16T00:08:09.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618328+0000", 2024-09-16T00:08:09.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0919000000000001e-05 2024-09-16T00:08:09.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.732 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621804+0000", 2024-09-16T00:08:09.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034756560000000001 2024-09-16T00:08:09.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.732 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621832+0000", 2024-09-16T00:08:09.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7966999999999998e-05 2024-09-16T00:08:09.732 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.732 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.732 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.732 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing_reply(2.1e epoch 22/19 rep_tid 0 lcod 22'1)", 2024-09-16T00:08:09.732 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:09.733 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.082076240999999, 2024-09-16T00:08:09.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1371000000000003e-05, 2024-09-16T00:08:09.733 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.733 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.733 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:09.733 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:09.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:09.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:08:09.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:08:09.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618662+0000", 2024-09-16T00:08:09.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1189999999999999e-06 2024-09-16T00:08:09.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618674+0000", 2024-09-16T00:08:09.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2317000000000001e-05 2024-09-16T00:08:09.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618681+0000", 2024-09-16T00:08:09.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7170000000000002e-06 2024-09-16T00:08:09.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:08:09.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6745000000000001e-05 2024-09-16T00:08:09.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618748+0000", 2024-09-16T00:08:09.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0794000000000001e-05 2024-09-16T00:08:09.736 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.736 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.736 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.736 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:09.736 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:09.736 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.081562906999999, 2024-09-16T00:08:09.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027183849999999998, 2024-09-16T00:08:09.736 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.736 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.737 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.737 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:09.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:09.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619171+0000", 2024-09-16T00:08:09.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8999999999999995e-07 2024-09-16T00:08:09.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619179+0000", 2024-09-16T00:08:09.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2679999999999994e-06 2024-09-16T00:08:09.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:08:09.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1246e-05 2024-09-16T00:08:09.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619197+0000", 2024-09-16T00:08:09.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2820000000000002e-06 2024-09-16T00:08:09.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619308+0000", 2024-09-16T00:08:09.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011141000000000001 2024-09-16T00:08:09.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619325+0000", 2024-09-16T00:08:09.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6469000000000001e-05 2024-09-16T00:08:09.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621860+0000", 2024-09-16T00:08:09.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025353590000000001 2024-09-16T00:08:09.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621889+0000", 2024-09-16T00:08:09.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8561e-05 2024-09-16T00:08:09.740 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.740 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.740 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.740 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 6 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:09.741 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:09.741 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.081520208000001, 2024-09-16T00:08:09.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026903809999999999, 2024-09-16T00:08:09.741 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.741 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.741 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:09.741 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:09.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:09.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:09.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2000000000000001e-07 2024-09-16T00:08:09.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619219+0000", 2024-09-16T00:08:09.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:08:09.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619234+0000", 2024-09-16T00:08:09.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5075000000000001e-05 2024-09-16T00:08:09.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619251+0000", 2024-09-16T00:08:09.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6863000000000001e-05 2024-09-16T00:08:09.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619556+0000", 2024-09-16T00:08:09.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030551599999999998 2024-09-16T00:08:09.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621903+0000", 2024-09-16T00:08:09.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023469979999999999 2024-09-16T00:08:09.744 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.744 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.744 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.744 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing_reply(2.1b epoch 22/19 rep_tid 1 lcod 22'1)", 2024-09-16T00:08:09.744 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:09.744 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.080441812, 2024-09-16T00:08:09.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6989000000000003e-05, 2024-09-16T00:08:09.745 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.745 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.745 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:09.745 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:09.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:09.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620292+0000", 2024-09-16T00:08:09.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.37e-07 2024-09-16T00:08:09.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620296+0000", 2024-09-16T00:08:09.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2420000000000002e-06 2024-09-16T00:08:09.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620308+0000", 2024-09-16T00:08:09.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1623e-05 2024-09-16T00:08:09.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620313+0000", 2024-09-16T00:08:09.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6290000000000001e-06 2024-09-16T00:08:09.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620359+0000", 2024-09-16T00:08:09.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6201e-05 2024-09-16T00:08:09.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620378+0000", 2024-09-16T00:08:09.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9556999999999999e-05 2024-09-16T00:08:09.748 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.748 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.748 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.748 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:38 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'6, mlcod=22'4)", 2024-09-16T00:08:09.748 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:09.748 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.077999562999999, 2024-09-16T00:08:09.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064551599999999995, 2024-09-16T00:08:09.749 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.749 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.749 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.749 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:09.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:09.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:09.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4399999999999999e-07 2024-09-16T00:08:09.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622739+0000", 2024-09-16T00:08:09.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0250000000000002e-06 2024-09-16T00:08:09.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622749+0000", 2024-09-16T00:08:09.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1220000000000001e-06 2024-09-16T00:08:09.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622760+0000", 2024-09-16T00:08:09.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1640999999999999e-05 2024-09-16T00:08:09.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622799+0000", 2024-09-16T00:08:09.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8537000000000001e-05 2024-09-16T00:08:09.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622810+0000", 2024-09-16T00:08:09.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1562e-05 2024-09-16T00:08:09.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.623356+0000", 2024-09-16T00:08:09.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054522299999999995 2024-09-16T00:08:09.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.623379+0000", 2024-09-16T00:08:09.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3662000000000001e-05 2024-09-16T00:08:09.752 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.752 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.752 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.753 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:40 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'7, mlcod=22'5)", 2024-09-16T00:08:09.753 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:09.753 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.860978661000001, 2024-09-16T00:08:09.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001017773, 2024-09-16T00:08:09.753 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.753 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.753 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.753 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:09.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:09.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839756+0000", 2024-09-16T00:08:09.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.491e-06 2024-09-16T00:08:09.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839768+0000", 2024-09-16T00:08:09.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1649e-05 2024-09-16T00:08:09.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839783+0000", 2024-09-16T00:08:09.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5023e-05 2024-09-16T00:08:09.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839791+0000", 2024-09-16T00:08:09.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5620000000000004e-06 2024-09-16T00:08:09.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839848+0000", 2024-09-16T00:08:09.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6563e-05 2024-09-16T00:08:09.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839868+0000", 2024-09-16T00:08:09.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9652000000000001e-05 2024-09-16T00:08:09.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.840729+0000", 2024-09-16T00:08:09.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086159500000000005 2024-09-16T00:08:09.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.840772+0000", 2024-09-16T00:08:09.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3238000000000002e-05 2024-09-16T00:08:09.757 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.757 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.757 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.757 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:41 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'8, mlcod=22'6)", 2024-09-16T00:08:09.758 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:09.758 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 16.587321750000001, 2024-09-16T00:08:09.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001040568, 2024-09-16T00:08:09.758 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:09.758 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:09.758 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:09.758 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:09.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:09.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:09.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:09.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:09.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:09.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:09.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113413+0000", 2024-09-16T00:08:09.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7409999999999999e-06 2024-09-16T00:08:09.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:09.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113428+0000", 2024-09-16T00:08:09.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5007999999999999e-05 2024-09-16T00:08:09.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:09.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113447+0000", 2024-09-16T00:08:09.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8731000000000001e-05 2024-09-16T00:08:09.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:09.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113457+0000", 2024-09-16T00:08:09.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0095000000000001e-05 2024-09-16T00:08:09.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:09.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113526+0000", 2024-09-16T00:08:09.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8552000000000004e-05 2024-09-16T00:08:09.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:09.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113552+0000", 2024-09-16T00:08:09.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6795999999999999e-05 2024-09-16T00:08:09.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:09.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.114417+0000", 2024-09-16T00:08:09.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086476299999999995 2024-09-16T00:08:09.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:09.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:09.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:09.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.114452+0000", 2024-09-16T00:08:09.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4882000000000002e-05 2024-09-16T00:08:09.762 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.762 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.762 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.762 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:09.762 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:09.762 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:09.764 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:08:09.764 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:09.763+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:09.864 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:09.864+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:09.879 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:09.879 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:09.879 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:09.879 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:09.888 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:08:09.965 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:09.964+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:10.034 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:10.034 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:10.034 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:10.034 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:10.034 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:10.043 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:08:10.066 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:10.065+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:10.148 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:10.148 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:10.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:10.148 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:10.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.148 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:10.148 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:10.148 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.532215097000002, 2024-09-16T00:08:10.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036770370000000002, 2024-09-16T00:08:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:10.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615154+0000", 2024-09-16T00:08:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1510000000000003e-06 2024-09-16T00:08:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615163+0000", 2024-09-16T00:08:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4490000000000005e-06 2024-09-16T00:08:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:10.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615182+0000", 2024-09-16T00:08:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8876e-05 2024-09-16T00:08:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615197+0000", 2024-09-16T00:08:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4671000000000001e-05 2024-09-16T00:08:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615349+0000", 2024-09-16T00:08:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015258700000000001 2024-09-16T00:08:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:08:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000204716 2024-09-16T00:08:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:08:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017250800000000001 2024-09-16T00:08:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616919+0000", 2024-09-16T00:08:10.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011921799999999999 2024-09-16T00:08:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:08:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018087769999999999 2024-09-16T00:08:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618774+0000", 2024-09-16T00:08:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6075999999999997e-05 2024-09-16T00:08:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:10.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618826+0000", 2024-09-16T00:08:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2046000000000003e-05 2024-09-16T00:08:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.530859309, 2024-09-16T00:08:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023501059999999998, 2024-09-16T00:08:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:10.154 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616506+0000", 2024-09-16T00:08:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4100000000000001e-06 2024-09-16T00:08:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617073+0000", 2024-09-16T00:08:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056725899999999997 2024-09-16T00:08:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617099+0000", 2024-09-16T00:08:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5860999999999999e-05 2024-09-16T00:08:10.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617114+0000", 2024-09-16T00:08:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5180999999999999e-05 2024-09-16T00:08:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617236+0000", 2024-09-16T00:08:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121662 2024-09-16T00:08:10.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001618733 2024-09-16T00:08:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:31 2.d e22/19 2:b50e409b:::mds_snaptable:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.529965161, 2024-09-16T00:08:10.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021851240000000001, 2024-09-16T00:08:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:10.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617400+0000", 2024-09-16T00:08:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.28e-06 2024-09-16T00:08:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617410+0000", 2024-09-16T00:08:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0179e-05 2024-09-16T00:08:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617435+0000", 2024-09-16T00:08:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5221000000000001e-05 2024-09-16T00:08:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617444+0000", 2024-09-16T00:08:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1989999999999995e-06 2024-09-16T00:08:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617514+0000", 2024-09-16T00:08:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9104999999999997e-05 2024-09-16T00:08:10.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617538+0000", 2024-09-16T00:08:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4383999999999999e-05 2024-09-16T00:08:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619536+0000", 2024-09-16T00:08:10.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019982789999999999 2024-09-16T00:08:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619584+0000", 2024-09-16T00:08:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7477e-05 2024-09-16T00:08:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:27 2.13 e22/19 2:cd5a64a3:::100.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:10.163 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.529659979000002, 2024-09-16T00:08:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018775459999999999, 2024-09-16T00:08:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617705+0000", 2024-09-16T00:08:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9900000000000009e-07 2024-09-16T00:08:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617718+0000", 2024-09-16T00:08:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3453e-05 2024-09-16T00:08:10.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617730+0000", 2024-09-16T00:08:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2221e-05 2024-09-16T00:08:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617739+0000", 2024-09-16T00:08:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0660000000000004e-06 2024-09-16T00:08:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617806+0000", 2024-09-16T00:08:10.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7397999999999994e-05 2024-09-16T00:08:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617823+0000", 2024-09-16T00:08:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7569999999999999e-05 2024-09-16T00:08:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619528+0000", 2024-09-16T00:08:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017045859999999999 2024-09-16T00:08:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619581+0000", 2024-09-16T00:08:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3253e-05 2024-09-16T00:08:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:28 2.13 e22/19 2:c91d4a1d:::mds0_inotable:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.529312708999999, 2024-09-16T00:08:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001558051, 2024-09-16T00:08:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:10.168 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618052+0000", 2024-09-16T00:08:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1399999999999997e-07 2024-09-16T00:08:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618065+0000", 2024-09-16T00:08:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3495e-05 2024-09-16T00:08:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:08:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0560999999999999e-05 2024-09-16T00:08:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:10.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618083+0000", 2024-09-16T00:08:10.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:08:10.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:10.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618147+0000", 2024-09-16T00:08:10.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4525999999999999e-05 2024-09-16T00:08:10.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:10.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618166+0000", 2024-09-16T00:08:10.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9045000000000001e-05 2024-09-16T00:08:10.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:10.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619585+0000", 2024-09-16T00:08:10.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014182660000000001 2024-09-16T00:08:10.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:10.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619609+0000", 2024-09-16T00:08:10.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4468e-05 2024-09-16T00:08:10.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.172 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:10.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.173 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:29 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:10.173 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:10.173 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.528962204999999, 2024-09-16T00:08:10.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012347219999999999, 2024-09-16T00:08:10.173 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:10.173 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:10.173 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:10.173 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:10.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:10.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:10.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:10.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:10.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:10.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:10.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3099999999999997e-07 2024-09-16T00:08:10.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:10.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618409+0000", 2024-09-16T00:08:10.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6239999999999996e-06 2024-09-16T00:08:10.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:10.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618420+0000", 2024-09-16T00:08:10.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1222e-05 2024-09-16T00:08:10.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:10.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618441+0000", 2024-09-16T00:08:10.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0564000000000002e-05 2024-09-16T00:08:10.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:10.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618519+0000", 2024-09-16T00:08:10.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7917999999999995e-05 2024-09-16T00:08:10.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:10.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618535+0000", 2024-09-16T00:08:10.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6266e-05 2024-09-16T00:08:10.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:10.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619612+0000", 2024-09-16T00:08:10.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010767039999999999 2024-09-16T00:08:10.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:10.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619636+0000", 2024-09-16T00:08:10.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4692999999999999e-05 2024-09-16T00:08:10.177 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.177 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:10.177 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.178 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:2 2.f e22/19 ondisk, result = 0)", 2024-09-16T00:08:10.178 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:10.178 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.528726177999999, 2024-09-16T00:08:10.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019782099999999999, 2024-09-16T00:08:10.178 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:10.178 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:10.178 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:10.178 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:10.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:10.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:10.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:10.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:10.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:10.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618639+0000", 2024-09-16T00:08:10.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5099999999999998e-07 2024-09-16T00:08:10.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:10.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618644+0000", 2024-09-16T00:08:10.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7440000000000004e-06 2024-09-16T00:08:10.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:10.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618655+0000", 2024-09-16T00:08:10.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0798999999999999e-05 2024-09-16T00:08:10.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:10.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618662+0000", 2024-09-16T00:08:10.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8700000000000003e-06 2024-09-16T00:08:10.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:10.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618717+0000", 2024-09-16T00:08:10.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5223999999999998e-05 2024-09-16T00:08:10.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:10.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618724+0000", 2024-09-16T00:08:10.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.046e-06 2024-09-16T00:08:10.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:10.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618835+0000", 2024-09-16T00:08:10.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111187 2024-09-16T00:08:10.182 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.182 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:10.182 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.182 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.b epoch 22/19 rep_tid 122 entries 22'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:10.182 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:10.183 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.527621785000001, 2024-09-16T00:08:10.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018692890000000001, 2024-09-16T00:08:10.183 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:10.183 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:10.183 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:10.183 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:10.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:10.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:10.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:10.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:10.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:10.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619743+0000", 2024-09-16T00:08:10.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1069999999999999e-06 2024-09-16T00:08:10.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:10.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619754+0000", 2024-09-16T00:08:10.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0789e-05 2024-09-16T00:08:10.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:10.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619777+0000", 2024-09-16T00:08:10.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2844e-05 2024-09-16T00:08:10.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:10.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619793+0000", 2024-09-16T00:08:10.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6231000000000001e-05 2024-09-16T00:08:10.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:10.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619875+0000", 2024-09-16T00:08:10.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2286000000000005e-05 2024-09-16T00:08:10.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:10.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621611+0000", 2024-09-16T00:08:10.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001736032 2024-09-16T00:08:10.186 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.186 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:10.186 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.186 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.186 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:10.186 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:10.188 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:08:10.188 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:10.187+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:10.288 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:10.288+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:10.332 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:10.332 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:10.332 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:10.341 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:08:10.389 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:10.388+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:10.445 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:10.445 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:10.445 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:10.445 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:10.445 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:10.454 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:08:10.498 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:10.498+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:10.558 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:10.558 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:10.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:10.559 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:10.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.559 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:14 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:10.559 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:10.559 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.942218187999998, 2024-09-16T00:08:10.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029190470000000001, 2024-09-16T00:08:10.559 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:10.559 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:10.559 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:10.559 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:10.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:10.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:10.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:10.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:10.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:10.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615982+0000", 2024-09-16T00:08:10.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7439999999999999e-06 2024-09-16T00:08:10.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:10.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:08:10.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3743999999999999e-05 2024-09-16T00:08:10.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:10.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:10.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2255e-05 2024-09-16T00:08:10.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:10.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616029+0000", 2024-09-16T00:08:10.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1175999999999999e-05 2024-09-16T00:08:10.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:10.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616098+0000", 2024-09-16T00:08:10.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9153999999999996e-05 2024-09-16T00:08:10.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:10.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616132+0000", 2024-09-16T00:08:10.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3924000000000002e-05 2024-09-16T00:08:10.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:10.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:10.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027226189999999999 2024-09-16T00:08:10.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:10.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618899+0000", 2024-09-16T00:08:10.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4431000000000002e-05 2024-09-16T00:08:10.564 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.564 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:10.564 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.564 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:15 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:10.564 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:10.564 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.939991368000001, 2024-09-16T00:08:10.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095618099999999996, 2024-09-16T00:08:10.565 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:10.565 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:10.565 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:10.565 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:10.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:10.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:10.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:10.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:10.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:10.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618208+0000", 2024-09-16T00:08:10.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1850000000000001e-06 2024-09-16T00:08:10.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:10.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618217+0000", 2024-09-16T00:08:10.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439999999999992e-06 2024-09-16T00:08:10.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:10.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618230+0000", 2024-09-16T00:08:10.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2435999999999999e-05 2024-09-16T00:08:10.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:10.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618238+0000", 2024-09-16T00:08:10.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1319999999999994e-06 2024-09-16T00:08:10.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:10.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618331+0000", 2024-09-16T00:08:10.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3453000000000001e-05 2024-09-16T00:08:10.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:10.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618352+0000", 2024-09-16T00:08:10.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0693999999999999e-05 2024-09-16T00:08:10.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:10.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619112+0000", 2024-09-16T00:08:10.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00076007099999999997 2024-09-16T00:08:10.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:10.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619163+0000", 2024-09-16T00:08:10.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1066000000000002e-05 2024-09-16T00:08:10.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.569 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:10.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.569 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:30 2.15 e22/19 2:aa448500:::500.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:10.570 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:10.570 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.939933882999998, 2024-09-16T00:08:10.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058031250000000001, 2024-09-16T00:08:10.570 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:10.570 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:10.570 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:10.570 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:10.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:10.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:10.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:10.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:10.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:10.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618265+0000", 2024-09-16T00:08:10.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9200000000000001e-07 2024-09-16T00:08:10.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:10.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618271+0000", 2024-09-16T00:08:10.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9580000000000004e-06 2024-09-16T00:08:10.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:10.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618279+0000", 2024-09-16T00:08:10.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2239999999999995e-06 2024-09-16T00:08:10.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:10.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618285+0000", 2024-09-16T00:08:10.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-06 2024-09-16T00:08:10.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:10.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618567+0000", 2024-09-16T00:08:10.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028204400000000001 2024-09-16T00:08:10.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:10.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:08:10.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9678999999999999e-05 2024-09-16T00:08:10.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:10.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624024+0000", 2024-09-16T00:08:10.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054376720000000002 2024-09-16T00:08:10.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:10.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624068+0000", 2024-09-16T00:08:10.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:08:10.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.574 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:10.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.574 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:20 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:10.574 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:10.574 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.939577519, 2024-09-16T00:08:10.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028808739999999998, 2024-09-16T00:08:10.574 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:10.574 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:10.574 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:10.575 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:10.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:10.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:10.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:10.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:10.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:10.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618622+0000", 2024-09-16T00:08:10.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1260000000000001e-06 2024-09-16T00:08:10.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:10.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618631+0000", 2024-09-16T00:08:10.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4700000000000008e-06 2024-09-16T00:08:10.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:10.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618645+0000", 2024-09-16T00:08:10.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3984e-05 2024-09-16T00:08:10.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:10.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:08:10.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2101e-05 2024-09-16T00:08:10.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:10.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618720+0000", 2024-09-16T00:08:10.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2761000000000004e-05 2024-09-16T00:08:10.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:10.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618740+0000", 2024-09-16T00:08:10.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9409e-05 2024-09-16T00:08:10.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:10.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621436+0000", 2024-09-16T00:08:10.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002696211 2024-09-16T00:08:10.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:10.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621502+0000", 2024-09-16T00:08:10.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5811999999999999e-05 2024-09-16T00:08:10.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.578 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:10.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.579 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:21 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:10.579 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:10.579 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.938754231000001, 2024-09-16T00:08:10.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046318540000000004, 2024-09-16T00:08:10.579 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:10.579 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:10.579 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:10.579 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:10.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:10.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:10.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:10.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:10.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:10.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:10.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619445+0000", 2024-09-16T00:08:10.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.083e-06 2024-09-16T00:08:10.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:10.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619455+0000", 2024-09-16T00:08:10.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5920000000000007e-06 2024-09-16T00:08:10.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:10.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619467+0000", 2024-09-16T00:08:10.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2711000000000001e-05 2024-09-16T00:08:10.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:10.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619476+0000", 2024-09-16T00:08:10.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0139999999999998e-06 2024-09-16T00:08:10.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:10.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619529+0000", 2024-09-16T00:08:10.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3078999999999998e-05 2024-09-16T00:08:10.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:10.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619548+0000", 2024-09-16T00:08:10.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8952000000000001e-05 2024-09-16T00:08:10.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:10.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624037+0000", 2024-09-16T00:08:10.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044895730000000002 2024-09-16T00:08:10.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:10.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:10.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624076+0000", 2024-09-16T00:08:10.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8850000000000002e-05 2024-09-16T00:08:10.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.583 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:10.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:10.583 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:10.583 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:10.584 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:08:10.625 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:10.625+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:10.692 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:10.692+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:10.730 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:10.730 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:10.730 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:10.730 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:10.739 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:08:10.772 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T00:08:10.815 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:10.815+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:10.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.876+0000 7f4029ee4640 1 Processor -- start 2024-09-16T00:08:10.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.876+0000 7f4029ee4640 1 -- start start 2024-09-16T00:08:10.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.876+0000 7f4029ee4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402415b490 0x7f402406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:10.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.876+0000 7f4029ee4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4024070c60 0x7f40240ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:10.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.876+0000 7f4029ee4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f402406d390 con 0x7f4024158050 2024-09-16T00:08:10.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.876+0000 7f4029ee4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f40240710d0 con 0x7f4024070c60 2024-09-16T00:08:10.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.876+0000 7f4029ee4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f402406cdd0 con 0x7f402415b490 2024-09-16T00:08:10.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.877+0000 7f40237fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4024158050 0x7f40240c93b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56318/0 (socket says 172.21.15.139:56318) 2024-09-16T00:08:10.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.877+0000 7f40237fe640 1 -- 172.21.15.139:0/1754706602 learned_addr learned my addr 172.21.15.139:0/1754706602 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:10.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.877+0000 7f4022ffd640 1 --2- 172.21.15.139:0/1754706602 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402415b490 0x7f402406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:10.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.877+0000 7f4023fff640 1 --2- 172.21.15.139:0/1754706602 >> v2:172.21.15.164:3300/0 conn(0x7f4024070c60 0x7f40240ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:10.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.877+0000 7f40227fc640 1 -- 172.21.15.139:0/1754706602 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1382715732 0 0) 0x7f402406d390 con 0x7f4024158050 2024-09-16T00:08:10.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.877+0000 7f40227fc640 1 -- 172.21.15.139:0/1754706602 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f400c003520 con 0x7f4024158050 2024-09-16T00:08:10.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.877+0000 7f4022ffd640 1 -- 172.21.15.139:0/1754706602 >> v2:172.21.15.164:3300/0 conn(0x7f4024070c60 msgr2=0x7f40240ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:10.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.877+0000 7f4022ffd640 1 --2- 172.21.15.139:0/1754706602 >> v2:172.21.15.164:3300/0 conn(0x7f4024070c60 0x7f40240ce1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:10.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.877+0000 7f4022ffd640 1 -- 172.21.15.139:0/1754706602 >> v1:172.21.15.139:6789/0 conn(0x7f4024158050 legacy=0x7f40240c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:10.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.877+0000 7f4022ffd640 1 -- 172.21.15.139:0/1754706602 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402406d0b0 con 0x7f402415b490 2024-09-16T00:08:10.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.877+0000 7f4023fff640 1 --2- 172.21.15.139:0/1754706602 >> v2:172.21.15.164:3300/0 conn(0x7f4024070c60 0x7f40240ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:10.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.878+0000 7f4022ffd640 1 --2- 172.21.15.139:0/1754706602 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402415b490 0x7f402406e8f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f40180097a0 tx=0x7f401802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a60d797a651142c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:10.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.878+0000 7f40227fc640 1 -- 172.21.15.139:0/1754706602 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018037070 con 0x7f402415b490 2024-09-16T00:08:10.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.878+0000 7f40227fc640 1 -- 172.21.15.139:0/1754706602 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4018032440 con 0x7f402415b490 2024-09-16T00:08:10.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.878+0000 7f40227fc640 1 -- 172.21.15.139:0/1754706602 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018032810 con 0x7f402415b490 2024-09-16T00:08:10.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.879+0000 7f4029ee4640 1 -- 172.21.15.139:0/1754706602 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402415b490 msgr2=0x7f402406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:10.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.879+0000 7f4029ee4640 1 --2- 172.21.15.139:0/1754706602 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402415b490 0x7f402406e8f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f40180097a0 tx=0x7f401802a180 comp rx=0 tx=0).stop 2024-09-16T00:08:10.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.879+0000 7f4029ee4640 1 -- 172.21.15.139:0/1754706602 shutdown_connections 2024-09-16T00:08:10.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.879+0000 7f4029ee4640 1 --2- 172.21.15.139:0/1754706602 >> v2:172.21.15.164:3300/0 conn(0x7f4024070c60 0x7f40240ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:10.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.879+0000 7f4029ee4640 1 --2- 172.21.15.139:0/1754706602 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402415b490 0x7f402406e8f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:10.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.879+0000 7f4029ee4640 1 -- 172.21.15.139:0/1754706602 >> 172.21.15.139:0/1754706602 conn(0x7f402406bd00 msgr2=0x7f4024067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:10.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.879+0000 7f4029ee4640 1 -- 172.21.15.139:0/1754706602 shutdown_connections 2024-09-16T00:08:10.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.879+0000 7f4029ee4640 1 -- 172.21.15.139:0/1754706602 wait complete. 2024-09-16T00:08:10.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.880+0000 7f4029ee4640 1 Processor -- start 2024-09-16T00:08:10.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.880+0000 7f4029ee4640 1 -- start start 2024-09-16T00:08:10.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f4029ee4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4024070c60 0x7f40240af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:10.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f4029ee4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4024158050 0x7f402415d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:10.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f4029ee4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402415b490 0x7f40241548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:10.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f4029ee4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4024158430 con 0x7f4024070c60 2024-09-16T00:08:10.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f4029ee4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f40240705d0 con 0x7f4024158050 2024-09-16T00:08:10.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f4029ee4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f40240ca140 con 0x7f402415b490 2024-09-16T00:08:10.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f40237fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4024070c60 0x7f40240af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:10.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f40237fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4024070c60 0x7f40240af470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35862/0 (socket says 172.21.15.139:35862) 2024-09-16T00:08:10.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f40237fe640 1 -- 172.21.15.139:0/1049609168 learned_addr learned my addr 172.21.15.139:0/1049609168 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:10.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f4022ffd640 1 --2- 172.21.15.139:0/1049609168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4024158050 0x7f402415d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:10.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f4023fff640 1 --2- 172.21.15.139:0/1049609168 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402415b490 0x7f40241548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:10.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f40237fe640 1 -- 172.21.15.139:0/1049609168 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402415b490 msgr2=0x7f40241548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:10.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f40237fe640 1 --2- 172.21.15.139:0/1049609168 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402415b490 0x7f40241548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:10.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f40237fe640 1 -- 172.21.15.139:0/1049609168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4024158050 msgr2=0x7f402415d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:10.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f40237fe640 1 --2- 172.21.15.139:0/1049609168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4024158050 0x7f402415d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:10.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f40237fe640 1 -- 172.21.15.139:0/1049609168 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40241f0ec0 con 0x7f4024070c60 2024-09-16T00:08:10.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f40237fe640 1 --2- 172.21.15.139:0/1049609168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4024070c60 0x7f40240af470 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f401400c430 tx=0x7f401400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:10.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f400bfff640 1 -- 172.21.15.139:0/1049609168 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014017070 con 0x7f4024070c60 2024-09-16T00:08:10.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.881+0000 7f400bfff640 1 -- 172.21.15.139:0/1049609168 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4014005ce0 con 0x7f4024070c60 2024-09-16T00:08:10.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.882+0000 7f4029ee4640 1 -- 172.21.15.139:0/1049609168 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40241f1180 con 0x7f4024070c60 2024-09-16T00:08:10.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.882+0000 7f400bfff640 1 -- 172.21.15.139:0/1049609168 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40140072f0 con 0x7f4024070c60 2024-09-16T00:08:10.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.882+0000 7f4029ee4640 1 -- 172.21.15.139:0/1049609168 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40241e64d0 con 0x7f4024070c60 2024-09-16T00:08:10.886 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:10.886 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:10.886 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:10.886 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:10.886 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:10.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.883+0000 7f4029ee4640 1 -- 172.21.15.139:0/1049609168 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f402406e8f0 con 0x7f4024070c60 2024-09-16T00:08:10.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.883+0000 7f400bfff640 1 -- 172.21.15.139:0/1049609168 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f401401a070 con 0x7f4024070c60 2024-09-16T00:08:10.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.884+0000 7f400bfff640 1 --2- 172.21.15.139:0/1049609168 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f401006c2b0 0x7f401006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:10.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.884+0000 7f400bfff640 1 -- 172.21.15.139:0/1049609168 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5738+0+0 (secure 0 0 0) 0x7f401408a7c0 con 0x7f4024070c60 2024-09-16T00:08:10.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.884+0000 7f4022ffd640 1 --2- 172.21.15.139:0/1049609168 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f401006c2b0 0x7f401006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:10.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.884+0000 7f4022ffd640 1 --2- 172.21.15.139:0/1049609168 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f401006c2b0 0x7f401006e770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4018034000 tx=0x7f401802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:10.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:10.888+0000 7f400bfff640 1 -- 172.21.15.139:0/1049609168 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f40140588c0 con 0x7f4024070c60 2024-09-16T00:08:10.893 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:08:10.894 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:10.894+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:10.995 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:10.994+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:10.999 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:10.999 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:10.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:10.999 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:10.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:10.999 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:10.999 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:10.999 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.382053675999998, 2024-09-16T00:08:10.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056100969999999997, 2024-09-16T00:08:10.999 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:10.999 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:10.999 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.000 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:11.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:11.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615352+0000", 2024-09-16T00:08:11.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9859999999999999e-06 2024-09-16T00:08:11.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615363+0000", 2024-09-16T00:08:11.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1168999999999999e-05 2024-09-16T00:08:11.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615382+0000", 2024-09-16T00:08:11.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8224e-05 2024-09-16T00:08:11.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615393+0000", 2024-09-16T00:08:11.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0925e-05 2024-09-16T00:08:11.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615624+0000", 2024-09-16T00:08:11.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023159600000000001 2024-09-16T00:08:11.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615721+0000", 2024-09-16T00:08:11.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7040000000000006e-05 2024-09-16T00:08:11.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:11.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615885+0000", 2024-09-16T00:08:11.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163456 2024-09-16T00:08:11.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:08:11.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002191368 2024-09-16T00:08:11.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620870+0000", 2024-09-16T00:08:11.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941839999999999 2024-09-16T00:08:11.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620922+0000", 2024-09-16T00:08:11.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1641999999999998e-05 2024-09-16T00:08:11.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620956+0000", 2024-09-16T00:08:11.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4507e-05 2024-09-16T00:08:11.007 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.007 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.007 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.007 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:11.007 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:11.007 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.381963317, 2024-09-16T00:08:11.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005695739, 2024-09-16T00:08:11.008 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.008 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.008 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.008 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:11.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:11.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:11.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4900000000000005e-07 2024-09-16T00:08:11.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615442+0000", 2024-09-16T00:08:11.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.994e-06 2024-09-16T00:08:11.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:08:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1081e-05 2024-09-16T00:08:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615473+0000", 2024-09-16T00:08:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1910000000000006e-06 2024-09-16T00:08:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616230+0000", 2024-09-16T00:08:11.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075700400000000003 2024-09-16T00:08:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616305+0000", 2024-09-16T00:08:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4759999999999996e-05 2024-09-16T00:08:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616466+0000", 2024-09-16T00:08:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000161714 2024-09-16T00:08:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619821+0000", 2024-09-16T00:08:11.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033547339999999998 2024-09-16T00:08:11.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621028+0000", 2024-09-16T00:08:11.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012067530000000001 2024-09-16T00:08:11.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621087+0000", 2024-09-16T00:08:11.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9364999999999998e-05 2024-09-16T00:08:11.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621132+0000", 2024-09-16T00:08:11.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5293999999999997e-05 2024-09-16T00:08:11.014 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.014 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.014 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.014 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:11.014 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:11.015 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.381901085000003, 2024-09-16T00:08:11.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050565760000000001, 2024-09-16T00:08:11.015 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.015 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.015 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.015 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:11.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:11.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615500+0000", 2024-09-16T00:08:11.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8199999999999999e-07 2024-09-16T00:08:11.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615505+0000", 2024-09-16T00:08:11.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5280000000000003e-06 2024-09-16T00:08:11.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:11.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0317e-05 2024-09-16T00:08:11.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615523+0000", 2024-09-16T00:08:11.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4580000000000004e-06 2024-09-16T00:08:11.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615607+0000", 2024-09-16T00:08:11.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4153999999999994e-05 2024-09-16T00:08:11.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615748+0000", 2024-09-16T00:08:11.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140533 2024-09-16T00:08:11.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:11.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615911+0000", 2024-09-16T00:08:11.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163678 2024-09-16T00:08:11.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619944+0000", 2024-09-16T00:08:11.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040330909999999999 2024-09-16T00:08:11.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620471+0000", 2024-09-16T00:08:11.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052645600000000002 2024-09-16T00:08:11.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620512+0000", 2024-09-16T00:08:11.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0738000000000002e-05 2024-09-16T00:08:11.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620555+0000", 2024-09-16T00:08:11.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3841000000000003e-05 2024-09-16T00:08:11.020 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.020 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.020 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.021 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:11.021 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:11.021 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.381848978000001, 2024-09-16T00:08:11.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089162479999999999, 2024-09-16T00:08:11.021 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.021 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.021 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.021 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:11.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:11.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615552+0000", 2024-09-16T00:08:11.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:08:11.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615556+0000", 2024-09-16T00:08:11.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3610000000000003e-06 2024-09-16T00:08:11.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615566+0000", 2024-09-16T00:08:11.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6099999999999995e-06 2024-09-16T00:08:11.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615574+0000", 2024-09-16T00:08:11.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9379999999999992e-06 2024-09-16T00:08:11.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616777+0000", 2024-09-16T00:08:11.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012031640000000001 2024-09-16T00:08:11.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616882+0000", 2024-09-16T00:08:11.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105146 2024-09-16T00:08:11.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:11.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619696+0000", 2024-09-16T00:08:11.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028142179999999998 2024-09-16T00:08:11.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624234+0000", 2024-09-16T00:08:11.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045381889999999998 2024-09-16T00:08:11.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624393+0000", 2024-09-16T00:08:11.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015871200000000001 2024-09-16T00:08:11.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624434+0000", 2024-09-16T00:08:11.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0908999999999997e-05 2024-09-16T00:08:11.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624467+0000", 2024-09-16T00:08:11.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3217999999999999e-05 2024-09-16T00:08:11.027 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.027 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.027 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.027 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:11.027 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:11.027 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.381779180999999, 2024-09-16T00:08:11.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064898359999999997, 2024-09-16T00:08:11.028 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.028 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.028 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.028 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:11.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:11.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615622+0000", 2024-09-16T00:08:11.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.181e-06 2024-09-16T00:08:11.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615628+0000", 2024-09-16T00:08:11.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2450000000000003e-06 2024-09-16T00:08:11.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:08:11.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2554999999999999e-05 2024-09-16T00:08:11.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615650+0000", 2024-09-16T00:08:11.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4350000000000003e-06 2024-09-16T00:08:11.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616772+0000", 2024-09-16T00:08:11.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001122058 2024-09-16T00:08:11.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616889+0000", 2024-09-16T00:08:11.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116623 2024-09-16T00:08:11.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:11.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617035+0000", 2024-09-16T00:08:11.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014579799999999999 2024-09-16T00:08:11.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619829+0000", 2024-09-16T00:08:11.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941379999999998 2024-09-16T00:08:11.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622024+0000", 2024-09-16T00:08:11.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021949920000000002 2024-09-16T00:08:11.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622087+0000", 2024-09-16T00:08:11.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3048999999999999e-05 2024-09-16T00:08:11.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622111+0000", 2024-09-16T00:08:11.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3762000000000001e-05 2024-09-16T00:08:11.034 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.034 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.034 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.034 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:11.034 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:11.034 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.381716619999999, 2024-09-16T00:08:11.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065192160000000004, 2024-09-16T00:08:11.034 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.035 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.035 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.035 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:11.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:11.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615684+0000", 2024-09-16T00:08:11.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8299999999999995e-07 2024-09-16T00:08:11.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615691+0000", 2024-09-16T00:08:11.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.866e-06 2024-09-16T00:08:11.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615707+0000", 2024-09-16T00:08:11.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6073999999999999e-05 2024-09-16T00:08:11.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615714+0000", 2024-09-16T00:08:11.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-16T00:08:11.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617403+0000", 2024-09-16T00:08:11.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016881629999999999 2024-09-16T00:08:11.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617559+0000", 2024-09-16T00:08:11.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015669699999999999 2024-09-16T00:08:11.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:11.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:11.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000144165 2024-09-16T00:08:11.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621413+0000", 2024-09-16T00:08:11.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037092200000000001 2024-09-16T00:08:11.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622147+0000", 2024-09-16T00:08:11.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073468800000000005 2024-09-16T00:08:11.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622180+0000", 2024-09-16T00:08:11.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2901999999999998e-05 2024-09-16T00:08:11.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622203+0000", 2024-09-16T00:08:11.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2211000000000001e-05 2024-09-16T00:08:11.040 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.040 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.040 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.040 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:11.041 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:11.041 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.381673526, 2024-09-16T00:08:11.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065825270000000003, 2024-09-16T00:08:11.041 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.041 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.041 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:11.041 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:11.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:11.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:08:11.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7299999999999996e-07 2024-09-16T00:08:11.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615730+0000", 2024-09-16T00:08:11.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.004e-06 2024-09-16T00:08:11.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615737+0000", 2024-09-16T00:08:11.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4789999999999999e-06 2024-09-16T00:08:11.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615745+0000", 2024-09-16T00:08:11.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6699999999999994e-06 2024-09-16T00:08:11.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618115+0000", 2024-09-16T00:08:11.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023701569999999999 2024-09-16T00:08:11.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618247+0000", 2024-09-16T00:08:11.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013185000000000001 2024-09-16T00:08:11.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622309+0000", 2024-09-16T00:08:11.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040617939999999996 2024-09-16T00:08:11.045 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.045 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.045 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.045 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:11.045 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:11.045 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.381646277000002, 2024-09-16T00:08:11.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036685950000000002, 2024-09-16T00:08:11.045 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.045 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.045 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:11.046 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:11.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:11.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:11.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T00:08:11.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615763+0000", 2024-09-16T00:08:11.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4360000000000002e-06 2024-09-16T00:08:11.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615770+0000", 2024-09-16T00:08:11.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7540000000000006e-06 2024-09-16T00:08:11.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615803+0000", 2024-09-16T00:08:11.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3008e-05 2024-09-16T00:08:11.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615888+0000", 2024-09-16T00:08:11.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5056999999999996e-05 2024-09-16T00:08:11.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:08:11.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013059700000000001 2024-09-16T00:08:11.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619422+0000", 2024-09-16T00:08:11.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034033230000000002 2024-09-16T00:08:11.049 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.050 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.050 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.050 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing_reply(2.1a epoch 22/19 rep_tid 2 lcod 22'1)", 2024-09-16T00:08:11.050 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:11.050 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.378204963999998, 2024-09-16T00:08:11.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024304200000000001, 2024-09-16T00:08:11.050 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.050 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.050 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:11.050 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:11.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:11.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619196+0000", 2024-09-16T00:08:11.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2759999999999999e-06 2024-09-16T00:08:11.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619203+0000", 2024-09-16T00:08:11.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1779999999999997e-06 2024-09-16T00:08:11.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619223+0000", 2024-09-16T00:08:11.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9159000000000001e-05 2024-09-16T00:08:11.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619231+0000", 2024-09-16T00:08:11.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7830000000000004e-06 2024-09-16T00:08:11.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619320+0000", 2024-09-16T00:08:11.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8805000000000001e-05 2024-09-16T00:08:11.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619438+0000", 2024-09-16T00:08:11.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117841 2024-09-16T00:08:11.054 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.054 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.054 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.054 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:16 2.1d e22/19 2:b871830b:::605.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:11.054 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:11.054 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.377657687000003, 2024-09-16T00:08:11.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045616900000000002, 2024-09-16T00:08:11.055 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.055 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.055 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.055 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:11.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:11.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:08:11.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:08:11.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619755+0000", 2024-09-16T00:08:11.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3599999999999996e-06 2024-09-16T00:08:11.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619767+0000", 2024-09-16T00:08:11.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2805e-05 2024-09-16T00:08:11.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619787+0000", 2024-09-16T00:08:11.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9264000000000001e-05 2024-09-16T00:08:11.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619869+0000", 2024-09-16T00:08:11.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2639999999999995e-05 2024-09-16T00:08:11.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619895+0000", 2024-09-16T00:08:11.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5890000000000001e-05 2024-09-16T00:08:11.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624264+0000", 2024-09-16T00:08:11.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043690049999999996 2024-09-16T00:08:11.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624304+0000", 2024-09-16T00:08:11.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9807e-05 2024-09-16T00:08:11.059 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.059 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.059 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.059 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:17 2.1d e22/19 2:b871830b:::605.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:11.059 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:11.059 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.377304977999998, 2024-09-16T00:08:11.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042370910000000001, 2024-09-16T00:08:11.059 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.060 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.060 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.060 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:11.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:11.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620096+0000", 2024-09-16T00:08:11.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:08:11.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620102+0000", 2024-09-16T00:08:11.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7220000000000002e-06 2024-09-16T00:08:11.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620112+0000", 2024-09-16T00:08:11.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0156999999999999e-05 2024-09-16T00:08:11.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620120+0000", 2024-09-16T00:08:11.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2880000000000004e-06 2024-09-16T00:08:11.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620171+0000", 2024-09-16T00:08:11.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.168e-05 2024-09-16T00:08:11.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620189+0000", 2024-09-16T00:08:11.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7419e-05 2024-09-16T00:08:11.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624306+0000", 2024-09-16T00:08:11.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041175430000000004 2024-09-16T00:08:11.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624332+0000", 2024-09-16T00:08:11.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5717000000000001e-05 2024-09-16T00:08:11.063 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.064 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.064 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.064 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:24 2.7 e22/19 ondisk, result = 0)", 2024-09-16T00:08:11.064 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:11.064 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.377039271000001, 2024-09-16T00:08:11.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000208018, 2024-09-16T00:08:11.064 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.064 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.064 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:11.064 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:11.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:11.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:11.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8500000000000001e-07 2024-09-16T00:08:11.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620366+0000", 2024-09-16T00:08:11.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5379999999999996e-06 2024-09-16T00:08:11.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620378+0000", 2024-09-16T00:08:11.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.204e-05 2024-09-16T00:08:11.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620385+0000", 2024-09-16T00:08:11.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.413e-06 2024-09-16T00:08:11.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620455+0000", 2024-09-16T00:08:11.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9886000000000001e-05 2024-09-16T00:08:11.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620463+0000", 2024-09-16T00:08:11.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1559999999999995e-06 2024-09-16T00:08:11.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620569+0000", 2024-09-16T00:08:11.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001054 2024-09-16T00:08:11.068 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.068 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.068 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.068 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:18 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:11.068 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:11.068 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.376653806999997, 2024-09-16T00:08:11.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021768100000000001, 2024-09-16T00:08:11.069 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.069 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.069 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:11.069 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:11.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:11.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620747+0000", 2024-09-16T00:08:11.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.02e-07 2024-09-16T00:08:11.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620752+0000", 2024-09-16T00:08:11.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4439999999999999e-06 2024-09-16T00:08:11.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620765+0000", 2024-09-16T00:08:11.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2714000000000001e-05 2024-09-16T00:08:11.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:08:11.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1160000000000004e-06 2024-09-16T00:08:11.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:11.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3374999999999994e-05 2024-09-16T00:08:11.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620864+0000", 2024-09-16T00:08:11.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4320000000000005e-06 2024-09-16T00:08:11.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620964+0000", 2024-09-16T00:08:11.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9698000000000007e-05 2024-09-16T00:08:11.073 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.073 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.073 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.073 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:19 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:11.073 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:11.073 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.376592262999999, 2024-09-16T00:08:11.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00033551400000000003, 2024-09-16T00:08:11.073 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.073 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.074 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:11.074 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:11.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:11.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:11.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0800000000000005e-07 2024-09-16T00:08:11.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620812+0000", 2024-09-16T00:08:11.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.427e-06 2024-09-16T00:08:11.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620820+0000", 2024-09-16T00:08:11.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6649999999999992e-06 2024-09-16T00:08:11.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620828+0000", 2024-09-16T00:08:11.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0010000000000001e-06 2024-09-16T00:08:11.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620995+0000", 2024-09-16T00:08:11.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016709400000000001 2024-09-16T00:08:11.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621005+0000", 2024-09-16T00:08:11.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7170000000000003e-06 2024-09-16T00:08:11.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621143+0000", 2024-09-16T00:08:11.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000138102 2024-09-16T00:08:11.077 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.077 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.077 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.077 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:3 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:11.077 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:11.077 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.375414992000003, 2024-09-16T00:08:11.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022247799999999999, 2024-09-16T00:08:11.078 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.078 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.078 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:11.078 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:11.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:11.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:11.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3300000000000003e-07 2024-09-16T00:08:11.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621989+0000", 2024-09-16T00:08:11.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2969999999999999e-06 2024-09-16T00:08:11.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621997+0000", 2024-09-16T00:08:11.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8990000000000001e-06 2024-09-16T00:08:11.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622002+0000", 2024-09-16T00:08:11.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5439999999999998e-06 2024-09-16T00:08:11.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622137+0000", 2024-09-16T00:08:11.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000135103 2024-09-16T00:08:11.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622143+0000", 2024-09-16T00:08:11.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9200000000000001e-06 2024-09-16T00:08:11.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622207+0000", 2024-09-16T00:08:11.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4282000000000006e-05 2024-09-16T00:08:11.081 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.082 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.082 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing_reply(2.14 epoch 22/19 rep_tid 6 lcod 22'5)", 2024-09-16T00:08:11.082 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:11.082 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.375383655999997, 2024-09-16T00:08:11.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029688000000000002, 2024-09-16T00:08:11.082 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.082 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.082 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:11.082 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:11.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:11.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622017+0000", 2024-09-16T00:08:11.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3200000000000001e-07 2024-09-16T00:08:11.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:08:11.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6240000000000002e-06 2024-09-16T00:08:11.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622026+0000", 2024-09-16T00:08:11.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.708e-06 2024-09-16T00:08:11.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622032+0000", 2024-09-16T00:08:11.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4699999999999999e-06 2024-09-16T00:08:11.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622230+0000", 2024-09-16T00:08:11.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019783800000000001 2024-09-16T00:08:11.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622313+0000", 2024-09-16T00:08:11.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2908e-05 2024-09-16T00:08:11.085 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.086 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.086 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.086 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:11.086 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:11.086 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.374938161000003, 2024-09-16T00:08:11.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044651079999999998, 2024-09-16T00:08:11.086 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.086 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.086 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.086 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:11.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:11.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622463+0000", 2024-09-16T00:08:11.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5349999999999999e-06 2024-09-16T00:08:11.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622470+0000", 2024-09-16T00:08:11.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7050000000000002e-06 2024-09-16T00:08:11.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622485+0000", 2024-09-16T00:08:11.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4978e-05 2024-09-16T00:08:11.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622493+0000", 2024-09-16T00:08:11.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2139999999999996e-06 2024-09-16T00:08:11.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622546+0000", 2024-09-16T00:08:11.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:08:11.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622595+0000", 2024-09-16T00:08:11.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9112e-05 2024-09-16T00:08:11.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:11.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622677+0000", 2024-09-16T00:08:11.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2332999999999999e-05 2024-09-16T00:08:11.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.623515+0000", 2024-09-16T00:08:11.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000837615 2024-09-16T00:08:11.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626867+0000", 2024-09-16T00:08:11.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033524819999999999 2024-09-16T00:08:11.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626902+0000", 2024-09-16T00:08:11.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4310000000000002e-05 2024-09-16T00:08:11.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626927+0000", 2024-09-16T00:08:11.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5128999999999999e-05 2024-09-16T00:08:11.091 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.091 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.091 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.092 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:39 2.6 e22/19 2:654134d2:::mds0_openfiles.0:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:11.092 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:11.092 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.356308884000001, 2024-09-16T00:08:11.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082120900000000004, 2024-09-16T00:08:11.092 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.092 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.092 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.092 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:11.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:11.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641092+0000", 2024-09-16T00:08:11.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.141e-06 2024-09-16T00:08:11.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641102+0000", 2024-09-16T00:08:11.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5990000000000008e-06 2024-09-16T00:08:11.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641116+0000", 2024-09-16T00:08:11.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4667999999999999e-05 2024-09-16T00:08:11.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641125+0000", 2024-09-16T00:08:11.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0849999999999997e-06 2024-09-16T00:08:11.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641188+0000", 2024-09-16T00:08:11.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3119000000000003e-05 2024-09-16T00:08:11.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641209+0000", 2024-09-16T00:08:11.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0897e-05 2024-09-16T00:08:11.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641874+0000", 2024-09-16T00:08:11.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00066517200000000001 2024-09-16T00:08:11.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641912+0000", 2024-09-16T00:08:11.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8528000000000001e-05 2024-09-16T00:08:11.100 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.100 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.100 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.100 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:40 2.14 2:2e2fa760:::200.00000001:head [write 940~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:11.100 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:11.100 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.158072768, 2024-09-16T00:08:11.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016643039999999999, 2024-09-16T00:08:11.100 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.100 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.101 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.101 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:11.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:11.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839329+0000", 2024-09-16T00:08:11.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2330000000000001e-06 2024-09-16T00:08:11.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839343+0000", 2024-09-16T00:08:11.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3634e-05 2024-09-16T00:08:11.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839362+0000", 2024-09-16T00:08:11.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9321999999999999e-05 2024-09-16T00:08:11.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839375+0000", 2024-09-16T00:08:11.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2945000000000001e-05 2024-09-16T00:08:11.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839462+0000", 2024-09-16T00:08:11.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6371000000000004e-05 2024-09-16T00:08:11.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839547+0000", 2024-09-16T00:08:11.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4988999999999996e-05 2024-09-16T00:08:11.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:11.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839657+0000", 2024-09-16T00:08:11.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110501 2024-09-16T00:08:11.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840598+0000", 2024-09-16T00:08:11.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00094087100000000003 2024-09-16T00:08:11.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840918+0000", 2024-09-16T00:08:11.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031968799999999999 2024-09-16T00:08:11.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840955+0000", 2024-09-16T00:08:11.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7360999999999997e-05 2024-09-16T00:08:11.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840991+0000", 2024-09-16T00:08:11.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6389e-05 2024-09-16T00:08:11.106 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.106 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.106 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.106 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:41 2.14 2:2e2fa760:::200.00000001:head [write 1383~2831 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:11.106 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:11.106 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.884419322999999, 2024-09-16T00:08:11.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017075809999999999, 2024-09-16T00:08:11.106 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.106 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.106 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.107 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:11.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:11.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112983+0000", 2024-09-16T00:08:11.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2560000000000001e-06 2024-09-16T00:08:11.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112993+0000", 2024-09-16T00:08:11.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0423e-05 2024-09-16T00:08:11.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113010+0000", 2024-09-16T00:08:11.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7169000000000001e-05 2024-09-16T00:08:11.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113023+0000", 2024-09-16T00:08:11.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2111e-05 2024-09-16T00:08:11.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113099+0000", 2024-09-16T00:08:11.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6798000000000006e-05 2024-09-16T00:08:11.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113174+0000", 2024-09-16T00:08:11.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.47e-05 2024-09-16T00:08:11.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:11.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113297+0000", 2024-09-16T00:08:11.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012250799999999999 2024-09-16T00:08:11.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114284+0000", 2024-09-16T00:08:11.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00098720399999999999 2024-09-16T00:08:11.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114604+0000", 2024-09-16T00:08:11.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031988699999999998 2024-09-16T00:08:11.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114644+0000", 2024-09-16T00:08:11.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9919999999999997e-05 2024-09-16T00:08:11.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114688+0000", 2024-09-16T00:08:11.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4604999999999997e-05 2024-09-16T00:08:11.112 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.112 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.112 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.112 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.112 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.112 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:11.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.071+0000 7f4029ee4640 1 -- 172.21.15.139:0/1049609168 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f40240c9b60 con 0x7f4024070c60 2024-09-16T00:08:11.114 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:08:11.114 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:11.113+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:11.214 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:11.213+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:11.259 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:11.259 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:11.259 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:11.260 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:11.269 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:08:11.315 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:11.315+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:11.334 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:08:11.372 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:11.372 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:11.381 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:08:11.422 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:11.422+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:11.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.432+0000 7f98953d8640 1 Processor -- start 2024-09-16T00:08:11.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.432+0000 7f98953d8640 1 -- start start 2024-09-16T00:08:11.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.433+0000 7f98953d8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9890071a40 0x7f989015c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.433+0000 7f98953d8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98900c74f0 con 0x7f9890071660 2024-09-16T00:08:11.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.433+0000 7f98953d8640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98900c77d0 con 0x7f98900cac90 2024-09-16T00:08:11.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.433+0000 7f98953d8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f989015c000 con 0x7f9890071a40 2024-09-16T00:08:11.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.433+0000 7f988e7fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9890071a40 0x7f989015c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.433+0000 7f988e7fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9890071a40 0x7f989015c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35878/0 (socket says 172.21.15.139:35878) 2024-09-16T00:08:11.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.433+0000 7f988e7fc640 1 -- 172.21.15.139:0/649262300 learned_addr learned my addr 172.21.15.139:0/649262300 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:11.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.433+0000 7f988e7fc640 1 -- 172.21.15.139:0/649262300 >> v1:172.21.15.164:6789/0 conn(0x7f98900cac90 legacy=0x7f9890152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.433+0000 7f988e7fc640 1 -- 172.21.15.139:0/649262300 >> v1:172.21.15.139:6789/0 conn(0x7f9890071660 legacy=0x7f98900cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.433+0000 7f988dffb640 1 -- 172.21.15.139:0/649262300 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 15515082 0 0) 0x7f98900c74f0 con 0x7f9890071660 2024-09-16T00:08:11.435 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.433+0000 7f988e7fc640 1 -- 172.21.15.139:0/649262300 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98900c7ab0 con 0x7f9890071a40 2024-09-16T00:08:11.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.435+0000 7f988e7fc640 1 --2- 172.21.15.139:0/649262300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9890071a40 0x7f989015c490 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9884009870 tx=0x7f988402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bfeb5736e8d630 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:11.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.435+0000 7f988dffb640 1 -- 172.21.15.139:0/649262300 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884037070 con 0x7f9890071a40 2024-09-16T00:08:11.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.435+0000 7f988dffb640 1 -- 172.21.15.139:0/649262300 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9884032440 con 0x7f9890071a40 2024-09-16T00:08:11.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.435+0000 7f988dffb640 1 -- 172.21.15.139:0/649262300 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884032810 con 0x7f9890071a40 2024-09-16T00:08:11.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.435+0000 7f98953d8640 1 -- 172.21.15.139:0/649262300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9890071a40 msgr2=0x7f989015c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.435+0000 7f98953d8640 1 --2- 172.21.15.139:0/649262300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9890071a40 0x7f989015c490 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9884009870 tx=0x7f988402a270 comp rx=0 tx=0).stop 2024-09-16T00:08:11.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.436+0000 7f98953d8640 1 -- 172.21.15.139:0/649262300 shutdown_connections 2024-09-16T00:08:11.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.436+0000 7f98953d8640 1 --2- 172.21.15.139:0/649262300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9890071a40 0x7f989015c490 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.436+0000 7f98953d8640 1 -- 172.21.15.139:0/649262300 >> 172.21.15.139:0/649262300 conn(0x7f9890073950 msgr2=0x7f989006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:11.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.436+0000 7f98953d8640 1 -- 172.21.15.139:0/649262300 shutdown_connections 2024-09-16T00:08:11.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.436+0000 7f98953d8640 1 -- 172.21.15.139:0/649262300 wait complete. 2024-09-16T00:08:11.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.437+0000 7f98953d8640 1 Processor -- start 2024-09-16T00:08:11.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.438+0000 7f98953d8640 1 -- start start 2024-09-16T00:08:11.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f98953d8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9890071660 0x7f9890149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f98953d8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9890071a40 0x7f98901eee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f98953d8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98900cac90 0x7f989014b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f98953d8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f98900c8ba0 con 0x7f9890071a40 2024-09-16T00:08:11.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f98953d8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f98900c8070 con 0x7f98900cac90 2024-09-16T00:08:11.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f98953d8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9890152e80 con 0x7f9890071660 2024-09-16T00:08:11.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f988e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9890071a40 0x7f98901eee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f988e7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9890071a40 0x7f98901eee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35880/0 (socket says 172.21.15.139:35880) 2024-09-16T00:08:11.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f988e7fc640 1 -- 172.21.15.139:0/3441475070 learned_addr learned my addr 172.21.15.139:0/3441475070 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:11.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f988effd640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9890071660 0x7f9890149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f988f7fe640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98900cac90 0x7f989014b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f988e7fc640 1 -- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9890071660 msgr2=0x7f9890149c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f988e7fc640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9890071660 0x7f9890149c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f988e7fc640 1 -- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98900cac90 msgr2=0x7f989014b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f988e7fc640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98900cac90 0x7f989014b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f988e7fc640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98901ee3e0 con 0x7f9890071a40 2024-09-16T00:08:11.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f988e7fc640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9890071a40 0x7f98901eee60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9884009870 tx=0x7f98840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:11.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884041020 con 0x7f9890071a40 2024-09-16T00:08:11.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9884030970 con 0x7f9890071a40 2024-09-16T00:08:11.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9884037040 con 0x7f9890071a40 2024-09-16T00:08:11.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.439+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989014a640 con 0x7f9890071a40 2024-09-16T00:08:11.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.440+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98901ee660 con 0x7f9890071a40 2024-09-16T00:08:11.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.440+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f98900b3600 con 0x7f9890071a40 2024-09-16T00:08:11.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.441+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9884044020 con 0x7f9890071a40 2024-09-16T00:08:11.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.442+0000 7f98777fe640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f987c06c2b0 0x7f987c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.442+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5738+0+0 (secure 0 0 0) 0x7f98840abce0 con 0x7f9890071a40 2024-09-16T00:08:11.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.442+0000 7f988effd640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f987c06c2b0 0x7f987c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.442+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v30) ==== 54+0+16 (secure 0 0 0) 0x7f98900b3600 con 0x7f9890071a40 2024-09-16T00:08:11.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.442+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f989015b860 0x7f98900c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.442+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f989015b4a0 con 0x7f989015b860 2024-09-16T00:08:11.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.442+0000 7f988effd640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f987c06c2b0 0x7f987c06e770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9880000f60 tx=0x7f9880009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:11.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.442+0000 7f988f7fe640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f989015b860 0x7f98900c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.443+0000 7f988f7fe640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f989015b860 0x7f98900c38e0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:11.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.443+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f989015b4a0 con 0x7f989015b860 2024-09-16T00:08:11.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.457+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9890152770 con 0x7f989015b860 2024-09-16T00:08:11.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.460+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9890152770 con 0x7f989015b860 2024-09-16T00:08:11.461 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:11.461 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:11.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.460+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f98901ee8a0 0x7f98901f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.460+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f98900c4330 con 0x7f98901ee8a0 2024-09-16T00:08:11.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.460+0000 7f988effd640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f98901ee8a0 0x7f98901f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.460+0000 7f988effd640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f98901ee8a0 0x7f98901f1950 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:11.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.461+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98900c4330 con 0x7f98901ee8a0 2024-09-16T00:08:11.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.475+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98900c90f0 con 0x7f98901ee8a0 2024-09-16T00:08:11.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.477+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98900c90f0 con 0x7f98901ee8a0 2024-09-16T00:08:11.478 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:11.479 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:11.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.477+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f98900c31f0 0x7f98901f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.477+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f9890150200 con 0x7f98900c31f0 2024-09-16T00:08:11.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.477+0000 7f988e7fc640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f98900c31f0 0x7f98901f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.478+0000 7f988e7fc640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f98900c31f0 0x7f98901f23a0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:11.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.478+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9890150200 con 0x7f98900c31f0 2024-09-16T00:08:11.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.492+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98901506e0 con 0x7f98900c31f0 2024-09-16T00:08:11.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.494+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98901506e0 con 0x7f98900c31f0 2024-09-16T00:08:11.495 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:11.495 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:11.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.494+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f9890062630 0x7f9890062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.494+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f9890150fb0 con 0x7f9890062630 2024-09-16T00:08:11.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.494+0000 7f988f7fe640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f9890062630 0x7f9890062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.495+0000 7f988f7fe640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f9890062630 0x7f9890062a30 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:11.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.495+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9890150fb0 con 0x7f9890062630 2024-09-16T00:08:11.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.508+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9890150fb0 con 0x7f9890062630 2024-09-16T00:08:11.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.510+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9890150fb0 con 0x7f9890062630 2024-09-16T00:08:11.511 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:11.512 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:11.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.510+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f989002c640 0x7f989002cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.510+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f9890150fb0 con 0x7f989002c640 2024-09-16T00:08:11.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.510+0000 7f988effd640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f989002c640 0x7f989002cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.511+0000 7f988effd640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f989002c640 0x7f989002cab0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:11.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.511+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9890150fb0 con 0x7f989002c640 2024-09-16T00:08:11.517 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:11.516+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:11.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.525+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9890150fb0 con 0x7f989002c640 2024-09-16T00:08:11.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.527+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9890150fb0 con 0x7f989002c640 2024-09-16T00:08:11.529 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:11.530 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.445234374999998, 2024-09-16T00:08:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:08:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:08:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:08:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:08:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:08:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:08:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:08:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:08:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:08:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:08:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:08:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:08:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:08:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:08:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:08:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:08:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:08:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:08:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:08:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:08:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:08:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:08:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:08:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:08:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:08:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:08:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:08:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:08:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:08:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.365702745, 2024-09-16T00:08:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093071190000000008, 2024-09-16T00:08:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 106 2024-09-16T00:08:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160362+0000", 2024-09-16T00:08:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160438+0000", 2024-09-16T00:08:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6711000000000005e-05 2024-09-16T00:08:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160444+0000", 2024-09-16T00:08:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0530000000000003e-06 2024-09-16T00:08:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160449+0000", 2024-09-16T00:08:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2189999999999998e-06 2024-09-16T00:08:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161270+0000", 2024-09-16T00:08:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082099199999999999 2024-09-16T00:08:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161348+0000", 2024-09-16T00:08:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8361000000000004e-05 2024-09-16T00:08:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161472+0000", 2024-09-16T00:08:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123523 2024-09-16T00:08:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163461+0000", 2024-09-16T00:08:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019891560000000002 2024-09-16T00:08:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169418+0000", 2024-09-16T00:08:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059568490000000002 2024-09-16T00:08:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169570+0000", 2024-09-16T00:08:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152461 2024-09-16T00:08:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169668+0000", 2024-09-16T00:08:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T00:08:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.365606974000002, 2024-09-16T00:08:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092931679999999992, 2024-09-16T00:08:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107 2024-09-16T00:08:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0699999999999998e-07 2024-09-16T00:08:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160459+0000", 2024-09-16T00:08:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0370000000000001e-06 2024-09-16T00:08:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160463+0000", 2024-09-16T00:08:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999999e-06 2024-09-16T00:08:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160466+0000", 2024-09-16T00:08:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.644e-06 2024-09-16T00:08:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161873+0000", 2024-09-16T00:08:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014074840000000001 2024-09-16T00:08:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161949+0000", 2024-09-16T00:08:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5749999999999998e-05 2024-09-16T00:08:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162069+0000", 2024-09-16T00:08:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120257 2024-09-16T00:08:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163506+0000", 2024-09-16T00:08:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001436344 2024-09-16T00:08:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169704+0000", 2024-09-16T00:08:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061982829999999997 2024-09-16T00:08:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169730+0000", 2024-09-16T00:08:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6060000000000001e-05 2024-09-16T00:08:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169750+0000", 2024-09-16T00:08:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0092e-05 2024-09-16T00:08:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.365497662000003, 2024-09-16T00:08:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092280039999999997, 2024-09-16T00:08:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.551 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 108 2024-09-16T00:08:11.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.551 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:11.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160567+0000", 2024-09-16T00:08:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7300000000000001e-07 2024-09-16T00:08:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160569+0000", 2024-09-16T00:08:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5299999999999999e-06 2024-09-16T00:08:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160574+0000", 2024-09-16T00:08:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-16T00:08:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160577+0000", 2024-09-16T00:08:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.01e-06 2024-09-16T00:08:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162518+0000", 2024-09-16T00:08:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001940703 2024-09-16T00:08:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162571+0000", 2024-09-16T00:08:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2670000000000002e-05 2024-09-16T00:08:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162647+0000", 2024-09-16T00:08:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6778000000000002e-05 2024-09-16T00:08:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163544+0000", 2024-09-16T00:08:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089702599999999999 2024-09-16T00:08:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169754+0000", 2024-09-16T00:08:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062091739999999996 2024-09-16T00:08:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169776+0000", 2024-09-16T00:08:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2391999999999998e-05 2024-09-16T00:08:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169794+0000", 2024-09-16T00:08:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8318999999999999e-05 2024-09-16T00:08:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.365449112, 2024-09-16T00:08:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092220679999999999, 2024-09-16T00:08:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 109 2024-09-16T00:08:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9900000000000001e-07 2024-09-16T00:08:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160707+0000", 2024-09-16T00:08:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1621999999999998e-05 2024-09-16T00:08:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160713+0000", 2024-09-16T00:08:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.596e-06 2024-09-16T00:08:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160717+0000", 2024-09-16T00:08:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5829999999999998e-06 2024-09-16T00:08:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162869+0000", 2024-09-16T00:08:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021516310000000002 2024-09-16T00:08:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162946+0000", 2024-09-16T00:08:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7696999999999998e-05 2024-09-16T00:08:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163065+0000", 2024-09-16T00:08:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118581 2024-09-16T00:08:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164431+0000", 2024-09-16T00:08:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001366003 2024-09-16T00:08:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169797+0000", 2024-09-16T00:08:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005366244 2024-09-16T00:08:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169819+0000", 2024-09-16T00:08:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2036999999999999e-05 2024-09-16T00:08:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169837+0000", 2024-09-16T00:08:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7575e-05 2024-09-16T00:08:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.365339763000001, 2024-09-16T00:08:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092871919999999997, 2024-09-16T00:08:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 110 2024-09-16T00:08:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:08:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160728+0000", 2024-09-16T00:08:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0620000000000001e-06 2024-09-16T00:08:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160731+0000", 2024-09-16T00:08:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6500000000000002e-06 2024-09-16T00:08:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160734+0000", 2024-09-16T00:08:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.762e-06 2024-09-16T00:08:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163580+0000", 2024-09-16T00:08:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028464549999999999 2024-09-16T00:08:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163657+0000", 2024-09-16T00:08:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6695999999999999e-05 2024-09-16T00:08:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163798+0000", 2024-09-16T00:08:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014055 2024-09-16T00:08:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165483+0000", 2024-09-16T00:08:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016853090000000001 2024-09-16T00:08:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169970+0000", 2024-09-16T00:08:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00448709 2024-09-16T00:08:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169994+0000", 2024-09-16T00:08:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3575999999999999e-05 2024-09-16T00:08:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170011+0000", 2024-09-16T00:08:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7702000000000001e-05 2024-09-16T00:08:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.365326093999997, 2024-09-16T00:08:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093174859999999998, 2024-09-16T00:08:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 111 2024-09-16T00:08:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1e-07 2024-09-16T00:08:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160741+0000", 2024-09-16T00:08:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6110000000000001e-06 2024-09-16T00:08:11.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160744+0000", 2024-09-16T00:08:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3299999999999999e-06 2024-09-16T00:08:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160746+0000", 2024-09-16T00:08:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5469999999999999e-06 2024-09-16T00:08:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164069+0000", 2024-09-16T00:08:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033222709999999999 2024-09-16T00:08:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164113+0000", 2024-09-16T00:08:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4212999999999998e-05 2024-09-16T00:08:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164187+0000", 2024-09-16T00:08:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4136999999999998e-05 2024-09-16T00:08:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166429+0000", 2024-09-16T00:08:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022423180000000001 2024-09-16T00:08:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170016+0000", 2024-09-16T00:08:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035869449999999998 2024-09-16T00:08:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170038+0000", 2024-09-16T00:08:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2053e-05 2024-09-16T00:08:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170055+0000", 2024-09-16T00:08:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6850999999999998e-05 2024-09-16T00:08:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.365312379000002, 2024-09-16T00:08:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093480279999999995, 2024-09-16T00:08:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.576 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.576 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.576 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.576 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.576 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 112 2024-09-16T00:08:11.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.576 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:11.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:11.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160752+0000", 2024-09-16T00:08:11.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2700000000000001e-07 2024-09-16T00:08:11.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160808+0000", 2024-09-16T00:08:11.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5797999999999997e-05 2024-09-16T00:08:11.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160814+0000", 2024-09-16T00:08:11.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5980000000000002e-06 2024-09-16T00:08:11.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160819+0000", 2024-09-16T00:08:11.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8960000000000002e-06 2024-09-16T00:08:11.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164468+0000", 2024-09-16T00:08:11.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003649476 2024-09-16T00:08:11.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164525+0000", 2024-09-16T00:08:11.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6227999999999998e-05 2024-09-16T00:08:11.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164595+0000", 2024-09-16T00:08:11.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759000000000005e-05 2024-09-16T00:08:11.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167237+0000", 2024-09-16T00:08:11.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026410470000000001 2024-09-16T00:08:11.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170060+0000", 2024-09-16T00:08:11.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028234969999999999 2024-09-16T00:08:11.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170081+0000", 2024-09-16T00:08:11.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1242000000000001e-05 2024-09-16T00:08:11.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170100+0000", 2024-09-16T00:08:11.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8260000000000001e-05 2024-09-16T00:08:11.581 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.581 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.581 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.581 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:11.581 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:11.581 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.365235284000001, 2024-09-16T00:08:11.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093139409999999992, 2024-09-16T00:08:11.581 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.581 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.582 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.582 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.582 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.582 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.582 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 113 2024-09-16T00:08:11.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.582 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:11.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:11.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:11.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-07 2024-09-16T00:08:11.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160833+0000", 2024-09-16T00:08:11.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.557e-06 2024-09-16T00:08:11.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160839+0000", 2024-09-16T00:08:11.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3829999999999997e-06 2024-09-16T00:08:11.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160844+0000", 2024-09-16T00:08:11.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6909999999999996e-06 2024-09-16T00:08:11.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165008+0000", 2024-09-16T00:08:11.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041642129999999999 2024-09-16T00:08:11.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165085+0000", 2024-09-16T00:08:11.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6651999999999997e-05 2024-09-16T00:08:11.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165205+0000", 2024-09-16T00:08:11.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012045900000000001 2024-09-16T00:08:11.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167283+0000", 2024-09-16T00:08:11.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020782909999999999 2024-09-16T00:08:11.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170104+0000", 2024-09-16T00:08:11.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028207950000000001 2024-09-16T00:08:11.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170125+0000", 2024-09-16T00:08:11.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0852e-05 2024-09-16T00:08:11.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170143+0000", 2024-09-16T00:08:11.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7467000000000001e-05 2024-09-16T00:08:11.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.587 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.587 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:11.587 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:11.588 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.365188801999999, 2024-09-16T00:08:11.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093126960000000005, 2024-09-16T00:08:11.588 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.588 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.588 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.588 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-16T00:08:11.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.588 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:11.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:11.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160876+0000", 2024-09-16T00:08:11.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5700000000000002e-07 2024-09-16T00:08:11.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160880+0000", 2024-09-16T00:08:11.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5539999999999999e-06 2024-09-16T00:08:11.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160888+0000", 2024-09-16T00:08:11.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.157e-06 2024-09-16T00:08:11.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160894+0000", 2024-09-16T00:08:11.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7690000000000002e-06 2024-09-16T00:08:11.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165519+0000", 2024-09-16T00:08:11.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046249539999999997 2024-09-16T00:08:11.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165593+0000", 2024-09-16T00:08:11.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4039e-05 2024-09-16T00:08:11.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165715+0000", 2024-09-16T00:08:11.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012131899999999999 2024-09-16T00:08:11.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167309+0000", 2024-09-16T00:08:11.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001594011 2024-09-16T00:08:11.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170147+0000", 2024-09-16T00:08:11.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028387130000000001 2024-09-16T00:08:11.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170170+0000", 2024-09-16T00:08:11.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2517000000000001e-05 2024-09-16T00:08:11.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170188+0000", 2024-09-16T00:08:11.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8006e-05 2024-09-16T00:08:11.593 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.593 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.593 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.593 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:11.594 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:11.594 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.365147774, 2024-09-16T00:08:11.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093140040000000007, 2024-09-16T00:08:11.594 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.594 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.594 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.594 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.594 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.594 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.594 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-16T00:08:11.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.594 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:11.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:11.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160917+0000", 2024-09-16T00:08:11.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7599999999999996e-07 2024-09-16T00:08:11.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160992+0000", 2024-09-16T00:08:11.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4760999999999998e-05 2024-09-16T00:08:11.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161000+0000", 2024-09-16T00:08:11.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9829999999999996e-06 2024-09-16T00:08:11.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161007+0000", 2024-09-16T00:08:11.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4909999999999999e-06 2024-09-16T00:08:11.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165983+0000", 2024-09-16T00:08:11.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049755399999999997 2024-09-16T00:08:11.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166027+0000", 2024-09-16T00:08:11.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4109000000000001e-05 2024-09-16T00:08:11.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166100+0000", 2024-09-16T00:08:11.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3360000000000002e-05 2024-09-16T00:08:11.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169698+0000", 2024-09-16T00:08:11.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597795 2024-09-16T00:08:11.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170192+0000", 2024-09-16T00:08:11.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049447300000000005 2024-09-16T00:08:11.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170214+0000", 2024-09-16T00:08:11.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1546e-05 2024-09-16T00:08:11.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170230+0000", 2024-09-16T00:08:11.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6169999999999999e-05 2024-09-16T00:08:11.599 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.599 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.599 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.600 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:11.600 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:11.600 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.365038044999999, 2024-09-16T00:08:11.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092457029999999992, 2024-09-16T00:08:11.600 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.600 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.600 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.600 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.600 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.600 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.600 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-16T00:08:11.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.600 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:11.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:11.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:11.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7899999999999999e-07 2024-09-16T00:08:11.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161029+0000", 2024-09-16T00:08:11.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6630000000000002e-06 2024-09-16T00:08:11.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161036+0000", 2024-09-16T00:08:11.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T00:08:11.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161041+0000", 2024-09-16T00:08:11.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:08:11.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166468+0000", 2024-09-16T00:08:11.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054266050000000001 2024-09-16T00:08:11.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166511+0000", 2024-09-16T00:08:11.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3869999999999998e-05 2024-09-16T00:08:11.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166606+0000", 2024-09-16T00:08:11.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4437000000000004e-05 2024-09-16T00:08:11.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169955+0000", 2024-09-16T00:08:11.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033494810000000001 2024-09-16T00:08:11.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170234+0000", 2024-09-16T00:08:11.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027900899999999998 2024-09-16T00:08:11.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170255+0000", 2024-09-16T00:08:11.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0537999999999999e-05 2024-09-16T00:08:11.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170272+0000", 2024-09-16T00:08:11.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6611000000000001e-05 2024-09-16T00:08:11.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.605 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.605 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.606 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:11.606 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:11.606 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.365012899999996, 2024-09-16T00:08:11.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093511010000000006, 2024-09-16T00:08:11.606 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.606 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.606 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.606 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.606 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-16T00:08:11.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.606 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:11.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:11.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:11.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.63e-07 2024-09-16T00:08:11.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161055+0000", 2024-09-16T00:08:11.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6770000000000001e-06 2024-09-16T00:08:11.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161063+0000", 2024-09-16T00:08:11.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6469999999999998e-06 2024-09-16T00:08:11.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161068+0000", 2024-09-16T00:08:11.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.746e-06 2024-09-16T00:08:11.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166817+0000", 2024-09-16T00:08:11.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057494959999999998 2024-09-16T00:08:11.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166893+0000", 2024-09-16T00:08:11.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6210000000000004e-05 2024-09-16T00:08:11.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167019+0000", 2024-09-16T00:08:11.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125585 2024-09-16T00:08:11.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170275+0000", 2024-09-16T00:08:11.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032563620000000001 2024-09-16T00:08:11.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170309+0000", 2024-09-16T00:08:11.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3865999999999997e-05 2024-09-16T00:08:11.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170362+0000", 2024-09-16T00:08:11.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3094999999999999e-05 2024-09-16T00:08:11.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170402+0000", 2024-09-16T00:08:11.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9953999999999997e-05 2024-09-16T00:08:11.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.611 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.612 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:11.612 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:11.612 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.916280815, 2024-09-16T00:08:11.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097364389999999995, 2024-09-16T00:08:11.612 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.612 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.612 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.612 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138 2024-09-16T00:08:11.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.612 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:11.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:11.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609784+0000", 2024-09-16T00:08:11.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T00:08:11.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609787+0000", 2024-09-16T00:08:11.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280000000000001e-06 2024-09-16T00:08:11.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609794+0000", 2024-09-16T00:08:11.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6409999999999996e-06 2024-09-16T00:08:11.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609799+0000", 2024-09-16T00:08:11.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9359999999999998e-06 2024-09-16T00:08:11.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615759+0000", 2024-09-16T00:08:11.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059599910000000004 2024-09-16T00:08:11.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615806+0000", 2024-09-16T00:08:11.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7290000000000003e-05 2024-09-16T00:08:11.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615877+0000", 2024-09-16T00:08:11.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1000000000000005e-05 2024-09-16T00:08:11.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618961+0000", 2024-09-16T00:08:11.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030841390000000001 2024-09-16T00:08:11.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619436+0000", 2024-09-16T00:08:11.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047452799999999999 2024-09-16T00:08:11.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619482+0000", 2024-09-16T00:08:11.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5689000000000003e-05 2024-09-16T00:08:11.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619519+0000", 2024-09-16T00:08:11.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7938000000000002e-05 2024-09-16T00:08:11.617 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.617 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.617 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.618 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:11.618 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:11.618 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.916257100999999, 2024-09-16T00:08:11.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011812954000000001, 2024-09-16T00:08:11.618 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.618 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.618 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.618 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.618 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139 2024-09-16T00:08:11.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.618 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:11.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:11.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:11.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999998e-07 2024-09-16T00:08:11.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609810+0000", 2024-09-16T00:08:11.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2100000000000002e-06 2024-09-16T00:08:11.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609817+0000", 2024-09-16T00:08:11.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.28e-06 2024-09-16T00:08:11.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609825+0000", 2024-09-16T00:08:11.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:11.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616116+0000", 2024-09-16T00:08:11.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006290543 2024-09-16T00:08:11.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616187+0000", 2024-09-16T00:08:11.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1830999999999997e-05 2024-09-16T00:08:11.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616298+0000", 2024-09-16T00:08:11.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011044699999999999 2024-09-16T00:08:11.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619559+0000", 2024-09-16T00:08:11.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032614710000000002 2024-09-16T00:08:11.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621562+0000", 2024-09-16T00:08:11.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020028680000000001 2024-09-16T00:08:11.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621597+0000", 2024-09-16T00:08:11.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4625000000000001e-05 2024-09-16T00:08:11.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621620+0000", 2024-09-16T00:08:11.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2864e-05 2024-09-16T00:08:11.623 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.623 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.623 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.624 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:11.624 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:11.624 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.916229305000002, 2024-09-16T00:08:11.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011828211999999999, 2024-09-16T00:08:11.624 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.624 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.624 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.624 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 140 2024-09-16T00:08:11.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.625 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:11.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:11.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:11.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T00:08:11.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:11.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961000000000003e-05 2024-09-16T00:08:11.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609892+0000", 2024-09-16T00:08:11.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0516000000000001e-05 2024-09-16T00:08:11.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609898+0000", 2024-09-16T00:08:11.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0449999999999997e-06 2024-09-16T00:08:11.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616797+0000", 2024-09-16T00:08:11.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068998840000000002 2024-09-16T00:08:11.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616844+0000", 2024-09-16T00:08:11.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6501000000000001e-05 2024-09-16T00:08:11.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616935+0000", 2024-09-16T00:08:11.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1222000000000002e-05 2024-09-16T00:08:11.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619808+0000", 2024-09-16T00:08:11.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873198 2024-09-16T00:08:11.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621624+0000", 2024-09-16T00:08:11.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001815933 2024-09-16T00:08:11.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621646+0000", 2024-09-16T00:08:11.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1622999999999999e-05 2024-09-16T00:08:11.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621663+0000", 2024-09-16T00:08:11.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6854000000000002e-05 2024-09-16T00:08:11.629 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.629 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.630 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.630 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:11.630 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:11.630 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.916143495999997, 2024-09-16T00:08:11.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014331501999999999, 2024-09-16T00:08:11.630 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.630 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.630 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.631 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-16T00:08:11.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.631 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:11.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:11.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609921+0000", 2024-09-16T00:08:11.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9599999999999999e-07 2024-09-16T00:08:11.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609925+0000", 2024-09-16T00:08:11.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T00:08:11.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609932+0000", 2024-09-16T00:08:11.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8240000000000002e-06 2024-09-16T00:08:11.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609937+0000", 2024-09-16T00:08:11.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1070000000000004e-06 2024-09-16T00:08:11.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618987+0000", 2024-09-16T00:08:11.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090502719999999998 2024-09-16T00:08:11.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619064+0000", 2024-09-16T00:08:11.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6868999999999999e-05 2024-09-16T00:08:11.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619183+0000", 2024-09-16T00:08:11.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011961 2024-09-16T00:08:11.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.620052+0000", 2024-09-16T00:08:11.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086822799999999997 2024-09-16T00:08:11.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624197+0000", 2024-09-16T00:08:11.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041454930000000001 2024-09-16T00:08:11.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624229+0000", 2024-09-16T00:08:11.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1758999999999999e-05 2024-09-16T00:08:11.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624252+0000", 2024-09-16T00:08:11.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:08:11.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.636 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.636 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:11.636 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:11.637 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.443377687999998, 2024-09-16T00:08:11.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010093276999999999, 2024-09-16T00:08:11.637 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.637 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.637 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.637 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.637 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.637 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.637 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 157 2024-09-16T00:08:11.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.637 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:11.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:11.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:11.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4699999999999998e-07 2024-09-16T00:08:11.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082688+0000", 2024-09-16T00:08:11.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0209999999999998e-06 2024-09-16T00:08:11.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082692+0000", 2024-09-16T00:08:11.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6890000000000002e-06 2024-09-16T00:08:11.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082695+0000", 2024-09-16T00:08:11.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5339999999999998e-06 2024-09-16T00:08:11.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085303+0000", 2024-09-16T00:08:11.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026084789999999999 2024-09-16T00:08:11.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085374+0000", 2024-09-16T00:08:11.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1317999999999997e-05 2024-09-16T00:08:11.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085506+0000", 2024-09-16T00:08:11.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013118 2024-09-16T00:08:11.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086928+0000", 2024-09-16T00:08:11.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014224559999999999 2024-09-16T00:08:11.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092665+0000", 2024-09-16T00:08:11.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057369830000000002 2024-09-16T00:08:11.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092727+0000", 2024-09-16T00:08:11.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1810000000000006e-05 2024-09-16T00:08:11.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092779+0000", 2024-09-16T00:08:11.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2559999999999998e-05 2024-09-16T00:08:11.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.642 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.642 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:11.642 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:11.642 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.44336552, 2024-09-16T00:08:11.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010176388, 2024-09-16T00:08:11.643 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.643 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.643 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.643 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.643 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 158 2024-09-16T00:08:11.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.643 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:11.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:11.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082699+0000", 2024-09-16T00:08:11.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.11e-07 2024-09-16T00:08:11.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082742+0000", 2024-09-16T00:08:11.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3699000000000003e-05 2024-09-16T00:08:11.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082748+0000", 2024-09-16T00:08:11.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8690000000000001e-06 2024-09-16T00:08:11.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082751+0000", 2024-09-16T00:08:11.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.942e-06 2024-09-16T00:08:11.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085881+0000", 2024-09-16T00:08:11.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031303450000000001 2024-09-16T00:08:11.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085959+0000", 2024-09-16T00:08:11.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7874999999999996e-05 2024-09-16T00:08:11.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086086+0000", 2024-09-16T00:08:11.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012685499999999999 2024-09-16T00:08:11.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087127+0000", 2024-09-16T00:08:11.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001041298 2024-09-16T00:08:11.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092790+0000", 2024-09-16T00:08:11.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056628 2024-09-16T00:08:11.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092832+0000", 2024-09-16T00:08:11.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2227000000000001e-05 2024-09-16T00:08:11.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092875+0000", 2024-09-16T00:08:11.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2267e-05 2024-09-16T00:08:11.648 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.648 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.648 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.649 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:11.649 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:11.649 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.443307482000002, 2024-09-16T00:08:11.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010199547, 2024-09-16T00:08:11.649 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:11.649 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:11.649 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:11.649 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:11.649 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:11.649 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:11.649 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159 2024-09-16T00:08:11.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.649 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082757+0000", 2024-09-16T00:08:11.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2500000000000001e-07 2024-09-16T00:08:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082759+0000", 2024-09-16T00:08:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-16T00:08:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082763+0000", 2024-09-16T00:08:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4889999999999997e-06 2024-09-16T00:08:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:11.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082766+0000", 2024-09-16T00:08:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:08:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086571+0000", 2024-09-16T00:08:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038053269999999998 2024-09-16T00:08:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086619+0000", 2024-09-16T00:08:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7924999999999999e-05 2024-09-16T00:08:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086693+0000", 2024-09-16T00:08:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000005e-05 2024-09-16T00:08:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087383+0000", 2024-09-16T00:08:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069008299999999997 2024-09-16T00:08:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092882+0000", 2024-09-16T00:08:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054987450000000002 2024-09-16T00:08:11.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092921+0000", 2024-09-16T00:08:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9388999999999998e-05 2024-09-16T00:08:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092956+0000", 2024-09-16T00:08:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4647999999999998e-05 2024-09-16T00:08:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:11.654 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:11.655 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:11.656 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:11.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.527+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f989002d500 0x7f9890204ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.527+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f989002d920 con 0x7f989002d500 2024-09-16T00:08:11.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.528+0000 7f988e7fc640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f989002d500 0x7f9890204ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.528+0000 7f988e7fc640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f989002d500 0x7f9890204ce0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:11.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.529+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f989002d920 con 0x7f989002d500 2024-09-16T00:08:11.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.543+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f989002db60 con 0x7f989002d500 2024-09-16T00:08:11.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.544+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f989002db60 con 0x7f989002d500 2024-09-16T00:08:11.657 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:11.658 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T00:08:11.658 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:08:11.658 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T00:08:11.658 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 1.000000 2024-09-16T00:08:11.658 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-16T00:08:11.659 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:11.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.545+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f9890205730 0x7f9890207b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.545+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f989002d920 con 0x7f9890205730 2024-09-16T00:08:11.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.545+0000 7f988f7fe640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f9890205730 0x7f9890207b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.545+0000 7f988f7fe640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f9890205730 0x7f9890207b20 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:11.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.546+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f989002d920 con 0x7f9890205730 2024-09-16T00:08:11.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.559+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f989002d920 con 0x7f9890205730 2024-09-16T00:08:11.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.561+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f989002d920 con 0x7f9890205730 2024-09-16T00:08:11.660 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:11.661 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:11.658+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:11.661 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:08:11.662 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:11.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.561+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9890208570 0x7f989020a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.561+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f989020d840 con 0x7f9890208570 2024-09-16T00:08:11.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.562+0000 7f988effd640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9890208570 0x7f989020a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.562+0000 7f988effd640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9890208570 0x7f989020a960 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:11.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.563+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f989020d840 con 0x7f9890208570 2024-09-16T00:08:11.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.578+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f989020d840 con 0x7f9890208570 2024-09-16T00:08:11.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.579+0000 7f98777fe640 1 -- 172.21.15.139:0/3441475070 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f989020d840 con 0x7f9890208570 2024-09-16T00:08:11.663 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:11.665 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:11.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f989015b860 msgr2=0x7f98900c38e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f989015b860 0x7f98900c38e0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f98901ee8a0 msgr2=0x7f98901f1950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f98901ee8a0 0x7f98901f1950 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f98900c31f0 msgr2=0x7f98901f23a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f98900c31f0 0x7f98901f23a0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f9890062630 msgr2=0x7f9890062a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f9890062630 0x7f9890062a30 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f989002c640 msgr2=0x7f989002cab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f989002c640 0x7f989002cab0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f989002d500 msgr2=0x7f9890204ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f989002d500 0x7f9890204ce0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f9890205730 msgr2=0x7f9890207b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f988f7fe640 1 -- 172.21.15.139:0/3441475070 reap_dead start 2024-09-16T00:08:11.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f9890205730 0x7f9890207b20 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f988f7fe640 1 -- 172.21.15.139:0/3441475070 reap_dead start 2024-09-16T00:08:11.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9890208570 msgr2=0x7f989020a960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9890208570 0x7f989020a960 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f988f7fe640 1 -- 172.21.15.139:0/3441475070 reap_dead start 2024-09-16T00:08:11.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f987c06c2b0 msgr2=0x7f987c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f987c06c2b0 0x7f987c06e770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9880000f60 tx=0x7f9880009040 comp rx=0 tx=0).stop 2024-09-16T00:08:11.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9890071a40 msgr2=0x7f98901eee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9890071a40 0x7f98901eee60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9884009870 tx=0x7f98840046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 shutdown_connections 2024-09-16T00:08:11.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f987c06c2b0 0x7f987c06e770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 --2- 172.21.15.139:0/3441475070 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9890071a40 0x7f98901eee60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 >> 172.21.15.139:0/3441475070 conn(0x7f9890073950 msgr2=0x7f98900cd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:11.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 shutdown_connections 2024-09-16T00:08:11.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.580+0000 7f98953d8640 1 -- 172.21.15.139:0/3441475070 wait complete. 2024-09-16T00:08:11.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.758+0000 7f400bfff640 1 -- 172.21.15.139:0/1049609168 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v31) ==== 91+0+0 (secure 0 0 0) 0x7f401405cbc0 con 0x7f4024070c60 2024-09-16T00:08:11.759 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is unset 2024-09-16T00:08:11.760 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:11.759+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:11.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.762+0000 7f4029ee4640 1 -- 172.21.15.139:0/1049609168 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f401006c2b0 msgr2=0x7f401006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.762+0000 7f4029ee4640 1 --2- 172.21.15.139:0/1049609168 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f401006c2b0 0x7f401006e770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4018034000 tx=0x7f401802e000 comp rx=0 tx=0).stop 2024-09-16T00:08:11.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.762+0000 7f4029ee4640 1 -- 172.21.15.139:0/1049609168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4024070c60 msgr2=0x7f40240af470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.762+0000 7f4029ee4640 1 --2- 172.21.15.139:0/1049609168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4024070c60 0x7f40240af470 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f401400c430 tx=0x7f401400c830 comp rx=0 tx=0).stop 2024-09-16T00:08:11.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.762+0000 7f4029ee4640 1 -- 172.21.15.139:0/1049609168 shutdown_connections 2024-09-16T00:08:11.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.762+0000 7f4029ee4640 1 --2- 172.21.15.139:0/1049609168 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f401006c2b0 0x7f401006e770 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.762+0000 7f4029ee4640 1 --2- 172.21.15.139:0/1049609168 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402415b490 0x7f40241548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.762+0000 7f4029ee4640 1 --2- 172.21.15.139:0/1049609168 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4024158050 0x7f402415d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.762+0000 7f4029ee4640 1 --2- 172.21.15.139:0/1049609168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4024070c60 0x7f40240af470 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.762+0000 7f4029ee4640 1 -- 172.21.15.139:0/1049609168 >> 172.21.15.139:0/1049609168 conn(0x7f402406bd00 msgr2=0x7f40240c3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:11.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.762+0000 7f4029ee4640 1 -- 172.21.15.139:0/1049609168 shutdown_connections 2024-09-16T00:08:11.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.762+0000 7f4029ee4640 1 -- 172.21.15.139:0/1049609168 wait complete. 2024-09-16T00:08:11.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.812+0000 7ff0f776d640 1 Processor -- start 2024-09-16T00:08:11.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.812+0000 7ff0f776d640 1 -- start start 2024-09-16T00:08:11.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.813+0000 7ff0f776d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff0f0068060 0x7ff0f015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.813+0000 7ff0f776d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0f00ccc90 0x7ff0f00cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.813+0000 7ff0f776d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0f00cc4b0 con 0x7ff0f0154c20 2024-09-16T00:08:11.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.813+0000 7ff0f776d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7ff0f00c5830 con 0x7ff0f0068060 2024-09-16T00:08:11.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.813+0000 7ff0f776d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f015b7c0 con 0x7ff0f00ccc90 2024-09-16T00:08:11.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.813+0000 7ff0f5509640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff0f0154c20 0x7ff0f00c5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56332/0 (socket says 172.21.15.139:56332) 2024-09-16T00:08:11.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.813+0000 7ff0f5509640 1 -- 172.21.15.139:0/60932895 learned_addr learned my addr 172.21.15.139:0/60932895 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:11.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.813+0000 7ff0f4d08640 1 --2- 172.21.15.139:0/60932895 >> v2:172.21.15.164:3300/0 conn(0x7ff0f0068060 0x7ff0f015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.813+0000 7ff0f5d0a640 1 --2- 172.21.15.139:0/60932895 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0f00ccc90 0x7ff0f00cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.813+0000 7ff0e7fff640 1 -- 172.21.15.139:0/60932895 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 381844547 0 0) 0x7ff0f00cc4b0 con 0x7ff0f0154c20 2024-09-16T00:08:11.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.813+0000 7ff0e7fff640 1 -- 172.21.15.139:0/60932895 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0dc003520 con 0x7ff0f0154c20 2024-09-16T00:08:11.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.813+0000 7ff0f5d0a640 1 -- 172.21.15.139:0/60932895 >> v2:172.21.15.164:3300/0 conn(0x7ff0f0068060 msgr2=0x7ff0f015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:11.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.813+0000 7ff0f5d0a640 1 --2- 172.21.15.139:0/60932895 >> v2:172.21.15.164:3300/0 conn(0x7ff0f0068060 0x7ff0f015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.813+0000 7ff0f5d0a640 1 -- 172.21.15.139:0/60932895 >> v1:172.21.15.139:6789/0 conn(0x7ff0f0154c20 legacy=0x7ff0f00c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.813+0000 7ff0f5d0a640 1 -- 172.21.15.139:0/60932895 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f00cc790 con 0x7ff0f00ccc90 2024-09-16T00:08:11.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.814+0000 7ff0f5d0a640 1 --2- 172.21.15.139:0/60932895 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0f00ccc90 0x7ff0f00cd0a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009d90 tx=0x7ff0e802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e76974c25b3ec595 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:11.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.814+0000 7ff0e7fff640 1 -- 172.21.15.139:0/60932895 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8007740 con 0x7ff0f00ccc90 2024-09-16T00:08:11.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.814+0000 7ff0e7fff640 1 -- 172.21.15.139:0/60932895 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e80078e0 con 0x7ff0f00ccc90 2024-09-16T00:08:11.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.814+0000 7ff0e7fff640 1 -- 172.21.15.139:0/60932895 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8007cb0 con 0x7ff0f00ccc90 2024-09-16T00:08:11.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.815+0000 7ff0f776d640 1 -- 172.21.15.139:0/60932895 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0f00ccc90 msgr2=0x7ff0f00cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.815+0000 7ff0f776d640 1 --2- 172.21.15.139:0/60932895 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0f00ccc90 0x7ff0f00cd0a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009d90 tx=0x7ff0e802a660 comp rx=0 tx=0).stop 2024-09-16T00:08:11.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.815+0000 7ff0f776d640 1 -- 172.21.15.139:0/60932895 shutdown_connections 2024-09-16T00:08:11.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.815+0000 7ff0f776d640 1 --2- 172.21.15.139:0/60932895 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0f00ccc90 0x7ff0f00cd0a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.815+0000 7ff0f776d640 1 --2- 172.21.15.139:0/60932895 >> v2:172.21.15.164:3300/0 conn(0x7ff0f0068060 0x7ff0f015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.815+0000 7ff0f776d640 1 -- 172.21.15.139:0/60932895 >> 172.21.15.139:0/60932895 conn(0x7ff0f006ca40 msgr2=0x7ff0f0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:11.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.815+0000 7ff0f776d640 1 -- 172.21.15.139:0/60932895 shutdown_connections 2024-09-16T00:08:11.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.815+0000 7ff0f776d640 1 -- 172.21.15.139:0/60932895 wait complete. 2024-09-16T00:08:11.818 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:11.818 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:11.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.817+0000 7ff0f776d640 1 Processor -- start 2024-09-16T00:08:11.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.817+0000 7ff0f776d640 1 -- start start 2024-09-16T00:08:11.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.818+0000 7ff0f776d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0f0068060 0x7ff0f00b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.818+0000 7ff0f776d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f00ccc90 0x7ff0f015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.818+0000 7ff0f776d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0f0154c20 0x7ff0f01f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.818+0000 7ff0f776d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f015e0d0 con 0x7ff0f00ccc90 2024-09-16T00:08:11.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.818+0000 7ff0f776d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f00cc260 con 0x7ff0f0068060 2024-09-16T00:08:11.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.818+0000 7ff0f776d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff0f015e3b0 con 0x7ff0f0154c20 2024-09-16T00:08:11.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0f5509640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0f0068060 0x7ff0f00b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0f4d08640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f00ccc90 0x7ff0f015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0f5509640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0f0068060 0x7ff0f00b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:60862/0 (socket says 172.21.15.139:60862) 2024-09-16T00:08:11.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0f5509640 1 -- 172.21.15.139:0/2009791174 learned_addr learned my addr 172.21.15.139:0/2009791174 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:11.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0f5d0a640 1 --2- 172.21.15.139:0/2009791174 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0f0154c20 0x7ff0f01f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0f5509640 1 -- 172.21.15.139:0/2009791174 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0f0154c20 msgr2=0x7ff0f01f44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0f5509640 1 --2- 172.21.15.139:0/2009791174 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0f0154c20 0x7ff0f01f44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0f5509640 1 -- 172.21.15.139:0/2009791174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f00ccc90 msgr2=0x7ff0f015b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:11.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0f5509640 1 --2- 172.21.15.139:0/2009791174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f00ccc90 0x7ff0f015b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:11.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0f5509640 1 -- 172.21.15.139:0/2009791174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f00d0f20 con 0x7ff0f0068060 2024-09-16T00:08:11.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0f5d0a640 1 --2- 172.21.15.139:0/2009791174 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0f0154c20 0x7ff0f01f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:11.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0f4d08640 1 --2- 172.21.15.139:0/2009791174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f00ccc90 0x7ff0f015b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:11.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0f5509640 1 --2- 172.21.15.139:0/2009791174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0f0068060 0x7ff0f00b55e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00c390 tx=0x7ff0ec00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:11.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0e5ffb640 1 -- 172.21.15.139:0/2009791174 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec017070 con 0x7ff0f0068060 2024-09-16T00:08:11.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0e5ffb640 1 -- 172.21.15.139:0/2009791174 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0ec005ce0 con 0x7ff0f0068060 2024-09-16T00:08:11.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0e5ffb640 1 -- 172.21.15.139:0/2009791174 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec0072f0 con 0x7ff0f0068060 2024-09-16T00:08:11.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.819+0000 7ff0f776d640 1 -- 172.21.15.139:0/2009791174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f00d0030 con 0x7ff0f0068060 2024-09-16T00:08:11.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.820+0000 7ff0f776d640 1 -- 172.21.15.139:0/2009791174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f00cefb0 con 0x7ff0f0068060 2024-09-16T00:08:11.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.821+0000 7ff0f776d640 1 -- 172.21.15.139:0/2009791174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f0152210 con 0x7ff0f0068060 2024-09-16T00:08:11.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.821+0000 7ff0e5ffb640 1 -- 172.21.15.139:0/2009791174 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff0ec01a070 con 0x7ff0f0068060 2024-09-16T00:08:11.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.822+0000 7ff0e5ffb640 1 --2- 172.21.15.139:0/2009791174 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff0d406c2e0 0x7ff0d406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:11.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.822+0000 7ff0e5ffb640 1 -- 172.21.15.139:0/2009791174 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5738+0+0 (secure 0 0 0) 0x7ff0ec08a780 con 0x7ff0f0068060 2024-09-16T00:08:11.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.822+0000 7ff0f4d08640 1 --2- 172.21.15.139:0/2009791174 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff0d406c2e0 0x7ff0d406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:11.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.822+0000 7ff0f4d08640 1 --2- 172.21.15.139:0/2009791174 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff0d406c2e0 0x7ff0d406e7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0148240 tx=0x7ff0e0009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:11.827 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:08:11.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:11.826+0000 7ff0e5ffb640 1 -- 172.21.15.139:0/2009791174 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff0f0152210 con 0x7ff0f0068060 2024-09-16T00:08:11.869 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:11.868+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:11.961 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:11.961+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:11.981 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:11.981 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:11.990 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:08:12.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:12.019+0000 7ff0f776d640 1 -- 172.21.15.139:0/2009791174 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} v 0) -- 0x7ff0f0075fc0 con 0x7ff0f0068060 2024-09-16T00:08:12.063 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:12.063+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:12.108 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:12.109 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:12.121 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:08:12.164 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:12.163+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:12.265 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:12.264+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:12.267 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:12.267 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:12.276 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:08:12.365 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:12.365+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:12.422 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:12.423 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:12.432 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:08:12.474 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:12.474+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:12.545 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:12.546 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:12.555 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:08:12.597 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:12.596+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:12.667 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:12.666+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:12.707 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:12.707 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:12.717 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:08:12.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:12.768+0000 7ff0e5ffb640 1 -- 172.21.15.139:0/2009791174 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]=0 set osd.6 primary-affinity to 1 (8655362) v32) ==== 133+0+0 (secure 0 0 0) 0x7ff0f0075fc0 con 0x7ff0f0068060 2024-09-16T00:08:12.769 INFO:teuthology.orchestra.run.smithi139.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-16T00:08:12.769 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:12.768+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:12.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:12.771+0000 7ff0f776d640 1 -- 172.21.15.139:0/2009791174 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff0d406c2e0 msgr2=0x7ff0d406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:12.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:12.771+0000 7ff0f776d640 1 --2- 172.21.15.139:0/2009791174 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff0d406c2e0 0x7ff0d406e7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0148240 tx=0x7ff0e0009110 comp rx=0 tx=0).stop 2024-09-16T00:08:12.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:12.771+0000 7ff0f776d640 1 -- 172.21.15.139:0/2009791174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0f0068060 msgr2=0x7ff0f00b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:12.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:12.771+0000 7ff0f776d640 1 --2- 172.21.15.139:0/2009791174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0f0068060 0x7ff0f00b55e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00c390 tx=0x7ff0ec00c850 comp rx=0 tx=0).stop 2024-09-16T00:08:12.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:12.771+0000 7ff0f776d640 1 -- 172.21.15.139:0/2009791174 shutdown_connections 2024-09-16T00:08:12.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:12.771+0000 7ff0f776d640 1 --2- 172.21.15.139:0/2009791174 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff0d406c2e0 0x7ff0d406e7a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:12.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:12.771+0000 7ff0f776d640 1 --2- 172.21.15.139:0/2009791174 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff0f0154c20 0x7ff0f01f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:12.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:12.771+0000 7ff0f776d640 1 --2- 172.21.15.139:0/2009791174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0f00ccc90 0x7ff0f015b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:12.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:12.771+0000 7ff0f776d640 1 --2- 172.21.15.139:0/2009791174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff0f0068060 0x7ff0f00b55e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:12.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:12.771+0000 7ff0f776d640 1 -- 172.21.15.139:0/2009791174 >> 172.21.15.139:0/2009791174 conn(0x7ff0f006ca40 msgr2=0x7ff0f0152db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:12.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:12.772+0000 7ff0f776d640 1 -- 172.21.15.139:0/2009791174 shutdown_connections 2024-09-16T00:08:12.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:12.772+0000 7ff0f776d640 1 -- 172.21.15.139:0/2009791174 wait complete. 2024-09-16T00:08:12.869 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:12.869+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:12.870 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:12.870 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:12.879 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:08:12.970 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:12.970+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:12.985 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:12.986 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:12.995 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:08:13.072 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:13.071+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:13.103 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:13.103 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:13.113 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:08:13.173 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:13.172+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:13.222 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:13.222 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:13.232 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:08:13.275 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:13.275+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:13.347 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:13.347 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:13.357 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:08:13.397 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:13.397+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:13.474 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:13.474 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:13.476 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:13.475+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:13.483 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:08:13.577 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:13.576+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:13.630 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:13.630 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:13.640 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:08:13.668 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:08:13.712 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:13.712+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:13.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.766+0000 7fbd51ae4640 1 Processor -- start 2024-09-16T00:08:13.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.766+0000 7fbd51ae4640 1 -- start start 2024-09-16T00:08:13.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.766+0000 7fbd51ae4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd4c0d0490 0x7fbd4c0b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.767+0000 7fbd51ae4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbd4c0b4750 0x7fbd4c1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.767+0000 7fbd51ae4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd4c0b3180 con 0x7fbd4c0d0870 2024-09-16T00:08:13.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.767+0000 7fbd51ae4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbd4c15b940 con 0x7fbd4c0b4750 2024-09-16T00:08:13.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.767+0000 7fbd51ae4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbd4c0b3bb0 con 0x7fbd4c0d0490 2024-09-16T00:08:13.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.767+0000 7fbd4bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbd4c0b4750 0x7fbd4c1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.767+0000 7fbd4bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbd4c0b4750 0x7fbd4c1546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35900/0 (socket says 172.21.15.139:35900) 2024-09-16T00:08:13.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.767+0000 7fbd4bfff640 1 -- 172.21.15.139:0/2500516501 learned_addr learned my addr 172.21.15.139:0/2500516501 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:13.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.767+0000 7fbd4b7fe640 1 --2- 172.21.15.139:0/2500516501 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd4c0d0490 0x7fbd4c0b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.767+0000 7fbd4bfff640 1 -- 172.21.15.139:0/2500516501 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd4c0d0490 msgr2=0x7fbd4c0b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:13.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.767+0000 7fbd4bfff640 1 --2- 172.21.15.139:0/2500516501 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd4c0d0490 0x7fbd4c0b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.767+0000 7fbd4bfff640 1 -- 172.21.15.139:0/2500516501 >> v1:172.21.15.164:6789/0 conn(0x7fbd4c0d0870 legacy=0x7fbd4c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.767+0000 7fbd4bfff640 1 -- 172.21.15.139:0/2500516501 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c15adc0 con 0x7fbd4c0b4750 2024-09-16T00:08:13.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.767+0000 7fbd4b7fe640 1 --2- 172.21.15.139:0/2500516501 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd4c0d0490 0x7fbd4c0b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:08:13.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.768+0000 7fbd4bfff640 1 --2- 172.21.15.139:0/2500516501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd4c0b4750 0x7fbd4c1546a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009e80 tx=0x7fbd4402a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89fa16e6dc3f631 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:13.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.768+0000 7fbd4a7fc640 1 -- 172.21.15.139:0/2500516501 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44036070 con 0x7fbd4c0b4750 2024-09-16T00:08:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.768+0000 7fbd4a7fc640 1 -- 172.21.15.139:0/2500516501 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd44031600 con 0x7fbd4c0b4750 2024-09-16T00:08:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.768+0000 7fbd4a7fc640 1 -- 172.21.15.139:0/2500516501 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd440319d0 con 0x7fbd4c0b4750 2024-09-16T00:08:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.768+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2500516501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd4c0b4750 msgr2=0x7fbd4c1546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.768+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2500516501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd4c0b4750 0x7fbd4c1546a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009e80 tx=0x7fbd4402a7e0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.768+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2500516501 shutdown_connections 2024-09-16T00:08:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.768+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2500516501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd4c0b4750 0x7fbd4c1546a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.768+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2500516501 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd4c0d0490 0x7fbd4c0b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.768+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2500516501 >> 172.21.15.139:0/2500516501 conn(0x7fbd4c06c520 msgr2=0x7fbd4c06c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.769+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2500516501 shutdown_connections 2024-09-16T00:08:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.769+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2500516501 wait complete. 2024-09-16T00:08:13.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.770+0000 7fbd51ae4640 1 Processor -- start 2024-09-16T00:08:13.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.770+0000 7fbd51ae4640 1 -- start start 2024-09-16T00:08:13.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd51ae4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd4c0b4750 0x7fbd4c14a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd51ae4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd4c0d0490 0x7fbd4c1f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd51ae4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4c0d0870 0x7fbd4c0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd51ae4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c15b660 con 0x7fbd4c0b4750 2024-09-16T00:08:13.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd4b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd4c0b4750 0x7fbd4c14a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd51ae4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c15b380 con 0x7fbd4c0d0870 2024-09-16T00:08:13.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd51ae4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbd4c15a980 con 0x7fbd4c0d0490 2024-09-16T00:08:13.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd4b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd4c0b4750 0x7fbd4c14a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35904/0 (socket says 172.21.15.139:35904) 2024-09-16T00:08:13.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd4b7fe640 1 -- 172.21.15.139:0/2368384283 learned_addr learned my addr 172.21.15.139:0/2368384283 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:13.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd4affd640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd4c0d0490 0x7fbd4c1f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd4bfff640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4c0d0870 0x7fbd4c0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd4b7fe640 1 -- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd4c0d0490 msgr2=0x7fbd4c1f3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd4b7fe640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbd4c0d0490 0x7fbd4c1f3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd4b7fe640 1 -- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4c0d0870 msgr2=0x7fbd4c0ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd4b7fe640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbd4c0d0870 0x7fbd4c0ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd4b7fe640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c0c9780 con 0x7fbd4c0b4750 2024-09-16T00:08:13.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.771+0000 7fbd4b7fe640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd4c0b4750 0x7fbd4c14a540 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00e7e0 tx=0x7fbd3c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:13.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.772+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c00f040 con 0x7fbd4c0b4750 2024-09-16T00:08:13.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.772+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4c0cfa00 con 0x7fbd4c0b4750 2024-09-16T00:08:13.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.772+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd3c004030 con 0x7fbd4c0b4750 2024-09-16T00:08:13.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.772+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c004400 con 0x7fbd4c0b4750 2024-09-16T00:08:13.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.772+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4c0c7990 con 0x7fbd4c0b4750 2024-09-16T00:08:13.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.773+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fbd4c0b3430 con 0x7fbd4c0b4750 2024-09-16T00:08:13.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.774+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbd3c00c6d0 con 0x7fbd4c0b4750 2024-09-16T00:08:13.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.774+0000 7fbd33fff640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbd3806c2b0 0x7fbd3806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.775+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5770+0+0 (secure 0 0 0) 0x7fbd3c08b4d0 con 0x7fbd4c0b4750 2024-09-16T00:08:13.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.775+0000 7fbd4affd640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbd3806c2b0 0x7fbd3806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.775+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v32) ==== 54+0+16 (secure 0 0 0) 0x7fbd4c0c9780 con 0x7fbd4c0b4750 2024-09-16T00:08:13.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.775+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbd4c1f4040 0x7fbd4c0cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.775+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd4c152380 con 0x7fbd4c1f4040 2024-09-16T00:08:13.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.775+0000 7fbd4affd640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbd3806c2b0 0x7fbd3806e770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0cd7c0 tx=0x7fbd40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:13.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.775+0000 7fbd4bfff640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbd4c1f4040 0x7fbd4c0cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.775+0000 7fbd4bfff640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbd4c1f4040 0x7fbd4c0cd1e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:13.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.776+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd4c152380 con 0x7fbd4c1f4040 2024-09-16T00:08:13.778 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T00:08:13.779 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:13.778+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:13.788 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:13.788 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:13.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.793+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbd4c15a680 con 0x7fbd4c1f4040 2024-09-16T00:08:13.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.794+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbd4c15a680 con 0x7fbd4c1f4040 2024-09-16T00:08:13.795 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:13.796 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:13.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.794+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbd4c14ae20 0x7fbd4c152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.795+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fbd4c0c8070 con 0x7fbd4c14ae20 2024-09-16T00:08:13.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.795+0000 7fbd4b7fe640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbd4c14ae20 0x7fbd4c152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.795+0000 7fbd4b7fe640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbd4c14ae20 0x7fbd4c152b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:13.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.796+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd4c0c8070 con 0x7fbd4c14ae20 2024-09-16T00:08:13.798 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:08:13.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.812+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbd4c15a3a0 con 0x7fbd4c14ae20 2024-09-16T00:08:13.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.814+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbd4c15a3a0 con 0x7fbd4c14ae20 2024-09-16T00:08:13.815 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:13.816 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:13.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.814+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbd4c0ce5c0 0x7fbd4c0cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.814+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fbd4c0ca4b0 con 0x7fbd4c0ce5c0 2024-09-16T00:08:13.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.814+0000 7fbd4affd640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbd4c0ce5c0 0x7fbd4c0cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.815+0000 7fbd4affd640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbd4c0ce5c0 0x7fbd4c0cea40 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:13.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.815+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd4c0ca4b0 con 0x7fbd4c0ce5c0 2024-09-16T00:08:13.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.830+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbd4c062880 con 0x7fbd4c0ce5c0 2024-09-16T00:08:13.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.832+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbd4c062880 con 0x7fbd4c0ce5c0 2024-09-16T00:08:13.833 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:13.834 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:13.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.832+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbd4c063360 0x7fbd4c0cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.832+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fbd4c0ca660 con 0x7fbd4c063360 2024-09-16T00:08:13.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.832+0000 7fbd4bfff640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbd4c063360 0x7fbd4c0cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.833+0000 7fbd4bfff640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbd4c063360 0x7fbd4c0cb0b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:13.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.833+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd4c0ca660 con 0x7fbd4c063360 2024-09-16T00:08:13.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.848+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbd4c0cb8c0 con 0x7fbd4c063360 2024-09-16T00:08:13.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.850+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbd4c0cb8c0 con 0x7fbd4c063360 2024-09-16T00:08:13.851 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.850+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbd4c02cb50 0x7fbd4c02cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.850+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fbd4c0cbe90 con 0x7fbd4c02cb50 2024-09-16T00:08:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.851+0000 7fbd4b7fe640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbd4c02cb50 0x7fbd4c02cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.851+0000 7fbd4b7fe640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbd4c02cb50 0x7fbd4c02cfd0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:13.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.852+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd4c0cbe90 con 0x7fbd4c02cb50 2024-09-16T00:08:13.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.866+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbd4c0cbe90 con 0x7fbd4c02cb50 2024-09-16T00:08:13.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.869+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbd4c0cbe90 con 0x7fbd4c02cb50 2024-09-16T00:08:13.870 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:13.871 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:13.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.869+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbd4c204ec0 0x7fbd4c2072b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.869+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fbd4c02da20 con 0x7fbd4c204ec0 2024-09-16T00:08:13.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.869+0000 7fbd4affd640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbd4c204ec0 0x7fbd4c2072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.870+0000 7fbd4affd640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbd4c204ec0 0x7fbd4c2072b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:13.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.870+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd4c02da20 con 0x7fbd4c204ec0 2024-09-16T00:08:13.880 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:13.879+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:13.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.881+0000 7fa5ac566640 1 Processor -- start 2024-09-16T00:08:13.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.881+0000 7fa5ac566640 1 -- start start 2024-09-16T00:08:13.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.882+0000 7fa5ac566640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa5a4154a10 0x7fa5a40cd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.882+0000 7fa5ac566640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a415a950 con 0x7fa5a406ea70 2024-09-16T00:08:13.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.882+0000 7fa5ac566640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a415ac30 con 0x7fa5a415e840 2024-09-16T00:08:13.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.882+0000 7fa5ac566640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a415af10 con 0x7fa5a4154a10 2024-09-16T00:08:13.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.883+0000 7fa5a9b01640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa5a406ea70 0x7fa5a415e130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56336/0 (socket says 172.21.15.139:56336) 2024-09-16T00:08:13.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.883+0000 7fa5a9b01640 1 -- 172.21.15.139:0/2558218271 learned_addr learned my addr 172.21.15.139:0/2558218271 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:13.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.883+0000 7fa5aa302640 1 --2- 172.21.15.139:0/2558218271 >> v2:172.21.15.164:3300/0 conn(0x7fa5a4154a10 0x7fa5a40cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.883+0000 7fa5a9300640 1 -- 172.21.15.139:0/2558218271 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 354145576 0 0) 0x7fa5a415a950 con 0x7fa5a406ea70 2024-09-16T00:08:13.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.883+0000 7fa5a9300640 1 -- 172.21.15.139:0/2558218271 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa590003520 con 0x7fa5a406ea70 2024-09-16T00:08:13.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.883+0000 7fa5a9300640 1 -- 172.21.15.139:0/2558218271 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3447475642 0 0) 0x7fa5a415ac30 con 0x7fa5a415e840 2024-09-16T00:08:13.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.883+0000 7fa5a9300640 1 -- 172.21.15.139:0/2558218271 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5a415a950 con 0x7fa5a415e840 2024-09-16T00:08:13.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.883+0000 7fa5aa302640 1 -- 172.21.15.139:0/2558218271 >> v1:172.21.15.164:6789/0 conn(0x7fa5a415e840 legacy=0x7fa5a41650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.883+0000 7fa5aa302640 1 -- 172.21.15.139:0/2558218271 >> v1:172.21.15.139:6789/0 conn(0x7fa5a406ea70 legacy=0x7fa5a415e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.883+0000 7fa5aa302640 1 -- 172.21.15.139:0/2558218271 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a415b1f0 con 0x7fa5a4154a10 2024-09-16T00:08:13.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.884+0000 7fa5aa302640 1 --2- 172.21.15.139:0/2558218271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5a4154a10 0x7fa5a40cd030 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00097a0 tx=0x7fa5a002a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee009e8d1d53c206 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:13.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.884+0000 7fa5a9300640 1 -- 172.21.15.139:0/2558218271 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0037070 con 0x7fa5a4154a10 2024-09-16T00:08:13.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.884+0000 7fa5a9300640 1 -- 172.21.15.139:0/2558218271 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a0032640 con 0x7fa5a4154a10 2024-09-16T00:08:13.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.884+0000 7fa5a9300640 1 -- 172.21.15.139:0/2558218271 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0032a10 con 0x7fa5a4154a10 2024-09-16T00:08:13.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.885+0000 7fa5ac566640 1 -- 172.21.15.139:0/2558218271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5a4154a10 msgr2=0x7fa5a40cd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.885+0000 7fa5ac566640 1 --2- 172.21.15.139:0/2558218271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5a4154a10 0x7fa5a40cd030 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa5a00097a0 tx=0x7fa5a002a2c0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.885+0000 7fa5ac566640 1 -- 172.21.15.139:0/2558218271 shutdown_connections 2024-09-16T00:08:13.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.885+0000 7fa5ac566640 1 --2- 172.21.15.139:0/2558218271 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5a4154a10 0x7fa5a40cd030 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.885+0000 7fa5ac566640 1 -- 172.21.15.139:0/2558218271 >> 172.21.15.139:0/2558218271 conn(0x7fa5a406cc30 msgr2=0x7fa5a406d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:13.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.885+0000 7fa5ac566640 1 -- 172.21.15.139:0/2558218271 shutdown_connections 2024-09-16T00:08:13.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.885+0000 7fa5ac566640 1 -- 172.21.15.139:0/2558218271 wait complete. 2024-09-16T00:08:13.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.886+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbd4c02da20 con 0x7fbd4c204ec0 2024-09-16T00:08:13.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.886+0000 7fa5ac566640 1 Processor -- start 2024-09-16T00:08:13.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.886+0000 7fa5ac566640 1 -- start start 2024-09-16T00:08:13.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.887+0000 7fa5ac566640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa5a406ea70 0x7fa5a415a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.887+0000 7fa5ac566640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5a4154a10 0x7fa5a414cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.887+0000 7fa5ac566640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a415e840 0x7fa5a414d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.887+0000 7fa5ac566640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a4154df0 con 0x7fa5a415e840 2024-09-16T00:08:13.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.887+0000 7fa5ac566640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a406da90 con 0x7fa5a4154a10 2024-09-16T00:08:13.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.887+0000 7fa5ac566640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a406c460 con 0x7fa5a406ea70 2024-09-16T00:08:13.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.887+0000 7fa5aab03640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a415e840 0x7fa5a414d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.887+0000 7fa5aab03640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a415e840 0x7fa5a414d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35920/0 (socket says 172.21.15.139:35920) 2024-09-16T00:08:13.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.887+0000 7fa5aab03640 1 -- 172.21.15.139:0/1873618472 learned_addr learned my addr 172.21.15.139:0/1873618472 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:13.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.887+0000 7fa5a9b01640 1 --2- 172.21.15.139:0/1873618472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5a4154a10 0x7fa5a414cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.887+0000 7fa5aa302640 1 --2- 172.21.15.139:0/1873618472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa5a406ea70 0x7fa5a415a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fa5aab03640 1 -- 172.21.15.139:0/1873618472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa5a406ea70 msgr2=0x7fa5a415a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fa5aab03640 1 --2- 172.21.15.139:0/1873618472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa5a406ea70 0x7fa5a415a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbd4c02da20 con 0x7fbd4c204ec0 2024-09-16T00:08:13.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fa5aab03640 1 -- 172.21.15.139:0/1873618472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5a4154a10 msgr2=0x7fa5a414cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fa5aab03640 1 --2- 172.21.15.139:0/1873618472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5a4154a10 0x7fa5a414cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.892 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:13.892 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:13.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fa5aab03640 1 -- 172.21.15.139:0/1873618472 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a41f13b0 con 0x7fa5a415e840 2024-09-16T00:08:13.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fa5a9b01640 1 --2- 172.21.15.139:0/1873618472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5a4154a10 0x7fa5a414cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:13.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbd4c207d00 0x7fbd4c20a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fa5aa302640 1 --2- 172.21.15.139:0/1873618472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa5a406ea70 0x7fa5a415a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:13.894 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fbd4c20cfc0 con 0x7fbd4c207d00 2024-09-16T00:08:13.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fa5aab03640 1 --2- 172.21.15.139:0/1873618472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a415e840 0x7fa5a414d4f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00c6e0 tx=0x7fa59c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:13.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fa59affd640 1 -- 172.21.15.139:0/1873618472 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c003b70 con 0x7fa5a415e840 2024-09-16T00:08:13.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fbd4bfff640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbd4c207d00 0x7fbd4c20a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fa59affd640 1 -- 172.21.15.139:0/1873618472 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59c003d10 con 0x7fa5a415e840 2024-09-16T00:08:13.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fa5ac566640 1 -- 172.21.15.139:0/1873618472 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a41f0550 con 0x7fa5a415e840 2024-09-16T00:08:13.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fa59affd640 1 -- 172.21.15.139:0/1873618472 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0052d0 con 0x7fa5a415e840 2024-09-16T00:08:13.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fa5ac566640 1 -- 172.21.15.139:0/1873618472 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a41f0b10 con 0x7fa5a415e840 2024-09-16T00:08:13.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.888+0000 7fbd4bfff640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbd4c207d00 0x7fbd4c20a0f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:13.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.889+0000 7fa5ac566640 1 -- 172.21.15.139:0/1873618472 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a406bd20 con 0x7fa5a415e840 2024-09-16T00:08:13.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.889+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd4c20cfc0 con 0x7fbd4c207d00 2024-09-16T00:08:13.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.890+0000 7fa59affd640 1 -- 172.21.15.139:0/1873618472 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa59c005470 con 0x7fa5a415e840 2024-09-16T00:08:13.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.890+0000 7fa59affd640 1 --2- 172.21.15.139:0/1873618472 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa58806c300 0x7fa58806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.890+0000 7fa59affd640 1 -- 172.21.15.139:0/1873618472 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5770+0+0 (secure 0 0 0) 0x7fa59c08b1f0 con 0x7fa5a415e840 2024-09-16T00:08:13.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.890+0000 7fa5aa302640 1 --2- 172.21.15.139:0/1873618472 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa58806c300 0x7fa58806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.891+0000 7fa5aa302640 1 --2- 172.21.15.139:0/1873618472 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa58806c300 0x7fa58806e7c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0034000 tx=0x7fa5a002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:13.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.895+0000 7fa59affd640 1 -- 172.21.15.139:0/1873618472 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa59c059320 con 0x7fa5a415e840 2024-09-16T00:08:13.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.916+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbd4c20a630 con 0x7fbd4c207d00 2024-09-16T00:08:13.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.918+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbd4c20a630 con 0x7fbd4c207d00 2024-09-16T00:08:13.919 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:13.920 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:13.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.918+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbd4c20ad50 0x7fbd4c20b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:13.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.918+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fbd4c210410 con 0x7fbd4c20ad50 2024-09-16T00:08:13.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.918+0000 7fbd4b7fe640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbd4c20ad50 0x7fbd4c20b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:13.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.918+0000 7fbd4b7fe640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbd4c20ad50 0x7fbd4c20b170 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:13.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.919+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd4c210410 con 0x7fbd4c20ad50 2024-09-16T00:08:13.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.935+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbd4c20b730 con 0x7fbd4c20ad50 2024-09-16T00:08:13.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd33fff640 1 -- 172.21.15.139:0/2368384283 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbd4c20b730 con 0x7fbd4c20ad50 2024-09-16T00:08:13.938 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:13.940 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:13.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbd4c1f4040 msgr2=0x7fbd4c0cd1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbd4c1f4040 0x7fbd4c0cd1e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbd4c14ae20 msgr2=0x7fbd4c152b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbd4c14ae20 0x7fbd4c152b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbd4c0ce5c0 msgr2=0x7fbd4c0cea40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbd4c0ce5c0 0x7fbd4c0cea40 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbd4c063360 msgr2=0x7fbd4c0cb0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbd4c063360 0x7fbd4c0cb0b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbd4c02cb50 msgr2=0x7fbd4c02cfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbd4c02cb50 0x7fbd4c02cfd0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd4bfff640 1 -- 172.21.15.139:0/2368384283 reap_dead start 2024-09-16T00:08:13.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbd4c204ec0 msgr2=0x7fbd4c2072b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbd4c204ec0 0x7fbd4c2072b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbd4c207d00 msgr2=0x7fbd4c20a0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbd4c207d00 0x7fbd4c20a0f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbd4c20ad50 msgr2=0x7fbd4c20b170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.937+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbd4c20ad50 0x7fbd4c20b170 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.938+0000 7fbd4bfff640 1 -- 172.21.15.139:0/2368384283 reap_dead start 2024-09-16T00:08:13.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.938+0000 7fbd4bfff640 1 -- 172.21.15.139:0/2368384283 reap_dead start 2024-09-16T00:08:13.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.938+0000 7fbd4bfff640 1 -- 172.21.15.139:0/2368384283 reap_dead start 2024-09-16T00:08:13.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.938+0000 7fbd4bfff640 1 -- 172.21.15.139:0/2368384283 reap_dead start 2024-09-16T00:08:13.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.938+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbd3806c2b0 msgr2=0x7fbd3806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.938+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbd3806c2b0 0x7fbd3806e770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0cd7c0 tx=0x7fbd40009040 comp rx=0 tx=0).stop 2024-09-16T00:08:13.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.938+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd4c0b4750 msgr2=0x7fbd4c14a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:13.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.938+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd4c0b4750 0x7fbd4c14a540 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00e7e0 tx=0x7fbd3c00eca0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.938+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 shutdown_connections 2024-09-16T00:08:13.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.938+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbd3806c2b0 0x7fbd3806e770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.938+0000 7fbd51ae4640 1 --2- 172.21.15.139:0/2368384283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd4c0b4750 0x7fbd4c14a540 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:13.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.938+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 >> 172.21.15.139:0/2368384283 conn(0x7fbd4c06c520 msgr2=0x7fbd4c1585e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:13.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.938+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 shutdown_connections 2024-09-16T00:08:13.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:13.938+0000 7fbd51ae4640 1 -- 172.21.15.139:0/2368384283 wait complete. 2024-09-16T00:08:13.945 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:13.945 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:13.945 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:13.945 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:13.954 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:08:13.993 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:13.993+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:14.057 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:14.058 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:14.058 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:14.058 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:14.058 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:14.066 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:08:14.082 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:14.081+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:14.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:14.092+0000 7fa5ac566640 1 -- 172.21.15.139:0/1873618472 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fa5a406c820 con 0x7fa5a415e840 2024-09-16T00:08:14.170 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:14 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.554440522, 2024-09-16T00:08:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029190470000000001, 2024-09-16T00:08:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.171 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:14.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615982+0000", 2024-09-16T00:08:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7439999999999999e-06 2024-09-16T00:08:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:08:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3743999999999999e-05 2024-09-16T00:08:14.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2255e-05 2024-09-16T00:08:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616029+0000", 2024-09-16T00:08:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1175999999999999e-05 2024-09-16T00:08:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616098+0000", 2024-09-16T00:08:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9153999999999996e-05 2024-09-16T00:08:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616132+0000", 2024-09-16T00:08:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3924000000000002e-05 2024-09-16T00:08:14.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027226189999999999 2024-09-16T00:08:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618899+0000", 2024-09-16T00:08:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4431000000000002e-05 2024-09-16T00:08:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:15 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.552213702000003, 2024-09-16T00:08:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095618099999999996, 2024-09-16T00:08:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618208+0000", 2024-09-16T00:08:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1850000000000001e-06 2024-09-16T00:08:14.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618217+0000", 2024-09-16T00:08:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439999999999992e-06 2024-09-16T00:08:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618230+0000", 2024-09-16T00:08:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2435999999999999e-05 2024-09-16T00:08:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618238+0000", 2024-09-16T00:08:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1319999999999994e-06 2024-09-16T00:08:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618331+0000", 2024-09-16T00:08:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3453000000000001e-05 2024-09-16T00:08:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618352+0000", 2024-09-16T00:08:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0693999999999999e-05 2024-09-16T00:08:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619112+0000", 2024-09-16T00:08:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00076007099999999997 2024-09-16T00:08:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619163+0000", 2024-09-16T00:08:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1066000000000002e-05 2024-09-16T00:08:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:30 2.15 e22/19 2:aa448500:::500.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.552156216999997, 2024-09-16T00:08:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058031250000000001, 2024-09-16T00:08:14.182 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:14.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618265+0000", 2024-09-16T00:08:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9200000000000001e-07 2024-09-16T00:08:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618271+0000", 2024-09-16T00:08:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9580000000000004e-06 2024-09-16T00:08:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618279+0000", 2024-09-16T00:08:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2239999999999995e-06 2024-09-16T00:08:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618285+0000", 2024-09-16T00:08:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-06 2024-09-16T00:08:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618567+0000", 2024-09-16T00:08:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028204400000000001 2024-09-16T00:08:14.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:08:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9678999999999999e-05 2024-09-16T00:08:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624024+0000", 2024-09-16T00:08:14.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054376720000000002 2024-09-16T00:08:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624068+0000", 2024-09-16T00:08:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:08:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:20 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.551799852999999, 2024-09-16T00:08:14.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028808739999999998, 2024-09-16T00:08:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618622+0000", 2024-09-16T00:08:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1260000000000001e-06 2024-09-16T00:08:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618631+0000", 2024-09-16T00:08:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4700000000000008e-06 2024-09-16T00:08:14.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618645+0000", 2024-09-16T00:08:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3984e-05 2024-09-16T00:08:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:08:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2101e-05 2024-09-16T00:08:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618720+0000", 2024-09-16T00:08:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2761000000000004e-05 2024-09-16T00:08:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618740+0000", 2024-09-16T00:08:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9409e-05 2024-09-16T00:08:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621436+0000", 2024-09-16T00:08:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002696211 2024-09-16T00:08:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621502+0000", 2024-09-16T00:08:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5811999999999999e-05 2024-09-16T00:08:14.192 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.193 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.193 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.193 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:21 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:14.193 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.550976564999999, 2024-09-16T00:08:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046318540000000004, 2024-09-16T00:08:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619445+0000", 2024-09-16T00:08:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.083e-06 2024-09-16T00:08:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619455+0000", 2024-09-16T00:08:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5920000000000007e-06 2024-09-16T00:08:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619467+0000", 2024-09-16T00:08:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2711000000000001e-05 2024-09-16T00:08:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619476+0000", 2024-09-16T00:08:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0139999999999998e-06 2024-09-16T00:08:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619529+0000", 2024-09-16T00:08:14.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3078999999999998e-05 2024-09-16T00:08:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619548+0000", 2024-09-16T00:08:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8952000000000001e-05 2024-09-16T00:08:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624037+0000", 2024-09-16T00:08:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044895730000000002 2024-09-16T00:08:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624076+0000", 2024-09-16T00:08:14.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8850000000000002e-05 2024-09-16T00:08:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.200 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.200 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:14.202 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:14.201+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:14.202 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:08:14.302 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:14.302+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:14.307 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:14.307 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:14.307 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:14.315 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:08:14.405 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:14.404+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:14.419 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:14.420 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:14.420 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:14.420 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:14.420 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:14.432 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:08:14.514 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:14.513+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:14.575 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:14.575 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:14.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:14.575 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:14.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.575 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:14.576 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:14.576 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.957970752999998, 2024-09-16T00:08:14.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056100969999999997, 2024-09-16T00:08:14.576 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.576 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.576 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.576 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:14.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:14.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615352+0000", 2024-09-16T00:08:14.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9859999999999999e-06 2024-09-16T00:08:14.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615363+0000", 2024-09-16T00:08:14.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1168999999999999e-05 2024-09-16T00:08:14.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615382+0000", 2024-09-16T00:08:14.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8224e-05 2024-09-16T00:08:14.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615393+0000", 2024-09-16T00:08:14.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0925e-05 2024-09-16T00:08:14.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615624+0000", 2024-09-16T00:08:14.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023159600000000001 2024-09-16T00:08:14.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615721+0000", 2024-09-16T00:08:14.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7040000000000006e-05 2024-09-16T00:08:14.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:14.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615885+0000", 2024-09-16T00:08:14.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163456 2024-09-16T00:08:14.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:14.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:08:14.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002191368 2024-09-16T00:08:14.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:14.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620870+0000", 2024-09-16T00:08:14.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941839999999999 2024-09-16T00:08:14.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620922+0000", 2024-09-16T00:08:14.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1641999999999998e-05 2024-09-16T00:08:14.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620956+0000", 2024-09-16T00:08:14.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4507e-05 2024-09-16T00:08:14.581 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.581 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.581 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.582 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:14.582 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:14.582 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.957880394, 2024-09-16T00:08:14.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005695739, 2024-09-16T00:08:14.582 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.582 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.582 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.582 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:14.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:14.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:14.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4900000000000005e-07 2024-09-16T00:08:14.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615442+0000", 2024-09-16T00:08:14.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.994e-06 2024-09-16T00:08:14.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:08:14.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1081e-05 2024-09-16T00:08:14.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615473+0000", 2024-09-16T00:08:14.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1910000000000006e-06 2024-09-16T00:08:14.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616230+0000", 2024-09-16T00:08:14.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075700400000000003 2024-09-16T00:08:14.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616305+0000", 2024-09-16T00:08:14.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4759999999999996e-05 2024-09-16T00:08:14.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:14.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616466+0000", 2024-09-16T00:08:14.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000161714 2024-09-16T00:08:14.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:14.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619821+0000", 2024-09-16T00:08:14.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033547339999999998 2024-09-16T00:08:14.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:14.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621028+0000", 2024-09-16T00:08:14.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012067530000000001 2024-09-16T00:08:14.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621087+0000", 2024-09-16T00:08:14.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9364999999999998e-05 2024-09-16T00:08:14.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621132+0000", 2024-09-16T00:08:14.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5293999999999997e-05 2024-09-16T00:08:14.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.587 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.587 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:14.587 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:14.587 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.957818162000002, 2024-09-16T00:08:14.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050565760000000001, 2024-09-16T00:08:14.588 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.588 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.588 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.588 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:14.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:14.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615500+0000", 2024-09-16T00:08:14.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8199999999999999e-07 2024-09-16T00:08:14.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615505+0000", 2024-09-16T00:08:14.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5280000000000003e-06 2024-09-16T00:08:14.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:14.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0317e-05 2024-09-16T00:08:14.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615523+0000", 2024-09-16T00:08:14.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4580000000000004e-06 2024-09-16T00:08:14.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615607+0000", 2024-09-16T00:08:14.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4153999999999994e-05 2024-09-16T00:08:14.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615748+0000", 2024-09-16T00:08:14.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140533 2024-09-16T00:08:14.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:14.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615911+0000", 2024-09-16T00:08:14.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163678 2024-09-16T00:08:14.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:14.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619944+0000", 2024-09-16T00:08:14.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040330909999999999 2024-09-16T00:08:14.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:14.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620471+0000", 2024-09-16T00:08:14.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052645600000000002 2024-09-16T00:08:14.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620512+0000", 2024-09-16T00:08:14.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0738000000000002e-05 2024-09-16T00:08:14.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620555+0000", 2024-09-16T00:08:14.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3841000000000003e-05 2024-09-16T00:08:14.593 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.593 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.593 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.593 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:14.593 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:14.593 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.957766055, 2024-09-16T00:08:14.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089162479999999999, 2024-09-16T00:08:14.594 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.594 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.594 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.594 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:14.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:14.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615552+0000", 2024-09-16T00:08:14.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:08:14.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615556+0000", 2024-09-16T00:08:14.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3610000000000003e-06 2024-09-16T00:08:14.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615566+0000", 2024-09-16T00:08:14.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6099999999999995e-06 2024-09-16T00:08:14.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615574+0000", 2024-09-16T00:08:14.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9379999999999992e-06 2024-09-16T00:08:14.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616777+0000", 2024-09-16T00:08:14.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012031640000000001 2024-09-16T00:08:14.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616882+0000", 2024-09-16T00:08:14.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105146 2024-09-16T00:08:14.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:14.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619696+0000", 2024-09-16T00:08:14.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028142179999999998 2024-09-16T00:08:14.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:14.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624234+0000", 2024-09-16T00:08:14.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045381889999999998 2024-09-16T00:08:14.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:14.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624393+0000", 2024-09-16T00:08:14.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015871200000000001 2024-09-16T00:08:14.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624434+0000", 2024-09-16T00:08:14.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0908999999999997e-05 2024-09-16T00:08:14.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624467+0000", 2024-09-16T00:08:14.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3217999999999999e-05 2024-09-16T00:08:14.598 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.598 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.599 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.599 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:14.599 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:14.599 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.957696257999999, 2024-09-16T00:08:14.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064898359999999997, 2024-09-16T00:08:14.599 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.599 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.599 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.599 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:14.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:14.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615622+0000", 2024-09-16T00:08:14.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.181e-06 2024-09-16T00:08:14.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615628+0000", 2024-09-16T00:08:14.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2450000000000003e-06 2024-09-16T00:08:14.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:08:14.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2554999999999999e-05 2024-09-16T00:08:14.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615650+0000", 2024-09-16T00:08:14.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4350000000000003e-06 2024-09-16T00:08:14.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616772+0000", 2024-09-16T00:08:14.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001122058 2024-09-16T00:08:14.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616889+0000", 2024-09-16T00:08:14.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116623 2024-09-16T00:08:14.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:14.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617035+0000", 2024-09-16T00:08:14.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014579799999999999 2024-09-16T00:08:14.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:14.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619829+0000", 2024-09-16T00:08:14.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941379999999998 2024-09-16T00:08:14.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:14.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622024+0000", 2024-09-16T00:08:14.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021949920000000002 2024-09-16T00:08:14.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622087+0000", 2024-09-16T00:08:14.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3048999999999999e-05 2024-09-16T00:08:14.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622111+0000", 2024-09-16T00:08:14.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3762000000000001e-05 2024-09-16T00:08:14.604 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.604 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.604 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.604 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:14.604 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:14.604 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.957633696999999, 2024-09-16T00:08:14.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065192160000000004, 2024-09-16T00:08:14.605 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.605 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.605 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.605 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:14.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:14.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615684+0000", 2024-09-16T00:08:14.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8299999999999995e-07 2024-09-16T00:08:14.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615691+0000", 2024-09-16T00:08:14.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.866e-06 2024-09-16T00:08:14.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615707+0000", 2024-09-16T00:08:14.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6073999999999999e-05 2024-09-16T00:08:14.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615714+0000", 2024-09-16T00:08:14.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-16T00:08:14.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617403+0000", 2024-09-16T00:08:14.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016881629999999999 2024-09-16T00:08:14.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617559+0000", 2024-09-16T00:08:14.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015669699999999999 2024-09-16T00:08:14.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:14.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:14.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000144165 2024-09-16T00:08:14.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:14.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621413+0000", 2024-09-16T00:08:14.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037092200000000001 2024-09-16T00:08:14.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:14.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622147+0000", 2024-09-16T00:08:14.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073468800000000005 2024-09-16T00:08:14.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622180+0000", 2024-09-16T00:08:14.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2901999999999998e-05 2024-09-16T00:08:14.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622203+0000", 2024-09-16T00:08:14.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2211000000000001e-05 2024-09-16T00:08:14.610 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.610 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.610 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.610 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:14.610 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:14.610 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.957590603, 2024-09-16T00:08:14.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065825270000000003, 2024-09-16T00:08:14.610 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.610 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.610 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:14.610 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:14.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:14.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:08:14.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7299999999999996e-07 2024-09-16T00:08:14.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615730+0000", 2024-09-16T00:08:14.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.004e-06 2024-09-16T00:08:14.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615737+0000", 2024-09-16T00:08:14.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4789999999999999e-06 2024-09-16T00:08:14.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615745+0000", 2024-09-16T00:08:14.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6699999999999994e-06 2024-09-16T00:08:14.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618115+0000", 2024-09-16T00:08:14.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023701569999999999 2024-09-16T00:08:14.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618247+0000", 2024-09-16T00:08:14.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013185000000000001 2024-09-16T00:08:14.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622309+0000", 2024-09-16T00:08:14.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040617939999999996 2024-09-16T00:08:14.614 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.614 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.614 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.614 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:14.614 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:14.614 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.957563354000001, 2024-09-16T00:08:14.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036685950000000002, 2024-09-16T00:08:14.614 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.614 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.615 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:14.615 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:14.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:14.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:14.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T00:08:14.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615763+0000", 2024-09-16T00:08:14.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4360000000000002e-06 2024-09-16T00:08:14.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615770+0000", 2024-09-16T00:08:14.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7540000000000006e-06 2024-09-16T00:08:14.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615803+0000", 2024-09-16T00:08:14.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3008e-05 2024-09-16T00:08:14.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615888+0000", 2024-09-16T00:08:14.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5056999999999996e-05 2024-09-16T00:08:14.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:08:14.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013059700000000001 2024-09-16T00:08:14.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619422+0000", 2024-09-16T00:08:14.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034033230000000002 2024-09-16T00:08:14.618 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.618 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing_reply(2.1a epoch 22/19 rep_tid 2 lcod 22'1)", 2024-09-16T00:08:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.954122040999998, 2024-09-16T00:08:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024304200000000001, 2024-09-16T00:08:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619196+0000", 2024-09-16T00:08:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2759999999999999e-06 2024-09-16T00:08:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619203+0000", 2024-09-16T00:08:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1779999999999997e-06 2024-09-16T00:08:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619223+0000", 2024-09-16T00:08:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9159000000000001e-05 2024-09-16T00:08:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619231+0000", 2024-09-16T00:08:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7830000000000004e-06 2024-09-16T00:08:14.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619320+0000", 2024-09-16T00:08:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8805000000000001e-05 2024-09-16T00:08:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619438+0000", 2024-09-16T00:08:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117841 2024-09-16T00:08:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:16 2.1d e22/19 2:b871830b:::605.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.953574764000003, 2024-09-16T00:08:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045616900000000002, 2024-09-16T00:08:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:08:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:08:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619755+0000", 2024-09-16T00:08:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3599999999999996e-06 2024-09-16T00:08:14.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619767+0000", 2024-09-16T00:08:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2805e-05 2024-09-16T00:08:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619787+0000", 2024-09-16T00:08:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9264000000000001e-05 2024-09-16T00:08:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619869+0000", 2024-09-16T00:08:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2639999999999995e-05 2024-09-16T00:08:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619895+0000", 2024-09-16T00:08:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5890000000000001e-05 2024-09-16T00:08:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624264+0000", 2024-09-16T00:08:14.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043690049999999996 2024-09-16T00:08:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624304+0000", 2024-09-16T00:08:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9807e-05 2024-09-16T00:08:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:17 2.1d e22/19 2:b871830b:::605.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.953222054999998, 2024-09-16T00:08:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042370910000000001, 2024-09-16T00:08:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620096+0000", 2024-09-16T00:08:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:08:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620102+0000", 2024-09-16T00:08:14.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7220000000000002e-06 2024-09-16T00:08:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620112+0000", 2024-09-16T00:08:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0156999999999999e-05 2024-09-16T00:08:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620120+0000", 2024-09-16T00:08:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2880000000000004e-06 2024-09-16T00:08:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620171+0000", 2024-09-16T00:08:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.168e-05 2024-09-16T00:08:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620189+0000", 2024-09-16T00:08:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7419e-05 2024-09-16T00:08:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624306+0000", 2024-09-16T00:08:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041175430000000004 2024-09-16T00:08:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624332+0000", 2024-09-16T00:08:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5717000000000001e-05 2024-09-16T00:08:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:24 2.7 e22/19 ondisk, result = 0)", 2024-09-16T00:08:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.952956348000001, 2024-09-16T00:08:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000208018, 2024-09-16T00:08:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.632 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8500000000000001e-07 2024-09-16T00:08:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620366+0000", 2024-09-16T00:08:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5379999999999996e-06 2024-09-16T00:08:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620378+0000", 2024-09-16T00:08:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.204e-05 2024-09-16T00:08:14.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620385+0000", 2024-09-16T00:08:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.413e-06 2024-09-16T00:08:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620455+0000", 2024-09-16T00:08:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9886000000000001e-05 2024-09-16T00:08:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620463+0000", 2024-09-16T00:08:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1559999999999995e-06 2024-09-16T00:08:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620569+0000", 2024-09-16T00:08:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001054 2024-09-16T00:08:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:18 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:14.636 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.952570883999996, 2024-09-16T00:08:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021768100000000001, 2024-09-16T00:08:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620747+0000", 2024-09-16T00:08:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.02e-07 2024-09-16T00:08:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620752+0000", 2024-09-16T00:08:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4439999999999999e-06 2024-09-16T00:08:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620765+0000", 2024-09-16T00:08:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2714000000000001e-05 2024-09-16T00:08:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:08:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1160000000000004e-06 2024-09-16T00:08:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3374999999999994e-05 2024-09-16T00:08:14.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620864+0000", 2024-09-16T00:08:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4320000000000005e-06 2024-09-16T00:08:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620964+0000", 2024-09-16T00:08:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9698000000000007e-05 2024-09-16T00:08:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:19 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.952509339999999, 2024-09-16T00:08:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00033551400000000003, 2024-09-16T00:08:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0800000000000005e-07 2024-09-16T00:08:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620812+0000", 2024-09-16T00:08:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.427e-06 2024-09-16T00:08:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620820+0000", 2024-09-16T00:08:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6649999999999992e-06 2024-09-16T00:08:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620828+0000", 2024-09-16T00:08:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0010000000000001e-06 2024-09-16T00:08:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620995+0000", 2024-09-16T00:08:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016709400000000001 2024-09-16T00:08:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621005+0000", 2024-09-16T00:08:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7170000000000003e-06 2024-09-16T00:08:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621143+0000", 2024-09-16T00:08:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000138102 2024-09-16T00:08:14.644 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:3 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.951332069000003, 2024-09-16T00:08:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022247799999999999, 2024-09-16T00:08:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3300000000000003e-07 2024-09-16T00:08:14.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621989+0000", 2024-09-16T00:08:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2969999999999999e-06 2024-09-16T00:08:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621997+0000", 2024-09-16T00:08:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8990000000000001e-06 2024-09-16T00:08:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622002+0000", 2024-09-16T00:08:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5439999999999998e-06 2024-09-16T00:08:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622137+0000", 2024-09-16T00:08:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000135103 2024-09-16T00:08:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622143+0000", 2024-09-16T00:08:14.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9200000000000001e-06 2024-09-16T00:08:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622207+0000", 2024-09-16T00:08:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4282000000000006e-05 2024-09-16T00:08:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing_reply(2.14 epoch 22/19 rep_tid 6 lcod 22'5)", 2024-09-16T00:08:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.951300732999997, 2024-09-16T00:08:14.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029688000000000002, 2024-09-16T00:08:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622017+0000", 2024-09-16T00:08:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3200000000000001e-07 2024-09-16T00:08:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:08:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6240000000000002e-06 2024-09-16T00:08:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622026+0000", 2024-09-16T00:08:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.708e-06 2024-09-16T00:08:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622032+0000", 2024-09-16T00:08:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4699999999999999e-06 2024-09-16T00:08:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622230+0000", 2024-09-16T00:08:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019783800000000001 2024-09-16T00:08:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622313+0000", 2024-09-16T00:08:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2908e-05 2024-09-16T00:08:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.950855238000003, 2024-09-16T00:08:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044651079999999998, 2024-09-16T00:08:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.653 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622463+0000", 2024-09-16T00:08:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5349999999999999e-06 2024-09-16T00:08:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622470+0000", 2024-09-16T00:08:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7050000000000002e-06 2024-09-16T00:08:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622485+0000", 2024-09-16T00:08:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4978e-05 2024-09-16T00:08:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622493+0000", 2024-09-16T00:08:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2139999999999996e-06 2024-09-16T00:08:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622546+0000", 2024-09-16T00:08:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:08:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622595+0000", 2024-09-16T00:08:14.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9112e-05 2024-09-16T00:08:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622677+0000", 2024-09-16T00:08:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2332999999999999e-05 2024-09-16T00:08:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.623515+0000", 2024-09-16T00:08:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000837615 2024-09-16T00:08:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:14.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626867+0000", 2024-09-16T00:08:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033524819999999999 2024-09-16T00:08:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626902+0000", 2024-09-16T00:08:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4310000000000002e-05 2024-09-16T00:08:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626927+0000", 2024-09-16T00:08:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5128999999999999e-05 2024-09-16T00:08:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.658 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:39 2.6 e22/19 2:654134d2:::mds0_openfiles.0:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 30.932225961, 2024-09-16T00:08:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082120900000000004, 2024-09-16T00:08:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:14.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641092+0000", 2024-09-16T00:08:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.141e-06 2024-09-16T00:08:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641102+0000", 2024-09-16T00:08:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5990000000000008e-06 2024-09-16T00:08:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641116+0000", 2024-09-16T00:08:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4667999999999999e-05 2024-09-16T00:08:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641125+0000", 2024-09-16T00:08:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0849999999999997e-06 2024-09-16T00:08:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641188+0000", 2024-09-16T00:08:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3119000000000003e-05 2024-09-16T00:08:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641209+0000", 2024-09-16T00:08:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0897e-05 2024-09-16T00:08:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641874+0000", 2024-09-16T00:08:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00066517200000000001 2024-09-16T00:08:14.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641912+0000", 2024-09-16T00:08:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8528000000000001e-05 2024-09-16T00:08:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:40 2.14 2:2e2fa760:::200.00000001:head [write 940~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.733989845, 2024-09-16T00:08:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016643039999999999, 2024-09-16T00:08:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839329+0000", 2024-09-16T00:08:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2330000000000001e-06 2024-09-16T00:08:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839343+0000", 2024-09-16T00:08:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3634e-05 2024-09-16T00:08:14.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839362+0000", 2024-09-16T00:08:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9321999999999999e-05 2024-09-16T00:08:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839375+0000", 2024-09-16T00:08:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2945000000000001e-05 2024-09-16T00:08:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839462+0000", 2024-09-16T00:08:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6371000000000004e-05 2024-09-16T00:08:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839547+0000", 2024-09-16T00:08:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4988999999999996e-05 2024-09-16T00:08:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839657+0000", 2024-09-16T00:08:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110501 2024-09-16T00:08:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:14.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840598+0000", 2024-09-16T00:08:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00094087100000000003 2024-09-16T00:08:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840918+0000", 2024-09-16T00:08:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031968799999999999 2024-09-16T00:08:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840955+0000", 2024-09-16T00:08:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7360999999999997e-05 2024-09-16T00:08:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840991+0000", 2024-09-16T00:08:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6389e-05 2024-09-16T00:08:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:41 2.14 2:2e2fa760:::200.00000001:head [write 1383~2831 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.460336399999999, 2024-09-16T00:08:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017075809999999999, 2024-09-16T00:08:14.669 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112983+0000", 2024-09-16T00:08:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2560000000000001e-06 2024-09-16T00:08:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112993+0000", 2024-09-16T00:08:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0423e-05 2024-09-16T00:08:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113010+0000", 2024-09-16T00:08:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7169000000000001e-05 2024-09-16T00:08:14.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113023+0000", 2024-09-16T00:08:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2111e-05 2024-09-16T00:08:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113099+0000", 2024-09-16T00:08:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6798000000000006e-05 2024-09-16T00:08:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:14.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113174+0000", 2024-09-16T00:08:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.47e-05 2024-09-16T00:08:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113297+0000", 2024-09-16T00:08:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012250799999999999 2024-09-16T00:08:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114284+0000", 2024-09-16T00:08:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00098720399999999999 2024-09-16T00:08:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114604+0000", 2024-09-16T00:08:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031988699999999998 2024-09-16T00:08:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:14.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114644+0000", 2024-09-16T00:08:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9919999999999997e-05 2024-09-16T00:08:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114688+0000", 2024-09-16T00:08:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4604999999999997e-05 2024-09-16T00:08:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:14.675 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:14.675 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:14.677 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:08:14.678 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:14.676+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:14.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:14.774+0000 7fa59affd640 1 -- 172.21.15.139:0/1873618472 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v33) ==== 101+0+0 (secure 0 0 0) 0x7fa59c05d620 con 0x7fa5a415e840 2024-09-16T00:08:14.775 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is unset 2024-09-16T00:08:14.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:14.776+0000 7fa5ac566640 1 -- 172.21.15.139:0/1873618472 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa58806c300 msgr2=0x7fa58806e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:14.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:14.776+0000 7fa5ac566640 1 --2- 172.21.15.139:0/1873618472 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa58806c300 0x7fa58806e7c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0034000 tx=0x7fa5a002e000 comp rx=0 tx=0).stop 2024-09-16T00:08:14.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:14.776+0000 7fa5ac566640 1 -- 172.21.15.139:0/1873618472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a415e840 msgr2=0x7fa5a414d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:14.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:14.776+0000 7fa5ac566640 1 --2- 172.21.15.139:0/1873618472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a415e840 0x7fa5a414d4f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00c6e0 tx=0x7fa59c00cba0 comp rx=0 tx=0).stop 2024-09-16T00:08:14.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:14.777+0000 7fa5ac566640 1 -- 172.21.15.139:0/1873618472 shutdown_connections 2024-09-16T00:08:14.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:14.777+0000 7fa5ac566640 1 --2- 172.21.15.139:0/1873618472 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa58806c300 0x7fa58806e7c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:14.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:14.777+0000 7fa5ac566640 1 --2- 172.21.15.139:0/1873618472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa5a415e840 0x7fa5a414d4f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:14.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:14.777+0000 7fa5ac566640 1 --2- 172.21.15.139:0/1873618472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa5a4154a10 0x7fa5a414cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:14.778 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:14.777+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:14.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:14.777+0000 7fa5ac566640 1 --2- 172.21.15.139:0/1873618472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa5a406ea70 0x7fa5a415a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:14.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:14.777+0000 7fa5ac566640 1 -- 172.21.15.139:0/1873618472 >> 172.21.15.139:0/1873618472 conn(0x7fa5a406cc30 msgr2=0x7fa5a4152230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:14.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:14.777+0000 7fa5ac566640 1 -- 172.21.15.139:0/1873618472 shutdown_connections 2024-09-16T00:08:14.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:14.777+0000 7fa5ac566640 1 -- 172.21.15.139:0/1873618472 wait complete. 2024-09-16T00:08:14.825 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:14.826 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:14.826 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:14.826 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:14.835 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:08:14.878 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:14.878+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:14.939 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:14.940 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:14.940 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:14.940 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:14.940 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:14.947 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:08:14.981 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:14.980+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:15.054 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:15.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.971447511999997, 2024-09-16T00:08:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:08:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.055 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:08:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:08:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:08:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:08:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:08:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:08:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:08:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:08:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:08:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:08:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:08:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:08:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:08:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:08:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:08:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:08:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:08:15.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:08:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:08:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:08:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:08:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:08:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:08:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:08:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:08:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:08:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:08:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:08:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.061 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.891915881999999, 2024-09-16T00:08:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093071190000000008, 2024-09-16T00:08:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.062 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 106 2024-09-16T00:08:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160362+0000", 2024-09-16T00:08:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160438+0000", 2024-09-16T00:08:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6711000000000005e-05 2024-09-16T00:08:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160444+0000", 2024-09-16T00:08:15.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0530000000000003e-06 2024-09-16T00:08:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160449+0000", 2024-09-16T00:08:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2189999999999998e-06 2024-09-16T00:08:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161270+0000", 2024-09-16T00:08:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082099199999999999 2024-09-16T00:08:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161348+0000", 2024-09-16T00:08:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8361000000000004e-05 2024-09-16T00:08:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161472+0000", 2024-09-16T00:08:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123523 2024-09-16T00:08:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163461+0000", 2024-09-16T00:08:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019891560000000002 2024-09-16T00:08:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169418+0000", 2024-09-16T00:08:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059568490000000002 2024-09-16T00:08:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169570+0000", 2024-09-16T00:08:15.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152461 2024-09-16T00:08:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169668+0000", 2024-09-16T00:08:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T00:08:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:15.068 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.891820111000001, 2024-09-16T00:08:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092931679999999992, 2024-09-16T00:08:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107 2024-09-16T00:08:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0699999999999998e-07 2024-09-16T00:08:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160459+0000", 2024-09-16T00:08:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0370000000000001e-06 2024-09-16T00:08:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160463+0000", 2024-09-16T00:08:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999999e-06 2024-09-16T00:08:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160466+0000", 2024-09-16T00:08:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.644e-06 2024-09-16T00:08:15.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161873+0000", 2024-09-16T00:08:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014074840000000001 2024-09-16T00:08:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161949+0000", 2024-09-16T00:08:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5749999999999998e-05 2024-09-16T00:08:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162069+0000", 2024-09-16T00:08:15.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120257 2024-09-16T00:08:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163506+0000", 2024-09-16T00:08:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001436344 2024-09-16T00:08:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169704+0000", 2024-09-16T00:08:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061982829999999997 2024-09-16T00:08:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169730+0000", 2024-09-16T00:08:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6060000000000001e-05 2024-09-16T00:08:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169750+0000", 2024-09-16T00:08:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0092e-05 2024-09-16T00:08:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.074 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.891710799000002, 2024-09-16T00:08:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092280039999999997, 2024-09-16T00:08:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 108 2024-09-16T00:08:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160567+0000", 2024-09-16T00:08:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7300000000000001e-07 2024-09-16T00:08:15.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160569+0000", 2024-09-16T00:08:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5299999999999999e-06 2024-09-16T00:08:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160574+0000", 2024-09-16T00:08:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-16T00:08:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160577+0000", 2024-09-16T00:08:15.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.01e-06 2024-09-16T00:08:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162518+0000", 2024-09-16T00:08:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001940703 2024-09-16T00:08:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162571+0000", 2024-09-16T00:08:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2670000000000002e-05 2024-09-16T00:08:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162647+0000", 2024-09-16T00:08:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6778000000000002e-05 2024-09-16T00:08:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163544+0000", 2024-09-16T00:08:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089702599999999999 2024-09-16T00:08:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169754+0000", 2024-09-16T00:08:15.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062091739999999996 2024-09-16T00:08:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169776+0000", 2024-09-16T00:08:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2391999999999998e-05 2024-09-16T00:08:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169794+0000", 2024-09-16T00:08:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8318999999999999e-05 2024-09-16T00:08:15.080 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.891662248999999, 2024-09-16T00:08:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092220679999999999, 2024-09-16T00:08:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.081 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 109 2024-09-16T00:08:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9900000000000001e-07 2024-09-16T00:08:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160707+0000", 2024-09-16T00:08:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1621999999999998e-05 2024-09-16T00:08:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160713+0000", 2024-09-16T00:08:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.596e-06 2024-09-16T00:08:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160717+0000", 2024-09-16T00:08:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5829999999999998e-06 2024-09-16T00:08:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162869+0000", 2024-09-16T00:08:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021516310000000002 2024-09-16T00:08:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162946+0000", 2024-09-16T00:08:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7696999999999998e-05 2024-09-16T00:08:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163065+0000", 2024-09-16T00:08:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118581 2024-09-16T00:08:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164431+0000", 2024-09-16T00:08:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001366003 2024-09-16T00:08:15.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169797+0000", 2024-09-16T00:08:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005366244 2024-09-16T00:08:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169819+0000", 2024-09-16T00:08:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2036999999999999e-05 2024-09-16T00:08:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169837+0000", 2024-09-16T00:08:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7575e-05 2024-09-16T00:08:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.891552900000001, 2024-09-16T00:08:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092871919999999997, 2024-09-16T00:08:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.087 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 110 2024-09-16T00:08:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:08:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160728+0000", 2024-09-16T00:08:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0620000000000001e-06 2024-09-16T00:08:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160731+0000", 2024-09-16T00:08:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6500000000000002e-06 2024-09-16T00:08:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160734+0000", 2024-09-16T00:08:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.762e-06 2024-09-16T00:08:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163580+0000", 2024-09-16T00:08:15.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028464549999999999 2024-09-16T00:08:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163657+0000", 2024-09-16T00:08:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6695999999999999e-05 2024-09-16T00:08:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163798+0000", 2024-09-16T00:08:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014055 2024-09-16T00:08:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165483+0000", 2024-09-16T00:08:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016853090000000001 2024-09-16T00:08:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169970+0000", 2024-09-16T00:08:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00448709 2024-09-16T00:08:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169994+0000", 2024-09-16T00:08:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3575999999999999e-05 2024-09-16T00:08:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170011+0000", 2024-09-16T00:08:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7702000000000001e-05 2024-09-16T00:08:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.891539231000003, 2024-09-16T00:08:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093174859999999998, 2024-09-16T00:08:15.093 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 111 2024-09-16T00:08:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1e-07 2024-09-16T00:08:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160741+0000", 2024-09-16T00:08:15.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6110000000000001e-06 2024-09-16T00:08:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160744+0000", 2024-09-16T00:08:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3299999999999999e-06 2024-09-16T00:08:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160746+0000", 2024-09-16T00:08:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5469999999999999e-06 2024-09-16T00:08:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164069+0000", 2024-09-16T00:08:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033222709999999999 2024-09-16T00:08:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164113+0000", 2024-09-16T00:08:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4212999999999998e-05 2024-09-16T00:08:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164187+0000", 2024-09-16T00:08:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4136999999999998e-05 2024-09-16T00:08:15.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166429+0000", 2024-09-16T00:08:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022423180000000001 2024-09-16T00:08:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170016+0000", 2024-09-16T00:08:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035869449999999998 2024-09-16T00:08:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170038+0000", 2024-09-16T00:08:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2053e-05 2024-09-16T00:08:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170055+0000", 2024-09-16T00:08:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6850999999999998e-05 2024-09-16T00:08:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:15.099 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.891525516000002, 2024-09-16T00:08:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093480279999999995, 2024-09-16T00:08:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 112 2024-09-16T00:08:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160752+0000", 2024-09-16T00:08:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2700000000000001e-07 2024-09-16T00:08:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160808+0000", 2024-09-16T00:08:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5797999999999997e-05 2024-09-16T00:08:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160814+0000", 2024-09-16T00:08:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5980000000000002e-06 2024-09-16T00:08:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160819+0000", 2024-09-16T00:08:15.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8960000000000002e-06 2024-09-16T00:08:15.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164468+0000", 2024-09-16T00:08:15.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003649476 2024-09-16T00:08:15.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164525+0000", 2024-09-16T00:08:15.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6227999999999998e-05 2024-09-16T00:08:15.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164595+0000", 2024-09-16T00:08:15.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759000000000005e-05 2024-09-16T00:08:15.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167237+0000", 2024-09-16T00:08:15.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026410470000000001 2024-09-16T00:08:15.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170060+0000", 2024-09-16T00:08:15.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028234969999999999 2024-09-16T00:08:15.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170081+0000", 2024-09-16T00:08:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1242000000000001e-05 2024-09-16T00:08:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170100+0000", 2024-09-16T00:08:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8260000000000001e-05 2024-09-16T00:08:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.105 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.891448421, 2024-09-16T00:08:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093139409999999992, 2024-09-16T00:08:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 113 2024-09-16T00:08:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-07 2024-09-16T00:08:15.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160833+0000", 2024-09-16T00:08:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.557e-06 2024-09-16T00:08:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160839+0000", 2024-09-16T00:08:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3829999999999997e-06 2024-09-16T00:08:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160844+0000", 2024-09-16T00:08:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6909999999999996e-06 2024-09-16T00:08:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165008+0000", 2024-09-16T00:08:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041642129999999999 2024-09-16T00:08:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165085+0000", 2024-09-16T00:08:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6651999999999997e-05 2024-09-16T00:08:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165205+0000", 2024-09-16T00:08:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012045900000000001 2024-09-16T00:08:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167283+0000", 2024-09-16T00:08:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020782909999999999 2024-09-16T00:08:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170104+0000", 2024-09-16T00:08:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028207950000000001 2024-09-16T00:08:15.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170125+0000", 2024-09-16T00:08:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0852e-05 2024-09-16T00:08:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170143+0000", 2024-09-16T00:08:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7467000000000001e-05 2024-09-16T00:08:15.111 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.891401938999998, 2024-09-16T00:08:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093126960000000005, 2024-09-16T00:08:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.112 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-16T00:08:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160876+0000", 2024-09-16T00:08:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5700000000000002e-07 2024-09-16T00:08:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160880+0000", 2024-09-16T00:08:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5539999999999999e-06 2024-09-16T00:08:15.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160888+0000", 2024-09-16T00:08:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.157e-06 2024-09-16T00:08:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160894+0000", 2024-09-16T00:08:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7690000000000002e-06 2024-09-16T00:08:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165519+0000", 2024-09-16T00:08:15.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046249539999999997 2024-09-16T00:08:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165593+0000", 2024-09-16T00:08:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4039e-05 2024-09-16T00:08:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165715+0000", 2024-09-16T00:08:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012131899999999999 2024-09-16T00:08:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167309+0000", 2024-09-16T00:08:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001594011 2024-09-16T00:08:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170147+0000", 2024-09-16T00:08:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028387130000000001 2024-09-16T00:08:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170170+0000", 2024-09-16T00:08:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2517000000000001e-05 2024-09-16T00:08:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170188+0000", 2024-09-16T00:08:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8006e-05 2024-09-16T00:08:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.891360911, 2024-09-16T00:08:15.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093140040000000007, 2024-09-16T00:08:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.119 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-16T00:08:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160917+0000", 2024-09-16T00:08:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7599999999999996e-07 2024-09-16T00:08:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160992+0000", 2024-09-16T00:08:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4760999999999998e-05 2024-09-16T00:08:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161000+0000", 2024-09-16T00:08:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9829999999999996e-06 2024-09-16T00:08:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161007+0000", 2024-09-16T00:08:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4909999999999999e-06 2024-09-16T00:08:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165983+0000", 2024-09-16T00:08:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049755399999999997 2024-09-16T00:08:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166027+0000", 2024-09-16T00:08:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4109000000000001e-05 2024-09-16T00:08:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166100+0000", 2024-09-16T00:08:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3360000000000002e-05 2024-09-16T00:08:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169698+0000", 2024-09-16T00:08:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597795 2024-09-16T00:08:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170192+0000", 2024-09-16T00:08:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049447300000000005 2024-09-16T00:08:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170214+0000", 2024-09-16T00:08:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1546e-05 2024-09-16T00:08:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170230+0000", 2024-09-16T00:08:15.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6169999999999999e-05 2024-09-16T00:08:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.891251181999998, 2024-09-16T00:08:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092457029999999992, 2024-09-16T00:08:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.126 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-16T00:08:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:15.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7899999999999999e-07 2024-09-16T00:08:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161029+0000", 2024-09-16T00:08:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6630000000000002e-06 2024-09-16T00:08:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161036+0000", 2024-09-16T00:08:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T00:08:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161041+0000", 2024-09-16T00:08:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:08:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166468+0000", 2024-09-16T00:08:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054266050000000001 2024-09-16T00:08:15.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166511+0000", 2024-09-16T00:08:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3869999999999998e-05 2024-09-16T00:08:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166606+0000", 2024-09-16T00:08:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4437000000000004e-05 2024-09-16T00:08:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169955+0000", 2024-09-16T00:08:15.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033494810000000001 2024-09-16T00:08:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170234+0000", 2024-09-16T00:08:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027900899999999998 2024-09-16T00:08:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170255+0000", 2024-09-16T00:08:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0537999999999999e-05 2024-09-16T00:08:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170272+0000", 2024-09-16T00:08:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6611000000000001e-05 2024-09-16T00:08:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.891226037000003, 2024-09-16T00:08:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093511010000000006, 2024-09-16T00:08:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.132 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-16T00:08:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.63e-07 2024-09-16T00:08:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161055+0000", 2024-09-16T00:08:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6770000000000001e-06 2024-09-16T00:08:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161063+0000", 2024-09-16T00:08:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6469999999999998e-06 2024-09-16T00:08:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161068+0000", 2024-09-16T00:08:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.746e-06 2024-09-16T00:08:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166817+0000", 2024-09-16T00:08:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057494959999999998 2024-09-16T00:08:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166893+0000", 2024-09-16T00:08:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6210000000000004e-05 2024-09-16T00:08:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167019+0000", 2024-09-16T00:08:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125585 2024-09-16T00:08:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170275+0000", 2024-09-16T00:08:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032563620000000001 2024-09-16T00:08:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170309+0000", 2024-09-16T00:08:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3865999999999997e-05 2024-09-16T00:08:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170362+0000", 2024-09-16T00:08:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3094999999999999e-05 2024-09-16T00:08:15.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170402+0000", 2024-09-16T00:08:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9953999999999997e-05 2024-09-16T00:08:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.442493952, 2024-09-16T00:08:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097364389999999995, 2024-09-16T00:08:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.138 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138 2024-09-16T00:08:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609784+0000", 2024-09-16T00:08:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T00:08:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609787+0000", 2024-09-16T00:08:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280000000000001e-06 2024-09-16T00:08:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609794+0000", 2024-09-16T00:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6409999999999996e-06 2024-09-16T00:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609799+0000", 2024-09-16T00:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9359999999999998e-06 2024-09-16T00:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615759+0000", 2024-09-16T00:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059599910000000004 2024-09-16T00:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615806+0000", 2024-09-16T00:08:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7290000000000003e-05 2024-09-16T00:08:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615877+0000", 2024-09-16T00:08:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1000000000000005e-05 2024-09-16T00:08:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618961+0000", 2024-09-16T00:08:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030841390000000001 2024-09-16T00:08:15.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619436+0000", 2024-09-16T00:08:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047452799999999999 2024-09-16T00:08:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619482+0000", 2024-09-16T00:08:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5689000000000003e-05 2024-09-16T00:08:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619519+0000", 2024-09-16T00:08:15.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7938000000000002e-05 2024-09-16T00:08:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.442470237999999, 2024-09-16T00:08:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011812954000000001, 2024-09-16T00:08:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.144 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139 2024-09-16T00:08:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999998e-07 2024-09-16T00:08:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609810+0000", 2024-09-16T00:08:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2100000000000002e-06 2024-09-16T00:08:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609817+0000", 2024-09-16T00:08:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.28e-06 2024-09-16T00:08:15.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609825+0000", 2024-09-16T00:08:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616116+0000", 2024-09-16T00:08:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006290543 2024-09-16T00:08:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616187+0000", 2024-09-16T00:08:15.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1830999999999997e-05 2024-09-16T00:08:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616298+0000", 2024-09-16T00:08:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011044699999999999 2024-09-16T00:08:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619559+0000", 2024-09-16T00:08:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032614710000000002 2024-09-16T00:08:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621562+0000", 2024-09-16T00:08:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020028680000000001 2024-09-16T00:08:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621597+0000", 2024-09-16T00:08:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4625000000000001e-05 2024-09-16T00:08:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621620+0000", 2024-09-16T00:08:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2864e-05 2024-09-16T00:08:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.442442442000001, 2024-09-16T00:08:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011828211999999999, 2024-09-16T00:08:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 140 2024-09-16T00:08:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.150 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T00:08:15.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961000000000003e-05 2024-09-16T00:08:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609892+0000", 2024-09-16T00:08:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0516000000000001e-05 2024-09-16T00:08:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609898+0000", 2024-09-16T00:08:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0449999999999997e-06 2024-09-16T00:08:15.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616797+0000", 2024-09-16T00:08:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068998840000000002 2024-09-16T00:08:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616844+0000", 2024-09-16T00:08:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6501000000000001e-05 2024-09-16T00:08:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616935+0000", 2024-09-16T00:08:15.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1222000000000002e-05 2024-09-16T00:08:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619808+0000", 2024-09-16T00:08:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873198 2024-09-16T00:08:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621624+0000", 2024-09-16T00:08:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001815933 2024-09-16T00:08:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621646+0000", 2024-09-16T00:08:15.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1622999999999999e-05 2024-09-16T00:08:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621663+0000", 2024-09-16T00:08:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6854000000000002e-05 2024-09-16T00:08:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.442356633000003, 2024-09-16T00:08:15.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014331501999999999, 2024-09-16T00:08:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-16T00:08:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:15.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609921+0000", 2024-09-16T00:08:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9599999999999999e-07 2024-09-16T00:08:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609925+0000", 2024-09-16T00:08:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T00:08:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609932+0000", 2024-09-16T00:08:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8240000000000002e-06 2024-09-16T00:08:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609937+0000", 2024-09-16T00:08:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1070000000000004e-06 2024-09-16T00:08:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618987+0000", 2024-09-16T00:08:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090502719999999998 2024-09-16T00:08:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619064+0000", 2024-09-16T00:08:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6868999999999999e-05 2024-09-16T00:08:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619183+0000", 2024-09-16T00:08:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011961 2024-09-16T00:08:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.620052+0000", 2024-09-16T00:08:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086822799999999997 2024-09-16T00:08:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624197+0000", 2024-09-16T00:08:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041454930000000001 2024-09-16T00:08:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624229+0000", 2024-09-16T00:08:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1758999999999999e-05 2024-09-16T00:08:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624252+0000", 2024-09-16T00:08:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:08:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:15.161 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 49.969590824999997, 2024-09-16T00:08:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010093276999999999, 2024-09-16T00:08:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 157 2024-09-16T00:08:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:15.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4699999999999998e-07 2024-09-16T00:08:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082688+0000", 2024-09-16T00:08:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0209999999999998e-06 2024-09-16T00:08:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082692+0000", 2024-09-16T00:08:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6890000000000002e-06 2024-09-16T00:08:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082695+0000", 2024-09-16T00:08:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5339999999999998e-06 2024-09-16T00:08:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085303+0000", 2024-09-16T00:08:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026084789999999999 2024-09-16T00:08:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085374+0000", 2024-09-16T00:08:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1317999999999997e-05 2024-09-16T00:08:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085506+0000", 2024-09-16T00:08:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013118 2024-09-16T00:08:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086928+0000", 2024-09-16T00:08:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014224559999999999 2024-09-16T00:08:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092665+0000", 2024-09-16T00:08:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057369830000000002 2024-09-16T00:08:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092727+0000", 2024-09-16T00:08:15.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1810000000000006e-05 2024-09-16T00:08:15.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092779+0000", 2024-09-16T00:08:15.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2559999999999998e-05 2024-09-16T00:08:15.167 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.167 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.167 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.167 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:15.168 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:15.168 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 49.969578657, 2024-09-16T00:08:15.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010176388, 2024-09-16T00:08:15.168 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.168 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.168 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.168 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.168 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 158 2024-09-16T00:08:15.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.168 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:15.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:15.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082699+0000", 2024-09-16T00:08:15.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.11e-07 2024-09-16T00:08:15.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082742+0000", 2024-09-16T00:08:15.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3699000000000003e-05 2024-09-16T00:08:15.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082748+0000", 2024-09-16T00:08:15.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8690000000000001e-06 2024-09-16T00:08:15.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082751+0000", 2024-09-16T00:08:15.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.942e-06 2024-09-16T00:08:15.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085881+0000", 2024-09-16T00:08:15.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031303450000000001 2024-09-16T00:08:15.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085959+0000", 2024-09-16T00:08:15.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7874999999999996e-05 2024-09-16T00:08:15.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086086+0000", 2024-09-16T00:08:15.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012685499999999999 2024-09-16T00:08:15.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087127+0000", 2024-09-16T00:08:15.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001041298 2024-09-16T00:08:15.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092790+0000", 2024-09-16T00:08:15.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056628 2024-09-16T00:08:15.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092832+0000", 2024-09-16T00:08:15.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2227000000000001e-05 2024-09-16T00:08:15.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092875+0000", 2024-09-16T00:08:15.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2267e-05 2024-09-16T00:08:15.173 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.173 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.173 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.173 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:15.173 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:15.174 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 49.969520619000001, 2024-09-16T00:08:15.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010199547, 2024-09-16T00:08:15.174 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:15.174 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:15.174 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.174 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:15.174 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:15.174 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:15.174 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159 2024-09-16T00:08:15.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.174 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:15.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:15.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:15.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:15.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:15.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:15.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:15.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082757+0000", 2024-09-16T00:08:15.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2500000000000001e-07 2024-09-16T00:08:15.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:15.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082759+0000", 2024-09-16T00:08:15.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-16T00:08:15.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:15.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082763+0000", 2024-09-16T00:08:15.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4889999999999997e-06 2024-09-16T00:08:15.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082766+0000", 2024-09-16T00:08:15.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:08:15.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:15.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086571+0000", 2024-09-16T00:08:15.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038053269999999998 2024-09-16T00:08:15.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:15.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086619+0000", 2024-09-16T00:08:15.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7924999999999999e-05 2024-09-16T00:08:15.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:15.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086693+0000", 2024-09-16T00:08:15.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000005e-05 2024-09-16T00:08:15.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087383+0000", 2024-09-16T00:08:15.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069008299999999997 2024-09-16T00:08:15.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:15.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092882+0000", 2024-09-16T00:08:15.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054987450000000002 2024-09-16T00:08:15.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:15.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092921+0000", 2024-09-16T00:08:15.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9388999999999998e-05 2024-09-16T00:08:15.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:15.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:15.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:15.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092956+0000", 2024-09-16T00:08:15.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4647999999999998e-05 2024-09-16T00:08:15.179 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.179 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.179 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.179 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:15.179 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:15.179 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:15.181 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:08:15.181 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:15.180+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:15.281 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:15.280+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:15.289 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:15.289 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:15.289 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:15.289 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:15.299 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:08:15.382 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:15.381+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:15.411 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:15.411 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:15.411 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:15.411 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:15.412 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:15.421 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:08:15.483 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:15.482+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:15.528 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'41, mlcod=17'39)", 2024-09-16T00:08:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.505798853999998, 2024-09-16T00:08:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001808488, 2024-09-16T00:08:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021032+0000", 2024-09-16T00:08:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3e-07 2024-09-16T00:08:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021037+0000", 2024-09-16T00:08:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0640000000000001e-06 2024-09-16T00:08:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021047+0000", 2024-09-16T00:08:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.023e-05 2024-09-16T00:08:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021053+0000", 2024-09-16T00:08:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3160000000000002e-06 2024-09-16T00:08:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021092+0000", 2024-09-16T00:08:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9033999999999999e-05 2024-09-16T00:08:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021105+0000", 2024-09-16T00:08:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2768e-05 2024-09-16T00:08:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022811+0000", 2024-09-16T00:08:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001705788 2024-09-16T00:08:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022840+0000", 2024-09-16T00:08:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8558e-05 2024-09-16T00:08:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:85 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'55, mlcod=17'50)", 2024-09-16T00:08:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.492769852999999, 2024-09-16T00:08:15.536 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017909530000000001, 2024-09-16T00:08:15.536 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.536 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.536 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.536 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.536 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.536 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.536 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:15.536 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.536 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.537 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.537 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.537 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:15.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.537 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.537 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.537 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.537 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034061+0000", 2024-09-16T00:08:15.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0399999999999996e-07 2024-09-16T00:08:15.537 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.538 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.538 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.538 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034065+0000", 2024-09-16T00:08:15.538 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8840000000000002e-06 2024-09-16T00:08:15.538 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.538 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.538 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.538 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034074+0000", 2024-09-16T00:08:15.538 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:15.539 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.539 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.539 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.539 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034079+0000", 2024-09-16T00:08:15.539 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0000000000000004e-06 2024-09-16T00:08:15.539 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.539 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.539 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.539 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034125+0000", 2024-09-16T00:08:15.540 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5905000000000002e-05 2024-09-16T00:08:15.540 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.540 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.540 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.540 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034144+0000", 2024-09-16T00:08:15.540 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9593e-05 2024-09-16T00:08:15.540 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.540 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.540 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.540 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035817+0000", 2024-09-16T00:08:15.541 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016728240000000001 2024-09-16T00:08:15.541 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.541 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.541 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.541 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035851+0000", 2024-09-16T00:08:15.541 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3809000000000001e-05 2024-09-16T00:08:15.541 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.541 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.541 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.541 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.542 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:112 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'74, mlcod=17'65)", 2024-09-16T00:08:15.542 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:15.542 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.362157990999997, 2024-09-16T00:08:15.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018457020000000001, 2024-09-16T00:08:15.542 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.542 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.542 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.542 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.542 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.542 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:15.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.543 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.543 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:15.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.543 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.543 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164673+0000", 2024-09-16T00:08:15.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1200000000000003e-07 2024-09-16T00:08:15.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.543 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.543 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164710+0000", 2024-09-16T00:08:15.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7756999999999998e-05 2024-09-16T00:08:15.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164719+0000", 2024-09-16T00:08:15.544 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2849999999999995e-06 2024-09-16T00:08:15.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164723+0000", 2024-09-16T00:08:15.544 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3220000000000003e-06 2024-09-16T00:08:15.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.545 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164760+0000", 2024-09-16T00:08:15.545 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7311999999999998e-05 2024-09-16T00:08:15.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.545 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.545 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.545 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164771+0000", 2024-09-16T00:08:15.545 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0869e-05 2024-09-16T00:08:15.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.545 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.545 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166495+0000", 2024-09-16T00:08:15.546 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001723863 2024-09-16T00:08:15.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.546 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.546 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166518+0000", 2024-09-16T00:08:15.546 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2682000000000001e-05 2024-09-16T00:08:15.546 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.546 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.546 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.546 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.547 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'66)", 2024-09-16T00:08:15.547 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:15.547 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.360654011999998, 2024-09-16T00:08:15.547 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032925509999999999, 2024-09-16T00:08:15.547 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.547 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.547 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.547 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.547 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.547 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.548 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:15.548 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.548 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.548 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:15.548 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.548 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.548 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166177+0000", 2024-09-16T00:08:15.548 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:15.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.549 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166216+0000", 2024-09-16T00:08:15.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9008000000000003e-05 2024-09-16T00:08:15.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.549 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166226+0000", 2024-09-16T00:08:15.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0081999999999999e-05 2024-09-16T00:08:15.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166230+0000", 2024-09-16T00:08:15.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5079999999999999e-06 2024-09-16T00:08:15.550 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.550 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166259+0000", 2024-09-16T00:08:15.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8928000000000002e-05 2024-09-16T00:08:15.550 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.550 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166271+0000", 2024-09-16T00:08:15.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1531e-05 2024-09-16T00:08:15.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.551 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.551 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.551 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169449+0000", 2024-09-16T00:08:15.551 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003178143 2024-09-16T00:08:15.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.551 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.551 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.551 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169468+0000", 2024-09-16T00:08:15.551 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9700000000000001e-05 2024-09-16T00:08:15.551 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.552 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.552 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.552 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.552 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.552 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'66)", 2024-09-16T00:08:15.552 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:15.552 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.360192552999997, 2024-09-16T00:08:15.552 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003179374, 2024-09-16T00:08:15.552 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.552 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.552 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.552 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.553 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:15.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.553 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:15.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.554 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166638+0000", 2024-09-16T00:08:15.554 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2300000000000002e-07 2024-09-16T00:08:15.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.554 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.554 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166641+0000", 2024-09-16T00:08:15.554 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0079999999999998e-06 2024-09-16T00:08:15.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.554 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.554 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166646+0000", 2024-09-16T00:08:15.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4970000000000001e-06 2024-09-16T00:08:15.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.555 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.555 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.555 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166650+0000", 2024-09-16T00:08:15.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5729999999999999e-06 2024-09-16T00:08:15.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.555 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.555 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.555 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166682+0000", 2024-09-16T00:08:15.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2218999999999998e-05 2024-09-16T00:08:15.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.555 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.556 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.556 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166694+0000", 2024-09-16T00:08:15.556 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1902999999999999e-05 2024-09-16T00:08:15.556 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.556 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.556 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.556 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169784+0000", 2024-09-16T00:08:15.556 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003089545 2024-09-16T00:08:15.556 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.556 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169817+0000", 2024-09-16T00:08:15.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3206e-05 2024-09-16T00:08:15.557 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.557 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.557 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.557 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:117 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'66)", 2024-09-16T00:08:15.557 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:15.557 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.359748492999998, 2024-09-16T00:08:15.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00275326, 2024-09-16T00:08:15.557 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.557 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.558 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.558 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.558 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:15.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.558 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:15.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.559 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167082+0000", 2024-09-16T00:08:15.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:08:15.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.559 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167086+0000", 2024-09-16T00:08:15.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1559999999999997e-06 2024-09-16T00:08:15.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.559 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167096+0000", 2024-09-16T00:08:15.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8390000000000002e-06 2024-09-16T00:08:15.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167101+0000", 2024-09-16T00:08:15.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:08:15.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167130+0000", 2024-09-16T00:08:15.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9649e-05 2024-09-16T00:08:15.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167142+0000", 2024-09-16T00:08:15.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1435e-05 2024-09-16T00:08:15.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.561 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169818+0000", 2024-09-16T00:08:15.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002676056 2024-09-16T00:08:15.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.561 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169835+0000", 2024-09-16T00:08:15.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6980999999999999e-05 2024-09-16T00:08:15.561 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.561 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.561 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.562 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.562 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.562 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:130 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'88, mlcod=17'85)", 2024-09-16T00:08:15.562 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:15.563 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 50.916949164000002, 2024-09-16T00:08:15.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002880456, 2024-09-16T00:08:15.563 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.563 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.563 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.563 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.563 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.563 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:15.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:15.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.564 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:15.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5699999999999998e-07 2024-09-16T00:08:15.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.564 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609885+0000", 2024-09-16T00:08:15.565 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.608e-06 2024-09-16T00:08:15.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.565 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.565 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.565 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609891+0000", 2024-09-16T00:08:15.565 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:15.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.565 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.565 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.565 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609895+0000", 2024-09-16T00:08:15.565 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.54e-06 2024-09-16T00:08:15.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.566 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.566 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609931+0000", 2024-09-16T00:08:15.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5893999999999999e-05 2024-09-16T00:08:15.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.566 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609945+0000", 2024-09-16T00:08:15.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4259e-05 2024-09-16T00:08:15.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612721+0000", 2024-09-16T00:08:15.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027757849999999998 2024-09-16T00:08:15.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612761+0000", 2024-09-16T00:08:15.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0638e-05 2024-09-16T00:08:15.568 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.568 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.568 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.568 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-16T00:08:15.568 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:15.568 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 50.916565190999997, 2024-09-16T00:08:15.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028016529999999999, 2024-09-16T00:08:15.568 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.568 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.568 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.568 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:15.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:15.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:15.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.63e-07 2024-09-16T00:08:15.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610344+0000", 2024-09-16T00:08:15.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8996000000000001e-05 2024-09-16T00:08:15.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610352+0000", 2024-09-16T00:08:15.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:15.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.571 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610356+0000", 2024-09-16T00:08:15.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0790000000000002e-06 2024-09-16T00:08:15.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.571 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.571 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610388+0000", 2024-09-16T00:08:15.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1541000000000002e-05 2024-09-16T00:08:15.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.571 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610400+0000", 2024-09-16T00:08:15.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2075e-05 2024-09-16T00:08:15.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613045+0000", 2024-09-16T00:08:15.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002645213 2024-09-16T00:08:15.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613066+0000", 2024-09-16T00:08:15.573 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1353e-05 2024-09-16T00:08:15.573 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.573 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.573 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.573 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.573 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.573 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-16T00:08:15.573 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:15.573 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 50.913305690999998, 2024-09-16T00:08:15.573 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027254850000000001, 2024-09-16T00:08:15.573 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.574 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.574 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.574 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.574 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:15.574 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.574 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:15.574 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613525+0000", 2024-09-16T00:08:15.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2800000000000002e-07 2024-09-16T00:08:15.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613528+0000", 2024-09-16T00:08:15.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9730000000000002e-06 2024-09-16T00:08:15.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.576 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.576 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.576 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613534+0000", 2024-09-16T00:08:15.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4339999999999997e-06 2024-09-16T00:08:15.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.576 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.576 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.576 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613538+0000", 2024-09-16T00:08:15.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5149999999999998e-06 2024-09-16T00:08:15.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613566+0000", 2024-09-16T00:08:15.577 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8104000000000001e-05 2024-09-16T00:08:15.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613578+0000", 2024-09-16T00:08:15.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2404e-05 2024-09-16T00:08:15.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616217+0000", 2024-09-16T00:08:15.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026389830000000001 2024-09-16T00:08:15.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616250+0000", 2024-09-16T00:08:15.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2644000000000001e-05 2024-09-16T00:08:15.579 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.579 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.579 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.579 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:134 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'92, mlcod=18'87)", 2024-09-16T00:08:15.579 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:15.579 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 50.912767551999998, 2024-09-16T00:08:15.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025322399999999998, 2024-09-16T00:08:15.579 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.579 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.579 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.579 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:15.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:15.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614063+0000", 2024-09-16T00:08:15.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8700000000000005e-07 2024-09-16T00:08:15.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.581 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.581 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614096+0000", 2024-09-16T00:08:15.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3136000000000003e-05 2024-09-16T00:08:15.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.581 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.581 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614106+0000", 2024-09-16T00:08:15.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9170000000000001e-06 2024-09-16T00:08:15.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.581 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.582 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614111+0000", 2024-09-16T00:08:15.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4390000000000002e-06 2024-09-16T00:08:15.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.582 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.582 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614141+0000", 2024-09-16T00:08:15.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.04e-05 2024-09-16T00:08:15.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.582 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.582 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614152+0000", 2024-09-16T00:08:15.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0987e-05 2024-09-16T00:08:15.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.583 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616573+0000", 2024-09-16T00:08:15.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002421351 2024-09-16T00:08:15.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.583 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616595+0000", 2024-09-16T00:08:15.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1322999999999999e-05 2024-09-16T00:08:15.583 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.583 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.583 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.584 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:135 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'93, mlcod=18'87)", 2024-09-16T00:08:15.584 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:15.584 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 50.912247514000001, 2024-09-16T00:08:15.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020304659999999999, 2024-09-16T00:08:15.584 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.584 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.584 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.584 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:15.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:15.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614583+0000", 2024-09-16T00:08:15.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:15.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614587+0000", 2024-09-16T00:08:15.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3100000000000002e-06 2024-09-16T00:08:15.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614597+0000", 2024-09-16T00:08:15.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7450000000000007e-06 2024-09-16T00:08:15.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.587 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.587 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614602+0000", 2024-09-16T00:08:15.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6099999999999999e-06 2024-09-16T00:08:15.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.587 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.587 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614641+0000", 2024-09-16T00:08:15.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9767000000000001e-05 2024-09-16T00:08:15.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.587 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614658+0000", 2024-09-16T00:08:15.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6205000000000002e-05 2024-09-16T00:08:15.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616596+0000", 2024-09-16T00:08:15.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019386659999999999 2024-09-16T00:08:15.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616613+0000", 2024-09-16T00:08:15.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6555999999999999e-05 2024-09-16T00:08:15.589 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.589 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.589 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.589 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:139 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'97, mlcod=18'87)", 2024-09-16T00:08:15.589 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:15.589 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 50.910486843000001, 2024-09-16T00:08:15.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030908139999999999, 2024-09-16T00:08:15.590 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.590 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.590 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.590 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.590 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:15.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.590 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:15.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616344+0000", 2024-09-16T00:08:15.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3000000000000001e-07 2024-09-16T00:08:15.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616347+0000", 2024-09-16T00:08:15.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7740000000000002e-06 2024-09-16T00:08:15.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616355+0000", 2024-09-16T00:08:15.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4699999999999996e-06 2024-09-16T00:08:15.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.592 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.592 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616358+0000", 2024-09-16T00:08:15.592 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5899999999999999e-06 2024-09-16T00:08:15.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.592 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.592 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616393+0000", 2024-09-16T00:08:15.592 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4529000000000001e-05 2024-09-16T00:08:15.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.592 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.593 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616404+0000", 2024-09-16T00:08:15.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0726e-05 2024-09-16T00:08:15.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.593 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.593 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.593 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619411+0000", 2024-09-16T00:08:15.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030076249999999999 2024-09-16T00:08:15.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.593 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.593 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.594 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619434+0000", 2024-09-16T00:08:15.594 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-05 2024-09-16T00:08:15.594 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.594 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.594 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.594 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.594 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.594 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:140 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'98, mlcod=18'89)", 2024-09-16T00:08:15.594 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:15.594 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 50.909845449999999, 2024-09-16T00:08:15.594 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027276150000000001, 2024-09-16T00:08:15.594 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.594 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.595 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.595 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.595 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.595 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.595 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:15.595 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.595 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.595 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.595 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:15.595 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.596 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.596 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.596 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616985+0000", 2024-09-16T00:08:15.596 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4099999999999999e-07 2024-09-16T00:08:15.596 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.596 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.596 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.596 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617055+0000", 2024-09-16T00:08:15.596 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9923000000000001e-05 2024-09-16T00:08:15.596 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.596 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.596 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.596 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617063+0000", 2024-09-16T00:08:15.596 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:15.597 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.597 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.597 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617070+0000", 2024-09-16T00:08:15.597 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9090000000000003e-06 2024-09-16T00:08:15.597 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.597 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.597 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617102+0000", 2024-09-16T00:08:15.597 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2620999999999998e-05 2024-09-16T00:08:15.597 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.597 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.598 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617114+0000", 2024-09-16T00:08:15.598 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1228e-05 2024-09-16T00:08:15.598 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.598 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.598 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.598 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619690+0000", 2024-09-16T00:08:15.598 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025763890000000001 2024-09-16T00:08:15.598 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.598 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.598 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.598 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619712+0000", 2024-09-16T00:08:15.598 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2171000000000002e-05 2024-09-16T00:08:15.598 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.598 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.599 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.599 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:15.599 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:15.599 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.911016349999997, 2024-09-16T00:08:15.599 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027355320000000002, 2024-09-16T00:08:15.599 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.599 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.599 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.599 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:15.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:15.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615815+0000", 2024-09-16T00:08:15.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1769999999999999e-06 2024-09-16T00:08:15.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.601 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.601 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615825+0000", 2024-09-16T00:08:15.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9000000000000001e-06 2024-09-16T00:08:15.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.601 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.601 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615840+0000", 2024-09-16T00:08:15.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4854e-05 2024-09-16T00:08:15.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615848+0000", 2024-09-16T00:08:15.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0940000000000008e-06 2024-09-16T00:08:15.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615926+0000", 2024-09-16T00:08:15.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8176999999999994e-05 2024-09-16T00:08:15.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.603 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.603 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615945+0000", 2024-09-16T00:08:15.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9663000000000001e-05 2024-09-16T00:08:15.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.603 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.603 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618509+0000", 2024-09-16T00:08:15.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002563802 2024-09-16T00:08:15.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.604 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.604 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618549+0000", 2024-09-16T00:08:15.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9864999999999999e-05 2024-09-16T00:08:15.604 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.604 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.604 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.605 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:15.605 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:15.605 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.910927209999997, 2024-09-16T00:08:15.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061593860000000002, 2024-09-16T00:08:15.605 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.605 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.605 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.605 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:15.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:15.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615907+0000", 2024-09-16T00:08:15.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9389999999999999e-06 2024-09-16T00:08:15.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.607 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.607 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615920+0000", 2024-09-16T00:08:15.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2989999999999999e-05 2024-09-16T00:08:15.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.608 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615931+0000", 2024-09-16T00:08:15.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.164e-05 2024-09-16T00:08:15.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.608 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615939+0000", 2024-09-16T00:08:15.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8029999999999997e-06 2024-09-16T00:08:15.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616310+0000", 2024-09-16T00:08:15.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00037111400000000003 2024-09-16T00:08:15.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616406+0000", 2024-09-16T00:08:15.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5372999999999996e-05 2024-09-16T00:08:15.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:15.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616514+0000", 2024-09-16T00:08:15.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108564 2024-09-16T00:08:15.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:15.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618158+0000", 2024-09-16T00:08:15.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016442449999999999 2024-09-16T00:08:15.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621972+0000", 2024-09-16T00:08:15.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038132299999999999 2024-09-16T00:08:15.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622018+0000", 2024-09-16T00:08:15.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6239000000000002e-05 2024-09-16T00:08:15.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622062+0000", 2024-09-16T00:08:15.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4249000000000003e-05 2024-09-16T00:08:15.611 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.611 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.611 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.612 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:15.612 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:15.612 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.910873742, 2024-09-16T00:08:15.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006258914, 2024-09-16T00:08:15.612 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.612 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.612 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.612 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.612 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.613 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:15.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.613 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.613 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:15.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615957+0000", 2024-09-16T00:08:15.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1399999999999997e-07 2024-09-16T00:08:15.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615960+0000", 2024-09-16T00:08:15.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0860000000000002e-06 2024-09-16T00:08:15.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.615 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615967+0000", 2024-09-16T00:08:15.615 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9059999999999996e-06 2024-09-16T00:08:15.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.615 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.615 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615972+0000", 2024-09-16T00:08:15.615 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5679999999999999e-06 2024-09-16T00:08:15.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.615 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.615 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616717+0000", 2024-09-16T00:08:15.615 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074438 2024-09-16T00:08:15.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616781+0000", 2024-09-16T00:08:15.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4808999999999996e-05 2024-09-16T00:08:15.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:15.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616885+0000", 2024-09-16T00:08:15.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010404900000000001 2024-09-16T00:08:15.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:15.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:15.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012773870000000001 2024-09-16T00:08:15.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622116+0000", 2024-09-16T00:08:15.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039533470000000003 2024-09-16T00:08:15.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.618 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622173+0000", 2024-09-16T00:08:15.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6347000000000001e-05 2024-09-16T00:08:15.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.618 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.618 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622215+0000", 2024-09-16T00:08:15.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2521000000000002e-05 2024-09-16T00:08:15.618 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.618 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.618 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.619 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:15.619 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:15.619 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.910850005, 2024-09-16T00:08:15.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041725529999999999, 2024-09-16T00:08:15.619 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.619 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.619 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.619 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.619 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.619 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:15.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:15.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:15.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8799999999999998e-07 2024-09-16T00:08:15.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615990+0000", 2024-09-16T00:08:15.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2150000000000007e-06 2024-09-16T00:08:15.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:08:15.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7159999999999997e-06 2024-09-16T00:08:15.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.622 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.622 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616001+0000", 2024-09-16T00:08:15.622 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9339999999999997e-06 2024-09-16T00:08:15.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.622 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.622 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616293+0000", 2024-09-16T00:08:15.622 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029210899999999998 2024-09-16T00:08:15.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.622 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616401+0000", 2024-09-16T00:08:15.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108087 2024-09-16T00:08:15.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:15.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616561+0000", 2024-09-16T00:08:15.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015940600000000001 2024-09-16T00:08:15.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:15.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618146+0000", 2024-09-16T00:08:15.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015857040000000001 2024-09-16T00:08:15.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.624 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620091+0000", 2024-09-16T00:08:15.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001944837 2024-09-16T00:08:15.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.624 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620126+0000", 2024-09-16T00:08:15.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4242999999999997e-05 2024-09-16T00:08:15.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.625 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620152+0000", 2024-09-16T00:08:15.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6914e-05 2024-09-16T00:08:15.625 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.625 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.625 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.625 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:15.625 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:15.625 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.910811780000003, 2024-09-16T00:08:15.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00630628, 2024-09-16T00:08:15.626 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.626 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.626 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.626 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.626 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:15.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.626 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:15.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.627 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:08:15.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5999999999999999e-07 2024-09-16T00:08:15.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.627 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616022+0000", 2024-09-16T00:08:15.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0149999999999999e-06 2024-09-16T00:08:15.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616031+0000", 2024-09-16T00:08:15.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9770000000000006e-06 2024-09-16T00:08:15.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616036+0000", 2024-09-16T00:08:15.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2639999999999999e-06 2024-09-16T00:08:15.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617057+0000", 2024-09-16T00:08:15.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010212999999999999 2024-09-16T00:08:15.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.629 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617146+0000", 2024-09-16T00:08:15.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9234000000000007e-05 2024-09-16T00:08:15.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.629 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:15.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617232+0000", 2024-09-16T00:08:15.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5218999999999998e-05 2024-09-16T00:08:15.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:15.630 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619107+0000", 2024-09-16T00:08:15.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001875016 2024-09-16T00:08:15.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.630 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622259+0000", 2024-09-16T00:08:15.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031527930000000001 2024-09-16T00:08:15.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.631 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.631 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622290+0000", 2024-09-16T00:08:15.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1022999999999998e-05 2024-09-16T00:08:15.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.631 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.631 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622324+0000", 2024-09-16T00:08:15.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3979000000000001e-05 2024-09-16T00:08:15.631 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.631 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.632 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.632 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:15.632 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:15.632 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.910786477999999, 2024-09-16T00:08:15.632 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042412739999999997, 2024-09-16T00:08:15.632 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.632 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.632 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:15.633 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:15.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:15.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.634 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616044+0000", 2024-09-16T00:08:15.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:15.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.634 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616052+0000", 2024-09-16T00:08:15.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.439e-06 2024-09-16T00:08:15.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616059+0000", 2024-09-16T00:08:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1389999999999998e-06 2024-09-16T00:08:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616065+0000", 2024-09-16T00:08:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:08:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616823+0000", 2024-09-16T00:08:15.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075781199999999996 2024-09-16T00:08:15.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616918+0000", 2024-09-16T00:08:15.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5174999999999996e-05 2024-09-16T00:08:15.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:15.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617019+0000", 2024-09-16T00:08:15.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101161 2024-09-16T00:08:15.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:15.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618503+0000", 2024-09-16T00:08:15.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014836750000000001 2024-09-16T00:08:15.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:15.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620201+0000", 2024-09-16T00:08:15.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00169805 2024-09-16T00:08:15.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:15.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620255+0000", 2024-09-16T00:08:15.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4499000000000001e-05 2024-09-16T00:08:15.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620285+0000", 2024-09-16T00:08:15.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9349e-05 2024-09-16T00:08:15.638 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.638 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.638 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.639 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:15.639 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:15.639 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.910757486999998, 2024-09-16T00:08:15.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063682799999999996, 2024-09-16T00:08:15.639 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:15.639 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:15.639 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:15.639 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:15.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:15.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:15.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:15.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:15.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:15.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:15.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:15.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9900000000000001e-07 2024-09-16T00:08:15.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:15.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616079+0000", 2024-09-16T00:08:15.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9009999999999999e-06 2024-09-16T00:08:15.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:15.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616086+0000", 2024-09-16T00:08:15.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.776e-06 2024-09-16T00:08:15.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:15.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616091+0000", 2024-09-16T00:08:15.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5600000000000001e-06 2024-09-16T00:08:15.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:15.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618339+0000", 2024-09-16T00:08:15.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022476179999999998 2024-09-16T00:08:15.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:15.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:15.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9025999999999997e-05 2024-09-16T00:08:15.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:15.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:15.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:15.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622441+0000", 2024-09-16T00:08:15.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040029999999999996 2024-09-16T00:08:15.642 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.643 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.643 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.643 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:15.643 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:15.643 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:15.644 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:08:15.645 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:15.643+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:15.745 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:15.744+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:15.749 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:15.749 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:15.749 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:15.750 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:15.759 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:08:15.846 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:15.845+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:15.870 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:15.870 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:15.871 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:15.871 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:15.871 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:15.880 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:08:15.953 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:08:15.966 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:15.965+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:16.033 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:16.033 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:16.033 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.415534123, 2024-09-16T00:08:16.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037514929999999998, 2024-09-16T00:08:16.035 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.035 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.035 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:16.035 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.036 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.036 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.036 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615364+0000", 2024-09-16T00:08:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8430000000000003e-06 2024-09-16T00:08:16.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.039 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.039 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.039 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615371+0000", 2024-09-16T00:08:16.039 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.148e-06 2024-09-16T00:08:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615380+0000", 2024-09-16T00:08:16.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5889999999999992e-06 2024-09-16T00:08:16.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.041 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.041 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.041 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615389+0000", 2024-09-16T00:08:16.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1620000000000008e-06 2024-09-16T00:08:16.042 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.042 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.042 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615525+0000", 2024-09-16T00:08:16.042 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000136083 2024-09-16T00:08:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615662+0000", 2024-09-16T00:08:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013696100000000001 2024-09-16T00:08:16.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.044 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:16.044 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615799+0000", 2024-09-16T00:08:16.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013781399999999999 2024-09-16T00:08:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002363502 2024-09-16T00:08:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619018+0000", 2024-09-16T00:08:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085522199999999999 2024-09-16T00:08:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619068+0000", 2024-09-16T00:08:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9545000000000001e-05 2024-09-16T00:08:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619111+0000", 2024-09-16T00:08:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3624000000000001e-05 2024-09-16T00:08:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.049 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.049 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.049 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.415482052000002, 2024-09-16T00:08:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043656110000000001, 2024-09-16T00:08:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2199999999999999e-07 2024-09-16T00:08:16.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615416+0000", 2024-09-16T00:08:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5590000000000001e-06 2024-09-16T00:08:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615423+0000", 2024-09-16T00:08:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6250000000000001e-06 2024-09-16T00:08:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615426+0000", 2024-09-16T00:08:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2540000000000001e-06 2024-09-16T00:08:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616075+0000", 2024-09-16T00:08:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064968800000000004 2024-09-16T00:08:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616142+0000", 2024-09-16T00:08:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6203000000000003e-05 2024-09-16T00:08:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616247+0000", 2024-09-16T00:08:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010572499999999999 2024-09-16T00:08:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618913+0000", 2024-09-16T00:08:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026654339999999999 2024-09-16T00:08:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619712+0000", 2024-09-16T00:08:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079885199999999996 2024-09-16T00:08:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619744+0000", 2024-09-16T00:08:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2790000000000003e-05 2024-09-16T00:08:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619778+0000", 2024-09-16T00:08:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3059000000000003e-05 2024-09-16T00:08:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.415445972999997, 2024-09-16T00:08:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087673219999999993, 2024-09-16T00:08:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:08:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5392000000000001e-05 2024-09-16T00:08:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615470+0000", 2024-09-16T00:08:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1999999999999999e-06 2024-09-16T00:08:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615474+0000", 2024-09-16T00:08:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7359999999999999e-06 2024-09-16T00:08:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617614+0000", 2024-09-16T00:08:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021398189999999998 2024-09-16T00:08:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617724+0000", 2024-09-16T00:08:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110808 2024-09-16T00:08:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617856+0000", 2024-09-16T00:08:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131676 2024-09-16T00:08:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029629280000000001 2024-09-16T00:08:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624158+0000", 2024-09-16T00:08:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033386549999999998 2024-09-16T00:08:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624189+0000", 2024-09-16T00:08:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1508999999999999e-05 2024-09-16T00:08:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624215+0000", 2024-09-16T00:08:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6228999999999999e-05 2024-09-16T00:08:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.415411958, 2024-09-16T00:08:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053285279999999999, 2024-09-16T00:08:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5600000000000002e-07 2024-09-16T00:08:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615484+0000", 2024-09-16T00:08:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.688e-06 2024-09-16T00:08:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615488+0000", 2024-09-16T00:08:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.658e-06 2024-09-16T00:08:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615490+0000", 2024-09-16T00:08:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.728e-06 2024-09-16T00:08:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616441+0000", 2024-09-16T00:08:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00095065500000000001 2024-09-16T00:08:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616539+0000", 2024-09-16T00:08:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8335e-05 2024-09-16T00:08:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616631+0000", 2024-09-16T00:08:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1260000000000004e-05 2024-09-16T00:08:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619837+0000", 2024-09-16T00:08:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003206899 2024-09-16T00:08:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620725+0000", 2024-09-16T00:08:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088713599999999996 2024-09-16T00:08:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620772+0000", 2024-09-16T00:08:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7639999999999998e-05 2024-09-16T00:08:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620811+0000", 2024-09-16T00:08:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8272999999999998e-05 2024-09-16T00:08:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.414996870000003, 2024-09-16T00:08:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029886169999999998, 2024-09-16T00:08:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615898+0000", 2024-09-16T00:08:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.237e-06 2024-09-16T00:08:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615904+0000", 2024-09-16T00:08:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615915+0000", 2024-09-16T00:08:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1003e-05 2024-09-16T00:08:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615922+0000", 2024-09-16T00:08:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9959999999999996e-06 2024-09-16T00:08:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615988+0000", 2024-09-16T00:08:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5806e-05 2024-09-16T00:08:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616009+0000", 2024-09-16T00:08:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0729000000000001e-05 2024-09-16T00:08:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028462130000000002 2024-09-16T00:08:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618886+0000", 2024-09-16T00:08:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.082e-05 2024-09-16T00:08:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.413385398000003, 2024-09-16T00:08:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032610299999999998, 2024-09-16T00:08:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617556+0000", 2024-09-16T00:08:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7431e-05 2024-09-16T00:08:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617847+0000", 2024-09-16T00:08:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029133400000000003 2024-09-16T00:08:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617858+0000", 2024-09-16T00:08:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0222e-05 2024-09-16T00:08:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617864+0000", 2024-09-16T00:08:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7619999999999998e-06 2024-09-16T00:08:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617909+0000", 2024-09-16T00:08:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4747999999999999e-05 2024-09-16T00:08:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617927+0000", 2024-09-16T00:08:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7785999999999999e-05 2024-09-16T00:08:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620734+0000", 2024-09-16T00:08:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002807305 2024-09-16T00:08:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620770+0000", 2024-09-16T00:08:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5441999999999997e-05 2024-09-16T00:08:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.412964713999997, 2024-09-16T00:08:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028641809999999999, 2024-09-16T00:08:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617930+0000", 2024-09-16T00:08:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0200000000000002e-07 2024-09-16T00:08:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617934+0000", 2024-09-16T00:08:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617943+0000", 2024-09-16T00:08:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5820000000000002e-06 2024-09-16T00:08:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617949+0000", 2024-09-16T00:08:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5069999999999997e-06 2024-09-16T00:08:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618094+0000", 2024-09-16T00:08:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014515599999999999 2024-09-16T00:08:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618111+0000", 2024-09-16T00:08:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6705999999999999e-05 2024-09-16T00:08:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:08:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026603070000000002 2024-09-16T00:08:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620793+0000", 2024-09-16T00:08:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2214000000000001e-05 2024-09-16T00:08:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.412933924999997, 2024-09-16T00:08:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028624549999999999, 2024-09-16T00:08:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0570000000000001e-06 2024-09-16T00:08:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617970+0000", 2024-09-16T00:08:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.669e-06 2024-09-16T00:08:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617976+0000", 2024-09-16T00:08:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1369999999999999e-06 2024-09-16T00:08:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618268+0000", 2024-09-16T00:08:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029164600000000001 2024-09-16T00:08:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618284+0000", 2024-09-16T00:08:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5824e-05 2024-09-16T00:08:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025127890000000001 2024-09-16T00:08:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620822+0000", 2024-09-16T00:08:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6009000000000001e-05 2024-09-16T00:08:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-16T00:08:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.412906344, 2024-09-16T00:08:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028650350000000002, 2024-09-16T00:08:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7599999999999998e-07 2024-09-16T00:08:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617991+0000", 2024-09-16T00:08:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1860000000000001e-06 2024-09-16T00:08:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617999+0000", 2024-09-16T00:08:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3250000000000002e-06 2024-09-16T00:08:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618004+0000", 2024-09-16T00:08:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7300000000000002e-06 2024-09-16T00:08:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618422+0000", 2024-09-16T00:08:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00041763199999999998 2024-09-16T00:08:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6112000000000001e-05 2024-09-16T00:08:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620824+0000", 2024-09-16T00:08:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002385954 2024-09-16T00:08:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620853+0000", 2024-09-16T00:08:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8719999999999999e-05 2024-09-16T00:08:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-16T00:08:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.412880522000002, 2024-09-16T00:08:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032703020000000001, 2024-09-16T00:08:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618014+0000", 2024-09-16T00:08:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7799999999999997e-07 2024-09-16T00:08:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618017+0000", 2024-09-16T00:08:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4309999999999998e-06 2024-09-16T00:08:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618023+0000", 2024-09-16T00:08:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8490000000000002e-06 2024-09-16T00:08:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618028+0000", 2024-09-16T00:08:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3489999999999999e-06 2024-09-16T00:08:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:08:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055724099999999999 2024-09-16T00:08:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618603+0000", 2024-09-16T00:08:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7486000000000002e-05 2024-09-16T00:08:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621235+0000", 2024-09-16T00:08:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002632087 2024-09-16T00:08:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621284+0000", 2024-09-16T00:08:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8581000000000001e-05 2024-09-16T00:08:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:14 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.411943542000003, 2024-09-16T00:08:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016807599999999999, 2024-09-16T00:08:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618951+0000", 2024-09-16T00:08:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618954+0000", 2024-09-16T00:08:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.145e-06 2024-09-16T00:08:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618963+0000", 2024-09-16T00:08:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5720000000000003e-06 2024-09-16T00:08:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618969+0000", 2024-09-16T00:08:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3319999999999997e-06 2024-09-16T00:08:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619008+0000", 2024-09-16T00:08:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9096e-05 2024-09-16T00:08:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619013+0000", 2024-09-16T00:08:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4710000000000001e-06 2024-09-16T00:08:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619118+0000", 2024-09-16T00:08:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000105809 2024-09-16T00:08:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-16T00:08:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.411905435000001, 2024-09-16T00:08:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002325307, 2024-09-16T00:08:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618993+0000", 2024-09-16T00:08:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.687e-06 2024-09-16T00:08:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619000+0000", 2024-09-16T00:08:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3590000000000002e-06 2024-09-16T00:08:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619005+0000", 2024-09-16T00:08:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.95e-06 2024-09-16T00:08:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619039+0000", 2024-09-16T00:08:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3942999999999997e-05 2024-09-16T00:08:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619055+0000", 2024-09-16T00:08:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6107e-05 2024-09-16T00:08:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621287+0000", 2024-09-16T00:08:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022318070000000001 2024-09-16T00:08:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7129999999999999e-05 2024-09-16T00:08:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-16T00:08:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.411867438999998, 2024-09-16T00:08:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002314328, 2024-09-16T00:08:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6399999999999998e-07 2024-09-16T00:08:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619030+0000", 2024-09-16T00:08:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.219e-06 2024-09-16T00:08:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619038+0000", 2024-09-16T00:08:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9899999999999997e-06 2024-09-16T00:08:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619043+0000", 2024-09-16T00:08:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5580000000000002e-06 2024-09-16T00:08:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619220+0000", 2024-09-16T00:08:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000176937 2024-09-16T00:08:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619237+0000", 2024-09-16T00:08:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7691999999999999e-05 2024-09-16T00:08:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621315+0000", 2024-09-16T00:08:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020778900000000002 2024-09-16T00:08:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621341+0000", 2024-09-16T00:08:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5678e-05 2024-09-16T00:08:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:15 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.411252470000001, 2024-09-16T00:08:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143416, 2024-09-16T00:08:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619642+0000", 2024-09-16T00:08:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5199999999999995e-07 2024-09-16T00:08:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619645+0000", 2024-09-16T00:08:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.134e-06 2024-09-16T00:08:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619655+0000", 2024-09-16T00:08:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3330000000000003e-06 2024-09-16T00:08:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619662+0000", 2024-09-16T00:08:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0500000000000003e-06 2024-09-16T00:08:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619704+0000", 2024-09-16T00:08:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1766e-05 2024-09-16T00:08:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619708+0000", 2024-09-16T00:08:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3440000000000003e-06 2024-09-16T00:08:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619785+0000", 2024-09-16T00:08:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6936999999999999e-05 2024-09-16T00:08:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:31 2.d e22/19 ondisk, result = 0)", 2024-09-16T00:08:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.411153661, 2024-09-16T00:08:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101826, 2024-09-16T00:08:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619741+0000", 2024-09-16T00:08:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3099999999999997e-07 2024-09-16T00:08:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:08:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1719999999999998e-06 2024-09-16T00:08:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619771+0000", 2024-09-16T00:08:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4754000000000001e-05 2024-09-16T00:08:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619790+0000", 2024-09-16T00:08:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9516000000000001e-05 2024-09-16T00:08:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619830+0000", 2024-09-16T00:08:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9848000000000002e-05 2024-09-16T00:08:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619834+0000", 2024-09-16T00:08:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1150000000000004e-06 2024-09-16T00:08:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619842+0000", 2024-09-16T00:08:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.79e-06 2024-09-16T00:08:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:30 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.406802673999998, 2024-09-16T00:08:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013076400000000001, 2024-09-16T00:08:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624092+0000", 2024-09-16T00:08:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5799999999999999e-07 2024-09-16T00:08:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624096+0000", 2024-09-16T00:08:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624106+0000", 2024-09-16T00:08:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7629999999999995e-06 2024-09-16T00:08:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624112+0000", 2024-09-16T00:08:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.626e-06 2024-09-16T00:08:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624149+0000", 2024-09-16T00:08:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7094000000000001e-05 2024-09-16T00:08:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624154+0000", 2024-09-16T00:08:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7359999999999998e-06 2024-09-16T00:08:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624222+0000", 2024-09-16T00:08:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7680000000000003e-05 2024-09-16T00:08:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:39 2.6 2:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.390616115999997, 2024-09-16T00:08:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050072119999999996, 2024-09-16T00:08:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640280+0000", 2024-09-16T00:08:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5380000000000001e-06 2024-09-16T00:08:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640296+0000", 2024-09-16T00:08:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5950000000000001e-05 2024-09-16T00:08:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640327+0000", 2024-09-16T00:08:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0309999999999999e-05 2024-09-16T00:08:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640340+0000", 2024-09-16T00:08:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3773e-05 2024-09-16T00:08:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640422+0000", 2024-09-16T00:08:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1122e-05 2024-09-16T00:08:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640572+0000", 2024-09-16T00:08:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015044099999999999 2024-09-16T00:08:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:08:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640691+0000", 2024-09-16T00:08:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000119393 2024-09-16T00:08:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645006+0000", 2024-09-16T00:08:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043143779999999998 2024-09-16T00:08:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645169+0000", 2024-09-16T00:08:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016323299999999999 2024-09-16T00:08:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645227+0000", 2024-09-16T00:08:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8e-05 2024-09-16T00:08:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645285+0000", 2024-09-16T00:08:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8074e-05 2024-09-16T00:08:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:39 2.6 e22/19 ondisk, result = 0)", 2024-09-16T00:08:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.386044005000002, 2024-09-16T00:08:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016541200000000001, 2024-09-16T00:08:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644852+0000", 2024-09-16T00:08:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1670000000000002e-06 2024-09-16T00:08:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644865+0000", 2024-09-16T00:08:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2826000000000001e-05 2024-09-16T00:08:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644894+0000", 2024-09-16T00:08:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9264999999999998e-05 2024-09-16T00:08:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644911+0000", 2024-09-16T00:08:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6827999999999999e-05 2024-09-16T00:08:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644989+0000", 2024-09-16T00:08:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7849999999999995e-05 2024-09-16T00:08:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644999+0000", 2024-09-16T00:08:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7289999999999995e-06 2024-09-16T00:08:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645015+0000", 2024-09-16T00:08:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6747000000000001e-05 2024-09-16T00:08:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:16.133 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:16.134 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:08:16.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.065+0000 7fbccc5c4640 1 Processor -- start 2024-09-16T00:08:16.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.065+0000 7fbccc5c4640 1 -- start start 2024-09-16T00:08:16.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.066+0000 7fbccc5c4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbcc40d0490 0x7fbcc40b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.066+0000 7fbccc5c4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbcc40d0870 0x7fbcc40b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.066+0000 7fbccc5c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbcc40b3430 0x7fbcc40b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.066+0000 7fbccc5c4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbcc415b940 con 0x7fbcc40d0870 2024-09-16T00:08:16.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.066+0000 7fbccc5c4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fbcc40b3bb0 con 0x7fbcc40d0490 2024-09-16T00:08:16.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.066+0000 7fbccc5c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc40b3180 con 0x7fbcc40b3430 2024-09-16T00:08:16.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.067+0000 7fbcc9b5f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbcc40d0870 0x7fbcc40b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.067+0000 7fbcc9b5f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbcc40d0870 0x7fbcc40b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35926/0 (socket says 172.21.15.139:35926) 2024-09-16T00:08:16.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.067+0000 7fbcc9b5f640 1 -- 172.21.15.139:0/542172319 learned_addr learned my addr 172.21.15.139:0/542172319 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:16.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.067+0000 7fbcca360640 1 --2- 172.21.15.139:0/542172319 >> v2:172.21.15.164:3300/0 conn(0x7fbcc40d0490 0x7fbcc40b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.067+0000 7fbccab61640 1 --2- 172.21.15.139:0/542172319 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbcc40b3430 0x7fbcc40b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.067+0000 7fbcc9b5f640 1 -- 172.21.15.139:0/542172319 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbcc40b3430 msgr2=0x7fbcc40b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:16.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.067+0000 7fbcc9b5f640 1 --2- 172.21.15.139:0/542172319 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbcc40b3430 0x7fbcc40b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.067+0000 7fbcc9b5f640 1 -- 172.21.15.139:0/542172319 >> v2:172.21.15.164:3300/0 conn(0x7fbcc40d0490 msgr2=0x7fbcc40b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:16.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.067+0000 7fbcc9b5f640 1 --2- 172.21.15.139:0/542172319 >> v2:172.21.15.164:3300/0 conn(0x7fbcc40d0490 0x7fbcc40b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.067+0000 7fbcc9b5f640 1 -- 172.21.15.139:0/542172319 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc415adc0 con 0x7fbcc40d0870 2024-09-16T00:08:16.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.068+0000 7fbcc9b5f640 1 --2- 172.21.15.139:0/542172319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc40d0870 0x7fbcc40b4040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40cddf0 tx=0x7fbcb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c3d202e7140303 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:16.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.068+0000 7fbcc935e640 1 -- 172.21.15.139:0/542172319 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4037070 con 0x7fbcc40d0870 2024-09-16T00:08:16.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.068+0000 7fbcc935e640 1 -- 172.21.15.139:0/542172319 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb4032440 con 0x7fbcc40d0870 2024-09-16T00:08:16.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.068+0000 7fbcc935e640 1 -- 172.21.15.139:0/542172319 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4032810 con 0x7fbcc40d0870 2024-09-16T00:08:16.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.069+0000 7fbccc5c4640 1 -- 172.21.15.139:0/542172319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc40d0870 msgr2=0x7fbcc40b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.069+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/542172319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc40d0870 0x7fbcc40b4040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40cddf0 tx=0x7fbcb402a270 comp rx=0 tx=0).stop 2024-09-16T00:08:16.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.069+0000 7fbccc5c4640 1 -- 172.21.15.139:0/542172319 shutdown_connections 2024-09-16T00:08:16.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.069+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/542172319 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbcc40b3430 0x7fbcc40b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.069+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/542172319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc40d0870 0x7fbcc40b4040 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.069+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/542172319 >> v2:172.21.15.164:3300/0 conn(0x7fbcc40d0490 0x7fbcc40b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.069+0000 7fbccc5c4640 1 -- 172.21.15.139:0/542172319 >> 172.21.15.139:0/542172319 conn(0x7fbcc406c520 msgr2=0x7fbcc406c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:16.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.069+0000 7fbccc5c4640 1 -- 172.21.15.139:0/542172319 shutdown_connections 2024-09-16T00:08:16.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.069+0000 7fbccc5c4640 1 -- 172.21.15.139:0/542172319 wait complete. 2024-09-16T00:08:16.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.070+0000 7fbccc5c4640 1 Processor -- start 2024-09-16T00:08:16.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.070+0000 7fbccc5c4640 1 -- start start 2024-09-16T00:08:16.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbccc5c4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcc40b3430 0x7fbcc4149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbccc5c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbcc40d0490 0x7fbcc41f0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbccc5c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc40d0870 0x7fbcc41484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbccc5c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc415b660 con 0x7fbcc40d0870 2024-09-16T00:08:16.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbccc5c4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc415b380 con 0x7fbcc40b3430 2024-09-16T00:08:16.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbccc5c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc415a980 con 0x7fbcc40d0490 2024-09-16T00:08:16.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbccab61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc40d0870 0x7fbcc41484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbccab61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc40d0870 0x7fbcc41484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35932/0 (socket says 172.21.15.139:35932) 2024-09-16T00:08:16.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbccab61640 1 -- 172.21.15.139:0/535993467 learned_addr learned my addr 172.21.15.139:0/535993467 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:16.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbcc9b5f640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbcc40d0490 0x7fbcc41f0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbcca360640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcc40b3430 0x7fbcc4149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbccab61640 1 -- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbcc40d0490 msgr2=0x7fbcc41f0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbccab61640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbcc40d0490 0x7fbcc41f0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbccab61640 1 -- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcc40b3430 msgr2=0x7fbcc4149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbccab61640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcc40b3430 0x7fbcc4149930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbccab61640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc41f46a0 con 0x7fbcc40d0870 2024-09-16T00:08:16.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbccab61640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc40d0870 0x7fbcc41484d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00ef00 tx=0x7fbcbc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:16.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.071+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc018020 con 0x7fbcc40d0870 2024-09-16T00:08:16.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.072+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcbc014400 con 0x7fbcc40d0870 2024-09-16T00:08:16.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.072+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0147f0 con 0x7fbcc40d0870 2024-09-16T00:08:16.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.072+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc41f4920 con 0x7fbcc40d0870 2024-09-16T00:08:16.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.072+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc40c7f50 con 0x7fbcc40d0870 2024-09-16T00:08:16.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.073+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fbcc40c8190 con 0x7fbcc40d0870 2024-09-16T00:08:16.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.073+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbcbc005d10 con 0x7fbcc40d0870 2024-09-16T00:08:16.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.073+0000 7fbcbaffd640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbca806c2e0 0x7fbca806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.074+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5915+0+0 (secure 0 0 0) 0x7fbcbc08ba40 con 0x7fbcc40d0870 2024-09-16T00:08:16.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.074+0000 7fbcca360640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbca806c2e0 0x7fbca806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.074+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v34) ==== 54+0+16 (secure 0 0 0) 0x7fbcbc08be20 con 0x7fbcc40d0870 2024-09-16T00:08:16.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.074+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbcc414ac70 0x7fbcc4154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.074+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbcc41f4c00 con 0x7fbcc414ac70 2024-09-16T00:08:16.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.074+0000 7fbcca360640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbca806c2e0 0x7fbca806e7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009b40 tx=0x7fbcc0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:16.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.074+0000 7fbcc9b5f640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbcc414ac70 0x7fbcc4154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.074+0000 7fbcc9b5f640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbcc414ac70 0x7fbcc4154450 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:16.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.075+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbcc41f4c00 con 0x7fbcc414ac70 2024-09-16T00:08:16.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.089+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbcc4148f20 con 0x7fbcc414ac70 2024-09-16T00:08:16.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.090+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbcc4148f20 con 0x7fbcc414ac70 2024-09-16T00:08:16.145 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:16.146 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:16.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.091+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbcc41f13c0 0x7fbcc414fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.091+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fbcc40b2c40 con 0x7fbcc41f13c0 2024-09-16T00:08:16.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.091+0000 7fbcca360640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbcc41f13c0 0x7fbcc414fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.091+0000 7fbcca360640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbcc41f13c0 0x7fbcc414fc00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:16.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.092+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbcc40b2c40 con 0x7fbcc41f13c0 2024-09-16T00:08:16.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.106+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbcc414e990 con 0x7fbcc41f13c0 2024-09-16T00:08:16.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.108+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbcc414e990 con 0x7fbcc41f13c0 2024-09-16T00:08:16.147 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:16.147 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:16.133+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:16.148 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:16.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.108+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbcc4150650 0x7fbcc4062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.108+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fbcc4151ce0 con 0x7fbcc4150650 2024-09-16T00:08:16.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.108+0000 7fbcc9b5f640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbcc4150650 0x7fbcc4062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.108+0000 7fbcc9b5f640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbcc4150650 0x7fbcc4062630 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:16.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.109+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbcc4151ce0 con 0x7fbcc4150650 2024-09-16T00:08:16.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.122+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbcc4159a30 con 0x7fbcc4150650 2024-09-16T00:08:16.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.124+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbcc4159a30 con 0x7fbcc4150650 2024-09-16T00:08:16.149 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:16.150 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:16.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.124+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbcc4063080 0x7fbcc40ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.124+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fbcc40cc0d0 con 0x7fbcc4063080 2024-09-16T00:08:16.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.124+0000 7fbccab61640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbcc4063080 0x7fbcc40ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.124+0000 7fbccab61640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbcc4063080 0x7fbcc40ce0b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:16.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.125+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbcc40cc0d0 con 0x7fbcc4063080 2024-09-16T00:08:16.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.141+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbcc40cc0d0 con 0x7fbcc4063080 2024-09-16T00:08:16.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.143+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbcc40cc0d0 con 0x7fbcc4063080 2024-09-16T00:08:16.151 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:16.151 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:16.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.143+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbcc40ceb00 0x7fbcc402c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.143+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fbcc40cef40 con 0x7fbcc40ceb00 2024-09-16T00:08:16.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.143+0000 7fbcca360640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbcc40ceb00 0x7fbcc402c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.144+0000 7fbcca360640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbcc40ceb00 0x7fbcc402c640 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:16.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.145+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbcc40cef40 con 0x7fbcc40ceb00 2024-09-16T00:08:16.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.159+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbcc40cef40 con 0x7fbcc40ceb00 2024-09-16T00:08:16.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.161+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbcc40cef40 con 0x7fbcc40ceb00 2024-09-16T00:08:16.162 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:16.162 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:16.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.161+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbcc402d090 0x7fbcc402d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.161+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fbcc402da30 con 0x7fbcc402d090 2024-09-16T00:08:16.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.161+0000 7fbcc9b5f640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbcc402d090 0x7fbcc402d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.162+0000 7fbcc9b5f640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbcc402d090 0x7fbcc402d4f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:16.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.162+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbcc402da30 con 0x7fbcc402d090 2024-09-16T00:08:16.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.177+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbcc402da30 con 0x7fbcc402d090 2024-09-16T00:08:16.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.179+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbcc402da30 con 0x7fbcc402d090 2024-09-16T00:08:16.180 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:16.181 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:16.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.179+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbcc4207560 0x7fbcc4209950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.179+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fbcc420c8b0 con 0x7fbcc4207560 2024-09-16T00:08:16.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.180+0000 7fbccab61640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbcc4207560 0x7fbcc4209950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.180+0000 7fbccab61640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbcc4207560 0x7fbcc4209950 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:16.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.181+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbcc420c8b0 con 0x7fbcc4207560 2024-09-16T00:08:16.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.194+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbcc420c880 con 0x7fbcc4207560 2024-09-16T00:08:16.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.196+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbcc420c880 con 0x7fbcc4207560 2024-09-16T00:08:16.197 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:16.198 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:16.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.196+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbcc420bae0 0x7fbcc420bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.196+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fbcc420fc80 con 0x7fbcc420bae0 2024-09-16T00:08:16.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.197+0000 7fbcca360640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbcc420bae0 0x7fbcc420bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.197+0000 7fbcca360640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbcc420bae0 0x7fbcc420bf40 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:16.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.198+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbcc420fc80 con 0x7fbcc420bae0 2024-09-16T00:08:16.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.211+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fbcc4209e90 con 0x7fbcc420bae0 2024-09-16T00:08:16.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.213+0000 7fbcbaffd640 1 -- 172.21.15.139:0/535993467 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbcc4209e90 con 0x7fbcc420bae0 2024-09-16T00:08:16.215 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:16.216 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:16.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbcc414ac70 msgr2=0x7fbcc4154450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbcc414ac70 0x7fbcc4154450 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbcc41f13c0 msgr2=0x7fbcc414fc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbcc41f13c0 0x7fbcc414fc00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbcc4150650 msgr2=0x7fbcc4062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbcc4150650 0x7fbcc4062630 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbcc4063080 msgr2=0x7fbcc40ce0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbcc4063080 0x7fbcc40ce0b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccab61640 1 -- 172.21.15.139:0/535993467 reap_dead start 2024-09-16T00:08:16.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbcc40ceb00 msgr2=0x7fbcc402c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbcc40ceb00 0x7fbcc402c640 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccab61640 1 -- 172.21.15.139:0/535993467 reap_dead start 2024-09-16T00:08:16.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbcc402d090 msgr2=0x7fbcc402d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbcc402d090 0x7fbcc402d4f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbcc4207560 msgr2=0x7fbcc4209950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbcc4207560 0x7fbcc4209950 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbcc420bae0 msgr2=0x7fbcc420bf40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbcc420bae0 0x7fbcc420bf40 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbca806c2e0 msgr2=0x7fbca806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbca806c2e0 0x7fbca806e7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009b40 tx=0x7fbcc0009190 comp rx=0 tx=0).stop 2024-09-16T00:08:16.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccab61640 1 -- 172.21.15.139:0/535993467 reap_dead start 2024-09-16T00:08:16.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc40d0870 msgr2=0x7fbcc41484d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc40d0870 0x7fbcc41484d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc00ef00 tx=0x7fbcbc00b480 comp rx=0 tx=0).stop 2024-09-16T00:08:16.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 shutdown_connections 2024-09-16T00:08:16.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 --2- 172.21.15.139:0/535993467 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbcc40d0870 0x7fbcc41484d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 >> 172.21.15.139:0/535993467 conn(0x7fbcc406c520 msgr2=0x7fbcc414a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:16.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 shutdown_connections 2024-09-16T00:08:16.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.214+0000 7fbccc5c4640 1 -- 172.21.15.139:0/535993467 wait complete. 2024-09-16T00:08:16.237 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:16.234+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:16.264 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:16.264 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:16.273 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:08:16.366 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:16.365+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:16.426 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:16.426 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:16.436 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:08:16.441 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:16.439+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:16.540 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:16.539+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:16.606 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:16.607 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:16.616 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:08:16.663 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:16.662+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:16.740 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:16.739+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:16.768 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:16.768 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:16.777 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:08:16.794 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T00:08:16.841 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:16.840+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:16.891 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:16.891 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:16.901 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:08:16.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.931+0000 7f4c31ff1640 1 Processor -- start 2024-09-16T00:08:16.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.931+0000 7f4c31ff1640 1 -- start start 2024-09-16T00:08:16.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.931+0000 7f4c31ff1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4c2c151850 0x7f4c2c0c6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.931+0000 7f4c31ff1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c2c0c8d30 con 0x7f4c2c151c30 2024-09-16T00:08:16.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.931+0000 7f4c31ff1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c2c0c6a90 con 0x7f4c2c0c7390 2024-09-16T00:08:16.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.931+0000 7f4c31ff1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f4c2c0c88a0 con 0x7f4c2c151850 2024-09-16T00:08:16.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2affd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4c2c151c30 0x7f4c2c15ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56352/0 (socket says 172.21.15.139:56352) 2024-09-16T00:08:16.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2affd640 1 -- 172.21.15.139:0/1875241163 learned_addr learned my addr 172.21.15.139:0/1875241163 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:16.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2b7fe640 1 --2- 172.21.15.139:0/1875241163 >> v2:172.21.15.164:3300/0 conn(0x7f4c2c151850 0x7f4c2c0c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2a7fc640 1 -- 172.21.15.139:0/1875241163 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1606933204 0 0) 0x7f4c2c0c8d30 con 0x7f4c2c151c30 2024-09-16T00:08:16.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2a7fc640 1 -- 172.21.15.139:0/1875241163 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c14003520 con 0x7f4c2c151c30 2024-09-16T00:08:16.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2a7fc640 1 -- 172.21.15.139:0/1875241163 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2592953638 0 0) 0x7f4c14003520 con 0x7f4c2c151c30 2024-09-16T00:08:16.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2a7fc640 1 -- 172.21.15.139:0/1875241163 >> v2:172.21.15.164:3300/0 conn(0x7f4c2c151850 msgr2=0x7f4c2c0c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:16.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2a7fc640 1 --2- 172.21.15.139:0/1875241163 >> v2:172.21.15.164:3300/0 conn(0x7f4c2c151850 0x7f4c2c0c6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2a7fc640 1 -- 172.21.15.139:0/1875241163 >> v1:172.21.15.164:6789/0 conn(0x7f4c2c0c7390 legacy=0x7f4c2c0c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2a7fc640 1 -- 172.21.15.139:0/1875241163 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2c0c67b0 con 0x7f4c2c151c30 2024-09-16T00:08:16.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2a7fc640 1 -- 172.21.15.139:0/1875241163 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7f4c20001670 con 0x7f4c2c151c30 2024-09-16T00:08:16.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2a7fc640 1 -- 172.21.15.139:0/1875241163 >> v1:172.21.15.139:6789/0 conn(0x7f4c2c151c30 legacy=0x7f4c2c15ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2a7fc640 1 --2- 172.21.15.139:0/1875241163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c140039b0 0x7f4c14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2a7fc640 1 --2- 172.21.15.139:0/1875241163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c14004800 0x7f4c14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2a7fc640 1 --2- 172.21.15.139:0/1875241163 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c140051c0 0x7f4c1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2a7fc640 1 -- 172.21.15.139:0/1875241163 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c0c88a0 con 0x7f4c14004800 2024-09-16T00:08:16.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2a7fc640 1 -- 172.21.15.139:0/1875241163 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1400c070 con 0x7f4c140039b0 2024-09-16T00:08:16.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2a7fc640 1 -- 172.21.15.139:0/1875241163 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4c1400c270 con 0x7f4c140051c0 2024-09-16T00:08:16.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.932+0000 7f4c2affd640 1 --2- 172.21.15.139:0/1875241163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c14004800 0x7f4c14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.933+0000 7f4c2b7fe640 1 --2- 172.21.15.139:0/1875241163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c140039b0 0x7f4c14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.933+0000 7f4c2bfff640 1 --2- 172.21.15.139:0/1875241163 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c140051c0 0x7f4c1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.933+0000 7f4c2affd640 1 -- 172.21.15.139:0/1875241163 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c140051c0 msgr2=0x7f4c1400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:16.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.933+0000 7f4c2affd640 1 --2- 172.21.15.139:0/1875241163 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c140051c0 0x7f4c1400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.933+0000 7f4c2affd640 1 -- 172.21.15.139:0/1875241163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c140039b0 msgr2=0x7f4c14003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:16.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.933+0000 7f4c2affd640 1 --2- 172.21.15.139:0/1875241163 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c140039b0 0x7f4c14003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.933+0000 7f4c2bfff640 1 -- 172.21.15.139:0/1875241163 reap_dead start 2024-09-16T00:08:16.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.933+0000 7f4c2affd640 1 -- 172.21.15.139:0/1875241163 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c14003520 con 0x7f4c14004800 2024-09-16T00:08:16.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.934+0000 7f4c2affd640 1 --2- 172.21.15.139:0/1875241163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c14004800 0x7f4c14004c80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000c210 tx=0x7f4c14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2658dfe810451bb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:16.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.934+0000 7f4c2a7fc640 1 -- 172.21.15.139:0/1875241163 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20007d30 con 0x7f4c14004800 2024-09-16T00:08:16.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.934+0000 7f4c2a7fc640 1 -- 172.21.15.139:0/1875241163 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c200058d0 con 0x7f4c14004800 2024-09-16T00:08:16.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.934+0000 7f4c2a7fc640 1 -- 172.21.15.139:0/1875241163 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20005d20 con 0x7f4c14004800 2024-09-16T00:08:16.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.935+0000 7f4c31ff1640 1 -- 172.21.15.139:0/1875241163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c14004800 msgr2=0x7f4c14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.935+0000 7f4c31ff1640 1 --2- 172.21.15.139:0/1875241163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c14004800 0x7f4c14004c80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000c210 tx=0x7f4c14005830 comp rx=0 tx=0).stop 2024-09-16T00:08:16.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.935+0000 7f4c31ff1640 1 -- 172.21.15.139:0/1875241163 shutdown_connections 2024-09-16T00:08:16.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.935+0000 7f4c31ff1640 1 --2- 172.21.15.139:0/1875241163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c14004800 0x7f4c14004c80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.935+0000 7f4c31ff1640 1 -- 172.21.15.139:0/1875241163 >> 172.21.15.139:0/1875241163 conn(0x7f4c2c06cc30 msgr2=0x7f4c2c06d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:16.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.935+0000 7f4c31ff1640 1 -- 172.21.15.139:0/1875241163 shutdown_connections 2024-09-16T00:08:16.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.935+0000 7f4c31ff1640 1 -- 172.21.15.139:0/1875241163 wait complete. 2024-09-16T00:08:16.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.936+0000 7f4c31ff1640 1 Processor -- start 2024-09-16T00:08:16.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.936+0000 7f4c31ff1640 1 -- start start 2024-09-16T00:08:16.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c31ff1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c14004800 0x7f4c2c154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c31ff1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c2c06f940 0x7f4c2c155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c31ff1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c2c1566c0 0x7f4c2c156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c31ff1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c0c8530 con 0x7f4c2c1566c0 2024-09-16T00:08:16.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c31ff1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c0c8250 con 0x7f4c14004800 2024-09-16T00:08:16.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c31ff1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2c0c45a0 con 0x7f4c2c06f940 2024-09-16T00:08:16.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c2bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c2c1566c0 0x7f4c2c156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c2bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c2c1566c0 0x7f4c2c156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:35960/0 (socket says 172.21.15.139:35960) 2024-09-16T00:08:16.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c2bfff640 1 -- 172.21.15.139:0/4021735588 learned_addr learned my addr 172.21.15.139:0/4021735588 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:16.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c2b7fe640 1 --2- 172.21.15.139:0/4021735588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c14004800 0x7f4c2c154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c2affd640 1 --2- 172.21.15.139:0/4021735588 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c2c06f940 0x7f4c2c155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c2bfff640 1 -- 172.21.15.139:0/4021735588 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c2c06f940 msgr2=0x7f4c2c155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c2bfff640 1 --2- 172.21.15.139:0/4021735588 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c2c06f940 0x7f4c2c155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c2bfff640 1 -- 172.21.15.139:0/4021735588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c14004800 msgr2=0x7f4c2c154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:16.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c2bfff640 1 --2- 172.21.15.139:0/4021735588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c14004800 0x7f4c2c154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:16.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c2bfff640 1 -- 172.21.15.139:0/4021735588 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2c06edc0 con 0x7f4c2c1566c0 2024-09-16T00:08:16.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c2b7fe640 1 --2- 172.21.15.139:0/4021735588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c14004800 0x7f4c2c154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:08:16.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.937+0000 7f4c2affd640 1 --2- 172.21.15.139:0/4021735588 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c2c06f940 0x7f4c2c155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:08:16.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.938+0000 7f4c2bfff640 1 --2- 172.21.15.139:0/4021735588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c2c1566c0 0x7f4c2c156b40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4c24005280 tx=0x7f4c24005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:16.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.938+0000 7f4c13fff640 1 -- 172.21.15.139:0/4021735588 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24001d10 con 0x7f4c2c1566c0 2024-09-16T00:08:16.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.938+0000 7f4c13fff640 1 -- 172.21.15.139:0/4021735588 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c24001ed0 con 0x7f4c2c1566c0 2024-09-16T00:08:16.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.938+0000 7f4c31ff1640 1 -- 172.21.15.139:0/4021735588 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2c158230 con 0x7f4c2c1566c0 2024-09-16T00:08:16.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.938+0000 7f4c13fff640 1 -- 172.21.15.139:0/4021735588 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400c690 con 0x7f4c2c1566c0 2024-09-16T00:08:16.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.938+0000 7f4c31ff1640 1 -- 172.21.15.139:0/4021735588 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c2c1ec210 con 0x7f4c2c1566c0 2024-09-16T00:08:16.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.939+0000 7f4c31ff1640 1 -- 172.21.15.139:0/4021735588 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c2c0c8a50 con 0x7f4c2c1566c0 2024-09-16T00:08:16.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.940+0000 7f4c13fff640 1 -- 172.21.15.139:0/4021735588 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4c24021020 con 0x7f4c2c1566c0 2024-09-16T00:08:16.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.941+0000 7f4c13fff640 1 --2- 172.21.15.139:0/4021735588 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c1806c2b0 0x7f4c1806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:16.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.941+0000 7f4c13fff640 1 -- 172.21.15.139:0/4021735588 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5915+0+0 (secure 0 0 0) 0x7f4c2408f080 con 0x7f4c2c1566c0 2024-09-16T00:08:16.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.941+0000 7f4c2b7fe640 1 --2- 172.21.15.139:0/4021735588 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c1806c2b0 0x7f4c1806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:16.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.941+0000 7f4c2b7fe640 1 --2- 172.21.15.139:0/4021735588 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c1806c2b0 0x7f4c1806e770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c069440 tx=0x7f4c1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:16.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:16.945+0000 7f4c13fff640 1 -- 172.21.15.139:0/4021735588 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c2c0c8a50 con 0x7f4c2c1566c0 2024-09-16T00:08:16.950 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:16.942+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:17.015 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:17.015 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:17.025 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:08:17.066 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:17.065+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:17.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:17.132+0000 7f4c31ff1640 1 -- 172.21.15.139:0/4021735588 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f4c2c0c6100 con 0x7f4c2c1566c0 2024-09-16T00:08:17.145 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:17.144+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:17.145 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:17.145 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:17.155 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:08:17.246 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:17.245+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:17.300 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:17.300 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:17.309 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:08:17.347 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:17.346+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:17.448 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:17.447+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:17.453 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:17.453 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:17.462 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:08:17.549 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:17.548+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:17.607 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:17.607 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:17.616 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:08:17.657 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:17.656+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:17.750 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:17.749+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:17.760 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:17.760 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:17.769 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:08:17.784 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T00:08:17.784 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:08:17.784 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-16T00:08:17.785 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T00:08:17.785 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T00:08:17.785 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-16T00:08:17.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:17.789+0000 7f4c13fff640 1 -- 172.21.15.139:0/4021735588 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v35) ==== 87+0+0 (secure 0 0 0) 0x7f4c24002080 con 0x7f4c2c1566c0 2024-09-16T00:08:17.790 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is set 2024-09-16T00:08:17.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:17.792+0000 7f4c31ff1640 1 -- 172.21.15.139:0/4021735588 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c1806c2b0 msgr2=0x7f4c1806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:17.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:17.792+0000 7f4c31ff1640 1 --2- 172.21.15.139:0/4021735588 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c1806c2b0 0x7f4c1806e770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c069440 tx=0x7f4c1c008040 comp rx=0 tx=0).stop 2024-09-16T00:08:17.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:17.792+0000 7f4c31ff1640 1 -- 172.21.15.139:0/4021735588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c2c1566c0 msgr2=0x7f4c2c156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:17.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:17.792+0000 7f4c31ff1640 1 --2- 172.21.15.139:0/4021735588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c2c1566c0 0x7f4c2c156b40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4c24005280 tx=0x7f4c24005740 comp rx=0 tx=0).stop 2024-09-16T00:08:17.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:17.792+0000 7f4c31ff1640 1 -- 172.21.15.139:0/4021735588 shutdown_connections 2024-09-16T00:08:17.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:17.792+0000 7f4c31ff1640 1 --2- 172.21.15.139:0/4021735588 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c1806c2b0 0x7f4c1806e770 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:17.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:17.792+0000 7f4c31ff1640 1 --2- 172.21.15.139:0/4021735588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c2c1566c0 0x7f4c2c156b40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:17.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:17.792+0000 7f4c31ff1640 1 --2- 172.21.15.139:0/4021735588 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c2c06f940 0x7f4c2c155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:17.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:17.792+0000 7f4c31ff1640 1 --2- 172.21.15.139:0/4021735588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c14004800 0x7f4c2c154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:17.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:17.792+0000 7f4c31ff1640 1 -- 172.21.15.139:0/4021735588 >> 172.21.15.139:0/4021735588 conn(0x7f4c2c06cc30 msgr2=0x7f4c2c150340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:17.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:17.792+0000 7f4c31ff1640 1 -- 172.21.15.139:0/4021735588 shutdown_connections 2024-09-16T00:08:17.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:17.792+0000 7f4c31ff1640 1 -- 172.21.15.139:0/4021735588 wait complete. 2024-09-16T00:08:17.851 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:17.850+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:17.895 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:17.896 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T00:08:17.896 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:17.914 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:17.914 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:17.923 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:08:17.951 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:17.951+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:18.029 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:18.029 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:18.038 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:08:18.053 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:18.053+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:18.142 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:18.142 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:18.151 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:08:18.155 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:18.153+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:18.227 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:08:18.257 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:18.257 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:18.259 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:18.256+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:18.264 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:08:18.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.326+0000 7f5bdf74b640 1 Processor -- start 2024-09-16T00:08:18.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.326+0000 7f5bdf74b640 1 -- start start 2024-09-16T00:08:18.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.327+0000 7f5bdf74b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5bd8071ad0 0x7f5bd80cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:18.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.327+0000 7f5bdf74b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd815bf90 con 0x7f5bd80716f0 2024-09-16T00:08:18.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.327+0000 7f5bdf74b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd80c7890 con 0x7f5bd8158880 2024-09-16T00:08:18.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.327+0000 7f5bdf74b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5bd80c8130 con 0x7f5bd8071ad0 2024-09-16T00:08:18.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.327+0000 7f5bdcce6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5bd8071ad0 0x7f5bd80cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:18.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.327+0000 7f5bdd4e7640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5bd80716f0 0x7f5bd815c750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35234/0 (socket says 172.21.15.139:35234) 2024-09-16T00:08:18.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.327+0000 7f5bdd4e7640 1 -- 172.21.15.139:0/1354538054 learned_addr learned my addr 172.21.15.139:0/1354538054 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:18.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.327+0000 7f5bdcce6640 1 -- 172.21.15.139:0/1354538054 >> v1:172.21.15.164:6789/0 conn(0x7f5bd8158880 legacy=0x7f5bd8152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:18.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.327+0000 7f5bdcce6640 1 -- 172.21.15.139:0/1354538054 >> v1:172.21.15.139:6789/0 conn(0x7f5bd80716f0 legacy=0x7f5bd815c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:18.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.327+0000 7f5bcffff640 1 -- 172.21.15.139:0/1354538054 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1169600202 0 0) 0x7f5bd80c7890 con 0x7f5bd8158880 2024-09-16T00:08:18.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.327+0000 7f5bdcce6640 1 -- 172.21.15.139:0/1354538054 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd80c7b70 con 0x7f5bd8071ad0 2024-09-16T00:08:18.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.328+0000 7f5bdcce6640 1 --2- 172.21.15.139:0/1354538054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd8071ad0 0x7f5bd80cb450 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80097a0 tx=0x7f5bc802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1229db1cabdcdc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:18.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.328+0000 7f5bcffff640 1 -- 172.21.15.139:0/1354538054 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8037070 con 0x7f5bd8071ad0 2024-09-16T00:08:18.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.328+0000 7f5bcffff640 1 -- 172.21.15.139:0/1354538054 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc8032640 con 0x7f5bd8071ad0 2024-09-16T00:08:18.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.328+0000 7f5bcffff640 1 -- 172.21.15.139:0/1354538054 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8032a10 con 0x7f5bd8071ad0 2024-09-16T00:08:18.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.329+0000 7f5bdf74b640 1 -- 172.21.15.139:0/1354538054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd8071ad0 msgr2=0x7f5bd80cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:18.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.329+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/1354538054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd8071ad0 0x7f5bd80cb450 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80097a0 tx=0x7f5bc802a2c0 comp rx=0 tx=0).stop 2024-09-16T00:08:18.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.329+0000 7f5bdf74b640 1 -- 172.21.15.139:0/1354538054 shutdown_connections 2024-09-16T00:08:18.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.329+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/1354538054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd8071ad0 0x7f5bd80cb450 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:18.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.329+0000 7f5bdf74b640 1 -- 172.21.15.139:0/1354538054 >> 172.21.15.139:0/1354538054 conn(0x7f5bd806fb70 msgr2=0x7f5bd8073940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:18.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.329+0000 7f5bdf74b640 1 -- 172.21.15.139:0/1354538054 shutdown_connections 2024-09-16T00:08:18.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.329+0000 7f5bdf74b640 1 -- 172.21.15.139:0/1354538054 wait complete. 2024-09-16T00:08:18.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.330+0000 7f5bdf74b640 1 Processor -- start 2024-09-16T00:08:18.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.330+0000 7f5bdf74b640 1 -- start start 2024-09-16T00:08:18.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bdf74b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd80716f0 0x7f5bd80b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:18.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bdf74b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd8071ad0 0x7f5bd80ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:18.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bdf74b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5bd8158880 0x7f5bd81f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:18.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bdf74b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd80cb990 con 0x7f5bd8071ad0 2024-09-16T00:08:18.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bdf74b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd815c2c0 con 0x7f5bd80716f0 2024-09-16T00:08:18.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bdf74b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd80c7430 con 0x7f5bd8158880 2024-09-16T00:08:18.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bdcce6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd8071ad0 0x7f5bd80ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:18.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bdcce6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd8071ad0 0x7f5bd80ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50702/0 (socket says 172.21.15.139:50702) 2024-09-16T00:08:18.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bdcce6640 1 -- 172.21.15.139:0/2591782279 learned_addr learned my addr 172.21.15.139:0/2591782279 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:18.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bdd4e7640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd80716f0 0x7f5bd80b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:18.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bddce8640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5bd8158880 0x7f5bd81f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:18.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bdcce6640 1 -- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5bd8158880 msgr2=0x7f5bd81f3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:18.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bdcce6640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5bd8158880 0x7f5bd81f3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:18.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bdcce6640 1 -- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd80716f0 msgr2=0x7f5bd80b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:18.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bdcce6640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd80716f0 0x7f5bd80b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:18.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bdcce6640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd806e070 con 0x7f5bd8071ad0 2024-09-16T00:08:18.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.331+0000 7f5bdd4e7640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5bd80716f0 0x7f5bd80b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:18.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.332+0000 7f5bdcce6640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd8071ad0 0x7f5bd80ce320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8000c00 tx=0x7f5bc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:18.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.332+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8041070 con 0x7f5bd8071ad0 2024-09-16T00:08:18.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.332+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc803c930 con 0x7f5bd8071ad0 2024-09-16T00:08:18.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.332+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd806e310 con 0x7f5bd8071ad0 2024-09-16T00:08:18.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.332+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8037040 con 0x7f5bd8071ad0 2024-09-16T00:08:18.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.332+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd806e8d0 con 0x7f5bd8071ad0 2024-09-16T00:08:18.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.333+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f5bd80c7890 con 0x7f5bd8071ad0 2024-09-16T00:08:18.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.334+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5bc8044070 con 0x7f5bd8071ad0 2024-09-16T00:08:18.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.334+0000 7f5bcdffb640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5bbc06c2b0 0x7f5bbc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:18.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.334+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5915+0+0 (secure 0 0 0) 0x7f5bc80ac100 con 0x7f5bd8071ad0 2024-09-16T00:08:18.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.334+0000 7f5bdd4e7640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5bbc06c2b0 0x7f5bbc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:18.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.334+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v35) ==== 54+0+16 (secure 0 0 0) 0x7f5bd80c7890 con 0x7f5bd8071ad0 2024-09-16T00:08:18.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.335+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5bd81531f0 0x7f5bd80c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:18.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.335+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5bd80caf00 con 0x7f5bd81531f0 2024-09-16T00:08:18.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.335+0000 7f5bdd4e7640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5bbc06c2b0 0x7f5bbc06e770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000f60 tx=0x7f5bd4007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:18.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.335+0000 7f5bddce8640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5bd81531f0 0x7f5bd80c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:18.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.335+0000 7f5bddce8640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5bd81531f0 0x7f5bd80c9830 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:18.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.336+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd80caf00 con 0x7f5bd81531f0 2024-09-16T00:08:18.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.354+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5bd8151f90 con 0x7f5bd81531f0 2024-09-16T00:08:18.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.356+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5bd8151f90 con 0x7f5bd81531f0 2024-09-16T00:08:18.356 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:18.357 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:18.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.356+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5bd814f840 0x7f5bd806cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:18.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.356+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f5bd8151cb0 con 0x7f5bd814f840 2024-09-16T00:08:18.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.356+0000 7f5bdd4e7640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5bd814f840 0x7f5bd806cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:18.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.356+0000 7f5bdd4e7640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5bd814f840 0x7f5bd806cc30 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:18.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.357+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd8151cb0 con 0x7f5bd814f840 2024-09-16T00:08:18.360 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:18.358+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:18.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.376+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5bd806eb10 con 0x7f5bd814f840 2024-09-16T00:08:18.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.378+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5bd806eb10 con 0x7f5bd814f840 2024-09-16T00:08:18.379 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:18.380 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:18.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.378+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5bd814fc20 0x7f5bd806d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:18.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.378+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f5bd81545a0 con 0x7f5bd814fc20 2024-09-16T00:08:18.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.378+0000 7f5bdcce6640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5bd814fc20 0x7f5bd806d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:18.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.379+0000 7f5bdcce6640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5bd814fc20 0x7f5bd806d680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:18.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.379+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd81545a0 con 0x7f5bd814fc20 2024-09-16T00:08:18.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.395+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5bd81545a0 con 0x7f5bd814fc20 2024-09-16T00:08:18.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.396+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5bd81545a0 con 0x7f5bd814fc20 2024-09-16T00:08:18.398 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:18.398 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:18.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.397+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f5bd8062f00 0x7f5bd80cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:18.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.397+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f5bd81545a0 con 0x7f5bd8062f00 2024-09-16T00:08:18.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.397+0000 7f5bddce8640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f5bd8062f00 0x7f5bd80cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:18.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.397+0000 7f5bddce8640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f5bd8062f00 0x7f5bd80cbee0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:18.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.398+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd81545a0 con 0x7f5bd8062f00 2024-09-16T00:08:18.409 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:18.409 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:18.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.411+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5bd8063590 con 0x7f5bd8062f00 2024-09-16T00:08:18.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.413+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5bd8063590 con 0x7f5bd8062f00 2024-09-16T00:08:18.414 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:18.414 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:18.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.413+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5bd802c640 0x7f5bd802ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:18.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.413+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f5bd8063590 con 0x7f5bd802c640 2024-09-16T00:08:18.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.413+0000 7f5bdd4e7640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5bd802c640 0x7f5bd802ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:18.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.414+0000 7f5bdd4e7640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5bd802c640 0x7f5bd802ca60 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:18.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.414+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd8063590 con 0x7f5bd802c640 2024-09-16T00:08:18.418 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:08:18.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.432+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5bd8063590 con 0x7f5bd802c640 2024-09-16T00:08:18.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.434+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5bd8063590 con 0x7f5bd802c640 2024-09-16T00:08:18.435 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:18.436 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:18.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.434+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5bd802d4b0 0x7f5bd8206fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:18.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.434+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f5bd80cccf0 con 0x7f5bd802d4b0 2024-09-16T00:08:18.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.434+0000 7f5bdcce6640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5bd802d4b0 0x7f5bd8206fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:18.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.434+0000 7f5bdcce6640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5bd802d4b0 0x7f5bd8206fc0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:18.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.435+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd80cccf0 con 0x7f5bd802d4b0 2024-09-16T00:08:18.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.448+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5bd81545a0 con 0x7f5bd802d4b0 2024-09-16T00:08:18.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.450+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5bd81545a0 con 0x7f5bd802d4b0 2024-09-16T00:08:18.451 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:18.452 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:18.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.451+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5bd802d910 0x7f5bd8209a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:18.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.451+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f5bd820c7f0 con 0x7f5bd802d910 2024-09-16T00:08:18.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.451+0000 7f5bddce8640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5bd802d910 0x7f5bd8209a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:18.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.451+0000 7f5bddce8640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5bd802d910 0x7f5bd8209a20 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:18.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.452+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd820c7f0 con 0x7f5bd802d910 2024-09-16T00:08:18.460 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:18.458+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:18.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.465+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5bd8209f60 con 0x7f5bd802d910 2024-09-16T00:08:18.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.467+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5bd8209f60 con 0x7f5bd802d910 2024-09-16T00:08:18.468 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:18.469 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:18.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.467+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5bd820a470 0x7f5bd820c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:18.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.467+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f5bd820f980 con 0x7f5bd820a470 2024-09-16T00:08:18.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.467+0000 7f5bdd4e7640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5bd820a470 0x7f5bd820c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:18.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.467+0000 7f5bdd4e7640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5bd820a470 0x7f5bd820c930 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:18.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.468+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5bd820f980 con 0x7f5bd820a470 2024-09-16T00:08:18.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.481+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5bd820f950 con 0x7f5bd820a470 2024-09-16T00:08:18.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bcdffb640 1 -- 172.21.15.139:0/2591782279 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5bd820f950 con 0x7f5bd820a470 2024-09-16T00:08:18.484 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:18.486 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:18.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5bd81531f0 msgr2=0x7f5bd80c9830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:18.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5bd81531f0 0x7f5bd80c9830 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:18.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5bd814f840 msgr2=0x7f5bd806cc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:18.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5bd814f840 0x7f5bd806cc30 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:18.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5bd814fc20 msgr2=0x7f5bd806d680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:18.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5bd814fc20 0x7f5bd806d680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:18.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f5bd8062f00 msgr2=0x7f5bd80cbee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:18.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f5bd8062f00 0x7f5bd80cbee0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:18.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5bd802c640 msgr2=0x7f5bd802ca60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:18.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5bd802c640 0x7f5bd802ca60 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:18.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bddce8640 1 -- 172.21.15.139:0/2591782279 reap_dead start 2024-09-16T00:08:18.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bddce8640 1 -- 172.21.15.139:0/2591782279 reap_dead start 2024-09-16T00:08:18.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5bd802d4b0 msgr2=0x7f5bd8206fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:18.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5bd802d4b0 0x7f5bd8206fc0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:18.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5bd802d910 msgr2=0x7f5bd8209a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:18.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5bd802d910 0x7f5bd8209a20 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:18.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5bd820a470 msgr2=0x7f5bd820c930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5bd820a470 0x7f5bd820c930 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5bbc06c2b0 msgr2=0x7f5bbc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5bbc06c2b0 0x7f5bbc06e770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4000f60 tx=0x7f5bd4007e80 comp rx=0 tx=0).stop 2024-09-16T00:08:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd8071ad0 msgr2=0x7f5bd80ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bddce8640 1 -- 172.21.15.139:0/2591782279 reap_dead start 2024-09-16T00:08:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.483+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd8071ad0 0x7f5bd80ce320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8000c00 tx=0x7f5bc80046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.484+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 shutdown_connections 2024-09-16T00:08:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.484+0000 7f5bdf74b640 1 --2- 172.21.15.139:0/2591782279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd8071ad0 0x7f5bd80ce320 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.484+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 >> 172.21.15.139:0/2591782279 conn(0x7f5bd806fb70 msgr2=0x7f5bd81564e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.484+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 shutdown_connections 2024-09-16T00:08:18.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:18.484+0000 7f5bdf74b640 1 -- 172.21.15.139:0/2591782279 wait complete. 2024-09-16T00:08:18.560 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:18.560+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:18.562 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:18.562 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:18.562 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:18.562 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:18.562 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:18.571 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:08:18.662 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:18.660+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:18.678 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:18.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.594903983000002, 2024-09-16T00:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:08:18.679 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.680 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:08:18.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:08:18.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:08:18.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:08:18.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.690 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.515372352999997, 2024-09-16T00:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093071190000000008, 2024-09-16T00:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.691 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 106 2024-09-16T00:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160362+0000", 2024-09-16T00:08:18.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160438+0000", 2024-09-16T00:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6711000000000005e-05 2024-09-16T00:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160444+0000", 2024-09-16T00:08:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0530000000000003e-06 2024-09-16T00:08:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160449+0000", 2024-09-16T00:08:18.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2189999999999998e-06 2024-09-16T00:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161270+0000", 2024-09-16T00:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082099199999999999 2024-09-16T00:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161348+0000", 2024-09-16T00:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8361000000000004e-05 2024-09-16T00:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161472+0000", 2024-09-16T00:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123523 2024-09-16T00:08:18.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163461+0000", 2024-09-16T00:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019891560000000002 2024-09-16T00:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169418+0000", 2024-09-16T00:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059568490000000002 2024-09-16T00:08:18.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169570+0000", 2024-09-16T00:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152461 2024-09-16T00:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169668+0000", 2024-09-16T00:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T00:08:18.699 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.515276581999998, 2024-09-16T00:08:18.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092931679999999992, 2024-09-16T00:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107 2024-09-16T00:08:18.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0699999999999998e-07 2024-09-16T00:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160459+0000", 2024-09-16T00:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0370000000000001e-06 2024-09-16T00:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160463+0000", 2024-09-16T00:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999999e-06 2024-09-16T00:08:18.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160466+0000", 2024-09-16T00:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.644e-06 2024-09-16T00:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161873+0000", 2024-09-16T00:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014074840000000001 2024-09-16T00:08:18.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161949+0000", 2024-09-16T00:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5749999999999998e-05 2024-09-16T00:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162069+0000", 2024-09-16T00:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120257 2024-09-16T00:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163506+0000", 2024-09-16T00:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001436344 2024-09-16T00:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169704+0000", 2024-09-16T00:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061982829999999997 2024-09-16T00:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169730+0000", 2024-09-16T00:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6060000000000001e-05 2024-09-16T00:08:18.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169750+0000", 2024-09-16T00:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0092e-05 2024-09-16T00:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.709 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.515167269999999, 2024-09-16T00:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092280039999999997, 2024-09-16T00:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.710 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 108 2024-09-16T00:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160567+0000", 2024-09-16T00:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7300000000000001e-07 2024-09-16T00:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160569+0000", 2024-09-16T00:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5299999999999999e-06 2024-09-16T00:08:18.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160574+0000", 2024-09-16T00:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-16T00:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160577+0000", 2024-09-16T00:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.01e-06 2024-09-16T00:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162518+0000", 2024-09-16T00:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001940703 2024-09-16T00:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162571+0000", 2024-09-16T00:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2670000000000002e-05 2024-09-16T00:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162647+0000", 2024-09-16T00:08:18.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6778000000000002e-05 2024-09-16T00:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163544+0000", 2024-09-16T00:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089702599999999999 2024-09-16T00:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169754+0000", 2024-09-16T00:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062091739999999996 2024-09-16T00:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169776+0000", 2024-09-16T00:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2391999999999998e-05 2024-09-16T00:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169794+0000", 2024-09-16T00:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8318999999999999e-05 2024-09-16T00:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:18.719 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.515118719999997, 2024-09-16T00:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092220679999999999, 2024-09-16T00:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.720 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 109 2024-09-16T00:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:18.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9900000000000001e-07 2024-09-16T00:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160707+0000", 2024-09-16T00:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1621999999999998e-05 2024-09-16T00:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160713+0000", 2024-09-16T00:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.596e-06 2024-09-16T00:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160717+0000", 2024-09-16T00:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5829999999999998e-06 2024-09-16T00:08:18.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162869+0000", 2024-09-16T00:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021516310000000002 2024-09-16T00:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162946+0000", 2024-09-16T00:08:18.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7696999999999998e-05 2024-09-16T00:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163065+0000", 2024-09-16T00:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118581 2024-09-16T00:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164431+0000", 2024-09-16T00:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001366003 2024-09-16T00:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169797+0000", 2024-09-16T00:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005366244 2024-09-16T00:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169819+0000", 2024-09-16T00:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2036999999999999e-05 2024-09-16T00:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169837+0000", 2024-09-16T00:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7575e-05 2024-09-16T00:08:18.728 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.515009370999998, 2024-09-16T00:08:18.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092871919999999997, 2024-09-16T00:08:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 110 2024-09-16T00:08:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.730 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:18.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:08:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160728+0000", 2024-09-16T00:08:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0620000000000001e-06 2024-09-16T00:08:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160731+0000", 2024-09-16T00:08:18.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6500000000000002e-06 2024-09-16T00:08:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160734+0000", 2024-09-16T00:08:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.762e-06 2024-09-16T00:08:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163580+0000", 2024-09-16T00:08:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028464549999999999 2024-09-16T00:08:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163657+0000", 2024-09-16T00:08:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6695999999999999e-05 2024-09-16T00:08:18.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163798+0000", 2024-09-16T00:08:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014055 2024-09-16T00:08:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165483+0000", 2024-09-16T00:08:18.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016853090000000001 2024-09-16T00:08:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169970+0000", 2024-09-16T00:08:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00448709 2024-09-16T00:08:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169994+0000", 2024-09-16T00:08:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3575999999999999e-05 2024-09-16T00:08:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170011+0000", 2024-09-16T00:08:18.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7702000000000001e-05 2024-09-16T00:08:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.514995702, 2024-09-16T00:08:18.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093174859999999998, 2024-09-16T00:08:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.740 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 111 2024-09-16T00:08:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1e-07 2024-09-16T00:08:18.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160741+0000", 2024-09-16T00:08:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6110000000000001e-06 2024-09-16T00:08:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160744+0000", 2024-09-16T00:08:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3299999999999999e-06 2024-09-16T00:08:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160746+0000", 2024-09-16T00:08:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5469999999999999e-06 2024-09-16T00:08:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164069+0000", 2024-09-16T00:08:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033222709999999999 2024-09-16T00:08:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164113+0000", 2024-09-16T00:08:18.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4212999999999998e-05 2024-09-16T00:08:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164187+0000", 2024-09-16T00:08:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4136999999999998e-05 2024-09-16T00:08:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166429+0000", 2024-09-16T00:08:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022423180000000001 2024-09-16T00:08:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170016+0000", 2024-09-16T00:08:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035869449999999998 2024-09-16T00:08:18.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170038+0000", 2024-09-16T00:08:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2053e-05 2024-09-16T00:08:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170055+0000", 2024-09-16T00:08:18.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6850999999999998e-05 2024-09-16T00:08:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:18.749 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.514981986999999, 2024-09-16T00:08:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093480279999999995, 2024-09-16T00:08:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.750 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 112 2024-09-16T00:08:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160752+0000", 2024-09-16T00:08:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2700000000000001e-07 2024-09-16T00:08:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160808+0000", 2024-09-16T00:08:18.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5797999999999997e-05 2024-09-16T00:08:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160814+0000", 2024-09-16T00:08:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5980000000000002e-06 2024-09-16T00:08:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160819+0000", 2024-09-16T00:08:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8960000000000002e-06 2024-09-16T00:08:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164468+0000", 2024-09-16T00:08:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003649476 2024-09-16T00:08:18.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164525+0000", 2024-09-16T00:08:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6227999999999998e-05 2024-09-16T00:08:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164595+0000", 2024-09-16T00:08:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759000000000005e-05 2024-09-16T00:08:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167237+0000", 2024-09-16T00:08:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026410470000000001 2024-09-16T00:08:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170060+0000", 2024-09-16T00:08:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028234969999999999 2024-09-16T00:08:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170081+0000", 2024-09-16T00:08:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1242000000000001e-05 2024-09-16T00:08:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170100+0000", 2024-09-16T00:08:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8260000000000001e-05 2024-09-16T00:08:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.514904891999997, 2024-09-16T00:08:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093139409999999992, 2024-09-16T00:08:18.759 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 113 2024-09-16T00:08:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-07 2024-09-16T00:08:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160833+0000", 2024-09-16T00:08:18.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.557e-06 2024-09-16T00:08:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160839+0000", 2024-09-16T00:08:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3829999999999997e-06 2024-09-16T00:08:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160844+0000", 2024-09-16T00:08:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6909999999999996e-06 2024-09-16T00:08:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165008+0000", 2024-09-16T00:08:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041642129999999999 2024-09-16T00:08:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165085+0000", 2024-09-16T00:08:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6651999999999997e-05 2024-09-16T00:08:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165205+0000", 2024-09-16T00:08:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012045900000000001 2024-09-16T00:08:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167283+0000", 2024-09-16T00:08:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020782909999999999 2024-09-16T00:08:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170104+0000", 2024-09-16T00:08:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028207950000000001 2024-09-16T00:08:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170125+0000", 2024-09-16T00:08:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0852e-05 2024-09-16T00:08:18.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170143+0000", 2024-09-16T00:08:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7467000000000001e-05 2024-09-16T00:08:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:18.765 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.514858410000002, 2024-09-16T00:08:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093126960000000005, 2024-09-16T00:08:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-16T00:08:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160876+0000", 2024-09-16T00:08:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5700000000000002e-07 2024-09-16T00:08:18.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160880+0000", 2024-09-16T00:08:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5539999999999999e-06 2024-09-16T00:08:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160888+0000", 2024-09-16T00:08:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.157e-06 2024-09-16T00:08:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160894+0000", 2024-09-16T00:08:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7690000000000002e-06 2024-09-16T00:08:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165519+0000", 2024-09-16T00:08:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046249539999999997 2024-09-16T00:08:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165593+0000", 2024-09-16T00:08:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4039e-05 2024-09-16T00:08:18.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165715+0000", 2024-09-16T00:08:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012131899999999999 2024-09-16T00:08:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167309+0000", 2024-09-16T00:08:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001594011 2024-09-16T00:08:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170147+0000", 2024-09-16T00:08:18.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028387130000000001 2024-09-16T00:08:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170170+0000", 2024-09-16T00:08:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2517000000000001e-05 2024-09-16T00:08:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170188+0000", 2024-09-16T00:08:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8006e-05 2024-09-16T00:08:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.514817381999997, 2024-09-16T00:08:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093140040000000007, 2024-09-16T00:08:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-16T00:08:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.772 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160917+0000", 2024-09-16T00:08:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7599999999999996e-07 2024-09-16T00:08:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160992+0000", 2024-09-16T00:08:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4760999999999998e-05 2024-09-16T00:08:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161000+0000", 2024-09-16T00:08:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9829999999999996e-06 2024-09-16T00:08:18.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161007+0000", 2024-09-16T00:08:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4909999999999999e-06 2024-09-16T00:08:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165983+0000", 2024-09-16T00:08:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049755399999999997 2024-09-16T00:08:18.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166027+0000", 2024-09-16T00:08:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4109000000000001e-05 2024-09-16T00:08:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166100+0000", 2024-09-16T00:08:18.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3360000000000002e-05 2024-09-16T00:08:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169698+0000", 2024-09-16T00:08:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597795 2024-09-16T00:08:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170192+0000", 2024-09-16T00:08:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049447300000000005 2024-09-16T00:08:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170214+0000", 2024-09-16T00:08:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1546e-05 2024-09-16T00:08:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170230+0000", 2024-09-16T00:08:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6169999999999999e-05 2024-09-16T00:08:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.514707653000002, 2024-09-16T00:08:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092457029999999992, 2024-09-16T00:08:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.779 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-16T00:08:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7899999999999999e-07 2024-09-16T00:08:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161029+0000", 2024-09-16T00:08:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6630000000000002e-06 2024-09-16T00:08:18.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161036+0000", 2024-09-16T00:08:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T00:08:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161041+0000", 2024-09-16T00:08:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:08:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166468+0000", 2024-09-16T00:08:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054266050000000001 2024-09-16T00:08:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166511+0000", 2024-09-16T00:08:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3869999999999998e-05 2024-09-16T00:08:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166606+0000", 2024-09-16T00:08:18.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4437000000000004e-05 2024-09-16T00:08:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169955+0000", 2024-09-16T00:08:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033494810000000001 2024-09-16T00:08:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170234+0000", 2024-09-16T00:08:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027900899999999998 2024-09-16T00:08:18.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170255+0000", 2024-09-16T00:08:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0537999999999999e-05 2024-09-16T00:08:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170272+0000", 2024-09-16T00:08:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6611000000000001e-05 2024-09-16T00:08:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.514682508, 2024-09-16T00:08:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093511010000000006, 2024-09-16T00:08:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-16T00:08:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.63e-07 2024-09-16T00:08:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161055+0000", 2024-09-16T00:08:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6770000000000001e-06 2024-09-16T00:08:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161063+0000", 2024-09-16T00:08:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6469999999999998e-06 2024-09-16T00:08:18.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161068+0000", 2024-09-16T00:08:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.746e-06 2024-09-16T00:08:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166817+0000", 2024-09-16T00:08:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057494959999999998 2024-09-16T00:08:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166893+0000", 2024-09-16T00:08:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6210000000000004e-05 2024-09-16T00:08:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167019+0000", 2024-09-16T00:08:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125585 2024-09-16T00:08:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170275+0000", 2024-09-16T00:08:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032563620000000001 2024-09-16T00:08:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170309+0000", 2024-09-16T00:08:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3865999999999997e-05 2024-09-16T00:08:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170362+0000", 2024-09-16T00:08:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3094999999999999e-05 2024-09-16T00:08:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170402+0000", 2024-09-16T00:08:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9953999999999997e-05 2024-09-16T00:08:18.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.065950422999997, 2024-09-16T00:08:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097364389999999995, 2024-09-16T00:08:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138 2024-09-16T00:08:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609784+0000", 2024-09-16T00:08:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T00:08:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609787+0000", 2024-09-16T00:08:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280000000000001e-06 2024-09-16T00:08:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609794+0000", 2024-09-16T00:08:18.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6409999999999996e-06 2024-09-16T00:08:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609799+0000", 2024-09-16T00:08:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9359999999999998e-06 2024-09-16T00:08:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615759+0000", 2024-09-16T00:08:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059599910000000004 2024-09-16T00:08:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615806+0000", 2024-09-16T00:08:18.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7290000000000003e-05 2024-09-16T00:08:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615877+0000", 2024-09-16T00:08:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1000000000000005e-05 2024-09-16T00:08:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618961+0000", 2024-09-16T00:08:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030841390000000001 2024-09-16T00:08:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619436+0000", 2024-09-16T00:08:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047452799999999999 2024-09-16T00:08:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619482+0000", 2024-09-16T00:08:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5689000000000003e-05 2024-09-16T00:08:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619519+0000", 2024-09-16T00:08:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7938000000000002e-05 2024-09-16T00:08:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.797 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.065926709000003, 2024-09-16T00:08:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011812954000000001, 2024-09-16T00:08:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.798 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139 2024-09-16T00:08:18.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.799 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:18.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:18.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:18.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999998e-07 2024-09-16T00:08:18.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609810+0000", 2024-09-16T00:08:18.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2100000000000002e-06 2024-09-16T00:08:18.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609817+0000", 2024-09-16T00:08:18.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.28e-06 2024-09-16T00:08:18.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609825+0000", 2024-09-16T00:08:18.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:18.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616116+0000", 2024-09-16T00:08:18.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006290543 2024-09-16T00:08:18.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616187+0000", 2024-09-16T00:08:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1830999999999997e-05 2024-09-16T00:08:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616298+0000", 2024-09-16T00:08:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011044699999999999 2024-09-16T00:08:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619559+0000", 2024-09-16T00:08:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032614710000000002 2024-09-16T00:08:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621562+0000", 2024-09-16T00:08:18.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020028680000000001 2024-09-16T00:08:18.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621597+0000", 2024-09-16T00:08:18.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4625000000000001e-05 2024-09-16T00:08:18.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621620+0000", 2024-09-16T00:08:18.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2864e-05 2024-09-16T00:08:18.803 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.803 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.803 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.804 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:18.804 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:18.804 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.065898912999998, 2024-09-16T00:08:18.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011828211999999999, 2024-09-16T00:08:18.804 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.804 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.804 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.804 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.804 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.804 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.804 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 140 2024-09-16T00:08:18.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.805 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:18.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:18.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:18.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T00:08:18.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:18.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961000000000003e-05 2024-09-16T00:08:18.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609892+0000", 2024-09-16T00:08:18.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0516000000000001e-05 2024-09-16T00:08:18.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609898+0000", 2024-09-16T00:08:18.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0449999999999997e-06 2024-09-16T00:08:18.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616797+0000", 2024-09-16T00:08:18.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068998840000000002 2024-09-16T00:08:18.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616844+0000", 2024-09-16T00:08:18.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6501000000000001e-05 2024-09-16T00:08:18.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616935+0000", 2024-09-16T00:08:18.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1222000000000002e-05 2024-09-16T00:08:18.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619808+0000", 2024-09-16T00:08:18.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873198 2024-09-16T00:08:18.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621624+0000", 2024-09-16T00:08:18.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001815933 2024-09-16T00:08:18.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621646+0000", 2024-09-16T00:08:18.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1622999999999999e-05 2024-09-16T00:08:18.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621663+0000", 2024-09-16T00:08:18.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6854000000000002e-05 2024-09-16T00:08:18.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.810 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.810 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:18.811 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:18.811 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.065813104, 2024-09-16T00:08:18.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014331501999999999, 2024-09-16T00:08:18.811 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.811 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.811 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.811 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.811 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.811 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.811 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-16T00:08:18.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.811 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:18.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:18.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609921+0000", 2024-09-16T00:08:18.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9599999999999999e-07 2024-09-16T00:08:18.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609925+0000", 2024-09-16T00:08:18.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T00:08:18.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609932+0000", 2024-09-16T00:08:18.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8240000000000002e-06 2024-09-16T00:08:18.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609937+0000", 2024-09-16T00:08:18.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1070000000000004e-06 2024-09-16T00:08:18.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618987+0000", 2024-09-16T00:08:18.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090502719999999998 2024-09-16T00:08:18.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619064+0000", 2024-09-16T00:08:18.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6868999999999999e-05 2024-09-16T00:08:18.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619183+0000", 2024-09-16T00:08:18.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011961 2024-09-16T00:08:18.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.620052+0000", 2024-09-16T00:08:18.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086822799999999997 2024-09-16T00:08:18.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624197+0000", 2024-09-16T00:08:18.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041454930000000001 2024-09-16T00:08:18.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624229+0000", 2024-09-16T00:08:18.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1758999999999999e-05 2024-09-16T00:08:18.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624252+0000", 2024-09-16T00:08:18.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:08:18.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.817 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.817 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.817 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:18.817 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:18.817 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 53.593047296000002, 2024-09-16T00:08:18.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010093276999999999, 2024-09-16T00:08:18.817 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.818 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.818 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.818 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.818 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 157 2024-09-16T00:08:18.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.818 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:18.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:18.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:18.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4699999999999998e-07 2024-09-16T00:08:18.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082688+0000", 2024-09-16T00:08:18.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0209999999999998e-06 2024-09-16T00:08:18.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082692+0000", 2024-09-16T00:08:18.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6890000000000002e-06 2024-09-16T00:08:18.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082695+0000", 2024-09-16T00:08:18.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5339999999999998e-06 2024-09-16T00:08:18.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085303+0000", 2024-09-16T00:08:18.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026084789999999999 2024-09-16T00:08:18.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085374+0000", 2024-09-16T00:08:18.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1317999999999997e-05 2024-09-16T00:08:18.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085506+0000", 2024-09-16T00:08:18.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013118 2024-09-16T00:08:18.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086928+0000", 2024-09-16T00:08:18.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014224559999999999 2024-09-16T00:08:18.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092665+0000", 2024-09-16T00:08:18.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057369830000000002 2024-09-16T00:08:18.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092727+0000", 2024-09-16T00:08:18.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1810000000000006e-05 2024-09-16T00:08:18.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092779+0000", 2024-09-16T00:08:18.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2559999999999998e-05 2024-09-16T00:08:18.823 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.823 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.823 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.823 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:18.823 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:18.823 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 53.593035127999997, 2024-09-16T00:08:18.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010176388, 2024-09-16T00:08:18.824 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.824 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.824 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.824 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.824 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 158 2024-09-16T00:08:18.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.824 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:18.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:18.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082699+0000", 2024-09-16T00:08:18.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.11e-07 2024-09-16T00:08:18.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082742+0000", 2024-09-16T00:08:18.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3699000000000003e-05 2024-09-16T00:08:18.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082748+0000", 2024-09-16T00:08:18.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8690000000000001e-06 2024-09-16T00:08:18.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082751+0000", 2024-09-16T00:08:18.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.942e-06 2024-09-16T00:08:18.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085881+0000", 2024-09-16T00:08:18.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031303450000000001 2024-09-16T00:08:18.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085959+0000", 2024-09-16T00:08:18.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7874999999999996e-05 2024-09-16T00:08:18.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086086+0000", 2024-09-16T00:08:18.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012685499999999999 2024-09-16T00:08:18.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087127+0000", 2024-09-16T00:08:18.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001041298 2024-09-16T00:08:18.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092790+0000", 2024-09-16T00:08:18.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056628 2024-09-16T00:08:18.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092832+0000", 2024-09-16T00:08:18.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2227000000000001e-05 2024-09-16T00:08:18.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092875+0000", 2024-09-16T00:08:18.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2267e-05 2024-09-16T00:08:18.829 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.829 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.829 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.829 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:18.830 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:18.830 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 53.592977089999998, 2024-09-16T00:08:18.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010199547, 2024-09-16T00:08:18.830 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:18.830 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:18.830 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:18.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:18.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:18.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:18.830 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159 2024-09-16T00:08:18.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.830 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:18.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:18.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:18.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:18.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:18.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:18.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:18.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082757+0000", 2024-09-16T00:08:18.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2500000000000001e-07 2024-09-16T00:08:18.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:18.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082759+0000", 2024-09-16T00:08:18.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-16T00:08:18.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:18.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082763+0000", 2024-09-16T00:08:18.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4889999999999997e-06 2024-09-16T00:08:18.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:18.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082766+0000", 2024-09-16T00:08:18.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:08:18.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:18.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086571+0000", 2024-09-16T00:08:18.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038053269999999998 2024-09-16T00:08:18.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:18.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086619+0000", 2024-09-16T00:08:18.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7924999999999999e-05 2024-09-16T00:08:18.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086693+0000", 2024-09-16T00:08:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000005e-05 2024-09-16T00:08:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087383+0000", 2024-09-16T00:08:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069008299999999997 2024-09-16T00:08:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092882+0000", 2024-09-16T00:08:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054987450000000002 2024-09-16T00:08:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092921+0000", 2024-09-16T00:08:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9388999999999998e-05 2024-09-16T00:08:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092956+0000", 2024-09-16T00:08:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4647999999999998e-05 2024-09-16T00:08:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:18.835 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:18.836 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:18.837 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:18.836+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:18.837 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:08:18.938 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:18.937+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:18.947 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:18.947 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:18.947 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:18.947 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:18.957 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:08:19.039 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:19.038+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:19.062 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:19.062 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:19.062 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:19.063 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:19.063 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:19.072 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:08:19.156 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:19.156+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:19.219 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:19.220 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:19.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:19.220 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:19.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.220 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'41, mlcod=17'39)", 2024-09-16T00:08:19.220 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:19.220 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.196506845999998, 2024-09-16T00:08:19.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001808488, 2024-09-16T00:08:19.220 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.220 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.220 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.220 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:19.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:19.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021032+0000", 2024-09-16T00:08:19.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3e-07 2024-09-16T00:08:19.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021037+0000", 2024-09-16T00:08:19.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0640000000000001e-06 2024-09-16T00:08:19.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021047+0000", 2024-09-16T00:08:19.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.023e-05 2024-09-16T00:08:19.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021053+0000", 2024-09-16T00:08:19.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3160000000000002e-06 2024-09-16T00:08:19.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021092+0000", 2024-09-16T00:08:19.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9033999999999999e-05 2024-09-16T00:08:19.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021105+0000", 2024-09-16T00:08:19.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2768e-05 2024-09-16T00:08:19.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022811+0000", 2024-09-16T00:08:19.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001705788 2024-09-16T00:08:19.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022840+0000", 2024-09-16T00:08:19.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8558e-05 2024-09-16T00:08:19.224 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.225 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.225 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.225 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:85 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'55, mlcod=17'50)", 2024-09-16T00:08:19.225 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:19.225 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.183477844999999, 2024-09-16T00:08:19.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017909530000000001, 2024-09-16T00:08:19.225 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.226 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.226 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.226 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:19.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:19.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034061+0000", 2024-09-16T00:08:19.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0399999999999996e-07 2024-09-16T00:08:19.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034065+0000", 2024-09-16T00:08:19.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8840000000000002e-06 2024-09-16T00:08:19.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034074+0000", 2024-09-16T00:08:19.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:19.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034079+0000", 2024-09-16T00:08:19.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0000000000000004e-06 2024-09-16T00:08:19.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034125+0000", 2024-09-16T00:08:19.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5905000000000002e-05 2024-09-16T00:08:19.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.229 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.229 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034144+0000", 2024-09-16T00:08:19.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9593e-05 2024-09-16T00:08:19.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.229 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.229 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035817+0000", 2024-09-16T00:08:19.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016728240000000001 2024-09-16T00:08:19.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035851+0000", 2024-09-16T00:08:19.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3809000000000001e-05 2024-09-16T00:08:19.230 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.230 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.230 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.231 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:112 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'74, mlcod=17'65)", 2024-09-16T00:08:19.231 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:19.231 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.052865982999997, 2024-09-16T00:08:19.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018457020000000001, 2024-09-16T00:08:19.231 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.231 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.231 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.231 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:19.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:19.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164673+0000", 2024-09-16T00:08:19.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1200000000000003e-07 2024-09-16T00:08:19.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164710+0000", 2024-09-16T00:08:19.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7756999999999998e-05 2024-09-16T00:08:19.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164719+0000", 2024-09-16T00:08:19.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2849999999999995e-06 2024-09-16T00:08:19.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164723+0000", 2024-09-16T00:08:19.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3220000000000003e-06 2024-09-16T00:08:19.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164760+0000", 2024-09-16T00:08:19.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7311999999999998e-05 2024-09-16T00:08:19.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.235 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164771+0000", 2024-09-16T00:08:19.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0869e-05 2024-09-16T00:08:19.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.235 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166495+0000", 2024-09-16T00:08:19.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001723863 2024-09-16T00:08:19.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.236 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166518+0000", 2024-09-16T00:08:19.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2682000000000001e-05 2024-09-16T00:08:19.236 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.236 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.236 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.236 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'66)", 2024-09-16T00:08:19.236 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:19.236 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.051362003999998, 2024-09-16T00:08:19.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032925509999999999, 2024-09-16T00:08:19.237 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.237 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.237 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.237 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:19.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:19.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166177+0000", 2024-09-16T00:08:19.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:19.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166216+0000", 2024-09-16T00:08:19.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9008000000000003e-05 2024-09-16T00:08:19.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166226+0000", 2024-09-16T00:08:19.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0081999999999999e-05 2024-09-16T00:08:19.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166230+0000", 2024-09-16T00:08:19.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5079999999999999e-06 2024-09-16T00:08:19.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166259+0000", 2024-09-16T00:08:19.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8928000000000002e-05 2024-09-16T00:08:19.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166271+0000", 2024-09-16T00:08:19.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1531e-05 2024-09-16T00:08:19.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169449+0000", 2024-09-16T00:08:19.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003178143 2024-09-16T00:08:19.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169468+0000", 2024-09-16T00:08:19.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9700000000000001e-05 2024-09-16T00:08:19.242 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.242 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.242 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.242 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'66)", 2024-09-16T00:08:19.242 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:19.243 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.050900544999998, 2024-09-16T00:08:19.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003179374, 2024-09-16T00:08:19.243 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.243 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.243 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.243 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:19.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:19.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166638+0000", 2024-09-16T00:08:19.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2300000000000002e-07 2024-09-16T00:08:19.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166641+0000", 2024-09-16T00:08:19.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0079999999999998e-06 2024-09-16T00:08:19.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166646+0000", 2024-09-16T00:08:19.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4970000000000001e-06 2024-09-16T00:08:19.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166650+0000", 2024-09-16T00:08:19.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5729999999999999e-06 2024-09-16T00:08:19.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166682+0000", 2024-09-16T00:08:19.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2218999999999998e-05 2024-09-16T00:08:19.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166694+0000", 2024-09-16T00:08:19.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1902999999999999e-05 2024-09-16T00:08:19.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169784+0000", 2024-09-16T00:08:19.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003089545 2024-09-16T00:08:19.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169817+0000", 2024-09-16T00:08:19.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3206e-05 2024-09-16T00:08:19.250 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.250 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.250 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.250 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:117 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'66)", 2024-09-16T00:08:19.250 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:19.250 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.050456484999998, 2024-09-16T00:08:19.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00275326, 2024-09-16T00:08:19.250 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.250 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.250 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.250 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167082+0000", 2024-09-16T00:08:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:08:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167086+0000", 2024-09-16T00:08:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1559999999999997e-06 2024-09-16T00:08:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167096+0000", 2024-09-16T00:08:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8390000000000002e-06 2024-09-16T00:08:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167101+0000", 2024-09-16T00:08:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:08:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167130+0000", 2024-09-16T00:08:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9649e-05 2024-09-16T00:08:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167142+0000", 2024-09-16T00:08:19.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1435e-05 2024-09-16T00:08:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169818+0000", 2024-09-16T00:08:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002676056 2024-09-16T00:08:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169835+0000", 2024-09-16T00:08:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6980999999999999e-05 2024-09-16T00:08:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.254 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:130 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'88, mlcod=17'85)", 2024-09-16T00:08:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 54.607657156000002, 2024-09-16T00:08:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002880456, 2024-09-16T00:08:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:19.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5699999999999998e-07 2024-09-16T00:08:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609885+0000", 2024-09-16T00:08:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.608e-06 2024-09-16T00:08:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609891+0000", 2024-09-16T00:08:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:19.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609895+0000", 2024-09-16T00:08:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.54e-06 2024-09-16T00:08:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609931+0000", 2024-09-16T00:08:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5893999999999999e-05 2024-09-16T00:08:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609945+0000", 2024-09-16T00:08:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4259e-05 2024-09-16T00:08:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612721+0000", 2024-09-16T00:08:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027757849999999998 2024-09-16T00:08:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612761+0000", 2024-09-16T00:08:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0638e-05 2024-09-16T00:08:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.260 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-16T00:08:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 54.607273182999997, 2024-09-16T00:08:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028016529999999999, 2024-09-16T00:08:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:19.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:19.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:19.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.63e-07 2024-09-16T00:08:19.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610344+0000", 2024-09-16T00:08:19.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8996000000000001e-05 2024-09-16T00:08:19.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610352+0000", 2024-09-16T00:08:19.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:19.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610356+0000", 2024-09-16T00:08:19.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0790000000000002e-06 2024-09-16T00:08:19.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610388+0000", 2024-09-16T00:08:19.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1541000000000002e-05 2024-09-16T00:08:19.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610400+0000", 2024-09-16T00:08:19.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2075e-05 2024-09-16T00:08:19.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613045+0000", 2024-09-16T00:08:19.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002645213 2024-09-16T00:08:19.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613066+0000", 2024-09-16T00:08:19.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1353e-05 2024-09-16T00:08:19.265 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.266 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.266 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.266 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-16T00:08:19.266 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:19.266 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 54.604013682999998, 2024-09-16T00:08:19.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027254850000000001, 2024-09-16T00:08:19.266 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.266 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.266 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.266 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:19.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:19.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613525+0000", 2024-09-16T00:08:19.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2800000000000002e-07 2024-09-16T00:08:19.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613528+0000", 2024-09-16T00:08:19.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9730000000000002e-06 2024-09-16T00:08:19.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613534+0000", 2024-09-16T00:08:19.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4339999999999997e-06 2024-09-16T00:08:19.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613538+0000", 2024-09-16T00:08:19.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5149999999999998e-06 2024-09-16T00:08:19.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613566+0000", 2024-09-16T00:08:19.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8104000000000001e-05 2024-09-16T00:08:19.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613578+0000", 2024-09-16T00:08:19.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2404e-05 2024-09-16T00:08:19.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616217+0000", 2024-09-16T00:08:19.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026389830000000001 2024-09-16T00:08:19.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616250+0000", 2024-09-16T00:08:19.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2644000000000001e-05 2024-09-16T00:08:19.271 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.271 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.271 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.271 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:134 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'92, mlcod=18'87)", 2024-09-16T00:08:19.271 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:19.271 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 54.603475543999998, 2024-09-16T00:08:19.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025322399999999998, 2024-09-16T00:08:19.271 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.271 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.271 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.271 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:19.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:19.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614063+0000", 2024-09-16T00:08:19.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8700000000000005e-07 2024-09-16T00:08:19.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614096+0000", 2024-09-16T00:08:19.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3136000000000003e-05 2024-09-16T00:08:19.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614106+0000", 2024-09-16T00:08:19.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9170000000000001e-06 2024-09-16T00:08:19.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614111+0000", 2024-09-16T00:08:19.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4390000000000002e-06 2024-09-16T00:08:19.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614141+0000", 2024-09-16T00:08:19.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.04e-05 2024-09-16T00:08:19.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614152+0000", 2024-09-16T00:08:19.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0987e-05 2024-09-16T00:08:19.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616573+0000", 2024-09-16T00:08:19.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002421351 2024-09-16T00:08:19.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616595+0000", 2024-09-16T00:08:19.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1322999999999999e-05 2024-09-16T00:08:19.276 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.276 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.276 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.276 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:135 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'93, mlcod=18'87)", 2024-09-16T00:08:19.276 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:19.276 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 54.602955506000001, 2024-09-16T00:08:19.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020304659999999999, 2024-09-16T00:08:19.276 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.276 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.277 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.277 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:19.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:19.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614583+0000", 2024-09-16T00:08:19.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:19.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614587+0000", 2024-09-16T00:08:19.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3100000000000002e-06 2024-09-16T00:08:19.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614597+0000", 2024-09-16T00:08:19.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7450000000000007e-06 2024-09-16T00:08:19.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614602+0000", 2024-09-16T00:08:19.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6099999999999999e-06 2024-09-16T00:08:19.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614641+0000", 2024-09-16T00:08:19.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9767000000000001e-05 2024-09-16T00:08:19.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614658+0000", 2024-09-16T00:08:19.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6205000000000002e-05 2024-09-16T00:08:19.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616596+0000", 2024-09-16T00:08:19.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019386659999999999 2024-09-16T00:08:19.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616613+0000", 2024-09-16T00:08:19.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6555999999999999e-05 2024-09-16T00:08:19.281 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.281 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.281 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.281 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:139 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'97, mlcod=18'87)", 2024-09-16T00:08:19.282 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:19.282 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 54.601194835000001, 2024-09-16T00:08:19.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030908139999999999, 2024-09-16T00:08:19.282 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.282 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.282 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.282 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.282 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:19.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:19.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616344+0000", 2024-09-16T00:08:19.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3000000000000001e-07 2024-09-16T00:08:19.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616347+0000", 2024-09-16T00:08:19.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7740000000000002e-06 2024-09-16T00:08:19.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616355+0000", 2024-09-16T00:08:19.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4699999999999996e-06 2024-09-16T00:08:19.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616358+0000", 2024-09-16T00:08:19.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5899999999999999e-06 2024-09-16T00:08:19.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616393+0000", 2024-09-16T00:08:19.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4529000000000001e-05 2024-09-16T00:08:19.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616404+0000", 2024-09-16T00:08:19.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0726e-05 2024-09-16T00:08:19.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619411+0000", 2024-09-16T00:08:19.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030076249999999999 2024-09-16T00:08:19.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.288 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619434+0000", 2024-09-16T00:08:19.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-05 2024-09-16T00:08:19.288 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.289 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.289 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.289 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:140 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'98, mlcod=18'89)", 2024-09-16T00:08:19.289 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:19.289 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 54.600553441999999, 2024-09-16T00:08:19.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027276150000000001, 2024-09-16T00:08:19.290 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.290 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.290 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.290 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:19.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:19.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616985+0000", 2024-09-16T00:08:19.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4099999999999999e-07 2024-09-16T00:08:19.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617055+0000", 2024-09-16T00:08:19.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9923000000000001e-05 2024-09-16T00:08:19.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617063+0000", 2024-09-16T00:08:19.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:19.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617070+0000", 2024-09-16T00:08:19.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9090000000000003e-06 2024-09-16T00:08:19.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617102+0000", 2024-09-16T00:08:19.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2620999999999998e-05 2024-09-16T00:08:19.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617114+0000", 2024-09-16T00:08:19.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1228e-05 2024-09-16T00:08:19.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619690+0000", 2024-09-16T00:08:19.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025763890000000001 2024-09-16T00:08:19.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619712+0000", 2024-09-16T00:08:19.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2171000000000002e-05 2024-09-16T00:08:19.295 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.295 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.295 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.295 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:19.295 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:19.295 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.601724341999997, 2024-09-16T00:08:19.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027355320000000002, 2024-09-16T00:08:19.295 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.295 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.296 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.296 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:19.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:19.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615815+0000", 2024-09-16T00:08:19.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1769999999999999e-06 2024-09-16T00:08:19.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615825+0000", 2024-09-16T00:08:19.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9000000000000001e-06 2024-09-16T00:08:19.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615840+0000", 2024-09-16T00:08:19.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4854e-05 2024-09-16T00:08:19.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615848+0000", 2024-09-16T00:08:19.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0940000000000008e-06 2024-09-16T00:08:19.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615926+0000", 2024-09-16T00:08:19.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8176999999999994e-05 2024-09-16T00:08:19.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615945+0000", 2024-09-16T00:08:19.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9663000000000001e-05 2024-09-16T00:08:19.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618509+0000", 2024-09-16T00:08:19.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002563802 2024-09-16T00:08:19.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618549+0000", 2024-09-16T00:08:19.300 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9864999999999999e-05 2024-09-16T00:08:19.300 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.300 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.300 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.300 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:19.300 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:19.301 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.601635201999997, 2024-09-16T00:08:19.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061593860000000002, 2024-09-16T00:08:19.301 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.301 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.301 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.301 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:19.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:19.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615907+0000", 2024-09-16T00:08:19.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9389999999999999e-06 2024-09-16T00:08:19.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615920+0000", 2024-09-16T00:08:19.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2989999999999999e-05 2024-09-16T00:08:19.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.303 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615931+0000", 2024-09-16T00:08:19.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.164e-05 2024-09-16T00:08:19.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.303 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615939+0000", 2024-09-16T00:08:19.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8029999999999997e-06 2024-09-16T00:08:19.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616310+0000", 2024-09-16T00:08:19.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00037111400000000003 2024-09-16T00:08:19.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.304 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.304 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616406+0000", 2024-09-16T00:08:19.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5372999999999996e-05 2024-09-16T00:08:19.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.304 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.304 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:19.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616514+0000", 2024-09-16T00:08:19.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108564 2024-09-16T00:08:19.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:19.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618158+0000", 2024-09-16T00:08:19.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016442449999999999 2024-09-16T00:08:19.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:19.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621972+0000", 2024-09-16T00:08:19.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038132299999999999 2024-09-16T00:08:19.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.306 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622018+0000", 2024-09-16T00:08:19.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6239000000000002e-05 2024-09-16T00:08:19.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.306 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.306 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622062+0000", 2024-09-16T00:08:19.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4249000000000003e-05 2024-09-16T00:08:19.306 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.306 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.306 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.306 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:19.306 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:19.307 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.601581734, 2024-09-16T00:08:19.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006258914, 2024-09-16T00:08:19.307 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.307 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.307 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.307 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:19.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:19.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615957+0000", 2024-09-16T00:08:19.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1399999999999997e-07 2024-09-16T00:08:19.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615960+0000", 2024-09-16T00:08:19.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0860000000000002e-06 2024-09-16T00:08:19.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615967+0000", 2024-09-16T00:08:19.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9059999999999996e-06 2024-09-16T00:08:19.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615972+0000", 2024-09-16T00:08:19.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5679999999999999e-06 2024-09-16T00:08:19.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616717+0000", 2024-09-16T00:08:19.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074438 2024-09-16T00:08:19.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616781+0000", 2024-09-16T00:08:19.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4808999999999996e-05 2024-09-16T00:08:19.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:19.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616885+0000", 2024-09-16T00:08:19.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010404900000000001 2024-09-16T00:08:19.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:19.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:19.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012773870000000001 2024-09-16T00:08:19.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:19.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622116+0000", 2024-09-16T00:08:19.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039533470000000003 2024-09-16T00:08:19.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622173+0000", 2024-09-16T00:08:19.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6347000000000001e-05 2024-09-16T00:08:19.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622215+0000", 2024-09-16T00:08:19.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2521000000000002e-05 2024-09-16T00:08:19.312 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.312 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.312 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.312 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:19.312 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:19.312 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.601557997, 2024-09-16T00:08:19.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041725529999999999, 2024-09-16T00:08:19.313 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.313 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.313 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.313 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:19.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:19.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:19.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8799999999999998e-07 2024-09-16T00:08:19.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615990+0000", 2024-09-16T00:08:19.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2150000000000007e-06 2024-09-16T00:08:19.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:08:19.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7159999999999997e-06 2024-09-16T00:08:19.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616001+0000", 2024-09-16T00:08:19.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9339999999999997e-06 2024-09-16T00:08:19.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616293+0000", 2024-09-16T00:08:19.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029210899999999998 2024-09-16T00:08:19.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616401+0000", 2024-09-16T00:08:19.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108087 2024-09-16T00:08:19.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:19.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616561+0000", 2024-09-16T00:08:19.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015940600000000001 2024-09-16T00:08:19.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:19.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618146+0000", 2024-09-16T00:08:19.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015857040000000001 2024-09-16T00:08:19.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:19.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620091+0000", 2024-09-16T00:08:19.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001944837 2024-09-16T00:08:19.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620126+0000", 2024-09-16T00:08:19.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4242999999999997e-05 2024-09-16T00:08:19.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620152+0000", 2024-09-16T00:08:19.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6914e-05 2024-09-16T00:08:19.318 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.318 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.318 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.319 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:19.319 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:19.319 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.601519772000003, 2024-09-16T00:08:19.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00630628, 2024-09-16T00:08:19.319 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.319 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.319 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.319 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:19.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:19.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:08:19.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5999999999999999e-07 2024-09-16T00:08:19.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616022+0000", 2024-09-16T00:08:19.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0149999999999999e-06 2024-09-16T00:08:19.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616031+0000", 2024-09-16T00:08:19.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9770000000000006e-06 2024-09-16T00:08:19.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616036+0000", 2024-09-16T00:08:19.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2639999999999999e-06 2024-09-16T00:08:19.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617057+0000", 2024-09-16T00:08:19.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010212999999999999 2024-09-16T00:08:19.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617146+0000", 2024-09-16T00:08:19.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9234000000000007e-05 2024-09-16T00:08:19.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:19.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617232+0000", 2024-09-16T00:08:19.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5218999999999998e-05 2024-09-16T00:08:19.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:19.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619107+0000", 2024-09-16T00:08:19.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001875016 2024-09-16T00:08:19.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:19.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622259+0000", 2024-09-16T00:08:19.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031527930000000001 2024-09-16T00:08:19.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622290+0000", 2024-09-16T00:08:19.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1022999999999998e-05 2024-09-16T00:08:19.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622324+0000", 2024-09-16T00:08:19.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3979000000000001e-05 2024-09-16T00:08:19.324 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.324 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.324 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.324 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:19.324 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:19.324 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.601494469999999, 2024-09-16T00:08:19.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042412739999999997, 2024-09-16T00:08:19.325 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.325 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.325 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.325 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:19.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:19.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616044+0000", 2024-09-16T00:08:19.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:19.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616052+0000", 2024-09-16T00:08:19.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.439e-06 2024-09-16T00:08:19.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616059+0000", 2024-09-16T00:08:19.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1389999999999998e-06 2024-09-16T00:08:19.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616065+0000", 2024-09-16T00:08:19.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:08:19.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616823+0000", 2024-09-16T00:08:19.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075781199999999996 2024-09-16T00:08:19.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616918+0000", 2024-09-16T00:08:19.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5174999999999996e-05 2024-09-16T00:08:19.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:19.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617019+0000", 2024-09-16T00:08:19.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101161 2024-09-16T00:08:19.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:19.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618503+0000", 2024-09-16T00:08:19.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014836750000000001 2024-09-16T00:08:19.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:19.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620201+0000", 2024-09-16T00:08:19.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00169805 2024-09-16T00:08:19.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620255+0000", 2024-09-16T00:08:19.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4499000000000001e-05 2024-09-16T00:08:19.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620285+0000", 2024-09-16T00:08:19.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9349e-05 2024-09-16T00:08:19.330 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.330 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.330 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.330 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:19.330 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:19.330 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.601465478999998, 2024-09-16T00:08:19.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063682799999999996, 2024-09-16T00:08:19.330 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.331 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.331 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:19.331 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:19.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:19.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:19.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9900000000000001e-07 2024-09-16T00:08:19.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616079+0000", 2024-09-16T00:08:19.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9009999999999999e-06 2024-09-16T00:08:19.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616086+0000", 2024-09-16T00:08:19.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.776e-06 2024-09-16T00:08:19.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.333 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616091+0000", 2024-09-16T00:08:19.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5600000000000001e-06 2024-09-16T00:08:19.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.333 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618339+0000", 2024-09-16T00:08:19.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022476179999999998 2024-09-16T00:08:19.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.333 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:19.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9025999999999997e-05 2024-09-16T00:08:19.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622441+0000", 2024-09-16T00:08:19.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040029999999999996 2024-09-16T00:08:19.334 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.334 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.334 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.334 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.334 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.334 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:19.336 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:08:19.336 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:19.335+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:19.436 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:19.436+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:19.483 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:19.483 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:19.483 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:19.483 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:19.492 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:08:19.577 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:19.576+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:19.638 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:19.637+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:19.640 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:19.640 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:19.640 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:19.640 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:19.640 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:19.649 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:08:19.739 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:19.738+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:19.801 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:19.801 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:19.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:19.802 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:19.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.802 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:19.802 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:19.802 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.184740310000002, 2024-09-16T00:08:19.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037514929999999998, 2024-09-16T00:08:19.802 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.802 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.802 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.802 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:19.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:19.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615364+0000", 2024-09-16T00:08:19.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8430000000000003e-06 2024-09-16T00:08:19.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615371+0000", 2024-09-16T00:08:19.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.148e-06 2024-09-16T00:08:19.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615380+0000", 2024-09-16T00:08:19.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5889999999999992e-06 2024-09-16T00:08:19.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615389+0000", 2024-09-16T00:08:19.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1620000000000008e-06 2024-09-16T00:08:19.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615525+0000", 2024-09-16T00:08:19.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000136083 2024-09-16T00:08:19.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615662+0000", 2024-09-16T00:08:19.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013696100000000001 2024-09-16T00:08:19.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:19.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615799+0000", 2024-09-16T00:08:19.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013781399999999999 2024-09-16T00:08:19.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:19.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:19.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002363502 2024-09-16T00:08:19.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:19.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619018+0000", 2024-09-16T00:08:19.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085522199999999999 2024-09-16T00:08:19.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619068+0000", 2024-09-16T00:08:19.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9545000000000001e-05 2024-09-16T00:08:19.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.807 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619111+0000", 2024-09-16T00:08:19.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3624000000000001e-05 2024-09-16T00:08:19.807 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.807 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.807 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.807 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:19.807 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:19.807 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.184688239000003, 2024-09-16T00:08:19.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043656110000000001, 2024-09-16T00:08:19.808 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.808 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.808 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.808 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:19.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:19.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:19.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2199999999999999e-07 2024-09-16T00:08:19.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615416+0000", 2024-09-16T00:08:19.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5590000000000001e-06 2024-09-16T00:08:19.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615423+0000", 2024-09-16T00:08:19.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6250000000000001e-06 2024-09-16T00:08:19.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615426+0000", 2024-09-16T00:08:19.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2540000000000001e-06 2024-09-16T00:08:19.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616075+0000", 2024-09-16T00:08:19.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064968800000000004 2024-09-16T00:08:19.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616142+0000", 2024-09-16T00:08:19.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6203000000000003e-05 2024-09-16T00:08:19.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:19.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616247+0000", 2024-09-16T00:08:19.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010572499999999999 2024-09-16T00:08:19.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:19.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618913+0000", 2024-09-16T00:08:19.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026654339999999999 2024-09-16T00:08:19.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:19.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619712+0000", 2024-09-16T00:08:19.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079885199999999996 2024-09-16T00:08:19.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619744+0000", 2024-09-16T00:08:19.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2790000000000003e-05 2024-09-16T00:08:19.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.813 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619778+0000", 2024-09-16T00:08:19.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3059000000000003e-05 2024-09-16T00:08:19.813 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.813 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.813 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.813 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:19.813 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:19.813 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.184652159999999, 2024-09-16T00:08:19.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087673219999999993, 2024-09-16T00:08:19.813 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.813 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.813 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.814 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:19.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:19.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:19.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:19.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:08:19.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5392000000000001e-05 2024-09-16T00:08:19.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615470+0000", 2024-09-16T00:08:19.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1999999999999999e-06 2024-09-16T00:08:19.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615474+0000", 2024-09-16T00:08:19.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7359999999999999e-06 2024-09-16T00:08:19.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617614+0000", 2024-09-16T00:08:19.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021398189999999998 2024-09-16T00:08:19.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617724+0000", 2024-09-16T00:08:19.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110808 2024-09-16T00:08:19.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:19.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617856+0000", 2024-09-16T00:08:19.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131676 2024-09-16T00:08:19.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:19.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:19.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029629280000000001 2024-09-16T00:08:19.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:19.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624158+0000", 2024-09-16T00:08:19.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033386549999999998 2024-09-16T00:08:19.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.818 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.818 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.818 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624189+0000", 2024-09-16T00:08:19.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1508999999999999e-05 2024-09-16T00:08:19.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.818 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.818 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.818 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624215+0000", 2024-09-16T00:08:19.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6228999999999999e-05 2024-09-16T00:08:19.818 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.818 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.818 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.819 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.819 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:19.819 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:19.819 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.184618145000002, 2024-09-16T00:08:19.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053285279999999999, 2024-09-16T00:08:19.819 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.819 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.819 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.819 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.819 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.819 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:19.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:19.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:19.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5600000000000002e-07 2024-09-16T00:08:19.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615484+0000", 2024-09-16T00:08:19.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.688e-06 2024-09-16T00:08:19.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615488+0000", 2024-09-16T00:08:19.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.658e-06 2024-09-16T00:08:19.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615490+0000", 2024-09-16T00:08:19.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.728e-06 2024-09-16T00:08:19.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616441+0000", 2024-09-16T00:08:19.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00095065500000000001 2024-09-16T00:08:19.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616539+0000", 2024-09-16T00:08:19.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8335e-05 2024-09-16T00:08:19.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:19.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616631+0000", 2024-09-16T00:08:19.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1260000000000004e-05 2024-09-16T00:08:19.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.824 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:19.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619837+0000", 2024-09-16T00:08:19.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003206899 2024-09-16T00:08:19.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.824 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:19.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620725+0000", 2024-09-16T00:08:19.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088713599999999996 2024-09-16T00:08:19.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620772+0000", 2024-09-16T00:08:19.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7639999999999998e-05 2024-09-16T00:08:19.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620811+0000", 2024-09-16T00:08:19.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8272999999999998e-05 2024-09-16T00:08:19.826 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.826 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.826 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.826 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:19.826 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:19.826 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.184203056999998, 2024-09-16T00:08:19.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029886169999999998, 2024-09-16T00:08:19.826 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.827 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.827 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.827 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:19.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:19.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615898+0000", 2024-09-16T00:08:19.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.237e-06 2024-09-16T00:08:19.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615904+0000", 2024-09-16T00:08:19.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:19.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615915+0000", 2024-09-16T00:08:19.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1003e-05 2024-09-16T00:08:19.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615922+0000", 2024-09-16T00:08:19.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9959999999999996e-06 2024-09-16T00:08:19.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615988+0000", 2024-09-16T00:08:19.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5806e-05 2024-09-16T00:08:19.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616009+0000", 2024-09-16T00:08:19.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0729000000000001e-05 2024-09-16T00:08:19.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:19.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028462130000000002 2024-09-16T00:08:19.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618886+0000", 2024-09-16T00:08:19.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.082e-05 2024-09-16T00:08:19.832 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.832 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.832 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.833 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:19.833 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:19.833 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.182591584999997, 2024-09-16T00:08:19.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032610299999999998, 2024-09-16T00:08:19.833 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.833 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.833 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.833 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:19.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:19.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617556+0000", 2024-09-16T00:08:19.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7431e-05 2024-09-16T00:08:19.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617847+0000", 2024-09-16T00:08:19.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029133400000000003 2024-09-16T00:08:19.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617858+0000", 2024-09-16T00:08:19.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0222e-05 2024-09-16T00:08:19.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617864+0000", 2024-09-16T00:08:19.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7619999999999998e-06 2024-09-16T00:08:19.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617909+0000", 2024-09-16T00:08:19.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4747999999999999e-05 2024-09-16T00:08:19.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617927+0000", 2024-09-16T00:08:19.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7785999999999999e-05 2024-09-16T00:08:19.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620734+0000", 2024-09-16T00:08:19.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002807305 2024-09-16T00:08:19.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620770+0000", 2024-09-16T00:08:19.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5441999999999997e-05 2024-09-16T00:08:19.839 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.839 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.839 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.840 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:19.840 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:19.840 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.182170900999999, 2024-09-16T00:08:19.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028641809999999999, 2024-09-16T00:08:19.840 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.840 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.840 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.841 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:19.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:19.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617930+0000", 2024-09-16T00:08:19.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0200000000000002e-07 2024-09-16T00:08:19.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617934+0000", 2024-09-16T00:08:19.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:19.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617943+0000", 2024-09-16T00:08:19.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5820000000000002e-06 2024-09-16T00:08:19.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617949+0000", 2024-09-16T00:08:19.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5069999999999997e-06 2024-09-16T00:08:19.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618094+0000", 2024-09-16T00:08:19.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014515599999999999 2024-09-16T00:08:19.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618111+0000", 2024-09-16T00:08:19.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6705999999999999e-05 2024-09-16T00:08:19.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:08:19.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026603070000000002 2024-09-16T00:08:19.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620793+0000", 2024-09-16T00:08:19.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2214000000000001e-05 2024-09-16T00:08:19.846 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.846 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.846 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.846 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:19.847 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:19.847 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.182140111999999, 2024-09-16T00:08:19.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028624549999999999, 2024-09-16T00:08:19.847 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.847 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.847 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.847 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:19.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:19.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:19.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:19.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:19.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0570000000000001e-06 2024-09-16T00:08:19.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617970+0000", 2024-09-16T00:08:19.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.669e-06 2024-09-16T00:08:19.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617976+0000", 2024-09-16T00:08:19.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1369999999999999e-06 2024-09-16T00:08:19.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618268+0000", 2024-09-16T00:08:19.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029164600000000001 2024-09-16T00:08:19.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618284+0000", 2024-09-16T00:08:19.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5824e-05 2024-09-16T00:08:19.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:19.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025127890000000001 2024-09-16T00:08:19.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620822+0000", 2024-09-16T00:08:19.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6009000000000001e-05 2024-09-16T00:08:19.851 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.852 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.852 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.852 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-16T00:08:19.852 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:19.852 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.182112531000001, 2024-09-16T00:08:19.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028650350000000002, 2024-09-16T00:08:19.852 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.852 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.852 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.852 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:19.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:19.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:19.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7599999999999998e-07 2024-09-16T00:08:19.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617991+0000", 2024-09-16T00:08:19.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1860000000000001e-06 2024-09-16T00:08:19.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617999+0000", 2024-09-16T00:08:19.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3250000000000002e-06 2024-09-16T00:08:19.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618004+0000", 2024-09-16T00:08:19.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7300000000000002e-06 2024-09-16T00:08:19.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618422+0000", 2024-09-16T00:08:19.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00041763199999999998 2024-09-16T00:08:19.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:19.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6112000000000001e-05 2024-09-16T00:08:19.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620824+0000", 2024-09-16T00:08:19.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002385954 2024-09-16T00:08:19.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620853+0000", 2024-09-16T00:08:19.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8719999999999999e-05 2024-09-16T00:08:19.857 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.857 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.857 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.858 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-16T00:08:19.858 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:19.858 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.182086709000004, 2024-09-16T00:08:19.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032703020000000001, 2024-09-16T00:08:19.858 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.858 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.858 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.858 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:19.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:19.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618014+0000", 2024-09-16T00:08:19.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7799999999999997e-07 2024-09-16T00:08:19.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618017+0000", 2024-09-16T00:08:19.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4309999999999998e-06 2024-09-16T00:08:19.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618023+0000", 2024-09-16T00:08:19.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8490000000000002e-06 2024-09-16T00:08:19.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618028+0000", 2024-09-16T00:08:19.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3489999999999999e-06 2024-09-16T00:08:19.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:08:19.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055724099999999999 2024-09-16T00:08:19.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618603+0000", 2024-09-16T00:08:19.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7486000000000002e-05 2024-09-16T00:08:19.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621235+0000", 2024-09-16T00:08:19.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002632087 2024-09-16T00:08:19.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621284+0000", 2024-09-16T00:08:19.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8581000000000001e-05 2024-09-16T00:08:19.862 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.862 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.862 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.863 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:14 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:19.863 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:19.863 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.181149728999998, 2024-09-16T00:08:19.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016807599999999999, 2024-09-16T00:08:19.863 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.863 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.863 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:19.863 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.864 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:19.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.864 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.864 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:19.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.864 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.864 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618951+0000", 2024-09-16T00:08:19.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:19.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618954+0000", 2024-09-16T00:08:19.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.145e-06 2024-09-16T00:08:19.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618963+0000", 2024-09-16T00:08:19.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5720000000000003e-06 2024-09-16T00:08:19.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618969+0000", 2024-09-16T00:08:19.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3319999999999997e-06 2024-09-16T00:08:19.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619008+0000", 2024-09-16T00:08:19.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9096e-05 2024-09-16T00:08:19.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619013+0000", 2024-09-16T00:08:19.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4710000000000001e-06 2024-09-16T00:08:19.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619118+0000", 2024-09-16T00:08:19.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000105809 2024-09-16T00:08:19.867 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.867 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.867 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.868 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-16T00:08:19.868 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:19.868 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.181111622000003, 2024-09-16T00:08:19.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002325307, 2024-09-16T00:08:19.868 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.868 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.868 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.868 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.868 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:19.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:19.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:19.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:19.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618993+0000", 2024-09-16T00:08:19.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.687e-06 2024-09-16T00:08:19.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619000+0000", 2024-09-16T00:08:19.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3590000000000002e-06 2024-09-16T00:08:19.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619005+0000", 2024-09-16T00:08:19.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.95e-06 2024-09-16T00:08:19.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619039+0000", 2024-09-16T00:08:19.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3942999999999997e-05 2024-09-16T00:08:19.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619055+0000", 2024-09-16T00:08:19.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6107e-05 2024-09-16T00:08:19.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.872 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621287+0000", 2024-09-16T00:08:19.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022318070000000001 2024-09-16T00:08:19.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.872 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:19.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7129999999999999e-05 2024-09-16T00:08:19.873 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.873 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.873 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.873 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-16T00:08:19.873 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:19.874 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.181073626, 2024-09-16T00:08:19.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002314328, 2024-09-16T00:08:19.874 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.874 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.874 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.874 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:19.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:19.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.876 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:19.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6399999999999998e-07 2024-09-16T00:08:19.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.876 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.876 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619030+0000", 2024-09-16T00:08:19.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.219e-06 2024-09-16T00:08:19.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.876 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.876 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619038+0000", 2024-09-16T00:08:19.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9899999999999997e-06 2024-09-16T00:08:19.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.877 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.877 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619043+0000", 2024-09-16T00:08:19.877 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5580000000000002e-06 2024-09-16T00:08:19.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619220+0000", 2024-09-16T00:08:19.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000176937 2024-09-16T00:08:19.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619237+0000", 2024-09-16T00:08:19.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7691999999999999e-05 2024-09-16T00:08:19.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621315+0000", 2024-09-16T00:08:19.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020778900000000002 2024-09-16T00:08:19.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621341+0000", 2024-09-16T00:08:19.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5678e-05 2024-09-16T00:08:19.879 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.880 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.880 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.880 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:15 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:19.880 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:19.880 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.180458657000003, 2024-09-16T00:08:19.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143416, 2024-09-16T00:08:19.880 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.880 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.881 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:19.881 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:19.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:19.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619642+0000", 2024-09-16T00:08:19.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5199999999999995e-07 2024-09-16T00:08:19.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619645+0000", 2024-09-16T00:08:19.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.134e-06 2024-09-16T00:08:19.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619655+0000", 2024-09-16T00:08:19.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3330000000000003e-06 2024-09-16T00:08:19.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.884 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619662+0000", 2024-09-16T00:08:19.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0500000000000003e-06 2024-09-16T00:08:19.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.884 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619704+0000", 2024-09-16T00:08:19.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1766e-05 2024-09-16T00:08:19.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619708+0000", 2024-09-16T00:08:19.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3440000000000003e-06 2024-09-16T00:08:19.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619785+0000", 2024-09-16T00:08:19.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6936999999999999e-05 2024-09-16T00:08:19.886 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.886 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.886 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.886 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:31 2.d e22/19 ondisk, result = 0)", 2024-09-16T00:08:19.887 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:19.887 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.180359848000002, 2024-09-16T00:08:19.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101826, 2024-09-16T00:08:19.887 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.887 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.887 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:19.887 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:19.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:19.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619741+0000", 2024-09-16T00:08:19.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3099999999999997e-07 2024-09-16T00:08:19.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:08:19.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1719999999999998e-06 2024-09-16T00:08:19.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619771+0000", 2024-09-16T00:08:19.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4754000000000001e-05 2024-09-16T00:08:19.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619790+0000", 2024-09-16T00:08:19.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9516000000000001e-05 2024-09-16T00:08:19.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619830+0000", 2024-09-16T00:08:19.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9848000000000002e-05 2024-09-16T00:08:19.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619834+0000", 2024-09-16T00:08:19.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1150000000000004e-06 2024-09-16T00:08:19.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619842+0000", 2024-09-16T00:08:19.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.79e-06 2024-09-16T00:08:19.891 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.891 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.891 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.892 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:30 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:19.892 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:19.892 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.176008861, 2024-09-16T00:08:19.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013076400000000001, 2024-09-16T00:08:19.892 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.892 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.892 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:19.892 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:19.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:19.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624092+0000", 2024-09-16T00:08:19.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5799999999999999e-07 2024-09-16T00:08:19.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624096+0000", 2024-09-16T00:08:19.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:19.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624106+0000", 2024-09-16T00:08:19.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7629999999999995e-06 2024-09-16T00:08:19.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624112+0000", 2024-09-16T00:08:19.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.626e-06 2024-09-16T00:08:19.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624149+0000", 2024-09-16T00:08:19.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7094000000000001e-05 2024-09-16T00:08:19.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.896 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624154+0000", 2024-09-16T00:08:19.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7359999999999998e-06 2024-09-16T00:08:19.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.896 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624222+0000", 2024-09-16T00:08:19.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7680000000000003e-05 2024-09-16T00:08:19.896 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.897 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.897 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.897 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:39 2.6 2:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:19.897 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:19.897 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.159822302999999, 2024-09-16T00:08:19.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050072119999999996, 2024-09-16T00:08:19.897 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.897 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.897 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:19.897 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:19.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:19.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640280+0000", 2024-09-16T00:08:19.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5380000000000001e-06 2024-09-16T00:08:19.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640296+0000", 2024-09-16T00:08:19.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5950000000000001e-05 2024-09-16T00:08:19.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640327+0000", 2024-09-16T00:08:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0309999999999999e-05 2024-09-16T00:08:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640340+0000", 2024-09-16T00:08:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3773e-05 2024-09-16T00:08:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640422+0000", 2024-09-16T00:08:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1122e-05 2024-09-16T00:08:19.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640572+0000", 2024-09-16T00:08:19.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015044099999999999 2024-09-16T00:08:19.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:08:19.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640691+0000", 2024-09-16T00:08:19.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000119393 2024-09-16T00:08:19.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:19.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645006+0000", 2024-09-16T00:08:19.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043143779999999998 2024-09-16T00:08:19.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:19.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645169+0000", 2024-09-16T00:08:19.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016323299999999999 2024-09-16T00:08:19.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:19.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645227+0000", 2024-09-16T00:08:19.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8e-05 2024-09-16T00:08:19.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645285+0000", 2024-09-16T00:08:19.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8074e-05 2024-09-16T00:08:19.903 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.903 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.903 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.903 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:39 2.6 e22/19 ondisk, result = 0)", 2024-09-16T00:08:19.903 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:19.904 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.155250191999997, 2024-09-16T00:08:19.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016541200000000001, 2024-09-16T00:08:19.904 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:19.904 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:19.904 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:19.904 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:19.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:19.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:19.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:19.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:19.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:19.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:19.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644852+0000", 2024-09-16T00:08:19.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1670000000000002e-06 2024-09-16T00:08:19.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:19.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644865+0000", 2024-09-16T00:08:19.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2826000000000001e-05 2024-09-16T00:08:19.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:19.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644894+0000", 2024-09-16T00:08:19.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9264999999999998e-05 2024-09-16T00:08:19.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:19.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644911+0000", 2024-09-16T00:08:19.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6827999999999999e-05 2024-09-16T00:08:19.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:19.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644989+0000", 2024-09-16T00:08:19.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7849999999999995e-05 2024-09-16T00:08:19.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:19.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644999+0000", 2024-09-16T00:08:19.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7289999999999995e-06 2024-09-16T00:08:19.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:19.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:19.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:19.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645015+0000", 2024-09-16T00:08:19.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6747000000000001e-05 2024-09-16T00:08:19.908 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.908 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.908 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.908 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:19.908 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:19.908 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:19.909 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T00:08:19.911 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:08:19.912 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:19.910+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:20.012 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:20.011+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:20.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.060+0000 7fa2ecc4c640 1 Processor -- start 2024-09-16T00:08:20.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.061+0000 7fa2ecc4c640 1 -- start start 2024-09-16T00:08:20.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.061+0000 7fa2ecc4c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa2e8154c20 0x7fa2e815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.061+0000 7fa2ecc4c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa2e80ccc90 0x7fa2e80cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.061+0000 7fa2ecc4c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2e80cc4b0 con 0x7fa2e8068060 2024-09-16T00:08:20.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.061+0000 7fa2ecc4c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa2e80c5830 con 0x7fa2e80ccc90 2024-09-16T00:08:20.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.061+0000 7fa2ecc4c640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fa2e815b7c0 con 0x7fa2e8154c20 2024-09-16T00:08:20.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.062+0000 7fa2e6d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa2e80ccc90 0x7fa2e80cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.062+0000 7fa2e5d9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa2e8068060 0x7fa2e80c5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35244/0 (socket says 172.21.15.139:35244) 2024-09-16T00:08:20.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.062+0000 7fa2e5d9b640 1 -- 172.21.15.139:0/86362006 learned_addr learned my addr 172.21.15.139:0/86362006 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:20.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.062+0000 7fa2e659c640 1 --2- 172.21.15.139:0/86362006 >> v2:172.21.15.164:3300/0 conn(0x7fa2e8154c20 0x7fa2e815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.062+0000 7fa2e559a640 1 -- 172.21.15.139:0/86362006 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2108289658 0 0) 0x7fa2e80cc4b0 con 0x7fa2e8068060 2024-09-16T00:08:20.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.062+0000 7fa2e559a640 1 -- 172.21.15.139:0/86362006 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2d0003520 con 0x7fa2e8068060 2024-09-16T00:08:20.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.062+0000 7fa2e6d9d640 1 -- 172.21.15.139:0/86362006 >> v2:172.21.15.164:3300/0 conn(0x7fa2e8154c20 msgr2=0x7fa2e815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:20.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.062+0000 7fa2e6d9d640 1 --2- 172.21.15.139:0/86362006 >> v2:172.21.15.164:3300/0 conn(0x7fa2e8154c20 0x7fa2e815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.062+0000 7fa2e6d9d640 1 -- 172.21.15.139:0/86362006 >> v1:172.21.15.139:6789/0 conn(0x7fa2e8068060 legacy=0x7fa2e80c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.062+0000 7fa2e6d9d640 1 -- 172.21.15.139:0/86362006 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e80cc790 con 0x7fa2e80ccc90 2024-09-16T00:08:20.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.062+0000 7fa2e659c640 1 --2- 172.21.15.139:0/86362006 >> v2:172.21.15.164:3300/0 conn(0x7fa2e8154c20 0x7fa2e815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:08:20.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.063+0000 7fa2e6d9d640 1 --2- 172.21.15.139:0/86362006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e80ccc90 0x7fa2e80cd0a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80d1290 tx=0x7fa2e002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23e482144ec7026 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:20.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.063+0000 7fa2e559a640 1 -- 172.21.15.139:0/86362006 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0007740 con 0x7fa2e80ccc90 2024-09-16T00:08:20.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.063+0000 7fa2e559a640 1 -- 172.21.15.139:0/86362006 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e00078e0 con 0x7fa2e80ccc90 2024-09-16T00:08:20.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.063+0000 7fa2e559a640 1 -- 172.21.15.139:0/86362006 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0007cb0 con 0x7fa2e80ccc90 2024-09-16T00:08:20.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.064+0000 7fa2ecc4c640 1 -- 172.21.15.139:0/86362006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e80ccc90 msgr2=0x7fa2e80cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.064+0000 7fa2ecc4c640 1 --2- 172.21.15.139:0/86362006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e80ccc90 0x7fa2e80cd0a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa2e80d1290 tx=0x7fa2e002a680 comp rx=0 tx=0).stop 2024-09-16T00:08:20.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.064+0000 7fa2ecc4c640 1 -- 172.21.15.139:0/86362006 shutdown_connections 2024-09-16T00:08:20.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.064+0000 7fa2ecc4c640 1 --2- 172.21.15.139:0/86362006 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e80ccc90 0x7fa2e80cd0a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.064+0000 7fa2ecc4c640 1 --2- 172.21.15.139:0/86362006 >> v2:172.21.15.164:3300/0 conn(0x7fa2e8154c20 0x7fa2e815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.064+0000 7fa2ecc4c640 1 -- 172.21.15.139:0/86362006 >> 172.21.15.139:0/86362006 conn(0x7fa2e806ca40 msgr2=0x7fa2e8067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:20.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.064+0000 7fa2ecc4c640 1 -- 172.21.15.139:0/86362006 shutdown_connections 2024-09-16T00:08:20.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.064+0000 7fa2ecc4c640 1 -- 172.21.15.139:0/86362006 wait complete. 2024-09-16T00:08:20.068 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:20.068 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:20.068 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:20.068 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:20.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.066+0000 7fa2ecc4c640 1 Processor -- start 2024-09-16T00:08:20.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.066+0000 7fa2ecc4c640 1 -- start start 2024-09-16T00:08:20.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.067+0000 7fa2ecc4c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2e8068060 0x7fa2e81f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.067+0000 7fa2ecc4c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2e80ccc90 0x7fa2e815bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.067+0000 7fa2ecc4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e8154c20 0x7fa2e8149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.067+0000 7fa2ecc4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e815e0d0 con 0x7fa2e8154c20 2024-09-16T00:08:20.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.067+0000 7fa2ecc4c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e80cc260 con 0x7fa2e80ccc90 2024-09-16T00:08:20.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.067+0000 7fa2ecc4c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e815e3b0 con 0x7fa2e8068060 2024-09-16T00:08:20.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.067+0000 7fa2e6d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e8154c20 0x7fa2e8149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.068+0000 7fa2e6d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e8154c20 0x7fa2e8149be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50726/0 (socket says 172.21.15.139:50726) 2024-09-16T00:08:20.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.068+0000 7fa2e6d9d640 1 -- 172.21.15.139:0/3941507516 learned_addr learned my addr 172.21.15.139:0/3941507516 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:20.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.068+0000 7fa2e5d9b640 1 --2- 172.21.15.139:0/3941507516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2e80ccc90 0x7fa2e815bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.068+0000 7fa2e659c640 1 --2- 172.21.15.139:0/3941507516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2e8068060 0x7fa2e81f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.068+0000 7fa2e6d9d640 1 -- 172.21.15.139:0/3941507516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2e8068060 msgr2=0x7fa2e81f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.068+0000 7fa2e6d9d640 1 --2- 172.21.15.139:0/3941507516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2e8068060 0x7fa2e81f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.068+0000 7fa2e6d9d640 1 -- 172.21.15.139:0/3941507516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2e80ccc90 msgr2=0x7fa2e815bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.068+0000 7fa2e6d9d640 1 --2- 172.21.15.139:0/3941507516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2e80ccc90 0x7fa2e815bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.068+0000 7fa2e6d9d640 1 -- 172.21.15.139:0/3941507516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e81506f0 con 0x7fa2e8154c20 2024-09-16T00:08:20.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.068+0000 7fa2e5d9b640 1 --2- 172.21.15.139:0/3941507516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2e80ccc90 0x7fa2e815bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:20.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.068+0000 7fa2e6d9d640 1 --2- 172.21.15.139:0/3941507516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e8154c20 0x7fa2e8149be0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa2e002ac70 tx=0x7fa2e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:20.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.068+0000 7fa2d6ffd640 1 -- 172.21.15.139:0/3941507516 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0040070 con 0x7fa2e8154c20 2024-09-16T00:08:20.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.069+0000 7fa2d6ffd640 1 -- 172.21.15.139:0/3941507516 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2e0031bc0 con 0x7fa2e8154c20 2024-09-16T00:08:20.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.069+0000 7fa2ecc4c640 1 -- 172.21.15.139:0/3941507516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e81511c0 con 0x7fa2e8154c20 2024-09-16T00:08:20.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.069+0000 7fa2d6ffd640 1 -- 172.21.15.139:0/3941507516 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0009450 con 0x7fa2e8154c20 2024-09-16T00:08:20.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.069+0000 7fa2ecc4c640 1 -- 172.21.15.139:0/3941507516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e81f3f40 con 0x7fa2e8154c20 2024-09-16T00:08:20.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.070+0000 7fa2ecc4c640 1 -- 172.21.15.139:0/3941507516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e815e690 con 0x7fa2e8154c20 2024-09-16T00:08:20.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.070+0000 7fa2d6ffd640 1 -- 172.21.15.139:0/3941507516 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa2e0039040 con 0x7fa2e8154c20 2024-09-16T00:08:20.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.071+0000 7fa2d6ffd640 1 --2- 172.21.15.139:0/3941507516 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa2c806c2b0 0x7fa2c806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.071+0000 7fa2d6ffd640 1 -- 172.21.15.139:0/3941507516 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5915+0+0 (secure 0 0 0) 0x7fa2e00acc00 con 0x7fa2e8154c20 2024-09-16T00:08:20.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.071+0000 7fa2e659c640 1 --2- 172.21.15.139:0/3941507516 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa2c806c2b0 0x7fa2c806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.071+0000 7fa2e659c640 1 --2- 172.21.15.139:0/3941507516 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa2c806c2b0 0x7fa2c806e770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8006fd0 tx=0x7fa2d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:20.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.076+0000 7fa2d6ffd640 1 -- 172.21.15.139:0/3941507516 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa2e00b3080 con 0x7fa2e8154c20 2024-09-16T00:08:20.078 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:08:20.122 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:20.121+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:20.188 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:20.189 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:20.189 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:20.189 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:20.189 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:20.199 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:08:20.215 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:20.214+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:20.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.299+0000 7fa2ecc4c640 1 -- 172.21.15.139:0/3941507516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fa2e80b63d0 con 0x7fa2e8154c20 2024-09-16T00:08:20.316 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:20.315+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:20.355 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:20.355 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:20.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:20.355 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:20.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.355 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:20.355 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:20.356 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.737533192000001, 2024-09-16T00:08:20.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038263220000000001, 2024-09-16T00:08:20.356 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.356 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.356 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:20.356 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:20.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:20.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615465+0000", 2024-09-16T00:08:20.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2769999999999999e-06 2024-09-16T00:08:20.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615471+0000", 2024-09-16T00:08:20.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5729999999999996e-06 2024-09-16T00:08:20.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615483+0000", 2024-09-16T00:08:20.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1838000000000001e-05 2024-09-16T00:08:20.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615492+0000", 2024-09-16T00:08:20.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2410000000000001e-06 2024-09-16T00:08:20.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615596+0000", 2024-09-16T00:08:20.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104097 2024-09-16T00:08:20.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615688+0000", 2024-09-16T00:08:20.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1225999999999997e-05 2024-09-16T00:08:20.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:20.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615804+0000", 2024-09-16T00:08:20.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000116601 2024-09-16T00:08:20.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:20.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618165+0000", 2024-09-16T00:08:20.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023609429999999999 2024-09-16T00:08:20.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:20.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:08:20.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010258509999999999 2024-09-16T00:08:20.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:20.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619243+0000", 2024-09-16T00:08:20.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2327999999999998e-05 2024-09-16T00:08:20.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619287+0000", 2024-09-16T00:08:20.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3346999999999997e-05 2024-09-16T00:08:20.362 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.362 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.362 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.362 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:20.362 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:20.363 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.737478115999998, 2024-09-16T00:08:20.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0054735510000000001, 2024-09-16T00:08:20.363 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.363 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.363 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:20.363 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:20.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.364 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.364 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:20.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.364 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.364 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615516+0000", 2024-09-16T00:08:20.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6899999999999998e-07 2024-09-16T00:08:20.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615519+0000", 2024-09-16T00:08:20.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1760000000000002e-06 2024-09-16T00:08:20.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615526+0000", 2024-09-16T00:08:20.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8399999999999997e-06 2024-09-16T00:08:20.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615531+0000", 2024-09-16T00:08:20.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4450000000000004e-06 2024-09-16T00:08:20.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:20.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054127099999999996 2024-09-16T00:08:20.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616140+0000", 2024-09-16T00:08:20.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7230999999999994e-05 2024-09-16T00:08:20.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:20.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616249+0000", 2024-09-16T00:08:20.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109122 2024-09-16T00:08:20.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:20.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618646+0000", 2024-09-16T00:08:20.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023972279999999999 2024-09-16T00:08:20.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:20.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:08:20.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022396220000000001 2024-09-16T00:08:20.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:20.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620939+0000", 2024-09-16T00:08:20.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2735999999999998e-05 2024-09-16T00:08:20.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620989+0000", 2024-09-16T00:08:20.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0411000000000002e-05 2024-09-16T00:08:20.369 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.369 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.369 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.369 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:20.369 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:20.370 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.737452636999997, 2024-09-16T00:08:20.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055872789999999997, 2024-09-16T00:08:20.370 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.370 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.370 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:20.370 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:20.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:20.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:20.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.01e-07 2024-09-16T00:08:20.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615547+0000", 2024-09-16T00:08:20.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7899999999999996e-06 2024-09-16T00:08:20.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:08:20.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1119999999999998e-06 2024-09-16T00:08:20.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615560+0000", 2024-09-16T00:08:20.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:08:20.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616424+0000", 2024-09-16T00:08:20.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086350800000000003 2024-09-16T00:08:20.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:08:20.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9675999999999999e-05 2024-09-16T00:08:20.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:20.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616621+0000", 2024-09-16T00:08:20.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7345999999999999e-05 2024-09-16T00:08:20.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:20.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618654+0000", 2024-09-16T00:08:20.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020335269999999998 2024-09-16T00:08:20.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:20.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621055+0000", 2024-09-16T00:08:20.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024009729999999998 2024-09-16T00:08:20.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:20.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621100+0000", 2024-09-16T00:08:20.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5402999999999999e-05 2024-09-16T00:08:20.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621128+0000", 2024-09-16T00:08:20.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7736e-05 2024-09-16T00:08:20.377 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.377 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.377 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.377 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:20.377 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:20.377 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.737426986999999, 2024-09-16T00:08:20.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057024509999999999, 2024-09-16T00:08:20.378 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.378 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.378 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:20.378 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:20.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:20.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:20.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6300000000000001e-07 2024-09-16T00:08:20.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615569+0000", 2024-09-16T00:08:20.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3709999999999998e-06 2024-09-16T00:08:20.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615580+0000", 2024-09-16T00:08:20.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0448000000000001e-05 2024-09-16T00:08:20.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615584+0000", 2024-09-16T00:08:20.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6020000000000002e-06 2024-09-16T00:08:20.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616850+0000", 2024-09-16T00:08:20.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012661110000000001 2024-09-16T00:08:20.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616915+0000", 2024-09-16T00:08:20.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4820000000000006e-05 2024-09-16T00:08:20.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:20.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617031+0000", 2024-09-16T00:08:20.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115258 2024-09-16T00:08:20.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:20.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618659+0000", 2024-09-16T00:08:20.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001627975 2024-09-16T00:08:20.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:20.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621179+0000", 2024-09-16T00:08:20.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025203410000000002 2024-09-16T00:08:20.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:20.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621238+0000", 2024-09-16T00:08:20.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9323000000000001e-05 2024-09-16T00:08:20.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621269+0000", 2024-09-16T00:08:20.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0839000000000001e-05 2024-09-16T00:08:20.384 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.384 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.384 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.384 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:20.384 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:20.384 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.737403237999999, 2024-09-16T00:08:20.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058012769999999996, 2024-09-16T00:08:20.385 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.385 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.385 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:20.385 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:20.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:20.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615591+0000", 2024-09-16T00:08:20.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6199999999999999e-07 2024-09-16T00:08:20.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615594+0000", 2024-09-16T00:08:20.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8739999999999999e-06 2024-09-16T00:08:20.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615599+0000", 2024-09-16T00:08:20.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:08:20.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615603+0000", 2024-09-16T00:08:20.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8489999999999999e-06 2024-09-16T00:08:20.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617235+0000", 2024-09-16T00:08:20.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001632037 2024-09-16T00:08:20.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617348+0000", 2024-09-16T00:08:20.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113387 2024-09-16T00:08:20.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:20.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617458+0000", 2024-09-16T00:08:20.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109765 2024-09-16T00:08:20.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:20.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618664+0000", 2024-09-16T00:08:20.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012059950000000001 2024-09-16T00:08:20.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:20.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621311+0000", 2024-09-16T00:08:20.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026468469999999999 2024-09-16T00:08:20.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:20.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621357+0000", 2024-09-16T00:08:20.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6360999999999999e-05 2024-09-16T00:08:20.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621392+0000", 2024-09-16T00:08:20.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4190999999999998e-05 2024-09-16T00:08:20.391 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.391 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.391 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.391 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:20.391 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:20.391 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.737384614, 2024-09-16T00:08:20.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058737279999999999, 2024-09-16T00:08:20.392 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.392 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.392 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:20.392 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:20.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:20.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:20.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3799999999999999e-07 2024-09-16T00:08:20.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615611+0000", 2024-09-16T00:08:20.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-06 2024-09-16T00:08:20.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615626+0000", 2024-09-16T00:08:20.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:08:20.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615634+0000", 2024-09-16T00:08:20.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-06 2024-09-16T00:08:20.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617692+0000", 2024-09-16T00:08:20.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002057718 2024-09-16T00:08:20.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617768+0000", 2024-09-16T00:08:20.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6878999999999994e-05 2024-09-16T00:08:20.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:20.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:20.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019501699999999999 2024-09-16T00:08:20.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:20.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:08:20.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034357200000000002 2024-09-16T00:08:20.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:20.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621429+0000", 2024-09-16T00:08:20.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0012999999999999e-05 2024-09-16T00:08:20.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:20.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621459+0000", 2024-09-16T00:08:20.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9498000000000001e-05 2024-09-16T00:08:20.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621483+0000", 2024-09-16T00:08:20.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3957000000000001e-05 2024-09-16T00:08:20.397 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.397 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.397 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.397 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:20.398 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:20.398 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.737353333999998, 2024-09-16T00:08:20.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059371789999999999, 2024-09-16T00:08:20.398 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.398 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.398 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:20.398 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:20.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:20.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:08:20.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4000000000000002e-07 2024-09-16T00:08:20.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615643+0000", 2024-09-16T00:08:20.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6860000000000002e-06 2024-09-16T00:08:20.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615649+0000", 2024-09-16T00:08:20.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7610000000000004e-06 2024-09-16T00:08:20.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615653+0000", 2024-09-16T00:08:20.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0380000000000001e-06 2024-09-16T00:08:20.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618197+0000", 2024-09-16T00:08:20.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025440079999999999 2024-09-16T00:08:20.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618312+0000", 2024-09-16T00:08:20.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115083 2024-09-16T00:08:20.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:20.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618412+0000", 2024-09-16T00:08:20.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010016799999999999 2024-09-16T00:08:20.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:20.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621518+0000", 2024-09-16T00:08:20.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031058679999999999 2024-09-16T00:08:20.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:20.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621526+0000", 2024-09-16T00:08:20.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2490000000000004e-06 2024-09-16T00:08:20.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:20.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621549+0000", 2024-09-16T00:08:20.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3686999999999999e-05 2024-09-16T00:08:20.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621577+0000", 2024-09-16T00:08:20.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8191000000000002e-05 2024-09-16T00:08:20.403 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.403 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.403 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.404 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:20.404 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:20.404 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.737335668, 2024-09-16T00:08:20.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005989622, 2024-09-16T00:08:20.404 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.404 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.404 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:20.404 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:20.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:20.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:20.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3799999999999998e-07 2024-09-16T00:08:20.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615660+0000", 2024-09-16T00:08:20.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9740000000000001e-06 2024-09-16T00:08:20.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615666+0000", 2024-09-16T00:08:20.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3499999999999996e-06 2024-09-16T00:08:20.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615671+0000", 2024-09-16T00:08:20.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.13e-06 2024-09-16T00:08:20.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618759+0000", 2024-09-16T00:08:20.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030880500000000002 2024-09-16T00:08:20.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618832+0000", 2024-09-16T00:08:20.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3725000000000003e-05 2024-09-16T00:08:20.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:20.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618958+0000", 2024-09-16T00:08:20.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012587899999999999 2024-09-16T00:08:20.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:20.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621583+0000", 2024-09-16T00:08:20.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026246469999999999 2024-09-16T00:08:20.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:20.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621602+0000", 2024-09-16T00:08:20.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9137e-05 2024-09-16T00:08:20.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:20.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621629+0000", 2024-09-16T00:08:20.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6451000000000001e-05 2024-09-16T00:08:20.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621648+0000", 2024-09-16T00:08:20.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8941e-05 2024-09-16T00:08:20.410 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.410 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.410 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.410 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:20.410 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:20.410 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.737318049000002, 2024-09-16T00:08:20.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053141380000000004, 2024-09-16T00:08:20.411 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.411 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.411 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:20.411 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:20.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:20.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:20.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-07 2024-09-16T00:08:20.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615678+0000", 2024-09-16T00:08:20.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:08:20.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615682+0000", 2024-09-16T00:08:20.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6659999999999999e-06 2024-09-16T00:08:20.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615686+0000", 2024-09-16T00:08:20.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6990000000000001e-06 2024-09-16T00:08:20.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615736+0000", 2024-09-16T00:08:20.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9656e-05 2024-09-16T00:08:20.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615864+0000", 2024-09-16T00:08:20.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012836100000000001 2024-09-16T00:08:20.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:20.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615989+0000", 2024-09-16T00:08:20.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000125525 2024-09-16T00:08:20.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:20.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617355+0000", 2024-09-16T00:08:20.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001365982 2024-09-16T00:08:20.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:20.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620897+0000", 2024-09-16T00:08:20.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035419610000000002 2024-09-16T00:08:20.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:20.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620941+0000", 2024-09-16T00:08:20.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3998999999999997e-05 2024-09-16T00:08:20.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620990+0000", 2024-09-16T00:08:20.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8199999999999999e-05 2024-09-16T00:08:20.417 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.417 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.417 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.417 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:20.417 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:20.417 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.737290791, 2024-09-16T00:08:20.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006316449, 2024-09-16T00:08:20.418 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.418 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.418 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:20.418 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:20.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:20.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:20.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3299999999999998e-07 2024-09-16T00:08:20.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615705+0000", 2024-09-16T00:08:20.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.278e-06 2024-09-16T00:08:20.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615711+0000", 2024-09-16T00:08:20.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8119999999999996e-06 2024-09-16T00:08:20.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615716+0000", 2024-09-16T00:08:20.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4909999999999999e-06 2024-09-16T00:08:20.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617551+0000", 2024-09-16T00:08:20.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00183549 2024-09-16T00:08:20.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617627+0000", 2024-09-16T00:08:20.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6163999999999998e-05 2024-09-16T00:08:20.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:20.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617776+0000", 2024-09-16T00:08:20.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014909499999999999 2024-09-16T00:08:20.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:20.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621072+0000", 2024-09-16T00:08:20.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032955150000000002 2024-09-16T00:08:20.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:20.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621957+0000", 2024-09-16T00:08:20.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088553199999999996 2024-09-16T00:08:20.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:20.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621998+0000", 2024-09-16T00:08:20.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0815000000000001e-05 2024-09-16T00:08:20.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:08:20.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0924000000000002e-05 2024-09-16T00:08:20.423 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.423 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.423 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.423 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:6 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:20.423 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:20.423 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.733888823000001, 2024-09-16T00:08:20.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000188111, 2024-09-16T00:08:20.423 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.424 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.424 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:20.424 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:20.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619106+0000", 2024-09-16T00:08:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4399999999999998e-07 2024-09-16T00:08:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619110+0000", 2024-09-16T00:08:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1110000000000001e-06 2024-09-16T00:08:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619120+0000", 2024-09-16T00:08:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7909999999999999e-06 2024-09-16T00:08:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619133+0000", 2024-09-16T00:08:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.359e-05 2024-09-16T00:08:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619179+0000", 2024-09-16T00:08:20.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5459e-05 2024-09-16T00:08:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619186+0000", 2024-09-16T00:08:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0480000000000001e-06 2024-09-16T00:08:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619293+0000", 2024-09-16T00:08:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000107168 2024-09-16T00:08:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:22 2.e e22/19 ondisk, result = 0)", 2024-09-16T00:08:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.732198122, 2024-09-16T00:08:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020058200000000001, 2024-09-16T00:08:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:20.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0109999999999999e-06 2024-09-16T00:08:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620801+0000", 2024-09-16T00:08:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6779999999999999e-06 2024-09-16T00:08:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620814+0000", 2024-09-16T00:08:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3131e-05 2024-09-16T00:08:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620821+0000", 2024-09-16T00:08:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0430000000000002e-06 2024-09-16T00:08:20.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3010999999999997e-05 2024-09-16T00:08:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620891+0000", 2024-09-16T00:08:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1760000000000004e-06 2024-09-16T00:08:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620996+0000", 2024-09-16T00:08:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010453199999999999 2024-09-16T00:08:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:7 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.732190138, 2024-09-16T00:08:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000195725, 2024-09-16T00:08:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.432 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620804+0000", 2024-09-16T00:08:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6099999999999999e-07 2024-09-16T00:08:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620809+0000", 2024-09-16T00:08:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4950000000000002e-06 2024-09-16T00:08:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0553e-05 2024-09-16T00:08:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620826+0000", 2024-09-16T00:08:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5579999999999997e-06 2024-09-16T00:08:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620874+0000", 2024-09-16T00:08:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7648999999999997e-05 2024-09-16T00:08:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620880+0000", 2024-09-16T00:08:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8290000000000002e-06 2024-09-16T00:08:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620999+0000", 2024-09-16T00:08:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011878 2024-09-16T00:08:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.436 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:8 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.732138077999998, 2024-09-16T00:08:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00027991099999999998, 2024-09-16T00:08:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620856+0000", 2024-09-16T00:08:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6800000000000001e-07 2024-09-16T00:08:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620859+0000", 2024-09-16T00:08:20.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6879999999999999e-06 2024-09-16T00:08:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620867+0000", 2024-09-16T00:08:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2819999999999996e-06 2024-09-16T00:08:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620872+0000", 2024-09-16T00:08:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5369999999999997e-06 2024-09-16T00:08:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621043+0000", 2024-09-16T00:08:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170791 2024-09-16T00:08:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621050+0000", 2024-09-16T00:08:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8580000000000002e-06 2024-09-16T00:08:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621135+0000", 2024-09-16T00:08:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5387000000000001e-05 2024-09-16T00:08:20.440 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:9 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.7321095, 2024-09-16T00:08:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00039143799999999997, 2024-09-16T00:08:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:20.441 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:20.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6800000000000002e-07 2024-09-16T00:08:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:08:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0310000000000001e-06 2024-09-16T00:08:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620892+0000", 2024-09-16T00:08:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9390000000000002e-06 2024-09-16T00:08:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620898+0000", 2024-09-16T00:08:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4480000000000002e-06 2024-09-16T00:08:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621167+0000", 2024-09-16T00:08:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026960599999999998 2024-09-16T00:08:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621174+0000", 2024-09-16T00:08:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.567e-06 2024-09-16T00:08:20.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621275+0000", 2024-09-16T00:08:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101579 2024-09-16T00:08:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:10 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.732087051999997, 2024-09-16T00:08:20.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00049416200000000003, 2024-09-16T00:08:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620907+0000", 2024-09-16T00:08:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.42e-07 2024-09-16T00:08:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620908+0000", 2024-09-16T00:08:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4300000000000001e-06 2024-09-16T00:08:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620911+0000", 2024-09-16T00:08:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3890000000000001e-06 2024-09-16T00:08:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620914+0000", 2024-09-16T00:08:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3690000000000001e-06 2024-09-16T00:08:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621297+0000", 2024-09-16T00:08:20.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038319200000000003 2024-09-16T00:08:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621305+0000", 2024-09-16T00:08:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4780000000000008e-06 2024-09-16T00:08:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621401+0000", 2024-09-16T00:08:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5161999999999995e-05 2024-09-16T00:08:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.449 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:11 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.731684303000002, 2024-09-16T00:08:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017943899999999999, 2024-09-16T00:08:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621310+0000", 2024-09-16T00:08:20.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5199999999999996e-07 2024-09-16T00:08:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1740000000000002e-06 2024-09-16T00:08:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621324+0000", 2024-09-16T00:08:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7170000000000003e-06 2024-09-16T00:08:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621331+0000", 2024-09-16T00:08:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2280000000000001e-06 2024-09-16T00:08:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621422+0000", 2024-09-16T00:08:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1052999999999997e-05 2024-09-16T00:08:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621426+0000", 2024-09-16T00:08:20.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.036e-06 2024-09-16T00:08:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621489+0000", 2024-09-16T00:08:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2478999999999996e-05 2024-09-16T00:08:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:12 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:20.454 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.731633447999997, 2024-09-16T00:08:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022422300000000001, 2024-09-16T00:08:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:20.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6699999999999999e-07 2024-09-16T00:08:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621363+0000", 2024-09-16T00:08:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7020000000000001e-06 2024-09-16T00:08:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621371+0000", 2024-09-16T00:08:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7209999999999994e-06 2024-09-16T00:08:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621376+0000", 2024-09-16T00:08:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7940000000000002e-06 2024-09-16T00:08:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621511+0000", 2024-09-16T00:08:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013518100000000001 2024-09-16T00:08:20.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621514+0000", 2024-09-16T00:08:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5599999999999998e-06 2024-09-16T00:08:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621584+0000", 2024-09-16T00:08:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9898000000000001e-05 2024-09-16T00:08:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.458 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:13 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.731607183999998, 2024-09-16T00:08:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026534099999999999, 2024-09-16T00:08:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621387+0000", 2024-09-16T00:08:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4299999999999999e-07 2024-09-16T00:08:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621389+0000", 2024-09-16T00:08:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.593e-06 2024-09-16T00:08:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621395+0000", 2024-09-16T00:08:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.553e-06 2024-09-16T00:08:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:08:20.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8410000000000002e-06 2024-09-16T00:08:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621597+0000", 2024-09-16T00:08:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019794100000000001 2024-09-16T00:08:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621600+0000", 2024-09-16T00:08:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.185e-06 2024-09-16T00:08:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621652+0000", 2024-09-16T00:08:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1984999999999998e-05 2024-09-16T00:08:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:23 2.e e22/19 ondisk, result = 0)", 2024-09-16T00:08:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.731124235000003, 2024-09-16T00:08:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015895699999999999, 2024-09-16T00:08:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:20.463 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:20.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621870+0000", 2024-09-16T00:08:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1600000000000001e-07 2024-09-16T00:08:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621873+0000", 2024-09-16T00:08:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3079999999999999e-06 2024-09-16T00:08:20.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621882+0000", 2024-09-16T00:08:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8429999999999999e-06 2024-09-16T00:08:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621896+0000", 2024-09-16T00:08:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:08:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621944+0000", 2024-09-16T00:08:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8062999999999997e-05 2024-09-16T00:08:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621951+0000", 2024-09-16T00:08:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7689999999999999e-06 2024-09-16T00:08:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622028+0000", 2024-09-16T00:08:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7243999999999994e-05 2024-09-16T00:08:20.467 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:20.468 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:20.468 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:20.469 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:08:20.470 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:20.468+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:20.497 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:08:20.571 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:20.570+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:20.620 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:20.620 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:20.620 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:20.620 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:20.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.622+0000 7f126278d640 1 Processor -- start 2024-09-16T00:08:20.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.622+0000 7f126278d640 1 -- start start 2024-09-16T00:08:20.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.623+0000 7f126278d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f125c15aa80 0x7f125c0b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.623+0000 7f126278d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f125c0b3900 0x7f125c15bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.623+0000 7f126278d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f125c0b3f50 con 0x7f125c0ca350 2024-09-16T00:08:20.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.623+0000 7f126278d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f125c0b4230 con 0x7f125c15aa80 2024-09-16T00:08:20.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.623+0000 7f126278d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f125c0b4510 con 0x7f125c0b3900 2024-09-16T00:08:20.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.623+0000 7f125b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f125c15aa80 0x7f125c0b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.623+0000 7f125b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f125c15aa80 0x7f125c0b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50742/0 (socket says 172.21.15.139:50742) 2024-09-16T00:08:20.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.623+0000 7f125b7fe640 1 -- 172.21.15.139:0/2077251482 learned_addr learned my addr 172.21.15.139:0/2077251482 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:20.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.623+0000 7f125bfff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f125c0ca350 0x7f125c0d0620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:56320/0 (socket says 172.21.15.139:56320) 2024-09-16T00:08:20.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.623+0000 7f1260d2a640 1 --2- 172.21.15.139:0/2077251482 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f125c0b3900 0x7f125c15bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.624+0000 7f125b7fe640 1 -- 172.21.15.139:0/2077251482 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f125c0b3900 msgr2=0x7f125c15bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:20.625 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.624+0000 7f125b7fe640 1 --2- 172.21.15.139:0/2077251482 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f125c0b3900 0x7f125c15bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.624+0000 7f125b7fe640 1 -- 172.21.15.139:0/2077251482 >> v1:172.21.15.164:6789/0 conn(0x7f125c0ca350 legacy=0x7f125c0d0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.624+0000 7f125affd640 1 -- 172.21.15.139:0/2077251482 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 552398040 0 0) 0x7f125c0b3f50 con 0x7f125c0ca350 2024-09-16T00:08:20.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.624+0000 7f125b7fe640 1 -- 172.21.15.139:0/2077251482 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125c0b47f0 con 0x7f125c15aa80 2024-09-16T00:08:20.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.625+0000 7f125b7fe640 1 --2- 172.21.15.139:0/2077251482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f125c15aa80 0x7f125c0b33c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1250009870 tx=0x7f125002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb806a61c9926862 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:20.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.625+0000 7f125affd640 1 -- 172.21.15.139:0/2077251482 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250037070 con 0x7f125c15aa80 2024-09-16T00:08:20.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.625+0000 7f125affd640 1 -- 172.21.15.139:0/2077251482 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f125002ae50 con 0x7f125c15aa80 2024-09-16T00:08:20.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.625+0000 7f125affd640 1 -- 172.21.15.139:0/2077251482 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1250032860 con 0x7f125c15aa80 2024-09-16T00:08:20.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.626+0000 7f126278d640 1 -- 172.21.15.139:0/2077251482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f125c15aa80 msgr2=0x7f125c0b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.626+0000 7f126278d640 1 --2- 172.21.15.139:0/2077251482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f125c15aa80 0x7f125c0b33c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1250009870 tx=0x7f125002a300 comp rx=0 tx=0).stop 2024-09-16T00:08:20.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.626+0000 7f126278d640 1 -- 172.21.15.139:0/2077251482 shutdown_connections 2024-09-16T00:08:20.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.626+0000 7f126278d640 1 --2- 172.21.15.139:0/2077251482 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f125c0b3900 0x7f125c15bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.626+0000 7f126278d640 1 --2- 172.21.15.139:0/2077251482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f125c15aa80 0x7f125c0b33c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.626+0000 7f126278d640 1 -- 172.21.15.139:0/2077251482 >> 172.21.15.139:0/2077251482 conn(0x7f125c06c5f0 msgr2=0x7f125c06ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:20.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.626+0000 7f126278d640 1 -- 172.21.15.139:0/2077251482 shutdown_connections 2024-09-16T00:08:20.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.626+0000 7f126278d640 1 -- 172.21.15.139:0/2077251482 wait complete. 2024-09-16T00:08:20.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.628+0000 7f126278d640 1 Processor -- start 2024-09-16T00:08:20.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.628+0000 7f126278d640 1 -- start start 2024-09-16T00:08:20.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.629+0000 7f126278d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f125c0b3900 0x7f125c150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.629+0000 7f126278d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f125c0ca350 0x7f125c157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.629+0000 7f126278d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f125c15aa80 0x7f125c157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.629+0000 7f126278d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f125c0ca730 con 0x7f125c15aa80 2024-09-16T00:08:20.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.629+0000 7f126278d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f125c0b4db0 con 0x7f125c0b3900 2024-09-16T00:08:20.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.629+0000 7f126278d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f125c0d0d30 con 0x7f125c0ca350 2024-09-16T00:08:20.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.629+0000 7f1260d2a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f125c15aa80 0x7f125c157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.629+0000 7f125bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f125c0b3900 0x7f125c150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.629+0000 7f1260d2a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f125c15aa80 0x7f125c157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50752/0 (socket says 172.21.15.139:50752) 2024-09-16T00:08:20.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.629+0000 7f1260d2a640 1 -- 172.21.15.139:0/2249452635 learned_addr learned my addr 172.21.15.139:0/2249452635 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:20.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.629+0000 7f125b7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f125c0ca350 0x7f125c157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.630+0000 7f125bfff640 1 -- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f125c0ca350 msgr2=0x7f125c157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.630+0000 7f125bfff640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f125c0ca350 0x7f125c157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.630+0000 7f125bfff640 1 -- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f125c15aa80 msgr2=0x7f125c157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.630+0000 7f125bfff640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f125c15aa80 0x7f125c157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.630+0000 7f125bfff640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125c1596f0 con 0x7f125c0b3900 2024-09-16T00:08:20.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.630+0000 7f125b7fe640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f125c0ca350 0x7f125c157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:20.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.630+0000 7f1260d2a640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f125c15aa80 0x7f125c157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:20.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.630+0000 7f125bfff640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f125c0b3900 0x7f125c150aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f124c00c450 tx=0x7f124c00c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:20.633 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:08:20.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.630+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c018070 con 0x7f125c0b3900 2024-09-16T00:08:20.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.630+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f124c0074e0 con 0x7f125c0b3900 2024-09-16T00:08:20.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.630+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125c15a650 con 0x7f125c0b3900 2024-09-16T00:08:20.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.631+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c004a20 con 0x7f125c0b3900 2024-09-16T00:08:20.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.631+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f125c1553f0 con 0x7f125c0b3900 2024-09-16T00:08:20.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.631+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f125c0b59e0 con 0x7f125c0b3900 2024-09-16T00:08:20.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.633+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f124c004bc0 con 0x7f125c0b3900 2024-09-16T00:08:20.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.633+0000 7f1258ff9640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f124806c2e0 0x7f124806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.633+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5915+0+0 (secure 0 0 0) 0x7f124c08a7a0 con 0x7f125c0b3900 2024-09-16T00:08:20.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.633+0000 7f125b7fe640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f124806c2e0 0x7f124806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.634+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v35) ==== 54+0+16 (secure 0 0 0) 0x7f125c0b59e0 con 0x7f125c0b3900 2024-09-16T00:08:20.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.634+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f125c0c3e10 0x7f125c156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.634+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f125c0b33c0 con 0x7f125c0c3e10 2024-09-16T00:08:20.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.634+0000 7f125b7fe640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f124806c2e0 0x7f124806e7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1250034000 tx=0x7f125002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:20.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.634+0000 7f1260d2a640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f125c0c3e10 0x7f125c156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.634+0000 7f1260d2a640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f125c0c3e10 0x7f125c156a50 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:20.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.635+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f125c0b33c0 con 0x7f125c0c3e10 2024-09-16T00:08:20.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.651+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f125c069890 con 0x7f125c0c3e10 2024-09-16T00:08:20.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.654+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f125c069890 con 0x7f125c0c3e10 2024-09-16T00:08:20.655 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:20.655 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:20.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.654+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f125c152520 0x7f125c158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.654+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f125c0680a0 con 0x7f125c152520 2024-09-16T00:08:20.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.654+0000 7f125bfff640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f125c152520 0x7f125c158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.654+0000 7f125bfff640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f125c152520 0x7f125c158640 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:20.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.655+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f125c0680a0 con 0x7f125c152520 2024-09-16T00:08:20.671 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:20.671+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:20.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.671+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f125c068380 con 0x7f125c152520 2024-09-16T00:08:20.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.673+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f125c068380 con 0x7f125c152520 2024-09-16T00:08:20.674 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:20.675 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:20.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.673+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f125c159200 0x7f125c1f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.673+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f125c1f47f0 con 0x7f125c159200 2024-09-16T00:08:20.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.673+0000 7f125b7fe640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f125c159200 0x7f125c1f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.674+0000 7f125b7fe640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f125c159200 0x7f125c1f42b0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:20.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.675+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f125c1f47f0 con 0x7f125c159200 2024-09-16T00:08:20.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.688+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f125c1f4bf0 con 0x7f125c159200 2024-09-16T00:08:20.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.691+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f125c1f4bf0 con 0x7f125c159200 2024-09-16T00:08:20.692 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:20.692 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:20.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.691+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f125c062b40 0x7f125c062fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.691+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f125c0634e0 con 0x7f125c062b40 2024-09-16T00:08:20.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.691+0000 7f1260d2a640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f125c062b40 0x7f125c062fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.691+0000 7f1260d2a640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f125c062b40 0x7f125c062fa0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:20.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.692+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f125c0634e0 con 0x7f125c062b40 2024-09-16T00:08:20.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.705+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f125c152d60 con 0x7f125c062b40 2024-09-16T00:08:20.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.707+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f125c152d60 con 0x7f125c062b40 2024-09-16T00:08:20.708 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:20.709 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:20.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.707+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f125c153480 0x7f125c153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.707+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f125c206740 con 0x7f125c153480 2024-09-16T00:08:20.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.708+0000 7f125bfff640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f125c153480 0x7f125c153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.708+0000 7f125bfff640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f125c153480 0x7f125c153900 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:20.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.709+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f125c206740 con 0x7f125c153480 2024-09-16T00:08:20.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.722+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f125c02c640 con 0x7f125c153480 2024-09-16T00:08:20.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.724+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f125c02c640 con 0x7f125c153480 2024-09-16T00:08:20.725 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:20.726 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:20.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.724+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f125c02cd60 0x7f125c02d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.724+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f125c02d6e0 con 0x7f125c02cd60 2024-09-16T00:08:20.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.725+0000 7f125b7fe640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f125c02cd60 0x7f125c02d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.725+0000 7f125b7fe640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f125c02cd60 0x7f125c02d1a0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:20.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.726+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f125c02d6e0 con 0x7f125c02cd60 2024-09-16T00:08:20.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.741+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f125c02d6e0 con 0x7f125c02cd60 2024-09-16T00:08:20.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.743+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f125c02d6e0 con 0x7f125c02cd60 2024-09-16T00:08:20.744 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:20.745 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:20.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.743+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f125c02d850 0x7f125c209af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.743+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f125c20c8c0 con 0x7f125c02d850 2024-09-16T00:08:20.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.744+0000 7f1260d2a640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f125c02d850 0x7f125c209af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.744+0000 7f1260d2a640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f125c02d850 0x7f125c209af0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:20.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.745+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f125c20c8c0 con 0x7f125c02d850 2024-09-16T00:08:20.754 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:20.754 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:20.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.760+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f125c20a030 con 0x7f125c02d850 2024-09-16T00:08:20.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.762+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f125c20a030 con 0x7f125c02d850 2024-09-16T00:08:20.763 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:20.764 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:20.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.762+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f125c20a6b0 0x7f125c20caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:20.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.762+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f125c20fa20 con 0x7f125c20a6b0 2024-09-16T00:08:20.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.763+0000 7f125bfff640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f125c20a6b0 0x7f125c20caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:20.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.763+0000 7f125bfff640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f125c20a6b0 0x7f125c20caa0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:20.765 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:08:20.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.764+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f125c20fa20 con 0x7f125c20a6b0 2024-09-16T00:08:20.772 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:20.771+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:20.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.778+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f125c0755b0 con 0x7f125c20a6b0 2024-09-16T00:08:20.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f1258ff9640 1 -- 172.21.15.139:0/2249452635 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f125c0755b0 con 0x7f125c20a6b0 2024-09-16T00:08:20.781 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:20.783 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:20.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f125c0c3e10 msgr2=0x7f125c156a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f125c0c3e10 0x7f125c156a50 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f125c152520 msgr2=0x7f125c158640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f125c152520 0x7f125c158640 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f125c159200 msgr2=0x7f125c1f42b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f125c159200 0x7f125c1f42b0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f125c062b40 msgr2=0x7f125c062fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f125c062b40 0x7f125c062fa0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f125c153480 msgr2=0x7f125c153900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f125c153480 0x7f125c153900 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f125c02cd60 msgr2=0x7f125c02d1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f125c02cd60 0x7f125c02d1a0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f125c02d850 msgr2=0x7f125c209af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f125c02d850 0x7f125c209af0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f125c20a6b0 msgr2=0x7f125c20caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.780+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f125c20a6b0 0x7f125c20caa0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.781+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f124806c2e0 msgr2=0x7f124806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.781+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f124806c2e0 0x7f124806e7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1250034000 tx=0x7f125002e000 comp rx=0 tx=0).stop 2024-09-16T00:08:20.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.781+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f125c0b3900 msgr2=0x7f125c150aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.781+0000 7f126278d640 1 --2- 172.21.15.139:0/2249452635 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f125c0b3900 0x7f125c150aa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f124c00c450 tx=0x7f124c00c910 comp rx=0 tx=0).stop 2024-09-16T00:08:20.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.781+0000 7f1260d2a640 1 -- 172.21.15.139:0/2249452635 reap_dead start 2024-09-16T00:08:20.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.781+0000 7f1260d2a640 1 -- 172.21.15.139:0/2249452635 reap_dead start 2024-09-16T00:08:20.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.781+0000 7f1260d2a640 1 -- 172.21.15.139:0/2249452635 reap_dead start 2024-09-16T00:08:20.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.781+0000 7f1260d2a640 1 -- 172.21.15.139:0/2249452635 reap_dead start 2024-09-16T00:08:20.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.781+0000 7f1260d2a640 1 -- 172.21.15.139:0/2249452635 reap_dead start 2024-09-16T00:08:20.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.781+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 shutdown_connections 2024-09-16T00:08:20.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.781+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 >> 172.21.15.139:0/2249452635 conn(0x7f125c06c5f0 msgr2=0x7f125c151900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:20.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.781+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 shutdown_connections 2024-09-16T00:08:20.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.781+0000 7f126278d640 1 -- 172.21.15.139:0/2249452635 wait complete. 2024-09-16T00:08:20.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.808+0000 7fa2d6ffd640 1 -- 172.21.15.139:0/3941507516 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v36) ==== 97+0+0 (secure 0 0 0) 0x7fa2e007ab40 con 0x7fa2e8154c20 2024-09-16T00:08:20.809 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is set 2024-09-16T00:08:20.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.811+0000 7fa2ecc4c640 1 -- 172.21.15.139:0/3941507516 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa2c806c2b0 msgr2=0x7fa2c806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.811+0000 7fa2ecc4c640 1 --2- 172.21.15.139:0/3941507516 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa2c806c2b0 0x7fa2c806e770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8006fd0 tx=0x7fa2d8008040 comp rx=0 tx=0).stop 2024-09-16T00:08:20.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.811+0000 7fa2ecc4c640 1 -- 172.21.15.139:0/3941507516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e8154c20 msgr2=0x7fa2e8149be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:20.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.811+0000 7fa2ecc4c640 1 --2- 172.21.15.139:0/3941507516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e8154c20 0x7fa2e8149be0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa2e002ac70 tx=0x7fa2e0004a40 comp rx=0 tx=0).stop 2024-09-16T00:08:20.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.811+0000 7fa2ecc4c640 1 -- 172.21.15.139:0/3941507516 shutdown_connections 2024-09-16T00:08:20.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.811+0000 7fa2ecc4c640 1 --2- 172.21.15.139:0/3941507516 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa2c806c2b0 0x7fa2c806e770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.811+0000 7fa2ecc4c640 1 --2- 172.21.15.139:0/3941507516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e8154c20 0x7fa2e8149be0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.811+0000 7fa2ecc4c640 1 --2- 172.21.15.139:0/3941507516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa2e80ccc90 0x7fa2e815bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.811+0000 7fa2ecc4c640 1 --2- 172.21.15.139:0/3941507516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa2e8068060 0x7fa2e81f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:20.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.811+0000 7fa2ecc4c640 1 -- 172.21.15.139:0/3941507516 >> 172.21.15.139:0/3941507516 conn(0x7fa2e806ca40 msgr2=0x7fa2e80c6810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:20.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.811+0000 7fa2ecc4c640 1 -- 172.21.15.139:0/3941507516 shutdown_connections 2024-09-16T00:08:20.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:20.811+0000 7fa2ecc4c640 1 -- 172.21.15.139:0/3941507516 wait complete. 2024-09-16T00:08:20.873 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:20.872+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:20.888 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:20.889 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:20.898 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:08:20.973 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:20.973+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:21.015 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:21.015 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:21.025 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:08:21.074 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:21.074+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:21.130 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:21.131 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:21.140 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:08:21.176 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:21.175+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:21.245 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:21.245 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:21.254 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:08:21.277 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:21.276+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:21.359 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:21.359 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:21.369 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:08:21.378 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:21.377+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:21.473 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:21.474 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:21.479 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:21.477+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:21.482 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:08:21.580 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:21.579+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:21.627 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:21.627 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:21.638 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:08:21.680 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:21.680+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:21.781 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:21.780+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:21.782 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:21.782 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:21.792 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:08:21.882 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:21.880+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:21.895 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:21.896 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:21.904 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:08:21.989 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:21.988+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:22.049 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:22.049 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:22.058 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:08:22.099 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:22.098+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:22.185 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:22.184+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:22.203 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:22.203 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:22.212 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:08:22.296 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:22.295+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:22.358 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:22.358 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:22.367 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:08:22.408 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:22.407+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:22.489 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:22.488+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:22.511 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:22.512 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:22.520 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:08:22.590 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:22.589+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:22.637 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:22.638 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:22.647 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:08:22.691 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:22.690+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:22.791 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:22.791+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:22.794 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:08:22.827 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T00:08:22.841 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:22.841 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:22.854 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:08:22.895 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:22.894+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:22.905 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T00:08:22.906 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:08:22.906 INFO:tasks.thrashosds.thrasher:Reweighting osd 6 to 0.13749944564937777 2024-09-16T00:08:22.906 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 0.13749944564937777 2024-09-16T00:08:22.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.913+0000 7ff83dbaf640 1 Processor -- start 2024-09-16T00:08:22.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.913+0000 7ff83dbaf640 1 -- start start 2024-09-16T00:08:22.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.915+0000 7ff83dbaf640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff8380c8e90 0x7ff8380c9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:22.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.915+0000 7ff83dbaf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8380c97b0 con 0x7ff83815a9f0 2024-09-16T00:08:22.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.915+0000 7ff83dbaf640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff838073e50 con 0x7ff8380c7880 2024-09-16T00:08:22.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.915+0000 7ff83dbaf640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7ff8380c5f80 con 0x7ff8380c8e90 2024-09-16T00:08:22.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.915+0000 7ff837fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff83815a9f0 0x7ff8380c6a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:35254/0 (socket says 172.21.15.139:35254) 2024-09-16T00:08:22.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.915+0000 7ff837fff640 1 -- 172.21.15.139:0/1949430583 learned_addr learned my addr 172.21.15.139:0/1949430583 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:22.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.915+0000 7ff8377fe640 1 --2- 172.21.15.139:0/1949430583 >> v2:172.21.15.164:3300/0 conn(0x7ff8380c8e90 0x7ff8380c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:22.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.915+0000 7ff8367fc640 1 -- 172.21.15.139:0/1949430583 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1575729670 0 0) 0x7ff830001490 con 0x7ff83815a9f0 2024-09-16T00:08:22.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.916+0000 7ff8367fc640 1 -- 172.21.15.139:0/1949430583 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff820003520 con 0x7ff83815a9f0 2024-09-16T00:08:22.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.916+0000 7ff8377fe640 1 -- 172.21.15.139:0/1949430583 >> v1:172.21.15.164:6789/0 conn(0x7ff8380c7880 legacy=0x7ff83815a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:22.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.916+0000 7ff8367fc640 1 -- 172.21.15.139:0/1949430583 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 412511357 0 0) 0x7ff820003520 con 0x7ff83815a9f0 2024-09-16T00:08:22.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.916+0000 7ff8377fe640 1 -- 172.21.15.139:0/1949430583 >> v1:172.21.15.139:6789/0 conn(0x7ff83815a9f0 legacy=0x7ff8380c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:22.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.916+0000 7ff8377fe640 1 -- 172.21.15.139:0/1949430583 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff838073890 con 0x7ff8380c8e90 2024-09-16T00:08:22.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.917+0000 7ff8377fe640 1 --2- 172.21.15.139:0/1949430583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8380c8e90 0x7ff8380c9270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff8280097a0 tx=0x7ff82802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a53133656d88827a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:22.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.917+0000 7ff8367fc640 1 -- 172.21.15.139:0/1949430583 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828037070 con 0x7ff8380c8e90 2024-09-16T00:08:22.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.917+0000 7ff8367fc640 1 -- 172.21.15.139:0/1949430583 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff82802ac50 con 0x7ff8380c8e90 2024-09-16T00:08:22.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.917+0000 7ff8367fc640 1 -- 172.21.15.139:0/1949430583 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830001bc0 con 0x7ff8380c8e90 2024-09-16T00:08:22.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.918+0000 7ff83dbaf640 1 -- 172.21.15.139:0/1949430583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8380c8e90 msgr2=0x7ff8380c9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:22.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.918+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/1949430583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8380c8e90 0x7ff8380c9270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff8280097a0 tx=0x7ff82802a1c0 comp rx=0 tx=0).stop 2024-09-16T00:08:22.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.918+0000 7ff83dbaf640 1 -- 172.21.15.139:0/1949430583 shutdown_connections 2024-09-16T00:08:22.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.918+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/1949430583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff8380c8e90 0x7ff8380c9270 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:22.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.918+0000 7ff83dbaf640 1 -- 172.21.15.139:0/1949430583 >> 172.21.15.139:0/1949430583 conn(0x7ff8380c9a80 msgr2=0x7ff8380c9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:22.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.918+0000 7ff83dbaf640 1 -- 172.21.15.139:0/1949430583 shutdown_connections 2024-09-16T00:08:22.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.919+0000 7ff83dbaf640 1 -- 172.21.15.139:0/1949430583 wait complete. 2024-09-16T00:08:22.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.920+0000 7ff83dbaf640 1 Processor -- start 2024-09-16T00:08:22.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.921+0000 7ff83dbaf640 1 -- start start 2024-09-16T00:08:22.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.922+0000 7ff83dbaf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8380c7880 0x7ff8381674e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:22.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.922+0000 7ff83dbaf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8380c8e90 0x7ff838165a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:22.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.922+0000 7ff83dbaf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff83815a9f0 0x7ff8380c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:22.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.922+0000 7ff83dbaf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff8380c59c0 con 0x7ff8380c7880 2024-09-16T00:08:22.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.922+0000 7ff83dbaf640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff8380c6220 con 0x7ff83815a9f0 2024-09-16T00:08:22.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.922+0000 7ff83dbaf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff838074140 con 0x7ff8380c8e90 2024-09-16T00:08:22.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.922+0000 7ff8377fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8380c7880 0x7ff8381674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:22.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.922+0000 7ff8377fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8380c7880 0x7ff8381674e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50756/0 (socket says 172.21.15.139:50756) 2024-09-16T00:08:22.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.922+0000 7ff8377fe640 1 -- 172.21.15.139:0/2691561132 learned_addr learned my addr 172.21.15.139:0/2691561132 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:22.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.923+0000 7ff836ffd640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8380c8e90 0x7ff838165a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:22.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.923+0000 7ff837fff640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff83815a9f0 0x7ff8380c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:22.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.923+0000 7ff836ffd640 1 -- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff83815a9f0 msgr2=0x7ff8380c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:22.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.923+0000 7ff836ffd640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff83815a9f0 0x7ff8380c2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:22.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.923+0000 7ff836ffd640 1 -- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8380c7880 msgr2=0x7ff8381674e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:22.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.923+0000 7ff836ffd640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8380c7880 0x7ff8381674e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:22.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.923+0000 7ff836ffd640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff838157700 con 0x7ff8380c8e90 2024-09-16T00:08:22.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.923+0000 7ff836ffd640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8380c8e90 0x7ff838165a60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff8380c7ca0 tx=0x7ff82c00c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:22.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.924+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c017070 con 0x7ff8380c8e90 2024-09-16T00:08:22.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.924+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff82c0052d0 con 0x7ff8380c8e90 2024-09-16T00:08:22.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.925+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c003830 con 0x7ff8380c8e90 2024-09-16T00:08:22.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.925+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff838167d90 con 0x7ff8380c8e90 2024-09-16T00:08:22.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.925+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83806db70 con 0x7ff8380c8e90 2024-09-16T00:08:22.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.926+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7ff838149850 con 0x7ff8380c8e90 2024-09-16T00:08:22.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.927+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff82c01a070 con 0x7ff8380c8e90 2024-09-16T00:08:22.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.928+0000 7ff81ffff640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff82406c380 0x7ff82406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:22.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.928+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 5915+0+0 (secure 0 0 0) 0x7ff82c08a730 con 0x7ff8380c8e90 2024-09-16T00:08:22.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.928+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v36) ==== 54+0+16 (secure 0 0 0) 0x7ff82c090170 con 0x7ff8380c8e90 2024-09-16T00:08:22.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.928+0000 7ff8377fe640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff82406c380 0x7ff82406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:22.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.928+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7ff8380c0e10 0x7ff8380c2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:22.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.928+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8380ca0b0 con 0x7ff8380c0e10 2024-09-16T00:08:22.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.929+0000 7ff837fff640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7ff8380c0e10 0x7ff8380c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:22.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.929+0000 7ff8377fe640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff82406c380 0x7ff82406e840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff82802a200 tx=0x7ff82802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:22.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.929+0000 7ff837fff640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7ff8380c0e10 0x7ff8380c2520 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:22.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.930+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff8380ca0b0 con 0x7ff8380c0e10 2024-09-16T00:08:22.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.955+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff8380c4370 con 0x7ff8380c0e10 2024-09-16T00:08:22.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.958+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff8380c4370 con 0x7ff8380c0e10 2024-09-16T00:08:22.959 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:22.960 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:22.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.959+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7ff8380c3e60 0x7ff83806c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.959+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7ff8380c2a60 con 0x7ff8380c3e60 2024-09-16T00:08:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.959+0000 7ff8377fe640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7ff8380c3e60 0x7ff83806c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.960+0000 7ff8377fe640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7ff8380c3e60 0x7ff83806c030 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:22.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.961+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff8380c2a60 con 0x7ff8380c3e60 2024-09-16T00:08:22.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.975+0000 7feabdec9640 1 Processor -- start 2024-09-16T00:08:22.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.976+0000 7feabdec9640 1 -- start start 2024-09-16T00:08:22.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.977+0000 7feabdec9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feab815fd90 0x7feab80ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:22.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.977+0000 7feabdec9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feab80ca4f0 con 0x7feab815f9b0 2024-09-16T00:08:22.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.977+0000 7feabdec9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feab80caa90 con 0x7feab80c8b50 2024-09-16T00:08:22.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.977+0000 7feabdec9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7feab806d040 con 0x7feab815fd90 2024-09-16T00:08:22.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.977+0000 7feab6ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feab815fd90 0x7feab80ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:22.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.977+0000 7feab6ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feab815fd90 0x7feab80ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56158/0 (socket says 172.21.15.139:56158) 2024-09-16T00:08:22.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.977+0000 7feab6ffd640 1 -- 172.21.15.139:0/3239226665 learned_addr learned my addr 172.21.15.139:0/3239226665 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:22.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.978+0000 7feab6ffd640 1 -- 172.21.15.139:0/3239226665 >> v1:172.21.15.164:6789/0 conn(0x7feab80c8b50 legacy=0x7feab81698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:22.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.978+0000 7feab6ffd640 1 -- 172.21.15.139:0/3239226665 >> v1:172.21.15.139:6789/0 conn(0x7feab815f9b0 legacy=0x7feab81625c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:22.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.978+0000 7feab6ffd640 1 -- 172.21.15.139:0/3239226665 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab80ca7b0 con 0x7feab815fd90 2024-09-16T00:08:22.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.979+0000 7feab6ffd640 1 --2- 172.21.15.139:0/3239226665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab815fd90 0x7feab80ce1c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feab806d1c0 tx=0x7feaa802a390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ebc5676a2b0f91d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:22.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.979+0000 7feab67fc640 1 -- 172.21.15.139:0/3239226665 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8037070 con 0x7feab815fd90 2024-09-16T00:08:22.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.979+0000 7feab67fc640 1 -- 172.21.15.139:0/3239226665 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa8032640 con 0x7feab815fd90 2024-09-16T00:08:22.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.979+0000 7feab67fc640 1 -- 172.21.15.139:0/3239226665 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa80329f0 con 0x7feab815fd90 2024-09-16T00:08:22.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.980+0000 7feabdec9640 1 -- 172.21.15.139:0/3239226665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab815fd90 msgr2=0x7feab80ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:22.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.980+0000 7feabdec9640 1 --2- 172.21.15.139:0/3239226665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab815fd90 0x7feab80ce1c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7feab806d1c0 tx=0x7feaa802a390 comp rx=0 tx=0).stop 2024-09-16T00:08:22.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.980+0000 7feabdec9640 1 -- 172.21.15.139:0/3239226665 shutdown_connections 2024-09-16T00:08:22.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.980+0000 7feabdec9640 1 --2- 172.21.15.139:0/3239226665 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab815fd90 0x7feab80ce1c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:22.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.980+0000 7feabdec9640 1 -- 172.21.15.139:0/3239226665 >> 172.21.15.139:0/3239226665 conn(0x7feab806c150 msgr2=0x7feab8067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:22.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.980+0000 7feabdec9640 1 -- 172.21.15.139:0/3239226665 shutdown_connections 2024-09-16T00:08:22.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.981+0000 7feabdec9640 1 -- 172.21.15.139:0/3239226665 wait complete. 2024-09-16T00:08:22.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.983+0000 7feabdec9640 1 Processor -- start 2024-09-16T00:08:22.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.983+0000 7feabdec9640 1 -- start start 2024-09-16T00:08:22.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.985+0000 7feabdec9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab80c8b50 0x7feab815e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:22.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.985+0000 7feabdec9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feab815f9b0 0x7feab81e5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:22.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.985+0000 7feabdec9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feab815fd90 0x7feab81e57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:22.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.985+0000 7feabdec9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feab806ec00 con 0x7feab815fd90 2024-09-16T00:08:22.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.985+0000 7feab7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feab815fd90 0x7feab81e57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:22.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.985+0000 7feabdec9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feab806ea80 con 0x7feab80c8b50 2024-09-16T00:08:22.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.985+0000 7feabdec9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7feab80c9460 con 0x7feab815f9b0 2024-09-16T00:08:22.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.985+0000 7feab7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feab815fd90 0x7feab81e57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50772/0 (socket says 172.21.15.139:50772) 2024-09-16T00:08:22.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.985+0000 7feab7fff640 1 -- 172.21.15.139:0/1820307379 learned_addr learned my addr 172.21.15.139:0/1820307379 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:22.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.985+0000 7feab6ffd640 1 --2- 172.21.15.139:0/1820307379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feab815f9b0 0x7feab81e5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:22.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.985+0000 7feab77fe640 1 --2- 172.21.15.139:0/1820307379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab80c8b50 0x7feab815e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:22.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.985+0000 7feab77fe640 1 -- 172.21.15.139:0/1820307379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feab815f9b0 msgr2=0x7feab81e5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:22.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.985+0000 7feab77fe640 1 --2- 172.21.15.139:0/1820307379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feab815f9b0 0x7feab81e5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:22.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.985+0000 7feab77fe640 1 -- 172.21.15.139:0/1820307379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feab815fd90 msgr2=0x7feab81e57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:22.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.985+0000 7feab77fe640 1 --2- 172.21.15.139:0/1820307379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feab815fd90 0x7feab81e57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:22.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.985+0000 7feab77fe640 1 -- 172.21.15.139:0/1820307379 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab81f8090 con 0x7feab80c8b50 2024-09-16T00:08:22.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.986+0000 7feab77fe640 1 --2- 172.21.15.139:0/1820307379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab80c8b50 0x7feab815e360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feaa000c370 tx=0x7feaa000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:22.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.986+0000 7feaaffff640 1 -- 172.21.15.139:0/1820307379 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa00052d0 con 0x7feab80c8b50 2024-09-16T00:08:22.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.986+0000 7feaaffff640 1 -- 172.21.15.139:0/1820307379 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa0005490 con 0x7feab80c8b50 2024-09-16T00:08:22.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.986+0000 7feaaffff640 1 -- 172.21.15.139:0/1820307379 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0007260 con 0x7feab80c8b50 2024-09-16T00:08:22.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.986+0000 7feabdec9640 1 -- 172.21.15.139:0/1820307379 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab81f8370 con 0x7feab80c8b50 2024-09-16T00:08:22.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.986+0000 7feabdec9640 1 -- 172.21.15.139:0/1820307379 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab81f8930 con 0x7feab80c8b50 2024-09-16T00:08:22.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.987+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff8380c3710 con 0x7ff8380c3e60 2024-09-16T00:08:22.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.988+0000 7feabdec9640 1 -- 172.21.15.139:0/1820307379 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feab80b2c10 con 0x7feab80c8b50 2024-09-16T00:08:22.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.988+0000 7feaaffff640 1 -- 172.21.15.139:0/1820307379 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7feaa0007400 con 0x7feab80c8b50 2024-09-16T00:08:22.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.989+0000 7feaaffff640 1 --2- 172.21.15.139:0/1820307379 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fea9806c300 0x7fea9806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:22.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.989+0000 7feaaffff640 1 -- 172.21.15.139:0/1820307379 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5915+0+0 (secure 0 0 0) 0x7feaa008a8a0 con 0x7feab80c8b50 2024-09-16T00:08:22.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.989+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff8380c3710 con 0x7ff8380c3e60 2024-09-16T00:08:22.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.989+0000 7feab6ffd640 1 --2- 172.21.15.139:0/1820307379 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fea9806c300 0x7fea9806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:22.991 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:22.991 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:22.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.990+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7ff8381f8530 0x7ff8381f8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:22.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.990+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7ff8381f8e50 con 0x7ff8381f8530 2024-09-16T00:08:22.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.990+0000 7ff836ffd640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7ff8381f8530 0x7ff8381f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:22.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.990+0000 7feab6ffd640 1 --2- 172.21.15.139:0/1820307379 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fea9806c300 0x7fea9806e7c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feaa8034000 tx=0x7feaa802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:22.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.990+0000 7ff836ffd640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7ff8381f8530 0x7ff8381f8910 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:22.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.991+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff8381f8e50 con 0x7ff8381f8530 2024-09-16T00:08:22.993 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:22.993+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:22.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:22.993+0000 7feaaffff640 1 -- 172.21.15.139:0/1820307379 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feaa00588e0 con 0x7feab80c8b50 2024-09-16T00:08:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.017+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff838062900 con 0x7ff8381f8530 2024-09-16T00:08:23.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.020+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff838062900 con 0x7ff8381f8530 2024-09-16T00:08:23.021 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:23.022 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:23.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.020+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7ff838158780 0x7ff838158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:23.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.020+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7ff83806c930 con 0x7ff838158780 2024-09-16T00:08:23.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.021+0000 7ff837fff640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7ff838158780 0x7ff838158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:23.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.021+0000 7ff837fff640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7ff838158780 0x7ff838158b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:23.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.022+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff83806c930 con 0x7ff838158780 2024-09-16T00:08:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.048+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff8381590c0 con 0x7ff838158780 2024-09-16T00:08:23.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.051+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff8381590c0 con 0x7ff838158780 2024-09-16T00:08:23.052 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:23.052 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:23.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.051+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7ff83802c640 0x7ff83802ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:23.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.051+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7ff83802cf80 con 0x7ff83802c640 2024-09-16T00:08:23.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.052+0000 7ff8377fe640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7ff83802c640 0x7ff83802ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:23.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.052+0000 7ff8377fe640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7ff83802c640 0x7ff83802ca40 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:23.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.053+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff83802cf80 con 0x7ff83802c640 2024-09-16T00:08:23.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.060+0000 7fcb1739f640 1 Processor -- start 2024-09-16T00:08:23.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.060+0000 7fcb1739f640 1 -- start start 2024-09-16T00:08:23.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.062+0000 7fcb1739f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcb1015bfb0 0x7fcb100c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:23.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.062+0000 7fcb1739f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcb10072be0 0x7fcb10072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:23.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.062+0000 7fcb1739f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb1015e9d0 con 0x7fcb10071e00 2024-09-16T00:08:23.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.062+0000 7fcb1739f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcb100c4110 con 0x7fcb10072be0 2024-09-16T00:08:23.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.062+0000 7fcb1739f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fcb1015ec70 con 0x7fcb1015bfb0 2024-09-16T00:08:23.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.062+0000 7fcb1639d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcb1015bfb0 0x7fcb100c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:23.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.062+0000 7fcb15b9c640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fcb10071e00 0x7fcb100c7180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:56348/0 (socket says 172.21.15.139:56348) 2024-09-16T00:08:23.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.062+0000 7fcb15b9c640 1 -- 172.21.15.139:0/3828266267 learned_addr learned my addr 172.21.15.139:0/3828266267 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:23.064 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:23.064 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:23.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:23.065 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:23.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.065 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'41, mlcod=17'39)", 2024-09-16T00:08:23.065 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:23.065 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 59.040172665, 2024-09-16T00:08:23.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001808488, 2024-09-16T00:08:23.065 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.065 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.065 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.065 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:23.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:23.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021032+0000", 2024-09-16T00:08:23.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3e-07 2024-09-16T00:08:23.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021037+0000", 2024-09-16T00:08:23.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0640000000000001e-06 2024-09-16T00:08:23.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021047+0000", 2024-09-16T00:08:23.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.023e-05 2024-09-16T00:08:23.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.068 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.068 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.068 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021053+0000", 2024-09-16T00:08:23.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3160000000000002e-06 2024-09-16T00:08:23.068 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.068 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.068 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.068 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021092+0000", 2024-09-16T00:08:23.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9033999999999999e-05 2024-09-16T00:08:23.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021105+0000", 2024-09-16T00:08:23.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2768e-05 2024-09-16T00:08:23.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022811+0000", 2024-09-16T00:08:23.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001705788 2024-09-16T00:08:23.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022840+0000", 2024-09-16T00:08:23.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8558e-05 2024-09-16T00:08:23.070 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.070 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.070 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.070 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:85 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'55, mlcod=17'50)", 2024-09-16T00:08:23.070 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:23.071 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 59.027143664, 2024-09-16T00:08:23.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017909530000000001, 2024-09-16T00:08:23.071 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.071 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.071 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.071 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:23.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:23.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.073 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.073 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034061+0000", 2024-09-16T00:08:23.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0399999999999996e-07 2024-09-16T00:08:23.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.073 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.073 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034065+0000", 2024-09-16T00:08:23.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8840000000000002e-06 2024-09-16T00:08:23.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034074+0000", 2024-09-16T00:08:23.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:23.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034079+0000", 2024-09-16T00:08:23.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0000000000000004e-06 2024-09-16T00:08:23.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034125+0000", 2024-09-16T00:08:23.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5905000000000002e-05 2024-09-16T00:08:23.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034144+0000", 2024-09-16T00:08:23.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9593e-05 2024-09-16T00:08:23.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035817+0000", 2024-09-16T00:08:23.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016728240000000001 2024-09-16T00:08:23.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.077 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.077 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.077 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035851+0000", 2024-09-16T00:08:23.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3809000000000001e-05 2024-09-16T00:08:23.077 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.077 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.078 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.078 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:112 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'74, mlcod=17'65)", 2024-09-16T00:08:23.078 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:23.078 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.896531801999998, 2024-09-16T00:08:23.079 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018457020000000001, 2024-09-16T00:08:23.079 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.079 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.079 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.079 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.080 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.080 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:23.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.080 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:23.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164673+0000", 2024-09-16T00:08:23.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1200000000000003e-07 2024-09-16T00:08:23.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164710+0000", 2024-09-16T00:08:23.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7756999999999998e-05 2024-09-16T00:08:23.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164719+0000", 2024-09-16T00:08:23.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2849999999999995e-06 2024-09-16T00:08:23.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164723+0000", 2024-09-16T00:08:23.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3220000000000003e-06 2024-09-16T00:08:23.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.084 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.084 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164760+0000", 2024-09-16T00:08:23.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7311999999999998e-05 2024-09-16T00:08:23.084 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.084 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.084 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.084 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164771+0000", 2024-09-16T00:08:23.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0869e-05 2024-09-16T00:08:23.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.085 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166495+0000", 2024-09-16T00:08:23.085 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001723863 2024-09-16T00:08:23.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166518+0000", 2024-09-16T00:08:23.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2682000000000001e-05 2024-09-16T00:08:23.086 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.086 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.086 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.086 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'66)", 2024-09-16T00:08:23.087 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:23.087 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.895027823, 2024-09-16T00:08:23.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032925509999999999, 2024-09-16T00:08:23.087 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.087 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.087 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.087 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:23.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:23.089 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.089 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166177+0000", 2024-09-16T00:08:23.089 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:23.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166216+0000", 2024-09-16T00:08:23.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9008000000000003e-05 2024-09-16T00:08:23.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.090 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.090 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166226+0000", 2024-09-16T00:08:23.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0081999999999999e-05 2024-09-16T00:08:23.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.091 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.091 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166230+0000", 2024-09-16T00:08:23.091 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5079999999999999e-06 2024-09-16T00:08:23.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.091 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.092 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166259+0000", 2024-09-16T00:08:23.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8928000000000002e-05 2024-09-16T00:08:23.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.092 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.092 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166271+0000", 2024-09-16T00:08:23.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1531e-05 2024-09-16T00:08:23.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.093 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169449+0000", 2024-09-16T00:08:23.093 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003178143 2024-09-16T00:08:23.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.093 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169468+0000", 2024-09-16T00:08:23.094 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9700000000000001e-05 2024-09-16T00:08:23.094 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.094 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.094 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.094 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.094 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'66)", 2024-09-16T00:08:23.094 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:23.095 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.894566363999999, 2024-09-16T00:08:23.095 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003179374, 2024-09-16T00:08:23.095 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.095 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.095 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.095 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.095 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.095 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.095 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:23.096 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.096 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.096 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.096 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.096 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:23.096 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.096 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.097 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.097 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.097 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166638+0000", 2024-09-16T00:08:23.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2300000000000002e-07 2024-09-16T00:08:23.097 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.097 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.097 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.097 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166641+0000", 2024-09-16T00:08:23.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0079999999999998e-06 2024-09-16T00:08:23.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166646+0000", 2024-09-16T00:08:23.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4970000000000001e-06 2024-09-16T00:08:23.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.099 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166650+0000", 2024-09-16T00:08:23.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5729999999999999e-06 2024-09-16T00:08:23.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.099 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166682+0000", 2024-09-16T00:08:23.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2218999999999998e-05 2024-09-16T00:08:23.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.100 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166694+0000", 2024-09-16T00:08:23.100 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1902999999999999e-05 2024-09-16T00:08:23.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169784+0000", 2024-09-16T00:08:23.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003089545 2024-09-16T00:08:23.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169817+0000", 2024-09-16T00:08:23.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3206e-05 2024-09-16T00:08:23.102 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.102 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.102 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.102 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.102 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.102 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:117 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'66)", 2024-09-16T00:08:23.102 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:23.102 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.894122304, 2024-09-16T00:08:23.103 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00275326, 2024-09-16T00:08:23.103 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.103 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.103 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.103 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.103 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.103 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.104 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:23.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.104 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.104 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.104 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:23.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.105 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167082+0000", 2024-09-16T00:08:23.105 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:08:23.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.105 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.106 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167086+0000", 2024-09-16T00:08:23.106 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1559999999999997e-06 2024-09-16T00:08:23.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.106 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.107 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167096+0000", 2024-09-16T00:08:23.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8390000000000002e-06 2024-09-16T00:08:23.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.107 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.107 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.107 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167101+0000", 2024-09-16T00:08:23.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:08:23.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.107 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.108 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.108 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167130+0000", 2024-09-16T00:08:23.108 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9649e-05 2024-09-16T00:08:23.108 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.108 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.108 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.109 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167142+0000", 2024-09-16T00:08:23.109 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1435e-05 2024-09-16T00:08:23.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.109 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.109 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.109 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169818+0000", 2024-09-16T00:08:23.109 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002676056 2024-09-16T00:08:23.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.110 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.110 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169835+0000", 2024-09-16T00:08:23.110 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6980999999999999e-05 2024-09-16T00:08:23.110 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.110 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.111 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.111 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.111 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:130 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'88, mlcod=17'85)", 2024-09-16T00:08:23.111 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:23.111 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.451322974999997, 2024-09-16T00:08:23.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002880456, 2024-09-16T00:08:23.111 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.112 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.112 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.112 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:23.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.113 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:23.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.114 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:23.114 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5699999999999998e-07 2024-09-16T00:08:23.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.114 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609885+0000", 2024-09-16T00:08:23.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.608e-06 2024-09-16T00:08:23.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609891+0000", 2024-09-16T00:08:23.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:23.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609895+0000", 2024-09-16T00:08:23.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.54e-06 2024-09-16T00:08:23.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609931+0000", 2024-09-16T00:08:23.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5893999999999999e-05 2024-09-16T00:08:23.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609945+0000", 2024-09-16T00:08:23.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4259e-05 2024-09-16T00:08:23.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612721+0000", 2024-09-16T00:08:23.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027757849999999998 2024-09-16T00:08:23.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612761+0000", 2024-09-16T00:08:23.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0638e-05 2024-09-16T00:08:23.118 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.118 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.119 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.119 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-16T00:08:23.119 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:23.119 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.450939001999998, 2024-09-16T00:08:23.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028016529999999999, 2024-09-16T00:08:23.119 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.120 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.120 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.120 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:23.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:23.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:23.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.63e-07 2024-09-16T00:08:23.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.123 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610344+0000", 2024-09-16T00:08:23.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8996000000000001e-05 2024-09-16T00:08:23.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.123 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610352+0000", 2024-09-16T00:08:23.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:23.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.124 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.124 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610356+0000", 2024-09-16T00:08:23.124 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0790000000000002e-06 2024-09-16T00:08:23.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.124 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.124 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610388+0000", 2024-09-16T00:08:23.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1541000000000002e-05 2024-09-16T00:08:23.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.125 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610400+0000", 2024-09-16T00:08:23.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2075e-05 2024-09-16T00:08:23.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613045+0000", 2024-09-16T00:08:23.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002645213 2024-09-16T00:08:23.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613066+0000", 2024-09-16T00:08:23.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1353e-05 2024-09-16T00:08:23.127 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.127 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.127 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.127 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-16T00:08:23.127 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:23.127 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.447679502, 2024-09-16T00:08:23.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027254850000000001, 2024-09-16T00:08:23.128 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.128 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.128 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.128 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.129 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:23.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.129 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.129 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:23.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613525+0000", 2024-09-16T00:08:23.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2800000000000002e-07 2024-09-16T00:08:23.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613528+0000", 2024-09-16T00:08:23.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9730000000000002e-06 2024-09-16T00:08:23.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613534+0000", 2024-09-16T00:08:23.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4339999999999997e-06 2024-09-16T00:08:23.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613538+0000", 2024-09-16T00:08:23.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5149999999999998e-06 2024-09-16T00:08:23.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613566+0000", 2024-09-16T00:08:23.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8104000000000001e-05 2024-09-16T00:08:23.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.133 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613578+0000", 2024-09-16T00:08:23.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2404e-05 2024-09-16T00:08:23.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616217+0000", 2024-09-16T00:08:23.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026389830000000001 2024-09-16T00:08:23.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616250+0000", 2024-09-16T00:08:23.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2644000000000001e-05 2024-09-16T00:08:23.135 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.135 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.135 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.135 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:134 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'92, mlcod=18'87)", 2024-09-16T00:08:23.135 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:23.136 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.447141363, 2024-09-16T00:08:23.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025322399999999998, 2024-09-16T00:08:23.136 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.136 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.136 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.136 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.136 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:23.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:23.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614063+0000", 2024-09-16T00:08:23.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8700000000000005e-07 2024-09-16T00:08:23.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614096+0000", 2024-09-16T00:08:23.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3136000000000003e-05 2024-09-16T00:08:23.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614106+0000", 2024-09-16T00:08:23.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9170000000000001e-06 2024-09-16T00:08:23.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.140 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.140 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614111+0000", 2024-09-16T00:08:23.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4390000000000002e-06 2024-09-16T00:08:23.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.140 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.140 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.140 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614141+0000", 2024-09-16T00:08:23.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.04e-05 2024-09-16T00:08:23.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614152+0000", 2024-09-16T00:08:23.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0987e-05 2024-09-16T00:08:23.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616573+0000", 2024-09-16T00:08:23.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002421351 2024-09-16T00:08:23.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616595+0000", 2024-09-16T00:08:23.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1322999999999999e-05 2024-09-16T00:08:23.142 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.142 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.143 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.143 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:135 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'93, mlcod=18'87)", 2024-09-16T00:08:23.143 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:23.143 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.446621325000002, 2024-09-16T00:08:23.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020304659999999999, 2024-09-16T00:08:23.143 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.143 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.144 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.144 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:23.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.145 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.145 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:23.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.145 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.145 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614583+0000", 2024-09-16T00:08:23.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:23.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614587+0000", 2024-09-16T00:08:23.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3100000000000002e-06 2024-09-16T00:08:23.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.147 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.147 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614597+0000", 2024-09-16T00:08:23.147 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7450000000000007e-06 2024-09-16T00:08:23.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.147 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614602+0000", 2024-09-16T00:08:23.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6099999999999999e-06 2024-09-16T00:08:23.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614641+0000", 2024-09-16T00:08:23.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9767000000000001e-05 2024-09-16T00:08:23.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.149 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614658+0000", 2024-09-16T00:08:23.149 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6205000000000002e-05 2024-09-16T00:08:23.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.150 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616596+0000", 2024-09-16T00:08:23.150 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019386659999999999 2024-09-16T00:08:23.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.150 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.150 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.150 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616613+0000", 2024-09-16T00:08:23.150 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6555999999999999e-05 2024-09-16T00:08:23.151 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.151 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.151 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.151 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.151 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:139 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'97, mlcod=18'87)", 2024-09-16T00:08:23.151 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:23.151 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.444860654000003, 2024-09-16T00:08:23.152 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030908139999999999, 2024-09-16T00:08:23.152 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.152 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.152 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.152 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.152 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.152 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.153 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:23.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.153 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.153 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.153 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:23.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.154 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.154 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.154 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616344+0000", 2024-09-16T00:08:23.154 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3000000000000001e-07 2024-09-16T00:08:23.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616347+0000", 2024-09-16T00:08:23.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7740000000000002e-06 2024-09-16T00:08:23.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.156 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.156 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616355+0000", 2024-09-16T00:08:23.156 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4699999999999996e-06 2024-09-16T00:08:23.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.156 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616358+0000", 2024-09-16T00:08:23.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5899999999999999e-06 2024-09-16T00:08:23.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616393+0000", 2024-09-16T00:08:23.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4529000000000001e-05 2024-09-16T00:08:23.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.158 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.158 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616404+0000", 2024-09-16T00:08:23.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0726e-05 2024-09-16T00:08:23.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.159 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.159 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619411+0000", 2024-09-16T00:08:23.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030076249999999999 2024-09-16T00:08:23.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.159 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.159 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.159 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619434+0000", 2024-09-16T00:08:23.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-05 2024-09-16T00:08:23.160 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.160 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.160 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.160 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:140 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'98, mlcod=18'89)", 2024-09-16T00:08:23.160 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:23.160 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.444219261000001, 2024-09-16T00:08:23.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027276150000000001, 2024-09-16T00:08:23.161 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.161 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.161 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.161 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.161 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.161 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.161 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:23.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:23.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616985+0000", 2024-09-16T00:08:23.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4099999999999999e-07 2024-09-16T00:08:23.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617055+0000", 2024-09-16T00:08:23.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9923000000000001e-05 2024-09-16T00:08:23.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.164 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617063+0000", 2024-09-16T00:08:23.164 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:23.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.164 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.164 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.164 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617070+0000", 2024-09-16T00:08:23.164 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9090000000000003e-06 2024-09-16T00:08:23.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.165 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.165 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617102+0000", 2024-09-16T00:08:23.165 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2620999999999998e-05 2024-09-16T00:08:23.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.165 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.165 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617114+0000", 2024-09-16T00:08:23.165 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1228e-05 2024-09-16T00:08:23.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619690+0000", 2024-09-16T00:08:23.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025763890000000001 2024-09-16T00:08:23.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619712+0000", 2024-09-16T00:08:23.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2171000000000002e-05 2024-09-16T00:08:23.167 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.167 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.167 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.168 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:23.168 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:23.168 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 44.445390160999999, 2024-09-16T00:08:23.168 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027355320000000002, 2024-09-16T00:08:23.168 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.168 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.168 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.168 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.168 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.168 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:23.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.169 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:23.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.169 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615815+0000", 2024-09-16T00:08:23.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1769999999999999e-06 2024-09-16T00:08:23.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.170 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.170 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615825+0000", 2024-09-16T00:08:23.170 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9000000000000001e-06 2024-09-16T00:08:23.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.170 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.170 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.170 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615840+0000", 2024-09-16T00:08:23.170 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4854e-05 2024-09-16T00:08:23.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.170 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615848+0000", 2024-09-16T00:08:23.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0940000000000008e-06 2024-09-16T00:08:23.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615926+0000", 2024-09-16T00:08:23.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8176999999999994e-05 2024-09-16T00:08:23.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615945+0000", 2024-09-16T00:08:23.172 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9663000000000001e-05 2024-09-16T00:08:23.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.172 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.172 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618509+0000", 2024-09-16T00:08:23.172 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002563802 2024-09-16T00:08:23.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.172 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.172 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618549+0000", 2024-09-16T00:08:23.172 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9864999999999999e-05 2024-09-16T00:08:23.172 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.173 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.173 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.173 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.173 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:23.173 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:23.173 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 44.445301020999999, 2024-09-16T00:08:23.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061593860000000002, 2024-09-16T00:08:23.173 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.173 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.173 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.174 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.174 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.174 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.174 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:23.174 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.174 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.174 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.174 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.174 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:23.174 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615907+0000", 2024-09-16T00:08:23.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9389999999999999e-06 2024-09-16T00:08:23.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615920+0000", 2024-09-16T00:08:23.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2989999999999999e-05 2024-09-16T00:08:23.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615931+0000", 2024-09-16T00:08:23.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.164e-05 2024-09-16T00:08:23.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615939+0000", 2024-09-16T00:08:23.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8029999999999997e-06 2024-09-16T00:08:23.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616310+0000", 2024-09-16T00:08:23.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00037111400000000003 2024-09-16T00:08:23.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616406+0000", 2024-09-16T00:08:23.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5372999999999996e-05 2024-09-16T00:08:23.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:23.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616514+0000", 2024-09-16T00:08:23.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108564 2024-09-16T00:08:23.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:23.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618158+0000", 2024-09-16T00:08:23.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016442449999999999 2024-09-16T00:08:23.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:23.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621972+0000", 2024-09-16T00:08:23.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038132299999999999 2024-09-16T00:08:23.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622018+0000", 2024-09-16T00:08:23.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6239000000000002e-05 2024-09-16T00:08:23.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.179 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622062+0000", 2024-09-16T00:08:23.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4249000000000003e-05 2024-09-16T00:08:23.179 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.179 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.179 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.180 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:23.180 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:23.180 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 44.445247553000002, 2024-09-16T00:08:23.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006258914, 2024-09-16T00:08:23.180 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.180 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.181 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.181 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:23.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:23.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615957+0000", 2024-09-16T00:08:23.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1399999999999997e-07 2024-09-16T00:08:23.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615960+0000", 2024-09-16T00:08:23.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0860000000000002e-06 2024-09-16T00:08:23.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615967+0000", 2024-09-16T00:08:23.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9059999999999996e-06 2024-09-16T00:08:23.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615972+0000", 2024-09-16T00:08:23.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5679999999999999e-06 2024-09-16T00:08:23.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616717+0000", 2024-09-16T00:08:23.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074438 2024-09-16T00:08:23.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.186 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.186 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616781+0000", 2024-09-16T00:08:23.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4808999999999996e-05 2024-09-16T00:08:23.186 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.186 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:23.186 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616885+0000", 2024-09-16T00:08:23.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010404900000000001 2024-09-16T00:08:23.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:23.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:23.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012773870000000001 2024-09-16T00:08:23.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:23.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622116+0000", 2024-09-16T00:08:23.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039533470000000003 2024-09-16T00:08:23.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622173+0000", 2024-09-16T00:08:23.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6347000000000001e-05 2024-09-16T00:08:23.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622215+0000", 2024-09-16T00:08:23.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2521000000000002e-05 2024-09-16T00:08:23.190 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.190 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.190 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.190 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:23.190 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:23.191 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 44.445223816000002, 2024-09-16T00:08:23.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041725529999999999, 2024-09-16T00:08:23.191 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.191 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.191 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.191 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:23.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.192 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:23.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:23.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8799999999999998e-07 2024-09-16T00:08:23.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615990+0000", 2024-09-16T00:08:23.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2150000000000007e-06 2024-09-16T00:08:23.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:08:23.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7159999999999997e-06 2024-09-16T00:08:23.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616001+0000", 2024-09-16T00:08:23.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9339999999999997e-06 2024-09-16T00:08:23.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616293+0000", 2024-09-16T00:08:23.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029210899999999998 2024-09-16T00:08:23.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616401+0000", 2024-09-16T00:08:23.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108087 2024-09-16T00:08:23.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:23.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616561+0000", 2024-09-16T00:08:23.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015940600000000001 2024-09-16T00:08:23.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:23.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618146+0000", 2024-09-16T00:08:23.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015857040000000001 2024-09-16T00:08:23.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:23.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620091+0000", 2024-09-16T00:08:23.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001944837 2024-09-16T00:08:23.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620126+0000", 2024-09-16T00:08:23.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4242999999999997e-05 2024-09-16T00:08:23.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620152+0000", 2024-09-16T00:08:23.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6914e-05 2024-09-16T00:08:23.200 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.200 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.200 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.200 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:23.200 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:23.201 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 44.445185590999998, 2024-09-16T00:08:23.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00630628, 2024-09-16T00:08:23.201 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.201 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.201 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.201 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:23.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:23.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:08:23.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5999999999999999e-07 2024-09-16T00:08:23.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616022+0000", 2024-09-16T00:08:23.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0149999999999999e-06 2024-09-16T00:08:23.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616031+0000", 2024-09-16T00:08:23.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9770000000000006e-06 2024-09-16T00:08:23.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616036+0000", 2024-09-16T00:08:23.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2639999999999999e-06 2024-09-16T00:08:23.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.205 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617057+0000", 2024-09-16T00:08:23.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010212999999999999 2024-09-16T00:08:23.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617146+0000", 2024-09-16T00:08:23.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9234000000000007e-05 2024-09-16T00:08:23.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:23.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617232+0000", 2024-09-16T00:08:23.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5218999999999998e-05 2024-09-16T00:08:23.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:23.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619107+0000", 2024-09-16T00:08:23.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001875016 2024-09-16T00:08:23.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:23.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622259+0000", 2024-09-16T00:08:23.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031527930000000001 2024-09-16T00:08:23.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622290+0000", 2024-09-16T00:08:23.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1022999999999998e-05 2024-09-16T00:08:23.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622324+0000", 2024-09-16T00:08:23.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3979000000000001e-05 2024-09-16T00:08:23.209 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.209 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.209 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.209 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:23.210 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:23.210 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 44.445160289, 2024-09-16T00:08:23.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042412739999999997, 2024-09-16T00:08:23.210 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.210 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.210 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.210 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:23.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:23.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.212 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.212 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616044+0000", 2024-09-16T00:08:23.212 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:23.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616052+0000", 2024-09-16T00:08:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.439e-06 2024-09-16T00:08:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616059+0000", 2024-09-16T00:08:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1389999999999998e-06 2024-09-16T00:08:23.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616065+0000", 2024-09-16T00:08:23.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:08:23.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616823+0000", 2024-09-16T00:08:23.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075781199999999996 2024-09-16T00:08:23.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616918+0000", 2024-09-16T00:08:23.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5174999999999996e-05 2024-09-16T00:08:23.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:23.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617019+0000", 2024-09-16T00:08:23.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101161 2024-09-16T00:08:23.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:23.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618503+0000", 2024-09-16T00:08:23.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014836750000000001 2024-09-16T00:08:23.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:23.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620201+0000", 2024-09-16T00:08:23.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00169805 2024-09-16T00:08:23.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.218 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620255+0000", 2024-09-16T00:08:23.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4499000000000001e-05 2024-09-16T00:08:23.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.219 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.219 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620285+0000", 2024-09-16T00:08:23.219 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9349e-05 2024-09-16T00:08:23.219 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.219 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.219 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.220 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:23.220 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:23.220 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 44.445131298, 2024-09-16T00:08:23.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063682799999999996, 2024-09-16T00:08:23.220 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.220 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.221 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:23.221 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:23.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:23.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:23.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9900000000000001e-07 2024-09-16T00:08:23.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616079+0000", 2024-09-16T00:08:23.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9009999999999999e-06 2024-09-16T00:08:23.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616086+0000", 2024-09-16T00:08:23.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.776e-06 2024-09-16T00:08:23.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616091+0000", 2024-09-16T00:08:23.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5600000000000001e-06 2024-09-16T00:08:23.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618339+0000", 2024-09-16T00:08:23.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022476179999999998 2024-09-16T00:08:23.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:23.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9025999999999997e-05 2024-09-16T00:08:23.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622441+0000", 2024-09-16T00:08:23.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040029999999999996 2024-09-16T00:08:23.228 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.228 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.228 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.228 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.228 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.228 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:23.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.062+0000 7fcb16b9e640 1 --2- 172.21.15.139:0/3828266267 >> v2:172.21.15.139:3300/0 conn(0x7fcb10072be0 0x7fcb10072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:23.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.062+0000 7fcb16b9e640 1 -- 172.21.15.139:0/3828266267 >> v2:172.21.15.164:3300/0 conn(0x7fcb1015bfb0 msgr2=0x7fcb100c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:23.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.062+0000 7fcb16b9e640 1 --2- 172.21.15.139:0/3828266267 >> v2:172.21.15.164:3300/0 conn(0x7fcb1015bfb0 0x7fcb100c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.063+0000 7fcb16b9e640 1 -- 172.21.15.139:0/3828266267 >> v1:172.21.15.164:6789/0 conn(0x7fcb10071e00 legacy=0x7fcb100c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:23.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.063+0000 7fcb16b9e640 1 -- 172.21.15.139:0/3828266267 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1015ef10 con 0x7fcb10072be0 2024-09-16T00:08:23.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.064+0000 7fcb16b9e640 1 --2- 172.21.15.139:0/3828266267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10072be0 0x7fcb10072ff0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcb10073e00 tx=0x7fcb0002a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3ad34859228d69 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:23.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.064+0000 7fcb1539b640 1 -- 172.21.15.139:0/3828266267 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00031640 con 0x7fcb10072be0 2024-09-16T00:08:23.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.064+0000 7fcb1539b640 1 -- 172.21.15.139:0/3828266267 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb00036070 con 0x7fcb10072be0 2024-09-16T00:08:23.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.064+0000 7fcb1539b640 1 -- 172.21.15.139:0/3828266267 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00031a00 con 0x7fcb10072be0 2024-09-16T00:08:23.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.065+0000 7fcb1739f640 1 -- 172.21.15.139:0/3828266267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10072be0 msgr2=0x7fcb10072ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:23.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.065+0000 7fcb1739f640 1 --2- 172.21.15.139:0/3828266267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10072be0 0x7fcb10072ff0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcb10073e00 tx=0x7fcb0002a8f0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.065+0000 7fcb1739f640 1 -- 172.21.15.139:0/3828266267 shutdown_connections 2024-09-16T00:08:23.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.065+0000 7fcb1739f640 1 --2- 172.21.15.139:0/3828266267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10072be0 0x7fcb10072ff0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.065+0000 7fcb1739f640 1 --2- 172.21.15.139:0/3828266267 >> v2:172.21.15.164:3300/0 conn(0x7fcb1015bfb0 0x7fcb100c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.065+0000 7fcb1739f640 1 -- 172.21.15.139:0/3828266267 >> 172.21.15.139:0/3828266267 conn(0x7fcb100c91d0 msgr2=0x7fcb100c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.065+0000 7fcb1739f640 1 -- 172.21.15.139:0/3828266267 shutdown_connections 2024-09-16T00:08:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.066+0000 7fcb1739f640 1 -- 172.21.15.139:0/3828266267 wait complete. 2024-09-16T00:08:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.068+0000 7fcb1739f640 1 Processor -- start 2024-09-16T00:08:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.068+0000 7fcb1739f640 1 -- start start 2024-09-16T00:08:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.069+0000 7fcb1739f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10071e00 0x7fcb10151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.069+0000 7fcb1739f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb10072be0 0x7fcb1015a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.069+0000 7fcb1739f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb1015bfb0 0x7fcb1015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.069+0000 7fcb1739f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1015f750 con 0x7fcb10071e00 2024-09-16T00:08:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.069+0000 7fcb1739f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1015f470 con 0x7fcb1015bfb0 2024-09-16T00:08:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.069+0000 7fcb1739f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcb10073530 con 0x7fcb10072be0 2024-09-16T00:08:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.069+0000 7fcb1639d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10071e00 0x7fcb10151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:23.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.069+0000 7fcb1639d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10071e00 0x7fcb10151f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50788/0 (socket says 172.21.15.139:50788) 2024-09-16T00:08:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.069+0000 7fcb1639d640 1 -- 172.21.15.139:0/1221615136 learned_addr learned my addr 172.21.15.139:0/1221615136 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.069+0000 7fcb16b9e640 1 --2- 172.21.15.139:0/1221615136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb1015bfb0 0x7fcb1015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.069+0000 7fcb1639d640 1 -- 172.21.15.139:0/1221615136 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb10072be0 msgr2=0x7fcb1015a7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:08:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.069+0000 7fcb1639d640 1 --2- 172.21.15.139:0/1221615136 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb10072be0 0x7fcb1015a7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.069+0000 7fcb1639d640 1 -- 172.21.15.139:0/1221615136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb1015bfb0 msgr2=0x7fcb1015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.069+0000 7fcb1639d640 1 --2- 172.21.15.139:0/1221615136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb1015bfb0 0x7fcb1015ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.069+0000 7fcb1639d640 1 -- 172.21.15.139:0/1221615136 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb101f4a90 con 0x7fcb10071e00 2024-09-16T00:08:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.070+0000 7fcb1639d640 1 --2- 172.21.15.139:0/1221615136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10071e00 0x7fcb10151f70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800d620 tx=0x7fcb0800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.070+0000 7fcb0effd640 1 -- 172.21.15.139:0/1221615136 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0800b620 con 0x7fcb10071e00 2024-09-16T00:08:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.070+0000 7fcb0effd640 1 -- 172.21.15.139:0/1221615136 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb08019070 con 0x7fcb10071e00 2024-09-16T00:08:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.070+0000 7fcb1739f640 1 -- 172.21.15.139:0/1221615136 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb101f4d70 con 0x7fcb10071e00 2024-09-16T00:08:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.070+0000 7fcb1739f640 1 -- 172.21.15.139:0/1221615136 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb101f5330 con 0x7fcb10071e00 2024-09-16T00:08:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.070+0000 7fcb0effd640 1 -- 172.21.15.139:0/1221615136 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0800b9e0 con 0x7fcb10071e00 2024-09-16T00:08:23.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.073+0000 7fcb0effd640 1 -- 172.21.15.139:0/1221615136 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcb0800bb80 con 0x7fcb10071e00 2024-09-16T00:08:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.074+0000 7fcb0effd640 1 --2- 172.21.15.139:0/1221615136 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcafc06c3b0 0x7fcafc06e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.074+0000 7fcb1739f640 1 -- 172.21.15.139:0/1221615136 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf0005290 con 0x7fcb10071e00 2024-09-16T00:08:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.074+0000 7fcb15b9c640 1 --2- 172.21.15.139:0/1221615136 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcafc06c3b0 0x7fcafc06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.075+0000 7fcb15b9c640 1 --2- 172.21.15.139:0/1221615136 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcafc06c3b0 0x7fcafc06e870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcb04007fd0 tx=0x7fcb04007aa0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:23.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.075+0000 7fcb0effd640 1 -- 172.21.15.139:0/1221615136 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5915+0+0 (secure 0 0 0) 0x7fcb0808b920 con 0x7fcb10071e00 2024-09-16T00:08:23.236 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:08:23.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.080+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff83802cf80 con 0x7ff83802c640 2024-09-16T00:08:23.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.082+0000 7fcb0effd640 1 -- 172.21.15.139:0/1221615136 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcb08059fa0 con 0x7fcb10071e00 2024-09-16T00:08:23.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.082+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff83802cf80 con 0x7ff83802c640 2024-09-16T00:08:23.237 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:23.238 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:23.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.082+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7ff83802d600 0x7ff83820b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.082+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7ff83802da60 con 0x7ff83802d600 2024-09-16T00:08:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.083+0000 7ff836ffd640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7ff83802d600 0x7ff83820b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.085+0000 7ff836ffd640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7ff83802d600 0x7ff83820b340 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.085+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff83802da60 con 0x7ff83802d600 2024-09-16T00:08:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.099+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff83820b880 con 0x7ff83802d600 2024-09-16T00:08:23.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.101+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff83820b880 con 0x7ff83802d600 2024-09-16T00:08:23.239 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:23.240 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:23.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.101+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7ff83820bf00 0x7ff83820e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:23.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.102+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7ff8382112d0 con 0x7ff83820bf00 2024-09-16T00:08:23.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.102+0000 7ff837fff640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7ff83820bf00 0x7ff83820e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:23.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.102+0000 7ff837fff640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7ff83820bf00 0x7ff83820e2f0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:23.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.103+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff8382112d0 con 0x7ff83820bf00 2024-09-16T00:08:23.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.117+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff838210010 con 0x7ff83820bf00 2024-09-16T00:08:23.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.119+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff838210010 con 0x7ff83820bf00 2024-09-16T00:08:23.241 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:23.242 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:23.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.119+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7ff838210730 0x7ff838210b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:23.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.119+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7ff8382146a0 con 0x7ff838210730 2024-09-16T00:08:23.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.119+0000 7ff8377fe640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7ff838210730 0x7ff838210b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:23.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.120+0000 7ff8377fe640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7ff838210730 0x7ff838210b50 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:23.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.120+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff8382146a0 con 0x7ff838210730 2024-09-16T00:08:23.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.142+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff838213520 con 0x7ff838210730 2024-09-16T00:08:23.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.144+0000 7ff81ffff640 1 -- 172.21.15.139:0/2691561132 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff838213520 con 0x7ff838210730 2024-09-16T00:08:23.243 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:23.244 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:23.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.144+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7ff8380c0e10 msgr2=0x7ff8380c2520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:23.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.144+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7ff8380c0e10 0x7ff8380c2520 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.144+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7ff8380c3e60 msgr2=0x7ff83806c030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:23.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.144+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7ff8380c3e60 0x7ff83806c030 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.144+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7ff8381f8530 msgr2=0x7ff8381f8910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:23.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.144+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7ff8381f8530 0x7ff8381f8910 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.144+0000 7ff837fff640 1 -- 172.21.15.139:0/2691561132 reap_dead start 2024-09-16T00:08:23.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.144+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7ff838158780 msgr2=0x7ff838158b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:23.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.144+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7ff838158780 0x7ff838158b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.144+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7ff83802c640 msgr2=0x7ff83802ca40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:23.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7ff83802c640 0x7ff83802ca40 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7ff83802d600 msgr2=0x7ff83820b340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:23.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7ff83802d600 0x7ff83820b340 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7ff83820bf00 msgr2=0x7ff83820e2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:23.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7ff83820bf00 0x7ff83820e2f0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7ff838210730 msgr2=0x7ff838210b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:23.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7ff838210730 0x7ff838210b50 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff837fff640 1 -- 172.21.15.139:0/2691561132 reap_dead start 2024-09-16T00:08:23.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff82406c380 msgr2=0x7ff82406e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:23.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff82406c380 0x7ff82406e840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff82802a200 tx=0x7ff82802e000 comp rx=0 tx=0).stop 2024-09-16T00:08:23.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8380c8e90 msgr2=0x7ff838165a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:23.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8380c8e90 0x7ff838165a60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff8380c7ca0 tx=0x7ff82c00c820 comp rx=0 tx=0).stop 2024-09-16T00:08:23.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 shutdown_connections 2024-09-16T00:08:23.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff82406c380 0x7ff82406e840 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 --2- 172.21.15.139:0/2691561132 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff8380c8e90 0x7ff838165a60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 >> 172.21.15.139:0/2691561132 conn(0x7ff8380c9a80 msgr2=0x7ff8380cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 shutdown_connections 2024-09-16T00:08:23.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.145+0000 7ff83dbaf640 1 -- 172.21.15.139:0/2691561132 wait complete. 2024-09-16T00:08:23.247 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:23.230+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:23.332 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:23.332+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:23.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.339+0000 7feabdec9640 1 -- 172.21.15.139:0/1820307379 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7feab80b2930 con 0x7feab80c8b50 2024-09-16T00:08:23.394 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:23.394 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:23.394 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:23.394 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:23.403 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:08:23.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.414+0000 7fcb1739f640 1 -- 172.21.15.139:0/1221615136 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 0.13749944564937777} v 0) -- 0x7fcaf0005d90 con 0x7fcb10071e00 2024-09-16T00:08:23.434 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:23.432+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:23.519 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:23.519 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:23.519 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:23.519 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:23.519 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:23.528 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:08:23.534 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:23.533+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:23.635 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:23.634+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:23.667 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:23.668 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:23.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:23.668 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:23.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.668 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:23.668 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:23.668 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.050802578999999, 2024-09-16T00:08:23.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037514929999999998, 2024-09-16T00:08:23.668 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.668 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.668 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.669 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.669 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.669 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:23.669 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.669 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.669 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.669 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:23.669 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.669 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.669 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615364+0000", 2024-09-16T00:08:23.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8430000000000003e-06 2024-09-16T00:08:23.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615371+0000", 2024-09-16T00:08:23.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.148e-06 2024-09-16T00:08:23.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615380+0000", 2024-09-16T00:08:23.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5889999999999992e-06 2024-09-16T00:08:23.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615389+0000", 2024-09-16T00:08:23.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1620000000000008e-06 2024-09-16T00:08:23.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615525+0000", 2024-09-16T00:08:23.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000136083 2024-09-16T00:08:23.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615662+0000", 2024-09-16T00:08:23.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013696100000000001 2024-09-16T00:08:23.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:23.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615799+0000", 2024-09-16T00:08:23.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013781399999999999 2024-09-16T00:08:23.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:23.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:23.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002363502 2024-09-16T00:08:23.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:23.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619018+0000", 2024-09-16T00:08:23.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085522199999999999 2024-09-16T00:08:23.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619068+0000", 2024-09-16T00:08:23.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9545000000000001e-05 2024-09-16T00:08:23.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619111+0000", 2024-09-16T00:08:23.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3624000000000001e-05 2024-09-16T00:08:23.674 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.674 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.674 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.674 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:23.674 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:23.674 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.050750508, 2024-09-16T00:08:23.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043656110000000001, 2024-09-16T00:08:23.675 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.675 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.675 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.675 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.675 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.675 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.675 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:23.675 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.675 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.675 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.675 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:23.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.676 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.676 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:23.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2199999999999999e-07 2024-09-16T00:08:23.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615416+0000", 2024-09-16T00:08:23.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5590000000000001e-06 2024-09-16T00:08:23.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615423+0000", 2024-09-16T00:08:23.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6250000000000001e-06 2024-09-16T00:08:23.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615426+0000", 2024-09-16T00:08:23.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2540000000000001e-06 2024-09-16T00:08:23.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616075+0000", 2024-09-16T00:08:23.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064968800000000004 2024-09-16T00:08:23.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616142+0000", 2024-09-16T00:08:23.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6203000000000003e-05 2024-09-16T00:08:23.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:23.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616247+0000", 2024-09-16T00:08:23.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010572499999999999 2024-09-16T00:08:23.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:23.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618913+0000", 2024-09-16T00:08:23.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026654339999999999 2024-09-16T00:08:23.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:23.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619712+0000", 2024-09-16T00:08:23.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079885199999999996 2024-09-16T00:08:23.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.681 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619744+0000", 2024-09-16T00:08:23.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2790000000000003e-05 2024-09-16T00:08:23.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.681 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.681 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619778+0000", 2024-09-16T00:08:23.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3059000000000003e-05 2024-09-16T00:08:23.681 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.681 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.682 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.682 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:23.682 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:23.682 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.050714429000003, 2024-09-16T00:08:23.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087673219999999993, 2024-09-16T00:08:23.682 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.682 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.682 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.682 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:23.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:23.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:23.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:23.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:08:23.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5392000000000001e-05 2024-09-16T00:08:23.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615470+0000", 2024-09-16T00:08:23.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1999999999999999e-06 2024-09-16T00:08:23.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615474+0000", 2024-09-16T00:08:23.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7359999999999999e-06 2024-09-16T00:08:23.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617614+0000", 2024-09-16T00:08:23.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021398189999999998 2024-09-16T00:08:23.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617724+0000", 2024-09-16T00:08:23.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110808 2024-09-16T00:08:23.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:23.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617856+0000", 2024-09-16T00:08:23.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131676 2024-09-16T00:08:23.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:23.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:23.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029629280000000001 2024-09-16T00:08:23.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:23.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624158+0000", 2024-09-16T00:08:23.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033386549999999998 2024-09-16T00:08:23.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624189+0000", 2024-09-16T00:08:23.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1508999999999999e-05 2024-09-16T00:08:23.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624215+0000", 2024-09-16T00:08:23.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6228999999999999e-05 2024-09-16T00:08:23.688 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.688 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.688 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.689 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:23.689 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:23.689 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.050680413999999, 2024-09-16T00:08:23.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053285279999999999, 2024-09-16T00:08:23.689 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.689 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.689 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.689 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:23.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:23.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:23.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5600000000000002e-07 2024-09-16T00:08:23.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615484+0000", 2024-09-16T00:08:23.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.688e-06 2024-09-16T00:08:23.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615488+0000", 2024-09-16T00:08:23.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.658e-06 2024-09-16T00:08:23.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615490+0000", 2024-09-16T00:08:23.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.728e-06 2024-09-16T00:08:23.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616441+0000", 2024-09-16T00:08:23.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00095065500000000001 2024-09-16T00:08:23.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616539+0000", 2024-09-16T00:08:23.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8335e-05 2024-09-16T00:08:23.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:23.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616631+0000", 2024-09-16T00:08:23.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1260000000000004e-05 2024-09-16T00:08:23.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:23.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619837+0000", 2024-09-16T00:08:23.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003206899 2024-09-16T00:08:23.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:23.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620725+0000", 2024-09-16T00:08:23.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088713599999999996 2024-09-16T00:08:23.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620772+0000", 2024-09-16T00:08:23.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7639999999999998e-05 2024-09-16T00:08:23.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620811+0000", 2024-09-16T00:08:23.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8272999999999998e-05 2024-09-16T00:08:23.696 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.696 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.696 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.697 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:23.697 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:23.697 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.050265326000002, 2024-09-16T00:08:23.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029886169999999998, 2024-09-16T00:08:23.697 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.697 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.697 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.698 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:23.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:23.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615898+0000", 2024-09-16T00:08:23.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.237e-06 2024-09-16T00:08:23.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615904+0000", 2024-09-16T00:08:23.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:23.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615915+0000", 2024-09-16T00:08:23.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1003e-05 2024-09-16T00:08:23.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615922+0000", 2024-09-16T00:08:23.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9959999999999996e-06 2024-09-16T00:08:23.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615988+0000", 2024-09-16T00:08:23.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5806e-05 2024-09-16T00:08:23.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616009+0000", 2024-09-16T00:08:23.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0729000000000001e-05 2024-09-16T00:08:23.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:23.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028462130000000002 2024-09-16T00:08:23.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618886+0000", 2024-09-16T00:08:23.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.082e-05 2024-09-16T00:08:23.702 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.702 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.702 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.703 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:23.703 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:23.703 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.048653854000001, 2024-09-16T00:08:23.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032610299999999998, 2024-09-16T00:08:23.703 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.703 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.703 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.703 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:23.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:23.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617556+0000", 2024-09-16T00:08:23.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7431e-05 2024-09-16T00:08:23.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617847+0000", 2024-09-16T00:08:23.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029133400000000003 2024-09-16T00:08:23.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617858+0000", 2024-09-16T00:08:23.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0222e-05 2024-09-16T00:08:23.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617864+0000", 2024-09-16T00:08:23.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7619999999999998e-06 2024-09-16T00:08:23.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617909+0000", 2024-09-16T00:08:23.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4747999999999999e-05 2024-09-16T00:08:23.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617927+0000", 2024-09-16T00:08:23.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7785999999999999e-05 2024-09-16T00:08:23.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620734+0000", 2024-09-16T00:08:23.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002807305 2024-09-16T00:08:23.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620770+0000", 2024-09-16T00:08:23.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5441999999999997e-05 2024-09-16T00:08:23.708 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.708 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.708 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.708 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:23.708 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:23.708 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.048233170000003, 2024-09-16T00:08:23.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028641809999999999, 2024-09-16T00:08:23.709 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.709 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.709 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.709 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:23.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:23.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617930+0000", 2024-09-16T00:08:23.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0200000000000002e-07 2024-09-16T00:08:23.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617934+0000", 2024-09-16T00:08:23.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:23.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617943+0000", 2024-09-16T00:08:23.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5820000000000002e-06 2024-09-16T00:08:23.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617949+0000", 2024-09-16T00:08:23.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5069999999999997e-06 2024-09-16T00:08:23.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618094+0000", 2024-09-16T00:08:23.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014515599999999999 2024-09-16T00:08:23.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618111+0000", 2024-09-16T00:08:23.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6705999999999999e-05 2024-09-16T00:08:23.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:08:23.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026603070000000002 2024-09-16T00:08:23.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620793+0000", 2024-09-16T00:08:23.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2214000000000001e-05 2024-09-16T00:08:23.714 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.714 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.714 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.714 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:23.714 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:23.715 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.048202381000003, 2024-09-16T00:08:23.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028624549999999999, 2024-09-16T00:08:23.715 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.715 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.715 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.715 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:23.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:23.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:23.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:23.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:23.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0570000000000001e-06 2024-09-16T00:08:23.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617970+0000", 2024-09-16T00:08:23.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.669e-06 2024-09-16T00:08:23.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617976+0000", 2024-09-16T00:08:23.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1369999999999999e-06 2024-09-16T00:08:23.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618268+0000", 2024-09-16T00:08:23.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029164600000000001 2024-09-16T00:08:23.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618284+0000", 2024-09-16T00:08:23.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5824e-05 2024-09-16T00:08:23.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:23.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025127890000000001 2024-09-16T00:08:23.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620822+0000", 2024-09-16T00:08:23.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6009000000000001e-05 2024-09-16T00:08:23.719 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.719 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.719 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.720 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-16T00:08:23.720 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:23.720 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.048174799999998, 2024-09-16T00:08:23.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028650350000000002, 2024-09-16T00:08:23.720 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.720 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.720 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.720 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:23.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:23.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:23.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7599999999999998e-07 2024-09-16T00:08:23.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617991+0000", 2024-09-16T00:08:23.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1860000000000001e-06 2024-09-16T00:08:23.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617999+0000", 2024-09-16T00:08:23.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3250000000000002e-06 2024-09-16T00:08:23.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618004+0000", 2024-09-16T00:08:23.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7300000000000002e-06 2024-09-16T00:08:23.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618422+0000", 2024-09-16T00:08:23.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00041763199999999998 2024-09-16T00:08:23.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:23.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6112000000000001e-05 2024-09-16T00:08:23.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620824+0000", 2024-09-16T00:08:23.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002385954 2024-09-16T00:08:23.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620853+0000", 2024-09-16T00:08:23.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8719999999999999e-05 2024-09-16T00:08:23.724 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.724 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.724 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.725 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-16T00:08:23.725 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:23.726 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.048148978, 2024-09-16T00:08:23.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032703020000000001, 2024-09-16T00:08:23.726 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.726 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.726 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.726 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:23.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:23.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618014+0000", 2024-09-16T00:08:23.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7799999999999997e-07 2024-09-16T00:08:23.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618017+0000", 2024-09-16T00:08:23.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4309999999999998e-06 2024-09-16T00:08:23.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618023+0000", 2024-09-16T00:08:23.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8490000000000002e-06 2024-09-16T00:08:23.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618028+0000", 2024-09-16T00:08:23.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3489999999999999e-06 2024-09-16T00:08:23.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:08:23.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055724099999999999 2024-09-16T00:08:23.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.732 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618603+0000", 2024-09-16T00:08:23.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7486000000000002e-05 2024-09-16T00:08:23.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621235+0000", 2024-09-16T00:08:23.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002632087 2024-09-16T00:08:23.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621284+0000", 2024-09-16T00:08:23.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8581000000000001e-05 2024-09-16T00:08:23.734 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.734 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.734 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.734 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:14 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:23.734 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:23.735 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.047211998000002, 2024-09-16T00:08:23.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016807599999999999, 2024-09-16T00:08:23.735 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.735 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.735 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:23.735 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:23.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:23.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618951+0000", 2024-09-16T00:08:23.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:23.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618954+0000", 2024-09-16T00:08:23.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.145e-06 2024-09-16T00:08:23.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618963+0000", 2024-09-16T00:08:23.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5720000000000003e-06 2024-09-16T00:08:23.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618969+0000", 2024-09-16T00:08:23.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3319999999999997e-06 2024-09-16T00:08:23.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619008+0000", 2024-09-16T00:08:23.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9096e-05 2024-09-16T00:08:23.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619013+0000", 2024-09-16T00:08:23.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4710000000000001e-06 2024-09-16T00:08:23.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619118+0000", 2024-09-16T00:08:23.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000105809 2024-09-16T00:08:23.741 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.741 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.741 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.742 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-16T00:08:23.742 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:23.742 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.047173891, 2024-09-16T00:08:23.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002325307, 2024-09-16T00:08:23.742 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.742 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.743 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.743 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:23.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:23.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:23.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:23.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618993+0000", 2024-09-16T00:08:23.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.687e-06 2024-09-16T00:08:23.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619000+0000", 2024-09-16T00:08:23.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3590000000000002e-06 2024-09-16T00:08:23.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619005+0000", 2024-09-16T00:08:23.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.95e-06 2024-09-16T00:08:23.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619039+0000", 2024-09-16T00:08:23.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3942999999999997e-05 2024-09-16T00:08:23.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619055+0000", 2024-09-16T00:08:23.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6107e-05 2024-09-16T00:08:23.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621287+0000", 2024-09-16T00:08:23.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022318070000000001 2024-09-16T00:08:23.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:23.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7129999999999999e-05 2024-09-16T00:08:23.749 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.749 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.749 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.750 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-16T00:08:23.750 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:23.750 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.047135894999997, 2024-09-16T00:08:23.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002314328, 2024-09-16T00:08:23.750 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.750 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.751 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.751 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:23.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:23.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:23.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6399999999999998e-07 2024-09-16T00:08:23.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619030+0000", 2024-09-16T00:08:23.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.219e-06 2024-09-16T00:08:23.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619038+0000", 2024-09-16T00:08:23.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9899999999999997e-06 2024-09-16T00:08:23.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619043+0000", 2024-09-16T00:08:23.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5580000000000002e-06 2024-09-16T00:08:23.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619220+0000", 2024-09-16T00:08:23.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000176937 2024-09-16T00:08:23.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619237+0000", 2024-09-16T00:08:23.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7691999999999999e-05 2024-09-16T00:08:23.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621315+0000", 2024-09-16T00:08:23.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020778900000000002 2024-09-16T00:08:23.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621341+0000", 2024-09-16T00:08:23.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5678e-05 2024-09-16T00:08:23.757 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.757 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.757 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.758 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:15 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:23.758 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:23.758 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.046520925999999, 2024-09-16T00:08:23.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143416, 2024-09-16T00:08:23.759 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.759 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.759 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:23.759 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:23.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:23.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619642+0000", 2024-09-16T00:08:23.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5199999999999995e-07 2024-09-16T00:08:23.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619645+0000", 2024-09-16T00:08:23.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.134e-06 2024-09-16T00:08:23.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.763 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619655+0000", 2024-09-16T00:08:23.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3330000000000003e-06 2024-09-16T00:08:23.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.764 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.764 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619662+0000", 2024-09-16T00:08:23.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0500000000000003e-06 2024-09-16T00:08:23.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.764 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619704+0000", 2024-09-16T00:08:23.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1766e-05 2024-09-16T00:08:23.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619708+0000", 2024-09-16T00:08:23.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3440000000000003e-06 2024-09-16T00:08:23.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619785+0000", 2024-09-16T00:08:23.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6936999999999999e-05 2024-09-16T00:08:23.766 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.767 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.767 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.767 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:31 2.d e22/19 ondisk, result = 0)", 2024-09-16T00:08:23.767 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:23.768 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.046422116999999, 2024-09-16T00:08:23.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101826, 2024-09-16T00:08:23.768 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.768 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.768 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:23.769 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:23.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.770 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.770 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:23.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619741+0000", 2024-09-16T00:08:23.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3099999999999997e-07 2024-09-16T00:08:23.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:08:23.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1719999999999998e-06 2024-09-16T00:08:23.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619771+0000", 2024-09-16T00:08:23.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4754000000000001e-05 2024-09-16T00:08:23.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619790+0000", 2024-09-16T00:08:23.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9516000000000001e-05 2024-09-16T00:08:23.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619830+0000", 2024-09-16T00:08:23.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9848000000000002e-05 2024-09-16T00:08:23.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619834+0000", 2024-09-16T00:08:23.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1150000000000004e-06 2024-09-16T00:08:23.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619842+0000", 2024-09-16T00:08:23.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.79e-06 2024-09-16T00:08:23.775 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.775 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.775 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.775 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:30 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:23.775 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:23.776 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.042071129999997, 2024-09-16T00:08:23.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013076400000000001, 2024-09-16T00:08:23.776 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.776 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.776 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:23.776 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:23.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:23.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624092+0000", 2024-09-16T00:08:23.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5799999999999999e-07 2024-09-16T00:08:23.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624096+0000", 2024-09-16T00:08:23.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:23.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.782 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624106+0000", 2024-09-16T00:08:23.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7629999999999995e-06 2024-09-16T00:08:23.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.782 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.782 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.782 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624112+0000", 2024-09-16T00:08:23.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.626e-06 2024-09-16T00:08:23.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624149+0000", 2024-09-16T00:08:23.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7094000000000001e-05 2024-09-16T00:08:23.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624154+0000", 2024-09-16T00:08:23.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7359999999999998e-06 2024-09-16T00:08:23.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624222+0000", 2024-09-16T00:08:23.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7680000000000003e-05 2024-09-16T00:08:23.785 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.785 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.785 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.785 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:39 2.6 2:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:23.785 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:23.785 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.025884572000002, 2024-09-16T00:08:23.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050072119999999996, 2024-09-16T00:08:23.786 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.786 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.786 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:23.786 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.786 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:23.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:23.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.788 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.788 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.788 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640280+0000", 2024-09-16T00:08:23.788 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5380000000000001e-06 2024-09-16T00:08:23.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.788 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.788 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.788 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640296+0000", 2024-09-16T00:08:23.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5950000000000001e-05 2024-09-16T00:08:23.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.789 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.789 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640327+0000", 2024-09-16T00:08:23.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0309999999999999e-05 2024-09-16T00:08:23.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640340+0000", 2024-09-16T00:08:23.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3773e-05 2024-09-16T00:08:23.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640422+0000", 2024-09-16T00:08:23.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1122e-05 2024-09-16T00:08:23.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640572+0000", 2024-09-16T00:08:23.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015044099999999999 2024-09-16T00:08:23.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:08:23.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640691+0000", 2024-09-16T00:08:23.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000119393 2024-09-16T00:08:23.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:23.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645006+0000", 2024-09-16T00:08:23.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043143779999999998 2024-09-16T00:08:23.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:23.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645169+0000", 2024-09-16T00:08:23.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016323299999999999 2024-09-16T00:08:23.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:23.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645227+0000", 2024-09-16T00:08:23.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8e-05 2024-09-16T00:08:23.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645285+0000", 2024-09-16T00:08:23.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8074e-05 2024-09-16T00:08:23.795 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.795 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.795 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.795 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.795 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.795 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:39 2.6 e22/19 ondisk, result = 0)", 2024-09-16T00:08:23.796 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:23.796 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.021312461000001, 2024-09-16T00:08:23.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016541200000000001, 2024-09-16T00:08:23.796 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:23.796 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:23.796 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:23.796 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:23.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:23.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:23.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:23.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:23.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:23.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:23.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644852+0000", 2024-09-16T00:08:23.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1670000000000002e-06 2024-09-16T00:08:23.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:23.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644865+0000", 2024-09-16T00:08:23.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2826000000000001e-05 2024-09-16T00:08:23.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.800 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:23.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644894+0000", 2024-09-16T00:08:23.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9264999999999998e-05 2024-09-16T00:08:23.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.800 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:23.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644911+0000", 2024-09-16T00:08:23.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6827999999999999e-05 2024-09-16T00:08:23.801 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.801 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.801 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:23.801 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644989+0000", 2024-09-16T00:08:23.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7849999999999995e-05 2024-09-16T00:08:23.801 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:23.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644999+0000", 2024-09-16T00:08:23.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7289999999999995e-06 2024-09-16T00:08:23.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:23.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:23.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:23.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645015+0000", 2024-09-16T00:08:23.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6747000000000001e-05 2024-09-16T00:08:23.803 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.803 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.803 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.804 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:23.804 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:23.804 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:23.807 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:08:23.807 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:23.805+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:23.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.817+0000 7fcb0effd640 1 -- 172.21.15.139:0/1221615136 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 0.13749944564937777}]=0 reweighted osd.6 to 0.137499 (2333) v37) ==== 135+0+0 (secure 0 0 0) 0x7fcb080598a0 con 0x7fcb10071e00 2024-09-16T00:08:23.818 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.6 to 0.137499 (2333) 2024-09-16T00:08:23.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.819+0000 7fcb1739f640 1 -- 172.21.15.139:0/1221615136 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcafc06c3b0 msgr2=0x7fcafc06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.819+0000 7fcb1739f640 1 --2- 172.21.15.139:0/1221615136 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcafc06c3b0 0x7fcafc06e870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcb04007fd0 tx=0x7fcb04007aa0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.819+0000 7fcb1739f640 1 -- 172.21.15.139:0/1221615136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10071e00 msgr2=0x7fcb10151f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.819+0000 7fcb1739f640 1 --2- 172.21.15.139:0/1221615136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10071e00 0x7fcb10151f70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800d620 tx=0x7fcb0800dae0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.820+0000 7fcb1739f640 1 -- 172.21.15.139:0/1221615136 shutdown_connections 2024-09-16T00:08:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.820+0000 7fcb1739f640 1 --2- 172.21.15.139:0/1221615136 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcafc06c3b0 0x7fcafc06e870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.820+0000 7fcb1739f640 1 --2- 172.21.15.139:0/1221615136 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb1015bfb0 0x7fcb1015ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.820+0000 7fcb1739f640 1 --2- 172.21.15.139:0/1221615136 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb10072be0 0x7fcb1015a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.820+0000 7fcb1739f640 1 --2- 172.21.15.139:0/1221615136 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10071e00 0x7fcb10151f70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.820+0000 7fcb1739f640 1 -- 172.21.15.139:0/1221615136 >> 172.21.15.139:0/1221615136 conn(0x7fcb100c91d0 msgr2=0x7fcb101575f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.820+0000 7fcb1739f640 1 -- 172.21.15.139:0/1221615136 shutdown_connections 2024-09-16T00:08:23.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.821+0000 7fcb1739f640 1 -- 172.21.15.139:0/1221615136 wait complete. 2024-09-16T00:08:23.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.824+0000 7feaaffff640 1 -- 172.21.15.139:0/1820307379 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v37) ==== 91+0+0 (secure 0 0 0) 0x7feaa005cbe0 con 0x7feab80c8b50 2024-09-16T00:08:23.826 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is unset 2024-09-16T00:08:23.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.828+0000 7feabdec9640 1 -- 172.21.15.139:0/1820307379 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fea9806c300 msgr2=0x7fea9806e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:23.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.828+0000 7feabdec9640 1 --2- 172.21.15.139:0/1820307379 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fea9806c300 0x7fea9806e7c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feaa8034000 tx=0x7feaa802e000 comp rx=0 tx=0).stop 2024-09-16T00:08:23.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.828+0000 7feabdec9640 1 -- 172.21.15.139:0/1820307379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab80c8b50 msgr2=0x7feab815e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:23.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.828+0000 7feabdec9640 1 --2- 172.21.15.139:0/1820307379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab80c8b50 0x7feab815e360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feaa000c370 tx=0x7feaa000c830 comp rx=0 tx=0).stop 2024-09-16T00:08:23.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.831+0000 7feabdec9640 1 -- 172.21.15.139:0/1820307379 shutdown_connections 2024-09-16T00:08:23.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.831+0000 7feabdec9640 1 --2- 172.21.15.139:0/1820307379 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fea9806c300 0x7fea9806e7c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.831+0000 7feabdec9640 1 --2- 172.21.15.139:0/1820307379 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feab815fd90 0x7feab81e57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.831+0000 7feabdec9640 1 --2- 172.21.15.139:0/1820307379 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feab815f9b0 0x7feab81e5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.831+0000 7feabdec9640 1 --2- 172.21.15.139:0/1820307379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feab80c8b50 0x7feab815e360 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:23.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.831+0000 7feabdec9640 1 -- 172.21.15.139:0/1820307379 >> 172.21.15.139:0/1820307379 conn(0x7feab806c150 msgr2=0x7feab80cdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:23.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.833+0000 7feabdec9640 1 -- 172.21.15.139:0/1820307379 shutdown_connections 2024-09-16T00:08:23.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:23.833+0000 7feabdec9640 1 -- 172.21.15.139:0/1820307379 wait complete. 2024-09-16T00:08:23.906 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:23.906+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:23.956 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:23.956 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:23.956 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:23.956 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:23.968 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:08:24.010 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:24.009+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:24.108 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:24.107+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:24.155 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:24.156 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:24.156 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:24.156 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:24.156 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:24.167 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:08:24.212 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:24.211+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:24.305 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:24.306 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:24.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:24.306 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:24.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.306 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:24.306 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:24.306 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.687865201000001, 2024-09-16T00:08:24.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038263220000000001, 2024-09-16T00:08:24.306 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.306 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.306 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.306 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:24.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:24.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615465+0000", 2024-09-16T00:08:24.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2769999999999999e-06 2024-09-16T00:08:24.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615471+0000", 2024-09-16T00:08:24.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5729999999999996e-06 2024-09-16T00:08:24.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615483+0000", 2024-09-16T00:08:24.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1838000000000001e-05 2024-09-16T00:08:24.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615492+0000", 2024-09-16T00:08:24.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2410000000000001e-06 2024-09-16T00:08:24.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615596+0000", 2024-09-16T00:08:24.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104097 2024-09-16T00:08:24.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615688+0000", 2024-09-16T00:08:24.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1225999999999997e-05 2024-09-16T00:08:24.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:24.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615804+0000", 2024-09-16T00:08:24.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000116601 2024-09-16T00:08:24.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:24.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618165+0000", 2024-09-16T00:08:24.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023609429999999999 2024-09-16T00:08:24.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:24.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:08:24.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010258509999999999 2024-09-16T00:08:24.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619243+0000", 2024-09-16T00:08:24.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2327999999999998e-05 2024-09-16T00:08:24.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619287+0000", 2024-09-16T00:08:24.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3346999999999997e-05 2024-09-16T00:08:24.313 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.313 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.313 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.313 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:24.313 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:24.313 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.687810124999999, 2024-09-16T00:08:24.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0054735510000000001, 2024-09-16T00:08:24.314 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.314 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.314 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.314 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:24.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:24.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615516+0000", 2024-09-16T00:08:24.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6899999999999998e-07 2024-09-16T00:08:24.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615519+0000", 2024-09-16T00:08:24.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1760000000000002e-06 2024-09-16T00:08:24.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615526+0000", 2024-09-16T00:08:24.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8399999999999997e-06 2024-09-16T00:08:24.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615531+0000", 2024-09-16T00:08:24.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4450000000000004e-06 2024-09-16T00:08:24.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:24.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054127099999999996 2024-09-16T00:08:24.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616140+0000", 2024-09-16T00:08:24.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7230999999999994e-05 2024-09-16T00:08:24.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:24.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616249+0000", 2024-09-16T00:08:24.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109122 2024-09-16T00:08:24.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:24.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618646+0000", 2024-09-16T00:08:24.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023972279999999999 2024-09-16T00:08:24.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:24.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:08:24.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022396220000000001 2024-09-16T00:08:24.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620939+0000", 2024-09-16T00:08:24.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2735999999999998e-05 2024-09-16T00:08:24.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620989+0000", 2024-09-16T00:08:24.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0411000000000002e-05 2024-09-16T00:08:24.318 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.319 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.319 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.319 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:24.319 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:24.319 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.687784645999997, 2024-09-16T00:08:24.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055872789999999997, 2024-09-16T00:08:24.319 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.319 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.319 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.319 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:24.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:24.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:24.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.01e-07 2024-09-16T00:08:24.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615547+0000", 2024-09-16T00:08:24.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7899999999999996e-06 2024-09-16T00:08:24.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:08:24.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1119999999999998e-06 2024-09-16T00:08:24.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615560+0000", 2024-09-16T00:08:24.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:08:24.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616424+0000", 2024-09-16T00:08:24.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086350800000000003 2024-09-16T00:08:24.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:08:24.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9675999999999999e-05 2024-09-16T00:08:24.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:24.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616621+0000", 2024-09-16T00:08:24.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7345999999999999e-05 2024-09-16T00:08:24.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:24.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618654+0000", 2024-09-16T00:08:24.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020335269999999998 2024-09-16T00:08:24.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:24.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621055+0000", 2024-09-16T00:08:24.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024009729999999998 2024-09-16T00:08:24.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621100+0000", 2024-09-16T00:08:24.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5402999999999999e-05 2024-09-16T00:08:24.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621128+0000", 2024-09-16T00:08:24.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7736e-05 2024-09-16T00:08:24.325 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.325 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.325 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.325 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:24.325 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:24.325 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.687758995999999, 2024-09-16T00:08:24.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057024509999999999, 2024-09-16T00:08:24.326 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.326 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.326 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.326 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:24.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:24.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:24.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6300000000000001e-07 2024-09-16T00:08:24.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615569+0000", 2024-09-16T00:08:24.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3709999999999998e-06 2024-09-16T00:08:24.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615580+0000", 2024-09-16T00:08:24.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0448000000000001e-05 2024-09-16T00:08:24.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615584+0000", 2024-09-16T00:08:24.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6020000000000002e-06 2024-09-16T00:08:24.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616850+0000", 2024-09-16T00:08:24.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012661110000000001 2024-09-16T00:08:24.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616915+0000", 2024-09-16T00:08:24.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4820000000000006e-05 2024-09-16T00:08:24.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:24.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617031+0000", 2024-09-16T00:08:24.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115258 2024-09-16T00:08:24.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:24.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618659+0000", 2024-09-16T00:08:24.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001627975 2024-09-16T00:08:24.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:24.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621179+0000", 2024-09-16T00:08:24.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025203410000000002 2024-09-16T00:08:24.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621238+0000", 2024-09-16T00:08:24.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9323000000000001e-05 2024-09-16T00:08:24.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621269+0000", 2024-09-16T00:08:24.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0839000000000001e-05 2024-09-16T00:08:24.331 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.331 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.331 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.332 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:24.332 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:24.332 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.687735246999999, 2024-09-16T00:08:24.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058012769999999996, 2024-09-16T00:08:24.332 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.332 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.332 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.332 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:24.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.333 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:24.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.333 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615591+0000", 2024-09-16T00:08:24.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6199999999999999e-07 2024-09-16T00:08:24.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615594+0000", 2024-09-16T00:08:24.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8739999999999999e-06 2024-09-16T00:08:24.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615599+0000", 2024-09-16T00:08:24.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:08:24.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615603+0000", 2024-09-16T00:08:24.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8489999999999999e-06 2024-09-16T00:08:24.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617235+0000", 2024-09-16T00:08:24.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001632037 2024-09-16T00:08:24.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617348+0000", 2024-09-16T00:08:24.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113387 2024-09-16T00:08:24.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:24.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617458+0000", 2024-09-16T00:08:24.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109765 2024-09-16T00:08:24.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:24.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618664+0000", 2024-09-16T00:08:24.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012059950000000001 2024-09-16T00:08:24.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:24.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621311+0000", 2024-09-16T00:08:24.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026468469999999999 2024-09-16T00:08:24.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621357+0000", 2024-09-16T00:08:24.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6360999999999999e-05 2024-09-16T00:08:24.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621392+0000", 2024-09-16T00:08:24.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4190999999999998e-05 2024-09-16T00:08:24.337 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.337 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.337 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.338 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:24.338 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:24.338 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.687716623, 2024-09-16T00:08:24.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058737279999999999, 2024-09-16T00:08:24.338 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.338 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.338 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.338 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:24.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:24.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:24.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3799999999999999e-07 2024-09-16T00:08:24.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615611+0000", 2024-09-16T00:08:24.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-06 2024-09-16T00:08:24.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.340 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615626+0000", 2024-09-16T00:08:24.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:08:24.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.340 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615634+0000", 2024-09-16T00:08:24.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-06 2024-09-16T00:08:24.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617692+0000", 2024-09-16T00:08:24.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002057718 2024-09-16T00:08:24.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617768+0000", 2024-09-16T00:08:24.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6878999999999994e-05 2024-09-16T00:08:24.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:24.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:24.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019501699999999999 2024-09-16T00:08:24.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:24.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:08:24.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034357200000000002 2024-09-16T00:08:24.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:24.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621429+0000", 2024-09-16T00:08:24.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0012999999999999e-05 2024-09-16T00:08:24.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621459+0000", 2024-09-16T00:08:24.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9498000000000001e-05 2024-09-16T00:08:24.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621483+0000", 2024-09-16T00:08:24.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3957000000000001e-05 2024-09-16T00:08:24.344 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.344 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.344 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.344 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.344 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:24.344 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:24.344 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.687685342999998, 2024-09-16T00:08:24.344 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059371789999999999, 2024-09-16T00:08:24.344 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.344 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.344 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.344 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:24.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:24.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:08:24.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4000000000000002e-07 2024-09-16T00:08:24.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.346 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615643+0000", 2024-09-16T00:08:24.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6860000000000002e-06 2024-09-16T00:08:24.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615649+0000", 2024-09-16T00:08:24.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7610000000000004e-06 2024-09-16T00:08:24.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615653+0000", 2024-09-16T00:08:24.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0380000000000001e-06 2024-09-16T00:08:24.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618197+0000", 2024-09-16T00:08:24.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025440079999999999 2024-09-16T00:08:24.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618312+0000", 2024-09-16T00:08:24.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115083 2024-09-16T00:08:24.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:24.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618412+0000", 2024-09-16T00:08:24.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010016799999999999 2024-09-16T00:08:24.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:24.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621518+0000", 2024-09-16T00:08:24.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031058679999999999 2024-09-16T00:08:24.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:24.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621526+0000", 2024-09-16T00:08:24.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2490000000000004e-06 2024-09-16T00:08:24.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621549+0000", 2024-09-16T00:08:24.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3686999999999999e-05 2024-09-16T00:08:24.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621577+0000", 2024-09-16T00:08:24.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8191000000000002e-05 2024-09-16T00:08:24.350 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.350 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.350 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.350 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:24.350 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:24.350 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.687667677, 2024-09-16T00:08:24.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005989622, 2024-09-16T00:08:24.351 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.351 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.351 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.351 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.351 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.351 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.351 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:24.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.351 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.351 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.351 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.351 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:24.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.352 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.352 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.352 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:24.352 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3799999999999998e-07 2024-09-16T00:08:24.352 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615660+0000", 2024-09-16T00:08:24.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9740000000000001e-06 2024-09-16T00:08:24.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615666+0000", 2024-09-16T00:08:24.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3499999999999996e-06 2024-09-16T00:08:24.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615671+0000", 2024-09-16T00:08:24.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.13e-06 2024-09-16T00:08:24.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618759+0000", 2024-09-16T00:08:24.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030880500000000002 2024-09-16T00:08:24.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618832+0000", 2024-09-16T00:08:24.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3725000000000003e-05 2024-09-16T00:08:24.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:24.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618958+0000", 2024-09-16T00:08:24.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012587899999999999 2024-09-16T00:08:24.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:24.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621583+0000", 2024-09-16T00:08:24.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026246469999999999 2024-09-16T00:08:24.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:24.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621602+0000", 2024-09-16T00:08:24.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9137e-05 2024-09-16T00:08:24.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621629+0000", 2024-09-16T00:08:24.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6451000000000001e-05 2024-09-16T00:08:24.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621648+0000", 2024-09-16T00:08:24.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8941e-05 2024-09-16T00:08:24.357 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.357 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.357 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.357 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:24.357 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:24.357 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.687650058000003, 2024-09-16T00:08:24.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053141380000000004, 2024-09-16T00:08:24.357 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.357 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.357 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.357 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:24.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:24.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:24.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-07 2024-09-16T00:08:24.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615678+0000", 2024-09-16T00:08:24.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:08:24.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615682+0000", 2024-09-16T00:08:24.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6659999999999999e-06 2024-09-16T00:08:24.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615686+0000", 2024-09-16T00:08:24.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6990000000000001e-06 2024-09-16T00:08:24.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615736+0000", 2024-09-16T00:08:24.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9656e-05 2024-09-16T00:08:24.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615864+0000", 2024-09-16T00:08:24.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012836100000000001 2024-09-16T00:08:24.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:24.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615989+0000", 2024-09-16T00:08:24.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000125525 2024-09-16T00:08:24.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:24.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617355+0000", 2024-09-16T00:08:24.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001365982 2024-09-16T00:08:24.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:24.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620897+0000", 2024-09-16T00:08:24.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035419610000000002 2024-09-16T00:08:24.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620941+0000", 2024-09-16T00:08:24.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3998999999999997e-05 2024-09-16T00:08:24.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.364 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620990+0000", 2024-09-16T00:08:24.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8199999999999999e-05 2024-09-16T00:08:24.364 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.364 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.364 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.364 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:24.364 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:24.364 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.6876228, 2024-09-16T00:08:24.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006316449, 2024-09-16T00:08:24.364 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.364 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.364 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.365 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:24.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:24.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:24.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3299999999999998e-07 2024-09-16T00:08:24.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615705+0000", 2024-09-16T00:08:24.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.278e-06 2024-09-16T00:08:24.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615711+0000", 2024-09-16T00:08:24.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8119999999999996e-06 2024-09-16T00:08:24.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615716+0000", 2024-09-16T00:08:24.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4909999999999999e-06 2024-09-16T00:08:24.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617551+0000", 2024-09-16T00:08:24.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00183549 2024-09-16T00:08:24.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617627+0000", 2024-09-16T00:08:24.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6163999999999998e-05 2024-09-16T00:08:24.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:24.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617776+0000", 2024-09-16T00:08:24.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014909499999999999 2024-09-16T00:08:24.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:24.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621072+0000", 2024-09-16T00:08:24.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032955150000000002 2024-09-16T00:08:24.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:24.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621957+0000", 2024-09-16T00:08:24.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088553199999999996 2024-09-16T00:08:24.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621998+0000", 2024-09-16T00:08:24.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0815000000000001e-05 2024-09-16T00:08:24.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:08:24.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0924000000000002e-05 2024-09-16T00:08:24.369 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.369 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.369 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.370 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:6 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:24.370 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:24.370 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.684220832000001, 2024-09-16T00:08:24.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000188111, 2024-09-16T00:08:24.370 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.370 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.370 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:24.370 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:24.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:24.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619106+0000", 2024-09-16T00:08:24.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4399999999999998e-07 2024-09-16T00:08:24.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619110+0000", 2024-09-16T00:08:24.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1110000000000001e-06 2024-09-16T00:08:24.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619120+0000", 2024-09-16T00:08:24.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7909999999999999e-06 2024-09-16T00:08:24.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619133+0000", 2024-09-16T00:08:24.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.359e-05 2024-09-16T00:08:24.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619179+0000", 2024-09-16T00:08:24.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5459e-05 2024-09-16T00:08:24.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619186+0000", 2024-09-16T00:08:24.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0480000000000001e-06 2024-09-16T00:08:24.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619293+0000", 2024-09-16T00:08:24.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000107168 2024-09-16T00:08:24.373 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.373 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.373 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.374 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:22 2.e e22/19 ondisk, result = 0)", 2024-09-16T00:08:24.374 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:24.374 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.682530131, 2024-09-16T00:08:24.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020058200000000001, 2024-09-16T00:08:24.374 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.374 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.374 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:24.374 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:24.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:24.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:24.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0109999999999999e-06 2024-09-16T00:08:24.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620801+0000", 2024-09-16T00:08:24.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6779999999999999e-06 2024-09-16T00:08:24.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620814+0000", 2024-09-16T00:08:24.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3131e-05 2024-09-16T00:08:24.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620821+0000", 2024-09-16T00:08:24.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0430000000000002e-06 2024-09-16T00:08:24.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:24.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3010999999999997e-05 2024-09-16T00:08:24.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620891+0000", 2024-09-16T00:08:24.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1760000000000004e-06 2024-09-16T00:08:24.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620996+0000", 2024-09-16T00:08:24.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010453199999999999 2024-09-16T00:08:24.377 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.378 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.378 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.378 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:7 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:24.378 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:24.378 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.682522147, 2024-09-16T00:08:24.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000195725, 2024-09-16T00:08:24.378 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.378 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.378 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:24.378 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:24.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:24.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620804+0000", 2024-09-16T00:08:24.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6099999999999999e-07 2024-09-16T00:08:24.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620809+0000", 2024-09-16T00:08:24.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4950000000000002e-06 2024-09-16T00:08:24.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:24.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0553e-05 2024-09-16T00:08:24.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620826+0000", 2024-09-16T00:08:24.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5579999999999997e-06 2024-09-16T00:08:24.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620874+0000", 2024-09-16T00:08:24.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7648999999999997e-05 2024-09-16T00:08:24.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620880+0000", 2024-09-16T00:08:24.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8290000000000002e-06 2024-09-16T00:08:24.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620999+0000", 2024-09-16T00:08:24.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011878 2024-09-16T00:08:24.381 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.382 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.382 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.382 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:8 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:24.382 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:24.382 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.682470086999999, 2024-09-16T00:08:24.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00027991099999999998, 2024-09-16T00:08:24.382 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.382 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.382 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:24.382 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:24.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:24.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620856+0000", 2024-09-16T00:08:24.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6800000000000001e-07 2024-09-16T00:08:24.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620859+0000", 2024-09-16T00:08:24.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6879999999999999e-06 2024-09-16T00:08:24.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620867+0000", 2024-09-16T00:08:24.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2819999999999996e-06 2024-09-16T00:08:24.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620872+0000", 2024-09-16T00:08:24.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5369999999999997e-06 2024-09-16T00:08:24.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621043+0000", 2024-09-16T00:08:24.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170791 2024-09-16T00:08:24.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621050+0000", 2024-09-16T00:08:24.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8580000000000002e-06 2024-09-16T00:08:24.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621135+0000", 2024-09-16T00:08:24.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5387000000000001e-05 2024-09-16T00:08:24.385 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.386 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.386 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.386 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:9 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:24.386 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:24.386 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.682441509, 2024-09-16T00:08:24.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00039143799999999997, 2024-09-16T00:08:24.386 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.386 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.386 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:24.386 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:24.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:24.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:24.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6800000000000002e-07 2024-09-16T00:08:24.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:08:24.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0310000000000001e-06 2024-09-16T00:08:24.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620892+0000", 2024-09-16T00:08:24.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9390000000000002e-06 2024-09-16T00:08:24.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620898+0000", 2024-09-16T00:08:24.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4480000000000002e-06 2024-09-16T00:08:24.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621167+0000", 2024-09-16T00:08:24.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026960599999999998 2024-09-16T00:08:24.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621174+0000", 2024-09-16T00:08:24.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.567e-06 2024-09-16T00:08:24.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621275+0000", 2024-09-16T00:08:24.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101579 2024-09-16T00:08:24.390 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.390 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.390 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.390 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:10 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:24.390 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:24.390 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.682419060999997, 2024-09-16T00:08:24.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00049416200000000003, 2024-09-16T00:08:24.390 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.390 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.390 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:24.390 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:24.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:24.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620907+0000", 2024-09-16T00:08:24.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.42e-07 2024-09-16T00:08:24.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620908+0000", 2024-09-16T00:08:24.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4300000000000001e-06 2024-09-16T00:08:24.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620911+0000", 2024-09-16T00:08:24.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3890000000000001e-06 2024-09-16T00:08:24.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620914+0000", 2024-09-16T00:08:24.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3690000000000001e-06 2024-09-16T00:08:24.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621297+0000", 2024-09-16T00:08:24.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038319200000000003 2024-09-16T00:08:24.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621305+0000", 2024-09-16T00:08:24.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4780000000000008e-06 2024-09-16T00:08:24.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621401+0000", 2024-09-16T00:08:24.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5161999999999995e-05 2024-09-16T00:08:24.394 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.394 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.396 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.397 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:11 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:24.397 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:24.397 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.682016312000002, 2024-09-16T00:08:24.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017943899999999999, 2024-09-16T00:08:24.397 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.397 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.397 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:24.397 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:24.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:24.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621310+0000", 2024-09-16T00:08:24.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5199999999999996e-07 2024-09-16T00:08:24.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:24.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1740000000000002e-06 2024-09-16T00:08:24.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621324+0000", 2024-09-16T00:08:24.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7170000000000003e-06 2024-09-16T00:08:24.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621331+0000", 2024-09-16T00:08:24.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2280000000000001e-06 2024-09-16T00:08:24.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621422+0000", 2024-09-16T00:08:24.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1052999999999997e-05 2024-09-16T00:08:24.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621426+0000", 2024-09-16T00:08:24.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.036e-06 2024-09-16T00:08:24.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621489+0000", 2024-09-16T00:08:24.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2478999999999996e-05 2024-09-16T00:08:24.400 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.401 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.401 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.401 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:12 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:24.401 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:24.401 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.681965456999997, 2024-09-16T00:08:24.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022422300000000001, 2024-09-16T00:08:24.401 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.401 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.401 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:24.401 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:24.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:24.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:24.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6699999999999999e-07 2024-09-16T00:08:24.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621363+0000", 2024-09-16T00:08:24.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7020000000000001e-06 2024-09-16T00:08:24.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621371+0000", 2024-09-16T00:08:24.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7209999999999994e-06 2024-09-16T00:08:24.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621376+0000", 2024-09-16T00:08:24.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7940000000000002e-06 2024-09-16T00:08:24.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621511+0000", 2024-09-16T00:08:24.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013518100000000001 2024-09-16T00:08:24.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621514+0000", 2024-09-16T00:08:24.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5599999999999998e-06 2024-09-16T00:08:24.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621584+0000", 2024-09-16T00:08:24.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9898000000000001e-05 2024-09-16T00:08:24.405 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.405 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.405 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.405 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:13 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:24.405 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:24.405 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.681939192999998, 2024-09-16T00:08:24.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026534099999999999, 2024-09-16T00:08:24.405 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.405 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.405 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:24.406 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:24.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:24.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621387+0000", 2024-09-16T00:08:24.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4299999999999999e-07 2024-09-16T00:08:24.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621389+0000", 2024-09-16T00:08:24.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.593e-06 2024-09-16T00:08:24.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621395+0000", 2024-09-16T00:08:24.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.553e-06 2024-09-16T00:08:24.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:08:24.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8410000000000002e-06 2024-09-16T00:08:24.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621597+0000", 2024-09-16T00:08:24.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019794100000000001 2024-09-16T00:08:24.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621600+0000", 2024-09-16T00:08:24.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.185e-06 2024-09-16T00:08:24.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621652+0000", 2024-09-16T00:08:24.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1984999999999998e-05 2024-09-16T00:08:24.409 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.409 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.409 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.409 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:23 2.e e22/19 ondisk, result = 0)", 2024-09-16T00:08:24.409 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:24.409 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.681456244000003, 2024-09-16T00:08:24.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015895699999999999, 2024-09-16T00:08:24.409 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.410 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.410 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:24.410 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:24.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:24.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621870+0000", 2024-09-16T00:08:24.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1600000000000001e-07 2024-09-16T00:08:24.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621873+0000", 2024-09-16T00:08:24.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3079999999999999e-06 2024-09-16T00:08:24.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621882+0000", 2024-09-16T00:08:24.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8429999999999999e-06 2024-09-16T00:08:24.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621896+0000", 2024-09-16T00:08:24.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:08:24.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621944+0000", 2024-09-16T00:08:24.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8062999999999997e-05 2024-09-16T00:08:24.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621951+0000", 2024-09-16T00:08:24.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7689999999999999e-06 2024-09-16T00:08:24.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622028+0000", 2024-09-16T00:08:24.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7243999999999994e-05 2024-09-16T00:08:24.413 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.413 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.413 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.413 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.413 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.413 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:24.415 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:08:24.415 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:24.414+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:24.516 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:24.515+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:24.560 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:24.560 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:24.561 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:24.561 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:24.570 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:08:24.616 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:24.615+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:24.675 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:24.675 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:24.676 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:24.676 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:24.676 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:24.684 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:08:24.717 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:24.716+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:24.799 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:22 2.e e22/19 2:717a0223:::608.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 46.181276427, 2024-09-16T00:08:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042195870000000003, 2024-09-16T00:08:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616426+0000", 2024-09-16T00:08:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.037e-06 2024-09-16T00:08:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616435+0000", 2024-09-16T00:08:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6440000000000006e-06 2024-09-16T00:08:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616447+0000", 2024-09-16T00:08:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2466000000000001e-05 2024-09-16T00:08:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616456+0000", 2024-09-16T00:08:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8799999999999997e-06 2024-09-16T00:08:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616517+0000", 2024-09-16T00:08:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1171000000000001e-05 2024-09-16T00:08:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616540+0000", 2024-09-16T00:08:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2991000000000001e-05 2024-09-16T00:08:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620599+0000", 2024-09-16T00:08:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040586140000000003 2024-09-16T00:08:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620645+0000", 2024-09-16T00:08:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5784000000000001e-05 2024-09-16T00:08:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 46.181186697000001, 2024-09-16T00:08:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044836249999999998, 2024-09-16T00:08:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616518+0000", 2024-09-16T00:08:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3699999999999999e-06 2024-09-16T00:08:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:08:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1479999999999997e-06 2024-09-16T00:08:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616534+0000", 2024-09-16T00:08:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0641e-05 2024-09-16T00:08:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616541+0000", 2024-09-16T00:08:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3339999999999996e-06 2024-09-16T00:08:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617318+0000", 2024-09-16T00:08:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00077694799999999996 2024-09-16T00:08:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617409+0000", 2024-09-16T00:08:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0740000000000002e-05 2024-09-16T00:08:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620998+0000", 2024-09-16T00:08:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035894439999999998 2024-09-16T00:08:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 46.181138793999999, 2024-09-16T00:08:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041597320000000002, 2024-09-16T00:08:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:08:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616566+0000", 2024-09-16T00:08:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6979999999999998e-06 2024-09-16T00:08:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616574+0000", 2024-09-16T00:08:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7759999999999997e-06 2024-09-16T00:08:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616580+0000", 2024-09-16T00:08:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616800+0000", 2024-09-16T00:08:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022016700000000001 2024-09-16T00:08:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616904+0000", 2024-09-16T00:08:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104579 2024-09-16T00:08:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620722+0000", 2024-09-16T00:08:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003818133 2024-09-16T00:08:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 46.180888109000001, 2024-09-16T00:08:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038451570000000001, 2024-09-16T00:08:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616814+0000", 2024-09-16T00:08:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9299999999999997e-07 2024-09-16T00:08:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616824+0000", 2024-09-16T00:08:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6600000000000007e-06 2024-09-16T00:08:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616835+0000", 2024-09-16T00:08:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0713000000000001e-05 2024-09-16T00:08:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616841+0000", 2024-09-16T00:08:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.934e-06 2024-09-16T00:08:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616948+0000", 2024-09-16T00:08:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010631100000000001 2024-09-16T00:08:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616967+0000", 2024-09-16T00:08:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9598000000000001e-05 2024-09-16T00:08:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620619+0000", 2024-09-16T00:08:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036515810000000001 2024-09-16T00:08:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620659+0000", 2024-09-16T00:08:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9666999999999998e-05 2024-09-16T00:08:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 46.180841157000003, 2024-09-16T00:08:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038240129999999998, 2024-09-16T00:08:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616861+0000", 2024-09-16T00:08:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0200000000000003e-07 2024-09-16T00:08:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616864+0000", 2024-09-16T00:08:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5090000000000002e-06 2024-09-16T00:08:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616871+0000", 2024-09-16T00:08:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9249999999999998e-06 2024-09-16T00:08:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616877+0000", 2024-09-16T00:08:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.981e-06 2024-09-16T00:08:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617168+0000", 2024-09-16T00:08:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029035699999999998 2024-09-16T00:08:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617184+0000", 2024-09-16T00:08:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6005999999999999e-05 2024-09-16T00:08:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620661+0000", 2024-09-16T00:08:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034775330000000001 2024-09-16T00:08:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620684+0000", 2024-09-16T00:08:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3300000000000001e-05 2024-09-16T00:08:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 46.180385714000003, 2024-09-16T00:08:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045415610000000004, 2024-09-16T00:08:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8800000000000002e-07 2024-09-16T00:08:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617321+0000", 2024-09-16T00:08:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6979999999999997e-06 2024-09-16T00:08:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617330+0000", 2024-09-16T00:08:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6729999999999998e-06 2024-09-16T00:08:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617336+0000", 2024-09-16T00:08:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2110000000000003e-06 2024-09-16T00:08:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617394+0000", 2024-09-16T00:08:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8087999999999997e-05 2024-09-16T00:08:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617412+0000", 2024-09-16T00:08:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8099999999999999e-05 2024-09-16T00:08:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621818+0000", 2024-09-16T00:08:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044061209999999998 2024-09-16T00:08:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621857+0000", 2024-09-16T00:08:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9082000000000003e-05 2024-09-16T00:08:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.855 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:23 2.e e22/19 2:717a0223:::608.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:24.855 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:24.855 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 46.179489693000001, 2024-09-16T00:08:24.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036198279999999998, 2024-09-16T00:08:24.855 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.855 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.855 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.855 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:24.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:24.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618213+0000", 2024-09-16T00:08:24.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:08:24.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618218+0000", 2024-09-16T00:08:24.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4099999999999999e-06 2024-09-16T00:08:24.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618227+0000", 2024-09-16T00:08:24.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5009999999999993e-06 2024-09-16T00:08:24.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618234+0000", 2024-09-16T00:08:24.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:24.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618307+0000", 2024-09-16T00:08:24.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3421000000000001e-05 2024-09-16T00:08:24.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618328+0000", 2024-09-16T00:08:24.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0919000000000001e-05 2024-09-16T00:08:24.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621804+0000", 2024-09-16T00:08:24.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034756560000000001 2024-09-16T00:08:24.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621832+0000", 2024-09-16T00:08:24.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7966999999999998e-05 2024-09-16T00:08:24.862 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.862 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.862 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.863 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing_reply(2.1e epoch 22/19 rep_tid 0 lcod 22'1)", 2024-09-16T00:08:24.863 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:24.863 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 46.179044525000002, 2024-09-16T00:08:24.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1371000000000003e-05, 2024-09-16T00:08:24.864 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.864 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.864 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:24.864 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.864 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:24.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:24.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:08:24.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:08:24.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618662+0000", 2024-09-16T00:08:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1189999999999999e-06 2024-09-16T00:08:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618674+0000", 2024-09-16T00:08:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2317000000000001e-05 2024-09-16T00:08:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618681+0000", 2024-09-16T00:08:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7170000000000002e-06 2024-09-16T00:08:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:08:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6745000000000001e-05 2024-09-16T00:08:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618748+0000", 2024-09-16T00:08:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0794000000000001e-05 2024-09-16T00:08:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:24.868 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:24.869 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 46.178531190999998, 2024-09-16T00:08:24.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027183849999999998, 2024-09-16T00:08:24.869 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.869 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.869 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.869 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:24.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:24.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619171+0000", 2024-09-16T00:08:24.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8999999999999995e-07 2024-09-16T00:08:24.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619179+0000", 2024-09-16T00:08:24.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2679999999999994e-06 2024-09-16T00:08:24.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:08:24.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1246e-05 2024-09-16T00:08:24.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619197+0000", 2024-09-16T00:08:24.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2820000000000002e-06 2024-09-16T00:08:24.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619308+0000", 2024-09-16T00:08:24.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011141000000000001 2024-09-16T00:08:24.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.872 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619325+0000", 2024-09-16T00:08:24.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6469000000000001e-05 2024-09-16T00:08:24.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.872 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621860+0000", 2024-09-16T00:08:24.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025353590000000001 2024-09-16T00:08:24.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.872 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621889+0000", 2024-09-16T00:08:24.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8561e-05 2024-09-16T00:08:24.873 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.873 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.873 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.873 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 6 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:24.873 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:24.873 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 46.178488492, 2024-09-16T00:08:24.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026903809999999999, 2024-09-16T00:08:24.873 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.873 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.873 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:24.874 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:24.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:24.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:24.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2000000000000001e-07 2024-09-16T00:08:24.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619219+0000", 2024-09-16T00:08:24.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:08:24.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619234+0000", 2024-09-16T00:08:24.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5075000000000001e-05 2024-09-16T00:08:24.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619251+0000", 2024-09-16T00:08:24.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6863000000000001e-05 2024-09-16T00:08:24.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.876 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.876 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619556+0000", 2024-09-16T00:08:24.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030551599999999998 2024-09-16T00:08:24.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.876 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.876 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621903+0000", 2024-09-16T00:08:24.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023469979999999999 2024-09-16T00:08:24.876 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.876 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.877 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.877 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing_reply(2.1b epoch 22/19 rep_tid 1 lcod 22'1)", 2024-09-16T00:08:24.877 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:24.877 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 46.177410096000003, 2024-09-16T00:08:24.877 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6989000000000003e-05, 2024-09-16T00:08:24.877 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.877 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.877 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:24.877 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.877 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.877 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:24.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:24.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620292+0000", 2024-09-16T00:08:24.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.37e-07 2024-09-16T00:08:24.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620296+0000", 2024-09-16T00:08:24.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2420000000000002e-06 2024-09-16T00:08:24.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620308+0000", 2024-09-16T00:08:24.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1623e-05 2024-09-16T00:08:24.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620313+0000", 2024-09-16T00:08:24.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6290000000000001e-06 2024-09-16T00:08:24.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620359+0000", 2024-09-16T00:08:24.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6201e-05 2024-09-16T00:08:24.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620378+0000", 2024-09-16T00:08:24.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9556999999999999e-05 2024-09-16T00:08:24.880 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.880 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.880 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.880 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:38 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'6, mlcod=22'4)", 2024-09-16T00:08:24.880 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:24.881 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 46.174967846999998, 2024-09-16T00:08:24.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064551599999999995, 2024-09-16T00:08:24.881 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.881 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.881 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.881 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:24.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:24.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:24.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4399999999999999e-07 2024-09-16T00:08:24.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622739+0000", 2024-09-16T00:08:24.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0250000000000002e-06 2024-09-16T00:08:24.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622749+0000", 2024-09-16T00:08:24.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1220000000000001e-06 2024-09-16T00:08:24.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622760+0000", 2024-09-16T00:08:24.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1640999999999999e-05 2024-09-16T00:08:24.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622799+0000", 2024-09-16T00:08:24.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8537000000000001e-05 2024-09-16T00:08:24.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622810+0000", 2024-09-16T00:08:24.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1562e-05 2024-09-16T00:08:24.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.884 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.623356+0000", 2024-09-16T00:08:24.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054522299999999995 2024-09-16T00:08:24.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.884 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.623379+0000", 2024-09-16T00:08:24.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3662000000000001e-05 2024-09-16T00:08:24.884 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.884 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.885 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.885 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:40 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'7, mlcod=22'5)", 2024-09-16T00:08:24.885 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:24.885 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.957946945000003, 2024-09-16T00:08:24.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001017773, 2024-09-16T00:08:24.885 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.885 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.885 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.885 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:24.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:24.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839756+0000", 2024-09-16T00:08:24.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.491e-06 2024-09-16T00:08:24.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839768+0000", 2024-09-16T00:08:24.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1649e-05 2024-09-16T00:08:24.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839783+0000", 2024-09-16T00:08:24.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5023e-05 2024-09-16T00:08:24.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839791+0000", 2024-09-16T00:08:24.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5620000000000004e-06 2024-09-16T00:08:24.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839848+0000", 2024-09-16T00:08:24.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6563e-05 2024-09-16T00:08:24.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839868+0000", 2024-09-16T00:08:24.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9652000000000001e-05 2024-09-16T00:08:24.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.840729+0000", 2024-09-16T00:08:24.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086159500000000005 2024-09-16T00:08:24.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.840772+0000", 2024-09-16T00:08:24.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3238000000000002e-05 2024-09-16T00:08:24.889 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.889 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.889 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.889 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:41 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'8, mlcod=22'6)", 2024-09-16T00:08:24.889 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:24.889 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.684290034, 2024-09-16T00:08:24.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001040568, 2024-09-16T00:08:24.889 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:24.890 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:24.890 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:24.890 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:24.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:24.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:24.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:24.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:24.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:24.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:24.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113413+0000", 2024-09-16T00:08:24.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7409999999999999e-06 2024-09-16T00:08:24.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:24.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113428+0000", 2024-09-16T00:08:24.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5007999999999999e-05 2024-09-16T00:08:24.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:24.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113447+0000", 2024-09-16T00:08:24.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8731000000000001e-05 2024-09-16T00:08:24.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.892 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:24.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113457+0000", 2024-09-16T00:08:24.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0095000000000001e-05 2024-09-16T00:08:24.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.892 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:24.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113526+0000", 2024-09-16T00:08:24.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8552000000000004e-05 2024-09-16T00:08:24.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.892 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:24.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113552+0000", 2024-09-16T00:08:24.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6795999999999999e-05 2024-09-16T00:08:24.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:24.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.114417+0000", 2024-09-16T00:08:24.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086476299999999995 2024-09-16T00:08:24.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:24.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:24.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:24.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.114452+0000", 2024-09-16T00:08:24.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4882000000000002e-05 2024-09-16T00:08:24.893 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.893 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.893 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.894 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:24.894 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:24.894 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:24.895 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:08:24.896 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:24.894+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:24.996 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:24.995+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:25.009 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:25.009 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:25.009 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:25.018 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:08:25.103 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:25.102+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:25.163 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:25.164 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:25.172 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:08:25.198 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:25.197+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:25.248 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:08:25.298 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:25.298+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:25.317 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:25.318 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:25.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:25.318 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:25.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.318 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:25.318 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:25.318 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.701920453, 2024-09-16T00:08:25.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036770370000000002, 2024-09-16T00:08:25.318 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:25.318 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:25.318 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:25.318 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:25.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:25.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:25.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:25.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:25.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:25.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615154+0000", 2024-09-16T00:08:25.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1510000000000003e-06 2024-09-16T00:08:25.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:25.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615163+0000", 2024-09-16T00:08:25.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4490000000000005e-06 2024-09-16T00:08:25.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:25.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615182+0000", 2024-09-16T00:08:25.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8876e-05 2024-09-16T00:08:25.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:25.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615197+0000", 2024-09-16T00:08:25.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4671000000000001e-05 2024-09-16T00:08:25.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:25.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615349+0000", 2024-09-16T00:08:25.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015258700000000001 2024-09-16T00:08:25.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:25.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:08:25.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000204716 2024-09-16T00:08:25.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:25.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:08:25.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017250800000000001 2024-09-16T00:08:25.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:25.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616919+0000", 2024-09-16T00:08:25.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011921799999999999 2024-09-16T00:08:25.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:25.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:08:25.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018087769999999999 2024-09-16T00:08:25.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:25.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618774+0000", 2024-09-16T00:08:25.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6075999999999997e-05 2024-09-16T00:08:25.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:25.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618826+0000", 2024-09-16T00:08:25.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2046000000000003e-05 2024-09-16T00:08:25.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.324 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:25.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.324 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:25.324 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:25.324 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.700564665000002, 2024-09-16T00:08:25.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023501059999999998, 2024-09-16T00:08:25.324 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:25.324 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:25.324 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:25.324 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:25.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:25.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:25.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:25.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:25.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:25.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616506+0000", 2024-09-16T00:08:25.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4100000000000001e-06 2024-09-16T00:08:25.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:25.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617073+0000", 2024-09-16T00:08:25.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056725899999999997 2024-09-16T00:08:25.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:25.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617099+0000", 2024-09-16T00:08:25.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5860999999999999e-05 2024-09-16T00:08:25.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:25.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617114+0000", 2024-09-16T00:08:25.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5180999999999999e-05 2024-09-16T00:08:25.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:25.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617236+0000", 2024-09-16T00:08:25.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121662 2024-09-16T00:08:25.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:25.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:25.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001618733 2024-09-16T00:08:25.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.327 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:25.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.328 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:31 2.d e22/19 2:b50e409b:::mds_snaptable:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:25.328 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:25.328 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.699670517000001, 2024-09-16T00:08:25.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021851240000000001, 2024-09-16T00:08:25.328 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:25.328 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:25.328 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:25.328 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:25.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:25.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:25.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:25.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:25.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:25.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617400+0000", 2024-09-16T00:08:25.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.28e-06 2024-09-16T00:08:25.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:25.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617410+0000", 2024-09-16T00:08:25.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0179e-05 2024-09-16T00:08:25.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:25.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617435+0000", 2024-09-16T00:08:25.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5221000000000001e-05 2024-09-16T00:08:25.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:25.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617444+0000", 2024-09-16T00:08:25.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1989999999999995e-06 2024-09-16T00:08:25.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:25.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617514+0000", 2024-09-16T00:08:25.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9104999999999997e-05 2024-09-16T00:08:25.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:25.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617538+0000", 2024-09-16T00:08:25.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4383999999999999e-05 2024-09-16T00:08:25.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:25.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619536+0000", 2024-09-16T00:08:25.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019982789999999999 2024-09-16T00:08:25.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:25.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619584+0000", 2024-09-16T00:08:25.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7477e-05 2024-09-16T00:08:25.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.333 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:25.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.333 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:27 2.13 e22/19 2:cd5a64a3:::100.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:25.334 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:25.334 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.699365335000003, 2024-09-16T00:08:25.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018775459999999999, 2024-09-16T00:08:25.334 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:25.334 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:25.334 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:25.334 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:25.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:25.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:25.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:25.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:25.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:25.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617705+0000", 2024-09-16T00:08:25.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9900000000000009e-07 2024-09-16T00:08:25.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:25.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617718+0000", 2024-09-16T00:08:25.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3453e-05 2024-09-16T00:08:25.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:25.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617730+0000", 2024-09-16T00:08:25.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2221e-05 2024-09-16T00:08:25.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:25.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617739+0000", 2024-09-16T00:08:25.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0660000000000004e-06 2024-09-16T00:08:25.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:25.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617806+0000", 2024-09-16T00:08:25.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7397999999999994e-05 2024-09-16T00:08:25.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:25.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617823+0000", 2024-09-16T00:08:25.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7569999999999999e-05 2024-09-16T00:08:25.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:25.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619528+0000", 2024-09-16T00:08:25.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017045859999999999 2024-09-16T00:08:25.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:25.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619581+0000", 2024-09-16T00:08:25.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3253e-05 2024-09-16T00:08:25.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.338 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:25.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.338 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:28 2.13 e22/19 2:c91d4a1d:::mds0_inotable:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:25.338 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:25.338 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.699018064999997, 2024-09-16T00:08:25.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001558051, 2024-09-16T00:08:25.338 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:25.338 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:25.338 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:25.339 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:25.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:25.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:25.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:25.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:25.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:25.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618052+0000", 2024-09-16T00:08:25.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1399999999999997e-07 2024-09-16T00:08:25.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:25.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618065+0000", 2024-09-16T00:08:25.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3495e-05 2024-09-16T00:08:25.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:25.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:08:25.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0560999999999999e-05 2024-09-16T00:08:25.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:25.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618083+0000", 2024-09-16T00:08:25.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:08:25.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:25.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618147+0000", 2024-09-16T00:08:25.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4525999999999999e-05 2024-09-16T00:08:25.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:25.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618166+0000", 2024-09-16T00:08:25.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9045000000000001e-05 2024-09-16T00:08:25.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:25.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619585+0000", 2024-09-16T00:08:25.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014182660000000001 2024-09-16T00:08:25.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:25.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619609+0000", 2024-09-16T00:08:25.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4468e-05 2024-09-16T00:08:25.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.342 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:25.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.343 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:29 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:25.343 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:25.343 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.698667561000001, 2024-09-16T00:08:25.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012347219999999999, 2024-09-16T00:08:25.343 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:25.343 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:25.343 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:25.343 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:25.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:25.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:25.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:25.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:25.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:25.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:25.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3099999999999997e-07 2024-09-16T00:08:25.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:25.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618409+0000", 2024-09-16T00:08:25.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6239999999999996e-06 2024-09-16T00:08:25.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:25.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618420+0000", 2024-09-16T00:08:25.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1222e-05 2024-09-16T00:08:25.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:25.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618441+0000", 2024-09-16T00:08:25.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0564000000000002e-05 2024-09-16T00:08:25.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:25.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618519+0000", 2024-09-16T00:08:25.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7917999999999995e-05 2024-09-16T00:08:25.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:25.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618535+0000", 2024-09-16T00:08:25.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6266e-05 2024-09-16T00:08:25.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:25.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619612+0000", 2024-09-16T00:08:25.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010767039999999999 2024-09-16T00:08:25.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:25.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619636+0000", 2024-09-16T00:08:25.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4692999999999999e-05 2024-09-16T00:08:25.347 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.347 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:25.347 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.347 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:2 2.f e22/19 ondisk, result = 0)", 2024-09-16T00:08:25.347 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:25.347 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.698431534000001, 2024-09-16T00:08:25.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019782099999999999, 2024-09-16T00:08:25.347 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:25.347 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:25.347 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:25.348 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:25.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:25.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:25.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:25.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:25.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:25.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618639+0000", 2024-09-16T00:08:25.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5099999999999998e-07 2024-09-16T00:08:25.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:25.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618644+0000", 2024-09-16T00:08:25.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7440000000000004e-06 2024-09-16T00:08:25.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:25.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618655+0000", 2024-09-16T00:08:25.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0798999999999999e-05 2024-09-16T00:08:25.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:25.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618662+0000", 2024-09-16T00:08:25.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8700000000000003e-06 2024-09-16T00:08:25.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:25.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618717+0000", 2024-09-16T00:08:25.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5223999999999998e-05 2024-09-16T00:08:25.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:25.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618724+0000", 2024-09-16T00:08:25.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.046e-06 2024-09-16T00:08:25.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:25.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618835+0000", 2024-09-16T00:08:25.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111187 2024-09-16T00:08:25.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.351 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:25.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.351 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.b epoch 22/19 rep_tid 122 entries 22'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:25.351 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:25.351 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.697327141000002, 2024-09-16T00:08:25.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018692890000000001, 2024-09-16T00:08:25.351 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:25.352 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:25.352 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:25.352 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:25.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:25.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:25.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:25.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:25.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:25.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619743+0000", 2024-09-16T00:08:25.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1069999999999999e-06 2024-09-16T00:08:25.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:25.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619754+0000", 2024-09-16T00:08:25.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0789e-05 2024-09-16T00:08:25.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:25.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619777+0000", 2024-09-16T00:08:25.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2844e-05 2024-09-16T00:08:25.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:25.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619793+0000", 2024-09-16T00:08:25.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6231000000000001e-05 2024-09-16T00:08:25.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:25.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619875+0000", 2024-09-16T00:08:25.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2286000000000005e-05 2024-09-16T00:08:25.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:25.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621611+0000", 2024-09-16T00:08:25.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001736032 2024-09-16T00:08:25.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.355 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:25.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.355 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.6 38/37 [PushOp(2:654134d2:::mds0_openfiles.0:head, version: 26'3, data_included: [], data_size: 0, omap_header_size: 42, omap_entries_size: 2, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:654134d2:::mds0_openfiles.0:head@26'3, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:25.355 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.839473+0000", 2024-09-16T00:08:25.355 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.47759638100000001, 2024-09-16T00:08:25.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044119500000000002, 2024-09-16T00:08:25.355 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:25.355 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:25.355 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:25.355 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:25.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:25.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839473+0000", 2024-09-16T00:08:25.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:25.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839473+0000", 2024-09-16T00:08:25.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:25.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:25.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839474+0000", 2024-09-16T00:08:25.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1999999999999998e-07 2024-09-16T00:08:25.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:25.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839512+0000", 2024-09-16T00:08:25.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8546e-05 2024-09-16T00:08:25.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:25.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839530+0000", 2024-09-16T00:08:25.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7663e-05 2024-09-16T00:08:25.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:25.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839550+0000", 2024-09-16T00:08:25.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9817e-05 2024-09-16T00:08:25.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:25.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839658+0000", 2024-09-16T00:08:25.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108213 2024-09-16T00:08:25.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:25.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839672+0000", 2024-09-16T00:08:25.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4518000000000001e-05 2024-09-16T00:08:25.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:25.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:25.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:25.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839914+0000", 2024-09-16T00:08:25.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000241618 2024-09-16T00:08:25.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.359 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:25.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:25.359 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:25.359 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:25.360 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:08:25.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.361+0000 7f3bcc131640 1 Processor -- start 2024-09-16T00:08:25.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.361+0000 7f3bcc131640 1 -- start start 2024-09-16T00:08:25.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.362+0000 7f3bcc131640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3bc40c8cc0 0x7f3bc4071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.362+0000 7f3bcc131640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bc4071b50 0x7f3bc40c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.362+0000 7f3bcc131640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bc415a930 con 0x7f3bc4154980 2024-09-16T00:08:25.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.362+0000 7f3bcc131640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3bc415ac10 con 0x7f3bc40c8cc0 2024-09-16T00:08:25.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.362+0000 7f3bcc131640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc415aef0 con 0x7f3bc4071b50 2024-09-16T00:08:25.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.362+0000 7f3bc96cc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3bc40c8cc0 0x7f3bc4071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.362+0000 7f3bc96cc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3bc40c8cc0 0x7f3bc4071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50790/0 (socket says 172.21.15.139:50790) 2024-09-16T00:08:25.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.362+0000 7f3bc9ecd640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3bc4154980 0x7f3bc415e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:56354/0 (socket says 172.21.15.139:56354) 2024-09-16T00:08:25.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.363+0000 7f3bc96cc640 1 -- 172.21.15.139:0/2384192601 learned_addr learned my addr 172.21.15.139:0/2384192601 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:25.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.363+0000 7f3bca6ce640 1 --2- 172.21.15.139:0/2384192601 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bc4071b50 0x7f3bc40c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.363+0000 7f3bc96cc640 1 -- 172.21.15.139:0/2384192601 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bc4071b50 msgr2=0x7f3bc40c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:25.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.363+0000 7f3bc96cc640 1 --2- 172.21.15.139:0/2384192601 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bc4071b50 0x7f3bc40c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.363+0000 7f3bc96cc640 1 -- 172.21.15.139:0/2384192601 >> v1:172.21.15.164:6789/0 conn(0x7f3bc4154980 legacy=0x7f3bc415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.363+0000 7f3bc8ecb640 1 -- 172.21.15.139:0/2384192601 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 244228303 0 0) 0x7f3bc415a930 con 0x7f3bc4154980 2024-09-16T00:08:25.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.363+0000 7f3bc96cc640 1 -- 172.21.15.139:0/2384192601 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc415b1d0 con 0x7f3bc40c8cc0 2024-09-16T00:08:25.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.364+0000 7f3bc96cc640 1 --2- 172.21.15.139:0/2384192601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bc40c8cc0 0x7f3bc4071610 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009870 tx=0x7f3bb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0458585676e5908 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:25.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.364+0000 7f3bc8ecb640 1 -- 172.21.15.139:0/2384192601 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4037070 con 0x7f3bc40c8cc0 2024-09-16T00:08:25.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.364+0000 7f3bc8ecb640 1 -- 172.21.15.139:0/2384192601 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb4032440 con 0x7f3bc40c8cc0 2024-09-16T00:08:25.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.364+0000 7f3bc8ecb640 1 -- 172.21.15.139:0/2384192601 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4032810 con 0x7f3bc40c8cc0 2024-09-16T00:08:25.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.364+0000 7f3bcc131640 1 -- 172.21.15.139:0/2384192601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bc40c8cc0 msgr2=0x7f3bc4071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.364+0000 7f3bcc131640 1 --2- 172.21.15.139:0/2384192601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bc40c8cc0 0x7f3bc4071610 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009870 tx=0x7f3bb402a270 comp rx=0 tx=0).stop 2024-09-16T00:08:25.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.364+0000 7f3bcc131640 1 -- 172.21.15.139:0/2384192601 shutdown_connections 2024-09-16T00:08:25.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.364+0000 7f3bcc131640 1 --2- 172.21.15.139:0/2384192601 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bc4071b50 0x7f3bc40c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.364+0000 7f3bcc131640 1 --2- 172.21.15.139:0/2384192601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bc40c8cc0 0x7f3bc4071610 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.364+0000 7f3bcc131640 1 -- 172.21.15.139:0/2384192601 >> 172.21.15.139:0/2384192601 conn(0x7f3bc406fbe0 msgr2=0x7f3bc406fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:25.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.365+0000 7f3bcc131640 1 -- 172.21.15.139:0/2384192601 shutdown_connections 2024-09-16T00:08:25.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.365+0000 7f3bcc131640 1 -- 172.21.15.139:0/2384192601 wait complete. 2024-09-16T00:08:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.366+0000 7f3bcc131640 1 Processor -- start 2024-09-16T00:08:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.366+0000 7f3bcc131640 1 -- start start 2024-09-16T00:08:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.366+0000 7f3bcc131640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bc4071b50 0x7f3bc40cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.366+0000 7f3bcc131640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bc40c8cc0 0x7f3bc40cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.366+0000 7f3bcc131640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bc4154980 0x7f3bc4150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.366+0000 7f3bcc131640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc40cbd00 con 0x7f3bc4154980 2024-09-16T00:08:25.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.366+0000 7f3bcc131640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc40cbb80 con 0x7f3bc4071b50 2024-09-16T00:08:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.366+0000 7f3bcc131640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc4154d60 con 0x7f3bc40c8cc0 2024-09-16T00:08:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.367+0000 7f3bc9ecd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bc4071b50 0x7f3bc40cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.367+0000 7f3bc9ecd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bc4071b50 0x7f3bc40cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56194/0 (socket says 172.21.15.139:56194) 2024-09-16T00:08:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.367+0000 7f3bc9ecd640 1 -- 172.21.15.139:0/918628588 learned_addr learned my addr 172.21.15.139:0/918628588 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.367+0000 7f3bca6ce640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bc4154980 0x7f3bc4150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.367+0000 7f3bc96cc640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bc40c8cc0 0x7f3bc40cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.367+0000 7f3bc9ecd640 1 -- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bc40c8cc0 msgr2=0x7f3bc40cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.367+0000 7f3bc9ecd640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3bc40c8cc0 0x7f3bc40cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.367+0000 7f3bc9ecd640 1 -- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bc4154980 msgr2=0x7f3bc4150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.367+0000 7f3bc9ecd640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bc4154980 0x7f3bc4150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.367+0000 7f3bc9ecd640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc4159cf0 con 0x7f3bc4071b50 2024-09-16T00:08:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.367+0000 7f3bc9ecd640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bc4071b50 0x7f3bc40cd200 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000c490 tx=0x7f3bc000c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:25.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.367+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0018070 con 0x7f3bc4071b50 2024-09-16T00:08:25.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.367+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc40cc160 con 0x7f3bc4071b50 2024-09-16T00:08:25.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.367+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bc0007500 con 0x7f3bc4071b50 2024-09-16T00:08:25.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.367+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc40cc720 con 0x7f3bc4071b50 2024-09-16T00:08:25.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.367+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0004a20 con 0x7f3bc4071b50 2024-09-16T00:08:25.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.368+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f3bac005270 con 0x7f3bc4071b50 2024-09-16T00:08:25.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.369+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3bc001b070 con 0x7f3bc4071b50 2024-09-16T00:08:25.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.369+0000 7f3bba7fc640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3ba806c2e0 0x7f3ba806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.370+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5915+0+0 (secure 0 0 0) 0x7f3bc008a6e0 con 0x7f3bc4071b50 2024-09-16T00:08:25.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.370+0000 7f3bc96cc640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3ba806c2e0 0x7f3ba806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.370+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v38) ==== 54+0+16 (secure 0 0 0) 0x7f3bc0090170 con 0x7f3bc4071b50 2024-09-16T00:08:25.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.370+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f3bac000fa0 0x7f3bac0045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.370+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3bac005550 con 0x7f3bac000fa0 2024-09-16T00:08:25.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.370+0000 7f3bca6ce640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f3bac000fa0 0x7f3bac0045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.370+0000 7f3bc96cc640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3ba806c2e0 0x7f3ba806e7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4034000 tx=0x7f3bb402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:25.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.371+0000 7f3bca6ce640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f3bac000fa0 0x7f3bac0045e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:25.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.371+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3bac005550 con 0x7f3bac000fa0 2024-09-16T00:08:25.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.385+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3bac004fe0 con 0x7f3bac000fa0 2024-09-16T00:08:25.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.388+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3bac004fe0 con 0x7f3bac000fa0 2024-09-16T00:08:25.389 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:25.390 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:25.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.388+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f3bac001d00 0x7f3bac002990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.388+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f3bac00b390 con 0x7f3bac001d00 2024-09-16T00:08:25.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.388+0000 7f3bc9ecd640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f3bac001d00 0x7f3bac002990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.388+0000 7f3bc9ecd640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f3bac001d00 0x7f3bac002990 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:25.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.389+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3bac00b390 con 0x7f3bac001d00 2024-09-16T00:08:25.404 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:25.403+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:25.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.416+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3bac0031a0 con 0x7f3bac001d00 2024-09-16T00:08:25.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.418+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3bac0031a0 con 0x7f3bac001d00 2024-09-16T00:08:25.421 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:25.422 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:25.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.418+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f3bac003c20 0x7f3bac009c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.418+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f3bac00a3a0 con 0x7f3bac003c20 2024-09-16T00:08:25.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.419+0000 7f3bc96cc640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f3bac003c20 0x7f3bac009c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.419+0000 7f3bc96cc640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f3bac003c20 0x7f3bac009c80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:25.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.420+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3bac00a3a0 con 0x7f3bac003c20 2024-09-16T00:08:25.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.446+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3bac00a8b0 con 0x7f3bac003c20 2024-09-16T00:08:25.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.448+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3bac00a8b0 con 0x7f3bac003c20 2024-09-16T00:08:25.449 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:25.450 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:25.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.448+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f3bac00d550 0x7f3bac00f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.448+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f3bac012940 con 0x7f3bac00d550 2024-09-16T00:08:25.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.448+0000 7f3bca6ce640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f3bac00d550 0x7f3bac00f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.449+0000 7f3bca6ce640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f3bac00d550 0x7f3bac00f940 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:25.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.449+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3bac012940 con 0x7f3bac00d550 2024-09-16T00:08:25.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.463+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3bac011930 con 0x7f3bac00d550 2024-09-16T00:08:25.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.464+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3bac011930 con 0x7f3bac00d550 2024-09-16T00:08:25.465 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:25.466 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:25.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.464+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f3bac012050 0x7f3bac00ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.464+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f3bac0162f0 con 0x7f3bac012050 2024-09-16T00:08:25.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.465+0000 7f3bc9ecd640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f3bac012050 0x7f3bac00ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.465+0000 7f3bc9ecd640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f3bac012050 0x7f3bac00ff20 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:25.466 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:25.467 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:25.467 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:25.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.466+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3bac0162f0 con 0x7f3bac012050 2024-09-16T00:08:25.475 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:08:25.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.479+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3bac010460 con 0x7f3bac012050 2024-09-16T00:08:25.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.481+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3bac010460 con 0x7f3bac012050 2024-09-16T00:08:25.482 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:25.482 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:25.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.481+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f3bac010ae0 0x7f3bac010f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.481+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f3bac019500 con 0x7f3bac010ae0 2024-09-16T00:08:25.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.481+0000 7f3bc96cc640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f3bac010ae0 0x7f3bac010f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.481+0000 7f3bc96cc640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f3bac010ae0 0x7f3bac010f40 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:25.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.482+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3bac019500 con 0x7f3bac010ae0 2024-09-16T00:08:25.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.495+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3bac016c70 con 0x7f3bac010ae0 2024-09-16T00:08:25.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.497+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3bac016c70 con 0x7f3bac010ae0 2024-09-16T00:08:25.498 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:25.498 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:25.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.497+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f3bac0172f0 0x7f3bac017730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.497+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f3bac01c870 con 0x7f3bac0172f0 2024-09-16T00:08:25.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.497+0000 7f3bca6ce640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f3bac0172f0 0x7f3bac017730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.498+0000 7f3bca6ce640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f3bac0172f0 0x7f3bac017730 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:25.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.498+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3bac01c870 con 0x7f3bac0172f0 2024-09-16T00:08:25.516 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:25.515+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:25.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.524+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3bac017d80 con 0x7f3bac0172f0 2024-09-16T00:08:25.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.525+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3bac017d80 con 0x7f3bac0172f0 2024-09-16T00:08:25.527 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:25.527 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:25.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.526+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f3bac018400 0x7f3bac018860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.526+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f3bac018f00 con 0x7f3bac018400 2024-09-16T00:08:25.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.526+0000 7f3bc9ecd640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f3bac018400 0x7f3bac018860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.526+0000 7f3bc9ecd640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f3bac018400 0x7f3bac018860 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:25.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.527+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3bac018f00 con 0x7f3bac018400 2024-09-16T00:08:25.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.540+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f3bac000950 con 0x7f3bac018400 2024-09-16T00:08:25.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bba7fc640 1 -- 172.21.15.139:0/918628588 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3bac000950 con 0x7f3bac018400 2024-09-16T00:08:25.543 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:25.546 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:25.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f3bac000fa0 msgr2=0x7f3bac0045e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f3bac000fa0 0x7f3bac0045e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f3bac001d00 msgr2=0x7f3bac002990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f3bac001d00 0x7f3bac002990 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f3bac003c20 msgr2=0x7f3bac009c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f3bac003c20 0x7f3bac009c80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bca6ce640 1 -- 172.21.15.139:0/918628588 reap_dead start 2024-09-16T00:08:25.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f3bac00d550 msgr2=0x7f3bac00f940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f3bac00d550 0x7f3bac00f940 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f3bac012050 msgr2=0x7f3bac00ff20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f3bac012050 0x7f3bac00ff20 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f3bac010ae0 msgr2=0x7f3bac010f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f3bac010ae0 0x7f3bac010f40 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f3bac0172f0 msgr2=0x7f3bac017730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f3bac0172f0 0x7f3bac017730 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f3bac018400 msgr2=0x7f3bac018860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f3bac018400 0x7f3bac018860 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.542+0000 7f3bca6ce640 1 -- 172.21.15.139:0/918628588 reap_dead start 2024-09-16T00:08:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.543+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3ba806c2e0 msgr2=0x7f3ba806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.543+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3ba806c2e0 0x7f3ba806e7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4034000 tx=0x7f3bb402e000 comp rx=0 tx=0).stop 2024-09-16T00:08:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.543+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bc4071b50 msgr2=0x7f3bc40cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.543+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bc4071b50 0x7f3bc40cd200 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000c490 tx=0x7f3bc000c950 comp rx=0 tx=0).stop 2024-09-16T00:08:25.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.543+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 shutdown_connections 2024-09-16T00:08:25.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.543+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3ba806c2e0 0x7f3ba806e7a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.543+0000 7f3bcc131640 1 --2- 172.21.15.139:0/918628588 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3bc4071b50 0x7f3bc40cd200 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.543+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 >> 172.21.15.139:0/918628588 conn(0x7f3bc406fbe0 msgr2=0x7f3bc415b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:25.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.543+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 shutdown_connections 2024-09-16T00:08:25.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.543+0000 7f3bcc131640 1 -- 172.21.15.139:0/918628588 wait complete. 2024-09-16T00:08:25.600 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:25.599+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:25.621 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:25.621 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:25.630 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:08:25.714 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:25.713+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:25.775 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:25.775 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:25.783 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:08:25.801 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:25.801+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:25.853 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T00:08:25.903 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:25.902+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:25.928 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:25.928 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:25.937 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:08:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.952+0000 7f4c36465640 1 Processor -- start 2024-09-16T00:08:25.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.952+0000 7f4c36465640 1 -- start start 2024-09-16T00:08:25.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.953+0000 7f4c36465640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4c30154c20 0x7f4c3015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.953+0000 7f4c36465640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c300ccc90 0x7f4c300cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.953+0000 7f4c36465640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c300cc4b0 con 0x7f4c30068060 2024-09-16T00:08:25.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.953+0000 7f4c36465640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4c300c5830 con 0x7f4c30154c20 2024-09-16T00:08:25.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.953+0000 7f4c36465640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4c3015b7c0 con 0x7f4c300ccc90 2024-09-16T00:08:25.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.953+0000 7f4c2ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4c30154c20 0x7f4c3015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.953+0000 7f4c2ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4c30154c20 0x7f4c3015b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50800/0 (socket says 172.21.15.139:50800) 2024-09-16T00:08:25.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.953+0000 7f4c2ffff640 1 -- 172.21.15.139:0/1234183657 learned_addr learned my addr 172.21.15.139:0/1234183657 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:25.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.953+0000 7f4c34a02640 1 --2- 172.21.15.139:0/1234183657 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c300ccc90 0x7f4c300cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.954+0000 7f4c2ffff640 1 -- 172.21.15.139:0/1234183657 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c300ccc90 msgr2=0x7f4c300cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:25.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.954+0000 7f4c2ffff640 1 --2- 172.21.15.139:0/1234183657 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c300ccc90 0x7f4c300cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.954+0000 7f4c2effd640 1 -- 172.21.15.139:0/1234183657 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 415797816 0 0) 0x7f4c300cc4b0 con 0x7f4c30068060 2024-09-16T00:08:25.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.954+0000 7f4c2ffff640 1 -- 172.21.15.139:0/1234183657 >> v1:172.21.15.164:6789/0 conn(0x7f4c30068060 legacy=0x7f4c300c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.954+0000 7f4c2ffff640 1 -- 172.21.15.139:0/1234183657 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c300cc790 con 0x7f4c30154c20 2024-09-16T00:08:25.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.954+0000 7f4c2ffff640 1 --2- 172.21.15.139:0/1234183657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c30154c20 0x7f4c3015b280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009870 tx=0x7f4c2002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a37d51fe634c0dc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:25.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.955+0000 7f4c2effd640 1 -- 172.21.15.139:0/1234183657 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20037070 con 0x7f4c30154c20 2024-09-16T00:08:25.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.955+0000 7f4c2effd640 1 -- 172.21.15.139:0/1234183657 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c20032440 con 0x7f4c30154c20 2024-09-16T00:08:25.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.955+0000 7f4c2effd640 1 -- 172.21.15.139:0/1234183657 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20032810 con 0x7f4c30154c20 2024-09-16T00:08:25.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.955+0000 7f4c36465640 1 -- 172.21.15.139:0/1234183657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c30154c20 msgr2=0x7f4c3015b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.955+0000 7f4c36465640 1 --2- 172.21.15.139:0/1234183657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c30154c20 0x7f4c3015b280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009870 tx=0x7f4c2002a270 comp rx=0 tx=0).stop 2024-09-16T00:08:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.956+0000 7f4c36465640 1 -- 172.21.15.139:0/1234183657 shutdown_connections 2024-09-16T00:08:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.956+0000 7f4c36465640 1 --2- 172.21.15.139:0/1234183657 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c300ccc90 0x7f4c300cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.956+0000 7f4c36465640 1 --2- 172.21.15.139:0/1234183657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c30154c20 0x7f4c3015b280 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.956+0000 7f4c36465640 1 -- 172.21.15.139:0/1234183657 >> 172.21.15.139:0/1234183657 conn(0x7f4c3006ca40 msgr2=0x7f4c30067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.956+0000 7f4c36465640 1 -- 172.21.15.139:0/1234183657 shutdown_connections 2024-09-16T00:08:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.956+0000 7f4c36465640 1 -- 172.21.15.139:0/1234183657 wait complete. 2024-09-16T00:08:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.957+0000 7f4c36465640 1 Processor -- start 2024-09-16T00:08:25.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.958+0000 7f4c36465640 1 -- start start 2024-09-16T00:08:25.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.959+0000 7f4c36465640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c30068060 0x7f4c300c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.959+0000 7f4c36465640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c300ccc90 0x7f4c300b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.959+0000 7f4c36465640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c30154c20 0x7f4c301f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.959+0000 7f4c36465640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3015e0d0 con 0x7f4c300ccc90 2024-09-16T00:08:25.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.959+0000 7f4c36465640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c300cc260 con 0x7f4c30154c20 2024-09-16T00:08:25.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.959+0000 7f4c36465640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4c3015e3b0 con 0x7f4c30068060 2024-09-16T00:08:25.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.959+0000 7f4c2f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c300ccc90 0x7f4c300b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.959+0000 7f4c2f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c300ccc90 0x7f4c300b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50804/0 (socket says 172.21.15.139:50804) 2024-09-16T00:08:25.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.959+0000 7f4c2ffff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c30068060 0x7f4c300c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.959+0000 7f4c2f7fe640 1 -- 172.21.15.139:0/832508009 learned_addr learned my addr 172.21.15.139:0/832508009 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:25.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.959+0000 7f4c34a02640 1 --2- 172.21.15.139:0/832508009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c30154c20 0x7f4c301f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.960+0000 7f4c2ffff640 1 -- 172.21.15.139:0/832508009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c30154c20 msgr2=0x7f4c301f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.960+0000 7f4c2ffff640 1 --2- 172.21.15.139:0/832508009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c30154c20 0x7f4c301f2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.960+0000 7f4c2ffff640 1 -- 172.21.15.139:0/832508009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c300ccc90 msgr2=0x7f4c300b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:25.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.960+0000 7f4c2ffff640 1 --2- 172.21.15.139:0/832508009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c300ccc90 0x7f4c300b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:25.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.960+0000 7f4c2ffff640 1 -- 172.21.15.139:0/832508009 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c301f23a0 con 0x7f4c30068060 2024-09-16T00:08:25.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.960+0000 7f4c2ffff640 1 --2- 172.21.15.139:0/832508009 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c30068060 0x7f4c300c9ed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4c20031000 tx=0x7f4c200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:25.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.960+0000 7f4c2cff9640 1 -- 172.21.15.139:0/832508009 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20041020 con 0x7f4c30068060 2024-09-16T00:08:25.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.960+0000 7f4c2cff9640 1 -- 172.21.15.139:0/832508009 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c20030660 con 0x7f4c30068060 2024-09-16T00:08:25.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.960+0000 7f4c36465640 1 -- 172.21.15.139:0/832508009 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3015c620 con 0x7f4c30068060 2024-09-16T00:08:25.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.960+0000 7f4c2cff9640 1 -- 172.21.15.139:0/832508009 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c20037040 con 0x7f4c30068060 2024-09-16T00:08:25.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.960+0000 7f4c36465640 1 -- 172.21.15.139:0/832508009 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c301f2660 con 0x7f4c30068060 2024-09-16T00:08:25.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.962+0000 7f4c36465640 1 -- 172.21.15.139:0/832508009 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c300cc4b0 con 0x7f4c30068060 2024-09-16T00:08:25.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.962+0000 7f4c2cff9640 1 -- 172.21.15.139:0/832508009 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4c20044020 con 0x7f4c30068060 2024-09-16T00:08:25.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.962+0000 7f4c2cff9640 1 --2- 172.21.15.139:0/832508009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c1c06c2e0 0x7f4c1c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:25.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.963+0000 7f4c2f7fe640 1 --2- 172.21.15.139:0/832508009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c1c06c2e0 0x7f4c1c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:25.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.963+0000 7f4c2cff9640 1 -- 172.21.15.139:0/832508009 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(38..38 src has 1..38) ==== 5915+0+0 (secure 0 0 0) 0x7f4c200ab980 con 0x7f4c30068060 2024-09-16T00:08:25.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.963+0000 7f4c2f7fe640 1 --2- 172.21.15.139:0/832508009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c1c06c2e0 0x7f4c1c06e7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4c301eeac0 tx=0x7f4c24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:25.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:25.967+0000 7f4c2cff9640 1 -- 172.21.15.139:0/832508009 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c300cc4b0 con 0x7f4c30068060 2024-09-16T00:08:26.004 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:26.003+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:26.082 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:26.082 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:26.091 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:08:26.105 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:26.104+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:26.194 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:26.195 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:26.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:26.201+0000 7f4c36465640 1 -- 172.21.15.139:0/832508009 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f4c3015e690 con 0x7f4c30068060 2024-09-16T00:08:26.204 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:08:26.206 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:26.205+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:26.307 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:26.306+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:26.307 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:26.308 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:26.316 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:08:26.408 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:26.407+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:26.421 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:26.421 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:26.430 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:08:26.509 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:26.508+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:26.573 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:26.574 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:26.582 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:08:26.610 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:26.608+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:26.710 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:26.710+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:26.733 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:26.733 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:26.745 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:08:26.812 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:26.811+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:26.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:26.834+0000 7f4c2cff9640 1 -- 172.21.15.139:0/832508009 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v39) ==== 101+0+0 (secure 0 0 0) 0x7f4c2004b030 con 0x7f4c30068060 2024-09-16T00:08:26.837 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is unset 2024-09-16T00:08:26.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:26.838+0000 7f4c36465640 1 -- 172.21.15.139:0/832508009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c1c06c2e0 msgr2=0x7f4c1c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:26.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:26.838+0000 7f4c36465640 1 --2- 172.21.15.139:0/832508009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c1c06c2e0 0x7f4c1c06e7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4c301eeac0 tx=0x7f4c24009040 comp rx=0 tx=0).stop 2024-09-16T00:08:26.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:26.838+0000 7f4c36465640 1 -- 172.21.15.139:0/832508009 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c30068060 msgr2=0x7f4c300c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:26.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:26.838+0000 7f4c36465640 1 --2- 172.21.15.139:0/832508009 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c30068060 0x7f4c300c9ed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4c20031000 tx=0x7f4c200046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:26.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:26.838+0000 7f4c36465640 1 -- 172.21.15.139:0/832508009 shutdown_connections 2024-09-16T00:08:26.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:26.838+0000 7f4c36465640 1 --2- 172.21.15.139:0/832508009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c1c06c2e0 0x7f4c1c06e7a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:26.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:26.838+0000 7f4c36465640 1 --2- 172.21.15.139:0/832508009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c30154c20 0x7f4c301f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:26.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:26.838+0000 7f4c36465640 1 --2- 172.21.15.139:0/832508009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c300ccc90 0x7f4c300b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:26.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:26.838+0000 7f4c36465640 1 --2- 172.21.15.139:0/832508009 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c30068060 0x7f4c300c9ed0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:26.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:26.838+0000 7f4c36465640 1 -- 172.21.15.139:0/832508009 >> 172.21.15.139:0/832508009 conn(0x7f4c3006ca40 msgr2=0x7f4c3014f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:26.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:26.838+0000 7f4c36465640 1 -- 172.21.15.139:0/832508009 shutdown_connections 2024-09-16T00:08:26.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:26.838+0000 7f4c36465640 1 -- 172.21.15.139:0/832508009 wait complete. 2024-09-16T00:08:26.913 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:26.912+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:26.915 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:26.915 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:26.927 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:08:27.014 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:27.013+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:27.038 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:27.038 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:27.047 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:08:27.114 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:27.114+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:27.202 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:27.203 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:27.215 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:08:27.216 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:27.214+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:27.316 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:27.315+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:27.414 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:27.414 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:27.417 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:27.415+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:27.426 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:08:27.518 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:27.517+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:27.531 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:27.531 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:27.541 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:08:27.555 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:08:27.620 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:27.619+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:27.650 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:27.650 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:27.660 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:08:27.721 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:27.720+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:27.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.729+0000 7f51b2c0e640 1 Processor -- start 2024-09-16T00:08:27.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.729+0000 7f51b2c0e640 1 -- start start 2024-09-16T00:08:27.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.730+0000 7f51b2c0e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51ac0c8600 0x7f51ac0c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:27.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.730+0000 7f51b2c0e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f51ac0c8c10 0x7f51ac0c9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:27.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.730+0000 7f51b2c0e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51ac0c1320 con 0x7f51ac151150 2024-09-16T00:08:27.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.730+0000 7f51b2c0e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f51ac0c1ea0 con 0x7f51ac0c8c10 2024-09-16T00:08:27.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.730+0000 7f51b2c0e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f51ac0c1600 con 0x7f51ac0c8600 2024-09-16T00:08:27.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.731+0000 7f51b11ab640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f51ac0c8c10 0x7f51ac0c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:27.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.731+0000 7f51b11ab640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f51ac0c8c10 0x7f51ac0c9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43032/0 (socket says 172.21.15.139:43032) 2024-09-16T00:08:27.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.731+0000 7f51b11ab640 1 -- 172.21.15.139:0/976698639 learned_addr learned my addr 172.21.15.139:0/976698639 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:27.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.731+0000 7f51abfff640 1 --2- 172.21.15.139:0/976698639 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51ac0c8600 0x7f51ac0c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:27.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.731+0000 7f51b11ab640 1 -- 172.21.15.139:0/976698639 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51ac0c8600 msgr2=0x7f51ac0c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:27.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.731+0000 7f51b11ab640 1 --2- 172.21.15.139:0/976698639 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51ac0c8600 0x7f51ac0c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:27.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.731+0000 7f51b11ab640 1 -- 172.21.15.139:0/976698639 >> v1:172.21.15.164:6789/0 conn(0x7f51ac151150 legacy=0x7f51ac157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:27.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.731+0000 7f51b11ab640 1 -- 172.21.15.139:0/976698639 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ac0c18e0 con 0x7f51ac0c8c10 2024-09-16T00:08:27.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.732+0000 7f51b11ab640 1 --2- 172.21.15.139:0/976698639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51ac0c8c10 0x7f51ac0c9080 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f51a400bef0 tx=0x7f51a400a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5d787031a87cec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:27.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.732+0000 7f51ab7fe640 1 -- 172.21.15.139:0/976698639 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4038070 con 0x7f51ac0c8c10 2024-09-16T00:08:27.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.732+0000 7f51ab7fe640 1 -- 172.21.15.139:0/976698639 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a4030480 con 0x7f51ac0c8c10 2024-09-16T00:08:27.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.732+0000 7f51ab7fe640 1 -- 172.21.15.139:0/976698639 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4030870 con 0x7f51ac0c8c10 2024-09-16T00:08:27.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.733+0000 7f51b2c0e640 1 -- 172.21.15.139:0/976698639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51ac0c8c10 msgr2=0x7f51ac0c9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:27.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.733+0000 7f51b2c0e640 1 --2- 172.21.15.139:0/976698639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51ac0c8c10 0x7f51ac0c9080 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f51a400bef0 tx=0x7f51a400a4c0 comp rx=0 tx=0).stop 2024-09-16T00:08:27.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.734+0000 7f51b2c0e640 1 -- 172.21.15.139:0/976698639 shutdown_connections 2024-09-16T00:08:27.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.734+0000 7f51b2c0e640 1 --2- 172.21.15.139:0/976698639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51ac0c8c10 0x7f51ac0c9080 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:27.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.734+0000 7f51b2c0e640 1 --2- 172.21.15.139:0/976698639 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51ac0c8600 0x7f51ac0c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:27.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.734+0000 7f51b2c0e640 1 -- 172.21.15.139:0/976698639 >> 172.21.15.139:0/976698639 conn(0x7f51ac06e960 msgr2=0x7f51ac06f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:27.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.734+0000 7f51b2c0e640 1 -- 172.21.15.139:0/976698639 shutdown_connections 2024-09-16T00:08:27.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.734+0000 7f51b2c0e640 1 -- 172.21.15.139:0/976698639 wait complete. 2024-09-16T00:08:27.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.735+0000 7f51b2c0e640 1 Processor -- start 2024-09-16T00:08:27.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.736+0000 7f51b2c0e640 1 -- start start 2024-09-16T00:08:27.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.737+0000 7f51b2c0e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51ac0c8600 0x7f51ac147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:27.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.737+0000 7f51b2c0e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51ac0c8c10 0x7f51ac1efdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:27.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.737+0000 7f51b2c0e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51ac151150 0x7f51ac153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:27.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.737+0000 7f51b2c0e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac151530 con 0x7f51ac0c8600 2024-09-16T00:08:27.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.737+0000 7f51b2c0e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f51ac157000 con 0x7f51ac151150 2024-09-16T00:08:27.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.737+0000 7f51b2c0e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f51ac157c30 con 0x7f51ac0c8c10 2024-09-16T00:08:27.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.737+0000 7f51b09aa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51ac0c8600 0x7f51ac147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.737+0000 7f51b09aa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51ac0c8600 0x7f51ac147790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43042/0 (socket says 172.21.15.139:43042) 2024-09-16T00:08:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.737+0000 7f51b09aa640 1 -- 172.21.15.139:0/3916614935 learned_addr learned my addr 172.21.15.139:0/3916614935 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.737+0000 7f51abfff640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51ac0c8c10 0x7f51ac1efdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.737+0000 7f51b11ab640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51ac151150 0x7f51ac153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.738+0000 7f51b09aa640 1 -- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51ac0c8c10 msgr2=0x7f51ac1efdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.738+0000 7f51b09aa640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51ac0c8c10 0x7f51ac1efdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:27.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.738+0000 7f51b09aa640 1 -- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51ac151150 msgr2=0x7f51ac153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:27.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.738+0000 7f51b09aa640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51ac151150 0x7f51ac153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:27.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.738+0000 7f51b09aa640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ac148050 con 0x7f51ac0c8600 2024-09-16T00:08:27.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.738+0000 7f51b09aa640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51ac0c8600 0x7f51ac147790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f51a000c3c0 tx=0x7f51a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:27.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.738+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a00052d0 con 0x7f51ac0c8600 2024-09-16T00:08:27.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.738+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a0005490 con 0x7f51ac0c8600 2024-09-16T00:08:27.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.738+0000 7f51b2c0e640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ac153580 con 0x7f51ac0c8600 2024-09-16T00:08:27.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.738+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0007260 con 0x7f51ac0c8600 2024-09-16T00:08:27.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.738+0000 7f51b2c0e640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ac1485b0 con 0x7f51ac0c8600 2024-09-16T00:08:27.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.740+0000 7f51b2c0e640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f51ac150690 con 0x7f51ac0c8600 2024-09-16T00:08:27.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.741+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f51a0007400 con 0x7f51ac0c8600 2024-09-16T00:08:27.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.741+0000 7f51a97fa640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f519006c2b0 0x7f519006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:27.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.742+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5915+0+0 (secure 0 0 0) 0x7f51a008a9b0 con 0x7f51ac0c8600 2024-09-16T00:08:27.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.742+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v39) ==== 54+0+16 (secure 0 0 0) 0x7f51ac150690 con 0x7f51ac0c8600 2024-09-16T00:08:27.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.742+0000 7f51abfff640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f519006c2b0 0x7f519006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:27.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.742+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5184001650 0x7f5184003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:27.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.742+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5184006bb0 con 0x7f5184001650 2024-09-16T00:08:27.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.742+0000 7f51b11ab640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5184001650 0x7f5184003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:27.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.743+0000 7f51abfff640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f519006c2b0 0x7f519006e770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f519800d440 tx=0x7f5198018040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:27.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.743+0000 7f51b11ab640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5184001650 0x7f5184003b10 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:27.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.744+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5184006bb0 con 0x7f5184001650 2024-09-16T00:08:27.770 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:27.770 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:27.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.772+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5184005c00 con 0x7f5184001650 2024-09-16T00:08:27.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.775+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5184005c00 con 0x7f5184001650 2024-09-16T00:08:27.776 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:27.776 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:27.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.775+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5184004190 0x7f5184004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:27.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.775+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f5184004b80 con 0x7f5184004190 2024-09-16T00:08:27.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.775+0000 7f51b09aa640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5184004190 0x7f5184004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:27.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.776+0000 7f51b09aa640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5184004190 0x7f5184004640 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:27.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.777+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5184004b80 con 0x7f5184004190 2024-09-16T00:08:27.779 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:08:27.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.805+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5184005090 con 0x7f5184004190 2024-09-16T00:08:27.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.807+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5184005090 con 0x7f5184004190 2024-09-16T00:08:27.808 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:27.809 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:27.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.808+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5184009340 0x7f518400b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:27.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.808+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f518400e5b0 con 0x7f5184009340 2024-09-16T00:08:27.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.808+0000 7f51abfff640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5184009340 0x7f518400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:27.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.808+0000 7f51abfff640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5184009340 0x7f518400b730 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:27.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.809+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f518400e5b0 con 0x7f5184009340 2024-09-16T00:08:27.822 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:27.821+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:27.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.836+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f518400c000 con 0x7f5184009340 2024-09-16T00:08:27.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.839+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f518400c000 con 0x7f5184009340 2024-09-16T00:08:27.840 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:27.840 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:27.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.839+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f518400bf80 0x7f518400c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:27.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.839+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f518400c920 con 0x7f518400bf80 2024-09-16T00:08:27.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.839+0000 7f51b11ab640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f518400bf80 0x7f518400c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:27.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.840+0000 7f51b11ab640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f518400bf80 0x7f518400c3e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:27.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.841+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f518400c920 con 0x7f518400bf80 2024-09-16T00:08:27.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.868+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f518400ce30 con 0x7f518400bf80 2024-09-16T00:08:27.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.870+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f518400ce30 con 0x7f518400bf80 2024-09-16T00:08:27.874 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:27.874 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:27.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.871+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f518400d1f0 0x7f5184012810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:27.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.871+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f51840155e0 con 0x7f518400d1f0 2024-09-16T00:08:27.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.871+0000 7f51b09aa640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f518400d1f0 0x7f5184012810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:27.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.871+0000 7f51b09aa640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f518400d1f0 0x7f5184012810 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:27.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.872+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f51840155e0 con 0x7f518400d1f0 2024-09-16T00:08:27.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.898+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5184012d50 con 0x7f518400d1f0 2024-09-16T00:08:27.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.900+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5184012d50 con 0x7f518400d1f0 2024-09-16T00:08:27.901 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:27.902 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:27.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.901+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5184014bd0 0x7f5184015030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:27.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.901+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f5184018950 con 0x7f5184014bd0 2024-09-16T00:08:27.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.901+0000 7f51abfff640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5184014bd0 0x7f5184015030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:27.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.901+0000 7f51abfff640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5184014bd0 0x7f5184015030 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:27.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.902+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5184018950 con 0x7f5184014bd0 2024-09-16T00:08:27.911 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:27.911 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:27.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:27.911 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:27.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.911 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:27.911 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:27.911 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.293223273000002, 2024-09-16T00:08:27.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038263220000000001, 2024-09-16T00:08:27.911 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:27.911 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:27.912 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:27.912 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:27.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:27.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:27.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:27.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:27.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:27.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615465+0000", 2024-09-16T00:08:27.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2769999999999999e-06 2024-09-16T00:08:27.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:27.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615471+0000", 2024-09-16T00:08:27.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5729999999999996e-06 2024-09-16T00:08:27.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:27.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615483+0000", 2024-09-16T00:08:27.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1838000000000001e-05 2024-09-16T00:08:27.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:27.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615492+0000", 2024-09-16T00:08:27.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2410000000000001e-06 2024-09-16T00:08:27.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:27.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615596+0000", 2024-09-16T00:08:27.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104097 2024-09-16T00:08:27.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:27.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615688+0000", 2024-09-16T00:08:27.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1225999999999997e-05 2024-09-16T00:08:27.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.915 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:27.915 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615804+0000", 2024-09-16T00:08:27.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000116601 2024-09-16T00:08:27.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.915 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:27.915 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618165+0000", 2024-09-16T00:08:27.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023609429999999999 2024-09-16T00:08:27.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:27.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:08:27.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010258509999999999 2024-09-16T00:08:27.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:27.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619243+0000", 2024-09-16T00:08:27.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2327999999999998e-05 2024-09-16T00:08:27.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:27.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619287+0000", 2024-09-16T00:08:27.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3346999999999997e-05 2024-09-16T00:08:27.917 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.917 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:27.917 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.917 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:27.917 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:27.917 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.293168197, 2024-09-16T00:08:27.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0054735510000000001, 2024-09-16T00:08:27.917 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:27.917 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:27.917 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:27.917 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:27.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:27.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:27.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:27.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:27.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:27.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615516+0000", 2024-09-16T00:08:27.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6899999999999998e-07 2024-09-16T00:08:27.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:27.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615519+0000", 2024-09-16T00:08:27.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1760000000000002e-06 2024-09-16T00:08:27.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:27.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615526+0000", 2024-09-16T00:08:27.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8399999999999997e-06 2024-09-16T00:08:27.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:27.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615531+0000", 2024-09-16T00:08:27.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4450000000000004e-06 2024-09-16T00:08:27.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:27.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:27.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054127099999999996 2024-09-16T00:08:27.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:27.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616140+0000", 2024-09-16T00:08:27.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7230999999999994e-05 2024-09-16T00:08:27.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.921 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:27.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616249+0000", 2024-09-16T00:08:27.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109122 2024-09-16T00:08:27.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.921 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:27.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618646+0000", 2024-09-16T00:08:27.922 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023972279999999999 2024-09-16T00:08:27.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.922 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:27.922 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:08:27.922 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022396220000000001 2024-09-16T00:08:27.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.922 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:27.922 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620939+0000", 2024-09-16T00:08:27.922 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2735999999999998e-05 2024-09-16T00:08:27.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.923 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:27.923 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620989+0000", 2024-09-16T00:08:27.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0411000000000002e-05 2024-09-16T00:08:27.923 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.923 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:27.923 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.924 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:27.924 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:27.924 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.293142717999999, 2024-09-16T00:08:27.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055872789999999997, 2024-09-16T00:08:27.924 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:27.924 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:27.924 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:27.924 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:27.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:27.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:27.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:27.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:27.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:27.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:27.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.01e-07 2024-09-16T00:08:27.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:27.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615547+0000", 2024-09-16T00:08:27.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7899999999999996e-06 2024-09-16T00:08:27.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:27.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:08:27.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1119999999999998e-06 2024-09-16T00:08:27.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:27.927 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615560+0000", 2024-09-16T00:08:27.927 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:08:27.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.927 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:27.927 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616424+0000", 2024-09-16T00:08:27.927 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086350800000000003 2024-09-16T00:08:27.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.927 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:27.927 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:08:27.927 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9675999999999999e-05 2024-09-16T00:08:27.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:27.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616621+0000", 2024-09-16T00:08:27.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7345999999999999e-05 2024-09-16T00:08:27.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:27.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618654+0000", 2024-09-16T00:08:27.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020335269999999998 2024-09-16T00:08:27.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:27.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621055+0000", 2024-09-16T00:08:27.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024009729999999998 2024-09-16T00:08:27.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:27.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621100+0000", 2024-09-16T00:08:27.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5402999999999999e-05 2024-09-16T00:08:27.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:27.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621128+0000", 2024-09-16T00:08:27.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7736e-05 2024-09-16T00:08:27.930 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.930 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:27.930 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.930 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:27.930 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:27.930 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.293117068000001, 2024-09-16T00:08:27.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057024509999999999, 2024-09-16T00:08:27.931 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:27.931 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:27.931 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:27.931 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:27.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:27.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:27.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:27.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:27.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:27.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:27.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6300000000000001e-07 2024-09-16T00:08:27.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:27.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615569+0000", 2024-09-16T00:08:27.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3709999999999998e-06 2024-09-16T00:08:27.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:27.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615580+0000", 2024-09-16T00:08:27.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0448000000000001e-05 2024-09-16T00:08:27.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:27.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615584+0000", 2024-09-16T00:08:27.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6020000000000002e-06 2024-09-16T00:08:27.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:27.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616850+0000", 2024-09-16T00:08:27.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012661110000000001 2024-09-16T00:08:27.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.934 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:27.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616915+0000", 2024-09-16T00:08:27.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4820000000000006e-05 2024-09-16T00:08:27.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.934 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:27.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617031+0000", 2024-09-16T00:08:27.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115258 2024-09-16T00:08:27.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:27.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618659+0000", 2024-09-16T00:08:27.935 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001627975 2024-09-16T00:08:27.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:27.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621179+0000", 2024-09-16T00:08:27.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025203410000000002 2024-09-16T00:08:27.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.936 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:27.936 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621238+0000", 2024-09-16T00:08:27.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9323000000000001e-05 2024-09-16T00:08:27.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.936 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:27.936 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621269+0000", 2024-09-16T00:08:27.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0839000000000001e-05 2024-09-16T00:08:27.937 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.937 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:27.937 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.937 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:27.937 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:27.937 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.293093319, 2024-09-16T00:08:27.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058012769999999996, 2024-09-16T00:08:27.938 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:27.938 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:27.938 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:27.938 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:27.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:27.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:27.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:27.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:27.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:27.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615591+0000", 2024-09-16T00:08:27.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6199999999999999e-07 2024-09-16T00:08:27.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:27.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615594+0000", 2024-09-16T00:08:27.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8739999999999999e-06 2024-09-16T00:08:27.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:27.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615599+0000", 2024-09-16T00:08:27.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:08:27.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:27.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615603+0000", 2024-09-16T00:08:27.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8489999999999999e-06 2024-09-16T00:08:27.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:27.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617235+0000", 2024-09-16T00:08:27.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001632037 2024-09-16T00:08:27.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:27.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617348+0000", 2024-09-16T00:08:27.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113387 2024-09-16T00:08:27.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:27.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617458+0000", 2024-09-16T00:08:27.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109765 2024-09-16T00:08:27.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:27.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618664+0000", 2024-09-16T00:08:27.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012059950000000001 2024-09-16T00:08:27.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:27.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621311+0000", 2024-09-16T00:08:27.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026468469999999999 2024-09-16T00:08:27.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:27.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621357+0000", 2024-09-16T00:08:27.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6360999999999999e-05 2024-09-16T00:08:27.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:27.943 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621392+0000", 2024-09-16T00:08:27.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4190999999999998e-05 2024-09-16T00:08:27.943 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.943 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:27.943 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.943 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:27.943 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:27.943 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.293074695000001, 2024-09-16T00:08:27.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058737279999999999, 2024-09-16T00:08:27.943 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:27.943 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:27.944 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:27.944 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:27.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:27.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:27.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:27.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:27.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:27.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:27.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3799999999999999e-07 2024-09-16T00:08:27.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:27.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615611+0000", 2024-09-16T00:08:27.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-06 2024-09-16T00:08:27.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:27.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615626+0000", 2024-09-16T00:08:27.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:08:27.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:27.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615634+0000", 2024-09-16T00:08:27.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-06 2024-09-16T00:08:27.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:27.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617692+0000", 2024-09-16T00:08:27.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002057718 2024-09-16T00:08:27.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:27.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617768+0000", 2024-09-16T00:08:27.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6878999999999994e-05 2024-09-16T00:08:27.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.947 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:27.947 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:27.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019501699999999999 2024-09-16T00:08:27.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.947 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:27.947 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:08:27.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034357200000000002 2024-09-16T00:08:27.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.947 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:27.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621429+0000", 2024-09-16T00:08:27.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0012999999999999e-05 2024-09-16T00:08:27.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.948 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:27.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621459+0000", 2024-09-16T00:08:27.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9498000000000001e-05 2024-09-16T00:08:27.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.948 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:27.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621483+0000", 2024-09-16T00:08:27.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3957000000000001e-05 2024-09-16T00:08:27.948 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.949 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:27.949 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.949 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.949 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:27.949 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:27.949 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.293043415, 2024-09-16T00:08:27.949 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059371789999999999, 2024-09-16T00:08:27.949 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:27.949 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:27.949 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:27.949 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:27.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.949 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:27.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:27.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:27.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:27.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:27.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:08:27.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4000000000000002e-07 2024-09-16T00:08:27.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:27.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615643+0000", 2024-09-16T00:08:27.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6860000000000002e-06 2024-09-16T00:08:27.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:27.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615649+0000", 2024-09-16T00:08:27.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7610000000000004e-06 2024-09-16T00:08:27.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:27.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615653+0000", 2024-09-16T00:08:27.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0380000000000001e-06 2024-09-16T00:08:27.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.952 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:27.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618197+0000", 2024-09-16T00:08:27.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025440079999999999 2024-09-16T00:08:27.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.952 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:27.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618312+0000", 2024-09-16T00:08:27.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115083 2024-09-16T00:08:27.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.952 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:27.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618412+0000", 2024-09-16T00:08:27.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010016799999999999 2024-09-16T00:08:27.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.953 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.953 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:27.953 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621518+0000", 2024-09-16T00:08:27.953 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031058679999999999 2024-09-16T00:08:27.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.953 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.953 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:27.953 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621526+0000", 2024-09-16T00:08:27.953 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2490000000000004e-06 2024-09-16T00:08:27.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.953 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.953 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:27.953 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621549+0000", 2024-09-16T00:08:27.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3686999999999999e-05 2024-09-16T00:08:27.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.954 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:27.954 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621577+0000", 2024-09-16T00:08:27.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8191000000000002e-05 2024-09-16T00:08:27.954 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.954 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:27.954 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.954 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:27.954 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:27.955 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.293025749000002, 2024-09-16T00:08:27.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005989622, 2024-09-16T00:08:27.955 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:27.955 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:27.955 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:27.955 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:27.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:27.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:27.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:27.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:27.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.956 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:27.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:27.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3799999999999998e-07 2024-09-16T00:08:27.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.956 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:27.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615660+0000", 2024-09-16T00:08:27.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9740000000000001e-06 2024-09-16T00:08:27.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:27.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615666+0000", 2024-09-16T00:08:27.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3499999999999996e-06 2024-09-16T00:08:27.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:27.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615671+0000", 2024-09-16T00:08:27.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.13e-06 2024-09-16T00:08:27.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:27.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618759+0000", 2024-09-16T00:08:27.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030880500000000002 2024-09-16T00:08:27.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:27.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618832+0000", 2024-09-16T00:08:27.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3725000000000003e-05 2024-09-16T00:08:27.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:27.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618958+0000", 2024-09-16T00:08:27.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012587899999999999 2024-09-16T00:08:27.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:27.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621583+0000", 2024-09-16T00:08:27.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026246469999999999 2024-09-16T00:08:27.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.959 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:27.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621602+0000", 2024-09-16T00:08:27.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9137e-05 2024-09-16T00:08:27.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.959 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:27.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621629+0000", 2024-09-16T00:08:27.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6451000000000001e-05 2024-09-16T00:08:27.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:27.960 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621648+0000", 2024-09-16T00:08:27.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8941e-05 2024-09-16T00:08:27.960 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.960 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:27.960 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.960 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:27.960 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:27.961 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.293008129999997, 2024-09-16T00:08:27.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053141380000000004, 2024-09-16T00:08:27.961 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:27.961 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:27.961 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:27.961 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:27.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:27.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:27.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:27.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:27.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:27.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:27.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-07 2024-09-16T00:08:27.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:27.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615678+0000", 2024-09-16T00:08:27.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:08:27.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:27.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615682+0000", 2024-09-16T00:08:27.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6659999999999999e-06 2024-09-16T00:08:27.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:27.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615686+0000", 2024-09-16T00:08:27.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6990000000000001e-06 2024-09-16T00:08:27.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:27.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615736+0000", 2024-09-16T00:08:27.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9656e-05 2024-09-16T00:08:27.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:27.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615864+0000", 2024-09-16T00:08:27.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012836100000000001 2024-09-16T00:08:27.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:27.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615989+0000", 2024-09-16T00:08:27.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000125525 2024-09-16T00:08:27.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:27.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617355+0000", 2024-09-16T00:08:27.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001365982 2024-09-16T00:08:27.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:27.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620897+0000", 2024-09-16T00:08:27.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035419610000000002 2024-09-16T00:08:27.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:27.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620941+0000", 2024-09-16T00:08:27.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3998999999999997e-05 2024-09-16T00:08:27.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:27.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620990+0000", 2024-09-16T00:08:27.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8199999999999999e-05 2024-09-16T00:08:27.967 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.967 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:27.967 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.967 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:27.967 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:27.967 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.292980872000001, 2024-09-16T00:08:27.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006316449, 2024-09-16T00:08:27.968 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:27.968 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:27.968 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:27.968 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:27.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:27.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:27.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:27.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:27.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:27.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:27.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3299999999999998e-07 2024-09-16T00:08:27.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:27.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615705+0000", 2024-09-16T00:08:27.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.278e-06 2024-09-16T00:08:27.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:27.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615711+0000", 2024-09-16T00:08:27.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8119999999999996e-06 2024-09-16T00:08:27.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:27.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615716+0000", 2024-09-16T00:08:27.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4909999999999999e-06 2024-09-16T00:08:27.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.971 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:27.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617551+0000", 2024-09-16T00:08:27.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00183549 2024-09-16T00:08:27.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.971 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:27.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617627+0000", 2024-09-16T00:08:27.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6163999999999998e-05 2024-09-16T00:08:27.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.971 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:27.972 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617776+0000", 2024-09-16T00:08:27.972 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014909499999999999 2024-09-16T00:08:27.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.972 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:27.972 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621072+0000", 2024-09-16T00:08:27.972 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032955150000000002 2024-09-16T00:08:27.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.972 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:27.972 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621957+0000", 2024-09-16T00:08:27.972 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088553199999999996 2024-09-16T00:08:27.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.973 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.973 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:27.973 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621998+0000", 2024-09-16T00:08:27.973 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0815000000000001e-05 2024-09-16T00:08:27.973 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.973 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.973 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:27.973 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:08:27.974 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0924000000000002e-05 2024-09-16T00:08:27.974 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.974 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:27.974 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.974 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.974 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.974 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:6 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:27.974 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:27.974 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.289578904000003, 2024-09-16T00:08:27.975 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000188111, 2024-09-16T00:08:27.975 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:27.975 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:27.975 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:27.975 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:27.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.975 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:27.975 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:27.975 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.976 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:27.976 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:27.976 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.976 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.976 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.976 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:27.976 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619106+0000", 2024-09-16T00:08:27.976 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4399999999999998e-07 2024-09-16T00:08:27.976 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.976 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.976 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:27.976 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619110+0000", 2024-09-16T00:08:27.977 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1110000000000001e-06 2024-09-16T00:08:27.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.977 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:27.977 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619120+0000", 2024-09-16T00:08:27.977 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7909999999999999e-06 2024-09-16T00:08:27.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.977 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:27.977 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619133+0000", 2024-09-16T00:08:27.977 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.359e-05 2024-09-16T00:08:27.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.977 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:27.978 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619179+0000", 2024-09-16T00:08:27.978 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5459e-05 2024-09-16T00:08:27.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.978 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.978 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:27.978 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619186+0000", 2024-09-16T00:08:27.978 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0480000000000001e-06 2024-09-16T00:08:27.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.978 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.978 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:27.978 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619293+0000", 2024-09-16T00:08:27.979 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000107168 2024-09-16T00:08:27.979 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.979 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:27.979 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.979 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.979 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.979 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:22 2.e e22/19 ondisk, result = 0)", 2024-09-16T00:08:27.979 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:27.979 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.287888203000001, 2024-09-16T00:08:27.979 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020058200000000001, 2024-09-16T00:08:27.979 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:27.979 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:27.979 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:27.979 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:27.980 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.980 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:27.980 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:27.980 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.980 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.980 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.980 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:27.980 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:27.980 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.980 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.980 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.980 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:27.980 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:27.980 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0109999999999999e-06 2024-09-16T00:08:27.981 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.981 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.981 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:27.981 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620801+0000", 2024-09-16T00:08:27.981 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6779999999999999e-06 2024-09-16T00:08:27.981 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.981 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.981 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:27.981 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620814+0000", 2024-09-16T00:08:27.981 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3131e-05 2024-09-16T00:08:27.981 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.981 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.981 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:27.982 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620821+0000", 2024-09-16T00:08:27.982 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0430000000000002e-06 2024-09-16T00:08:27.982 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.982 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:27.982 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:27.982 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3010999999999997e-05 2024-09-16T00:08:27.982 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.982 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:27.982 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620891+0000", 2024-09-16T00:08:27.982 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1760000000000004e-06 2024-09-16T00:08:27.982 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.983 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:27.983 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620996+0000", 2024-09-16T00:08:27.983 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010453199999999999 2024-09-16T00:08:27.983 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.983 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:27.983 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.983 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.983 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.983 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:7 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:27.983 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:27.983 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.287880219000002, 2024-09-16T00:08:27.983 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000195725, 2024-09-16T00:08:27.983 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:27.983 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:27.984 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:27.984 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:27.984 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.984 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:27.984 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:27.984 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.984 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.984 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.984 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:27.984 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:27.984 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.984 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.984 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.984 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:27.985 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620804+0000", 2024-09-16T00:08:27.985 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6099999999999999e-07 2024-09-16T00:08:27.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.985 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.985 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:27.985 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620809+0000", 2024-09-16T00:08:27.985 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4950000000000002e-06 2024-09-16T00:08:27.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.985 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.985 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:27.985 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:27.985 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0553e-05 2024-09-16T00:08:27.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.985 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.986 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:27.986 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620826+0000", 2024-09-16T00:08:27.986 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5579999999999997e-06 2024-09-16T00:08:27.986 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.986 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.986 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:27.986 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620874+0000", 2024-09-16T00:08:27.986 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7648999999999997e-05 2024-09-16T00:08:27.986 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.986 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.986 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:27.986 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620880+0000", 2024-09-16T00:08:27.986 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8290000000000002e-06 2024-09-16T00:08:27.986 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.986 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.987 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:27.987 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620999+0000", 2024-09-16T00:08:27.987 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011878 2024-09-16T00:08:27.987 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.987 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:27.987 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.987 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.987 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.987 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:8 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:27.987 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:27.987 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.287828159, 2024-09-16T00:08:27.987 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00027991099999999998, 2024-09-16T00:08:27.987 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:27.987 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:27.988 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:27.988 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:27.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.988 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:27.988 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:27.988 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.988 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.988 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:27.988 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:27.988 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.988 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.988 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:27.989 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620856+0000", 2024-09-16T00:08:27.989 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6800000000000001e-07 2024-09-16T00:08:27.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.989 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.989 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:27.989 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620859+0000", 2024-09-16T00:08:27.989 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6879999999999999e-06 2024-09-16T00:08:27.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.989 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.989 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:27.989 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620867+0000", 2024-09-16T00:08:27.989 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2819999999999996e-06 2024-09-16T00:08:27.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.990 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:27.990 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620872+0000", 2024-09-16T00:08:27.990 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5369999999999997e-06 2024-09-16T00:08:27.990 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.990 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:27.990 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621043+0000", 2024-09-16T00:08:27.990 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170791 2024-09-16T00:08:27.990 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.990 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:27.990 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621050+0000", 2024-09-16T00:08:27.990 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8580000000000002e-06 2024-09-16T00:08:27.990 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.991 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.991 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:27.991 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621135+0000", 2024-09-16T00:08:27.991 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5387000000000001e-05 2024-09-16T00:08:27.991 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.991 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:27.991 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.991 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.991 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.991 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:9 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:27.991 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:27.991 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.287799581000002, 2024-09-16T00:08:27.991 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00039143799999999997, 2024-09-16T00:08:27.991 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:27.992 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:27.992 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:27.992 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:27.992 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.992 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:27.992 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:27.992 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.992 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.992 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.992 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:27.992 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:27.992 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.992 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.992 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.993 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:27.993 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:27.993 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6800000000000002e-07 2024-09-16T00:08:27.993 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.993 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.993 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:27.993 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:08:27.993 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0310000000000001e-06 2024-09-16T00:08:27.993 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.993 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.993 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:27.993 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620892+0000", 2024-09-16T00:08:27.993 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9390000000000002e-06 2024-09-16T00:08:27.993 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.994 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.994 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:27.994 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620898+0000", 2024-09-16T00:08:27.994 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4480000000000002e-06 2024-09-16T00:08:27.994 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.994 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.994 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:27.994 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621167+0000", 2024-09-16T00:08:27.994 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026960599999999998 2024-09-16T00:08:27.994 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.994 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.994 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:27.994 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621174+0000", 2024-09-16T00:08:27.995 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.567e-06 2024-09-16T00:08:27.995 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.995 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.995 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:27.995 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621275+0000", 2024-09-16T00:08:27.995 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101579 2024-09-16T00:08:27.995 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.995 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:27.995 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.995 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.995 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.995 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:10 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:27.995 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:27.995 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.287777132999999, 2024-09-16T00:08:27.996 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00049416200000000003, 2024-09-16T00:08:27.996 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:27.996 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:27.996 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:27.996 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:27.996 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.996 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:27.996 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:27.996 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.996 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.996 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.996 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:27.996 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:27.996 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:27.997 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.997 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.997 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:27.997 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620907+0000", 2024-09-16T00:08:27.997 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.42e-07 2024-09-16T00:08:27.997 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.997 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.997 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:27.997 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620908+0000", 2024-09-16T00:08:27.997 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4300000000000001e-06 2024-09-16T00:08:27.997 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.997 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.997 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:27.997 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620911+0000", 2024-09-16T00:08:27.998 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3890000000000001e-06 2024-09-16T00:08:27.998 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.998 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.998 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:27.998 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620914+0000", 2024-09-16T00:08:27.998 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3690000000000001e-06 2024-09-16T00:08:27.998 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.998 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.998 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:27.998 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621297+0000", 2024-09-16T00:08:27.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038319200000000003 2024-09-16T00:08:27.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:27.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621305+0000", 2024-09-16T00:08:27.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4780000000000008e-06 2024-09-16T00:08:27.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:27.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:27.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:27.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621401+0000", 2024-09-16T00:08:27.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5161999999999995e-05 2024-09-16T00:08:27.999 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:27.999 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.000 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.000 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:11 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:28.000 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:28.000 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.287374384000003, 2024-09-16T00:08:28.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017943899999999999, 2024-09-16T00:08:28.000 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.000 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.000 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:28.000 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.001 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.001 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:28.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.001 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.001 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:28.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.001 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.001 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621310+0000", 2024-09-16T00:08:28.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5199999999999996e-07 2024-09-16T00:08:28.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.002 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.002 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:28.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1740000000000002e-06 2024-09-16T00:08:28.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.002 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.002 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621324+0000", 2024-09-16T00:08:28.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7170000000000003e-06 2024-09-16T00:08:28.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621331+0000", 2024-09-16T00:08:28.003 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2280000000000001e-06 2024-09-16T00:08:28.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621422+0000", 2024-09-16T00:08:28.003 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1052999999999997e-05 2024-09-16T00:08:28.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:28.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621426+0000", 2024-09-16T00:08:28.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.036e-06 2024-09-16T00:08:28.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.004 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621489+0000", 2024-09-16T00:08:28.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2478999999999996e-05 2024-09-16T00:08:28.004 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.004 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.004 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.005 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:12 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:28.005 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:28.005 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.287323528999998, 2024-09-16T00:08:28.005 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022422300000000001, 2024-09-16T00:08:28.005 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.005 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.005 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:28.005 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.005 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.005 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:28.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:28.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:28.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6699999999999999e-07 2024-09-16T00:08:28.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621363+0000", 2024-09-16T00:08:28.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7020000000000001e-06 2024-09-16T00:08:28.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621371+0000", 2024-09-16T00:08:28.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7209999999999994e-06 2024-09-16T00:08:28.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621376+0000", 2024-09-16T00:08:28.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7940000000000002e-06 2024-09-16T00:08:28.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.009 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621511+0000", 2024-09-16T00:08:28.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013518100000000001 2024-09-16T00:08:28.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.009 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:28.009 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621514+0000", 2024-09-16T00:08:28.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5599999999999998e-06 2024-09-16T00:08:28.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.010 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621584+0000", 2024-09-16T00:08:28.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9898000000000001e-05 2024-09-16T00:08:28.010 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.010 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.010 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.010 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:13 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:28.011 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:28.011 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.287297264999999, 2024-09-16T00:08:28.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026534099999999999, 2024-09-16T00:08:28.011 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.011 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.011 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:28.011 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.011 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.011 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:28.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.011 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:28.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.012 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621387+0000", 2024-09-16T00:08:28.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4299999999999999e-07 2024-09-16T00:08:28.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.012 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621389+0000", 2024-09-16T00:08:28.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.593e-06 2024-09-16T00:08:28.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621395+0000", 2024-09-16T00:08:28.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.553e-06 2024-09-16T00:08:28.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:08:28.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8410000000000002e-06 2024-09-16T00:08:28.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621597+0000", 2024-09-16T00:08:28.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019794100000000001 2024-09-16T00:08:28.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:28.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621600+0000", 2024-09-16T00:08:28.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.185e-06 2024-09-16T00:08:28.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621652+0000", 2024-09-16T00:08:28.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1984999999999998e-05 2024-09-16T00:08:28.014 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.014 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.014 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.015 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:23 2.e e22/19 ondisk, result = 0)", 2024-09-16T00:08:28.015 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:28.015 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.286814315999997, 2024-09-16T00:08:28.015 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015895699999999999, 2024-09-16T00:08:28.015 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.015 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.015 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:28.015 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.015 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.015 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.015 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:28.015 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.015 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.016 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.016 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.016 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:28.016 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.016 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.016 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.016 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.016 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621870+0000", 2024-09-16T00:08:28.016 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1600000000000001e-07 2024-09-16T00:08:28.016 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.016 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.016 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.016 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621873+0000", 2024-09-16T00:08:28.016 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3079999999999999e-06 2024-09-16T00:08:28.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.017 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.017 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.017 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621882+0000", 2024-09-16T00:08:28.017 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8429999999999999e-06 2024-09-16T00:08:28.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.017 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.017 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.017 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621896+0000", 2024-09-16T00:08:28.017 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:08:28.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.017 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.017 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.017 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621944+0000", 2024-09-16T00:08:28.018 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8062999999999997e-05 2024-09-16T00:08:28.018 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.018 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:28.018 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621951+0000", 2024-09-16T00:08:28.018 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7689999999999999e-06 2024-09-16T00:08:28.018 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.018 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.018 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622028+0000", 2024-09-16T00:08:28.018 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7243999999999994e-05 2024-09-16T00:08:28.018 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.018 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.018 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.018 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.019 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.019 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:28.020 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:08:28.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.929+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5184012f60 con 0x7f5184014bd0 2024-09-16T00:08:28.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.931+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5184012f60 con 0x7f5184014bd0 2024-09-16T00:08:28.021 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:28.022 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:28.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.931+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5184013680 0x7f5184013ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:28.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.931+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f5184014340 con 0x7f5184013680 2024-09-16T00:08:28.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.932+0000 7f51b11ab640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5184013680 0x7f5184013ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:28.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.932+0000 7f51b11ab640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5184013680 0x7f5184013ac0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:28.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.933+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5184014340 con 0x7f5184013680 2024-09-16T00:08:28.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.959+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5184014100 con 0x7f5184013680 2024-09-16T00:08:28.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.961+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5184014100 con 0x7f5184013680 2024-09-16T00:08:28.023 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:28.024 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:28.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.962+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5184019cf0 0x7f518401c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:28.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.962+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f518401ef30 con 0x7f5184019cf0 2024-09-16T00:08:28.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.962+0000 7f51b09aa640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5184019cf0 0x7f518401c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:28.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.962+0000 7f51b09aa640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5184019cf0 0x7f518401c0e0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:28.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.963+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f518401ef30 con 0x7f5184019cf0 2024-09-16T00:08:28.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.990+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f518401c6a0 con 0x7f5184019cf0 2024-09-16T00:08:28.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f51a97fa640 1 -- 172.21.15.139:0/3916614935 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f518401c6a0 con 0x7f5184019cf0 2024-09-16T00:08:28.025 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:28.027 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:28.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5184001650 msgr2=0x7f5184003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:28.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5184001650 0x7f5184003b10 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5184004190 msgr2=0x7f5184004640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:28.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5184004190 0x7f5184004640 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5184009340 msgr2=0x7f518400b730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:28.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5184009340 0x7f518400b730 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f518400bf80 msgr2=0x7f518400c3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:28.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f518400bf80 0x7f518400c3e0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f518400d1f0 msgr2=0x7f5184012810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:28.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f518400d1f0 0x7f5184012810 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5184014bd0 msgr2=0x7f5184015030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f51b11ab640 1 -- 172.21.15.139:0/3916614935 reap_dead start 2024-09-16T00:08:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5184014bd0 0x7f5184015030 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f51b11ab640 1 -- 172.21.15.139:0/3916614935 reap_dead start 2024-09-16T00:08:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f51b11ab640 1 -- 172.21.15.139:0/3916614935 reap_dead start 2024-09-16T00:08:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5184013680 msgr2=0x7f5184013ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5184013680 0x7f5184013ac0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5184019cf0 msgr2=0x7f518401c0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5184019cf0 0x7f518401c0e0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f519006c2b0 msgr2=0x7f519006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f519006c2b0 0x7f519006e770 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f519800d440 tx=0x7f5198018040 comp rx=0 tx=0).stop 2024-09-16T00:08:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51ac0c8600 msgr2=0x7f51ac147790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:28.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51ac0c8600 0x7f51ac147790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f51a000c3c0 tx=0x7f51a000c880 comp rx=0 tx=0).stop 2024-09-16T00:08:28.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 shutdown_connections 2024-09-16T00:08:28.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5184013680 0x7f5184013ac0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5184019cf0 0x7f518401c0e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f519006c2b0 0x7f519006e770 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 --2- 172.21.15.139:0/3916614935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51ac0c8600 0x7f51ac147790 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 >> 172.21.15.139:0/3916614935 conn(0x7f51ac06e960 msgr2=0x7f51ac073600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:28.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 shutdown_connections 2024-09-16T00:08:28.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:27.992+0000 7f5196ffd640 1 -- 172.21.15.139:0/3916614935 wait complete. 2024-09-16T00:08:28.029 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:28.019+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:28.120 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:28.120+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:28.166 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:28.166 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:28.166 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:28.166 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:28.175 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:08:28.222 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:28.221+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:28.282 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:28.282 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:28.283 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:28.283 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:28.283 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:28.292 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:08:28.335 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:28.335+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:28.399 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:28.400 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:28.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:28.400 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:28.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.400 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:22 2.e e22/19 2:717a0223:::608.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:28.401 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:28.401 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.781218121000002, 2024-09-16T00:08:28.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042195870000000003, 2024-09-16T00:08:28.401 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.401 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.401 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:28.401 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:28.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:28.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616426+0000", 2024-09-16T00:08:28.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.037e-06 2024-09-16T00:08:28.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616435+0000", 2024-09-16T00:08:28.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6440000000000006e-06 2024-09-16T00:08:28.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616447+0000", 2024-09-16T00:08:28.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2466000000000001e-05 2024-09-16T00:08:28.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616456+0000", 2024-09-16T00:08:28.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8799999999999997e-06 2024-09-16T00:08:28.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616517+0000", 2024-09-16T00:08:28.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1171000000000001e-05 2024-09-16T00:08:28.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:28.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616540+0000", 2024-09-16T00:08:28.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2991000000000001e-05 2024-09-16T00:08:28.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:28.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620599+0000", 2024-09-16T00:08:28.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040586140000000003 2024-09-16T00:08:28.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620645+0000", 2024-09-16T00:08:28.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5784000000000001e-05 2024-09-16T00:08:28.405 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.405 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.405 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.405 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:28.405 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:28.405 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.781128391000003, 2024-09-16T00:08:28.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044836249999999998, 2024-09-16T00:08:28.405 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.405 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.405 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:28.405 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:28.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:28.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616518+0000", 2024-09-16T00:08:28.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3699999999999999e-06 2024-09-16T00:08:28.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:08:28.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1479999999999997e-06 2024-09-16T00:08:28.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616534+0000", 2024-09-16T00:08:28.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0641e-05 2024-09-16T00:08:28.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616541+0000", 2024-09-16T00:08:28.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3339999999999996e-06 2024-09-16T00:08:28.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617318+0000", 2024-09-16T00:08:28.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00077694799999999996 2024-09-16T00:08:28.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:28.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617409+0000", 2024-09-16T00:08:28.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0740000000000002e-05 2024-09-16T00:08:28.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620998+0000", 2024-09-16T00:08:28.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035894439999999998 2024-09-16T00:08:28.409 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.409 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.409 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.409 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:28.409 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:28.409 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.781080488000001, 2024-09-16T00:08:28.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041597320000000002, 2024-09-16T00:08:28.410 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.410 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.410 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:28.410 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:28.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:28.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:28.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:08:28.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616566+0000", 2024-09-16T00:08:28.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6979999999999998e-06 2024-09-16T00:08:28.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616574+0000", 2024-09-16T00:08:28.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7759999999999997e-06 2024-09-16T00:08:28.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616580+0000", 2024-09-16T00:08:28.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:28.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616800+0000", 2024-09-16T00:08:28.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022016700000000001 2024-09-16T00:08:28.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:28.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616904+0000", 2024-09-16T00:08:28.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104579 2024-09-16T00:08:28.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620722+0000", 2024-09-16T00:08:28.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003818133 2024-09-16T00:08:28.414 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.414 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.414 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.414 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:28.414 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:28.414 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.780829803000003, 2024-09-16T00:08:28.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038451570000000001, 2024-09-16T00:08:28.414 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.414 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.414 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:28.415 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:28.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:28.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616814+0000", 2024-09-16T00:08:28.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9299999999999997e-07 2024-09-16T00:08:28.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616824+0000", 2024-09-16T00:08:28.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6600000000000007e-06 2024-09-16T00:08:28.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616835+0000", 2024-09-16T00:08:28.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0713000000000001e-05 2024-09-16T00:08:28.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616841+0000", 2024-09-16T00:08:28.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.934e-06 2024-09-16T00:08:28.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616948+0000", 2024-09-16T00:08:28.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010631100000000001 2024-09-16T00:08:28.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:28.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616967+0000", 2024-09-16T00:08:28.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9598000000000001e-05 2024-09-16T00:08:28.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:28.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620619+0000", 2024-09-16T00:08:28.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036515810000000001 2024-09-16T00:08:28.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620659+0000", 2024-09-16T00:08:28.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9666999999999998e-05 2024-09-16T00:08:28.418 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.418 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.418 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.419 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:28.419 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:28.419 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.780782850999998, 2024-09-16T00:08:28.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038240129999999998, 2024-09-16T00:08:28.419 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.419 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.419 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:28.419 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:28.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:28.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616861+0000", 2024-09-16T00:08:28.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0200000000000003e-07 2024-09-16T00:08:28.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616864+0000", 2024-09-16T00:08:28.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5090000000000002e-06 2024-09-16T00:08:28.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616871+0000", 2024-09-16T00:08:28.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9249999999999998e-06 2024-09-16T00:08:28.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616877+0000", 2024-09-16T00:08:28.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.981e-06 2024-09-16T00:08:28.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617168+0000", 2024-09-16T00:08:28.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029035699999999998 2024-09-16T00:08:28.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:28.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617184+0000", 2024-09-16T00:08:28.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6005999999999999e-05 2024-09-16T00:08:28.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:28.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620661+0000", 2024-09-16T00:08:28.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034775330000000001 2024-09-16T00:08:28.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620684+0000", 2024-09-16T00:08:28.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3300000000000001e-05 2024-09-16T00:08:28.424 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.424 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.424 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.424 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:28.424 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:28.424 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.780327407999998, 2024-09-16T00:08:28.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045415610000000004, 2024-09-16T00:08:28.424 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.424 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.424 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:28.424 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:28.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:28.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:28.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8800000000000002e-07 2024-09-16T00:08:28.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617321+0000", 2024-09-16T00:08:28.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6979999999999997e-06 2024-09-16T00:08:28.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617330+0000", 2024-09-16T00:08:28.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6729999999999998e-06 2024-09-16T00:08:28.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617336+0000", 2024-09-16T00:08:28.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2110000000000003e-06 2024-09-16T00:08:28.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617394+0000", 2024-09-16T00:08:28.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8087999999999997e-05 2024-09-16T00:08:28.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:28.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617412+0000", 2024-09-16T00:08:28.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8099999999999999e-05 2024-09-16T00:08:28.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:28.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621818+0000", 2024-09-16T00:08:28.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044061209999999998 2024-09-16T00:08:28.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621857+0000", 2024-09-16T00:08:28.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9082000000000003e-05 2024-09-16T00:08:28.429 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.429 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.429 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.430 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:23 2.e e22/19 2:717a0223:::608.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:28.430 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:28.430 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.779431387000002, 2024-09-16T00:08:28.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036198279999999998, 2024-09-16T00:08:28.430 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.430 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.430 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:28.430 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:28.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:28.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618213+0000", 2024-09-16T00:08:28.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:08:28.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618218+0000", 2024-09-16T00:08:28.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4099999999999999e-06 2024-09-16T00:08:28.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618227+0000", 2024-09-16T00:08:28.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5009999999999993e-06 2024-09-16T00:08:28.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618234+0000", 2024-09-16T00:08:28.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:28.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618307+0000", 2024-09-16T00:08:28.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3421000000000001e-05 2024-09-16T00:08:28.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:28.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618328+0000", 2024-09-16T00:08:28.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0919000000000001e-05 2024-09-16T00:08:28.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:28.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621804+0000", 2024-09-16T00:08:28.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034756560000000001 2024-09-16T00:08:28.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621832+0000", 2024-09-16T00:08:28.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7966999999999998e-05 2024-09-16T00:08:28.435 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.435 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.435 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.435 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing_reply(2.1e epoch 22/19 rep_tid 0 lcod 22'1)", 2024-09-16T00:08:28.436 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:28.436 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.778986218999997, 2024-09-16T00:08:28.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1371000000000003e-05, 2024-09-16T00:08:28.436 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.436 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.436 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:28.436 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:28.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:28.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:08:28.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:08:28.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618662+0000", 2024-09-16T00:08:28.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1189999999999999e-06 2024-09-16T00:08:28.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618674+0000", 2024-09-16T00:08:28.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2317000000000001e-05 2024-09-16T00:08:28.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618681+0000", 2024-09-16T00:08:28.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7170000000000002e-06 2024-09-16T00:08:28.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:08:28.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6745000000000001e-05 2024-09-16T00:08:28.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618748+0000", 2024-09-16T00:08:28.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0794000000000001e-05 2024-09-16T00:08:28.439 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.439 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.439 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.439 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:28.439 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:28.439 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.778472884999999, 2024-09-16T00:08:28.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027183849999999998, 2024-09-16T00:08:28.440 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.440 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.440 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:28.440 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:28.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:28.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619171+0000", 2024-09-16T00:08:28.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8999999999999995e-07 2024-09-16T00:08:28.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619179+0000", 2024-09-16T00:08:28.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2679999999999994e-06 2024-09-16T00:08:28.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:08:28.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1246e-05 2024-09-16T00:08:28.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619197+0000", 2024-09-16T00:08:28.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2820000000000002e-06 2024-09-16T00:08:28.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619308+0000", 2024-09-16T00:08:28.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011141000000000001 2024-09-16T00:08:28.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:28.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619325+0000", 2024-09-16T00:08:28.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6469000000000001e-05 2024-09-16T00:08:28.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:28.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621860+0000", 2024-09-16T00:08:28.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025353590000000001 2024-09-16T00:08:28.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621889+0000", 2024-09-16T00:08:28.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8561e-05 2024-09-16T00:08:28.444 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.444 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.444 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.445 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 6 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:28.445 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:28.445 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.778430186000001, 2024-09-16T00:08:28.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026903809999999999, 2024-09-16T00:08:28.445 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.445 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.445 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:28.445 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:28.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:28.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:28.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2000000000000001e-07 2024-09-16T00:08:28.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619219+0000", 2024-09-16T00:08:28.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:08:28.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619234+0000", 2024-09-16T00:08:28.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5075000000000001e-05 2024-09-16T00:08:28.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619251+0000", 2024-09-16T00:08:28.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6863000000000001e-05 2024-09-16T00:08:28.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619556+0000", 2024-09-16T00:08:28.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030551599999999998 2024-09-16T00:08:28.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621903+0000", 2024-09-16T00:08:28.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023469979999999999 2024-09-16T00:08:28.449 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.449 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.449 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.449 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing_reply(2.1b epoch 22/19 rep_tid 1 lcod 22'1)", 2024-09-16T00:08:28.449 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:28.449 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.777351789999997, 2024-09-16T00:08:28.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6989000000000003e-05, 2024-09-16T00:08:28.449 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.449 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.449 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:28.450 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:28.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:28.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620292+0000", 2024-09-16T00:08:28.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.37e-07 2024-09-16T00:08:28.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620296+0000", 2024-09-16T00:08:28.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2420000000000002e-06 2024-09-16T00:08:28.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620308+0000", 2024-09-16T00:08:28.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1623e-05 2024-09-16T00:08:28.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620313+0000", 2024-09-16T00:08:28.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6290000000000001e-06 2024-09-16T00:08:28.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620359+0000", 2024-09-16T00:08:28.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6201e-05 2024-09-16T00:08:28.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620378+0000", 2024-09-16T00:08:28.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9556999999999999e-05 2024-09-16T00:08:28.452 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.452 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.452 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.453 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:38 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'6, mlcod=22'4)", 2024-09-16T00:08:28.453 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:28.453 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.774909541, 2024-09-16T00:08:28.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064551599999999995, 2024-09-16T00:08:28.453 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.453 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.453 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:28.453 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:28.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:28.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:28.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4399999999999999e-07 2024-09-16T00:08:28.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622739+0000", 2024-09-16T00:08:28.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0250000000000002e-06 2024-09-16T00:08:28.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622749+0000", 2024-09-16T00:08:28.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1220000000000001e-06 2024-09-16T00:08:28.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622760+0000", 2024-09-16T00:08:28.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1640999999999999e-05 2024-09-16T00:08:28.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622799+0000", 2024-09-16T00:08:28.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8537000000000001e-05 2024-09-16T00:08:28.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:28.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622810+0000", 2024-09-16T00:08:28.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1562e-05 2024-09-16T00:08:28.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:28.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.623356+0000", 2024-09-16T00:08:28.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054522299999999995 2024-09-16T00:08:28.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.623379+0000", 2024-09-16T00:08:28.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3662000000000001e-05 2024-09-16T00:08:28.457 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.457 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.458 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.458 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:40 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'7, mlcod=22'5)", 2024-09-16T00:08:28.458 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:28.458 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.557888638999998, 2024-09-16T00:08:28.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001017773, 2024-09-16T00:08:28.458 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.458 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.458 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:28.458 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:28.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:28.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839756+0000", 2024-09-16T00:08:28.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.491e-06 2024-09-16T00:08:28.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839768+0000", 2024-09-16T00:08:28.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1649e-05 2024-09-16T00:08:28.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839783+0000", 2024-09-16T00:08:28.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5023e-05 2024-09-16T00:08:28.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839791+0000", 2024-09-16T00:08:28.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5620000000000004e-06 2024-09-16T00:08:28.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839848+0000", 2024-09-16T00:08:28.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6563e-05 2024-09-16T00:08:28.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:28.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839868+0000", 2024-09-16T00:08:28.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9652000000000001e-05 2024-09-16T00:08:28.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:28.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.840729+0000", 2024-09-16T00:08:28.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086159500000000005 2024-09-16T00:08:28.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.463 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.840772+0000", 2024-09-16T00:08:28.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3238000000000002e-05 2024-09-16T00:08:28.463 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.463 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.463 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.464 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:41 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'8, mlcod=22'6)", 2024-09-16T00:08:28.464 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:28.464 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 35.284231728000002, 2024-09-16T00:08:28.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001040568, 2024-09-16T00:08:28.464 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.464 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.464 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:28.464 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:28.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:28.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113413+0000", 2024-09-16T00:08:28.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7409999999999999e-06 2024-09-16T00:08:28.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113428+0000", 2024-09-16T00:08:28.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5007999999999999e-05 2024-09-16T00:08:28.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113447+0000", 2024-09-16T00:08:28.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8731000000000001e-05 2024-09-16T00:08:28.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113457+0000", 2024-09-16T00:08:28.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0095000000000001e-05 2024-09-16T00:08:28.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113526+0000", 2024-09-16T00:08:28.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8552000000000004e-05 2024-09-16T00:08:28.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:28.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113552+0000", 2024-09-16T00:08:28.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6795999999999999e-05 2024-09-16T00:08:28.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:28.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.114417+0000", 2024-09-16T00:08:28.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086476299999999995 2024-09-16T00:08:28.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.114452+0000", 2024-09-16T00:08:28.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4882000000000002e-05 2024-09-16T00:08:28.468 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.468 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.469 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.469 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.14 38/37 [PushReplyOp(2:2f55791f:::606.00000000:head)])", 2024-09-16T00:08:28.469 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.839962+0000", 2024-09-16T00:08:28.469 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.5576813070000002, 2024-09-16T00:08:28.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030257, 2024-09-16T00:08:28.469 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.470 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.470 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:28.470 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839962+0000", 2024-09-16T00:08:28.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839962+0000", 2024-09-16T00:08:28.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839962+0000", 2024-09-16T00:08:28.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5400000000000001e-07 2024-09-16T00:08:28.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839967+0000", 2024-09-16T00:08:28.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.245e-06 2024-09-16T00:08:28.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839978+0000", 2024-09-16T00:08:28.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1388e-05 2024-09-16T00:08:28.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839987+0000", 2024-09-16T00:08:28.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1500000000000005e-06 2024-09-16T00:08:28.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.840043+0000", 2024-09-16T00:08:28.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5396000000000003e-05 2024-09-16T00:08:28.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.840265+0000", 2024-09-16T00:08:28.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022183700000000001 2024-09-16T00:08:28.473 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.473 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.473 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.473 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.14 38/37 [PushReplyOp(2:287034c4:::1.00000000.inode:head)])", 2024-09-16T00:08:28.473 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.848174+0000", 2024-09-16T00:08:28.473 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.5494691010000001, 2024-09-16T00:08:28.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029495900000000002, 2024-09-16T00:08:28.474 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.474 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.474 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:28.474 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848174+0000", 2024-09-16T00:08:28.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848174+0000", 2024-09-16T00:08:28.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848175+0000", 2024-09-16T00:08:28.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.68e-07 2024-09-16T00:08:28.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848179+0000", 2024-09-16T00:08:28.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1400000000000002e-06 2024-09-16T00:08:28.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848191+0000", 2024-09-16T00:08:28.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1823e-05 2024-09-16T00:08:28.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848196+0000", 2024-09-16T00:08:28.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.626e-06 2024-09-16T00:08:28.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848251+0000", 2024-09-16T00:08:28.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4342999999999998e-05 2024-09-16T00:08:28.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848469+0000", 2024-09-16T00:08:28.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000218359 2024-09-16T00:08:28.477 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.477 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.477 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.477 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.14 38/37 [PushReplyOp(2:2e2fa760:::200.00000001:head)])", 2024-09-16T00:08:28.477 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.850960+0000", 2024-09-16T00:08:28.477 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.5466828100000001, 2024-09-16T00:08:28.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00028471600000000002, 2024-09-16T00:08:28.477 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.477 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.477 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:28.478 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850960+0000", 2024-09-16T00:08:28.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850960+0000", 2024-09-16T00:08:28.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850961+0000", 2024-09-16T00:08:28.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6000000000000003e-07 2024-09-16T00:08:28.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850965+0000", 2024-09-16T00:08:28.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1400000000000002e-06 2024-09-16T00:08:28.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850977+0000", 2024-09-16T00:08:28.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2128e-05 2024-09-16T00:08:28.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850985+0000", 2024-09-16T00:08:28.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3220000000000004e-06 2024-09-16T00:08:28.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.851036+0000", 2024-09-16T00:08:28.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1100000000000002e-05 2024-09-16T00:08:28.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.851245+0000", 2024-09-16T00:08:28.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000209266 2024-09-16T00:08:28.481 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.481 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.481 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.481 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.14 38/37 [PushReplyOp(2:292cf221:::200.00000000:head)])", 2024-09-16T00:08:28.486 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.853964+0000", 2024-09-16T00:08:28.486 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.543679622, 2024-09-16T00:08:28.486 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026211799999999999, 2024-09-16T00:08:28.486 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:28.486 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:28.486 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:28.486 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:28.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.486 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:28.486 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853964+0000", 2024-09-16T00:08:28.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:28.487 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853964+0000", 2024-09-16T00:08:28.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:28.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:28.487 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853964+0000", 2024-09-16T00:08:28.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8000000000000002e-07 2024-09-16T00:08:28.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:28.487 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853967+0000", 2024-09-16T00:08:28.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6529999999999998e-06 2024-09-16T00:08:28.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:28.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853973+0000", 2024-09-16T00:08:28.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9110000000000004e-06 2024-09-16T00:08:28.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853981+0000", 2024-09-16T00:08:28.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.25e-06 2024-09-16T00:08:28.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:28.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.854030+0000", 2024-09-16T00:08:28.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8909000000000002e-05 2024-09-16T00:08:28.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:28.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:28.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:28.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.854226+0000", 2024-09-16T00:08:28.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019611499999999999 2024-09-16T00:08:28.489 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.489 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.489 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.489 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:28.489 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:28.489 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:28.491 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:08:28.491 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:28.491+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:28.592 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:28.591+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:28.607 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:28.607 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:28.607 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:28.607 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:28.616 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:08:28.693 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:28.692+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:28.721 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:28.721 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:28.722 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:28.722 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:28.722 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:28.730 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:08:28.814 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:28.813+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:28.841 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T00:08:28.841 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T00:08:28.858 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T00:08:28.875 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.260020441000002, 2024-09-16T00:08:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036770370000000002, 2024-09-16T00:08:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:28.876 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:28.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615154+0000", 2024-09-16T00:08:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1510000000000003e-06 2024-09-16T00:08:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615163+0000", 2024-09-16T00:08:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4490000000000005e-06 2024-09-16T00:08:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615182+0000", 2024-09-16T00:08:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8876e-05 2024-09-16T00:08:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615197+0000", 2024-09-16T00:08:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4671000000000001e-05 2024-09-16T00:08:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615349+0000", 2024-09-16T00:08:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015258700000000001 2024-09-16T00:08:28.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:08:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000204716 2024-09-16T00:08:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:08:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017250800000000001 2024-09-16T00:08:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616919+0000", 2024-09-16T00:08:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011921799999999999 2024-09-16T00:08:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:08:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018087769999999999 2024-09-16T00:08:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618774+0000", 2024-09-16T00:08:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6075999999999997e-05 2024-09-16T00:08:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618826+0000", 2024-09-16T00:08:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2046000000000003e-05 2024-09-16T00:08:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.882 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.258664652999997, 2024-09-16T00:08:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023501059999999998, 2024-09-16T00:08:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616506+0000", 2024-09-16T00:08:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4100000000000001e-06 2024-09-16T00:08:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617073+0000", 2024-09-16T00:08:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056725899999999997 2024-09-16T00:08:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617099+0000", 2024-09-16T00:08:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5860999999999999e-05 2024-09-16T00:08:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617114+0000", 2024-09-16T00:08:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5180999999999999e-05 2024-09-16T00:08:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617236+0000", 2024-09-16T00:08:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121662 2024-09-16T00:08:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001618733 2024-09-16T00:08:28.886 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:31 2.d e22/19 2:b50e409b:::mds_snaptable:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.257770505000003, 2024-09-16T00:08:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021851240000000001, 2024-09-16T00:08:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:28.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617400+0000", 2024-09-16T00:08:28.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.28e-06 2024-09-16T00:08:28.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617410+0000", 2024-09-16T00:08:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0179e-05 2024-09-16T00:08:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617435+0000", 2024-09-16T00:08:28.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5221000000000001e-05 2024-09-16T00:08:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617444+0000", 2024-09-16T00:08:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1989999999999995e-06 2024-09-16T00:08:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617514+0000", 2024-09-16T00:08:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9104999999999997e-05 2024-09-16T00:08:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617538+0000", 2024-09-16T00:08:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4383999999999999e-05 2024-09-16T00:08:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619536+0000", 2024-09-16T00:08:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019982789999999999 2024-09-16T00:08:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619584+0000", 2024-09-16T00:08:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7477e-05 2024-09-16T00:08:28.895 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:27 2.13 e22/19 2:cd5a64a3:::100.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.257465322999998, 2024-09-16T00:08:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018775459999999999, 2024-09-16T00:08:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:28.896 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617705+0000", 2024-09-16T00:08:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9900000000000009e-07 2024-09-16T00:08:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617718+0000", 2024-09-16T00:08:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3453e-05 2024-09-16T00:08:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617730+0000", 2024-09-16T00:08:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2221e-05 2024-09-16T00:08:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617739+0000", 2024-09-16T00:08:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0660000000000004e-06 2024-09-16T00:08:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617806+0000", 2024-09-16T00:08:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7397999999999994e-05 2024-09-16T00:08:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617823+0000", 2024-09-16T00:08:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7569999999999999e-05 2024-09-16T00:08:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619528+0000", 2024-09-16T00:08:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017045859999999999 2024-09-16T00:08:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619581+0000", 2024-09-16T00:08:28.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3253e-05 2024-09-16T00:08:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:28 2.13 e22/19 2:c91d4a1d:::mds0_inotable:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.257118052999999, 2024-09-16T00:08:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001558051, 2024-09-16T00:08:28.902 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:28.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618052+0000", 2024-09-16T00:08:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1399999999999997e-07 2024-09-16T00:08:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618065+0000", 2024-09-16T00:08:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3495e-05 2024-09-16T00:08:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:08:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0560999999999999e-05 2024-09-16T00:08:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618083+0000", 2024-09-16T00:08:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:08:28.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618147+0000", 2024-09-16T00:08:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4525999999999999e-05 2024-09-16T00:08:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618166+0000", 2024-09-16T00:08:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9045000000000001e-05 2024-09-16T00:08:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619585+0000", 2024-09-16T00:08:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014182660000000001 2024-09-16T00:08:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619609+0000", 2024-09-16T00:08:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4468e-05 2024-09-16T00:08:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:29 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.256767549000003, 2024-09-16T00:08:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012347219999999999, 2024-09-16T00:08:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:28.910 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3099999999999997e-07 2024-09-16T00:08:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618409+0000", 2024-09-16T00:08:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6239999999999996e-06 2024-09-16T00:08:28.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618420+0000", 2024-09-16T00:08:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1222e-05 2024-09-16T00:08:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618441+0000", 2024-09-16T00:08:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0564000000000002e-05 2024-09-16T00:08:28.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618519+0000", 2024-09-16T00:08:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7917999999999995e-05 2024-09-16T00:08:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618535+0000", 2024-09-16T00:08:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6266e-05 2024-09-16T00:08:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:28.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619612+0000", 2024-09-16T00:08:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010767039999999999 2024-09-16T00:08:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619636+0000", 2024-09-16T00:08:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4692999999999999e-05 2024-09-16T00:08:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:2 2.f e22/19 ondisk, result = 0)", 2024-09-16T00:08:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.256531522000003, 2024-09-16T00:08:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019782099999999999, 2024-09-16T00:08:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:28.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:28.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618639+0000", 2024-09-16T00:08:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5099999999999998e-07 2024-09-16T00:08:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618644+0000", 2024-09-16T00:08:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7440000000000004e-06 2024-09-16T00:08:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618655+0000", 2024-09-16T00:08:28.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0798999999999999e-05 2024-09-16T00:08:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618662+0000", 2024-09-16T00:08:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8700000000000003e-06 2024-09-16T00:08:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618717+0000", 2024-09-16T00:08:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5223999999999998e-05 2024-09-16T00:08:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618724+0000", 2024-09-16T00:08:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.046e-06 2024-09-16T00:08:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618835+0000", 2024-09-16T00:08:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111187 2024-09-16T00:08:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.921 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.b epoch 22/19 rep_tid 122 entries 22'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.255427128999997, 2024-09-16T00:08:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018692890000000001, 2024-09-16T00:08:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619743+0000", 2024-09-16T00:08:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1069999999999999e-06 2024-09-16T00:08:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619754+0000", 2024-09-16T00:08:28.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0789e-05 2024-09-16T00:08:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619777+0000", 2024-09-16T00:08:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2844e-05 2024-09-16T00:08:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619793+0000", 2024-09-16T00:08:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6231000000000001e-05 2024-09-16T00:08:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619875+0000", 2024-09-16T00:08:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2286000000000005e-05 2024-09-16T00:08:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621611+0000", 2024-09-16T00:08:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001736032 2024-09-16T00:08:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.925 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.6 38/37 [PushOp(2:654134d2:::mds0_openfiles.0:head, version: 26'3, data_included: [], data_size: 0, omap_header_size: 42, omap_entries_size: 2, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:654134d2:::mds0_openfiles.0:head@26'3, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.839473+0000", 2024-09-16T00:08:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0356963690000001, 2024-09-16T00:08:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044119500000000002, 2024-09-16T00:08:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839473+0000", 2024-09-16T00:08:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839473+0000", 2024-09-16T00:08:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839474+0000", 2024-09-16T00:08:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1999999999999998e-07 2024-09-16T00:08:28.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839512+0000", 2024-09-16T00:08:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8546e-05 2024-09-16T00:08:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839530+0000", 2024-09-16T00:08:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7663e-05 2024-09-16T00:08:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839550+0000", 2024-09-16T00:08:28.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9817e-05 2024-09-16T00:08:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839658+0000", 2024-09-16T00:08:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108213 2024-09-16T00:08:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839672+0000", 2024-09-16T00:08:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4518000000000001e-05 2024-09-16T00:08:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:28.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839914+0000", 2024-09-16T00:08:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000241618 2024-09-16T00:08:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:28.930 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:28.930 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:28.932 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:08:28.932 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:28.930+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:28.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.948+0000 7fb0b0e37640 1 Processor -- start 2024-09-16T00:08:28.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.948+0000 7fb0b0e37640 1 -- start start 2024-09-16T00:08:28.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.950+0000 7fb0b0e37640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb0ac151a10 0x7fb0ac067950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:28.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.950+0000 7fb0b0e37640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0ac0c3c40 0x7fb0ac0c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:28.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.950+0000 7fb0b0e37640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0ac0c38d0 con 0x7fb0ac0c5d60 2024-09-16T00:08:28.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.950+0000 7fb0b0e37640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb0ac0c3160 con 0x7fb0ac151a10 2024-09-16T00:08:28.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.950+0000 7fb0b0e37640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb0ac0c3440 con 0x7fb0ac0c3c40 2024-09-16T00:08:28.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.950+0000 7fb0aad9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb0ac0c5d60 0x7fb0ac0c86e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:50254/0 (socket says 172.21.15.139:50254) 2024-09-16T00:08:28.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.950+0000 7fb0aa59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb0ac151a10 0x7fb0ac067950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:28.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.950+0000 7fb0aad9d640 1 -- 172.21.15.139:0/3382291084 learned_addr learned my addr 172.21.15.139:0/3382291084 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:28.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.950+0000 7fb0a9d9b640 1 --2- 172.21.15.139:0/3382291084 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0ac0c3c40 0x7fb0ac0c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:28.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.950+0000 7fb0a9d9b640 1 -- 172.21.15.139:0/3382291084 >> v2:172.21.15.139:3300/0 conn(0x7fb0ac151a10 msgr2=0x7fb0ac067950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:28.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.950+0000 7fb0a9d9b640 1 --2- 172.21.15.139:0/3382291084 >> v2:172.21.15.139:3300/0 conn(0x7fb0ac151a10 0x7fb0ac067950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.950+0000 7fb0a9d9b640 1 -- 172.21.15.139:0/3382291084 >> v1:172.21.15.139:6789/0 conn(0x7fb0ac0c5d60 legacy=0x7fb0ac0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:28.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.950+0000 7fb0a9d9b640 1 -- 172.21.15.139:0/3382291084 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac0c4110 con 0x7fb0ac0c3c40 2024-09-16T00:08:28.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.951+0000 7fb0aa59c640 1 --2- 172.21.15.139:0/3382291084 >> v2:172.21.15.139:3300/0 conn(0x7fb0ac151a10 0x7fb0ac067950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:28.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.952+0000 7fb0a9d9b640 1 --2- 172.21.15.139:0/3382291084 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0ac0c3c40 0x7fb0ac0c5820 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00097a0 tx=0x7fb0a002a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=858abd789911c2db server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:28.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.952+0000 7fb0a959a640 1 -- 172.21.15.139:0/3382291084 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0037070 con 0x7fb0ac0c3c40 2024-09-16T00:08:28.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.952+0000 7fb0a959a640 1 -- 172.21.15.139:0/3382291084 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0a0032440 con 0x7fb0ac0c3c40 2024-09-16T00:08:28.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.952+0000 7fb0a959a640 1 -- 172.21.15.139:0/3382291084 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0032810 con 0x7fb0ac0c3c40 2024-09-16T00:08:28.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.953+0000 7fb0b0e37640 1 -- 172.21.15.139:0/3382291084 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0ac0c3c40 msgr2=0x7fb0ac0c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:28.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.953+0000 7fb0b0e37640 1 --2- 172.21.15.139:0/3382291084 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0ac0c3c40 0x7fb0ac0c5820 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00097a0 tx=0x7fb0a002a230 comp rx=0 tx=0).stop 2024-09-16T00:08:28.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.953+0000 7fb0b0e37640 1 -- 172.21.15.139:0/3382291084 shutdown_connections 2024-09-16T00:08:28.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.953+0000 7fb0b0e37640 1 --2- 172.21.15.139:0/3382291084 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0ac0c3c40 0x7fb0ac0c5820 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.953+0000 7fb0b0e37640 1 --2- 172.21.15.139:0/3382291084 >> v2:172.21.15.139:3300/0 conn(0x7fb0ac151a10 0x7fb0ac067950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.953+0000 7fb0b0e37640 1 -- 172.21.15.139:0/3382291084 >> 172.21.15.139:0/3382291084 conn(0x7fb0ac0cc740 msgr2=0x7fb0ac0ccb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:28.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.953+0000 7fb0b0e37640 1 -- 172.21.15.139:0/3382291084 shutdown_connections 2024-09-16T00:08:28.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.953+0000 7fb0b0e37640 1 -- 172.21.15.139:0/3382291084 wait complete. 2024-09-16T00:08:28.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.956+0000 7fb0b0e37640 1 Processor -- start 2024-09-16T00:08:28.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.956+0000 7fb0b0e37640 1 -- start start 2024-09-16T00:08:28.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.957+0000 7fb0b0e37640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0ac0c3c40 0x7fb0ac13fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:28.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.957+0000 7fb0b0e37640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0ac0c5d60 0x7fb0ac153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:28.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.958+0000 7fb0b0e37640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ac151a10 0x7fb0ac153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:28.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.958+0000 7fb0aa59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0ac0c3c40 0x7fb0ac13fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:28.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.958+0000 7fb0aa59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0ac0c3c40 0x7fb0ac13fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:34050/0 (socket says 172.21.15.139:34050) 2024-09-16T00:08:28.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.958+0000 7fb0a9d9b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0ac0c5d60 0x7fb0ac153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:28.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.958+0000 7fb0a9d9b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0ac0c5d60 0x7fb0ac153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:59942/0 (socket says 172.21.15.139:59942) 2024-09-16T00:08:28.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.958+0000 7fb0aad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ac151a10 0x7fb0ac153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:28.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.958+0000 7fb0aad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ac151a10 0x7fb0ac153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43062/0 (socket says 172.21.15.139:43062) 2024-09-16T00:08:28.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.958+0000 7fb0aa59c640 1 -- 172.21.15.139:0/2086386987 learned_addr learned my addr 172.21.15.139:0/2086386987 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:28.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.958+0000 7fb0b0e37640 1 -- 172.21.15.139:0/2086386987 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac151df0 con 0x7fb0ac151a10 2024-09-16T00:08:28.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.958+0000 7fb0b0e37640 1 -- 172.21.15.139:0/2086386987 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb0ac06d1f0 con 0x7fb0ac0c3c40 2024-09-16T00:08:28.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.958+0000 7fb0b0e37640 1 -- 172.21.15.139:0/2086386987 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb0ac0c8df0 con 0x7fb0ac0c5d60 2024-09-16T00:08:28.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.959+0000 7fb0aad9d640 1 -- 172.21.15.139:0/2086386987 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0ac0c5d60 msgr2=0x7fb0ac153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:28.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.959+0000 7fb0aad9d640 1 --2- 172.21.15.139:0/2086386987 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0ac0c5d60 0x7fb0ac153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.959+0000 7fb0aad9d640 1 -- 172.21.15.139:0/2086386987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0ac0c3c40 msgr2=0x7fb0ac13fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:28.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.959+0000 7fb0aad9d640 1 --2- 172.21.15.139:0/2086386987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0ac0c3c40 0x7fb0ac13fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:28.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.959+0000 7fb0aad9d640 1 -- 172.21.15.139:0/2086386987 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0ac152260 con 0x7fb0ac151a10 2024-09-16T00:08:28.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.959+0000 7fb0aa59c640 1 --2- 172.21.15.139:0/2086386987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0ac0c3c40 0x7fb0ac13fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:28.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.959+0000 7fb0aad9d640 1 --2- 172.21.15.139:0/2086386987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ac151a10 0x7fb0ac153840 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400ca70 tx=0x7fb0a400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:28.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.960+0000 7fb09affd640 1 -- 172.21.15.139:0/2086386987 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4017070 con 0x7fb0ac151a10 2024-09-16T00:08:28.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.960+0000 7fb09affd640 1 -- 172.21.15.139:0/2086386987 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0a400ab70 con 0x7fb0ac151a10 2024-09-16T00:08:28.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.961+0000 7fb09affd640 1 -- 172.21.15.139:0/2086386987 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4012400 con 0x7fb0ac151a10 2024-09-16T00:08:28.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.961+0000 7fb0b0e37640 1 -- 172.21.15.139:0/2086386987 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0ac152540 con 0x7fb0ac151a10 2024-09-16T00:08:28.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.961+0000 7fb0b0e37640 1 -- 172.21.15.139:0/2086386987 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0ac152b00 con 0x7fb0ac151a10 2024-09-16T00:08:28.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.963+0000 7fb0b0e37640 1 -- 172.21.15.139:0/2086386987 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0ac0c7140 con 0x7fb0ac151a10 2024-09-16T00:08:28.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.963+0000 7fb09affd640 1 -- 172.21.15.139:0/2086386987 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb0a401a070 con 0x7fb0ac151a10 2024-09-16T00:08:28.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.963+0000 7fb09affd640 1 --2- 172.21.15.139:0/2086386987 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb08c06c2b0 0x7fb08c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:28.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.964+0000 7fb09affd640 1 -- 172.21.15.139:0/2086386987 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5915+0+0 (secure 0 0 0) 0x7fb0a408b100 con 0x7fb0ac151a10 2024-09-16T00:08:28.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.964+0000 7fb0aa59c640 1 --2- 172.21.15.139:0/2086386987 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb08c06c2b0 0x7fb08c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:28.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.964+0000 7fb0aa59c640 1 --2- 172.21.15.139:0/2086386987 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb08c06c2b0 0x7fb08c06e770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0095e0 tx=0x7fb09c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:28.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:28.972+0000 7fb09affd640 1 -- 172.21.15.139:0/2086386987 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb0a40590c0 con 0x7fb0ac151a10 2024-09-16T00:08:29.032 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:29.031+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:29.077 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:29.077 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:29.077 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:29.078 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:29.086 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:08:29.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.085+0000 7f736e45e640 1 Processor -- start 2024-09-16T00:08:29.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.085+0000 7f736e45e640 1 -- start start 2024-09-16T00:08:29.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.086+0000 7f736e45e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f736815b280 0x7f7368166190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:29.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.086+0000 7f736e45e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73680cc4b0 con 0x7f7368154c20 2024-09-16T00:08:29.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.086+0000 7f736e45e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73680cc790 con 0x7f7368068060 2024-09-16T00:08:29.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.086+0000 7f736e45e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f73680c5830 con 0x7f736815b280 2024-09-16T00:08:29.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.087+0000 7f73677fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f7368068060 0x7f73680ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:59750/0 (socket says 172.21.15.139:59750) 2024-09-16T00:08:29.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.087+0000 7f7367fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f7368154c20 0x7f73680c5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:50258/0 (socket says 172.21.15.139:50258) 2024-09-16T00:08:29.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.087+0000 7f73677fe640 1 -- 172.21.15.139:0/1219995492 learned_addr learned my addr 172.21.15.139:0/1219995492 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:29.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.087+0000 7f736c9fb640 1 --2- 172.21.15.139:0/1219995492 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f736815b280 0x7f7368166190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:29.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.087+0000 7f736c9fb640 1 -- 172.21.15.139:0/1219995492 >> v1:172.21.15.164:6789/0 conn(0x7f7368068060 legacy=0x7f73680ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:29.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.087+0000 7f736c9fb640 1 -- 172.21.15.139:0/1219995492 >> v1:172.21.15.139:6789/0 conn(0x7f7368154c20 legacy=0x7f73680c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:29.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.087+0000 7f736c9fb640 1 -- 172.21.15.139:0/1219995492 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736815ddf0 con 0x7f736815b280 2024-09-16T00:08:29.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.088+0000 7f736c9fb640 1 --2- 172.21.15.139:0/1219995492 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f736815b280 0x7f7368166190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7360009dc0 tx=0x7f736002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d2a73106727764f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:29.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.088+0000 7f7366ffd640 1 -- 172.21.15.139:0/1219995492 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360007740 con 0x7f736815b280 2024-09-16T00:08:29.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.088+0000 7f7366ffd640 1 -- 172.21.15.139:0/1219995492 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73600078e0 con 0x7f736815b280 2024-09-16T00:08:29.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.088+0000 7f7366ffd640 1 -- 172.21.15.139:0/1219995492 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360007cb0 con 0x7f736815b280 2024-09-16T00:08:29.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.089+0000 7f736e45e640 1 -- 172.21.15.139:0/1219995492 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f736815b280 msgr2=0x7f7368166190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:29.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.089+0000 7f736e45e640 1 --2- 172.21.15.139:0/1219995492 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f736815b280 0x7f7368166190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7360009dc0 tx=0x7f736002a5f0 comp rx=0 tx=0).stop 2024-09-16T00:08:29.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.089+0000 7f736e45e640 1 -- 172.21.15.139:0/1219995492 shutdown_connections 2024-09-16T00:08:29.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.089+0000 7f736e45e640 1 --2- 172.21.15.139:0/1219995492 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f736815b280 0x7f7368166190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:29.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.089+0000 7f736e45e640 1 -- 172.21.15.139:0/1219995492 >> 172.21.15.139:0/1219995492 conn(0x7f736806ca40 msgr2=0x7f7368067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:29.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.089+0000 7f736e45e640 1 -- 172.21.15.139:0/1219995492 shutdown_connections 2024-09-16T00:08:29.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.089+0000 7f736e45e640 1 -- 172.21.15.139:0/1219995492 wait complete. 2024-09-16T00:08:29.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.091+0000 7f736e45e640 1 Processor -- start 2024-09-16T00:08:29.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.091+0000 7f736e45e640 1 -- start start 2024-09-16T00:08:29.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.093+0000 7f736e45e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7368068060 0x7f73680c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:29.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.093+0000 7f7367fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7368068060 0x7f73680c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:29.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.093+0000 7f7367fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7368068060 0x7f73680c98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43070/0 (socket says 172.21.15.139:43070) 2024-09-16T00:08:29.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.093+0000 7f7367fff640 1 -- 172.21.15.139:0/2996540618 learned_addr learned my addr 172.21.15.139:0/2996540618 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:29.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.093+0000 7f736e45e640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7368154c20 0x7f73681f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:29.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.093+0000 7f736e45e640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f736815b280 0x7f73680c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:29.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.093+0000 7f736e45e640 1 -- 172.21.15.139:0/2996540618 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f736815e3b0 con 0x7f7368068060 2024-09-16T00:08:29.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.093+0000 7f736e45e640 1 -- 172.21.15.139:0/2996540618 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f73680cc260 con 0x7f7368154c20 2024-09-16T00:08:29.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.093+0000 7f736e45e640 1 -- 172.21.15.139:0/2996540618 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f736815e690 con 0x7f736815b280 2024-09-16T00:08:29.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.094+0000 7f73677fe640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7368154c20 0x7f73681f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:29.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.094+0000 7f736c9fb640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f736815b280 0x7f73680c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:29.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.094+0000 7f73677fe640 1 -- 172.21.15.139:0/2996540618 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f736815b280 msgr2=0x7f73680c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:29.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.094+0000 7f73677fe640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f736815b280 0x7f73680c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:29.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.094+0000 7f73677fe640 1 -- 172.21.15.139:0/2996540618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7368068060 msgr2=0x7f73680c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:29.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.094+0000 7f73677fe640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7368068060 0x7f73680c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:29.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.094+0000 7f73677fe640 1 -- 172.21.15.139:0/2996540618 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736814a3a0 con 0x7f7368154c20 2024-09-16T00:08:29.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.094+0000 7f7367fff640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7368068060 0x7f73680c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:29.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.094+0000 7f73677fe640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7368154c20 0x7f73681f40a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f735c00c570 tx=0x7f735c00ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:29.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.094+0000 7f7364ff9640 1 -- 172.21.15.139:0/2996540618 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c018070 con 0x7f7368154c20 2024-09-16T00:08:29.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.095+0000 7f7364ff9640 1 -- 172.21.15.139:0/2996540618 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f735c0074e0 con 0x7f7368154c20 2024-09-16T00:08:29.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.095+0000 7f7364ff9640 1 -- 172.21.15.139:0/2996540618 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735c004a50 con 0x7f7368154c20 2024-09-16T00:08:29.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.095+0000 7f736e45e640 1 -- 172.21.15.139:0/2996540618 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73681f4960 con 0x7f7368154c20 2024-09-16T00:08:29.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.095+0000 7f736e45e640 1 -- 172.21.15.139:0/2996540618 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73680cb460 con 0x7f7368154c20 2024-09-16T00:08:29.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.096+0000 7f736e45e640 1 -- 172.21.15.139:0/2996540618 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73681f4ba0 con 0x7f7368154c20 2024-09-16T00:08:29.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.097+0000 7f7364ff9640 1 -- 172.21.15.139:0/2996540618 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f735c004bf0 con 0x7f7368154c20 2024-09-16T00:08:29.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.097+0000 7f7364ff9640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f735406c2e0 0x7f735406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:29.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.098+0000 7f7364ff9640 1 -- 172.21.15.139:0/2996540618 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5915+0+0 (secure 0 0 0) 0x7f735c08a720 con 0x7f7368154c20 2024-09-16T00:08:29.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.098+0000 7f7367fff640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f735406c2e0 0x7f735406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:29.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.098+0000 7f7367fff640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f735406c2e0 0x7f735406e7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7358004e00 tx=0x7f7358007f10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:29.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.102+0000 7f7364ff9640 1 -- 172.21.15.139:0/2996540618 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f73681f4ba0 con 0x7f7368154c20 2024-09-16T00:08:29.170 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:29.169+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:29.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.207+0000 7fb0b0e37640 1 -- 172.21.15.139:0/2086386987 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) -- 0x7fb0ac067f30 con 0x7fb0ac151a10 2024-09-16T00:08:29.230 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:29.230 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:29.231 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:29.231 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:29.231 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:29.234 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:29.233+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:29.240 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:08:29.335 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:29.334+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:29.385 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:29.385 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:29.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:29.385 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:29.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.386 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:14 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:29.386 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:29.386 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.768289946000003, 2024-09-16T00:08:29.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029190470000000001, 2024-09-16T00:08:29.386 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.386 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.386 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.386 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:29.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:29.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615982+0000", 2024-09-16T00:08:29.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7439999999999999e-06 2024-09-16T00:08:29.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:08:29.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3743999999999999e-05 2024-09-16T00:08:29.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:29.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2255e-05 2024-09-16T00:08:29.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616029+0000", 2024-09-16T00:08:29.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1175999999999999e-05 2024-09-16T00:08:29.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616098+0000", 2024-09-16T00:08:29.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9153999999999996e-05 2024-09-16T00:08:29.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616132+0000", 2024-09-16T00:08:29.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3924000000000002e-05 2024-09-16T00:08:29.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:29.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027226189999999999 2024-09-16T00:08:29.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618899+0000", 2024-09-16T00:08:29.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4431000000000002e-05 2024-09-16T00:08:29.392 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.392 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.392 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.392 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:15 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:29.392 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:29.392 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.766063125999999, 2024-09-16T00:08:29.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095618099999999996, 2024-09-16T00:08:29.393 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.393 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.393 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.393 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:29.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:29.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618208+0000", 2024-09-16T00:08:29.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1850000000000001e-06 2024-09-16T00:08:29.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618217+0000", 2024-09-16T00:08:29.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439999999999992e-06 2024-09-16T00:08:29.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618230+0000", 2024-09-16T00:08:29.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2435999999999999e-05 2024-09-16T00:08:29.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618238+0000", 2024-09-16T00:08:29.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1319999999999994e-06 2024-09-16T00:08:29.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618331+0000", 2024-09-16T00:08:29.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3453000000000001e-05 2024-09-16T00:08:29.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618352+0000", 2024-09-16T00:08:29.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0693999999999999e-05 2024-09-16T00:08:29.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619112+0000", 2024-09-16T00:08:29.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00076007099999999997 2024-09-16T00:08:29.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619163+0000", 2024-09-16T00:08:29.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1066000000000002e-05 2024-09-16T00:08:29.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.397 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.397 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:30 2.15 e22/19 2:aa448500:::500.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:29.397 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:29.397 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.766005641, 2024-09-16T00:08:29.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058031250000000001, 2024-09-16T00:08:29.397 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.397 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.397 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.398 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:29.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:29.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618265+0000", 2024-09-16T00:08:29.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9200000000000001e-07 2024-09-16T00:08:29.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618271+0000", 2024-09-16T00:08:29.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9580000000000004e-06 2024-09-16T00:08:29.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618279+0000", 2024-09-16T00:08:29.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2239999999999995e-06 2024-09-16T00:08:29.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618285+0000", 2024-09-16T00:08:29.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-06 2024-09-16T00:08:29.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618567+0000", 2024-09-16T00:08:29.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028204400000000001 2024-09-16T00:08:29.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:08:29.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9678999999999999e-05 2024-09-16T00:08:29.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624024+0000", 2024-09-16T00:08:29.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054376720000000002 2024-09-16T00:08:29.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624068+0000", 2024-09-16T00:08:29.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:08:29.401 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.401 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.401 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.401 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:20 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:29.402 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:29.402 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.765649277000001, 2024-09-16T00:08:29.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028808739999999998, 2024-09-16T00:08:29.402 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.402 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.402 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.402 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:29.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:29.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618622+0000", 2024-09-16T00:08:29.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1260000000000001e-06 2024-09-16T00:08:29.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618631+0000", 2024-09-16T00:08:29.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4700000000000008e-06 2024-09-16T00:08:29.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618645+0000", 2024-09-16T00:08:29.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3984e-05 2024-09-16T00:08:29.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:08:29.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2101e-05 2024-09-16T00:08:29.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618720+0000", 2024-09-16T00:08:29.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2761000000000004e-05 2024-09-16T00:08:29.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618740+0000", 2024-09-16T00:08:29.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9409e-05 2024-09-16T00:08:29.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621436+0000", 2024-09-16T00:08:29.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002696211 2024-09-16T00:08:29.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621502+0000", 2024-09-16T00:08:29.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5811999999999999e-05 2024-09-16T00:08:29.406 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.406 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.406 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.406 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:21 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:29.406 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:29.406 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.764825989000002, 2024-09-16T00:08:29.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046318540000000004, 2024-09-16T00:08:29.406 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.407 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.407 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.407 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:29.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:29.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619445+0000", 2024-09-16T00:08:29.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.083e-06 2024-09-16T00:08:29.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619455+0000", 2024-09-16T00:08:29.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5920000000000007e-06 2024-09-16T00:08:29.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619467+0000", 2024-09-16T00:08:29.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2711000000000001e-05 2024-09-16T00:08:29.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619476+0000", 2024-09-16T00:08:29.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0139999999999998e-06 2024-09-16T00:08:29.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619529+0000", 2024-09-16T00:08:29.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3078999999999998e-05 2024-09-16T00:08:29.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619548+0000", 2024-09-16T00:08:29.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8952000000000001e-05 2024-09-16T00:08:29.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624037+0000", 2024-09-16T00:08:29.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044895730000000002 2024-09-16T00:08:29.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624076+0000", 2024-09-16T00:08:29.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8850000000000002e-05 2024-09-16T00:08:29.410 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.410 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.410 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.411 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.14 38/37 [PushOp(2:2f55791f:::606.00000000:head, version: 22'2, data_included: [], data_size: 0, omap_header_size: 274, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(2:2f55791f:::606.00000000:head@22'2, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:29.411 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.838548+0000", 2024-09-16T00:08:29.411 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.545722091, 2024-09-16T00:08:29.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044033700000000001, 2024-09-16T00:08:29.411 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.411 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.411 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:29.411 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838548+0000", 2024-09-16T00:08:29.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838548+0000", 2024-09-16T00:08:29.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838549+0000", 2024-09-16T00:08:29.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5600000000000004e-07 2024-09-16T00:08:29.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838567+0000", 2024-09-16T00:08:29.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7850999999999999e-05 2024-09-16T00:08:29.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838597+0000", 2024-09-16T00:08:29.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0416000000000002e-05 2024-09-16T00:08:29.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838619+0000", 2024-09-16T00:08:29.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1274e-05 2024-09-16T00:08:29.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838706+0000", 2024-09-16T00:08:29.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7755000000000003e-05 2024-09-16T00:08:29.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838721+0000", 2024-09-16T00:08:29.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4953999999999999e-05 2024-09-16T00:08:29.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838988+0000", 2024-09-16T00:08:29.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026713099999999999 2024-09-16T00:08:29.415 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.415 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.415 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.416 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.14 38/37 [PushOp(2:287034c4:::1.00000000.inode:head, version: 22'4, data_included: [0~582], data_size: 582, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:287034c4:::1.00000000.inode:head@22'4, size: 582, copy_subset: [(0, 582)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 582, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:29.416 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.840824+0000", 2024-09-16T00:08:29.416 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.5434462509999998, 2024-09-16T00:08:29.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00055177600000000002, 2024-09-16T00:08:29.416 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.416 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.416 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:29.416 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840824+0000", 2024-09-16T00:08:29.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840824+0000", 2024-09-16T00:08:29.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840825+0000", 2024-09-16T00:08:29.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1100000000000005e-07 2024-09-16T00:08:29.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840840+0000", 2024-09-16T00:08:29.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5308999999999999e-05 2024-09-16T00:08:29.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840863+0000", 2024-09-16T00:08:29.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2663999999999998e-05 2024-09-16T00:08:29.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840884+0000", 2024-09-16T00:08:29.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1798e-05 2024-09-16T00:08:29.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840980+0000", 2024-09-16T00:08:29.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5299999999999999e-05 2024-09-16T00:08:29.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840995+0000", 2024-09-16T00:08:29.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4895999999999999e-05 2024-09-16T00:08:29.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.841376+0000", 2024-09-16T00:08:29.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038099800000000001 2024-09-16T00:08:29.420 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.421 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.421 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.421 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.14 38/37 [PushOp(2:2e2fa760:::200.00000001:head, version: 28'9, data_included: [0~5883], data_size: 5883, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:2e2fa760:::200.00000001:head@28'9, size: 5883, copy_subset: [(0, 5883)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5883, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:29.421 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.848988+0000", 2024-09-16T00:08:29.421 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.5352819310000001, 2024-09-16T00:08:29.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00037334, 2024-09-16T00:08:29.421 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.421 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.422 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:29.422 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.848988+0000", 2024-09-16T00:08:29.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.848988+0000", 2024-09-16T00:08:29.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.848989+0000", 2024-09-16T00:08:29.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5099999999999998e-07 2024-09-16T00:08:29.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849004+0000", 2024-09-16T00:08:29.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5192e-05 2024-09-16T00:08:29.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849023+0000", 2024-09-16T00:08:29.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.844e-05 2024-09-16T00:08:29.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849033+0000", 2024-09-16T00:08:29.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0061000000000001e-05 2024-09-16T00:08:29.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849096+0000", 2024-09-16T00:08:29.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3627999999999995e-05 2024-09-16T00:08:29.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849105+0000", 2024-09-16T00:08:29.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5890000000000003e-06 2024-09-16T00:08:29.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849362+0000", 2024-09-16T00:08:29.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000256579 2024-09-16T00:08:29.425 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.425 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.425 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.426 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.14 38/37 [PushOp(2:292cf221:::200.00000000:head, version: 28'10, data_included: [0~90], data_size: 90, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:292cf221:::200.00000000:head@28'10, size: 90, copy_subset: [(0, 90)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 90, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:29.426 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.851655+0000", 2024-09-16T00:08:29.426 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.5326152659999996, 2024-09-16T00:08:29.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036283000000000002, 2024-09-16T00:08:29.426 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.426 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.426 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:29.426 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851655+0000", 2024-09-16T00:08:29.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851655+0000", 2024-09-16T00:08:29.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851656+0000", 2024-09-16T00:08:29.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0399999999999995e-07 2024-09-16T00:08:29.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851672+0000", 2024-09-16T00:08:29.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6801000000000001e-05 2024-09-16T00:08:29.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851689+0000", 2024-09-16T00:08:29.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6810999999999999e-05 2024-09-16T00:08:29.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851699+0000", 2024-09-16T00:08:29.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.995e-06 2024-09-16T00:08:29.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851767+0000", 2024-09-16T00:08:29.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8194000000000005e-05 2024-09-16T00:08:29.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851777+0000", 2024-09-16T00:08:29.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2450000000000004e-06 2024-09-16T00:08:29.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.852018+0000", 2024-09-16T00:08:29.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024107999999999999 2024-09-16T00:08:29.431 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.431 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.431 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.431 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.431 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.431 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:29.432 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:08:29.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.466+0000 7f736e45e640 1 -- 172.21.15.139:0/2996540618 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f73680686b0 con 0x7f7368154c20 2024-09-16T00:08:29.473 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:29.472+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:29.538 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:29.537+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:29.577 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:29.577 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:29.577 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:29.577 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:29.586 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:08:29.638 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:29.638+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:29.692 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:29.692 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:29.692 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:29.701 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:08:29.745 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:29.744+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:29.807 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.189907093999999, 2024-09-16T00:08:29.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056100969999999997, 2024-09-16T00:08:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:29.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615352+0000", 2024-09-16T00:08:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9859999999999999e-06 2024-09-16T00:08:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615363+0000", 2024-09-16T00:08:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1168999999999999e-05 2024-09-16T00:08:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615382+0000", 2024-09-16T00:08:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8224e-05 2024-09-16T00:08:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615393+0000", 2024-09-16T00:08:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0925e-05 2024-09-16T00:08:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615624+0000", 2024-09-16T00:08:29.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023159600000000001 2024-09-16T00:08:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615721+0000", 2024-09-16T00:08:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7040000000000006e-05 2024-09-16T00:08:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615885+0000", 2024-09-16T00:08:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163456 2024-09-16T00:08:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:08:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002191368 2024-09-16T00:08:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620870+0000", 2024-09-16T00:08:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941839999999999 2024-09-16T00:08:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620922+0000", 2024-09-16T00:08:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1641999999999998e-05 2024-09-16T00:08:29.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620956+0000", 2024-09-16T00:08:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4507e-05 2024-09-16T00:08:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.189816735000001, 2024-09-16T00:08:29.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005695739, 2024-09-16T00:08:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4900000000000005e-07 2024-09-16T00:08:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615442+0000", 2024-09-16T00:08:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.994e-06 2024-09-16T00:08:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:08:29.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1081e-05 2024-09-16T00:08:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615473+0000", 2024-09-16T00:08:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1910000000000006e-06 2024-09-16T00:08:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616230+0000", 2024-09-16T00:08:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075700400000000003 2024-09-16T00:08:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616305+0000", 2024-09-16T00:08:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4759999999999996e-05 2024-09-16T00:08:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616466+0000", 2024-09-16T00:08:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000161714 2024-09-16T00:08:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619821+0000", 2024-09-16T00:08:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033547339999999998 2024-09-16T00:08:29.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621028+0000", 2024-09-16T00:08:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012067530000000001 2024-09-16T00:08:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621087+0000", 2024-09-16T00:08:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9364999999999998e-05 2024-09-16T00:08:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621132+0000", 2024-09-16T00:08:29.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5293999999999997e-05 2024-09-16T00:08:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.189754503000003, 2024-09-16T00:08:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050565760000000001, 2024-09-16T00:08:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.819 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615500+0000", 2024-09-16T00:08:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8199999999999999e-07 2024-09-16T00:08:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615505+0000", 2024-09-16T00:08:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5280000000000003e-06 2024-09-16T00:08:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0317e-05 2024-09-16T00:08:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615523+0000", 2024-09-16T00:08:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4580000000000004e-06 2024-09-16T00:08:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615607+0000", 2024-09-16T00:08:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4153999999999994e-05 2024-09-16T00:08:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615748+0000", 2024-09-16T00:08:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140533 2024-09-16T00:08:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615911+0000", 2024-09-16T00:08:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163678 2024-09-16T00:08:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619944+0000", 2024-09-16T00:08:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040330909999999999 2024-09-16T00:08:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620471+0000", 2024-09-16T00:08:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052645600000000002 2024-09-16T00:08:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620512+0000", 2024-09-16T00:08:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0738000000000002e-05 2024-09-16T00:08:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620555+0000", 2024-09-16T00:08:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3841000000000003e-05 2024-09-16T00:08:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.189702396000001, 2024-09-16T00:08:29.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089162479999999999, 2024-09-16T00:08:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:29.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615552+0000", 2024-09-16T00:08:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:08:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615556+0000", 2024-09-16T00:08:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3610000000000003e-06 2024-09-16T00:08:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615566+0000", 2024-09-16T00:08:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6099999999999995e-06 2024-09-16T00:08:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615574+0000", 2024-09-16T00:08:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9379999999999992e-06 2024-09-16T00:08:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616777+0000", 2024-09-16T00:08:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012031640000000001 2024-09-16T00:08:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616882+0000", 2024-09-16T00:08:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105146 2024-09-16T00:08:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619696+0000", 2024-09-16T00:08:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028142179999999998 2024-09-16T00:08:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624234+0000", 2024-09-16T00:08:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045381889999999998 2024-09-16T00:08:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624393+0000", 2024-09-16T00:08:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015871200000000001 2024-09-16T00:08:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624434+0000", 2024-09-16T00:08:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0908999999999997e-05 2024-09-16T00:08:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624467+0000", 2024-09-16T00:08:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3217999999999999e-05 2024-09-16T00:08:29.832 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:29.833 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.189632598999999, 2024-09-16T00:08:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064898359999999997, 2024-09-16T00:08:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615622+0000", 2024-09-16T00:08:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.181e-06 2024-09-16T00:08:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615628+0000", 2024-09-16T00:08:29.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2450000000000003e-06 2024-09-16T00:08:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:08:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2554999999999999e-05 2024-09-16T00:08:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615650+0000", 2024-09-16T00:08:29.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4350000000000003e-06 2024-09-16T00:08:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616772+0000", 2024-09-16T00:08:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001122058 2024-09-16T00:08:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616889+0000", 2024-09-16T00:08:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116623 2024-09-16T00:08:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617035+0000", 2024-09-16T00:08:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014579799999999999 2024-09-16T00:08:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619829+0000", 2024-09-16T00:08:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941379999999998 2024-09-16T00:08:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622024+0000", 2024-09-16T00:08:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021949920000000002 2024-09-16T00:08:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622087+0000", 2024-09-16T00:08:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3048999999999999e-05 2024-09-16T00:08:29.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622111+0000", 2024-09-16T00:08:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3762000000000001e-05 2024-09-16T00:08:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.840 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.189570037999999, 2024-09-16T00:08:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065192160000000004, 2024-09-16T00:08:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.841 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615684+0000", 2024-09-16T00:08:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8299999999999995e-07 2024-09-16T00:08:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615691+0000", 2024-09-16T00:08:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.866e-06 2024-09-16T00:08:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615707+0000", 2024-09-16T00:08:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6073999999999999e-05 2024-09-16T00:08:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615714+0000", 2024-09-16T00:08:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-16T00:08:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617403+0000", 2024-09-16T00:08:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016881629999999999 2024-09-16T00:08:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617559+0000", 2024-09-16T00:08:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015669699999999999 2024-09-16T00:08:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000144165 2024-09-16T00:08:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621413+0000", 2024-09-16T00:08:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037092200000000001 2024-09-16T00:08:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622147+0000", 2024-09-16T00:08:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073468800000000005 2024-09-16T00:08:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622180+0000", 2024-09-16T00:08:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2901999999999998e-05 2024-09-16T00:08:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622203+0000", 2024-09-16T00:08:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2211000000000001e-05 2024-09-16T00:08:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.189526944000001, 2024-09-16T00:08:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065825270000000003, 2024-09-16T00:08:29.848 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.849 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.849 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:29.849 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:29.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:29.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:08:29.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7299999999999996e-07 2024-09-16T00:08:29.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615730+0000", 2024-09-16T00:08:29.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.004e-06 2024-09-16T00:08:29.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615737+0000", 2024-09-16T00:08:29.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4789999999999999e-06 2024-09-16T00:08:29.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615745+0000", 2024-09-16T00:08:29.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6699999999999994e-06 2024-09-16T00:08:29.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618115+0000", 2024-09-16T00:08:29.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023701569999999999 2024-09-16T00:08:29.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618247+0000", 2024-09-16T00:08:29.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013185000000000001 2024-09-16T00:08:29.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622309+0000", 2024-09-16T00:08:29.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040617939999999996 2024-09-16T00:08:29.853 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.853 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.853 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.853 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:29.853 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:29.853 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.189499695000002, 2024-09-16T00:08:29.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036685950000000002, 2024-09-16T00:08:29.854 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.854 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.854 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:29.854 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:29.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:29.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:29.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T00:08:29.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615763+0000", 2024-09-16T00:08:29.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4360000000000002e-06 2024-09-16T00:08:29.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615770+0000", 2024-09-16T00:08:29.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7540000000000006e-06 2024-09-16T00:08:29.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615803+0000", 2024-09-16T00:08:29.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3008e-05 2024-09-16T00:08:29.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615888+0000", 2024-09-16T00:08:29.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5056999999999996e-05 2024-09-16T00:08:29.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:08:29.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013059700000000001 2024-09-16T00:08:29.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619422+0000", 2024-09-16T00:08:29.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034033230000000002 2024-09-16T00:08:29.858 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.858 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.858 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.858 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing_reply(2.1a epoch 22/19 rep_tid 2 lcod 22'1)", 2024-09-16T00:08:29.858 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:29.859 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.186058381999999, 2024-09-16T00:08:29.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024304200000000001, 2024-09-16T00:08:29.859 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.859 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.859 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:29.859 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:29.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:29.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619196+0000", 2024-09-16T00:08:29.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2759999999999999e-06 2024-09-16T00:08:29.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619203+0000", 2024-09-16T00:08:29.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1779999999999997e-06 2024-09-16T00:08:29.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619223+0000", 2024-09-16T00:08:29.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9159000000000001e-05 2024-09-16T00:08:29.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619231+0000", 2024-09-16T00:08:29.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7830000000000004e-06 2024-09-16T00:08:29.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619320+0000", 2024-09-16T00:08:29.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8805000000000001e-05 2024-09-16T00:08:29.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619438+0000", 2024-09-16T00:08:29.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117841 2024-09-16T00:08:29.863 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.863 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.863 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.863 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:16 2.1d e22/19 2:b871830b:::605.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:29.863 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:29.863 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.185511105000003, 2024-09-16T00:08:29.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045616900000000002, 2024-09-16T00:08:29.863 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.863 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.863 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.863 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:29.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:29.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:08:29.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:08:29.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619755+0000", 2024-09-16T00:08:29.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3599999999999996e-06 2024-09-16T00:08:29.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619767+0000", 2024-09-16T00:08:29.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2805e-05 2024-09-16T00:08:29.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619787+0000", 2024-09-16T00:08:29.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9264000000000001e-05 2024-09-16T00:08:29.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619869+0000", 2024-09-16T00:08:29.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2639999999999995e-05 2024-09-16T00:08:29.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619895+0000", 2024-09-16T00:08:29.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5890000000000001e-05 2024-09-16T00:08:29.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624264+0000", 2024-09-16T00:08:29.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043690049999999996 2024-09-16T00:08:29.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624304+0000", 2024-09-16T00:08:29.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9807e-05 2024-09-16T00:08:29.868 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.868 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.868 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.868 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:17 2.1d e22/19 2:b871830b:::605.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:29.868 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:29.869 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.185158395999999, 2024-09-16T00:08:29.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042370910000000001, 2024-09-16T00:08:29.869 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.869 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.869 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.869 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:29.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:29.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620096+0000", 2024-09-16T00:08:29.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:08:29.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620102+0000", 2024-09-16T00:08:29.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7220000000000002e-06 2024-09-16T00:08:29.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620112+0000", 2024-09-16T00:08:29.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0156999999999999e-05 2024-09-16T00:08:29.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620120+0000", 2024-09-16T00:08:29.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2880000000000004e-06 2024-09-16T00:08:29.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620171+0000", 2024-09-16T00:08:29.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.168e-05 2024-09-16T00:08:29.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620189+0000", 2024-09-16T00:08:29.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7419e-05 2024-09-16T00:08:29.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624306+0000", 2024-09-16T00:08:29.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041175430000000004 2024-09-16T00:08:29.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624332+0000", 2024-09-16T00:08:29.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5717000000000001e-05 2024-09-16T00:08:29.875 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.875 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.875 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.876 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:24 2.7 e22/19 ondisk, result = 0)", 2024-09-16T00:08:29.876 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:29.876 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.184892689000002, 2024-09-16T00:08:29.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000208018, 2024-09-16T00:08:29.876 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.876 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.877 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:29.877 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:29.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:29.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:29.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8500000000000001e-07 2024-09-16T00:08:29.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620366+0000", 2024-09-16T00:08:29.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5379999999999996e-06 2024-09-16T00:08:29.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620378+0000", 2024-09-16T00:08:29.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.204e-05 2024-09-16T00:08:29.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620385+0000", 2024-09-16T00:08:29.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.413e-06 2024-09-16T00:08:29.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620455+0000", 2024-09-16T00:08:29.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9886000000000001e-05 2024-09-16T00:08:29.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620463+0000", 2024-09-16T00:08:29.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1559999999999995e-06 2024-09-16T00:08:29.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620569+0000", 2024-09-16T00:08:29.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001054 2024-09-16T00:08:29.881 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.881 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.881 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.881 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:18 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:29.881 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:29.882 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.184507224999997, 2024-09-16T00:08:29.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021768100000000001, 2024-09-16T00:08:29.882 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.882 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.882 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:29.882 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:29.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:29.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620747+0000", 2024-09-16T00:08:29.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.02e-07 2024-09-16T00:08:29.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620752+0000", 2024-09-16T00:08:29.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4439999999999999e-06 2024-09-16T00:08:29.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620765+0000", 2024-09-16T00:08:29.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2714000000000001e-05 2024-09-16T00:08:29.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:08:29.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1160000000000004e-06 2024-09-16T00:08:29.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:29.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3374999999999994e-05 2024-09-16T00:08:29.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620864+0000", 2024-09-16T00:08:29.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4320000000000005e-06 2024-09-16T00:08:29.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620964+0000", 2024-09-16T00:08:29.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9698000000000007e-05 2024-09-16T00:08:29.885 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.885 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.885 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.886 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:19 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:29.886 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:29.886 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.184445681, 2024-09-16T00:08:29.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00033551400000000003, 2024-09-16T00:08:29.886 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.886 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.886 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:29.886 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:29.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:29.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:29.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0800000000000005e-07 2024-09-16T00:08:29.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620812+0000", 2024-09-16T00:08:29.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.427e-06 2024-09-16T00:08:29.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620820+0000", 2024-09-16T00:08:29.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6649999999999992e-06 2024-09-16T00:08:29.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620828+0000", 2024-09-16T00:08:29.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0010000000000001e-06 2024-09-16T00:08:29.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620995+0000", 2024-09-16T00:08:29.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016709400000000001 2024-09-16T00:08:29.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621005+0000", 2024-09-16T00:08:29.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7170000000000003e-06 2024-09-16T00:08:29.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621143+0000", 2024-09-16T00:08:29.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000138102 2024-09-16T00:08:29.889 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.889 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.889 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.890 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:3 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:29.890 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:29.890 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.183268409999997, 2024-09-16T00:08:29.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022247799999999999, 2024-09-16T00:08:29.890 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.890 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.890 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:29.890 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:29.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:29.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:29.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3300000000000003e-07 2024-09-16T00:08:29.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621989+0000", 2024-09-16T00:08:29.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2969999999999999e-06 2024-09-16T00:08:29.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621997+0000", 2024-09-16T00:08:29.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8990000000000001e-06 2024-09-16T00:08:29.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622002+0000", 2024-09-16T00:08:29.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5439999999999998e-06 2024-09-16T00:08:29.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622137+0000", 2024-09-16T00:08:29.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000135103 2024-09-16T00:08:29.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622143+0000", 2024-09-16T00:08:29.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9200000000000001e-06 2024-09-16T00:08:29.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622207+0000", 2024-09-16T00:08:29.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4282000000000006e-05 2024-09-16T00:08:29.893 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.894 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.894 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.894 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing_reply(2.14 epoch 22/19 rep_tid 6 lcod 22'5)", 2024-09-16T00:08:29.894 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:29.894 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.183237073999997, 2024-09-16T00:08:29.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029688000000000002, 2024-09-16T00:08:29.894 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.894 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.894 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:29.894 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:29.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:29.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622017+0000", 2024-09-16T00:08:29.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3200000000000001e-07 2024-09-16T00:08:29.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:08:29.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6240000000000002e-06 2024-09-16T00:08:29.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622026+0000", 2024-09-16T00:08:29.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.708e-06 2024-09-16T00:08:29.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622032+0000", 2024-09-16T00:08:29.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4699999999999999e-06 2024-09-16T00:08:29.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622230+0000", 2024-09-16T00:08:29.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019783800000000001 2024-09-16T00:08:29.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622313+0000", 2024-09-16T00:08:29.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2908e-05 2024-09-16T00:08:29.897 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.897 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.897 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.898 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:29.898 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:29.898 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.182791579000003, 2024-09-16T00:08:29.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044651079999999998, 2024-09-16T00:08:29.898 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.898 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.898 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.898 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622463+0000", 2024-09-16T00:08:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5349999999999999e-06 2024-09-16T00:08:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622470+0000", 2024-09-16T00:08:29.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7050000000000002e-06 2024-09-16T00:08:29.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622485+0000", 2024-09-16T00:08:29.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4978e-05 2024-09-16T00:08:29.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622493+0000", 2024-09-16T00:08:29.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2139999999999996e-06 2024-09-16T00:08:29.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622546+0000", 2024-09-16T00:08:29.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:08:29.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622595+0000", 2024-09-16T00:08:29.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9112e-05 2024-09-16T00:08:29.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:29.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622677+0000", 2024-09-16T00:08:29.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2332999999999999e-05 2024-09-16T00:08:29.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:29.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.623515+0000", 2024-09-16T00:08:29.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000837615 2024-09-16T00:08:29.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:29.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626867+0000", 2024-09-16T00:08:29.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033524819999999999 2024-09-16T00:08:29.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626902+0000", 2024-09-16T00:08:29.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4310000000000002e-05 2024-09-16T00:08:29.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626927+0000", 2024-09-16T00:08:29.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5128999999999999e-05 2024-09-16T00:08:29.903 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.903 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.903 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.904 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:39 2.6 e22/19 2:654134d2:::mds0_openfiles.0:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:29.904 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:29.904 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.164162302000001, 2024-09-16T00:08:29.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082120900000000004, 2024-09-16T00:08:29.904 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.904 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.904 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.904 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:29.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:29.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641092+0000", 2024-09-16T00:08:29.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.141e-06 2024-09-16T00:08:29.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641102+0000", 2024-09-16T00:08:29.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5990000000000008e-06 2024-09-16T00:08:29.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641116+0000", 2024-09-16T00:08:29.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4667999999999999e-05 2024-09-16T00:08:29.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641125+0000", 2024-09-16T00:08:29.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0849999999999997e-06 2024-09-16T00:08:29.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641188+0000", 2024-09-16T00:08:29.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3119000000000003e-05 2024-09-16T00:08:29.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641209+0000", 2024-09-16T00:08:29.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0897e-05 2024-09-16T00:08:29.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641874+0000", 2024-09-16T00:08:29.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00066517200000000001 2024-09-16T00:08:29.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641912+0000", 2024-09-16T00:08:29.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8528000000000001e-05 2024-09-16T00:08:29.908 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.908 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.908 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.908 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:40 2.14 2:2e2fa760:::200.00000001:head [write 940~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:29.908 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:29.908 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.965926185999997, 2024-09-16T00:08:29.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016643039999999999, 2024-09-16T00:08:29.908 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.908 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.909 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.909 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:29.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:29.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839329+0000", 2024-09-16T00:08:29.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2330000000000001e-06 2024-09-16T00:08:29.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839343+0000", 2024-09-16T00:08:29.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3634e-05 2024-09-16T00:08:29.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839362+0000", 2024-09-16T00:08:29.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9321999999999999e-05 2024-09-16T00:08:29.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839375+0000", 2024-09-16T00:08:29.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2945000000000001e-05 2024-09-16T00:08:29.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839462+0000", 2024-09-16T00:08:29.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6371000000000004e-05 2024-09-16T00:08:29.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839547+0000", 2024-09-16T00:08:29.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4988999999999996e-05 2024-09-16T00:08:29.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:29.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839657+0000", 2024-09-16T00:08:29.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110501 2024-09-16T00:08:29.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:29.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840598+0000", 2024-09-16T00:08:29.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00094087100000000003 2024-09-16T00:08:29.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:29.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840918+0000", 2024-09-16T00:08:29.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031968799999999999 2024-09-16T00:08:29.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840955+0000", 2024-09-16T00:08:29.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7360999999999997e-05 2024-09-16T00:08:29.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840991+0000", 2024-09-16T00:08:29.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6389e-05 2024-09-16T00:08:29.917 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.917 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.917 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.918 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:41 2.14 2:2e2fa760:::200.00000001:head [write 1383~2831 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:29.918 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:29.918 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 36.692272740999996, 2024-09-16T00:08:29.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017075809999999999, 2024-09-16T00:08:29.918 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:29.918 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:29.918 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:29.919 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:29.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:29.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:29.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:29.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:29.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:29.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:29.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112983+0000", 2024-09-16T00:08:29.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2560000000000001e-06 2024-09-16T00:08:29.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:29.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112993+0000", 2024-09-16T00:08:29.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0423e-05 2024-09-16T00:08:29.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:29.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113010+0000", 2024-09-16T00:08:29.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7169000000000001e-05 2024-09-16T00:08:29.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:29.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113023+0000", 2024-09-16T00:08:29.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2111e-05 2024-09-16T00:08:29.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:29.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113099+0000", 2024-09-16T00:08:29.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6798000000000006e-05 2024-09-16T00:08:29.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:29.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113174+0000", 2024-09-16T00:08:29.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.47e-05 2024-09-16T00:08:29.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:29.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113297+0000", 2024-09-16T00:08:29.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012250799999999999 2024-09-16T00:08:29.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:29.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114284+0000", 2024-09-16T00:08:29.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00098720399999999999 2024-09-16T00:08:29.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:29.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114604+0000", 2024-09-16T00:08:29.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031988699999999998 2024-09-16T00:08:29.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:29.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114644+0000", 2024-09-16T00:08:29.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9919999999999997e-05 2024-09-16T00:08:29.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:29.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:29.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:29.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114688+0000", 2024-09-16T00:08:29.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4604999999999997e-05 2024-09-16T00:08:29.924 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.924 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.924 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.924 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:29.924 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:29.924 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:29.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.830+0000 7fb09affd640 1 -- 172.21.15.139:0/2086386987 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v40) ==== 113+0+0 (secure 0 0 0) 0x7fb0a405d3c0 con 0x7fb0ac151a10 2024-09-16T00:08:29.925 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T00:08:29.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.834+0000 7f7364ff9640 1 -- 172.21.15.139:0/2996540618 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v40) ==== 87+0+0 (secure 0 0 0) 0x7f735c058800 con 0x7f7368154c20 2024-09-16T00:08:29.927 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:08:29.927 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:29.924+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:29.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.834+0000 7fb0b0e37640 1 -- 172.21.15.139:0/2086386987 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb08c06c2b0 msgr2=0x7fb08c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:29.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.834+0000 7fb0b0e37640 1 --2- 172.21.15.139:0/2086386987 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb08c06c2b0 0x7fb08c06e770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0095e0 tx=0x7fb09c009190 comp rx=0 tx=0).stop 2024-09-16T00:08:29.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.834+0000 7fb0b0e37640 1 -- 172.21.15.139:0/2086386987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ac151a10 msgr2=0x7fb0ac153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:29.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.834+0000 7fb0b0e37640 1 --2- 172.21.15.139:0/2086386987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ac151a10 0x7fb0ac153840 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400ca70 tx=0x7fb0a400cf30 comp rx=0 tx=0).stop 2024-09-16T00:08:29.928 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is set 2024-09-16T00:08:29.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.834+0000 7fb0b0e37640 1 -- 172.21.15.139:0/2086386987 shutdown_connections 2024-09-16T00:08:29.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.834+0000 7fb0b0e37640 1 --2- 172.21.15.139:0/2086386987 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb08c06c2b0 0x7fb08c06e770 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:29.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.834+0000 7fb0b0e37640 1 --2- 172.21.15.139:0/2086386987 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0ac151a10 0x7fb0ac153840 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:29.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.834+0000 7fb0b0e37640 1 --2- 172.21.15.139:0/2086386987 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb0ac0c5d60 0x7fb0ac153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:29.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.834+0000 7fb0b0e37640 1 --2- 172.21.15.139:0/2086386987 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0ac0c3c40 0x7fb0ac13fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:29.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.834+0000 7fb0b0e37640 1 -- 172.21.15.139:0/2086386987 >> 172.21.15.139:0/2086386987 conn(0x7fb0ac0cc740 msgr2=0x7fb0ac0ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:29.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.834+0000 7fb0b0e37640 1 -- 172.21.15.139:0/2086386987 shutdown_connections 2024-09-16T00:08:29.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.835+0000 7fb0b0e37640 1 -- 172.21.15.139:0/2086386987 wait complete. 2024-09-16T00:08:29.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.837+0000 7f736e45e640 1 -- 172.21.15.139:0/2996540618 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f735406c2e0 msgr2=0x7f735406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:29.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.837+0000 7f736e45e640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f735406c2e0 0x7f735406e7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7358004e00 tx=0x7f7358007f10 comp rx=0 tx=0).stop 2024-09-16T00:08:29.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.837+0000 7f736e45e640 1 -- 172.21.15.139:0/2996540618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7368154c20 msgr2=0x7f73681f40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:29.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.837+0000 7f736e45e640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7368154c20 0x7f73681f40a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f735c00c570 tx=0x7f735c00ca30 comp rx=0 tx=0).stop 2024-09-16T00:08:29.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.837+0000 7f736e45e640 1 -- 172.21.15.139:0/2996540618 shutdown_connections 2024-09-16T00:08:29.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.837+0000 7f736e45e640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f735406c2e0 0x7f735406e7a0 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7358004e00 tx=0x7f7358007f10 comp rx=0 tx=0).stop 2024-09-16T00:08:29.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.837+0000 7f736e45e640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f736815b280 0x7f73680c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:29.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.837+0000 7f736e45e640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7368154c20 0x7f73681f40a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:29.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.837+0000 7f736e45e640 1 --2- 172.21.15.139:0/2996540618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7368068060 0x7f73680c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:29.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.837+0000 7f736e45e640 1 -- 172.21.15.139:0/2996540618 >> 172.21.15.139:0/2996540618 conn(0x7f736806ca40 msgr2=0x7f73680ca8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:29.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.837+0000 7f736e45e640 1 -- 172.21.15.139:0/2996540618 shutdown_connections 2024-09-16T00:08:29.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:29.837+0000 7f736e45e640 1 -- 172.21.15.139:0/2996540618 wait complete. 2024-09-16T00:08:29.932 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T00:08:30.026 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:30.025+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:30.030 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:08:30.071 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:30.072 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:30.072 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:30.072 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:30.080 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:08:30.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.092+0000 7f4265a82640 1 Processor -- start 2024-09-16T00:08:30.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.092+0000 7f4265a82640 1 -- start start 2024-09-16T00:08:30.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.093+0000 7f4265a82640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4260158e60 0x7f42600c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.093+0000 7f4265a82640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f42601621d0 0x7f426006f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.093+0000 7f4265a82640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42600c9110 con 0x7f42600c8740 2024-09-16T00:08:30.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.093+0000 7f4265a82640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f426006f760 con 0x7f42601621d0 2024-09-16T00:08:30.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.093+0000 7f4265a82640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f426006e7d0 con 0x7f4260158e60 2024-09-16T00:08:30.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.094+0000 7f425f7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4260158e60 0x7f42600c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.094+0000 7f425f7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4260158e60 0x7f42600c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:59980/0 (socket says 172.21.15.139:59980) 2024-09-16T00:08:30.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.094+0000 7f425effd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f42601621d0 0x7f426006f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.094+0000 7f425f7fe640 1 -- 172.21.15.139:0/1703709596 learned_addr learned my addr 172.21.15.139:0/1703709596 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:30.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.094+0000 7f425ffff640 1 --1- 172.21.15.139:0/1703709596 >> v1:172.21.15.164:6789/0 conn(0x7f42600c8740 0x7f42600c9a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:59762/0 (socket says 172.21.15.139:59762) 2024-09-16T00:08:30.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.094+0000 7f425e7fc640 1 -- 172.21.15.139:0/1703709596 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4122235512 0 0) 0x7f42600c9110 con 0x7f42600c8740 2024-09-16T00:08:30.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.094+0000 7f425f7fe640 1 -- 172.21.15.139:0/1703709596 >> v2:172.21.15.164:3300/0 conn(0x7f42601621d0 msgr2=0x7f426006f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:30.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.094+0000 7f425f7fe640 1 --2- 172.21.15.139:0/1703709596 >> v2:172.21.15.164:3300/0 conn(0x7f42601621d0 0x7f426006f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.094+0000 7f425f7fe640 1 -- 172.21.15.139:0/1703709596 >> v1:172.21.15.164:6789/0 conn(0x7f42600c8740 legacy=0x7f42600c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.094+0000 7f425f7fe640 1 -- 172.21.15.139:0/1703709596 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42600c93d0 con 0x7f4260158e60 2024-09-16T00:08:30.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.095+0000 7f425f7fe640 1 --2- 172.21.15.139:0/1703709596 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4260158e60 0x7f42600c6f90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4250009870 tx=0x7f425002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3efccb5d9124a715 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:30.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.095+0000 7f425e7fc640 1 -- 172.21.15.139:0/1703709596 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250037070 con 0x7f4260158e60 2024-09-16T00:08:30.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.095+0000 7f425e7fc640 1 -- 172.21.15.139:0/1703709596 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4250032440 con 0x7f4260158e60 2024-09-16T00:08:30.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.095+0000 7f425e7fc640 1 -- 172.21.15.139:0/1703709596 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250032810 con 0x7f4260158e60 2024-09-16T00:08:30.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.096+0000 7f4265a82640 1 -- 172.21.15.139:0/1703709596 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4260158e60 msgr2=0x7f42600c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.096+0000 7f4265a82640 1 --2- 172.21.15.139:0/1703709596 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4260158e60 0x7f42600c6f90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4250009870 tx=0x7f425002a270 comp rx=0 tx=0).stop 2024-09-16T00:08:30.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.096+0000 7f4265a82640 1 -- 172.21.15.139:0/1703709596 shutdown_connections 2024-09-16T00:08:30.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.096+0000 7f4265a82640 1 --2- 172.21.15.139:0/1703709596 >> v2:172.21.15.164:3300/0 conn(0x7f42601621d0 0x7f426006f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.096+0000 7f4265a82640 1 --2- 172.21.15.139:0/1703709596 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4260158e60 0x7f42600c6f90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.096+0000 7f4265a82640 1 -- 172.21.15.139:0/1703709596 >> 172.21.15.139:0/1703709596 conn(0x7f4260070460 msgr2=0x7f4260070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:30.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.096+0000 7f4265a82640 1 -- 172.21.15.139:0/1703709596 shutdown_connections 2024-09-16T00:08:30.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.096+0000 7f4265a82640 1 -- 172.21.15.139:0/1703709596 wait complete. 2024-09-16T00:08:30.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.098+0000 7f4265a82640 1 Processor -- start 2024-09-16T00:08:30.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.098+0000 7f4265a82640 1 -- start start 2024-09-16T00:08:30.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.099+0000 7f4265a82640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42600c8740 0x7f4260150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.099+0000 7f4265a82640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4260158e60 0x7f42601537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.099+0000 7f425f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42600c8740 0x7f4260150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.099+0000 7f425f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42600c8740 0x7f4260150e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:34084/0 (socket says 172.21.15.139:34084) 2024-09-16T00:08:30.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.099+0000 7f425effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4260158e60 0x7f42601537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.099+0000 7f425effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4260158e60 0x7f42601537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43080/0 (socket says 172.21.15.139:43080) 2024-09-16T00:08:30.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.100+0000 7f4265a82640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42601621d0 0x7f426015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.100+0000 7f425effd640 1 -- 172.21.15.139:0/3556870019 learned_addr learned my addr 172.21.15.139:0/3556870019 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:30.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.100+0000 7f4265a82640 1 -- 172.21.15.139:0/3556870019 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f426006ed00 con 0x7f4260158e60 2024-09-16T00:08:30.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.100+0000 7f4265a82640 1 -- 172.21.15.139:0/3556870019 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f426006ea20 con 0x7f42600c8740 2024-09-16T00:08:30.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.100+0000 7f4265a82640 1 -- 172.21.15.139:0/3556870019 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4260159240 con 0x7f42601621d0 2024-09-16T00:08:30.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.100+0000 7f425ffff640 1 --2- 172.21.15.139:0/3556870019 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42601621d0 0x7f426015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.100+0000 7f425f7fe640 1 -- 172.21.15.139:0/3556870019 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42601621d0 msgr2=0x7f426015ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.100+0000 7f425f7fe640 1 --2- 172.21.15.139:0/3556870019 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42601621d0 0x7f426015ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.100+0000 7f425f7fe640 1 -- 172.21.15.139:0/3556870019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4260158e60 msgr2=0x7f42601537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.100+0000 7f425f7fe640 1 --2- 172.21.15.139:0/3556870019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4260158e60 0x7f42601537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.100+0000 7f425f7fe640 1 -- 172.21.15.139:0/3556870019 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42601548a0 con 0x7f42600c8740 2024-09-16T00:08:30.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.100+0000 7f425effd640 1 --2- 172.21.15.139:0/3556870019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4260158e60 0x7f42601537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:30.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.100+0000 7f425f7fe640 1 --2- 172.21.15.139:0/3556870019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42600c8740 0x7f4260150e90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4250031000 tx=0x7f42500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:30.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.101+0000 7f4247fff640 1 -- 172.21.15.139:0/3556870019 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250037040 con 0x7f42600c8740 2024-09-16T00:08:30.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.101+0000 7f4265a82640 1 -- 172.21.15.139:0/3556870019 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426015b360 con 0x7f42600c8740 2024-09-16T00:08:30.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.101+0000 7f4247fff640 1 -- 172.21.15.139:0/3556870019 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4250004020 con 0x7f42600c8740 2024-09-16T00:08:30.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.101+0000 7f4265a82640 1 -- 172.21.15.139:0/3556870019 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42601f5230 con 0x7f42600c8740 2024-09-16T00:08:30.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.101+0000 7f4247fff640 1 -- 172.21.15.139:0/3556870019 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42500043f0 con 0x7f42600c8740 2024-09-16T00:08:30.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.102+0000 7f4265a82640 1 -- 172.21.15.139:0/3556870019 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4240005290 con 0x7f42600c8740 2024-09-16T00:08:30.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.103+0000 7f4247fff640 1 -- 172.21.15.139:0/3556870019 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f425003a070 con 0x7f42600c8740 2024-09-16T00:08:30.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.104+0000 7f4247fff640 1 --2- 172.21.15.139:0/3556870019 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f424c06c2e0 0x7f424c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.104+0000 7f4247fff640 1 -- 172.21.15.139:0/3556870019 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5915+0+0 (secure 0 0 0) 0x7f42500aba00 con 0x7f42600c8740 2024-09-16T00:08:30.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.104+0000 7f425effd640 1 --2- 172.21.15.139:0/3556870019 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f424c06c2e0 0x7f424c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.105+0000 7f425effd640 1 --2- 172.21.15.139:0/3556870019 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f424c06c2e0 0x7f424c06e7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f426006c230 tx=0x7f4254009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:30.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.108+0000 7f4247fff640 1 -- 172.21.15.139:0/3556870019 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f425004a030 con 0x7f42600c8740 2024-09-16T00:08:30.127 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:30.126+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:30.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.167+0000 7f973c247640 1 Processor -- start 2024-09-16T00:08:30.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.167+0000 7f973c247640 1 -- start start 2024-09-16T00:08:30.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.168+0000 7f973c247640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9734071660 0x7f97340c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.168+0000 7f973c247640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f973406e9d0 0x7f973415cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.168+0000 7f973c247640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97340c8e40 con 0x7f9734071a40 2024-09-16T00:08:30.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.168+0000 7f973c247640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f973406d2e0 con 0x7f9734071660 2024-09-16T00:08:30.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.168+0000 7f973c247640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f973406ccd0 con 0x7f973406e9d0 2024-09-16T00:08:30.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.168+0000 7f9739fe3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9734071660 0x7f97340c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.168+0000 7f97397e2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f973406e9d0 0x7f973415cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.168+0000 7f97397e2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f973406e9d0 0x7f973415cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:59998/0 (socket says 172.21.15.139:59998) 2024-09-16T00:08:30.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.168+0000 7f97397e2640 1 -- 172.21.15.139:0/4126910651 learned_addr learned my addr 172.21.15.139:0/4126910651 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:30.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.169+0000 7f97397e2640 1 -- 172.21.15.139:0/4126910651 >> v2:172.21.15.139:3300/0 conn(0x7f9734071660 msgr2=0x7f97340c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:30.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.169+0000 7f97397e2640 1 --2- 172.21.15.139:0/4126910651 >> v2:172.21.15.139:3300/0 conn(0x7f9734071660 0x7f97340c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.169+0000 7f97397e2640 1 -- 172.21.15.139:0/4126910651 >> v1:172.21.15.164:6789/0 conn(0x7f9734071a40 legacy=0x7f973415b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.169+0000 7f97397e2640 1 -- 172.21.15.139:0/4126910651 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97340ce910 con 0x7f973406e9d0 2024-09-16T00:08:30.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.170+0000 7f97397e2640 1 --2- 172.21.15.139:0/4126910651 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f973406e9d0 0x7f973415cc90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9724009870 tx=0x7f972402a320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a5feee2f6f9f5cb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:30.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.170+0000 7f9738fe1640 1 -- 172.21.15.139:0/4126910651 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724037070 con 0x7f973406e9d0 2024-09-16T00:08:30.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.170+0000 7f9738fe1640 1 -- 172.21.15.139:0/4126910651 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f972402ae50 con 0x7f973406e9d0 2024-09-16T00:08:30.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.170+0000 7f9738fe1640 1 -- 172.21.15.139:0/4126910651 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97240328c0 con 0x7f973406e9d0 2024-09-16T00:08:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.171+0000 7f973c247640 1 -- 172.21.15.139:0/4126910651 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f973406e9d0 msgr2=0x7f973415cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.171+0000 7f973c247640 1 --2- 172.21.15.139:0/4126910651 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f973406e9d0 0x7f973415cc90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9724009870 tx=0x7f972402a320 comp rx=0 tx=0).stop 2024-09-16T00:08:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.171+0000 7f973c247640 1 -- 172.21.15.139:0/4126910651 shutdown_connections 2024-09-16T00:08:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.171+0000 7f973c247640 1 --2- 172.21.15.139:0/4126910651 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f973406e9d0 0x7f973415cc90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.171+0000 7f973c247640 1 --2- 172.21.15.139:0/4126910651 >> v2:172.21.15.139:3300/0 conn(0x7f9734071660 0x7f97340c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.171+0000 7f973c247640 1 -- 172.21.15.139:0/4126910651 >> 172.21.15.139:0/4126910651 conn(0x7f973406fb00 msgr2=0x7f97340738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.171+0000 7f973c247640 1 -- 172.21.15.139:0/4126910651 shutdown_connections 2024-09-16T00:08:30.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.171+0000 7f973c247640 1 -- 172.21.15.139:0/4126910651 wait complete. 2024-09-16T00:08:30.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.173+0000 7f973c247640 1 Processor -- start 2024-09-16T00:08:30.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.173+0000 7f973c247640 1 -- start start 2024-09-16T00:08:30.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.174+0000 7f973c247640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f973406e9d0 0x7f9734157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.174+0000 7f973c247640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9734071660 0x7f97340c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.174+0000 7f973c247640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9734071a40 0x7f97341541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.174+0000 7f973c247640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f97340ceed0 con 0x7f973406e9d0 2024-09-16T00:08:30.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.174+0000 7f973c247640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f973406cf70 con 0x7f9734071a40 2024-09-16T00:08:30.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.174+0000 7f973c247640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f973415b9a0 con 0x7f9734071660 2024-09-16T00:08:30.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.175+0000 7f9739fe3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f973406e9d0 0x7f9734157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.175+0000 7f97397e2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9734071660 0x7f97340c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.175+0000 7f97397e2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9734071660 0x7f97340c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:60012/0 (socket says 172.21.15.139:60012) 2024-09-16T00:08:30.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.175+0000 7f97397e2640 1 -- 172.21.15.139:0/3411531773 learned_addr learned my addr 172.21.15.139:0/3411531773 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:30.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.175+0000 7f973a7e4640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9734071a40 0x7f97341541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.175+0000 7f973a7e4640 1 -- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9734071660 msgr2=0x7f97340c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.175+0000 7f973a7e4640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9734071660 0x7f97340c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.175+0000 7f973a7e4640 1 -- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f973406e9d0 msgr2=0x7f9734157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.175+0000 7f973a7e4640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f973406e9d0 0x7f9734157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.175+0000 7f973a7e4640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9734153000 con 0x7f9734071a40 2024-09-16T00:08:30.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.176+0000 7f973a7e4640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9734071a40 0x7f97341541e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9734071e50 tx=0x7f972c014420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:30.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.176+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c014d10 con 0x7f9734071a40 2024-09-16T00:08:30.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.176+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f972c0120d0 con 0x7f9734071a40 2024-09-16T00:08:30.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.176+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c012480 con 0x7f9734071a40 2024-09-16T00:08:30.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.176+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9734156330 con 0x7f9734071a40 2024-09-16T00:08:30.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.176+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97341568f0 con 0x7f9734071a40 2024-09-16T00:08:30.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.177+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f9734069600 con 0x7f9734071a40 2024-09-16T00:08:30.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.178+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f972c010850 con 0x7f9734071a40 2024-09-16T00:08:30.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.179+0000 7f972a7fc640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f971806c2b0 0x7f971806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.179+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5915+0+0 (secure 0 0 0) 0x7f972c093300 con 0x7f9734071a40 2024-09-16T00:08:30.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.179+0000 7f9739fe3640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f971806c2b0 0x7f971806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.179+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v40) ==== 54+0+16 (secure 0 0 0) 0x7f972c0958e0 con 0x7f9734071a40 2024-09-16T00:08:30.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.179+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f97340c1a20 0x7f973415b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.179+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f97340ca800 con 0x7f97340c1a20 2024-09-16T00:08:30.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.179+0000 7f97397e2640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f97340c1a20 0x7f973415b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.179+0000 7f9739fe3640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f971806c2b0 0x7f971806e770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9730009b40 tx=0x7f9730009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:30.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.180+0000 7f97397e2640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f97340c1a20 0x7f973415b040 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:30.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.181+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f97340ca800 con 0x7f97340c1a20 2024-09-16T00:08:30.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.208+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9734157650 con 0x7f97340c1a20 2024-09-16T00:08:30.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.211+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9734157650 con 0x7f97340c1a20 2024-09-16T00:08:30.212 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:30.213 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:30.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.211+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f9734150ad0 0x7f9734150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.211+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f9734153d80 con 0x7f9734150ad0 2024-09-16T00:08:30.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.211+0000 7f9739fe3640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f9734150ad0 0x7f9734150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.212+0000 7f9739fe3640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f9734150ad0 0x7f9734150ef0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:30.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.213+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9734153d80 con 0x7f9734150ad0 2024-09-16T00:08:30.225 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:30.225 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:30.225 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:30.225 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:30.226 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:30.228 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:30.227+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:30.234 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:08:30.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.240+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f97340ae470 con 0x7f9734150ad0 2024-09-16T00:08:30.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.243+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f97340ae470 con 0x7f9734150ad0 2024-09-16T00:08:30.245 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:30.246 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:30.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.243+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f97341f3ce0 0x7f9734062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.243+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f97340ae680 con 0x7f97341f3ce0 2024-09-16T00:08:30.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.244+0000 7f97397e2640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f97341f3ce0 0x7f9734062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.245+0000 7f97397e2640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f97341f3ce0 0x7f9734062630 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:30.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.246+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f97340ae680 con 0x7f97341f3ce0 2024-09-16T00:08:30.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.273+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9734062b70 con 0x7f97341f3ce0 2024-09-16T00:08:30.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.275+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9734062b70 con 0x7f97341f3ce0 2024-09-16T00:08:30.276 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:30.277 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:30.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.275+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f97340c9010 0x7f97340c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.275+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f97340c99b0 con 0x7f97340c9010 2024-09-16T00:08:30.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.276+0000 7f973a7e4640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f97340c9010 0x7f97340c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.276+0000 7f973a7e4640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f97340c9010 0x7f97340c9470 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:30.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.277+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f97340c99b0 con 0x7f97340c9010 2024-09-16T00:08:30.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.304+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f97340c9ec0 con 0x7f97340c9010 2024-09-16T00:08:30.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.306+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f97340c9ec0 con 0x7f97340c9010 2024-09-16T00:08:30.307 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:30.308 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:30.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.306+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f973402cb50 0x7f973402cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.307+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f97340c99b0 con 0x7f973402cb50 2024-09-16T00:08:30.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.307+0000 7f9739fe3640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f973402cb50 0x7f973402cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.307+0000 7f9739fe3640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f973402cb50 0x7f973402cf90 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:30.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.308+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f97340c99b0 con 0x7f973402cb50 2024-09-16T00:08:30.329 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:30.328+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:30.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.335+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f97340c99b0 con 0x7f973402cb50 2024-09-16T00:08:30.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.338+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f97340c99b0 con 0x7f973402cb50 2024-09-16T00:08:30.338 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:30.339 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:30.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.338+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f9734200770 0x7f9734202b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.338+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f973402d9e0 con 0x7f9734200770 2024-09-16T00:08:30.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.338+0000 7f97397e2640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f9734200770 0x7f9734202b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.339+0000 7f97397e2640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f9734200770 0x7f9734202b60 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:30.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.339+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f973402d9e0 con 0x7f9734200770 2024-09-16T00:08:30.343 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 67.260246320999997, 2024-09-16T00:08:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:08:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.343 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:08:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:08:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:08:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:08:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:08:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:08:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:08:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:08:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:08:30.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:08:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:08:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:08:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:08:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:08:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:08:30.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:08:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:08:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:08:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:08:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:08:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:08:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:08:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:08:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:08:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:08:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:08:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:08:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:08:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.180714691000006, 2024-09-16T00:08:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093071190000000008, 2024-09-16T00:08:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.350 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.351 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.351 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.351 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 106 2024-09-16T00:08:30.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.351 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160362+0000", 2024-09-16T00:08:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160438+0000", 2024-09-16T00:08:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6711000000000005e-05 2024-09-16T00:08:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160444+0000", 2024-09-16T00:08:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0530000000000003e-06 2024-09-16T00:08:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160449+0000", 2024-09-16T00:08:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2189999999999998e-06 2024-09-16T00:08:30.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161270+0000", 2024-09-16T00:08:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082099199999999999 2024-09-16T00:08:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161348+0000", 2024-09-16T00:08:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8361000000000004e-05 2024-09-16T00:08:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161472+0000", 2024-09-16T00:08:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123523 2024-09-16T00:08:30.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163461+0000", 2024-09-16T00:08:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019891560000000002 2024-09-16T00:08:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169418+0000", 2024-09-16T00:08:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059568490000000002 2024-09-16T00:08:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169570+0000", 2024-09-16T00:08:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152461 2024-09-16T00:08:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169668+0000", 2024-09-16T00:08:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T00:08:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:30.356 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.180618920000001, 2024-09-16T00:08:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092931679999999992, 2024-09-16T00:08:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107 2024-09-16T00:08:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0699999999999998e-07 2024-09-16T00:08:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160459+0000", 2024-09-16T00:08:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0370000000000001e-06 2024-09-16T00:08:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160463+0000", 2024-09-16T00:08:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999999e-06 2024-09-16T00:08:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160466+0000", 2024-09-16T00:08:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.644e-06 2024-09-16T00:08:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161873+0000", 2024-09-16T00:08:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014074840000000001 2024-09-16T00:08:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161949+0000", 2024-09-16T00:08:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5749999999999998e-05 2024-09-16T00:08:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162069+0000", 2024-09-16T00:08:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120257 2024-09-16T00:08:30.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163506+0000", 2024-09-16T00:08:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001436344 2024-09-16T00:08:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169704+0000", 2024-09-16T00:08:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061982829999999997 2024-09-16T00:08:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169730+0000", 2024-09-16T00:08:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6060000000000001e-05 2024-09-16T00:08:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169750+0000", 2024-09-16T00:08:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0092e-05 2024-09-16T00:08:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:30.362 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.180509607999994, 2024-09-16T00:08:30.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092280039999999997, 2024-09-16T00:08:30.363 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.363 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.363 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.363 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.363 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.363 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.363 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 108 2024-09-16T00:08:30.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160567+0000", 2024-09-16T00:08:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7300000000000001e-07 2024-09-16T00:08:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160569+0000", 2024-09-16T00:08:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5299999999999999e-06 2024-09-16T00:08:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160574+0000", 2024-09-16T00:08:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-16T00:08:30.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160577+0000", 2024-09-16T00:08:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.01e-06 2024-09-16T00:08:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162518+0000", 2024-09-16T00:08:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001940703 2024-09-16T00:08:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162571+0000", 2024-09-16T00:08:30.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2670000000000002e-05 2024-09-16T00:08:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162647+0000", 2024-09-16T00:08:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6778000000000002e-05 2024-09-16T00:08:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163544+0000", 2024-09-16T00:08:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089702599999999999 2024-09-16T00:08:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169754+0000", 2024-09-16T00:08:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062091739999999996 2024-09-16T00:08:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169776+0000", 2024-09-16T00:08:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2391999999999998e-05 2024-09-16T00:08:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169794+0000", 2024-09-16T00:08:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8318999999999999e-05 2024-09-16T00:08:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.368 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.180461058000006, 2024-09-16T00:08:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092220679999999999, 2024-09-16T00:08:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.369 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 109 2024-09-16T00:08:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9900000000000001e-07 2024-09-16T00:08:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160707+0000", 2024-09-16T00:08:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1621999999999998e-05 2024-09-16T00:08:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160713+0000", 2024-09-16T00:08:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.596e-06 2024-09-16T00:08:30.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160717+0000", 2024-09-16T00:08:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5829999999999998e-06 2024-09-16T00:08:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162869+0000", 2024-09-16T00:08:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021516310000000002 2024-09-16T00:08:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162946+0000", 2024-09-16T00:08:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7696999999999998e-05 2024-09-16T00:08:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163065+0000", 2024-09-16T00:08:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118581 2024-09-16T00:08:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164431+0000", 2024-09-16T00:08:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001366003 2024-09-16T00:08:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169797+0000", 2024-09-16T00:08:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005366244 2024-09-16T00:08:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169819+0000", 2024-09-16T00:08:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2036999999999999e-05 2024-09-16T00:08:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169837+0000", 2024-09-16T00:08:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7575e-05 2024-09-16T00:08:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.180351709000007, 2024-09-16T00:08:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092871919999999997, 2024-09-16T00:08:30.375 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 110 2024-09-16T00:08:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:08:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160728+0000", 2024-09-16T00:08:30.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0620000000000001e-06 2024-09-16T00:08:30.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160731+0000", 2024-09-16T00:08:30.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6500000000000002e-06 2024-09-16T00:08:30.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160734+0000", 2024-09-16T00:08:30.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.762e-06 2024-09-16T00:08:30.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163580+0000", 2024-09-16T00:08:30.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028464549999999999 2024-09-16T00:08:30.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163657+0000", 2024-09-16T00:08:30.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6695999999999999e-05 2024-09-16T00:08:30.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163798+0000", 2024-09-16T00:08:30.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014055 2024-09-16T00:08:30.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165483+0000", 2024-09-16T00:08:30.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016853090000000001 2024-09-16T00:08:30.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169970+0000", 2024-09-16T00:08:30.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00448709 2024-09-16T00:08:30.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169994+0000", 2024-09-16T00:08:30.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3575999999999999e-05 2024-09-16T00:08:30.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170011+0000", 2024-09-16T00:08:30.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7702000000000001e-05 2024-09-16T00:08:30.381 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.381 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.381 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.381 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:30.382 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:30.382 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.180338039999995, 2024-09-16T00:08:30.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093174859999999998, 2024-09-16T00:08:30.382 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.382 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.382 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.382 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.382 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.382 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.382 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 111 2024-09-16T00:08:30.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.382 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:30.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:30.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:30.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1e-07 2024-09-16T00:08:30.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160741+0000", 2024-09-16T00:08:30.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6110000000000001e-06 2024-09-16T00:08:30.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160744+0000", 2024-09-16T00:08:30.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3299999999999999e-06 2024-09-16T00:08:30.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160746+0000", 2024-09-16T00:08:30.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5469999999999999e-06 2024-09-16T00:08:30.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164069+0000", 2024-09-16T00:08:30.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033222709999999999 2024-09-16T00:08:30.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164113+0000", 2024-09-16T00:08:30.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4212999999999998e-05 2024-09-16T00:08:30.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164187+0000", 2024-09-16T00:08:30.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4136999999999998e-05 2024-09-16T00:08:30.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166429+0000", 2024-09-16T00:08:30.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022423180000000001 2024-09-16T00:08:30.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170016+0000", 2024-09-16T00:08:30.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035869449999999998 2024-09-16T00:08:30.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170038+0000", 2024-09-16T00:08:30.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2053e-05 2024-09-16T00:08:30.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170055+0000", 2024-09-16T00:08:30.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6850999999999998e-05 2024-09-16T00:08:30.387 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.387 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.387 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.388 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:30.388 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:30.388 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.180324325000001, 2024-09-16T00:08:30.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093480279999999995, 2024-09-16T00:08:30.388 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.388 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.388 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.388 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 112 2024-09-16T00:08:30.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.388 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:30.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:30.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160752+0000", 2024-09-16T00:08:30.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2700000000000001e-07 2024-09-16T00:08:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160808+0000", 2024-09-16T00:08:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5797999999999997e-05 2024-09-16T00:08:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160814+0000", 2024-09-16T00:08:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5980000000000002e-06 2024-09-16T00:08:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160819+0000", 2024-09-16T00:08:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8960000000000002e-06 2024-09-16T00:08:30.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164468+0000", 2024-09-16T00:08:30.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003649476 2024-09-16T00:08:30.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164525+0000", 2024-09-16T00:08:30.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6227999999999998e-05 2024-09-16T00:08:30.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164595+0000", 2024-09-16T00:08:30.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759000000000005e-05 2024-09-16T00:08:30.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167237+0000", 2024-09-16T00:08:30.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026410470000000001 2024-09-16T00:08:30.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170060+0000", 2024-09-16T00:08:30.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028234969999999999 2024-09-16T00:08:30.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170081+0000", 2024-09-16T00:08:30.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1242000000000001e-05 2024-09-16T00:08:30.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170100+0000", 2024-09-16T00:08:30.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8260000000000001e-05 2024-09-16T00:08:30.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.394 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.394 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:30.394 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:30.395 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.180247230000006, 2024-09-16T00:08:30.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093139409999999992, 2024-09-16T00:08:30.395 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.395 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.395 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.395 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 113 2024-09-16T00:08:30.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.395 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:30.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:30.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:30.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-07 2024-09-16T00:08:30.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160833+0000", 2024-09-16T00:08:30.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.557e-06 2024-09-16T00:08:30.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160839+0000", 2024-09-16T00:08:30.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3829999999999997e-06 2024-09-16T00:08:30.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160844+0000", 2024-09-16T00:08:30.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6909999999999996e-06 2024-09-16T00:08:30.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165008+0000", 2024-09-16T00:08:30.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041642129999999999 2024-09-16T00:08:30.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165085+0000", 2024-09-16T00:08:30.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6651999999999997e-05 2024-09-16T00:08:30.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165205+0000", 2024-09-16T00:08:30.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012045900000000001 2024-09-16T00:08:30.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167283+0000", 2024-09-16T00:08:30.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020782909999999999 2024-09-16T00:08:30.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170104+0000", 2024-09-16T00:08:30.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028207950000000001 2024-09-16T00:08:30.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170125+0000", 2024-09-16T00:08:30.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0852e-05 2024-09-16T00:08:30.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170143+0000", 2024-09-16T00:08:30.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7467000000000001e-05 2024-09-16T00:08:30.400 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.400 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.400 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.401 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:30.401 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:30.401 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.180200748000004, 2024-09-16T00:08:30.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093126960000000005, 2024-09-16T00:08:30.401 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.401 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.401 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.401 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-16T00:08:30.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.402 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:30.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:30.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160876+0000", 2024-09-16T00:08:30.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5700000000000002e-07 2024-09-16T00:08:30.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160880+0000", 2024-09-16T00:08:30.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5539999999999999e-06 2024-09-16T00:08:30.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160888+0000", 2024-09-16T00:08:30.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.157e-06 2024-09-16T00:08:30.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160894+0000", 2024-09-16T00:08:30.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7690000000000002e-06 2024-09-16T00:08:30.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165519+0000", 2024-09-16T00:08:30.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046249539999999997 2024-09-16T00:08:30.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165593+0000", 2024-09-16T00:08:30.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4039e-05 2024-09-16T00:08:30.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165715+0000", 2024-09-16T00:08:30.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012131899999999999 2024-09-16T00:08:30.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167309+0000", 2024-09-16T00:08:30.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001594011 2024-09-16T00:08:30.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170147+0000", 2024-09-16T00:08:30.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028387130000000001 2024-09-16T00:08:30.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170170+0000", 2024-09-16T00:08:30.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2517000000000001e-05 2024-09-16T00:08:30.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170188+0000", 2024-09-16T00:08:30.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8006e-05 2024-09-16T00:08:30.406 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.406 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.406 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.407 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:30.407 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:30.407 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.180159720000006, 2024-09-16T00:08:30.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093140040000000007, 2024-09-16T00:08:30.407 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.407 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.407 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.407 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.407 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.407 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.407 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-16T00:08:30.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.408 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:30.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:30.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160917+0000", 2024-09-16T00:08:30.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7599999999999996e-07 2024-09-16T00:08:30.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160992+0000", 2024-09-16T00:08:30.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4760999999999998e-05 2024-09-16T00:08:30.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161000+0000", 2024-09-16T00:08:30.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9829999999999996e-06 2024-09-16T00:08:30.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161007+0000", 2024-09-16T00:08:30.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4909999999999999e-06 2024-09-16T00:08:30.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165983+0000", 2024-09-16T00:08:30.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049755399999999997 2024-09-16T00:08:30.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166027+0000", 2024-09-16T00:08:30.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4109000000000001e-05 2024-09-16T00:08:30.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166100+0000", 2024-09-16T00:08:30.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3360000000000002e-05 2024-09-16T00:08:30.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169698+0000", 2024-09-16T00:08:30.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597795 2024-09-16T00:08:30.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170192+0000", 2024-09-16T00:08:30.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049447300000000005 2024-09-16T00:08:30.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170214+0000", 2024-09-16T00:08:30.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1546e-05 2024-09-16T00:08:30.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170230+0000", 2024-09-16T00:08:30.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6169999999999999e-05 2024-09-16T00:08:30.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.413 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.413 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:30.413 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:30.413 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.180049991000004, 2024-09-16T00:08:30.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092457029999999992, 2024-09-16T00:08:30.413 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.414 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.414 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.414 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.414 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.414 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.414 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-16T00:08:30.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.414 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:30.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:30.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:30.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7899999999999999e-07 2024-09-16T00:08:30.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161029+0000", 2024-09-16T00:08:30.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6630000000000002e-06 2024-09-16T00:08:30.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161036+0000", 2024-09-16T00:08:30.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T00:08:30.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161041+0000", 2024-09-16T00:08:30.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:08:30.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166468+0000", 2024-09-16T00:08:30.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054266050000000001 2024-09-16T00:08:30.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166511+0000", 2024-09-16T00:08:30.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3869999999999998e-05 2024-09-16T00:08:30.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166606+0000", 2024-09-16T00:08:30.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4437000000000004e-05 2024-09-16T00:08:30.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169955+0000", 2024-09-16T00:08:30.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033494810000000001 2024-09-16T00:08:30.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170234+0000", 2024-09-16T00:08:30.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027900899999999998 2024-09-16T00:08:30.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170255+0000", 2024-09-16T00:08:30.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0537999999999999e-05 2024-09-16T00:08:30.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170272+0000", 2024-09-16T00:08:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6611000000000001e-05 2024-09-16T00:08:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.180024845999995, 2024-09-16T00:08:30.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093511010000000006, 2024-09-16T00:08:30.420 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.420 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.420 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.420 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-16T00:08:30.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.420 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:30.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:30.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:30.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.63e-07 2024-09-16T00:08:30.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161055+0000", 2024-09-16T00:08:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6770000000000001e-06 2024-09-16T00:08:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161063+0000", 2024-09-16T00:08:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6469999999999998e-06 2024-09-16T00:08:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161068+0000", 2024-09-16T00:08:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.746e-06 2024-09-16T00:08:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166817+0000", 2024-09-16T00:08:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057494959999999998 2024-09-16T00:08:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166893+0000", 2024-09-16T00:08:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6210000000000004e-05 2024-09-16T00:08:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167019+0000", 2024-09-16T00:08:30.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125585 2024-09-16T00:08:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170275+0000", 2024-09-16T00:08:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032563620000000001 2024-09-16T00:08:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170309+0000", 2024-09-16T00:08:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3865999999999997e-05 2024-09-16T00:08:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170362+0000", 2024-09-16T00:08:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3094999999999999e-05 2024-09-16T00:08:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170402+0000", 2024-09-16T00:08:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9953999999999997e-05 2024-09-16T00:08:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:30.425 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.731292761000006, 2024-09-16T00:08:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097364389999999995, 2024-09-16T00:08:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138 2024-09-16T00:08:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:30.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:30.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609784+0000", 2024-09-16T00:08:30.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T00:08:30.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609787+0000", 2024-09-16T00:08:30.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280000000000001e-06 2024-09-16T00:08:30.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609794+0000", 2024-09-16T00:08:30.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6409999999999996e-06 2024-09-16T00:08:30.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609799+0000", 2024-09-16T00:08:30.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9359999999999998e-06 2024-09-16T00:08:30.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615759+0000", 2024-09-16T00:08:30.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059599910000000004 2024-09-16T00:08:30.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615806+0000", 2024-09-16T00:08:30.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7290000000000003e-05 2024-09-16T00:08:30.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615877+0000", 2024-09-16T00:08:30.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1000000000000005e-05 2024-09-16T00:08:30.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618961+0000", 2024-09-16T00:08:30.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030841390000000001 2024-09-16T00:08:30.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619436+0000", 2024-09-16T00:08:30.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047452799999999999 2024-09-16T00:08:30.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619482+0000", 2024-09-16T00:08:30.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5689000000000003e-05 2024-09-16T00:08:30.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619519+0000", 2024-09-16T00:08:30.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7938000000000002e-05 2024-09-16T00:08:30.432 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.432 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.432 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.432 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:30.432 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:30.432 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.731269046999998, 2024-09-16T00:08:30.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011812954000000001, 2024-09-16T00:08:30.432 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.432 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.432 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.433 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.433 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.433 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.433 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139 2024-09-16T00:08:30.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.433 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:30.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:30.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:30.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999998e-07 2024-09-16T00:08:30.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609810+0000", 2024-09-16T00:08:30.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2100000000000002e-06 2024-09-16T00:08:30.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609817+0000", 2024-09-16T00:08:30.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.28e-06 2024-09-16T00:08:30.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609825+0000", 2024-09-16T00:08:30.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:30.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616116+0000", 2024-09-16T00:08:30.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006290543 2024-09-16T00:08:30.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616187+0000", 2024-09-16T00:08:30.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1830999999999997e-05 2024-09-16T00:08:30.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616298+0000", 2024-09-16T00:08:30.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011044699999999999 2024-09-16T00:08:30.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619559+0000", 2024-09-16T00:08:30.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032614710000000002 2024-09-16T00:08:30.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621562+0000", 2024-09-16T00:08:30.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020028680000000001 2024-09-16T00:08:30.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621597+0000", 2024-09-16T00:08:30.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4625000000000001e-05 2024-09-16T00:08:30.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621620+0000", 2024-09-16T00:08:30.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2864e-05 2024-09-16T00:08:30.438 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.438 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.438 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.438 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:30.438 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:30.438 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.731241251, 2024-09-16T00:08:30.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011828211999999999, 2024-09-16T00:08:30.439 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.439 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.439 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.439 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.439 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 140 2024-09-16T00:08:30.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.439 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:30.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:30.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:30.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T00:08:30.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:30.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961000000000003e-05 2024-09-16T00:08:30.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609892+0000", 2024-09-16T00:08:30.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0516000000000001e-05 2024-09-16T00:08:30.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609898+0000", 2024-09-16T00:08:30.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0449999999999997e-06 2024-09-16T00:08:30.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616797+0000", 2024-09-16T00:08:30.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068998840000000002 2024-09-16T00:08:30.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616844+0000", 2024-09-16T00:08:30.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6501000000000001e-05 2024-09-16T00:08:30.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616935+0000", 2024-09-16T00:08:30.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1222000000000002e-05 2024-09-16T00:08:30.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619808+0000", 2024-09-16T00:08:30.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873198 2024-09-16T00:08:30.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621624+0000", 2024-09-16T00:08:30.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001815933 2024-09-16T00:08:30.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621646+0000", 2024-09-16T00:08:30.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1622999999999999e-05 2024-09-16T00:08:30.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621663+0000", 2024-09-16T00:08:30.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6854000000000002e-05 2024-09-16T00:08:30.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.444 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.444 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:30.444 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:30.444 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.731155442000002, 2024-09-16T00:08:30.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014331501999999999, 2024-09-16T00:08:30.445 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.445 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.445 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.445 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-16T00:08:30.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.445 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:30.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:30.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609921+0000", 2024-09-16T00:08:30.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9599999999999999e-07 2024-09-16T00:08:30.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609925+0000", 2024-09-16T00:08:30.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T00:08:30.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609932+0000", 2024-09-16T00:08:30.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8240000000000002e-06 2024-09-16T00:08:30.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609937+0000", 2024-09-16T00:08:30.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1070000000000004e-06 2024-09-16T00:08:30.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618987+0000", 2024-09-16T00:08:30.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090502719999999998 2024-09-16T00:08:30.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619064+0000", 2024-09-16T00:08:30.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6868999999999999e-05 2024-09-16T00:08:30.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619183+0000", 2024-09-16T00:08:30.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011961 2024-09-16T00:08:30.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.620052+0000", 2024-09-16T00:08:30.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086822799999999997 2024-09-16T00:08:30.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624197+0000", 2024-09-16T00:08:30.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041454930000000001 2024-09-16T00:08:30.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624229+0000", 2024-09-16T00:08:30.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1758999999999999e-05 2024-09-16T00:08:30.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624252+0000", 2024-09-16T00:08:30.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:08:30.450 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.450 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.450 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.450 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:30.450 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:30.451 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.258389633999997, 2024-09-16T00:08:30.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010093276999999999, 2024-09-16T00:08:30.451 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.451 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.451 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.451 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.451 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.451 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.451 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 157 2024-09-16T00:08:30.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.451 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:30.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:30.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:30.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4699999999999998e-07 2024-09-16T00:08:30.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082688+0000", 2024-09-16T00:08:30.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0209999999999998e-06 2024-09-16T00:08:30.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082692+0000", 2024-09-16T00:08:30.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6890000000000002e-06 2024-09-16T00:08:30.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082695+0000", 2024-09-16T00:08:30.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5339999999999998e-06 2024-09-16T00:08:30.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085303+0000", 2024-09-16T00:08:30.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026084789999999999 2024-09-16T00:08:30.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085374+0000", 2024-09-16T00:08:30.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1317999999999997e-05 2024-09-16T00:08:30.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085506+0000", 2024-09-16T00:08:30.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013118 2024-09-16T00:08:30.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086928+0000", 2024-09-16T00:08:30.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014224559999999999 2024-09-16T00:08:30.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092665+0000", 2024-09-16T00:08:30.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057369830000000002 2024-09-16T00:08:30.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092727+0000", 2024-09-16T00:08:30.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1810000000000006e-05 2024-09-16T00:08:30.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092779+0000", 2024-09-16T00:08:30.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2559999999999998e-05 2024-09-16T00:08:30.456 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.456 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.456 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.457 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:30.457 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:30.457 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.258377465999999, 2024-09-16T00:08:30.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010176388, 2024-09-16T00:08:30.457 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.457 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.457 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.457 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 158 2024-09-16T00:08:30.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.458 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:30.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:30.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082699+0000", 2024-09-16T00:08:30.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.11e-07 2024-09-16T00:08:30.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082742+0000", 2024-09-16T00:08:30.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3699000000000003e-05 2024-09-16T00:08:30.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082748+0000", 2024-09-16T00:08:30.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8690000000000001e-06 2024-09-16T00:08:30.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082751+0000", 2024-09-16T00:08:30.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.942e-06 2024-09-16T00:08:30.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085881+0000", 2024-09-16T00:08:30.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031303450000000001 2024-09-16T00:08:30.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085959+0000", 2024-09-16T00:08:30.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7874999999999996e-05 2024-09-16T00:08:30.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086086+0000", 2024-09-16T00:08:30.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012685499999999999 2024-09-16T00:08:30.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087127+0000", 2024-09-16T00:08:30.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001041298 2024-09-16T00:08:30.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092790+0000", 2024-09-16T00:08:30.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056628 2024-09-16T00:08:30.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092832+0000", 2024-09-16T00:08:30.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2227000000000001e-05 2024-09-16T00:08:30.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092875+0000", 2024-09-16T00:08:30.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2267e-05 2024-09-16T00:08:30.462 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.462 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.463 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.463 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:30.463 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:30.463 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.258319427999993, 2024-09-16T00:08:30.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010199547, 2024-09-16T00:08:30.463 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:30.463 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:30.463 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:30.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:30.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:30.463 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:30.464 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159 2024-09-16T00:08:30.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.464 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:30.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:30.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:30.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:30.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:30.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:30.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:30.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082757+0000", 2024-09-16T00:08:30.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2500000000000001e-07 2024-09-16T00:08:30.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:30.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082759+0000", 2024-09-16T00:08:30.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-16T00:08:30.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:30.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082763+0000", 2024-09-16T00:08:30.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4889999999999997e-06 2024-09-16T00:08:30.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:30.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082766+0000", 2024-09-16T00:08:30.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:08:30.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:30.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086571+0000", 2024-09-16T00:08:30.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038053269999999998 2024-09-16T00:08:30.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:30.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086619+0000", 2024-09-16T00:08:30.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7924999999999999e-05 2024-09-16T00:08:30.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:30.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086693+0000", 2024-09-16T00:08:30.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000005e-05 2024-09-16T00:08:30.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:30.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087383+0000", 2024-09-16T00:08:30.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069008299999999997 2024-09-16T00:08:30.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:30.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092882+0000", 2024-09-16T00:08:30.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054987450000000002 2024-09-16T00:08:30.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:30.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092921+0000", 2024-09-16T00:08:30.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9388999999999998e-05 2024-09-16T00:08:30.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:30.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:30.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:30.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092956+0000", 2024-09-16T00:08:30.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4647999999999998e-05 2024-09-16T00:08:30.468 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.469 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.469 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.469 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:30.469 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:30.469 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:30.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.365+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f973402d9e0 con 0x7f9734200770 2024-09-16T00:08:30.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.367+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f973402d9e0 con 0x7f9734200770 2024-09-16T00:08:30.470 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:30.471 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:30.469+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:30.472 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:08:30.473 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:30.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.367+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f97342035b0 0x7f97342059a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.367+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f9734208770 con 0x7f97342035b0 2024-09-16T00:08:30.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.367+0000 7f973a7e4640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f97342035b0 0x7f97342059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.368+0000 7f973a7e4640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f97342035b0 0x7f97342059a0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:30.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.368+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9734208770 con 0x7f97342035b0 2024-09-16T00:08:30.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.393+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9734205ee0 con 0x7f97342035b0 2024-09-16T00:08:30.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.395+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9734205ee0 con 0x7f97342035b0 2024-09-16T00:08:30.473 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:30.474 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:30.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.395+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9734206600 0x7f97342089f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:30.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.395+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f973420b7c0 con 0x7f9734206600 2024-09-16T00:08:30.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.396+0000 7f9739fe3640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9734206600 0x7f97342089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:30.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.396+0000 7f9739fe3640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9734206600 0x7f97342089f0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:30.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.397+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f973420b7c0 con 0x7f9734206600 2024-09-16T00:08:30.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.423+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9734208f30 con 0x7f9734206600 2024-09-16T00:08:30.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.424+0000 7f4265a82640 1 -- 172.21.15.139:0/3556870019 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) -- 0x7f4240005580 con 0x7f42600c8740 2024-09-16T00:08:30.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.425+0000 7f972a7fc640 1 -- 172.21.15.139:0/3411531773 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9734208f30 con 0x7f9734206600 2024-09-16T00:08:30.476 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:30.478 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:30.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.425+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f97340c1a20 msgr2=0x7f973415b040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.425+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f97340c1a20 0x7f973415b040 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.425+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f9734150ad0 msgr2=0x7f9734150ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.425+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f9734150ad0 0x7f9734150ef0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f97341f3ce0 msgr2=0x7f9734062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f97341f3ce0 0x7f9734062630 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973a7e4640 1 -- 172.21.15.139:0/3411531773 reap_dead start 2024-09-16T00:08:30.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f97340c9010 msgr2=0x7f97340c9470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f97340c9010 0x7f97340c9470 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f973402cb50 msgr2=0x7f973402cf90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f973402cb50 0x7f973402cf90 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f9734200770 msgr2=0x7f9734202b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f9734200770 0x7f9734202b60 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f97342035b0 msgr2=0x7f97342059a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f97342035b0 0x7f97342059a0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9734206600 msgr2=0x7f97342089f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9734206600 0x7f97342089f0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973a7e4640 1 -- 172.21.15.139:0/3411531773 reap_dead start 2024-09-16T00:08:30.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f971806c2b0 msgr2=0x7f971806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f971806c2b0 0x7f971806e770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9730009b40 tx=0x7f9730009190 comp rx=0 tx=0).stop 2024-09-16T00:08:30.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9734071a40 msgr2=0x7f97341541e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9734071a40 0x7f97341541e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9734071e50 tx=0x7f972c014420 comp rx=0 tx=0).stop 2024-09-16T00:08:30.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 shutdown_connections 2024-09-16T00:08:30.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f971806c2b0 0x7f971806e770 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9730009b40 tx=0x7f9730009190 comp rx=0 tx=0).stop 2024-09-16T00:08:30.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 --2- 172.21.15.139:0/3411531773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9734071a40 0x7f97341541e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 >> 172.21.15.139:0/3411531773 conn(0x7f973406fb00 msgr2=0x7f9734068230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:30.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 shutdown_connections 2024-09-16T00:08:30.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.426+0000 7f973c247640 1 -- 172.21.15.139:0/3411531773 wait complete. 2024-09-16T00:08:30.571 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:30.570+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:30.632 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:30.632 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:30.641 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:08:30.672 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:30.671+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:30.772 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:30.772+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:30.791 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:30.791 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:30.800 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:08:30.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.839+0000 7f4247fff640 1 -- 172.21.15.139:0/3556870019 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v41) ==== 113+0+0 (secure 0 0 0) 0x7f4250041030 con 0x7f42600c8740 2024-09-16T00:08:30.840 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T00:08:30.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.843+0000 7f4265a82640 1 -- 172.21.15.139:0/3556870019 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f424c06c2e0 msgr2=0x7f424c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.843+0000 7f4265a82640 1 --2- 172.21.15.139:0/3556870019 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f424c06c2e0 0x7f424c06e7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f426006c230 tx=0x7f4254009190 comp rx=0 tx=0).stop 2024-09-16T00:08:30.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.843+0000 7f4265a82640 1 -- 172.21.15.139:0/3556870019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42600c8740 msgr2=0x7f4260150e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:30.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.843+0000 7f4265a82640 1 --2- 172.21.15.139:0/3556870019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42600c8740 0x7f4260150e90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4250031000 tx=0x7f42500046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.843+0000 7f4265a82640 1 -- 172.21.15.139:0/3556870019 shutdown_connections 2024-09-16T00:08:30.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.843+0000 7f4265a82640 1 --2- 172.21.15.139:0/3556870019 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f424c06c2e0 0x7f424c06e7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.843+0000 7f4265a82640 1 --2- 172.21.15.139:0/3556870019 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42601621d0 0x7f426015ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.843+0000 7f4265a82640 1 --2- 172.21.15.139:0/3556870019 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4260158e60 0x7f42601537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.843+0000 7f4265a82640 1 --2- 172.21.15.139:0/3556870019 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42600c8740 0x7f4260150e90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:30.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.843+0000 7f4265a82640 1 -- 172.21.15.139:0/3556870019 >> 172.21.15.139:0/3556870019 conn(0x7f4260070460 msgr2=0x7f4260160f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:30.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.843+0000 7f4265a82640 1 -- 172.21.15.139:0/3556870019 shutdown_connections 2024-09-16T00:08:30.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:30.844+0000 7f4265a82640 1 -- 172.21.15.139:0/3556870019 wait complete. 2024-09-16T00:08:30.860 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T00:08:30.874 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:30.873+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:30.917 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:30.917 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:30.926 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:08:30.974 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:30.974+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:31.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.014+0000 7f89ec123640 1 Processor -- start 2024-09-16T00:08:31.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.014+0000 7f89ec123640 1 -- start start 2024-09-16T00:08:31.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.015+0000 7f89ec123640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f89e40ce430 0x7f89e40b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:31.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.015+0000 7f89ec123640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f89e40ce810 0x7f89e40c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:31.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.015+0000 7f89ec123640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89e40703e0 con 0x7f89e40c9430 2024-09-16T00:08:31.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.015+0000 7f89e9ebf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f89e40ce430 0x7f89e40b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:31.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.016+0000 7f89e9ebf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f89e40ce430 0x7f89e40b31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43114/0 (socket says 172.21.15.139:43114) 2024-09-16T00:08:31.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.016+0000 7f89ec123640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f89e40c5750 con 0x7f89e40ce430 2024-09-16T00:08:31.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.016+0000 7f89e9ebf640 1 -- 172.21.15.139:0/4189125991 learned_addr learned my addr 172.21.15.139:0/4189125991 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:31.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.016+0000 7f89ec123640 1 -- 172.21.15.139:0/4189125991 --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f89e4070680 con 0x7f89e40ce810 2024-09-16T00:08:31.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.016+0000 7f89e96be640 1 --2- 172.21.15.139:0/4189125991 >> v2:172.21.15.164:3300/0 conn(0x7f89e40ce810 0x7f89e40c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:31.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.016+0000 7f89e9ebf640 1 -- 172.21.15.139:0/4189125991 >> v2:172.21.15.164:3300/0 conn(0x7f89e40ce810 msgr2=0x7f89e40c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:31.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.016+0000 7f89e9ebf640 1 --2- 172.21.15.139:0/4189125991 >> v2:172.21.15.164:3300/0 conn(0x7f89e40ce810 0x7f89e40c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:31.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.016+0000 7f89e9ebf640 1 -- 172.21.15.139:0/4189125991 >> v1:172.21.15.164:6789/0 conn(0x7f89e40c9430 legacy=0x7f89e40ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:31.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.016+0000 7f89e9ebf640 1 -- 172.21.15.139:0/4189125991 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e4070920 con 0x7f89e40ce430 2024-09-16T00:08:31.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.017+0000 7f89e9ebf640 1 --2- 172.21.15.139:0/4189125991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89e40ce430 0x7f89e40b31c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f89e40c1930 tx=0x7f89e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2088f540fd178ad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:31.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.017+0000 7f89e8ebd640 1 -- 172.21.15.139:0/4189125991 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0037070 con 0x7f89e40ce430 2024-09-16T00:08:31.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.017+0000 7f89e8ebd640 1 -- 172.21.15.139:0/4189125991 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89e0032440 con 0x7f89e40ce430 2024-09-16T00:08:31.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.017+0000 7f89e8ebd640 1 -- 172.21.15.139:0/4189125991 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0032810 con 0x7f89e40ce430 2024-09-16T00:08:31.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.018+0000 7f89ec123640 1 -- 172.21.15.139:0/4189125991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89e40ce430 msgr2=0x7f89e40b31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:31.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.018+0000 7f89ec123640 1 --2- 172.21.15.139:0/4189125991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89e40ce430 0x7f89e40b31c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f89e40c1930 tx=0x7f89e002a270 comp rx=0 tx=0).stop 2024-09-16T00:08:31.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.018+0000 7f89ec123640 1 -- 172.21.15.139:0/4189125991 shutdown_connections 2024-09-16T00:08:31.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.018+0000 7f89ec123640 1 --2- 172.21.15.139:0/4189125991 >> v2:172.21.15.164:3300/0 conn(0x7f89e40ce810 0x7f89e40c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:31.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.018+0000 7f89ec123640 1 --2- 172.21.15.139:0/4189125991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89e40ce430 0x7f89e40b31c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:31.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.018+0000 7f89ec123640 1 -- 172.21.15.139:0/4189125991 >> 172.21.15.139:0/4189125991 conn(0x7f89e406c150 msgr2=0x7f89e4067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:31.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.018+0000 7f89ec123640 1 -- 172.21.15.139:0/4189125991 shutdown_connections 2024-09-16T00:08:31.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.018+0000 7f89ec123640 1 -- 172.21.15.139:0/4189125991 wait complete. 2024-09-16T00:08:31.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.020+0000 7f89ec123640 1 Processor -- start 2024-09-16T00:08:31.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.020+0000 7f89ec123640 1 -- start start 2024-09-16T00:08:31.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89ec123640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89e40c9430 0x7f89e40c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:31.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89ec123640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89e40ce430 0x7f89e414d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:31.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89ec123640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f89e40ce810 0x7f89e414dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:31.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89ec123640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f89e40b3700 con 0x7f89e40c9430 2024-09-16T00:08:31.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89ec123640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f89e4070ee0 con 0x7f89e40ce430 2024-09-16T00:08:31.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89ec123640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f89e40b3880 con 0x7f89e40ce810 2024-09-16T00:08:31.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89e9ebf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89e40c9430 0x7f89e40c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:31.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89e9ebf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89e40c9430 0x7f89e40c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43128/0 (socket says 172.21.15.139:43128) 2024-09-16T00:08:31.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89e9ebf640 1 -- 172.21.15.139:0/3937377604 learned_addr learned my addr 172.21.15.139:0/3937377604 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:31.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89e96be640 1 --2- 172.21.15.139:0/3937377604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89e40ce430 0x7f89e414d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:31.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89ea6c0640 1 --2- 172.21.15.139:0/3937377604 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f89e40ce810 0x7f89e414dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:31.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89e9ebf640 1 -- 172.21.15.139:0/3937377604 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f89e40ce810 msgr2=0x7f89e414dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:31.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89e9ebf640 1 --2- 172.21.15.139:0/3937377604 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f89e40ce810 0x7f89e414dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:31.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89e9ebf640 1 -- 172.21.15.139:0/3937377604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89e40ce430 msgr2=0x7f89e414d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:31.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89e9ebf640 1 --2- 172.21.15.139:0/3937377604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89e40ce430 0x7f89e414d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:31.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89e9ebf640 1 -- 172.21.15.139:0/3937377604 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e41522e0 con 0x7f89e40c9430 2024-09-16T00:08:31.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89ea6c0640 1 --2- 172.21.15.139:0/3937377604 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f89e40ce810 0x7f89e414dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:08:31.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.021+0000 7f89e96be640 1 --2- 172.21.15.139:0/3937377604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89e40ce430 0x7f89e414d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:08:31.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.022+0000 7f89e9ebf640 1 --2- 172.21.15.139:0/3937377604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89e40c9430 0x7f89e40c6d60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f89e0000c00 tx=0x7f89e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:31.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.022+0000 7f89da7fc640 1 -- 172.21.15.139:0/3937377604 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0037040 con 0x7f89e40c9430 2024-09-16T00:08:31.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.022+0000 7f89da7fc640 1 -- 172.21.15.139:0/3937377604 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89e0042070 con 0x7f89e40c9430 2024-09-16T00:08:31.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.022+0000 7f89da7fc640 1 -- 172.21.15.139:0/3937377604 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0030880 con 0x7f89e40c9430 2024-09-16T00:08:31.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.022+0000 7f89ec123640 1 -- 172.21.15.139:0/3937377604 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e40c4dd0 con 0x7f89e40c9430 2024-09-16T00:08:31.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.022+0000 7f89ec123640 1 -- 172.21.15.139:0/3937377604 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e4151370 con 0x7f89e40c9430 2024-09-16T00:08:31.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.023+0000 7f89ec123640 1 -- 172.21.15.139:0/3937377604 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89e40cefb0 con 0x7f89e40c9430 2024-09-16T00:08:31.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.024+0000 7f89da7fc640 1 -- 172.21.15.139:0/3937377604 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f89e0030a20 con 0x7f89e40c9430 2024-09-16T00:08:31.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.025+0000 7f89da7fc640 1 --2- 172.21.15.139:0/3937377604 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f89c80706f0 0x7f89c8072bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:31.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.025+0000 7f89da7fc640 1 -- 172.21.15.139:0/3937377604 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5915+0+0 (secure 0 0 0) 0x7f89e00abe90 con 0x7f89e40c9430 2024-09-16T00:08:31.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.025+0000 7f89e96be640 1 --2- 172.21.15.139:0/3937377604 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f89c80706f0 0x7f89c8072bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:31.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.026+0000 7f89e96be640 1 --2- 172.21.15.139:0/3937377604 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f89c80706f0 0x7f89c8072bb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f89e40c5020 tx=0x7f89d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:31.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.034+0000 7f89da7fc640 1 -- 172.21.15.139:0/3937377604 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f89e40cefb0 con 0x7f89e40c9430 2024-09-16T00:08:31.076 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:31.075+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:31.095 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:31.096 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:31.105 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:08:31.177 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:31.176+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:31.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.222+0000 7f89ec123640 1 -- 172.21.15.139:0/3937377604 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) -- 0x7f89e406ffc0 con 0x7f89e40c9430 2024-09-16T00:08:31.250 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:31.250 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:31.259 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:08:31.278 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:31.277+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:31.366 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:31.366 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:31.376 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:08:31.378 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:31.378+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:31.479 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:31.478+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:31.483 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:31.483 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:31.492 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:08:31.580 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:31.579+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:31.599 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:31.599 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:31.612 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:08:31.681 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:31.681+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:31.720 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:31.720 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:31.729 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:08:31.783 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:31.782+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:31.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.844+0000 7f89da7fc640 1 -- 172.21.15.139:0/3937377604 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v42) ==== 113+0+0 (secure 0 0 0) 0x7f89e007a440 con 0x7f89e40c9430 2024-09-16T00:08:32.003 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T00:08:32.003 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T00:08:32.004 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:08:32.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.847+0000 7f89ec123640 1 -- 172.21.15.139:0/3937377604 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f89c80706f0 msgr2=0x7f89c8072bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.847+0000 7f89ec123640 1 --2- 172.21.15.139:0/3937377604 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f89c80706f0 0x7f89c8072bb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f89e40c5020 tx=0x7f89d4008040 comp rx=0 tx=0).stop 2024-09-16T00:08:32.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.847+0000 7f89ec123640 1 -- 172.21.15.139:0/3937377604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89e40c9430 msgr2=0x7f89e40c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.847+0000 7f89ec123640 1 --2- 172.21.15.139:0/3937377604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89e40c9430 0x7f89e40c6d60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f89e0000c00 tx=0x7f89e00046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.847+0000 7f89ec123640 1 -- 172.21.15.139:0/3937377604 shutdown_connections 2024-09-16T00:08:32.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.847+0000 7f89ec123640 1 --2- 172.21.15.139:0/3937377604 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f89c80706f0 0x7f89c8072bb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.847+0000 7f89ec123640 1 --2- 172.21.15.139:0/3937377604 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f89e40ce810 0x7f89e414dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.847+0000 7f89ec123640 1 --2- 172.21.15.139:0/3937377604 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f89e40ce430 0x7f89e414d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.847+0000 7f89ec123640 1 --2- 172.21.15.139:0/3937377604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89e40c9430 0x7f89e40c6d60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.847+0000 7f89ec123640 1 -- 172.21.15.139:0/3937377604 >> 172.21.15.139:0/3937377604 conn(0x7f89e406c150 msgr2=0x7f89e40c7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:32.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.847+0000 7f89ec123640 1 -- 172.21.15.139:0/3937377604 shutdown_connections 2024-09-16T00:08:32.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:31.847+0000 7f89ec123640 1 -- 172.21.15.139:0/3937377604 wait complete. 2024-09-16T00:08:32.007 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T00:08:32.007 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:32.008 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:32.008 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:08:32.008 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:32.003+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:32.021 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:08:32.106 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:32.106+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:32.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.183+0000 7fbfdf59e640 1 Processor -- start 2024-09-16T00:08:32.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.183+0000 7fbfdf59e640 1 -- start start 2024-09-16T00:08:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.183+0000 7f220c677640 1 Processor -- start 2024-09-16T00:08:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.184+0000 7f220c677640 1 -- start start 2024-09-16T00:08:32.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.184+0000 7fbfdf59e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbfe0150c50 0x7fbfe006ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.184+0000 7fbfdf59e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbfe0151030 0x7fbfe0070450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.184+0000 7fbfdf59e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfe015a470 con 0x7fbfe00c8b00 2024-09-16T00:08:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.184+0000 7fbfdf59e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fbfe0068060 con 0x7fbfe0150c50 2024-09-16T00:08:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.184+0000 7fbfdf59e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe0072d90 con 0x7fbfe0151030 2024-09-16T00:08:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.185+0000 7fbfde59c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbfe0150c50 0x7fbfe006ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.185+0000 7fbfded9d640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fbfe00c8b00 0x7fbfe0067950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:59790/0 (socket says 172.21.15.139:59790) 2024-09-16T00:08:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.185+0000 7fbfded9d640 1 -- 172.21.15.139:0/286219406 learned_addr learned my addr 172.21.15.139:0/286219406 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:32.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.185+0000 7fbfddd9b640 1 --2- 172.21.15.139:0/286219406 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbfe0151030 0x7fbfe0070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.185+0000 7f220c677640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f22040c63c0 0x7f220406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.185+0000 7f220a413640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f22040c63c0 0x7f220406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.185+0000 7f220a413640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f22040c63c0 0x7f220406f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:34140/0 (socket says 172.21.15.139:34140) 2024-09-16T00:08:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.185+0000 7f220a413640 1 -- 172.21.15.139:0/451559327 learned_addr learned my addr 172.21.15.139:0/451559327 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.185+0000 7f220c677640 1 -- 172.21.15.139:0/451559327 --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f220406ed80 con 0x7f22040c67a0 2024-09-16T00:08:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.186+0000 7f220c677640 1 -- 172.21.15.139:0/451559327 --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22040c90f0 con 0x7f2204151a30 2024-09-16T00:08:32.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.186+0000 7f220c677640 1 -- 172.21.15.139:0/451559327 --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f22040c93d0 con 0x7f22040c63c0 2024-09-16T00:08:32.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.185+0000 7fbfdd59a640 1 -- 172.21.15.139:0/286219406 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 425602284 0 0) 0x7fbfe015a470 con 0x7fbfe00c8b00 2024-09-16T00:08:32.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.185+0000 7fbfdd59a640 1 -- 172.21.15.139:0/286219406 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfc8003520 con 0x7fbfe00c8b00 2024-09-16T00:08:32.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.185+0000 7fbfde59c640 1 -- 172.21.15.139:0/286219406 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbfe0151030 msgr2=0x7fbfe0070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:32.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.185+0000 7fbfde59c640 1 --2- 172.21.15.139:0/286219406 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbfe0151030 0x7fbfe0070450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.185+0000 7fbfde59c640 1 -- 172.21.15.139:0/286219406 >> v1:172.21.15.164:6789/0 conn(0x7fbfe00c8b00 legacy=0x7fbfe0067950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.185+0000 7fbfde59c640 1 -- 172.21.15.139:0/286219406 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe015a750 con 0x7fbfe0150c50 2024-09-16T00:08:32.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.186+0000 7f2209411640 1 -- 172.21.15.139:0/451559327 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2289179715 0 0) 0x7f220406ed80 con 0x7f22040c67a0 2024-09-16T00:08:32.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.186+0000 7f2209411640 1 -- 172.21.15.139:0/451559327 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21f0003520 con 0x7f22040c67a0 2024-09-16T00:08:32.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.186+0000 7f2209411640 1 -- 172.21.15.139:0/451559327 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1767927999 0 0) 0x7f22040c90f0 con 0x7f2204151a30 2024-09-16T00:08:32.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.186+0000 7f2209411640 1 -- 172.21.15.139:0/451559327 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21f0003880 con 0x7f2204151a30 2024-09-16T00:08:32.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.186+0000 7f220a413640 1 -- 172.21.15.139:0/451559327 >> v1:172.21.15.164:6789/0 conn(0x7f2204151a30 legacy=0x7f2204161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.186+0000 7f220a413640 1 -- 172.21.15.139:0/451559327 >> v1:172.21.15.139:6789/0 conn(0x7f22040c67a0 legacy=0x7f2204151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.186+0000 7f220a413640 1 -- 172.21.15.139:0/451559327 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22040c96b0 con 0x7f22040c63c0 2024-09-16T00:08:32.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.187+0000 7fbfde59c640 1 --2- 172.21.15.139:0/286219406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe0150c50 0x7fbfe006ff10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00097a0 tx=0x7fbfd002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f124979b6e2babb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.187+0000 7f220a413640 1 --2- 172.21.15.139:0/451559327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22040c63c0 0x7f220406f930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2200009870 tx=0x7f220002e7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25596f620fda01f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.187+0000 7f2209411640 1 -- 172.21.15.139:0/451559327 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220003a070 con 0x7f22040c63c0 2024-09-16T00:08:32.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.187+0000 7f2209411640 1 -- 172.21.15.139:0/451559327 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2200033710 con 0x7f22040c63c0 2024-09-16T00:08:32.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.187+0000 7f2209411640 1 -- 172.21.15.139:0/451559327 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200033ae0 con 0x7f22040c63c0 2024-09-16T00:08:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.187+0000 7fbfdd59a640 1 -- 172.21.15.139:0/286219406 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0037070 con 0x7fbfe0150c50 2024-09-16T00:08:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.187+0000 7fbfdd59a640 1 -- 172.21.15.139:0/286219406 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd0032440 con 0x7fbfe0150c50 2024-09-16T00:08:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.187+0000 7fbfdd59a640 1 -- 172.21.15.139:0/286219406 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0032810 con 0x7fbfe0150c50 2024-09-16T00:08:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.188+0000 7fbfdf59e640 1 -- 172.21.15.139:0/286219406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe0150c50 msgr2=0x7fbfe006ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.188+0000 7fbfdf59e640 1 --2- 172.21.15.139:0/286219406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe0150c50 0x7fbfe006ff10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00097a0 tx=0x7fbfd002a210 comp rx=0 tx=0).stop 2024-09-16T00:08:32.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.188+0000 7fbfdf59e640 1 -- 172.21.15.139:0/286219406 shutdown_connections 2024-09-16T00:08:32.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.188+0000 7fbfdf59e640 1 --2- 172.21.15.139:0/286219406 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbfe0151030 0x7fbfe0070450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.188+0000 7fbfdf59e640 1 --2- 172.21.15.139:0/286219406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe0150c50 0x7fbfe006ff10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.188+0000 7fbfdf59e640 1 -- 172.21.15.139:0/286219406 >> 172.21.15.139:0/286219406 conn(0x7fbfe006f3f0 msgr2=0x7fbfe006f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:32.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.188+0000 7fbfdf59e640 1 -- 172.21.15.139:0/286219406 shutdown_connections 2024-09-16T00:08:32.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.188+0000 7fbfdf59e640 1 -- 172.21.15.139:0/286219406 wait complete. 2024-09-16T00:08:32.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.188+0000 7f220c677640 1 -- 172.21.15.139:0/451559327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22040c63c0 msgr2=0x7f220406f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.188+0000 7f220c677640 1 --2- 172.21.15.139:0/451559327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22040c63c0 0x7f220406f930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2200009870 tx=0x7f220002e7d0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.188+0000 7f220c677640 1 -- 172.21.15.139:0/451559327 shutdown_connections 2024-09-16T00:08:32.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.188+0000 7f220c677640 1 --2- 172.21.15.139:0/451559327 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22040c63c0 0x7f220406f930 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.188+0000 7f220c677640 1 -- 172.21.15.139:0/451559327 >> 172.21.15.139:0/451559327 conn(0x7f2204072b30 msgr2=0x7f2204070510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:32.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.189+0000 7f220c677640 1 -- 172.21.15.139:0/451559327 shutdown_connections 2024-09-16T00:08:32.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.189+0000 7f220c677640 1 -- 172.21.15.139:0/451559327 wait complete. 2024-09-16T00:08:32.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.190+0000 7fbfdf59e640 1 Processor -- start 2024-09-16T00:08:32.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.190+0000 7fbfdf59e640 1 -- start start 2024-09-16T00:08:32.194 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:32.194 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:32.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.191+0000 7f220c677640 1 Processor -- start 2024-09-16T00:08:32.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.191+0000 7f220c677640 1 -- start start 2024-09-16T00:08:32.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.191+0000 7fbfdf59e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe00c8b00 0x7fbfe01f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7fbfdf59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfe0150c50 0x7fbfe00c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7fbfdf59e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbfe0151030 0x7fbfe00ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7fbfddd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfe0150c50 0x7fbfe00c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7fbfddd9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfe0150c50 0x7fbfe00c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43142/0 (socket says 172.21.15.139:43142) 2024-09-16T00:08:32.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7fbfde59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe00c8b00 0x7fbfe01f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7fbfde59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe00c8b00 0x7fbfe01f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:34148/0 (socket says 172.21.15.139:34148) 2024-09-16T00:08:32.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7f220c677640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22040c63c0 0x7f220414aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7fbfddd9b640 1 -- 172.21.15.139:0/3578205631 learned_addr learned my addr 172.21.15.139:0/3578205631 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:32.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7f220c677640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22040c67a0 0x7f2204158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7fbfdf59e640 1 -- 172.21.15.139:0/3578205631 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe0072860 con 0x7fbfe0150c50 2024-09-16T00:08:32.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7fbfdf59e640 1 -- 172.21.15.139:0/3578205631 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe015ad10 con 0x7fbfe00c8b00 2024-09-16T00:08:32.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7fbfded9d640 1 --2- 172.21.15.139:0/3578205631 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbfe0151030 0x7fbfe00ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7f220a413640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22040c63c0 0x7f220414aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7f220a413640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22040c63c0 0x7f220414aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:34162/0 (socket says 172.21.15.139:34162) 2024-09-16T00:08:32.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7f220a413640 1 -- 172.21.15.139:0/2698950411 learned_addr learned my addr 172.21.15.139:0/2698950411 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:32.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7fbfdf59e640 1 -- 172.21.15.139:0/3578205631 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe015a030 con 0x7fbfe0151030 2024-09-16T00:08:32.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7f220c677640 1 --2- 172.21.15.139:0/2698950411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2204151a30 0x7f2204158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7f2209c12640 1 --2- 172.21.15.139:0/2698950411 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22040c67a0 0x7f2204158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.192+0000 7f220c677640 1 -- 172.21.15.139:0/2698950411 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f220406f410 con 0x7f2204151a30 2024-09-16T00:08:32.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.193+0000 7f220ac14640 1 --2- 172.21.15.139:0/2698950411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2204151a30 0x7f2204158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.193+0000 7f220c677640 1 -- 172.21.15.139:0/2698950411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f220406f130 con 0x7f22040c63c0 2024-09-16T00:08:32.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.193+0000 7f220c677640 1 -- 172.21.15.139:0/2698950411 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f22040c46a0 con 0x7f22040c67a0 2024-09-16T00:08:32.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.193+0000 7fbfddd9b640 1 -- 172.21.15.139:0/3578205631 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbfe0151030 msgr2=0x7fbfe00ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.193+0000 7fbfddd9b640 1 --2- 172.21.15.139:0/3578205631 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbfe0151030 0x7fbfe00ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.193+0000 7fbfddd9b640 1 -- 172.21.15.139:0/3578205631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe00c8b00 msgr2=0x7fbfe01f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.193+0000 7fbfddd9b640 1 --2- 172.21.15.139:0/3578205631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe00c8b00 0x7fbfe01f4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.193+0000 7fbfddd9b640 1 -- 172.21.15.139:0/3578205631 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe01f48b0 con 0x7fbfe0150c50 2024-09-16T00:08:32.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.193+0000 7fbfddd9b640 1 --2- 172.21.15.139:0/3578205631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfe0150c50 0x7fbfe00c5cc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400e800 tx=0x7fbfd400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.193+0000 7f220a413640 1 -- 172.21.15.139:0/2698950411 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22040c67a0 msgr2=0x7f2204158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.193+0000 7f220a413640 1 --2- 172.21.15.139:0/2698950411 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22040c67a0 0x7f2204158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.193+0000 7fbfceffd640 1 -- 172.21.15.139:0/3578205631 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4017070 con 0x7fbfe0150c50 2024-09-16T00:08:32.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.193+0000 7fbfceffd640 1 -- 172.21.15.139:0/3578205631 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd400c940 con 0x7fbfe0150c50 2024-09-16T00:08:32.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.193+0000 7f220a413640 1 -- 172.21.15.139:0/2698950411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2204151a30 msgr2=0x7f2204158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.193+0000 7fbfceffd640 1 -- 172.21.15.139:0/3578205631 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd400ccf0 con 0x7fbfe0150c50 2024-09-16T00:08:32.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.194+0000 7fbfdf59e640 1 -- 172.21.15.139:0/3578205631 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe00cbd90 con 0x7fbfe0150c50 2024-09-16T00:08:32.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.194+0000 7f220a413640 1 --2- 172.21.15.139:0/2698950411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2204151a30 0x7f2204158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.194+0000 7f220a413640 1 -- 172.21.15.139:0/2698950411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2204152060 con 0x7f22040c63c0 2024-09-16T00:08:32.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.194+0000 7f220ac14640 1 --2- 172.21.15.139:0/2698950411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2204151a30 0x7f2204158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:32.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.194+0000 7f220a413640 1 --2- 172.21.15.139:0/2698950411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22040c63c0 0x7f220414aa00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2200009990 tx=0x7f22000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.195+0000 7fbfdf59e640 1 -- 172.21.15.139:0/3578205631 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe01f3b70 con 0x7fbfe0150c50 2024-09-16T00:08:32.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.194+0000 7f21faffd640 1 -- 172.21.15.139:0/2698950411 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220003a040 con 0x7f22040c63c0 2024-09-16T00:08:32.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.194+0000 7f21faffd640 1 -- 172.21.15.139:0/2698950411 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2200045070 con 0x7f22040c63c0 2024-09-16T00:08:32.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.194+0000 7f21faffd640 1 -- 172.21.15.139:0/2698950411 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200035c60 con 0x7f22040c63c0 2024-09-16T00:08:32.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.195+0000 7f220c677640 1 -- 172.21.15.139:0/2698950411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22041ec250 con 0x7f22040c63c0 2024-09-16T00:08:32.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.196+0000 7f220c677640 1 -- 172.21.15.139:0/2698950411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22041e1d00 con 0x7f22040c63c0 2024-09-16T00:08:32.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.196+0000 7fbfdf59e640 1 -- 172.21.15.139:0/3578205631 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe00cc2b0 con 0x7fbfe0150c50 2024-09-16T00:08:32.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.196+0000 7fbfceffd640 1 -- 172.21.15.139:0/3578205631 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbfd401a070 con 0x7fbfe0150c50 2024-09-16T00:08:32.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.197+0000 7fbfceffd640 1 --2- 172.21.15.139:0/3578205631 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbfc006c2b0 0x7fbfc006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.197+0000 7fbfceffd640 1 -- 172.21.15.139:0/3578205631 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5915+0+0 (secure 0 0 0) 0x7fbfd408b780 con 0x7fbfe0150c50 2024-09-16T00:08:32.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.198+0000 7fbfde59c640 1 --2- 172.21.15.139:0/3578205631 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbfc006c2b0 0x7fbfc006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.198+0000 7fbfde59c640 1 --2- 172.21.15.139:0/3578205631 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbfc006c2b0 0x7fbfc006e770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0034000 tx=0x7fbfd0040040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.198+0000 7f21faffd640 1 -- 172.21.15.139:0/2698950411 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f220003d070 con 0x7f22040c63c0 2024-09-16T00:08:32.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.199+0000 7f220c677640 1 -- 172.21.15.139:0/2698950411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22040ce290 con 0x7f22040c63c0 2024-09-16T00:08:32.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.199+0000 7f21faffd640 1 --2- 172.21.15.139:0/2698950411 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f21e806c380 0x7f21e806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.199+0000 7f21faffd640 1 -- 172.21.15.139:0/2698950411 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5915+0+0 (secure 0 0 0) 0x7f22000af930 con 0x7f22040c63c0 2024-09-16T00:08:32.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.199+0000 7f2209c12640 1 --2- 172.21.15.139:0/2698950411 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f21e806c380 0x7f21e806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.200+0000 7f2209c12640 1 --2- 172.21.15.139:0/2698950411 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f21e806c380 0x7f21e806e840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2204150ec0 tx=0x7f21f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.204+0000 7f21faffd640 1 -- 172.21.15.139:0/2698950411 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f220007e040 con 0x7f22040c63c0 2024-09-16T00:08:32.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.204+0000 7fbfceffd640 1 -- 172.21.15.139:0/3578205631 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbfe00cc2b0 con 0x7fbfe0150c50 2024-09-16T00:08:32.206 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:08:32.251 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:32.250+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:32.309 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:32.308+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:32.410 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:32.409+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:32.415 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:32.415 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:32.428 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:08:32.481 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:08:32.511 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:32.510+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:32.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.515+0000 7f220c677640 1 -- 172.21.15.139:0/2698950411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) -- 0x7f22040cd880 con 0x7f22040c63c0 2024-09-16T00:08:32.545 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:32.545 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:32.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.548+0000 7fbfdf59e640 1 -- 172.21.15.139:0/3578205631 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fbfe006d280 con 0x7fbfe0150c50 2024-09-16T00:08:32.555 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:08:32.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.589+0000 7f0bd8b96640 1 Processor -- start 2024-09-16T00:08:32.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.589+0000 7f0bd8b96640 1 -- start start 2024-09-16T00:08:32.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.590+0000 7f0bd8b96640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0bd40d0490 0x7f0bd40b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.590+0000 7f0bd8b96640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bd40b3180 con 0x7f0bd40b4750 2024-09-16T00:08:32.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.590+0000 7f0bd8b96640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bd415adc0 con 0x7f0bd40d0870 2024-09-16T00:08:32.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.590+0000 7f0bd8b96640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f0bd415b940 con 0x7f0bd40d0490 2024-09-16T00:08:32.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.590+0000 7f0bd2d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f0bd40b4750 0x7f0bd41546a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:50272/0 (socket says 172.21.15.139:50272) 2024-09-16T00:08:32.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.590+0000 7f0bd2d9d640 1 -- 172.21.15.139:0/4068359032 learned_addr learned my addr 172.21.15.139:0/4068359032 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:32.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.591+0000 7f0bd259c640 1 --2- 172.21.15.139:0/4068359032 >> v2:172.21.15.164:3300/0 conn(0x7f0bd40d0490 0x7f0bd40b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.591+0000 7f0bd159a640 1 -- 172.21.15.139:0/4068359032 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1041377581 0 0) 0x7f0bd40b3180 con 0x7f0bd40b4750 2024-09-16T00:08:32.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.591+0000 7f0bd159a640 1 -- 172.21.15.139:0/4068359032 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bbc003520 con 0x7f0bd40b4750 2024-09-16T00:08:32.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.591+0000 7f0bd259c640 1 -- 172.21.15.139:0/4068359032 >> v1:172.21.15.164:6789/0 conn(0x7f0bd40d0870 legacy=0x7f0bd40b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.591+0000 7f0bd159a640 1 -- 172.21.15.139:0/4068359032 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 514698066 0 0) 0x7f0bd415adc0 con 0x7f0bd40d0870 2024-09-16T00:08:32.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.591+0000 7f0bd259c640 1 -- 172.21.15.139:0/4068359032 >> v1:172.21.15.139:6789/0 conn(0x7f0bd40b4750 legacy=0x7f0bd41546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.591+0000 7f0bd259c640 1 -- 172.21.15.139:0/4068359032 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd415b0a0 con 0x7f0bd40d0490 2024-09-16T00:08:32.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.592+0000 7f0bd259c640 1 --2- 172.21.15.139:0/4068359032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bd40d0490 0x7f0bd40b2c40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40097d0 tx=0x7f0bc402a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e07998c91c58bd7e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.592+0000 7f0bd159a640 1 -- 172.21.15.139:0/4068359032 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4037070 con 0x7f0bd40d0490 2024-09-16T00:08:32.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.592+0000 7f0bd159a640 1 -- 172.21.15.139:0/4068359032 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc4032640 con 0x7f0bd40d0490 2024-09-16T00:08:32.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.592+0000 7f0bd159a640 1 -- 172.21.15.139:0/4068359032 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4032a10 con 0x7f0bd40d0490 2024-09-16T00:08:32.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.593+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4068359032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bd40d0490 msgr2=0x7f0bd40b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.593+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4068359032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bd40d0490 0x7f0bd40b2c40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40097d0 tx=0x7f0bc402a300 comp rx=0 tx=0).stop 2024-09-16T00:08:32.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.593+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4068359032 shutdown_connections 2024-09-16T00:08:32.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.593+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4068359032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bd40d0490 0x7f0bd40b2c40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.593+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4068359032 >> 172.21.15.139:0/4068359032 conn(0x7f0bd406c520 msgr2=0x7f0bd406c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:32.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.593+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4068359032 shutdown_connections 2024-09-16T00:08:32.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.593+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4068359032 wait complete. 2024-09-16T00:08:32.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.594+0000 7f0bd8b96640 1 Processor -- start 2024-09-16T00:08:32.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.594+0000 7f0bd8b96640 1 -- start start 2024-09-16T00:08:32.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.595+0000 7f0bd8b96640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bd40b4750 0x7f0bd414acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.595+0000 7f0bd8b96640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bd40d0490 0x7f0bd40c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.595+0000 7f0bd8b96640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0bd40d0870 0x7f0bd414fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.595+0000 7f0bd8b96640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd415b660 con 0x7f0bd40d0490 2024-09-16T00:08:32.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.595+0000 7f0bd8b96640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd40b3bb0 con 0x7f0bd40b4750 2024-09-16T00:08:32.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.595+0000 7f0bd8b96640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd415a980 con 0x7f0bd40d0870 2024-09-16T00:08:32.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.595+0000 7f0bd259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bd40b4750 0x7f0bd414acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.595+0000 7f0bd1d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bd40d0490 0x7f0bd40c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.595+0000 7f0bd259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bd40b4750 0x7f0bd414acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:34170/0 (socket says 172.21.15.139:34170) 2024-09-16T00:08:32.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.595+0000 7f0bd259c640 1 -- 172.21.15.139:0/4059165522 learned_addr learned my addr 172.21.15.139:0/4059165522 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:32.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.595+0000 7f0bd2d9d640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0bd40d0870 0x7f0bd414fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.595+0000 7f0bd259c640 1 -- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0bd40d0870 msgr2=0x7f0bd414fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.595+0000 7f0bd259c640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0bd40d0870 0x7f0bd414fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.595+0000 7f0bd259c640 1 -- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bd40d0490 msgr2=0x7f0bd40c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.596+0000 7f0bd259c640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bd40d0490 0x7f0bd40c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.596+0000 7f0bd259c640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd40c96e0 con 0x7f0bd40b4750 2024-09-16T00:08:32.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.596+0000 7f0bd1d9b640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0bd40d0490 0x7f0bd40c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:32.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.596+0000 7f0bd259c640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bd40b4750 0x7f0bd414acb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4035040 tx=0x7f0bc40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.596+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4041020 con 0x7f0bd40b4750 2024-09-16T00:08:32.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.596+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc40306f0 con 0x7f0bd40b4750 2024-09-16T00:08:32.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.596+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4037040 con 0x7f0bd40b4750 2024-09-16T00:08:32.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.596+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd40c9980 con 0x7f0bd40b4750 2024-09-16T00:08:32.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.596+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd41f08d0 con 0x7f0bd40b4750 2024-09-16T00:08:32.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.597+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f0bd40b3430 con 0x7f0bd40b4750 2024-09-16T00:08:32.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.598+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0bc4044020 con 0x7f0bd40b4750 2024-09-16T00:08:32.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.598+0000 7f0bc2ffd640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0bb406c2b0 0x7f0bb406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.598+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5915+0+0 (secure 0 0 0) 0x7f0bc40abc20 con 0x7f0bd40b4750 2024-09-16T00:08:32.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.598+0000 7f0bd1d9b640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0bb406c2b0 0x7f0bb406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.598+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v42) ==== 54+0+16 (secure 0 0 0) 0x7f0bc404b030 con 0x7f0bd40b4750 2024-09-16T00:08:32.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.599+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f0bd414e660 0x7f0bd41f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.599+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0bd41f0fe0 con 0x7f0bd414e660 2024-09-16T00:08:32.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.599+0000 7f0bd2d9d640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f0bd414e660 0x7f0bd41f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.599+0000 7f0bd1d9b640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0bb406c2b0 0x7f0bb406e770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8007fd0 tx=0x7f0bc8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.599+0000 7f0bd2d9d640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f0bd414e660 0x7f0bd41f0aa0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.600+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0bd41f0fe0 con 0x7f0bd414e660 2024-09-16T00:08:32.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.628+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0bd40cc560 con 0x7f0bd414e660 2024-09-16T00:08:32.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.630+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0bd40cc560 con 0x7f0bd414e660 2024-09-16T00:08:32.631 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:32.632 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:32.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.631+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f0bd40c7880 0x7f0bd40c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.631+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f0bd40cbfa0 con 0x7f0bd40c7880 2024-09-16T00:08:32.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.631+0000 7f0bd259c640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f0bd40c7880 0x7f0bd40c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.631+0000 7f0bd259c640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f0bd40c7880 0x7f0bd40c7c60 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.632+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0bd40cbfa0 con 0x7f0bd40c7880 2024-09-16T00:08:32.646 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:32.645+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:32.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.659+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0bd40cbcc0 con 0x7f0bd40c7880 2024-09-16T00:08:32.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.661+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0bd40cbcc0 con 0x7f0bd40c7880 2024-09-16T00:08:32.662 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:32.663 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:32.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.661+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f0bd4072de0 0x7f0bd41f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.661+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f0bd41f4260 con 0x7f0bd4072de0 2024-09-16T00:08:32.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.662+0000 7f0bd1d9b640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f0bd4072de0 0x7f0bd41f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.662+0000 7f0bd1d9b640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f0bd4072de0 0x7f0bd41f3d20 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.663+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0bd41f4260 con 0x7f0bd4072de0 2024-09-16T00:08:32.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.690+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0bd41f4260 con 0x7f0bd4072de0 2024-09-16T00:08:32.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.692+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0bd41f4260 con 0x7f0bd4072de0 2024-09-16T00:08:32.693 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:32.694 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:32.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.692+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0bd4062f00 0x7f0bd40ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.692+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f0bd41f4260 con 0x7f0bd4062f00 2024-09-16T00:08:32.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.693+0000 7f0bd2d9d640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0bd4062f00 0x7f0bd40ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.693+0000 7f0bd2d9d640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0bd4062f00 0x7f0bd40ce2b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.694+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0bd41f4260 con 0x7f0bd4062f00 2024-09-16T00:08:32.703 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:32.703 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:32.712 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:08:32.713 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:32.711+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:32.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.721+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0bd41f4260 con 0x7f0bd4062f00 2024-09-16T00:08:32.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.723+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0bd41f4260 con 0x7f0bd4062f00 2024-09-16T00:08:32.724 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:32.725 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:32.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.723+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f0bd40ced00 0x7f0bd402c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.723+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f0bd41f4260 con 0x7f0bd40ced00 2024-09-16T00:08:32.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.724+0000 7f0bd259c640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f0bd40ced00 0x7f0bd402c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.724+0000 7f0bd259c640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f0bd40ced00 0x7f0bd402c640 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.725+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0bd41f4260 con 0x7f0bd40ced00 2024-09-16T00:08:32.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.751+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0bd41f4260 con 0x7f0bd40ced00 2024-09-16T00:08:32.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.754+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0bd41f4260 con 0x7f0bd40ced00 2024-09-16T00:08:32.755 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:32.756 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:32.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.754+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f0bd402d090 0x7f0bd402d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.754+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f0bd40cf160 con 0x7f0bd402d090 2024-09-16T00:08:32.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.754+0000 7f0bd1d9b640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f0bd402d090 0x7f0bd402d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.754+0000 7f0bd1d9b640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f0bd402d090 0x7f0bd402d510 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.755+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0bd40cf160 con 0x7f0bd402d090 2024-09-16T00:08:32.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.774+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0bd40cf160 con 0x7f0bd402d090 2024-09-16T00:08:32.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.776+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0bd40cf160 con 0x7f0bd402d090 2024-09-16T00:08:32.777 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:32.778 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:32.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.776+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f0bd4203090 0x7f0bd4205480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.776+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f0bd40cf160 con 0x7f0bd4203090 2024-09-16T00:08:32.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.776+0000 7f0bd2d9d640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f0bd4203090 0x7f0bd4205480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.777+0000 7f0bd2d9d640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f0bd4203090 0x7f0bd4205480 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.778+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0bd40cf160 con 0x7f0bd4203090 2024-09-16T00:08:32.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.791+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0bd40cf160 con 0x7f0bd4203090 2024-09-16T00:08:32.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.792+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0bd40cf160 con 0x7f0bd4203090 2024-09-16T00:08:32.793 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:32.794 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:32.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.793+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f0bd4205ed0 0x7f0bd42082c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:32.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.793+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f0bd420b090 con 0x7f0bd4205ed0 2024-09-16T00:08:32.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.793+0000 7f0bd259c640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f0bd4205ed0 0x7f0bd42082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:32.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.793+0000 7f0bd259c640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f0bd4205ed0 0x7f0bd42082c0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:32.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.794+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0bd420b090 con 0x7f0bd4205ed0 2024-09-16T00:08:32.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.807+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0bd4208800 con 0x7f0bd4205ed0 2024-09-16T00:08:32.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bc2ffd640 1 -- 172.21.15.139:0/4059165522 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0bd4208800 con 0x7f0bd4205ed0 2024-09-16T00:08:32.810 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:32.811 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f0bd414e660 msgr2=0x7f0bd41f0aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f0bd414e660 0x7f0bd41f0aa0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f0bd40c7880 msgr2=0x7f0bd40c7c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f0bd40c7880 0x7f0bd40c7c60 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f0bd4072de0 msgr2=0x7f0bd41f3d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f0bd4072de0 0x7f0bd41f3d20 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0bd4062f00 msgr2=0x7f0bd40ce2b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0bd4062f00 0x7f0bd40ce2b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f0bd40ced00 msgr2=0x7f0bd402c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f0bd40ced00 0x7f0bd402c640 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f0bd402d090 msgr2=0x7f0bd402d510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f0bd402d090 0x7f0bd402d510 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f0bd4203090 msgr2=0x7f0bd4205480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f0bd4203090 0x7f0bd4205480 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f0bd4205ed0 msgr2=0x7f0bd42082c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f0bd4205ed0 0x7f0bd42082c0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd2d9d640 1 -- 172.21.15.139:0/4059165522 reap_dead start 2024-09-16T00:08:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd2d9d640 1 -- 172.21.15.139:0/4059165522 reap_dead start 2024-09-16T00:08:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd2d9d640 1 -- 172.21.15.139:0/4059165522 reap_dead start 2024-09-16T00:08:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd2d9d640 1 -- 172.21.15.139:0/4059165522 reap_dead start 2024-09-16T00:08:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd2d9d640 1 -- 172.21.15.139:0/4059165522 reap_dead start 2024-09-16T00:08:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0bb406c2b0 msgr2=0x7f0bb406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0bb406c2b0 0x7f0bb406e770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8007fd0 tx=0x7f0bc8009040 comp rx=0 tx=0).stop 2024-09-16T00:08:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bd40b4750 msgr2=0x7f0bd414acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bd40b4750 0x7f0bd414acb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4035040 tx=0x7f0bc40046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 shutdown_connections 2024-09-16T00:08:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0bb406c2b0 0x7f0bb406e770 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 --2- 172.21.15.139:0/4059165522 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0bd40b4750 0x7f0bd414acb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.809+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 >> 172.21.15.139:0/4059165522 conn(0x7f0bd406c520 msgr2=0x7f0bd41585e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.810+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 shutdown_connections 2024-09-16T00:08:32.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.810+0000 7f0bd8b96640 1 -- 172.21.15.139:0/4059165522 wait complete. 2024-09-16T00:08:32.815 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:32.814+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:32.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.856+0000 7fbfceffd640 1 -- 172.21.15.139:0/3578205631 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v43) ==== 97+0+0 (secure 0 0 0) 0x7fbfd40597c0 con 0x7fbfe0150c50 2024-09-16T00:08:32.857 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is set 2024-09-16T00:08:32.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.860+0000 7f21faffd640 1 -- 172.21.15.139:0/2698950411 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v43) ==== 113+0+0 (secure 0 0 0) 0x7f220007d940 con 0x7f22040c63c0 2024-09-16T00:08:32.861 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T00:08:32.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.860+0000 7fbfdf59e640 1 -- 172.21.15.139:0/3578205631 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbfc006c2b0 msgr2=0x7fbfc006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.860+0000 7fbfdf59e640 1 --2- 172.21.15.139:0/3578205631 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbfc006c2b0 0x7fbfc006e770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0034000 tx=0x7fbfd0040040 comp rx=0 tx=0).stop 2024-09-16T00:08:32.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.860+0000 7fbfdf59e640 1 -- 172.21.15.139:0/3578205631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfe0150c50 msgr2=0x7fbfe00c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.860+0000 7fbfdf59e640 1 --2- 172.21.15.139:0/3578205631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfe0150c50 0x7fbfe00c5cc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400e800 tx=0x7fbfd400ecc0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.860+0000 7fbfdf59e640 1 -- 172.21.15.139:0/3578205631 shutdown_connections 2024-09-16T00:08:32.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.860+0000 7fbfdf59e640 1 --2- 172.21.15.139:0/3578205631 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbfc006c2b0 0x7fbfc006e770 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.860+0000 7fbfdf59e640 1 --2- 172.21.15.139:0/3578205631 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbfe0151030 0x7fbfe00ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.860+0000 7fbfdf59e640 1 --2- 172.21.15.139:0/3578205631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbfe0150c50 0x7fbfe00c5cc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.860+0000 7fbfdf59e640 1 --2- 172.21.15.139:0/3578205631 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbfe00c8b00 0x7fbfe01f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.860+0000 7fbfdf59e640 1 -- 172.21.15.139:0/3578205631 >> 172.21.15.139:0/3578205631 conn(0x7fbfe006f3f0 msgr2=0x7fbfe006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:32.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.860+0000 7fbfdf59e640 1 -- 172.21.15.139:0/3578205631 shutdown_connections 2024-09-16T00:08:32.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.860+0000 7fbfdf59e640 1 -- 172.21.15.139:0/3578205631 wait complete. 2024-09-16T00:08:32.864 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:32.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.245351362000001, 2024-09-16T00:08:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036770370000000002, 2024-09-16T00:08:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:32.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615154+0000", 2024-09-16T00:08:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1510000000000003e-06 2024-09-16T00:08:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615163+0000", 2024-09-16T00:08:32.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4490000000000005e-06 2024-09-16T00:08:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615182+0000", 2024-09-16T00:08:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8876e-05 2024-09-16T00:08:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615197+0000", 2024-09-16T00:08:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4671000000000001e-05 2024-09-16T00:08:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615349+0000", 2024-09-16T00:08:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015258700000000001 2024-09-16T00:08:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:08:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000204716 2024-09-16T00:08:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:08:32.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017250800000000001 2024-09-16T00:08:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616919+0000", 2024-09-16T00:08:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011921799999999999 2024-09-16T00:08:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:08:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018087769999999999 2024-09-16T00:08:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:32.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618774+0000", 2024-09-16T00:08:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6075999999999997e-05 2024-09-16T00:08:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618826+0000", 2024-09-16T00:08:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2046000000000003e-05 2024-09-16T00:08:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:32.871 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.243995574000003, 2024-09-16T00:08:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023501059999999998, 2024-09-16T00:08:32.872 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616506+0000", 2024-09-16T00:08:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4100000000000001e-06 2024-09-16T00:08:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617073+0000", 2024-09-16T00:08:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056725899999999997 2024-09-16T00:08:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617099+0000", 2024-09-16T00:08:32.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5860999999999999e-05 2024-09-16T00:08:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617114+0000", 2024-09-16T00:08:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5180999999999999e-05 2024-09-16T00:08:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617236+0000", 2024-09-16T00:08:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121662 2024-09-16T00:08:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:32.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001618733 2024-09-16T00:08:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:31 2.d e22/19 2:b50e409b:::mds_snaptable:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.243101426000003, 2024-09-16T00:08:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021851240000000001, 2024-09-16T00:08:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:32.876 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:32.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617400+0000", 2024-09-16T00:08:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.28e-06 2024-09-16T00:08:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617410+0000", 2024-09-16T00:08:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0179e-05 2024-09-16T00:08:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617435+0000", 2024-09-16T00:08:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5221000000000001e-05 2024-09-16T00:08:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617444+0000", 2024-09-16T00:08:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1989999999999995e-06 2024-09-16T00:08:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617514+0000", 2024-09-16T00:08:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9104999999999997e-05 2024-09-16T00:08:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617538+0000", 2024-09-16T00:08:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4383999999999999e-05 2024-09-16T00:08:32.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619536+0000", 2024-09-16T00:08:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019982789999999999 2024-09-16T00:08:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619584+0000", 2024-09-16T00:08:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7477e-05 2024-09-16T00:08:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:32.880 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:27 2.13 e22/19 2:cd5a64a3:::100.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.242796243999997, 2024-09-16T00:08:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018775459999999999, 2024-09-16T00:08:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:32.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617705+0000", 2024-09-16T00:08:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9900000000000009e-07 2024-09-16T00:08:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617718+0000", 2024-09-16T00:08:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3453e-05 2024-09-16T00:08:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617730+0000", 2024-09-16T00:08:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2221e-05 2024-09-16T00:08:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617739+0000", 2024-09-16T00:08:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0660000000000004e-06 2024-09-16T00:08:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617806+0000", 2024-09-16T00:08:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7397999999999994e-05 2024-09-16T00:08:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617823+0000", 2024-09-16T00:08:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7569999999999999e-05 2024-09-16T00:08:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619528+0000", 2024-09-16T00:08:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017045859999999999 2024-09-16T00:08:32.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619581+0000", 2024-09-16T00:08:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3253e-05 2024-09-16T00:08:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:28 2.13 e22/19 2:c91d4a1d:::mds0_inotable:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.242448973999998, 2024-09-16T00:08:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001558051, 2024-09-16T00:08:32.885 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618052+0000", 2024-09-16T00:08:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1399999999999997e-07 2024-09-16T00:08:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618065+0000", 2024-09-16T00:08:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3495e-05 2024-09-16T00:08:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:08:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0560999999999999e-05 2024-09-16T00:08:32.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618083+0000", 2024-09-16T00:08:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:08:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618147+0000", 2024-09-16T00:08:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4525999999999999e-05 2024-09-16T00:08:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:32.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618166+0000", 2024-09-16T00:08:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9045000000000001e-05 2024-09-16T00:08:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619585+0000", 2024-09-16T00:08:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014182660000000001 2024-09-16T00:08:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619609+0000", 2024-09-16T00:08:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4468e-05 2024-09-16T00:08:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:32.889 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:29 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.242098470000002, 2024-09-16T00:08:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012347219999999999, 2024-09-16T00:08:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:32.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3099999999999997e-07 2024-09-16T00:08:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:32.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618409+0000", 2024-09-16T00:08:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6239999999999996e-06 2024-09-16T00:08:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618420+0000", 2024-09-16T00:08:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1222e-05 2024-09-16T00:08:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618441+0000", 2024-09-16T00:08:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0564000000000002e-05 2024-09-16T00:08:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618519+0000", 2024-09-16T00:08:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7917999999999995e-05 2024-09-16T00:08:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618535+0000", 2024-09-16T00:08:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6266e-05 2024-09-16T00:08:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619612+0000", 2024-09-16T00:08:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010767039999999999 2024-09-16T00:08:32.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619636+0000", 2024-09-16T00:08:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4692999999999999e-05 2024-09-16T00:08:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:2 2.f e22/19 ondisk, result = 0)", 2024-09-16T00:08:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.241862443000002, 2024-09-16T00:08:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019782099999999999, 2024-09-16T00:08:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:32.894 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618639+0000", 2024-09-16T00:08:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5099999999999998e-07 2024-09-16T00:08:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618644+0000", 2024-09-16T00:08:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7440000000000004e-06 2024-09-16T00:08:32.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618655+0000", 2024-09-16T00:08:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0798999999999999e-05 2024-09-16T00:08:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618662+0000", 2024-09-16T00:08:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8700000000000003e-06 2024-09-16T00:08:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618717+0000", 2024-09-16T00:08:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5223999999999998e-05 2024-09-16T00:08:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618724+0000", 2024-09-16T00:08:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.046e-06 2024-09-16T00:08:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:32.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618835+0000", 2024-09-16T00:08:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111187 2024-09-16T00:08:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.b epoch 22/19 rep_tid 122 entries 22'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.240758049999997, 2024-09-16T00:08:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018692890000000001, 2024-09-16T00:08:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:32.899 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:32.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619743+0000", 2024-09-16T00:08:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1069999999999999e-06 2024-09-16T00:08:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619754+0000", 2024-09-16T00:08:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0789e-05 2024-09-16T00:08:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619777+0000", 2024-09-16T00:08:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2844e-05 2024-09-16T00:08:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619793+0000", 2024-09-16T00:08:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6231000000000001e-05 2024-09-16T00:08:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619875+0000", 2024-09-16T00:08:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2286000000000005e-05 2024-09-16T00:08:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621611+0000", 2024-09-16T00:08:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001736032 2024-09-16T00:08:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:32.903 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.6 38/37 [PushOp(2:654134d2:::mds0_openfiles.0:head, version: 26'3, data_included: [], data_size: 0, omap_header_size: 42, omap_entries_size: 2, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:654134d2:::mds0_openfiles.0:head@26'3, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.839473+0000", 2024-09-16T00:08:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.0210272899999993, 2024-09-16T00:08:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044119500000000002, 2024-09-16T00:08:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:32.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839473+0000", 2024-09-16T00:08:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839473+0000", 2024-09-16T00:08:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839474+0000", 2024-09-16T00:08:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1999999999999998e-07 2024-09-16T00:08:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839512+0000", 2024-09-16T00:08:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8546e-05 2024-09-16T00:08:32.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839530+0000", 2024-09-16T00:08:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7663e-05 2024-09-16T00:08:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839550+0000", 2024-09-16T00:08:32.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9817e-05 2024-09-16T00:08:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839658+0000", 2024-09-16T00:08:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108213 2024-09-16T00:08:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839672+0000", 2024-09-16T00:08:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4518000000000001e-05 2024-09-16T00:08:32.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839914+0000", 2024-09-16T00:08:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000241618 2024-09-16T00:08:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:32.909 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:32.909 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:32.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.863+0000 7f220c677640 1 -- 172.21.15.139:0/2698950411 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f21e806c380 msgr2=0x7f21e806e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.863+0000 7f220c677640 1 --2- 172.21.15.139:0/2698950411 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f21e806c380 0x7f21e806e840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2204150ec0 tx=0x7f21f4009040 comp rx=0 tx=0).stop 2024-09-16T00:08:32.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.863+0000 7f220c677640 1 -- 172.21.15.139:0/2698950411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22040c63c0 msgr2=0x7f220414aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:32.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.863+0000 7f220c677640 1 --2- 172.21.15.139:0/2698950411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22040c63c0 0x7f220414aa00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2200009990 tx=0x7f22000046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.863+0000 7f220c677640 1 -- 172.21.15.139:0/2698950411 shutdown_connections 2024-09-16T00:08:32.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.863+0000 7f220c677640 1 --2- 172.21.15.139:0/2698950411 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f21e806c380 0x7f21e806e840 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.863+0000 7f220c677640 1 --2- 172.21.15.139:0/2698950411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2204151a30 0x7f2204158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.863+0000 7f220c677640 1 --2- 172.21.15.139:0/2698950411 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22040c67a0 0x7f2204158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.863+0000 7f220c677640 1 --2- 172.21.15.139:0/2698950411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22040c63c0 0x7f220414aa00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:32.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.863+0000 7f220c677640 1 -- 172.21.15.139:0/2698950411 >> 172.21.15.139:0/2698950411 conn(0x7f2204072b30 msgr2=0x7f22041510f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:32.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.863+0000 7f220c677640 1 -- 172.21.15.139:0/2698950411 shutdown_connections 2024-09-16T00:08:32.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:32.864+0000 7f220c677640 1 -- 172.21.15.139:0/2698950411 wait complete. 2024-09-16T00:08:32.912 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T00:08:32.913 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:08:32.955 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:32.953+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:33.016 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:33.015+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:33.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.056+0000 7f7b3959f640 1 Processor -- start 2024-09-16T00:08:33.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.056+0000 7f7b3959f640 1 -- start start 2024-09-16T00:08:33.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.057+0000 7f7b3959f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7b340737e0 0x7f7b34072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:33.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.057+0000 7f7b3959f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b34073010 con 0x7f7b340c7c40 2024-09-16T00:08:33.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.057+0000 7f7b3959f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b3415aad0 con 0x7f7b34073400 2024-09-16T00:08:33.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.057+0000 7f7b3959f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7b3415adb0 con 0x7f7b340737e0 2024-09-16T00:08:33.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.057+0000 7f7b38d9e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f7b340c7c40 0x7f7b340c8050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:50274/0 (socket says 172.21.15.139:50274) 2024-09-16T00:08:33.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.057+0000 7f7b38d9e640 1 -- 172.21.15.139:0/3127779368 learned_addr learned my addr 172.21.15.139:0/3127779368 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:33.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.057+0000 7f7b337fe640 1 --2- 172.21.15.139:0/3127779368 >> v2:172.21.15.164:3300/0 conn(0x7f7b340737e0 0x7f7b34072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:33.061 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:33.061 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:33.061 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:33.061 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:33.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.058+0000 7f7b32ffd640 1 -- 172.21.15.139:0/3127779368 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3607295157 0 0) 0x7f7b34073010 con 0x7f7b340c7c40 2024-09-16T00:08:33.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.058+0000 7f7b32ffd640 1 -- 172.21.15.139:0/3127779368 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b1c003520 con 0x7f7b340c7c40 2024-09-16T00:08:33.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.058+0000 7f7b32ffd640 1 -- 172.21.15.139:0/3127779368 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3492685785 0 0) 0x7f7b3415aad0 con 0x7f7b34073400 2024-09-16T00:08:33.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.058+0000 7f7b32ffd640 1 -- 172.21.15.139:0/3127779368 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b34073010 con 0x7f7b34073400 2024-09-16T00:08:33.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.058+0000 7f7b337fe640 1 -- 172.21.15.139:0/3127779368 >> v1:172.21.15.164:6789/0 conn(0x7f7b34073400 legacy=0x7f7b34157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:33.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.058+0000 7f7b337fe640 1 -- 172.21.15.139:0/3127779368 >> v1:172.21.15.139:6789/0 conn(0x7f7b340c7c40 legacy=0x7f7b340c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:33.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.058+0000 7f7b337fe640 1 -- 172.21.15.139:0/3127779368 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3415b090 con 0x7f7b340737e0 2024-09-16T00:08:33.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.058+0000 7f7b337fe640 1 --2- 172.21.15.139:0/3127779368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b340737e0 0x7f7b34072680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7b280097a0 tx=0x7f7b2802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8aaa556b3f21ac3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:33.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.059+0000 7f7b32ffd640 1 -- 172.21.15.139:0/3127779368 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28037070 con 0x7f7b340737e0 2024-09-16T00:08:33.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.059+0000 7f7b32ffd640 1 -- 172.21.15.139:0/3127779368 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b28032440 con 0x7f7b340737e0 2024-09-16T00:08:33.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.059+0000 7f7b32ffd640 1 -- 172.21.15.139:0/3127779368 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28032810 con 0x7f7b340737e0 2024-09-16T00:08:33.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.059+0000 7f7b3959f640 1 -- 172.21.15.139:0/3127779368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b340737e0 msgr2=0x7f7b34072680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:33.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.059+0000 7f7b3959f640 1 --2- 172.21.15.139:0/3127779368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b340737e0 0x7f7b34072680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7b280097a0 tx=0x7f7b2802a230 comp rx=0 tx=0).stop 2024-09-16T00:08:33.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.059+0000 7f7b3959f640 1 -- 172.21.15.139:0/3127779368 shutdown_connections 2024-09-16T00:08:33.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.059+0000 7f7b3959f640 1 --2- 172.21.15.139:0/3127779368 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b340737e0 0x7f7b34072680 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:33.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.059+0000 7f7b3959f640 1 -- 172.21.15.139:0/3127779368 >> 172.21.15.139:0/3127779368 conn(0x7f7b340c9a10 msgr2=0x7f7b340c9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:33.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.059+0000 7f7b3959f640 1 -- 172.21.15.139:0/3127779368 shutdown_connections 2024-09-16T00:08:33.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.059+0000 7f7b3959f640 1 -- 172.21.15.139:0/3127779368 wait complete. 2024-09-16T00:08:33.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.061+0000 7f7b3959f640 1 Processor -- start 2024-09-16T00:08:33.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.061+0000 7f7b3959f640 1 -- start start 2024-09-16T00:08:33.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b3959f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b34073400 0x7f7b34149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:33.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b3959f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b340737e0 0x7f7b34143a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:33.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b3959f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b340c7c40 0x7f7b341f0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:33.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b340737e0 0x7f7b34143a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:33.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b337fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b340737e0 0x7f7b34143a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43168/0 (socket says 172.21.15.139:43168) 2024-09-16T00:08:33.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b337fe640 1 -- 172.21.15.139:0/3867529695 learned_addr learned my addr 172.21.15.139:0/3867529695 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:33.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b3959f640 1 -- 172.21.15.139:0/3867529695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34151a00 con 0x7f7b340737e0 2024-09-16T00:08:33.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b3959f640 1 -- 172.21.15.139:0/3867529695 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b34073c10 con 0x7f7b34073400 2024-09-16T00:08:33.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b3959f640 1 -- 172.21.15.139:0/3867529695 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7b34151b80 con 0x7f7b340c7c40 2024-09-16T00:08:33.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b33fff640 1 --2- 172.21.15.139:0/3867529695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b34073400 0x7f7b34149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:33.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b38d9e640 1 --2- 172.21.15.139:0/3867529695 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b340c7c40 0x7f7b341f0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:33.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b337fe640 1 -- 172.21.15.139:0/3867529695 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b340c7c40 msgr2=0x7f7b341f0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:33.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b337fe640 1 --2- 172.21.15.139:0/3867529695 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b340c7c40 0x7f7b341f0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:33.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b337fe640 1 -- 172.21.15.139:0/3867529695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b34073400 msgr2=0x7f7b34149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:33.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b337fe640 1 --2- 172.21.15.139:0/3867529695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b34073400 0x7f7b34149e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:33.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b337fe640 1 -- 172.21.15.139:0/3867529695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b34154b80 con 0x7f7b340737e0 2024-09-16T00:08:33.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b38d9e640 1 --2- 172.21.15.139:0/3867529695 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b340c7c40 0x7f7b341f0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:08:33.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.063+0000 7f7b337fe640 1 --2- 172.21.15.139:0/3867529695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b340737e0 0x7f7b34143a00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7b28035040 tx=0x7f7b280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:33.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.064+0000 7f7b30ff9640 1 -- 172.21.15.139:0/3867529695 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28041030 con 0x7f7b340737e0 2024-09-16T00:08:33.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.064+0000 7f7b30ff9640 1 -- 172.21.15.139:0/3867529695 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b2803d4e0 con 0x7f7b340737e0 2024-09-16T00:08:33.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.064+0000 7f7b30ff9640 1 -- 172.21.15.139:0/3867529695 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28037040 con 0x7f7b340737e0 2024-09-16T00:08:33.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.064+0000 7f7b3959f640 1 -- 172.21.15.139:0/3867529695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b34151eb0 con 0x7f7b340737e0 2024-09-16T00:08:33.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.064+0000 7f7b3959f640 1 -- 172.21.15.139:0/3867529695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b341434e0 con 0x7f7b340737e0 2024-09-16T00:08:33.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.065+0000 7f7b3959f640 1 -- 172.21.15.139:0/3867529695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b34157e40 con 0x7f7b340737e0 2024-09-16T00:08:33.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.066+0000 7f7b30ff9640 1 -- 172.21.15.139:0/3867529695 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7b28044070 con 0x7f7b340737e0 2024-09-16T00:08:33.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.067+0000 7f7b30ff9640 1 --2- 172.21.15.139:0/3867529695 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b2006c2b0 0x7f7b2006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:33.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.068+0000 7f7b30ff9640 1 -- 172.21.15.139:0/3867529695 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5915+0+0 (secure 0 0 0) 0x7f7b28076fb0 con 0x7f7b340737e0 2024-09-16T00:08:33.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.068+0000 7f7b33fff640 1 --2- 172.21.15.139:0/3867529695 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b2006c2b0 0x7f7b2006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:33.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.068+0000 7f7b33fff640 1 --2- 172.21.15.139:0/3867529695 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b2006c2b0 0x7f7b2006e770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7b24007fd0 tx=0x7f7b24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:33.071 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:08:33.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.075+0000 7f7b30ff9640 1 -- 172.21.15.139:0/3867529695 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7b28079610 con 0x7f7b340737e0 2024-09-16T00:08:33.154 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:33.153+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:33.215 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:33.216 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:33.216 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:33.216 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:33.216 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:33.218 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:33.217+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:33.224 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:08:33.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.259+0000 7f7b3959f640 1 -- 172.21.15.139:0/3867529695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) -- 0x7f7b340731e0 con 0x7f7b340737e0 2024-09-16T00:08:33.319 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:33.318+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:33.370 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.370 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:14 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.753627252000001, 2024-09-16T00:08:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029190470000000001, 2024-09-16T00:08:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615982+0000", 2024-09-16T00:08:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7439999999999999e-06 2024-09-16T00:08:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:08:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3743999999999999e-05 2024-09-16T00:08:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2255e-05 2024-09-16T00:08:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616029+0000", 2024-09-16T00:08:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1175999999999999e-05 2024-09-16T00:08:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616098+0000", 2024-09-16T00:08:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9153999999999996e-05 2024-09-16T00:08:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616132+0000", 2024-09-16T00:08:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3924000000000002e-05 2024-09-16T00:08:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:33.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027226189999999999 2024-09-16T00:08:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618899+0000", 2024-09-16T00:08:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4431000000000002e-05 2024-09-16T00:08:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:15 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.751400431999997, 2024-09-16T00:08:33.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095618099999999996, 2024-09-16T00:08:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:33.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618208+0000", 2024-09-16T00:08:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1850000000000001e-06 2024-09-16T00:08:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618217+0000", 2024-09-16T00:08:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439999999999992e-06 2024-09-16T00:08:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618230+0000", 2024-09-16T00:08:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2435999999999999e-05 2024-09-16T00:08:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618238+0000", 2024-09-16T00:08:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1319999999999994e-06 2024-09-16T00:08:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618331+0000", 2024-09-16T00:08:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3453000000000001e-05 2024-09-16T00:08:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618352+0000", 2024-09-16T00:08:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0693999999999999e-05 2024-09-16T00:08:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619112+0000", 2024-09-16T00:08:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00076007099999999997 2024-09-16T00:08:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619163+0000", 2024-09-16T00:08:33.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1066000000000002e-05 2024-09-16T00:08:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:30 2.15 e22/19 2:aa448500:::500.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.751342946999998, 2024-09-16T00:08:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058031250000000001, 2024-09-16T00:08:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.380 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618265+0000", 2024-09-16T00:08:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9200000000000001e-07 2024-09-16T00:08:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618271+0000", 2024-09-16T00:08:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9580000000000004e-06 2024-09-16T00:08:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618279+0000", 2024-09-16T00:08:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2239999999999995e-06 2024-09-16T00:08:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618285+0000", 2024-09-16T00:08:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-06 2024-09-16T00:08:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618567+0000", 2024-09-16T00:08:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028204400000000001 2024-09-16T00:08:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:08:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9678999999999999e-05 2024-09-16T00:08:33.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624024+0000", 2024-09-16T00:08:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054376720000000002 2024-09-16T00:08:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624068+0000", 2024-09-16T00:08:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:08:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:20 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.750986583, 2024-09-16T00:08:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028808739999999998, 2024-09-16T00:08:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:33.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618622+0000", 2024-09-16T00:08:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1260000000000001e-06 2024-09-16T00:08:33.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618631+0000", 2024-09-16T00:08:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4700000000000008e-06 2024-09-16T00:08:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618645+0000", 2024-09-16T00:08:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3984e-05 2024-09-16T00:08:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:08:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2101e-05 2024-09-16T00:08:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618720+0000", 2024-09-16T00:08:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2761000000000004e-05 2024-09-16T00:08:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618740+0000", 2024-09-16T00:08:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9409e-05 2024-09-16T00:08:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621436+0000", 2024-09-16T00:08:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002696211 2024-09-16T00:08:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621502+0000", 2024-09-16T00:08:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5811999999999999e-05 2024-09-16T00:08:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:21 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.750163295, 2024-09-16T00:08:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046318540000000004, 2024-09-16T00:08:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619445+0000", 2024-09-16T00:08:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.083e-06 2024-09-16T00:08:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619455+0000", 2024-09-16T00:08:33.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5920000000000007e-06 2024-09-16T00:08:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619467+0000", 2024-09-16T00:08:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2711000000000001e-05 2024-09-16T00:08:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619476+0000", 2024-09-16T00:08:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0139999999999998e-06 2024-09-16T00:08:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619529+0000", 2024-09-16T00:08:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3078999999999998e-05 2024-09-16T00:08:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619548+0000", 2024-09-16T00:08:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8952000000000001e-05 2024-09-16T00:08:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624037+0000", 2024-09-16T00:08:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044895730000000002 2024-09-16T00:08:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624076+0000", 2024-09-16T00:08:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8850000000000002e-05 2024-09-16T00:08:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.14 38/37 [PushOp(2:2f55791f:::606.00000000:head, version: 22'2, data_included: [], data_size: 0, omap_header_size: 274, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(2:2f55791f:::606.00000000:head@22'2, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.838548+0000", 2024-09-16T00:08:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5310593969999999, 2024-09-16T00:08:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044033700000000001, 2024-09-16T00:08:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.394 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838548+0000", 2024-09-16T00:08:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838548+0000", 2024-09-16T00:08:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838549+0000", 2024-09-16T00:08:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5600000000000004e-07 2024-09-16T00:08:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838567+0000", 2024-09-16T00:08:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7850999999999999e-05 2024-09-16T00:08:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838597+0000", 2024-09-16T00:08:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0416000000000002e-05 2024-09-16T00:08:33.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838619+0000", 2024-09-16T00:08:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1274e-05 2024-09-16T00:08:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838706+0000", 2024-09-16T00:08:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7755000000000003e-05 2024-09-16T00:08:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838721+0000", 2024-09-16T00:08:33.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4953999999999999e-05 2024-09-16T00:08:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838988+0000", 2024-09-16T00:08:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026713099999999999 2024-09-16T00:08:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.14 38/37 [PushOp(2:287034c4:::1.00000000.inode:head, version: 22'4, data_included: [0~582], data_size: 582, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:287034c4:::1.00000000.inode:head@22'4, size: 582, copy_subset: [(0, 582)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 582, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.840824+0000", 2024-09-16T00:08:33.398 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5287835570000006, 2024-09-16T00:08:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00055177600000000002, 2024-09-16T00:08:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840824+0000", 2024-09-16T00:08:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840824+0000", 2024-09-16T00:08:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840825+0000", 2024-09-16T00:08:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1100000000000005e-07 2024-09-16T00:08:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840840+0000", 2024-09-16T00:08:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5308999999999999e-05 2024-09-16T00:08:33.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840863+0000", 2024-09-16T00:08:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2663999999999998e-05 2024-09-16T00:08:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840884+0000", 2024-09-16T00:08:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1798e-05 2024-09-16T00:08:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840980+0000", 2024-09-16T00:08:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5299999999999999e-05 2024-09-16T00:08:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840995+0000", 2024-09-16T00:08:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4895999999999999e-05 2024-09-16T00:08:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.841376+0000", 2024-09-16T00:08:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038099800000000001 2024-09-16T00:08:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.402 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.14 38/37 [PushOp(2:2e2fa760:::200.00000001:head, version: 28'9, data_included: [0~5883], data_size: 5883, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:2e2fa760:::200.00000001:head@28'9, size: 5883, copy_subset: [(0, 5883)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5883, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.848988+0000", 2024-09-16T00:08:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.520619237, 2024-09-16T00:08:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00037334, 2024-09-16T00:08:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.848988+0000", 2024-09-16T00:08:33.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.848988+0000", 2024-09-16T00:08:33.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.848989+0000", 2024-09-16T00:08:33.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5099999999999998e-07 2024-09-16T00:08:33.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849004+0000", 2024-09-16T00:08:33.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5192e-05 2024-09-16T00:08:33.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849023+0000", 2024-09-16T00:08:33.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.844e-05 2024-09-16T00:08:33.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849033+0000", 2024-09-16T00:08:33.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0061000000000001e-05 2024-09-16T00:08:33.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849096+0000", 2024-09-16T00:08:33.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3627999999999995e-05 2024-09-16T00:08:33.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849105+0000", 2024-09-16T00:08:33.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5890000000000003e-06 2024-09-16T00:08:33.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849362+0000", 2024-09-16T00:08:33.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000256579 2024-09-16T00:08:33.407 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.407 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.407 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.407 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.14 38/37 [PushOp(2:292cf221:::200.00000000:head, version: 28'10, data_included: [0~90], data_size: 90, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:292cf221:::200.00000000:head@28'10, size: 90, copy_subset: [(0, 90)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 90, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:33.407 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.851655+0000", 2024-09-16T00:08:33.407 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5179525720000004, 2024-09-16T00:08:33.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036283000000000002, 2024-09-16T00:08:33.407 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.407 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.407 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:33.407 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851655+0000", 2024-09-16T00:08:33.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851655+0000", 2024-09-16T00:08:33.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851656+0000", 2024-09-16T00:08:33.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0399999999999995e-07 2024-09-16T00:08:33.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851672+0000", 2024-09-16T00:08:33.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6801000000000001e-05 2024-09-16T00:08:33.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851689+0000", 2024-09-16T00:08:33.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6810999999999999e-05 2024-09-16T00:08:33.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851699+0000", 2024-09-16T00:08:33.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.995e-06 2024-09-16T00:08:33.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851767+0000", 2024-09-16T00:08:33.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8194000000000005e-05 2024-09-16T00:08:33.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851777+0000", 2024-09-16T00:08:33.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2450000000000004e-06 2024-09-16T00:08:33.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.852018+0000", 2024-09-16T00:08:33.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024107999999999999 2024-09-16T00:08:33.411 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.411 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.411 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.411 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.411 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.411 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:33.412 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:08:33.453 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:33.452+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:33.521 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:33.519+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:33.558 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:33.558 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:33.558 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:33.559 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:33.567 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:08:33.651 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:33.650+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:33.711 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:33.711 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:33.711 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:33.711 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:33.711 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:33.720 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:08:33.722 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:33.721+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:33.823 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:33.822+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:33.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.862+0000 7f7b30ff9640 1 -- 172.21.15.139:0/3867529695 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v44) ==== 113+0+0 (secure 0 0 0) 0x7f7b28078f10 con 0x7f7b340737e0 2024-09-16T00:08:33.863 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T00:08:33.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.865+0000 7f7b3959f640 1 -- 172.21.15.139:0/3867529695 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b2006c2b0 msgr2=0x7f7b2006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:33.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.865+0000 7f7b3959f640 1 --2- 172.21.15.139:0/3867529695 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b2006c2b0 0x7f7b2006e770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7b24007fd0 tx=0x7f7b24009040 comp rx=0 tx=0).stop 2024-09-16T00:08:33.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.866+0000 7f7b3959f640 1 -- 172.21.15.139:0/3867529695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b340737e0 msgr2=0x7f7b34143a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:33.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.866+0000 7f7b3959f640 1 --2- 172.21.15.139:0/3867529695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b340737e0 0x7f7b34143a00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7b28035040 tx=0x7f7b280046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:33.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.866+0000 7f7b3959f640 1 -- 172.21.15.139:0/3867529695 shutdown_connections 2024-09-16T00:08:33.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.866+0000 7f7b3959f640 1 --2- 172.21.15.139:0/3867529695 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7b2006c2b0 0x7f7b2006e770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:33.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.866+0000 7f7b3959f640 1 --2- 172.21.15.139:0/3867529695 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b340c7c40 0x7f7b341f0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:33.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.866+0000 7f7b3959f640 1 --2- 172.21.15.139:0/3867529695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b340737e0 0x7f7b34143a00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:33.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.866+0000 7f7b3959f640 1 --2- 172.21.15.139:0/3867529695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b34073400 0x7f7b34149e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:33.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.866+0000 7f7b3959f640 1 -- 172.21.15.139:0/3867529695 >> 172.21.15.139:0/3867529695 conn(0x7f7b340c9a10 msgr2=0x7f7b3406e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:33.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.866+0000 7f7b3959f640 1 -- 172.21.15.139:0/3867529695 shutdown_connections 2024-09-16T00:08:33.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:33.866+0000 7f7b3959f640 1 -- 172.21.15.139:0/3867529695 wait complete. 2024-09-16T00:08:33.869 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:33.869 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:33.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:33.869 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:33.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.869 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:33.869 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:33.869 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.251377605999998, 2024-09-16T00:08:33.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056100969999999997, 2024-09-16T00:08:33.870 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.870 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.870 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.870 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:33.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:33.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615352+0000", 2024-09-16T00:08:33.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9859999999999999e-06 2024-09-16T00:08:33.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615363+0000", 2024-09-16T00:08:33.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1168999999999999e-05 2024-09-16T00:08:33.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615382+0000", 2024-09-16T00:08:33.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8224e-05 2024-09-16T00:08:33.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615393+0000", 2024-09-16T00:08:33.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0925e-05 2024-09-16T00:08:33.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615624+0000", 2024-09-16T00:08:33.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023159600000000001 2024-09-16T00:08:33.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615721+0000", 2024-09-16T00:08:33.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7040000000000006e-05 2024-09-16T00:08:33.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:33.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615885+0000", 2024-09-16T00:08:33.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163456 2024-09-16T00:08:33.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:33.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:08:33.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002191368 2024-09-16T00:08:33.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:33.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620870+0000", 2024-09-16T00:08:33.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941839999999999 2024-09-16T00:08:33.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620922+0000", 2024-09-16T00:08:33.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1641999999999998e-05 2024-09-16T00:08:33.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620956+0000", 2024-09-16T00:08:33.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4507e-05 2024-09-16T00:08:33.876 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.876 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.876 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.876 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:33.876 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:33.876 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.251287247, 2024-09-16T00:08:33.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005695739, 2024-09-16T00:08:33.877 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.877 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.877 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.877 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:33.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:33.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:33.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4900000000000005e-07 2024-09-16T00:08:33.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615442+0000", 2024-09-16T00:08:33.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.994e-06 2024-09-16T00:08:33.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:08:33.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1081e-05 2024-09-16T00:08:33.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615473+0000", 2024-09-16T00:08:33.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1910000000000006e-06 2024-09-16T00:08:33.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616230+0000", 2024-09-16T00:08:33.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075700400000000003 2024-09-16T00:08:33.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616305+0000", 2024-09-16T00:08:33.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4759999999999996e-05 2024-09-16T00:08:33.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:33.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616466+0000", 2024-09-16T00:08:33.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000161714 2024-09-16T00:08:33.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:33.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619821+0000", 2024-09-16T00:08:33.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033547339999999998 2024-09-16T00:08:33.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:33.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621028+0000", 2024-09-16T00:08:33.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012067530000000001 2024-09-16T00:08:33.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621087+0000", 2024-09-16T00:08:33.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9364999999999998e-05 2024-09-16T00:08:33.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621132+0000", 2024-09-16T00:08:33.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5293999999999997e-05 2024-09-16T00:08:33.882 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.882 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.882 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.883 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:33.883 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:33.883 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.251225015000003, 2024-09-16T00:08:33.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050565760000000001, 2024-09-16T00:08:33.883 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.883 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.883 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.883 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:33.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:33.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615500+0000", 2024-09-16T00:08:33.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8199999999999999e-07 2024-09-16T00:08:33.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615505+0000", 2024-09-16T00:08:33.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5280000000000003e-06 2024-09-16T00:08:33.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:33.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0317e-05 2024-09-16T00:08:33.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615523+0000", 2024-09-16T00:08:33.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4580000000000004e-06 2024-09-16T00:08:33.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615607+0000", 2024-09-16T00:08:33.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4153999999999994e-05 2024-09-16T00:08:33.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615748+0000", 2024-09-16T00:08:33.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140533 2024-09-16T00:08:33.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:33.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615911+0000", 2024-09-16T00:08:33.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163678 2024-09-16T00:08:33.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:33.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619944+0000", 2024-09-16T00:08:33.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040330909999999999 2024-09-16T00:08:33.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:33.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620471+0000", 2024-09-16T00:08:33.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052645600000000002 2024-09-16T00:08:33.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620512+0000", 2024-09-16T00:08:33.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0738000000000002e-05 2024-09-16T00:08:33.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620555+0000", 2024-09-16T00:08:33.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3841000000000003e-05 2024-09-16T00:08:33.888 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.889 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.889 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.889 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:33.889 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:33.889 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.251172908000001, 2024-09-16T00:08:33.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089162479999999999, 2024-09-16T00:08:33.889 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.889 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.889 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.889 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:33.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:33.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615552+0000", 2024-09-16T00:08:33.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:08:33.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615556+0000", 2024-09-16T00:08:33.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3610000000000003e-06 2024-09-16T00:08:33.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615566+0000", 2024-09-16T00:08:33.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6099999999999995e-06 2024-09-16T00:08:33.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615574+0000", 2024-09-16T00:08:33.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9379999999999992e-06 2024-09-16T00:08:33.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616777+0000", 2024-09-16T00:08:33.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012031640000000001 2024-09-16T00:08:33.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616882+0000", 2024-09-16T00:08:33.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105146 2024-09-16T00:08:33.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:33.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619696+0000", 2024-09-16T00:08:33.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028142179999999998 2024-09-16T00:08:33.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:33.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624234+0000", 2024-09-16T00:08:33.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045381889999999998 2024-09-16T00:08:33.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:33.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624393+0000", 2024-09-16T00:08:33.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015871200000000001 2024-09-16T00:08:33.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624434+0000", 2024-09-16T00:08:33.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0908999999999997e-05 2024-09-16T00:08:33.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624467+0000", 2024-09-16T00:08:33.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3217999999999999e-05 2024-09-16T00:08:33.894 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.894 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.894 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.894 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:33.894 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:33.895 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.251103110999999, 2024-09-16T00:08:33.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064898359999999997, 2024-09-16T00:08:33.895 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.895 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.895 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.895 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:33.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:33.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615622+0000", 2024-09-16T00:08:33.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.181e-06 2024-09-16T00:08:33.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615628+0000", 2024-09-16T00:08:33.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2450000000000003e-06 2024-09-16T00:08:33.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:08:33.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2554999999999999e-05 2024-09-16T00:08:33.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615650+0000", 2024-09-16T00:08:33.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4350000000000003e-06 2024-09-16T00:08:33.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616772+0000", 2024-09-16T00:08:33.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001122058 2024-09-16T00:08:33.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616889+0000", 2024-09-16T00:08:33.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116623 2024-09-16T00:08:33.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:33.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617035+0000", 2024-09-16T00:08:33.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014579799999999999 2024-09-16T00:08:33.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:33.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619829+0000", 2024-09-16T00:08:33.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941379999999998 2024-09-16T00:08:33.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:33.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622024+0000", 2024-09-16T00:08:33.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021949920000000002 2024-09-16T00:08:33.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622087+0000", 2024-09-16T00:08:33.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3048999999999999e-05 2024-09-16T00:08:33.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622111+0000", 2024-09-16T00:08:33.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3762000000000001e-05 2024-09-16T00:08:33.900 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.900 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.900 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.900 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:33.900 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:33.900 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.251040549999999, 2024-09-16T00:08:33.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065192160000000004, 2024-09-16T00:08:33.900 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.900 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.901 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.901 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:33.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:33.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615684+0000", 2024-09-16T00:08:33.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8299999999999995e-07 2024-09-16T00:08:33.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615691+0000", 2024-09-16T00:08:33.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.866e-06 2024-09-16T00:08:33.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615707+0000", 2024-09-16T00:08:33.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6073999999999999e-05 2024-09-16T00:08:33.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615714+0000", 2024-09-16T00:08:33.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-16T00:08:33.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617403+0000", 2024-09-16T00:08:33.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016881629999999999 2024-09-16T00:08:33.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617559+0000", 2024-09-16T00:08:33.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015669699999999999 2024-09-16T00:08:33.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:33.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:33.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000144165 2024-09-16T00:08:33.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:33.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621413+0000", 2024-09-16T00:08:33.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037092200000000001 2024-09-16T00:08:33.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:33.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622147+0000", 2024-09-16T00:08:33.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073468800000000005 2024-09-16T00:08:33.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622180+0000", 2024-09-16T00:08:33.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2901999999999998e-05 2024-09-16T00:08:33.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622203+0000", 2024-09-16T00:08:33.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2211000000000001e-05 2024-09-16T00:08:33.905 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.905 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.906 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.906 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:33.906 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:33.906 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.250997456, 2024-09-16T00:08:33.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065825270000000003, 2024-09-16T00:08:33.906 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.906 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.906 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:33.906 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:33.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:33.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:08:33.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7299999999999996e-07 2024-09-16T00:08:33.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615730+0000", 2024-09-16T00:08:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.004e-06 2024-09-16T00:08:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615737+0000", 2024-09-16T00:08:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4789999999999999e-06 2024-09-16T00:08:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615745+0000", 2024-09-16T00:08:33.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6699999999999994e-06 2024-09-16T00:08:33.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618115+0000", 2024-09-16T00:08:33.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023701569999999999 2024-09-16T00:08:33.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618247+0000", 2024-09-16T00:08:33.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013185000000000001 2024-09-16T00:08:33.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622309+0000", 2024-09-16T00:08:33.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040617939999999996 2024-09-16T00:08:33.910 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.910 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.910 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.910 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:33.910 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:33.911 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.250970207000002, 2024-09-16T00:08:33.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036685950000000002, 2024-09-16T00:08:33.911 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.911 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.911 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:33.911 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:33.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:33.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:33.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T00:08:33.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615763+0000", 2024-09-16T00:08:33.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4360000000000002e-06 2024-09-16T00:08:33.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615770+0000", 2024-09-16T00:08:33.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7540000000000006e-06 2024-09-16T00:08:33.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615803+0000", 2024-09-16T00:08:33.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3008e-05 2024-09-16T00:08:33.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615888+0000", 2024-09-16T00:08:33.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5056999999999996e-05 2024-09-16T00:08:33.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:08:33.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013059700000000001 2024-09-16T00:08:33.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619422+0000", 2024-09-16T00:08:33.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034033230000000002 2024-09-16T00:08:33.915 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.915 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.915 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.915 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing_reply(2.1a epoch 22/19 rep_tid 2 lcod 22'1)", 2024-09-16T00:08:33.915 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:33.915 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.247528893999998, 2024-09-16T00:08:33.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024304200000000001, 2024-09-16T00:08:33.916 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.916 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.916 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:33.916 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:33.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:33.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619196+0000", 2024-09-16T00:08:33.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2759999999999999e-06 2024-09-16T00:08:33.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619203+0000", 2024-09-16T00:08:33.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1779999999999997e-06 2024-09-16T00:08:33.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619223+0000", 2024-09-16T00:08:33.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9159000000000001e-05 2024-09-16T00:08:33.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619231+0000", 2024-09-16T00:08:33.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7830000000000004e-06 2024-09-16T00:08:33.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619320+0000", 2024-09-16T00:08:33.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8805000000000001e-05 2024-09-16T00:08:33.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619438+0000", 2024-09-16T00:08:33.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117841 2024-09-16T00:08:33.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.920 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.921 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:16 2.1d e22/19 2:b871830b:::605.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:33.921 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:33.921 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.246981617000003, 2024-09-16T00:08:33.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045616900000000002, 2024-09-16T00:08:33.921 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.921 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.921 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.921 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:33.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:33.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:08:33.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:08:33.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619755+0000", 2024-09-16T00:08:33.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3599999999999996e-06 2024-09-16T00:08:33.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619767+0000", 2024-09-16T00:08:33.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2805e-05 2024-09-16T00:08:33.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619787+0000", 2024-09-16T00:08:33.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9264000000000001e-05 2024-09-16T00:08:33.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619869+0000", 2024-09-16T00:08:33.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2639999999999995e-05 2024-09-16T00:08:33.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619895+0000", 2024-09-16T00:08:33.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5890000000000001e-05 2024-09-16T00:08:33.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624264+0000", 2024-09-16T00:08:33.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043690049999999996 2024-09-16T00:08:33.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624304+0000", 2024-09-16T00:08:33.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9807e-05 2024-09-16T00:08:33.925 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.925 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.925 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.925 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:17 2.1d e22/19 2:b871830b:::605.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:33.925 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:33.925 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.246628907999998, 2024-09-16T00:08:33.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042370910000000001, 2024-09-16T00:08:33.925 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.925 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.926 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.926 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:33.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:33.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620096+0000", 2024-09-16T00:08:33.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:08:33.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620102+0000", 2024-09-16T00:08:33.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7220000000000002e-06 2024-09-16T00:08:33.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620112+0000", 2024-09-16T00:08:33.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0156999999999999e-05 2024-09-16T00:08:33.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620120+0000", 2024-09-16T00:08:33.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2880000000000004e-06 2024-09-16T00:08:33.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620171+0000", 2024-09-16T00:08:33.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.168e-05 2024-09-16T00:08:33.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620189+0000", 2024-09-16T00:08:33.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7419e-05 2024-09-16T00:08:33.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624306+0000", 2024-09-16T00:08:33.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041175430000000004 2024-09-16T00:08:33.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624332+0000", 2024-09-16T00:08:33.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5717000000000001e-05 2024-09-16T00:08:33.929 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.929 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.929 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.930 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:24 2.7 e22/19 ondisk, result = 0)", 2024-09-16T00:08:33.930 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:33.930 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.246363201000001, 2024-09-16T00:08:33.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000208018, 2024-09-16T00:08:33.930 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.930 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.930 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:33.930 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:33.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:33.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:33.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8500000000000001e-07 2024-09-16T00:08:33.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620366+0000", 2024-09-16T00:08:33.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5379999999999996e-06 2024-09-16T00:08:33.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620378+0000", 2024-09-16T00:08:33.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.204e-05 2024-09-16T00:08:33.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620385+0000", 2024-09-16T00:08:33.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.413e-06 2024-09-16T00:08:33.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620455+0000", 2024-09-16T00:08:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9886000000000001e-05 2024-09-16T00:08:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620463+0000", 2024-09-16T00:08:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1559999999999995e-06 2024-09-16T00:08:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620569+0000", 2024-09-16T00:08:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001054 2024-09-16T00:08:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.933 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.934 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.934 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:18 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:33.934 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:33.934 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.245977736999997, 2024-09-16T00:08:33.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021768100000000001, 2024-09-16T00:08:33.934 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.934 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.934 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:33.934 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:33.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:33.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620747+0000", 2024-09-16T00:08:33.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.02e-07 2024-09-16T00:08:33.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620752+0000", 2024-09-16T00:08:33.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4439999999999999e-06 2024-09-16T00:08:33.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620765+0000", 2024-09-16T00:08:33.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2714000000000001e-05 2024-09-16T00:08:33.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:08:33.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1160000000000004e-06 2024-09-16T00:08:33.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:33.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3374999999999994e-05 2024-09-16T00:08:33.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620864+0000", 2024-09-16T00:08:33.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4320000000000005e-06 2024-09-16T00:08:33.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620964+0000", 2024-09-16T00:08:33.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9698000000000007e-05 2024-09-16T00:08:33.938 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.938 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.938 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.938 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:19 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:33.938 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:33.938 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.245916192999999, 2024-09-16T00:08:33.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00033551400000000003, 2024-09-16T00:08:33.938 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.938 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.938 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:33.938 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:33.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:33.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:33.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0800000000000005e-07 2024-09-16T00:08:33.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620812+0000", 2024-09-16T00:08:33.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.427e-06 2024-09-16T00:08:33.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620820+0000", 2024-09-16T00:08:33.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6649999999999992e-06 2024-09-16T00:08:33.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620828+0000", 2024-09-16T00:08:33.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0010000000000001e-06 2024-09-16T00:08:33.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620995+0000", 2024-09-16T00:08:33.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016709400000000001 2024-09-16T00:08:33.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621005+0000", 2024-09-16T00:08:33.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7170000000000003e-06 2024-09-16T00:08:33.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621143+0000", 2024-09-16T00:08:33.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000138102 2024-09-16T00:08:33.942 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.942 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.942 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.942 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:3 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:33.942 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:33.942 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.244738922000003, 2024-09-16T00:08:33.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022247799999999999, 2024-09-16T00:08:33.942 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.942 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.942 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:33.943 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:33.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:33.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:33.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3300000000000003e-07 2024-09-16T00:08:33.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621989+0000", 2024-09-16T00:08:33.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2969999999999999e-06 2024-09-16T00:08:33.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621997+0000", 2024-09-16T00:08:33.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8990000000000001e-06 2024-09-16T00:08:33.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622002+0000", 2024-09-16T00:08:33.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5439999999999998e-06 2024-09-16T00:08:33.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622137+0000", 2024-09-16T00:08:33.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000135103 2024-09-16T00:08:33.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622143+0000", 2024-09-16T00:08:33.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9200000000000001e-06 2024-09-16T00:08:33.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622207+0000", 2024-09-16T00:08:33.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4282000000000006e-05 2024-09-16T00:08:33.946 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.946 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.947 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.947 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing_reply(2.14 epoch 22/19 rep_tid 6 lcod 22'5)", 2024-09-16T00:08:33.947 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:33.947 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.244707585999997, 2024-09-16T00:08:33.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029688000000000002, 2024-09-16T00:08:33.947 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.947 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.947 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:33.947 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:33.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:33.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622017+0000", 2024-09-16T00:08:33.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3200000000000001e-07 2024-09-16T00:08:33.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:08:33.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6240000000000002e-06 2024-09-16T00:08:33.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622026+0000", 2024-09-16T00:08:33.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.708e-06 2024-09-16T00:08:33.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622032+0000", 2024-09-16T00:08:33.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4699999999999999e-06 2024-09-16T00:08:33.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622230+0000", 2024-09-16T00:08:33.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019783800000000001 2024-09-16T00:08:33.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622313+0000", 2024-09-16T00:08:33.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2908e-05 2024-09-16T00:08:33.951 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.951 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.951 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.952 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:33.952 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:33.952 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.244262091000003, 2024-09-16T00:08:33.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044651079999999998, 2024-09-16T00:08:33.952 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.952 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.952 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.953 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:33.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:33.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622463+0000", 2024-09-16T00:08:33.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5349999999999999e-06 2024-09-16T00:08:33.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622470+0000", 2024-09-16T00:08:33.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7050000000000002e-06 2024-09-16T00:08:33.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622485+0000", 2024-09-16T00:08:33.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4978e-05 2024-09-16T00:08:33.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622493+0000", 2024-09-16T00:08:33.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2139999999999996e-06 2024-09-16T00:08:33.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622546+0000", 2024-09-16T00:08:33.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:08:33.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622595+0000", 2024-09-16T00:08:33.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9112e-05 2024-09-16T00:08:33.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:33.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622677+0000", 2024-09-16T00:08:33.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2332999999999999e-05 2024-09-16T00:08:33.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:33.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.623515+0000", 2024-09-16T00:08:33.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000837615 2024-09-16T00:08:33.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:33.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626867+0000", 2024-09-16T00:08:33.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033524819999999999 2024-09-16T00:08:33.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626902+0000", 2024-09-16T00:08:33.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4310000000000002e-05 2024-09-16T00:08:33.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626927+0000", 2024-09-16T00:08:33.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5128999999999999e-05 2024-09-16T00:08:33.958 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.958 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.958 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.959 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:39 2.6 e22/19 2:654134d2:::mds0_openfiles.0:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:33.959 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:33.959 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.225632814000001, 2024-09-16T00:08:33.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082120900000000004, 2024-09-16T00:08:33.959 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.959 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.959 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.959 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:33.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:33.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641092+0000", 2024-09-16T00:08:33.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.141e-06 2024-09-16T00:08:33.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641102+0000", 2024-09-16T00:08:33.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5990000000000008e-06 2024-09-16T00:08:33.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641116+0000", 2024-09-16T00:08:33.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4667999999999999e-05 2024-09-16T00:08:33.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641125+0000", 2024-09-16T00:08:33.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0849999999999997e-06 2024-09-16T00:08:33.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641188+0000", 2024-09-16T00:08:33.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3119000000000003e-05 2024-09-16T00:08:33.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641209+0000", 2024-09-16T00:08:33.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0897e-05 2024-09-16T00:08:33.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641874+0000", 2024-09-16T00:08:33.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00066517200000000001 2024-09-16T00:08:33.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641912+0000", 2024-09-16T00:08:33.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8528000000000001e-05 2024-09-16T00:08:33.963 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.963 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.963 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.963 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:40 2.14 2:2e2fa760:::200.00000001:head [write 940~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:33.964 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:33.964 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.027396697999997, 2024-09-16T00:08:33.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016643039999999999, 2024-09-16T00:08:33.964 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.964 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.964 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.964 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:33.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:33.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839329+0000", 2024-09-16T00:08:33.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2330000000000001e-06 2024-09-16T00:08:33.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839343+0000", 2024-09-16T00:08:33.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3634e-05 2024-09-16T00:08:33.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839362+0000", 2024-09-16T00:08:33.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9321999999999999e-05 2024-09-16T00:08:33.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839375+0000", 2024-09-16T00:08:33.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2945000000000001e-05 2024-09-16T00:08:33.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839462+0000", 2024-09-16T00:08:33.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6371000000000004e-05 2024-09-16T00:08:33.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839547+0000", 2024-09-16T00:08:33.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4988999999999996e-05 2024-09-16T00:08:33.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:33.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839657+0000", 2024-09-16T00:08:33.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110501 2024-09-16T00:08:33.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:33.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840598+0000", 2024-09-16T00:08:33.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00094087100000000003 2024-09-16T00:08:33.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:33.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840918+0000", 2024-09-16T00:08:33.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031968799999999999 2024-09-16T00:08:33.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840955+0000", 2024-09-16T00:08:33.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7360999999999997e-05 2024-09-16T00:08:33.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840991+0000", 2024-09-16T00:08:33.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6389e-05 2024-09-16T00:08:33.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.969 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.969 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:41 2.14 2:2e2fa760:::200.00000001:head [write 1383~2831 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:33.969 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:33.969 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 40.753743253000003, 2024-09-16T00:08:33.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017075809999999999, 2024-09-16T00:08:33.969 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:33.969 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:33.969 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:33.969 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:33.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:33.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:33.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:33.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:33.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:33.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:33.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112983+0000", 2024-09-16T00:08:33.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2560000000000001e-06 2024-09-16T00:08:33.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:33.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112993+0000", 2024-09-16T00:08:33.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0423e-05 2024-09-16T00:08:33.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:33.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113010+0000", 2024-09-16T00:08:33.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7169000000000001e-05 2024-09-16T00:08:33.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:33.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113023+0000", 2024-09-16T00:08:33.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2111e-05 2024-09-16T00:08:33.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:33.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113099+0000", 2024-09-16T00:08:33.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6798000000000006e-05 2024-09-16T00:08:33.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:33.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113174+0000", 2024-09-16T00:08:33.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.47e-05 2024-09-16T00:08:33.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:33.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113297+0000", 2024-09-16T00:08:33.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012250799999999999 2024-09-16T00:08:33.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:33.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114284+0000", 2024-09-16T00:08:33.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00098720399999999999 2024-09-16T00:08:33.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:33.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114604+0000", 2024-09-16T00:08:33.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031988699999999998 2024-09-16T00:08:33.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:33.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114644+0000", 2024-09-16T00:08:33.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9919999999999997e-05 2024-09-16T00:08:33.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:33.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:33.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:33.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114688+0000", 2024-09-16T00:08:33.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4604999999999997e-05 2024-09-16T00:08:33.974 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.974 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.974 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.974 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:33.974 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:33.974 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:33.976 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:08:33.977 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T00:08:33.977 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:33.975+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:34.076 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:34.076+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:34.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.118+0000 7fbe177f6640 1 Processor -- start 2024-09-16T00:08:34.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.118+0000 7fbe177f6640 1 -- start start 2024-09-16T00:08:34.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.119+0000 7fbe177f6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe10154c20 0x7fbe1015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:34.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.119+0000 7fbe177f6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbe10068060 0x7fbe100c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:34.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.119+0000 7fbe177f6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe100cc4b0 con 0x7fbe100ccc90 2024-09-16T00:08:34.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.119+0000 7fbe177f6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbe1015b7c0 con 0x7fbe10068060 2024-09-16T00:08:34.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.119+0000 7fbe177f6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbe100cc260 con 0x7fbe10154c20 2024-09-16T00:08:34.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.119+0000 7fbe14d91640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbe10068060 0x7fbe100c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:34.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.119+0000 7fbe14d91640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbe10068060 0x7fbe100c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43178/0 (socket says 172.21.15.139:43178) 2024-09-16T00:08:34.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.119+0000 7fbe14d91640 1 -- 172.21.15.139:0/1841942366 learned_addr learned my addr 172.21.15.139:0/1841942366 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:34.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.119+0000 7fbe15592640 1 --2- 172.21.15.139:0/1841942366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe10154c20 0x7fbe1015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:34.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.119+0000 7fbe14d91640 1 -- 172.21.15.139:0/1841942366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe10154c20 msgr2=0x7fbe1015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:34.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.119+0000 7fbe14d91640 1 --2- 172.21.15.139:0/1841942366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe10154c20 0x7fbe1015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:34.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.119+0000 7fbe14d91640 1 -- 172.21.15.139:0/1841942366 >> v1:172.21.15.164:6789/0 conn(0x7fbe100ccc90 legacy=0x7fbe100cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:34.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.119+0000 7fbe14d91640 1 -- 172.21.15.139:0/1841942366 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe100cc790 con 0x7fbe10068060 2024-09-16T00:08:34.122 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:34.122 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:34.122 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:34.122 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:34.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.120+0000 7fbe14d91640 1 --2- 172.21.15.139:0/1841942366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe10068060 0x7fbe100c5120 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009860 tx=0x7fbe0002a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4c4fab86d6237a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:34.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.120+0000 7fbe07fff640 1 -- 172.21.15.139:0/1841942366 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00037070 con 0x7fbe10068060 2024-09-16T00:08:34.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.120+0000 7fbe07fff640 1 -- 172.21.15.139:0/1841942366 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe00032640 con 0x7fbe10068060 2024-09-16T00:08:34.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.121+0000 7fbe07fff640 1 -- 172.21.15.139:0/1841942366 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00032a10 con 0x7fbe10068060 2024-09-16T00:08:34.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.121+0000 7fbe177f6640 1 -- 172.21.15.139:0/1841942366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe10068060 msgr2=0x7fbe100c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:34.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.121+0000 7fbe177f6640 1 --2- 172.21.15.139:0/1841942366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe10068060 0x7fbe100c5120 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009860 tx=0x7fbe0002a2e0 comp rx=0 tx=0).stop 2024-09-16T00:08:34.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.121+0000 7fbe177f6640 1 -- 172.21.15.139:0/1841942366 shutdown_connections 2024-09-16T00:08:34.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.121+0000 7fbe177f6640 1 --2- 172.21.15.139:0/1841942366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe10068060 0x7fbe100c5120 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:34.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.121+0000 7fbe177f6640 1 --2- 172.21.15.139:0/1841942366 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe10154c20 0x7fbe1015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:34.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.121+0000 7fbe177f6640 1 -- 172.21.15.139:0/1841942366 >> 172.21.15.139:0/1841942366 conn(0x7fbe1006ca40 msgr2=0x7fbe10067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:34.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.121+0000 7fbe177f6640 1 -- 172.21.15.139:0/1841942366 shutdown_connections 2024-09-16T00:08:34.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.121+0000 7fbe177f6640 1 -- 172.21.15.139:0/1841942366 wait complete. 2024-09-16T00:08:34.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.123+0000 7fbe177f6640 1 Processor -- start 2024-09-16T00:08:34.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.123+0000 7fbe177f6640 1 -- start start 2024-09-16T00:08:34.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.124+0000 7fbe177f6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe100ccc90 0x7fbe101f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:34.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.124+0000 7fbe177f6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe10154c20 0x7fbe100ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:34.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.124+0000 7fbe177f6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe100c7580 0x7fbe101f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:34.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.124+0000 7fbe177f6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1015e3b0 con 0x7fbe100ccc90 2024-09-16T00:08:34.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.124+0000 7fbe177f6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1015e0d0 con 0x7fbe100c7580 2024-09-16T00:08:34.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.124+0000 7fbe177f6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbe1015e690 con 0x7fbe10154c20 2024-09-16T00:08:34.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.124+0000 7fbe15592640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe100ccc90 0x7fbe101f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:34.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.124+0000 7fbe15592640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe100ccc90 0x7fbe101f52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43192/0 (socket says 172.21.15.139:43192) 2024-09-16T00:08:34.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.124+0000 7fbe15592640 1 -- 172.21.15.139:0/3038477142 learned_addr learned my addr 172.21.15.139:0/3038477142 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:34.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.124+0000 7fbe14d91640 1 --2- 172.21.15.139:0/3038477142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe10154c20 0x7fbe100ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:34.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.125+0000 7fbe14d91640 1 -- 172.21.15.139:0/3038477142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe100c7580 msgr2=0x7fbe101f11f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:08:34.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.125+0000 7fbe14d91640 1 --2- 172.21.15.139:0/3038477142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe100c7580 0x7fbe101f11f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:34.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.125+0000 7fbe14d91640 1 -- 172.21.15.139:0/3038477142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe100ccc90 msgr2=0x7fbe101f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:34.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.125+0000 7fbe14d91640 1 --2- 172.21.15.139:0/3038477142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe100ccc90 0x7fbe101f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:34.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.125+0000 7fbe14d91640 1 -- 172.21.15.139:0/3038477142 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1015ccd0 con 0x7fbe10154c20 2024-09-16T00:08:34.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.125+0000 7fbe14d91640 1 --2- 172.21.15.139:0/3038477142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe10154c20 0x7fbe100ce410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe00031000 tx=0x7fbe000046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:34.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.125+0000 7fbe05ffb640 1 -- 172.21.15.139:0/3038477142 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00037040 con 0x7fbe10154c20 2024-09-16T00:08:34.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.125+0000 7fbe05ffb640 1 -- 172.21.15.139:0/3038477142 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe00004020 con 0x7fbe10154c20 2024-09-16T00:08:34.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.125+0000 7fbe177f6640 1 -- 172.21.15.139:0/3038477142 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1015d920 con 0x7fbe10154c20 2024-09-16T00:08:34.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.125+0000 7fbe05ffb640 1 -- 172.21.15.139:0/3038477142 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00004410 con 0x7fbe10154c20 2024-09-16T00:08:34.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.125+0000 7fbe177f6640 1 -- 172.21.15.139:0/3038477142 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe101f4800 con 0x7fbe10154c20 2024-09-16T00:08:34.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.127+0000 7fbe177f6640 1 -- 172.21.15.139:0/3038477142 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe100686b0 con 0x7fbe10154c20 2024-09-16T00:08:34.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.127+0000 7fbe05ffb640 1 -- 172.21.15.139:0/3038477142 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbe00046060 con 0x7fbe10154c20 2024-09-16T00:08:34.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.127+0000 7fbe05ffb640 1 --2- 172.21.15.139:0/3038477142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbdf406c2b0 0x7fbdf406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:34.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.127+0000 7fbe05ffb640 1 -- 172.21.15.139:0/3038477142 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(44..44 src has 1..44) ==== 5915+0+0 (secure 0 0 0) 0x7fbe000aaf50 con 0x7fbe10154c20 2024-09-16T00:08:34.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.127+0000 7fbe15592640 1 --2- 172.21.15.139:0/3038477142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbdf406c2b0 0x7fbdf406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:34.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.128+0000 7fbe15592640 1 --2- 172.21.15.139:0/3038477142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbdf406c2b0 0x7fbdf406e770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009b60 tx=0x7fbe0c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:34.132 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:08:34.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.133+0000 7fbe05ffb640 1 -- 172.21.15.139:0/3038477142 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbe00079540 con 0x7fbe10154c20 2024-09-16T00:08:34.178 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:34.177+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:34.235 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:34.235 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:34.235 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:34.235 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:34.235 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:34.244 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:08:34.288 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:34.287+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:34.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.327+0000 7fbe177f6640 1 -- 172.21.15.139:0/3038477142 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) -- 0x7fbe100cca70 con 0x7fbe10154c20 2024-09-16T00:08:34.351 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:34.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 71.266713838000001, 2024-09-16T00:08:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:08:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.352 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.353 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.353 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:08:34.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:34.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:08:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:08:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:08:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:08:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:08:34.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:08:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:08:34.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:08:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:08:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:08:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:08:34.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:08:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:08:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:08:34.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:08:34.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:08:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:08:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:08:34.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:08:34.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:08:34.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:08:34.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:08:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:08:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:08:34.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:08:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:08:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:08:34.368 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:34.369 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.187182207999996, 2024-09-16T00:08:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093071190000000008, 2024-09-16T00:08:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.370 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 106 2024-09-16T00:08:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:34.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160362+0000", 2024-09-16T00:08:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160438+0000", 2024-09-16T00:08:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6711000000000005e-05 2024-09-16T00:08:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160444+0000", 2024-09-16T00:08:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0530000000000003e-06 2024-09-16T00:08:34.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160449+0000", 2024-09-16T00:08:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2189999999999998e-06 2024-09-16T00:08:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161270+0000", 2024-09-16T00:08:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082099199999999999 2024-09-16T00:08:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161348+0000", 2024-09-16T00:08:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8361000000000004e-05 2024-09-16T00:08:34.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161472+0000", 2024-09-16T00:08:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123523 2024-09-16T00:08:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163461+0000", 2024-09-16T00:08:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019891560000000002 2024-09-16T00:08:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169418+0000", 2024-09-16T00:08:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059568490000000002 2024-09-16T00:08:34.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169570+0000", 2024-09-16T00:08:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152461 2024-09-16T00:08:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169668+0000", 2024-09-16T00:08:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T00:08:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.187086437000005, 2024-09-16T00:08:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092931679999999992, 2024-09-16T00:08:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.381 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107 2024-09-16T00:08:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.382 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:34.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0699999999999998e-07 2024-09-16T00:08:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160459+0000", 2024-09-16T00:08:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0370000000000001e-06 2024-09-16T00:08:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160463+0000", 2024-09-16T00:08:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999999e-06 2024-09-16T00:08:34.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160466+0000", 2024-09-16T00:08:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.644e-06 2024-09-16T00:08:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161873+0000", 2024-09-16T00:08:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014074840000000001 2024-09-16T00:08:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161949+0000", 2024-09-16T00:08:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5749999999999998e-05 2024-09-16T00:08:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162069+0000", 2024-09-16T00:08:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120257 2024-09-16T00:08:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163506+0000", 2024-09-16T00:08:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001436344 2024-09-16T00:08:34.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169704+0000", 2024-09-16T00:08:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061982829999999997 2024-09-16T00:08:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169730+0000", 2024-09-16T00:08:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6060000000000001e-05 2024-09-16T00:08:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169750+0000", 2024-09-16T00:08:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0092e-05 2024-09-16T00:08:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.389 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.186977124999999, 2024-09-16T00:08:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092280039999999997, 2024-09-16T00:08:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.390 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 108 2024-09-16T00:08:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160567+0000", 2024-09-16T00:08:34.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7300000000000001e-07 2024-09-16T00:08:34.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160569+0000", 2024-09-16T00:08:34.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5299999999999999e-06 2024-09-16T00:08:34.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160574+0000", 2024-09-16T00:08:34.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-16T00:08:34.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160577+0000", 2024-09-16T00:08:34.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.01e-06 2024-09-16T00:08:34.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162518+0000", 2024-09-16T00:08:34.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001940703 2024-09-16T00:08:34.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162571+0000", 2024-09-16T00:08:34.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2670000000000002e-05 2024-09-16T00:08:34.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162647+0000", 2024-09-16T00:08:34.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6778000000000002e-05 2024-09-16T00:08:34.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163544+0000", 2024-09-16T00:08:34.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089702599999999999 2024-09-16T00:08:34.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169754+0000", 2024-09-16T00:08:34.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062091739999999996 2024-09-16T00:08:34.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169776+0000", 2024-09-16T00:08:34.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2391999999999998e-05 2024-09-16T00:08:34.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169794+0000", 2024-09-16T00:08:34.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8318999999999999e-05 2024-09-16T00:08:34.398 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.398 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.398 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.398 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:34.398 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:34.398 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.186928574999996, 2024-09-16T00:08:34.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092220679999999999, 2024-09-16T00:08:34.399 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.399 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.399 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.399 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.399 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.399 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.399 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 109 2024-09-16T00:08:34.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.399 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:34.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:34.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:34.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9900000000000001e-07 2024-09-16T00:08:34.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160707+0000", 2024-09-16T00:08:34.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1621999999999998e-05 2024-09-16T00:08:34.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160713+0000", 2024-09-16T00:08:34.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.596e-06 2024-09-16T00:08:34.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160717+0000", 2024-09-16T00:08:34.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5829999999999998e-06 2024-09-16T00:08:34.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162869+0000", 2024-09-16T00:08:34.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021516310000000002 2024-09-16T00:08:34.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162946+0000", 2024-09-16T00:08:34.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7696999999999998e-05 2024-09-16T00:08:34.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163065+0000", 2024-09-16T00:08:34.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118581 2024-09-16T00:08:34.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164431+0000", 2024-09-16T00:08:34.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001366003 2024-09-16T00:08:34.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169797+0000", 2024-09-16T00:08:34.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005366244 2024-09-16T00:08:34.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169819+0000", 2024-09-16T00:08:34.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2036999999999999e-05 2024-09-16T00:08:34.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169837+0000", 2024-09-16T00:08:34.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7575e-05 2024-09-16T00:08:34.406 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.406 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.406 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.407 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:34.407 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:34.407 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.186819225999997, 2024-09-16T00:08:34.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092871919999999997, 2024-09-16T00:08:34.407 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.407 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.407 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.408 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.408 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.408 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.408 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 110 2024-09-16T00:08:34.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.408 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:34.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:34.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:34.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:08:34.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160728+0000", 2024-09-16T00:08:34.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0620000000000001e-06 2024-09-16T00:08:34.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160731+0000", 2024-09-16T00:08:34.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6500000000000002e-06 2024-09-16T00:08:34.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160734+0000", 2024-09-16T00:08:34.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.762e-06 2024-09-16T00:08:34.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163580+0000", 2024-09-16T00:08:34.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028464549999999999 2024-09-16T00:08:34.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163657+0000", 2024-09-16T00:08:34.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6695999999999999e-05 2024-09-16T00:08:34.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163798+0000", 2024-09-16T00:08:34.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014055 2024-09-16T00:08:34.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165483+0000", 2024-09-16T00:08:34.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016853090000000001 2024-09-16T00:08:34.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169970+0000", 2024-09-16T00:08:34.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00448709 2024-09-16T00:08:34.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169994+0000", 2024-09-16T00:08:34.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3575999999999999e-05 2024-09-16T00:08:34.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170011+0000", 2024-09-16T00:08:34.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7702000000000001e-05 2024-09-16T00:08:34.418 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.418 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.418 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.418 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:34.418 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:34.419 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.186805557, 2024-09-16T00:08:34.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093174859999999998, 2024-09-16T00:08:34.419 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.419 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.419 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.419 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.420 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.420 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 111 2024-09-16T00:08:34.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.420 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:34.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:34.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:34.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1e-07 2024-09-16T00:08:34.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160741+0000", 2024-09-16T00:08:34.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6110000000000001e-06 2024-09-16T00:08:34.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160744+0000", 2024-09-16T00:08:34.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3299999999999999e-06 2024-09-16T00:08:34.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160746+0000", 2024-09-16T00:08:34.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5469999999999999e-06 2024-09-16T00:08:34.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164069+0000", 2024-09-16T00:08:34.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033222709999999999 2024-09-16T00:08:34.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164113+0000", 2024-09-16T00:08:34.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4212999999999998e-05 2024-09-16T00:08:34.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164187+0000", 2024-09-16T00:08:34.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4136999999999998e-05 2024-09-16T00:08:34.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166429+0000", 2024-09-16T00:08:34.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022423180000000001 2024-09-16T00:08:34.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170016+0000", 2024-09-16T00:08:34.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035869449999999998 2024-09-16T00:08:34.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170038+0000", 2024-09-16T00:08:34.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2053e-05 2024-09-16T00:08:34.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170055+0000", 2024-09-16T00:08:34.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6850999999999998e-05 2024-09-16T00:08:34.432 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.432 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.432 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.433 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:34.433 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:34.433 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.186791842000005, 2024-09-16T00:08:34.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093480279999999995, 2024-09-16T00:08:34.433 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.433 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.434 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.434 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.434 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.434 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.434 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 112 2024-09-16T00:08:34.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.434 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:34.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:34.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160752+0000", 2024-09-16T00:08:34.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2700000000000001e-07 2024-09-16T00:08:34.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160808+0000", 2024-09-16T00:08:34.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5797999999999997e-05 2024-09-16T00:08:34.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160814+0000", 2024-09-16T00:08:34.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5980000000000002e-06 2024-09-16T00:08:34.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160819+0000", 2024-09-16T00:08:34.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8960000000000002e-06 2024-09-16T00:08:34.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164468+0000", 2024-09-16T00:08:34.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003649476 2024-09-16T00:08:34.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164525+0000", 2024-09-16T00:08:34.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6227999999999998e-05 2024-09-16T00:08:34.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164595+0000", 2024-09-16T00:08:34.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759000000000005e-05 2024-09-16T00:08:34.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167237+0000", 2024-09-16T00:08:34.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026410470000000001 2024-09-16T00:08:34.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170060+0000", 2024-09-16T00:08:34.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028234969999999999 2024-09-16T00:08:34.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170081+0000", 2024-09-16T00:08:34.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1242000000000001e-05 2024-09-16T00:08:34.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170100+0000", 2024-09-16T00:08:34.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8260000000000001e-05 2024-09-16T00:08:34.443 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.444 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.444 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:34.444 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:34.444 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.186714746999996, 2024-09-16T00:08:34.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093139409999999992, 2024-09-16T00:08:34.445 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.445 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.445 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.446 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 113 2024-09-16T00:08:34.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.446 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:34.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:34.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:34.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-07 2024-09-16T00:08:34.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160833+0000", 2024-09-16T00:08:34.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.557e-06 2024-09-16T00:08:34.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160839+0000", 2024-09-16T00:08:34.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3829999999999997e-06 2024-09-16T00:08:34.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160844+0000", 2024-09-16T00:08:34.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6909999999999996e-06 2024-09-16T00:08:34.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165008+0000", 2024-09-16T00:08:34.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041642129999999999 2024-09-16T00:08:34.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165085+0000", 2024-09-16T00:08:34.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6651999999999997e-05 2024-09-16T00:08:34.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165205+0000", 2024-09-16T00:08:34.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012045900000000001 2024-09-16T00:08:34.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167283+0000", 2024-09-16T00:08:34.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020782909999999999 2024-09-16T00:08:34.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170104+0000", 2024-09-16T00:08:34.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028207950000000001 2024-09-16T00:08:34.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170125+0000", 2024-09-16T00:08:34.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0852e-05 2024-09-16T00:08:34.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170143+0000", 2024-09-16T00:08:34.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7467000000000001e-05 2024-09-16T00:08:34.455 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.456 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.456 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.456 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:34.456 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:34.456 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.186668264999994, 2024-09-16T00:08:34.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093126960000000005, 2024-09-16T00:08:34.457 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.457 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.457 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.457 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.458 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-16T00:08:34.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.458 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:34.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:34.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160876+0000", 2024-09-16T00:08:34.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5700000000000002e-07 2024-09-16T00:08:34.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160880+0000", 2024-09-16T00:08:34.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5539999999999999e-06 2024-09-16T00:08:34.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160888+0000", 2024-09-16T00:08:34.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.157e-06 2024-09-16T00:08:34.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160894+0000", 2024-09-16T00:08:34.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7690000000000002e-06 2024-09-16T00:08:34.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165519+0000", 2024-09-16T00:08:34.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046249539999999997 2024-09-16T00:08:34.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165593+0000", 2024-09-16T00:08:34.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4039e-05 2024-09-16T00:08:34.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165715+0000", 2024-09-16T00:08:34.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012131899999999999 2024-09-16T00:08:34.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167309+0000", 2024-09-16T00:08:34.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001594011 2024-09-16T00:08:34.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170147+0000", 2024-09-16T00:08:34.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028387130000000001 2024-09-16T00:08:34.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170170+0000", 2024-09-16T00:08:34.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2517000000000001e-05 2024-09-16T00:08:34.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170188+0000", 2024-09-16T00:08:34.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8006e-05 2024-09-16T00:08:34.467 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.467 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.467 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.468 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:34.468 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:34.468 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.186627236999996, 2024-09-16T00:08:34.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093140040000000007, 2024-09-16T00:08:34.468 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.468 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.469 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.469 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.469 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-16T00:08:34.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.469 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:34.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:34.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160917+0000", 2024-09-16T00:08:34.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7599999999999996e-07 2024-09-16T00:08:34.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160992+0000", 2024-09-16T00:08:34.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4760999999999998e-05 2024-09-16T00:08:34.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161000+0000", 2024-09-16T00:08:34.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9829999999999996e-06 2024-09-16T00:08:34.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161007+0000", 2024-09-16T00:08:34.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4909999999999999e-06 2024-09-16T00:08:34.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165983+0000", 2024-09-16T00:08:34.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049755399999999997 2024-09-16T00:08:34.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166027+0000", 2024-09-16T00:08:34.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4109000000000001e-05 2024-09-16T00:08:34.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166100+0000", 2024-09-16T00:08:34.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3360000000000002e-05 2024-09-16T00:08:34.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169698+0000", 2024-09-16T00:08:34.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597795 2024-09-16T00:08:34.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170192+0000", 2024-09-16T00:08:34.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049447300000000005 2024-09-16T00:08:34.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170214+0000", 2024-09-16T00:08:34.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1546e-05 2024-09-16T00:08:34.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170230+0000", 2024-09-16T00:08:34.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6169999999999999e-05 2024-09-16T00:08:34.478 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.478 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.478 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.478 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:34.478 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:34.478 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.186517507999994, 2024-09-16T00:08:34.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092457029999999992, 2024-09-16T00:08:34.478 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.479 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.479 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.479 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.479 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-16T00:08:34.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.479 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:34.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:34.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:34.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7899999999999999e-07 2024-09-16T00:08:34.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161029+0000", 2024-09-16T00:08:34.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6630000000000002e-06 2024-09-16T00:08:34.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161036+0000", 2024-09-16T00:08:34.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T00:08:34.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161041+0000", 2024-09-16T00:08:34.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:08:34.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166468+0000", 2024-09-16T00:08:34.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054266050000000001 2024-09-16T00:08:34.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166511+0000", 2024-09-16T00:08:34.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3869999999999998e-05 2024-09-16T00:08:34.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166606+0000", 2024-09-16T00:08:34.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4437000000000004e-05 2024-09-16T00:08:34.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169955+0000", 2024-09-16T00:08:34.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033494810000000001 2024-09-16T00:08:34.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170234+0000", 2024-09-16T00:08:34.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027900899999999998 2024-09-16T00:08:34.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170255+0000", 2024-09-16T00:08:34.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0537999999999999e-05 2024-09-16T00:08:34.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170272+0000", 2024-09-16T00:08:34.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6611000000000001e-05 2024-09-16T00:08:34.484 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.484 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.484 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.484 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:34.484 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:34.485 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.186492362999999, 2024-09-16T00:08:34.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093511010000000006, 2024-09-16T00:08:34.485 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.485 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.485 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.485 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.485 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-16T00:08:34.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.485 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:34.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:34.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:34.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.63e-07 2024-09-16T00:08:34.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161055+0000", 2024-09-16T00:08:34.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6770000000000001e-06 2024-09-16T00:08:34.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161063+0000", 2024-09-16T00:08:34.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6469999999999998e-06 2024-09-16T00:08:34.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161068+0000", 2024-09-16T00:08:34.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.746e-06 2024-09-16T00:08:34.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166817+0000", 2024-09-16T00:08:34.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057494959999999998 2024-09-16T00:08:34.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166893+0000", 2024-09-16T00:08:34.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6210000000000004e-05 2024-09-16T00:08:34.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167019+0000", 2024-09-16T00:08:34.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125585 2024-09-16T00:08:34.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170275+0000", 2024-09-16T00:08:34.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032563620000000001 2024-09-16T00:08:34.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170309+0000", 2024-09-16T00:08:34.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3865999999999997e-05 2024-09-16T00:08:34.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170362+0000", 2024-09-16T00:08:34.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3094999999999999e-05 2024-09-16T00:08:34.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170402+0000", 2024-09-16T00:08:34.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9953999999999997e-05 2024-09-16T00:08:34.490 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.490 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.490 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.490 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:34.491 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:34.491 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.737760277999996, 2024-09-16T00:08:34.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097364389999999995, 2024-09-16T00:08:34.491 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.491 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.491 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.491 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.491 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138 2024-09-16T00:08:34.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.491 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:34.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:34.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609784+0000", 2024-09-16T00:08:34.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T00:08:34.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609787+0000", 2024-09-16T00:08:34.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280000000000001e-06 2024-09-16T00:08:34.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609794+0000", 2024-09-16T00:08:34.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6409999999999996e-06 2024-09-16T00:08:34.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609799+0000", 2024-09-16T00:08:34.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9359999999999998e-06 2024-09-16T00:08:34.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615759+0000", 2024-09-16T00:08:34.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059599910000000004 2024-09-16T00:08:34.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615806+0000", 2024-09-16T00:08:34.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7290000000000003e-05 2024-09-16T00:08:34.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615877+0000", 2024-09-16T00:08:34.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1000000000000005e-05 2024-09-16T00:08:34.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618961+0000", 2024-09-16T00:08:34.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030841390000000001 2024-09-16T00:08:34.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619436+0000", 2024-09-16T00:08:34.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047452799999999999 2024-09-16T00:08:34.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619482+0000", 2024-09-16T00:08:34.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5689000000000003e-05 2024-09-16T00:08:34.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619519+0000", 2024-09-16T00:08:34.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7938000000000002e-05 2024-09-16T00:08:34.496 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.496 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.496 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.497 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:34.497 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:34.497 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.737736564000002, 2024-09-16T00:08:34.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011812954000000001, 2024-09-16T00:08:34.497 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.497 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.497 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.497 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.497 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139 2024-09-16T00:08:34.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.498 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:34.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:34.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:34.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999998e-07 2024-09-16T00:08:34.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609810+0000", 2024-09-16T00:08:34.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2100000000000002e-06 2024-09-16T00:08:34.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609817+0000", 2024-09-16T00:08:34.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.28e-06 2024-09-16T00:08:34.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609825+0000", 2024-09-16T00:08:34.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:34.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616116+0000", 2024-09-16T00:08:34.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006290543 2024-09-16T00:08:34.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616187+0000", 2024-09-16T00:08:34.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1830999999999997e-05 2024-09-16T00:08:34.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616298+0000", 2024-09-16T00:08:34.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011044699999999999 2024-09-16T00:08:34.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619559+0000", 2024-09-16T00:08:34.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032614710000000002 2024-09-16T00:08:34.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621562+0000", 2024-09-16T00:08:34.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020028680000000001 2024-09-16T00:08:34.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621597+0000", 2024-09-16T00:08:34.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4625000000000001e-05 2024-09-16T00:08:34.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621620+0000", 2024-09-16T00:08:34.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2864e-05 2024-09-16T00:08:34.502 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.502 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.503 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:34.503 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:34.503 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.737708768000005, 2024-09-16T00:08:34.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011828211999999999, 2024-09-16T00:08:34.503 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.503 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.503 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.503 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.503 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 140 2024-09-16T00:08:34.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.504 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:34.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:34.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:34.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T00:08:34.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:34.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961000000000003e-05 2024-09-16T00:08:34.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609892+0000", 2024-09-16T00:08:34.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0516000000000001e-05 2024-09-16T00:08:34.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609898+0000", 2024-09-16T00:08:34.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0449999999999997e-06 2024-09-16T00:08:34.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616797+0000", 2024-09-16T00:08:34.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068998840000000002 2024-09-16T00:08:34.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616844+0000", 2024-09-16T00:08:34.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6501000000000001e-05 2024-09-16T00:08:34.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616935+0000", 2024-09-16T00:08:34.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1222000000000002e-05 2024-09-16T00:08:34.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619808+0000", 2024-09-16T00:08:34.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873198 2024-09-16T00:08:34.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621624+0000", 2024-09-16T00:08:34.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001815933 2024-09-16T00:08:34.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621646+0000", 2024-09-16T00:08:34.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1622999999999999e-05 2024-09-16T00:08:34.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621663+0000", 2024-09-16T00:08:34.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6854000000000002e-05 2024-09-16T00:08:34.508 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.509 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.509 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:34.509 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:34.509 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.737622959000007, 2024-09-16T00:08:34.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014331501999999999, 2024-09-16T00:08:34.509 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.509 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.509 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.509 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.510 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-16T00:08:34.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.510 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:34.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:34.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609921+0000", 2024-09-16T00:08:34.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9599999999999999e-07 2024-09-16T00:08:34.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609925+0000", 2024-09-16T00:08:34.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T00:08:34.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609932+0000", 2024-09-16T00:08:34.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8240000000000002e-06 2024-09-16T00:08:34.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609937+0000", 2024-09-16T00:08:34.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1070000000000004e-06 2024-09-16T00:08:34.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618987+0000", 2024-09-16T00:08:34.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090502719999999998 2024-09-16T00:08:34.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619064+0000", 2024-09-16T00:08:34.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6868999999999999e-05 2024-09-16T00:08:34.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619183+0000", 2024-09-16T00:08:34.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011961 2024-09-16T00:08:34.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.620052+0000", 2024-09-16T00:08:34.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086822799999999997 2024-09-16T00:08:34.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624197+0000", 2024-09-16T00:08:34.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041454930000000001 2024-09-16T00:08:34.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624229+0000", 2024-09-16T00:08:34.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1758999999999999e-05 2024-09-16T00:08:34.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624252+0000", 2024-09-16T00:08:34.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:08:34.515 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.515 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.516 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:34.516 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:34.516 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.264857151000001, 2024-09-16T00:08:34.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010093276999999999, 2024-09-16T00:08:34.516 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.516 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.516 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.517 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 157 2024-09-16T00:08:34.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.517 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:34.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:34.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:34.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4699999999999998e-07 2024-09-16T00:08:34.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082688+0000", 2024-09-16T00:08:34.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0209999999999998e-06 2024-09-16T00:08:34.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082692+0000", 2024-09-16T00:08:34.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6890000000000002e-06 2024-09-16T00:08:34.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082695+0000", 2024-09-16T00:08:34.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5339999999999998e-06 2024-09-16T00:08:34.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085303+0000", 2024-09-16T00:08:34.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026084789999999999 2024-09-16T00:08:34.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085374+0000", 2024-09-16T00:08:34.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1317999999999997e-05 2024-09-16T00:08:34.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085506+0000", 2024-09-16T00:08:34.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013118 2024-09-16T00:08:34.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086928+0000", 2024-09-16T00:08:34.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014224559999999999 2024-09-16T00:08:34.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092665+0000", 2024-09-16T00:08:34.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057369830000000002 2024-09-16T00:08:34.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092727+0000", 2024-09-16T00:08:34.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1810000000000006e-05 2024-09-16T00:08:34.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092779+0000", 2024-09-16T00:08:34.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2559999999999998e-05 2024-09-16T00:08:34.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.523 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.523 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:34.523 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:34.523 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.264844983000003, 2024-09-16T00:08:34.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010176388, 2024-09-16T00:08:34.523 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.524 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.524 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.524 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 158 2024-09-16T00:08:34.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.524 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:34.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:34.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082699+0000", 2024-09-16T00:08:34.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.11e-07 2024-09-16T00:08:34.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082742+0000", 2024-09-16T00:08:34.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3699000000000003e-05 2024-09-16T00:08:34.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082748+0000", 2024-09-16T00:08:34.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8690000000000001e-06 2024-09-16T00:08:34.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082751+0000", 2024-09-16T00:08:34.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.942e-06 2024-09-16T00:08:34.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085881+0000", 2024-09-16T00:08:34.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031303450000000001 2024-09-16T00:08:34.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085959+0000", 2024-09-16T00:08:34.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7874999999999996e-05 2024-09-16T00:08:34.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086086+0000", 2024-09-16T00:08:34.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012685499999999999 2024-09-16T00:08:34.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087127+0000", 2024-09-16T00:08:34.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001041298 2024-09-16T00:08:34.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092790+0000", 2024-09-16T00:08:34.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056628 2024-09-16T00:08:34.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092832+0000", 2024-09-16T00:08:34.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2227000000000001e-05 2024-09-16T00:08:34.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092875+0000", 2024-09-16T00:08:34.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2267e-05 2024-09-16T00:08:34.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.529 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.529 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:34.530 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:34.530 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.264786944999997, 2024-09-16T00:08:34.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010199547, 2024-09-16T00:08:34.530 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:34.530 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:34.530 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:34.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:34.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:34.530 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159 2024-09-16T00:08:34.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.530 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:34.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:34.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:34.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:34.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:34.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:34.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:34.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082757+0000", 2024-09-16T00:08:34.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2500000000000001e-07 2024-09-16T00:08:34.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:34.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082759+0000", 2024-09-16T00:08:34.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-16T00:08:34.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:34.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082763+0000", 2024-09-16T00:08:34.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4889999999999997e-06 2024-09-16T00:08:34.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082766+0000", 2024-09-16T00:08:34.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:08:34.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:34.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086571+0000", 2024-09-16T00:08:34.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038053269999999998 2024-09-16T00:08:34.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:34.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086619+0000", 2024-09-16T00:08:34.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7924999999999999e-05 2024-09-16T00:08:34.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:34.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086693+0000", 2024-09-16T00:08:34.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000005e-05 2024-09-16T00:08:34.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:34.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087383+0000", 2024-09-16T00:08:34.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069008299999999997 2024-09-16T00:08:34.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:34.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092882+0000", 2024-09-16T00:08:34.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054987450000000002 2024-09-16T00:08:34.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:34.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092921+0000", 2024-09-16T00:08:34.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9388999999999998e-05 2024-09-16T00:08:34.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:34.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:34.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:34.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092956+0000", 2024-09-16T00:08:34.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4647999999999998e-05 2024-09-16T00:08:34.535 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.535 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.535 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.535 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:34.535 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:34.535 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:34.537 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:08:34.538 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:34.536+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:34.638 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:34.637+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:34.649 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:34.650 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:34.650 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:34.650 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:34.659 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:08:34.739 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:34.738+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:34.766 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:34.766 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:34.766 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:34.766 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:34.767 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:34.776 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:08:34.822 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:08:34.840 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:34.838+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:34.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.871+0000 7fbe05ffb640 1 -- 172.21.15.139:0/3038477142 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v45) ==== 113+0+0 (secure 0 0 0) 0x7fbe00078e40 con 0x7fbe10154c20 2024-09-16T00:08:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T00:08:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.875+0000 7fbe177f6640 1 -- 172.21.15.139:0/3038477142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbdf406c2b0 msgr2=0x7fbdf406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.875+0000 7fbe177f6640 1 --2- 172.21.15.139:0/3038477142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbdf406c2b0 0x7fbdf406e770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009b60 tx=0x7fbe0c009190 comp rx=0 tx=0).stop 2024-09-16T00:08:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.875+0000 7fbe177f6640 1 -- 172.21.15.139:0/3038477142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe10154c20 msgr2=0x7fbe100ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.875+0000 7fbe177f6640 1 --2- 172.21.15.139:0/3038477142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe10154c20 0x7fbe100ce410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbe00031000 tx=0x7fbe000046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:34.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.876+0000 7fbe177f6640 1 -- 172.21.15.139:0/3038477142 shutdown_connections 2024-09-16T00:08:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.876+0000 7fbe177f6640 1 --2- 172.21.15.139:0/3038477142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbdf406c2b0 0x7fbdf406e770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.876+0000 7fbe177f6640 1 --2- 172.21.15.139:0/3038477142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe100c7580 0x7fbe101f11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.876+0000 7fbe177f6640 1 --2- 172.21.15.139:0/3038477142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe10154c20 0x7fbe100ce410 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.876+0000 7fbe177f6640 1 --2- 172.21.15.139:0/3038477142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe100ccc90 0x7fbe101f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.876+0000 7fbe177f6640 1 -- 172.21.15.139:0/3038477142 >> 172.21.15.139:0/3038477142 conn(0x7fbe1006ca40 msgr2=0x7fbe1014ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:34.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.876+0000 7fbe177f6640 1 -- 172.21.15.139:0/3038477142 shutdown_connections 2024-09-16T00:08:34.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.876+0000 7fbe177f6640 1 -- 172.21.15.139:0/3038477142 wait complete. 2024-09-16T00:08:34.894 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T00:08:34.907 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:34.907 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:34.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:34.908 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:34.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.908 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'41, mlcod=17'39)", 2024-09-16T00:08:34.908 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:34.908 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 70.883744024999999, 2024-09-16T00:08:34.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001808488, 2024-09-16T00:08:34.909 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:34.909 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:34.909 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.909 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:34.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.910 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:34.910 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:34.910 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:34.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:34.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:34.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021032+0000", 2024-09-16T00:08:34.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3e-07 2024-09-16T00:08:34.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:34.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021037+0000", 2024-09-16T00:08:34.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0640000000000001e-06 2024-09-16T00:08:34.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:34.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021047+0000", 2024-09-16T00:08:34.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.023e-05 2024-09-16T00:08:34.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.915 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.915 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021053+0000", 2024-09-16T00:08:34.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3160000000000002e-06 2024-09-16T00:08:34.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:34.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021092+0000", 2024-09-16T00:08:34.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9033999999999999e-05 2024-09-16T00:08:34.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:34.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021105+0000", 2024-09-16T00:08:34.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2768e-05 2024-09-16T00:08:34.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022811+0000", 2024-09-16T00:08:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001705788 2024-09-16T00:08:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022840+0000", 2024-09-16T00:08:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8558e-05 2024-09-16T00:08:34.918 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.919 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:34.919 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.919 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:85 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'55, mlcod=17'50)", 2024-09-16T00:08:34.919 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:34.919 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 70.870715024000006, 2024-09-16T00:08:34.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017909530000000001, 2024-09-16T00:08:34.919 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:34.919 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:34.919 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.919 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:34.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:34.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:34.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:34.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:34.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:34.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034061+0000", 2024-09-16T00:08:34.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0399999999999996e-07 2024-09-16T00:08:34.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.921 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:34.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034065+0000", 2024-09-16T00:08:34.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8840000000000002e-06 2024-09-16T00:08:34.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.922 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:34.922 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034074+0000", 2024-09-16T00:08:34.922 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:34.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.922 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.922 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034079+0000", 2024-09-16T00:08:34.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0000000000000004e-06 2024-09-16T00:08:34.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.923 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:34.923 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034125+0000", 2024-09-16T00:08:34.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5905000000000002e-05 2024-09-16T00:08:34.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.923 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:34.923 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034144+0000", 2024-09-16T00:08:34.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9593e-05 2024-09-16T00:08:34.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:34.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035817+0000", 2024-09-16T00:08:34.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016728240000000001 2024-09-16T00:08:34.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:34.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035851+0000", 2024-09-16T00:08:34.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3809000000000001e-05 2024-09-16T00:08:34.925 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.925 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:34.925 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.926 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:112 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'74, mlcod=17'65)", 2024-09-16T00:08:34.926 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:34.926 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 70.740103161999997, 2024-09-16T00:08:34.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018457020000000001, 2024-09-16T00:08:34.926 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:34.927 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:34.927 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.927 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:34.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.927 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:34.927 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:34.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:34.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:34.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:34.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164673+0000", 2024-09-16T00:08:34.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1200000000000003e-07 2024-09-16T00:08:34.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:34.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164710+0000", 2024-09-16T00:08:34.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7756999999999998e-05 2024-09-16T00:08:34.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:34.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164719+0000", 2024-09-16T00:08:34.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2849999999999995e-06 2024-09-16T00:08:34.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164723+0000", 2024-09-16T00:08:34.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3220000000000003e-06 2024-09-16T00:08:34.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:34.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164760+0000", 2024-09-16T00:08:34.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7311999999999998e-05 2024-09-16T00:08:34.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:34.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164771+0000", 2024-09-16T00:08:34.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0869e-05 2024-09-16T00:08:34.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:34.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166495+0000", 2024-09-16T00:08:34.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001723863 2024-09-16T00:08:34.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:34.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166518+0000", 2024-09-16T00:08:34.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2682000000000001e-05 2024-09-16T00:08:34.933 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'66)", 2024-09-16T00:08:34.934 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:34.935 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 70.738599183000005, 2024-09-16T00:08:34.935 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032925509999999999, 2024-09-16T00:08:34.935 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:34.935 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:34.935 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.935 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:34.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:34.936 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:34.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.936 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:34.936 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:34.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.936 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:34.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166177+0000", 2024-09-16T00:08:34.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:34.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:34.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166216+0000", 2024-09-16T00:08:34.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9008000000000003e-05 2024-09-16T00:08:34.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:34.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166226+0000", 2024-09-16T00:08:34.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0081999999999999e-05 2024-09-16T00:08:34.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166230+0000", 2024-09-16T00:08:34.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5079999999999999e-06 2024-09-16T00:08:34.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:34.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166259+0000", 2024-09-16T00:08:34.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8928000000000002e-05 2024-09-16T00:08:34.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:34.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166271+0000", 2024-09-16T00:08:34.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1531e-05 2024-09-16T00:08:34.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:34.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169449+0000", 2024-09-16T00:08:34.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003178143 2024-09-16T00:08:34.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:34.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169468+0000", 2024-09-16T00:08:34.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9700000000000001e-05 2024-09-16T00:08:34.941 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.942 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:34.942 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.942 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'66)", 2024-09-16T00:08:34.942 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:34.943 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 70.738137723999998, 2024-09-16T00:08:34.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003179374, 2024-09-16T00:08:34.943 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:34.943 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:34.943 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.943 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:34.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.943 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:34.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:34.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:34.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:34.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:34.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166638+0000", 2024-09-16T00:08:34.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2300000000000002e-07 2024-09-16T00:08:34.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:34.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166641+0000", 2024-09-16T00:08:34.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0079999999999998e-06 2024-09-16T00:08:34.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:34.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166646+0000", 2024-09-16T00:08:34.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4970000000000001e-06 2024-09-16T00:08:34.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166650+0000", 2024-09-16T00:08:34.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5729999999999999e-06 2024-09-16T00:08:34.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:34.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166682+0000", 2024-09-16T00:08:34.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2218999999999998e-05 2024-09-16T00:08:34.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.947 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:34.947 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166694+0000", 2024-09-16T00:08:34.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1902999999999999e-05 2024-09-16T00:08:34.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.947 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:34.947 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169784+0000", 2024-09-16T00:08:34.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003089545 2024-09-16T00:08:34.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.948 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:34.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169817+0000", 2024-09-16T00:08:34.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3206e-05 2024-09-16T00:08:34.948 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.948 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:34.949 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.949 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.949 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:117 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'66)", 2024-09-16T00:08:34.949 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:34.949 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 70.737693664000005, 2024-09-16T00:08:34.949 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00275326, 2024-09-16T00:08:34.949 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:34.949 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:34.949 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.949 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:34.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:34.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:34.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:34.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:34.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:34.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167082+0000", 2024-09-16T00:08:34.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:08:34.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:34.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167086+0000", 2024-09-16T00:08:34.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1559999999999997e-06 2024-09-16T00:08:34.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:34.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167096+0000", 2024-09-16T00:08:34.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8390000000000002e-06 2024-09-16T00:08:34.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.952 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167101+0000", 2024-09-16T00:08:34.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:08:34.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.953 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.953 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:34.953 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167130+0000", 2024-09-16T00:08:34.953 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9649e-05 2024-09-16T00:08:34.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.954 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:34.954 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167142+0000", 2024-09-16T00:08:34.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1435e-05 2024-09-16T00:08:34.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:34.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169818+0000", 2024-09-16T00:08:34.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002676056 2024-09-16T00:08:34.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:34.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169835+0000", 2024-09-16T00:08:34.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6980999999999999e-05 2024-09-16T00:08:34.956 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.956 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:34.956 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.956 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:130 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'88, mlcod=17'85)", 2024-09-16T00:08:34.956 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:34.956 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 70.294894334999995, 2024-09-16T00:08:34.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002880456, 2024-09-16T00:08:34.957 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:34.957 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:34.957 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.957 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:34.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:34.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:34.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:34.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:34.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:34.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:34.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5699999999999998e-07 2024-09-16T00:08:34.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.959 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:34.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609885+0000", 2024-09-16T00:08:34.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.608e-06 2024-09-16T00:08:34.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:34.960 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609891+0000", 2024-09-16T00:08:34.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:34.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.960 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609895+0000", 2024-09-16T00:08:34.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.54e-06 2024-09-16T00:08:34.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:34.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609931+0000", 2024-09-16T00:08:34.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5893999999999999e-05 2024-09-16T00:08:34.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:34.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609945+0000", 2024-09-16T00:08:34.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4259e-05 2024-09-16T00:08:34.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:34.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612721+0000", 2024-09-16T00:08:34.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027757849999999998 2024-09-16T00:08:34.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:34.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612761+0000", 2024-09-16T00:08:34.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0638e-05 2024-09-16T00:08:34.962 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.962 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:34.963 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.963 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-16T00:08:34.963 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:34.963 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 70.294510361999997, 2024-09-16T00:08:34.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028016529999999999, 2024-09-16T00:08:34.964 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:34.964 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:34.964 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.964 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:34.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:34.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:34.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:34.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:34.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:34.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:34.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.63e-07 2024-09-16T00:08:34.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:34.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610344+0000", 2024-09-16T00:08:34.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8996000000000001e-05 2024-09-16T00:08:34.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:34.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610352+0000", 2024-09-16T00:08:34.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:34.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610356+0000", 2024-09-16T00:08:34.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0790000000000002e-06 2024-09-16T00:08:34.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.967 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:34.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610388+0000", 2024-09-16T00:08:34.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1541000000000002e-05 2024-09-16T00:08:34.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.967 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:34.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610400+0000", 2024-09-16T00:08:34.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2075e-05 2024-09-16T00:08:34.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:34.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613045+0000", 2024-09-16T00:08:34.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002645213 2024-09-16T00:08:34.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:34.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613066+0000", 2024-09-16T00:08:34.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1353e-05 2024-09-16T00:08:34.968 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.968 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:34.969 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.969 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-16T00:08:34.969 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:34.969 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 70.291250861999998, 2024-09-16T00:08:34.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027254850000000001, 2024-09-16T00:08:34.969 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:34.969 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:34.969 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.969 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:34.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:34.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:34.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:34.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:34.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:34.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613525+0000", 2024-09-16T00:08:34.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2800000000000002e-07 2024-09-16T00:08:34.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.971 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:34.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613528+0000", 2024-09-16T00:08:34.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9730000000000002e-06 2024-09-16T00:08:34.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:34.972 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613534+0000", 2024-09-16T00:08:34.972 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4339999999999997e-06 2024-09-16T00:08:34.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.972 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.972 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613538+0000", 2024-09-16T00:08:34.972 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5149999999999998e-06 2024-09-16T00:08:34.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.972 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:34.973 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613566+0000", 2024-09-16T00:08:34.973 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8104000000000001e-05 2024-09-16T00:08:34.973 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.973 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.973 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:34.973 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613578+0000", 2024-09-16T00:08:34.974 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2404e-05 2024-09-16T00:08:34.974 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.974 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.974 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:34.974 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616217+0000", 2024-09-16T00:08:34.974 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026389830000000001 2024-09-16T00:08:34.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.975 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:34.975 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616250+0000", 2024-09-16T00:08:34.975 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2644000000000001e-05 2024-09-16T00:08:34.975 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.975 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:34.975 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.975 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:134 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'92, mlcod=18'87)", 2024-09-16T00:08:34.976 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:34.976 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 70.290712722999999, 2024-09-16T00:08:34.976 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025322399999999998, 2024-09-16T00:08:34.976 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:34.976 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:34.976 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.976 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:34.976 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.976 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:34.976 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:34.976 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.977 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:34.977 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:34.977 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.977 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:34.977 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614063+0000", 2024-09-16T00:08:34.977 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8700000000000005e-07 2024-09-16T00:08:34.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.978 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.978 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:34.978 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614096+0000", 2024-09-16T00:08:34.978 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3136000000000003e-05 2024-09-16T00:08:34.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.978 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.978 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:34.978 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614106+0000", 2024-09-16T00:08:34.978 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9170000000000001e-06 2024-09-16T00:08:34.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.979 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.979 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.979 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614111+0000", 2024-09-16T00:08:34.979 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4390000000000002e-06 2024-09-16T00:08:34.979 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.979 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.979 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:34.979 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614141+0000", 2024-09-16T00:08:34.979 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.04e-05 2024-09-16T00:08:34.979 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.979 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.979 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:34.980 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614152+0000", 2024-09-16T00:08:34.980 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0987e-05 2024-09-16T00:08:34.980 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.980 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.980 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:34.980 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616573+0000", 2024-09-16T00:08:34.980 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002421351 2024-09-16T00:08:34.980 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.980 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.981 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:34.981 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616595+0000", 2024-09-16T00:08:34.981 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1322999999999999e-05 2024-09-16T00:08:34.981 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.981 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:34.981 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.981 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.981 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.981 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:135 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'93, mlcod=18'87)", 2024-09-16T00:08:34.981 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:34.981 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 70.290192684999994, 2024-09-16T00:08:34.981 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020304659999999999, 2024-09-16T00:08:34.982 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:34.982 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:34.982 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.982 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:34.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.982 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:34.982 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:34.982 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.982 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.983 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:34.983 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:34.983 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.983 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.983 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.983 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:34.983 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614583+0000", 2024-09-16T00:08:34.984 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:34.984 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.984 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.984 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:34.984 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614587+0000", 2024-09-16T00:08:34.984 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3100000000000002e-06 2024-09-16T00:08:34.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.985 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.985 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:34.985 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614597+0000", 2024-09-16T00:08:34.985 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7450000000000007e-06 2024-09-16T00:08:34.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.986 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.986 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.986 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614602+0000", 2024-09-16T00:08:34.986 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6099999999999999e-06 2024-09-16T00:08:34.986 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.986 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.987 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:34.987 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614641+0000", 2024-09-16T00:08:34.987 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9767000000000001e-05 2024-09-16T00:08:34.987 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.987 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.987 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:34.987 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614658+0000", 2024-09-16T00:08:34.988 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6205000000000002e-05 2024-09-16T00:08:34.988 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.988 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:34.988 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616596+0000", 2024-09-16T00:08:34.988 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019386659999999999 2024-09-16T00:08:34.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.989 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.989 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:34.989 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616613+0000", 2024-09-16T00:08:34.989 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6555999999999999e-05 2024-09-16T00:08:34.989 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.990 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:34.990 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:34.990 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.990 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:139 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'97, mlcod=18'87)", 2024-09-16T00:08:34.991 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:34.991 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 70.288432013999994, 2024-09-16T00:08:34.991 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030908139999999999, 2024-09-16T00:08:34.991 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:34.991 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:34.991 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:34.992 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:34.992 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.992 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:34.992 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:34.992 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.992 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.993 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.993 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:34.993 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:34.993 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:34.993 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.993 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.994 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:34.994 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616344+0000", 2024-09-16T00:08:34.994 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3000000000000001e-07 2024-09-16T00:08:34.994 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.994 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.994 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:34.995 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616347+0000", 2024-09-16T00:08:34.995 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7740000000000002e-06 2024-09-16T00:08:34.995 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.995 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.995 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:34.996 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616355+0000", 2024-09-16T00:08:34.996 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4699999999999996e-06 2024-09-16T00:08:34.996 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.996 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.996 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:34.996 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616358+0000", 2024-09-16T00:08:34.997 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5899999999999999e-06 2024-09-16T00:08:34.997 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.997 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.997 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:34.997 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616393+0000", 2024-09-16T00:08:34.997 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4529000000000001e-05 2024-09-16T00:08:34.998 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.998 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.998 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:34.998 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616404+0000", 2024-09-16T00:08:34.998 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0726e-05 2024-09-16T00:08:34.998 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:34.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:34.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619411+0000", 2024-09-16T00:08:34.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030076249999999999 2024-09-16T00:08:34.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:34.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:35.000 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619434+0000", 2024-09-16T00:08:35.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-05 2024-09-16T00:08:35.000 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.000 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:35.000 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.001 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:140 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'98, mlcod=18'89)", 2024-09-16T00:08:35.001 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:35.001 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 70.287790620999999, 2024-09-16T00:08:35.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027276150000000001, 2024-09-16T00:08:35.002 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:35.002 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:35.002 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:35.002 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:35.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.002 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:35.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:35.003 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:35.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:35.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:35.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:35.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.004 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:35.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616985+0000", 2024-09-16T00:08:35.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4099999999999999e-07 2024-09-16T00:08:35.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.005 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:35.005 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617055+0000", 2024-09-16T00:08:35.005 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9923000000000001e-05 2024-09-16T00:08:35.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:35.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617063+0000", 2024-09-16T00:08:35.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:35.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:35.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617070+0000", 2024-09-16T00:08:35.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9090000000000003e-06 2024-09-16T00:08:35.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:35.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617102+0000", 2024-09-16T00:08:35.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2620999999999998e-05 2024-09-16T00:08:35.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.009 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:35.009 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617114+0000", 2024-09-16T00:08:35.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1228e-05 2024-09-16T00:08:35.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.010 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:35.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619690+0000", 2024-09-16T00:08:35.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025763890000000001 2024-09-16T00:08:35.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.010 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:35.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619712+0000", 2024-09-16T00:08:35.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2171000000000002e-05 2024-09-16T00:08:35.011 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.011 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:35.011 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.012 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:35.012 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:35.012 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 56.288961520999997, 2024-09-16T00:08:35.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027355320000000002, 2024-09-16T00:08:35.012 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:35.012 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:35.013 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:35.013 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:35.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:35.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:35.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:35.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:35.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:35.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:35.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:35.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615815+0000", 2024-09-16T00:08:35.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1769999999999999e-06 2024-09-16T00:08:35.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:35.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615825+0000", 2024-09-16T00:08:35.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9000000000000001e-06 2024-09-16T00:08:35.015 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.015 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.015 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:35.015 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615840+0000", 2024-09-16T00:08:35.015 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4854e-05 2024-09-16T00:08:35.015 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.015 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.015 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:35.015 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615848+0000", 2024-09-16T00:08:35.015 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0940000000000008e-06 2024-09-16T00:08:35.015 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.015 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.015 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:35.015 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615926+0000", 2024-09-16T00:08:35.016 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8176999999999994e-05 2024-09-16T00:08:35.016 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.016 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.016 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:35.016 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615945+0000", 2024-09-16T00:08:35.016 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9663000000000001e-05 2024-09-16T00:08:35.016 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.016 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.016 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:35.016 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618509+0000", 2024-09-16T00:08:35.016 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002563802 2024-09-16T00:08:35.016 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.017 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.017 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:35.017 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618549+0000", 2024-09-16T00:08:35.017 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9864999999999999e-05 2024-09-16T00:08:35.017 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.017 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:35.017 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.017 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.017 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:35.017 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:35.017 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 56.288872380999997, 2024-09-16T00:08:35.017 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061593860000000002, 2024-09-16T00:08:35.018 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:35.018 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:35.018 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:35.018 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:35.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.018 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:35.018 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:35.018 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:35.018 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.018 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:35.018 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:35.019 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:35.019 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.019 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.019 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:35.019 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615907+0000", 2024-09-16T00:08:35.019 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9389999999999999e-06 2024-09-16T00:08:35.019 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.019 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.019 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:35.019 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615920+0000", 2024-09-16T00:08:35.019 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2989999999999999e-05 2024-09-16T00:08:35.019 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.019 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.019 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:35.020 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615931+0000", 2024-09-16T00:08:35.020 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.164e-05 2024-09-16T00:08:35.020 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.020 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.020 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:35.020 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615939+0000", 2024-09-16T00:08:35.020 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8029999999999997e-06 2024-09-16T00:08:35.020 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.020 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.020 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:35.020 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616310+0000", 2024-09-16T00:08:35.020 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00037111400000000003 2024-09-16T00:08:35.020 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.020 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.021 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:35.021 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616406+0000", 2024-09-16T00:08:35.021 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5372999999999996e-05 2024-09-16T00:08:35.021 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.021 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.021 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:35.021 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616514+0000", 2024-09-16T00:08:35.021 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108564 2024-09-16T00:08:35.021 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.021 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.021 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:35.021 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618158+0000", 2024-09-16T00:08:35.021 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016442449999999999 2024-09-16T00:08:35.021 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.022 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.022 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:35.022 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621972+0000", 2024-09-16T00:08:35.022 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038132299999999999 2024-09-16T00:08:35.022 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.022 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.022 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:35.022 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622018+0000", 2024-09-16T00:08:35.022 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6239000000000002e-05 2024-09-16T00:08:35.022 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.022 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.022 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:35.022 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622062+0000", 2024-09-16T00:08:35.023 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4249000000000003e-05 2024-09-16T00:08:35.023 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.023 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:35.023 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.023 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.023 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.023 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:35.023 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:35.023 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 56.288818913, 2024-09-16T00:08:35.023 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006258914, 2024-09-16T00:08:35.023 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:35.023 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:35.023 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:35.023 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:35.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:35.024 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:35.024 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:35.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:35.024 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:35.024 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:35.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:35.024 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615957+0000", 2024-09-16T00:08:35.024 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1399999999999997e-07 2024-09-16T00:08:35.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.025 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.025 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:35.025 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615960+0000", 2024-09-16T00:08:35.025 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0860000000000002e-06 2024-09-16T00:08:35.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.025 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.025 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:35.025 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615967+0000", 2024-09-16T00:08:35.025 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9059999999999996e-06 2024-09-16T00:08:35.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.025 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.025 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:35.026 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615972+0000", 2024-09-16T00:08:35.026 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5679999999999999e-06 2024-09-16T00:08:35.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.026 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.026 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:35.026 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616717+0000", 2024-09-16T00:08:35.026 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074438 2024-09-16T00:08:35.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.026 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.026 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:35.026 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616781+0000", 2024-09-16T00:08:35.026 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4808999999999996e-05 2024-09-16T00:08:35.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.026 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616885+0000", 2024-09-16T00:08:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010404900000000001 2024-09-16T00:08:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012773870000000001 2024-09-16T00:08:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622116+0000", 2024-09-16T00:08:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039533470000000003 2024-09-16T00:08:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622173+0000", 2024-09-16T00:08:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6347000000000001e-05 2024-09-16T00:08:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622215+0000", 2024-09-16T00:08:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2521000000000002e-05 2024-09-16T00:08:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 56.288795176000001, 2024-09-16T00:08:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041725529999999999, 2024-09-16T00:08:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8799999999999998e-07 2024-09-16T00:08:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615990+0000", 2024-09-16T00:08:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2150000000000007e-06 2024-09-16T00:08:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:08:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7159999999999997e-06 2024-09-16T00:08:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616001+0000", 2024-09-16T00:08:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9339999999999997e-06 2024-09-16T00:08:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616293+0000", 2024-09-16T00:08:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029210899999999998 2024-09-16T00:08:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616401+0000", 2024-09-16T00:08:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108087 2024-09-16T00:08:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616561+0000", 2024-09-16T00:08:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015940600000000001 2024-09-16T00:08:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618146+0000", 2024-09-16T00:08:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015857040000000001 2024-09-16T00:08:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620091+0000", 2024-09-16T00:08:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001944837 2024-09-16T00:08:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620126+0000", 2024-09-16T00:08:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4242999999999997e-05 2024-09-16T00:08:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620152+0000", 2024-09-16T00:08:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6914e-05 2024-09-16T00:08:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 56.288756951000003, 2024-09-16T00:08:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00630628, 2024-09-16T00:08:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:08:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5999999999999999e-07 2024-09-16T00:08:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616022+0000", 2024-09-16T00:08:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0149999999999999e-06 2024-09-16T00:08:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616031+0000", 2024-09-16T00:08:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9770000000000006e-06 2024-09-16T00:08:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616036+0000", 2024-09-16T00:08:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2639999999999999e-06 2024-09-16T00:08:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617057+0000", 2024-09-16T00:08:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010212999999999999 2024-09-16T00:08:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617146+0000", 2024-09-16T00:08:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9234000000000007e-05 2024-09-16T00:08:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617232+0000", 2024-09-16T00:08:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5218999999999998e-05 2024-09-16T00:08:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619107+0000", 2024-09-16T00:08:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001875016 2024-09-16T00:08:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622259+0000", 2024-09-16T00:08:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031527930000000001 2024-09-16T00:08:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622290+0000", 2024-09-16T00:08:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1022999999999998e-05 2024-09-16T00:08:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622324+0000", 2024-09-16T00:08:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3979000000000001e-05 2024-09-16T00:08:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 56.288731648999999, 2024-09-16T00:08:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042412739999999997, 2024-09-16T00:08:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616044+0000", 2024-09-16T00:08:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616052+0000", 2024-09-16T00:08:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.439e-06 2024-09-16T00:08:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616059+0000", 2024-09-16T00:08:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1389999999999998e-06 2024-09-16T00:08:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616065+0000", 2024-09-16T00:08:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:08:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616823+0000", 2024-09-16T00:08:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075781199999999996 2024-09-16T00:08:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616918+0000", 2024-09-16T00:08:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5174999999999996e-05 2024-09-16T00:08:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617019+0000", 2024-09-16T00:08:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101161 2024-09-16T00:08:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618503+0000", 2024-09-16T00:08:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014836750000000001 2024-09-16T00:08:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620201+0000", 2024-09-16T00:08:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00169805 2024-09-16T00:08:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620255+0000", 2024-09-16T00:08:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4499000000000001e-05 2024-09-16T00:08:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620285+0000", 2024-09-16T00:08:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9349e-05 2024-09-16T00:08:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 56.288702657999998, 2024-09-16T00:08:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063682799999999996, 2024-09-16T00:08:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9900000000000001e-07 2024-09-16T00:08:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616079+0000", 2024-09-16T00:08:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9009999999999999e-06 2024-09-16T00:08:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616086+0000", 2024-09-16T00:08:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.776e-06 2024-09-16T00:08:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616091+0000", 2024-09-16T00:08:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5600000000000001e-06 2024-09-16T00:08:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618339+0000", 2024-09-16T00:08:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022476179999999998 2024-09-16T00:08:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9025999999999997e-05 2024-09-16T00:08:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622441+0000", 2024-09-16T00:08:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040029999999999996 2024-09-16T00:08:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:35.052 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:35.052 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T00:08:35.054 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:08:35.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.954+0000 7f22191d8640 1 Processor -- start 2024-09-16T00:08:35.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.955+0000 7f22191d8640 1 -- start start 2024-09-16T00:08:35.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.956+0000 7f22191d8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f221415c700 0x7f2214164050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.956+0000 7f22191d8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2214071830 con 0x7f22140c72a0 2024-09-16T00:08:35.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.956+0000 7f22191d8640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2214071b10 con 0x7f22140c8070 2024-09-16T00:08:35.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.956+0000 7f22191d8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f221415bcb0 con 0x7f221415c700 2024-09-16T00:08:35.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.956+0000 7f2212d9d640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f22140c8070 0x7f22140c5500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:59818/0 (socket says 172.21.15.139:59818) 2024-09-16T00:08:35.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.956+0000 7f221359e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f221415c700 0x7f2214164050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.956+0000 7f2212d9d640 1 -- 172.21.15.139:0/2539505877 learned_addr learned my addr 172.21.15.139:0/2539505877 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:35.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.956+0000 7f220bfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f22140c72a0 0x7f22140c9220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:50288/0 (socket says 172.21.15.139:50288) 2024-09-16T00:08:35.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.957+0000 7f221259c640 1 -- 172.21.15.139:0/2539505877 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1106959925 0 0) 0x7f2214071b10 con 0x7f22140c8070 2024-09-16T00:08:35.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.957+0000 7f221259c640 1 -- 172.21.15.139:0/2539505877 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2200003520 con 0x7f22140c8070 2024-09-16T00:08:35.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.957+0000 7f221359e640 1 -- 172.21.15.139:0/2539505877 >> v1:172.21.15.164:6789/0 conn(0x7f22140c8070 legacy=0x7f22140c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.957+0000 7f221359e640 1 -- 172.21.15.139:0/2539505877 >> v1:172.21.15.139:6789/0 conn(0x7f22140c72a0 legacy=0x7f22140c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.957+0000 7f221359e640 1 -- 172.21.15.139:0/2539505877 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2214071df0 con 0x7f221415c700 2024-09-16T00:08:35.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.958+0000 7f221359e640 1 --2- 172.21.15.139:0/2539505877 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f221415c700 0x7f2214164050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f220c009dc0 tx=0x7f220c02a780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=329b7d25b1fd148c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.958+0000 7f221259c640 1 -- 172.21.15.139:0/2539505877 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c037070 con 0x7f221415c700 2024-09-16T00:08:35.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.958+0000 7f221259c640 1 -- 172.21.15.139:0/2539505877 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220c007740 con 0x7f221415c700 2024-09-16T00:08:35.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.958+0000 7f221259c640 1 -- 172.21.15.139:0/2539505877 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c007b10 con 0x7f221415c700 2024-09-16T00:08:35.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.959+0000 7f22191d8640 1 -- 172.21.15.139:0/2539505877 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f221415c700 msgr2=0x7f2214164050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.959+0000 7f22191d8640 1 --2- 172.21.15.139:0/2539505877 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f221415c700 0x7f2214164050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f220c009dc0 tx=0x7f220c02a780 comp rx=0 tx=0).stop 2024-09-16T00:08:35.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.959+0000 7f22191d8640 1 -- 172.21.15.139:0/2539505877 shutdown_connections 2024-09-16T00:08:35.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.959+0000 7f22191d8640 1 --2- 172.21.15.139:0/2539505877 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f221415c700 0x7f2214164050 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.959+0000 7f22191d8640 1 -- 172.21.15.139:0/2539505877 >> 172.21.15.139:0/2539505877 conn(0x7f221406fb00 msgr2=0x7f22140738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:35.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.959+0000 7f22191d8640 1 -- 172.21.15.139:0/2539505877 shutdown_connections 2024-09-16T00:08:35.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.959+0000 7f22191d8640 1 -- 172.21.15.139:0/2539505877 wait complete. 2024-09-16T00:08:35.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.961+0000 7f22191d8640 1 Processor -- start 2024-09-16T00:08:35.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.961+0000 7f22191d8640 1 -- start start 2024-09-16T00:08:35.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.962+0000 7f22191d8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22140c72a0 0x7f2214158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.963+0000 7f22191d8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22140c8070 0x7f22140b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.963+0000 7f22191d8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f221415c700 0x7f22141563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.963+0000 7f2212d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22140c72a0 0x7f2214158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.963+0000 7f22191d8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f221415cb90 con 0x7f22140c72a0 2024-09-16T00:08:35.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.963+0000 7f22191d8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f221415c1e0 con 0x7f221415c700 2024-09-16T00:08:35.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.963+0000 7f22191d8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f22140c8450 con 0x7f22140c8070 2024-09-16T00:08:35.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.963+0000 7f2212d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22140c72a0 0x7f2214158530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43206/0 (socket says 172.21.15.139:43206) 2024-09-16T00:08:35.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.963+0000 7f2212d9d640 1 -- 172.21.15.139:0/3762149440 learned_addr learned my addr 172.21.15.139:0/3762149440 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:35.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.963+0000 7f221359e640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f221415c700 0x7f22141563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.963+0000 7f220bfff640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22140c8070 0x7f22140b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.963+0000 7f221359e640 1 -- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22140c8070 msgr2=0x7f22140b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.963+0000 7f221359e640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22140c8070 0x7f22140b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.963+0000 7f221359e640 1 -- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22140c72a0 msgr2=0x7f2214158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.964+0000 7f221359e640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22140c72a0 0x7f2214158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.964+0000 7f221359e640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2214158080 con 0x7f221415c700 2024-09-16T00:08:35.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.964+0000 7f2212d9d640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22140c72a0 0x7f2214158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:35.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.964+0000 7f221359e640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f221415c700 0x7f22141563a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2214073880 tx=0x7f220c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.964+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c042020 con 0x7f221415c700 2024-09-16T00:08:35.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.964+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220c0327f0 con 0x7f221415c700 2024-09-16T00:08:35.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.964+0000 7f22191d8640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22140c4d20 con 0x7f221415c700 2024-09-16T00:08:35.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.964+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c037040 con 0x7f221415c700 2024-09-16T00:08:35.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.964+0000 7f22191d8640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2214158fe0 con 0x7f221415c700 2024-09-16T00:08:35.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.965+0000 7f22191d8640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f22140cedc0 con 0x7f221415c700 2024-09-16T00:08:35.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.966+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f220c045020 con 0x7f221415c700 2024-09-16T00:08:35.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.966+0000 7f220b7fe640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f21f406c2b0 0x7f21f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.967+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5915+0+0 (secure 0 0 0) 0x7f220c0ac500 con 0x7f221415c700 2024-09-16T00:08:35.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.967+0000 7f2212d9d640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f21f406c2b0 0x7f21f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.967+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v45) ==== 54+0+16 (secure 0 0 0) 0x7f220c04c030 con 0x7f221415c700 2024-09-16T00:08:35.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.968+0000 7f2212d9d640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f21f406c2b0 0x7f21f406e770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f22040011e0 tx=0x7f2204009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.972+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f21ec001650 0x7f21ec003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.972+0000 7f220bfff640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f21ec001650 0x7f21ec003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.972+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f21ec006bb0 con 0x7f21ec001650 2024-09-16T00:08:35.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.973+0000 7f220bfff640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f21ec001650 0x7f21ec003b10 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.974+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21ec006bb0 con 0x7f21ec001650 2024-09-16T00:08:35.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.988+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f21ec005c60 con 0x7f21ec001650 2024-09-16T00:08:35.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.991+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f21ec005c60 con 0x7f21ec001650 2024-09-16T00:08:35.064 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:35.065 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:35.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.991+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f21ec004190 0x7f21ec004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.991+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f21ec004c00 con 0x7f21ec004190 2024-09-16T00:08:35.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.992+0000 7f2212d9d640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f21ec004190 0x7f21ec004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.992+0000 7f2212d9d640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f21ec004190 0x7f21ec004640 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:34.993+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21ec004c00 con 0x7f21ec004190 2024-09-16T00:08:35.066 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:35.053+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.014+0000 7fc45979f640 1 Processor -- start 2024-09-16T00:08:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.014+0000 7fc45979f640 1 -- start start 2024-09-16T00:08:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.015+0000 7fc45979f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc454158050 0x7fc45406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.015+0000 7fc45979f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc454070c60 0x7fc4540ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.015+0000 7fc45979f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc45406d390 con 0x7fc45415b490 2024-09-16T00:08:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.015+0000 7fc45979f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc4540710b0 con 0x7fc454070c60 2024-09-16T00:08:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.015+0000 7fc45979f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc45406cdd0 con 0x7fc454158050 2024-09-16T00:08:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.015+0000 7fc452ffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc454158050 0x7fc45406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.015+0000 7fc4527fc640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fc45415b490 0x7fc4540c93b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:59822/0 (socket says 172.21.15.139:59822) 2024-09-16T00:08:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.015+0000 7fc452ffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc454158050 0x7fc45406e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:60108/0 (socket says 172.21.15.139:60108) 2024-09-16T00:08:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.015+0000 7fc4527fc640 1 -- 172.21.15.139:0/392193567 learned_addr learned my addr 172.21.15.139:0/392193567 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.015+0000 7fc4537fe640 1 --2- 172.21.15.139:0/392193567 >> v2:172.21.15.139:3300/0 conn(0x7fc454070c60 0x7fc4540ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.015+0000 7fc4537fe640 1 -- 172.21.15.139:0/392193567 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc454158050 msgr2=0x7fc45406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.015+0000 7fc4537fe640 1 --2- 172.21.15.139:0/392193567 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc454158050 0x7fc45406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.015+0000 7fc4537fe640 1 -- 172.21.15.139:0/392193567 >> v1:172.21.15.164:6789/0 conn(0x7fc45415b490 legacy=0x7fc4540c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.015+0000 7fc4537fe640 1 -- 172.21.15.139:0/392193567 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45406d0b0 con 0x7fc454070c60 2024-09-16T00:08:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.016+0000 7fc452ffd640 1 --2- 172.21.15.139:0/392193567 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc454158050 0x7fc45406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:08:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.016+0000 7fc4537fe640 1 --2- 172.21.15.139:0/392193567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc454070c60 0x7fc4540ce1e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009d90 tx=0x7fc44c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8803993028702eef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.017+0000 7fc451ffb640 1 -- 172.21.15.139:0/392193567 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c007740 con 0x7fc454070c60 2024-09-16T00:08:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.017+0000 7fc451ffb640 1 -- 172.21.15.139:0/392193567 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc44c0078e0 con 0x7fc454070c60 2024-09-16T00:08:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.017+0000 7fc451ffb640 1 -- 172.21.15.139:0/392193567 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c007cf0 con 0x7fc454070c60 2024-09-16T00:08:35.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.018+0000 7fc45979f640 1 -- 172.21.15.139:0/392193567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc454070c60 msgr2=0x7fc4540ce1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.018+0000 7fc45979f640 1 --2- 172.21.15.139:0/392193567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc454070c60 0x7fc4540ce1e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc44c009d90 tx=0x7fc44c02a680 comp rx=0 tx=0).stop 2024-09-16T00:08:35.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.019+0000 7fc45979f640 1 -- 172.21.15.139:0/392193567 shutdown_connections 2024-09-16T00:08:35.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.019+0000 7fc45979f640 1 --2- 172.21.15.139:0/392193567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc454070c60 0x7fc4540ce1e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.019+0000 7fc45979f640 1 --2- 172.21.15.139:0/392193567 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc454158050 0x7fc45406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.019+0000 7fc45979f640 1 -- 172.21.15.139:0/392193567 >> 172.21.15.139:0/392193567 conn(0x7fc45406bd00 msgr2=0x7fc454067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:35.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.019+0000 7fc45979f640 1 -- 172.21.15.139:0/392193567 shutdown_connections 2024-09-16T00:08:35.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.019+0000 7fc45979f640 1 -- 172.21.15.139:0/392193567 wait complete. 2024-09-16T00:08:35.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.021+0000 7fc45979f640 1 Processor -- start 2024-09-16T00:08:35.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.021+0000 7fc45979f640 1 -- start start 2024-09-16T00:08:35.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.021+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f21ec0051a0 con 0x7f21ec004190 2024-09-16T00:08:35.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.022+0000 7fc45979f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc454070c60 0x7fc4540c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.022+0000 7fc45979f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc454158050 0x7fc4541f1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.022+0000 7fc45979f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc45415b490 0x7fc4540c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.023+0000 7fc452ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc454070c60 0x7fc4540c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.023+0000 7fc452ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc454070c60 0x7fc4540c5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43224/0 (socket says 172.21.15.139:43224) 2024-09-16T00:08:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.023+0000 7fc452ffd640 1 -- 172.21.15.139:0/1718521705 learned_addr learned my addr 172.21.15.139:0/1718521705 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.023+0000 7fc45979f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc454158430 con 0x7fc454070c60 2024-09-16T00:08:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.023+0000 7fc45979f640 1 -- 172.21.15.139:0/1718521705 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc4540705d0 con 0x7fc45415b490 2024-09-16T00:08:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.023+0000 7fc4537fe640 1 --2- 172.21.15.139:0/1718521705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc45415b490 0x7fc4540c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.023+0000 7fc4527fc640 1 --2- 172.21.15.139:0/1718521705 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc454158050 0x7fc4541f1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.023+0000 7fc45979f640 1 -- 172.21.15.139:0/1718521705 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc4540ca140 con 0x7fc454158050 2024-09-16T00:08:35.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.023+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f21ec0051a0 con 0x7f21ec004190 2024-09-16T00:08:35.073 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:35.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.023+0000 7fc4527fc640 1 -- 172.21.15.139:0/1718521705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc45415b490 msgr2=0x7fc4540c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.023+0000 7fc4527fc640 1 --2- 172.21.15.139:0/1718521705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc45415b490 0x7fc4540c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.023+0000 7fc4527fc640 1 -- 172.21.15.139:0/1718521705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc454070c60 msgr2=0x7fc4540c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.023+0000 7fc4527fc640 1 --2- 172.21.15.139:0/1718521705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc454070c60 0x7fc4540c5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.023+0000 7fc4527fc640 1 -- 172.21.15.139:0/1718521705 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4540b2430 con 0x7fc454158050 2024-09-16T00:08:35.074 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:35.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.024+0000 7fc4527fc640 1 --2- 172.21.15.139:0/1718521705 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc454158050 0x7fc4541f1370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc44800c390 tx=0x7fc44800c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.024+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f21ec009370 0x7f21ec00b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.024+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f21ec00e720 con 0x7f21ec009370 2024-09-16T00:08:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.024+0000 7fc43b7fe640 1 -- 172.21.15.139:0/1718521705 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448017070 con 0x7fc454158050 2024-09-16T00:08:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.024+0000 7f220bfff640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f21ec009370 0x7f21ec00b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.024+0000 7fc45979f640 1 -- 172.21.15.139:0/1718521705 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4540c7760 con 0x7fc454158050 2024-09-16T00:08:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.024+0000 7fc43b7fe640 1 -- 172.21.15.139:0/1718521705 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc448005ce0 con 0x7fc454158050 2024-09-16T00:08:35.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.024+0000 7fc43b7fe640 1 -- 172.21.15.139:0/1718521705 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4480072d0 con 0x7fc454158050 2024-09-16T00:08:35.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.024+0000 7fc45979f640 1 -- 172.21.15.139:0/1718521705 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4540b26a0 con 0x7fc454158050 2024-09-16T00:08:35.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.024+0000 7f220bfff640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f21ec009370 0x7f21ec00b760 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.025+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21ec00e720 con 0x7f21ec009370 2024-09-16T00:08:35.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.025+0000 7fc45979f640 1 -- 172.21.15.139:0/1718521705 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc45406d590 con 0x7fc454158050 2024-09-16T00:08:35.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.026+0000 7fc43b7fe640 1 -- 172.21.15.139:0/1718521705 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc44801a070 con 0x7fc454158050 2024-09-16T00:08:35.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.026+0000 7fc43b7fe640 1 --2- 172.21.15.139:0/1718521705 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc44006c2b0 0x7fc44006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.027+0000 7fc43b7fe640 1 -- 172.21.15.139:0/1718521705 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(45..45 src has 1..45) ==== 5915+0+0 (secure 0 0 0) 0x7fc44808a690 con 0x7fc454158050 2024-09-16T00:08:35.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.027+0000 7fc452ffd640 1 --2- 172.21.15.139:0/1718521705 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc44006c2b0 0x7fc44006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.027+0000 7fc452ffd640 1 --2- 172.21.15.139:0/1718521705 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc44006c2b0 0x7fc44006e770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc444009b20 tx=0x7fc444008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.031+0000 7fc43b7fe640 1 -- 172.21.15.139:0/1718521705 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc45406d590 con 0x7fc454158050 2024-09-16T00:08:35.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.053+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f21ec00d730 con 0x7f21ec009370 2024-09-16T00:08:35.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.055+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f21ec00d730 con 0x7f21ec009370 2024-09-16T00:08:35.078 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:35.079 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:35.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.056+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f21ec00df80 0x7f21ec00bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.056+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f21ec012070 con 0x7f21ec00df80 2024-09-16T00:08:35.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.056+0000 7f221359e640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f21ec00df80 0x7f21ec00bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.056+0000 7f221359e640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f21ec00df80 0x7f21ec00bd20 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.057+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21ec012070 con 0x7f21ec00df80 2024-09-16T00:08:35.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.083+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f21ec00c5f0 con 0x7f21ec00df80 2024-09-16T00:08:35.087 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:35.087 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:35.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.086+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f21ec00c5f0 con 0x7f21ec00df80 2024-09-16T00:08:35.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.086+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f21ec00cd10 0x7f21ec0108e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.087+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f21ec00d150 con 0x7f21ec00cd10 2024-09-16T00:08:35.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.087+0000 7f2212d9d640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f21ec00cd10 0x7f21ec0108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.088+0000 7f2212d9d640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f21ec00cd10 0x7f21ec0108e0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.088+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21ec00d150 con 0x7f21ec00cd10 2024-09-16T00:08:35.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.114+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f21ec00d150 con 0x7f21ec00cd10 2024-09-16T00:08:35.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.116+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f21ec00d150 con 0x7f21ec00cd10 2024-09-16T00:08:35.117 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:35.118 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:35.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.117+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f21ec0113e0 0x7f21ec011800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.117+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f21ec018b60 con 0x7f21ec0113e0 2024-09-16T00:08:35.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.117+0000 7f220bfff640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f21ec0113e0 0x7f21ec011800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.118+0000 7f220bfff640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f21ec0113e0 0x7f21ec011800 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.119+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21ec018b60 con 0x7f21ec0113e0 2024-09-16T00:08:35.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.146+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f21ec0162d0 con 0x7f21ec0113e0 2024-09-16T00:08:35.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.148+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f21ec0162d0 con 0x7f21ec0113e0 2024-09-16T00:08:35.149 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:35.149 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:35.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.148+0000 7f22191d8640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f21ec018100 0x7f21ec018540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.148+0000 7f22191d8640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f21ec01d660 con 0x7f21ec018100 2024-09-16T00:08:35.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.149+0000 7f221359e640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f21ec018100 0x7f21ec018540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.149+0000 7f221359e640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f21ec018100 0x7f21ec018540 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.150+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21ec01d660 con 0x7f21ec018100 2024-09-16T00:08:35.154 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:35.153+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:35.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.176+0000 7f22191d8640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f21ec01add0 con 0x7f21ec018100 2024-09-16T00:08:35.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.178+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f21ec01add0 con 0x7f21ec018100 2024-09-16T00:08:35.179 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:35.180 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:35.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.179+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f21ec0169f0 0x7f21ec016e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.179+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f21ec017480 con 0x7f21ec0169f0 2024-09-16T00:08:35.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.179+0000 7f2212d9d640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f21ec0169f0 0x7f21ec016e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.180+0000 7f2212d9d640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f21ec0169f0 0x7f21ec016e50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.181+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f21ec017480 con 0x7f21ec0169f0 2024-09-16T00:08:35.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.208+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f21ec000950 con 0x7f21ec0169f0 2024-09-16T00:08:35.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f220b7fe640 1 -- 172.21.15.139:0/3762149440 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f21ec000950 con 0x7f21ec0169f0 2024-09-16T00:08:35.211 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:35.211 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:35.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f21ec001650 msgr2=0x7f21ec003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f21ec001650 0x7f21ec003b10 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f21ec004190 msgr2=0x7f21ec004640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f21ec004190 0x7f21ec004640 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f21ec009370 msgr2=0x7f21ec00b760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f21ec009370 0x7f21ec00b760 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f21ec00df80 msgr2=0x7f21ec00bd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f21ec00df80 0x7f21ec00bd20 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f21ec00cd10 msgr2=0x7f21ec0108e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f21ec00cd10 0x7f21ec0108e0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f21ec0113e0 msgr2=0x7f21ec011800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f21ec0113e0 0x7f21ec011800 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f21ec018100 msgr2=0x7f21ec018540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f21ec018100 0x7f21ec018540 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f21ec0169f0 msgr2=0x7f21ec016e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f21ec0169f0 0x7f21ec016e50 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f221359e640 1 -- 172.21.15.139:0/3762149440 reap_dead start 2024-09-16T00:08:35.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f221359e640 1 -- 172.21.15.139:0/3762149440 reap_dead start 2024-09-16T00:08:35.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f221359e640 1 -- 172.21.15.139:0/3762149440 reap_dead start 2024-09-16T00:08:35.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f221359e640 1 -- 172.21.15.139:0/3762149440 reap_dead start 2024-09-16T00:08:35.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.210+0000 7f221359e640 1 -- 172.21.15.139:0/3762149440 reap_dead start 2024-09-16T00:08:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.211+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f21f406c2b0 msgr2=0x7f21f406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.211+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f21f406c2b0 0x7f21f406e770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f22040011e0 tx=0x7f2204009040 comp rx=0 tx=0).stop 2024-09-16T00:08:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.211+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f221415c700 msgr2=0x7f22141563a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.211+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f221415c700 0x7f22141563a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2214073880 tx=0x7f220c004a40 comp rx=0 tx=0).stop 2024-09-16T00:08:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.211+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 shutdown_connections 2024-09-16T00:08:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.211+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f21f406c2b0 0x7f21f406e770 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f22040011e0 tx=0x7f2204009040 comp rx=0 tx=0).stop 2024-09-16T00:08:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.211+0000 7f22097fa640 1 --2- 172.21.15.139:0/3762149440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f221415c700 0x7f22141563a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.211+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 >> 172.21.15.139:0/3762149440 conn(0x7f221406fb00 msgr2=0x7f221406d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.212+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 shutdown_connections 2024-09-16T00:08:35.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.212+0000 7f22097fa640 1 -- 172.21.15.139:0/3762149440 wait complete. 2024-09-16T00:08:35.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.235+0000 7fcbd176d640 1 Processor -- start 2024-09-16T00:08:35.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.235+0000 7fcbd176d640 1 -- start start 2024-09-16T00:08:35.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.236+0000 7fcbd176d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcbcc0cc6f0 0x7fcbcc072660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.236+0000 7fcbd176d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcbcc162220 0x7fcbcc160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.236+0000 7fcbd176d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbcc072f60 con 0x7fcbcc0ccad0 2024-09-16T00:08:35.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.236+0000 7fcbd176d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fcbcc073240 con 0x7fcbcc0cc6f0 2024-09-16T00:08:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.236+0000 7fcbd176d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcbcc073520 con 0x7fcbcc162220 2024-09-16T00:08:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.236+0000 7fcbcaffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcbcc0cc6f0 0x7fcbcc072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.237+0000 7fcbcaffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcbcc0cc6f0 0x7fcbcc072660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:34240/0 (socket says 172.21.15.139:34240) 2024-09-16T00:08:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.237+0000 7fcbcaffd640 1 -- 172.21.15.139:0/1074704785 learned_addr learned my addr 172.21.15.139:0/1074704785 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:35.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.237+0000 7fcbcb7fe640 1 --2- 172.21.15.139:0/1074704785 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcbcc162220 0x7fcbcc160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.237+0000 7fcbcaffd640 1 -- 172.21.15.139:0/1074704785 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcbcc162220 msgr2=0x7fcbcc160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:35.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.237+0000 7fcbcaffd640 1 --2- 172.21.15.139:0/1074704785 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcbcc162220 0x7fcbcc160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.237+0000 7fcbcaffd640 1 -- 172.21.15.139:0/1074704785 >> v1:172.21.15.139:6789/0 conn(0x7fcbcc0ccad0 legacy=0x7fcbcc162890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.237+0000 7fcbcaffd640 1 -- 172.21.15.139:0/1074704785 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbcc1602c0 con 0x7fcbcc0cc6f0 2024-09-16T00:08:35.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.238+0000 7fcbcaffd640 1 --2- 172.21.15.139:0/1074704785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc0cc6f0 0x7fcbcc072660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009870 tx=0x7fcbbc02a2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d37f2d9f371f380a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.238+0000 7fcbc9ffb640 1 -- 172.21.15.139:0/1074704785 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc037070 con 0x7fcbcc0cc6f0 2024-09-16T00:08:35.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.238+0000 7fcbc9ffb640 1 -- 172.21.15.139:0/1074704785 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbbc032640 con 0x7fcbcc0cc6f0 2024-09-16T00:08:35.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.238+0000 7fcbc9ffb640 1 -- 172.21.15.139:0/1074704785 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc032a10 con 0x7fcbcc0cc6f0 2024-09-16T00:08:35.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.239+0000 7fcbd176d640 1 -- 172.21.15.139:0/1074704785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc0cc6f0 msgr2=0x7fcbcc072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.239+0000 7fcbd176d640 1 --2- 172.21.15.139:0/1074704785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc0cc6f0 0x7fcbcc072660 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009870 tx=0x7fcbbc02a2f0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.239+0000 7fcbd176d640 1 -- 172.21.15.139:0/1074704785 shutdown_connections 2024-09-16T00:08:35.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.239+0000 7fcbd176d640 1 --2- 172.21.15.139:0/1074704785 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcbcc162220 0x7fcbcc160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.239+0000 7fcbd176d640 1 --2- 172.21.15.139:0/1074704785 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc0cc6f0 0x7fcbcc072660 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.239+0000 7fcbd176d640 1 -- 172.21.15.139:0/1074704785 >> 172.21.15.139:0/1074704785 conn(0x7fcbcc06bb20 msgr2=0x7fcbcc06bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:35.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.239+0000 7fcbd176d640 1 -- 172.21.15.139:0/1074704785 shutdown_connections 2024-09-16T00:08:35.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.239+0000 7fcbd176d640 1 -- 172.21.15.139:0/1074704785 wait complete. 2024-09-16T00:08:35.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.240+0000 7fcbd176d640 1 Processor -- start 2024-09-16T00:08:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.240+0000 7fcbd176d640 1 -- start start 2024-09-16T00:08:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.241+0000 7fcbd176d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc0cc6f0 0x7fcbcc15f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.241+0000 7fcbd176d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcbcc0ccad0 0x7fcbcc0c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.241+0000 7fcbd176d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbcc162220 0x7fcbcc1f7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.241+0000 7fcbd176d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc072ba0 con 0x7fcbcc162220 2024-09-16T00:08:35.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.241+0000 7fcbd176d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc0c7160 con 0x7fcbcc0cc6f0 2024-09-16T00:08:35.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.241+0000 7fcbd176d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcbcc072d20 con 0x7fcbcc0ccad0 2024-09-16T00:08:35.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.242+0000 7fcbcb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbcc162220 0x7fcbcc1f7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.242+0000 7fcbcb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbcc162220 0x7fcbcc1f7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43226/0 (socket says 172.21.15.139:43226) 2024-09-16T00:08:35.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.242+0000 7fcbcb7fe640 1 -- 172.21.15.139:0/2244617898 learned_addr learned my addr 172.21.15.139:0/2244617898 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:35.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.242+0000 7fcbca7fc640 1 --2- 172.21.15.139:0/2244617898 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcbcc0ccad0 0x7fcbcc0c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.242+0000 7fcbcaffd640 1 --2- 172.21.15.139:0/2244617898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc0cc6f0 0x7fcbcc15f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.242+0000 7fcbcb7fe640 1 -- 172.21.15.139:0/2244617898 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcbcc0ccad0 msgr2=0x7fcbcc0c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.242+0000 7fcbcb7fe640 1 --2- 172.21.15.139:0/2244617898 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcbcc0ccad0 0x7fcbcc0c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.242+0000 7fcbcb7fe640 1 -- 172.21.15.139:0/2244617898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc0cc6f0 msgr2=0x7fcbcc15f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.242+0000 7fcbcb7fe640 1 --2- 172.21.15.139:0/2244617898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc0cc6f0 0x7fcbcc15f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.242+0000 7fcbcb7fe640 1 -- 172.21.15.139:0/2244617898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbcc15ec60 con 0x7fcbcc162220 2024-09-16T00:08:35.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.242+0000 7fcbcb7fe640 1 --2- 172.21.15.139:0/2244617898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbcc162220 0x7fcbcc1f7860 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcbc400efc0 tx=0x7fcbc400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.243+0000 7fcbc37fe640 1 -- 172.21.15.139:0/2244617898 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4018020 con 0x7fcbcc162220 2024-09-16T00:08:35.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.243+0000 7fcbc37fe640 1 -- 172.21.15.139:0/2244617898 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc4014480 con 0x7fcbcc162220 2024-09-16T00:08:35.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.243+0000 7fcbc37fe640 1 -- 172.21.15.139:0/2244617898 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4014870 con 0x7fcbcc162220 2024-09-16T00:08:35.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.243+0000 7fcbd176d640 1 -- 172.21.15.139:0/2244617898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbcc15ef40 con 0x7fcbcc162220 2024-09-16T00:08:35.253 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:35.253 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:35.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.243+0000 7fcbd176d640 1 -- 172.21.15.139:0/2244617898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbcc0c3430 con 0x7fcbcc162220 2024-09-16T00:08:35.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.245+0000 7fcbd176d640 1 -- 172.21.15.139:0/2244617898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbcc0c3630 con 0x7fcbcc162220 2024-09-16T00:08:35.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.245+0000 7fcbc37fe640 1 -- 172.21.15.139:0/2244617898 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcbc401b020 con 0x7fcbcc162220 2024-09-16T00:08:35.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.246+0000 7fcbc37fe640 1 --2- 172.21.15.139:0/2244617898 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcbac06c300 0x7fcbac06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:35.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.246+0000 7fcbc37fe640 1 -- 172.21.15.139:0/2244617898 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5915+0+0 (secure 0 0 0) 0x7fcbc408bec0 con 0x7fcbcc162220 2024-09-16T00:08:35.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.247+0000 7fcbcaffd640 1 --2- 172.21.15.139:0/2244617898 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcbac06c300 0x7fcbac06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:35.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.247+0000 7fcbcaffd640 1 --2- 172.21.15.139:0/2244617898 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcbac06c300 0x7fcbac06e7c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc034000 tx=0x7fcbbc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:35.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.251+0000 7fcbc37fe640 1 -- 172.21.15.139:0/2244617898 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcbcc0c3630 con 0x7fcbcc162220 2024-09-16T00:08:35.258 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:08:35.259 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:35.257+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:35.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.354+0000 7fc45979f640 1 -- 172.21.15.139:0/1718521705 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) -- 0x7fc454157610 con 0x7fc454158050 2024-09-16T00:08:35.359 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:35.358+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:35.460 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:35.459+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:35.475 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:35.475 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:35.488 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:08:35.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.533+0000 7fcbd176d640 1 -- 172.21.15.139:0/2244617898 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fcbcc06c030 con 0x7fcbcc162220 2024-09-16T00:08:35.561 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:35.559+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:35.619 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:35.619 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:35.628 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:08:35.674 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:35.673+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:35.737 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:35.737 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:35.746 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:08:35.788 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:35.787+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:35.863 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:35.862+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:35.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.877+0000 7fcbc37fe640 1 -- 172.21.15.139:0/2244617898 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v46) ==== 91+0+0 (secure 0 0 0) 0x7fcbc4022030 con 0x7fcbcc162220 2024-09-16T00:08:35.879 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is unset 2024-09-16T00:08:35.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.883+0000 7fc43b7fe640 1 -- 172.21.15.139:0/1718521705 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v46) ==== 113+0+0 (secure 0 0 0) 0x7fc4480586d0 con 0x7fc454158050 2024-09-16T00:08:35.887 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T00:08:35.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.881+0000 7fcbd176d640 1 -- 172.21.15.139:0/2244617898 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcbac06c300 msgr2=0x7fcbac06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.881+0000 7fcbd176d640 1 --2- 172.21.15.139:0/2244617898 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcbac06c300 0x7fcbac06e7c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc034000 tx=0x7fcbbc02e000 comp rx=0 tx=0).stop 2024-09-16T00:08:35.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.881+0000 7fcbd176d640 1 -- 172.21.15.139:0/2244617898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbcc162220 msgr2=0x7fcbcc1f7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.881+0000 7fcbd176d640 1 --2- 172.21.15.139:0/2244617898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbcc162220 0x7fcbcc1f7860 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcbc400efc0 tx=0x7fcbc400c490 comp rx=0 tx=0).stop 2024-09-16T00:08:35.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.881+0000 7fcbd176d640 1 -- 172.21.15.139:0/2244617898 shutdown_connections 2024-09-16T00:08:35.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.881+0000 7fcbd176d640 1 --2- 172.21.15.139:0/2244617898 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcbac06c300 0x7fcbac06e7c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.881+0000 7fcbd176d640 1 --2- 172.21.15.139:0/2244617898 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbcc162220 0x7fcbcc1f7860 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.881+0000 7fcbd176d640 1 --2- 172.21.15.139:0/2244617898 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcbcc0ccad0 0x7fcbcc0c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.881+0000 7fcbd176d640 1 --2- 172.21.15.139:0/2244617898 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcbcc0cc6f0 0x7fcbcc15f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.881+0000 7fcbd176d640 1 -- 172.21.15.139:0/2244617898 >> 172.21.15.139:0/2244617898 conn(0x7fcbcc06bb20 msgr2=0x7fcbcc0c91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:35.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.886+0000 7fcbd176d640 1 -- 172.21.15.139:0/2244617898 shutdown_connections 2024-09-16T00:08:35.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.886+0000 7fcbd176d640 1 -- 172.21.15.139:0/2244617898 wait complete. 2024-09-16T00:08:35.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.887+0000 7fc45979f640 1 -- 172.21.15.139:0/1718521705 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc44006c2b0 msgr2=0x7fc44006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.887+0000 7fc45979f640 1 --2- 172.21.15.139:0/1718521705 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc44006c2b0 0x7fc44006e770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc444009b20 tx=0x7fc444008040 comp rx=0 tx=0).stop 2024-09-16T00:08:35.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.887+0000 7fc45979f640 1 -- 172.21.15.139:0/1718521705 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc454158050 msgr2=0x7fc4541f1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:35.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.887+0000 7fc45979f640 1 --2- 172.21.15.139:0/1718521705 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc454158050 0x7fc4541f1370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc44800c390 tx=0x7fc44800c790 comp rx=0 tx=0).stop 2024-09-16T00:08:35.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.888+0000 7fc45979f640 1 -- 172.21.15.139:0/1718521705 shutdown_connections 2024-09-16T00:08:35.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.888+0000 7fc45979f640 1 --2- 172.21.15.139:0/1718521705 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc44006c2b0 0x7fc44006e770 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.888+0000 7fc45979f640 1 --2- 172.21.15.139:0/1718521705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc45415b490 0x7fc4540c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.888+0000 7fc45979f640 1 --2- 172.21.15.139:0/1718521705 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc454158050 0x7fc4541f1370 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.888+0000 7fc45979f640 1 --2- 172.21.15.139:0/1718521705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc454070c60 0x7fc4540c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:35.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.888+0000 7fc45979f640 1 -- 172.21.15.139:0/1718521705 >> 172.21.15.139:0/1718521705 conn(0x7fc45406bd00 msgr2=0x7fc4540c46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:35.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.888+0000 7fc45979f640 1 -- 172.21.15.139:0/1718521705 shutdown_connections 2024-09-16T00:08:35.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:35.888+0000 7fc45979f640 1 -- 172.21.15.139:0/1718521705 wait complete. 2024-09-16T00:08:35.910 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T00:08:35.932 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:35.932 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:35.942 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:08:35.964 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:35.963+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:36.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.016+0000 7f70bf0dd640 1 Processor -- start 2024-09-16T00:08:36.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.016+0000 7f70bf0dd640 1 -- start start 2024-09-16T00:08:36.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.017+0000 7f70bf0dd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f70b8154ae0 0x7f70b806bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:36.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.017+0000 7f70bf0dd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f70b815b460 0x7f70b8072eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:36.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.017+0000 7f70bf0dd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70b815a9f0 con 0x7f70b80c7140 2024-09-16T00:08:36.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.017+0000 7f70bf0dd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f70b80c8860 con 0x7f70b8154ae0 2024-09-16T00:08:36.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.017+0000 7f70bf0dd640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f70b815afd0 con 0x7f70b815b460 2024-09-16T00:08:36.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.017+0000 7f70bce79640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f70b8154ae0 0x7f70b806bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:36.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.017+0000 7f70b7fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f70b80c7140 0x7f70b80727a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:50304/0 (socket says 172.21.15.139:50304) 2024-09-16T00:08:36.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.017+0000 7f70b7fff640 1 -- 172.21.15.139:0/2893823292 learned_addr learned my addr 172.21.15.139:0/2893823292 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:36.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.018+0000 7f70bd67a640 1 --2- 172.21.15.139:0/2893823292 >> v2:172.21.15.164:3300/0 conn(0x7f70b815b460 0x7f70b8072eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:36.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.018+0000 7f70bd67a640 1 -- 172.21.15.139:0/2893823292 >> v2:172.21.15.139:3300/0 conn(0x7f70b8154ae0 msgr2=0x7f70b806bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:36.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.018+0000 7f70bd67a640 1 --2- 172.21.15.139:0/2893823292 >> v2:172.21.15.139:3300/0 conn(0x7f70b8154ae0 0x7f70b806bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:36.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.018+0000 7f70bd67a640 1 -- 172.21.15.139:0/2893823292 >> v1:172.21.15.139:6789/0 conn(0x7f70b80c7140 legacy=0x7f70b80727a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:36.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.018+0000 7f70bd67a640 1 -- 172.21.15.139:0/2893823292 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b815b8b0 con 0x7f70b815b460 2024-09-16T00:08:36.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.019+0000 7f70bd67a640 1 --2- 172.21.15.139:0/2893823292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70b815b460 0x7f70b8072eb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f70b80c3900 tx=0x7f70b002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55c766d25542ceef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:36.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.019+0000 7f70b77fe640 1 -- 172.21.15.139:0/2893823292 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0007740 con 0x7f70b815b460 2024-09-16T00:08:36.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.019+0000 7f70b77fe640 1 -- 172.21.15.139:0/2893823292 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70b00078e0 con 0x7f70b815b460 2024-09-16T00:08:36.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.019+0000 7f70b77fe640 1 -- 172.21.15.139:0/2893823292 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0007cb0 con 0x7f70b815b460 2024-09-16T00:08:36.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.020+0000 7f70bf0dd640 1 -- 172.21.15.139:0/2893823292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70b815b460 msgr2=0x7f70b8072eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:36.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.020+0000 7f70bf0dd640 1 --2- 172.21.15.139:0/2893823292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70b815b460 0x7f70b8072eb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f70b80c3900 tx=0x7f70b002a710 comp rx=0 tx=0).stop 2024-09-16T00:08:36.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.020+0000 7f70bf0dd640 1 -- 172.21.15.139:0/2893823292 shutdown_connections 2024-09-16T00:08:36.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.020+0000 7f70bf0dd640 1 --2- 172.21.15.139:0/2893823292 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70b815b460 0x7f70b8072eb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:36.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.020+0000 7f70bf0dd640 1 --2- 172.21.15.139:0/2893823292 >> v2:172.21.15.139:3300/0 conn(0x7f70b8154ae0 0x7f70b806bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:36.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.020+0000 7f70bf0dd640 1 -- 172.21.15.139:0/2893823292 >> 172.21.15.139:0/2893823292 conn(0x7f70b80cc740 msgr2=0x7f70b80ccb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:36.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.020+0000 7f70bf0dd640 1 -- 172.21.15.139:0/2893823292 shutdown_connections 2024-09-16T00:08:36.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.020+0000 7f70bf0dd640 1 -- 172.21.15.139:0/2893823292 wait complete. 2024-09-16T00:08:36.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.022+0000 7f70bf0dd640 1 Processor -- start 2024-09-16T00:08:36.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.022+0000 7f70bf0dd640 1 -- start start 2024-09-16T00:08:36.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.023+0000 7f70bf0dd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70b80c7140 0x7f70b8157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:36.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.023+0000 7f70bf0dd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70b8154ae0 0x7f70b8157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:36.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.023+0000 7f70bf0dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70b815b460 0x7f70b81553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:36.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.023+0000 7f70bf0dd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f70b815ac60 con 0x7f70b815b460 2024-09-16T00:08:36.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.023+0000 7f70bf0dd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f70b80c7550 con 0x7f70b8154ae0 2024-09-16T00:08:36.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.023+0000 7f70bf0dd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f70b806f600 con 0x7f70b80c7140 2024-09-16T00:08:36.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.023+0000 7f70bd67a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70b815b460 0x7f70b81553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:36.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.023+0000 7f70bd67a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70b815b460 0x7f70b81553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43238/0 (socket says 172.21.15.139:43238) 2024-09-16T00:08:36.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.023+0000 7f70bd67a640 1 -- 172.21.15.139:0/3102302652 learned_addr learned my addr 172.21.15.139:0/3102302652 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:36.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.023+0000 7f70bce79640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70b80c7140 0x7f70b8157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:36.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.023+0000 7f70b7fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70b8154ae0 0x7f70b8157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:36.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.024+0000 7f70bce79640 1 -- 172.21.15.139:0/3102302652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70b8154ae0 msgr2=0x7f70b8157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:36.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.024+0000 7f70bce79640 1 --2- 172.21.15.139:0/3102302652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70b8154ae0 0x7f70b8157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:36.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.024+0000 7f70bce79640 1 -- 172.21.15.139:0/3102302652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70b815b460 msgr2=0x7f70b81553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:36.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.024+0000 7f70bce79640 1 --2- 172.21.15.139:0/3102302652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70b815b460 0x7f70b81553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:36.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.024+0000 7f70bce79640 1 -- 172.21.15.139:0/3102302652 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b81f0f50 con 0x7f70b80c7140 2024-09-16T00:08:36.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.024+0000 7f70bd67a640 1 --2- 172.21.15.139:0/3102302652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70b815b460 0x7f70b81553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:36.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.024+0000 7f70bce79640 1 --2- 172.21.15.139:0/3102302652 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70b80c7140 0x7f70b8157960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00e8a0 tx=0x7f70ac00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:36.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.024+0000 7f70b57fa640 1 -- 172.21.15.139:0/3102302652 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac017070 con 0x7f70b80c7140 2024-09-16T00:08:36.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.024+0000 7f70b57fa640 1 -- 172.21.15.139:0/3102302652 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70ac00c940 con 0x7f70b80c7140 2024-09-16T00:08:36.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.024+0000 7f70bf0dd640 1 -- 172.21.15.139:0/3102302652 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b81f1210 con 0x7f70b80c7140 2024-09-16T00:08:36.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.024+0000 7f70b57fa640 1 -- 172.21.15.139:0/3102302652 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ac009e60 con 0x7f70b80c7140 2024-09-16T00:08:36.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.024+0000 7f70bf0dd640 1 -- 172.21.15.139:0/3102302652 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b81f17d0 con 0x7f70b80c7140 2024-09-16T00:08:36.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.026+0000 7f70bf0dd640 1 -- 172.21.15.139:0/3102302652 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70b81f1e90 con 0x7f70b80c7140 2024-09-16T00:08:36.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.026+0000 7f70b57fa640 1 -- 172.21.15.139:0/3102302652 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f70ac01a070 con 0x7f70b80c7140 2024-09-16T00:08:36.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.026+0000 7f70b57fa640 1 --2- 172.21.15.139:0/3102302652 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f709c06c2b0 0x7f709c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:36.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.027+0000 7f70b57fa640 1 -- 172.21.15.139:0/3102302652 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 5915+0+0 (secure 0 0 0) 0x7f70ac08b600 con 0x7f70b80c7140 2024-09-16T00:08:36.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.027+0000 7f70b7fff640 1 --2- 172.21.15.139:0/3102302652 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f709c06c2b0 0x7f709c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:36.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.027+0000 7f70b7fff640 1 --2- 172.21.15.139:0/3102302652 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f709c06c2b0 0x7f709c06e770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f70b815efd0 tx=0x7f70a4007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:36.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.031+0000 7f70b57fa640 1 -- 172.21.15.139:0/3102302652 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f70b81f1e90 con 0x7f70b80c7140 2024-09-16T00:08:36.066 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:36.065+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:36.107 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:36.107 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:36.117 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:08:36.201 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:36.200+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:36.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.235+0000 7f70bf0dd640 1 -- 172.21.15.139:0/3102302652 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) -- 0x7f70b80c8d90 con 0x7f70b80c7140 2024-09-16T00:08:36.263 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:36.263 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:36.269 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:36.266+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:36.272 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:08:36.368 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:36.366+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:36.418 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:36.418 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:36.428 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:08:36.469 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:36.468+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:36.534 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:36.534 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:36.543 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:08:36.588 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:36.586+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:36.661 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:36.661 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:36.671 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:08:36.672 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:36.670+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:36.772 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:36.771+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:36.816 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:36.817 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:36.825 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:08:36.873 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:36.872+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:36.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.883+0000 7f70b57fa640 1 -- 172.21.15.139:0/3102302652 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v47) ==== 113+0+0 (secure 0 0 0) 0x7f70ac059650 con 0x7f70b80c7140 2024-09-16T00:08:36.885 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T00:08:36.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.887+0000 7f70bf0dd640 1 -- 172.21.15.139:0/3102302652 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f709c06c2b0 msgr2=0x7f709c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:36.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.887+0000 7f70bf0dd640 1 --2- 172.21.15.139:0/3102302652 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f709c06c2b0 0x7f709c06e770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f70b815efd0 tx=0x7f70a4007e80 comp rx=0 tx=0).stop 2024-09-16T00:08:36.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.887+0000 7f70bf0dd640 1 -- 172.21.15.139:0/3102302652 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70b80c7140 msgr2=0x7f70b8157960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:36.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.887+0000 7f70bf0dd640 1 --2- 172.21.15.139:0/3102302652 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70b80c7140 0x7f70b8157960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00e8a0 tx=0x7f70ac00ed60 comp rx=0 tx=0).stop 2024-09-16T00:08:36.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.891+0000 7f70bf0dd640 1 -- 172.21.15.139:0/3102302652 shutdown_connections 2024-09-16T00:08:36.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.891+0000 7f70bf0dd640 1 --2- 172.21.15.139:0/3102302652 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f709c06c2b0 0x7f709c06e770 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:36.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.891+0000 7f70bf0dd640 1 --2- 172.21.15.139:0/3102302652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70b815b460 0x7f70b81553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:36.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.891+0000 7f70bf0dd640 1 --2- 172.21.15.139:0/3102302652 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70b8154ae0 0x7f70b8157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:36.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.891+0000 7f70bf0dd640 1 --2- 172.21.15.139:0/3102302652 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70b80c7140 0x7f70b8157960 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:36.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.891+0000 7f70bf0dd640 1 -- 172.21.15.139:0/3102302652 >> 172.21.15.139:0/3102302652 conn(0x7f70b80cc740 msgr2=0x7f70b814c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:36.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.892+0000 7f70bf0dd640 1 -- 172.21.15.139:0/3102302652 shutdown_connections 2024-09-16T00:08:36.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:36.892+0000 7f70bf0dd640 1 -- 172.21.15.139:0/3102302652 wait complete. 2024-09-16T00:08:36.911 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T00:08:36.911 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:08:36.939 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:36.939 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:36.948 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:08:36.989 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:36.988+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:37.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.056+0000 7fab1f6d0640 1 Processor -- start 2024-09-16T00:08:37.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.056+0000 7fab1f6d0640 1 -- start start 2024-09-16T00:08:37.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.057+0000 7fab1f6d0640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fab180c8cc0 0x7fab1806fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.057+0000 7fab1f6d0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fab18070120 0x7fab18070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.057+0000 7fab1f6d0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab180cb640 con 0x7fab181528b0 2024-09-16T00:08:37.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.057+0000 7fab1f6d0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fab18152c90 con 0x7fab18070120 2024-09-16T00:08:37.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.057+0000 7fab1f6d0640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fab180c9990 con 0x7fab180c8cc0 2024-09-16T00:08:37.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.057+0000 7fab1d46c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fab181528b0 0x7fab180caf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:50316/0 (socket says 172.21.15.139:50316) 2024-09-16T00:08:37.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.057+0000 7fab1d46c640 1 -- 172.21.15.139:0/4191273623 learned_addr learned my addr 172.21.15.139:0/4191273623 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:37.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.057+0000 7fab1cc6b640 1 --2- 172.21.15.139:0/4191273623 >> v2:172.21.15.164:3300/0 conn(0x7fab180c8cc0 0x7fab1806fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.057+0000 7fab1dc6d640 1 --2- 172.21.15.139:0/4191273623 >> v2:172.21.15.139:3300/0 conn(0x7fab18070120 0x7fab18070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.057+0000 7fab0ffff640 1 -- 172.21.15.139:0/4191273623 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1487748185 0 0) 0x7fab180cb640 con 0x7fab181528b0 2024-09-16T00:08:37.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.057+0000 7fab0ffff640 1 -- 172.21.15.139:0/4191273623 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab04003520 con 0x7fab181528b0 2024-09-16T00:08:37.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.057+0000 7fab0ffff640 1 -- 172.21.15.139:0/4191273623 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 633041946 0 0) 0x7fab04003520 con 0x7fab181528b0 2024-09-16T00:08:37.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.057+0000 7fab0ffff640 1 -- 172.21.15.139:0/4191273623 >> v2:172.21.15.164:3300/0 conn(0x7fab180c8cc0 msgr2=0x7fab1806fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:37.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.057+0000 7fab0ffff640 1 --2- 172.21.15.139:0/4191273623 >> v2:172.21.15.164:3300/0 conn(0x7fab180c8cc0 0x7fab1806fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.057+0000 7fab0ffff640 1 -- 172.21.15.139:0/4191273623 >> v2:172.21.15.139:3300/0 conn(0x7fab18070120 msgr2=0x7fab18070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:37.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab0ffff640 1 --2- 172.21.15.139:0/4191273623 >> v2:172.21.15.139:3300/0 conn(0x7fab18070120 0x7fab18070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab0ffff640 1 -- 172.21.15.139:0/4191273623 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab180c32e0 con 0x7fab181528b0 2024-09-16T00:08:37.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab0ffff640 1 -- 172.21.15.139:0/4191273623 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7fab14001670 con 0x7fab181528b0 2024-09-16T00:08:37.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab0ffff640 1 -- 172.21.15.139:0/4191273623 >> v1:172.21.15.139:6789/0 conn(0x7fab181528b0 legacy=0x7fab180caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab0ffff640 1 --2- 172.21.15.139:0/4191273623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab04003a00 0x7fab04003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab0ffff640 1 --2- 172.21.15.139:0/4191273623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab04004800 0x7fab04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab0ffff640 1 --2- 172.21.15.139:0/4191273623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab040051c0 0x7fab0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab0ffff640 1 -- 172.21.15.139:0/4191273623 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fab18152c90 con 0x7fab04004800 2024-09-16T00:08:37.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab0ffff640 1 -- 172.21.15.139:0/4191273623 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab180c9990 con 0x7fab04003a00 2024-09-16T00:08:37.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab0ffff640 1 -- 172.21.15.139:0/4191273623 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fab0400bfb0 con 0x7fab040051c0 2024-09-16T00:08:37.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab1cc6b640 1 --2- 172.21.15.139:0/4191273623 >> v2:172.21.15.164:3300/0 conn(0x7fab180c8cc0 0x7fab1806fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:37.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab1dc6d640 1 --2- 172.21.15.139:0/4191273623 >> v2:172.21.15.139:3300/0 conn(0x7fab18070120 0x7fab18070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:37.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab1d46c640 1 --2- 172.21.15.139:0/4191273623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab04003a00 0x7fab04003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab1cc6b640 1 --2- 172.21.15.139:0/4191273623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab04004800 0x7fab04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab1dc6d640 1 --2- 172.21.15.139:0/4191273623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab040051c0 0x7fab0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab1d46c640 1 -- 172.21.15.139:0/4191273623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab040051c0 msgr2=0x7fab0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:37.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab1d46c640 1 --2- 172.21.15.139:0/4191273623 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab040051c0 0x7fab0400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab1d46c640 1 -- 172.21.15.139:0/4191273623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab04004800 msgr2=0x7fab04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:37.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab1d46c640 1 --2- 172.21.15.139:0/4191273623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab04004800 0x7fab04004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab1d46c640 1 -- 172.21.15.139:0/4191273623 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab04003520 con 0x7fab04003a00 2024-09-16T00:08:37.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.058+0000 7fab1dc6d640 1 -- 172.21.15.139:0/4191273623 reap_dead start 2024-09-16T00:08:37.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.059+0000 7fab1d46c640 1 --2- 172.21.15.139:0/4191273623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab04003a00 0x7fab04003e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fab1400c190 tx=0x7fab04005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23ae9995a9108d9d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.060+0000 7fab0ffff640 1 -- 172.21.15.139:0/4191273623 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14007c60 con 0x7fab04003a00 2024-09-16T00:08:37.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.060+0000 7fab0ffff640 1 -- 172.21.15.139:0/4191273623 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab14005ad0 con 0x7fab04003a00 2024-09-16T00:08:37.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.060+0000 7fab0ffff640 1 -- 172.21.15.139:0/4191273623 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14007070 con 0x7fab04003a00 2024-09-16T00:08:37.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.060+0000 7fab1f6d0640 1 -- 172.21.15.139:0/4191273623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab04003a00 msgr2=0x7fab04003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.060+0000 7fab1f6d0640 1 --2- 172.21.15.139:0/4191273623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab04003a00 0x7fab04003e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fab1400c190 tx=0x7fab04005830 comp rx=0 tx=0).stop 2024-09-16T00:08:37.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.061+0000 7fab1f6d0640 1 -- 172.21.15.139:0/4191273623 shutdown_connections 2024-09-16T00:08:37.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.061+0000 7fab1f6d0640 1 --2- 172.21.15.139:0/4191273623 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab04003a00 0x7fab04003e20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.061+0000 7fab1f6d0640 1 -- 172.21.15.139:0/4191273623 >> 172.21.15.139:0/4191273623 conn(0x7fab18067d70 msgr2=0x7fab18068180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:37.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.061+0000 7fab1f6d0640 1 -- 172.21.15.139:0/4191273623 shutdown_connections 2024-09-16T00:08:37.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.061+0000 7fab1f6d0640 1 -- 172.21.15.139:0/4191273623 wait complete. 2024-09-16T00:08:37.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.062+0000 7fab1f6d0640 1 Processor -- start 2024-09-16T00:08:37.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.062+0000 7fab1f6d0640 1 -- start start 2024-09-16T00:08:37.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.063+0000 7fab1f6d0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab04003a00 0x7fab181559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.063+0000 7fab1f6d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab1815b490 0x7fab180c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.063+0000 7fab1f6d0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab180c8470 0x7fab1815a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.063+0000 7fab1f6d0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fab1815c860 con 0x7fab1815b490 2024-09-16T00:08:37.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.063+0000 7fab1f6d0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab180caaa0 con 0x7fab180c8470 2024-09-16T00:08:37.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.063+0000 7fab1f6d0640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fab180c96b0 con 0x7fab04003a00 2024-09-16T00:08:37.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.063+0000 7fab1d46c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab04003a00 0x7fab181559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.063+0000 7fab1d46c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab04003a00 0x7fab181559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:60178/0 (socket says 172.21.15.139:60178) 2024-09-16T00:08:37.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.063+0000 7fab1d46c640 1 -- 172.21.15.139:0/2678331644 learned_addr learned my addr 172.21.15.139:0/2678331644 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:37.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.063+0000 7fab1cc6b640 1 --2- 172.21.15.139:0/2678331644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab1815b490 0x7fab180c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.063+0000 7fab1dc6d640 1 --2- 172.21.15.139:0/2678331644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab180c8470 0x7fab1815a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.064+0000 7fab1d46c640 1 -- 172.21.15.139:0/2678331644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab180c8470 msgr2=0x7fab1815a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.064+0000 7fab1d46c640 1 --2- 172.21.15.139:0/2678331644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab180c8470 0x7fab1815a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.064+0000 7fab1d46c640 1 -- 172.21.15.139:0/2678331644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab1815b490 msgr2=0x7fab180c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.064+0000 7fab1d46c640 1 --2- 172.21.15.139:0/2678331644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab1815b490 0x7fab180c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.064+0000 7fab1d46c640 1 -- 172.21.15.139:0/2678331644 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab181539d0 con 0x7fab04003a00 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.064+0000 7fab1cc6b640 1 --2- 172.21.15.139:0/2678331644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab1815b490 0x7fab180c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.064+0000 7fab1d46c640 1 --2- 172.21.15.139:0/2678331644 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab04003a00 0x7fab181559b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fab1400c190 tx=0x7fab180c9c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.064+0000 7fab0dffb640 1 -- 172.21.15.139:0/2678331644 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14041020 con 0x7fab04003a00 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.064+0000 7fab0dffb640 1 -- 172.21.15.139:0/2678331644 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab14038040 con 0x7fab04003a00 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.064+0000 7fab0dffb640 1 -- 172.21.15.139:0/2678331644 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14007070 con 0x7fab04003a00 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.064+0000 7fab1f6d0640 1 -- 172.21.15.139:0/2678331644 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab181efbd0 con 0x7fab04003a00 2024-09-16T00:08:37.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.065+0000 7fab1f6d0640 1 -- 172.21.15.139:0/2678331644 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fab181f0190 con 0x7fab04003a00 2024-09-16T00:08:37.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.066+0000 7fab1f6d0640 1 -- 172.21.15.139:0/2678331644 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab180c89b0 con 0x7fab04003a00 2024-09-16T00:08:37.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.066+0000 7fab0dffb640 1 -- 172.21.15.139:0/2678331644 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fab14044020 con 0x7fab04003a00 2024-09-16T00:08:37.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.067+0000 7fab0dffb640 1 --2- 172.21.15.139:0/2678331644 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faafc06c2b0 0x7faafc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.067+0000 7fab0dffb640 1 -- 172.21.15.139:0/2678331644 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(47..47 src has 1..47) ==== 5915+0+0 (secure 0 0 0) 0x7fab140ab660 con 0x7fab04003a00 2024-09-16T00:08:37.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.067+0000 7fab1cc6b640 1 --2- 172.21.15.139:0/2678331644 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faafc06c2b0 0x7faafc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.067+0000 7fab1cc6b640 1 --2- 172.21.15.139:0/2678331644 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faafc06c2b0 0x7faafc06e770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fab1806e7c0 tx=0x7fab08009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.072+0000 7fab0dffb640 1 -- 172.21.15.139:0/2678331644 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fab180c89b0 con 0x7fab04003a00 2024-09-16T00:08:37.075 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:37.074+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:37.094 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:37.094 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:37.103 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:08:37.175 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:37.175+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:37.206 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:37.206 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:37.215 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:08:37.233 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:08:37.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.244+0000 7fab1f6d0640 1 -- 172.21.15.139:0/2678331644 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fab180ca730 con 0x7faafc06c2b0 2024-09-16T00:08:37.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.257+0000 7fab0dffb640 1 -- 172.21.15.139:0/2678331644 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251828 (secure 0 0 0) 0x7fab180ca730 con 0x7faafc06c2b0 2024-09-16T00:08:37.261 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:08:37.264 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:08:37.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.261+0000 7fab1f6d0640 1 -- 172.21.15.139:0/2678331644 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faafc06c2b0 msgr2=0x7faafc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.261+0000 7fab1f6d0640 1 --2- 172.21.15.139:0/2678331644 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faafc06c2b0 0x7faafc06e770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fab1806e7c0 tx=0x7fab08009190 comp rx=0 tx=0).stop 2024-09-16T00:08:37.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.261+0000 7fab1f6d0640 1 -- 172.21.15.139:0/2678331644 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab04003a00 msgr2=0x7fab181559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.261+0000 7fab1f6d0640 1 --2- 172.21.15.139:0/2678331644 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab04003a00 0x7fab181559b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fab1400c190 tx=0x7fab180c9c40 comp rx=0 tx=0).stop 2024-09-16T00:08:37.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.261+0000 7fab1f6d0640 1 -- 172.21.15.139:0/2678331644 shutdown_connections 2024-09-16T00:08:37.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.261+0000 7fab1f6d0640 1 --2- 172.21.15.139:0/2678331644 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faafc06c2b0 0x7faafc06e770 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.261+0000 7fab1f6d0640 1 --2- 172.21.15.139:0/2678331644 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab180c8470 0x7fab1815a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.261+0000 7fab1f6d0640 1 --2- 172.21.15.139:0/2678331644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab1815b490 0x7fab180c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.261+0000 7fab1f6d0640 1 --2- 172.21.15.139:0/2678331644 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab04003a00 0x7fab181559b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.261+0000 7fab1f6d0640 1 -- 172.21.15.139:0/2678331644 >> 172.21.15.139:0/2678331644 conn(0x7fab18067d70 msgr2=0x7fab180cda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:37.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.261+0000 7fab1f6d0640 1 -- 172.21.15.139:0/2678331644 shutdown_connections 2024-09-16T00:08:37.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.261+0000 7fab1f6d0640 1 -- 172.21.15.139:0/2678331644 wait complete. 2024-09-16T00:08:37.280 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-09-16T00:08:36.880498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597721,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":212,"num_write_kb":2439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":6555,"num_keys_recovered":2,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220200,"kb_used_data":3944,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754520,"statfs":{"total":773094113280,"available":772868628480,"internally_reserved":0,"allocated":4038656,"data_stored":2838298,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022623"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":54,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839717+0000","last_change":"2024-09-16T00:08:24.838661+0000","last_active":"2024-09-16T00:08:30.839717+0000","last_peered":"2024-09-16T00:08:30.839717+0000","last_clean":"2024-09-16T00:08:30.839717+0000","last_became_active":"2024-09-16T00:08:24.838273+0000","last_became_peered":"2024-09-16T00:08:24.838273+0000","last_unstale":"2024-09-16T00:08:30.839717+0000","last_undegraded":"2024-09-16T00:08:30.839717+0000","last_fullsized":"2024-09-16T00:08:30.839717+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:32.768723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882794+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:08:35.882794+0000","last_peered":"2024-09-16T00:08:35.882794+0000","last_clean":"2024-09-16T00:08:35.882794+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:08:35.882794+0000","last_undegraded":"2024-09-16T00:08:35.882794+0000","last_fullsized":"2024-09-16T00:08:35.882794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.890336+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:08:35.890336+0000","last_peered":"2024-09-16T00:08:35.890336+0000","last_clean":"2024-09-16T00:08:35.890336+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:08:35.890336+0000","last_undegraded":"2024-09-16T00:08:35.890336+0000","last_fullsized":"2024-09-16T00:08:35.890336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847983+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:08:31.847983+0000","last_peered":"2024-09-16T00:08:31.847983+0000","last_clean":"2024-09-16T00:08:31.847983+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:08:31.847983+0000","last_undegraded":"2024-09-16T00:08:31.847983+0000","last_fullsized":"2024-09-16T00:08:31.847983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882059+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:08:35.882059+0000","last_peered":"2024-09-16T00:08:35.882059+0000","last_clean":"2024-09-16T00:08:35.882059+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:08:35.882059+0000","last_undegraded":"2024-09-16T00:08:35.882059+0000","last_fullsized":"2024-09-16T00:08:35.882059+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847578+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:08:31.847578+0000","last_peered":"2024-09-16T00:08:31.847578+0000","last_clean":"2024-09-16T00:08:31.847578+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:08:31.847578+0000","last_undegraded":"2024-09-16T00:08:31.847578+0000","last_fullsized":"2024-09-16T00:08:31.847578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839199+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:08:30.839199+0000","last_peered":"2024-09-16T00:08:30.839199+0000","last_clean":"2024-09-16T00:08:30.839199+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:08:30.839199+0000","last_undegraded":"2024-09-16T00:08:30.839199+0000","last_fullsized":"2024-09-16T00:08:30.839199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":32,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874260+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:08:34.874260+0000","last_peered":"2024-09-16T00:08:34.874260+0000","last_clean":"2024-09-16T00:08:34.874260+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:08:34.874260+0000","last_undegraded":"2024-09-16T00:08:34.874260+0000","last_fullsized":"2024-09-16T00:08:34.874260+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839738+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:08:30.839738+0000","last_peered":"2024-09-16T00:08:30.839738+0000","last_clean":"2024-09-16T00:08:30.839738+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:08:30.839738+0000","last_undegraded":"2024-09-16T00:08:30.839738+0000","last_fullsized":"2024-09-16T00:08:30.839738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.848014+0000","last_change":"2024-09-16T00:08:24.835738+0000","last_active":"2024-09-16T00:08:31.848014+0000","last_peered":"2024-09-16T00:08:31.848014+0000","last_clean":"2024-09-16T00:08:31.848014+0000","last_became_active":"2024-09-16T00:08:24.835368+0000","last_became_peered":"2024-09-16T00:08:24.835368+0000","last_unstale":"2024-09-16T00:08:31.848014+0000","last_undegraded":"2024-09-16T00:08:31.848014+0000","last_fullsized":"2024-09-16T00:08:31.848014+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:40.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847765+0000","last_change":"2024-09-16T00:08:24.840483+0000","last_active":"2024-09-16T00:08:31.847765+0000","last_peered":"2024-09-16T00:08:31.847765+0000","last_clean":"2024-09-16T00:08:31.847765+0000","last_became_active":"2024-09-16T00:08:24.840085+0000","last_became_peered":"2024-09-16T00:08:24.840085+0000","last_unstale":"2024-09-16T00:08:31.847765+0000","last_undegraded":"2024-09-16T00:08:31.847765+0000","last_fullsized":"2024-09-16T00:08:31.847765+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:55.997551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":54,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831329+0000","last_change":"2024-09-16T00:08:24.839776+0000","last_active":"2024-09-16T00:08:31.831329+0000","last_peered":"2024-09-16T00:08:31.831329+0000","last_clean":"2024-09-16T00:08:31.831329+0000","last_became_active":"2024-09-16T00:08:24.839415+0000","last_became_peered":"2024-09-16T00:08:24.839415+0000","last_unstale":"2024-09-16T00:08:31.831329+0000","last_undegraded":"2024-09-16T00:08:31.831329+0000","last_fullsized":"2024-09-16T00:08:31.831329+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"peering","last_fresh":"2024-09-16T00:08:35.190711+0000","last_change":"2024-09-16T00:08:35.892992+0000","last_active":"2024-09-16T00:08:35.892992+0000","last_peered":"2024-09-16T00:08:35.190711+0000","last_clean":"2024-09-16T00:08:35.190711+0000","last_became_active":"2024-09-16T00:08:24.840637+0000","last_became_peered":"2024-09-16T00:08:24.840637+0000","last_unstale":"2024-09-16T00:08:35.190711+0000","last_undegraded":"2024-09-16T00:08:35.190711+0000","last_fullsized":"2024-09-16T00:08:35.190711+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888448+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:08:35.888448+0000","last_peered":"2024-09-16T00:08:35.888448+0000","last_clean":"2024-09-16T00:08:35.888448+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:08:35.888448+0000","last_undegraded":"2024-09-16T00:08:35.888448+0000","last_fullsized":"2024-09-16T00:08:35.888448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881726+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:08:35.881726+0000","last_peered":"2024-09-16T00:08:35.881726+0000","last_clean":"2024-09-16T00:08:35.881726+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:08:35.881726+0000","last_undegraded":"2024-09-16T00:08:35.881726+0000","last_fullsized":"2024-09-16T00:08:35.881726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847508+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:08:31.847508+0000","last_peered":"2024-09-16T00:08:31.847508+0000","last_clean":"2024-09-16T00:08:31.847508+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:08:31.847508+0000","last_undegraded":"2024-09-16T00:08:31.847508+0000","last_fullsized":"2024-09-16T00:08:31.847508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.848137+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:08:31.848137+0000","last_peered":"2024-09-16T00:08:31.848137+0000","last_clean":"2024-09-16T00:08:31.848137+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:08:31.848137+0000","last_undegraded":"2024-09-16T00:08:31.848137+0000","last_fullsized":"2024-09-16T00:08:31.848137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874802+0000","last_change":"2024-09-16T00:08:24.836818+0000","last_active":"2024-09-16T00:08:34.874802+0000","last_peered":"2024-09-16T00:08:34.874802+0000","last_clean":"2024-09-16T00:08:34.874802+0000","last_became_active":"2024-09-16T00:08:24.836029+0000","last_became_peered":"2024-09-16T00:08:24.836029+0000","last_unstale":"2024-09-16T00:08:34.874802+0000","last_undegraded":"2024-09-16T00:08:34.874802+0000","last_fullsized":"2024-09-16T00:08:34.874802+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:38:55.816393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839639+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:08:30.839639+0000","last_peered":"2024-09-16T00:08:30.839639+0000","last_clean":"2024-09-16T00:08:30.839639+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:08:30.839639+0000","last_undegraded":"2024-09-16T00:08:30.839639+0000","last_fullsized":"2024-09-16T00:08:30.839639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831171+0000","last_change":"2024-09-16T00:08:24.839823+0000","last_active":"2024-09-16T00:08:31.831171+0000","last_peered":"2024-09-16T00:08:31.831171+0000","last_clean":"2024-09-16T00:08:31.831171+0000","last_became_active":"2024-09-16T00:08:24.839507+0000","last_became_peered":"2024-09-16T00:08:24.839507+0000","last_unstale":"2024-09-16T00:08:31.831171+0000","last_undegraded":"2024-09-16T00:08:31.831171+0000","last_fullsized":"2024-09-16T00:08:31.831171+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888586+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:08:35.888586+0000","last_peered":"2024-09-16T00:08:35.888586+0000","last_clean":"2024-09-16T00:08:35.888586+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:08:35.888586+0000","last_undegraded":"2024-09-16T00:08:35.888586+0000","last_fullsized":"2024-09-16T00:08:35.888586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847663+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:08:31.847663+0000","last_peered":"2024-09-16T00:08:31.847663+0000","last_clean":"2024-09-16T00:08:31.847663+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:08:31.847663+0000","last_undegraded":"2024-09-16T00:08:31.847663+0000","last_fullsized":"2024-09-16T00:08:31.847663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.891321+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:08:35.891321+0000","last_peered":"2024-09-16T00:08:35.891321+0000","last_clean":"2024-09-16T00:08:35.891321+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:08:35.891321+0000","last_undegraded":"2024-09-16T00:08:35.891321+0000","last_fullsized":"2024-09-16T00:08:35.891321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847518+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:08:31.847518+0000","last_peered":"2024-09-16T00:08:31.847518+0000","last_clean":"2024-09-16T00:08:31.847518+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:08:31.847518+0000","last_undegraded":"2024-09-16T00:08:31.847518+0000","last_fullsized":"2024-09-16T00:08:31.847518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.894305+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:08:35.894305+0000","last_peered":"2024-09-16T00:08:35.894305+0000","last_clean":"2024-09-16T00:08:35.894305+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:08:35.894305+0000","last_undegraded":"2024-09-16T00:08:35.894305+0000","last_fullsized":"2024-09-16T00:08:35.894305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847213+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:08:31.847213+0000","last_peered":"2024-09-16T00:08:31.847213+0000","last_clean":"2024-09-16T00:08:31.847213+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:08:31.847213+0000","last_undegraded":"2024-09-16T00:08:31.847213+0000","last_fullsized":"2024-09-16T00:08:31.847213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882462+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:08:35.882462+0000","last_peered":"2024-09-16T00:08:35.882462+0000","last_clean":"2024-09-16T00:08:35.882462+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:08:35.882462+0000","last_undegraded":"2024-09-16T00:08:35.882462+0000","last_fullsized":"2024-09-16T00:08:35.882462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.892094+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:08:35.892094+0000","last_peered":"2024-09-16T00:08:35.892094+0000","last_clean":"2024-09-16T00:08:35.892094+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:08:35.892094+0000","last_undegraded":"2024-09-16T00:08:35.892094+0000","last_fullsized":"2024-09-16T00:08:35.892094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888666+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:08:35.888666+0000","last_peered":"2024-09-16T00:08:35.888666+0000","last_clean":"2024-09-16T00:08:35.888666+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:08:35.888666+0000","last_undegraded":"2024-09-16T00:08:35.888666+0000","last_fullsized":"2024-09-16T00:08:35.888666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.875423+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:08:34.875423+0000","last_peered":"2024-09-16T00:08:34.875423+0000","last_clean":"2024-09-16T00:08:34.875423+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:08:34.875423+0000","last_undegraded":"2024-09-16T00:08:34.875423+0000","last_fullsized":"2024-09-16T00:08:34.875423+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874580+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:08:34.874580+0000","last_peered":"2024-09-16T00:08:34.874580+0000","last_clean":"2024-09-16T00:08:34.874580+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:08:34.874580+0000","last_undegraded":"2024-09-16T00:08:34.874580+0000","last_fullsized":"2024-09-16T00:08:34.874580+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.025362+0000","last_change":"2024-09-16T00:08:24.839775+0000","last_active":"2024-09-16T00:08:30.025362+0000","last_peered":"2024-09-16T00:08:30.025362+0000","last_clean":"2024-09-16T00:08:30.025362+0000","last_became_active":"2024-09-16T00:08:24.839222+0000","last_became_peered":"2024-09-16T00:08:24.839222+0000","last_unstale":"2024-09-16T00:08:30.025362+0000","last_undegraded":"2024-09-16T00:08:30.025362+0000","last_fullsized":"2024-09-16T00:08:30.025362+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:33:48.945330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831152+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:08:31.831152+0000","last_peered":"2024-09-16T00:08:31.831152+0000","last_clean":"2024-09-16T00:08:31.831152+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:08:31.831152+0000","last_undegraded":"2024-09-16T00:08:31.831152+0000","last_fullsized":"2024-09-16T00:08:31.831152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":63,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.893461+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:08:35.893461+0000","last_peered":"2024-09-16T00:08:35.893461+0000","last_clean":"2024-09-16T00:08:35.893461+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:08:35.893461+0000","last_undegraded":"2024-09-16T00:08:35.893461+0000","last_fullsized":"2024-09-16T00:08:35.893461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888853+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:08:35.888853+0000","last_peered":"2024-09-16T00:08:35.888853+0000","last_clean":"2024-09-16T00:08:35.888853+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:08:35.888853+0000","last_undegraded":"2024-09-16T00:08:35.888853+0000","last_fullsized":"2024-09-16T00:08:35.888853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831068+0000","last_change":"2024-09-16T00:08:24.835401+0000","last_active":"2024-09-16T00:08:31.831068+0000","last_peered":"2024-09-16T00:08:31.831068+0000","last_clean":"2024-09-16T00:08:31.831068+0000","last_became_active":"2024-09-16T00:08:24.835017+0000","last_became_peered":"2024-09-16T00:08:24.835017+0000","last_unstale":"2024-09-16T00:08:31.831068+0000","last_undegraded":"2024-09-16T00:08:31.831068+0000","last_fullsized":"2024-09-16T00:08:31.831068+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:02.003294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"peering","last_fresh":"2024-09-16T00:08:35.189391+0000","last_change":"2024-09-16T00:08:35.890927+0000","last_active":"2024-09-16T00:08:35.890927+0000","last_peered":"2024-09-16T00:08:35.189391+0000","last_clean":"2024-09-16T00:08:35.189391+0000","last_became_active":"2024-09-16T00:08:24.838130+0000","last_became_peered":"2024-09-16T00:08:24.838130+0000","last_unstale":"2024-09-16T00:08:35.189391+0000","last_undegraded":"2024-09-16T00:08:35.189391+0000","last_fullsized":"2024-09-16T00:08:35.189391+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.892368+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:08:35.892368+0000","last_peered":"2024-09-16T00:08:35.892368+0000","last_clean":"2024-09-16T00:08:35.892368+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:08:35.892368+0000","last_undegraded":"2024-09-16T00:08:35.892368+0000","last_fullsized":"2024-09-16T00:08:35.892368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":54,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847637+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:08:31.847637+0000","last_peered":"2024-09-16T00:08:31.847637+0000","last_clean":"2024-09-16T00:08:31.847637+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:08:31.847637+0000","last_undegraded":"2024-09-16T00:08:31.847637+0000","last_fullsized":"2024-09-16T00:08:31.847637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881638+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:08:35.881638+0000","last_peered":"2024-09-16T00:08:35.881638+0000","last_clean":"2024-09-16T00:08:35.881638+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:08:35.881638+0000","last_undegraded":"2024-09-16T00:08:35.881638+0000","last_fullsized":"2024-09-16T00:08:35.881638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":54,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847413+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:08:31.847413+0000","last_peered":"2024-09-16T00:08:31.847413+0000","last_clean":"2024-09-16T00:08:31.847413+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:08:31.847413+0000","last_undegraded":"2024-09-16T00:08:31.847413+0000","last_fullsized":"2024-09-16T00:08:31.847413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847960+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:08:31.847960+0000","last_peered":"2024-09-16T00:08:31.847960+0000","last_clean":"2024-09-16T00:08:31.847960+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:08:31.847960+0000","last_undegraded":"2024-09-16T00:08:31.847960+0000","last_fullsized":"2024-09-16T00:08:31.847960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":54,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.848115+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:08:31.848115+0000","last_peered":"2024-09-16T00:08:31.848115+0000","last_clean":"2024-09-16T00:08:31.848115+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:08:31.848115+0000","last_undegraded":"2024-09-16T00:08:31.848115+0000","last_fullsized":"2024-09-16T00:08:31.848115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.891379+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:08:35.891379+0000","last_peered":"2024-09-16T00:08:35.891379+0000","last_clean":"2024-09-16T00:08:35.891379+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:08:35.891379+0000","last_undegraded":"2024-09-16T00:08:35.891379+0000","last_fullsized":"2024-09-16T00:08:35.891379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":64,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882552+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:08:35.882552+0000","last_peered":"2024-09-16T00:08:35.882552+0000","last_clean":"2024-09-16T00:08:35.882552+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:08:35.882552+0000","last_undegraded":"2024-09-16T00:08:35.882552+0000","last_fullsized":"2024-09-16T00:08:35.882552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831420+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:08:31.831420+0000","last_peered":"2024-09-16T00:08:31.831420+0000","last_clean":"2024-09-16T00:08:31.831420+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:08:31.831420+0000","last_undegraded":"2024-09-16T00:08:31.831420+0000","last_fullsized":"2024-09-16T00:08:31.831420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"28'10","reported_seq":60,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.840236+0000","last_change":"2024-09-16T00:08:24.854457+0000","last_active":"2024-09-16T00:08:30.840236+0000","last_peered":"2024-09-16T00:08:30.840236+0000","last_clean":"2024-09-16T00:08:30.840236+0000","last_became_active":"2024-09-16T00:08:24.837069+0000","last_became_peered":"2024-09-16T00:08:24.837069+0000","last_unstale":"2024-09-16T00:08:30.840236+0000","last_undegraded":"2024-09-16T00:08:30.840236+0000","last_fullsized":"2024-09-16T00:08:30.840236+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:04.061152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6555,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":6555,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839507+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:08:30.839507+0000","last_peered":"2024-09-16T00:08:30.839507+0000","last_clean":"2024-09-16T00:08:30.839507+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:08:30.839507+0000","last_undegraded":"2024-09-16T00:08:30.839507+0000","last_fullsized":"2024-09-16T00:08:30.839507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":63,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888938+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:08:35.888938+0000","last_peered":"2024-09-16T00:08:35.888938+0000","last_clean":"2024-09-16T00:08:35.888938+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:08:35.888938+0000","last_undegraded":"2024-09-16T00:08:35.888938+0000","last_fullsized":"2024-09-16T00:08:35.888938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888319+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:08:35.888319+0000","last_peered":"2024-09-16T00:08:35.888319+0000","last_clean":"2024-09-16T00:08:35.888319+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:08:35.888319+0000","last_undegraded":"2024-09-16T00:08:35.888319+0000","last_fullsized":"2024-09-16T00:08:35.888319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":54,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847693+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:08:31.847693+0000","last_peered":"2024-09-16T00:08:31.847693+0000","last_clean":"2024-09-16T00:08:31.847693+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:08:31.847693+0000","last_undegraded":"2024-09-16T00:08:31.847693+0000","last_fullsized":"2024-09-16T00:08:31.847693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839384+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:08:30.839384+0000","last_peered":"2024-09-16T00:08:30.839384+0000","last_clean":"2024-09-16T00:08:30.839384+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:08:30.839384+0000","last_undegraded":"2024-09-16T00:08:30.839384+0000","last_fullsized":"2024-09-16T00:08:30.839384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831022+0000","last_change":"2024-09-16T00:08:24.834930+0000","last_active":"2024-09-16T00:08:31.831022+0000","last_peered":"2024-09-16T00:08:31.831022+0000","last_clean":"2024-09-16T00:08:31.831022+0000","last_became_active":"2024-09-16T00:08:24.834556+0000","last_became_peered":"2024-09-16T00:08:24.834556+0000","last_unstale":"2024-09-16T00:08:31.831022+0000","last_undegraded":"2024-09-16T00:08:31.831022+0000","last_fullsized":"2024-09-16T00:08:31.831022+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:28.660189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.025267+0000","last_change":"2024-09-16T00:08:24.839984+0000","last_active":"2024-09-16T00:08:30.025267+0000","last_peered":"2024-09-16T00:08:30.025267+0000","last_clean":"2024-09-16T00:08:30.025267+0000","last_became_active":"2024-09-16T00:08:24.839497+0000","last_became_peered":"2024-09-16T00:08:24.839497+0000","last_unstale":"2024-09-16T00:08:30.025267+0000","last_undegraded":"2024-09-16T00:08:30.025267+0000","last_fullsized":"2024-09-16T00:08:30.025267+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874425+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:08:34.874425+0000","last_peered":"2024-09-16T00:08:34.874425+0000","last_clean":"2024-09-16T00:08:34.874425+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:08:34.874425+0000","last_undegraded":"2024-09-16T00:08:34.874425+0000","last_fullsized":"2024-09-16T00:08:34.874425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839214+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:08:30.839214+0000","last_peered":"2024-09-16T00:08:30.839214+0000","last_clean":"2024-09-16T00:08:30.839214+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:08:30.839214+0000","last_undegraded":"2024-09-16T00:08:30.839214+0000","last_fullsized":"2024-09-16T00:08:30.839214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":52,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831448+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:08:31.831448+0000","last_peered":"2024-09-16T00:08:31.831448+0000","last_clean":"2024-09-16T00:08:31.831448+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:08:31.831448+0000","last_undegraded":"2024-09-16T00:08:31.831448+0000","last_fullsized":"2024-09-16T00:08:31.831448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839647+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:08:30.839647+0000","last_peered":"2024-09-16T00:08:30.839647+0000","last_clean":"2024-09-16T00:08:30.839647+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:08:30.839647+0000","last_undegraded":"2024-09-16T00:08:30.839647+0000","last_fullsized":"2024-09-16T00:08:30.839647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":62,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874882+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:08:34.874882+0000","last_peered":"2024-09-16T00:08:34.874882+0000","last_clean":"2024-09-16T00:08:34.874882+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:08:34.874882+0000","last_undegraded":"2024-09-16T00:08:34.874882+0000","last_fullsized":"2024-09-16T00:08:34.874882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839795+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:08:30.839795+0000","last_peered":"2024-09-16T00:08:30.839795+0000","last_clean":"2024-09-16T00:08:30.839795+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:08:30.839795+0000","last_undegraded":"2024-09-16T00:08:30.839795+0000","last_fullsized":"2024-09-16T00:08:30.839795+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":62,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882644+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:08:35.882644+0000","last_peered":"2024-09-16T00:08:35.882644+0000","last_clean":"2024-09-16T00:08:35.882644+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:08:35.882644+0000","last_undegraded":"2024-09-16T00:08:35.882644+0000","last_fullsized":"2024-09-16T00:08:35.882644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.848103+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:08:31.848103+0000","last_peered":"2024-09-16T00:08:31.848103+0000","last_clean":"2024-09-16T00:08:31.848103+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:08:31.848103+0000","last_undegraded":"2024-09-16T00:08:31.848103+0000","last_fullsized":"2024-09-16T00:08:31.848103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.875230+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:08:34.875230+0000","last_peered":"2024-09-16T00:08:34.875230+0000","last_clean":"2024-09-16T00:08:34.875230+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:08:34.875230+0000","last_undegraded":"2024-09-16T00:08:34.875230+0000","last_fullsized":"2024-09-16T00:08:34.875230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":54,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839539+0000","last_change":"2024-09-16T00:08:15.795467+0000","last_active":"2024-09-16T00:08:30.839539+0000","last_peered":"2024-09-16T00:08:30.839539+0000","last_clean":"2024-09-16T00:08:30.839539+0000","last_became_active":"2024-09-16T00:08:15.795167+0000","last_became_peered":"2024-09-16T00:08:15.795167+0000","last_unstale":"2024-09-16T00:08:30.839539+0000","last_undegraded":"2024-09-16T00:08:30.839539+0000","last_fullsized":"2024-09-16T00:08:30.839539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":207,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839287+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:08:30.839287+0000","last_peered":"2024-09-16T00:08:30.839287+0000","last_clean":"2024-09-16T00:08:30.839287+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:08:30.839287+0000","last_undegraded":"2024-09-16T00:08:30.839287+0000","last_fullsized":"2024-09-16T00:08:30.839287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":54,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847636+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:08:31.847636+0000","last_peered":"2024-09-16T00:08:31.847636+0000","last_clean":"2024-09-16T00:08:31.847636+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:08:31.847636+0000","last_undegraded":"2024-09-16T00:08:31.847636+0000","last_fullsized":"2024-09-16T00:08:31.847636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.893654+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:08:35.893654+0000","last_peered":"2024-09-16T00:08:35.893654+0000","last_clean":"2024-09-16T00:08:35.893654+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:08:35.893654+0000","last_undegraded":"2024-09-16T00:08:35.893654+0000","last_fullsized":"2024-09-16T00:08:35.893654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839181+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:08:30.839181+0000","last_peered":"2024-09-16T00:08:30.839181+0000","last_clean":"2024-09-16T00:08:30.839181+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:08:30.839181+0000","last_undegraded":"2024-09-16T00:08:30.839181+0000","last_fullsized":"2024-09-16T00:08:30.839181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.887981+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:08:35.887981+0000","last_peered":"2024-09-16T00:08:35.887981+0000","last_clean":"2024-09-16T00:08:35.887981+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:08:35.887981+0000","last_undegraded":"2024-09-16T00:08:35.887981+0000","last_fullsized":"2024-09-16T00:08:35.887981+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874658+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:08:34.874658+0000","last_peered":"2024-09-16T00:08:34.874658+0000","last_clean":"2024-09-16T00:08:34.874658+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:08:34.874658+0000","last_undegraded":"2024-09-16T00:08:34.874658+0000","last_fullsized":"2024-09-16T00:08:34.874658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881158+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:08:35.881158+0000","last_peered":"2024-09-16T00:08:35.881158+0000","last_clean":"2024-09-16T00:08:35.881158+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:08:35.881158+0000","last_undegraded":"2024-09-16T00:08:35.881158+0000","last_fullsized":"2024-09-16T00:08:35.881158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":54,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847952+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:08:31.847952+0000","last_peered":"2024-09-16T00:08:31.847952+0000","last_clean":"2024-09-16T00:08:31.847952+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:08:31.847952+0000","last_undegraded":"2024-09-16T00:08:31.847952+0000","last_fullsized":"2024-09-16T00:08:31.847952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831392+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:08:31.831392+0000","last_peered":"2024-09-16T00:08:31.831392+0000","last_clean":"2024-09-16T00:08:31.831392+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:08:31.831392+0000","last_undegraded":"2024-09-16T00:08:31.831392+0000","last_fullsized":"2024-09-16T00:08:31.831392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882294+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:08:35.882294+0000","last_peered":"2024-09-16T00:08:35.882294+0000","last_clean":"2024-09-16T00:08:35.882294+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:08:35.882294+0000","last_undegraded":"2024-09-16T00:08:35.882294+0000","last_fullsized":"2024-09-16T00:08:35.882294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847873+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:08:31.847873+0000","last_peered":"2024-09-16T00:08:31.847873+0000","last_clean":"2024-09-16T00:08:31.847873+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:08:31.847873+0000","last_undegraded":"2024-09-16T00:08:31.847873+0000","last_fullsized":"2024-09-16T00:08:31.847873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839727+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:08:30.839727+0000","last_peered":"2024-09-16T00:08:30.839727+0000","last_clean":"2024-09-16T00:08:30.839727+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:08:30.839727+0000","last_undegraded":"2024-09-16T00:08:30.839727+0000","last_fullsized":"2024-09-16T00:08:30.839727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882179+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:08:35.882179+0000","last_peered":"2024-09-16T00:08:35.882179+0000","last_clean":"2024-09-16T00:08:35.882179+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:08:35.882179+0000","last_undegraded":"2024-09-16T00:08:35.882179+0000","last_fullsized":"2024-09-16T00:08:35.882179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"26'3","reported_seq":72,"reported_epoch":45,"state":"peering","last_fresh":"2024-09-16T00:08:35.191937+0000","last_change":"2024-09-16T00:08:35.883363+0000","last_active":"2024-09-16T00:08:35.883363+0000","last_peered":"2024-09-16T00:08:35.191937+0000","last_clean":"2024-09-16T00:08:35.191937+0000","last_became_active":"2024-09-16T00:08:24.835764+0000","last_became_peered":"2024-09-16T00:08:24.835764+0000","last_unstale":"2024-09-16T00:08:35.191937+0000","last_undegraded":"2024-09-16T00:08:35.191937+0000","last_fullsized":"2024-09-16T00:08:35.191937+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.894399+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:08:35.894399+0000","last_peered":"2024-09-16T00:08:35.894399+0000","last_clean":"2024-09-16T00:08:35.894399+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:08:35.894399+0000","last_undegraded":"2024-09-16T00:08:35.894399+0000","last_fullsized":"2024-09-16T00:08:35.894399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":56,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847288+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:08:31.847288+0000","last_peered":"2024-09-16T00:08:31.847288+0000","last_clean":"2024-09-16T00:08:31.847288+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:08:31.847288+0000","last_undegraded":"2024-09-16T00:08:31.847288+0000","last_fullsized":"2024-09-16T00:08:31.847288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.890491+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:08:35.890491+0000","last_peered":"2024-09-16T00:08:35.890491+0000","last_clean":"2024-09-16T00:08:35.890491+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:08:35.890491+0000","last_undegraded":"2024-09-16T00:08:35.890491+0000","last_fullsized":"2024-09-16T00:08:35.890491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":51,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839174+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:08:30.839174+0000","last_peered":"2024-09-16T00:08:30.839174+0000","last_clean":"2024-09-16T00:08:30.839174+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:08:30.839174+0000","last_undegraded":"2024-09-16T00:08:30.839174+0000","last_fullsized":"2024-09-16T00:08:30.839174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831075+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:08:31.831075+0000","last_peered":"2024-09-16T00:08:31.831075+0000","last_clean":"2024-09-16T00:08:31.831075+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:08:31.831075+0000","last_undegraded":"2024-09-16T00:08:31.831075+0000","last_fullsized":"2024-09-16T00:08:31.831075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":61,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881815+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:08:35.881815+0000","last_peered":"2024-09-16T00:08:35.881815+0000","last_clean":"2024-09-16T00:08:35.881815+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:08:35.881815+0000","last_undegraded":"2024-09-16T00:08:35.881815+0000","last_fullsized":"2024-09-16T00:08:35.881815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.892453+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:08:35.892453+0000","last_peered":"2024-09-16T00:08:35.892453+0000","last_clean":"2024-09-16T00:08:35.892453+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:08:35.892453+0000","last_undegraded":"2024-09-16T00:08:35.892453+0000","last_fullsized":"2024-09-16T00:08:35.892453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":61,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881852+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:08:35.881852+0000","last_peered":"2024-09-16T00:08:35.881852+0000","last_clean":"2024-09-16T00:08:35.881852+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:08:35.881852+0000","last_undegraded":"2024-09-16T00:08:35.881852+0000","last_fullsized":"2024-09-16T00:08:35.881852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.893352+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:08:35.893352+0000","last_peered":"2024-09-16T00:08:35.893352+0000","last_clean":"2024-09-16T00:08:35.893352+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:08:35.893352+0000","last_undegraded":"2024-09-16T00:08:35.893352+0000","last_fullsized":"2024-09-16T00:08:35.893352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":53,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839419+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:08:30.839419+0000","last_peered":"2024-09-16T00:08:30.839419+0000","last_clean":"2024-09-16T00:08:30.839419+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:08:30.839419+0000","last_undegraded":"2024-09-16T00:08:30.839419+0000","last_fullsized":"2024-09-16T00:08:30.839419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831024+0000","last_change":"2024-09-16T00:08:24.835323+0000","last_active":"2024-09-16T00:08:31.831024+0000","last_peered":"2024-09-16T00:08:31.831024+0000","last_clean":"2024-09-16T00:08:31.831024+0000","last_became_active":"2024-09-16T00:08:24.834976+0000","last_became_peered":"2024-09-16T00:08:24.834976+0000","last_unstale":"2024-09-16T00:08:31.831024+0000","last_undegraded":"2024-09-16T00:08:31.831024+0000","last_fullsized":"2024-09-16T00:08:31.831024+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847790+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:08:31.847790+0000","last_peered":"2024-09-16T00:08:31.847790+0000","last_clean":"2024-09-16T00:08:31.847790+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:08:31.847790+0000","last_undegraded":"2024-09-16T00:08:31.847790+0000","last_fullsized":"2024-09-16T00:08:31.847790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":53,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839492+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:08:30.839492+0000","last_peered":"2024-09-16T00:08:30.839492+0000","last_clean":"2024-09-16T00:08:30.839492+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:08:30.839492+0000","last_undegraded":"2024-09-16T00:08:30.839492+0000","last_fullsized":"2024-09-16T00:08:30.839492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.848046+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:08:31.848046+0000","last_peered":"2024-09-16T00:08:31.848046+0000","last_clean":"2024-09-16T00:08:31.848046+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:08:31.848046+0000","last_undegraded":"2024-09-16T00:08:31.848046+0000","last_fullsized":"2024-09-16T00:08:31.848046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839608+0000","last_change":"2024-09-16T00:08:24.844324+0000","last_active":"2024-09-16T00:08:30.839608+0000","last_peered":"2024-09-16T00:08:30.839608+0000","last_clean":"2024-09-16T00:08:30.839608+0000","last_became_active":"2024-09-16T00:08:24.840798+0000","last_became_peered":"2024-09-16T00:08:24.840798+0000","last_unstale":"2024-09-16T00:08:30.839608+0000","last_undegraded":"2024-09-16T00:08:30.839608+0000","last_fullsized":"2024-09-16T00:08:30.839608+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:34.657833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874986+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:08:34.874986+0000","last_peered":"2024-09-16T00:08:34.874986+0000","last_clean":"2024-09-16T00:08:34.874986+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:08:34.874986+0000","last_undegraded":"2024-09-16T00:08:34.874986+0000","last_fullsized":"2024-09-16T00:08:34.874986+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":63,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.895206+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:08:35.895206+0000","last_peered":"2024-09-16T00:08:35.895206+0000","last_clean":"2024-09-16T00:08:35.895206+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:08:35.895206+0000","last_undegraded":"2024-09-16T00:08:35.895206+0000","last_fullsized":"2024-09-16T00:08:35.895206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"peering","last_fresh":"2024-09-16T00:08:35.190279+0000","last_change":"2024-09-16T00:08:35.895010+0000","last_active":"2024-09-16T00:08:35.895010+0000","last_peered":"2024-09-16T00:08:35.190279+0000","last_clean":"2024-09-16T00:08:35.190279+0000","last_became_active":"2024-09-16T00:08:24.838285+0000","last_became_peered":"2024-09-16T00:08:24.838285+0000","last_unstale":"2024-09-16T00:08:35.190279+0000","last_undegraded":"2024-09-16T00:08:35.190279+0000","last_fullsized":"2024-09-16T00:08:35.190279+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":65,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.891464+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:08:35.891464+0000","last_peered":"2024-09-16T00:08:35.891464+0000","last_clean":"2024-09-16T00:08:35.891464+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:08:35.891464+0000","last_undegraded":"2024-09-16T00:08:35.891464+0000","last_fullsized":"2024-09-16T00:08:35.891464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7353,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":6555,"num_keys_recovered":2,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":14706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":524288,"data_stored":784061,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":199234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":475136,"data_stored":199756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":466944,"data_stored":193237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":520192,"data_stored":224158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":217763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":581632,"data_stored":808609,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":491520,"data_stored":211480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":6555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":6645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:08:37.292 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:08:37.292 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:37.282+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:37.319 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:37.320 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:37.329 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:08:37.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.341+0000 7fcae40d7640 1 Processor -- start 2024-09-16T00:08:37.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.342+0000 7fcae40d7640 1 -- start start 2024-09-16T00:08:37.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.342+0000 7fcae40d7640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fcadc0c8e90 0x7fcadc0c9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.342+0000 7fcae40d7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcadc0c97b0 con 0x7fcadc0c7880 2024-09-16T00:08:37.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.342+0000 7fcae40d7640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcadc073e50 con 0x7fcadc15a9f0 2024-09-16T00:08:37.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.342+0000 7fcae40d7640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fcadc0c5f80 con 0x7fcadc0c8e90 2024-09-16T00:08:37.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.343+0000 7fcae1672640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fcadc0c7880 0x7fcadc15a2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45554/0 (socket says 172.21.15.139:45554) 2024-09-16T00:08:37.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.343+0000 7fcae1672640 1 -- 172.21.15.139:0/785055174 learned_addr learned my addr 172.21.15.139:0/785055174 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:37.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.343+0000 7fcae0e71640 1 -- 172.21.15.139:0/785055174 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3940379190 0 0) 0x7fcadc073e50 con 0x7fcadc15a9f0 2024-09-16T00:08:37.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.343+0000 7fcae0e71640 1 -- 172.21.15.139:0/785055174 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcac8003520 con 0x7fcadc15a9f0 2024-09-16T00:08:37.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.343+0000 7fcae0e71640 1 -- 172.21.15.139:0/785055174 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 611443477 0 0) 0x7fcacc001110 con 0x7fcadc0c7880 2024-09-16T00:08:37.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.343+0000 7fcae0e71640 1 -- 172.21.15.139:0/785055174 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcadc073e50 con 0x7fcadc0c7880 2024-09-16T00:08:37.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.343+0000 7fcae1e73640 1 --2- 172.21.15.139:0/785055174 >> v2:172.21.15.164:3300/0 conn(0x7fcadc0c8e90 0x7fcadc0c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.344+0000 7fcae1e73640 1 -- 172.21.15.139:0/785055174 >> v1:172.21.15.164:6789/0 conn(0x7fcadc15a9f0 legacy=0x7fcadc0c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.344+0000 7fcae1e73640 1 -- 172.21.15.139:0/785055174 >> v1:172.21.15.139:6789/0 conn(0x7fcadc0c7880 legacy=0x7fcadc15a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.345 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.344+0000 7fcae1e73640 1 -- 172.21.15.139:0/785055174 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcadc073890 con 0x7fcadc0c8e90 2024-09-16T00:08:37.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.345+0000 7fcae1e73640 1 --2- 172.21.15.139:0/785055174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcadc0c8e90 0x7fcadc0c9270 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009720 tx=0x7fcad802a150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cf2936d7f910c17 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.345+0000 7fcae0e71640 1 -- 172.21.15.139:0/785055174 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8037070 con 0x7fcadc0c8e90 2024-09-16T00:08:37.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.345+0000 7fcae0e71640 1 -- 172.21.15.139:0/785055174 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad802ac60 con 0x7fcadc0c8e90 2024-09-16T00:08:37.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.345+0000 7fcae0e71640 1 -- 172.21.15.139:0/785055174 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad40019f0 con 0x7fcadc0c8e90 2024-09-16T00:08:37.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.345+0000 7fcae40d7640 1 -- 172.21.15.139:0/785055174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcadc0c8e90 msgr2=0x7fcadc0c9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.345+0000 7fcae40d7640 1 --2- 172.21.15.139:0/785055174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcadc0c8e90 0x7fcadc0c9270 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009720 tx=0x7fcad802a150 comp rx=0 tx=0).stop 2024-09-16T00:08:37.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.346+0000 7fcae40d7640 1 -- 172.21.15.139:0/785055174 shutdown_connections 2024-09-16T00:08:37.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.346+0000 7fcae40d7640 1 --2- 172.21.15.139:0/785055174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcadc0c8e90 0x7fcadc0c9270 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.346+0000 7fcae40d7640 1 -- 172.21.15.139:0/785055174 >> 172.21.15.139:0/785055174 conn(0x7fcadc0c9a80 msgr2=0x7fcadc0c9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:37.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.346+0000 7fcae40d7640 1 -- 172.21.15.139:0/785055174 shutdown_connections 2024-09-16T00:08:37.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.346+0000 7fcae40d7640 1 -- 172.21.15.139:0/785055174 wait complete. 2024-09-16T00:08:37.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.347+0000 7fcae40d7640 1 Processor -- start 2024-09-16T00:08:37.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.347+0000 7fcae40d7640 1 -- start start 2024-09-16T00:08:37.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.348+0000 7fcae40d7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcadc0c7880 0x7fcadc159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.348+0000 7fcae40d7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcadc0c8e90 0x7fcadc0c3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.348+0000 7fcae40d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc15a9f0 0x7fcadc14d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.348+0000 7fcae40d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc0c59c0 con 0x7fcadc15a9f0 2024-09-16T00:08:37.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.348+0000 7fcae40d7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc0c6220 con 0x7fcadc0c7880 2024-09-16T00:08:37.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.348+0000 7fcae40d7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcadc074140 con 0x7fcadc0c8e90 2024-09-16T00:08:37.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.348+0000 7fcae2674640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc15a9f0 0x7fcadc14d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.348+0000 7fcae1672640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcadc0c8e90 0x7fcadc0c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.348+0000 7fcae2674640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc15a9f0 0x7fcadc14d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49236/0 (socket says 172.21.15.139:49236) 2024-09-16T00:08:37.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.348+0000 7fcae2674640 1 -- 172.21.15.139:0/338262905 learned_addr learned my addr 172.21.15.139:0/338262905 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:37.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.348+0000 7fcae1e73640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcadc0c7880 0x7fcadc159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.349+0000 7fcae2674640 1 -- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcadc0c8e90 msgr2=0x7fcadc0c3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.349+0000 7fcae2674640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcadc0c8e90 0x7fcadc0c3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.349+0000 7fcae2674640 1 -- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcadc0c7880 msgr2=0x7fcadc159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.349+0000 7fcae2674640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcadc0c7880 0x7fcadc159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.349+0000 7fcae2674640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcadc0c2050 con 0x7fcadc15a9f0 2024-09-16T00:08:37.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.349+0000 7fcae2674640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc15a9f0 0x7fcadc14d030 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcad400cb20 tx=0x7fcad4002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.349+0000 7fcae1672640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcadc0c8e90 0x7fcadc0c3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:37.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.349+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4017070 con 0x7fcadc15a9f0 2024-09-16T00:08:37.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.349+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad4003d80 con 0x7fcadc15a9f0 2024-09-16T00:08:37.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.349+0000 7fcae40d7640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcadc150d80 con 0x7fcadc15a9f0 2024-09-16T00:08:37.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.349+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad40052b0 con 0x7fcadc15a9f0 2024-09-16T00:08:37.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.349+0000 7fcae40d7640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcadc0c2bf0 con 0x7fcadc15a9f0 2024-09-16T00:08:37.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.350+0000 7fcae40d7640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fcadc150f50 con 0x7fcadc15a9f0 2024-09-16T00:08:37.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.351+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcad401a070 con 0x7fcadc15a9f0 2024-09-16T00:08:37.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.352+0000 7fcad27fc640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcac006c2b0 0x7fcac006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.352+0000 7fcae1e73640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcac006c2b0 0x7fcac006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.352+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5915+0+0 (secure 0 0 0) 0x7fcad408ab50 con 0x7fcadc15a9f0 2024-09-16T00:08:37.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.353+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v47) ==== 54+0+16 (secure 0 0 0) 0x7fcad408f240 con 0x7fcadc15a9f0 2024-09-16T00:08:37.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.353+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fcadc0c3650 0x7fcadc14d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.353+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcadc0c2e30 con 0x7fcadc0c3650 2024-09-16T00:08:37.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.353+0000 7fcae1e73640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcac006c2b0 0x7fcac006e770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcad8034000 tx=0x7fcad802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.353+0000 7fcae1672640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fcadc0c3650 0x7fcadc14d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.354+0000 7fcae1672640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fcadc0c3650 0x7fcadc14d570 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.354+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcacc0019d0 con 0x7fcadc0c3650 2024-09-16T00:08:37.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.371+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcadc167210 con 0x7fcadc0c3650 2024-09-16T00:08:37.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.373+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcadc167210 con 0x7fcadc0c3650 2024-09-16T00:08:37.374 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.373+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fcadc0c1320 0x7fcadc0c1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.373+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fcadc0c1c40 con 0x7fcadc0c1320 2024-09-16T00:08:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.373+0000 7fcae1e73640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fcadc0c1320 0x7fcadc0c1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.373+0000 7fcae1e73640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fcadc0c1320 0x7fcadc0c1700 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.374+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcadc0c1c40 con 0x7fcadc0c1320 2024-09-16T00:08:37.378 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:37.377+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:37.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.388+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcadc1429e0 con 0x7fcadc0c1320 2024-09-16T00:08:37.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.390+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcadc1429e0 con 0x7fcadc0c1320 2024-09-16T00:08:37.391 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:37.393 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:37.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.390+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fcadc062f00 0x7fcadc143780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.390+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fcadc142700 con 0x7fcadc062f00 2024-09-16T00:08:37.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.391+0000 7fcae1672640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fcadc062f00 0x7fcadc143780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.391+0000 7fcae1672640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fcadc062f00 0x7fcadc143780 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.392+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcadc142700 con 0x7fcadc062f00 2024-09-16T00:08:37.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.405+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcadc0635b0 con 0x7fcadc062f00 2024-09-16T00:08:37.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.408+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcadc0635b0 con 0x7fcadc062f00 2024-09-16T00:08:37.409 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:37.410 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:37.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.408+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fcadc1fc1d0 0x7fcadc1fe5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.408+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fcadc142420 con 0x7fcadc1fc1d0 2024-09-16T00:08:37.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.408+0000 7fcae2674640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fcadc1fc1d0 0x7fcadc1fe5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.409+0000 7fcae2674640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fcadc1fc1d0 0x7fcadc1fe5c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.409+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcadc142420 con 0x7fcadc1fc1d0 2024-09-16T00:08:37.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.436+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcadc1feb00 con 0x7fcadc1fc1d0 2024-09-16T00:08:37.438 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:37.438 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:37.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.438+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcadc1feb00 con 0x7fcadc1fc1d0 2024-09-16T00:08:37.439 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:37.440 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:37.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.438+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fcadc1ff330 0x7fcadc201720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.438+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fcadc2044f0 con 0x7fcadc1ff330 2024-09-16T00:08:37.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.438+0000 7fcae1e73640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fcadc1ff330 0x7fcadc201720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.439+0000 7fcae1e73640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fcadc1ff330 0x7fcadc201720 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.440+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcadc2044f0 con 0x7fcadc1ff330 2024-09-16T00:08:37.447 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:08:37.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.465+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcadc201c60 con 0x7fcadc1ff330 2024-09-16T00:08:37.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.467+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcadc201c60 con 0x7fcadc1ff330 2024-09-16T00:08:37.468 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:37.469 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:37.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.468+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fcadc2022e0 0x7fcadc2046d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.468+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fcadc207570 con 0x7fcadc2022e0 2024-09-16T00:08:37.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.468+0000 7fcae1672640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fcadc2022e0 0x7fcadc2046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.468+0000 7fcae1672640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fcadc2022e0 0x7fcadc2046d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.469+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcadc207570 con 0x7fcadc2022e0 2024-09-16T00:08:37.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.477+0000 7f2596d91640 1 Processor -- start 2024-09-16T00:08:37.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.478+0000 7f2596d91640 1 -- start start 2024-09-16T00:08:37.479 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:37.478+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:37.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.479+0000 7f2596d91640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f25900c67e0 0x7f25900c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.479+0000 7f2596d91640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f259006c110 0x7f259015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.479+0000 7f2596d91640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f259006c560 0x7f259015ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.479+0000 7f2596d91640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f259006cd10 con 0x7f25900c67e0 2024-09-16T00:08:37.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.479+0000 7f2596d91640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f25900c6bc0 con 0x7f259006c110 2024-09-16T00:08:37.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.479+0000 7f2596d91640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f259015e850 con 0x7f259006c560 2024-09-16T00:08:37.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.479+0000 7f2594b2d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f25900c67e0 0x7f25900c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.479+0000 7f2594b2d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f25900c67e0 0x7f25900c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49244/0 (socket says 172.21.15.139:49244) 2024-09-16T00:08:37.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.479+0000 7f2594b2d640 1 -- 172.21.15.139:0/1559807787 learned_addr learned my addr 172.21.15.139:0/1559807787 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:37.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.479+0000 7f259532e640 1 --2- 172.21.15.139:0/1559807787 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f259006c560 0x7f259015ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.479+0000 7f258ffff640 1 --2- 172.21.15.139:0/1559807787 >> v2:172.21.15.164:3300/0 conn(0x7f259006c110 0x7f259015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.480+0000 7f2594b2d640 1 -- 172.21.15.139:0/1559807787 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f259006c560 msgr2=0x7f259015ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:37.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.480+0000 7f2594b2d640 1 --2- 172.21.15.139:0/1559807787 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f259006c560 0x7f259015ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.480+0000 7f2594b2d640 1 -- 172.21.15.139:0/1559807787 >> v2:172.21.15.164:3300/0 conn(0x7f259006c110 msgr2=0x7f259015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:37.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.480+0000 7f2594b2d640 1 --2- 172.21.15.139:0/1559807787 >> v2:172.21.15.164:3300/0 conn(0x7f259006c110 0x7f259015a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.480+0000 7f2594b2d640 1 -- 172.21.15.139:0/1559807787 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259006ca30 con 0x7f25900c67e0 2024-09-16T00:08:37.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.481+0000 7f2594b2d640 1 --2- 172.21.15.139:0/1559807787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25900c67e0 0x7f25900c7b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2590158440 tx=0x7f258402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e8e6358f22a149 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.481+0000 7f258f7fe640 1 -- 172.21.15.139:0/1559807787 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2584037070 con 0x7f25900c67e0 2024-09-16T00:08:37.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.481+0000 7f258f7fe640 1 -- 172.21.15.139:0/1559807787 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2584032440 con 0x7f25900c67e0 2024-09-16T00:08:37.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.481+0000 7f258f7fe640 1 -- 172.21.15.139:0/1559807787 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2584032810 con 0x7f25900c67e0 2024-09-16T00:08:37.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.482+0000 7f2596d91640 1 -- 172.21.15.139:0/1559807787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25900c67e0 msgr2=0x7f25900c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.482+0000 7f2596d91640 1 --2- 172.21.15.139:0/1559807787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25900c67e0 0x7f25900c7b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2590158440 tx=0x7f258402a270 comp rx=0 tx=0).stop 2024-09-16T00:08:37.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.482+0000 7f2596d91640 1 -- 172.21.15.139:0/1559807787 shutdown_connections 2024-09-16T00:08:37.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.482+0000 7f2596d91640 1 --2- 172.21.15.139:0/1559807787 >> v2:172.21.15.164:3300/0 conn(0x7f259006c110 0x7f259015a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.482+0000 7f2596d91640 1 --2- 172.21.15.139:0/1559807787 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f259006c560 0x7f259015ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.482+0000 7f2596d91640 1 --2- 172.21.15.139:0/1559807787 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25900c67e0 0x7f25900c7b90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.482+0000 7f2596d91640 1 -- 172.21.15.139:0/1559807787 >> 172.21.15.139:0/1559807787 conn(0x7f25900d05f0 msgr2=0x7f25900d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:37.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.482+0000 7f2596d91640 1 -- 172.21.15.139:0/1559807787 shutdown_connections 2024-09-16T00:08:37.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.482+0000 7f2596d91640 1 -- 172.21.15.139:0/1559807787 wait complete. 2024-09-16T00:08:37.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.484+0000 7f2596d91640 1 Processor -- start 2024-09-16T00:08:37.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.484+0000 7f2596d91640 1 -- start start 2024-09-16T00:08:37.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.486+0000 7f2596d91640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f259006c110 0x7f259015c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.486+0000 7f2596d91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f259006c560 0x7f259015b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.486+0000 7f258ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f259006c560 0x7f259015b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.486+0000 7f258ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f259006c560 0x7f259015b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49252/0 (socket says 172.21.15.139:49252) 2024-09-16T00:08:37.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.486+0000 7f2596d91640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25900c67e0 0x7f2590150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.486+0000 7f258ffff640 1 -- 172.21.15.139:0/2957892134 learned_addr learned my addr 172.21.15.139:0/2957892134 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:37.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.486+0000 7f2596d91640 1 -- 172.21.15.139:0/2957892134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f259015e4e0 con 0x7f259006c560 2024-09-16T00:08:37.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.486+0000 7f2596d91640 1 -- 172.21.15.139:0/2957892134 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f259015e200 con 0x7f25900c67e0 2024-09-16T00:08:37.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.486+0000 7f2596d91640 1 -- 172.21.15.139:0/2957892134 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f25900cbe50 con 0x7f259006c110 2024-09-16T00:08:37.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.486+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcadc000950 con 0x7fcadc2022e0 2024-09-16T00:08:37.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.486+0000 7f2594b2d640 1 --2- 172.21.15.139:0/2957892134 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f259006c110 0x7f259015c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.486+0000 7f259532e640 1 --2- 172.21.15.139:0/2957892134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25900c67e0 0x7f2590150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.487+0000 7f258ffff640 1 -- 172.21.15.139:0/2957892134 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f259006c110 msgr2=0x7f259015c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.487+0000 7f258ffff640 1 --2- 172.21.15.139:0/2957892134 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f259006c110 0x7f259015c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.487+0000 7f258ffff640 1 -- 172.21.15.139:0/2957892134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25900c67e0 msgr2=0x7f2590150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.487+0000 7f258ffff640 1 --2- 172.21.15.139:0/2957892134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25900c67e0 0x7f2590150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.487+0000 7f258ffff640 1 -- 172.21.15.139:0/2957892134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2590151230 con 0x7f259006c560 2024-09-16T00:08:37.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.487+0000 7f258ffff640 1 --2- 172.21.15.139:0/2957892134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f259006c560 0x7f259015b010 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f257c00d830 tx=0x7f257c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.487+0000 7f258d7fa640 1 -- 172.21.15.139:0/2957892134 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c00b6c0 con 0x7f259006c560 2024-09-16T00:08:37.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.487+0000 7f258d7fa640 1 -- 172.21.15.139:0/2957892134 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257c019070 con 0x7f259006c560 2024-09-16T00:08:37.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.487+0000 7f258d7fa640 1 -- 172.21.15.139:0/2957892134 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c00ba80 con 0x7f259006c560 2024-09-16T00:08:37.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.487+0000 7f2596d91640 1 -- 172.21.15.139:0/2957892134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25901f0f20 con 0x7f259006c560 2024-09-16T00:08:37.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.487+0000 7f2596d91640 1 -- 172.21.15.139:0/2957892134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f259015d030 con 0x7f259006c560 2024-09-16T00:08:37.493 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:37.493 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:37.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.488+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcadc000950 con 0x7fcadc2022e0 2024-09-16T00:08:37.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.488+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fcadc205120 0x7fcadc205560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.488+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fcadc20a990 con 0x7fcadc205120 2024-09-16T00:08:37.493 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.489+0000 7fcae2674640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fcadc205120 0x7fcadc205560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.489+0000 7fcae2674640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fcadc205120 0x7fcadc205560 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.489+0000 7f258d7fa640 1 -- 172.21.15.139:0/2957892134 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f257c00bc20 con 0x7f259006c560 2024-09-16T00:08:37.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.490+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcadc20a990 con 0x7fcadc205120 2024-09-16T00:08:37.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.490+0000 7f258d7fa640 1 --2- 172.21.15.139:0/2957892134 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f257406c2b0 0x7f257406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.490+0000 7f2596d91640 1 -- 172.21.15.139:0/2957892134 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25900c7b90 con 0x7f259006c560 2024-09-16T00:08:37.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.490+0000 7f258d7fa640 1 -- 172.21.15.139:0/2957892134 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5915+0+0 (secure 0 0 0) 0x7f257c08b890 con 0x7f259006c560 2024-09-16T00:08:37.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.490+0000 7f2594b2d640 1 --2- 172.21.15.139:0/2957892134 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f257406c2b0 0x7f257406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.491+0000 7f2594b2d640 1 --2- 172.21.15.139:0/2957892134 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f257406c2b0 0x7f257406e770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2584034000 tx=0x7f258402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.499+0000 7f258d7fa640 1 -- 172.21.15.139:0/2957892134 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f257c0598c0 con 0x7f259006c560 2024-09-16T00:08:37.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.517+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcadc205bb0 con 0x7fcadc205120 2024-09-16T00:08:37.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.518+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcadc205bb0 con 0x7fcadc205120 2024-09-16T00:08:37.519 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:37.520 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:37.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.519+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fcadc206230 0x7fcadc206670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.519+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fcadc206cf0 con 0x7fcadc206230 2024-09-16T00:08:37.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.519+0000 7fcae1e73640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fcadc206230 0x7fcadc206670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.519+0000 7fcae1e73640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fcadc206230 0x7fcadc206670 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.521+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcadc206cf0 con 0x7fcadc206230 2024-09-16T00:08:37.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.548+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcadc207570 con 0x7fcadc206230 2024-09-16T00:08:37.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.549+0000 7fcad27fc640 1 -- 172.21.15.139:0/338262905 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcadc207570 con 0x7fcadc206230 2024-09-16T00:08:37.550 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fcadc0c3650 msgr2=0x7fcadc14d570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fcadc0c3650 0x7fcadc14d570 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fcadc0c1320 msgr2=0x7fcadc0c1700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fcadc0c1320 0x7fcadc0c1700 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fcadc062f00 msgr2=0x7fcadc143780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fcadc062f00 0x7fcadc143780 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fcadc1fc1d0 msgr2=0x7fcadc1fe5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fcadc1fc1d0 0x7fcadc1fe5c0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fcadc1ff330 msgr2=0x7fcadc201720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fcadc1ff330 0x7fcadc201720 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fcadc2022e0 msgr2=0x7fcadc2046d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fcadc2022e0 0x7fcadc2046d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fcadc205120 msgr2=0x7fcadc205560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fcadc205120 0x7fcadc205560 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fcadc206230 msgr2=0x7fcadc206670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fcadc206230 0x7fcadc206670 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcae2674640 1 -- 172.21.15.139:0/338262905 reap_dead start 2024-09-16T00:08:37.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcae2674640 1 -- 172.21.15.139:0/338262905 reap_dead start 2024-09-16T00:08:37.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcae2674640 1 -- 172.21.15.139:0/338262905 reap_dead start 2024-09-16T00:08:37.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcae2674640 1 -- 172.21.15.139:0/338262905 reap_dead start 2024-09-16T00:08:37.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcae2674640 1 -- 172.21.15.139:0/338262905 reap_dead start 2024-09-16T00:08:37.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcac006c2b0 msgr2=0x7fcac006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.550+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcac006c2b0 0x7fcac006e770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcad8034000 tx=0x7fcad802e000 comp rx=0 tx=0).stop 2024-09-16T00:08:37.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.551+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc15a9f0 msgr2=0x7fcadc14d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.551+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc15a9f0 0x7fcadc14d030 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcad400cb20 tx=0x7fcad4002170 comp rx=0 tx=0).stop 2024-09-16T00:08:37.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.551+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 shutdown_connections 2024-09-16T00:08:37.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.551+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcac006c2b0 0x7fcac006e770 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.551+0000 7fcabffff640 1 --2- 172.21.15.139:0/338262905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc15a9f0 0x7fcadc14d030 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.551+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 >> 172.21.15.139:0/338262905 conn(0x7fcadc0c9a80 msgr2=0x7fcadc0c9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:37.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.552+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 shutdown_connections 2024-09-16T00:08:37.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.552+0000 7fcabffff640 1 -- 172.21.15.139:0/338262905 wait complete. 2024-09-16T00:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:37.555 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:37.556 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:37.556 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:37.561 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:08:37.608 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:37.608+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:37.669 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:37.669 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:37.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:37.670 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:37.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.670 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:37.670 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:37.670 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.052567965000001, 2024-09-16T00:08:37.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056100969999999997, 2024-09-16T00:08:37.670 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.670 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.670 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:37.670 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:37.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:37.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615352+0000", 2024-09-16T00:08:37.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9859999999999999e-06 2024-09-16T00:08:37.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615363+0000", 2024-09-16T00:08:37.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1168999999999999e-05 2024-09-16T00:08:37.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615382+0000", 2024-09-16T00:08:37.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8224e-05 2024-09-16T00:08:37.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615393+0000", 2024-09-16T00:08:37.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0925e-05 2024-09-16T00:08:37.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615624+0000", 2024-09-16T00:08:37.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023159600000000001 2024-09-16T00:08:37.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615721+0000", 2024-09-16T00:08:37.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7040000000000006e-05 2024-09-16T00:08:37.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:37.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615885+0000", 2024-09-16T00:08:37.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163456 2024-09-16T00:08:37.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:37.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:08:37.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002191368 2024-09-16T00:08:37.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:37.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620870+0000", 2024-09-16T00:08:37.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941839999999999 2024-09-16T00:08:37.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:37.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620922+0000", 2024-09-16T00:08:37.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1641999999999998e-05 2024-09-16T00:08:37.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620956+0000", 2024-09-16T00:08:37.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4507e-05 2024-09-16T00:08:37.675 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.676 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.676 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.676 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:37.676 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:37.676 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.052477605999997, 2024-09-16T00:08:37.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005695739, 2024-09-16T00:08:37.676 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.676 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.676 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:37.676 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:37.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:37.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:37.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4900000000000005e-07 2024-09-16T00:08:37.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615442+0000", 2024-09-16T00:08:37.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.994e-06 2024-09-16T00:08:37.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:08:37.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1081e-05 2024-09-16T00:08:37.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615473+0000", 2024-09-16T00:08:37.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1910000000000006e-06 2024-09-16T00:08:37.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616230+0000", 2024-09-16T00:08:37.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075700400000000003 2024-09-16T00:08:37.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616305+0000", 2024-09-16T00:08:37.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4759999999999996e-05 2024-09-16T00:08:37.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:37.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616466+0000", 2024-09-16T00:08:37.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000161714 2024-09-16T00:08:37.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:37.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619821+0000", 2024-09-16T00:08:37.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033547339999999998 2024-09-16T00:08:37.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:37.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621028+0000", 2024-09-16T00:08:37.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012067530000000001 2024-09-16T00:08:37.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:37.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621087+0000", 2024-09-16T00:08:37.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9364999999999998e-05 2024-09-16T00:08:37.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621132+0000", 2024-09-16T00:08:37.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5293999999999997e-05 2024-09-16T00:08:37.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.682 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.682 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:37.682 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:37.682 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.052415373999999, 2024-09-16T00:08:37.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050565760000000001, 2024-09-16T00:08:37.682 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.682 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.682 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:37.682 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:37.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:37.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615500+0000", 2024-09-16T00:08:37.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8199999999999999e-07 2024-09-16T00:08:37.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615505+0000", 2024-09-16T00:08:37.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5280000000000003e-06 2024-09-16T00:08:37.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:37.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0317e-05 2024-09-16T00:08:37.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615523+0000", 2024-09-16T00:08:37.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4580000000000004e-06 2024-09-16T00:08:37.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615607+0000", 2024-09-16T00:08:37.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4153999999999994e-05 2024-09-16T00:08:37.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615748+0000", 2024-09-16T00:08:37.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140533 2024-09-16T00:08:37.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:37.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615911+0000", 2024-09-16T00:08:37.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163678 2024-09-16T00:08:37.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:37.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619944+0000", 2024-09-16T00:08:37.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040330909999999999 2024-09-16T00:08:37.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:37.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620471+0000", 2024-09-16T00:08:37.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052645600000000002 2024-09-16T00:08:37.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:37.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620512+0000", 2024-09-16T00:08:37.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0738000000000002e-05 2024-09-16T00:08:37.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620555+0000", 2024-09-16T00:08:37.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3841000000000003e-05 2024-09-16T00:08:37.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.687 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.688 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:37.688 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:37.688 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.052363266999997, 2024-09-16T00:08:37.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089162479999999999, 2024-09-16T00:08:37.688 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.688 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.688 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:37.688 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:37.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:37.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615552+0000", 2024-09-16T00:08:37.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:08:37.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615556+0000", 2024-09-16T00:08:37.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3610000000000003e-06 2024-09-16T00:08:37.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615566+0000", 2024-09-16T00:08:37.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6099999999999995e-06 2024-09-16T00:08:37.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615574+0000", 2024-09-16T00:08:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9379999999999992e-06 2024-09-16T00:08:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616777+0000", 2024-09-16T00:08:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012031640000000001 2024-09-16T00:08:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616882+0000", 2024-09-16T00:08:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105146 2024-09-16T00:08:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619696+0000", 2024-09-16T00:08:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028142179999999998 2024-09-16T00:08:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624234+0000", 2024-09-16T00:08:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045381889999999998 2024-09-16T00:08:37.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624393+0000", 2024-09-16T00:08:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015871200000000001 2024-09-16T00:08:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624434+0000", 2024-09-16T00:08:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0908999999999997e-05 2024-09-16T00:08:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624467+0000", 2024-09-16T00:08:37.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3217999999999999e-05 2024-09-16T00:08:37.694 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.694 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.694 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.694 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:37.694 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:37.694 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.052293470000002, 2024-09-16T00:08:37.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064898359999999997, 2024-09-16T00:08:37.694 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.694 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.696 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:37.696 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:37.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:37.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615622+0000", 2024-09-16T00:08:37.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.181e-06 2024-09-16T00:08:37.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615628+0000", 2024-09-16T00:08:37.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2450000000000003e-06 2024-09-16T00:08:37.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:08:37.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2554999999999999e-05 2024-09-16T00:08:37.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615650+0000", 2024-09-16T00:08:37.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4350000000000003e-06 2024-09-16T00:08:37.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616772+0000", 2024-09-16T00:08:37.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001122058 2024-09-16T00:08:37.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616889+0000", 2024-09-16T00:08:37.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116623 2024-09-16T00:08:37.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:37.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617035+0000", 2024-09-16T00:08:37.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014579799999999999 2024-09-16T00:08:37.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:37.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619829+0000", 2024-09-16T00:08:37.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941379999999998 2024-09-16T00:08:37.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:37.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622024+0000", 2024-09-16T00:08:37.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021949920000000002 2024-09-16T00:08:37.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:37.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622087+0000", 2024-09-16T00:08:37.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3048999999999999e-05 2024-09-16T00:08:37.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622111+0000", 2024-09-16T00:08:37.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3762000000000001e-05 2024-09-16T00:08:37.701 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.701 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.701 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.701 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:37.701 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:37.701 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.052230909000002, 2024-09-16T00:08:37.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065192160000000004, 2024-09-16T00:08:37.702 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.702 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.702 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:37.702 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:37.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:37.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615684+0000", 2024-09-16T00:08:37.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8299999999999995e-07 2024-09-16T00:08:37.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615691+0000", 2024-09-16T00:08:37.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.866e-06 2024-09-16T00:08:37.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615707+0000", 2024-09-16T00:08:37.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6073999999999999e-05 2024-09-16T00:08:37.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615714+0000", 2024-09-16T00:08:37.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-16T00:08:37.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617403+0000", 2024-09-16T00:08:37.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016881629999999999 2024-09-16T00:08:37.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617559+0000", 2024-09-16T00:08:37.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015669699999999999 2024-09-16T00:08:37.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:37.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:37.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000144165 2024-09-16T00:08:37.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:37.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621413+0000", 2024-09-16T00:08:37.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037092200000000001 2024-09-16T00:08:37.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:37.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622147+0000", 2024-09-16T00:08:37.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073468800000000005 2024-09-16T00:08:37.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:37.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622180+0000", 2024-09-16T00:08:37.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2901999999999998e-05 2024-09-16T00:08:37.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622203+0000", 2024-09-16T00:08:37.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2211000000000001e-05 2024-09-16T00:08:37.706 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.707 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.707 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.707 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:37.707 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:37.707 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.052187815000003, 2024-09-16T00:08:37.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065825270000000003, 2024-09-16T00:08:37.707 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.707 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.707 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:37.707 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:37.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:37.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:08:37.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7299999999999996e-07 2024-09-16T00:08:37.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615730+0000", 2024-09-16T00:08:37.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.004e-06 2024-09-16T00:08:37.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615737+0000", 2024-09-16T00:08:37.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4789999999999999e-06 2024-09-16T00:08:37.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615745+0000", 2024-09-16T00:08:37.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6699999999999994e-06 2024-09-16T00:08:37.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618115+0000", 2024-09-16T00:08:37.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023701569999999999 2024-09-16T00:08:37.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618247+0000", 2024-09-16T00:08:37.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013185000000000001 2024-09-16T00:08:37.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622309+0000", 2024-09-16T00:08:37.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040617939999999996 2024-09-16T00:08:37.711 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.711 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.711 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.711 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:37.711 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:37.711 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.052160565999998, 2024-09-16T00:08:37.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036685950000000002, 2024-09-16T00:08:37.711 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.711 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.711 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:37.711 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:37.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:37.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:37.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T00:08:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615763+0000", 2024-09-16T00:08:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4360000000000002e-06 2024-09-16T00:08:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615770+0000", 2024-09-16T00:08:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7540000000000006e-06 2024-09-16T00:08:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615803+0000", 2024-09-16T00:08:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3008e-05 2024-09-16T00:08:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615888+0000", 2024-09-16T00:08:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5056999999999996e-05 2024-09-16T00:08:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:08:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013059700000000001 2024-09-16T00:08:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619422+0000", 2024-09-16T00:08:37.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034033230000000002 2024-09-16T00:08:37.715 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.715 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.715 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.715 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing_reply(2.1a epoch 22/19 rep_tid 2 lcod 22'1)", 2024-09-16T00:08:37.715 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:37.715 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.048719253000002, 2024-09-16T00:08:37.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024304200000000001, 2024-09-16T00:08:37.715 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.715 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.715 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:37.716 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:37.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:37.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619196+0000", 2024-09-16T00:08:37.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2759999999999999e-06 2024-09-16T00:08:37.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619203+0000", 2024-09-16T00:08:37.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1779999999999997e-06 2024-09-16T00:08:37.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619223+0000", 2024-09-16T00:08:37.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9159000000000001e-05 2024-09-16T00:08:37.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619231+0000", 2024-09-16T00:08:37.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7830000000000004e-06 2024-09-16T00:08:37.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619320+0000", 2024-09-16T00:08:37.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8805000000000001e-05 2024-09-16T00:08:37.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619438+0000", 2024-09-16T00:08:37.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117841 2024-09-16T00:08:37.718 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.719 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.719 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.719 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:16 2.1d e22/19 2:b871830b:::605.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:37.719 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:37.719 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.048171975999999, 2024-09-16T00:08:37.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045616900000000002, 2024-09-16T00:08:37.719 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.719 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.719 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:37.719 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:37.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:37.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:08:37.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:08:37.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619755+0000", 2024-09-16T00:08:37.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3599999999999996e-06 2024-09-16T00:08:37.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619767+0000", 2024-09-16T00:08:37.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2805e-05 2024-09-16T00:08:37.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619787+0000", 2024-09-16T00:08:37.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9264000000000001e-05 2024-09-16T00:08:37.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619869+0000", 2024-09-16T00:08:37.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2639999999999995e-05 2024-09-16T00:08:37.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619895+0000", 2024-09-16T00:08:37.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5890000000000001e-05 2024-09-16T00:08:37.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:37.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624264+0000", 2024-09-16T00:08:37.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043690049999999996 2024-09-16T00:08:37.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624304+0000", 2024-09-16T00:08:37.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9807e-05 2024-09-16T00:08:37.723 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.723 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.723 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.723 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:17 2.1d e22/19 2:b871830b:::605.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:37.723 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:37.723 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.047819267000001, 2024-09-16T00:08:37.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042370910000000001, 2024-09-16T00:08:37.724 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.724 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.724 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:37.724 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:37.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:37.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620096+0000", 2024-09-16T00:08:37.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:08:37.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620102+0000", 2024-09-16T00:08:37.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7220000000000002e-06 2024-09-16T00:08:37.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620112+0000", 2024-09-16T00:08:37.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0156999999999999e-05 2024-09-16T00:08:37.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620120+0000", 2024-09-16T00:08:37.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2880000000000004e-06 2024-09-16T00:08:37.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620171+0000", 2024-09-16T00:08:37.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.168e-05 2024-09-16T00:08:37.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620189+0000", 2024-09-16T00:08:37.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7419e-05 2024-09-16T00:08:37.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:37.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624306+0000", 2024-09-16T00:08:37.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041175430000000004 2024-09-16T00:08:37.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624332+0000", 2024-09-16T00:08:37.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5717000000000001e-05 2024-09-16T00:08:37.727 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.728 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.728 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.728 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:24 2.7 e22/19 ondisk, result = 0)", 2024-09-16T00:08:37.728 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:37.728 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.047553559999997, 2024-09-16T00:08:37.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000208018, 2024-09-16T00:08:37.728 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.728 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.728 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:37.728 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:37.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:37.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:37.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8500000000000001e-07 2024-09-16T00:08:37.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620366+0000", 2024-09-16T00:08:37.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5379999999999996e-06 2024-09-16T00:08:37.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620378+0000", 2024-09-16T00:08:37.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.204e-05 2024-09-16T00:08:37.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620385+0000", 2024-09-16T00:08:37.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.413e-06 2024-09-16T00:08:37.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620455+0000", 2024-09-16T00:08:37.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9886000000000001e-05 2024-09-16T00:08:37.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620463+0000", 2024-09-16T00:08:37.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1559999999999995e-06 2024-09-16T00:08:37.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620569+0000", 2024-09-16T00:08:37.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001054 2024-09-16T00:08:37.732 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.732 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.732 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.732 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:18 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:37.732 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:37.732 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.047168096, 2024-09-16T00:08:37.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021768100000000001, 2024-09-16T00:08:37.732 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.732 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.732 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:37.732 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:37.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:37.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620747+0000", 2024-09-16T00:08:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.02e-07 2024-09-16T00:08:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620752+0000", 2024-09-16T00:08:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4439999999999999e-06 2024-09-16T00:08:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620765+0000", 2024-09-16T00:08:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2714000000000001e-05 2024-09-16T00:08:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:08:37.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1160000000000004e-06 2024-09-16T00:08:37.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:37.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3374999999999994e-05 2024-09-16T00:08:37.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620864+0000", 2024-09-16T00:08:37.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4320000000000005e-06 2024-09-16T00:08:37.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620964+0000", 2024-09-16T00:08:37.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9698000000000007e-05 2024-09-16T00:08:37.736 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.736 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.736 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.736 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:19 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:37.736 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:37.736 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.047106552000002, 2024-09-16T00:08:37.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00033551400000000003, 2024-09-16T00:08:37.737 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.737 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.737 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:37.737 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:37.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:37.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:37.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0800000000000005e-07 2024-09-16T00:08:37.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620812+0000", 2024-09-16T00:08:37.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.427e-06 2024-09-16T00:08:37.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620820+0000", 2024-09-16T00:08:37.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6649999999999992e-06 2024-09-16T00:08:37.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620828+0000", 2024-09-16T00:08:37.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0010000000000001e-06 2024-09-16T00:08:37.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620995+0000", 2024-09-16T00:08:37.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016709400000000001 2024-09-16T00:08:37.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621005+0000", 2024-09-16T00:08:37.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7170000000000003e-06 2024-09-16T00:08:37.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621143+0000", 2024-09-16T00:08:37.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000138102 2024-09-16T00:08:37.740 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.740 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.740 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.740 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:3 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:37.740 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:37.740 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.045929280999999, 2024-09-16T00:08:37.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022247799999999999, 2024-09-16T00:08:37.741 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.741 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.741 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:37.741 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:37.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:37.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:37.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3300000000000003e-07 2024-09-16T00:08:37.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621989+0000", 2024-09-16T00:08:37.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2969999999999999e-06 2024-09-16T00:08:37.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621997+0000", 2024-09-16T00:08:37.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8990000000000001e-06 2024-09-16T00:08:37.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622002+0000", 2024-09-16T00:08:37.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5439999999999998e-06 2024-09-16T00:08:37.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622137+0000", 2024-09-16T00:08:37.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000135103 2024-09-16T00:08:37.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622143+0000", 2024-09-16T00:08:37.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9200000000000001e-06 2024-09-16T00:08:37.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622207+0000", 2024-09-16T00:08:37.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4282000000000006e-05 2024-09-16T00:08:37.744 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.744 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.744 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.745 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing_reply(2.14 epoch 22/19 rep_tid 6 lcod 22'5)", 2024-09-16T00:08:37.745 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:37.745 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.045897945, 2024-09-16T00:08:37.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029688000000000002, 2024-09-16T00:08:37.745 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.745 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.745 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:37.745 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:37.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:37.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622017+0000", 2024-09-16T00:08:37.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3200000000000001e-07 2024-09-16T00:08:37.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:08:37.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6240000000000002e-06 2024-09-16T00:08:37.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622026+0000", 2024-09-16T00:08:37.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.708e-06 2024-09-16T00:08:37.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622032+0000", 2024-09-16T00:08:37.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4699999999999999e-06 2024-09-16T00:08:37.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622230+0000", 2024-09-16T00:08:37.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019783800000000001 2024-09-16T00:08:37.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622313+0000", 2024-09-16T00:08:37.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2908e-05 2024-09-16T00:08:37.748 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.748 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.748 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.748 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:37.749 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:37.749 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.045452449999999, 2024-09-16T00:08:37.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044651079999999998, 2024-09-16T00:08:37.749 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.749 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.749 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:37.749 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:37.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622463+0000", 2024-09-16T00:08:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5349999999999999e-06 2024-09-16T00:08:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622470+0000", 2024-09-16T00:08:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7050000000000002e-06 2024-09-16T00:08:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622485+0000", 2024-09-16T00:08:37.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4978e-05 2024-09-16T00:08:37.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622493+0000", 2024-09-16T00:08:37.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2139999999999996e-06 2024-09-16T00:08:37.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622546+0000", 2024-09-16T00:08:37.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:08:37.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622595+0000", 2024-09-16T00:08:37.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9112e-05 2024-09-16T00:08:37.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:37.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622677+0000", 2024-09-16T00:08:37.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2332999999999999e-05 2024-09-16T00:08:37.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:37.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.623515+0000", 2024-09-16T00:08:37.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000837615 2024-09-16T00:08:37.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:37.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626867+0000", 2024-09-16T00:08:37.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033524819999999999 2024-09-16T00:08:37.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:37.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626902+0000", 2024-09-16T00:08:37.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4310000000000002e-05 2024-09-16T00:08:37.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626927+0000", 2024-09-16T00:08:37.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5128999999999999e-05 2024-09-16T00:08:37.754 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.754 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.754 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.754 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:39 2.6 e22/19 2:654134d2:::mds0_openfiles.0:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:37.754 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:37.754 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.026823172999997, 2024-09-16T00:08:37.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082120900000000004, 2024-09-16T00:08:37.754 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.754 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.754 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:37.754 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:37.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:37.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641092+0000", 2024-09-16T00:08:37.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.141e-06 2024-09-16T00:08:37.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641102+0000", 2024-09-16T00:08:37.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5990000000000008e-06 2024-09-16T00:08:37.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641116+0000", 2024-09-16T00:08:37.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4667999999999999e-05 2024-09-16T00:08:37.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641125+0000", 2024-09-16T00:08:37.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0849999999999997e-06 2024-09-16T00:08:37.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641188+0000", 2024-09-16T00:08:37.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3119000000000003e-05 2024-09-16T00:08:37.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641209+0000", 2024-09-16T00:08:37.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0897e-05 2024-09-16T00:08:37.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:37.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641874+0000", 2024-09-16T00:08:37.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00066517200000000001 2024-09-16T00:08:37.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641912+0000", 2024-09-16T00:08:37.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8528000000000001e-05 2024-09-16T00:08:37.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.758 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.759 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:40 2.14 2:2e2fa760:::200.00000001:head [write 940~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:37.759 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:37.759 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.828587057, 2024-09-16T00:08:37.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016643039999999999, 2024-09-16T00:08:37.759 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.759 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.759 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:37.759 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:37.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:37.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839329+0000", 2024-09-16T00:08:37.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2330000000000001e-06 2024-09-16T00:08:37.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839343+0000", 2024-09-16T00:08:37.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3634e-05 2024-09-16T00:08:37.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839362+0000", 2024-09-16T00:08:37.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9321999999999999e-05 2024-09-16T00:08:37.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839375+0000", 2024-09-16T00:08:37.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2945000000000001e-05 2024-09-16T00:08:37.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839462+0000", 2024-09-16T00:08:37.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6371000000000004e-05 2024-09-16T00:08:37.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839547+0000", 2024-09-16T00:08:37.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4988999999999996e-05 2024-09-16T00:08:37.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:37.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839657+0000", 2024-09-16T00:08:37.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110501 2024-09-16T00:08:37.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:37.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840598+0000", 2024-09-16T00:08:37.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00094087100000000003 2024-09-16T00:08:37.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:37.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840918+0000", 2024-09-16T00:08:37.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031968799999999999 2024-09-16T00:08:37.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:37.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840955+0000", 2024-09-16T00:08:37.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7360999999999997e-05 2024-09-16T00:08:37.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840991+0000", 2024-09-16T00:08:37.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6389e-05 2024-09-16T00:08:37.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.764 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.765 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.765 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:41 2.14 2:2e2fa760:::200.00000001:head [write 1383~2831 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:37.765 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:37.765 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.554933611999999, 2024-09-16T00:08:37.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017075809999999999, 2024-09-16T00:08:37.765 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:37.765 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:37.765 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:37.765 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:37.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:37.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:37.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:37.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:37.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:37.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:37.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112983+0000", 2024-09-16T00:08:37.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2560000000000001e-06 2024-09-16T00:08:37.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:37.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112993+0000", 2024-09-16T00:08:37.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0423e-05 2024-09-16T00:08:37.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:37.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113010+0000", 2024-09-16T00:08:37.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7169000000000001e-05 2024-09-16T00:08:37.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:37.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113023+0000", 2024-09-16T00:08:37.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2111e-05 2024-09-16T00:08:37.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:37.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113099+0000", 2024-09-16T00:08:37.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6798000000000006e-05 2024-09-16T00:08:37.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:37.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113174+0000", 2024-09-16T00:08:37.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.47e-05 2024-09-16T00:08:37.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:37.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113297+0000", 2024-09-16T00:08:37.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012250799999999999 2024-09-16T00:08:37.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:37.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114284+0000", 2024-09-16T00:08:37.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00098720399999999999 2024-09-16T00:08:37.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:37.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114604+0000", 2024-09-16T00:08:37.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031988699999999998 2024-09-16T00:08:37.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:37.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114644+0000", 2024-09-16T00:08:37.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9919999999999997e-05 2024-09-16T00:08:37.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:37.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:37.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:37.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114688+0000", 2024-09-16T00:08:37.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4604999999999997e-05 2024-09-16T00:08:37.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.770 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:37.770 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:37.770 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:37.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.684+0000 7f2596d91640 1 -- 172.21.15.139:0/2957892134 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25900cbc30 con 0x7f257406c2b0 2024-09-16T00:08:37.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.695+0000 7f258d7fa640 1 -- 172.21.15.139:0/2957892134 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251828 (secure 0 0 0) 0x7f25900cbc30 con 0x7f257406c2b0 2024-09-16T00:08:37.773 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:08:37.775 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:08:37.776 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:08:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.699+0000 7f2596d91640 1 -- 172.21.15.139:0/2957892134 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f257406c2b0 msgr2=0x7f257406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.699+0000 7f2596d91640 1 --2- 172.21.15.139:0/2957892134 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f257406c2b0 0x7f257406e770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2584034000 tx=0x7f258402e000 comp rx=0 tx=0).stop 2024-09-16T00:08:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.699+0000 7f2596d91640 1 -- 172.21.15.139:0/2957892134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f259006c560 msgr2=0x7f259015b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.699+0000 7f2596d91640 1 --2- 172.21.15.139:0/2957892134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f259006c560 0x7f259015b010 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f257c00d830 tx=0x7f257c00dcf0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.699+0000 7f2596d91640 1 -- 172.21.15.139:0/2957892134 shutdown_connections 2024-09-16T00:08:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.699+0000 7f2596d91640 1 --2- 172.21.15.139:0/2957892134 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f257406c2b0 0x7f257406e770 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.699+0000 7f2596d91640 1 --2- 172.21.15.139:0/2957892134 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25900c67e0 0x7f2590150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.699+0000 7f2596d91640 1 --2- 172.21.15.139:0/2957892134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f259006c560 0x7f259015b010 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.699+0000 7f2596d91640 1 --2- 172.21.15.139:0/2957892134 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f259006c110 0x7f259015c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.699+0000 7f2596d91640 1 -- 172.21.15.139:0/2957892134 >> 172.21.15.139:0/2957892134 conn(0x7f25900d05f0 msgr2=0x7f25900d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:37.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.699+0000 7f2596d91640 1 -- 172.21.15.139:0/2957892134 shutdown_connections 2024-09-16T00:08:37.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.700+0000 7f2596d91640 1 -- 172.21.15.139:0/2957892134 wait complete. 2024-09-16T00:08:37.780 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-09-16T00:08:36.880498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597721,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":212,"num_write_kb":2439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":6555,"num_keys_recovered":2,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220200,"kb_used_data":3944,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754520,"statfs":{"total":773094113280,"available":772868628480,"internally_reserved":0,"allocated":4038656,"data_stored":2838298,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022623"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":54,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839717+0000","last_change":"2024-09-16T00:08:24.838661+0000","last_active":"2024-09-16T00:08:30.839717+0000","last_peered":"2024-09-16T00:08:30.839717+0000","last_clean":"2024-09-16T00:08:30.839717+0000","last_became_active":"2024-09-16T00:08:24.838273+0000","last_became_peered":"2024-09-16T00:08:24.838273+0000","last_unstale":"2024-09-16T00:08:30.839717+0000","last_undegraded":"2024-09-16T00:08:30.839717+0000","last_fullsized":"2024-09-16T00:08:30.839717+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:32.768723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882794+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:08:35.882794+0000","last_peered":"2024-09-16T00:08:35.882794+0000","last_clean":"2024-09-16T00:08:35.882794+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:08:35.882794+0000","last_undegraded":"2024-09-16T00:08:35.882794+0000","last_fullsized":"2024-09-16T00:08:35.882794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.890336+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:08:35.890336+0000","last_peered":"2024-09-16T00:08:35.890336+0000","last_clean":"2024-09-16T00:08:35.890336+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:08:35.890336+0000","last_undegraded":"2024-09-16T00:08:35.890336+0000","last_fullsized":"2024-09-16T00:08:35.890336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847983+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:08:31.847983+0000","last_peered":"2024-09-16T00:08:31.847983+0000","last_clean":"2024-09-16T00:08:31.847983+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:08:31.847983+0000","last_undegraded":"2024-09-16T00:08:31.847983+0000","last_fullsized":"2024-09-16T00:08:31.847983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882059+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:08:35.882059+0000","last_peered":"2024-09-16T00:08:35.882059+0000","last_clean":"2024-09-16T00:08:35.882059+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:08:35.882059+0000","last_undegraded":"2024-09-16T00:08:35.882059+0000","last_fullsized":"2024-09-16T00:08:35.882059+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847578+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:08:31.847578+0000","last_peered":"2024-09-16T00:08:31.847578+0000","last_clean":"2024-09-16T00:08:31.847578+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:08:31.847578+0000","last_undegraded":"2024-09-16T00:08:31.847578+0000","last_fullsized":"2024-09-16T00:08:31.847578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839199+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:08:30.839199+0000","last_peered":"2024-09-16T00:08:30.839199+0000","last_clean":"2024-09-16T00:08:30.839199+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:08:30.839199+0000","last_undegraded":"2024-09-16T00:08:30.839199+0000","last_fullsized":"2024-09-16T00:08:30.839199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":32,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874260+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:08:34.874260+0000","last_peered":"2024-09-16T00:08:34.874260+0000","last_clean":"2024-09-16T00:08:34.874260+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:08:34.874260+0000","last_undegraded":"2024-09-16T00:08:34.874260+0000","last_fullsized":"2024-09-16T00:08:34.874260+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839738+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:08:30.839738+0000","last_peered":"2024-09-16T00:08:30.839738+0000","last_clean":"2024-09-16T00:08:30.839738+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:08:30.839738+0000","last_undegraded":"2024-09-16T00:08:30.839738+0000","last_fullsized":"2024-09-16T00:08:30.839738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.848014+0000","last_change":"2024-09-16T00:08:24.835738+0000","last_active":"2024-09-16T00:08:31.848014+0000","last_peered":"2024-09-16T00:08:31.848014+0000","last_clean":"2024-09-16T00:08:31.848014+0000","last_became_active":"2024-09-16T00:08:24.835368+0000","last_became_peered":"2024-09-16T00:08:24.835368+0000","last_unstale":"2024-09-16T00:08:31.848014+0000","last_undegraded":"2024-09-16T00:08:31.848014+0000","last_fullsized":"2024-09-16T00:08:31.848014+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:40.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847765+0000","last_change":"2024-09-16T00:08:24.840483+0000","last_active":"2024-09-16T00:08:31.847765+0000","last_peered":"2024-09-16T00:08:31.847765+0000","last_clean":"2024-09-16T00:08:31.847765+0000","last_became_active":"2024-09-16T00:08:24.840085+0000","last_became_peered":"2024-09-16T00:08:24.840085+0000","last_unstale":"2024-09-16T00:08:31.847765+0000","last_undegraded":"2024-09-16T00:08:31.847765+0000","last_fullsized":"2024-09-16T00:08:31.847765+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:55.997551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":54,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831329+0000","last_change":"2024-09-16T00:08:24.839776+0000","last_active":"2024-09-16T00:08:31.831329+0000","last_peered":"2024-09-16T00:08:31.831329+0000","last_clean":"2024-09-16T00:08:31.831329+0000","last_became_active":"2024-09-16T00:08:24.839415+0000","last_became_peered":"2024-09-16T00:08:24.839415+0000","last_unstale":"2024-09-16T00:08:31.831329+0000","last_undegraded":"2024-09-16T00:08:31.831329+0000","last_fullsized":"2024-09-16T00:08:31.831329+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:44.442568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"peering","last_fresh":"2024-09-16T00:08:35.190711+0000","last_change":"2024-09-16T00:08:35.892992+0000","last_active":"2024-09-16T00:08:35.892992+0000","last_peered":"2024-09-16T00:08:35.190711+0000","last_clean":"2024-09-16T00:08:35.190711+0000","last_became_active":"2024-09-16T00:08:24.840637+0000","last_became_peered":"2024-09-16T00:08:24.840637+0000","last_unstale":"2024-09-16T00:08:35.190711+0000","last_undegraded":"2024-09-16T00:08:35.190711+0000","last_fullsized":"2024-09-16T00:08:35.190711+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888448+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:08:35.888448+0000","last_peered":"2024-09-16T00:08:35.888448+0000","last_clean":"2024-09-16T00:08:35.888448+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:08:35.888448+0000","last_undegraded":"2024-09-16T00:08:35.888448+0000","last_fullsized":"2024-09-16T00:08:35.888448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881726+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:08:35.881726+0000","last_peered":"2024-09-16T00:08:35.881726+0000","last_clean":"2024-09-16T00:08:35.881726+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:08:35.881726+0000","last_undegraded":"2024-09-16T00:08:35.881726+0000","last_fullsized":"2024-09-16T00:08:35.881726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847508+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:08:31.847508+0000","last_peered":"2024-09-16T00:08:31.847508+0000","last_clean":"2024-09-16T00:08:31.847508+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:08:31.847508+0000","last_undegraded":"2024-09-16T00:08:31.847508+0000","last_fullsized":"2024-09-16T00:08:31.847508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.848137+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:08:31.848137+0000","last_peered":"2024-09-16T00:08:31.848137+0000","last_clean":"2024-09-16T00:08:31.848137+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:08:31.848137+0000","last_undegraded":"2024-09-16T00:08:31.848137+0000","last_fullsized":"2024-09-16T00:08:31.848137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874802+0000","last_change":"2024-09-16T00:08:24.836818+0000","last_active":"2024-09-16T00:08:34.874802+0000","last_peered":"2024-09-16T00:08:34.874802+0000","last_clean":"2024-09-16T00:08:34.874802+0000","last_became_active":"2024-09-16T00:08:24.836029+0000","last_became_peered":"2024-09-16T00:08:24.836029+0000","last_unstale":"2024-09-16T00:08:34.874802+0000","last_undegraded":"2024-09-16T00:08:34.874802+0000","last_fullsized":"2024-09-16T00:08:34.874802+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:38:55.816393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839639+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:08:30.839639+0000","last_peered":"2024-09-16T00:08:30.839639+0000","last_clean":"2024-09-16T00:08:30.839639+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:08:30.839639+0000","last_undegraded":"2024-09-16T00:08:30.839639+0000","last_fullsized":"2024-09-16T00:08:30.839639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831171+0000","last_change":"2024-09-16T00:08:24.839823+0000","last_active":"2024-09-16T00:08:31.831171+0000","last_peered":"2024-09-16T00:08:31.831171+0000","last_clean":"2024-09-16T00:08:31.831171+0000","last_became_active":"2024-09-16T00:08:24.839507+0000","last_became_peered":"2024-09-16T00:08:24.839507+0000","last_unstale":"2024-09-16T00:08:31.831171+0000","last_undegraded":"2024-09-16T00:08:31.831171+0000","last_fullsized":"2024-09-16T00:08:31.831171+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888586+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:08:35.888586+0000","last_peered":"2024-09-16T00:08:35.888586+0000","last_clean":"2024-09-16T00:08:35.888586+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:08:35.888586+0000","last_undegraded":"2024-09-16T00:08:35.888586+0000","last_fullsized":"2024-09-16T00:08:35.888586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847663+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:08:31.847663+0000","last_peered":"2024-09-16T00:08:31.847663+0000","last_clean":"2024-09-16T00:08:31.847663+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:08:31.847663+0000","last_undegraded":"2024-09-16T00:08:31.847663+0000","last_fullsized":"2024-09-16T00:08:31.847663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.891321+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:08:35.891321+0000","last_peered":"2024-09-16T00:08:35.891321+0000","last_clean":"2024-09-16T00:08:35.891321+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:08:35.891321+0000","last_undegraded":"2024-09-16T00:08:35.891321+0000","last_fullsized":"2024-09-16T00:08:35.891321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847518+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:08:31.847518+0000","last_peered":"2024-09-16T00:08:31.847518+0000","last_clean":"2024-09-16T00:08:31.847518+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:08:31.847518+0000","last_undegraded":"2024-09-16T00:08:31.847518+0000","last_fullsized":"2024-09-16T00:08:31.847518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.894305+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:08:35.894305+0000","last_peered":"2024-09-16T00:08:35.894305+0000","last_clean":"2024-09-16T00:08:35.894305+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:08:35.894305+0000","last_undegraded":"2024-09-16T00:08:35.894305+0000","last_fullsized":"2024-09-16T00:08:35.894305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847213+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:08:31.847213+0000","last_peered":"2024-09-16T00:08:31.847213+0000","last_clean":"2024-09-16T00:08:31.847213+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:08:31.847213+0000","last_undegraded":"2024-09-16T00:08:31.847213+0000","last_fullsized":"2024-09-16T00:08:31.847213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882462+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:08:35.882462+0000","last_peered":"2024-09-16T00:08:35.882462+0000","last_clean":"2024-09-16T00:08:35.882462+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:08:35.882462+0000","last_undegraded":"2024-09-16T00:08:35.882462+0000","last_fullsized":"2024-09-16T00:08:35.882462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.892094+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:08:35.892094+0000","last_peered":"2024-09-16T00:08:35.892094+0000","last_clean":"2024-09-16T00:08:35.892094+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:08:35.892094+0000","last_undegraded":"2024-09-16T00:08:35.892094+0000","last_fullsized":"2024-09-16T00:08:35.892094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888666+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:08:35.888666+0000","last_peered":"2024-09-16T00:08:35.888666+0000","last_clean":"2024-09-16T00:08:35.888666+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:08:35.888666+0000","last_undegraded":"2024-09-16T00:08:35.888666+0000","last_fullsized":"2024-09-16T00:08:35.888666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.875423+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:08:34.875423+0000","last_peered":"2024-09-16T00:08:34.875423+0000","last_clean":"2024-09-16T00:08:34.875423+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:08:34.875423+0000","last_undegraded":"2024-09-16T00:08:34.875423+0000","last_fullsized":"2024-09-16T00:08:34.875423+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874580+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:08:34.874580+0000","last_peered":"2024-09-16T00:08:34.874580+0000","last_clean":"2024-09-16T00:08:34.874580+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:08:34.874580+0000","last_undegraded":"2024-09-16T00:08:34.874580+0000","last_fullsized":"2024-09-16T00:08:34.874580+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.025362+0000","last_change":"2024-09-16T00:08:24.839775+0000","last_active":"2024-09-16T00:08:30.025362+0000","last_peered":"2024-09-16T00:08:30.025362+0000","last_clean":"2024-09-16T00:08:30.025362+0000","last_became_active":"2024-09-16T00:08:24.839222+0000","last_became_peered":"2024-09-16T00:08:24.839222+0000","last_unstale":"2024-09-16T00:08:30.025362+0000","last_undegraded":"2024-09-16T00:08:30.025362+0000","last_fullsized":"2024-09-16T00:08:30.025362+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:33:48.945330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831152+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:08:31.831152+0000","last_peered":"2024-09-16T00:08:31.831152+0000","last_clean":"2024-09-16T00:08:31.831152+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:08:31.831152+0000","last_undegraded":"2024-09-16T00:08:31.831152+0000","last_fullsized":"2024-09-16T00:08:31.831152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":63,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.893461+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:08:35.893461+0000","last_peered":"2024-09-16T00:08:35.893461+0000","last_clean":"2024-09-16T00:08:35.893461+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:08:35.893461+0000","last_undegraded":"2024-09-16T00:08:35.893461+0000","last_fullsized":"2024-09-16T00:08:35.893461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888853+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:08:35.888853+0000","last_peered":"2024-09-16T00:08:35.888853+0000","last_clean":"2024-09-16T00:08:35.888853+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:08:35.888853+0000","last_undegraded":"2024-09-16T00:08:35.888853+0000","last_fullsized":"2024-09-16T00:08:35.888853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831068+0000","last_change":"2024-09-16T00:08:24.835401+0000","last_active":"2024-09-16T00:08:31.831068+0000","last_peered":"2024-09-16T00:08:31.831068+0000","last_clean":"2024-09-16T00:08:31.831068+0000","last_became_active":"2024-09-16T00:08:24.835017+0000","last_became_peered":"2024-09-16T00:08:24.835017+0000","last_unstale":"2024-09-16T00:08:31.831068+0000","last_undegraded":"2024-09-16T00:08:31.831068+0000","last_fullsized":"2024-09-16T00:08:31.831068+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:02.003294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"peering","last_fresh":"2024-09-16T00:08:35.189391+0000","last_change":"2024-09-16T00:08:35.890927+0000","last_active":"2024-09-16T00:08:35.890927+0000","last_peered":"2024-09-16T00:08:35.189391+0000","last_clean":"2024-09-16T00:08:35.189391+0000","last_became_active":"2024-09-16T00:08:24.838130+0000","last_became_peered":"2024-09-16T00:08:24.838130+0000","last_unstale":"2024-09-16T00:08:35.189391+0000","last_undegraded":"2024-09-16T00:08:35.189391+0000","last_fullsized":"2024-09-16T00:08:35.189391+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.892368+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:08:35.892368+0000","last_peered":"2024-09-16T00:08:35.892368+0000","last_clean":"2024-09-16T00:08:35.892368+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:08:35.892368+0000","last_undegraded":"2024-09-16T00:08:35.892368+0000","last_fullsized":"2024-09-16T00:08:35.892368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":54,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847637+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:08:31.847637+0000","last_peered":"2024-09-16T00:08:31.847637+0000","last_clean":"2024-09-16T00:08:31.847637+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:08:31.847637+0000","last_undegraded":"2024-09-16T00:08:31.847637+0000","last_fullsized":"2024-09-16T00:08:31.847637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881638+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:08:35.881638+0000","last_peered":"2024-09-16T00:08:35.881638+0000","last_clean":"2024-09-16T00:08:35.881638+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:08:35.881638+0000","last_undegraded":"2024-09-16T00:08:35.881638+0000","last_fullsized":"2024-09-16T00:08:35.881638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":54,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847413+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:08:31.847413+0000","last_peered":"2024-09-16T00:08:31.847413+0000","last_clean":"2024-09-16T00:08:31.847413+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:08:31.847413+0000","last_undegraded":"2024-09-16T00:08:31.847413+0000","last_fullsized":"2024-09-16T00:08:31.847413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847960+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:08:31.847960+0000","last_peered":"2024-09-16T00:08:31.847960+0000","last_clean":"2024-09-16T00:08:31.847960+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:08:31.847960+0000","last_undegraded":"2024-09-16T00:08:31.847960+0000","last_fullsized":"2024-09-16T00:08:31.847960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":54,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.848115+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:08:31.848115+0000","last_peered":"2024-09-16T00:08:31.848115+0000","last_clean":"2024-09-16T00:08:31.848115+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:08:31.848115+0000","last_undegraded":"2024-09-16T00:08:31.848115+0000","last_fullsized":"2024-09-16T00:08:31.848115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.891379+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:08:35.891379+0000","last_peered":"2024-09-16T00:08:35.891379+0000","last_clean":"2024-09-16T00:08:35.891379+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:08:35.891379+0000","last_undegraded":"2024-09-16T00:08:35.891379+0000","last_fullsized":"2024-09-16T00:08:35.891379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":64,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882552+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:08:35.882552+0000","last_peered":"2024-09-16T00:08:35.882552+0000","last_clean":"2024-09-16T00:08:35.882552+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:08:35.882552+0000","last_undegraded":"2024-09-16T00:08:35.882552+0000","last_fullsized":"2024-09-16T00:08:35.882552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831420+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:08:31.831420+0000","last_peered":"2024-09-16T00:08:31.831420+0000","last_clean":"2024-09-16T00:08:31.831420+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:08:31.831420+0000","last_undegraded":"2024-09-16T00:08:31.831420+0000","last_fullsized":"2024-09-16T00:08:31.831420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"28'10","reported_seq":60,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.840236+0000","last_change":"2024-09-16T00:08:24.854457+0000","last_active":"2024-09-16T00:08:30.840236+0000","last_peered":"2024-09-16T00:08:30.840236+0000","last_clean":"2024-09-16T00:08:30.840236+0000","last_became_active":"2024-09-16T00:08:24.837069+0000","last_became_peered":"2024-09-16T00:08:24.837069+0000","last_unstale":"2024-09-16T00:08:30.840236+0000","last_undegraded":"2024-09-16T00:08:30.840236+0000","last_fullsized":"2024-09-16T00:08:30.840236+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:44:04.061152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6555,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":6555,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839507+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:08:30.839507+0000","last_peered":"2024-09-16T00:08:30.839507+0000","last_clean":"2024-09-16T00:08:30.839507+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:08:30.839507+0000","last_undegraded":"2024-09-16T00:08:30.839507+0000","last_fullsized":"2024-09-16T00:08:30.839507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":63,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888938+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:08:35.888938+0000","last_peered":"2024-09-16T00:08:35.888938+0000","last_clean":"2024-09-16T00:08:35.888938+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:08:35.888938+0000","last_undegraded":"2024-09-16T00:08:35.888938+0000","last_fullsized":"2024-09-16T00:08:35.888938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888319+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:08:35.888319+0000","last_peered":"2024-09-16T00:08:35.888319+0000","last_clean":"2024-09-16T00:08:35.888319+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:08:35.888319+0000","last_undegraded":"2024-09-16T00:08:35.888319+0000","last_fullsized":"2024-09-16T00:08:35.888319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":54,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847693+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:08:31.847693+0000","last_peered":"2024-09-16T00:08:31.847693+0000","last_clean":"2024-09-16T00:08:31.847693+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:08:31.847693+0000","last_undegraded":"2024-09-16T00:08:31.847693+0000","last_fullsized":"2024-09-16T00:08:31.847693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839384+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:08:30.839384+0000","last_peered":"2024-09-16T00:08:30.839384+0000","last_clean":"2024-09-16T00:08:30.839384+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:08:30.839384+0000","last_undegraded":"2024-09-16T00:08:30.839384+0000","last_fullsized":"2024-09-16T00:08:30.839384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831022+0000","last_change":"2024-09-16T00:08:24.834930+0000","last_active":"2024-09-16T00:08:31.831022+0000","last_peered":"2024-09-16T00:08:31.831022+0000","last_clean":"2024-09-16T00:08:31.831022+0000","last_became_active":"2024-09-16T00:08:24.834556+0000","last_became_peered":"2024-09-16T00:08:24.834556+0000","last_unstale":"2024-09-16T00:08:31.831022+0000","last_undegraded":"2024-09-16T00:08:31.831022+0000","last_fullsized":"2024-09-16T00:08:31.831022+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:28.660189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.025267+0000","last_change":"2024-09-16T00:08:24.839984+0000","last_active":"2024-09-16T00:08:30.025267+0000","last_peered":"2024-09-16T00:08:30.025267+0000","last_clean":"2024-09-16T00:08:30.025267+0000","last_became_active":"2024-09-16T00:08:24.839497+0000","last_became_peered":"2024-09-16T00:08:24.839497+0000","last_unstale":"2024-09-16T00:08:30.025267+0000","last_undegraded":"2024-09-16T00:08:30.025267+0000","last_fullsized":"2024-09-16T00:08:30.025267+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874425+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:08:34.874425+0000","last_peered":"2024-09-16T00:08:34.874425+0000","last_clean":"2024-09-16T00:08:34.874425+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:08:34.874425+0000","last_undegraded":"2024-09-16T00:08:34.874425+0000","last_fullsized":"2024-09-16T00:08:34.874425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839214+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:08:30.839214+0000","last_peered":"2024-09-16T00:08:30.839214+0000","last_clean":"2024-09-16T00:08:30.839214+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:08:30.839214+0000","last_undegraded":"2024-09-16T00:08:30.839214+0000","last_fullsized":"2024-09-16T00:08:30.839214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":52,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831448+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:08:31.831448+0000","last_peered":"2024-09-16T00:08:31.831448+0000","last_clean":"2024-09-16T00:08:31.831448+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:08:31.831448+0000","last_undegraded":"2024-09-16T00:08:31.831448+0000","last_fullsized":"2024-09-16T00:08:31.831448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839647+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:08:30.839647+0000","last_peered":"2024-09-16T00:08:30.839647+0000","last_clean":"2024-09-16T00:08:30.839647+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:08:30.839647+0000","last_undegraded":"2024-09-16T00:08:30.839647+0000","last_fullsized":"2024-09-16T00:08:30.839647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":62,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874882+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:08:34.874882+0000","last_peered":"2024-09-16T00:08:34.874882+0000","last_clean":"2024-09-16T00:08:34.874882+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:08:34.874882+0000","last_undegraded":"2024-09-16T00:08:34.874882+0000","last_fullsized":"2024-09-16T00:08:34.874882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839795+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:08:30.839795+0000","last_peered":"2024-09-16T00:08:30.839795+0000","last_clean":"2024-09-16T00:08:30.839795+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:08:30.839795+0000","last_undegraded":"2024-09-16T00:08:30.839795+0000","last_fullsized":"2024-09-16T00:08:30.839795+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":62,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882644+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:08:35.882644+0000","last_peered":"2024-09-16T00:08:35.882644+0000","last_clean":"2024-09-16T00:08:35.882644+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:08:35.882644+0000","last_undegraded":"2024-09-16T00:08:35.882644+0000","last_fullsized":"2024-09-16T00:08:35.882644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.848103+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:08:31.848103+0000","last_peered":"2024-09-16T00:08:31.848103+0000","last_clean":"2024-09-16T00:08:31.848103+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:08:31.848103+0000","last_undegraded":"2024-09-16T00:08:31.848103+0000","last_fullsized":"2024-09-16T00:08:31.848103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.875230+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:08:34.875230+0000","last_peered":"2024-09-16T00:08:34.875230+0000","last_clean":"2024-09-16T00:08:34.875230+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:08:34.875230+0000","last_undegraded":"2024-09-16T00:08:34.875230+0000","last_fullsized":"2024-09-16T00:08:34.875230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":54,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839539+0000","last_change":"2024-09-16T00:08:15.795467+0000","last_active":"2024-09-16T00:08:30.839539+0000","last_peered":"2024-09-16T00:08:30.839539+0000","last_clean":"2024-09-16T00:08:30.839539+0000","last_became_active":"2024-09-16T00:08:15.795167+0000","last_became_peered":"2024-09-16T00:08:15.795167+0000","last_unstale":"2024-09-16T00:08:30.839539+0000","last_undegraded":"2024-09-16T00:08:30.839539+0000","last_fullsized":"2024-09-16T00:08:30.839539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":207,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839287+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:08:30.839287+0000","last_peered":"2024-09-16T00:08:30.839287+0000","last_clean":"2024-09-16T00:08:30.839287+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:08:30.839287+0000","last_undegraded":"2024-09-16T00:08:30.839287+0000","last_fullsized":"2024-09-16T00:08:30.839287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":54,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847636+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:08:31.847636+0000","last_peered":"2024-09-16T00:08:31.847636+0000","last_clean":"2024-09-16T00:08:31.847636+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:08:31.847636+0000","last_undegraded":"2024-09-16T00:08:31.847636+0000","last_fullsized":"2024-09-16T00:08:31.847636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.893654+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:08:35.893654+0000","last_peered":"2024-09-16T00:08:35.893654+0000","last_clean":"2024-09-16T00:08:35.893654+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:08:35.893654+0000","last_undegraded":"2024-09-16T00:08:35.893654+0000","last_fullsized":"2024-09-16T00:08:35.893654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839181+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:08:30.839181+0000","last_peered":"2024-09-16T00:08:30.839181+0000","last_clean":"2024-09-16T00:08:30.839181+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:08:30.839181+0000","last_undegraded":"2024-09-16T00:08:30.839181+0000","last_fullsized":"2024-09-16T00:08:30.839181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.887981+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:08:35.887981+0000","last_peered":"2024-09-16T00:08:35.887981+0000","last_clean":"2024-09-16T00:08:35.887981+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:08:35.887981+0000","last_undegraded":"2024-09-16T00:08:35.887981+0000","last_fullsized":"2024-09-16T00:08:35.887981+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874658+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:08:34.874658+0000","last_peered":"2024-09-16T00:08:34.874658+0000","last_clean":"2024-09-16T00:08:34.874658+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:08:34.874658+0000","last_undegraded":"2024-09-16T00:08:34.874658+0000","last_fullsized":"2024-09-16T00:08:34.874658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881158+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:08:35.881158+0000","last_peered":"2024-09-16T00:08:35.881158+0000","last_clean":"2024-09-16T00:08:35.881158+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:08:35.881158+0000","last_undegraded":"2024-09-16T00:08:35.881158+0000","last_fullsized":"2024-09-16T00:08:35.881158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":54,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847952+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:08:31.847952+0000","last_peered":"2024-09-16T00:08:31.847952+0000","last_clean":"2024-09-16T00:08:31.847952+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:08:31.847952+0000","last_undegraded":"2024-09-16T00:08:31.847952+0000","last_fullsized":"2024-09-16T00:08:31.847952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831392+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:08:31.831392+0000","last_peered":"2024-09-16T00:08:31.831392+0000","last_clean":"2024-09-16T00:08:31.831392+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:08:31.831392+0000","last_undegraded":"2024-09-16T00:08:31.831392+0000","last_fullsized":"2024-09-16T00:08:31.831392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882294+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:08:35.882294+0000","last_peered":"2024-09-16T00:08:35.882294+0000","last_clean":"2024-09-16T00:08:35.882294+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:08:35.882294+0000","last_undegraded":"2024-09-16T00:08:35.882294+0000","last_fullsized":"2024-09-16T00:08:35.882294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847873+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:08:31.847873+0000","last_peered":"2024-09-16T00:08:31.847873+0000","last_clean":"2024-09-16T00:08:31.847873+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:08:31.847873+0000","last_undegraded":"2024-09-16T00:08:31.847873+0000","last_fullsized":"2024-09-16T00:08:31.847873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839727+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:08:30.839727+0000","last_peered":"2024-09-16T00:08:30.839727+0000","last_clean":"2024-09-16T00:08:30.839727+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:08:30.839727+0000","last_undegraded":"2024-09-16T00:08:30.839727+0000","last_fullsized":"2024-09-16T00:08:30.839727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882179+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:08:35.882179+0000","last_peered":"2024-09-16T00:08:35.882179+0000","last_clean":"2024-09-16T00:08:35.882179+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:08:35.882179+0000","last_undegraded":"2024-09-16T00:08:35.882179+0000","last_fullsized":"2024-09-16T00:08:35.882179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"26'3","reported_seq":72,"reported_epoch":45,"state":"peering","last_fresh":"2024-09-16T00:08:35.191937+0000","last_change":"2024-09-16T00:08:35.883363+0000","last_active":"2024-09-16T00:08:35.883363+0000","last_peered":"2024-09-16T00:08:35.191937+0000","last_clean":"2024-09-16T00:08:35.191937+0000","last_became_active":"2024-09-16T00:08:24.835764+0000","last_became_peered":"2024-09-16T00:08:24.835764+0000","last_unstale":"2024-09-16T00:08:35.191937+0000","last_undegraded":"2024-09-16T00:08:35.191937+0000","last_fullsized":"2024-09-16T00:08:35.191937+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.894399+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:08:35.894399+0000","last_peered":"2024-09-16T00:08:35.894399+0000","last_clean":"2024-09-16T00:08:35.894399+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:08:35.894399+0000","last_undegraded":"2024-09-16T00:08:35.894399+0000","last_fullsized":"2024-09-16T00:08:35.894399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":56,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847288+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:08:31.847288+0000","last_peered":"2024-09-16T00:08:31.847288+0000","last_clean":"2024-09-16T00:08:31.847288+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:08:31.847288+0000","last_undegraded":"2024-09-16T00:08:31.847288+0000","last_fullsized":"2024-09-16T00:08:31.847288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.890491+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:08:35.890491+0000","last_peered":"2024-09-16T00:08:35.890491+0000","last_clean":"2024-09-16T00:08:35.890491+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:08:35.890491+0000","last_undegraded":"2024-09-16T00:08:35.890491+0000","last_fullsized":"2024-09-16T00:08:35.890491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":51,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839174+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:08:30.839174+0000","last_peered":"2024-09-16T00:08:30.839174+0000","last_clean":"2024-09-16T00:08:30.839174+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:08:30.839174+0000","last_undegraded":"2024-09-16T00:08:30.839174+0000","last_fullsized":"2024-09-16T00:08:30.839174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":49,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831075+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:08:31.831075+0000","last_peered":"2024-09-16T00:08:31.831075+0000","last_clean":"2024-09-16T00:08:31.831075+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:08:31.831075+0000","last_undegraded":"2024-09-16T00:08:31.831075+0000","last_fullsized":"2024-09-16T00:08:31.831075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":61,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881815+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:08:35.881815+0000","last_peered":"2024-09-16T00:08:35.881815+0000","last_clean":"2024-09-16T00:08:35.881815+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:08:35.881815+0000","last_undegraded":"2024-09-16T00:08:35.881815+0000","last_fullsized":"2024-09-16T00:08:35.881815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.892453+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:08:35.892453+0000","last_peered":"2024-09-16T00:08:35.892453+0000","last_clean":"2024-09-16T00:08:35.892453+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:08:35.892453+0000","last_undegraded":"2024-09-16T00:08:35.892453+0000","last_fullsized":"2024-09-16T00:08:35.892453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":61,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881852+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:08:35.881852+0000","last_peered":"2024-09-16T00:08:35.881852+0000","last_clean":"2024-09-16T00:08:35.881852+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:08:35.881852+0000","last_undegraded":"2024-09-16T00:08:35.881852+0000","last_fullsized":"2024-09-16T00:08:35.881852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.893352+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:08:35.893352+0000","last_peered":"2024-09-16T00:08:35.893352+0000","last_clean":"2024-09-16T00:08:35.893352+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:08:35.893352+0000","last_undegraded":"2024-09-16T00:08:35.893352+0000","last_fullsized":"2024-09-16T00:08:35.893352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":53,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839419+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:08:30.839419+0000","last_peered":"2024-09-16T00:08:30.839419+0000","last_clean":"2024-09-16T00:08:30.839419+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:08:30.839419+0000","last_undegraded":"2024-09-16T00:08:30.839419+0000","last_fullsized":"2024-09-16T00:08:30.839419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.831024+0000","last_change":"2024-09-16T00:08:24.835323+0000","last_active":"2024-09-16T00:08:31.831024+0000","last_peered":"2024-09-16T00:08:31.831024+0000","last_clean":"2024-09-16T00:08:31.831024+0000","last_became_active":"2024-09-16T00:08:24.834976+0000","last_became_peered":"2024-09-16T00:08:24.834976+0000","last_unstale":"2024-09-16T00:08:31.831024+0000","last_undegraded":"2024-09-16T00:08:31.831024+0000","last_fullsized":"2024-09-16T00:08:31.831024+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.847790+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:08:31.847790+0000","last_peered":"2024-09-16T00:08:31.847790+0000","last_clean":"2024-09-16T00:08:31.847790+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:08:31.847790+0000","last_undegraded":"2024-09-16T00:08:31.847790+0000","last_fullsized":"2024-09-16T00:08:31.847790+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":53,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839492+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:08:30.839492+0000","last_peered":"2024-09-16T00:08:30.839492+0000","last_clean":"2024-09-16T00:08:30.839492+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:08:30.839492+0000","last_undegraded":"2024-09-16T00:08:30.839492+0000","last_fullsized":"2024-09-16T00:08:30.839492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":52,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-16T00:08:31.848046+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:08:31.848046+0000","last_peered":"2024-09-16T00:08:31.848046+0000","last_clean":"2024-09-16T00:08:31.848046+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:08:31.848046+0000","last_undegraded":"2024-09-16T00:08:31.848046+0000","last_fullsized":"2024-09-16T00:08:31.848046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-16T00:08:30.839608+0000","last_change":"2024-09-16T00:08:24.844324+0000","last_active":"2024-09-16T00:08:30.839608+0000","last_peered":"2024-09-16T00:08:30.839608+0000","last_clean":"2024-09-16T00:08:30.839608+0000","last_became_active":"2024-09-16T00:08:24.840798+0000","last_became_peered":"2024-09-16T00:08:24.840798+0000","last_unstale":"2024-09-16T00:08:30.839608+0000","last_undegraded":"2024-09-16T00:08:30.839608+0000","last_fullsized":"2024-09-16T00:08:30.839608+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:34.657833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874986+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:08:34.874986+0000","last_peered":"2024-09-16T00:08:34.874986+0000","last_clean":"2024-09-16T00:08:34.874986+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:08:34.874986+0000","last_undegraded":"2024-09-16T00:08:34.874986+0000","last_fullsized":"2024-09-16T00:08:34.874986+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":63,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.895206+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:08:35.895206+0000","last_peered":"2024-09-16T00:08:35.895206+0000","last_clean":"2024-09-16T00:08:35.895206+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:08:35.895206+0000","last_undegraded":"2024-09-16T00:08:35.895206+0000","last_fullsized":"2024-09-16T00:08:35.895206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"peering","last_fresh":"2024-09-16T00:08:35.190279+0000","last_change":"2024-09-16T00:08:35.895010+0000","last_active":"2024-09-16T00:08:35.895010+0000","last_peered":"2024-09-16T00:08:35.190279+0000","last_clean":"2024-09-16T00:08:35.190279+0000","last_became_active":"2024-09-16T00:08:24.838285+0000","last_became_peered":"2024-09-16T00:08:24.838285+0000","last_unstale":"2024-09-16T00:08:35.190279+0000","last_undegraded":"2024-09-16T00:08:35.190279+0000","last_fullsized":"2024-09-16T00:08:35.190279+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":65,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.891464+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:08:35.891464+0000","last_peered":"2024-09-16T00:08:35.891464+0000","last_clean":"2024-09-16T00:08:35.891464+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:08:35.891464+0000","last_undegraded":"2024-09-16T00:08:35.891464+0000","last_fullsized":"2024-09-16T00:08:35.891464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7353,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":6555,"num_keys_recovered":2,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":14706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":524288,"data_stored":784061,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":199234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":475136,"data_stored":199756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":466944,"data_stored":193237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":520192,"data_stored":224158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":217763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":581632,"data_stored":808609,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":491520,"data_stored":211480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":6555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":6645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:08:37.784 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:08:37.784 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:37.771+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:37.872 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:37.872+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:37.882 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:37.882 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:37.882 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:37.882 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:37.890 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:08:37.910 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T00:08:37.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.928+0000 7f69104dd640 1 Processor -- start 2024-09-16T00:08:37.929 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.928+0000 7f69104dd640 1 -- start start 2024-09-16T00:08:37.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.929+0000 7f69104dd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6908150df0 0x7f6908073510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.929+0000 7f69104dd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f69080c3310 0x7f6908073a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.929+0000 7f69104dd640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69080c7c40 con 0x7f69080c8430 2024-09-16T00:08:37.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.929+0000 7f69104dd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6908068060 con 0x7f6908150df0 2024-09-16T00:08:37.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.929+0000 7f69104dd640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f69081511d0 con 0x7f69080c3310 2024-09-16T00:08:37.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.929+0000 7f690da78640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f69080c3310 0x7f6908073a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.929+0000 7f690da78640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f69080c3310 0x7f6908073a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:49426/0 (socket says 172.21.15.139:49426) 2024-09-16T00:08:37.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.930+0000 7f690da78640 1 -- 172.21.15.139:0/2943443182 learned_addr learned my addr 172.21.15.139:0/2943443182 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:37.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.930+0000 7f690e279640 1 --2- 172.21.15.139:0/2943443182 >> v2:172.21.15.139:3300/0 conn(0x7f6908150df0 0x7f6908073510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.930+0000 7f690da78640 1 -- 172.21.15.139:0/2943443182 >> v2:172.21.15.139:3300/0 conn(0x7f6908150df0 msgr2=0x7f6908073510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:37.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.930+0000 7f690da78640 1 --2- 172.21.15.139:0/2943443182 >> v2:172.21.15.139:3300/0 conn(0x7f6908150df0 0x7f6908073510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.930+0000 7f690da78640 1 -- 172.21.15.139:0/2943443182 >> v1:172.21.15.164:6789/0 conn(0x7f69080c8430 legacy=0x7f6908067950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.932 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.930+0000 7f690da78640 1 -- 172.21.15.139:0/2943443182 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69080c76c0 con 0x7f69080c3310 2024-09-16T00:08:37.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.931+0000 7f690da78640 1 --2- 172.21.15.139:0/2943443182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69080c3310 0x7f6908073a50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f69080c75b0 tx=0x7f68f802c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68309b1efd6f6714 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.931+0000 7f690d277640 1 -- 172.21.15.139:0/2943443182 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8038070 con 0x7f69080c3310 2024-09-16T00:08:37.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.931+0000 7f690d277640 1 -- 172.21.15.139:0/2943443182 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f8031710 con 0x7f69080c3310 2024-09-16T00:08:37.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.931+0000 7f690d277640 1 -- 172.21.15.139:0/2943443182 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8031ae0 con 0x7f69080c3310 2024-09-16T00:08:37.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.932+0000 7f69104dd640 1 -- 172.21.15.139:0/2943443182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69080c3310 msgr2=0x7f6908073a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.932+0000 7f69104dd640 1 --2- 172.21.15.139:0/2943443182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69080c3310 0x7f6908073a50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f69080c75b0 tx=0x7f68f802c510 comp rx=0 tx=0).stop 2024-09-16T00:08:37.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.932+0000 7f69104dd640 1 -- 172.21.15.139:0/2943443182 shutdown_connections 2024-09-16T00:08:37.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.932+0000 7f69104dd640 1 --2- 172.21.15.139:0/2943443182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69080c3310 0x7f6908073a50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.932+0000 7f69104dd640 1 --2- 172.21.15.139:0/2943443182 >> v2:172.21.15.139:3300/0 conn(0x7f6908150df0 0x7f6908073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.932+0000 7f69104dd640 1 -- 172.21.15.139:0/2943443182 >> 172.21.15.139:0/2943443182 conn(0x7f69080cc6d0 msgr2=0x7f69080ccab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:37.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.932+0000 7f69104dd640 1 -- 172.21.15.139:0/2943443182 shutdown_connections 2024-09-16T00:08:37.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.932+0000 7f69104dd640 1 -- 172.21.15.139:0/2943443182 wait complete. 2024-09-16T00:08:37.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.933+0000 7f69104dd640 1 Processor -- start 2024-09-16T00:08:37.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.933+0000 7f69104dd640 1 -- start start 2024-09-16T00:08:37.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.934+0000 7f69104dd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69080c3310 0x7f690814c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.934+0000 7f69104dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f69080c8430 0x7f69081f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.934+0000 7f69104dd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6908150df0 0x7f69081e29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.934+0000 7f69104dd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f69080724e0 con 0x7f69080c8430 2024-09-16T00:08:37.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.934+0000 7f69104dd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6908072200 con 0x7f69080c3310 2024-09-16T00:08:37.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.934+0000 7f69104dd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f69080c3850 con 0x7f6908150df0 2024-09-16T00:08:37.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.934+0000 7f690e279640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69080c3310 0x7f690814c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.934+0000 7f690e279640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69080c3310 0x7f690814c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:49430/0 (socket says 172.21.15.139:49430) 2024-09-16T00:08:37.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.934+0000 7f690e279640 1 -- 172.21.15.139:0/1307058562 learned_addr learned my addr 172.21.15.139:0/1307058562 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:37.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.934+0000 7f690da78640 1 --2- 172.21.15.139:0/1307058562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f69080c8430 0x7f69081f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.934+0000 7f690ea7a640 1 --2- 172.21.15.139:0/1307058562 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6908150df0 0x7f69081e29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.934+0000 7f690e279640 1 -- 172.21.15.139:0/1307058562 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6908150df0 msgr2=0x7f69081e29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.934+0000 7f690e279640 1 --2- 172.21.15.139:0/1307058562 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6908150df0 0x7f69081e29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.934+0000 7f690e279640 1 -- 172.21.15.139:0/1307058562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f69080c8430 msgr2=0x7f69081f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:37.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.934+0000 7f690e279640 1 --2- 172.21.15.139:0/1307058562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f69080c8430 0x7f69081f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:37.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.934+0000 7f690e279640 1 -- 172.21.15.139:0/1307058562 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69081f3440 con 0x7f69080c3310 2024-09-16T00:08:37.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.935+0000 7f690e279640 1 --2- 172.21.15.139:0/1307058562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69080c3310 0x7f690814c5c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f690400e870 tx=0x7f690400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.935+0000 7f68feffd640 1 -- 172.21.15.139:0/1307058562 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6904017070 con 0x7f69080c3310 2024-09-16T00:08:37.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.935+0000 7f68feffd640 1 -- 172.21.15.139:0/1307058562 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f690400c940 con 0x7f69080c3310 2024-09-16T00:08:37.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.935+0000 7f68feffd640 1 -- 172.21.15.139:0/1307058562 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690400cd30 con 0x7f69080c3310 2024-09-16T00:08:37.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.936+0000 7f69104dd640 1 -- 172.21.15.139:0/1307058562 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690814cbc0 con 0x7f69080c3310 2024-09-16T00:08:37.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.936+0000 7f69104dd640 1 -- 172.21.15.139:0/1307058562 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69081f36e0 con 0x7f69080c3310 2024-09-16T00:08:37.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.937+0000 7f69104dd640 1 -- 172.21.15.139:0/1307058562 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6908069690 con 0x7f69080c3310 2024-09-16T00:08:37.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.938+0000 7f68feffd640 1 -- 172.21.15.139:0/1307058562 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f690401a070 con 0x7f69080c3310 2024-09-16T00:08:37.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.938+0000 7f68feffd640 1 --2- 172.21.15.139:0/1307058562 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f68ec06c2b0 0x7f68ec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:37.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.939+0000 7f68feffd640 1 -- 172.21.15.139:0/1307058562 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5915+0+0 (secure 0 0 0) 0x7f690408b430 con 0x7f69080c3310 2024-09-16T00:08:37.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.939+0000 7f690da78640 1 --2- 172.21.15.139:0/1307058562 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f68ec06c2b0 0x7f68ec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:37.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.939+0000 7f690da78640 1 --2- 172.21.15.139:0/1307058562 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f68ec06c2b0 0x7f68ec06e770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f68f8035000 tx=0x7f68f8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:37.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:37.947+0000 7f68feffd640 1 -- 172.21.15.139:0/1307058562 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6904059350 con 0x7f69080c3310 2024-09-16T00:08:37.973 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:37.972+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:38.001 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:38.001 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:38.001 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:38.001 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:38.001 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:38.010 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:08:38.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.058+0000 7fb7c3217640 1 Processor -- start 2024-09-16T00:08:38.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.058+0000 7fb7c3217640 1 -- start start 2024-09-16T00:08:38.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.059+0000 7fb7c3217640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb7bc154c20 0x7fb7bc15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:38.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.059+0000 7fb7c3217640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb7bc068060 0x7fb7bc0c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:38.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.059+0000 7fb7c3217640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7bc0cc4b0 con 0x7fb7bc0ccc90 2024-09-16T00:08:38.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.059+0000 7fb7c3217640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb7bc15b7c0 con 0x7fb7bc068060 2024-09-16T00:08:38.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.059+0000 7fb7c3217640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb7bc0cc260 con 0x7fb7bc154c20 2024-09-16T00:08:38.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.059+0000 7fb7bbfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb7bc068060 0x7fb7bc0c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:38.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.059+0000 7fb7c17b4640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb7bc0ccc90 0x7fb7bc0cac00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:50604/0 (socket says 172.21.15.139:50604) 2024-09-16T00:08:38.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.059+0000 7fb7c17b4640 1 -- 172.21.15.139:0/2355024455 learned_addr learned my addr 172.21.15.139:0/2355024455 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:38.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.059+0000 7fb7bb7fe640 1 -- 172.21.15.139:0/2355024455 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 833639097 0 0) 0x7fb7bc0cc4b0 con 0x7fb7bc0ccc90 2024-09-16T00:08:38.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.059+0000 7fb7bb7fe640 1 -- 172.21.15.139:0/2355024455 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7ac003520 con 0x7fb7bc0ccc90 2024-09-16T00:08:38.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.059+0000 7fb7bbfff640 1 -- 172.21.15.139:0/2355024455 >> v2:172.21.15.164:3300/0 conn(0x7fb7bc154c20 msgr2=0x7fb7bc15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:38.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.059+0000 7fb7c0fb3640 1 --2- 172.21.15.139:0/2355024455 >> v2:172.21.15.164:3300/0 conn(0x7fb7bc154c20 0x7fb7bc15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:38.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.059+0000 7fb7bbfff640 1 --2- 172.21.15.139:0/2355024455 >> v2:172.21.15.164:3300/0 conn(0x7fb7bc154c20 0x7fb7bc15b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:38.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.059+0000 7fb7bbfff640 1 -- 172.21.15.139:0/2355024455 >> v1:172.21.15.164:6789/0 conn(0x7fb7bc0ccc90 legacy=0x7fb7bc0cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:38.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.059+0000 7fb7bbfff640 1 -- 172.21.15.139:0/2355024455 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7bc0cc790 con 0x7fb7bc068060 2024-09-16T00:08:38.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.059+0000 7fb7c0fb3640 1 --2- 172.21.15.139:0/2355024455 >> v2:172.21.15.164:3300/0 conn(0x7fb7bc154c20 0x7fb7bc15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:08:38.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.060+0000 7fb7bbfff640 1 --2- 172.21.15.139:0/2355024455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7bc068060 0x7fb7bc0c5120 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0d1290 tx=0x7fb7a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9860e71ea24a13cf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:38.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.060+0000 7fb7bb7fe640 1 -- 172.21.15.139:0/2355024455 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8037070 con 0x7fb7bc068060 2024-09-16T00:08:38.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.060+0000 7fb7bb7fe640 1 -- 172.21.15.139:0/2355024455 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a8032440 con 0x7fb7bc068060 2024-09-16T00:08:38.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.061+0000 7fb7bb7fe640 1 -- 172.21.15.139:0/2355024455 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8032810 con 0x7fb7bc068060 2024-09-16T00:08:38.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.061+0000 7fb7c3217640 1 -- 172.21.15.139:0/2355024455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7bc068060 msgr2=0x7fb7bc0c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:38.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.061+0000 7fb7c3217640 1 --2- 172.21.15.139:0/2355024455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7bc068060 0x7fb7bc0c5120 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0d1290 tx=0x7fb7a802a1a0 comp rx=0 tx=0).stop 2024-09-16T00:08:38.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.061+0000 7fb7c3217640 1 -- 172.21.15.139:0/2355024455 shutdown_connections 2024-09-16T00:08:38.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.061+0000 7fb7c3217640 1 --2- 172.21.15.139:0/2355024455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7bc068060 0x7fb7bc0c5120 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:38.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.061+0000 7fb7c3217640 1 --2- 172.21.15.139:0/2355024455 >> v2:172.21.15.164:3300/0 conn(0x7fb7bc154c20 0x7fb7bc15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:38.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.061+0000 7fb7c3217640 1 -- 172.21.15.139:0/2355024455 >> 172.21.15.139:0/2355024455 conn(0x7fb7bc06ca40 msgr2=0x7fb7bc067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:38.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.061+0000 7fb7c3217640 1 -- 172.21.15.139:0/2355024455 shutdown_connections 2024-09-16T00:08:38.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.061+0000 7fb7c3217640 1 -- 172.21.15.139:0/2355024455 wait complete. 2024-09-16T00:08:38.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.063+0000 7fb7c3217640 1 Processor -- start 2024-09-16T00:08:38.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.063+0000 7fb7c3217640 1 -- start start 2024-09-16T00:08:38.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.064+0000 7fb7c3217640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7bc068060 0x7fb7bc0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.064+0000 7fb7c3217640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7bc0ccc90 0x7fb7bc1f1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.064+0000 7fb7c3217640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7bc154c20 0x7fb7bc146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.064+0000 7fb7c3217640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc15e3b0 con 0x7fb7bc154c20 2024-09-16T00:08:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.064+0000 7fb7c3217640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc15e0d0 con 0x7fb7bc068060 2024-09-16T00:08:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.064+0000 7fb7c3217640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb7bc15e690 con 0x7fb7bc0ccc90 2024-09-16T00:08:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.064+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7bc0ccc90 0x7fb7bc1f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.064+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7bc0ccc90 0x7fb7bc1f1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:48714/0 (socket says 172.21.15.139:48714) 2024-09-16T00:08:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.064+0000 7fb7bbfff640 1 -- 172.21.15.139:0/1210806254 learned_addr learned my addr 172.21.15.139:0/1210806254 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.064+0000 7fb7c0fb3640 1 --2- 172.21.15.139:0/1210806254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7bc068060 0x7fb7bc0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.064+0000 7fb7c17b4640 1 --2- 172.21.15.139:0/1210806254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7bc154c20 0x7fb7bc146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.064+0000 7fb7bbfff640 1 -- 172.21.15.139:0/1210806254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7bc068060 msgr2=0x7fb7bc0ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.064+0000 7fb7bbfff640 1 --2- 172.21.15.139:0/1210806254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7bc068060 0x7fb7bc0ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:38.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.064+0000 7fb7bbfff640 1 -- 172.21.15.139:0/1210806254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7bc154c20 msgr2=0x7fb7bc146870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:38.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.064+0000 7fb7bbfff640 1 --2- 172.21.15.139:0/1210806254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7bc154c20 0x7fb7bc146870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:38.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.064+0000 7fb7bbfff640 1 -- 172.21.15.139:0/1210806254 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7bc1f0770 con 0x7fb7bc0ccc90 2024-09-16T00:08:38.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.065+0000 7fb7bbfff640 1 --2- 172.21.15.139:0/1210806254 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7bc0ccc90 0x7fb7bc1f1230 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009850 tx=0x7fb7a80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:38.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.065+0000 7fb7b97fa640 1 -- 172.21.15.139:0/1210806254 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8037050 con 0x7fb7bc0ccc90 2024-09-16T00:08:38.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.065+0000 7fb7c3217640 1 -- 172.21.15.139:0/1210806254 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7bc0cfb00 con 0x7fb7bc0ccc90 2024-09-16T00:08:38.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.065+0000 7fb7b97fa640 1 -- 172.21.15.139:0/1210806254 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7a8043070 con 0x7fb7bc0ccc90 2024-09-16T00:08:38.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.065+0000 7fb7c3217640 1 -- 172.21.15.139:0/1210806254 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7bc1f0a30 con 0x7fb7bc0ccc90 2024-09-16T00:08:38.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.066+0000 7fb7b97fa640 1 -- 172.21.15.139:0/1210806254 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8030cf0 con 0x7fb7bc0ccc90 2024-09-16T00:08:38.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.067+0000 7fb7c3217640 1 -- 172.21.15.139:0/1210806254 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb79c005290 con 0x7fb7bc0ccc90 2024-09-16T00:08:38.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.067+0000 7fb7b97fa640 1 -- 172.21.15.139:0/1210806254 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb7a8048460 con 0x7fb7bc0ccc90 2024-09-16T00:08:38.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.067+0000 7fb7b97fa640 1 --2- 172.21.15.139:0/1210806254 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb7a006c2b0 0x7fb7a006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:38.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.067+0000 7fb7c0fb3640 1 --2- 172.21.15.139:0/1210806254 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb7a006c2b0 0x7fb7a006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:38.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.068+0000 7fb7b97fa640 1 -- 172.21.15.139:0/1210806254 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(47..47 src has 1..47) ==== 5915+0+0 (secure 0 0 0) 0x7fb7a80ac2f0 con 0x7fb7bc0ccc90 2024-09-16T00:08:38.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.068+0000 7fb7c0fb3640 1 --2- 172.21.15.139:0/1210806254 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb7a006c2b0 0x7fb7a006e770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000ab60 tx=0x7fb7b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:38.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.072+0000 7fb7b97fa640 1 -- 172.21.15.139:0/1210806254 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb7a807a2e0 con 0x7fb7bc0ccc90 2024-09-16T00:08:38.074 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:38.073+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:38.156 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 75.073989052000002, 2024-09-16T00:08:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:08:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.157 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.158 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.158 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.158 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.158 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:08:38.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.158 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:38.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:38.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:08:38.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:08:38.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:08:38.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:08:38.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:08:38.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:08:38.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:08:38.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:08:38.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:08:38.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:08:38.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:08:38.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:08:38.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:08:38.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:08:38.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:08:38.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:08:38.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:08:38.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:08:38.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:08:38.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:08:38.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:08:38.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:08:38.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:08:38.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:08:38.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:08:38.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:08:38.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:08:38.165 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.165 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.165 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.165 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:38.165 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:38.166 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.994457421999996, 2024-09-16T00:08:38.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093071190000000008, 2024-09-16T00:08:38.166 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.166 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.166 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.166 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.166 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 106 2024-09-16T00:08:38.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.166 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:38.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:38.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160362+0000", 2024-09-16T00:08:38.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:38.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160438+0000", 2024-09-16T00:08:38.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6711000000000005e-05 2024-09-16T00:08:38.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160444+0000", 2024-09-16T00:08:38.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0530000000000003e-06 2024-09-16T00:08:38.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160449+0000", 2024-09-16T00:08:38.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2189999999999998e-06 2024-09-16T00:08:38.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161270+0000", 2024-09-16T00:08:38.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082099199999999999 2024-09-16T00:08:38.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161348+0000", 2024-09-16T00:08:38.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8361000000000004e-05 2024-09-16T00:08:38.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161472+0000", 2024-09-16T00:08:38.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123523 2024-09-16T00:08:38.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163461+0000", 2024-09-16T00:08:38.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019891560000000002 2024-09-16T00:08:38.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169418+0000", 2024-09-16T00:08:38.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059568490000000002 2024-09-16T00:08:38.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169570+0000", 2024-09-16T00:08:38.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152461 2024-09-16T00:08:38.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169668+0000", 2024-09-16T00:08:38.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T00:08:38.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.172 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.172 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.172 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:38.172 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:38.172 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.994361651000006, 2024-09-16T00:08:38.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092931679999999992, 2024-09-16T00:08:38.172 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.173 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.173 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.173 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.173 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107 2024-09-16T00:08:38.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.173 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:38.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:38.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:38.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0699999999999998e-07 2024-09-16T00:08:38.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160459+0000", 2024-09-16T00:08:38.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0370000000000001e-06 2024-09-16T00:08:38.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160463+0000", 2024-09-16T00:08:38.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999999e-06 2024-09-16T00:08:38.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160466+0000", 2024-09-16T00:08:38.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.644e-06 2024-09-16T00:08:38.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161873+0000", 2024-09-16T00:08:38.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014074840000000001 2024-09-16T00:08:38.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161949+0000", 2024-09-16T00:08:38.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5749999999999998e-05 2024-09-16T00:08:38.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162069+0000", 2024-09-16T00:08:38.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120257 2024-09-16T00:08:38.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163506+0000", 2024-09-16T00:08:38.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001436344 2024-09-16T00:08:38.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169704+0000", 2024-09-16T00:08:38.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061982829999999997 2024-09-16T00:08:38.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169730+0000", 2024-09-16T00:08:38.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6060000000000001e-05 2024-09-16T00:08:38.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169750+0000", 2024-09-16T00:08:38.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0092e-05 2024-09-16T00:08:38.178 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.178 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.178 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.178 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:38.178 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:38.178 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.994252338999999, 2024-09-16T00:08:38.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092280039999999997, 2024-09-16T00:08:38.178 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.179 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.179 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.179 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.179 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 108 2024-09-16T00:08:38.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.179 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:38.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:38.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160567+0000", 2024-09-16T00:08:38.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7300000000000001e-07 2024-09-16T00:08:38.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160569+0000", 2024-09-16T00:08:38.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5299999999999999e-06 2024-09-16T00:08:38.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160574+0000", 2024-09-16T00:08:38.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-16T00:08:38.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160577+0000", 2024-09-16T00:08:38.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.01e-06 2024-09-16T00:08:38.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162518+0000", 2024-09-16T00:08:38.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001940703 2024-09-16T00:08:38.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162571+0000", 2024-09-16T00:08:38.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2670000000000002e-05 2024-09-16T00:08:38.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162647+0000", 2024-09-16T00:08:38.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6778000000000002e-05 2024-09-16T00:08:38.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163544+0000", 2024-09-16T00:08:38.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089702599999999999 2024-09-16T00:08:38.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169754+0000", 2024-09-16T00:08:38.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062091739999999996 2024-09-16T00:08:38.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169776+0000", 2024-09-16T00:08:38.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2391999999999998e-05 2024-09-16T00:08:38.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169794+0000", 2024-09-16T00:08:38.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8318999999999999e-05 2024-09-16T00:08:38.184 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.184 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.184 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.184 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:38.184 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:38.184 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.994203788999997, 2024-09-16T00:08:38.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092220679999999999, 2024-09-16T00:08:38.184 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.185 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.185 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.185 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.185 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.185 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.185 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 109 2024-09-16T00:08:38.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.185 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:38.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:38.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:38.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9900000000000001e-07 2024-09-16T00:08:38.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160707+0000", 2024-09-16T00:08:38.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1621999999999998e-05 2024-09-16T00:08:38.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160713+0000", 2024-09-16T00:08:38.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.596e-06 2024-09-16T00:08:38.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160717+0000", 2024-09-16T00:08:38.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5829999999999998e-06 2024-09-16T00:08:38.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162869+0000", 2024-09-16T00:08:38.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021516310000000002 2024-09-16T00:08:38.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162946+0000", 2024-09-16T00:08:38.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7696999999999998e-05 2024-09-16T00:08:38.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163065+0000", 2024-09-16T00:08:38.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118581 2024-09-16T00:08:38.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164431+0000", 2024-09-16T00:08:38.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001366003 2024-09-16T00:08:38.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169797+0000", 2024-09-16T00:08:38.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005366244 2024-09-16T00:08:38.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169819+0000", 2024-09-16T00:08:38.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2036999999999999e-05 2024-09-16T00:08:38.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169837+0000", 2024-09-16T00:08:38.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7575e-05 2024-09-16T00:08:38.190 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.190 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.190 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.190 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:38.190 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:38.190 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.994094439999998, 2024-09-16T00:08:38.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092871919999999997, 2024-09-16T00:08:38.190 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.191 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.191 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.191 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.191 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.191 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.191 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 110 2024-09-16T00:08:38.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.191 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:38.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:38.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:38.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:08:38.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160728+0000", 2024-09-16T00:08:38.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0620000000000001e-06 2024-09-16T00:08:38.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160731+0000", 2024-09-16T00:08:38.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6500000000000002e-06 2024-09-16T00:08:38.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160734+0000", 2024-09-16T00:08:38.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.762e-06 2024-09-16T00:08:38.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163580+0000", 2024-09-16T00:08:38.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028464549999999999 2024-09-16T00:08:38.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163657+0000", 2024-09-16T00:08:38.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6695999999999999e-05 2024-09-16T00:08:38.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163798+0000", 2024-09-16T00:08:38.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014055 2024-09-16T00:08:38.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165483+0000", 2024-09-16T00:08:38.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016853090000000001 2024-09-16T00:08:38.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169970+0000", 2024-09-16T00:08:38.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00448709 2024-09-16T00:08:38.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169994+0000", 2024-09-16T00:08:38.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3575999999999999e-05 2024-09-16T00:08:38.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170011+0000", 2024-09-16T00:08:38.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7702000000000001e-05 2024-09-16T00:08:38.196 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.196 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.196 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.196 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:38.196 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:38.196 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.994080771, 2024-09-16T00:08:38.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093174859999999998, 2024-09-16T00:08:38.197 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.197 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.197 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.197 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.197 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 111 2024-09-16T00:08:38.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.197 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:38.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:38.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:38.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1e-07 2024-09-16T00:08:38.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160741+0000", 2024-09-16T00:08:38.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6110000000000001e-06 2024-09-16T00:08:38.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160744+0000", 2024-09-16T00:08:38.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3299999999999999e-06 2024-09-16T00:08:38.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160746+0000", 2024-09-16T00:08:38.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5469999999999999e-06 2024-09-16T00:08:38.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164069+0000", 2024-09-16T00:08:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033222709999999999 2024-09-16T00:08:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164113+0000", 2024-09-16T00:08:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4212999999999998e-05 2024-09-16T00:08:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164187+0000", 2024-09-16T00:08:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4136999999999998e-05 2024-09-16T00:08:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166429+0000", 2024-09-16T00:08:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022423180000000001 2024-09-16T00:08:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170016+0000", 2024-09-16T00:08:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035869449999999998 2024-09-16T00:08:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170038+0000", 2024-09-16T00:08:38.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2053e-05 2024-09-16T00:08:38.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170055+0000", 2024-09-16T00:08:38.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6850999999999998e-05 2024-09-16T00:08:38.202 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.202 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.202 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.202 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:38.202 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:38.203 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.994067056000006, 2024-09-16T00:08:38.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093480279999999995, 2024-09-16T00:08:38.203 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.203 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.203 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.203 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.203 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.203 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.203 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 112 2024-09-16T00:08:38.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.203 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:38.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:38.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160752+0000", 2024-09-16T00:08:38.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2700000000000001e-07 2024-09-16T00:08:38.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160808+0000", 2024-09-16T00:08:38.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5797999999999997e-05 2024-09-16T00:08:38.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160814+0000", 2024-09-16T00:08:38.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5980000000000002e-06 2024-09-16T00:08:38.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160819+0000", 2024-09-16T00:08:38.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8960000000000002e-06 2024-09-16T00:08:38.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164468+0000", 2024-09-16T00:08:38.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003649476 2024-09-16T00:08:38.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164525+0000", 2024-09-16T00:08:38.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6227999999999998e-05 2024-09-16T00:08:38.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164595+0000", 2024-09-16T00:08:38.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759000000000005e-05 2024-09-16T00:08:38.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167237+0000", 2024-09-16T00:08:38.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026410470000000001 2024-09-16T00:08:38.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170060+0000", 2024-09-16T00:08:38.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028234969999999999 2024-09-16T00:08:38.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170081+0000", 2024-09-16T00:08:38.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1242000000000001e-05 2024-09-16T00:08:38.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170100+0000", 2024-09-16T00:08:38.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8260000000000001e-05 2024-09-16T00:08:38.208 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.208 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.208 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.208 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:38.208 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:38.208 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.993989960999997, 2024-09-16T00:08:38.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093139409999999992, 2024-09-16T00:08:38.209 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.209 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.209 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.209 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.209 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.209 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.209 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 113 2024-09-16T00:08:38.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.209 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:38.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:38.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:38.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-07 2024-09-16T00:08:38.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160833+0000", 2024-09-16T00:08:38.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.557e-06 2024-09-16T00:08:38.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160839+0000", 2024-09-16T00:08:38.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3829999999999997e-06 2024-09-16T00:08:38.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160844+0000", 2024-09-16T00:08:38.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6909999999999996e-06 2024-09-16T00:08:38.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165008+0000", 2024-09-16T00:08:38.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041642129999999999 2024-09-16T00:08:38.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165085+0000", 2024-09-16T00:08:38.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6651999999999997e-05 2024-09-16T00:08:38.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165205+0000", 2024-09-16T00:08:38.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012045900000000001 2024-09-16T00:08:38.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167283+0000", 2024-09-16T00:08:38.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020782909999999999 2024-09-16T00:08:38.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170104+0000", 2024-09-16T00:08:38.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028207950000000001 2024-09-16T00:08:38.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170125+0000", 2024-09-16T00:08:38.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0852e-05 2024-09-16T00:08:38.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170143+0000", 2024-09-16T00:08:38.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7467000000000001e-05 2024-09-16T00:08:38.214 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.214 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.214 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.214 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:38.214 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:38.214 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.993943478999995, 2024-09-16T00:08:38.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093126960000000005, 2024-09-16T00:08:38.215 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.215 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.215 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.215 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.215 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-16T00:08:38.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.215 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:38.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:38.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160876+0000", 2024-09-16T00:08:38.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5700000000000002e-07 2024-09-16T00:08:38.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160880+0000", 2024-09-16T00:08:38.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5539999999999999e-06 2024-09-16T00:08:38.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160888+0000", 2024-09-16T00:08:38.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.157e-06 2024-09-16T00:08:38.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160894+0000", 2024-09-16T00:08:38.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7690000000000002e-06 2024-09-16T00:08:38.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165519+0000", 2024-09-16T00:08:38.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046249539999999997 2024-09-16T00:08:38.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165593+0000", 2024-09-16T00:08:38.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4039e-05 2024-09-16T00:08:38.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165715+0000", 2024-09-16T00:08:38.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012131899999999999 2024-09-16T00:08:38.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167309+0000", 2024-09-16T00:08:38.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001594011 2024-09-16T00:08:38.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170147+0000", 2024-09-16T00:08:38.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028387130000000001 2024-09-16T00:08:38.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170170+0000", 2024-09-16T00:08:38.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2517000000000001e-05 2024-09-16T00:08:38.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170188+0000", 2024-09-16T00:08:38.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8006e-05 2024-09-16T00:08:38.220 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.220 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.220 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.220 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:38.220 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:38.220 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.993902450999997, 2024-09-16T00:08:38.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093140040000000007, 2024-09-16T00:08:38.221 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.221 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.221 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.221 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.221 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.221 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.221 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-16T00:08:38.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.221 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:38.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:38.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160917+0000", 2024-09-16T00:08:38.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7599999999999996e-07 2024-09-16T00:08:38.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160992+0000", 2024-09-16T00:08:38.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4760999999999998e-05 2024-09-16T00:08:38.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161000+0000", 2024-09-16T00:08:38.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9829999999999996e-06 2024-09-16T00:08:38.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161007+0000", 2024-09-16T00:08:38.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4909999999999999e-06 2024-09-16T00:08:38.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165983+0000", 2024-09-16T00:08:38.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049755399999999997 2024-09-16T00:08:38.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166027+0000", 2024-09-16T00:08:38.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4109000000000001e-05 2024-09-16T00:08:38.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166100+0000", 2024-09-16T00:08:38.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3360000000000002e-05 2024-09-16T00:08:38.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169698+0000", 2024-09-16T00:08:38.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597795 2024-09-16T00:08:38.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170192+0000", 2024-09-16T00:08:38.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049447300000000005 2024-09-16T00:08:38.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170214+0000", 2024-09-16T00:08:38.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1546e-05 2024-09-16T00:08:38.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170230+0000", 2024-09-16T00:08:38.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6169999999999999e-05 2024-09-16T00:08:38.226 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.227 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.227 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.227 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:38.227 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:38.227 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.993792721999995, 2024-09-16T00:08:38.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092457029999999992, 2024-09-16T00:08:38.227 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.227 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.227 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.227 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.227 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.227 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.228 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-16T00:08:38.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.228 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:38.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:38.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:38.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7899999999999999e-07 2024-09-16T00:08:38.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161029+0000", 2024-09-16T00:08:38.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6630000000000002e-06 2024-09-16T00:08:38.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161036+0000", 2024-09-16T00:08:38.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T00:08:38.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161041+0000", 2024-09-16T00:08:38.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:08:38.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166468+0000", 2024-09-16T00:08:38.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054266050000000001 2024-09-16T00:08:38.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166511+0000", 2024-09-16T00:08:38.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3869999999999998e-05 2024-09-16T00:08:38.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166606+0000", 2024-09-16T00:08:38.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4437000000000004e-05 2024-09-16T00:08:38.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169955+0000", 2024-09-16T00:08:38.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033494810000000001 2024-09-16T00:08:38.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170234+0000", 2024-09-16T00:08:38.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027900899999999998 2024-09-16T00:08:38.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170255+0000", 2024-09-16T00:08:38.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0537999999999999e-05 2024-09-16T00:08:38.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170272+0000", 2024-09-16T00:08:38.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6611000000000001e-05 2024-09-16T00:08:38.233 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.233 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.233 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.233 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:38.233 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:38.233 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.993767577, 2024-09-16T00:08:38.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093511010000000006, 2024-09-16T00:08:38.234 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.234 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.234 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.234 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.234 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.234 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.234 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-16T00:08:38.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.234 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:38.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:38.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:38.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.63e-07 2024-09-16T00:08:38.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161055+0000", 2024-09-16T00:08:38.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6770000000000001e-06 2024-09-16T00:08:38.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161063+0000", 2024-09-16T00:08:38.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6469999999999998e-06 2024-09-16T00:08:38.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161068+0000", 2024-09-16T00:08:38.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.746e-06 2024-09-16T00:08:38.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166817+0000", 2024-09-16T00:08:38.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057494959999999998 2024-09-16T00:08:38.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166893+0000", 2024-09-16T00:08:38.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6210000000000004e-05 2024-09-16T00:08:38.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167019+0000", 2024-09-16T00:08:38.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125585 2024-09-16T00:08:38.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170275+0000", 2024-09-16T00:08:38.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032563620000000001 2024-09-16T00:08:38.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170309+0000", 2024-09-16T00:08:38.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3865999999999997e-05 2024-09-16T00:08:38.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170362+0000", 2024-09-16T00:08:38.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3094999999999999e-05 2024-09-16T00:08:38.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170402+0000", 2024-09-16T00:08:38.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9953999999999997e-05 2024-09-16T00:08:38.239 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.239 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.239 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.239 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:38.239 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:38.239 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.545035491999997, 2024-09-16T00:08:38.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097364389999999995, 2024-09-16T00:08:38.240 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.240 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.240 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.240 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.240 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.240 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.240 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138 2024-09-16T00:08:38.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.240 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:38.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:38.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609784+0000", 2024-09-16T00:08:38.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T00:08:38.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609787+0000", 2024-09-16T00:08:38.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280000000000001e-06 2024-09-16T00:08:38.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609794+0000", 2024-09-16T00:08:38.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6409999999999996e-06 2024-09-16T00:08:38.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609799+0000", 2024-09-16T00:08:38.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9359999999999998e-06 2024-09-16T00:08:38.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615759+0000", 2024-09-16T00:08:38.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059599910000000004 2024-09-16T00:08:38.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615806+0000", 2024-09-16T00:08:38.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7290000000000003e-05 2024-09-16T00:08:38.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615877+0000", 2024-09-16T00:08:38.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1000000000000005e-05 2024-09-16T00:08:38.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618961+0000", 2024-09-16T00:08:38.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030841390000000001 2024-09-16T00:08:38.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619436+0000", 2024-09-16T00:08:38.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047452799999999999 2024-09-16T00:08:38.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619482+0000", 2024-09-16T00:08:38.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5689000000000003e-05 2024-09-16T00:08:38.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619519+0000", 2024-09-16T00:08:38.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7938000000000002e-05 2024-09-16T00:08:38.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.245 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.245 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:38.245 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:38.245 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.545011778000003, 2024-09-16T00:08:38.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011812954000000001, 2024-09-16T00:08:38.246 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.246 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.246 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.246 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139 2024-09-16T00:08:38.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.246 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:38.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:38.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:38.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999998e-07 2024-09-16T00:08:38.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609810+0000", 2024-09-16T00:08:38.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2100000000000002e-06 2024-09-16T00:08:38.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609817+0000", 2024-09-16T00:08:38.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.28e-06 2024-09-16T00:08:38.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609825+0000", 2024-09-16T00:08:38.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:38.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616116+0000", 2024-09-16T00:08:38.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006290543 2024-09-16T00:08:38.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616187+0000", 2024-09-16T00:08:38.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1830999999999997e-05 2024-09-16T00:08:38.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616298+0000", 2024-09-16T00:08:38.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011044699999999999 2024-09-16T00:08:38.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619559+0000", 2024-09-16T00:08:38.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032614710000000002 2024-09-16T00:08:38.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621562+0000", 2024-09-16T00:08:38.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020028680000000001 2024-09-16T00:08:38.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621597+0000", 2024-09-16T00:08:38.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4625000000000001e-05 2024-09-16T00:08:38.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621620+0000", 2024-09-16T00:08:38.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2864e-05 2024-09-16T00:08:38.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.251 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.251 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:38.252 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:38.252 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.544983982000005, 2024-09-16T00:08:38.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011828211999999999, 2024-09-16T00:08:38.252 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.252 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.252 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.252 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.252 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.252 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.252 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 140 2024-09-16T00:08:38.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.252 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:38.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:38.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:38.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T00:08:38.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:38.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961000000000003e-05 2024-09-16T00:08:38.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609892+0000", 2024-09-16T00:08:38.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0516000000000001e-05 2024-09-16T00:08:38.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609898+0000", 2024-09-16T00:08:38.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0449999999999997e-06 2024-09-16T00:08:38.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616797+0000", 2024-09-16T00:08:38.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068998840000000002 2024-09-16T00:08:38.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616844+0000", 2024-09-16T00:08:38.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6501000000000001e-05 2024-09-16T00:08:38.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616935+0000", 2024-09-16T00:08:38.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1222000000000002e-05 2024-09-16T00:08:38.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619808+0000", 2024-09-16T00:08:38.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873198 2024-09-16T00:08:38.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621624+0000", 2024-09-16T00:08:38.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001815933 2024-09-16T00:08:38.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621646+0000", 2024-09-16T00:08:38.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1622999999999999e-05 2024-09-16T00:08:38.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621663+0000", 2024-09-16T00:08:38.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6854000000000002e-05 2024-09-16T00:08:38.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.257 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.257 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:38.258 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:38.258 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.544898173000007, 2024-09-16T00:08:38.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014331501999999999, 2024-09-16T00:08:38.258 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.258 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.258 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.258 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.258 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-16T00:08:38.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.258 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:38.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:38.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609921+0000", 2024-09-16T00:08:38.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9599999999999999e-07 2024-09-16T00:08:38.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609925+0000", 2024-09-16T00:08:38.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T00:08:38.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609932+0000", 2024-09-16T00:08:38.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8240000000000002e-06 2024-09-16T00:08:38.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609937+0000", 2024-09-16T00:08:38.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1070000000000004e-06 2024-09-16T00:08:38.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618987+0000", 2024-09-16T00:08:38.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090502719999999998 2024-09-16T00:08:38.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619064+0000", 2024-09-16T00:08:38.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6868999999999999e-05 2024-09-16T00:08:38.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619183+0000", 2024-09-16T00:08:38.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011961 2024-09-16T00:08:38.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.620052+0000", 2024-09-16T00:08:38.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086822799999999997 2024-09-16T00:08:38.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624197+0000", 2024-09-16T00:08:38.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041454930000000001 2024-09-16T00:08:38.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624229+0000", 2024-09-16T00:08:38.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1758999999999999e-05 2024-09-16T00:08:38.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624252+0000", 2024-09-16T00:08:38.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:08:38.263 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.263 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.072132365000002, 2024-09-16T00:08:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010093276999999999, 2024-09-16T00:08:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.264 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 157 2024-09-16T00:08:38.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.265 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:38.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:38.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:38.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4699999999999998e-07 2024-09-16T00:08:38.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082688+0000", 2024-09-16T00:08:38.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0209999999999998e-06 2024-09-16T00:08:38.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082692+0000", 2024-09-16T00:08:38.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6890000000000002e-06 2024-09-16T00:08:38.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082695+0000", 2024-09-16T00:08:38.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5339999999999998e-06 2024-09-16T00:08:38.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085303+0000", 2024-09-16T00:08:38.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026084789999999999 2024-09-16T00:08:38.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085374+0000", 2024-09-16T00:08:38.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1317999999999997e-05 2024-09-16T00:08:38.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085506+0000", 2024-09-16T00:08:38.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013118 2024-09-16T00:08:38.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086928+0000", 2024-09-16T00:08:38.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014224559999999999 2024-09-16T00:08:38.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092665+0000", 2024-09-16T00:08:38.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057369830000000002 2024-09-16T00:08:38.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092727+0000", 2024-09-16T00:08:38.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1810000000000006e-05 2024-09-16T00:08:38.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092779+0000", 2024-09-16T00:08:38.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2559999999999998e-05 2024-09-16T00:08:38.269 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.269 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.269 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.270 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:38.270 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:38.270 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.072120197000004, 2024-09-16T00:08:38.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010176388, 2024-09-16T00:08:38.270 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.270 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.270 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.270 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.270 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.270 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.270 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 158 2024-09-16T00:08:38.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.271 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:38.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:38.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082699+0000", 2024-09-16T00:08:38.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.11e-07 2024-09-16T00:08:38.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082742+0000", 2024-09-16T00:08:38.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3699000000000003e-05 2024-09-16T00:08:38.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082748+0000", 2024-09-16T00:08:38.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8690000000000001e-06 2024-09-16T00:08:38.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082751+0000", 2024-09-16T00:08:38.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.942e-06 2024-09-16T00:08:38.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085881+0000", 2024-09-16T00:08:38.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031303450000000001 2024-09-16T00:08:38.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085959+0000", 2024-09-16T00:08:38.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7874999999999996e-05 2024-09-16T00:08:38.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086086+0000", 2024-09-16T00:08:38.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012685499999999999 2024-09-16T00:08:38.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087127+0000", 2024-09-16T00:08:38.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001041298 2024-09-16T00:08:38.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092790+0000", 2024-09-16T00:08:38.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056628 2024-09-16T00:08:38.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092832+0000", 2024-09-16T00:08:38.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2227000000000001e-05 2024-09-16T00:08:38.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092875+0000", 2024-09-16T00:08:38.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2267e-05 2024-09-16T00:08:38.275 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.275 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.276 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.276 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:38.276 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:38.276 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.072062158999998, 2024-09-16T00:08:38.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010199547, 2024-09-16T00:08:38.276 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:38.276 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:38.276 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:38.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:38.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:38.277 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159 2024-09-16T00:08:38.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.277 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:38.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:38.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:38.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:38.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:38.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:38.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:38.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082757+0000", 2024-09-16T00:08:38.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2500000000000001e-07 2024-09-16T00:08:38.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:38.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082759+0000", 2024-09-16T00:08:38.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-16T00:08:38.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:38.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082763+0000", 2024-09-16T00:08:38.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4889999999999997e-06 2024-09-16T00:08:38.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082766+0000", 2024-09-16T00:08:38.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:08:38.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:38.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086571+0000", 2024-09-16T00:08:38.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038053269999999998 2024-09-16T00:08:38.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:38.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086619+0000", 2024-09-16T00:08:38.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7924999999999999e-05 2024-09-16T00:08:38.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:38.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086693+0000", 2024-09-16T00:08:38.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000005e-05 2024-09-16T00:08:38.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087383+0000", 2024-09-16T00:08:38.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069008299999999997 2024-09-16T00:08:38.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:38.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092882+0000", 2024-09-16T00:08:38.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054987450000000002 2024-09-16T00:08:38.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:38.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092921+0000", 2024-09-16T00:08:38.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9388999999999998e-05 2024-09-16T00:08:38.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:38.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:38.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:38.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092956+0000", 2024-09-16T00:08:38.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4647999999999998e-05 2024-09-16T00:08:38.281 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.281 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.282 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.282 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:38.282 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:38.282 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:38.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.198+0000 7f69104dd640 1 -- 172.21.15.139:0/1307058562 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f69080c8210 con 0x7f68ec06c2b0 2024-09-16T00:08:38.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.204+0000 7f68feffd640 1 -- 172.21.15.139:0/1307058562 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251876 (secure 0 0 0) 0x7f69080c8210 con 0x7f68ec06c2b0 2024-09-16T00:08:38.285 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:08:38.286 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:08:38.288 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:08:38.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.207+0000 7f69104dd640 1 -- 172.21.15.139:0/1307058562 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f68ec06c2b0 msgr2=0x7f68ec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:38.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.207+0000 7f69104dd640 1 --2- 172.21.15.139:0/1307058562 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f68ec06c2b0 0x7f68ec06e770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f68f8035000 tx=0x7f68f8008030 comp rx=0 tx=0).stop 2024-09-16T00:08:38.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.207+0000 7f69104dd640 1 -- 172.21.15.139:0/1307058562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69080c3310 msgr2=0x7f690814c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:38.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.207+0000 7f69104dd640 1 --2- 172.21.15.139:0/1307058562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69080c3310 0x7f690814c5c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f690400e870 tx=0x7f690400ed30 comp rx=0 tx=0).stop 2024-09-16T00:08:38.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.207+0000 7f69104dd640 1 -- 172.21.15.139:0/1307058562 shutdown_connections 2024-09-16T00:08:38.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.207+0000 7f69104dd640 1 --2- 172.21.15.139:0/1307058562 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f68ec06c2b0 0x7f68ec06e770 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:38.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.207+0000 7f69104dd640 1 --2- 172.21.15.139:0/1307058562 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6908150df0 0x7f69081e29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:38.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.207+0000 7f69104dd640 1 --2- 172.21.15.139:0/1307058562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f69080c8430 0x7f69081f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:38.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.207+0000 7f69104dd640 1 --2- 172.21.15.139:0/1307058562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f69080c3310 0x7f690814c5c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:38.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.207+0000 7f69104dd640 1 -- 172.21.15.139:0/1307058562 >> 172.21.15.139:0/1307058562 conn(0x7f69080cc6d0 msgr2=0x7f690814ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:38.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.207+0000 7f69104dd640 1 -- 172.21.15.139:0/1307058562 shutdown_connections 2024-09-16T00:08:38.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.208+0000 7f69104dd640 1 -- 172.21.15.139:0/1307058562 wait complete. 2024-09-16T00:08:38.291 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-09-16T00:08:38.036612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597721,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":212,"num_write_kb":2439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":13110,"num_keys_recovered":2,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220472,"kb_used_data":4208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754248,"statfs":{"total":773094113280,"available":772868349952,"internally_reserved":0,"allocated":4308992,"data_stored":2974870,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":6555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.172814"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888570+0000","last_change":"2024-09-16T00:08:36.888570+0000","last_active":"2024-09-16T00:08:36.888570+0000","last_peered":"2024-09-16T00:08:36.888570+0000","last_clean":"2024-09-16T00:08:36.888570+0000","last_became_active":"2024-09-16T00:08:36.888143+0000","last_became_peered":"2024-09-16T00:08:36.888143+0000","last_unstale":"2024-09-16T00:08:36.888570+0000","last_undegraded":"2024-09-16T00:08:36.888570+0000","last_fullsized":"2024-09-16T00:08:36.888570+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:30.579681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882794+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:08:35.882794+0000","last_peered":"2024-09-16T00:08:35.882794+0000","last_clean":"2024-09-16T00:08:35.882794+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:08:35.882794+0000","last_undegraded":"2024-09-16T00:08:35.882794+0000","last_fullsized":"2024-09-16T00:08:35.882794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.890336+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:08:35.890336+0000","last_peered":"2024-09-16T00:08:35.890336+0000","last_clean":"2024-09-16T00:08:35.890336+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:08:35.890336+0000","last_undegraded":"2024-09-16T00:08:35.890336+0000","last_fullsized":"2024-09-16T00:08:35.890336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888438+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:08:36.888438+0000","last_peered":"2024-09-16T00:08:36.888438+0000","last_clean":"2024-09-16T00:08:36.888438+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:08:36.888438+0000","last_undegraded":"2024-09-16T00:08:36.888438+0000","last_fullsized":"2024-09-16T00:08:36.888438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882059+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:08:35.882059+0000","last_peered":"2024-09-16T00:08:35.882059+0000","last_clean":"2024-09-16T00:08:35.882059+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:08:35.882059+0000","last_undegraded":"2024-09-16T00:08:35.882059+0000","last_fullsized":"2024-09-16T00:08:35.882059+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888197+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:08:36.888197+0000","last_peered":"2024-09-16T00:08:36.888197+0000","last_clean":"2024-09-16T00:08:36.888197+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:08:36.888197+0000","last_undegraded":"2024-09-16T00:08:36.888197+0000","last_fullsized":"2024-09-16T00:08:36.888197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882645+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:08:36.882645+0000","last_peered":"2024-09-16T00:08:36.882645+0000","last_clean":"2024-09-16T00:08:36.882645+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:08:36.882645+0000","last_undegraded":"2024-09-16T00:08:36.882645+0000","last_fullsized":"2024-09-16T00:08:36.882645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":32,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874260+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:08:34.874260+0000","last_peered":"2024-09-16T00:08:34.874260+0000","last_clean":"2024-09-16T00:08:34.874260+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:08:34.874260+0000","last_undegraded":"2024-09-16T00:08:34.874260+0000","last_fullsized":"2024-09-16T00:08:34.874260+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883724+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:08:36.883724+0000","last_peered":"2024-09-16T00:08:36.883724+0000","last_clean":"2024-09-16T00:08:36.883724+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:08:36.883724+0000","last_undegraded":"2024-09-16T00:08:36.883724+0000","last_fullsized":"2024-09-16T00:08:36.883724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.893637+0000","last_change":"2024-09-16T00:08:36.891868+0000","last_active":"2024-09-16T00:08:36.893637+0000","last_peered":"2024-09-16T00:08:36.893637+0000","last_clean":"2024-09-16T00:08:36.893637+0000","last_became_active":"2024-09-16T00:08:36.891605+0000","last_became_peered":"2024-09-16T00:08:36.891605+0000","last_unstale":"2024-09-16T00:08:36.893637+0000","last_undegraded":"2024-09-16T00:08:36.893637+0000","last_fullsized":"2024-09-16T00:08:36.893637+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:46:00.730436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.893332+0000","last_change":"2024-09-16T00:08:36.891589+0000","last_active":"2024-09-16T00:08:36.893332+0000","last_peered":"2024-09-16T00:08:36.893332+0000","last_clean":"2024-09-16T00:08:36.893332+0000","last_became_active":"2024-09-16T00:08:36.891279+0000","last_became_peered":"2024-09-16T00:08:36.891279+0000","last_unstale":"2024-09-16T00:08:36.893332+0000","last_undegraded":"2024-09-16T00:08:36.893332+0000","last_fullsized":"2024-09-16T00:08:36.893332+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888576+0000","last_change":"2024-09-16T00:08:36.888576+0000","last_active":"2024-09-16T00:08:36.888576+0000","last_peered":"2024-09-16T00:08:36.888576+0000","last_clean":"2024-09-16T00:08:36.888576+0000","last_became_active":"2024-09-16T00:08:36.888301+0000","last_became_peered":"2024-09-16T00:08:36.888301+0000","last_unstale":"2024-09-16T00:08:36.888576+0000","last_undegraded":"2024-09-16T00:08:36.888576+0000","last_fullsized":"2024-09-16T00:08:36.888576+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"peering","last_fresh":"2024-09-16T00:08:35.190711+0000","last_change":"2024-09-16T00:08:35.892992+0000","last_active":"2024-09-16T00:08:35.892992+0000","last_peered":"2024-09-16T00:08:35.190711+0000","last_clean":"2024-09-16T00:08:35.190711+0000","last_became_active":"2024-09-16T00:08:24.840637+0000","last_became_peered":"2024-09-16T00:08:24.840637+0000","last_unstale":"2024-09-16T00:08:35.190711+0000","last_undegraded":"2024-09-16T00:08:35.190711+0000","last_fullsized":"2024-09-16T00:08:35.190711+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888448+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:08:35.888448+0000","last_peered":"2024-09-16T00:08:35.888448+0000","last_clean":"2024-09-16T00:08:35.888448+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:08:35.888448+0000","last_undegraded":"2024-09-16T00:08:35.888448+0000","last_fullsized":"2024-09-16T00:08:35.888448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881726+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:08:35.881726+0000","last_peered":"2024-09-16T00:08:35.881726+0000","last_clean":"2024-09-16T00:08:35.881726+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:08:35.881726+0000","last_undegraded":"2024-09-16T00:08:35.881726+0000","last_fullsized":"2024-09-16T00:08:35.881726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889590+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:08:36.889590+0000","last_peered":"2024-09-16T00:08:36.889590+0000","last_clean":"2024-09-16T00:08:36.889590+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:08:36.889590+0000","last_undegraded":"2024-09-16T00:08:36.889590+0000","last_fullsized":"2024-09-16T00:08:36.889590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888829+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:08:36.888829+0000","last_peered":"2024-09-16T00:08:36.888829+0000","last_clean":"2024-09-16T00:08:36.888829+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:08:36.888829+0000","last_undegraded":"2024-09-16T00:08:36.888829+0000","last_fullsized":"2024-09-16T00:08:36.888829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874802+0000","last_change":"2024-09-16T00:08:24.836818+0000","last_active":"2024-09-16T00:08:34.874802+0000","last_peered":"2024-09-16T00:08:34.874802+0000","last_clean":"2024-09-16T00:08:34.874802+0000","last_became_active":"2024-09-16T00:08:24.836029+0000","last_became_peered":"2024-09-16T00:08:24.836029+0000","last_unstale":"2024-09-16T00:08:34.874802+0000","last_undegraded":"2024-09-16T00:08:34.874802+0000","last_fullsized":"2024-09-16T00:08:34.874802+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:38:55.816393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884263+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:08:36.884263+0000","last_peered":"2024-09-16T00:08:36.884263+0000","last_clean":"2024-09-16T00:08:36.884263+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:08:36.884263+0000","last_undegraded":"2024-09-16T00:08:36.884263+0000","last_fullsized":"2024-09-16T00:08:36.884263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.892945+0000","last_change":"2024-09-16T00:08:36.890676+0000","last_active":"2024-09-16T00:08:36.892945+0000","last_peered":"2024-09-16T00:08:36.892945+0000","last_clean":"2024-09-16T00:08:36.892945+0000","last_became_active":"2024-09-16T00:08:36.889743+0000","last_became_peered":"2024-09-16T00:08:36.889743+0000","last_unstale":"2024-09-16T00:08:36.892945+0000","last_undegraded":"2024-09-16T00:08:36.892945+0000","last_fullsized":"2024-09-16T00:08:36.892945+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:02.709496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888586+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:08:35.888586+0000","last_peered":"2024-09-16T00:08:35.888586+0000","last_clean":"2024-09-16T00:08:35.888586+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:08:35.888586+0000","last_undegraded":"2024-09-16T00:08:35.888586+0000","last_fullsized":"2024-09-16T00:08:35.888586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888289+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:08:36.888289+0000","last_peered":"2024-09-16T00:08:36.888289+0000","last_clean":"2024-09-16T00:08:36.888289+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:08:36.888289+0000","last_undegraded":"2024-09-16T00:08:36.888289+0000","last_fullsized":"2024-09-16T00:08:36.888289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.891321+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:08:35.891321+0000","last_peered":"2024-09-16T00:08:35.891321+0000","last_clean":"2024-09-16T00:08:35.891321+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:08:35.891321+0000","last_undegraded":"2024-09-16T00:08:35.891321+0000","last_fullsized":"2024-09-16T00:08:35.891321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889686+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:08:36.889686+0000","last_peered":"2024-09-16T00:08:36.889686+0000","last_clean":"2024-09-16T00:08:36.889686+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:08:36.889686+0000","last_undegraded":"2024-09-16T00:08:36.889686+0000","last_fullsized":"2024-09-16T00:08:36.889686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.894305+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:08:35.894305+0000","last_peered":"2024-09-16T00:08:35.894305+0000","last_clean":"2024-09-16T00:08:35.894305+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:08:35.894305+0000","last_undegraded":"2024-09-16T00:08:35.894305+0000","last_fullsized":"2024-09-16T00:08:35.894305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.886830+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:08:36.886830+0000","last_peered":"2024-09-16T00:08:36.886830+0000","last_clean":"2024-09-16T00:08:36.886830+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:08:36.886830+0000","last_undegraded":"2024-09-16T00:08:36.886830+0000","last_fullsized":"2024-09-16T00:08:36.886830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882462+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:08:35.882462+0000","last_peered":"2024-09-16T00:08:35.882462+0000","last_clean":"2024-09-16T00:08:35.882462+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:08:35.882462+0000","last_undegraded":"2024-09-16T00:08:35.882462+0000","last_fullsized":"2024-09-16T00:08:35.882462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.892094+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:08:35.892094+0000","last_peered":"2024-09-16T00:08:35.892094+0000","last_clean":"2024-09-16T00:08:35.892094+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:08:35.892094+0000","last_undegraded":"2024-09-16T00:08:35.892094+0000","last_fullsized":"2024-09-16T00:08:35.892094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888666+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:08:35.888666+0000","last_peered":"2024-09-16T00:08:35.888666+0000","last_clean":"2024-09-16T00:08:35.888666+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:08:35.888666+0000","last_undegraded":"2024-09-16T00:08:35.888666+0000","last_fullsized":"2024-09-16T00:08:35.888666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.875423+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:08:34.875423+0000","last_peered":"2024-09-16T00:08:34.875423+0000","last_clean":"2024-09-16T00:08:34.875423+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:08:34.875423+0000","last_undegraded":"2024-09-16T00:08:34.875423+0000","last_fullsized":"2024-09-16T00:08:34.875423+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874580+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:08:34.874580+0000","last_peered":"2024-09-16T00:08:34.874580+0000","last_clean":"2024-09-16T00:08:34.874580+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:08:34.874580+0000","last_undegraded":"2024-09-16T00:08:34.874580+0000","last_fullsized":"2024-09-16T00:08:34.874580+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.894891+0000","last_change":"2024-09-16T00:08:36.894101+0000","last_active":"2024-09-16T00:08:36.894891+0000","last_peered":"2024-09-16T00:08:36.894891+0000","last_clean":"2024-09-16T00:08:36.894891+0000","last_became_active":"2024-09-16T00:08:36.893807+0000","last_became_peered":"2024-09-16T00:08:36.893807+0000","last_unstale":"2024-09-16T00:08:36.894891+0000","last_undegraded":"2024-09-16T00:08:36.894891+0000","last_fullsized":"2024-09-16T00:08:36.894891+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:40.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883196+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:08:36.883196+0000","last_peered":"2024-09-16T00:08:36.883196+0000","last_clean":"2024-09-16T00:08:36.883196+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:08:36.883196+0000","last_undegraded":"2024-09-16T00:08:36.883196+0000","last_fullsized":"2024-09-16T00:08:36.883196+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":63,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.893461+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:08:35.893461+0000","last_peered":"2024-09-16T00:08:35.893461+0000","last_clean":"2024-09-16T00:08:35.893461+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:08:35.893461+0000","last_undegraded":"2024-09-16T00:08:35.893461+0000","last_fullsized":"2024-09-16T00:08:35.893461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888853+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:08:35.888853+0000","last_peered":"2024-09-16T00:08:35.888853+0000","last_clean":"2024-09-16T00:08:35.888853+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:08:35.888853+0000","last_undegraded":"2024-09-16T00:08:35.888853+0000","last_fullsized":"2024-09-16T00:08:35.888853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.892709+0000","last_change":"2024-09-16T00:08:36.890316+0000","last_active":"2024-09-16T00:08:36.892709+0000","last_peered":"2024-09-16T00:08:36.892709+0000","last_clean":"2024-09-16T00:08:36.892709+0000","last_became_active":"2024-09-16T00:08:36.889635+0000","last_became_peered":"2024-09-16T00:08:36.889635+0000","last_unstale":"2024-09-16T00:08:36.892709+0000","last_undegraded":"2024-09-16T00:08:36.892709+0000","last_fullsized":"2024-09-16T00:08:36.892709+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:28.843883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"peering","last_fresh":"2024-09-16T00:08:35.189391+0000","last_change":"2024-09-16T00:08:35.890927+0000","last_active":"2024-09-16T00:08:35.890927+0000","last_peered":"2024-09-16T00:08:35.189391+0000","last_clean":"2024-09-16T00:08:35.189391+0000","last_became_active":"2024-09-16T00:08:24.838130+0000","last_became_peered":"2024-09-16T00:08:24.838130+0000","last_unstale":"2024-09-16T00:08:35.189391+0000","last_undegraded":"2024-09-16T00:08:35.189391+0000","last_fullsized":"2024-09-16T00:08:35.189391+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.892368+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:08:35.892368+0000","last_peered":"2024-09-16T00:08:35.892368+0000","last_clean":"2024-09-16T00:08:35.892368+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:08:35.892368+0000","last_undegraded":"2024-09-16T00:08:35.892368+0000","last_fullsized":"2024-09-16T00:08:35.892368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889773+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:08:36.889773+0000","last_peered":"2024-09-16T00:08:36.889773+0000","last_clean":"2024-09-16T00:08:36.889773+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:08:36.889773+0000","last_undegraded":"2024-09-16T00:08:36.889773+0000","last_fullsized":"2024-09-16T00:08:36.889773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881638+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:08:35.881638+0000","last_peered":"2024-09-16T00:08:35.881638+0000","last_clean":"2024-09-16T00:08:35.881638+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:08:35.881638+0000","last_undegraded":"2024-09-16T00:08:35.881638+0000","last_fullsized":"2024-09-16T00:08:35.881638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.886560+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:08:36.886560+0000","last_peered":"2024-09-16T00:08:36.886560+0000","last_clean":"2024-09-16T00:08:36.886560+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:08:36.886560+0000","last_undegraded":"2024-09-16T00:08:36.886560+0000","last_fullsized":"2024-09-16T00:08:36.886560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889167+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:08:36.889167+0000","last_peered":"2024-09-16T00:08:36.889167+0000","last_clean":"2024-09-16T00:08:36.889167+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:08:36.889167+0000","last_undegraded":"2024-09-16T00:08:36.889167+0000","last_fullsized":"2024-09-16T00:08:36.889167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889247+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:08:36.889247+0000","last_peered":"2024-09-16T00:08:36.889247+0000","last_clean":"2024-09-16T00:08:36.889247+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:08:36.889247+0000","last_undegraded":"2024-09-16T00:08:36.889247+0000","last_fullsized":"2024-09-16T00:08:36.889247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.891379+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:08:35.891379+0000","last_peered":"2024-09-16T00:08:35.891379+0000","last_clean":"2024-09-16T00:08:35.891379+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:08:35.891379+0000","last_undegraded":"2024-09-16T00:08:35.891379+0000","last_fullsized":"2024-09-16T00:08:35.891379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":64,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882552+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:08:35.882552+0000","last_peered":"2024-09-16T00:08:35.882552+0000","last_clean":"2024-09-16T00:08:35.882552+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:08:35.882552+0000","last_undegraded":"2024-09-16T00:08:35.882552+0000","last_fullsized":"2024-09-16T00:08:35.882552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884405+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:08:36.884405+0000","last_peered":"2024-09-16T00:08:36.884405+0000","last_clean":"2024-09-16T00:08:36.884405+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:08:36.884405+0000","last_undegraded":"2024-09-16T00:08:36.884405+0000","last_fullsized":"2024-09-16T00:08:36.884405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"28'10","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.916626+0000","last_change":"2024-09-16T00:08:36.914308+0000","last_active":"2024-09-16T00:08:36.916626+0000","last_peered":"2024-09-16T00:08:36.916626+0000","last_clean":"2024-09-16T00:08:36.916626+0000","last_became_active":"2024-09-16T00:08:36.890227+0000","last_became_peered":"2024-09-16T00:08:36.890227+0000","last_unstale":"2024-09-16T00:08:36.916626+0000","last_undegraded":"2024-09-16T00:08:36.916626+0000","last_fullsized":"2024-09-16T00:08:36.916626+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:31.761195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6555,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13110,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884049+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:08:36.884049+0000","last_peered":"2024-09-16T00:08:36.884049+0000","last_clean":"2024-09-16T00:08:36.884049+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:08:36.884049+0000","last_undegraded":"2024-09-16T00:08:36.884049+0000","last_fullsized":"2024-09-16T00:08:36.884049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":63,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888938+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:08:35.888938+0000","last_peered":"2024-09-16T00:08:35.888938+0000","last_clean":"2024-09-16T00:08:35.888938+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:08:35.888938+0000","last_undegraded":"2024-09-16T00:08:35.888938+0000","last_fullsized":"2024-09-16T00:08:35.888938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888319+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:08:35.888319+0000","last_peered":"2024-09-16T00:08:35.888319+0000","last_clean":"2024-09-16T00:08:35.888319+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:08:35.888319+0000","last_undegraded":"2024-09-16T00:08:35.888319+0000","last_fullsized":"2024-09-16T00:08:35.888319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888369+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:08:36.888369+0000","last_peered":"2024-09-16T00:08:36.888369+0000","last_clean":"2024-09-16T00:08:36.888369+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:08:36.888369+0000","last_undegraded":"2024-09-16T00:08:36.888369+0000","last_fullsized":"2024-09-16T00:08:36.888369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882721+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:08:36.882721+0000","last_peered":"2024-09-16T00:08:36.882721+0000","last_clean":"2024-09-16T00:08:36.882721+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:08:36.882721+0000","last_undegraded":"2024-09-16T00:08:36.882721+0000","last_fullsized":"2024-09-16T00:08:36.882721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.891475+0000","last_change":"2024-09-16T00:08:36.889720+0000","last_active":"2024-09-16T00:08:36.891475+0000","last_peered":"2024-09-16T00:08:36.891475+0000","last_clean":"2024-09-16T00:08:36.891475+0000","last_became_active":"2024-09-16T00:08:36.889345+0000","last_became_peered":"2024-09-16T00:08:36.889345+0000","last_unstale":"2024-09-16T00:08:36.891475+0000","last_undegraded":"2024-09-16T00:08:36.891475+0000","last_fullsized":"2024-09-16T00:08:36.891475+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:03.170073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.900265+0000","last_change":"2024-09-16T00:08:36.896787+0000","last_active":"2024-09-16T00:08:36.900265+0000","last_peered":"2024-09-16T00:08:36.900265+0000","last_clean":"2024-09-16T00:08:36.900265+0000","last_became_active":"2024-09-16T00:08:36.896505+0000","last_became_peered":"2024-09-16T00:08:36.896505+0000","last_unstale":"2024-09-16T00:08:36.900265+0000","last_undegraded":"2024-09-16T00:08:36.900265+0000","last_fullsized":"2024-09-16T00:08:36.900265+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:55.997551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874425+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:08:34.874425+0000","last_peered":"2024-09-16T00:08:34.874425+0000","last_clean":"2024-09-16T00:08:34.874425+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:08:34.874425+0000","last_undegraded":"2024-09-16T00:08:34.874425+0000","last_fullsized":"2024-09-16T00:08:34.874425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882734+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:08:36.882734+0000","last_peered":"2024-09-16T00:08:36.882734+0000","last_clean":"2024-09-16T00:08:36.882734+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:08:36.882734+0000","last_undegraded":"2024-09-16T00:08:36.882734+0000","last_fullsized":"2024-09-16T00:08:36.882734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883994+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:08:36.883994+0000","last_peered":"2024-09-16T00:08:36.883994+0000","last_clean":"2024-09-16T00:08:36.883994+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:08:36.883994+0000","last_undegraded":"2024-09-16T00:08:36.883994+0000","last_fullsized":"2024-09-16T00:08:36.883994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883532+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:08:36.883532+0000","last_peered":"2024-09-16T00:08:36.883532+0000","last_clean":"2024-09-16T00:08:36.883532+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:08:36.883532+0000","last_undegraded":"2024-09-16T00:08:36.883532+0000","last_fullsized":"2024-09-16T00:08:36.883532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":62,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874882+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:08:34.874882+0000","last_peered":"2024-09-16T00:08:34.874882+0000","last_clean":"2024-09-16T00:08:34.874882+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:08:34.874882+0000","last_undegraded":"2024-09-16T00:08:34.874882+0000","last_fullsized":"2024-09-16T00:08:34.874882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883810+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:08:36.883810+0000","last_peered":"2024-09-16T00:08:36.883810+0000","last_clean":"2024-09-16T00:08:36.883810+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:08:36.883810+0000","last_undegraded":"2024-09-16T00:08:36.883810+0000","last_fullsized":"2024-09-16T00:08:36.883810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":62,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882644+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:08:35.882644+0000","last_peered":"2024-09-16T00:08:35.882644+0000","last_clean":"2024-09-16T00:08:35.882644+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:08:35.882644+0000","last_undegraded":"2024-09-16T00:08:35.882644+0000","last_fullsized":"2024-09-16T00:08:35.882644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888593+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:08:36.888593+0000","last_peered":"2024-09-16T00:08:36.888593+0000","last_clean":"2024-09-16T00:08:36.888593+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:08:36.888593+0000","last_undegraded":"2024-09-16T00:08:36.888593+0000","last_fullsized":"2024-09-16T00:08:36.888593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.875230+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:08:34.875230+0000","last_peered":"2024-09-16T00:08:34.875230+0000","last_clean":"2024-09-16T00:08:34.875230+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:08:34.875230+0000","last_undegraded":"2024-09-16T00:08:34.875230+0000","last_fullsized":"2024-09-16T00:08:34.875230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884151+0000","last_change":"2024-09-16T00:08:15.795467+0000","last_active":"2024-09-16T00:08:36.884151+0000","last_peered":"2024-09-16T00:08:36.884151+0000","last_clean":"2024-09-16T00:08:36.884151+0000","last_became_active":"2024-09-16T00:08:15.795167+0000","last_became_peered":"2024-09-16T00:08:15.795167+0000","last_unstale":"2024-09-16T00:08:36.884151+0000","last_undegraded":"2024-09-16T00:08:36.884151+0000","last_fullsized":"2024-09-16T00:08:36.884151+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":219,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882705+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:08:36.882705+0000","last_peered":"2024-09-16T00:08:36.882705+0000","last_clean":"2024-09-16T00:08:36.882705+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:08:36.882705+0000","last_undegraded":"2024-09-16T00:08:36.882705+0000","last_fullsized":"2024-09-16T00:08:36.882705+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888542+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:08:36.888542+0000","last_peered":"2024-09-16T00:08:36.888542+0000","last_clean":"2024-09-16T00:08:36.888542+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:08:36.888542+0000","last_undegraded":"2024-09-16T00:08:36.888542+0000","last_fullsized":"2024-09-16T00:08:36.888542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.893654+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:08:35.893654+0000","last_peered":"2024-09-16T00:08:35.893654+0000","last_clean":"2024-09-16T00:08:35.893654+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:08:35.893654+0000","last_undegraded":"2024-09-16T00:08:35.893654+0000","last_fullsized":"2024-09-16T00:08:35.893654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882625+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:08:36.882625+0000","last_peered":"2024-09-16T00:08:36.882625+0000","last_clean":"2024-09-16T00:08:36.882625+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:08:36.882625+0000","last_undegraded":"2024-09-16T00:08:36.882625+0000","last_fullsized":"2024-09-16T00:08:36.882625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.887981+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:08:35.887981+0000","last_peered":"2024-09-16T00:08:35.887981+0000","last_clean":"2024-09-16T00:08:35.887981+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:08:35.887981+0000","last_undegraded":"2024-09-16T00:08:35.887981+0000","last_fullsized":"2024-09-16T00:08:35.887981+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874658+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:08:34.874658+0000","last_peered":"2024-09-16T00:08:34.874658+0000","last_clean":"2024-09-16T00:08:34.874658+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:08:34.874658+0000","last_undegraded":"2024-09-16T00:08:34.874658+0000","last_fullsized":"2024-09-16T00:08:34.874658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881158+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:08:35.881158+0000","last_peered":"2024-09-16T00:08:35.881158+0000","last_clean":"2024-09-16T00:08:35.881158+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:08:35.881158+0000","last_undegraded":"2024-09-16T00:08:35.881158+0000","last_fullsized":"2024-09-16T00:08:35.881158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888456+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:08:36.888456+0000","last_peered":"2024-09-16T00:08:36.888456+0000","last_clean":"2024-09-16T00:08:36.888456+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:08:36.888456+0000","last_undegraded":"2024-09-16T00:08:36.888456+0000","last_fullsized":"2024-09-16T00:08:36.888456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883562+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:08:36.883562+0000","last_peered":"2024-09-16T00:08:36.883562+0000","last_clean":"2024-09-16T00:08:36.883562+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:08:36.883562+0000","last_undegraded":"2024-09-16T00:08:36.883562+0000","last_fullsized":"2024-09-16T00:08:36.883562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882294+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:08:35.882294+0000","last_peered":"2024-09-16T00:08:35.882294+0000","last_clean":"2024-09-16T00:08:35.882294+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:08:35.882294+0000","last_undegraded":"2024-09-16T00:08:35.882294+0000","last_fullsized":"2024-09-16T00:08:35.882294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889077+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:08:36.889077+0000","last_peered":"2024-09-16T00:08:36.889077+0000","last_clean":"2024-09-16T00:08:36.889077+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:08:36.889077+0000","last_undegraded":"2024-09-16T00:08:36.889077+0000","last_fullsized":"2024-09-16T00:08:36.889077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884353+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:08:36.884353+0000","last_peered":"2024-09-16T00:08:36.884353+0000","last_clean":"2024-09-16T00:08:36.884353+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:08:36.884353+0000","last_undegraded":"2024-09-16T00:08:36.884353+0000","last_fullsized":"2024-09-16T00:08:36.884353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882179+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:08:35.882179+0000","last_peered":"2024-09-16T00:08:35.882179+0000","last_clean":"2024-09-16T00:08:35.882179+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:08:35.882179+0000","last_undegraded":"2024-09-16T00:08:35.882179+0000","last_fullsized":"2024-09-16T00:08:35.882179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"26'3","reported_seq":72,"reported_epoch":45,"state":"peering","last_fresh":"2024-09-16T00:08:35.191937+0000","last_change":"2024-09-16T00:08:35.883363+0000","last_active":"2024-09-16T00:08:35.883363+0000","last_peered":"2024-09-16T00:08:35.191937+0000","last_clean":"2024-09-16T00:08:35.191937+0000","last_became_active":"2024-09-16T00:08:24.835764+0000","last_became_peered":"2024-09-16T00:08:24.835764+0000","last_unstale":"2024-09-16T00:08:35.191937+0000","last_undegraded":"2024-09-16T00:08:35.191937+0000","last_fullsized":"2024-09-16T00:08:35.191937+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.894399+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:08:35.894399+0000","last_peered":"2024-09-16T00:08:35.894399+0000","last_clean":"2024-09-16T00:08:35.894399+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:08:35.894399+0000","last_undegraded":"2024-09-16T00:08:35.894399+0000","last_fullsized":"2024-09-16T00:08:35.894399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.886688+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:08:36.886688+0000","last_peered":"2024-09-16T00:08:36.886688+0000","last_clean":"2024-09-16T00:08:36.886688+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:08:36.886688+0000","last_undegraded":"2024-09-16T00:08:36.886688+0000","last_fullsized":"2024-09-16T00:08:36.886688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.890491+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:08:35.890491+0000","last_peered":"2024-09-16T00:08:35.890491+0000","last_clean":"2024-09-16T00:08:35.890491+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:08:35.890491+0000","last_undegraded":"2024-09-16T00:08:35.890491+0000","last_fullsized":"2024-09-16T00:08:35.890491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882517+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:08:36.882517+0000","last_peered":"2024-09-16T00:08:36.882517+0000","last_clean":"2024-09-16T00:08:36.882517+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:08:36.882517+0000","last_undegraded":"2024-09-16T00:08:36.882517+0000","last_fullsized":"2024-09-16T00:08:36.882517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884343+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:08:36.884343+0000","last_peered":"2024-09-16T00:08:36.884343+0000","last_clean":"2024-09-16T00:08:36.884343+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:08:36.884343+0000","last_undegraded":"2024-09-16T00:08:36.884343+0000","last_fullsized":"2024-09-16T00:08:36.884343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":61,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881815+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:08:35.881815+0000","last_peered":"2024-09-16T00:08:35.881815+0000","last_clean":"2024-09-16T00:08:35.881815+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:08:35.881815+0000","last_undegraded":"2024-09-16T00:08:35.881815+0000","last_fullsized":"2024-09-16T00:08:35.881815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.892453+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:08:35.892453+0000","last_peered":"2024-09-16T00:08:35.892453+0000","last_clean":"2024-09-16T00:08:35.892453+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:08:35.892453+0000","last_undegraded":"2024-09-16T00:08:35.892453+0000","last_fullsized":"2024-09-16T00:08:35.892453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":61,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881852+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:08:35.881852+0000","last_peered":"2024-09-16T00:08:35.881852+0000","last_clean":"2024-09-16T00:08:35.881852+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:08:35.881852+0000","last_undegraded":"2024-09-16T00:08:35.881852+0000","last_fullsized":"2024-09-16T00:08:35.881852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.893352+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:08:35.893352+0000","last_peered":"2024-09-16T00:08:35.893352+0000","last_clean":"2024-09-16T00:08:35.893352+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:08:35.893352+0000","last_undegraded":"2024-09-16T00:08:35.893352+0000","last_fullsized":"2024-09-16T00:08:35.893352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882815+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:08:36.882815+0000","last_peered":"2024-09-16T00:08:36.882815+0000","last_clean":"2024-09-16T00:08:36.882815+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:08:36.882815+0000","last_undegraded":"2024-09-16T00:08:36.882815+0000","last_fullsized":"2024-09-16T00:08:36.882815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.892834+0000","last_change":"2024-09-16T00:08:36.890347+0000","last_active":"2024-09-16T00:08:36.892834+0000","last_peered":"2024-09-16T00:08:36.892834+0000","last_clean":"2024-09-16T00:08:36.892834+0000","last_became_active":"2024-09-16T00:08:36.890009+0000","last_became_peered":"2024-09-16T00:08:36.890009+0000","last_unstale":"2024-09-16T00:08:36.892834+0000","last_undegraded":"2024-09-16T00:08:36.892834+0000","last_fullsized":"2024-09-16T00:08:36.892834+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:24.298978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888991+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:08:36.888991+0000","last_peered":"2024-09-16T00:08:36.888991+0000","last_clean":"2024-09-16T00:08:36.888991+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:08:36.888991+0000","last_undegraded":"2024-09-16T00:08:36.888991+0000","last_fullsized":"2024-09-16T00:08:36.888991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883181+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:08:36.883181+0000","last_peered":"2024-09-16T00:08:36.883181+0000","last_clean":"2024-09-16T00:08:36.883181+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:08:36.883181+0000","last_undegraded":"2024-09-16T00:08:36.883181+0000","last_fullsized":"2024-09-16T00:08:36.883181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888518+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:08:36.888518+0000","last_peered":"2024-09-16T00:08:36.888518+0000","last_clean":"2024-09-16T00:08:36.888518+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:08:36.888518+0000","last_undegraded":"2024-09-16T00:08:36.888518+0000","last_fullsized":"2024-09-16T00:08:36.888518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.900704+0000","last_change":"2024-09-16T00:08:36.900704+0000","last_active":"2024-09-16T00:08:36.900704+0000","last_peered":"2024-09-16T00:08:36.900704+0000","last_clean":"2024-09-16T00:08:36.900704+0000","last_became_active":"2024-09-16T00:08:36.892520+0000","last_became_peered":"2024-09-16T00:08:36.892520+0000","last_unstale":"2024-09-16T00:08:36.900704+0000","last_undegraded":"2024-09-16T00:08:36.900704+0000","last_fullsized":"2024-09-16T00:08:36.900704+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:30.056041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874986+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:08:34.874986+0000","last_peered":"2024-09-16T00:08:34.874986+0000","last_clean":"2024-09-16T00:08:34.874986+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:08:34.874986+0000","last_undegraded":"2024-09-16T00:08:34.874986+0000","last_fullsized":"2024-09-16T00:08:34.874986+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":63,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.895206+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:08:35.895206+0000","last_peered":"2024-09-16T00:08:35.895206+0000","last_clean":"2024-09-16T00:08:35.895206+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:08:35.895206+0000","last_undegraded":"2024-09-16T00:08:35.895206+0000","last_fullsized":"2024-09-16T00:08:35.895206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"peering","last_fresh":"2024-09-16T00:08:35.190279+0000","last_change":"2024-09-16T00:08:35.895010+0000","last_active":"2024-09-16T00:08:35.895010+0000","last_peered":"2024-09-16T00:08:35.190279+0000","last_clean":"2024-09-16T00:08:35.190279+0000","last_became_active":"2024-09-16T00:08:24.838285+0000","last_became_peered":"2024-09-16T00:08:24.838285+0000","last_unstale":"2024-09-16T00:08:35.190279+0000","last_undegraded":"2024-09-16T00:08:35.190279+0000","last_fullsized":"2024-09-16T00:08:35.190279+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":65,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.891464+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:08:35.891464+0000","last_peered":"2024-09-16T00:08:35.891464+0000","last_clean":"2024-09-16T00:08:35.891464+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:08:35.891464+0000","last_undegraded":"2024-09-16T00:08:35.891464+0000","last_fullsized":"2024-09-16T00:08:35.891464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7353,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":13110,"num_keys_recovered":2,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":14706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":585728,"data_stored":815033,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":540672,"data_stored":236719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":230728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":540672,"data_stored":230380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":520192,"data_stored":224158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":217763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":581632,"data_stored":808609,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":491520,"data_stored":211480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":6555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":6555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:08:38.295 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:38.282+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:38.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.289+0000 7fb7c3217640 1 -- 172.21.15.139:0/1210806254 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fb79c005580 con 0x7fb7bc0ccc90 2024-09-16T00:08:38.384 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:38.383+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:38.437 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:38.437 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:38.437 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:38.437 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:38.447 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:08:38.485 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:38.484+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:38.556 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:38.556 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:38.556 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:38.556 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:38.557 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:38.566 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:08:38.586 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:38.585+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:38.684 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:38.685 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:38.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:38.685 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:38.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.685 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'41, mlcod=17'39)", 2024-09-16T00:08:38.685 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:38.685 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 74.661315635999998, 2024-09-16T00:08:38.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001808488, 2024-09-16T00:08:38.685 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.685 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.685 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.685 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:38.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:38.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021032+0000", 2024-09-16T00:08:38.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3e-07 2024-09-16T00:08:38.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021037+0000", 2024-09-16T00:08:38.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0640000000000001e-06 2024-09-16T00:08:38.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021047+0000", 2024-09-16T00:08:38.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.023e-05 2024-09-16T00:08:38.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021053+0000", 2024-09-16T00:08:38.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3160000000000002e-06 2024-09-16T00:08:38.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021092+0000", 2024-09-16T00:08:38.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9033999999999999e-05 2024-09-16T00:08:38.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021105+0000", 2024-09-16T00:08:38.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2768e-05 2024-09-16T00:08:38.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022811+0000", 2024-09-16T00:08:38.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001705788 2024-09-16T00:08:38.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022840+0000", 2024-09-16T00:08:38.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8558e-05 2024-09-16T00:08:38.689 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.689 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.689 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.690 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:85 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'55, mlcod=17'50)", 2024-09-16T00:08:38.690 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:38.690 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 74.648286635000005, 2024-09-16T00:08:38.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017909530000000001, 2024-09-16T00:08:38.690 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.690 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.690 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.690 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:38.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:38.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034061+0000", 2024-09-16T00:08:38.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0399999999999996e-07 2024-09-16T00:08:38.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034065+0000", 2024-09-16T00:08:38.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8840000000000002e-06 2024-09-16T00:08:38.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034074+0000", 2024-09-16T00:08:38.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:38.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034079+0000", 2024-09-16T00:08:38.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0000000000000004e-06 2024-09-16T00:08:38.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034125+0000", 2024-09-16T00:08:38.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5905000000000002e-05 2024-09-16T00:08:38.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034144+0000", 2024-09-16T00:08:38.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9593e-05 2024-09-16T00:08:38.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035817+0000", 2024-09-16T00:08:38.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016728240000000001 2024-09-16T00:08:38.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035851+0000", 2024-09-16T00:08:38.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3809000000000001e-05 2024-09-16T00:08:38.695 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.695 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.695 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.695 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:112 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'74, mlcod=17'65)", 2024-09-16T00:08:38.695 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:38.695 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 74.517674772999996, 2024-09-16T00:08:38.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018457020000000001, 2024-09-16T00:08:38.695 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.695 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.695 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.696 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:38.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:38.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164673+0000", 2024-09-16T00:08:38.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1200000000000003e-07 2024-09-16T00:08:38.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164710+0000", 2024-09-16T00:08:38.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7756999999999998e-05 2024-09-16T00:08:38.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164719+0000", 2024-09-16T00:08:38.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2849999999999995e-06 2024-09-16T00:08:38.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164723+0000", 2024-09-16T00:08:38.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3220000000000003e-06 2024-09-16T00:08:38.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164760+0000", 2024-09-16T00:08:38.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7311999999999998e-05 2024-09-16T00:08:38.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164771+0000", 2024-09-16T00:08:38.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0869e-05 2024-09-16T00:08:38.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166495+0000", 2024-09-16T00:08:38.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001723863 2024-09-16T00:08:38.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166518+0000", 2024-09-16T00:08:38.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2682000000000001e-05 2024-09-16T00:08:38.701 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.701 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.701 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.701 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'66)", 2024-09-16T00:08:38.701 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:38.702 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 74.516170794000004, 2024-09-16T00:08:38.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032925509999999999, 2024-09-16T00:08:38.702 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.702 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.702 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.702 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:38.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:38.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166177+0000", 2024-09-16T00:08:38.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:38.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166216+0000", 2024-09-16T00:08:38.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9008000000000003e-05 2024-09-16T00:08:38.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166226+0000", 2024-09-16T00:08:38.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0081999999999999e-05 2024-09-16T00:08:38.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166230+0000", 2024-09-16T00:08:38.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5079999999999999e-06 2024-09-16T00:08:38.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166259+0000", 2024-09-16T00:08:38.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8928000000000002e-05 2024-09-16T00:08:38.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166271+0000", 2024-09-16T00:08:38.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1531e-05 2024-09-16T00:08:38.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169449+0000", 2024-09-16T00:08:38.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003178143 2024-09-16T00:08:38.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169468+0000", 2024-09-16T00:08:38.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9700000000000001e-05 2024-09-16T00:08:38.708 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.708 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.708 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.708 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'66)", 2024-09-16T00:08:38.709 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:38.709 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 74.515709334999997, 2024-09-16T00:08:38.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003179374, 2024-09-16T00:08:38.709 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.709 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.709 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.709 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:38.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:38.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166638+0000", 2024-09-16T00:08:38.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2300000000000002e-07 2024-09-16T00:08:38.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166641+0000", 2024-09-16T00:08:38.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0079999999999998e-06 2024-09-16T00:08:38.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166646+0000", 2024-09-16T00:08:38.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4970000000000001e-06 2024-09-16T00:08:38.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166650+0000", 2024-09-16T00:08:38.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5729999999999999e-06 2024-09-16T00:08:38.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166682+0000", 2024-09-16T00:08:38.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2218999999999998e-05 2024-09-16T00:08:38.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166694+0000", 2024-09-16T00:08:38.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1902999999999999e-05 2024-09-16T00:08:38.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169784+0000", 2024-09-16T00:08:38.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003089545 2024-09-16T00:08:38.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169817+0000", 2024-09-16T00:08:38.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3206e-05 2024-09-16T00:08:38.713 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.713 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.714 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.714 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:117 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'66)", 2024-09-16T00:08:38.714 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:38.714 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 74.515265275000004, 2024-09-16T00:08:38.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00275326, 2024-09-16T00:08:38.714 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.714 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.714 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.715 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:38.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:38.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167082+0000", 2024-09-16T00:08:38.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:08:38.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167086+0000", 2024-09-16T00:08:38.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1559999999999997e-06 2024-09-16T00:08:38.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167096+0000", 2024-09-16T00:08:38.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8390000000000002e-06 2024-09-16T00:08:38.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167101+0000", 2024-09-16T00:08:38.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:08:38.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167130+0000", 2024-09-16T00:08:38.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9649e-05 2024-09-16T00:08:38.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167142+0000", 2024-09-16T00:08:38.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1435e-05 2024-09-16T00:08:38.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169818+0000", 2024-09-16T00:08:38.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002676056 2024-09-16T00:08:38.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169835+0000", 2024-09-16T00:08:38.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6980999999999999e-05 2024-09-16T00:08:38.719 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.719 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.719 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.719 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:130 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'88, mlcod=17'85)", 2024-09-16T00:08:38.719 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:38.719 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 74.072465945999994, 2024-09-16T00:08:38.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002880456, 2024-09-16T00:08:38.719 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.719 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.719 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.720 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:38.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:38.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:38.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5699999999999998e-07 2024-09-16T00:08:38.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609885+0000", 2024-09-16T00:08:38.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.608e-06 2024-09-16T00:08:38.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609891+0000", 2024-09-16T00:08:38.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:38.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609895+0000", 2024-09-16T00:08:38.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.54e-06 2024-09-16T00:08:38.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609931+0000", 2024-09-16T00:08:38.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5893999999999999e-05 2024-09-16T00:08:38.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609945+0000", 2024-09-16T00:08:38.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4259e-05 2024-09-16T00:08:38.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612721+0000", 2024-09-16T00:08:38.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027757849999999998 2024-09-16T00:08:38.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612761+0000", 2024-09-16T00:08:38.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0638e-05 2024-09-16T00:08:38.724 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.724 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.724 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.724 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-16T00:08:38.724 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:38.724 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 74.072081972999996, 2024-09-16T00:08:38.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028016529999999999, 2024-09-16T00:08:38.724 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.725 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.725 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.725 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:38.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:38.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:38.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.63e-07 2024-09-16T00:08:38.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610344+0000", 2024-09-16T00:08:38.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8996000000000001e-05 2024-09-16T00:08:38.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610352+0000", 2024-09-16T00:08:38.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:38.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610356+0000", 2024-09-16T00:08:38.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0790000000000002e-06 2024-09-16T00:08:38.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610388+0000", 2024-09-16T00:08:38.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1541000000000002e-05 2024-09-16T00:08:38.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610400+0000", 2024-09-16T00:08:38.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2075e-05 2024-09-16T00:08:38.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613045+0000", 2024-09-16T00:08:38.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002645213 2024-09-16T00:08:38.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613066+0000", 2024-09-16T00:08:38.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1353e-05 2024-09-16T00:08:38.731 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.731 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.731 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.731 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-16T00:08:38.731 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:38.731 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 74.068822472999997, 2024-09-16T00:08:38.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027254850000000001, 2024-09-16T00:08:38.731 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.731 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.731 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.732 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.732 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:38.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613525+0000", 2024-09-16T00:08:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2800000000000002e-07 2024-09-16T00:08:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613528+0000", 2024-09-16T00:08:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9730000000000002e-06 2024-09-16T00:08:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613534+0000", 2024-09-16T00:08:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4339999999999997e-06 2024-09-16T00:08:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613538+0000", 2024-09-16T00:08:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5149999999999998e-06 2024-09-16T00:08:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613566+0000", 2024-09-16T00:08:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8104000000000001e-05 2024-09-16T00:08:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613578+0000", 2024-09-16T00:08:38.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2404e-05 2024-09-16T00:08:38.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616217+0000", 2024-09-16T00:08:38.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026389830000000001 2024-09-16T00:08:38.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616250+0000", 2024-09-16T00:08:38.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2644000000000001e-05 2024-09-16T00:08:38.737 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.737 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.737 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.737 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:134 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'92, mlcod=18'87)", 2024-09-16T00:08:38.737 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:38.737 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 74.068284333999998, 2024-09-16T00:08:38.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025322399999999998, 2024-09-16T00:08:38.737 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.738 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.738 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.738 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:38.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:38.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614063+0000", 2024-09-16T00:08:38.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8700000000000005e-07 2024-09-16T00:08:38.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614096+0000", 2024-09-16T00:08:38.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3136000000000003e-05 2024-09-16T00:08:38.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614106+0000", 2024-09-16T00:08:38.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9170000000000001e-06 2024-09-16T00:08:38.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614111+0000", 2024-09-16T00:08:38.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4390000000000002e-06 2024-09-16T00:08:38.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614141+0000", 2024-09-16T00:08:38.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.04e-05 2024-09-16T00:08:38.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614152+0000", 2024-09-16T00:08:38.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0987e-05 2024-09-16T00:08:38.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616573+0000", 2024-09-16T00:08:38.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002421351 2024-09-16T00:08:38.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616595+0000", 2024-09-16T00:08:38.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1322999999999999e-05 2024-09-16T00:08:38.742 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.742 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.742 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.743 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:135 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'93, mlcod=18'87)", 2024-09-16T00:08:38.743 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:38.743 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 74.067764296000007, 2024-09-16T00:08:38.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020304659999999999, 2024-09-16T00:08:38.743 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.743 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.743 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.743 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:38.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:38.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614583+0000", 2024-09-16T00:08:38.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:38.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614587+0000", 2024-09-16T00:08:38.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3100000000000002e-06 2024-09-16T00:08:38.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614597+0000", 2024-09-16T00:08:38.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7450000000000007e-06 2024-09-16T00:08:38.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614602+0000", 2024-09-16T00:08:38.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6099999999999999e-06 2024-09-16T00:08:38.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614641+0000", 2024-09-16T00:08:38.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9767000000000001e-05 2024-09-16T00:08:38.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614658+0000", 2024-09-16T00:08:38.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6205000000000002e-05 2024-09-16T00:08:38.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616596+0000", 2024-09-16T00:08:38.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019386659999999999 2024-09-16T00:08:38.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616613+0000", 2024-09-16T00:08:38.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6555999999999999e-05 2024-09-16T00:08:38.747 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.747 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.748 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.748 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:139 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'97, mlcod=18'87)", 2024-09-16T00:08:38.748 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:38.748 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 74.066003624999993, 2024-09-16T00:08:38.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030908139999999999, 2024-09-16T00:08:38.748 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.748 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.748 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.749 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:38.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:38.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616344+0000", 2024-09-16T00:08:38.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3000000000000001e-07 2024-09-16T00:08:38.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616347+0000", 2024-09-16T00:08:38.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7740000000000002e-06 2024-09-16T00:08:38.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616355+0000", 2024-09-16T00:08:38.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4699999999999996e-06 2024-09-16T00:08:38.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616358+0000", 2024-09-16T00:08:38.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5899999999999999e-06 2024-09-16T00:08:38.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616393+0000", 2024-09-16T00:08:38.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4529000000000001e-05 2024-09-16T00:08:38.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616404+0000", 2024-09-16T00:08:38.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0726e-05 2024-09-16T00:08:38.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619411+0000", 2024-09-16T00:08:38.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030076249999999999 2024-09-16T00:08:38.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619434+0000", 2024-09-16T00:08:38.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-05 2024-09-16T00:08:38.753 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.753 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.753 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.754 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:140 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'98, mlcod=18'89)", 2024-09-16T00:08:38.754 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:38.754 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 74.065362231999998, 2024-09-16T00:08:38.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027276150000000001, 2024-09-16T00:08:38.754 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.754 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.754 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.754 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:38.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:38.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616985+0000", 2024-09-16T00:08:38.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4099999999999999e-07 2024-09-16T00:08:38.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617055+0000", 2024-09-16T00:08:38.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9923000000000001e-05 2024-09-16T00:08:38.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617063+0000", 2024-09-16T00:08:38.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:38.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617070+0000", 2024-09-16T00:08:38.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9090000000000003e-06 2024-09-16T00:08:38.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617102+0000", 2024-09-16T00:08:38.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2620999999999998e-05 2024-09-16T00:08:38.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617114+0000", 2024-09-16T00:08:38.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1228e-05 2024-09-16T00:08:38.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619690+0000", 2024-09-16T00:08:38.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025763890000000001 2024-09-16T00:08:38.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619712+0000", 2024-09-16T00:08:38.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2171000000000002e-05 2024-09-16T00:08:38.759 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.759 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.759 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.759 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:38.759 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:38.759 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.066533132000004, 2024-09-16T00:08:38.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027355320000000002, 2024-09-16T00:08:38.759 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.760 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.760 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.760 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:38.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:38.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615815+0000", 2024-09-16T00:08:38.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1769999999999999e-06 2024-09-16T00:08:38.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615825+0000", 2024-09-16T00:08:38.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9000000000000001e-06 2024-09-16T00:08:38.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615840+0000", 2024-09-16T00:08:38.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4854e-05 2024-09-16T00:08:38.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615848+0000", 2024-09-16T00:08:38.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0940000000000008e-06 2024-09-16T00:08:38.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615926+0000", 2024-09-16T00:08:38.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8176999999999994e-05 2024-09-16T00:08:38.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.763 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.763 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615945+0000", 2024-09-16T00:08:38.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9663000000000001e-05 2024-09-16T00:08:38.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.764 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.764 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618509+0000", 2024-09-16T00:08:38.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002563802 2024-09-16T00:08:38.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.764 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.764 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.764 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618549+0000", 2024-09-16T00:08:38.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9864999999999999e-05 2024-09-16T00:08:38.764 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.765 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.765 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.765 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:38.765 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:38.765 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.066443992000004, 2024-09-16T00:08:38.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061593860000000002, 2024-09-16T00:08:38.765 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.765 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.766 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.766 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:38.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:38.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615907+0000", 2024-09-16T00:08:38.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9389999999999999e-06 2024-09-16T00:08:38.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615920+0000", 2024-09-16T00:08:38.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2989999999999999e-05 2024-09-16T00:08:38.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615931+0000", 2024-09-16T00:08:38.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.164e-05 2024-09-16T00:08:38.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615939+0000", 2024-09-16T00:08:38.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8029999999999997e-06 2024-09-16T00:08:38.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616310+0000", 2024-09-16T00:08:38.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00037111400000000003 2024-09-16T00:08:38.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616406+0000", 2024-09-16T00:08:38.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5372999999999996e-05 2024-09-16T00:08:38.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.770 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:38.770 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616514+0000", 2024-09-16T00:08:38.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108564 2024-09-16T00:08:38.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.770 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:38.770 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618158+0000", 2024-09-16T00:08:38.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016442449999999999 2024-09-16T00:08:38.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621972+0000", 2024-09-16T00:08:38.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038132299999999999 2024-09-16T00:08:38.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622018+0000", 2024-09-16T00:08:38.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6239000000000002e-05 2024-09-16T00:08:38.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622062+0000", 2024-09-16T00:08:38.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4249000000000003e-05 2024-09-16T00:08:38.772 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.772 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.772 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.772 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:38.773 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:38.773 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.066390523999999, 2024-09-16T00:08:38.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006258914, 2024-09-16T00:08:38.773 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.773 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.773 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.773 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:38.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:38.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615957+0000", 2024-09-16T00:08:38.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1399999999999997e-07 2024-09-16T00:08:38.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.775 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.775 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615960+0000", 2024-09-16T00:08:38.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0860000000000002e-06 2024-09-16T00:08:38.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.776 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.776 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615967+0000", 2024-09-16T00:08:38.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9059999999999996e-06 2024-09-16T00:08:38.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.776 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.776 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615972+0000", 2024-09-16T00:08:38.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5679999999999999e-06 2024-09-16T00:08:38.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616717+0000", 2024-09-16T00:08:38.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074438 2024-09-16T00:08:38.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616781+0000", 2024-09-16T00:08:38.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4808999999999996e-05 2024-09-16T00:08:38.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:38.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616885+0000", 2024-09-16T00:08:38.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010404900000000001 2024-09-16T00:08:38.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:38.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:38.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012773870000000001 2024-09-16T00:08:38.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622116+0000", 2024-09-16T00:08:38.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039533470000000003 2024-09-16T00:08:38.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622173+0000", 2024-09-16T00:08:38.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6347000000000001e-05 2024-09-16T00:08:38.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622215+0000", 2024-09-16T00:08:38.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2521000000000002e-05 2024-09-16T00:08:38.780 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.780 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.780 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.780 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:38.780 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:38.780 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.066366787, 2024-09-16T00:08:38.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041725529999999999, 2024-09-16T00:08:38.780 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.780 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.780 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.781 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:38.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:38.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8799999999999998e-07 2024-09-16T00:08:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615990+0000", 2024-09-16T00:08:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2150000000000007e-06 2024-09-16T00:08:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:08:38.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7159999999999997e-06 2024-09-16T00:08:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616001+0000", 2024-09-16T00:08:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9339999999999997e-06 2024-09-16T00:08:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616293+0000", 2024-09-16T00:08:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029210899999999998 2024-09-16T00:08:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616401+0000", 2024-09-16T00:08:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108087 2024-09-16T00:08:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616561+0000", 2024-09-16T00:08:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015940600000000001 2024-09-16T00:08:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618146+0000", 2024-09-16T00:08:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015857040000000001 2024-09-16T00:08:38.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620091+0000", 2024-09-16T00:08:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001944837 2024-09-16T00:08:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620126+0000", 2024-09-16T00:08:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4242999999999997e-05 2024-09-16T00:08:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620152+0000", 2024-09-16T00:08:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6914e-05 2024-09-16T00:08:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.066328562000002, 2024-09-16T00:08:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00630628, 2024-09-16T00:08:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.786 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.788 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.788 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:08:38.788 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5999999999999999e-07 2024-09-16T00:08:38.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.788 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.788 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.788 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616022+0000", 2024-09-16T00:08:38.788 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0149999999999999e-06 2024-09-16T00:08:38.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.788 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.788 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.788 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616031+0000", 2024-09-16T00:08:38.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9770000000000006e-06 2024-09-16T00:08:38.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.789 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.789 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616036+0000", 2024-09-16T00:08:38.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2639999999999999e-06 2024-09-16T00:08:38.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.789 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.789 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617057+0000", 2024-09-16T00:08:38.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010212999999999999 2024-09-16T00:08:38.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617146+0000", 2024-09-16T00:08:38.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9234000000000007e-05 2024-09-16T00:08:38.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:38.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617232+0000", 2024-09-16T00:08:38.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5218999999999998e-05 2024-09-16T00:08:38.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:38.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619107+0000", 2024-09-16T00:08:38.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001875016 2024-09-16T00:08:38.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622259+0000", 2024-09-16T00:08:38.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031527930000000001 2024-09-16T00:08:38.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622290+0000", 2024-09-16T00:08:38.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1022999999999998e-05 2024-09-16T00:08:38.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622324+0000", 2024-09-16T00:08:38.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3979000000000001e-05 2024-09-16T00:08:38.792 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.792 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.792 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.792 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:38.792 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:38.792 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.066303259999998, 2024-09-16T00:08:38.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042412739999999997, 2024-09-16T00:08:38.793 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.793 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.793 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:38.793 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:38.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:38.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616044+0000", 2024-09-16T00:08:38.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:38.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616052+0000", 2024-09-16T00:08:38.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.439e-06 2024-09-16T00:08:38.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.795 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616059+0000", 2024-09-16T00:08:38.795 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1389999999999998e-06 2024-09-16T00:08:38.795 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.795 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.795 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.795 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616065+0000", 2024-09-16T00:08:38.795 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:08:38.795 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.795 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.795 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.795 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616823+0000", 2024-09-16T00:08:38.795 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075781199999999996 2024-09-16T00:08:38.795 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616918+0000", 2024-09-16T00:08:38.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5174999999999996e-05 2024-09-16T00:08:38.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:38.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617019+0000", 2024-09-16T00:08:38.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101161 2024-09-16T00:08:38.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:38.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618503+0000", 2024-09-16T00:08:38.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014836750000000001 2024-09-16T00:08:38.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:38.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620201+0000", 2024-09-16T00:08:38.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00169805 2024-09-16T00:08:38.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:38.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620255+0000", 2024-09-16T00:08:38.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4499000000000001e-05 2024-09-16T00:08:38.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620285+0000", 2024-09-16T00:08:38.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9349e-05 2024-09-16T00:08:38.798 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.798 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.798 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.799 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:38.799 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:38.799 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.066274268999997, 2024-09-16T00:08:38.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063682799999999996, 2024-09-16T00:08:38.799 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:38.799 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:38.799 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:38.800 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:38.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.800 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:38.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:38.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.800 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:38.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:38.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:38.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.801 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.801 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:38.801 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:38.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9900000000000001e-07 2024-09-16T00:08:38.801 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.801 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.801 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:38.801 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616079+0000", 2024-09-16T00:08:38.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9009999999999999e-06 2024-09-16T00:08:38.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:38.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616086+0000", 2024-09-16T00:08:38.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.776e-06 2024-09-16T00:08:38.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:38.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616091+0000", 2024-09-16T00:08:38.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5600000000000001e-06 2024-09-16T00:08:38.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:38.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618339+0000", 2024-09-16T00:08:38.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022476179999999998 2024-09-16T00:08:38.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:38.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:38.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9025999999999997e-05 2024-09-16T00:08:38.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:38.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:38.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:38.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622441+0000", 2024-09-16T00:08:38.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040029999999999996 2024-09-16T00:08:38.804 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.804 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.804 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.805 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:38.805 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:38.805 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:38.807 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:08:38.808 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:38.805+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:38.907 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:38.906+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:38.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.914+0000 7fb7b97fa640 1 -- 172.21.15.139:0/1210806254 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v48) ==== 101+0+0 (secure 0 0 0) 0x7fb7a807e5e0 con 0x7fb7bc0ccc90 2024-09-16T00:08:38.915 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is unset 2024-09-16T00:08:38.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.918+0000 7fb7c3217640 1 -- 172.21.15.139:0/1210806254 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb7a006c2b0 msgr2=0x7fb7a006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:38.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.918+0000 7fb7c3217640 1 --2- 172.21.15.139:0/1210806254 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb7a006c2b0 0x7fb7a006e770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000ab60 tx=0x7fb7b0008040 comp rx=0 tx=0).stop 2024-09-16T00:08:38.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.918+0000 7fb7c3217640 1 -- 172.21.15.139:0/1210806254 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7bc0ccc90 msgr2=0x7fb7bc1f1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:38.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.918+0000 7fb7c3217640 1 --2- 172.21.15.139:0/1210806254 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7bc0ccc90 0x7fb7bc1f1230 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009850 tx=0x7fb7a80046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:38.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.918+0000 7fb7c3217640 1 -- 172.21.15.139:0/1210806254 shutdown_connections 2024-09-16T00:08:38.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.918+0000 7fb7c3217640 1 --2- 172.21.15.139:0/1210806254 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb7a006c2b0 0x7fb7a006e770 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:38.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.918+0000 7fb7c3217640 1 --2- 172.21.15.139:0/1210806254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7bc154c20 0x7fb7bc146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:38.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.918+0000 7fb7c3217640 1 --2- 172.21.15.139:0/1210806254 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7bc0ccc90 0x7fb7bc1f1230 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:38.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.918+0000 7fb7c3217640 1 --2- 172.21.15.139:0/1210806254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7bc068060 0x7fb7bc0ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:38.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.918+0000 7fb7c3217640 1 -- 172.21.15.139:0/1210806254 >> 172.21.15.139:0/1210806254 conn(0x7fb7bc06ca40 msgr2=0x7fb7bc0c68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:38.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.918+0000 7fb7c3217640 1 -- 172.21.15.139:0/1210806254 shutdown_connections 2024-09-16T00:08:38.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:38.918+0000 7fb7c3217640 1 -- 172.21.15.139:0/1210806254 wait complete. 2024-09-16T00:08:38.964 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:38.965 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:38.965 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:38.965 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:38.977 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:08:39.008 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:39.007+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:39.109 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:39.109+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:39.178 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:39.178 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:39.178 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:39.178 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:39.178 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:39.190 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:08:39.210 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:39.209+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:39.311 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:39.311+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:39.395 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:39.395 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:39.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:39.395 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:39.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.395 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:39.395 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:39.396 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.778107046000002, 2024-09-16T00:08:39.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037514929999999998, 2024-09-16T00:08:39.396 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.396 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.396 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:39.396 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:39.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:39.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615364+0000", 2024-09-16T00:08:39.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8430000000000003e-06 2024-09-16T00:08:39.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615371+0000", 2024-09-16T00:08:39.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.148e-06 2024-09-16T00:08:39.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615380+0000", 2024-09-16T00:08:39.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5889999999999992e-06 2024-09-16T00:08:39.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615389+0000", 2024-09-16T00:08:39.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1620000000000008e-06 2024-09-16T00:08:39.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615525+0000", 2024-09-16T00:08:39.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000136083 2024-09-16T00:08:39.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615662+0000", 2024-09-16T00:08:39.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013696100000000001 2024-09-16T00:08:39.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:39.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615799+0000", 2024-09-16T00:08:39.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013781399999999999 2024-09-16T00:08:39.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:39.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:39.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002363502 2024-09-16T00:08:39.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:39.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619018+0000", 2024-09-16T00:08:39.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085522199999999999 2024-09-16T00:08:39.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:39.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619068+0000", 2024-09-16T00:08:39.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9545000000000001e-05 2024-09-16T00:08:39.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619111+0000", 2024-09-16T00:08:39.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3624000000000001e-05 2024-09-16T00:08:39.401 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.401 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.401 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.401 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:39.401 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:39.401 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.778054975000003, 2024-09-16T00:08:39.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043656110000000001, 2024-09-16T00:08:39.401 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.401 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.401 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:39.402 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:39.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:39.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:39.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2199999999999999e-07 2024-09-16T00:08:39.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615416+0000", 2024-09-16T00:08:39.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5590000000000001e-06 2024-09-16T00:08:39.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615423+0000", 2024-09-16T00:08:39.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6250000000000001e-06 2024-09-16T00:08:39.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615426+0000", 2024-09-16T00:08:39.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2540000000000001e-06 2024-09-16T00:08:39.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616075+0000", 2024-09-16T00:08:39.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064968800000000004 2024-09-16T00:08:39.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616142+0000", 2024-09-16T00:08:39.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6203000000000003e-05 2024-09-16T00:08:39.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:39.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616247+0000", 2024-09-16T00:08:39.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010572499999999999 2024-09-16T00:08:39.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:39.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618913+0000", 2024-09-16T00:08:39.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026654339999999999 2024-09-16T00:08:39.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:39.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619712+0000", 2024-09-16T00:08:39.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079885199999999996 2024-09-16T00:08:39.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:39.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619744+0000", 2024-09-16T00:08:39.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2790000000000003e-05 2024-09-16T00:08:39.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619778+0000", 2024-09-16T00:08:39.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3059000000000003e-05 2024-09-16T00:08:39.406 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.406 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.406 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.407 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:39.407 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:39.407 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.778018895999999, 2024-09-16T00:08:39.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087673219999999993, 2024-09-16T00:08:39.407 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.407 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.407 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:39.407 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:39.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:39.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:39.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:39.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:08:39.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5392000000000001e-05 2024-09-16T00:08:39.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615470+0000", 2024-09-16T00:08:39.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1999999999999999e-06 2024-09-16T00:08:39.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615474+0000", 2024-09-16T00:08:39.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7359999999999999e-06 2024-09-16T00:08:39.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617614+0000", 2024-09-16T00:08:39.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021398189999999998 2024-09-16T00:08:39.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617724+0000", 2024-09-16T00:08:39.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110808 2024-09-16T00:08:39.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:39.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617856+0000", 2024-09-16T00:08:39.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131676 2024-09-16T00:08:39.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:39.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:39.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029629280000000001 2024-09-16T00:08:39.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:39.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624158+0000", 2024-09-16T00:08:39.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033386549999999998 2024-09-16T00:08:39.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:39.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624189+0000", 2024-09-16T00:08:39.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1508999999999999e-05 2024-09-16T00:08:39.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624215+0000", 2024-09-16T00:08:39.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6228999999999999e-05 2024-09-16T00:08:39.412 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.412 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.412 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.412 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:39.412 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:39.412 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.777984881000002, 2024-09-16T00:08:39.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053285279999999999, 2024-09-16T00:08:39.413 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.413 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.413 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:39.413 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:39.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:39.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:39.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5600000000000002e-07 2024-09-16T00:08:39.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615484+0000", 2024-09-16T00:08:39.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.688e-06 2024-09-16T00:08:39.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615488+0000", 2024-09-16T00:08:39.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.658e-06 2024-09-16T00:08:39.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615490+0000", 2024-09-16T00:08:39.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.728e-06 2024-09-16T00:08:39.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616441+0000", 2024-09-16T00:08:39.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00095065500000000001 2024-09-16T00:08:39.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616539+0000", 2024-09-16T00:08:39.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8335e-05 2024-09-16T00:08:39.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:39.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616631+0000", 2024-09-16T00:08:39.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1260000000000004e-05 2024-09-16T00:08:39.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:39.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619837+0000", 2024-09-16T00:08:39.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003206899 2024-09-16T00:08:39.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:39.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620725+0000", 2024-09-16T00:08:39.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088713599999999996 2024-09-16T00:08:39.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:39.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620772+0000", 2024-09-16T00:08:39.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7639999999999998e-05 2024-09-16T00:08:39.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620811+0000", 2024-09-16T00:08:39.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8272999999999998e-05 2024-09-16T00:08:39.418 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.418 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.418 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.418 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:39.418 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:39.418 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.777569792999998, 2024-09-16T00:08:39.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029886169999999998, 2024-09-16T00:08:39.418 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.419 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.419 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:39.419 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:39.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:39.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615898+0000", 2024-09-16T00:08:39.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.237e-06 2024-09-16T00:08:39.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615904+0000", 2024-09-16T00:08:39.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:39.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615915+0000", 2024-09-16T00:08:39.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1003e-05 2024-09-16T00:08:39.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615922+0000", 2024-09-16T00:08:39.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9959999999999996e-06 2024-09-16T00:08:39.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615988+0000", 2024-09-16T00:08:39.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5806e-05 2024-09-16T00:08:39.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616009+0000", 2024-09-16T00:08:39.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0729000000000001e-05 2024-09-16T00:08:39.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:39.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:39.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028462130000000002 2024-09-16T00:08:39.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618886+0000", 2024-09-16T00:08:39.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.082e-05 2024-09-16T00:08:39.422 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.422 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.422 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.423 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:39.423 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:39.423 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.775958320999997, 2024-09-16T00:08:39.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032610299999999998, 2024-09-16T00:08:39.423 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.423 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.423 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:39.423 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:39.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:39.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617556+0000", 2024-09-16T00:08:39.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7431e-05 2024-09-16T00:08:39.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617847+0000", 2024-09-16T00:08:39.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029133400000000003 2024-09-16T00:08:39.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617858+0000", 2024-09-16T00:08:39.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0222e-05 2024-09-16T00:08:39.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617864+0000", 2024-09-16T00:08:39.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7619999999999998e-06 2024-09-16T00:08:39.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617909+0000", 2024-09-16T00:08:39.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4747999999999999e-05 2024-09-16T00:08:39.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617927+0000", 2024-09-16T00:08:39.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7785999999999999e-05 2024-09-16T00:08:39.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:39.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620734+0000", 2024-09-16T00:08:39.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002807305 2024-09-16T00:08:39.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620770+0000", 2024-09-16T00:08:39.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5441999999999997e-05 2024-09-16T00:08:39.427 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.427 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.427 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.427 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:39.427 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:39.427 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.775537636999999, 2024-09-16T00:08:39.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028641809999999999, 2024-09-16T00:08:39.427 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.428 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.428 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:39.428 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:39.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:39.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617930+0000", 2024-09-16T00:08:39.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0200000000000002e-07 2024-09-16T00:08:39.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617934+0000", 2024-09-16T00:08:39.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:39.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617943+0000", 2024-09-16T00:08:39.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5820000000000002e-06 2024-09-16T00:08:39.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617949+0000", 2024-09-16T00:08:39.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5069999999999997e-06 2024-09-16T00:08:39.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618094+0000", 2024-09-16T00:08:39.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014515599999999999 2024-09-16T00:08:39.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618111+0000", 2024-09-16T00:08:39.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6705999999999999e-05 2024-09-16T00:08:39.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:39.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:08:39.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026603070000000002 2024-09-16T00:08:39.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620793+0000", 2024-09-16T00:08:39.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2214000000000001e-05 2024-09-16T00:08:39.431 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.431 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.431 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.432 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:39.432 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:39.432 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.775506847999999, 2024-09-16T00:08:39.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028624549999999999, 2024-09-16T00:08:39.432 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.432 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.432 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:39.432 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:39.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:39.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:39.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:39.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:39.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0570000000000001e-06 2024-09-16T00:08:39.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617970+0000", 2024-09-16T00:08:39.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.669e-06 2024-09-16T00:08:39.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617976+0000", 2024-09-16T00:08:39.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1369999999999999e-06 2024-09-16T00:08:39.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618268+0000", 2024-09-16T00:08:39.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029164600000000001 2024-09-16T00:08:39.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618284+0000", 2024-09-16T00:08:39.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5824e-05 2024-09-16T00:08:39.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:39.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:39.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025127890000000001 2024-09-16T00:08:39.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620822+0000", 2024-09-16T00:08:39.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6009000000000001e-05 2024-09-16T00:08:39.436 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.436 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.436 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.436 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-16T00:08:39.436 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:39.436 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.775479267000001, 2024-09-16T00:08:39.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028650350000000002, 2024-09-16T00:08:39.437 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.437 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.437 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:39.437 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:39.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:39.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:39.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7599999999999998e-07 2024-09-16T00:08:39.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617991+0000", 2024-09-16T00:08:39.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1860000000000001e-06 2024-09-16T00:08:39.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617999+0000", 2024-09-16T00:08:39.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3250000000000002e-06 2024-09-16T00:08:39.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618004+0000", 2024-09-16T00:08:39.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7300000000000002e-06 2024-09-16T00:08:39.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618422+0000", 2024-09-16T00:08:39.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00041763199999999998 2024-09-16T00:08:39.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:39.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6112000000000001e-05 2024-09-16T00:08:39.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:39.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620824+0000", 2024-09-16T00:08:39.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002385954 2024-09-16T00:08:39.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620853+0000", 2024-09-16T00:08:39.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8719999999999999e-05 2024-09-16T00:08:39.440 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.441 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.441 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.441 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-16T00:08:39.441 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:39.441 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.775453444999997, 2024-09-16T00:08:39.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032703020000000001, 2024-09-16T00:08:39.441 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.441 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.441 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:39.441 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:39.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:39.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618014+0000", 2024-09-16T00:08:39.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7799999999999997e-07 2024-09-16T00:08:39.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618017+0000", 2024-09-16T00:08:39.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4309999999999998e-06 2024-09-16T00:08:39.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618023+0000", 2024-09-16T00:08:39.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8490000000000002e-06 2024-09-16T00:08:39.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618028+0000", 2024-09-16T00:08:39.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3489999999999999e-06 2024-09-16T00:08:39.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:08:39.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055724099999999999 2024-09-16T00:08:39.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618603+0000", 2024-09-16T00:08:39.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7486000000000002e-05 2024-09-16T00:08:39.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:39.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621235+0000", 2024-09-16T00:08:39.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002632087 2024-09-16T00:08:39.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621284+0000", 2024-09-16T00:08:39.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8581000000000001e-05 2024-09-16T00:08:39.445 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.445 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.445 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.445 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:14 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:39.445 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:39.445 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.774516464999998, 2024-09-16T00:08:39.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016807599999999999, 2024-09-16T00:08:39.446 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.446 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.446 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:39.446 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:39.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:39.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618951+0000", 2024-09-16T00:08:39.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:39.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618954+0000", 2024-09-16T00:08:39.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.145e-06 2024-09-16T00:08:39.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618963+0000", 2024-09-16T00:08:39.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5720000000000003e-06 2024-09-16T00:08:39.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618969+0000", 2024-09-16T00:08:39.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3319999999999997e-06 2024-09-16T00:08:39.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619008+0000", 2024-09-16T00:08:39.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9096e-05 2024-09-16T00:08:39.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619013+0000", 2024-09-16T00:08:39.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4710000000000001e-06 2024-09-16T00:08:39.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619118+0000", 2024-09-16T00:08:39.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000105809 2024-09-16T00:08:39.449 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.449 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.449 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.449 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-16T00:08:39.449 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:39.450 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.774478358000003, 2024-09-16T00:08:39.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002325307, 2024-09-16T00:08:39.450 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.450 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.450 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:39.450 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:39.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:39.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:39.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:39.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618993+0000", 2024-09-16T00:08:39.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.687e-06 2024-09-16T00:08:39.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619000+0000", 2024-09-16T00:08:39.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3590000000000002e-06 2024-09-16T00:08:39.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619005+0000", 2024-09-16T00:08:39.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.95e-06 2024-09-16T00:08:39.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619039+0000", 2024-09-16T00:08:39.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3942999999999997e-05 2024-09-16T00:08:39.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619055+0000", 2024-09-16T00:08:39.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6107e-05 2024-09-16T00:08:39.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:39.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621287+0000", 2024-09-16T00:08:39.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022318070000000001 2024-09-16T00:08:39.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:39.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7129999999999999e-05 2024-09-16T00:08:39.454 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.454 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.454 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.454 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-16T00:08:39.454 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:39.454 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.774440362, 2024-09-16T00:08:39.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002314328, 2024-09-16T00:08:39.454 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.454 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.454 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:39.454 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:39.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:39.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:39.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6399999999999998e-07 2024-09-16T00:08:39.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619030+0000", 2024-09-16T00:08:39.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.219e-06 2024-09-16T00:08:39.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619038+0000", 2024-09-16T00:08:39.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9899999999999997e-06 2024-09-16T00:08:39.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619043+0000", 2024-09-16T00:08:39.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5580000000000002e-06 2024-09-16T00:08:39.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619220+0000", 2024-09-16T00:08:39.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000176937 2024-09-16T00:08:39.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619237+0000", 2024-09-16T00:08:39.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7691999999999999e-05 2024-09-16T00:08:39.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:39.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621315+0000", 2024-09-16T00:08:39.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020778900000000002 2024-09-16T00:08:39.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621341+0000", 2024-09-16T00:08:39.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5678e-05 2024-09-16T00:08:39.458 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.458 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.458 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.458 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:15 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:39.458 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:39.458 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.773825393000003, 2024-09-16T00:08:39.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143416, 2024-09-16T00:08:39.459 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.459 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.459 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:39.459 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:39.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:39.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619642+0000", 2024-09-16T00:08:39.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5199999999999995e-07 2024-09-16T00:08:39.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619645+0000", 2024-09-16T00:08:39.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.134e-06 2024-09-16T00:08:39.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619655+0000", 2024-09-16T00:08:39.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3330000000000003e-06 2024-09-16T00:08:39.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619662+0000", 2024-09-16T00:08:39.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0500000000000003e-06 2024-09-16T00:08:39.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619704+0000", 2024-09-16T00:08:39.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1766e-05 2024-09-16T00:08:39.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619708+0000", 2024-09-16T00:08:39.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3440000000000003e-06 2024-09-16T00:08:39.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619785+0000", 2024-09-16T00:08:39.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6936999999999999e-05 2024-09-16T00:08:39.462 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.462 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.462 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.462 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:31 2.d e22/19 ondisk, result = 0)", 2024-09-16T00:08:39.463 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:39.463 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.773726584000002, 2024-09-16T00:08:39.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101826, 2024-09-16T00:08:39.463 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.463 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.463 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:39.463 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.463 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:39.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:39.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619741+0000", 2024-09-16T00:08:39.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3099999999999997e-07 2024-09-16T00:08:39.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:08:39.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1719999999999998e-06 2024-09-16T00:08:39.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619771+0000", 2024-09-16T00:08:39.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4754000000000001e-05 2024-09-16T00:08:39.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619790+0000", 2024-09-16T00:08:39.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9516000000000001e-05 2024-09-16T00:08:39.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619830+0000", 2024-09-16T00:08:39.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9848000000000002e-05 2024-09-16T00:08:39.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619834+0000", 2024-09-16T00:08:39.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1150000000000004e-06 2024-09-16T00:08:39.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619842+0000", 2024-09-16T00:08:39.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.79e-06 2024-09-16T00:08:39.466 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.466 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.466 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.467 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:30 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:39.467 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:39.467 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.769375597, 2024-09-16T00:08:39.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013076400000000001, 2024-09-16T00:08:39.467 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.467 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.467 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:39.467 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:39.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:39.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624092+0000", 2024-09-16T00:08:39.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5799999999999999e-07 2024-09-16T00:08:39.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624096+0000", 2024-09-16T00:08:39.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:39.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624106+0000", 2024-09-16T00:08:39.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7629999999999995e-06 2024-09-16T00:08:39.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624112+0000", 2024-09-16T00:08:39.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.626e-06 2024-09-16T00:08:39.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624149+0000", 2024-09-16T00:08:39.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7094000000000001e-05 2024-09-16T00:08:39.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624154+0000", 2024-09-16T00:08:39.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7359999999999998e-06 2024-09-16T00:08:39.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624222+0000", 2024-09-16T00:08:39.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7680000000000003e-05 2024-09-16T00:08:39.470 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.471 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.471 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.471 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:39 2.6 2:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:39.471 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:39.471 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.753189038999999, 2024-09-16T00:08:39.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050072119999999996, 2024-09-16T00:08:39.471 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.471 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.471 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:39.471 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:39.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:39.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640280+0000", 2024-09-16T00:08:39.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5380000000000001e-06 2024-09-16T00:08:39.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640296+0000", 2024-09-16T00:08:39.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5950000000000001e-05 2024-09-16T00:08:39.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640327+0000", 2024-09-16T00:08:39.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0309999999999999e-05 2024-09-16T00:08:39.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640340+0000", 2024-09-16T00:08:39.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3773e-05 2024-09-16T00:08:39.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640422+0000", 2024-09-16T00:08:39.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1122e-05 2024-09-16T00:08:39.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640572+0000", 2024-09-16T00:08:39.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015044099999999999 2024-09-16T00:08:39.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:08:39.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640691+0000", 2024-09-16T00:08:39.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000119393 2024-09-16T00:08:39.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:39.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645006+0000", 2024-09-16T00:08:39.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043143779999999998 2024-09-16T00:08:39.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:39.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645169+0000", 2024-09-16T00:08:39.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016323299999999999 2024-09-16T00:08:39.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:39.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645227+0000", 2024-09-16T00:08:39.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8e-05 2024-09-16T00:08:39.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645285+0000", 2024-09-16T00:08:39.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8074e-05 2024-09-16T00:08:39.476 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.476 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.476 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.477 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:39 2.6 e22/19 ondisk, result = 0)", 2024-09-16T00:08:39.477 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:39.477 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.748616927999997, 2024-09-16T00:08:39.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016541200000000001, 2024-09-16T00:08:39.477 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.477 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.477 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:39.477 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:39.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:39.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644852+0000", 2024-09-16T00:08:39.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1670000000000002e-06 2024-09-16T00:08:39.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644865+0000", 2024-09-16T00:08:39.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2826000000000001e-05 2024-09-16T00:08:39.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644894+0000", 2024-09-16T00:08:39.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9264999999999998e-05 2024-09-16T00:08:39.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644911+0000", 2024-09-16T00:08:39.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6827999999999999e-05 2024-09-16T00:08:39.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644989+0000", 2024-09-16T00:08:39.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7849999999999995e-05 2024-09-16T00:08:39.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:39.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644999+0000", 2024-09-16T00:08:39.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7289999999999995e-06 2024-09-16T00:08:39.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645015+0000", 2024-09-16T00:08:39.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6747000000000001e-05 2024-09-16T00:08:39.480 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.480 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.480 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.481 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.6 38/37 [PushReplyOp(2:654134d2:::mds0_openfiles.0:head)])", 2024-09-16T00:08:39.481 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.845546+0000", 2024-09-16T00:08:39.481 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.547921077, 2024-09-16T00:08:39.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00023895899999999999, 2024-09-16T00:08:39.481 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:39.481 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:39.481 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:39.481 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:39.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:39.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845546+0000", 2024-09-16T00:08:39.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:39.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845546+0000", 2024-09-16T00:08:39.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:39.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:39.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845546+0000", 2024-09-16T00:08:39.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1299999999999999e-07 2024-09-16T00:08:39.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:39.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845549+0000", 2024-09-16T00:08:39.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6000000000000001e-06 2024-09-16T00:08:39.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:39.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845557+0000", 2024-09-16T00:08:39.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0970000000000006e-06 2024-09-16T00:08:39.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:39.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845564+0000", 2024-09-16T00:08:39.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3839999999999999e-06 2024-09-16T00:08:39.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:39.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845616+0000", 2024-09-16T00:08:39.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2120000000000002e-05 2024-09-16T00:08:39.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:39.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:39.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:39.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845785+0000", 2024-09-16T00:08:39.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016854500000000001 2024-09-16T00:08:39.484 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.484 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.484 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.484 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:39.484 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:39.484 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:39.486 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:08:39.486 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:39.485+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:39.567 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:08:39.586 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:39.586+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:39.669 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:39.669 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:39.669 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:39.669 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:39.682 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:08:39.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.710+0000 7fd9e616d640 1 Processor -- start 2024-09-16T00:08:39.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.710+0000 7fd9e616d640 1 -- start start 2024-09-16T00:08:39.711 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:39.710+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:39.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.711+0000 7fd9e616d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9e01510f0 0x7fd9e01576f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:39.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.711+0000 7fd9e616d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd9e006ca10 0x7fd9e006ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:39.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.711+0000 7fd9e616d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9e006d3c0 con 0x7fd9e00c3540 2024-09-16T00:08:39.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.711+0000 7fd9e616d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fd9e00c8a00 con 0x7fd9e006ca10 2024-09-16T00:08:39.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.711+0000 7fd9e616d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e006c580 con 0x7fd9e01510f0 2024-09-16T00:08:39.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.711+0000 7fd9deffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd9e00c3540 0x7fd9e00c7690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45558/0 (socket says 172.21.15.139:45558) 2024-09-16T00:08:39.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.711+0000 7fd9deffd640 1 -- 172.21.15.139:0/810710294 learned_addr learned my addr 172.21.15.139:0/810710294 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:39.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.711+0000 7fd9dffff640 1 --2- 172.21.15.139:0/810710294 >> v2:172.21.15.164:3300/0 conn(0x7fd9e006ca10 0x7fd9e006ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:39.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.711+0000 7fd9df7fe640 1 --2- 172.21.15.139:0/810710294 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9e01510f0 0x7fd9e01576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:39.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.711+0000 7fd9de7fc640 1 -- 172.21.15.139:0/810710294 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2263179816 0 0) 0x7fd9e006d3c0 con 0x7fd9e00c3540 2024-09-16T00:08:39.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.712+0000 7fd9de7fc640 1 -- 172.21.15.139:0/810710294 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9c8003520 con 0x7fd9e00c3540 2024-09-16T00:08:39.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.712+0000 7fd9df7fe640 1 -- 172.21.15.139:0/810710294 >> v2:172.21.15.164:3300/0 conn(0x7fd9e006ca10 msgr2=0x7fd9e006ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:39.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.712+0000 7fd9df7fe640 1 --2- 172.21.15.139:0/810710294 >> v2:172.21.15.164:3300/0 conn(0x7fd9e006ca10 0x7fd9e006ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:39.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.712+0000 7fd9df7fe640 1 -- 172.21.15.139:0/810710294 >> v1:172.21.15.139:6789/0 conn(0x7fd9e00c3540 legacy=0x7fd9e00c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:39.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.712+0000 7fd9df7fe640 1 -- 172.21.15.139:0/810710294 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e00c8480 con 0x7fd9e01510f0 2024-09-16T00:08:39.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.712+0000 7fd9dffff640 1 --2- 172.21.15.139:0/810710294 >> v2:172.21.15.164:3300/0 conn(0x7fd9e006ca10 0x7fd9e006ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.713+0000 7fd9df7fe640 1 --2- 172.21.15.139:0/810710294 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9e01510f0 0x7fd9e01576f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00097a0 tx=0x7fd9d002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9d28c93f4cf2a94 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.713+0000 7fd9de7fc640 1 -- 172.21.15.139:0/810710294 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0037070 con 0x7fd9e01510f0 2024-09-16T00:08:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.713+0000 7fd9de7fc640 1 -- 172.21.15.139:0/810710294 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d0032440 con 0x7fd9e01510f0 2024-09-16T00:08:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.713+0000 7fd9de7fc640 1 -- 172.21.15.139:0/810710294 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0032810 con 0x7fd9e01510f0 2024-09-16T00:08:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.713+0000 7fd9e616d640 1 -- 172.21.15.139:0/810710294 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9e01510f0 msgr2=0x7fd9e01576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.713+0000 7fd9e616d640 1 --2- 172.21.15.139:0/810710294 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9e01510f0 0x7fd9e01576f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00097a0 tx=0x7fd9d002a180 comp rx=0 tx=0).stop 2024-09-16T00:08:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.714+0000 7fd9e616d640 1 -- 172.21.15.139:0/810710294 shutdown_connections 2024-09-16T00:08:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.714+0000 7fd9e616d640 1 --2- 172.21.15.139:0/810710294 >> v2:172.21.15.164:3300/0 conn(0x7fd9e006ca10 0x7fd9e006ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.714+0000 7fd9e616d640 1 --2- 172.21.15.139:0/810710294 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9e01510f0 0x7fd9e01576f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.714+0000 7fd9e616d640 1 -- 172.21.15.139:0/810710294 >> 172.21.15.139:0/810710294 conn(0x7fd9e00cc740 msgr2=0x7fd9e00ccb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.714+0000 7fd9e616d640 1 -- 172.21.15.139:0/810710294 shutdown_connections 2024-09-16T00:08:39.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.714+0000 7fd9e616d640 1 -- 172.21.15.139:0/810710294 wait complete. 2024-09-16T00:08:39.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.715+0000 7fd9e616d640 1 Processor -- start 2024-09-16T00:08:39.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.715+0000 7fd9e616d640 1 -- start start 2024-09-16T00:08:39.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.716+0000 7fd9e616d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9e006ca10 0x7fd9e01330e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:39.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.716+0000 7fd9e616d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9e00c3540 0x7fd9e00cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:39.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.716+0000 7fd9e616d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e01510f0 0x7fd9e013f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.716+0000 7fd9e616d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e006bf30 con 0x7fd9e01510f0 2024-09-16T00:08:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.716+0000 7fd9e616d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e006bc50 con 0x7fd9e00c3540 2024-09-16T00:08:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.716+0000 7fd9e616d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e006c210 con 0x7fd9e006ca10 2024-09-16T00:08:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.716+0000 7fd9df7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9e006ca10 0x7fd9e01330e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.716+0000 7fd9deffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9e00c3540 0x7fd9e00cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.716+0000 7fd9df7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9e006ca10 0x7fd9e01330e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:48734/0 (socket says 172.21.15.139:48734) 2024-09-16T00:08:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.716+0000 7fd9df7fe640 1 -- 172.21.15.139:0/4260046182 learned_addr learned my addr 172.21.15.139:0/4260046182 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:39.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.716+0000 7fd9dffff640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e01510f0 0x7fd9e013f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:39.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.717+0000 7fd9df7fe640 1 -- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9e00c3540 msgr2=0x7fd9e00cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:39.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.717+0000 7fd9df7fe640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9e00c3540 0x7fd9e00cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:39.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.717+0000 7fd9df7fe640 1 -- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e01510f0 msgr2=0x7fd9e013f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:39.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.717+0000 7fd9df7fe640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e01510f0 0x7fd9e013f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:39.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.717+0000 7fd9df7fe640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e013d660 con 0x7fd9e006ca10 2024-09-16T00:08:39.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.717+0000 7fd9df7fe640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9e006ca10 0x7fd9e01330e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0000c00 tx=0x7fd9d00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:39.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.717+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0041020 con 0x7fd9e006ca10 2024-09-16T00:08:39.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.717+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d003d640 con 0x7fd9e006ca10 2024-09-16T00:08:39.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.717+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0037040 con 0x7fd9e006ca10 2024-09-16T00:08:39.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.718+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e013d940 con 0x7fd9e006ca10 2024-09-16T00:08:39.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.718+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e013e920 con 0x7fd9e006ca10 2024-09-16T00:08:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.719+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd9d0044020 con 0x7fd9e006ca10 2024-09-16T00:08:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.719+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fd9e00c7140 con 0x7fd9e006ca10 2024-09-16T00:08:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.720+0000 7fd9c7fff640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd9cc06c2b0 0x7fd9cc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.720+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(48..48 src has 1..48) ==== 5915+0+0 (secure 0 0 0) 0x7fd9d00abb00 con 0x7fd9e006ca10 2024-09-16T00:08:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.720+0000 7fd9deffd640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd9cc06c2b0 0x7fd9cc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:39.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.720+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v48) ==== 54+0+16 (secure 0 0 0) 0x7fd9d004b030 con 0x7fd9e006ca10 2024-09-16T00:08:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.721+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fd9e013eaf0 0x7fd9e00c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.721+0000 7fd9deffd640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd9cc06c2b0 0x7fd9cc06e770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd9e014f5f0 tx=0x7fd9d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.721+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd9e0133960 con 0x7fd9e013eaf0 2024-09-16T00:08:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.721+0000 7fd9dffff640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fd9e013eaf0 0x7fd9e00c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:39.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.722+0000 7fd9dffff640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fd9e013eaf0 0x7fd9e00c6120 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:39.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.723+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9d8011480 con 0x7fd9e013eaf0 2024-09-16T00:08:39.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.756+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd9e00cd610 con 0x7fd9e013eaf0 2024-09-16T00:08:39.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.759+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd9e00cd610 con 0x7fd9e013eaf0 2024-09-16T00:08:39.760 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:39.760 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:39.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.759+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fd9e013d120 0x7fd9e0137c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:39.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.759+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fd9e014f620 con 0x7fd9e013d120 2024-09-16T00:08:39.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.759+0000 7fd9df7fe640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fd9e013d120 0x7fd9e0137c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:39.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.760+0000 7fd9df7fe640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fd9e013d120 0x7fd9e0137c80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:39.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.761+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9e014f620 con 0x7fd9e013d120 2024-09-16T00:08:39.788 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:39.787+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:39.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.790+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd9e014f340 con 0x7fd9e013d120 2024-09-16T00:08:39.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.793+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd9e014f340 con 0x7fd9e013d120 2024-09-16T00:08:39.794 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:39.795 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:39.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.793+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fd9e00c6660 0x7fd9e01f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:39.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.793+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fd9e0074270 con 0x7fd9e00c6660 2024-09-16T00:08:39.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.794+0000 7fd9deffd640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fd9e00c6660 0x7fd9e01f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:39.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.794+0000 7fd9deffd640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fd9e00c6660 0x7fd9e01f4170 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:39.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.795+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9e0074270 con 0x7fd9e00c6660 2024-09-16T00:08:39.820 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:39.820 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:39.820 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:39.820 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:39.820 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:39.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.822+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd9e01f4980 con 0x7fd9e00c6660 2024-09-16T00:08:39.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.825+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd9e01f4980 con 0x7fd9e00c6660 2024-09-16T00:08:39.825 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:39.826 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:39.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.825+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fd9e0062b40 0x7fd9e0062f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:39.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.825+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fd9e01f4f50 con 0x7fd9e0062b40 2024-09-16T00:08:39.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.825+0000 7fd9dffff640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fd9e0062b40 0x7fd9e0062f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:39.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.825+0000 7fd9dffff640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fd9e0062b40 0x7fd9e0062f80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:39.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.826+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9e01f4f50 con 0x7fd9e0062b40 2024-09-16T00:08:39.831 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:08:39.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.842+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd9e01f4f50 con 0x7fd9e0062b40 2024-09-16T00:08:39.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.843+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd9e01f4f50 con 0x7fd9e0062b40 2024-09-16T00:08:39.844 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:39.845 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:39.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.843+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fd9e00cab30 0x7fd9e00cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:39.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.843+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fd9e01f4900 con 0x7fd9e00cab30 2024-09-16T00:08:39.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.844+0000 7fd9df7fe640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fd9e00cab30 0x7fd9e00cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:39.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.844+0000 7fd9df7fe640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fd9e00cab30 0x7fd9e00cafa0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:39.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.845+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9e01f4900 con 0x7fd9e00cab30 2024-09-16T00:08:39.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.861+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd9e00008d0 con 0x7fd9e00cab30 2024-09-16T00:08:39.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.863+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd9e00008d0 con 0x7fd9e00cab30 2024-09-16T00:08:39.864 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:39.864 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:39.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.863+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fd9e002cb50 0x7fd9e002cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:39.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.863+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fd9e00cb5f0 con 0x7fd9e002cb50 2024-09-16T00:08:39.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.863+0000 7fd9deffd640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fd9e002cb50 0x7fd9e002cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:39.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.864+0000 7fd9deffd640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fd9e002cb50 0x7fd9e002cf70 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:39.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.865+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9e00cb5f0 con 0x7fd9e002cb50 2024-09-16T00:08:39.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.878+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd9e00cb5f0 con 0x7fd9e002cb50 2024-09-16T00:08:39.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.880+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd9e00cb5f0 con 0x7fd9e002cb50 2024-09-16T00:08:39.881 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:39.882 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:39.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.881+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fd9e0207490 0x7fd9e0209880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:39.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.881+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fd9e020c650 con 0x7fd9e0207490 2024-09-16T00:08:39.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.881+0000 7fd9dffff640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fd9e0207490 0x7fd9e0209880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:39.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.881+0000 7fd9dffff640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fd9e0207490 0x7fd9e0209880 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:39.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.882+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9e020c650 con 0x7fd9e0207490 2024-09-16T00:08:39.889 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:39.888+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:39.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.896+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd9e0209dc0 con 0x7fd9e0207490 2024-09-16T00:08:39.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.898+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd9e0209dc0 con 0x7fd9e0207490 2024-09-16T00:08:39.898 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:39.899 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:39.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.898+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd9e020a440 0x7fd9e020a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:39.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.898+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fd9e020fa20 con 0x7fd9e020a440 2024-09-16T00:08:39.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.898+0000 7fd9df7fe640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd9e020a440 0x7fd9e020a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:39.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.899+0000 7fd9df7fe640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd9e020a440 0x7fd9e020a880 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:39.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.899+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9e020fa20 con 0x7fd9e020a440 2024-09-16T00:08:39.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.914+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd9e020ae30 con 0x7fd9e020a440 2024-09-16T00:08:39.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.916+0000 7fd9c7fff640 1 -- 172.21.15.139:0/4260046182 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd9e020ae30 con 0x7fd9e020a440 2024-09-16T00:08:39.917 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:39.918 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:39.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fd9e013eaf0 msgr2=0x7fd9e00c6120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fd9e013eaf0 0x7fd9e00c6120 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fd9e013d120 msgr2=0x7fd9e0137c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fd9e013d120 0x7fd9e0137c80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fd9e00c6660 msgr2=0x7fd9e01f4170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fd9e00c6660 0x7fd9e01f4170 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9dffff640 1 -- 172.21.15.139:0/4260046182 reap_dead start 2024-09-16T00:08:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fd9e0062b40 msgr2=0x7fd9e0062f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fd9e0062b40 0x7fd9e0062f80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fd9e00cab30 msgr2=0x7fd9e00cafa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fd9e00cab30 0x7fd9e00cafa0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fd9e002cb50 msgr2=0x7fd9e002cf70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fd9e002cb50 0x7fd9e002cf70 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:39.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fd9e0207490 msgr2=0x7fd9e0209880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fd9e0207490 0x7fd9e0209880 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd9e020a440 msgr2=0x7fd9e020a880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd9e020a440 0x7fd9e020a880 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9dffff640 1 -- 172.21.15.139:0/4260046182 reap_dead start 2024-09-16T00:08:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd9cc06c2b0 msgr2=0x7fd9cc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd9cc06c2b0 0x7fd9cc06e770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd9e014f5f0 tx=0x7fd9d4009040 comp rx=0 tx=0).stop 2024-09-16T00:08:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9e006ca10 msgr2=0x7fd9e01330e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9e006ca10 0x7fd9e01330e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0000c00 tx=0x7fd9d00046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 shutdown_connections 2024-09-16T00:08:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd9cc06c2b0 0x7fd9cc06e770 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 --2- 172.21.15.139:0/4260046182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9e006ca10 0x7fd9e01330e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.917+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 >> 172.21.15.139:0/4260046182 conn(0x7fd9e00cc740 msgr2=0x7fd9e00c7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:39.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.918+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 shutdown_connections 2024-09-16T00:08:39.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:39.918+0000 7fd9e616d640 1 -- 172.21.15.139:0/4260046182 wait complete. 2024-09-16T00:08:39.957 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:39.957 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:39.966 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:08:40.008 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:40.007+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:40.090 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:40.090+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:40.114 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:40.114 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:40.123 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:08:40.207 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:40.207+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:40.273 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:40.273 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:40.284 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:08:40.292 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:40.292+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:40.394 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:40.393+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:40.431 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:40.431 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:40.440 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:08:40.494 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:40.493+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:40.558 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:40.559 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:40.568 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:08:40.609 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:40.608+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:40.697 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:40.695+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:40.714 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:40.714 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:40.723 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:08:40.797 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:40.796+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:40.827 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:40.827 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:40.836 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:08:40.898 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:40.897+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:40.934 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T00:08:40.941 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:40.941 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:40.950 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:08:40.999 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:40.997+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:41.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.034+0000 7f8f27d0a640 1 Processor -- start 2024-09-16T00:08:41.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.034+0000 7f8f27d0a640 1 -- start start 2024-09-16T00:08:41.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.035+0000 7f8f27d0a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8f20068060 0x7f8f2015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.035+0000 7f8f27d0a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f200cc4b0 con 0x7f8f200ccc90 2024-09-16T00:08:41.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.035+0000 7f8f27d0a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f200cc790 con 0x7f8f20154c20 2024-09-16T00:08:41.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.035+0000 7f8f27d0a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f8f200c5830 con 0x7f8f20068060 2024-09-16T00:08:41.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.035+0000 7f8f262a7640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8f200ccc90 0x7f8f20165180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45560/0 (socket says 172.21.15.139:45560) 2024-09-16T00:08:41.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.035+0000 7f8f262a7640 1 -- 172.21.15.139:0/3156108730 learned_addr learned my addr 172.21.15.139:0/3156108730 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:41.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.035+0000 7f8f252a5640 1 --2- 172.21.15.139:0/3156108730 >> v2:172.21.15.164:3300/0 conn(0x7f8f20068060 0x7f8f2015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.035+0000 7f8f24aa4640 1 -- 172.21.15.139:0/3156108730 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3828600312 0 0) 0x7f8f200cc4b0 con 0x7f8f200ccc90 2024-09-16T00:08:41.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.035+0000 7f8f24aa4640 1 -- 172.21.15.139:0/3156108730 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f0c003520 con 0x7f8f200ccc90 2024-09-16T00:08:41.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.035+0000 7f8f24aa4640 1 -- 172.21.15.139:0/3156108730 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3008354787 0 0) 0x7f8f200cc790 con 0x7f8f20154c20 2024-09-16T00:08:41.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.035+0000 7f8f24aa4640 1 -- 172.21.15.139:0/3156108730 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f200cc4b0 con 0x7f8f20154c20 2024-09-16T00:08:41.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.036+0000 7f8f252a5640 1 -- 172.21.15.139:0/3156108730 >> v1:172.21.15.164:6789/0 conn(0x7f8f20154c20 legacy=0x7f8f200c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.036+0000 7f8f252a5640 1 -- 172.21.15.139:0/3156108730 >> v1:172.21.15.139:6789/0 conn(0x7f8f200ccc90 legacy=0x7f8f20165180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.036+0000 7f8f252a5640 1 -- 172.21.15.139:0/3156108730 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2015ddf0 con 0x7f8f20068060 2024-09-16T00:08:41.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.036+0000 7f8f252a5640 1 --2- 172.21.15.139:0/3156108730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f20068060 0x7f8f2015b280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009790 tx=0x7f8f1002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef471f4e8f9a0d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:41.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.037+0000 7f8f24aa4640 1 -- 172.21.15.139:0/3156108730 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10037070 con 0x7f8f20068060 2024-09-16T00:08:41.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.037+0000 7f8f24aa4640 1 -- 172.21.15.139:0/3156108730 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f10032440 con 0x7f8f20068060 2024-09-16T00:08:41.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.037+0000 7f8f24aa4640 1 -- 172.21.15.139:0/3156108730 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10032810 con 0x7f8f20068060 2024-09-16T00:08:41.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.037+0000 7f8f27d0a640 1 -- 172.21.15.139:0/3156108730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f20068060 msgr2=0x7f8f2015b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.037+0000 7f8f27d0a640 1 --2- 172.21.15.139:0/3156108730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f20068060 0x7f8f2015b280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009790 tx=0x7f8f1002a210 comp rx=0 tx=0).stop 2024-09-16T00:08:41.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.037+0000 7f8f27d0a640 1 -- 172.21.15.139:0/3156108730 shutdown_connections 2024-09-16T00:08:41.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.037+0000 7f8f27d0a640 1 --2- 172.21.15.139:0/3156108730 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f20068060 0x7f8f2015b280 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.037+0000 7f8f27d0a640 1 -- 172.21.15.139:0/3156108730 >> 172.21.15.139:0/3156108730 conn(0x7f8f2006ca40 msgr2=0x7f8f20067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:41.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.037+0000 7f8f27d0a640 1 -- 172.21.15.139:0/3156108730 shutdown_connections 2024-09-16T00:08:41.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.037+0000 7f8f27d0a640 1 -- 172.21.15.139:0/3156108730 wait complete. 2024-09-16T00:08:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.038+0000 7f8f27d0a640 1 Processor -- start 2024-09-16T00:08:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.039+0000 7f8f27d0a640 1 -- start start 2024-09-16T00:08:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.039+0000 7f8f27d0a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f20068060 0x7f8f2015cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.039+0000 7f8f27d0a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f200ccc90 0x7f8f2015d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.039+0000 7f8f27d0a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f20154c20 0x7f8f200b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.039+0000 7f8f27d0a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8f200cc260 con 0x7f8f20068060 2024-09-16T00:08:41.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.039+0000 7f8f27d0a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2015e3b0 con 0x7f8f20154c20 2024-09-16T00:08:41.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.039+0000 7f8f27d0a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8f2015e690 con 0x7f8f200ccc90 2024-09-16T00:08:41.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.039+0000 7f8f25aa6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f20068060 0x7f8f2015cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.039+0000 7f8f25aa6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f20068060 0x7f8f2015cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49328/0 (socket says 172.21.15.139:49328) 2024-09-16T00:08:41.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.039+0000 7f8f25aa6640 1 -- 172.21.15.139:0/3178549179 learned_addr learned my addr 172.21.15.139:0/3178549179 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:41.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.039+0000 7f8f262a7640 1 --2- 172.21.15.139:0/3178549179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f20154c20 0x7f8f200b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.039+0000 7f8f252a5640 1 --2- 172.21.15.139:0/3178549179 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f200ccc90 0x7f8f2015d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.040+0000 7f8f25aa6640 1 -- 172.21.15.139:0/3178549179 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f200ccc90 msgr2=0x7f8f2015d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.040+0000 7f8f25aa6640 1 --2- 172.21.15.139:0/3178549179 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f200ccc90 0x7f8f2015d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.040+0000 7f8f25aa6640 1 -- 172.21.15.139:0/3178549179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f20154c20 msgr2=0x7f8f200b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.040+0000 7f8f25aa6640 1 --2- 172.21.15.139:0/3178549179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f20154c20 0x7f8f200b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.040+0000 7f8f25aa6640 1 -- 172.21.15.139:0/3178549179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2015c2c0 con 0x7f8f20068060 2024-09-16T00:08:41.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.040+0000 7f8f252a5640 1 --2- 172.21.15.139:0/3178549179 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f200ccc90 0x7f8f2015d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:08:41.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.040+0000 7f8f25aa6640 1 --2- 172.21.15.139:0/3178549179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f20068060 0x7f8f2015cd60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00c430 tx=0x7f8f1c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:41.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.040+0000 7f8f167fc640 1 -- 172.21.15.139:0/3178549179 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c017070 con 0x7f8f20068060 2024-09-16T00:08:41.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.040+0000 7f8f167fc640 1 -- 172.21.15.139:0/3178549179 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f1c0052f0 con 0x7f8f20068060 2024-09-16T00:08:41.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.040+0000 7f8f167fc640 1 -- 172.21.15.139:0/3178549179 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c003830 con 0x7f8f20068060 2024-09-16T00:08:41.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.040+0000 7f8f27d0a640 1 -- 172.21.15.139:0/3178549179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f201f13d0 con 0x7f8f20068060 2024-09-16T00:08:41.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.040+0000 7f8f27d0a640 1 -- 172.21.15.139:0/3178549179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f20148510 con 0x7f8f20068060 2024-09-16T00:08:41.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.041+0000 7f8f27d0a640 1 -- 172.21.15.139:0/3178549179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f200cd720 con 0x7f8f20068060 2024-09-16T00:08:41.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.043+0000 7f8f167fc640 1 -- 172.21.15.139:0/3178549179 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8f1c01a070 con 0x7f8f20068060 2024-09-16T00:08:41.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.043+0000 7f8f167fc640 1 --2- 172.21.15.139:0/3178549179 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f0406c2b0 0x7f8f0406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.043+0000 7f8f252a5640 1 --2- 172.21.15.139:0/3178549179 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f0406c2b0 0x7f8f0406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.044+0000 7f8f167fc640 1 -- 172.21.15.139:0/3178549179 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5915+0+0 (secure 0 0 0) 0x7f8f1c08a970 con 0x7f8f20068060 2024-09-16T00:08:41.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.044+0000 7f8f252a5640 1 --2- 172.21.15.139:0/3178549179 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f0406c2b0 0x7f8f0406e770 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8f10034000 tx=0x7f8f1002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:41.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.048+0000 7f8f167fc640 1 -- 172.21.15.139:0/3178549179 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8f1c058890 con 0x7f8f20068060 2024-09-16T00:08:41.096 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:41.096 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:41.100 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:41.099+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:41.105 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:08:41.201 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:41.200+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:41.209 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:41.209 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:41.217 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:08:41.295 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:08:41.341 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:41.339+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:41.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.357+0000 7f8f27d0a640 1 -- 172.21.15.139:0/3178549179 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f8f200cca70 con 0x7f8f20068060 2024-09-16T00:08:41.362 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:41.362 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:41.371 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:08:41.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.394+0000 7f9c3f882640 1 Processor -- start 2024-09-16T00:08:41.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.394+0000 7f9c3f882640 1 -- start start 2024-09-16T00:08:41.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.395+0000 7f9c3f882640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9c380c8cc0 0x7f9c3806fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.395+0000 7f9c3f882640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9c38070120 0x7f9c38070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.395+0000 7f9c3f882640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c380cb640 con 0x7f9c381528b0 2024-09-16T00:08:41.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.395+0000 7f9c3f882640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9c38152c90 con 0x7f9c380c8cc0 2024-09-16T00:08:41.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.395+0000 7f9c3f882640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f9c380c9990 con 0x7f9c38070120 2024-09-16T00:08:41.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.395+0000 7f9c3ce1d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9c380c8cc0 0x7f9c3806fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.395+0000 7f9c3ce1d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9c380c8cc0 0x7f9c3806fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49340/0 (socket says 172.21.15.139:49340) 2024-09-16T00:08:41.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.396+0000 7f9c3ce1d640 1 -- 172.21.15.139:0/3050635853 learned_addr learned my addr 172.21.15.139:0/3050635853 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:41.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.396+0000 7f9c3de1f640 1 --2- 172.21.15.139:0/3050635853 >> v2:172.21.15.164:3300/0 conn(0x7f9c38070120 0x7f9c38070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.396+0000 7f9c3ce1d640 1 -- 172.21.15.139:0/3050635853 >> v2:172.21.15.164:3300/0 conn(0x7f9c38070120 msgr2=0x7f9c38070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:41.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.396+0000 7f9c3ce1d640 1 --2- 172.21.15.139:0/3050635853 >> v2:172.21.15.164:3300/0 conn(0x7f9c38070120 0x7f9c38070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.396+0000 7f9c3ce1d640 1 -- 172.21.15.139:0/3050635853 >> v1:172.21.15.164:6789/0 conn(0x7f9c381528b0 legacy=0x7f9c380caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.396+0000 7f9c3ce1d640 1 -- 172.21.15.139:0/3050635853 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c380c32e0 con 0x7f9c380c8cc0 2024-09-16T00:08:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.397+0000 7f9c3ce1d640 1 --2- 172.21.15.139:0/3050635853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c380c8cc0 0x7f9c3806fbe0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9c380739e0 tx=0x7f9c2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5305f4b80f98ab server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.397+0000 7f9c2ffff640 1 -- 172.21.15.139:0/3050635853 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c28037070 con 0x7f9c380c8cc0 2024-09-16T00:08:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.397+0000 7f9c2ffff640 1 -- 172.21.15.139:0/3050635853 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c28032440 con 0x7f9c380c8cc0 2024-09-16T00:08:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.397+0000 7f9c2ffff640 1 -- 172.21.15.139:0/3050635853 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c28032810 con 0x7f9c380c8cc0 2024-09-16T00:08:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.398+0000 7f9c3f882640 1 -- 172.21.15.139:0/3050635853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c380c8cc0 msgr2=0x7f9c3806fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.398+0000 7f9c3f882640 1 --2- 172.21.15.139:0/3050635853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c380c8cc0 0x7f9c3806fbe0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9c380739e0 tx=0x7f9c2802a270 comp rx=0 tx=0).stop 2024-09-16T00:08:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.398+0000 7f9c3f882640 1 -- 172.21.15.139:0/3050635853 shutdown_connections 2024-09-16T00:08:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.398+0000 7f9c3f882640 1 --2- 172.21.15.139:0/3050635853 >> v2:172.21.15.164:3300/0 conn(0x7f9c38070120 0x7f9c38070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.398+0000 7f9c3f882640 1 --2- 172.21.15.139:0/3050635853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c380c8cc0 0x7f9c3806fbe0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.398+0000 7f9c3f882640 1 -- 172.21.15.139:0/3050635853 >> 172.21.15.139:0/3050635853 conn(0x7f9c38067d70 msgr2=0x7f9c38068180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.398+0000 7f9c3f882640 1 -- 172.21.15.139:0/3050635853 shutdown_connections 2024-09-16T00:08:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.398+0000 7f9c3f882640 1 -- 172.21.15.139:0/3050635853 wait complete. 2024-09-16T00:08:41.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.400+0000 7f9c3f882640 1 Processor -- start 2024-09-16T00:08:41.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.400+0000 7f9c3f882640 1 -- start start 2024-09-16T00:08:41.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.401+0000 7f9c3f882640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c38070120 0x7f9c38155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.401+0000 7f9c3f882640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c380c8cc0 0x7f9c380c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.401+0000 7f9c3f882640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c381528b0 0x7f9c380c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.401+0000 7f9c3f882640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9c3815c860 con 0x7f9c380c8cc0 2024-09-16T00:08:41.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.401+0000 7f9c3f882640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9c380caaa0 con 0x7f9c38070120 2024-09-16T00:08:41.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.401+0000 7f9c3f882640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9c380c96b0 con 0x7f9c381528b0 2024-09-16T00:08:41.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.401+0000 7f9c3ce1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c380c8cc0 0x7f9c380c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.401+0000 7f9c3d61e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c38070120 0x7f9c38155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.401+0000 7f9c3ce1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c380c8cc0 0x7f9c380c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49342/0 (socket says 172.21.15.139:49342) 2024-09-16T00:08:41.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.401+0000 7f9c3ce1d640 1 -- 172.21.15.139:0/727930511 learned_addr learned my addr 172.21.15.139:0/727930511 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:41.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.402+0000 7f9c3de1f640 1 --2- 172.21.15.139:0/727930511 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c381528b0 0x7f9c380c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.402+0000 7f9c3ce1d640 1 -- 172.21.15.139:0/727930511 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c381528b0 msgr2=0x7f9c380c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.402+0000 7f9c3ce1d640 1 --2- 172.21.15.139:0/727930511 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c381528b0 0x7f9c380c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.402+0000 7f9c3ce1d640 1 -- 172.21.15.139:0/727930511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c38070120 msgr2=0x7f9c38155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.402+0000 7f9c3ce1d640 1 --2- 172.21.15.139:0/727930511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c38070120 0x7f9c38155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.402+0000 7f9c3ce1d640 1 -- 172.21.15.139:0/727930511 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c381f1bf0 con 0x7f9c380c8cc0 2024-09-16T00:08:41.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.402+0000 7f9c3de1f640 1 --2- 172.21.15.139:0/727930511 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c381528b0 0x7f9c380c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:08:41.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.402+0000 7f9c3ce1d640 1 --2- 172.21.15.139:0/727930511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c380c8cc0 0x7f9c380c7950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9c2802a2b0 tx=0x7f9c280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:41.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.402+0000 7f9c2dffb640 1 -- 172.21.15.139:0/727930511 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c28042070 con 0x7f9c380c8cc0 2024-09-16T00:08:41.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.402+0000 7f9c2dffb640 1 -- 172.21.15.139:0/727930511 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c28030640 con 0x7f9c380c8cc0 2024-09-16T00:08:41.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.402+0000 7f9c2dffb640 1 -- 172.21.15.139:0/727930511 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c28037040 con 0x7f9c380c8cc0 2024-09-16T00:08:41.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.402+0000 7f9c3f882640 1 -- 172.21.15.139:0/727930511 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c381f1ed0 con 0x7f9c380c8cc0 2024-09-16T00:08:41.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.403+0000 7f9c3f882640 1 -- 172.21.15.139:0/727930511 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c381f2490 con 0x7f9c380c8cc0 2024-09-16T00:08:41.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.404+0000 7f9c3f882640 1 -- 172.21.15.139:0/727930511 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c380ca730 con 0x7f9c380c8cc0 2024-09-16T00:08:41.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.405+0000 7f9c2dffb640 1 -- 172.21.15.139:0/727930511 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9c28045070 con 0x7f9c380c8cc0 2024-09-16T00:08:41.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.405+0000 7f9c2dffb640 1 --2- 172.21.15.139:0/727930511 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9c1c06c2e0 0x7f9c1c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.405+0000 7f9c2dffb640 1 -- 172.21.15.139:0/727930511 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5915+0+0 (secure 0 0 0) 0x7f9c280ab0b0 con 0x7f9c380c8cc0 2024-09-16T00:08:41.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.405+0000 7f9c3d61e640 1 --2- 172.21.15.139:0/727930511 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9c1c06c2e0 0x7f9c1c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.406+0000 7f9c3d61e640 1 --2- 172.21.15.139:0/727930511 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9c1c06c2e0 0x7f9c1c06e7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9c34014f50 tx=0x7f9c34016040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:41.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.410+0000 7f9c2dffb640 1 -- 172.21.15.139:0/727930511 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9c28079190 con 0x7f9c380c8cc0 2024-09-16T00:08:41.414 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:41.413+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:41.475 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:41.475 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:41.484 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:08:41.525 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:41.524+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:41.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.600+0000 7f9c3f882640 1 -- 172.21.15.139:0/727930511 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c380c47a0 con 0x7f9c1c06c2e0 2024-09-16T00:08:41.604 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:41.603+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.611+0000 7f9c2dffb640 1 -- 172.21.15.139:0/727930511 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251876 (secure 0 0 0) 0x7f9c380c47a0 con 0x7f9c1c06c2e0 2024-09-16T00:08:41.616 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:08:41.617 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:08:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.615+0000 7f9c3f882640 1 -- 172.21.15.139:0/727930511 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9c1c06c2e0 msgr2=0x7f9c1c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.615+0000 7f9c3f882640 1 --2- 172.21.15.139:0/727930511 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9c1c06c2e0 0x7f9c1c06e7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9c34014f50 tx=0x7f9c34016040 comp rx=0 tx=0).stop 2024-09-16T00:08:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.615+0000 7f9c3f882640 1 -- 172.21.15.139:0/727930511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c380c8cc0 msgr2=0x7f9c380c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.615+0000 7f9c3f882640 1 --2- 172.21.15.139:0/727930511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c380c8cc0 0x7f9c380c7950 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9c2802a2b0 tx=0x7f9c280046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.615+0000 7f9c3f882640 1 -- 172.21.15.139:0/727930511 shutdown_connections 2024-09-16T00:08:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.615+0000 7f9c3f882640 1 --2- 172.21.15.139:0/727930511 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9c1c06c2e0 0x7f9c1c06e7a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.615+0000 7f9c3f882640 1 --2- 172.21.15.139:0/727930511 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c381528b0 0x7f9c380c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.615+0000 7f9c3f882640 1 --2- 172.21.15.139:0/727930511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c380c8cc0 0x7f9c380c7950 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.615+0000 7f9c3f882640 1 --2- 172.21.15.139:0/727930511 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c38070120 0x7f9c38155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.615+0000 7f9c3f882640 1 -- 172.21.15.139:0/727930511 >> 172.21.15.139:0/727930511 conn(0x7f9c38067d70 msgr2=0x7f9c38071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.615+0000 7f9c3f882640 1 -- 172.21.15.139:0/727930511 shutdown_connections 2024-09-16T00:08:41.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.615+0000 7f9c3f882640 1 -- 172.21.15.139:0/727930511 wait complete. 2024-09-16T00:08:41.628 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:41.629 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:41.629 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-16T00:08:40.037034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597721,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":212,"num_write_kb":2439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":13110,"num_keys_recovered":2,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220472,"kb_used_data":4208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754248,"statfs":{"total":773094113280,"available":772868349952,"internally_reserved":0,"allocated":4308992,"data_stored":2974870,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":6555,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.167198"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888570+0000","last_change":"2024-09-16T00:08:36.888570+0000","last_active":"2024-09-16T00:08:36.888570+0000","last_peered":"2024-09-16T00:08:36.888570+0000","last_clean":"2024-09-16T00:08:36.888570+0000","last_became_active":"2024-09-16T00:08:36.888143+0000","last_became_peered":"2024-09-16T00:08:36.888143+0000","last_unstale":"2024-09-16T00:08:36.888570+0000","last_undegraded":"2024-09-16T00:08:36.888570+0000","last_fullsized":"2024-09-16T00:08:36.888570+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:30.579681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882794+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:08:35.882794+0000","last_peered":"2024-09-16T00:08:35.882794+0000","last_clean":"2024-09-16T00:08:35.882794+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:08:35.882794+0000","last_undegraded":"2024-09-16T00:08:35.882794+0000","last_fullsized":"2024-09-16T00:08:35.882794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.890336+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:08:35.890336+0000","last_peered":"2024-09-16T00:08:35.890336+0000","last_clean":"2024-09-16T00:08:35.890336+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:08:35.890336+0000","last_undegraded":"2024-09-16T00:08:35.890336+0000","last_fullsized":"2024-09-16T00:08:35.890336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888438+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:08:36.888438+0000","last_peered":"2024-09-16T00:08:36.888438+0000","last_clean":"2024-09-16T00:08:36.888438+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:08:36.888438+0000","last_undegraded":"2024-09-16T00:08:36.888438+0000","last_fullsized":"2024-09-16T00:08:36.888438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882059+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:08:35.882059+0000","last_peered":"2024-09-16T00:08:35.882059+0000","last_clean":"2024-09-16T00:08:35.882059+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:08:35.882059+0000","last_undegraded":"2024-09-16T00:08:35.882059+0000","last_fullsized":"2024-09-16T00:08:35.882059+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888197+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:08:36.888197+0000","last_peered":"2024-09-16T00:08:36.888197+0000","last_clean":"2024-09-16T00:08:36.888197+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:08:36.888197+0000","last_undegraded":"2024-09-16T00:08:36.888197+0000","last_fullsized":"2024-09-16T00:08:36.888197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882645+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:08:36.882645+0000","last_peered":"2024-09-16T00:08:36.882645+0000","last_clean":"2024-09-16T00:08:36.882645+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:08:36.882645+0000","last_undegraded":"2024-09-16T00:08:36.882645+0000","last_fullsized":"2024-09-16T00:08:36.882645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":32,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874260+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:08:34.874260+0000","last_peered":"2024-09-16T00:08:34.874260+0000","last_clean":"2024-09-16T00:08:34.874260+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:08:34.874260+0000","last_undegraded":"2024-09-16T00:08:34.874260+0000","last_fullsized":"2024-09-16T00:08:34.874260+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883724+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:08:36.883724+0000","last_peered":"2024-09-16T00:08:36.883724+0000","last_clean":"2024-09-16T00:08:36.883724+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:08:36.883724+0000","last_undegraded":"2024-09-16T00:08:36.883724+0000","last_fullsized":"2024-09-16T00:08:36.883724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.893637+0000","last_change":"2024-09-16T00:08:36.891868+0000","last_active":"2024-09-16T00:08:36.893637+0000","last_peered":"2024-09-16T00:08:36.893637+0000","last_clean":"2024-09-16T00:08:36.893637+0000","last_became_active":"2024-09-16T00:08:36.891605+0000","last_became_peered":"2024-09-16T00:08:36.891605+0000","last_unstale":"2024-09-16T00:08:36.893637+0000","last_undegraded":"2024-09-16T00:08:36.893637+0000","last_fullsized":"2024-09-16T00:08:36.893637+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:46:00.730436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.893332+0000","last_change":"2024-09-16T00:08:36.891589+0000","last_active":"2024-09-16T00:08:36.893332+0000","last_peered":"2024-09-16T00:08:36.893332+0000","last_clean":"2024-09-16T00:08:36.893332+0000","last_became_active":"2024-09-16T00:08:36.891279+0000","last_became_peered":"2024-09-16T00:08:36.891279+0000","last_unstale":"2024-09-16T00:08:36.893332+0000","last_undegraded":"2024-09-16T00:08:36.893332+0000","last_fullsized":"2024-09-16T00:08:36.893332+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888576+0000","last_change":"2024-09-16T00:08:36.888576+0000","last_active":"2024-09-16T00:08:36.888576+0000","last_peered":"2024-09-16T00:08:36.888576+0000","last_clean":"2024-09-16T00:08:36.888576+0000","last_became_active":"2024-09-16T00:08:36.888301+0000","last_became_peered":"2024-09-16T00:08:36.888301+0000","last_unstale":"2024-09-16T00:08:36.888576+0000","last_undegraded":"2024-09-16T00:08:36.888576+0000","last_fullsized":"2024-09-16T00:08:36.888576+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"peering","last_fresh":"2024-09-16T00:08:35.190711+0000","last_change":"2024-09-16T00:08:35.892992+0000","last_active":"2024-09-16T00:08:35.892992+0000","last_peered":"2024-09-16T00:08:35.190711+0000","last_clean":"2024-09-16T00:08:35.190711+0000","last_became_active":"2024-09-16T00:08:24.840637+0000","last_became_peered":"2024-09-16T00:08:24.840637+0000","last_unstale":"2024-09-16T00:08:35.190711+0000","last_undegraded":"2024-09-16T00:08:35.190711+0000","last_fullsized":"2024-09-16T00:08:35.190711+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888448+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:08:35.888448+0000","last_peered":"2024-09-16T00:08:35.888448+0000","last_clean":"2024-09-16T00:08:35.888448+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:08:35.888448+0000","last_undegraded":"2024-09-16T00:08:35.888448+0000","last_fullsized":"2024-09-16T00:08:35.888448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881726+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:08:35.881726+0000","last_peered":"2024-09-16T00:08:35.881726+0000","last_clean":"2024-09-16T00:08:35.881726+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:08:35.881726+0000","last_undegraded":"2024-09-16T00:08:35.881726+0000","last_fullsized":"2024-09-16T00:08:35.881726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889590+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:08:36.889590+0000","last_peered":"2024-09-16T00:08:36.889590+0000","last_clean":"2024-09-16T00:08:36.889590+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:08:36.889590+0000","last_undegraded":"2024-09-16T00:08:36.889590+0000","last_fullsized":"2024-09-16T00:08:36.889590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888829+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:08:36.888829+0000","last_peered":"2024-09-16T00:08:36.888829+0000","last_clean":"2024-09-16T00:08:36.888829+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:08:36.888829+0000","last_undegraded":"2024-09-16T00:08:36.888829+0000","last_fullsized":"2024-09-16T00:08:36.888829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874802+0000","last_change":"2024-09-16T00:08:24.836818+0000","last_active":"2024-09-16T00:08:34.874802+0000","last_peered":"2024-09-16T00:08:34.874802+0000","last_clean":"2024-09-16T00:08:34.874802+0000","last_became_active":"2024-09-16T00:08:24.836029+0000","last_became_peered":"2024-09-16T00:08:24.836029+0000","last_unstale":"2024-09-16T00:08:34.874802+0000","last_undegraded":"2024-09-16T00:08:34.874802+0000","last_fullsized":"2024-09-16T00:08:34.874802+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:38:55.816393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884263+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:08:36.884263+0000","last_peered":"2024-09-16T00:08:36.884263+0000","last_clean":"2024-09-16T00:08:36.884263+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:08:36.884263+0000","last_undegraded":"2024-09-16T00:08:36.884263+0000","last_fullsized":"2024-09-16T00:08:36.884263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.892945+0000","last_change":"2024-09-16T00:08:36.890676+0000","last_active":"2024-09-16T00:08:36.892945+0000","last_peered":"2024-09-16T00:08:36.892945+0000","last_clean":"2024-09-16T00:08:36.892945+0000","last_became_active":"2024-09-16T00:08:36.889743+0000","last_became_peered":"2024-09-16T00:08:36.889743+0000","last_unstale":"2024-09-16T00:08:36.892945+0000","last_undegraded":"2024-09-16T00:08:36.892945+0000","last_fullsized":"2024-09-16T00:08:36.892945+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:02.709496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888586+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:08:35.888586+0000","last_peered":"2024-09-16T00:08:35.888586+0000","last_clean":"2024-09-16T00:08:35.888586+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:08:35.888586+0000","last_undegraded":"2024-09-16T00:08:35.888586+0000","last_fullsized":"2024-09-16T00:08:35.888586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888289+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:08:36.888289+0000","last_peered":"2024-09-16T00:08:36.888289+0000","last_clean":"2024-09-16T00:08:36.888289+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:08:36.888289+0000","last_undegraded":"2024-09-16T00:08:36.888289+0000","last_fullsized":"2024-09-16T00:08:36.888289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.891321+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:08:35.891321+0000","last_peered":"2024-09-16T00:08:35.891321+0000","last_clean":"2024-09-16T00:08:35.891321+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:08:35.891321+0000","last_undegraded":"2024-09-16T00:08:35.891321+0000","last_fullsized":"2024-09-16T00:08:35.891321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889686+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:08:36.889686+0000","last_peered":"2024-09-16T00:08:36.889686+0000","last_clean":"2024-09-16T00:08:36.889686+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:08:36.889686+0000","last_undegraded":"2024-09-16T00:08:36.889686+0000","last_fullsized":"2024-09-16T00:08:36.889686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.894305+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:08:35.894305+0000","last_peered":"2024-09-16T00:08:35.894305+0000","last_clean":"2024-09-16T00:08:35.894305+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:08:35.894305+0000","last_undegraded":"2024-09-16T00:08:35.894305+0000","last_fullsized":"2024-09-16T00:08:35.894305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.886830+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:08:36.886830+0000","last_peered":"2024-09-16T00:08:36.886830+0000","last_clean":"2024-09-16T00:08:36.886830+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:08:36.886830+0000","last_undegraded":"2024-09-16T00:08:36.886830+0000","last_fullsized":"2024-09-16T00:08:36.886830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882462+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:08:35.882462+0000","last_peered":"2024-09-16T00:08:35.882462+0000","last_clean":"2024-09-16T00:08:35.882462+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:08:35.882462+0000","last_undegraded":"2024-09-16T00:08:35.882462+0000","last_fullsized":"2024-09-16T00:08:35.882462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.892094+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:08:35.892094+0000","last_peered":"2024-09-16T00:08:35.892094+0000","last_clean":"2024-09-16T00:08:35.892094+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:08:35.892094+0000","last_undegraded":"2024-09-16T00:08:35.892094+0000","last_fullsized":"2024-09-16T00:08:35.892094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888666+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:08:35.888666+0000","last_peered":"2024-09-16T00:08:35.888666+0000","last_clean":"2024-09-16T00:08:35.888666+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:08:35.888666+0000","last_undegraded":"2024-09-16T00:08:35.888666+0000","last_fullsized":"2024-09-16T00:08:35.888666+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.875423+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:08:34.875423+0000","last_peered":"2024-09-16T00:08:34.875423+0000","last_clean":"2024-09-16T00:08:34.875423+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:08:34.875423+0000","last_undegraded":"2024-09-16T00:08:34.875423+0000","last_fullsized":"2024-09-16T00:08:34.875423+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":63,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874580+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:08:34.874580+0000","last_peered":"2024-09-16T00:08:34.874580+0000","last_clean":"2024-09-16T00:08:34.874580+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:08:34.874580+0000","last_undegraded":"2024-09-16T00:08:34.874580+0000","last_fullsized":"2024-09-16T00:08:34.874580+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.894891+0000","last_change":"2024-09-16T00:08:36.894101+0000","last_active":"2024-09-16T00:08:36.894891+0000","last_peered":"2024-09-16T00:08:36.894891+0000","last_clean":"2024-09-16T00:08:36.894891+0000","last_became_active":"2024-09-16T00:08:36.893807+0000","last_became_peered":"2024-09-16T00:08:36.893807+0000","last_unstale":"2024-09-16T00:08:36.894891+0000","last_undegraded":"2024-09-16T00:08:36.894891+0000","last_fullsized":"2024-09-16T00:08:36.894891+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:40.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883196+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:08:36.883196+0000","last_peered":"2024-09-16T00:08:36.883196+0000","last_clean":"2024-09-16T00:08:36.883196+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:08:36.883196+0000","last_undegraded":"2024-09-16T00:08:36.883196+0000","last_fullsized":"2024-09-16T00:08:36.883196+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":63,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.893461+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:08:35.893461+0000","last_peered":"2024-09-16T00:08:35.893461+0000","last_clean":"2024-09-16T00:08:35.893461+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:08:35.893461+0000","last_undegraded":"2024-09-16T00:08:35.893461+0000","last_fullsized":"2024-09-16T00:08:35.893461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888853+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:08:35.888853+0000","last_peered":"2024-09-16T00:08:35.888853+0000","last_clean":"2024-09-16T00:08:35.888853+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:08:35.888853+0000","last_undegraded":"2024-09-16T00:08:35.888853+0000","last_fullsized":"2024-09-16T00:08:35.888853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.892709+0000","last_change":"2024-09-16T00:08:36.890316+0000","last_active":"2024-09-16T00:08:36.892709+0000","last_peered":"2024-09-16T00:08:36.892709+0000","last_clean":"2024-09-16T00:08:36.892709+0000","last_became_active":"2024-09-16T00:08:36.889635+0000","last_became_peered":"2024-09-16T00:08:36.889635+0000","last_unstale":"2024-09-16T00:08:36.892709+0000","last_undegraded":"2024-09-16T00:08:36.892709+0000","last_fullsized":"2024-09-16T00:08:36.892709+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:28.843883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"peering","last_fresh":"2024-09-16T00:08:35.189391+0000","last_change":"2024-09-16T00:08:35.890927+0000","last_active":"2024-09-16T00:08:35.890927+0000","last_peered":"2024-09-16T00:08:35.189391+0000","last_clean":"2024-09-16T00:08:35.189391+0000","last_became_active":"2024-09-16T00:08:24.838130+0000","last_became_peered":"2024-09-16T00:08:24.838130+0000","last_unstale":"2024-09-16T00:08:35.189391+0000","last_undegraded":"2024-09-16T00:08:35.189391+0000","last_fullsized":"2024-09-16T00:08:35.189391+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.892368+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:08:35.892368+0000","last_peered":"2024-09-16T00:08:35.892368+0000","last_clean":"2024-09-16T00:08:35.892368+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:08:35.892368+0000","last_undegraded":"2024-09-16T00:08:35.892368+0000","last_fullsized":"2024-09-16T00:08:35.892368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889773+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:08:36.889773+0000","last_peered":"2024-09-16T00:08:36.889773+0000","last_clean":"2024-09-16T00:08:36.889773+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:08:36.889773+0000","last_undegraded":"2024-09-16T00:08:36.889773+0000","last_fullsized":"2024-09-16T00:08:36.889773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881638+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:08:35.881638+0000","last_peered":"2024-09-16T00:08:35.881638+0000","last_clean":"2024-09-16T00:08:35.881638+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:08:35.881638+0000","last_undegraded":"2024-09-16T00:08:35.881638+0000","last_fullsized":"2024-09-16T00:08:35.881638+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.886560+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:08:36.886560+0000","last_peered":"2024-09-16T00:08:36.886560+0000","last_clean":"2024-09-16T00:08:36.886560+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:08:36.886560+0000","last_undegraded":"2024-09-16T00:08:36.886560+0000","last_fullsized":"2024-09-16T00:08:36.886560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889167+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:08:36.889167+0000","last_peered":"2024-09-16T00:08:36.889167+0000","last_clean":"2024-09-16T00:08:36.889167+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:08:36.889167+0000","last_undegraded":"2024-09-16T00:08:36.889167+0000","last_fullsized":"2024-09-16T00:08:36.889167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889247+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:08:36.889247+0000","last_peered":"2024-09-16T00:08:36.889247+0000","last_clean":"2024-09-16T00:08:36.889247+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:08:36.889247+0000","last_undegraded":"2024-09-16T00:08:36.889247+0000","last_fullsized":"2024-09-16T00:08:36.889247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.891379+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:08:35.891379+0000","last_peered":"2024-09-16T00:08:35.891379+0000","last_clean":"2024-09-16T00:08:35.891379+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:08:35.891379+0000","last_undegraded":"2024-09-16T00:08:35.891379+0000","last_fullsized":"2024-09-16T00:08:35.891379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":64,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882552+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:08:35.882552+0000","last_peered":"2024-09-16T00:08:35.882552+0000","last_clean":"2024-09-16T00:08:35.882552+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:08:35.882552+0000","last_undegraded":"2024-09-16T00:08:35.882552+0000","last_fullsized":"2024-09-16T00:08:35.882552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884405+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:08:36.884405+0000","last_peered":"2024-09-16T00:08:36.884405+0000","last_clean":"2024-09-16T00:08:36.884405+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:08:36.884405+0000","last_undegraded":"2024-09-16T00:08:36.884405+0000","last_fullsized":"2024-09-16T00:08:36.884405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"28'10","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.916626+0000","last_change":"2024-09-16T00:08:36.914308+0000","last_active":"2024-09-16T00:08:36.916626+0000","last_peered":"2024-09-16T00:08:36.916626+0000","last_clean":"2024-09-16T00:08:36.916626+0000","last_became_active":"2024-09-16T00:08:36.890227+0000","last_became_peered":"2024-09-16T00:08:36.890227+0000","last_unstale":"2024-09-16T00:08:36.916626+0000","last_undegraded":"2024-09-16T00:08:36.916626+0000","last_fullsized":"2024-09-16T00:08:36.916626+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:31.761195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6555,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13110,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884049+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:08:36.884049+0000","last_peered":"2024-09-16T00:08:36.884049+0000","last_clean":"2024-09-16T00:08:36.884049+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:08:36.884049+0000","last_undegraded":"2024-09-16T00:08:36.884049+0000","last_fullsized":"2024-09-16T00:08:36.884049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":63,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888938+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:08:35.888938+0000","last_peered":"2024-09-16T00:08:35.888938+0000","last_clean":"2024-09-16T00:08:35.888938+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:08:35.888938+0000","last_undegraded":"2024-09-16T00:08:35.888938+0000","last_fullsized":"2024-09-16T00:08:35.888938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.888319+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:08:35.888319+0000","last_peered":"2024-09-16T00:08:35.888319+0000","last_clean":"2024-09-16T00:08:35.888319+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:08:35.888319+0000","last_undegraded":"2024-09-16T00:08:35.888319+0000","last_fullsized":"2024-09-16T00:08:35.888319+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888369+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:08:36.888369+0000","last_peered":"2024-09-16T00:08:36.888369+0000","last_clean":"2024-09-16T00:08:36.888369+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:08:36.888369+0000","last_undegraded":"2024-09-16T00:08:36.888369+0000","last_fullsized":"2024-09-16T00:08:36.888369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882721+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:08:36.882721+0000","last_peered":"2024-09-16T00:08:36.882721+0000","last_clean":"2024-09-16T00:08:36.882721+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:08:36.882721+0000","last_undegraded":"2024-09-16T00:08:36.882721+0000","last_fullsized":"2024-09-16T00:08:36.882721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.891475+0000","last_change":"2024-09-16T00:08:36.889720+0000","last_active":"2024-09-16T00:08:36.891475+0000","last_peered":"2024-09-16T00:08:36.891475+0000","last_clean":"2024-09-16T00:08:36.891475+0000","last_became_active":"2024-09-16T00:08:36.889345+0000","last_became_peered":"2024-09-16T00:08:36.889345+0000","last_unstale":"2024-09-16T00:08:36.891475+0000","last_undegraded":"2024-09-16T00:08:36.891475+0000","last_fullsized":"2024-09-16T00:08:36.891475+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:03.170073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.900265+0000","last_change":"2024-09-16T00:08:36.896787+0000","last_active":"2024-09-16T00:08:36.900265+0000","last_peered":"2024-09-16T00:08:36.900265+0000","last_clean":"2024-09-16T00:08:36.900265+0000","last_became_active":"2024-09-16T00:08:36.896505+0000","last_became_peered":"2024-09-16T00:08:36.896505+0000","last_unstale":"2024-09-16T00:08:36.900265+0000","last_undegraded":"2024-09-16T00:08:36.900265+0000","last_fullsized":"2024-09-16T00:08:36.900265+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:55.997551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":68,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874425+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:08:34.874425+0000","last_peered":"2024-09-16T00:08:34.874425+0000","last_clean":"2024-09-16T00:08:34.874425+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:08:34.874425+0000","last_undegraded":"2024-09-16T00:08:34.874425+0000","last_fullsized":"2024-09-16T00:08:34.874425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882734+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:08:36.882734+0000","last_peered":"2024-09-16T00:08:36.882734+0000","last_clean":"2024-09-16T00:08:36.882734+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:08:36.882734+0000","last_undegraded":"2024-09-16T00:08:36.882734+0000","last_fullsized":"2024-09-16T00:08:36.882734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883994+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:08:36.883994+0000","last_peered":"2024-09-16T00:08:36.883994+0000","last_clean":"2024-09-16T00:08:36.883994+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:08:36.883994+0000","last_undegraded":"2024-09-16T00:08:36.883994+0000","last_fullsized":"2024-09-16T00:08:36.883994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883532+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:08:36.883532+0000","last_peered":"2024-09-16T00:08:36.883532+0000","last_clean":"2024-09-16T00:08:36.883532+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:08:36.883532+0000","last_undegraded":"2024-09-16T00:08:36.883532+0000","last_fullsized":"2024-09-16T00:08:36.883532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":62,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874882+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:08:34.874882+0000","last_peered":"2024-09-16T00:08:34.874882+0000","last_clean":"2024-09-16T00:08:34.874882+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:08:34.874882+0000","last_undegraded":"2024-09-16T00:08:34.874882+0000","last_fullsized":"2024-09-16T00:08:34.874882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883810+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:08:36.883810+0000","last_peered":"2024-09-16T00:08:36.883810+0000","last_clean":"2024-09-16T00:08:36.883810+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:08:36.883810+0000","last_undegraded":"2024-09-16T00:08:36.883810+0000","last_fullsized":"2024-09-16T00:08:36.883810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":62,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882644+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:08:35.882644+0000","last_peered":"2024-09-16T00:08:35.882644+0000","last_clean":"2024-09-16T00:08:35.882644+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:08:35.882644+0000","last_undegraded":"2024-09-16T00:08:35.882644+0000","last_fullsized":"2024-09-16T00:08:35.882644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888593+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:08:36.888593+0000","last_peered":"2024-09-16T00:08:36.888593+0000","last_clean":"2024-09-16T00:08:36.888593+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:08:36.888593+0000","last_undegraded":"2024-09-16T00:08:36.888593+0000","last_fullsized":"2024-09-16T00:08:36.888593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.875230+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:08:34.875230+0000","last_peered":"2024-09-16T00:08:34.875230+0000","last_clean":"2024-09-16T00:08:34.875230+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:08:34.875230+0000","last_undegraded":"2024-09-16T00:08:34.875230+0000","last_fullsized":"2024-09-16T00:08:34.875230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884151+0000","last_change":"2024-09-16T00:08:15.795467+0000","last_active":"2024-09-16T00:08:36.884151+0000","last_peered":"2024-09-16T00:08:36.884151+0000","last_clean":"2024-09-16T00:08:36.884151+0000","last_became_active":"2024-09-16T00:08:15.795167+0000","last_became_peered":"2024-09-16T00:08:15.795167+0000","last_unstale":"2024-09-16T00:08:36.884151+0000","last_undegraded":"2024-09-16T00:08:36.884151+0000","last_fullsized":"2024-09-16T00:08:36.884151+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":219,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882705+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:08:36.882705+0000","last_peered":"2024-09-16T00:08:36.882705+0000","last_clean":"2024-09-16T00:08:36.882705+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:08:36.882705+0000","last_undegraded":"2024-09-16T00:08:36.882705+0000","last_fullsized":"2024-09-16T00:08:36.882705+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888542+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:08:36.888542+0000","last_peered":"2024-09-16T00:08:36.888542+0000","last_clean":"2024-09-16T00:08:36.888542+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:08:36.888542+0000","last_undegraded":"2024-09-16T00:08:36.888542+0000","last_fullsized":"2024-09-16T00:08:36.888542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.893654+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:08:35.893654+0000","last_peered":"2024-09-16T00:08:35.893654+0000","last_clean":"2024-09-16T00:08:35.893654+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:08:35.893654+0000","last_undegraded":"2024-09-16T00:08:35.893654+0000","last_fullsized":"2024-09-16T00:08:35.893654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882625+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:08:36.882625+0000","last_peered":"2024-09-16T00:08:36.882625+0000","last_clean":"2024-09-16T00:08:36.882625+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:08:36.882625+0000","last_undegraded":"2024-09-16T00:08:36.882625+0000","last_fullsized":"2024-09-16T00:08:36.882625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.887981+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:08:35.887981+0000","last_peered":"2024-09-16T00:08:35.887981+0000","last_clean":"2024-09-16T00:08:35.887981+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:08:35.887981+0000","last_undegraded":"2024-09-16T00:08:35.887981+0000","last_fullsized":"2024-09-16T00:08:35.887981+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874658+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:08:34.874658+0000","last_peered":"2024-09-16T00:08:34.874658+0000","last_clean":"2024-09-16T00:08:34.874658+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:08:34.874658+0000","last_undegraded":"2024-09-16T00:08:34.874658+0000","last_fullsized":"2024-09-16T00:08:34.874658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881158+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:08:35.881158+0000","last_peered":"2024-09-16T00:08:35.881158+0000","last_clean":"2024-09-16T00:08:35.881158+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:08:35.881158+0000","last_undegraded":"2024-09-16T00:08:35.881158+0000","last_fullsized":"2024-09-16T00:08:35.881158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888456+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:08:36.888456+0000","last_peered":"2024-09-16T00:08:36.888456+0000","last_clean":"2024-09-16T00:08:36.888456+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:08:36.888456+0000","last_undegraded":"2024-09-16T00:08:36.888456+0000","last_fullsized":"2024-09-16T00:08:36.888456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883562+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:08:36.883562+0000","last_peered":"2024-09-16T00:08:36.883562+0000","last_clean":"2024-09-16T00:08:36.883562+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:08:36.883562+0000","last_undegraded":"2024-09-16T00:08:36.883562+0000","last_fullsized":"2024-09-16T00:08:36.883562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":61,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882294+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:08:35.882294+0000","last_peered":"2024-09-16T00:08:35.882294+0000","last_clean":"2024-09-16T00:08:35.882294+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:08:35.882294+0000","last_undegraded":"2024-09-16T00:08:35.882294+0000","last_fullsized":"2024-09-16T00:08:35.882294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889077+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:08:36.889077+0000","last_peered":"2024-09-16T00:08:36.889077+0000","last_clean":"2024-09-16T00:08:36.889077+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:08:36.889077+0000","last_undegraded":"2024-09-16T00:08:36.889077+0000","last_fullsized":"2024-09-16T00:08:36.889077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884353+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:08:36.884353+0000","last_peered":"2024-09-16T00:08:36.884353+0000","last_clean":"2024-09-16T00:08:36.884353+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:08:36.884353+0000","last_undegraded":"2024-09-16T00:08:36.884353+0000","last_fullsized":"2024-09-16T00:08:36.884353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.882179+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:08:35.882179+0000","last_peered":"2024-09-16T00:08:35.882179+0000","last_clean":"2024-09-16T00:08:35.882179+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:08:35.882179+0000","last_undegraded":"2024-09-16T00:08:35.882179+0000","last_fullsized":"2024-09-16T00:08:35.882179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"26'3","reported_seq":72,"reported_epoch":45,"state":"peering","last_fresh":"2024-09-16T00:08:35.191937+0000","last_change":"2024-09-16T00:08:35.883363+0000","last_active":"2024-09-16T00:08:35.883363+0000","last_peered":"2024-09-16T00:08:35.191937+0000","last_clean":"2024-09-16T00:08:35.191937+0000","last_became_active":"2024-09-16T00:08:24.835764+0000","last_became_peered":"2024-09-16T00:08:24.835764+0000","last_unstale":"2024-09-16T00:08:35.191937+0000","last_undegraded":"2024-09-16T00:08:35.191937+0000","last_fullsized":"2024-09-16T00:08:35.191937+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.894399+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:08:35.894399+0000","last_peered":"2024-09-16T00:08:35.894399+0000","last_clean":"2024-09-16T00:08:35.894399+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:08:35.894399+0000","last_undegraded":"2024-09-16T00:08:35.894399+0000","last_fullsized":"2024-09-16T00:08:35.894399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.886688+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:08:36.886688+0000","last_peered":"2024-09-16T00:08:36.886688+0000","last_clean":"2024-09-16T00:08:36.886688+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:08:36.886688+0000","last_undegraded":"2024-09-16T00:08:36.886688+0000","last_fullsized":"2024-09-16T00:08:36.886688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":59,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.890491+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:08:35.890491+0000","last_peered":"2024-09-16T00:08:35.890491+0000","last_clean":"2024-09-16T00:08:35.890491+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:08:35.890491+0000","last_undegraded":"2024-09-16T00:08:35.890491+0000","last_fullsized":"2024-09-16T00:08:35.890491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882517+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:08:36.882517+0000","last_peered":"2024-09-16T00:08:36.882517+0000","last_clean":"2024-09-16T00:08:36.882517+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:08:36.882517+0000","last_undegraded":"2024-09-16T00:08:36.882517+0000","last_fullsized":"2024-09-16T00:08:36.882517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884343+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:08:36.884343+0000","last_peered":"2024-09-16T00:08:36.884343+0000","last_clean":"2024-09-16T00:08:36.884343+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:08:36.884343+0000","last_undegraded":"2024-09-16T00:08:36.884343+0000","last_fullsized":"2024-09-16T00:08:36.884343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":61,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881815+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:08:35.881815+0000","last_peered":"2024-09-16T00:08:35.881815+0000","last_clean":"2024-09-16T00:08:35.881815+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:08:35.881815+0000","last_undegraded":"2024-09-16T00:08:35.881815+0000","last_fullsized":"2024-09-16T00:08:35.881815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.892453+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:08:35.892453+0000","last_peered":"2024-09-16T00:08:35.892453+0000","last_clean":"2024-09-16T00:08:35.892453+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:08:35.892453+0000","last_undegraded":"2024-09-16T00:08:35.892453+0000","last_fullsized":"2024-09-16T00:08:35.892453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":61,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.881852+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:08:35.881852+0000","last_peered":"2024-09-16T00:08:35.881852+0000","last_clean":"2024-09-16T00:08:35.881852+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:08:35.881852+0000","last_undegraded":"2024-09-16T00:08:35.881852+0000","last_fullsized":"2024-09-16T00:08:35.881852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":60,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.893352+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:08:35.893352+0000","last_peered":"2024-09-16T00:08:35.893352+0000","last_clean":"2024-09-16T00:08:35.893352+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:08:35.893352+0000","last_undegraded":"2024-09-16T00:08:35.893352+0000","last_fullsized":"2024-09-16T00:08:35.893352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882815+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:08:36.882815+0000","last_peered":"2024-09-16T00:08:36.882815+0000","last_clean":"2024-09-16T00:08:36.882815+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:08:36.882815+0000","last_undegraded":"2024-09-16T00:08:36.882815+0000","last_fullsized":"2024-09-16T00:08:36.882815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.892834+0000","last_change":"2024-09-16T00:08:36.890347+0000","last_active":"2024-09-16T00:08:36.892834+0000","last_peered":"2024-09-16T00:08:36.892834+0000","last_clean":"2024-09-16T00:08:36.892834+0000","last_became_active":"2024-09-16T00:08:36.890009+0000","last_became_peered":"2024-09-16T00:08:36.890009+0000","last_unstale":"2024-09-16T00:08:36.892834+0000","last_undegraded":"2024-09-16T00:08:36.892834+0000","last_fullsized":"2024-09-16T00:08:36.892834+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:24.298978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888991+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:08:36.888991+0000","last_peered":"2024-09-16T00:08:36.888991+0000","last_clean":"2024-09-16T00:08:36.888991+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:08:36.888991+0000","last_undegraded":"2024-09-16T00:08:36.888991+0000","last_fullsized":"2024-09-16T00:08:36.888991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883181+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:08:36.883181+0000","last_peered":"2024-09-16T00:08:36.883181+0000","last_clean":"2024-09-16T00:08:36.883181+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:08:36.883181+0000","last_undegraded":"2024-09-16T00:08:36.883181+0000","last_fullsized":"2024-09-16T00:08:36.883181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888518+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:08:36.888518+0000","last_peered":"2024-09-16T00:08:36.888518+0000","last_clean":"2024-09-16T00:08:36.888518+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:08:36.888518+0000","last_undegraded":"2024-09-16T00:08:36.888518+0000","last_fullsized":"2024-09-16T00:08:36.888518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.900704+0000","last_change":"2024-09-16T00:08:36.900704+0000","last_active":"2024-09-16T00:08:36.900704+0000","last_peered":"2024-09-16T00:08:36.900704+0000","last_clean":"2024-09-16T00:08:36.900704+0000","last_became_active":"2024-09-16T00:08:36.892520+0000","last_became_peered":"2024-09-16T00:08:36.892520+0000","last_unstale":"2024-09-16T00:08:36.900704+0000","last_undegraded":"2024-09-16T00:08:36.900704+0000","last_fullsized":"2024-09-16T00:08:36.900704+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:30.056041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-16T00:08:34.874986+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:08:34.874986+0000","last_peered":"2024-09-16T00:08:34.874986+0000","last_clean":"2024-09-16T00:08:34.874986+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:08:34.874986+0000","last_undegraded":"2024-09-16T00:08:34.874986+0000","last_fullsized":"2024-09-16T00:08:34.874986+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":63,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.895206+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:08:35.895206+0000","last_peered":"2024-09-16T00:08:35.895206+0000","last_clean":"2024-09-16T00:08:35.895206+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:08:35.895206+0000","last_undegraded":"2024-09-16T00:08:35.895206+0000","last_fullsized":"2024-09-16T00:08:35.895206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":64,"reported_epoch":45,"state":"peering","last_fresh":"2024-09-16T00:08:35.190279+0000","last_change":"2024-09-16T00:08:35.895010+0000","last_active":"2024-09-16T00:08:35.895010+0000","last_peered":"2024-09-16T00:08:35.190279+0000","last_clean":"2024-09-16T00:08:35.190279+0000","last_became_active":"2024-09-16T00:08:24.838285+0000","last_became_peered":"2024-09-16T00:08:24.838285+0000","last_unstale":"2024-09-16T00:08:35.190279+0000","last_undegraded":"2024-09-16T00:08:35.190279+0000","last_fullsized":"2024-09-16T00:08:35.190279+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":65,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-16T00:08:35.891464+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:08:35.891464+0000","last_peered":"2024-09-16T00:08:35.891464+0000","last_clean":"2024-09-16T00:08:35.891464+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:08:35.891464+0000","last_undegraded":"2024-09-16T00:08:35.891464+0000","last_fullsized":"2024-09-16T00:08:35.891464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7353,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":13110,"num_keys_recovered":2,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":14706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":585728,"data_stored":815033,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":540672,"data_stored":236719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":230728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":540672,"data_stored":230380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":520192,"data_stored":224158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":217763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":581632,"data_stored":808609,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":491520,"data_stored":211480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":6555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":6555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:08:41.633 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:08:41.638 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:08:41.722 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:41.721+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:41.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.769+0000 7fd483333640 1 Processor -- start 2024-09-16T00:08:41.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.769+0000 7fd483333640 1 -- start start 2024-09-16T00:08:41.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd483333640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd47c0c67e0 0x7fd47c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd483333640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd47c06c110 0x7fd47c15a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd483333640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd47c06c530 0x7fd47c15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd483333640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd47c06cd10 con 0x7fd47c06c530 2024-09-16T00:08:41.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd483333640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fd47c0c6bc0 con 0x7fd47c06c110 2024-09-16T00:08:41.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd483333640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd47c15e850 con 0x7fd47c0c67e0 2024-09-16T00:08:41.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd4808ce640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd47c06c110 0x7fd47c15a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd4810cf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd47c0c67e0 0x7fd47c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd4810cf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd47c0c67e0 0x7fd47c0c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:48760/0 (socket says 172.21.15.139:48760) 2024-09-16T00:08:41.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd4808ce640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd47c06c110 0x7fd47c15a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:49516/0 (socket says 172.21.15.139:49516) 2024-09-16T00:08:41.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd4810cf640 1 -- 172.21.15.139:0/3970087129 learned_addr learned my addr 172.21.15.139:0/3970087129 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:41.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd4818d0640 1 --2- 172.21.15.139:0/3970087129 >> v2:172.21.15.139:3300/0 conn(0x7fd47c06c530 0x7fd47c15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd4808ce640 1 -- 172.21.15.139:0/3970087129 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd47c0c67e0 msgr2=0x7fd47c0c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:41.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd4808ce640 1 --2- 172.21.15.139:0/3970087129 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd47c0c67e0 0x7fd47c0c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd4808ce640 1 -- 172.21.15.139:0/3970087129 >> v2:172.21.15.139:3300/0 conn(0x7fd47c06c530 msgr2=0x7fd47c15ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:41.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd4808ce640 1 --2- 172.21.15.139:0/3970087129 >> v2:172.21.15.139:3300/0 conn(0x7fd47c06c530 0x7fd47c15ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd4808ce640 1 -- 172.21.15.139:0/3970087129 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47c06ca30 con 0x7fd47c06c110 2024-09-16T00:08:41.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd4818d0640 1 --2- 172.21.15.139:0/3970087129 >> v2:172.21.15.139:3300/0 conn(0x7fd47c06c530 0x7fd47c15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:41.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.770+0000 7fd4810cf640 1 --2- 172.21.15.139:0/3970087129 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd47c0c67e0 0x7fd47c0c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:41.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.771+0000 7fd4808ce640 1 --2- 172.21.15.139:0/3970087129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd47c06c110 0x7fd47c15a810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd47c08d500 tx=0x7fd46c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2980158e8a4b9b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:41.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.772+0000 7fd473fff640 1 -- 172.21.15.139:0/3970087129 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c037070 con 0x7fd47c06c110 2024-09-16T00:08:41.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.772+0000 7fd473fff640 1 -- 172.21.15.139:0/3970087129 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd46c032440 con 0x7fd47c06c110 2024-09-16T00:08:41.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.772+0000 7fd473fff640 1 -- 172.21.15.139:0/3970087129 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c032810 con 0x7fd47c06c110 2024-09-16T00:08:41.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.772+0000 7fd483333640 1 -- 172.21.15.139:0/3970087129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd47c06c110 msgr2=0x7fd47c15a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.772+0000 7fd483333640 1 --2- 172.21.15.139:0/3970087129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd47c06c110 0x7fd47c15a810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd47c08d500 tx=0x7fd46c02a1a0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.772+0000 7fd483333640 1 -- 172.21.15.139:0/3970087129 shutdown_connections 2024-09-16T00:08:41.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.772+0000 7fd483333640 1 --2- 172.21.15.139:0/3970087129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd47c06c110 0x7fd47c15a810 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.773+0000 7fd483333640 1 --2- 172.21.15.139:0/3970087129 >> v2:172.21.15.139:3300/0 conn(0x7fd47c06c530 0x7fd47c15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.773+0000 7fd483333640 1 --2- 172.21.15.139:0/3970087129 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd47c0c67e0 0x7fd47c0c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.773+0000 7fd483333640 1 -- 172.21.15.139:0/3970087129 >> 172.21.15.139:0/3970087129 conn(0x7fd47c0d05f0 msgr2=0x7fd47c0d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:41.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.773+0000 7fd483333640 1 -- 172.21.15.139:0/3970087129 shutdown_connections 2024-09-16T00:08:41.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.773+0000 7fd483333640 1 -- 172.21.15.139:0/3970087129 wait complete. 2024-09-16T00:08:41.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.774+0000 7fd483333640 1 Processor -- start 2024-09-16T00:08:41.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.774+0000 7fd483333640 1 -- start start 2024-09-16T00:08:41.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.776+0000 7fd483333640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd47c06c110 0x7fd47c154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.776+0000 7fd483333640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd47c06c530 0x7fd47c153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.776+0000 7fd483333640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd47c0c67e0 0x7fd47c156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.776+0000 7fd483333640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd47c15e4e0 con 0x7fd47c06c530 2024-09-16T00:08:41.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.776+0000 7fd4808ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd47c06c530 0x7fd47c153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.776+0000 7fd4808ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd47c06c530 0x7fd47c153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49352/0 (socket says 172.21.15.139:49352) 2024-09-16T00:08:41.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.776+0000 7fd4808ce640 1 -- 172.21.15.139:0/4115265037 learned_addr learned my addr 172.21.15.139:0/4115265037 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:41.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.776+0000 7fd483333640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd47c15e200 con 0x7fd47c06c110 2024-09-16T00:08:41.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.776+0000 7fd483333640 1 -- 172.21.15.139:0/4115265037 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd47c0cbe50 con 0x7fd47c0c67e0 2024-09-16T00:08:41.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.776+0000 7fd4810cf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd47c06c110 0x7fd47c154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.776+0000 7fd4818d0640 1 --2- 172.21.15.139:0/4115265037 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd47c0c67e0 0x7fd47c156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.776+0000 7fd4808ce640 1 -- 172.21.15.139:0/4115265037 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd47c0c67e0 msgr2=0x7fd47c156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.776+0000 7fd4808ce640 1 --2- 172.21.15.139:0/4115265037 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd47c0c67e0 0x7fd47c156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.776+0000 7fd4808ce640 1 -- 172.21.15.139:0/4115265037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd47c06c110 msgr2=0x7fd47c154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.776+0000 7fd4808ce640 1 --2- 172.21.15.139:0/4115265037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd47c06c110 0x7fd47c154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.776+0000 7fd4808ce640 1 -- 172.21.15.139:0/4115265037 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47c1f1290 con 0x7fd47c06c530 2024-09-16T00:08:41.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.777+0000 7fd4808ce640 1 --2- 172.21.15.139:0/4115265037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd47c06c530 0x7fd47c153fd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009890 tx=0x7fd46c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:41.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.777+0000 7fd471ffb640 1 -- 172.21.15.139:0/4115265037 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c037040 con 0x7fd47c06c530 2024-09-16T00:08:41.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.777+0000 7fd471ffb640 1 -- 172.21.15.139:0/4115265037 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd46c049610 con 0x7fd47c06c530 2024-09-16T00:08:41.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.777+0000 7fd483333640 1 -- 172.21.15.139:0/4115265037 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd47c15b570 con 0x7fd47c06c530 2024-09-16T00:08:41.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.777+0000 7fd471ffb640 1 -- 172.21.15.139:0/4115265037 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c049a00 con 0x7fd47c06c530 2024-09-16T00:08:41.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.777+0000 7fd483333640 1 -- 172.21.15.139:0/4115265037 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd47c155290 con 0x7fd47c06c530 2024-09-16T00:08:41.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.779+0000 7fd483333640 1 -- 172.21.15.139:0/4115265037 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd47c0cbc30 con 0x7fd47c06c530 2024-09-16T00:08:41.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.779+0000 7fd471ffb640 1 -- 172.21.15.139:0/4115265037 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd46c049ba0 con 0x7fd47c06c530 2024-09-16T00:08:41.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.780+0000 7fd471ffb640 1 --2- 172.21.15.139:0/4115265037 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd46006c2e0 0x7fd46006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:41.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.780+0000 7fd471ffb640 1 -- 172.21.15.139:0/4115265037 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5915+0+0 (secure 0 0 0) 0x7fd46c0b3ef0 con 0x7fd47c06c530 2024-09-16T00:08:41.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.780+0000 7fd4810cf640 1 --2- 172.21.15.139:0/4115265037 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd46006c2e0 0x7fd46006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:41.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.781+0000 7fd4810cf640 1 --2- 172.21.15.139:0/4115265037 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd46006c2e0 0x7fd46006e7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd478009630 tx=0x7fd478009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:41.783 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:41.783 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:41.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.784+0000 7fd471ffb640 1 -- 172.21.15.139:0/4115265037 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd46c044070 con 0x7fd47c06c530 2024-09-16T00:08:41.792 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:08:41.806 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:41.804+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:41.907 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:41.906+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:41.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.917+0000 7f8f167fc640 1 -- 172.21.15.139:0/3178549179 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v49) ==== 87+0+0 (secure 0 0 0) 0x7f8f1c05cb90 con 0x7f8f20068060 2024-09-16T00:08:41.919 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is set 2024-09-16T00:08:41.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.921+0000 7f8f27d0a640 1 -- 172.21.15.139:0/3178549179 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f0406c2b0 msgr2=0x7f8f0406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.921+0000 7f8f27d0a640 1 --2- 172.21.15.139:0/3178549179 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f0406c2b0 0x7f8f0406e770 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8f10034000 tx=0x7f8f1002e000 comp rx=0 tx=0).stop 2024-09-16T00:08:41.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.921+0000 7f8f27d0a640 1 -- 172.21.15.139:0/3178549179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f20068060 msgr2=0x7f8f2015cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.921+0000 7f8f27d0a640 1 --2- 172.21.15.139:0/3178549179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f20068060 0x7f8f2015cd60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00c430 tx=0x7f8f1c00c8f0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.921+0000 7f8f27d0a640 1 -- 172.21.15.139:0/3178549179 shutdown_connections 2024-09-16T00:08:41.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.921+0000 7f8f27d0a640 1 --2- 172.21.15.139:0/3178549179 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f0406c2b0 0x7f8f0406e770 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.921+0000 7f8f27d0a640 1 --2- 172.21.15.139:0/3178549179 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f20154c20 0x7f8f200b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.921+0000 7f8f27d0a640 1 --2- 172.21.15.139:0/3178549179 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f200ccc90 0x7f8f2015d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.921+0000 7f8f27d0a640 1 --2- 172.21.15.139:0/3178549179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f20068060 0x7f8f2015cd60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.921+0000 7f8f27d0a640 1 -- 172.21.15.139:0/3178549179 >> 172.21.15.139:0/3178549179 conn(0x7f8f2006ca40 msgr2=0x7f8f20067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.922+0000 7f8f27d0a640 1 -- 172.21.15.139:0/3178549179 shutdown_connections 2024-09-16T00:08:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.922+0000 7f8f27d0a640 1 -- 172.21.15.139:0/3178549179 wait complete. 2024-09-16T00:08:41.934 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:08:41.939 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:41.940 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:41.949 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:08:41.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.974+0000 7fd483333640 1 -- 172.21.15.139:0/4115265037 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd47c06db10 con 0x7fd46006c2e0 2024-09-16T00:08:41.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.986+0000 7fd471ffb640 1 -- 172.21.15.139:0/4115265037 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252032 (secure 0 0 0) 0x7fd47c06db10 con 0x7fd46006c2e0 2024-09-16T00:08:41.990 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:08:41.992 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:08:41.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.990+0000 7fd483333640 1 -- 172.21.15.139:0/4115265037 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd46006c2e0 msgr2=0x7fd46006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.990+0000 7fd483333640 1 --2- 172.21.15.139:0/4115265037 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd46006c2e0 0x7fd46006e7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd478009630 tx=0x7fd478009240 comp rx=0 tx=0).stop 2024-09-16T00:08:41.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.991+0000 7fd483333640 1 -- 172.21.15.139:0/4115265037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd47c06c530 msgr2=0x7fd47c153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:41.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.991+0000 7fd483333640 1 --2- 172.21.15.139:0/4115265037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd47c06c530 0x7fd47c153fd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009890 tx=0x7fd46c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.991+0000 7fd483333640 1 -- 172.21.15.139:0/4115265037 shutdown_connections 2024-09-16T00:08:41.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.991+0000 7fd483333640 1 --2- 172.21.15.139:0/4115265037 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd46006c2e0 0x7fd46006e7a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.991+0000 7fd483333640 1 --2- 172.21.15.139:0/4115265037 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd47c0c67e0 0x7fd47c156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.991+0000 7fd483333640 1 --2- 172.21.15.139:0/4115265037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd47c06c530 0x7fd47c153fd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.991+0000 7fd483333640 1 --2- 172.21.15.139:0/4115265037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd47c06c110 0x7fd47c154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:41.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.991+0000 7fd483333640 1 -- 172.21.15.139:0/4115265037 >> 172.21.15.139:0/4115265037 conn(0x7fd47c0d05f0 msgr2=0x7fd47c0c3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:41.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.991+0000 7fd483333640 1 -- 172.21.15.139:0/4115265037 shutdown_connections 2024-09-16T00:08:41.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:41.991+0000 7fd483333640 1 -- 172.21.15.139:0/4115265037 wait complete. 2024-09-16T00:08:42.008 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-16T00:08:41.919661+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597721,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":212,"num_write_kb":2439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":13110,"num_keys_recovered":4,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220596,"kb_used_data":4340,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754124,"statfs":{"total":773094113280,"available":772868222976,"internally_reserved":0,"allocated":4444160,"data_stored":3043827,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":6555,"num_keys_recovered":2,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.040564"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888570+0000","last_change":"2024-09-16T00:08:36.888570+0000","last_active":"2024-09-16T00:08:36.888570+0000","last_peered":"2024-09-16T00:08:36.888570+0000","last_clean":"2024-09-16T00:08:36.888570+0000","last_became_active":"2024-09-16T00:08:36.888143+0000","last_became_peered":"2024-09-16T00:08:36.888143+0000","last_unstale":"2024-09-16T00:08:36.888570+0000","last_undegraded":"2024-09-16T00:08:36.888570+0000","last_fullsized":"2024-09-16T00:08:36.888570+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:30.579681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913558+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:08:38.913558+0000","last_peered":"2024-09-16T00:08:38.913558+0000","last_clean":"2024-09-16T00:08:38.913558+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:08:38.913558+0000","last_undegraded":"2024-09-16T00:08:38.913558+0000","last_fullsized":"2024-09-16T00:08:38.913558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504598+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:08:40.504598+0000","last_peered":"2024-09-16T00:08:40.504598+0000","last_clean":"2024-09-16T00:08:40.504598+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:08:40.504598+0000","last_undegraded":"2024-09-16T00:08:40.504598+0000","last_fullsized":"2024-09-16T00:08:40.504598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888438+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:08:36.888438+0000","last_peered":"2024-09-16T00:08:36.888438+0000","last_clean":"2024-09-16T00:08:36.888438+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:08:36.888438+0000","last_undegraded":"2024-09-16T00:08:36.888438+0000","last_fullsized":"2024-09-16T00:08:36.888438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913518+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:08:38.913518+0000","last_peered":"2024-09-16T00:08:38.913518+0000","last_clean":"2024-09-16T00:08:38.913518+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:08:38.913518+0000","last_undegraded":"2024-09-16T00:08:38.913518+0000","last_fullsized":"2024-09-16T00:08:38.913518+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888197+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:08:36.888197+0000","last_peered":"2024-09-16T00:08:36.888197+0000","last_clean":"2024-09-16T00:08:36.888197+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:08:36.888197+0000","last_undegraded":"2024-09-16T00:08:36.888197+0000","last_fullsized":"2024-09-16T00:08:36.888197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882645+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:08:36.882645+0000","last_peered":"2024-09-16T00:08:36.882645+0000","last_clean":"2024-09-16T00:08:36.882645+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:08:36.882645+0000","last_undegraded":"2024-09-16T00:08:36.882645+0000","last_fullsized":"2024-09-16T00:08:36.882645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.786683+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:08:39.786683+0000","last_peered":"2024-09-16T00:08:39.786683+0000","last_clean":"2024-09-16T00:08:39.786683+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:08:39.786683+0000","last_undegraded":"2024-09-16T00:08:39.786683+0000","last_fullsized":"2024-09-16T00:08:39.786683+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883724+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:08:36.883724+0000","last_peered":"2024-09-16T00:08:36.883724+0000","last_clean":"2024-09-16T00:08:36.883724+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:08:36.883724+0000","last_undegraded":"2024-09-16T00:08:36.883724+0000","last_fullsized":"2024-09-16T00:08:36.883724+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.893637+0000","last_change":"2024-09-16T00:08:36.891868+0000","last_active":"2024-09-16T00:08:36.893637+0000","last_peered":"2024-09-16T00:08:36.893637+0000","last_clean":"2024-09-16T00:08:36.893637+0000","last_became_active":"2024-09-16T00:08:36.891605+0000","last_became_peered":"2024-09-16T00:08:36.891605+0000","last_unstale":"2024-09-16T00:08:36.893637+0000","last_undegraded":"2024-09-16T00:08:36.893637+0000","last_fullsized":"2024-09-16T00:08:36.893637+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:46:00.730436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.893332+0000","last_change":"2024-09-16T00:08:36.891589+0000","last_active":"2024-09-16T00:08:36.893332+0000","last_peered":"2024-09-16T00:08:36.893332+0000","last_clean":"2024-09-16T00:08:36.893332+0000","last_became_active":"2024-09-16T00:08:36.891279+0000","last_became_peered":"2024-09-16T00:08:36.891279+0000","last_unstale":"2024-09-16T00:08:36.893332+0000","last_undegraded":"2024-09-16T00:08:36.893332+0000","last_fullsized":"2024-09-16T00:08:36.893332+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888576+0000","last_change":"2024-09-16T00:08:36.888576+0000","last_active":"2024-09-16T00:08:36.888576+0000","last_peered":"2024-09-16T00:08:36.888576+0000","last_clean":"2024-09-16T00:08:36.888576+0000","last_became_active":"2024-09-16T00:08:36.888301+0000","last_became_peered":"2024-09-16T00:08:36.888301+0000","last_unstale":"2024-09-16T00:08:36.888576+0000","last_undegraded":"2024-09-16T00:08:36.888576+0000","last_fullsized":"2024-09-16T00:08:36.888576+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389414+0000","last_change":"2024-09-16T00:08:36.892595+0000","last_active":"2024-09-16T00:08:40.389414+0000","last_peered":"2024-09-16T00:08:40.389414+0000","last_clean":"2024-09-16T00:08:40.389414+0000","last_became_active":"2024-09-16T00:08:36.892316+0000","last_became_peered":"2024-09-16T00:08:36.892316+0000","last_unstale":"2024-09-16T00:08:40.389414+0000","last_undegraded":"2024-09-16T00:08:40.389414+0000","last_fullsized":"2024-09-16T00:08:40.389414+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:47.480168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504439+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:08:40.504439+0000","last_peered":"2024-09-16T00:08:40.504439+0000","last_clean":"2024-09-16T00:08:40.504439+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:08:40.504439+0000","last_undegraded":"2024-09-16T00:08:40.504439+0000","last_fullsized":"2024-09-16T00:08:40.504439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913756+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:08:38.913756+0000","last_peered":"2024-09-16T00:08:38.913756+0000","last_clean":"2024-09-16T00:08:38.913756+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:08:38.913756+0000","last_undegraded":"2024-09-16T00:08:38.913756+0000","last_fullsized":"2024-09-16T00:08:38.913756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889590+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:08:36.889590+0000","last_peered":"2024-09-16T00:08:36.889590+0000","last_clean":"2024-09-16T00:08:36.889590+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:08:36.889590+0000","last_undegraded":"2024-09-16T00:08:36.889590+0000","last_fullsized":"2024-09-16T00:08:36.889590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888829+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:08:36.888829+0000","last_peered":"2024-09-16T00:08:36.888829+0000","last_clean":"2024-09-16T00:08:36.888829+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:08:36.888829+0000","last_undegraded":"2024-09-16T00:08:36.888829+0000","last_fullsized":"2024-09-16T00:08:36.888829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.787096+0000","last_change":"2024-09-16T00:08:36.894430+0000","last_active":"2024-09-16T00:08:39.787096+0000","last_peered":"2024-09-16T00:08:39.787096+0000","last_clean":"2024-09-16T00:08:39.787096+0000","last_became_active":"2024-09-16T00:08:36.894138+0000","last_became_peered":"2024-09-16T00:08:36.894138+0000","last_unstale":"2024-09-16T00:08:39.787096+0000","last_undegraded":"2024-09-16T00:08:39.787096+0000","last_fullsized":"2024-09-16T00:08:39.787096+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:19.478353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884263+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:08:36.884263+0000","last_peered":"2024-09-16T00:08:36.884263+0000","last_clean":"2024-09-16T00:08:36.884263+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:08:36.884263+0000","last_undegraded":"2024-09-16T00:08:36.884263+0000","last_fullsized":"2024-09-16T00:08:36.884263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.892945+0000","last_change":"2024-09-16T00:08:36.890676+0000","last_active":"2024-09-16T00:08:36.892945+0000","last_peered":"2024-09-16T00:08:36.892945+0000","last_clean":"2024-09-16T00:08:36.892945+0000","last_became_active":"2024-09-16T00:08:36.889743+0000","last_became_peered":"2024-09-16T00:08:36.889743+0000","last_unstale":"2024-09-16T00:08:36.892945+0000","last_undegraded":"2024-09-16T00:08:36.892945+0000","last_fullsized":"2024-09-16T00:08:36.892945+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:02.709496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504430+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:08:40.504430+0000","last_peered":"2024-09-16T00:08:40.504430+0000","last_clean":"2024-09-16T00:08:40.504430+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:08:40.504430+0000","last_undegraded":"2024-09-16T00:08:40.504430+0000","last_fullsized":"2024-09-16T00:08:40.504430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888289+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:08:36.888289+0000","last_peered":"2024-09-16T00:08:36.888289+0000","last_clean":"2024-09-16T00:08:36.888289+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:08:36.888289+0000","last_undegraded":"2024-09-16T00:08:36.888289+0000","last_fullsized":"2024-09-16T00:08:36.888289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389236+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:08:40.389236+0000","last_peered":"2024-09-16T00:08:40.389236+0000","last_clean":"2024-09-16T00:08:40.389236+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:08:40.389236+0000","last_undegraded":"2024-09-16T00:08:40.389236+0000","last_fullsized":"2024-09-16T00:08:40.389236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889686+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:08:36.889686+0000","last_peered":"2024-09-16T00:08:36.889686+0000","last_clean":"2024-09-16T00:08:36.889686+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:08:36.889686+0000","last_undegraded":"2024-09-16T00:08:36.889686+0000","last_fullsized":"2024-09-16T00:08:36.889686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389795+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:08:40.389795+0000","last_peered":"2024-09-16T00:08:40.389795+0000","last_clean":"2024-09-16T00:08:40.389795+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:08:40.389795+0000","last_undegraded":"2024-09-16T00:08:40.389795+0000","last_fullsized":"2024-09-16T00:08:40.389795+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.886830+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:08:36.886830+0000","last_peered":"2024-09-16T00:08:36.886830+0000","last_clean":"2024-09-16T00:08:36.886830+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:08:36.886830+0000","last_undegraded":"2024-09-16T00:08:36.886830+0000","last_fullsized":"2024-09-16T00:08:36.886830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913978+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:08:38.913978+0000","last_peered":"2024-09-16T00:08:38.913978+0000","last_clean":"2024-09-16T00:08:38.913978+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:08:38.913978+0000","last_undegraded":"2024-09-16T00:08:38.913978+0000","last_fullsized":"2024-09-16T00:08:38.913978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389479+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:08:40.389479+0000","last_peered":"2024-09-16T00:08:40.389479+0000","last_clean":"2024-09-16T00:08:40.389479+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:08:40.389479+0000","last_undegraded":"2024-09-16T00:08:40.389479+0000","last_fullsized":"2024-09-16T00:08:40.389479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504452+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:08:40.504452+0000","last_peered":"2024-09-16T00:08:40.504452+0000","last_clean":"2024-09-16T00:08:40.504452+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:08:40.504452+0000","last_undegraded":"2024-09-16T00:08:40.504452+0000","last_fullsized":"2024-09-16T00:08:40.504452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.786805+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:08:39.786805+0000","last_peered":"2024-09-16T00:08:39.786805+0000","last_clean":"2024-09-16T00:08:39.786805+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:08:39.786805+0000","last_undegraded":"2024-09-16T00:08:39.786805+0000","last_fullsized":"2024-09-16T00:08:39.786805+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":69,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.786660+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:08:39.786660+0000","last_peered":"2024-09-16T00:08:39.786660+0000","last_clean":"2024-09-16T00:08:39.786660+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:08:39.786660+0000","last_undegraded":"2024-09-16T00:08:39.786660+0000","last_fullsized":"2024-09-16T00:08:39.786660+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.894891+0000","last_change":"2024-09-16T00:08:36.894101+0000","last_active":"2024-09-16T00:08:36.894891+0000","last_peered":"2024-09-16T00:08:36.894891+0000","last_clean":"2024-09-16T00:08:36.894891+0000","last_became_active":"2024-09-16T00:08:36.893807+0000","last_became_peered":"2024-09-16T00:08:36.893807+0000","last_unstale":"2024-09-16T00:08:36.894891+0000","last_undegraded":"2024-09-16T00:08:36.894891+0000","last_fullsized":"2024-09-16T00:08:36.894891+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:40.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883196+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:08:36.883196+0000","last_peered":"2024-09-16T00:08:36.883196+0000","last_clean":"2024-09-16T00:08:36.883196+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:08:36.883196+0000","last_undegraded":"2024-09-16T00:08:36.883196+0000","last_fullsized":"2024-09-16T00:08:36.883196+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":67,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389579+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:08:40.389579+0000","last_peered":"2024-09-16T00:08:40.389579+0000","last_clean":"2024-09-16T00:08:40.389579+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:08:40.389579+0000","last_undegraded":"2024-09-16T00:08:40.389579+0000","last_fullsized":"2024-09-16T00:08:40.389579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504774+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:08:40.504774+0000","last_peered":"2024-09-16T00:08:40.504774+0000","last_clean":"2024-09-16T00:08:40.504774+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:08:40.504774+0000","last_undegraded":"2024-09-16T00:08:40.504774+0000","last_fullsized":"2024-09-16T00:08:40.504774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.892709+0000","last_change":"2024-09-16T00:08:36.890316+0000","last_active":"2024-09-16T00:08:36.892709+0000","last_peered":"2024-09-16T00:08:36.892709+0000","last_clean":"2024-09-16T00:08:36.892709+0000","last_became_active":"2024-09-16T00:08:36.889635+0000","last_became_peered":"2024-09-16T00:08:36.889635+0000","last_unstale":"2024-09-16T00:08:36.892709+0000","last_undegraded":"2024-09-16T00:08:36.892709+0000","last_fullsized":"2024-09-16T00:08:36.892709+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:28.843883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":76,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389324+0000","last_change":"2024-09-16T00:08:36.888349+0000","last_active":"2024-09-16T00:08:40.389324+0000","last_peered":"2024-09-16T00:08:40.389324+0000","last_clean":"2024-09-16T00:08:40.389324+0000","last_became_active":"2024-09-16T00:08:36.888052+0000","last_became_peered":"2024-09-16T00:08:36.888052+0000","last_unstale":"2024-09-16T00:08:40.389324+0000","last_undegraded":"2024-09-16T00:08:40.389324+0000","last_fullsized":"2024-09-16T00:08:40.389324+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:49.607254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389802+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:08:40.389802+0000","last_peered":"2024-09-16T00:08:40.389802+0000","last_clean":"2024-09-16T00:08:40.389802+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:08:40.389802+0000","last_undegraded":"2024-09-16T00:08:40.389802+0000","last_fullsized":"2024-09-16T00:08:40.389802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889773+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:08:36.889773+0000","last_peered":"2024-09-16T00:08:36.889773+0000","last_clean":"2024-09-16T00:08:36.889773+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:08:36.889773+0000","last_undegraded":"2024-09-16T00:08:36.889773+0000","last_fullsized":"2024-09-16T00:08:36.889773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913558+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:08:38.913558+0000","last_peered":"2024-09-16T00:08:38.913558+0000","last_clean":"2024-09-16T00:08:38.913558+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:08:38.913558+0000","last_undegraded":"2024-09-16T00:08:38.913558+0000","last_fullsized":"2024-09-16T00:08:38.913558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.886560+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:08:36.886560+0000","last_peered":"2024-09-16T00:08:36.886560+0000","last_clean":"2024-09-16T00:08:36.886560+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:08:36.886560+0000","last_undegraded":"2024-09-16T00:08:36.886560+0000","last_fullsized":"2024-09-16T00:08:36.886560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889167+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:08:36.889167+0000","last_peered":"2024-09-16T00:08:36.889167+0000","last_clean":"2024-09-16T00:08:36.889167+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:08:36.889167+0000","last_undegraded":"2024-09-16T00:08:36.889167+0000","last_fullsized":"2024-09-16T00:08:36.889167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889247+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:08:36.889247+0000","last_peered":"2024-09-16T00:08:36.889247+0000","last_clean":"2024-09-16T00:08:36.889247+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:08:36.889247+0000","last_undegraded":"2024-09-16T00:08:36.889247+0000","last_fullsized":"2024-09-16T00:08:36.889247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504691+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:08:40.504691+0000","last_peered":"2024-09-16T00:08:40.504691+0000","last_clean":"2024-09-16T00:08:40.504691+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:08:40.504691+0000","last_undegraded":"2024-09-16T00:08:40.504691+0000","last_fullsized":"2024-09-16T00:08:40.504691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":68,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.914027+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:08:38.914027+0000","last_peered":"2024-09-16T00:08:38.914027+0000","last_clean":"2024-09-16T00:08:38.914027+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:08:38.914027+0000","last_undegraded":"2024-09-16T00:08:38.914027+0000","last_fullsized":"2024-09-16T00:08:38.914027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884405+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:08:36.884405+0000","last_peered":"2024-09-16T00:08:36.884405+0000","last_clean":"2024-09-16T00:08:36.884405+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:08:36.884405+0000","last_undegraded":"2024-09-16T00:08:36.884405+0000","last_fullsized":"2024-09-16T00:08:36.884405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"28'10","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.916626+0000","last_change":"2024-09-16T00:08:36.914308+0000","last_active":"2024-09-16T00:08:36.916626+0000","last_peered":"2024-09-16T00:08:36.916626+0000","last_clean":"2024-09-16T00:08:36.916626+0000","last_became_active":"2024-09-16T00:08:36.890227+0000","last_became_peered":"2024-09-16T00:08:36.890227+0000","last_unstale":"2024-09-16T00:08:36.916626+0000","last_undegraded":"2024-09-16T00:08:36.916626+0000","last_fullsized":"2024-09-16T00:08:36.916626+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:31.761195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6555,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13110,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884049+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:08:36.884049+0000","last_peered":"2024-09-16T00:08:36.884049+0000","last_clean":"2024-09-16T00:08:36.884049+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:08:36.884049+0000","last_undegraded":"2024-09-16T00:08:36.884049+0000","last_fullsized":"2024-09-16T00:08:36.884049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":67,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504540+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:08:40.504540+0000","last_peered":"2024-09-16T00:08:40.504540+0000","last_clean":"2024-09-16T00:08:40.504540+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:08:40.504540+0000","last_undegraded":"2024-09-16T00:08:40.504540+0000","last_fullsized":"2024-09-16T00:08:40.504540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504365+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:08:40.504365+0000","last_peered":"2024-09-16T00:08:40.504365+0000","last_clean":"2024-09-16T00:08:40.504365+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:08:40.504365+0000","last_undegraded":"2024-09-16T00:08:40.504365+0000","last_fullsized":"2024-09-16T00:08:40.504365+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888369+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:08:36.888369+0000","last_peered":"2024-09-16T00:08:36.888369+0000","last_clean":"2024-09-16T00:08:36.888369+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:08:36.888369+0000","last_undegraded":"2024-09-16T00:08:36.888369+0000","last_fullsized":"2024-09-16T00:08:36.888369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882721+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:08:36.882721+0000","last_peered":"2024-09-16T00:08:36.882721+0000","last_clean":"2024-09-16T00:08:36.882721+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:08:36.882721+0000","last_undegraded":"2024-09-16T00:08:36.882721+0000","last_fullsized":"2024-09-16T00:08:36.882721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.891475+0000","last_change":"2024-09-16T00:08:36.889720+0000","last_active":"2024-09-16T00:08:36.891475+0000","last_peered":"2024-09-16T00:08:36.891475+0000","last_clean":"2024-09-16T00:08:36.891475+0000","last_became_active":"2024-09-16T00:08:36.889345+0000","last_became_peered":"2024-09-16T00:08:36.889345+0000","last_unstale":"2024-09-16T00:08:36.891475+0000","last_undegraded":"2024-09-16T00:08:36.891475+0000","last_fullsized":"2024-09-16T00:08:36.891475+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:03.170073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.900265+0000","last_change":"2024-09-16T00:08:36.896787+0000","last_active":"2024-09-16T00:08:36.900265+0000","last_peered":"2024-09-16T00:08:36.900265+0000","last_clean":"2024-09-16T00:08:36.900265+0000","last_became_active":"2024-09-16T00:08:36.896505+0000","last_became_peered":"2024-09-16T00:08:36.896505+0000","last_unstale":"2024-09-16T00:08:36.900265+0000","last_undegraded":"2024-09-16T00:08:36.900265+0000","last_fullsized":"2024-09-16T00:08:36.900265+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:55.997551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.786661+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:08:39.786661+0000","last_peered":"2024-09-16T00:08:39.786661+0000","last_clean":"2024-09-16T00:08:39.786661+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:08:39.786661+0000","last_undegraded":"2024-09-16T00:08:39.786661+0000","last_fullsized":"2024-09-16T00:08:39.786661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882734+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:08:36.882734+0000","last_peered":"2024-09-16T00:08:36.882734+0000","last_clean":"2024-09-16T00:08:36.882734+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:08:36.882734+0000","last_undegraded":"2024-09-16T00:08:36.882734+0000","last_fullsized":"2024-09-16T00:08:36.882734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883994+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:08:36.883994+0000","last_peered":"2024-09-16T00:08:36.883994+0000","last_clean":"2024-09-16T00:08:36.883994+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:08:36.883994+0000","last_undegraded":"2024-09-16T00:08:36.883994+0000","last_fullsized":"2024-09-16T00:08:36.883994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883532+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:08:36.883532+0000","last_peered":"2024-09-16T00:08:36.883532+0000","last_clean":"2024-09-16T00:08:36.883532+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:08:36.883532+0000","last_undegraded":"2024-09-16T00:08:36.883532+0000","last_fullsized":"2024-09-16T00:08:36.883532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":68,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.787141+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:08:39.787141+0000","last_peered":"2024-09-16T00:08:39.787141+0000","last_clean":"2024-09-16T00:08:39.787141+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:08:39.787141+0000","last_undegraded":"2024-09-16T00:08:39.787141+0000","last_fullsized":"2024-09-16T00:08:39.787141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883810+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:08:36.883810+0000","last_peered":"2024-09-16T00:08:36.883810+0000","last_clean":"2024-09-16T00:08:36.883810+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:08:36.883810+0000","last_undegraded":"2024-09-16T00:08:36.883810+0000","last_fullsized":"2024-09-16T00:08:36.883810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.914050+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:08:38.914050+0000","last_peered":"2024-09-16T00:08:38.914050+0000","last_clean":"2024-09-16T00:08:38.914050+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:08:38.914050+0000","last_undegraded":"2024-09-16T00:08:38.914050+0000","last_fullsized":"2024-09-16T00:08:38.914050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888593+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:08:36.888593+0000","last_peered":"2024-09-16T00:08:36.888593+0000","last_clean":"2024-09-16T00:08:36.888593+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:08:36.888593+0000","last_undegraded":"2024-09-16T00:08:36.888593+0000","last_fullsized":"2024-09-16T00:08:36.888593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.786984+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:08:39.786984+0000","last_peered":"2024-09-16T00:08:39.786984+0000","last_clean":"2024-09-16T00:08:39.786984+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:08:39.786984+0000","last_undegraded":"2024-09-16T00:08:39.786984+0000","last_fullsized":"2024-09-16T00:08:39.786984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884151+0000","last_change":"2024-09-16T00:08:15.795467+0000","last_active":"2024-09-16T00:08:36.884151+0000","last_peered":"2024-09-16T00:08:36.884151+0000","last_clean":"2024-09-16T00:08:36.884151+0000","last_became_active":"2024-09-16T00:08:15.795167+0000","last_became_peered":"2024-09-16T00:08:15.795167+0000","last_unstale":"2024-09-16T00:08:36.884151+0000","last_undegraded":"2024-09-16T00:08:36.884151+0000","last_fullsized":"2024-09-16T00:08:36.884151+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":219,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882705+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:08:36.882705+0000","last_peered":"2024-09-16T00:08:36.882705+0000","last_clean":"2024-09-16T00:08:36.882705+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:08:36.882705+0000","last_undegraded":"2024-09-16T00:08:36.882705+0000","last_fullsized":"2024-09-16T00:08:36.882705+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888542+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:08:36.888542+0000","last_peered":"2024-09-16T00:08:36.888542+0000","last_clean":"2024-09-16T00:08:36.888542+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:08:36.888542+0000","last_undegraded":"2024-09-16T00:08:36.888542+0000","last_fullsized":"2024-09-16T00:08:36.888542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389905+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:08:40.389905+0000","last_peered":"2024-09-16T00:08:40.389905+0000","last_clean":"2024-09-16T00:08:40.389905+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:08:40.389905+0000","last_undegraded":"2024-09-16T00:08:40.389905+0000","last_fullsized":"2024-09-16T00:08:40.389905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882625+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:08:36.882625+0000","last_peered":"2024-09-16T00:08:36.882625+0000","last_clean":"2024-09-16T00:08:36.882625+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:08:36.882625+0000","last_undegraded":"2024-09-16T00:08:36.882625+0000","last_fullsized":"2024-09-16T00:08:36.882625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504238+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:08:40.504238+0000","last_peered":"2024-09-16T00:08:40.504238+0000","last_clean":"2024-09-16T00:08:40.504238+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:08:40.504238+0000","last_undegraded":"2024-09-16T00:08:40.504238+0000","last_fullsized":"2024-09-16T00:08:40.504238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.786772+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:08:39.786772+0000","last_peered":"2024-09-16T00:08:39.786772+0000","last_clean":"2024-09-16T00:08:39.786772+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:08:39.786772+0000","last_undegraded":"2024-09-16T00:08:39.786772+0000","last_fullsized":"2024-09-16T00:08:39.786772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913422+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:08:38.913422+0000","last_peered":"2024-09-16T00:08:38.913422+0000","last_clean":"2024-09-16T00:08:38.913422+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:08:38.913422+0000","last_undegraded":"2024-09-16T00:08:38.913422+0000","last_fullsized":"2024-09-16T00:08:38.913422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":64,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888456+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:08:36.888456+0000","last_peered":"2024-09-16T00:08:36.888456+0000","last_clean":"2024-09-16T00:08:36.888456+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:08:36.888456+0000","last_undegraded":"2024-09-16T00:08:36.888456+0000","last_fullsized":"2024-09-16T00:08:36.888456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883562+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:08:36.883562+0000","last_peered":"2024-09-16T00:08:36.883562+0000","last_clean":"2024-09-16T00:08:36.883562+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:08:36.883562+0000","last_undegraded":"2024-09-16T00:08:36.883562+0000","last_fullsized":"2024-09-16T00:08:36.883562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913851+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:08:38.913851+0000","last_peered":"2024-09-16T00:08:38.913851+0000","last_clean":"2024-09-16T00:08:38.913851+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:08:38.913851+0000","last_undegraded":"2024-09-16T00:08:38.913851+0000","last_fullsized":"2024-09-16T00:08:38.913851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.889077+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:08:36.889077+0000","last_peered":"2024-09-16T00:08:36.889077+0000","last_clean":"2024-09-16T00:08:36.889077+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:08:36.889077+0000","last_undegraded":"2024-09-16T00:08:36.889077+0000","last_fullsized":"2024-09-16T00:08:36.889077+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884353+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:08:36.884353+0000","last_peered":"2024-09-16T00:08:36.884353+0000","last_clean":"2024-09-16T00:08:36.884353+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:08:36.884353+0000","last_undegraded":"2024-09-16T00:08:36.884353+0000","last_fullsized":"2024-09-16T00:08:36.884353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913770+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:08:38.913770+0000","last_peered":"2024-09-16T00:08:38.913770+0000","last_clean":"2024-09-16T00:08:38.913770+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:08:38.913770+0000","last_undegraded":"2024-09-16T00:08:38.913770+0000","last_fullsized":"2024-09-16T00:08:38.913770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"26'3","reported_seq":86,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913666+0000","last_change":"2024-09-16T00:08:36.906253+0000","last_active":"2024-09-16T00:08:38.913666+0000","last_peered":"2024-09-16T00:08:38.913666+0000","last_clean":"2024-09-16T00:08:38.913666+0000","last_became_active":"2024-09-16T00:08:36.894363+0000","last_became_peered":"2024-09-16T00:08:36.894363+0000","last_unstale":"2024-09-16T00:08:38.913666+0000","last_undegraded":"2024-09-16T00:08:38.913666+0000","last_fullsized":"2024-09-16T00:08:38.913666+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:00.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389880+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:08:40.389880+0000","last_peered":"2024-09-16T00:08:40.389880+0000","last_clean":"2024-09-16T00:08:40.389880+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:08:40.389880+0000","last_undegraded":"2024-09-16T00:08:40.389880+0000","last_fullsized":"2024-09-16T00:08:40.389880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":66,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.886688+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:08:36.886688+0000","last_peered":"2024-09-16T00:08:36.886688+0000","last_clean":"2024-09-16T00:08:36.886688+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:08:36.886688+0000","last_undegraded":"2024-09-16T00:08:36.886688+0000","last_fullsized":"2024-09-16T00:08:36.886688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504700+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:08:40.504700+0000","last_peered":"2024-09-16T00:08:40.504700+0000","last_clean":"2024-09-16T00:08:40.504700+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:08:40.504700+0000","last_undegraded":"2024-09-16T00:08:40.504700+0000","last_fullsized":"2024-09-16T00:08:40.504700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":63,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882517+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:08:36.882517+0000","last_peered":"2024-09-16T00:08:36.882517+0000","last_clean":"2024-09-16T00:08:36.882517+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:08:36.882517+0000","last_undegraded":"2024-09-16T00:08:36.882517+0000","last_fullsized":"2024-09-16T00:08:36.882517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":61,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.884343+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:08:36.884343+0000","last_peered":"2024-09-16T00:08:36.884343+0000","last_clean":"2024-09-16T00:08:36.884343+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:08:36.884343+0000","last_undegraded":"2024-09-16T00:08:36.884343+0000","last_fullsized":"2024-09-16T00:08:36.884343+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913841+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:08:38.913841+0000","last_peered":"2024-09-16T00:08:38.913841+0000","last_clean":"2024-09-16T00:08:38.913841+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:08:38.913841+0000","last_undegraded":"2024-09-16T00:08:38.913841+0000","last_fullsized":"2024-09-16T00:08:38.913841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389182+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:08:40.389182+0000","last_peered":"2024-09-16T00:08:40.389182+0000","last_clean":"2024-09-16T00:08:40.389182+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:08:40.389182+0000","last_undegraded":"2024-09-16T00:08:40.389182+0000","last_fullsized":"2024-09-16T00:08:40.389182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913666+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:08:38.913666+0000","last_peered":"2024-09-16T00:08:38.913666+0000","last_clean":"2024-09-16T00:08:38.913666+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:08:38.913666+0000","last_undegraded":"2024-09-16T00:08:38.913666+0000","last_fullsized":"2024-09-16T00:08:38.913666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389497+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:08:40.389497+0000","last_peered":"2024-09-16T00:08:40.389497+0000","last_clean":"2024-09-16T00:08:40.389497+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:08:40.389497+0000","last_undegraded":"2024-09-16T00:08:40.389497+0000","last_fullsized":"2024-09-16T00:08:40.389497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.882815+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:08:36.882815+0000","last_peered":"2024-09-16T00:08:36.882815+0000","last_clean":"2024-09-16T00:08:36.882815+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:08:36.882815+0000","last_undegraded":"2024-09-16T00:08:36.882815+0000","last_fullsized":"2024-09-16T00:08:36.882815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.892834+0000","last_change":"2024-09-16T00:08:36.890347+0000","last_active":"2024-09-16T00:08:36.892834+0000","last_peered":"2024-09-16T00:08:36.892834+0000","last_clean":"2024-09-16T00:08:36.892834+0000","last_became_active":"2024-09-16T00:08:36.890009+0000","last_became_peered":"2024-09-16T00:08:36.890009+0000","last_unstale":"2024-09-16T00:08:36.892834+0000","last_undegraded":"2024-09-16T00:08:36.892834+0000","last_fullsized":"2024-09-16T00:08:36.892834+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:24.298978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888991+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:08:36.888991+0000","last_peered":"2024-09-16T00:08:36.888991+0000","last_clean":"2024-09-16T00:08:36.888991+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:08:36.888991+0000","last_undegraded":"2024-09-16T00:08:36.888991+0000","last_fullsized":"2024-09-16T00:08:36.888991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":65,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.883181+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:08:36.883181+0000","last_peered":"2024-09-16T00:08:36.883181+0000","last_clean":"2024-09-16T00:08:36.883181+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:08:36.883181+0000","last_undegraded":"2024-09-16T00:08:36.883181+0000","last_fullsized":"2024-09-16T00:08:36.883181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":62,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.888518+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:08:36.888518+0000","last_peered":"2024-09-16T00:08:36.888518+0000","last_clean":"2024-09-16T00:08:36.888518+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:08:36.888518+0000","last_undegraded":"2024-09-16T00:08:36.888518+0000","last_fullsized":"2024-09-16T00:08:36.888518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":83,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-16T00:08:36.900704+0000","last_change":"2024-09-16T00:08:36.900704+0000","last_active":"2024-09-16T00:08:36.900704+0000","last_peered":"2024-09-16T00:08:36.900704+0000","last_clean":"2024-09-16T00:08:36.900704+0000","last_became_active":"2024-09-16T00:08:36.892520+0000","last_became_peered":"2024-09-16T00:08:36.892520+0000","last_unstale":"2024-09-16T00:08:36.900704+0000","last_undegraded":"2024-09-16T00:08:36.900704+0000","last_fullsized":"2024-09-16T00:08:36.900704+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:30.056041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.787096+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:08:39.787096+0000","last_peered":"2024-09-16T00:08:39.787096+0000","last_clean":"2024-09-16T00:08:39.787096+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:08:39.787096+0000","last_undegraded":"2024-09-16T00:08:39.787096+0000","last_fullsized":"2024-09-16T00:08:39.787096+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":67,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389975+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:08:40.389975+0000","last_peered":"2024-09-16T00:08:40.389975+0000","last_clean":"2024-09-16T00:08:40.389975+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:08:40.389975+0000","last_undegraded":"2024-09-16T00:08:40.389975+0000","last_fullsized":"2024-09-16T00:08:40.389975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389058+0000","last_change":"2024-09-16T00:08:36.889082+0000","last_active":"2024-09-16T00:08:40.389058+0000","last_peered":"2024-09-16T00:08:40.389058+0000","last_clean":"2024-09-16T00:08:40.389058+0000","last_became_active":"2024-09-16T00:08:36.888807+0000","last_became_peered":"2024-09-16T00:08:36.888807+0000","last_unstale":"2024-09-16T00:08:40.389058+0000","last_undegraded":"2024-09-16T00:08:40.389058+0000","last_fullsized":"2024-09-16T00:08:40.389058+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:22.825324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":69,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504726+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:08:40.504726+0000","last_peered":"2024-09-16T00:08:40.504726+0000","last_clean":"2024-09-16T00:08:40.504726+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:08:40.504726+0000","last_undegraded":"2024-09-16T00:08:40.504726+0000","last_fullsized":"2024-09-16T00:08:40.504726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7353,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":13110,"num_keys_recovered":4,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":14706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":585728,"data_stored":815033,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":540672,"data_stored":236719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":230728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":540672,"data_stored":230380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":544768,"data_stored":236809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":548864,"data_stored":236529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":618496,"data_stored":827375,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":528384,"data_stored":230254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":6555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":6555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:08:42.037 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:42.036+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:42.098 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:42.098 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:42.107 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:08:42.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.106+0000 7f51710dd640 1 Processor -- start 2024-09-16T00:08:42.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.106+0000 7f51710dd640 1 -- start start 2024-09-16T00:08:42.109 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:42.107+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:42.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.107+0000 7f51710dd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f516c0d0490 0x7f516c0b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:42.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.107+0000 7f51710dd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f516c0d0870 0x7f516c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:42.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.107+0000 7f51710dd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f516c0b3430 0x7f516c0b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:42.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.107+0000 7f51710dd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f516c15b940 con 0x7f516c0d0490 2024-09-16T00:08:42.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.107+0000 7f51710dd640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f516c0b3bb0 con 0x7f516c0b3430 2024-09-16T00:08:42.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.107+0000 7f51710dd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f516c0b3180 con 0x7f516c0d0870 2024-09-16T00:08:42.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.107+0000 7f516ad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f516c0d0490 0x7f516c0b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:42.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.107+0000 7f516ad9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f516c0d0490 0x7f516c0b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49362/0 (socket says 172.21.15.139:49362) 2024-09-16T00:08:42.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.107+0000 7f516ad9d640 1 -- 172.21.15.139:0/2137026415 learned_addr learned my addr 172.21.15.139:0/2137026415 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:42.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.107+0000 7f516a59c640 1 --2- 172.21.15.139:0/2137026415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f516c0d0870 0x7f516c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:42.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.108+0000 7f516ad9d640 1 -- 172.21.15.139:0/2137026415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f516c0d0870 msgr2=0x7f516c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:42.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.108+0000 7f516ad9d640 1 --2- 172.21.15.139:0/2137026415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f516c0d0870 0x7f516c0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.108+0000 7f516ad9d640 1 -- 172.21.15.139:0/2137026415 >> v2:172.21.15.164:3300/0 conn(0x7f516c0b3430 msgr2=0x7f516c0b4580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:08:42.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.108+0000 7f516ad9d640 1 --2- 172.21.15.139:0/2137026415 >> v2:172.21.15.164:3300/0 conn(0x7f516c0b3430 0x7f516c0b4580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.108+0000 7f516ad9d640 1 -- 172.21.15.139:0/2137026415 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516c15adc0 con 0x7f516c0d0490 2024-09-16T00:08:42.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.109+0000 7f516ad9d640 1 --2- 172.21.15.139:0/2137026415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f516c0d0490 0x7f516c0b2c40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f516c0c7780 tx=0x7f516002a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8da1c07aba1856 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:42.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.109+0000 7f5169d9b640 1 -- 172.21.15.139:0/2137026415 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5160037070 con 0x7f516c0d0490 2024-09-16T00:08:42.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.109+0000 7f5169d9b640 1 -- 172.21.15.139:0/2137026415 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5160032440 con 0x7f516c0d0490 2024-09-16T00:08:42.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.109+0000 7f5169d9b640 1 -- 172.21.15.139:0/2137026415 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5160037070 con 0x7f516c0d0490 2024-09-16T00:08:42.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.110+0000 7f51710dd640 1 -- 172.21.15.139:0/2137026415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f516c0d0490 msgr2=0x7f516c0b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:42.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.110+0000 7f51710dd640 1 --2- 172.21.15.139:0/2137026415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f516c0d0490 0x7f516c0b2c40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f516c0c7780 tx=0x7f516002a290 comp rx=0 tx=0).stop 2024-09-16T00:08:42.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.110+0000 7f51710dd640 1 -- 172.21.15.139:0/2137026415 shutdown_connections 2024-09-16T00:08:42.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.110+0000 7f51710dd640 1 --2- 172.21.15.139:0/2137026415 >> v2:172.21.15.164:3300/0 conn(0x7f516c0b3430 0x7f516c0b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.110+0000 7f51710dd640 1 --2- 172.21.15.139:0/2137026415 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f516c0d0870 0x7f516c0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.110+0000 7f51710dd640 1 --2- 172.21.15.139:0/2137026415 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f516c0d0490 0x7f516c0b2c40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.110+0000 7f51710dd640 1 -- 172.21.15.139:0/2137026415 >> 172.21.15.139:0/2137026415 conn(0x7f516c06c520 msgr2=0x7f516c06c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:42.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.110+0000 7f51710dd640 1 -- 172.21.15.139:0/2137026415 shutdown_connections 2024-09-16T00:08:42.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.110+0000 7f51710dd640 1 -- 172.21.15.139:0/2137026415 wait complete. 2024-09-16T00:08:42.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.112+0000 7f51710dd640 1 Processor -- start 2024-09-16T00:08:42.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.112+0000 7f51710dd640 1 -- start start 2024-09-16T00:08:42.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.113+0000 7f51710dd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f516c0b3430 0x7f516c0c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:42.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.113+0000 7f51710dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f516c0d0870 0x7f516c157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:42.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.113+0000 7f51710dd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c0c8100 0x7f516c157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:42.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.113+0000 7f51710dd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f516c15b660 con 0x7f516c0d0870 2024-09-16T00:08:42.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.113+0000 7f51710dd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f516c15b380 con 0x7f516c0c8100 2024-09-16T00:08:42.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.113+0000 7f51710dd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f516c15a980 con 0x7f516c0b3430 2024-09-16T00:08:42.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.113+0000 7f516a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f516c0d0870 0x7f516c157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:42.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.113+0000 7f516a59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f516c0d0870 0x7f516c157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49376/0 (socket says 172.21.15.139:49376) 2024-09-16T00:08:42.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.113+0000 7f516a59c640 1 -- 172.21.15.139:0/3389922829 learned_addr learned my addr 172.21.15.139:0/3389922829 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:42.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.113+0000 7f516b59e640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c0c8100 0x7f516c157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:42.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.113+0000 7f516ad9d640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f516c0b3430 0x7f516c0c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:42.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.113+0000 7f516b59e640 1 -- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f516c0b3430 msgr2=0x7f516c0c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:42.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.113+0000 7f516b59e640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f516c0b3430 0x7f516c0c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.113+0000 7f516b59e640 1 -- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f516c0d0870 msgr2=0x7f516c157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:42.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.113+0000 7f516b59e640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f516c0d0870 0x7f516c157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.113+0000 7f516b59e640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516c1f3800 con 0x7f516c0c8100 2024-09-16T00:08:42.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.114+0000 7f516b59e640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c0c8100 0x7f516c157c80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f516400be20 tx=0x7f516400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:42.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.114+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164019070 con 0x7f516c0c8100 2024-09-16T00:08:42.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.114+0000 7f51710dd640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516c0c7880 con 0x7f516c0c8100 2024-09-16T00:08:42.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.114+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5164014440 con 0x7f516c0c8100 2024-09-16T00:08:42.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.114+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5164014810 con 0x7f516c0c8100 2024-09-16T00:08:42.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.115+0000 7f51710dd640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f516c1f2480 con 0x7f516c0c8100 2024-09-16T00:08:42.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.116+0000 7f51710dd640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f516c0b2c40 con 0x7f516c0c8100 2024-09-16T00:08:42.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.116+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f516401c070 con 0x7f516c0c8100 2024-09-16T00:08:42.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.117+0000 7f51537fe640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f515806c2b0 0x7f515806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:42.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.117+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5915+0+0 (secure 0 0 0) 0x7f516408b470 con 0x7f516c0c8100 2024-09-16T00:08:42.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.117+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v49) ==== 54+0+16 (secure 0 0 0) 0x7f5164059390 con 0x7f516c0c8100 2024-09-16T00:08:42.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.117+0000 7f516ad9d640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f515806c2b0 0x7f515806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:42.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.118+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5144001650 0x7f5144003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:42.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.118+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5144006bb0 con 0x7f5144001650 2024-09-16T00:08:42.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.118+0000 7f516ad9d640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f515806c2b0 0x7f515806e770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5160034000 tx=0x7f516002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:42.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.118+0000 7f516a59c640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5144001650 0x7f5144003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:42.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.119+0000 7f516a59c640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5144001650 0x7f5144003b10 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:42.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.119+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5144006bb0 con 0x7f5144001650 2024-09-16T00:08:42.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.136+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5144005c60 con 0x7f5144001650 2024-09-16T00:08:42.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.138+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5144005c60 con 0x7f5144001650 2024-09-16T00:08:42.140 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:42.141 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:42.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.138+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5144004190 0x7f5144004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:42.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.138+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f5144004c00 con 0x7f5144004190 2024-09-16T00:08:42.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.139+0000 7f516ad9d640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5144004190 0x7f5144004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:42.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.139+0000 7f516ad9d640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5144004190 0x7f5144004640 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:42.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.140+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5144004c00 con 0x7f5144004190 2024-09-16T00:08:42.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.167+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5144005140 con 0x7f5144004190 2024-09-16T00:08:42.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.171+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5144005140 con 0x7f5144004190 2024-09-16T00:08:42.173 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:42.174 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:42.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.172+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5144009370 0x7f514400b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:42.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.172+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f514400e6c0 con 0x7f5144009370 2024-09-16T00:08:42.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.172+0000 7f516a59c640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5144009370 0x7f514400b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:42.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.172+0000 7f516a59c640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5144009370 0x7f514400b760 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:42.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.173+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f514400e6c0 con 0x7f5144009370 2024-09-16T00:08:42.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.200+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f514400d6b0 con 0x7f5144009370 2024-09-16T00:08:42.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.203+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f514400d6b0 con 0x7f5144009370 2024-09-16T00:08:42.205 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:42.206 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:42.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.203+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f514400e130 0x7f514400bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:42.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.203+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f514400c490 con 0x7f514400e130 2024-09-16T00:08:42.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.204+0000 7f516b59e640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f514400e130 0x7f514400bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:42.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.204+0000 7f516b59e640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f514400e130 0x7f514400bca0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:42.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.205+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f514400c490 con 0x7f514400e130 2024-09-16T00:08:42.209 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:42.209+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:42.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.218+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f514400c9a0 con 0x7f514400e130 2024-09-16T00:08:42.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.220+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f514400c9a0 con 0x7f514400e130 2024-09-16T00:08:42.222 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:42.223 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:42.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.220+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f514400cf10 0x7f5144012890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:42.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.220+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f51440158e0 con 0x7f514400cf10 2024-09-16T00:08:42.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.221+0000 7f516ad9d640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f514400cf10 0x7f5144012890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:42.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.221+0000 7f516ad9d640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f514400cf10 0x7f5144012890 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:42.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.222+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f51440158e0 con 0x7f514400cf10 2024-09-16T00:08:42.224 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:42.224 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:42.224 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:42.224 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:42.233 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:08:42.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.235+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5144014600 con 0x7f514400cf10 2024-09-16T00:08:42.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.237+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5144014600 con 0x7f514400cf10 2024-09-16T00:08:42.240 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:42.241 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:42.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.237+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5144014d20 0x7f5144015160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:42.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.237+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f5144018cb0 con 0x7f5144014d20 2024-09-16T00:08:42.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.238+0000 7f516a59c640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5144014d20 0x7f5144015160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:42.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.238+0000 7f516a59c640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5144014d20 0x7f5144015160 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:42.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.239+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5144018cb0 con 0x7f5144014d20 2024-09-16T00:08:42.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.265+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5144017bb0 con 0x7f5144014d20 2024-09-16T00:08:42.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.266+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5144017bb0 con 0x7f5144014d20 2024-09-16T00:08:42.268 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:42.269 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:42.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.267+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f51440182d0 0x7f5144018710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:42.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.267+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f514401c080 con 0x7f51440182d0 2024-09-16T00:08:42.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.267+0000 7f516b59e640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f51440182d0 0x7f5144018710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:42.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.268+0000 7f516b59e640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f51440182d0 0x7f5144018710 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:42.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.268+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f514401c080 con 0x7f51440182d0 2024-09-16T00:08:42.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.294+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f514401af00 con 0x7f51440182d0 2024-09-16T00:08:42.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.296+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f514401af00 con 0x7f51440182d0 2024-09-16T00:08:42.298 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:42.298 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:42.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.296+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f514401b620 0x7f514401ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:42.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.296+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f514401f3f0 con 0x7f514401b620 2024-09-16T00:08:42.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.296+0000 7f516ad9d640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f514401b620 0x7f514401ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:42.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.297+0000 7f516ad9d640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f514401b620 0x7f514401ba60 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:42.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.298+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f514401f3f0 con 0x7f514401b620 2024-09-16T00:08:42.310 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:42.309+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:42.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.324+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5144012ec0 con 0x7f514401b620 2024-09-16T00:08:42.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51537fe640 1 -- 172.21.15.139:0/3389922829 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5144012ec0 con 0x7f514401b620 2024-09-16T00:08:42.327 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:42.329 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:42.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5144001650 msgr2=0x7f5144003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:42.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5144001650 0x7f5144003b10 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5144004190 msgr2=0x7f5144004640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:42.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5144004190 0x7f5144004640 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5144009370 msgr2=0x7f514400b760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:42.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5144009370 0x7f514400b760 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f516b59e640 1 -- 172.21.15.139:0/3389922829 reap_dead start 2024-09-16T00:08:42.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f514400e130 msgr2=0x7f514400bca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:42.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f514400e130 0x7f514400bca0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f514400cf10 msgr2=0x7f5144012890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:42.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f514400cf10 0x7f5144012890 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5144014d20 msgr2=0x7f5144015160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:42.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5144014d20 0x7f5144015160 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f51440182d0 msgr2=0x7f5144018710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:42.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f51440182d0 0x7f5144018710 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f514401b620 msgr2=0x7f514401ba60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:42.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f514401b620 0x7f514401ba60 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.326+0000 7f516b59e640 1 -- 172.21.15.139:0/3389922829 reap_dead start 2024-09-16T00:08:42.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.327+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f515806c2b0 msgr2=0x7f515806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:42.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.327+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f515806c2b0 0x7f515806e770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5160034000 tx=0x7f516002e000 comp rx=0 tx=0).stop 2024-09-16T00:08:42.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.327+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c0c8100 msgr2=0x7f516c157c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:42.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.327+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c0c8100 0x7f516c157c80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f516400be20 tx=0x7f516400bf10 comp rx=0 tx=0).stop 2024-09-16T00:08:42.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.327+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 shutdown_connections 2024-09-16T00:08:42.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.327+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f515806c2b0 0x7f515806e770 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.327+0000 7f51517fa640 1 --2- 172.21.15.139:0/3389922829 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f516c0c8100 0x7f516c157c80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:42.331 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.327+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 >> 172.21.15.139:0/3389922829 conn(0x7f516c06c520 msgr2=0x7f516c14a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:42.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.328+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 shutdown_connections 2024-09-16T00:08:42.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:42.328+0000 7f51517fa640 1 -- 172.21.15.139:0/3389922829 wait complete. 2024-09-16T00:08:42.368 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:42.368 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:42.368 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:42.368 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:42.368 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:42.378 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:08:42.411 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:42.411+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:42.512 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:42.512+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:42.566 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:42.566 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:42.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:42.566 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:42.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'41, mlcod=17'39)", 2024-09-16T00:08:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 78.542899977000005, 2024-09-16T00:08:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001808488, 2024-09-16T00:08:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021031+0000", 2024-09-16T00:08:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021032+0000", 2024-09-16T00:08:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3e-07 2024-09-16T00:08:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021037+0000", 2024-09-16T00:08:42.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0640000000000001e-06 2024-09-16T00:08:42.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021047+0000", 2024-09-16T00:08:42.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.023e-05 2024-09-16T00:08:42.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021053+0000", 2024-09-16T00:08:42.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3160000000000002e-06 2024-09-16T00:08:42.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021092+0000", 2024-09-16T00:08:42.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9033999999999999e-05 2024-09-16T00:08:42.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.021105+0000", 2024-09-16T00:08:42.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2768e-05 2024-09-16T00:08:42.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022811+0000", 2024-09-16T00:08:42.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001705788 2024-09-16T00:08:42.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.571 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.571 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.022840+0000", 2024-09-16T00:08:42.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8558e-05 2024-09-16T00:08:42.571 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.571 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.571 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.571 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:85 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'55, mlcod=17'50)", 2024-09-16T00:08:42.571 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:42.571 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 78.529870975999998, 2024-09-16T00:08:42.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017909530000000001, 2024-09-16T00:08:42.572 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.572 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.572 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.572 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:42.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.573 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034060+0000", 2024-09-16T00:08:42.573 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.573 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.573 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.573 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.573 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034061+0000", 2024-09-16T00:08:42.573 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0399999999999996e-07 2024-09-16T00:08:42.573 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.573 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.573 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.573 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034065+0000", 2024-09-16T00:08:42.574 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8840000000000002e-06 2024-09-16T00:08:42.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.574 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034074+0000", 2024-09-16T00:08:42.574 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:42.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.574 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034079+0000", 2024-09-16T00:08:42.574 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0000000000000004e-06 2024-09-16T00:08:42.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034125+0000", 2024-09-16T00:08:42.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5905000000000002e-05 2024-09-16T00:08:42.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.034144+0000", 2024-09-16T00:08:42.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9593e-05 2024-09-16T00:08:42.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035817+0000", 2024-09-16T00:08:42.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016728240000000001 2024-09-16T00:08:42.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.576 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.576 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.035851+0000", 2024-09-16T00:08:42.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3809000000000001e-05 2024-09-16T00:08:42.576 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.576 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.576 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.576 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.576 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:112 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'74, mlcod=17'65)", 2024-09-16T00:08:42.576 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:42.576 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 78.399259114000003, 2024-09-16T00:08:42.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018457020000000001, 2024-09-16T00:08:42.576 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.577 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.577 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.577 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:42.577 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164672+0000", 2024-09-16T00:08:42.577 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164673+0000", 2024-09-16T00:08:42.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1200000000000003e-07 2024-09-16T00:08:42.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164710+0000", 2024-09-16T00:08:42.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7756999999999998e-05 2024-09-16T00:08:42.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164719+0000", 2024-09-16T00:08:42.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2849999999999995e-06 2024-09-16T00:08:42.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164723+0000", 2024-09-16T00:08:42.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3220000000000003e-06 2024-09-16T00:08:42.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164760+0000", 2024-09-16T00:08:42.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7311999999999998e-05 2024-09-16T00:08:42.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.164771+0000", 2024-09-16T00:08:42.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0869e-05 2024-09-16T00:08:42.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166495+0000", 2024-09-16T00:08:42.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001723863 2024-09-16T00:08:42.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166518+0000", 2024-09-16T00:08:42.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2682000000000001e-05 2024-09-16T00:08:42.580 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.580 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.580 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.581 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'66)", 2024-09-16T00:08:42.581 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:42.581 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 78.397755134999997, 2024-09-16T00:08:42.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032925509999999999, 2024-09-16T00:08:42.581 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.581 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.582 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.582 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.582 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.582 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:42.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.582 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.582 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:42.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.582 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.583 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166177+0000", 2024-09-16T00:08:42.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:42.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.583 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166216+0000", 2024-09-16T00:08:42.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9008000000000003e-05 2024-09-16T00:08:42.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.584 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166226+0000", 2024-09-16T00:08:42.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0081999999999999e-05 2024-09-16T00:08:42.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.584 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.584 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166230+0000", 2024-09-16T00:08:42.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5079999999999999e-06 2024-09-16T00:08:42.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166259+0000", 2024-09-16T00:08:42.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8928000000000002e-05 2024-09-16T00:08:42.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166271+0000", 2024-09-16T00:08:42.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1531e-05 2024-09-16T00:08:42.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169449+0000", 2024-09-16T00:08:42.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003178143 2024-09-16T00:08:42.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169468+0000", 2024-09-16T00:08:42.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9700000000000001e-05 2024-09-16T00:08:42.586 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.586 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.587 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.587 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'66)", 2024-09-16T00:08:42.587 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:42.587 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 78.397293676000004, 2024-09-16T00:08:42.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003179374, 2024-09-16T00:08:42.587 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.587 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.587 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.587 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.587 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.587 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:42.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:42.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166638+0000", 2024-09-16T00:08:42.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2300000000000002e-07 2024-09-16T00:08:42.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166641+0000", 2024-09-16T00:08:42.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0079999999999998e-06 2024-09-16T00:08:42.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166646+0000", 2024-09-16T00:08:42.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4970000000000001e-06 2024-09-16T00:08:42.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166650+0000", 2024-09-16T00:08:42.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5729999999999999e-06 2024-09-16T00:08:42.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.590 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166682+0000", 2024-09-16T00:08:42.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2218999999999998e-05 2024-09-16T00:08:42.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.590 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166694+0000", 2024-09-16T00:08:42.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1902999999999999e-05 2024-09-16T00:08:42.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169784+0000", 2024-09-16T00:08:42.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003089545 2024-09-16T00:08:42.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169817+0000", 2024-09-16T00:08:42.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3206e-05 2024-09-16T00:08:42.591 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.591 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.591 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.591 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:117 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'66)", 2024-09-16T00:08:42.592 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:42.592 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 78.396849615999997, 2024-09-16T00:08:42.592 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00275326, 2024-09-16T00:08:42.592 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.592 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.592 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.592 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.592 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.592 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:42.592 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.593 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.593 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:42.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.593 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.593 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.593 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167082+0000", 2024-09-16T00:08:42.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:08:42.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.593 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.593 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.593 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167086+0000", 2024-09-16T00:08:42.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1559999999999997e-06 2024-09-16T00:08:42.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.594 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.594 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.594 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167096+0000", 2024-09-16T00:08:42.594 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8390000000000002e-06 2024-09-16T00:08:42.594 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.594 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.594 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.594 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167101+0000", 2024-09-16T00:08:42.594 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:08:42.594 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.595 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.595 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.595 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167130+0000", 2024-09-16T00:08:42.595 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9649e-05 2024-09-16T00:08:42.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.595 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.595 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.595 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167142+0000", 2024-09-16T00:08:42.595 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1435e-05 2024-09-16T00:08:42.596 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.596 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.596 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.596 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169818+0000", 2024-09-16T00:08:42.596 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002676056 2024-09-16T00:08:42.596 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.596 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.596 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.596 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169835+0000", 2024-09-16T00:08:42.597 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6980999999999999e-05 2024-09-16T00:08:42.597 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.597 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.598 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.598 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.598 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.598 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:130 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'88, mlcod=17'85)", 2024-09-16T00:08:42.598 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:42.598 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 77.954050287000001, 2024-09-16T00:08:42.598 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002880456, 2024-09-16T00:08:42.598 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.598 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.598 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.599 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.599 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.599 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:42.599 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.599 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:42.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:42.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5699999999999998e-07 2024-09-16T00:08:42.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609885+0000", 2024-09-16T00:08:42.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.608e-06 2024-09-16T00:08:42.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.601 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.601 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609891+0000", 2024-09-16T00:08:42.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:42.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.601 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.601 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609895+0000", 2024-09-16T00:08:42.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.54e-06 2024-09-16T00:08:42.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609931+0000", 2024-09-16T00:08:42.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5893999999999999e-05 2024-09-16T00:08:42.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609945+0000", 2024-09-16T00:08:42.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4259e-05 2024-09-16T00:08:42.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.603 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612721+0000", 2024-09-16T00:08:42.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027757849999999998 2024-09-16T00:08:42.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.603 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.603 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612761+0000", 2024-09-16T00:08:42.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0638e-05 2024-09-16T00:08:42.603 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.603 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.603 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.603 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-16T00:08:42.604 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:42.604 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 77.953666314000003, 2024-09-16T00:08:42.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028016529999999999, 2024-09-16T00:08:42.604 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.604 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.604 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.604 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.604 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.604 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:42.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.604 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:42.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:42.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.63e-07 2024-09-16T00:08:42.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610344+0000", 2024-09-16T00:08:42.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8996000000000001e-05 2024-09-16T00:08:42.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610352+0000", 2024-09-16T00:08:42.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:42.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610356+0000", 2024-09-16T00:08:42.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0790000000000002e-06 2024-09-16T00:08:42.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610388+0000", 2024-09-16T00:08:42.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1541000000000002e-05 2024-09-16T00:08:42.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.607 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.607 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610400+0000", 2024-09-16T00:08:42.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2075e-05 2024-09-16T00:08:42.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.607 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.607 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613045+0000", 2024-09-16T00:08:42.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002645213 2024-09-16T00:08:42.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.607 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.607 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613066+0000", 2024-09-16T00:08:42.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1353e-05 2024-09-16T00:08:42.608 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.608 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.608 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.608 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-16T00:08:42.608 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:42.608 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 77.950406814000004, 2024-09-16T00:08:42.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027254850000000001, 2024-09-16T00:08:42.609 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.609 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.609 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.609 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:42.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:42.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613525+0000", 2024-09-16T00:08:42.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2800000000000002e-07 2024-09-16T00:08:42.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613528+0000", 2024-09-16T00:08:42.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9730000000000002e-06 2024-09-16T00:08:42.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.612 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.612 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613534+0000", 2024-09-16T00:08:42.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4339999999999997e-06 2024-09-16T00:08:42.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.612 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.612 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613538+0000", 2024-09-16T00:08:42.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5149999999999998e-06 2024-09-16T00:08:42.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.613 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.613 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613566+0000", 2024-09-16T00:08:42.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8104000000000001e-05 2024-09-16T00:08:42.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.613 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613578+0000", 2024-09-16T00:08:42.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2404e-05 2024-09-16T00:08:42.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616217+0000", 2024-09-16T00:08:42.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026389830000000001 2024-09-16T00:08:42.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.615 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616250+0000", 2024-09-16T00:08:42.615 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2644000000000001e-05 2024-09-16T00:08:42.615 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.615 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.615 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.616 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:134 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'92, mlcod=18'87)", 2024-09-16T00:08:42.616 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:42.616 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 77.949868675000005, 2024-09-16T00:08:42.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025322399999999998, 2024-09-16T00:08:42.616 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.616 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.616 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.616 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:42.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:42.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.618 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614063+0000", 2024-09-16T00:08:42.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8700000000000005e-07 2024-09-16T00:08:42.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.618 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.618 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614096+0000", 2024-09-16T00:08:42.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3136000000000003e-05 2024-09-16T00:08:42.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.618 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.618 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614106+0000", 2024-09-16T00:08:42.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9170000000000001e-06 2024-09-16T00:08:42.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.619 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.619 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614111+0000", 2024-09-16T00:08:42.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4390000000000002e-06 2024-09-16T00:08:42.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.619 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.619 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614141+0000", 2024-09-16T00:08:42.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.04e-05 2024-09-16T00:08:42.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614152+0000", 2024-09-16T00:08:42.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0987e-05 2024-09-16T00:08:42.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616573+0000", 2024-09-16T00:08:42.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002421351 2024-09-16T00:08:42.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616595+0000", 2024-09-16T00:08:42.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1322999999999999e-05 2024-09-16T00:08:42.621 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.621 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.621 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.622 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:135 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'93, mlcod=18'87)", 2024-09-16T00:08:42.622 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:42.622 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 77.949348637, 2024-09-16T00:08:42.622 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020304659999999999, 2024-09-16T00:08:42.622 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.622 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.622 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.622 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:42.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614582+0000", 2024-09-16T00:08:42.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.624 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614583+0000", 2024-09-16T00:08:42.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:42.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.625 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614587+0000", 2024-09-16T00:08:42.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3100000000000002e-06 2024-09-16T00:08:42.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614597+0000", 2024-09-16T00:08:42.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7450000000000007e-06 2024-09-16T00:08:42.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.626 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614602+0000", 2024-09-16T00:08:42.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6099999999999999e-06 2024-09-16T00:08:42.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.627 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614641+0000", 2024-09-16T00:08:42.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9767000000000001e-05 2024-09-16T00:08:42.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614658+0000", 2024-09-16T00:08:42.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6205000000000002e-05 2024-09-16T00:08:42.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616596+0000", 2024-09-16T00:08:42.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019386659999999999 2024-09-16T00:08:42.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.629 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616613+0000", 2024-09-16T00:08:42.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6555999999999999e-05 2024-09-16T00:08:42.630 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.630 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.630 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.630 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:139 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'97, mlcod=18'87)", 2024-09-16T00:08:42.631 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:42.631 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 77.947587966, 2024-09-16T00:08:42.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030908139999999999, 2024-09-16T00:08:42.631 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.631 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.631 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.632 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.632 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.632 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:42.632 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:42.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.634 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616344+0000", 2024-09-16T00:08:42.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3000000000000001e-07 2024-09-16T00:08:42.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616347+0000", 2024-09-16T00:08:42.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7740000000000002e-06 2024-09-16T00:08:42.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616355+0000", 2024-09-16T00:08:42.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4699999999999996e-06 2024-09-16T00:08:42.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616358+0000", 2024-09-16T00:08:42.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5899999999999999e-06 2024-09-16T00:08:42.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616393+0000", 2024-09-16T00:08:42.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4529000000000001e-05 2024-09-16T00:08:42.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616404+0000", 2024-09-16T00:08:42.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0726e-05 2024-09-16T00:08:42.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619411+0000", 2024-09-16T00:08:42.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030076249999999999 2024-09-16T00:08:42.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619434+0000", 2024-09-16T00:08:42.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-05 2024-09-16T00:08:42.640 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.640 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.640 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.641 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:140 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'98, mlcod=18'89)", 2024-09-16T00:08:42.641 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:42.641 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 77.946946573000005, 2024-09-16T00:08:42.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027276150000000001, 2024-09-16T00:08:42.641 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.642 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.642 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.642 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:42.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:42.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616985+0000", 2024-09-16T00:08:42.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4099999999999999e-07 2024-09-16T00:08:42.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.645 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617055+0000", 2024-09-16T00:08:42.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9923000000000001e-05 2024-09-16T00:08:42.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.646 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.646 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617063+0000", 2024-09-16T00:08:42.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:42.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617070+0000", 2024-09-16T00:08:42.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9090000000000003e-06 2024-09-16T00:08:42.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617102+0000", 2024-09-16T00:08:42.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2620999999999998e-05 2024-09-16T00:08:42.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617114+0000", 2024-09-16T00:08:42.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1228e-05 2024-09-16T00:08:42.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619690+0000", 2024-09-16T00:08:42.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025763890000000001 2024-09-16T00:08:42.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.650 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619712+0000", 2024-09-16T00:08:42.650 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2171000000000002e-05 2024-09-16T00:08:42.650 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.650 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.650 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.651 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:42.651 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:42.651 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 63.948117473000003, 2024-09-16T00:08:42.651 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027355320000000002, 2024-09-16T00:08:42.651 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.651 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.651 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.652 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.652 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.652 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:42.652 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.652 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.652 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.652 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:42.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.653 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.653 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615815+0000", 2024-09-16T00:08:42.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1769999999999999e-06 2024-09-16T00:08:42.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.653 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.654 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615825+0000", 2024-09-16T00:08:42.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9000000000000001e-06 2024-09-16T00:08:42.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.654 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615840+0000", 2024-09-16T00:08:42.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4854e-05 2024-09-16T00:08:42.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.655 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615848+0000", 2024-09-16T00:08:42.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0940000000000008e-06 2024-09-16T00:08:42.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.656 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.656 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615926+0000", 2024-09-16T00:08:42.656 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8176999999999994e-05 2024-09-16T00:08:42.656 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.656 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.656 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.656 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615945+0000", 2024-09-16T00:08:42.657 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9663000000000001e-05 2024-09-16T00:08:42.657 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.657 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.657 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.657 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618509+0000", 2024-09-16T00:08:42.657 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002563802 2024-09-16T00:08:42.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.658 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618549+0000", 2024-09-16T00:08:42.658 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9864999999999999e-05 2024-09-16T00:08:42.658 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.658 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.658 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.659 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:42.659 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:42.659 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 63.948028333000003, 2024-09-16T00:08:42.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061593860000000002, 2024-09-16T00:08:42.659 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.659 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.660 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.660 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.660 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.660 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.660 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:42.660 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.660 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.660 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.661 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.661 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:42.661 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.661 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.662 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.662 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.662 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615907+0000", 2024-09-16T00:08:42.662 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9389999999999999e-06 2024-09-16T00:08:42.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.662 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.663 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.663 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615920+0000", 2024-09-16T00:08:42.663 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2989999999999999e-05 2024-09-16T00:08:42.663 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.663 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.663 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.663 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615931+0000", 2024-09-16T00:08:42.664 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.164e-05 2024-09-16T00:08:42.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.664 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.664 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.664 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615939+0000", 2024-09-16T00:08:42.664 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8029999999999997e-06 2024-09-16T00:08:42.665 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.665 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.665 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.665 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616310+0000", 2024-09-16T00:08:42.665 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00037111400000000003 2024-09-16T00:08:42.665 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616406+0000", 2024-09-16T00:08:42.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5372999999999996e-05 2024-09-16T00:08:42.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:42.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616514+0000", 2024-09-16T00:08:42.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108564 2024-09-16T00:08:42.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:42.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618158+0000", 2024-09-16T00:08:42.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016442449999999999 2024-09-16T00:08:42.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.668 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:42.669 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621972+0000", 2024-09-16T00:08:42.669 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038132299999999999 2024-09-16T00:08:42.669 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.669 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.669 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622018+0000", 2024-09-16T00:08:42.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6239000000000002e-05 2024-09-16T00:08:42.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622062+0000", 2024-09-16T00:08:42.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4249000000000003e-05 2024-09-16T00:08:42.671 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.671 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.671 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.671 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:42.671 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:42.672 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 63.947974864999999, 2024-09-16T00:08:42.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006258914, 2024-09-16T00:08:42.672 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.672 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.672 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.673 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:42.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.674 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.674 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:42.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.674 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.675 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615957+0000", 2024-09-16T00:08:42.675 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1399999999999997e-07 2024-09-16T00:08:42.675 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.675 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.675 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.675 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615960+0000", 2024-09-16T00:08:42.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0860000000000002e-06 2024-09-16T00:08:42.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.676 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.676 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615967+0000", 2024-09-16T00:08:42.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9059999999999996e-06 2024-09-16T00:08:42.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615972+0000", 2024-09-16T00:08:42.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5679999999999999e-06 2024-09-16T00:08:42.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616717+0000", 2024-09-16T00:08:42.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074438 2024-09-16T00:08:42.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616781+0000", 2024-09-16T00:08:42.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4808999999999996e-05 2024-09-16T00:08:42.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:42.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616885+0000", 2024-09-16T00:08:42.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010404900000000001 2024-09-16T00:08:42.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:42.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:42.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012773870000000001 2024-09-16T00:08:42.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.681 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.681 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:42.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622116+0000", 2024-09-16T00:08:42.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039533470000000003 2024-09-16T00:08:42.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622173+0000", 2024-09-16T00:08:42.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6347000000000001e-05 2024-09-16T00:08:42.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622215+0000", 2024-09-16T00:08:42.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2521000000000002e-05 2024-09-16T00:08:42.683 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.684 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.684 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.684 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:42.685 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:42.685 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 63.947951128, 2024-09-16T00:08:42.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041725529999999999, 2024-09-16T00:08:42.685 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.685 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.685 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.686 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:42.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:42.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:42.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8799999999999998e-07 2024-09-16T00:08:42.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615990+0000", 2024-09-16T00:08:42.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2150000000000007e-06 2024-09-16T00:08:42.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:08:42.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7159999999999997e-06 2024-09-16T00:08:42.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616001+0000", 2024-09-16T00:08:42.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9339999999999997e-06 2024-09-16T00:08:42.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616293+0000", 2024-09-16T00:08:42.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029210899999999998 2024-09-16T00:08:42.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616401+0000", 2024-09-16T00:08:42.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108087 2024-09-16T00:08:42.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:42.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616561+0000", 2024-09-16T00:08:42.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015940600000000001 2024-09-16T00:08:42.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:42.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618146+0000", 2024-09-16T00:08:42.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015857040000000001 2024-09-16T00:08:42.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:42.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620091+0000", 2024-09-16T00:08:42.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001944837 2024-09-16T00:08:42.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620126+0000", 2024-09-16T00:08:42.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4242999999999997e-05 2024-09-16T00:08:42.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620152+0000", 2024-09-16T00:08:42.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6914e-05 2024-09-16T00:08:42.697 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.697 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.698 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.698 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:42.698 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:42.698 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 63.947912903000002, 2024-09-16T00:08:42.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00630628, 2024-09-16T00:08:42.699 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.699 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.699 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.699 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:42.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:42.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:08:42.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5999999999999999e-07 2024-09-16T00:08:42.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616022+0000", 2024-09-16T00:08:42.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0149999999999999e-06 2024-09-16T00:08:42.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616031+0000", 2024-09-16T00:08:42.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9770000000000006e-06 2024-09-16T00:08:42.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616036+0000", 2024-09-16T00:08:42.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2639999999999999e-06 2024-09-16T00:08:42.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617057+0000", 2024-09-16T00:08:42.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010212999999999999 2024-09-16T00:08:42.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617146+0000", 2024-09-16T00:08:42.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9234000000000007e-05 2024-09-16T00:08:42.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:42.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617232+0000", 2024-09-16T00:08:42.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5218999999999998e-05 2024-09-16T00:08:42.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:42.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619107+0000", 2024-09-16T00:08:42.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001875016 2024-09-16T00:08:42.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:42.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622259+0000", 2024-09-16T00:08:42.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031527930000000001 2024-09-16T00:08:42.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622290+0000", 2024-09-16T00:08:42.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1022999999999998e-05 2024-09-16T00:08:42.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622324+0000", 2024-09-16T00:08:42.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3979000000000001e-05 2024-09-16T00:08:42.709 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.710 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.710 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.710 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:42.710 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:42.711 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 63.947887600999998, 2024-09-16T00:08:42.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042412739999999997, 2024-09-16T00:08:42.711 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.711 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.711 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:42.711 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:42.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:42.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616044+0000", 2024-09-16T00:08:42.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:42.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616052+0000", 2024-09-16T00:08:42.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.439e-06 2024-09-16T00:08:42.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616059+0000", 2024-09-16T00:08:42.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1389999999999998e-06 2024-09-16T00:08:42.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616065+0000", 2024-09-16T00:08:42.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:08:42.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616823+0000", 2024-09-16T00:08:42.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075781199999999996 2024-09-16T00:08:42.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616918+0000", 2024-09-16T00:08:42.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5174999999999996e-05 2024-09-16T00:08:42.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:42.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617019+0000", 2024-09-16T00:08:42.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101161 2024-09-16T00:08:42.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:42.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618503+0000", 2024-09-16T00:08:42.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014836750000000001 2024-09-16T00:08:42.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:42.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620201+0000", 2024-09-16T00:08:42.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00169805 2024-09-16T00:08:42.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:42.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620255+0000", 2024-09-16T00:08:42.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4499000000000001e-05 2024-09-16T00:08:42.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620285+0000", 2024-09-16T00:08:42.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9349e-05 2024-09-16T00:08:42.721 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.722 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.722 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.722 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:42.722 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:42.723 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 63.947858609999997, 2024-09-16T00:08:42.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063682799999999996, 2024-09-16T00:08:42.723 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:42.723 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:42.723 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:42.724 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:42.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:42.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:42.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:42.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:42.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:42.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:42.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:42.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9900000000000001e-07 2024-09-16T00:08:42.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:42.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616079+0000", 2024-09-16T00:08:42.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9009999999999999e-06 2024-09-16T00:08:42.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:42.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616086+0000", 2024-09-16T00:08:42.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.776e-06 2024-09-16T00:08:42.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:42.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616091+0000", 2024-09-16T00:08:42.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5600000000000001e-06 2024-09-16T00:08:42.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:42.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618339+0000", 2024-09-16T00:08:42.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022476179999999998 2024-09-16T00:08:42.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:42.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:42.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9025999999999997e-05 2024-09-16T00:08:42.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:42.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:42.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:42.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622441+0000", 2024-09-16T00:08:42.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040029999999999996 2024-09-16T00:08:42.731 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.731 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.731 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.732 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:42.732 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:42.732 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:42.735 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:08:42.735 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:42.732+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:42.835 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:42.833+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:42.911 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:42.911 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:42.911 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:42.911 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:42.924 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:08:42.936 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:42.934+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:43.037 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:43.037+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:43.077 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:43.077 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:43.077 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:43.077 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:43.078 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:43.086 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:08:43.138 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:43.138+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:43.209 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:43.209 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:43.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:43.209 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:43.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.209 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:43.210 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:43.210 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.591713677000001, 2024-09-16T00:08:43.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037514929999999998, 2024-09-16T00:08:43.210 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.210 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.210 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.210 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:43.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:43.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615364+0000", 2024-09-16T00:08:43.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8430000000000003e-06 2024-09-16T00:08:43.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615371+0000", 2024-09-16T00:08:43.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.148e-06 2024-09-16T00:08:43.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.212 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.212 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615380+0000", 2024-09-16T00:08:43.212 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5889999999999992e-06 2024-09-16T00:08:43.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.212 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.212 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615389+0000", 2024-09-16T00:08:43.212 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1620000000000008e-06 2024-09-16T00:08:43.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.212 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615525+0000", 2024-09-16T00:08:43.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000136083 2024-09-16T00:08:43.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615662+0000", 2024-09-16T00:08:43.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013696100000000001 2024-09-16T00:08:43.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:43.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615799+0000", 2024-09-16T00:08:43.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013781399999999999 2024-09-16T00:08:43.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:43.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:43.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002363502 2024-09-16T00:08:43.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:43.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619018+0000", 2024-09-16T00:08:43.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085522199999999999 2024-09-16T00:08:43.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619068+0000", 2024-09-16T00:08:43.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9545000000000001e-05 2024-09-16T00:08:43.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619111+0000", 2024-09-16T00:08:43.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3624000000000001e-05 2024-09-16T00:08:43.215 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.215 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.215 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.215 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:43.215 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:43.215 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.591661606000002, 2024-09-16T00:08:43.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043656110000000001, 2024-09-16T00:08:43.216 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.216 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.216 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.216 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:43.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:43.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:43.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2199999999999999e-07 2024-09-16T00:08:43.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615416+0000", 2024-09-16T00:08:43.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5590000000000001e-06 2024-09-16T00:08:43.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615423+0000", 2024-09-16T00:08:43.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6250000000000001e-06 2024-09-16T00:08:43.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.218 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615426+0000", 2024-09-16T00:08:43.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2540000000000001e-06 2024-09-16T00:08:43.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.218 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616075+0000", 2024-09-16T00:08:43.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064968800000000004 2024-09-16T00:08:43.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616142+0000", 2024-09-16T00:08:43.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6203000000000003e-05 2024-09-16T00:08:43.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:43.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616247+0000", 2024-09-16T00:08:43.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010572499999999999 2024-09-16T00:08:43.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:43.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618913+0000", 2024-09-16T00:08:43.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026654339999999999 2024-09-16T00:08:43.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:43.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619712+0000", 2024-09-16T00:08:43.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079885199999999996 2024-09-16T00:08:43.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619744+0000", 2024-09-16T00:08:43.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2790000000000003e-05 2024-09-16T00:08:43.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619778+0000", 2024-09-16T00:08:43.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3059000000000003e-05 2024-09-16T00:08:43.222 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.222 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.222 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.222 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:43.222 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:43.222 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.591625527000005, 2024-09-16T00:08:43.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087673219999999993, 2024-09-16T00:08:43.222 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.223 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.223 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.223 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:43.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:43.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:43.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:43.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:08:43.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5392000000000001e-05 2024-09-16T00:08:43.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615470+0000", 2024-09-16T00:08:43.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1999999999999999e-06 2024-09-16T00:08:43.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615474+0000", 2024-09-16T00:08:43.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7359999999999999e-06 2024-09-16T00:08:43.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617614+0000", 2024-09-16T00:08:43.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021398189999999998 2024-09-16T00:08:43.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617724+0000", 2024-09-16T00:08:43.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110808 2024-09-16T00:08:43.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:43.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617856+0000", 2024-09-16T00:08:43.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131676 2024-09-16T00:08:43.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:43.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:43.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029629280000000001 2024-09-16T00:08:43.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:43.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624158+0000", 2024-09-16T00:08:43.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033386549999999998 2024-09-16T00:08:43.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624189+0000", 2024-09-16T00:08:43.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1508999999999999e-05 2024-09-16T00:08:43.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624215+0000", 2024-09-16T00:08:43.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6228999999999999e-05 2024-09-16T00:08:43.227 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.227 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.228 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.228 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:43.228 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:43.228 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.591591511999994, 2024-09-16T00:08:43.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053285279999999999, 2024-09-16T00:08:43.228 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.228 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.228 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.228 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.229 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:43.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.229 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.229 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:43.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.229 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.229 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:43.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5600000000000002e-07 2024-09-16T00:08:43.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615484+0000", 2024-09-16T00:08:43.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.688e-06 2024-09-16T00:08:43.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615488+0000", 2024-09-16T00:08:43.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.658e-06 2024-09-16T00:08:43.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615490+0000", 2024-09-16T00:08:43.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.728e-06 2024-09-16T00:08:43.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616441+0000", 2024-09-16T00:08:43.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00095065500000000001 2024-09-16T00:08:43.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616539+0000", 2024-09-16T00:08:43.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8335e-05 2024-09-16T00:08:43.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:43.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616631+0000", 2024-09-16T00:08:43.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1260000000000004e-05 2024-09-16T00:08:43.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:43.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619837+0000", 2024-09-16T00:08:43.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003206899 2024-09-16T00:08:43.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:43.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620725+0000", 2024-09-16T00:08:43.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088713599999999996 2024-09-16T00:08:43.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620772+0000", 2024-09-16T00:08:43.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7639999999999998e-05 2024-09-16T00:08:43.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620811+0000", 2024-09-16T00:08:43.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8272999999999998e-05 2024-09-16T00:08:43.233 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.233 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.233 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.233 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:43.233 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:43.233 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.591176423999997, 2024-09-16T00:08:43.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029886169999999998, 2024-09-16T00:08:43.234 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.234 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.234 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.234 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:43.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:43.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.235 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615898+0000", 2024-09-16T00:08:43.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.237e-06 2024-09-16T00:08:43.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.235 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615904+0000", 2024-09-16T00:08:43.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:43.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.235 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615915+0000", 2024-09-16T00:08:43.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1003e-05 2024-09-16T00:08:43.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.236 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615922+0000", 2024-09-16T00:08:43.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9959999999999996e-06 2024-09-16T00:08:43.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.236 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615988+0000", 2024-09-16T00:08:43.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5806e-05 2024-09-16T00:08:43.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616009+0000", 2024-09-16T00:08:43.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0729000000000001e-05 2024-09-16T00:08:43.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:43.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028462130000000002 2024-09-16T00:08:43.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618886+0000", 2024-09-16T00:08:43.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.082e-05 2024-09-16T00:08:43.237 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.237 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.238 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.238 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:43.238 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:43.238 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.589564952000003, 2024-09-16T00:08:43.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032610299999999998, 2024-09-16T00:08:43.238 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.238 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.238 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.238 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:43.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:43.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617556+0000", 2024-09-16T00:08:43.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7431e-05 2024-09-16T00:08:43.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617847+0000", 2024-09-16T00:08:43.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029133400000000003 2024-09-16T00:08:43.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617858+0000", 2024-09-16T00:08:43.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0222e-05 2024-09-16T00:08:43.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617864+0000", 2024-09-16T00:08:43.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7619999999999998e-06 2024-09-16T00:08:43.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617909+0000", 2024-09-16T00:08:43.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4747999999999999e-05 2024-09-16T00:08:43.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617927+0000", 2024-09-16T00:08:43.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7785999999999999e-05 2024-09-16T00:08:43.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620734+0000", 2024-09-16T00:08:43.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002807305 2024-09-16T00:08:43.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.242 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.242 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620770+0000", 2024-09-16T00:08:43.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5441999999999997e-05 2024-09-16T00:08:43.242 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.242 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.242 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.242 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:43.242 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:43.242 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.589144267999998, 2024-09-16T00:08:43.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028641809999999999, 2024-09-16T00:08:43.243 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.243 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.243 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.243 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:43.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:43.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617930+0000", 2024-09-16T00:08:43.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0200000000000002e-07 2024-09-16T00:08:43.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617934+0000", 2024-09-16T00:08:43.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:43.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617943+0000", 2024-09-16T00:08:43.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5820000000000002e-06 2024-09-16T00:08:43.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617949+0000", 2024-09-16T00:08:43.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5069999999999997e-06 2024-09-16T00:08:43.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618094+0000", 2024-09-16T00:08:43.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014515599999999999 2024-09-16T00:08:43.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618111+0000", 2024-09-16T00:08:43.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6705999999999999e-05 2024-09-16T00:08:43.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:08:43.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026603070000000002 2024-09-16T00:08:43.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620793+0000", 2024-09-16T00:08:43.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2214000000000001e-05 2024-09-16T00:08:43.246 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.246 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.247 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.247 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:43.247 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:43.247 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.589113479000005, 2024-09-16T00:08:43.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028624549999999999, 2024-09-16T00:08:43.247 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.247 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.247 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.247 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:43.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:43.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:43.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:43.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:43.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0570000000000001e-06 2024-09-16T00:08:43.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617970+0000", 2024-09-16T00:08:43.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.669e-06 2024-09-16T00:08:43.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617976+0000", 2024-09-16T00:08:43.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1369999999999999e-06 2024-09-16T00:08:43.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618268+0000", 2024-09-16T00:08:43.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029164600000000001 2024-09-16T00:08:43.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618284+0000", 2024-09-16T00:08:43.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5824e-05 2024-09-16T00:08:43.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:43.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025127890000000001 2024-09-16T00:08:43.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620822+0000", 2024-09-16T00:08:43.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6009000000000001e-05 2024-09-16T00:08:43.251 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.251 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.251 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.251 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-16T00:08:43.251 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:43.251 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.589085897999993, 2024-09-16T00:08:43.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028650350000000002, 2024-09-16T00:08:43.252 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.252 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.252 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.252 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:43.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:43.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:43.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7599999999999998e-07 2024-09-16T00:08:43.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617991+0000", 2024-09-16T00:08:43.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1860000000000001e-06 2024-09-16T00:08:43.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617999+0000", 2024-09-16T00:08:43.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3250000000000002e-06 2024-09-16T00:08:43.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618004+0000", 2024-09-16T00:08:43.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7300000000000002e-06 2024-09-16T00:08:43.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618422+0000", 2024-09-16T00:08:43.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00041763199999999998 2024-09-16T00:08:43.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:43.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6112000000000001e-05 2024-09-16T00:08:43.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620824+0000", 2024-09-16T00:08:43.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002385954 2024-09-16T00:08:43.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620853+0000", 2024-09-16T00:08:43.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8719999999999999e-05 2024-09-16T00:08:43.255 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.256 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.256 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.256 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-16T00:08:43.256 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:43.256 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.589060075999996, 2024-09-16T00:08:43.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032703020000000001, 2024-09-16T00:08:43.256 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.256 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.256 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.256 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:43.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:43.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618014+0000", 2024-09-16T00:08:43.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7799999999999997e-07 2024-09-16T00:08:43.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618017+0000", 2024-09-16T00:08:43.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4309999999999998e-06 2024-09-16T00:08:43.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618023+0000", 2024-09-16T00:08:43.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8490000000000002e-06 2024-09-16T00:08:43.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618028+0000", 2024-09-16T00:08:43.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3489999999999999e-06 2024-09-16T00:08:43.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:08:43.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055724099999999999 2024-09-16T00:08:43.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618603+0000", 2024-09-16T00:08:43.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7486000000000002e-05 2024-09-16T00:08:43.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621235+0000", 2024-09-16T00:08:43.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002632087 2024-09-16T00:08:43.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621284+0000", 2024-09-16T00:08:43.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8581000000000001e-05 2024-09-16T00:08:43.260 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.260 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.260 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.260 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:14 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:43.260 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:43.260 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.588123096000004, 2024-09-16T00:08:43.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016807599999999999, 2024-09-16T00:08:43.261 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.261 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.261 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:43.261 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:43.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:43.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618951+0000", 2024-09-16T00:08:43.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:43.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618954+0000", 2024-09-16T00:08:43.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.145e-06 2024-09-16T00:08:43.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618963+0000", 2024-09-16T00:08:43.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5720000000000003e-06 2024-09-16T00:08:43.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618969+0000", 2024-09-16T00:08:43.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3319999999999997e-06 2024-09-16T00:08:43.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619008+0000", 2024-09-16T00:08:43.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9096e-05 2024-09-16T00:08:43.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619013+0000", 2024-09-16T00:08:43.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4710000000000001e-06 2024-09-16T00:08:43.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619118+0000", 2024-09-16T00:08:43.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000105809 2024-09-16T00:08:43.264 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.264 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.264 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.264 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-16T00:08:43.265 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:43.265 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.588084988999995, 2024-09-16T00:08:43.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002325307, 2024-09-16T00:08:43.265 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.265 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.265 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.265 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:43.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:43.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:43.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:43.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618993+0000", 2024-09-16T00:08:43.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.687e-06 2024-09-16T00:08:43.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619000+0000", 2024-09-16T00:08:43.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3590000000000002e-06 2024-09-16T00:08:43.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619005+0000", 2024-09-16T00:08:43.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.95e-06 2024-09-16T00:08:43.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619039+0000", 2024-09-16T00:08:43.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3942999999999997e-05 2024-09-16T00:08:43.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619055+0000", 2024-09-16T00:08:43.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6107e-05 2024-09-16T00:08:43.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621287+0000", 2024-09-16T00:08:43.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022318070000000001 2024-09-16T00:08:43.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:43.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7129999999999999e-05 2024-09-16T00:08:43.269 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.269 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.269 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.269 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-16T00:08:43.269 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:43.269 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.588046993000006, 2024-09-16T00:08:43.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002314328, 2024-09-16T00:08:43.269 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.269 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.270 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.270 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:43.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:43.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:43.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6399999999999998e-07 2024-09-16T00:08:43.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619030+0000", 2024-09-16T00:08:43.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.219e-06 2024-09-16T00:08:43.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619038+0000", 2024-09-16T00:08:43.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9899999999999997e-06 2024-09-16T00:08:43.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619043+0000", 2024-09-16T00:08:43.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5580000000000002e-06 2024-09-16T00:08:43.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619220+0000", 2024-09-16T00:08:43.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000176937 2024-09-16T00:08:43.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619237+0000", 2024-09-16T00:08:43.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7691999999999999e-05 2024-09-16T00:08:43.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621315+0000", 2024-09-16T00:08:43.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020778900000000002 2024-09-16T00:08:43.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621341+0000", 2024-09-16T00:08:43.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5678e-05 2024-09-16T00:08:43.273 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.273 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.273 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.274 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:15 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:43.274 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:43.274 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.587432023999995, 2024-09-16T00:08:43.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143416, 2024-09-16T00:08:43.274 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.274 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.274 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:43.274 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:43.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:43.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619642+0000", 2024-09-16T00:08:43.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5199999999999995e-07 2024-09-16T00:08:43.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619645+0000", 2024-09-16T00:08:43.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.134e-06 2024-09-16T00:08:43.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.276 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.276 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619655+0000", 2024-09-16T00:08:43.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3330000000000003e-06 2024-09-16T00:08:43.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.276 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.276 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619662+0000", 2024-09-16T00:08:43.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0500000000000003e-06 2024-09-16T00:08:43.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619704+0000", 2024-09-16T00:08:43.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1766e-05 2024-09-16T00:08:43.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619708+0000", 2024-09-16T00:08:43.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3440000000000003e-06 2024-09-16T00:08:43.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619785+0000", 2024-09-16T00:08:43.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6936999999999999e-05 2024-09-16T00:08:43.277 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.278 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.278 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.278 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:31 2.d e22/19 ondisk, result = 0)", 2024-09-16T00:08:43.278 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:43.278 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.587333215000001, 2024-09-16T00:08:43.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101826, 2024-09-16T00:08:43.278 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.278 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.278 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:43.278 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:43.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:43.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619741+0000", 2024-09-16T00:08:43.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3099999999999997e-07 2024-09-16T00:08:43.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:08:43.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1719999999999998e-06 2024-09-16T00:08:43.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619771+0000", 2024-09-16T00:08:43.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4754000000000001e-05 2024-09-16T00:08:43.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619790+0000", 2024-09-16T00:08:43.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9516000000000001e-05 2024-09-16T00:08:43.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619830+0000", 2024-09-16T00:08:43.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9848000000000002e-05 2024-09-16T00:08:43.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619834+0000", 2024-09-16T00:08:43.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1150000000000004e-06 2024-09-16T00:08:43.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619842+0000", 2024-09-16T00:08:43.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.79e-06 2024-09-16T00:08:43.282 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.282 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.282 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.282 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.282 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:30 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:43.282 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:43.282 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.582982228000006, 2024-09-16T00:08:43.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013076400000000001, 2024-09-16T00:08:43.282 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.282 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.282 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:43.283 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:43.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:43.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624092+0000", 2024-09-16T00:08:43.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5799999999999999e-07 2024-09-16T00:08:43.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624096+0000", 2024-09-16T00:08:43.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:43.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624106+0000", 2024-09-16T00:08:43.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7629999999999995e-06 2024-09-16T00:08:43.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624112+0000", 2024-09-16T00:08:43.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.626e-06 2024-09-16T00:08:43.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624149+0000", 2024-09-16T00:08:43.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7094000000000001e-05 2024-09-16T00:08:43.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624154+0000", 2024-09-16T00:08:43.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7359999999999998e-06 2024-09-16T00:08:43.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624222+0000", 2024-09-16T00:08:43.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7680000000000003e-05 2024-09-16T00:08:43.286 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.286 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.286 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.286 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:39 2.6 2:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:43.286 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:43.286 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 59.566795669999998, 2024-09-16T00:08:43.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050072119999999996, 2024-09-16T00:08:43.286 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.286 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.287 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.287 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:43.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:43.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640280+0000", 2024-09-16T00:08:43.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5380000000000001e-06 2024-09-16T00:08:43.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.288 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640296+0000", 2024-09-16T00:08:43.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5950000000000001e-05 2024-09-16T00:08:43.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.288 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640327+0000", 2024-09-16T00:08:43.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0309999999999999e-05 2024-09-16T00:08:43.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640340+0000", 2024-09-16T00:08:43.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3773e-05 2024-09-16T00:08:43.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640422+0000", 2024-09-16T00:08:43.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1122e-05 2024-09-16T00:08:43.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640572+0000", 2024-09-16T00:08:43.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015044099999999999 2024-09-16T00:08:43.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:08:43.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640691+0000", 2024-09-16T00:08:43.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000119393 2024-09-16T00:08:43.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:43.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645006+0000", 2024-09-16T00:08:43.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043143779999999998 2024-09-16T00:08:43.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:43.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645169+0000", 2024-09-16T00:08:43.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016323299999999999 2024-09-16T00:08:43.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645227+0000", 2024-09-16T00:08:43.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8e-05 2024-09-16T00:08:43.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645285+0000", 2024-09-16T00:08:43.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8074e-05 2024-09-16T00:08:43.291 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.291 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.291 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.292 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:39 2.6 e22/19 ondisk, result = 0)", 2024-09-16T00:08:43.292 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:43.292 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 59.562223559000003, 2024-09-16T00:08:43.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016541200000000001, 2024-09-16T00:08:43.292 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.292 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.292 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:43.292 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:43.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:43.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644852+0000", 2024-09-16T00:08:43.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1670000000000002e-06 2024-09-16T00:08:43.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644865+0000", 2024-09-16T00:08:43.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2826000000000001e-05 2024-09-16T00:08:43.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644894+0000", 2024-09-16T00:08:43.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9264999999999998e-05 2024-09-16T00:08:43.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644911+0000", 2024-09-16T00:08:43.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6827999999999999e-05 2024-09-16T00:08:43.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644989+0000", 2024-09-16T00:08:43.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7849999999999995e-05 2024-09-16T00:08:43.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644999+0000", 2024-09-16T00:08:43.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7289999999999995e-06 2024-09-16T00:08:43.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645015+0000", 2024-09-16T00:08:43.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6747000000000001e-05 2024-09-16T00:08:43.295 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.295 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.295 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.296 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.6 38/37 [PushReplyOp(2:654134d2:::mds0_openfiles.0:head)])", 2024-09-16T00:08:43.296 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.845546+0000", 2024-09-16T00:08:43.296 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.361527708000001, 2024-09-16T00:08:43.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00023895899999999999, 2024-09-16T00:08:43.296 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.296 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.296 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:43.296 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845546+0000", 2024-09-16T00:08:43.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845546+0000", 2024-09-16T00:08:43.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845546+0000", 2024-09-16T00:08:43.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1299999999999999e-07 2024-09-16T00:08:43.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845549+0000", 2024-09-16T00:08:43.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6000000000000001e-06 2024-09-16T00:08:43.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845557+0000", 2024-09-16T00:08:43.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0970000000000006e-06 2024-09-16T00:08:43.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845564+0000", 2024-09-16T00:08:43.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3839999999999999e-06 2024-09-16T00:08:43.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845616+0000", 2024-09-16T00:08:43.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2120000000000002e-05 2024-09-16T00:08:43.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845785+0000", 2024-09-16T00:08:43.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016854500000000001 2024-09-16T00:08:43.299 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.299 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.299 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.299 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.299 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.300 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:43.301 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:08:43.301 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:43.299+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:43.402 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:43.401+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:43.502 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:43.502 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:43.502 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:43.502 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:43.503 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:43.502+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:43.514 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:08:43.604 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:43.602+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:43.646 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:43.646 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:43.646 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:43.646 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:43.646 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:43.655 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:08:43.704 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:43.703+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:43.805 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:43.805 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:43.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:43.805 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:43.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.805 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:43.805 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:43.805 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.186999368000002, 2024-09-16T00:08:43.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038263220000000001, 2024-09-16T00:08:43.806 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.806 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.806 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.806 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:43.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:43.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.807 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615465+0000", 2024-09-16T00:08:43.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2769999999999999e-06 2024-09-16T00:08:43.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.807 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615471+0000", 2024-09-16T00:08:43.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5729999999999996e-06 2024-09-16T00:08:43.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.807 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615483+0000", 2024-09-16T00:08:43.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1838000000000001e-05 2024-09-16T00:08:43.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615492+0000", 2024-09-16T00:08:43.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2410000000000001e-06 2024-09-16T00:08:43.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615596+0000", 2024-09-16T00:08:43.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104097 2024-09-16T00:08:43.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615688+0000", 2024-09-16T00:08:43.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1225999999999997e-05 2024-09-16T00:08:43.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:43.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615804+0000", 2024-09-16T00:08:43.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000116601 2024-09-16T00:08:43.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:43.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618165+0000", 2024-09-16T00:08:43.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023609429999999999 2024-09-16T00:08:43.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:43.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:08:43.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010258509999999999 2024-09-16T00:08:43.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619243+0000", 2024-09-16T00:08:43.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2327999999999998e-05 2024-09-16T00:08:43.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619287+0000", 2024-09-16T00:08:43.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3346999999999997e-05 2024-09-16T00:08:43.810 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.810 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.810 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.811 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:43.811 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:43.811 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.186944292000007, 2024-09-16T00:08:43.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0054735510000000001, 2024-09-16T00:08:43.811 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.811 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.811 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.811 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:43.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:43.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615516+0000", 2024-09-16T00:08:43.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6899999999999998e-07 2024-09-16T00:08:43.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615519+0000", 2024-09-16T00:08:43.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1760000000000002e-06 2024-09-16T00:08:43.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.813 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.813 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615526+0000", 2024-09-16T00:08:43.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8399999999999997e-06 2024-09-16T00:08:43.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.813 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.813 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615531+0000", 2024-09-16T00:08:43.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4450000000000004e-06 2024-09-16T00:08:43.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.813 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.813 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:43.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054127099999999996 2024-09-16T00:08:43.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616140+0000", 2024-09-16T00:08:43.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7230999999999994e-05 2024-09-16T00:08:43.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:43.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616249+0000", 2024-09-16T00:08:43.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109122 2024-09-16T00:08:43.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:43.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618646+0000", 2024-09-16T00:08:43.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023972279999999999 2024-09-16T00:08:43.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:43.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:08:43.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022396220000000001 2024-09-16T00:08:43.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620939+0000", 2024-09-16T00:08:43.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2735999999999998e-05 2024-09-16T00:08:43.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620989+0000", 2024-09-16T00:08:43.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0411000000000002e-05 2024-09-16T00:08:43.816 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.816 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.816 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.816 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:43.816 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:43.816 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.186918813000005, 2024-09-16T00:08:43.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055872789999999997, 2024-09-16T00:08:43.816 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.816 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.817 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.817 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:43.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:43.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.818 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:43.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.01e-07 2024-09-16T00:08:43.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.818 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.818 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.818 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615547+0000", 2024-09-16T00:08:43.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7899999999999996e-06 2024-09-16T00:08:43.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.818 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.818 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.818 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:08:43.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1119999999999998e-06 2024-09-16T00:08:43.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.819 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.819 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615560+0000", 2024-09-16T00:08:43.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:08:43.819 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.819 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.819 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616424+0000", 2024-09-16T00:08:43.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086350800000000003 2024-09-16T00:08:43.819 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.819 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.819 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:08:43.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9675999999999999e-05 2024-09-16T00:08:43.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:43.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616621+0000", 2024-09-16T00:08:43.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7345999999999999e-05 2024-09-16T00:08:43.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:43.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618654+0000", 2024-09-16T00:08:43.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020335269999999998 2024-09-16T00:08:43.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:43.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621055+0000", 2024-09-16T00:08:43.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024009729999999998 2024-09-16T00:08:43.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621100+0000", 2024-09-16T00:08:43.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5402999999999999e-05 2024-09-16T00:08:43.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621128+0000", 2024-09-16T00:08:43.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7736e-05 2024-09-16T00:08:43.821 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.821 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.821 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.822 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:43.822 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:43.822 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.186893162999993, 2024-09-16T00:08:43.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057024509999999999, 2024-09-16T00:08:43.822 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.822 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.822 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.822 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:43.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:43.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:43.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6300000000000001e-07 2024-09-16T00:08:43.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615569+0000", 2024-09-16T00:08:43.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3709999999999998e-06 2024-09-16T00:08:43.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.824 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615580+0000", 2024-09-16T00:08:43.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0448000000000001e-05 2024-09-16T00:08:43.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.824 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615584+0000", 2024-09-16T00:08:43.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6020000000000002e-06 2024-09-16T00:08:43.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616850+0000", 2024-09-16T00:08:43.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012661110000000001 2024-09-16T00:08:43.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616915+0000", 2024-09-16T00:08:43.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4820000000000006e-05 2024-09-16T00:08:43.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:43.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617031+0000", 2024-09-16T00:08:43.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115258 2024-09-16T00:08:43.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:43.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618659+0000", 2024-09-16T00:08:43.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001627975 2024-09-16T00:08:43.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:43.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621179+0000", 2024-09-16T00:08:43.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025203410000000002 2024-09-16T00:08:43.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621238+0000", 2024-09-16T00:08:43.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9323000000000001e-05 2024-09-16T00:08:43.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621269+0000", 2024-09-16T00:08:43.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0839000000000001e-05 2024-09-16T00:08:43.827 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.827 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.827 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.827 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:43.827 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:43.827 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.186869414, 2024-09-16T00:08:43.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058012769999999996, 2024-09-16T00:08:43.827 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.828 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.828 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.828 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:43.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:43.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615591+0000", 2024-09-16T00:08:43.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6199999999999999e-07 2024-09-16T00:08:43.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615594+0000", 2024-09-16T00:08:43.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8739999999999999e-06 2024-09-16T00:08:43.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615599+0000", 2024-09-16T00:08:43.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:08:43.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615603+0000", 2024-09-16T00:08:43.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8489999999999999e-06 2024-09-16T00:08:43.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617235+0000", 2024-09-16T00:08:43.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001632037 2024-09-16T00:08:43.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617348+0000", 2024-09-16T00:08:43.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113387 2024-09-16T00:08:43.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:43.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617458+0000", 2024-09-16T00:08:43.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109765 2024-09-16T00:08:43.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:43.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618664+0000", 2024-09-16T00:08:43.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012059950000000001 2024-09-16T00:08:43.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:43.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621311+0000", 2024-09-16T00:08:43.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026468469999999999 2024-09-16T00:08:43.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621357+0000", 2024-09-16T00:08:43.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6360999999999999e-05 2024-09-16T00:08:43.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621392+0000", 2024-09-16T00:08:43.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4190999999999998e-05 2024-09-16T00:08:43.832 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.832 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.833 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.833 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:43.833 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:43.833 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.186850789999994, 2024-09-16T00:08:43.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058737279999999999, 2024-09-16T00:08:43.833 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.833 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.833 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.833 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:43.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:43.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:43.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3799999999999999e-07 2024-09-16T00:08:43.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615611+0000", 2024-09-16T00:08:43.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-06 2024-09-16T00:08:43.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615626+0000", 2024-09-16T00:08:43.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:08:43.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615634+0000", 2024-09-16T00:08:43.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-06 2024-09-16T00:08:43.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617692+0000", 2024-09-16T00:08:43.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002057718 2024-09-16T00:08:43.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617768+0000", 2024-09-16T00:08:43.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6878999999999994e-05 2024-09-16T00:08:43.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:43.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:43.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019501699999999999 2024-09-16T00:08:43.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:43.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:08:43.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034357200000000002 2024-09-16T00:08:43.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:43.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621429+0000", 2024-09-16T00:08:43.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0012999999999999e-05 2024-09-16T00:08:43.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621459+0000", 2024-09-16T00:08:43.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9498000000000001e-05 2024-09-16T00:08:43.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621483+0000", 2024-09-16T00:08:43.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3957000000000001e-05 2024-09-16T00:08:43.838 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.838 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.838 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.838 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:43.838 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:43.838 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.186819510000007, 2024-09-16T00:08:43.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059371789999999999, 2024-09-16T00:08:43.838 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.838 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.839 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.839 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:43.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:43.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:08:43.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4000000000000002e-07 2024-09-16T00:08:43.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615643+0000", 2024-09-16T00:08:43.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6860000000000002e-06 2024-09-16T00:08:43.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615649+0000", 2024-09-16T00:08:43.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7610000000000004e-06 2024-09-16T00:08:43.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615653+0000", 2024-09-16T00:08:43.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0380000000000001e-06 2024-09-16T00:08:43.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618197+0000", 2024-09-16T00:08:43.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025440079999999999 2024-09-16T00:08:43.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618312+0000", 2024-09-16T00:08:43.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115083 2024-09-16T00:08:43.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:43.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618412+0000", 2024-09-16T00:08:43.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010016799999999999 2024-09-16T00:08:43.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:43.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621518+0000", 2024-09-16T00:08:43.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031058679999999999 2024-09-16T00:08:43.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:43.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621526+0000", 2024-09-16T00:08:43.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2490000000000004e-06 2024-09-16T00:08:43.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621549+0000", 2024-09-16T00:08:43.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3686999999999999e-05 2024-09-16T00:08:43.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621577+0000", 2024-09-16T00:08:43.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8191000000000002e-05 2024-09-16T00:08:43.843 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.843 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.843 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.844 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:43.844 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:43.844 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.186801844000001, 2024-09-16T00:08:43.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005989622, 2024-09-16T00:08:43.844 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.844 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.844 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.844 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:43.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:43.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:43.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3799999999999998e-07 2024-09-16T00:08:43.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615660+0000", 2024-09-16T00:08:43.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9740000000000001e-06 2024-09-16T00:08:43.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615666+0000", 2024-09-16T00:08:43.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3499999999999996e-06 2024-09-16T00:08:43.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615671+0000", 2024-09-16T00:08:43.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.13e-06 2024-09-16T00:08:43.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618759+0000", 2024-09-16T00:08:43.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030880500000000002 2024-09-16T00:08:43.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618832+0000", 2024-09-16T00:08:43.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3725000000000003e-05 2024-09-16T00:08:43.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:43.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618958+0000", 2024-09-16T00:08:43.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012587899999999999 2024-09-16T00:08:43.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:43.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621583+0000", 2024-09-16T00:08:43.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026246469999999999 2024-09-16T00:08:43.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:43.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621602+0000", 2024-09-16T00:08:43.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9137e-05 2024-09-16T00:08:43.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621629+0000", 2024-09-16T00:08:43.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6451000000000001e-05 2024-09-16T00:08:43.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621648+0000", 2024-09-16T00:08:43.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8941e-05 2024-09-16T00:08:43.849 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.849 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.849 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.849 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:43.849 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:43.849 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.186784224999997, 2024-09-16T00:08:43.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053141380000000004, 2024-09-16T00:08:43.849 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.850 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.850 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.850 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:43.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:43.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:43.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-07 2024-09-16T00:08:43.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615678+0000", 2024-09-16T00:08:43.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:08:43.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615682+0000", 2024-09-16T00:08:43.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6659999999999999e-06 2024-09-16T00:08:43.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615686+0000", 2024-09-16T00:08:43.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6990000000000001e-06 2024-09-16T00:08:43.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615736+0000", 2024-09-16T00:08:43.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9656e-05 2024-09-16T00:08:43.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615864+0000", 2024-09-16T00:08:43.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012836100000000001 2024-09-16T00:08:43.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:43.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615989+0000", 2024-09-16T00:08:43.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000125525 2024-09-16T00:08:43.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:43.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617355+0000", 2024-09-16T00:08:43.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001365982 2024-09-16T00:08:43.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:43.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620897+0000", 2024-09-16T00:08:43.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035419610000000002 2024-09-16T00:08:43.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620941+0000", 2024-09-16T00:08:43.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3998999999999997e-05 2024-09-16T00:08:43.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620990+0000", 2024-09-16T00:08:43.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8199999999999999e-05 2024-09-16T00:08:43.854 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.854 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.854 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.854 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:43.855 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:43.855 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.186756966999994, 2024-09-16T00:08:43.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006316449, 2024-09-16T00:08:43.855 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.855 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.855 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:43.855 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:43.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:43.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:43.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3299999999999998e-07 2024-09-16T00:08:43.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615705+0000", 2024-09-16T00:08:43.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.278e-06 2024-09-16T00:08:43.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615711+0000", 2024-09-16T00:08:43.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8119999999999996e-06 2024-09-16T00:08:43.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615716+0000", 2024-09-16T00:08:43.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4909999999999999e-06 2024-09-16T00:08:43.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617551+0000", 2024-09-16T00:08:43.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00183549 2024-09-16T00:08:43.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617627+0000", 2024-09-16T00:08:43.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6163999999999998e-05 2024-09-16T00:08:43.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:43.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617776+0000", 2024-09-16T00:08:43.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014909499999999999 2024-09-16T00:08:43.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:43.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621072+0000", 2024-09-16T00:08:43.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032955150000000002 2024-09-16T00:08:43.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:43.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621957+0000", 2024-09-16T00:08:43.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088553199999999996 2024-09-16T00:08:43.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:43.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621998+0000", 2024-09-16T00:08:43.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0815000000000001e-05 2024-09-16T00:08:43.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:08:43.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0924000000000002e-05 2024-09-16T00:08:43.860 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.860 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.860 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.860 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:6 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:43.860 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:43.860 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.183354999000002, 2024-09-16T00:08:43.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000188111, 2024-09-16T00:08:43.860 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.860 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.860 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:43.860 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:43.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:43.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619106+0000", 2024-09-16T00:08:43.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4399999999999998e-07 2024-09-16T00:08:43.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619110+0000", 2024-09-16T00:08:43.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1110000000000001e-06 2024-09-16T00:08:43.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619120+0000", 2024-09-16T00:08:43.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7909999999999999e-06 2024-09-16T00:08:43.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619133+0000", 2024-09-16T00:08:43.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.359e-05 2024-09-16T00:08:43.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619179+0000", 2024-09-16T00:08:43.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5459e-05 2024-09-16T00:08:43.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619186+0000", 2024-09-16T00:08:43.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0480000000000001e-06 2024-09-16T00:08:43.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.864 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619293+0000", 2024-09-16T00:08:43.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000107168 2024-09-16T00:08:43.864 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.864 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.864 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.864 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:22 2.e e22/19 ondisk, result = 0)", 2024-09-16T00:08:43.864 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:43.864 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.181664298000001, 2024-09-16T00:08:43.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020058200000000001, 2024-09-16T00:08:43.864 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.864 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.864 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:43.865 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:43.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:43.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:43.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0109999999999999e-06 2024-09-16T00:08:43.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620801+0000", 2024-09-16T00:08:43.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6779999999999999e-06 2024-09-16T00:08:43.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620814+0000", 2024-09-16T00:08:43.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3131e-05 2024-09-16T00:08:43.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620821+0000", 2024-09-16T00:08:43.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0430000000000002e-06 2024-09-16T00:08:43.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:43.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3010999999999997e-05 2024-09-16T00:08:43.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620891+0000", 2024-09-16T00:08:43.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1760000000000004e-06 2024-09-16T00:08:43.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620996+0000", 2024-09-16T00:08:43.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010453199999999999 2024-09-16T00:08:43.868 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.868 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.868 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.868 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:7 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:43.868 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:43.868 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.181656313999994, 2024-09-16T00:08:43.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000195725, 2024-09-16T00:08:43.868 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.868 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.868 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:43.869 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:43.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:43.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620804+0000", 2024-09-16T00:08:43.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6099999999999999e-07 2024-09-16T00:08:43.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620809+0000", 2024-09-16T00:08:43.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4950000000000002e-06 2024-09-16T00:08:43.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:43.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0553e-05 2024-09-16T00:08:43.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620826+0000", 2024-09-16T00:08:43.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5579999999999997e-06 2024-09-16T00:08:43.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620874+0000", 2024-09-16T00:08:43.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7648999999999997e-05 2024-09-16T00:08:43.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620880+0000", 2024-09-16T00:08:43.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8290000000000002e-06 2024-09-16T00:08:43.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620999+0000", 2024-09-16T00:08:43.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011878 2024-09-16T00:08:43.872 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.872 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.872 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.872 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:8 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:43.872 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:43.872 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.181604254000007, 2024-09-16T00:08:43.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00027991099999999998, 2024-09-16T00:08:43.872 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.872 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.872 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:43.873 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:43.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:43.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620856+0000", 2024-09-16T00:08:43.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6800000000000001e-07 2024-09-16T00:08:43.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620859+0000", 2024-09-16T00:08:43.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6879999999999999e-06 2024-09-16T00:08:43.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620867+0000", 2024-09-16T00:08:43.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2819999999999996e-06 2024-09-16T00:08:43.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620872+0000", 2024-09-16T00:08:43.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5369999999999997e-06 2024-09-16T00:08:43.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621043+0000", 2024-09-16T00:08:43.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170791 2024-09-16T00:08:43.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621050+0000", 2024-09-16T00:08:43.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8580000000000002e-06 2024-09-16T00:08:43.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.876 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.876 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621135+0000", 2024-09-16T00:08:43.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5387000000000001e-05 2024-09-16T00:08:43.876 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.876 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.876 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.876 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:9 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:43.876 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:43.876 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.181575675999994, 2024-09-16T00:08:43.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00039143799999999997, 2024-09-16T00:08:43.876 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.876 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.877 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:43.877 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.877 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.877 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:43.877 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.877 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.877 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:43.877 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.877 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:43.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6800000000000002e-07 2024-09-16T00:08:43.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:08:43.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0310000000000001e-06 2024-09-16T00:08:43.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620892+0000", 2024-09-16T00:08:43.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9390000000000002e-06 2024-09-16T00:08:43.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620898+0000", 2024-09-16T00:08:43.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4480000000000002e-06 2024-09-16T00:08:43.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621167+0000", 2024-09-16T00:08:43.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026960599999999998 2024-09-16T00:08:43.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621174+0000", 2024-09-16T00:08:43.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.567e-06 2024-09-16T00:08:43.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621275+0000", 2024-09-16T00:08:43.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101579 2024-09-16T00:08:43.880 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.880 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.880 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.880 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:10 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:43.880 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:43.880 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.181553227999999, 2024-09-16T00:08:43.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00049416200000000003, 2024-09-16T00:08:43.880 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.880 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.881 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:43.881 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:43.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:43.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620907+0000", 2024-09-16T00:08:43.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.42e-07 2024-09-16T00:08:43.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620908+0000", 2024-09-16T00:08:43.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4300000000000001e-06 2024-09-16T00:08:43.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620911+0000", 2024-09-16T00:08:43.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3890000000000001e-06 2024-09-16T00:08:43.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620914+0000", 2024-09-16T00:08:43.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3690000000000001e-06 2024-09-16T00:08:43.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621297+0000", 2024-09-16T00:08:43.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038319200000000003 2024-09-16T00:08:43.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621305+0000", 2024-09-16T00:08:43.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4780000000000008e-06 2024-09-16T00:08:43.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.884 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621401+0000", 2024-09-16T00:08:43.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5161999999999995e-05 2024-09-16T00:08:43.884 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.884 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.884 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.884 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:11 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:43.884 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:43.885 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.181150478999996, 2024-09-16T00:08:43.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017943899999999999, 2024-09-16T00:08:43.885 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.885 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.885 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:43.885 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:43.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:43.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621310+0000", 2024-09-16T00:08:43.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5199999999999996e-07 2024-09-16T00:08:43.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:43.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1740000000000002e-06 2024-09-16T00:08:43.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621324+0000", 2024-09-16T00:08:43.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7170000000000003e-06 2024-09-16T00:08:43.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621331+0000", 2024-09-16T00:08:43.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2280000000000001e-06 2024-09-16T00:08:43.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621422+0000", 2024-09-16T00:08:43.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1052999999999997e-05 2024-09-16T00:08:43.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621426+0000", 2024-09-16T00:08:43.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.036e-06 2024-09-16T00:08:43.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621489+0000", 2024-09-16T00:08:43.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2478999999999996e-05 2024-09-16T00:08:43.888 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.888 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.888 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.888 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:12 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:43.888 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:43.889 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.181099623999998, 2024-09-16T00:08:43.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022422300000000001, 2024-09-16T00:08:43.889 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.889 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.889 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:43.889 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:43.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:43.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:43.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6699999999999999e-07 2024-09-16T00:08:43.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621363+0000", 2024-09-16T00:08:43.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7020000000000001e-06 2024-09-16T00:08:43.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621371+0000", 2024-09-16T00:08:43.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7209999999999994e-06 2024-09-16T00:08:43.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621376+0000", 2024-09-16T00:08:43.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7940000000000002e-06 2024-09-16T00:08:43.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621511+0000", 2024-09-16T00:08:43.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013518100000000001 2024-09-16T00:08:43.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621514+0000", 2024-09-16T00:08:43.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5599999999999998e-06 2024-09-16T00:08:43.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.892 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621584+0000", 2024-09-16T00:08:43.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9898000000000001e-05 2024-09-16T00:08:43.892 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.892 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.892 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.892 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:13 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:43.892 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:43.893 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.181073359999999, 2024-09-16T00:08:43.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026534099999999999, 2024-09-16T00:08:43.893 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.893 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.893 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:43.893 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:43.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:43.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621387+0000", 2024-09-16T00:08:43.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4299999999999999e-07 2024-09-16T00:08:43.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621389+0000", 2024-09-16T00:08:43.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.593e-06 2024-09-16T00:08:43.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621395+0000", 2024-09-16T00:08:43.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.553e-06 2024-09-16T00:08:43.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:08:43.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8410000000000002e-06 2024-09-16T00:08:43.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621597+0000", 2024-09-16T00:08:43.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019794100000000001 2024-09-16T00:08:43.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621600+0000", 2024-09-16T00:08:43.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.185e-06 2024-09-16T00:08:43.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.896 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621652+0000", 2024-09-16T00:08:43.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1984999999999998e-05 2024-09-16T00:08:43.896 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.896 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.896 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.896 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:23 2.e e22/19 ondisk, result = 0)", 2024-09-16T00:08:43.896 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:43.897 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.180590410999997, 2024-09-16T00:08:43.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015895699999999999, 2024-09-16T00:08:43.897 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:43.897 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:43.897 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:43.897 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:43.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:43.897 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:43.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:43.897 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:43.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:43.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:43.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621870+0000", 2024-09-16T00:08:43.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1600000000000001e-07 2024-09-16T00:08:43.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:43.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621873+0000", 2024-09-16T00:08:43.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3079999999999999e-06 2024-09-16T00:08:43.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:43.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621882+0000", 2024-09-16T00:08:43.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8429999999999999e-06 2024-09-16T00:08:43.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:43.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621896+0000", 2024-09-16T00:08:43.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:08:43.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:43.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621944+0000", 2024-09-16T00:08:43.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8062999999999997e-05 2024-09-16T00:08:43.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:43.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621951+0000", 2024-09-16T00:08:43.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7689999999999999e-06 2024-09-16T00:08:43.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:43.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:43.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:43.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622028+0000", 2024-09-16T00:08:43.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7243999999999994e-05 2024-09-16T00:08:43.900 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.900 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.900 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.900 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:43.900 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:43.900 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:43.901 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:43.803+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:43.902 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:08:43.905 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:43.905+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:43.937 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T00:08:44.007 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:44.005+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:44.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.102+0000 7fb8358f1640 1 Processor -- start 2024-09-16T00:08:44.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.102+0000 7fb8358f1640 1 -- start start 2024-09-16T00:08:44.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.103+0000 7fb8358f1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb830068060 0x7fb83015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.103+0000 7fb8358f1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8300cc4b0 con 0x7fb830154c20 2024-09-16T00:08:44.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.104+0000 7fb82effd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb830154c20 0x7fb8300c5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45570/0 (socket says 172.21.15.139:45570) 2024-09-16T00:08:44.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.104+0000 7fb82effd640 1 -- 172.21.15.139:0/3850556517 learned_addr learned my addr 172.21.15.139:0/3850556517 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:44.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.104+0000 7fb82e7fc640 1 --2- 172.21.15.139:0/3850556517 >> v2:172.21.15.139:3300/0 conn(0x7fb830068060 0x7fb83015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.104+0000 7fb8358f1640 1 -- 172.21.15.139:0/3850556517 --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8300cc790 con 0x7fb8300ccc90 2024-09-16T00:08:44.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.104+0000 7fb8358f1640 1 -- 172.21.15.139:0/3850556517 --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb8300c5830 con 0x7fb830068060 2024-09-16T00:08:44.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.104+0000 7fb82dffb640 1 -- 172.21.15.139:0/3850556517 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 601849022 0 0) 0x7fb8300cc4b0 con 0x7fb830154c20 2024-09-16T00:08:44.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.104+0000 7fb82dffb640 1 -- 172.21.15.139:0/3850556517 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb818003520 con 0x7fb830154c20 2024-09-16T00:08:44.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.104+0000 7fb82e7fc640 1 -- 172.21.15.139:0/3850556517 >> v1:172.21.15.164:6789/0 conn(0x7fb8300ccc90 legacy=0x7fb830165180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.104+0000 7fb82e7fc640 1 -- 172.21.15.139:0/3850556517 >> v1:172.21.15.139:6789/0 conn(0x7fb830154c20 legacy=0x7fb8300c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.104+0000 7fb82e7fc640 1 -- 172.21.15.139:0/3850556517 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83015ddf0 con 0x7fb830068060 2024-09-16T00:08:44.107 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:44.105+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.105+0000 7fb82e7fc640 1 --2- 172.21.15.139:0/3850556517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb830068060 0x7fb83015b280 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb824009790 tx=0x7fb82402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e781ccde5767083 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.105+0000 7fb82dffb640 1 -- 172.21.15.139:0/3850556517 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824037070 con 0x7fb830068060 2024-09-16T00:08:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.105+0000 7fb82dffb640 1 -- 172.21.15.139:0/3850556517 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb824032440 con 0x7fb830068060 2024-09-16T00:08:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.105+0000 7fb82dffb640 1 -- 172.21.15.139:0/3850556517 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824032810 con 0x7fb830068060 2024-09-16T00:08:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.106+0000 7fb8358f1640 1 -- 172.21.15.139:0/3850556517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb830068060 msgr2=0x7fb83015b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.106+0000 7fb8358f1640 1 --2- 172.21.15.139:0/3850556517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb830068060 0x7fb83015b280 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb824009790 tx=0x7fb82402a230 comp rx=0 tx=0).stop 2024-09-16T00:08:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.107+0000 7fb8358f1640 1 -- 172.21.15.139:0/3850556517 shutdown_connections 2024-09-16T00:08:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.107+0000 7fb8358f1640 1 --2- 172.21.15.139:0/3850556517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb830068060 0x7fb83015b280 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.107+0000 7fb8358f1640 1 -- 172.21.15.139:0/3850556517 >> 172.21.15.139:0/3850556517 conn(0x7fb83006ca40 msgr2=0x7fb830067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:44.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.107+0000 7fb8358f1640 1 -- 172.21.15.139:0/3850556517 shutdown_connections 2024-09-16T00:08:44.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.107+0000 7fb8358f1640 1 -- 172.21.15.139:0/3850556517 wait complete. 2024-09-16T00:08:44.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.108+0000 7fb8358f1640 1 Processor -- start 2024-09-16T00:08:44.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.109+0000 7fb8358f1640 1 -- start start 2024-09-16T00:08:44.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.110+0000 7fb8358f1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8300ccc90 0x7fb8301f0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.110+0000 7fb8358f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb830154c20 0x7fb83015c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.110+0000 7fb82effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8300ccc90 0x7fb8301f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.110+0000 7fb8358f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8300cb070 0x7fb8300cf450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.110+0000 7fb8358f1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb8300cc260 con 0x7fb8300ccc90 2024-09-16T00:08:44.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.110+0000 7fb8358f1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb83015e3b0 con 0x7fb8300cb070 2024-09-16T00:08:44.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.110+0000 7fb8358f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb83015e690 con 0x7fb830154c20 2024-09-16T00:08:44.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.110+0000 7fb82effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8300ccc90 0x7fb8301f0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49392/0 (socket says 172.21.15.139:49392) 2024-09-16T00:08:44.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.110+0000 7fb82effd640 1 -- 172.21.15.139:0/3416015283 learned_addr learned my addr 172.21.15.139:0/3416015283 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:44.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.110+0000 7fb82f7fe640 1 --2- 172.21.15.139:0/3416015283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8300cb070 0x7fb8300cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.110+0000 7fb82effd640 1 -- 172.21.15.139:0/3416015283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb830154c20 msgr2=0x7fb83015c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.110+0000 7fb82effd640 1 --2- 172.21.15.139:0/3416015283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb830154c20 0x7fb83015c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.110+0000 7fb82effd640 1 -- 172.21.15.139:0/3416015283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8300cb070 msgr2=0x7fb8300cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.110+0000 7fb82effd640 1 --2- 172.21.15.139:0/3416015283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8300cb070 0x7fb8300cf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.110+0000 7fb82effd640 1 -- 172.21.15.139:0/3416015283 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83014ef60 con 0x7fb8300ccc90 2024-09-16T00:08:44.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.111+0000 7fb82effd640 1 --2- 172.21.15.139:0/3416015283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8300ccc90 0x7fb8301f0720 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb82000c440 tx=0x7fb82000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:44.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.111+0000 7fb817fff640 1 -- 172.21.15.139:0/3416015283 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb820017070 con 0x7fb8300ccc90 2024-09-16T00:08:44.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.111+0000 7fb8358f1640 1 -- 172.21.15.139:0/3416015283 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83014f200 con 0x7fb8300ccc90 2024-09-16T00:08:44.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.111+0000 7fb8358f1640 1 -- 172.21.15.139:0/3416015283 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb83015bd70 con 0x7fb8300ccc90 2024-09-16T00:08:44.116 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:44.116 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:44.116 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:44.116 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:44.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.111+0000 7fb817fff640 1 -- 172.21.15.139:0/3416015283 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb820005ce0 con 0x7fb8300ccc90 2024-09-16T00:08:44.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.112+0000 7fb817fff640 1 -- 172.21.15.139:0/3416015283 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8200072d0 con 0x7fb8300ccc90 2024-09-16T00:08:44.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.113+0000 7fb8358f1640 1 -- 172.21.15.139:0/3416015283 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb810005290 con 0x7fb8300ccc90 2024-09-16T00:08:44.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.114+0000 7fb817fff640 1 -- 172.21.15.139:0/3416015283 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb82001a070 con 0x7fb8300ccc90 2024-09-16T00:08:44.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.114+0000 7fb817fff640 1 --2- 172.21.15.139:0/3416015283 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb81c06c2e0 0x7fb81c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.114+0000 7fb817fff640 1 -- 172.21.15.139:0/3416015283 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5915+0+0 (secure 0 0 0) 0x7fb82008f060 con 0x7fb8300ccc90 2024-09-16T00:08:44.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.115+0000 7fb82e7fc640 1 --2- 172.21.15.139:0/3416015283 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb81c06c2e0 0x7fb81c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.115+0000 7fb82e7fc640 1 --2- 172.21.15.139:0/3416015283 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb81c06c2e0 0x7fb81c06e7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb824034000 tx=0x7fb82402c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:44.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.124+0000 7fb817fff640 1 -- 172.21.15.139:0/3416015283 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb820058d20 con 0x7fb8300ccc90 2024-09-16T00:08:44.127 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:08:44.208 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:44.207+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:44.278 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:44.279 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:44.279 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:44.279 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:44.279 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:44.288 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:08:44.309 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:44.307+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:44.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.335+0000 7fb8358f1640 1 -- 172.21.15.139:0/3416015283 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fb810005850 con 0x7fb8300ccc90 2024-09-16T00:08:44.346 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:08:44.404 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:44.404 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:44.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:44.404 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:44.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.404 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:22 2.e e22/19 2:717a0223:::608.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:44.405 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:44.405 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.785341219000003, 2024-09-16T00:08:44.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042195870000000003, 2024-09-16T00:08:44.405 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.405 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.405 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:44.405 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:44.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:44.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616426+0000", 2024-09-16T00:08:44.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.037e-06 2024-09-16T00:08:44.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616435+0000", 2024-09-16T00:08:44.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6440000000000006e-06 2024-09-16T00:08:44.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616447+0000", 2024-09-16T00:08:44.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2466000000000001e-05 2024-09-16T00:08:44.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616456+0000", 2024-09-16T00:08:44.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8799999999999997e-06 2024-09-16T00:08:44.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616517+0000", 2024-09-16T00:08:44.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1171000000000001e-05 2024-09-16T00:08:44.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:44.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616540+0000", 2024-09-16T00:08:44.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2991000000000001e-05 2024-09-16T00:08:44.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:44.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620599+0000", 2024-09-16T00:08:44.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040586140000000003 2024-09-16T00:08:44.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620645+0000", 2024-09-16T00:08:44.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5784000000000001e-05 2024-09-16T00:08:44.411 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.411 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.411 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.411 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:44.411 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:44.411 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.785251489000004, 2024-09-16T00:08:44.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044836249999999998, 2024-09-16T00:08:44.412 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.412 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.412 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:44.412 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:44.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:44.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616518+0000", 2024-09-16T00:08:44.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3699999999999999e-06 2024-09-16T00:08:44.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:08:44.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1479999999999997e-06 2024-09-16T00:08:44.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616534+0000", 2024-09-16T00:08:44.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0641e-05 2024-09-16T00:08:44.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616541+0000", 2024-09-16T00:08:44.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3339999999999996e-06 2024-09-16T00:08:44.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617318+0000", 2024-09-16T00:08:44.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00077694799999999996 2024-09-16T00:08:44.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:44.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617409+0000", 2024-09-16T00:08:44.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0740000000000002e-05 2024-09-16T00:08:44.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620998+0000", 2024-09-16T00:08:44.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035894439999999998 2024-09-16T00:08:44.417 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.417 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.417 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.417 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:44.417 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:44.418 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.785203585999994, 2024-09-16T00:08:44.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041597320000000002, 2024-09-16T00:08:44.418 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.418 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.418 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:44.418 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:44.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:44.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:44.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:08:44.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616566+0000", 2024-09-16T00:08:44.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6979999999999998e-06 2024-09-16T00:08:44.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616574+0000", 2024-09-16T00:08:44.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7759999999999997e-06 2024-09-16T00:08:44.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616580+0000", 2024-09-16T00:08:44.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:44.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616800+0000", 2024-09-16T00:08:44.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022016700000000001 2024-09-16T00:08:44.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:44.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616904+0000", 2024-09-16T00:08:44.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104579 2024-09-16T00:08:44.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620722+0000", 2024-09-16T00:08:44.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003818133 2024-09-16T00:08:44.422 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.422 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.422 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.422 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:44.422 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:44.422 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.784952900999997, 2024-09-16T00:08:44.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038451570000000001, 2024-09-16T00:08:44.422 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.422 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.422 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:44.422 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:44.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:44.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616814+0000", 2024-09-16T00:08:44.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9299999999999997e-07 2024-09-16T00:08:44.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616824+0000", 2024-09-16T00:08:44.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6600000000000007e-06 2024-09-16T00:08:44.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616835+0000", 2024-09-16T00:08:44.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0713000000000001e-05 2024-09-16T00:08:44.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616841+0000", 2024-09-16T00:08:44.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.934e-06 2024-09-16T00:08:44.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616948+0000", 2024-09-16T00:08:44.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010631100000000001 2024-09-16T00:08:44.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:44.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616967+0000", 2024-09-16T00:08:44.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9598000000000001e-05 2024-09-16T00:08:44.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:44.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620619+0000", 2024-09-16T00:08:44.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036515810000000001 2024-09-16T00:08:44.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620659+0000", 2024-09-16T00:08:44.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9666999999999998e-05 2024-09-16T00:08:44.426 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.426 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.427 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.427 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:44.427 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:44.427 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.784905949000006, 2024-09-16T00:08:44.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038240129999999998, 2024-09-16T00:08:44.427 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.427 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.427 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:44.427 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:44.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:44.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616861+0000", 2024-09-16T00:08:44.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0200000000000003e-07 2024-09-16T00:08:44.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616864+0000", 2024-09-16T00:08:44.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5090000000000002e-06 2024-09-16T00:08:44.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616871+0000", 2024-09-16T00:08:44.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9249999999999998e-06 2024-09-16T00:08:44.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616877+0000", 2024-09-16T00:08:44.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.981e-06 2024-09-16T00:08:44.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617168+0000", 2024-09-16T00:08:44.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029035699999999998 2024-09-16T00:08:44.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:44.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617184+0000", 2024-09-16T00:08:44.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6005999999999999e-05 2024-09-16T00:08:44.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:44.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620661+0000", 2024-09-16T00:08:44.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034775330000000001 2024-09-16T00:08:44.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620684+0000", 2024-09-16T00:08:44.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3300000000000001e-05 2024-09-16T00:08:44.431 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.431 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.431 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.431 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:44.431 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:44.431 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.784450505999999, 2024-09-16T00:08:44.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045415610000000004, 2024-09-16T00:08:44.432 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.432 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.432 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:44.432 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:44.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:44.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:44.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8800000000000002e-07 2024-09-16T00:08:44.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617321+0000", 2024-09-16T00:08:44.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6979999999999997e-06 2024-09-16T00:08:44.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617330+0000", 2024-09-16T00:08:44.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6729999999999998e-06 2024-09-16T00:08:44.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617336+0000", 2024-09-16T00:08:44.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2110000000000003e-06 2024-09-16T00:08:44.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617394+0000", 2024-09-16T00:08:44.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8087999999999997e-05 2024-09-16T00:08:44.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:44.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617412+0000", 2024-09-16T00:08:44.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8099999999999999e-05 2024-09-16T00:08:44.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:44.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621818+0000", 2024-09-16T00:08:44.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044061209999999998 2024-09-16T00:08:44.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621857+0000", 2024-09-16T00:08:44.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9082000000000003e-05 2024-09-16T00:08:44.436 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.436 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.436 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.436 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:23 2.e e22/19 2:717a0223:::608.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:44.436 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:44.436 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.783554484999996, 2024-09-16T00:08:44.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036198279999999998, 2024-09-16T00:08:44.436 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.436 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.436 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:44.437 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:44.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:44.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618213+0000", 2024-09-16T00:08:44.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:08:44.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618218+0000", 2024-09-16T00:08:44.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4099999999999999e-06 2024-09-16T00:08:44.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618227+0000", 2024-09-16T00:08:44.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5009999999999993e-06 2024-09-16T00:08:44.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618234+0000", 2024-09-16T00:08:44.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:44.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618307+0000", 2024-09-16T00:08:44.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3421000000000001e-05 2024-09-16T00:08:44.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:44.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618328+0000", 2024-09-16T00:08:44.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0919000000000001e-05 2024-09-16T00:08:44.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:44.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621804+0000", 2024-09-16T00:08:44.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034756560000000001 2024-09-16T00:08:44.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621832+0000", 2024-09-16T00:08:44.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7966999999999998e-05 2024-09-16T00:08:44.441 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.441 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.441 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.441 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing_reply(2.1e epoch 22/19 rep_tid 0 lcod 22'1)", 2024-09-16T00:08:44.441 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:44.441 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.783109316999997, 2024-09-16T00:08:44.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1371000000000003e-05, 2024-09-16T00:08:44.442 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.442 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.442 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:44.442 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:44.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:44.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:08:44.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:08:44.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618662+0000", 2024-09-16T00:08:44.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1189999999999999e-06 2024-09-16T00:08:44.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618674+0000", 2024-09-16T00:08:44.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2317000000000001e-05 2024-09-16T00:08:44.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618681+0000", 2024-09-16T00:08:44.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7170000000000002e-06 2024-09-16T00:08:44.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:08:44.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6745000000000001e-05 2024-09-16T00:08:44.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618748+0000", 2024-09-16T00:08:44.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0794000000000001e-05 2024-09-16T00:08:44.447 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.447 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.447 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.447 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:44.447 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:44.447 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.782595982999993, 2024-09-16T00:08:44.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027183849999999998, 2024-09-16T00:08:44.448 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.448 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.448 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:44.448 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:44.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:44.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619171+0000", 2024-09-16T00:08:44.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8999999999999995e-07 2024-09-16T00:08:44.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619179+0000", 2024-09-16T00:08:44.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2679999999999994e-06 2024-09-16T00:08:44.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:08:44.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1246e-05 2024-09-16T00:08:44.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619197+0000", 2024-09-16T00:08:44.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2820000000000002e-06 2024-09-16T00:08:44.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619308+0000", 2024-09-16T00:08:44.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011141000000000001 2024-09-16T00:08:44.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:44.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619325+0000", 2024-09-16T00:08:44.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6469000000000001e-05 2024-09-16T00:08:44.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:44.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621860+0000", 2024-09-16T00:08:44.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025353590000000001 2024-09-16T00:08:44.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621889+0000", 2024-09-16T00:08:44.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8561e-05 2024-09-16T00:08:44.452 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.452 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.452 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.452 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 6 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:44.452 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:44.452 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.782553284000002, 2024-09-16T00:08:44.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026903809999999999, 2024-09-16T00:08:44.452 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.453 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.453 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:44.453 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:44.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:44.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:44.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2000000000000001e-07 2024-09-16T00:08:44.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619219+0000", 2024-09-16T00:08:44.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:08:44.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619234+0000", 2024-09-16T00:08:44.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5075000000000001e-05 2024-09-16T00:08:44.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619251+0000", 2024-09-16T00:08:44.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6863000000000001e-05 2024-09-16T00:08:44.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619556+0000", 2024-09-16T00:08:44.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030551599999999998 2024-09-16T00:08:44.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621903+0000", 2024-09-16T00:08:44.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023469979999999999 2024-09-16T00:08:44.456 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.456 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.456 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.456 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing_reply(2.1b epoch 22/19 rep_tid 1 lcod 22'1)", 2024-09-16T00:08:44.456 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:44.456 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.781474888000005, 2024-09-16T00:08:44.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6989000000000003e-05, 2024-09-16T00:08:44.457 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.457 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.457 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:44.457 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:44.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:44.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620292+0000", 2024-09-16T00:08:44.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.37e-07 2024-09-16T00:08:44.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620296+0000", 2024-09-16T00:08:44.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2420000000000002e-06 2024-09-16T00:08:44.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620308+0000", 2024-09-16T00:08:44.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1623e-05 2024-09-16T00:08:44.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620313+0000", 2024-09-16T00:08:44.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6290000000000001e-06 2024-09-16T00:08:44.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620359+0000", 2024-09-16T00:08:44.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6201e-05 2024-09-16T00:08:44.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620378+0000", 2024-09-16T00:08:44.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9556999999999999e-05 2024-09-16T00:08:44.461 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.461 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.461 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.461 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:38 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'6, mlcod=22'4)", 2024-09-16T00:08:44.461 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:44.461 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.779032638999993, 2024-09-16T00:08:44.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064551599999999995, 2024-09-16T00:08:44.461 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.461 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.461 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:44.461 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:44.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:44.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:44.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4399999999999999e-07 2024-09-16T00:08:44.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.463 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622739+0000", 2024-09-16T00:08:44.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0250000000000002e-06 2024-09-16T00:08:44.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.463 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622749+0000", 2024-09-16T00:08:44.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1220000000000001e-06 2024-09-16T00:08:44.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622760+0000", 2024-09-16T00:08:44.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1640999999999999e-05 2024-09-16T00:08:44.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622799+0000", 2024-09-16T00:08:44.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8537000000000001e-05 2024-09-16T00:08:44.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:44.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622810+0000", 2024-09-16T00:08:44.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1562e-05 2024-09-16T00:08:44.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:44.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.623356+0000", 2024-09-16T00:08:44.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054522299999999995 2024-09-16T00:08:44.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.623379+0000", 2024-09-16T00:08:44.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3662000000000001e-05 2024-09-16T00:08:44.466 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.466 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.466 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.467 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:40 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'7, mlcod=22'5)", 2024-09-16T00:08:44.467 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:44.467 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 53.562011736999999, 2024-09-16T00:08:44.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001017773, 2024-09-16T00:08:44.467 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.467 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.467 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:44.467 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:44.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:44.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839756+0000", 2024-09-16T00:08:44.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.491e-06 2024-09-16T00:08:44.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839768+0000", 2024-09-16T00:08:44.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1649e-05 2024-09-16T00:08:44.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839783+0000", 2024-09-16T00:08:44.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5023e-05 2024-09-16T00:08:44.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839791+0000", 2024-09-16T00:08:44.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5620000000000004e-06 2024-09-16T00:08:44.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839848+0000", 2024-09-16T00:08:44.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6563e-05 2024-09-16T00:08:44.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:44.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839868+0000", 2024-09-16T00:08:44.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9652000000000001e-05 2024-09-16T00:08:44.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:44.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.840729+0000", 2024-09-16T00:08:44.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086159500000000005 2024-09-16T00:08:44.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.840772+0000", 2024-09-16T00:08:44.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3238000000000002e-05 2024-09-16T00:08:44.471 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.471 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.471 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.471 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:41 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'8, mlcod=22'6)", 2024-09-16T00:08:44.471 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:44.471 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.288354826000003, 2024-09-16T00:08:44.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001040568, 2024-09-16T00:08:44.472 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.472 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.472 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:44.472 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:44.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:44.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113413+0000", 2024-09-16T00:08:44.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7409999999999999e-06 2024-09-16T00:08:44.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113428+0000", 2024-09-16T00:08:44.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5007999999999999e-05 2024-09-16T00:08:44.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113447+0000", 2024-09-16T00:08:44.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8731000000000001e-05 2024-09-16T00:08:44.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113457+0000", 2024-09-16T00:08:44.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0095000000000001e-05 2024-09-16T00:08:44.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113526+0000", 2024-09-16T00:08:44.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8552000000000004e-05 2024-09-16T00:08:44.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:44.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113552+0000", 2024-09-16T00:08:44.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6795999999999999e-05 2024-09-16T00:08:44.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:44.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.114417+0000", 2024-09-16T00:08:44.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086476299999999995 2024-09-16T00:08:44.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.114452+0000", 2024-09-16T00:08:44.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4882000000000002e-05 2024-09-16T00:08:44.475 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.476 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.476 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.476 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.14 38/37 [PushReplyOp(2:2f55791f:::606.00000000:head)])", 2024-09-16T00:08:44.476 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.839962+0000", 2024-09-16T00:08:44.476 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.561804405, 2024-09-16T00:08:44.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030257, 2024-09-16T00:08:44.476 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.476 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.476 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:44.476 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839962+0000", 2024-09-16T00:08:44.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839962+0000", 2024-09-16T00:08:44.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839962+0000", 2024-09-16T00:08:44.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5400000000000001e-07 2024-09-16T00:08:44.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839967+0000", 2024-09-16T00:08:44.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.245e-06 2024-09-16T00:08:44.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839978+0000", 2024-09-16T00:08:44.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1388e-05 2024-09-16T00:08:44.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839987+0000", 2024-09-16T00:08:44.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1500000000000005e-06 2024-09-16T00:08:44.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.840043+0000", 2024-09-16T00:08:44.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5396000000000003e-05 2024-09-16T00:08:44.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.840265+0000", 2024-09-16T00:08:44.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022183700000000001 2024-09-16T00:08:44.482 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.482 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.483 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.483 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.14 38/37 [PushReplyOp(2:287034c4:::1.00000000.inode:head)])", 2024-09-16T00:08:44.483 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.848174+0000", 2024-09-16T00:08:44.483 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.553592199000001, 2024-09-16T00:08:44.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029495900000000002, 2024-09-16T00:08:44.484 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.484 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.484 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:44.484 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848174+0000", 2024-09-16T00:08:44.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848174+0000", 2024-09-16T00:08:44.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.486 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.486 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848175+0000", 2024-09-16T00:08:44.486 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.68e-07 2024-09-16T00:08:44.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.487 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848179+0000", 2024-09-16T00:08:44.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1400000000000002e-06 2024-09-16T00:08:44.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848191+0000", 2024-09-16T00:08:44.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1823e-05 2024-09-16T00:08:44.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848196+0000", 2024-09-16T00:08:44.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.626e-06 2024-09-16T00:08:44.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848251+0000", 2024-09-16T00:08:44.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4342999999999998e-05 2024-09-16T00:08:44.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848469+0000", 2024-09-16T00:08:44.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000218359 2024-09-16T00:08:44.491 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.491 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.491 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.491 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.14 38/37 [PushReplyOp(2:2e2fa760:::200.00000001:head)])", 2024-09-16T00:08:44.491 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.850960+0000", 2024-09-16T00:08:44.491 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.550805908000001, 2024-09-16T00:08:44.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00028471600000000002, 2024-09-16T00:08:44.492 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.492 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.492 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:44.492 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.492 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.492 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850960+0000", 2024-09-16T00:08:44.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.493 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.493 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850960+0000", 2024-09-16T00:08:44.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850961+0000", 2024-09-16T00:08:44.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6000000000000003e-07 2024-09-16T00:08:44.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850965+0000", 2024-09-16T00:08:44.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1400000000000002e-06 2024-09-16T00:08:44.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850977+0000", 2024-09-16T00:08:44.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2128e-05 2024-09-16T00:08:44.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.496 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.496 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850985+0000", 2024-09-16T00:08:44.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3220000000000004e-06 2024-09-16T00:08:44.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.497 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.851036+0000", 2024-09-16T00:08:44.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1100000000000002e-05 2024-09-16T00:08:44.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.498 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.498 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.851245+0000", 2024-09-16T00:08:44.498 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000209266 2024-09-16T00:08:44.498 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.498 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.498 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.498 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.498 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.14 38/37 [PushReplyOp(2:292cf221:::200.00000000:head)])", 2024-09-16T00:08:44.499 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.853964+0000", 2024-09-16T00:08:44.499 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.54780272, 2024-09-16T00:08:44.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026211799999999999, 2024-09-16T00:08:44.499 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:44.499 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:44.499 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:44.499 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:44.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:44.499 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853964+0000", 2024-09-16T00:08:44.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:44.500 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853964+0000", 2024-09-16T00:08:44.500 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:44.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.500 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:44.500 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853964+0000", 2024-09-16T00:08:44.500 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8000000000000002e-07 2024-09-16T00:08:44.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.500 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:44.500 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853967+0000", 2024-09-16T00:08:44.500 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6529999999999998e-06 2024-09-16T00:08:44.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.501 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:44.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853973+0000", 2024-09-16T00:08:44.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9110000000000004e-06 2024-09-16T00:08:44.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.501 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:44.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853981+0000", 2024-09-16T00:08:44.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.25e-06 2024-09-16T00:08:44.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.501 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:44.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.854030+0000", 2024-09-16T00:08:44.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8909000000000002e-05 2024-09-16T00:08:44.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:44.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:44.502 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:44.502 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.854226+0000", 2024-09-16T00:08:44.502 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019611499999999999 2024-09-16T00:08:44.502 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.502 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.502 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.502 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:44.502 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:44.502 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:44.504 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:08:44.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.484+0000 7f8d49d78640 1 Processor -- start 2024-09-16T00:08:44.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.485+0000 7f8d49d78640 1 -- start start 2024-09-16T00:08:44.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.485+0000 7f8d49d78640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8d440d0870 0x7f8d440b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.485+0000 7f8d49d78640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d440b3430 0x7f8d440b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.485+0000 7f8d49d78640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d4415adc0 con 0x7f8d440d0490 2024-09-16T00:08:44.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.485+0000 7f8d49d78640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8d4415b940 con 0x7f8d440d0870 2024-09-16T00:08:44.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.485+0000 7f8d49d78640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8d440b3bb0 con 0x7f8d440b3430 2024-09-16T00:08:44.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.486+0000 7f8d42ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8d440d0870 0x7f8d440b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.486+0000 7f8d42ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8d440d0870 0x7f8d440b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49406/0 (socket says 172.21.15.139:49406) 2024-09-16T00:08:44.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.486+0000 7f8d42ffd640 1 -- 172.21.15.139:0/1357235633 learned_addr learned my addr 172.21.15.139:0/1357235633 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:44.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.486+0000 7f8d43fff640 1 --2- 172.21.15.139:0/1357235633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d440b3430 0x7f8d440b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.486+0000 7f8d42ffd640 1 -- 172.21.15.139:0/1357235633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d440b3430 msgr2=0x7f8d440b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:44.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.486+0000 7f8d42ffd640 1 --2- 172.21.15.139:0/1357235633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d440b3430 0x7f8d440b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.486+0000 7f8d42ffd640 1 -- 172.21.15.139:0/1357235633 >> v1:172.21.15.164:6789/0 conn(0x7f8d440d0490 legacy=0x7f8d440b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.486+0000 7f8d42ffd640 1 -- 172.21.15.139:0/1357235633 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4415b0a0 con 0x7f8d440d0870 2024-09-16T00:08:44.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.487+0000 7f8d42ffd640 1 --2- 172.21.15.139:0/1357235633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d440d0870 0x7f8d440b4040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009870 tx=0x7f8d3802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a8584b0fd9493e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:44.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.487+0000 7f8d427fc640 1 -- 172.21.15.139:0/1357235633 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38037070 con 0x7f8d440d0870 2024-09-16T00:08:44.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.487+0000 7f8d427fc640 1 -- 172.21.15.139:0/1357235633 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d3802ae50 con 0x7f8d440d0870 2024-09-16T00:08:44.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.487+0000 7f8d427fc640 1 -- 172.21.15.139:0/1357235633 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d38032860 con 0x7f8d440d0870 2024-09-16T00:08:44.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.488+0000 7f8d49d78640 1 -- 172.21.15.139:0/1357235633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d440d0870 msgr2=0x7f8d440b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.488+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1357235633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d440d0870 0x7f8d440b4040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009870 tx=0x7f8d3802a300 comp rx=0 tx=0).stop 2024-09-16T00:08:44.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.488+0000 7f8d49d78640 1 -- 172.21.15.139:0/1357235633 shutdown_connections 2024-09-16T00:08:44.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.488+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1357235633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d440b3430 0x7f8d440b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.488+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1357235633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d440d0870 0x7f8d440b4040 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.488+0000 7f8d49d78640 1 -- 172.21.15.139:0/1357235633 >> 172.21.15.139:0/1357235633 conn(0x7f8d4406c520 msgr2=0x7f8d4406c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:44.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.488+0000 7f8d49d78640 1 -- 172.21.15.139:0/1357235633 shutdown_connections 2024-09-16T00:08:44.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.488+0000 7f8d49d78640 1 -- 172.21.15.139:0/1357235633 wait complete. 2024-09-16T00:08:44.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.490+0000 7f8d49d78640 1 Processor -- start 2024-09-16T00:08:44.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.490+0000 7f8d49d78640 1 -- start start 2024-09-16T00:08:44.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.491+0000 7f8d49d78640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d440b3430 0x7f8d4415d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.491+0000 7f8d49d78640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d440d0490 0x7f8d4415e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.491+0000 7f8d49d78640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d440d0870 0x7f8d4414ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.491+0000 7f8d49d78640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4415a980 con 0x7f8d440d0870 2024-09-16T00:08:44.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.491+0000 7f8d49d78640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4415b660 con 0x7f8d440d0490 2024-09-16T00:08:44.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.491+0000 7f8d49d78640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8d4415ab00 con 0x7f8d440b3430 2024-09-16T00:08:44.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.491+0000 7f8d43fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d440d0870 0x7f8d4414ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.491+0000 7f8d43fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d440d0870 0x7f8d4414ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49422/0 (socket says 172.21.15.139:49422) 2024-09-16T00:08:44.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.491+0000 7f8d43fff640 1 -- 172.21.15.139:0/1826063683 learned_addr learned my addr 172.21.15.139:0/1826063683 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:44.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.491+0000 7f8d42ffd640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d440d0490 0x7f8d4415e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.491+0000 7f8d437fe640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d440b3430 0x7f8d4415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.492+0000 7f8d43fff640 1 -- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d440b3430 msgr2=0x7f8d4415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.492+0000 7f8d43fff640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d440b3430 0x7f8d4415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.492+0000 7f8d43fff640 1 -- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d440d0490 msgr2=0x7f8d4415e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.492+0000 7f8d43fff640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d440d0490 0x7f8d4415e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.492+0000 7f8d43fff640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d44158280 con 0x7f8d440d0870 2024-09-16T00:08:44.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.492+0000 7f8d43fff640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d440d0870 0x7f8d4414ec10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c007e70 tx=0x7f8d3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:44.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.492+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c014620 con 0x7f8d440d0870 2024-09-16T00:08:44.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.492+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d3c019070 con 0x7f8d440d0870 2024-09-16T00:08:44.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.492+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c014a20 con 0x7f8d440d0870 2024-09-16T00:08:44.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.492+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4415bc10 con 0x7f8d440d0870 2024-09-16T00:08:44.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.492+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4415c1d0 con 0x7f8d440d0870 2024-09-16T00:08:44.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.493+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f8d44154fa0 con 0x7f8d440d0870 2024-09-16T00:08:44.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.494+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8d3c014bc0 con 0x7f8d440d0870 2024-09-16T00:08:44.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.495+0000 7f8d2bfff640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8d3006c2b0 0x7f8d3006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.495+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5915+0+0 (secure 0 0 0) 0x7f8d3c08bca0 con 0x7f8d440d0870 2024-09-16T00:08:44.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.495+0000 7f8d437fe640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8d3006c2b0 0x7f8d3006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.495+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v49) ==== 54+0+16 (secure 0 0 0) 0x7f8d44154fa0 con 0x7f8d440d0870 2024-09-16T00:08:44.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.495+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f8d440cb860 0x7f8d4415c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.495+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d441f0e20 con 0x7f8d440cb860 2024-09-16T00:08:44.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.496+0000 7f8d437fe640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8d3006c2b0 0x7f8d3006e770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8d34001780 tx=0x7f8d34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:44.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.496+0000 7f8d42ffd640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f8d440cb860 0x7f8d4415c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.496+0000 7f8d42ffd640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f8d440cb860 0x7f8d4415c3a0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:44.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.497+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d441f0e20 con 0x7f8d440cb860 2024-09-16T00:08:44.514 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:44.503+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:44.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.524+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8d441590a0 con 0x7f8d440cb860 2024-09-16T00:08:44.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.526+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d441590a0 con 0x7f8d440cb860 2024-09-16T00:08:44.528 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:44.529 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:44.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.527+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f8d440cbc40 0x7f8d44149980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.527+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f8d44149ec0 con 0x7f8d440cbc40 2024-09-16T00:08:44.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.527+0000 7f8d437fe640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f8d440cbc40 0x7f8d44149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.527+0000 7f8d437fe640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f8d440cbc40 0x7f8d44149980 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:44.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.528+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d44149ec0 con 0x7f8d440cbc40 2024-09-16T00:08:44.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.556+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8d440b2c40 con 0x7f8d440cbc40 2024-09-16T00:08:44.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.558+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d440b2c40 con 0x7f8d440cbc40 2024-09-16T00:08:44.560 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:44.561 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:44.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.558+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8d441f4530 0x7f8d441f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.558+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f8d4414a4b0 con 0x7f8d441f4530 2024-09-16T00:08:44.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.559+0000 7f8d42ffd640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8d441f4530 0x7f8d441f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.559+0000 7f8d42ffd640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8d441f4530 0x7f8d441f4970 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:44.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.560+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d4414a4b0 con 0x7f8d441f4530 2024-09-16T00:08:44.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.586+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8d4414a1d0 con 0x7f8d441f4530 2024-09-16T00:08:44.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.588+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d4414a1d0 con 0x7f8d441f4530 2024-09-16T00:08:44.590 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:44.591 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:44.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.589+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8d44062b40 0x7f8d44062fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.589+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f8d44151510 con 0x7f8d44062b40 2024-09-16T00:08:44.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.589+0000 7f8d43fff640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8d44062b40 0x7f8d44062fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.589+0000 7f8d43fff640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8d44062b40 0x7f8d44062fa0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:44.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.590+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d44151510 con 0x7f8d44062b40 2024-09-16T00:08:44.604 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:44.603+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:44.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.617+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8d44151510 con 0x7f8d44062b40 2024-09-16T00:08:44.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.619+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d44151510 con 0x7f8d44062b40 2024-09-16T00:08:44.622 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:44.623 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:44.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.620+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f8d44152640 0x7f8d44152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.620+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f8d44153040 con 0x7f8d44152640 2024-09-16T00:08:44.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.620+0000 7f8d437fe640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f8d44152640 0x7f8d44152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.620+0000 7f8d437fe640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f8d44152640 0x7f8d44152a80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:44.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.621+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d44153040 con 0x7f8d44152640 2024-09-16T00:08:44.631 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:44.631 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:44.631 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:44.632 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:44.641 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:08:44.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.648+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8d44153040 con 0x7f8d44152640 2024-09-16T00:08:44.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.650+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d44153040 con 0x7f8d44152640 2024-09-16T00:08:44.652 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:44.653 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:44.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.650+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f8d442058b0 0x7f8d44205d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.650+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f8d44209700 con 0x7f8d442058b0 2024-09-16T00:08:44.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.650+0000 7f8d42ffd640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f8d442058b0 0x7f8d44205d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.651+0000 7f8d42ffd640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f8d442058b0 0x7f8d44205d10 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:44.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.651+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d44209700 con 0x7f8d442058b0 2024-09-16T00:08:44.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.678+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8d4402c640 con 0x7f8d442058b0 2024-09-16T00:08:44.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.679+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d4402c640 con 0x7f8d442058b0 2024-09-16T00:08:44.681 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:44.682 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:44.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.680+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f8d4402cd60 0x7f8d4402d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.680+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f8d4402d810 con 0x7f8d4402cd60 2024-09-16T00:08:44.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.680+0000 7f8d43fff640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f8d4402cd60 0x7f8d4402d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.680+0000 7f8d43fff640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f8d4402cd60 0x7f8d4402d1c0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:44.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.681+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d4402d810 con 0x7f8d4402cd60 2024-09-16T00:08:44.705 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:44.703+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:44.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.707+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8d4402d810 con 0x7f8d4402cd60 2024-09-16T00:08:44.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.710+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d4402d810 con 0x7f8d4402cd60 2024-09-16T00:08:44.712 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:44.712 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:44.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.710+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8d4420aaa0 0x7f8d4420ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:44.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.710+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f8d4402dba0 con 0x7f8d4420aaa0 2024-09-16T00:08:44.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.710+0000 7f8d437fe640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8d4420aaa0 0x7f8d4420ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:44.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.711+0000 7f8d437fe640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8d4420aaa0 0x7f8d4420ce90 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:44.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.712+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d4402dba0 con 0x7f8d4420aaa0 2024-09-16T00:08:44.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.738+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8d4420d3d0 con 0x7f8d4420aaa0 2024-09-16T00:08:44.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.740+0000 7f8d2bfff640 1 -- 172.21.15.139:0/1826063683 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d4420d3d0 con 0x7f8d4420aaa0 2024-09-16T00:08:44.742 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:44.744 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:44.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f8d440cb860 msgr2=0x7f8d4415c3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f8d440cb860 0x7f8d4415c3a0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f8d440cbc40 msgr2=0x7f8d44149980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f8d440cbc40 0x7f8d44149980 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8d441f4530 msgr2=0x7f8d441f4970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8d441f4530 0x7f8d441f4970 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8d44062b40 msgr2=0x7f8d44062fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8d44062b40 0x7f8d44062fa0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f8d44152640 msgr2=0x7f8d44152a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f8d44152640 0x7f8d44152a80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f8d442058b0 msgr2=0x7f8d44205d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f8d442058b0 0x7f8d44205d10 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f8d4402cd60 msgr2=0x7f8d4402d1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f8d4402cd60 0x7f8d4402d1c0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8d4420aaa0 msgr2=0x7f8d4420ce90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8d4420aaa0 0x7f8d4420ce90 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d43fff640 1 -- 172.21.15.139:0/1826063683 reap_dead start 2024-09-16T00:08:44.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8d3006c2b0 msgr2=0x7f8d3006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8d3006c2b0 0x7f8d3006e770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8d34001780 tx=0x7f8d34009040 comp rx=0 tx=0).stop 2024-09-16T00:08:44.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d43fff640 1 -- 172.21.15.139:0/1826063683 reap_dead start 2024-09-16T00:08:44.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d440d0870 msgr2=0x7f8d4414ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 --2- 172.21.15.139:0/1826063683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d440d0870 0x7f8d4414ec10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c007e70 tx=0x7f8d3c00e4b0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d43fff640 1 -- 172.21.15.139:0/1826063683 reap_dead start 2024-09-16T00:08:44.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d43fff640 1 -- 172.21.15.139:0/1826063683 reap_dead start 2024-09-16T00:08:44.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d43fff640 1 -- 172.21.15.139:0/1826063683 reap_dead start 2024-09-16T00:08:44.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.741+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 shutdown_connections 2024-09-16T00:08:44.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.742+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 >> 172.21.15.139:0/1826063683 conn(0x7f8d4406c520 msgr2=0x7f8d441587f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:44.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.742+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 shutdown_connections 2024-09-16T00:08:44.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.742+0000 7f8d49d78640 1 -- 172.21.15.139:0/1826063683 wait complete. 2024-09-16T00:08:44.750 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:44.750 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:44.759 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:08:44.806 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:44.805+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:44.864 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:44.864 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:44.873 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:08:44.907 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:44.906+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:44.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.931+0000 7fb817fff640 1 -- 172.21.15.139:0/3416015283 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v50) ==== 97+0+0 (secure 0 0 0) 0x7fb820058620 con 0x7fb8300ccc90 2024-09-16T00:08:44.934 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is set 2024-09-16T00:08:44.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.935+0000 7fb8358f1640 1 -- 172.21.15.139:0/3416015283 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb81c06c2e0 msgr2=0x7fb81c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.935+0000 7fb8358f1640 1 --2- 172.21.15.139:0/3416015283 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb81c06c2e0 0x7fb81c06e7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb824034000 tx=0x7fb82402c000 comp rx=0 tx=0).stop 2024-09-16T00:08:44.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.935+0000 7fb8358f1640 1 -- 172.21.15.139:0/3416015283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8300ccc90 msgr2=0x7fb8301f0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:44.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.935+0000 7fb8358f1640 1 --2- 172.21.15.139:0/3416015283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8300ccc90 0x7fb8301f0720 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb82000c440 tx=0x7fb82000c900 comp rx=0 tx=0).stop 2024-09-16T00:08:44.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.935+0000 7fb8358f1640 1 -- 172.21.15.139:0/3416015283 shutdown_connections 2024-09-16T00:08:44.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.935+0000 7fb8358f1640 1 --2- 172.21.15.139:0/3416015283 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb81c06c2e0 0x7fb81c06e7a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.935+0000 7fb8358f1640 1 --2- 172.21.15.139:0/3416015283 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8300cb070 0x7fb8300cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.935+0000 7fb8358f1640 1 --2- 172.21.15.139:0/3416015283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb830154c20 0x7fb83015c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.935+0000 7fb8358f1640 1 --2- 172.21.15.139:0/3416015283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8300ccc90 0x7fb8301f0720 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:44.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.935+0000 7fb8358f1640 1 -- 172.21.15.139:0/3416015283 >> 172.21.15.139:0/3416015283 conn(0x7fb83006ca40 msgr2=0x7fb8300c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:44.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.936+0000 7fb8358f1640 1 -- 172.21.15.139:0/3416015283 shutdown_connections 2024-09-16T00:08:44.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:44.936+0000 7fb8358f1640 1 -- 172.21.15.139:0/3416015283 wait complete. 2024-09-16T00:08:45.009 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:45.007+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:45.013 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:08:45.018 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:45.018 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:45.027 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:08:45.110 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:45.108+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:45.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.116+0000 7f8d3385e640 1 Processor -- start 2024-09-16T00:08:45.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.116+0000 7f8d3385e640 1 -- start start 2024-09-16T00:08:45.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.117+0000 7f8d3385e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8d2c0c67e0 0x7f8d2c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:45.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.117+0000 7f8d3385e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8d2c06c560 0x7f8d2c0cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:45.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.117+0000 7f8d3385e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d2c06ca30 con 0x7f8d2c06c110 2024-09-16T00:08:45.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.117+0000 7f8d3385e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8d2c06cd10 con 0x7f8d2c0c67e0 2024-09-16T00:08:45.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.117+0000 7f8d3385e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f8d2c0c6bc0 con 0x7f8d2c06c560 2024-09-16T00:08:45.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.117+0000 7f8d315fa640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8d2c0c67e0 0x7f8d2c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:45.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.117+0000 7f8d315fa640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8d2c0c67e0 0x7f8d2c0c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49424/0 (socket says 172.21.15.139:49424) 2024-09-16T00:08:45.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.117+0000 7f8d30df9640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8d2c06c110 0x7f8d2c15a810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:50640/0 (socket says 172.21.15.139:50640) 2024-09-16T00:08:45.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.117+0000 7f8d315fa640 1 -- 172.21.15.139:0/2158391242 learned_addr learned my addr 172.21.15.139:0/2158391242 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:45.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.117+0000 7f8d31dfb640 1 --2- 172.21.15.139:0/2158391242 >> v2:172.21.15.164:3300/0 conn(0x7f8d2c06c560 0x7f8d2c0cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:45.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.117+0000 7f8d315fa640 1 -- 172.21.15.139:0/2158391242 >> v2:172.21.15.164:3300/0 conn(0x7f8d2c06c560 msgr2=0x7f8d2c0cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:45.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.117+0000 7f8d315fa640 1 --2- 172.21.15.139:0/2158391242 >> v2:172.21.15.164:3300/0 conn(0x7f8d2c06c560 0x7f8d2c0cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.117+0000 7f8d315fa640 1 -- 172.21.15.139:0/2158391242 >> v1:172.21.15.164:6789/0 conn(0x7f8d2c06c110 legacy=0x7f8d2c15a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:45.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.117+0000 7f8d23fff640 1 -- 172.21.15.139:0/2158391242 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1736749081 0 0) 0x7f8d2c06ca30 con 0x7f8d2c06c110 2024-09-16T00:08:45.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.117+0000 7f8d315fa640 1 -- 172.21.15.139:0/2158391242 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2c15df20 con 0x7f8d2c0c67e0 2024-09-16T00:08:45.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.117+0000 7f8d31dfb640 1 --2- 172.21.15.139:0/2158391242 >> v2:172.21.15.164:3300/0 conn(0x7f8d2c06c560 0x7f8d2c0cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:45.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.118+0000 7f8d315fa640 1 --2- 172.21.15.139:0/2158391242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d2c0c67e0 0x7f8d2c0c7b90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c06bd50 tx=0x7f8d2802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89b32d4c7e1f7e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:45.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.118+0000 7f8d23fff640 1 -- 172.21.15.139:0/2158391242 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28037070 con 0x7f8d2c0c67e0 2024-09-16T00:08:45.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.119+0000 7f8d23fff640 1 -- 172.21.15.139:0/2158391242 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d2802ae50 con 0x7f8d2c0c67e0 2024-09-16T00:08:45.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.119+0000 7f8d23fff640 1 -- 172.21.15.139:0/2158391242 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28032860 con 0x7f8d2c0c67e0 2024-09-16T00:08:45.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.119+0000 7f8d3385e640 1 -- 172.21.15.139:0/2158391242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d2c0c67e0 msgr2=0x7f8d2c0c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:45.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.119+0000 7f8d3385e640 1 --2- 172.21.15.139:0/2158391242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d2c0c67e0 0x7f8d2c0c7b90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c06bd50 tx=0x7f8d2802a300 comp rx=0 tx=0).stop 2024-09-16T00:08:45.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.119+0000 7f8d3385e640 1 -- 172.21.15.139:0/2158391242 shutdown_connections 2024-09-16T00:08:45.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.119+0000 7f8d3385e640 1 --2- 172.21.15.139:0/2158391242 >> v2:172.21.15.164:3300/0 conn(0x7f8d2c06c560 0x7f8d2c0cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.119+0000 7f8d3385e640 1 --2- 172.21.15.139:0/2158391242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d2c0c67e0 0x7f8d2c0c7b90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.119+0000 7f8d3385e640 1 -- 172.21.15.139:0/2158391242 >> 172.21.15.139:0/2158391242 conn(0x7f8d2c0d05f0 msgr2=0x7f8d2c0d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:45.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.119+0000 7f8d3385e640 1 -- 172.21.15.139:0/2158391242 shutdown_connections 2024-09-16T00:08:45.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.119+0000 7f8d3385e640 1 -- 172.21.15.139:0/2158391242 wait complete. 2024-09-16T00:08:45.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.121+0000 7f8d3385e640 1 Processor -- start 2024-09-16T00:08:45.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.121+0000 7f8d3385e640 1 -- start start 2024-09-16T00:08:45.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.121+0000 7f8d3385e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d2c06c110 0x7f8d2c0c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:45.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.121+0000 7f8d3385e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d2c06c560 0x7f8d2c1e4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:45.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.121+0000 7f8d3385e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d2c0c67e0 0x7f8d2c1f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:45.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.121+0000 7f8d3385e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c15e4e0 con 0x7f8d2c06c560 2024-09-16T00:08:45.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.121+0000 7f8d3385e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c15e850 con 0x7f8d2c06c110 2024-09-16T00:08:45.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.121+0000 7f8d30df9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d2c06c560 0x7f8d2c1e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:45.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.121+0000 7f8d3385e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8d2c15af20 con 0x7f8d2c0c67e0 2024-09-16T00:08:45.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.121+0000 7f8d30df9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d2c06c560 0x7f8d2c1e4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49426/0 (socket says 172.21.15.139:49426) 2024-09-16T00:08:45.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.121+0000 7f8d30df9640 1 -- 172.21.15.139:0/3269950566 learned_addr learned my addr 172.21.15.139:0/3269950566 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:45.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.121+0000 7f8d315fa640 1 --2- 172.21.15.139:0/3269950566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d2c06c110 0x7f8d2c0c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:45.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.122+0000 7f8d31dfb640 1 --2- 172.21.15.139:0/3269950566 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d2c0c67e0 0x7f8d2c1f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:45.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.122+0000 7f8d315fa640 1 -- 172.21.15.139:0/3269950566 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d2c0c67e0 msgr2=0x7f8d2c1f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:45.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.122+0000 7f8d315fa640 1 --2- 172.21.15.139:0/3269950566 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d2c0c67e0 0x7f8d2c1f37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.122+0000 7f8d315fa640 1 -- 172.21.15.139:0/3269950566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d2c06c560 msgr2=0x7f8d2c1e4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:45.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.122+0000 7f8d315fa640 1 --2- 172.21.15.139:0/3269950566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d2c06c560 0x7f8d2c1e4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.122+0000 7f8d315fa640 1 -- 172.21.15.139:0/3269950566 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2c0cfaf0 con 0x7f8d2c06c110 2024-09-16T00:08:45.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.122+0000 7f8d30df9640 1 --2- 172.21.15.139:0/3269950566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d2c06c560 0x7f8d2c1e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:45.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.122+0000 7f8d315fa640 1 --2- 172.21.15.139:0/3269950566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d2c06c110 0x7f8d2c0c49c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8d28031000 tx=0x7f8d280046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:45.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.122+0000 7f8d21ffb640 1 -- 172.21.15.139:0/3269950566 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28037040 con 0x7f8d2c06c110 2024-09-16T00:08:45.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.122+0000 7f8d21ffb640 1 -- 172.21.15.139:0/3269950566 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d28032cf0 con 0x7f8d2c06c110 2024-09-16T00:08:45.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.123+0000 7f8d21ffb640 1 -- 172.21.15.139:0/3269950566 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28004250 con 0x7f8d2c06c110 2024-09-16T00:08:45.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.123+0000 7f8d3385e640 1 -- 172.21.15.139:0/3269950566 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2c0d09d0 con 0x7f8d2c06c110 2024-09-16T00:08:45.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.123+0000 7f8d3385e640 1 -- 172.21.15.139:0/3269950566 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2c0ce150 con 0x7f8d2c06c110 2024-09-16T00:08:45.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.124+0000 7f8d3385e640 1 -- 172.21.15.139:0/3269950566 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d2c0c7b90 con 0x7f8d2c06c110 2024-09-16T00:08:45.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.125+0000 7f8d21ffb640 1 -- 172.21.15.139:0/3269950566 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8d28050060 con 0x7f8d2c06c110 2024-09-16T00:08:45.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.126+0000 7f8d21ffb640 1 --2- 172.21.15.139:0/3269950566 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8d1006c2b0 0x7f8d1006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:45.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.126+0000 7f8d21ffb640 1 -- 172.21.15.139:0/3269950566 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5915+0+0 (secure 0 0 0) 0x7f8d280b35a0 con 0x7f8d2c06c110 2024-09-16T00:08:45.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.126+0000 7f8d30df9640 1 --2- 172.21.15.139:0/3269950566 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8d1006c2b0 0x7f8d1006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:45.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.126+0000 7f8d30df9640 1 --2- 172.21.15.139:0/3269950566 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8d1006c2b0 0x7f8d1006e770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c007b10 tx=0x7f8d1c007a60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:45.131 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:45.131 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:45.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.130+0000 7f8d21ffb640 1 -- 172.21.15.139:0/3269950566 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8d28044070 con 0x7f8d2c06c110 2024-09-16T00:08:45.140 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:08:45.211 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:45.209+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:45.245 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:45.245 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:45.254 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:08:45.338 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:45.337+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:45.399 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:45.399 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:45.408 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:08:45.412 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:45.411+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:45.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.467+0000 7f8d3385e640 1 -- 172.21.15.139:0/3269950566 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8d2c0cbc30 con 0x7f8d1006c2b0 2024-09-16T00:08:45.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.479+0000 7f8d21ffb640 1 -- 172.21.15.139:0/3269950566 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252002 (secure 0 0 0) 0x7f8d2c0cbc30 con 0x7f8d1006c2b0 2024-09-16T00:08:45.483 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:08:45.485 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:08:45.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.483+0000 7f8d3385e640 1 -- 172.21.15.139:0/3269950566 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8d1006c2b0 msgr2=0x7f8d1006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:45.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.483+0000 7f8d3385e640 1 --2- 172.21.15.139:0/3269950566 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8d1006c2b0 0x7f8d1006e770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c007b10 tx=0x7f8d1c007a60 comp rx=0 tx=0).stop 2024-09-16T00:08:45.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.483+0000 7f8d3385e640 1 -- 172.21.15.139:0/3269950566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d2c06c110 msgr2=0x7f8d2c0c49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:45.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.483+0000 7f8d3385e640 1 --2- 172.21.15.139:0/3269950566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d2c06c110 0x7f8d2c0c49c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8d28031000 tx=0x7f8d280046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.483+0000 7f8d3385e640 1 -- 172.21.15.139:0/3269950566 shutdown_connections 2024-09-16T00:08:45.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.483+0000 7f8d3385e640 1 --2- 172.21.15.139:0/3269950566 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8d1006c2b0 0x7f8d1006e770 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.483+0000 7f8d3385e640 1 --2- 172.21.15.139:0/3269950566 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8d2c0c67e0 0x7f8d2c1f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.483+0000 7f8d3385e640 1 --2- 172.21.15.139:0/3269950566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d2c06c560 0x7f8d2c1e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.483+0000 7f8d3385e640 1 --2- 172.21.15.139:0/3269950566 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8d2c06c110 0x7f8d2c0c49c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.483+0000 7f8d3385e640 1 -- 172.21.15.139:0/3269950566 >> 172.21.15.139:0/3269950566 conn(0x7f8d2c0d05f0 msgr2=0x7f8d2c06d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:45.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.483+0000 7f8d3385e640 1 -- 172.21.15.139:0/3269950566 shutdown_connections 2024-09-16T00:08:45.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.483+0000 7f8d3385e640 1 -- 172.21.15.139:0/3269950566 wait complete. 2024-09-16T00:08:45.496 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-09-16T00:08:44.934222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597721,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":212,"num_write_kb":2439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":13110,"num_keys_recovered":4,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220648,"kb_used_data":4412,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754072,"statfs":{"total":773094113280,"available":772868169728,"internally_reserved":0,"allocated":4517888,"data_stored":3080279,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":2,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.897610"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":73,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455566+0000","last_change":"2024-09-16T00:08:36.888570+0000","last_active":"2024-09-16T00:08:40.455566+0000","last_peered":"2024-09-16T00:08:40.455566+0000","last_clean":"2024-09-16T00:08:40.455566+0000","last_became_active":"2024-09-16T00:08:36.888143+0000","last_became_peered":"2024-09-16T00:08:36.888143+0000","last_unstale":"2024-09-16T00:08:40.455566+0000","last_undegraded":"2024-09-16T00:08:40.455566+0000","last_fullsized":"2024-09-16T00:08:40.455566+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:30.579681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913558+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:08:38.913558+0000","last_peered":"2024-09-16T00:08:38.913558+0000","last_clean":"2024-09-16T00:08:38.913558+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:08:38.913558+0000","last_undegraded":"2024-09-16T00:08:38.913558+0000","last_fullsized":"2024-09-16T00:08:38.913558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504598+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:08:40.504598+0000","last_peered":"2024-09-16T00:08:40.504598+0000","last_clean":"2024-09-16T00:08:40.504598+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:08:40.504598+0000","last_undegraded":"2024-09-16T00:08:40.504598+0000","last_fullsized":"2024-09-16T00:08:40.504598+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.922026+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:08:41.922026+0000","last_peered":"2024-09-16T00:08:41.922026+0000","last_clean":"2024-09-16T00:08:41.922026+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:08:41.922026+0000","last_undegraded":"2024-09-16T00:08:41.922026+0000","last_fullsized":"2024-09-16T00:08:41.922026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913518+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:08:38.913518+0000","last_peered":"2024-09-16T00:08:38.913518+0000","last_clean":"2024-09-16T00:08:38.913518+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:08:38.913518+0000","last_undegraded":"2024-09-16T00:08:38.913518+0000","last_fullsized":"2024-09-16T00:08:38.913518+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.921831+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:08:41.921831+0000","last_peered":"2024-09-16T00:08:41.921831+0000","last_clean":"2024-09-16T00:08:41.921831+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:08:41.921831+0000","last_undegraded":"2024-09-16T00:08:41.921831+0000","last_fullsized":"2024-09-16T00:08:41.921831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455098+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:08:40.455098+0000","last_peered":"2024-09-16T00:08:40.455098+0000","last_clean":"2024-09-16T00:08:40.455098+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:08:40.455098+0000","last_undegraded":"2024-09-16T00:08:40.455098+0000","last_fullsized":"2024-09-16T00:08:40.455098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.786683+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:08:39.786683+0000","last_peered":"2024-09-16T00:08:39.786683+0000","last_clean":"2024-09-16T00:08:39.786683+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:08:39.786683+0000","last_undegraded":"2024-09-16T00:08:39.786683+0000","last_fullsized":"2024-09-16T00:08:39.786683+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455611+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:08:40.455611+0000","last_peered":"2024-09-16T00:08:40.455611+0000","last_clean":"2024-09-16T00:08:40.455611+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:08:40.455611+0000","last_undegraded":"2024-09-16T00:08:40.455611+0000","last_fullsized":"2024-09-16T00:08:40.455611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.404411+0000","last_change":"2024-09-16T00:08:36.891868+0000","last_active":"2024-09-16T00:08:39.404411+0000","last_peered":"2024-09-16T00:08:39.404411+0000","last_clean":"2024-09-16T00:08:39.404411+0000","last_became_active":"2024-09-16T00:08:36.891605+0000","last_became_peered":"2024-09-16T00:08:36.891605+0000","last_unstale":"2024-09-16T00:08:39.404411+0000","last_undegraded":"2024-09-16T00:08:39.404411+0000","last_fullsized":"2024-09-16T00:08:39.404411+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:46:00.730436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.404250+0000","last_change":"2024-09-16T00:08:36.891589+0000","last_active":"2024-09-16T00:08:39.404250+0000","last_peered":"2024-09-16T00:08:39.404250+0000","last_clean":"2024-09-16T00:08:39.404250+0000","last_became_active":"2024-09-16T00:08:36.891279+0000","last_became_peered":"2024-09-16T00:08:36.891279+0000","last_unstale":"2024-09-16T00:08:39.404250+0000","last_undegraded":"2024-09-16T00:08:39.404250+0000","last_fullsized":"2024-09-16T00:08:39.404250+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.922917+0000","last_change":"2024-09-16T00:08:36.888576+0000","last_active":"2024-09-16T00:08:41.922917+0000","last_peered":"2024-09-16T00:08:41.922917+0000","last_clean":"2024-09-16T00:08:41.922917+0000","last_became_active":"2024-09-16T00:08:36.888301+0000","last_became_peered":"2024-09-16T00:08:36.888301+0000","last_unstale":"2024-09-16T00:08:41.922917+0000","last_undegraded":"2024-09-16T00:08:41.922917+0000","last_fullsized":"2024-09-16T00:08:41.922917+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389414+0000","last_change":"2024-09-16T00:08:36.892595+0000","last_active":"2024-09-16T00:08:40.389414+0000","last_peered":"2024-09-16T00:08:40.389414+0000","last_clean":"2024-09-16T00:08:40.389414+0000","last_became_active":"2024-09-16T00:08:36.892316+0000","last_became_peered":"2024-09-16T00:08:36.892316+0000","last_unstale":"2024-09-16T00:08:40.389414+0000","last_undegraded":"2024-09-16T00:08:40.389414+0000","last_fullsized":"2024-09-16T00:08:40.389414+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:47.480168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504439+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:08:40.504439+0000","last_peered":"2024-09-16T00:08:40.504439+0000","last_clean":"2024-09-16T00:08:40.504439+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:08:40.504439+0000","last_undegraded":"2024-09-16T00:08:40.504439+0000","last_fullsized":"2024-09-16T00:08:40.504439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913756+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:08:38.913756+0000","last_peered":"2024-09-16T00:08:38.913756+0000","last_clean":"2024-09-16T00:08:38.913756+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:08:38.913756+0000","last_undegraded":"2024-09-16T00:08:38.913756+0000","last_fullsized":"2024-09-16T00:08:38.913756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.921713+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:08:41.921713+0000","last_peered":"2024-09-16T00:08:41.921713+0000","last_clean":"2024-09-16T00:08:41.921713+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:08:41.921713+0000","last_undegraded":"2024-09-16T00:08:41.921713+0000","last_fullsized":"2024-09-16T00:08:41.921713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.922363+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:08:41.922363+0000","last_peered":"2024-09-16T00:08:41.922363+0000","last_clean":"2024-09-16T00:08:41.922363+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:08:41.922363+0000","last_undegraded":"2024-09-16T00:08:41.922363+0000","last_fullsized":"2024-09-16T00:08:41.922363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":84,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.787096+0000","last_change":"2024-09-16T00:08:36.894430+0000","last_active":"2024-09-16T00:08:39.787096+0000","last_peered":"2024-09-16T00:08:39.787096+0000","last_clean":"2024-09-16T00:08:39.787096+0000","last_became_active":"2024-09-16T00:08:36.894138+0000","last_became_peered":"2024-09-16T00:08:36.894138+0000","last_unstale":"2024-09-16T00:08:39.787096+0000","last_undegraded":"2024-09-16T00:08:39.787096+0000","last_fullsized":"2024-09-16T00:08:39.787096+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:19.478353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455446+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:08:40.455446+0000","last_peered":"2024-09-16T00:08:40.455446+0000","last_clean":"2024-09-16T00:08:40.455446+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:08:40.455446+0000","last_undegraded":"2024-09-16T00:08:40.455446+0000","last_fullsized":"2024-09-16T00:08:40.455446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.404297+0000","last_change":"2024-09-16T00:08:36.890676+0000","last_active":"2024-09-16T00:08:39.404297+0000","last_peered":"2024-09-16T00:08:39.404297+0000","last_clean":"2024-09-16T00:08:39.404297+0000","last_became_active":"2024-09-16T00:08:36.889743+0000","last_became_peered":"2024-09-16T00:08:36.889743+0000","last_unstale":"2024-09-16T00:08:39.404297+0000","last_undegraded":"2024-09-16T00:08:39.404297+0000","last_fullsized":"2024-09-16T00:08:39.404297+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:02.709496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504430+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:08:40.504430+0000","last_peered":"2024-09-16T00:08:40.504430+0000","last_clean":"2024-09-16T00:08:40.504430+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:08:40.504430+0000","last_undegraded":"2024-09-16T00:08:40.504430+0000","last_fullsized":"2024-09-16T00:08:40.504430+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.921935+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:08:41.921935+0000","last_peered":"2024-09-16T00:08:41.921935+0000","last_clean":"2024-09-16T00:08:41.921935+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:08:41.921935+0000","last_undegraded":"2024-09-16T00:08:41.921935+0000","last_fullsized":"2024-09-16T00:08:41.921935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389236+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:08:40.389236+0000","last_peered":"2024-09-16T00:08:40.389236+0000","last_clean":"2024-09-16T00:08:40.389236+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:08:40.389236+0000","last_undegraded":"2024-09-16T00:08:40.389236+0000","last_fullsized":"2024-09-16T00:08:40.389236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.921742+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:08:41.921742+0000","last_peered":"2024-09-16T00:08:41.921742+0000","last_clean":"2024-09-16T00:08:41.921742+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:08:41.921742+0000","last_undegraded":"2024-09-16T00:08:41.921742+0000","last_fullsized":"2024-09-16T00:08:41.921742+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389795+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:08:40.389795+0000","last_peered":"2024-09-16T00:08:40.389795+0000","last_clean":"2024-09-16T00:08:40.389795+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:08:40.389795+0000","last_undegraded":"2024-09-16T00:08:40.389795+0000","last_fullsized":"2024-09-16T00:08:40.389795+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.404480+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:08:39.404480+0000","last_peered":"2024-09-16T00:08:39.404480+0000","last_clean":"2024-09-16T00:08:39.404480+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:08:39.404480+0000","last_undegraded":"2024-09-16T00:08:39.404480+0000","last_fullsized":"2024-09-16T00:08:39.404480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913978+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:08:38.913978+0000","last_peered":"2024-09-16T00:08:38.913978+0000","last_clean":"2024-09-16T00:08:38.913978+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:08:38.913978+0000","last_undegraded":"2024-09-16T00:08:38.913978+0000","last_fullsized":"2024-09-16T00:08:38.913978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389479+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:08:40.389479+0000","last_peered":"2024-09-16T00:08:40.389479+0000","last_clean":"2024-09-16T00:08:40.389479+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:08:40.389479+0000","last_undegraded":"2024-09-16T00:08:40.389479+0000","last_fullsized":"2024-09-16T00:08:40.389479+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504452+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:08:40.504452+0000","last_peered":"2024-09-16T00:08:40.504452+0000","last_clean":"2024-09-16T00:08:40.504452+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:08:40.504452+0000","last_undegraded":"2024-09-16T00:08:40.504452+0000","last_fullsized":"2024-09-16T00:08:40.504452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.786805+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:08:39.786805+0000","last_peered":"2024-09-16T00:08:39.786805+0000","last_clean":"2024-09-16T00:08:39.786805+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:08:39.786805+0000","last_undegraded":"2024-09-16T00:08:39.786805+0000","last_fullsized":"2024-09-16T00:08:39.786805+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":69,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.786660+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:08:39.786660+0000","last_peered":"2024-09-16T00:08:39.786660+0000","last_clean":"2024-09-16T00:08:39.786660+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:08:39.786660+0000","last_undegraded":"2024-09-16T00:08:39.786660+0000","last_fullsized":"2024-09-16T00:08:39.786660+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.404025+0000","last_change":"2024-09-16T00:08:36.894101+0000","last_active":"2024-09-16T00:08:39.404025+0000","last_peered":"2024-09-16T00:08:39.404025+0000","last_clean":"2024-09-16T00:08:39.404025+0000","last_became_active":"2024-09-16T00:08:36.893807+0000","last_became_peered":"2024-09-16T00:08:36.893807+0000","last_unstale":"2024-09-16T00:08:39.404025+0000","last_undegraded":"2024-09-16T00:08:39.404025+0000","last_fullsized":"2024-09-16T00:08:39.404025+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:40.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.922750+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:08:41.922750+0000","last_peered":"2024-09-16T00:08:41.922750+0000","last_clean":"2024-09-16T00:08:41.922750+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:08:41.922750+0000","last_undegraded":"2024-09-16T00:08:41.922750+0000","last_fullsized":"2024-09-16T00:08:41.922750+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":67,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389579+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:08:40.389579+0000","last_peered":"2024-09-16T00:08:40.389579+0000","last_clean":"2024-09-16T00:08:40.389579+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:08:40.389579+0000","last_undegraded":"2024-09-16T00:08:40.389579+0000","last_fullsized":"2024-09-16T00:08:40.389579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504774+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:08:40.504774+0000","last_peered":"2024-09-16T00:08:40.504774+0000","last_clean":"2024-09-16T00:08:40.504774+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:08:40.504774+0000","last_undegraded":"2024-09-16T00:08:40.504774+0000","last_fullsized":"2024-09-16T00:08:40.504774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.404124+0000","last_change":"2024-09-16T00:08:36.890316+0000","last_active":"2024-09-16T00:08:39.404124+0000","last_peered":"2024-09-16T00:08:39.404124+0000","last_clean":"2024-09-16T00:08:39.404124+0000","last_became_active":"2024-09-16T00:08:36.889635+0000","last_became_peered":"2024-09-16T00:08:36.889635+0000","last_unstale":"2024-09-16T00:08:39.404124+0000","last_undegraded":"2024-09-16T00:08:39.404124+0000","last_fullsized":"2024-09-16T00:08:39.404124+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:28.843883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":76,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389324+0000","last_change":"2024-09-16T00:08:36.888349+0000","last_active":"2024-09-16T00:08:40.389324+0000","last_peered":"2024-09-16T00:08:40.389324+0000","last_clean":"2024-09-16T00:08:40.389324+0000","last_became_active":"2024-09-16T00:08:36.888052+0000","last_became_peered":"2024-09-16T00:08:36.888052+0000","last_unstale":"2024-09-16T00:08:40.389324+0000","last_undegraded":"2024-09-16T00:08:40.389324+0000","last_fullsized":"2024-09-16T00:08:40.389324+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:49.607254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389802+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:08:40.389802+0000","last_peered":"2024-09-16T00:08:40.389802+0000","last_clean":"2024-09-16T00:08:40.389802+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:08:40.389802+0000","last_undegraded":"2024-09-16T00:08:40.389802+0000","last_fullsized":"2024-09-16T00:08:40.389802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.921830+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:08:41.921830+0000","last_peered":"2024-09-16T00:08:41.921830+0000","last_clean":"2024-09-16T00:08:41.921830+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:08:41.921830+0000","last_undegraded":"2024-09-16T00:08:41.921830+0000","last_fullsized":"2024-09-16T00:08:41.921830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913558+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:08:38.913558+0000","last_peered":"2024-09-16T00:08:38.913558+0000","last_clean":"2024-09-16T00:08:38.913558+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:08:38.913558+0000","last_undegraded":"2024-09-16T00:08:38.913558+0000","last_fullsized":"2024-09-16T00:08:38.913558+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.404394+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:08:39.404394+0000","last_peered":"2024-09-16T00:08:39.404394+0000","last_clean":"2024-09-16T00:08:39.404394+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:08:39.404394+0000","last_undegraded":"2024-09-16T00:08:39.404394+0000","last_fullsized":"2024-09-16T00:08:39.404394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.922171+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:08:41.922171+0000","last_peered":"2024-09-16T00:08:41.922171+0000","last_clean":"2024-09-16T00:08:41.922171+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:08:41.922171+0000","last_undegraded":"2024-09-16T00:08:41.922171+0000","last_fullsized":"2024-09-16T00:08:41.922171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.922247+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:08:41.922247+0000","last_peered":"2024-09-16T00:08:41.922247+0000","last_clean":"2024-09-16T00:08:41.922247+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:08:41.922247+0000","last_undegraded":"2024-09-16T00:08:41.922247+0000","last_fullsized":"2024-09-16T00:08:41.922247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504691+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:08:40.504691+0000","last_peered":"2024-09-16T00:08:40.504691+0000","last_clean":"2024-09-16T00:08:40.504691+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:08:40.504691+0000","last_undegraded":"2024-09-16T00:08:40.504691+0000","last_fullsized":"2024-09-16T00:08:40.504691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":68,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.914027+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:08:38.914027+0000","last_peered":"2024-09-16T00:08:38.914027+0000","last_clean":"2024-09-16T00:08:38.914027+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:08:38.914027+0000","last_undegraded":"2024-09-16T00:08:38.914027+0000","last_fullsized":"2024-09-16T00:08:38.914027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.922976+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:08:41.922976+0000","last_peered":"2024-09-16T00:08:41.922976+0000","last_clean":"2024-09-16T00:08:41.922976+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:08:41.922976+0000","last_undegraded":"2024-09-16T00:08:41.922976+0000","last_fullsized":"2024-09-16T00:08:41.922976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"28'10","reported_seq":22,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.404160+0000","last_change":"2024-09-16T00:08:36.914308+0000","last_active":"2024-09-16T00:08:39.404160+0000","last_peered":"2024-09-16T00:08:39.404160+0000","last_clean":"2024-09-16T00:08:39.404160+0000","last_became_active":"2024-09-16T00:08:36.890227+0000","last_became_peered":"2024-09-16T00:08:36.890227+0000","last_unstale":"2024-09-16T00:08:39.404160+0000","last_undegraded":"2024-09-16T00:08:39.404160+0000","last_fullsized":"2024-09-16T00:08:39.404160+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:31.761195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6555,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13110,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455330+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:08:40.455330+0000","last_peered":"2024-09-16T00:08:40.455330+0000","last_clean":"2024-09-16T00:08:40.455330+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:08:40.455330+0000","last_undegraded":"2024-09-16T00:08:40.455330+0000","last_fullsized":"2024-09-16T00:08:40.455330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":67,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504540+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:08:40.504540+0000","last_peered":"2024-09-16T00:08:40.504540+0000","last_clean":"2024-09-16T00:08:40.504540+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:08:40.504540+0000","last_undegraded":"2024-09-16T00:08:40.504540+0000","last_fullsized":"2024-09-16T00:08:40.504540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504365+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:08:40.504365+0000","last_peered":"2024-09-16T00:08:40.504365+0000","last_clean":"2024-09-16T00:08:40.504365+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:08:40.504365+0000","last_undegraded":"2024-09-16T00:08:40.504365+0000","last_fullsized":"2024-09-16T00:08:40.504365+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.921974+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:08:41.921974+0000","last_peered":"2024-09-16T00:08:41.921974+0000","last_clean":"2024-09-16T00:08:41.921974+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:08:41.921974+0000","last_undegraded":"2024-09-16T00:08:41.921974+0000","last_fullsized":"2024-09-16T00:08:41.921974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455373+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:08:40.455373+0000","last_peered":"2024-09-16T00:08:40.455373+0000","last_clean":"2024-09-16T00:08:40.455373+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:08:40.455373+0000","last_undegraded":"2024-09-16T00:08:40.455373+0000","last_fullsized":"2024-09-16T00:08:40.455373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.404083+0000","last_change":"2024-09-16T00:08:36.889720+0000","last_active":"2024-09-16T00:08:39.404083+0000","last_peered":"2024-09-16T00:08:39.404083+0000","last_clean":"2024-09-16T00:08:39.404083+0000","last_became_active":"2024-09-16T00:08:36.889345+0000","last_became_peered":"2024-09-16T00:08:36.889345+0000","last_unstale":"2024-09-16T00:08:39.404083+0000","last_undegraded":"2024-09-16T00:08:39.404083+0000","last_fullsized":"2024-09-16T00:08:39.404083+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:03.170073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.404046+0000","last_change":"2024-09-16T00:08:36.896787+0000","last_active":"2024-09-16T00:08:39.404046+0000","last_peered":"2024-09-16T00:08:39.404046+0000","last_clean":"2024-09-16T00:08:39.404046+0000","last_became_active":"2024-09-16T00:08:36.896505+0000","last_became_peered":"2024-09-16T00:08:36.896505+0000","last_unstale":"2024-09-16T00:08:39.404046+0000","last_undegraded":"2024-09-16T00:08:39.404046+0000","last_fullsized":"2024-09-16T00:08:39.404046+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:55.997551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.786661+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:08:39.786661+0000","last_peered":"2024-09-16T00:08:39.786661+0000","last_clean":"2024-09-16T00:08:39.786661+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:08:39.786661+0000","last_undegraded":"2024-09-16T00:08:39.786661+0000","last_fullsized":"2024-09-16T00:08:39.786661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455116+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:08:40.455116+0000","last_peered":"2024-09-16T00:08:40.455116+0000","last_clean":"2024-09-16T00:08:40.455116+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:08:40.455116+0000","last_undegraded":"2024-09-16T00:08:40.455116+0000","last_fullsized":"2024-09-16T00:08:40.455116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.922986+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:08:41.922986+0000","last_peered":"2024-09-16T00:08:41.922986+0000","last_clean":"2024-09-16T00:08:41.922986+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:08:41.922986+0000","last_undegraded":"2024-09-16T00:08:41.922986+0000","last_fullsized":"2024-09-16T00:08:41.922986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455552+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:08:40.455552+0000","last_peered":"2024-09-16T00:08:40.455552+0000","last_clean":"2024-09-16T00:08:40.455552+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:08:40.455552+0000","last_undegraded":"2024-09-16T00:08:40.455552+0000","last_fullsized":"2024-09-16T00:08:40.455552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":68,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.787141+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:08:39.787141+0000","last_peered":"2024-09-16T00:08:39.787141+0000","last_clean":"2024-09-16T00:08:39.787141+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:08:39.787141+0000","last_undegraded":"2024-09-16T00:08:39.787141+0000","last_fullsized":"2024-09-16T00:08:39.787141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455661+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:08:40.455661+0000","last_peered":"2024-09-16T00:08:40.455661+0000","last_clean":"2024-09-16T00:08:40.455661+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:08:40.455661+0000","last_undegraded":"2024-09-16T00:08:40.455661+0000","last_fullsized":"2024-09-16T00:08:40.455661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.914050+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:08:38.914050+0000","last_peered":"2024-09-16T00:08:38.914050+0000","last_clean":"2024-09-16T00:08:38.914050+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:08:38.914050+0000","last_undegraded":"2024-09-16T00:08:38.914050+0000","last_fullsized":"2024-09-16T00:08:38.914050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.922050+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:08:41.922050+0000","last_peered":"2024-09-16T00:08:41.922050+0000","last_clean":"2024-09-16T00:08:41.922050+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:08:41.922050+0000","last_undegraded":"2024-09-16T00:08:41.922050+0000","last_fullsized":"2024-09-16T00:08:41.922050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.786984+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:08:39.786984+0000","last_peered":"2024-09-16T00:08:39.786984+0000","last_clean":"2024-09-16T00:08:39.786984+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:08:39.786984+0000","last_undegraded":"2024-09-16T00:08:39.786984+0000","last_fullsized":"2024-09-16T00:08:39.786984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":68,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455018+0000","last_change":"2024-09-16T00:08:15.795467+0000","last_active":"2024-09-16T00:08:40.455018+0000","last_peered":"2024-09-16T00:08:40.455018+0000","last_clean":"2024-09-16T00:08:40.455018+0000","last_became_active":"2024-09-16T00:08:15.795167+0000","last_became_peered":"2024-09-16T00:08:15.795167+0000","last_unstale":"2024-09-16T00:08:40.455018+0000","last_undegraded":"2024-09-16T00:08:40.455018+0000","last_fullsized":"2024-09-16T00:08:40.455018+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":221,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455228+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:08:40.455228+0000","last_peered":"2024-09-16T00:08:40.455228+0000","last_clean":"2024-09-16T00:08:40.455228+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:08:40.455228+0000","last_undegraded":"2024-09-16T00:08:40.455228+0000","last_fullsized":"2024-09-16T00:08:40.455228+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.921842+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:08:41.921842+0000","last_peered":"2024-09-16T00:08:41.921842+0000","last_clean":"2024-09-16T00:08:41.921842+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:08:41.921842+0000","last_undegraded":"2024-09-16T00:08:41.921842+0000","last_fullsized":"2024-09-16T00:08:41.921842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389905+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:08:40.389905+0000","last_peered":"2024-09-16T00:08:40.389905+0000","last_clean":"2024-09-16T00:08:40.389905+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:08:40.389905+0000","last_undegraded":"2024-09-16T00:08:40.389905+0000","last_fullsized":"2024-09-16T00:08:40.389905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455091+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:08:40.455091+0000","last_peered":"2024-09-16T00:08:40.455091+0000","last_clean":"2024-09-16T00:08:40.455091+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:08:40.455091+0000","last_undegraded":"2024-09-16T00:08:40.455091+0000","last_fullsized":"2024-09-16T00:08:40.455091+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504238+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:08:40.504238+0000","last_peered":"2024-09-16T00:08:40.504238+0000","last_clean":"2024-09-16T00:08:40.504238+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:08:40.504238+0000","last_undegraded":"2024-09-16T00:08:40.504238+0000","last_fullsized":"2024-09-16T00:08:40.504238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":66,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.786772+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:08:39.786772+0000","last_peered":"2024-09-16T00:08:39.786772+0000","last_clean":"2024-09-16T00:08:39.786772+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:08:39.786772+0000","last_undegraded":"2024-09-16T00:08:39.786772+0000","last_fullsized":"2024-09-16T00:08:39.786772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913422+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:08:38.913422+0000","last_peered":"2024-09-16T00:08:38.913422+0000","last_clean":"2024-09-16T00:08:38.913422+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:08:38.913422+0000","last_undegraded":"2024-09-16T00:08:38.913422+0000","last_fullsized":"2024-09-16T00:08:38.913422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":68,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.922015+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:08:41.922015+0000","last_peered":"2024-09-16T00:08:41.922015+0000","last_clean":"2024-09-16T00:08:41.922015+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:08:41.922015+0000","last_undegraded":"2024-09-16T00:08:41.922015+0000","last_fullsized":"2024-09-16T00:08:41.922015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.922869+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:08:41.922869+0000","last_peered":"2024-09-16T00:08:41.922869+0000","last_clean":"2024-09-16T00:08:41.922869+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:08:41.922869+0000","last_undegraded":"2024-09-16T00:08:41.922869+0000","last_fullsized":"2024-09-16T00:08:41.922869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913851+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:08:38.913851+0000","last_peered":"2024-09-16T00:08:38.913851+0000","last_clean":"2024-09-16T00:08:38.913851+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:08:38.913851+0000","last_undegraded":"2024-09-16T00:08:38.913851+0000","last_fullsized":"2024-09-16T00:08:38.913851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.922251+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:08:41.922251+0000","last_peered":"2024-09-16T00:08:41.922251+0000","last_clean":"2024-09-16T00:08:41.922251+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:08:41.922251+0000","last_undegraded":"2024-09-16T00:08:41.922251+0000","last_fullsized":"2024-09-16T00:08:41.922251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455624+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:08:40.455624+0000","last_peered":"2024-09-16T00:08:40.455624+0000","last_clean":"2024-09-16T00:08:40.455624+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:08:40.455624+0000","last_undegraded":"2024-09-16T00:08:40.455624+0000","last_fullsized":"2024-09-16T00:08:40.455624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913770+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:08:38.913770+0000","last_peered":"2024-09-16T00:08:38.913770+0000","last_clean":"2024-09-16T00:08:38.913770+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:08:38.913770+0000","last_undegraded":"2024-09-16T00:08:38.913770+0000","last_fullsized":"2024-09-16T00:08:38.913770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"26'3","reported_seq":86,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913666+0000","last_change":"2024-09-16T00:08:36.906253+0000","last_active":"2024-09-16T00:08:38.913666+0000","last_peered":"2024-09-16T00:08:38.913666+0000","last_clean":"2024-09-16T00:08:38.913666+0000","last_became_active":"2024-09-16T00:08:36.894363+0000","last_became_peered":"2024-09-16T00:08:36.894363+0000","last_unstale":"2024-09-16T00:08:38.913666+0000","last_undegraded":"2024-09-16T00:08:38.913666+0000","last_fullsized":"2024-09-16T00:08:38.913666+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:00.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389880+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:08:40.389880+0000","last_peered":"2024-09-16T00:08:40.389880+0000","last_clean":"2024-09-16T00:08:40.389880+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:08:40.389880+0000","last_undegraded":"2024-09-16T00:08:40.389880+0000","last_fullsized":"2024-09-16T00:08:40.389880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":68,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.404451+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:08:39.404451+0000","last_peered":"2024-09-16T00:08:39.404451+0000","last_clean":"2024-09-16T00:08:39.404451+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:08:39.404451+0000","last_undegraded":"2024-09-16T00:08:39.404451+0000","last_fullsized":"2024-09-16T00:08:39.404451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504700+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:08:40.504700+0000","last_peered":"2024-09-16T00:08:40.504700+0000","last_clean":"2024-09-16T00:08:40.504700+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:08:40.504700+0000","last_undegraded":"2024-09-16T00:08:40.504700+0000","last_fullsized":"2024-09-16T00:08:40.504700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455078+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:08:40.455078+0000","last_peered":"2024-09-16T00:08:40.455078+0000","last_clean":"2024-09-16T00:08:40.455078+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:08:40.455078+0000","last_undegraded":"2024-09-16T00:08:40.455078+0000","last_fullsized":"2024-09-16T00:08:40.455078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":65,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.922574+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:08:41.922574+0000","last_peered":"2024-09-16T00:08:41.922574+0000","last_clean":"2024-09-16T00:08:41.922574+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:08:41.922574+0000","last_undegraded":"2024-09-16T00:08:41.922574+0000","last_fullsized":"2024-09-16T00:08:41.922574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913841+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:08:38.913841+0000","last_peered":"2024-09-16T00:08:38.913841+0000","last_clean":"2024-09-16T00:08:38.913841+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:08:38.913841+0000","last_undegraded":"2024-09-16T00:08:38.913841+0000","last_fullsized":"2024-09-16T00:08:38.913841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389182+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:08:40.389182+0000","last_peered":"2024-09-16T00:08:40.389182+0000","last_clean":"2024-09-16T00:08:40.389182+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:08:40.389182+0000","last_undegraded":"2024-09-16T00:08:40.389182+0000","last_fullsized":"2024-09-16T00:08:40.389182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":65,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:38.913666+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:08:38.913666+0000","last_peered":"2024-09-16T00:08:38.913666+0000","last_clean":"2024-09-16T00:08:38.913666+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:08:38.913666+0000","last_undegraded":"2024-09-16T00:08:38.913666+0000","last_fullsized":"2024-09-16T00:08:38.913666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":64,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389497+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:08:40.389497+0000","last_peered":"2024-09-16T00:08:40.389497+0000","last_clean":"2024-09-16T00:08:40.389497+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:08:40.389497+0000","last_undegraded":"2024-09-16T00:08:40.389497+0000","last_fullsized":"2024-09-16T00:08:40.389497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":67,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455478+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:08:40.455478+0000","last_peered":"2024-09-16T00:08:40.455478+0000","last_clean":"2024-09-16T00:08:40.455478+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:08:40.455478+0000","last_undegraded":"2024-09-16T00:08:40.455478+0000","last_fullsized":"2024-09-16T00:08:40.455478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.404092+0000","last_change":"2024-09-16T00:08:36.890347+0000","last_active":"2024-09-16T00:08:39.404092+0000","last_peered":"2024-09-16T00:08:39.404092+0000","last_clean":"2024-09-16T00:08:39.404092+0000","last_became_active":"2024-09-16T00:08:36.890009+0000","last_became_peered":"2024-09-16T00:08:36.890009+0000","last_unstale":"2024-09-16T00:08:39.404092+0000","last_undegraded":"2024-09-16T00:08:39.404092+0000","last_fullsized":"2024-09-16T00:08:39.404092+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:24.298978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.922141+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:08:41.922141+0000","last_peered":"2024-09-16T00:08:41.922141+0000","last_clean":"2024-09-16T00:08:41.922141+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:08:41.922141+0000","last_undegraded":"2024-09-16T00:08:41.922141+0000","last_fullsized":"2024-09-16T00:08:41.922141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":67,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455344+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:08:40.455344+0000","last_peered":"2024-09-16T00:08:40.455344+0000","last_clean":"2024-09-16T00:08:40.455344+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:08:40.455344+0000","last_undegraded":"2024-09-16T00:08:40.455344+0000","last_fullsized":"2024-09-16T00:08:40.455344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":66,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-16T00:08:41.921965+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:08:41.921965+0000","last_peered":"2024-09-16T00:08:41.921965+0000","last_clean":"2024-09-16T00:08:41.921965+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:08:41.921965+0000","last_undegraded":"2024-09-16T00:08:41.921965+0000","last_fullsized":"2024-09-16T00:08:41.921965+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":85,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.455403+0000","last_change":"2024-09-16T00:08:36.900704+0000","last_active":"2024-09-16T00:08:40.455403+0000","last_peered":"2024-09-16T00:08:40.455403+0000","last_clean":"2024-09-16T00:08:40.455403+0000","last_became_active":"2024-09-16T00:08:36.892520+0000","last_became_peered":"2024-09-16T00:08:36.892520+0000","last_unstale":"2024-09-16T00:08:40.455403+0000","last_undegraded":"2024-09-16T00:08:40.455403+0000","last_fullsized":"2024-09-16T00:08:40.455403+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:30.056041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:39.787096+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:08:39.787096+0000","last_peered":"2024-09-16T00:08:39.787096+0000","last_clean":"2024-09-16T00:08:39.787096+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:08:39.787096+0000","last_undegraded":"2024-09-16T00:08:39.787096+0000","last_fullsized":"2024-09-16T00:08:39.787096+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":67,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389975+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:08:40.389975+0000","last_peered":"2024-09-16T00:08:40.389975+0000","last_clean":"2024-09-16T00:08:40.389975+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:08:40.389975+0000","last_undegraded":"2024-09-16T00:08:40.389975+0000","last_fullsized":"2024-09-16T00:08:40.389975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":74,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.389058+0000","last_change":"2024-09-16T00:08:36.889082+0000","last_active":"2024-09-16T00:08:40.389058+0000","last_peered":"2024-09-16T00:08:40.389058+0000","last_clean":"2024-09-16T00:08:40.389058+0000","last_became_active":"2024-09-16T00:08:36.888807+0000","last_became_peered":"2024-09-16T00:08:36.888807+0000","last_unstale":"2024-09-16T00:08:40.389058+0000","last_undegraded":"2024-09-16T00:08:40.389058+0000","last_fullsized":"2024-09-16T00:08:40.389058+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:22.825324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":69,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-16T00:08:40.504726+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:08:40.504726+0000","last_peered":"2024-09-16T00:08:40.504726+0000","last_clean":"2024-09-16T00:08:40.504726+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:08:40.504726+0000","last_undegraded":"2024-09-16T00:08:40.504726+0000","last_fullsized":"2024-09-16T00:08:40.504726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7353,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":53,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":13110,"num_keys_recovered":4,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":14706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509460,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":610304,"data_stored":827319,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509460,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":557056,"data_stored":242924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":544768,"data_stored":236459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":565248,"data_stored":242610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":544768,"data_stored":236809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":548864,"data_stored":236529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":618496,"data_stored":827375,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":528384,"data_stored":230254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":6555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":6555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:08:45.501 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T00:08:45.501 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:08:45.501 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T00:08:45.501 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T00:08:45.539 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:45.537+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:45.552 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:45.552 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:45.561 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:08:45.614 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:45.613+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:45.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.643+0000 7f2f0b152640 1 Processor -- start 2024-09-16T00:08:45.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.644+0000 7f2f0b152640 1 -- start start 2024-09-16T00:08:45.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.644+0000 7f2f0b152640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f040c7b90 0x7f2f0415a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:45.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.644+0000 7f2f0b152640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f040c7fb0 con 0x7f2f040c67e0 2024-09-16T00:08:45.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.644+0000 7f2f0b152640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f0406ca30 con 0x7f2f0415ad50 2024-09-16T00:08:45.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.644+0000 7f2f0b152640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2f0406cd10 con 0x7f2f040c7b90 2024-09-16T00:08:45.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.645+0000 7f2f08eee640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2f040c67e0 0x7f2f0406c110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45582/0 (socket says 172.21.15.139:45582) 2024-09-16T00:08:45.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.645+0000 7f2f08eee640 1 -- 172.21.15.139:0/1382308034 learned_addr learned my addr 172.21.15.139:0/1382308034 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:45.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.645+0000 7f2f03fff640 1 --2- 172.21.15.139:0/1382308034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f040c7b90 0x7f2f0415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:45.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.645+0000 7f2f03fff640 1 -- 172.21.15.139:0/1382308034 >> v1:172.21.15.164:6789/0 conn(0x7f2f0415ad50 legacy=0x7f2f040cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:45.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.645+0000 7f2f03fff640 1 -- 172.21.15.139:0/1382308034 >> v1:172.21.15.139:6789/0 conn(0x7f2f040c67e0 legacy=0x7f2f0406c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:45.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.645+0000 7f2f03fff640 1 -- 172.21.15.139:0/1382308034 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0415df20 con 0x7f2f040c7b90 2024-09-16T00:08:45.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.646+0000 7f2f03fff640 1 --2- 172.21.15.139:0/1382308034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f040c7b90 0x7f2f0415a810 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00097d0 tx=0x7f2ef002a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76924b8722f5016 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:45.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.646+0000 7f2f037fe640 1 -- 172.21.15.139:0/1382308034 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0037070 con 0x7f2f040c7b90 2024-09-16T00:08:45.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.646+0000 7f2f037fe640 1 -- 172.21.15.139:0/1382308034 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef0032440 con 0x7f2f040c7b90 2024-09-16T00:08:45.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.646+0000 7f2f037fe640 1 -- 172.21.15.139:0/1382308034 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef00327f0 con 0x7f2f040c7b90 2024-09-16T00:08:45.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.646+0000 7f2f0b152640 1 -- 172.21.15.139:0/1382308034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f040c7b90 msgr2=0x7f2f0415a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:45.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.646+0000 7f2f0b152640 1 --2- 172.21.15.139:0/1382308034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f040c7b90 0x7f2f0415a810 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00097d0 tx=0x7f2ef002a290 comp rx=0 tx=0).stop 2024-09-16T00:08:45.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.647+0000 7f2f0b152640 1 -- 172.21.15.139:0/1382308034 shutdown_connections 2024-09-16T00:08:45.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.647+0000 7f2f0b152640 1 --2- 172.21.15.139:0/1382308034 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f040c7b90 0x7f2f0415a810 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.647+0000 7f2f0b152640 1 -- 172.21.15.139:0/1382308034 >> 172.21.15.139:0/1382308034 conn(0x7f2f040d05f0 msgr2=0x7f2f040d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:45.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.647+0000 7f2f0b152640 1 -- 172.21.15.139:0/1382308034 shutdown_connections 2024-09-16T00:08:45.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.647+0000 7f2f0b152640 1 -- 172.21.15.139:0/1382308034 wait complete. 2024-09-16T00:08:45.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.648+0000 7f2f0b152640 1 Processor -- start 2024-09-16T00:08:45.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.648+0000 7f2f0b152640 1 -- start start 2024-09-16T00:08:45.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.649+0000 7f2f0b152640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f040c67e0 0x7f2f040b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:45.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.649+0000 7f2f0b152640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f040c7b90 0x7f2f040addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:45.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.649+0000 7f2f03fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f040c7b90 0x7f2f040addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:45.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.649+0000 7f2f08eee640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f040c67e0 0x7f2f040b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:45.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.649+0000 7f2f03fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f040c7b90 0x7f2f040addf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49432/0 (socket says 172.21.15.139:49432) 2024-09-16T00:08:45.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.649+0000 7f2f03fff640 1 -- 172.21.15.139:0/3437805213 learned_addr learned my addr 172.21.15.139:0/3437805213 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:45.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.649+0000 7f2f0b152640 1 --2- 172.21.15.139:0/3437805213 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0415ad50 0x7f2f04155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:45.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.649+0000 7f2f0b152640 1 -- 172.21.15.139:0/3437805213 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0415e850 con 0x7f2f040c7b90 2024-09-16T00:08:45.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.649+0000 7f2f0b152640 1 -- 172.21.15.139:0/3437805213 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0415e4e0 con 0x7f2f040c67e0 2024-09-16T00:08:45.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.649+0000 7f2f0b152640 1 -- 172.21.15.139:0/3437805213 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2f040cc560 con 0x7f2f0415ad50 2024-09-16T00:08:45.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.650+0000 7f2f096ef640 1 --2- 172.21.15.139:0/3437805213 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0415ad50 0x7f2f04155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:45.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.650+0000 7f2f03fff640 1 -- 172.21.15.139:0/3437805213 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0415ad50 msgr2=0x7f2f04155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:45.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.650+0000 7f2f03fff640 1 --2- 172.21.15.139:0/3437805213 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0415ad50 0x7f2f04155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.650+0000 7f2f03fff640 1 -- 172.21.15.139:0/3437805213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f040c67e0 msgr2=0x7f2f040b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:45.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.650+0000 7f2f03fff640 1 --2- 172.21.15.139:0/3437805213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f040c67e0 0x7f2f040b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.650+0000 7f2f03fff640 1 -- 172.21.15.139:0/3437805213 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f041f49f0 con 0x7f2f040c7b90 2024-09-16T00:08:45.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.650+0000 7f2f096ef640 1 --2- 172.21.15.139:0/3437805213 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0415ad50 0x7f2f04155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:08:45.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.650+0000 7f2f03fff640 1 --2- 172.21.15.139:0/3437805213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f040c7b90 0x7f2f040addf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00098f0 tx=0x7f2ef00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:45.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.650+0000 7f2f017fa640 1 -- 172.21.15.139:0/3437805213 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0037040 con 0x7f2f040c7b90 2024-09-16T00:08:45.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.650+0000 7f2f017fa640 1 -- 172.21.15.139:0/3437805213 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef0042070 con 0x7f2f040c7b90 2024-09-16T00:08:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.650+0000 7f2f017fa640 1 -- 172.21.15.139:0/3437805213 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0030ca0 con 0x7f2f040c7b90 2024-09-16T00:08:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.650+0000 7f2f0b152640 1 -- 172.21.15.139:0/3437805213 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f041f4710 con 0x7f2f040c7b90 2024-09-16T00:08:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.650+0000 7f2f0b152640 1 -- 172.21.15.139:0/3437805213 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f04157e50 con 0x7f2f040c7b90 2024-09-16T00:08:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.652+0000 7f2f0b152640 1 -- 172.21.15.139:0/3437805213 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f040c35a0 con 0x7f2f040c7b90 2024-09-16T00:08:45.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.652+0000 7f2f017fa640 1 -- 172.21.15.139:0/3437805213 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2ef0047460 con 0x7f2f040c7b90 2024-09-16T00:08:45.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.653+0000 7f2f017fa640 1 --2- 172.21.15.139:0/3437805213 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2ee806c2b0 0x7f2ee806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:45.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.653+0000 7f2f017fa640 1 -- 172.21.15.139:0/3437805213 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5915+0+0 (secure 0 0 0) 0x7f2ef00ac0a0 con 0x7f2f040c7b90 2024-09-16T00:08:45.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.653+0000 7f2f08eee640 1 --2- 172.21.15.139:0/3437805213 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2ee806c2b0 0x7f2ee806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:45.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.654+0000 7f2f08eee640 1 --2- 172.21.15.139:0/3437805213 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2ee806c2b0 0x7f2ee806e770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8005880 tx=0x7f2ef8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:45.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.657+0000 7f2f017fa640 1 -- 172.21.15.139:0/3437805213 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2ef007a7d0 con 0x7f2f040c7b90 2024-09-16T00:08:45.666 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:45.666 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:45.677 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:08:45.715 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:45.713+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:45.780 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:45.781 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:45.789 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:08:45.834 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:45.833+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:45.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.864+0000 7f2f0b152640 1 -- 172.21.15.139:0/3437805213 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f2f0415a810 con 0x7f2f040c7b90 2024-09-16T00:08:45.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.866+0000 7f2f017fa640 1 -- 172.21.15.139:0/3437805213 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) ==== 81+0+35490 (secure 0 0 0) 0x7f2ef007a0d0 con 0x7f2f040c7b90 2024-09-16T00:08:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:08:45.868 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 50, 2024-09-16T00:08:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: "fsid": "2d3366e5-6a59-4b59-93b2-615e72ac7015", 2024-09-16T00:08:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: "created": "2024-09-16T00:07:09.967833+0000", 2024-09-16T00:08:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: "modified": "2024-09-16T00:08:44.926559+0000", 2024-09-16T00:08:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "last_up_change": "2024-09-16T00:07:21.049678+0000", 2024-09-16T00:08:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "last_in_change": "2024-09-16T00:07:12.025620+0000", 2024-09-16T00:08:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T00:08:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_num": 5806080, 2024-09-16T00:08:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_set": [ 2024-09-16T00:08:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "nodeep-scrub", 2024-09-16T00:08:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "noscrub", 2024-09-16T00:08:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "pglog_hardlimit", 2024-09-16T00:08:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "purged_snapdirs", 2024-09-16T00:08:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_deletes", 2024-09-16T00:08:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "sortbitwise" 2024-09-16T00:08:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:08:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_version": 6, 2024-09-16T00:08:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T00:08:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T00:08:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T00:08:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_snapshot": "", 2024-09-16T00:08:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_max": 3, 2024-09-16T00:08:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "max_osd": 8, 2024-09-16T00:08:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "require_min_compat_client": "luminous", 2024-09-16T00:08:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "min_compat_client": "luminous", 2024-09-16T00:08:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "require_osd_release": "squid", 2024-09-16T00:08:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "allow_crimson": false, 2024-09-16T00:08:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "pools": [ 2024-09-16T00:08:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 1, 2024-09-16T00:08:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": ".mgr", 2024-09-16T00:08:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:22.056303+0000", 2024-09-16T00:08:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:08:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:08:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:08:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:08:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:08:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:08:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:08:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:08:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:08:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:08:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:08:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:08:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:08:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 1, 2024-09-16T00:08:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 1, 2024-09-16T00:08:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 1, 2024-09-16T00:08:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 1, 2024-09-16T00:08:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 1, 2024-09-16T00:08:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:08:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:08:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:08:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:08:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:08:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:08:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:08:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "18", 2024-09-16T00:08:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:08:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:08:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:08:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:08:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:08:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:08:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:08:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:08:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:08:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:08:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:08:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:08:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:08:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:08:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:08:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:08:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:08:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:08:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:08:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:08:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:08:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:08:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:08:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:08:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:08:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:08:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:08:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:08:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:08:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:08:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:08:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:08:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:08:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:08:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:08:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:08:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:08:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:08:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_max": 32, 2024-09-16T00:08:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 1 2024-09-16T00:08:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:08:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": {} 2024-09-16T00:08:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:08:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:08:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 8, 2024-09-16T00:08:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 8, 2024-09-16T00:08:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 0.25, 2024-09-16T00:08:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 2, 2024-09-16T00:08:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 2, 2024-09-16T00:08:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:08:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:08:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:08:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 2, 2024-09-16T00:08:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": "cephfs_metadata", 2024-09-16T00:08:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:30.768649+0000", 2024-09-16T00:08:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:08:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:08:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:08:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:08:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:08:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:08:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:08:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:08:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:08:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:08:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:08:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:08:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:08:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 32, 2024-09-16T00:08:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 32, 2024-09-16T00:08:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 32, 2024-09-16T00:08:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 32, 2024-09-16T00:08:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 32, 2024-09-16T00:08:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:08:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:08:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:08:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:08:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:08:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:08:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:08:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "21", 2024-09-16T00:08:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:08:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:08:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:08:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:08:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:08:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:08:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:08:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:08:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:08:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:08:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:08:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:08:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:08:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:08:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:08:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:08:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:08:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:08:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:08:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:08:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:08:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:08:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:08:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:08:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:08:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:08:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:08:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:08:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:08:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:08:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:08:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:08:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:08:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:08:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:08:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:08:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:08:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:08:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_bias": 4, 2024-09-16T00:08:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 16, 2024-09-16T00:08:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_priority": 5 2024-09-16T00:08:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:08:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs": { 2024-09-16T00:08:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": "cephfs" 2024-09-16T00:08:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:08:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:08:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 1.5, 2024-09-16T00:08:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 1.5, 2024-09-16T00:08:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 1, 2024-09-16T00:08:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 1.5, 2024-09-16T00:08:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 1.5, 2024-09-16T00:08:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:08:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:08:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:08:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 3, 2024-09-16T00:08:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": "cephfs_data", 2024-09-16T00:08:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:32.008209+0000", 2024-09-16T00:08:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:08:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:08:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:08:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:08:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:08:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:08:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:08:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:08:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:08:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:08:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:08:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:08:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:08:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 64, 2024-09-16T00:08:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 64, 2024-09-16T00:08:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 64, 2024-09-16T00:08:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 64, 2024-09-16T00:08:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 64, 2024-09-16T00:08:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:08:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:08:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:08:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:08:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:08:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:08:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:08:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "22", 2024-09-16T00:08:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:08:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:08:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:08:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:08:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:08:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:08:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:08:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:08:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:08:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:08:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:08:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:08:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:08:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:08:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:08:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:08:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:08:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:08:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:08:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:08:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:08:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:08:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:08:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:08:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:08:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:08:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:08:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:08:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:08:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:08:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:08:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:08:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:08:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:08:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:08:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:08:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:08:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:08:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 64, 2024-09-16T00:08:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: "target_size_ratio": 0.89999997615814209 2024-09-16T00:08:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:08:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs": { 2024-09-16T00:08:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: "data": "cephfs" 2024-09-16T00:08:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:08:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:08:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 1.3799999952316284, 2024-09-16T00:08:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 1.3799999952316284, 2024-09-16T00:08:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 1, 2024-09-16T00:08:45.891 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-16T00:08:45.891 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-16T00:08:45.891 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:08:45.891 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:08:45.891 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:08:45.891 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.891 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.892 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:08:45.892 INFO:teuthology.orchestra.run.smithi139.stdout: "osds": [ 2024-09-16T00:08:45.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.892 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 0, 2024-09-16T00:08:45.892 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "83699a05-4937-43c3-aefc-b669bed9bb6f", 2024-09-16T00:08:45.892 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:08:45.892 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:08:45.892 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:08:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:08:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:08:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:08:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 14, 2024-09-16T00:08:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 37, 2024-09-16T00:08:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:08:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:08:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:08:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6816", 2024-09-16T00:08:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:08:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6817", 2024-09-16T00:08:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:08:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:08:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6818", 2024-09-16T00:08:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:08:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6819", 2024-09-16T00:08:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:08:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:08:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6822", 2024-09-16T00:08:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:08:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6823", 2024-09-16T00:08:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:08:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:08:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6820", 2024-09-16T00:08:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:08:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6821", 2024-09-16T00:08:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:08:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6817/1974901438", 2024-09-16T00:08:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6819/1974901438", 2024-09-16T00:08:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6823/1974901438", 2024-09-16T00:08:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6821/1974901438", 2024-09-16T00:08:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:08:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:08:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:08:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 1, 2024-09-16T00:08:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "9c04991e-9695-4bbb-8011-bf613bf404b3", 2024-09-16T00:08:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:08:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:08:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:08:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:08:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:08:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:08:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 14, 2024-09-16T00:08:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:08:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:08:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:08:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:08:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6808", 2024-09-16T00:08:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:08:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6809", 2024-09-16T00:08:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:08:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:08:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6810", 2024-09-16T00:08:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:08:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6811", 2024-09-16T00:08:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:08:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:08:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6814", 2024-09-16T00:08:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:08:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6815", 2024-09-16T00:08:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:08:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:08:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6812", 2024-09-16T00:08:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:08:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6813", 2024-09-16T00:08:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:08:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6809/2232380998", 2024-09-16T00:08:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6811/2232380998", 2024-09-16T00:08:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6815/2232380998", 2024-09-16T00:08:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6813/2232380998", 2024-09-16T00:08:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:08:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:08:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:08:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 2, 2024-09-16T00:08:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "8048fc64-72b3-4866-ac62-6a8b2bf6f5c8", 2024-09-16T00:08:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:08:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:08:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:08:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:08:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:08:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:08:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 13, 2024-09-16T00:08:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:08:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:08:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:08:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:08:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6800", 2024-09-16T00:08:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:08:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6801", 2024-09-16T00:08:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:08:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:08:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6802", 2024-09-16T00:08:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:08:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6803", 2024-09-16T00:08:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:08:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:08:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6806", 2024-09-16T00:08:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:08:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6807", 2024-09-16T00:08:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:08:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:08:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6804", 2024-09-16T00:08:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:08:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6805", 2024-09-16T00:08:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:08:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6801/4180588905", 2024-09-16T00:08:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6803/4180588905", 2024-09-16T00:08:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6807/4180588905", 2024-09-16T00:08:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6805/4180588905", 2024-09-16T00:08:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:08:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:08:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:08:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 3, 2024-09-16T00:08:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "c34ac77b-354c-4884-902d-3e2a84f5eb27", 2024-09-16T00:08:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:08:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:08:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:08:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:08:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:08:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:08:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:08:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:08:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:08:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:08:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:08:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6826", 2024-09-16T00:08:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:08:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6827", 2024-09-16T00:08:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:08:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:08:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6828", 2024-09-16T00:08:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:08:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6829", 2024-09-16T00:08:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:08:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:08:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6832", 2024-09-16T00:08:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:08:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6833", 2024-09-16T00:08:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:08:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:08:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6830", 2024-09-16T00:08:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:08:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6831", 2024-09-16T00:08:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:08:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6827/3947768492", 2024-09-16T00:08:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6829/3947768492", 2024-09-16T00:08:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6833/3947768492", 2024-09-16T00:08:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6831/3947768492", 2024-09-16T00:08:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:08:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:08:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:08:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 4, 2024-09-16T00:08:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "ba9a7c93-9672-43e1-8c98-2878cdf7f548", 2024-09-16T00:08:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:08:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:08:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:08:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:08:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:08:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:08:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:08:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:08:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:08:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:08:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:08:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6816", 2024-09-16T00:08:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:08:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6817", 2024-09-16T00:08:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:08:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:08:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6818", 2024-09-16T00:08:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:08:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6819", 2024-09-16T00:08:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:08:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:08:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6822", 2024-09-16T00:08:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:08:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6823", 2024-09-16T00:08:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:08:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:08:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6820", 2024-09-16T00:08:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:08:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6821", 2024-09-16T00:08:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:08:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6817/642632916", 2024-09-16T00:08:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6819/642632916", 2024-09-16T00:08:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6823/642632916", 2024-09-16T00:08:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6821/642632916", 2024-09-16T00:08:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:08:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:08:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:08:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 5, 2024-09-16T00:08:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "46a0916b-474c-411b-84ad-86f27d4cbf53", 2024-09-16T00:08:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:08:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:08:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:08:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:08:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:08:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:08:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:08:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 37, 2024-09-16T00:08:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:08:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:08:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:08:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6800", 2024-09-16T00:08:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:08:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6801", 2024-09-16T00:08:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:08:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:08:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6802", 2024-09-16T00:08:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:08:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6803", 2024-09-16T00:08:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:08:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:08:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6806", 2024-09-16T00:08:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:08:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6807", 2024-09-16T00:08:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:08:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:08:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6804", 2024-09-16T00:08:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:08:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6805", 2024-09-16T00:08:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:08:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6801/1051778383", 2024-09-16T00:08:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6803/1051778383", 2024-09-16T00:08:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6807/1051778383", 2024-09-16T00:08:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6805/1051778383", 2024-09-16T00:08:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:08:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:08:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:08:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 6, 2024-09-16T00:08:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "9b6b33cc-d6d9-4a0a-8143-526073a58b93", 2024-09-16T00:08:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:08:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:08:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:08:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:08:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:08:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:08:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:08:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:08:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:08:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:08:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:08:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6824", 2024-09-16T00:08:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:08:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6825", 2024-09-16T00:08:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:08:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:08:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6826", 2024-09-16T00:08:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:08:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6827", 2024-09-16T00:08:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:08:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:08:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6830", 2024-09-16T00:08:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:08:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6831", 2024-09-16T00:08:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:08:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:08:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6828", 2024-09-16T00:08:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:08:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6829", 2024-09-16T00:08:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:08:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6825/1908495807", 2024-09-16T00:08:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6827/1908495807", 2024-09-16T00:08:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6831/1908495807", 2024-09-16T00:08:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6829/1908495807", 2024-09-16T00:08:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:08:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:08:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:08:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 7, 2024-09-16T00:08:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "50bcd808-3d3e-4a72-88e4-30decb53cfea", 2024-09-16T00:08:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:08:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:08:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:08:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:08:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:08:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:08:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:08:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:08:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:08:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:08:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:08:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6808", 2024-09-16T00:08:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:08:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6809", 2024-09-16T00:08:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:08:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:08:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6810", 2024-09-16T00:08:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:08:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6811", 2024-09-16T00:08:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:08:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:08:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6814", 2024-09-16T00:08:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:08:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6815", 2024-09-16T00:08:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:08:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:08:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:08:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:08:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6812", 2024-09-16T00:08:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:08:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:08:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6813", 2024-09-16T00:08:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:08:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6809/1158838808", 2024-09-16T00:08:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6811/1158838808", 2024-09-16T00:08:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6815/1158838808", 2024-09-16T00:08:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6813/1158838808", 2024-09-16T00:08:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:08:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:08:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:08:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:08:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_xinfo": [ 2024-09-16T00:08:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 0, 2024-09-16T00:08:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:08:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:08:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:08:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:08:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:08:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.843117+0000", 2024-09-16T00:08:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:08:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 1, 2024-09-16T00:08:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:08:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:08:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:08:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:08:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:08:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.622789+0000", 2024-09-16T00:08:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:08:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 2, 2024-09-16T00:08:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:08:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:08:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:08:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:08:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:08:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:17.919457+0000", 2024-09-16T00:08:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:08:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 3, 2024-09-16T00:08:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:08:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:08:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:08:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:08:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:08:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:19.330341+0000", 2024-09-16T00:08:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:08:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 4, 2024-09-16T00:08:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:08:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:08:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:08:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:08:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:08:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.815640+0000", 2024-09-16T00:08:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:08:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 5, 2024-09-16T00:08:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:08:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:08:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:08:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:08:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:08:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.083046+0000", 2024-09-16T00:08:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:08:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 6, 2024-09-16T00:08:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:08:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:08:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:08:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:08:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:08:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.551499+0000", 2024-09-16T00:08:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:08:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 7, 2024-09-16T00:08:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:08:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:08:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:08:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:08:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:08:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.377179+0000", 2024-09-16T00:08:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:08:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:08:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap": [], 2024-09-16T00:08:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap_items": [ 2024-09-16T00:08:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.d", 2024-09-16T00:08:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:08:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 5, 2024-09-16T00:08:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 1 2024-09-16T00:08:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.1f", 2024-09-16T00:08:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:08:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 0, 2024-09-16T00:08:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:08:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.21", 2024-09-16T00:08:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:08:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 5, 2024-09-16T00:08:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 1 2024-09-16T00:08:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.22", 2024-09-16T00:08:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:08:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 5, 2024-09-16T00:08:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:08:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.38", 2024-09-16T00:08:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:08:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 3, 2024-09-16T00:08:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:08:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.3b", 2024-09-16T00:08:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:08:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 7, 2024-09-16T00:08:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 1 2024-09-16T00:08:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:08:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap_primaries": [], 2024-09-16T00:08:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_temp": [], 2024-09-16T00:08:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_temp": [], 2024-09-16T00:08:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklist": {}, 2024-09-16T00:08:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: "range_blocklist": {}, 2024-09-16T00:08:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profiles": { 2024-09-16T00:08:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: "default": { 2024-09-16T00:08:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: "crush-failure-domain": "osd", 2024-09-16T00:08:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: "k": "2", 2024-09-16T00:08:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: "m": "1", 2024-09-16T00:08:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: "plugin": "jerasure", 2024-09-16T00:08:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: "technique": "reed_sol_van" 2024-09-16T00:08:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps_queue": [], 2024-09-16T00:08:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "new_removed_snaps": [], 2024-09-16T00:08:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "new_purged_snaps": [], 2024-09-16T00:08:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_node_flags": {}, 2024-09-16T00:08:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "device_class_flags": {}, 2024-09-16T00:08:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode": { 2024-09-16T00:08:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode_enabled": false, 2024-09-16T00:08:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_bucket_count": 0, 2024-09-16T00:08:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "degraded_stretch_mode": 0, 2024-09-16T00:08:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "recovering_stretch_mode": 0, 2024-09-16T00:08:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode_bucket": 0 2024-09-16T00:08:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:45.950 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:45.952 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:45.953 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:45.953 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:08:45.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.868+0000 7f2f0b152640 1 -- 172.21.15.139:0/3437805213 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2ee806c2b0 msgr2=0x7f2ee806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:45.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.868+0000 7f2f0b152640 1 --2- 172.21.15.139:0/3437805213 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2ee806c2b0 0x7f2ee806e770 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8005880 tx=0x7f2ef8009040 comp rx=0 tx=0).stop 2024-09-16T00:08:45.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.868+0000 7f2f0b152640 1 -- 172.21.15.139:0/3437805213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f040c7b90 msgr2=0x7f2f040addf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:45.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.868+0000 7f2f0b152640 1 --2- 172.21.15.139:0/3437805213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f040c7b90 0x7f2f040addf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00098f0 tx=0x7f2ef00046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.868+0000 7f2f0b152640 1 -- 172.21.15.139:0/3437805213 shutdown_connections 2024-09-16T00:08:45.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.868+0000 7f2f0b152640 1 --2- 172.21.15.139:0/3437805213 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2ee806c2b0 0x7f2ee806e770 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.868+0000 7f2f0b152640 1 --2- 172.21.15.139:0/3437805213 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0415ad50 0x7f2f04155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.868+0000 7f2f0b152640 1 --2- 172.21.15.139:0/3437805213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f040c7b90 0x7f2f040addf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.868+0000 7f2f0b152640 1 --2- 172.21.15.139:0/3437805213 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f040c67e0 0x7f2f040b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:45.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.868+0000 7f2f0b152640 1 -- 172.21.15.139:0/3437805213 >> 172.21.15.139:0/3437805213 conn(0x7f2f040d05f0 msgr2=0x7f2f040cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:45.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.868+0000 7f2f0b152640 1 -- 172.21.15.139:0/3437805213 shutdown_connections 2024-09-16T00:08:45.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:45.868+0000 7f2f0b152640 1 -- 172.21.15.139:0/3437805213 wait complete. 2024-09-16T00:08:45.956 INFO:tasks.thrashosds.thrasher:j is {'epoch': 50, 'fsid': '2d3366e5-6a59-4b59-93b2-615e72ac7015', 'created': '2024-09-16T00:07:09.967833+0000', 'modified': '2024-09-16T00:08:44.926559+0000', 'last_up_change': '2024-09-16T00:07:21.049678+0000', 'last_in_change': '2024-09-16T00:07:12.025620+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:07:22.056303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T00:07:30.768649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T00:07:32.008209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '83699a05-4937-43c3-aefc-b669bed9bb6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6816', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6817', 'nonce': 1974901438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6818', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6819', 'nonce': 1974901438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6822', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6823', 'nonce': 1974901438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6820', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6821', 'nonce': 1974901438}]}, 'public_addr': '172.21.15.139:6817/1974901438', 'cluster_addr': '172.21.15.139:6819/1974901438', 'heartbeat_back_addr': '172.21.15.139:6823/1974901438', 'heartbeat_front_addr': '172.21.15.139:6821/1974901438', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9c04991e-9695-4bbb-8011-bf613bf404b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6808', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6809', 'nonce': 2232380998}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6810', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6811', 'nonce': 2232380998}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6814', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6815', 'nonce': 2232380998}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6812', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6813', 'nonce': 2232380998}]}, 'public_addr': '172.21.15.139:6809/2232380998', 'cluster_addr': '172.21.15.139:6811/2232380998', 'heartbeat_back_addr': '172.21.15.139:6815/2232380998', 'heartbeat_front_addr': '172.21.15.139:6813/2232380998', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8048fc64-72b3-4866-ac62-6a8b2bf6f5c8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6800', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6801', 'nonce': 4180588905}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6802', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6803', 'nonce': 4180588905}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6806', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6807', 'nonce': 4180588905}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6804', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6805', 'nonce': 4180588905}]}, 'public_addr': '172.21.15.139:6801/4180588905', 'cluster_addr': '172.21.15.139:6803/4180588905', 'heartbeat_back_addr': '172.21.15.139:6807/4180588905', 'heartbeat_front_addr': '172.21.15.139:6805/4180588905', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c34ac77b-354c-4884-902d-3e2a84f5eb27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6826', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6827', 'nonce': 3947768492}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6828', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6829', 'nonce': 3947768492}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6832', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6833', 'nonce': 3947768492}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6830', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6831', 'nonce': 3947768492}]}, 'public_addr': '172.21.15.139:6827/3947768492', 'cluster_addr': '172.21.15.139:6829/3947768492', 'heartbeat_back_addr': '172.21.15.139:6833/3947768492', 'heartbeat_front_addr': '172.21.15.139:6831/3947768492', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ba9a7c93-9672-43e1-8c98-2878cdf7f548', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6816', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6817', 'nonce': 642632916}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6818', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 642632916}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 642632916}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6821', 'nonce': 642632916}]}, 'public_addr': '172.21.15.164:6817/642632916', 'cluster_addr': '172.21.15.164:6819/642632916', 'heartbeat_back_addr': '172.21.15.164:6823/642632916', 'heartbeat_front_addr': '172.21.15.164:6821/642632916', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '46a0916b-474c-411b-84ad-86f27d4cbf53', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 1051778383}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 1051778383}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 1051778383}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 1051778383}]}, 'public_addr': '172.21.15.164:6801/1051778383', 'cluster_addr': '172.21.15.164:6803/1051778383', 'heartbeat_back_addr': '172.21.15.164:6807/1051778383', 'heartbeat_front_addr': '172.21.15.164:6805/1051778383', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b6b33cc-d6d9-4a0a-8143-526073a58b93', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6824', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 1908495807}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6827', 'nonce': 1908495807}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6830', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 1908495807}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 1908495807}]}, 'public_addr': '172.21.15.164:6825/1908495807', 'cluster_addr': '172.21.15.164:6827/1908495807', 'heartbeat_back_addr': '172.21.15.164:6831/1908495807', 'heartbeat_front_addr': '172.21.15.164:6829/1908495807', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '50bcd808-3d3e-4a72-88e4-30decb53cfea', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6809', 'nonce': 1158838808}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 1158838808}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6815', 'nonce': 1158838808}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6812', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 1158838808}]}, 'public_addr': '172.21.15.164:6809/1158838808', 'cluster_addr': '172.21.15.164:6811/1158838808', 'heartbeat_back_addr': '172.21.15.164:6815/1158838808', 'heartbeat_front_addr': '172.21.15.164:6813/1158838808', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.843117+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.622789+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:17.919457+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:19.330341+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.815640+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.083046+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.551499+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.377179+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.d', 'mappings': [{'from': 5, 'to': 1}]}, {'pgid': '3.1f', 'mappings': [{'from': 0, 'to': 2}]}, {'pgid': '3.21', 'mappings': [{'from': 5, 'to': 1}]}, {'pgid': '3.22', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '3.38', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '3.3b', 'mappings': [{'from': 7, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T00:08:45.956 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.d 2024-09-16T00:08:45.956 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.d 2024-09-16T00:08:45.957 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:45.951+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:46.052 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:46.051+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:46.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.097+0000 7f1d668e4640 1 Processor -- start 2024-09-16T00:08:46.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.097+0000 7f1d668e4640 1 -- start start 2024-09-16T00:08:46.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.097+0000 7f1d668e4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d60154c20 0x7f1d6015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:46.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.097+0000 7f1d668e4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1d60068060 0x7f1d600c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:46.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.097+0000 7f1d668e4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d600cc4b0 con 0x7f1d600ccc90 2024-09-16T00:08:46.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.097+0000 7f1d668e4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1d6015b7c0 con 0x7f1d60068060 2024-09-16T00:08:46.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.097+0000 7f1d668e4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1d600cc260 con 0x7f1d60154c20 2024-09-16T00:08:46.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.098+0000 7f1d5f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1d60068060 0x7f1d600c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:46.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.098+0000 7f1d64e81640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f1d600ccc90 0x7f1d600cac00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:50668/0 (socket says 172.21.15.139:50668) 2024-09-16T00:08:46.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.098+0000 7f1d64e81640 1 -- 172.21.15.139:0/2267626318 learned_addr learned my addr 172.21.15.139:0/2267626318 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:46.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.098+0000 7f1d5ffff640 1 --2- 172.21.15.139:0/2267626318 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d60154c20 0x7f1d6015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:46.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.098+0000 7f1d5effd640 1 -- 172.21.15.139:0/2267626318 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 285359712 0 0) 0x7f1d600cc4b0 con 0x7f1d600ccc90 2024-09-16T00:08:46.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.098+0000 7f1d5f7fe640 1 -- 172.21.15.139:0/2267626318 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d60154c20 msgr2=0x7f1d6015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:46.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.098+0000 7f1d5f7fe640 1 --2- 172.21.15.139:0/2267626318 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d60154c20 0x7f1d6015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:46.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.098+0000 7f1d5f7fe640 1 -- 172.21.15.139:0/2267626318 >> v1:172.21.15.164:6789/0 conn(0x7f1d600ccc90 legacy=0x7f1d600cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:46.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.098+0000 7f1d5f7fe640 1 -- 172.21.15.139:0/2267626318 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d600cc790 con 0x7f1d60068060 2024-09-16T00:08:46.102 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:46.102 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:46.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.099+0000 7f1d5f7fe640 1 --2- 172.21.15.139:0/2267626318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60068060 0x7f1d600c5120 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009790 tx=0x7f1d5402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddd164ee453922e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:46.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.099+0000 7f1d5effd640 1 -- 172.21.15.139:0/2267626318 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54037070 con 0x7f1d60068060 2024-09-16T00:08:46.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.099+0000 7f1d5effd640 1 -- 172.21.15.139:0/2267626318 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d54032440 con 0x7f1d60068060 2024-09-16T00:08:46.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.099+0000 7f1d5effd640 1 -- 172.21.15.139:0/2267626318 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54032810 con 0x7f1d60068060 2024-09-16T00:08:46.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.100+0000 7f1d668e4640 1 -- 172.21.15.139:0/2267626318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60068060 msgr2=0x7f1d600c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:46.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.100+0000 7f1d668e4640 1 --2- 172.21.15.139:0/2267626318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60068060 0x7f1d600c5120 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009790 tx=0x7f1d5402a200 comp rx=0 tx=0).stop 2024-09-16T00:08:46.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.100+0000 7f1d668e4640 1 -- 172.21.15.139:0/2267626318 shutdown_connections 2024-09-16T00:08:46.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.100+0000 7f1d668e4640 1 --2- 172.21.15.139:0/2267626318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60068060 0x7f1d600c5120 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:46.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.100+0000 7f1d668e4640 1 --2- 172.21.15.139:0/2267626318 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d60154c20 0x7f1d6015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:46.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.100+0000 7f1d668e4640 1 -- 172.21.15.139:0/2267626318 >> 172.21.15.139:0/2267626318 conn(0x7f1d6006ca40 msgr2=0x7f1d60067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:46.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.100+0000 7f1d668e4640 1 -- 172.21.15.139:0/2267626318 shutdown_connections 2024-09-16T00:08:46.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.100+0000 7f1d668e4640 1 -- 172.21.15.139:0/2267626318 wait complete. 2024-09-16T00:08:46.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.102+0000 7f1d668e4640 1 Processor -- start 2024-09-16T00:08:46.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.102+0000 7f1d668e4640 1 -- start start 2024-09-16T00:08:46.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.103+0000 7f1d668e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60068060 0x7f1d600d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:46.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.103+0000 7f1d668e4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d600ccc90 0x7f1d60159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:46.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.103+0000 7f1d668e4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d60154c20 0x7f1d6015cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:46.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.103+0000 7f1d668e4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6015e3b0 con 0x7f1d60068060 2024-09-16T00:08:46.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.103+0000 7f1d668e4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6015e0d0 con 0x7f1d60154c20 2024-09-16T00:08:46.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.103+0000 7f1d668e4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6015e690 con 0x7f1d600ccc90 2024-09-16T00:08:46.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.103+0000 7f1d5ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60068060 0x7f1d600d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:46.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.103+0000 7f1d5ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60068060 0x7f1d600d08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49436/0 (socket says 172.21.15.139:49436) 2024-09-16T00:08:46.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.103+0000 7f1d5ffff640 1 -- 172.21.15.139:0/2403283816 learned_addr learned my addr 172.21.15.139:0/2403283816 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:46.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.103+0000 7f1d5f7fe640 1 --2- 172.21.15.139:0/2403283816 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d600ccc90 0x7f1d60159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:46.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.104+0000 7f1d64e81640 1 --2- 172.21.15.139:0/2403283816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d60154c20 0x7f1d6015cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:46.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.104+0000 7f1d5ffff640 1 -- 172.21.15.139:0/2403283816 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d600ccc90 msgr2=0x7f1d60159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:46.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.104+0000 7f1d5ffff640 1 --2- 172.21.15.139:0/2403283816 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d600ccc90 0x7f1d60159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:46.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.104+0000 7f1d5ffff640 1 -- 172.21.15.139:0/2403283816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d60154c20 msgr2=0x7f1d6015cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:46.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.104+0000 7f1d5ffff640 1 --2- 172.21.15.139:0/2403283816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d60154c20 0x7f1d6015cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:46.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.104+0000 7f1d5ffff640 1 -- 172.21.15.139:0/2403283816 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d601f4b60 con 0x7f1d60068060 2024-09-16T00:08:46.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.104+0000 7f1d5ffff640 1 --2- 172.21.15.139:0/2403283816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60068060 0x7f1d600d08c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000e870 tx=0x7f1d5000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:46.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.104+0000 7f1d5cff9640 1 -- 172.21.15.139:0/2403283816 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50017070 con 0x7f1d60068060 2024-09-16T00:08:46.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.104+0000 7f1d5cff9640 1 -- 172.21.15.139:0/2403283816 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d5000c940 con 0x7f1d60068060 2024-09-16T00:08:46.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.104+0000 7f1d668e4640 1 -- 172.21.15.139:0/2403283816 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d601f50e0 con 0x7f1d60068060 2024-09-16T00:08:46.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.104+0000 7f1d5cff9640 1 -- 172.21.15.139:0/2403283816 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5000cd30 con 0x7f1d60068060 2024-09-16T00:08:46.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.104+0000 7f1d668e4640 1 -- 172.21.15.139:0/2403283816 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d600ce470 con 0x7f1d60068060 2024-09-16T00:08:46.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.106+0000 7f1d668e4640 1 -- 172.21.15.139:0/2403283816 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d600686b0 con 0x7f1d60068060 2024-09-16T00:08:46.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.106+0000 7f1d5cff9640 1 -- 172.21.15.139:0/2403283816 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1d5001a070 con 0x7f1d60068060 2024-09-16T00:08:46.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.107+0000 7f1d5cff9640 1 --2- 172.21.15.139:0/2403283816 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1d4c06c2b0 0x7f1d4c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:46.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.107+0000 7f1d5cff9640 1 -- 172.21.15.139:0/2403283816 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5915+0+0 (secure 0 0 0) 0x7f1d5008b390 con 0x7f1d60068060 2024-09-16T00:08:46.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.107+0000 7f1d5f7fe640 1 --2- 172.21.15.139:0/2403283816 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1d4c06c2b0 0x7f1d4c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:46.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.107+0000 7f1d5f7fe640 1 --2- 172.21.15.139:0/2403283816 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1d4c06c2b0 0x7f1d4c06e770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1d54034000 tx=0x7f1d5402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:46.111 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:08:46.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.115+0000 7f1d5cff9640 1 -- 172.21.15.139:0/2403283816 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1d500593e0 con 0x7f1d60068060 2024-09-16T00:08:46.230 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:46.230 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:46.240 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:08:46.254 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:46.152+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:46.254 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:46.253+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:46.354 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:46.353+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:46.390 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:46.390 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:46.400 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:08:46.456 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:46.454+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:46.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.476+0000 7f1d668e4640 1 -- 172.21.15.139:0/2403283816 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd rm-pg-upmap-items", "pgid": "3.d"} v 0) -- 0x7f1d600cca70 con 0x7f1d60068060 2024-09-16T00:08:46.507 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:46.508 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:46.517 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:08:46.558 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:46.557+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:46.658 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:46.656+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:46.666 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:46.666 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:46.676 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:08:46.758 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:46.758+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:46.759 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:08:46.836 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:46.837 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:46.849 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:08:46.859 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:46.858+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:46.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.870+0000 7fbb2719f640 1 Processor -- start 2024-09-16T00:08:46.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.871+0000 7fbb2719f640 1 -- start start 2024-09-16T00:08:46.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.872+0000 7fbb2719f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbb2015c700 0x7fbb20164050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:46.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.872+0000 7fbb2719f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb20071830 con 0x7fbb200c8070 2024-09-16T00:08:46.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.872+0000 7fbb24f3b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fbb200c8070 0x7fbb200c5500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45598/0 (socket says 172.21.15.139:45598) 2024-09-16T00:08:46.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.872+0000 7fbb24f3b640 1 -- 172.21.15.139:0/3473383379 learned_addr learned my addr 172.21.15.139:0/3473383379 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:46.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.872+0000 7fbb2719f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb20071b10 con 0x7fbb200c72a0 2024-09-16T00:08:46.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.872+0000 7fbb2573c640 1 --2- 172.21.15.139:0/3473383379 >> v2:172.21.15.164:3300/0 conn(0x7fbb2015c700 0x7fbb20164050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:46.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.873+0000 7fbb2719f640 1 -- 172.21.15.139:0/3473383379 --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fbb2015bcb0 con 0x7fbb2015c700 2024-09-16T00:08:46.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.873+0000 7fbb1f7fe640 1 -- 172.21.15.139:0/3473383379 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4261084418 0 0) 0x7fbb20071b10 con 0x7fbb200c72a0 2024-09-16T00:08:46.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.873+0000 7fbb1f7fe640 1 -- 172.21.15.139:0/3473383379 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb10003520 con 0x7fbb200c72a0 2024-09-16T00:08:46.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.873+0000 7fbb1f7fe640 1 -- 172.21.15.139:0/3473383379 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 491100344 0 0) 0x7fbb20071830 con 0x7fbb200c8070 2024-09-16T00:08:46.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.873+0000 7fbb1f7fe640 1 -- 172.21.15.139:0/3473383379 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb20071b10 con 0x7fbb200c8070 2024-09-16T00:08:46.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.873+0000 7fbb2573c640 1 -- 172.21.15.139:0/3473383379 >> v1:172.21.15.164:6789/0 conn(0x7fbb200c72a0 legacy=0x7fbb200c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:46.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.873+0000 7fbb2573c640 1 -- 172.21.15.139:0/3473383379 >> v1:172.21.15.139:6789/0 conn(0x7fbb200c8070 legacy=0x7fbb200c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:46.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.873+0000 7fbb2573c640 1 -- 172.21.15.139:0/3473383379 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb20071df0 con 0x7fbb2015c700 2024-09-16T00:08:46.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.874+0000 7fbb2573c640 1 --2- 172.21.15.139:0/3473383379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb2015c700 0x7fbb20164050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009d90 tx=0x7fbb1802a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e22cd8d85414127 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:46.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.874+0000 7fbb1f7fe640 1 -- 172.21.15.139:0/3473383379 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18035020 con 0x7fbb2015c700 2024-09-16T00:08:46.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.874+0000 7fbb1f7fe640 1 -- 172.21.15.139:0/3473383379 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb18007540 con 0x7fbb2015c700 2024-09-16T00:08:46.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.874+0000 7fbb1f7fe640 1 -- 172.21.15.139:0/3473383379 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c001670 con 0x7fbb2015c700 2024-09-16T00:08:46.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.875+0000 7fbb2719f640 1 -- 172.21.15.139:0/3473383379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb2015c700 msgr2=0x7fbb20164050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:46.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.875+0000 7fbb2719f640 1 --2- 172.21.15.139:0/3473383379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb2015c700 0x7fbb20164050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009d90 tx=0x7fbb1802a730 comp rx=0 tx=0).stop 2024-09-16T00:08:46.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.875+0000 7fbb2719f640 1 -- 172.21.15.139:0/3473383379 shutdown_connections 2024-09-16T00:08:46.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.875+0000 7fbb2719f640 1 --2- 172.21.15.139:0/3473383379 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb2015c700 0x7fbb20164050 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:46.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.875+0000 7fbb2719f640 1 -- 172.21.15.139:0/3473383379 >> 172.21.15.139:0/3473383379 conn(0x7fbb2006fb00 msgr2=0x7fbb200738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:46.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.876+0000 7fbb2719f640 1 -- 172.21.15.139:0/3473383379 shutdown_connections 2024-09-16T00:08:46.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.876+0000 7fbb2719f640 1 -- 172.21.15.139:0/3473383379 wait complete. 2024-09-16T00:08:46.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.877+0000 7fbb2719f640 1 Processor -- start 2024-09-16T00:08:46.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.877+0000 7fbb2719f640 1 -- start start 2024-09-16T00:08:46.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.878+0000 7fbb2719f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb200c72a0 0x7fbb201e2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:46.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.878+0000 7fbb2719f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb200c8070 0x7fbb200c2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:46.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.878+0000 7fbb2719f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb2015c700 0x7fbb2014b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:46.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.878+0000 7fbb2719f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2015cb50 con 0x7fbb200c8070 2024-09-16T00:08:46.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.878+0000 7fbb2719f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2015c1e0 con 0x7fbb200c72a0 2024-09-16T00:08:46.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.878+0000 7fbb2719f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbb200c8450 con 0x7fbb2015c700 2024-09-16T00:08:46.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.878+0000 7fbb1ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb200c8070 0x7fbb200c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:46.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.878+0000 7fbb24f3b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb200c72a0 0x7fbb201e2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:46.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.878+0000 7fbb1ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb200c8070 0x7fbb200c2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49452/0 (socket says 172.21.15.139:49452) 2024-09-16T00:08:46.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.878+0000 7fbb1ffff640 1 -- 172.21.15.139:0/1851320694 learned_addr learned my addr 172.21.15.139:0/1851320694 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:46.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.878+0000 7fbb2573c640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb2015c700 0x7fbb2014b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:46.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.879+0000 7fbb2573c640 1 -- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb200c72a0 msgr2=0x7fbb201e2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:46.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.879+0000 7fbb2573c640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb200c72a0 0x7fbb201e2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:46.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.879+0000 7fbb2573c640 1 -- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb200c8070 msgr2=0x7fbb200c2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:46.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.879+0000 7fbb2573c640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb200c8070 0x7fbb200c2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:46.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.879+0000 7fbb2573c640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb201e2ab0 con 0x7fbb2015c700 2024-09-16T00:08:46.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.879+0000 7fbb2573c640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb2015c700 0x7fbb2014b2e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbb18031000 tx=0x7fbb18004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:46.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.879+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1803f070 con 0x7fbb2015c700 2024-09-16T00:08:46.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.879+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb18031bc0 con 0x7fbb2015c700 2024-09-16T00:08:46.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.880+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb18035070 con 0x7fbb2015c700 2024-09-16T00:08:46.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.880+0000 7fbb2719f640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb201f2e80 con 0x7fbb2015c700 2024-09-16T00:08:46.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.880+0000 7fbb2719f640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb201f2ba0 con 0x7fbb2015c700 2024-09-16T00:08:46.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.881+0000 7fbb2719f640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fbb200cc0c0 con 0x7fbb2015c700 2024-09-16T00:08:46.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.882+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbb18003990 con 0x7fbb2015c700 2024-09-16T00:08:46.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.882+0000 7fbb1d7fa640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb0406c2e0 0x7fbb0406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:46.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.883+0000 7fbb24f3b640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb0406c2e0 0x7fbb0406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:46.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.883+0000 7fbb24f3b640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb0406c2e0 0x7fbb0406e7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbb14000f60 tx=0x7fbb14007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:46.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.883+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(50..50 src has 1..50) ==== 5915+0+0 (secure 0 0 0) 0x7fbb180ac220 con 0x7fbb2015c700 2024-09-16T00:08:46.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.883+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v50) ==== 54+0+16 (secure 0 0 0) 0x7fbb200cc0c0 con 0x7fbb2015c700 2024-09-16T00:08:46.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.884+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbaf8001650 0x7fbaf8003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:46.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.884+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbaf8006bb0 con 0x7fbaf8001650 2024-09-16T00:08:46.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.884+0000 7fbb1ffff640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbaf8001650 0x7fbaf8003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:46.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.884+0000 7fbb1ffff640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbaf8001650 0x7fbaf8003b10 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:46.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.885+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf8006bb0 con 0x7fbaf8001650 2024-09-16T00:08:46.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.915+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbaf8005c60 con 0x7fbaf8001650 2024-09-16T00:08:46.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.917+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbaf8005c60 con 0x7fbaf8001650 2024-09-16T00:08:46.919 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:46.919 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:46.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.917+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbaf8004190 0x7fbaf8004620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:46.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.917+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fbaf8004c10 con 0x7fbaf8004190 2024-09-16T00:08:46.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.919+0000 7fbb24f3b640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbaf8004190 0x7fbaf8004620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:46.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.919+0000 7fbb24f3b640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbaf8004190 0x7fbaf8004620 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:46.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.920+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf8004c10 con 0x7fbaf8004190 2024-09-16T00:08:46.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.940+0000 7f1d5cff9640 1 -- 172.21.15.139:0/2403283816 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd rm-pg-upmap-items", "pgid": "3.d"}]=0 clear 3.d pg_upmap_items mapping v51) ==== 116+0+0 (secure 0 0 0) 0x7f1d5005d6e0 con 0x7f1d60068060 2024-09-16T00:08:46.942 INFO:teuthology.orchestra.run.smithi139.stderr:clear 3.d pg_upmap_items mapping 2024-09-16T00:08:46.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.945+0000 7f1d668e4640 1 -- 172.21.15.139:0/2403283816 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1d4c06c2b0 msgr2=0x7f1d4c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:46.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.945+0000 7f1d668e4640 1 --2- 172.21.15.139:0/2403283816 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1d4c06c2b0 0x7f1d4c06e770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1d54034000 tx=0x7f1d5402e000 comp rx=0 tx=0).stop 2024-09-16T00:08:46.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.945+0000 7f1d668e4640 1 -- 172.21.15.139:0/2403283816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60068060 msgr2=0x7f1d600d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:46.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.945+0000 7f1d668e4640 1 --2- 172.21.15.139:0/2403283816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60068060 0x7f1d600d08c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000e870 tx=0x7f1d5000ed30 comp rx=0 tx=0).stop 2024-09-16T00:08:46.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.945+0000 7f1d668e4640 1 -- 172.21.15.139:0/2403283816 shutdown_connections 2024-09-16T00:08:46.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.945+0000 7f1d668e4640 1 --2- 172.21.15.139:0/2403283816 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1d4c06c2b0 0x7f1d4c06e770 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:46.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.945+0000 7f1d668e4640 1 --2- 172.21.15.139:0/2403283816 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d60154c20 0x7f1d6015cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:46.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.945+0000 7f1d668e4640 1 --2- 172.21.15.139:0/2403283816 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d600ccc90 0x7f1d60159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:46.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.945+0000 7f1d668e4640 1 --2- 172.21.15.139:0/2403283816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d60068060 0x7f1d600d08c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:46.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.945+0000 7f1d668e4640 1 -- 172.21.15.139:0/2403283816 >> 172.21.15.139:0/2403283816 conn(0x7f1d6006ca40 msgr2=0x7f1d6014fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:46.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.948+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbaf80051e0 con 0x7fbaf8004190 2024-09-16T00:08:46.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.948+0000 7f1d668e4640 1 -- 172.21.15.139:0/2403283816 shutdown_connections 2024-09-16T00:08:46.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.950+0000 7f1d668e4640 1 -- 172.21.15.139:0/2403283816 wait complete. 2024-09-16T00:08:46.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.951+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbaf80051e0 con 0x7fbaf8004190 2024-09-16T00:08:46.953 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T00:08:46.953 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:46.954 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:46.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.951+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbaf8009370 0x7fbaf800b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:46.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.952+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fbaf800e760 con 0x7fbaf8009370 2024-09-16T00:08:46.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.952+0000 7fbb1ffff640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbaf8009370 0x7fbaf800b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:46.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.952+0000 7fbb1ffff640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbaf8009370 0x7fbaf800b760 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:46.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.954+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf800e760 con 0x7fbaf8009370 2024-09-16T00:08:46.972 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:46.970+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:46.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.981+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbaf800d750 con 0x7fbaf8009370 2024-09-16T00:08:46.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.984+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbaf800d750 con 0x7fbaf8009370 2024-09-16T00:08:46.986 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:46.987 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:46.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.984+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbaf800bd40 0x7fbaf800c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:46.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.984+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fbaf800e370 con 0x7fbaf800bd40 2024-09-16T00:08:46.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.985+0000 7fbb2573c640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbaf800bd40 0x7fbaf800c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:46.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.985+0000 7fbb2573c640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbaf800bd40 0x7fbaf800c1f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:46.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:46.986+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf800e370 con 0x7fbaf800bd40 2024-09-16T00:08:46.998 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:46.999 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:46.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:46.999 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:46.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:46.999 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:46.999 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:46.999 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.380594602000002, 2024-09-16T00:08:46.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037514929999999998, 2024-09-16T00:08:46.999 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:46.999 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:46.999 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:46.999 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.000 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:47.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.000 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:47.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.000 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615364+0000", 2024-09-16T00:08:47.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8430000000000003e-06 2024-09-16T00:08:47.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.001 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.001 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615371+0000", 2024-09-16T00:08:47.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.148e-06 2024-09-16T00:08:47.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.001 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.001 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615380+0000", 2024-09-16T00:08:47.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5889999999999992e-06 2024-09-16T00:08:47.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.002 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.002 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615389+0000", 2024-09-16T00:08:47.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1620000000000008e-06 2024-09-16T00:08:47.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.002 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.002 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615525+0000", 2024-09-16T00:08:47.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000136083 2024-09-16T00:08:47.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.002 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615662+0000", 2024-09-16T00:08:47.003 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013696100000000001 2024-09-16T00:08:47.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:47.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615799+0000", 2024-09-16T00:08:47.003 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013781399999999999 2024-09-16T00:08:47.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:47.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:47.003 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002363502 2024-09-16T00:08:47.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.004 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:47.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619018+0000", 2024-09-16T00:08:47.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085522199999999999 2024-09-16T00:08:47.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.004 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619068+0000", 2024-09-16T00:08:47.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9545000000000001e-05 2024-09-16T00:08:47.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.004 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619111+0000", 2024-09-16T00:08:47.005 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3624000000000001e-05 2024-09-16T00:08:47.005 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.005 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.005 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.005 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:47.005 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:47.005 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.380542531000003, 2024-09-16T00:08:47.005 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043656110000000001, 2024-09-16T00:08:47.005 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.005 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.006 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.006 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:47.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:47.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:47.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2199999999999999e-07 2024-09-16T00:08:47.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615416+0000", 2024-09-16T00:08:47.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5590000000000001e-06 2024-09-16T00:08:47.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615423+0000", 2024-09-16T00:08:47.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6250000000000001e-06 2024-09-16T00:08:47.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615426+0000", 2024-09-16T00:08:47.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2540000000000001e-06 2024-09-16T00:08:47.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616075+0000", 2024-09-16T00:08:47.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064968800000000004 2024-09-16T00:08:47.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616142+0000", 2024-09-16T00:08:47.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6203000000000003e-05 2024-09-16T00:08:47.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.009 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:47.009 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616247+0000", 2024-09-16T00:08:47.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010572499999999999 2024-09-16T00:08:47.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.009 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:47.009 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618913+0000", 2024-09-16T00:08:47.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026654339999999999 2024-09-16T00:08:47.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.009 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:47.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619712+0000", 2024-09-16T00:08:47.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079885199999999996 2024-09-16T00:08:47.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.010 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619744+0000", 2024-09-16T00:08:47.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2790000000000003e-05 2024-09-16T00:08:47.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.010 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619778+0000", 2024-09-16T00:08:47.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3059000000000003e-05 2024-09-16T00:08:47.010 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.011 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.011 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.011 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:47.011 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:47.011 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.380506452000006, 2024-09-16T00:08:47.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087673219999999993, 2024-09-16T00:08:47.011 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.011 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.011 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.011 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.011 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:47.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.012 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:47.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.012 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:47.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:47.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:08:47.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5392000000000001e-05 2024-09-16T00:08:47.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615470+0000", 2024-09-16T00:08:47.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1999999999999999e-06 2024-09-16T00:08:47.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615474+0000", 2024-09-16T00:08:47.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7359999999999999e-06 2024-09-16T00:08:47.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617614+0000", 2024-09-16T00:08:47.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021398189999999998 2024-09-16T00:08:47.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617724+0000", 2024-09-16T00:08:47.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110808 2024-09-16T00:08:47.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:47.015 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617856+0000", 2024-09-16T00:08:47.015 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131676 2024-09-16T00:08:47.015 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.015 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.015 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:47.015 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:47.015 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029629280000000001 2024-09-16T00:08:47.015 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.015 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.015 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:47.015 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624158+0000", 2024-09-16T00:08:47.015 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033386549999999998 2024-09-16T00:08:47.015 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.015 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.016 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.016 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624189+0000", 2024-09-16T00:08:47.016 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1508999999999999e-05 2024-09-16T00:08:47.016 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.016 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.016 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.016 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624215+0000", 2024-09-16T00:08:47.016 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6228999999999999e-05 2024-09-16T00:08:47.016 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.016 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.016 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.016 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.016 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.017 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:47.017 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:47.017 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.380472436999995, 2024-09-16T00:08:47.017 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053285279999999999, 2024-09-16T00:08:47.017 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.017 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.017 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.017 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.017 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.017 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.017 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:47.017 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.018 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.018 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.018 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:47.018 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.018 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.018 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.018 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:47.018 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5600000000000002e-07 2024-09-16T00:08:47.018 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.018 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.019 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615484+0000", 2024-09-16T00:08:47.019 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.688e-06 2024-09-16T00:08:47.019 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.019 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.019 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.019 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615488+0000", 2024-09-16T00:08:47.019 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.658e-06 2024-09-16T00:08:47.019 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.019 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.019 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.019 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615490+0000", 2024-09-16T00:08:47.019 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.728e-06 2024-09-16T00:08:47.019 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.019 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.020 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.020 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616441+0000", 2024-09-16T00:08:47.020 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00095065500000000001 2024-09-16T00:08:47.020 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.020 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.020 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.020 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616539+0000", 2024-09-16T00:08:47.020 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8335e-05 2024-09-16T00:08:47.020 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.020 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.020 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:47.020 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616631+0000", 2024-09-16T00:08:47.020 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1260000000000004e-05 2024-09-16T00:08:47.021 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.021 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.021 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:47.021 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619837+0000", 2024-09-16T00:08:47.021 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003206899 2024-09-16T00:08:47.021 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.021 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.021 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:47.021 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620725+0000", 2024-09-16T00:08:47.021 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088713599999999996 2024-09-16T00:08:47.021 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.021 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.021 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.021 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620772+0000", 2024-09-16T00:08:47.022 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7639999999999998e-05 2024-09-16T00:08:47.022 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.022 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.022 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.022 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620811+0000", 2024-09-16T00:08:47.022 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8272999999999998e-05 2024-09-16T00:08:47.022 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.022 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.022 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.022 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.022 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.022 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:47.022 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:47.022 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.380057348999998, 2024-09-16T00:08:47.023 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029886169999999998, 2024-09-16T00:08:47.023 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.023 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.023 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.023 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.023 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.023 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.023 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:47.023 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.023 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.023 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.023 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.023 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:47.023 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.024 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615898+0000", 2024-09-16T00:08:47.024 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.237e-06 2024-09-16T00:08:47.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.024 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615904+0000", 2024-09-16T00:08:47.024 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:47.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.024 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615915+0000", 2024-09-16T00:08:47.025 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1003e-05 2024-09-16T00:08:47.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.025 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.025 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.025 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615922+0000", 2024-09-16T00:08:47.025 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9959999999999996e-06 2024-09-16T00:08:47.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.025 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.025 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.025 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615988+0000", 2024-09-16T00:08:47.025 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5806e-05 2024-09-16T00:08:47.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.025 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.026 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.026 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616009+0000", 2024-09-16T00:08:47.026 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0729000000000001e-05 2024-09-16T00:08:47.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.026 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.026 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.026 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:47.026 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028462130000000002 2024-09-16T00:08:47.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.026 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.026 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.026 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618886+0000", 2024-09-16T00:08:47.026 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.082e-05 2024-09-16T00:08:47.026 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.027 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.027 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.027 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.027 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:47.027 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:47.027 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.378445877000004, 2024-09-16T00:08:47.027 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032610299999999998, 2024-09-16T00:08:47.027 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.027 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.027 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.027 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.027 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.028 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:47.028 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.028 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.028 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:47.028 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.028 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.028 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617556+0000", 2024-09-16T00:08:47.028 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7431e-05 2024-09-16T00:08:47.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.029 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.029 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617847+0000", 2024-09-16T00:08:47.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029133400000000003 2024-09-16T00:08:47.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.029 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.029 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617858+0000", 2024-09-16T00:08:47.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0222e-05 2024-09-16T00:08:47.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.029 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.029 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617864+0000", 2024-09-16T00:08:47.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7619999999999998e-06 2024-09-16T00:08:47.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.030 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617909+0000", 2024-09-16T00:08:47.030 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4747999999999999e-05 2024-09-16T00:08:47.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.030 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617927+0000", 2024-09-16T00:08:47.030 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7785999999999999e-05 2024-09-16T00:08:47.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.031 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620734+0000", 2024-09-16T00:08:47.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002807305 2024-09-16T00:08:47.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.031 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.031 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620770+0000", 2024-09-16T00:08:47.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5441999999999997e-05 2024-09-16T00:08:47.031 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.031 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.031 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.031 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:47.031 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:47.032 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.378025192999999, 2024-09-16T00:08:47.032 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028641809999999999, 2024-09-16T00:08:47.032 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.032 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.032 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.032 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.032 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.032 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.032 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:47.032 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.032 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.032 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.033 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.033 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:47.033 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.033 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.033 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.033 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.033 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617930+0000", 2024-09-16T00:08:47.033 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0200000000000002e-07 2024-09-16T00:08:47.033 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.033 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.033 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.033 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617934+0000", 2024-09-16T00:08:47.033 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:47.033 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.034 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.034 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617943+0000", 2024-09-16T00:08:47.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5820000000000002e-06 2024-09-16T00:08:47.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.034 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.034 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617949+0000", 2024-09-16T00:08:47.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5069999999999997e-06 2024-09-16T00:08:47.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.034 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.034 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618094+0000", 2024-09-16T00:08:47.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014515599999999999 2024-09-16T00:08:47.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618111+0000", 2024-09-16T00:08:47.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6705999999999999e-05 2024-09-16T00:08:47.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:08:47.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026603070000000002 2024-09-16T00:08:47.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620793+0000", 2024-09-16T00:08:47.036 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2214000000000001e-05 2024-09-16T00:08:47.036 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.036 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.036 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.036 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.036 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:47.036 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:47.036 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.377994404000006, 2024-09-16T00:08:47.036 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028624549999999999, 2024-09-16T00:08:47.036 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.036 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.036 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.037 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.037 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.037 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:47.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.037 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.037 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:47.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.037 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.037 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:47.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:47.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.038 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.038 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:47.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0570000000000001e-06 2024-09-16T00:08:47.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.038 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.038 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617970+0000", 2024-09-16T00:08:47.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.669e-06 2024-09-16T00:08:47.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.038 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.039 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617976+0000", 2024-09-16T00:08:47.039 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1369999999999999e-06 2024-09-16T00:08:47.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.039 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.039 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.039 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618268+0000", 2024-09-16T00:08:47.039 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029164600000000001 2024-09-16T00:08:47.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.039 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.039 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.039 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618284+0000", 2024-09-16T00:08:47.039 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5824e-05 2024-09-16T00:08:47.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.039 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.040 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.040 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:47.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025127890000000001 2024-09-16T00:08:47.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.040 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.040 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620822+0000", 2024-09-16T00:08:47.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6009000000000001e-05 2024-09-16T00:08:47.040 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.040 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.040 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.041 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-16T00:08:47.041 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:47.041 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.377966822999994, 2024-09-16T00:08:47.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028650350000000002, 2024-09-16T00:08:47.041 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.041 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.041 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.041 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.041 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.041 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.041 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:47.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.041 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.042 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.042 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:47.042 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.042 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.042 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.042 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:47.042 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7599999999999998e-07 2024-09-16T00:08:47.042 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.042 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.042 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617991+0000", 2024-09-16T00:08:47.042 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1860000000000001e-06 2024-09-16T00:08:47.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617999+0000", 2024-09-16T00:08:47.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3250000000000002e-06 2024-09-16T00:08:47.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618004+0000", 2024-09-16T00:08:47.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7300000000000002e-06 2024-09-16T00:08:47.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618422+0000", 2024-09-16T00:08:47.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00041763199999999998 2024-09-16T00:08:47.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.044 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.044 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:47.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6112000000000001e-05 2024-09-16T00:08:47.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.044 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.044 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620824+0000", 2024-09-16T00:08:47.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002385954 2024-09-16T00:08:47.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620853+0000", 2024-09-16T00:08:47.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8719999999999999e-05 2024-09-16T00:08:47.045 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.045 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.045 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.045 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-16T00:08:47.045 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:47.045 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.377941000999996, 2024-09-16T00:08:47.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032703020000000001, 2024-09-16T00:08:47.045 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.046 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.046 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.046 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.046 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:47.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.046 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:47.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.047 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.047 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618014+0000", 2024-09-16T00:08:47.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7799999999999997e-07 2024-09-16T00:08:47.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.047 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.047 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.047 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618017+0000", 2024-09-16T00:08:47.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4309999999999998e-06 2024-09-16T00:08:47.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.047 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.047 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.047 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618023+0000", 2024-09-16T00:08:47.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8490000000000002e-06 2024-09-16T00:08:47.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618028+0000", 2024-09-16T00:08:47.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3489999999999999e-06 2024-09-16T00:08:47.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:08:47.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055724099999999999 2024-09-16T00:08:47.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618603+0000", 2024-09-16T00:08:47.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7486000000000002e-05 2024-09-16T00:08:47.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.049 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.049 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621235+0000", 2024-09-16T00:08:47.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002632087 2024-09-16T00:08:47.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.049 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.049 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621284+0000", 2024-09-16T00:08:47.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8581000000000001e-05 2024-09-16T00:08:47.049 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.049 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.049 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.050 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.050 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.050 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:14 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:47.050 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:47.050 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.377004021000005, 2024-09-16T00:08:47.050 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016807599999999999, 2024-09-16T00:08:47.050 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.050 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.050 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:47.050 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.050 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.050 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.050 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:47.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.051 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.051 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618950+0000", 2024-09-16T00:08:47.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.051 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.051 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618951+0000", 2024-09-16T00:08:47.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:47.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.051 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618954+0000", 2024-09-16T00:08:47.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.145e-06 2024-09-16T00:08:47.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618963+0000", 2024-09-16T00:08:47.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5720000000000003e-06 2024-09-16T00:08:47.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618969+0000", 2024-09-16T00:08:47.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3319999999999997e-06 2024-09-16T00:08:47.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619008+0000", 2024-09-16T00:08:47.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9096e-05 2024-09-16T00:08:47.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619013+0000", 2024-09-16T00:08:47.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4710000000000001e-06 2024-09-16T00:08:47.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.054 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.054 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619118+0000", 2024-09-16T00:08:47.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000105809 2024-09-16T00:08:47.054 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.054 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.054 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.054 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-16T00:08:47.054 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:47.054 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.376965913999996, 2024-09-16T00:08:47.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002325307, 2024-09-16T00:08:47.054 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.055 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.055 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.055 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.055 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.055 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:47.055 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.055 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.055 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.055 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:47.055 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.055 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.056 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.056 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:47.056 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:47.056 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.056 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.056 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.056 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618993+0000", 2024-09-16T00:08:47.056 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.687e-06 2024-09-16T00:08:47.056 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.056 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.056 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.056 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619000+0000", 2024-09-16T00:08:47.056 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3590000000000002e-06 2024-09-16T00:08:47.056 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619005+0000", 2024-09-16T00:08:47.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.95e-06 2024-09-16T00:08:47.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619039+0000", 2024-09-16T00:08:47.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3942999999999997e-05 2024-09-16T00:08:47.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619055+0000", 2024-09-16T00:08:47.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6107e-05 2024-09-16T00:08:47.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.058 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.058 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621287+0000", 2024-09-16T00:08:47.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022318070000000001 2024-09-16T00:08:47.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.058 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.058 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:47.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7129999999999999e-05 2024-09-16T00:08:47.058 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.058 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.058 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.059 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-16T00:08:47.059 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:47.059 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.376927918000007, 2024-09-16T00:08:47.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002314328, 2024-09-16T00:08:47.059 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.059 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.059 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.059 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.059 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:47.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:47.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:47.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6399999999999998e-07 2024-09-16T00:08:47.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619030+0000", 2024-09-16T00:08:47.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.219e-06 2024-09-16T00:08:47.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619038+0000", 2024-09-16T00:08:47.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9899999999999997e-06 2024-09-16T00:08:47.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619043+0000", 2024-09-16T00:08:47.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5580000000000002e-06 2024-09-16T00:08:47.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.062 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.062 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619220+0000", 2024-09-16T00:08:47.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000176937 2024-09-16T00:08:47.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.062 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.062 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619237+0000", 2024-09-16T00:08:47.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7691999999999999e-05 2024-09-16T00:08:47.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.062 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621315+0000", 2024-09-16T00:08:47.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020778900000000002 2024-09-16T00:08:47.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621341+0000", 2024-09-16T00:08:47.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5678e-05 2024-09-16T00:08:47.063 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.063 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.063 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.063 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:15 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:47.063 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:47.064 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.376312948999995, 2024-09-16T00:08:47.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143416, 2024-09-16T00:08:47.064 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.064 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.064 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:47.064 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:47.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619641+0000", 2024-09-16T00:08:47.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619642+0000", 2024-09-16T00:08:47.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5199999999999995e-07 2024-09-16T00:08:47.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619645+0000", 2024-09-16T00:08:47.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.134e-06 2024-09-16T00:08:47.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619655+0000", 2024-09-16T00:08:47.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3330000000000003e-06 2024-09-16T00:08:47.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619662+0000", 2024-09-16T00:08:47.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0500000000000003e-06 2024-09-16T00:08:47.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619704+0000", 2024-09-16T00:08:47.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1766e-05 2024-09-16T00:08:47.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619708+0000", 2024-09-16T00:08:47.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3440000000000003e-06 2024-09-16T00:08:47.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619785+0000", 2024-09-16T00:08:47.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6936999999999999e-05 2024-09-16T00:08:47.067 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.067 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.067 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.068 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.068 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:31 2.d e22/19 ondisk, result = 0)", 2024-09-16T00:08:47.068 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:47.068 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.376214140000002, 2024-09-16T00:08:47.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101826, 2024-09-16T00:08:47.068 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.068 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.068 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:47.068 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.068 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.068 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.068 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:47.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619740+0000", 2024-09-16T00:08:47.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619741+0000", 2024-09-16T00:08:47.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3099999999999997e-07 2024-09-16T00:08:47.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:08:47.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1719999999999998e-06 2024-09-16T00:08:47.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619771+0000", 2024-09-16T00:08:47.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4754000000000001e-05 2024-09-16T00:08:47.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619790+0000", 2024-09-16T00:08:47.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9516000000000001e-05 2024-09-16T00:08:47.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619830+0000", 2024-09-16T00:08:47.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9848000000000002e-05 2024-09-16T00:08:47.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619834+0000", 2024-09-16T00:08:47.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1150000000000004e-06 2024-09-16T00:08:47.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619842+0000", 2024-09-16T00:08:47.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.79e-06 2024-09-16T00:08:47.071 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.071 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.072 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.072 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:30 2.15 e22/19 ondisk, result = 0)", 2024-09-16T00:08:47.072 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:47.072 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.371863153000007, 2024-09-16T00:08:47.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013076400000000001, 2024-09-16T00:08:47.072 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.072 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.072 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:47.073 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.073 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.073 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:47.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.073 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.073 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624091+0000", 2024-09-16T00:08:47.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.073 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624092+0000", 2024-09-16T00:08:47.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5799999999999999e-07 2024-09-16T00:08:47.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624096+0000", 2024-09-16T00:08:47.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:47.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624106+0000", 2024-09-16T00:08:47.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7629999999999995e-06 2024-09-16T00:08:47.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624112+0000", 2024-09-16T00:08:47.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.626e-06 2024-09-16T00:08:47.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624149+0000", 2024-09-16T00:08:47.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7094000000000001e-05 2024-09-16T00:08:47.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624154+0000", 2024-09-16T00:08:47.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7359999999999998e-06 2024-09-16T00:08:47.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624222+0000", 2024-09-16T00:08:47.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7680000000000003e-05 2024-09-16T00:08:47.076 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.076 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.076 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.077 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.077 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:39 2.6 2:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:47.077 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:47.077 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 63.355676594999998, 2024-09-16T00:08:47.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050072119999999996, 2024-09-16T00:08:47.077 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.077 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.077 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.077 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.077 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.077 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.077 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:47.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:47.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640280+0000", 2024-09-16T00:08:47.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5380000000000001e-06 2024-09-16T00:08:47.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640296+0000", 2024-09-16T00:08:47.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5950000000000001e-05 2024-09-16T00:08:47.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.079 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640327+0000", 2024-09-16T00:08:47.079 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0309999999999999e-05 2024-09-16T00:08:47.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.079 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640340+0000", 2024-09-16T00:08:47.079 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3773e-05 2024-09-16T00:08:47.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.080 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640422+0000", 2024-09-16T00:08:47.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1122e-05 2024-09-16T00:08:47.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.080 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.080 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640572+0000", 2024-09-16T00:08:47.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015044099999999999 2024-09-16T00:08:47.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.080 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:08:47.080 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640691+0000", 2024-09-16T00:08:47.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000119393 2024-09-16T00:08:47.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:47.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645006+0000", 2024-09-16T00:08:47.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043143779999999998 2024-09-16T00:08:47.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:47.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645169+0000", 2024-09-16T00:08:47.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016323299999999999 2024-09-16T00:08:47.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645227+0000", 2024-09-16T00:08:47.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8e-05 2024-09-16T00:08:47.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645285+0000", 2024-09-16T00:08:47.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8074e-05 2024-09-16T00:08:47.082 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.082 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.082 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.082 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:39 2.6 e22/19 ondisk, result = 0)", 2024-09-16T00:08:47.082 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:47.082 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 63.351104483999997, 2024-09-16T00:08:47.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016541200000000001, 2024-09-16T00:08:47.083 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.083 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.083 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:47.083 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:47.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644850+0000", 2024-09-16T00:08:47.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.084 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.084 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.084 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644852+0000", 2024-09-16T00:08:47.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1670000000000002e-06 2024-09-16T00:08:47.084 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.084 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.084 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.084 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644865+0000", 2024-09-16T00:08:47.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2826000000000001e-05 2024-09-16T00:08:47.084 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.084 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.084 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.084 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644894+0000", 2024-09-16T00:08:47.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9264999999999998e-05 2024-09-16T00:08:47.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.085 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644911+0000", 2024-09-16T00:08:47.085 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6827999999999999e-05 2024-09-16T00:08:47.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.085 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644989+0000", 2024-09-16T00:08:47.085 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7849999999999995e-05 2024-09-16T00:08:47.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.644999+0000", 2024-09-16T00:08:47.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7289999999999995e-06 2024-09-16T00:08:47.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.086 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645015+0000", 2024-09-16T00:08:47.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6747000000000001e-05 2024-09-16T00:08:47.086 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.086 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.086 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.086 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.6 38/37 [PushReplyOp(2:654134d2:::mds0_openfiles.0:head)])", 2024-09-16T00:08:47.086 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.845546+0000", 2024-09-16T00:08:47.087 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.150408633000001, 2024-09-16T00:08:47.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00023895899999999999, 2024-09-16T00:08:47.087 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.087 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.087 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:47.087 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.087 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.087 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845546+0000", 2024-09-16T00:08:47.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.087 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845546+0000", 2024-09-16T00:08:47.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845546+0000", 2024-09-16T00:08:47.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1299999999999999e-07 2024-09-16T00:08:47.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845549+0000", 2024-09-16T00:08:47.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6000000000000001e-06 2024-09-16T00:08:47.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.089 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845557+0000", 2024-09-16T00:08:47.089 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0970000000000006e-06 2024-09-16T00:08:47.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.089 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845564+0000", 2024-09-16T00:08:47.089 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3839999999999999e-06 2024-09-16T00:08:47.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.089 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845616+0000", 2024-09-16T00:08:47.089 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2120000000000002e-05 2024-09-16T00:08:47.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.090 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.090 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.845785+0000", 2024-09-16T00:08:47.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016854500000000001 2024-09-16T00:08:47.090 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.090 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.090 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.090 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.090 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.090 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:47.092 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:08:47.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.013+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbaf800c7b0 con 0x7fbaf800bd40 2024-09-16T00:08:47.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.016+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbaf800c7b0 con 0x7fbaf800bd40 2024-09-16T00:08:47.093 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:47.094 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:47.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.016+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbaf800cdb0 0x7fbaf80126d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:47.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.016+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fbaf8015520 con 0x7fbaf800cdb0 2024-09-16T00:08:47.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.016+0000 7fbb24f3b640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbaf800cdb0 0x7fbaf80126d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:47.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.017+0000 7fbb24f3b640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbaf800cdb0 0x7fbaf80126d0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:47.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.018+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf8015520 con 0x7fbaf800cdb0 2024-09-16T00:08:47.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.044+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbaf8012ee0 con 0x7fbaf800cdb0 2024-09-16T00:08:47.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.047+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbaf8012ee0 con 0x7fbaf800cdb0 2024-09-16T00:08:47.094 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:47.095 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:47.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.047+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbaf8013630 0x7fbaf8013a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:47.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.047+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fbaf80188c0 con 0x7fbaf8013630 2024-09-16T00:08:47.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.047+0000 7fbb1ffff640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbaf8013630 0x7fbaf8013a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:47.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.048+0000 7fbb1ffff640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbaf8013630 0x7fbaf8013a70 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:47.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.048+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf80188c0 con 0x7fbaf8013630 2024-09-16T00:08:47.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.070+0000 7f1633fb9640 1 Processor -- start 2024-09-16T00:08:47.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.070+0000 7f1633fb9640 1 -- start start 2024-09-16T00:08:47.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.071+0000 7f1633fb9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f162c0c8040 0x7f162c06bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:47.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.071+0000 7f1633fb9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f162c15b390 0x7f162c0cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:47.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.071+0000 7f1633fb9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f162c06c3f0 con 0x7f162c0ccff0 2024-09-16T00:08:47.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.071+0000 7f1633fb9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f162c06f510 con 0x7f162c0c8040 2024-09-16T00:08:47.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.071+0000 7f1633fb9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f162c15a780 con 0x7f162c15b390 2024-09-16T00:08:47.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.072+0000 7f1631554640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f162c0c8040 0x7f162c06bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:47.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.072+0000 7f1631554640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f162c0c8040 0x7f162c06bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49468/0 (socket says 172.21.15.139:49468) 2024-09-16T00:08:47.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.072+0000 7f1631554640 1 -- 172.21.15.139:0/3798129567 learned_addr learned my addr 172.21.15.139:0/3798129567 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:47.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.072+0000 7f1632556640 1 --2- 172.21.15.139:0/3798129567 >> v2:172.21.15.164:3300/0 conn(0x7f162c15b390 0x7f162c0cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:47.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.072+0000 7f1631554640 1 -- 172.21.15.139:0/3798129567 >> v2:172.21.15.164:3300/0 conn(0x7f162c15b390 msgr2=0x7f162c0cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:47.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.072+0000 7f1631554640 1 --2- 172.21.15.139:0/3798129567 >> v2:172.21.15.164:3300/0 conn(0x7f162c15b390 0x7f162c0cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.072+0000 7f1631554640 1 -- 172.21.15.139:0/3798129567 >> v1:172.21.15.164:6789/0 conn(0x7f162c0ccff0 legacy=0x7f162c15ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:47.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.072+0000 7f1631554640 1 -- 172.21.15.139:0/3798129567 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c06f7b0 con 0x7f162c0c8040 2024-09-16T00:08:47.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.073+0000 7f1631554640 1 --2- 172.21.15.139:0/3798129567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f162c0c8040 0x7f162c06bcb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f162c0c95e0 tx=0x7f161c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9221ff95a39fc327 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:47.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.073+0000 7f1630d53640 1 -- 172.21.15.139:0/3798129567 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c037070 con 0x7f162c0c8040 2024-09-16T00:08:47.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.073+0000 7f1630d53640 1 -- 172.21.15.139:0/3798129567 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161c032440 con 0x7f162c0c8040 2024-09-16T00:08:47.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.073+0000 7f1630d53640 1 -- 172.21.15.139:0/3798129567 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c032810 con 0x7f162c0c8040 2024-09-16T00:08:47.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.074+0000 7f1633fb9640 1 -- 172.21.15.139:0/3798129567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f162c0c8040 msgr2=0x7f162c06bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:47.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.074+0000 7f1633fb9640 1 --2- 172.21.15.139:0/3798129567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f162c0c8040 0x7f162c06bcb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f162c0c95e0 tx=0x7f161c02a270 comp rx=0 tx=0).stop 2024-09-16T00:08:47.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.074+0000 7f1633fb9640 1 -- 172.21.15.139:0/3798129567 shutdown_connections 2024-09-16T00:08:47.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.074+0000 7f1633fb9640 1 --2- 172.21.15.139:0/3798129567 >> v2:172.21.15.164:3300/0 conn(0x7f162c15b390 0x7f162c0cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.074+0000 7f1633fb9640 1 --2- 172.21.15.139:0/3798129567 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f162c0c8040 0x7f162c06bcb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.074+0000 7f1633fb9640 1 -- 172.21.15.139:0/3798129567 >> 172.21.15.139:0/3798129567 conn(0x7f162c06cc30 msgr2=0x7f162c06d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:47.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.074+0000 7f1633fb9640 1 -- 172.21.15.139:0/3798129567 shutdown_connections 2024-09-16T00:08:47.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.074+0000 7f1633fb9640 1 -- 172.21.15.139:0/3798129567 wait complete. 2024-09-16T00:08:47.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.075+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbaf8017740 con 0x7fbaf8013630 2024-09-16T00:08:47.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.075+0000 7f1633fb9640 1 Processor -- start 2024-09-16T00:08:47.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.075+0000 7f1633fb9640 1 -- start start 2024-09-16T00:08:47.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.076+0000 7f1633fb9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f162c0c8040 0x7f162c0c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:47.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.076+0000 7f1633fb9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f162c0ccff0 0x7f162c159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:47.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.076+0000 7f1633fb9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f162c15b390 0x7f162c150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:47.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.076+0000 7f1631d55640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f162c0c8040 0x7f162c0c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:47.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.076+0000 7f1631d55640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f162c0c8040 0x7f162c0c5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49484/0 (socket says 172.21.15.139:49484) 2024-09-16T00:08:47.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.076+0000 7f1633fb9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f162c154b30 con 0x7f162c0c8040 2024-09-16T00:08:47.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.076+0000 7f1631554640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f162c0ccff0 0x7f162c159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:47.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.076+0000 7f1633fb9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f162c06fd70 con 0x7f162c0ccff0 2024-09-16T00:08:47.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.076+0000 7f1633fb9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f162c154cb0 con 0x7f162c15b390 2024-09-16T00:08:47.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.076+0000 7f1631d55640 1 -- 172.21.15.139:0/1884282057 learned_addr learned my addr 172.21.15.139:0/1884282057 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:47.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.076+0000 7f1632556640 1 --2- 172.21.15.139:0/1884282057 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f162c15b390 0x7f162c150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:47.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.077+0000 7f1631554640 1 -- 172.21.15.139:0/1884282057 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f162c15b390 msgr2=0x7f162c150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:47.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.077+0000 7f1631554640 1 --2- 172.21.15.139:0/1884282057 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f162c15b390 0x7f162c150640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.077+0000 7f1631554640 1 -- 172.21.15.139:0/1884282057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f162c0c8040 msgr2=0x7f162c0c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:47.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.077+0000 7f1631554640 1 --2- 172.21.15.139:0/1884282057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f162c0c8040 0x7f162c0c5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.077+0000 7f1631554640 1 -- 172.21.15.139:0/1884282057 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c1e3ef0 con 0x7f162c0ccff0 2024-09-16T00:08:47.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.077+0000 7f1631554640 1 --2- 172.21.15.139:0/1884282057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f162c0ccff0 0x7f162c159c50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f161c031000 tx=0x7f161c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:47.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.077+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbaf8017740 con 0x7fbaf8013630 2024-09-16T00:08:47.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.077+0000 7f16227fc640 1 -- 172.21.15.139:0/1884282057 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c041030 con 0x7f162c0ccff0 2024-09-16T00:08:47.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.077+0000 7f1633fb9640 1 -- 172.21.15.139:0/1884282057 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c0c5ba0 con 0x7f162c0ccff0 2024-09-16T00:08:47.103 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:47.104 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:47.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.077+0000 7f1633fb9640 1 -- 172.21.15.139:0/1884282057 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c1e57f0 con 0x7f162c0ccff0 2024-09-16T00:08:47.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.078+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbaf8017e90 0x7fbaf80182b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:47.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.078+0000 7f16227fc640 1 -- 172.21.15.139:0/1884282057 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161c032440 con 0x7f162c0ccff0 2024-09-16T00:08:47.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.078+0000 7f16227fc640 1 -- 172.21.15.139:0/1884282057 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c037040 con 0x7f162c0ccff0 2024-09-16T00:08:47.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.078+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fbaf801bc60 con 0x7fbaf8017e90 2024-09-16T00:08:47.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.078+0000 7fbb2573c640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbaf8017e90 0x7fbaf80182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:47.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.079+0000 7fbb2573c640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbaf8017e90 0x7fbaf80182b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:47.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.079+0000 7f1633fb9640 1 -- 172.21.15.139:0/1884282057 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1614005290 con 0x7f162c0ccff0 2024-09-16T00:08:47.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.079+0000 7f16227fc640 1 -- 172.21.15.139:0/1884282057 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f161c044070 con 0x7f162c0ccff0 2024-09-16T00:08:47.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.079+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf801bc60 con 0x7fbaf8017e90 2024-09-16T00:08:47.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.080+0000 7f16227fc640 1 --2- 172.21.15.139:0/1884282057 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f161006c2b0 0x7f161006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:47.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.080+0000 7f16227fc640 1 -- 172.21.15.139:0/1884282057 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5886+0+0 (secure 0 0 0) 0x7f161c0aba60 con 0x7f162c0ccff0 2024-09-16T00:08:47.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.080+0000 7f1631d55640 1 --2- 172.21.15.139:0/1884282057 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f161006c2b0 0x7f161006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:47.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.081+0000 7f1631d55640 1 --2- 172.21.15.139:0/1884282057 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f161006c2b0 0x7f161006e770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1628001840 tx=0x7f1628009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:47.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.084+0000 7f16227fc640 1 -- 172.21.15.139:0/1884282057 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f162c0c5ba0 con 0x7f162c0ccff0 2024-09-16T00:08:47.107 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:47.091+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:47.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.106+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbaf8013fb0 con 0x7fbaf8017e90 2024-09-16T00:08:47.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.108+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbaf8013fb0 con 0x7fbaf8017e90 2024-09-16T00:08:47.110 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:47.111 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:47.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.108+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbaf80144c0 0x7fbaf8014920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:47.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.108+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fbaf8014e60 con 0x7fbaf80144c0 2024-09-16T00:08:47.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.109+0000 7fbb24f3b640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbaf80144c0 0x7fbaf8014920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:47.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.109+0000 7fbb24f3b640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbaf80144c0 0x7fbaf8014920 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:47.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.110+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbaf8014e60 con 0x7fbaf80144c0 2024-09-16T00:08:47.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.132+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbaf8014e60 con 0x7fbaf80144c0 2024-09-16T00:08:47.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb1d7fa640 1 -- 172.21.15.139:0/1851320694 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbaf8014e60 con 0x7fbaf80144c0 2024-09-16T00:08:47.137 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:47.139 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:47.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbaf8001650 msgr2=0x7fbaf8003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:47.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fbaf8001650 0x7fbaf8003b10 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbaf8004190 msgr2=0x7fbaf8004620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:47.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fbaf8004190 0x7fbaf8004620 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbaf8009370 msgr2=0x7fbaf800b760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:47.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fbaf8009370 0x7fbaf800b760 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb2573c640 1 -- 172.21.15.139:0/1851320694 reap_dead start 2024-09-16T00:08:47.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbaf800bd40 msgr2=0x7fbaf800c1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:47.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fbaf800bd40 0x7fbaf800c1f0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbaf800cdb0 msgr2=0x7fbaf80126d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:47.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fbaf800cdb0 0x7fbaf80126d0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbaf8013630 msgr2=0x7fbaf8013a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:47.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fbaf8013630 0x7fbaf8013a70 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbaf8017e90 msgr2=0x7fbaf80182b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:47.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fbaf8017e90 0x7fbaf80182b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.135+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbaf80144c0 msgr2=0x7fbaf8014920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:47.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.136+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fbaf80144c0 0x7fbaf8014920 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.136+0000 7fbb2573c640 1 -- 172.21.15.139:0/1851320694 reap_dead start 2024-09-16T00:08:47.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.136+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb0406c2e0 msgr2=0x7fbb0406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:47.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.136+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb0406c2e0 0x7fbb0406e7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbb14000f60 tx=0x7fbb14007d30 comp rx=0 tx=0).stop 2024-09-16T00:08:47.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.136+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb2015c700 msgr2=0x7fbb2014b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:47.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.136+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb2015c700 0x7fbb2014b2e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbb18031000 tx=0x7fbb18004a40 comp rx=0 tx=0).stop 2024-09-16T00:08:47.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.136+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 shutdown_connections 2024-09-16T00:08:47.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.136+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb0406c2e0 0x7fbb0406e7a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.136+0000 7fbb0affd640 1 --2- 172.21.15.139:0/1851320694 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb2015c700 0x7fbb2014b2e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.136+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 >> 172.21.15.139:0/1851320694 conn(0x7fbb2006fb00 msgr2=0x7fbb20151a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:47.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.136+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 shutdown_connections 2024-09-16T00:08:47.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.136+0000 7fbb0affd640 1 -- 172.21.15.139:0/1851320694 wait complete. 2024-09-16T00:08:47.192 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:47.191+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:47.227 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:47.227 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:47.228 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:47.228 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:47.237 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:08:47.321 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:47.319+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:47.385 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:47.385 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:47.385 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:47.386 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:47.386 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:47.396 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:08:47.396 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:47.393+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:47.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.446+0000 7f1633fb9640 1 -- 172.21.15.139:0/1884282057 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f1614005850 con 0x7f162c0ccff0 2024-09-16T00:08:47.496 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:47.494+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:47.597 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:47.596+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:47.599 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:47.599 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:47.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:47.600 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:47.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.600 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:47.600 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:47.600 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.981829606000005, 2024-09-16T00:08:47.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038263220000000001, 2024-09-16T00:08:47.600 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.600 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.600 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.600 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.601 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:47.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.601 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.601 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:08:47.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.601 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.601 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615465+0000", 2024-09-16T00:08:47.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2769999999999999e-06 2024-09-16T00:08:47.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615471+0000", 2024-09-16T00:08:47.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5729999999999996e-06 2024-09-16T00:08:47.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615483+0000", 2024-09-16T00:08:47.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1838000000000001e-05 2024-09-16T00:08:47.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615492+0000", 2024-09-16T00:08:47.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2410000000000001e-06 2024-09-16T00:08:47.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.603 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.604 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615596+0000", 2024-09-16T00:08:47.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104097 2024-09-16T00:08:47.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615688+0000", 2024-09-16T00:08:47.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1225999999999997e-05 2024-09-16T00:08:47.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:47.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615804+0000", 2024-09-16T00:08:47.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000116601 2024-09-16T00:08:47.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:47.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618165+0000", 2024-09-16T00:08:47.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023609429999999999 2024-09-16T00:08:47.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:47.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:08:47.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010258509999999999 2024-09-16T00:08:47.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619243+0000", 2024-09-16T00:08:47.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2327999999999998e-05 2024-09-16T00:08:47.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.607 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619287+0000", 2024-09-16T00:08:47.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3346999999999997e-05 2024-09-16T00:08:47.607 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.607 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.607 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.607 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:47.607 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:47.607 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.981774529999996, 2024-09-16T00:08:47.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0054735510000000001, 2024-09-16T00:08:47.607 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.607 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.608 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.608 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.608 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:47.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.608 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:47.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615516+0000", 2024-09-16T00:08:47.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6899999999999998e-07 2024-09-16T00:08:47.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615519+0000", 2024-09-16T00:08:47.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1760000000000002e-06 2024-09-16T00:08:47.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615526+0000", 2024-09-16T00:08:47.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8399999999999997e-06 2024-09-16T00:08:47.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615531+0000", 2024-09-16T00:08:47.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4450000000000004e-06 2024-09-16T00:08:47.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:47.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054127099999999996 2024-09-16T00:08:47.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616140+0000", 2024-09-16T00:08:47.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7230999999999994e-05 2024-09-16T00:08:47.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:47.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616249+0000", 2024-09-16T00:08:47.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109122 2024-09-16T00:08:47.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:47.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618646+0000", 2024-09-16T00:08:47.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023972279999999999 2024-09-16T00:08:47.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.612 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:47.612 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:08:47.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022396220000000001 2024-09-16T00:08:47.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.612 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.612 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620939+0000", 2024-09-16T00:08:47.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2735999999999998e-05 2024-09-16T00:08:47.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.612 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.613 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620989+0000", 2024-09-16T00:08:47.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0411000000000002e-05 2024-09-16T00:08:47.613 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.613 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.613 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.613 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:47.613 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:47.613 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.981749050999994, 2024-09-16T00:08:47.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055872789999999997, 2024-09-16T00:08:47.613 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.613 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.614 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.614 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:47.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:47.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.615 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.615 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:08:47.615 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.01e-07 2024-09-16T00:08:47.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.615 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.615 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615547+0000", 2024-09-16T00:08:47.615 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7899999999999996e-06 2024-09-16T00:08:47.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.615 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.615 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:08:47.615 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1119999999999998e-06 2024-09-16T00:08:47.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615560+0000", 2024-09-16T00:08:47.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:08:47.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616424+0000", 2024-09-16T00:08:47.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086350800000000003 2024-09-16T00:08:47.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:08:47.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9675999999999999e-05 2024-09-16T00:08:47.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:47.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616621+0000", 2024-09-16T00:08:47.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7345999999999999e-05 2024-09-16T00:08:47.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:47.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618654+0000", 2024-09-16T00:08:47.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020335269999999998 2024-09-16T00:08:47.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.618 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:47.618 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621055+0000", 2024-09-16T00:08:47.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024009729999999998 2024-09-16T00:08:47.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.618 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.618 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621100+0000", 2024-09-16T00:08:47.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5402999999999999e-05 2024-09-16T00:08:47.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.618 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.618 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621128+0000", 2024-09-16T00:08:47.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7736e-05 2024-09-16T00:08:47.619 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.619 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.619 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.619 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:47.619 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:47.619 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.981723400999996, 2024-09-16T00:08:47.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057024509999999999, 2024-09-16T00:08:47.619 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.619 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.619 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.620 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:47.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:47.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:08:47.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6300000000000001e-07 2024-09-16T00:08:47.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615569+0000", 2024-09-16T00:08:47.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3709999999999998e-06 2024-09-16T00:08:47.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615580+0000", 2024-09-16T00:08:47.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0448000000000001e-05 2024-09-16T00:08:47.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.622 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.622 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615584+0000", 2024-09-16T00:08:47.622 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6020000000000002e-06 2024-09-16T00:08:47.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.622 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.622 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616850+0000", 2024-09-16T00:08:47.622 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012661110000000001 2024-09-16T00:08:47.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.622 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616915+0000", 2024-09-16T00:08:47.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4820000000000006e-05 2024-09-16T00:08:47.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:47.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617031+0000", 2024-09-16T00:08:47.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115258 2024-09-16T00:08:47.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:47.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618659+0000", 2024-09-16T00:08:47.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001627975 2024-09-16T00:08:47.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.624 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:47.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621179+0000", 2024-09-16T00:08:47.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025203410000000002 2024-09-16T00:08:47.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.624 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621238+0000", 2024-09-16T00:08:47.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9323000000000001e-05 2024-09-16T00:08:47.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.624 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621269+0000", 2024-09-16T00:08:47.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0839000000000001e-05 2024-09-16T00:08:47.625 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.625 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.625 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.625 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:47.625 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:47.625 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.981699652000003, 2024-09-16T00:08:47.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058012769999999996, 2024-09-16T00:08:47.625 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.625 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.625 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.625 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.626 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:47.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.626 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:08:47.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.626 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615591+0000", 2024-09-16T00:08:47.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6199999999999999e-07 2024-09-16T00:08:47.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.627 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615594+0000", 2024-09-16T00:08:47.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8739999999999999e-06 2024-09-16T00:08:47.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.627 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615599+0000", 2024-09-16T00:08:47.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:08:47.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615603+0000", 2024-09-16T00:08:47.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8489999999999999e-06 2024-09-16T00:08:47.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617235+0000", 2024-09-16T00:08:47.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001632037 2024-09-16T00:08:47.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617348+0000", 2024-09-16T00:08:47.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113387 2024-09-16T00:08:47.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.629 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:47.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617458+0000", 2024-09-16T00:08:47.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109765 2024-09-16T00:08:47.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.629 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:47.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618664+0000", 2024-09-16T00:08:47.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012059950000000001 2024-09-16T00:08:47.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.629 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:47.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621311+0000", 2024-09-16T00:08:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026468469999999999 2024-09-16T00:08:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621357+0000", 2024-09-16T00:08:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6360999999999999e-05 2024-09-16T00:08:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621392+0000", 2024-09-16T00:08:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4190999999999998e-05 2024-09-16T00:08:47.630 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.981681027999997, 2024-09-16T00:08:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058737279999999999, 2024-09-16T00:08:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.631 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:08:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3799999999999999e-07 2024-09-16T00:08:47.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615611+0000", 2024-09-16T00:08:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-06 2024-09-16T00:08:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615626+0000", 2024-09-16T00:08:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:08:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615634+0000", 2024-09-16T00:08:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-06 2024-09-16T00:08:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617692+0000", 2024-09-16T00:08:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002057718 2024-09-16T00:08:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617768+0000", 2024-09-16T00:08:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6878999999999994e-05 2024-09-16T00:08:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019501699999999999 2024-09-16T00:08:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:08:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034357200000000002 2024-09-16T00:08:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621429+0000", 2024-09-16T00:08:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0012999999999999e-05 2024-09-16T00:08:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621459+0000", 2024-09-16T00:08:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9498000000000001e-05 2024-09-16T00:08:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621483+0000", 2024-09-16T00:08:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3957000000000001e-05 2024-09-16T00:08:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.981649747999995, 2024-09-16T00:08:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059371789999999999, 2024-09-16T00:08:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:08:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:08:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4000000000000002e-07 2024-09-16T00:08:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615643+0000", 2024-09-16T00:08:47.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6860000000000002e-06 2024-09-16T00:08:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615649+0000", 2024-09-16T00:08:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7610000000000004e-06 2024-09-16T00:08:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615653+0000", 2024-09-16T00:08:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0380000000000001e-06 2024-09-16T00:08:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618197+0000", 2024-09-16T00:08:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025440079999999999 2024-09-16T00:08:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618312+0000", 2024-09-16T00:08:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115083 2024-09-16T00:08:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618412+0000", 2024-09-16T00:08:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010016799999999999 2024-09-16T00:08:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621518+0000", 2024-09-16T00:08:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031058679999999999 2024-09-16T00:08:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621526+0000", 2024-09-16T00:08:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2490000000000004e-06 2024-09-16T00:08:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621549+0000", 2024-09-16T00:08:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3686999999999999e-05 2024-09-16T00:08:47.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621577+0000", 2024-09-16T00:08:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8191000000000002e-05 2024-09-16T00:08:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.981632082000004, 2024-09-16T00:08:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005989622, 2024-09-16T00:08:47.642 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:08:47.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3799999999999998e-07 2024-09-16T00:08:47.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615660+0000", 2024-09-16T00:08:47.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9740000000000001e-06 2024-09-16T00:08:47.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615666+0000", 2024-09-16T00:08:47.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3499999999999996e-06 2024-09-16T00:08:47.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.645 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615671+0000", 2024-09-16T00:08:47.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.13e-06 2024-09-16T00:08:47.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.645 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618759+0000", 2024-09-16T00:08:47.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030880500000000002 2024-09-16T00:08:47.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.645 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618832+0000", 2024-09-16T00:08:47.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3725000000000003e-05 2024-09-16T00:08:47.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.646 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:08:47.646 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618958+0000", 2024-09-16T00:08:47.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012587899999999999 2024-09-16T00:08:47.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.646 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:47.646 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621583+0000", 2024-09-16T00:08:47.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026246469999999999 2024-09-16T00:08:47.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:47.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621602+0000", 2024-09-16T00:08:47.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9137e-05 2024-09-16T00:08:47.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621629+0000", 2024-09-16T00:08:47.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6451000000000001e-05 2024-09-16T00:08:47.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621648+0000", 2024-09-16T00:08:47.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8941e-05 2024-09-16T00:08:47.647 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.648 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.648 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.648 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:47.648 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:47.648 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.981614463, 2024-09-16T00:08:47.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053141380000000004, 2024-09-16T00:08:47.648 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.648 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.648 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.648 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:47.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:47.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:08:47.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-07 2024-09-16T00:08:47.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.650 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.650 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615678+0000", 2024-09-16T00:08:47.650 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:08:47.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.650 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.650 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615682+0000", 2024-09-16T00:08:47.650 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6659999999999999e-06 2024-09-16T00:08:47.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.650 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.650 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615686+0000", 2024-09-16T00:08:47.651 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6990000000000001e-06 2024-09-16T00:08:47.651 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.651 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.651 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.651 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615736+0000", 2024-09-16T00:08:47.651 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9656e-05 2024-09-16T00:08:47.651 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.651 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.651 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.651 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615864+0000", 2024-09-16T00:08:47.651 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012836100000000001 2024-09-16T00:08:47.651 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.651 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.652 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:47.652 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615989+0000", 2024-09-16T00:08:47.652 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000125525 2024-09-16T00:08:47.652 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.652 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:47.652 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617355+0000", 2024-09-16T00:08:47.652 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001365982 2024-09-16T00:08:47.652 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.652 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:47.652 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620897+0000", 2024-09-16T00:08:47.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035419610000000002 2024-09-16T00:08:47.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.653 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.653 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620941+0000", 2024-09-16T00:08:47.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3998999999999997e-05 2024-09-16T00:08:47.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.653 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.653 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620990+0000", 2024-09-16T00:08:47.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8199999999999999e-05 2024-09-16T00:08:47.653 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.653 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.654 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.654 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:47.654 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:47.654 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.981587204999997, 2024-09-16T00:08:47.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006316449, 2024-09-16T00:08:47.654 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.654 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.654 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:47.654 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:47.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.655 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:47.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.655 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:08:47.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3299999999999998e-07 2024-09-16T00:08:47.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.656 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.656 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.656 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615705+0000", 2024-09-16T00:08:47.656 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.278e-06 2024-09-16T00:08:47.656 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.656 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.656 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.656 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615711+0000", 2024-09-16T00:08:47.656 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8119999999999996e-06 2024-09-16T00:08:47.656 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.656 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.656 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.657 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615716+0000", 2024-09-16T00:08:47.657 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4909999999999999e-06 2024-09-16T00:08:47.657 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.657 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.657 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.657 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617551+0000", 2024-09-16T00:08:47.657 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00183549 2024-09-16T00:08:47.657 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.657 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.657 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.657 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617627+0000", 2024-09-16T00:08:47.657 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6163999999999998e-05 2024-09-16T00:08:47.657 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:47.658 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617776+0000", 2024-09-16T00:08:47.658 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014909499999999999 2024-09-16T00:08:47.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:47.658 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621072+0000", 2024-09-16T00:08:47.658 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032955150000000002 2024-09-16T00:08:47.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:47.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621957+0000", 2024-09-16T00:08:47.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088553199999999996 2024-09-16T00:08:47.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.659 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:47.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621998+0000", 2024-09-16T00:08:47.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0815000000000001e-05 2024-09-16T00:08:47.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.659 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:08:47.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0924000000000002e-05 2024-09-16T00:08:47.659 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.660 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.660 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.660 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.660 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.660 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:6 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:47.660 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:47.660 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.978185237000005, 2024-09-16T00:08:47.660 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000188111, 2024-09-16T00:08:47.660 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.660 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.660 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:47.660 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.661 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.661 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.661 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:47.661 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.661 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.661 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.661 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.661 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619105+0000", 2024-09-16T00:08:47.661 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.661 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.661 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.661 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.661 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619106+0000", 2024-09-16T00:08:47.662 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4399999999999998e-07 2024-09-16T00:08:47.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.662 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.662 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.662 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619110+0000", 2024-09-16T00:08:47.662 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1110000000000001e-06 2024-09-16T00:08:47.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.662 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.662 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.662 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619120+0000", 2024-09-16T00:08:47.662 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7909999999999999e-06 2024-09-16T00:08:47.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.663 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.663 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.663 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619133+0000", 2024-09-16T00:08:47.663 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.359e-05 2024-09-16T00:08:47.663 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.663 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.663 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.663 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619179+0000", 2024-09-16T00:08:47.663 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5459e-05 2024-09-16T00:08:47.663 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.663 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.663 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.663 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619186+0000", 2024-09-16T00:08:47.664 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0480000000000001e-06 2024-09-16T00:08:47.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.664 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.664 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.664 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619293+0000", 2024-09-16T00:08:47.664 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000107168 2024-09-16T00:08:47.664 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.664 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.664 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.664 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.664 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:22 2.e e22/19 ondisk, result = 0)", 2024-09-16T00:08:47.665 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:47.665 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.976494536000004, 2024-09-16T00:08:47.665 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020058200000000001, 2024-09-16T00:08:47.665 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.665 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.665 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:47.665 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.665 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.665 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.665 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:47.665 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.665 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.665 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620795+0000", 2024-09-16T00:08:47.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:47.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0109999999999999e-06 2024-09-16T00:08:47.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620801+0000", 2024-09-16T00:08:47.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6779999999999999e-06 2024-09-16T00:08:47.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620814+0000", 2024-09-16T00:08:47.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3131e-05 2024-09-16T00:08:47.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620821+0000", 2024-09-16T00:08:47.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0430000000000002e-06 2024-09-16T00:08:47.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:47.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3010999999999997e-05 2024-09-16T00:08:47.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.668 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620891+0000", 2024-09-16T00:08:47.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1760000000000004e-06 2024-09-16T00:08:47.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.668 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620996+0000", 2024-09-16T00:08:47.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010453199999999999 2024-09-16T00:08:47.668 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.669 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.669 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.669 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.669 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:7 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:47.669 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:47.669 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.976486551999997, 2024-09-16T00:08:47.669 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000195725, 2024-09-16T00:08:47.669 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.669 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.669 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:47.669 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:47.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620803+0000", 2024-09-16T00:08:47.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620804+0000", 2024-09-16T00:08:47.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6099999999999999e-07 2024-09-16T00:08:47.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620809+0000", 2024-09-16T00:08:47.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4950000000000002e-06 2024-09-16T00:08:47.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:47.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0553e-05 2024-09-16T00:08:47.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620826+0000", 2024-09-16T00:08:47.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5579999999999997e-06 2024-09-16T00:08:47.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620874+0000", 2024-09-16T00:08:47.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7648999999999997e-05 2024-09-16T00:08:47.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620880+0000", 2024-09-16T00:08:47.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8290000000000002e-06 2024-09-16T00:08:47.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620999+0000", 2024-09-16T00:08:47.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011878 2024-09-16T00:08:47.673 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.673 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.673 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.673 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:8 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:47.673 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:47.673 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.976434491999996, 2024-09-16T00:08:47.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00027991099999999998, 2024-09-16T00:08:47.674 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.674 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.674 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:47.674 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.674 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.674 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:47.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.674 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.674 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:47.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.675 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.675 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.675 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.675 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620856+0000", 2024-09-16T00:08:47.675 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6800000000000001e-07 2024-09-16T00:08:47.675 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.675 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.675 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.675 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620859+0000", 2024-09-16T00:08:47.675 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6879999999999999e-06 2024-09-16T00:08:47.675 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.675 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.676 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620867+0000", 2024-09-16T00:08:47.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2819999999999996e-06 2024-09-16T00:08:47.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.676 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.676 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620872+0000", 2024-09-16T00:08:47.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5369999999999997e-06 2024-09-16T00:08:47.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.676 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.676 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621043+0000", 2024-09-16T00:08:47.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170791 2024-09-16T00:08:47.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621050+0000", 2024-09-16T00:08:47.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8580000000000002e-06 2024-09-16T00:08:47.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621135+0000", 2024-09-16T00:08:47.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5387000000000001e-05 2024-09-16T00:08:47.677 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.677 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.677 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.678 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:9 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:47.678 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:47.678 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.976405913999997, 2024-09-16T00:08:47.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00039143799999999997, 2024-09-16T00:08:47.678 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.678 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.678 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:47.678 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:47.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:47.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620884+0000", 2024-09-16T00:08:47.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6800000000000002e-07 2024-09-16T00:08:47.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:08:47.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0310000000000001e-06 2024-09-16T00:08:47.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620892+0000", 2024-09-16T00:08:47.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9390000000000002e-06 2024-09-16T00:08:47.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620898+0000", 2024-09-16T00:08:47.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4480000000000002e-06 2024-09-16T00:08:47.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621167+0000", 2024-09-16T00:08:47.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026960599999999998 2024-09-16T00:08:47.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.681 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.681 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621174+0000", 2024-09-16T00:08:47.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.567e-06 2024-09-16T00:08:47.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.681 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.681 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621275+0000", 2024-09-16T00:08:47.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101579 2024-09-16T00:08:47.681 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.681 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.682 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.682 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:10 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:47.682 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:47.682 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.976383466000001, 2024-09-16T00:08:47.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00049416200000000003, 2024-09-16T00:08:47.682 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.682 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.682 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:47.682 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:47.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620906+0000", 2024-09-16T00:08:47.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620907+0000", 2024-09-16T00:08:47.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.42e-07 2024-09-16T00:08:47.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620908+0000", 2024-09-16T00:08:47.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4300000000000001e-06 2024-09-16T00:08:47.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620911+0000", 2024-09-16T00:08:47.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3890000000000001e-06 2024-09-16T00:08:47.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620914+0000", 2024-09-16T00:08:47.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3690000000000001e-06 2024-09-16T00:08:47.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621297+0000", 2024-09-16T00:08:47.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038319200000000003 2024-09-16T00:08:47.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621305+0000", 2024-09-16T00:08:47.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4780000000000008e-06 2024-09-16T00:08:47.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621401+0000", 2024-09-16T00:08:47.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5161999999999995e-05 2024-09-16T00:08:47.685 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.686 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.686 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.686 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:11 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:47.686 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:47.686 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.975980716999999, 2024-09-16T00:08:47.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017943899999999999, 2024-09-16T00:08:47.686 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.686 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.686 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:47.686 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:47.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621309+0000", 2024-09-16T00:08:47.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621310+0000", 2024-09-16T00:08:47.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5199999999999996e-07 2024-09-16T00:08:47.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:47.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1740000000000002e-06 2024-09-16T00:08:47.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621324+0000", 2024-09-16T00:08:47.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7170000000000003e-06 2024-09-16T00:08:47.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621331+0000", 2024-09-16T00:08:47.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2280000000000001e-06 2024-09-16T00:08:47.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621422+0000", 2024-09-16T00:08:47.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1052999999999997e-05 2024-09-16T00:08:47.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621426+0000", 2024-09-16T00:08:47.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.036e-06 2024-09-16T00:08:47.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621489+0000", 2024-09-16T00:08:47.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2478999999999996e-05 2024-09-16T00:08:47.690 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.690 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.690 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.690 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:12 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:47.690 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:47.690 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.975929862000001, 2024-09-16T00:08:47.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022422300000000001, 2024-09-16T00:08:47.690 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.691 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.691 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:47.691 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:47.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:47.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621360+0000", 2024-09-16T00:08:47.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6699999999999999e-07 2024-09-16T00:08:47.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621363+0000", 2024-09-16T00:08:47.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7020000000000001e-06 2024-09-16T00:08:47.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621371+0000", 2024-09-16T00:08:47.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7209999999999994e-06 2024-09-16T00:08:47.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621376+0000", 2024-09-16T00:08:47.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7940000000000002e-06 2024-09-16T00:08:47.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621511+0000", 2024-09-16T00:08:47.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013518100000000001 2024-09-16T00:08:47.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621514+0000", 2024-09-16T00:08:47.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5599999999999998e-06 2024-09-16T00:08:47.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621584+0000", 2024-09-16T00:08:47.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9898000000000001e-05 2024-09-16T00:08:47.694 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.695 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.695 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.695 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:13 2.10 e22/19 ondisk, result = 0)", 2024-09-16T00:08:47.695 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:47.695 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.975903598000002, 2024-09-16T00:08:47.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026534099999999999, 2024-09-16T00:08:47.695 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.695 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.695 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:47.695 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:47.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621386+0000", 2024-09-16T00:08:47.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621387+0000", 2024-09-16T00:08:47.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4299999999999999e-07 2024-09-16T00:08:47.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621389+0000", 2024-09-16T00:08:47.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.593e-06 2024-09-16T00:08:47.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621395+0000", 2024-09-16T00:08:47.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.553e-06 2024-09-16T00:08:47.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:08:47.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8410000000000002e-06 2024-09-16T00:08:47.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621597+0000", 2024-09-16T00:08:47.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019794100000000001 2024-09-16T00:08:47.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621600+0000", 2024-09-16T00:08:47.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.185e-06 2024-09-16T00:08:47.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621652+0000", 2024-09-16T00:08:47.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1984999999999998e-05 2024-09-16T00:08:47.699 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.699 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.699 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.700 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop_reply(mds.0.4:23 2.e e22/19 ondisk, result = 0)", 2024-09-16T00:08:47.700 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:47.700 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.975420649, 2024-09-16T00:08:47.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015895699999999999, 2024-09-16T00:08:47.700 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:47.700 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:47.700 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:47.700 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:47.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:47.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:47.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:47.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621869+0000", 2024-09-16T00:08:47.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:47.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:47.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621870+0000", 2024-09-16T00:08:47.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1600000000000001e-07 2024-09-16T00:08:47.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:47.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621873+0000", 2024-09-16T00:08:47.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3079999999999999e-06 2024-09-16T00:08:47.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:47.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621882+0000", 2024-09-16T00:08:47.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8429999999999999e-06 2024-09-16T00:08:47.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:47.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621896+0000", 2024-09-16T00:08:47.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:08:47.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:47.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621944+0000", 2024-09-16T00:08:47.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8062999999999997e-05 2024-09-16T00:08:47.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:47.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621951+0000", 2024-09-16T00:08:47.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7689999999999999e-06 2024-09-16T00:08:47.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:47.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:47.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:47.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622028+0000", 2024-09-16T00:08:47.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7243999999999994e-05 2024-09-16T00:08:47.704 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.704 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.704 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.704 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:47.704 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:47.704 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:47.706 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:08:47.707 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:47.707+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:47.807 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:47.805+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:47.856 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:47.856 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:47.856 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:47.856 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:47.866 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:08:47.908 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:47.908+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:47.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.946+0000 7f16227fc640 1 -- 172.21.15.139:0/1884282057 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v52) ==== 91+0+0 (secure 0 0 0) 0x7f161c0799c0 con 0x7f162c0ccff0 2024-09-16T00:08:47.949 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is unset 2024-09-16T00:08:47.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.950+0000 7f1633fb9640 1 -- 172.21.15.139:0/1884282057 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f161006c2b0 msgr2=0x7f161006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:47.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.950+0000 7f1633fb9640 1 --2- 172.21.15.139:0/1884282057 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f161006c2b0 0x7f161006e770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1628001840 tx=0x7f1628009040 comp rx=0 tx=0).stop 2024-09-16T00:08:47.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.950+0000 7f1633fb9640 1 -- 172.21.15.139:0/1884282057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f162c0ccff0 msgr2=0x7f162c159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:47.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.950+0000 7f1633fb9640 1 --2- 172.21.15.139:0/1884282057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f162c0ccff0 0x7f162c159c50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f161c031000 tx=0x7f161c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.950+0000 7f1633fb9640 1 -- 172.21.15.139:0/1884282057 shutdown_connections 2024-09-16T00:08:47.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.950+0000 7f1633fb9640 1 --2- 172.21.15.139:0/1884282057 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f161006c2b0 0x7f161006e770 secure :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1628001840 tx=0x7f1628009040 comp rx=0 tx=0).stop 2024-09-16T00:08:47.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.950+0000 7f1633fb9640 1 --2- 172.21.15.139:0/1884282057 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f162c15b390 0x7f162c150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.950+0000 7f1633fb9640 1 --2- 172.21.15.139:0/1884282057 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f162c0ccff0 0x7f162c159c50 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f161c031000 tx=0x7f161c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.950+0000 7f1633fb9640 1 --2- 172.21.15.139:0/1884282057 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f162c0c8040 0x7f162c0c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:47.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.950+0000 7f1633fb9640 1 -- 172.21.15.139:0/1884282057 >> 172.21.15.139:0/1884282057 conn(0x7f162c06cc30 msgr2=0x7f162c06d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:47.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.950+0000 7f1633fb9640 1 -- 172.21.15.139:0/1884282057 shutdown_connections 2024-09-16T00:08:47.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:47.950+0000 7f1633fb9640 1 -- 172.21.15.139:0/1884282057 wait complete. 2024-09-16T00:08:48.009 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:48.009+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:48.046 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:48.046 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:48.046 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:48.046 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:48.047 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:48.056 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:08:48.110 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:48.109+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:48.186 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:48.186 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:48.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:48.186 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:48.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.186 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:22 2.e e22/19 2:717a0223:::608.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:48.186 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:48.186 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 69.567554954000002, 2024-09-16T00:08:48.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042195870000000003, 2024-09-16T00:08:48.186 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.187 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.187 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:48.187 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:48.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:48.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616426+0000", 2024-09-16T00:08:48.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.037e-06 2024-09-16T00:08:48.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616435+0000", 2024-09-16T00:08:48.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6440000000000006e-06 2024-09-16T00:08:48.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616447+0000", 2024-09-16T00:08:48.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2466000000000001e-05 2024-09-16T00:08:48.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616456+0000", 2024-09-16T00:08:48.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8799999999999997e-06 2024-09-16T00:08:48.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616517+0000", 2024-09-16T00:08:48.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1171000000000001e-05 2024-09-16T00:08:48.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:48.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616540+0000", 2024-09-16T00:08:48.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2991000000000001e-05 2024-09-16T00:08:48.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:48.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620599+0000", 2024-09-16T00:08:48.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040586140000000003 2024-09-16T00:08:48.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620645+0000", 2024-09-16T00:08:48.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5784000000000001e-05 2024-09-16T00:08:48.191 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.191 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.191 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.192 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:48.192 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:48.192 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 69.567465224000003, 2024-09-16T00:08:48.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044836249999999998, 2024-09-16T00:08:48.192 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.192 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.192 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:48.192 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.192 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:48.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:48.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616518+0000", 2024-09-16T00:08:48.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3699999999999999e-06 2024-09-16T00:08:48.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:08:48.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1479999999999997e-06 2024-09-16T00:08:48.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616534+0000", 2024-09-16T00:08:48.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0641e-05 2024-09-16T00:08:48.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616541+0000", 2024-09-16T00:08:48.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3339999999999996e-06 2024-09-16T00:08:48.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617318+0000", 2024-09-16T00:08:48.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00077694799999999996 2024-09-16T00:08:48.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:48.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617409+0000", 2024-09-16T00:08:48.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0740000000000002e-05 2024-09-16T00:08:48.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620998+0000", 2024-09-16T00:08:48.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035894439999999998 2024-09-16T00:08:48.196 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.197 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.197 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.197 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:48.197 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:48.197 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 69.567417320999994, 2024-09-16T00:08:48.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041597320000000002, 2024-09-16T00:08:48.198 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.198 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.198 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:48.198 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:48.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:48.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:48.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:08:48.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616566+0000", 2024-09-16T00:08:48.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6979999999999998e-06 2024-09-16T00:08:48.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616574+0000", 2024-09-16T00:08:48.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7759999999999997e-06 2024-09-16T00:08:48.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616580+0000", 2024-09-16T00:08:48.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:48.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616800+0000", 2024-09-16T00:08:48.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022016700000000001 2024-09-16T00:08:48.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:48.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616904+0000", 2024-09-16T00:08:48.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104579 2024-09-16T00:08:48.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620722+0000", 2024-09-16T00:08:48.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003818133 2024-09-16T00:08:48.204 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.205 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.205 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.205 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:48.205 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:48.205 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 69.567166635999996, 2024-09-16T00:08:48.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038451570000000001, 2024-09-16T00:08:48.206 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.206 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.206 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:48.206 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:48.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:48.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616814+0000", 2024-09-16T00:08:48.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9299999999999997e-07 2024-09-16T00:08:48.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616824+0000", 2024-09-16T00:08:48.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6600000000000007e-06 2024-09-16T00:08:48.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616835+0000", 2024-09-16T00:08:48.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0713000000000001e-05 2024-09-16T00:08:48.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616841+0000", 2024-09-16T00:08:48.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.934e-06 2024-09-16T00:08:48.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616948+0000", 2024-09-16T00:08:48.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010631100000000001 2024-09-16T00:08:48.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:48.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616967+0000", 2024-09-16T00:08:48.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9598000000000001e-05 2024-09-16T00:08:48.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:48.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620619+0000", 2024-09-16T00:08:48.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036515810000000001 2024-09-16T00:08:48.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620659+0000", 2024-09-16T00:08:48.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9666999999999998e-05 2024-09-16T00:08:48.211 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.211 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.211 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.211 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:48.211 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:48.212 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 69.567119684000005, 2024-09-16T00:08:48.212 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038240129999999998, 2024-09-16T00:08:48.212 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.212 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.212 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:48.212 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.212 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.212 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:48.212 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.212 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:48.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616861+0000", 2024-09-16T00:08:48.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0200000000000003e-07 2024-09-16T00:08:48.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616864+0000", 2024-09-16T00:08:48.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5090000000000002e-06 2024-09-16T00:08:48.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616871+0000", 2024-09-16T00:08:48.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9249999999999998e-06 2024-09-16T00:08:48.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616877+0000", 2024-09-16T00:08:48.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.981e-06 2024-09-16T00:08:48.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617168+0000", 2024-09-16T00:08:48.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029035699999999998 2024-09-16T00:08:48.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:48.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617184+0000", 2024-09-16T00:08:48.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6005999999999999e-05 2024-09-16T00:08:48.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:48.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620661+0000", 2024-09-16T00:08:48.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034775330000000001 2024-09-16T00:08:48.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620684+0000", 2024-09-16T00:08:48.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3300000000000001e-05 2024-09-16T00:08:48.216 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.216 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.217 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.217 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:48.217 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:48.217 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 69.566664240999998, 2024-09-16T00:08:48.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045415610000000004, 2024-09-16T00:08:48.217 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.217 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.217 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:48.217 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.218 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:48.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.218 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:48.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.218 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:48.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8800000000000002e-07 2024-09-16T00:08:48.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.219 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.219 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.219 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617321+0000", 2024-09-16T00:08:48.219 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6979999999999997e-06 2024-09-16T00:08:48.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.219 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.219 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.219 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617330+0000", 2024-09-16T00:08:48.219 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6729999999999998e-06 2024-09-16T00:08:48.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.219 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.219 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.219 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617336+0000", 2024-09-16T00:08:48.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2110000000000003e-06 2024-09-16T00:08:48.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617394+0000", 2024-09-16T00:08:48.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8087999999999997e-05 2024-09-16T00:08:48.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:48.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617412+0000", 2024-09-16T00:08:48.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8099999999999999e-05 2024-09-16T00:08:48.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:48.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621818+0000", 2024-09-16T00:08:48.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044061209999999998 2024-09-16T00:08:48.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621857+0000", 2024-09-16T00:08:48.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9082000000000003e-05 2024-09-16T00:08:48.221 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.221 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.222 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.222 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:23 2.e e22/19 2:717a0223:::608.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:48.222 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:48.222 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 69.565768219999995, 2024-09-16T00:08:48.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036198279999999998, 2024-09-16T00:08:48.222 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.222 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.222 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:48.222 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:48.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:48.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618213+0000", 2024-09-16T00:08:48.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:08:48.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618218+0000", 2024-09-16T00:08:48.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4099999999999999e-06 2024-09-16T00:08:48.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618227+0000", 2024-09-16T00:08:48.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5009999999999993e-06 2024-09-16T00:08:48.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618234+0000", 2024-09-16T00:08:48.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:48.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618307+0000", 2024-09-16T00:08:48.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3421000000000001e-05 2024-09-16T00:08:48.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:48.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618328+0000", 2024-09-16T00:08:48.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0919000000000001e-05 2024-09-16T00:08:48.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:48.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621804+0000", 2024-09-16T00:08:48.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034756560000000001 2024-09-16T00:08:48.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621832+0000", 2024-09-16T00:08:48.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7966999999999998e-05 2024-09-16T00:08:48.227 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.227 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.227 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.227 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing_reply(2.1e epoch 22/19 rep_tid 0 lcod 22'1)", 2024-09-16T00:08:48.227 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:48.227 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 69.565323051999997, 2024-09-16T00:08:48.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1371000000000003e-05, 2024-09-16T00:08:48.227 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.227 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.228 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:48.228 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:48.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:48.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.229 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.229 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:08:48.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:08:48.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.229 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618662+0000", 2024-09-16T00:08:48.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1189999999999999e-06 2024-09-16T00:08:48.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618674+0000", 2024-09-16T00:08:48.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2317000000000001e-05 2024-09-16T00:08:48.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618681+0000", 2024-09-16T00:08:48.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7170000000000002e-06 2024-09-16T00:08:48.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:08:48.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6745000000000001e-05 2024-09-16T00:08:48.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618748+0000", 2024-09-16T00:08:48.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0794000000000001e-05 2024-09-16T00:08:48.233 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.234 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.234 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.234 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:48.234 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:48.234 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 69.564809718000006, 2024-09-16T00:08:48.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027183849999999998, 2024-09-16T00:08:48.234 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.235 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.235 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:48.235 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.235 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:48.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.236 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:48.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619171+0000", 2024-09-16T00:08:48.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8999999999999995e-07 2024-09-16T00:08:48.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619179+0000", 2024-09-16T00:08:48.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2679999999999994e-06 2024-09-16T00:08:48.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:08:48.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1246e-05 2024-09-16T00:08:48.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619197+0000", 2024-09-16T00:08:48.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2820000000000002e-06 2024-09-16T00:08:48.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619308+0000", 2024-09-16T00:08:48.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011141000000000001 2024-09-16T00:08:48.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:48.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619325+0000", 2024-09-16T00:08:48.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6469000000000001e-05 2024-09-16T00:08:48.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:48.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621860+0000", 2024-09-16T00:08:48.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025353590000000001 2024-09-16T00:08:48.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621889+0000", 2024-09-16T00:08:48.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8561e-05 2024-09-16T00:08:48.242 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.242 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.242 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.242 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 6 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:48.242 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:48.242 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 69.564767019000001, 2024-09-16T00:08:48.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026903809999999999, 2024-09-16T00:08:48.243 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.243 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.243 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:48.243 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:48.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:48.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:48.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2000000000000001e-07 2024-09-16T00:08:48.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619219+0000", 2024-09-16T00:08:48.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:08:48.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619234+0000", 2024-09-16T00:08:48.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5075000000000001e-05 2024-09-16T00:08:48.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619251+0000", 2024-09-16T00:08:48.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6863000000000001e-05 2024-09-16T00:08:48.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619556+0000", 2024-09-16T00:08:48.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030551599999999998 2024-09-16T00:08:48.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621903+0000", 2024-09-16T00:08:48.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023469979999999999 2024-09-16T00:08:48.247 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.247 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.247 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.248 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing_reply(2.1b epoch 22/19 rep_tid 1 lcod 22'1)", 2024-09-16T00:08:48.248 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:48.248 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 69.563688623000004, 2024-09-16T00:08:48.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6989000000000003e-05, 2024-09-16T00:08:48.248 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.248 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.248 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:48.248 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:48.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:48.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620292+0000", 2024-09-16T00:08:48.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.37e-07 2024-09-16T00:08:48.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620296+0000", 2024-09-16T00:08:48.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2420000000000002e-06 2024-09-16T00:08:48.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620308+0000", 2024-09-16T00:08:48.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1623e-05 2024-09-16T00:08:48.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620313+0000", 2024-09-16T00:08:48.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6290000000000001e-06 2024-09-16T00:08:48.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620359+0000", 2024-09-16T00:08:48.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6201e-05 2024-09-16T00:08:48.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620378+0000", 2024-09-16T00:08:48.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9556999999999999e-05 2024-09-16T00:08:48.252 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.252 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.252 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.252 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:38 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'6, mlcod=22'4)", 2024-09-16T00:08:48.252 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:48.252 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 69.561246374000007, 2024-09-16T00:08:48.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064551599999999995, 2024-09-16T00:08:48.252 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.253 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.253 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:48.253 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:48.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:48.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:48.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4399999999999999e-07 2024-09-16T00:08:48.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622739+0000", 2024-09-16T00:08:48.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0250000000000002e-06 2024-09-16T00:08:48.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622749+0000", 2024-09-16T00:08:48.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1220000000000001e-06 2024-09-16T00:08:48.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622760+0000", 2024-09-16T00:08:48.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1640999999999999e-05 2024-09-16T00:08:48.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622799+0000", 2024-09-16T00:08:48.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8537000000000001e-05 2024-09-16T00:08:48.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:48.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622810+0000", 2024-09-16T00:08:48.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1562e-05 2024-09-16T00:08:48.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:48.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.623356+0000", 2024-09-16T00:08:48.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054522299999999995 2024-09-16T00:08:48.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.623379+0000", 2024-09-16T00:08:48.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3662000000000001e-05 2024-09-16T00:08:48.257 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.257 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.257 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.257 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:40 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'7, mlcod=22'5)", 2024-09-16T00:08:48.258 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:48.258 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 57.344225471999998, 2024-09-16T00:08:48.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001017773, 2024-09-16T00:08:48.258 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.258 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.259 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:48.259 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:48.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:48.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839756+0000", 2024-09-16T00:08:48.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.491e-06 2024-09-16T00:08:48.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839768+0000", 2024-09-16T00:08:48.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1649e-05 2024-09-16T00:08:48.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839783+0000", 2024-09-16T00:08:48.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5023e-05 2024-09-16T00:08:48.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839791+0000", 2024-09-16T00:08:48.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5620000000000004e-06 2024-09-16T00:08:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839848+0000", 2024-09-16T00:08:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6563e-05 2024-09-16T00:08:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839868+0000", 2024-09-16T00:08:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9652000000000001e-05 2024-09-16T00:08:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:48.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.840729+0000", 2024-09-16T00:08:48.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086159500000000005 2024-09-16T00:08:48.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.840772+0000", 2024-09-16T00:08:48.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3238000000000002e-05 2024-09-16T00:08:48.265 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.265 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.265 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.265 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:41 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'8, mlcod=22'6)", 2024-09-16T00:08:48.265 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:48.265 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.070568561000002, 2024-09-16T00:08:48.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001040568, 2024-09-16T00:08:48.266 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.266 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.266 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:48.266 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:48.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:48.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113413+0000", 2024-09-16T00:08:48.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7409999999999999e-06 2024-09-16T00:08:48.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113428+0000", 2024-09-16T00:08:48.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5007999999999999e-05 2024-09-16T00:08:48.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113447+0000", 2024-09-16T00:08:48.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8731000000000001e-05 2024-09-16T00:08:48.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113457+0000", 2024-09-16T00:08:48.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0095000000000001e-05 2024-09-16T00:08:48.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113526+0000", 2024-09-16T00:08:48.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8552000000000004e-05 2024-09-16T00:08:48.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:48.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113552+0000", 2024-09-16T00:08:48.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6795999999999999e-05 2024-09-16T00:08:48.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:48.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.114417+0000", 2024-09-16T00:08:48.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086476299999999995 2024-09-16T00:08:48.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.114452+0000", 2024-09-16T00:08:48.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4882000000000002e-05 2024-09-16T00:08:48.271 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.272 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.272 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.272 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.14 38/37 [PushReplyOp(2:2f55791f:::606.00000000:head)])", 2024-09-16T00:08:48.272 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.839962+0000", 2024-09-16T00:08:48.272 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.344018139999999, 2024-09-16T00:08:48.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030257, 2024-09-16T00:08:48.273 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.273 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.273 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:48.273 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839962+0000", 2024-09-16T00:08:48.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839962+0000", 2024-09-16T00:08:48.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839962+0000", 2024-09-16T00:08:48.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5400000000000001e-07 2024-09-16T00:08:48.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839967+0000", 2024-09-16T00:08:48.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.245e-06 2024-09-16T00:08:48.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.276 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.276 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839978+0000", 2024-09-16T00:08:48.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1388e-05 2024-09-16T00:08:48.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839987+0000", 2024-09-16T00:08:48.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1500000000000005e-06 2024-09-16T00:08:48.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.840043+0000", 2024-09-16T00:08:48.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5396000000000003e-05 2024-09-16T00:08:48.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.840265+0000", 2024-09-16T00:08:48.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022183700000000001 2024-09-16T00:08:48.278 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.278 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.278 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.278 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.14 38/37 [PushReplyOp(2:287034c4:::1.00000000.inode:head)])", 2024-09-16T00:08:48.278 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.848174+0000", 2024-09-16T00:08:48.279 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.335805934, 2024-09-16T00:08:48.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029495900000000002, 2024-09-16T00:08:48.279 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.279 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.279 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:48.279 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848174+0000", 2024-09-16T00:08:48.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848174+0000", 2024-09-16T00:08:48.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848175+0000", 2024-09-16T00:08:48.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.68e-07 2024-09-16T00:08:48.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848179+0000", 2024-09-16T00:08:48.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1400000000000002e-06 2024-09-16T00:08:48.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848191+0000", 2024-09-16T00:08:48.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1823e-05 2024-09-16T00:08:48.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848196+0000", 2024-09-16T00:08:48.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.626e-06 2024-09-16T00:08:48.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.282 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848251+0000", 2024-09-16T00:08:48.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4342999999999998e-05 2024-09-16T00:08:48.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.282 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.282 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.282 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848469+0000", 2024-09-16T00:08:48.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000218359 2024-09-16T00:08:48.282 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.282 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.282 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.283 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.14 38/37 [PushReplyOp(2:2e2fa760:::200.00000001:head)])", 2024-09-16T00:08:48.283 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.850960+0000", 2024-09-16T00:08:48.283 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.333019643, 2024-09-16T00:08:48.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00028471600000000002, 2024-09-16T00:08:48.283 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.283 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.283 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:48.283 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850960+0000", 2024-09-16T00:08:48.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850960+0000", 2024-09-16T00:08:48.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850961+0000", 2024-09-16T00:08:48.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6000000000000003e-07 2024-09-16T00:08:48.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850965+0000", 2024-09-16T00:08:48.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1400000000000002e-06 2024-09-16T00:08:48.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850977+0000", 2024-09-16T00:08:48.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2128e-05 2024-09-16T00:08:48.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850985+0000", 2024-09-16T00:08:48.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3220000000000004e-06 2024-09-16T00:08:48.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.851036+0000", 2024-09-16T00:08:48.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1100000000000002e-05 2024-09-16T00:08:48.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.288 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.851245+0000", 2024-09-16T00:08:48.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000209266 2024-09-16T00:08:48.288 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.288 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.288 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.289 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.14 38/37 [PushReplyOp(2:292cf221:::200.00000000:head)])", 2024-09-16T00:08:48.289 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.853964+0000", 2024-09-16T00:08:48.289 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.330016454999999, 2024-09-16T00:08:48.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026211799999999999, 2024-09-16T00:08:48.289 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:48.289 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:48.289 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:48.289 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:48.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:48.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853964+0000", 2024-09-16T00:08:48.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:48.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853964+0000", 2024-09-16T00:08:48.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:48.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:48.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853964+0000", 2024-09-16T00:08:48.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8000000000000002e-07 2024-09-16T00:08:48.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:48.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853967+0000", 2024-09-16T00:08:48.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6529999999999998e-06 2024-09-16T00:08:48.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:48.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853973+0000", 2024-09-16T00:08:48.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9110000000000004e-06 2024-09-16T00:08:48.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853981+0000", 2024-09-16T00:08:48.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.25e-06 2024-09-16T00:08:48.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:48.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.854030+0000", 2024-09-16T00:08:48.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8909000000000002e-05 2024-09-16T00:08:48.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:48.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:48.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:48.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.854226+0000", 2024-09-16T00:08:48.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019611499999999999 2024-09-16T00:08:48.292 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.292 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.292 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.292 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:48.293 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:48.293 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:48.294 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:08:48.295 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:48.292+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:48.395 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:48.394+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:48.412 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:48.412 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:48.412 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:48.412 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:48.421 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:08:48.496 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:48.494+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:48.526 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:48.526 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:48.527 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:48.527 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:48.527 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:48.535 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:08:48.597 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:48.596+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:48.640 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:48.640 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:48.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:48.641 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:48.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.641 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:48.641 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:48.641 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.024706086999998, 2024-09-16T00:08:48.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036770370000000002, 2024-09-16T00:08:48.641 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:48.641 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:48.641 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:48.641 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:48.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:48.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:48.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:48.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:48.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:48.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615154+0000", 2024-09-16T00:08:48.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1510000000000003e-06 2024-09-16T00:08:48.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:48.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615163+0000", 2024-09-16T00:08:48.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4490000000000005e-06 2024-09-16T00:08:48.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:48.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615182+0000", 2024-09-16T00:08:48.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8876e-05 2024-09-16T00:08:48.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615197+0000", 2024-09-16T00:08:48.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4671000000000001e-05 2024-09-16T00:08:48.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:48.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615349+0000", 2024-09-16T00:08:48.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015258700000000001 2024-09-16T00:08:48.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:48.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:08:48.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000204716 2024-09-16T00:08:48.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:48.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:08:48.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017250800000000001 2024-09-16T00:08:48.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:48.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616919+0000", 2024-09-16T00:08:48.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011921799999999999 2024-09-16T00:08:48.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:48.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:08:48.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018087769999999999 2024-09-16T00:08:48.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:48.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618774+0000", 2024-09-16T00:08:48.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6075999999999997e-05 2024-09-16T00:08:48.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:48.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618826+0000", 2024-09-16T00:08:48.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2046000000000003e-05 2024-09-16T00:08:48.647 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.647 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:48.647 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.647 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:48.647 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:48.647 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.023350299000001, 2024-09-16T00:08:48.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023501059999999998, 2024-09-16T00:08:48.647 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:48.647 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:48.647 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:48.647 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:48.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:48.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:48.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:48.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:48.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:48.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616506+0000", 2024-09-16T00:08:48.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4100000000000001e-06 2024-09-16T00:08:48.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:48.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617073+0000", 2024-09-16T00:08:48.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056725899999999997 2024-09-16T00:08:48.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:48.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617099+0000", 2024-09-16T00:08:48.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5860999999999999e-05 2024-09-16T00:08:48.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617114+0000", 2024-09-16T00:08:48.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5180999999999999e-05 2024-09-16T00:08:48.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:48.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617236+0000", 2024-09-16T00:08:48.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121662 2024-09-16T00:08:48.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:48.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:48.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001618733 2024-09-16T00:08:48.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.651 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:48.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.651 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:31 2.d e22/19 2:b50e409b:::mds_snaptable:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:48.651 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:48.651 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.022456151, 2024-09-16T00:08:48.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021851240000000001, 2024-09-16T00:08:48.651 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:48.652 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:48.652 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:48.652 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:48.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:48.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:48.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:48.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:48.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:48.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617400+0000", 2024-09-16T00:08:48.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.28e-06 2024-09-16T00:08:48.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:48.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617410+0000", 2024-09-16T00:08:48.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0179e-05 2024-09-16T00:08:48.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:48.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617435+0000", 2024-09-16T00:08:48.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5221000000000001e-05 2024-09-16T00:08:48.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617444+0000", 2024-09-16T00:08:48.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1989999999999995e-06 2024-09-16T00:08:48.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:48.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617514+0000", 2024-09-16T00:08:48.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9104999999999997e-05 2024-09-16T00:08:48.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:48.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617538+0000", 2024-09-16T00:08:48.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4383999999999999e-05 2024-09-16T00:08:48.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:48.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619536+0000", 2024-09-16T00:08:48.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019982789999999999 2024-09-16T00:08:48.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:48.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619584+0000", 2024-09-16T00:08:48.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7477e-05 2024-09-16T00:08:48.656 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.656 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:48.656 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.656 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:27 2.13 e22/19 2:cd5a64a3:::100.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:48.656 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:48.656 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.022150968999995, 2024-09-16T00:08:48.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018775459999999999, 2024-09-16T00:08:48.656 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:48.656 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:48.656 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:48.656 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:48.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:48.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:48.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:48.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:48.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:48.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617705+0000", 2024-09-16T00:08:48.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9900000000000009e-07 2024-09-16T00:08:48.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:48.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617718+0000", 2024-09-16T00:08:48.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3453e-05 2024-09-16T00:08:48.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:48.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617730+0000", 2024-09-16T00:08:48.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2221e-05 2024-09-16T00:08:48.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617739+0000", 2024-09-16T00:08:48.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0660000000000004e-06 2024-09-16T00:08:48.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:48.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617806+0000", 2024-09-16T00:08:48.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7397999999999994e-05 2024-09-16T00:08:48.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:48.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617823+0000", 2024-09-16T00:08:48.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7569999999999999e-05 2024-09-16T00:08:48.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:48.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619528+0000", 2024-09-16T00:08:48.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017045859999999999 2024-09-16T00:08:48.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:48.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619581+0000", 2024-09-16T00:08:48.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3253e-05 2024-09-16T00:08:48.660 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.660 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:48.660 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.661 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:28 2.13 e22/19 2:c91d4a1d:::mds0_inotable:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:48.661 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:48.661 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.021803699000003, 2024-09-16T00:08:48.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001558051, 2024-09-16T00:08:48.661 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:48.661 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:48.661 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:48.661 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:48.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:48.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:48.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:48.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:48.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:48.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618052+0000", 2024-09-16T00:08:48.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1399999999999997e-07 2024-09-16T00:08:48.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:48.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618065+0000", 2024-09-16T00:08:48.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3495e-05 2024-09-16T00:08:48.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:48.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:08:48.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0560999999999999e-05 2024-09-16T00:08:48.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618083+0000", 2024-09-16T00:08:48.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:08:48.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:48.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618147+0000", 2024-09-16T00:08:48.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4525999999999999e-05 2024-09-16T00:08:48.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:48.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618166+0000", 2024-09-16T00:08:48.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9045000000000001e-05 2024-09-16T00:08:48.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:48.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619585+0000", 2024-09-16T00:08:48.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014182660000000001 2024-09-16T00:08:48.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:48.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619609+0000", 2024-09-16T00:08:48.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4468e-05 2024-09-16T00:08:48.665 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.665 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:48.665 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.665 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:29 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:48.665 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:48.665 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.021453195000007, 2024-09-16T00:08:48.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012347219999999999, 2024-09-16T00:08:48.666 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:48.666 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:48.666 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:48.666 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:48.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:48.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:48.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:48.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:48.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:48.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:48.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3099999999999997e-07 2024-09-16T00:08:48.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:48.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618409+0000", 2024-09-16T00:08:48.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6239999999999996e-06 2024-09-16T00:08:48.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:48.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618420+0000", 2024-09-16T00:08:48.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1222e-05 2024-09-16T00:08:48.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618441+0000", 2024-09-16T00:08:48.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0564000000000002e-05 2024-09-16T00:08:48.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:48.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618519+0000", 2024-09-16T00:08:48.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7917999999999995e-05 2024-09-16T00:08:48.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:48.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618535+0000", 2024-09-16T00:08:48.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6266e-05 2024-09-16T00:08:48.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:48.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619612+0000", 2024-09-16T00:08:48.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010767039999999999 2024-09-16T00:08:48.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:48.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619636+0000", 2024-09-16T00:08:48.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4692999999999999e-05 2024-09-16T00:08:48.669 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.670 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:48.670 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.670 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:2 2.f e22/19 ondisk, result = 0)", 2024-09-16T00:08:48.670 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:48.670 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.021217168000007, 2024-09-16T00:08:48.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019782099999999999, 2024-09-16T00:08:48.670 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:48.670 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:48.670 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:48.670 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:48.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:48.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:48.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:48.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:48.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:48.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618639+0000", 2024-09-16T00:08:48.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5099999999999998e-07 2024-09-16T00:08:48.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:48.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618644+0000", 2024-09-16T00:08:48.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7440000000000004e-06 2024-09-16T00:08:48.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:48.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618655+0000", 2024-09-16T00:08:48.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0798999999999999e-05 2024-09-16T00:08:48.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618662+0000", 2024-09-16T00:08:48.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8700000000000003e-06 2024-09-16T00:08:48.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:48.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618717+0000", 2024-09-16T00:08:48.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5223999999999998e-05 2024-09-16T00:08:48.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:48.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618724+0000", 2024-09-16T00:08:48.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.046e-06 2024-09-16T00:08:48.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:48.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618835+0000", 2024-09-16T00:08:48.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111187 2024-09-16T00:08:48.674 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.674 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:48.674 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.674 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.b epoch 22/19 rep_tid 122 entries 22'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:48.674 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:48.674 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.020112775000001, 2024-09-16T00:08:48.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018692890000000001, 2024-09-16T00:08:48.674 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:48.675 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:48.675 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:48.675 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:48.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:48.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:48.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:48.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:48.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:48.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619743+0000", 2024-09-16T00:08:48.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1069999999999999e-06 2024-09-16T00:08:48.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:48.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619754+0000", 2024-09-16T00:08:48.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0789e-05 2024-09-16T00:08:48.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:48.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619777+0000", 2024-09-16T00:08:48.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2844e-05 2024-09-16T00:08:48.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619793+0000", 2024-09-16T00:08:48.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6231000000000001e-05 2024-09-16T00:08:48.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:48.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619875+0000", 2024-09-16T00:08:48.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2286000000000005e-05 2024-09-16T00:08:48.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:48.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621611+0000", 2024-09-16T00:08:48.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001736032 2024-09-16T00:08:48.678 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.678 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:48.678 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.678 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.6 38/37 [PushOp(2:654134d2:::mds0_openfiles.0:head, version: 26'3, data_included: [], data_size: 0, omap_header_size: 42, omap_entries_size: 2, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:654134d2:::mds0_openfiles.0:head@26'3, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:48.678 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.839473+0000", 2024-09-16T00:08:48.678 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.800382015, 2024-09-16T00:08:48.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044119500000000002, 2024-09-16T00:08:48.678 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:48.678 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:48.679 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:48.679 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:48.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:48.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839473+0000", 2024-09-16T00:08:48.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:48.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839473+0000", 2024-09-16T00:08:48.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:48.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:48.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839474+0000", 2024-09-16T00:08:48.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1999999999999998e-07 2024-09-16T00:08:48.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:48.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839512+0000", 2024-09-16T00:08:48.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8546e-05 2024-09-16T00:08:48.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:48.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839530+0000", 2024-09-16T00:08:48.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7663e-05 2024-09-16T00:08:48.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:48.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839550+0000", 2024-09-16T00:08:48.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9817e-05 2024-09-16T00:08:48.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:48.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839658+0000", 2024-09-16T00:08:48.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108213 2024-09-16T00:08:48.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:48.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839672+0000", 2024-09-16T00:08:48.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4518000000000001e-05 2024-09-16T00:08:48.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:48.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:48.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:48.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839914+0000", 2024-09-16T00:08:48.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000241618 2024-09-16T00:08:48.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.682 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:48.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:48.682 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:48.682 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:48.683 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:08:48.724 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:48.724+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:48.799 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:48.799+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:48.829 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:48.829 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:48.829 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:48.830 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:48.839 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:08:48.900 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:48.899+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:48.985 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:48.985 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:48.985 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:48.985 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:48.986 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:48.994 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:08:49.000 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:48.999+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:49.098 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:49.098 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:49.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:49.098 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:49.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.098 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:14 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:49.099 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:49.099 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.481506100000004, 2024-09-16T00:08:49.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029190470000000001, 2024-09-16T00:08:49.099 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:49.099 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:49.099 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:49.099 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:49.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:49.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:49.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:49.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:49.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:49.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615982+0000", 2024-09-16T00:08:49.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7439999999999999e-06 2024-09-16T00:08:49.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:49.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:08:49.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3743999999999999e-05 2024-09-16T00:08:49.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:49.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:49.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2255e-05 2024-09-16T00:08:49.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:49.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616029+0000", 2024-09-16T00:08:49.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1175999999999999e-05 2024-09-16T00:08:49.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:49.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616098+0000", 2024-09-16T00:08:49.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9153999999999996e-05 2024-09-16T00:08:49.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:49.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616132+0000", 2024-09-16T00:08:49.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3924000000000002e-05 2024-09-16T00:08:49.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:49.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:49.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027226189999999999 2024-09-16T00:08:49.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:49.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618899+0000", 2024-09-16T00:08:49.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4431000000000002e-05 2024-09-16T00:08:49.103 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.103 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:49.103 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.103 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:15 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:49.103 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:49.103 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.47927928, 2024-09-16T00:08:49.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095618099999999996, 2024-09-16T00:08:49.103 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:49.103 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:49.103 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:49.103 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:49.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:49.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:49.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:49.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:49.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:49.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618208+0000", 2024-09-16T00:08:49.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1850000000000001e-06 2024-09-16T00:08:49.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:49.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618217+0000", 2024-09-16T00:08:49.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439999999999992e-06 2024-09-16T00:08:49.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:49.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618230+0000", 2024-09-16T00:08:49.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2435999999999999e-05 2024-09-16T00:08:49.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:49.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618238+0000", 2024-09-16T00:08:49.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1319999999999994e-06 2024-09-16T00:08:49.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:49.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618331+0000", 2024-09-16T00:08:49.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3453000000000001e-05 2024-09-16T00:08:49.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:49.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618352+0000", 2024-09-16T00:08:49.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0693999999999999e-05 2024-09-16T00:08:49.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:49.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619112+0000", 2024-09-16T00:08:49.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00076007099999999997 2024-09-16T00:08:49.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:49.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619163+0000", 2024-09-16T00:08:49.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1066000000000002e-05 2024-09-16T00:08:49.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.108 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:49.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.108 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:30 2.15 e22/19 2:aa448500:::500.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:49.108 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:49.108 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.479221795000001, 2024-09-16T00:08:49.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058031250000000001, 2024-09-16T00:08:49.109 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:49.109 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:49.109 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:49.109 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:49.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:49.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:49.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:49.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:49.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:49.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618265+0000", 2024-09-16T00:08:49.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9200000000000001e-07 2024-09-16T00:08:49.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:49.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618271+0000", 2024-09-16T00:08:49.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9580000000000004e-06 2024-09-16T00:08:49.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:49.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618279+0000", 2024-09-16T00:08:49.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2239999999999995e-06 2024-09-16T00:08:49.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:49.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618285+0000", 2024-09-16T00:08:49.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-06 2024-09-16T00:08:49.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:49.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618567+0000", 2024-09-16T00:08:49.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028204400000000001 2024-09-16T00:08:49.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:49.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:08:49.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9678999999999999e-05 2024-09-16T00:08:49.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:49.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624024+0000", 2024-09-16T00:08:49.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054376720000000002 2024-09-16T00:08:49.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:49.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624068+0000", 2024-09-16T00:08:49.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:08:49.114 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.114 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:49.114 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.114 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:20 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:49.114 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:49.114 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.478865431000003, 2024-09-16T00:08:49.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028808739999999998, 2024-09-16T00:08:49.114 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:49.114 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:49.115 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:49.115 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:49.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:49.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:49.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:49.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:49.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:49.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618622+0000", 2024-09-16T00:08:49.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1260000000000001e-06 2024-09-16T00:08:49.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:49.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618631+0000", 2024-09-16T00:08:49.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4700000000000008e-06 2024-09-16T00:08:49.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:49.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618645+0000", 2024-09-16T00:08:49.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3984e-05 2024-09-16T00:08:49.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:49.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:08:49.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2101e-05 2024-09-16T00:08:49.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:49.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618720+0000", 2024-09-16T00:08:49.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2761000000000004e-05 2024-09-16T00:08:49.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:49.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618740+0000", 2024-09-16T00:08:49.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9409e-05 2024-09-16T00:08:49.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:49.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621436+0000", 2024-09-16T00:08:49.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002696211 2024-09-16T00:08:49.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:49.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621502+0000", 2024-09-16T00:08:49.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5811999999999999e-05 2024-09-16T00:08:49.119 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.120 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:49.120 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.120 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:21 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:49.120 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:49.120 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 70.478042142999996, 2024-09-16T00:08:49.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046318540000000004, 2024-09-16T00:08:49.120 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:49.120 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:49.121 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:49.121 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:49.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:49.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:49.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:49.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:49.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:49.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619445+0000", 2024-09-16T00:08:49.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.083e-06 2024-09-16T00:08:49.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:49.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619455+0000", 2024-09-16T00:08:49.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5920000000000007e-06 2024-09-16T00:08:49.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:49.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619467+0000", 2024-09-16T00:08:49.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2711000000000001e-05 2024-09-16T00:08:49.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:49.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619476+0000", 2024-09-16T00:08:49.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0139999999999998e-06 2024-09-16T00:08:49.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:49.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619529+0000", 2024-09-16T00:08:49.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3078999999999998e-05 2024-09-16T00:08:49.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:49.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619548+0000", 2024-09-16T00:08:49.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8952000000000001e-05 2024-09-16T00:08:49.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:49.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624037+0000", 2024-09-16T00:08:49.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044895730000000002 2024-09-16T00:08:49.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:49.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624076+0000", 2024-09-16T00:08:49.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8850000000000002e-05 2024-09-16T00:08:49.125 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.125 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:49.125 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.125 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.14 38/37 [PushOp(2:2f55791f:::606.00000000:head, version: 22'2, data_included: [], data_size: 0, omap_header_size: 274, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(2:2f55791f:::606.00000000:head@22'2, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:49.125 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.838548+0000", 2024-09-16T00:08:49.125 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.258938245, 2024-09-16T00:08:49.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044033700000000001, 2024-09-16T00:08:49.125 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:49.126 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:49.126 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:49.126 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:49.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:49.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838548+0000", 2024-09-16T00:08:49.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:49.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838548+0000", 2024-09-16T00:08:49.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:49.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838549+0000", 2024-09-16T00:08:49.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5600000000000004e-07 2024-09-16T00:08:49.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:49.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838567+0000", 2024-09-16T00:08:49.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7850999999999999e-05 2024-09-16T00:08:49.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:49.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838597+0000", 2024-09-16T00:08:49.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0416000000000002e-05 2024-09-16T00:08:49.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:49.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838619+0000", 2024-09-16T00:08:49.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1274e-05 2024-09-16T00:08:49.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:49.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838706+0000", 2024-09-16T00:08:49.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7755000000000003e-05 2024-09-16T00:08:49.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:49.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838721+0000", 2024-09-16T00:08:49.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4953999999999999e-05 2024-09-16T00:08:49.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:49.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838988+0000", 2024-09-16T00:08:49.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026713099999999999 2024-09-16T00:08:49.129 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.129 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:49.129 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.129 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.14 38/37 [PushOp(2:287034c4:::1.00000000.inode:head, version: 22'4, data_included: [0~582], data_size: 582, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:287034c4:::1.00000000.inode:head@22'4, size: 582, copy_subset: [(0, 582)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 582, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:49.129 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.840824+0000", 2024-09-16T00:08:49.129 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.256662405, 2024-09-16T00:08:49.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00055177600000000002, 2024-09-16T00:08:49.130 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:49.130 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:49.130 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:49.130 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:49.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:49.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840824+0000", 2024-09-16T00:08:49.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:49.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840824+0000", 2024-09-16T00:08:49.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:49.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840825+0000", 2024-09-16T00:08:49.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1100000000000005e-07 2024-09-16T00:08:49.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:49.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840840+0000", 2024-09-16T00:08:49.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5308999999999999e-05 2024-09-16T00:08:49.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:49.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840863+0000", 2024-09-16T00:08:49.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2663999999999998e-05 2024-09-16T00:08:49.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:49.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840884+0000", 2024-09-16T00:08:49.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1798e-05 2024-09-16T00:08:49.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:49.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840980+0000", 2024-09-16T00:08:49.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5299999999999999e-05 2024-09-16T00:08:49.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:49.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840995+0000", 2024-09-16T00:08:49.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4895999999999999e-05 2024-09-16T00:08:49.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:49.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.841376+0000", 2024-09-16T00:08:49.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038099800000000001 2024-09-16T00:08:49.133 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.133 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:49.133 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.133 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.14 38/37 [PushOp(2:2e2fa760:::200.00000001:head, version: 28'9, data_included: [0~5883], data_size: 5883, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:2e2fa760:::200.00000001:head@28'9, size: 5883, copy_subset: [(0, 5883)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5883, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:49.133 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.848988+0000", 2024-09-16T00:08:49.133 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.248498085000001, 2024-09-16T00:08:49.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00037334, 2024-09-16T00:08:49.134 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:49.134 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:49.134 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:49.134 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:49.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:49.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.848988+0000", 2024-09-16T00:08:49.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:49.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.848988+0000", 2024-09-16T00:08:49.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:49.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.848989+0000", 2024-09-16T00:08:49.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5099999999999998e-07 2024-09-16T00:08:49.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:49.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849004+0000", 2024-09-16T00:08:49.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5192e-05 2024-09-16T00:08:49.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:49.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849023+0000", 2024-09-16T00:08:49.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.844e-05 2024-09-16T00:08:49.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:49.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849033+0000", 2024-09-16T00:08:49.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0061000000000001e-05 2024-09-16T00:08:49.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:49.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849096+0000", 2024-09-16T00:08:49.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3627999999999995e-05 2024-09-16T00:08:49.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:49.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849105+0000", 2024-09-16T00:08:49.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5890000000000003e-06 2024-09-16T00:08:49.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:49.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849362+0000", 2024-09-16T00:08:49.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000256579 2024-09-16T00:08:49.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.137 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:49.137 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.137 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.14 38/37 [PushOp(2:292cf221:::200.00000000:head, version: 28'10, data_included: [0~90], data_size: 90, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:292cf221:::200.00000000:head@28'10, size: 90, copy_subset: [(0, 90)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 90, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:49.137 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.851655+0000", 2024-09-16T00:08:49.138 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.245831419999998, 2024-09-16T00:08:49.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036283000000000002, 2024-09-16T00:08:49.138 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:49.138 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:49.138 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:49.138 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:49.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:49.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851655+0000", 2024-09-16T00:08:49.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:49.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851655+0000", 2024-09-16T00:08:49.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:49.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:49.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851656+0000", 2024-09-16T00:08:49.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0399999999999995e-07 2024-09-16T00:08:49.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:49.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851672+0000", 2024-09-16T00:08:49.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6801000000000001e-05 2024-09-16T00:08:49.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:49.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851689+0000", 2024-09-16T00:08:49.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6810999999999999e-05 2024-09-16T00:08:49.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:49.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851699+0000", 2024-09-16T00:08:49.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.995e-06 2024-09-16T00:08:49.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:49.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851767+0000", 2024-09-16T00:08:49.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8194000000000005e-05 2024-09-16T00:08:49.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:49.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851777+0000", 2024-09-16T00:08:49.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2450000000000004e-06 2024-09-16T00:08:49.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:49.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:49.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:49.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.852018+0000", 2024-09-16T00:08:49.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024107999999999999 2024-09-16T00:08:49.141 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.141 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:49.141 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.141 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:49.141 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:49.141 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:49.143 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:08:49.143 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:49.141+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:49.153 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:08:49.244 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:49.244+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:49.248 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:49.248 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:49.248 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:49.248 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:49.258 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:08:49.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.297+0000 7f18030b7640 1 Processor -- start 2024-09-16T00:08:49.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.297+0000 7f18030b7640 1 -- start start 2024-09-16T00:08:49.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.299+0000 7f18030b7640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f17fc0d0490 0x7f17fc160430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:49.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.299+0000 7f18030b7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17fc0cb550 con 0x7f17fc0d0870 2024-09-16T00:08:49.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.299+0000 7f18030b7640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17fc0b61d0 con 0x7f17fc0ccad0 2024-09-16T00:08:49.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.299+0000 7f18030b7640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f17fc0cc6b0 con 0x7f17fc0d0490 2024-09-16T00:08:49.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.299+0000 7f1800e53640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f17fc0d0490 0x7f17fc160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:49.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.299+0000 7f1800e53640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f17fc0d0490 0x7f17fc160430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:38298/0 (socket says 172.21.15.139:38298) 2024-09-16T00:08:49.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.299+0000 7f1800e53640 1 -- 172.21.15.139:0/2527738591 learned_addr learned my addr 172.21.15.139:0/2527738591 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:49.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.299+0000 7f1800e53640 1 -- 172.21.15.139:0/2527738591 >> v1:172.21.15.164:6789/0 conn(0x7f17fc0ccad0 legacy=0x7f17fc169760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:49.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.299+0000 7f1800e53640 1 -- 172.21.15.139:0/2527738591 >> v1:172.21.15.139:6789/0 conn(0x7f17fc0d0870 legacy=0x7f17fc15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:49.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.299+0000 7f1800e53640 1 -- 172.21.15.139:0/2527738591 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc0cb7e0 con 0x7f17fc0d0490 2024-09-16T00:08:49.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.300+0000 7f1800e53640 1 --2- 172.21.15.139:0/2527738591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17fc0d0490 0x7f17fc160430 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0cc830 tx=0x7f17f002a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a2e35cb36e4126d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:49.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.301+0000 7f17fb7fe640 1 -- 172.21.15.139:0/2527738591 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0037070 con 0x7f17fc0d0490 2024-09-16T00:08:49.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.301+0000 7f17fb7fe640 1 -- 172.21.15.139:0/2527738591 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17f0032640 con 0x7f17fc0d0490 2024-09-16T00:08:49.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.301+0000 7f17fb7fe640 1 -- 172.21.15.139:0/2527738591 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f0032a10 con 0x7f17fc0d0490 2024-09-16T00:08:49.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.301+0000 7f18030b7640 1 -- 172.21.15.139:0/2527738591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17fc0d0490 msgr2=0x7f17fc160430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:49.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.301+0000 7f18030b7640 1 --2- 172.21.15.139:0/2527738591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17fc0d0490 0x7f17fc160430 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0cc830 tx=0x7f17f002a2e0 comp rx=0 tx=0).stop 2024-09-16T00:08:49.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.302+0000 7f18030b7640 1 -- 172.21.15.139:0/2527738591 shutdown_connections 2024-09-16T00:08:49.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.302+0000 7f18030b7640 1 --2- 172.21.15.139:0/2527738591 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17fc0d0490 0x7f17fc160430 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:49.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.302+0000 7f18030b7640 1 -- 172.21.15.139:0/2527738591 >> 172.21.15.139:0/2527738591 conn(0x7f17fc06c520 msgr2=0x7f17fc06c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:49.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.302+0000 7f18030b7640 1 -- 172.21.15.139:0/2527738591 shutdown_connections 2024-09-16T00:08:49.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.302+0000 7f18030b7640 1 -- 172.21.15.139:0/2527738591 wait complete. 2024-09-16T00:08:49.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.303+0000 7f18030b7640 1 Processor -- start 2024-09-16T00:08:49.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.304+0000 7f18030b7640 1 -- start start 2024-09-16T00:08:49.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f18030b7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f17fc0ccad0 0x7f17fc15db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:49.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f18030b7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17fc0d0490 0x7f17fc1f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:49.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f18030b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17fc0d0870 0x7f17fc1f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:49.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f18030b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc0cc060 con 0x7f17fc0d0870 2024-09-16T00:08:49.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f18030b7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc0cbd80 con 0x7f17fc0d0490 2024-09-16T00:08:49.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f18030b7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f17fc0cc340 con 0x7f17fc0ccad0 2024-09-16T00:08:49.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17fc0d0490 0x7f17fc1f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:49.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17fc0d0490 0x7f17fc1f49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:38310/0 (socket says 172.21.15.139:38310) 2024-09-16T00:08:49.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f17fbfff640 1 -- 172.21.15.139:0/832556141 learned_addr learned my addr 172.21.15.139:0/832556141 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:49.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f1801654640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17fc0d0870 0x7f17fc1f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:49.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f1800e53640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f17fc0ccad0 0x7f17fc15db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:49.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f17fbfff640 1 -- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f17fc0ccad0 msgr2=0x7f17fc15db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:49.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f17fbfff640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f17fc0ccad0 0x7f17fc15db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:49.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f17fbfff640 1 -- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17fc0d0870 msgr2=0x7f17fc1f4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:49.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f17fbfff640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17fc0d0870 0x7f17fc1f4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:49.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f17fbfff640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc1f7f20 con 0x7f17fc0d0490 2024-09-16T00:08:49.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f1800e53640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f17fc0ccad0 0x7f17fc15db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:08:49.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.305+0000 7f1801654640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17fc0d0870 0x7f17fc1f4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:08:49.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.306+0000 7f17fbfff640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17fc0d0490 0x7f17fc1f49c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f17e800c3f0 tx=0x7f17e800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:49.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.306+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8017070 con 0x7f17fc0d0490 2024-09-16T00:08:49.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.306+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17e80052d0 con 0x7f17fc0d0490 2024-09-16T00:08:49.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.306+0000 7f18030b7640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17fc1f8200 con 0x7f17fc0d0490 2024-09-16T00:08:49.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.306+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8003830 con 0x7f17fc0d0490 2024-09-16T00:08:49.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.306+0000 7f18030b7640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17fc1f87c0 con 0x7f17fc0d0490 2024-09-16T00:08:49.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.307+0000 7f18030b7640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f17fc1f8ba0 con 0x7f17fc0d0490 2024-09-16T00:08:49.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.308+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f17e801a070 con 0x7f17fc0d0490 2024-09-16T00:08:49.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.309+0000 7f17f97fa640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f17e006c2b0 0x7f17e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:49.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.309+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5886+0+0 (secure 0 0 0) 0x7f17e808a950 con 0x7f17fc0d0490 2024-09-16T00:08:49.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.309+0000 7f1800e53640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f17e006c2b0 0x7f17e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:49.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.309+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v52) ==== 54+0+16 (secure 0 0 0) 0x7f17fc1f8ba0 con 0x7f17fc0d0490 2024-09-16T00:08:49.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.309+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f17d4001650 0x7f17d4003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:49.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.309+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17d4006bb0 con 0x7f17d4001650 2024-09-16T00:08:49.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.309+0000 7f1801654640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f17d4001650 0x7f17d4003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:49.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.310+0000 7f1800e53640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f17e006c2b0 0x7f17e006e770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f17f0034000 tx=0x7f17f002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:49.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.310+0000 7f1801654640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f17d4001650 0x7f17d4003b10 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:49.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.311+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17d4006bb0 con 0x7f17d4001650 2024-09-16T00:08:49.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.336+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f17d4005c60 con 0x7f17d4001650 2024-09-16T00:08:49.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.339+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f17d4005c60 con 0x7f17d4001650 2024-09-16T00:08:49.341 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:49.342 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:49.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.339+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f17d4004190 0x7f17d4004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:49.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.339+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f17d4004c00 con 0x7f17d4004190 2024-09-16T00:08:49.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.339+0000 7f1800e53640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f17d4004190 0x7f17d4004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:49.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.340+0000 7f1800e53640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f17d4004190 0x7f17d4004640 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:49.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.340+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17d4004c00 con 0x7f17d4004190 2024-09-16T00:08:49.344 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:49.344+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:49.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.355+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f17d4005140 con 0x7f17d4004190 2024-09-16T00:08:49.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.356+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f17d4005140 con 0x7f17d4004190 2024-09-16T00:08:49.358 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:49.359 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:49.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.357+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f17d4009370 0x7f17d400b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:49.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.357+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f17d400e6c0 con 0x7f17d4009370 2024-09-16T00:08:49.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.357+0000 7f17fbfff640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f17d4009370 0x7f17d400b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:49.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.357+0000 7f17fbfff640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f17d4009370 0x7f17d400b760 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:49.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.358+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17d400e6c0 con 0x7f17d4009370 2024-09-16T00:08:49.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.372+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f17d400d6b0 con 0x7f17d4009370 2024-09-16T00:08:49.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.374+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f17d400d6b0 con 0x7f17d4009370 2024-09-16T00:08:49.376 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:49.377 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:49.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.374+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f17d400e130 0x7f17d400bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:49.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.374+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f17d400c4f0 con 0x7f17d400e130 2024-09-16T00:08:49.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.375+0000 7f1801654640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f17d400e130 0x7f17d400bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:49.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.375+0000 7f1801654640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f17d400e130 0x7f17d400bce0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:49.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.376+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17d400c4f0 con 0x7f17d400e130 2024-09-16T00:08:49.402 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:49.402 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:49.402 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:49.402 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:49.402 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:49.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.403+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f17d400ca60 con 0x7f17d400e130 2024-09-16T00:08:49.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.404+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f17d400ca60 con 0x7f17d400e130 2024-09-16T00:08:49.406 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:49.407 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:49.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.405+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f17d4010880 0x7f17d4012c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:49.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.405+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f17d4015a40 con 0x7f17d4010880 2024-09-16T00:08:49.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.405+0000 7f1800e53640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f17d4010880 0x7f17d4012c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:49.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.405+0000 7f1800e53640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f17d4010880 0x7f17d4012c70 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:49.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.406+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17d4015a40 con 0x7f17d4010880 2024-09-16T00:08:49.410 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:08:49.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.420+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f17d40131b0 con 0x7f17d4010880 2024-09-16T00:08:49.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.422+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f17d40131b0 con 0x7f17d4010880 2024-09-16T00:08:49.424 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:49.424 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:49.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.422+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f17d4013830 0x7f17d4013c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:49.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.422+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f17d4018e30 con 0x7f17d4013830 2024-09-16T00:08:49.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.422+0000 7f17fbfff640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f17d4013830 0x7f17d4013c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:49.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.422+0000 7f17fbfff640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f17d4013830 0x7f17d4013c70 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:49.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.423+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17d4018e30 con 0x7f17d4013830 2024-09-16T00:08:49.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.436+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f17d40142c0 con 0x7f17d4013830 2024-09-16T00:08:49.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.438+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f17d40142c0 con 0x7f17d4013830 2024-09-16T00:08:49.440 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:49.441 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:49.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.438+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f17d40149e0 0x7f17d4014e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:49.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.438+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f17d40154e0 con 0x7f17d40149e0 2024-09-16T00:08:49.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.439+0000 7f1801654640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f17d40149e0 0x7f17d4014e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:49.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.439+0000 7f1801654640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f17d40149e0 0x7f17d4014e40 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:49.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.440+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17d40154e0 con 0x7f17d40149e0 2024-09-16T00:08:49.451 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:49.451+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:49.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.453+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f17d40154e0 con 0x7f17d40149e0 2024-09-16T00:08:49.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.455+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f17d40154e0 con 0x7f17d40149e0 2024-09-16T00:08:49.457 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:49.457 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:49.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.455+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f17d401a1d0 0x7f17d401c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:49.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.455+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f17d401f390 con 0x7f17d401a1d0 2024-09-16T00:08:49.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.455+0000 7f1800e53640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f17d401a1d0 0x7f17d401c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:49.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.456+0000 7f1800e53640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f17d401a1d0 0x7f17d401c5c0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:49.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.456+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17d401f390 con 0x7f17d401a1d0 2024-09-16T00:08:49.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.470+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f17d401cb00 con 0x7f17d401a1d0 2024-09-16T00:08:49.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.471+0000 7f17f97fa640 1 -- 172.21.15.139:0/832556141 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f17d401cb00 con 0x7f17d401a1d0 2024-09-16T00:08:49.473 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:49.475 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:49.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f17d4001650 msgr2=0x7f17d4003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:49.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f17d4001650 0x7f17d4003b10 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:49.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f17d4004190 msgr2=0x7f17d4004640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:49.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f17d4004190 0x7f17d4004640 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:49.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f17d4009370 msgr2=0x7f17d400b760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:49.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f17d4009370 0x7f17d400b760 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:49.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f1801654640 1 -- 172.21.15.139:0/832556141 reap_dead start 2024-09-16T00:08:49.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f17d400e130 msgr2=0x7f17d400bce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:49.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f17d400e130 0x7f17d400bce0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:49.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f17d4010880 msgr2=0x7f17d4012c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:49.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f17d4010880 0x7f17d4012c70 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:49.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f17d4013830 msgr2=0x7f17d4013c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:49.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f17d4013830 0x7f17d4013c70 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:49.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f17d40149e0 msgr2=0x7f17d4014e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:49.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f17d40149e0 0x7f17d4014e40 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:49.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f17d401a1d0 msgr2=0x7f17d401c5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:49.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f17d401a1d0 0x7f17d401c5c0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:49.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f1801654640 1 -- 172.21.15.139:0/832556141 reap_dead start 2024-09-16T00:08:49.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f17e006c2b0 msgr2=0x7f17e006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:49.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f17e006c2b0 0x7f17e006e770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f17f0034000 tx=0x7f17f002e000 comp rx=0 tx=0).stop 2024-09-16T00:08:49.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17fc0d0490 msgr2=0x7f17fc1f49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:49.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17fc0d0490 0x7f17fc1f49c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f17e800c3f0 tx=0x7f17e800c8b0 comp rx=0 tx=0).stop 2024-09-16T00:08:49.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 shutdown_connections 2024-09-16T00:08:49.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f17e006c2b0 0x7f17e006e770 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:49.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 --2- 172.21.15.139:0/832556141 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f17fc0d0490 0x7f17fc1f49c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:49.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 >> 172.21.15.139:0/832556141 conn(0x7f17fc06c520 msgr2=0x7f17fc0ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:49.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 shutdown_connections 2024-09-16T00:08:49.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:49.472+0000 7f17e6ffd640 1 -- 172.21.15.139:0/832556141 wait complete. 2024-09-16T00:08:49.547 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:49.545+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:49.553 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:49.553 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:49.562 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:08:49.648 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:49.647+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:49.667 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:49.667 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:49.675 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:08:49.759 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:49.759+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:49.819 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:49.819 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:49.828 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:08:49.849 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:49.847+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:49.950 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:49.947+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:49.968 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T00:08:49.974 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:49.974 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:49.985 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:08:50.050 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:50.050+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:50.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.092+0000 7ff032850640 1 Processor -- start 2024-09-16T00:08:50.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.093+0000 7ff032850640 1 -- start start 2024-09-16T00:08:50.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.094+0000 7ff032850640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff02c152b20 0x7ff02c0c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:50.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.094+0000 7ff032850640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff02c1587b0 0x7ff02c158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:50.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.094+0000 7ff032850640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff02c0ce310 con 0x7ff02c0c8c50 2024-09-16T00:08:50.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.094+0000 7ff032850640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff02c0c5470 con 0x7ff02c1587b0 2024-09-16T00:08:50.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.094+0000 7ff032850640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7ff02c0cbba0 con 0x7ff02c152b20 2024-09-16T00:08:50.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.094+0000 7ff030ded640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff02c1587b0 0x7ff02c158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:50.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.094+0000 7ff030ded640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff02c1587b0 0x7ff02c158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59250/0 (socket says 172.21.15.139:59250) 2024-09-16T00:08:50.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.094+0000 7ff030ded640 1 -- 172.21.15.139:0/3944866548 learned_addr learned my addr 172.21.15.139:0/3944866548 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:50.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.094+0000 7ff02bfff640 1 --2- 172.21.15.139:0/3944866548 >> v2:172.21.15.164:3300/0 conn(0x7ff02c152b20 0x7ff02c0c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:50.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.095+0000 7ff030ded640 1 -- 172.21.15.139:0/3944866548 >> v2:172.21.15.164:3300/0 conn(0x7ff02c152b20 msgr2=0x7ff02c0c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:50.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.095+0000 7ff030ded640 1 --2- 172.21.15.139:0/3944866548 >> v2:172.21.15.164:3300/0 conn(0x7ff02c152b20 0x7ff02c0c4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:50.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.095+0000 7ff030ded640 1 -- 172.21.15.139:0/3944866548 >> v1:172.21.15.164:6789/0 conn(0x7ff02c0c8c50 legacy=0x7ff02c06ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:50.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.095+0000 7ff030ded640 1 -- 172.21.15.139:0/3944866548 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02c159130 con 0x7ff02c1587b0 2024-09-16T00:08:50.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.096+0000 7ff030ded640 1 --2- 172.21.15.139:0/3944866548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff02c1587b0 0x7ff02c158bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff02c068860 tx=0x7ff02402ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9761353c00597e2a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:50.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.096+0000 7ff02affd640 1 -- 172.21.15.139:0/3944866548 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024036070 con 0x7ff02c1587b0 2024-09-16T00:08:50.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.096+0000 7ff02affd640 1 -- 172.21.15.139:0/3944866548 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff024032640 con 0x7ff02c1587b0 2024-09-16T00:08:50.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.096+0000 7ff02affd640 1 -- 172.21.15.139:0/3944866548 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024032a10 con 0x7ff02c1587b0 2024-09-16T00:08:50.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.097+0000 7ff032850640 1 -- 172.21.15.139:0/3944866548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff02c1587b0 msgr2=0x7ff02c158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:50.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.097+0000 7ff032850640 1 --2- 172.21.15.139:0/3944866548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff02c1587b0 0x7ff02c158bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff02c068860 tx=0x7ff02402ead0 comp rx=0 tx=0).stop 2024-09-16T00:08:50.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.097+0000 7ff032850640 1 -- 172.21.15.139:0/3944866548 shutdown_connections 2024-09-16T00:08:50.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.097+0000 7ff032850640 1 --2- 172.21.15.139:0/3944866548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff02c1587b0 0x7ff02c158bf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:50.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.097+0000 7ff032850640 1 --2- 172.21.15.139:0/3944866548 >> v2:172.21.15.164:3300/0 conn(0x7ff02c152b20 0x7ff02c0c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:50.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.097+0000 7ff032850640 1 -- 172.21.15.139:0/3944866548 >> 172.21.15.139:0/3944866548 conn(0x7ff02c06bd00 msgr2=0x7ff02c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:50.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.097+0000 7ff032850640 1 -- 172.21.15.139:0/3944866548 shutdown_connections 2024-09-16T00:08:50.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.097+0000 7ff032850640 1 -- 172.21.15.139:0/3944866548 wait complete. 2024-09-16T00:08:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.099+0000 7ff032850640 1 Processor -- start 2024-09-16T00:08:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.099+0000 7ff032850640 1 -- start start 2024-09-16T00:08:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.100+0000 7ff032850640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff02c0c8c50 0x7ff02c0b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.100+0000 7ff032850640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff02c152b20 0x7ff02c154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.100+0000 7ff032850640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff02c1587b0 0x7ff02c151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.100+0000 7ff032850640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c0ce5b0 con 0x7ff02c152b20 2024-09-16T00:08:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.100+0000 7ff032850640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c0c3660 con 0x7ff02c0c8c50 2024-09-16T00:08:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.100+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff02c0c8c50 0x7ff02c0b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.100+0000 7ff02bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff02c0c8c50 0x7ff02c0b0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:38328/0 (socket says 172.21.15.139:38328) 2024-09-16T00:08:50.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.100+0000 7ff02b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff02c152b20 0x7ff02c154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.100+0000 7ff02bfff640 1 -- 172.21.15.139:0/3616080450 learned_addr learned my addr 172.21.15.139:0/3616080450 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.100+0000 7ff032850640 1 -- 172.21.15.139:0/3616080450 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff02c0c3940 con 0x7ff02c1587b0 2024-09-16T00:08:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.100+0000 7ff030ded640 1 --2- 172.21.15.139:0/3616080450 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff02c1587b0 0x7ff02c151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.101+0000 7ff02bfff640 1 -- 172.21.15.139:0/3616080450 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff02c1587b0 msgr2=0x7ff02c151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.101+0000 7ff02bfff640 1 --2- 172.21.15.139:0/3616080450 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff02c1587b0 0x7ff02c151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.101+0000 7ff02bfff640 1 -- 172.21.15.139:0/3616080450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff02c152b20 msgr2=0x7ff02c154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.101+0000 7ff02bfff640 1 --2- 172.21.15.139:0/3616080450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff02c152b20 0x7ff02c154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.101+0000 7ff02bfff640 1 -- 172.21.15.139:0/3616080450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02c155ea0 con 0x7ff02c0c8c50 2024-09-16T00:08:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.101+0000 7ff030ded640 1 --2- 172.21.15.139:0/3616080450 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff02c1587b0 0x7ff02c151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:08:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.101+0000 7ff02b7fe640 1 --2- 172.21.15.139:0/3616080450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff02c152b20 0x7ff02c154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:08:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.101+0000 7ff02bfff640 1 --2- 172.21.15.139:0/3616080450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff02c0c8c50 0x7ff02c0b0df0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00e9b0 tx=0x7ff01c00ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.101+0000 7ff028ff9640 1 -- 172.21.15.139:0/3616080450 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c017070 con 0x7ff02c0c8c50 2024-09-16T00:08:50.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.101+0000 7ff028ff9640 1 -- 172.21.15.139:0/3616080450 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff01c00c9a0 con 0x7ff02c0c8c50 2024-09-16T00:08:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.101+0000 7ff032850640 1 -- 172.21.15.139:0/3616080450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02c153020 con 0x7ff02c0c8c50 2024-09-16T00:08:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.101+0000 7ff028ff9640 1 -- 172.21.15.139:0/3616080450 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01c00cd90 con 0x7ff02c0c8c50 2024-09-16T00:08:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.101+0000 7ff032850640 1 -- 172.21.15.139:0/3616080450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff02c1535e0 con 0x7ff02c0c8c50 2024-09-16T00:08:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.102+0000 7ff032850640 1 -- 172.21.15.139:0/3616080450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff02c153ca0 con 0x7ff02c0c8c50 2024-09-16T00:08:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.103+0000 7ff028ff9640 1 -- 172.21.15.139:0/3616080450 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff01c01a070 con 0x7ff02c0c8c50 2024-09-16T00:08:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.103+0000 7ff028ff9640 1 --2- 172.21.15.139:0/3616080450 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff01806c2b0 0x7ff01806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.103+0000 7ff028ff9640 1 -- 172.21.15.139:0/3616080450 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5886+0+0 (secure 0 0 0) 0x7ff01c08b7c0 con 0x7ff02c0c8c50 2024-09-16T00:08:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.103+0000 7ff02b7fe640 1 --2- 172.21.15.139:0/3616080450 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff01806c2b0 0x7ff01806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.104+0000 7ff02b7fe640 1 --2- 172.21.15.139:0/3616080450 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff01806c2b0 0x7ff01806e770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff020006fd0 tx=0x7ff020008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:50.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.108+0000 7ff028ff9640 1 -- 172.21.15.139:0/3616080450 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff02c153ca0 con 0x7ff02c0c8c50 2024-09-16T00:08:50.151 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:50.150+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:50.156 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:50.156 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:50.166 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:08:50.252 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:50.251+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:50.271 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:50.271 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:50.281 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:08:50.352 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:50.350+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:50.386 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:50.386 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:50.398 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:08:50.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.411+0000 7ff032850640 1 -- 172.21.15.139:0/3616080450 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7ff02c0b0960 con 0x7ff02c0c8c50 2024-09-16T00:08:50.453 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:50.453+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:50.517 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:50.518 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:50.527 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:08:50.554 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:50.554+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:50.655 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:50.655+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:50.680 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:50.680 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:50.692 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:08:50.756 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:50.754+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:50.857 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:50.857 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:50.858 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:50.856+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:50.868 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:08:50.958 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:50.957+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:50.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.965+0000 7ff028ff9640 1 -- 172.21.15.139:0/3616080450 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v53) ==== 101+0+0 (secure 0 0 0) 0x7ff01c059f10 con 0x7ff02c0c8c50 2024-09-16T00:08:50.968 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is unset 2024-09-16T00:08:50.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.968+0000 7ff032850640 1 -- 172.21.15.139:0/3616080450 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff01806c2b0 msgr2=0x7ff01806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:50.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.968+0000 7ff032850640 1 --2- 172.21.15.139:0/3616080450 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff01806c2b0 0x7ff01806e770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff020006fd0 tx=0x7ff020008040 comp rx=0 tx=0).stop 2024-09-16T00:08:50.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.969+0000 7ff032850640 1 -- 172.21.15.139:0/3616080450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff02c0c8c50 msgr2=0x7ff02c0b0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:50.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.969+0000 7ff032850640 1 --2- 172.21.15.139:0/3616080450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff02c0c8c50 0x7ff02c0b0df0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00e9b0 tx=0x7ff01c00ee70 comp rx=0 tx=0).stop 2024-09-16T00:08:50.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.969+0000 7ff032850640 1 -- 172.21.15.139:0/3616080450 shutdown_connections 2024-09-16T00:08:50.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.969+0000 7ff032850640 1 --2- 172.21.15.139:0/3616080450 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff01806c2b0 0x7ff01806e770 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:50.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.969+0000 7ff032850640 1 --2- 172.21.15.139:0/3616080450 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff02c1587b0 0x7ff02c151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:50.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.969+0000 7ff032850640 1 --2- 172.21.15.139:0/3616080450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff02c152b20 0x7ff02c154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:50.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.969+0000 7ff032850640 1 --2- 172.21.15.139:0/3616080450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff02c0c8c50 0x7ff02c0b0df0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:50.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.969+0000 7ff032850640 1 -- 172.21.15.139:0/3616080450 >> 172.21.15.139:0/3616080450 conn(0x7ff02c06bd00 msgr2=0x7ff02c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:50.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.969+0000 7ff032850640 1 -- 172.21.15.139:0/3616080450 shutdown_connections 2024-09-16T00:08:50.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:50.969+0000 7ff032850640 1 -- 172.21.15.139:0/3616080450 wait complete. 2024-09-16T00:08:51.059 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:51.057+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:51.065 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:51.065 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:51.077 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:08:51.162 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:51.160+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:51.233 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:51.233 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:51.241 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:08:51.261 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:51.261+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:51.362 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:51.360+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:51.432 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:51.433 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:51.445 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:08:51.486 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:08:51.486 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:51.484+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:51.565 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:51.562+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:51.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.590+0000 7ff33ecf1640 1 Processor -- start 2024-09-16T00:08:51.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.590+0000 7ff33ecf1640 1 -- start start 2024-09-16T00:08:51.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.591+0000 7ff33ecf1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff3380c6790 0x7ff338158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:51.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.591+0000 7ff33ecf1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3380c7030 0x7ff3380c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:51.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.591+0000 7ff33ecf1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff3380c8660 0x7ff338072770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:51.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.591+0000 7ff33ecf1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff338073a20 con 0x7ff3380c8660 2024-09-16T00:08:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.591+0000 7ff33ecf1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7ff3380c8b10 con 0x7ff3380c6790 2024-09-16T00:08:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.591+0000 7ff33ecf1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff3380c7990 con 0x7ff3380c7030 2024-09-16T00:08:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.592+0000 7ff33d28e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff3380c8660 0x7ff338072770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.592+0000 7ff33d28e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff3380c8660 0x7ff338072770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59280/0 (socket says 172.21.15.139:59280) 2024-09-16T00:08:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.592+0000 7ff33d28e640 1 -- 172.21.15.139:0/295903350 learned_addr learned my addr 172.21.15.139:0/295903350 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.592+0000 7ff337fff640 1 --2- 172.21.15.139:0/295903350 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3380c7030 0x7ff3380c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.592+0000 7ff33ca8d640 1 --2- 172.21.15.139:0/295903350 >> v2:172.21.15.164:3300/0 conn(0x7ff3380c6790 0x7ff338158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.592+0000 7ff33d28e640 1 -- 172.21.15.139:0/295903350 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3380c7030 msgr2=0x7ff3380c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.592+0000 7ff33d28e640 1 --2- 172.21.15.139:0/295903350 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3380c7030 0x7ff3380c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.592+0000 7ff33d28e640 1 -- 172.21.15.139:0/295903350 >> v2:172.21.15.164:3300/0 conn(0x7ff3380c6790 msgr2=0x7ff338158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.592+0000 7ff33d28e640 1 --2- 172.21.15.139:0/295903350 >> v2:172.21.15.164:3300/0 conn(0x7ff3380c6790 0x7ff338158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.592+0000 7ff33d28e640 1 -- 172.21.15.139:0/295903350 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3381586b0 con 0x7ff3380c8660 2024-09-16T00:08:51.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.593+0000 7ff33d28e640 1 --2- 172.21.15.139:0/295903350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3380c8660 0x7ff338072770 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff3380b3320 tx=0x7ff33002c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcacd430fa95eff server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:51.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.593+0000 7ff3377fe640 1 -- 172.21.15.139:0/295903350 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33000b2e0 con 0x7ff3380c8660 2024-09-16T00:08:51.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.593+0000 7ff3377fe640 1 -- 172.21.15.139:0/295903350 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff330009600 con 0x7ff3380c8660 2024-09-16T00:08:51.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.593+0000 7ff3377fe640 1 -- 172.21.15.139:0/295903350 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33000b480 con 0x7ff3380c8660 2024-09-16T00:08:51.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.594+0000 7ff33ecf1640 1 -- 172.21.15.139:0/295903350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3380c8660 msgr2=0x7ff338072770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:51.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.594+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/295903350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3380c8660 0x7ff338072770 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff3380b3320 tx=0x7ff33002c8a0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.594+0000 7ff33ecf1640 1 -- 172.21.15.139:0/295903350 shutdown_connections 2024-09-16T00:08:51.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.594+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/295903350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3380c8660 0x7ff338072770 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.594+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/295903350 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3380c7030 0x7ff3380c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.594+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/295903350 >> v2:172.21.15.164:3300/0 conn(0x7ff3380c6790 0x7ff338158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.594+0000 7ff33ecf1640 1 -- 172.21.15.139:0/295903350 >> 172.21.15.139:0/295903350 conn(0x7ff3380c9af0 msgr2=0x7ff3380c9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:51.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.594+0000 7ff33ecf1640 1 -- 172.21.15.139:0/295903350 shutdown_connections 2024-09-16T00:08:51.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.594+0000 7ff33ecf1640 1 -- 172.21.15.139:0/295903350 wait complete. 2024-09-16T00:08:51.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.595+0000 7ff33ecf1640 1 Processor -- start 2024-09-16T00:08:51.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.595+0000 7ff33ecf1640 1 -- start start 2024-09-16T00:08:51.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.596+0000 7ff33ecf1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3380c6790 0x7ff3381f11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:51.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.596+0000 7ff33ecf1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3380c7030 0x7ff338154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:51.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.596+0000 7ff33ecf1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3380c8660 0x7ff338149880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:51.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.596+0000 7ff33ecf1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff3380c7c00 con 0x7ff3380c6790 2024-09-16T00:08:51.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.596+0000 7ff33ecf1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff338073c70 con 0x7ff3380c7030 2024-09-16T00:08:51.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.596+0000 7ff33ecf1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff3380c6b70 con 0x7ff3380c8660 2024-09-16T00:08:51.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.596+0000 7ff33ca8d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3380c6790 0x7ff3381f11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:51.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.596+0000 7ff33ca8d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3380c6790 0x7ff3381f11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59288/0 (socket says 172.21.15.139:59288) 2024-09-16T00:08:51.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.596+0000 7ff33ca8d640 1 -- 172.21.15.139:0/3348542191 learned_addr learned my addr 172.21.15.139:0/3348542191 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:51.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.596+0000 7ff337fff640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3380c7030 0x7ff338154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:51.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.596+0000 7ff33d28e640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3380c8660 0x7ff338149880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:51.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.597+0000 7ff33ca8d640 1 -- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3380c8660 msgr2=0x7ff338149880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:51.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.597+0000 7ff33ca8d640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3380c8660 0x7ff338149880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.597+0000 7ff33ca8d640 1 -- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3380c7030 msgr2=0x7ff338154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:51.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.597+0000 7ff33ca8d640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3380c7030 0x7ff338154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.597+0000 7ff33ca8d640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3381f1710 con 0x7ff3380c6790 2024-09-16T00:08:51.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.597+0000 7ff33ca8d640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3380c6790 0x7ff3381f11d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00d830 tx=0x7ff32c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:51.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.597+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c018030 con 0x7ff3380c6790 2024-09-16T00:08:51.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.597+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff32c00be10 con 0x7ff3380c6790 2024-09-16T00:08:51.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.597+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3381533a0 con 0x7ff3380c6790 2024-09-16T00:08:51.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.597+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0098b0 con 0x7ff3380c6790 2024-09-16T00:08:51.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.597+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff338153960 con 0x7ff3380c6790 2024-09-16T00:08:51.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.598+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7ff3380b2550 con 0x7ff3380c6790 2024-09-16T00:08:51.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.599+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff32c01b070 con 0x7ff3380c6790 2024-09-16T00:08:51.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.599+0000 7ff3357fa640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff31c06c2b0 0x7ff31c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:51.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.599+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5886+0+0 (secure 0 0 0) 0x7ff32c08b5f0 con 0x7ff3380c6790 2024-09-16T00:08:51.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.599+0000 7ff337fff640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff31c06c2b0 0x7ff31c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:51.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.599+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v53) ==== 54+0+16 (secure 0 0 0) 0x7ff32c091170 con 0x7ff3380c6790 2024-09-16T00:08:51.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.600+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7ff338151470 0x7ff33814f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:51.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.600+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff338153150 con 0x7ff338151470 2024-09-16T00:08:51.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.600+0000 7ff337fff640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff31c06c2b0 0x7ff31c06e770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff3380692b0 tx=0x7ff324008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:51.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.600+0000 7ff33d28e640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7ff338151470 0x7ff33814f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:51.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.600+0000 7ff33d28e640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7ff338151470 0x7ff33814f320 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:51.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.601+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff338153150 con 0x7ff338151470 2024-09-16T00:08:51.606 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:51.606 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:51.613 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:08:51.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.629+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff33814fec0 con 0x7ff338151470 2024-09-16T00:08:51.633 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:51.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.630+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff33814fec0 con 0x7ff338151470 2024-09-16T00:08:51.633 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:51.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.630+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7ff338149dc0 0x7ff3381f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:51.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.630+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7ff338150be0 con 0x7ff338149dc0 2024-09-16T00:08:51.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.631+0000 7ff33ca8d640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7ff338149dc0 0x7ff3381f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:51.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.631+0000 7ff33ca8d640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7ff338149dc0 0x7ff3381f3ac0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:51.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.632+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff338150be0 con 0x7ff338149dc0 2024-09-16T00:08:51.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.647+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff338150900 con 0x7ff338149dc0 2024-09-16T00:08:51.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.650+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff338150900 con 0x7ff338149dc0 2024-09-16T00:08:51.652 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.650+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7ff338062630 0x7ff338062a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.650+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7ff338150620 con 0x7ff338062630 2024-09-16T00:08:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.650+0000 7ff337fff640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7ff338062630 0x7ff338062a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.650+0000 7ff337fff640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7ff338062630 0x7ff338062a70 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:51.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.651+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff338150620 con 0x7ff338062630 2024-09-16T00:08:51.665 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:51.664+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:51.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.665+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff3380c5b30 con 0x7ff338062630 2024-09-16T00:08:51.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.667+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff3380c5b30 con 0x7ff338062630 2024-09-16T00:08:51.670 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:51.671 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:51.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.667+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7ff338063370 0x7ff33802cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:51.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.667+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7ff33814e5a0 con 0x7ff338063370 2024-09-16T00:08:51.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.668+0000 7ff33d28e640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7ff338063370 0x7ff33802cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:51.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.668+0000 7ff33d28e640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7ff338063370 0x7ff33802cb50 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:51.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.669+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff33814e5a0 con 0x7ff338063370 2024-09-16T00:08:51.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.683+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff33802d360 con 0x7ff338063370 2024-09-16T00:08:51.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.685+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff33802d360 con 0x7ff338063370 2024-09-16T00:08:51.687 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:51.688 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:51.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.686+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7ff33814ce90 0x7ff33814d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:51.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.686+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7ff33802da80 con 0x7ff33814ce90 2024-09-16T00:08:51.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.686+0000 7ff33ca8d640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7ff33814ce90 0x7ff33814d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:51.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.686+0000 7ff33ca8d640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7ff33814ce90 0x7ff33814d2d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:51.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.687+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff33802da80 con 0x7ff33814ce90 2024-09-16T00:08:51.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.713+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff33802da80 con 0x7ff33814ce90 2024-09-16T00:08:51.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.715+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff33802da80 con 0x7ff33814ce90 2024-09-16T00:08:51.719 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:51.721 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:51.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.716+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7ff33814dd20 0x7ff33814e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:51.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.716+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7ff338209730 con 0x7ff33814dd20 2024-09-16T00:08:51.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.716+0000 7ff337fff640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7ff33814dd20 0x7ff33814e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:51.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.716+0000 7ff337fff640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7ff33814dd20 0x7ff33814e1b0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:51.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.717+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff338209730 con 0x7ff33814dd20 2024-09-16T00:08:51.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.731+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff338206ea0 con 0x7ff33814dd20 2024-09-16T00:08:51.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.733+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff338206ea0 con 0x7ff33814dd20 2024-09-16T00:08:51.735 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:51.736 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:51.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.733+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7ff3382075c0 0x7ff3382079e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:51.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.733+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7ff33820cb00 con 0x7ff3382075c0 2024-09-16T00:08:51.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.734+0000 7ff33d28e640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7ff3382075c0 0x7ff3382079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:51.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.734+0000 7ff33d28e640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7ff3382075c0 0x7ff3382079e0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:51.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.735+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff33820cb00 con 0x7ff3382075c0 2024-09-16T00:08:51.739 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:51.739 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:51.739 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:51.739 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:51.739 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:51.749 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:08:51.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.749+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff338207fe0 con 0x7ff3382075c0 2024-09-16T00:08:51.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.751+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff338207fe0 con 0x7ff3382075c0 2024-09-16T00:08:51.753 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:51.754 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:51.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.751+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7ff338208660 0x7ff338208ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:51.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.751+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7ff338209160 con 0x7ff338208660 2024-09-16T00:08:51.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.751+0000 7ff33ca8d640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7ff338208660 0x7ff338208ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:51.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.752+0000 7ff33ca8d640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7ff338208660 0x7ff338208ac0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:51.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.752+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff338209160 con 0x7ff338208660 2024-09-16T00:08:51.766 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:51.765+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:51.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.767+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff3380755b0 con 0x7ff338208660 2024-09-16T00:08:51.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff3357fa640 1 -- 172.21.15.139:0/3348542191 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff3380755b0 con 0x7ff338208660 2024-09-16T00:08:51.771 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:51.773 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:51.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7ff338151470 msgr2=0x7ff33814f320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:51.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7ff338151470 0x7ff33814f320 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7ff338149dc0 msgr2=0x7ff3381f3ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:51.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7ff338149dc0 0x7ff3381f3ac0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7ff338062630 msgr2=0x7ff338062a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:51.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7ff338062630 0x7ff338062a70 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7ff338063370 msgr2=0x7ff33802cb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:51.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7ff338063370 0x7ff33802cb50 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7ff33814ce90 msgr2=0x7ff33814d2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:51.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7ff33814ce90 0x7ff33814d2d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33d28e640 1 -- 172.21.15.139:0/3348542191 reap_dead start 2024-09-16T00:08:51.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7ff33814dd20 msgr2=0x7ff33814e1b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:51.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7ff33814dd20 0x7ff33814e1b0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33d28e640 1 -- 172.21.15.139:0/3348542191 reap_dead start 2024-09-16T00:08:51.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7ff3382075c0 msgr2=0x7ff3382079e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:51.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7ff3382075c0 0x7ff3382079e0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7ff338208660 msgr2=0x7ff338208ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:51.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7ff338208660 0x7ff338208ac0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33d28e640 1 -- 172.21.15.139:0/3348542191 reap_dead start 2024-09-16T00:08:51.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff31c06c2b0 msgr2=0x7ff31c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:51.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff31c06c2b0 0x7ff31c06e770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff3380692b0 tx=0x7ff324008040 comp rx=0 tx=0).stop 2024-09-16T00:08:51.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3380c6790 msgr2=0x7ff3381f11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:51.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3380c6790 0x7ff3381f11d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00d830 tx=0x7ff32c00dcf0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 shutdown_connections 2024-09-16T00:08:51.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff31c06c2b0 0x7ff31c06e770 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 --2- 172.21.15.139:0/3348542191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3380c6790 0x7ff3381f11d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:51.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 >> 172.21.15.139:0/3348542191 conn(0x7ff3380c9af0 msgr2=0x7ff338072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:51.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 shutdown_connections 2024-09-16T00:08:51.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:51.770+0000 7ff33ecf1640 1 -- 172.21.15.139:0/3348542191 wait complete. 2024-09-16T00:08:51.862 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:51.863 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:51.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:51.863 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:51.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.864 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:22 2.e e22/19 2:717a0223:::608.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:51.864 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:51.865 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 73.242838523000003, 2024-09-16T00:08:51.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042195870000000003, 2024-09-16T00:08:51.865 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.865 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.865 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:51.865 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:51.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:08:51.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.868 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616426+0000", 2024-09-16T00:08:51.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.037e-06 2024-09-16T00:08:51.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616435+0000", 2024-09-16T00:08:51.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6440000000000006e-06 2024-09-16T00:08:51.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616447+0000", 2024-09-16T00:08:51.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2466000000000001e-05 2024-09-16T00:08:51.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616456+0000", 2024-09-16T00:08:51.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8799999999999997e-06 2024-09-16T00:08:51.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616517+0000", 2024-09-16T00:08:51.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1171000000000001e-05 2024-09-16T00:08:51.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.872 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:51.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616540+0000", 2024-09-16T00:08:51.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2991000000000001e-05 2024-09-16T00:08:51.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:51.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620599+0000", 2024-09-16T00:08:51.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040586140000000003 2024-09-16T00:08:51.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620645+0000", 2024-09-16T00:08:51.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5784000000000001e-05 2024-09-16T00:08:51.876 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.876 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.876 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.877 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:51.877 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:51.878 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 73.242748793000004, 2024-09-16T00:08:51.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044836249999999998, 2024-09-16T00:08:51.878 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.878 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.878 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:51.878 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:51.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:08:51.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616518+0000", 2024-09-16T00:08:51.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3699999999999999e-06 2024-09-16T00:08:51.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:08:51.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1479999999999997e-06 2024-09-16T00:08:51.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616534+0000", 2024-09-16T00:08:51.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0641e-05 2024-09-16T00:08:51.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616541+0000", 2024-09-16T00:08:51.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3339999999999996e-06 2024-09-16T00:08:51.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617318+0000", 2024-09-16T00:08:51.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00077694799999999996 2024-09-16T00:08:51.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:51.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617409+0000", 2024-09-16T00:08:51.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0740000000000002e-05 2024-09-16T00:08:51.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620998+0000", 2024-09-16T00:08:51.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035894439999999998 2024-09-16T00:08:51.883 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.883 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.883 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.883 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:51.884 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:51.884 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 73.242700889999995, 2024-09-16T00:08:51.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041597320000000002, 2024-09-16T00:08:51.884 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.884 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.884 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:51.884 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.884 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:51.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:51.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:08:51.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:08:51.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616566+0000", 2024-09-16T00:08:51.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6979999999999998e-06 2024-09-16T00:08:51.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616574+0000", 2024-09-16T00:08:51.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7759999999999997e-06 2024-09-16T00:08:51.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616580+0000", 2024-09-16T00:08:51.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:51.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616800+0000", 2024-09-16T00:08:51.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022016700000000001 2024-09-16T00:08:51.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:51.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616904+0000", 2024-09-16T00:08:51.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104579 2024-09-16T00:08:51.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620722+0000", 2024-09-16T00:08:51.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003818133 2024-09-16T00:08:51.887 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.887 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.887 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.887 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:51.888 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:51.888 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 73.242450204999997, 2024-09-16T00:08:51.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038451570000000001, 2024-09-16T00:08:51.888 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.888 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.888 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:51.888 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:51.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:08:51.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616814+0000", 2024-09-16T00:08:51.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9299999999999997e-07 2024-09-16T00:08:51.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616824+0000", 2024-09-16T00:08:51.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6600000000000007e-06 2024-09-16T00:08:51.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616835+0000", 2024-09-16T00:08:51.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0713000000000001e-05 2024-09-16T00:08:51.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616841+0000", 2024-09-16T00:08:51.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.934e-06 2024-09-16T00:08:51.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616948+0000", 2024-09-16T00:08:51.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010631100000000001 2024-09-16T00:08:51.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:51.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616967+0000", 2024-09-16T00:08:51.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9598000000000001e-05 2024-09-16T00:08:51.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:51.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620619+0000", 2024-09-16T00:08:51.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036515810000000001 2024-09-16T00:08:51.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620659+0000", 2024-09-16T00:08:51.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9666999999999998e-05 2024-09-16T00:08:51.892 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.892 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.892 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.892 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:51.892 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:51.892 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 73.242403253000006, 2024-09-16T00:08:51.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038240129999999998, 2024-09-16T00:08:51.892 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.892 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.892 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:51.893 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:51.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:08:51.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616861+0000", 2024-09-16T00:08:51.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0200000000000003e-07 2024-09-16T00:08:51.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616864+0000", 2024-09-16T00:08:51.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5090000000000002e-06 2024-09-16T00:08:51.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616871+0000", 2024-09-16T00:08:51.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9249999999999998e-06 2024-09-16T00:08:51.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616877+0000", 2024-09-16T00:08:51.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.981e-06 2024-09-16T00:08:51.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617168+0000", 2024-09-16T00:08:51.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029035699999999998 2024-09-16T00:08:51.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:51.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617184+0000", 2024-09-16T00:08:51.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6005999999999999e-05 2024-09-16T00:08:51.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:51.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620661+0000", 2024-09-16T00:08:51.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034775330000000001 2024-09-16T00:08:51.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.896 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620684+0000", 2024-09-16T00:08:51.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3300000000000001e-05 2024-09-16T00:08:51.896 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.896 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.896 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.896 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:51.896 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:51.897 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 73.241947809999999, 2024-09-16T00:08:51.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045415610000000004, 2024-09-16T00:08:51.897 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.897 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.897 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:51.897 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.897 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:51.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.897 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:51.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:08:51.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8800000000000002e-07 2024-09-16T00:08:51.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617321+0000", 2024-09-16T00:08:51.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6979999999999997e-06 2024-09-16T00:08:51.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617330+0000", 2024-09-16T00:08:51.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6729999999999998e-06 2024-09-16T00:08:51.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617336+0000", 2024-09-16T00:08:51.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2110000000000003e-06 2024-09-16T00:08:51.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617394+0000", 2024-09-16T00:08:51.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8087999999999997e-05 2024-09-16T00:08:51.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:51.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617412+0000", 2024-09-16T00:08:51.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8099999999999999e-05 2024-09-16T00:08:51.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:51.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621818+0000", 2024-09-16T00:08:51.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044061209999999998 2024-09-16T00:08:51.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621857+0000", 2024-09-16T00:08:51.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9082000000000003e-05 2024-09-16T00:08:51.900 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.900 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.901 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.901 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:23 2.e e22/19 2:717a0223:::608.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:51.901 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:51.901 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 73.241051788999997, 2024-09-16T00:08:51.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036198279999999998, 2024-09-16T00:08:51.901 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.901 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.901 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:51.901 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:51.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:08:51.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618213+0000", 2024-09-16T00:08:51.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:08:51.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618218+0000", 2024-09-16T00:08:51.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4099999999999999e-06 2024-09-16T00:08:51.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618227+0000", 2024-09-16T00:08:51.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5009999999999993e-06 2024-09-16T00:08:51.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618234+0000", 2024-09-16T00:08:51.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:51.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618307+0000", 2024-09-16T00:08:51.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3421000000000001e-05 2024-09-16T00:08:51.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:51.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618328+0000", 2024-09-16T00:08:51.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0919000000000001e-05 2024-09-16T00:08:51.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:51.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621804+0000", 2024-09-16T00:08:51.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034756560000000001 2024-09-16T00:08:51.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621832+0000", 2024-09-16T00:08:51.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7966999999999998e-05 2024-09-16T00:08:51.905 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.905 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.905 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.905 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing_reply(2.1e epoch 22/19 rep_tid 0 lcod 22'1)", 2024-09-16T00:08:51.905 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:51.905 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 73.240606620999998, 2024-09-16T00:08:51.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1371000000000003e-05, 2024-09-16T00:08:51.905 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.906 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.906 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:51.906 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:51.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618657+0000", 2024-09-16T00:08:51.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:08:51.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:08:51.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618662+0000", 2024-09-16T00:08:51.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1189999999999999e-06 2024-09-16T00:08:51.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618674+0000", 2024-09-16T00:08:51.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2317000000000001e-05 2024-09-16T00:08:51.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618681+0000", 2024-09-16T00:08:51.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7170000000000002e-06 2024-09-16T00:08:51.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:08:51.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6745000000000001e-05 2024-09-16T00:08:51.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618748+0000", 2024-09-16T00:08:51.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0794000000000001e-05 2024-09-16T00:08:51.909 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.909 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.909 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.909 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.909 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:51.909 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:51.909 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 73.240093286999993, 2024-09-16T00:08:51.909 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027183849999999998, 2024-09-16T00:08:51.909 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.909 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.909 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:51.909 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.909 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.910 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.910 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:51.910 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.910 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.910 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619170+0000", 2024-09-16T00:08:51.910 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.910 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.910 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619171+0000", 2024-09-16T00:08:51.910 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8999999999999995e-07 2024-09-16T00:08:51.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619179+0000", 2024-09-16T00:08:51.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2679999999999994e-06 2024-09-16T00:08:51.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:08:51.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1246e-05 2024-09-16T00:08:51.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619197+0000", 2024-09-16T00:08:51.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2820000000000002e-06 2024-09-16T00:08:51.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619308+0000", 2024-09-16T00:08:51.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011141000000000001 2024-09-16T00:08:51.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:51.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619325+0000", 2024-09-16T00:08:51.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6469000000000001e-05 2024-09-16T00:08:51.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:51.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621860+0000", 2024-09-16T00:08:51.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025353590000000001 2024-09-16T00:08:51.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621889+0000", 2024-09-16T00:08:51.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8561e-05 2024-09-16T00:08:51.915 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.915 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.915 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.915 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 6 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:51.915 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:51.915 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 73.240050588000003, 2024-09-16T00:08:51.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026903809999999999, 2024-09-16T00:08:51.915 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.915 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.915 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:51.915 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:51.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:51.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619213+0000", 2024-09-16T00:08:51.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2000000000000001e-07 2024-09-16T00:08:51.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619219+0000", 2024-09-16T00:08:51.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:08:51.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619234+0000", 2024-09-16T00:08:51.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5075000000000001e-05 2024-09-16T00:08:51.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619251+0000", 2024-09-16T00:08:51.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6863000000000001e-05 2024-09-16T00:08:51.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619556+0000", 2024-09-16T00:08:51.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030551599999999998 2024-09-16T00:08:51.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621903+0000", 2024-09-16T00:08:51.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023469979999999999 2024-09-16T00:08:51.918 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.918 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.919 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.919 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "pg_update_log_missing_reply(2.1b epoch 22/19 rep_tid 1 lcod 22'1)", 2024-09-16T00:08:51.919 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:51.919 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 73.238972192000006, 2024-09-16T00:08:51.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6989000000000003e-05, 2024-09-16T00:08:51.919 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.919 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.919 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:51.919 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:51.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620291+0000", 2024-09-16T00:08:51.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620292+0000", 2024-09-16T00:08:51.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.37e-07 2024-09-16T00:08:51.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620296+0000", 2024-09-16T00:08:51.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2420000000000002e-06 2024-09-16T00:08:51.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.921 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620308+0000", 2024-09-16T00:08:51.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1623e-05 2024-09-16T00:08:51.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.921 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620313+0000", 2024-09-16T00:08:51.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6290000000000001e-06 2024-09-16T00:08:51.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.922 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.922 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620359+0000", 2024-09-16T00:08:51.922 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6201e-05 2024-09-16T00:08:51.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.922 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.922 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620378+0000", 2024-09-16T00:08:51.922 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9556999999999999e-05 2024-09-16T00:08:51.922 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.922 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.922 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.922 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:38 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'6, mlcod=22'4)", 2024-09-16T00:08:51.923 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:51.923 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 73.236529942999994, 2024-09-16T00:08:51.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064551599999999995, 2024-09-16T00:08:51.923 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.923 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.923 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:51.923 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.923 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.923 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:51.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.923 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:51.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622734+0000", 2024-09-16T00:08:51.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4399999999999999e-07 2024-09-16T00:08:51.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622739+0000", 2024-09-16T00:08:51.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0250000000000002e-06 2024-09-16T00:08:51.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622749+0000", 2024-09-16T00:08:51.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1220000000000001e-06 2024-09-16T00:08:51.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622760+0000", 2024-09-16T00:08:51.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1640999999999999e-05 2024-09-16T00:08:51.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622799+0000", 2024-09-16T00:08:51.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8537000000000001e-05 2024-09-16T00:08:51.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:51.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622810+0000", 2024-09-16T00:08:51.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1562e-05 2024-09-16T00:08:51.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:51.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.623356+0000", 2024-09-16T00:08:51.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054522299999999995 2024-09-16T00:08:51.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.623379+0000", 2024-09-16T00:08:51.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3662000000000001e-05 2024-09-16T00:08:51.927 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.927 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.927 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.927 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:40 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'7, mlcod=22'5)", 2024-09-16T00:08:51.927 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:51.927 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 61.019509040999999, 2024-09-16T00:08:51.927 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001017773, 2024-09-16T00:08:51.927 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.927 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.927 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:51.927 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.927 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:51.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839755+0000", 2024-09-16T00:08:51.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839756+0000", 2024-09-16T00:08:51.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.491e-06 2024-09-16T00:08:51.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839768+0000", 2024-09-16T00:08:51.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1649e-05 2024-09-16T00:08:51.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839783+0000", 2024-09-16T00:08:51.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5023e-05 2024-09-16T00:08:51.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839791+0000", 2024-09-16T00:08:51.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5620000000000004e-06 2024-09-16T00:08:51.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839848+0000", 2024-09-16T00:08:51.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6563e-05 2024-09-16T00:08:51.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:51.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.839868+0000", 2024-09-16T00:08:51.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9652000000000001e-05 2024-09-16T00:08:51.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:51.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.840729+0000", 2024-09-16T00:08:51.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086159500000000005 2024-09-16T00:08:51.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:50.840772+0000", 2024-09-16T00:08:51.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3238000000000002e-05 2024-09-16T00:08:51.931 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.931 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.931 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.931 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:41 2.14 e22/19 2:2e2fa760:::200.00000001:head v 22'8, mlcod=22'6)", 2024-09-16T00:08:51.931 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:51.931 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.745852130000003, 2024-09-16T00:08:51.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001040568, 2024-09-16T00:08:51.932 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.932 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.932 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:51.932 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:51.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113412+0000", 2024-09-16T00:08:51.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113413+0000", 2024-09-16T00:08:51.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7409999999999999e-06 2024-09-16T00:08:51.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113428+0000", 2024-09-16T00:08:51.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5007999999999999e-05 2024-09-16T00:08:51.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113447+0000", 2024-09-16T00:08:51.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8731000000000001e-05 2024-09-16T00:08:51.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.934 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113457+0000", 2024-09-16T00:08:51.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0095000000000001e-05 2024-09-16T00:08:51.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.934 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113526+0000", 2024-09-16T00:08:51.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8552000000000004e-05 2024-09-16T00:08:51.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.934 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:51.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.113552+0000", 2024-09-16T00:08:51.935 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6795999999999999e-05 2024-09-16T00:08:51.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:51.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.114417+0000", 2024-09-16T00:08:51.935 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086476299999999995 2024-09-16T00:08:51.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:53.114452+0000", 2024-09-16T00:08:51.935 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4882000000000002e-05 2024-09-16T00:08:51.935 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.935 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.935 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.936 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.14 38/37 [PushReplyOp(2:2f55791f:::606.00000000:head)])", 2024-09-16T00:08:51.936 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.839962+0000", 2024-09-16T00:08:51.936 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.019301709, 2024-09-16T00:08:51.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030257, 2024-09-16T00:08:51.936 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.936 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.936 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:51.936 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.936 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.936 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839962+0000", 2024-09-16T00:08:51.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839962+0000", 2024-09-16T00:08:51.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839962+0000", 2024-09-16T00:08:51.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5400000000000001e-07 2024-09-16T00:08:51.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839967+0000", 2024-09-16T00:08:51.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.245e-06 2024-09-16T00:08:51.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839978+0000", 2024-09-16T00:08:51.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1388e-05 2024-09-16T00:08:51.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.839987+0000", 2024-09-16T00:08:51.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1500000000000005e-06 2024-09-16T00:08:51.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.840043+0000", 2024-09-16T00:08:51.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5396000000000003e-05 2024-09-16T00:08:51.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.840265+0000", 2024-09-16T00:08:51.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022183700000000001 2024-09-16T00:08:51.939 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.939 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.939 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.939 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.14 38/37 [PushReplyOp(2:287034c4:::1.00000000.inode:head)])", 2024-09-16T00:08:51.939 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.848174+0000", 2024-09-16T00:08:51.940 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.011089503000001, 2024-09-16T00:08:51.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029495900000000002, 2024-09-16T00:08:51.940 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.940 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.940 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:51.940 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848174+0000", 2024-09-16T00:08:51.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848174+0000", 2024-09-16T00:08:51.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848175+0000", 2024-09-16T00:08:51.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.68e-07 2024-09-16T00:08:51.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848179+0000", 2024-09-16T00:08:51.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1400000000000002e-06 2024-09-16T00:08:51.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848191+0000", 2024-09-16T00:08:51.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1823e-05 2024-09-16T00:08:51.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848196+0000", 2024-09-16T00:08:51.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.626e-06 2024-09-16T00:08:51.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848251+0000", 2024-09-16T00:08:51.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4342999999999998e-05 2024-09-16T00:08:51.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.943 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.848469+0000", 2024-09-16T00:08:51.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000218359 2024-09-16T00:08:51.943 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.943 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.943 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.943 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.14 38/37 [PushReplyOp(2:2e2fa760:::200.00000001:head)])", 2024-09-16T00:08:51.943 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.850960+0000", 2024-09-16T00:08:51.943 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.008303212000001, 2024-09-16T00:08:51.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00028471600000000002, 2024-09-16T00:08:51.943 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.943 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.943 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:51.944 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850960+0000", 2024-09-16T00:08:51.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850960+0000", 2024-09-16T00:08:51.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850961+0000", 2024-09-16T00:08:51.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6000000000000003e-07 2024-09-16T00:08:51.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850965+0000", 2024-09-16T00:08:51.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1400000000000002e-06 2024-09-16T00:08:51.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850977+0000", 2024-09-16T00:08:51.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2128e-05 2024-09-16T00:08:51.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.850985+0000", 2024-09-16T00:08:51.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3220000000000004e-06 2024-09-16T00:08:51.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.851036+0000", 2024-09-16T00:08:51.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1100000000000002e-05 2024-09-16T00:08:51.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.851245+0000", 2024-09-16T00:08:51.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000209266 2024-09-16T00:08:51.946 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.946 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.946 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.947 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "MOSDPGPushReply(2.14 38/37 [PushReplyOp(2:292cf221:::200.00000000:head)])", 2024-09-16T00:08:51.947 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:24.853964+0000", 2024-09-16T00:08:51.947 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.005300024, 2024-09-16T00:08:51.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00026211799999999999, 2024-09-16T00:08:51.947 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:51.947 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:51.947 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:08:51.947 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:51.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.947 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:51.947 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853964+0000", 2024-09-16T00:08:51.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.948 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:51.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853964+0000", 2024-09-16T00:08:51.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:51.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.948 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:51.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853964+0000", 2024-09-16T00:08:51.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8000000000000002e-07 2024-09-16T00:08:51.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.948 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:51.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853967+0000", 2024-09-16T00:08:51.949 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6529999999999998e-06 2024-09-16T00:08:51.949 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.949 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:51.949 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853973+0000", 2024-09-16T00:08:51.949 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9110000000000004e-06 2024-09-16T00:08:51.949 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.949 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:51.949 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.853981+0000", 2024-09-16T00:08:51.949 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.25e-06 2024-09-16T00:08:51.949 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.949 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:51.949 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.854030+0000", 2024-09-16T00:08:51.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8909000000000002e-05 2024-09-16T00:08:51.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:51.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:51.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:51.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:24.854226+0000", 2024-09-16T00:08:51.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019611499999999999 2024-09-16T00:08:51.950 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.950 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.950 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.950 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:51.950 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:51.950 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:51.952 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:08:51.953 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:51.951+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:51.973 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T00:08:51.974 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T00:08:52.053 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:52.051+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:52.069 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:52.069 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:52.069 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:52.069 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:52.079 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:08:52.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.078+0000 7f6ba6e82640 1 Processor -- start 2024-09-16T00:08:52.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.078+0000 7f6ba6e82640 1 -- start start 2024-09-16T00:08:52.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.078+0000 7f6ba6e82640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6ba00ce430 0x7f6ba00b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:52.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.078+0000 7f6ba6e82640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6ba00c9430 0x7f6ba015bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:52.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.078+0000 7f6ba6e82640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba00703e0 con 0x7f6ba00ce810 2024-09-16T00:08:52.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.079+0000 7f6ba6e82640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba00c5750 con 0x7f6ba00c9430 2024-09-16T00:08:52.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.079+0000 7f6ba6e82640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba0070680 con 0x7f6ba00ce430 2024-09-16T00:08:52.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.079+0000 7f6ba541f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6ba00c9430 0x7f6ba015bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:52.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.079+0000 7f6ba541f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6ba00c9430 0x7f6ba015bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59300/0 (socket says 172.21.15.139:59300) 2024-09-16T00:08:52.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.079+0000 7f6ba541f640 1 -- 172.21.15.139:0/442443434 learned_addr learned my addr 172.21.15.139:0/442443434 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:52.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.079+0000 7f6ba4c1e640 1 --2- 172.21.15.139:0/442443434 >> v2:172.21.15.164:3300/0 conn(0x7f6ba00ce430 0x7f6ba00b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:52.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.079+0000 7f6ba541f640 1 -- 172.21.15.139:0/442443434 >> v2:172.21.15.164:3300/0 conn(0x7f6ba00ce430 msgr2=0x7f6ba00b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:52.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.079+0000 7f6ba541f640 1 --2- 172.21.15.139:0/442443434 >> v2:172.21.15.164:3300/0 conn(0x7f6ba00ce430 0x7f6ba00b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:52.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.079+0000 7f6ba541f640 1 -- 172.21.15.139:0/442443434 >> v1:172.21.15.164:6789/0 conn(0x7f6ba00ce810 legacy=0x7f6ba00c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:52.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.079+0000 7f6ba541f640 1 -- 172.21.15.139:0/442443434 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba0070920 con 0x7f6ba00c9430 2024-09-16T00:08:52.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.080+0000 7f6ba541f640 1 --2- 172.21.15.139:0/442443434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba00c9430 0x7f6ba015bf50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00733a0 tx=0x7f6b9802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72405b61f5935c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:52.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.080+0000 7f6b9f7fe640 1 -- 172.21.15.139:0/442443434 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98036070 con 0x7f6ba00c9430 2024-09-16T00:08:52.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.080+0000 7f6b9f7fe640 1 -- 172.21.15.139:0/442443434 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b98031600 con 0x7f6ba00c9430 2024-09-16T00:08:52.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.080+0000 7f6b9f7fe640 1 -- 172.21.15.139:0/442443434 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b980319d0 con 0x7f6ba00c9430 2024-09-16T00:08:52.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.080+0000 7f6ba6e82640 1 -- 172.21.15.139:0/442443434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba00c9430 msgr2=0x7f6ba015bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:52.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.081+0000 7f6ba6e82640 1 --2- 172.21.15.139:0/442443434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba00c9430 0x7f6ba015bf50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00733a0 tx=0x7f6b9802a730 comp rx=0 tx=0).stop 2024-09-16T00:08:52.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.081+0000 7f6ba6e82640 1 -- 172.21.15.139:0/442443434 shutdown_connections 2024-09-16T00:08:52.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.081+0000 7f6ba6e82640 1 --2- 172.21.15.139:0/442443434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba00c9430 0x7f6ba015bf50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:52.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.081+0000 7f6ba6e82640 1 --2- 172.21.15.139:0/442443434 >> v2:172.21.15.164:3300/0 conn(0x7f6ba00ce430 0x7f6ba00b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:52.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.081+0000 7f6ba6e82640 1 -- 172.21.15.139:0/442443434 >> 172.21.15.139:0/442443434 conn(0x7f6ba006c150 msgr2=0x7f6ba0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:52.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.081+0000 7f6ba6e82640 1 -- 172.21.15.139:0/442443434 shutdown_connections 2024-09-16T00:08:52.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.081+0000 7f6ba6e82640 1 -- 172.21.15.139:0/442443434 wait complete. 2024-09-16T00:08:52.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.082+0000 7f6ba6e82640 1 Processor -- start 2024-09-16T00:08:52.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.082+0000 7f6ba6e82640 1 -- start start 2024-09-16T00:08:52.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba6e82640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ba00c9430 0x7f6ba014fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:52.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba6e82640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6ba00ce430 0x7f6ba014ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba6e82640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba00ce810 0x7f6ba00cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba6e82640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba00b3700 con 0x7f6ba00ce810 2024-09-16T00:08:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba6e82640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba0070ee0 con 0x7f6ba00c9430 2024-09-16T00:08:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba6e82640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba00b3880 con 0x7f6ba00ce430 2024-09-16T00:08:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba541f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba00ce810 0x7f6ba00cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba4c1e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ba00c9430 0x7f6ba014fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba541f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba00ce810 0x7f6ba00cd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59316/0 (socket says 172.21.15.139:59316) 2024-09-16T00:08:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba541f640 1 -- 172.21.15.139:0/2962447759 learned_addr learned my addr 172.21.15.139:0/2962447759 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6b9ffff640 1 --2- 172.21.15.139:0/2962447759 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6ba00ce430 0x7f6ba014ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba541f640 1 -- 172.21.15.139:0/2962447759 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6ba00ce430 msgr2=0x7f6ba014ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba541f640 1 --2- 172.21.15.139:0/2962447759 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6ba00ce430 0x7f6ba014ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba541f640 1 -- 172.21.15.139:0/2962447759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ba00c9430 msgr2=0x7f6ba014fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba541f640 1 --2- 172.21.15.139:0/2962447759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ba00c9430 0x7f6ba014fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:52.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba541f640 1 -- 172.21.15.139:0/2962447759 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba01502d0 con 0x7f6ba00ce810 2024-09-16T00:08:52.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba4c1e640 1 --2- 172.21.15.139:0/2962447759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ba00c9430 0x7f6ba014fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:52.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6ba541f640 1 --2- 172.21.15.139:0/2962447759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba00ce810 0x7f6ba00cd1d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009f40 tx=0x7f6b98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:52.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.083+0000 7f6b9d7fa640 1 -- 172.21.15.139:0/2962447759 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98036050 con 0x7f6ba00ce810 2024-09-16T00:08:52.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.084+0000 7f6b9d7fa640 1 -- 172.21.15.139:0/2962447759 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b98042070 con 0x7f6ba00ce810 2024-09-16T00:08:52.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.084+0000 7f6ba6e82640 1 -- 172.21.15.139:0/2962447759 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba01f1540 con 0x7f6ba00ce810 2024-09-16T00:08:52.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.084+0000 7f6b9d7fa640 1 -- 172.21.15.139:0/2962447759 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98009510 con 0x7f6ba00ce810 2024-09-16T00:08:52.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.084+0000 7f6ba6e82640 1 -- 172.21.15.139:0/2962447759 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba01f1b00 con 0x7f6ba00ce810 2024-09-16T00:08:52.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.085+0000 7f6ba6e82640 1 -- 172.21.15.139:0/2962447759 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba006ffc0 con 0x7f6ba00ce810 2024-09-16T00:08:52.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.085+0000 7f6b9d7fa640 1 -- 172.21.15.139:0/2962447759 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6b98007bf0 con 0x7f6ba00ce810 2024-09-16T00:08:52.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.086+0000 7f6b9d7fa640 1 --2- 172.21.15.139:0/2962447759 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6b8406c2e0 0x7f6b8406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:52.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.086+0000 7f6b9d7fa640 1 -- 172.21.15.139:0/2962447759 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5886+0+0 (secure 0 0 0) 0x7f6b980ac060 con 0x7f6ba00ce810 2024-09-16T00:08:52.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.086+0000 7f6ba4c1e640 1 --2- 172.21.15.139:0/2962447759 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6b8406c2e0 0x7f6b8406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:52.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.086+0000 7f6ba4c1e640 1 --2- 172.21.15.139:0/2962447759 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6b8406c2e0 0x7f6b8406e7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6b94009f30 tx=0x7f6b94009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:52.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.090+0000 7f6b9d7fa640 1 -- 172.21.15.139:0/2962447759 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6b9807a0e0 con 0x7f6ba00ce810 2024-09-16T00:08:52.153 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:52.152+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:52.183 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:52.183 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:52.183 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:52.183 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:52.183 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:52.191 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:08:52.254 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:52.254+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:52.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.273+0000 7f6ba6e82640 1 -- 172.21.15.139:0/2962447759 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) -- 0x7f6ba014cf50 con 0x7f6ba00ce810 2024-09-16T00:08:52.295 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:52.295 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:52.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:52.295 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:52.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.295 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:52.295 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:52.295 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.679291059999997, 2024-09-16T00:08:52.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036770370000000002, 2024-09-16T00:08:52.296 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.296 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.296 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:52.296 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:52.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:08:52.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615154+0000", 2024-09-16T00:08:52.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1510000000000003e-06 2024-09-16T00:08:52.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615163+0000", 2024-09-16T00:08:52.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4490000000000005e-06 2024-09-16T00:08:52.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615182+0000", 2024-09-16T00:08:52.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8876e-05 2024-09-16T00:08:52.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615197+0000", 2024-09-16T00:08:52.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4671000000000001e-05 2024-09-16T00:08:52.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615349+0000", 2024-09-16T00:08:52.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015258700000000001 2024-09-16T00:08:52.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:52.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:08:52.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000204716 2024-09-16T00:08:52.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:52.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:08:52.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017250800000000001 2024-09-16T00:08:52.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:52.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616919+0000", 2024-09-16T00:08:52.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011921799999999999 2024-09-16T00:08:52.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:52.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:08:52.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018087769999999999 2024-09-16T00:08:52.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:52.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618774+0000", 2024-09-16T00:08:52.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6075999999999997e-05 2024-09-16T00:08:52.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618826+0000", 2024-09-16T00:08:52.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2046000000000003e-05 2024-09-16T00:08:52.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:52.301 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:52.301 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.677935271999999, 2024-09-16T00:08:52.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023501059999999998, 2024-09-16T00:08:52.301 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.301 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:52.302 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:52.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:08:52.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616506+0000", 2024-09-16T00:08:52.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4100000000000001e-06 2024-09-16T00:08:52.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617073+0000", 2024-09-16T00:08:52.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056725899999999997 2024-09-16T00:08:52.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617099+0000", 2024-09-16T00:08:52.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5860999999999999e-05 2024-09-16T00:08:52.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617114+0000", 2024-09-16T00:08:52.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5180999999999999e-05 2024-09-16T00:08:52.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617236+0000", 2024-09-16T00:08:52.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121662 2024-09-16T00:08:52.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:52.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001618733 2024-09-16T00:08:52.304 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.304 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.305 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.305 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:31 2.d e22/19 2:b50e409b:::mds_snaptable:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:52.305 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:52.305 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.677041123999999, 2024-09-16T00:08:52.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021851240000000001, 2024-09-16T00:08:52.305 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.305 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.305 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:52.305 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:52.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:08:52.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617400+0000", 2024-09-16T00:08:52.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.28e-06 2024-09-16T00:08:52.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617410+0000", 2024-09-16T00:08:52.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0179e-05 2024-09-16T00:08:52.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617435+0000", 2024-09-16T00:08:52.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5221000000000001e-05 2024-09-16T00:08:52.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617444+0000", 2024-09-16T00:08:52.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1989999999999995e-06 2024-09-16T00:08:52.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617514+0000", 2024-09-16T00:08:52.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9104999999999997e-05 2024-09-16T00:08:52.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:52.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617538+0000", 2024-09-16T00:08:52.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4383999999999999e-05 2024-09-16T00:08:52.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:52.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619536+0000", 2024-09-16T00:08:52.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019982789999999999 2024-09-16T00:08:52.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619584+0000", 2024-09-16T00:08:52.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7477e-05 2024-09-16T00:08:52.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.310 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.310 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:27 2.13 e22/19 2:cd5a64a3:::100.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:52.310 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:52.311 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.676735941999993, 2024-09-16T00:08:52.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018775459999999999, 2024-09-16T00:08:52.311 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.311 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.311 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:52.311 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:52.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:52.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617705+0000", 2024-09-16T00:08:52.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9900000000000009e-07 2024-09-16T00:08:52.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617718+0000", 2024-09-16T00:08:52.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3453e-05 2024-09-16T00:08:52.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617730+0000", 2024-09-16T00:08:52.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2221e-05 2024-09-16T00:08:52.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617739+0000", 2024-09-16T00:08:52.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0660000000000004e-06 2024-09-16T00:08:52.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617806+0000", 2024-09-16T00:08:52.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7397999999999994e-05 2024-09-16T00:08:52.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:52.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617823+0000", 2024-09-16T00:08:52.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7569999999999999e-05 2024-09-16T00:08:52.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:52.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619528+0000", 2024-09-16T00:08:52.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017045859999999999 2024-09-16T00:08:52.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619581+0000", 2024-09-16T00:08:52.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3253e-05 2024-09-16T00:08:52.315 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.315 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.315 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.316 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:28 2.13 e22/19 2:c91d4a1d:::mds0_inotable:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:52.316 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:52.316 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.676388672000002, 2024-09-16T00:08:52.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001558051, 2024-09-16T00:08:52.316 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.316 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.316 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:52.316 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:52.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618051+0000", 2024-09-16T00:08:52.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618052+0000", 2024-09-16T00:08:52.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1399999999999997e-07 2024-09-16T00:08:52.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618065+0000", 2024-09-16T00:08:52.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3495e-05 2024-09-16T00:08:52.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:08:52.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0560999999999999e-05 2024-09-16T00:08:52.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618083+0000", 2024-09-16T00:08:52.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:08:52.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618147+0000", 2024-09-16T00:08:52.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4525999999999999e-05 2024-09-16T00:08:52.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:52.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618166+0000", 2024-09-16T00:08:52.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9045000000000001e-05 2024-09-16T00:08:52.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:52.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619585+0000", 2024-09-16T00:08:52.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014182660000000001 2024-09-16T00:08:52.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619609+0000", 2024-09-16T00:08:52.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4468e-05 2024-09-16T00:08:52.320 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.320 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.320 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.320 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:29 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:52.320 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:52.321 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.676038168000005, 2024-09-16T00:08:52.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012347219999999999, 2024-09-16T00:08:52.321 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.321 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.321 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:52.321 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:52.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:52.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618402+0000", 2024-09-16T00:08:52.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3099999999999997e-07 2024-09-16T00:08:52.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618409+0000", 2024-09-16T00:08:52.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6239999999999996e-06 2024-09-16T00:08:52.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618420+0000", 2024-09-16T00:08:52.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1222e-05 2024-09-16T00:08:52.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618441+0000", 2024-09-16T00:08:52.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0564000000000002e-05 2024-09-16T00:08:52.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618519+0000", 2024-09-16T00:08:52.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7917999999999995e-05 2024-09-16T00:08:52.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:52.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618535+0000", 2024-09-16T00:08:52.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6266e-05 2024-09-16T00:08:52.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:52.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619612+0000", 2024-09-16T00:08:52.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010767039999999999 2024-09-16T00:08:52.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619636+0000", 2024-09-16T00:08:52.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4692999999999999e-05 2024-09-16T00:08:52.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.325 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.325 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:2 2.f e22/19 ondisk, result = 0)", 2024-09-16T00:08:52.325 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:52.325 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.675802141000005, 2024-09-16T00:08:52.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019782099999999999, 2024-09-16T00:08:52.325 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.326 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.326 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:52.326 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:52.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618638+0000", 2024-09-16T00:08:52.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618639+0000", 2024-09-16T00:08:52.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5099999999999998e-07 2024-09-16T00:08:52.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618644+0000", 2024-09-16T00:08:52.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7440000000000004e-06 2024-09-16T00:08:52.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618655+0000", 2024-09-16T00:08:52.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0798999999999999e-05 2024-09-16T00:08:52.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618662+0000", 2024-09-16T00:08:52.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8700000000000003e-06 2024-09-16T00:08:52.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618717+0000", 2024-09-16T00:08:52.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5223999999999998e-05 2024-09-16T00:08:52.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:52.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618724+0000", 2024-09-16T00:08:52.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.046e-06 2024-09-16T00:08:52.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618835+0000", 2024-09-16T00:08:52.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111187 2024-09-16T00:08:52.329 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.329 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.329 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.330 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.b epoch 22/19 rep_tid 122 entries 22'2 (0'0) error 2:d5c7a900:::200.00000003:head by mds.0.4:34 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:08:52.330 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:52.330 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 73.674697748, 2024-09-16T00:08:52.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018692890000000001, 2024-09-16T00:08:52.330 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.330 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.330 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:52.330 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:52.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:52.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619743+0000", 2024-09-16T00:08:52.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1069999999999999e-06 2024-09-16T00:08:52.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619754+0000", 2024-09-16T00:08:52.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0789e-05 2024-09-16T00:08:52.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619777+0000", 2024-09-16T00:08:52.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2844e-05 2024-09-16T00:08:52.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619793+0000", 2024-09-16T00:08:52.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6231000000000001e-05 2024-09-16T00:08:52.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619875+0000", 2024-09-16T00:08:52.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2286000000000005e-05 2024-09-16T00:08:52.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621611+0000", 2024-09-16T00:08:52.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001736032 2024-09-16T00:08:52.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.334 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.334 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.6 38/37 [PushOp(2:654134d2:::mds0_openfiles.0:head, version: 26'3, data_included: [], data_size: 0, omap_header_size: 42, omap_entries_size: 2, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:654134d2:::mds0_openfiles.0:head@26'3, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:52.334 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.839473+0000", 2024-09-16T00:08:52.334 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.454966987999999, 2024-09-16T00:08:52.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044119500000000002, 2024-09-16T00:08:52.334 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.334 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.334 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:52.334 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839473+0000", 2024-09-16T00:08:52.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839473+0000", 2024-09-16T00:08:52.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839474+0000", 2024-09-16T00:08:52.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1999999999999998e-07 2024-09-16T00:08:52.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839512+0000", 2024-09-16T00:08:52.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8546e-05 2024-09-16T00:08:52.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839530+0000", 2024-09-16T00:08:52.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7663e-05 2024-09-16T00:08:52.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839550+0000", 2024-09-16T00:08:52.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9817e-05 2024-09-16T00:08:52.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839658+0000", 2024-09-16T00:08:52.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108213 2024-09-16T00:08:52.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:52.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839672+0000", 2024-09-16T00:08:52.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4518000000000001e-05 2024-09-16T00:08:52.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.839914+0000", 2024-09-16T00:08:52.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000241618 2024-09-16T00:08:52.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.338 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.338 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.338 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:52.340 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:08:52.355 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:52.354+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:52.457 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:52.456+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:52.485 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:52.485 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:52.485 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:52.485 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:52.494 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:08:52.559 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:52.558+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:52.638 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:52.638 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:52.638 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:52.639 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:52.639 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:52.647 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:08:52.691 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:52.691+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:52.751 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:52.751 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:52.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:52.751 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:52.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.751 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:14 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:52.751 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:52.751 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.134468322000004, 2024-09-16T00:08:52.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029190470000000001, 2024-09-16T00:08:52.752 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.752 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.752 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:52.752 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:52.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:52.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615982+0000", 2024-09-16T00:08:52.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7439999999999999e-06 2024-09-16T00:08:52.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:08:52.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3743999999999999e-05 2024-09-16T00:08:52.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:52.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2255e-05 2024-09-16T00:08:52.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616029+0000", 2024-09-16T00:08:52.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1175999999999999e-05 2024-09-16T00:08:52.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616098+0000", 2024-09-16T00:08:52.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9153999999999996e-05 2024-09-16T00:08:52.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:52.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616132+0000", 2024-09-16T00:08:52.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3924000000000002e-05 2024-09-16T00:08:52.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:52.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:52.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027226189999999999 2024-09-16T00:08:52.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618899+0000", 2024-09-16T00:08:52.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4431000000000002e-05 2024-09-16T00:08:52.756 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.756 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.756 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.756 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:15 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:52.756 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:52.756 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.132241501999999, 2024-09-16T00:08:52.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00095618099999999996, 2024-09-16T00:08:52.756 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.756 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.756 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:52.757 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:52.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618207+0000", 2024-09-16T00:08:52.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618208+0000", 2024-09-16T00:08:52.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1850000000000001e-06 2024-09-16T00:08:52.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618217+0000", 2024-09-16T00:08:52.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1439999999999992e-06 2024-09-16T00:08:52.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618230+0000", 2024-09-16T00:08:52.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2435999999999999e-05 2024-09-16T00:08:52.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618238+0000", 2024-09-16T00:08:52.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1319999999999994e-06 2024-09-16T00:08:52.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618331+0000", 2024-09-16T00:08:52.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3453000000000001e-05 2024-09-16T00:08:52.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:52.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618352+0000", 2024-09-16T00:08:52.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0693999999999999e-05 2024-09-16T00:08:52.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:52.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619112+0000", 2024-09-16T00:08:52.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00076007099999999997 2024-09-16T00:08:52.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619163+0000", 2024-09-16T00:08:52.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1066000000000002e-05 2024-09-16T00:08:52.760 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.760 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.760 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.761 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:30 2.15 e22/19 2:aa448500:::500.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:52.761 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:52.761 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.132184017, 2024-09-16T00:08:52.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058031250000000001, 2024-09-16T00:08:52.761 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.761 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.761 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:52.761 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:52.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:08:52.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618265+0000", 2024-09-16T00:08:52.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9200000000000001e-07 2024-09-16T00:08:52.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618271+0000", 2024-09-16T00:08:52.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9580000000000004e-06 2024-09-16T00:08:52.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618279+0000", 2024-09-16T00:08:52.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2239999999999995e-06 2024-09-16T00:08:52.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618285+0000", 2024-09-16T00:08:52.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-06 2024-09-16T00:08:52.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618567+0000", 2024-09-16T00:08:52.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028204400000000001 2024-09-16T00:08:52.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:52.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:08:52.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9678999999999999e-05 2024-09-16T00:08:52.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:52.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624024+0000", 2024-09-16T00:08:52.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054376720000000002 2024-09-16T00:08:52.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624068+0000", 2024-09-16T00:08:52.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:08:52.765 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.765 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.765 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.765 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:20 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:52.766 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:52.766 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.131827653000002, 2024-09-16T00:08:52.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028808739999999998, 2024-09-16T00:08:52.766 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.766 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.766 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:52.766 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:52.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:08:52.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618622+0000", 2024-09-16T00:08:52.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1260000000000001e-06 2024-09-16T00:08:52.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618631+0000", 2024-09-16T00:08:52.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4700000000000008e-06 2024-09-16T00:08:52.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618645+0000", 2024-09-16T00:08:52.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3984e-05 2024-09-16T00:08:52.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:08:52.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2101e-05 2024-09-16T00:08:52.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618720+0000", 2024-09-16T00:08:52.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2761000000000004e-05 2024-09-16T00:08:52.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:52.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618740+0000", 2024-09-16T00:08:52.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9409e-05 2024-09-16T00:08:52.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:52.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621436+0000", 2024-09-16T00:08:52.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002696211 2024-09-16T00:08:52.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621502+0000", 2024-09-16T00:08:52.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5811999999999999e-05 2024-09-16T00:08:52.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.770 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.770 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:21 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:52.770 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:52.770 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.131004364999995, 2024-09-16T00:08:52.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046318540000000004, 2024-09-16T00:08:52.770 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.770 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.770 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:52.770 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:52.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:08:52.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619445+0000", 2024-09-16T00:08:52.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.083e-06 2024-09-16T00:08:52.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619455+0000", 2024-09-16T00:08:52.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5920000000000007e-06 2024-09-16T00:08:52.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619467+0000", 2024-09-16T00:08:52.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2711000000000001e-05 2024-09-16T00:08:52.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619476+0000", 2024-09-16T00:08:52.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0139999999999998e-06 2024-09-16T00:08:52.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619529+0000", 2024-09-16T00:08:52.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3078999999999998e-05 2024-09-16T00:08:52.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:52.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619548+0000", 2024-09-16T00:08:52.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8952000000000001e-05 2024-09-16T00:08:52.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:52.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624037+0000", 2024-09-16T00:08:52.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044895730000000002 2024-09-16T00:08:52.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624076+0000", 2024-09-16T00:08:52.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8850000000000002e-05 2024-09-16T00:08:52.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.774 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.774 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.775 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.14 38/37 [PushOp(2:2f55791f:::606.00000000:head, version: 22'2, data_included: [], data_size: 0, omap_header_size: 274, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(2:2f55791f:::606.00000000:head@22'2, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:52.775 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.838548+0000", 2024-09-16T00:08:52.775 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.911900466999999, 2024-09-16T00:08:52.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044033700000000001, 2024-09-16T00:08:52.775 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.775 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.775 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:52.775 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838548+0000", 2024-09-16T00:08:52.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838548+0000", 2024-09-16T00:08:52.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838549+0000", 2024-09-16T00:08:52.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5600000000000004e-07 2024-09-16T00:08:52.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838567+0000", 2024-09-16T00:08:52.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7850999999999999e-05 2024-09-16T00:08:52.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838597+0000", 2024-09-16T00:08:52.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0416000000000002e-05 2024-09-16T00:08:52.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838619+0000", 2024-09-16T00:08:52.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1274e-05 2024-09-16T00:08:52.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838706+0000", 2024-09-16T00:08:52.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7755000000000003e-05 2024-09-16T00:08:52.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:52.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838721+0000", 2024-09-16T00:08:52.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4953999999999999e-05 2024-09-16T00:08:52.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.838988+0000", 2024-09-16T00:08:52.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026713099999999999 2024-09-16T00:08:52.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.779 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.779 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.14 38/37 [PushOp(2:287034c4:::1.00000000.inode:head, version: 22'4, data_included: [0~582], data_size: 582, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:287034c4:::1.00000000.inode:head@22'4, size: 582, copy_subset: [(0, 582)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 582, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:52.779 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.840824+0000", 2024-09-16T00:08:52.779 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.909624626999999, 2024-09-16T00:08:52.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00055177600000000002, 2024-09-16T00:08:52.779 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.779 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.779 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:52.779 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840824+0000", 2024-09-16T00:08:52.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840824+0000", 2024-09-16T00:08:52.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840825+0000", 2024-09-16T00:08:52.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1100000000000005e-07 2024-09-16T00:08:52.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840840+0000", 2024-09-16T00:08:52.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5308999999999999e-05 2024-09-16T00:08:52.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840863+0000", 2024-09-16T00:08:52.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2663999999999998e-05 2024-09-16T00:08:52.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840884+0000", 2024-09-16T00:08:52.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1798e-05 2024-09-16T00:08:52.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840980+0000", 2024-09-16T00:08:52.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5299999999999999e-05 2024-09-16T00:08:52.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:52.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.840995+0000", 2024-09-16T00:08:52.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4895999999999999e-05 2024-09-16T00:08:52.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.841376+0000", 2024-09-16T00:08:52.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00038099800000000001 2024-09-16T00:08:52.783 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.783 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.783 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.783 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.14 38/37 [PushOp(2:2e2fa760:::200.00000001:head, version: 28'9, data_included: [0~5883], data_size: 5883, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:2e2fa760:::200.00000001:head@28'9, size: 5883, copy_subset: [(0, 5883)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 5883, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:52.783 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.848988+0000", 2024-09-16T00:08:52.783 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.901460307000001, 2024-09-16T00:08:52.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00037334, 2024-09-16T00:08:52.783 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.783 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.783 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:52.784 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.848988+0000", 2024-09-16T00:08:52.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.848988+0000", 2024-09-16T00:08:52.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.848989+0000", 2024-09-16T00:08:52.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5099999999999998e-07 2024-09-16T00:08:52.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849004+0000", 2024-09-16T00:08:52.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5192e-05 2024-09-16T00:08:52.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849023+0000", 2024-09-16T00:08:52.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.844e-05 2024-09-16T00:08:52.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849033+0000", 2024-09-16T00:08:52.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0061000000000001e-05 2024-09-16T00:08:52.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849096+0000", 2024-09-16T00:08:52.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3627999999999995e-05 2024-09-16T00:08:52.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:52.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849105+0000", 2024-09-16T00:08:52.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5890000000000003e-06 2024-09-16T00:08:52.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.849362+0000", 2024-09-16T00:08:52.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000256579 2024-09-16T00:08:52.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.787 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.787 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "MOSDPGPush(2.14 38/37 [PushOp(2:292cf221:::200.00000000:head, version: 28'10, data_included: [0~90], data_size: 90, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:292cf221:::200.00000000:head@28'10, size: 90, copy_subset: [(0, 90)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 90, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T00:08:52.787 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:24.851655+0000", 2024-09-16T00:08:52.787 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.898793642000001, 2024-09-16T00:08:52.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036283000000000002, 2024-09-16T00:08:52.787 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:52.787 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:52.788 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:52.788 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:52.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:52.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851655+0000", 2024-09-16T00:08:52.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:52.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851655+0000", 2024-09-16T00:08:52.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:52.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:52.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851656+0000", 2024-09-16T00:08:52.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0399999999999995e-07 2024-09-16T00:08:52.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:52.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851672+0000", 2024-09-16T00:08:52.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6801000000000001e-05 2024-09-16T00:08:52.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:52.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851689+0000", 2024-09-16T00:08:52.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6810999999999999e-05 2024-09-16T00:08:52.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:52.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851699+0000", 2024-09-16T00:08:52.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.995e-06 2024-09-16T00:08:52.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:52.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851767+0000", 2024-09-16T00:08:52.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8194000000000005e-05 2024-09-16T00:08:52.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:52.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.851777+0000", 2024-09-16T00:08:52.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2450000000000004e-06 2024-09-16T00:08:52.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:52.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:52.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:52.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:24.852018+0000", 2024-09-16T00:08:52.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024107999999999999 2024-09-16T00:08:52.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.791 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:52.791 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:52.791 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:52.793 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:08:52.793 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:52.792+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:52.895 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:52.895+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:52.941 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:52.941 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:52.941 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:52.941 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:52.950 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:08:52.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.964+0000 7f6b9d7fa640 1 -- 172.21.15.139:0/2962447759 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v54) ==== 113+0+0 (secure 0 0 0) 0x7f6b9807e3e0 con 0x7f6ba00ce810 2024-09-16T00:08:52.965 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T00:08:52.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.967+0000 7f6ba6e82640 1 -- 172.21.15.139:0/2962447759 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6b8406c2e0 msgr2=0x7f6b8406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:52.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.967+0000 7f6ba6e82640 1 --2- 172.21.15.139:0/2962447759 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6b8406c2e0 0x7f6b8406e7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6b94009f30 tx=0x7f6b94009190 comp rx=0 tx=0).stop 2024-09-16T00:08:52.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.967+0000 7f6ba6e82640 1 -- 172.21.15.139:0/2962447759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba00ce810 msgr2=0x7f6ba00cd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:52.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.967+0000 7f6ba6e82640 1 --2- 172.21.15.139:0/2962447759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba00ce810 0x7f6ba00cd1d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009f40 tx=0x7f6b98004a40 comp rx=0 tx=0).stop 2024-09-16T00:08:52.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.967+0000 7f6ba6e82640 1 -- 172.21.15.139:0/2962447759 shutdown_connections 2024-09-16T00:08:52.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.967+0000 7f6ba6e82640 1 --2- 172.21.15.139:0/2962447759 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6b8406c2e0 0x7f6b8406e7a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:52.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.967+0000 7f6ba6e82640 1 --2- 172.21.15.139:0/2962447759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ba00ce810 0x7f6ba00cd1d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:52.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.967+0000 7f6ba6e82640 1 --2- 172.21.15.139:0/2962447759 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6ba00ce430 0x7f6ba014ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:52.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.967+0000 7f6ba6e82640 1 --2- 172.21.15.139:0/2962447759 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6ba00c9430 0x7f6ba014fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:52.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.967+0000 7f6ba6e82640 1 -- 172.21.15.139:0/2962447759 >> 172.21.15.139:0/2962447759 conn(0x7f6ba006c150 msgr2=0x7f6ba0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:52.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.967+0000 7f6ba6e82640 1 -- 172.21.15.139:0/2962447759 shutdown_connections 2024-09-16T00:08:52.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:52.967+0000 7f6ba6e82640 1 -- 172.21.15.139:0/2962447759 wait complete. 2024-09-16T00:08:52.980 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T00:08:52.988 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T00:08:52.995 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:52.994+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:53.057 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:53.057 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:53.057 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:53.057 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:53.057 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:53.066 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:08:53.108 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:53.107+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:53.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.122+0000 7ffb0fd31640 1 Processor -- start 2024-09-16T00:08:53.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.122+0000 7ffb0fd31640 1 -- start start 2024-09-16T00:08:53.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.123+0000 7ffb0fd31640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffb08068060 0x7ffb0815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.123+0000 7ffb0fd31640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb080cc4b0 con 0x7ffb08154c20 2024-09-16T00:08:53.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.123+0000 7ffb0fd31640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb080cc790 con 0x7ffb080ccc90 2024-09-16T00:08:53.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.123+0000 7ffb0fd31640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ffb080c5830 con 0x7ffb08068060 2024-09-16T00:08:53.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.123+0000 7f0a2f6ba640 1 Processor -- start 2024-09-16T00:08:53.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.123+0000 7ffb0d2cc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffb08068060 0x7ffb0815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.123+0000 7ffb0dacd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ffb08154c20 0x7ffb080c5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:44438/0 (socket says 172.21.15.139:44438) 2024-09-16T00:08:53.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.123+0000 7ffb0dacd640 1 -- 172.21.15.139:0/605299325 learned_addr learned my addr 172.21.15.139:0/605299325 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:53.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.123+0000 7f0a2f6ba640 1 -- start start 2024-09-16T00:08:53.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.123+0000 7ffb0d2cc640 1 -- 172.21.15.139:0/605299325 >> v1:172.21.15.164:6789/0 conn(0x7ffb080ccc90 legacy=0x7ffb08165180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.123+0000 7ffb0cacb640 1 -- 172.21.15.139:0/605299325 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1607676636 0 0) 0x7ffb080cc4b0 con 0x7ffb08154c20 2024-09-16T00:08:53.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.123+0000 7ffb0d2cc640 1 -- 172.21.15.139:0/605299325 >> v1:172.21.15.139:6789/0 conn(0x7ffb08154c20 legacy=0x7ffb080c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.123+0000 7ffb0d2cc640 1 -- 172.21.15.139:0/605299325 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0815ddf0 con 0x7ffb08068060 2024-09-16T00:08:53.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7f0a2f6ba640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a28154aa0 0x7f0a2815e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7f0a2f6ba640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0a280c6da0 0x7f0a280cd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7f0a2f6ba640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0a280c7aa0 0x7f0a280cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7f0a2f6ba640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0a280c8760 con 0x7f0a280c6da0 2024-09-16T00:08:53.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7f0a2f6ba640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f0a280c8a40 con 0x7f0a280c7aa0 2024-09-16T00:08:53.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7f0a2f6ba640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0a280c8d20 con 0x7f0a28154aa0 2024-09-16T00:08:53.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7ffb0d2cc640 1 --2- 172.21.15.139:0/605299325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb08068060 0x7ffb0815b280 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009710 tx=0x7ffaf802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51b2991402387d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:53.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7f0a2cc55640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0a280c6da0 0x7f0a280cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7f0a2cc55640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0a280c6da0 0x7f0a280cd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59322/0 (socket says 172.21.15.139:59322) 2024-09-16T00:08:53.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7f0a2cc55640 1 -- 172.21.15.139:0/2569601833 learned_addr learned my addr 172.21.15.139:0/2569601833 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:53.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7f0a2dc57640 1 --2- 172.21.15.139:0/2569601833 >> v2:172.21.15.164:3300/0 conn(0x7f0a280c7aa0 0x7f0a280cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7f0a2d456640 1 --2- 172.21.15.139:0/2569601833 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a28154aa0 0x7f0a2815e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7ffb0cacb640 1 -- 172.21.15.139:0/605299325 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8037070 con 0x7ffb08068060 2024-09-16T00:08:53.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7ffb0cacb640 1 -- 172.21.15.139:0/605299325 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf802adc0 con 0x7ffb08068060 2024-09-16T00:08:53.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7ffb0cacb640 1 -- 172.21.15.139:0/605299325 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8032660 con 0x7ffb08068060 2024-09-16T00:08:53.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7f0a2cc55640 1 -- 172.21.15.139:0/2569601833 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a28154aa0 msgr2=0x7f0a2815e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:53.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7f0a2cc55640 1 --2- 172.21.15.139:0/2569601833 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a28154aa0 0x7f0a2815e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7f0a2cc55640 1 -- 172.21.15.139:0/2569601833 >> v2:172.21.15.164:3300/0 conn(0x7f0a280c7aa0 msgr2=0x7f0a280cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:53.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7f0a2cc55640 1 --2- 172.21.15.139:0/2569601833 >> v2:172.21.15.164:3300/0 conn(0x7f0a280c7aa0 0x7f0a280cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.124+0000 7f0a2cc55640 1 -- 172.21.15.139:0/2569601833 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2815deb0 con 0x7f0a280c6da0 2024-09-16T00:08:53.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.125+0000 7ffb0fd31640 1 -- 172.21.15.139:0/605299325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb08068060 msgr2=0x7ffb0815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.125+0000 7f0a2cc55640 1 --2- 172.21.15.139:0/2569601833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a280c6da0 0x7f0a280cd030 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0a28073550 tx=0x7f0a1802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761a940abd722d92 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:53.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.125+0000 7ffb0fd31640 1 --2- 172.21.15.139:0/605299325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb08068060 0x7ffb0815b280 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009710 tx=0x7ffaf802a0b0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.125+0000 7ffb0fd31640 1 -- 172.21.15.139:0/605299325 shutdown_connections 2024-09-16T00:08:53.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.125+0000 7ffb0fd31640 1 --2- 172.21.15.139:0/605299325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb08068060 0x7ffb0815b280 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.125+0000 7ffb0fd31640 1 -- 172.21.15.139:0/605299325 >> 172.21.15.139:0/605299325 conn(0x7ffb0806ca40 msgr2=0x7ffb08067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:53.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.125+0000 7f0a1ffff640 1 -- 172.21.15.139:0/2569601833 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18037070 con 0x7f0a280c6da0 2024-09-16T00:08:53.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.125+0000 7ffb0fd31640 1 -- 172.21.15.139:0/605299325 shutdown_connections 2024-09-16T00:08:53.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.125+0000 7ffb0fd31640 1 -- 172.21.15.139:0/605299325 wait complete. 2024-09-16T00:08:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.125+0000 7f0a1ffff640 1 -- 172.21.15.139:0/2569601833 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a18032440 con 0x7f0a280c6da0 2024-09-16T00:08:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.125+0000 7f0a1ffff640 1 -- 172.21.15.139:0/2569601833 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18032810 con 0x7f0a280c6da0 2024-09-16T00:08:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.126+0000 7f0a2f6ba640 1 -- 172.21.15.139:0/2569601833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a280c6da0 msgr2=0x7f0a280cd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.126+0000 7f0a2f6ba640 1 --2- 172.21.15.139:0/2569601833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a280c6da0 0x7f0a280cd030 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0a28073550 tx=0x7f0a1802a1e0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.126+0000 7f0a2f6ba640 1 -- 172.21.15.139:0/2569601833 shutdown_connections 2024-09-16T00:08:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.126+0000 7f0a2f6ba640 1 --2- 172.21.15.139:0/2569601833 >> v2:172.21.15.164:3300/0 conn(0x7f0a280c7aa0 0x7f0a280cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.126+0000 7f0a2f6ba640 1 --2- 172.21.15.139:0/2569601833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a280c6da0 0x7f0a280cd030 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.126+0000 7f0a2f6ba640 1 --2- 172.21.15.139:0/2569601833 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a28154aa0 0x7f0a2815e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.126+0000 7f0a2f6ba640 1 -- 172.21.15.139:0/2569601833 >> 172.21.15.139:0/2569601833 conn(0x7f0a2806cc30 msgr2=0x7f0a2806d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.126+0000 7f0a2f6ba640 1 -- 172.21.15.139:0/2569601833 shutdown_connections 2024-09-16T00:08:53.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.126+0000 7f0a2f6ba640 1 -- 172.21.15.139:0/2569601833 wait complete. 2024-09-16T00:08:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.127+0000 7ffb0fd31640 1 Processor -- start 2024-09-16T00:08:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.127+0000 7ffb0fd31640 1 -- start start 2024-09-16T00:08:53.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.127+0000 7f0a2f6ba640 1 Processor -- start 2024-09-16T00:08:53.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.127+0000 7f0a2f6ba640 1 -- start start 2024-09-16T00:08:53.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.127+0000 7ffb0fd31640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb08068060 0x7ffb0814c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.127+0000 7ffb0fd31640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb080ccc90 0x7ffb080cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.127+0000 7ffb0fd31640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb08154c20 0x7ffb080b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.127+0000 7ffb0fd31640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffb080cc260 con 0x7ffb08068060 2024-09-16T00:08:53.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.127+0000 7ffb0fd31640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0815e3b0 con 0x7ffb080ccc90 2024-09-16T00:08:53.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.127+0000 7ffb0fd31640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ffb0815e690 con 0x7ffb08154c20 2024-09-16T00:08:53.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.127+0000 7ffb0dacd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb08068060 0x7ffb0814c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.127+0000 7ffb0dacd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb08068060 0x7ffb0814c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59330/0 (socket says 172.21.15.139:59330) 2024-09-16T00:08:53.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.127+0000 7ffb0d2cc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb080ccc90 0x7ffb080cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.127+0000 7ffb0dacd640 1 -- 172.21.15.139:0/438708207 learned_addr learned my addr 172.21.15.139:0/438708207 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:53.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.127+0000 7ffb0e2ce640 1 --2- 172.21.15.139:0/438708207 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb08154c20 0x7ffb080b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.128+0000 7ffb0dacd640 1 -- 172.21.15.139:0/438708207 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb08154c20 msgr2=0x7ffb080b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.128+0000 7ffb0dacd640 1 --2- 172.21.15.139:0/438708207 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb08154c20 0x7ffb080b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.128+0000 7ffb0dacd640 1 -- 172.21.15.139:0/438708207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb080ccc90 msgr2=0x7ffb080cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.128+0000 7ffb0dacd640 1 --2- 172.21.15.139:0/438708207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb080ccc90 0x7ffb080cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.128+0000 7ffb0dacd640 1 -- 172.21.15.139:0/438708207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0814dc40 con 0x7ffb08068060 2024-09-16T00:08:53.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.128+0000 7ffb0d2cc640 1 --2- 172.21.15.139:0/438708207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb080ccc90 0x7ffb080cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:53.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.128+0000 7ffb0dacd640 1 --2- 172.21.15.139:0/438708207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb08068060 0x7ffb0814c350 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400c460 tx=0x7ffb0400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:53.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.128+0000 7ffafe7fc640 1 -- 172.21.15.139:0/438708207 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04018070 con 0x7ffb08068060 2024-09-16T00:08:53.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.128+0000 7ffafe7fc640 1 -- 172.21.15.139:0/438708207 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb04007500 con 0x7ffb08068060 2024-09-16T00:08:53.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.128+0000 7ffb0fd31640 1 -- 172.21.15.139:0/438708207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb0814deb0 con 0x7ffb08068060 2024-09-16T00:08:53.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.128+0000 7ffafe7fc640 1 -- 172.21.15.139:0/438708207 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb04004a20 con 0x7ffb08068060 2024-09-16T00:08:53.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.128+0000 7ffb0fd31640 1 -- 172.21.15.139:0/438708207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb0814e470 con 0x7ffb08068060 2024-09-16T00:08:53.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.128+0000 7f0a2f6ba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a280c6da0 0x7f0a281596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.128+0000 7f0a2f6ba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a280c7aa0 0x7f0a28155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.128+0000 7f0a2f6ba640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a28154aa0 0x7f0a281576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.128+0000 7f0a2f6ba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0a28154e80 con 0x7f0a280c6da0 2024-09-16T00:08:53.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a2f6ba640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a280c9000 con 0x7f0a280c7aa0 2024-09-16T00:08:53.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a2f6ba640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0a280c8420 con 0x7f0a28154aa0 2024-09-16T00:08:53.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a2d456640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a280c6da0 0x7f0a281596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a2cc55640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a280c7aa0 0x7f0a28155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a2d456640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a280c6da0 0x7f0a281596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59346/0 (socket says 172.21.15.139:59346) 2024-09-16T00:08:53.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a2d456640 1 -- 172.21.15.139:0/1390873008 learned_addr learned my addr 172.21.15.139:0/1390873008 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:53.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a2cc55640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a280c7aa0 0x7f0a28155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:38372/0 (socket says 172.21.15.139:38372) 2024-09-16T00:08:53.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a2dc57640 1 --2- 172.21.15.139:0/1390873008 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a28154aa0 0x7f0a281576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a2dc57640 1 -- 172.21.15.139:0/1390873008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a280c7aa0 msgr2=0x7f0a28155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a2dc57640 1 --2- 172.21.15.139:0/1390873008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a280c7aa0 0x7f0a28155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a2dc57640 1 -- 172.21.15.139:0/1390873008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a280c6da0 msgr2=0x7f0a281596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a2dc57640 1 --2- 172.21.15.139:0/1390873008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a280c6da0 0x7f0a281596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a2dc57640 1 -- 172.21.15.139:0/1390873008 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a281f12d0 con 0x7f0a28154aa0 2024-09-16T00:08:53.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a2dc57640 1 --2- 172.21.15.139:0/1390873008 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a28154aa0 0x7f0a281576e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009d90 tx=0x7f0a2000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:53.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a1dffb640 1 -- 172.21.15.139:0/1390873008 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a20018020 con 0x7f0a28154aa0 2024-09-16T00:08:53.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7ffb0fd31640 1 -- 172.21.15.139:0/438708207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb080686b0 con 0x7ffb08068060 2024-09-16T00:08:53.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a1dffb640 1 -- 172.21.15.139:0/1390873008 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a20014480 con 0x7f0a28154aa0 2024-09-16T00:08:53.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a1dffb640 1 -- 172.21.15.139:0/1390873008 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a20014870 con 0x7f0a28154aa0 2024-09-16T00:08:53.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.129+0000 7f0a2f6ba640 1 -- 172.21.15.139:0/1390873008 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a280bd2c0 con 0x7f0a28154aa0 2024-09-16T00:08:53.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.130+0000 7f0a2f6ba640 1 -- 172.21.15.139:0/1390873008 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a281f0ff0 con 0x7f0a28154aa0 2024-09-16T00:08:53.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.130+0000 7ffafe7fc640 1 -- 172.21.15.139:0/438708207 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ffb0401b070 con 0x7ffb08068060 2024-09-16T00:08:53.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.130+0000 7ffafe7fc640 1 --2- 172.21.15.139:0/438708207 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffaec06c2e0 0x7ffaec06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.131+0000 7ffafe7fc640 1 -- 172.21.15.139:0/438708207 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5886+0+0 (secure 0 0 0) 0x7ffb0408a600 con 0x7ffb08068060 2024-09-16T00:08:53.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.131+0000 7ffb0d2cc640 1 --2- 172.21.15.139:0/438708207 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffaec06c2e0 0x7ffaec06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.131+0000 7f0a2f6ba640 1 -- 172.21.15.139:0/1390873008 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a280c73d0 con 0x7f0a28154aa0 2024-09-16T00:08:53.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.131+0000 7ffb0d2cc640 1 --2- 172.21.15.139:0/438708207 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffaec06c2e0 0x7ffaec06e7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8034000 tx=0x7ffaf802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:53.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.131+0000 7f0a1dffb640 1 -- 172.21.15.139:0/1390873008 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0a20005d10 con 0x7f0a28154aa0 2024-09-16T00:08:53.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.132+0000 7f0a1dffb640 1 --2- 172.21.15.139:0/1390873008 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a0c06c2b0 0x7f0a0c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.132+0000 7f0a1dffb640 1 -- 172.21.15.139:0/1390873008 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(54..54 src has 1..54) ==== 5886+0+0 (secure 0 0 0) 0x7f0a2008c190 con 0x7f0a28154aa0 2024-09-16T00:08:53.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.132+0000 7f0a2d456640 1 --2- 172.21.15.139:0/1390873008 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a0c06c2b0 0x7f0a0c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.132+0000 7f0a2d456640 1 --2- 172.21.15.139:0/1390873008 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a0c06c2b0 0x7f0a0c06e770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009ae0 tx=0x7f0a24009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:53.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.135+0000 7ffafe7fc640 1 -- 172.21.15.139:0/438708207 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffb04058700 con 0x7ffb08068060 2024-09-16T00:08:53.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.136+0000 7f0a1dffb640 1 -- 172.21.15.139:0/1390873008 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0a20022030 con 0x7f0a28154aa0 2024-09-16T00:08:53.196 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:53.196+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:53.214 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:53.214 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:53.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:53.214 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:53.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.214 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:53.216 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:53.216 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.597042580999997, 2024-09-16T00:08:53.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056100969999999997, 2024-09-16T00:08:53.217 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.217 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.217 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.217 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:53.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:53.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615352+0000", 2024-09-16T00:08:53.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9859999999999999e-06 2024-09-16T00:08:53.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615363+0000", 2024-09-16T00:08:53.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1168999999999999e-05 2024-09-16T00:08:53.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615382+0000", 2024-09-16T00:08:53.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8224e-05 2024-09-16T00:08:53.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615393+0000", 2024-09-16T00:08:53.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0925e-05 2024-09-16T00:08:53.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615624+0000", 2024-09-16T00:08:53.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023159600000000001 2024-09-16T00:08:53.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615721+0000", 2024-09-16T00:08:53.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7040000000000006e-05 2024-09-16T00:08:53.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:53.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615885+0000", 2024-09-16T00:08:53.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163456 2024-09-16T00:08:53.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:08:53.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002191368 2024-09-16T00:08:53.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620870+0000", 2024-09-16T00:08:53.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941839999999999 2024-09-16T00:08:53.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620922+0000", 2024-09-16T00:08:53.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1641999999999998e-05 2024-09-16T00:08:53.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620956+0000", 2024-09-16T00:08:53.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4507e-05 2024-09-16T00:08:53.223 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.224 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.224 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.224 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:53.224 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:53.224 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.596952221999999, 2024-09-16T00:08:53.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005695739, 2024-09-16T00:08:53.224 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.224 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.224 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.225 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:53.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:53.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:53.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4900000000000005e-07 2024-09-16T00:08:53.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615442+0000", 2024-09-16T00:08:53.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.994e-06 2024-09-16T00:08:53.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:08:53.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1081e-05 2024-09-16T00:08:53.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615473+0000", 2024-09-16T00:08:53.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1910000000000006e-06 2024-09-16T00:08:53.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616230+0000", 2024-09-16T00:08:53.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075700400000000003 2024-09-16T00:08:53.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616305+0000", 2024-09-16T00:08:53.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4759999999999996e-05 2024-09-16T00:08:53.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:53.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616466+0000", 2024-09-16T00:08:53.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000161714 2024-09-16T00:08:53.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619821+0000", 2024-09-16T00:08:53.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033547339999999998 2024-09-16T00:08:53.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621028+0000", 2024-09-16T00:08:53.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012067530000000001 2024-09-16T00:08:53.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621087+0000", 2024-09-16T00:08:53.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9364999999999998e-05 2024-09-16T00:08:53.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621132+0000", 2024-09-16T00:08:53.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5293999999999997e-05 2024-09-16T00:08:53.231 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.231 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.231 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.231 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:53.231 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:53.232 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.596889989999994, 2024-09-16T00:08:53.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050565760000000001, 2024-09-16T00:08:53.232 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.232 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.233 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.233 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:53.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:53.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615500+0000", 2024-09-16T00:08:53.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8199999999999999e-07 2024-09-16T00:08:53.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615505+0000", 2024-09-16T00:08:53.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5280000000000003e-06 2024-09-16T00:08:53.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:53.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0317e-05 2024-09-16T00:08:53.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615523+0000", 2024-09-16T00:08:53.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4580000000000004e-06 2024-09-16T00:08:53.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615607+0000", 2024-09-16T00:08:53.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4153999999999994e-05 2024-09-16T00:08:53.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615748+0000", 2024-09-16T00:08:53.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140533 2024-09-16T00:08:53.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:53.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615911+0000", 2024-09-16T00:08:53.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163678 2024-09-16T00:08:53.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619944+0000", 2024-09-16T00:08:53.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040330909999999999 2024-09-16T00:08:53.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620471+0000", 2024-09-16T00:08:53.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052645600000000002 2024-09-16T00:08:53.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620512+0000", 2024-09-16T00:08:53.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0738000000000002e-05 2024-09-16T00:08:53.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620555+0000", 2024-09-16T00:08:53.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3841000000000003e-05 2024-09-16T00:08:53.239 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.239 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.239 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.240 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:53.240 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:53.240 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.596837883000006, 2024-09-16T00:08:53.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089162479999999999, 2024-09-16T00:08:53.240 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.240 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.240 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.240 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:53.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:53.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615552+0000", 2024-09-16T00:08:53.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:08:53.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615556+0000", 2024-09-16T00:08:53.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3610000000000003e-06 2024-09-16T00:08:53.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615566+0000", 2024-09-16T00:08:53.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6099999999999995e-06 2024-09-16T00:08:53.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615574+0000", 2024-09-16T00:08:53.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9379999999999992e-06 2024-09-16T00:08:53.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616777+0000", 2024-09-16T00:08:53.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012031640000000001 2024-09-16T00:08:53.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616882+0000", 2024-09-16T00:08:53.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105146 2024-09-16T00:08:53.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:53.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619696+0000", 2024-09-16T00:08:53.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028142179999999998 2024-09-16T00:08:53.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624234+0000", 2024-09-16T00:08:53.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045381889999999998 2024-09-16T00:08:53.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624393+0000", 2024-09-16T00:08:53.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015871200000000001 2024-09-16T00:08:53.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624434+0000", 2024-09-16T00:08:53.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0908999999999997e-05 2024-09-16T00:08:53.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624467+0000", 2024-09-16T00:08:53.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3217999999999999e-05 2024-09-16T00:08:53.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.245 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.246 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:53.246 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:53.246 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.596768085999997, 2024-09-16T00:08:53.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064898359999999997, 2024-09-16T00:08:53.246 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.246 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.246 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.246 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:53.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:53.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615622+0000", 2024-09-16T00:08:53.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.181e-06 2024-09-16T00:08:53.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615628+0000", 2024-09-16T00:08:53.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2450000000000003e-06 2024-09-16T00:08:53.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:08:53.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2554999999999999e-05 2024-09-16T00:08:53.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615650+0000", 2024-09-16T00:08:53.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4350000000000003e-06 2024-09-16T00:08:53.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616772+0000", 2024-09-16T00:08:53.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001122058 2024-09-16T00:08:53.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616889+0000", 2024-09-16T00:08:53.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116623 2024-09-16T00:08:53.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:53.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617035+0000", 2024-09-16T00:08:53.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014579799999999999 2024-09-16T00:08:53.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619829+0000", 2024-09-16T00:08:53.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941379999999998 2024-09-16T00:08:53.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622024+0000", 2024-09-16T00:08:53.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021949920000000002 2024-09-16T00:08:53.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622087+0000", 2024-09-16T00:08:53.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3048999999999999e-05 2024-09-16T00:08:53.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622111+0000", 2024-09-16T00:08:53.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3762000000000001e-05 2024-09-16T00:08:53.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.251 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.251 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:53.252 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:53.252 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.596705525000004, 2024-09-16T00:08:53.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065192160000000004, 2024-09-16T00:08:53.252 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.252 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.252 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.252 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:53.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:53.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615684+0000", 2024-09-16T00:08:53.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8299999999999995e-07 2024-09-16T00:08:53.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615691+0000", 2024-09-16T00:08:53.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.866e-06 2024-09-16T00:08:53.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615707+0000", 2024-09-16T00:08:53.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6073999999999999e-05 2024-09-16T00:08:53.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615714+0000", 2024-09-16T00:08:53.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-16T00:08:53.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617403+0000", 2024-09-16T00:08:53.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016881629999999999 2024-09-16T00:08:53.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617559+0000", 2024-09-16T00:08:53.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015669699999999999 2024-09-16T00:08:53.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:53.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:53.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000144165 2024-09-16T00:08:53.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621413+0000", 2024-09-16T00:08:53.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037092200000000001 2024-09-16T00:08:53.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622147+0000", 2024-09-16T00:08:53.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073468800000000005 2024-09-16T00:08:53.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622180+0000", 2024-09-16T00:08:53.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2901999999999998e-05 2024-09-16T00:08:53.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622203+0000", 2024-09-16T00:08:53.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2211000000000001e-05 2024-09-16T00:08:53.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.257 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.258 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:53.260 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:53.260 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.596662430999999, 2024-09-16T00:08:53.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065825270000000003, 2024-09-16T00:08:53.260 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.260 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.260 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:53.261 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:53.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:53.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:08:53.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7299999999999996e-07 2024-09-16T00:08:53.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615730+0000", 2024-09-16T00:08:53.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.004e-06 2024-09-16T00:08:53.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615737+0000", 2024-09-16T00:08:53.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4789999999999999e-06 2024-09-16T00:08:53.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615745+0000", 2024-09-16T00:08:53.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6699999999999994e-06 2024-09-16T00:08:53.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618115+0000", 2024-09-16T00:08:53.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023701569999999999 2024-09-16T00:08:53.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618247+0000", 2024-09-16T00:08:53.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013185000000000001 2024-09-16T00:08:53.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622309+0000", 2024-09-16T00:08:53.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040617939999999996 2024-09-16T00:08:53.264 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.264 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.264 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.264 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:53.264 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:53.264 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.596635182, 2024-09-16T00:08:53.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036685950000000002, 2024-09-16T00:08:53.265 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.265 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.265 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:53.265 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:53.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:53.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:53.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T00:08:53.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615763+0000", 2024-09-16T00:08:53.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4360000000000002e-06 2024-09-16T00:08:53.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615770+0000", 2024-09-16T00:08:53.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7540000000000006e-06 2024-09-16T00:08:53.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615803+0000", 2024-09-16T00:08:53.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3008e-05 2024-09-16T00:08:53.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615888+0000", 2024-09-16T00:08:53.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5056999999999996e-05 2024-09-16T00:08:53.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:08:53.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013059700000000001 2024-09-16T00:08:53.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619422+0000", 2024-09-16T00:08:53.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034033230000000002 2024-09-16T00:08:53.269 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.269 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.269 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.269 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing_reply(2.1a epoch 22/19 rep_tid 2 lcod 22'1)", 2024-09-16T00:08:53.269 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:53.269 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.593193869000004, 2024-09-16T00:08:53.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024304200000000001, 2024-09-16T00:08:53.269 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.269 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.269 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:53.269 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619195+0000", 2024-09-16T00:08:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619196+0000", 2024-09-16T00:08:53.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2759999999999999e-06 2024-09-16T00:08:53.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619203+0000", 2024-09-16T00:08:53.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1779999999999997e-06 2024-09-16T00:08:53.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619223+0000", 2024-09-16T00:08:53.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9159000000000001e-05 2024-09-16T00:08:53.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619231+0000", 2024-09-16T00:08:53.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7830000000000004e-06 2024-09-16T00:08:53.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619320+0000", 2024-09-16T00:08:53.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8805000000000001e-05 2024-09-16T00:08:53.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619438+0000", 2024-09-16T00:08:53.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000117841 2024-09-16T00:08:53.272 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.272 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.273 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:16 2.1d e22/19 2:b871830b:::605.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:53.273 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:53.273 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.592646591999994, 2024-09-16T00:08:53.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045616900000000002, 2024-09-16T00:08:53.273 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.273 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.273 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.273 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:53.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:53.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:08:53.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:08:53.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619755+0000", 2024-09-16T00:08:53.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3599999999999996e-06 2024-09-16T00:08:53.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619767+0000", 2024-09-16T00:08:53.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2805e-05 2024-09-16T00:08:53.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619787+0000", 2024-09-16T00:08:53.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9264000000000001e-05 2024-09-16T00:08:53.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619869+0000", 2024-09-16T00:08:53.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2639999999999995e-05 2024-09-16T00:08:53.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619895+0000", 2024-09-16T00:08:53.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5890000000000001e-05 2024-09-16T00:08:53.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624264+0000", 2024-09-16T00:08:53.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043690049999999996 2024-09-16T00:08:53.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624304+0000", 2024-09-16T00:08:53.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9807e-05 2024-09-16T00:08:53.277 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.277 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.277 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.277 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:17 2.1d e22/19 2:b871830b:::605.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:53.277 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:53.277 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.592293882999996, 2024-09-16T00:08:53.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042370910000000001, 2024-09-16T00:08:53.278 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.278 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.278 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.278 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:53.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:53.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620096+0000", 2024-09-16T00:08:53.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:08:53.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620102+0000", 2024-09-16T00:08:53.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7220000000000002e-06 2024-09-16T00:08:53.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620112+0000", 2024-09-16T00:08:53.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0156999999999999e-05 2024-09-16T00:08:53.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620120+0000", 2024-09-16T00:08:53.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2880000000000004e-06 2024-09-16T00:08:53.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620171+0000", 2024-09-16T00:08:53.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.168e-05 2024-09-16T00:08:53.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620189+0000", 2024-09-16T00:08:53.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7419e-05 2024-09-16T00:08:53.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624306+0000", 2024-09-16T00:08:53.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041175430000000004 2024-09-16T00:08:53.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624332+0000", 2024-09-16T00:08:53.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5717000000000001e-05 2024-09-16T00:08:53.281 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.282 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.282 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.282 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:24 2.7 e22/19 ondisk, result = 0)", 2024-09-16T00:08:53.282 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:53.282 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.592028175999999, 2024-09-16T00:08:53.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000208018, 2024-09-16T00:08:53.282 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.282 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.282 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:53.282 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:53.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:53.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620361+0000", 2024-09-16T00:08:53.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8500000000000001e-07 2024-09-16T00:08:53.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620366+0000", 2024-09-16T00:08:53.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5379999999999996e-06 2024-09-16T00:08:53.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620378+0000", 2024-09-16T00:08:53.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.204e-05 2024-09-16T00:08:53.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620385+0000", 2024-09-16T00:08:53.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.413e-06 2024-09-16T00:08:53.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620455+0000", 2024-09-16T00:08:53.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9886000000000001e-05 2024-09-16T00:08:53.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620463+0000", 2024-09-16T00:08:53.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1559999999999995e-06 2024-09-16T00:08:53.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620569+0000", 2024-09-16T00:08:53.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001054 2024-09-16T00:08:53.286 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.286 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.286 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.287 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:18 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:53.287 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:53.287 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.591642711999995, 2024-09-16T00:08:53.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021768100000000001, 2024-09-16T00:08:53.287 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.287 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.287 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:53.287 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:53.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620746+0000", 2024-09-16T00:08:53.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620747+0000", 2024-09-16T00:08:53.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.02e-07 2024-09-16T00:08:53.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620752+0000", 2024-09-16T00:08:53.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4439999999999999e-06 2024-09-16T00:08:53.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620765+0000", 2024-09-16T00:08:53.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2714000000000001e-05 2024-09-16T00:08:53.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:08:53.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1160000000000004e-06 2024-09-16T00:08:53.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620855+0000", 2024-09-16T00:08:53.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3374999999999994e-05 2024-09-16T00:08:53.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620864+0000", 2024-09-16T00:08:53.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4320000000000005e-06 2024-09-16T00:08:53.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620964+0000", 2024-09-16T00:08:53.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9698000000000007e-05 2024-09-16T00:08:53.291 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.291 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.291 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.291 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:19 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:53.291 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:53.291 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.591581168000005, 2024-09-16T00:08:53.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00033551400000000003, 2024-09-16T00:08:53.291 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.292 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.292 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:53.292 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:53.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:53.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620808+0000", 2024-09-16T00:08:53.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0800000000000005e-07 2024-09-16T00:08:53.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620812+0000", 2024-09-16T00:08:53.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.427e-06 2024-09-16T00:08:53.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620820+0000", 2024-09-16T00:08:53.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6649999999999992e-06 2024-09-16T00:08:53.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620828+0000", 2024-09-16T00:08:53.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0010000000000001e-06 2024-09-16T00:08:53.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620995+0000", 2024-09-16T00:08:53.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016709400000000001 2024-09-16T00:08:53.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621005+0000", 2024-09-16T00:08:53.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7170000000000003e-06 2024-09-16T00:08:53.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621143+0000", 2024-09-16T00:08:53.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000138102 2024-09-16T00:08:53.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.295 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.295 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop_reply(mds.0.4:3 2.14 e22/19 ondisk, result = 0)", 2024-09-16T00:08:53.295 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:53.296 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.590403897000002, 2024-09-16T00:08:53.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00022247799999999999, 2024-09-16T00:08:53.296 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.296 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.296 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:53.296 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:53.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:53.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621985+0000", 2024-09-16T00:08:53.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3300000000000003e-07 2024-09-16T00:08:53.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621989+0000", 2024-09-16T00:08:53.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2969999999999999e-06 2024-09-16T00:08:53.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621997+0000", 2024-09-16T00:08:53.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8990000000000001e-06 2024-09-16T00:08:53.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622002+0000", 2024-09-16T00:08:53.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5439999999999998e-06 2024-09-16T00:08:53.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622137+0000", 2024-09-16T00:08:53.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000135103 2024-09-16T00:08:53.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622143+0000", 2024-09-16T00:08:53.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9200000000000001e-06 2024-09-16T00:08:53.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622207+0000", 2024-09-16T00:08:53.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4282000000000006e-05 2024-09-16T00:08:53.299 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.299 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.299 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.300 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing_reply(2.14 epoch 22/19 rep_tid 6 lcod 22'5)", 2024-09-16T00:08:53.300 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:53.300 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.590372560999995, 2024-09-16T00:08:53.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029688000000000002, 2024-09-16T00:08:53.300 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.300 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.300 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:08:53.300 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:53.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622016+0000", 2024-09-16T00:08:53.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622017+0000", 2024-09-16T00:08:53.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3200000000000001e-07 2024-09-16T00:08:53.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:08:53.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6240000000000002e-06 2024-09-16T00:08:53.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622026+0000", 2024-09-16T00:08:53.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.708e-06 2024-09-16T00:08:53.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622032+0000", 2024-09-16T00:08:53.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4699999999999999e-06 2024-09-16T00:08:53.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622230+0000", 2024-09-16T00:08:53.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019783800000000001 2024-09-16T00:08:53.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622313+0000", 2024-09-16T00:08:53.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2908e-05 2024-09-16T00:08:53.303 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.303 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.303 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.303 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:53.303 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:53.304 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 74.589927066000001, 2024-09-16T00:08:53.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044651079999999998, 2024-09-16T00:08:53.304 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.304 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.304 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.304 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:53.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:53.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622463+0000", 2024-09-16T00:08:53.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5349999999999999e-06 2024-09-16T00:08:53.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622470+0000", 2024-09-16T00:08:53.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7050000000000002e-06 2024-09-16T00:08:53.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622485+0000", 2024-09-16T00:08:53.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4978e-05 2024-09-16T00:08:53.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622493+0000", 2024-09-16T00:08:53.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2139999999999996e-06 2024-09-16T00:08:53.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622546+0000", 2024-09-16T00:08:53.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:08:53.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622595+0000", 2024-09-16T00:08:53.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9112e-05 2024-09-16T00:08:53.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:53.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622677+0000", 2024-09-16T00:08:53.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2332999999999999e-05 2024-09-16T00:08:53.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.623515+0000", 2024-09-16T00:08:53.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000837615 2024-09-16T00:08:53.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626867+0000", 2024-09-16T00:08:53.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033524819999999999 2024-09-16T00:08:53.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626902+0000", 2024-09-16T00:08:53.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4310000000000002e-05 2024-09-16T00:08:53.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626927+0000", 2024-09-16T00:08:53.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5128999999999999e-05 2024-09-16T00:08:53.309 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.309 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.309 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.309 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:39 2.6 e22/19 2:654134d2:::mds0_openfiles.0:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:53.309 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:53.309 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 69.571297788999999, 2024-09-16T00:08:53.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082120900000000004, 2024-09-16T00:08:53.309 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.309 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.310 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.310 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:53.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641091+0000", 2024-09-16T00:08:53.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641092+0000", 2024-09-16T00:08:53.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.141e-06 2024-09-16T00:08:53.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641102+0000", 2024-09-16T00:08:53.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5990000000000008e-06 2024-09-16T00:08:53.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641116+0000", 2024-09-16T00:08:53.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4667999999999999e-05 2024-09-16T00:08:53.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641125+0000", 2024-09-16T00:08:53.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0849999999999997e-06 2024-09-16T00:08:53.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641188+0000", 2024-09-16T00:08:53.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3119000000000003e-05 2024-09-16T00:08:53.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641209+0000", 2024-09-16T00:08:53.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0897e-05 2024-09-16T00:08:53.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641874+0000", 2024-09-16T00:08:53.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00066517200000000001 2024-09-16T00:08:53.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:43.641912+0000", 2024-09-16T00:08:53.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8528000000000001e-05 2024-09-16T00:08:53.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.313 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.314 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:40 2.14 2:2e2fa760:::200.00000001:head [write 940~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:53.314 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:53.314 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 62.373061673000002, 2024-09-16T00:08:53.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016643039999999999, 2024-09-16T00:08:53.314 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.314 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.314 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.314 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:53.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:53.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839329+0000", 2024-09-16T00:08:53.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2330000000000001e-06 2024-09-16T00:08:53.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839343+0000", 2024-09-16T00:08:53.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3634e-05 2024-09-16T00:08:53.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839362+0000", 2024-09-16T00:08:53.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9321999999999999e-05 2024-09-16T00:08:53.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839375+0000", 2024-09-16T00:08:53.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2945000000000001e-05 2024-09-16T00:08:53.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839462+0000", 2024-09-16T00:08:53.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6371000000000004e-05 2024-09-16T00:08:53.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839547+0000", 2024-09-16T00:08:53.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4988999999999996e-05 2024-09-16T00:08:53.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:53.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839657+0000", 2024-09-16T00:08:53.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110501 2024-09-16T00:08:53.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840598+0000", 2024-09-16T00:08:53.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00094087100000000003 2024-09-16T00:08:53.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840918+0000", 2024-09-16T00:08:53.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031968799999999999 2024-09-16T00:08:53.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840955+0000", 2024-09-16T00:08:53.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7360999999999997e-05 2024-09-16T00:08:53.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840991+0000", 2024-09-16T00:08:53.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6389e-05 2024-09-16T00:08:53.319 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.319 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.319 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.319 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:41 2.14 2:2e2fa760:::200.00000001:head [write 1383~2831 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:53.320 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:53.320 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.099408228000001, 2024-09-16T00:08:53.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017075809999999999, 2024-09-16T00:08:53.320 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.320 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.320 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.320 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:53.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:53.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112983+0000", 2024-09-16T00:08:53.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2560000000000001e-06 2024-09-16T00:08:53.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112993+0000", 2024-09-16T00:08:53.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0423e-05 2024-09-16T00:08:53.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113010+0000", 2024-09-16T00:08:53.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7169000000000001e-05 2024-09-16T00:08:53.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113023+0000", 2024-09-16T00:08:53.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2111e-05 2024-09-16T00:08:53.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113099+0000", 2024-09-16T00:08:53.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6798000000000006e-05 2024-09-16T00:08:53.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113174+0000", 2024-09-16T00:08:53.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.47e-05 2024-09-16T00:08:53.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:53.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113297+0000", 2024-09-16T00:08:53.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012250799999999999 2024-09-16T00:08:53.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114284+0000", 2024-09-16T00:08:53.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00098720399999999999 2024-09-16T00:08:53.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114604+0000", 2024-09-16T00:08:53.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031988699999999998 2024-09-16T00:08:53.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114644+0000", 2024-09-16T00:08:53.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9919999999999997e-05 2024-09-16T00:08:53.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114688+0000", 2024-09-16T00:08:53.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4604999999999997e-05 2024-09-16T00:08:53.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.325 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.325 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.325 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:53.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.319+0000 7ffb0fd31640 1 -- 172.21.15.139:0/438708207 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7ffb08068f40 con 0x7ffb08068060 2024-09-16T00:08:53.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.320+0000 7f0a2f6ba640 1 -- 172.21.15.139:0/1390873008 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) -- 0x7f0a280c9270 con 0x7f0a28154aa0 2024-09-16T00:08:53.327 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:08:53.328 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:53.326+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:53.427 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:53.427+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:53.473 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:53.473 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:53.473 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:53.474 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:53.482 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:08:53.529 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:53.528+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:53.586 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:53.586 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:53.586 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:53.586 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:53.586 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:53.595 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:08:53.639 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:53.639+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:53.703 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:53.703 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:53.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:53.703 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:53.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.703 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:53.704 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:53.704 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 90.620332512000004, 2024-09-16T00:08:53.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:08:53.704 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.704 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.704 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.704 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.704 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.704 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.704 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:08:53.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.704 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:53.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:53.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:08:53.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:08:53.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:08:53.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:08:53.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:08:53.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:08:53.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:08:53.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:08:53.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:08:53.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:08:53.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:08:53.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:08:53.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:08:53.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:08:53.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:08:53.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:08:53.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:08:53.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:08:53.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:08:53.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:08:53.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:08:53.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:08:53.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:08:53.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:08:53.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:08:53.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:08:53.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:08:53.710 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.710 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.710 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.710 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:53.710 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:53.711 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.540800881999999, 2024-09-16T00:08:53.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093071190000000008, 2024-09-16T00:08:53.711 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.711 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.711 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.711 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 106 2024-09-16T00:08:53.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.711 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:53.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:53.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160362+0000", 2024-09-16T00:08:53.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:53.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160438+0000", 2024-09-16T00:08:53.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6711000000000005e-05 2024-09-16T00:08:53.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160444+0000", 2024-09-16T00:08:53.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0530000000000003e-06 2024-09-16T00:08:53.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160449+0000", 2024-09-16T00:08:53.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2189999999999998e-06 2024-09-16T00:08:53.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161270+0000", 2024-09-16T00:08:53.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082099199999999999 2024-09-16T00:08:53.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161348+0000", 2024-09-16T00:08:53.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8361000000000004e-05 2024-09-16T00:08:53.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161472+0000", 2024-09-16T00:08:53.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123523 2024-09-16T00:08:53.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163461+0000", 2024-09-16T00:08:53.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019891560000000002 2024-09-16T00:08:53.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169418+0000", 2024-09-16T00:08:53.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059568490000000002 2024-09-16T00:08:53.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169570+0000", 2024-09-16T00:08:53.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152461 2024-09-16T00:08:53.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169668+0000", 2024-09-16T00:08:53.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T00:08:53.717 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.717 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.717 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.717 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:53.717 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:53.718 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.540705110999994, 2024-09-16T00:08:53.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092931679999999992, 2024-09-16T00:08:53.718 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.718 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.719 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.719 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.719 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.719 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.719 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107 2024-09-16T00:08:53.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.719 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:53.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:53.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:53.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0699999999999998e-07 2024-09-16T00:08:53.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160459+0000", 2024-09-16T00:08:53.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0370000000000001e-06 2024-09-16T00:08:53.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160463+0000", 2024-09-16T00:08:53.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999999e-06 2024-09-16T00:08:53.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160466+0000", 2024-09-16T00:08:53.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.644e-06 2024-09-16T00:08:53.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161873+0000", 2024-09-16T00:08:53.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014074840000000001 2024-09-16T00:08:53.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161949+0000", 2024-09-16T00:08:53.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5749999999999998e-05 2024-09-16T00:08:53.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162069+0000", 2024-09-16T00:08:53.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120257 2024-09-16T00:08:53.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163506+0000", 2024-09-16T00:08:53.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001436344 2024-09-16T00:08:53.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169704+0000", 2024-09-16T00:08:53.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061982829999999997 2024-09-16T00:08:53.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169730+0000", 2024-09-16T00:08:53.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6060000000000001e-05 2024-09-16T00:08:53.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169750+0000", 2024-09-16T00:08:53.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0092e-05 2024-09-16T00:08:53.724 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.724 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.724 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.725 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:53.725 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:53.725 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.540595799000002, 2024-09-16T00:08:53.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092280039999999997, 2024-09-16T00:08:53.725 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.725 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.725 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.725 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.725 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.725 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.725 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 108 2024-09-16T00:08:53.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.726 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:53.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:53.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160567+0000", 2024-09-16T00:08:53.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7300000000000001e-07 2024-09-16T00:08:53.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160569+0000", 2024-09-16T00:08:53.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5299999999999999e-06 2024-09-16T00:08:53.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160574+0000", 2024-09-16T00:08:53.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-16T00:08:53.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160577+0000", 2024-09-16T00:08:53.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.01e-06 2024-09-16T00:08:53.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162518+0000", 2024-09-16T00:08:53.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001940703 2024-09-16T00:08:53.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162571+0000", 2024-09-16T00:08:53.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2670000000000002e-05 2024-09-16T00:08:53.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162647+0000", 2024-09-16T00:08:53.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6778000000000002e-05 2024-09-16T00:08:53.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163544+0000", 2024-09-16T00:08:53.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089702599999999999 2024-09-16T00:08:53.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169754+0000", 2024-09-16T00:08:53.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062091739999999996 2024-09-16T00:08:53.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169776+0000", 2024-09-16T00:08:53.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2391999999999998e-05 2024-09-16T00:08:53.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169794+0000", 2024-09-16T00:08:53.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8318999999999999e-05 2024-09-16T00:08:53.731 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.731 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.731 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.731 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:53.731 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:53.731 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.540547248999999, 2024-09-16T00:08:53.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092220679999999999, 2024-09-16T00:08:53.731 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.732 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.732 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.732 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.732 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 109 2024-09-16T00:08:53.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.732 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:53.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:53.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:53.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9900000000000001e-07 2024-09-16T00:08:53.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160707+0000", 2024-09-16T00:08:53.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1621999999999998e-05 2024-09-16T00:08:53.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160713+0000", 2024-09-16T00:08:53.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.596e-06 2024-09-16T00:08:53.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160717+0000", 2024-09-16T00:08:53.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5829999999999998e-06 2024-09-16T00:08:53.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162869+0000", 2024-09-16T00:08:53.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021516310000000002 2024-09-16T00:08:53.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162946+0000", 2024-09-16T00:08:53.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7696999999999998e-05 2024-09-16T00:08:53.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163065+0000", 2024-09-16T00:08:53.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118581 2024-09-16T00:08:53.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164431+0000", 2024-09-16T00:08:53.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001366003 2024-09-16T00:08:53.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169797+0000", 2024-09-16T00:08:53.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005366244 2024-09-16T00:08:53.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169819+0000", 2024-09-16T00:08:53.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2036999999999999e-05 2024-09-16T00:08:53.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169837+0000", 2024-09-16T00:08:53.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7575e-05 2024-09-16T00:08:53.737 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.737 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.737 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.738 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:53.738 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:53.738 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.540437900000001, 2024-09-16T00:08:53.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092871919999999997, 2024-09-16T00:08:53.738 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.738 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.738 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.738 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.738 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 110 2024-09-16T00:08:53.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.738 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:53.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:53.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:53.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:08:53.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160728+0000", 2024-09-16T00:08:53.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0620000000000001e-06 2024-09-16T00:08:53.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160731+0000", 2024-09-16T00:08:53.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6500000000000002e-06 2024-09-16T00:08:53.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160734+0000", 2024-09-16T00:08:53.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.762e-06 2024-09-16T00:08:53.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163580+0000", 2024-09-16T00:08:53.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028464549999999999 2024-09-16T00:08:53.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163657+0000", 2024-09-16T00:08:53.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6695999999999999e-05 2024-09-16T00:08:53.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163798+0000", 2024-09-16T00:08:53.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014055 2024-09-16T00:08:53.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165483+0000", 2024-09-16T00:08:53.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016853090000000001 2024-09-16T00:08:53.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169970+0000", 2024-09-16T00:08:53.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00448709 2024-09-16T00:08:53.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169994+0000", 2024-09-16T00:08:53.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3575999999999999e-05 2024-09-16T00:08:53.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170011+0000", 2024-09-16T00:08:53.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7702000000000001e-05 2024-09-16T00:08:53.743 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.743 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.743 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.744 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:53.744 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:53.744 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.540424231000003, 2024-09-16T00:08:53.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093174859999999998, 2024-09-16T00:08:53.744 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.744 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.744 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.744 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.744 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.744 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.744 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 111 2024-09-16T00:08:53.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.744 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:53.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:53.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:53.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1e-07 2024-09-16T00:08:53.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160741+0000", 2024-09-16T00:08:53.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6110000000000001e-06 2024-09-16T00:08:53.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160744+0000", 2024-09-16T00:08:53.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3299999999999999e-06 2024-09-16T00:08:53.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160746+0000", 2024-09-16T00:08:53.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5469999999999999e-06 2024-09-16T00:08:53.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164069+0000", 2024-09-16T00:08:53.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033222709999999999 2024-09-16T00:08:53.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164113+0000", 2024-09-16T00:08:53.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4212999999999998e-05 2024-09-16T00:08:53.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164187+0000", 2024-09-16T00:08:53.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4136999999999998e-05 2024-09-16T00:08:53.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166429+0000", 2024-09-16T00:08:53.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022423180000000001 2024-09-16T00:08:53.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170016+0000", 2024-09-16T00:08:53.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035869449999999998 2024-09-16T00:08:53.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170038+0000", 2024-09-16T00:08:53.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2053e-05 2024-09-16T00:08:53.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170055+0000", 2024-09-16T00:08:53.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6850999999999998e-05 2024-09-16T00:08:53.750 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.750 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.751 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.751 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:53.751 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:53.751 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.540410515999994, 2024-09-16T00:08:53.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093480279999999995, 2024-09-16T00:08:53.751 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.751 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.751 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.752 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.752 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 112 2024-09-16T00:08:53.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.752 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:53.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:53.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160752+0000", 2024-09-16T00:08:53.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2700000000000001e-07 2024-09-16T00:08:53.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160808+0000", 2024-09-16T00:08:53.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5797999999999997e-05 2024-09-16T00:08:53.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160814+0000", 2024-09-16T00:08:53.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5980000000000002e-06 2024-09-16T00:08:53.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160819+0000", 2024-09-16T00:08:53.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8960000000000002e-06 2024-09-16T00:08:53.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164468+0000", 2024-09-16T00:08:53.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003649476 2024-09-16T00:08:53.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164525+0000", 2024-09-16T00:08:53.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6227999999999998e-05 2024-09-16T00:08:53.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164595+0000", 2024-09-16T00:08:53.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759000000000005e-05 2024-09-16T00:08:53.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167237+0000", 2024-09-16T00:08:53.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026410470000000001 2024-09-16T00:08:53.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170060+0000", 2024-09-16T00:08:53.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028234969999999999 2024-09-16T00:08:53.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170081+0000", 2024-09-16T00:08:53.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1242000000000001e-05 2024-09-16T00:08:53.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170100+0000", 2024-09-16T00:08:53.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8260000000000001e-05 2024-09-16T00:08:53.757 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.757 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.758 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.758 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:53.758 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:53.758 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.540333421, 2024-09-16T00:08:53.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093139409999999992, 2024-09-16T00:08:53.758 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.758 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.758 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.759 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 113 2024-09-16T00:08:53.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.759 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:53.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:53.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:53.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-07 2024-09-16T00:08:53.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160833+0000", 2024-09-16T00:08:53.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.557e-06 2024-09-16T00:08:53.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160839+0000", 2024-09-16T00:08:53.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3829999999999997e-06 2024-09-16T00:08:53.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160844+0000", 2024-09-16T00:08:53.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6909999999999996e-06 2024-09-16T00:08:53.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165008+0000", 2024-09-16T00:08:53.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041642129999999999 2024-09-16T00:08:53.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165085+0000", 2024-09-16T00:08:53.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6651999999999997e-05 2024-09-16T00:08:53.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165205+0000", 2024-09-16T00:08:53.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012045900000000001 2024-09-16T00:08:53.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167283+0000", 2024-09-16T00:08:53.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020782909999999999 2024-09-16T00:08:53.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170104+0000", 2024-09-16T00:08:53.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028207950000000001 2024-09-16T00:08:53.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170125+0000", 2024-09-16T00:08:53.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0852e-05 2024-09-16T00:08:53.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170143+0000", 2024-09-16T00:08:53.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7467000000000001e-05 2024-09-16T00:08:53.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.764 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.764 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:53.764 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:53.764 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.540286938999998, 2024-09-16T00:08:53.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093126960000000005, 2024-09-16T00:08:53.764 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.765 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.765 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.765 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-16T00:08:53.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.765 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:53.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:53.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160876+0000", 2024-09-16T00:08:53.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5700000000000002e-07 2024-09-16T00:08:53.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160880+0000", 2024-09-16T00:08:53.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5539999999999999e-06 2024-09-16T00:08:53.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160888+0000", 2024-09-16T00:08:53.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.157e-06 2024-09-16T00:08:53.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160894+0000", 2024-09-16T00:08:53.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7690000000000002e-06 2024-09-16T00:08:53.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165519+0000", 2024-09-16T00:08:53.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046249539999999997 2024-09-16T00:08:53.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165593+0000", 2024-09-16T00:08:53.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4039e-05 2024-09-16T00:08:53.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165715+0000", 2024-09-16T00:08:53.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012131899999999999 2024-09-16T00:08:53.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167309+0000", 2024-09-16T00:08:53.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001594011 2024-09-16T00:08:53.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170147+0000", 2024-09-16T00:08:53.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028387130000000001 2024-09-16T00:08:53.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170170+0000", 2024-09-16T00:08:53.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2517000000000001e-05 2024-09-16T00:08:53.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170188+0000", 2024-09-16T00:08:53.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8006e-05 2024-09-16T00:08:53.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.770 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.770 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.770 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:53.770 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:53.770 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.540245911, 2024-09-16T00:08:53.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093140040000000007, 2024-09-16T00:08:53.770 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.770 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.771 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.771 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.771 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-16T00:08:53.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.771 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:53.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:53.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160917+0000", 2024-09-16T00:08:53.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7599999999999996e-07 2024-09-16T00:08:53.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160992+0000", 2024-09-16T00:08:53.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4760999999999998e-05 2024-09-16T00:08:53.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161000+0000", 2024-09-16T00:08:53.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9829999999999996e-06 2024-09-16T00:08:53.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161007+0000", 2024-09-16T00:08:53.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4909999999999999e-06 2024-09-16T00:08:53.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165983+0000", 2024-09-16T00:08:53.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049755399999999997 2024-09-16T00:08:53.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166027+0000", 2024-09-16T00:08:53.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4109000000000001e-05 2024-09-16T00:08:53.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166100+0000", 2024-09-16T00:08:53.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3360000000000002e-05 2024-09-16T00:08:53.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169698+0000", 2024-09-16T00:08:53.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597795 2024-09-16T00:08:53.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170192+0000", 2024-09-16T00:08:53.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049447300000000005 2024-09-16T00:08:53.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170214+0000", 2024-09-16T00:08:53.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1546e-05 2024-09-16T00:08:53.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170230+0000", 2024-09-16T00:08:53.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6169999999999999e-05 2024-09-16T00:08:53.776 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.776 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.776 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.776 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:53.776 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:53.776 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.540136181999998, 2024-09-16T00:08:53.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092457029999999992, 2024-09-16T00:08:53.776 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.776 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.776 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.777 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.777 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.777 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.777 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-16T00:08:53.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.777 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:53.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:53.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:53.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7899999999999999e-07 2024-09-16T00:08:53.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161029+0000", 2024-09-16T00:08:53.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6630000000000002e-06 2024-09-16T00:08:53.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161036+0000", 2024-09-16T00:08:53.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T00:08:53.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161041+0000", 2024-09-16T00:08:53.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:08:53.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166468+0000", 2024-09-16T00:08:53.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054266050000000001 2024-09-16T00:08:53.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166511+0000", 2024-09-16T00:08:53.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3869999999999998e-05 2024-09-16T00:08:53.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166606+0000", 2024-09-16T00:08:53.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4437000000000004e-05 2024-09-16T00:08:53.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169955+0000", 2024-09-16T00:08:53.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033494810000000001 2024-09-16T00:08:53.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170234+0000", 2024-09-16T00:08:53.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027900899999999998 2024-09-16T00:08:53.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170255+0000", 2024-09-16T00:08:53.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0537999999999999e-05 2024-09-16T00:08:53.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170272+0000", 2024-09-16T00:08:53.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6611000000000001e-05 2024-09-16T00:08:53.783 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.783 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.783 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.783 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:53.783 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:53.783 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.540111037000003, 2024-09-16T00:08:53.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093511010000000006, 2024-09-16T00:08:53.783 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.783 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.783 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.783 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.784 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.784 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.784 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-16T00:08:53.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.784 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:53.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:53.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:53.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.63e-07 2024-09-16T00:08:53.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161055+0000", 2024-09-16T00:08:53.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6770000000000001e-06 2024-09-16T00:08:53.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161063+0000", 2024-09-16T00:08:53.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6469999999999998e-06 2024-09-16T00:08:53.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161068+0000", 2024-09-16T00:08:53.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.746e-06 2024-09-16T00:08:53.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166817+0000", 2024-09-16T00:08:53.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057494959999999998 2024-09-16T00:08:53.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166893+0000", 2024-09-16T00:08:53.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6210000000000004e-05 2024-09-16T00:08:53.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167019+0000", 2024-09-16T00:08:53.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125585 2024-09-16T00:08:53.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170275+0000", 2024-09-16T00:08:53.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032563620000000001 2024-09-16T00:08:53.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170309+0000", 2024-09-16T00:08:53.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3865999999999997e-05 2024-09-16T00:08:53.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170362+0000", 2024-09-16T00:08:53.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3094999999999999e-05 2024-09-16T00:08:53.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170402+0000", 2024-09-16T00:08:53.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9953999999999997e-05 2024-09-16T00:08:53.789 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.789 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.789 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.789 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:53.789 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:53.789 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.091378951999999, 2024-09-16T00:08:53.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097364389999999995, 2024-09-16T00:08:53.789 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.789 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.790 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.790 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.790 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.790 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.790 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138 2024-09-16T00:08:53.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.790 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:53.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:53.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609784+0000", 2024-09-16T00:08:53.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T00:08:53.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609787+0000", 2024-09-16T00:08:53.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280000000000001e-06 2024-09-16T00:08:53.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609794+0000", 2024-09-16T00:08:53.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6409999999999996e-06 2024-09-16T00:08:53.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609799+0000", 2024-09-16T00:08:53.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9359999999999998e-06 2024-09-16T00:08:53.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615759+0000", 2024-09-16T00:08:53.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059599910000000004 2024-09-16T00:08:53.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615806+0000", 2024-09-16T00:08:53.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7290000000000003e-05 2024-09-16T00:08:53.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615877+0000", 2024-09-16T00:08:53.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1000000000000005e-05 2024-09-16T00:08:53.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618961+0000", 2024-09-16T00:08:53.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030841390000000001 2024-09-16T00:08:53.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619436+0000", 2024-09-16T00:08:53.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047452799999999999 2024-09-16T00:08:53.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619482+0000", 2024-09-16T00:08:53.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5689000000000003e-05 2024-09-16T00:08:53.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619519+0000", 2024-09-16T00:08:53.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7938000000000002e-05 2024-09-16T00:08:53.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.795 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.795 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.795 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:53.795 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:53.795 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.091355238000006, 2024-09-16T00:08:53.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011812954000000001, 2024-09-16T00:08:53.795 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.796 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.796 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.796 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.796 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139 2024-09-16T00:08:53.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.796 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:53.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:53.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:53.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999998e-07 2024-09-16T00:08:53.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609810+0000", 2024-09-16T00:08:53.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2100000000000002e-06 2024-09-16T00:08:53.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609817+0000", 2024-09-16T00:08:53.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.28e-06 2024-09-16T00:08:53.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609825+0000", 2024-09-16T00:08:53.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:53.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616116+0000", 2024-09-16T00:08:53.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006290543 2024-09-16T00:08:53.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616187+0000", 2024-09-16T00:08:53.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1830999999999997e-05 2024-09-16T00:08:53.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616298+0000", 2024-09-16T00:08:53.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011044699999999999 2024-09-16T00:08:53.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619559+0000", 2024-09-16T00:08:53.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032614710000000002 2024-09-16T00:08:53.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621562+0000", 2024-09-16T00:08:53.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020028680000000001 2024-09-16T00:08:53.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621597+0000", 2024-09-16T00:08:53.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4625000000000001e-05 2024-09-16T00:08:53.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621620+0000", 2024-09-16T00:08:53.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2864e-05 2024-09-16T00:08:53.801 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.801 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.801 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.802 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:53.802 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:53.802 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.091327441999994, 2024-09-16T00:08:53.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011828211999999999, 2024-09-16T00:08:53.802 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.802 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.802 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.802 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.803 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 140 2024-09-16T00:08:53.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.803 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:53.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:53.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:53.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T00:08:53.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:53.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961000000000003e-05 2024-09-16T00:08:53.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609892+0000", 2024-09-16T00:08:53.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0516000000000001e-05 2024-09-16T00:08:53.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609898+0000", 2024-09-16T00:08:53.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0449999999999997e-06 2024-09-16T00:08:53.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616797+0000", 2024-09-16T00:08:53.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068998840000000002 2024-09-16T00:08:53.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616844+0000", 2024-09-16T00:08:53.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6501000000000001e-05 2024-09-16T00:08:53.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616935+0000", 2024-09-16T00:08:53.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1222000000000002e-05 2024-09-16T00:08:53.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619808+0000", 2024-09-16T00:08:53.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873198 2024-09-16T00:08:53.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621624+0000", 2024-09-16T00:08:53.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001815933 2024-09-16T00:08:53.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621646+0000", 2024-09-16T00:08:53.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1622999999999999e-05 2024-09-16T00:08:53.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621663+0000", 2024-09-16T00:08:53.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6854000000000002e-05 2024-09-16T00:08:53.808 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.808 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.809 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.809 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:53.809 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:53.809 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 89.091241632999996, 2024-09-16T00:08:53.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014331501999999999, 2024-09-16T00:08:53.809 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.809 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.809 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.809 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.809 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.809 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.810 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-16T00:08:53.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.810 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:53.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:53.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609921+0000", 2024-09-16T00:08:53.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9599999999999999e-07 2024-09-16T00:08:53.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609925+0000", 2024-09-16T00:08:53.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T00:08:53.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609932+0000", 2024-09-16T00:08:53.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8240000000000002e-06 2024-09-16T00:08:53.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609937+0000", 2024-09-16T00:08:53.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1070000000000004e-06 2024-09-16T00:08:53.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618987+0000", 2024-09-16T00:08:53.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090502719999999998 2024-09-16T00:08:53.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619064+0000", 2024-09-16T00:08:53.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6868999999999999e-05 2024-09-16T00:08:53.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619183+0000", 2024-09-16T00:08:53.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011961 2024-09-16T00:08:53.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.620052+0000", 2024-09-16T00:08:53.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086822799999999997 2024-09-16T00:08:53.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624197+0000", 2024-09-16T00:08:53.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041454930000000001 2024-09-16T00:08:53.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624229+0000", 2024-09-16T00:08:53.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1758999999999999e-05 2024-09-16T00:08:53.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624252+0000", 2024-09-16T00:08:53.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:08:53.814 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.814 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.815 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.815 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:53.815 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:53.815 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.618475825000004, 2024-09-16T00:08:53.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010093276999999999, 2024-09-16T00:08:53.815 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.815 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.815 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.815 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.815 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.815 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.815 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 157 2024-09-16T00:08:53.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.816 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:53.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:53.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:53.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4699999999999998e-07 2024-09-16T00:08:53.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082688+0000", 2024-09-16T00:08:53.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0209999999999998e-06 2024-09-16T00:08:53.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082692+0000", 2024-09-16T00:08:53.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6890000000000002e-06 2024-09-16T00:08:53.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082695+0000", 2024-09-16T00:08:53.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5339999999999998e-06 2024-09-16T00:08:53.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085303+0000", 2024-09-16T00:08:53.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026084789999999999 2024-09-16T00:08:53.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085374+0000", 2024-09-16T00:08:53.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1317999999999997e-05 2024-09-16T00:08:53.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085506+0000", 2024-09-16T00:08:53.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013118 2024-09-16T00:08:53.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086928+0000", 2024-09-16T00:08:53.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014224559999999999 2024-09-16T00:08:53.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092665+0000", 2024-09-16T00:08:53.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057369830000000002 2024-09-16T00:08:53.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092727+0000", 2024-09-16T00:08:53.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1810000000000006e-05 2024-09-16T00:08:53.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092779+0000", 2024-09-16T00:08:53.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2559999999999998e-05 2024-09-16T00:08:53.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.821 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.822 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.822 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:53.822 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:53.822 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.618463657000007, 2024-09-16T00:08:53.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010176388, 2024-09-16T00:08:53.822 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.822 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.822 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.823 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.823 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 158 2024-09-16T00:08:53.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.823 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:53.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:53.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082699+0000", 2024-09-16T00:08:53.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.11e-07 2024-09-16T00:08:53.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082742+0000", 2024-09-16T00:08:53.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3699000000000003e-05 2024-09-16T00:08:53.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082748+0000", 2024-09-16T00:08:53.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8690000000000001e-06 2024-09-16T00:08:53.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082751+0000", 2024-09-16T00:08:53.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.942e-06 2024-09-16T00:08:53.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085881+0000", 2024-09-16T00:08:53.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031303450000000001 2024-09-16T00:08:53.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085959+0000", 2024-09-16T00:08:53.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7874999999999996e-05 2024-09-16T00:08:53.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086086+0000", 2024-09-16T00:08:53.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012685499999999999 2024-09-16T00:08:53.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087127+0000", 2024-09-16T00:08:53.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001041298 2024-09-16T00:08:53.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092790+0000", 2024-09-16T00:08:53.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056628 2024-09-16T00:08:53.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092832+0000", 2024-09-16T00:08:53.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2227000000000001e-05 2024-09-16T00:08:53.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092875+0000", 2024-09-16T00:08:53.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2267e-05 2024-09-16T00:08:53.829 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.829 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.829 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.829 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:53.829 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:53.829 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 88.618405619000001, 2024-09-16T00:08:53.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010199547, 2024-09-16T00:08:53.830 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:53.830 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:53.830 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:53.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:53.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:53.830 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:53.830 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159 2024-09-16T00:08:53.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.830 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:53.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:53.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:53.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:53.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:53.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:53.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:53.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082757+0000", 2024-09-16T00:08:53.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2500000000000001e-07 2024-09-16T00:08:53.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:53.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082759+0000", 2024-09-16T00:08:53.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-16T00:08:53.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:53.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082763+0000", 2024-09-16T00:08:53.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4889999999999997e-06 2024-09-16T00:08:53.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:53.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082766+0000", 2024-09-16T00:08:53.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:08:53.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:53.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086571+0000", 2024-09-16T00:08:53.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038053269999999998 2024-09-16T00:08:53.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:53.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086619+0000", 2024-09-16T00:08:53.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7924999999999999e-05 2024-09-16T00:08:53.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:53.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086693+0000", 2024-09-16T00:08:53.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000005e-05 2024-09-16T00:08:53.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:53.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087383+0000", 2024-09-16T00:08:53.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069008299999999997 2024-09-16T00:08:53.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:53.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092882+0000", 2024-09-16T00:08:53.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054987450000000002 2024-09-16T00:08:53.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:53.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092921+0000", 2024-09-16T00:08:53.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9388999999999998e-05 2024-09-16T00:08:53.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:53.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:53.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:53.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092956+0000", 2024-09-16T00:08:53.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4647999999999998e-05 2024-09-16T00:08:53.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.835 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:53.835 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:53.835 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:53.836 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:08:53.837 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:53.836+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:53.838 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:08:53.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.936+0000 7f374232a640 1 Processor -- start 2024-09-16T00:08:53.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.936+0000 7f374232a640 1 -- start start 2024-09-16T00:08:53.938 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:53.937+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:53.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.937+0000 7f374232a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0c6900 0x7f373c073e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.937+0000 7f374232a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f373c0c6ce0 0x7f373c152740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.937+0000 7f374232a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f373c06e010 con 0x7f373c06d2a0 2024-09-16T00:08:53.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.937+0000 7f374232a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f373c06bce0 con 0x7f373c0c6ce0 2024-09-16T00:08:53.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.937+0000 7f374232a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f373c15cde0 con 0x7f373c0c6900 2024-09-16T00:08:53.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.938+0000 7f373bfff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0c6900 0x7f373c073e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.938+0000 7f37408c7640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f373c0c6ce0 0x7f373c152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.938+0000 7f373bfff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0c6900 0x7f373c073e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:51424/0 (socket says 172.21.15.139:51424) 2024-09-16T00:08:53.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.938+0000 7f373bfff640 1 -- 172.21.15.139:0/867149270 learned_addr learned my addr 172.21.15.139:0/867149270 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:53.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.938+0000 7f373b7fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f373c06d2a0 0x7f373c15c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:33234/0 (socket says 172.21.15.139:33234) 2024-09-16T00:08:53.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.938+0000 7f373bfff640 1 -- 172.21.15.139:0/867149270 >> v2:172.21.15.164:3300/0 conn(0x7f373c0c6ce0 msgr2=0x7f373c152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:53.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.938+0000 7f373bfff640 1 --2- 172.21.15.139:0/867149270 >> v2:172.21.15.164:3300/0 conn(0x7f373c0c6ce0 0x7f373c152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.938+0000 7f373bfff640 1 -- 172.21.15.139:0/867149270 >> v1:172.21.15.164:6789/0 conn(0x7f373c06d2a0 legacy=0x7f373c15c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.938+0000 7f373bfff640 1 -- 172.21.15.139:0/867149270 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373c06d770 con 0x7f373c0c6900 2024-09-16T00:08:53.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.938+0000 7f37408c7640 1 --2- 172.21.15.139:0/867149270 >> v2:172.21.15.164:3300/0 conn(0x7f373c0c6ce0 0x7f373c152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:53.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.939+0000 7f373bfff640 1 --2- 172.21.15.139:0/867149270 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0c6900 0x7f373c073e20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f372c009870 tx=0x7f372c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8529cebae2e28c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:53.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.939+0000 7f373affd640 1 -- 172.21.15.139:0/867149270 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c037070 con 0x7f373c0c6900 2024-09-16T00:08:53.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.939+0000 7f373affd640 1 -- 172.21.15.139:0/867149270 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f372c032440 con 0x7f373c0c6900 2024-09-16T00:08:53.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.939+0000 7f373affd640 1 -- 172.21.15.139:0/867149270 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c032810 con 0x7f373c0c6900 2024-09-16T00:08:53.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.940+0000 7f374232a640 1 -- 172.21.15.139:0/867149270 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0c6900 msgr2=0x7f373c073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.940+0000 7f374232a640 1 --2- 172.21.15.139:0/867149270 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0c6900 0x7f373c073e20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f372c009870 tx=0x7f372c02a270 comp rx=0 tx=0).stop 2024-09-16T00:08:53.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.940+0000 7f374232a640 1 -- 172.21.15.139:0/867149270 shutdown_connections 2024-09-16T00:08:53.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.940+0000 7f374232a640 1 --2- 172.21.15.139:0/867149270 >> v2:172.21.15.164:3300/0 conn(0x7f373c0c6ce0 0x7f373c152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.940+0000 7f374232a640 1 --2- 172.21.15.139:0/867149270 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0c6900 0x7f373c073e20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.940+0000 7f374232a640 1 -- 172.21.15.139:0/867149270 >> 172.21.15.139:0/867149270 conn(0x7f373c070280 msgr2=0x7f373c070690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:53.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.940+0000 7f374232a640 1 -- 172.21.15.139:0/867149270 shutdown_connections 2024-09-16T00:08:53.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.940+0000 7f374232a640 1 -- 172.21.15.139:0/867149270 wait complete. 2024-09-16T00:08:53.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.941+0000 7f374232a640 1 Processor -- start 2024-09-16T00:08:53.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.941+0000 7f374232a640 1 -- start start 2024-09-16T00:08:53.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f374232a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f373c06d2a0 0x7f373c15a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f374232a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373c0c6900 0x7f373c155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f374232a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0c6ce0 0x7f373c1558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f374232a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f373c06dd30 con 0x7f373c06d2a0 2024-09-16T00:08:53.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f374232a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f373c15c240 con 0x7f373c0c6900 2024-09-16T00:08:53.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f374232a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f373c15bed0 con 0x7f373c0c6ce0 2024-09-16T00:08:53.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f373bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f373c06d2a0 0x7f373c15a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f373bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f373c06d2a0 0x7f373c15a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59356/0 (socket says 172.21.15.139:59356) 2024-09-16T00:08:53.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f373bfff640 1 -- 172.21.15.139:0/1656807802 learned_addr learned my addr 172.21.15.139:0/1656807802 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:53.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f373b7fe640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373c0c6900 0x7f373c155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f37408c7640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0c6ce0 0x7f373c1558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f373b7fe640 1 -- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0c6ce0 msgr2=0x7f373c1558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f373b7fe640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0c6ce0 0x7f373c1558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f373b7fe640 1 -- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f373c06d2a0 msgr2=0x7f373c15a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f373b7fe640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f373c06d2a0 0x7f373c15a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f373b7fe640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373c1568b0 con 0x7f373c0c6900 2024-09-16T00:08:53.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f373bfff640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f373c06d2a0 0x7f373c15a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:53.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.942+0000 7f37408c7640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f373c0c6ce0 0x7f373c1558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:53.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.943+0000 7f373b7fe640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373c0c6900 0x7f373c155380 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3730000f60 tx=0x7f373000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:53.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.943+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37300074e0 con 0x7f373c0c6900 2024-09-16T00:08:53.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.943+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37300047e0 con 0x7f373c0c6900 2024-09-16T00:08:53.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.943+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730004bd0 con 0x7f373c0c6900 2024-09-16T00:08:53.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.943+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f373c0cd270 con 0x7f373c0c6900 2024-09-16T00:08:53.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.943+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f373c0cd830 con 0x7f373c0c6900 2024-09-16T00:08:53.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.944+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f373c0c7880 con 0x7f373c0c6900 2024-09-16T00:08:53.947 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:53.947 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:53.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.945+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f373001a020 con 0x7f373c0c6900 2024-09-16T00:08:53.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.945+0000 7f3738ff9640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f372806c2b0 0x7f372806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.945+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5886+0+0 (secure 0 0 0) 0x7f373008ab40 con 0x7f373c0c6900 2024-09-16T00:08:53.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.945+0000 7f373bfff640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f372806c2b0 0x7f372806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.946+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v54) ==== 54+0+16 (secure 0 0 0) 0x7f373008f1e0 con 0x7f373c0c6900 2024-09-16T00:08:53.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.946+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f373c1547e0 0x7f373c153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.946+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f373c14ff70 con 0x7f373c1547e0 2024-09-16T00:08:53.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.946+0000 7f373bfff640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f372806c2b0 0x7f372806e770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f372c0090b0 tx=0x7f372c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:53.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.946+0000 7f37408c7640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f373c1547e0 0x7f373c153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.946+0000 7f37408c7640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f373c1547e0 0x7f373c153210 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:53.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.947+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f373c14ff70 con 0x7f373c1547e0 2024-09-16T00:08:53.956 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:08:53.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.961+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f373c073080 con 0x7f373c1547e0 2024-09-16T00:08:53.965 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:53.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.963+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f373c073080 con 0x7f373c1547e0 2024-09-16T00:08:53.965 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:53.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.963+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f373c0cda00 0x7f373c153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.963+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f373c156500 con 0x7f373c0cda00 2024-09-16T00:08:53.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.964+0000 7f373bfff640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f373c0cda00 0x7f373c153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.964+0000 7f373bfff640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f373c0cda00 0x7f373c153c60 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:53.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.965+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f373c156500 con 0x7f373c0cda00 2024-09-16T00:08:53.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.969+0000 7ffafe7fc640 1 -- 172.21.15.139:0/438708207 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v55) ==== 87+0+0 (secure 0 0 0) 0x7ffb0405ca00 con 0x7ffb08068060 2024-09-16T00:08:53.969 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is set 2024-09-16T00:08:53.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.971+0000 7f0a1dffb640 1 -- 172.21.15.139:0/1390873008 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v55) ==== 113+0+0 (secure 0 0 0) 0x7f0a20010040 con 0x7f0a28154aa0 2024-09-16T00:08:53.972 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T00:08:53.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.972+0000 7ffb0fd31640 1 -- 172.21.15.139:0/438708207 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffaec06c2e0 msgr2=0x7ffaec06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.972+0000 7ffb0fd31640 1 --2- 172.21.15.139:0/438708207 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffaec06c2e0 0x7ffaec06e7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8034000 tx=0x7ffaf802e000 comp rx=0 tx=0).stop 2024-09-16T00:08:53.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.972+0000 7ffb0fd31640 1 -- 172.21.15.139:0/438708207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb08068060 msgr2=0x7ffb0814c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.972+0000 7ffb0fd31640 1 --2- 172.21.15.139:0/438708207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb08068060 0x7ffb0814c350 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400c460 tx=0x7ffb0400c920 comp rx=0 tx=0).stop 2024-09-16T00:08:53.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.973+0000 7ffb0fd31640 1 -- 172.21.15.139:0/438708207 shutdown_connections 2024-09-16T00:08:53.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.973+0000 7ffb0fd31640 1 --2- 172.21.15.139:0/438708207 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffaec06c2e0 0x7ffaec06e7a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.973+0000 7ffb0fd31640 1 --2- 172.21.15.139:0/438708207 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffb08154c20 0x7ffb080b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.973+0000 7ffb0fd31640 1 --2- 172.21.15.139:0/438708207 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffb080ccc90 0x7ffb080cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.973+0000 7ffb0fd31640 1 --2- 172.21.15.139:0/438708207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb08068060 0x7ffb0814c350 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.973+0000 7ffb0fd31640 1 -- 172.21.15.139:0/438708207 >> 172.21.15.139:0/438708207 conn(0x7ffb0806ca40 msgr2=0x7ffb080c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:53.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.973+0000 7ffb0fd31640 1 -- 172.21.15.139:0/438708207 shutdown_connections 2024-09-16T00:08:53.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.973+0000 7ffb0fd31640 1 -- 172.21.15.139:0/438708207 wait complete. 2024-09-16T00:08:53.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.975+0000 7f0a2f6ba640 1 -- 172.21.15.139:0/1390873008 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a0c06c2b0 msgr2=0x7f0a0c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.975+0000 7f0a2f6ba640 1 --2- 172.21.15.139:0/1390873008 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a0c06c2b0 0x7f0a0c06e770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009ae0 tx=0x7f0a24009190 comp rx=0 tx=0).stop 2024-09-16T00:08:53.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.975+0000 7f0a2f6ba640 1 -- 172.21.15.139:0/1390873008 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a28154aa0 msgr2=0x7f0a281576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:53.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.975+0000 7f0a2f6ba640 1 --2- 172.21.15.139:0/1390873008 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a28154aa0 0x7f0a281576e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009d90 tx=0x7f0a2000c490 comp rx=0 tx=0).stop 2024-09-16T00:08:53.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.975+0000 7f0a2f6ba640 1 -- 172.21.15.139:0/1390873008 shutdown_connections 2024-09-16T00:08:53.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.975+0000 7f0a2f6ba640 1 --2- 172.21.15.139:0/1390873008 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a0c06c2b0 0x7f0a0c06e770 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.975+0000 7f0a2f6ba640 1 --2- 172.21.15.139:0/1390873008 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a28154aa0 0x7f0a281576e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.975+0000 7f0a2f6ba640 1 --2- 172.21.15.139:0/1390873008 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a280c7aa0 0x7f0a28155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.975+0000 7f0a2f6ba640 1 --2- 172.21.15.139:0/1390873008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a280c6da0 0x7f0a281596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:53.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.975+0000 7f0a2f6ba640 1 -- 172.21.15.139:0/1390873008 >> 172.21.15.139:0/1390873008 conn(0x7f0a2806cc30 msgr2=0x7f0a280c5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:53.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.975+0000 7f0a2f6ba640 1 -- 172.21.15.139:0/1390873008 shutdown_connections 2024-09-16T00:08:53.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.975+0000 7f0a2f6ba640 1 -- 172.21.15.139:0/1390873008 wait complete. 2024-09-16T00:08:53.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.983+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f373c157af0 con 0x7f373c0cda00 2024-09-16T00:08:53.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.985+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f373c157af0 con 0x7f373c0cda00 2024-09-16T00:08:53.986 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:53.986 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:53.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.985+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f373c1f42f0 0x7f373c062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:53.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.985+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f373c1f46f0 con 0x7f373c1f42f0 2024-09-16T00:08:53.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.985+0000 7f373b7fe640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f373c1f42f0 0x7f373c062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:53.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.986+0000 7f373b7fe640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f373c1f42f0 0x7f373c062630 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:53.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:53.986+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f373c1f46f0 con 0x7f373c1f42f0 2024-09-16T00:08:53.994 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T00:08:54.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.001+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f373c062b70 con 0x7f373c1f42f0 2024-09-16T00:08:54.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.003+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f373c062b70 con 0x7f373c1f42f0 2024-09-16T00:08:54.004 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:54.005 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:54.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.003+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f373c0caf00 0x7f373c0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:54.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.003+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f373c062b70 con 0x7f373c0caf00 2024-09-16T00:08:54.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.003+0000 7f37408c7640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f373c0caf00 0x7f373c0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:54.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.004+0000 7f37408c7640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f373c0caf00 0x7f373c0cb300 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:54.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.004+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f373c062b70 con 0x7f373c0caf00 2024-09-16T00:08:54.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.018+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f373c062b70 con 0x7f373c0caf00 2024-09-16T00:08:54.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.020+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f373c062b70 con 0x7f373c0caf00 2024-09-16T00:08:54.021 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:54.022 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:54.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.020+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f373c02c640 0x7f373c02cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:54.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.020+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f373c062b70 con 0x7f373c02c640 2024-09-16T00:08:54.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.021+0000 7f373bfff640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f373c02c640 0x7f373c02cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:54.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.021+0000 7f373bfff640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f373c02c640 0x7f373c02cac0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:54.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.022+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f373c062b70 con 0x7f373c02c640 2024-09-16T00:08:54.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.036+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f373c062b70 con 0x7f373c02c640 2024-09-16T00:08:54.038 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:54.037+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:54.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.038+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f373c062b70 con 0x7f373c02c640 2024-09-16T00:08:54.039 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:54.040 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:54.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.038+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f373c02d510 0x7f373c206eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:54.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.038+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f373c02d990 con 0x7f373c02d510 2024-09-16T00:08:54.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.039+0000 7f373b7fe640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f373c02d510 0x7f373c206eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:54.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.039+0000 7f373b7fe640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f373c02d510 0x7f373c206eb0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:54.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.040+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f373c02d990 con 0x7f373c02d510 2024-09-16T00:08:54.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.054+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f373c02d990 con 0x7f373c02d510 2024-09-16T00:08:54.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.056+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f373c02d990 con 0x7f373c02d510 2024-09-16T00:08:54.056 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:54.057 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:54.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.056+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f373c207900 0x7f373c209cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:54.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.056+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f373c02db00 con 0x7f373c207900 2024-09-16T00:08:54.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.056+0000 7f37408c7640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f373c207900 0x7f373c209cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:54.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.057+0000 7f37408c7640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f373c207900 0x7f373c209cf0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:54.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.057+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f373c02db00 con 0x7f373c207900 2024-09-16T00:08:54.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.071+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f373c02db00 con 0x7f373c207900 2024-09-16T00:08:54.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.073+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f373c02db00 con 0x7f373c207900 2024-09-16T00:08:54.074 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:54.075 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:54.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.073+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f373c20a740 0x7f373c20cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:54.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.073+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f373c20fa20 con 0x7f373c20a740 2024-09-16T00:08:54.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.074+0000 7f373bfff640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f373c20a740 0x7f373c20cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:54.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.074+0000 7f373bfff640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f373c20a740 0x7f373c20cb30 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:54.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.075+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f373c20fa20 con 0x7f373c20a740 2024-09-16T00:08:54.080 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:54.080 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:54.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.089+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f373c000950 con 0x7f373c20a740 2024-09-16T00:08:54.091 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:08:54.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f3738ff9640 1 -- 172.21.15.139:0/1656807802 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f373c000950 con 0x7f373c20a740 2024-09-16T00:08:54.092 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:54.094 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:54.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f373c1547e0 msgr2=0x7f373c153210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:54.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f373c1547e0 0x7f373c153210 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f373c0cda00 msgr2=0x7f373c153c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:54.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f373c0cda00 0x7f373c153c60 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f373c1f42f0 msgr2=0x7f373c062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:54.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f373c1f42f0 0x7f373c062630 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f373c0caf00 msgr2=0x7f373c0cb300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:54.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f373c0caf00 0x7f373c0cb300 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f373c02c640 msgr2=0x7f373c02cac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:54.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f373c02c640 0x7f373c02cac0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f373c02d510 msgr2=0x7f373c206eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:54.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f373c02d510 0x7f373c206eb0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f373c207900 msgr2=0x7f373c209cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f373c207900 0x7f373c209cf0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f373c20a740 msgr2=0x7f373c20cb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f373c20a740 0x7f373c20cb30 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f37408c7640 1 -- 172.21.15.139:0/1656807802 reap_dead start 2024-09-16T00:08:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f37408c7640 1 -- 172.21.15.139:0/1656807802 reap_dead start 2024-09-16T00:08:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f372806c2b0 msgr2=0x7f372806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f372806c2b0 0x7f372806e770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f372c0090b0 tx=0x7f372c02e000 comp rx=0 tx=0).stop 2024-09-16T00:08:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373c0c6900 msgr2=0x7f373c155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f374232a640 1 --2- 172.21.15.139:0/1656807802 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f373c0c6900 0x7f373c155380 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3730000f60 tx=0x7f373000c800 comp rx=0 tx=0).stop 2024-09-16T00:08:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f37408c7640 1 -- 172.21.15.139:0/1656807802 reap_dead start 2024-09-16T00:08:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f37408c7640 1 -- 172.21.15.139:0/1656807802 reap_dead start 2024-09-16T00:08:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.091+0000 7f37408c7640 1 -- 172.21.15.139:0/1656807802 reap_dead start 2024-09-16T00:08:54.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.092+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 shutdown_connections 2024-09-16T00:08:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.092+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 >> 172.21.15.139:0/1656807802 conn(0x7f373c070280 msgr2=0x7f373c1501d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.092+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 shutdown_connections 2024-09-16T00:08:54.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.092+0000 7f374232a640 1 -- 172.21.15.139:0/1656807802 wait complete. 2024-09-16T00:08:54.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.108+0000 7f5893ef1640 1 Processor -- start 2024-09-16T00:08:54.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.108+0000 7f5893ef1640 1 -- start start 2024-09-16T00:08:54.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.109+0000 7f5893ef1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f588c0c8110 0x7f588c154690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:54.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.109+0000 7f5893ef1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f588c0cbfa0 0x7f588c154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:54.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.109+0000 7f5893ef1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f588c06bcc0 con 0x7f588c0c7d30 2024-09-16T00:08:54.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.109+0000 7f5893ef1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f588c06c8d0 con 0x7f588c0cbfa0 2024-09-16T00:08:54.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.109+0000 7f5893ef1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f588c06bfa0 con 0x7f588c0c8110 2024-09-16T00:08:54.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.109+0000 7f589248e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f588c0cbfa0 0x7f588c154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:54.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.109+0000 7f589248e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f588c0cbfa0 0x7f588c154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59360/0 (socket says 172.21.15.139:59360) 2024-09-16T00:08:54.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.109+0000 7f589248e640 1 -- 172.21.15.139:0/1084516446 learned_addr learned my addr 172.21.15.139:0/1084516446 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:54.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.109+0000 7f589148c640 1 --2- 172.21.15.139:0/1084516446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f588c0c8110 0x7f588c154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:54.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.109+0000 7f589248e640 1 -- 172.21.15.139:0/1084516446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f588c0c8110 msgr2=0x7f588c154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:54.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.109+0000 7f589248e640 1 --2- 172.21.15.139:0/1084516446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f588c0c8110 0x7f588c154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.109+0000 7f589248e640 1 -- 172.21.15.139:0/1084516446 >> v1:172.21.15.164:6789/0 conn(0x7f588c0c7d30 legacy=0x7f588c06cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:54.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.109+0000 7f589248e640 1 -- 172.21.15.139:0/1084516446 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c06c280 con 0x7f588c0cbfa0 2024-09-16T00:08:54.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.110+0000 7f589248e640 1 --2- 172.21.15.139:0/1084516446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f588c0cbfa0 0x7f588c154bd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5884009e50 tx=0x7f588402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c244bbff542ed15 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:54.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.110+0000 7f5890c8b640 1 -- 172.21.15.139:0/1084516446 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884036070 con 0x7f588c0cbfa0 2024-09-16T00:08:54.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.111+0000 7f5890c8b640 1 -- 172.21.15.139:0/1084516446 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5884031600 con 0x7f588c0cbfa0 2024-09-16T00:08:54.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.111+0000 7f5890c8b640 1 -- 172.21.15.139:0/1084516446 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58840319d0 con 0x7f588c0cbfa0 2024-09-16T00:08:54.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.111+0000 7f5893ef1640 1 -- 172.21.15.139:0/1084516446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f588c0cbfa0 msgr2=0x7f588c154bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:54.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.111+0000 7f5893ef1640 1 --2- 172.21.15.139:0/1084516446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f588c0cbfa0 0x7f588c154bd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5884009e50 tx=0x7f588402a7a0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.111+0000 7f5893ef1640 1 -- 172.21.15.139:0/1084516446 shutdown_connections 2024-09-16T00:08:54.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.111+0000 7f5893ef1640 1 --2- 172.21.15.139:0/1084516446 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f588c0cbfa0 0x7f588c154bd0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.111+0000 7f5893ef1640 1 --2- 172.21.15.139:0/1084516446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f588c0c8110 0x7f588c154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.111+0000 7f5893ef1640 1 -- 172.21.15.139:0/1084516446 >> 172.21.15.139:0/1084516446 conn(0x7f588c0c91d0 msgr2=0x7f588c0c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:54.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.111+0000 7f5893ef1640 1 -- 172.21.15.139:0/1084516446 shutdown_connections 2024-09-16T00:08:54.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.111+0000 7f5893ef1640 1 -- 172.21.15.139:0/1084516446 wait complete. 2024-09-16T00:08:54.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.113+0000 7f5893ef1640 1 Processor -- start 2024-09-16T00:08:54.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.113+0000 7f5893ef1640 1 -- start start 2024-09-16T00:08:54.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.113+0000 7f5893ef1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f588c0c7d30 0x7f588c15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:54.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.113+0000 7f5893ef1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f588c0c8110 0x7f588c1f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:54.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.113+0000 7f5893ef1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f588c0cbfa0 0x7f588c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:54.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.113+0000 7f5893ef1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f588c06ea00 con 0x7f588c0c7d30 2024-09-16T00:08:54.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.113+0000 7f5893ef1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f588c0c9cf0 con 0x7f588c0cbfa0 2024-09-16T00:08:54.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.113+0000 7f5893ef1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f588c0ca4d0 con 0x7f588c0c8110 2024-09-16T00:08:54.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.113+0000 7f5891c8d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f588c0c7d30 0x7f588c15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:54.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.113+0000 7f589148c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f588c0c8110 0x7f588c1f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:54.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.113+0000 7f589248e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f588c0cbfa0 0x7f588c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:54.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.113+0000 7f5891c8d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f588c0c7d30 0x7f588c15cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59374/0 (socket says 172.21.15.139:59374) 2024-09-16T00:08:54.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.114+0000 7f5891c8d640 1 -- 172.21.15.139:0/106491865 learned_addr learned my addr 172.21.15.139:0/106491865 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:54.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.113+0000 7f589148c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f588c0c8110 0x7f588c1f11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:51462/0 (socket says 172.21.15.139:51462) 2024-09-16T00:08:54.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.114+0000 7f589248e640 1 -- 172.21.15.139:0/106491865 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f588c0c8110 msgr2=0x7f588c1f11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:54.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.114+0000 7f589248e640 1 --2- 172.21.15.139:0/106491865 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f588c0c8110 0x7f588c1f11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.114+0000 7f589248e640 1 -- 172.21.15.139:0/106491865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f588c0c7d30 msgr2=0x7f588c15cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:54.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.114+0000 7f589248e640 1 --2- 172.21.15.139:0/106491865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f588c0c7d30 0x7f588c15cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.114+0000 7f589248e640 1 -- 172.21.15.139:0/106491865 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c15da70 con 0x7f588c0cbfa0 2024-09-16T00:08:54.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.114+0000 7f589248e640 1 --2- 172.21.15.139:0/106491865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f588c0cbfa0 0x7f588c15c1c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5884030000 tx=0x7f5884004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:54.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.114+0000 7f58827fc640 1 -- 172.21.15.139:0/106491865 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884036050 con 0x7f588c0cbfa0 2024-09-16T00:08:54.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.114+0000 7f58827fc640 1 -- 172.21.15.139:0/106491865 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5884042070 con 0x7f588c0cbfa0 2024-09-16T00:08:54.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.114+0000 7f58827fc640 1 -- 172.21.15.139:0/106491865 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884009510 con 0x7f588c0cbfa0 2024-09-16T00:08:54.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.114+0000 7f5893ef1640 1 -- 172.21.15.139:0/106491865 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588c0bd2c0 con 0x7f588c0cbfa0 2024-09-16T00:08:54.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.114+0000 7f5893ef1640 1 -- 172.21.15.139:0/106491865 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f588c0bd560 con 0x7f588c0cbfa0 2024-09-16T00:08:54.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.116+0000 7f5893ef1640 1 -- 172.21.15.139:0/106491865 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f588c15e800 con 0x7f588c0cbfa0 2024-09-16T00:08:54.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.116+0000 7f58827fc640 1 -- 172.21.15.139:0/106491865 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5884007bf0 con 0x7f588c0cbfa0 2024-09-16T00:08:54.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.117+0000 7f58827fc640 1 --2- 172.21.15.139:0/106491865 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f587006c2b0 0x7f587006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:54.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.117+0000 7f58827fc640 1 -- 172.21.15.139:0/106491865 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5886+0+0 (secure 0 0 0) 0x7f58840ac0d0 con 0x7f588c0cbfa0 2024-09-16T00:08:54.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.117+0000 7f5891c8d640 1 --2- 172.21.15.139:0/106491865 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f587006c2b0 0x7f587006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:54.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.117+0000 7f5891c8d640 1 --2- 172.21.15.139:0/106491865 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f587006c2b0 0x7f587006e770 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5888001840 tx=0x7f5888009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:54.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.121+0000 7f58827fc640 1 -- 172.21.15.139:0/106491865 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f588c15da70 con 0x7f588c0cbfa0 2024-09-16T00:08:54.139 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:54.139+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:54.201 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:54.201 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:54.214 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:08:54.256 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:54.255+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:54.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.321+0000 7f5893ef1640 1 -- 172.21.15.139:0/106491865 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) -- 0x7f588c0cb1f0 con 0x7f588c0cbfa0 2024-09-16T00:08:54.341 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:54.340+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:54.361 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:54.361 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:54.370 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:08:54.442 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:54.441+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:54.477 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:54.478 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:54.488 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:08:54.542 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:54.542+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:54.632 INFO:tasks.workunit.client.0.smithi139.stderr:Note: switching to 'ad35db06e8924912f01e32b35576297c0f1352b4'. 2024-09-16T00:08:54.633 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T00:08:54.633 INFO:tasks.workunit.client.0.smithi139.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T00:08:54.633 INFO:tasks.workunit.client.0.smithi139.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T00:08:54.634 INFO:tasks.workunit.client.0.smithi139.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T00:08:54.634 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T00:08:54.634 INFO:tasks.workunit.client.0.smithi139.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T00:08:54.635 INFO:tasks.workunit.client.0.smithi139.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T00:08:54.635 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T00:08:54.635 INFO:tasks.workunit.client.0.smithi139.stderr: git switch -c 2024-09-16T00:08:54.635 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T00:08:54.635 INFO:tasks.workunit.client.0.smithi139.stderr:Or undo this operation with: 2024-09-16T00:08:54.635 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T00:08:54.636 INFO:tasks.workunit.client.0.smithi139.stderr: git switch - 2024-09-16T00:08:54.636 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T00:08:54.636 INFO:tasks.workunit.client.0.smithi139.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T00:08:54.636 INFO:tasks.workunit.client.0.smithi139.stderr: 2024-09-16T00:08:54.637 INFO:tasks.workunit.client.0.smithi139.stderr:HEAD is now at ad35db06e89 Merge pull request #59794 from zdover23/wip-doc-2024-09-15-backport-59613-to-squid 2024-09-16T00:08:54.639 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:54.639 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:54.640 DEBUG:teuthology.orchestra.run.smithi139:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-16T00:08:54.642 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:54.642+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:54.644 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:08:54.663 INFO:tasks.workunit.client.0.smithi139.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T00:08:54.665 INFO:tasks.workunit.client.0.smithi139.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T00:08:54.665 INFO:tasks.workunit.client.0.smithi139.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T00:08:54.744 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:54.744+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:54.766 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:54.767 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:54.775 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:08:54.845 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:54.844+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:54.879 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:54.879 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:54.888 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:08:54.945 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:54.945+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:54.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.975+0000 7f58827fc640 1 -- 172.21.15.139:0/106491865 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v56) ==== 113+0+0 (secure 0 0 0) 0x7f588407a1d0 con 0x7f588c0cbfa0 2024-09-16T00:08:54.976 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T00:08:54.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.977+0000 7f5893ef1640 1 -- 172.21.15.139:0/106491865 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f587006c2b0 msgr2=0x7f587006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:54.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.977+0000 7f5893ef1640 1 --2- 172.21.15.139:0/106491865 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f587006c2b0 0x7f587006e770 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5888001840 tx=0x7f5888009040 comp rx=0 tx=0).stop 2024-09-16T00:08:54.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.977+0000 7f5893ef1640 1 -- 172.21.15.139:0/106491865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f588c0cbfa0 msgr2=0x7f588c15c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:54.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.977+0000 7f5893ef1640 1 --2- 172.21.15.139:0/106491865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f588c0cbfa0 0x7f588c15c1c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5884030000 tx=0x7f5884004a40 comp rx=0 tx=0).stop 2024-09-16T00:08:54.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.977+0000 7f5893ef1640 1 -- 172.21.15.139:0/106491865 shutdown_connections 2024-09-16T00:08:54.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.977+0000 7f5893ef1640 1 --2- 172.21.15.139:0/106491865 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f587006c2b0 0x7f587006e770 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.977+0000 7f5893ef1640 1 --2- 172.21.15.139:0/106491865 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f588c0cbfa0 0x7f588c15c1c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.977+0000 7f5893ef1640 1 --2- 172.21.15.139:0/106491865 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f588c0c8110 0x7f588c1f11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.977+0000 7f5893ef1640 1 --2- 172.21.15.139:0/106491865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f588c0c7d30 0x7f588c15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:54.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.977+0000 7f5893ef1640 1 -- 172.21.15.139:0/106491865 >> 172.21.15.139:0/106491865 conn(0x7f588c0c91d0 msgr2=0x7f588c14f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:54.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.977+0000 7f5893ef1640 1 -- 172.21.15.139:0/106491865 shutdown_connections 2024-09-16T00:08:54.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:54.977+0000 7f5893ef1640 1 -- 172.21.15.139:0/106491865 wait complete. 2024-09-16T00:08:54.990 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T00:08:55.036 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:55.036 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:55.045 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:08:55.047 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:55.046+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:55.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.094+0000 7fe63dd8d640 1 Processor -- start 2024-09-16T00:08:55.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.094+0000 7fe63dd8d640 1 -- start start 2024-09-16T00:08:55.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.095+0000 7fe63dd8d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe638154c20 0x7fe63815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:55.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.095+0000 7fe63dd8d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe638068060 0x7fe6380c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.095+0000 7fe63dd8d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6380cc4b0 con 0x7fe6380ccc90 2024-09-16T00:08:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.095+0000 7fe63dd8d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe63815b7c0 con 0x7fe638154c20 2024-09-16T00:08:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.095+0000 7fe63dd8d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe6380cc260 con 0x7fe638068060 2024-09-16T00:08:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.095+0000 7fe6377fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe638154c20 0x7fe63815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.095+0000 7fe6377fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe638154c20 0x7fe63815b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59384/0 (socket says 172.21.15.139:59384) 2024-09-16T00:08:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.095+0000 7fe6377fe640 1 -- 172.21.15.139:0/3156930273 learned_addr learned my addr 172.21.15.139:0/3156930273 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.095+0000 7fe6377fe640 1 -- 172.21.15.139:0/3156930273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe638068060 msgr2=0x7fe6380c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.095+0000 7fe6377fe640 1 --2- 172.21.15.139:0/3156930273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe638068060 0x7fe6380c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.095+0000 7fe6377fe640 1 -- 172.21.15.139:0/3156930273 >> v1:172.21.15.164:6789/0 conn(0x7fe6380ccc90 legacy=0x7fe6380cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:55.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.095+0000 7fe6377fe640 1 -- 172.21.15.139:0/3156930273 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6380cc790 con 0x7fe638154c20 2024-09-16T00:08:55.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.096+0000 7fe6377fe640 1 --2- 172.21.15.139:0/3156930273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe638154c20 0x7fe63815b280 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe628009e50 tx=0x7fe62802a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d6d55037423f22 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:55.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.096+0000 7fe6367fc640 1 -- 172.21.15.139:0/3156930273 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628036070 con 0x7fe638154c20 2024-09-16T00:08:55.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.096+0000 7fe6367fc640 1 -- 172.21.15.139:0/3156930273 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe628031600 con 0x7fe638154c20 2024-09-16T00:08:55.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.096+0000 7fe6367fc640 1 -- 172.21.15.139:0/3156930273 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6280319d0 con 0x7fe638154c20 2024-09-16T00:08:55.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.097+0000 7fe63dd8d640 1 -- 172.21.15.139:0/3156930273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe638154c20 msgr2=0x7fe63815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:55.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.097+0000 7fe63dd8d640 1 --2- 172.21.15.139:0/3156930273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe638154c20 0x7fe63815b280 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe628009e50 tx=0x7fe62802a750 comp rx=0 tx=0).stop 2024-09-16T00:08:55.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.097+0000 7fe63dd8d640 1 -- 172.21.15.139:0/3156930273 shutdown_connections 2024-09-16T00:08:55.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.097+0000 7fe63dd8d640 1 --2- 172.21.15.139:0/3156930273 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe638068060 0x7fe6380c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:55.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.097+0000 7fe63dd8d640 1 --2- 172.21.15.139:0/3156930273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe638154c20 0x7fe63815b280 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:55.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.097+0000 7fe63dd8d640 1 -- 172.21.15.139:0/3156930273 >> 172.21.15.139:0/3156930273 conn(0x7fe63806ca40 msgr2=0x7fe638067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:55.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.097+0000 7fe63dd8d640 1 -- 172.21.15.139:0/3156930273 shutdown_connections 2024-09-16T00:08:55.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.097+0000 7fe63dd8d640 1 -- 172.21.15.139:0/3156930273 wait complete. 2024-09-16T00:08:55.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.098+0000 7fe63dd8d640 1 Processor -- start 2024-09-16T00:08:55.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.098+0000 7fe63dd8d640 1 -- start start 2024-09-16T00:08:55.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.099+0000 7fe63dd8d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe638068060 0x7fe63814b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:55.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.099+0000 7fe63dd8d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6380ccc90 0x7fe6380b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:55.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.099+0000 7fe63dd8d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe638154c20 0x7fe6380cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:55.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.099+0000 7fe63dd8d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe63815e3b0 con 0x7fe6380ccc90 2024-09-16T00:08:55.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.099+0000 7fe63dd8d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe63815e0d0 con 0x7fe638154c20 2024-09-16T00:08:55.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.099+0000 7fe63dd8d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe63815e690 con 0x7fe638068060 2024-09-16T00:08:55.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.099+0000 7fe636ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6380ccc90 0x7fe6380b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:55.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.099+0000 7fe636ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6380ccc90 0x7fe6380b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59400/0 (socket says 172.21.15.139:59400) 2024-09-16T00:08:55.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.099+0000 7fe636ffd640 1 -- 172.21.15.139:0/1644458499 learned_addr learned my addr 172.21.15.139:0/1644458499 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:55.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.099+0000 7fe6377fe640 1 --2- 172.21.15.139:0/1644458499 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe638068060 0x7fe63814b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:55.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.099+0000 7fe637fff640 1 --2- 172.21.15.139:0/1644458499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe638154c20 0x7fe6380cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:55.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.100+0000 7fe636ffd640 1 -- 172.21.15.139:0/1644458499 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe638068060 msgr2=0x7fe63814b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:55.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.100+0000 7fe636ffd640 1 --2- 172.21.15.139:0/1644458499 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe638068060 0x7fe63814b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:55.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.100+0000 7fe636ffd640 1 -- 172.21.15.139:0/1644458499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe638154c20 msgr2=0x7fe6380cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:55.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.100+0000 7fe636ffd640 1 --2- 172.21.15.139:0/1644458499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe638154c20 0x7fe6380cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:55.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.100+0000 7fe636ffd640 1 -- 172.21.15.139:0/1644458499 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63815d480 con 0x7fe6380ccc90 2024-09-16T00:08:55.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.100+0000 7fe636ffd640 1 --2- 172.21.15.139:0/1644458499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6380ccc90 0x7fe6380b5670 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b4a0 tx=0x7fe62c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:55.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.100+0000 7fe61ffff640 1 -- 172.21.15.139:0/1644458499 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c014070 con 0x7fe6380ccc90 2024-09-16T00:08:55.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.100+0000 7fe61ffff640 1 -- 172.21.15.139:0/1644458499 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe62c005a10 con 0x7fe6380ccc90 2024-09-16T00:08:55.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.100+0000 7fe63dd8d640 1 -- 172.21.15.139:0/1644458499 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6381f1040 con 0x7fe6380ccc90 2024-09-16T00:08:55.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.100+0000 7fe61ffff640 1 -- 172.21.15.139:0/1644458499 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c005e40 con 0x7fe6380ccc90 2024-09-16T00:08:55.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.100+0000 7fe63dd8d640 1 -- 172.21.15.139:0/1644458499 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6381f1600 con 0x7fe6380ccc90 2024-09-16T00:08:55.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.101+0000 7fe63dd8d640 1 -- 172.21.15.139:0/1644458499 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6380686b0 con 0x7fe6380ccc90 2024-09-16T00:08:55.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.101+0000 7fe61ffff640 1 -- 172.21.15.139:0/1644458499 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe62c010610 con 0x7fe6380ccc90 2024-09-16T00:08:55.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.102+0000 7fe61ffff640 1 --2- 172.21.15.139:0/1644458499 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe62406c2b0 0x7fe62406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:55.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.102+0000 7fe61ffff640 1 -- 172.21.15.139:0/1644458499 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5886+0+0 (secure 0 0 0) 0x7fe62c08ae20 con 0x7fe6380ccc90 2024-09-16T00:08:55.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.102+0000 7fe6377fe640 1 --2- 172.21.15.139:0/1644458499 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe62406c2b0 0x7fe62406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:55.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.102+0000 7fe6377fe640 1 --2- 172.21.15.139:0/1644458499 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe62406c2b0 0x7fe62406e770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe628033000 tx=0x7fe628009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:55.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.106+0000 7fe61ffff640 1 -- 172.21.15.139:0/1644458499 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe62c058ea0 con 0x7fe6380ccc90 2024-09-16T00:08:55.148 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:55.148+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:55.153 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:55.154 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:55.163 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:08:55.249 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:55.248+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:55.269 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:55.269 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:55.278 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:08:55.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.292+0000 7fe63dd8d640 1 -- 172.21.15.139:0/1644458499 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) -- 0x7fe63814ea70 con 0x7fe6380ccc90 2024-09-16T00:08:55.350 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:55.349+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:55.355 INFO:tasks.workunit.client.0.smithi139.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T00:08:55.399 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:55.399 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:55.409 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:08:55.417 INFO:tasks.workunit.client.0.smithi139.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T00:08:55.453 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:55.453+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:55.457 INFO:tasks.workunit.client.0.smithi139.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T00:08:55.459 INFO:tasks.workunit.client.0.smithi139.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T00:08:55.459 INFO:tasks.workunit.client.0.smithi139.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T00:08:55.497 INFO:tasks.workunit.client.0.smithi139.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T00:08:55.500 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:08:55.500 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-16T00:08:55.526 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:55.526 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:55.535 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:08:55.556 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-09-16T00:08:55.558 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-09-16T00:08:55.558 DEBUG:teuthology.orchestra.run.smithi139:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-09-16T00:08:55.577 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:55.576+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:55.622 INFO:tasks.workunit.client.0.smithi139.stderr:+ mkdir -p fsstress 2024-09-16T00:08:55.625 INFO:tasks.workunit.client.0.smithi139.stderr:+ pushd fsstress 2024-09-16T00:08:55.625 INFO:tasks.workunit.client.0.smithi139.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-16T00:08:55.626 INFO:tasks.workunit.client.0.smithi139.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-09-16T00:08:55.653 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:55.653+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:55.681 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:55.681 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:55.690 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:08:55.774 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:55.774+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:55.833 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:55.834 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:55.842 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:08:55.884 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:55.883+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:55.955 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:55.955+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:55.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.976+0000 7fe61ffff640 1 -- 172.21.15.139:0/1644458499 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v57) ==== 113+0+0 (secure 0 0 0) 0x7fe62c05d1a0 con 0x7fe6380ccc90 2024-09-16T00:08:55.978 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T00:08:55.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.980+0000 7fe63dd8d640 1 -- 172.21.15.139:0/1644458499 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe62406c2b0 msgr2=0x7fe62406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:55.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.980+0000 7fe63dd8d640 1 --2- 172.21.15.139:0/1644458499 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe62406c2b0 0x7fe62406e770 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe628033000 tx=0x7fe628009000 comp rx=0 tx=0).stop 2024-09-16T00:08:55.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.980+0000 7fe63dd8d640 1 -- 172.21.15.139:0/1644458499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6380ccc90 msgr2=0x7fe6380b5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:55.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.980+0000 7fe63dd8d640 1 --2- 172.21.15.139:0/1644458499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6380ccc90 0x7fe6380b5670 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b4a0 tx=0x7fe62c00b960 comp rx=0 tx=0).stop 2024-09-16T00:08:55.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.980+0000 7fe63dd8d640 1 -- 172.21.15.139:0/1644458499 shutdown_connections 2024-09-16T00:08:55.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.980+0000 7fe63dd8d640 1 --2- 172.21.15.139:0/1644458499 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe62406c2b0 0x7fe62406e770 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:55.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.980+0000 7fe63dd8d640 1 --2- 172.21.15.139:0/1644458499 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe638154c20 0x7fe6380cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:55.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.980+0000 7fe63dd8d640 1 --2- 172.21.15.139:0/1644458499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6380ccc90 0x7fe6380b5670 secure :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b4a0 tx=0x7fe62c00b960 comp rx=0 tx=0).stop 2024-09-16T00:08:55.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.980+0000 7fe63dd8d640 1 --2- 172.21.15.139:0/1644458499 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe638068060 0x7fe63814b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:55.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.980+0000 7fe63dd8d640 1 -- 172.21.15.139:0/1644458499 >> 172.21.15.139:0/1644458499 conn(0x7fe63806ca40 msgr2=0x7fe6380c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:55.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.980+0000 7fe63dd8d640 1 -- 172.21.15.139:0/1644458499 shutdown_connections 2024-09-16T00:08:55.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:55.980+0000 7fe63dd8d640 1 -- 172.21.15.139:0/1644458499 wait complete. 2024-09-16T00:08:55.991 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:55.991 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:55.991 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T00:08:55.993 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T00:08:56.000 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:08:56.084 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:56.084+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:56.105 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:08:56.145 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:56.145 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:56.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.149+0000 7fbad3c53640 1 Processor -- start 2024-09-16T00:08:56.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.149+0000 7fbad3c53640 1 -- start start 2024-09-16T00:08:56.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.151+0000 7fbad3c53640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbacc0ce430 0x7fbacc0b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.151+0000 7fbad3c53640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbacc0c9430 0x7fbacc15bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.151+0000 7fbad3c53640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbacc0703e0 con 0x7fbacc0ce810 2024-09-16T00:08:56.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.151+0000 7fbad3c53640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbacc0c5750 con 0x7fbacc0ce430 2024-09-16T00:08:56.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.151+0000 7fbad3c53640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbacc070680 con 0x7fbacc0c9430 2024-09-16T00:08:56.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.151+0000 7fbad11ee640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fbacc0ce810 0x7fbacc0c5be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:44444/0 (socket says 172.21.15.139:44444) 2024-09-16T00:08:56.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.151+0000 7fbad11ee640 1 -- 172.21.15.139:0/3546774003 learned_addr learned my addr 172.21.15.139:0/3546774003 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:56.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.151+0000 7fbad19ef640 1 --2- 172.21.15.139:0/3546774003 >> v2:172.21.15.139:3300/0 conn(0x7fbacc0ce430 0x7fbacc0b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.151+0000 7fbad21f0640 1 --2- 172.21.15.139:0/3546774003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbacc0c9430 0x7fbacc15bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.151+0000 7fbad09ed640 1 -- 172.21.15.139:0/3546774003 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4090845447 0 0) 0x7fbacc0703e0 con 0x7fbacc0ce810 2024-09-16T00:08:56.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.151+0000 7fbad09ed640 1 -- 172.21.15.139:0/3546774003 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbab8003520 con 0x7fbacc0ce810 2024-09-16T00:08:56.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.151+0000 7fbad19ef640 1 -- 172.21.15.139:0/3546774003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbacc0c9430 msgr2=0x7fbacc15bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:56.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.151+0000 7fbad19ef640 1 --2- 172.21.15.139:0/3546774003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbacc0c9430 0x7fbacc15bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.151+0000 7fbad19ef640 1 -- 172.21.15.139:0/3546774003 >> v1:172.21.15.139:6789/0 conn(0x7fbacc0ce810 legacy=0x7fbacc0c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.151+0000 7fbad19ef640 1 -- 172.21.15.139:0/3546774003 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc070920 con 0x7fbacc0ce430 2024-09-16T00:08:56.155 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:08:56.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.152+0000 7fbad19ef640 1 --2- 172.21.15.139:0/3546774003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbacc0ce430 0x7fbacc0b31c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbac80097a0 tx=0x7fbac802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f04666f37e7ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.153+0000 7fbad09ed640 1 -- 172.21.15.139:0/3546774003 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8037070 con 0x7fbacc0ce430 2024-09-16T00:08:56.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.153+0000 7fbad09ed640 1 -- 172.21.15.139:0/3546774003 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac8032440 con 0x7fbacc0ce430 2024-09-16T00:08:56.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.153+0000 7fbad09ed640 1 -- 172.21.15.139:0/3546774003 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8032810 con 0x7fbacc0ce430 2024-09-16T00:08:56.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.153+0000 7fbad3c53640 1 -- 172.21.15.139:0/3546774003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbacc0ce430 msgr2=0x7fbacc0b31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.153+0000 7fbad3c53640 1 --2- 172.21.15.139:0/3546774003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbacc0ce430 0x7fbacc0b31c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbac80097a0 tx=0x7fbac802a180 comp rx=0 tx=0).stop 2024-09-16T00:08:56.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.154+0000 7fbad3c53640 1 -- 172.21.15.139:0/3546774003 shutdown_connections 2024-09-16T00:08:56.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.154+0000 7fbad3c53640 1 --2- 172.21.15.139:0/3546774003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbacc0c9430 0x7fbacc15bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.154+0000 7fbad3c53640 1 --2- 172.21.15.139:0/3546774003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbacc0ce430 0x7fbacc0b31c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.154+0000 7fbad3c53640 1 -- 172.21.15.139:0/3546774003 >> 172.21.15.139:0/3546774003 conn(0x7fbacc06c150 msgr2=0x7fbacc067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:56.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.154+0000 7fbad3c53640 1 -- 172.21.15.139:0/3546774003 shutdown_connections 2024-09-16T00:08:56.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.154+0000 7fbad3c53640 1 -- 172.21.15.139:0/3546774003 wait complete. 2024-09-16T00:08:56.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.155+0000 7fbad3c53640 1 Processor -- start 2024-09-16T00:08:56.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.155+0000 7fbad3c53640 1 -- start start 2024-09-16T00:08:56.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.156+0000 7fbad3c53640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbacc0c9430 0x7fbacc06d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.156+0000 7fbad3c53640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbacc0ce430 0x7fbacc0cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.156+0000 7fbad3c53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbacc0ce810 0x7fbacc0c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.156+0000 7fbad3c53640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc0b3700 con 0x7fbacc0ce810 2024-09-16T00:08:56.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.156+0000 7fbad3c53640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc070ee0 con 0x7fbacc0ce430 2024-09-16T00:08:56.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.156+0000 7fbad3c53640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbacc0b3880 con 0x7fbacc0c9430 2024-09-16T00:08:56.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.156+0000 7fbad21f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbacc0ce810 0x7fbacc0c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.156+0000 7fbad21f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbacc0ce810 0x7fbacc0c1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59406/0 (socket says 172.21.15.139:59406) 2024-09-16T00:08:56.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.156+0000 7fbad21f0640 1 -- 172.21.15.139:0/4247989383 learned_addr learned my addr 172.21.15.139:0/4247989383 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:56.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.156+0000 7fbad19ef640 1 --2- 172.21.15.139:0/4247989383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbacc0c9430 0x7fbacc06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.156+0000 7fbad11ee640 1 --2- 172.21.15.139:0/4247989383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbacc0ce430 0x7fbacc0cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.156+0000 7fbad21f0640 1 -- 172.21.15.139:0/4247989383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbacc0c9430 msgr2=0x7fbacc06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.156+0000 7fbad21f0640 1 --2- 172.21.15.139:0/4247989383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbacc0c9430 0x7fbacc06d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.156+0000 7fbad21f0640 1 -- 172.21.15.139:0/4247989383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbacc0ce430 msgr2=0x7fbacc0cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.156+0000 7fbad21f0640 1 --2- 172.21.15.139:0/4247989383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbacc0ce430 0x7fbacc0cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.156+0000 7fbad21f0640 1 -- 172.21.15.139:0/4247989383 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc1ebde0 con 0x7fbacc0ce810 2024-09-16T00:08:56.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.157+0000 7fbad21f0640 1 --2- 172.21.15.139:0/4247989383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbacc0ce810 0x7fbacc0c1eb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbac400efc0 tx=0x7fbac400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.157+0000 7fbac27fc640 1 -- 172.21.15.139:0/4247989383 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4018020 con 0x7fbacc0ce810 2024-09-16T00:08:56.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.157+0000 7fbac27fc640 1 -- 172.21.15.139:0/4247989383 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac4014480 con 0x7fbacc0ce810 2024-09-16T00:08:56.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.157+0000 7fbac27fc640 1 -- 172.21.15.139:0/4247989383 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4014870 con 0x7fbacc0ce810 2024-09-16T00:08:56.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.157+0000 7fbad3c53640 1 -- 172.21.15.139:0/4247989383 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbacc152b10 con 0x7fbacc0ce810 2024-09-16T00:08:56.161 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:56.157+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:56.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.157+0000 7fbad3c53640 1 -- 172.21.15.139:0/4247989383 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbacc1ec340 con 0x7fbacc0ce810 2024-09-16T00:08:56.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.159+0000 7fbad3c53640 1 -- 172.21.15.139:0/4247989383 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbacc06ffc0 con 0x7fbacc0ce810 2024-09-16T00:08:56.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.159+0000 7fbac27fc640 1 -- 172.21.15.139:0/4247989383 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbac4005d10 con 0x7fbacc0ce810 2024-09-16T00:08:56.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.160+0000 7fbac27fc640 1 --2- 172.21.15.139:0/4247989383 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbab006c300 0x7fbab006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.160+0000 7fbac27fc640 1 -- 172.21.15.139:0/4247989383 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5886+0+0 (secure 0 0 0) 0x7fbac408c190 con 0x7fbacc0ce810 2024-09-16T00:08:56.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.160+0000 7fbad19ef640 1 --2- 172.21.15.139:0/4247989383 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbab006c300 0x7fbab006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.161+0000 7fbad19ef640 1 --2- 172.21.15.139:0/4247989383 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbab006c300 0x7fbab006e7c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbac8034000 tx=0x7fbac802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.166+0000 7fe8a4aba640 1 Processor -- start 2024-09-16T00:08:56.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.166+0000 7fe8a4aba640 1 -- start start 2024-09-16T00:08:56.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.167+0000 7fbac27fc640 1 -- 172.21.15.139:0/4247989383 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbac4022030 con 0x7fbacc0ce810 2024-09-16T00:08:56.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.168+0000 7fe8a4aba640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe8a0068060 0x7fe8a015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.168+0000 7fe8a4aba640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe8a00ccc90 0x7fe8a00cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.168+0000 7fe8a4aba640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8a00cc4b0 con 0x7fe8a0154c20 2024-09-16T00:08:56.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.168+0000 7fe8a4aba640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe8a00c5830 con 0x7fe8a00ccc90 2024-09-16T00:08:56.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.168+0000 7fe8a4aba640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fe8a015b7c0 con 0x7fe8a0068060 2024-09-16T00:08:56.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.168+0000 7fe89ed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe8a00ccc90 0x7fe8a00cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.168+0000 7fe89e59c640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe8a0154c20 0x7fe8a00c5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:33256/0 (socket says 172.21.15.139:33256) 2024-09-16T00:08:56.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.168+0000 7fe89e59c640 1 -- 172.21.15.139:0/1361498695 learned_addr learned my addr 172.21.15.139:0/1361498695 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:56.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.168+0000 7fe89dd9b640 1 --2- 172.21.15.139:0/1361498695 >> v2:172.21.15.164:3300/0 conn(0x7fe8a0068060 0x7fe8a015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.168+0000 7fe89dd9b640 1 -- 172.21.15.139:0/1361498695 >> v2:172.21.15.139:3300/0 conn(0x7fe8a00ccc90 msgr2=0x7fe8a00cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:56.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.168+0000 7fe89dd9b640 1 --2- 172.21.15.139:0/1361498695 >> v2:172.21.15.139:3300/0 conn(0x7fe8a00ccc90 0x7fe8a00cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.168+0000 7fe89d59a640 1 -- 172.21.15.139:0/1361498695 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3980077713 0 0) 0x7fe8a00cc4b0 con 0x7fe8a0154c20 2024-09-16T00:08:56.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.168+0000 7fe89dd9b640 1 -- 172.21.15.139:0/1361498695 >> v1:172.21.15.164:6789/0 conn(0x7fe8a0154c20 legacy=0x7fe8a00c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.168+0000 7fe89dd9b640 1 -- 172.21.15.139:0/1361498695 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a00cc790 con 0x7fe8a0068060 2024-09-16T00:08:56.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.169+0000 7fe89dd9b640 1 --2- 172.21.15.139:0/1361498695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a0068060 0x7fe8a015b280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00685f0 tx=0x7fe89402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffceb838ba0a4402 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.169+0000 7fe89d59a640 1 -- 172.21.15.139:0/1361498695 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894037070 con 0x7fe8a0068060 2024-09-16T00:08:56.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.169+0000 7fe89d59a640 1 -- 172.21.15.139:0/1361498695 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe894032440 con 0x7fe8a0068060 2024-09-16T00:08:56.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.169+0000 7fe89d59a640 1 -- 172.21.15.139:0/1361498695 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894032810 con 0x7fe8a0068060 2024-09-16T00:08:56.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.170+0000 7fe8a4aba640 1 -- 172.21.15.139:0/1361498695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a0068060 msgr2=0x7fe8a015b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.170+0000 7fe8a4aba640 1 --2- 172.21.15.139:0/1361498695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a0068060 0x7fe8a015b280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00685f0 tx=0x7fe89402a1a0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.170+0000 7fe8a4aba640 1 -- 172.21.15.139:0/1361498695 shutdown_connections 2024-09-16T00:08:56.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.170+0000 7fe8a4aba640 1 --2- 172.21.15.139:0/1361498695 >> v2:172.21.15.139:3300/0 conn(0x7fe8a00ccc90 0x7fe8a00cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.170+0000 7fe8a4aba640 1 --2- 172.21.15.139:0/1361498695 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a0068060 0x7fe8a015b280 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.170+0000 7fe8a4aba640 1 -- 172.21.15.139:0/1361498695 >> 172.21.15.139:0/1361498695 conn(0x7fe8a006ca40 msgr2=0x7fe8a0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:56.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.170+0000 7fe8a4aba640 1 -- 172.21.15.139:0/1361498695 shutdown_connections 2024-09-16T00:08:56.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.170+0000 7fe8a4aba640 1 -- 172.21.15.139:0/1361498695 wait complete. 2024-09-16T00:08:56.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.172+0000 7fe8a4aba640 1 Processor -- start 2024-09-16T00:08:56.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.172+0000 7fe8a4aba640 1 -- start start 2024-09-16T00:08:56.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.173+0000 7fe8a4aba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a0068060 0x7fe8a014eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.173+0000 7fe8a4aba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a00ccc90 0x7fe8a01497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.173+0000 7fe8a4aba640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a0154c20 0x7fe8a0149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.173+0000 7fe8a4aba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a015e0d0 con 0x7fe8a0068060 2024-09-16T00:08:56.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.173+0000 7fe8a4aba640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a00cc260 con 0x7fe8a00ccc90 2024-09-16T00:08:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.173+0000 7fe8a4aba640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a015e3b0 con 0x7fe8a0154c20 2024-09-16T00:08:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.173+0000 7fe89dd9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a00ccc90 0x7fe8a01497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.173+0000 7fe89e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a0068060 0x7fe8a014eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.173+0000 7fe89dd9b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a00ccc90 0x7fe8a01497e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:38432/0 (socket says 172.21.15.139:38432) 2024-09-16T00:08:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.173+0000 7fe89ed9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a0154c20 0x7fe8a0149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.173+0000 7fe89dd9b640 1 -- 172.21.15.139:0/3275165472 learned_addr learned my addr 172.21.15.139:0/3275165472 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.173+0000 7fe89dd9b640 1 -- 172.21.15.139:0/3275165472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a0154c20 msgr2=0x7fe8a0149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.173+0000 7fe89dd9b640 1 --2- 172.21.15.139:0/3275165472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a0154c20 0x7fe8a0149d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.173+0000 7fe89dd9b640 1 -- 172.21.15.139:0/3275165472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a0068060 msgr2=0x7fe8a014eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.173+0000 7fe89dd9b640 1 --2- 172.21.15.139:0/3275165472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a0068060 0x7fe8a014eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.173+0000 7fe89dd9b640 1 -- 172.21.15.139:0/3275165472 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a0153180 con 0x7fe8a00ccc90 2024-09-16T00:08:56.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.174+0000 7fe89ed9d640 1 --2- 172.21.15.139:0/3275165472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a0154c20 0x7fe8a0149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.174+0000 7fe89e59c640 1 --2- 172.21.15.139:0/3275165472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a0068060 0x7fe8a014eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.174+0000 7fe89dd9b640 1 --2- 172.21.15.139:0/3275165472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a00ccc90 0x7fe8a01497e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe894000c00 tx=0x7fe8940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.174+0000 7fe88effd640 1 -- 172.21.15.139:0/3275165472 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894037040 con 0x7fe8a00ccc90 2024-09-16T00:08:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.174+0000 7fe88effd640 1 -- 172.21.15.139:0/3275165472 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe894042070 con 0x7fe8a00ccc90 2024-09-16T00:08:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.174+0000 7fe88effd640 1 -- 172.21.15.139:0/3275165472 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894030960 con 0x7fe8a00ccc90 2024-09-16T00:08:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.174+0000 7fe8a4aba640 1 -- 172.21.15.139:0/3275165472 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a0153420 con 0x7fe8a00ccc90 2024-09-16T00:08:56.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.174+0000 7fe8a4aba640 1 -- 172.21.15.139:0/3275165472 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a01539e0 con 0x7fe8a00ccc90 2024-09-16T00:08:56.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.175+0000 7fe8a4aba640 1 -- 172.21.15.139:0/3275165472 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a015e690 con 0x7fe8a00ccc90 2024-09-16T00:08:56.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.176+0000 7fe88effd640 1 -- 172.21.15.139:0/3275165472 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe894030b00 con 0x7fe8a00ccc90 2024-09-16T00:08:56.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.176+0000 7fe88effd640 1 --2- 172.21.15.139:0/3275165472 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe88006c2e0 0x7fe88006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.176+0000 7fe88effd640 1 -- 172.21.15.139:0/3275165472 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5886+0+0 (secure 0 0 0) 0x7fe8940abcd0 con 0x7fe8a00ccc90 2024-09-16T00:08:56.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.176+0000 7fe89e59c640 1 --2- 172.21.15.139:0/3275165472 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe88006c2e0 0x7fe88006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.177+0000 7fe89e59c640 1 --2- 172.21.15.139:0/3275165472 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe88006c2e0 0x7fe88006e7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe890007fd0 tx=0x7fe890007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.181+0000 7fe88effd640 1 -- 172.21.15.139:0/3275165472 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe894079d40 con 0x7fe8a00ccc90 2024-09-16T00:08:56.205 INFO:tasks.workunit.client.0.smithi139.stderr:+ tar xzf ltp-full.tgz 2024-09-16T00:08:56.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.235+0000 7f800a7c4640 1 Processor -- start 2024-09-16T00:08:56.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.236+0000 7f800a7c4640 1 -- start start 2024-09-16T00:08:56.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.236+0000 7f800a7c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f80040d0490 0x7f8004160430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.236+0000 7f800a7c4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f80040ccad0 0x7f8004160970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.236+0000 7f800a7c4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80040cb550 con 0x7f80040d0870 2024-09-16T00:08:56.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.237+0000 7f8003fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f80040d0490 0x7f8004160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.237+0000 7f800a7c4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f80040cc6b0 con 0x7f80040ccad0 2024-09-16T00:08:56.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.237+0000 7f8003fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f80040d0490 0x7f8004160430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:51508/0 (socket says 172.21.15.139:51508) 2024-09-16T00:08:56.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.237+0000 7f8003fff640 1 -- 172.21.15.139:0/428871138 learned_addr learned my addr 172.21.15.139:0/428871138 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:56.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.237+0000 7f800a7c4640 1 -- 172.21.15.139:0/428871138 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f80040b61d0 con 0x7f80040d0490 2024-09-16T00:08:56.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.237+0000 7f8008d61640 1 --2- 172.21.15.139:0/428871138 >> v2:172.21.15.139:3300/0 conn(0x7f80040ccad0 0x7f8004160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.237+0000 7f8003fff640 1 -- 172.21.15.139:0/428871138 >> v2:172.21.15.139:3300/0 conn(0x7f80040ccad0 msgr2=0x7f8004160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:56.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.237+0000 7f8003fff640 1 --2- 172.21.15.139:0/428871138 >> v2:172.21.15.139:3300/0 conn(0x7f80040ccad0 0x7f8004160970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.237+0000 7f8003fff640 1 -- 172.21.15.139:0/428871138 >> v1:172.21.15.164:6789/0 conn(0x7f80040d0870 legacy=0x7f800415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.237+0000 7f8003fff640 1 -- 172.21.15.139:0/428871138 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80040cb7e0 con 0x7f80040d0490 2024-09-16T00:08:56.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.238+0000 7f8003fff640 1 --2- 172.21.15.139:0/428871138 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f80040d0490 0x7f8004160430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009870 tx=0x7f7ff402a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e6701165f562b06 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.238+0000 7f8002ffd640 1 -- 172.21.15.139:0/428871138 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4037070 con 0x7f80040d0490 2024-09-16T00:08:56.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.238+0000 7f8002ffd640 1 -- 172.21.15.139:0/428871138 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff4032440 con 0x7f80040d0490 2024-09-16T00:08:56.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.238+0000 7f8002ffd640 1 -- 172.21.15.139:0/428871138 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4032810 con 0x7f80040d0490 2024-09-16T00:08:56.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.239+0000 7f800a7c4640 1 -- 172.21.15.139:0/428871138 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f80040d0490 msgr2=0x7f8004160430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.239+0000 7f800a7c4640 1 --2- 172.21.15.139:0/428871138 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f80040d0490 0x7f8004160430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009870 tx=0x7f7ff402a250 comp rx=0 tx=0).stop 2024-09-16T00:08:56.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.239+0000 7f800a7c4640 1 -- 172.21.15.139:0/428871138 shutdown_connections 2024-09-16T00:08:56.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.239+0000 7f800a7c4640 1 --2- 172.21.15.139:0/428871138 >> v2:172.21.15.139:3300/0 conn(0x7f80040ccad0 0x7f8004160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.239+0000 7f800a7c4640 1 --2- 172.21.15.139:0/428871138 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f80040d0490 0x7f8004160430 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.239+0000 7f800a7c4640 1 -- 172.21.15.139:0/428871138 >> 172.21.15.139:0/428871138 conn(0x7f800406c520 msgr2=0x7f800406c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:56.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.239+0000 7f800a7c4640 1 -- 172.21.15.139:0/428871138 shutdown_connections 2024-09-16T00:08:56.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.239+0000 7f800a7c4640 1 -- 172.21.15.139:0/428871138 wait complete. 2024-09-16T00:08:56.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.240+0000 7f800a7c4640 1 Processor -- start 2024-09-16T00:08:56.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.241+0000 7f800a7c4640 1 -- start start 2024-09-16T00:08:56.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.241+0000 7f800a7c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80040ccad0 0x7f80041f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.242+0000 7f800a7c4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80040d0490 0x7f8004158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.242+0000 7f800a7c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f80040d0870 0x7f800415c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.242+0000 7f8003fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80040ccad0 0x7f80041f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.242+0000 7f800a7c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f80040cc060 con 0x7f80040ccad0 2024-09-16T00:08:56.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.242+0000 7f8003fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80040ccad0 0x7f80041f4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59440/0 (socket says 172.21.15.139:59440) 2024-09-16T00:08:56.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.242+0000 7f8003fff640 1 -- 172.21.15.139:0/3161308764 learned_addr learned my addr 172.21.15.139:0/3161308764 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:56.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.242+0000 7f80037fe640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80040d0490 0x7f8004158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.242+0000 7f8008d61640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f80040d0870 0x7f800415c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.242+0000 7f800a7c4640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f80040cbd80 con 0x7f80040d0490 2024-09-16T00:08:56.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.242+0000 7f800a7c4640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f80040cc340 con 0x7f80040d0870 2024-09-16T00:08:56.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.242+0000 7f8008d61640 1 -- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80040d0490 msgr2=0x7f8004158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.242+0000 7f8008d61640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80040d0490 0x7f8004158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.242+0000 7f8008d61640 1 -- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80040ccad0 msgr2=0x7f80041f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.242+0000 7f8008d61640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80040ccad0 0x7f80041f4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.242+0000 7f8008d61640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80041f8200 con 0x7f80040d0870 2024-09-16T00:08:56.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.242+0000 7f80037fe640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f80040d0490 0x7f8004158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:56.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.243+0000 7f8008d61640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f80040d0870 0x7f800415c010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00efc0 tx=0x7f7ffc00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.243+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc018020 con 0x7f80040d0870 2024-09-16T00:08:56.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.243+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ffc014480 con 0x7f80040d0870 2024-09-16T00:08:56.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.243+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc014870 con 0x7f80040d0870 2024-09-16T00:08:56.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.243+0000 7f800a7c4640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80041f4470 con 0x7f80040d0870 2024-09-16T00:08:56.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.243+0000 7f800a7c4640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f80041f4710 con 0x7f80040d0870 2024-09-16T00:08:56.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.244+0000 7f800a7c4640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f80040b6440 con 0x7f80040d0870 2024-09-16T00:08:56.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.244+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7ffc005d10 con 0x7f80040d0870 2024-09-16T00:08:56.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.245+0000 7f8000ff9640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7ff006c2e0 0x7f7ff006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.245+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(57..57 src has 1..57) ==== 5886+0+0 (secure 0 0 0) 0x7f7ffc08c1d0 con 0x7f80040d0870 2024-09-16T00:08:56.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.245+0000 7f8003fff640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7ff006c2e0 0x7f7ff006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.245+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v57) ==== 54+0+16 (secure 0 0 0) 0x7f7ffc022030 con 0x7f80040d0870 2024-09-16T00:08:56.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.245+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f80040c7a10 0x7f800415acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.246+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80041f8760 con 0x7f80040c7a10 2024-09-16T00:08:56.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.246+0000 7f80037fe640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f80040c7a10 0x7f800415acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.246+0000 7f8003fff640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7ff006c2e0 0x7f7ff006e7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40090b0 tx=0x7f7ff402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.246+0000 7f80037fe640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f80040c7a10 0x7f800415acd0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.247+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f80041f8760 con 0x7f80040c7a10 2024-09-16T00:08:56.259 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:56.258+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:56.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.264+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f800415c550 con 0x7f80040c7a10 2024-09-16T00:08:56.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.266+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f800415c550 con 0x7f80040c7a10 2024-09-16T00:08:56.267 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:56.268 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:56.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.267+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f80040c7f70 0x7f80040c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.267+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f8004159030 con 0x7f80040c7f70 2024-09-16T00:08:56.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.267+0000 7f8003fff640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f80040c7f70 0x7f80040c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.267+0000 7f8003fff640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f80040c7f70 0x7f80040c8350 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.268+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8004159030 con 0x7f80040c7f70 2024-09-16T00:08:56.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.286+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f80040c8ae0 con 0x7f80040c7f70 2024-09-16T00:08:56.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.289+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f80040c8ae0 con 0x7f80040c7f70 2024-09-16T00:08:56.290 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:56.290 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:56.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.289+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8004062b40 0x7f8004062f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.289+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f80040634c0 con 0x7f8004062b40 2024-09-16T00:08:56.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.289+0000 7f80037fe640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8004062b40 0x7f8004062f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.290+0000 7f80037fe640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8004062b40 0x7f8004062f80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.291+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f80040634c0 con 0x7f8004062b40 2024-09-16T00:08:56.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.307+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f80040b2fc0 con 0x7f8004062b40 2024-09-16T00:08:56.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.309+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f80040b2fc0 con 0x7f8004062b40 2024-09-16T00:08:56.310 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:56.311 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:56.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.310+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8004204c30 0x7f8004207020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.310+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f80040b2f40 con 0x7f8004204c30 2024-09-16T00:08:56.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.310+0000 7f8008d61640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8004204c30 0x7f8004207020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.310+0000 7f8008d61640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8004204c30 0x7f8004207020 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.311+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f80040b2f40 con 0x7f8004204c30 2024-09-16T00:08:56.313 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:56.313 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:56.323 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:08:56.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.329+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f80040b2f40 con 0x7f8004204c30 2024-09-16T00:08:56.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.330+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f80040b2f40 con 0x7f8004204c30 2024-09-16T00:08:56.333 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:56.334 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:56.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.331+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f8004207a70 0x7f8004209e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.331+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f80040b3c10 con 0x7f8004207a70 2024-09-16T00:08:56.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.331+0000 7f8003fff640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f8004207a70 0x7f8004209e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.331+0000 7f8003fff640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f8004207a70 0x7f8004209e60 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.332+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f80040b3c10 con 0x7f8004207a70 2024-09-16T00:08:56.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.350+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f80040b3c10 con 0x7f8004207a70 2024-09-16T00:08:56.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.352+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f80040b3c10 con 0x7f8004207a70 2024-09-16T00:08:56.353 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:56.354 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:56.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.352+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f800420a8b0 0x7f800420cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.352+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f800420fa70 con 0x7f800420a8b0 2024-09-16T00:08:56.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.353+0000 7f80037fe640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f800420a8b0 0x7f800420cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.353+0000 7f80037fe640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f800420a8b0 0x7f800420cca0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.354+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f800420fa70 con 0x7f800420a8b0 2024-09-16T00:08:56.367 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:56.367+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:56.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.373+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f800420d1e0 con 0x7f800420a8b0 2024-09-16T00:08:56.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.374+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f800420d1e0 con 0x7f800420a8b0 2024-09-16T00:08:56.375 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:56.376 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:56.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.375+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f800420d900 0x7f800420fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.375+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f8004212b90 con 0x7f800420d900 2024-09-16T00:08:56.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.375+0000 7f8008d61640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f800420d900 0x7f800420fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.375+0000 7f8008d61640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f800420d900 0x7f800420fcf0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.376+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8004212b90 con 0x7f800420d900 2024-09-16T00:08:56.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.394+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f80040755b0 con 0x7f800420d900 2024-09-16T00:08:56.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.396+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f80040755b0 con 0x7f800420d900 2024-09-16T00:08:56.397 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:56.397 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:56.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.396+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8004210740 0x7f8004210bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:56.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.396+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f8004215fe0 con 0x7f8004210740 2024-09-16T00:08:56.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.396+0000 7f8003fff640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8004210740 0x7f8004210bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:56.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.397+0000 7f8003fff640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8004210740 0x7f8004210bc0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:56.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.397+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8004215fe0 con 0x7f8004210740 2024-09-16T00:08:56.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.410+0000 7fe8a4aba640 1 -- 172.21.15.139:0/3275165472 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fe8a00c6000 con 0x7fe8a00ccc90 2024-09-16T00:08:56.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.415+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8004211210 con 0x7f8004210740 2024-09-16T00:08:56.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.416+0000 7f8000ff9640 1 -- 172.21.15.139:0/3161308764 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8004211210 con 0x7f8004210740 2024-09-16T00:08:56.417 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:56.419 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:56.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f80040c7a10 msgr2=0x7f800415acd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f80040c7a10 0x7f800415acd0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f80040c7f70 msgr2=0x7f80040c8350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f80040c7f70 0x7f80040c8350 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8004062b40 msgr2=0x7f8004062f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8004062b40 0x7f8004062f80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8004204c30 msgr2=0x7f8004207020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8004204c30 0x7f8004207020 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f8008d61640 1 -- 172.21.15.139:0/3161308764 reap_dead start 2024-09-16T00:08:56.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f8004207a70 msgr2=0x7f8004209e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f8004207a70 0x7f8004209e60 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f8008d61640 1 -- 172.21.15.139:0/3161308764 reap_dead start 2024-09-16T00:08:56.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f800420a8b0 msgr2=0x7f800420cca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f800420a8b0 0x7f800420cca0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f800420d900 msgr2=0x7f800420fcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f800420d900 0x7f800420fcf0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8004210740 msgr2=0x7f8004210bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8004210740 0x7f8004210bc0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7ff006c2e0 msgr2=0x7f7ff006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7ff006c2e0 0x7f7ff006e7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40090b0 tx=0x7f7ff402e000 comp rx=0 tx=0).stop 2024-09-16T00:08:56.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f80040d0870 msgr2=0x7f800415c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 --2- 172.21.15.139:0/3161308764 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f80040d0870 0x7f800415c010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00efc0 tx=0x7f7ffc00c490 comp rx=0 tx=0).stop 2024-09-16T00:08:56.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f8008d61640 1 -- 172.21.15.139:0/3161308764 reap_dead start 2024-09-16T00:08:56.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 shutdown_connections 2024-09-16T00:08:56.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 >> 172.21.15.139:0/3161308764 conn(0x7f800406c520 msgr2=0x7f80040ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:56.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 shutdown_connections 2024-09-16T00:08:56.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.417+0000 7f7fea7fc640 1 -- 172.21.15.139:0/3161308764 wait complete. 2024-09-16T00:08:56.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.420+0000 7fbad3c53640 1 -- 172.21.15.139:0/4247989383 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) -- 0x7fbacc067ad0 con 0x7fbacc0ce810 2024-09-16T00:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:56.439 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:56.449 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:08:56.490 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:56.490+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:56.561 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:56.561+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:56.604 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:56.605 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:56.605 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:56.605 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:56.605 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:56.615 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:08:56.661 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:56.661+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:56.762 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:56.762+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:56.776 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.159245702000007, 2024-09-16T00:08:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056100969999999997, 2024-09-16T00:08:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.777 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:08:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615352+0000", 2024-09-16T00:08:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9859999999999999e-06 2024-09-16T00:08:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615363+0000", 2024-09-16T00:08:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1168999999999999e-05 2024-09-16T00:08:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615382+0000", 2024-09-16T00:08:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8224e-05 2024-09-16T00:08:56.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615393+0000", 2024-09-16T00:08:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0925e-05 2024-09-16T00:08:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615624+0000", 2024-09-16T00:08:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023159600000000001 2024-09-16T00:08:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615721+0000", 2024-09-16T00:08:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7040000000000006e-05 2024-09-16T00:08:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615885+0000", 2024-09-16T00:08:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163456 2024-09-16T00:08:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:08:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002191368 2024-09-16T00:08:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620870+0000", 2024-09-16T00:08:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941839999999999 2024-09-16T00:08:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620922+0000", 2024-09-16T00:08:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1641999999999998e-05 2024-09-16T00:08:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620956+0000", 2024-09-16T00:08:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4507e-05 2024-09-16T00:08:56.782 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.159155342999995, 2024-09-16T00:08:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005695739, 2024-09-16T00:08:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:08:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4900000000000005e-07 2024-09-16T00:08:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615442+0000", 2024-09-16T00:08:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.994e-06 2024-09-16T00:08:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:08:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1081e-05 2024-09-16T00:08:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615473+0000", 2024-09-16T00:08:56.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1910000000000006e-06 2024-09-16T00:08:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616230+0000", 2024-09-16T00:08:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075700400000000003 2024-09-16T00:08:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616305+0000", 2024-09-16T00:08:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4759999999999996e-05 2024-09-16T00:08:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616466+0000", 2024-09-16T00:08:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000161714 2024-09-16T00:08:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619821+0000", 2024-09-16T00:08:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033547339999999998 2024-09-16T00:08:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621028+0000", 2024-09-16T00:08:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012067530000000001 2024-09-16T00:08:56.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621087+0000", 2024-09-16T00:08:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9364999999999998e-05 2024-09-16T00:08:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621132+0000", 2024-09-16T00:08:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5293999999999997e-05 2024-09-16T00:08:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.159093111000004, 2024-09-16T00:08:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050565760000000001, 2024-09-16T00:08:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:56.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:08:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615500+0000", 2024-09-16T00:08:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8199999999999999e-07 2024-09-16T00:08:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615505+0000", 2024-09-16T00:08:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5280000000000003e-06 2024-09-16T00:08:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:08:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0317e-05 2024-09-16T00:08:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615523+0000", 2024-09-16T00:08:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4580000000000004e-06 2024-09-16T00:08:56.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615607+0000", 2024-09-16T00:08:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4153999999999994e-05 2024-09-16T00:08:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615748+0000", 2024-09-16T00:08:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140533 2024-09-16T00:08:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:56.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615911+0000", 2024-09-16T00:08:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163678 2024-09-16T00:08:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619944+0000", 2024-09-16T00:08:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040330909999999999 2024-09-16T00:08:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620471+0000", 2024-09-16T00:08:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052645600000000002 2024-09-16T00:08:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620512+0000", 2024-09-16T00:08:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0738000000000002e-05 2024-09-16T00:08:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620555+0000", 2024-09-16T00:08:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3841000000000003e-05 2024-09-16T00:08:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.794 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.159041004000002, 2024-09-16T00:08:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089162479999999999, 2024-09-16T00:08:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:08:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615552+0000", 2024-09-16T00:08:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:08:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615556+0000", 2024-09-16T00:08:56.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3610000000000003e-06 2024-09-16T00:08:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615566+0000", 2024-09-16T00:08:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6099999999999995e-06 2024-09-16T00:08:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615574+0000", 2024-09-16T00:08:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9379999999999992e-06 2024-09-16T00:08:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616777+0000", 2024-09-16T00:08:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012031640000000001 2024-09-16T00:08:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616882+0000", 2024-09-16T00:08:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105146 2024-09-16T00:08:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619696+0000", 2024-09-16T00:08:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028142179999999998 2024-09-16T00:08:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624234+0000", 2024-09-16T00:08:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045381889999999998 2024-09-16T00:08:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624393+0000", 2024-09-16T00:08:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015871200000000001 2024-09-16T00:08:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624434+0000", 2024-09-16T00:08:56.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0908999999999997e-05 2024-09-16T00:08:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624467+0000", 2024-09-16T00:08:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3217999999999999e-05 2024-09-16T00:08:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.158971206999993, 2024-09-16T00:08:56.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064898359999999997, 2024-09-16T00:08:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:08:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615622+0000", 2024-09-16T00:08:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.181e-06 2024-09-16T00:08:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615628+0000", 2024-09-16T00:08:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2450000000000003e-06 2024-09-16T00:08:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:08:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2554999999999999e-05 2024-09-16T00:08:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615650+0000", 2024-09-16T00:08:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4350000000000003e-06 2024-09-16T00:08:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616772+0000", 2024-09-16T00:08:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001122058 2024-09-16T00:08:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616889+0000", 2024-09-16T00:08:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116623 2024-09-16T00:08:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617035+0000", 2024-09-16T00:08:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014579799999999999 2024-09-16T00:08:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619829+0000", 2024-09-16T00:08:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941379999999998 2024-09-16T00:08:56.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622024+0000", 2024-09-16T00:08:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021949920000000002 2024-09-16T00:08:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622087+0000", 2024-09-16T00:08:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3048999999999999e-05 2024-09-16T00:08:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622111+0000", 2024-09-16T00:08:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3762000000000001e-05 2024-09-16T00:08:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.158908646, 2024-09-16T00:08:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065192160000000004, 2024-09-16T00:08:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.806 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:08:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615684+0000", 2024-09-16T00:08:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8299999999999995e-07 2024-09-16T00:08:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615691+0000", 2024-09-16T00:08:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.866e-06 2024-09-16T00:08:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615707+0000", 2024-09-16T00:08:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6073999999999999e-05 2024-09-16T00:08:56.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615714+0000", 2024-09-16T00:08:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-16T00:08:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617403+0000", 2024-09-16T00:08:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016881629999999999 2024-09-16T00:08:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617559+0000", 2024-09-16T00:08:56.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015669699999999999 2024-09-16T00:08:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:08:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000144165 2024-09-16T00:08:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621413+0000", 2024-09-16T00:08:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037092200000000001 2024-09-16T00:08:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622147+0000", 2024-09-16T00:08:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073468800000000005 2024-09-16T00:08:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622180+0000", 2024-09-16T00:08:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2901999999999998e-05 2024-09-16T00:08:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622203+0000", 2024-09-16T00:08:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2211000000000001e-05 2024-09-16T00:08:56.811 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.158865551999995, 2024-09-16T00:08:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065825270000000003, 2024-09-16T00:08:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:08:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:08:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7299999999999996e-07 2024-09-16T00:08:56.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615730+0000", 2024-09-16T00:08:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.004e-06 2024-09-16T00:08:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615737+0000", 2024-09-16T00:08:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4789999999999999e-06 2024-09-16T00:08:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615745+0000", 2024-09-16T00:08:56.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6699999999999994e-06 2024-09-16T00:08:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618115+0000", 2024-09-16T00:08:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023701569999999999 2024-09-16T00:08:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618247+0000", 2024-09-16T00:08:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013185000000000001 2024-09-16T00:08:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622309+0000", 2024-09-16T00:08:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040617939999999996 2024-09-16T00:08:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.158838302999996, 2024-09-16T00:08:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036685950000000002, 2024-09-16T00:08:56.816 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:08:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:08:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T00:08:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615763+0000", 2024-09-16T00:08:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4360000000000002e-06 2024-09-16T00:08:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615770+0000", 2024-09-16T00:08:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7540000000000006e-06 2024-09-16T00:08:56.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615803+0000", 2024-09-16T00:08:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3008e-05 2024-09-16T00:08:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615888+0000", 2024-09-16T00:08:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5056999999999996e-05 2024-09-16T00:08:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:08:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013059700000000001 2024-09-16T00:08:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619422+0000", 2024-09-16T00:08:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034033230000000002 2024-09-16T00:08:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.820 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:16 2.1d e22/19 2:b871830b:::605.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.154849713000004, 2024-09-16T00:08:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045616900000000002, 2024-09-16T00:08:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:08:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:08:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:08:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619755+0000", 2024-09-16T00:08:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3599999999999996e-06 2024-09-16T00:08:56.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619767+0000", 2024-09-16T00:08:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2805e-05 2024-09-16T00:08:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619787+0000", 2024-09-16T00:08:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9264000000000001e-05 2024-09-16T00:08:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619869+0000", 2024-09-16T00:08:56.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2639999999999995e-05 2024-09-16T00:08:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619895+0000", 2024-09-16T00:08:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5890000000000001e-05 2024-09-16T00:08:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624264+0000", 2024-09-16T00:08:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043690049999999996 2024-09-16T00:08:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624304+0000", 2024-09-16T00:08:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9807e-05 2024-09-16T00:08:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:17 2.1d e22/19 2:b871830b:::605.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.154497004000007, 2024-09-16T00:08:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042370910000000001, 2024-09-16T00:08:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.825 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:08:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620096+0000", 2024-09-16T00:08:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:08:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620102+0000", 2024-09-16T00:08:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7220000000000002e-06 2024-09-16T00:08:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620112+0000", 2024-09-16T00:08:56.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0156999999999999e-05 2024-09-16T00:08:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620120+0000", 2024-09-16T00:08:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2880000000000004e-06 2024-09-16T00:08:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620171+0000", 2024-09-16T00:08:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.168e-05 2024-09-16T00:08:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620189+0000", 2024-09-16T00:08:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7419e-05 2024-09-16T00:08:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624306+0000", 2024-09-16T00:08:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041175430000000004 2024-09-16T00:08:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624332+0000", 2024-09-16T00:08:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5717000000000001e-05 2024-09-16T00:08:56.829 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 78.152130186999997, 2024-09-16T00:08:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044651079999999998, 2024-09-16T00:08:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:08:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622463+0000", 2024-09-16T00:08:56.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5349999999999999e-06 2024-09-16T00:08:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622470+0000", 2024-09-16T00:08:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7050000000000002e-06 2024-09-16T00:08:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622485+0000", 2024-09-16T00:08:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4978e-05 2024-09-16T00:08:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622493+0000", 2024-09-16T00:08:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2139999999999996e-06 2024-09-16T00:08:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622546+0000", 2024-09-16T00:08:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:08:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622595+0000", 2024-09-16T00:08:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9112e-05 2024-09-16T00:08:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622677+0000", 2024-09-16T00:08:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2332999999999999e-05 2024-09-16T00:08:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.623515+0000", 2024-09-16T00:08:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000837615 2024-09-16T00:08:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626867+0000", 2024-09-16T00:08:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033524819999999999 2024-09-16T00:08:56.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626902+0000", 2024-09-16T00:08:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4310000000000002e-05 2024-09-16T00:08:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626927+0000", 2024-09-16T00:08:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5128999999999999e-05 2024-09-16T00:08:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:40 2.14 2:2e2fa760:::200.00000001:head [write 940~443 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 65.935264794000005, 2024-09-16T00:08:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016643039999999999, 2024-09-16T00:08:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:56.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839327+0000", 2024-09-16T00:08:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839329+0000", 2024-09-16T00:08:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2330000000000001e-06 2024-09-16T00:08:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839343+0000", 2024-09-16T00:08:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3634e-05 2024-09-16T00:08:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839362+0000", 2024-09-16T00:08:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9321999999999999e-05 2024-09-16T00:08:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839375+0000", 2024-09-16T00:08:56.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2945000000000001e-05 2024-09-16T00:08:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839462+0000", 2024-09-16T00:08:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6371000000000004e-05 2024-09-16T00:08:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839547+0000", 2024-09-16T00:08:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4988999999999996e-05 2024-09-16T00:08:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.839657+0000", 2024-09-16T00:08:56.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110501 2024-09-16T00:08:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840598+0000", 2024-09-16T00:08:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00094087100000000003 2024-09-16T00:08:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840918+0000", 2024-09-16T00:08:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031968799999999999 2024-09-16T00:08:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840955+0000", 2024-09-16T00:08:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7360999999999997e-05 2024-09-16T00:08:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:50.840991+0000", 2024-09-16T00:08:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6389e-05 2024-09-16T00:08:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:41 2.14 2:2e2fa760:::200.00000001:head [write 1383~2831 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:56.841 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 63.661611348999998, 2024-09-16T00:08:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017075809999999999, 2024-09-16T00:08:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112981+0000", 2024-09-16T00:08:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112983+0000", 2024-09-16T00:08:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2560000000000001e-06 2024-09-16T00:08:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.112993+0000", 2024-09-16T00:08:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0423e-05 2024-09-16T00:08:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113010+0000", 2024-09-16T00:08:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7169000000000001e-05 2024-09-16T00:08:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113023+0000", 2024-09-16T00:08:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2111e-05 2024-09-16T00:08:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113099+0000", 2024-09-16T00:08:56.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6798000000000006e-05 2024-09-16T00:08:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113174+0000", 2024-09-16T00:08:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.47e-05 2024-09-16T00:08:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.113297+0000", 2024-09-16T00:08:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012250799999999999 2024-09-16T00:08:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114284+0000", 2024-09-16T00:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00098720399999999999 2024-09-16T00:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114604+0000", 2024-09-16T00:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031988699999999998 2024-09-16T00:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114644+0000", 2024-09-16T00:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9919999999999997e-05 2024-09-16T00:08:56.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:53.114688+0000", 2024-09-16T00:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4604999999999997e-05 2024-09-16T00:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:68 3.a 3:53d16f76:::1000000023b.00000000:head [write 0~1274] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.415151+0000", 2024-09-16T00:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.35944091099999997, 2024-09-16T00:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020653619999999998, 2024-09-16T00:08:56.847 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 68 2024-09-16T00:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.415151+0000", 2024-09-16T00:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.415151+0000", 2024-09-16T00:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.415153+0000", 2024-09-16T00:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.559e-06 2024-09-16T00:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.415160+0000", 2024-09-16T00:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2219999999999996e-06 2024-09-16T00:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.415175+0000", 2024-09-16T00:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4854999999999999e-05 2024-09-16T00:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.415182+0000", 2024-09-16T00:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9630000000000001e-06 2024-09-16T00:08:56.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.415242+0000", 2024-09-16T00:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0627000000000001e-05 2024-09-16T00:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.415332+0000", 2024-09-16T00:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0020999999999998e-05 2024-09-16T00:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.415406+0000", 2024-09-16T00:08:56.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4047000000000004e-05 2024-09-16T00:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.416839+0000", 2024-09-16T00:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014323910000000001 2024-09-16T00:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.417161+0000", 2024-09-16T00:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032209699999999998 2024-09-16T00:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.417191+0000", 2024-09-16T00:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9728999999999999e-05 2024-09-16T00:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.417216+0000", 2024-09-16T00:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5851e-05 2024-09-16T00:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.853 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:71 3.10 3:08500202:::1000000023e.00000000:head [write 0~1105] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.422609+0000", 2024-09-16T00:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.351983184, 2024-09-16T00:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013521239999999999, 2024-09-16T00:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 71 2024-09-16T00:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.422609+0000", 2024-09-16T00:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.422609+0000", 2024-09-16T00:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.422610+0000", 2024-09-16T00:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6199999999999999e-06 2024-09-16T00:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.422619+0000", 2024-09-16T00:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1349999999999992e-06 2024-09-16T00:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.422634+0000", 2024-09-16T00:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4975e-05 2024-09-16T00:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.422647+0000", 2024-09-16T00:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3274e-05 2024-09-16T00:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.422704+0000", 2024-09-16T00:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7370000000000001e-05 2024-09-16T00:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.422825+0000", 2024-09-16T00:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120345 2024-09-16T00:08:56.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.422950+0000", 2024-09-16T00:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125931 2024-09-16T00:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.423571+0000", 2024-09-16T00:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00062045100000000005 2024-09-16T00:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:56.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.423865+0000", 2024-09-16T00:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029406899999999999 2024-09-16T00:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.423920+0000", 2024-09-16T00:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4970999999999999e-05 2024-09-16T00:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.423961+0000", 2024-09-16T00:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0982999999999997e-05 2024-09-16T00:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.859 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:73 3.26 3:65e03fee:::10000000240.00000000:head [write 0~1307] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.426597+0000", 2024-09-16T00:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.347995322, 2024-09-16T00:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002101201, 2024-09-16T00:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:08:56.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 73 2024-09-16T00:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.426597+0000", 2024-09-16T00:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.426597+0000", 2024-09-16T00:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.426597+0000", 2024-09-16T00:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2900000000000003e-07 2024-09-16T00:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.426601+0000", 2024-09-16T00:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9380000000000002e-06 2024-09-16T00:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.426611+0000", 2024-09-16T00:08:56.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.251e-06 2024-09-16T00:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.426616+0000", 2024-09-16T00:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9790000000000001e-06 2024-09-16T00:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.426665+0000", 2024-09-16T00:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9889999999999998e-05 2024-09-16T00:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.426743+0000", 2024-09-16T00:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7482000000000001e-05 2024-09-16T00:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.426817+0000", 2024-09-16T00:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4394999999999995e-05 2024-09-16T00:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428414+0000", 2024-09-16T00:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001596262 2024-09-16T00:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428624+0000", 2024-09-16T00:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021063200000000001 2024-09-16T00:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428657+0000", 2024-09-16T00:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2499999999999997e-05 2024-09-16T00:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428698+0000", 2024-09-16T00:08:56.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1143000000000002e-05 2024-09-16T00:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:103 3.26 3:6519881c:::1000000025e.00000000:head [write 0~2892] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.487113+0000", 2024-09-16T00:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.28747903800000002, 2024-09-16T00:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034171290000000001, 2024-09-16T00:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:56.866 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103 2024-09-16T00:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487113+0000", 2024-09-16T00:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487113+0000", 2024-09-16T00:08:56.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487114+0000", 2024-09-16T00:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3030000000000001e-06 2024-09-16T00:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487122+0000", 2024-09-16T00:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3889999999999999e-06 2024-09-16T00:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487133+0000", 2024-09-16T00:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1325000000000001e-05 2024-09-16T00:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487138+0000", 2024-09-16T00:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0660000000000003e-06 2024-09-16T00:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487182+0000", 2024-09-16T00:08:56.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3655999999999997e-05 2024-09-16T00:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487307+0000", 2024-09-16T00:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012541599999999999 2024-09-16T00:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487391+0000", 2024-09-16T00:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4404e-05 2024-09-16T00:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:56.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487908+0000", 2024-09-16T00:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000516608 2024-09-16T00:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.490429+0000", 2024-09-16T00:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025209339999999998 2024-09-16T00:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.490482+0000", 2024-09-16T00:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2655000000000003e-05 2024-09-16T00:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.490530+0000", 2024-09-16T00:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8372999999999999e-05 2024-09-16T00:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:109 3.2c 3:36882206:::10000000265.00000000:head [write 0~2269] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.503797+0000", 2024-09-16T00:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.27079462599999998, 2024-09-16T00:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001280389, 2024-09-16T00:08:56.872 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 109 2024-09-16T00:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.503797+0000", 2024-09-16T00:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.503797+0000", 2024-09-16T00:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.503799+0000", 2024-09-16T00:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.39e-06 2024-09-16T00:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.503806+0000", 2024-09-16T00:08:56.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2219999999999996e-06 2024-09-16T00:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.503817+0000", 2024-09-16T00:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0991e-05 2024-09-16T00:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.503825+0000", 2024-09-16T00:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7430000000000002e-06 2024-09-16T00:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.503880+0000", 2024-09-16T00:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4991999999999998e-05 2024-09-16T00:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.503972+0000", 2024-09-16T00:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2429999999999994e-05 2024-09-16T00:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.504054+0000", 2024-09-16T00:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2095999999999995e-05 2024-09-16T00:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.504499+0000", 2024-09-16T00:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00044441899999999999 2024-09-16T00:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.504986+0000", 2024-09-16T00:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048715100000000001 2024-09-16T00:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.505036+0000", 2024-09-16T00:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0135e-05 2024-09-16T00:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.505078+0000", 2024-09-16T00:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1820000000000003e-05 2024-09-16T00:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:118 3.26 3:6559b46a:::1000000026e.00000000:head [write 0~59119] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:08:56.878 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.525492+0000", 2024-09-16T00:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.24910006700000001, 2024-09-16T00:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001401708, 2024-09-16T00:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 118 2024-09-16T00:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.525492+0000", 2024-09-16T00:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.525492+0000", 2024-09-16T00:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.525494+0000", 2024-09-16T00:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7999999999999999e-06 2024-09-16T00:08:56.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.525525+0000", 2024-09-16T00:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1050999999999998e-05 2024-09-16T00:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.525542+0000", 2024-09-16T00:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7380999999999998e-05 2024-09-16T00:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.525551+0000", 2024-09-16T00:08:56.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6090000000000001e-06 2024-09-16T00:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.525605+0000", 2024-09-16T00:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3949999999999997e-05 2024-09-16T00:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.525714+0000", 2024-09-16T00:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010889499999999999 2024-09-16T00:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.525808+0000", 2024-09-16T00:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4772000000000007e-05 2024-09-16T00:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.526512+0000", 2024-09-16T00:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00070334800000000002 2024-09-16T00:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.526832+0000", 2024-09-16T00:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00031998299999999998 2024-09-16T00:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.526865+0000", 2024-09-16T00:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3309999999999998e-05 2024-09-16T00:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.526894+0000", 2024-09-16T00:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8609e-05 2024-09-16T00:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:196 3.36 3:6fd73218:::100000002bd.00000000:head [write 0~98] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.695066+0000", 2024-09-16T00:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.079526330000000006, 2024-09-16T00:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018856649999999999, 2024-09-16T00:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 196 2024-09-16T00:08:56.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.695066+0000", 2024-09-16T00:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.695066+0000", 2024-09-16T00:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:56.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.695068+0000", 2024-09-16T00:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9819999999999998e-06 2024-09-16T00:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.695075+0000", 2024-09-16T00:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.374e-06 2024-09-16T00:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.695092+0000", 2024-09-16T00:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6742e-05 2024-09-16T00:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.695102+0000", 2024-09-16T00:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0699e-05 2024-09-16T00:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.695156+0000", 2024-09-16T00:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3517999999999998e-05 2024-09-16T00:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.695238+0000", 2024-09-16T00:08:56.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2470000000000002e-05 2024-09-16T00:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.695320+0000", 2024-09-16T00:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1403000000000006e-05 2024-09-16T00:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.695864+0000", 2024-09-16T00:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00054465799999999995 2024-09-16T00:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.696895+0000", 2024-09-16T00:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001030785 2024-09-16T00:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.696926+0000", 2024-09-16T00:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1148000000000001e-05 2024-09-16T00:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.696951+0000", 2024-09-16T00:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4885999999999999e-05 2024-09-16T00:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.890 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:56.891 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:56.892 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:08:56.893 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:56.892+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:56.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.980+0000 7fbac27fc640 1 -- 172.21.15.139:0/4247989383 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v58) ==== 113+0+0 (secure 0 0 0) 0x7fbacc1ebde0 con 0x7fbacc0ce810 2024-09-16T00:08:56.982 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T00:08:56.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.983+0000 7fbad3c53640 1 -- 172.21.15.139:0/4247989383 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbab006c300 msgr2=0x7fbab006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.983+0000 7fbad3c53640 1 --2- 172.21.15.139:0/4247989383 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbab006c300 0x7fbab006e7c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbac8034000 tx=0x7fbac802e000 comp rx=0 tx=0).stop 2024-09-16T00:08:56.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.983+0000 7fbad3c53640 1 -- 172.21.15.139:0/4247989383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbacc0ce810 msgr2=0x7fbacc0c1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.983+0000 7fbad3c53640 1 --2- 172.21.15.139:0/4247989383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbacc0ce810 0x7fbacc0c1eb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbac400efc0 tx=0x7fbac400c490 comp rx=0 tx=0).stop 2024-09-16T00:08:56.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.983+0000 7fbad3c53640 1 -- 172.21.15.139:0/4247989383 shutdown_connections 2024-09-16T00:08:56.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.983+0000 7fbad3c53640 1 --2- 172.21.15.139:0/4247989383 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbab006c300 0x7fbab006e7c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.983+0000 7fbad3c53640 1 --2- 172.21.15.139:0/4247989383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbacc0ce810 0x7fbacc0c1eb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.983+0000 7fbad3c53640 1 --2- 172.21.15.139:0/4247989383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbacc0ce430 0x7fbacc0cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.983+0000 7fbad3c53640 1 --2- 172.21.15.139:0/4247989383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbacc0c9430 0x7fbacc06d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.983+0000 7fbad3c53640 1 -- 172.21.15.139:0/4247989383 >> 172.21.15.139:0/4247989383 conn(0x7fbacc06c150 msgr2=0x7fbacc0ca5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:56.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.983+0000 7fbad3c53640 1 -- 172.21.15.139:0/4247989383 shutdown_connections 2024-09-16T00:08:56.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.983+0000 7fbad3c53640 1 -- 172.21.15.139:0/4247989383 wait complete. 2024-09-16T00:08:56.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.984+0000 7fe88effd640 1 -- 172.21.15.139:0/3275165472 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v58) ==== 97+0+0 (secure 0 0 0) 0x7fe89407e040 con 0x7fe8a00ccc90 2024-09-16T00:08:56.985 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is set 2024-09-16T00:08:56.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.987+0000 7fe8a4aba640 1 -- 172.21.15.139:0/3275165472 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe88006c2e0 msgr2=0x7fe88006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.987+0000 7fe8a4aba640 1 --2- 172.21.15.139:0/3275165472 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe88006c2e0 0x7fe88006e7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe890007fd0 tx=0x7fe890007de0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.987+0000 7fe8a4aba640 1 -- 172.21.15.139:0/3275165472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a00ccc90 msgr2=0x7fe8a01497e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:56.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.987+0000 7fe8a4aba640 1 --2- 172.21.15.139:0/3275165472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a00ccc90 0x7fe8a01497e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe894000c00 tx=0x7fe8940046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.987+0000 7fe8a4aba640 1 -- 172.21.15.139:0/3275165472 shutdown_connections 2024-09-16T00:08:56.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.987+0000 7fe8a4aba640 1 --2- 172.21.15.139:0/3275165472 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe88006c2e0 0x7fe88006e7a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.987+0000 7fe8a4aba640 1 --2- 172.21.15.139:0/3275165472 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a0154c20 0x7fe8a0149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.987+0000 7fe8a4aba640 1 --2- 172.21.15.139:0/3275165472 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a00ccc90 0x7fe8a01497e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.987+0000 7fe8a4aba640 1 --2- 172.21.15.139:0/3275165472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a0068060 0x7fe8a014eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:56.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.987+0000 7fe8a4aba640 1 -- 172.21.15.139:0/3275165472 >> 172.21.15.139:0/3275165472 conn(0x7fe8a006ca40 msgr2=0x7fe8a00c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:56.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.987+0000 7fe8a4aba640 1 -- 172.21.15.139:0/3275165472 shutdown_connections 2024-09-16T00:08:56.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:56.987+0000 7fe8a4aba640 1 -- 172.21.15.139:0/3275165472 wait complete. 2024-09-16T00:08:56.992 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:56.992+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:56.998 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T00:08:57.048 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:57.048 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:57.048 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:08:57.048 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:57.058 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:08:57.094 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:57.093+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:57.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.142+0000 7f1ab21a6640 1 Processor -- start 2024-09-16T00:08:57.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.142+0000 7f1ab21a6640 1 -- start start 2024-09-16T00:08:57.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.143+0000 7f1ab21a6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1aac150bd0 0x7f1aac0718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.143+0000 7f1ab21a6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aac073230 con 0x7f1aac15afc0 2024-09-16T00:08:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.143+0000 7f1ab21a6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aac072020 con 0x7f1aac150fb0 2024-09-16T00:08:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.143+0000 7f1ab21a6640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f1aac071df0 con 0x7f1aac150bd0 2024-09-16T00:08:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.143+0000 7f1aabfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f1aac15afc0 0x7f1aac0c4a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:44452/0 (socket says 172.21.15.139:44452) 2024-09-16T00:08:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.143+0000 7f1aabfff640 1 -- 172.21.15.139:0/2982403697 learned_addr learned my addr 172.21.15.139:0/2982403697 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.143+0000 7f1aab7fe640 1 --2- 172.21.15.139:0/2982403697 >> v2:172.21.15.164:3300/0 conn(0x7f1aac150bd0 0x7f1aac0718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.144+0000 7f1aaaffd640 1 -- 172.21.15.139:0/2982403697 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 664909040 0 0) 0x7f1aac073230 con 0x7f1aac15afc0 2024-09-16T00:08:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.144+0000 7f1aaaffd640 1 -- 172.21.15.139:0/2982403697 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a98003520 con 0x7f1aac15afc0 2024-09-16T00:08:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.144+0000 7f1aab7fe640 1 -- 172.21.15.139:0/2982403697 >> v1:172.21.15.164:6789/0 conn(0x7f1aac150fb0 legacy=0x7f1aac157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.144+0000 7f1aab7fe640 1 -- 172.21.15.139:0/2982403697 >> v1:172.21.15.139:6789/0 conn(0x7f1aac15afc0 legacy=0x7f1aac0c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.144+0000 7f1aab7fe640 1 -- 172.21.15.139:0/2982403697 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aac0722c0 con 0x7f1aac150bd0 2024-09-16T00:08:57.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.144+0000 7f1aab7fe640 1 --2- 172.21.15.139:0/2982403697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac150bd0 0x7f1aac0718b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00a940 tx=0x7f1a9c02b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae4690d27c84544d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:57.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.145+0000 7f1aaaffd640 1 -- 172.21.15.139:0/2982403697 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c00b040 con 0x7f1aac150bd0 2024-09-16T00:08:57.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.145+0000 7f1aaaffd640 1 -- 172.21.15.139:0/2982403697 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a9c030710 con 0x7f1aac150bd0 2024-09-16T00:08:57.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.145+0000 7f1aaaffd640 1 -- 172.21.15.139:0/2982403697 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c030ac0 con 0x7f1aac150bd0 2024-09-16T00:08:57.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.145+0000 7f1ab21a6640 1 -- 172.21.15.139:0/2982403697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac150bd0 msgr2=0x7f1aac0718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:57.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.145+0000 7f1ab21a6640 1 --2- 172.21.15.139:0/2982403697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac150bd0 0x7f1aac0718b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00a940 tx=0x7f1a9c02b670 comp rx=0 tx=0).stop 2024-09-16T00:08:57.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.145+0000 7f1ab21a6640 1 -- 172.21.15.139:0/2982403697 shutdown_connections 2024-09-16T00:08:57.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.145+0000 7f1ab21a6640 1 --2- 172.21.15.139:0/2982403697 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac150bd0 0x7f1aac0718b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:57.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.145+0000 7f1ab21a6640 1 -- 172.21.15.139:0/2982403697 >> 172.21.15.139:0/2982403697 conn(0x7f1aac06da40 msgr2=0x7f1aac06f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:57.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.145+0000 7f1ab21a6640 1 -- 172.21.15.139:0/2982403697 shutdown_connections 2024-09-16T00:08:57.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.145+0000 7f1ab21a6640 1 -- 172.21.15.139:0/2982403697 wait complete. 2024-09-16T00:08:57.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.146+0000 7f1ab21a6640 1 Processor -- start 2024-09-16T00:08:57.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.146+0000 7f1ab21a6640 1 -- start start 2024-09-16T00:08:57.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.147+0000 7f1ab21a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1aac150bd0 0x7f1aac158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:57.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.147+0000 7f1ab21a6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac150fb0 0x7f1aac158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:57.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.147+0000 7f1ab21a6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1aac15afc0 0x7f1aac1f3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:57.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.147+0000 7f1ab21a6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac15a7c0 con 0x7f1aac150bd0 2024-09-16T00:08:57.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.147+0000 7f1ab21a6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1aac15ab30 con 0x7f1aac150fb0 2024-09-16T00:08:57.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.147+0000 7f1ab21a6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1aac06fb70 con 0x7f1aac15afc0 2024-09-16T00:08:57.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.147+0000 7f1aab7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1aac150bd0 0x7f1aac158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:57.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.147+0000 7f1aab7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1aac150bd0 0x7f1aac158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59444/0 (socket says 172.21.15.139:59444) 2024-09-16T00:08:57.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.147+0000 7f1aab7fe640 1 -- 172.21.15.139:0/2261865974 learned_addr learned my addr 172.21.15.139:0/2261865974 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:57.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.147+0000 7f1aa3fff640 1 --2- 172.21.15.139:0/2261865974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac150fb0 0x7f1aac158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:57.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.147+0000 7f1aabfff640 1 --2- 172.21.15.139:0/2261865974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1aac15afc0 0x7f1aac1f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:57.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.147+0000 7f1aab7fe640 1 -- 172.21.15.139:0/2261865974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1aac15afc0 msgr2=0x7f1aac1f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:57.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.147+0000 7f1aab7fe640 1 --2- 172.21.15.139:0/2261865974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1aac15afc0 0x7f1aac1f3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:57.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.147+0000 7f1aab7fe640 1 -- 172.21.15.139:0/2261865974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac150fb0 msgr2=0x7f1aac158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:57.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.147+0000 7f1aab7fe640 1 --2- 172.21.15.139:0/2261865974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac150fb0 0x7f1aac158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:57.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.147+0000 7f1aab7fe640 1 -- 172.21.15.139:0/2261865974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aac1f4190 con 0x7f1aac150bd0 2024-09-16T00:08:57.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.148+0000 7f1aab7fe640 1 --2- 172.21.15.139:0/2261865974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1aac150bd0 0x7f1aac158830 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c000c00 tx=0x7f1a9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:57.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.148+0000 7f1aa8ff9640 1 -- 172.21.15.139:0/2261865974 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c040070 con 0x7f1aac150bd0 2024-09-16T00:08:57.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.148+0000 7f1aa8ff9640 1 -- 172.21.15.139:0/2261865974 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a9c030e10 con 0x7f1aac150bd0 2024-09-16T00:08:57.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.148+0000 7f1ab21a6640 1 -- 172.21.15.139:0/2261865974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aac157620 con 0x7f1aac150bd0 2024-09-16T00:08:57.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.148+0000 7f1aa8ff9640 1 -- 172.21.15.139:0/2261865974 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c006e10 con 0x7f1aac150bd0 2024-09-16T00:08:57.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.148+0000 7f1ab21a6640 1 -- 172.21.15.139:0/2261865974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aac0c2ca0 con 0x7f1aac150bd0 2024-09-16T00:08:57.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.149+0000 7f1ab21a6640 1 -- 172.21.15.139:0/2261865974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aac06ff00 con 0x7f1aac150bd0 2024-09-16T00:08:57.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.149+0000 7f1aa8ff9640 1 -- 172.21.15.139:0/2261865974 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1a9c043070 con 0x7f1aac150bd0 2024-09-16T00:08:57.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.150+0000 7f1aa8ff9640 1 --2- 172.21.15.139:0/2261865974 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1a8c06c2e0 0x7f1a8c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:57.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.150+0000 7f1aa8ff9640 1 -- 172.21.15.139:0/2261865974 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5886+0+0 (secure 0 0 0) 0x7f1a9c0ac0f0 con 0x7f1aac150bd0 2024-09-16T00:08:57.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.150+0000 7f1aa3fff640 1 --2- 172.21.15.139:0/2261865974 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1a8c06c2e0 0x7f1a8c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:57.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.150+0000 7f1aa3fff640 1 --2- 172.21.15.139:0/2261865974 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1a8c06c2e0 0x7f1a8c06e7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0ccce0 tx=0x7f1a94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:57.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.154+0000 7f1aa8ff9640 1 -- 172.21.15.139:0/2261865974 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1a9c07a160 con 0x7f1aac150bd0 2024-09-16T00:08:57.164 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:57.164 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:08:57.165 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:08:57.165 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:08:57.165 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:57.174 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:08:57.219 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:57.219+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:57.289 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:08:57.289 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:08:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:08:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:08:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 94.206535285000001, 2024-09-16T00:08:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:08:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:08:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:08:57.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:08:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:08:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:08:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:08:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:08:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:08:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:08:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:08:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:08:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:08:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:08:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:08:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:08:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:08:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:08:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:08:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:08:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:08:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:08:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:08:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:08:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:08:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:08:57.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:08:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:08:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:08:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:08:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.127003654999996, 2024-09-16T00:08:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093071190000000008, 2024-09-16T00:08:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 106 2024-09-16T00:08:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.297 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:08:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160362+0000", 2024-09-16T00:08:57.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:08:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160438+0000", 2024-09-16T00:08:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6711000000000005e-05 2024-09-16T00:08:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160444+0000", 2024-09-16T00:08:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0530000000000003e-06 2024-09-16T00:08:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160449+0000", 2024-09-16T00:08:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2189999999999998e-06 2024-09-16T00:08:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161270+0000", 2024-09-16T00:08:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082099199999999999 2024-09-16T00:08:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161348+0000", 2024-09-16T00:08:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8361000000000004e-05 2024-09-16T00:08:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161472+0000", 2024-09-16T00:08:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123523 2024-09-16T00:08:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163461+0000", 2024-09-16T00:08:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019891560000000002 2024-09-16T00:08:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169418+0000", 2024-09-16T00:08:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059568490000000002 2024-09-16T00:08:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169570+0000", 2024-09-16T00:08:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152461 2024-09-16T00:08:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169668+0000", 2024-09-16T00:08:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T00:08:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.302 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.126907884000005, 2024-09-16T00:08:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092931679999999992, 2024-09-16T00:08:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107 2024-09-16T00:08:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:08:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0699999999999998e-07 2024-09-16T00:08:57.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160459+0000", 2024-09-16T00:08:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0370000000000001e-06 2024-09-16T00:08:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160463+0000", 2024-09-16T00:08:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999999e-06 2024-09-16T00:08:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160466+0000", 2024-09-16T00:08:57.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.644e-06 2024-09-16T00:08:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161873+0000", 2024-09-16T00:08:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014074840000000001 2024-09-16T00:08:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161949+0000", 2024-09-16T00:08:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5749999999999998e-05 2024-09-16T00:08:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162069+0000", 2024-09-16T00:08:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120257 2024-09-16T00:08:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163506+0000", 2024-09-16T00:08:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001436344 2024-09-16T00:08:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169704+0000", 2024-09-16T00:08:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061982829999999997 2024-09-16T00:08:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169730+0000", 2024-09-16T00:08:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6060000000000001e-05 2024-09-16T00:08:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169750+0000", 2024-09-16T00:08:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0092e-05 2024-09-16T00:08:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:57.308 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.126798571999998, 2024-09-16T00:08:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092280039999999997, 2024-09-16T00:08:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 108 2024-09-16T00:08:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:08:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160567+0000", 2024-09-16T00:08:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7300000000000001e-07 2024-09-16T00:08:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160569+0000", 2024-09-16T00:08:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5299999999999999e-06 2024-09-16T00:08:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160574+0000", 2024-09-16T00:08:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-16T00:08:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160577+0000", 2024-09-16T00:08:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.01e-06 2024-09-16T00:08:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162518+0000", 2024-09-16T00:08:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001940703 2024-09-16T00:08:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162571+0000", 2024-09-16T00:08:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2670000000000002e-05 2024-09-16T00:08:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162647+0000", 2024-09-16T00:08:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6778000000000002e-05 2024-09-16T00:08:57.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163544+0000", 2024-09-16T00:08:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089702599999999999 2024-09-16T00:08:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169754+0000", 2024-09-16T00:08:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062091739999999996 2024-09-16T00:08:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169776+0000", 2024-09-16T00:08:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2391999999999998e-05 2024-09-16T00:08:57.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169794+0000", 2024-09-16T00:08:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8318999999999999e-05 2024-09-16T00:08:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.126750021999996, 2024-09-16T00:08:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092220679999999999, 2024-09-16T00:08:57.314 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 109 2024-09-16T00:08:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:08:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9900000000000001e-07 2024-09-16T00:08:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160707+0000", 2024-09-16T00:08:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1621999999999998e-05 2024-09-16T00:08:57.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160713+0000", 2024-09-16T00:08:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.596e-06 2024-09-16T00:08:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160717+0000", 2024-09-16T00:08:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5829999999999998e-06 2024-09-16T00:08:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162869+0000", 2024-09-16T00:08:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021516310000000002 2024-09-16T00:08:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162946+0000", 2024-09-16T00:08:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7696999999999998e-05 2024-09-16T00:08:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163065+0000", 2024-09-16T00:08:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118581 2024-09-16T00:08:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164431+0000", 2024-09-16T00:08:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001366003 2024-09-16T00:08:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169797+0000", 2024-09-16T00:08:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005366244 2024-09-16T00:08:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169819+0000", 2024-09-16T00:08:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2036999999999999e-05 2024-09-16T00:08:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169837+0000", 2024-09-16T00:08:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7575e-05 2024-09-16T00:08:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.126640672999997, 2024-09-16T00:08:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092871919999999997, 2024-09-16T00:08:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.320 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 110 2024-09-16T00:08:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:08:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:08:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160728+0000", 2024-09-16T00:08:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0620000000000001e-06 2024-09-16T00:08:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160731+0000", 2024-09-16T00:08:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6500000000000002e-06 2024-09-16T00:08:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160734+0000", 2024-09-16T00:08:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.762e-06 2024-09-16T00:08:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163580+0000", 2024-09-16T00:08:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028464549999999999 2024-09-16T00:08:57.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163657+0000", 2024-09-16T00:08:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6695999999999999e-05 2024-09-16T00:08:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163798+0000", 2024-09-16T00:08:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014055 2024-09-16T00:08:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165483+0000", 2024-09-16T00:08:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016853090000000001 2024-09-16T00:08:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169970+0000", 2024-09-16T00:08:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00448709 2024-09-16T00:08:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169994+0000", 2024-09-16T00:08:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3575999999999999e-05 2024-09-16T00:08:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170011+0000", 2024-09-16T00:08:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7702000000000001e-05 2024-09-16T00:08:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.126627003999999, 2024-09-16T00:08:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093174859999999998, 2024-09-16T00:08:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.326 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 111 2024-09-16T00:08:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:08:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1e-07 2024-09-16T00:08:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160741+0000", 2024-09-16T00:08:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6110000000000001e-06 2024-09-16T00:08:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160744+0000", 2024-09-16T00:08:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3299999999999999e-06 2024-09-16T00:08:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160746+0000", 2024-09-16T00:08:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5469999999999999e-06 2024-09-16T00:08:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164069+0000", 2024-09-16T00:08:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033222709999999999 2024-09-16T00:08:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164113+0000", 2024-09-16T00:08:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4212999999999998e-05 2024-09-16T00:08:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164187+0000", 2024-09-16T00:08:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4136999999999998e-05 2024-09-16T00:08:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166429+0000", 2024-09-16T00:08:57.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022423180000000001 2024-09-16T00:08:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170016+0000", 2024-09-16T00:08:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035869449999999998 2024-09-16T00:08:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170038+0000", 2024-09-16T00:08:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2053e-05 2024-09-16T00:08:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170055+0000", 2024-09-16T00:08:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6850999999999998e-05 2024-09-16T00:08:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.126613289000005, 2024-09-16T00:08:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093480279999999995, 2024-09-16T00:08:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.332 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 112 2024-09-16T00:08:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:08:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160752+0000", 2024-09-16T00:08:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2700000000000001e-07 2024-09-16T00:08:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160808+0000", 2024-09-16T00:08:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5797999999999997e-05 2024-09-16T00:08:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160814+0000", 2024-09-16T00:08:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5980000000000002e-06 2024-09-16T00:08:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160819+0000", 2024-09-16T00:08:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8960000000000002e-06 2024-09-16T00:08:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164468+0000", 2024-09-16T00:08:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003649476 2024-09-16T00:08:57.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164525+0000", 2024-09-16T00:08:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6227999999999998e-05 2024-09-16T00:08:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164595+0000", 2024-09-16T00:08:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759000000000005e-05 2024-09-16T00:08:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167237+0000", 2024-09-16T00:08:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026410470000000001 2024-09-16T00:08:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170060+0000", 2024-09-16T00:08:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028234969999999999 2024-09-16T00:08:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170081+0000", 2024-09-16T00:08:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1242000000000001e-05 2024-09-16T00:08:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170100+0000", 2024-09-16T00:08:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8260000000000001e-05 2024-09-16T00:08:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.126536193999996, 2024-09-16T00:08:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093139409999999992, 2024-09-16T00:08:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.338 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 113 2024-09-16T00:08:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:08:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-07 2024-09-16T00:08:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160833+0000", 2024-09-16T00:08:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.557e-06 2024-09-16T00:08:57.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160839+0000", 2024-09-16T00:08:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3829999999999997e-06 2024-09-16T00:08:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160844+0000", 2024-09-16T00:08:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6909999999999996e-06 2024-09-16T00:08:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165008+0000", 2024-09-16T00:08:57.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041642129999999999 2024-09-16T00:08:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165085+0000", 2024-09-16T00:08:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6651999999999997e-05 2024-09-16T00:08:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165205+0000", 2024-09-16T00:08:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012045900000000001 2024-09-16T00:08:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167283+0000", 2024-09-16T00:08:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020782909999999999 2024-09-16T00:08:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170104+0000", 2024-09-16T00:08:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028207950000000001 2024-09-16T00:08:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170125+0000", 2024-09-16T00:08:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0852e-05 2024-09-16T00:08:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170143+0000", 2024-09-16T00:08:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7467000000000001e-05 2024-09-16T00:08:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.126489711999994, 2024-09-16T00:08:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093126960000000005, 2024-09-16T00:08:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.344 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-16T00:08:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:08:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160876+0000", 2024-09-16T00:08:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5700000000000002e-07 2024-09-16T00:08:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160880+0000", 2024-09-16T00:08:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5539999999999999e-06 2024-09-16T00:08:57.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160888+0000", 2024-09-16T00:08:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.157e-06 2024-09-16T00:08:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160894+0000", 2024-09-16T00:08:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7690000000000002e-06 2024-09-16T00:08:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165519+0000", 2024-09-16T00:08:57.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046249539999999997 2024-09-16T00:08:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165593+0000", 2024-09-16T00:08:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4039e-05 2024-09-16T00:08:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165715+0000", 2024-09-16T00:08:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012131899999999999 2024-09-16T00:08:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167309+0000", 2024-09-16T00:08:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001594011 2024-09-16T00:08:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170147+0000", 2024-09-16T00:08:57.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028387130000000001 2024-09-16T00:08:57.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170170+0000", 2024-09-16T00:08:57.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2517000000000001e-05 2024-09-16T00:08:57.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170188+0000", 2024-09-16T00:08:57.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8006e-05 2024-09-16T00:08:57.353 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.353 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.353 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.353 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:57.353 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:57.354 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.126448683999996, 2024-09-16T00:08:57.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093140040000000007, 2024-09-16T00:08:57.354 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.354 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.354 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.354 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.354 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.354 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.354 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-16T00:08:57.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.354 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:57.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:08:57.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160917+0000", 2024-09-16T00:08:57.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7599999999999996e-07 2024-09-16T00:08:57.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160992+0000", 2024-09-16T00:08:57.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4760999999999998e-05 2024-09-16T00:08:57.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161000+0000", 2024-09-16T00:08:57.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9829999999999996e-06 2024-09-16T00:08:57.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161007+0000", 2024-09-16T00:08:57.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4909999999999999e-06 2024-09-16T00:08:57.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165983+0000", 2024-09-16T00:08:57.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049755399999999997 2024-09-16T00:08:57.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166027+0000", 2024-09-16T00:08:57.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4109000000000001e-05 2024-09-16T00:08:57.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166100+0000", 2024-09-16T00:08:57.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3360000000000002e-05 2024-09-16T00:08:57.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169698+0000", 2024-09-16T00:08:57.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597795 2024-09-16T00:08:57.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170192+0000", 2024-09-16T00:08:57.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049447300000000005 2024-09-16T00:08:57.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170214+0000", 2024-09-16T00:08:57.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1546e-05 2024-09-16T00:08:57.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170230+0000", 2024-09-16T00:08:57.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6169999999999999e-05 2024-09-16T00:08:57.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.359 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.359 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:57.359 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:57.359 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.126338954999994, 2024-09-16T00:08:57.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092457029999999992, 2024-09-16T00:08:57.360 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.360 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.360 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.360 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-16T00:08:57.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.360 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:57.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:57.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:08:57.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7899999999999999e-07 2024-09-16T00:08:57.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161029+0000", 2024-09-16T00:08:57.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6630000000000002e-06 2024-09-16T00:08:57.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161036+0000", 2024-09-16T00:08:57.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T00:08:57.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161041+0000", 2024-09-16T00:08:57.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:08:57.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166468+0000", 2024-09-16T00:08:57.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054266050000000001 2024-09-16T00:08:57.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166511+0000", 2024-09-16T00:08:57.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3869999999999998e-05 2024-09-16T00:08:57.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166606+0000", 2024-09-16T00:08:57.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4437000000000004e-05 2024-09-16T00:08:57.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169955+0000", 2024-09-16T00:08:57.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033494810000000001 2024-09-16T00:08:57.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170234+0000", 2024-09-16T00:08:57.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027900899999999998 2024-09-16T00:08:57.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170255+0000", 2024-09-16T00:08:57.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0537999999999999e-05 2024-09-16T00:08:57.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170272+0000", 2024-09-16T00:08:57.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6611000000000001e-05 2024-09-16T00:08:57.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.365 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.365 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:08:57.365 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:57.365 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.126313809999999, 2024-09-16T00:08:57.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093511010000000006, 2024-09-16T00:08:57.366 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.366 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.366 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.366 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-16T00:08:57.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.366 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:57.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:57.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:08:57.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.63e-07 2024-09-16T00:08:57.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161055+0000", 2024-09-16T00:08:57.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6770000000000001e-06 2024-09-16T00:08:57.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161063+0000", 2024-09-16T00:08:57.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6469999999999998e-06 2024-09-16T00:08:57.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161068+0000", 2024-09-16T00:08:57.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.746e-06 2024-09-16T00:08:57.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166817+0000", 2024-09-16T00:08:57.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057494959999999998 2024-09-16T00:08:57.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166893+0000", 2024-09-16T00:08:57.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6210000000000004e-05 2024-09-16T00:08:57.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167019+0000", 2024-09-16T00:08:57.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125585 2024-09-16T00:08:57.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170275+0000", 2024-09-16T00:08:57.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032563620000000001 2024-09-16T00:08:57.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170309+0000", 2024-09-16T00:08:57.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3865999999999997e-05 2024-09-16T00:08:57.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170362+0000", 2024-09-16T00:08:57.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3094999999999999e-05 2024-09-16T00:08:57.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170402+0000", 2024-09-16T00:08:57.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9953999999999997e-05 2024-09-16T00:08:57.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.371 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.371 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:57.371 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:57.371 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.677581724999996, 2024-09-16T00:08:57.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097364389999999995, 2024-09-16T00:08:57.371 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.371 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.371 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.371 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.372 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.372 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.372 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138 2024-09-16T00:08:57.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.372 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:57.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:08:57.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609784+0000", 2024-09-16T00:08:57.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T00:08:57.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609787+0000", 2024-09-16T00:08:57.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280000000000001e-06 2024-09-16T00:08:57.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609794+0000", 2024-09-16T00:08:57.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6409999999999996e-06 2024-09-16T00:08:57.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609799+0000", 2024-09-16T00:08:57.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9359999999999998e-06 2024-09-16T00:08:57.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615759+0000", 2024-09-16T00:08:57.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059599910000000004 2024-09-16T00:08:57.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615806+0000", 2024-09-16T00:08:57.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7290000000000003e-05 2024-09-16T00:08:57.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615877+0000", 2024-09-16T00:08:57.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1000000000000005e-05 2024-09-16T00:08:57.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618961+0000", 2024-09-16T00:08:57.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030841390000000001 2024-09-16T00:08:57.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619436+0000", 2024-09-16T00:08:57.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047452799999999999 2024-09-16T00:08:57.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619482+0000", 2024-09-16T00:08:57.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5689000000000003e-05 2024-09-16T00:08:57.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619519+0000", 2024-09-16T00:08:57.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7938000000000002e-05 2024-09-16T00:08:57.376 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.377 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.377 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.377 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:57.377 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:57.377 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.677558011000002, 2024-09-16T00:08:57.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011812954000000001, 2024-09-16T00:08:57.377 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.377 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.377 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.377 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.378 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139 2024-09-16T00:08:57.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.378 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:57.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:57.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:08:57.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999998e-07 2024-09-16T00:08:57.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609810+0000", 2024-09-16T00:08:57.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2100000000000002e-06 2024-09-16T00:08:57.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609817+0000", 2024-09-16T00:08:57.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.28e-06 2024-09-16T00:08:57.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609825+0000", 2024-09-16T00:08:57.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:08:57.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616116+0000", 2024-09-16T00:08:57.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006290543 2024-09-16T00:08:57.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616187+0000", 2024-09-16T00:08:57.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1830999999999997e-05 2024-09-16T00:08:57.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616298+0000", 2024-09-16T00:08:57.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011044699999999999 2024-09-16T00:08:57.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619559+0000", 2024-09-16T00:08:57.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032614710000000002 2024-09-16T00:08:57.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621562+0000", 2024-09-16T00:08:57.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020028680000000001 2024-09-16T00:08:57.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621597+0000", 2024-09-16T00:08:57.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4625000000000001e-05 2024-09-16T00:08:57.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621620+0000", 2024-09-16T00:08:57.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2864e-05 2024-09-16T00:08:57.382 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.382 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.382 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.383 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:57.383 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:57.383 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.677530215000004, 2024-09-16T00:08:57.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011828211999999999, 2024-09-16T00:08:57.383 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.383 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.383 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.383 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 140 2024-09-16T00:08:57.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.384 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:57.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:57.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:08:57.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T00:08:57.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:57.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961000000000003e-05 2024-09-16T00:08:57.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609892+0000", 2024-09-16T00:08:57.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0516000000000001e-05 2024-09-16T00:08:57.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609898+0000", 2024-09-16T00:08:57.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0449999999999997e-06 2024-09-16T00:08:57.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616797+0000", 2024-09-16T00:08:57.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068998840000000002 2024-09-16T00:08:57.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616844+0000", 2024-09-16T00:08:57.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6501000000000001e-05 2024-09-16T00:08:57.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616935+0000", 2024-09-16T00:08:57.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1222000000000002e-05 2024-09-16T00:08:57.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619808+0000", 2024-09-16T00:08:57.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873198 2024-09-16T00:08:57.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621624+0000", 2024-09-16T00:08:57.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001815933 2024-09-16T00:08:57.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621646+0000", 2024-09-16T00:08:57.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1622999999999999e-05 2024-09-16T00:08:57.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621663+0000", 2024-09-16T00:08:57.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6854000000000002e-05 2024-09-16T00:08:57.388 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.388 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.388 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.388 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:57.389 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:57.389 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.677444406000006, 2024-09-16T00:08:57.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014331501999999999, 2024-09-16T00:08:57.389 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.389 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.389 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.389 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.389 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.389 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.389 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-16T00:08:57.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.389 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:57.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:08:57.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609921+0000", 2024-09-16T00:08:57.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9599999999999999e-07 2024-09-16T00:08:57.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609925+0000", 2024-09-16T00:08:57.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T00:08:57.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609932+0000", 2024-09-16T00:08:57.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8240000000000002e-06 2024-09-16T00:08:57.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609937+0000", 2024-09-16T00:08:57.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1070000000000004e-06 2024-09-16T00:08:57.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618987+0000", 2024-09-16T00:08:57.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090502719999999998 2024-09-16T00:08:57.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619064+0000", 2024-09-16T00:08:57.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6868999999999999e-05 2024-09-16T00:08:57.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619183+0000", 2024-09-16T00:08:57.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011961 2024-09-16T00:08:57.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.620052+0000", 2024-09-16T00:08:57.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086822799999999997 2024-09-16T00:08:57.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624197+0000", 2024-09-16T00:08:57.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041454930000000001 2024-09-16T00:08:57.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624229+0000", 2024-09-16T00:08:57.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1758999999999999e-05 2024-09-16T00:08:57.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624252+0000", 2024-09-16T00:08:57.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:08:57.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.394 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.394 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.394 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:57.394 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:57.394 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.204678598000001, 2024-09-16T00:08:57.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010093276999999999, 2024-09-16T00:08:57.395 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.395 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.395 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.395 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.395 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 157 2024-09-16T00:08:57.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.395 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:57.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:57.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:08:57.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4699999999999998e-07 2024-09-16T00:08:57.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082688+0000", 2024-09-16T00:08:57.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0209999999999998e-06 2024-09-16T00:08:57.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082692+0000", 2024-09-16T00:08:57.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6890000000000002e-06 2024-09-16T00:08:57.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082695+0000", 2024-09-16T00:08:57.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5339999999999998e-06 2024-09-16T00:08:57.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085303+0000", 2024-09-16T00:08:57.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026084789999999999 2024-09-16T00:08:57.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085374+0000", 2024-09-16T00:08:57.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1317999999999997e-05 2024-09-16T00:08:57.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085506+0000", 2024-09-16T00:08:57.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013118 2024-09-16T00:08:57.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086928+0000", 2024-09-16T00:08:57.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014224559999999999 2024-09-16T00:08:57.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092665+0000", 2024-09-16T00:08:57.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057369830000000002 2024-09-16T00:08:57.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092727+0000", 2024-09-16T00:08:57.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1810000000000006e-05 2024-09-16T00:08:57.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092779+0000", 2024-09-16T00:08:57.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2559999999999998e-05 2024-09-16T00:08:57.400 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.400 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.400 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.400 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:57.400 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:57.400 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.204666430000003, 2024-09-16T00:08:57.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010176388, 2024-09-16T00:08:57.401 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.401 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.401 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.401 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.401 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 158 2024-09-16T00:08:57.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.401 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:57.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:08:57.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082699+0000", 2024-09-16T00:08:57.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.11e-07 2024-09-16T00:08:57.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082742+0000", 2024-09-16T00:08:57.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3699000000000003e-05 2024-09-16T00:08:57.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082748+0000", 2024-09-16T00:08:57.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8690000000000001e-06 2024-09-16T00:08:57.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082751+0000", 2024-09-16T00:08:57.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.942e-06 2024-09-16T00:08:57.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085881+0000", 2024-09-16T00:08:57.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031303450000000001 2024-09-16T00:08:57.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085959+0000", 2024-09-16T00:08:57.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7874999999999996e-05 2024-09-16T00:08:57.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086086+0000", 2024-09-16T00:08:57.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012685499999999999 2024-09-16T00:08:57.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087127+0000", 2024-09-16T00:08:57.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001041298 2024-09-16T00:08:57.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092790+0000", 2024-09-16T00:08:57.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056628 2024-09-16T00:08:57.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092832+0000", 2024-09-16T00:08:57.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2227000000000001e-05 2024-09-16T00:08:57.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092875+0000", 2024-09-16T00:08:57.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2267e-05 2024-09-16T00:08:57.406 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.406 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.406 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.406 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:08:57.406 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:57.406 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 92.204608391999997, 2024-09-16T00:08:57.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010199547, 2024-09-16T00:08:57.406 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:08:57.406 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:08:57.407 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.407 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:08:57.407 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:08:57.407 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:08:57.407 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159 2024-09-16T00:08:57.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.407 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:08:57.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:08:57.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:57.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:08:57.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:08:57.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:08:57.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:08:57.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082757+0000", 2024-09-16T00:08:57.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2500000000000001e-07 2024-09-16T00:08:57.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:08:57.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082759+0000", 2024-09-16T00:08:57.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-16T00:08:57.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:08:57.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082763+0000", 2024-09-16T00:08:57.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4889999999999997e-06 2024-09-16T00:08:57.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082766+0000", 2024-09-16T00:08:57.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:08:57.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:08:57.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086571+0000", 2024-09-16T00:08:57.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038053269999999998 2024-09-16T00:08:57.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:08:57.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086619+0000", 2024-09-16T00:08:57.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7924999999999999e-05 2024-09-16T00:08:57.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:57.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086693+0000", 2024-09-16T00:08:57.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000005e-05 2024-09-16T00:08:57.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087383+0000", 2024-09-16T00:08:57.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069008299999999997 2024-09-16T00:08:57.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:08:57.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092882+0000", 2024-09-16T00:08:57.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054987450000000002 2024-09-16T00:08:57.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:08:57.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092921+0000", 2024-09-16T00:08:57.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9388999999999998e-05 2024-09-16T00:08:57.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:08:57.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:08:57.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:08:57.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092956+0000", 2024-09-16T00:08:57.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4647999999999998e-05 2024-09-16T00:08:57.412 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.412 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.412 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.412 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:08:57.412 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:08:57.412 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:08:57.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.376+0000 7f1ab21a6640 1 -- 172.21.15.139:0/2261865974 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) -- 0x7f1aac06c590 con 0x7f1aac150bd0 2024-09-16T00:08:57.413 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:57.412+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:57.414 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:08:57.514 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:57.514+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:57.522 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:57.523 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:57.523 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:57.523 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:57.531 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:08:57.615 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:57.615+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:57.688 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:57.688 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:57.689 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:57.689 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:57.689 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:57.698 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:08:57.715 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:57.715+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:57.816 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:57.815+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:57.819 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'66)", 2024-09-16T00:08:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 93.651231228, 2024-09-16T00:08:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032925509999999999, 2024-09-16T00:08:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:08:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166177+0000", 2024-09-16T00:08:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:08:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166216+0000", 2024-09-16T00:08:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9008000000000003e-05 2024-09-16T00:08:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166226+0000", 2024-09-16T00:08:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0081999999999999e-05 2024-09-16T00:08:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166230+0000", 2024-09-16T00:08:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5079999999999999e-06 2024-09-16T00:08:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166259+0000", 2024-09-16T00:08:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8928000000000002e-05 2024-09-16T00:08:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166271+0000", 2024-09-16T00:08:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1531e-05 2024-09-16T00:08:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169449+0000", 2024-09-16T00:08:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003178143 2024-09-16T00:08:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169468+0000", 2024-09-16T00:08:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9700000000000001e-05 2024-09-16T00:08:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'66)", 2024-09-16T00:08:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 93.650769768999993, 2024-09-16T00:08:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003179374, 2024-09-16T00:08:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:08:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166638+0000", 2024-09-16T00:08:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2300000000000002e-07 2024-09-16T00:08:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166641+0000", 2024-09-16T00:08:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0079999999999998e-06 2024-09-16T00:08:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166646+0000", 2024-09-16T00:08:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4970000000000001e-06 2024-09-16T00:08:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166650+0000", 2024-09-16T00:08:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5729999999999999e-06 2024-09-16T00:08:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166682+0000", 2024-09-16T00:08:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2218999999999998e-05 2024-09-16T00:08:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166694+0000", 2024-09-16T00:08:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1902999999999999e-05 2024-09-16T00:08:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169784+0000", 2024-09-16T00:08:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003089545 2024-09-16T00:08:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169817+0000", 2024-09-16T00:08:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3206e-05 2024-09-16T00:08:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:117 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'66)", 2024-09-16T00:08:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 93.650325709000001, 2024-09-16T00:08:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00275326, 2024-09-16T00:08:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:08:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167082+0000", 2024-09-16T00:08:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:08:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167086+0000", 2024-09-16T00:08:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1559999999999997e-06 2024-09-16T00:08:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167096+0000", 2024-09-16T00:08:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8390000000000002e-06 2024-09-16T00:08:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167101+0000", 2024-09-16T00:08:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:08:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167130+0000", 2024-09-16T00:08:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9649e-05 2024-09-16T00:08:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167142+0000", 2024-09-16T00:08:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1435e-05 2024-09-16T00:08:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169818+0000", 2024-09-16T00:08:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002676056 2024-09-16T00:08:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169835+0000", 2024-09-16T00:08:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6980999999999999e-05 2024-09-16T00:08:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:130 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'88, mlcod=17'85)", 2024-09-16T00:08:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 93.207526380000004, 2024-09-16T00:08:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002880456, 2024-09-16T00:08:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:08:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5699999999999998e-07 2024-09-16T00:08:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609885+0000", 2024-09-16T00:08:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.608e-06 2024-09-16T00:08:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609891+0000", 2024-09-16T00:08:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:08:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609895+0000", 2024-09-16T00:08:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.54e-06 2024-09-16T00:08:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609931+0000", 2024-09-16T00:08:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5893999999999999e-05 2024-09-16T00:08:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609945+0000", 2024-09-16T00:08:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4259e-05 2024-09-16T00:08:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612721+0000", 2024-09-16T00:08:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027757849999999998 2024-09-16T00:08:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612761+0000", 2024-09-16T00:08:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0638e-05 2024-09-16T00:08:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-16T00:08:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 93.207142407000006, 2024-09-16T00:08:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028016529999999999, 2024-09-16T00:08:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:08:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.63e-07 2024-09-16T00:08:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610344+0000", 2024-09-16T00:08:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8996000000000001e-05 2024-09-16T00:08:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610352+0000", 2024-09-16T00:08:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610356+0000", 2024-09-16T00:08:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0790000000000002e-06 2024-09-16T00:08:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610388+0000", 2024-09-16T00:08:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1541000000000002e-05 2024-09-16T00:08:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610400+0000", 2024-09-16T00:08:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2075e-05 2024-09-16T00:08:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613045+0000", 2024-09-16T00:08:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002645213 2024-09-16T00:08:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613066+0000", 2024-09-16T00:08:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1353e-05 2024-09-16T00:08:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-16T00:08:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 93.203882906999993, 2024-09-16T00:08:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027254850000000001, 2024-09-16T00:08:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:08:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613525+0000", 2024-09-16T00:08:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2800000000000002e-07 2024-09-16T00:08:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613528+0000", 2024-09-16T00:08:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9730000000000002e-06 2024-09-16T00:08:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613534+0000", 2024-09-16T00:08:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4339999999999997e-06 2024-09-16T00:08:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613538+0000", 2024-09-16T00:08:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5149999999999998e-06 2024-09-16T00:08:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613566+0000", 2024-09-16T00:08:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8104000000000001e-05 2024-09-16T00:08:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613578+0000", 2024-09-16T00:08:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2404e-05 2024-09-16T00:08:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616217+0000", 2024-09-16T00:08:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026389830000000001 2024-09-16T00:08:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616250+0000", 2024-09-16T00:08:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2644000000000001e-05 2024-09-16T00:08:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:134 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'92, mlcod=18'87)", 2024-09-16T00:08:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 93.203344767999994, 2024-09-16T00:08:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025322399999999998, 2024-09-16T00:08:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614062+0000", 2024-09-16T00:08:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614063+0000", 2024-09-16T00:08:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8700000000000005e-07 2024-09-16T00:08:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614096+0000", 2024-09-16T00:08:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3136000000000003e-05 2024-09-16T00:08:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614106+0000", 2024-09-16T00:08:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9170000000000001e-06 2024-09-16T00:08:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614111+0000", 2024-09-16T00:08:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4390000000000002e-06 2024-09-16T00:08:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614141+0000", 2024-09-16T00:08:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.04e-05 2024-09-16T00:08:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.614152+0000", 2024-09-16T00:08:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0987e-05 2024-09-16T00:08:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616573+0000", 2024-09-16T00:08:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002421351 2024-09-16T00:08:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616595+0000", 2024-09-16T00:08:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1322999999999999e-05 2024-09-16T00:08:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:139 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'97, mlcod=18'87)", 2024-09-16T00:08:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 93.201064059000004, 2024-09-16T00:08:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030908139999999999, 2024-09-16T00:08:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:08:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616344+0000", 2024-09-16T00:08:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3000000000000001e-07 2024-09-16T00:08:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616347+0000", 2024-09-16T00:08:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7740000000000002e-06 2024-09-16T00:08:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616355+0000", 2024-09-16T00:08:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4699999999999996e-06 2024-09-16T00:08:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616358+0000", 2024-09-16T00:08:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5899999999999999e-06 2024-09-16T00:08:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616393+0000", 2024-09-16T00:08:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4529000000000001e-05 2024-09-16T00:08:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616404+0000", 2024-09-16T00:08:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0726e-05 2024-09-16T00:08:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619411+0000", 2024-09-16T00:08:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030076249999999999 2024-09-16T00:08:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619434+0000", 2024-09-16T00:08:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-05 2024-09-16T00:08:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:140 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'98, mlcod=18'89)", 2024-09-16T00:08:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 93.200422665999994, 2024-09-16T00:08:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027276150000000001, 2024-09-16T00:08:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:08:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616985+0000", 2024-09-16T00:08:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4099999999999999e-07 2024-09-16T00:08:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617055+0000", 2024-09-16T00:08:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9923000000000001e-05 2024-09-16T00:08:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617063+0000", 2024-09-16T00:08:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:08:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617070+0000", 2024-09-16T00:08:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9090000000000003e-06 2024-09-16T00:08:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617102+0000", 2024-09-16T00:08:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2620999999999998e-05 2024-09-16T00:08:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617114+0000", 2024-09-16T00:08:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1228e-05 2024-09-16T00:08:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619690+0000", 2024-09-16T00:08:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025763890000000001 2024-09-16T00:08:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619712+0000", 2024-09-16T00:08:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2171000000000002e-05 2024-09-16T00:08:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.201593566, 2024-09-16T00:08:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027355320000000002, 2024-09-16T00:08:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:08:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615815+0000", 2024-09-16T00:08:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1769999999999999e-06 2024-09-16T00:08:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615825+0000", 2024-09-16T00:08:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9000000000000001e-06 2024-09-16T00:08:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615840+0000", 2024-09-16T00:08:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4854e-05 2024-09-16T00:08:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615848+0000", 2024-09-16T00:08:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0940000000000008e-06 2024-09-16T00:08:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615926+0000", 2024-09-16T00:08:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8176999999999994e-05 2024-09-16T00:08:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615945+0000", 2024-09-16T00:08:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9663000000000001e-05 2024-09-16T00:08:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618509+0000", 2024-09-16T00:08:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002563802 2024-09-16T00:08:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618549+0000", 2024-09-16T00:08:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9864999999999999e-05 2024-09-16T00:08:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.201504426, 2024-09-16T00:08:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061593860000000002, 2024-09-16T00:08:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:08:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615907+0000", 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9389999999999999e-06 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615920+0000", 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2989999999999999e-05 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615931+0000", 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.164e-05 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615939+0000", 2024-09-16T00:08:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8029999999999997e-06 2024-09-16T00:08:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616310+0000", 2024-09-16T00:08:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00037111400000000003 2024-09-16T00:08:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616406+0000", 2024-09-16T00:08:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5372999999999996e-05 2024-09-16T00:08:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616514+0000", 2024-09-16T00:08:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108564 2024-09-16T00:08:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618158+0000", 2024-09-16T00:08:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016442449999999999 2024-09-16T00:08:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621972+0000", 2024-09-16T00:08:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038132299999999999 2024-09-16T00:08:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622018+0000", 2024-09-16T00:08:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6239000000000002e-05 2024-09-16T00:08:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622062+0000", 2024-09-16T00:08:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4249000000000003e-05 2024-09-16T00:08:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.871 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:57.871 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:57.871 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.201450957999995, 2024-09-16T00:08:57.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006258914, 2024-09-16T00:08:57.871 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.871 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.871 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.871 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:57.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.872 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:08:57.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.872 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615957+0000", 2024-09-16T00:08:57.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1399999999999997e-07 2024-09-16T00:08:57.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.872 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615960+0000", 2024-09-16T00:08:57.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0860000000000002e-06 2024-09-16T00:08:57.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615967+0000", 2024-09-16T00:08:57.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9059999999999996e-06 2024-09-16T00:08:57.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615972+0000", 2024-09-16T00:08:57.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5679999999999999e-06 2024-09-16T00:08:57.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616717+0000", 2024-09-16T00:08:57.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074438 2024-09-16T00:08:57.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616781+0000", 2024-09-16T00:08:57.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4808999999999996e-05 2024-09-16T00:08:57.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:57.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616885+0000", 2024-09-16T00:08:57.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010404900000000001 2024-09-16T00:08:57.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:57.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:57.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012773870000000001 2024-09-16T00:08:57.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622116+0000", 2024-09-16T00:08:57.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039533470000000003 2024-09-16T00:08:57.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622173+0000", 2024-09-16T00:08:57.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6347000000000001e-05 2024-09-16T00:08:57.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.876 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.876 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622215+0000", 2024-09-16T00:08:57.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2521000000000002e-05 2024-09-16T00:08:57.876 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.876 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.876 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.876 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:57.876 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:57.876 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.201427221000003, 2024-09-16T00:08:57.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041725529999999999, 2024-09-16T00:08:57.877 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.877 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.877 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.877 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.877 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.877 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:57.877 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.877 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.877 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:57.877 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:08:57.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8799999999999998e-07 2024-09-16T00:08:57.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615990+0000", 2024-09-16T00:08:57.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2150000000000007e-06 2024-09-16T00:08:57.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:08:57.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7159999999999997e-06 2024-09-16T00:08:57.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616001+0000", 2024-09-16T00:08:57.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9339999999999997e-06 2024-09-16T00:08:57.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616293+0000", 2024-09-16T00:08:57.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029210899999999998 2024-09-16T00:08:57.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616401+0000", 2024-09-16T00:08:57.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108087 2024-09-16T00:08:57.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:57.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616561+0000", 2024-09-16T00:08:57.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015940600000000001 2024-09-16T00:08:57.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:57.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618146+0000", 2024-09-16T00:08:57.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015857040000000001 2024-09-16T00:08:57.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620091+0000", 2024-09-16T00:08:57.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001944837 2024-09-16T00:08:57.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620126+0000", 2024-09-16T00:08:57.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4242999999999997e-05 2024-09-16T00:08:57.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620152+0000", 2024-09-16T00:08:57.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6914e-05 2024-09-16T00:08:57.882 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.882 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.882 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.882 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:57.882 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:57.882 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.201388996000006, 2024-09-16T00:08:57.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00630628, 2024-09-16T00:08:57.882 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.882 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.882 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.882 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:57.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:08:57.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:08:57.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5999999999999999e-07 2024-09-16T00:08:57.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.884 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616022+0000", 2024-09-16T00:08:57.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0149999999999999e-06 2024-09-16T00:08:57.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.884 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616031+0000", 2024-09-16T00:08:57.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9770000000000006e-06 2024-09-16T00:08:57.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.884 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616036+0000", 2024-09-16T00:08:57.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2639999999999999e-06 2024-09-16T00:08:57.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617057+0000", 2024-09-16T00:08:57.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010212999999999999 2024-09-16T00:08:57.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617146+0000", 2024-09-16T00:08:57.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9234000000000007e-05 2024-09-16T00:08:57.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:08:57.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617232+0000", 2024-09-16T00:08:57.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5218999999999998e-05 2024-09-16T00:08:57.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:57.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619107+0000", 2024-09-16T00:08:57.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001875016 2024-09-16T00:08:57.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622259+0000", 2024-09-16T00:08:57.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031527930000000001 2024-09-16T00:08:57.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622290+0000", 2024-09-16T00:08:57.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1022999999999998e-05 2024-09-16T00:08:57.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622324+0000", 2024-09-16T00:08:57.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3979000000000001e-05 2024-09-16T00:08:57.887 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.887 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.887 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.888 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:57.888 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:57.888 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.201363693999994, 2024-09-16T00:08:57.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042412739999999997, 2024-09-16T00:08:57.888 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.888 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.888 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.888 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:57.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:08:57.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616044+0000", 2024-09-16T00:08:57.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:57.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616052+0000", 2024-09-16T00:08:57.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.439e-06 2024-09-16T00:08:57.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616059+0000", 2024-09-16T00:08:57.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1389999999999998e-06 2024-09-16T00:08:57.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616065+0000", 2024-09-16T00:08:57.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:08:57.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616823+0000", 2024-09-16T00:08:57.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075781199999999996 2024-09-16T00:08:57.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616918+0000", 2024-09-16T00:08:57.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5174999999999996e-05 2024-09-16T00:08:57.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:57.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617019+0000", 2024-09-16T00:08:57.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101161 2024-09-16T00:08:57.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:57.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618503+0000", 2024-09-16T00:08:57.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014836750000000001 2024-09-16T00:08:57.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.892 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620201+0000", 2024-09-16T00:08:57.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00169805 2024-09-16T00:08:57.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.892 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620255+0000", 2024-09-16T00:08:57.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4499000000000001e-05 2024-09-16T00:08:57.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620285+0000", 2024-09-16T00:08:57.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9349e-05 2024-09-16T00:08:57.893 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.893 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.893 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.893 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:57.893 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:57.893 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.201334703000001, 2024-09-16T00:08:57.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063682799999999996, 2024-09-16T00:08:57.893 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.893 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.894 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:08:57.894 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:57.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:08:57.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:08:57.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9900000000000001e-07 2024-09-16T00:08:57.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616079+0000", 2024-09-16T00:08:57.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9009999999999999e-06 2024-09-16T00:08:57.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616086+0000", 2024-09-16T00:08:57.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.776e-06 2024-09-16T00:08:57.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616091+0000", 2024-09-16T00:08:57.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5600000000000001e-06 2024-09-16T00:08:57.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.896 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618339+0000", 2024-09-16T00:08:57.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022476179999999998 2024-09-16T00:08:57.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.896 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:57.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9025999999999997e-05 2024-09-16T00:08:57.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.897 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622441+0000", 2024-09-16T00:08:57.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040029999999999996 2024-09-16T00:08:57.897 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.897 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.897 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.897 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:114 3.1b 3:d9cb9acc:::1000000026a.00000000:head [write 0~111] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:08:57.897 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.513698+0000", 2024-09-16T00:08:57.897 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.3037093909999999, 2024-09-16T00:08:57.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024303630000000001, 2024-09-16T00:08:57.897 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.898 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.898 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.898 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:08:57.898 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:08:57.898 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:08:57.898 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 114 2024-09-16T00:08:57.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.898 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.513698+0000", 2024-09-16T00:08:57.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.513698+0000", 2024-09-16T00:08:57.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.513699+0000", 2024-09-16T00:08:57.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.468e-06 2024-09-16T00:08:57.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.513707+0000", 2024-09-16T00:08:57.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4189999999999996e-06 2024-09-16T00:08:57.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.513722+0000", 2024-09-16T00:08:57.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4983e-05 2024-09-16T00:08:57.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.513729+0000", 2024-09-16T00:08:57.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0299999999999996e-06 2024-09-16T00:08:57.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.513787+0000", 2024-09-16T00:08:57.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8405000000000001e-05 2024-09-16T00:08:57.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.513898+0000", 2024-09-16T00:08:57.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011106400000000001 2024-09-16T00:08:57.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:57.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.514014+0000", 2024-09-16T00:08:57.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011633699999999999 2024-09-16T00:08:57.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:57.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.514555+0000", 2024-09-16T00:08:57.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054078999999999998 2024-09-16T00:08:57.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.516055+0000", 2024-09-16T00:08:57.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014994870000000001 2024-09-16T00:08:57.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.516093+0000", 2024-09-16T00:08:57.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8025000000000003e-05 2024-09-16T00:08:57.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.516128+0000", 2024-09-16T00:08:57.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5355000000000003e-05 2024-09-16T00:08:57.903 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.903 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.903 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.903 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:221 3.1b 3:dad7a251:::100000002d7.00000000:head [write 0~30412] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:08:57.903 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:08:57.903 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.0681265129999999, 2024-09-16T00:08:57.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034396769999999999, 2024-09-16T00:08:57.903 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.903 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.903 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.903 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:08:57.904 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:08:57.904 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:08:57.904 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 221 2024-09-16T00:08:57.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.904 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:08:57.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:08:57.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749282+0000", 2024-09-16T00:08:57.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3769999999999999e-06 2024-09-16T00:08:57.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749300+0000", 2024-09-16T00:08:57.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8204999999999999e-05 2024-09-16T00:08:57.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749311+0000", 2024-09-16T00:08:57.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0599e-05 2024-09-16T00:08:57.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749318+0000", 2024-09-16T00:08:57.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9789999999999996e-06 2024-09-16T00:08:57.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749367+0000", 2024-09-16T00:08:57.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9382999999999998e-05 2024-09-16T00:08:57.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749450+0000", 2024-09-16T00:08:57.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-05 2024-09-16T00:08:57.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:57.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749526+0000", 2024-09-16T00:08:57.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5564999999999999e-05 2024-09-16T00:08:57.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.750601+0000", 2024-09-16T00:08:57.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010751199999999999 2024-09-16T00:08:57.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:57.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752665+0000", 2024-09-16T00:08:57.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00206442 2024-09-16T00:08:57.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752696+0000", 2024-09-16T00:08:57.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0476000000000001e-05 2024-09-16T00:08:57.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752720+0000", 2024-09-16T00:08:57.909 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4482999999999999e-05 2024-09-16T00:08:57.909 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.909 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.909 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.909 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.909 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:244 3.1b 3:da01a9ef:::100000002f1.00000000:head [write 0~9324] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:08:57.909 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.800988+0000", 2024-09-16T00:08:57.909 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.016418826, 2024-09-16T00:08:57.909 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026477689999999999, 2024-09-16T00:08:57.909 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.909 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.909 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.909 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:08:57.910 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:08:57.910 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:08:57.910 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 244 2024-09-16T00:08:57.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.910 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.910 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.910 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.800988+0000", 2024-09-16T00:08:57.910 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.910 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.910 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.800988+0000", 2024-09-16T00:08:57.910 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.800989+0000", 2024-09-16T00:08:57.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.33e-07 2024-09-16T00:08:57.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801004+0000", 2024-09-16T00:08:57.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4531e-05 2024-09-16T00:08:57.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801015+0000", 2024-09-16T00:08:57.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0988000000000001e-05 2024-09-16T00:08:57.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801020+0000", 2024-09-16T00:08:57.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2820000000000004e-06 2024-09-16T00:08:57.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801072+0000", 2024-09-16T00:08:57.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1830999999999999e-05 2024-09-16T00:08:57.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801142+0000", 2024-09-16T00:08:57.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0182e-05 2024-09-16T00:08:57.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:57.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801218+0000", 2024-09-16T00:08:57.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6093999999999993e-05 2024-09-16T00:08:57.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:57.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801707+0000", 2024-09-16T00:08:57.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000488837 2024-09-16T00:08:57.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:57.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803548+0000", 2024-09-16T00:08:57.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018406519999999999 2024-09-16T00:08:57.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803600+0000", 2024-09-16T00:08:57.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2429999999999998e-05 2024-09-16T00:08:57.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803636+0000", 2024-09-16T00:08:57.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6009e-05 2024-09-16T00:08:57.914 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.914 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.915 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.915 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:323 3.3 e58/20 3:c319d237:::10000000354.00000000:head v 58'3, mlcod=57'1)", 2024-09-16T00:08:57.915 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:08:57.915 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.78474674200000005, 2024-09-16T00:08:57.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034011000000000002, 2024-09-16T00:08:57.915 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:57.915 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:57.915 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:57.915 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:57.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.915 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:57.915 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:08:57.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:57.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:08:57.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:57.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:57.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032661+0000", 2024-09-16T00:08:57.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7000000000000003e-07 2024-09-16T00:08:57.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:57.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032670+0000", 2024-09-16T00:08:57.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.602e-06 2024-09-16T00:08:57.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:57.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032682+0000", 2024-09-16T00:08:57.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1754e-05 2024-09-16T00:08:57.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:57.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032688+0000", 2024-09-16T00:08:57.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8140000000000003e-06 2024-09-16T00:08:57.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:57.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032740+0000", 2024-09-16T00:08:57.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1981000000000003e-05 2024-09-16T00:08:57.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:57.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032765+0000", 2024-09-16T00:08:57.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4097e-05 2024-09-16T00:08:57.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:57.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.036023+0000", 2024-09-16T00:08:57.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032585240000000001 2024-09-16T00:08:57.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:57.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:57.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:57.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.036061+0000", 2024-09-16T00:08:57.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8457999999999997e-05 2024-09-16T00:08:57.919 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.919 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.919 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.919 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:57.919 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:57.919 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:57.921 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:08:57.921 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:57.920+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:57.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.989+0000 7f1aa8ff9640 1 -- 172.21.15.139:0/2261865974 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v59) ==== 113+0+0 (secure 0 0 0) 0x7f1a9c07e460 con 0x7f1aac150bd0 2024-09-16T00:08:57.990 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T00:08:57.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.991+0000 7f1ab21a6640 1 -- 172.21.15.139:0/2261865974 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1a8c06c2e0 msgr2=0x7f1a8c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:57.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.991+0000 7f1ab21a6640 1 --2- 172.21.15.139:0/2261865974 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1a8c06c2e0 0x7f1a8c06e7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0ccce0 tx=0x7f1a94009040 comp rx=0 tx=0).stop 2024-09-16T00:08:57.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.991+0000 7f1ab21a6640 1 -- 172.21.15.139:0/2261865974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1aac150bd0 msgr2=0x7f1aac158830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:57.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.991+0000 7f1ab21a6640 1 --2- 172.21.15.139:0/2261865974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1aac150bd0 0x7f1aac158830 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c000c00 tx=0x7f1a9c004730 comp rx=0 tx=0).stop 2024-09-16T00:08:57.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.991+0000 7f1ab21a6640 1 -- 172.21.15.139:0/2261865974 shutdown_connections 2024-09-16T00:08:57.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.991+0000 7f1ab21a6640 1 --2- 172.21.15.139:0/2261865974 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1a8c06c2e0 0x7f1a8c06e7a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:57.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.991+0000 7f1ab21a6640 1 --2- 172.21.15.139:0/2261865974 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1aac15afc0 0x7f1aac1f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:57.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.991+0000 7f1ab21a6640 1 --2- 172.21.15.139:0/2261865974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1aac150fb0 0x7f1aac158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:57.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.991+0000 7f1ab21a6640 1 --2- 172.21.15.139:0/2261865974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1aac150bd0 0x7f1aac158830 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:57.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.991+0000 7f1ab21a6640 1 -- 172.21.15.139:0/2261865974 >> 172.21.15.139:0/2261865974 conn(0x7f1aac06da40 msgr2=0x7f1aac0c7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:57.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.992+0000 7f1ab21a6640 1 -- 172.21.15.139:0/2261865974 shutdown_connections 2024-09-16T00:08:57.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:57.992+0000 7f1ab21a6640 1 -- 172.21.15.139:0/2261865974 wait complete. 2024-09-16T00:08:58.008 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T00:08:58.021 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:58.021+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:58.091 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:58.091 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:58.091 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:58.091 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:58.102 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:08:58.121 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:58.121+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:58.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.167+0000 7f2f09d31640 1 Processor -- start 2024-09-16T00:08:58.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.167+0000 7f2f09d31640 1 -- start start 2024-09-16T00:08:58.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.168+0000 7f2f09d31640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2f04154a10 0x7f2f040cd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.168+0000 7f2f09d31640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0406ea70 0x7f2f040cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.168+0000 7f2f09d31640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f0415a950 con 0x7f2f0406bd20 2024-09-16T00:08:58.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.168+0000 7f2f09d31640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f2f0415ac30 con 0x7f2f04154a10 2024-09-16T00:08:58.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.168+0000 7f2f09d31640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2f0415af10 con 0x7f2f0406ea70 2024-09-16T00:08:58.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.168+0000 7f2f03fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2f0406bd20 0x7f2f0415e130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:60338/0 (socket says 172.21.15.139:60338) 2024-09-16T00:08:58.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.168+0000 7f2f03fff640 1 -- 172.21.15.139:0/3522689110 learned_addr learned my addr 172.21.15.139:0/3522689110 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:58.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.168+0000 7f2f02ffd640 1 --2- 172.21.15.139:0/3522689110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0406ea70 0x7f2f040cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.168+0000 7f2f037fe640 1 --2- 172.21.15.139:0/3522689110 >> v2:172.21.15.164:3300/0 conn(0x7f2f04154a10 0x7f2f040cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.168+0000 7f2f027fc640 1 -- 172.21.15.139:0/3522689110 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3588371277 0 0) 0x7f2f0415a950 con 0x7f2f0406bd20 2024-09-16T00:08:58.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.168+0000 7f2f037fe640 1 -- 172.21.15.139:0/3522689110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0406ea70 msgr2=0x7f2f040cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:58.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.168+0000 7f2f037fe640 1 --2- 172.21.15.139:0/3522689110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0406ea70 0x7f2f040cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.168+0000 7f2f037fe640 1 -- 172.21.15.139:0/3522689110 >> v1:172.21.15.139:6789/0 conn(0x7f2f0406bd20 legacy=0x7f2f0415e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.168+0000 7f2f037fe640 1 -- 172.21.15.139:0/3522689110 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0415b1f0 con 0x7f2f04154a10 2024-09-16T00:08:58.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.168+0000 7f2f02ffd640 1 --2- 172.21.15.139:0/3522689110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0406ea70 0x7f2f040cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:58.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.169+0000 7f2f037fe640 1 --2- 172.21.15.139:0/3522689110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f04154a10 0x7f2f040cd030 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40097a0 tx=0x7f2ef402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a5cfb4ff37b544a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:58.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.169+0000 7f2f027fc640 1 -- 172.21.15.139:0/3522689110 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4037070 con 0x7f2f04154a10 2024-09-16T00:08:58.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.169+0000 7f2f027fc640 1 -- 172.21.15.139:0/3522689110 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef4032440 con 0x7f2f04154a10 2024-09-16T00:08:58.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.169+0000 7f2f027fc640 1 -- 172.21.15.139:0/3522689110 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4032810 con 0x7f2f04154a10 2024-09-16T00:08:58.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.170+0000 7f2f09d31640 1 -- 172.21.15.139:0/3522689110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f04154a10 msgr2=0x7f2f040cd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.170+0000 7f2f09d31640 1 --2- 172.21.15.139:0/3522689110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f04154a10 0x7f2f040cd030 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40097a0 tx=0x7f2ef402a210 comp rx=0 tx=0).stop 2024-09-16T00:08:58.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.170+0000 7f2f09d31640 1 -- 172.21.15.139:0/3522689110 shutdown_connections 2024-09-16T00:08:58.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.170+0000 7f2f09d31640 1 --2- 172.21.15.139:0/3522689110 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0406ea70 0x7f2f040cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.170+0000 7f2f09d31640 1 --2- 172.21.15.139:0/3522689110 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f04154a10 0x7f2f040cd030 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.170+0000 7f2f09d31640 1 -- 172.21.15.139:0/3522689110 >> 172.21.15.139:0/3522689110 conn(0x7f2f0406cc30 msgr2=0x7f2f0406d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:58.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.170+0000 7f2f09d31640 1 -- 172.21.15.139:0/3522689110 shutdown_connections 2024-09-16T00:08:58.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.170+0000 7f2f09d31640 1 -- 172.21.15.139:0/3522689110 wait complete. 2024-09-16T00:08:58.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.172+0000 7f2f09d31640 1 Processor -- start 2024-09-16T00:08:58.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.172+0000 7f2f09d31640 1 -- start start 2024-09-16T00:08:58.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.172+0000 7f2f09d31640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0406bd20 0x7f2f0414df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.172+0000 7f2f09d31640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f0406ea70 0x7f2f0415cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.173+0000 7f2f09d31640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f04154a10 0x7f2f040c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.173+0000 7f2f09d31640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2f04154df0 con 0x7f2f0406ea70 2024-09-16T00:08:58.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.173+0000 7f2f09d31640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0406da90 con 0x7f2f04154a10 2024-09-16T00:08:58.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.173+0000 7f2f09d31640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2f0406c460 con 0x7f2f0406bd20 2024-09-16T00:08:58.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.173+0000 7f2f037fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0406bd20 0x7f2f0414df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.173+0000 7f2f037fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0406bd20 0x7f2f0414df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:56714/0 (socket says 172.21.15.139:56714) 2024-09-16T00:08:58.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.173+0000 7f2f037fe640 1 -- 172.21.15.139:0/1857143196 learned_addr learned my addr 172.21.15.139:0/1857143196 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:58.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.173+0000 7f2f02ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f0406ea70 0x7f2f0415cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.173+0000 7f2f03fff640 1 --2- 172.21.15.139:0/1857143196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f04154a10 0x7f2f040c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.173+0000 7f2f037fe640 1 -- 172.21.15.139:0/1857143196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f04154a10 msgr2=0x7f2f040c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.173+0000 7f2f037fe640 1 --2- 172.21.15.139:0/1857143196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f04154a10 0x7f2f040c6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.173+0000 7f2f037fe640 1 -- 172.21.15.139:0/1857143196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f0406ea70 msgr2=0x7f2f0415cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.173+0000 7f2f037fe640 1 --2- 172.21.15.139:0/1857143196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f0406ea70 0x7f2f0415cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.173+0000 7f2f037fe640 1 -- 172.21.15.139:0/1857143196 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f041f0d30 con 0x7f2f0406bd20 2024-09-16T00:08:58.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.173+0000 7f2f037fe640 1 --2- 172.21.15.139:0/1857143196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0406bd20 0x7f2f0414df40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40097a0 tx=0x7f2ef40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:58.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.173+0000 7f2eebfff640 1 -- 172.21.15.139:0/1857143196 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4037050 con 0x7f2f0406bd20 2024-09-16T00:08:58.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.174+0000 7f2eebfff640 1 -- 172.21.15.139:0/1857143196 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef4038040 con 0x7f2f0406bd20 2024-09-16T00:08:58.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.174+0000 7f2eebfff640 1 -- 172.21.15.139:0/1857143196 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4004490 con 0x7f2f0406bd20 2024-09-16T00:08:58.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.174+0000 7f2f09d31640 1 -- 172.21.15.139:0/1857143196 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f040c50c0 con 0x7f2f0406bd20 2024-09-16T00:08:58.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.174+0000 7f2f09d31640 1 -- 172.21.15.139:0/1857143196 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f040c5680 con 0x7f2f0406bd20 2024-09-16T00:08:58.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.175+0000 7f2f09d31640 1 -- 172.21.15.139:0/1857143196 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f040cd030 con 0x7f2f0406bd20 2024-09-16T00:08:58.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.175+0000 7f2eebfff640 1 -- 172.21.15.139:0/1857143196 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2ef403c6d0 con 0x7f2f0406bd20 2024-09-16T00:08:58.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.176+0000 7f2eebfff640 1 --2- 172.21.15.139:0/1857143196 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2ef006c2e0 0x7f2ef006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.176+0000 7f2eebfff640 1 -- 172.21.15.139:0/1857143196 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(59..59 src has 1..59) ==== 5886+0+0 (secure 0 0 0) 0x7f2ef40abcc0 con 0x7f2f0406bd20 2024-09-16T00:08:58.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.176+0000 7f2f02ffd640 1 --2- 172.21.15.139:0/1857143196 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2ef006c2e0 0x7f2ef006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.177+0000 7f2f02ffd640 1 --2- 172.21.15.139:0/1857143196 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2ef006c2e0 0x7f2ef006e7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009560 tx=0x7f2ef8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:58.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.181+0000 7f2eebfff640 1 -- 172.21.15.139:0/1857143196 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2ef4079da0 con 0x7f2f0406bd20 2024-09-16T00:08:58.222 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:58.222+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:58.228 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:58.228 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:58.228 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:08:58.228 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:08:58.228 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:58.239 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:08:58.323 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:58.322+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:58.402 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:58.402 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:08:58.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:08:58.403 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:08:58.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.403 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:58.403 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:58.403 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.784554263000004, 2024-09-16T00:08:58.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037514929999999998, 2024-09-16T00:08:58.403 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.404 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.404 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.404 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:58.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:08:58.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615364+0000", 2024-09-16T00:08:58.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8430000000000003e-06 2024-09-16T00:08:58.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615371+0000", 2024-09-16T00:08:58.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.148e-06 2024-09-16T00:08:58.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615380+0000", 2024-09-16T00:08:58.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5889999999999992e-06 2024-09-16T00:08:58.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615389+0000", 2024-09-16T00:08:58.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1620000000000008e-06 2024-09-16T00:08:58.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615525+0000", 2024-09-16T00:08:58.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000136083 2024-09-16T00:08:58.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615662+0000", 2024-09-16T00:08:58.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013696100000000001 2024-09-16T00:08:58.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:58.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615799+0000", 2024-09-16T00:08:58.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013781399999999999 2024-09-16T00:08:58.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:58.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:08:58.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002363502 2024-09-16T00:08:58.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:58.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619018+0000", 2024-09-16T00:08:58.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085522199999999999 2024-09-16T00:08:58.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619068+0000", 2024-09-16T00:08:58.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9545000000000001e-05 2024-09-16T00:08:58.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619111+0000", 2024-09-16T00:08:58.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3624000000000001e-05 2024-09-16T00:08:58.410 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.410 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.410 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.410 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:58.410 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:58.410 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.784502192000005, 2024-09-16T00:08:58.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043656110000000001, 2024-09-16T00:08:58.410 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.411 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.411 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.411 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:58.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:58.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:08:58.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2199999999999999e-07 2024-09-16T00:08:58.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615416+0000", 2024-09-16T00:08:58.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5590000000000001e-06 2024-09-16T00:08:58.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615423+0000", 2024-09-16T00:08:58.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6250000000000001e-06 2024-09-16T00:08:58.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615426+0000", 2024-09-16T00:08:58.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2540000000000001e-06 2024-09-16T00:08:58.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616075+0000", 2024-09-16T00:08:58.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064968800000000004 2024-09-16T00:08:58.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616142+0000", 2024-09-16T00:08:58.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6203000000000003e-05 2024-09-16T00:08:58.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:58.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616247+0000", 2024-09-16T00:08:58.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010572499999999999 2024-09-16T00:08:58.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:58.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618913+0000", 2024-09-16T00:08:58.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026654339999999999 2024-09-16T00:08:58.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:58.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619712+0000", 2024-09-16T00:08:58.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079885199999999996 2024-09-16T00:08:58.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619744+0000", 2024-09-16T00:08:58.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2790000000000003e-05 2024-09-16T00:08:58.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619778+0000", 2024-09-16T00:08:58.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3059000000000003e-05 2024-09-16T00:08:58.415 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.415 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.415 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.416 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:58.416 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:58.416 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.784466112999993, 2024-09-16T00:08:58.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087673219999999993, 2024-09-16T00:08:58.416 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.416 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.416 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.416 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:58.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:58.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:08:58.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:08:58.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:08:58.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5392000000000001e-05 2024-09-16T00:08:58.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615470+0000", 2024-09-16T00:08:58.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1999999999999999e-06 2024-09-16T00:08:58.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615474+0000", 2024-09-16T00:08:58.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7359999999999999e-06 2024-09-16T00:08:58.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617614+0000", 2024-09-16T00:08:58.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021398189999999998 2024-09-16T00:08:58.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617724+0000", 2024-09-16T00:08:58.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110808 2024-09-16T00:08:58.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:08:58.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617856+0000", 2024-09-16T00:08:58.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131676 2024-09-16T00:08:58.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:58.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:08:58.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029629280000000001 2024-09-16T00:08:58.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:58.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624158+0000", 2024-09-16T00:08:58.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033386549999999998 2024-09-16T00:08:58.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624189+0000", 2024-09-16T00:08:58.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1508999999999999e-05 2024-09-16T00:08:58.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624215+0000", 2024-09-16T00:08:58.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6228999999999999e-05 2024-09-16T00:08:58.421 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.421 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.421 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.421 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:08:58.421 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:58.421 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.784432097999996, 2024-09-16T00:08:58.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053285279999999999, 2024-09-16T00:08:58.422 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.422 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.422 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.422 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:58.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:58.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:08:58.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5600000000000002e-07 2024-09-16T00:08:58.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615484+0000", 2024-09-16T00:08:58.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.688e-06 2024-09-16T00:08:58.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615488+0000", 2024-09-16T00:08:58.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.658e-06 2024-09-16T00:08:58.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615490+0000", 2024-09-16T00:08:58.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.728e-06 2024-09-16T00:08:58.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616441+0000", 2024-09-16T00:08:58.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00095065500000000001 2024-09-16T00:08:58.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616539+0000", 2024-09-16T00:08:58.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8335e-05 2024-09-16T00:08:58.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:58.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616631+0000", 2024-09-16T00:08:58.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1260000000000004e-05 2024-09-16T00:08:58.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:58.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619837+0000", 2024-09-16T00:08:58.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003206899 2024-09-16T00:08:58.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:58.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620725+0000", 2024-09-16T00:08:58.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088713599999999996 2024-09-16T00:08:58.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620772+0000", 2024-09-16T00:08:58.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7639999999999998e-05 2024-09-16T00:08:58.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620811+0000", 2024-09-16T00:08:58.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8272999999999998e-05 2024-09-16T00:08:58.426 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.426 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.426 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.427 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:08:58.427 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:58.427 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.784017009999999, 2024-09-16T00:08:58.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029886169999999998, 2024-09-16T00:08:58.427 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.427 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.427 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.427 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:58.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:08:58.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615898+0000", 2024-09-16T00:08:58.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.237e-06 2024-09-16T00:08:58.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615904+0000", 2024-09-16T00:08:58.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:08:58.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615915+0000", 2024-09-16T00:08:58.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1003e-05 2024-09-16T00:08:58.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615922+0000", 2024-09-16T00:08:58.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9959999999999996e-06 2024-09-16T00:08:58.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615988+0000", 2024-09-16T00:08:58.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5806e-05 2024-09-16T00:08:58.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616009+0000", 2024-09-16T00:08:58.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0729000000000001e-05 2024-09-16T00:08:58.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:08:58.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028462130000000002 2024-09-16T00:08:58.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618886+0000", 2024-09-16T00:08:58.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.082e-05 2024-09-16T00:08:58.431 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.431 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.431 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.431 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:08:58.431 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:58.431 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.782405538000006, 2024-09-16T00:08:58.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032610299999999998, 2024-09-16T00:08:58.431 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.432 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.432 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.432 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:58.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:08:58.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617556+0000", 2024-09-16T00:08:58.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7431e-05 2024-09-16T00:08:58.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617847+0000", 2024-09-16T00:08:58.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029133400000000003 2024-09-16T00:08:58.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617858+0000", 2024-09-16T00:08:58.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0222e-05 2024-09-16T00:08:58.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617864+0000", 2024-09-16T00:08:58.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7619999999999998e-06 2024-09-16T00:08:58.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617909+0000", 2024-09-16T00:08:58.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4747999999999999e-05 2024-09-16T00:08:58.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617927+0000", 2024-09-16T00:08:58.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7785999999999999e-05 2024-09-16T00:08:58.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620734+0000", 2024-09-16T00:08:58.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002807305 2024-09-16T00:08:58.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620770+0000", 2024-09-16T00:08:58.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5441999999999997e-05 2024-09-16T00:08:58.435 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.435 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.436 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.436 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:08:58.436 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:58.436 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.781984854000001, 2024-09-16T00:08:58.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028641809999999999, 2024-09-16T00:08:58.436 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.436 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.436 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.436 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:58.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:08:58.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617930+0000", 2024-09-16T00:08:58.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0200000000000002e-07 2024-09-16T00:08:58.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617934+0000", 2024-09-16T00:08:58.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:08:58.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617943+0000", 2024-09-16T00:08:58.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5820000000000002e-06 2024-09-16T00:08:58.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617949+0000", 2024-09-16T00:08:58.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5069999999999997e-06 2024-09-16T00:08:58.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618094+0000", 2024-09-16T00:08:58.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014515599999999999 2024-09-16T00:08:58.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618111+0000", 2024-09-16T00:08:58.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6705999999999999e-05 2024-09-16T00:08:58.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:08:58.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026603070000000002 2024-09-16T00:08:58.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620793+0000", 2024-09-16T00:08:58.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2214000000000001e-05 2024-09-16T00:08:58.440 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.441 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.441 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.441 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-16T00:08:58.441 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:58.441 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.781954064999994, 2024-09-16T00:08:58.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028624549999999999, 2024-09-16T00:08:58.441 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.441 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.441 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.441 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:58.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:58.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:08:58.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:58.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:08:58.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0570000000000001e-06 2024-09-16T00:08:58.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617970+0000", 2024-09-16T00:08:58.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.669e-06 2024-09-16T00:08:58.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617976+0000", 2024-09-16T00:08:58.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1369999999999999e-06 2024-09-16T00:08:58.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618268+0000", 2024-09-16T00:08:58.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029164600000000001 2024-09-16T00:08:58.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618284+0000", 2024-09-16T00:08:58.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5824e-05 2024-09-16T00:08:58.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:08:58.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025127890000000001 2024-09-16T00:08:58.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620822+0000", 2024-09-16T00:08:58.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6009000000000001e-05 2024-09-16T00:08:58.445 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.445 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.445 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.445 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-16T00:08:58.446 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:58.446 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.781926483999996, 2024-09-16T00:08:58.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028650350000000002, 2024-09-16T00:08:58.446 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.446 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.446 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.446 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:58.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:58.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:08:58.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7599999999999998e-07 2024-09-16T00:08:58.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617991+0000", 2024-09-16T00:08:58.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1860000000000001e-06 2024-09-16T00:08:58.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617999+0000", 2024-09-16T00:08:58.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3250000000000002e-06 2024-09-16T00:08:58.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618004+0000", 2024-09-16T00:08:58.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7300000000000002e-06 2024-09-16T00:08:58.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618422+0000", 2024-09-16T00:08:58.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00041763199999999998 2024-09-16T00:08:58.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:08:58.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6112000000000001e-05 2024-09-16T00:08:58.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620824+0000", 2024-09-16T00:08:58.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002385954 2024-09-16T00:08:58.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620853+0000", 2024-09-16T00:08:58.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8719999999999999e-05 2024-09-16T00:08:58.450 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.450 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.450 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.450 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-16T00:08:58.450 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:58.450 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.781900661999998, 2024-09-16T00:08:58.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032703020000000001, 2024-09-16T00:08:58.450 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.450 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.450 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.450 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:58.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:08:58.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618014+0000", 2024-09-16T00:08:58.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7799999999999997e-07 2024-09-16T00:08:58.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618017+0000", 2024-09-16T00:08:58.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4309999999999998e-06 2024-09-16T00:08:58.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618023+0000", 2024-09-16T00:08:58.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8490000000000002e-06 2024-09-16T00:08:58.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618028+0000", 2024-09-16T00:08:58.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3489999999999999e-06 2024-09-16T00:08:58.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:08:58.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055724099999999999 2024-09-16T00:08:58.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618603+0000", 2024-09-16T00:08:58.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7486000000000002e-05 2024-09-16T00:08:58.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621235+0000", 2024-09-16T00:08:58.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002632087 2024-09-16T00:08:58.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621284+0000", 2024-09-16T00:08:58.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8581000000000001e-05 2024-09-16T00:08:58.454 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.454 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.454 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.454 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-16T00:08:58.454 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:58.455 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.780925574999998, 2024-09-16T00:08:58.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002325307, 2024-09-16T00:08:58.455 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.455 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.455 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.455 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:58.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:58.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:08:58.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:08:58.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618993+0000", 2024-09-16T00:08:58.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.687e-06 2024-09-16T00:08:58.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619000+0000", 2024-09-16T00:08:58.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3590000000000002e-06 2024-09-16T00:08:58.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619005+0000", 2024-09-16T00:08:58.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.95e-06 2024-09-16T00:08:58.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619039+0000", 2024-09-16T00:08:58.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3942999999999997e-05 2024-09-16T00:08:58.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619055+0000", 2024-09-16T00:08:58.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6107e-05 2024-09-16T00:08:58.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621287+0000", 2024-09-16T00:08:58.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022318070000000001 2024-09-16T00:08:58.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:08:58.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7129999999999999e-05 2024-09-16T00:08:58.458 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.458 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.458 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.459 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-16T00:08:58.459 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:58.459 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.780887578999995, 2024-09-16T00:08:58.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002314328, 2024-09-16T00:08:58.459 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.459 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.459 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.459 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:58.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:58.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:08:58.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6399999999999998e-07 2024-09-16T00:08:58.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619030+0000", 2024-09-16T00:08:58.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.219e-06 2024-09-16T00:08:58.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619038+0000", 2024-09-16T00:08:58.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9899999999999997e-06 2024-09-16T00:08:58.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619043+0000", 2024-09-16T00:08:58.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5580000000000002e-06 2024-09-16T00:08:58.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619220+0000", 2024-09-16T00:08:58.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000176937 2024-09-16T00:08:58.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619237+0000", 2024-09-16T00:08:58.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7691999999999999e-05 2024-09-16T00:08:58.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621315+0000", 2024-09-16T00:08:58.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020778900000000002 2024-09-16T00:08:58.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.463 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621341+0000", 2024-09-16T00:08:58.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5678e-05 2024-09-16T00:08:58.463 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.463 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.463 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.463 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:39 2.6 2:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:08:58.463 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:58.463 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 74.759636255999993, 2024-09-16T00:08:58.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050072119999999996, 2024-09-16T00:08:58.464 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.464 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.464 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.464 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:58.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:08:58.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640280+0000", 2024-09-16T00:08:58.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5380000000000001e-06 2024-09-16T00:08:58.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640296+0000", 2024-09-16T00:08:58.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5950000000000001e-05 2024-09-16T00:08:58.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640327+0000", 2024-09-16T00:08:58.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0309999999999999e-05 2024-09-16T00:08:58.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640340+0000", 2024-09-16T00:08:58.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3773e-05 2024-09-16T00:08:58.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640422+0000", 2024-09-16T00:08:58.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1122e-05 2024-09-16T00:08:58.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640572+0000", 2024-09-16T00:08:58.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015044099999999999 2024-09-16T00:08:58.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:08:58.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640691+0000", 2024-09-16T00:08:58.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000119393 2024-09-16T00:08:58.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:58.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645006+0000", 2024-09-16T00:08:58.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043143779999999998 2024-09-16T00:08:58.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:58.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645169+0000", 2024-09-16T00:08:58.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016323299999999999 2024-09-16T00:08:58.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645227+0000", 2024-09-16T00:08:58.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8e-05 2024-09-16T00:08:58.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645285+0000", 2024-09-16T00:08:58.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8074e-05 2024-09-16T00:08:58.468 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.468 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.469 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.469 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:31 3.31 3:8c51d9b0:::10000000216.00000000:head [write 0~1217] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:08:58.469 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.316438+0000", 2024-09-16T00:08:58.469 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 2.0834758550000001, 2024-09-16T00:08:58.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001562192, 2024-09-16T00:08:58.469 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.469 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.469 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.469 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:08:58.469 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:08:58.469 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:08:58.470 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 31 2024-09-16T00:08:58.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.470 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.316438+0000", 2024-09-16T00:08:58.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.316438+0000", 2024-09-16T00:08:58.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.316440+0000", 2024-09-16T00:08:58.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2010000000000002e-06 2024-09-16T00:08:58.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.316453+0000", 2024-09-16T00:08:58.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.207e-05 2024-09-16T00:08:58.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.316471+0000", 2024-09-16T00:08:58.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8771e-05 2024-09-16T00:08:58.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.316481+0000", 2024-09-16T00:08:58.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8400000000000007e-06 2024-09-16T00:08:58.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.316559+0000", 2024-09-16T00:08:58.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7980999999999998e-05 2024-09-16T00:08:58.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.316678+0000", 2024-09-16T00:08:58.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000119196 2024-09-16T00:08:58.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:58.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.316809+0000", 2024-09-16T00:08:58.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131049 2024-09-16T00:08:58.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:58.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.317548+0000", 2024-09-16T00:08:58.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00073885300000000004 2024-09-16T00:08:58.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:58.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.317914+0000", 2024-09-16T00:08:58.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00036567099999999998 2024-09-16T00:08:58.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.317960+0000", 2024-09-16T00:08:58.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6430000000000001e-05 2024-09-16T00:08:58.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.318000+0000", 2024-09-16T00:08:58.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0129999999999997e-05 2024-09-16T00:08:58.474 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.475 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.475 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.475 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:52 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e57)", 2024-09-16T00:08:58.475 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.651358+0000", 2024-09-16T00:08:58.475 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.748555614, 2024-09-16T00:08:58.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002097018, 2024-09-16T00:08:58.475 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.475 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.475 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.475 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.651358+0000", 2024-09-16T00:08:58.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.651358+0000", 2024-09-16T00:08:58.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.651360+0000", 2024-09-16T00:08:58.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0979999999999999e-06 2024-09-16T00:08:58.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.651375+0000", 2024-09-16T00:08:58.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.579e-05 2024-09-16T00:08:58.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.651390+0000", 2024-09-16T00:08:58.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4477e-05 2024-09-16T00:08:58.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.651398+0000", 2024-09-16T00:08:58.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2239999999999995e-06 2024-09-16T00:08:58.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.651454+0000", 2024-09-16T00:08:58.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6076999999999999e-05 2024-09-16T00:08:58.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.651533+0000", 2024-09-16T00:08:58.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8998000000000005e-05 2024-09-16T00:08:58.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:08:58.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.652316+0000", 2024-09-16T00:08:58.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000783306 2024-09-16T00:08:58.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:58.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.653144+0000", 2024-09-16T00:08:58.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00082749400000000002 2024-09-16T00:08:58.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:58.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.653364+0000", 2024-09-16T00:08:58.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022024699999999999 2024-09-16T00:08:58.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.653418+0000", 2024-09-16T00:08:58.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3554000000000003e-05 2024-09-16T00:08:58.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.653456+0000", 2024-09-16T00:08:58.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7753000000000003e-05 2024-09-16T00:08:58.480 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.480 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.480 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.480 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:188 3.3b 3:deb70790:::100000002b5.00000000:head [write 0~111] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:08:58.480 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.677997+0000", 2024-09-16T00:08:58.480 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.7219174850000001, 2024-09-16T00:08:58.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018159579999999999, 2024-09-16T00:08:58.481 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.481 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.481 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.481 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:08:58.481 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:08:58.481 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:08:58.481 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 188 2024-09-16T00:08:58.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.481 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.677997+0000", 2024-09-16T00:08:58.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.677997+0000", 2024-09-16T00:08:58.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.677998+0000", 2024-09-16T00:08:58.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.454e-06 2024-09-16T00:08:58.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.678005+0000", 2024-09-16T00:08:58.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2710000000000003e-06 2024-09-16T00:08:58.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.678018+0000", 2024-09-16T00:08:58.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2908000000000001e-05 2024-09-16T00:08:58.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.678025+0000", 2024-09-16T00:08:58.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.404e-06 2024-09-16T00:08:58.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.678070+0000", 2024-09-16T00:08:58.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5528999999999997e-05 2024-09-16T00:08:58.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.678214+0000", 2024-09-16T00:08:58.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143843 2024-09-16T00:08:58.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:08:58.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.678292+0000", 2024-09-16T00:08:58.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7659999999999998e-05 2024-09-16T00:08:58.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:58.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.679617+0000", 2024-09-16T00:08:58.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001324899 2024-09-16T00:08:58.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:58.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.679757+0000", 2024-09-16T00:08:58.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000140786 2024-09-16T00:08:58.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.679788+0000", 2024-09-16T00:08:58.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0759000000000001e-05 2024-09-16T00:08:58.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.486 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.486 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.679813+0000", 2024-09-16T00:08:58.486 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4445000000000001e-05 2024-09-16T00:08:58.486 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.486 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.486 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.486 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:320 3.31 3:8eea9cd9:::10000000351.00000000:head [write 0~111] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:08:58.486 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.025490+0000", 2024-09-16T00:08:58.486 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.3744236060000001, 2024-09-16T00:08:58.486 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019909250000000002, 2024-09-16T00:08:58.486 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.486 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.487 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.487 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:08:58.487 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:08:58.487 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:08:58.487 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 320 2024-09-16T00:08:58.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.487 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.487 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025490+0000", 2024-09-16T00:08:58.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025490+0000", 2024-09-16T00:08:58.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025492+0000", 2024-09-16T00:08:58.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0729999999999999e-06 2024-09-16T00:08:58.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025496+0000", 2024-09-16T00:08:58.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7570000000000004e-06 2024-09-16T00:08:58.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025506+0000", 2024-09-16T00:08:58.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0023e-05 2024-09-16T00:08:58.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025514+0000", 2024-09-16T00:08:58.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3429999999999998e-06 2024-09-16T00:08:58.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025565+0000", 2024-09-16T00:08:58.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0905999999999997e-05 2024-09-16T00:08:58.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025646+0000", 2024-09-16T00:08:58.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1705000000000005e-05 2024-09-16T00:08:58.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:08:58.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025723+0000", 2024-09-16T00:08:58.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6619999999999995e-05 2024-09-16T00:08:58.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:58.491 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.026282+0000", 2024-09-16T00:08:58.491 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055940999999999999 2024-09-16T00:08:58.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.491 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:58.491 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.027384+0000", 2024-09-16T00:08:58.491 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00110177 2024-09-16T00:08:58.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.491 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.491 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.027439+0000", 2024-09-16T00:08:58.491 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5139999999999997e-05 2024-09-16T00:08:58.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.491 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.491 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.027481+0000", 2024-09-16T00:08:58.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2178000000000002e-05 2024-09-16T00:08:58.492 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.492 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.492 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.492 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.492 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:54 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e58)", 2024-09-16T00:08:58.492 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.652050+0000", 2024-09-16T00:08:58.492 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.74786441299999995, 2024-09-16T00:08:58.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039842300000000001, 2024-09-16T00:08:58.492 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.492 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.492 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.492 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.493 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.493 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652050+0000", 2024-09-16T00:08:58.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.493 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.493 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652050+0000", 2024-09-16T00:08:58.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.493 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.493 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652052+0000", 2024-09-16T00:08:58.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.857e-06 2024-09-16T00:08:58.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652101+0000", 2024-09-16T00:08:58.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9812999999999999e-05 2024-09-16T00:08:58.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652122+0000", 2024-09-16T00:08:58.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0659e-05 2024-09-16T00:08:58.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652133+0000", 2024-09-16T00:08:58.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0977e-05 2024-09-16T00:08:58.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652198+0000", 2024-09-16T00:08:58.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5229999999999997e-05 2024-09-16T00:08:58.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652293+0000", 2024-09-16T00:08:58.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4596e-05 2024-09-16T00:08:58.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:08:58.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.654316+0000", 2024-09-16T00:08:58.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020229699999999998 2024-09-16T00:08:58.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.496 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:58.496 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.655783+0000", 2024-09-16T00:08:58.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014669240000000001 2024-09-16T00:08:58.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.496 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:58.496 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.655974+0000", 2024-09-16T00:08:58.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000191437 2024-09-16T00:08:58.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.496 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.496 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.656008+0000", 2024-09-16T00:08:58.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3603000000000003e-05 2024-09-16T00:08:58.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.497 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.656034+0000", 2024-09-16T00:08:58.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6163999999999999e-05 2024-09-16T00:08:58.497 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.497 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.497 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.497 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:650 3.3e 3:7f23b022:::100000004e2.00000000:head [write 0~9466] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:08:58.497 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.174315+0000", 2024-09-16T00:08:58.497 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.22559902000000001, 2024-09-16T00:08:58.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001879726, 2024-09-16T00:08:58.498 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.498 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.498 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.498 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:08:58.498 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:08:58.498 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:08:58.498 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 650 2024-09-16T00:08:58.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.498 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.498 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.498 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.498 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174315+0000", 2024-09-16T00:08:58.498 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.499 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174315+0000", 2024-09-16T00:08:58.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.499 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174318+0000", 2024-09-16T00:08:58.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8430000000000001e-06 2024-09-16T00:08:58.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.499 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174343+0000", 2024-09-16T00:08:58.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5551e-05 2024-09-16T00:08:58.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.500 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.500 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174368+0000", 2024-09-16T00:08:58.500 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4142e-05 2024-09-16T00:08:58.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.500 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.500 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174380+0000", 2024-09-16T00:08:58.500 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2269000000000001e-05 2024-09-16T00:08:58.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.500 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174451+0000", 2024-09-16T00:08:58.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0669999999999999e-05 2024-09-16T00:08:58.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.501 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174571+0000", 2024-09-16T00:08:58.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000120919 2024-09-16T00:08:58.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.501 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:08:58.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174672+0000", 2024-09-16T00:08:58.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010018699999999999 2024-09-16T00:08:58.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.502 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:58.502 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.175572+0000", 2024-09-16T00:08:58.502 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00090064699999999997 2024-09-16T00:08:58.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.502 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:58.502 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.176083+0000", 2024-09-16T00:08:58.502 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00051112200000000003 2024-09-16T00:08:58.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.502 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.502 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.176144+0000", 2024-09-16T00:08:58.502 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0297000000000002e-05 2024-09-16T00:08:58.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.503 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.503 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.176195+0000", 2024-09-16T00:08:58.503 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1078999999999997e-05 2024-09-16T00:08:58.503 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.503 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.503 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.503 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:715 3.25 3:a71cd800:::10000000537.00000000:head [write 0~6367] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:08:58.503 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.358984+0000", 2024-09-16T00:08:58.503 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.040930154000000003, 2024-09-16T00:08:58.503 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016671050000000001, 2024-09-16T00:08:58.504 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:08:58.504 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:08:58.504 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:08:58.504 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:08:58.504 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:08:58.504 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:08:58.504 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 715 2024-09-16T00:08:58.504 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.504 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:08:58.504 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.504 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:08:58.504 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.358984+0000", 2024-09-16T00:08:58.504 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.505 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:08:58.505 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.358984+0000", 2024-09-16T00:08:58.505 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:08:58.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.505 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:08:58.505 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.358986+0000", 2024-09-16T00:08:58.505 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8279999999999999e-06 2024-09-16T00:08:58.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.505 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:08:58.506 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.358998+0000", 2024-09-16T00:08:58.506 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2542e-05 2024-09-16T00:08:58.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.506 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.506 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:08:58.506 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.359014+0000", 2024-09-16T00:08:58.506 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6106000000000001e-05 2024-09-16T00:08:58.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.506 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.506 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:08:58.506 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.359024+0000", 2024-09-16T00:08:58.506 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4939999999999992e-06 2024-09-16T00:08:58.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.507 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:08:58.507 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.359086+0000", 2024-09-16T00:08:58.507 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1859999999999994e-05 2024-09-16T00:08:58.507 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.507 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:08:58.507 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.359220+0000", 2024-09-16T00:08:58.507 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013377199999999999 2024-09-16T00:08:58.507 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.507 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:08:58.507 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.359299+0000", 2024-09-16T00:08:58.507 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9708000000000003e-05 2024-09-16T00:08:58.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.508 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:08:58.508 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360215+0000", 2024-09-16T00:08:58.508 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00091609700000000005 2024-09-16T00:08:58.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.508 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:08:58.508 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360578+0000", 2024-09-16T00:08:58.508 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00036229300000000002 2024-09-16T00:08:58.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.508 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:08:58.508 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360609+0000", 2024-09-16T00:08:58.509 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0876000000000001e-05 2024-09-16T00:08:58.509 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:08:58.509 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:08:58.509 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:08:58.509 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360651+0000", 2024-09-16T00:08:58.509 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2528999999999999e-05 2024-09-16T00:08:58.509 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.509 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.509 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.509 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:08:58.509 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:08:58.509 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:58.510 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:08:58.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.408+0000 7f2f09d31640 1 -- 172.21.15.139:0/1857143196 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) -- 0x7f2f0406c820 con 0x7f2f0406bd20 2024-09-16T00:08:58.511 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:58.510+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:08:58.512 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:08:58.611 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:58.610+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:58.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.667+0000 7fe91e511640 1 Processor -- start 2024-09-16T00:08:58.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.667+0000 7fe91e511640 1 -- start start 2024-09-16T00:08:58.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.668+0000 7fe91e511640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe9180b2170 0x7fe91815a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.668+0000 7fe91e511640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe91806da30 0x7fe9180cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.668+0000 7fe91e511640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9180c8800 con 0x7fe918073bf0 2024-09-16T00:08:58.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.668+0000 7fe91e511640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe9180c7980 con 0x7fe91806da30 2024-09-16T00:08:58.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.668+0000 7fe91e511640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fe918073480 con 0x7fe9180b2170 2024-09-16T00:08:58.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.668+0000 7fe9177fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe91806da30 0x7fe9180cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.668+0000 7fe9177fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe91806da30 0x7fe9180cd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41850/0 (socket says 172.21.15.139:41850) 2024-09-16T00:08:58.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.668+0000 7fe9177fe640 1 -- 172.21.15.139:0/2773592525 learned_addr learned my addr 172.21.15.139:0/2773592525 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:58.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.668+0000 7fe917fff640 1 --2- 172.21.15.139:0/2773592525 >> v2:172.21.15.164:3300/0 conn(0x7fe9180b2170 0x7fe91815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.668+0000 7fe9177fe640 1 -- 172.21.15.139:0/2773592525 >> v2:172.21.15.164:3300/0 conn(0x7fe9180b2170 msgr2=0x7fe91815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:58.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.668+0000 7fe9177fe640 1 --2- 172.21.15.139:0/2773592525 >> v2:172.21.15.164:3300/0 conn(0x7fe9180b2170 0x7fe91815a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.668+0000 7fe9177fe640 1 -- 172.21.15.139:0/2773592525 >> v1:172.21.15.139:6789/0 conn(0x7fe918073bf0 legacy=0x7fe91815de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.668+0000 7fe9177fe640 1 -- 172.21.15.139:0/2773592525 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9180c7bd0 con 0x7fe91806da30 2024-09-16T00:08:58.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.669+0000 7fe9177fe640 1 --2- 172.21.15.139:0/2773592525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe91806da30 0x7fe9180cd1c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe9180b2c20 tx=0x7fe90c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d964de57757065 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:58.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.669+0000 7fe916ffd640 1 -- 172.21.15.139:0/2773592525 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c037070 con 0x7fe91806da30 2024-09-16T00:08:58.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.669+0000 7fe916ffd640 1 -- 172.21.15.139:0/2773592525 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe90c032440 con 0x7fe91806da30 2024-09-16T00:08:58.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.669+0000 7fe916ffd640 1 -- 172.21.15.139:0/2773592525 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c032810 con 0x7fe91806da30 2024-09-16T00:08:58.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.670+0000 7fe91e511640 1 -- 172.21.15.139:0/2773592525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe91806da30 msgr2=0x7fe9180cd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.670+0000 7fe91e511640 1 --2- 172.21.15.139:0/2773592525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe91806da30 0x7fe9180cd1c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe9180b2c20 tx=0x7fe90c02a270 comp rx=0 tx=0).stop 2024-09-16T00:08:58.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.670+0000 7fe91e511640 1 -- 172.21.15.139:0/2773592525 shutdown_connections 2024-09-16T00:08:58.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.670+0000 7fe91e511640 1 --2- 172.21.15.139:0/2773592525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe91806da30 0x7fe9180cd1c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.670+0000 7fe91e511640 1 --2- 172.21.15.139:0/2773592525 >> v2:172.21.15.164:3300/0 conn(0x7fe9180b2170 0x7fe91815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.670+0000 7fe91e511640 1 -- 172.21.15.139:0/2773592525 >> 172.21.15.139:0/2773592525 conn(0x7fe918067950 msgr2=0x7fe918067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:58.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.670+0000 7fe91e511640 1 -- 172.21.15.139:0/2773592525 shutdown_connections 2024-09-16T00:08:58.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.670+0000 7fe91e511640 1 -- 172.21.15.139:0/2773592525 wait complete. 2024-09-16T00:08:58.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.671+0000 7fe91e511640 1 Processor -- start 2024-09-16T00:08:58.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.671+0000 7fe91e511640 1 -- start start 2024-09-16T00:08:58.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.672+0000 7fe91e511640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe91806da30 0x7fe9181e4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.672+0000 7fe91e511640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe918073bf0 0x7fe918156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.674 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.672+0000 7fe91e511640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9180b2170 0x7fe918156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.672+0000 7fe91e511640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe91815ae50 con 0x7fe91806da30 2024-09-16T00:08:58.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.672+0000 7fe91e511640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe9180736d0 con 0x7fe9180b2170 2024-09-16T00:08:58.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.672+0000 7fe91e511640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe9180c4020 con 0x7fe918073bf0 2024-09-16T00:08:58.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.673+0000 7fe9177fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe918073bf0 0x7fe918156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.673+0000 7fe917fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe91806da30 0x7fe9181e4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.673+0000 7fe9177fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe918073bf0 0x7fe918156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:56722/0 (socket says 172.21.15.139:56722) 2024-09-16T00:08:58.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.673+0000 7fe917fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe91806da30 0x7fe9181e4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41864/0 (socket says 172.21.15.139:41864) 2024-09-16T00:08:58.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.673+0000 7fe9177fe640 1 -- 172.21.15.139:0/1983459977 learned_addr learned my addr 172.21.15.139:0/1983459977 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:58.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.673+0000 7fe91caae640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9180b2170 0x7fe918156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.673+0000 7fe9177fe640 1 -- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9180b2170 msgr2=0x7fe918156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.673+0000 7fe9177fe640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe9180b2170 0x7fe918156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.673+0000 7fe9177fe640 1 -- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe91806da30 msgr2=0x7fe9181e4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.673+0000 7fe9177fe640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe91806da30 0x7fe9181e4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.673+0000 7fe9177fe640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe918157260 con 0x7fe918073bf0 2024-09-16T00:08:58.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.673+0000 7fe9177fe640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe918073bf0 0x7fe918156200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe90c02a2b0 tx=0x7fe90c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:58.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.673+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c037040 con 0x7fe918073bf0 2024-09-16T00:08:58.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.673+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe90c042070 con 0x7fe918073bf0 2024-09-16T00:08:58.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.673+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9181f51f0 con 0x7fe918073bf0 2024-09-16T00:08:58.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.673+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c030880 con 0x7fe918073bf0 2024-09-16T00:08:58.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.674+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9181f54d0 con 0x7fe918073bf0 2024-09-16T00:08:58.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.675+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fe9180cdb20 con 0x7fe918073bf0 2024-09-16T00:08:58.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.675+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe90c030a20 con 0x7fe918073bf0 2024-09-16T00:08:58.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.676+0000 7fe914ff9640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe90406c2b0 0x7fe90406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.676+0000 7fe917fff640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe90406c2b0 0x7fe90406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.676+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(59..59 src has 1..59) ==== 5886+0+0 (secure 0 0 0) 0x7fe90c0abd70 con 0x7fe918073bf0 2024-09-16T00:08:58.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.676+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v59) ==== 54+0+16 (secure 0 0 0) 0x7fe9180cdb20 con 0x7fe918073bf0 2024-09-16T00:08:58.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.676+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe9180c2ed0 0x7fe91814e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.676+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe91806d610 con 0x7fe9180c2ed0 2024-09-16T00:08:58.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.676+0000 7fe91caae640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe9180c2ed0 0x7fe91814e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.676+0000 7fe917fff640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe90406c2b0 0x7fe90406e770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe908009aa0 tx=0x7fe908009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:58.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.677+0000 7fe91caae640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe9180c2ed0 0x7fe91814e970 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:58.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.678+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe91806d610 con 0x7fe9180c2ed0 2024-09-16T00:08:58.681 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:08:58.681 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:08:58.682 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:08:58.682 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:08:58.694 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:08:58.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.700+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe91806c6f0 con 0x7fe9180c2ed0 2024-09-16T00:08:58.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.702+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe91806c6f0 con 0x7fe9180c2ed0 2024-09-16T00:08:58.703 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:08:58.704 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:08:58.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.702+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe9180c32b0 0x7fe9181e1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.702+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fe91814f3c0 con 0x7fe9180c32b0 2024-09-16T00:08:58.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.702+0000 7fe917fff640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe9180c32b0 0x7fe9181e1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.702+0000 7fe917fff640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe9180c32b0 0x7fe9181e1cb0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:58.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.703+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe91814f3c0 con 0x7fe9180c32b0 2024-09-16T00:08:58.712 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:58.712+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:58.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.723+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe918074070 con 0x7fe9180c32b0 2024-09-16T00:08:58.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.725+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe918074070 con 0x7fe9180c32b0 2024-09-16T00:08:58.726 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:08:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:08:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.726+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fe9181e2700 0x7fe918062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.726+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fe9181e2ae0 con 0x7fe9181e2700 2024-09-16T00:08:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.726+0000 7fe9177fe640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fe9181e2700 0x7fe918062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.726+0000 7fe9177fe640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fe9181e2700 0x7fe918062630 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:58.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.727+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9181e2ae0 con 0x7fe9181e2700 2024-09-16T00:08:58.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.744+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe9180caf00 con 0x7fe9181e2700 2024-09-16T00:08:58.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.746+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe9180caf00 con 0x7fe9181e2700 2024-09-16T00:08:58.747 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:08:58.747 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:08:58.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.746+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fe918063080 0x7fe9180c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.746+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fe91815b6d0 con 0x7fe918063080 2024-09-16T00:08:58.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.746+0000 7fe91caae640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fe918063080 0x7fe9180c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.747+0000 7fe91caae640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fe918063080 0x7fe9180c9020 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:58.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.748+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe91815b6d0 con 0x7fe918063080 2024-09-16T00:08:58.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.766+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe91815b6d0 con 0x7fe918063080 2024-09-16T00:08:58.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.768+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe91815b6d0 con 0x7fe918063080 2024-09-16T00:08:58.768 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:08:58.769 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:08:58.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.768+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fe9180c9a70 0x7fe91802c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.768+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fe9180680b0 con 0x7fe9180c9a70 2024-09-16T00:08:58.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.768+0000 7fe917fff640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fe9180c9a70 0x7fe91802c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.768+0000 7fe917fff640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fe9180c9a70 0x7fe91802c640 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:58.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.770+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9180680b0 con 0x7fe9180c9a70 2024-09-16T00:08:58.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.788+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe9180680b0 con 0x7fe9180c9a70 2024-09-16T00:08:58.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.789+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe9180680b0 con 0x7fe9180c9a70 2024-09-16T00:08:58.790 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:08:58.791 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:08:58.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.790+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fe91802d090 0x7fe91802d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.790+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fe9180c9ed0 con 0x7fe91802d090 2024-09-16T00:08:58.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.790+0000 7fe9177fe640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fe91802d090 0x7fe91802d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.791+0000 7fe9177fe640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fe91802d090 0x7fe91802d4f0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:58.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.791+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9180c9ed0 con 0x7fe91802d090 2024-09-16T00:08:58.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.810+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe91806d2c0 con 0x7fe91802d090 2024-09-16T00:08:58.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.812+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe91806d2c0 con 0x7fe91802d090 2024-09-16T00:08:58.813 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:08:58.813 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:08:58.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.812+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fe918207ed0 0x7fe91820a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.812+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fe91802daf0 con 0x7fe918207ed0 2024-09-16T00:08:58.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.812+0000 7fe91caae640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fe918207ed0 0x7fe91820a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.813+0000 7fe91caae640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fe918207ed0 0x7fe91820a2c0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:58.814 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:58.813+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:58.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.813+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe91802daf0 con 0x7fe918207ed0 2024-09-16T00:08:58.825 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:58.825 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:58.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.829+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe918000950 con 0x7fe918207ed0 2024-09-16T00:08:58.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.831+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe918000950 con 0x7fe918207ed0 2024-09-16T00:08:58.832 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:08:58.832 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:08:58.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.831+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fe91820ad10 0x7fe91820d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:58.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.831+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fe91820fed0 con 0x7fe91820ad10 2024-09-16T00:08:58.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.831+0000 7fe917fff640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fe91820ad10 0x7fe91820d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:58.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.832+0000 7fe917fff640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fe91820ad10 0x7fe91820d100 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:58.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.832+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe91820fed0 con 0x7fe91820ad10 2024-09-16T00:08:58.836 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:08:58.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.850+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe91820d640 con 0x7fe91820ad10 2024-09-16T00:08:58.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.852+0000 7fe914ff9640 1 -- 172.21.15.139:0/1983459977 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe91820d640 con 0x7fe91820ad10 2024-09-16T00:08:58.855 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:08:58.858 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:08:58.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe9180c2ed0 msgr2=0x7fe91814e970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe9180c2ed0 0x7fe91814e970 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe9180c32b0 msgr2=0x7fe9181e1cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe9180c32b0 0x7fe9181e1cb0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fe9181e2700 msgr2=0x7fe918062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fe9181e2700 0x7fe918062630 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fe918063080 msgr2=0x7fe9180c9020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fe918063080 0x7fe9180c9020 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fe9180c9a70 msgr2=0x7fe91802c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91caae640 1 -- 172.21.15.139:0/1983459977 reap_dead start 2024-09-16T00:08:58.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fe9180c9a70 0x7fe91802c640 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91caae640 1 -- 172.21.15.139:0/1983459977 reap_dead start 2024-09-16T00:08:58.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fe91802d090 msgr2=0x7fe91802d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fe91802d090 0x7fe91802d4f0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fe918207ed0 msgr2=0x7fe91820a2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fe918207ed0 0x7fe91820a2c0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fe91820ad10 msgr2=0x7fe91820d100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fe91820ad10 0x7fe91820d100 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe90406c2b0 msgr2=0x7fe90406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe90406c2b0 0x7fe90406e770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe908009aa0 tx=0x7fe908009190 comp rx=0 tx=0).stop 2024-09-16T00:08:58.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe918073bf0 msgr2=0x7fe918156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe918073bf0 0x7fe918156200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe90c02a2b0 tx=0x7fe90c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91caae640 1 -- 172.21.15.139:0/1983459977 reap_dead start 2024-09-16T00:08:58.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 shutdown_connections 2024-09-16T00:08:58.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 --2- 172.21.15.139:0/1983459977 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe918073bf0 0x7fe918156200 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 >> 172.21.15.139:0/1983459977 conn(0x7fe918067950 msgr2=0x7fe9180c27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:58.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 shutdown_connections 2024-09-16T00:08:58.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.853+0000 7fe91e511640 1 -- 172.21.15.139:0/1983459977 wait complete. 2024-09-16T00:08:58.913 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:08:58.913+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:08:58.965 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:58.965 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:58.975 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:08:58.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.993+0000 7f2eebfff640 1 -- 172.21.15.139:0/1857143196 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v60) ==== 113+0+0 (secure 0 0 0) 0x7f2ef407e0a0 con 0x7f2f0406bd20 2024-09-16T00:08:58.995 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T00:08:58.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.996+0000 7f2f09d31640 1 -- 172.21.15.139:0/1857143196 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2ef006c2e0 msgr2=0x7f2ef006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.996+0000 7f2f09d31640 1 --2- 172.21.15.139:0/1857143196 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2ef006c2e0 0x7f2ef006e7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009560 tx=0x7f2ef8009190 comp rx=0 tx=0).stop 2024-09-16T00:08:58.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.996+0000 7f2f09d31640 1 -- 172.21.15.139:0/1857143196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0406bd20 msgr2=0x7f2f0414df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:58.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.996+0000 7f2f09d31640 1 --2- 172.21.15.139:0/1857143196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0406bd20 0x7f2f0414df40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40097a0 tx=0x7f2ef40046c0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.997+0000 7f2f09d31640 1 -- 172.21.15.139:0/1857143196 shutdown_connections 2024-09-16T00:08:58.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.997+0000 7f2f09d31640 1 --2- 172.21.15.139:0/1857143196 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2ef006c2e0 0x7f2ef006e7a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.997+0000 7f2f09d31640 1 --2- 172.21.15.139:0/1857143196 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f04154a10 0x7f2f040c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.997+0000 7f2f09d31640 1 --2- 172.21.15.139:0/1857143196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f0406ea70 0x7f2f0415cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.997+0000 7f2f09d31640 1 --2- 172.21.15.139:0/1857143196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2f0406bd20 0x7f2f0414df40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:58.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.997+0000 7f2f09d31640 1 -- 172.21.15.139:0/1857143196 >> 172.21.15.139:0/1857143196 conn(0x7f2f0406cc30 msgr2=0x7f2f04152410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:58.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.997+0000 7f2f09d31640 1 -- 172.21.15.139:0/1857143196 shutdown_connections 2024-09-16T00:08:58.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:58.997+0000 7f2f09d31640 1 -- 172.21.15.139:0/1857143196 wait complete. 2024-09-16T00:08:59.001 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T00:08:59.013 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T00:08:59.054 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:08:59.053+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:08:59.105 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:59.105 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:59.115 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:59.114+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:59.116 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:08:59.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.121+0000 7f403376d640 1 Processor -- start 2024-09-16T00:08:59.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.122+0000 7f403376d640 1 -- start start 2024-09-16T00:08:59.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.123+0000 7f403376d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402c151850 0x7f402c0c6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:59.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.123+0000 7f403376d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f402c0c7390 0x7f402c0c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:59.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.123+0000 7f403376d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f402c0c8d30 con 0x7f402c151c30 2024-09-16T00:08:59.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.123+0000 7f403376d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f402c0c88a0 con 0x7f402c0c7390 2024-09-16T00:08:59.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.123+0000 7f403376d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f402c0c6a90 con 0x7f402c151850 2024-09-16T00:08:59.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.123+0000 7f4031d0a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f402c0c7390 0x7f402c0c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:59.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.123+0000 7f4030d08640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f402c151c30 0x7f402c15ac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:45580/0 (socket says 172.21.15.139:45580) 2024-09-16T00:08:59.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.123+0000 7f4030d08640 1 -- 172.21.15.139:0/2964782091 learned_addr learned my addr 172.21.15.139:0/2964782091 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:59.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.123+0000 7f4031509640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402c151850 0x7f402c0c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:59.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.123+0000 7f4031d0a640 1 -- 172.21.15.139:0/2964782091 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402c151850 msgr2=0x7f402c0c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:08:59.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.123+0000 7f4023fff640 1 -- 172.21.15.139:0/2964782091 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1144204004 0 0) 0x7f402c0c8d30 con 0x7f402c151c30 2024-09-16T00:08:59.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.123+0000 7f4031d0a640 1 --2- 172.21.15.139:0/2964782091 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402c151850 0x7f402c0c6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:59.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.123+0000 7f4031d0a640 1 -- 172.21.15.139:0/2964782091 >> v1:172.21.15.164:6789/0 conn(0x7f402c151c30 legacy=0x7f402c15ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:59.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.123+0000 7f4031d0a640 1 -- 172.21.15.139:0/2964782091 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c0c67b0 con 0x7f402c0c7390 2024-09-16T00:08:59.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.123+0000 7f4031509640 1 --2- 172.21.15.139:0/2964782091 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402c151850 0x7f402c0c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:59.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.124+0000 7f4031d0a640 1 --2- 172.21.15.139:0/2964782091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f402c0c7390 0x7f402c0c8fb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f402400b050 tx=0x7f402402a960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0aa1c1afd91a964 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:59.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.124+0000 7f4023fff640 1 -- 172.21.15.139:0/2964782091 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024034020 con 0x7f402c0c7390 2024-09-16T00:08:59.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.125+0000 7f4023fff640 1 -- 172.21.15.139:0/2964782091 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f402402c040 con 0x7f402c0c7390 2024-09-16T00:08:59.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.125+0000 7f4023fff640 1 -- 172.21.15.139:0/2964782091 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024034020 con 0x7f402c0c7390 2024-09-16T00:08:59.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.125+0000 7f403376d640 1 -- 172.21.15.139:0/2964782091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f402c0c7390 msgr2=0x7f402c0c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:59.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.125+0000 7f403376d640 1 --2- 172.21.15.139:0/2964782091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f402c0c7390 0x7f402c0c8fb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f402400b050 tx=0x7f402402a960 comp rx=0 tx=0).stop 2024-09-16T00:08:59.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.126+0000 7f403376d640 1 -- 172.21.15.139:0/2964782091 shutdown_connections 2024-09-16T00:08:59.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.126+0000 7f403376d640 1 --2- 172.21.15.139:0/2964782091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f402c0c7390 0x7f402c0c8fb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:59.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.126+0000 7f403376d640 1 --2- 172.21.15.139:0/2964782091 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402c151850 0x7f402c0c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:59.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.126+0000 7f403376d640 1 -- 172.21.15.139:0/2964782091 >> 172.21.15.139:0/2964782091 conn(0x7f402c06cc30 msgr2=0x7f402c06d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:59.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.126+0000 7f403376d640 1 -- 172.21.15.139:0/2964782091 shutdown_connections 2024-09-16T00:08:59.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.126+0000 7f403376d640 1 -- 172.21.15.139:0/2964782091 wait complete. 2024-09-16T00:08:59.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.127+0000 7f403376d640 1 Processor -- start 2024-09-16T00:08:59.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.127+0000 7f403376d640 1 -- start start 2024-09-16T00:08:59.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.128+0000 7f403376d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402c0c7390 0x7f402c1443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:59.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.128+0000 7f403376d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402c151850 0x7f402c159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:59.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.128+0000 7f403376d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f402c151c30 0x7f402c163fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:59.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.128+0000 7f403376d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f402c0c8530 con 0x7f402c151c30 2024-09-16T00:08:59.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.128+0000 7f403376d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f402c0c8250 con 0x7f402c0c7390 2024-09-16T00:08:59.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.128+0000 7f403376d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f402c15b320 con 0x7f402c151850 2024-09-16T00:08:59.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.128+0000 7f4031509640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402c0c7390 0x7f402c1443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:59.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.128+0000 7f4031d0a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f402c151c30 0x7f402c163fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:59.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.128+0000 7f4030d08640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402c151850 0x7f402c159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:59.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.128+0000 7f4031d0a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f402c151c30 0x7f402c163fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41890/0 (socket says 172.21.15.139:41890) 2024-09-16T00:08:59.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.128+0000 7f4031d0a640 1 -- 172.21.15.139:0/1758535848 learned_addr learned my addr 172.21.15.139:0/1758535848 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:59.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.129+0000 7f4030d08640 1 -- 172.21.15.139:0/1758535848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402c0c7390 msgr2=0x7f402c1443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:59.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.129+0000 7f4030d08640 1 --2- 172.21.15.139:0/1758535848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402c0c7390 0x7f402c1443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:59.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.129+0000 7f4030d08640 1 -- 172.21.15.139:0/1758535848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f402c151c30 msgr2=0x7f402c163fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:59.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.129+0000 7f4030d08640 1 --2- 172.21.15.139:0/1758535848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f402c151c30 0x7f402c163fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:59.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.129+0000 7f4030d08640 1 -- 172.21.15.139:0/1758535848 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c14dec0 con 0x7f402c151850 2024-09-16T00:08:59.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.129+0000 7f4031509640 1 --2- 172.21.15.139:0/1758535848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402c0c7390 0x7f402c1443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:59.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.129+0000 7f4031d0a640 1 --2- 172.21.15.139:0/1758535848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f402c151c30 0x7f402c163fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:08:59.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.129+0000 7f4030d08640 1 --2- 172.21.15.139:0/1758535848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402c151850 0x7f402c159980 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f401c00c3e0 tx=0x7f401c00c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:59.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.129+0000 7f4021ffb640 1 -- 172.21.15.139:0/1758535848 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c0074e0 con 0x7f402c151850 2024-09-16T00:08:59.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.129+0000 7f4021ffb640 1 -- 172.21.15.139:0/1758535848 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f401c0047e0 con 0x7f402c151850 2024-09-16T00:08:59.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.129+0000 7f4021ffb640 1 -- 172.21.15.139:0/1758535848 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401c004bd0 con 0x7f402c151850 2024-09-16T00:08:59.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.129+0000 7f403376d640 1 -- 172.21.15.139:0/1758535848 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402c1f4290 con 0x7f402c151850 2024-09-16T00:08:59.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.130+0000 7f403376d640 1 -- 172.21.15.139:0/1758535848 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f402c1f4850 con 0x7f402c151850 2024-09-16T00:08:59.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.131+0000 7f403376d640 1 -- 172.21.15.139:0/1758535848 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f402c0c45a0 con 0x7f402c151850 2024-09-16T00:08:59.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.131+0000 7f4021ffb640 1 -- 172.21.15.139:0/1758535848 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f401c011070 con 0x7f402c151850 2024-09-16T00:08:59.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.132+0000 7f4021ffb640 1 --2- 172.21.15.139:0/1758535848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f401006c2b0 0x7f401006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:59.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.132+0000 7f4021ffb640 1 -- 172.21.15.139:0/1758535848 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(60..60 src has 1..60) ==== 5886+0+0 (secure 0 0 0) 0x7f401c08af40 con 0x7f402c151850 2024-09-16T00:08:59.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.132+0000 7f4031509640 1 --2- 172.21.15.139:0/1758535848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f401006c2b0 0x7f401006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:59.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.133+0000 7f4031509640 1 --2- 172.21.15.139:0/1758535848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f401006c2b0 0x7f401006e770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4028009560 tx=0x7f4028009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:59.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.136+0000 7f4021ffb640 1 -- 172.21.15.139:0/1758535848 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f401c059070 con 0x7f402c151850 2024-09-16T00:08:59.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.177+0000 7fb6f95ed640 1 Processor -- start 2024-09-16T00:08:59.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.177+0000 7fb6f95ed640 1 -- start start 2024-09-16T00:08:59.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.178+0000 7fb6f95ed640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6f40c3c40 0x7fb6f4067950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:59.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.179+0000 7fb6f95ed640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f40c38d0 con 0x7fb6f4151a10 2024-09-16T00:08:59.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.179+0000 7fb6f95ed640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6f40c4110 con 0x7fb6f40c7140 2024-09-16T00:08:59.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.179+0000 7fb6f2ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb6f4151a10 0x7fb6f40c5820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:60342/0 (socket says 172.21.15.139:60342) 2024-09-16T00:08:59.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.179+0000 7fb6f95ed640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f40c3160 con 0x7fb6f40c3c40 2024-09-16T00:08:59.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.179+0000 7fb6f2ffd640 1 -- 172.21.15.139:0/2547022680 learned_addr learned my addr 172.21.15.139:0/2547022680 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:08:59.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.179+0000 7fb6f27fc640 1 --2- 172.21.15.139:0/2547022680 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6f40c3c40 0x7fb6f4067950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:59.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.179+0000 7fb6f1ffb640 1 -- 172.21.15.139:0/2547022680 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 857672345 0 0) 0x7fb6f40c38d0 con 0x7fb6f4151a10 2024-09-16T00:08:59.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.179+0000 7fb6f1ffb640 1 -- 172.21.15.139:0/2547022680 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6dc003520 con 0x7fb6f4151a10 2024-09-16T00:08:59.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.179+0000 7fb6f1ffb640 1 -- 172.21.15.139:0/2547022680 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 718656068 0 0) 0x7fb6f40c4110 con 0x7fb6f40c7140 2024-09-16T00:08:59.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.179+0000 7fb6f1ffb640 1 -- 172.21.15.139:0/2547022680 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6f40c38d0 con 0x7fb6f40c7140 2024-09-16T00:08:59.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.179+0000 7fb6f27fc640 1 -- 172.21.15.139:0/2547022680 >> v1:172.21.15.164:6789/0 conn(0x7fb6f40c7140 legacy=0x7fb6f40c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:59.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.179+0000 7fb6f27fc640 1 -- 172.21.15.139:0/2547022680 >> v1:172.21.15.139:6789/0 conn(0x7fb6f4151a10 legacy=0x7fb6f40c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:59.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.179+0000 7fb6f27fc640 1 -- 172.21.15.139:0/2547022680 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f406cc30 con 0x7fb6f40c3c40 2024-09-16T00:08:59.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.180+0000 7fb6f27fc640 1 --2- 172.21.15.139:0/2547022680 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6f40c3c40 0x7fb6f4067950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80097d0 tx=0x7fb6e802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65a11f0dd37892e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:59.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.181+0000 7fb6f1ffb640 1 -- 172.21.15.139:0/2547022680 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8037070 con 0x7fb6f40c3c40 2024-09-16T00:08:59.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.181+0000 7fb6f1ffb640 1 -- 172.21.15.139:0/2547022680 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e8032440 con 0x7fb6f40c3c40 2024-09-16T00:08:59.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.181+0000 7fb6f1ffb640 1 -- 172.21.15.139:0/2547022680 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8032810 con 0x7fb6f40c3c40 2024-09-16T00:08:59.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.182+0000 7fb6f95ed640 1 -- 172.21.15.139:0/2547022680 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6f40c3c40 msgr2=0x7fb6f4067950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:59.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.182+0000 7fb6f95ed640 1 --2- 172.21.15.139:0/2547022680 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6f40c3c40 0x7fb6f4067950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80097d0 tx=0x7fb6e802a250 comp rx=0 tx=0).stop 2024-09-16T00:08:59.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.182+0000 7fb6f95ed640 1 -- 172.21.15.139:0/2547022680 shutdown_connections 2024-09-16T00:08:59.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.182+0000 7fb6f95ed640 1 --2- 172.21.15.139:0/2547022680 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6f40c3c40 0x7fb6f4067950 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:59.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.182+0000 7fb6f95ed640 1 -- 172.21.15.139:0/2547022680 >> 172.21.15.139:0/2547022680 conn(0x7fb6f40cc740 msgr2=0x7fb6f40ccb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:08:59.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.182+0000 7fb6f95ed640 1 -- 172.21.15.139:0/2547022680 shutdown_connections 2024-09-16T00:08:59.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.182+0000 7fb6f95ed640 1 -- 172.21.15.139:0/2547022680 wait complete. 2024-09-16T00:08:59.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.184+0000 7fb6f95ed640 1 Processor -- start 2024-09-16T00:08:59.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.184+0000 7fb6f95ed640 1 -- start start 2024-09-16T00:08:59.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.185+0000 7fb6f95ed640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6f40c3c40 0x7fb6f414cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:59.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.185+0000 7fb6f95ed640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6f40c7140 0x7fb6f4150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:59.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.185+0000 7fb6f2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6f40c3c40 0x7fb6f414cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:59.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.185+0000 7fb6f2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6f40c3c40 0x7fb6f414cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41892/0 (socket says 172.21.15.139:41892) 2024-09-16T00:08:59.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.185+0000 7fb6f2ffd640 1 -- 172.21.15.139:0/436806324 learned_addr learned my addr 172.21.15.139:0/436806324 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:08:59.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.185+0000 7fb6f95ed640 1 --2- 172.21.15.139:0/436806324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6f4151a10 0x7fb6f4144c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:59.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.186+0000 7fb6f27fc640 1 --2- 172.21.15.139:0/436806324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6f40c7140 0x7fb6f4150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:59.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.186+0000 7fb6f95ed640 1 -- 172.21.15.139:0/436806324 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f406d1f0 con 0x7fb6f40c3c40 2024-09-16T00:08:59.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.186+0000 7fb6f95ed640 1 -- 172.21.15.139:0/436806324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f40c3440 con 0x7fb6f40c7140 2024-09-16T00:08:59.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.186+0000 7fb6f95ed640 1 -- 172.21.15.139:0/436806324 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f40c5f30 con 0x7fb6f4151a10 2024-09-16T00:08:59.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.186+0000 7fb6f37fe640 1 --2- 172.21.15.139:0/436806324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6f4151a10 0x7fb6f4144c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:59.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.186+0000 7fb6f27fc640 1 -- 172.21.15.139:0/436806324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6f4151a10 msgr2=0x7fb6f4144c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:59.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.186+0000 7fb6f27fc640 1 --2- 172.21.15.139:0/436806324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6f4151a10 0x7fb6f4144c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:59.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.186+0000 7fb6f27fc640 1 -- 172.21.15.139:0/436806324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6f40c3c40 msgr2=0x7fb6f414cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:08:59.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.186+0000 7fb6f27fc640 1 --2- 172.21.15.139:0/436806324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6f40c3c40 0x7fb6f414cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:08:59.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.186+0000 7fb6f27fc640 1 -- 172.21.15.139:0/436806324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f414f780 con 0x7fb6f40c7140 2024-09-16T00:08:59.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.186+0000 7fb6f27fc640 1 --2- 172.21.15.139:0/436806324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6f40c7140 0x7fb6f4150b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8031000 tx=0x7fb6e80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:59.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.187+0000 7fb6db7fe640 1 -- 172.21.15.139:0/436806324 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8041030 con 0x7fb6f40c7140 2024-09-16T00:08:59.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.187+0000 7fb6db7fe640 1 -- 172.21.15.139:0/436806324 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e803d4e0 con 0x7fb6f40c7140 2024-09-16T00:08:59.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.187+0000 7fb6f95ed640 1 -- 172.21.15.139:0/436806324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f414fa60 con 0x7fb6f40c7140 2024-09-16T00:08:59.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.187+0000 7fb6db7fe640 1 -- 172.21.15.139:0/436806324 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8037040 con 0x7fb6f40c7140 2024-09-16T00:08:59.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.187+0000 7fb6f95ed640 1 -- 172.21.15.139:0/436806324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f4150020 con 0x7fb6f40c7140 2024-09-16T00:08:59.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.188+0000 7fb6f95ed640 1 -- 172.21.15.139:0/436806324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f4151df0 con 0x7fb6f40c7140 2024-09-16T00:08:59.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.189+0000 7fb6db7fe640 1 -- 172.21.15.139:0/436806324 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb6e8044070 con 0x7fb6f40c7140 2024-09-16T00:08:59.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.189+0000 7fb6db7fe640 1 --2- 172.21.15.139:0/436806324 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb6e006c2b0 0x7fb6e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:08:59.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.189+0000 7fb6db7fe640 1 -- 172.21.15.139:0/436806324 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5886+0+0 (secure 0 0 0) 0x7fb6e80ab540 con 0x7fb6f40c7140 2024-09-16T00:08:59.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.189+0000 7fb6f2ffd640 1 --2- 172.21.15.139:0/436806324 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb6e006c2b0 0x7fb6e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:08:59.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.190+0000 7fb6f2ffd640 1 --2- 172.21.15.139:0/436806324 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb6e006c2b0 0x7fb6e006e770 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4000f60 tx=0x7fb6e400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:08:59.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.194+0000 7fb6db7fe640 1 -- 172.21.15.139:0/436806324 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb6e8079640 con 0x7fb6f40c7140 2024-09-16T00:08:59.216 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:59.215+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:59.257 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:59.258 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:59.271 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:08:59.355 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:08:59.355+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:08:59.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.381+0000 7f403376d640 1 -- 172.21.15.139:0/1758535848 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f402c06f660 con 0x7f402c151850 2024-09-16T00:08:59.417 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:08:59.417+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:08:59.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:08:59.422+0000 7fb6f95ed640 1 -- 172.21.15.139:0/436806324 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) -- 0x7fb6f40c6460 con 0x7fb6f40c7140 2024-09-16T00:08:59.435 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:59.435 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:59.446 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:08:59.518 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:59.518+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:59.558 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:59.558 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:59.567 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:08:59.619 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:08:59.619+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:08:59.720 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:08:59.719+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:08:59.721 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:59.721 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:59.732 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:08:59.821 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:08:59.821+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:08:59.846 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:08:59.846 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:08:59.856 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:08:59.940 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:08:59.940+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:00.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.005+0000 7f4021ffb640 1 -- 172.21.15.139:0/1758535848 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v61) ==== 91+0+0 (secure 0 0 0) 0x7f401c05d370 con 0x7f402c151850 2024-09-16T00:09:00.006 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is unset 2024-09-16T00:09:00.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.006+0000 7fb6db7fe640 1 -- 172.21.15.139:0/436806324 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v61) ==== 113+0+0 (secure 0 0 0) 0x7fb6e807d940 con 0x7fb6f40c7140 2024-09-16T00:09:00.007 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T00:09:00.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.008+0000 7f403376d640 1 -- 172.21.15.139:0/1758535848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f401006c2b0 msgr2=0x7f401006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.008+0000 7f403376d640 1 --2- 172.21.15.139:0/1758535848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f401006c2b0 0x7f401006e770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4028009560 tx=0x7f4028009190 comp rx=0 tx=0).stop 2024-09-16T00:09:00.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.008+0000 7f403376d640 1 -- 172.21.15.139:0/1758535848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402c151850 msgr2=0x7f402c159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.008+0000 7f403376d640 1 --2- 172.21.15.139:0/1758535848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402c151850 0x7f402c159980 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f401c00c3e0 tx=0x7f401c00c8a0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.008+0000 7f403376d640 1 -- 172.21.15.139:0/1758535848 shutdown_connections 2024-09-16T00:09:00.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.008+0000 7f403376d640 1 --2- 172.21.15.139:0/1758535848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f401006c2b0 0x7f401006e770 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.008+0000 7f403376d640 1 --2- 172.21.15.139:0/1758535848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f402c151c30 0x7f402c163fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.008+0000 7f403376d640 1 --2- 172.21.15.139:0/1758535848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f402c151850 0x7f402c159980 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.008+0000 7f403376d640 1 --2- 172.21.15.139:0/1758535848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f402c0c7390 0x7f402c1443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.008+0000 7f403376d640 1 -- 172.21.15.139:0/1758535848 >> 172.21.15.139:0/1758535848 conn(0x7f402c06cc30 msgr2=0x7f402c0c8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:00.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.009+0000 7f403376d640 1 -- 172.21.15.139:0/1758535848 shutdown_connections 2024-09-16T00:09:00.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.009+0000 7fb6f95ed640 1 -- 172.21.15.139:0/436806324 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb6e006c2b0 msgr2=0x7fb6e006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.009+0000 7fb6f95ed640 1 --2- 172.21.15.139:0/436806324 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb6e006c2b0 0x7fb6e006e770 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4000f60 tx=0x7fb6e400b040 comp rx=0 tx=0).stop 2024-09-16T00:09:00.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.009+0000 7fb6f95ed640 1 -- 172.21.15.139:0/436806324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6f40c7140 msgr2=0x7fb6f4150b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.009+0000 7fb6f95ed640 1 --2- 172.21.15.139:0/436806324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6f40c7140 0x7fb6f4150b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8031000 tx=0x7fb6e80046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.009+0000 7f403376d640 1 -- 172.21.15.139:0/1758535848 wait complete. 2024-09-16T00:09:00.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.009+0000 7fb6f95ed640 1 -- 172.21.15.139:0/436806324 shutdown_connections 2024-09-16T00:09:00.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.009+0000 7fb6f95ed640 1 --2- 172.21.15.139:0/436806324 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb6e006c2b0 0x7fb6e006e770 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.009+0000 7fb6f95ed640 1 --2- 172.21.15.139:0/436806324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6f4151a10 0x7fb6f4144c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.009+0000 7fb6f95ed640 1 --2- 172.21.15.139:0/436806324 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6f40c7140 0x7fb6f4150b00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.009+0000 7fb6f95ed640 1 --2- 172.21.15.139:0/436806324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6f40c3c40 0x7fb6f414cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.009+0000 7fb6f95ed640 1 -- 172.21.15.139:0/436806324 >> 172.21.15.139:0/436806324 conn(0x7fb6f40cc740 msgr2=0x7fb6f40c86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:00.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.009+0000 7fb6f95ed640 1 -- 172.21.15.139:0/436806324 shutdown_connections 2024-09-16T00:09:00.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.009+0000 7fb6f95ed640 1 -- 172.21.15.139:0/436806324 wait complete. 2024-09-16T00:09:00.016 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:00.016 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:00.022 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:00.022+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:00.023 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T00:09:00.024 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:09:00.026 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:09:00.123 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:00.123+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:00.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.167+0000 7f93fc12c640 1 Processor -- start 2024-09-16T00:09:00.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.167+0000 7f93fc12c640 1 -- start start 2024-09-16T00:09:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.168+0000 7f93fc12c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93f41528b0 0x7f93f40caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.168+0000 7f93fc12c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f93f40c8cc0 0x7f93f406fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.168+0000 7f93fc12c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93f40c32e0 con 0x7f93f4070120 2024-09-16T00:09:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.168+0000 7f93fc12c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f93f4152c90 con 0x7f93f40c8cc0 2024-09-16T00:09:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.168+0000 7f93fc12c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f93f40c9990 con 0x7f93f41528b0 2024-09-16T00:09:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.168+0000 7f93f96c7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f93f40c8cc0 0x7f93f406fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.168+0000 7f93f96c7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f93f40c8cc0 0x7f93f406fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41898/0 (socket says 172.21.15.139:41898) 2024-09-16T00:09:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.168+0000 7f93f96c7640 1 -- 172.21.15.139:0/1701952985 learned_addr learned my addr 172.21.15.139:0/1701952985 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.168+0000 7f93f9ec8640 1 --2- 172.21.15.139:0/1701952985 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93f41528b0 0x7f93f40caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.168+0000 7f93f96c7640 1 -- 172.21.15.139:0/1701952985 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93f41528b0 msgr2=0x7f93f40caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.168+0000 7f93f96c7640 1 --2- 172.21.15.139:0/1701952985 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93f41528b0 0x7f93f40caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.168+0000 7f93f96c7640 1 -- 172.21.15.139:0/1701952985 >> v1:172.21.15.164:6789/0 conn(0x7f93f4070120 legacy=0x7f93f41584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.169+0000 7f93f96c7640 1 -- 172.21.15.139:0/1701952985 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f40c93f0 con 0x7f93f40c8cc0 2024-09-16T00:09:00.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.169+0000 7f93f96c7640 1 --2- 172.21.15.139:0/1701952985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93f40c8cc0 0x7f93f406fbe0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009870 tx=0x7f93e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6197085c7ef16c96 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.169+0000 7f93f8ec6640 1 -- 172.21.15.139:0/1701952985 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4037070 con 0x7f93f40c8cc0 2024-09-16T00:09:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.169+0000 7f93f8ec6640 1 -- 172.21.15.139:0/1701952985 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e4032440 con 0x7f93f40c8cc0 2024-09-16T00:09:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.169+0000 7f93f8ec6640 1 -- 172.21.15.139:0/1701952985 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4032810 con 0x7f93f40c8cc0 2024-09-16T00:09:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.170+0000 7f93fc12c640 1 -- 172.21.15.139:0/1701952985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93f40c8cc0 msgr2=0x7f93f406fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.170+0000 7f93fc12c640 1 --2- 172.21.15.139:0/1701952985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93f40c8cc0 0x7f93f406fbe0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009870 tx=0x7f93e402a270 comp rx=0 tx=0).stop 2024-09-16T00:09:00.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.170+0000 7f93fc12c640 1 -- 172.21.15.139:0/1701952985 shutdown_connections 2024-09-16T00:09:00.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.170+0000 7f93fc12c640 1 --2- 172.21.15.139:0/1701952985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93f40c8cc0 0x7f93f406fbe0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.170+0000 7f93fc12c640 1 --2- 172.21.15.139:0/1701952985 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93f41528b0 0x7f93f40caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.170+0000 7f93fc12c640 1 -- 172.21.15.139:0/1701952985 >> 172.21.15.139:0/1701952985 conn(0x7f93f4067d70 msgr2=0x7f93f4068180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:00.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.170+0000 7f93fc12c640 1 -- 172.21.15.139:0/1701952985 shutdown_connections 2024-09-16T00:09:00.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.170+0000 7f93fc12c640 1 -- 172.21.15.139:0/1701952985 wait complete. 2024-09-16T00:09:00.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.171+0000 7f93fc12c640 1 Processor -- start 2024-09-16T00:09:00.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.171+0000 7f93fc12c640 1 -- start start 2024-09-16T00:09:00.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93fc12c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93f4070120 0x7f93f40b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93fc12c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93f40c8cc0 0x7f93f41519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93fc12c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f41528b0 0x7f93f41f0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93fc12c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f93f40c96b0 con 0x7f93f40c8cc0 2024-09-16T00:09:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93fc12c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f93f40caaa0 con 0x7f93f41528b0 2024-09-16T00:09:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93fc12c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f93f40ca730 con 0x7f93f4070120 2024-09-16T00:09:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93f9ec8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93f4070120 0x7f93f40b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93f96c7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93f40c8cc0 0x7f93f41519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93f9ec8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93f4070120 0x7f93f40b0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:56788/0 (socket says 172.21.15.139:56788) 2024-09-16T00:09:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93f9ec8640 1 -- 172.21.15.139:0/553285677 learned_addr learned my addr 172.21.15.139:0/553285677 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93f96c7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93f40c8cc0 0x7f93f41519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41912/0 (socket says 172.21.15.139:41912) 2024-09-16T00:09:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93fa6c9640 1 --2- 172.21.15.139:0/553285677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f41528b0 0x7f93f41f0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93f9ec8640 1 -- 172.21.15.139:0/553285677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f41528b0 msgr2=0x7f93f41f0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93f9ec8640 1 --2- 172.21.15.139:0/553285677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f41528b0 0x7f93f41f0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93f9ec8640 1 -- 172.21.15.139:0/553285677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93f40c8cc0 msgr2=0x7f93f41519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93f9ec8640 1 --2- 172.21.15.139:0/553285677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93f40c8cc0 0x7f93f41519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.172+0000 7f93f9ec8640 1 -- 172.21.15.139:0/553285677 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f41f4360 con 0x7f93f4070120 2024-09-16T00:09:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.173+0000 7f93f9ec8640 1 --2- 172.21.15.139:0/553285677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93f4070120 0x7f93f40b0de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f93f000e9e0 tx=0x7f93f000eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.173+0000 7f93ea7fc640 1 -- 172.21.15.139:0/553285677 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0017070 con 0x7f93f4070120 2024-09-16T00:09:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.173+0000 7f93ea7fc640 1 -- 172.21.15.139:0/553285677 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93f000c9d0 con 0x7f93f4070120 2024-09-16T00:09:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.173+0000 7f93fc12c640 1 -- 172.21.15.139:0/553285677 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f41f4640 con 0x7f93f4070120 2024-09-16T00:09:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.173+0000 7f93ea7fc640 1 -- 172.21.15.139:0/553285677 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f000cdc0 con 0x7f93f4070120 2024-09-16T00:09:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.173+0000 7f93fc12c640 1 -- 172.21.15.139:0/553285677 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f41f4c00 con 0x7f93f4070120 2024-09-16T00:09:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.174+0000 7f93fc12c640 1 -- 172.21.15.139:0/553285677 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93f414dd30 con 0x7f93f4070120 2024-09-16T00:09:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.175+0000 7f93ea7fc640 1 -- 172.21.15.139:0/553285677 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f93f001a070 con 0x7f93f4070120 2024-09-16T00:09:00.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.175+0000 7f93ea7fc640 1 --2- 172.21.15.139:0/553285677 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f93d806c2b0 0x7f93d806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.175+0000 7f93ea7fc640 1 -- 172.21.15.139:0/553285677 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(61..61 src has 1..61) ==== 5886+0+0 (secure 0 0 0) 0x7f93f008b880 con 0x7f93f4070120 2024-09-16T00:09:00.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.175+0000 7f93f96c7640 1 --2- 172.21.15.139:0/553285677 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f93d806c2b0 0x7f93d806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.178 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:00.178 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:00.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.175+0000 7f93f96c7640 1 --2- 172.21.15.139:0/553285677 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f93d806c2b0 0x7f93d806e770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93e4034000 tx=0x7f93e402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:00.185 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:09:00.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.184+0000 7f93ea7fc640 1 -- 172.21.15.139:0/553285677 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f93f414dd30 con 0x7f93f4070120 2024-09-16T00:09:00.224 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:00.223+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:00.293 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:00.293 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:00.302 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:09:00.346 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:00.346+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:00.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.377+0000 7f93fc12c640 1 -- 172.21.15.139:0/553285677 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93f40cb9b0 con 0x7f93d806c2b0 2024-09-16T00:09:00.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.384+0000 7f93ea7fc640 1 -- 172.21.15.139:0/553285677 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252397 (secure 0 0 0) 0x7f93f40cb9b0 con 0x7f93d806c2b0 2024-09-16T00:09:00.391 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:09:00.392 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:09:00.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.387+0000 7f93fc12c640 1 -- 172.21.15.139:0/553285677 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f93d806c2b0 msgr2=0x7f93d806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.387+0000 7f93fc12c640 1 --2- 172.21.15.139:0/553285677 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f93d806c2b0 0x7f93d806e770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93e4034000 tx=0x7f93e402e000 comp rx=0 tx=0).stop 2024-09-16T00:09:00.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.387+0000 7f93fc12c640 1 -- 172.21.15.139:0/553285677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93f4070120 msgr2=0x7f93f40b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.387+0000 7f93fc12c640 1 --2- 172.21.15.139:0/553285677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93f4070120 0x7f93f40b0de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f93f000e9e0 tx=0x7f93f000eea0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.387+0000 7f93fc12c640 1 -- 172.21.15.139:0/553285677 shutdown_connections 2024-09-16T00:09:00.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.387+0000 7f93fc12c640 1 --2- 172.21.15.139:0/553285677 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f93d806c2b0 0x7f93d806e770 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.387+0000 7f93fc12c640 1 --2- 172.21.15.139:0/553285677 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93f41528b0 0x7f93f41f0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.387+0000 7f93fc12c640 1 --2- 172.21.15.139:0/553285677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93f40c8cc0 0x7f93f41519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.387+0000 7f93fc12c640 1 --2- 172.21.15.139:0/553285677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f93f4070120 0x7f93f40b0de0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.387+0000 7f93fc12c640 1 -- 172.21.15.139:0/553285677 >> 172.21.15.139:0/553285677 conn(0x7f93f4067d70 msgr2=0x7f93f414f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:00.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.387+0000 7f93fc12c640 1 -- 172.21.15.139:0/553285677 shutdown_connections 2024-09-16T00:09:00.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.387+0000 7f93fc12c640 1 -- 172.21.15.139:0/553285677 wait complete. 2024-09-16T00:09:00.400 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-16T00:09:00.040495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8047325,"num_objects":194,"num_object_clones":0,"num_object_copies":388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":194,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":389,"num_write_kb":9818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":13110,"num_keys_recovered":4,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":338,"ondisk_log_size":338,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231536,"kb_used_data":15288,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754743184,"statfs":{"total":773094113280,"available":772857020416,"internally_reserved":0,"allocated":15654912,"data_stored":13801740,"data_compressed":263129,"data_compressed_allocated":339968,"data_compressed_original":2060288,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7449604,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":7379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000555"},"pg_stats":[{"pgid":"3.3f","version":"57'4","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984959+0000","last_change":"2024-09-16T00:08:36.888570+0000","last_active":"2024-09-16T00:08:56.984959+0000","last_peered":"2024-09-16T00:08:56.984959+0000","last_clean":"2024-09-16T00:08:56.984959+0000","last_became_active":"2024-09-16T00:08:36.888143+0000","last_became_peered":"2024-09-16T00:08:36.888143+0000","last_unstale":"2024-09-16T00:08:56.984959+0000","last_undegraded":"2024-09-16T00:08:56.984959+0000","last_fullsized":"2024-09-16T00:08:56.984959+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:30.579681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7173,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980158+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:08:55.980158+0000","last_peered":"2024-09-16T00:08:55.980158+0000","last_clean":"2024-09-16T00:08:55.980158+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:08:55.980158+0000","last_undegraded":"2024-09-16T00:08:55.980158+0000","last_fullsized":"2024-09-16T00:08:55.980158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973955+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:08:54.973955+0000","last_peered":"2024-09-16T00:08:54.973955+0000","last_clean":"2024-09-16T00:08:54.973955+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:08:54.973955+0000","last_undegraded":"2024-09-16T00:08:54.973955+0000","last_fullsized":"2024-09-16T00:08:54.973955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"57'1","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005040+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:08:57.005040+0000","last_peered":"2024-09-16T00:08:57.005040+0000","last_clean":"2024-09-16T00:08:57.005040+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:08:57.005040+0000","last_undegraded":"2024-09-16T00:08:57.005040+0000","last_fullsized":"2024-09-16T00:08:57.005040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":56,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980028+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:08:55.980028+0000","last_peered":"2024-09-16T00:08:55.980028+0000","last_clean":"2024-09-16T00:08:55.980028+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:08:55.980028+0000","last_undegraded":"2024-09-16T00:08:55.980028+0000","last_fullsized":"2024-09-16T00:08:55.980028+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"57'2","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004829+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:08:57.004829+0000","last_peered":"2024-09-16T00:08:57.004829+0000","last_clean":"2024-09-16T00:08:57.004829+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:08:57.004829+0000","last_undegraded":"2024-09-16T00:08:57.004829+0000","last_fullsized":"2024-09-16T00:08:57.004829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12750,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"57'2","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984334+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:08:56.984334+0000","last_peered":"2024-09-16T00:08:56.984334+0000","last_clean":"2024-09-16T00:08:56.984334+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:08:56.984334+0000","last_undegraded":"2024-09-16T00:08:56.984334+0000","last_fullsized":"2024-09-16T00:08:56.984334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8617,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":52,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.423823+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:08:54.423823+0000","last_peered":"2024-09-16T00:08:54.423823+0000","last_clean":"2024-09-16T00:08:54.423823+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:08:54.423823+0000","last_undegraded":"2024-09-16T00:08:54.423823+0000","last_fullsized":"2024-09-16T00:08:54.423823+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"58'2","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.985069+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:08:56.985069+0000","last_peered":"2024-09-16T00:08:56.985069+0000","last_clean":"2024-09-16T00:08:56.985069+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:08:56.985069+0000","last_undegraded":"2024-09-16T00:08:56.985069+0000","last_fullsized":"2024-09-16T00:08:56.985069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27085,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"57'7","reported_seq":38,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983777+0000","last_change":"2024-09-16T00:08:36.891868+0000","last_active":"2024-09-16T00:08:56.983777+0000","last_peered":"2024-09-16T00:08:56.983777+0000","last_clean":"2024-09-16T00:08:56.983777+0000","last_became_active":"2024-09-16T00:08:36.891605+0000","last_became_peered":"2024-09-16T00:08:36.891605+0000","last_unstale":"2024-09-16T00:08:56.983777+0000","last_undegraded":"2024-09-16T00:08:56.983777+0000","last_fullsized":"2024-09-16T00:08:56.983777+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:46:00.730436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44377,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"57'5","reported_seq":36,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983513+0000","last_change":"2024-09-16T00:08:36.891589+0000","last_active":"2024-09-16T00:08:56.983513+0000","last_peered":"2024-09-16T00:08:56.983513+0000","last_clean":"2024-09-16T00:08:56.983513+0000","last_became_active":"2024-09-16T00:08:36.891279+0000","last_became_peered":"2024-09-16T00:08:36.891279+0000","last_unstale":"2024-09-16T00:08:56.983513+0000","last_undegraded":"2024-09-16T00:08:56.983513+0000","last_fullsized":"2024-09-16T00:08:56.983513+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3123,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"58'4","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.008913+0000","last_change":"2024-09-16T00:08:36.888576+0000","last_active":"2024-09-16T00:08:57.008913+0000","last_peered":"2024-09-16T00:08:57.008913+0000","last_clean":"2024-09-16T00:08:57.008913+0000","last_became_active":"2024-09-16T00:08:36.888301+0000","last_became_peered":"2024-09-16T00:08:36.888301+0000","last_unstale":"2024-09-16T00:08:57.008913+0000","last_undegraded":"2024-09-16T00:08:57.008913+0000","last_fullsized":"2024-09-16T00:08:57.008913+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21479,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991795+0000","last_change":"2024-09-16T00:08:36.892595+0000","last_active":"2024-09-16T00:08:55.991795+0000","last_peered":"2024-09-16T00:08:55.991795+0000","last_clean":"2024-09-16T00:08:55.991795+0000","last_became_active":"2024-09-16T00:08:36.892316+0000","last_became_peered":"2024-09-16T00:08:36.892316+0000","last_unstale":"2024-09-16T00:08:55.991795+0000","last_undegraded":"2024-09-16T00:08:55.991795+0000","last_fullsized":"2024-09-16T00:08:55.991795+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:47.480168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"56'1","reported_seq":80,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973769+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:08:54.973769+0000","last_peered":"2024-09-16T00:08:54.973769+0000","last_clean":"2024-09-16T00:08:54.973769+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:08:54.973769+0000","last_undegraded":"2024-09-16T00:08:54.973769+0000","last_fullsized":"2024-09-16T00:08:54.973769+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979862+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:08:55.979862+0000","last_peered":"2024-09-16T00:08:55.979862+0000","last_clean":"2024-09-16T00:08:55.979862+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:08:55.979862+0000","last_undegraded":"2024-09-16T00:08:55.979862+0000","last_fullsized":"2024-09-16T00:08:55.979862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"57'4","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004785+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:08:57.004785+0000","last_peered":"2024-09-16T00:08:57.004785+0000","last_clean":"2024-09-16T00:08:57.004785+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:08:57.004785+0000","last_undegraded":"2024-09-16T00:08:57.004785+0000","last_fullsized":"2024-09-16T00:08:57.004785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8128,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"57'4","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005170+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:08:57.005170+0000","last_peered":"2024-09-16T00:08:57.005170+0000","last_clean":"2024-09-16T00:08:57.005170+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:08:57.005170+0000","last_undegraded":"2024-09-16T00:08:57.005170+0000","last_fullsized":"2024-09-16T00:08:57.005170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4240,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424154+0000","last_change":"2024-09-16T00:08:36.894430+0000","last_active":"2024-09-16T00:08:54.424154+0000","last_peered":"2024-09-16T00:08:54.424154+0000","last_clean":"2024-09-16T00:08:54.424154+0000","last_became_active":"2024-09-16T00:08:36.894138+0000","last_became_peered":"2024-09-16T00:08:36.894138+0000","last_unstale":"2024-09-16T00:08:54.424154+0000","last_undegraded":"2024-09-16T00:08:54.424154+0000","last_fullsized":"2024-09-16T00:08:54.424154+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:19.478353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"57'4","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.985021+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:08:56.985021+0000","last_peered":"2024-09-16T00:08:56.985021+0000","last_clean":"2024-09-16T00:08:56.985021+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:08:56.985021+0000","last_undegraded":"2024-09-16T00:08:56.985021+0000","last_fullsized":"2024-09-16T00:08:56.985021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1845,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"58'8","reported_seq":39,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983626+0000","last_change":"2024-09-16T00:08:36.890676+0000","last_active":"2024-09-16T00:08:56.983626+0000","last_peered":"2024-09-16T00:08:56.983626+0000","last_clean":"2024-09-16T00:08:56.983626+0000","last_became_active":"2024-09-16T00:08:36.889743+0000","last_became_peered":"2024-09-16T00:08:36.889743+0000","last_unstale":"2024-09-16T00:08:56.983626+0000","last_undegraded":"2024-09-16T00:08:56.983626+0000","last_fullsized":"2024-09-16T00:08:56.983626+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:02.709496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21485,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973839+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:08:54.973839+0000","last_peered":"2024-09-16T00:08:54.973839+0000","last_clean":"2024-09-16T00:08:54.973839+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:08:54.973839+0000","last_undegraded":"2024-09-16T00:08:54.973839+0000","last_fullsized":"2024-09-16T00:08:54.973839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"57'7","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004869+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:08:57.004869+0000","last_peered":"2024-09-16T00:08:57.004869+0000","last_clean":"2024-09-16T00:08:57.004869+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:08:57.004869+0000","last_undegraded":"2024-09-16T00:08:57.004869+0000","last_fullsized":"2024-09-16T00:08:57.004869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69449,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991605+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:08:55.991605+0000","last_peered":"2024-09-16T00:08:55.991605+0000","last_clean":"2024-09-16T00:08:55.991605+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:08:55.991605+0000","last_undegraded":"2024-09-16T00:08:55.991605+0000","last_fullsized":"2024-09-16T00:08:55.991605+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"57'7","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004863+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:08:57.004863+0000","last_peered":"2024-09-16T00:08:57.004863+0000","last_clean":"2024-09-16T00:08:57.004863+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:08:57.004863+0000","last_undegraded":"2024-09-16T00:08:57.004863+0000","last_fullsized":"2024-09-16T00:08:57.004863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14047,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.992055+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:08:55.992055+0000","last_peered":"2024-09-16T00:08:55.992055+0000","last_clean":"2024-09-16T00:08:55.992055+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:08:55.992055+0000","last_undegraded":"2024-09-16T00:08:55.992055+0000","last_fullsized":"2024-09-16T00:08:55.992055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"57'7","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983874+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:08:56.983874+0000","last_peered":"2024-09-16T00:08:56.983874+0000","last_clean":"2024-09-16T00:08:56.983874+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:08:56.983874+0000","last_undegraded":"2024-09-16T00:08:56.983874+0000","last_fullsized":"2024-09-16T00:08:56.983874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92931,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979860+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:08:55.979860+0000","last_peered":"2024-09-16T00:08:55.979860+0000","last_clean":"2024-09-16T00:08:55.979860+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:08:55.979860+0000","last_undegraded":"2024-09-16T00:08:55.979860+0000","last_fullsized":"2024-09-16T00:08:55.979860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991901+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:08:55.991901+0000","last_peered":"2024-09-16T00:08:55.991901+0000","last_clean":"2024-09-16T00:08:55.991901+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:08:55.991901+0000","last_undegraded":"2024-09-16T00:08:55.991901+0000","last_fullsized":"2024-09-16T00:08:55.991901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973891+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:08:54.973891+0000","last_peered":"2024-09-16T00:08:54.973891+0000","last_clean":"2024-09-16T00:08:54.973891+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:08:54.973891+0000","last_undegraded":"2024-09-16T00:08:54.973891+0000","last_fullsized":"2024-09-16T00:08:54.973891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":52,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424099+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:08:54.424099+0000","last_peered":"2024-09-16T00:08:54.424099+0000","last_clean":"2024-09-16T00:08:54.424099+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:08:54.424099+0000","last_undegraded":"2024-09-16T00:08:54.424099+0000","last_fullsized":"2024-09-16T00:08:54.424099+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":83,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424334+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:08:54.424334+0000","last_peered":"2024-09-16T00:08:54.424334+0000","last_clean":"2024-09-16T00:08:54.424334+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:08:54.424334+0000","last_undegraded":"2024-09-16T00:08:54.424334+0000","last_fullsized":"2024-09-16T00:08:54.424334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"57'6","reported_seq":37,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983162+0000","last_change":"2024-09-16T00:08:36.894101+0000","last_active":"2024-09-16T00:08:56.983162+0000","last_peered":"2024-09-16T00:08:56.983162+0000","last_clean":"2024-09-16T00:08:56.983162+0000","last_became_active":"2024-09-16T00:08:36.893807+0000","last_became_peered":"2024-09-16T00:08:36.893807+0000","last_unstale":"2024-09-16T00:08:56.983162+0000","last_undegraded":"2024-09-16T00:08:56.983162+0000","last_fullsized":"2024-09-16T00:08:56.983162+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:40.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"58'7","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.010727+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:08:57.010727+0000","last_peered":"2024-09-16T00:08:57.010727+0000","last_clean":"2024-09-16T00:08:57.010727+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:08:57.010727+0000","last_undegraded":"2024-09-16T00:08:57.010727+0000","last_fullsized":"2024-09-16T00:08:57.010727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16184,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":85,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991911+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:08:55.991911+0000","last_peered":"2024-09-16T00:08:55.991911+0000","last_clean":"2024-09-16T00:08:55.991911+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:08:55.991911+0000","last_undegraded":"2024-09-16T00:08:55.991911+0000","last_fullsized":"2024-09-16T00:08:55.991911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973910+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:08:54.973910+0000","last_peered":"2024-09-16T00:08:54.973910+0000","last_clean":"2024-09-16T00:08:54.973910+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:08:54.973910+0000","last_undegraded":"2024-09-16T00:08:54.973910+0000","last_fullsized":"2024-09-16T00:08:54.973910+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":31,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983375+0000","last_change":"2024-09-16T00:08:36.890316+0000","last_active":"2024-09-16T00:08:56.983375+0000","last_peered":"2024-09-16T00:08:56.983375+0000","last_clean":"2024-09-16T00:08:56.983375+0000","last_became_active":"2024-09-16T00:08:36.889635+0000","last_became_peered":"2024-09-16T00:08:36.889635+0000","last_unstale":"2024-09-16T00:08:56.983375+0000","last_undegraded":"2024-09-16T00:08:56.983375+0000","last_fullsized":"2024-09-16T00:08:56.983375+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:28.843883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":94,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991691+0000","last_change":"2024-09-16T00:08:36.888349+0000","last_active":"2024-09-16T00:08:55.991691+0000","last_peered":"2024-09-16T00:08:55.991691+0000","last_clean":"2024-09-16T00:08:55.991691+0000","last_became_active":"2024-09-16T00:08:36.888052+0000","last_became_peered":"2024-09-16T00:08:36.888052+0000","last_unstale":"2024-09-16T00:08:55.991691+0000","last_undegraded":"2024-09-16T00:08:55.991691+0000","last_fullsized":"2024-09-16T00:08:55.991691+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:49.607254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991502+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:08:55.991502+0000","last_peered":"2024-09-16T00:08:55.991502+0000","last_clean":"2024-09-16T00:08:55.991502+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:08:55.991502+0000","last_undegraded":"2024-09-16T00:08:55.991502+0000","last_fullsized":"2024-09-16T00:08:55.991502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004896+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:08:57.004896+0000","last_peered":"2024-09-16T00:08:57.004896+0000","last_clean":"2024-09-16T00:08:57.004896+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:08:57.004896+0000","last_undegraded":"2024-09-16T00:08:57.004896+0000","last_fullsized":"2024-09-16T00:08:57.004896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979829+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:08:55.979829+0000","last_peered":"2024-09-16T00:08:55.979829+0000","last_clean":"2024-09-16T00:08:55.979829+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:08:55.979829+0000","last_undegraded":"2024-09-16T00:08:55.979829+0000","last_fullsized":"2024-09-16T00:08:55.979829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983626+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:08:56.983626+0000","last_peered":"2024-09-16T00:08:56.983626+0000","last_clean":"2024-09-16T00:08:56.983626+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:08:56.983626+0000","last_undegraded":"2024-09-16T00:08:56.983626+0000","last_fullsized":"2024-09-16T00:08:56.983626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"57'5","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005138+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:08:57.005138+0000","last_peered":"2024-09-16T00:08:57.005138+0000","last_clean":"2024-09-16T00:08:57.005138+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:08:57.005138+0000","last_undegraded":"2024-09-16T00:08:57.005138+0000","last_fullsized":"2024-09-16T00:08:57.005138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":80946,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005068+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:08:57.005068+0000","last_peered":"2024-09-16T00:08:57.005068+0000","last_clean":"2024-09-16T00:08:57.005068+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:08:57.005068+0000","last_undegraded":"2024-09-16T00:08:57.005068+0000","last_fullsized":"2024-09-16T00:08:57.005068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.974071+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:08:54.974071+0000","last_peered":"2024-09-16T00:08:54.974071+0000","last_clean":"2024-09-16T00:08:54.974071+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:08:54.974071+0000","last_undegraded":"2024-09-16T00:08:54.974071+0000","last_fullsized":"2024-09-16T00:08:54.974071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":86,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980005+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:08:55.980005+0000","last_peered":"2024-09-16T00:08:55.980005+0000","last_clean":"2024-09-16T00:08:55.980005+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:08:55.980005+0000","last_undegraded":"2024-09-16T00:08:55.980005+0000","last_fullsized":"2024-09-16T00:08:55.980005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"57'8","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.008894+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:08:57.008894+0000","last_peered":"2024-09-16T00:08:57.008894+0000","last_clean":"2024-09-16T00:08:57.008894+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:08:57.008894+0000","last_undegraded":"2024-09-16T00:08:57.008894+0000","last_fullsized":"2024-09-16T00:08:57.008894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84046,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"57'15","reported_seq":46,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983597+0000","last_change":"2024-09-16T00:08:36.914308+0000","last_active":"2024-09-16T00:08:56.983597+0000","last_peered":"2024-09-16T00:08:56.983597+0000","last_clean":"2024-09-16T00:08:56.983597+0000","last_became_active":"2024-09-16T00:08:36.890227+0000","last_became_peered":"2024-09-16T00:08:36.890227+0000","last_unstale":"2024-09-16T00:08:56.983597+0000","last_undegraded":"2024-09-16T00:08:56.983597+0000","last_fullsized":"2024-09-16T00:08:56.983597+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:31.761195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21522,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13110,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"57'3","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984771+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:08:56.984771+0000","last_peered":"2024-09-16T00:08:56.984771+0000","last_clean":"2024-09-16T00:08:56.984771+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:08:56.984771+0000","last_undegraded":"2024-09-16T00:08:56.984771+0000","last_fullsized":"2024-09-16T00:08:56.984771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2379,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":83,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973988+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:08:54.973988+0000","last_peered":"2024-09-16T00:08:54.973988+0000","last_clean":"2024-09-16T00:08:54.973988+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:08:54.973988+0000","last_undegraded":"2024-09-16T00:08:54.973988+0000","last_fullsized":"2024-09-16T00:08:54.973988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973740+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:08:54.973740+0000","last_peered":"2024-09-16T00:08:54.973740+0000","last_clean":"2024-09-16T00:08:54.973740+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:08:54.973740+0000","last_undegraded":"2024-09-16T00:08:54.973740+0000","last_fullsized":"2024-09-16T00:08:54.973740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004924+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:08:57.004924+0000","last_peered":"2024-09-16T00:08:57.004924+0000","last_clean":"2024-09-16T00:08:57.004924+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:08:57.004924+0000","last_undegraded":"2024-09-16T00:08:57.004924+0000","last_fullsized":"2024-09-16T00:08:57.004924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"57'10","reported_seq":93,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984391+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:08:56.984391+0000","last_peered":"2024-09-16T00:08:56.984391+0000","last_clean":"2024-09-16T00:08:56.984391+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:08:56.984391+0000","last_undegraded":"2024-09-16T00:08:56.984391+0000","last_fullsized":"2024-09-16T00:08:56.984391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2388208,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":2338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":31,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983231+0000","last_change":"2024-09-16T00:08:36.889720+0000","last_active":"2024-09-16T00:08:56.983231+0000","last_peered":"2024-09-16T00:08:56.983231+0000","last_clean":"2024-09-16T00:08:56.983231+0000","last_became_active":"2024-09-16T00:08:36.889345+0000","last_became_peered":"2024-09-16T00:08:36.889345+0000","last_unstale":"2024-09-16T00:08:56.983231+0000","last_undegraded":"2024-09-16T00:08:56.983231+0000","last_fullsized":"2024-09-16T00:08:56.983231+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:03.170073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"58'7","reported_seq":38,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983217+0000","last_change":"2024-09-16T00:08:36.896787+0000","last_active":"2024-09-16T00:08:56.983217+0000","last_peered":"2024-09-16T00:08:56.983217+0000","last_clean":"2024-09-16T00:08:56.983217+0000","last_became_active":"2024-09-16T00:08:36.896505+0000","last_became_peered":"2024-09-16T00:08:36.896505+0000","last_unstale":"2024-09-16T00:08:56.983217+0000","last_undegraded":"2024-09-16T00:08:56.983217+0000","last_fullsized":"2024-09-16T00:08:56.983217+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:55.997551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4202646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424000+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:08:54.424000+0000","last_peered":"2024-09-16T00:08:54.424000+0000","last_clean":"2024-09-16T00:08:54.424000+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:08:54.424000+0000","last_undegraded":"2024-09-16T00:08:54.424000+0000","last_fullsized":"2024-09-16T00:08:54.424000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"57'5","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984434+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:08:56.984434+0000","last_peered":"2024-09-16T00:08:56.984434+0000","last_clean":"2024-09-16T00:08:56.984434+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:08:56.984434+0000","last_undegraded":"2024-09-16T00:08:56.984434+0000","last_fullsized":"2024-09-16T00:08:56.984434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.008735+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:08:57.008735+0000","last_peered":"2024-09-16T00:08:57.008735+0000","last_clean":"2024-09-16T00:08:57.008735+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:08:57.008735+0000","last_undegraded":"2024-09-16T00:08:57.008735+0000","last_fullsized":"2024-09-16T00:08:57.008735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"57'5","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984850+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:08:56.984850+0000","last_peered":"2024-09-16T00:08:56.984850+0000","last_clean":"2024-09-16T00:08:56.984850+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:08:56.984850+0000","last_undegraded":"2024-09-16T00:08:56.984850+0000","last_fullsized":"2024-09-16T00:08:56.984850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81483,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":82,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424201+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:08:54.424201+0000","last_peered":"2024-09-16T00:08:54.424201+0000","last_clean":"2024-09-16T00:08:54.424201+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:08:54.424201+0000","last_undegraded":"2024-09-16T00:08:54.424201+0000","last_fullsized":"2024-09-16T00:08:54.424201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"57'6","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.985051+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:08:56.985051+0000","last_peered":"2024-09-16T00:08:56.985051+0000","last_clean":"2024-09-16T00:08:56.985051+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:08:56.985051+0000","last_undegraded":"2024-09-16T00:08:56.985051+0000","last_fullsized":"2024-09-16T00:08:56.985051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22306,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":84,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980097+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:08:55.980097+0000","last_peered":"2024-09-16T00:08:55.980097+0000","last_clean":"2024-09-16T00:08:55.980097+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:08:55.980097+0000","last_undegraded":"2024-09-16T00:08:55.980097+0000","last_fullsized":"2024-09-16T00:08:55.980097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"57'8","reported_seq":92,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005154+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:08:57.005154+0000","last_peered":"2024-09-16T00:08:57.005154+0000","last_clean":"2024-09-16T00:08:57.005154+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:08:57.005154+0000","last_undegraded":"2024-09-16T00:08:57.005154+0000","last_fullsized":"2024-09-16T00:08:57.005154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32783,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":80,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424063+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:08:54.424063+0000","last_peered":"2024-09-16T00:08:54.424063+0000","last_clean":"2024-09-16T00:08:54.424063+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:08:54.424063+0000","last_undegraded":"2024-09-16T00:08:54.424063+0000","last_fullsized":"2024-09-16T00:08:54.424063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"58'7","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.029232+0000","last_change":"2024-09-16T00:08:47.953122+0000","last_active":"2024-09-16T00:08:57.029232+0000","last_peered":"2024-09-16T00:08:57.029232+0000","last_clean":"2024-09-16T00:08:57.029232+0000","last_became_active":"2024-09-16T00:08:47.952799+0000","last_became_peered":"2024-09-16T00:08:47.952799+0000","last_unstale":"2024-09-16T00:08:57.029232+0000","last_undegraded":"2024-09-16T00:08:57.029232+0000","last_fullsized":"2024-09-16T00:08:57.029232+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:03.447562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61360,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":241,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984180+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:08:56.984180+0000","last_peered":"2024-09-16T00:08:56.984180+0000","last_clean":"2024-09-16T00:08:56.984180+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:08:56.984180+0000","last_undegraded":"2024-09-16T00:08:56.984180+0000","last_fullsized":"2024-09-16T00:08:56.984180+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004860+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:08:57.004860+0000","last_peered":"2024-09-16T00:08:57.004860+0000","last_clean":"2024-09-16T00:08:57.004860+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:08:57.004860+0000","last_undegraded":"2024-09-16T00:08:57.004860+0000","last_fullsized":"2024-09-16T00:08:57.004860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991784+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:08:55.991784+0000","last_peered":"2024-09-16T00:08:55.991784+0000","last_clean":"2024-09-16T00:08:55.991784+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:08:55.991784+0000","last_undegraded":"2024-09-16T00:08:55.991784+0000","last_fullsized":"2024-09-16T00:08:55.991784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984166+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:08:56.984166+0000","last_peered":"2024-09-16T00:08:56.984166+0000","last_clean":"2024-09-16T00:08:56.984166+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:08:56.984166+0000","last_undegraded":"2024-09-16T00:08:56.984166+0000","last_fullsized":"2024-09-16T00:08:56.984166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973713+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:08:54.973713+0000","last_peered":"2024-09-16T00:08:54.973713+0000","last_clean":"2024-09-16T00:08:54.973713+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:08:54.973713+0000","last_undegraded":"2024-09-16T00:08:54.973713+0000","last_fullsized":"2024-09-16T00:08:54.973713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":80,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424424+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:08:54.424424+0000","last_peered":"2024-09-16T00:08:54.424424+0000","last_clean":"2024-09-16T00:08:54.424424+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:08:54.424424+0000","last_undegraded":"2024-09-16T00:08:54.424424+0000","last_fullsized":"2024-09-16T00:08:54.424424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979829+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:08:55.979829+0000","last_peered":"2024-09-16T00:08:55.979829+0000","last_clean":"2024-09-16T00:08:55.979829+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:08:55.979829+0000","last_undegraded":"2024-09-16T00:08:55.979829+0000","last_fullsized":"2024-09-16T00:08:55.979829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004966+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:08:57.004966+0000","last_peered":"2024-09-16T00:08:57.004966+0000","last_clean":"2024-09-16T00:08:57.004966+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:08:57.004966+0000","last_undegraded":"2024-09-16T00:08:57.004966+0000","last_fullsized":"2024-09-16T00:08:57.004966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"58'3","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.015492+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:08:57.015492+0000","last_peered":"2024-09-16T00:08:57.015492+0000","last_clean":"2024-09-16T00:08:57.015492+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:08:57.015492+0000","last_undegraded":"2024-09-16T00:08:57.015492+0000","last_fullsized":"2024-09-16T00:08:57.015492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1838,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":83,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979978+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:08:55.979978+0000","last_peered":"2024-09-16T00:08:55.979978+0000","last_clean":"2024-09-16T00:08:55.979978+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:08:55.979978+0000","last_undegraded":"2024-09-16T00:08:55.979978+0000","last_fullsized":"2024-09-16T00:08:55.979978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"57'5","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005147+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:08:57.005147+0000","last_peered":"2024-09-16T00:08:57.005147+0000","last_clean":"2024-09-16T00:08:57.005147+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:08:57.005147+0000","last_undegraded":"2024-09-16T00:08:57.005147+0000","last_fullsized":"2024-09-16T00:08:57.005147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13443,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.985261+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:08:56.985261+0000","last_peered":"2024-09-16T00:08:56.985261+0000","last_clean":"2024-09-16T00:08:56.985261+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:08:56.985261+0000","last_undegraded":"2024-09-16T00:08:56.985261+0000","last_fullsized":"2024-09-16T00:08:56.985261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979860+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:08:55.979860+0000","last_peered":"2024-09-16T00:08:55.979860+0000","last_clean":"2024-09-16T00:08:55.979860+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:08:55.979860+0000","last_undegraded":"2024-09-16T00:08:55.979860+0000","last_fullsized":"2024-09-16T00:08:55.979860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"56'4","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980204+0000","last_change":"2024-09-16T00:08:36.906253+0000","last_active":"2024-09-16T00:08:55.980204+0000","last_peered":"2024-09-16T00:08:55.980204+0000","last_clean":"2024-09-16T00:08:55.980204+0000","last_became_active":"2024-09-16T00:08:36.894363+0000","last_became_peered":"2024-09-16T00:08:36.894363+0000","last_unstale":"2024-09-16T00:08:55.980204+0000","last_undegraded":"2024-09-16T00:08:55.980204+0000","last_fullsized":"2024-09-16T00:08:55.980204+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:00.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.992092+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:08:55.992092+0000","last_peered":"2024-09-16T00:08:55.992092+0000","last_clean":"2024-09-16T00:08:55.992092+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:08:55.992092+0000","last_undegraded":"2024-09-16T00:08:55.992092+0000","last_fullsized":"2024-09-16T00:08:55.992092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983637+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:08:56.983637+0000","last_peered":"2024-09-16T00:08:56.983637+0000","last_clean":"2024-09-16T00:08:56.983637+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:08:56.983637+0000","last_undegraded":"2024-09-16T00:08:56.983637+0000","last_fullsized":"2024-09-16T00:08:56.983637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.974120+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:08:54.974120+0000","last_peered":"2024-09-16T00:08:54.974120+0000","last_clean":"2024-09-16T00:08:54.974120+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:08:54.974120+0000","last_undegraded":"2024-09-16T00:08:54.974120+0000","last_fullsized":"2024-09-16T00:08:54.974120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984026+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:08:56.984026+0000","last_peered":"2024-09-16T00:08:56.984026+0000","last_clean":"2024-09-16T00:08:56.984026+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:08:56.984026+0000","last_undegraded":"2024-09-16T00:08:56.984026+0000","last_fullsized":"2024-09-16T00:08:56.984026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"57'4","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.008594+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:08:57.008594+0000","last_peered":"2024-09-16T00:08:57.008594+0000","last_clean":"2024-09-16T00:08:57.008594+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:08:57.008594+0000","last_undegraded":"2024-09-16T00:08:57.008594+0000","last_fullsized":"2024-09-16T00:08:57.008594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22937,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":83,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979932+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:08:55.979932+0000","last_peered":"2024-09-16T00:08:55.979932+0000","last_clean":"2024-09-16T00:08:55.979932+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:08:55.979932+0000","last_undegraded":"2024-09-16T00:08:55.979932+0000","last_fullsized":"2024-09-16T00:08:55.979932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991568+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:08:55.991568+0000","last_peered":"2024-09-16T00:08:55.991568+0000","last_clean":"2024-09-16T00:08:55.991568+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:08:55.991568+0000","last_undegraded":"2024-09-16T00:08:55.991568+0000","last_fullsized":"2024-09-16T00:08:55.991568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":83,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980089+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:08:55.980089+0000","last_peered":"2024-09-16T00:08:55.980089+0000","last_clean":"2024-09-16T00:08:55.980089+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:08:55.980089+0000","last_undegraded":"2024-09-16T00:08:55.980089+0000","last_fullsized":"2024-09-16T00:08:55.980089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.992004+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:08:55.992004+0000","last_peered":"2024-09-16T00:08:55.992004+0000","last_clean":"2024-09-16T00:08:55.992004+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:08:55.992004+0000","last_undegraded":"2024-09-16T00:08:55.992004+0000","last_fullsized":"2024-09-16T00:08:55.992004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984458+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:08:56.984458+0000","last_peered":"2024-09-16T00:08:56.984458+0000","last_clean":"2024-09-16T00:08:56.984458+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:08:56.984458+0000","last_undegraded":"2024-09-16T00:08:56.984458+0000","last_fullsized":"2024-09-16T00:08:56.984458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"58'6","reported_seq":37,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983319+0000","last_change":"2024-09-16T00:08:36.890347+0000","last_active":"2024-09-16T00:08:56.983319+0000","last_peered":"2024-09-16T00:08:56.983319+0000","last_clean":"2024-09-16T00:08:56.983319+0000","last_became_active":"2024-09-16T00:08:36.890009+0000","last_became_peered":"2024-09-16T00:08:36.890009+0000","last_unstale":"2024-09-16T00:08:56.983319+0000","last_undegraded":"2024-09-16T00:08:56.983319+0000","last_fullsized":"2024-09-16T00:08:56.983319+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:24.298978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19739,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"57'4","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005223+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:08:57.005223+0000","last_peered":"2024-09-16T00:08:57.005223+0000","last_clean":"2024-09-16T00:08:57.005223+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:08:57.005223+0000","last_undegraded":"2024-09-16T00:08:57.005223+0000","last_fullsized":"2024-09-16T00:08:57.005223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2557,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984758+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:08:56.984758+0000","last_peered":"2024-09-16T00:08:56.984758+0000","last_clean":"2024-09-16T00:08:56.984758+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:08:56.984758+0000","last_undegraded":"2024-09-16T00:08:56.984758+0000","last_fullsized":"2024-09-16T00:08:56.984758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'2","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005087+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:08:57.005087+0000","last_peered":"2024-09-16T00:08:57.005087+0000","last_clean":"2024-09-16T00:08:57.005087+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:08:57.005087+0000","last_undegraded":"2024-09-16T00:08:57.005087+0000","last_fullsized":"2024-09-16T00:08:57.005087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":105,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984904+0000","last_change":"2024-09-16T00:08:36.900704+0000","last_active":"2024-09-16T00:08:56.984904+0000","last_peered":"2024-09-16T00:08:56.984904+0000","last_clean":"2024-09-16T00:08:56.984904+0000","last_became_active":"2024-09-16T00:08:36.892520+0000","last_became_peered":"2024-09-16T00:08:36.892520+0000","last_unstale":"2024-09-16T00:08:56.984904+0000","last_undegraded":"2024-09-16T00:08:56.984904+0000","last_fullsized":"2024-09-16T00:08:56.984904+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:30.056041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":52,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424040+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:08:54.424040+0000","last_peered":"2024-09-16T00:08:54.424040+0000","last_clean":"2024-09-16T00:08:54.424040+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:08:54.424040+0000","last_undegraded":"2024-09-16T00:08:54.424040+0000","last_fullsized":"2024-09-16T00:08:54.424040+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":85,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.992032+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:08:55.992032+0000","last_peered":"2024-09-16T00:08:55.992032+0000","last_clean":"2024-09-16T00:08:55.992032+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:08:55.992032+0000","last_undegraded":"2024-09-16T00:08:55.992032+0000","last_fullsized":"2024-09-16T00:08:55.992032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991703+0000","last_change":"2024-09-16T00:08:36.889082+0000","last_active":"2024-09-16T00:08:55.991703+0000","last_peered":"2024-09-16T00:08:55.991703+0000","last_clean":"2024-09-16T00:08:55.991703+0000","last_became_active":"2024-09-16T00:08:36.888807+0000","last_became_peered":"2024-09-16T00:08:36.888807+0000","last_unstale":"2024-09-16T00:08:55.991703+0000","last_undegraded":"2024-09-16T00:08:55.991703+0000","last_fullsized":"2024-09-16T00:08:55.991703+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:22.825324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":85,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.974121+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:08:54.974121+0000","last_peered":"2024-09-16T00:08:54.974121+0000","last_clean":"2024-09-16T00:08:54.974121+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:08:54.974121+0000","last_undegraded":"2024-09-16T00:08:54.974121+0000","last_fullsized":"2024-09-16T00:08:54.974121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":7434637,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10895360,"data_stored":10700843,"data_compressed":270015,"data_compressed_allocated":344064,"data_compressed_original":1134592,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":22320,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":13110,"num_keys_recovered":4,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":40694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30268,"kb_used_data":3240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341572,"statfs":{"total":96636764160,"available":96605769728,"internally_reserved":0,"allocated":3317760,"data_stored":3592203,"data_compressed":73051,"data_compressed_allocated":94208,"data_compressed_original":831488,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":5144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":5267456,"data_stored":4867710,"data_compressed":57794,"data_compressed_allocated":73728,"data_compressed_original":237568,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343940,"statfs":{"total":96636764160,"available":96608194560,"internally_reserved":0,"allocated":897024,"data_stored":548421,"data_compressed":30333,"data_compressed_allocated":40960,"data_compressed_original":180224,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30468,"kb_used_data":3444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341372,"statfs":{"total":96636764160,"available":96605564928,"internally_reserved":0,"allocated":3526656,"data_stored":3061123,"data_compressed":69574,"data_compressed_allocated":86016,"data_compressed_original":237568,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":655360,"data_stored":291719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":647168,"data_stored":285324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":716800,"data_stored":876170,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":626688,"data_stored":279070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":17576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":21522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2936832,"data_stored":2786732,"data_compressed":69574,"data_compressed_allocated":86016,"data_compressed_original":237568,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":398309,"data_compressed":60175,"data_compressed_allocated":77824,"data_compressed_original":237568,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4579328,"data_stored":4549565,"data_compressed":57794,"data_compressed_allocated":73728,"data_compressed_original":237568,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2908160,"data_stored":2966237,"data_compressed":82472,"data_compressed_allocated":106496,"data_compressed_original":421888,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:09:00.405 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:09:00.414 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:00.414 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:00.425 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:09:00.466 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:00.466+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:00.528 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:00.528+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:00.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.554+0000 7fab39053640 1 Processor -- start 2024-09-16T00:09:00.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.555+0000 7fab39053640 1 -- start start 2024-09-16T00:09:00.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.555+0000 7fab39053640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fab341516a0 0x7fab340c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.555+0000 7fab39053640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab340c47b0 0x7fab3406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.555+0000 7fab39053640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fab3406f040 0x7fab3415a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.555+0000 7fab39053640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fab34158060 con 0x7fab341516a0 2024-09-16T00:09:00.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.555+0000 7fab39053640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fab340c76c0 con 0x7fab3406f040 2024-09-16T00:09:00.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.555+0000 7fab39053640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fab34157730 con 0x7fab340c47b0 2024-09-16T00:09:00.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.556+0000 7fab32d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fab341516a0 0x7fab340c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.556+0000 7fab32d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fab341516a0 0x7fab340c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41922/0 (socket says 172.21.15.139:41922) 2024-09-16T00:09:00.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.556+0000 7fab32d9d640 1 -- 172.21.15.139:0/333181239 learned_addr learned my addr 172.21.15.139:0/333181239 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:00.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.556+0000 7fab2bfff640 1 --2- 172.21.15.139:0/333181239 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab340c47b0 0x7fab3406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.556+0000 7fab3359e640 1 --2- 172.21.15.139:0/333181239 >> v2:172.21.15.164:3300/0 conn(0x7fab3406f040 0x7fab3415a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.556+0000 7fab32d9d640 1 -- 172.21.15.139:0/333181239 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab340c47b0 msgr2=0x7fab3406eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:00.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.556+0000 7fab32d9d640 1 --2- 172.21.15.139:0/333181239 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab340c47b0 0x7fab3406eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.556+0000 7fab32d9d640 1 -- 172.21.15.139:0/333181239 >> v2:172.21.15.164:3300/0 conn(0x7fab3406f040 msgr2=0x7fab3415a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:00.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.556+0000 7fab32d9d640 1 --2- 172.21.15.139:0/333181239 >> v2:172.21.15.164:3300/0 conn(0x7fab3406f040 0x7fab3415a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.556+0000 7fab32d9d640 1 -- 172.21.15.139:0/333181239 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab34157a10 con 0x7fab341516a0 2024-09-16T00:09:00.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.556+0000 7fab2bfff640 1 --2- 172.21.15.139:0/333181239 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab340c47b0 0x7fab3406eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:00.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.556+0000 7fab3359e640 1 --2- 172.21.15.139:0/333181239 >> v2:172.21.15.164:3300/0 conn(0x7fab3406f040 0x7fab3415a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:00.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.557+0000 7fab32d9d640 1 --2- 172.21.15.139:0/333181239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab341516a0 0x7fab340c7080 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fab3415ba40 tx=0x7fab2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57e2cb4875c1108 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:00.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.557+0000 7fab3259c640 1 -- 172.21.15.139:0/333181239 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24037070 con 0x7fab341516a0 2024-09-16T00:09:00.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.557+0000 7fab3259c640 1 -- 172.21.15.139:0/333181239 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab24032440 con 0x7fab341516a0 2024-09-16T00:09:00.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.557+0000 7fab3259c640 1 -- 172.21.15.139:0/333181239 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24032810 con 0x7fab341516a0 2024-09-16T00:09:00.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.558+0000 7fab39053640 1 -- 172.21.15.139:0/333181239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab341516a0 msgr2=0x7fab340c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.558+0000 7fab39053640 1 --2- 172.21.15.139:0/333181239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab341516a0 0x7fab340c7080 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fab3415ba40 tx=0x7fab2402a270 comp rx=0 tx=0).stop 2024-09-16T00:09:00.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.558+0000 7fab39053640 1 -- 172.21.15.139:0/333181239 shutdown_connections 2024-09-16T00:09:00.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.558+0000 7fab39053640 1 --2- 172.21.15.139:0/333181239 >> v2:172.21.15.164:3300/0 conn(0x7fab3406f040 0x7fab3415a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.558+0000 7fab39053640 1 --2- 172.21.15.139:0/333181239 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab340c47b0 0x7fab3406eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.558+0000 7fab39053640 1 --2- 172.21.15.139:0/333181239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab341516a0 0x7fab340c7080 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.558+0000 7fab39053640 1 -- 172.21.15.139:0/333181239 >> 172.21.15.139:0/333181239 conn(0x7fab34067950 msgr2=0x7fab34067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:00.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.558+0000 7fab39053640 1 -- 172.21.15.139:0/333181239 shutdown_connections 2024-09-16T00:09:00.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.558+0000 7fab39053640 1 -- 172.21.15.139:0/333181239 wait complete. 2024-09-16T00:09:00.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.559+0000 7fab39053640 1 Processor -- start 2024-09-16T00:09:00.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.559+0000 7fab39053640 1 -- start start 2024-09-16T00:09:00.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab39053640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab3406f040 0x7fab341e1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab39053640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab340c47b0 0x7fab341569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab39053640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab341516a0 0x7fab340cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab39053640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fab341573e0 con 0x7fab341516a0 2024-09-16T00:09:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab39053640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab340c6be0 con 0x7fab340c47b0 2024-09-16T00:09:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab39053640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fab34151a80 con 0x7fab3406f040 2024-09-16T00:09:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab3359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab341516a0 0x7fab340cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab32d9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab3406f040 0x7fab341e1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab340c47b0 0x7fab341569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab3359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab341516a0 0x7fab340cc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41934/0 (socket says 172.21.15.139:41934) 2024-09-16T00:09:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab3359e640 1 -- 172.21.15.139:0/1954861748 learned_addr learned my addr 172.21.15.139:0/1954861748 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab32d9d640 1 -- 172.21.15.139:0/1954861748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab340c47b0 msgr2=0x7fab341569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab32d9d640 1 --2- 172.21.15.139:0/1954861748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab340c47b0 0x7fab341569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab32d9d640 1 -- 172.21.15.139:0/1954861748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab341516a0 msgr2=0x7fab340cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab32d9d640 1 --2- 172.21.15.139:0/1954861748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab341516a0 0x7fab340cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab32d9d640 1 -- 172.21.15.139:0/1954861748 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab34150060 con 0x7fab3406f040 2024-09-16T00:09:00.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.560+0000 7fab2bfff640 1 --2- 172.21.15.139:0/1954861748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab340c47b0 0x7fab341569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:00.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.561+0000 7fab32d9d640 1 --2- 172.21.15.139:0/1954861748 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab3406f040 0x7fab341e1ea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab24031000 tx=0x7fab240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:00.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.561+0000 7fab2b7fe640 1 -- 172.21.15.139:0/1954861748 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24041020 con 0x7fab3406f040 2024-09-16T00:09:00.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.561+0000 7fab2b7fe640 1 -- 172.21.15.139:0/1954861748 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab2403d660 con 0x7fab3406f040 2024-09-16T00:09:00.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.561+0000 7fab2b7fe640 1 -- 172.21.15.139:0/1954861748 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24037040 con 0x7fab3406f040 2024-09-16T00:09:00.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.561+0000 7fab39053640 1 -- 172.21.15.139:0/1954861748 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab34150790 con 0x7fab3406f040 2024-09-16T00:09:00.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.561+0000 7fab39053640 1 -- 172.21.15.139:0/1954861748 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fab3415b7c0 con 0x7fab3406f040 2024-09-16T00:09:00.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.562+0000 7fab39053640 1 -- 172.21.15.139:0/1954861748 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab34150cb0 con 0x7fab3406f040 2024-09-16T00:09:00.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.563+0000 7fab2b7fe640 1 -- 172.21.15.139:0/1954861748 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fab24044020 con 0x7fab3406f040 2024-09-16T00:09:00.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.563+0000 7fab2b7fe640 1 --2- 172.21.15.139:0/1954861748 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fab1406c2e0 0x7fab1406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.563+0000 7fab2b7fe640 1 -- 172.21.15.139:0/1954861748 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(61..61 src has 1..61) ==== 5886+0+0 (secure 0 0 0) 0x7fab240aba00 con 0x7fab3406f040 2024-09-16T00:09:00.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.563+0000 7fab2bfff640 1 --2- 172.21.15.139:0/1954861748 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fab1406c2e0 0x7fab1406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.564+0000 7fab2bfff640 1 --2- 172.21.15.139:0/1954861748 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fab1406c2e0 0x7fab1406e7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fab341e2b80 tx=0x7fab1c009110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:00.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.568+0000 7fab2b7fe640 1 -- 172.21.15.139:0/1954861748 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fab34150cb0 con 0x7fab3406f040 2024-09-16T00:09:00.580 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:00.580 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:00.590 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:09:00.629 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:00.629+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:00.730 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:00.729+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:00.742 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:00.743 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:00.753 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:09:00.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.775+0000 7fab39053640 1 -- 172.21.15.139:0/1954861748 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fab3414fe10 con 0x7fab1406c2e0 2024-09-16T00:09:00.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.783+0000 7fab2b7fe640 1 -- 172.21.15.139:0/1954861748 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252397 (secure 0 0 0) 0x7fab3414fe10 con 0x7fab1406c2e0 2024-09-16T00:09:00.786 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:09:00.787 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:09:00.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.786+0000 7fab39053640 1 -- 172.21.15.139:0/1954861748 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fab1406c2e0 msgr2=0x7fab1406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.786+0000 7fab39053640 1 --2- 172.21.15.139:0/1954861748 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fab1406c2e0 0x7fab1406e7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fab341e2b80 tx=0x7fab1c009110 comp rx=0 tx=0).stop 2024-09-16T00:09:00.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.786+0000 7fab39053640 1 -- 172.21.15.139:0/1954861748 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab3406f040 msgr2=0x7fab341e1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.786+0000 7fab39053640 1 --2- 172.21.15.139:0/1954861748 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab3406f040 0x7fab341e1ea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab24031000 tx=0x7fab240046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.786+0000 7fab39053640 1 -- 172.21.15.139:0/1954861748 shutdown_connections 2024-09-16T00:09:00.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.786+0000 7fab39053640 1 --2- 172.21.15.139:0/1954861748 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fab1406c2e0 0x7fab1406e7a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.786+0000 7fab39053640 1 --2- 172.21.15.139:0/1954861748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab341516a0 0x7fab340cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.786+0000 7fab39053640 1 --2- 172.21.15.139:0/1954861748 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab340c47b0 0x7fab341569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.786+0000 7fab39053640 1 --2- 172.21.15.139:0/1954861748 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab3406f040 0x7fab341e1ea0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.786+0000 7fab39053640 1 -- 172.21.15.139:0/1954861748 >> 172.21.15.139:0/1954861748 conn(0x7fab34067950 msgr2=0x7fab34070400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:00.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.786+0000 7fab39053640 1 -- 172.21.15.139:0/1954861748 shutdown_connections 2024-09-16T00:09:00.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.786+0000 7fab39053640 1 -- 172.21.15.139:0/1954861748 wait complete. 2024-09-16T00:09:00.801 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-16T00:09:00.040495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8047325,"num_objects":194,"num_object_clones":0,"num_object_copies":388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":194,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":389,"num_write_kb":9818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":13110,"num_keys_recovered":4,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":338,"ondisk_log_size":338,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231536,"kb_used_data":15288,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754743184,"statfs":{"total":773094113280,"available":772857020416,"internally_reserved":0,"allocated":15654912,"data_stored":13801740,"data_compressed":263129,"data_compressed_allocated":339968,"data_compressed_original":2060288,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7449604,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":7379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000555"},"pg_stats":[{"pgid":"3.3f","version":"57'4","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984959+0000","last_change":"2024-09-16T00:08:36.888570+0000","last_active":"2024-09-16T00:08:56.984959+0000","last_peered":"2024-09-16T00:08:56.984959+0000","last_clean":"2024-09-16T00:08:56.984959+0000","last_became_active":"2024-09-16T00:08:36.888143+0000","last_became_peered":"2024-09-16T00:08:36.888143+0000","last_unstale":"2024-09-16T00:08:56.984959+0000","last_undegraded":"2024-09-16T00:08:56.984959+0000","last_fullsized":"2024-09-16T00:08:56.984959+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:30.579681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7173,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980158+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:08:55.980158+0000","last_peered":"2024-09-16T00:08:55.980158+0000","last_clean":"2024-09-16T00:08:55.980158+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:08:55.980158+0000","last_undegraded":"2024-09-16T00:08:55.980158+0000","last_fullsized":"2024-09-16T00:08:55.980158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973955+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:08:54.973955+0000","last_peered":"2024-09-16T00:08:54.973955+0000","last_clean":"2024-09-16T00:08:54.973955+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:08:54.973955+0000","last_undegraded":"2024-09-16T00:08:54.973955+0000","last_fullsized":"2024-09-16T00:08:54.973955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"57'1","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005040+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:08:57.005040+0000","last_peered":"2024-09-16T00:08:57.005040+0000","last_clean":"2024-09-16T00:08:57.005040+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:08:57.005040+0000","last_undegraded":"2024-09-16T00:08:57.005040+0000","last_fullsized":"2024-09-16T00:08:57.005040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":56,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980028+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:08:55.980028+0000","last_peered":"2024-09-16T00:08:55.980028+0000","last_clean":"2024-09-16T00:08:55.980028+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:08:55.980028+0000","last_undegraded":"2024-09-16T00:08:55.980028+0000","last_fullsized":"2024-09-16T00:08:55.980028+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"57'2","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004829+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:08:57.004829+0000","last_peered":"2024-09-16T00:08:57.004829+0000","last_clean":"2024-09-16T00:08:57.004829+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:08:57.004829+0000","last_undegraded":"2024-09-16T00:08:57.004829+0000","last_fullsized":"2024-09-16T00:08:57.004829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12750,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"57'2","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984334+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:08:56.984334+0000","last_peered":"2024-09-16T00:08:56.984334+0000","last_clean":"2024-09-16T00:08:56.984334+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:08:56.984334+0000","last_undegraded":"2024-09-16T00:08:56.984334+0000","last_fullsized":"2024-09-16T00:08:56.984334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8617,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":52,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.423823+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:08:54.423823+0000","last_peered":"2024-09-16T00:08:54.423823+0000","last_clean":"2024-09-16T00:08:54.423823+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:08:54.423823+0000","last_undegraded":"2024-09-16T00:08:54.423823+0000","last_fullsized":"2024-09-16T00:08:54.423823+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"58'2","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.985069+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:08:56.985069+0000","last_peered":"2024-09-16T00:08:56.985069+0000","last_clean":"2024-09-16T00:08:56.985069+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:08:56.985069+0000","last_undegraded":"2024-09-16T00:08:56.985069+0000","last_fullsized":"2024-09-16T00:08:56.985069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27085,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"57'7","reported_seq":38,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983777+0000","last_change":"2024-09-16T00:08:36.891868+0000","last_active":"2024-09-16T00:08:56.983777+0000","last_peered":"2024-09-16T00:08:56.983777+0000","last_clean":"2024-09-16T00:08:56.983777+0000","last_became_active":"2024-09-16T00:08:36.891605+0000","last_became_peered":"2024-09-16T00:08:36.891605+0000","last_unstale":"2024-09-16T00:08:56.983777+0000","last_undegraded":"2024-09-16T00:08:56.983777+0000","last_fullsized":"2024-09-16T00:08:56.983777+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:46:00.730436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44377,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"57'5","reported_seq":36,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983513+0000","last_change":"2024-09-16T00:08:36.891589+0000","last_active":"2024-09-16T00:08:56.983513+0000","last_peered":"2024-09-16T00:08:56.983513+0000","last_clean":"2024-09-16T00:08:56.983513+0000","last_became_active":"2024-09-16T00:08:36.891279+0000","last_became_peered":"2024-09-16T00:08:36.891279+0000","last_unstale":"2024-09-16T00:08:56.983513+0000","last_undegraded":"2024-09-16T00:08:56.983513+0000","last_fullsized":"2024-09-16T00:08:56.983513+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3123,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"58'4","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.008913+0000","last_change":"2024-09-16T00:08:36.888576+0000","last_active":"2024-09-16T00:08:57.008913+0000","last_peered":"2024-09-16T00:08:57.008913+0000","last_clean":"2024-09-16T00:08:57.008913+0000","last_became_active":"2024-09-16T00:08:36.888301+0000","last_became_peered":"2024-09-16T00:08:36.888301+0000","last_unstale":"2024-09-16T00:08:57.008913+0000","last_undegraded":"2024-09-16T00:08:57.008913+0000","last_fullsized":"2024-09-16T00:08:57.008913+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21479,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991795+0000","last_change":"2024-09-16T00:08:36.892595+0000","last_active":"2024-09-16T00:08:55.991795+0000","last_peered":"2024-09-16T00:08:55.991795+0000","last_clean":"2024-09-16T00:08:55.991795+0000","last_became_active":"2024-09-16T00:08:36.892316+0000","last_became_peered":"2024-09-16T00:08:36.892316+0000","last_unstale":"2024-09-16T00:08:55.991795+0000","last_undegraded":"2024-09-16T00:08:55.991795+0000","last_fullsized":"2024-09-16T00:08:55.991795+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:47.480168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"56'1","reported_seq":80,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973769+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:08:54.973769+0000","last_peered":"2024-09-16T00:08:54.973769+0000","last_clean":"2024-09-16T00:08:54.973769+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:08:54.973769+0000","last_undegraded":"2024-09-16T00:08:54.973769+0000","last_fullsized":"2024-09-16T00:08:54.973769+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979862+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:08:55.979862+0000","last_peered":"2024-09-16T00:08:55.979862+0000","last_clean":"2024-09-16T00:08:55.979862+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:08:55.979862+0000","last_undegraded":"2024-09-16T00:08:55.979862+0000","last_fullsized":"2024-09-16T00:08:55.979862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"57'4","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004785+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:08:57.004785+0000","last_peered":"2024-09-16T00:08:57.004785+0000","last_clean":"2024-09-16T00:08:57.004785+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:08:57.004785+0000","last_undegraded":"2024-09-16T00:08:57.004785+0000","last_fullsized":"2024-09-16T00:08:57.004785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8128,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"57'4","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005170+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:08:57.005170+0000","last_peered":"2024-09-16T00:08:57.005170+0000","last_clean":"2024-09-16T00:08:57.005170+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:08:57.005170+0000","last_undegraded":"2024-09-16T00:08:57.005170+0000","last_fullsized":"2024-09-16T00:08:57.005170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4240,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424154+0000","last_change":"2024-09-16T00:08:36.894430+0000","last_active":"2024-09-16T00:08:54.424154+0000","last_peered":"2024-09-16T00:08:54.424154+0000","last_clean":"2024-09-16T00:08:54.424154+0000","last_became_active":"2024-09-16T00:08:36.894138+0000","last_became_peered":"2024-09-16T00:08:36.894138+0000","last_unstale":"2024-09-16T00:08:54.424154+0000","last_undegraded":"2024-09-16T00:08:54.424154+0000","last_fullsized":"2024-09-16T00:08:54.424154+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:19.478353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"57'4","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.985021+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:08:56.985021+0000","last_peered":"2024-09-16T00:08:56.985021+0000","last_clean":"2024-09-16T00:08:56.985021+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:08:56.985021+0000","last_undegraded":"2024-09-16T00:08:56.985021+0000","last_fullsized":"2024-09-16T00:08:56.985021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1845,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"58'8","reported_seq":39,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983626+0000","last_change":"2024-09-16T00:08:36.890676+0000","last_active":"2024-09-16T00:08:56.983626+0000","last_peered":"2024-09-16T00:08:56.983626+0000","last_clean":"2024-09-16T00:08:56.983626+0000","last_became_active":"2024-09-16T00:08:36.889743+0000","last_became_peered":"2024-09-16T00:08:36.889743+0000","last_unstale":"2024-09-16T00:08:56.983626+0000","last_undegraded":"2024-09-16T00:08:56.983626+0000","last_fullsized":"2024-09-16T00:08:56.983626+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:02.709496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21485,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973839+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:08:54.973839+0000","last_peered":"2024-09-16T00:08:54.973839+0000","last_clean":"2024-09-16T00:08:54.973839+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:08:54.973839+0000","last_undegraded":"2024-09-16T00:08:54.973839+0000","last_fullsized":"2024-09-16T00:08:54.973839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"57'7","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004869+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:08:57.004869+0000","last_peered":"2024-09-16T00:08:57.004869+0000","last_clean":"2024-09-16T00:08:57.004869+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:08:57.004869+0000","last_undegraded":"2024-09-16T00:08:57.004869+0000","last_fullsized":"2024-09-16T00:08:57.004869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69449,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991605+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:08:55.991605+0000","last_peered":"2024-09-16T00:08:55.991605+0000","last_clean":"2024-09-16T00:08:55.991605+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:08:55.991605+0000","last_undegraded":"2024-09-16T00:08:55.991605+0000","last_fullsized":"2024-09-16T00:08:55.991605+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"57'7","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004863+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:08:57.004863+0000","last_peered":"2024-09-16T00:08:57.004863+0000","last_clean":"2024-09-16T00:08:57.004863+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:08:57.004863+0000","last_undegraded":"2024-09-16T00:08:57.004863+0000","last_fullsized":"2024-09-16T00:08:57.004863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14047,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.992055+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:08:55.992055+0000","last_peered":"2024-09-16T00:08:55.992055+0000","last_clean":"2024-09-16T00:08:55.992055+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:08:55.992055+0000","last_undegraded":"2024-09-16T00:08:55.992055+0000","last_fullsized":"2024-09-16T00:08:55.992055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"57'7","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983874+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:08:56.983874+0000","last_peered":"2024-09-16T00:08:56.983874+0000","last_clean":"2024-09-16T00:08:56.983874+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:08:56.983874+0000","last_undegraded":"2024-09-16T00:08:56.983874+0000","last_fullsized":"2024-09-16T00:08:56.983874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92931,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979860+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:08:55.979860+0000","last_peered":"2024-09-16T00:08:55.979860+0000","last_clean":"2024-09-16T00:08:55.979860+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:08:55.979860+0000","last_undegraded":"2024-09-16T00:08:55.979860+0000","last_fullsized":"2024-09-16T00:08:55.979860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991901+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:08:55.991901+0000","last_peered":"2024-09-16T00:08:55.991901+0000","last_clean":"2024-09-16T00:08:55.991901+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:08:55.991901+0000","last_undegraded":"2024-09-16T00:08:55.991901+0000","last_fullsized":"2024-09-16T00:08:55.991901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973891+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:08:54.973891+0000","last_peered":"2024-09-16T00:08:54.973891+0000","last_clean":"2024-09-16T00:08:54.973891+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:08:54.973891+0000","last_undegraded":"2024-09-16T00:08:54.973891+0000","last_fullsized":"2024-09-16T00:08:54.973891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":52,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424099+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:08:54.424099+0000","last_peered":"2024-09-16T00:08:54.424099+0000","last_clean":"2024-09-16T00:08:54.424099+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:08:54.424099+0000","last_undegraded":"2024-09-16T00:08:54.424099+0000","last_fullsized":"2024-09-16T00:08:54.424099+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":83,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424334+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:08:54.424334+0000","last_peered":"2024-09-16T00:08:54.424334+0000","last_clean":"2024-09-16T00:08:54.424334+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:08:54.424334+0000","last_undegraded":"2024-09-16T00:08:54.424334+0000","last_fullsized":"2024-09-16T00:08:54.424334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"57'6","reported_seq":37,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983162+0000","last_change":"2024-09-16T00:08:36.894101+0000","last_active":"2024-09-16T00:08:56.983162+0000","last_peered":"2024-09-16T00:08:56.983162+0000","last_clean":"2024-09-16T00:08:56.983162+0000","last_became_active":"2024-09-16T00:08:36.893807+0000","last_became_peered":"2024-09-16T00:08:36.893807+0000","last_unstale":"2024-09-16T00:08:56.983162+0000","last_undegraded":"2024-09-16T00:08:56.983162+0000","last_fullsized":"2024-09-16T00:08:56.983162+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:40.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"58'7","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.010727+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:08:57.010727+0000","last_peered":"2024-09-16T00:08:57.010727+0000","last_clean":"2024-09-16T00:08:57.010727+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:08:57.010727+0000","last_undegraded":"2024-09-16T00:08:57.010727+0000","last_fullsized":"2024-09-16T00:08:57.010727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16184,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":85,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991911+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:08:55.991911+0000","last_peered":"2024-09-16T00:08:55.991911+0000","last_clean":"2024-09-16T00:08:55.991911+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:08:55.991911+0000","last_undegraded":"2024-09-16T00:08:55.991911+0000","last_fullsized":"2024-09-16T00:08:55.991911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973910+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:08:54.973910+0000","last_peered":"2024-09-16T00:08:54.973910+0000","last_clean":"2024-09-16T00:08:54.973910+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:08:54.973910+0000","last_undegraded":"2024-09-16T00:08:54.973910+0000","last_fullsized":"2024-09-16T00:08:54.973910+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":31,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983375+0000","last_change":"2024-09-16T00:08:36.890316+0000","last_active":"2024-09-16T00:08:56.983375+0000","last_peered":"2024-09-16T00:08:56.983375+0000","last_clean":"2024-09-16T00:08:56.983375+0000","last_became_active":"2024-09-16T00:08:36.889635+0000","last_became_peered":"2024-09-16T00:08:36.889635+0000","last_unstale":"2024-09-16T00:08:56.983375+0000","last_undegraded":"2024-09-16T00:08:56.983375+0000","last_fullsized":"2024-09-16T00:08:56.983375+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:28.843883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":94,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991691+0000","last_change":"2024-09-16T00:08:36.888349+0000","last_active":"2024-09-16T00:08:55.991691+0000","last_peered":"2024-09-16T00:08:55.991691+0000","last_clean":"2024-09-16T00:08:55.991691+0000","last_became_active":"2024-09-16T00:08:36.888052+0000","last_became_peered":"2024-09-16T00:08:36.888052+0000","last_unstale":"2024-09-16T00:08:55.991691+0000","last_undegraded":"2024-09-16T00:08:55.991691+0000","last_fullsized":"2024-09-16T00:08:55.991691+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:49.607254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991502+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:08:55.991502+0000","last_peered":"2024-09-16T00:08:55.991502+0000","last_clean":"2024-09-16T00:08:55.991502+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:08:55.991502+0000","last_undegraded":"2024-09-16T00:08:55.991502+0000","last_fullsized":"2024-09-16T00:08:55.991502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004896+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:08:57.004896+0000","last_peered":"2024-09-16T00:08:57.004896+0000","last_clean":"2024-09-16T00:08:57.004896+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:08:57.004896+0000","last_undegraded":"2024-09-16T00:08:57.004896+0000","last_fullsized":"2024-09-16T00:08:57.004896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979829+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:08:55.979829+0000","last_peered":"2024-09-16T00:08:55.979829+0000","last_clean":"2024-09-16T00:08:55.979829+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:08:55.979829+0000","last_undegraded":"2024-09-16T00:08:55.979829+0000","last_fullsized":"2024-09-16T00:08:55.979829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983626+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:08:56.983626+0000","last_peered":"2024-09-16T00:08:56.983626+0000","last_clean":"2024-09-16T00:08:56.983626+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:08:56.983626+0000","last_undegraded":"2024-09-16T00:08:56.983626+0000","last_fullsized":"2024-09-16T00:08:56.983626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"57'5","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005138+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:08:57.005138+0000","last_peered":"2024-09-16T00:08:57.005138+0000","last_clean":"2024-09-16T00:08:57.005138+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:08:57.005138+0000","last_undegraded":"2024-09-16T00:08:57.005138+0000","last_fullsized":"2024-09-16T00:08:57.005138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":80946,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005068+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:08:57.005068+0000","last_peered":"2024-09-16T00:08:57.005068+0000","last_clean":"2024-09-16T00:08:57.005068+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:08:57.005068+0000","last_undegraded":"2024-09-16T00:08:57.005068+0000","last_fullsized":"2024-09-16T00:08:57.005068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.974071+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:08:54.974071+0000","last_peered":"2024-09-16T00:08:54.974071+0000","last_clean":"2024-09-16T00:08:54.974071+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:08:54.974071+0000","last_undegraded":"2024-09-16T00:08:54.974071+0000","last_fullsized":"2024-09-16T00:08:54.974071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":86,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980005+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:08:55.980005+0000","last_peered":"2024-09-16T00:08:55.980005+0000","last_clean":"2024-09-16T00:08:55.980005+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:08:55.980005+0000","last_undegraded":"2024-09-16T00:08:55.980005+0000","last_fullsized":"2024-09-16T00:08:55.980005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"57'8","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.008894+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:08:57.008894+0000","last_peered":"2024-09-16T00:08:57.008894+0000","last_clean":"2024-09-16T00:08:57.008894+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:08:57.008894+0000","last_undegraded":"2024-09-16T00:08:57.008894+0000","last_fullsized":"2024-09-16T00:08:57.008894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84046,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"57'15","reported_seq":46,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983597+0000","last_change":"2024-09-16T00:08:36.914308+0000","last_active":"2024-09-16T00:08:56.983597+0000","last_peered":"2024-09-16T00:08:56.983597+0000","last_clean":"2024-09-16T00:08:56.983597+0000","last_became_active":"2024-09-16T00:08:36.890227+0000","last_became_peered":"2024-09-16T00:08:36.890227+0000","last_unstale":"2024-09-16T00:08:56.983597+0000","last_undegraded":"2024-09-16T00:08:56.983597+0000","last_fullsized":"2024-09-16T00:08:56.983597+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:31.761195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21522,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13110,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"57'3","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984771+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:08:56.984771+0000","last_peered":"2024-09-16T00:08:56.984771+0000","last_clean":"2024-09-16T00:08:56.984771+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:08:56.984771+0000","last_undegraded":"2024-09-16T00:08:56.984771+0000","last_fullsized":"2024-09-16T00:08:56.984771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2379,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":83,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973988+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:08:54.973988+0000","last_peered":"2024-09-16T00:08:54.973988+0000","last_clean":"2024-09-16T00:08:54.973988+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:08:54.973988+0000","last_undegraded":"2024-09-16T00:08:54.973988+0000","last_fullsized":"2024-09-16T00:08:54.973988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973740+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:08:54.973740+0000","last_peered":"2024-09-16T00:08:54.973740+0000","last_clean":"2024-09-16T00:08:54.973740+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:08:54.973740+0000","last_undegraded":"2024-09-16T00:08:54.973740+0000","last_fullsized":"2024-09-16T00:08:54.973740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004924+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:08:57.004924+0000","last_peered":"2024-09-16T00:08:57.004924+0000","last_clean":"2024-09-16T00:08:57.004924+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:08:57.004924+0000","last_undegraded":"2024-09-16T00:08:57.004924+0000","last_fullsized":"2024-09-16T00:08:57.004924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"57'10","reported_seq":93,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984391+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:08:56.984391+0000","last_peered":"2024-09-16T00:08:56.984391+0000","last_clean":"2024-09-16T00:08:56.984391+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:08:56.984391+0000","last_undegraded":"2024-09-16T00:08:56.984391+0000","last_fullsized":"2024-09-16T00:08:56.984391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2388208,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":2338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":31,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983231+0000","last_change":"2024-09-16T00:08:36.889720+0000","last_active":"2024-09-16T00:08:56.983231+0000","last_peered":"2024-09-16T00:08:56.983231+0000","last_clean":"2024-09-16T00:08:56.983231+0000","last_became_active":"2024-09-16T00:08:36.889345+0000","last_became_peered":"2024-09-16T00:08:36.889345+0000","last_unstale":"2024-09-16T00:08:56.983231+0000","last_undegraded":"2024-09-16T00:08:56.983231+0000","last_fullsized":"2024-09-16T00:08:56.983231+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:03.170073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"58'7","reported_seq":38,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983217+0000","last_change":"2024-09-16T00:08:36.896787+0000","last_active":"2024-09-16T00:08:56.983217+0000","last_peered":"2024-09-16T00:08:56.983217+0000","last_clean":"2024-09-16T00:08:56.983217+0000","last_became_active":"2024-09-16T00:08:36.896505+0000","last_became_peered":"2024-09-16T00:08:36.896505+0000","last_unstale":"2024-09-16T00:08:56.983217+0000","last_undegraded":"2024-09-16T00:08:56.983217+0000","last_fullsized":"2024-09-16T00:08:56.983217+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:55.997551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4202646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424000+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:08:54.424000+0000","last_peered":"2024-09-16T00:08:54.424000+0000","last_clean":"2024-09-16T00:08:54.424000+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:08:54.424000+0000","last_undegraded":"2024-09-16T00:08:54.424000+0000","last_fullsized":"2024-09-16T00:08:54.424000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"57'5","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984434+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:08:56.984434+0000","last_peered":"2024-09-16T00:08:56.984434+0000","last_clean":"2024-09-16T00:08:56.984434+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:08:56.984434+0000","last_undegraded":"2024-09-16T00:08:56.984434+0000","last_fullsized":"2024-09-16T00:08:56.984434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.008735+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:08:57.008735+0000","last_peered":"2024-09-16T00:08:57.008735+0000","last_clean":"2024-09-16T00:08:57.008735+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:08:57.008735+0000","last_undegraded":"2024-09-16T00:08:57.008735+0000","last_fullsized":"2024-09-16T00:08:57.008735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"57'5","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984850+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:08:56.984850+0000","last_peered":"2024-09-16T00:08:56.984850+0000","last_clean":"2024-09-16T00:08:56.984850+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:08:56.984850+0000","last_undegraded":"2024-09-16T00:08:56.984850+0000","last_fullsized":"2024-09-16T00:08:56.984850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81483,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":82,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424201+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:08:54.424201+0000","last_peered":"2024-09-16T00:08:54.424201+0000","last_clean":"2024-09-16T00:08:54.424201+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:08:54.424201+0000","last_undegraded":"2024-09-16T00:08:54.424201+0000","last_fullsized":"2024-09-16T00:08:54.424201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"57'6","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.985051+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:08:56.985051+0000","last_peered":"2024-09-16T00:08:56.985051+0000","last_clean":"2024-09-16T00:08:56.985051+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:08:56.985051+0000","last_undegraded":"2024-09-16T00:08:56.985051+0000","last_fullsized":"2024-09-16T00:08:56.985051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22306,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":84,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980097+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:08:55.980097+0000","last_peered":"2024-09-16T00:08:55.980097+0000","last_clean":"2024-09-16T00:08:55.980097+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:08:55.980097+0000","last_undegraded":"2024-09-16T00:08:55.980097+0000","last_fullsized":"2024-09-16T00:08:55.980097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"57'8","reported_seq":92,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005154+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:08:57.005154+0000","last_peered":"2024-09-16T00:08:57.005154+0000","last_clean":"2024-09-16T00:08:57.005154+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:08:57.005154+0000","last_undegraded":"2024-09-16T00:08:57.005154+0000","last_fullsized":"2024-09-16T00:08:57.005154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32783,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":80,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424063+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:08:54.424063+0000","last_peered":"2024-09-16T00:08:54.424063+0000","last_clean":"2024-09-16T00:08:54.424063+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:08:54.424063+0000","last_undegraded":"2024-09-16T00:08:54.424063+0000","last_fullsized":"2024-09-16T00:08:54.424063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"58'7","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.029232+0000","last_change":"2024-09-16T00:08:47.953122+0000","last_active":"2024-09-16T00:08:57.029232+0000","last_peered":"2024-09-16T00:08:57.029232+0000","last_clean":"2024-09-16T00:08:57.029232+0000","last_became_active":"2024-09-16T00:08:47.952799+0000","last_became_peered":"2024-09-16T00:08:47.952799+0000","last_unstale":"2024-09-16T00:08:57.029232+0000","last_undegraded":"2024-09-16T00:08:57.029232+0000","last_fullsized":"2024-09-16T00:08:57.029232+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:03.447562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61360,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":241,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984180+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:08:56.984180+0000","last_peered":"2024-09-16T00:08:56.984180+0000","last_clean":"2024-09-16T00:08:56.984180+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:08:56.984180+0000","last_undegraded":"2024-09-16T00:08:56.984180+0000","last_fullsized":"2024-09-16T00:08:56.984180+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004860+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:08:57.004860+0000","last_peered":"2024-09-16T00:08:57.004860+0000","last_clean":"2024-09-16T00:08:57.004860+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:08:57.004860+0000","last_undegraded":"2024-09-16T00:08:57.004860+0000","last_fullsized":"2024-09-16T00:08:57.004860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991784+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:08:55.991784+0000","last_peered":"2024-09-16T00:08:55.991784+0000","last_clean":"2024-09-16T00:08:55.991784+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:08:55.991784+0000","last_undegraded":"2024-09-16T00:08:55.991784+0000","last_fullsized":"2024-09-16T00:08:55.991784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984166+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:08:56.984166+0000","last_peered":"2024-09-16T00:08:56.984166+0000","last_clean":"2024-09-16T00:08:56.984166+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:08:56.984166+0000","last_undegraded":"2024-09-16T00:08:56.984166+0000","last_fullsized":"2024-09-16T00:08:56.984166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973713+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:08:54.973713+0000","last_peered":"2024-09-16T00:08:54.973713+0000","last_clean":"2024-09-16T00:08:54.973713+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:08:54.973713+0000","last_undegraded":"2024-09-16T00:08:54.973713+0000","last_fullsized":"2024-09-16T00:08:54.973713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":80,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424424+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:08:54.424424+0000","last_peered":"2024-09-16T00:08:54.424424+0000","last_clean":"2024-09-16T00:08:54.424424+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:08:54.424424+0000","last_undegraded":"2024-09-16T00:08:54.424424+0000","last_fullsized":"2024-09-16T00:08:54.424424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979829+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:08:55.979829+0000","last_peered":"2024-09-16T00:08:55.979829+0000","last_clean":"2024-09-16T00:08:55.979829+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:08:55.979829+0000","last_undegraded":"2024-09-16T00:08:55.979829+0000","last_fullsized":"2024-09-16T00:08:55.979829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004966+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:08:57.004966+0000","last_peered":"2024-09-16T00:08:57.004966+0000","last_clean":"2024-09-16T00:08:57.004966+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:08:57.004966+0000","last_undegraded":"2024-09-16T00:08:57.004966+0000","last_fullsized":"2024-09-16T00:08:57.004966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"58'3","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.015492+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:08:57.015492+0000","last_peered":"2024-09-16T00:08:57.015492+0000","last_clean":"2024-09-16T00:08:57.015492+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:08:57.015492+0000","last_undegraded":"2024-09-16T00:08:57.015492+0000","last_fullsized":"2024-09-16T00:08:57.015492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1838,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":83,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979978+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:08:55.979978+0000","last_peered":"2024-09-16T00:08:55.979978+0000","last_clean":"2024-09-16T00:08:55.979978+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:08:55.979978+0000","last_undegraded":"2024-09-16T00:08:55.979978+0000","last_fullsized":"2024-09-16T00:08:55.979978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"57'5","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005147+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:08:57.005147+0000","last_peered":"2024-09-16T00:08:57.005147+0000","last_clean":"2024-09-16T00:08:57.005147+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:08:57.005147+0000","last_undegraded":"2024-09-16T00:08:57.005147+0000","last_fullsized":"2024-09-16T00:08:57.005147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13443,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.985261+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:08:56.985261+0000","last_peered":"2024-09-16T00:08:56.985261+0000","last_clean":"2024-09-16T00:08:56.985261+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:08:56.985261+0000","last_undegraded":"2024-09-16T00:08:56.985261+0000","last_fullsized":"2024-09-16T00:08:56.985261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979860+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:08:55.979860+0000","last_peered":"2024-09-16T00:08:55.979860+0000","last_clean":"2024-09-16T00:08:55.979860+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:08:55.979860+0000","last_undegraded":"2024-09-16T00:08:55.979860+0000","last_fullsized":"2024-09-16T00:08:55.979860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"56'4","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980204+0000","last_change":"2024-09-16T00:08:36.906253+0000","last_active":"2024-09-16T00:08:55.980204+0000","last_peered":"2024-09-16T00:08:55.980204+0000","last_clean":"2024-09-16T00:08:55.980204+0000","last_became_active":"2024-09-16T00:08:36.894363+0000","last_became_peered":"2024-09-16T00:08:36.894363+0000","last_unstale":"2024-09-16T00:08:55.980204+0000","last_undegraded":"2024-09-16T00:08:55.980204+0000","last_fullsized":"2024-09-16T00:08:55.980204+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:00.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.992092+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:08:55.992092+0000","last_peered":"2024-09-16T00:08:55.992092+0000","last_clean":"2024-09-16T00:08:55.992092+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:08:55.992092+0000","last_undegraded":"2024-09-16T00:08:55.992092+0000","last_fullsized":"2024-09-16T00:08:55.992092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983637+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:08:56.983637+0000","last_peered":"2024-09-16T00:08:56.983637+0000","last_clean":"2024-09-16T00:08:56.983637+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:08:56.983637+0000","last_undegraded":"2024-09-16T00:08:56.983637+0000","last_fullsized":"2024-09-16T00:08:56.983637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.974120+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:08:54.974120+0000","last_peered":"2024-09-16T00:08:54.974120+0000","last_clean":"2024-09-16T00:08:54.974120+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:08:54.974120+0000","last_undegraded":"2024-09-16T00:08:54.974120+0000","last_fullsized":"2024-09-16T00:08:54.974120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984026+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:08:56.984026+0000","last_peered":"2024-09-16T00:08:56.984026+0000","last_clean":"2024-09-16T00:08:56.984026+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:08:56.984026+0000","last_undegraded":"2024-09-16T00:08:56.984026+0000","last_fullsized":"2024-09-16T00:08:56.984026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"57'4","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.008594+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:08:57.008594+0000","last_peered":"2024-09-16T00:08:57.008594+0000","last_clean":"2024-09-16T00:08:57.008594+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:08:57.008594+0000","last_undegraded":"2024-09-16T00:08:57.008594+0000","last_fullsized":"2024-09-16T00:08:57.008594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22937,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":83,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979932+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:08:55.979932+0000","last_peered":"2024-09-16T00:08:55.979932+0000","last_clean":"2024-09-16T00:08:55.979932+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:08:55.979932+0000","last_undegraded":"2024-09-16T00:08:55.979932+0000","last_fullsized":"2024-09-16T00:08:55.979932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991568+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:08:55.991568+0000","last_peered":"2024-09-16T00:08:55.991568+0000","last_clean":"2024-09-16T00:08:55.991568+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:08:55.991568+0000","last_undegraded":"2024-09-16T00:08:55.991568+0000","last_fullsized":"2024-09-16T00:08:55.991568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":83,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980089+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:08:55.980089+0000","last_peered":"2024-09-16T00:08:55.980089+0000","last_clean":"2024-09-16T00:08:55.980089+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:08:55.980089+0000","last_undegraded":"2024-09-16T00:08:55.980089+0000","last_fullsized":"2024-09-16T00:08:55.980089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.992004+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:08:55.992004+0000","last_peered":"2024-09-16T00:08:55.992004+0000","last_clean":"2024-09-16T00:08:55.992004+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:08:55.992004+0000","last_undegraded":"2024-09-16T00:08:55.992004+0000","last_fullsized":"2024-09-16T00:08:55.992004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984458+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:08:56.984458+0000","last_peered":"2024-09-16T00:08:56.984458+0000","last_clean":"2024-09-16T00:08:56.984458+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:08:56.984458+0000","last_undegraded":"2024-09-16T00:08:56.984458+0000","last_fullsized":"2024-09-16T00:08:56.984458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"58'6","reported_seq":37,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983319+0000","last_change":"2024-09-16T00:08:36.890347+0000","last_active":"2024-09-16T00:08:56.983319+0000","last_peered":"2024-09-16T00:08:56.983319+0000","last_clean":"2024-09-16T00:08:56.983319+0000","last_became_active":"2024-09-16T00:08:36.890009+0000","last_became_peered":"2024-09-16T00:08:36.890009+0000","last_unstale":"2024-09-16T00:08:56.983319+0000","last_undegraded":"2024-09-16T00:08:56.983319+0000","last_fullsized":"2024-09-16T00:08:56.983319+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:24.298978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19739,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"57'4","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005223+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:08:57.005223+0000","last_peered":"2024-09-16T00:08:57.005223+0000","last_clean":"2024-09-16T00:08:57.005223+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:08:57.005223+0000","last_undegraded":"2024-09-16T00:08:57.005223+0000","last_fullsized":"2024-09-16T00:08:57.005223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2557,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984758+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:08:56.984758+0000","last_peered":"2024-09-16T00:08:56.984758+0000","last_clean":"2024-09-16T00:08:56.984758+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:08:56.984758+0000","last_undegraded":"2024-09-16T00:08:56.984758+0000","last_fullsized":"2024-09-16T00:08:56.984758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'2","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005087+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:08:57.005087+0000","last_peered":"2024-09-16T00:08:57.005087+0000","last_clean":"2024-09-16T00:08:57.005087+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:08:57.005087+0000","last_undegraded":"2024-09-16T00:08:57.005087+0000","last_fullsized":"2024-09-16T00:08:57.005087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":105,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984904+0000","last_change":"2024-09-16T00:08:36.900704+0000","last_active":"2024-09-16T00:08:56.984904+0000","last_peered":"2024-09-16T00:08:56.984904+0000","last_clean":"2024-09-16T00:08:56.984904+0000","last_became_active":"2024-09-16T00:08:36.892520+0000","last_became_peered":"2024-09-16T00:08:36.892520+0000","last_unstale":"2024-09-16T00:08:56.984904+0000","last_undegraded":"2024-09-16T00:08:56.984904+0000","last_fullsized":"2024-09-16T00:08:56.984904+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:30.056041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":52,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424040+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:08:54.424040+0000","last_peered":"2024-09-16T00:08:54.424040+0000","last_clean":"2024-09-16T00:08:54.424040+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:08:54.424040+0000","last_undegraded":"2024-09-16T00:08:54.424040+0000","last_fullsized":"2024-09-16T00:08:54.424040+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":85,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.992032+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:08:55.992032+0000","last_peered":"2024-09-16T00:08:55.992032+0000","last_clean":"2024-09-16T00:08:55.992032+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:08:55.992032+0000","last_undegraded":"2024-09-16T00:08:55.992032+0000","last_fullsized":"2024-09-16T00:08:55.992032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991703+0000","last_change":"2024-09-16T00:08:36.889082+0000","last_active":"2024-09-16T00:08:55.991703+0000","last_peered":"2024-09-16T00:08:55.991703+0000","last_clean":"2024-09-16T00:08:55.991703+0000","last_became_active":"2024-09-16T00:08:36.888807+0000","last_became_peered":"2024-09-16T00:08:36.888807+0000","last_unstale":"2024-09-16T00:08:55.991703+0000","last_undegraded":"2024-09-16T00:08:55.991703+0000","last_fullsized":"2024-09-16T00:08:55.991703+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:22.825324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":85,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.974121+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:08:54.974121+0000","last_peered":"2024-09-16T00:08:54.974121+0000","last_clean":"2024-09-16T00:08:54.974121+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:08:54.974121+0000","last_undegraded":"2024-09-16T00:08:54.974121+0000","last_fullsized":"2024-09-16T00:08:54.974121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":7434637,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10895360,"data_stored":10700843,"data_compressed":270015,"data_compressed_allocated":344064,"data_compressed_original":1134592,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":22320,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":13110,"num_keys_recovered":4,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":40694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30268,"kb_used_data":3240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341572,"statfs":{"total":96636764160,"available":96605769728,"internally_reserved":0,"allocated":3317760,"data_stored":3592203,"data_compressed":73051,"data_compressed_allocated":94208,"data_compressed_original":831488,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":5144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":5267456,"data_stored":4867710,"data_compressed":57794,"data_compressed_allocated":73728,"data_compressed_original":237568,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343940,"statfs":{"total":96636764160,"available":96608194560,"internally_reserved":0,"allocated":897024,"data_stored":548421,"data_compressed":30333,"data_compressed_allocated":40960,"data_compressed_original":180224,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30468,"kb_used_data":3444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341372,"statfs":{"total":96636764160,"available":96605564928,"internally_reserved":0,"allocated":3526656,"data_stored":3061123,"data_compressed":69574,"data_compressed_allocated":86016,"data_compressed_original":237568,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":655360,"data_stored":291719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":647168,"data_stored":285324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":716800,"data_stored":876170,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":626688,"data_stored":279070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":17576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":21522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2936832,"data_stored":2786732,"data_compressed":69574,"data_compressed_allocated":86016,"data_compressed_original":237568,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":398309,"data_compressed":60175,"data_compressed_allocated":77824,"data_compressed_original":237568,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4579328,"data_stored":4549565,"data_compressed":57794,"data_compressed_allocated":73728,"data_compressed_original":237568,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2908160,"data_stored":2966237,"data_compressed":82472,"data_compressed_allocated":106496,"data_compressed_original":421888,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:09:00.806 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T00:09:00.806 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:09:00.807 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T00:09:00.807 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T00:09:00.842 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:00.842+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:00.867 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:09:00.916 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:00.916 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:00.925 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:09:00.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.956+0000 7f4c58680640 1 Processor -- start 2024-09-16T00:09:00.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.956+0000 7f4c58680640 1 -- start start 2024-09-16T00:09:00.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.957+0000 7f4c58680640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c500c67e0 0x7f4c500c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.957+0000 7f4c58680640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4c5006c110 0x7f4c5015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.957+0000 7f4c58680640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c5006ca30 con 0x7f4c5006c530 2024-09-16T00:09:00.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.957+0000 7f4c58680640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f4c5006cd10 con 0x7f4c5006c110 2024-09-16T00:09:00.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.957+0000 7f4c58680640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4c500c6bc0 con 0x7f4c500c67e0 2024-09-16T00:09:00.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.957+0000 7f4c55c1b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4c5006c110 0x7f4c5015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.957+0000 7f4c5641c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c500c67e0 0x7f4c500c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.957+0000 7f4c56c1d640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f4c5006c530 0x7f4c500cbe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:45616/0 (socket says 172.21.15.139:45616) 2024-09-16T00:09:00.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.957+0000 7f4c56c1d640 1 -- 172.21.15.139:0/3566360428 learned_addr learned my addr 172.21.15.139:0/3566360428 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:00.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.957+0000 7f4c55c1b640 1 -- 172.21.15.139:0/3566360428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c500c67e0 msgr2=0x7f4c500c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:00.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.957+0000 7f4c55c1b640 1 --2- 172.21.15.139:0/3566360428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c500c67e0 0x7f4c500c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.957+0000 7f4c55c1b640 1 -- 172.21.15.139:0/3566360428 >> v1:172.21.15.164:6789/0 conn(0x7f4c5006c530 legacy=0x7f4c500cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.957+0000 7f4c55c1b640 1 -- 172.21.15.139:0/3566360428 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5015df20 con 0x7f4c5006c110 2024-09-16T00:09:00.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.958+0000 7f4c55c1b640 1 --2- 172.21.15.139:0/3566360428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c5006c110 0x7f4c5015a810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4c400097d0 tx=0x7f4c4002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b4983ea01c13d48 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:00.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.958+0000 7f4c5541a640 1 -- 172.21.15.139:0/3566360428 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40037070 con 0x7f4c5006c110 2024-09-16T00:09:00.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.959+0000 7f4c5541a640 1 -- 172.21.15.139:0/3566360428 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c40032440 con 0x7f4c5006c110 2024-09-16T00:09:00.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.959+0000 7f4c5541a640 1 -- 172.21.15.139:0/3566360428 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c40032810 con 0x7f4c5006c110 2024-09-16T00:09:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.959+0000 7f4c58680640 1 -- 172.21.15.139:0/3566360428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c5006c110 msgr2=0x7f4c5015a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.959+0000 7f4c58680640 1 --2- 172.21.15.139:0/3566360428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c5006c110 0x7f4c5015a810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4c400097d0 tx=0x7f4c4002a250 comp rx=0 tx=0).stop 2024-09-16T00:09:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.960+0000 7f4c58680640 1 -- 172.21.15.139:0/3566360428 shutdown_connections 2024-09-16T00:09:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.960+0000 7f4c58680640 1 --2- 172.21.15.139:0/3566360428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c5006c110 0x7f4c5015a810 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.960+0000 7f4c58680640 1 --2- 172.21.15.139:0/3566360428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c500c67e0 0x7f4c500c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.960+0000 7f4c58680640 1 -- 172.21.15.139:0/3566360428 >> 172.21.15.139:0/3566360428 conn(0x7f4c500d05f0 msgr2=0x7f4c500d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.960+0000 7f4c58680640 1 -- 172.21.15.139:0/3566360428 shutdown_connections 2024-09-16T00:09:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.960+0000 7f4c58680640 1 -- 172.21.15.139:0/3566360428 wait complete. 2024-09-16T00:09:00.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.961+0000 7f4c58680640 1 Processor -- start 2024-09-16T00:09:00.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.961+0000 7f4c58680640 1 -- start start 2024-09-16T00:09:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c58680640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c5006c110 0x7f4c5015a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c58680640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c5006c530 0x7f4c500cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c58680640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c500c67e0 0x7f4c50152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c58680640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5015e4e0 con 0x7f4c500c67e0 2024-09-16T00:09:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c58680640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5015e850 con 0x7f4c5006c110 2024-09-16T00:09:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c58680640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4c500cc560 con 0x7f4c5006c530 2024-09-16T00:09:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c5641c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c5006c110 0x7f4c5015a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c55c1b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c5006c530 0x7f4c500cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c56c1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c500c67e0 0x7f4c50152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c5641c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c5006c110 0x7f4c5015a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56828/0 (socket says 172.21.15.139:56828) 2024-09-16T00:09:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c5641c640 1 -- 172.21.15.139:0/4285820389 learned_addr learned my addr 172.21.15.139:0/4285820389 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c5641c640 1 -- 172.21.15.139:0/4285820389 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c5006c530 msgr2=0x7f4c500cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c5641c640 1 --2- 172.21.15.139:0/4285820389 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c5006c530 0x7f4c500cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c5641c640 1 -- 172.21.15.139:0/4285820389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c500c67e0 msgr2=0x7f4c50152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:00.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c5641c640 1 --2- 172.21.15.139:0/4285820389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c500c67e0 0x7f4c50152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:00.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c5641c640 1 -- 172.21.15.139:0/4285820389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c501587b0 con 0x7f4c5006c110 2024-09-16T00:09:00.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c55c1b640 1 --2- 172.21.15.139:0/4285820389 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c5006c530 0x7f4c500cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:00.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.962+0000 7f4c56c1d640 1 --2- 172.21.15.139:0/4285820389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c500c67e0 0x7f4c50152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:00.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.963+0000 7f4c5641c640 1 --2- 172.21.15.139:0/4285820389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c5006c110 0x7f4c5015a0a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00e870 tx=0x7f4c4c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:00.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.963+0000 7f4c46ffd640 1 -- 172.21.15.139:0/4285820389 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c017070 con 0x7f4c5006c110 2024-09-16T00:09:00.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.963+0000 7f4c46ffd640 1 -- 172.21.15.139:0/4285820389 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c4c00c940 con 0x7f4c5006c110 2024-09-16T00:09:00.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.963+0000 7f4c46ffd640 1 -- 172.21.15.139:0/4285820389 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00cd30 con 0x7f4c5006c110 2024-09-16T00:09:00.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.963+0000 7f4c58680640 1 -- 172.21.15.139:0/4285820389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c50158a50 con 0x7f4c5006c110 2024-09-16T00:09:00.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.963+0000 7f4c58680640 1 -- 172.21.15.139:0/4285820389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c50158d10 con 0x7f4c5006c110 2024-09-16T00:09:00.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.964+0000 7f4c58680640 1 -- 172.21.15.139:0/4285820389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c500c7b90 con 0x7f4c5006c110 2024-09-16T00:09:00.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.965+0000 7f4c46ffd640 1 -- 172.21.15.139:0/4285820389 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4c4c01a070 con 0x7f4c5006c110 2024-09-16T00:09:00.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.965+0000 7f4c46ffd640 1 --2- 172.21.15.139:0/4285820389 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c3406c2b0 0x7f4c3406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:00.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.965+0000 7f4c46ffd640 1 -- 172.21.15.139:0/4285820389 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5886+0+0 (secure 0 0 0) 0x7f4c4c08b690 con 0x7f4c5006c110 2024-09-16T00:09:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.966+0000 7f4c55c1b640 1 --2- 172.21.15.139:0/4285820389 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c3406c2b0 0x7f4c3406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.966+0000 7f4c55c1b640 1 --2- 172.21.15.139:0/4285820389 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c3406c2b0 0x7f4c3406e770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4c40034000 tx=0x7f4c4002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:00.968 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:00.968+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:00.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:00.970+0000 7f4c46ffd640 1 -- 172.21.15.139:0/4285820389 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4c4c059700 con 0x7f4c5006c110 2024-09-16T00:09:01.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.000+0000 7f8556e8d640 1 Processor -- start 2024-09-16T00:09:01.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.000+0000 7f8556e8d640 1 -- start start 2024-09-16T00:09:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.001+0000 7f8556e8d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f85500d0490 0x7f85500b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.001+0000 7f8556e8d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f85500d0870 0x7f85500b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.001+0000 7f8556e8d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f85500b3430 0x7f85500b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.001+0000 7f8556e8d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f855015b940 con 0x7f85500d0870 2024-09-16T00:09:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.001+0000 7f8556e8d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f85500b3bb0 con 0x7f85500d0490 2024-09-16T00:09:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.001+0000 7f8556e8d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f85500b3180 con 0x7f85500b3430 2024-09-16T00:09:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.001+0000 7f854ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f85500d0870 0x7f85500b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.001+0000 7f854ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f85500d0870 0x7f85500b4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41950/0 (socket says 172.21.15.139:41950) 2024-09-16T00:09:01.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.001+0000 7f854ffff640 1 -- 172.21.15.139:0/2277986677 learned_addr learned my addr 172.21.15.139:0/2277986677 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.001+0000 7f855542a640 1 --2- 172.21.15.139:0/2277986677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f85500b3430 0x7f85500b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.001+0000 7f8554c29640 1 --2- 172.21.15.139:0/2277986677 >> v2:172.21.15.164:3300/0 conn(0x7f85500d0490 0x7f85500b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.001+0000 7f854ffff640 1 -- 172.21.15.139:0/2277986677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f85500b3430 msgr2=0x7f85500b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.001+0000 7f854ffff640 1 --2- 172.21.15.139:0/2277986677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f85500b3430 0x7f85500b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.001+0000 7f854ffff640 1 -- 172.21.15.139:0/2277986677 >> v2:172.21.15.164:3300/0 conn(0x7f85500d0490 msgr2=0x7f85500b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.001+0000 7f854ffff640 1 --2- 172.21.15.139:0/2277986677 >> v2:172.21.15.164:3300/0 conn(0x7f85500d0490 0x7f85500b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.001+0000 7f854ffff640 1 -- 172.21.15.139:0/2277986677 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f855015adc0 con 0x7f85500d0870 2024-09-16T00:09:01.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.003+0000 7f854ffff640 1 --2- 172.21.15.139:0/2277986677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85500d0870 0x7f85500b4040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f85500cddf0 tx=0x7f853c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713ca9fa46860e4f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:01.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.004+0000 7f854f7fe640 1 -- 172.21.15.139:0/2277986677 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c037070 con 0x7f85500d0870 2024-09-16T00:09:01.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.004+0000 7f854f7fe640 1 -- 172.21.15.139:0/2277986677 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f853c032440 con 0x7f85500d0870 2024-09-16T00:09:01.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.004+0000 7f854f7fe640 1 -- 172.21.15.139:0/2277986677 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c032810 con 0x7f85500d0870 2024-09-16T00:09:01.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.005+0000 7f8556e8d640 1 -- 172.21.15.139:0/2277986677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85500d0870 msgr2=0x7f85500b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.005+0000 7f8556e8d640 1 --2- 172.21.15.139:0/2277986677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85500d0870 0x7f85500b4040 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f85500cddf0 tx=0x7f853c02a270 comp rx=0 tx=0).stop 2024-09-16T00:09:01.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.005+0000 7f8556e8d640 1 -- 172.21.15.139:0/2277986677 shutdown_connections 2024-09-16T00:09:01.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.005+0000 7f8556e8d640 1 --2- 172.21.15.139:0/2277986677 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f85500b3430 0x7f85500b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.005+0000 7f8556e8d640 1 --2- 172.21.15.139:0/2277986677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85500d0870 0x7f85500b4040 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.005+0000 7f8556e8d640 1 --2- 172.21.15.139:0/2277986677 >> v2:172.21.15.164:3300/0 conn(0x7f85500d0490 0x7f85500b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.005+0000 7f8556e8d640 1 -- 172.21.15.139:0/2277986677 >> 172.21.15.139:0/2277986677 conn(0x7f855006c520 msgr2=0x7f855006c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:01.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.005+0000 7f8556e8d640 1 -- 172.21.15.139:0/2277986677 shutdown_connections 2024-09-16T00:09:01.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.005+0000 7f8556e8d640 1 -- 172.21.15.139:0/2277986677 wait complete. 2024-09-16T00:09:01.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.007+0000 7f8556e8d640 1 Processor -- start 2024-09-16T00:09:01.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.007+0000 7f8556e8d640 1 -- start start 2024-09-16T00:09:01.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f8556e8d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85500b3430 0x7f8550149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f8556e8d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f85500d0490 0x7f85501f0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f8556e8d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85500d0870 0x7f85501484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f8556e8d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f855015b660 con 0x7f85500b3430 2024-09-16T00:09:01.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f8556e8d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f855015b380 con 0x7f85500d0870 2024-09-16T00:09:01.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f8556e8d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f855015a980 con 0x7f85500d0490 2024-09-16T00:09:01.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f8554c29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85500b3430 0x7f8550149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f8554c29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85500b3430 0x7f8550149930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41964/0 (socket says 172.21.15.139:41964) 2024-09-16T00:09:01.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f8554c29640 1 -- 172.21.15.139:0/3214667393 learned_addr learned my addr 172.21.15.139:0/3214667393 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:01.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f854ffff640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f85500d0490 0x7f85501f0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f855542a640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85500d0870 0x7f85501484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f854ffff640 1 -- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85500d0870 msgr2=0x7f85501484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f854ffff640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85500d0870 0x7f85501484d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f854ffff640 1 -- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85500b3430 msgr2=0x7f8550149930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f854ffff640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85500b3430 0x7f8550149930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f854ffff640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85501f46a0 con 0x7f85500d0490 2024-09-16T00:09:01.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.009+0000 7f855542a640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f85500d0870 0x7f85501484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:01.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.010+0000 7f854ffff640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f85500d0490 0x7f85501f0e80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f853c031000 tx=0x7f853c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:01.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.010+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c041020 con 0x7f85500d0490 2024-09-16T00:09:01.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.010+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f853c03d620 con 0x7f85500d0490 2024-09-16T00:09:01.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.010+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85501f4920 con 0x7f85500d0490 2024-09-16T00:09:01.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.010+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c037040 con 0x7f85500d0490 2024-09-16T00:09:01.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.010+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f85500c7f50 con 0x7f85500d0490 2024-09-16T00:09:01.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.011+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f85501f0750 con 0x7f85500d0490 2024-09-16T00:09:01.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.012+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f853c044020 con 0x7f85500d0490 2024-09-16T00:09:01.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.012+0000 7f854d7fa640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f853406c330 0x7f853406e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.013+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(61..61 src has 1..61) ==== 5886+0+0 (secure 0 0 0) 0x7f853c0ab910 con 0x7f85500d0490 2024-09-16T00:09:01.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.013+0000 7f8554c29640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f853406c330 0x7f853406e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.013+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v61) ==== 54+0+16 (secure 0 0 0) 0x7f853c04b030 con 0x7f85500d0490 2024-09-16T00:09:01.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.013+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f855014ac70 0x7f8550154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.013+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85501f4c00 con 0x7f855014ac70 2024-09-16T00:09:01.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.013+0000 7f8554c29640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f853406c330 0x7f853406e7f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8544009c00 tx=0x7f8544009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:01.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.013+0000 7f855542a640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f855014ac70 0x7f8550154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.013+0000 7f855542a640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f855014ac70 0x7f8550154450 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:01.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.014+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f85501f4c00 con 0x7f855014ac70 2024-09-16T00:09:01.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.032+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f8550148f20 con 0x7f855014ac70 2024-09-16T00:09:01.033 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:01.032+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:01.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.034+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8550148f20 con 0x7f855014ac70 2024-09-16T00:09:01.035 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:01.035 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:01.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.034+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f85501f13c0 0x7f855014fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.034+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f85500b2c40 con 0x7f85501f13c0 2024-09-16T00:09:01.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.034+0000 7f8554c29640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f85501f13c0 0x7f855014fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.035+0000 7f8554c29640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f85501f13c0 0x7f855014fc00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:01.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.036+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f85500b2c40 con 0x7f85501f13c0 2024-09-16T00:09:01.042 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:01.043 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:01.052 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:09:01.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.054+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f855014e990 con 0x7f85501f13c0 2024-09-16T00:09:01.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.056+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f855014e990 con 0x7f85501f13c0 2024-09-16T00:09:01.057 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:01.058 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:01.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.057+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8550150650 0x7f8550062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.057+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f8550151ce0 con 0x7f8550150650 2024-09-16T00:09:01.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.057+0000 7f854ffff640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8550150650 0x7f8550062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.057+0000 7f854ffff640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8550150650 0x7f8550062630 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:01.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.058+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8550151ce0 con 0x7f8550150650 2024-09-16T00:09:01.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.075+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f85501599d0 con 0x7f8550150650 2024-09-16T00:09:01.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.076+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f85501599d0 con 0x7f8550150650 2024-09-16T00:09:01.077 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:01.078 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:01.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.077+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8550063080 0x7f85500ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.077+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f85500634a0 con 0x7f8550063080 2024-09-16T00:09:01.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.077+0000 7f855542a640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8550063080 0x7f85500ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.077+0000 7f855542a640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8550063080 0x7f85500ce0b0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:01.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.078+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f85500634a0 con 0x7f8550063080 2024-09-16T00:09:01.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.095+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f85500634a0 con 0x7f8550063080 2024-09-16T00:09:01.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.096+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f85500634a0 con 0x7f8550063080 2024-09-16T00:09:01.097 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:01.098 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:01.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.096+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f85500ceb00 0x7f855002c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.096+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f85500cef90 con 0x7f85500ceb00 2024-09-16T00:09:01.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.097+0000 7f8554c29640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f85500ceb00 0x7f855002c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.097+0000 7f8554c29640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f85500ceb00 0x7f855002c640 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:01.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.098+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f85500cef90 con 0x7f85500ceb00 2024-09-16T00:09:01.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.116+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f85500cef90 con 0x7f85500ceb00 2024-09-16T00:09:01.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.118+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f85500cef90 con 0x7f85500ceb00 2024-09-16T00:09:01.119 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:01.119 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:01.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.118+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f855002d090 0x7f855002d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.118+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f855002dad0 con 0x7f855002d090 2024-09-16T00:09:01.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.118+0000 7f854ffff640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f855002d090 0x7f855002d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.119+0000 7f854ffff640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f855002d090 0x7f855002d510 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:01.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.119+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f855002dad0 con 0x7f855002d090 2024-09-16T00:09:01.133 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:01.133+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:01.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.151+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f855002dad0 con 0x7f855002d090 2024-09-16T00:09:01.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.152+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f855002dad0 con 0x7f855002d090 2024-09-16T00:09:01.153 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:01.154 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:01.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.153+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f85502075c0 0x7f85502099b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.153+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f855020c9f0 con 0x7f85502075c0 2024-09-16T00:09:01.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.153+0000 7f855542a640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f85502075c0 0x7f85502099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.153+0000 7f855542a640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f85502075c0 0x7f85502099b0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:01.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.154+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f855020c9f0 con 0x7f85502075c0 2024-09-16T00:09:01.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.171+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f85500755b0 con 0x7f85502075c0 2024-09-16T00:09:01.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.173+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f85500755b0 con 0x7f85502075c0 2024-09-16T00:09:01.174 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:01.175 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:09:01.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.173+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f855020bc50 0x7f855020c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.173+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f855020fe10 con 0x7f855020bc50 2024-09-16T00:09:01.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.174+0000 7f8554c29640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f855020bc50 0x7f855020c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.174+0000 7f8554c29640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f855020bc50 0x7f855020c0b0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:01.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.175+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f855020fe10 con 0x7f855020bc50 2024-09-16T00:09:01.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.206+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f855020a000 con 0x7f855020bc50 2024-09-16T00:09:01.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.207+0000 7f854d7fa640 1 -- 172.21.15.139:0/3214667393 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f855020a000 con 0x7f855020bc50 2024-09-16T00:09:01.208 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:01.210 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:01.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f855014ac70 msgr2=0x7f8550154450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f855014ac70 0x7f8550154450 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f85501f13c0 msgr2=0x7f855014fc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f85501f13c0 0x7f855014fc00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8550150650 msgr2=0x7f8550062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8550150650 0x7f8550062630 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f855542a640 1 -- 172.21.15.139:0/3214667393 reap_dead start 2024-09-16T00:09:01.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8550063080 msgr2=0x7f85500ce0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8550063080 0x7f85500ce0b0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f85500ceb00 msgr2=0x7f855002c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f85500ceb00 0x7f855002c640 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f855002d090 msgr2=0x7f855002d510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f855002d090 0x7f855002d510 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f85502075c0 msgr2=0x7f85502099b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f85502075c0 0x7f85502099b0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f855020bc50 msgr2=0x7f855020c0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f855020bc50 0x7f855020c0b0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f855542a640 1 -- 172.21.15.139:0/3214667393 reap_dead start 2024-09-16T00:09:01.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f853406c330 msgr2=0x7f853406e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f853406c330 0x7f853406e7f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8544009c00 tx=0x7f8544009190 comp rx=0 tx=0).stop 2024-09-16T00:09:01.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f85500d0490 msgr2=0x7f85501f0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f85500d0490 0x7f85501f0e80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f853c031000 tx=0x7f853c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 shutdown_connections 2024-09-16T00:09:01.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f853406c330 0x7f853406e7f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 --2- 172.21.15.139:0/3214667393 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f85500d0490 0x7f85501f0e80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 >> 172.21.15.139:0/3214667393 conn(0x7f855006c520 msgr2=0x7f855014a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:01.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 shutdown_connections 2024-09-16T00:09:01.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.208+0000 7f8556e8d640 1 -- 172.21.15.139:0/3214667393 wait complete. 2024-09-16T00:09:01.216 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:01.217 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:01.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:01.217 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:01.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.217 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:01.217 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:01.217 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.132893338000002, 2024-09-16T00:09:01.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:09:01.217 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.217 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.218 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.218 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.218 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.218 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.218 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:09:01.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.218 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:01.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:01.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:09:01.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:09:01.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:09:01.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:09:01.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:09:01.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:09:01.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:09:01.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:09:01.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:09:01.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:09:01.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:09:01.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:09:01.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:09:01.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:09:01.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:09:01.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:09:01.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:09:01.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:09:01.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:09:01.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:09:01.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:09:01.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:09:01.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:09:01.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:09:01.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:09:01.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:09:01.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:09:01.228 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.228 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.228 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.228 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:01.228 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:09:01.228 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.053361707999997, 2024-09-16T00:09:01.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093071190000000008, 2024-09-16T00:09:01.228 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.229 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.229 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.229 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.229 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.229 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.229 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 106 2024-09-16T00:09:01.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.229 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:09:01.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:09:01.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160362+0000", 2024-09-16T00:09:01.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:09:01.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160438+0000", 2024-09-16T00:09:01.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6711000000000005e-05 2024-09-16T00:09:01.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160444+0000", 2024-09-16T00:09:01.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0530000000000003e-06 2024-09-16T00:09:01.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160449+0000", 2024-09-16T00:09:01.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2189999999999998e-06 2024-09-16T00:09:01.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161270+0000", 2024-09-16T00:09:01.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082099199999999999 2024-09-16T00:09:01.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161348+0000", 2024-09-16T00:09:01.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8361000000000004e-05 2024-09-16T00:09:01.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161472+0000", 2024-09-16T00:09:01.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123523 2024-09-16T00:09:01.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163461+0000", 2024-09-16T00:09:01.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019891560000000002 2024-09-16T00:09:01.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169418+0000", 2024-09-16T00:09:01.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059568490000000002 2024-09-16T00:09:01.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169570+0000", 2024-09-16T00:09:01.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152461 2024-09-16T00:09:01.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169668+0000", 2024-09-16T00:09:01.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T00:09:01.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.241 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.241 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:01.242 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:09:01.242 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.053265937000006, 2024-09-16T00:09:01.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092931679999999992, 2024-09-16T00:09:01.242 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.242 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.242 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.242 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.242 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.242 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.242 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107 2024-09-16T00:09:01.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.243 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:09:01.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:09:01.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:09:01.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0699999999999998e-07 2024-09-16T00:09:01.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160459+0000", 2024-09-16T00:09:01.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0370000000000001e-06 2024-09-16T00:09:01.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160463+0000", 2024-09-16T00:09:01.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999999e-06 2024-09-16T00:09:01.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160466+0000", 2024-09-16T00:09:01.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.644e-06 2024-09-16T00:09:01.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161873+0000", 2024-09-16T00:09:01.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014074840000000001 2024-09-16T00:09:01.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161949+0000", 2024-09-16T00:09:01.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5749999999999998e-05 2024-09-16T00:09:01.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162069+0000", 2024-09-16T00:09:01.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120257 2024-09-16T00:09:01.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163506+0000", 2024-09-16T00:09:01.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001436344 2024-09-16T00:09:01.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169704+0000", 2024-09-16T00:09:01.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061982829999999997 2024-09-16T00:09:01.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169730+0000", 2024-09-16T00:09:01.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6060000000000001e-05 2024-09-16T00:09:01.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169750+0000", 2024-09-16T00:09:01.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0092e-05 2024-09-16T00:09:01.250 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.250 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.250 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.250 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:01.251 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:09:01.251 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.053156625, 2024-09-16T00:09:01.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092280039999999997, 2024-09-16T00:09:01.251 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.251 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.251 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.251 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.251 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 108 2024-09-16T00:09:01.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.252 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:09:01.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:09:01.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160567+0000", 2024-09-16T00:09:01.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7300000000000001e-07 2024-09-16T00:09:01.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160569+0000", 2024-09-16T00:09:01.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5299999999999999e-06 2024-09-16T00:09:01.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160574+0000", 2024-09-16T00:09:01.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-16T00:09:01.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160577+0000", 2024-09-16T00:09:01.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.01e-06 2024-09-16T00:09:01.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162518+0000", 2024-09-16T00:09:01.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001940703 2024-09-16T00:09:01.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162571+0000", 2024-09-16T00:09:01.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2670000000000002e-05 2024-09-16T00:09:01.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162647+0000", 2024-09-16T00:09:01.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6778000000000002e-05 2024-09-16T00:09:01.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163544+0000", 2024-09-16T00:09:01.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089702599999999999 2024-09-16T00:09:01.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169754+0000", 2024-09-16T00:09:01.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062091739999999996 2024-09-16T00:09:01.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169776+0000", 2024-09-16T00:09:01.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2391999999999998e-05 2024-09-16T00:09:01.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169794+0000", 2024-09-16T00:09:01.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8318999999999999e-05 2024-09-16T00:09:01.258 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.258 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.258 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.258 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:109 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:01.258 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:09:01.259 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.053108074999997, 2024-09-16T00:09:01.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092220679999999999, 2024-09-16T00:09:01.259 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.259 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.259 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.259 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.259 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.259 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.259 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 109 2024-09-16T00:09:01.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.259 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:09:01.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:09:01.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160615+0000", 2024-09-16T00:09:01.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9900000000000001e-07 2024-09-16T00:09:01.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160707+0000", 2024-09-16T00:09:01.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1621999999999998e-05 2024-09-16T00:09:01.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160713+0000", 2024-09-16T00:09:01.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.596e-06 2024-09-16T00:09:01.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160717+0000", 2024-09-16T00:09:01.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5829999999999998e-06 2024-09-16T00:09:01.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162869+0000", 2024-09-16T00:09:01.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021516310000000002 2024-09-16T00:09:01.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162946+0000", 2024-09-16T00:09:01.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7696999999999998e-05 2024-09-16T00:09:01.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163065+0000", 2024-09-16T00:09:01.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118581 2024-09-16T00:09:01.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164431+0000", 2024-09-16T00:09:01.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001366003 2024-09-16T00:09:01.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169797+0000", 2024-09-16T00:09:01.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005366244 2024-09-16T00:09:01.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169819+0000", 2024-09-16T00:09:01.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2036999999999999e-05 2024-09-16T00:09:01.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169837+0000", 2024-09-16T00:09:01.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7575e-05 2024-09-16T00:09:01.266 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.266 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.266 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.266 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:01.266 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:09:01.266 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.052998725999998, 2024-09-16T00:09:01.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092871919999999997, 2024-09-16T00:09:01.267 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.267 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.267 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.267 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.267 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.267 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.267 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 110 2024-09-16T00:09:01.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.268 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:09:01.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:09:01.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:09:01.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:09:01.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160728+0000", 2024-09-16T00:09:01.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0620000000000001e-06 2024-09-16T00:09:01.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160731+0000", 2024-09-16T00:09:01.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6500000000000002e-06 2024-09-16T00:09:01.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160734+0000", 2024-09-16T00:09:01.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.762e-06 2024-09-16T00:09:01.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163580+0000", 2024-09-16T00:09:01.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028464549999999999 2024-09-16T00:09:01.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163657+0000", 2024-09-16T00:09:01.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6695999999999999e-05 2024-09-16T00:09:01.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163798+0000", 2024-09-16T00:09:01.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014055 2024-09-16T00:09:01.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165483+0000", 2024-09-16T00:09:01.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016853090000000001 2024-09-16T00:09:01.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169970+0000", 2024-09-16T00:09:01.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00448709 2024-09-16T00:09:01.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169994+0000", 2024-09-16T00:09:01.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3575999999999999e-05 2024-09-16T00:09:01.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170011+0000", 2024-09-16T00:09:01.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7702000000000001e-05 2024-09-16T00:09:01.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.274 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.274 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:01.274 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:09:01.275 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.052985057000001, 2024-09-16T00:09:01.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093174859999999998, 2024-09-16T00:09:01.275 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.275 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.275 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.275 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.275 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 111 2024-09-16T00:09:01.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.275 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:09:01.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:09:01.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:09:01.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1e-07 2024-09-16T00:09:01.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160741+0000", 2024-09-16T00:09:01.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6110000000000001e-06 2024-09-16T00:09:01.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160744+0000", 2024-09-16T00:09:01.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3299999999999999e-06 2024-09-16T00:09:01.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160746+0000", 2024-09-16T00:09:01.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5469999999999999e-06 2024-09-16T00:09:01.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164069+0000", 2024-09-16T00:09:01.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033222709999999999 2024-09-16T00:09:01.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164113+0000", 2024-09-16T00:09:01.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4212999999999998e-05 2024-09-16T00:09:01.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164187+0000", 2024-09-16T00:09:01.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4136999999999998e-05 2024-09-16T00:09:01.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166429+0000", 2024-09-16T00:09:01.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022423180000000001 2024-09-16T00:09:01.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170016+0000", 2024-09-16T00:09:01.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035869449999999998 2024-09-16T00:09:01.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170038+0000", 2024-09-16T00:09:01.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2053e-05 2024-09-16T00:09:01.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170055+0000", 2024-09-16T00:09:01.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6850999999999998e-05 2024-09-16T00:09:01.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.283 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.284 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:01.284 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:09:01.284 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.052971342000006, 2024-09-16T00:09:01.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093480279999999995, 2024-09-16T00:09:01.284 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.284 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.284 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.284 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.285 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 112 2024-09-16T00:09:01.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.285 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:09:01.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:09:01.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160752+0000", 2024-09-16T00:09:01.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2700000000000001e-07 2024-09-16T00:09:01.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160808+0000", 2024-09-16T00:09:01.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5797999999999997e-05 2024-09-16T00:09:01.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160814+0000", 2024-09-16T00:09:01.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5980000000000002e-06 2024-09-16T00:09:01.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160819+0000", 2024-09-16T00:09:01.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8960000000000002e-06 2024-09-16T00:09:01.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164468+0000", 2024-09-16T00:09:01.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003649476 2024-09-16T00:09:01.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164525+0000", 2024-09-16T00:09:01.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6227999999999998e-05 2024-09-16T00:09:01.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164595+0000", 2024-09-16T00:09:01.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759000000000005e-05 2024-09-16T00:09:01.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167237+0000", 2024-09-16T00:09:01.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026410470000000001 2024-09-16T00:09:01.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170060+0000", 2024-09-16T00:09:01.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028234969999999999 2024-09-16T00:09:01.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170081+0000", 2024-09-16T00:09:01.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1242000000000001e-05 2024-09-16T00:09:01.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170100+0000", 2024-09-16T00:09:01.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8260000000000001e-05 2024-09-16T00:09:01.293 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.293 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.293 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.293 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:01.293 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:09:01.293 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.052894246999998, 2024-09-16T00:09:01.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093139409999999992, 2024-09-16T00:09:01.294 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.294 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.294 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.294 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.294 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 113 2024-09-16T00:09:01.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.294 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:09:01.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:09:01.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:09:01.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-07 2024-09-16T00:09:01.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160833+0000", 2024-09-16T00:09:01.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.557e-06 2024-09-16T00:09:01.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160839+0000", 2024-09-16T00:09:01.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3829999999999997e-06 2024-09-16T00:09:01.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160844+0000", 2024-09-16T00:09:01.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6909999999999996e-06 2024-09-16T00:09:01.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165008+0000", 2024-09-16T00:09:01.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041642129999999999 2024-09-16T00:09:01.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165085+0000", 2024-09-16T00:09:01.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6651999999999997e-05 2024-09-16T00:09:01.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165205+0000", 2024-09-16T00:09:01.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012045900000000001 2024-09-16T00:09:01.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167283+0000", 2024-09-16T00:09:01.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020782909999999999 2024-09-16T00:09:01.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170104+0000", 2024-09-16T00:09:01.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028207950000000001 2024-09-16T00:09:01.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170125+0000", 2024-09-16T00:09:01.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0852e-05 2024-09-16T00:09:01.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170143+0000", 2024-09-16T00:09:01.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7467000000000001e-05 2024-09-16T00:09:01.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:01.301 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:09:01.302 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.052847764999996, 2024-09-16T00:09:01.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093126960000000005, 2024-09-16T00:09:01.302 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.302 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.302 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.302 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-16T00:09:01.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.303 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:09:01.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:09:01.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160876+0000", 2024-09-16T00:09:01.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5700000000000002e-07 2024-09-16T00:09:01.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160880+0000", 2024-09-16T00:09:01.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5539999999999999e-06 2024-09-16T00:09:01.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160888+0000", 2024-09-16T00:09:01.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.157e-06 2024-09-16T00:09:01.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160894+0000", 2024-09-16T00:09:01.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7690000000000002e-06 2024-09-16T00:09:01.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165519+0000", 2024-09-16T00:09:01.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046249539999999997 2024-09-16T00:09:01.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165593+0000", 2024-09-16T00:09:01.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4039e-05 2024-09-16T00:09:01.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165715+0000", 2024-09-16T00:09:01.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012131899999999999 2024-09-16T00:09:01.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167309+0000", 2024-09-16T00:09:01.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001594011 2024-09-16T00:09:01.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170147+0000", 2024-09-16T00:09:01.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028387130000000001 2024-09-16T00:09:01.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170170+0000", 2024-09-16T00:09:01.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2517000000000001e-05 2024-09-16T00:09:01.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170188+0000", 2024-09-16T00:09:01.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8006e-05 2024-09-16T00:09:01.309 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.309 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.309 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.309 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:01.310 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:09:01.310 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.052806736999997, 2024-09-16T00:09:01.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093140040000000007, 2024-09-16T00:09:01.310 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.310 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.310 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.310 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.310 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-16T00:09:01.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.311 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:09:01.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:09:01.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160917+0000", 2024-09-16T00:09:01.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7599999999999996e-07 2024-09-16T00:09:01.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160992+0000", 2024-09-16T00:09:01.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4760999999999998e-05 2024-09-16T00:09:01.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161000+0000", 2024-09-16T00:09:01.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9829999999999996e-06 2024-09-16T00:09:01.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161007+0000", 2024-09-16T00:09:01.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4909999999999999e-06 2024-09-16T00:09:01.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165983+0000", 2024-09-16T00:09:01.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049755399999999997 2024-09-16T00:09:01.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166027+0000", 2024-09-16T00:09:01.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4109000000000001e-05 2024-09-16T00:09:01.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166100+0000", 2024-09-16T00:09:01.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3360000000000002e-05 2024-09-16T00:09:01.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169698+0000", 2024-09-16T00:09:01.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597795 2024-09-16T00:09:01.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170192+0000", 2024-09-16T00:09:01.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049447300000000005 2024-09-16T00:09:01.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170214+0000", 2024-09-16T00:09:01.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1546e-05 2024-09-16T00:09:01.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170230+0000", 2024-09-16T00:09:01.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6169999999999999e-05 2024-09-16T00:09:01.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.317 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.317 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:01.317 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:09:01.317 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.052697007999996, 2024-09-16T00:09:01.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092457029999999992, 2024-09-16T00:09:01.318 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.318 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.318 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.318 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-16T00:09:01.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.318 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:09:01.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:09:01.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:09:01.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7899999999999999e-07 2024-09-16T00:09:01.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161029+0000", 2024-09-16T00:09:01.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6630000000000002e-06 2024-09-16T00:09:01.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161036+0000", 2024-09-16T00:09:01.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T00:09:01.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161041+0000", 2024-09-16T00:09:01.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:09:01.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166468+0000", 2024-09-16T00:09:01.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054266050000000001 2024-09-16T00:09:01.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166511+0000", 2024-09-16T00:09:01.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3869999999999998e-05 2024-09-16T00:09:01.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166606+0000", 2024-09-16T00:09:01.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4437000000000004e-05 2024-09-16T00:09:01.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169955+0000", 2024-09-16T00:09:01.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033494810000000001 2024-09-16T00:09:01.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170234+0000", 2024-09-16T00:09:01.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027900899999999998 2024-09-16T00:09:01.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170255+0000", 2024-09-16T00:09:01.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0537999999999999e-05 2024-09-16T00:09:01.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170272+0000", 2024-09-16T00:09:01.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6611000000000001e-05 2024-09-16T00:09:01.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.324 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.324 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:01.325 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:09:01.325 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.052671863, 2024-09-16T00:09:01.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093511010000000006, 2024-09-16T00:09:01.325 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.325 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.325 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.325 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-16T00:09:01.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.326 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:09:01.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:09:01.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:09:01.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.63e-07 2024-09-16T00:09:01.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161055+0000", 2024-09-16T00:09:01.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6770000000000001e-06 2024-09-16T00:09:01.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161063+0000", 2024-09-16T00:09:01.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6469999999999998e-06 2024-09-16T00:09:01.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161068+0000", 2024-09-16T00:09:01.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.746e-06 2024-09-16T00:09:01.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166817+0000", 2024-09-16T00:09:01.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057494959999999998 2024-09-16T00:09:01.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166893+0000", 2024-09-16T00:09:01.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6210000000000004e-05 2024-09-16T00:09:01.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167019+0000", 2024-09-16T00:09:01.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125585 2024-09-16T00:09:01.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170275+0000", 2024-09-16T00:09:01.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032563620000000001 2024-09-16T00:09:01.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170309+0000", 2024-09-16T00:09:01.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3865999999999997e-05 2024-09-16T00:09:01.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170362+0000", 2024-09-16T00:09:01.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3094999999999999e-05 2024-09-16T00:09:01.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170402+0000", 2024-09-16T00:09:01.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9953999999999997e-05 2024-09-16T00:09:01.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.332 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.332 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:01.332 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:09:01.333 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 96.603939777999997, 2024-09-16T00:09:01.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097364389999999995, 2024-09-16T00:09:01.333 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.333 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.333 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.333 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138 2024-09-16T00:09:01.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.334 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:09:01.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:09:01.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609784+0000", 2024-09-16T00:09:01.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T00:09:01.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609787+0000", 2024-09-16T00:09:01.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280000000000001e-06 2024-09-16T00:09:01.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609794+0000", 2024-09-16T00:09:01.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6409999999999996e-06 2024-09-16T00:09:01.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609799+0000", 2024-09-16T00:09:01.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9359999999999998e-06 2024-09-16T00:09:01.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615759+0000", 2024-09-16T00:09:01.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059599910000000004 2024-09-16T00:09:01.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615806+0000", 2024-09-16T00:09:01.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7290000000000003e-05 2024-09-16T00:09:01.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615877+0000", 2024-09-16T00:09:01.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1000000000000005e-05 2024-09-16T00:09:01.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618961+0000", 2024-09-16T00:09:01.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030841390000000001 2024-09-16T00:09:01.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619436+0000", 2024-09-16T00:09:01.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047452799999999999 2024-09-16T00:09:01.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619482+0000", 2024-09-16T00:09:01.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5689000000000003e-05 2024-09-16T00:09:01.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619519+0000", 2024-09-16T00:09:01.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7938000000000002e-05 2024-09-16T00:09:01.340 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.340 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.340 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.340 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:01.340 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:09:01.340 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 96.603916064000003, 2024-09-16T00:09:01.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011812954000000001, 2024-09-16T00:09:01.341 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.341 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.341 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.341 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139 2024-09-16T00:09:01.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.341 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:09:01.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:09:01.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:09:01.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999998e-07 2024-09-16T00:09:01.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609810+0000", 2024-09-16T00:09:01.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2100000000000002e-06 2024-09-16T00:09:01.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609817+0000", 2024-09-16T00:09:01.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.28e-06 2024-09-16T00:09:01.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609825+0000", 2024-09-16T00:09:01.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:09:01.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616116+0000", 2024-09-16T00:09:01.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006290543 2024-09-16T00:09:01.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616187+0000", 2024-09-16T00:09:01.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1830999999999997e-05 2024-09-16T00:09:01.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616298+0000", 2024-09-16T00:09:01.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011044699999999999 2024-09-16T00:09:01.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619559+0000", 2024-09-16T00:09:01.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032614710000000002 2024-09-16T00:09:01.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621562+0000", 2024-09-16T00:09:01.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020028680000000001 2024-09-16T00:09:01.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621597+0000", 2024-09-16T00:09:01.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4625000000000001e-05 2024-09-16T00:09:01.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621620+0000", 2024-09-16T00:09:01.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2864e-05 2024-09-16T00:09:01.348 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.348 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.348 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.349 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:01.349 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:09:01.349 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 96.603888268000006, 2024-09-16T00:09:01.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011828211999999999, 2024-09-16T00:09:01.349 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.349 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.349 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.350 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.350 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.350 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.350 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 140 2024-09-16T00:09:01.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.350 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:09:01.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:09:01.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:09:01.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T00:09:01.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:01.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961000000000003e-05 2024-09-16T00:09:01.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609892+0000", 2024-09-16T00:09:01.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0516000000000001e-05 2024-09-16T00:09:01.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609898+0000", 2024-09-16T00:09:01.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0449999999999997e-06 2024-09-16T00:09:01.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616797+0000", 2024-09-16T00:09:01.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068998840000000002 2024-09-16T00:09:01.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616844+0000", 2024-09-16T00:09:01.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6501000000000001e-05 2024-09-16T00:09:01.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616935+0000", 2024-09-16T00:09:01.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1222000000000002e-05 2024-09-16T00:09:01.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619808+0000", 2024-09-16T00:09:01.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873198 2024-09-16T00:09:01.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621624+0000", 2024-09-16T00:09:01.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001815933 2024-09-16T00:09:01.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621646+0000", 2024-09-16T00:09:01.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1622999999999999e-05 2024-09-16T00:09:01.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621663+0000", 2024-09-16T00:09:01.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6854000000000002e-05 2024-09-16T00:09:01.357 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.358 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.358 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.358 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:01.358 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:01.358 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 96.603802458999994, 2024-09-16T00:09:01.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014331501999999999, 2024-09-16T00:09:01.358 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.359 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.359 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.359 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.359 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.359 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.359 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-16T00:09:01.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.359 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:01.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:01.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609921+0000", 2024-09-16T00:09:01.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9599999999999999e-07 2024-09-16T00:09:01.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609925+0000", 2024-09-16T00:09:01.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T00:09:01.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609932+0000", 2024-09-16T00:09:01.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8240000000000002e-06 2024-09-16T00:09:01.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609937+0000", 2024-09-16T00:09:01.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1070000000000004e-06 2024-09-16T00:09:01.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618987+0000", 2024-09-16T00:09:01.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090502719999999998 2024-09-16T00:09:01.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619064+0000", 2024-09-16T00:09:01.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6868999999999999e-05 2024-09-16T00:09:01.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619183+0000", 2024-09-16T00:09:01.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011961 2024-09-16T00:09:01.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.620052+0000", 2024-09-16T00:09:01.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086822799999999997 2024-09-16T00:09:01.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624197+0000", 2024-09-16T00:09:01.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041454930000000001 2024-09-16T00:09:01.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624229+0000", 2024-09-16T00:09:01.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1758999999999999e-05 2024-09-16T00:09:01.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624252+0000", 2024-09-16T00:09:01.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:09:01.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.366 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.367 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:01.367 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:09:01.367 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 96.131036651000002, 2024-09-16T00:09:01.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010093276999999999, 2024-09-16T00:09:01.367 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.367 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.367 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.367 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.367 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 157 2024-09-16T00:09:01.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.368 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:09:01.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:09:01.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:09:01.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4699999999999998e-07 2024-09-16T00:09:01.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082688+0000", 2024-09-16T00:09:01.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0209999999999998e-06 2024-09-16T00:09:01.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082692+0000", 2024-09-16T00:09:01.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6890000000000002e-06 2024-09-16T00:09:01.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082695+0000", 2024-09-16T00:09:01.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5339999999999998e-06 2024-09-16T00:09:01.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085303+0000", 2024-09-16T00:09:01.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026084789999999999 2024-09-16T00:09:01.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085374+0000", 2024-09-16T00:09:01.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1317999999999997e-05 2024-09-16T00:09:01.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085506+0000", 2024-09-16T00:09:01.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013118 2024-09-16T00:09:01.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086928+0000", 2024-09-16T00:09:01.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014224559999999999 2024-09-16T00:09:01.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092665+0000", 2024-09-16T00:09:01.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057369830000000002 2024-09-16T00:09:01.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092727+0000", 2024-09-16T00:09:01.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1810000000000006e-05 2024-09-16T00:09:01.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092779+0000", 2024-09-16T00:09:01.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2559999999999998e-05 2024-09-16T00:09:01.374 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.374 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.374 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.375 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:01.375 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:09:01.375 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 96.131024483000004, 2024-09-16T00:09:01.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010176388, 2024-09-16T00:09:01.375 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.375 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.375 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.375 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.375 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.375 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.375 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 158 2024-09-16T00:09:01.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.376 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:09:01.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:09:01.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082699+0000", 2024-09-16T00:09:01.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.11e-07 2024-09-16T00:09:01.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082742+0000", 2024-09-16T00:09:01.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3699000000000003e-05 2024-09-16T00:09:01.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082748+0000", 2024-09-16T00:09:01.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8690000000000001e-06 2024-09-16T00:09:01.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082751+0000", 2024-09-16T00:09:01.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.942e-06 2024-09-16T00:09:01.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085881+0000", 2024-09-16T00:09:01.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031303450000000001 2024-09-16T00:09:01.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085959+0000", 2024-09-16T00:09:01.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7874999999999996e-05 2024-09-16T00:09:01.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086086+0000", 2024-09-16T00:09:01.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012685499999999999 2024-09-16T00:09:01.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087127+0000", 2024-09-16T00:09:01.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001041298 2024-09-16T00:09:01.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092790+0000", 2024-09-16T00:09:01.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056628 2024-09-16T00:09:01.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092832+0000", 2024-09-16T00:09:01.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2227000000000001e-05 2024-09-16T00:09:01.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092875+0000", 2024-09-16T00:09:01.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2267e-05 2024-09-16T00:09:01.382 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.382 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.382 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.382 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:01.383 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:09:01.383 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 96.130966444999999, 2024-09-16T00:09:01.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010199547, 2024-09-16T00:09:01.383 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:01.383 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:01.383 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:01.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:01.383 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:01.383 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159 2024-09-16T00:09:01.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.384 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:01.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:01.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:09:01.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:01.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:09:01.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:01.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:01.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082757+0000", 2024-09-16T00:09:01.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2500000000000001e-07 2024-09-16T00:09:01.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:01.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082759+0000", 2024-09-16T00:09:01.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-16T00:09:01.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:01.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082763+0000", 2024-09-16T00:09:01.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4889999999999997e-06 2024-09-16T00:09:01.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082766+0000", 2024-09-16T00:09:01.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:09:01.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:01.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086571+0000", 2024-09-16T00:09:01.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038053269999999998 2024-09-16T00:09:01.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:01.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086619+0000", 2024-09-16T00:09:01.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7924999999999999e-05 2024-09-16T00:09:01.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:01.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086693+0000", 2024-09-16T00:09:01.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000005e-05 2024-09-16T00:09:01.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:01.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087383+0000", 2024-09-16T00:09:01.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069008299999999997 2024-09-16T00:09:01.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:01.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092882+0000", 2024-09-16T00:09:01.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054987450000000002 2024-09-16T00:09:01.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:01.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092921+0000", 2024-09-16T00:09:01.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9388999999999998e-05 2024-09-16T00:09:01.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:01.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:01.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:01.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092956+0000", 2024-09-16T00:09:01.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4647999999999998e-05 2024-09-16T00:09:01.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.391 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.392 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:01.392 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:01.392 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:01.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.227+0000 7f4c58680640 1 -- 172.21.15.139:0/4285820389 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f4c500d0c20 con 0x7f4c5006c110 2024-09-16T00:09:01.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.228+0000 7f4c46ffd640 1 -- 172.21.15.139:0/4285820389 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v61) ==== 81+0+35280 (secure 0 0 0) 0x7f4c4c05da00 con 0x7f4c5006c110 2024-09-16T00:09:01.396 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:09:01.396 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:09:01.397 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:01.397 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 61, 2024-09-16T00:09:01.397 INFO:teuthology.orchestra.run.smithi139.stdout: "fsid": "2d3366e5-6a59-4b59-93b2-615e72ac7015", 2024-09-16T00:09:01.397 INFO:teuthology.orchestra.run.smithi139.stdout: "created": "2024-09-16T00:07:09.967833+0000", 2024-09-16T00:09:01.397 INFO:teuthology.orchestra.run.smithi139.stdout: "modified": "2024-09-16T00:08:59.995558+0000", 2024-09-16T00:09:01.397 INFO:teuthology.orchestra.run.smithi139.stdout: "last_up_change": "2024-09-16T00:07:21.049678+0000", 2024-09-16T00:09:01.397 INFO:teuthology.orchestra.run.smithi139.stdout: "last_in_change": "2024-09-16T00:07:12.025620+0000", 2024-09-16T00:09:01.397 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T00:09:01.397 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_num": 5804032, 2024-09-16T00:09:01.397 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_set": [ 2024-09-16T00:09:01.397 INFO:teuthology.orchestra.run.smithi139.stdout: "nodeep-scrub", 2024-09-16T00:09:01.398 INFO:teuthology.orchestra.run.smithi139.stdout: "pglog_hardlimit", 2024-09-16T00:09:01.398 INFO:teuthology.orchestra.run.smithi139.stdout: "purged_snapdirs", 2024-09-16T00:09:01.398 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_deletes", 2024-09-16T00:09:01.398 INFO:teuthology.orchestra.run.smithi139.stdout: "sortbitwise" 2024-09-16T00:09:01.398 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:01.398 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_version": 6, 2024-09-16T00:09:01.398 INFO:teuthology.orchestra.run.smithi139.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T00:09:01.398 INFO:teuthology.orchestra.run.smithi139.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T00:09:01.398 INFO:teuthology.orchestra.run.smithi139.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T00:09:01.398 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_snapshot": "", 2024-09-16T00:09:01.398 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_max": 3, 2024-09-16T00:09:01.399 INFO:teuthology.orchestra.run.smithi139.stdout: "max_osd": 8, 2024-09-16T00:09:01.399 INFO:teuthology.orchestra.run.smithi139.stdout: "require_min_compat_client": "luminous", 2024-09-16T00:09:01.399 INFO:teuthology.orchestra.run.smithi139.stdout: "min_compat_client": "luminous", 2024-09-16T00:09:01.399 INFO:teuthology.orchestra.run.smithi139.stdout: "require_osd_release": "squid", 2024-09-16T00:09:01.399 INFO:teuthology.orchestra.run.smithi139.stdout: "allow_crimson": false, 2024-09-16T00:09:01.399 INFO:teuthology.orchestra.run.smithi139.stdout: "pools": [ 2024-09-16T00:09:01.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.399 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 1, 2024-09-16T00:09:01.399 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": ".mgr", 2024-09-16T00:09:01.399 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:22.056303+0000", 2024-09-16T00:09:01.399 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:09:01.400 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:09:01.400 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:09:01.400 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:09:01.400 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:09:01.400 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:09:01.400 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:09:01.400 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:09:01.400 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:09:01.400 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:09:01.400 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:09:01.401 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:09:01.401 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:09:01.401 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 1, 2024-09-16T00:09:01.401 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 1, 2024-09-16T00:09:01.401 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 1, 2024-09-16T00:09:01.401 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 1, 2024-09-16T00:09:01.401 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 1, 2024-09-16T00:09:01.401 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:09:01.401 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:09:01.401 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:09:01.401 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:09:01.402 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:09:01.402 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:09:01.402 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:09:01.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.402 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "18", 2024-09-16T00:09:01.402 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:09:01.402 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:09:01.402 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:09:01.402 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:09:01.402 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:09:01.402 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:09:01.403 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:09:01.403 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:09:01.403 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:09:01.403 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:09:01.403 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:09:01.403 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:09:01.403 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:09:01.403 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:09:01.403 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:09:01.403 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:09:01.403 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:09:01.404 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:09:01.404 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:09:01.404 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:09:01.404 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:09:01.404 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:09:01.404 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:09:01.404 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:09:01.404 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:09:01.404 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:09:01.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.404 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:09:01.405 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:09:01.405 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:09:01.405 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:09:01.405 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:09:01.405 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:09:01.405 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:09:01.405 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:09:01.405 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:09:01.405 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:09:01.405 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:09:01.405 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:09:01.406 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_max": 32, 2024-09-16T00:09:01.406 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 1 2024-09-16T00:09:01.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.406 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:09:01.406 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": {} 2024-09-16T00:09:01.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.406 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:09:01.406 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:09:01.406 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 8, 2024-09-16T00:09:01.406 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 8, 2024-09-16T00:09:01.406 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 0.25, 2024-09-16T00:09:01.407 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 2, 2024-09-16T00:09:01.407 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 2, 2024-09-16T00:09:01.407 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:09:01.407 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:09:01.407 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:09:01.407 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.407 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 2, 2024-09-16T00:09:01.407 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": "cephfs_metadata", 2024-09-16T00:09:01.407 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:30.768649+0000", 2024-09-16T00:09:01.408 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:09:01.408 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:09:01.408 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:09:01.408 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:09:01.408 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:09:01.408 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:09:01.408 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:09:01.408 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:09:01.408 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:09:01.408 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:09:01.408 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:09:01.409 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:09:01.409 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:09:01.409 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 32, 2024-09-16T00:09:01.409 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 32, 2024-09-16T00:09:01.409 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 32, 2024-09-16T00:09:01.409 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 32, 2024-09-16T00:09:01.409 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 32, 2024-09-16T00:09:01.409 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:09:01.409 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:09:01.409 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:09:01.409 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:09:01.409 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:09:01.410 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:09:01.410 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:09:01.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.410 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "21", 2024-09-16T00:09:01.410 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:09:01.410 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:09:01.410 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:09:01.410 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:09:01.410 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:09:01.410 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:09:01.410 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:09:01.411 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:09:01.411 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:09:01.411 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:09:01.411 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:09:01.411 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:09:01.411 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:09:01.411 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:09:01.411 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:09:01.411 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:09:01.411 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:09:01.411 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:09:01.412 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:09:01.412 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:09:01.412 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:09:01.412 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:09:01.412 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:09:01.412 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:09:01.412 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:09:01.412 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:09:01.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.412 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:09:01.413 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:09:01.413 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:09:01.413 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:09:01.413 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:09:01.413 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:09:01.413 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:09:01.413 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:09:01.413 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:09:01.413 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:09:01.413 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:09:01.413 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:09:01.414 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_bias": 4, 2024-09-16T00:09:01.414 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 16, 2024-09-16T00:09:01.414 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_priority": 5 2024-09-16T00:09:01.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.414 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:09:01.414 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs": { 2024-09-16T00:09:01.414 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": "cephfs" 2024-09-16T00:09:01.414 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.414 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:09:01.414 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:09:01.415 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 1.5, 2024-09-16T00:09:01.415 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 1.5, 2024-09-16T00:09:01.415 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 1, 2024-09-16T00:09:01.415 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 1.5, 2024-09-16T00:09:01.415 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 1.5, 2024-09-16T00:09:01.415 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:09:01.415 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:09:01.415 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:09:01.415 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.416 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 3, 2024-09-16T00:09:01.416 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": "cephfs_data", 2024-09-16T00:09:01.416 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:32.008209+0000", 2024-09-16T00:09:01.416 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:09:01.416 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:09:01.416 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:09:01.416 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:09:01.417 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:09:01.417 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:09:01.417 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:09:01.417 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:09:01.417 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:09:01.417 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:09:01.417 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:09:01.417 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:09:01.417 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:09:01.417 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 64, 2024-09-16T00:09:01.418 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 64, 2024-09-16T00:09:01.418 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 64, 2024-09-16T00:09:01.418 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 64, 2024-09-16T00:09:01.418 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 64, 2024-09-16T00:09:01.418 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:09:01.418 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:09:01.418 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:09:01.419 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:09:01.419 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:09:01.419 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:09:01.419 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:09:01.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.419 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "22", 2024-09-16T00:09:01.419 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:09:01.419 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:09:01.419 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:09:01.419 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:09:01.420 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:09:01.420 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:09:01.420 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:09:01.420 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:09:01.420 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:09:01.420 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:09:01.420 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:09:01.421 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:09:01.421 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:09:01.421 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:09:01.421 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:09:01.421 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:09:01.421 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:09:01.421 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:09:01.421 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:09:01.421 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:09:01.421 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:09:01.422 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:09:01.422 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:09:01.422 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:09:01.422 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:09:01.422 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:09:01.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.422 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:09:01.422 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:09:01.423 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:09:01.423 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:09:01.423 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:09:01.423 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:09:01.423 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:09:01.423 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:09:01.423 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:09:01.423 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:09:01.423 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:09:01.423 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:09:01.424 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 64, 2024-09-16T00:09:01.424 INFO:teuthology.orchestra.run.smithi139.stdout: "target_size_ratio": 0.89999997615814209 2024-09-16T00:09:01.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.424 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:09:01.424 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs": { 2024-09-16T00:09:01.424 INFO:teuthology.orchestra.run.smithi139.stdout: "data": "cephfs" 2024-09-16T00:09:01.424 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.424 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:09:01.424 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:09:01.425 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 1.3799999952316284, 2024-09-16T00:09:01.425 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 1.3799999952316284, 2024-09-16T00:09:01.425 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 1, 2024-09-16T00:09:01.425 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 1.3799999952316284, 2024-09-16T00:09:01.425 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 1.3799999952316284, 2024-09-16T00:09:01.425 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:09:01.425 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:09:01.425 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:09:01.425 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.425 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.426 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:01.426 INFO:teuthology.orchestra.run.smithi139.stdout: "osds": [ 2024-09-16T00:09:01.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.426 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 0, 2024-09-16T00:09:01.426 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "83699a05-4937-43c3-aefc-b669bed9bb6f", 2024-09-16T00:09:01.426 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:01.426 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:01.426 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:01.426 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:01.426 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:01.426 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:01.427 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 14, 2024-09-16T00:09:01.427 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 37, 2024-09-16T00:09:01.427 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:01.427 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:01.427 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:01.427 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.427 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.427 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6816", 2024-09-16T00:09:01.427 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:01.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.428 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.428 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6817", 2024-09-16T00:09:01.428 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:01.428 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.428 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.428 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:01.428 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.429 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.429 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6818", 2024-09-16T00:09:01.429 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:01.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.429 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.429 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6819", 2024-09-16T00:09:01.429 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:01.429 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.429 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.430 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:01.430 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.430 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.430 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6822", 2024-09-16T00:09:01.430 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:01.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.430 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.431 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6823", 2024-09-16T00:09:01.431 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:01.431 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.431 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.431 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:01.431 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.431 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.431 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6820", 2024-09-16T00:09:01.431 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:01.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.432 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.432 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6821", 2024-09-16T00:09:01.432 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:01.432 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.432 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.432 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6817/1974901438", 2024-09-16T00:09:01.432 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6819/1974901438", 2024-09-16T00:09:01.432 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6823/1974901438", 2024-09-16T00:09:01.433 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6821/1974901438", 2024-09-16T00:09:01.433 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:01.433 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:01.433 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:01.433 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.433 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 1, 2024-09-16T00:09:01.433 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "9c04991e-9695-4bbb-8011-bf613bf404b3", 2024-09-16T00:09:01.433 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:01.434 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:01.434 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:01.434 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:01.434 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:01.434 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:01.434 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 14, 2024-09-16T00:09:01.434 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:09:01.434 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:01.434 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:01.434 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:01.434 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.435 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.435 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6808", 2024-09-16T00:09:01.435 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:01.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.435 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.435 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6809", 2024-09-16T00:09:01.435 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:01.435 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.435 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.436 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:01.436 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.436 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.436 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6810", 2024-09-16T00:09:01.436 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:01.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.436 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.437 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6811", 2024-09-16T00:09:01.437 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:01.437 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.437 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.437 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:01.437 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.437 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.437 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6814", 2024-09-16T00:09:01.437 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:01.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.438 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.438 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6815", 2024-09-16T00:09:01.438 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:01.438 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.438 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.438 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:01.438 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.439 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.439 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6812", 2024-09-16T00:09:01.439 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:01.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.439 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.439 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6813", 2024-09-16T00:09:01.439 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:01.439 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.439 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.440 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6809/2232380998", 2024-09-16T00:09:01.440 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6811/2232380998", 2024-09-16T00:09:01.440 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6815/2232380998", 2024-09-16T00:09:01.440 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6813/2232380998", 2024-09-16T00:09:01.440 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:01.440 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:01.440 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:01.440 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.441 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 2, 2024-09-16T00:09:01.441 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "8048fc64-72b3-4866-ac62-6a8b2bf6f5c8", 2024-09-16T00:09:01.441 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:01.441 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:01.441 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:01.441 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:01.441 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:01.441 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:01.441 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 13, 2024-09-16T00:09:01.442 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:09:01.442 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:01.442 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:01.442 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:01.444 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.444 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.445 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6800", 2024-09-16T00:09:01.445 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:01.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.445 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.445 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6801", 2024-09-16T00:09:01.445 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:01.445 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.445 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.446 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:01.446 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.446 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.446 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6802", 2024-09-16T00:09:01.446 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:01.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.447 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.447 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6803", 2024-09-16T00:09:01.447 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:01.447 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.447 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.447 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:01.447 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.447 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.448 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6806", 2024-09-16T00:09:01.448 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:01.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.448 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.448 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6807", 2024-09-16T00:09:01.448 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:01.448 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.449 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.449 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:01.449 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.449 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.449 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6804", 2024-09-16T00:09:01.449 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:01.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.450 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.450 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6805", 2024-09-16T00:09:01.450 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:01.450 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.450 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.450 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6801/4180588905", 2024-09-16T00:09:01.450 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6803/4180588905", 2024-09-16T00:09:01.451 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6807/4180588905", 2024-09-16T00:09:01.451 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6805/4180588905", 2024-09-16T00:09:01.451 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:01.451 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:01.451 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:01.451 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.452 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 3, 2024-09-16T00:09:01.452 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "c34ac77b-354c-4884-902d-3e2a84f5eb27", 2024-09-16T00:09:01.452 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:01.452 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:01.452 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:01.452 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:01.452 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:01.452 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:01.452 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:09:01.452 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:09:01.453 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:01.453 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:01.453 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:01.453 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.453 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.453 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6826", 2024-09-16T00:09:01.454 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:01.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.454 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.454 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6827", 2024-09-16T00:09:01.454 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:01.454 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.454 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.454 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:01.454 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.455 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.455 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6828", 2024-09-16T00:09:01.455 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:01.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.455 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.455 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6829", 2024-09-16T00:09:01.455 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:01.455 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.455 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.456 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:01.456 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.456 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.456 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6832", 2024-09-16T00:09:01.456 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:01.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.456 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.456 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6833", 2024-09-16T00:09:01.457 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:01.457 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.457 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.457 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:01.457 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.457 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.457 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6830", 2024-09-16T00:09:01.457 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:01.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.458 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.458 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6831", 2024-09-16T00:09:01.458 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:01.458 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.458 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.458 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6827/3947768492", 2024-09-16T00:09:01.458 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6829/3947768492", 2024-09-16T00:09:01.458 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6833/3947768492", 2024-09-16T00:09:01.459 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6831/3947768492", 2024-09-16T00:09:01.459 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:01.459 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:01.459 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:01.459 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.459 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 4, 2024-09-16T00:09:01.459 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "ba9a7c93-9672-43e1-8c98-2878cdf7f548", 2024-09-16T00:09:01.459 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:01.460 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:01.460 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:01.460 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:01.460 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:01.460 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:01.460 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:09:01.460 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:09:01.460 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:01.460 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:01.461 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:01.461 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.461 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.461 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6816", 2024-09-16T00:09:01.461 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:01.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.461 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.461 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6817", 2024-09-16T00:09:01.461 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:01.462 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.462 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.462 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:01.462 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.462 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.462 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6818", 2024-09-16T00:09:01.462 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:01.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.463 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.463 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6819", 2024-09-16T00:09:01.463 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:01.463 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.463 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.464 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:01.464 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.464 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.464 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6822", 2024-09-16T00:09:01.464 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:01.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.464 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.465 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6823", 2024-09-16T00:09:01.465 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:01.465 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.465 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.465 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:01.465 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.466 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.466 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6820", 2024-09-16T00:09:01.466 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:01.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.466 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.467 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6821", 2024-09-16T00:09:01.467 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:01.467 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.467 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.467 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6817/642632916", 2024-09-16T00:09:01.467 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6819/642632916", 2024-09-16T00:09:01.468 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6823/642632916", 2024-09-16T00:09:01.468 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6821/642632916", 2024-09-16T00:09:01.468 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:01.468 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:01.468 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:01.468 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.468 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 5, 2024-09-16T00:09:01.469 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "46a0916b-474c-411b-84ad-86f27d4cbf53", 2024-09-16T00:09:01.469 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:01.469 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:01.469 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:01.469 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:01.469 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:01.469 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:01.469 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:09:01.469 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 37, 2024-09-16T00:09:01.469 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:01.469 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:01.470 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:01.470 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.470 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.470 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6800", 2024-09-16T00:09:01.470 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:01.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.470 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.470 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6801", 2024-09-16T00:09:01.471 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:01.471 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.471 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.471 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:01.471 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.471 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.471 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6802", 2024-09-16T00:09:01.471 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:01.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.472 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.472 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6803", 2024-09-16T00:09:01.472 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:01.472 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.472 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.472 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:01.472 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.473 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.473 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6806", 2024-09-16T00:09:01.473 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:01.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.473 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.473 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6807", 2024-09-16T00:09:01.473 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:01.473 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.473 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.474 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:01.474 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.474 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.474 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6804", 2024-09-16T00:09:01.474 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:01.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.475 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.475 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6805", 2024-09-16T00:09:01.475 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:01.475 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.475 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.475 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6801/1051778383", 2024-09-16T00:09:01.475 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6803/1051778383", 2024-09-16T00:09:01.476 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6807/1051778383", 2024-09-16T00:09:01.476 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6805/1051778383", 2024-09-16T00:09:01.476 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:01.476 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:01.476 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:01.476 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.477 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 6, 2024-09-16T00:09:01.477 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "9b6b33cc-d6d9-4a0a-8143-526073a58b93", 2024-09-16T00:09:01.477 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:01.477 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:01.477 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:01.477 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:01.477 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:01.477 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:01.478 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:09:01.478 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:09:01.478 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:01.478 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:01.478 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:01.478 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.478 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.479 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6824", 2024-09-16T00:09:01.479 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:01.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.479 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.479 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6825", 2024-09-16T00:09:01.479 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:01.479 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.479 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.480 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:01.480 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.480 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.480 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6826", 2024-09-16T00:09:01.480 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:01.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.481 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.481 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6827", 2024-09-16T00:09:01.481 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:01.481 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.481 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.481 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:01.481 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.482 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.482 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6830", 2024-09-16T00:09:01.482 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:01.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.482 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.482 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6831", 2024-09-16T00:09:01.482 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:01.483 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.483 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.483 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:01.483 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.483 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.483 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6828", 2024-09-16T00:09:01.483 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:01.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.484 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.484 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6829", 2024-09-16T00:09:01.484 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:01.484 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.484 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.484 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6825/1908495807", 2024-09-16T00:09:01.484 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6827/1908495807", 2024-09-16T00:09:01.485 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6831/1908495807", 2024-09-16T00:09:01.485 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6829/1908495807", 2024-09-16T00:09:01.485 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:01.485 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:01.485 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:01.485 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.486 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 7, 2024-09-16T00:09:01.486 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "50bcd808-3d3e-4a72-88e4-30decb53cfea", 2024-09-16T00:09:01.486 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:01.486 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:01.486 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:01.486 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:01.486 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:01.486 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:01.487 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:09:01.487 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 51, 2024-09-16T00:09:01.487 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:01.487 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:01.487 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:01.487 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.487 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.487 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6808", 2024-09-16T00:09:01.488 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:01.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.488 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.488 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6809", 2024-09-16T00:09:01.488 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:01.488 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.488 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.489 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:01.489 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.489 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.489 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6810", 2024-09-16T00:09:01.489 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:01.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.490 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.490 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6811", 2024-09-16T00:09:01.490 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:01.490 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.490 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.491 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:01.491 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.491 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.491 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6814", 2024-09-16T00:09:01.492 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:01.492 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.492 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.492 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6815", 2024-09-16T00:09:01.492 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:01.492 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.492 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.492 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.493 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:01.493 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:01.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.493 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:01.493 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6812", 2024-09-16T00:09:01.493 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:01.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.494 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:01.494 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6813", 2024-09-16T00:09:01.494 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:01.494 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.494 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.494 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6809/1158838808", 2024-09-16T00:09:01.495 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6811/1158838808", 2024-09-16T00:09:01.495 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6815/1158838808", 2024-09-16T00:09:01.495 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6813/1158838808", 2024-09-16T00:09:01.495 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:01.495 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:01.495 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:01.495 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.495 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.496 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:01.496 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_xinfo": [ 2024-09-16T00:09:01.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.496 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 0, 2024-09-16T00:09:01.496 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:01.496 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:01.496 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:01.497 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:01.497 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:01.497 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.843117+0000", 2024-09-16T00:09:01.497 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:01.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.497 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 1, 2024-09-16T00:09:01.497 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:01.497 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:01.497 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:01.498 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:01.498 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:01.498 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.622789+0000", 2024-09-16T00:09:01.498 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:01.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.498 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.498 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 2, 2024-09-16T00:09:01.498 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:01.498 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:01.498 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:01.499 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:01.499 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:01.499 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:17.919457+0000", 2024-09-16T00:09:01.499 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:01.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.499 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 3, 2024-09-16T00:09:01.499 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:01.500 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:01.500 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:01.500 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:01.500 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:01.500 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:19.330341+0000", 2024-09-16T00:09:01.500 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:01.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.501 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 4, 2024-09-16T00:09:01.501 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:01.501 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:01.501 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:01.501 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:01.501 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:01.501 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.815640+0000", 2024-09-16T00:09:01.501 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:01.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.502 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 5, 2024-09-16T00:09:01.502 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:01.502 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:01.502 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:01.502 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:01.502 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:01.502 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.083046+0000", 2024-09-16T00:09:01.502 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:01.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.503 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 6, 2024-09-16T00:09:01.503 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:01.503 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:01.503 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:01.503 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:01.503 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:01.503 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.551499+0000", 2024-09-16T00:09:01.504 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:01.504 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.504 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.504 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 7, 2024-09-16T00:09:01.504 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:01.504 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:01.504 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:01.504 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:01.505 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:01.505 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.377179+0000", 2024-09-16T00:09:01.505 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:01.505 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.505 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:01.505 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap": [], 2024-09-16T00:09:01.505 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap_items": [ 2024-09-16T00:09:01.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.506 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.1f", 2024-09-16T00:09:01.506 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:01.506 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.506 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 0, 2024-09-16T00:09:01.506 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:09:01.506 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.506 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.506 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.506 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.21", 2024-09-16T00:09:01.507 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:01.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.507 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 5, 2024-09-16T00:09:01.507 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 1 2024-09-16T00:09:01.507 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.507 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.507 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.507 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.22", 2024-09-16T00:09:01.507 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:01.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.508 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 5, 2024-09-16T00:09:01.508 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:09:01.508 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.508 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.509 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.38", 2024-09-16T00:09:01.509 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:01.509 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.509 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 3, 2024-09-16T00:09:01.509 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:09:01.509 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.509 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.510 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.510 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.3b", 2024-09-16T00:09:01.510 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:01.510 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.510 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 7, 2024-09-16T00:09:01.510 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 1 2024-09-16T00:09:01.510 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.510 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.511 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.511 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:01.511 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap_primaries": [], 2024-09-16T00:09:01.511 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_temp": [], 2024-09-16T00:09:01.511 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_temp": [], 2024-09-16T00:09:01.511 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklist": {}, 2024-09-16T00:09:01.511 INFO:teuthology.orchestra.run.smithi139.stdout: "range_blocklist": {}, 2024-09-16T00:09:01.511 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profiles": { 2024-09-16T00:09:01.511 INFO:teuthology.orchestra.run.smithi139.stdout: "default": { 2024-09-16T00:09:01.511 INFO:teuthology.orchestra.run.smithi139.stdout: "crush-failure-domain": "osd", 2024-09-16T00:09:01.512 INFO:teuthology.orchestra.run.smithi139.stdout: "k": "2", 2024-09-16T00:09:01.512 INFO:teuthology.orchestra.run.smithi139.stdout: "m": "1", 2024-09-16T00:09:01.512 INFO:teuthology.orchestra.run.smithi139.stdout: "plugin": "jerasure", 2024-09-16T00:09:01.512 INFO:teuthology.orchestra.run.smithi139.stdout: "technique": "reed_sol_van" 2024-09-16T00:09:01.512 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.512 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.512 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps_queue": [], 2024-09-16T00:09:01.512 INFO:teuthology.orchestra.run.smithi139.stdout: "new_removed_snaps": [], 2024-09-16T00:09:01.512 INFO:teuthology.orchestra.run.smithi139.stdout: "new_purged_snaps": [], 2024-09-16T00:09:01.513 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_node_flags": {}, 2024-09-16T00:09:01.513 INFO:teuthology.orchestra.run.smithi139.stdout: "device_class_flags": {}, 2024-09-16T00:09:01.513 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode": { 2024-09-16T00:09:01.513 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode_enabled": false, 2024-09-16T00:09:01.513 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_bucket_count": 0, 2024-09-16T00:09:01.513 INFO:teuthology.orchestra.run.smithi139.stdout: "degraded_stretch_mode": 0, 2024-09-16T00:09:01.513 INFO:teuthology.orchestra.run.smithi139.stdout: "recovering_stretch_mode": 0, 2024-09-16T00:09:01.514 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode_bucket": 0 2024-09-16T00:09:01.514 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.514 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:01.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.231+0000 7f4c58680640 1 -- 172.21.15.139:0/4285820389 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c3406c2b0 msgr2=0x7f4c3406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.231+0000 7f4c58680640 1 --2- 172.21.15.139:0/4285820389 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c3406c2b0 0x7f4c3406e770 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4c40034000 tx=0x7f4c4002e000 comp rx=0 tx=0).stop 2024-09-16T00:09:01.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.231+0000 7f4c58680640 1 -- 172.21.15.139:0/4285820389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c5006c110 msgr2=0x7f4c5015a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.231+0000 7f4c58680640 1 --2- 172.21.15.139:0/4285820389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c5006c110 0x7f4c5015a0a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00e870 tx=0x7f4c4c00ed30 comp rx=0 tx=0).stop 2024-09-16T00:09:01.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.231+0000 7f4c58680640 1 -- 172.21.15.139:0/4285820389 shutdown_connections 2024-09-16T00:09:01.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.231+0000 7f4c58680640 1 --2- 172.21.15.139:0/4285820389 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4c3406c2b0 0x7f4c3406e770 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.231+0000 7f4c58680640 1 --2- 172.21.15.139:0/4285820389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c500c67e0 0x7f4c50152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.231+0000 7f4c58680640 1 --2- 172.21.15.139:0/4285820389 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c5006c530 0x7f4c500cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.231+0000 7f4c58680640 1 --2- 172.21.15.139:0/4285820389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c5006c110 0x7f4c5015a0a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.231+0000 7f4c58680640 1 -- 172.21.15.139:0/4285820389 >> 172.21.15.139:0/4285820389 conn(0x7f4c500d05f0 msgr2=0x7f4c500c57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:01.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.231+0000 7f4c58680640 1 -- 172.21.15.139:0/4285820389 shutdown_connections 2024-09-16T00:09:01.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.231+0000 7f4c58680640 1 -- 172.21.15.139:0/4285820389 wait complete. 2024-09-16T00:09:01.519 INFO:tasks.thrashosds.thrasher:j is {'epoch': 61, 'fsid': '2d3366e5-6a59-4b59-93b2-615e72ac7015', 'created': '2024-09-16T00:07:09.967833+0000', 'modified': '2024-09-16T00:08:59.995558+0000', 'last_up_change': '2024-09-16T00:07:21.049678+0000', 'last_in_change': '2024-09-16T00:07:12.025620+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:07:22.056303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T00:07:30.768649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T00:07:32.008209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '83699a05-4937-43c3-aefc-b669bed9bb6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6816', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6817', 'nonce': 1974901438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6818', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6819', 'nonce': 1974901438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6822', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6823', 'nonce': 1974901438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6820', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6821', 'nonce': 1974901438}]}, 'public_addr': '172.21.15.139:6817/1974901438', 'cluster_addr': '172.21.15.139:6819/1974901438', 'heartbeat_back_addr': '172.21.15.139:6823/1974901438', 'heartbeat_front_addr': '172.21.15.139:6821/1974901438', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9c04991e-9695-4bbb-8011-bf613bf404b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6808', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6809', 'nonce': 2232380998}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6810', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6811', 'nonce': 2232380998}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6814', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6815', 'nonce': 2232380998}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6812', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6813', 'nonce': 2232380998}]}, 'public_addr': '172.21.15.139:6809/2232380998', 'cluster_addr': '172.21.15.139:6811/2232380998', 'heartbeat_back_addr': '172.21.15.139:6815/2232380998', 'heartbeat_front_addr': '172.21.15.139:6813/2232380998', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8048fc64-72b3-4866-ac62-6a8b2bf6f5c8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6800', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6801', 'nonce': 4180588905}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6802', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6803', 'nonce': 4180588905}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6806', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6807', 'nonce': 4180588905}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6804', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6805', 'nonce': 4180588905}]}, 'public_addr': '172.21.15.139:6801/4180588905', 'cluster_addr': '172.21.15.139:6803/4180588905', 'heartbeat_back_addr': '172.21.15.139:6807/4180588905', 'heartbeat_front_addr': '172.21.15.139:6805/4180588905', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c34ac77b-354c-4884-902d-3e2a84f5eb27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6826', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6827', 'nonce': 3947768492}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6828', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6829', 'nonce': 3947768492}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6832', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6833', 'nonce': 3947768492}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6830', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6831', 'nonce': 3947768492}]}, 'public_addr': '172.21.15.139:6827/3947768492', 'cluster_addr': '172.21.15.139:6829/3947768492', 'heartbeat_back_addr': '172.21.15.139:6833/3947768492', 'heartbeat_front_addr': '172.21.15.139:6831/3947768492', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ba9a7c93-9672-43e1-8c98-2878cdf7f548', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6816', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6817', 'nonce': 642632916}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6818', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 642632916}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 642632916}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6821', 'nonce': 642632916}]}, 'public_addr': '172.21.15.164:6817/642632916', 'cluster_addr': '172.21.15.164:6819/642632916', 'heartbeat_back_addr': '172.21.15.164:6823/642632916', 'heartbeat_front_addr': '172.21.15.164:6821/642632916', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '46a0916b-474c-411b-84ad-86f27d4cbf53', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 1051778383}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 1051778383}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 1051778383}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 1051778383}]}, 'public_addr': '172.21.15.164:6801/1051778383', 'cluster_addr': '172.21.15.164:6803/1051778383', 'heartbeat_back_addr': '172.21.15.164:6807/1051778383', 'heartbeat_front_addr': '172.21.15.164:6805/1051778383', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b6b33cc-d6d9-4a0a-8143-526073a58b93', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6824', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 1908495807}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6827', 'nonce': 1908495807}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6830', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 1908495807}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 1908495807}]}, 'public_addr': '172.21.15.164:6825/1908495807', 'cluster_addr': '172.21.15.164:6827/1908495807', 'heartbeat_back_addr': '172.21.15.164:6831/1908495807', 'heartbeat_front_addr': '172.21.15.164:6829/1908495807', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '50bcd808-3d3e-4a72-88e4-30decb53cfea', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 51, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6809', 'nonce': 1158838808}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 1158838808}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6815', 'nonce': 1158838808}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6812', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 1158838808}]}, 'public_addr': '172.21.15.164:6809/1158838808', 'cluster_addr': '172.21.15.164:6811/1158838808', 'heartbeat_back_addr': '172.21.15.164:6815/1158838808', 'heartbeat_front_addr': '172.21.15.164:6813/1158838808', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.843117+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.622789+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:17.919457+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:19.330341+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.815640+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.083046+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.551499+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.377179+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.1f', 'mappings': [{'from': 0, 'to': 2}]}, {'pgid': '3.21', 'mappings': [{'from': 5, 'to': 1}]}, {'pgid': '3.22', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '3.38', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '3.3b', 'mappings': [{'from': 7, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T00:09:01.519 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:09:01.520 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:01.392+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:01.520 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:01.516+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:01.618 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:01.618+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:01.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.674+0000 7fa7eef86640 1 Processor -- start 2024-09-16T00:09:01.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.674+0000 7fa7eef86640 1 -- start start 2024-09-16T00:09:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7eef86640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa7e8073a00 0x7fa7e80c5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7eef86640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa7e8073de0 0x7fa7e80c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7eef86640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa7e81513c0 0x7fa7e8151870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7eef86640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e80c5e50 con 0x7fa7e81513c0 2024-09-16T00:09:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7eef86640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e80c6130 con 0x7fa7e8073de0 2024-09-16T00:09:01.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7eef86640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e80c80a0 con 0x7fa7e8073a00 2024-09-16T00:09:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7ed523640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa7e81513c0 0x7fa7e8151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7ecd22640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa7e8073a00 0x7fa7e80c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7ed523640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa7e81513c0 0x7fa7e8151870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41966/0 (socket says 172.21.15.139:41966) 2024-09-16T00:09:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7ed523640 1 -- 172.21.15.139:0/3800920105 learned_addr learned my addr 172.21.15.139:0/3800920105 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7e7fff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa7e8073de0 0x7fa7e80c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7ecd22640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa7e8073a00 0x7fa7e80c5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:56842/0 (socket says 172.21.15.139:56842) 2024-09-16T00:09:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7ed523640 1 -- 172.21.15.139:0/3800920105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa7e8073a00 msgr2=0x7fa7e80c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7ed523640 1 --2- 172.21.15.139:0/3800920105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa7e8073a00 0x7fa7e80c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7ed523640 1 -- 172.21.15.139:0/3800920105 >> v2:172.21.15.164:3300/0 conn(0x7fa7e8073de0 msgr2=0x7fa7e80c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7ed523640 1 --2- 172.21.15.139:0/3800920105 >> v2:172.21.15.164:3300/0 conn(0x7fa7e8073de0 0x7fa7e80c39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7ed523640 1 -- 172.21.15.139:0/3800920105 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e80c8380 con 0x7fa7e81513c0 2024-09-16T00:09:01.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.675+0000 7fa7e7fff640 1 --2- 172.21.15.139:0/3800920105 >> v2:172.21.15.164:3300/0 conn(0x7fa7e8073de0 0x7fa7e80c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:09:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.676+0000 7fa7ed523640 1 --2- 172.21.15.139:0/3800920105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e81513c0 0x7fa7e8151870 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa7e806cad0 tx=0x7fa7e002a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b18bb7e9d2eab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.676+0000 7fa7e77fe640 1 -- 172.21.15.139:0/3800920105 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0036070 con 0x7fa7e81513c0 2024-09-16T00:09:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.676+0000 7fa7e77fe640 1 -- 172.21.15.139:0/3800920105 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e0031600 con 0x7fa7e81513c0 2024-09-16T00:09:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.676+0000 7fa7e77fe640 1 -- 172.21.15.139:0/3800920105 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00319d0 con 0x7fa7e81513c0 2024-09-16T00:09:01.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.677+0000 7fa7eef86640 1 -- 172.21.15.139:0/3800920105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e81513c0 msgr2=0x7fa7e8151870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.677+0000 7fa7eef86640 1 --2- 172.21.15.139:0/3800920105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e81513c0 0x7fa7e8151870 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa7e806cad0 tx=0x7fa7e002a720 comp rx=0 tx=0).stop 2024-09-16T00:09:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.677+0000 7fa7eef86640 1 -- 172.21.15.139:0/3800920105 shutdown_connections 2024-09-16T00:09:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.677+0000 7fa7eef86640 1 --2- 172.21.15.139:0/3800920105 >> v2:172.21.15.164:3300/0 conn(0x7fa7e8073de0 0x7fa7e80c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.677+0000 7fa7eef86640 1 --2- 172.21.15.139:0/3800920105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e81513c0 0x7fa7e8151870 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.677+0000 7fa7eef86640 1 --2- 172.21.15.139:0/3800920105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa7e8073a00 0x7fa7e80c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.677+0000 7fa7eef86640 1 -- 172.21.15.139:0/3800920105 >> 172.21.15.139:0/3800920105 conn(0x7fa7e806bb20 msgr2=0x7fa7e806bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.677+0000 7fa7eef86640 1 -- 172.21.15.139:0/3800920105 shutdown_connections 2024-09-16T00:09:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.677+0000 7fa7eef86640 1 -- 172.21.15.139:0/3800920105 wait complete. 2024-09-16T00:09:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.678+0000 7fa7eef86640 1 Processor -- start 2024-09-16T00:09:01.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.678+0000 7fa7eef86640 1 -- start start 2024-09-16T00:09:01.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7eef86640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa7e8073a00 0x7fa7e81f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7eef86640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e8073de0 0x7fa7e80cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7eef86640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7e81513c0 0x7fa7e80cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7eef86640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e80c5b50 con 0x7fa7e8073de0 2024-09-16T00:09:01.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7eef86640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e80c8940 con 0x7fa7e81513c0 2024-09-16T00:09:01.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7eef86640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e80c3f30 con 0x7fa7e8073a00 2024-09-16T00:09:01.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7e7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e8073de0 0x7fa7e80cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7e7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e8073de0 0x7fa7e80cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41978/0 (socket says 172.21.15.139:41978) 2024-09-16T00:09:01.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7e7fff640 1 -- 172.21.15.139:0/3288359993 learned_addr learned my addr 172.21.15.139:0/3288359993 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7ed523640 1 --2- 172.21.15.139:0/3288359993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7e81513c0 0x7fa7e80cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7ecd22640 1 --2- 172.21.15.139:0/3288359993 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa7e8073a00 0x7fa7e81f0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7e7fff640 1 -- 172.21.15.139:0/3288359993 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa7e8073a00 msgr2=0x7fa7e81f0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7e7fff640 1 --2- 172.21.15.139:0/3288359993 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa7e8073a00 0x7fa7e81f0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7e7fff640 1 -- 172.21.15.139:0/3288359993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7e81513c0 msgr2=0x7fa7e80cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7e7fff640 1 --2- 172.21.15.139:0/3288359993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7e81513c0 0x7fa7e80cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7e7fff640 1 -- 172.21.15.139:0/3288359993 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e80cbe70 con 0x7fa7e8073de0 2024-09-16T00:09:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7ed523640 1 --2- 172.21.15.139:0/3288359993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7e81513c0 0x7fa7e80cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.680+0000 7fa7e7fff640 1 --2- 172.21.15.139:0/3288359993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e8073de0 0x7fa7e80cac90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa7d400e840 tx=0x7fa7d400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.681+0000 7fa7e57fa640 1 -- 172.21.15.139:0/3288359993 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4017070 con 0x7fa7e8073de0 2024-09-16T00:09:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.681+0000 7fa7e57fa640 1 -- 172.21.15.139:0/3288359993 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d400c6d0 con 0x7fa7e8073de0 2024-09-16T00:09:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.681+0000 7fa7e57fa640 1 -- 172.21.15.139:0/3288359993 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d400f040 con 0x7fa7e8073de0 2024-09-16T00:09:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.681+0000 7fa7eef86640 1 -- 172.21.15.139:0/3288359993 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e80cb8d0 con 0x7fa7e8073de0 2024-09-16T00:09:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.681+0000 7fa7eef86640 1 -- 172.21.15.139:0/3288359993 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e8154a20 con 0x7fa7e8073de0 2024-09-16T00:09:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.682+0000 7fa7eef86640 1 -- 172.21.15.139:0/3288359993 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e80c5610 con 0x7fa7e8073de0 2024-09-16T00:09:01.685 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:01.685 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:01.685 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:01.685 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:01.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.683+0000 7fa7e57fa640 1 -- 172.21.15.139:0/3288359993 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa7d401a070 con 0x7fa7e8073de0 2024-09-16T00:09:01.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.683+0000 7fa7e57fa640 1 --2- 172.21.15.139:0/3288359993 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa7cc06c2b0 0x7fa7cc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:01.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.683+0000 7fa7e57fa640 1 -- 172.21.15.139:0/3288359993 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5886+0+0 (secure 0 0 0) 0x7fa7d408afe0 con 0x7fa7e8073de0 2024-09-16T00:09:01.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.684+0000 7fa7ecd22640 1 --2- 172.21.15.139:0/3288359993 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa7cc06c2b0 0x7fa7cc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:01.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.684+0000 7fa7ecd22640 1 --2- 172.21.15.139:0/3288359993 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa7cc06c2b0 0x7fa7cc06e770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009a20 tx=0x7fa7dc009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:01.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.688+0000 7fa7e57fa640 1 -- 172.21.15.139:0/3288359993 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa7d4059050 con 0x7fa7e8073de0 2024-09-16T00:09:01.694 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:09:01.738 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:01.738+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:01.820 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:01.819+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:01.824 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:01.824 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:01.824 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:01.824 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:01.824 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:01.835 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:09:01.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.901+0000 7fa7eef86640 1 -- 172.21.15.139:0/3288359993 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa7e80cbbb0 con 0x7fa7cc06c2b0 2024-09-16T00:09:01.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.907+0000 7fa7e57fa640 1 -- 172.21.15.139:0/3288359993 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252397 (secure 0 0 0) 0x7fa7e80cbbb0 con 0x7fa7cc06c2b0 2024-09-16T00:09:01.911 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:09:01.912 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:09:01.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.910+0000 7fa7eef86640 1 -- 172.21.15.139:0/3288359993 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa7cc06c2b0 msgr2=0x7fa7cc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.910+0000 7fa7eef86640 1 --2- 172.21.15.139:0/3288359993 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa7cc06c2b0 0x7fa7cc06e770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009a20 tx=0x7fa7dc009190 comp rx=0 tx=0).stop 2024-09-16T00:09:01.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.910+0000 7fa7eef86640 1 -- 172.21.15.139:0/3288359993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e8073de0 msgr2=0x7fa7e80cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:01.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.910+0000 7fa7eef86640 1 --2- 172.21.15.139:0/3288359993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e8073de0 0x7fa7e80cac90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa7d400e840 tx=0x7fa7d400ed00 comp rx=0 tx=0).stop 2024-09-16T00:09:01.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.910+0000 7fa7eef86640 1 -- 172.21.15.139:0/3288359993 shutdown_connections 2024-09-16T00:09:01.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.910+0000 7fa7eef86640 1 --2- 172.21.15.139:0/3288359993 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa7cc06c2b0 0x7fa7cc06e770 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.910+0000 7fa7eef86640 1 --2- 172.21.15.139:0/3288359993 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa7e81513c0 0x7fa7e80cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.910+0000 7fa7eef86640 1 --2- 172.21.15.139:0/3288359993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7e8073de0 0x7fa7e80cac90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.910+0000 7fa7eef86640 1 --2- 172.21.15.139:0/3288359993 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa7e8073a00 0x7fa7e81f0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:01.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.910+0000 7fa7eef86640 1 -- 172.21.15.139:0/3288359993 >> 172.21.15.139:0/3288359993 conn(0x7fa7e806bb20 msgr2=0x7fa7e806fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:01.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.911+0000 7fa7eef86640 1 -- 172.21.15.139:0/3288359993 shutdown_connections 2024-09-16T00:09:01.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:01.911+0000 7fa7eef86640 1 -- 172.21.15.139:0/3288359993 wait complete. 2024-09-16T00:09:01.921 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:01.920+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:01.922 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-16T00:09:00.040495+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8047325,"num_objects":194,"num_object_clones":0,"num_object_copies":388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":194,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":389,"num_write_kb":9818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":13110,"num_keys_recovered":4,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":338,"ondisk_log_size":338,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231536,"kb_used_data":15288,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754743184,"statfs":{"total":773094113280,"available":772857020416,"internally_reserved":0,"allocated":15654912,"data_stored":13801740,"data_compressed":263129,"data_compressed_allocated":339968,"data_compressed_original":2060288,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7449604,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":177,"num_write_kb":7379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000555"},"pg_stats":[{"pgid":"3.3f","version":"57'4","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984959+0000","last_change":"2024-09-16T00:08:36.888570+0000","last_active":"2024-09-16T00:08:56.984959+0000","last_peered":"2024-09-16T00:08:56.984959+0000","last_clean":"2024-09-16T00:08:56.984959+0000","last_became_active":"2024-09-16T00:08:36.888143+0000","last_became_peered":"2024-09-16T00:08:36.888143+0000","last_unstale":"2024-09-16T00:08:56.984959+0000","last_undegraded":"2024-09-16T00:08:56.984959+0000","last_fullsized":"2024-09-16T00:08:56.984959+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:30.579681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7173,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980158+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:08:55.980158+0000","last_peered":"2024-09-16T00:08:55.980158+0000","last_clean":"2024-09-16T00:08:55.980158+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:08:55.980158+0000","last_undegraded":"2024-09-16T00:08:55.980158+0000","last_fullsized":"2024-09-16T00:08:55.980158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973955+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:08:54.973955+0000","last_peered":"2024-09-16T00:08:54.973955+0000","last_clean":"2024-09-16T00:08:54.973955+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:08:54.973955+0000","last_undegraded":"2024-09-16T00:08:54.973955+0000","last_fullsized":"2024-09-16T00:08:54.973955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"57'1","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005040+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:08:57.005040+0000","last_peered":"2024-09-16T00:08:57.005040+0000","last_clean":"2024-09-16T00:08:57.005040+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:08:57.005040+0000","last_undegraded":"2024-09-16T00:08:57.005040+0000","last_fullsized":"2024-09-16T00:08:57.005040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":56,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980028+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:08:55.980028+0000","last_peered":"2024-09-16T00:08:55.980028+0000","last_clean":"2024-09-16T00:08:55.980028+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:08:55.980028+0000","last_undegraded":"2024-09-16T00:08:55.980028+0000","last_fullsized":"2024-09-16T00:08:55.980028+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"57'2","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004829+0000","last_change":"2024-09-16T00:07:33.627649+0000","last_active":"2024-09-16T00:08:57.004829+0000","last_peered":"2024-09-16T00:08:57.004829+0000","last_clean":"2024-09-16T00:08:57.004829+0000","last_became_active":"2024-09-16T00:07:33.626471+0000","last_became_peered":"2024-09-16T00:07:33.626471+0000","last_unstale":"2024-09-16T00:08:57.004829+0000","last_undegraded":"2024-09-16T00:08:57.004829+0000","last_fullsized":"2024-09-16T00:08:57.004829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12750,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"57'2","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984334+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:08:56.984334+0000","last_peered":"2024-09-16T00:08:56.984334+0000","last_clean":"2024-09-16T00:08:56.984334+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:08:56.984334+0000","last_undegraded":"2024-09-16T00:08:56.984334+0000","last_fullsized":"2024-09-16T00:08:56.984334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8617,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":52,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.423823+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:08:54.423823+0000","last_peered":"2024-09-16T00:08:54.423823+0000","last_clean":"2024-09-16T00:08:54.423823+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:08:54.423823+0000","last_undegraded":"2024-09-16T00:08:54.423823+0000","last_fullsized":"2024-09-16T00:08:54.423823+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"58'2","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.985069+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:08:56.985069+0000","last_peered":"2024-09-16T00:08:56.985069+0000","last_clean":"2024-09-16T00:08:56.985069+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:08:56.985069+0000","last_undegraded":"2024-09-16T00:08:56.985069+0000","last_fullsized":"2024-09-16T00:08:56.985069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27085,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"57'7","reported_seq":38,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983777+0000","last_change":"2024-09-16T00:08:36.891868+0000","last_active":"2024-09-16T00:08:56.983777+0000","last_peered":"2024-09-16T00:08:56.983777+0000","last_clean":"2024-09-16T00:08:56.983777+0000","last_became_active":"2024-09-16T00:08:36.891605+0000","last_became_peered":"2024-09-16T00:08:36.891605+0000","last_unstale":"2024-09-16T00:08:56.983777+0000","last_undegraded":"2024-09-16T00:08:56.983777+0000","last_fullsized":"2024-09-16T00:08:56.983777+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:46:00.730436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44377,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"57'5","reported_seq":36,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983513+0000","last_change":"2024-09-16T00:08:36.891589+0000","last_active":"2024-09-16T00:08:56.983513+0000","last_peered":"2024-09-16T00:08:56.983513+0000","last_clean":"2024-09-16T00:08:56.983513+0000","last_became_active":"2024-09-16T00:08:36.891279+0000","last_became_peered":"2024-09-16T00:08:36.891279+0000","last_unstale":"2024-09-16T00:08:56.983513+0000","last_undegraded":"2024-09-16T00:08:56.983513+0000","last_fullsized":"2024-09-16T00:08:56.983513+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3123,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"58'4","reported_seq":97,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.008913+0000","last_change":"2024-09-16T00:08:36.888576+0000","last_active":"2024-09-16T00:08:57.008913+0000","last_peered":"2024-09-16T00:08:57.008913+0000","last_clean":"2024-09-16T00:08:57.008913+0000","last_became_active":"2024-09-16T00:08:36.888301+0000","last_became_peered":"2024-09-16T00:08:36.888301+0000","last_unstale":"2024-09-16T00:08:57.008913+0000","last_undegraded":"2024-09-16T00:08:57.008913+0000","last_fullsized":"2024-09-16T00:08:57.008913+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21479,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991795+0000","last_change":"2024-09-16T00:08:36.892595+0000","last_active":"2024-09-16T00:08:55.991795+0000","last_peered":"2024-09-16T00:08:55.991795+0000","last_clean":"2024-09-16T00:08:55.991795+0000","last_became_active":"2024-09-16T00:08:36.892316+0000","last_became_peered":"2024-09-16T00:08:36.892316+0000","last_unstale":"2024-09-16T00:08:55.991795+0000","last_undegraded":"2024-09-16T00:08:55.991795+0000","last_fullsized":"2024-09-16T00:08:55.991795+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:47.480168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"56'1","reported_seq":80,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973769+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:08:54.973769+0000","last_peered":"2024-09-16T00:08:54.973769+0000","last_clean":"2024-09-16T00:08:54.973769+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:08:54.973769+0000","last_undegraded":"2024-09-16T00:08:54.973769+0000","last_fullsized":"2024-09-16T00:08:54.973769+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979862+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:08:55.979862+0000","last_peered":"2024-09-16T00:08:55.979862+0000","last_clean":"2024-09-16T00:08:55.979862+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:08:55.979862+0000","last_undegraded":"2024-09-16T00:08:55.979862+0000","last_fullsized":"2024-09-16T00:08:55.979862+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"57'4","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004785+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:08:57.004785+0000","last_peered":"2024-09-16T00:08:57.004785+0000","last_clean":"2024-09-16T00:08:57.004785+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:08:57.004785+0000","last_undegraded":"2024-09-16T00:08:57.004785+0000","last_fullsized":"2024-09-16T00:08:57.004785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8128,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"57'4","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005170+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:08:57.005170+0000","last_peered":"2024-09-16T00:08:57.005170+0000","last_clean":"2024-09-16T00:08:57.005170+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:08:57.005170+0000","last_undegraded":"2024-09-16T00:08:57.005170+0000","last_fullsized":"2024-09-16T00:08:57.005170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4240,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424154+0000","last_change":"2024-09-16T00:08:36.894430+0000","last_active":"2024-09-16T00:08:54.424154+0000","last_peered":"2024-09-16T00:08:54.424154+0000","last_clean":"2024-09-16T00:08:54.424154+0000","last_became_active":"2024-09-16T00:08:36.894138+0000","last_became_peered":"2024-09-16T00:08:36.894138+0000","last_unstale":"2024-09-16T00:08:54.424154+0000","last_undegraded":"2024-09-16T00:08:54.424154+0000","last_fullsized":"2024-09-16T00:08:54.424154+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:19.478353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"57'4","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.985021+0000","last_change":"2024-09-16T00:07:33.630711+0000","last_active":"2024-09-16T00:08:56.985021+0000","last_peered":"2024-09-16T00:08:56.985021+0000","last_clean":"2024-09-16T00:08:56.985021+0000","last_became_active":"2024-09-16T00:07:33.630453+0000","last_became_peered":"2024-09-16T00:07:33.630453+0000","last_unstale":"2024-09-16T00:08:56.985021+0000","last_undegraded":"2024-09-16T00:08:56.985021+0000","last_fullsized":"2024-09-16T00:08:56.985021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1845,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"58'8","reported_seq":39,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983626+0000","last_change":"2024-09-16T00:08:36.890676+0000","last_active":"2024-09-16T00:08:56.983626+0000","last_peered":"2024-09-16T00:08:56.983626+0000","last_clean":"2024-09-16T00:08:56.983626+0000","last_became_active":"2024-09-16T00:08:36.889743+0000","last_became_peered":"2024-09-16T00:08:36.889743+0000","last_unstale":"2024-09-16T00:08:56.983626+0000","last_undegraded":"2024-09-16T00:08:56.983626+0000","last_fullsized":"2024-09-16T00:08:56.983626+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:02.709496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21485,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973839+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:08:54.973839+0000","last_peered":"2024-09-16T00:08:54.973839+0000","last_clean":"2024-09-16T00:08:54.973839+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:08:54.973839+0000","last_undegraded":"2024-09-16T00:08:54.973839+0000","last_fullsized":"2024-09-16T00:08:54.973839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"57'7","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004869+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:08:57.004869+0000","last_peered":"2024-09-16T00:08:57.004869+0000","last_clean":"2024-09-16T00:08:57.004869+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:08:57.004869+0000","last_undegraded":"2024-09-16T00:08:57.004869+0000","last_fullsized":"2024-09-16T00:08:57.004869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69449,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991605+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:08:55.991605+0000","last_peered":"2024-09-16T00:08:55.991605+0000","last_clean":"2024-09-16T00:08:55.991605+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:08:55.991605+0000","last_undegraded":"2024-09-16T00:08:55.991605+0000","last_fullsized":"2024-09-16T00:08:55.991605+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"57'7","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004863+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:08:57.004863+0000","last_peered":"2024-09-16T00:08:57.004863+0000","last_clean":"2024-09-16T00:08:57.004863+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:08:57.004863+0000","last_undegraded":"2024-09-16T00:08:57.004863+0000","last_fullsized":"2024-09-16T00:08:57.004863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14047,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.992055+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:08:55.992055+0000","last_peered":"2024-09-16T00:08:55.992055+0000","last_clean":"2024-09-16T00:08:55.992055+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:08:55.992055+0000","last_undegraded":"2024-09-16T00:08:55.992055+0000","last_fullsized":"2024-09-16T00:08:55.992055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"57'7","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983874+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:08:56.983874+0000","last_peered":"2024-09-16T00:08:56.983874+0000","last_clean":"2024-09-16T00:08:56.983874+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:08:56.983874+0000","last_undegraded":"2024-09-16T00:08:56.983874+0000","last_fullsized":"2024-09-16T00:08:56.983874+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92931,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979860+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:08:55.979860+0000","last_peered":"2024-09-16T00:08:55.979860+0000","last_clean":"2024-09-16T00:08:55.979860+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:08:55.979860+0000","last_undegraded":"2024-09-16T00:08:55.979860+0000","last_fullsized":"2024-09-16T00:08:55.979860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991901+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:08:55.991901+0000","last_peered":"2024-09-16T00:08:55.991901+0000","last_clean":"2024-09-16T00:08:55.991901+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:08:55.991901+0000","last_undegraded":"2024-09-16T00:08:55.991901+0000","last_fullsized":"2024-09-16T00:08:55.991901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973891+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:08:54.973891+0000","last_peered":"2024-09-16T00:08:54.973891+0000","last_clean":"2024-09-16T00:08:54.973891+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:08:54.973891+0000","last_undegraded":"2024-09-16T00:08:54.973891+0000","last_fullsized":"2024-09-16T00:08:54.973891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":52,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424099+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:08:54.424099+0000","last_peered":"2024-09-16T00:08:54.424099+0000","last_clean":"2024-09-16T00:08:54.424099+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:08:54.424099+0000","last_undegraded":"2024-09-16T00:08:54.424099+0000","last_fullsized":"2024-09-16T00:08:54.424099+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":83,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424334+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:08:54.424334+0000","last_peered":"2024-09-16T00:08:54.424334+0000","last_clean":"2024-09-16T00:08:54.424334+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:08:54.424334+0000","last_undegraded":"2024-09-16T00:08:54.424334+0000","last_fullsized":"2024-09-16T00:08:54.424334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"57'6","reported_seq":37,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983162+0000","last_change":"2024-09-16T00:08:36.894101+0000","last_active":"2024-09-16T00:08:56.983162+0000","last_peered":"2024-09-16T00:08:56.983162+0000","last_clean":"2024-09-16T00:08:56.983162+0000","last_became_active":"2024-09-16T00:08:36.893807+0000","last_became_peered":"2024-09-16T00:08:36.893807+0000","last_unstale":"2024-09-16T00:08:56.983162+0000","last_undegraded":"2024-09-16T00:08:56.983162+0000","last_fullsized":"2024-09-16T00:08:56.983162+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:40.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59387,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"58'7","reported_seq":90,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.010727+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:08:57.010727+0000","last_peered":"2024-09-16T00:08:57.010727+0000","last_clean":"2024-09-16T00:08:57.010727+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:08:57.010727+0000","last_undegraded":"2024-09-16T00:08:57.010727+0000","last_fullsized":"2024-09-16T00:08:57.010727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16184,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":85,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991911+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:08:55.991911+0000","last_peered":"2024-09-16T00:08:55.991911+0000","last_clean":"2024-09-16T00:08:55.991911+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:08:55.991911+0000","last_undegraded":"2024-09-16T00:08:55.991911+0000","last_fullsized":"2024-09-16T00:08:55.991911+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973910+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:08:54.973910+0000","last_peered":"2024-09-16T00:08:54.973910+0000","last_clean":"2024-09-16T00:08:54.973910+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:08:54.973910+0000","last_undegraded":"2024-09-16T00:08:54.973910+0000","last_fullsized":"2024-09-16T00:08:54.973910+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":31,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983375+0000","last_change":"2024-09-16T00:08:36.890316+0000","last_active":"2024-09-16T00:08:56.983375+0000","last_peered":"2024-09-16T00:08:56.983375+0000","last_clean":"2024-09-16T00:08:56.983375+0000","last_became_active":"2024-09-16T00:08:36.889635+0000","last_became_peered":"2024-09-16T00:08:36.889635+0000","last_unstale":"2024-09-16T00:08:56.983375+0000","last_undegraded":"2024-09-16T00:08:56.983375+0000","last_fullsized":"2024-09-16T00:08:56.983375+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:28.843883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":94,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991691+0000","last_change":"2024-09-16T00:08:36.888349+0000","last_active":"2024-09-16T00:08:55.991691+0000","last_peered":"2024-09-16T00:08:55.991691+0000","last_clean":"2024-09-16T00:08:55.991691+0000","last_became_active":"2024-09-16T00:08:36.888052+0000","last_became_peered":"2024-09-16T00:08:36.888052+0000","last_unstale":"2024-09-16T00:08:55.991691+0000","last_undegraded":"2024-09-16T00:08:55.991691+0000","last_fullsized":"2024-09-16T00:08:55.991691+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:49.607254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991502+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:08:55.991502+0000","last_peered":"2024-09-16T00:08:55.991502+0000","last_clean":"2024-09-16T00:08:55.991502+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:08:55.991502+0000","last_undegraded":"2024-09-16T00:08:55.991502+0000","last_fullsized":"2024-09-16T00:08:55.991502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004896+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:08:57.004896+0000","last_peered":"2024-09-16T00:08:57.004896+0000","last_clean":"2024-09-16T00:08:57.004896+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:08:57.004896+0000","last_undegraded":"2024-09-16T00:08:57.004896+0000","last_fullsized":"2024-09-16T00:08:57.004896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979829+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:08:55.979829+0000","last_peered":"2024-09-16T00:08:55.979829+0000","last_clean":"2024-09-16T00:08:55.979829+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:08:55.979829+0000","last_undegraded":"2024-09-16T00:08:55.979829+0000","last_fullsized":"2024-09-16T00:08:55.979829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983626+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:08:56.983626+0000","last_peered":"2024-09-16T00:08:56.983626+0000","last_clean":"2024-09-16T00:08:56.983626+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:08:56.983626+0000","last_undegraded":"2024-09-16T00:08:56.983626+0000","last_fullsized":"2024-09-16T00:08:56.983626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"57'5","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005138+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:08:57.005138+0000","last_peered":"2024-09-16T00:08:57.005138+0000","last_clean":"2024-09-16T00:08:57.005138+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:08:57.005138+0000","last_undegraded":"2024-09-16T00:08:57.005138+0000","last_fullsized":"2024-09-16T00:08:57.005138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":80946,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005068+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:08:57.005068+0000","last_peered":"2024-09-16T00:08:57.005068+0000","last_clean":"2024-09-16T00:08:57.005068+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:08:57.005068+0000","last_undegraded":"2024-09-16T00:08:57.005068+0000","last_fullsized":"2024-09-16T00:08:57.005068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.974071+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:08:54.974071+0000","last_peered":"2024-09-16T00:08:54.974071+0000","last_clean":"2024-09-16T00:08:54.974071+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:08:54.974071+0000","last_undegraded":"2024-09-16T00:08:54.974071+0000","last_fullsized":"2024-09-16T00:08:54.974071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":86,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980005+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:08:55.980005+0000","last_peered":"2024-09-16T00:08:55.980005+0000","last_clean":"2024-09-16T00:08:55.980005+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:08:55.980005+0000","last_undegraded":"2024-09-16T00:08:55.980005+0000","last_fullsized":"2024-09-16T00:08:55.980005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"57'8","reported_seq":91,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.008894+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:08:57.008894+0000","last_peered":"2024-09-16T00:08:57.008894+0000","last_clean":"2024-09-16T00:08:57.008894+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:08:57.008894+0000","last_undegraded":"2024-09-16T00:08:57.008894+0000","last_fullsized":"2024-09-16T00:08:57.008894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84046,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"57'15","reported_seq":46,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983597+0000","last_change":"2024-09-16T00:08:36.914308+0000","last_active":"2024-09-16T00:08:56.983597+0000","last_peered":"2024-09-16T00:08:56.983597+0000","last_clean":"2024-09-16T00:08:56.983597+0000","last_became_active":"2024-09-16T00:08:36.890227+0000","last_became_peered":"2024-09-16T00:08:36.890227+0000","last_unstale":"2024-09-16T00:08:56.983597+0000","last_undegraded":"2024-09-16T00:08:56.983597+0000","last_fullsized":"2024-09-16T00:08:56.983597+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:31.761195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21522,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13110,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"57'3","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984771+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:08:56.984771+0000","last_peered":"2024-09-16T00:08:56.984771+0000","last_clean":"2024-09-16T00:08:56.984771+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:08:56.984771+0000","last_undegraded":"2024-09-16T00:08:56.984771+0000","last_fullsized":"2024-09-16T00:08:56.984771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2379,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":83,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973988+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:08:54.973988+0000","last_peered":"2024-09-16T00:08:54.973988+0000","last_clean":"2024-09-16T00:08:54.973988+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:08:54.973988+0000","last_undegraded":"2024-09-16T00:08:54.973988+0000","last_fullsized":"2024-09-16T00:08:54.973988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973740+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:08:54.973740+0000","last_peered":"2024-09-16T00:08:54.973740+0000","last_clean":"2024-09-16T00:08:54.973740+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:08:54.973740+0000","last_undegraded":"2024-09-16T00:08:54.973740+0000","last_fullsized":"2024-09-16T00:08:54.973740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004924+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:08:57.004924+0000","last_peered":"2024-09-16T00:08:57.004924+0000","last_clean":"2024-09-16T00:08:57.004924+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:08:57.004924+0000","last_undegraded":"2024-09-16T00:08:57.004924+0000","last_fullsized":"2024-09-16T00:08:57.004924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"57'10","reported_seq":93,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984391+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:08:56.984391+0000","last_peered":"2024-09-16T00:08:56.984391+0000","last_clean":"2024-09-16T00:08:56.984391+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:08:56.984391+0000","last_undegraded":"2024-09-16T00:08:56.984391+0000","last_fullsized":"2024-09-16T00:08:56.984391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2388208,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":2338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":31,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983231+0000","last_change":"2024-09-16T00:08:36.889720+0000","last_active":"2024-09-16T00:08:56.983231+0000","last_peered":"2024-09-16T00:08:56.983231+0000","last_clean":"2024-09-16T00:08:56.983231+0000","last_became_active":"2024-09-16T00:08:36.889345+0000","last_became_peered":"2024-09-16T00:08:36.889345+0000","last_unstale":"2024-09-16T00:08:56.983231+0000","last_undegraded":"2024-09-16T00:08:56.983231+0000","last_fullsized":"2024-09-16T00:08:56.983231+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:03.170073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"58'7","reported_seq":38,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983217+0000","last_change":"2024-09-16T00:08:36.896787+0000","last_active":"2024-09-16T00:08:56.983217+0000","last_peered":"2024-09-16T00:08:56.983217+0000","last_clean":"2024-09-16T00:08:56.983217+0000","last_became_active":"2024-09-16T00:08:36.896505+0000","last_became_peered":"2024-09-16T00:08:36.896505+0000","last_unstale":"2024-09-16T00:08:56.983217+0000","last_undegraded":"2024-09-16T00:08:56.983217+0000","last_fullsized":"2024-09-16T00:08:56.983217+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:55.997551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4202646,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424000+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:08:54.424000+0000","last_peered":"2024-09-16T00:08:54.424000+0000","last_clean":"2024-09-16T00:08:54.424000+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:08:54.424000+0000","last_undegraded":"2024-09-16T00:08:54.424000+0000","last_fullsized":"2024-09-16T00:08:54.424000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"57'5","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984434+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:08:56.984434+0000","last_peered":"2024-09-16T00:08:56.984434+0000","last_clean":"2024-09-16T00:08:56.984434+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:08:56.984434+0000","last_undegraded":"2024-09-16T00:08:56.984434+0000","last_fullsized":"2024-09-16T00:08:56.984434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5111,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.008735+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:08:57.008735+0000","last_peered":"2024-09-16T00:08:57.008735+0000","last_clean":"2024-09-16T00:08:57.008735+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:08:57.008735+0000","last_undegraded":"2024-09-16T00:08:57.008735+0000","last_fullsized":"2024-09-16T00:08:57.008735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"57'5","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984850+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:08:56.984850+0000","last_peered":"2024-09-16T00:08:56.984850+0000","last_clean":"2024-09-16T00:08:56.984850+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:08:56.984850+0000","last_undegraded":"2024-09-16T00:08:56.984850+0000","last_fullsized":"2024-09-16T00:08:56.984850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":81483,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":82,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424201+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:08:54.424201+0000","last_peered":"2024-09-16T00:08:54.424201+0000","last_clean":"2024-09-16T00:08:54.424201+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:08:54.424201+0000","last_undegraded":"2024-09-16T00:08:54.424201+0000","last_fullsized":"2024-09-16T00:08:54.424201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"57'6","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.985051+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:08:56.985051+0000","last_peered":"2024-09-16T00:08:56.985051+0000","last_clean":"2024-09-16T00:08:56.985051+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:08:56.985051+0000","last_undegraded":"2024-09-16T00:08:56.985051+0000","last_fullsized":"2024-09-16T00:08:56.985051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22306,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":84,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980097+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:08:55.980097+0000","last_peered":"2024-09-16T00:08:55.980097+0000","last_clean":"2024-09-16T00:08:55.980097+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:08:55.980097+0000","last_undegraded":"2024-09-16T00:08:55.980097+0000","last_fullsized":"2024-09-16T00:08:55.980097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"57'8","reported_seq":92,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005154+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:08:57.005154+0000","last_peered":"2024-09-16T00:08:57.005154+0000","last_clean":"2024-09-16T00:08:57.005154+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:08:57.005154+0000","last_undegraded":"2024-09-16T00:08:57.005154+0000","last_fullsized":"2024-09-16T00:08:57.005154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32783,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":80,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424063+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:08:54.424063+0000","last_peered":"2024-09-16T00:08:54.424063+0000","last_clean":"2024-09-16T00:08:54.424063+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:08:54.424063+0000","last_undegraded":"2024-09-16T00:08:54.424063+0000","last_fullsized":"2024-09-16T00:08:54.424063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"58'7","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.029232+0000","last_change":"2024-09-16T00:08:47.953122+0000","last_active":"2024-09-16T00:08:57.029232+0000","last_peered":"2024-09-16T00:08:57.029232+0000","last_clean":"2024-09-16T00:08:57.029232+0000","last_became_active":"2024-09-16T00:08:47.952799+0000","last_became_peered":"2024-09-16T00:08:47.952799+0000","last_unstale":"2024-09-16T00:08:57.029232+0000","last_undegraded":"2024-09-16T00:08:57.029232+0000","last_fullsized":"2024-09-16T00:08:57.029232+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:03.447562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61360,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":241,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984180+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:08:56.984180+0000","last_peered":"2024-09-16T00:08:56.984180+0000","last_clean":"2024-09-16T00:08:56.984180+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:08:56.984180+0000","last_undegraded":"2024-09-16T00:08:56.984180+0000","last_fullsized":"2024-09-16T00:08:56.984180+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004860+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:08:57.004860+0000","last_peered":"2024-09-16T00:08:57.004860+0000","last_clean":"2024-09-16T00:08:57.004860+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:08:57.004860+0000","last_undegraded":"2024-09-16T00:08:57.004860+0000","last_fullsized":"2024-09-16T00:08:57.004860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991784+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:08:55.991784+0000","last_peered":"2024-09-16T00:08:55.991784+0000","last_clean":"2024-09-16T00:08:55.991784+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:08:55.991784+0000","last_undegraded":"2024-09-16T00:08:55.991784+0000","last_fullsized":"2024-09-16T00:08:55.991784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984166+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:08:56.984166+0000","last_peered":"2024-09-16T00:08:56.984166+0000","last_clean":"2024-09-16T00:08:56.984166+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:08:56.984166+0000","last_undegraded":"2024-09-16T00:08:56.984166+0000","last_fullsized":"2024-09-16T00:08:56.984166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.973713+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:08:54.973713+0000","last_peered":"2024-09-16T00:08:54.973713+0000","last_clean":"2024-09-16T00:08:54.973713+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:08:54.973713+0000","last_undegraded":"2024-09-16T00:08:54.973713+0000","last_fullsized":"2024-09-16T00:08:54.973713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":80,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424424+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:08:54.424424+0000","last_peered":"2024-09-16T00:08:54.424424+0000","last_clean":"2024-09-16T00:08:54.424424+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:08:54.424424+0000","last_undegraded":"2024-09-16T00:08:54.424424+0000","last_fullsized":"2024-09-16T00:08:54.424424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979829+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:08:55.979829+0000","last_peered":"2024-09-16T00:08:55.979829+0000","last_clean":"2024-09-16T00:08:55.979829+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:08:55.979829+0000","last_undegraded":"2024-09-16T00:08:55.979829+0000","last_fullsized":"2024-09-16T00:08:55.979829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.004966+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:08:57.004966+0000","last_peered":"2024-09-16T00:08:57.004966+0000","last_clean":"2024-09-16T00:08:57.004966+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:08:57.004966+0000","last_undegraded":"2024-09-16T00:08:57.004966+0000","last_fullsized":"2024-09-16T00:08:57.004966+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"58'3","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.015492+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:08:57.015492+0000","last_peered":"2024-09-16T00:08:57.015492+0000","last_clean":"2024-09-16T00:08:57.015492+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:08:57.015492+0000","last_undegraded":"2024-09-16T00:08:57.015492+0000","last_fullsized":"2024-09-16T00:08:57.015492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1838,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":83,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979978+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:08:55.979978+0000","last_peered":"2024-09-16T00:08:55.979978+0000","last_clean":"2024-09-16T00:08:55.979978+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:08:55.979978+0000","last_undegraded":"2024-09-16T00:08:55.979978+0000","last_fullsized":"2024-09-16T00:08:55.979978+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"57'5","reported_seq":89,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005147+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:08:57.005147+0000","last_peered":"2024-09-16T00:08:57.005147+0000","last_clean":"2024-09-16T00:08:57.005147+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:08:57.005147+0000","last_undegraded":"2024-09-16T00:08:57.005147+0000","last_fullsized":"2024-09-16T00:08:57.005147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13443,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.985261+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:08:56.985261+0000","last_peered":"2024-09-16T00:08:56.985261+0000","last_clean":"2024-09-16T00:08:56.985261+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:08:56.985261+0000","last_undegraded":"2024-09-16T00:08:56.985261+0000","last_fullsized":"2024-09-16T00:08:56.985261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979860+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:08:55.979860+0000","last_peered":"2024-09-16T00:08:55.979860+0000","last_clean":"2024-09-16T00:08:55.979860+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:08:55.979860+0000","last_undegraded":"2024-09-16T00:08:55.979860+0000","last_fullsized":"2024-09-16T00:08:55.979860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"56'4","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980204+0000","last_change":"2024-09-16T00:08:36.906253+0000","last_active":"2024-09-16T00:08:55.980204+0000","last_peered":"2024-09-16T00:08:55.980204+0000","last_clean":"2024-09-16T00:08:55.980204+0000","last_became_active":"2024-09-16T00:08:36.894363+0000","last_became_peered":"2024-09-16T00:08:36.894363+0000","last_unstale":"2024-09-16T00:08:55.980204+0000","last_undegraded":"2024-09-16T00:08:55.980204+0000","last_fullsized":"2024-09-16T00:08:55.980204+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:00.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.992092+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:08:55.992092+0000","last_peered":"2024-09-16T00:08:55.992092+0000","last_clean":"2024-09-16T00:08:55.992092+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:08:55.992092+0000","last_undegraded":"2024-09-16T00:08:55.992092+0000","last_fullsized":"2024-09-16T00:08:55.992092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983637+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:08:56.983637+0000","last_peered":"2024-09-16T00:08:56.983637+0000","last_clean":"2024-09-16T00:08:56.983637+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:08:56.983637+0000","last_undegraded":"2024-09-16T00:08:56.983637+0000","last_fullsized":"2024-09-16T00:08:56.983637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.974120+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:08:54.974120+0000","last_peered":"2024-09-16T00:08:54.974120+0000","last_clean":"2024-09-16T00:08:54.974120+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:08:54.974120+0000","last_undegraded":"2024-09-16T00:08:54.974120+0000","last_fullsized":"2024-09-16T00:08:54.974120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":85,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984026+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:08:56.984026+0000","last_peered":"2024-09-16T00:08:56.984026+0000","last_clean":"2024-09-16T00:08:56.984026+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:08:56.984026+0000","last_undegraded":"2024-09-16T00:08:56.984026+0000","last_fullsized":"2024-09-16T00:08:56.984026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"57'4","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.008594+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:08:57.008594+0000","last_peered":"2024-09-16T00:08:57.008594+0000","last_clean":"2024-09-16T00:08:57.008594+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:08:57.008594+0000","last_undegraded":"2024-09-16T00:08:57.008594+0000","last_fullsized":"2024-09-16T00:08:57.008594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22937,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":83,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.979932+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:08:55.979932+0000","last_peered":"2024-09-16T00:08:55.979932+0000","last_clean":"2024-09-16T00:08:55.979932+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:08:55.979932+0000","last_undegraded":"2024-09-16T00:08:55.979932+0000","last_fullsized":"2024-09-16T00:08:55.979932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991568+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:08:55.991568+0000","last_peered":"2024-09-16T00:08:55.991568+0000","last_clean":"2024-09-16T00:08:55.991568+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:08:55.991568+0000","last_undegraded":"2024-09-16T00:08:55.991568+0000","last_fullsized":"2024-09-16T00:08:55.991568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":83,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.980089+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:08:55.980089+0000","last_peered":"2024-09-16T00:08:55.980089+0000","last_clean":"2024-09-16T00:08:55.980089+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:08:55.980089+0000","last_undegraded":"2024-09-16T00:08:55.980089+0000","last_fullsized":"2024-09-16T00:08:55.980089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":82,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.992004+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:08:55.992004+0000","last_peered":"2024-09-16T00:08:55.992004+0000","last_clean":"2024-09-16T00:08:55.992004+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:08:55.992004+0000","last_undegraded":"2024-09-16T00:08:55.992004+0000","last_fullsized":"2024-09-16T00:08:55.992004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984458+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:08:56.984458+0000","last_peered":"2024-09-16T00:08:56.984458+0000","last_clean":"2024-09-16T00:08:56.984458+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:08:56.984458+0000","last_undegraded":"2024-09-16T00:08:56.984458+0000","last_fullsized":"2024-09-16T00:08:56.984458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"58'6","reported_seq":37,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.983319+0000","last_change":"2024-09-16T00:08:36.890347+0000","last_active":"2024-09-16T00:08:56.983319+0000","last_peered":"2024-09-16T00:08:56.983319+0000","last_clean":"2024-09-16T00:08:56.983319+0000","last_became_active":"2024-09-16T00:08:36.890009+0000","last_became_peered":"2024-09-16T00:08:36.890009+0000","last_unstale":"2024-09-16T00:08:56.983319+0000","last_undegraded":"2024-09-16T00:08:56.983319+0000","last_fullsized":"2024-09-16T00:08:56.983319+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:24.298978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19739,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"57'4","reported_seq":88,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005223+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:08:57.005223+0000","last_peered":"2024-09-16T00:08:57.005223+0000","last_clean":"2024-09-16T00:08:57.005223+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:08:57.005223+0000","last_undegraded":"2024-09-16T00:08:57.005223+0000","last_fullsized":"2024-09-16T00:08:57.005223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2557,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":87,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984758+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:08:56.984758+0000","last_peered":"2024-09-16T00:08:56.984758+0000","last_clean":"2024-09-16T00:08:56.984758+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:08:56.984758+0000","last_undegraded":"2024-09-16T00:08:56.984758+0000","last_fullsized":"2024-09-16T00:08:56.984758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"57'2","reported_seq":86,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:57.005087+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:08:57.005087+0000","last_peered":"2024-09-16T00:08:57.005087+0000","last_clean":"2024-09-16T00:08:57.005087+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:08:57.005087+0000","last_undegraded":"2024-09-16T00:08:57.005087+0000","last_fullsized":"2024-09-16T00:08:57.005087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":105,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-16T00:08:56.984904+0000","last_change":"2024-09-16T00:08:36.900704+0000","last_active":"2024-09-16T00:08:56.984904+0000","last_peered":"2024-09-16T00:08:56.984904+0000","last_clean":"2024-09-16T00:08:56.984904+0000","last_became_active":"2024-09-16T00:08:36.892520+0000","last_became_peered":"2024-09-16T00:08:36.892520+0000","last_unstale":"2024-09-16T00:08:56.984904+0000","last_undegraded":"2024-09-16T00:08:56.984904+0000","last_fullsized":"2024-09-16T00:08:56.984904+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:30.056041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":52,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.424040+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:08:54.424040+0000","last_peered":"2024-09-16T00:08:54.424040+0000","last_clean":"2024-09-16T00:08:54.424040+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:08:54.424040+0000","last_undegraded":"2024-09-16T00:08:54.424040+0000","last_fullsized":"2024-09-16T00:08:54.424040+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":85,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.992032+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:08:55.992032+0000","last_peered":"2024-09-16T00:08:55.992032+0000","last_clean":"2024-09-16T00:08:55.992032+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:08:55.992032+0000","last_undegraded":"2024-09-16T00:08:55.992032+0000","last_fullsized":"2024-09-16T00:08:55.992032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-16T00:08:55.991703+0000","last_change":"2024-09-16T00:08:36.889082+0000","last_active":"2024-09-16T00:08:55.991703+0000","last_peered":"2024-09-16T00:08:55.991703+0000","last_clean":"2024-09-16T00:08:55.991703+0000","last_became_active":"2024-09-16T00:08:36.888807+0000","last_became_peered":"2024-09-16T00:08:36.888807+0000","last_unstale":"2024-09-16T00:08:55.991703+0000","last_undegraded":"2024-09-16T00:08:55.991703+0000","last_fullsized":"2024-09-16T00:08:55.991703+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:22.825324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":85,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-16T00:08:54.974121+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:08:54.974121+0000","last_peered":"2024-09-16T00:08:54.974121+0000","last_clean":"2024-09-16T00:08:54.974121+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:08:54.974121+0000","last_undegraded":"2024-09-16T00:08:54.974121+0000","last_fullsized":"2024-09-16T00:08:54.974121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":7434637,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":7362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10895360,"data_stored":10700843,"data_compressed":270015,"data_compressed_allocated":344064,"data_compressed_original":1134592,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":22320,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":13110,"num_keys_recovered":4,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":40694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30268,"kb_used_data":3240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341572,"statfs":{"total":96636764160,"available":96605769728,"internally_reserved":0,"allocated":3317760,"data_stored":3592203,"data_compressed":73051,"data_compressed_allocated":94208,"data_compressed_original":831488,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":5144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":5267456,"data_stored":4867710,"data_compressed":57794,"data_compressed_allocated":73728,"data_compressed_original":237568,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343940,"statfs":{"total":96636764160,"available":96608194560,"internally_reserved":0,"allocated":897024,"data_stored":548421,"data_compressed":30333,"data_compressed_allocated":40960,"data_compressed_original":180224,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30468,"kb_used_data":3444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341372,"statfs":{"total":96636764160,"available":96605564928,"internally_reserved":0,"allocated":3526656,"data_stored":3061123,"data_compressed":69574,"data_compressed_allocated":86016,"data_compressed_original":237568,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":655360,"data_stored":291719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":647168,"data_stored":285324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":716800,"data_stored":876170,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":626688,"data_stored":279070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":17576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":21522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2936832,"data_stored":2786732,"data_compressed":69574,"data_compressed_allocated":86016,"data_compressed_original":237568,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":471040,"data_stored":398309,"data_compressed":60175,"data_compressed_allocated":77824,"data_compressed_original":237568,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4579328,"data_stored":4549565,"data_compressed":57794,"data_compressed_allocated":73728,"data_compressed_original":237568,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2908160,"data_stored":2966237,"data_compressed":82472,"data_compressed_allocated":106496,"data_compressed_original":421888,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:09:01.926 INFO:tasks.thrashosds.thrasher:Setting 3.2d to [7, 5] 2024-09-16T00:09:01.926 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.2d', '7', '5'] 2024-09-16T00:09:01.926 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.2d 7 5 2024-09-16T00:09:01.987 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:01.987 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:01.987 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:01.987 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:01.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.988 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'66)", 2024-09-16T00:09:01.988 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:09:01.988 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 97.819499934000007, 2024-09-16T00:09:01.988 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032925509999999999, 2024-09-16T00:09:01.988 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:01.988 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:01.988 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.988 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:01.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.988 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:01.988 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:09:01.988 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:01.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.989 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.989 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:01.989 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:09:01.989 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:01.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.989 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.989 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:01.989 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166177+0000", 2024-09-16T00:09:01.989 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:09:01.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.989 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.989 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:01.989 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166216+0000", 2024-09-16T00:09:01.990 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9008000000000003e-05 2024-09-16T00:09:01.990 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.990 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:01.990 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166226+0000", 2024-09-16T00:09:01.990 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0081999999999999e-05 2024-09-16T00:09:01.990 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.990 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.990 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166230+0000", 2024-09-16T00:09:01.990 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5079999999999999e-06 2024-09-16T00:09:01.990 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.990 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:01.991 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166259+0000", 2024-09-16T00:09:01.991 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8928000000000002e-05 2024-09-16T00:09:01.991 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.991 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.991 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:01.991 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166271+0000", 2024-09-16T00:09:01.991 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1531e-05 2024-09-16T00:09:01.991 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.991 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.991 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:01.991 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169449+0000", 2024-09-16T00:09:01.991 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003178143 2024-09-16T00:09:01.991 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.991 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.992 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:01.992 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169468+0000", 2024-09-16T00:09:01.992 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9700000000000001e-05 2024-09-16T00:09:01.992 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.992 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.992 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.992 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.992 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.992 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'66)", 2024-09-16T00:09:01.992 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:09:01.992 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 97.819038474999999, 2024-09-16T00:09:01.992 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003179374, 2024-09-16T00:09:01.992 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:01.993 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:01.993 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.993 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:01.993 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.993 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:01.993 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:09:01.993 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:01.993 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.993 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.993 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:01.993 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:09:01.993 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:01.993 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.993 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.994 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:01.994 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166638+0000", 2024-09-16T00:09:01.994 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2300000000000002e-07 2024-09-16T00:09:01.994 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.994 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.994 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:01.994 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166641+0000", 2024-09-16T00:09:01.994 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0079999999999998e-06 2024-09-16T00:09:01.994 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.994 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.994 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:01.994 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166646+0000", 2024-09-16T00:09:01.994 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4970000000000001e-06 2024-09-16T00:09:01.994 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.995 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.995 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.995 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166650+0000", 2024-09-16T00:09:01.995 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5729999999999999e-06 2024-09-16T00:09:01.995 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.995 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.995 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:01.995 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166682+0000", 2024-09-16T00:09:01.995 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2218999999999998e-05 2024-09-16T00:09:01.995 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.995 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.995 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:01.995 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166694+0000", 2024-09-16T00:09:01.995 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1902999999999999e-05 2024-09-16T00:09:01.996 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.996 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.996 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:01.996 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169784+0000", 2024-09-16T00:09:01.996 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003089545 2024-09-16T00:09:01.996 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.996 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.996 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:01.996 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169817+0000", 2024-09-16T00:09:01.996 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3206e-05 2024-09-16T00:09:01.996 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.996 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:01.996 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:01.996 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.997 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.997 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:117 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'66)", 2024-09-16T00:09:01.997 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:09:01.997 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 97.818594415000007, 2024-09-16T00:09:01.997 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00275326, 2024-09-16T00:09:01.997 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:01.997 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:01.997 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:01.997 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:01.997 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.997 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:01.997 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:09:01.997 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:01.997 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.998 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.998 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:01.998 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:09:01.998 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:01.998 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.998 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.998 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:01.998 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167082+0000", 2024-09-16T00:09:01.998 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:09:01.998 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.998 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.998 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:01.998 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167086+0000", 2024-09-16T00:09:01.998 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1559999999999997e-06 2024-09-16T00:09:01.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:01.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167096+0000", 2024-09-16T00:09:01.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8390000000000002e-06 2024-09-16T00:09:01.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:01.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167101+0000", 2024-09-16T00:09:01.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:09:01.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:01.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:01.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:01.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167130+0000", 2024-09-16T00:09:02.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9649e-05 2024-09-16T00:09:02.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.000 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167142+0000", 2024-09-16T00:09:02.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1435e-05 2024-09-16T00:09:02.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.000 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169818+0000", 2024-09-16T00:09:02.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002676056 2024-09-16T00:09:02.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.001 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169835+0000", 2024-09-16T00:09:02.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6980999999999999e-05 2024-09-16T00:09:02.001 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.001 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.001 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.001 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:130 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'88, mlcod=17'85)", 2024-09-16T00:09:02.001 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:02.001 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 97.375795085999997, 2024-09-16T00:09:02.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002880456, 2024-09-16T00:09:02.001 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.001 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.001 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.002 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.002 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.002 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:02.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.002 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.002 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:02.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.002 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.002 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:02.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5699999999999998e-07 2024-09-16T00:09:02.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609885+0000", 2024-09-16T00:09:02.003 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.608e-06 2024-09-16T00:09:02.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609891+0000", 2024-09-16T00:09:02.003 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:09:02.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609895+0000", 2024-09-16T00:09:02.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.54e-06 2024-09-16T00:09:02.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.004 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609931+0000", 2024-09-16T00:09:02.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5893999999999999e-05 2024-09-16T00:09:02.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.004 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609945+0000", 2024-09-16T00:09:02.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4259e-05 2024-09-16T00:09:02.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.004 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.005 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612721+0000", 2024-09-16T00:09:02.005 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027757849999999998 2024-09-16T00:09:02.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.005 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.005 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612761+0000", 2024-09-16T00:09:02.005 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0638e-05 2024-09-16T00:09:02.005 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.005 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.005 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.005 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-16T00:09:02.005 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:09:02.006 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 97.375411112999998, 2024-09-16T00:09:02.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028016529999999999, 2024-09-16T00:09:02.006 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.006 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.006 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.006 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:09:02.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:09:02.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:09:02.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.63e-07 2024-09-16T00:09:02.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610344+0000", 2024-09-16T00:09:02.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8996000000000001e-05 2024-09-16T00:09:02.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610352+0000", 2024-09-16T00:09:02.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:09:02.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610356+0000", 2024-09-16T00:09:02.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0790000000000002e-06 2024-09-16T00:09:02.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610388+0000", 2024-09-16T00:09:02.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1541000000000002e-05 2024-09-16T00:09:02.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.009 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.009 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610400+0000", 2024-09-16T00:09:02.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2075e-05 2024-09-16T00:09:02.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.009 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.009 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613045+0000", 2024-09-16T00:09:02.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002645213 2024-09-16T00:09:02.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.009 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.009 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613066+0000", 2024-09-16T00:09:02.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1353e-05 2024-09-16T00:09:02.010 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.010 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.010 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.010 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-16T00:09:02.010 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:09:02.010 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 97.372151613, 2024-09-16T00:09:02.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027254850000000001, 2024-09-16T00:09:02.010 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.010 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.010 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.010 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.011 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.011 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:09:02.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.011 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.011 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:09:02.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.011 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.011 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613525+0000", 2024-09-16T00:09:02.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2800000000000002e-07 2024-09-16T00:09:02.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.012 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613528+0000", 2024-09-16T00:09:02.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9730000000000002e-06 2024-09-16T00:09:02.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.012 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613534+0000", 2024-09-16T00:09:02.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4339999999999997e-06 2024-09-16T00:09:02.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.012 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613538+0000", 2024-09-16T00:09:02.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5149999999999998e-06 2024-09-16T00:09:02.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613566+0000", 2024-09-16T00:09:02.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8104000000000001e-05 2024-09-16T00:09:02.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613578+0000", 2024-09-16T00:09:02.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2404e-05 2024-09-16T00:09:02.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616217+0000", 2024-09-16T00:09:02.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026389830000000001 2024-09-16T00:09:02.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616250+0000", 2024-09-16T00:09:02.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2644000000000001e-05 2024-09-16T00:09:02.015 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.015 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.015 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.015 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.015 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.015 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:139 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'97, mlcod=18'87)", 2024-09-16T00:09:02.015 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:09:02.015 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 97.369332764999996, 2024-09-16T00:09:02.015 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030908139999999999, 2024-09-16T00:09:02.015 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.015 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.015 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.015 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.016 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.016 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.016 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:09:02.016 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.016 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.016 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.016 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.016 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:09:02.017 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.017 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.017 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.017 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616344+0000", 2024-09-16T00:09:02.017 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3000000000000001e-07 2024-09-16T00:09:02.018 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.018 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.018 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616347+0000", 2024-09-16T00:09:02.018 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7740000000000002e-06 2024-09-16T00:09:02.018 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.018 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.018 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616355+0000", 2024-09-16T00:09:02.018 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4699999999999996e-06 2024-09-16T00:09:02.018 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.019 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.019 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616358+0000", 2024-09-16T00:09:02.019 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5899999999999999e-06 2024-09-16T00:09:02.019 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.019 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.019 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.019 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616393+0000", 2024-09-16T00:09:02.019 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4529000000000001e-05 2024-09-16T00:09:02.019 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.019 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.019 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.020 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616404+0000", 2024-09-16T00:09:02.020 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0726e-05 2024-09-16T00:09:02.020 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.020 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.020 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.020 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619411+0000", 2024-09-16T00:09:02.020 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030076249999999999 2024-09-16T00:09:02.020 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.020 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.020 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.020 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619434+0000", 2024-09-16T00:09:02.020 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-05 2024-09-16T00:09:02.021 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.021 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.021 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.021 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.021 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.021 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:140 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'98, mlcod=18'89)", 2024-09-16T00:09:02.021 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:09:02.021 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 97.368691372000001, 2024-09-16T00:09:02.021 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027276150000000001, 2024-09-16T00:09:02.021 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.021 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.022 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.022 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.022 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.022 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.022 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:09:02.022 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.022 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.022 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.022 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.022 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:09:02.023 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.023 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.023 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.023 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.023 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616985+0000", 2024-09-16T00:09:02.023 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4099999999999999e-07 2024-09-16T00:09:02.023 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.023 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.023 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.024 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617055+0000", 2024-09-16T00:09:02.024 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9923000000000001e-05 2024-09-16T00:09:02.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.024 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617063+0000", 2024-09-16T00:09:02.024 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:09:02.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.024 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617070+0000", 2024-09-16T00:09:02.025 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9090000000000003e-06 2024-09-16T00:09:02.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.025 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.025 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.025 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617102+0000", 2024-09-16T00:09:02.025 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2620999999999998e-05 2024-09-16T00:09:02.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.025 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.025 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.025 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617114+0000", 2024-09-16T00:09:02.025 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1228e-05 2024-09-16T00:09:02.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.026 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.026 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.026 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619690+0000", 2024-09-16T00:09:02.026 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025763890000000001 2024-09-16T00:09:02.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.026 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.026 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.026 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619712+0000", 2024-09-16T00:09:02.026 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2171000000000002e-05 2024-09-16T00:09:02.026 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.026 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.027 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.027 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.027 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:02.027 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:09:02.027 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 83.369862272000006, 2024-09-16T00:09:02.027 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027355320000000002, 2024-09-16T00:09:02.027 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.027 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.027 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.027 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.027 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.027 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:09:02.028 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.028 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.028 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:09:02.028 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.028 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.028 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615815+0000", 2024-09-16T00:09:02.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1769999999999999e-06 2024-09-16T00:09:02.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.029 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.029 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615825+0000", 2024-09-16T00:09:02.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9000000000000001e-06 2024-09-16T00:09:02.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.029 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.029 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615840+0000", 2024-09-16T00:09:02.030 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4854e-05 2024-09-16T00:09:02.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.030 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615848+0000", 2024-09-16T00:09:02.030 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0940000000000008e-06 2024-09-16T00:09:02.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.031 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.031 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615926+0000", 2024-09-16T00:09:02.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8176999999999994e-05 2024-09-16T00:09:02.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.031 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.031 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615945+0000", 2024-09-16T00:09:02.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9663000000000001e-05 2024-09-16T00:09:02.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.032 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.032 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.032 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618509+0000", 2024-09-16T00:09:02.032 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002563802 2024-09-16T00:09:02.032 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.032 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.032 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.032 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618549+0000", 2024-09-16T00:09:02.032 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9864999999999999e-05 2024-09-16T00:09:02.032 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.033 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.033 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.033 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.033 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.033 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:02.033 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:09:02.033 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 83.369773132000006, 2024-09-16T00:09:02.033 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061593860000000002, 2024-09-16T00:09:02.033 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.033 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.034 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.034 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.034 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.034 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:09:02.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.034 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.034 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:09:02.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615907+0000", 2024-09-16T00:09:02.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9389999999999999e-06 2024-09-16T00:09:02.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615920+0000", 2024-09-16T00:09:02.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2989999999999999e-05 2024-09-16T00:09:02.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.036 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.036 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615931+0000", 2024-09-16T00:09:02.036 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.164e-05 2024-09-16T00:09:02.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.036 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.036 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.036 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615939+0000", 2024-09-16T00:09:02.036 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8029999999999997e-06 2024-09-16T00:09:02.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.036 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.036 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.036 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616310+0000", 2024-09-16T00:09:02.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00037111400000000003 2024-09-16T00:09:02.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.037 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.037 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616406+0000", 2024-09-16T00:09:02.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5372999999999996e-05 2024-09-16T00:09:02.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.037 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:02.037 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616514+0000", 2024-09-16T00:09:02.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108564 2024-09-16T00:09:02.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.038 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.038 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618158+0000", 2024-09-16T00:09:02.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016442449999999999 2024-09-16T00:09:02.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.038 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.038 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621972+0000", 2024-09-16T00:09:02.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038132299999999999 2024-09-16T00:09:02.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.039 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.039 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622018+0000", 2024-09-16T00:09:02.039 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6239000000000002e-05 2024-09-16T00:09:02.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.039 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.039 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.039 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622062+0000", 2024-09-16T00:09:02.039 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4249000000000003e-05 2024-09-16T00:09:02.039 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.039 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.039 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.039 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.039 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:02.040 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:09:02.040 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 83.369719664000002, 2024-09-16T00:09:02.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006258914, 2024-09-16T00:09:02.040 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.040 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.040 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.040 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.040 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.040 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:09:02.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.040 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.041 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:09:02.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.041 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.041 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.041 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615957+0000", 2024-09-16T00:09:02.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1399999999999997e-07 2024-09-16T00:09:02.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.041 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.041 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.042 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615960+0000", 2024-09-16T00:09:02.042 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0860000000000002e-06 2024-09-16T00:09:02.042 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.042 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.042 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615967+0000", 2024-09-16T00:09:02.042 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9059999999999996e-06 2024-09-16T00:09:02.042 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.042 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615972+0000", 2024-09-16T00:09:02.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5679999999999999e-06 2024-09-16T00:09:02.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616717+0000", 2024-09-16T00:09:02.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074438 2024-09-16T00:09:02.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616781+0000", 2024-09-16T00:09:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4808999999999996e-05 2024-09-16T00:09:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616885+0000", 2024-09-16T00:09:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010404900000000001 2024-09-16T00:09:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:09:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012773870000000001 2024-09-16T00:09:02.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622116+0000", 2024-09-16T00:09:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039533470000000003 2024-09-16T00:09:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622173+0000", 2024-09-16T00:09:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6347000000000001e-05 2024-09-16T00:09:02.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622215+0000", 2024-09-16T00:09:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2521000000000002e-05 2024-09-16T00:09:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.046 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:02.047 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:02.047 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 83.369695926999995, 2024-09-16T00:09:02.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041725529999999999, 2024-09-16T00:09:02.047 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.047 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.047 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.047 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:02.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:02.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.049 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.049 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:02.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8799999999999998e-07 2024-09-16T00:09:02.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.049 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.049 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615990+0000", 2024-09-16T00:09:02.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2150000000000007e-06 2024-09-16T00:09:02.050 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.050 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.050 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.050 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:09:02.050 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7159999999999997e-06 2024-09-16T00:09:02.050 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.051 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.051 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616001+0000", 2024-09-16T00:09:02.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9339999999999997e-06 2024-09-16T00:09:02.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616293+0000", 2024-09-16T00:09:02.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029210899999999998 2024-09-16T00:09:02.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616401+0000", 2024-09-16T00:09:02.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108087 2024-09-16T00:09:02.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:02.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616561+0000", 2024-09-16T00:09:02.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015940600000000001 2024-09-16T00:09:02.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618146+0000", 2024-09-16T00:09:02.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015857040000000001 2024-09-16T00:09:02.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.054 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620091+0000", 2024-09-16T00:09:02.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001944837 2024-09-16T00:09:02.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.054 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.054 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620126+0000", 2024-09-16T00:09:02.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4242999999999997e-05 2024-09-16T00:09:02.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.055 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.055 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620152+0000", 2024-09-16T00:09:02.055 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6914e-05 2024-09-16T00:09:02.055 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.055 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.055 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.055 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.055 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:02.055 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:02.055 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 83.369657701999998, 2024-09-16T00:09:02.056 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00630628, 2024-09-16T00:09:02.056 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.056 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.056 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.056 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.056 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.056 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.056 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:02.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:02.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:09:02.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5999999999999999e-07 2024-09-16T00:09:02.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616022+0000", 2024-09-16T00:09:02.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0149999999999999e-06 2024-09-16T00:09:02.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616031+0000", 2024-09-16T00:09:02.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9770000000000006e-06 2024-09-16T00:09:02.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.062 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616036+0000", 2024-09-16T00:09:02.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2639999999999999e-06 2024-09-16T00:09:02.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.062 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.062 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617057+0000", 2024-09-16T00:09:02.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010212999999999999 2024-09-16T00:09:02.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617146+0000", 2024-09-16T00:09:02.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9234000000000007e-05 2024-09-16T00:09:02.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:02.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617232+0000", 2024-09-16T00:09:02.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5218999999999998e-05 2024-09-16T00:09:02.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619107+0000", 2024-09-16T00:09:02.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001875016 2024-09-16T00:09:02.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622259+0000", 2024-09-16T00:09:02.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031527930000000001 2024-09-16T00:09:02.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622290+0000", 2024-09-16T00:09:02.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1022999999999998e-05 2024-09-16T00:09:02.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622324+0000", 2024-09-16T00:09:02.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3979000000000001e-05 2024-09-16T00:09:02.066 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.066 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.066 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.066 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:02.067 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:09:02.067 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 83.3696324, 2024-09-16T00:09:02.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042412739999999997, 2024-09-16T00:09:02.067 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.067 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.067 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.067 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:09:02.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.068 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.068 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.068 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.068 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:09:02.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.068 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616044+0000", 2024-09-16T00:09:02.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:09:02.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616052+0000", 2024-09-16T00:09:02.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.439e-06 2024-09-16T00:09:02.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616059+0000", 2024-09-16T00:09:02.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1389999999999998e-06 2024-09-16T00:09:02.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616065+0000", 2024-09-16T00:09:02.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:09:02.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616823+0000", 2024-09-16T00:09:02.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075781199999999996 2024-09-16T00:09:02.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616918+0000", 2024-09-16T00:09:02.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5174999999999996e-05 2024-09-16T00:09:02.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:02.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617019+0000", 2024-09-16T00:09:02.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101161 2024-09-16T00:09:02.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618503+0000", 2024-09-16T00:09:02.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014836750000000001 2024-09-16T00:09:02.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620201+0000", 2024-09-16T00:09:02.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00169805 2024-09-16T00:09:02.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620255+0000", 2024-09-16T00:09:02.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4499000000000001e-05 2024-09-16T00:09:02.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.073 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.073 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620285+0000", 2024-09-16T00:09:02.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9349e-05 2024-09-16T00:09:02.073 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.073 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.073 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.073 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:02.073 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:09:02.073 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 83.369603409000007, 2024-09-16T00:09:02.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063682799999999996, 2024-09-16T00:09:02.074 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.074 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.074 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:09:02.074 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:09:02.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:09:02.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:09:02.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9900000000000001e-07 2024-09-16T00:09:02.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616079+0000", 2024-09-16T00:09:02.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9009999999999999e-06 2024-09-16T00:09:02.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616086+0000", 2024-09-16T00:09:02.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.776e-06 2024-09-16T00:09:02.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616091+0000", 2024-09-16T00:09:02.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5600000000000001e-06 2024-09-16T00:09:02.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618339+0000", 2024-09-16T00:09:02.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022476179999999998 2024-09-16T00:09:02.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.077 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.077 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.077 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:09:02.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9025999999999997e-05 2024-09-16T00:09:02.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.077 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.077 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.077 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622441+0000", 2024-09-16T00:09:02.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040029999999999996 2024-09-16T00:09:02.077 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.077 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.078 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.078 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:221 3.1b 3:dad7a251:::100000002d7.00000000:head [write 0~30412] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:02.078 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:09:02.078 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.2363952190000003, 2024-09-16T00:09:02.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034396769999999999, 2024-09-16T00:09:02.078 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.078 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.078 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.078 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:02.078 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:02.078 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:02.078 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 221 2024-09-16T00:09:02.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.079 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.079 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:09:02.079 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.079 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:09:02.079 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.080 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749282+0000", 2024-09-16T00:09:02.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3769999999999999e-06 2024-09-16T00:09:02.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.080 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.080 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749300+0000", 2024-09-16T00:09:02.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8204999999999999e-05 2024-09-16T00:09:02.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.080 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.080 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749311+0000", 2024-09-16T00:09:02.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0599e-05 2024-09-16T00:09:02.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749318+0000", 2024-09-16T00:09:02.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9789999999999996e-06 2024-09-16T00:09:02.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749367+0000", 2024-09-16T00:09:02.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9382999999999998e-05 2024-09-16T00:09:02.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749450+0000", 2024-09-16T00:09:02.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-05 2024-09-16T00:09:02.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:02.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749526+0000", 2024-09-16T00:09:02.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5564999999999999e-05 2024-09-16T00:09:02.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.750601+0000", 2024-09-16T00:09:02.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010751199999999999 2024-09-16T00:09:02.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752665+0000", 2024-09-16T00:09:02.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00206442 2024-09-16T00:09:02.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752696+0000", 2024-09-16T00:09:02.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0476000000000001e-05 2024-09-16T00:09:02.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752720+0000", 2024-09-16T00:09:02.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4482999999999999e-05 2024-09-16T00:09:02.084 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.084 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.084 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.084 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.084 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.084 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:244 3.1b 3:da01a9ef:::100000002f1.00000000:head [write 0~9324] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:02.084 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.800988+0000", 2024-09-16T00:09:02.084 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.1846875319999999, 2024-09-16T00:09:02.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026477689999999999, 2024-09-16T00:09:02.084 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.084 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.084 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.084 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:02.084 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:02.085 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:02.085 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 244 2024-09-16T00:09:02.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.085 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.085 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.800988+0000", 2024-09-16T00:09:02.085 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.085 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.800988+0000", 2024-09-16T00:09:02.085 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.086 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.800989+0000", 2024-09-16T00:09:02.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.33e-07 2024-09-16T00:09:02.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.086 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801004+0000", 2024-09-16T00:09:02.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4531e-05 2024-09-16T00:09:02.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.086 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801015+0000", 2024-09-16T00:09:02.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0988000000000001e-05 2024-09-16T00:09:02.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.087 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.087 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801020+0000", 2024-09-16T00:09:02.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2820000000000004e-06 2024-09-16T00:09:02.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.087 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.087 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801072+0000", 2024-09-16T00:09:02.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1830999999999999e-05 2024-09-16T00:09:02.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.087 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801142+0000", 2024-09-16T00:09:02.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0182e-05 2024-09-16T00:09:02.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:02.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801218+0000", 2024-09-16T00:09:02.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6093999999999993e-05 2024-09-16T00:09:02.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801707+0000", 2024-09-16T00:09:02.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000488837 2024-09-16T00:09:02.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.089 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803548+0000", 2024-09-16T00:09:02.089 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018406519999999999 2024-09-16T00:09:02.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.089 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803600+0000", 2024-09-16T00:09:02.089 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2429999999999998e-05 2024-09-16T00:09:02.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.089 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803636+0000", 2024-09-16T00:09:02.089 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6009e-05 2024-09-16T00:09:02.090 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.090 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.090 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.090 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.090 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:323 3.3 e58/20 3:c319d237:::10000000354.00000000:head v 58'3, mlcod=57'1)", 2024-09-16T00:09:02.090 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:09:02.090 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.9530154480000004, 2024-09-16T00:09:02.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034011000000000002, 2024-09-16T00:09:02.090 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.090 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.090 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.090 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.091 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.091 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:09:02.091 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.091 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.091 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:09:02.091 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.091 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.091 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032661+0000", 2024-09-16T00:09:02.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7000000000000003e-07 2024-09-16T00:09:02.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.092 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.092 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032670+0000", 2024-09-16T00:09:02.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.602e-06 2024-09-16T00:09:02.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.092 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.092 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032682+0000", 2024-09-16T00:09:02.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1754e-05 2024-09-16T00:09:02.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032688+0000", 2024-09-16T00:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8140000000000003e-06 2024-09-16T00:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032740+0000", 2024-09-16T00:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1981000000000003e-05 2024-09-16T00:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032765+0000", 2024-09-16T00:09:02.093 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4097e-05 2024-09-16T00:09:02.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.094 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.094 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.094 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.036023+0000", 2024-09-16T00:09:02.094 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032585240000000001 2024-09-16T00:09:02.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.094 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.094 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.094 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.036061+0000", 2024-09-16T00:09:02.094 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8457999999999997e-05 2024-09-16T00:09:02.094 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.094 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.094 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.095 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.095 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:664 3.39 e59/20 3:9c9d3e51:::100000004f6.00000000:head v 59'7, mlcod=58'5)", 2024-09-16T00:09:02.095 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:02.095 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.765346289, 2024-09-16T00:09:02.095 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035253329999999999, 2024-09-16T00:09:02.095 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.095 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.095 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.095 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.095 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.095 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.095 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:02.095 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.096 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.096 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.096 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.096 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:02.096 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.096 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.096 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.096 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.096 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:02.096 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2399999999999996e-07 2024-09-16T00:09:02.096 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.096 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.096 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.096 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220338+0000", 2024-09-16T00:09:02.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0120000000000005e-06 2024-09-16T00:09:02.097 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.097 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.097 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.097 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220353+0000", 2024-09-16T00:09:02.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4718999999999999e-05 2024-09-16T00:09:02.097 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.097 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.097 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.097 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220361+0000", 2024-09-16T00:09:02.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6559999999999992e-06 2024-09-16T00:09:02.097 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.097 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.097 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220422+0000", 2024-09-16T00:09:02.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.071e-05 2024-09-16T00:09:02.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220446+0000", 2024-09-16T00:09:02.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4556e-05 2024-09-16T00:09:02.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.223813+0000", 2024-09-16T00:09:02.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033666439999999998 2024-09-16T00:09:02.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.099 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.223855+0000", 2024-09-16T00:09:02.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2111999999999999e-05 2024-09-16T00:09:02.099 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.099 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.099 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.099 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:681 3.17 3:ea55746f:::1000000050b.00000000:head [write 0~1071] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:02.099 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:02.099 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.7206033239999998, 2024-09-16T00:09:02.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036588620000000001, 2024-09-16T00:09:02.099 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.100 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.100 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.100 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:02.100 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:02.100 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:02.100 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 681 2024-09-16T00:09:02.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.100 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.100 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:02.100 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:02.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:02.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7600000000000006e-07 2024-09-16T00:09:02.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265080+0000", 2024-09-16T00:09:02.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8759999999999999e-06 2024-09-16T00:09:02.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.102 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.102 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.102 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265094+0000", 2024-09-16T00:09:02.102 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.411e-05 2024-09-16T00:09:02.102 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.102 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.102 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.102 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265102+0000", 2024-09-16T00:09:02.102 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9640000000000003e-06 2024-09-16T00:09:02.102 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.102 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.102 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.102 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265156+0000", 2024-09-16T00:09:02.102 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3375000000000003e-05 2024-09-16T00:09:02.103 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.103 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.103 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.103 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265229+0000", 2024-09-16T00:09:02.103 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3147999999999998e-05 2024-09-16T00:09:02.103 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.103 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.103 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:02.103 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265304+0000", 2024-09-16T00:09:02.103 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4568999999999997e-05 2024-09-16T00:09:02.103 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.103 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.103 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.103 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265747+0000", 2024-09-16T00:09:02.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00044363099999999999 2024-09-16T00:09:02.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.104 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.104 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.104 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.268635+0000", 2024-09-16T00:09:02.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028873459999999998 2024-09-16T00:09:02.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.104 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.104 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.104 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.268688+0000", 2024-09-16T00:09:02.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3155999999999997e-05 2024-09-16T00:09:02.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.104 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.268731+0000", 2024-09-16T00:09:02.105 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3711000000000002e-05 2024-09-16T00:09:02.105 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.105 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.105 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.105 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.105 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.105 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:02.106 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T00:09:02.106 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:09:02.107 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:02.106+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:02.108 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:09:02.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.078+0000 7f00e645e640 1 Processor -- start 2024-09-16T00:09:02.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.078+0000 7f00e645e640 1 -- start start 2024-09-16T00:09:02.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.079+0000 7f00e645e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f00e0068060 0x7f00e015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:02.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.079+0000 7f00e645e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f00e00ccc90 0x7f00e00cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:02.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.079+0000 7f00e645e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e00cc4b0 con 0x7f00e0154c20 2024-09-16T00:09:02.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.079+0000 7f00e645e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f00e00c5830 con 0x7f00e0068060 2024-09-16T00:09:02.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.079+0000 7f00e645e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f00e015b7c0 con 0x7f00e00ccc90 2024-09-16T00:09:02.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.079+0000 7f00df7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f00e0068060 0x7f00e015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:02.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.079+0000 7f00dffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f00e0154c20 0x7f00e00c5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:60354/0 (socket says 172.21.15.139:60354) 2024-09-16T00:09:02.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.079+0000 7f00dffff640 1 -- 172.21.15.139:0/3823155832 learned_addr learned my addr 172.21.15.139:0/3823155832 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:02.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.079+0000 7f00e49fb640 1 --2- 172.21.15.139:0/3823155832 >> v2:172.21.15.164:3300/0 conn(0x7f00e00ccc90 0x7f00e00cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:02.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.080+0000 7f00e49fb640 1 -- 172.21.15.139:0/3823155832 >> v2:172.21.15.139:3300/0 conn(0x7f00e0068060 msgr2=0x7f00e015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:02.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.080+0000 7f00e49fb640 1 --2- 172.21.15.139:0/3823155832 >> v2:172.21.15.139:3300/0 conn(0x7f00e0068060 0x7f00e015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:02.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.080+0000 7f00e49fb640 1 -- 172.21.15.139:0/3823155832 >> v1:172.21.15.139:6789/0 conn(0x7f00e0154c20 legacy=0x7f00e00c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:02.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.080+0000 7f00deffd640 1 -- 172.21.15.139:0/3823155832 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2629060186 0 0) 0x7f00e00cc4b0 con 0x7f00e0154c20 2024-09-16T00:09:02.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.080+0000 7f00e49fb640 1 -- 172.21.15.139:0/3823155832 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e00cc790 con 0x7f00e00ccc90 2024-09-16T00:09:02.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.080+0000 7f00e49fb640 1 --2- 172.21.15.139:0/3823155832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e00ccc90 0x7f00e00cd0a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f00e00c7300 tx=0x7f00d802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b461935612b603f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:02.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.081+0000 7f00deffd640 1 -- 172.21.15.139:0/3823155832 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8007740 con 0x7f00e00ccc90 2024-09-16T00:09:02.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.081+0000 7f00deffd640 1 -- 172.21.15.139:0/3823155832 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d80078e0 con 0x7f00e00ccc90 2024-09-16T00:09:02.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.081+0000 7f00deffd640 1 -- 172.21.15.139:0/3823155832 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d8007cb0 con 0x7f00e00ccc90 2024-09-16T00:09:02.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.081+0000 7f00e645e640 1 -- 172.21.15.139:0/3823155832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e00ccc90 msgr2=0x7f00e00cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:02.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.082+0000 7f00e645e640 1 --2- 172.21.15.139:0/3823155832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e00ccc90 0x7f00e00cd0a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f00e00c7300 tx=0x7f00d802a680 comp rx=0 tx=0).stop 2024-09-16T00:09:02.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.082+0000 7f00e645e640 1 -- 172.21.15.139:0/3823155832 shutdown_connections 2024-09-16T00:09:02.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.082+0000 7f00e645e640 1 --2- 172.21.15.139:0/3823155832 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e00ccc90 0x7f00e00cd0a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:02.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.082+0000 7f00e645e640 1 --2- 172.21.15.139:0/3823155832 >> v2:172.21.15.139:3300/0 conn(0x7f00e0068060 0x7f00e015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:02.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.082+0000 7f00e645e640 1 -- 172.21.15.139:0/3823155832 >> 172.21.15.139:0/3823155832 conn(0x7f00e006ca40 msgr2=0x7f00e0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:02.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.082+0000 7f00e645e640 1 -- 172.21.15.139:0/3823155832 shutdown_connections 2024-09-16T00:09:02.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.082+0000 7f00e645e640 1 -- 172.21.15.139:0/3823155832 wait complete. 2024-09-16T00:09:02.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.083+0000 7f00e645e640 1 Processor -- start 2024-09-16T00:09:02.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.083+0000 7f00e645e640 1 -- start start 2024-09-16T00:09:02.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00e645e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00e0068060 0x7f00e0156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:02.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00e645e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00e00ccc90 0x7f00e015cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:02.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00e645e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e0154c20 0x7f00e015d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:02.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00e645e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f00e015e0d0 con 0x7f00e00ccc90 2024-09-16T00:09:02.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00e645e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f00e00cc260 con 0x7f00e0154c20 2024-09-16T00:09:02.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00e645e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f00e015e3b0 con 0x7f00e0068060 2024-09-16T00:09:02.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00df7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00e00ccc90 0x7f00e015cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:02.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00df7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00e00ccc90 0x7f00e015cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:41998/0 (socket says 172.21.15.139:41998) 2024-09-16T00:09:02.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00df7fe640 1 -- 172.21.15.139:0/371335274 learned_addr learned my addr 172.21.15.139:0/371335274 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:02.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00e49fb640 1 --2- 172.21.15.139:0/371335274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e0154c20 0x7f00e015d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:02.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00dffff640 1 --2- 172.21.15.139:0/371335274 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00e0068060 0x7f00e0156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:02.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00df7fe640 1 -- 172.21.15.139:0/371335274 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00e0068060 msgr2=0x7f00e0156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:02.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00df7fe640 1 --2- 172.21.15.139:0/371335274 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00e0068060 0x7f00e0156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:02.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00df7fe640 1 -- 172.21.15.139:0/371335274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e0154c20 msgr2=0x7f00e015d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:02.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00df7fe640 1 --2- 172.21.15.139:0/371335274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e0154c20 0x7f00e015d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:02.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00df7fe640 1 -- 172.21.15.139:0/371335274 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e00b6300 con 0x7f00e00ccc90 2024-09-16T00:09:02.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00df7fe640 1 --2- 172.21.15.139:0/371335274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00e00ccc90 0x7f00e015cff0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f00d400e900 tx=0x7f00d400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:02.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00dcff9640 1 -- 172.21.15.139:0/371335274 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4017070 con 0x7f00e00ccc90 2024-09-16T00:09:02.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00dcff9640 1 -- 172.21.15.139:0/371335274 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d400c9a0 con 0x7f00e00ccc90 2024-09-16T00:09:02.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00dcff9640 1 -- 172.21.15.139:0/371335274 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d400cd50 con 0x7f00e00ccc90 2024-09-16T00:09:02.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00e645e640 1 -- 172.21.15.139:0/371335274 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e015cc00 con 0x7f00e00ccc90 2024-09-16T00:09:02.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.084+0000 7f00e645e640 1 -- 172.21.15.139:0/371335274 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e01f0f20 con 0x7f00e00ccc90 2024-09-16T00:09:02.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.085+0000 7f00e645e640 1 -- 172.21.15.139:0/371335274 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e01f1450 con 0x7f00e00ccc90 2024-09-16T00:09:02.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.086+0000 7f00dcff9640 1 -- 172.21.15.139:0/371335274 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f00d401a070 con 0x7f00e00ccc90 2024-09-16T00:09:02.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.086+0000 7f00dcff9640 1 --2- 172.21.15.139:0/371335274 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f00cc06c2e0 0x7f00cc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:02.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.086+0000 7f00dcff9640 1 -- 172.21.15.139:0/371335274 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5886+0+0 (secure 0 0 0) 0x7f00d408b6f0 con 0x7f00e00ccc90 2024-09-16T00:09:02.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.086+0000 7f00dffff640 1 --2- 172.21.15.139:0/371335274 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f00cc06c2e0 0x7f00cc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:02.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.087+0000 7f00dffff640 1 --2- 172.21.15.139:0/371335274 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f00cc06c2e0 0x7f00cc06e7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f00d0007fd0 tx=0x7f00d0007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:02.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.091+0000 7f00dcff9640 1 -- 172.21.15.139:0/371335274 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f00d40597f0 con 0x7f00e00ccc90 2024-09-16T00:09:02.152 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:09:02.207 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:02.207+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:02.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.287+0000 7f92a6c53640 1 Processor -- start 2024-09-16T00:09:02.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.287+0000 7f92a6c53640 1 -- start start 2024-09-16T00:09:02.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.289+0000 7f92a6c53640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f92a006d0c0 0x7f92a00ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:02.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.289+0000 7f92a6c53640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f92a006e940 0x7f92a00ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:02.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.289+0000 7f92a49ef640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f92a0154ae0 0x7f92a015a710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:45618/0 (socket says 172.21.15.139:45618) 2024-09-16T00:09:02.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.289+0000 7f92a51f0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f92a006e940 0x7f92a00ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:02.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.289+0000 7f92a51f0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f92a006e940 0x7f92a00ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42002/0 (socket says 172.21.15.139:42002) 2024-09-16T00:09:02.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.289+0000 7f92a49ef640 1 -- 172.21.15.139:0/3012741315 learned_addr learned my addr 172.21.15.139:0/3012741315 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:02.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.289+0000 7f92a6c53640 1 -- 172.21.15.139:0/3012741315 --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92a00c8d90 con 0x7f92a0154ae0 2024-09-16T00:09:02.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.289+0000 7f92a6c53640 1 -- 172.21.15.139:0/3012741315 --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f92a006cc30 con 0x7f92a006e940 2024-09-16T00:09:02.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.289+0000 7f92a6c53640 1 -- 172.21.15.139:0/3012741315 --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f92a00701b0 con 0x7f92a006d0c0 2024-09-16T00:09:02.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.289+0000 7f929ffff640 1 --2- 172.21.15.139:0/3012741315 >> v2:172.21.15.164:3300/0 conn(0x7f92a006d0c0 0x7f92a00ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:02.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.289+0000 7f929f7fe640 1 -- 172.21.15.139:0/3012741315 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3329594862 0 0) 0x7f92a00c8d90 con 0x7f92a0154ae0 2024-09-16T00:09:02.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.289+0000 7f929f7fe640 1 -- 172.21.15.139:0/3012741315 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9290003520 con 0x7f92a0154ae0 2024-09-16T00:09:02.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.289+0000 7f929ffff640 1 -- 172.21.15.139:0/3012741315 >> v2:172.21.15.139:3300/0 conn(0x7f92a006e940 msgr2=0x7f92a00ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:02.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.289+0000 7f929ffff640 1 --2- 172.21.15.139:0/3012741315 >> v2:172.21.15.139:3300/0 conn(0x7f92a006e940 0x7f92a00ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:02.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.289+0000 7f929ffff640 1 -- 172.21.15.139:0/3012741315 >> v1:172.21.15.164:6789/0 conn(0x7f92a0154ae0 legacy=0x7f92a015a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:02.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.289+0000 7f929ffff640 1 -- 172.21.15.139:0/3012741315 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a0070400 con 0x7f92a006d0c0 2024-09-16T00:09:02.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.290+0000 7f929ffff640 1 --2- 172.21.15.139:0/3012741315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92a006d0c0 0x7f92a00ce140 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f92a00cbcd0 tx=0x7f928c02a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ff7e5d3b8445e23 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:02.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.290+0000 7f929f7fe640 1 -- 172.21.15.139:0/3012741315 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c037070 con 0x7f92a006d0c0 2024-09-16T00:09:02.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.290+0000 7f929f7fe640 1 -- 172.21.15.139:0/3012741315 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928c032440 con 0x7f92a006d0c0 2024-09-16T00:09:02.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.290+0000 7f929f7fe640 1 -- 172.21.15.139:0/3012741315 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c0327f0 con 0x7f92a006d0c0 2024-09-16T00:09:02.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.291+0000 7f92a6c53640 1 -- 172.21.15.139:0/3012741315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92a006d0c0 msgr2=0x7f92a00ce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:02.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.291+0000 7f92a6c53640 1 --2- 172.21.15.139:0/3012741315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92a006d0c0 0x7f92a00ce140 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f92a00cbcd0 tx=0x7f928c02a1e0 comp rx=0 tx=0).stop 2024-09-16T00:09:02.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.291+0000 7f92a6c53640 1 -- 172.21.15.139:0/3012741315 shutdown_connections 2024-09-16T00:09:02.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.291+0000 7f92a6c53640 1 --2- 172.21.15.139:0/3012741315 >> v2:172.21.15.139:3300/0 conn(0x7f92a006e940 0x7f92a00ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:02.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.291+0000 7f92a6c53640 1 --2- 172.21.15.139:0/3012741315 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92a006d0c0 0x7f92a00ce140 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:02.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.291+0000 7f92a6c53640 1 -- 172.21.15.139:0/3012741315 >> 172.21.15.139:0/3012741315 conn(0x7f92a006c150 msgr2=0x7f92a0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:02.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.291+0000 7f92a6c53640 1 -- 172.21.15.139:0/3012741315 shutdown_connections 2024-09-16T00:09:02.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.291+0000 7f92a6c53640 1 -- 172.21.15.139:0/3012741315 wait complete. 2024-09-16T00:09:02.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.293+0000 7f92a6c53640 1 Processor -- start 2024-09-16T00:09:02.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.293+0000 7f92a6c53640 1 -- start start 2024-09-16T00:09:02.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.294+0000 7f92a6c53640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92a006d0c0 0x7f92a00c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:02.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.294+0000 7f92a6c53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a006e940 0x7f92a0148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:02.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.294+0000 7f92a6c53640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92a0154ae0 0x7f92a0148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:02.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.294+0000 7f92a6c53640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f92a0154ec0 con 0x7f92a006e940 2024-09-16T00:09:02.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.294+0000 7f929ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a006e940 0x7f92a0148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:02.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.294+0000 7f929ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a006e940 0x7f92a0148730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42012/0 (socket says 172.21.15.139:42012) 2024-09-16T00:09:02.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.294+0000 7f92a6c53640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f92a0070770 con 0x7f92a0154ae0 2024-09-16T00:09:02.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.294+0000 7f92a6c53640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f92a00c9ce0 con 0x7f92a006d0c0 2024-09-16T00:09:02.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.294+0000 7f929ffff640 1 -- 172.21.15.139:0/3697163344 learned_addr learned my addr 172.21.15.139:0/3697163344 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:02.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.294+0000 7f92a49ef640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92a006d0c0 0x7f92a00c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:02.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.294+0000 7f92a51f0640 1 --2- 172.21.15.139:0/3697163344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92a0154ae0 0x7f92a0148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:02.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.295+0000 7f929ffff640 1 -- 172.21.15.139:0/3697163344 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92a006d0c0 msgr2=0x7f92a00c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:02.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.295+0000 7f929ffff640 1 --2- 172.21.15.139:0/3697163344 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92a006d0c0 0x7f92a00c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:02.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.295+0000 7f929ffff640 1 -- 172.21.15.139:0/3697163344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92a0154ae0 msgr2=0x7f92a0148c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:02.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.295+0000 7f929ffff640 1 --2- 172.21.15.139:0/3697163344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92a0154ae0 0x7f92a0148c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:02.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.295+0000 7f929ffff640 1 -- 172.21.15.139:0/3697163344 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a01f1380 con 0x7f92a006e940 2024-09-16T00:09:02.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.295+0000 7f92a49ef640 1 --2- 172.21.15.139:0/3697163344 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92a006d0c0 0x7f92a00c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:09:02.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.295+0000 7f929ffff640 1 --2- 172.21.15.139:0/3697163344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a006e940 0x7f92a0148730 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f928c000c00 tx=0x7f928c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:02.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.295+0000 7f929d7fa640 1 -- 172.21.15.139:0/3697163344 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c037040 con 0x7f92a006e940 2024-09-16T00:09:02.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.295+0000 7f92a6c53640 1 -- 172.21.15.139:0/3697163344 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a01e6690 con 0x7f92a006e940 2024-09-16T00:09:02.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.295+0000 7f929d7fa640 1 -- 172.21.15.139:0/3697163344 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928c042070 con 0x7f92a006e940 2024-09-16T00:09:02.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.295+0000 7f92a6c53640 1 -- 172.21.15.139:0/3697163344 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a014d680 con 0x7f92a006e940 2024-09-16T00:09:02.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.295+0000 7f929d7fa640 1 -- 172.21.15.139:0/3697163344 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c047660 con 0x7f92a006e940 2024-09-16T00:09:02.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.296+0000 7f92a6c53640 1 -- 172.21.15.139:0/3697163344 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a0067b80 con 0x7f92a006e940 2024-09-16T00:09:02.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.297+0000 7f929d7fa640 1 -- 172.21.15.139:0/3697163344 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f928c047800 con 0x7f92a006e940 2024-09-16T00:09:02.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.297+0000 7f929d7fa640 1 --2- 172.21.15.139:0/3697163344 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f928406c2b0 0x7f928406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:02.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.297+0000 7f929d7fa640 1 -- 172.21.15.139:0/3697163344 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5886+0+0 (secure 0 0 0) 0x7f928c0abc30 con 0x7f92a006e940 2024-09-16T00:09:02.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.297+0000 7f92a49ef640 1 --2- 172.21.15.139:0/3697163344 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f928406c2b0 0x7f928406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:02.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.298+0000 7f92a49ef640 1 --2- 172.21.15.139:0/3697163344 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f928406c2b0 0x7f928406e770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f92940011e0 tx=0x7f9294007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:02.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.303+0000 7f929d7fa640 1 -- 172.21.15.139:0/3697163344 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f92a0067b80 con 0x7f92a006e940 2024-09-16T00:09:02.308 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:02.307+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:02.309 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:02.309 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:02.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.309 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:1941 3.25 3:a4711b34:::10000000aa1.00000000:head [write 0~2491 in=2491b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T00:09:02.309 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:02.308750+0000", 2024-09-16T00:09:02.309 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.00012968800000000001, 2024-09-16T00:09:02.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000152258, 2024-09-16T00:09:02.309 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.309 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.310 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:09:02.310 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:02.310 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:02.310 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:02.310 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 1941 2024-09-16T00:09:02.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.310 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.308750+0000", 2024-09-16T00:09:02.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.308750+0000", 2024-09-16T00:09:02.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.308760+0000", 2024-09-16T00:09:02.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0358e-05 2024-09-16T00:09:02.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.308767+0000", 2024-09-16T00:09:02.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7569999999999998e-06 2024-09-16T00:09:02.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.308781+0000", 2024-09-16T00:09:02.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4168999999999999e-05 2024-09-16T00:09:02.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.308789+0000", 2024-09-16T00:09:02.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9039999999999992e-06 2024-09-16T00:09:02.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.308847+0000", 2024-09-16T00:09:02.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8235000000000001e-05 2024-09-16T00:09:02.313 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.313 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.313 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.313 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.313 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:02.313 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 1 2024-09-16T00:09:02.313 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:02.319 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:09:02.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.336+0000 7f00e645e640 1 -- 172.21.15.139:0/371335274 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pg-upmap", "pgid": "3.2d", "id": [7, 5]} v 0) -- 0x7f00e015e690 con 0x7f00e00ccc90 2024-09-16T00:09:02.409 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:02.408+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:02.458 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:02.459 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:02.459 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:02.459 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:02.459 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:02.470 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:09:02.512 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:02.511+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:02.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:02.523+0000 7f92a6c53640 1 -- 172.21.15.139:0/3697163344 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f92a006d600 con 0x7f92a006e940 2024-09-16T00:09:02.611 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:02.610+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:02.633 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:02.634 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:02.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:02.634 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:02.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.634 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:02.634 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:09:02.634 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.016592556999996, 2024-09-16T00:09:02.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037514929999999998, 2024-09-16T00:09:02.634 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.634 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.634 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.635 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:09:02.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:09:02.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615364+0000", 2024-09-16T00:09:02.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8430000000000003e-06 2024-09-16T00:09:02.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615371+0000", 2024-09-16T00:09:02.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.148e-06 2024-09-16T00:09:02.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615380+0000", 2024-09-16T00:09:02.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5889999999999992e-06 2024-09-16T00:09:02.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615389+0000", 2024-09-16T00:09:02.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1620000000000008e-06 2024-09-16T00:09:02.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615525+0000", 2024-09-16T00:09:02.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000136083 2024-09-16T00:09:02.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615662+0000", 2024-09-16T00:09:02.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013696100000000001 2024-09-16T00:09:02.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:02.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615799+0000", 2024-09-16T00:09:02.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013781399999999999 2024-09-16T00:09:02.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:09:02.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002363502 2024-09-16T00:09:02.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619018+0000", 2024-09-16T00:09:02.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085522199999999999 2024-09-16T00:09:02.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619068+0000", 2024-09-16T00:09:02.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9545000000000001e-05 2024-09-16T00:09:02.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619111+0000", 2024-09-16T00:09:02.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3624000000000001e-05 2024-09-16T00:09:02.639 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.639 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.639 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.640 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:02.640 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:09:02.640 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.016540485999997, 2024-09-16T00:09:02.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043656110000000001, 2024-09-16T00:09:02.640 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.640 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.640 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.640 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:09:02.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:09:02.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:09:02.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2199999999999999e-07 2024-09-16T00:09:02.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615416+0000", 2024-09-16T00:09:02.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5590000000000001e-06 2024-09-16T00:09:02.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615423+0000", 2024-09-16T00:09:02.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6250000000000001e-06 2024-09-16T00:09:02.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615426+0000", 2024-09-16T00:09:02.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2540000000000001e-06 2024-09-16T00:09:02.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616075+0000", 2024-09-16T00:09:02.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064968800000000004 2024-09-16T00:09:02.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616142+0000", 2024-09-16T00:09:02.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6203000000000003e-05 2024-09-16T00:09:02.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:02.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616247+0000", 2024-09-16T00:09:02.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010572499999999999 2024-09-16T00:09:02.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618913+0000", 2024-09-16T00:09:02.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026654339999999999 2024-09-16T00:09:02.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619712+0000", 2024-09-16T00:09:02.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079885199999999996 2024-09-16T00:09:02.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619744+0000", 2024-09-16T00:09:02.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2790000000000003e-05 2024-09-16T00:09:02.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.645 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619778+0000", 2024-09-16T00:09:02.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3059000000000003e-05 2024-09-16T00:09:02.645 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.645 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.645 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.645 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:02.645 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:09:02.645 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.016504406999999, 2024-09-16T00:09:02.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087673219999999993, 2024-09-16T00:09:02.646 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.646 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.646 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.646 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.646 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.646 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:09:02.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.646 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.646 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:09:02.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:09:02.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:09:02.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:09:02.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5392000000000001e-05 2024-09-16T00:09:02.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615470+0000", 2024-09-16T00:09:02.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1999999999999999e-06 2024-09-16T00:09:02.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615474+0000", 2024-09-16T00:09:02.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7359999999999999e-06 2024-09-16T00:09:02.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617614+0000", 2024-09-16T00:09:02.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021398189999999998 2024-09-16T00:09:02.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617724+0000", 2024-09-16T00:09:02.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110808 2024-09-16T00:09:02.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:02.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617856+0000", 2024-09-16T00:09:02.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131676 2024-09-16T00:09:02.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:09:02.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029629280000000001 2024-09-16T00:09:02.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.650 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624158+0000", 2024-09-16T00:09:02.650 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033386549999999998 2024-09-16T00:09:02.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.650 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.650 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624189+0000", 2024-09-16T00:09:02.650 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1508999999999999e-05 2024-09-16T00:09:02.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.650 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.650 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624215+0000", 2024-09-16T00:09:02.650 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6228999999999999e-05 2024-09-16T00:09:02.650 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.650 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.650 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.651 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.651 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.651 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:02.651 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:09:02.651 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.016470392000002, 2024-09-16T00:09:02.651 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053285279999999999, 2024-09-16T00:09:02.651 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.651 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.651 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.651 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.651 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.651 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.651 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:09:02.651 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.652 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.652 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.652 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:09:02.652 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.652 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.652 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.652 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:09:02.652 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5600000000000002e-07 2024-09-16T00:09:02.652 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.652 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.652 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615484+0000", 2024-09-16T00:09:02.652 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.688e-06 2024-09-16T00:09:02.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.653 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.653 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615488+0000", 2024-09-16T00:09:02.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.658e-06 2024-09-16T00:09:02.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.653 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.653 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615490+0000", 2024-09-16T00:09:02.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.728e-06 2024-09-16T00:09:02.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.653 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.653 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616441+0000", 2024-09-16T00:09:02.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00095065500000000001 2024-09-16T00:09:02.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.654 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616539+0000", 2024-09-16T00:09:02.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8335e-05 2024-09-16T00:09:02.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:02.654 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616631+0000", 2024-09-16T00:09:02.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1260000000000004e-05 2024-09-16T00:09:02.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619837+0000", 2024-09-16T00:09:02.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003206899 2024-09-16T00:09:02.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.655 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620725+0000", 2024-09-16T00:09:02.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088713599999999996 2024-09-16T00:09:02.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.655 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620772+0000", 2024-09-16T00:09:02.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7639999999999998e-05 2024-09-16T00:09:02.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.655 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.656 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620811+0000", 2024-09-16T00:09:02.656 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8272999999999998e-05 2024-09-16T00:09:02.656 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.656 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.656 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.656 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.656 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.656 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:02.656 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:09:02.656 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.016055304000005, 2024-09-16T00:09:02.656 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029886169999999998, 2024-09-16T00:09:02.656 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.656 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.656 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.657 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.657 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.657 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.657 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:09:02.657 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.657 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.657 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.657 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.657 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:09:02.657 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.657 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.657 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.657 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.657 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615898+0000", 2024-09-16T00:09:02.658 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.237e-06 2024-09-16T00:09:02.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.658 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615904+0000", 2024-09-16T00:09:02.658 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:09:02.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.658 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615915+0000", 2024-09-16T00:09:02.658 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1003e-05 2024-09-16T00:09:02.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.658 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615922+0000", 2024-09-16T00:09:02.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9959999999999996e-06 2024-09-16T00:09:02.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.659 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615988+0000", 2024-09-16T00:09:02.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5806e-05 2024-09-16T00:09:02.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.659 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616009+0000", 2024-09-16T00:09:02.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0729000000000001e-05 2024-09-16T00:09:02.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.659 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.660 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:09:02.660 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028462130000000002 2024-09-16T00:09:02.660 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.660 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.660 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.660 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618886+0000", 2024-09-16T00:09:02.660 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.082e-05 2024-09-16T00:09:02.660 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.660 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.660 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.660 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.660 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.660 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:02.660 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:09:02.661 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.014443831999998, 2024-09-16T00:09:02.661 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032610299999999998, 2024-09-16T00:09:02.661 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.661 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.661 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.661 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.661 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.661 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.661 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:09:02.661 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.661 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.661 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.661 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.661 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:09:02.662 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.662 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.662 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.662 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617556+0000", 2024-09-16T00:09:02.662 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7431e-05 2024-09-16T00:09:02.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.662 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.662 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.662 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617847+0000", 2024-09-16T00:09:02.662 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029133400000000003 2024-09-16T00:09:02.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.662 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.662 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.663 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617858+0000", 2024-09-16T00:09:02.663 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0222e-05 2024-09-16T00:09:02.663 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.663 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.663 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.663 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617864+0000", 2024-09-16T00:09:02.663 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7619999999999998e-06 2024-09-16T00:09:02.663 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.663 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.663 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.663 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617909+0000", 2024-09-16T00:09:02.663 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4747999999999999e-05 2024-09-16T00:09:02.663 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.664 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.664 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.664 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617927+0000", 2024-09-16T00:09:02.664 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7785999999999999e-05 2024-09-16T00:09:02.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.664 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.664 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.664 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620734+0000", 2024-09-16T00:09:02.664 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002807305 2024-09-16T00:09:02.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.664 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.664 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.664 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620770+0000", 2024-09-16T00:09:02.664 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5441999999999997e-05 2024-09-16T00:09:02.665 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.665 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.665 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.665 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.665 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.665 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:09:02.665 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:09:02.665 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.014023148000007, 2024-09-16T00:09:02.665 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028641809999999999, 2024-09-16T00:09:02.665 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.665 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.665 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.665 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:09:02.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:09:02.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617930+0000", 2024-09-16T00:09:02.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0200000000000002e-07 2024-09-16T00:09:02.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617934+0000", 2024-09-16T00:09:02.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:09:02.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617943+0000", 2024-09-16T00:09:02.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5820000000000002e-06 2024-09-16T00:09:02.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617949+0000", 2024-09-16T00:09:02.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5069999999999997e-06 2024-09-16T00:09:02.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.668 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618094+0000", 2024-09-16T00:09:02.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014515599999999999 2024-09-16T00:09:02.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.668 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618111+0000", 2024-09-16T00:09:02.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6705999999999999e-05 2024-09-16T00:09:02.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.668 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:09:02.669 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026603070000000002 2024-09-16T00:09:02.669 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.669 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.669 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620793+0000", 2024-09-16T00:09:02.669 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2214000000000001e-05 2024-09-16T00:09:02.669 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.669 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.669 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.669 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.669 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-16T00:09:02.669 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:09:02.669 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.013992359, 2024-09-16T00:09:02.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028624549999999999, 2024-09-16T00:09:02.670 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.670 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.670 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.670 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:09:02.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:09:02.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:09:02.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:09:02.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:09:02.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0570000000000001e-06 2024-09-16T00:09:02.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617970+0000", 2024-09-16T00:09:02.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.669e-06 2024-09-16T00:09:02.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617976+0000", 2024-09-16T00:09:02.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1369999999999999e-06 2024-09-16T00:09:02.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618268+0000", 2024-09-16T00:09:02.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029164600000000001 2024-09-16T00:09:02.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618284+0000", 2024-09-16T00:09:02.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5824e-05 2024-09-16T00:09:02.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:09:02.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025127890000000001 2024-09-16T00:09:02.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620822+0000", 2024-09-16T00:09:02.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6009000000000001e-05 2024-09-16T00:09:02.673 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.673 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.673 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.674 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-16T00:09:02.674 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:09:02.674 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.013964778000002, 2024-09-16T00:09:02.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028650350000000002, 2024-09-16T00:09:02.674 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.674 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.674 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.674 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.674 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.674 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:09:02.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.675 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.675 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.675 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.675 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:09:02.675 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.675 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.675 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.675 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.675 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:09:02.675 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7599999999999998e-07 2024-09-16T00:09:02.675 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.675 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.675 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617991+0000", 2024-09-16T00:09:02.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1860000000000001e-06 2024-09-16T00:09:02.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.676 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.676 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617999+0000", 2024-09-16T00:09:02.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3250000000000002e-06 2024-09-16T00:09:02.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.676 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.676 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618004+0000", 2024-09-16T00:09:02.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7300000000000002e-06 2024-09-16T00:09:02.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618422+0000", 2024-09-16T00:09:02.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00041763199999999998 2024-09-16T00:09:02.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:09:02.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6112000000000001e-05 2024-09-16T00:09:02.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620824+0000", 2024-09-16T00:09:02.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002385954 2024-09-16T00:09:02.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620853+0000", 2024-09-16T00:09:02.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8719999999999999e-05 2024-09-16T00:09:02.678 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.678 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.678 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.678 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-16T00:09:02.678 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:09:02.678 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.013938956000004, 2024-09-16T00:09:02.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032703020000000001, 2024-09-16T00:09:02.678 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.679 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.679 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.679 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:09:02.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:09:02.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618014+0000", 2024-09-16T00:09:02.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7799999999999997e-07 2024-09-16T00:09:02.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618017+0000", 2024-09-16T00:09:02.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4309999999999998e-06 2024-09-16T00:09:02.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618023+0000", 2024-09-16T00:09:02.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8490000000000002e-06 2024-09-16T00:09:02.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.681 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.681 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618028+0000", 2024-09-16T00:09:02.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3489999999999999e-06 2024-09-16T00:09:02.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.681 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.681 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:09:02.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055724099999999999 2024-09-16T00:09:02.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.681 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.681 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618603+0000", 2024-09-16T00:09:02.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7486000000000002e-05 2024-09-16T00:09:02.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621235+0000", 2024-09-16T00:09:02.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002632087 2024-09-16T00:09:02.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621284+0000", 2024-09-16T00:09:02.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8581000000000001e-05 2024-09-16T00:09:02.682 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.682 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.682 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.683 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-16T00:09:02.683 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:09:02.683 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.012963869000004, 2024-09-16T00:09:02.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002325307, 2024-09-16T00:09:02.683 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.683 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.683 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.683 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:09:02.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:09:02.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:09:02.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:09:02.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618993+0000", 2024-09-16T00:09:02.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.687e-06 2024-09-16T00:09:02.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619000+0000", 2024-09-16T00:09:02.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3590000000000002e-06 2024-09-16T00:09:02.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619005+0000", 2024-09-16T00:09:02.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.95e-06 2024-09-16T00:09:02.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619039+0000", 2024-09-16T00:09:02.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3942999999999997e-05 2024-09-16T00:09:02.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619055+0000", 2024-09-16T00:09:02.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6107e-05 2024-09-16T00:09:02.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621287+0000", 2024-09-16T00:09:02.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022318070000000001 2024-09-16T00:09:02.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:09:02.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7129999999999999e-05 2024-09-16T00:09:02.687 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.687 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.687 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.687 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-16T00:09:02.687 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:09:02.687 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.012925873, 2024-09-16T00:09:02.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002314328, 2024-09-16T00:09:02.687 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.687 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.687 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.687 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:09:02.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:09:02.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:09:02.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6399999999999998e-07 2024-09-16T00:09:02.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619030+0000", 2024-09-16T00:09:02.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.219e-06 2024-09-16T00:09:02.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619038+0000", 2024-09-16T00:09:02.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9899999999999997e-06 2024-09-16T00:09:02.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619043+0000", 2024-09-16T00:09:02.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5580000000000002e-06 2024-09-16T00:09:02.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619220+0000", 2024-09-16T00:09:02.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000176937 2024-09-16T00:09:02.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619237+0000", 2024-09-16T00:09:02.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7691999999999999e-05 2024-09-16T00:09:02.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621315+0000", 2024-09-16T00:09:02.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020778900000000002 2024-09-16T00:09:02.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621341+0000", 2024-09-16T00:09:02.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5678e-05 2024-09-16T00:09:02.691 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.691 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.691 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.691 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:39 2.6 2:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:09:02.691 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:09:02.691 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 78.991674549999999, 2024-09-16T00:09:02.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050072119999999996, 2024-09-16T00:09:02.692 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.692 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.692 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.692 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:09:02.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:09:02.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640280+0000", 2024-09-16T00:09:02.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5380000000000001e-06 2024-09-16T00:09:02.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640296+0000", 2024-09-16T00:09:02.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5950000000000001e-05 2024-09-16T00:09:02.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640327+0000", 2024-09-16T00:09:02.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0309999999999999e-05 2024-09-16T00:09:02.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640340+0000", 2024-09-16T00:09:02.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3773e-05 2024-09-16T00:09:02.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640422+0000", 2024-09-16T00:09:02.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1122e-05 2024-09-16T00:09:02.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640572+0000", 2024-09-16T00:09:02.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015044099999999999 2024-09-16T00:09:02.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:02.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640691+0000", 2024-09-16T00:09:02.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000119393 2024-09-16T00:09:02.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645006+0000", 2024-09-16T00:09:02.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043143779999999998 2024-09-16T00:09:02.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645169+0000", 2024-09-16T00:09:02.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016323299999999999 2024-09-16T00:09:02.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645227+0000", 2024-09-16T00:09:02.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8e-05 2024-09-16T00:09:02.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645285+0000", 2024-09-16T00:09:02.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8074e-05 2024-09-16T00:09:02.696 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.697 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.697 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.697 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:52 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e57)", 2024-09-16T00:09:02.697 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.651358+0000", 2024-09-16T00:09:02.697 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.9805939080000003, 2024-09-16T00:09:02.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002097018, 2024-09-16T00:09:02.697 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.697 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.697 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.697 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.651358+0000", 2024-09-16T00:09:02.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.651358+0000", 2024-09-16T00:09:02.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.651360+0000", 2024-09-16T00:09:02.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0979999999999999e-06 2024-09-16T00:09:02.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.651375+0000", 2024-09-16T00:09:02.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.579e-05 2024-09-16T00:09:02.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.651390+0000", 2024-09-16T00:09:02.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4477e-05 2024-09-16T00:09:02.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.651398+0000", 2024-09-16T00:09:02.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2239999999999995e-06 2024-09-16T00:09:02.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.651454+0000", 2024-09-16T00:09:02.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6076999999999999e-05 2024-09-16T00:09:02.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.651533+0000", 2024-09-16T00:09:02.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8998000000000005e-05 2024-09-16T00:09:02.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:02.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.652316+0000", 2024-09-16T00:09:02.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000783306 2024-09-16T00:09:02.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.653144+0000", 2024-09-16T00:09:02.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00082749400000000002 2024-09-16T00:09:02.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.653364+0000", 2024-09-16T00:09:02.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022024699999999999 2024-09-16T00:09:02.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.653418+0000", 2024-09-16T00:09:02.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3554000000000003e-05 2024-09-16T00:09:02.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.653456+0000", 2024-09-16T00:09:02.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7753000000000003e-05 2024-09-16T00:09:02.702 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.702 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.702 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.702 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:320 3.31 3:8eea9cd9:::10000000351.00000000:head [write 0~111] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:02.702 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.025490+0000", 2024-09-16T00:09:02.702 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.6064619000000002, 2024-09-16T00:09:02.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019909250000000002, 2024-09-16T00:09:02.703 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.703 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.703 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.703 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:02.703 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:02.703 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:02.703 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 320 2024-09-16T00:09:02.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.703 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025490+0000", 2024-09-16T00:09:02.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025490+0000", 2024-09-16T00:09:02.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025492+0000", 2024-09-16T00:09:02.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0729999999999999e-06 2024-09-16T00:09:02.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025496+0000", 2024-09-16T00:09:02.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7570000000000004e-06 2024-09-16T00:09:02.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025506+0000", 2024-09-16T00:09:02.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0023e-05 2024-09-16T00:09:02.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025514+0000", 2024-09-16T00:09:02.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3429999999999998e-06 2024-09-16T00:09:02.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025565+0000", 2024-09-16T00:09:02.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0905999999999997e-05 2024-09-16T00:09:02.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025646+0000", 2024-09-16T00:09:02.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1705000000000005e-05 2024-09-16T00:09:02.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:02.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.025723+0000", 2024-09-16T00:09:02.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6619999999999995e-05 2024-09-16T00:09:02.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.026282+0000", 2024-09-16T00:09:02.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055940999999999999 2024-09-16T00:09:02.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.027384+0000", 2024-09-16T00:09:02.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00110177 2024-09-16T00:09:02.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.027439+0000", 2024-09-16T00:09:02.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5139999999999997e-05 2024-09-16T00:09:02.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.027481+0000", 2024-09-16T00:09:02.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2178000000000002e-05 2024-09-16T00:09:02.708 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.708 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.708 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.708 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:54 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e58)", 2024-09-16T00:09:02.708 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.652050+0000", 2024-09-16T00:09:02.708 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.9799027069999999, 2024-09-16T00:09:02.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039842300000000001, 2024-09-16T00:09:02.709 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.709 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.709 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.709 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652050+0000", 2024-09-16T00:09:02.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652050+0000", 2024-09-16T00:09:02.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652052+0000", 2024-09-16T00:09:02.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.857e-06 2024-09-16T00:09:02.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652101+0000", 2024-09-16T00:09:02.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9812999999999999e-05 2024-09-16T00:09:02.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652122+0000", 2024-09-16T00:09:02.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0659e-05 2024-09-16T00:09:02.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652133+0000", 2024-09-16T00:09:02.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0977e-05 2024-09-16T00:09:02.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652198+0000", 2024-09-16T00:09:02.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5229999999999997e-05 2024-09-16T00:09:02.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652293+0000", 2024-09-16T00:09:02.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4596e-05 2024-09-16T00:09:02.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:02.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.654316+0000", 2024-09-16T00:09:02.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020229699999999998 2024-09-16T00:09:02.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.655783+0000", 2024-09-16T00:09:02.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014669240000000001 2024-09-16T00:09:02.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.655974+0000", 2024-09-16T00:09:02.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000191437 2024-09-16T00:09:02.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.656008+0000", 2024-09-16T00:09:02.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3603000000000003e-05 2024-09-16T00:09:02.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.656034+0000", 2024-09-16T00:09:02.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6163999999999999e-05 2024-09-16T00:09:02.714 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.714 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.714 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.714 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:650 3.3e 3:7f23b022:::100000004e2.00000000:head [write 0~9466] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:02.714 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.174315+0000", 2024-09-16T00:09:02.714 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.4576373140000003, 2024-09-16T00:09:02.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001879726, 2024-09-16T00:09:02.714 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.714 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.714 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.714 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:02.714 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:02.714 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:02.715 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 650 2024-09-16T00:09:02.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.715 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174315+0000", 2024-09-16T00:09:02.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174315+0000", 2024-09-16T00:09:02.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174318+0000", 2024-09-16T00:09:02.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8430000000000001e-06 2024-09-16T00:09:02.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174343+0000", 2024-09-16T00:09:02.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5551e-05 2024-09-16T00:09:02.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174368+0000", 2024-09-16T00:09:02.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4142e-05 2024-09-16T00:09:02.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174380+0000", 2024-09-16T00:09:02.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2269000000000001e-05 2024-09-16T00:09:02.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174451+0000", 2024-09-16T00:09:02.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0669999999999999e-05 2024-09-16T00:09:02.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174571+0000", 2024-09-16T00:09:02.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000120919 2024-09-16T00:09:02.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:02.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.174672+0000", 2024-09-16T00:09:02.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010018699999999999 2024-09-16T00:09:02.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.175572+0000", 2024-09-16T00:09:02.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00090064699999999997 2024-09-16T00:09:02.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.176083+0000", 2024-09-16T00:09:02.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00051112200000000003 2024-09-16T00:09:02.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.176144+0000", 2024-09-16T00:09:02.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0297000000000002e-05 2024-09-16T00:09:02.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.176195+0000", 2024-09-16T00:09:02.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1078999999999997e-05 2024-09-16T00:09:02.720 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.720 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.720 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.720 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:56 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e59)", 2024-09-16T00:09:02.720 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.652483+0000", 2024-09-16T00:09:02.720 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.9794695120000001, 2024-09-16T00:09:02.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003414191, 2024-09-16T00:09:02.721 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.721 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.721 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.721 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652483+0000", 2024-09-16T00:09:02.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652483+0000", 2024-09-16T00:09:02.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652485+0000", 2024-09-16T00:09:02.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4059999999999999e-06 2024-09-16T00:09:02.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652527+0000", 2024-09-16T00:09:02.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1319000000000002e-05 2024-09-16T00:09:02.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652548+0000", 2024-09-16T00:09:02.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1149999999999999e-05 2024-09-16T00:09:02.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652562+0000", 2024-09-16T00:09:02.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3971000000000001e-05 2024-09-16T00:09:02.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652631+0000", 2024-09-16T00:09:02.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9411000000000003e-05 2024-09-16T00:09:02.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652717+0000", 2024-09-16T00:09:02.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5556999999999994e-05 2024-09-16T00:09:02.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:02.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.654006+0000", 2024-09-16T00:09:02.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012897200000000001 2024-09-16T00:09:02.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.655280+0000", 2024-09-16T00:09:02.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012732609999999999 2024-09-16T00:09:02.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.655811+0000", 2024-09-16T00:09:02.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00053082400000000005 2024-09-16T00:09:02.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.655862+0000", 2024-09-16T00:09:02.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1672000000000003e-05 2024-09-16T00:09:02.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.655897+0000", 2024-09-16T00:09:02.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4900000000000001e-05 2024-09-16T00:09:02.725 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.725 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.725 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.726 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:71 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:02.726 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:01.656409+0000", 2024-09-16T00:09:02.726 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.97554316299999999, 2024-09-16T00:09:02.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023563120000000002, 2024-09-16T00:09:02.726 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.726 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.726 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.726 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.656409+0000", 2024-09-16T00:09:02.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.656409+0000", 2024-09-16T00:09:02.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.656411+0000", 2024-09-16T00:09:02.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.812e-06 2024-09-16T00:09:02.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.656457+0000", 2024-09-16T00:09:02.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5921999999999998e-05 2024-09-16T00:09:02.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.656470+0000", 2024-09-16T00:09:02.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2559e-05 2024-09-16T00:09:02.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.656476+0000", 2024-09-16T00:09:02.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1190000000000002e-06 2024-09-16T00:09:02.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.656518+0000", 2024-09-16T00:09:02.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2228000000000003e-05 2024-09-16T00:09:02.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.656577+0000", 2024-09-16T00:09:02.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9036999999999997e-05 2024-09-16T00:09:02.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:02.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.657373+0000", 2024-09-16T00:09:02.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079629799999999997 2024-09-16T00:09:02.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.658546+0000", 2024-09-16T00:09:02.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001172962 2024-09-16T00:09:02.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.658655+0000", 2024-09-16T00:09:02.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108863 2024-09-16T00:09:02.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.658713+0000", 2024-09-16T00:09:02.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8242999999999998e-05 2024-09-16T00:09:02.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.658766+0000", 2024-09-16T00:09:02.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2268999999999997e-05 2024-09-16T00:09:02.731 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.731 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.731 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.731 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:82 2.15 2:aaefc04a:::10000000261.03a00000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:02.731 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:02.076823+0000", 2024-09-16T00:09:02.731 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.55512963199999998, 2024-09-16T00:09:02.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001915513, 2024-09-16T00:09:02.731 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:02.731 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:02.732 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:02.732 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:02.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:02.732 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.076823+0000", 2024-09-16T00:09:02.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:02.732 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.076823+0000", 2024-09-16T00:09:02.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:02.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:02.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.076823+0000", 2024-09-16T00:09:02.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2200000000000003e-07 2024-09-16T00:09:02.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:02.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.076849+0000", 2024-09-16T00:09:02.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5354000000000002e-05 2024-09-16T00:09:02.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:02.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.076858+0000", 2024-09-16T00:09:02.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7390000000000005e-06 2024-09-16T00:09:02.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:02.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.076866+0000", 2024-09-16T00:09:02.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1659999999999994e-06 2024-09-16T00:09:02.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:02.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.076918+0000", 2024-09-16T00:09:02.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1898999999999999e-05 2024-09-16T00:09:02.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:02.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.077026+0000", 2024-09-16T00:09:02.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108151 2024-09-16T00:09:02.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:02.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.077175+0000", 2024-09-16T00:09:02.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014873100000000001 2024-09-16T00:09:02.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:02.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.078217+0000", 2024-09-16T00:09:02.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001042149 2024-09-16T00:09:02.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:02.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.078648+0000", 2024-09-16T00:09:02.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00043103500000000002 2024-09-16T00:09:02.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:02.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.078702+0000", 2024-09-16T00:09:02.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4092999999999999e-05 2024-09-16T00:09:02.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:02.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:02.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:02.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.078738+0000", 2024-09-16T00:09:02.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6473999999999997e-05 2024-09-16T00:09:02.736 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.736 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.736 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.736 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:02.736 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:02.737 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:02.738 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:09:02.738 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:02.737+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:02.839 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:02.838+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:02.902 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:02.902 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:02.902 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:02.902 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:02.911 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:09:02.953 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:02.952+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:03.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.016+0000 7f00dcff9640 1 -- 172.21.15.139:0/371335274 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap", "pgid": "3.2d", "id": [7, 5]}]=0 set 3.2d pg_upmap mapping to [7,5] v62) ==== 124+0+0 (secure 0 0 0) 0x7f00d405daf0 con 0x7f00e00ccc90 2024-09-16T00:09:03.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.016+0000 7f929d7fa640 1 -- 172.21.15.139:0/3697163344 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v62) ==== 101+0+0 (secure 0 0 0) 0x7f928c0b2080 con 0x7f92a006e940 2024-09-16T00:09:03.018 INFO:teuthology.orchestra.run.smithi139.stderr:set 3.2d pg_upmap mapping to [7,5] 2024-09-16T00:09:03.018 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is unset 2024-09-16T00:09:03.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.018+0000 7f92a6c53640 1 -- 172.21.15.139:0/3697163344 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f928406c2b0 msgr2=0x7f928406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.018+0000 7f92a6c53640 1 --2- 172.21.15.139:0/3697163344 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f928406c2b0 0x7f928406e770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f92940011e0 tx=0x7f9294007c90 comp rx=0 tx=0).stop 2024-09-16T00:09:03.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.018+0000 7f92a6c53640 1 -- 172.21.15.139:0/3697163344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a006e940 msgr2=0x7f92a0148730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.018+0000 7f92a6c53640 1 --2- 172.21.15.139:0/3697163344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a006e940 0x7f92a0148730 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f928c000c00 tx=0x7f928c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f92a6c53640 1 -- 172.21.15.139:0/3697163344 shutdown_connections 2024-09-16T00:09:03.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f92a6c53640 1 --2- 172.21.15.139:0/3697163344 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f928406c2b0 0x7f928406e770 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f92a6c53640 1 --2- 172.21.15.139:0/3697163344 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f92a0154ae0 0x7f92a0148c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f92a6c53640 1 --2- 172.21.15.139:0/3697163344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a006e940 0x7f92a0148730 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f92a6c53640 1 --2- 172.21.15.139:0/3697163344 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f92a006d0c0 0x7f92a00c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f92a6c53640 1 -- 172.21.15.139:0/3697163344 >> 172.21.15.139:0/3697163344 conn(0x7f92a006c150 msgr2=0x7f92a00ca940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:03.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f00e645e640 1 -- 172.21.15.139:0/371335274 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f00cc06c2e0 msgr2=0x7f00cc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f00e645e640 1 --2- 172.21.15.139:0/371335274 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f00cc06c2e0 0x7f00cc06e7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f00d0007fd0 tx=0x7f00d0007de0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f00e645e640 1 -- 172.21.15.139:0/371335274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00e00ccc90 msgr2=0x7f00e015cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f00e645e640 1 --2- 172.21.15.139:0/371335274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00e00ccc90 0x7f00e015cff0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f00d400e900 tx=0x7f00d400edc0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f00e645e640 1 -- 172.21.15.139:0/371335274 shutdown_connections 2024-09-16T00:09:03.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f00e645e640 1 --2- 172.21.15.139:0/371335274 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f00cc06c2e0 0x7f00cc06e7a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f00e645e640 1 --2- 172.21.15.139:0/371335274 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00e0154c20 0x7f00e015d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f00e645e640 1 --2- 172.21.15.139:0/371335274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00e00ccc90 0x7f00e015cff0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f00e645e640 1 --2- 172.21.15.139:0/371335274 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00e0068060 0x7f00e0156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f00e645e640 1 -- 172.21.15.139:0/371335274 >> 172.21.15.139:0/371335274 conn(0x7f00e006ca40 msgr2=0x7f00e0149b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:03.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f92a6c53640 1 -- 172.21.15.139:0/3697163344 shutdown_connections 2024-09-16T00:09:03.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f92a6c53640 1 -- 172.21.15.139:0/3697163344 wait complete. 2024-09-16T00:09:03.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f00e645e640 1 -- 172.21.15.139:0/371335274 shutdown_connections 2024-09-16T00:09:03.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.019+0000 7f00e645e640 1 -- 172.21.15.139:0/371335274 wait complete. 2024-09-16T00:09:03.040 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:03.040+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:03.122 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:03.122 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:03.122 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:03.123 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:03.123 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:03.123 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:09:03.164 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:03.163+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:03.241 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:03.241+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:03.288 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:03.288 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:03.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:03.289 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:03.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.289 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:03.289 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:09:03.289 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.668822543000005, 2024-09-16T00:09:03.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038263220000000001, 2024-09-16T00:09:03.289 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.289 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.289 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.289 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:09:03.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:09:03.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615465+0000", 2024-09-16T00:09:03.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2769999999999999e-06 2024-09-16T00:09:03.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615471+0000", 2024-09-16T00:09:03.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5729999999999996e-06 2024-09-16T00:09:03.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615483+0000", 2024-09-16T00:09:03.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1838000000000001e-05 2024-09-16T00:09:03.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615492+0000", 2024-09-16T00:09:03.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2410000000000001e-06 2024-09-16T00:09:03.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615596+0000", 2024-09-16T00:09:03.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104097 2024-09-16T00:09:03.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615688+0000", 2024-09-16T00:09:03.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1225999999999997e-05 2024-09-16T00:09:03.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:03.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615804+0000", 2024-09-16T00:09:03.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000116601 2024-09-16T00:09:03.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618165+0000", 2024-09-16T00:09:03.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023609429999999999 2024-09-16T00:09:03.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:09:03.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010258509999999999 2024-09-16T00:09:03.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619243+0000", 2024-09-16T00:09:03.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2327999999999998e-05 2024-09-16T00:09:03.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619287+0000", 2024-09-16T00:09:03.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3346999999999997e-05 2024-09-16T00:09:03.294 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.294 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.294 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.294 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:03.294 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:03.295 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.668767466999995, 2024-09-16T00:09:03.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0054735510000000001, 2024-09-16T00:09:03.295 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.295 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.295 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.295 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:03.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:03.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615516+0000", 2024-09-16T00:09:03.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6899999999999998e-07 2024-09-16T00:09:03.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615519+0000", 2024-09-16T00:09:03.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1760000000000002e-06 2024-09-16T00:09:03.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615526+0000", 2024-09-16T00:09:03.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8399999999999997e-06 2024-09-16T00:09:03.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615531+0000", 2024-09-16T00:09:03.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4450000000000004e-06 2024-09-16T00:09:03.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:09:03.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054127099999999996 2024-09-16T00:09:03.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616140+0000", 2024-09-16T00:09:03.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7230999999999994e-05 2024-09-16T00:09:03.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:03.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616249+0000", 2024-09-16T00:09:03.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109122 2024-09-16T00:09:03.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618646+0000", 2024-09-16T00:09:03.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023972279999999999 2024-09-16T00:09:03.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:09:03.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022396220000000001 2024-09-16T00:09:03.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620939+0000", 2024-09-16T00:09:03.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2735999999999998e-05 2024-09-16T00:09:03.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620989+0000", 2024-09-16T00:09:03.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0411000000000002e-05 2024-09-16T00:09:03.299 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.300 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.300 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.300 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:03.300 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:03.300 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.668741987999994, 2024-09-16T00:09:03.300 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055872789999999997, 2024-09-16T00:09:03.300 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.300 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.300 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.300 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.300 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:03.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:03.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:03.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.01e-07 2024-09-16T00:09:03.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615547+0000", 2024-09-16T00:09:03.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7899999999999996e-06 2024-09-16T00:09:03.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:09:03.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1119999999999998e-06 2024-09-16T00:09:03.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615560+0000", 2024-09-16T00:09:03.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:09:03.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.303 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616424+0000", 2024-09-16T00:09:03.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086350800000000003 2024-09-16T00:09:03.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.303 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:09:03.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9675999999999999e-05 2024-09-16T00:09:03.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:03.303 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616621+0000", 2024-09-16T00:09:03.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7345999999999999e-05 2024-09-16T00:09:03.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.304 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.304 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618654+0000", 2024-09-16T00:09:03.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020335269999999998 2024-09-16T00:09:03.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.304 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.304 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621055+0000", 2024-09-16T00:09:03.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024009729999999998 2024-09-16T00:09:03.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.304 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.304 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621100+0000", 2024-09-16T00:09:03.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5402999999999999e-05 2024-09-16T00:09:03.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621128+0000", 2024-09-16T00:09:03.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7736e-05 2024-09-16T00:09:03.305 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.305 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.305 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.305 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:03.305 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:03.305 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.668716337999996, 2024-09-16T00:09:03.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057024509999999999, 2024-09-16T00:09:03.306 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.306 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.306 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.306 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.306 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.306 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:03.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.306 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.306 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:03.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:03.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6300000000000001e-07 2024-09-16T00:09:03.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615569+0000", 2024-09-16T00:09:03.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3709999999999998e-06 2024-09-16T00:09:03.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615580+0000", 2024-09-16T00:09:03.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0448000000000001e-05 2024-09-16T00:09:03.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615584+0000", 2024-09-16T00:09:03.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6020000000000002e-06 2024-09-16T00:09:03.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616850+0000", 2024-09-16T00:09:03.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012661110000000001 2024-09-16T00:09:03.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616915+0000", 2024-09-16T00:09:03.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4820000000000006e-05 2024-09-16T00:09:03.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:03.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617031+0000", 2024-09-16T00:09:03.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115258 2024-09-16T00:09:03.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618659+0000", 2024-09-16T00:09:03.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001627975 2024-09-16T00:09:03.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621179+0000", 2024-09-16T00:09:03.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025203410000000002 2024-09-16T00:09:03.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621238+0000", 2024-09-16T00:09:03.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9323000000000001e-05 2024-09-16T00:09:03.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621269+0000", 2024-09-16T00:09:03.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0839000000000001e-05 2024-09-16T00:09:03.310 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.310 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.311 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.311 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:03.311 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:09:03.311 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.668692589000003, 2024-09-16T00:09:03.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058012769999999996, 2024-09-16T00:09:03.311 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.311 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.311 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.311 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:09:03.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:09:03.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615591+0000", 2024-09-16T00:09:03.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6199999999999999e-07 2024-09-16T00:09:03.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615594+0000", 2024-09-16T00:09:03.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8739999999999999e-06 2024-09-16T00:09:03.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615599+0000", 2024-09-16T00:09:03.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:09:03.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615603+0000", 2024-09-16T00:09:03.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8489999999999999e-06 2024-09-16T00:09:03.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617235+0000", 2024-09-16T00:09:03.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001632037 2024-09-16T00:09:03.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617348+0000", 2024-09-16T00:09:03.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113387 2024-09-16T00:09:03.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:03.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617458+0000", 2024-09-16T00:09:03.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109765 2024-09-16T00:09:03.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618664+0000", 2024-09-16T00:09:03.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012059950000000001 2024-09-16T00:09:03.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621311+0000", 2024-09-16T00:09:03.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026468469999999999 2024-09-16T00:09:03.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621357+0000", 2024-09-16T00:09:03.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6360999999999999e-05 2024-09-16T00:09:03.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621392+0000", 2024-09-16T00:09:03.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4190999999999998e-05 2024-09-16T00:09:03.316 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.316 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.316 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.316 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:03.316 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:03.317 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.668673964999996, 2024-09-16T00:09:03.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058737279999999999, 2024-09-16T00:09:03.317 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.317 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.317 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.317 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:03.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:03.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:03.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3799999999999999e-07 2024-09-16T00:09:03.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615611+0000", 2024-09-16T00:09:03.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-06 2024-09-16T00:09:03.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615626+0000", 2024-09-16T00:09:03.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:09:03.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615634+0000", 2024-09-16T00:09:03.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-06 2024-09-16T00:09:03.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617692+0000", 2024-09-16T00:09:03.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002057718 2024-09-16T00:09:03.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617768+0000", 2024-09-16T00:09:03.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6878999999999994e-05 2024-09-16T00:09:03.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:03.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:09:03.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019501699999999999 2024-09-16T00:09:03.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:09:03.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034357200000000002 2024-09-16T00:09:03.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621429+0000", 2024-09-16T00:09:03.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0012999999999999e-05 2024-09-16T00:09:03.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621459+0000", 2024-09-16T00:09:03.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9498000000000001e-05 2024-09-16T00:09:03.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621483+0000", 2024-09-16T00:09:03.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3957000000000001e-05 2024-09-16T00:09:03.323 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.323 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.323 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.324 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:03.324 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:09:03.324 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.668642684999995, 2024-09-16T00:09:03.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059371789999999999, 2024-09-16T00:09:03.324 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.324 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.324 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.324 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:09:03.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:09:03.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:09:03.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4000000000000002e-07 2024-09-16T00:09:03.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615643+0000", 2024-09-16T00:09:03.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6860000000000002e-06 2024-09-16T00:09:03.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615649+0000", 2024-09-16T00:09:03.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7610000000000004e-06 2024-09-16T00:09:03.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615653+0000", 2024-09-16T00:09:03.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0380000000000001e-06 2024-09-16T00:09:03.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618197+0000", 2024-09-16T00:09:03.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025440079999999999 2024-09-16T00:09:03.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618312+0000", 2024-09-16T00:09:03.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115083 2024-09-16T00:09:03.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:03.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618412+0000", 2024-09-16T00:09:03.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010016799999999999 2024-09-16T00:09:03.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621518+0000", 2024-09-16T00:09:03.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031058679999999999 2024-09-16T00:09:03.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621526+0000", 2024-09-16T00:09:03.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2490000000000004e-06 2024-09-16T00:09:03.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621549+0000", 2024-09-16T00:09:03.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3686999999999999e-05 2024-09-16T00:09:03.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621577+0000", 2024-09-16T00:09:03.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8191000000000002e-05 2024-09-16T00:09:03.329 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.329 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.329 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.329 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:03.330 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:03.330 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.668625019000004, 2024-09-16T00:09:03.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005989622, 2024-09-16T00:09:03.330 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.330 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.330 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.330 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:03.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:03.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:03.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3799999999999998e-07 2024-09-16T00:09:03.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615660+0000", 2024-09-16T00:09:03.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9740000000000001e-06 2024-09-16T00:09:03.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615666+0000", 2024-09-16T00:09:03.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3499999999999996e-06 2024-09-16T00:09:03.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615671+0000", 2024-09-16T00:09:03.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.13e-06 2024-09-16T00:09:03.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618759+0000", 2024-09-16T00:09:03.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030880500000000002 2024-09-16T00:09:03.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.333 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618832+0000", 2024-09-16T00:09:03.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3725000000000003e-05 2024-09-16T00:09:03.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.333 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:03.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618958+0000", 2024-09-16T00:09:03.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012587899999999999 2024-09-16T00:09:03.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621583+0000", 2024-09-16T00:09:03.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026246469999999999 2024-09-16T00:09:03.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621602+0000", 2024-09-16T00:09:03.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9137e-05 2024-09-16T00:09:03.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621629+0000", 2024-09-16T00:09:03.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6451000000000001e-05 2024-09-16T00:09:03.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621648+0000", 2024-09-16T00:09:03.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8941e-05 2024-09-16T00:09:03.336 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.337 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.337 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.337 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:03.337 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:03.337 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.668607399999999, 2024-09-16T00:09:03.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053141380000000004, 2024-09-16T00:09:03.337 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.337 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.338 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.338 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:03.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:03.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:03.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-07 2024-09-16T00:09:03.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.340 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615678+0000", 2024-09-16T00:09:03.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:09:03.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.340 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615682+0000", 2024-09-16T00:09:03.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6659999999999999e-06 2024-09-16T00:09:03.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.340 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615686+0000", 2024-09-16T00:09:03.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6990000000000001e-06 2024-09-16T00:09:03.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615736+0000", 2024-09-16T00:09:03.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9656e-05 2024-09-16T00:09:03.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615864+0000", 2024-09-16T00:09:03.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012836100000000001 2024-09-16T00:09:03.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:03.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615989+0000", 2024-09-16T00:09:03.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000125525 2024-09-16T00:09:03.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617355+0000", 2024-09-16T00:09:03.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001365982 2024-09-16T00:09:03.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620897+0000", 2024-09-16T00:09:03.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035419610000000002 2024-09-16T00:09:03.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620941+0000", 2024-09-16T00:09:03.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3998999999999997e-05 2024-09-16T00:09:03.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.344 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.344 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.344 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620990+0000", 2024-09-16T00:09:03.344 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8199999999999999e-05 2024-09-16T00:09:03.344 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.344 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.344 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.344 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.344 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:03.344 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:03.345 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 84.668580141999996, 2024-09-16T00:09:03.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006316449, 2024-09-16T00:09:03.345 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.345 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.345 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.345 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:03.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.346 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:03.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.346 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:03.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3299999999999998e-07 2024-09-16T00:09:03.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615705+0000", 2024-09-16T00:09:03.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.278e-06 2024-09-16T00:09:03.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615711+0000", 2024-09-16T00:09:03.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8119999999999996e-06 2024-09-16T00:09:03.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615716+0000", 2024-09-16T00:09:03.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4909999999999999e-06 2024-09-16T00:09:03.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617551+0000", 2024-09-16T00:09:03.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00183549 2024-09-16T00:09:03.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617627+0000", 2024-09-16T00:09:03.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6163999999999998e-05 2024-09-16T00:09:03.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:03.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617776+0000", 2024-09-16T00:09:03.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014909499999999999 2024-09-16T00:09:03.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621072+0000", 2024-09-16T00:09:03.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032955150000000002 2024-09-16T00:09:03.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621957+0000", 2024-09-16T00:09:03.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088553199999999996 2024-09-16T00:09:03.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621998+0000", 2024-09-16T00:09:03.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0815000000000001e-05 2024-09-16T00:09:03.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.350 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:09:03.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0924000000000002e-05 2024-09-16T00:09:03.350 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.350 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.350 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.350 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:29 3.1f 3:f8e2d5cc:::10000000214.00000000:head [write 0~2395] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:03.350 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.311759+0000", 2024-09-16T00:09:03.350 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.9725235770000005, 2024-09-16T00:09:03.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001606505, 2024-09-16T00:09:03.351 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.351 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.351 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.351 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:03.351 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:03.351 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:03.351 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 29 2024-09-16T00:09:03.351 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.351 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.351 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.352 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.352 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.311759+0000", 2024-09-16T00:09:03.352 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.352 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.352 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.352 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.311759+0000", 2024-09-16T00:09:03.352 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.352 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.352 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.352 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.311761+0000", 2024-09-16T00:09:03.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.198e-06 2024-09-16T00:09:03.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.311768+0000", 2024-09-16T00:09:03.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6499999999999996e-06 2024-09-16T00:09:03.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.311779+0000", 2024-09-16T00:09:03.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1279e-05 2024-09-16T00:09:03.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.311787+0000", 2024-09-16T00:09:03.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0090000000000007e-06 2024-09-16T00:09:03.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.311846+0000", 2024-09-16T00:09:03.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8400999999999999e-05 2024-09-16T00:09:03.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.311987+0000", 2024-09-16T00:09:03.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000141571 2024-09-16T00:09:03.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:03.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.312122+0000", 2024-09-16T00:09:03.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013493700000000001 2024-09-16T00:09:03.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.313195+0000", 2024-09-16T00:09:03.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010728630000000001 2024-09-16T00:09:03.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.313262+0000", 2024-09-16T00:09:03.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7013000000000004e-05 2024-09-16T00:09:03.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.313316+0000", 2024-09-16T00:09:03.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4073000000000002e-05 2024-09-16T00:09:03.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.313366+0000", 2024-09-16T00:09:03.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9511000000000001e-05 2024-09-16T00:09:03.357 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.357 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.357 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.357 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:501 3.38 3:1f24713e:::1000000042b.00000000:head [write 0~71] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:03.357 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.761706+0000", 2024-09-16T00:09:03.357 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.5225773350000003, 2024-09-16T00:09:03.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001845941, 2024-09-16T00:09:03.357 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.357 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.358 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.358 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:03.358 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:03.358 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:03.358 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 501 2024-09-16T00:09:03.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.358 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761706+0000", 2024-09-16T00:09:03.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761706+0000", 2024-09-16T00:09:03.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761707+0000", 2024-09-16T00:09:03.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8509999999999999e-06 2024-09-16T00:09:03.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761717+0000", 2024-09-16T00:09:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9250000000000007e-06 2024-09-16T00:09:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761732+0000", 2024-09-16T00:09:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4201e-05 2024-09-16T00:09:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761738+0000", 2024-09-16T00:09:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7299999999999999e-06 2024-09-16T00:09:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761819+0000", 2024-09-16T00:09:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1044999999999994e-05 2024-09-16T00:09:03.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761936+0000", 2024-09-16T00:09:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011626299999999999 2024-09-16T00:09:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.762051+0000", 2024-09-16T00:09:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011582999999999999 2024-09-16T00:09:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.762671+0000", 2024-09-16T00:09:03.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00061922099999999999 2024-09-16T00:09:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.763496+0000", 2024-09-16T00:09:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00082528899999999999 2024-09-16T00:09:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.763525+0000", 2024-09-16T00:09:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9488999999999998e-05 2024-09-16T00:09:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.763552+0000", 2024-09-16T00:09:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6097000000000001e-05 2024-09-16T00:09:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:584 3.22 3:47e2f9af:::10000000492.00000000:head [write 0~10710] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.989570+0000", 2024-09-16T00:09:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.2947125589999997, 2024-09-16T00:09:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038063620000000002, 2024-09-16T00:09:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.364 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 584 2024-09-16T00:09:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989570+0000", 2024-09-16T00:09:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989570+0000", 2024-09-16T00:09:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989573+0000", 2024-09-16T00:09:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9909999999999998e-06 2024-09-16T00:09:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989602+0000", 2024-09-16T00:09:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8353000000000001e-05 2024-09-16T00:09:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989622+0000", 2024-09-16T00:09:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9905999999999999e-05 2024-09-16T00:09:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989631+0000", 2024-09-16T00:09:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6360000000000006e-06 2024-09-16T00:09:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989706+0000", 2024-09-16T00:09:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4493000000000007e-05 2024-09-16T00:09:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989856+0000", 2024-09-16T00:09:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015023700000000001 2024-09-16T00:09:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.990004+0000", 2024-09-16T00:09:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000148157 2024-09-16T00:09:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.992924+0000", 2024-09-16T00:09:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029203250000000001 2024-09-16T00:09:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.993270+0000", 2024-09-16T00:09:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00034548899999999998 2024-09-16T00:09:03.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.993325+0000", 2024-09-16T00:09:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5337999999999998e-05 2024-09-16T00:09:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.993377+0000", 2024-09-16T00:09:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1437000000000002e-05 2024-09-16T00:09:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:589 3.22 3:44283f82:::10000000498.00000000:head [write 0~1058] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.007191+0000", 2024-09-16T00:09:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.2770914959999997, 2024-09-16T00:09:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003237729, 2024-09-16T00:09:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 589 2024-09-16T00:09:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007191+0000", 2024-09-16T00:09:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007191+0000", 2024-09-16T00:09:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007193+0000", 2024-09-16T00:09:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8449999999999999e-06 2024-09-16T00:09:03.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007206+0000", 2024-09-16T00:09:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2509e-05 2024-09-16T00:09:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007223+0000", 2024-09-16T00:09:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7220000000000001e-05 2024-09-16T00:09:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007231+0000", 2024-09-16T00:09:03.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9370000000000004e-06 2024-09-16T00:09:03.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007285+0000", 2024-09-16T00:09:03.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4472000000000003e-05 2024-09-16T00:09:03.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op must wait for map", 2024-09-16T00:09:03.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007289+0000", 2024-09-16T00:09:03.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0679999999999998e-06 2024-09-16T00:09:03.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009018+0000", 2024-09-16T00:09:03.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017285270000000001 2024-09-16T00:09:03.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009121+0000", 2024-09-16T00:09:03.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010267 2024-09-16T00:09:03.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:03.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009236+0000", 2024-09-16T00:09:03.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115563 2024-09-16T00:09:03.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009903+0000", 2024-09-16T00:09:03.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00066633999999999997 2024-09-16T00:09:03.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.010348+0000", 2024-09-16T00:09:03.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00044550999999999998 2024-09-16T00:09:03.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.010388+0000", 2024-09-16T00:09:03.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0240000000000001e-05 2024-09-16T00:09:03.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.010429+0000", 2024-09-16T00:09:03.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0828000000000003e-05 2024-09-16T00:09:03.377 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.377 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.377 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.377 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:590 3.21 3:86dbbdca:::10000000499.00000000:head [write 0~3104] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:03.377 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.009656+0000", 2024-09-16T00:09:03.378 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.2746270400000004, 2024-09-16T00:09:03.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001717283, 2024-09-16T00:09:03.378 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.378 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.378 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.378 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:03.378 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:03.378 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:03.378 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 590 2024-09-16T00:09:03.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.378 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009656+0000", 2024-09-16T00:09:03.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009656+0000", 2024-09-16T00:09:03.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009657+0000", 2024-09-16T00:09:03.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.514e-06 2024-09-16T00:09:03.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009664+0000", 2024-09-16T00:09:03.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6819999999999997e-06 2024-09-16T00:09:03.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009677+0000", 2024-09-16T00:09:03.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2841e-05 2024-09-16T00:09:03.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009687+0000", 2024-09-16T00:09:03.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0171e-05 2024-09-16T00:09:03.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009742+0000", 2024-09-16T00:09:03.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4855999999999998e-05 2024-09-16T00:09:03.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009874+0000", 2024-09-16T00:09:03.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013217099999999999 2024-09-16T00:09:03.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:03.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009994+0000", 2024-09-16T00:09:03.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000120252 2024-09-16T00:09:03.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.010890+0000", 2024-09-16T00:09:03.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00089574900000000005 2024-09-16T00:09:03.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.011289+0000", 2024-09-16T00:09:03.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00039914799999999999 2024-09-16T00:09:03.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.011336+0000", 2024-09-16T00:09:03.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6306e-05 2024-09-16T00:09:03.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.011373+0000", 2024-09-16T00:09:03.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7592999999999997e-05 2024-09-16T00:09:03.383 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.383 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.383 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.383 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:665 3.3d e59/20 3:be78f56d:::100000004f7.00000000:head v 59'10, mlcod=58'8)", 2024-09-16T00:09:03.383 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.222373+0000", 2024-09-16T00:09:03.384 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.0619102839999996, 2024-09-16T00:09:03.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017385230000000001, 2024-09-16T00:09:03.384 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.384 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.384 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.384 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222373+0000", 2024-09-16T00:09:03.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222373+0000", 2024-09-16T00:09:03.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222374+0000", 2024-09-16T00:09:03.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5680000000000001e-06 2024-09-16T00:09:03.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222384+0000", 2024-09-16T00:09:03.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9690000000000006e-06 2024-09-16T00:09:03.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222405+0000", 2024-09-16T00:09:03.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0840999999999999e-05 2024-09-16T00:09:03.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222427+0000", 2024-09-16T00:09:03.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2348e-05 2024-09-16T00:09:03.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222489+0000", 2024-09-16T00:09:03.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1884000000000006e-05 2024-09-16T00:09:03.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222512+0000", 2024-09-16T00:09:03.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2464999999999999e-05 2024-09-16T00:09:03.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.224062+0000", 2024-09-16T00:09:03.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001550685 2024-09-16T00:09:03.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.224111+0000", 2024-09-16T00:09:03.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8763000000000001e-05 2024-09-16T00:09:03.388 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.388 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.388 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.388 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:714 3.22 3:453cbc5d:::10000000536.00000000:head [write 0~260] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:03.388 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.356475+0000", 2024-09-16T00:09:03.388 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.927808261, 2024-09-16T00:09:03.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003898926, 2024-09-16T00:09:03.388 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.388 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.388 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.389 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:03.389 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:03.389 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:03.389 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 714 2024-09-16T00:09:03.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.389 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356475+0000", 2024-09-16T00:09:03.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356475+0000", 2024-09-16T00:09:03.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356476+0000", 2024-09-16T00:09:03.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6750000000000001e-06 2024-09-16T00:09:03.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356493+0000", 2024-09-16T00:09:03.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6562000000000002e-05 2024-09-16T00:09:03.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356509+0000", 2024-09-16T00:09:03.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5908000000000001e-05 2024-09-16T00:09:03.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356516+0000", 2024-09-16T00:09:03.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9079999999999998e-06 2024-09-16T00:09:03.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356573+0000", 2024-09-16T00:09:03.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7243000000000001e-05 2024-09-16T00:09:03.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356657+0000", 2024-09-16T00:09:03.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3809999999999999e-05 2024-09-16T00:09:03.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:03.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356742+0000", 2024-09-16T00:09:03.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5704000000000005e-05 2024-09-16T00:09:03.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360000+0000", 2024-09-16T00:09:03.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003257916 2024-09-16T00:09:03.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360279+0000", 2024-09-16T00:09:03.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00027872500000000001 2024-09-16T00:09:03.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360324+0000", 2024-09-16T00:09:03.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5359999999999999e-05 2024-09-16T00:09:03.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360374+0000", 2024-09-16T00:09:03.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9115e-05 2024-09-16T00:09:03.394 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.394 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.394 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.394 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:766 3.1f 3:f8970b5b:::10000000571.00000000:head [write 0~1989] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:03.394 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.481586+0000", 2024-09-16T00:09:03.394 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.8026966059999996, 2024-09-16T00:09:03.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038531529999999998, 2024-09-16T00:09:03.394 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.394 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.394 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.395 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:03.395 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:03.395 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:03.395 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 766 2024-09-16T00:09:03.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.395 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481586+0000", 2024-09-16T00:09:03.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481586+0000", 2024-09-16T00:09:03.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481588+0000", 2024-09-16T00:09:03.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.691e-06 2024-09-16T00:09:03.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481595+0000", 2024-09-16T00:09:03.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5780000000000004e-06 2024-09-16T00:09:03.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481605+0000", 2024-09-16T00:09:03.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0729e-05 2024-09-16T00:09:03.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481610+0000", 2024-09-16T00:09:03.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:09:03.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481663+0000", 2024-09-16T00:09:03.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2429999999999998e-05 2024-09-16T00:09:03.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481740+0000", 2024-09-16T00:09:03.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7526000000000003e-05 2024-09-16T00:09:03.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:03.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481829+0000", 2024-09-16T00:09:03.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8265999999999998e-05 2024-09-16T00:09:03.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.482578+0000", 2024-09-16T00:09:03.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074936600000000001 2024-09-16T00:09:03.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.485314+0000", 2024-09-16T00:09:03.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027358949999999999 2024-09-16T00:09:03.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.485398+0000", 2024-09-16T00:09:03.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4462000000000005e-05 2024-09-16T00:09:03.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.485439+0000", 2024-09-16T00:09:03.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1066999999999998e-05 2024-09-16T00:09:03.400 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.400 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.400 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.400 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:816 3.21 3:86b80525:::100000005a6.00000000:head [write 0~4693] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:03.400 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.596020+0000", 2024-09-16T00:09:03.400 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.6882631720000001, 2024-09-16T00:09:03.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016622049999999999, 2024-09-16T00:09:03.400 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.400 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.401 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.401 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:03.401 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:03.401 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:03.401 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 816 2024-09-16T00:09:03.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.401 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596020+0000", 2024-09-16T00:09:03.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596020+0000", 2024-09-16T00:09:03.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596023+0000", 2024-09-16T00:09:03.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9790000000000002e-06 2024-09-16T00:09:03.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596041+0000", 2024-09-16T00:09:03.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8003999999999999e-05 2024-09-16T00:09:03.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596065+0000", 2024-09-16T00:09:03.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4646000000000001e-05 2024-09-16T00:09:03.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596079+0000", 2024-09-16T00:09:03.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.382e-05 2024-09-16T00:09:03.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596151+0000", 2024-09-16T00:09:03.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1727e-05 2024-09-16T00:09:03.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596305+0000", 2024-09-16T00:09:03.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015395799999999999 2024-09-16T00:09:03.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:03.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596468+0000", 2024-09-16T00:09:03.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016360499999999999 2024-09-16T00:09:03.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.597256+0000", 2024-09-16T00:09:03.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00078736799999999999 2024-09-16T00:09:03.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.597596+0000", 2024-09-16T00:09:03.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00034032499999999999 2024-09-16T00:09:03.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.597651+0000", 2024-09-16T00:09:03.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5148999999999997e-05 2024-09-16T00:09:03.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.597682+0000", 2024-09-16T00:09:03.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0623999999999997e-05 2024-09-16T00:09:03.406 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.406 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.406 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.406 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:839 3.38 3:1d7725f9:::100000005c8.00000000:head [write 0~3439] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:03.406 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.686484+0000", 2024-09-16T00:09:03.406 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.5977985209999996, 2024-09-16T00:09:03.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016765650000000001, 2024-09-16T00:09:03.406 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:03.407 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:03.407 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:03.407 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:03.407 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:03.407 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:03.407 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 839 2024-09-16T00:09:03.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.407 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:03.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:03.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686484+0000", 2024-09-16T00:09:03.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:03.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686484+0000", 2024-09-16T00:09:03.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:03.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:03.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686487+0000", 2024-09-16T00:09:03.409 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:09:03.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2989999999999999e-06 2024-09-16T00:09:03.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:03.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686501+0000", 2024-09-16T00:09:03.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4592e-05 2024-09-16T00:09:03.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:03.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686521+0000", 2024-09-16T00:09:03.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9808000000000001e-05 2024-09-16T00:09:03.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:03.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686534+0000", 2024-09-16T00:09:03.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2561000000000001e-05 2024-09-16T00:09:03.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:03.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686609+0000", 2024-09-16T00:09:03.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5315999999999995e-05 2024-09-16T00:09:03.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:03.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686820+0000", 2024-09-16T00:09:03.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00021137900000000001 2024-09-16T00:09:03.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:03.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686930+0000", 2024-09-16T00:09:03.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110158 2024-09-16T00:09:03.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:03.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.687631+0000", 2024-09-16T00:09:03.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000700393 2024-09-16T00:09:03.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:03.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.688087+0000", 2024-09-16T00:09:03.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000456432 2024-09-16T00:09:03.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:03.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.688122+0000", 2024-09-16T00:09:03.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4944000000000003e-05 2024-09-16T00:09:03.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:03.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:03.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:03.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.688161+0000", 2024-09-16T00:09:03.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8683000000000002e-05 2024-09-16T00:09:03.414 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.414 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.414 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.414 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:03.414 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:03.414 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:03.414 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:09:03.415 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:03.408+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:03.510 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:03.509+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:03.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.567+0000 7fa4694ae640 1 Processor -- start 2024-09-16T00:09:03.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.567+0000 7fa4694ae640 1 -- start start 2024-09-16T00:09:03.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.568+0000 7fa4694ae640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa4640c6e20 0x7fa46415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:03.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.568+0000 7fa4694ae640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa46415b080 con 0x7fa4640d0970 2024-09-16T00:09:03.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.568+0000 7fa4694ae640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa46415b360 con 0x7fa4640d0590 2024-09-16T00:09:03.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.568+0000 7fa4694ae640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa46415b920 con 0x7fa4640c6e20 2024-09-16T00:09:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.568+0000 7fa4637fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa4640c6e20 0x7fa46415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.568+0000 7fa462ffd640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa4640d0590 0x7fa4640c5350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:45622/0 (socket says 172.21.15.139:45622) 2024-09-16T00:09:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.568+0000 7fa462ffd640 1 -- 172.21.15.139:0/1782992741 learned_addr learned my addr 172.21.15.139:0/1782992741 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.568+0000 7fa45bfff640 1 --1- 172.21.15.139:0/1782992741 >> v1:172.21.15.139:6789/0 conn(0x7fa4640d0970 0x7fa4640c6710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:60368/0 (socket says 172.21.15.139:60368) 2024-09-16T00:09:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.568+0000 7fa4627fc640 1 -- 172.21.15.139:0/1782992741 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1125480096 0 0) 0x7fa46415b360 con 0x7fa4640d0590 2024-09-16T00:09:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.568+0000 7fa4637fe640 1 -- 172.21.15.139:0/1782992741 >> v1:172.21.15.164:6789/0 conn(0x7fa4640d0590 legacy=0x7fa4640c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.568+0000 7fa4637fe640 1 -- 172.21.15.139:0/1782992741 >> v1:172.21.15.139:6789/0 conn(0x7fa4640d0970 legacy=0x7fa4640c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.568+0000 7fa4637fe640 1 -- 172.21.15.139:0/1782992741 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46415b640 con 0x7fa4640c6e20 2024-09-16T00:09:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.569+0000 7fa4637fe640 1 --2- 172.21.15.139:0/1782992741 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4640c6e20 0x7fa46415a9f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009d90 tx=0x7fa45c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572fe75d373f198f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.569+0000 7fa4627fc640 1 -- 172.21.15.139:0/1782992741 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c007740 con 0x7fa4640c6e20 2024-09-16T00:09:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.570+0000 7fa4627fc640 1 -- 172.21.15.139:0/1782992741 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa45c0078e0 con 0x7fa4640c6e20 2024-09-16T00:09:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.570+0000 7fa4627fc640 1 -- 172.21.15.139:0/1782992741 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c007cb0 con 0x7fa4640c6e20 2024-09-16T00:09:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.570+0000 7fa4694ae640 1 -- 172.21.15.139:0/1782992741 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4640c6e20 msgr2=0x7fa46415a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.570+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1782992741 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4640c6e20 0x7fa46415a9f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009d90 tx=0x7fa45c02a680 comp rx=0 tx=0).stop 2024-09-16T00:09:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.570+0000 7fa4694ae640 1 -- 172.21.15.139:0/1782992741 shutdown_connections 2024-09-16T00:09:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.570+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1782992741 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4640c6e20 0x7fa46415a9f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.570+0000 7fa4694ae640 1 -- 172.21.15.139:0/1782992741 >> 172.21.15.139:0/1782992741 conn(0x7fa46406c3c0 msgr2=0x7fa46406c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.571+0000 7fa4694ae640 1 -- 172.21.15.139:0/1782992741 shutdown_connections 2024-09-16T00:09:03.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.571+0000 7fa4694ae640 1 -- 172.21.15.139:0/1782992741 wait complete. 2024-09-16T00:09:03.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.572+0000 7fa4694ae640 1 Processor -- start 2024-09-16T00:09:03.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.572+0000 7fa4694ae640 1 -- start start 2024-09-16T00:09:03.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa4694ae640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa4640c6e20 0x7fa4641495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:03.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa4694ae640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4640d0590 0x7fa4641586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:03.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa4694ae640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4640d0970 0x7fa464158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:03.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa4694ae640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4640c5a60 con 0x7fa4640d0590 2024-09-16T00:09:03.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa4694ae640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa4640c6280 con 0x7fa4640d0970 2024-09-16T00:09:03.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa4694ae640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa4640c5be0 con 0x7fa4640c6e20 2024-09-16T00:09:03.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa45bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4640d0590 0x7fa4641586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:03.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa45bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4640d0590 0x7fa4641586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42026/0 (socket says 172.21.15.139:42026) 2024-09-16T00:09:03.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa4637fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4640d0970 0x7fa464158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:03.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa45bfff640 1 -- 172.21.15.139:0/1624966129 learned_addr learned my addr 172.21.15.139:0/1624966129 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:03.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa462ffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa4640c6e20 0x7fa4641495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:03.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa45bfff640 1 -- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa4640c6e20 msgr2=0x7fa4641495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa45bfff640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa4640c6e20 0x7fa4641495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa45bfff640 1 -- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4640d0970 msgr2=0x7fa464158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa45bfff640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4640d0970 0x7fa464158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa45bfff640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4640c7c30 con 0x7fa4640d0590 2024-09-16T00:09:03.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa4637fe640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa4640d0970 0x7fa464158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:03.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.573+0000 7fa45bfff640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4640d0590 0x7fa4641586e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00c570 tx=0x7fa44c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:03.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.574+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c018070 con 0x7fa4640d0590 2024-09-16T00:09:03.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.574+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa44c0074e0 con 0x7fa4640d0590 2024-09-16T00:09:03.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.574+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa46415a690 con 0x7fa4640d0590 2024-09-16T00:09:03.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.574+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c007690 con 0x7fa4640d0590 2024-09-16T00:09:03.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.574+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa464149f40 con 0x7fa4640d0590 2024-09-16T00:09:03.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.575+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fa46415a9f0 con 0x7fa4640d0590 2024-09-16T00:09:03.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.576+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa44c01b070 con 0x7fa4640d0590 2024-09-16T00:09:03.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.576+0000 7fa45b7fe640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa44406c2b0 0x7fa44406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:03.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.576+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5915+0+0 (secure 0 0 0) 0x7fa44c08a980 con 0x7fa4640d0590 2024-09-16T00:09:03.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.576+0000 7fa462ffd640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa44406c2b0 0x7fa44406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:03.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.576+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v62) ==== 54+0+16 (secure 0 0 0) 0x7fa44c08f200 con 0x7fa4640d0590 2024-09-16T00:09:03.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.577+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fa464149b30 0x7fa464146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:03.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.577+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa4641f0d60 con 0x7fa464149b30 2024-09-16T00:09:03.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.577+0000 7fa4637fe640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fa464149b30 0x7fa464146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:03.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.577+0000 7fa462ffd640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa44406c2b0 0x7fa44406e770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa454007f70 tx=0x7fa454007f00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:03.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.577+0000 7fa4637fe640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fa464149b30 0x7fa464146b00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:03.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.578+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa4641f0d60 con 0x7fa464149b30 2024-09-16T00:09:03.579 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:03.579 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:03.580 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:03.580 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:03.590 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:09:03.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.595+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa4641e5f90 con 0x7fa464149b30 2024-09-16T00:09:03.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.597+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa4641e5f90 con 0x7fa464149b30 2024-09-16T00:09:03.598 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:03.598 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:03.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.597+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fa464148d10 0x7fa4641f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:03.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.597+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fa464147550 con 0x7fa464148d10 2024-09-16T00:09:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.597+0000 7fa462ffd640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fa464148d10 0x7fa4641f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.598+0000 7fa462ffd640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fa464148d10 0x7fa4641f3850 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:03.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.598+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa464147550 con 0x7fa464148d10 2024-09-16T00:09:03.611 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:03.610+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:03.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.616+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa4640cf1c0 con 0x7fa464148d10 2024-09-16T00:09:03.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.618+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa4640cf1c0 con 0x7fa464148d10 2024-09-16T00:09:03.619 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:03.619 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:03.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.618+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fa464147b30 0x7fa4641f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:03.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.618+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fa4641e6790 con 0x7fa464147b30 2024-09-16T00:09:03.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.618+0000 7fa45bfff640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fa464147b30 0x7fa4641f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:03.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.619+0000 7fa45bfff640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fa464147b30 0x7fa4641f42a0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:03.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.619+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa4641e6790 con 0x7fa464147b30 2024-09-16T00:09:03.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.637+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa4641e6bf0 con 0x7fa464147b30 2024-09-16T00:09:03.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.639+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa4641e6bf0 con 0x7fa464147b30 2024-09-16T00:09:03.640 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:03.641 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:03.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.640+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fa464062630 0x7fa464062a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:03.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.640+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fa4641e76d0 con 0x7fa464062630 2024-09-16T00:09:03.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.640+0000 7fa4637fe640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fa464062630 0x7fa464062a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:03.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.640+0000 7fa4637fe640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fa464062630 0x7fa464062a50 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:03.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.641+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa4641e76d0 con 0x7fa464062630 2024-09-16T00:09:03.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.659+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa464063260 con 0x7fa464062630 2024-09-16T00:09:03.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.662+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa464063260 con 0x7fa464062630 2024-09-16T00:09:03.663 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:03.663 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:03.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.662+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fa4640633d0 0x7fa46402cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:03.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.662+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fa46402d090 con 0x7fa4640633d0 2024-09-16T00:09:03.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.662+0000 7fa462ffd640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fa4640633d0 0x7fa46402cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:03.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.663+0000 7fa462ffd640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fa4640633d0 0x7fa46402cb50 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:03.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.663+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa46402d090 con 0x7fa4640633d0 2024-09-16T00:09:03.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.688+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa46402d090 con 0x7fa4640633d0 2024-09-16T00:09:03.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.689+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa46402d090 con 0x7fa4640633d0 2024-09-16T00:09:03.690 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:03.691 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:03.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.690+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fa46402d710 0x7fa464206be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:03.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.690+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fa46402db90 con 0x7fa46402d710 2024-09-16T00:09:03.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.690+0000 7fa45bfff640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fa46402d710 0x7fa464206be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:03.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.690+0000 7fa45bfff640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fa46402d710 0x7fa464206be0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:03.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.691+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa44c090020 con 0x7fa46402d710 2024-09-16T00:09:03.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.708+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa464207120 con 0x7fa46402d710 2024-09-16T00:09:03.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.709+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa464207120 con 0x7fa46402d710 2024-09-16T00:09:03.710 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:03.711 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:03.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.710+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fa464207840 0x7fa464209c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:03.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.710+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fa46420cbb0 con 0x7fa464207840 2024-09-16T00:09:03.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.710+0000 7fa4637fe640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fa464207840 0x7fa464209c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:03.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.710+0000 7fa4637fe640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fa464207840 0x7fa464209c30 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:03.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.711+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa46420cbb0 con 0x7fa464207840 2024-09-16T00:09:03.712 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:03.712+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:03.730 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:03.730 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:03.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.729+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa4640755b0 con 0x7fa464207840 2024-09-16T00:09:03.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.731+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa4640755b0 con 0x7fa464207840 2024-09-16T00:09:03.732 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:03.733 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:09:03.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.732+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fa46420be60 0x7fa46420c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:03.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.732+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fa46420ff80 con 0x7fa46420be60 2024-09-16T00:09:03.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.732+0000 7fa462ffd640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fa46420be60 0x7fa46420c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:03.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.732+0000 7fa462ffd640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fa46420be60 0x7fa46420c2a0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:03.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.733+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa46420ff80 con 0x7fa46420be60 2024-09-16T00:09:03.741 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:09:03.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.750+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fa46420a210 con 0x7fa46420be60 2024-09-16T00:09:03.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa45b7fe640 1 -- 172.21.15.139:0/1624966129 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fa46420a210 con 0x7fa46420be60 2024-09-16T00:09:03.754 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:03.755 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:03.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fa464149b30 msgr2=0x7fa464146b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fa464149b30 0x7fa464146b00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fa464148d10 msgr2=0x7fa4641f3850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fa464148d10 0x7fa4641f3850 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fa464147b30 msgr2=0x7fa4641f42a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fa464147b30 0x7fa4641f42a0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fa464062630 msgr2=0x7fa464062a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fa464062630 0x7fa464062a50 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fa4640633d0 msgr2=0x7fa46402cb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fa4640633d0 0x7fa46402cb50 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fa46402d710 msgr2=0x7fa464206be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fa46402d710 0x7fa464206be0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4637fe640 1 -- 172.21.15.139:0/1624966129 reap_dead start 2024-09-16T00:09:03.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fa464207840 msgr2=0x7fa464209c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fa464207840 0x7fa464209c30 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4637fe640 1 -- 172.21.15.139:0/1624966129 reap_dead start 2024-09-16T00:09:03.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fa46420be60 msgr2=0x7fa46420c2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fa46420be60 0x7fa46420c2a0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4637fe640 1 -- 172.21.15.139:0/1624966129 reap_dead start 2024-09-16T00:09:03.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4637fe640 1 -- 172.21.15.139:0/1624966129 reap_dead start 2024-09-16T00:09:03.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa44406c2b0 msgr2=0x7fa44406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa44406c2b0 0x7fa44406e770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa454007f70 tx=0x7fa454007f00 comp rx=0 tx=0).stop 2024-09-16T00:09:03.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4640d0590 msgr2=0x7fa4641586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:03.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4640d0590 0x7fa4641586e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00c570 tx=0x7fa44c00ca30 comp rx=0 tx=0).stop 2024-09-16T00:09:03.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 shutdown_connections 2024-09-16T00:09:03.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa44406c2b0 0x7fa44406e770 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 --2- 172.21.15.139:0/1624966129 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4640d0590 0x7fa4641586e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:03.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 >> 172.21.15.139:0/1624966129 conn(0x7fa46406c3c0 msgr2=0x7fa4640b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:03.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 shutdown_connections 2024-09-16T00:09:03.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:03.753+0000 7fa4694ae640 1 -- 172.21.15.139:0/1624966129 wait complete. 2024-09-16T00:09:03.813 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:03.812+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:03.877 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:03.877 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:03.888 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:09:03.930 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:03.929+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:04.015 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:04.015+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:04.032 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:04.032 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:04.042 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:09:04.126 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:04.126+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:04.212 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:04.212 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:04.217 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:04.216+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:04.225 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:09:04.318 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:04.318+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:04.390 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:04.390 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:04.400 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:09:04.419 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:04.419+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:04.520 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:04.519+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:04.560 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:04.560 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:04.570 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:09:04.621 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:04.620+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:04.689 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:04.689 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:04.699 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:09:04.744 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:04.744+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:04.816 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:04.816 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:04.822 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:04.822+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:04.826 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:09:04.924 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:04.923+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:04.979 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:04.979 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:04.988 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:09:05.025 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:05.024+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:05.036 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T00:09:05.113 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:05.113 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:05.123 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:09:05.125 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:05.124+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:05.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.140+0000 7faa1a25e640 1 Processor -- start 2024-09-16T00:09:05.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.140+0000 7faa1a25e640 1 -- start start 2024-09-16T00:09:05.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.141+0000 7faa1a25e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faa14068060 0x7faa1415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.141+0000 7faa1a25e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa140ccc90 0x7faa140cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.141+0000 7faa1a25e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa140cc4b0 con 0x7faa14154c20 2024-09-16T00:09:05.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.141+0000 7faa1a25e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7faa140c5830 con 0x7faa14068060 2024-09-16T00:09:05.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.141+0000 7faa1a25e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7faa1415b7c0 con 0x7faa140ccc90 2024-09-16T00:09:05.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.141+0000 7faa12ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faa14068060 0x7faa1415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.141+0000 7faa137fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7faa14154c20 0x7faa140c5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:60372/0 (socket says 172.21.15.139:60372) 2024-09-16T00:09:05.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.141+0000 7faa137fe640 1 -- 172.21.15.139:0/376363692 learned_addr learned my addr 172.21.15.139:0/376363692 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:05.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.141+0000 7faa13fff640 1 --2- 172.21.15.139:0/376363692 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa140ccc90 0x7faa140cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.142+0000 7faa12ffd640 1 -- 172.21.15.139:0/376363692 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa140ccc90 msgr2=0x7faa140cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:05.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.142+0000 7faa12ffd640 1 --2- 172.21.15.139:0/376363692 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa140ccc90 0x7faa140cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:05.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.142+0000 7faa12ffd640 1 -- 172.21.15.139:0/376363692 >> v1:172.21.15.139:6789/0 conn(0x7faa14154c20 legacy=0x7faa140c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:05.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.142+0000 7faa12ffd640 1 -- 172.21.15.139:0/376363692 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa140cc790 con 0x7faa14068060 2024-09-16T00:09:05.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.142+0000 7faa127fc640 1 -- 172.21.15.139:0/376363692 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4254308003 0 0) 0x7faa140cc4b0 con 0x7faa14154c20 2024-09-16T00:09:05.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.142+0000 7faa12ffd640 1 --2- 172.21.15.139:0/376363692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14068060 0x7faa1415b280 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faa08009790 tx=0x7faa0802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a06e6762e2af88 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:05.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.143+0000 7faa127fc640 1 -- 172.21.15.139:0/376363692 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08037070 con 0x7faa14068060 2024-09-16T00:09:05.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.143+0000 7faa127fc640 1 -- 172.21.15.139:0/376363692 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa08032440 con 0x7faa14068060 2024-09-16T00:09:05.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.143+0000 7faa127fc640 1 -- 172.21.15.139:0/376363692 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08032810 con 0x7faa14068060 2024-09-16T00:09:05.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.143+0000 7faa1a25e640 1 -- 172.21.15.139:0/376363692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14068060 msgr2=0x7faa1415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:05.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.143+0000 7faa1a25e640 1 --2- 172.21.15.139:0/376363692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14068060 0x7faa1415b280 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faa08009790 tx=0x7faa0802a180 comp rx=0 tx=0).stop 2024-09-16T00:09:05.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.143+0000 7faa1a25e640 1 -- 172.21.15.139:0/376363692 shutdown_connections 2024-09-16T00:09:05.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.143+0000 7faa1a25e640 1 --2- 172.21.15.139:0/376363692 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa140ccc90 0x7faa140cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:05.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.143+0000 7faa1a25e640 1 --2- 172.21.15.139:0/376363692 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14068060 0x7faa1415b280 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:05.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.143+0000 7faa1a25e640 1 -- 172.21.15.139:0/376363692 >> 172.21.15.139:0/376363692 conn(0x7faa1406ca40 msgr2=0x7faa14067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:05.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.143+0000 7faa1a25e640 1 -- 172.21.15.139:0/376363692 shutdown_connections 2024-09-16T00:09:05.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.144+0000 7faa1a25e640 1 -- 172.21.15.139:0/376363692 wait complete. 2024-09-16T00:09:05.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.145+0000 7faa1a25e640 1 Processor -- start 2024-09-16T00:09:05.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.145+0000 7faa1a25e640 1 -- start start 2024-09-16T00:09:05.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa1a25e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa14068060 0x7faa141f0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa1a25e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa140ccc90 0x7faa140ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa1a25e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14154c20 0x7faa140b5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa1a25e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faa1415e0d0 con 0x7faa14154c20 2024-09-16T00:09:05.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa1a25e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faa140cc260 con 0x7faa14068060 2024-09-16T00:09:05.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa1a25e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7faa1415e3b0 con 0x7faa140ccc90 2024-09-16T00:09:05.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa13fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14154c20 0x7faa140b5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa12ffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa140ccc90 0x7faa140ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa13fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14154c20 0x7faa140b5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42044/0 (socket says 172.21.15.139:42044) 2024-09-16T00:09:05.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa13fff640 1 -- 172.21.15.139:0/2673534691 learned_addr learned my addr 172.21.15.139:0/2673534691 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:05.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa137fe640 1 --2- 172.21.15.139:0/2673534691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa14068060 0x7faa141f0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa12ffd640 1 -- 172.21.15.139:0/2673534691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa14068060 msgr2=0x7faa141f0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:05.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa12ffd640 1 --2- 172.21.15.139:0/2673534691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa14068060 0x7faa141f0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:05.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa12ffd640 1 -- 172.21.15.139:0/2673534691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14154c20 msgr2=0x7faa140b5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:05.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa12ffd640 1 --2- 172.21.15.139:0/2673534691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14154c20 0x7faa140b5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:05.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa12ffd640 1 -- 172.21.15.139:0/2673534691 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1415ec50 con 0x7faa140ccc90 2024-09-16T00:09:05.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa13fff640 1 --2- 172.21.15.139:0/2673534691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14154c20 0x7faa140b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:05.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.146+0000 7faa12ffd640 1 --2- 172.21.15.139:0/2673534691 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa140ccc90 0x7faa140ce160 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faa14074130 tx=0x7faa080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:05.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.147+0000 7fa9fbfff640 1 -- 172.21.15.139:0/2673534691 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08037040 con 0x7faa140ccc90 2024-09-16T00:09:05.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.147+0000 7fa9fbfff640 1 -- 172.21.15.139:0/2673534691 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa08042070 con 0x7faa140ccc90 2024-09-16T00:09:05.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.147+0000 7fa9fbfff640 1 -- 172.21.15.139:0/2673534691 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08006030 con 0x7faa140ccc90 2024-09-16T00:09:05.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.147+0000 7faa1a25e640 1 -- 172.21.15.139:0/2673534691 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa140d0d20 con 0x7faa140ccc90 2024-09-16T00:09:05.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.147+0000 7faa1a25e640 1 -- 172.21.15.139:0/2673534691 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faa140cd2c0 con 0x7faa140ccc90 2024-09-16T00:09:05.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.148+0000 7faa1a25e640 1 -- 172.21.15.139:0/2673534691 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa1415e690 con 0x7faa140ccc90 2024-09-16T00:09:05.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.148+0000 7fa9fbfff640 1 -- 172.21.15.139:0/2673534691 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7faa080061d0 con 0x7faa140ccc90 2024-09-16T00:09:05.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.149+0000 7fa9fbfff640 1 --2- 172.21.15.139:0/2673534691 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faa0006c2b0 0x7faa0006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.149+0000 7fa9fbfff640 1 -- 172.21.15.139:0/2673534691 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(63..63 src has 1..63) ==== 5915+0+0 (secure 0 0 0) 0x7faa080abca0 con 0x7faa140ccc90 2024-09-16T00:09:05.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.149+0000 7faa137fe640 1 --2- 172.21.15.139:0/2673534691 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faa0006c2b0 0x7faa0006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.149+0000 7faa137fe640 1 --2- 172.21.15.139:0/2673534691 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faa0006c2b0 0x7faa0006e770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faa04000f60 tx=0x7faa04007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:05.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.154+0000 7fa9fbfff640 1 -- 172.21.15.139:0/2673534691 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faa1415ec50 con 0x7faa140ccc90 2024-09-16T00:09:05.226 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:05.225+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:05.240 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:05.240 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:05.249 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:09:05.326 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:05.326+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:05.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.371+0000 7faa1a25e640 1 -- 172.21.15.139:0/2673534691 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7faa140cca70 con 0x7faa140ccc90 2024-09-16T00:09:05.409 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:05.410 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:05.420 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:09:05.428 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:05.427+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:05.528 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:05.528+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:05.540 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:05.541 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:05.550 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:09:05.629 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:05.628+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:05.671 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:05.672 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:05.681 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:09:05.730 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:05.730+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:05.767 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:09:05.804 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:05.804 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:05.816 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:09:05.831 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:05.831+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:05.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.878+0000 7f5838406640 1 Processor -- start 2024-09-16T00:09:05.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.878+0000 7f5838406640 1 -- start start 2024-09-16T00:09:05.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.879+0000 7f5838406640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f58300c6790 0x7f5830158170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.879+0000 7f5838406640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f58300c8830 0x7f5830072770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.880+0000 7f5838406640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58300c7990 con 0x7f58300c7030 2024-09-16T00:09:05.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.880+0000 7f5838406640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5830073a20 con 0x7f58300c6790 2024-09-16T00:09:05.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.880+0000 7f5838406640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f58301586b0 con 0x7f58300c8830 2024-09-16T00:09:05.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.880+0000 7f58361a2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f58300c6790 0x7f5830158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.880+0000 7f58361a2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f58300c6790 0x7f5830158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42056/0 (socket says 172.21.15.139:42056) 2024-09-16T00:09:05.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.880+0000 7f58361a2640 1 -- 172.21.15.139:0/3890611951 learned_addr learned my addr 172.21.15.139:0/3890611951 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:05.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.880+0000 7f58369a3640 1 --2- 172.21.15.139:0/3890611951 >> v2:172.21.15.164:3300/0 conn(0x7f58300c8830 0x7f5830072770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.880+0000 7f58351a0640 1 -- 172.21.15.139:0/3890611951 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1485827735 0 0) 0x7f58300c7990 con 0x7f58300c7030 2024-09-16T00:09:05.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.880+0000 7f58361a2640 1 -- 172.21.15.139:0/3890611951 >> v2:172.21.15.164:3300/0 conn(0x7f58300c8830 msgr2=0x7f5830072770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:05.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.880+0000 7f58361a2640 1 --2- 172.21.15.139:0/3890611951 >> v2:172.21.15.164:3300/0 conn(0x7f58300c8830 0x7f5830072770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:05.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.880+0000 7f58361a2640 1 -- 172.21.15.139:0/3890611951 >> v1:172.21.15.164:6789/0 conn(0x7f58300c7030 legacy=0x7f58300c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:05.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.880+0000 7f58361a2640 1 -- 172.21.15.139:0/3890611951 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5830073640 con 0x7f58300c6790 2024-09-16T00:09:05.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.881+0000 7f58361a2640 1 --2- 172.21.15.139:0/3890611951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58300c6790 0x7f5830158170 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f58300caea0 tx=0x7f582c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a7fd26671b4a5f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:05.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.881+0000 7f58351a0640 1 -- 172.21.15.139:0/3890611951 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c02f710 con 0x7f58300c6790 2024-09-16T00:09:05.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.882+0000 7f58351a0640 1 -- 172.21.15.139:0/3890611951 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f582c02f8b0 con 0x7f58300c6790 2024-09-16T00:09:05.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.882+0000 7f58351a0640 1 -- 172.21.15.139:0/3890611951 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c02fc80 con 0x7f58300c6790 2024-09-16T00:09:05.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.882+0000 7f5838406640 1 -- 172.21.15.139:0/3890611951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58300c6790 msgr2=0x7f5830158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:05.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.882+0000 7f5838406640 1 --2- 172.21.15.139:0/3890611951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58300c6790 0x7f5830158170 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f58300caea0 tx=0x7f582c02a710 comp rx=0 tx=0).stop 2024-09-16T00:09:05.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.882+0000 7f5838406640 1 -- 172.21.15.139:0/3890611951 shutdown_connections 2024-09-16T00:09:05.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.882+0000 7f5838406640 1 --2- 172.21.15.139:0/3890611951 >> v2:172.21.15.164:3300/0 conn(0x7f58300c8830 0x7f5830072770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:05.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.882+0000 7f5838406640 1 --2- 172.21.15.139:0/3890611951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58300c6790 0x7f5830158170 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:05.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.882+0000 7f5838406640 1 -- 172.21.15.139:0/3890611951 >> 172.21.15.139:0/3890611951 conn(0x7f58300c9af0 msgr2=0x7f58300c9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:05.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.883+0000 7f5838406640 1 -- 172.21.15.139:0/3890611951 shutdown_connections 2024-09-16T00:09:05.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.883+0000 7f5838406640 1 -- 172.21.15.139:0/3890611951 wait complete. 2024-09-16T00:09:05.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.884+0000 7f5838406640 1 Processor -- start 2024-09-16T00:09:05.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.884+0000 7f5838406640 1 -- start start 2024-09-16T00:09:05.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.885+0000 7f5838406640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58300c6790 0x7f58300c3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.885+0000 7f5838406640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58300c7030 0x7f5830156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.885+0000 7f5838406640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58300c8830 0x7f5830149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.885+0000 7f5838406640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f58300c6b70 con 0x7f58300c7030 2024-09-16T00:09:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.885+0000 7f5838406640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58300c7c00 con 0x7f58300c8830 2024-09-16T00:09:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.885+0000 7f5838406640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5830073f50 con 0x7f58300c6790 2024-09-16T00:09:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.885+0000 7f58361a2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58300c6790 0x7f58300c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.885+0000 7f58369a3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58300c8830 0x7f5830149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.885+0000 7f58359a1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58300c7030 0x7f5830156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.885+0000 7f58369a3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58300c8830 0x7f5830149820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56916/0 (socket says 172.21.15.139:56916) 2024-09-16T00:09:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.885+0000 7f58361a2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58300c6790 0x7f58300c3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:56902/0 (socket says 172.21.15.139:56902) 2024-09-16T00:09:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.885+0000 7f58369a3640 1 -- 172.21.15.139:0/1463545917 learned_addr learned my addr 172.21.15.139:0/1463545917 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:05.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.886+0000 7f58369a3640 1 -- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58300c6790 msgr2=0x7f58300c3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:05.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.886+0000 7f58369a3640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58300c6790 0x7f58300c3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:05.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.886+0000 7f58369a3640 1 -- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58300c7030 msgr2=0x7f5830156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:05.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.886+0000 7f58369a3640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58300c7030 0x7f5830156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:05.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.886+0000 7f58369a3640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58300c3640 con 0x7f58300c8830 2024-09-16T00:09:05.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.886+0000 7f58359a1640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58300c7030 0x7f5830156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:05.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.886+0000 7f58369a3640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58300c8830 0x7f5830149820 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f58280123f0 tx=0x7f58280128b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:05.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.886+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582800d4e0 con 0x7f58300c8830 2024-09-16T00:09:05.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.886+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f582801d070 con 0x7f58300c8830 2024-09-16T00:09:05.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.886+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58301543d0 con 0x7f58300c8830 2024-09-16T00:09:05.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.886+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5830154bc0 con 0x7f58300c8830 2024-09-16T00:09:05.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.886+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582800b980 con 0x7f58300c8830 2024-09-16T00:09:05.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.887+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f5830158170 con 0x7f58300c8830 2024-09-16T00:09:05.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.888+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5828005d10 con 0x7f58300c8830 2024-09-16T00:09:05.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.888+0000 7f5826ffd640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f581406c2e0 0x7f581406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.888+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5915+0+0 (secure 0 0 0) 0x7f582808fd80 con 0x7f58300c8830 2024-09-16T00:09:05.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.888+0000 7f58361a2640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f581406c2e0 0x7f581406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.888+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v63) ==== 54+0+16 (secure 0 0 0) 0x7f5828014040 con 0x7f58300c8830 2024-09-16T00:09:05.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.889+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5830152cf0 0x7f58300cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.889+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f58300cc6d0 con 0x7f5830152cf0 2024-09-16T00:09:05.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.889+0000 7f58359a1640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5830152cf0 0x7f58300cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.889+0000 7f58361a2640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f581406c2e0 0x7f581406e7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f582c034000 tx=0x7f582c02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:05.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.889+0000 7f58359a1640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5830152cf0 0x7f58300cc190 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:05.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.890+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f58300cc6d0 con 0x7f5830152cf0 2024-09-16T00:09:05.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.908+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5830157bc0 con 0x7f5830152cf0 2024-09-16T00:09:05.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.910+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5830157bc0 con 0x7f5830152cf0 2024-09-16T00:09:05.911 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:05.911 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:05.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.910+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5830153a70 0x7f58301ef590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.910+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f5830154f00 con 0x7f5830153a70 2024-09-16T00:09:05.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.910+0000 7f58361a2640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5830153a70 0x7f58301ef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.911+0000 7f58361a2640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5830153a70 0x7f58301ef590 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:05.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.912+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5830154f00 con 0x7f5830153a70 2024-09-16T00:09:05.932 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:05.932+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:05.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.941+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f58301efd20 con 0x7f5830153a70 2024-09-16T00:09:05.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.944+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f58301efd20 con 0x7f5830153a70 2024-09-16T00:09:05.945 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:05.946 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:05.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.944+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5830148780 0x7f5830062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.944+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f5830148bc0 con 0x7f5830148780 2024-09-16T00:09:05.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.944+0000 7f58359a1640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5830148780 0x7f5830062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.944+0000 7f58359a1640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5830148780 0x7f5830062630 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:05.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.945+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5830148bc0 con 0x7f5830148780 2024-09-16T00:09:05.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.959+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5830062e40 con 0x7f5830148780 2024-09-16T00:09:05.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.961+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5830062e40 con 0x7f5830148780 2024-09-16T00:09:05.962 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:05.962 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:05.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.961+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f583006d0d0 0x7f583006d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.961+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f583006da30 con 0x7f583006d0d0 2024-09-16T00:09:05.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.961+0000 7f58369a3640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f583006d0d0 0x7f583006d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.961+0000 7f58369a3640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f583006d0d0 0x7f583006d4f0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:05.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.962+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f583006da30 con 0x7f583006d0d0 2024-09-16T00:09:05.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.975+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f58301efed0 con 0x7f583006d0d0 2024-09-16T00:09:05.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.977+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f582805ddd0 con 0x7f583006d0d0 2024-09-16T00:09:05.978 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:05.979 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:05.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.977+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f583002ccc0 0x7f583002d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.977+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f583002d640 con 0x7f583002ccc0 2024-09-16T00:09:05.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.977+0000 7f58361a2640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f583002ccc0 0x7f583002d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.978+0000 7f58361a2640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f583002ccc0 0x7f583002d100 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:05.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.978+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f583002d640 con 0x7f583002ccc0 2024-09-16T00:09:05.981 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:05.981 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:05.981 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:05.981 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:05.981 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:05.990 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:09:05.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.992+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f583002d640 con 0x7f583002ccc0 2024-09-16T00:09:05.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.994+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f583002d640 con 0x7f583002ccc0 2024-09-16T00:09:05.995 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:05.995 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:05.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.994+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5830200610 0x7f5830202a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:05.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.994+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f58302057d0 con 0x7f5830200610 2024-09-16T00:09:05.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.994+0000 7f58359a1640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5830200610 0x7f5830202a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:05.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.995+0000 7f58359a1640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5830200610 0x7f5830202a00 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:05.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:05.995+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f58302057d0 con 0x7f5830200610 2024-09-16T00:09:06.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.009+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5830202f40 con 0x7f5830200610 2024-09-16T00:09:06.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.010+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5830202f40 con 0x7f5830200610 2024-09-16T00:09:06.011 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.011+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5830203660 0x7f5830205a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.011+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f5830208820 con 0x7f5830203660 2024-09-16T00:09:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.011+0000 7f58369a3640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5830203660 0x7f5830205a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.011+0000 7f58369a3640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5830203660 0x7f5830205a50 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:06.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.012+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5830208820 con 0x7f5830203660 2024-09-16T00:09:06.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.026+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5830205f90 con 0x7f5830203660 2024-09-16T00:09:06.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.027+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5830205f90 con 0x7f5830203660 2024-09-16T00:09:06.028 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:06.029 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:09:06.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.028+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f58302066b0 0x7f5830208aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:06.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.028+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f583020b9f0 con 0x7f58302066b0 2024-09-16T00:09:06.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.028+0000 7f58361a2640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f58302066b0 0x7f5830208aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:06.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.029+0000 7f58361a2640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f58302066b0 0x7f5830208aa0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:06.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.029+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f583020b9f0 con 0x7f58302066b0 2024-09-16T00:09:06.033 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:06.033+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:06.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.033+0000 7fa9fbfff640 1 -- 172.21.15.139:0/2673534691 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v64) ==== 87+0+0 (secure 0 0 0) 0x7faa08079d70 con 0x7faa140ccc90 2024-09-16T00:09:06.034 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is set 2024-09-16T00:09:06.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.035+0000 7faa1a25e640 1 -- 172.21.15.139:0/2673534691 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faa0006c2b0 msgr2=0x7faa0006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:06.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.035+0000 7faa1a25e640 1 --2- 172.21.15.139:0/2673534691 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faa0006c2b0 0x7faa0006e770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faa04000f60 tx=0x7faa04007e40 comp rx=0 tx=0).stop 2024-09-16T00:09:06.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.035+0000 7faa1a25e640 1 -- 172.21.15.139:0/2673534691 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa140ccc90 msgr2=0x7faa140ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:06.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.035+0000 7faa1a25e640 1 --2- 172.21.15.139:0/2673534691 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa140ccc90 0x7faa140ce160 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faa14074130 tx=0x7faa080046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:06.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.035+0000 7faa1a25e640 1 -- 172.21.15.139:0/2673534691 shutdown_connections 2024-09-16T00:09:06.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.035+0000 7faa1a25e640 1 --2- 172.21.15.139:0/2673534691 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faa0006c2b0 0x7faa0006e770 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:06.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.035+0000 7faa1a25e640 1 --2- 172.21.15.139:0/2673534691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa14154c20 0x7faa140b5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:06.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.035+0000 7faa1a25e640 1 --2- 172.21.15.139:0/2673534691 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faa140ccc90 0x7faa140ce160 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:06.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.035+0000 7faa1a25e640 1 --2- 172.21.15.139:0/2673534691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faa14068060 0x7faa141f0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:06.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.035+0000 7faa1a25e640 1 -- 172.21.15.139:0/2673534691 >> 172.21.15.139:0/2673534691 conn(0x7faa1406ca40 msgr2=0x7faa14153c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:06.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.035+0000 7faa1a25e640 1 -- 172.21.15.139:0/2673534691 shutdown_connections 2024-09-16T00:09:06.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.035+0000 7faa1a25e640 1 -- 172.21.15.139:0/2673534691 wait complete. 2024-09-16T00:09:06.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.044+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5830000950 con 0x7f58302066b0 2024-09-16T00:09:06.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.046+0000 7f5826ffd640 1 -- 172.21.15.139:0/1463545917 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5830000950 con 0x7f58302066b0 2024-09-16T00:09:06.047 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:06.049 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:06.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5830152cf0 msgr2=0x7f58300cc190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:06.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5830152cf0 0x7f58300cc190 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:06.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5830153a70 msgr2=0x7f58301ef590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:06.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5830153a70 0x7f58301ef590 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:06.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5830148780 msgr2=0x7f5830062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:06.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5830148780 0x7f5830062630 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:06.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f583006d0d0 msgr2=0x7f583006d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:06.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f583006d0d0 0x7f583006d4f0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:06.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f583002ccc0 msgr2=0x7f583002d100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:06.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f58369a3640 1 -- 172.21.15.139:0/1463545917 reap_dead start 2024-09-16T00:09:06.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f583002ccc0 0x7f583002d100 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:06.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f58369a3640 1 -- 172.21.15.139:0/1463545917 reap_dead start 2024-09-16T00:09:06.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5830200610 msgr2=0x7f5830202a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:06.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5830200610 0x7f5830202a00 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:06.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5830203660 msgr2=0x7f5830205a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:06.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f5830203660 0x7f5830205a50 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:06.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f58302066b0 msgr2=0x7f5830208aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:06.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f58302066b0 0x7f5830208aa0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:06.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f581406c2e0 msgr2=0x7f581406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:06.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f581406c2e0 0x7f581406e7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f582c034000 tx=0x7f582c02f000 comp rx=0 tx=0).stop 2024-09-16T00:09:06.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58300c8830 msgr2=0x7f5830149820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:06.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 --2- 172.21.15.139:0/1463545917 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58300c8830 0x7f5830149820 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f58280123f0 tx=0x7f58280128b0 comp rx=0 tx=0).stop 2024-09-16T00:09:06.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f58369a3640 1 -- 172.21.15.139:0/1463545917 reap_dead start 2024-09-16T00:09:06.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f58369a3640 1 -- 172.21.15.139:0/1463545917 reap_dead start 2024-09-16T00:09:06.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 shutdown_connections 2024-09-16T00:09:06.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 >> 172.21.15.139:0/1463545917 conn(0x7f58300c9af0 msgr2=0x7f583006c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:06.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 shutdown_connections 2024-09-16T00:09:06.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:06.047+0000 7f5838406640 1 -- 172.21.15.139:0/1463545917 wait complete. 2024-09-16T00:09:06.108 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:06.108 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:06.108 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:06.108 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:06.108 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.108 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'66)", 2024-09-16T00:09:06.108 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:09:06.108 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 101.939899694, 2024-09-16T00:09:06.108 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032925509999999999, 2024-09-16T00:09:06.108 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.109 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.109 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.109 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.109 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.109 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.109 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:09:06.109 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.110 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.110 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:09:06.110 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.110 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.110 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166177+0000", 2024-09-16T00:09:06.110 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:09:06.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.111 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.111 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166216+0000", 2024-09-16T00:09:06.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9008000000000003e-05 2024-09-16T00:09:06.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.111 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.111 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.111 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166226+0000", 2024-09-16T00:09:06.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0081999999999999e-05 2024-09-16T00:09:06.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.111 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.111 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.111 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166230+0000", 2024-09-16T00:09:06.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5079999999999999e-06 2024-09-16T00:09:06.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.112 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166259+0000", 2024-09-16T00:09:06.112 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8928000000000002e-05 2024-09-16T00:09:06.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.112 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166271+0000", 2024-09-16T00:09:06.112 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1531e-05 2024-09-16T00:09:06.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.113 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169449+0000", 2024-09-16T00:09:06.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003178143 2024-09-16T00:09:06.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.113 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169468+0000", 2024-09-16T00:09:06.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9700000000000001e-05 2024-09-16T00:09:06.113 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.113 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.114 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.114 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'66)", 2024-09-16T00:09:06.114 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:09:06.114 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 101.939438235, 2024-09-16T00:09:06.114 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003179374, 2024-09-16T00:09:06.114 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.114 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.115 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.115 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:09:06.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:09:06.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166638+0000", 2024-09-16T00:09:06.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2300000000000002e-07 2024-09-16T00:09:06.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166641+0000", 2024-09-16T00:09:06.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0079999999999998e-06 2024-09-16T00:09:06.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166646+0000", 2024-09-16T00:09:06.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4970000000000001e-06 2024-09-16T00:09:06.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166650+0000", 2024-09-16T00:09:06.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5729999999999999e-06 2024-09-16T00:09:06.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166682+0000", 2024-09-16T00:09:06.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2218999999999998e-05 2024-09-16T00:09:06.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166694+0000", 2024-09-16T00:09:06.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1902999999999999e-05 2024-09-16T00:09:06.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169784+0000", 2024-09-16T00:09:06.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003089545 2024-09-16T00:09:06.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.119 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.119 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169817+0000", 2024-09-16T00:09:06.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3206e-05 2024-09-16T00:09:06.119 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.119 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.119 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.119 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:117 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'66)", 2024-09-16T00:09:06.119 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:09:06.119 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 101.938994175, 2024-09-16T00:09:06.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00275326, 2024-09-16T00:09:06.120 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.120 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.120 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.120 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:09:06.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:09:06.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167082+0000", 2024-09-16T00:09:06.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:09:06.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167086+0000", 2024-09-16T00:09:06.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1559999999999997e-06 2024-09-16T00:09:06.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167096+0000", 2024-09-16T00:09:06.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8390000000000002e-06 2024-09-16T00:09:06.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167101+0000", 2024-09-16T00:09:06.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:09:06.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167130+0000", 2024-09-16T00:09:06.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9649e-05 2024-09-16T00:09:06.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167142+0000", 2024-09-16T00:09:06.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1435e-05 2024-09-16T00:09:06.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.123 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169818+0000", 2024-09-16T00:09:06.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002676056 2024-09-16T00:09:06.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.123 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169835+0000", 2024-09-16T00:09:06.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6980999999999999e-05 2024-09-16T00:09:06.123 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.124 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.124 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.124 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:130 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'88, mlcod=17'85)", 2024-09-16T00:09:06.124 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:06.124 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 101.49619484599999, 2024-09-16T00:09:06.124 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002880456, 2024-09-16T00:09:06.124 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.124 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.124 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.124 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.125 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:06.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.125 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:06.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:06.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5699999999999998e-07 2024-09-16T00:09:06.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.127 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.127 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609885+0000", 2024-09-16T00:09:06.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.608e-06 2024-09-16T00:09:06.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.127 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.127 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609891+0000", 2024-09-16T00:09:06.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:09:06.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609895+0000", 2024-09-16T00:09:06.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.54e-06 2024-09-16T00:09:06.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609931+0000", 2024-09-16T00:09:06.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5893999999999999e-05 2024-09-16T00:09:06.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.129 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.129 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609945+0000", 2024-09-16T00:09:06.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4259e-05 2024-09-16T00:09:06.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.129 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.129 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612721+0000", 2024-09-16T00:09:06.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027757849999999998 2024-09-16T00:09:06.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612761+0000", 2024-09-16T00:09:06.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0638e-05 2024-09-16T00:09:06.130 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.130 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.130 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.131 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-16T00:09:06.131 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:09:06.131 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 101.495810873, 2024-09-16T00:09:06.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028016529999999999, 2024-09-16T00:09:06.131 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.131 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.131 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.131 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:09:06.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:09:06.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:09:06.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.63e-07 2024-09-16T00:09:06.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610344+0000", 2024-09-16T00:09:06.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8996000000000001e-05 2024-09-16T00:09:06.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.133 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610352+0000", 2024-09-16T00:09:06.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:09:06.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.133 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610356+0000", 2024-09-16T00:09:06.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0790000000000002e-06 2024-09-16T00:09:06.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610388+0000", 2024-09-16T00:09:06.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1541000000000002e-05 2024-09-16T00:09:06.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610400+0000", 2024-09-16T00:09:06.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2075e-05 2024-09-16T00:09:06.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.135 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.135 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613045+0000", 2024-09-16T00:09:06.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002645213 2024-09-16T00:09:06.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.135 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.135 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613066+0000", 2024-09-16T00:09:06.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1353e-05 2024-09-16T00:09:06.135 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.135 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.135 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.135 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-16T00:09:06.136 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:09:06.136 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 101.492551373, 2024-09-16T00:09:06.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027254850000000001, 2024-09-16T00:09:06.136 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.136 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.136 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.136 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.136 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.136 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:09:06.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.136 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613524+0000", 2024-09-16T00:09:06.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613525+0000", 2024-09-16T00:09:06.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2800000000000002e-07 2024-09-16T00:09:06.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613528+0000", 2024-09-16T00:09:06.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9730000000000002e-06 2024-09-16T00:09:06.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613534+0000", 2024-09-16T00:09:06.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4339999999999997e-06 2024-09-16T00:09:06.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613538+0000", 2024-09-16T00:09:06.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5149999999999998e-06 2024-09-16T00:09:06.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613566+0000", 2024-09-16T00:09:06.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8104000000000001e-05 2024-09-16T00:09:06.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613578+0000", 2024-09-16T00:09:06.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2404e-05 2024-09-16T00:09:06.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616217+0000", 2024-09-16T00:09:06.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026389830000000001 2024-09-16T00:09:06.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.140 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.140 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616250+0000", 2024-09-16T00:09:06.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2644000000000001e-05 2024-09-16T00:09:06.140 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.140 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.140 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.140 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.140 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:139 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'97, mlcod=18'87)", 2024-09-16T00:09:06.140 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:09:06.140 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 101.48973252499999, 2024-09-16T00:09:06.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030908139999999999, 2024-09-16T00:09:06.140 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.141 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.141 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.141 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:09:06.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:09:06.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616344+0000", 2024-09-16T00:09:06.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3000000000000001e-07 2024-09-16T00:09:06.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616347+0000", 2024-09-16T00:09:06.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7740000000000002e-06 2024-09-16T00:09:06.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616355+0000", 2024-09-16T00:09:06.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4699999999999996e-06 2024-09-16T00:09:06.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616358+0000", 2024-09-16T00:09:06.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5899999999999999e-06 2024-09-16T00:09:06.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616393+0000", 2024-09-16T00:09:06.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4529000000000001e-05 2024-09-16T00:09:06.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616404+0000", 2024-09-16T00:09:06.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0726e-05 2024-09-16T00:09:06.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619411+0000", 2024-09-16T00:09:06.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030076249999999999 2024-09-16T00:09:06.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619434+0000", 2024-09-16T00:09:06.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-05 2024-09-16T00:09:06.144 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.145 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.145 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.145 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:140 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'98, mlcod=18'89)", 2024-09-16T00:09:06.145 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:09:06.145 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 101.489091132, 2024-09-16T00:09:06.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027276150000000001, 2024-09-16T00:09:06.145 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.145 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.145 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.145 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.145 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:09:06.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616984+0000", 2024-09-16T00:09:06.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616985+0000", 2024-09-16T00:09:06.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4099999999999999e-07 2024-09-16T00:09:06.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.147 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.147 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617055+0000", 2024-09-16T00:09:06.147 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9923000000000001e-05 2024-09-16T00:09:06.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.147 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.147 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617063+0000", 2024-09-16T00:09:06.147 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:09:06.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.147 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.147 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617070+0000", 2024-09-16T00:09:06.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9090000000000003e-06 2024-09-16T00:09:06.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617102+0000", 2024-09-16T00:09:06.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2620999999999998e-05 2024-09-16T00:09:06.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.617114+0000", 2024-09-16T00:09:06.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1228e-05 2024-09-16T00:09:06.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.149 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619690+0000", 2024-09-16T00:09:06.149 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025763890000000001 2024-09-16T00:09:06.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.149 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619712+0000", 2024-09-16T00:09:06.149 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2171000000000002e-05 2024-09-16T00:09:06.149 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.149 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.149 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.149 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:06.150 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:09:06.150 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 87.490262032000004, 2024-09-16T00:09:06.150 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027355320000000002, 2024-09-16T00:09:06.150 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.150 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.150 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.150 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.150 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.150 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.150 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:09:06.150 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.150 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.151 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.151 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615814+0000", 2024-09-16T00:09:06.151 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.151 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.152 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.152 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615815+0000", 2024-09-16T00:09:06.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1769999999999999e-06 2024-09-16T00:09:06.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.153 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.153 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.153 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615825+0000", 2024-09-16T00:09:06.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9000000000000001e-06 2024-09-16T00:09:06.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.153 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.153 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.153 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615840+0000", 2024-09-16T00:09:06.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4854e-05 2024-09-16T00:09:06.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.153 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.154 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.154 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615848+0000", 2024-09-16T00:09:06.154 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0940000000000008e-06 2024-09-16T00:09:06.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.154 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.154 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.154 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615926+0000", 2024-09-16T00:09:06.154 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8176999999999994e-05 2024-09-16T00:09:06.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.154 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.154 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.154 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615945+0000", 2024-09-16T00:09:06.154 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9663000000000001e-05 2024-09-16T00:09:06.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618509+0000", 2024-09-16T00:09:06.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002563802 2024-09-16T00:09:06.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618549+0000", 2024-09-16T00:09:06.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9864999999999999e-05 2024-09-16T00:09:06.155 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.155 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.155 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.156 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:06.156 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:09:06.156 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 87.490172892000004, 2024-09-16T00:09:06.156 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061593860000000002, 2024-09-16T00:09:06.156 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.156 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.156 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.156 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.156 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.156 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.156 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:09:06.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:09:06.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615907+0000", 2024-09-16T00:09:06.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9389999999999999e-06 2024-09-16T00:09:06.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.158 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.158 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615920+0000", 2024-09-16T00:09:06.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2989999999999999e-05 2024-09-16T00:09:06.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.158 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.158 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615931+0000", 2024-09-16T00:09:06.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.164e-05 2024-09-16T00:09:06.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.158 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.158 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615939+0000", 2024-09-16T00:09:06.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8029999999999997e-06 2024-09-16T00:09:06.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.159 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.159 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.159 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616310+0000", 2024-09-16T00:09:06.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00037111400000000003 2024-09-16T00:09:06.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.159 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.159 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.159 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616406+0000", 2024-09-16T00:09:06.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5372999999999996e-05 2024-09-16T00:09:06.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.159 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.159 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:06.159 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616514+0000", 2024-09-16T00:09:06.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108564 2024-09-16T00:09:06.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.160 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.160 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618158+0000", 2024-09-16T00:09:06.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016442449999999999 2024-09-16T00:09:06.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.160 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.160 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621972+0000", 2024-09-16T00:09:06.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038132299999999999 2024-09-16T00:09:06.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.161 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.161 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622018+0000", 2024-09-16T00:09:06.161 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6239000000000002e-05 2024-09-16T00:09:06.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.161 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.161 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.161 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622062+0000", 2024-09-16T00:09:06.161 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4249000000000003e-05 2024-09-16T00:09:06.161 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.161 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.161 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.161 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.161 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:06.162 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:09:06.162 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 87.490119424, 2024-09-16T00:09:06.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006258914, 2024-09-16T00:09:06.162 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.162 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.162 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.162 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:09:06.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:09:06.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615957+0000", 2024-09-16T00:09:06.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1399999999999997e-07 2024-09-16T00:09:06.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.164 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.164 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615960+0000", 2024-09-16T00:09:06.164 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0860000000000002e-06 2024-09-16T00:09:06.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.164 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.164 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.164 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615967+0000", 2024-09-16T00:09:06.164 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9059999999999996e-06 2024-09-16T00:09:06.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.164 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.164 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.164 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615972+0000", 2024-09-16T00:09:06.164 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5679999999999999e-06 2024-09-16T00:09:06.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.165 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.165 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616717+0000", 2024-09-16T00:09:06.165 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074438 2024-09-16T00:09:06.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.165 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.165 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616781+0000", 2024-09-16T00:09:06.165 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4808999999999996e-05 2024-09-16T00:09:06.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.165 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:06.165 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616885+0000", 2024-09-16T00:09:06.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010404900000000001 2024-09-16T00:09:06.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:09:06.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012773870000000001 2024-09-16T00:09:06.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622116+0000", 2024-09-16T00:09:06.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039533470000000003 2024-09-16T00:09:06.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622173+0000", 2024-09-16T00:09:06.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6347000000000001e-05 2024-09-16T00:09:06.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622215+0000", 2024-09-16T00:09:06.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2521000000000002e-05 2024-09-16T00:09:06.167 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.167 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.168 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.168 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.168 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.168 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:06.168 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:06.168 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 87.490095686999993, 2024-09-16T00:09:06.168 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041725529999999999, 2024-09-16T00:09:06.168 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.168 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.168 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.168 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.168 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.168 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:06.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.169 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:06.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.169 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:06.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8799999999999998e-07 2024-09-16T00:09:06.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.170 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.170 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615990+0000", 2024-09-16T00:09:06.170 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2150000000000007e-06 2024-09-16T00:09:06.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.170 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.170 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.170 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:09:06.170 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7159999999999997e-06 2024-09-16T00:09:06.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.170 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.170 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.170 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616001+0000", 2024-09-16T00:09:06.170 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9339999999999997e-06 2024-09-16T00:09:06.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616293+0000", 2024-09-16T00:09:06.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029210899999999998 2024-09-16T00:09:06.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616401+0000", 2024-09-16T00:09:06.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108087 2024-09-16T00:09:06.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:06.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616561+0000", 2024-09-16T00:09:06.172 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015940600000000001 2024-09-16T00:09:06.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.172 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.172 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618146+0000", 2024-09-16T00:09:06.172 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015857040000000001 2024-09-16T00:09:06.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.172 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.172 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620091+0000", 2024-09-16T00:09:06.172 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001944837 2024-09-16T00:09:06.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.173 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.173 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.173 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620126+0000", 2024-09-16T00:09:06.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4242999999999997e-05 2024-09-16T00:09:06.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.173 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.173 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.173 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620152+0000", 2024-09-16T00:09:06.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6914e-05 2024-09-16T00:09:06.173 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.173 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.173 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.174 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.174 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:06.174 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:06.174 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 87.490057461999996, 2024-09-16T00:09:06.174 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00630628, 2024-09-16T00:09:06.174 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.174 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.174 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.174 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.174 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.174 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.174 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:06.174 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:06.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:09:06.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5999999999999999e-07 2024-09-16T00:09:06.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616022+0000", 2024-09-16T00:09:06.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0149999999999999e-06 2024-09-16T00:09:06.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616031+0000", 2024-09-16T00:09:06.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9770000000000006e-06 2024-09-16T00:09:06.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616036+0000", 2024-09-16T00:09:06.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2639999999999999e-06 2024-09-16T00:09:06.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617057+0000", 2024-09-16T00:09:06.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010212999999999999 2024-09-16T00:09:06.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617146+0000", 2024-09-16T00:09:06.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9234000000000007e-05 2024-09-16T00:09:06.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:06.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617232+0000", 2024-09-16T00:09:06.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5218999999999998e-05 2024-09-16T00:09:06.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619107+0000", 2024-09-16T00:09:06.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001875016 2024-09-16T00:09:06.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622259+0000", 2024-09-16T00:09:06.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031527930000000001 2024-09-16T00:09:06.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622290+0000", 2024-09-16T00:09:06.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1022999999999998e-05 2024-09-16T00:09:06.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.179 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622324+0000", 2024-09-16T00:09:06.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3979000000000001e-05 2024-09-16T00:09:06.179 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.179 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.179 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.179 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:06.179 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:09:06.180 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 87.490032159999998, 2024-09-16T00:09:06.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042412739999999997, 2024-09-16T00:09:06.180 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.180 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.180 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.180 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.180 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:09:06.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:09:06.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616044+0000", 2024-09-16T00:09:06.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:09:06.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616052+0000", 2024-09-16T00:09:06.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.439e-06 2024-09-16T00:09:06.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616059+0000", 2024-09-16T00:09:06.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1389999999999998e-06 2024-09-16T00:09:06.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616065+0000", 2024-09-16T00:09:06.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:09:06.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616823+0000", 2024-09-16T00:09:06.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075781199999999996 2024-09-16T00:09:06.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616918+0000", 2024-09-16T00:09:06.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5174999999999996e-05 2024-09-16T00:09:06.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:06.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617019+0000", 2024-09-16T00:09:06.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101161 2024-09-16T00:09:06.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618503+0000", 2024-09-16T00:09:06.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014836750000000001 2024-09-16T00:09:06.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620201+0000", 2024-09-16T00:09:06.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00169805 2024-09-16T00:09:06.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620255+0000", 2024-09-16T00:09:06.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4499000000000001e-05 2024-09-16T00:09:06.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620285+0000", 2024-09-16T00:09:06.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9349e-05 2024-09-16T00:09:06.185 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.185 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.185 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.185 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:06.185 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:09:06.185 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 87.490003169000005, 2024-09-16T00:09:06.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063682799999999996, 2024-09-16T00:09:06.185 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.185 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.186 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:09:06.186 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.186 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.186 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:09:06.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.186 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.186 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.186 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:09:06.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.186 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.186 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:09:06.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9900000000000001e-07 2024-09-16T00:09:06.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616079+0000", 2024-09-16T00:09:06.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9009999999999999e-06 2024-09-16T00:09:06.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616086+0000", 2024-09-16T00:09:06.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.776e-06 2024-09-16T00:09:06.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616091+0000", 2024-09-16T00:09:06.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5600000000000001e-06 2024-09-16T00:09:06.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618339+0000", 2024-09-16T00:09:06.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022476179999999998 2024-09-16T00:09:06.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:09:06.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9025999999999997e-05 2024-09-16T00:09:06.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622441+0000", 2024-09-16T00:09:06.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040029999999999996 2024-09-16T00:09:06.189 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.189 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.189 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.189 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:221 3.1b 3:dad7a251:::100000002d7.00000000:head [write 0~30412] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:06.189 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:09:06.189 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.356794979, 2024-09-16T00:09:06.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034396769999999999, 2024-09-16T00:09:06.190 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.190 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.190 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.190 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:06.190 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:06.190 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:06.190 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 221 2024-09-16T00:09:06.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.190 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:09:06.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:09:06.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749282+0000", 2024-09-16T00:09:06.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3769999999999999e-06 2024-09-16T00:09:06.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749300+0000", 2024-09-16T00:09:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8204999999999999e-05 2024-09-16T00:09:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749311+0000", 2024-09-16T00:09:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0599e-05 2024-09-16T00:09:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749318+0000", 2024-09-16T00:09:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9789999999999996e-06 2024-09-16T00:09:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749367+0000", 2024-09-16T00:09:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9382999999999998e-05 2024-09-16T00:09:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749450+0000", 2024-09-16T00:09:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-05 2024-09-16T00:09:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749526+0000", 2024-09-16T00:09:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5564999999999999e-05 2024-09-16T00:09:06.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.750601+0000", 2024-09-16T00:09:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010751199999999999 2024-09-16T00:09:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752665+0000", 2024-09-16T00:09:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00206442 2024-09-16T00:09:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752696+0000", 2024-09-16T00:09:06.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0476000000000001e-05 2024-09-16T00:09:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752720+0000", 2024-09-16T00:09:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4482999999999999e-05 2024-09-16T00:09:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:244 3.1b 3:da01a9ef:::100000002f1.00000000:head [write 0~9324] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.800988+0000", 2024-09-16T00:09:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.3050872919999996, 2024-09-16T00:09:06.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026477689999999999, 2024-09-16T00:09:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 244 2024-09-16T00:09:06.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.800988+0000", 2024-09-16T00:09:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.800988+0000", 2024-09-16T00:09:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.800989+0000", 2024-09-16T00:09:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.33e-07 2024-09-16T00:09:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801004+0000", 2024-09-16T00:09:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4531e-05 2024-09-16T00:09:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801015+0000", 2024-09-16T00:09:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0988000000000001e-05 2024-09-16T00:09:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801020+0000", 2024-09-16T00:09:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2820000000000004e-06 2024-09-16T00:09:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801072+0000", 2024-09-16T00:09:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1830999999999999e-05 2024-09-16T00:09:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801142+0000", 2024-09-16T00:09:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0182e-05 2024-09-16T00:09:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801218+0000", 2024-09-16T00:09:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6093999999999993e-05 2024-09-16T00:09:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.801707+0000", 2024-09-16T00:09:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000488837 2024-09-16T00:09:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803548+0000", 2024-09-16T00:09:06.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018406519999999999 2024-09-16T00:09:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803600+0000", 2024-09-16T00:09:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2429999999999998e-05 2024-09-16T00:09:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803636+0000", 2024-09-16T00:09:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6009e-05 2024-09-16T00:09:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.201 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:323 3.3 e58/20 3:c319d237:::10000000354.00000000:head v 58'3, mlcod=57'1)", 2024-09-16T00:09:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:09:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.0734152080000001, 2024-09-16T00:09:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034011000000000002, 2024-09-16T00:09:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:09:06.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:09:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032661+0000", 2024-09-16T00:09:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7000000000000003e-07 2024-09-16T00:09:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032670+0000", 2024-09-16T00:09:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.602e-06 2024-09-16T00:09:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032682+0000", 2024-09-16T00:09:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1754e-05 2024-09-16T00:09:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032688+0000", 2024-09-16T00:09:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8140000000000003e-06 2024-09-16T00:09:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032740+0000", 2024-09-16T00:09:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1981000000000003e-05 2024-09-16T00:09:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032765+0000", 2024-09-16T00:09:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4097e-05 2024-09-16T00:09:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.036023+0000", 2024-09-16T00:09:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032585240000000001 2024-09-16T00:09:06.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.036061+0000", 2024-09-16T00:09:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8457999999999997e-05 2024-09-16T00:09:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:664 3.39 e59/20 3:9c9d3e51:::100000004f6.00000000:head v 59'7, mlcod=58'5)", 2024-09-16T00:09:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 7.8857460489999998, 2024-09-16T00:09:06.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035253329999999999, 2024-09-16T00:09:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2399999999999996e-07 2024-09-16T00:09:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220338+0000", 2024-09-16T00:09:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0120000000000005e-06 2024-09-16T00:09:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220353+0000", 2024-09-16T00:09:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4718999999999999e-05 2024-09-16T00:09:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220361+0000", 2024-09-16T00:09:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6559999999999992e-06 2024-09-16T00:09:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220422+0000", 2024-09-16T00:09:06.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.071e-05 2024-09-16T00:09:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220446+0000", 2024-09-16T00:09:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4556e-05 2024-09-16T00:09:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.223813+0000", 2024-09-16T00:09:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033666439999999998 2024-09-16T00:09:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.223855+0000", 2024-09-16T00:09:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2111999999999999e-05 2024-09-16T00:09:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:681 3.17 3:ea55746f:::1000000050b.00000000:head [write 0~1071] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 7.8410030840000005, 2024-09-16T00:09:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036588620000000001, 2024-09-16T00:09:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.211 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 681 2024-09-16T00:09:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.212 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7600000000000006e-07 2024-09-16T00:09:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265080+0000", 2024-09-16T00:09:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8759999999999999e-06 2024-09-16T00:09:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265094+0000", 2024-09-16T00:09:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.411e-05 2024-09-16T00:09:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265102+0000", 2024-09-16T00:09:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9640000000000003e-06 2024-09-16T00:09:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265156+0000", 2024-09-16T00:09:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3375000000000003e-05 2024-09-16T00:09:06.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265229+0000", 2024-09-16T00:09:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3147999999999998e-05 2024-09-16T00:09:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265304+0000", 2024-09-16T00:09:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4568999999999997e-05 2024-09-16T00:09:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265747+0000", 2024-09-16T00:09:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00044363099999999999 2024-09-16T00:09:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.268635+0000", 2024-09-16T00:09:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028873459999999998 2024-09-16T00:09:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.268688+0000", 2024-09-16T00:09:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3155999999999997e-05 2024-09-16T00:09:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.268731+0000", 2024-09-16T00:09:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3711000000000002e-05 2024-09-16T00:09:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.217 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.217 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:06.219 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:09:06.219 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:06.218+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:06.319 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:06.319+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:06.378 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:06.378 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:06.378 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:06.378 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:06.388 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:09:06.430 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:06.429+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:06.522 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:06.522+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:06.548 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:06.548 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:06.548 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:06.549 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:06.549 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:06.562 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:09:06.646 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:06.645+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:06.723 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:06.723 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:06.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:06.723 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:06.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.723 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:06.723 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:09:06.724 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.105695604000005, 2024-09-16T00:09:06.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037514929999999998, 2024-09-16T00:09:06.724 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.724 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.724 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.724 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:09:06.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:09:06.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615364+0000", 2024-09-16T00:09:06.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8430000000000003e-06 2024-09-16T00:09:06.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615371+0000", 2024-09-16T00:09:06.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.148e-06 2024-09-16T00:09:06.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615380+0000", 2024-09-16T00:09:06.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5889999999999992e-06 2024-09-16T00:09:06.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615389+0000", 2024-09-16T00:09:06.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1620000000000008e-06 2024-09-16T00:09:06.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615525+0000", 2024-09-16T00:09:06.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000136083 2024-09-16T00:09:06.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615662+0000", 2024-09-16T00:09:06.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013696100000000001 2024-09-16T00:09:06.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:06.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615799+0000", 2024-09-16T00:09:06.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013781399999999999 2024-09-16T00:09:06.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:09:06.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002363502 2024-09-16T00:09:06.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619018+0000", 2024-09-16T00:09:06.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085522199999999999 2024-09-16T00:09:06.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619068+0000", 2024-09-16T00:09:06.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9545000000000001e-05 2024-09-16T00:09:06.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619111+0000", 2024-09-16T00:09:06.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3624000000000001e-05 2024-09-16T00:09:06.730 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.730 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.730 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.731 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:06.731 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:09:06.731 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.105643533000006, 2024-09-16T00:09:06.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043656110000000001, 2024-09-16T00:09:06.732 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.732 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.732 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.732 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.732 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:09:06.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:09:06.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:09:06.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2199999999999999e-07 2024-09-16T00:09:06.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615416+0000", 2024-09-16T00:09:06.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5590000000000001e-06 2024-09-16T00:09:06.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615423+0000", 2024-09-16T00:09:06.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6250000000000001e-06 2024-09-16T00:09:06.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615426+0000", 2024-09-16T00:09:06.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2540000000000001e-06 2024-09-16T00:09:06.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616075+0000", 2024-09-16T00:09:06.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064968800000000004 2024-09-16T00:09:06.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616142+0000", 2024-09-16T00:09:06.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6203000000000003e-05 2024-09-16T00:09:06.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:06.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616247+0000", 2024-09-16T00:09:06.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010572499999999999 2024-09-16T00:09:06.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618913+0000", 2024-09-16T00:09:06.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026654339999999999 2024-09-16T00:09:06.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619712+0000", 2024-09-16T00:09:06.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079885199999999996 2024-09-16T00:09:06.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619744+0000", 2024-09-16T00:09:06.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2790000000000003e-05 2024-09-16T00:09:06.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619778+0000", 2024-09-16T00:09:06.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3059000000000003e-05 2024-09-16T00:09:06.738 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.738 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.738 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.739 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:06.739 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:09:06.739 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.105607453999994, 2024-09-16T00:09:06.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087673219999999993, 2024-09-16T00:09:06.739 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.739 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.739 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.739 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:09:06.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:09:06.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:09:06.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:09:06.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:09:06.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5392000000000001e-05 2024-09-16T00:09:06.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615470+0000", 2024-09-16T00:09:06.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1999999999999999e-06 2024-09-16T00:09:06.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615474+0000", 2024-09-16T00:09:06.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7359999999999999e-06 2024-09-16T00:09:06.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617614+0000", 2024-09-16T00:09:06.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021398189999999998 2024-09-16T00:09:06.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617724+0000", 2024-09-16T00:09:06.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110808 2024-09-16T00:09:06.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:06.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617856+0000", 2024-09-16T00:09:06.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131676 2024-09-16T00:09:06.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:09:06.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029629280000000001 2024-09-16T00:09:06.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624158+0000", 2024-09-16T00:09:06.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033386549999999998 2024-09-16T00:09:06.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624189+0000", 2024-09-16T00:09:06.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1508999999999999e-05 2024-09-16T00:09:06.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624215+0000", 2024-09-16T00:09:06.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6228999999999999e-05 2024-09-16T00:09:06.745 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.745 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.745 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.745 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:06.745 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:09:06.745 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.105573438999997, 2024-09-16T00:09:06.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053285279999999999, 2024-09-16T00:09:06.745 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.745 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.746 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.746 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:09:06.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:09:06.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:09:06.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5600000000000002e-07 2024-09-16T00:09:06.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615484+0000", 2024-09-16T00:09:06.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.688e-06 2024-09-16T00:09:06.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615488+0000", 2024-09-16T00:09:06.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.658e-06 2024-09-16T00:09:06.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615490+0000", 2024-09-16T00:09:06.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.728e-06 2024-09-16T00:09:06.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616441+0000", 2024-09-16T00:09:06.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00095065500000000001 2024-09-16T00:09:06.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616539+0000", 2024-09-16T00:09:06.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8335e-05 2024-09-16T00:09:06.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:06.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616631+0000", 2024-09-16T00:09:06.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1260000000000004e-05 2024-09-16T00:09:06.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619837+0000", 2024-09-16T00:09:06.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003206899 2024-09-16T00:09:06.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620725+0000", 2024-09-16T00:09:06.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088713599999999996 2024-09-16T00:09:06.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620772+0000", 2024-09-16T00:09:06.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7639999999999998e-05 2024-09-16T00:09:06.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620811+0000", 2024-09-16T00:09:06.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8272999999999998e-05 2024-09-16T00:09:06.750 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.751 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.751 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.751 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:6 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:06.751 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:09:06.751 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.105158351, 2024-09-16T00:09:06.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029886169999999998, 2024-09-16T00:09:06.751 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.751 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.751 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.752 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:09:06.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615897+0000", 2024-09-16T00:09:06.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615898+0000", 2024-09-16T00:09:06.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.237e-06 2024-09-16T00:09:06.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615904+0000", 2024-09-16T00:09:06.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:09:06.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615915+0000", 2024-09-16T00:09:06.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1003e-05 2024-09-16T00:09:06.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615922+0000", 2024-09-16T00:09:06.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9959999999999996e-06 2024-09-16T00:09:06.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615988+0000", 2024-09-16T00:09:06.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5806e-05 2024-09-16T00:09:06.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616009+0000", 2024-09-16T00:09:06.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0729000000000001e-05 2024-09-16T00:09:06.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:09:06.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028462130000000002 2024-09-16T00:09:06.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618886+0000", 2024-09-16T00:09:06.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.082e-05 2024-09-16T00:09:06.756 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.756 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.756 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.756 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:06.756 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:09:06.756 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.103546879000007, 2024-09-16T00:09:06.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032610299999999998, 2024-09-16T00:09:06.756 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.756 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.757 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.757 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:09:06.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:09:06.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617556+0000", 2024-09-16T00:09:06.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7431e-05 2024-09-16T00:09:06.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617847+0000", 2024-09-16T00:09:06.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029133400000000003 2024-09-16T00:09:06.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617858+0000", 2024-09-16T00:09:06.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0222e-05 2024-09-16T00:09:06.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617864+0000", 2024-09-16T00:09:06.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7619999999999998e-06 2024-09-16T00:09:06.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617909+0000", 2024-09-16T00:09:06.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4747999999999999e-05 2024-09-16T00:09:06.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617927+0000", 2024-09-16T00:09:06.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7785999999999999e-05 2024-09-16T00:09:06.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620734+0000", 2024-09-16T00:09:06.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002807305 2024-09-16T00:09:06.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620770+0000", 2024-09-16T00:09:06.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5441999999999997e-05 2024-09-16T00:09:06.761 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.761 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.761 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.765 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:09:06.765 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:09:06.765 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.103126195000002, 2024-09-16T00:09:06.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028641809999999999, 2024-09-16T00:09:06.765 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.765 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.765 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.765 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:09:06.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617929+0000", 2024-09-16T00:09:06.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617930+0000", 2024-09-16T00:09:06.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0200000000000002e-07 2024-09-16T00:09:06.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617934+0000", 2024-09-16T00:09:06.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2069999999999997e-06 2024-09-16T00:09:06.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617943+0000", 2024-09-16T00:09:06.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5820000000000002e-06 2024-09-16T00:09:06.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617949+0000", 2024-09-16T00:09:06.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5069999999999997e-06 2024-09-16T00:09:06.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618094+0000", 2024-09-16T00:09:06.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014515599999999999 2024-09-16T00:09:06.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618111+0000", 2024-09-16T00:09:06.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6705999999999999e-05 2024-09-16T00:09:06.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620771+0000", 2024-09-16T00:09:06.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026603070000000002 2024-09-16T00:09:06.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620793+0000", 2024-09-16T00:09:06.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2214000000000001e-05 2024-09-16T00:09:06.770 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.770 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.770 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.770 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-16T00:09:06.770 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:09:06.770 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.103095405999994, 2024-09-16T00:09:06.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028624549999999999, 2024-09-16T00:09:06.770 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.770 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.771 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.771 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:09:06.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:09:06.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617960+0000", 2024-09-16T00:09:06.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:09:06.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:09:06.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0570000000000001e-06 2024-09-16T00:09:06.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617970+0000", 2024-09-16T00:09:06.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.669e-06 2024-09-16T00:09:06.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617976+0000", 2024-09-16T00:09:06.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1369999999999999e-06 2024-09-16T00:09:06.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618268+0000", 2024-09-16T00:09:06.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029164600000000001 2024-09-16T00:09:06.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.775 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.775 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618284+0000", 2024-09-16T00:09:06.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5824e-05 2024-09-16T00:09:06.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.776 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.776 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620796+0000", 2024-09-16T00:09:06.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025127890000000001 2024-09-16T00:09:06.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.776 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.776 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620822+0000", 2024-09-16T00:09:06.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6009000000000001e-05 2024-09-16T00:09:06.776 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.776 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.776 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.777 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-16T00:09:06.777 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:09:06.777 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.103067824999997, 2024-09-16T00:09:06.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028650350000000002, 2024-09-16T00:09:06.777 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.777 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.777 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.777 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:09:06.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:09:06.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617988+0000", 2024-09-16T00:09:06.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7599999999999998e-07 2024-09-16T00:09:06.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617991+0000", 2024-09-16T00:09:06.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1860000000000001e-06 2024-09-16T00:09:06.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617999+0000", 2024-09-16T00:09:06.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3250000000000002e-06 2024-09-16T00:09:06.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618004+0000", 2024-09-16T00:09:06.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7300000000000002e-06 2024-09-16T00:09:06.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618422+0000", 2024-09-16T00:09:06.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00041763199999999998 2024-09-16T00:09:06.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:09:06.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6112000000000001e-05 2024-09-16T00:09:06.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620824+0000", 2024-09-16T00:09:06.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002385954 2024-09-16T00:09:06.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620853+0000", 2024-09-16T00:09:06.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8719999999999999e-05 2024-09-16T00:09:06.781 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.781 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.781 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.782 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-16T00:09:06.782 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:09:06.782 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.103042002999999, 2024-09-16T00:09:06.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032703020000000001, 2024-09-16T00:09:06.782 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.782 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.782 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.783 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:09:06.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:09:06.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618014+0000", 2024-09-16T00:09:06.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7799999999999997e-07 2024-09-16T00:09:06.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618017+0000", 2024-09-16T00:09:06.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4309999999999998e-06 2024-09-16T00:09:06.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618023+0000", 2024-09-16T00:09:06.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8490000000000002e-06 2024-09-16T00:09:06.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618028+0000", 2024-09-16T00:09:06.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3489999999999999e-06 2024-09-16T00:09:06.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.786 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.786 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:09:06.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055724099999999999 2024-09-16T00:09:06.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.786 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.786 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618603+0000", 2024-09-16T00:09:06.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7486000000000002e-05 2024-09-16T00:09:06.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621235+0000", 2024-09-16T00:09:06.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002632087 2024-09-16T00:09:06.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621284+0000", 2024-09-16T00:09:06.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8581000000000001e-05 2024-09-16T00:09:06.788 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.788 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.788 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.788 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.788 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-16T00:09:06.788 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:09:06.788 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.102066915999998, 2024-09-16T00:09:06.788 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002325307, 2024-09-16T00:09:06.788 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.789 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.789 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.789 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.789 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.789 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:09:06.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.789 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.789 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:09:06.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618989+0000", 2024-09-16T00:09:06.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2399999999999999e-07 2024-09-16T00:09:06.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618993+0000", 2024-09-16T00:09:06.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.687e-06 2024-09-16T00:09:06.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619000+0000", 2024-09-16T00:09:06.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3590000000000002e-06 2024-09-16T00:09:06.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619005+0000", 2024-09-16T00:09:06.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.95e-06 2024-09-16T00:09:06.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619039+0000", 2024-09-16T00:09:06.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3942999999999997e-05 2024-09-16T00:09:06.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619055+0000", 2024-09-16T00:09:06.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6107e-05 2024-09-16T00:09:06.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621287+0000", 2024-09-16T00:09:06.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022318070000000001 2024-09-16T00:09:06.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621314+0000", 2024-09-16T00:09:06.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7129999999999999e-05 2024-09-16T00:09:06.794 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.794 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.794 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.795 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.795 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-16T00:09:06.795 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:09:06.795 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.102028919999995, 2024-09-16T00:09:06.795 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002314328, 2024-09-16T00:09:06.795 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.795 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.795 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.795 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.795 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.795 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.795 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:09:06.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:09:06.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619027+0000", 2024-09-16T00:09:06.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6399999999999998e-07 2024-09-16T00:09:06.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619030+0000", 2024-09-16T00:09:06.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.219e-06 2024-09-16T00:09:06.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619038+0000", 2024-09-16T00:09:06.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9899999999999997e-06 2024-09-16T00:09:06.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619043+0000", 2024-09-16T00:09:06.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5580000000000002e-06 2024-09-16T00:09:06.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619220+0000", 2024-09-16T00:09:06.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000176937 2024-09-16T00:09:06.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619237+0000", 2024-09-16T00:09:06.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7691999999999999e-05 2024-09-16T00:09:06.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621315+0000", 2024-09-16T00:09:06.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020778900000000002 2024-09-16T00:09:06.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.800 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621341+0000", 2024-09-16T00:09:06.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5678e-05 2024-09-16T00:09:06.800 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.800 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.800 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.800 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:39 2.6 2:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:09:06.801 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:09:06.801 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 83.080777596999994, 2024-09-16T00:09:06.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050072119999999996, 2024-09-16T00:09:06.801 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.801 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.801 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.801 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.801 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.801 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.801 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:09:06.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.801 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:09:06.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640280+0000", 2024-09-16T00:09:06.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5380000000000001e-06 2024-09-16T00:09:06.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640296+0000", 2024-09-16T00:09:06.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5950000000000001e-05 2024-09-16T00:09:06.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640327+0000", 2024-09-16T00:09:06.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0309999999999999e-05 2024-09-16T00:09:06.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640340+0000", 2024-09-16T00:09:06.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3773e-05 2024-09-16T00:09:06.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640422+0000", 2024-09-16T00:09:06.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1122e-05 2024-09-16T00:09:06.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640572+0000", 2024-09-16T00:09:06.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015044099999999999 2024-09-16T00:09:06.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:06.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640691+0000", 2024-09-16T00:09:06.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000119393 2024-09-16T00:09:06.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645006+0000", 2024-09-16T00:09:06.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043143779999999998 2024-09-16T00:09:06.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645169+0000", 2024-09-16T00:09:06.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016323299999999999 2024-09-16T00:09:06.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645227+0000", 2024-09-16T00:09:06.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8e-05 2024-09-16T00:09:06.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645285+0000", 2024-09-16T00:09:06.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8074e-05 2024-09-16T00:09:06.806 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.806 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.806 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.807 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:54 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e58)", 2024-09-16T00:09:06.807 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.652050+0000", 2024-09-16T00:09:06.807 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.0690057540000009, 2024-09-16T00:09:06.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039842300000000001, 2024-09-16T00:09:06.807 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.807 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.807 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.807 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.807 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652050+0000", 2024-09-16T00:09:06.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652050+0000", 2024-09-16T00:09:06.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652052+0000", 2024-09-16T00:09:06.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.857e-06 2024-09-16T00:09:06.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652101+0000", 2024-09-16T00:09:06.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9812999999999999e-05 2024-09-16T00:09:06.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652122+0000", 2024-09-16T00:09:06.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0659e-05 2024-09-16T00:09:06.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652133+0000", 2024-09-16T00:09:06.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0977e-05 2024-09-16T00:09:06.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652198+0000", 2024-09-16T00:09:06.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5229999999999997e-05 2024-09-16T00:09:06.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652293+0000", 2024-09-16T00:09:06.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4596e-05 2024-09-16T00:09:06.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:06.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.654316+0000", 2024-09-16T00:09:06.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020229699999999998 2024-09-16T00:09:06.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.655783+0000", 2024-09-16T00:09:06.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014669240000000001 2024-09-16T00:09:06.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.655974+0000", 2024-09-16T00:09:06.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000191437 2024-09-16T00:09:06.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.656008+0000", 2024-09-16T00:09:06.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3603000000000003e-05 2024-09-16T00:09:06.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.656034+0000", 2024-09-16T00:09:06.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6163999999999999e-05 2024-09-16T00:09:06.812 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.812 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.812 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.812 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:56 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e59)", 2024-09-16T00:09:06.812 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.652483+0000", 2024-09-16T00:09:06.812 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.0685725589999997, 2024-09-16T00:09:06.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003414191, 2024-09-16T00:09:06.813 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.813 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.813 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.813 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.813 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.813 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652483+0000", 2024-09-16T00:09:06.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652483+0000", 2024-09-16T00:09:06.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652485+0000", 2024-09-16T00:09:06.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4059999999999999e-06 2024-09-16T00:09:06.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652527+0000", 2024-09-16T00:09:06.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1319000000000002e-05 2024-09-16T00:09:06.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652548+0000", 2024-09-16T00:09:06.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1149999999999999e-05 2024-09-16T00:09:06.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652562+0000", 2024-09-16T00:09:06.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3971000000000001e-05 2024-09-16T00:09:06.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652631+0000", 2024-09-16T00:09:06.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9411000000000003e-05 2024-09-16T00:09:06.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652717+0000", 2024-09-16T00:09:06.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5556999999999994e-05 2024-09-16T00:09:06.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.818 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:06.818 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.654006+0000", 2024-09-16T00:09:06.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012897200000000001 2024-09-16T00:09:06.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.818 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.818 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.818 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.655280+0000", 2024-09-16T00:09:06.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012732609999999999 2024-09-16T00:09:06.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.819 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.819 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.655811+0000", 2024-09-16T00:09:06.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00053082400000000005 2024-09-16T00:09:06.819 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.819 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.819 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.655862+0000", 2024-09-16T00:09:06.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1672000000000003e-05 2024-09-16T00:09:06.819 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.655897+0000", 2024-09-16T00:09:06.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4900000000000001e-05 2024-09-16T00:09:06.820 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.820 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.820 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.820 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:71 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:06.820 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:01.656409+0000", 2024-09-16T00:09:06.820 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.0646462100000003, 2024-09-16T00:09:06.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023563120000000002, 2024-09-16T00:09:06.821 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.821 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.821 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.821 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.656409+0000", 2024-09-16T00:09:06.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.656409+0000", 2024-09-16T00:09:06.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.656411+0000", 2024-09-16T00:09:06.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.812e-06 2024-09-16T00:09:06.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.656457+0000", 2024-09-16T00:09:06.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5921999999999998e-05 2024-09-16T00:09:06.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.656470+0000", 2024-09-16T00:09:06.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2559e-05 2024-09-16T00:09:06.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.656476+0000", 2024-09-16T00:09:06.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1190000000000002e-06 2024-09-16T00:09:06.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.824 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.656518+0000", 2024-09-16T00:09:06.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2228000000000003e-05 2024-09-16T00:09:06.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.824 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.656577+0000", 2024-09-16T00:09:06.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9036999999999997e-05 2024-09-16T00:09:06.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:06.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.657373+0000", 2024-09-16T00:09:06.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079629799999999997 2024-09-16T00:09:06.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.658546+0000", 2024-09-16T00:09:06.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001172962 2024-09-16T00:09:06.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.658655+0000", 2024-09-16T00:09:06.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108863 2024-09-16T00:09:06.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.658713+0000", 2024-09-16T00:09:06.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8242999999999998e-05 2024-09-16T00:09:06.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.658766+0000", 2024-09-16T00:09:06.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2268999999999997e-05 2024-09-16T00:09:06.827 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.827 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.827 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.827 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:85 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:06.828 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:02.658347+0000", 2024-09-16T00:09:06.828 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.0627084330000001, 2024-09-16T00:09:06.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047582939999999997, 2024-09-16T00:09:06.828 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.828 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.828 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.828 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658347+0000", 2024-09-16T00:09:06.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658347+0000", 2024-09-16T00:09:06.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658349+0000", 2024-09-16T00:09:06.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7999999999999999e-06 2024-09-16T00:09:06.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658439+0000", 2024-09-16T00:09:06.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0276000000000001e-05 2024-09-16T00:09:06.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658454+0000", 2024-09-16T00:09:06.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5105e-05 2024-09-16T00:09:06.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658462+0000", 2024-09-16T00:09:06.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.525e-06 2024-09-16T00:09:06.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658512+0000", 2024-09-16T00:09:06.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9877999999999999e-05 2024-09-16T00:09:06.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658569+0000", 2024-09-16T00:09:06.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7022999999999999e-05 2024-09-16T00:09:06.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:06.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.660742+0000", 2024-09-16T00:09:06.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021734419999999998 2024-09-16T00:09:06.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.662841+0000", 2024-09-16T00:09:06.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020994120000000002 2024-09-16T00:09:06.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.663045+0000", 2024-09-16T00:09:06.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000203422 2024-09-16T00:09:06.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.663079+0000", 2024-09-16T00:09:06.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3769999999999997e-05 2024-09-16T00:09:06.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.663105+0000", 2024-09-16T00:09:06.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6641000000000001e-05 2024-09-16T00:09:06.835 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.835 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.835 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.836 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:2297 3.25 3:a5f7fd7e:::10000000c4d.00000000:head [write 0~2713] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T00:09:06.836 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.384329+0000", 2024-09-16T00:09:06.836 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.3367259740000001, 2024-09-16T00:09:06.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023695169999999998, 2024-09-16T00:09:06.836 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.836 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.836 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.836 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:06.836 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:06.836 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:06.837 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 2297 2024-09-16T00:09:06.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.837 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.384329+0000", 2024-09-16T00:09:06.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.384329+0000", 2024-09-16T00:09:06.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.384330+0000", 2024-09-16T00:09:06.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1099999999999995e-07 2024-09-16T00:09:06.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.384335+0000", 2024-09-16T00:09:06.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6609999999999999e-06 2024-09-16T00:09:06.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.384345+0000", 2024-09-16T00:09:06.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0237999999999999e-05 2024-09-16T00:09:06.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.384350+0000", 2024-09-16T00:09:06.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3210000000000004e-06 2024-09-16T00:09:06.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.384391+0000", 2024-09-16T00:09:06.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0243000000000001e-05 2024-09-16T00:09:06.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.384470+0000", 2024-09-16T00:09:06.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9066000000000005e-05 2024-09-16T00:09:06.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:06.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.384552+0000", 2024-09-16T00:09:06.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1904999999999996e-05 2024-09-16T00:09:06.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.385239+0000", 2024-09-16T00:09:06.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00068786100000000001 2024-09-16T00:09:06.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.386650+0000", 2024-09-16T00:09:06.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001410498 2024-09-16T00:09:06.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.386678+0000", 2024-09-16T00:09:06.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8410000000000001e-05 2024-09-16T00:09:06.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.386699+0000", 2024-09-16T00:09:06.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0602999999999999e-05 2024-09-16T00:09:06.842 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.842 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.842 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.842 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:3098 3.31 3:8cd33917:::10000000f77.00000000:head [write 0~2537] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T00:09:06.842 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:06.093624+0000", 2024-09-16T00:09:06.842 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.62743170199999998, 2024-09-16T00:09:06.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024968910000000002, 2024-09-16T00:09:06.842 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.842 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.843 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.843 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:06.843 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:06.843 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:06.843 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 3098 2024-09-16T00:09:06.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.843 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.093624+0000", 2024-09-16T00:09:06.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.093624+0000", 2024-09-16T00:09:06.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.093625+0000", 2024-09-16T00:09:06.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7209999999999999e-06 2024-09-16T00:09:06.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.093634+0000", 2024-09-16T00:09:06.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.772e-06 2024-09-16T00:09:06.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.093650+0000", 2024-09-16T00:09:06.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.552e-05 2024-09-16T00:09:06.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.093657+0000", 2024-09-16T00:09:06.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3880000000000002e-06 2024-09-16T00:09:06.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.093719+0000", 2024-09-16T00:09:06.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1945000000000004e-05 2024-09-16T00:09:06.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op must wait for map", 2024-09-16T00:09:06.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.093727+0000", 2024-09-16T00:09:06.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9899999999999997e-06 2024-09-16T00:09:06.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.095015+0000", 2024-09-16T00:09:06.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012883980000000001 2024-09-16T00:09:06.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.095141+0000", 2024-09-16T00:09:06.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012560499999999999 2024-09-16T00:09:06.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:06.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.095269+0000", 2024-09-16T00:09:06.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000127427 2024-09-16T00:09:06.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.095661+0000", 2024-09-16T00:09:06.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00039213399999999998 2024-09-16T00:09:06.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096060+0000", 2024-09-16T00:09:06.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00039922299999999998 2024-09-16T00:09:06.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096088+0000", 2024-09-16T00:09:06.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7647000000000001e-05 2024-09-16T00:09:06.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096121+0000", 2024-09-16T00:09:06.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3120999999999997e-05 2024-09-16T00:09:06.849 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.849 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.849 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.849 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:160 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e64)", 2024-09-16T00:09:06.849 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:06.662147+0000", 2024-09-16T00:09:06.849 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.058908039000000002, 2024-09-16T00:09:06.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034276770000000001, 2024-09-16T00:09:06.849 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:06.849 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:06.849 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:06.849 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:06.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:06.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.662147+0000", 2024-09-16T00:09:06.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:06.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.662147+0000", 2024-09-16T00:09:06.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:06.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:06.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.662150+0000", 2024-09-16T00:09:06.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2400000000000002e-06 2024-09-16T00:09:06.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:06.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.662188+0000", 2024-09-16T00:09:06.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8176000000000002e-05 2024-09-16T00:09:06.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:06.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.662211+0000", 2024-09-16T00:09:06.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3603000000000001e-05 2024-09-16T00:09:06.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:06.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.662242+0000", 2024-09-16T00:09:06.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0446e-05 2024-09-16T00:09:06.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:06.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.662313+0000", 2024-09-16T00:09:06.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0730999999999998e-05 2024-09-16T00:09:06.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:06.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.662413+0000", 2024-09-16T00:09:06.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000100204 2024-09-16T00:09:06.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:06.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.663804+0000", 2024-09-16T00:09:06.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013914890000000001 2024-09-16T00:09:06.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:06.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.665348+0000", 2024-09-16T00:09:06.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001543574 2024-09-16T00:09:06.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:06.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.665465+0000", 2024-09-16T00:09:06.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117224 2024-09-16T00:09:06.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:06.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.665521+0000", 2024-09-16T00:09:06.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5634999999999998e-05 2024-09-16T00:09:06.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:06.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:06.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:06.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.665575+0000", 2024-09-16T00:09:06.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4354999999999997e-05 2024-09-16T00:09:06.854 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.854 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.854 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.854 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:06.854 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:06.855 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:06.856 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:09:06.857 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:06.855+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:06.956 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:06.956+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:07.016 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:07.016 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:07.016 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:07.016 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:07.024 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:09:07.069 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:07.068+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:07.133 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:07.133 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:07.133 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:07.133 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:07.133 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:07.143 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:09:07.185 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:07.184+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:07.258 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:07.258+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:07.308 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:07.308 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:07.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:07.308 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:07.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.309 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:07.309 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:09:07.309 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.690543474999998, 2024-09-16T00:09:07.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038263220000000001, 2024-09-16T00:09:07.309 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.309 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.309 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.309 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:09:07.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:09:07.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615465+0000", 2024-09-16T00:09:07.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2769999999999999e-06 2024-09-16T00:09:07.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615471+0000", 2024-09-16T00:09:07.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5729999999999996e-06 2024-09-16T00:09:07.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615483+0000", 2024-09-16T00:09:07.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1838000000000001e-05 2024-09-16T00:09:07.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615492+0000", 2024-09-16T00:09:07.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2410000000000001e-06 2024-09-16T00:09:07.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615596+0000", 2024-09-16T00:09:07.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104097 2024-09-16T00:09:07.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615688+0000", 2024-09-16T00:09:07.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1225999999999997e-05 2024-09-16T00:09:07.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:07.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615804+0000", 2024-09-16T00:09:07.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000116601 2024-09-16T00:09:07.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618165+0000", 2024-09-16T00:09:07.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023609429999999999 2024-09-16T00:09:07.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:09:07.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010258509999999999 2024-09-16T00:09:07.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619243+0000", 2024-09-16T00:09:07.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2327999999999998e-05 2024-09-16T00:09:07.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619287+0000", 2024-09-16T00:09:07.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3346999999999997e-05 2024-09-16T00:09:07.314 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.314 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.314 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.315 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:07.315 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:07.315 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.690488399000003, 2024-09-16T00:09:07.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0054735510000000001, 2024-09-16T00:09:07.315 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.315 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.316 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.316 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:07.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:07.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615516+0000", 2024-09-16T00:09:07.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6899999999999998e-07 2024-09-16T00:09:07.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615519+0000", 2024-09-16T00:09:07.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1760000000000002e-06 2024-09-16T00:09:07.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615526+0000", 2024-09-16T00:09:07.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8399999999999997e-06 2024-09-16T00:09:07.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615531+0000", 2024-09-16T00:09:07.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4450000000000004e-06 2024-09-16T00:09:07.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:09:07.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054127099999999996 2024-09-16T00:09:07.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616140+0000", 2024-09-16T00:09:07.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7230999999999994e-05 2024-09-16T00:09:07.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:07.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616249+0000", 2024-09-16T00:09:07.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109122 2024-09-16T00:09:07.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618646+0000", 2024-09-16T00:09:07.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023972279999999999 2024-09-16T00:09:07.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:09:07.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022396220000000001 2024-09-16T00:09:07.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620939+0000", 2024-09-16T00:09:07.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2735999999999998e-05 2024-09-16T00:09:07.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620989+0000", 2024-09-16T00:09:07.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0411000000000002e-05 2024-09-16T00:09:07.322 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.322 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.322 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.322 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:07.322 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:07.322 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.690462920000002, 2024-09-16T00:09:07.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055872789999999997, 2024-09-16T00:09:07.322 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.322 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.322 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.322 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:07.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:07.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:07.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.01e-07 2024-09-16T00:09:07.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615547+0000", 2024-09-16T00:09:07.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7899999999999996e-06 2024-09-16T00:09:07.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:09:07.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1119999999999998e-06 2024-09-16T00:09:07.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615560+0000", 2024-09-16T00:09:07.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:09:07.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616424+0000", 2024-09-16T00:09:07.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086350800000000003 2024-09-16T00:09:07.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:09:07.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9675999999999999e-05 2024-09-16T00:09:07.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:07.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616621+0000", 2024-09-16T00:09:07.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7345999999999999e-05 2024-09-16T00:09:07.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618654+0000", 2024-09-16T00:09:07.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020335269999999998 2024-09-16T00:09:07.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621055+0000", 2024-09-16T00:09:07.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024009729999999998 2024-09-16T00:09:07.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621100+0000", 2024-09-16T00:09:07.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5402999999999999e-05 2024-09-16T00:09:07.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621128+0000", 2024-09-16T00:09:07.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7736e-05 2024-09-16T00:09:07.327 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.328 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.328 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.328 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:07.328 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:07.328 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.690437270000004, 2024-09-16T00:09:07.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057024509999999999, 2024-09-16T00:09:07.328 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.328 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.328 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.328 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:07.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:07.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:07.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6300000000000001e-07 2024-09-16T00:09:07.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615569+0000", 2024-09-16T00:09:07.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3709999999999998e-06 2024-09-16T00:09:07.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615580+0000", 2024-09-16T00:09:07.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0448000000000001e-05 2024-09-16T00:09:07.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615584+0000", 2024-09-16T00:09:07.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6020000000000002e-06 2024-09-16T00:09:07.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616850+0000", 2024-09-16T00:09:07.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012661110000000001 2024-09-16T00:09:07.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616915+0000", 2024-09-16T00:09:07.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4820000000000006e-05 2024-09-16T00:09:07.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:07.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617031+0000", 2024-09-16T00:09:07.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115258 2024-09-16T00:09:07.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618659+0000", 2024-09-16T00:09:07.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001627975 2024-09-16T00:09:07.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621179+0000", 2024-09-16T00:09:07.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025203410000000002 2024-09-16T00:09:07.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.333 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621238+0000", 2024-09-16T00:09:07.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9323000000000001e-05 2024-09-16T00:09:07.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.333 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621269+0000", 2024-09-16T00:09:07.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0839000000000001e-05 2024-09-16T00:09:07.333 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.333 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.333 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.334 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:07.334 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:09:07.334 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.690413520999996, 2024-09-16T00:09:07.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058012769999999996, 2024-09-16T00:09:07.334 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.334 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.334 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.334 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:09:07.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:09:07.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615591+0000", 2024-09-16T00:09:07.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6199999999999999e-07 2024-09-16T00:09:07.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615594+0000", 2024-09-16T00:09:07.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8739999999999999e-06 2024-09-16T00:09:07.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615599+0000", 2024-09-16T00:09:07.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:09:07.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615603+0000", 2024-09-16T00:09:07.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8489999999999999e-06 2024-09-16T00:09:07.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617235+0000", 2024-09-16T00:09:07.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001632037 2024-09-16T00:09:07.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617348+0000", 2024-09-16T00:09:07.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113387 2024-09-16T00:09:07.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:07.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617458+0000", 2024-09-16T00:09:07.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109765 2024-09-16T00:09:07.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618664+0000", 2024-09-16T00:09:07.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012059950000000001 2024-09-16T00:09:07.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621311+0000", 2024-09-16T00:09:07.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026468469999999999 2024-09-16T00:09:07.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621357+0000", 2024-09-16T00:09:07.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6360999999999999e-05 2024-09-16T00:09:07.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621392+0000", 2024-09-16T00:09:07.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4190999999999998e-05 2024-09-16T00:09:07.339 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.339 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.339 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.340 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:07.340 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:07.340 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.690394897000004, 2024-09-16T00:09:07.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058737279999999999, 2024-09-16T00:09:07.340 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.340 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.340 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.340 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.340 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:07.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:07.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:07.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3799999999999999e-07 2024-09-16T00:09:07.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615611+0000", 2024-09-16T00:09:07.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-06 2024-09-16T00:09:07.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615626+0000", 2024-09-16T00:09:07.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:09:07.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615634+0000", 2024-09-16T00:09:07.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-06 2024-09-16T00:09:07.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617692+0000", 2024-09-16T00:09:07.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002057718 2024-09-16T00:09:07.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617768+0000", 2024-09-16T00:09:07.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6878999999999994e-05 2024-09-16T00:09:07.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.344 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:07.344 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:09:07.344 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019501699999999999 2024-09-16T00:09:07.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.344 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.344 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:09:07.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034357200000000002 2024-09-16T00:09:07.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621429+0000", 2024-09-16T00:09:07.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0012999999999999e-05 2024-09-16T00:09:07.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621459+0000", 2024-09-16T00:09:07.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9498000000000001e-05 2024-09-16T00:09:07.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.346 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621483+0000", 2024-09-16T00:09:07.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3957000000000001e-05 2024-09-16T00:09:07.346 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.346 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.346 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.346 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:07.346 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:09:07.346 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.690363617000003, 2024-09-16T00:09:07.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059371789999999999, 2024-09-16T00:09:07.346 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.347 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.347 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.347 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:09:07.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:09:07.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:09:07.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4000000000000002e-07 2024-09-16T00:09:07.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615643+0000", 2024-09-16T00:09:07.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6860000000000002e-06 2024-09-16T00:09:07.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615649+0000", 2024-09-16T00:09:07.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7610000000000004e-06 2024-09-16T00:09:07.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615653+0000", 2024-09-16T00:09:07.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0380000000000001e-06 2024-09-16T00:09:07.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618197+0000", 2024-09-16T00:09:07.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025440079999999999 2024-09-16T00:09:07.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.350 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618312+0000", 2024-09-16T00:09:07.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115083 2024-09-16T00:09:07.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.350 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:07.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618412+0000", 2024-09-16T00:09:07.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010016799999999999 2024-09-16T00:09:07.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.351 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.351 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621518+0000", 2024-09-16T00:09:07.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031058679999999999 2024-09-16T00:09:07.351 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.351 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.351 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.351 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621526+0000", 2024-09-16T00:09:07.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2490000000000004e-06 2024-09-16T00:09:07.351 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.351 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.351 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.352 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621549+0000", 2024-09-16T00:09:07.352 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3686999999999999e-05 2024-09-16T00:09:07.352 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.352 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.352 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621577+0000", 2024-09-16T00:09:07.352 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8191000000000002e-05 2024-09-16T00:09:07.352 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.352 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.352 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.352 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.353 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:07.353 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:07.353 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.690345950999998, 2024-09-16T00:09:07.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005989622, 2024-09-16T00:09:07.353 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.353 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.353 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.353 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:07.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:07.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:07.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3799999999999998e-07 2024-09-16T00:09:07.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615660+0000", 2024-09-16T00:09:07.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9740000000000001e-06 2024-09-16T00:09:07.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615666+0000", 2024-09-16T00:09:07.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3499999999999996e-06 2024-09-16T00:09:07.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615671+0000", 2024-09-16T00:09:07.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.13e-06 2024-09-16T00:09:07.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618759+0000", 2024-09-16T00:09:07.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030880500000000002 2024-09-16T00:09:07.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618832+0000", 2024-09-16T00:09:07.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3725000000000003e-05 2024-09-16T00:09:07.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:07.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618958+0000", 2024-09-16T00:09:07.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012587899999999999 2024-09-16T00:09:07.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621583+0000", 2024-09-16T00:09:07.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026246469999999999 2024-09-16T00:09:07.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621602+0000", 2024-09-16T00:09:07.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9137e-05 2024-09-16T00:09:07.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621629+0000", 2024-09-16T00:09:07.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6451000000000001e-05 2024-09-16T00:09:07.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621648+0000", 2024-09-16T00:09:07.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8941e-05 2024-09-16T00:09:07.358 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.358 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.358 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.359 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:07.359 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:07.359 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.690328332000007, 2024-09-16T00:09:07.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053141380000000004, 2024-09-16T00:09:07.359 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.359 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.359 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.359 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:07.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:07.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:07.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-07 2024-09-16T00:09:07.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615678+0000", 2024-09-16T00:09:07.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:09:07.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615682+0000", 2024-09-16T00:09:07.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6659999999999999e-06 2024-09-16T00:09:07.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615686+0000", 2024-09-16T00:09:07.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6990000000000001e-06 2024-09-16T00:09:07.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615736+0000", 2024-09-16T00:09:07.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9656e-05 2024-09-16T00:09:07.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615864+0000", 2024-09-16T00:09:07.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012836100000000001 2024-09-16T00:09:07.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:07.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615989+0000", 2024-09-16T00:09:07.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000125525 2024-09-16T00:09:07.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617355+0000", 2024-09-16T00:09:07.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001365982 2024-09-16T00:09:07.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620897+0000", 2024-09-16T00:09:07.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035419610000000002 2024-09-16T00:09:07.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.364 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.364 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620941+0000", 2024-09-16T00:09:07.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3998999999999997e-05 2024-09-16T00:09:07.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.364 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.364 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620990+0000", 2024-09-16T00:09:07.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8199999999999999e-05 2024-09-16T00:09:07.364 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.364 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.364 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.365 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:07.365 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:07.365 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 88.690301074000004, 2024-09-16T00:09:07.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006316449, 2024-09-16T00:09:07.365 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.365 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.365 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.365 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:07.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:07.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:07.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3299999999999998e-07 2024-09-16T00:09:07.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615705+0000", 2024-09-16T00:09:07.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.278e-06 2024-09-16T00:09:07.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615711+0000", 2024-09-16T00:09:07.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8119999999999996e-06 2024-09-16T00:09:07.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615716+0000", 2024-09-16T00:09:07.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4909999999999999e-06 2024-09-16T00:09:07.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617551+0000", 2024-09-16T00:09:07.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00183549 2024-09-16T00:09:07.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617627+0000", 2024-09-16T00:09:07.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6163999999999998e-05 2024-09-16T00:09:07.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:07.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617776+0000", 2024-09-16T00:09:07.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014909499999999999 2024-09-16T00:09:07.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621072+0000", 2024-09-16T00:09:07.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032955150000000002 2024-09-16T00:09:07.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621957+0000", 2024-09-16T00:09:07.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088553199999999996 2024-09-16T00:09:07.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621998+0000", 2024-09-16T00:09:07.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0815000000000001e-05 2024-09-16T00:09:07.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:09:07.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0924000000000002e-05 2024-09-16T00:09:07.371 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.371 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.371 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.371 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:501 3.38 3:1f24713e:::1000000042b.00000000:head [write 0~71] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:07.371 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.761706+0000", 2024-09-16T00:09:07.371 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.5442982670000003, 2024-09-16T00:09:07.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001845941, 2024-09-16T00:09:07.372 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.372 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.372 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.372 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:07.372 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:07.372 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:07.372 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 501 2024-09-16T00:09:07.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.373 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761706+0000", 2024-09-16T00:09:07.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761706+0000", 2024-09-16T00:09:07.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761707+0000", 2024-09-16T00:09:07.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8509999999999999e-06 2024-09-16T00:09:07.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761717+0000", 2024-09-16T00:09:07.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9250000000000007e-06 2024-09-16T00:09:07.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761732+0000", 2024-09-16T00:09:07.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4201e-05 2024-09-16T00:09:07.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761738+0000", 2024-09-16T00:09:07.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7299999999999999e-06 2024-09-16T00:09:07.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761819+0000", 2024-09-16T00:09:07.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1044999999999994e-05 2024-09-16T00:09:07.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761936+0000", 2024-09-16T00:09:07.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011626299999999999 2024-09-16T00:09:07.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:07.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.762051+0000", 2024-09-16T00:09:07.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011582999999999999 2024-09-16T00:09:07.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.762671+0000", 2024-09-16T00:09:07.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00061922099999999999 2024-09-16T00:09:07.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.763496+0000", 2024-09-16T00:09:07.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00082528899999999999 2024-09-16T00:09:07.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.763525+0000", 2024-09-16T00:09:07.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9488999999999998e-05 2024-09-16T00:09:07.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.763552+0000", 2024-09-16T00:09:07.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6097000000000001e-05 2024-09-16T00:09:07.381 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.381 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.381 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.381 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:584 3.22 3:47e2f9af:::10000000492.00000000:head [write 0~10710] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:07.381 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.989570+0000", 2024-09-16T00:09:07.381 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.3164334909999997, 2024-09-16T00:09:07.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038063620000000002, 2024-09-16T00:09:07.381 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.381 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.381 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.381 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:07.382 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:07.382 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:07.382 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 584 2024-09-16T00:09:07.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.383 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989570+0000", 2024-09-16T00:09:07.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989570+0000", 2024-09-16T00:09:07.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989573+0000", 2024-09-16T00:09:07.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9909999999999998e-06 2024-09-16T00:09:07.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989602+0000", 2024-09-16T00:09:07.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8353000000000001e-05 2024-09-16T00:09:07.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989622+0000", 2024-09-16T00:09:07.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9905999999999999e-05 2024-09-16T00:09:07.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989631+0000", 2024-09-16T00:09:07.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6360000000000006e-06 2024-09-16T00:09:07.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989706+0000", 2024-09-16T00:09:07.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4493000000000007e-05 2024-09-16T00:09:07.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989856+0000", 2024-09-16T00:09:07.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015023700000000001 2024-09-16T00:09:07.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:07.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.990004+0000", 2024-09-16T00:09:07.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000148157 2024-09-16T00:09:07.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.992924+0000", 2024-09-16T00:09:07.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029203250000000001 2024-09-16T00:09:07.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.993270+0000", 2024-09-16T00:09:07.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00034548899999999998 2024-09-16T00:09:07.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.993325+0000", 2024-09-16T00:09:07.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5337999999999998e-05 2024-09-16T00:09:07.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.993377+0000", 2024-09-16T00:09:07.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1437000000000002e-05 2024-09-16T00:09:07.391 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.391 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.391 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.392 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:589 3.22 3:44283f82:::10000000498.00000000:head [write 0~1058] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:07.392 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.007191+0000", 2024-09-16T00:09:07.392 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.2988124279999997, 2024-09-16T00:09:07.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003237729, 2024-09-16T00:09:07.392 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.392 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.392 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.392 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:07.392 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:07.392 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:07.393 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 589 2024-09-16T00:09:07.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.393 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007191+0000", 2024-09-16T00:09:07.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007191+0000", 2024-09-16T00:09:07.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007193+0000", 2024-09-16T00:09:07.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8449999999999999e-06 2024-09-16T00:09:07.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007206+0000", 2024-09-16T00:09:07.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2509e-05 2024-09-16T00:09:07.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007223+0000", 2024-09-16T00:09:07.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7220000000000001e-05 2024-09-16T00:09:07.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007231+0000", 2024-09-16T00:09:07.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9370000000000004e-06 2024-09-16T00:09:07.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007285+0000", 2024-09-16T00:09:07.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4472000000000003e-05 2024-09-16T00:09:07.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op must wait for map", 2024-09-16T00:09:07.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007289+0000", 2024-09-16T00:09:07.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0679999999999998e-06 2024-09-16T00:09:07.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009018+0000", 2024-09-16T00:09:07.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017285270000000001 2024-09-16T00:09:07.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009121+0000", 2024-09-16T00:09:07.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010267 2024-09-16T00:09:07.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:07.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009236+0000", 2024-09-16T00:09:07.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115563 2024-09-16T00:09:07.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009903+0000", 2024-09-16T00:09:07.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00066633999999999997 2024-09-16T00:09:07.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.010348+0000", 2024-09-16T00:09:07.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00044550999999999998 2024-09-16T00:09:07.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.010388+0000", 2024-09-16T00:09:07.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0240000000000001e-05 2024-09-16T00:09:07.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.010429+0000", 2024-09-16T00:09:07.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0828000000000003e-05 2024-09-16T00:09:07.398 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.398 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.398 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.399 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:590 3.21 3:86dbbdca:::10000000499.00000000:head [write 0~3104] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:07.399 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.009656+0000", 2024-09-16T00:09:07.399 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.2963479719999995, 2024-09-16T00:09:07.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001717283, 2024-09-16T00:09:07.399 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.399 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.399 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.399 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:07.399 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:07.399 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:07.399 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 590 2024-09-16T00:09:07.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.400 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009656+0000", 2024-09-16T00:09:07.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009656+0000", 2024-09-16T00:09:07.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009657+0000", 2024-09-16T00:09:07.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.514e-06 2024-09-16T00:09:07.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009664+0000", 2024-09-16T00:09:07.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6819999999999997e-06 2024-09-16T00:09:07.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009677+0000", 2024-09-16T00:09:07.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2841e-05 2024-09-16T00:09:07.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009687+0000", 2024-09-16T00:09:07.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0171e-05 2024-09-16T00:09:07.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009742+0000", 2024-09-16T00:09:07.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4855999999999998e-05 2024-09-16T00:09:07.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009874+0000", 2024-09-16T00:09:07.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013217099999999999 2024-09-16T00:09:07.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:07.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009994+0000", 2024-09-16T00:09:07.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000120252 2024-09-16T00:09:07.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.010890+0000", 2024-09-16T00:09:07.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00089574900000000005 2024-09-16T00:09:07.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.011289+0000", 2024-09-16T00:09:07.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00039914799999999999 2024-09-16T00:09:07.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.011336+0000", 2024-09-16T00:09:07.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6306e-05 2024-09-16T00:09:07.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.011373+0000", 2024-09-16T00:09:07.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7592999999999997e-05 2024-09-16T00:09:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:665 3.3d e59/20 3:be78f56d:::100000004f7.00000000:head v 59'10, mlcod=58'8)", 2024-09-16T00:09:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.222373+0000", 2024-09-16T00:09:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.0836312160000006, 2024-09-16T00:09:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017385230000000001, 2024-09-16T00:09:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222373+0000", 2024-09-16T00:09:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222373+0000", 2024-09-16T00:09:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222374+0000", 2024-09-16T00:09:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5680000000000001e-06 2024-09-16T00:09:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222384+0000", 2024-09-16T00:09:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9690000000000006e-06 2024-09-16T00:09:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222405+0000", 2024-09-16T00:09:07.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0840999999999999e-05 2024-09-16T00:09:07.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222427+0000", 2024-09-16T00:09:07.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2348e-05 2024-09-16T00:09:07.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222489+0000", 2024-09-16T00:09:07.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1884000000000006e-05 2024-09-16T00:09:07.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222512+0000", 2024-09-16T00:09:07.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2464999999999999e-05 2024-09-16T00:09:07.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.224062+0000", 2024-09-16T00:09:07.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001550685 2024-09-16T00:09:07.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.224111+0000", 2024-09-16T00:09:07.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8763000000000001e-05 2024-09-16T00:09:07.410 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.410 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.410 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.410 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:714 3.22 3:453cbc5d:::10000000536.00000000:head [write 0~260] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:07.410 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.356475+0000", 2024-09-16T00:09:07.410 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.949529193, 2024-09-16T00:09:07.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003898926, 2024-09-16T00:09:07.410 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.410 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.411 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.411 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:07.411 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:07.411 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:07.411 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 714 2024-09-16T00:09:07.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.411 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356475+0000", 2024-09-16T00:09:07.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356475+0000", 2024-09-16T00:09:07.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356476+0000", 2024-09-16T00:09:07.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6750000000000001e-06 2024-09-16T00:09:07.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356493+0000", 2024-09-16T00:09:07.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6562000000000002e-05 2024-09-16T00:09:07.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356509+0000", 2024-09-16T00:09:07.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5908000000000001e-05 2024-09-16T00:09:07.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356516+0000", 2024-09-16T00:09:07.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9079999999999998e-06 2024-09-16T00:09:07.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356573+0000", 2024-09-16T00:09:07.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7243000000000001e-05 2024-09-16T00:09:07.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356657+0000", 2024-09-16T00:09:07.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3809999999999999e-05 2024-09-16T00:09:07.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:07.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356742+0000", 2024-09-16T00:09:07.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5704000000000005e-05 2024-09-16T00:09:07.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360000+0000", 2024-09-16T00:09:07.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003257916 2024-09-16T00:09:07.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360279+0000", 2024-09-16T00:09:07.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00027872500000000001 2024-09-16T00:09:07.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360324+0000", 2024-09-16T00:09:07.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5359999999999999e-05 2024-09-16T00:09:07.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360374+0000", 2024-09-16T00:09:07.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9115e-05 2024-09-16T00:09:07.416 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.416 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.416 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.417 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:766 3.1f 3:f8970b5b:::10000000571.00000000:head [write 0~1989] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:07.417 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.481586+0000", 2024-09-16T00:09:07.417 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.8244175380000005, 2024-09-16T00:09:07.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038531529999999998, 2024-09-16T00:09:07.417 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.417 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.417 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.417 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:07.417 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:07.417 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:07.417 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 766 2024-09-16T00:09:07.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.418 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481586+0000", 2024-09-16T00:09:07.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481586+0000", 2024-09-16T00:09:07.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481588+0000", 2024-09-16T00:09:07.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.691e-06 2024-09-16T00:09:07.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481595+0000", 2024-09-16T00:09:07.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5780000000000004e-06 2024-09-16T00:09:07.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481605+0000", 2024-09-16T00:09:07.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0729e-05 2024-09-16T00:09:07.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481610+0000", 2024-09-16T00:09:07.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:09:07.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481663+0000", 2024-09-16T00:09:07.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2429999999999998e-05 2024-09-16T00:09:07.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481740+0000", 2024-09-16T00:09:07.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7526000000000003e-05 2024-09-16T00:09:07.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:07.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481829+0000", 2024-09-16T00:09:07.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8265999999999998e-05 2024-09-16T00:09:07.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.482578+0000", 2024-09-16T00:09:07.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074936600000000001 2024-09-16T00:09:07.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.485314+0000", 2024-09-16T00:09:07.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027358949999999999 2024-09-16T00:09:07.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.485398+0000", 2024-09-16T00:09:07.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4462000000000005e-05 2024-09-16T00:09:07.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.485439+0000", 2024-09-16T00:09:07.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1066999999999998e-05 2024-09-16T00:09:07.422 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.423 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.423 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.423 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:816 3.21 3:86b80525:::100000005a6.00000000:head [write 0~4693] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:07.423 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.596020+0000", 2024-09-16T00:09:07.423 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.7099841040000001, 2024-09-16T00:09:07.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016622049999999999, 2024-09-16T00:09:07.423 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.423 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.423 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.423 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:07.424 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:07.424 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:07.424 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 816 2024-09-16T00:09:07.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.424 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596020+0000", 2024-09-16T00:09:07.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596020+0000", 2024-09-16T00:09:07.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596023+0000", 2024-09-16T00:09:07.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9790000000000002e-06 2024-09-16T00:09:07.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596041+0000", 2024-09-16T00:09:07.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8003999999999999e-05 2024-09-16T00:09:07.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596065+0000", 2024-09-16T00:09:07.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4646000000000001e-05 2024-09-16T00:09:07.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596079+0000", 2024-09-16T00:09:07.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.382e-05 2024-09-16T00:09:07.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596151+0000", 2024-09-16T00:09:07.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1727e-05 2024-09-16T00:09:07.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596305+0000", 2024-09-16T00:09:07.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015395799999999999 2024-09-16T00:09:07.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:07.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.596468+0000", 2024-09-16T00:09:07.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016360499999999999 2024-09-16T00:09:07.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.597256+0000", 2024-09-16T00:09:07.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00078736799999999999 2024-09-16T00:09:07.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.597596+0000", 2024-09-16T00:09:07.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00034032499999999999 2024-09-16T00:09:07.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.597651+0000", 2024-09-16T00:09:07.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5148999999999997e-05 2024-09-16T00:09:07.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.597682+0000", 2024-09-16T00:09:07.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0623999999999997e-05 2024-09-16T00:09:07.429 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.429 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.429 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.429 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:839 3.38 3:1d7725f9:::100000005c8.00000000:head [write 0~3439] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:07.429 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.686484+0000", 2024-09-16T00:09:07.430 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.6195194530000006, 2024-09-16T00:09:07.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016765650000000001, 2024-09-16T00:09:07.430 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.430 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.430 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.430 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:07.430 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:07.430 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:07.430 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 839 2024-09-16T00:09:07.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.430 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686484+0000", 2024-09-16T00:09:07.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686484+0000", 2024-09-16T00:09:07.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686487+0000", 2024-09-16T00:09:07.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2989999999999999e-06 2024-09-16T00:09:07.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686501+0000", 2024-09-16T00:09:07.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4592e-05 2024-09-16T00:09:07.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686521+0000", 2024-09-16T00:09:07.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9808000000000001e-05 2024-09-16T00:09:07.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686534+0000", 2024-09-16T00:09:07.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2561000000000001e-05 2024-09-16T00:09:07.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686609+0000", 2024-09-16T00:09:07.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5315999999999995e-05 2024-09-16T00:09:07.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686820+0000", 2024-09-16T00:09:07.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00021137900000000001 2024-09-16T00:09:07.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:07.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686930+0000", 2024-09-16T00:09:07.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110158 2024-09-16T00:09:07.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.687631+0000", 2024-09-16T00:09:07.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000700393 2024-09-16T00:09:07.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.688087+0000", 2024-09-16T00:09:07.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000456432 2024-09-16T00:09:07.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.688122+0000", 2024-09-16T00:09:07.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4944000000000003e-05 2024-09-16T00:09:07.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.688161+0000", 2024-09-16T00:09:07.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8683000000000002e-05 2024-09-16T00:09:07.436 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.436 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.436 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.436 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:2315 3.1f 3:f84a2a2d:::10000000c60.00000000:head [write 0~648229] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T00:09:07.436 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.440012+0000", 2024-09-16T00:09:07.436 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.8659914959999999, 2024-09-16T00:09:07.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038521150000000001, 2024-09-16T00:09:07.436 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.437 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.437 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.437 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:07.437 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:07.437 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:07.437 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 2315 2024-09-16T00:09:07.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.437 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440012+0000", 2024-09-16T00:09:07.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440012+0000", 2024-09-16T00:09:07.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440014+0000", 2024-09-16T00:09:07.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5689999999999999e-06 2024-09-16T00:09:07.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440415+0000", 2024-09-16T00:09:07.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00040101999999999998 2024-09-16T00:09:07.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440434+0000", 2024-09-16T00:09:07.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8833000000000001e-05 2024-09-16T00:09:07.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440444+0000", 2024-09-16T00:09:07.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0552e-05 2024-09-16T00:09:07.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440508+0000", 2024-09-16T00:09:07.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4156e-05 2024-09-16T00:09:07.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440610+0000", 2024-09-16T00:09:07.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101584 2024-09-16T00:09:07.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:07.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440698+0000", 2024-09-16T00:09:07.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7824999999999993e-05 2024-09-16T00:09:07.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.443176+0000", 2024-09-16T00:09:07.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024776379999999999 2024-09-16T00:09:07.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.443789+0000", 2024-09-16T00:09:07.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00061301699999999999 2024-09-16T00:09:07.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.443820+0000", 2024-09-16T00:09:07.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1392e-05 2024-09-16T00:09:07.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.443864+0000", 2024-09-16T00:09:07.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4529e-05 2024-09-16T00:09:07.442 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.442 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.442 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.442 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.442 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.443 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:07.444 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:09:07.445 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:07.443+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:07.545 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:07.545+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:07.599 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:07.599 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:07.600 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:07.600 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:07.609 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:09:07.653 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:07.652+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:07.716 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:07.716 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:07.716 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:07.716 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:07.716 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:07.725 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:09:07.747 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:07.747+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:07.848 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:07.848+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:07.877 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:22 2.e e22/19 2:717a0223:::608.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:09:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 89.259621291000002, 2024-09-16T00:09:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042195870000000003, 2024-09-16T00:09:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:09:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:09:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616426+0000", 2024-09-16T00:09:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.037e-06 2024-09-16T00:09:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616435+0000", 2024-09-16T00:09:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6440000000000006e-06 2024-09-16T00:09:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616447+0000", 2024-09-16T00:09:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2466000000000001e-05 2024-09-16T00:09:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616456+0000", 2024-09-16T00:09:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8799999999999997e-06 2024-09-16T00:09:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616517+0000", 2024-09-16T00:09:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1171000000000001e-05 2024-09-16T00:09:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616540+0000", 2024-09-16T00:09:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2991000000000001e-05 2024-09-16T00:09:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620599+0000", 2024-09-16T00:09:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040586140000000003 2024-09-16T00:09:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620645+0000", 2024-09-16T00:09:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5784000000000001e-05 2024-09-16T00:09:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:09:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 89.259531561000003, 2024-09-16T00:09:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044836249999999998, 2024-09-16T00:09:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:09:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:09:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:09:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616518+0000", 2024-09-16T00:09:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3699999999999999e-06 2024-09-16T00:09:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:09:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1479999999999997e-06 2024-09-16T00:09:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616534+0000", 2024-09-16T00:09:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0641e-05 2024-09-16T00:09:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616541+0000", 2024-09-16T00:09:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3339999999999996e-06 2024-09-16T00:09:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617318+0000", 2024-09-16T00:09:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00077694799999999996 2024-09-16T00:09:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617409+0000", 2024-09-16T00:09:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0740000000000002e-05 2024-09-16T00:09:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620998+0000", 2024-09-16T00:09:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035894439999999998 2024-09-16T00:09:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:09:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 89.259483657999994, 2024-09-16T00:09:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041597320000000002, 2024-09-16T00:09:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:09:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:09:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:09:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:09:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:09:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616566+0000", 2024-09-16T00:09:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6979999999999998e-06 2024-09-16T00:09:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616574+0000", 2024-09-16T00:09:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7759999999999997e-06 2024-09-16T00:09:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616580+0000", 2024-09-16T00:09:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:09:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616800+0000", 2024-09-16T00:09:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022016700000000001 2024-09-16T00:09:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616904+0000", 2024-09-16T00:09:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104579 2024-09-16T00:09:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620722+0000", 2024-09-16T00:09:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003818133 2024-09-16T00:09:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:09:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 89.259232972999996, 2024-09-16T00:09:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038451570000000001, 2024-09-16T00:09:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:09:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:09:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616814+0000", 2024-09-16T00:09:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9299999999999997e-07 2024-09-16T00:09:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616824+0000", 2024-09-16T00:09:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6600000000000007e-06 2024-09-16T00:09:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616835+0000", 2024-09-16T00:09:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0713000000000001e-05 2024-09-16T00:09:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616841+0000", 2024-09-16T00:09:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.934e-06 2024-09-16T00:09:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616948+0000", 2024-09-16T00:09:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010631100000000001 2024-09-16T00:09:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616967+0000", 2024-09-16T00:09:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9598000000000001e-05 2024-09-16T00:09:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620619+0000", 2024-09-16T00:09:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036515810000000001 2024-09-16T00:09:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620659+0000", 2024-09-16T00:09:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9666999999999998e-05 2024-09-16T00:09:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:09:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 89.259186021000005, 2024-09-16T00:09:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038240129999999998, 2024-09-16T00:09:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:09:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:09:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616861+0000", 2024-09-16T00:09:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0200000000000003e-07 2024-09-16T00:09:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616864+0000", 2024-09-16T00:09:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5090000000000002e-06 2024-09-16T00:09:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616871+0000", 2024-09-16T00:09:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9249999999999998e-06 2024-09-16T00:09:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616877+0000", 2024-09-16T00:09:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.981e-06 2024-09-16T00:09:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617168+0000", 2024-09-16T00:09:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029035699999999998 2024-09-16T00:09:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617184+0000", 2024-09-16T00:09:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6005999999999999e-05 2024-09-16T00:09:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620661+0000", 2024-09-16T00:09:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034775330000000001 2024-09-16T00:09:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620684+0000", 2024-09-16T00:09:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3300000000000001e-05 2024-09-16T00:09:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:09:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:09:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 89.258730577999998, 2024-09-16T00:09:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045415610000000004, 2024-09-16T00:09:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:09:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:09:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:09:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8800000000000002e-07 2024-09-16T00:09:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617321+0000", 2024-09-16T00:09:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6979999999999997e-06 2024-09-16T00:09:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617330+0000", 2024-09-16T00:09:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6729999999999998e-06 2024-09-16T00:09:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617336+0000", 2024-09-16T00:09:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2110000000000003e-06 2024-09-16T00:09:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617394+0000", 2024-09-16T00:09:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8087999999999997e-05 2024-09-16T00:09:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617412+0000", 2024-09-16T00:09:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8099999999999999e-05 2024-09-16T00:09:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621818+0000", 2024-09-16T00:09:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044061209999999998 2024-09-16T00:09:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621857+0000", 2024-09-16T00:09:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9082000000000003e-05 2024-09-16T00:09:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:23 2.e e22/19 2:717a0223:::608.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:09:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 89.257834556999995, 2024-09-16T00:09:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036198279999999998, 2024-09-16T00:09:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:09:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:09:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618213+0000", 2024-09-16T00:09:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:09:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618218+0000", 2024-09-16T00:09:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4099999999999999e-06 2024-09-16T00:09:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618227+0000", 2024-09-16T00:09:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5009999999999993e-06 2024-09-16T00:09:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618234+0000", 2024-09-16T00:09:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:09:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618307+0000", 2024-09-16T00:09:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3421000000000001e-05 2024-09-16T00:09:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618328+0000", 2024-09-16T00:09:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0919000000000001e-05 2024-09-16T00:09:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621804+0000", 2024-09-16T00:09:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034756560000000001 2024-09-16T00:09:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621832+0000", 2024-09-16T00:09:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7966999999999998e-05 2024-09-16T00:09:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:243 3.24 3:25048551:::100000002f0.00000000:head [write 0~1428] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.799025+0000", 2024-09-16T00:09:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 11.077021222999999, 2024-09-16T00:09:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042804829999999999, 2024-09-16T00:09:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 243 2024-09-16T00:09:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799025+0000", 2024-09-16T00:09:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799025+0000", 2024-09-16T00:09:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799026+0000", 2024-09-16T00:09:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9099999999999991e-07 2024-09-16T00:09:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799032+0000", 2024-09-16T00:09:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:09:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799044+0000", 2024-09-16T00:09:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1861999999999999e-05 2024-09-16T00:09:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799049+0000", 2024-09-16T00:09:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6570000000000002e-06 2024-09-16T00:09:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799095+0000", 2024-09-16T00:09:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5479999999999998e-05 2024-09-16T00:09:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799175+0000", 2024-09-16T00:09:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0181999999999999e-05 2024-09-16T00:09:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799252+0000", 2024-09-16T00:09:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6822000000000004e-05 2024-09-16T00:09:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799855+0000", 2024-09-16T00:09:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00060294199999999997 2024-09-16T00:09:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803230+0000", 2024-09-16T00:09:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033752719999999999 2024-09-16T00:09:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803263+0000", 2024-09-16T00:09:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3339e-05 2024-09-16T00:09:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803306+0000", 2024-09-16T00:09:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2330999999999998e-05 2024-09-16T00:09:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:324 3.29 3:960c7fc3:::10000000355.00000000:head [write 0~1506] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.034622+0000", 2024-09-16T00:09:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 10.841424202000001, 2024-09-16T00:09:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003641746, 2024-09-16T00:09:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 324 2024-09-16T00:09:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034622+0000", 2024-09-16T00:09:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034622+0000", 2024-09-16T00:09:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034623+0000", 2024-09-16T00:09:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1689999999999999e-06 2024-09-16T00:09:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034629+0000", 2024-09-16T00:09:07.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4859999999999997e-06 2024-09-16T00:09:07.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034639+0000", 2024-09-16T00:09:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0142999999999999e-05 2024-09-16T00:09:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034644+0000", 2024-09-16T00:09:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1159999999999998e-06 2024-09-16T00:09:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034693+0000", 2024-09-16T00:09:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9228999999999999e-05 2024-09-16T00:09:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034813+0000", 2024-09-16T00:09:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012003200000000001 2024-09-16T00:09:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034940+0000", 2024-09-16T00:09:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012655699999999999 2024-09-16T00:09:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.036244+0000", 2024-09-16T00:09:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001304326 2024-09-16T00:09:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.038175+0000", 2024-09-16T00:09:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001930768 2024-09-16T00:09:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.038225+0000", 2024-09-16T00:09:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0046000000000002e-05 2024-09-16T00:09:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.038264+0000", 2024-09-16T00:09:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8874000000000001e-05 2024-09-16T00:09:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:411 3.29 3:96aa9846:::100000003bf.00000000:head [write 0~1161] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.363361+0000", 2024-09-16T00:09:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 10.512685152, 2024-09-16T00:09:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038090889999999999, 2024-09-16T00:09:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 411 2024-09-16T00:09:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363361+0000", 2024-09-16T00:09:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363361+0000", 2024-09-16T00:09:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363363+0000", 2024-09-16T00:09:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.421e-06 2024-09-16T00:09:07.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363369+0000", 2024-09-16T00:09:07.935 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7270000000000001e-06 2024-09-16T00:09:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363382+0000", 2024-09-16T00:09:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2700999999999999e-05 2024-09-16T00:09:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363388+0000", 2024-09-16T00:09:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0009999999999998e-06 2024-09-16T00:09:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363438+0000", 2024-09-16T00:09:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9789000000000001e-05 2024-09-16T00:09:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363562+0000", 2024-09-16T00:09:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000123798 2024-09-16T00:09:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363639+0000", 2024-09-16T00:09:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7074000000000001e-05 2024-09-16T00:09:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.364153+0000", 2024-09-16T00:09:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00051393999999999995 2024-09-16T00:09:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.367097+0000", 2024-09-16T00:09:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00294414 2024-09-16T00:09:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.367145+0000", 2024-09-16T00:09:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8591999999999998e-05 2024-09-16T00:09:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.367170+0000", 2024-09-16T00:09:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4905999999999998e-05 2024-09-16T00:09:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:53 2.14 e58/46 2:2e2fa760:::200.00000001:head v 58'16, mlcod=57'14)", 2024-09-16T00:09:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.453518+0000", 2024-09-16T00:09:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 10.422528625, 2024-09-16T00:09:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036209150000000002, 2024-09-16T00:09:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.453518+0000", 2024-09-16T00:09:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.453518+0000", 2024-09-16T00:09:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.453519+0000", 2024-09-16T00:09:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.277e-06 2024-09-16T00:09:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.454703+0000", 2024-09-16T00:09:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011842440000000001 2024-09-16T00:09:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.454724+0000", 2024-09-16T00:09:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0835999999999998e-05 2024-09-16T00:09:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.454734+0000", 2024-09-16T00:09:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0299000000000001e-05 2024-09-16T00:09:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.454802+0000", 2024-09-16T00:09:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7127999999999998e-05 2024-09-16T00:09:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.454820+0000", 2024-09-16T00:09:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8858000000000001e-05 2024-09-16T00:09:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.457101+0000", 2024-09-16T00:09:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022802539999999998 2024-09-16T00:09:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.457139+0000", 2024-09-16T00:09:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8019000000000003e-05 2024-09-16T00:09:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:714 3.22 e59/33 3:453cbc5d:::10000000536.00000000:head v 59'18, mlcod=59'16)", 2024-09-16T00:09:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.356842+0000", 2024-09-16T00:09:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.5192039600000005, 2024-09-16T00:09:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032768649999999999, 2024-09-16T00:09:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356842+0000", 2024-09-16T00:09:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356842+0000", 2024-09-16T00:09:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356844+0000", 2024-09-16T00:09:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4219999999999999e-06 2024-09-16T00:09:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356853+0000", 2024-09-16T00:09:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7760000000000003e-06 2024-09-16T00:09:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356871+0000", 2024-09-16T00:09:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8114e-05 2024-09-16T00:09:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356881+0000", 2024-09-16T00:09:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0233e-05 2024-09-16T00:09:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356949+0000", 2024-09-16T00:09:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7780000000000005e-05 2024-09-16T00:09:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356972+0000", 2024-09-16T00:09:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3144999999999999e-05 2024-09-16T00:09:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360073+0000", 2024-09-16T00:09:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031013170000000001 2024-09-16T00:09:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360119+0000", 2024-09-16T00:09:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6078000000000002e-05 2024-09-16T00:09:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:55 2.14 e59/46 2:2e2fa760:::200.00000001:head v 59'17, mlcod=57'15)", 2024-09-16T00:09:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.643358+0000", 2024-09-16T00:09:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.2326883300000002, 2024-09-16T00:09:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031410520000000001, 2024-09-16T00:09:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.643358+0000", 2024-09-16T00:09:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.643358+0000", 2024-09-16T00:09:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.643360+0000", 2024-09-16T00:09:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.136e-06 2024-09-16T00:09:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.644215+0000", 2024-09-16T00:09:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085427799999999998 2024-09-16T00:09:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.644236+0000", 2024-09-16T00:09:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1693000000000001e-05 2024-09-16T00:09:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.644248+0000", 2024-09-16T00:09:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1958e-05 2024-09-16T00:09:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.644310+0000", 2024-09-16T00:09:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1853000000000006e-05 2024-09-16T00:09:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.644327+0000", 2024-09-16T00:09:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7258999999999999e-05 2024-09-16T00:09:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.646455+0000", 2024-09-16T00:09:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021274710000000001 2024-09-16T00:09:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.646499+0000", 2024-09-16T00:09:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4403999999999997e-05 2024-09-16T00:09:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:72 2.14 e61/46 2:2e2fa760:::200.00000001:head v 61'21, mlcod=61'19)", 2024-09-16T00:09:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:01.672924+0000", 2024-09-16T00:09:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.203122284, 2024-09-16T00:09:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003655784, 2024-09-16T00:09:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.672924+0000", 2024-09-16T00:09:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.672924+0000", 2024-09-16T00:09:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.672925+0000", 2024-09-16T00:09:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0189999999999999e-06 2024-09-16T00:09:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674096+0000", 2024-09-16T00:09:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011709660000000001 2024-09-16T00:09:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674112+0000", 2024-09-16T00:09:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6149000000000001e-05 2024-09-16T00:09:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674120+0000", 2024-09-16T00:09:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.892e-06 2024-09-16T00:09:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674179+0000", 2024-09-16T00:09:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8876999999999999e-05 2024-09-16T00:09:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674197+0000", 2024-09-16T00:09:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8048999999999999e-05 2024-09-16T00:09:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.676539+0000", 2024-09-16T00:09:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023414830000000001 2024-09-16T00:09:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.676580+0000", 2024-09-16T00:09:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1349e-05 2024-09-16T00:09:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:86 2.1b 2:ddb4e4d7:::200.00000002:head [write 368603~2220227 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.644964+0000", 2024-09-16T00:09:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.2310822110000004, 2024-09-16T00:09:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011714478, 2024-09-16T00:09:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.644964+0000", 2024-09-16T00:09:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.644964+0000", 2024-09-16T00:09:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.644966+0000", 2024-09-16T00:09:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4789999999999999e-06 2024-09-16T00:09:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.648979+0000", 2024-09-16T00:09:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040136900000000003 2024-09-16T00:09:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.648993+0000", 2024-09-16T00:09:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3129000000000001e-05 2024-09-16T00:09:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649000+0000", 2024-09-16T00:09:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5700000000000004e-06 2024-09-16T00:09:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649055+0000", 2024-09-16T00:09:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4914999999999999e-05 2024-09-16T00:09:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649131+0000", 2024-09-16T00:09:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6055000000000003e-05 2024-09-16T00:09:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649258+0000", 2024-09-16T00:09:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000127229 2024-09-16T00:09:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.971 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656218+0000", 2024-09-16T00:09:07.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069600469999999996 2024-09-16T00:09:07.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.971 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656602+0000", 2024-09-16T00:09:07.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038380700000000001 2024-09-16T00:09:07.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.972 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656638+0000", 2024-09-16T00:09:07.972 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5725000000000001e-05 2024-09-16T00:09:07.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.972 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.973 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.973 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656679+0000", 2024-09-16T00:09:07.973 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0831999999999998e-05 2024-09-16T00:09:07.973 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.973 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.973 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.973 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.974 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.974 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:87 2.1b 2:ddb4e4d7:::200.00000002:head [write 2588830~1400 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:07.974 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649020+0000", 2024-09-16T00:09:07.974 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.2270260200000003, 2024-09-16T00:09:07.974 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0083707460000000001, 2024-09-16T00:09:07.974 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.974 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.974 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.974 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.974 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.974 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.974 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649020+0000", 2024-09-16T00:09:07.974 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.975 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.975 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649020+0000", 2024-09-16T00:09:07.975 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.975 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.975 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649021+0000", 2024-09-16T00:09:07.975 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6000000000000002e-07 2024-09-16T00:09:07.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.975 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.976 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649034+0000", 2024-09-16T00:09:07.976 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2627e-05 2024-09-16T00:09:07.976 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.976 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.976 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.976 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649045+0000", 2024-09-16T00:09:07.976 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0744000000000001e-05 2024-09-16T00:09:07.976 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.977 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.977 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649051+0000", 2024-09-16T00:09:07.977 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5459999999999997e-06 2024-09-16T00:09:07.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.977 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.977 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654369+0000", 2024-09-16T00:09:07.977 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053179519999999999 2024-09-16T00:09:07.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.977 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.978 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654435+0000", 2024-09-16T00:09:07.978 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6175999999999998e-05 2024-09-16T00:09:07.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.978 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.978 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:07.978 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654525+0000", 2024-09-16T00:09:07.978 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9856000000000002e-05 2024-09-16T00:09:07.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.978 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.978 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.978 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656225+0000", 2024-09-16T00:09:07.979 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001699386 2024-09-16T00:09:07.979 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.979 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.979 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.979 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657339+0000", 2024-09-16T00:09:07.979 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011145250000000001 2024-09-16T00:09:07.979 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.979 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.979 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.979 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657366+0000", 2024-09-16T00:09:07.979 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7172e-05 2024-09-16T00:09:07.979 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.979 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.980 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.980 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657391+0000", 2024-09-16T00:09:07.980 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4902e-05 2024-09-16T00:09:07.980 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.980 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.980 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.980 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.980 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.980 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:88 2.1b 2:ddb4e4d7:::200.00000002:head [write 2590230~1400 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:07.980 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649060+0000", 2024-09-16T00:09:07.980 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.2269862180000004, 2024-09-16T00:09:07.980 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0088748730000000001, 2024-09-16T00:09:07.981 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.981 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.981 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.981 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.981 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.981 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.981 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649060+0000", 2024-09-16T00:09:07.981 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.981 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.981 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.981 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.982 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649060+0000", 2024-09-16T00:09:07.982 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.982 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.982 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.982 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649061+0000", 2024-09-16T00:09:07.982 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2300000000000001e-07 2024-09-16T00:09:07.982 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.982 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.982 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649064+0000", 2024-09-16T00:09:07.982 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3160000000000001e-06 2024-09-16T00:09:07.982 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.983 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.983 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649070+0000", 2024-09-16T00:09:07.983 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9880000000000001e-06 2024-09-16T00:09:07.983 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.983 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.983 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.983 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649074+0000", 2024-09-16T00:09:07.983 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4329999999999997e-06 2024-09-16T00:09:07.983 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.983 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.983 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.983 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654769+0000", 2024-09-16T00:09:07.983 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0056944500000000002 2024-09-16T00:09:07.984 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.984 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.984 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.984 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654875+0000", 2024-09-16T00:09:07.984 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000106183 2024-09-16T00:09:07.984 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.984 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.984 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:07.984 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654994+0000", 2024-09-16T00:09:07.984 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011867900000000001 2024-09-16T00:09:07.984 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.984 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.984 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.985 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657697+0000", 2024-09-16T00:09:07.985 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027033920000000002 2024-09-16T00:09:07.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.985 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.985 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.985 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657882+0000", 2024-09-16T00:09:07.985 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018478900000000001 2024-09-16T00:09:07.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.985 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.985 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.985 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657913+0000", 2024-09-16T00:09:07.985 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1306000000000001e-05 2024-09-16T00:09:07.986 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.986 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.986 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.986 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657935+0000", 2024-09-16T00:09:07.986 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1814000000000002e-05 2024-09-16T00:09:07.986 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.986 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.986 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.986 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.986 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.986 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:89 2.1b 2:ddb4e4d7:::200.00000002:head [write 2591630~1401 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:07.986 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:07.986 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.2269659590000002, 2024-09-16T00:09:07.987 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011802609, 2024-09-16T00:09:07.987 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.987 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.987 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.987 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.987 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.987 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.987 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:07.987 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.987 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.987 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.987 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.987 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:07.988 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.988 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.988 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.988 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649081+0000", 2024-09-16T00:09:07.988 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7100000000000002e-07 2024-09-16T00:09:07.988 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.988 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:07.988 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649089+0000", 2024-09-16T00:09:07.988 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7640000000000005e-06 2024-09-16T00:09:07.988 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.988 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:07.989 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649094+0000", 2024-09-16T00:09:07.989 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6219999999999997e-06 2024-09-16T00:09:07.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.989 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.989 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:07.989 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649099+0000", 2024-09-16T00:09:07.989 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6970000000000001e-06 2024-09-16T00:09:07.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.989 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.989 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:07.989 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655288+0000", 2024-09-16T00:09:07.989 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061885569999999999 2024-09-16T00:09:07.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.990 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:07.990 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655365+0000", 2024-09-16T00:09:07.990 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7723000000000001e-05 2024-09-16T00:09:07.990 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.990 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:07.990 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655483+0000", 2024-09-16T00:09:07.991 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117895 2024-09-16T00:09:07.991 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.991 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.991 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:07.991 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657758+0000", 2024-09-16T00:09:07.991 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022752369999999998 2024-09-16T00:09:07.992 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.992 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.992 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:07.992 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660803+0000", 2024-09-16T00:09:07.992 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030441499999999998 2024-09-16T00:09:07.992 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.993 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.993 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:07.997 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660850+0000", 2024-09-16T00:09:07.997 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7017000000000001e-05 2024-09-16T00:09:07.997 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.997 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.997 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:07.997 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660883+0000", 2024-09-16T00:09:07.998 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3476000000000003e-05 2024-09-16T00:09:07.998 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.998 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:07.998 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:07.998 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.998 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.998 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:90 2.1b 2:ddb4e4d7:::200.00000002:head [write 2593031~1404 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:07.998 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:07.998 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.226928054, 2024-09-16T00:09:07.998 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011882886, 2024-09-16T00:09:07.998 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:07.998 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:07.998 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:07.999 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:07.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:07.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:07.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:07.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:07.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:07.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:07.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:07.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:07.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649119+0000", 2024-09-16T00:09:08.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:09:08.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:08.000 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649125+0000", 2024-09-16T00:09:08.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1650000000000003e-06 2024-09-16T00:09:08.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:08.000 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649134+0000", 2024-09-16T00:09:08.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5669999999999995e-06 2024-09-16T00:09:08.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.001 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:08.001 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649141+0000", 2024-09-16T00:09:08.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9040000000000003e-06 2024-09-16T00:09:08.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.001 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:08.001 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655796+0000", 2024-09-16T00:09:08.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066553649999999999 2024-09-16T00:09:08.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.001 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:08.002 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655868+0000", 2024-09-16T00:09:08.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2219999999999996e-05 2024-09-16T00:09:08.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.002 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:08.002 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655986+0000", 2024-09-16T00:09:08.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117457 2024-09-16T00:09:08.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.002 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:08.002 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657794+0000", 2024-09-16T00:09:08.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001807894 2024-09-16T00:09:08.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:08.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660890+0000", 2024-09-16T00:09:08.003 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030958449999999998 2024-09-16T00:09:08.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:08.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660966+0000", 2024-09-16T00:09:08.003 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6382000000000002e-05 2024-09-16T00:09:08.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:08.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.661001+0000", 2024-09-16T00:09:08.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5349000000000003e-05 2024-09-16T00:09:08.004 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:08.004 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:08.004 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:08.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.004 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:2397 3.29 3:948abdad:::10000000cb5.00000000:head [write 0~1373] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T00:09:08.004 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.655571+0000", 2024-09-16T00:09:08.004 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.2204757690000001, 2024-09-16T00:09:08.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052117400000000003, 2024-09-16T00:09:08.004 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:08.005 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:08.005 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:08.005 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:08.005 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:08.005 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:08.005 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 2397 2024-09-16T00:09:08.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.005 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:08.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:08.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655571+0000", 2024-09-16T00:09:08.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:08.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:08.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655571+0000", 2024-09-16T00:09:08.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:08.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:08.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655573+0000", 2024-09-16T00:09:08.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0789999999999999e-06 2024-09-16T00:09:08.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:08.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655580+0000", 2024-09-16T00:09:08.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7859999999999999e-06 2024-09-16T00:09:08.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.009 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:08.009 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655592+0000", 2024-09-16T00:09:08.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2541000000000001e-05 2024-09-16T00:09:08.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.009 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:08.009 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655601+0000", 2024-09-16T00:09:08.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.721e-06 2024-09-16T00:09:08.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.009 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:08.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655663+0000", 2024-09-16T00:09:08.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2287999999999997e-05 2024-09-16T00:09:08.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.010 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:08.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655765+0000", 2024-09-16T00:09:08.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000102212 2024-09-16T00:09:08.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.010 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:08.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655893+0000", 2024-09-16T00:09:08.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012742499999999999 2024-09-16T00:09:08.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.011 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:08.011 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657022+0000", 2024-09-16T00:09:08.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011293340000000001 2024-09-16T00:09:08.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.011 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:08.011 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660686+0000", 2024-09-16T00:09:08.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036637409999999999 2024-09-16T00:09:08.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.011 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:08.011 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660737+0000", 2024-09-16T00:09:08.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1711e-05 2024-09-16T00:09:08.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:08.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:08.012 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:08.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660782+0000", 2024-09-16T00:09:08.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4901999999999998e-05 2024-09-16T00:09:08.012 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:08.012 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:08.012 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:08.012 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:08.012 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:08.012 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:08.014 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:08.013+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:08.015 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:09:08.035 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T00:09:08.035 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:09:08.036 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-16T00:09:08.036 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T00:09:08.036 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T00:09:08.037 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-16T00:09:08.052 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T00:09:08.060 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:09:08.125 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:08.124+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:08.146 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:08.146 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:08.146 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:08.147 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:08.149 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:08.149 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T00:09:08.149 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:08.155 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:09:08.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.182+0000 7f3353d03640 1 Processor -- start 2024-09-16T00:09:08.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.182+0000 7f3353d03640 1 -- start start 2024-09-16T00:09:08.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.183+0000 7f3353d03640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f334c0d0490 0x7f334c0b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.183+0000 7f3353d03640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f334c0d0870 0x7f334c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.183+0000 7f3353d03640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f334c0b3180 con 0x7f334c0b3430 2024-09-16T00:09:08.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.183+0000 7f3353d03640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f334c15b940 con 0x7f334c0d0870 2024-09-16T00:09:08.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.183+0000 7f3353d03640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f334c0b3bb0 con 0x7f334c0d0490 2024-09-16T00:09:08.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.183+0000 7f33522a0640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f334c0b3430 0x7f334c1546a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46722/0 (socket says 172.21.15.139:46722) 2024-09-16T00:09:08.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.183+0000 7f33522a0640 1 -- 172.21.15.139:0/3865531446 learned_addr learned my addr 172.21.15.139:0/3865531446 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:08.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.183+0000 7f335129e640 1 --2- 172.21.15.139:0/3865531446 >> v2:172.21.15.164:3300/0 conn(0x7f334c0d0870 0x7f334c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.183+0000 7f3351a9f640 1 --2- 172.21.15.139:0/3865531446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f334c0d0490 0x7f334c0b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.183+0000 7f3350a9d640 1 -- 172.21.15.139:0/3865531446 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3080083515 0 0) 0x7f334c0b3180 con 0x7f334c0b3430 2024-09-16T00:09:08.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.183+0000 7f3350a9d640 1 -- 172.21.15.139:0/3865531446 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3338003520 con 0x7f334c0b3430 2024-09-16T00:09:08.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.183+0000 7f3351a9f640 1 -- 172.21.15.139:0/3865531446 >> v2:172.21.15.164:3300/0 conn(0x7f334c0d0870 msgr2=0x7f334c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:08.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.183+0000 7f3351a9f640 1 --2- 172.21.15.139:0/3865531446 >> v2:172.21.15.164:3300/0 conn(0x7f334c0d0870 0x7f334c0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.183+0000 7f3351a9f640 1 -- 172.21.15.139:0/3865531446 >> v1:172.21.15.139:6789/0 conn(0x7f334c0b3430 legacy=0x7f334c1546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.183+0000 7f3351a9f640 1 -- 172.21.15.139:0/3865531446 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c15adc0 con 0x7f334c0d0490 2024-09-16T00:09:08.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.183+0000 7f335129e640 1 --2- 172.21.15.139:0/3865531446 >> v2:172.21.15.164:3300/0 conn(0x7f334c0d0870 0x7f334c0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:08.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.184+0000 7f3351a9f640 1 --2- 172.21.15.139:0/3865531446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f334c0d0490 0x7f334c0b2c40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f33480097a0 tx=0x7f334802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4987d769c4809304 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:08.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.184+0000 7f3350a9d640 1 -- 172.21.15.139:0/3865531446 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348037070 con 0x7f334c0d0490 2024-09-16T00:09:08.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.184+0000 7f3350a9d640 1 -- 172.21.15.139:0/3865531446 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3348032440 con 0x7f334c0d0490 2024-09-16T00:09:08.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.184+0000 7f3350a9d640 1 -- 172.21.15.139:0/3865531446 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33480327f0 con 0x7f334c0d0490 2024-09-16T00:09:08.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.185+0000 7f3353d03640 1 -- 172.21.15.139:0/3865531446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f334c0d0490 msgr2=0x7f334c0b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.185+0000 7f3353d03640 1 --2- 172.21.15.139:0/3865531446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f334c0d0490 0x7f334c0b2c40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f33480097a0 tx=0x7f334802a210 comp rx=0 tx=0).stop 2024-09-16T00:09:08.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.185+0000 7f3353d03640 1 -- 172.21.15.139:0/3865531446 shutdown_connections 2024-09-16T00:09:08.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.185+0000 7f3353d03640 1 --2- 172.21.15.139:0/3865531446 >> v2:172.21.15.164:3300/0 conn(0x7f334c0d0870 0x7f334c0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.185+0000 7f3353d03640 1 --2- 172.21.15.139:0/3865531446 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f334c0d0490 0x7f334c0b2c40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.185+0000 7f3353d03640 1 -- 172.21.15.139:0/3865531446 >> 172.21.15.139:0/3865531446 conn(0x7f334c06c520 msgr2=0x7f334c06c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:08.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.185+0000 7f3353d03640 1 -- 172.21.15.139:0/3865531446 shutdown_connections 2024-09-16T00:09:08.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.185+0000 7f3353d03640 1 -- 172.21.15.139:0/3865531446 wait complete. 2024-09-16T00:09:08.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.186+0000 7fa63b827640 1 Processor -- start 2024-09-16T00:09:08.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.186+0000 7fa63b827640 1 -- start start 2024-09-16T00:09:08.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.186+0000 7f3353d03640 1 Processor -- start 2024-09-16T00:09:08.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.186+0000 7f3353d03640 1 -- start start 2024-09-16T00:09:08.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7fa63b827640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa634154ae0 0x7fa63415a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7fa63b827640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa63406e940 0x7fa634164ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7fa63b827640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6340c8d90 con 0x7fa63406d0c0 2024-09-16T00:09:08.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7fa63b827640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fa63406cc30 con 0x7fa63406e940 2024-09-16T00:09:08.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7fa63b827640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa6340701b0 con 0x7fa634154ae0 2024-09-16T00:09:08.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7fa6395c3640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa634154ae0 0x7fa63415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7fa638dc2640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa63406d0c0 0x7fa6340ce140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:41608/0 (socket says 172.21.15.139:41608) 2024-09-16T00:09:08.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7fa638dc2640 1 -- 172.21.15.139:0/3932292511 learned_addr learned my addr 172.21.15.139:0/3932292511 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:08.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7f3353d03640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f334c0b3430 0x7f334c1509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7fa639dc4640 1 --2- 172.21.15.139:0/3932292511 >> v2:172.21.15.164:3300/0 conn(0x7fa63406e940 0x7fa634164ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7f3353d03640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f334c0d0490 0x7f334c0cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7f3353d03640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f334c0d0870 0x7f334c152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7f3353d03640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f334c15b660 con 0x7f334c0b3430 2024-09-16T00:09:08.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7f3353d03640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f334c15b380 con 0x7f334c0d0490 2024-09-16T00:09:08.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7f3353d03640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f334c15a980 con 0x7f334c0d0870 2024-09-16T00:09:08.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7f3351a9f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f334c0b3430 0x7f334c1509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7f3351a9f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f334c0b3430 0x7f334c1509b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58710/0 (socket says 172.21.15.139:58710) 2024-09-16T00:09:08.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7f3351a9f640 1 -- 172.21.15.139:0/3527454115 learned_addr learned my addr 172.21.15.139:0/3527454115 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:08.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7f335129e640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f334c0d0490 0x7f334c0cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7f33522a0640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f334c0d0870 0x7f334c152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7fa6395c3640 1 -- 172.21.15.139:0/3932292511 >> v2:172.21.15.164:3300/0 conn(0x7fa63406e940 msgr2=0x7fa634164ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:08.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7fa6395c3640 1 --2- 172.21.15.139:0/3932292511 >> v2:172.21.15.164:3300/0 conn(0x7fa63406e940 0x7fa634164ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7fa6395c3640 1 -- 172.21.15.139:0/3932292511 >> v1:172.21.15.164:6789/0 conn(0x7fa63406d0c0 legacy=0x7fa6340ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7fa6395c3640 1 -- 172.21.15.139:0/3932292511 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa634070400 con 0x7fa634154ae0 2024-09-16T00:09:08.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.187+0000 7fa62bfff640 1 -- 172.21.15.139:0/3932292511 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1487972167 0 0) 0x7fa6340c8d90 con 0x7fa63406d0c0 2024-09-16T00:09:08.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.188+0000 7f3351a9f640 1 -- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f334c0d0870 msgr2=0x7f334c152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.188+0000 7f3351a9f640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f334c0d0870 0x7f334c152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.188+0000 7f3351a9f640 1 -- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f334c0d0490 msgr2=0x7f334c0cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.188+0000 7f3351a9f640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f334c0d0490 0x7f334c0cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.188+0000 7f3351a9f640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334c157200 con 0x7f334c0b3430 2024-09-16T00:09:08.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.188+0000 7f3351a9f640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f334c0b3430 0x7f334c1509b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3348035040 tx=0x7f33480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:08.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.188+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348037040 con 0x7f334c0b3430 2024-09-16T00:09:08.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.188+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3348042070 con 0x7f334c0b3430 2024-09-16T00:09:08.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.188+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f334c1574e0 con 0x7f334c0b3430 2024-09-16T00:09:08.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.188+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334803b930 con 0x7f334c0b3430 2024-09-16T00:09:08.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.188+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f334c157aa0 con 0x7f334c0b3430 2024-09-16T00:09:08.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.188+0000 7fa6395c3640 1 --2- 172.21.15.139:0/3932292511 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa634154ae0 0x7fa63415a710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa6300097a0 tx=0x7fa63002a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=650b16a9e1d106a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:08.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.188+0000 7fa62bfff640 1 -- 172.21.15.139:0/3932292511 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630037070 con 0x7fa634154ae0 2024-09-16T00:09:08.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.188+0000 7fa62bfff640 1 -- 172.21.15.139:0/3932292511 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa630032440 con 0x7fa634154ae0 2024-09-16T00:09:08.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.188+0000 7fa62bfff640 1 -- 172.21.15.139:0/3932292511 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630032810 con 0x7fa634154ae0 2024-09-16T00:09:08.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.189+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f334c0b2c40 con 0x7f334c0b3430 2024-09-16T00:09:08.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.189+0000 7fa63b827640 1 -- 172.21.15.139:0/3932292511 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa634154ae0 msgr2=0x7fa63415a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.189+0000 7fa63b827640 1 --2- 172.21.15.139:0/3932292511 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa634154ae0 0x7fa63415a710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa6300097a0 tx=0x7fa63002a230 comp rx=0 tx=0).stop 2024-09-16T00:09:08.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.189+0000 7fa63b827640 1 -- 172.21.15.139:0/3932292511 shutdown_connections 2024-09-16T00:09:08.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.189+0000 7fa63b827640 1 --2- 172.21.15.139:0/3932292511 >> v2:172.21.15.164:3300/0 conn(0x7fa63406e940 0x7fa634164ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.189+0000 7fa63b827640 1 --2- 172.21.15.139:0/3932292511 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa634154ae0 0x7fa63415a710 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.189+0000 7fa63b827640 1 -- 172.21.15.139:0/3932292511 >> 172.21.15.139:0/3932292511 conn(0x7fa63406c150 msgr2=0x7fa634067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:08.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.189+0000 7fa63b827640 1 -- 172.21.15.139:0/3932292511 shutdown_connections 2024-09-16T00:09:08.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.189+0000 7fa63b827640 1 -- 172.21.15.139:0/3932292511 wait complete. 2024-09-16T00:09:08.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.190+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f334803bad0 con 0x7f334c0b3430 2024-09-16T00:09:08.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.190+0000 7f33427fc640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f333006c2b0 0x7f333006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.191+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5915+0+0 (secure 0 0 0) 0x7f33480abbf0 con 0x7f334c0b3430 2024-09-16T00:09:08.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.191+0000 7f335129e640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f333006c2b0 0x7f333006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.191+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v64) ==== 54+0+16 (secure 0 0 0) 0x7f33480ae270 con 0x7f334c0b3430 2024-09-16T00:09:08.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.191+0000 7fa63b827640 1 Processor -- start 2024-09-16T00:09:08.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.191+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f334c153970 0x7f334c1528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.191+0000 7fa63b827640 1 -- start start 2024-09-16T00:09:08.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.191+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f334c0cc070 con 0x7f334c153970 2024-09-16T00:09:08.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.191+0000 7f33522a0640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f334c153970 0x7f334c1528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.191+0000 7f335129e640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f333006c2b0 0x7f333006e770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f333c009560 tx=0x7f333c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:08.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.191+0000 7f33522a0640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f334c153970 0x7f334c1528b0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:08.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa63b827640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa63406d0c0 0x7fa6340cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa63b827640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa63406e940 0x7fa6340c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa63b827640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa634154ae0 0x7fa6341f0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa63b827640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa634154ec0 con 0x7fa63406e940 2024-09-16T00:09:08.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa63b827640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa634070770 con 0x7fa634154ae0 2024-09-16T00:09:08.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa63b827640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa63406f9d0 con 0x7fa63406d0c0 2024-09-16T00:09:08.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa638dc2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa63406e940 0x7fa6340c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa6395c3640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa63406d0c0 0x7fa6340cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa638dc2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa63406e940 0x7fa6340c7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58724/0 (socket says 172.21.15.139:58724) 2024-09-16T00:09:08.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa638dc2640 1 -- 172.21.15.139:0/3638757848 learned_addr learned my addr 172.21.15.139:0/3638757848 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:08.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa639dc4640 1 --2- 172.21.15.139:0/3638757848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa634154ae0 0x7fa6341f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.203 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f334c0cc070 con 0x7f334c153970 2024-09-16T00:09:08.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa6395c3640 1 -- 172.21.15.139:0/3638757848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa634154ae0 msgr2=0x7fa6341f0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa6395c3640 1 --2- 172.21.15.139:0/3638757848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa634154ae0 0x7fa6341f0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa6395c3640 1 -- 172.21.15.139:0/3638757848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa63406e940 msgr2=0x7fa6340c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa6395c3640 1 --2- 172.21.15.139:0/3638757848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa63406e940 0x7fa6340c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa6395c3640 1 -- 172.21.15.139:0/3638757848 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa634156af0 con 0x7fa63406d0c0 2024-09-16T00:09:08.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa638dc2640 1 --2- 172.21.15.139:0/3638757848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa63406e940 0x7fa6340c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:08.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa6395c3640 1 --2- 172.21.15.139:0/3638757848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa63406d0c0 0x7fa6340cc890 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa634070330 tx=0x7fa6300046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:08.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.192+0000 7fa629ffb640 1 -- 172.21.15.139:0/3638757848 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630037050 con 0x7fa63406d0c0 2024-09-16T00:09:08.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.193+0000 7fa629ffb640 1 -- 172.21.15.139:0/3638757848 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa630038040 con 0x7fa63406d0c0 2024-09-16T00:09:08.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.193+0000 7fa63b827640 1 -- 172.21.15.139:0/3638757848 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa634156dd0 con 0x7fa63406d0c0 2024-09-16T00:09:08.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.193+0000 7fa629ffb640 1 -- 172.21.15.139:0/3638757848 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630004430 con 0x7fa63406d0c0 2024-09-16T00:09:08.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.193+0000 7fa63b827640 1 -- 172.21.15.139:0/3638757848 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa634157390 con 0x7fa63406d0c0 2024-09-16T00:09:08.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.194+0000 7fa63b827640 1 -- 172.21.15.139:0/3638757848 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6340c9ce0 con 0x7fa63406d0c0 2024-09-16T00:09:08.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.194+0000 7fa629ffb640 1 -- 172.21.15.139:0/3638757848 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa63003c6d0 con 0x7fa63406d0c0 2024-09-16T00:09:08.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.195+0000 7fa629ffb640 1 --2- 172.21.15.139:0/3638757848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa61806c2b0 0x7fa61806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.195+0000 7fa629ffb640 1 -- 172.21.15.139:0/3638757848 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(64..64 src has 1..64) ==== 5915+0+0 (secure 0 0 0) 0x7fa6300abd10 con 0x7fa63406d0c0 2024-09-16T00:09:08.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.195+0000 7fa638dc2640 1 --2- 172.21.15.139:0/3638757848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa61806c2b0 0x7fa61806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.195+0000 7fa638dc2640 1 --2- 172.21.15.139:0/3638757848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa61806c2b0 0x7fa61806e770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa634156140 tx=0x7fa624007e40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:08.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.203+0000 7fa629ffb640 1 -- 172.21.15.139:0/3638757848 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa630079df0 con 0x7fa63406d0c0 2024-09-16T00:09:08.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.207+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f334c0cbd90 con 0x7f334c153970 2024-09-16T00:09:08.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.209+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f334c0cbd90 con 0x7f334c153970 2024-09-16T00:09:08.210 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:08.211 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:08.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.209+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f334c14a580 0x7f334c14a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.210+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f334c0cbab0 con 0x7f334c14a580 2024-09-16T00:09:08.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.210+0000 7f3351a9f640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f334c14a580 0x7f334c14a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.210+0000 7f3351a9f640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f334c14a580 0x7f334c14a960 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:08.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.211+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f334c0cbab0 con 0x7f334c14a580 2024-09-16T00:09:08.215 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:08.214+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:08.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.226+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f334c0cb7d0 con 0x7f334c14a580 2024-09-16T00:09:08.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.228+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f334c0cb7d0 con 0x7f334c14a580 2024-09-16T00:09:08.229 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:08.230 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:08.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.228+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f334c1f3ed0 0x7f334c1f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.228+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f334c0cd8d0 con 0x7f334c1f3ed0 2024-09-16T00:09:08.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.229+0000 7f335129e640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f334c1f3ed0 0x7f334c1f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.229+0000 7f335129e640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f334c1f3ed0 0x7f334c1f42f0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:08.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.229+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f334c0cd8d0 con 0x7f334c1f3ed0 2024-09-16T00:09:08.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.244+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f334c062900 con 0x7f334c1f3ed0 2024-09-16T00:09:08.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.247+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f334c062900 con 0x7f334c1f3ed0 2024-09-16T00:09:08.248 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:08.249 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:08.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.247+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f334c0ce390 0x7f334c0ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.247+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f334c0633e0 con 0x7f334c0ce390 2024-09-16T00:09:08.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.247+0000 7f33522a0640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f334c0ce390 0x7f334c0ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.248+0000 7f33522a0640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f334c0ce390 0x7f334c0ce7f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:08.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.249+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f334c0633e0 con 0x7f334c0ce390 2024-09-16T00:09:08.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.268+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f334c0cf000 con 0x7f334c0ce390 2024-09-16T00:09:08.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.270+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f334c0cf000 con 0x7f334c0ce390 2024-09-16T00:09:08.270 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:08.271 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:08.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.270+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f334c02cb50 0x7f334c02cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.270+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f334c0cef80 con 0x7f334c02cb50 2024-09-16T00:09:08.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.270+0000 7f3351a9f640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f334c02cb50 0x7f334c02cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.270+0000 7f3351a9f640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f334c02cb50 0x7f334c02cf50 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:08.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.271+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f334c0cef80 con 0x7f334c02cb50 2024-09-16T00:09:08.285 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:08.286 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:08.286 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:08.286 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:08.286 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:08.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.287+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f334c0cef80 con 0x7f334c02cb50 2024-09-16T00:09:08.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.288+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f334c0cef80 con 0x7f334c02cb50 2024-09-16T00:09:08.289 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:08.290 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:08.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.289+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f334c204d40 0x7f334c207130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.289+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f334c02d9a0 con 0x7f334c204d40 2024-09-16T00:09:08.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.289+0000 7f335129e640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f334c204d40 0x7f334c207130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.289+0000 7f335129e640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f334c204d40 0x7f334c207130 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:08.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.290+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f334c02d9a0 con 0x7f334c204d40 2024-09-16T00:09:08.295 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:09:08.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.306+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f334c02d9a0 con 0x7f334c204d40 2024-09-16T00:09:08.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.308+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f334c02d9a0 con 0x7f334c204d40 2024-09-16T00:09:08.309 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:08.309 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:08.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.308+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f334c207b80 0x7f334c209f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.308+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f334c20cd40 con 0x7f334c207b80 2024-09-16T00:09:08.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.308+0000 7f33522a0640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f334c207b80 0x7f334c209f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.309+0000 7f33522a0640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f334c207b80 0x7f334c209f70 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:08.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.309+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f334c20cd40 con 0x7f334c207b80 2024-09-16T00:09:08.316 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:08.315+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:08.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.325+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f334c20a4b0 con 0x7f334c207b80 2024-09-16T00:09:08.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.327+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f334c20a4b0 con 0x7f334c207b80 2024-09-16T00:09:08.328 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:08.329 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:09:08.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.328+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f334c20ab30 0x7f334c20cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:08.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.328+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f334c20fee0 con 0x7f334c20ab30 2024-09-16T00:09:08.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.328+0000 7f3351a9f640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f334c20ab30 0x7f334c20cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:08.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.328+0000 7f3351a9f640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f334c20ab30 0x7f334c20cf20 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:08.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.329+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f334c20fee0 con 0x7f334c20ab30 2024-09-16T00:09:08.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.346+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f334c20ec20 con 0x7f334c20ab30 2024-09-16T00:09:08.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f33427fc640 1 -- 172.21.15.139:0/3527454115 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f334c20ec20 con 0x7f334c20ab30 2024-09-16T00:09:08.349 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:08.350 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:08.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f334c153970 msgr2=0x7f334c1528b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f334c153970 0x7f334c1528b0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f334c14a580 msgr2=0x7f334c14a960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f334c14a580 0x7f334c14a960 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f334c1f3ed0 msgr2=0x7f334c1f42f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f334c1f3ed0 0x7f334c1f42f0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f33522a0640 1 -- 172.21.15.139:0/3527454115 reap_dead start 2024-09-16T00:09:08.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f334c0ce390 msgr2=0x7f334c0ce7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f334c0ce390 0x7f334c0ce7f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f334c02cb50 msgr2=0x7f334c02cf50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f334c02cb50 0x7f334c02cf50 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f334c204d40 msgr2=0x7f334c207130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f334c204d40 0x7f334c207130 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f334c207b80 msgr2=0x7f334c209f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f334c207b80 0x7f334c209f70 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f334c20ab30 msgr2=0x7f334c20cf20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f334c20ab30 0x7f334c20cf20 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.348+0000 7f33522a0640 1 -- 172.21.15.139:0/3527454115 reap_dead start 2024-09-16T00:09:08.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.349+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f333006c2b0 msgr2=0x7f333006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.349+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f333006c2b0 0x7f333006e770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f333c009560 tx=0x7f333c009190 comp rx=0 tx=0).stop 2024-09-16T00:09:08.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.349+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f334c0b3430 msgr2=0x7f334c1509b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:08.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.349+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f334c0b3430 0x7f334c1509b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3348035040 tx=0x7f33480046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.349+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 shutdown_connections 2024-09-16T00:09:08.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.349+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f333006c2b0 0x7f333006e770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.349+0000 7f3353d03640 1 --2- 172.21.15.139:0/3527454115 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f334c0b3430 0x7f334c1509b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:08.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.349+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 >> 172.21.15.139:0/3527454115 conn(0x7f334c06c520 msgr2=0x7f334c0b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:08.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.349+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 shutdown_connections 2024-09-16T00:09:08.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.349+0000 7f3353d03640 1 -- 172.21.15.139:0/3527454115 wait complete. 2024-09-16T00:09:08.410 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:08.410 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:08.417 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:08.417+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:08.419 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:09:08.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:08.429+0000 7fa63b827640 1 -- 172.21.15.139:0/3638757848 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fa6341f06a0 con 0x7fa63406d0c0 2024-09-16T00:09:08.518 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:08.517+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:08.535 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:08.535 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:08.545 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:09:08.619 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:08.619+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:08.666 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:08.666 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:08.676 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:09:08.720 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:08.719+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:08.786 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:08.786 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:08.796 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:09:08.821 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:08.820+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:08.915 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:08.915 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:08.921 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:08.921+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:08.925 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:09:09.022 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:09.022+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:09.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:09.050+0000 7fa629ffb640 1 -- 172.21.15.139:0/3638757848 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v65) ==== 97+0+0 (secure 0 0 0) 0x7fa63007e0f0 con 0x7fa63406d0c0 2024-09-16T00:09:09.051 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is set 2024-09-16T00:09:09.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:09.052+0000 7fa63b827640 1 -- 172.21.15.139:0/3638757848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa61806c2b0 msgr2=0x7fa61806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:09.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:09.052+0000 7fa63b827640 1 --2- 172.21.15.139:0/3638757848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa61806c2b0 0x7fa61806e770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa634156140 tx=0x7fa624007e40 comp rx=0 tx=0).stop 2024-09-16T00:09:09.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:09.052+0000 7fa63b827640 1 -- 172.21.15.139:0/3638757848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa63406d0c0 msgr2=0x7fa6340cc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:09.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:09.052+0000 7fa63b827640 1 --2- 172.21.15.139:0/3638757848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa63406d0c0 0x7fa6340cc890 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa634070330 tx=0x7fa6300046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:09.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:09.052+0000 7fa63b827640 1 -- 172.21.15.139:0/3638757848 shutdown_connections 2024-09-16T00:09:09.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:09.052+0000 7fa63b827640 1 --2- 172.21.15.139:0/3638757848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa61806c2b0 0x7fa61806e770 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:09.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:09.052+0000 7fa63b827640 1 --2- 172.21.15.139:0/3638757848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa634154ae0 0x7fa6341f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:09.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:09.052+0000 7fa63b827640 1 --2- 172.21.15.139:0/3638757848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa63406e940 0x7fa6340c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:09.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:09.052+0000 7fa63b827640 1 --2- 172.21.15.139:0/3638757848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa63406d0c0 0x7fa6340cc890 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:09.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:09.052+0000 7fa63b827640 1 -- 172.21.15.139:0/3638757848 >> 172.21.15.139:0/3638757848 conn(0x7fa63406c150 msgr2=0x7fa63415a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:09.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:09.052+0000 7fa63b827640 1 -- 172.21.15.139:0/3638757848 shutdown_connections 2024-09-16T00:09:09.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:09.052+0000 7fa63b827640 1 -- 172.21.15.139:0/3638757848 wait complete. 2024-09-16T00:09:09.054 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:09.055 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:09.064 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:09:09.148 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:09.148+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:09.220 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:09.220 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:09.223 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:09.223+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:09.230 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:09:09.325 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:09.324+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:09.390 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:09.390 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:09.396 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:09:09.425 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:09.424+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:09.511 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:09.511 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:09.521 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:09:09.526 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:09.525+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:09.627 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:09.626+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:09.641 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:09.641 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:09.650 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:09:09.728 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:09.727+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:09.770 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:09.770 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:09.780 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:09:09.864 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:09.864+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:09.930 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:09.929+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:09.937 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:09.938 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:09.948 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:09:10.031 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:10.031+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:10.095 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:10.096 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:10.105 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:09:10.132 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:10.132+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:10.223 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:10.223 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:10.232 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:09:10.233 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:10.233+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:10.335 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:10.334+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:10.353 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:10.354 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:10.363 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:09:10.364 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:09:10.436 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:10.435+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:10.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.475+0000 7f7353dbd640 1 Processor -- start 2024-09-16T00:09:10.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.475+0000 7f7353dbd640 1 -- start start 2024-09-16T00:09:10.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.476+0000 7f7353dbd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f734c071660 0x7f734c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:10.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.476+0000 7f7353dbd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f734c15e340 con 0x7f734c071a40 2024-09-16T00:09:10.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.477+0000 7f7353dbd640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f734c15e620 con 0x7f734c15b4b0 2024-09-16T00:09:10.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.477+0000 7f7353dbd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f734c0cefd0 con 0x7f734c071660 2024-09-16T00:09:10.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.477+0000 7f7351b59640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f734c071660 0x7f734c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:10.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.477+0000 7f7351b59640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f734c071660 0x7f734c0cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58734/0 (socket says 172.21.15.139:58734) 2024-09-16T00:09:10.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.477+0000 7f7351b59640 1 -- 172.21.15.139:0/2026573866 learned_addr learned my addr 172.21.15.139:0/2026573866 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:10.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.477+0000 7f7351b59640 1 -- 172.21.15.139:0/2026573866 >> v1:172.21.15.164:6789/0 conn(0x7f734c15b4b0 legacy=0x7f734c0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:10.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.477+0000 7f7350b57640 1 -- 172.21.15.139:0/2026573866 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3282059138 0 0) 0x7f734c15e620 con 0x7f734c15b4b0 2024-09-16T00:09:10.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.477+0000 7f7351b59640 1 -- 172.21.15.139:0/2026573866 >> v1:172.21.15.139:6789/0 conn(0x7f734c071a40 legacy=0x7f734c06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:10.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.477+0000 7f7351b59640 1 -- 172.21.15.139:0/2026573866 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c15e900 con 0x7f734c071660 2024-09-16T00:09:10.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.478+0000 7f7351b59640 1 --2- 172.21.15.139:0/2026573866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f734c071660 0x7f734c0cea90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f73480098a0 tx=0x7f734802a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d88277c35918607 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:10.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.478+0000 7f7350b57640 1 -- 172.21.15.139:0/2026573866 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348037070 con 0x7f734c071660 2024-09-16T00:09:10.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.478+0000 7f7350b57640 1 -- 172.21.15.139:0/2026573866 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7348032640 con 0x7f734c071660 2024-09-16T00:09:10.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.478+0000 7f7350b57640 1 -- 172.21.15.139:0/2026573866 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348032a10 con 0x7f734c071660 2024-09-16T00:09:10.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.479+0000 7f7353dbd640 1 -- 172.21.15.139:0/2026573866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f734c071660 msgr2=0x7f734c0cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:10.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.479+0000 7f7353dbd640 1 --2- 172.21.15.139:0/2026573866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f734c071660 0x7f734c0cea90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f73480098a0 tx=0x7f734802a3c0 comp rx=0 tx=0).stop 2024-09-16T00:09:10.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.479+0000 7f7353dbd640 1 -- 172.21.15.139:0/2026573866 shutdown_connections 2024-09-16T00:09:10.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.479+0000 7f7353dbd640 1 --2- 172.21.15.139:0/2026573866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f734c071660 0x7f734c0cea90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:10.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.479+0000 7f7353dbd640 1 -- 172.21.15.139:0/2026573866 >> 172.21.15.139:0/2026573866 conn(0x7f734c073950 msgr2=0x7f734c06bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:10.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.479+0000 7f7353dbd640 1 -- 172.21.15.139:0/2026573866 shutdown_connections 2024-09-16T00:09:10.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.479+0000 7f7353dbd640 1 -- 172.21.15.139:0/2026573866 wait complete. 2024-09-16T00:09:10.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.480+0000 7f7353dbd640 1 Processor -- start 2024-09-16T00:09:10.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.480+0000 7f7353dbd640 1 -- start start 2024-09-16T00:09:10.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.481+0000 7f7353dbd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f734c071660 0x7f734c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:10.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.481+0000 7f7353dbd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f734c071a40 0x7f734c156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:10.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.481+0000 7f7353dbd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f734c15b4b0 0x7f734c150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:10.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.481+0000 7f7353dbd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f734c15eec0 con 0x7f734c071a40 2024-09-16T00:09:10.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.481+0000 7f7353dbd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f734c06d450 con 0x7f734c15b4b0 2024-09-16T00:09:10.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.481+0000 7f7353dbd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f734c06ecb0 con 0x7f734c071660 2024-09-16T00:09:10.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.481+0000 7f7351358640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f734c071a40 0x7f734c156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:10.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.481+0000 7f7351358640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f734c071a40 0x7f734c156520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58746/0 (socket says 172.21.15.139:58746) 2024-09-16T00:09:10.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.481+0000 7f7351358640 1 -- 172.21.15.139:0/3038497962 learned_addr learned my addr 172.21.15.139:0/3038497962 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:10.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.481+0000 7f7351b59640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f734c071660 0x7f734c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:10.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.481+0000 7f735235a640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f734c15b4b0 0x7f734c150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:10.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.481+0000 7f7351358640 1 -- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f734c071660 msgr2=0x7f734c155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:10.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.481+0000 7f7351358640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f734c071660 0x7f734c155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:10.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.481+0000 7f7351358640 1 -- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f734c15b4b0 msgr2=0x7f734c150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:10.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.481+0000 7f7351358640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f734c15b4b0 0x7f734c150420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:10.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.481+0000 7f7351358640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c151cc0 con 0x7f734c071a40 2024-09-16T00:09:10.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.482+0000 7f7351358640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f734c071a40 0x7f734c156520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f733c001eb0 tx=0x7f733c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:10.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.482+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c017070 con 0x7f734c071a40 2024-09-16T00:09:10.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.482+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f733c0052d0 con 0x7f734c071a40 2024-09-16T00:09:10.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.482+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734c151f40 con 0x7f734c071a40 2024-09-16T00:09:10.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.482+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c003830 con 0x7f734c071a40 2024-09-16T00:09:10.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.482+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c152500 con 0x7f734c071a40 2024-09-16T00:09:10.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.483+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f734c0cea90 con 0x7f734c071a40 2024-09-16T00:09:10.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.484+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f733c01a070 con 0x7f734c071a40 2024-09-16T00:09:10.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.484+0000 7f73427fc640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f733006c2b0 0x7f733006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:10.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.484+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5915+0+0 (secure 0 0 0) 0x7f733c08a880 con 0x7f734c071a40 2024-09-16T00:09:10.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.484+0000 7f7351b59640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f733006c2b0 0x7f733006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:10.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.485+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v65) ==== 54+0+16 (secure 0 0 0) 0x7f734c151cc0 con 0x7f734c071a40 2024-09-16T00:09:10.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.485+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f734c0b27b0 0x7f734c1f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:10.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.485+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f734c069220 con 0x7f734c0b27b0 2024-09-16T00:09:10.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.485+0000 7f7351b59640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f733006c2b0 0x7f733006e770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7348034000 tx=0x7f734802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:10.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.485+0000 7f735235a640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f734c0b27b0 0x7f734c1f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:10.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.485+0000 7f735235a640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f734c0b27b0 0x7f734c1f3db0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:10.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.486+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f734c069220 con 0x7f734c0b27b0 2024-09-16T00:09:10.489 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:10.489 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:10.500 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:09:10.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.503+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f734c0c8e10 con 0x7f734c0b27b0 2024-09-16T00:09:10.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.506+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f734c0c8e10 con 0x7f734c0b27b0 2024-09-16T00:09:10.507 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:10.507 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:10.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.506+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f734c1f0840 0x7f734c1f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:10.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.506+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f734c15a160 con 0x7f734c1f0840 2024-09-16T00:09:10.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.506+0000 7f7351b59640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f734c1f0840 0x7f734c1f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:10.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.507+0000 7f7351b59640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f734c1f0840 0x7f734c1f4800 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:10.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.507+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f734c15a160 con 0x7f734c1f0840 2024-09-16T00:09:10.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.526+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f734c152700 con 0x7f734c1f0840 2024-09-16T00:09:10.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.528+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f734c152700 con 0x7f734c1f0840 2024-09-16T00:09:10.529 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:10.529 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:10.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.528+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f734c14f890 0x7f734c062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:10.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.528+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f734c152910 con 0x7f734c14f890 2024-09-16T00:09:10.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.528+0000 7f7351358640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f734c14f890 0x7f734c062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:10.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.529+0000 7f7351358640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f734c14f890 0x7f734c062630 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:10.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.530+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f734c152910 con 0x7f734c14f890 2024-09-16T00:09:10.536 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:10.536+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:10.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.546+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f734c062b70 con 0x7f734c14f890 2024-09-16T00:09:10.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.549+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f734c062b70 con 0x7f734c14f890 2024-09-16T00:09:10.550 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:10.551 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:10.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.549+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f734c02c640 0x7f734c02ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:10.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.549+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f734c0cd670 con 0x7f734c02c640 2024-09-16T00:09:10.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.549+0000 7f735235a640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f734c02c640 0x7f734c02ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:10.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.550+0000 7f735235a640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f734c02c640 0x7f734c02ca40 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:10.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.550+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f734c0cd670 con 0x7f734c02c640 2024-09-16T00:09:10.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.567+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f734c02d250 con 0x7f734c02c640 2024-09-16T00:09:10.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.569+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f734c02d250 con 0x7f734c02c640 2024-09-16T00:09:10.570 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:10.571 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:10.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.569+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f734c02d8d0 0x7f734c0cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:10.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.569+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f734c206a50 con 0x7f734c02d8d0 2024-09-16T00:09:10.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.570+0000 7f7351b59640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f734c02d8d0 0x7f734c0cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:10.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.570+0000 7f7351b59640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f734c02d8d0 0x7f734c0cbf60 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:10.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.571+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f734c206a50 con 0x7f734c02d8d0 2024-09-16T00:09:10.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.587+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f734c0cc4a0 con 0x7f734c02d8d0 2024-09-16T00:09:10.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.589+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f734c0cc4a0 con 0x7f734c02d8d0 2024-09-16T00:09:10.589 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:10.590 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:10.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.589+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f734c0ccbc0 0x7f734c0cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:10.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.589+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f734c209be0 con 0x7f734c0ccbc0 2024-09-16T00:09:10.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.589+0000 7f7351358640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f734c0ccbc0 0x7f734c0cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:10.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.589+0000 7f7351358640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f734c0ccbc0 0x7f734c0cd000 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:10.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.590+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f734c209be0 con 0x7f734c0ccbc0 2024-09-16T00:09:10.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.608+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f734c207350 con 0x7f734c0ccbc0 2024-09-16T00:09:10.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.609+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f734c207350 con 0x7f734c0ccbc0 2024-09-16T00:09:10.610 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:10.611 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:10.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.609+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f734c2079d0 0x7f734c209dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:10.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.609+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f734c20cca0 con 0x7f734c2079d0 2024-09-16T00:09:10.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.610+0000 7f735235a640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f734c2079d0 0x7f734c209dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:10.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.610+0000 7f735235a640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f734c2079d0 0x7f734c209dc0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:10.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.611+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f734c20cca0 con 0x7f734c2079d0 2024-09-16T00:09:10.626 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:10.626 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:10.626 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:10.626 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:10.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.628+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f734c0755b0 con 0x7f734c2079d0 2024-09-16T00:09:10.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.630+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f734c0755b0 con 0x7f734c2079d0 2024-09-16T00:09:10.631 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:10.631 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:09:10.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.630+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f734c20a810 0x7f734c20ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:10.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.630+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f734c2100c0 con 0x7f734c20a810 2024-09-16T00:09:10.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.630+0000 7f7351b59640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f734c20a810 0x7f734c20ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:10.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.631+0000 7f7351b59640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f734c20a810 0x7f734c20ac70 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:10.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.631+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f734c2100c0 con 0x7f734c20a810 2024-09-16T00:09:10.637 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:09:10.638 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:10.636+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:10.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.648+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f734c20b2c0 con 0x7f734c20a810 2024-09-16T00:09:10.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f73427fc640 1 -- 172.21.15.139:0/3038497962 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f734c20b2c0 con 0x7f734c20a810 2024-09-16T00:09:10.651 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:10.652 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:10.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f734c0b27b0 msgr2=0x7f734c1f3db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:10.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f734c0b27b0 0x7f734c1f3db0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:10.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f734c1f0840 msgr2=0x7f734c1f4800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:10.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f734c1f0840 0x7f734c1f4800 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:10.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f734c14f890 msgr2=0x7f734c062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:10.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f734c14f890 0x7f734c062630 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:10.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f734c02c640 msgr2=0x7f734c02ca40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:10.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f735235a640 1 -- 172.21.15.139:0/3038497962 reap_dead start 2024-09-16T00:09:10.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f734c02c640 0x7f734c02ca40 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:10.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f734c02d8d0 msgr2=0x7f734c0cbf60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:10.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f734c02d8d0 0x7f734c0cbf60 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:10.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f734c0ccbc0 msgr2=0x7f734c0cd000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:10.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f734c0ccbc0 0x7f734c0cd000 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:10.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f734c2079d0 msgr2=0x7f734c209dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:10.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f734c2079d0 0x7f734c209dc0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:10.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f734c20a810 msgr2=0x7f734c20ac70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:10.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f734c20a810 0x7f734c20ac70 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:10.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f733006c2b0 msgr2=0x7f733006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:10.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f733006c2b0 0x7f733006e770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7348034000 tx=0x7f734802e000 comp rx=0 tx=0).stop 2024-09-16T00:09:10.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f734c071a40 msgr2=0x7f734c156520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:10.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f735235a640 1 -- 172.21.15.139:0/3038497962 reap_dead start 2024-09-16T00:09:10.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f734c071a40 0x7f734c156520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f733c001eb0 tx=0x7f733c00c890 comp rx=0 tx=0).stop 2024-09-16T00:09:10.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f735235a640 1 -- 172.21.15.139:0/3038497962 reap_dead start 2024-09-16T00:09:10.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 shutdown_connections 2024-09-16T00:09:10.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 --2- 172.21.15.139:0/3038497962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f734c071a40 0x7f734c156520 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:10.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 >> 172.21.15.139:0/3038497962 conn(0x7f734c073950 msgr2=0x7f734c0c92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:10.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 shutdown_connections 2024-09-16T00:09:10.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:10.650+0000 7f7353dbd640 1 -- 172.21.15.139:0/3038497962 wait complete. 2024-09-16T00:09:10.739 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:10.738+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:10.769 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:10.769 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:10.769 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:10.769 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:10.769 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:10.779 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:09:10.839 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:10.839+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:10.898 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:10.898 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:10.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:10.898 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:10.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.898 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:10.898 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:09:10.898 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.280485276999997, 2024-09-16T00:09:10.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038263220000000001, 2024-09-16T00:09:10.898 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.899 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.899 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.899 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:09:10.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:09:10.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615465+0000", 2024-09-16T00:09:10.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2769999999999999e-06 2024-09-16T00:09:10.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615471+0000", 2024-09-16T00:09:10.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5729999999999996e-06 2024-09-16T00:09:10.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615483+0000", 2024-09-16T00:09:10.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1838000000000001e-05 2024-09-16T00:09:10.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615492+0000", 2024-09-16T00:09:10.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2410000000000001e-06 2024-09-16T00:09:10.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615596+0000", 2024-09-16T00:09:10.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104097 2024-09-16T00:09:10.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615688+0000", 2024-09-16T00:09:10.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1225999999999997e-05 2024-09-16T00:09:10.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:10.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615804+0000", 2024-09-16T00:09:10.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000116601 2024-09-16T00:09:10.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:10.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618165+0000", 2024-09-16T00:09:10.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023609429999999999 2024-09-16T00:09:10.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:10.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:09:10.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010258509999999999 2024-09-16T00:09:10.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619243+0000", 2024-09-16T00:09:10.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2327999999999998e-05 2024-09-16T00:09:10.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619287+0000", 2024-09-16T00:09:10.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3346999999999997e-05 2024-09-16T00:09:10.904 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.904 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.904 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.904 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:10.904 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:10.905 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.280430201000001, 2024-09-16T00:09:10.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0054735510000000001, 2024-09-16T00:09:10.905 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.905 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.905 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.905 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:10.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:10.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615516+0000", 2024-09-16T00:09:10.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6899999999999998e-07 2024-09-16T00:09:10.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615519+0000", 2024-09-16T00:09:10.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1760000000000002e-06 2024-09-16T00:09:10.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615526+0000", 2024-09-16T00:09:10.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8399999999999997e-06 2024-09-16T00:09:10.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615531+0000", 2024-09-16T00:09:10.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4450000000000004e-06 2024-09-16T00:09:10.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:09:10.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054127099999999996 2024-09-16T00:09:10.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616140+0000", 2024-09-16T00:09:10.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7230999999999994e-05 2024-09-16T00:09:10.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:10.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616249+0000", 2024-09-16T00:09:10.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109122 2024-09-16T00:09:10.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.909 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.909 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:10.909 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618646+0000", 2024-09-16T00:09:10.909 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023972279999999999 2024-09-16T00:09:10.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.909 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.909 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:10.909 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:09:10.909 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022396220000000001 2024-09-16T00:09:10.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.909 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.909 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.909 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620939+0000", 2024-09-16T00:09:10.910 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2735999999999998e-05 2024-09-16T00:09:10.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.910 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.910 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620989+0000", 2024-09-16T00:09:10.910 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0411000000000002e-05 2024-09-16T00:09:10.910 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.910 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.910 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.910 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:10.910 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:10.910 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.280404722, 2024-09-16T00:09:10.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055872789999999997, 2024-09-16T00:09:10.911 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.911 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.911 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.911 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:10.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:10.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:10.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.01e-07 2024-09-16T00:09:10.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615547+0000", 2024-09-16T00:09:10.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7899999999999996e-06 2024-09-16T00:09:10.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:09:10.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1119999999999998e-06 2024-09-16T00:09:10.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615560+0000", 2024-09-16T00:09:10.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:09:10.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616424+0000", 2024-09-16T00:09:10.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086350800000000003 2024-09-16T00:09:10.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:09:10.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9675999999999999e-05 2024-09-16T00:09:10.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:10.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616621+0000", 2024-09-16T00:09:10.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7345999999999999e-05 2024-09-16T00:09:10.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:10.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618654+0000", 2024-09-16T00:09:10.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020335269999999998 2024-09-16T00:09:10.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.915 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:10.915 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621055+0000", 2024-09-16T00:09:10.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024009729999999998 2024-09-16T00:09:10.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.915 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.915 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621100+0000", 2024-09-16T00:09:10.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5402999999999999e-05 2024-09-16T00:09:10.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.915 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.915 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621128+0000", 2024-09-16T00:09:10.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7736e-05 2024-09-16T00:09:10.916 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.916 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.916 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.916 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:10.916 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:10.916 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.280379072000002, 2024-09-16T00:09:10.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057024509999999999, 2024-09-16T00:09:10.916 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.916 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.916 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.916 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:10.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:10.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:10.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6300000000000001e-07 2024-09-16T00:09:10.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615569+0000", 2024-09-16T00:09:10.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3709999999999998e-06 2024-09-16T00:09:10.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615580+0000", 2024-09-16T00:09:10.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0448000000000001e-05 2024-09-16T00:09:10.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615584+0000", 2024-09-16T00:09:10.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6020000000000002e-06 2024-09-16T00:09:10.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616850+0000", 2024-09-16T00:09:10.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012661110000000001 2024-09-16T00:09:10.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616915+0000", 2024-09-16T00:09:10.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4820000000000006e-05 2024-09-16T00:09:10.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:10.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617031+0000", 2024-09-16T00:09:10.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115258 2024-09-16T00:09:10.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:10.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618659+0000", 2024-09-16T00:09:10.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001627975 2024-09-16T00:09:10.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:10.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621179+0000", 2024-09-16T00:09:10.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025203410000000002 2024-09-16T00:09:10.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621238+0000", 2024-09-16T00:09:10.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9323000000000001e-05 2024-09-16T00:09:10.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.921 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621269+0000", 2024-09-16T00:09:10.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0839000000000001e-05 2024-09-16T00:09:10.921 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.921 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.921 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.921 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:10.921 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:09:10.922 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.280355322999995, 2024-09-16T00:09:10.922 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058012769999999996, 2024-09-16T00:09:10.922 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.922 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.922 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.922 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.922 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.922 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:09:10.922 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.922 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.922 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:09:10.922 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.923 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.923 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615591+0000", 2024-09-16T00:09:10.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6199999999999999e-07 2024-09-16T00:09:10.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.923 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.923 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615594+0000", 2024-09-16T00:09:10.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8739999999999999e-06 2024-09-16T00:09:10.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.923 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.923 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615599+0000", 2024-09-16T00:09:10.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:09:10.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615603+0000", 2024-09-16T00:09:10.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8489999999999999e-06 2024-09-16T00:09:10.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617235+0000", 2024-09-16T00:09:10.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001632037 2024-09-16T00:09:10.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617348+0000", 2024-09-16T00:09:10.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113387 2024-09-16T00:09:10.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:10.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617458+0000", 2024-09-16T00:09:10.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109765 2024-09-16T00:09:10.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:10.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618664+0000", 2024-09-16T00:09:10.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012059950000000001 2024-09-16T00:09:10.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:10.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621311+0000", 2024-09-16T00:09:10.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026468469999999999 2024-09-16T00:09:10.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621357+0000", 2024-09-16T00:09:10.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6360999999999999e-05 2024-09-16T00:09:10.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621392+0000", 2024-09-16T00:09:10.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4190999999999998e-05 2024-09-16T00:09:10.927 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.927 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.927 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.927 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:10.927 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:10.927 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.280336699000003, 2024-09-16T00:09:10.927 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058737279999999999, 2024-09-16T00:09:10.927 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.927 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.927 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.927 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:10.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:10.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:10.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3799999999999999e-07 2024-09-16T00:09:10.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615611+0000", 2024-09-16T00:09:10.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-06 2024-09-16T00:09:10.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615626+0000", 2024-09-16T00:09:10.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:09:10.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615634+0000", 2024-09-16T00:09:10.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-06 2024-09-16T00:09:10.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617692+0000", 2024-09-16T00:09:10.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002057718 2024-09-16T00:09:10.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617768+0000", 2024-09-16T00:09:10.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6878999999999994e-05 2024-09-16T00:09:10.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:10.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:09:10.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019501699999999999 2024-09-16T00:09:10.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:10.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:09:10.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034357200000000002 2024-09-16T00:09:10.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:10.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621429+0000", 2024-09-16T00:09:10.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0012999999999999e-05 2024-09-16T00:09:10.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621459+0000", 2024-09-16T00:09:10.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9498000000000001e-05 2024-09-16T00:09:10.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621483+0000", 2024-09-16T00:09:10.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3957000000000001e-05 2024-09-16T00:09:10.932 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.932 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.932 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.933 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:10.933 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:09:10.933 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.280305419000001, 2024-09-16T00:09:10.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059371789999999999, 2024-09-16T00:09:10.933 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.933 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.933 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.933 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:09:10.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.934 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:09:10.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.934 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:09:10.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4000000000000002e-07 2024-09-16T00:09:10.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.934 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615643+0000", 2024-09-16T00:09:10.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6860000000000002e-06 2024-09-16T00:09:10.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615649+0000", 2024-09-16T00:09:10.935 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7610000000000004e-06 2024-09-16T00:09:10.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615653+0000", 2024-09-16T00:09:10.935 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0380000000000001e-06 2024-09-16T00:09:10.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618197+0000", 2024-09-16T00:09:10.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025440079999999999 2024-09-16T00:09:10.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.936 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.936 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618312+0000", 2024-09-16T00:09:10.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115083 2024-09-16T00:09:10.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.936 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:10.936 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618412+0000", 2024-09-16T00:09:10.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010016799999999999 2024-09-16T00:09:10.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.936 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:10.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621518+0000", 2024-09-16T00:09:10.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031058679999999999 2024-09-16T00:09:10.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:10.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621526+0000", 2024-09-16T00:09:10.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2490000000000004e-06 2024-09-16T00:09:10.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621549+0000", 2024-09-16T00:09:10.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3686999999999999e-05 2024-09-16T00:09:10.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621577+0000", 2024-09-16T00:09:10.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8191000000000002e-05 2024-09-16T00:09:10.938 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.938 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.938 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.938 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:10.938 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:10.938 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.280287752999996, 2024-09-16T00:09:10.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005989622, 2024-09-16T00:09:10.938 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.939 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.939 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.939 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:10.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:10.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:10.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3799999999999998e-07 2024-09-16T00:09:10.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615660+0000", 2024-09-16T00:09:10.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9740000000000001e-06 2024-09-16T00:09:10.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615666+0000", 2024-09-16T00:09:10.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3499999999999996e-06 2024-09-16T00:09:10.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615671+0000", 2024-09-16T00:09:10.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.13e-06 2024-09-16T00:09:10.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618759+0000", 2024-09-16T00:09:10.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030880500000000002 2024-09-16T00:09:10.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618832+0000", 2024-09-16T00:09:10.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3725000000000003e-05 2024-09-16T00:09:10.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:10.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618958+0000", 2024-09-16T00:09:10.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012587899999999999 2024-09-16T00:09:10.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:10.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621583+0000", 2024-09-16T00:09:10.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026246469999999999 2024-09-16T00:09:10.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.943 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:10.943 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621602+0000", 2024-09-16T00:09:10.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9137e-05 2024-09-16T00:09:10.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.943 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.943 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621629+0000", 2024-09-16T00:09:10.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6451000000000001e-05 2024-09-16T00:09:10.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.943 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.943 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621648+0000", 2024-09-16T00:09:10.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8941e-05 2024-09-16T00:09:10.943 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.944 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.944 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.944 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:10.944 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:10.944 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.280270134000006, 2024-09-16T00:09:10.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053141380000000004, 2024-09-16T00:09:10.944 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.944 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.944 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.944 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:10.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:10.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:10.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-07 2024-09-16T00:09:10.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615678+0000", 2024-09-16T00:09:10.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:09:10.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615682+0000", 2024-09-16T00:09:10.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6659999999999999e-06 2024-09-16T00:09:10.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615686+0000", 2024-09-16T00:09:10.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6990000000000001e-06 2024-09-16T00:09:10.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.947 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.947 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615736+0000", 2024-09-16T00:09:10.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9656e-05 2024-09-16T00:09:10.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.947 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.947 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615864+0000", 2024-09-16T00:09:10.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012836100000000001 2024-09-16T00:09:10.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.947 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:10.947 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615989+0000", 2024-09-16T00:09:10.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000125525 2024-09-16T00:09:10.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.948 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:10.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617355+0000", 2024-09-16T00:09:10.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001365982 2024-09-16T00:09:10.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.948 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:10.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620897+0000", 2024-09-16T00:09:10.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035419610000000002 2024-09-16T00:09:10.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.948 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.949 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620941+0000", 2024-09-16T00:09:10.949 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3998999999999997e-05 2024-09-16T00:09:10.949 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.949 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.949 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620990+0000", 2024-09-16T00:09:10.949 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8199999999999999e-05 2024-09-16T00:09:10.949 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.949 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.949 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.949 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.949 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:10.950 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:10.950 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.280242876000003, 2024-09-16T00:09:10.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006316449, 2024-09-16T00:09:10.950 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.950 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.950 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.950 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:10.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:10.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:10.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3299999999999998e-07 2024-09-16T00:09:10.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615705+0000", 2024-09-16T00:09:10.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.278e-06 2024-09-16T00:09:10.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.952 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615711+0000", 2024-09-16T00:09:10.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8119999999999996e-06 2024-09-16T00:09:10.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.952 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615716+0000", 2024-09-16T00:09:10.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4909999999999999e-06 2024-09-16T00:09:10.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.952 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617551+0000", 2024-09-16T00:09:10.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00183549 2024-09-16T00:09:10.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.953 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.953 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.953 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617627+0000", 2024-09-16T00:09:10.953 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6163999999999998e-05 2024-09-16T00:09:10.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.953 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.953 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:10.953 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617776+0000", 2024-09-16T00:09:10.953 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014909499999999999 2024-09-16T00:09:10.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.953 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.953 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:10.953 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621072+0000", 2024-09-16T00:09:10.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032955150000000002 2024-09-16T00:09:10.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.954 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:10.954 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621957+0000", 2024-09-16T00:09:10.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088553199999999996 2024-09-16T00:09:10.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.954 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.954 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621998+0000", 2024-09-16T00:09:10.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0815000000000001e-05 2024-09-16T00:09:10.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:09:10.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0924000000000002e-05 2024-09-16T00:09:10.955 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.955 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.955 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.955 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:501 3.38 3:1f24713e:::1000000042b.00000000:head [write 0~71] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:10.955 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.761706+0000", 2024-09-16T00:09:10.955 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 13.134240069000001, 2024-09-16T00:09:10.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001845941, 2024-09-16T00:09:10.955 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.955 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.956 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.956 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:10.956 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:10.956 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:10.956 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 501 2024-09-16T00:09:10.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.956 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.956 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761706+0000", 2024-09-16T00:09:10.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761706+0000", 2024-09-16T00:09:10.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761707+0000", 2024-09-16T00:09:10.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8509999999999999e-06 2024-09-16T00:09:10.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761717+0000", 2024-09-16T00:09:10.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9250000000000007e-06 2024-09-16T00:09:10.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761732+0000", 2024-09-16T00:09:10.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4201e-05 2024-09-16T00:09:10.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761738+0000", 2024-09-16T00:09:10.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7299999999999999e-06 2024-09-16T00:09:10.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761819+0000", 2024-09-16T00:09:10.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1044999999999994e-05 2024-09-16T00:09:10.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.959 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.761936+0000", 2024-09-16T00:09:10.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011626299999999999 2024-09-16T00:09:10.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.959 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:10.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.762051+0000", 2024-09-16T00:09:10.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011582999999999999 2024-09-16T00:09:10.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.959 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:10.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.762671+0000", 2024-09-16T00:09:10.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00061922099999999999 2024-09-16T00:09:10.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:10.960 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.763496+0000", 2024-09-16T00:09:10.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00082528899999999999 2024-09-16T00:09:10.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.960 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.763525+0000", 2024-09-16T00:09:10.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9488999999999998e-05 2024-09-16T00:09:10.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.763552+0000", 2024-09-16T00:09:10.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6097000000000001e-05 2024-09-16T00:09:10.961 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.961 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.961 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.961 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:584 3.22 3:47e2f9af:::10000000492.00000000:head [write 0~10710] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:10.961 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.989570+0000", 2024-09-16T00:09:10.961 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 12.906375293, 2024-09-16T00:09:10.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038063620000000002, 2024-09-16T00:09:10.961 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.961 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.962 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.962 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:10.962 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:10.962 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:10.962 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 584 2024-09-16T00:09:10.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.962 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989570+0000", 2024-09-16T00:09:10.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989570+0000", 2024-09-16T00:09:10.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989573+0000", 2024-09-16T00:09:10.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9909999999999998e-06 2024-09-16T00:09:10.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989602+0000", 2024-09-16T00:09:10.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8353000000000001e-05 2024-09-16T00:09:10.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989622+0000", 2024-09-16T00:09:10.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9905999999999999e-05 2024-09-16T00:09:10.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989631+0000", 2024-09-16T00:09:10.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6360000000000006e-06 2024-09-16T00:09:10.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989706+0000", 2024-09-16T00:09:10.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4493000000000007e-05 2024-09-16T00:09:10.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989856+0000", 2024-09-16T00:09:10.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015023700000000001 2024-09-16T00:09:10.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:10.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.990004+0000", 2024-09-16T00:09:10.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000148157 2024-09-16T00:09:10.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:10.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.992924+0000", 2024-09-16T00:09:10.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029203250000000001 2024-09-16T00:09:10.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:10.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.993270+0000", 2024-09-16T00:09:10.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00034548899999999998 2024-09-16T00:09:10.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.993325+0000", 2024-09-16T00:09:10.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5337999999999998e-05 2024-09-16T00:09:10.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.993377+0000", 2024-09-16T00:09:10.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1437000000000002e-05 2024-09-16T00:09:10.967 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.967 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.967 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.967 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:589 3.22 3:44283f82:::10000000498.00000000:head [write 0~1058] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:10.967 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.007191+0000", 2024-09-16T00:09:10.967 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 12.88875423, 2024-09-16T00:09:10.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003237729, 2024-09-16T00:09:10.967 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.967 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.967 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.967 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:10.968 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:10.968 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:10.968 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 589 2024-09-16T00:09:10.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.968 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007191+0000", 2024-09-16T00:09:10.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007191+0000", 2024-09-16T00:09:10.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007193+0000", 2024-09-16T00:09:10.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8449999999999999e-06 2024-09-16T00:09:10.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007206+0000", 2024-09-16T00:09:10.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2509e-05 2024-09-16T00:09:10.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007223+0000", 2024-09-16T00:09:10.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7220000000000001e-05 2024-09-16T00:09:10.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007231+0000", 2024-09-16T00:09:10.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9370000000000004e-06 2024-09-16T00:09:10.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007285+0000", 2024-09-16T00:09:10.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4472000000000003e-05 2024-09-16T00:09:10.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op must wait for map", 2024-09-16T00:09:10.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.007289+0000", 2024-09-16T00:09:10.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0679999999999998e-06 2024-09-16T00:09:10.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.971 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009018+0000", 2024-09-16T00:09:10.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017285270000000001 2024-09-16T00:09:10.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.971 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009121+0000", 2024-09-16T00:09:10.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010267 2024-09-16T00:09:10.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:10.972 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009236+0000", 2024-09-16T00:09:10.972 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115563 2024-09-16T00:09:10.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.972 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:10.972 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009903+0000", 2024-09-16T00:09:10.972 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00066633999999999997 2024-09-16T00:09:10.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.972 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:10.972 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.010348+0000", 2024-09-16T00:09:10.972 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00044550999999999998 2024-09-16T00:09:10.973 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.973 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.973 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.973 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.010388+0000", 2024-09-16T00:09:10.973 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0240000000000001e-05 2024-09-16T00:09:10.973 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.973 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.973 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.973 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.010429+0000", 2024-09-16T00:09:10.973 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0828000000000003e-05 2024-09-16T00:09:10.973 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.973 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.973 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.973 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.974 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.974 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:590 3.21 3:86dbbdca:::10000000499.00000000:head [write 0~3104] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:10.974 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.009656+0000", 2024-09-16T00:09:10.974 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 12.886289774, 2024-09-16T00:09:10.974 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001717283, 2024-09-16T00:09:10.974 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.974 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.974 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.974 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:10.974 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:10.974 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:10.974 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 590 2024-09-16T00:09:10.974 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.974 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.975 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.975 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009656+0000", 2024-09-16T00:09:10.975 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.975 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.975 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009656+0000", 2024-09-16T00:09:10.975 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.975 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.975 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009657+0000", 2024-09-16T00:09:10.975 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.514e-06 2024-09-16T00:09:10.976 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.976 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.976 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.976 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009664+0000", 2024-09-16T00:09:10.976 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6819999999999997e-06 2024-09-16T00:09:10.976 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.976 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.976 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.976 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009677+0000", 2024-09-16T00:09:10.976 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2841e-05 2024-09-16T00:09:10.976 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.976 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.976 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.976 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009687+0000", 2024-09-16T00:09:10.977 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0171e-05 2024-09-16T00:09:10.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.977 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.977 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009742+0000", 2024-09-16T00:09:10.977 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4855999999999998e-05 2024-09-16T00:09:10.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.977 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.977 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009874+0000", 2024-09-16T00:09:10.977 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013217099999999999 2024-09-16T00:09:10.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.977 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:10.978 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.009994+0000", 2024-09-16T00:09:10.978 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000120252 2024-09-16T00:09:10.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.978 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.978 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:10.978 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.010890+0000", 2024-09-16T00:09:10.978 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00089574900000000005 2024-09-16T00:09:10.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.978 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.978 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:10.978 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.011289+0000", 2024-09-16T00:09:10.978 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00039914799999999999 2024-09-16T00:09:10.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.978 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.979 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.979 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.011336+0000", 2024-09-16T00:09:10.979 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6306e-05 2024-09-16T00:09:10.979 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.979 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.979 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.979 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.011373+0000", 2024-09-16T00:09:10.979 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7592999999999997e-05 2024-09-16T00:09:10.979 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.979 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.979 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.979 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.979 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.980 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:665 3.3d e59/20 3:be78f56d:::100000004f7.00000000:head v 59'10, mlcod=58'8)", 2024-09-16T00:09:10.980 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.222373+0000", 2024-09-16T00:09:10.980 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 12.673573018000001, 2024-09-16T00:09:10.980 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017385230000000001, 2024-09-16T00:09:10.980 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.980 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.980 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.980 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.980 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.980 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.980 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222373+0000", 2024-09-16T00:09:10.980 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.980 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.980 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.981 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.981 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222373+0000", 2024-09-16T00:09:10.981 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.981 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.981 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.981 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.981 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222374+0000", 2024-09-16T00:09:10.981 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5680000000000001e-06 2024-09-16T00:09:10.981 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.981 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.981 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.981 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222384+0000", 2024-09-16T00:09:10.981 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9690000000000006e-06 2024-09-16T00:09:10.981 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.982 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.982 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222405+0000", 2024-09-16T00:09:10.982 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0840999999999999e-05 2024-09-16T00:09:10.982 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.982 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.982 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222427+0000", 2024-09-16T00:09:10.982 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2348e-05 2024-09-16T00:09:10.982 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.982 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.982 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222489+0000", 2024-09-16T00:09:10.983 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1884000000000006e-05 2024-09-16T00:09:10.983 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.983 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.983 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.983 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.222512+0000", 2024-09-16T00:09:10.983 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2464999999999999e-05 2024-09-16T00:09:10.983 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.983 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.983 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.983 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.224062+0000", 2024-09-16T00:09:10.983 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001550685 2024-09-16T00:09:10.983 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.983 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.983 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.984 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.224111+0000", 2024-09-16T00:09:10.984 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8763000000000001e-05 2024-09-16T00:09:10.984 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.984 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.984 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.984 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.984 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.984 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:714 3.22 3:453cbc5d:::10000000536.00000000:head [write 0~260] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:10.984 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.356475+0000", 2024-09-16T00:09:10.984 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 12.539470995, 2024-09-16T00:09:10.984 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003898926, 2024-09-16T00:09:10.984 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.984 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.985 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.985 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:10.985 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:10.985 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:10.985 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 714 2024-09-16T00:09:10.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.985 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.985 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.985 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.985 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356475+0000", 2024-09-16T00:09:10.985 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.985 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.985 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.985 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356475+0000", 2024-09-16T00:09:10.986 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.986 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.986 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.986 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.986 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356476+0000", 2024-09-16T00:09:10.986 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6750000000000001e-06 2024-09-16T00:09:10.986 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.986 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.986 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.986 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356493+0000", 2024-09-16T00:09:10.986 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6562000000000002e-05 2024-09-16T00:09:10.986 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.986 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.987 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.987 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356509+0000", 2024-09-16T00:09:10.987 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5908000000000001e-05 2024-09-16T00:09:10.987 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.987 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.987 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.987 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356516+0000", 2024-09-16T00:09:10.987 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9079999999999998e-06 2024-09-16T00:09:10.987 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.987 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.987 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.987 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356573+0000", 2024-09-16T00:09:10.987 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7243000000000001e-05 2024-09-16T00:09:10.987 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.988 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.988 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356657+0000", 2024-09-16T00:09:10.988 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3809999999999999e-05 2024-09-16T00:09:10.988 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.988 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:10.988 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356742+0000", 2024-09-16T00:09:10.988 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5704000000000005e-05 2024-09-16T00:09:10.988 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.988 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:10.988 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360000+0000", 2024-09-16T00:09:10.988 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003257916 2024-09-16T00:09:10.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.989 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.989 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:10.989 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360279+0000", 2024-09-16T00:09:10.989 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00027872500000000001 2024-09-16T00:09:10.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.989 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.989 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.989 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360324+0000", 2024-09-16T00:09:10.989 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5359999999999999e-05 2024-09-16T00:09:10.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.989 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.989 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.990 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360374+0000", 2024-09-16T00:09:10.990 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9115e-05 2024-09-16T00:09:10.990 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.990 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.990 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.990 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.990 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:766 3.1f 3:f8970b5b:::10000000571.00000000:head [write 0~1989] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:10.990 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.481586+0000", 2024-09-16T00:09:10.990 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 12.414359340000001, 2024-09-16T00:09:10.990 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038531529999999998, 2024-09-16T00:09:10.990 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.990 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.990 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.991 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:10.991 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:10.991 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:10.991 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 766 2024-09-16T00:09:10.991 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.991 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.991 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.991 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.991 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481586+0000", 2024-09-16T00:09:10.991 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.991 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.991 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.991 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.992 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481586+0000", 2024-09-16T00:09:10.992 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.992 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.992 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.992 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.992 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481588+0000", 2024-09-16T00:09:10.992 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.691e-06 2024-09-16T00:09:10.992 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.992 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.992 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.992 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481595+0000", 2024-09-16T00:09:10.992 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5780000000000004e-06 2024-09-16T00:09:10.992 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.992 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.993 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.993 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481605+0000", 2024-09-16T00:09:10.993 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0729e-05 2024-09-16T00:09:10.993 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.993 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.993 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.993 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481610+0000", 2024-09-16T00:09:10.993 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:09:10.993 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.993 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.993 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.993 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481663+0000", 2024-09-16T00:09:10.993 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2429999999999998e-05 2024-09-16T00:09:10.994 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.994 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.994 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:10.994 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481740+0000", 2024-09-16T00:09:10.994 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7526000000000003e-05 2024-09-16T00:09:10.994 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.994 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.994 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:10.994 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481829+0000", 2024-09-16T00:09:10.994 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8265999999999998e-05 2024-09-16T00:09:10.994 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.994 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.994 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:10.994 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.482578+0000", 2024-09-16T00:09:10.995 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074936600000000001 2024-09-16T00:09:10.995 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.995 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.995 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:10.995 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.485314+0000", 2024-09-16T00:09:10.995 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027358949999999999 2024-09-16T00:09:10.995 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.995 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.995 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:10.995 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.485398+0000", 2024-09-16T00:09:10.995 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4462000000000005e-05 2024-09-16T00:09:10.995 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.995 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.995 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:10.996 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.485439+0000", 2024-09-16T00:09:10.996 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1066999999999998e-05 2024-09-16T00:09:10.996 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.996 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:10.996 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:10.996 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.996 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.996 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:839 3.38 3:1d7725f9:::100000005c8.00000000:head [write 0~3439] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:10.996 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.686484+0000", 2024-09-16T00:09:10.996 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 12.209461255000001, 2024-09-16T00:09:10.996 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016765650000000001, 2024-09-16T00:09:10.996 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:10.996 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:10.997 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:10.997 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:10.997 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:10.997 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:10.997 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 839 2024-09-16T00:09:10.997 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.997 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:10.997 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.997 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:10.997 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686484+0000", 2024-09-16T00:09:10.997 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.997 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.997 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686484+0000", 2024-09-16T00:09:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686487+0000", 2024-09-16T00:09:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2989999999999999e-06 2024-09-16T00:09:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686501+0000", 2024-09-16T00:09:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4592e-05 2024-09-16T00:09:10.998 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:10.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686521+0000", 2024-09-16T00:09:10.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9808000000000001e-05 2024-09-16T00:09:10.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:10.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686534+0000", 2024-09-16T00:09:10.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2561000000000001e-05 2024-09-16T00:09:10.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:10.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:10.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:10.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686609+0000", 2024-09-16T00:09:10.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5315999999999995e-05 2024-09-16T00:09:11.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.000 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686820+0000", 2024-09-16T00:09:11.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00021137900000000001 2024-09-16T00:09:11.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:11.000 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.686930+0000", 2024-09-16T00:09:11.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110158 2024-09-16T00:09:11.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:11.001 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.687631+0000", 2024-09-16T00:09:11.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000700393 2024-09-16T00:09:11.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.001 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:11.001 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.688087+0000", 2024-09-16T00:09:11.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000456432 2024-09-16T00:09:11.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.001 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.001 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.688122+0000", 2024-09-16T00:09:11.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4944000000000003e-05 2024-09-16T00:09:11.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.688161+0000", 2024-09-16T00:09:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8683000000000002e-05 2024-09-16T00:09:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:2315 3.1f 3:f84a2a2d:::10000000c60.00000000:head [write 0~648229] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T00:09:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.440012+0000", 2024-09-16T00:09:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 7.4559332979999997, 2024-09-16T00:09:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038521150000000001, 2024-09-16T00:09:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.002 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 2315 2024-09-16T00:09:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440012+0000", 2024-09-16T00:09:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440012+0000", 2024-09-16T00:09:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440014+0000", 2024-09-16T00:09:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5689999999999999e-06 2024-09-16T00:09:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440415+0000", 2024-09-16T00:09:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00040101999999999998 2024-09-16T00:09:11.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440434+0000", 2024-09-16T00:09:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8833000000000001e-05 2024-09-16T00:09:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440444+0000", 2024-09-16T00:09:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0552e-05 2024-09-16T00:09:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440508+0000", 2024-09-16T00:09:11.005 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4156e-05 2024-09-16T00:09:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440610+0000", 2024-09-16T00:09:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101584 2024-09-16T00:09:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440698+0000", 2024-09-16T00:09:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7824999999999993e-05 2024-09-16T00:09:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:11.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.443176+0000", 2024-09-16T00:09:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024776379999999999 2024-09-16T00:09:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.443789+0000", 2024-09-16T00:09:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00061301699999999999 2024-09-16T00:09:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.443820+0000", 2024-09-16T00:09:11.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1392e-05 2024-09-16T00:09:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.443864+0000", 2024-09-16T00:09:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4529e-05 2024-09-16T00:09:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:4497 3.21 3:85a4fa67:::10000001537.00000000:head [write 0~302] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e65)", 2024-09-16T00:09:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:10.670074+0000", 2024-09-16T00:09:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.22587165200000001, 2024-09-16T00:09:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024600920000000001, 2024-09-16T00:09:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 4497 2024-09-16T00:09:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.009 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.670074+0000", 2024-09-16T00:09:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.670074+0000", 2024-09-16T00:09:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.670076+0000", 2024-09-16T00:09:11.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7439999999999999e-06 2024-09-16T00:09:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.670084+0000", 2024-09-16T00:09:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4950000000000008e-06 2024-09-16T00:09:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.670102+0000", 2024-09-16T00:09:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7303999999999999e-05 2024-09-16T00:09:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.011 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.670109+0000", 2024-09-16T00:09:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9009999999999994e-06 2024-09-16T00:09:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.670161+0000", 2024-09-16T00:09:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1579000000000002e-05 2024-09-16T00:09:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.670298+0000", 2024-09-16T00:09:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000137014 2024-09-16T00:09:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.670385+0000", 2024-09-16T00:09:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6541999999999999e-05 2024-09-16T00:09:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.670894+0000", 2024-09-16T00:09:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000509929 2024-09-16T00:09:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.672467+0000", 2024-09-16T00:09:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001572396 2024-09-16T00:09:11.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.672500+0000", 2024-09-16T00:09:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3281999999999997e-05 2024-09-16T00:09:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.672534+0000", 2024-09-16T00:09:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3905999999999997e-05 2024-09-16T00:09:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.014 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.015 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:11.016 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:09:11.016 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:11.015+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:11.064 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T00:09:11.145 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:11.144+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:11.167 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:11.167 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:11.167 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:11.167 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:11.178 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:09:11.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.202+0000 7fbe8f1cf640 1 Processor -- start 2024-09-16T00:09:11.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.202+0000 7fbe8f1cf640 1 -- start start 2024-09-16T00:09:11.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.203+0000 7fbe8f1cf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbe88070d90 0x7fbe8815de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:11.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.203+0000 7fbe8f1cf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe8806ccd0 con 0x7fbe8815da90 2024-09-16T00:09:11.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.203+0000 7fbe8f1cf640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe880ca720 con 0x7fbe880c8b50 2024-09-16T00:09:11.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.203+0000 7fbe8f1cf640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbe8815e3b0 con 0x7fbe88070d90 2024-09-16T00:09:11.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.203+0000 7fbe8cf6b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fbe8815da90 0x7fbe88070410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46736/0 (socket says 172.21.15.139:46736) 2024-09-16T00:09:11.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.203+0000 7fbe8cf6b640 1 -- 172.21.15.139:0/893766515 learned_addr learned my addr 172.21.15.139:0/893766515 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:11.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.203+0000 7fbe8d76c640 1 --2- 172.21.15.139:0/893766515 >> v2:172.21.15.139:3300/0 conn(0x7fbe88070d90 0x7fbe8815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:11.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.204+0000 7fbe8d76c640 1 -- 172.21.15.139:0/893766515 >> v1:172.21.15.164:6789/0 conn(0x7fbe880c8b50 legacy=0x7fbe880ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:11.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.204+0000 7fbe8d76c640 1 -- 172.21.15.139:0/893766515 >> v1:172.21.15.139:6789/0 conn(0x7fbe8815da90 legacy=0x7fbe88070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:11.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.204+0000 7fbe8d76c640 1 -- 172.21.15.139:0/893766515 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8806cf40 con 0x7fbe88070d90 2024-09-16T00:09:11.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.204+0000 7fbe8d76c640 1 --2- 172.21.15.139:0/893766515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe88070d90 0x7fbe8815de70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009d90 tx=0x7fbe8002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b669bb9cd5945aaf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:11.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.205+0000 7fbe7f7fe640 1 -- 172.21.15.139:0/893766515 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80007740 con 0x7fbe88070d90 2024-09-16T00:09:11.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.205+0000 7fbe7f7fe640 1 -- 172.21.15.139:0/893766515 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe800078e0 con 0x7fbe88070d90 2024-09-16T00:09:11.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.205+0000 7fbe7f7fe640 1 -- 172.21.15.139:0/893766515 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80007cb0 con 0x7fbe88070d90 2024-09-16T00:09:11.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.205+0000 7fbe8f1cf640 1 -- 172.21.15.139:0/893766515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe88070d90 msgr2=0x7fbe8815de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.205+0000 7fbe8f1cf640 1 --2- 172.21.15.139:0/893766515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe88070d90 0x7fbe8815de70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009d90 tx=0x7fbe8002a6f0 comp rx=0 tx=0).stop 2024-09-16T00:09:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.205+0000 7fbe8f1cf640 1 -- 172.21.15.139:0/893766515 shutdown_connections 2024-09-16T00:09:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.205+0000 7fbe8f1cf640 1 --2- 172.21.15.139:0/893766515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe88070d90 0x7fbe8815de70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.205+0000 7fbe8f1cf640 1 -- 172.21.15.139:0/893766515 >> 172.21.15.139:0/893766515 conn(0x7fbe8806c150 msgr2=0x7fbe88067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.205+0000 7fbe8f1cf640 1 -- 172.21.15.139:0/893766515 shutdown_connections 2024-09-16T00:09:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.205+0000 7fbe8f1cf640 1 -- 172.21.15.139:0/893766515 wait complete. 2024-09-16T00:09:11.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.207+0000 7fbe8f1cf640 1 Processor -- start 2024-09-16T00:09:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.207+0000 7fbe8f1cf640 1 -- start start 2024-09-16T00:09:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe8f1cf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe88070d90 0x7fbe880c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe8f1cf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe880c8b50 0x7fbe880b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe8f1cf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe8815da90 0x7fbe880cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe8f1cf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbe880ce8d0 con 0x7fbe880c8b50 2024-09-16T00:09:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe8f1cf640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbe880caa00 con 0x7fbe88070d90 2024-09-16T00:09:11.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe8f1cf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbe880cea50 con 0x7fbe8815da90 2024-09-16T00:09:11.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe7ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe880c8b50 0x7fbe880b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:11.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe7ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe880c8b50 0x7fbe880b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58770/0 (socket says 172.21.15.139:58770) 2024-09-16T00:09:11.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe7ffff640 1 -- 172.21.15.139:0/4246979151 learned_addr learned my addr 172.21.15.139:0/4246979151 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:11.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe8cf6b640 1 --2- 172.21.15.139:0/4246979151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe88070d90 0x7fbe880c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:11.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe8d76c640 1 --2- 172.21.15.139:0/4246979151 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe8815da90 0x7fbe880cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:11.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe7ffff640 1 -- 172.21.15.139:0/4246979151 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe8815da90 msgr2=0x7fbe880cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:11.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe7ffff640 1 --2- 172.21.15.139:0/4246979151 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe8815da90 0x7fbe880cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:11.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe7ffff640 1 -- 172.21.15.139:0/4246979151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe88070d90 msgr2=0x7fbe880c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:11.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe7ffff640 1 --2- 172.21.15.139:0/4246979151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe88070d90 0x7fbe880c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:11.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe7ffff640 1 -- 172.21.15.139:0/4246979151 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe880b2ca0 con 0x7fbe880c8b50 2024-09-16T00:09:11.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe7ffff640 1 --2- 172.21.15.139:0/4246979151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe880c8b50 0x7fbe880b1ce0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800c440 tx=0x7fbe78001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:11.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe7d7fa640 1 -- 172.21.15.139:0/4246979151 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe780052d0 con 0x7fbe880c8b50 2024-09-16T00:09:11.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe7d7fa640 1 -- 172.21.15.139:0/4246979151 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe78005470 con 0x7fbe880c8b50 2024-09-16T00:09:11.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.208+0000 7fbe8f1cf640 1 -- 172.21.15.139:0/4246979151 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe880b0c40 con 0x7fbe880c8b50 2024-09-16T00:09:11.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.209+0000 7fbe7d7fa640 1 -- 172.21.15.139:0/4246979151 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe78007260 con 0x7fbe880c8b50 2024-09-16T00:09:11.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.209+0000 7fbe8f1cf640 1 -- 172.21.15.139:0/4246979151 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe880b1200 con 0x7fbe880c8b50 2024-09-16T00:09:11.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.210+0000 7fbe8f1cf640 1 -- 172.21.15.139:0/4246979151 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe68005290 con 0x7fbe880c8b50 2024-09-16T00:09:11.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.210+0000 7fbe7d7fa640 1 -- 172.21.15.139:0/4246979151 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbe78007400 con 0x7fbe880c8b50 2024-09-16T00:09:11.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.211+0000 7fbe7d7fa640 1 --2- 172.21.15.139:0/4246979151 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbe7406c2b0 0x7fbe7406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:11.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.211+0000 7fbe7d7fa640 1 -- 172.21.15.139:0/4246979151 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5915+0+0 (secure 0 0 0) 0x7fbe7808a8e0 con 0x7fbe880c8b50 2024-09-16T00:09:11.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.211+0000 7fbe8cf6b640 1 --2- 172.21.15.139:0/4246979151 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbe7406c2b0 0x7fbe7406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:11.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.211+0000 7fbe8cf6b640 1 --2- 172.21.15.139:0/4246979151 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbe7406c2b0 0x7fbe7406e770 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbe84005880 tx=0x7fbe84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:11.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.215+0000 7fbe7d7fa640 1 -- 172.21.15.139:0/4246979151 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbe78058930 con 0x7fbe880c8b50 2024-09-16T00:09:11.222 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:11.221+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:11.318 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:11.318 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:11.319 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:11.319 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:11.319 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:11.322 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:11.322+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:11.327 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:09:11.423 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:11.422+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:11.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:11.431+0000 7fbe8f1cf640 1 -- 172.21.15.139:0/4246979151 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fbe68005850 con 0x7fbe880c8b50 2024-09-16T00:09:11.488 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:11.488 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:11.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:11.489 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:11.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.489 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:22 2.e e22/19 2:717a0223:::608.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:11.489 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:09:11.489 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.869982913000001, 2024-09-16T00:09:11.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042195870000000003, 2024-09-16T00:09:11.489 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.489 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.489 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.489 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:09:11.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:09:11.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616426+0000", 2024-09-16T00:09:11.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.037e-06 2024-09-16T00:09:11.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.491 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.491 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616435+0000", 2024-09-16T00:09:11.491 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6440000000000006e-06 2024-09-16T00:09:11.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.491 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.491 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616447+0000", 2024-09-16T00:09:11.491 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2466000000000001e-05 2024-09-16T00:09:11.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.491 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.491 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616456+0000", 2024-09-16T00:09:11.491 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8799999999999997e-06 2024-09-16T00:09:11.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.492 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.492 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616517+0000", 2024-09-16T00:09:11.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1171000000000001e-05 2024-09-16T00:09:11.492 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.492 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.492 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616540+0000", 2024-09-16T00:09:11.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2991000000000001e-05 2024-09-16T00:09:11.492 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.492 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.492 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620599+0000", 2024-09-16T00:09:11.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040586140000000003 2024-09-16T00:09:11.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.493 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.493 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620645+0000", 2024-09-16T00:09:11.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5784000000000001e-05 2024-09-16T00:09:11.493 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.493 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.493 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.494 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:11.494 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:09:11.494 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.869893183000002, 2024-09-16T00:09:11.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044836249999999998, 2024-09-16T00:09:11.494 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.494 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.494 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:09:11.494 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:09:11.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:09:11.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616518+0000", 2024-09-16T00:09:11.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3699999999999999e-06 2024-09-16T00:09:11.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.496 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.496 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:09:11.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1479999999999997e-06 2024-09-16T00:09:11.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.496 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.496 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616534+0000", 2024-09-16T00:09:11.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0641e-05 2024-09-16T00:09:11.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.496 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.496 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616541+0000", 2024-09-16T00:09:11.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3339999999999996e-06 2024-09-16T00:09:11.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.497 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617318+0000", 2024-09-16T00:09:11.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00077694799999999996 2024-09-16T00:09:11.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.497 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617409+0000", 2024-09-16T00:09:11.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0740000000000002e-05 2024-09-16T00:09:11.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.497 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620998+0000", 2024-09-16T00:09:11.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035894439999999998 2024-09-16T00:09:11.498 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.498 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.498 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.498 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.498 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:11.498 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:09:11.498 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.869845280000007, 2024-09-16T00:09:11.498 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041597320000000002, 2024-09-16T00:09:11.498 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.498 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.498 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:09:11.498 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.499 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:09:11.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.499 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:09:11.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.499 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:09:11.500 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:09:11.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.500 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.500 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616566+0000", 2024-09-16T00:09:11.500 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6979999999999998e-06 2024-09-16T00:09:11.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.500 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.500 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616574+0000", 2024-09-16T00:09:11.500 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7759999999999997e-06 2024-09-16T00:09:11.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.500 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616580+0000", 2024-09-16T00:09:11.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:09:11.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.501 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616800+0000", 2024-09-16T00:09:11.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022016700000000001 2024-09-16T00:09:11.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.501 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616904+0000", 2024-09-16T00:09:11.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104579 2024-09-16T00:09:11.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.502 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.502 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620722+0000", 2024-09-16T00:09:11.502 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003818133 2024-09-16T00:09:11.502 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.502 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.502 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.502 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:11.502 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:09:11.502 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.869594594999995, 2024-09-16T00:09:11.502 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038451570000000001, 2024-09-16T00:09:11.502 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.503 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.503 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.503 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.503 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.503 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:09:11.503 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.503 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.503 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:09:11.503 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.504 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.504 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.504 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616814+0000", 2024-09-16T00:09:11.504 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9299999999999997e-07 2024-09-16T00:09:11.504 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.504 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.504 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.504 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616824+0000", 2024-09-16T00:09:11.504 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6600000000000007e-06 2024-09-16T00:09:11.504 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.504 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.504 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.504 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616835+0000", 2024-09-16T00:09:11.505 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0713000000000001e-05 2024-09-16T00:09:11.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.505 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.505 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616841+0000", 2024-09-16T00:09:11.505 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.934e-06 2024-09-16T00:09:11.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.505 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.505 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616948+0000", 2024-09-16T00:09:11.505 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010631100000000001 2024-09-16T00:09:11.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.506 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.506 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616967+0000", 2024-09-16T00:09:11.506 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9598000000000001e-05 2024-09-16T00:09:11.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.506 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.506 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.506 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620619+0000", 2024-09-16T00:09:11.506 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036515810000000001 2024-09-16T00:09:11.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.506 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.506 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.506 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620659+0000", 2024-09-16T00:09:11.506 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9666999999999998e-05 2024-09-16T00:09:11.506 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.507 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.507 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.507 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.507 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:11.507 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:09:11.507 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.869547643000004, 2024-09-16T00:09:11.507 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038240129999999998, 2024-09-16T00:09:11.507 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.507 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.507 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.507 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.507 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.508 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:09:11.508 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.508 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.508 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:09:11.508 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.508 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.508 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616861+0000", 2024-09-16T00:09:11.508 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0200000000000003e-07 2024-09-16T00:09:11.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.509 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.509 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.509 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616864+0000", 2024-09-16T00:09:11.509 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5090000000000002e-06 2024-09-16T00:09:11.509 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.509 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.509 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.509 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616871+0000", 2024-09-16T00:09:11.509 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9249999999999998e-06 2024-09-16T00:09:11.509 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.509 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.509 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.509 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616877+0000", 2024-09-16T00:09:11.509 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.981e-06 2024-09-16T00:09:11.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.510 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.510 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.510 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617168+0000", 2024-09-16T00:09:11.510 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029035699999999998 2024-09-16T00:09:11.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.510 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.510 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.510 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617184+0000", 2024-09-16T00:09:11.510 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6005999999999999e-05 2024-09-16T00:09:11.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.510 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.510 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.510 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620661+0000", 2024-09-16T00:09:11.511 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034775330000000001 2024-09-16T00:09:11.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.511 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.511 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.511 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620684+0000", 2024-09-16T00:09:11.511 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3300000000000001e-05 2024-09-16T00:09:11.511 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.511 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.511 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.511 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.511 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:09:11.512 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:09:11.512 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.869092199999997, 2024-09-16T00:09:11.512 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045415610000000004, 2024-09-16T00:09:11.512 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.512 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.512 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.512 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.512 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.512 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.512 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:09:11.512 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.512 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.512 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.512 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.513 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:09:11.513 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.513 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.513 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.513 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.513 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:09:11.513 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8800000000000002e-07 2024-09-16T00:09:11.513 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.513 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.513 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.513 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617321+0000", 2024-09-16T00:09:11.513 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6979999999999997e-06 2024-09-16T00:09:11.513 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.513 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.514 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.514 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617330+0000", 2024-09-16T00:09:11.514 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6729999999999998e-06 2024-09-16T00:09:11.514 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.514 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.514 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.514 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617336+0000", 2024-09-16T00:09:11.514 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2110000000000003e-06 2024-09-16T00:09:11.514 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.514 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.514 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.514 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617394+0000", 2024-09-16T00:09:11.514 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8087999999999997e-05 2024-09-16T00:09:11.514 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.515 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.515 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.515 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617412+0000", 2024-09-16T00:09:11.515 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8099999999999999e-05 2024-09-16T00:09:11.515 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.515 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.515 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.515 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621818+0000", 2024-09-16T00:09:11.515 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044061209999999998 2024-09-16T00:09:11.515 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.515 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.515 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.515 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621857+0000", 2024-09-16T00:09:11.515 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9082000000000003e-05 2024-09-16T00:09:11.515 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.516 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.516 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.516 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.516 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.516 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:23 2.e e22/19 2:717a0223:::608.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:11.516 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:09:11.516 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.868196178999995, 2024-09-16T00:09:11.516 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036198279999999998, 2024-09-16T00:09:11.516 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.516 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.516 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.516 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.516 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.517 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.517 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:09:11.517 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.517 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.517 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.517 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:09:11.517 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.517 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.517 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.517 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618213+0000", 2024-09-16T00:09:11.517 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:09:11.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.518 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.518 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.518 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618218+0000", 2024-09-16T00:09:11.518 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4099999999999999e-06 2024-09-16T00:09:11.518 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.518 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.518 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.518 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618227+0000", 2024-09-16T00:09:11.518 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5009999999999993e-06 2024-09-16T00:09:11.518 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.518 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.518 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.518 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618234+0000", 2024-09-16T00:09:11.519 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:09:11.519 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.519 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.519 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.519 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618307+0000", 2024-09-16T00:09:11.519 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3421000000000001e-05 2024-09-16T00:09:11.519 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.519 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.519 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.519 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618328+0000", 2024-09-16T00:09:11.519 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0919000000000001e-05 2024-09-16T00:09:11.519 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.519 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.519 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.520 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621804+0000", 2024-09-16T00:09:11.520 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034756560000000001 2024-09-16T00:09:11.520 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.520 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.520 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.520 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621832+0000", 2024-09-16T00:09:11.520 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7966999999999998e-05 2024-09-16T00:09:11.520 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.520 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.520 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.520 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.520 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.520 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:243 3.24 3:25048551:::100000002f0.00000000:head [write 0~1428] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:11.520 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.799025+0000", 2024-09-16T00:09:11.521 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.687382845, 2024-09-16T00:09:11.521 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042804829999999999, 2024-09-16T00:09:11.521 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.521 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.521 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.521 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:11.521 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:11.521 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:11.521 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 243 2024-09-16T00:09:11.521 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.521 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.521 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.521 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.522 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799025+0000", 2024-09-16T00:09:11.522 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.522 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.522 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.522 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.522 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799025+0000", 2024-09-16T00:09:11.522 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.522 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.522 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.522 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.522 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799026+0000", 2024-09-16T00:09:11.522 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9099999999999991e-07 2024-09-16T00:09:11.522 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.522 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.522 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.523 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799032+0000", 2024-09-16T00:09:11.523 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:09:11.523 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.523 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.523 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.523 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799044+0000", 2024-09-16T00:09:11.523 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1861999999999999e-05 2024-09-16T00:09:11.523 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.523 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.523 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.523 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799049+0000", 2024-09-16T00:09:11.523 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6570000000000002e-06 2024-09-16T00:09:11.523 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.523 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.524 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.524 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799095+0000", 2024-09-16T00:09:11.524 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5479999999999998e-05 2024-09-16T00:09:11.524 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.524 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.524 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.524 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799175+0000", 2024-09-16T00:09:11.524 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0181999999999999e-05 2024-09-16T00:09:11.524 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.524 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.524 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:11.524 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799252+0000", 2024-09-16T00:09:11.524 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6822000000000004e-05 2024-09-16T00:09:11.524 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.525 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.525 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:11.525 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799855+0000", 2024-09-16T00:09:11.525 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00060294199999999997 2024-09-16T00:09:11.525 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.525 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.525 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:11.525 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803230+0000", 2024-09-16T00:09:11.525 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033752719999999999 2024-09-16T00:09:11.525 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.525 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.525 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.525 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803263+0000", 2024-09-16T00:09:11.525 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3339e-05 2024-09-16T00:09:11.526 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.526 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.526 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.526 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803306+0000", 2024-09-16T00:09:11.526 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2330999999999998e-05 2024-09-16T00:09:11.526 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.526 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.526 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.526 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.526 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.526 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:324 3.29 3:960c7fc3:::10000000355.00000000:head [write 0~1506] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:11.526 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.034622+0000", 2024-09-16T00:09:11.526 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.451785824, 2024-09-16T00:09:11.526 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003641746, 2024-09-16T00:09:11.527 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.527 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.527 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.527 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:11.527 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:11.527 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:11.527 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 324 2024-09-16T00:09:11.527 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.527 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.527 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.527 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.527 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034622+0000", 2024-09-16T00:09:11.527 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.528 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.528 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.528 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.528 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034622+0000", 2024-09-16T00:09:11.528 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.528 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.528 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.528 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.528 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034623+0000", 2024-09-16T00:09:11.528 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1689999999999999e-06 2024-09-16T00:09:11.528 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.528 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.528 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.528 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034629+0000", 2024-09-16T00:09:11.529 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4859999999999997e-06 2024-09-16T00:09:11.529 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.529 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.529 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.529 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034639+0000", 2024-09-16T00:09:11.529 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0142999999999999e-05 2024-09-16T00:09:11.529 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.529 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.529 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.529 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034644+0000", 2024-09-16T00:09:11.529 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1159999999999998e-06 2024-09-16T00:09:11.529 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.529 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.530 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.530 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034693+0000", 2024-09-16T00:09:11.530 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9228999999999999e-05 2024-09-16T00:09:11.530 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.530 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.530 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.530 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034813+0000", 2024-09-16T00:09:11.530 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012003200000000001 2024-09-16T00:09:11.530 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.530 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.530 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:11.530 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034940+0000", 2024-09-16T00:09:11.530 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012655699999999999 2024-09-16T00:09:11.531 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.531 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.531 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:11.531 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.036244+0000", 2024-09-16T00:09:11.531 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001304326 2024-09-16T00:09:11.531 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.531 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.531 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:11.531 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.038175+0000", 2024-09-16T00:09:11.531 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001930768 2024-09-16T00:09:11.531 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.531 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.531 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.531 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.038225+0000", 2024-09-16T00:09:11.531 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0046000000000002e-05 2024-09-16T00:09:11.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.532 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.532 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.532 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.038264+0000", 2024-09-16T00:09:11.532 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8874000000000001e-05 2024-09-16T00:09:11.532 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.532 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.532 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.532 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.532 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:411 3.29 3:96aa9846:::100000003bf.00000000:head [write 0~1161] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:11.532 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.363361+0000", 2024-09-16T00:09:11.532 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.123046774000001, 2024-09-16T00:09:11.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038090889999999999, 2024-09-16T00:09:11.533 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.533 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.533 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.533 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:11.533 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:11.533 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:11.533 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 411 2024-09-16T00:09:11.533 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.533 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.533 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.533 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363361+0000", 2024-09-16T00:09:11.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.534 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.534 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.534 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363361+0000", 2024-09-16T00:09:11.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.534 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.534 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.534 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363363+0000", 2024-09-16T00:09:11.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.421e-06 2024-09-16T00:09:11.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.534 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.534 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.534 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363369+0000", 2024-09-16T00:09:11.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7270000000000001e-06 2024-09-16T00:09:11.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.535 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.535 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.535 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363382+0000", 2024-09-16T00:09:11.535 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2700999999999999e-05 2024-09-16T00:09:11.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.535 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.535 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.535 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363388+0000", 2024-09-16T00:09:11.535 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0009999999999998e-06 2024-09-16T00:09:11.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.535 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.535 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.535 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363438+0000", 2024-09-16T00:09:11.536 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9789000000000001e-05 2024-09-16T00:09:11.536 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.536 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.536 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.536 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363562+0000", 2024-09-16T00:09:11.536 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000123798 2024-09-16T00:09:11.536 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.536 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.536 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:11.536 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363639+0000", 2024-09-16T00:09:11.536 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7074000000000001e-05 2024-09-16T00:09:11.536 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.536 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.536 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:11.537 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.364153+0000", 2024-09-16T00:09:11.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00051393999999999995 2024-09-16T00:09:11.537 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.537 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.537 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:11.537 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.367097+0000", 2024-09-16T00:09:11.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00294414 2024-09-16T00:09:11.537 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.537 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.537 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.537 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.367145+0000", 2024-09-16T00:09:11.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8591999999999998e-05 2024-09-16T00:09:11.537 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.537 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.367170+0000", 2024-09-16T00:09:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4905999999999998e-05 2024-09-16T00:09:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:53 2.14 e58/46 2:2e2fa760:::200.00000001:head v 58'16, mlcod=57'14)", 2024-09-16T00:09:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.453518+0000", 2024-09-16T00:09:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.032890246999999, 2024-09-16T00:09:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036209150000000002, 2024-09-16T00:09:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.453518+0000", 2024-09-16T00:09:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.453518+0000", 2024-09-16T00:09:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.453519+0000", 2024-09-16T00:09:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.277e-06 2024-09-16T00:09:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.454703+0000", 2024-09-16T00:09:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011842440000000001 2024-09-16T00:09:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.454724+0000", 2024-09-16T00:09:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0835999999999998e-05 2024-09-16T00:09:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.541 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.541 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.454734+0000", 2024-09-16T00:09:11.541 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0299000000000001e-05 2024-09-16T00:09:11.541 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.541 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.541 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.541 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.454802+0000", 2024-09-16T00:09:11.541 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7127999999999998e-05 2024-09-16T00:09:11.541 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.541 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.541 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.541 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.454820+0000", 2024-09-16T00:09:11.541 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8858000000000001e-05 2024-09-16T00:09:11.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.542 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.542 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.457101+0000", 2024-09-16T00:09:11.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022802539999999998 2024-09-16T00:09:11.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.542 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.542 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.457139+0000", 2024-09-16T00:09:11.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8019000000000003e-05 2024-09-16T00:09:11.542 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.542 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.542 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.543 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:714 3.22 e59/33 3:453cbc5d:::10000000536.00000000:head v 59'18, mlcod=59'16)", 2024-09-16T00:09:11.543 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.356842+0000", 2024-09-16T00:09:11.543 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 13.129565582, 2024-09-16T00:09:11.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032768649999999999, 2024-09-16T00:09:11.543 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.543 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.543 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.543 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.543 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.543 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356842+0000", 2024-09-16T00:09:11.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356842+0000", 2024-09-16T00:09:11.544 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356844+0000", 2024-09-16T00:09:11.544 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4219999999999999e-06 2024-09-16T00:09:11.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356853+0000", 2024-09-16T00:09:11.545 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7760000000000003e-06 2024-09-16T00:09:11.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.545 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.545 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.545 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356871+0000", 2024-09-16T00:09:11.545 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8114e-05 2024-09-16T00:09:11.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.545 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.545 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.545 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356881+0000", 2024-09-16T00:09:11.545 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0233e-05 2024-09-16T00:09:11.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.545 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.545 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356949+0000", 2024-09-16T00:09:11.546 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7780000000000005e-05 2024-09-16T00:09:11.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.546 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.546 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356972+0000", 2024-09-16T00:09:11.546 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3144999999999999e-05 2024-09-16T00:09:11.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.546 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.546 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360073+0000", 2024-09-16T00:09:11.546 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031013170000000001 2024-09-16T00:09:11.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.546 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.547 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.547 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360119+0000", 2024-09-16T00:09:11.547 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6078000000000002e-05 2024-09-16T00:09:11.547 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.547 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.547 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.547 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.547 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.547 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:55 2.14 e59/46 2:2e2fa760:::200.00000001:head v 59'17, mlcod=57'15)", 2024-09-16T00:09:11.547 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.643358+0000", 2024-09-16T00:09:11.547 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 12.843049951999999, 2024-09-16T00:09:11.547 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031410520000000001, 2024-09-16T00:09:11.547 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.547 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.548 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.548 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.548 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.548 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.643358+0000", 2024-09-16T00:09:11.548 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.548 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.548 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.643358+0000", 2024-09-16T00:09:11.548 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.548 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.549 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.643360+0000", 2024-09-16T00:09:11.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.136e-06 2024-09-16T00:09:11.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.549 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.644215+0000", 2024-09-16T00:09:11.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085427799999999998 2024-09-16T00:09:11.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.549 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.644236+0000", 2024-09-16T00:09:11.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1693000000000001e-05 2024-09-16T00:09:11.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.644248+0000", 2024-09-16T00:09:11.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1958e-05 2024-09-16T00:09:11.550 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.550 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.644310+0000", 2024-09-16T00:09:11.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1853000000000006e-05 2024-09-16T00:09:11.550 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.550 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.644327+0000", 2024-09-16T00:09:11.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7258999999999999e-05 2024-09-16T00:09:11.550 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.551 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.551 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.646455+0000", 2024-09-16T00:09:11.551 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021274710000000001 2024-09-16T00:09:11.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.551 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.551 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.551 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.646499+0000", 2024-09-16T00:09:11.551 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4403999999999997e-05 2024-09-16T00:09:11.551 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.551 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.551 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.551 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.551 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:72 2.14 e61/46 2:2e2fa760:::200.00000001:head v 61'21, mlcod=61'19)", 2024-09-16T00:09:11.552 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:01.672924+0000", 2024-09-16T00:09:11.552 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.8134839060000001, 2024-09-16T00:09:11.552 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003655784, 2024-09-16T00:09:11.552 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.552 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.552 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.552 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.552 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.552 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.552 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.672924+0000", 2024-09-16T00:09:11.552 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.552 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.552 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.552 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.672924+0000", 2024-09-16T00:09:11.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.553 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.672925+0000", 2024-09-16T00:09:11.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0189999999999999e-06 2024-09-16T00:09:11.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.553 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674096+0000", 2024-09-16T00:09:11.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011709660000000001 2024-09-16T00:09:11.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.554 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.554 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674112+0000", 2024-09-16T00:09:11.554 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6149000000000001e-05 2024-09-16T00:09:11.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.554 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.554 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674120+0000", 2024-09-16T00:09:11.554 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.892e-06 2024-09-16T00:09:11.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.554 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.554 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674179+0000", 2024-09-16T00:09:11.554 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8876999999999999e-05 2024-09-16T00:09:11.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.555 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.555 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.555 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674197+0000", 2024-09-16T00:09:11.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8048999999999999e-05 2024-09-16T00:09:11.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.555 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.555 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.555 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.676539+0000", 2024-09-16T00:09:11.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023414830000000001 2024-09-16T00:09:11.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.555 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.555 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.556 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.676580+0000", 2024-09-16T00:09:11.556 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1349e-05 2024-09-16T00:09:11.556 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.556 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.556 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.556 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.556 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.556 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:86 2.1b 2:ddb4e4d7:::200.00000002:head [write 368603~2220227 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:11.556 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.644964+0000", 2024-09-16T00:09:11.556 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 7.8414438329999996, 2024-09-16T00:09:11.556 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011714478, 2024-09-16T00:09:11.556 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.556 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.556 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.557 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.644964+0000", 2024-09-16T00:09:11.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.644964+0000", 2024-09-16T00:09:11.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.644966+0000", 2024-09-16T00:09:11.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4789999999999999e-06 2024-09-16T00:09:11.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.558 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.648979+0000", 2024-09-16T00:09:11.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040136900000000003 2024-09-16T00:09:11.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.558 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.648993+0000", 2024-09-16T00:09:11.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3129000000000001e-05 2024-09-16T00:09:11.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649000+0000", 2024-09-16T00:09:11.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5700000000000004e-06 2024-09-16T00:09:11.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.559 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649055+0000", 2024-09-16T00:09:11.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4914999999999999e-05 2024-09-16T00:09:11.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.559 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649131+0000", 2024-09-16T00:09:11.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6055000000000003e-05 2024-09-16T00:09:11.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:11.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649258+0000", 2024-09-16T00:09:11.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000127229 2024-09-16T00:09:11.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:11.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656218+0000", 2024-09-16T00:09:11.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069600469999999996 2024-09-16T00:09:11.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:11.561 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656602+0000", 2024-09-16T00:09:11.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038380700000000001 2024-09-16T00:09:11.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.561 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656638+0000", 2024-09-16T00:09:11.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5725000000000001e-05 2024-09-16T00:09:11.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.561 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656679+0000", 2024-09-16T00:09:11.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0831999999999998e-05 2024-09-16T00:09:11.561 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.562 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.562 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.562 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.562 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.562 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:87 2.1b 2:ddb4e4d7:::200.00000002:head [write 2588830~1400 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:11.562 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649020+0000", 2024-09-16T00:09:11.562 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 7.8373876420000004, 2024-09-16T00:09:11.562 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0083707460000000001, 2024-09-16T00:09:11.562 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.562 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.562 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.562 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.562 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.562 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.563 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649020+0000", 2024-09-16T00:09:11.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.563 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.563 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649020+0000", 2024-09-16T00:09:11.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.563 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.563 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649021+0000", 2024-09-16T00:09:11.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6000000000000002e-07 2024-09-16T00:09:11.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649034+0000", 2024-09-16T00:09:11.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2627e-05 2024-09-16T00:09:11.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.564 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649045+0000", 2024-09-16T00:09:11.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0744000000000001e-05 2024-09-16T00:09:11.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.564 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649051+0000", 2024-09-16T00:09:11.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5459999999999997e-06 2024-09-16T00:09:11.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.565 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.565 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.565 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654369+0000", 2024-09-16T00:09:11.565 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053179519999999999 2024-09-16T00:09:11.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.565 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.565 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.565 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654435+0000", 2024-09-16T00:09:11.565 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6175999999999998e-05 2024-09-16T00:09:11.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.565 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.565 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:11.565 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654525+0000", 2024-09-16T00:09:11.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9856000000000002e-05 2024-09-16T00:09:11.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.566 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:11.566 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656225+0000", 2024-09-16T00:09:11.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001699386 2024-09-16T00:09:11.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.566 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:11.566 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657339+0000", 2024-09-16T00:09:11.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011145250000000001 2024-09-16T00:09:11.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.566 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.566 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657366+0000", 2024-09-16T00:09:11.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7172e-05 2024-09-16T00:09:11.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657391+0000", 2024-09-16T00:09:11.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4902e-05 2024-09-16T00:09:11.567 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.567 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.567 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.567 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:88 2.1b 2:ddb4e4d7:::200.00000002:head [write 2590230~1400 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:11.567 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649060+0000", 2024-09-16T00:09:11.567 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 7.8373478399999996, 2024-09-16T00:09:11.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0088748730000000001, 2024-09-16T00:09:11.568 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.568 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.568 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.568 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.568 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.568 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649060+0000", 2024-09-16T00:09:11.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.568 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.568 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649060+0000", 2024-09-16T00:09:11.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649061+0000", 2024-09-16T00:09:11.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2300000000000001e-07 2024-09-16T00:09:11.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649064+0000", 2024-09-16T00:09:11.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3160000000000001e-06 2024-09-16T00:09:11.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649070+0000", 2024-09-16T00:09:11.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9880000000000001e-06 2024-09-16T00:09:11.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649074+0000", 2024-09-16T00:09:11.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4329999999999997e-06 2024-09-16T00:09:11.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654769+0000", 2024-09-16T00:09:11.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0056944500000000002 2024-09-16T00:09:11.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.571 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654875+0000", 2024-09-16T00:09:11.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000106183 2024-09-16T00:09:11.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.571 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:11.571 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654994+0000", 2024-09-16T00:09:11.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011867900000000001 2024-09-16T00:09:11.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.571 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:11.571 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657697+0000", 2024-09-16T00:09:11.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027033920000000002 2024-09-16T00:09:11.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:11.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657882+0000", 2024-09-16T00:09:11.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018478900000000001 2024-09-16T00:09:11.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657913+0000", 2024-09-16T00:09:11.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1306000000000001e-05 2024-09-16T00:09:11.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657935+0000", 2024-09-16T00:09:11.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1814000000000002e-05 2024-09-16T00:09:11.572 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.573 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.573 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.573 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.573 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.573 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:89 2.1b 2:ddb4e4d7:::200.00000002:head [write 2591630~1401 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:11.573 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:11.573 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 7.8373275810000003, 2024-09-16T00:09:11.573 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011802609, 2024-09-16T00:09:11.573 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.573 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.573 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.573 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.573 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.573 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.574 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:11.574 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.574 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:11.574 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.574 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649081+0000", 2024-09-16T00:09:11.574 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7100000000000002e-07 2024-09-16T00:09:11.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649089+0000", 2024-09-16T00:09:11.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7640000000000005e-06 2024-09-16T00:09:11.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649094+0000", 2024-09-16T00:09:11.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6219999999999997e-06 2024-09-16T00:09:11.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649099+0000", 2024-09-16T00:09:11.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6970000000000001e-06 2024-09-16T00:09:11.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.576 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.576 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.576 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655288+0000", 2024-09-16T00:09:11.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061885569999999999 2024-09-16T00:09:11.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.576 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.576 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.576 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655365+0000", 2024-09-16T00:09:11.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7723000000000001e-05 2024-09-16T00:09:11.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.576 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.576 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:11.576 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655483+0000", 2024-09-16T00:09:11.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117895 2024-09-16T00:09:11.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:11.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657758+0000", 2024-09-16T00:09:11.577 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022752369999999998 2024-09-16T00:09:11.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:11.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660803+0000", 2024-09-16T00:09:11.577 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030441499999999998 2024-09-16T00:09:11.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660850+0000", 2024-09-16T00:09:11.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7017000000000001e-05 2024-09-16T00:09:11.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660883+0000", 2024-09-16T00:09:11.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3476000000000003e-05 2024-09-16T00:09:11.578 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.578 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.578 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.578 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:90 2.1b 2:ddb4e4d7:::200.00000002:head [write 2593031~1404 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:11.578 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:11.578 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 7.8372896760000001, 2024-09-16T00:09:11.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011882886, 2024-09-16T00:09:11.579 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.579 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.579 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.579 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:11.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:11.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649119+0000", 2024-09-16T00:09:11.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:09:11.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649125+0000", 2024-09-16T00:09:11.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1650000000000003e-06 2024-09-16T00:09:11.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.581 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649134+0000", 2024-09-16T00:09:11.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5669999999999995e-06 2024-09-16T00:09:11.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.581 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.581 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649141+0000", 2024-09-16T00:09:11.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9040000000000003e-06 2024-09-16T00:09:11.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.581 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.581 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655796+0000", 2024-09-16T00:09:11.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066553649999999999 2024-09-16T00:09:11.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.582 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.582 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655868+0000", 2024-09-16T00:09:11.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2219999999999996e-05 2024-09-16T00:09:11.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.582 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:11.582 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655986+0000", 2024-09-16T00:09:11.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117457 2024-09-16T00:09:11.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.582 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:11.582 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657794+0000", 2024-09-16T00:09:11.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001807894 2024-09-16T00:09:11.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:11.583 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660890+0000", 2024-09-16T00:09:11.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030958449999999998 2024-09-16T00:09:11.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.583 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660966+0000", 2024-09-16T00:09:11.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6382000000000002e-05 2024-09-16T00:09:11.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.583 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.661001+0000", 2024-09-16T00:09:11.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5349000000000003e-05 2024-09-16T00:09:11.583 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.584 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.584 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.584 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:2397 3.29 3:948abdad:::10000000cb5.00000000:head [write 0~1373] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T00:09:11.584 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.655571+0000", 2024-09-16T00:09:11.584 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 7.8308373910000002, 2024-09-16T00:09:11.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052117400000000003, 2024-09-16T00:09:11.584 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:11.584 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:11.584 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.584 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:11.585 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:11.585 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:11.585 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 2397 2024-09-16T00:09:11.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.585 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:11.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:11.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655571+0000", 2024-09-16T00:09:11.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:11.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655571+0000", 2024-09-16T00:09:11.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:11.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:11.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655573+0000", 2024-09-16T00:09:11.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0789999999999999e-06 2024-09-16T00:09:11.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:11.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655580+0000", 2024-09-16T00:09:11.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7859999999999999e-06 2024-09-16T00:09:11.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.587 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:11.587 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655592+0000", 2024-09-16T00:09:11.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2541000000000001e-05 2024-09-16T00:09:11.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.587 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.587 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655601+0000", 2024-09-16T00:09:11.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.721e-06 2024-09-16T00:09:11.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:11.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655663+0000", 2024-09-16T00:09:11.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2287999999999997e-05 2024-09-16T00:09:11.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:11.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655765+0000", 2024-09-16T00:09:11.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000102212 2024-09-16T00:09:11.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:11.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655893+0000", 2024-09-16T00:09:11.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012742499999999999 2024-09-16T00:09:11.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:11.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657022+0000", 2024-09-16T00:09:11.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011293340000000001 2024-09-16T00:09:11.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:11.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660686+0000", 2024-09-16T00:09:11.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036637409999999999 2024-09-16T00:09:11.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:11.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660737+0000", 2024-09-16T00:09:11.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1711e-05 2024-09-16T00:09:11.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:11.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:11.590 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:11.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660782+0000", 2024-09-16T00:09:11.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4901999999999998e-05 2024-09-16T00:09:11.590 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.590 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.590 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.590 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:11.590 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:11.590 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:11.592 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:09:11.592 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:11.591+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:11.693 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:11.692+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:11.705 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:11.705 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:11.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.705 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:4689 3.2c e65/46 3:35f050ce:::1000000160c.00000000:head v 65'91, mlcod=65'89)", 2024-09-16T00:09:11.705 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:11.704281+0000", 2024-09-16T00:09:11.705 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00048029700000000003, 2024-09-16T00:09:11.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049675100000000003, 2024-09-16T00:09:11.705 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:11.706 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:11.706 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:09:11.706 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:11.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:11.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:11.704281+0000", 2024-09-16T00:09:11.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:11.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:11.704281+0000", 2024-09-16T00:09:11.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:11.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:11.704281+0000", 2024-09-16T00:09:11.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4799999999999997e-07 2024-09-16T00:09:11.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:11.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:11.704288+0000", 2024-09-16T00:09:11.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3709999999999996e-06 2024-09-16T00:09:11.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:11.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:11.704299+0000", 2024-09-16T00:09:11.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1076e-05 2024-09-16T00:09:11.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:11.704305+0000", 2024-09-16T00:09:11.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3080000000000004e-06 2024-09-16T00:09:11.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:11.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:11.704357+0000", 2024-09-16T00:09:11.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1681000000000002e-05 2024-09-16T00:09:11.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:11.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:11.704376+0000", 2024-09-16T00:09:11.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9403999999999999e-05 2024-09-16T00:09:11.709 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.709 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:11.709 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.709 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.709 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-16T00:09:11.709 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 1 2024-09-16T00:09:11.709 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:11.715 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:09:11.794 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:11.793+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:11.827 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:11.827 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:11.827 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:11.827 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:11.827 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:11.836 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:09:11.895 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:11.894+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:11.951 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:11.951 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:11.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:11.951 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:11.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.951 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:11.951 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:09:11.951 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.333737325000001, 2024-09-16T00:09:11.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036770370000000002, 2024-09-16T00:09:11.952 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:11.952 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:11.952 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.952 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:11.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:11.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:09:11.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:11.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:09:11.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:11.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615154+0000", 2024-09-16T00:09:11.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1510000000000003e-06 2024-09-16T00:09:11.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:11.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615163+0000", 2024-09-16T00:09:11.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4490000000000005e-06 2024-09-16T00:09:11.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:11.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615182+0000", 2024-09-16T00:09:11.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8876e-05 2024-09-16T00:09:11.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615197+0000", 2024-09-16T00:09:11.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4671000000000001e-05 2024-09-16T00:09:11.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:11.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615349+0000", 2024-09-16T00:09:11.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015258700000000001 2024-09-16T00:09:11.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:11.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:09:11.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000204716 2024-09-16T00:09:11.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:11.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:09:11.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017250800000000001 2024-09-16T00:09:11.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:11.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616919+0000", 2024-09-16T00:09:11.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011921799999999999 2024-09-16T00:09:11.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:11.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:09:11.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018087769999999999 2024-09-16T00:09:11.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:11.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618774+0000", 2024-09-16T00:09:11.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6075999999999997e-05 2024-09-16T00:09:11.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:11.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618826+0000", 2024-09-16T00:09:11.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2046000000000003e-05 2024-09-16T00:09:11.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.957 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:11.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.958 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:09:11.958 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:09:11.958 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.332381537000003, 2024-09-16T00:09:11.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023501059999999998, 2024-09-16T00:09:11.958 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:11.958 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:11.958 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:09:11.958 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:11.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:11.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:09:11.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:11.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:09:11.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:11.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616506+0000", 2024-09-16T00:09:11.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4100000000000001e-06 2024-09-16T00:09:11.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:11.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617073+0000", 2024-09-16T00:09:11.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056725899999999997 2024-09-16T00:09:11.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:11.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617099+0000", 2024-09-16T00:09:11.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5860999999999999e-05 2024-09-16T00:09:11.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617114+0000", 2024-09-16T00:09:11.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5180999999999999e-05 2024-09-16T00:09:11.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:11.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617236+0000", 2024-09-16T00:09:11.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121662 2024-09-16T00:09:11.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:11.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:09:11.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001618733 2024-09-16T00:09:11.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.961 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:11.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.961 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:31 2.d e22/19 2:b50e409b:::mds_snaptable:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:11.961 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:09:11.961 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.331487389000003, 2024-09-16T00:09:11.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021851240000000001, 2024-09-16T00:09:11.962 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:11.962 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:11.962 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.962 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:11.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:11.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:09:11.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:11.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:09:11.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:11.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617400+0000", 2024-09-16T00:09:11.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.28e-06 2024-09-16T00:09:11.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:11.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617410+0000", 2024-09-16T00:09:11.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0179e-05 2024-09-16T00:09:11.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:11.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617435+0000", 2024-09-16T00:09:11.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5221000000000001e-05 2024-09-16T00:09:11.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617444+0000", 2024-09-16T00:09:11.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1989999999999995e-06 2024-09-16T00:09:11.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:11.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617514+0000", 2024-09-16T00:09:11.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9104999999999997e-05 2024-09-16T00:09:11.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:11.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617538+0000", 2024-09-16T00:09:11.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4383999999999999e-05 2024-09-16T00:09:11.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:11.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619536+0000", 2024-09-16T00:09:11.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019982789999999999 2024-09-16T00:09:11.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:11.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619584+0000", 2024-09-16T00:09:11.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7477e-05 2024-09-16T00:09:11.966 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.966 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:11.966 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.966 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:47 3.1a 3:582f7b2d:::10000000226.00000000:head [write 0~5348] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:11.966 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.358277+0000", 2024-09-16T00:09:11.966 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.590609156999999, 2024-09-16T00:09:11.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003561935, 2024-09-16T00:09:11.966 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:11.966 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:11.966 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.966 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:11.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:11.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:11.967 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 47 2024-09-16T00:09:11.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.967 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:11.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:11.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358277+0000", 2024-09-16T00:09:11.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:11.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358277+0000", 2024-09-16T00:09:11.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:11.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358278+0000", 2024-09-16T00:09:11.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.404e-06 2024-09-16T00:09:11.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:11.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358290+0000", 2024-09-16T00:09:11.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1926999999999999e-05 2024-09-16T00:09:11.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:11.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358307+0000", 2024-09-16T00:09:11.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6529e-05 2024-09-16T00:09:11.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358316+0000", 2024-09-16T00:09:11.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1779999999999992e-06 2024-09-16T00:09:11.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:11.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358398+0000", 2024-09-16T00:09:11.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2056000000000002e-05 2024-09-16T00:09:11.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:11.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358488+0000", 2024-09-16T00:09:11.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9761999999999999e-05 2024-09-16T00:09:11.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:11.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358567+0000", 2024-09-16T00:09:11.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9173999999999998e-05 2024-09-16T00:09:11.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:11.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.359485+0000", 2024-09-16T00:09:11.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00091798800000000001 2024-09-16T00:09:11.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:11.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.361743+0000", 2024-09-16T00:09:11.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002258038 2024-09-16T00:09:11.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:11.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.361784+0000", 2024-09-16T00:09:11.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1529999999999997e-05 2024-09-16T00:09:11.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:11.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.361839+0000", 2024-09-16T00:09:11.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4348999999999998e-05 2024-09-16T00:09:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:74 3.13 e57/20 3:c933d8fa:::10000000241.00000000:head v 57'5, mlcod=57'3)", 2024-09-16T00:09:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.428708+0000", 2024-09-16T00:09:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.520178071, 2024-09-16T00:09:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023048140000000001, 2024-09-16T00:09:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.972 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:11.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:11.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428708+0000", 2024-09-16T00:09:11.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:11.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428708+0000", 2024-09-16T00:09:11.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:11.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428709+0000", 2024-09-16T00:09:11.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7000000000000003e-07 2024-09-16T00:09:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428716+0000", 2024-09-16T00:09:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5449999999999998e-06 2024-09-16T00:09:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428730+0000", 2024-09-16T00:09:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3464e-05 2024-09-16T00:09:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428738+0000", 2024-09-16T00:09:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7919999999999993e-06 2024-09-16T00:09:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428788+0000", 2024-09-16T00:09:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0813999999999998e-05 2024-09-16T00:09:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428810+0000", 2024-09-16T00:09:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1543e-05 2024-09-16T00:09:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.430972+0000", 2024-09-16T00:09:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021622130000000001 2024-09-16T00:09:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.431013+0000", 2024-09-16T00:09:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0572999999999999e-05 2024-09-16T00:09:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.976 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:101 3.11 e57/20 3:88015c6d:::1000000025c.00000000:head v 57'1, mlcod=0'0)", 2024-09-16T00:09:11.977 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.484094+0000", 2024-09-16T00:09:11.977 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.464792213000001, 2024-09-16T00:09:11.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030040079999999999, 2024-09-16T00:09:11.977 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:11.977 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:11.977 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.977 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:11.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:11.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484094+0000", 2024-09-16T00:09:11.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:11.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484094+0000", 2024-09-16T00:09:11.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:11.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484094+0000", 2024-09-16T00:09:11.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8999999999999996e-07 2024-09-16T00:09:11.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:11.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484099+0000", 2024-09-16T00:09:11.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6589999999999998e-06 2024-09-16T00:09:11.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:11.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484108+0000", 2024-09-16T00:09:11.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1910000000000006e-06 2024-09-16T00:09:11.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484115+0000", 2024-09-16T00:09:11.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6270000000000002e-06 2024-09-16T00:09:11.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:11.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484163+0000", 2024-09-16T00:09:11.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8137999999999999e-05 2024-09-16T00:09:11.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:11.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484182+0000", 2024-09-16T00:09:11.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8600000000000001e-05 2024-09-16T00:09:11.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:11.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487055+0000", 2024-09-16T00:09:11.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873858 2024-09-16T00:09:11.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:11.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487098+0000", 2024-09-16T00:09:11.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2345000000000002e-05 2024-09-16T00:09:11.981 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.981 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:11.981 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.981 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:132 3.13 e57/20 3:c93e5698:::1000000027d.00000000:head v 57'7, mlcod=57'5)", 2024-09-16T00:09:11.981 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.557897+0000", 2024-09-16T00:09:11.981 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.390988798, 2024-09-16T00:09:11.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029674789999999999, 2024-09-16T00:09:11.981 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:11.981 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:11.982 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.982 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:11.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:11.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557897+0000", 2024-09-16T00:09:11.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:11.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557897+0000", 2024-09-16T00:09:11.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:11.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557898+0000", 2024-09-16T00:09:11.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1800000000000005e-07 2024-09-16T00:09:11.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:11.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557906+0000", 2024-09-16T00:09:11.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8390000000000007e-06 2024-09-16T00:09:11.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:11.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557919+0000", 2024-09-16T00:09:11.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2914999999999999e-05 2024-09-16T00:09:11.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557931+0000", 2024-09-16T00:09:11.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2734999999999999e-05 2024-09-16T00:09:11.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:11.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557982+0000", 2024-09-16T00:09:11.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0293000000000001e-05 2024-09-16T00:09:11.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:11.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557997+0000", 2024-09-16T00:09:11.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5160999999999999e-05 2024-09-16T00:09:11.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:11.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.560830+0000", 2024-09-16T00:09:11.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002832776 2024-09-16T00:09:11.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:11.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.560865+0000", 2024-09-16T00:09:11.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4941999999999999e-05 2024-09-16T00:09:11.986 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.986 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:11.986 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.986 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:243 3.24 e57/20 3:25048551:::100000002f0.00000000:head v 57'4, mlcod=57'2)", 2024-09-16T00:09:11.986 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.799371+0000", 2024-09-16T00:09:11.986 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.149515148000001, 2024-09-16T00:09:11.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037781889999999999, 2024-09-16T00:09:11.986 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:11.986 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:11.986 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.986 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:11.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:11.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799371+0000", 2024-09-16T00:09:11.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:11.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799371+0000", 2024-09-16T00:09:11.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:11.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799372+0000", 2024-09-16T00:09:11.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9100000000000003e-07 2024-09-16T00:09:11.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:11.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799379+0000", 2024-09-16T00:09:11.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.99e-06 2024-09-16T00:09:11.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:11.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799387+0000", 2024-09-16T00:09:11.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5790000000000004e-06 2024-09-16T00:09:11.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799392+0000", 2024-09-16T00:09:11.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5210000000000004e-06 2024-09-16T00:09:11.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:11.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799434+0000", 2024-09-16T00:09:11.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2341e-05 2024-09-16T00:09:11.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:11.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799447+0000", 2024-09-16T00:09:11.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3379e-05 2024-09-16T00:09:11.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:11.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.803117+0000", 2024-09-16T00:09:11.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036697169999999999 2024-09-16T00:09:11.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:11.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.803149+0000", 2024-09-16T00:09:11.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1971000000000003e-05 2024-09-16T00:09:11.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.990 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:11.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.990 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:244 3.1b e57/20 3:da01a9ef:::100000002f1.00000000:head v 57'4, mlcod=57'2)", 2024-09-16T00:09:11.991 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.801331+0000", 2024-09-16T00:09:11.991 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.147555313, 2024-09-16T00:09:11.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021400989999999999, 2024-09-16T00:09:11.991 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:11.991 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:11.991 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.991 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:11.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:11.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.801331+0000", 2024-09-16T00:09:11.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:11.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.801331+0000", 2024-09-16T00:09:11.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:11.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.801331+0000", 2024-09-16T00:09:11.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0900000000000001e-07 2024-09-16T00:09:11.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:11.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.801366+0000", 2024-09-16T00:09:11.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5006999999999999e-05 2024-09-16T00:09:11.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:11.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.801376+0000", 2024-09-16T00:09:11.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1980000000000007e-06 2024-09-16T00:09:11.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.801380+0000", 2024-09-16T00:09:11.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5560000000000001e-06 2024-09-16T00:09:11.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:11.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.801438+0000", 2024-09-16T00:09:11.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7868999999999998e-05 2024-09-16T00:09:11.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:11.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.801453+0000", 2024-09-16T00:09:11.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5126999999999999e-05 2024-09-16T00:09:11.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:11.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.803404+0000", 2024-09-16T00:09:11.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001950799 2024-09-16T00:09:11.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:11.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.803471+0000", 2024-09-16T00:09:11.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6834000000000004e-05 2024-09-16T00:09:11.995 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.995 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:11.995 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:11.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.995 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:319 3.1a 3:5a6e0114:::10000000350.00000000:head [write 0~3550] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:11.995 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.023452+0000", 2024-09-16T00:09:11.995 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.925434246, 2024-09-16T00:09:11.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597186, 2024-09-16T00:09:11.995 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:11.996 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:11.996 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:11.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:11.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:11.996 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:11.996 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 319 2024-09-16T00:09:11.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.996 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:11.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:11.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023452+0000", 2024-09-16T00:09:11.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:11.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023452+0000", 2024-09-16T00:09:11.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:11.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:11.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023452+0000", 2024-09-16T00:09:11.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5199999999999996e-07 2024-09-16T00:09:11.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:11.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023457+0000", 2024-09-16T00:09:11.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8289999999999998e-06 2024-09-16T00:09:11.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:11.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023467+0000", 2024-09-16T00:09:11.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8190000000000003e-06 2024-09-16T00:09:11.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:11.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023472+0000", 2024-09-16T00:09:11.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0810000000000001e-06 2024-09-16T00:09:11.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:11.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023524+0000", 2024-09-16T00:09:11.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2017e-05 2024-09-16T00:09:11.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:11.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023649+0000", 2024-09-16T00:09:11.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491999999999999 2024-09-16T00:09:11.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:11.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:11.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023772+0000", 2024-09-16T00:09:11.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012327599999999999 2024-09-16T00:09:11.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:11.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:12.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.024507+0000", 2024-09-16T00:09:12.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073499100000000005 2024-09-16T00:09:12.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:12.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.026948+0000", 2024-09-16T00:09:12.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002441033 2024-09-16T00:09:12.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.026997+0000", 2024-09-16T00:09:12.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8778e-05 2024-09-16T00:09:12.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.027049+0000", 2024-09-16T00:09:12.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690000000000001e-05 2024-09-16T00:09:12.001 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.001 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.001 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.001 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:323 3.3 3:c319d237:::10000000354.00000000:head [write 0~1622] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:12.001 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.032376+0000", 2024-09-16T00:09:12.001 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.916510223, 2024-09-16T00:09:12.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039255940000000001, 2024-09-16T00:09:12.002 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.002 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.002 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:12.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:12.002 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:12.002 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 323 2024-09-16T00:09:12.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.002 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032376+0000", 2024-09-16T00:09:12.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032376+0000", 2024-09-16T00:09:12.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032377+0000", 2024-09-16T00:09:12.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.42e-06 2024-09-16T00:09:12.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032383+0000", 2024-09-16T00:09:12.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9900000000000002e-06 2024-09-16T00:09:12.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032394+0000", 2024-09-16T00:09:12.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1216999999999999e-05 2024-09-16T00:09:12.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032400+0000", 2024-09-16T00:09:12.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8039999999999998e-06 2024-09-16T00:09:12.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032451+0000", 2024-09-16T00:09:12.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0612999999999998e-05 2024-09-16T00:09:12.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032536+0000", 2024-09-16T00:09:12.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4956000000000004e-05 2024-09-16T00:09:12.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:12.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032614+0000", 2024-09-16T00:09:12.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8716999999999999e-05 2024-09-16T00:09:12.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:12.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.033109+0000", 2024-09-16T00:09:12.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049412399999999997 2024-09-16T00:09:12.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:12.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.036222+0000", 2024-09-16T00:09:12.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031137949999999999 2024-09-16T00:09:12.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.036264+0000", 2024-09-16T00:09:12.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1572000000000001e-05 2024-09-16T00:09:12.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.036301+0000", 2024-09-16T00:09:12.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7385999999999997e-05 2024-09-16T00:09:12.007 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.007 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.007 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.007 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:324 3.29 e58/20 3:960c7fc3:::10000000355.00000000:head v 58'7, mlcod=57'5)", 2024-09-16T00:09:12.007 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.035116+0000", 2024-09-16T00:09:12.008 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.913770449999999, 2024-09-16T00:09:12.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029776550000000001, 2024-09-16T00:09:12.008 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.008 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.008 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.008 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035116+0000", 2024-09-16T00:09:12.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035116+0000", 2024-09-16T00:09:12.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035116+0000", 2024-09-16T00:09:12.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.91e-07 2024-09-16T00:09:12.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035119+0000", 2024-09-16T00:09:12.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.755e-06 2024-09-16T00:09:12.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035130+0000", 2024-09-16T00:09:12.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169999999999999e-05 2024-09-16T00:09:12.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035135+0000", 2024-09-16T00:09:12.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0019999999999997e-06 2024-09-16T00:09:12.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035175+0000", 2024-09-16T00:09:12.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0027999999999997e-05 2024-09-16T00:09:12.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035188+0000", 2024-09-16T00:09:12.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3386e-05 2024-09-16T00:09:12.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.038054+0000", 2024-09-16T00:09:12.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002866324 2024-09-16T00:09:12.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.038093+0000", 2024-09-16T00:09:12.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8798999999999999e-05 2024-09-16T00:09:12.012 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.012 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.012 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.012 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:392 3.f e58/20 3:f03826fe:::100000003a5.00000000:head v 58'8, mlcod=57'6)", 2024-09-16T00:09:12.012 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.307961+0000", 2024-09-16T00:09:12.013 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.640924736000001, 2024-09-16T00:09:12.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003252202, 2024-09-16T00:09:12.013 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.013 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.013 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.013 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.307961+0000", 2024-09-16T00:09:12.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.307961+0000", 2024-09-16T00:09:12.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.307962+0000", 2024-09-16T00:09:12.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5000000000000001e-07 2024-09-16T00:09:12.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.307993+0000", 2024-09-16T00:09:12.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0414000000000001e-05 2024-09-16T00:09:12.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.308003+0000", 2024-09-16T00:09:12.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0618999999999999e-05 2024-09-16T00:09:12.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.308010+0000", 2024-09-16T00:09:12.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5200000000000003e-06 2024-09-16T00:09:12.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.308062+0000", 2024-09-16T00:09:12.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.185e-05 2024-09-16T00:09:12.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.308083+0000", 2024-09-16T00:09:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1489e-05 2024-09-16T00:09:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.311178+0000", 2024-09-16T00:09:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030948630000000002 2024-09-16T00:09:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.311213+0000", 2024-09-16T00:09:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5497000000000003e-05 2024-09-16T00:09:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:411 3.29 e58/20 3:96aa9846:::100000003bf.00000000:head v 58'9, mlcod=58'7)", 2024-09-16T00:09:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.363807+0000", 2024-09-16T00:09:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.585079191, 2024-09-16T00:09:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032126009999999998, 2024-09-16T00:09:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.017 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363807+0000", 2024-09-16T00:09:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363807+0000", 2024-09-16T00:09:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363807+0000", 2024-09-16T00:09:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4899999999999995e-07 2024-09-16T00:09:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363812+0000", 2024-09-16T00:09:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4769999999999997e-06 2024-09-16T00:09:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363822+0000", 2024-09-16T00:09:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7950000000000002e-06 2024-09-16T00:09:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363827+0000", 2024-09-16T00:09:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8049999999999997e-06 2024-09-16T00:09:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363867+0000", 2024-09-16T00:09:12.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0694e-05 2024-09-16T00:09:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363880+0000", 2024-09-16T00:09:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3022e-05 2024-09-16T00:09:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.366979+0000", 2024-09-16T00:09:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003099161 2024-09-16T00:09:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.367019+0000", 2024-09-16T00:09:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9997999999999999e-05 2024-09-16T00:09:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:500 3.13 e58/20 3:c8c66365:::1000000042a.00000000:head v 58'14, mlcod=58'12)", 2024-09-16T00:09:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.760251+0000", 2024-09-16T00:09:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.18863539, 2024-09-16T00:09:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002913793, 2024-09-16T00:09:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.022 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760251+0000", 2024-09-16T00:09:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760251+0000", 2024-09-16T00:09:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760251+0000", 2024-09-16T00:09:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7400000000000003e-07 2024-09-16T00:09:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760257+0000", 2024-09-16T00:09:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0000000000000002e-06 2024-09-16T00:09:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760268+0000", 2024-09-16T00:09:12.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0397999999999999e-05 2024-09-16T00:09:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760273+0000", 2024-09-16T00:09:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2889999999999997e-06 2024-09-16T00:09:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760322+0000", 2024-09-16T00:09:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9193999999999997e-05 2024-09-16T00:09:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760343+0000", 2024-09-16T00:09:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1228000000000001e-05 2024-09-16T00:09:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.763123+0000", 2024-09-16T00:09:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027797189999999999 2024-09-16T00:09:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.763164+0000", 2024-09-16T00:09:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1291000000000002e-05 2024-09-16T00:09:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.026 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:811 3.1c e59/20 3:3b54038d:::100000005a1.00000000:head v 59'10, mlcod=59'8)", 2024-09-16T00:09:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:58.585639+0000", 2024-09-16T00:09:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.363246659, 2024-09-16T00:09:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002996163, 2024-09-16T00:09:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585639+0000", 2024-09-16T00:09:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585639+0000", 2024-09-16T00:09:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585640+0000", 2024-09-16T00:09:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1800000000000005e-07 2024-09-16T00:09:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585657+0000", 2024-09-16T00:09:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7249e-05 2024-09-16T00:09:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585669+0000", 2024-09-16T00:09:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1212e-05 2024-09-16T00:09:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585675+0000", 2024-09-16T00:09:12.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6200000000000001e-06 2024-09-16T00:09:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585723+0000", 2024-09-16T00:09:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7991000000000002e-05 2024-09-16T00:09:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585738+0000", 2024-09-16T00:09:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4578e-05 2024-09-16T00:09:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588596+0000", 2024-09-16T00:09:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028585780000000001 2024-09-16T00:09:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588635+0000", 2024-09-16T00:09:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9116999999999998e-05 2024-09-16T00:09:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:1618 3.1c e61/20 3:3b08c232:::1000000094e.00000000:head v 61'20, mlcod=61'18)", 2024-09-16T00:09:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.253213+0000", 2024-09-16T00:09:12.031 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.695673153, 2024-09-16T00:09:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025388519999999999, 2024-09-16T00:09:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253213+0000", 2024-09-16T00:09:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253213+0000", 2024-09-16T00:09:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253214+0000", 2024-09-16T00:09:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4600000000000004e-07 2024-09-16T00:09:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253231+0000", 2024-09-16T00:09:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6982000000000001e-05 2024-09-16T00:09:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253243+0000", 2024-09-16T00:09:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2412000000000001e-05 2024-09-16T00:09:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253250+0000", 2024-09-16T00:09:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.922e-06 2024-09-16T00:09:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253293+0000", 2024-09-16T00:09:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2823e-05 2024-09-16T00:09:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253309+0000", 2024-09-16T00:09:12.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6049000000000002e-05 2024-09-16T00:09:12.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255709+0000", 2024-09-16T00:09:12.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002400639 2024-09-16T00:09:12.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255752+0000", 2024-09-16T00:09:12.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2279e-05 2024-09-16T00:09:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:1691 3.a e61/46 3:518183a8:::10000000999.00000000:head v 61'30, mlcod=61'28)", 2024-09-16T00:09:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.411769+0000", 2024-09-16T00:09:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.537117047000001, 2024-09-16T00:09:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027805550000000001, 2024-09-16T00:09:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411769+0000", 2024-09-16T00:09:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411769+0000", 2024-09-16T00:09:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411770+0000", 2024-09-16T00:09:12.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3e-07 2024-09-16T00:09:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411787+0000", 2024-09-16T00:09:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7632999999999999e-05 2024-09-16T00:09:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411798+0000", 2024-09-16T00:09:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.094e-05 2024-09-16T00:09:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411807+0000", 2024-09-16T00:09:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3930000000000008e-06 2024-09-16T00:09:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411860+0000", 2024-09-16T00:09:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3477999999999999e-05 2024-09-16T00:09:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411879+0000", 2024-09-16T00:09:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9148999999999999e-05 2024-09-16T00:09:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.414504+0000", 2024-09-16T00:09:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026246759999999998 2024-09-16T00:09:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.414549+0000", 2024-09-16T00:09:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5556000000000002e-05 2024-09-16T00:09:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.040 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:1889 3.f e61/20 3:f2db18a4:::10000000a6a.00000000:head v 61'27, mlcod=61'25)", 2024-09-16T00:09:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.171388+0000", 2024-09-16T00:09:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.7774978800000003, 2024-09-16T00:09:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00302002, 2024-09-16T00:09:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171388+0000", 2024-09-16T00:09:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171388+0000", 2024-09-16T00:09:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171389+0000", 2024-09-16T00:09:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0470000000000001e-06 2024-09-16T00:09:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171399+0000", 2024-09-16T00:09:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0088000000000001e-05 2024-09-16T00:09:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171415+0000", 2024-09-16T00:09:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5526e-05 2024-09-16T00:09:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171422+0000", 2024-09-16T00:09:12.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.764e-06 2024-09-16T00:09:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171488+0000", 2024-09-16T00:09:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6308000000000003e-05 2024-09-16T00:09:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171511+0000", 2024-09-16T00:09:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2891000000000002e-05 2024-09-16T00:09:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.174359+0000", 2024-09-16T00:09:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028479669999999999 2024-09-16T00:09:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.174408+0000", 2024-09-16T00:09:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9428999999999997e-05 2024-09-16T00:09:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:161 2.b e64/19 2:d5c7a900:::200.00000003:head v 64'5, mlcod=61'3)", 2024-09-16T00:09:12.045 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:07.541046+0000", 2024-09-16T00:09:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4078395620000004, 2024-09-16T00:09:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010081302, 2024-09-16T00:09:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.541046+0000", 2024-09-16T00:09:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.541046+0000", 2024-09-16T00:09:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.541048+0000", 2024-09-16T00:09:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5859999999999999e-06 2024-09-16T00:09:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543134+0000", 2024-09-16T00:09:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020863589999999999 2024-09-16T00:09:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543155+0000", 2024-09-16T00:09:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0344999999999999e-05 2024-09-16T00:09:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543167+0000", 2024-09-16T00:09:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2071e-05 2024-09-16T00:09:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543242+0000", 2024-09-16T00:09:12.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5108e-05 2024-09-16T00:09:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543268+0000", 2024-09-16T00:09:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6471000000000001e-05 2024-09-16T00:09:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551087+0000", 2024-09-16T00:09:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0078188670000000002 2024-09-16T00:09:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551128+0000", 2024-09-16T00:09:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0494999999999998e-05 2024-09-16T00:09:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.050 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.050 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:12.051 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:09:12.052 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:12.051+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:12.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.059+0000 7fbe7d7fa640 1 -- 172.21.15.139:0/4246979151 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v66) ==== 91+0+0 (secure 0 0 0) 0x7fbe7805cc30 con 0x7fbe880c8b50 2024-09-16T00:09:12.060 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is unset 2024-09-16T00:09:12.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.062+0000 7fbe8f1cf640 1 -- 172.21.15.139:0/4246979151 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbe7406c2b0 msgr2=0x7fbe7406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:12.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.062+0000 7fbe8f1cf640 1 --2- 172.21.15.139:0/4246979151 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbe7406c2b0 0x7fbe7406e770 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbe84005880 tx=0x7fbe84009040 comp rx=0 tx=0).stop 2024-09-16T00:09:12.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.062+0000 7fbe8f1cf640 1 -- 172.21.15.139:0/4246979151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe880c8b50 msgr2=0x7fbe880b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:12.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.062+0000 7fbe8f1cf640 1 --2- 172.21.15.139:0/4246979151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe880c8b50 0x7fbe880b1ce0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800c440 tx=0x7fbe78001e70 comp rx=0 tx=0).stop 2024-09-16T00:09:12.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.063+0000 7fbe8f1cf640 1 -- 172.21.15.139:0/4246979151 shutdown_connections 2024-09-16T00:09:12.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.063+0000 7fbe8f1cf640 1 --2- 172.21.15.139:0/4246979151 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbe7406c2b0 0x7fbe7406e770 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.063+0000 7fbe8f1cf640 1 --2- 172.21.15.139:0/4246979151 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbe8815da90 0x7fbe880cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.063+0000 7fbe8f1cf640 1 --2- 172.21.15.139:0/4246979151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe880c8b50 0x7fbe880b1ce0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.063+0000 7fbe8f1cf640 1 --2- 172.21.15.139:0/4246979151 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbe88070d90 0x7fbe880c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.063+0000 7fbe8f1cf640 1 -- 172.21.15.139:0/4246979151 >> 172.21.15.139:0/4246979151 conn(0x7fbe8806c150 msgr2=0x7fbe88070410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:12.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.063+0000 7fbe8f1cf640 1 -- 172.21.15.139:0/4246979151 shutdown_connections 2024-09-16T00:09:12.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.063+0000 7fbe8f1cf640 1 -- 172.21.15.139:0/4246979151 wait complete. 2024-09-16T00:09:12.152 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:12.151+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:12.199 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:12.199 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:12.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.199 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:4839 3.3c 3:3eba03ff:::100000016b5.00000000:head [write 0~462 in=462b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-16T00:09:12.199 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:12.197933+0000", 2024-09-16T00:09:12.199 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00076160300000000002, 2024-09-16T00:09:12.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00077905900000000002, 2024-09-16T00:09:12.199 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.199 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.199 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "waiting for sub ops", 2024-09-16T00:09:12.199 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:12.199 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:12.200 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:12.200 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 4839 2024-09-16T00:09:12.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.200 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.197933+0000", 2024-09-16T00:09:12.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.197933+0000", 2024-09-16T00:09:12.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.197933+0000", 2024-09-16T00:09:12.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5499999999999997e-07 2024-09-16T00:09:12.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.197938+0000", 2024-09-16T00:09:12.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8840000000000002e-06 2024-09-16T00:09:12.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.197952+0000", 2024-09-16T00:09:12.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.396e-05 2024-09-16T00:09:12.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.197960+0000", 2024-09-16T00:09:12.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5240000000000003e-06 2024-09-16T00:09:12.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.198001+0000", 2024-09-16T00:09:12.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0905999999999998e-05 2024-09-16T00:09:12.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.198079+0000", 2024-09-16T00:09:12.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7923999999999995e-05 2024-09-16T00:09:12.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:12.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.198155+0000", 2024-09-16T00:09:12.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5859000000000007e-05 2024-09-16T00:09:12.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:12.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.198580+0000", 2024-09-16T00:09:12.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00042559800000000001 2024-09-16T00:09:12.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.203 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.204 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.204 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-16T00:09:12.204 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 1 2024-09-16T00:09:12.204 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:12.207 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:09:12.252 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:12.251+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:12.353 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:12.352+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:12.363 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:12.363 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:12.363 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:12.363 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:12.363 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:12.373 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:09:12.458 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:12.458+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:12.532 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:14 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.914128452, 2024-09-16T00:09:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029190470000000001, 2024-09-16T00:09:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.533 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615982+0000", 2024-09-16T00:09:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7439999999999999e-06 2024-09-16T00:09:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:09:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3743999999999999e-05 2024-09-16T00:09:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2255e-05 2024-09-16T00:09:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616029+0000", 2024-09-16T00:09:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1175999999999999e-05 2024-09-16T00:09:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616098+0000", 2024-09-16T00:09:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9153999999999996e-05 2024-09-16T00:09:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616132+0000", 2024-09-16T00:09:12.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3924000000000002e-05 2024-09-16T00:09:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:09:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027226189999999999 2024-09-16T00:09:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618899+0000", 2024-09-16T00:09:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4431000000000002e-05 2024-09-16T00:09:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:30 2.15 e22/19 2:aa448500:::500.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:09:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:09:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.911844146999996, 2024-09-16T00:09:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058031250000000001, 2024-09-16T00:09:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:09:12.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:09:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618265+0000", 2024-09-16T00:09:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9200000000000001e-07 2024-09-16T00:09:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618271+0000", 2024-09-16T00:09:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9580000000000004e-06 2024-09-16T00:09:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618279+0000", 2024-09-16T00:09:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2239999999999995e-06 2024-09-16T00:09:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618285+0000", 2024-09-16T00:09:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-06 2024-09-16T00:09:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618567+0000", 2024-09-16T00:09:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028204400000000001 2024-09-16T00:09:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:09:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9678999999999999e-05 2024-09-16T00:09:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624024+0000", 2024-09-16T00:09:12.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054376720000000002 2024-09-16T00:09:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624068+0000", 2024-09-16T00:09:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:09:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:20 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:09:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.911487782999998, 2024-09-16T00:09:12.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028808739999999998, 2024-09-16T00:09:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:09:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:09:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618622+0000", 2024-09-16T00:09:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1260000000000001e-06 2024-09-16T00:09:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618631+0000", 2024-09-16T00:09:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4700000000000008e-06 2024-09-16T00:09:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618645+0000", 2024-09-16T00:09:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3984e-05 2024-09-16T00:09:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:09:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2101e-05 2024-09-16T00:09:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618720+0000", 2024-09-16T00:09:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2761000000000004e-05 2024-09-16T00:09:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618740+0000", 2024-09-16T00:09:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9409e-05 2024-09-16T00:09:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621436+0000", 2024-09-16T00:09:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002696211 2024-09-16T00:09:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621502+0000", 2024-09-16T00:09:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5811999999999999e-05 2024-09-16T00:09:12.546 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:21 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:09:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 93.910664495000006, 2024-09-16T00:09:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046318540000000004, 2024-09-16T00:09:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:09:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:09:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619445+0000", 2024-09-16T00:09:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.083e-06 2024-09-16T00:09:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619455+0000", 2024-09-16T00:09:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5920000000000007e-06 2024-09-16T00:09:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619467+0000", 2024-09-16T00:09:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2711000000000001e-05 2024-09-16T00:09:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619476+0000", 2024-09-16T00:09:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0139999999999998e-06 2024-09-16T00:09:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619529+0000", 2024-09-16T00:09:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3078999999999998e-05 2024-09-16T00:09:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619548+0000", 2024-09-16T00:09:12.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8952000000000001e-05 2024-09-16T00:09:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624037+0000", 2024-09-16T00:09:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044895730000000002 2024-09-16T00:09:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624076+0000", 2024-09-16T00:09:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8850000000000002e-05 2024-09-16T00:09:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:93 3.5 3:a3a50212:::10000000254.00000000:head [write 0~6832] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.467788+0000", 2024-09-16T00:09:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 16.062320745000001, 2024-09-16T00:09:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001903882, 2024-09-16T00:09:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93 2024-09-16T00:09:12.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467788+0000", 2024-09-16T00:09:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467788+0000", 2024-09-16T00:09:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467789+0000", 2024-09-16T00:09:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4399999999999999e-07 2024-09-16T00:09:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467798+0000", 2024-09-16T00:09:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0510000000000002e-06 2024-09-16T00:09:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467809+0000", 2024-09-16T00:09:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0859000000000001e-05 2024-09-16T00:09:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467814+0000", 2024-09-16T00:09:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0409999999999997e-06 2024-09-16T00:09:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467860+0000", 2024-09-16T00:09:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6604999999999998e-05 2024-09-16T00:09:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467935+0000", 2024-09-16T00:09:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4417000000000003e-05 2024-09-16T00:09:12.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.468012+0000", 2024-09-16T00:09:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6971000000000006e-05 2024-09-16T00:09:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.468821+0000", 2024-09-16T00:09:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00080984099999999995 2024-09-16T00:09:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:12.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.469636+0000", 2024-09-16T00:09:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00081428499999999999 2024-09-16T00:09:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.469668+0000", 2024-09-16T00:09:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2322e-05 2024-09-16T00:09:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.469692+0000", 2024-09-16T00:09:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3745999999999999e-05 2024-09-16T00:09:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:249 3.28 3:14c9b427:::100000002f7.00000000:head [write 0~5746] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:12.558 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.812683+0000", 2024-09-16T00:09:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.717425257, 2024-09-16T00:09:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019975150000000001, 2024-09-16T00:09:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 249 2024-09-16T00:09:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812683+0000", 2024-09-16T00:09:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812683+0000", 2024-09-16T00:09:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812684+0000", 2024-09-16T00:09:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1100000000000004e-07 2024-09-16T00:09:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812708+0000", 2024-09-16T00:09:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4074e-05 2024-09-16T00:09:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812736+0000", 2024-09-16T00:09:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7767e-05 2024-09-16T00:09:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812741+0000", 2024-09-16T00:09:12.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3319999999999999e-06 2024-09-16T00:09:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812790+0000", 2024-09-16T00:09:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8294000000000002e-05 2024-09-16T00:09:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812881+0000", 2024-09-16T00:09:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1085e-05 2024-09-16T00:09:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:12.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812960+0000", 2024-09-16T00:09:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8824000000000003e-05 2024-09-16T00:09:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.813562+0000", 2024-09-16T00:09:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00060270500000000004 2024-09-16T00:09:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.814630+0000", 2024-09-16T00:09:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010679489999999999 2024-09-16T00:09:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.814660+0000", 2024-09-16T00:09:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.936e-05 2024-09-16T00:09:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.814681+0000", 2024-09-16T00:09:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1214000000000001e-05 2024-09-16T00:09:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.564 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:311 3.2a 3:548c374b:::10000000347.00000000:head [write 0~4872] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.002861+0000", 2024-09-16T00:09:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.527247981, 2024-09-16T00:09:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001734665, 2024-09-16T00:09:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 311 2024-09-16T00:09:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002861+0000", 2024-09-16T00:09:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002861+0000", 2024-09-16T00:09:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002863+0000", 2024-09-16T00:09:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0540000000000002e-06 2024-09-16T00:09:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002877+0000", 2024-09-16T00:09:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4677e-05 2024-09-16T00:09:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002896+0000", 2024-09-16T00:09:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9038999999999998e-05 2024-09-16T00:09:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002905+0000", 2024-09-16T00:09:12.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0599999999999997e-06 2024-09-16T00:09:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002973+0000", 2024-09-16T00:09:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7203e-05 2024-09-16T00:09:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.003111+0000", 2024-09-16T00:09:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013858100000000001 2024-09-16T00:09:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:12.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.003269+0000", 2024-09-16T00:09:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015809 2024-09-16T00:09:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.004171+0000", 2024-09-16T00:09:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00090205600000000004 2024-09-16T00:09:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.004494+0000", 2024-09-16T00:09:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000322126 2024-09-16T00:09:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.004549+0000", 2024-09-16T00:09:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5049000000000001e-05 2024-09-16T00:09:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.004595+0000", 2024-09-16T00:09:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6730000000000002e-05 2024-09-16T00:09:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:333 3.c 3:329df26d:::1000000035f.00000000:head [write 0~657] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:12.570 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.059938+0000", 2024-09-16T00:09:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.470170282, 2024-09-16T00:09:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001777437, 2024-09-16T00:09:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 333 2024-09-16T00:09:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059938+0000", 2024-09-16T00:09:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059938+0000", 2024-09-16T00:09:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059940+0000", 2024-09-16T00:09:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1129999999999999e-06 2024-09-16T00:09:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059953+0000", 2024-09-16T00:09:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.278e-05 2024-09-16T00:09:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059980+0000", 2024-09-16T00:09:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6533000000000002e-05 2024-09-16T00:09:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059994+0000", 2024-09-16T00:09:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4049999999999999e-05 2024-09-16T00:09:12.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.060136+0000", 2024-09-16T00:09:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014184399999999999 2024-09-16T00:09:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.060396+0000", 2024-09-16T00:09:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026020600000000002 2024-09-16T00:09:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.060540+0000", 2024-09-16T00:09:12.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143838 2024-09-16T00:09:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.061336+0000", 2024-09-16T00:09:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00079669099999999998 2024-09-16T00:09:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.061621+0000", 2024-09-16T00:09:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028509599999999998 2024-09-16T00:09:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.061674+0000", 2024-09-16T00:09:12.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2685000000000001e-05 2024-09-16T00:09:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.061716+0000", 2024-09-16T00:09:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1600999999999997e-05 2024-09-16T00:09:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:490 3.d e58/51 3:b3311d72:::1000000041b.00000000:head v 58'8, mlcod=58'6)", 2024-09-16T00:09:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.728943+0000", 2024-09-16T00:09:12.576 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.801165983000001, 2024-09-16T00:09:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002073142, 2024-09-16T00:09:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728943+0000", 2024-09-16T00:09:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728943+0000", 2024-09-16T00:09:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728943+0000", 2024-09-16T00:09:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:09:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728949+0000", 2024-09-16T00:09:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T00:09:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728958+0000", 2024-09-16T00:09:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3049999999999998e-06 2024-09-16T00:09:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728964+0000", 2024-09-16T00:09:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9699999999999996e-06 2024-09-16T00:09:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.729018+0000", 2024-09-16T00:09:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4045000000000002e-05 2024-09-16T00:09:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.729039+0000", 2024-09-16T00:09:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1098999999999999e-05 2024-09-16T00:09:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.730988+0000", 2024-09-16T00:09:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019486919999999999 2024-09-16T00:09:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.731016+0000", 2024-09-16T00:09:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7971e-05 2024-09-16T00:09:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:681 3.17 e59/20 3:ea55746f:::1000000050b.00000000:head v 59'12, mlcod=59'10)", 2024-09-16T00:09:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:58.265463+0000", 2024-09-16T00:09:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.264645107, 2024-09-16T00:09:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030780130000000001, 2024-09-16T00:09:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.581 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265463+0000", 2024-09-16T00:09:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265463+0000", 2024-09-16T00:09:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265464+0000", 2024-09-16T00:09:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7999999999999995e-07 2024-09-16T00:09:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265471+0000", 2024-09-16T00:09:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5010000000000001e-06 2024-09-16T00:09:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265480+0000", 2024-09-16T00:09:12.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.302e-06 2024-09-16T00:09:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265485+0000", 2024-09-16T00:09:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:09:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265526+0000", 2024-09-16T00:09:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1229000000000001e-05 2024-09-16T00:09:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265539+0000", 2024-09-16T00:09:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3083e-05 2024-09-16T00:09:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.268498+0000", 2024-09-16T00:09:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029594320000000001 2024-09-16T00:09:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.268542+0000", 2024-09-16T00:09:12.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3207000000000002e-05 2024-09-16T00:09:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:811 3.1c 3:3b54038d:::100000005a1.00000000:head [write 0~2412] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:58.585366+0000", 2024-09-16T00:09:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.944743076, 2024-09-16T00:09:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003510216, 2024-09-16T00:09:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.586 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 811 2024-09-16T00:09:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585366+0000", 2024-09-16T00:09:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585366+0000", 2024-09-16T00:09:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585366+0000", 2024-09-16T00:09:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5300000000000003e-07 2024-09-16T00:09:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585371+0000", 2024-09-16T00:09:12.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2849999999999996e-06 2024-09-16T00:09:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585380+0000", 2024-09-16T00:09:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9219999999999992e-06 2024-09-16T00:09:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585390+0000", 2024-09-16T00:09:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2960000000000004e-06 2024-09-16T00:09:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585436+0000", 2024-09-16T00:09:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5874000000000002e-05 2024-09-16T00:09:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585510+0000", 2024-09-16T00:09:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4070000000000001e-05 2024-09-16T00:09:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585586+0000", 2024-09-16T00:09:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6044999999999994e-05 2024-09-16T00:09:12.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.586070+0000", 2024-09-16T00:09:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048407300000000001 2024-09-16T00:09:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588779+0000", 2024-09-16T00:09:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027090339999999999 2024-09-16T00:09:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588834+0000", 2024-09-16T00:09:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5513000000000002e-05 2024-09-16T00:09:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588876+0000", 2024-09-16T00:09:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1350999999999998e-05 2024-09-16T00:09:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.592 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:1617 3.36 e61/46 3:6fb28598:::1000000094d.00000000:head v 61'29, mlcod=61'27)", 2024-09-16T00:09:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.251467+0000", 2024-09-16T00:09:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.278641421, 2024-09-16T00:09:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040699999999999998, 2024-09-16T00:09:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251467+0000", 2024-09-16T00:09:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251467+0000", 2024-09-16T00:09:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251468+0000", 2024-09-16T00:09:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6400000000000001e-07 2024-09-16T00:09:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251478+0000", 2024-09-16T00:09:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0057e-05 2024-09-16T00:09:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251489+0000", 2024-09-16T00:09:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0992000000000001e-05 2024-09-16T00:09:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251494+0000", 2024-09-16T00:09:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8999999999999997e-06 2024-09-16T00:09:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251537+0000", 2024-09-16T00:09:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3513999999999997e-05 2024-09-16T00:09:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251550+0000", 2024-09-16T00:09:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3083e-05 2024-09-16T00:09:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255500+0000", 2024-09-16T00:09:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039494090000000001 2024-09-16T00:09:12.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255537+0000", 2024-09-16T00:09:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280999999999998e-05 2024-09-16T00:09:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:1618 3.1c 3:3b08c232:::1000000094e.00000000:head [write 0~2553] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T00:09:12.597 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.252928+0000", 2024-09-16T00:09:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.277180279, 2024-09-16T00:09:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030458360000000001, 2024-09-16T00:09:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1618 2024-09-16T00:09:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252928+0000", 2024-09-16T00:09:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252928+0000", 2024-09-16T00:09:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252929+0000", 2024-09-16T00:09:12.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.13e-06 2024-09-16T00:09:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252939+0000", 2024-09-16T00:09:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8239999999999995e-06 2024-09-16T00:09:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252950+0000", 2024-09-16T00:09:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251999999999999e-05 2024-09-16T00:09:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252962+0000", 2024-09-16T00:09:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2332e-05 2024-09-16T00:09:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253005+0000", 2024-09-16T00:09:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3380000000000001e-05 2024-09-16T00:09:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253084+0000", 2024-09-16T00:09:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9097000000000005e-05 2024-09-16T00:09:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253162+0000", 2024-09-16T00:09:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7890000000000001e-05 2024-09-16T00:09:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255878+0000", 2024-09-16T00:09:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027157589999999999 2024-09-16T00:09:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255892+0000", 2024-09-16T00:09:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4329e-05 2024-09-16T00:09:12.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255940+0000", 2024-09-16T00:09:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7979000000000003e-05 2024-09-16T00:09:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255974+0000", 2024-09-16T00:09:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3864e-05 2024-09-16T00:09:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:84 2.17 e61/19 2:ee44d6a7:::10000000261.03e00000:head v 61'1, mlcod=0'0)", 2024-09-16T00:09:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.077563+0000", 2024-09-16T00:09:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.452545895, 2024-09-16T00:09:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033709450000000002, 2024-09-16T00:09:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077563+0000", 2024-09-16T00:09:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077563+0000", 2024-09-16T00:09:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077563+0000", 2024-09-16T00:09:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.61e-07 2024-09-16T00:09:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077577+0000", 2024-09-16T00:09:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3108000000000001e-05 2024-09-16T00:09:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077587+0000", 2024-09-16T00:09:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0273e-05 2024-09-16T00:09:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077593+0000", 2024-09-16T00:09:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5209999999999999e-06 2024-09-16T00:09:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077641+0000", 2024-09-16T00:09:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7579e-05 2024-09-16T00:09:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077658+0000", 2024-09-16T00:09:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7297999999999999e-05 2024-09-16T00:09:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080906+0000", 2024-09-16T00:09:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032481760000000002 2024-09-16T00:09:12.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080934+0000", 2024-09-16T00:09:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7229e-05 2024-09-16T00:09:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:80 2.12 2:4a871970:::10000000261.03600000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.077908+0000", 2024-09-16T00:09:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.452200918000001, 2024-09-16T00:09:12.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030856009999999999, 2024-09-16T00:09:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077908+0000", 2024-09-16T00:09:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077908+0000", 2024-09-16T00:09:12.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077909+0000", 2024-09-16T00:09:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5719999999999999e-06 2024-09-16T00:09:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077920+0000", 2024-09-16T00:09:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0631e-05 2024-09-16T00:09:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077930+0000", 2024-09-16T00:09:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0180000000000001e-05 2024-09-16T00:09:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077937+0000", 2024-09-16T00:09:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:09:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077993+0000", 2024-09-16T00:09:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6174999999999997e-05 2024-09-16T00:09:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078106+0000", 2024-09-16T00:09:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113133 2024-09-16T00:09:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078218+0000", 2024-09-16T00:09:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011114 2024-09-16T00:09:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.079121+0000", 2024-09-16T00:09:12.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00090328499999999998 2024-09-16T00:09:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080928+0000", 2024-09-16T00:09:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018072310000000001 2024-09-16T00:09:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080967+0000", 2024-09-16T00:09:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8982e-05 2024-09-16T00:09:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080993+0000", 2024-09-16T00:09:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6157e-05 2024-09-16T00:09:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.614 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:1878 3.3c 3:3f4f0e87:::10000000a5f.00000000:head [write 0~2494] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T00:09:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.078761+0000", 2024-09-16T00:09:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.451347987, 2024-09-16T00:09:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025239199999999998, 2024-09-16T00:09:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.615 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1878 2024-09-16T00:09:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078761+0000", 2024-09-16T00:09:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078761+0000", 2024-09-16T00:09:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078762+0000", 2024-09-16T00:09:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7600000000000006e-07 2024-09-16T00:09:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078767+0000", 2024-09-16T00:09:12.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5929999999999996e-06 2024-09-16T00:09:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078781+0000", 2024-09-16T00:09:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3804e-05 2024-09-16T00:09:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078789+0000", 2024-09-16T00:09:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6960000000000005e-06 2024-09-16T00:09:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078838+0000", 2024-09-16T00:09:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9416000000000003e-05 2024-09-16T00:09:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078940+0000", 2024-09-16T00:09:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101431 2024-09-16T00:09:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.079032+0000", 2024-09-16T00:09:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2559999999999995e-05 2024-09-16T00:09:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.079823+0000", 2024-09-16T00:09:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00079055300000000005 2024-09-16T00:09:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081201+0000", 2024-09-16T00:09:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013780579999999999 2024-09-16T00:09:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081247+0000", 2024-09-16T00:09:12.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6213e-05 2024-09-16T00:09:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081285+0000", 2024-09-16T00:09:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7620000000000002e-05 2024-09-16T00:09:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:2173 3.30 3:0ff9340b:::10000000bb6.00000000:head [write 0~80] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T00:09:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:03.053795+0000", 2024-09-16T00:09:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.4763139540000001, 2024-09-16T00:09:12.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020696619999999999, 2024-09-16T00:09:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 2173 2024-09-16T00:09:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053795+0000", 2024-09-16T00:09:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053795+0000", 2024-09-16T00:09:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053796+0000", 2024-09-16T00:09:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.257e-06 2024-09-16T00:09:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053802+0000", 2024-09-16T00:09:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3430000000000001e-06 2024-09-16T00:09:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053813+0000", 2024-09-16T00:09:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.101e-05 2024-09-16T00:09:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053819+0000", 2024-09-16T00:09:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0750000000000003e-06 2024-09-16T00:09:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053864+0000", 2024-09-16T00:09:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4258999999999998e-05 2024-09-16T00:09:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053954+0000", 2024-09-16T00:09:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0195e-05 2024-09-16T00:09:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.054034+0000", 2024-09-16T00:09:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0319000000000001e-05 2024-09-16T00:09:12.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.054604+0000", 2024-09-16T00:09:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00057022099999999999 2024-09-16T00:09:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.055768+0000", 2024-09-16T00:09:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001164113 2024-09-16T00:09:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.055821+0000", 2024-09-16T00:09:12.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2364000000000002e-05 2024-09-16T00:09:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.055864+0000", 2024-09-16T00:09:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3506e-05 2024-09-16T00:09:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:2315 3.1f e62/33 3:f84a2a2d:::10000000c60.00000000:head v 62'45, mlcod=62'43)", 2024-09-16T00:09:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:03.440894+0000", 2024-09-16T00:09:12.627 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0892147940000001, 2024-09-16T00:09:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028251109999999999, 2024-09-16T00:09:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.440894+0000", 2024-09-16T00:09:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.440894+0000", 2024-09-16T00:09:12.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.440894+0000", 2024-09-16T00:09:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7700000000000004e-07 2024-09-16T00:09:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441598+0000", 2024-09-16T00:09:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00070308100000000004 2024-09-16T00:09:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441607+0000", 2024-09-16T00:09:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7489999999999993e-06 2024-09-16T00:09:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441612+0000", 2024-09-16T00:09:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6519999999999997e-06 2024-09-16T00:09:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441650+0000", 2024-09-16T00:09:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7867000000000002e-05 2024-09-16T00:09:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441663+0000", 2024-09-16T00:09:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2788e-05 2024-09-16T00:09:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.443681+0000", 2024-09-16T00:09:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020186599999999998 2024-09-16T00:09:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.443719+0000", 2024-09-16T00:09:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7636999999999999e-05 2024-09-16T00:09:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.631 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:2326 3.1c 3:39be8b5c:::10000000c6c.00000000:head [write 0~173460] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T00:09:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:03.473107+0000", 2024-09-16T00:09:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0570015099999992, 2024-09-16T00:09:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001822029, 2024-09-16T00:09:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:12.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 2326 2024-09-16T00:09:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473107+0000", 2024-09-16T00:09:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473107+0000", 2024-09-16T00:09:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473108+0000", 2024-09-16T00:09:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0139999999999999e-06 2024-09-16T00:09:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473292+0000", 2024-09-16T00:09:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000184072 2024-09-16T00:09:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473305+0000", 2024-09-16T00:09:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2444e-05 2024-09-16T00:09:12.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473310+0000", 2024-09-16T00:09:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5720000000000002e-06 2024-09-16T00:09:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473353+0000", 2024-09-16T00:09:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3041999999999999e-05 2024-09-16T00:09:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473455+0000", 2024-09-16T00:09:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101297 2024-09-16T00:09:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473542+0000", 2024-09-16T00:09:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7787999999999994e-05 2024-09-16T00:09:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.474469+0000", 2024-09-16T00:09:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00092618500000000005 2024-09-16T00:09:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.474868+0000", 2024-09-16T00:09:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00039969400000000003 2024-09-16T00:09:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.474900+0000", 2024-09-16T00:09:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1341999999999999e-05 2024-09-16T00:09:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.474929+0000", 2024-09-16T00:09:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9578999999999999e-05 2024-09-16T00:09:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.637 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:3184 3.28 3:144e5f80:::10000000fd3.00000000:head [write 0~36526] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T00:09:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:06.300316+0000", 2024-09-16T00:09:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 6.2297922520000002, 2024-09-16T00:09:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016609940000000001, 2024-09-16T00:09:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:12.638 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 3184 2024-09-16T00:09:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:06.300316+0000", 2024-09-16T00:09:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:06.300316+0000", 2024-09-16T00:09:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:06.300318+0000", 2024-09-16T00:09:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0700000000000001e-06 2024-09-16T00:09:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:06.300355+0000", 2024-09-16T00:09:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6529000000000002e-05 2024-09-16T00:09:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:06.300381+0000", 2024-09-16T00:09:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6327000000000001e-05 2024-09-16T00:09:12.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:06.300396+0000", 2024-09-16T00:09:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.434e-05 2024-09-16T00:09:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:06.300512+0000", 2024-09-16T00:09:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116688 2024-09-16T00:09:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:12.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:06.300702+0000", 2024-09-16T00:09:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000190044 2024-09-16T00:09:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:06.300840+0000", 2024-09-16T00:09:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013776000000000001 2024-09-16T00:09:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:06.301502+0000", 2024-09-16T00:09:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00066227199999999999 2024-09-16T00:09:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:12.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:06.301923+0000", 2024-09-16T00:09:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00042100299999999998 2024-09-16T00:09:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:06.301953+0000", 2024-09-16T00:09:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9476999999999999e-05 2024-09-16T00:09:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:06.301977+0000", 2024-09-16T00:09:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4484000000000001e-05 2024-09-16T00:09:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.643 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.644 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.644 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:12.644 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:12.644 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:12.645 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:09:12.646 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:12.645+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:12.664 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:09:12.745 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:12.745+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:12.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.762+0000 7f468e5e4640 1 Processor -- start 2024-09-16T00:09:12.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.762+0000 7f468e5e4640 1 -- start start 2024-09-16T00:09:12.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.762+0000 7f468e5e4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f46880d0490 0x7f46880b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:12.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.763+0000 7f468e5e4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46880b4750 0x7f46881546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:12.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.763+0000 7f468e5e4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46880b3180 con 0x7f46880d0870 2024-09-16T00:09:12.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.763+0000 7f468e5e4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f468815b940 con 0x7f46880d0490 2024-09-16T00:09:12.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.763+0000 7f468e5e4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f46880b3bb0 con 0x7f46880b4750 2024-09-16T00:09:12.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.763+0000 7f468cb81640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46880b4750 0x7f46881546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:12.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.763+0000 7f4687fff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f46880d0490 0x7f46880b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:12.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.763+0000 7f4687fff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f46880d0490 0x7f46880b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:35398/0 (socket says 172.21.15.139:35398) 2024-09-16T00:09:12.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.763+0000 7f46877fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f46880d0870 0x7f46880b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:41638/0 (socket says 172.21.15.139:41638) 2024-09-16T00:09:12.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.763+0000 7f468cb81640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46880b4750 0x7f46881546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:41328/0 (socket says 172.21.15.139:41328) 2024-09-16T00:09:12.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.763+0000 7f4687fff640 1 -- 172.21.15.139:0/1683170819 learned_addr learned my addr 172.21.15.139:0/1683170819 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:12.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.763+0000 7f4687fff640 1 -- 172.21.15.139:0/1683170819 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46880b4750 msgr2=0x7f46881546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:12.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.763+0000 7f4687fff640 1 --2- 172.21.15.139:0/1683170819 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46880b4750 0x7f46881546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.763+0000 7f4687fff640 1 -- 172.21.15.139:0/1683170819 >> v1:172.21.15.164:6789/0 conn(0x7f46880d0870 legacy=0x7f46880b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:12.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.763+0000 7f4687fff640 1 -- 172.21.15.139:0/1683170819 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468815adc0 con 0x7f46880d0490 2024-09-16T00:09:12.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.764+0000 7f4687fff640 1 --2- 172.21.15.139:0/1683170819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46880d0490 0x7f46880b2c40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4678009870 tx=0x7f467802a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcd45bd1c06317e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:12.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.764+0000 7f4686ffd640 1 -- 172.21.15.139:0/1683170819 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4678037070 con 0x7f46880d0490 2024-09-16T00:09:12.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.764+0000 7f4686ffd640 1 -- 172.21.15.139:0/1683170819 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f467802ae50 con 0x7f46880d0490 2024-09-16T00:09:12.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.764+0000 7f4686ffd640 1 -- 172.21.15.139:0/1683170819 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4678032860 con 0x7f46880d0490 2024-09-16T00:09:12.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.765+0000 7f468e5e4640 1 -- 172.21.15.139:0/1683170819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46880d0490 msgr2=0x7f46880b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:12.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.765+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1683170819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46880d0490 0x7f46880b2c40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4678009870 tx=0x7f467802a300 comp rx=0 tx=0).stop 2024-09-16T00:09:12.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.765+0000 7f468e5e4640 1 -- 172.21.15.139:0/1683170819 shutdown_connections 2024-09-16T00:09:12.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.765+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1683170819 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46880b4750 0x7f46881546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.765+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1683170819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46880d0490 0x7f46880b2c40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.765+0000 7f468e5e4640 1 -- 172.21.15.139:0/1683170819 >> 172.21.15.139:0/1683170819 conn(0x7f468806c520 msgr2=0x7f468806c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:12.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.765+0000 7f468e5e4640 1 -- 172.21.15.139:0/1683170819 shutdown_connections 2024-09-16T00:09:12.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.765+0000 7f468e5e4640 1 -- 172.21.15.139:0/1683170819 wait complete. 2024-09-16T00:09:12.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.766+0000 7f468e5e4640 1 Processor -- start 2024-09-16T00:09:12.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.766+0000 7f468e5e4640 1 -- start start 2024-09-16T00:09:12.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.767+0000 7f468e5e4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46880b4750 0x7f468815bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:12.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.767+0000 7f468e5e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46880d0490 0x7f468815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:12.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.767+0000 7f468e5e4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46880d0870 0x7f468814a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:12.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.767+0000 7f468e5e4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f468815b660 con 0x7f46880d0490 2024-09-16T00:09:12.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.767+0000 7f468e5e4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f468815b380 con 0x7f46880b4750 2024-09-16T00:09:12.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.767+0000 7f468e5e4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f468815a980 con 0x7f46880d0870 2024-09-16T00:09:12.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.767+0000 7f4687fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46880b4750 0x7f468815bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:12.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.767+0000 7f46877fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46880d0490 0x7f468815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:12.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.767+0000 7f4687fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46880b4750 0x7f468815bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:35414/0 (socket says 172.21.15.139:35414) 2024-09-16T00:09:12.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.767+0000 7f468cb81640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46880d0870 0x7f468814a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:12.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.767+0000 7f4687fff640 1 -- 172.21.15.139:0/1574013148 learned_addr learned my addr 172.21.15.139:0/1574013148 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:12.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.767+0000 7f4687fff640 1 -- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46880d0870 msgr2=0x7f468814a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:12.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.768+0000 7f4687fff640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46880d0870 0x7f468814a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.768+0000 7f4687fff640 1 -- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46880d0490 msgr2=0x7f468815dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:12.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.768+0000 7f4687fff640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46880d0490 0x7f468815dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.768+0000 7f4687fff640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f468815e750 con 0x7f46880b4750 2024-09-16T00:09:12.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.768+0000 7f4687fff640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46880b4750 0x7f468815bae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4678000c00 tx=0x7f46780046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:12.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.768+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4678041070 con 0x7f46880b4750 2024-09-16T00:09:12.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.768+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f467803c660 con 0x7f46880b4750 2024-09-16T00:09:12.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.768+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f468814fc30 con 0x7f46880b4750 2024-09-16T00:09:12.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.768+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4678037040 con 0x7f46880b4750 2024-09-16T00:09:12.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.768+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46881501f0 con 0x7f46880b4750 2024-09-16T00:09:12.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.769+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f46880b3430 con 0x7f46880b4750 2024-09-16T00:09:12.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.770+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4678044070 con 0x7f46880b4750 2024-09-16T00:09:12.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.770+0000 7f4684ff9640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f467406c2e0 0x7f467406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:12.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.770+0000 7f46877fe640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f467406c2e0 0x7f467406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:12.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.770+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5915+0+0 (secure 0 0 0) 0x7f46780abed0 con 0x7f46880b4750 2024-09-16T00:09:12.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.771+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v66) ==== 54+0+16 (secure 0 0 0) 0x7f46780ac360 con 0x7f46880b4750 2024-09-16T00:09:12.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.771+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f468815e2d0 0x7f46880cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:12.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.771+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4688151ce0 con 0x7f468815e2d0 2024-09-16T00:09:12.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.771+0000 7f46877fe640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f467406c2e0 0x7f467406e7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4688151c00 tx=0x7f467c007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:12.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.771+0000 7f468cb81640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f468815e2d0 0x7f46880cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:12.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.771+0000 7f468cb81640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f468815e2d0 0x7f46880cf2f0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:12.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.772+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4688151ce0 con 0x7f468815e2d0 2024-09-16T00:09:12.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.788+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f46880ceaf0 con 0x7f468815e2d0 2024-09-16T00:09:12.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.790+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f46880ceaf0 con 0x7f468815e2d0 2024-09-16T00:09:12.791 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:12.791 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:12.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.790+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f468814e760 0x7f46881f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:12.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.790+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f46880ce810 con 0x7f468814e760 2024-09-16T00:09:12.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.790+0000 7f4687fff640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f468814e760 0x7f46881f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:12.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.790+0000 7f4687fff640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f468814e760 0x7f46881f4390 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:12.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.791+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f46880ce810 con 0x7f468814e760 2024-09-16T00:09:12.796 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:12.796 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:12.796 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:12.796 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:12.805 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:09:12.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.807+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f46880ce530 con 0x7f468814e760 2024-09-16T00:09:12.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.809+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f46880ce530 con 0x7f468814e760 2024-09-16T00:09:12.810 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:12.810 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:12.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.809+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f468814f5c0 0x7f4688062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:12.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.809+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f468815c020 con 0x7f468814f5c0 2024-09-16T00:09:12.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.809+0000 7f46877fe640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f468814f5c0 0x7f4688062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:12.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.809+0000 7f46877fe640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f468814f5c0 0x7f4688062630 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:12.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.810+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f468815c020 con 0x7f468814f5c0 2024-09-16T00:09:12.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.825+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f468815c1d0 con 0x7f468814f5c0 2024-09-16T00:09:12.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.827+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f468815c1d0 con 0x7f468814f5c0 2024-09-16T00:09:12.828 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:12.828 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:12.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.827+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f4688063080 0x7f46880c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:12.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.827+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f46880634a0 con 0x7f4688063080 2024-09-16T00:09:12.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.827+0000 7f468cb81640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f4688063080 0x7f46880c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:12.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.828+0000 7f468cb81640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f4688063080 0x7f46880c7880 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:12.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.828+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f46880634a0 con 0x7f4688063080 2024-09-16T00:09:12.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.842+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f46880634a0 con 0x7f4688063080 2024-09-16T00:09:12.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.844+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f46880634a0 con 0x7f4688063080 2024-09-16T00:09:12.845 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:12.845 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:12.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.844+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f468802c640 0x7f468802ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:12.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.844+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f46880634a0 con 0x7f468802c640 2024-09-16T00:09:12.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.844+0000 7f4687fff640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f468802c640 0x7f468802ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:12.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.845+0000 7f4687fff640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f468802c640 0x7f468802ca80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:12.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.846+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f46880634a0 con 0x7f468802c640 2024-09-16T00:09:12.847 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:12.846+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:12.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.860+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f46880634a0 con 0x7f468802c640 2024-09-16T00:09:12.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.862+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f46880634a0 con 0x7f468802c640 2024-09-16T00:09:12.863 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:12.863 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:12.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.862+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f468802d4d0 0x7f4688207210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:12.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.862+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f468802d930 con 0x7f468802d4d0 2024-09-16T00:09:12.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.862+0000 7f46877fe640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f468802d4d0 0x7f4688207210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:12.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.863+0000 7f46877fe640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f468802d4d0 0x7f4688207210 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:12.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.864+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f468802d930 con 0x7f468802d4d0 2024-09-16T00:09:12.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.879+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f468802d930 con 0x7f468802d4d0 2024-09-16T00:09:12.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.881+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f468802d930 con 0x7f468802d4d0 2024-09-16T00:09:12.882 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:12.883 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:12.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.881+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f4688207c60 0x7f468820a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:12.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.881+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f468802db40 con 0x7f4688207c60 2024-09-16T00:09:12.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.882+0000 7f468cb81640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f4688207c60 0x7f468820a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:12.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.882+0000 7f468cb81640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f4688207c60 0x7f468820a050 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:12.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.883+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f468802db40 con 0x7f4688207c60 2024-09-16T00:09:12.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.896+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f468802db40 con 0x7f4688207c60 2024-09-16T00:09:12.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.899+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f468802db40 con 0x7f4688207c60 2024-09-16T00:09:12.900 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:12.900 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:09:12.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.899+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f468820aaa0 0x7f468820af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:12.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.899+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f468820ffe0 con 0x7f468820aaa0 2024-09-16T00:09:12.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.899+0000 7f4687fff640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f468820aaa0 0x7f468820af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:12.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.900+0000 7f4687fff640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f468820aaa0 0x7f468820af20 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:12.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.900+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f468820ffe0 con 0x7f468820aaa0 2024-09-16T00:09:12.913 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:12.913 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:12.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.914+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f468820b460 con 0x7f468820aaa0 2024-09-16T00:09:12.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f4684ff9640 1 -- 172.21.15.139:0/1574013148 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f468820b460 con 0x7f468820aaa0 2024-09-16T00:09:12.917 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:12.918 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:12.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f468815e2d0 msgr2=0x7f46880cf2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:12.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f468815e2d0 0x7f46880cf2f0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f468814e760 msgr2=0x7f46881f4390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f468814e760 0x7f46881f4390 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f468814f5c0 msgr2=0x7f4688062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f468814f5c0 0x7f4688062630 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f4688063080 msgr2=0x7f46880c7880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f4688063080 0x7f46880c7880 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468cb81640 1 -- 172.21.15.139:0/1574013148 reap_dead start 2024-09-16T00:09:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f468802c640 msgr2=0x7f468802ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f468802c640 0x7f468802ca80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468cb81640 1 -- 172.21.15.139:0/1574013148 reap_dead start 2024-09-16T00:09:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f468802d4d0 msgr2=0x7f4688207210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f468802d4d0 0x7f4688207210 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f4688207c60 msgr2=0x7f468820a050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f4688207c60 0x7f468820a050 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f468820aaa0 msgr2=0x7f468820af20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.916+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f468820aaa0 0x7f468820af20 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.917+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f467406c2e0 msgr2=0x7f467406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.917+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f467406c2e0 0x7f467406e7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4688151c00 tx=0x7f467c007e80 comp rx=0 tx=0).stop 2024-09-16T00:09:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.917+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46880b4750 msgr2=0x7f468815bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.917+0000 7f468e5e4640 1 --2- 172.21.15.139:0/1574013148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46880b4750 0x7f468815bae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4678000c00 tx=0x7f46780046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.917+0000 7f468cb81640 1 -- 172.21.15.139:0/1574013148 reap_dead start 2024-09-16T00:09:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.917+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 shutdown_connections 2024-09-16T00:09:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.917+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 >> 172.21.15.139:0/1574013148 conn(0x7f468806c520 msgr2=0x7f4688159ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.917+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 shutdown_connections 2024-09-16T00:09:12.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:12.917+0000 7f468e5e4640 1 -- 172.21.15.139:0/1574013148 wait complete. 2024-09-16T00:09:12.922 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:09:12.966 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:12.966+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:13.039 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:13.039 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:13.048 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:09:13.049 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:13.048+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:13.150 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:13.150+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:13.161 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T00:09:13.161 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T00:09:13.207 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:13.208 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:13.218 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:09:13.251 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:13.251+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:13.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.270+0000 7f7b136c9640 1 Processor -- start 2024-09-16T00:09:13.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.270+0000 7f7b136c9640 1 -- start start 2024-09-16T00:09:13.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.271+0000 7f7b136c9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7b0c154aa0 0x7f7b0c15e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:13.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.271+0000 7f7b136c9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7b0c0c7aa0 0x7f7b0c165130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:13.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.271+0000 7f7b136c9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b0c0c8760 con 0x7f7b0c0c6da0 2024-09-16T00:09:13.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.271+0000 7f7b136c9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7b0c0c8a40 con 0x7f7b0c154aa0 2024-09-16T00:09:13.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.271+0000 7f7b136c9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7b0c0c8d20 con 0x7f7b0c0c7aa0 2024-09-16T00:09:13.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.271+0000 7f7b11465640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7b0c154aa0 0x7f7b0c15e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:13.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.271+0000 7f7b11465640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7b0c154aa0 0x7f7b0c15e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58778/0 (socket says 172.21.15.139:58778) 2024-09-16T00:09:13.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.271+0000 7f7b11465640 1 -- 172.21.15.139:0/159531275 learned_addr learned my addr 172.21.15.139:0/159531275 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:13.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.271+0000 7f7b11c66640 1 --2- 172.21.15.139:0/159531275 >> v2:172.21.15.164:3300/0 conn(0x7f7b0c0c7aa0 0x7f7b0c165130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:13.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.272+0000 7f7b11465640 1 -- 172.21.15.139:0/159531275 >> v2:172.21.15.164:3300/0 conn(0x7f7b0c0c7aa0 msgr2=0x7f7b0c165130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:13.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.272+0000 7f7b11465640 1 --2- 172.21.15.139:0/159531275 >> v2:172.21.15.164:3300/0 conn(0x7f7b0c0c7aa0 0x7f7b0c165130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:13.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.272+0000 7f7b11465640 1 -- 172.21.15.139:0/159531275 >> v1:172.21.15.164:6789/0 conn(0x7f7b0c0c6da0 legacy=0x7f7b0c0cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:13.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.272+0000 7f7b11465640 1 -- 172.21.15.139:0/159531275 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0c15deb0 con 0x7f7b0c154aa0 2024-09-16T00:09:13.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.272+0000 7f7b11465640 1 --2- 172.21.15.139:0/159531275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0c154aa0 0x7f7b0c15e6b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c06dc70 tx=0x7f7b0802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713f88263f0acb0a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:13.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.272+0000 7f7b03fff640 1 -- 172.21.15.139:0/159531275 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08037070 con 0x7f7b0c154aa0 2024-09-16T00:09:13.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.273+0000 7f7b03fff640 1 -- 172.21.15.139:0/159531275 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b0802ae50 con 0x7f7b0c154aa0 2024-09-16T00:09:13.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.273+0000 7f7b03fff640 1 -- 172.21.15.139:0/159531275 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08032860 con 0x7f7b0c154aa0 2024-09-16T00:09:13.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.273+0000 7f7b136c9640 1 -- 172.21.15.139:0/159531275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0c154aa0 msgr2=0x7f7b0c15e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:13.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.273+0000 7f7b136c9640 1 --2- 172.21.15.139:0/159531275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0c154aa0 0x7f7b0c15e6b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c06dc70 tx=0x7f7b0802a300 comp rx=0 tx=0).stop 2024-09-16T00:09:13.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.273+0000 7f7b136c9640 1 -- 172.21.15.139:0/159531275 shutdown_connections 2024-09-16T00:09:13.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.273+0000 7f7b136c9640 1 --2- 172.21.15.139:0/159531275 >> v2:172.21.15.164:3300/0 conn(0x7f7b0c0c7aa0 0x7f7b0c165130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:13.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.273+0000 7f7b136c9640 1 --2- 172.21.15.139:0/159531275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0c154aa0 0x7f7b0c15e6b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:13.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.273+0000 7f7b136c9640 1 -- 172.21.15.139:0/159531275 >> 172.21.15.139:0/159531275 conn(0x7f7b0c06cc30 msgr2=0x7f7b0c06d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:13.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.273+0000 7f7b136c9640 1 -- 172.21.15.139:0/159531275 shutdown_connections 2024-09-16T00:09:13.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.273+0000 7f7b136c9640 1 -- 172.21.15.139:0/159531275 wait complete. 2024-09-16T00:09:13.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.275+0000 7f7b136c9640 1 Processor -- start 2024-09-16T00:09:13.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.275+0000 7f7b136c9640 1 -- start start 2024-09-16T00:09:13.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b136c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0c0c6da0 0x7f7b0c1e4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:13.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b136c9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b0c0c7aa0 0x7f7b0c1556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:13.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b136c9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c154aa0 0x7f7b0c155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:13.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b136c9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c154e80 con 0x7f7b0c0c6da0 2024-09-16T00:09:13.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b136c9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7b0c0c9000 con 0x7f7b0c154aa0 2024-09-16T00:09:13.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b136c9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7b0c0c8420 con 0x7f7b0c0c7aa0 2024-09-16T00:09:13.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b11465640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0c0c6da0 0x7f7b0c1e4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:13.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b11465640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0c0c6da0 0x7f7b0c1e4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58780/0 (socket says 172.21.15.139:58780) 2024-09-16T00:09:13.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b11465640 1 -- 172.21.15.139:0/4107980562 learned_addr learned my addr 172.21.15.139:0/4107980562 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:13.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b11c66640 1 --2- 172.21.15.139:0/4107980562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c154aa0 0x7f7b0c155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:13.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b10c64640 1 --2- 172.21.15.139:0/4107980562 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b0c0c7aa0 0x7f7b0c1556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:13.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b11465640 1 -- 172.21.15.139:0/4107980562 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b0c0c7aa0 msgr2=0x7f7b0c1556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:13.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b11465640 1 --2- 172.21.15.139:0/4107980562 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b0c0c7aa0 0x7f7b0c1556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:13.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b11465640 1 -- 172.21.15.139:0/4107980562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c154aa0 msgr2=0x7f7b0c155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:13.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b11465640 1 --2- 172.21.15.139:0/4107980562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c154aa0 0x7f7b0c155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:13.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b11465640 1 -- 172.21.15.139:0/4107980562 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0c1e34d0 con 0x7f7b0c0c6da0 2024-09-16T00:09:13.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b11465640 1 --2- 172.21.15.139:0/4107980562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0c0c6da0 0x7f7b0c1e4600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7b08000c00 tx=0x7f7b080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:13.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b01ffb640 1 -- 172.21.15.139:0/4107980562 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08041020 con 0x7f7b0c0c6da0 2024-09-16T00:09:13.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b01ffb640 1 -- 172.21.15.139:0/4107980562 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b0803c890 con 0x7f7b0c0c6da0 2024-09-16T00:09:13.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b136c9640 1 -- 172.21.15.139:0/4107980562 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0c1e37b0 con 0x7f7b0c0c6da0 2024-09-16T00:09:13.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.276+0000 7f7b01ffb640 1 -- 172.21.15.139:0/4107980562 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b08037040 con 0x7f7b0c0c6da0 2024-09-16T00:09:13.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.277+0000 7f7b136c9640 1 -- 172.21.15.139:0/4107980562 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b0c1569d0 con 0x7f7b0c0c6da0 2024-09-16T00:09:13.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.278+0000 7f7b136c9640 1 -- 172.21.15.139:0/4107980562 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b0c0c73d0 con 0x7f7b0c0c6da0 2024-09-16T00:09:13.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.278+0000 7f7b01ffb640 1 -- 172.21.15.139:0/4107980562 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7b08044020 con 0x7f7b0c0c6da0 2024-09-16T00:09:13.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.279+0000 7f7b01ffb640 1 --2- 172.21.15.139:0/4107980562 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7af006c2b0 0x7f7af006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:13.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.279+0000 7f7b01ffb640 1 -- 172.21.15.139:0/4107980562 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5915+0+0 (secure 0 0 0) 0x7f7b080abd20 con 0x7f7b0c0c6da0 2024-09-16T00:09:13.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.279+0000 7f7b10c64640 1 --2- 172.21.15.139:0/4107980562 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7af006c2b0 0x7f7af006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:13.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.279+0000 7f7b10c64640 1 --2- 172.21.15.139:0/4107980562 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7af006c2b0 0x7f7af006e770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c06f170 tx=0x7f7afc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:13.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.284+0000 7f7b01ffb640 1 -- 172.21.15.139:0/4107980562 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7b0804b030 con 0x7f7b0c0c6da0 2024-09-16T00:09:13.352 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:13.351+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:13.374 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:13.374 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:13.383 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:09:13.453 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:13.453+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:13.496 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:13.496 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:13.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:13.501+0000 7f7b136c9640 1 -- 172.21.15.139:0/4107980562 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) -- 0x7f7b0c0c58a0 con 0x7f7b0c0c6da0 2024-09-16T00:09:13.506 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:09:13.554 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:13.554+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:13.627 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:13.627 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:13.638 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:09:13.655 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:13.655+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:13.756 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:13.756+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:13.797 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:13.797 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:13.807 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:09:13.857 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:13.856+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:13.923 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:13.924 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:13.934 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:09:13.974 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:13.974+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:14.058 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:14.058+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:14.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.061+0000 7f7b01ffb640 1 -- 172.21.15.139:0/4107980562 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v67) ==== 113+0+0 (secure 0 0 0) 0x7f7b08079d70 con 0x7f7b0c0c6da0 2024-09-16T00:09:14.062 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T00:09:14.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.064+0000 7f7b136c9640 1 -- 172.21.15.139:0/4107980562 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7af006c2b0 msgr2=0x7f7af006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:14.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.064+0000 7f7b136c9640 1 --2- 172.21.15.139:0/4107980562 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7af006c2b0 0x7f7af006e770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c06f170 tx=0x7f7afc009040 comp rx=0 tx=0).stop 2024-09-16T00:09:14.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.064+0000 7f7b136c9640 1 -- 172.21.15.139:0/4107980562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0c0c6da0 msgr2=0x7f7b0c1e4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:14.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.064+0000 7f7b136c9640 1 --2- 172.21.15.139:0/4107980562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0c0c6da0 0x7f7b0c1e4600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7b08000c00 tx=0x7f7b080046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:14.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.064+0000 7f7b136c9640 1 -- 172.21.15.139:0/4107980562 shutdown_connections 2024-09-16T00:09:14.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.064+0000 7f7b136c9640 1 --2- 172.21.15.139:0/4107980562 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7af006c2b0 0x7f7af006e770 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:14.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.064+0000 7f7b136c9640 1 --2- 172.21.15.139:0/4107980562 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7b0c154aa0 0x7f7b0c155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:14.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.064+0000 7f7b136c9640 1 --2- 172.21.15.139:0/4107980562 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7b0c0c7aa0 0x7f7b0c1556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:14.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.064+0000 7f7b136c9640 1 --2- 172.21.15.139:0/4107980562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b0c0c6da0 0x7f7b0c1e4600 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:14.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.064+0000 7f7b136c9640 1 -- 172.21.15.139:0/4107980562 >> 172.21.15.139:0/4107980562 conn(0x7f7b0c06cc30 msgr2=0x7f7b0c0caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:14.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.064+0000 7f7b136c9640 1 -- 172.21.15.139:0/4107980562 shutdown_connections 2024-09-16T00:09:14.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.064+0000 7f7b136c9640 1 -- 172.21.15.139:0/4107980562 wait complete. 2024-09-16T00:09:14.075 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T00:09:14.082 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T00:09:14.101 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:14.101 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:14.111 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:09:14.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.180+0000 7f2456bc4640 1 Processor -- start 2024-09-16T00:09:14.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.180+0000 7f2456bc4640 1 -- start start 2024-09-16T00:09:14.181 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:14.180+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:14.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.181+0000 7f2456bc4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24500c7aa0 0x7f2450166140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:14.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.181+0000 7f2456bc4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24500c8760 con 0x7f2450154aa0 2024-09-16T00:09:14.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.181+0000 7f2456bc4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24500c8a40 con 0x7f24500c6da0 2024-09-16T00:09:14.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.181+0000 7f2456bc4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f24500c8d20 con 0x7f24500c7aa0 2024-09-16T00:09:14.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.181+0000 7f2454960640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2450154aa0 0x7f245015e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46744/0 (socket says 172.21.15.139:46744) 2024-09-16T00:09:14.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.181+0000 7f2454960640 1 -- 172.21.15.139:0/851274398 learned_addr learned my addr 172.21.15.139:0/851274398 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:14.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.181+0000 7f2455161640 1 --2- 172.21.15.139:0/851274398 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24500c7aa0 0x7f2450166140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:14.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.182+0000 7f244f7fe640 1 -- 172.21.15.139:0/851274398 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2515934554 0 0) 0x7f24500c8a40 con 0x7f24500c6da0 2024-09-16T00:09:14.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.182+0000 7f244f7fe640 1 -- 172.21.15.139:0/851274398 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2440003520 con 0x7f24500c6da0 2024-09-16T00:09:14.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.182+0000 7f244f7fe640 1 -- 172.21.15.139:0/851274398 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3417057906 0 0) 0x7f24500c8760 con 0x7f2450154aa0 2024-09-16T00:09:14.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.182+0000 7f244f7fe640 1 -- 172.21.15.139:0/851274398 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24500c8a40 con 0x7f2450154aa0 2024-09-16T00:09:14.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.182+0000 7f2455161640 1 -- 172.21.15.139:0/851274398 >> v1:172.21.15.164:6789/0 conn(0x7f24500c6da0 legacy=0x7f24500cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:14.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.182+0000 7f2455161640 1 -- 172.21.15.139:0/851274398 >> v1:172.21.15.139:6789/0 conn(0x7f2450154aa0 legacy=0x7f245015e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:14.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.182+0000 7f2455161640 1 -- 172.21.15.139:0/851274398 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245015deb0 con 0x7f24500c7aa0 2024-09-16T00:09:14.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.183+0000 7f2455161640 1 --2- 172.21.15.139:0/851274398 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24500c7aa0 0x7f2450166140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2448009dc0 tx=0x7f244802a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0d0e6b202728803 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:14.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.184+0000 7f244f7fe640 1 -- 172.21.15.139:0/851274398 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448007740 con 0x7f24500c7aa0 2024-09-16T00:09:14.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.184+0000 7f244f7fe640 1 -- 172.21.15.139:0/851274398 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24480078e0 con 0x7f24500c7aa0 2024-09-16T00:09:14.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.184+0000 7f244f7fe640 1 -- 172.21.15.139:0/851274398 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448007cb0 con 0x7f24500c7aa0 2024-09-16T00:09:14.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.184+0000 7f2456bc4640 1 -- 172.21.15.139:0/851274398 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24500c7aa0 msgr2=0x7f2450166140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:14.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.184+0000 7f2456bc4640 1 --2- 172.21.15.139:0/851274398 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24500c7aa0 0x7f2450166140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2448009dc0 tx=0x7f244802a760 comp rx=0 tx=0).stop 2024-09-16T00:09:14.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.184+0000 7f2456bc4640 1 -- 172.21.15.139:0/851274398 shutdown_connections 2024-09-16T00:09:14.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.184+0000 7f2456bc4640 1 --2- 172.21.15.139:0/851274398 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24500c7aa0 0x7f2450166140 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:14.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.185+0000 7f2456bc4640 1 -- 172.21.15.139:0/851274398 >> 172.21.15.139:0/851274398 conn(0x7f245006cc30 msgr2=0x7f245006d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:14.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.185+0000 7f2456bc4640 1 -- 172.21.15.139:0/851274398 shutdown_connections 2024-09-16T00:09:14.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.185+0000 7f2456bc4640 1 -- 172.21.15.139:0/851274398 wait complete. 2024-09-16T00:09:14.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.186+0000 7f2456bc4640 1 Processor -- start 2024-09-16T00:09:14.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.186+0000 7f2456bc4640 1 -- start start 2024-09-16T00:09:14.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.187+0000 7f2456bc4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24500c6da0 0x7f24500b97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:14.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.187+0000 7f2456bc4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24500c7aa0 0x7f24501f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:14.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.187+0000 7f2456bc4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2450154aa0 0x7f2450156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:14.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.187+0000 7f2456bc4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2450154e80 con 0x7f24500c6da0 2024-09-16T00:09:14.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.187+0000 7f2456bc4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f24500c9000 con 0x7f24500c7aa0 2024-09-16T00:09:14.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.187+0000 7f2456bc4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f24500c8420 con 0x7f2450154aa0 2024-09-16T00:09:14.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.187+0000 7f2454960640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24500c6da0 0x7f24500b97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:14.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.187+0000 7f244ffff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24500c7aa0 0x7f24501f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:14.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.187+0000 7f2454960640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24500c6da0 0x7f24500b97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58788/0 (socket says 172.21.15.139:58788) 2024-09-16T00:09:14.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.187+0000 7f2454960640 1 -- 172.21.15.139:0/2049351731 learned_addr learned my addr 172.21.15.139:0/2049351731 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:14.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.187+0000 7f2455161640 1 --2- 172.21.15.139:0/2049351731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2450154aa0 0x7f2450156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:14.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.188+0000 7f2454960640 1 -- 172.21.15.139:0/2049351731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2450154aa0 msgr2=0x7f2450156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:14.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.188+0000 7f2454960640 1 --2- 172.21.15.139:0/2049351731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2450154aa0 0x7f2450156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:14.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.188+0000 7f2454960640 1 -- 172.21.15.139:0/2049351731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24500c7aa0 msgr2=0x7f24501f87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:14.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.188+0000 7f2454960640 1 --2- 172.21.15.139:0/2049351731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24500c7aa0 0x7f24501f87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:14.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.188+0000 7f2454960640 1 -- 172.21.15.139:0/2049351731 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24501f8d20 con 0x7f24500c6da0 2024-09-16T00:09:14.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.188+0000 7f2455161640 1 --2- 172.21.15.139:0/2049351731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2450154aa0 0x7f2450156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:14.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.188+0000 7f244ffff640 1 --2- 172.21.15.139:0/2049351731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24500c7aa0 0x7f24501f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:14.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.188+0000 7f2454960640 1 --2- 172.21.15.139:0/2049351731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24500c6da0 0x7f24500b97c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f244400c410 tx=0x7f244400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:14.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.188+0000 7f244d7fa640 1 -- 172.21.15.139:0/2049351731 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444017070 con 0x7f24500c6da0 2024-09-16T00:09:14.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.188+0000 7f244d7fa640 1 -- 172.21.15.139:0/2049351731 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2444005ce0 con 0x7f24500c6da0 2024-09-16T00:09:14.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.188+0000 7f244d7fa640 1 -- 172.21.15.139:0/2049351731 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24440072f0 con 0x7f24500c6da0 2024-09-16T00:09:14.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.188+0000 7f2456bc4640 1 -- 172.21.15.139:0/2049351731 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245015c770 con 0x7f24500c6da0 2024-09-16T00:09:14.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.188+0000 7f2456bc4640 1 -- 172.21.15.139:0/2049351731 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24501552c0 con 0x7f24500c6da0 2024-09-16T00:09:14.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.190+0000 7f2456bc4640 1 -- 172.21.15.139:0/2049351731 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24500c73d0 con 0x7f24500c6da0 2024-09-16T00:09:14.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.190+0000 7f244d7fa640 1 -- 172.21.15.139:0/2049351731 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f244401a070 con 0x7f24500c6da0 2024-09-16T00:09:14.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.191+0000 7f244d7fa640 1 --2- 172.21.15.139:0/2049351731 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f243406c2b0 0x7f243406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:14.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.191+0000 7f244d7fa640 1 -- 172.21.15.139:0/2049351731 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5915+0+0 (secure 0 0 0) 0x7f244408a7e0 con 0x7f24500c6da0 2024-09-16T00:09:14.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.191+0000 7f244ffff640 1 --2- 172.21.15.139:0/2049351731 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f243406c2b0 0x7f243406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:14.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.191+0000 7f244ffff640 1 --2- 172.21.15.139:0/2049351731 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f243406c2b0 0x7f243406e770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f243c007fd0 tx=0x7f243c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:14.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.200+0000 7f244d7fa640 1 -- 172.21.15.139:0/2049351731 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2444058980 con 0x7f24500c6da0 2024-09-16T00:09:14.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.214+0000 7fb79519f640 1 Processor -- start 2024-09-16T00:09:14.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.214+0000 7fb79519f640 1 -- start start 2024-09-16T00:09:14.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.216+0000 7fb79519f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb79015b280 0x7fb790166190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:14.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.216+0000 7fb79519f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7900cc4b0 con 0x7fb790154c20 2024-09-16T00:09:14.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.216+0000 7fb79519f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7900cc790 con 0x7fb790068060 2024-09-16T00:09:14.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.216+0000 7fb79519f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb7900c5830 con 0x7fb79015b280 2024-09-16T00:09:14.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.216+0000 7fb78e59c640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb790068060 0x7fb7900ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:41646/0 (socket says 172.21.15.139:41646) 2024-09-16T00:09:14.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.216+0000 7fb78e59c640 1 -- 172.21.15.139:0/3637455668 learned_addr learned my addr 172.21.15.139:0/3637455668 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:14.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.216+0000 7fb78f59e640 1 --2- 172.21.15.139:0/3637455668 >> v2:172.21.15.164:3300/0 conn(0x7fb79015b280 0x7fb790166190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:14.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.216+0000 7fb78dd9b640 1 -- 172.21.15.139:0/3637455668 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1702190415 0 0) 0x7fb7900cc790 con 0x7fb790068060 2024-09-16T00:09:14.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.216+0000 7fb78dd9b640 1 -- 172.21.15.139:0/3637455668 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb778003520 con 0x7fb790068060 2024-09-16T00:09:14.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.216+0000 7fb78f59e640 1 -- 172.21.15.139:0/3637455668 >> v1:172.21.15.164:6789/0 conn(0x7fb790068060 legacy=0x7fb7900ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:14.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.216+0000 7fb78f59e640 1 -- 172.21.15.139:0/3637455668 >> v1:172.21.15.139:6789/0 conn(0x7fb790154c20 legacy=0x7fb7900c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:14.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.216+0000 7fb78dd9b640 1 -- 172.21.15.139:0/3637455668 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1810471415 0 0) 0x7fb7900cc4b0 con 0x7fb790154c20 2024-09-16T00:09:14.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.216+0000 7fb78f59e640 1 -- 172.21.15.139:0/3637455668 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79015ddf0 con 0x7fb79015b280 2024-09-16T00:09:14.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.218+0000 7fb78f59e640 1 --2- 172.21.15.139:0/3637455668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb79015b280 0x7fb790166190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb788009d90 tx=0x7fb78802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b4f69fac830d3b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:14.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.218+0000 7fb78dd9b640 1 -- 172.21.15.139:0/3637455668 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788007740 con 0x7fb79015b280 2024-09-16T00:09:14.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.218+0000 7fb78dd9b640 1 -- 172.21.15.139:0/3637455668 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7880078e0 con 0x7fb79015b280 2024-09-16T00:09:14.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.218+0000 7fb78dd9b640 1 -- 172.21.15.139:0/3637455668 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788007cb0 con 0x7fb79015b280 2024-09-16T00:09:14.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.219+0000 7fb79519f640 1 -- 172.21.15.139:0/3637455668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb79015b280 msgr2=0x7fb790166190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:14.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.219+0000 7fb79519f640 1 --2- 172.21.15.139:0/3637455668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb79015b280 0x7fb790166190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb788009d90 tx=0x7fb78802a6f0 comp rx=0 tx=0).stop 2024-09-16T00:09:14.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.219+0000 7fb79519f640 1 -- 172.21.15.139:0/3637455668 shutdown_connections 2024-09-16T00:09:14.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.219+0000 7fb79519f640 1 --2- 172.21.15.139:0/3637455668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb79015b280 0x7fb790166190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:14.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.219+0000 7fb79519f640 1 -- 172.21.15.139:0/3637455668 >> 172.21.15.139:0/3637455668 conn(0x7fb79006ca40 msgr2=0x7fb790067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:14.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.219+0000 7fb79519f640 1 -- 172.21.15.139:0/3637455668 shutdown_connections 2024-09-16T00:09:14.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.219+0000 7fb79519f640 1 -- 172.21.15.139:0/3637455668 wait complete. 2024-09-16T00:09:14.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.221+0000 7fb79519f640 1 Processor -- start 2024-09-16T00:09:14.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.221+0000 7fb79519f640 1 -- start start 2024-09-16T00:09:14.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.222+0000 7fb79519f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb790068060 0x7fb79015d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:14.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.222+0000 7fb79519f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb790154c20 0x7fb7900cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:14.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.222+0000 7fb79519f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb79015b280 0x7fb790152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:14.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb79519f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb79015e3b0 con 0x7fb79015b280 2024-09-16T00:09:14.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb79519f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb7900cc260 con 0x7fb790154c20 2024-09-16T00:09:14.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb79519f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb79015e690 con 0x7fb790068060 2024-09-16T00:09:14.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb78e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb790154c20 0x7fb7900cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:14.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb78f59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb79015b280 0x7fb790152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:14.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb78e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb790154c20 0x7fb7900cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:35448/0 (socket says 172.21.15.139:35448) 2024-09-16T00:09:14.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb78ed9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb790068060 0x7fb79015d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:14.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb78e59c640 1 -- 172.21.15.139:0/3698984434 learned_addr learned my addr 172.21.15.139:0/3698984434 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:14.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb78e59c640 1 -- 172.21.15.139:0/3698984434 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb790068060 msgr2=0x7fb79015d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:14.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb78e59c640 1 --2- 172.21.15.139:0/3698984434 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb790068060 0x7fb79015d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:14.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb78e59c640 1 -- 172.21.15.139:0/3698984434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb79015b280 msgr2=0x7fb790152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:14.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb78e59c640 1 --2- 172.21.15.139:0/3698984434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb79015b280 0x7fb790152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:14.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb78e59c640 1 -- 172.21.15.139:0/3698984434 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79015d850 con 0x7fb790154c20 2024-09-16T00:09:14.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb78e59c640 1 --2- 172.21.15.139:0/3698984434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb790154c20 0x7fb7900cf8c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb78400c460 tx=0x7fb78400c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:14.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb7777fe640 1 -- 172.21.15.139:0/3698984434 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784017070 con 0x7fb790154c20 2024-09-16T00:09:14.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb7777fe640 1 -- 172.21.15.139:0/3698984434 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb784005d30 con 0x7fb790154c20 2024-09-16T00:09:14.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb7777fe640 1 -- 172.21.15.139:0/3698984434 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7840072f0 con 0x7fb790154c20 2024-09-16T00:09:14.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb79519f640 1 -- 172.21.15.139:0/3698984434 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7900b47f0 con 0x7fb790154c20 2024-09-16T00:09:14.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.223+0000 7fb79519f640 1 -- 172.21.15.139:0/3698984434 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb79015bdd0 con 0x7fb790154c20 2024-09-16T00:09:14.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.225+0000 7fb79519f640 1 -- 172.21.15.139:0/3698984434 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7900cd720 con 0x7fb790154c20 2024-09-16T00:09:14.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.226+0000 7fb7777fe640 1 -- 172.21.15.139:0/3698984434 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb78401a070 con 0x7fb790154c20 2024-09-16T00:09:14.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.226+0000 7fb7777fe640 1 --2- 172.21.15.139:0/3698984434 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb77c06c2b0 0x7fb77c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:14.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.226+0000 7fb7777fe640 1 -- 172.21.15.139:0/3698984434 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5915+0+0 (secure 0 0 0) 0x7fb78408a7b0 con 0x7fb790154c20 2024-09-16T00:09:14.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.226+0000 7fb78ed9d640 1 --2- 172.21.15.139:0/3698984434 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb77c06c2b0 0x7fb77c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:14.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.227+0000 7fb78ed9d640 1 --2- 172.21.15.139:0/3698984434 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb77c06c2b0 0x7fb77c06e770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb780000f60 tx=0x7fb780009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:14.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.231+0000 7fb7777fe640 1 -- 172.21.15.139:0/3698984434 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb7840587f0 con 0x7fb790154c20 2024-09-16T00:09:14.259 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:14.259+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:14.275 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:14.275 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:14.286 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:09:14.360 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:14.360+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:14.455 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:14.456 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:14.461 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:14.460+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:14.466 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:09:14.561 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:14.561+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:14.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.578+0000 7f2456bc4640 1 -- 172.21.15.139:0/2049351731 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f24500c9270 con 0x7f24500c6da0 2024-09-16T00:09:14.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:14.608+0000 7fb79519f640 1 -- 172.21.15.139:0/3698984434 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) -- 0x7fb7900cca70 con 0x7fb790154c20 2024-09-16T00:09:14.632 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:14.632 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:14.642 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:09:14.662 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:14.662+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:14.764 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:14.763+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:14.805 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:14.805 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:14.815 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:09:14.900 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:14.900+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:14.929 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:09:14.973 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:14.973+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:14.982 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:14.982 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:14.992 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:09:15.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.047+0000 7f13aa731640 1 Processor -- start 2024-09-16T00:09:15.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.047+0000 7f13aa731640 1 -- start start 2024-09-16T00:09:15.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.048+0000 7f13aa731640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f13a4073bf0 0x7f13a41661c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.048+0000 7f13aa731640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13a40c8800 con 0x7f13a40b2170 2024-09-16T00:09:15.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.048+0000 7f13aa731640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13a40c7bd0 con 0x7f13a406da30 2024-09-16T00:09:15.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.048+0000 7f13aa731640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f13a40c7980 con 0x7f13a4073bf0 2024-09-16T00:09:15.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.048+0000 7f13a3fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f13a40b2170 0x7f13a415a910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46764/0 (socket says 172.21.15.139:46764) 2024-09-16T00:09:15.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.048+0000 7f13a3fff640 1 -- 172.21.15.139:0/3423544950 learned_addr learned my addr 172.21.15.139:0/3423544950 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:15.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.048+0000 7f13a8cce640 1 --2- 172.21.15.139:0/3423544950 >> v2:172.21.15.164:3300/0 conn(0x7f13a4073bf0 0x7f13a41661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.049+0000 7f13a2ffd640 1 -- 172.21.15.139:0/3423544950 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2548794821 0 0) 0x7f13a40c8800 con 0x7f13a40b2170 2024-09-16T00:09:15.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.049+0000 7f13a8cce640 1 -- 172.21.15.139:0/3423544950 >> v1:172.21.15.164:6789/0 conn(0x7f13a406da30 legacy=0x7f13a40cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.049+0000 7f13a8cce640 1 -- 172.21.15.139:0/3423544950 >> v1:172.21.15.139:6789/0 conn(0x7f13a40b2170 legacy=0x7f13a415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.049+0000 7f13a8cce640 1 -- 172.21.15.139:0/3423544950 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a40c7eb0 con 0x7f13a4073bf0 2024-09-16T00:09:15.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.050+0000 7f13a8cce640 1 --2- 172.21.15.139:0/3423544950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a4073bf0 0x7f13a41661c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f139c009d90 tx=0x7f139c02a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=498019d74c41cf23 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:15.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.050+0000 7f13a2ffd640 1 -- 172.21.15.139:0/3423544950 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c007740 con 0x7f13a4073bf0 2024-09-16T00:09:15.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.050+0000 7f13a2ffd640 1 -- 172.21.15.139:0/3423544950 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139c0078e0 con 0x7f13a4073bf0 2024-09-16T00:09:15.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.050+0000 7f13a2ffd640 1 -- 172.21.15.139:0/3423544950 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c007cb0 con 0x7f13a4073bf0 2024-09-16T00:09:15.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.051+0000 7f13aa731640 1 -- 172.21.15.139:0/3423544950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a4073bf0 msgr2=0x7f13a41661c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.051+0000 7f13aa731640 1 --2- 172.21.15.139:0/3423544950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a4073bf0 0x7f13a41661c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f139c009d90 tx=0x7f139c02a610 comp rx=0 tx=0).stop 2024-09-16T00:09:15.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.051+0000 7f13aa731640 1 -- 172.21.15.139:0/3423544950 shutdown_connections 2024-09-16T00:09:15.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.051+0000 7f13aa731640 1 --2- 172.21.15.139:0/3423544950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a4073bf0 0x7f13a41661c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.051+0000 7f13aa731640 1 -- 172.21.15.139:0/3423544950 >> 172.21.15.139:0/3423544950 conn(0x7f13a4067950 msgr2=0x7f13a4067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:15.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.051+0000 7f13aa731640 1 -- 172.21.15.139:0/3423544950 shutdown_connections 2024-09-16T00:09:15.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.051+0000 7f13aa731640 1 -- 172.21.15.139:0/3423544950 wait complete. 2024-09-16T00:09:15.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.052+0000 7f13aa731640 1 Processor -- start 2024-09-16T00:09:15.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.052+0000 7f13aa731640 1 -- start start 2024-09-16T00:09:15.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.053+0000 7f13aa731640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a406da30 0x7f13a40ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.053+0000 7f13aa731640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13a4073bf0 0x7f13a41e4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.053+0000 7f13aa731640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a40b2170 0x7f13a41e1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.053+0000 7f13aa731640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f13a40c4020 con 0x7f13a4073bf0 2024-09-16T00:09:15.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.053+0000 7f13aa731640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4073480 con 0x7f13a406da30 2024-09-16T00:09:15.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.053+0000 7f13aa731640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f13a40c41a0 con 0x7f13a40b2170 2024-09-16T00:09:15.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.053+0000 7f13a3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a406da30 0x7f13a40ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.053+0000 7f13a37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13a4073bf0 0x7f13a41e4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.053+0000 7f13a3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a406da30 0x7f13a40ae9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:35462/0 (socket says 172.21.15.139:35462) 2024-09-16T00:09:15.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.053+0000 7f13a3fff640 1 -- 172.21.15.139:0/1561463188 learned_addr learned my addr 172.21.15.139:0/1561463188 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:15.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.053+0000 7f13a8cce640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a40b2170 0x7f13a41e1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.054+0000 7f13a3fff640 1 -- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a40b2170 msgr2=0x7f13a41e1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.054+0000 7f13a3fff640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a40b2170 0x7f13a41e1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.054+0000 7f13a3fff640 1 -- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13a4073bf0 msgr2=0x7f13a41e4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.054+0000 7f13a3fff640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13a4073bf0 0x7f13a41e4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.054+0000 7f13a3fff640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a4157910 con 0x7f13a406da30 2024-09-16T00:09:15.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.054+0000 7f13a37fe640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13a4073bf0 0x7f13a41e4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:15.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.054+0000 7f13a3fff640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a406da30 0x7f13a40ae9f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f139400c3c0 tx=0x7f139400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:15.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.054+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394017070 con 0x7f13a406da30 2024-09-16T00:09:15.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.054+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1394005ce0 con 0x7f13a406da30 2024-09-16T00:09:15.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.054+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a4157bd0 con 0x7f13a406da30 2024-09-16T00:09:15.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.054+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394007310 con 0x7f13a406da30 2024-09-16T00:09:15.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.054+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a41574f0 con 0x7f13a406da30 2024-09-16T00:09:15.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.055+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f13a41576f0 con 0x7f13a406da30 2024-09-16T00:09:15.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.056+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f139401a070 con 0x7f13a406da30 2024-09-16T00:09:15.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.056+0000 7f13a0ff9640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f139006c2b0 0x7f139006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.057+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5915+0+0 (secure 0 0 0) 0x7f139408a800 con 0x7f13a406da30 2024-09-16T00:09:15.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.057+0000 7f13a37fe640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f139006c2b0 0x7f139006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.057+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v67) ==== 54+0+16 (secure 0 0 0) 0x7f1394090170 con 0x7f13a406da30 2024-09-16T00:09:15.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.057+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f13a41e4000 0x7f13a40c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.057+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13a406d250 con 0x7f13a41e4000 2024-09-16T00:09:15.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.057+0000 7f13a37fe640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f139006c2b0 0x7f139006e770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f13a40b1a60 tx=0x7f1398009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:15.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.057+0000 7f13a8cce640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f13a41e4000 0x7f13a40c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.058+0000 7f13a8cce640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f13a41e4000 0x7f13a40c8b20 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:15.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.058+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13a406d250 con 0x7f13a41e4000 2024-09-16T00:09:15.066 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:15.065+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:15.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.070+0000 7f244d7fa640 1 -- 172.21.15.139:0/2049351731 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v68) ==== 101+0+0 (secure 0 0 0) 0x7f244405cc80 con 0x7f24500c6da0 2024-09-16T00:09:15.071 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is unset 2024-09-16T00:09:15.074 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T00:09:15.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.073+0000 7fb7777fe640 1 -- 172.21.15.139:0/3698984434 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v68) ==== 113+0+0 (secure 0 0 0) 0x7fb78405caf0 con 0x7fb790154c20 2024-09-16T00:09:15.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.073+0000 7f2456bc4640 1 -- 172.21.15.139:0/2049351731 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f243406c2b0 msgr2=0x7f243406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.073+0000 7f2456bc4640 1 --2- 172.21.15.139:0/2049351731 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f243406c2b0 0x7f243406e770 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f243c007fd0 tx=0x7f243c009040 comp rx=0 tx=0).stop 2024-09-16T00:09:15.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.074+0000 7f2456bc4640 1 -- 172.21.15.139:0/2049351731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24500c6da0 msgr2=0x7f24500b97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.074+0000 7f2456bc4640 1 --2- 172.21.15.139:0/2049351731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24500c6da0 0x7f24500b97c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f244400c410 tx=0x7f244400c8d0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.074+0000 7f2456bc4640 1 -- 172.21.15.139:0/2049351731 shutdown_connections 2024-09-16T00:09:15.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.074+0000 7f2456bc4640 1 --2- 172.21.15.139:0/2049351731 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f243406c2b0 0x7f243406e770 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.074+0000 7f2456bc4640 1 --2- 172.21.15.139:0/2049351731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2450154aa0 0x7f2450156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.074+0000 7f2456bc4640 1 --2- 172.21.15.139:0/2049351731 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24500c7aa0 0x7f24501f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.074+0000 7f2456bc4640 1 --2- 172.21.15.139:0/2049351731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24500c6da0 0x7f24500b97c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.074+0000 7f2456bc4640 1 -- 172.21.15.139:0/2049351731 >> 172.21.15.139:0/2049351731 conn(0x7f245006cc30 msgr2=0x7f245015c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:15.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.074+0000 7f2456bc4640 1 -- 172.21.15.139:0/2049351731 shutdown_connections 2024-09-16T00:09:15.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.074+0000 7f2456bc4640 1 -- 172.21.15.139:0/2049351731 wait complete. 2024-09-16T00:09:15.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.077+0000 7fb79519f640 1 -- 172.21.15.139:0/3698984434 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb77c06c2b0 msgr2=0x7fb77c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.077+0000 7fb79519f640 1 --2- 172.21.15.139:0/3698984434 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb77c06c2b0 0x7fb77c06e770 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb780000f60 tx=0x7fb780009040 comp rx=0 tx=0).stop 2024-09-16T00:09:15.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.077+0000 7fb79519f640 1 -- 172.21.15.139:0/3698984434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb790154c20 msgr2=0x7fb7900cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.077+0000 7fb79519f640 1 --2- 172.21.15.139:0/3698984434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb790154c20 0x7fb7900cf8c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb78400c460 tx=0x7fb78400c920 comp rx=0 tx=0).stop 2024-09-16T00:09:15.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.077+0000 7fb79519f640 1 -- 172.21.15.139:0/3698984434 shutdown_connections 2024-09-16T00:09:15.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.077+0000 7fb79519f640 1 --2- 172.21.15.139:0/3698984434 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb77c06c2b0 0x7fb77c06e770 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.077+0000 7fb79519f640 1 --2- 172.21.15.139:0/3698984434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb79015b280 0x7fb790152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.077+0000 7fb79519f640 1 --2- 172.21.15.139:0/3698984434 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb790154c20 0x7fb7900cf8c0 secure :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb78400c460 tx=0x7fb78400c920 comp rx=0 tx=0).stop 2024-09-16T00:09:15.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.077+0000 7fb79519f640 1 --2- 172.21.15.139:0/3698984434 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb790068060 0x7fb79015d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.077+0000 7fb79519f640 1 -- 172.21.15.139:0/3698984434 >> 172.21.15.139:0/3698984434 conn(0x7fb79006ca40 msgr2=0x7fb7900c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:15.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.078+0000 7fb79519f640 1 -- 172.21.15.139:0/3698984434 shutdown_connections 2024-09-16T00:09:15.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.078+0000 7fb79519f640 1 -- 172.21.15.139:0/3698984434 wait complete. 2024-09-16T00:09:15.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.081+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f13a406c6f0 con 0x7f13a41e4000 2024-09-16T00:09:15.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.083+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f13a406c6f0 con 0x7f13a41e4000 2024-09-16T00:09:15.084 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:15.085 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:15.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.084+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f13a41f1300 0x7f13a41507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.084+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f13a40c97f0 con 0x7f13a41f1300 2024-09-16T00:09:15.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.084+0000 7f13a3fff640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f13a41f1300 0x7f13a41507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.084+0000 7f13a3fff640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f13a41f1300 0x7f13a41507d0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:15.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.086+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13a40c97f0 con 0x7f13a41f1300 2024-09-16T00:09:15.098 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T00:09:15.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.116+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f13a4068140 con 0x7f13a41f1300 2024-09-16T00:09:15.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.118+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f13a4068140 con 0x7f13a41f1300 2024-09-16T00:09:15.119 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:15.120 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:15.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.118+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f13a4150f60 0x7f13a41f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.119+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f13a41f4720 con 0x7f13a4150f60 2024-09-16T00:09:15.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.119+0000 7f13a37fe640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f13a4150f60 0x7f13a41f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.119+0000 7f13a37fe640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f13a4150f60 0x7f13a41f41e0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:15.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.120+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13a41f4720 con 0x7f13a4150f60 2024-09-16T00:09:15.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.137+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f13a41f4b20 con 0x7f13a4150f60 2024-09-16T00:09:15.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.138+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1398014550 con 0x7f13a4150f60 2024-09-16T00:09:15.139 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:15.140 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:15.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.139+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f13a4062f00 0x7f13a41e0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.139+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f13a4063380 con 0x7f13a4062f00 2024-09-16T00:09:15.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.139+0000 7f13a8cce640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f13a4062f00 0x7f13a41e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.139+0000 7f13a8cce640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f13a4062f00 0x7f13a41e0990 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:15.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.140+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13a4063380 con 0x7f13a4062f00 2024-09-16T00:09:15.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.153+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f13a41e11a0 con 0x7f13a4062f00 2024-09-16T00:09:15.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.156+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f13a41e11a0 con 0x7f13a4062f00 2024-09-16T00:09:15.157 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.156+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f13a41e1630 0x7f13a402c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.156+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f13a4063590 con 0x7f13a41e1630 2024-09-16T00:09:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.157+0000 7f13a3fff640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f13a41e1630 0x7f13a402c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.157+0000 7f13a3fff640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f13a41e1630 0x7f13a402c640 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.158+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13a4063590 con 0x7f13a41e1630 2024-09-16T00:09:15.162 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:15.162 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:15.162 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:15.162 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:15.167 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:15.166+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:15.171 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:09:15.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.171+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f13a402cb80 con 0x7f13a41e1630 2024-09-16T00:09:15.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.173+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f13a402cb80 con 0x7f13a41e1630 2024-09-16T00:09:15.174 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:15.174 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:15.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.173+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f13a402d2a0 0x7f13a402d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.173+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f13a4209950 con 0x7f13a402d2a0 2024-09-16T00:09:15.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.173+0000 7f13a37fe640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f13a402d2a0 0x7f13a402d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.174+0000 7f13a37fe640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f13a402d2a0 0x7f13a402d6c0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:15.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.174+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13a4209950 con 0x7f13a402d2a0 2024-09-16T00:09:15.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.188+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f13a42070c0 con 0x7f13a402d2a0 2024-09-16T00:09:15.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.190+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f13a42070c0 con 0x7f13a402d2a0 2024-09-16T00:09:15.191 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:15.192 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:15.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.190+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f13a42077e0 0x7f13a4209bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.190+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f13a420ca70 con 0x7f13a42077e0 2024-09-16T00:09:15.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.191+0000 7f13a8cce640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f13a42077e0 0x7f13a4209bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.191+0000 7f13a8cce640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f13a42077e0 0x7f13a4209bd0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:15.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.192+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13a420ca70 con 0x7f13a42077e0 2024-09-16T00:09:15.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.206+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f13a40635e0 con 0x7f13a42077e0 2024-09-16T00:09:15.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.209+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f13a40635e0 con 0x7f13a42077e0 2024-09-16T00:09:15.209 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:15.210 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:09:15.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.209+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f13a420a620 0x7f13a420aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.209+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f13a420c6f0 con 0x7f13a420a620 2024-09-16T00:09:15.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.209+0000 7f13a3fff640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f13a420a620 0x7f13a420aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.209+0000 7f13a3fff640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f13a420a620 0x7f13a420aaa0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:15.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.210+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13a420c6f0 con 0x7f13a420a620 2024-09-16T00:09:15.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.220+0000 7f51c59f8640 1 Processor -- start 2024-09-16T00:09:15.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.220+0000 7f51c59f8640 1 -- start start 2024-09-16T00:09:15.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.221+0000 7f51c59f8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51c0152b20 0x7f51c00c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.221+0000 7f51c59f8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f51c00c8c50 0x7f51c006ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.221+0000 7f51c59f8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51c00ce310 con 0x7f51c0070440 2024-09-16T00:09:15.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.221+0000 7f51c59f8640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f51c00c5470 con 0x7f51c00c8c50 2024-09-16T00:09:15.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.221+0000 7f51c59f8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f51c00cbba0 con 0x7f51c0152b20 2024-09-16T00:09:15.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.221+0000 7f51bf7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f51c0070440 0x7f51c01587b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46772/0 (socket says 172.21.15.139:46772) 2024-09-16T00:09:15.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.221+0000 7f51bf7fe640 1 -- 172.21.15.139:0/2755365633 learned_addr learned my addr 172.21.15.139:0/2755365633 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:15.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.221+0000 7f51b7fff640 1 --2- 172.21.15.139:0/2755365633 >> v2:172.21.15.164:3300/0 conn(0x7f51c00c8c50 0x7f51c006ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.221+0000 7f51beffd640 1 --2- 172.21.15.139:0/2755365633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51c0152b20 0x7f51c00c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.221+0000 7f51be7fc640 1 -- 172.21.15.139:0/2755365633 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4264314805 0 0) 0x7f51c00ce310 con 0x7f51c0070440 2024-09-16T00:09:15.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.221+0000 7f51be7fc640 1 -- 172.21.15.139:0/2755365633 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51ac003520 con 0x7f51c0070440 2024-09-16T00:09:15.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.222+0000 7f51b7fff640 1 -- 172.21.15.139:0/2755365633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51c0152b20 msgr2=0x7f51c00c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:15.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.222+0000 7f51b7fff640 1 --2- 172.21.15.139:0/2755365633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51c0152b20 0x7f51c00c4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.222+0000 7f51b7fff640 1 -- 172.21.15.139:0/2755365633 >> v1:172.21.15.139:6789/0 conn(0x7f51c0070440 legacy=0x7f51c01587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.222+0000 7f51b7fff640 1 -- 172.21.15.139:0/2755365633 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c00c3380 con 0x7f51c00c8c50 2024-09-16T00:09:15.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.222+0000 7f51beffd640 1 --2- 172.21.15.139:0/2755365633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51c0152b20 0x7f51c00c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:15.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.222+0000 7f51b7fff640 1 --2- 172.21.15.139:0/2755365633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51c00c8c50 0x7f51c006ff00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f51a80097d0 tx=0x7f51a802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3563eedb121dd65e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:15.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.223+0000 7f51be7fc640 1 -- 172.21.15.139:0/2755365633 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8037070 con 0x7f51c00c8c50 2024-09-16T00:09:15.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.223+0000 7f51be7fc640 1 -- 172.21.15.139:0/2755365633 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a802ae40 con 0x7f51c00c8c50 2024-09-16T00:09:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.223+0000 7f51be7fc640 1 -- 172.21.15.139:0/2755365633 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8032660 con 0x7f51c00c8c50 2024-09-16T00:09:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.223+0000 7f51c59f8640 1 -- 172.21.15.139:0/2755365633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51c00c8c50 msgr2=0x7f51c006ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.223+0000 7f51c59f8640 1 --2- 172.21.15.139:0/2755365633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51c00c8c50 0x7f51c006ff00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f51a80097d0 tx=0x7f51a802a130 comp rx=0 tx=0).stop 2024-09-16T00:09:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.223+0000 7f51c59f8640 1 -- 172.21.15.139:0/2755365633 shutdown_connections 2024-09-16T00:09:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.223+0000 7f51c59f8640 1 --2- 172.21.15.139:0/2755365633 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51c00c8c50 0x7f51c006ff00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.223+0000 7f51c59f8640 1 --2- 172.21.15.139:0/2755365633 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51c0152b20 0x7f51c00c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.223+0000 7f51c59f8640 1 -- 172.21.15.139:0/2755365633 >> 172.21.15.139:0/2755365633 conn(0x7f51c006bd00 msgr2=0x7f51c0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.223+0000 7f51c59f8640 1 -- 172.21.15.139:0/2755365633 shutdown_connections 2024-09-16T00:09:15.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.223+0000 7f51c59f8640 1 -- 172.21.15.139:0/2755365633 wait complete. 2024-09-16T00:09:15.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.224+0000 7f51c59f8640 1 Processor -- start 2024-09-16T00:09:15.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.225+0000 7f51c59f8640 1 -- start start 2024-09-16T00:09:15.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.225+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f13a420afe0 con 0x7f13a420a620 2024-09-16T00:09:15.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.225+0000 7f51c59f8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51c0070440 0x7f51c01e40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.225+0000 7f51c59f8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51c00c8c50 0x7f51c01f1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.225+0000 7f51c59f8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51c0152b20 0x7f51c00cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.225+0000 7f51c59f8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f51c00c3940 con 0x7f51c00c8c50 2024-09-16T00:09:15.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.225+0000 7f51c59f8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f51c00ce5b0 con 0x7f51c0152b20 2024-09-16T00:09:15.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.225+0000 7f51c59f8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f51c0070cd0 con 0x7f51c0070440 2024-09-16T00:09:15.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.225+0000 7f51b7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51c00c8c50 0x7f51c01f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.225+0000 7f51b7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51c00c8c50 0x7f51c01f1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58824/0 (socket says 172.21.15.139:58824) 2024-09-16T00:09:15.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.225+0000 7f51b7fff640 1 -- 172.21.15.139:0/3385353640 learned_addr learned my addr 172.21.15.139:0/3385353640 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:15.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.225+0000 7f51beffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51c0070440 0x7f51c01e40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.225+0000 7f51bf7fe640 1 --2- 172.21.15.139:0/3385353640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51c0152b20 0x7f51c00cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.226+0000 7f51b7fff640 1 -- 172.21.15.139:0/3385353640 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51c0070440 msgr2=0x7f51c01e40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.226+0000 7f51b7fff640 1 --2- 172.21.15.139:0/3385353640 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51c0070440 0x7f51c01e40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.226+0000 7f51b7fff640 1 -- 172.21.15.139:0/3385353640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51c0152b20 msgr2=0x7f51c00cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.226+0000 7f51b7fff640 1 --2- 172.21.15.139:0/3385353640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51c0152b20 0x7f51c00cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.226+0000 7f51b7fff640 1 -- 172.21.15.139:0/3385353640 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c01f4630 con 0x7f51c00c8c50 2024-09-16T00:09:15.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.226+0000 7f51beffd640 1 --2- 172.21.15.139:0/3385353640 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51c0070440 0x7f51c01e40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:15.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.226+0000 7f51b7fff640 1 --2- 172.21.15.139:0/3385353640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51c00c8c50 0x7f51c01f1670 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f51a8035040 tx=0x7f51a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:15.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.226+0000 7f51bcff9640 1 -- 172.21.15.139:0/3385353640 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8041070 con 0x7f51c00c8c50 2024-09-16T00:09:15.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.226+0000 7f51bcff9640 1 -- 172.21.15.139:0/3385353640 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a803c460 con 0x7f51c00c8c50 2024-09-16T00:09:15.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.226+0000 7f51c59f8640 1 -- 172.21.15.139:0/3385353640 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c01f48f0 con 0x7f51c00c8c50 2024-09-16T00:09:15.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.226+0000 7f51bcff9640 1 -- 172.21.15.139:0/3385353640 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8037040 con 0x7f51c00c8c50 2024-09-16T00:09:15.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.226+0000 7f51c59f8640 1 -- 172.21.15.139:0/3385353640 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c01f4eb0 con 0x7f51c00c8c50 2024-09-16T00:09:15.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.227+0000 7f51c59f8640 1 -- 172.21.15.139:0/3385353640 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c006ff00 con 0x7f51c00c8c50 2024-09-16T00:09:15.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.227+0000 7f13a0ff9640 1 -- 172.21.15.139:0/1561463188 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f13a420afe0 con 0x7f13a420a620 2024-09-16T00:09:15.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f51bcff9640 1 -- 172.21.15.139:0/3385353640 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f51a803a070 con 0x7f51c00c8c50 2024-09-16T00:09:15.237 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:15.238 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:15.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f51bcff9640 1 --2- 172.21.15.139:0/3385353640 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f51a006c2b0 0x7f51a006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:15.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f13a41e4000 msgr2=0x7f13a40c8b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f13a41e4000 0x7f13a40c8b20 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f13a41f1300 msgr2=0x7f13a41507d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f13a41f1300 0x7f13a41507d0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f13a4150f60 msgr2=0x7f13a41f41e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f13a4150f60 0x7f13a41f41e0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f51bcff9640 1 -- 172.21.15.139:0/3385353640 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5944+0+0 (secure 0 0 0) 0x7f51a80ab710 con 0x7f51c00c8c50 2024-09-16T00:09:15.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13a8cce640 1 -- 172.21.15.139:0/1561463188 reap_dead start 2024-09-16T00:09:15.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f13a4062f00 msgr2=0x7f13a41e0990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f13a4062f00 0x7f13a41e0990 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f13a41e1630 msgr2=0x7f13a402c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f13a41e1630 0x7f13a402c640 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f13a402d2a0 msgr2=0x7f13a402d6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f13a402d2a0 0x7f13a402d6c0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f13a42077e0 msgr2=0x7f13a4209bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f13a42077e0 0x7f13a4209bd0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f51beffd640 1 --2- 172.21.15.139:0/3385353640 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f51a006c2b0 0x7f51a006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:15.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f13a420a620 msgr2=0x7f13a420aaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f13a420a620 0x7f13a420aaa0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.228+0000 7f13a8cce640 1 -- 172.21.15.139:0/1561463188 reap_dead start 2024-09-16T00:09:15.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.229+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f139006c2b0 msgr2=0x7f139006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.229+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f139006c2b0 0x7f139006e770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f13a40b1a60 tx=0x7f1398009040 comp rx=0 tx=0).stop 2024-09-16T00:09:15.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.229+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a406da30 msgr2=0x7f13a40ae9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:15.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.229+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a406da30 0x7f13a40ae9f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f139400c3c0 tx=0x7f139400c880 comp rx=0 tx=0).stop 2024-09-16T00:09:15.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.229+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 shutdown_connections 2024-09-16T00:09:15.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.229+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f139006c2b0 0x7f139006e770 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.229+0000 7f13aa731640 1 --2- 172.21.15.139:0/1561463188 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a406da30 0x7f13a40ae9f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:15.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.229+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 >> 172.21.15.139:0/1561463188 conn(0x7f13a4067950 msgr2=0x7f13a415aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:15.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.229+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 shutdown_connections 2024-09-16T00:09:15.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.229+0000 7f13aa731640 1 -- 172.21.15.139:0/1561463188 wait complete. 2024-09-16T00:09:15.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.229+0000 7f51beffd640 1 --2- 172.21.15.139:0/3385353640 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f51a006c2b0 0x7f51a006e770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009560 tx=0x7f51b0009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:15.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.233+0000 7f51bcff9640 1 -- 172.21.15.139:0/3385353640 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f51a80796a0 con 0x7f51c00c8c50 2024-09-16T00:09:15.268 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:15.267+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:15.289 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:15.289 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:15.289 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:15.289 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:15.289 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:15.300 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:09:15.369 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:15.369+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:15.428 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:15.428 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:15.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:15.429 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:15.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.429 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:22 2.e e22/19 2:717a0223:::608.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:15.429 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:09:15.429 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 96.809356159999993, 2024-09-16T00:09:15.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042195870000000003, 2024-09-16T00:09:15.429 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.429 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.429 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.429 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:09:15.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:09:15.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616426+0000", 2024-09-16T00:09:15.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.037e-06 2024-09-16T00:09:15.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616435+0000", 2024-09-16T00:09:15.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6440000000000006e-06 2024-09-16T00:09:15.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616447+0000", 2024-09-16T00:09:15.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2466000000000001e-05 2024-09-16T00:09:15.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616456+0000", 2024-09-16T00:09:15.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8799999999999997e-06 2024-09-16T00:09:15.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616517+0000", 2024-09-16T00:09:15.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1171000000000001e-05 2024-09-16T00:09:15.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616540+0000", 2024-09-16T00:09:15.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2991000000000001e-05 2024-09-16T00:09:15.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620599+0000", 2024-09-16T00:09:15.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040586140000000003 2024-09-16T00:09:15.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620645+0000", 2024-09-16T00:09:15.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5784000000000001e-05 2024-09-16T00:09:15.433 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.433 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.433 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.434 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:15.434 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:09:15.434 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 96.809266429999994, 2024-09-16T00:09:15.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044836249999999998, 2024-09-16T00:09:15.434 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.434 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.434 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:09:15.434 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:09:15.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:09:15.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616518+0000", 2024-09-16T00:09:15.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3699999999999999e-06 2024-09-16T00:09:15.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:09:15.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1479999999999997e-06 2024-09-16T00:09:15.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616534+0000", 2024-09-16T00:09:15.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0641e-05 2024-09-16T00:09:15.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616541+0000", 2024-09-16T00:09:15.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3339999999999996e-06 2024-09-16T00:09:15.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617318+0000", 2024-09-16T00:09:15.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00077694799999999996 2024-09-16T00:09:15.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617409+0000", 2024-09-16T00:09:15.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0740000000000002e-05 2024-09-16T00:09:15.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620998+0000", 2024-09-16T00:09:15.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035894439999999998 2024-09-16T00:09:15.438 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.438 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.438 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.438 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:15.438 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:09:15.438 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 96.809218526999999, 2024-09-16T00:09:15.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041597320000000002, 2024-09-16T00:09:15.438 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.439 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.439 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:09:15.439 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:09:15.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:09:15.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:09:15.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:09:15.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616566+0000", 2024-09-16T00:09:15.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6979999999999998e-06 2024-09-16T00:09:15.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616574+0000", 2024-09-16T00:09:15.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7759999999999997e-06 2024-09-16T00:09:15.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616580+0000", 2024-09-16T00:09:15.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:09:15.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616800+0000", 2024-09-16T00:09:15.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022016700000000001 2024-09-16T00:09:15.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616904+0000", 2024-09-16T00:09:15.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104579 2024-09-16T00:09:15.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620722+0000", 2024-09-16T00:09:15.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003818133 2024-09-16T00:09:15.442 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.442 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.442 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.443 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:15.443 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:09:15.443 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 96.808967842000001, 2024-09-16T00:09:15.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038451570000000001, 2024-09-16T00:09:15.443 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.443 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.443 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.443 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:09:15.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:09:15.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616814+0000", 2024-09-16T00:09:15.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9299999999999997e-07 2024-09-16T00:09:15.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616824+0000", 2024-09-16T00:09:15.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6600000000000007e-06 2024-09-16T00:09:15.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616835+0000", 2024-09-16T00:09:15.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0713000000000001e-05 2024-09-16T00:09:15.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616841+0000", 2024-09-16T00:09:15.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.934e-06 2024-09-16T00:09:15.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616948+0000", 2024-09-16T00:09:15.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010631100000000001 2024-09-16T00:09:15.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616967+0000", 2024-09-16T00:09:15.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9598000000000001e-05 2024-09-16T00:09:15.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620619+0000", 2024-09-16T00:09:15.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036515810000000001 2024-09-16T00:09:15.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620659+0000", 2024-09-16T00:09:15.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9666999999999998e-05 2024-09-16T00:09:15.447 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.447 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.448 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.448 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:15.448 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:09:15.448 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 96.808920889999996, 2024-09-16T00:09:15.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038240129999999998, 2024-09-16T00:09:15.448 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.448 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.448 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.448 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:09:15.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:09:15.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616861+0000", 2024-09-16T00:09:15.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0200000000000003e-07 2024-09-16T00:09:15.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616864+0000", 2024-09-16T00:09:15.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5090000000000002e-06 2024-09-16T00:09:15.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616871+0000", 2024-09-16T00:09:15.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9249999999999998e-06 2024-09-16T00:09:15.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616877+0000", 2024-09-16T00:09:15.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.981e-06 2024-09-16T00:09:15.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617168+0000", 2024-09-16T00:09:15.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029035699999999998 2024-09-16T00:09:15.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617184+0000", 2024-09-16T00:09:15.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6005999999999999e-05 2024-09-16T00:09:15.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620661+0000", 2024-09-16T00:09:15.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034775330000000001 2024-09-16T00:09:15.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620684+0000", 2024-09-16T00:09:15.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3300000000000001e-05 2024-09-16T00:09:15.452 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.453 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.453 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.453 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:09:15.453 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:09:15.453 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 96.808465447000003, 2024-09-16T00:09:15.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045415610000000004, 2024-09-16T00:09:15.453 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.453 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.453 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.453 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:09:15.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:09:15.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:09:15.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8800000000000002e-07 2024-09-16T00:09:15.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617321+0000", 2024-09-16T00:09:15.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6979999999999997e-06 2024-09-16T00:09:15.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617330+0000", 2024-09-16T00:09:15.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6729999999999998e-06 2024-09-16T00:09:15.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617336+0000", 2024-09-16T00:09:15.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2110000000000003e-06 2024-09-16T00:09:15.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617394+0000", 2024-09-16T00:09:15.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8087999999999997e-05 2024-09-16T00:09:15.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617412+0000", 2024-09-16T00:09:15.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8099999999999999e-05 2024-09-16T00:09:15.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621818+0000", 2024-09-16T00:09:15.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044061209999999998 2024-09-16T00:09:15.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621857+0000", 2024-09-16T00:09:15.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9082000000000003e-05 2024-09-16T00:09:15.457 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.457 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.457 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.458 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:23 2.e e22/19 2:717a0223:::608.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:15.458 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:09:15.458 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 96.807569426000001, 2024-09-16T00:09:15.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036198279999999998, 2024-09-16T00:09:15.458 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.458 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.458 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.458 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:09:15.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618212+0000", 2024-09-16T00:09:15.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618213+0000", 2024-09-16T00:09:15.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7899999999999998e-07 2024-09-16T00:09:15.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618218+0000", 2024-09-16T00:09:15.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4099999999999999e-06 2024-09-16T00:09:15.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618227+0000", 2024-09-16T00:09:15.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5009999999999993e-06 2024-09-16T00:09:15.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618234+0000", 2024-09-16T00:09:15.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:09:15.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618307+0000", 2024-09-16T00:09:15.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3421000000000001e-05 2024-09-16T00:09:15.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618328+0000", 2024-09-16T00:09:15.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0919000000000001e-05 2024-09-16T00:09:15.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621804+0000", 2024-09-16T00:09:15.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034756560000000001 2024-09-16T00:09:15.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621832+0000", 2024-09-16T00:09:15.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7966999999999998e-05 2024-09-16T00:09:15.462 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.462 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.462 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.462 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:243 3.24 3:25048551:::100000002f0.00000000:head [write 0~1428] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:15.463 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.799025+0000", 2024-09-16T00:09:15.463 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.626756092000001, 2024-09-16T00:09:15.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042804829999999999, 2024-09-16T00:09:15.463 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.463 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.463 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.463 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:15.463 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:15.463 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:15.463 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 243 2024-09-16T00:09:15.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.463 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799025+0000", 2024-09-16T00:09:15.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799025+0000", 2024-09-16T00:09:15.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799026+0000", 2024-09-16T00:09:15.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9099999999999991e-07 2024-09-16T00:09:15.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799032+0000", 2024-09-16T00:09:15.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:09:15.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799044+0000", 2024-09-16T00:09:15.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1861999999999999e-05 2024-09-16T00:09:15.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799049+0000", 2024-09-16T00:09:15.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6570000000000002e-06 2024-09-16T00:09:15.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799095+0000", 2024-09-16T00:09:15.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5479999999999998e-05 2024-09-16T00:09:15.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799175+0000", 2024-09-16T00:09:15.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0181999999999999e-05 2024-09-16T00:09:15.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:15.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799252+0000", 2024-09-16T00:09:15.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6822000000000004e-05 2024-09-16T00:09:15.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:15.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799855+0000", 2024-09-16T00:09:15.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00060294199999999997 2024-09-16T00:09:15.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:15.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803230+0000", 2024-09-16T00:09:15.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033752719999999999 2024-09-16T00:09:15.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803263+0000", 2024-09-16T00:09:15.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3339e-05 2024-09-16T00:09:15.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803306+0000", 2024-09-16T00:09:15.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2330999999999998e-05 2024-09-16T00:09:15.469 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.469 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.469 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.469 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:324 3.29 3:960c7fc3:::10000000355.00000000:head [write 0~1506] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:15.469 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.034622+0000", 2024-09-16T00:09:15.469 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.391159071000001, 2024-09-16T00:09:15.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003641746, 2024-09-16T00:09:15.470 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.470 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.470 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.470 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:15.470 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:15.470 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:15.470 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 324 2024-09-16T00:09:15.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.470 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034622+0000", 2024-09-16T00:09:15.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034622+0000", 2024-09-16T00:09:15.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034623+0000", 2024-09-16T00:09:15.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1689999999999999e-06 2024-09-16T00:09:15.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034629+0000", 2024-09-16T00:09:15.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4859999999999997e-06 2024-09-16T00:09:15.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034639+0000", 2024-09-16T00:09:15.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0142999999999999e-05 2024-09-16T00:09:15.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034644+0000", 2024-09-16T00:09:15.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1159999999999998e-06 2024-09-16T00:09:15.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034693+0000", 2024-09-16T00:09:15.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9228999999999999e-05 2024-09-16T00:09:15.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034813+0000", 2024-09-16T00:09:15.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012003200000000001 2024-09-16T00:09:15.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:15.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034940+0000", 2024-09-16T00:09:15.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012655699999999999 2024-09-16T00:09:15.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:15.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.036244+0000", 2024-09-16T00:09:15.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001304326 2024-09-16T00:09:15.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:15.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.038175+0000", 2024-09-16T00:09:15.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001930768 2024-09-16T00:09:15.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.038225+0000", 2024-09-16T00:09:15.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0046000000000002e-05 2024-09-16T00:09:15.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.038264+0000", 2024-09-16T00:09:15.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8874000000000001e-05 2024-09-16T00:09:15.475 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.475 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.475 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.476 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:411 3.29 3:96aa9846:::100000003bf.00000000:head [write 0~1161] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:15.476 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.363361+0000", 2024-09-16T00:09:15.476 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.062420021000001, 2024-09-16T00:09:15.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038090889999999999, 2024-09-16T00:09:15.476 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.476 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.476 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.476 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:15.476 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:15.476 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:15.476 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 411 2024-09-16T00:09:15.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.477 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363361+0000", 2024-09-16T00:09:15.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363361+0000", 2024-09-16T00:09:15.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363363+0000", 2024-09-16T00:09:15.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.421e-06 2024-09-16T00:09:15.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363369+0000", 2024-09-16T00:09:15.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7270000000000001e-06 2024-09-16T00:09:15.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363382+0000", 2024-09-16T00:09:15.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2700999999999999e-05 2024-09-16T00:09:15.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363388+0000", 2024-09-16T00:09:15.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0009999999999998e-06 2024-09-16T00:09:15.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363438+0000", 2024-09-16T00:09:15.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9789000000000001e-05 2024-09-16T00:09:15.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363562+0000", 2024-09-16T00:09:15.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000123798 2024-09-16T00:09:15.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:15.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363639+0000", 2024-09-16T00:09:15.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7074000000000001e-05 2024-09-16T00:09:15.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:15.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.364153+0000", 2024-09-16T00:09:15.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00051393999999999995 2024-09-16T00:09:15.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:15.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.367097+0000", 2024-09-16T00:09:15.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00294414 2024-09-16T00:09:15.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.367145+0000", 2024-09-16T00:09:15.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8591999999999998e-05 2024-09-16T00:09:15.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.367170+0000", 2024-09-16T00:09:15.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4905999999999998e-05 2024-09-16T00:09:15.481 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.482 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.482 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.482 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:53 2.14 e58/46 2:2e2fa760:::200.00000001:head v 58'16, mlcod=57'14)", 2024-09-16T00:09:15.482 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.453518+0000", 2024-09-16T00:09:15.482 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 17.972263494, 2024-09-16T00:09:15.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036209150000000002, 2024-09-16T00:09:15.482 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.482 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.482 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.482 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.453518+0000", 2024-09-16T00:09:15.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.453518+0000", 2024-09-16T00:09:15.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.453519+0000", 2024-09-16T00:09:15.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.277e-06 2024-09-16T00:09:15.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.454703+0000", 2024-09-16T00:09:15.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011842440000000001 2024-09-16T00:09:15.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.454724+0000", 2024-09-16T00:09:15.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0835999999999998e-05 2024-09-16T00:09:15.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.454734+0000", 2024-09-16T00:09:15.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0299000000000001e-05 2024-09-16T00:09:15.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.454802+0000", 2024-09-16T00:09:15.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7127999999999998e-05 2024-09-16T00:09:15.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.454820+0000", 2024-09-16T00:09:15.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8858000000000001e-05 2024-09-16T00:09:15.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.457101+0000", 2024-09-16T00:09:15.486 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022802539999999998 2024-09-16T00:09:15.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.486 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.486 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.457139+0000", 2024-09-16T00:09:15.486 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8019000000000003e-05 2024-09-16T00:09:15.486 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.486 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.486 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.486 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:714 3.22 e59/33 3:453cbc5d:::10000000536.00000000:head v 59'18, mlcod=59'16)", 2024-09-16T00:09:15.486 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.356842+0000", 2024-09-16T00:09:15.487 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 17.068938829, 2024-09-16T00:09:15.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032768649999999999, 2024-09-16T00:09:15.487 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.487 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.487 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.487 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.487 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356842+0000", 2024-09-16T00:09:15.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356842+0000", 2024-09-16T00:09:15.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356844+0000", 2024-09-16T00:09:15.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4219999999999999e-06 2024-09-16T00:09:15.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356853+0000", 2024-09-16T00:09:15.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7760000000000003e-06 2024-09-16T00:09:15.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356871+0000", 2024-09-16T00:09:15.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8114e-05 2024-09-16T00:09:15.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356881+0000", 2024-09-16T00:09:15.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0233e-05 2024-09-16T00:09:15.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356949+0000", 2024-09-16T00:09:15.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7780000000000005e-05 2024-09-16T00:09:15.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356972+0000", 2024-09-16T00:09:15.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3144999999999999e-05 2024-09-16T00:09:15.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360073+0000", 2024-09-16T00:09:15.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031013170000000001 2024-09-16T00:09:15.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.491 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.491 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360119+0000", 2024-09-16T00:09:15.491 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6078000000000002e-05 2024-09-16T00:09:15.491 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.491 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.491 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.491 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:55 2.14 e59/46 2:2e2fa760:::200.00000001:head v 59'17, mlcod=57'15)", 2024-09-16T00:09:15.491 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.643358+0000", 2024-09-16T00:09:15.491 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 16.782423199, 2024-09-16T00:09:15.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031410520000000001, 2024-09-16T00:09:15.492 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.492 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.492 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.492 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.492 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.492 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.643358+0000", 2024-09-16T00:09:15.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.492 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.492 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.492 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.643358+0000", 2024-09-16T00:09:15.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.493 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.493 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.643360+0000", 2024-09-16T00:09:15.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.136e-06 2024-09-16T00:09:15.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.493 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.493 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.644215+0000", 2024-09-16T00:09:15.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085427799999999998 2024-09-16T00:09:15.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.644236+0000", 2024-09-16T00:09:15.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1693000000000001e-05 2024-09-16T00:09:15.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.644248+0000", 2024-09-16T00:09:15.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1958e-05 2024-09-16T00:09:15.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.644310+0000", 2024-09-16T00:09:15.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1853000000000006e-05 2024-09-16T00:09:15.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.644327+0000", 2024-09-16T00:09:15.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7258999999999999e-05 2024-09-16T00:09:15.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.646455+0000", 2024-09-16T00:09:15.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021274710000000001 2024-09-16T00:09:15.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.496 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.646499+0000", 2024-09-16T00:09:15.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4403999999999997e-05 2024-09-16T00:09:15.496 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.496 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.496 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.496 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:72 2.14 e61/46 2:2e2fa760:::200.00000001:head v 61'21, mlcod=61'19)", 2024-09-16T00:09:15.496 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:01.672924+0000", 2024-09-16T00:09:15.496 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 13.752857153000001, 2024-09-16T00:09:15.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003655784, 2024-09-16T00:09:15.496 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.496 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.497 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.497 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.497 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.672924+0000", 2024-09-16T00:09:15.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.497 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.672924+0000", 2024-09-16T00:09:15.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.498 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.498 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.672925+0000", 2024-09-16T00:09:15.498 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0189999999999999e-06 2024-09-16T00:09:15.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.498 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.498 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.498 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674096+0000", 2024-09-16T00:09:15.498 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011709660000000001 2024-09-16T00:09:15.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.498 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.498 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.498 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674112+0000", 2024-09-16T00:09:15.498 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6149000000000001e-05 2024-09-16T00:09:15.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.499 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674120+0000", 2024-09-16T00:09:15.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.892e-06 2024-09-16T00:09:15.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.499 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674179+0000", 2024-09-16T00:09:15.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8876999999999999e-05 2024-09-16T00:09:15.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.499 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674197+0000", 2024-09-16T00:09:15.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8048999999999999e-05 2024-09-16T00:09:15.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.500 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.500 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.676539+0000", 2024-09-16T00:09:15.500 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023414830000000001 2024-09-16T00:09:15.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.500 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.500 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.676580+0000", 2024-09-16T00:09:15.500 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1349e-05 2024-09-16T00:09:15.500 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.500 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.501 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.501 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:86 2.1b 2:ddb4e4d7:::200.00000002:head [write 368603~2220227 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:15.501 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.644964+0000", 2024-09-16T00:09:15.501 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 11.78081708, 2024-09-16T00:09:15.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011714478, 2024-09-16T00:09:15.501 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.501 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.501 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.501 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.501 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.644964+0000", 2024-09-16T00:09:15.502 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.502 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.502 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.644964+0000", 2024-09-16T00:09:15.502 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.502 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.502 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.644966+0000", 2024-09-16T00:09:15.502 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4789999999999999e-06 2024-09-16T00:09:15.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.503 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.503 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.648979+0000", 2024-09-16T00:09:15.503 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040136900000000003 2024-09-16T00:09:15.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.503 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.503 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.648993+0000", 2024-09-16T00:09:15.503 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3129000000000001e-05 2024-09-16T00:09:15.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.503 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.503 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649000+0000", 2024-09-16T00:09:15.503 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5700000000000004e-06 2024-09-16T00:09:15.504 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.504 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.504 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.504 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649055+0000", 2024-09-16T00:09:15.504 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4914999999999999e-05 2024-09-16T00:09:15.504 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.504 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.504 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.504 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649131+0000", 2024-09-16T00:09:15.504 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6055000000000003e-05 2024-09-16T00:09:15.504 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.504 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.504 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:15.505 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649258+0000", 2024-09-16T00:09:15.505 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000127229 2024-09-16T00:09:15.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.505 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:15.505 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656218+0000", 2024-09-16T00:09:15.505 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069600469999999996 2024-09-16T00:09:15.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.505 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:15.505 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656602+0000", 2024-09-16T00:09:15.506 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038380700000000001 2024-09-16T00:09:15.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.506 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.506 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.506 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656638+0000", 2024-09-16T00:09:15.506 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5725000000000001e-05 2024-09-16T00:09:15.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.506 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.506 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.506 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656679+0000", 2024-09-16T00:09:15.506 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0831999999999998e-05 2024-09-16T00:09:15.506 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.506 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.507 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.507 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.507 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:87 2.1b 2:ddb4e4d7:::200.00000002:head [write 2588830~1400 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:15.507 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649020+0000", 2024-09-16T00:09:15.507 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 11.776760889, 2024-09-16T00:09:15.507 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0083707460000000001, 2024-09-16T00:09:15.507 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.507 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.507 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.507 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.507 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.508 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649020+0000", 2024-09-16T00:09:15.508 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.508 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.508 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649020+0000", 2024-09-16T00:09:15.508 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.508 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.508 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649021+0000", 2024-09-16T00:09:15.508 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6000000000000002e-07 2024-09-16T00:09:15.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.509 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.509 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.509 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649034+0000", 2024-09-16T00:09:15.509 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2627e-05 2024-09-16T00:09:15.509 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.509 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.509 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.509 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649045+0000", 2024-09-16T00:09:15.509 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0744000000000001e-05 2024-09-16T00:09:15.509 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.509 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.509 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.509 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649051+0000", 2024-09-16T00:09:15.510 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5459999999999997e-06 2024-09-16T00:09:15.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.510 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.510 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.510 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654369+0000", 2024-09-16T00:09:15.510 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053179519999999999 2024-09-16T00:09:15.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.510 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.510 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.510 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654435+0000", 2024-09-16T00:09:15.510 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6175999999999998e-05 2024-09-16T00:09:15.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.510 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.510 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:15.510 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654525+0000", 2024-09-16T00:09:15.511 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9856000000000002e-05 2024-09-16T00:09:15.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.511 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.511 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:15.511 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656225+0000", 2024-09-16T00:09:15.511 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001699386 2024-09-16T00:09:15.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.511 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.511 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:15.511 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657339+0000", 2024-09-16T00:09:15.511 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011145250000000001 2024-09-16T00:09:15.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.511 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657366+0000", 2024-09-16T00:09:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7172e-05 2024-09-16T00:09:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657391+0000", 2024-09-16T00:09:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4902e-05 2024-09-16T00:09:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.512 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:88 2.1b 2:ddb4e4d7:::200.00000002:head [write 2590230~1400 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:15.513 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649060+0000", 2024-09-16T00:09:15.513 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 11.776721087, 2024-09-16T00:09:15.513 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0088748730000000001, 2024-09-16T00:09:15.513 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.513 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.513 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.513 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.513 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.513 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.513 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649060+0000", 2024-09-16T00:09:15.513 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.513 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.513 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.514 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.514 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649060+0000", 2024-09-16T00:09:15.514 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.514 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.514 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.514 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.514 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649061+0000", 2024-09-16T00:09:15.514 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2300000000000001e-07 2024-09-16T00:09:15.514 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.514 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.514 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.514 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649064+0000", 2024-09-16T00:09:15.514 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3160000000000001e-06 2024-09-16T00:09:15.515 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.515 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.515 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.515 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649070+0000", 2024-09-16T00:09:15.515 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9880000000000001e-06 2024-09-16T00:09:15.515 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.515 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.515 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.515 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649074+0000", 2024-09-16T00:09:15.515 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4329999999999997e-06 2024-09-16T00:09:15.515 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.515 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.515 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.516 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654769+0000", 2024-09-16T00:09:15.516 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0056944500000000002 2024-09-16T00:09:15.516 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.516 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.516 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.516 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654875+0000", 2024-09-16T00:09:15.516 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000106183 2024-09-16T00:09:15.516 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.516 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.516 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:15.516 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654994+0000", 2024-09-16T00:09:15.516 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011867900000000001 2024-09-16T00:09:15.516 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.517 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.517 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:15.517 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657697+0000", 2024-09-16T00:09:15.517 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027033920000000002 2024-09-16T00:09:15.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.517 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.517 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:15.517 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657882+0000", 2024-09-16T00:09:15.517 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018478900000000001 2024-09-16T00:09:15.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.517 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.517 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.518 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657913+0000", 2024-09-16T00:09:15.518 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1306000000000001e-05 2024-09-16T00:09:15.518 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.518 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.518 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.518 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657935+0000", 2024-09-16T00:09:15.518 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1814000000000002e-05 2024-09-16T00:09:15.518 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.518 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.518 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.518 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.518 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.518 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:89 2.1b 2:ddb4e4d7:::200.00000002:head [write 2591630~1401 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:15.519 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:15.519 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 11.776700827999999, 2024-09-16T00:09:15.519 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011802609, 2024-09-16T00:09:15.519 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.519 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.519 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.519 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.519 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.519 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.519 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:15.519 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.519 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.519 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.520 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.520 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:15.520 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.520 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.520 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.520 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.520 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649081+0000", 2024-09-16T00:09:15.520 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7100000000000002e-07 2024-09-16T00:09:15.520 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.520 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.520 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.520 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649089+0000", 2024-09-16T00:09:15.520 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7640000000000005e-06 2024-09-16T00:09:15.520 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.521 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.521 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.521 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649094+0000", 2024-09-16T00:09:15.521 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6219999999999997e-06 2024-09-16T00:09:15.521 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.521 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.521 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.521 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649099+0000", 2024-09-16T00:09:15.521 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6970000000000001e-06 2024-09-16T00:09:15.521 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.521 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.521 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.521 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655288+0000", 2024-09-16T00:09:15.521 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061885569999999999 2024-09-16T00:09:15.522 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.522 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.522 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.522 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655365+0000", 2024-09-16T00:09:15.522 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7723000000000001e-05 2024-09-16T00:09:15.522 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.522 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.522 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:15.522 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655483+0000", 2024-09-16T00:09:15.522 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117895 2024-09-16T00:09:15.522 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.522 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.522 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:15.523 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657758+0000", 2024-09-16T00:09:15.523 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022752369999999998 2024-09-16T00:09:15.523 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.523 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.523 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:15.523 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660803+0000", 2024-09-16T00:09:15.523 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030441499999999998 2024-09-16T00:09:15.523 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.523 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.523 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.523 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660850+0000", 2024-09-16T00:09:15.523 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7017000000000001e-05 2024-09-16T00:09:15.523 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.524 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.524 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.524 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660883+0000", 2024-09-16T00:09:15.524 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3476000000000003e-05 2024-09-16T00:09:15.524 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.524 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.524 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.524 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.524 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.524 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:90 2.1b 2:ddb4e4d7:::200.00000002:head [write 2593031~1404 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:15.524 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:15.524 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 11.776662923, 2024-09-16T00:09:15.524 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011882886, 2024-09-16T00:09:15.525 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.525 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.525 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.525 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.525 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.525 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.525 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:15.525 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.525 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.525 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.525 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.525 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:15.525 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.525 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.526 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.526 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.526 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649119+0000", 2024-09-16T00:09:15.526 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:09:15.526 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.526 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.526 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.526 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649125+0000", 2024-09-16T00:09:15.526 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1650000000000003e-06 2024-09-16T00:09:15.526 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.526 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.526 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.526 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649134+0000", 2024-09-16T00:09:15.526 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5669999999999995e-06 2024-09-16T00:09:15.527 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.527 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.527 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.527 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649141+0000", 2024-09-16T00:09:15.527 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9040000000000003e-06 2024-09-16T00:09:15.527 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.527 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.527 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.527 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655796+0000", 2024-09-16T00:09:15.527 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066553649999999999 2024-09-16T00:09:15.527 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.527 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.527 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.528 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655868+0000", 2024-09-16T00:09:15.528 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2219999999999996e-05 2024-09-16T00:09:15.528 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.528 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.528 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:15.528 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655986+0000", 2024-09-16T00:09:15.528 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117457 2024-09-16T00:09:15.528 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.528 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.528 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:15.528 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657794+0000", 2024-09-16T00:09:15.528 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001807894 2024-09-16T00:09:15.528 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.528 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660890+0000", 2024-09-16T00:09:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030958449999999998 2024-09-16T00:09:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660966+0000", 2024-09-16T00:09:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6382000000000002e-05 2024-09-16T00:09:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.661001+0000", 2024-09-16T00:09:15.529 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5349000000000003e-05 2024-09-16T00:09:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:2397 3.29 3:948abdad:::10000000cb5.00000000:head [write 0~1373] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T00:09:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.655571+0000", 2024-09-16T00:09:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 11.770210638, 2024-09-16T00:09:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052117400000000003, 2024-09-16T00:09:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:15.530 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 2397 2024-09-16T00:09:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655571+0000", 2024-09-16T00:09:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:15.531 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655571+0000", 2024-09-16T00:09:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655573+0000", 2024-09-16T00:09:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0789999999999999e-06 2024-09-16T00:09:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655580+0000", 2024-09-16T00:09:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7859999999999999e-06 2024-09-16T00:09:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.532 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655592+0000", 2024-09-16T00:09:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2541000000000001e-05 2024-09-16T00:09:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655601+0000", 2024-09-16T00:09:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.721e-06 2024-09-16T00:09:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655663+0000", 2024-09-16T00:09:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2287999999999997e-05 2024-09-16T00:09:15.533 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655765+0000", 2024-09-16T00:09:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000102212 2024-09-16T00:09:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655893+0000", 2024-09-16T00:09:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012742499999999999 2024-09-16T00:09:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657022+0000", 2024-09-16T00:09:15.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011293340000000001 2024-09-16T00:09:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660686+0000", 2024-09-16T00:09:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036637409999999999 2024-09-16T00:09:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660737+0000", 2024-09-16T00:09:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1711e-05 2024-09-16T00:09:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:15.535 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660782+0000", 2024-09-16T00:09:15.536 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4901999999999998e-05 2024-09-16T00:09:15.536 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.536 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.536 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.536 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:15.536 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:15.536 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:15.538 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:09:15.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:15.460+0000 7f51c59f8640 1 -- 172.21.15.139:0/3385353640 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) -- 0x7f51c0068fb0 con 0x7f51c00c8c50 2024-09-16T00:09:15.538 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:15.536+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:15.637 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:15.636+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:15.651 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:15.651 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:15.651 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:15.661 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:09:15.745 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:15.744+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:15.818 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:15.819 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:15.819 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:15.819 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:15.819 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:15.829 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:09:15.839 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:15.838+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:15.940 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:15.938+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:15.992 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:09:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.374879136000004, 2024-09-16T00:09:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036770370000000002, 2024-09-16T00:09:15.992 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:09:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:09:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:15.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615154+0000", 2024-09-16T00:09:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1510000000000003e-06 2024-09-16T00:09:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615163+0000", 2024-09-16T00:09:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4490000000000005e-06 2024-09-16T00:09:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:15.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615182+0000", 2024-09-16T00:09:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8876e-05 2024-09-16T00:09:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615197+0000", 2024-09-16T00:09:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4671000000000001e-05 2024-09-16T00:09:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615349+0000", 2024-09-16T00:09:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015258700000000001 2024-09-16T00:09:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:09:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000204716 2024-09-16T00:09:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:09:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017250800000000001 2024-09-16T00:09:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616919+0000", 2024-09-16T00:09:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011921799999999999 2024-09-16T00:09:15.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:09:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018087769999999999 2024-09-16T00:09:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618774+0000", 2024-09-16T00:09:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6075999999999997e-05 2024-09-16T00:09:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618826+0000", 2024-09-16T00:09:15.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2046000000000003e-05 2024-09-16T00:09:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:09:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:09:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.373523348000006, 2024-09-16T00:09:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023501059999999998, 2024-09-16T00:09:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:09:15.998 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:09:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:09:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616506+0000", 2024-09-16T00:09:15.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4100000000000001e-06 2024-09-16T00:09:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617073+0000", 2024-09-16T00:09:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056725899999999997 2024-09-16T00:09:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617099+0000", 2024-09-16T00:09:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5860999999999999e-05 2024-09-16T00:09:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617114+0000", 2024-09-16T00:09:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5180999999999999e-05 2024-09-16T00:09:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617236+0000", 2024-09-16T00:09:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121662 2024-09-16T00:09:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:09:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001618733 2024-09-16T00:09:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.001 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:31 2.d e22/19 2:b50e409b:::mds_snaptable:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:09:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.372629200000006, 2024-09-16T00:09:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021851240000000001, 2024-09-16T00:09:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:09:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617399+0000", 2024-09-16T00:09:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617400+0000", 2024-09-16T00:09:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.28e-06 2024-09-16T00:09:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617410+0000", 2024-09-16T00:09:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0179e-05 2024-09-16T00:09:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617435+0000", 2024-09-16T00:09:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5221000000000001e-05 2024-09-16T00:09:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617444+0000", 2024-09-16T00:09:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1989999999999995e-06 2024-09-16T00:09:16.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617514+0000", 2024-09-16T00:09:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9104999999999997e-05 2024-09-16T00:09:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617538+0000", 2024-09-16T00:09:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4383999999999999e-05 2024-09-16T00:09:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619536+0000", 2024-09-16T00:09:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019982789999999999 2024-09-16T00:09:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619584+0000", 2024-09-16T00:09:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7477e-05 2024-09-16T00:09:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:47 3.1a 3:582f7b2d:::10000000226.00000000:head [write 0~5348] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:16.006 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.358277+0000", 2024-09-16T00:09:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.631750967999999, 2024-09-16T00:09:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003561935, 2024-09-16T00:09:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 47 2024-09-16T00:09:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358277+0000", 2024-09-16T00:09:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358277+0000", 2024-09-16T00:09:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358278+0000", 2024-09-16T00:09:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.404e-06 2024-09-16T00:09:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358290+0000", 2024-09-16T00:09:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1926999999999999e-05 2024-09-16T00:09:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358307+0000", 2024-09-16T00:09:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6529e-05 2024-09-16T00:09:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358316+0000", 2024-09-16T00:09:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1779999999999992e-06 2024-09-16T00:09:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358398+0000", 2024-09-16T00:09:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2056000000000002e-05 2024-09-16T00:09:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358488+0000", 2024-09-16T00:09:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9761999999999999e-05 2024-09-16T00:09:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358567+0000", 2024-09-16T00:09:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9173999999999998e-05 2024-09-16T00:09:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.359485+0000", 2024-09-16T00:09:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00091798800000000001 2024-09-16T00:09:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.361743+0000", 2024-09-16T00:09:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002258038 2024-09-16T00:09:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.361784+0000", 2024-09-16T00:09:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1529999999999997e-05 2024-09-16T00:09:16.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.361839+0000", 2024-09-16T00:09:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4348999999999998e-05 2024-09-16T00:09:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:74 3.13 e57/20 3:c933d8fa:::10000000241.00000000:head v 57'5, mlcod=57'3)", 2024-09-16T00:09:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.428708+0000", 2024-09-16T00:09:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.561319881999999, 2024-09-16T00:09:16.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023048140000000001, 2024-09-16T00:09:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428708+0000", 2024-09-16T00:09:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428708+0000", 2024-09-16T00:09:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428709+0000", 2024-09-16T00:09:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7000000000000003e-07 2024-09-16T00:09:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428716+0000", 2024-09-16T00:09:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5449999999999998e-06 2024-09-16T00:09:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428730+0000", 2024-09-16T00:09:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3464e-05 2024-09-16T00:09:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428738+0000", 2024-09-16T00:09:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7919999999999993e-06 2024-09-16T00:09:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428788+0000", 2024-09-16T00:09:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0813999999999998e-05 2024-09-16T00:09:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428810+0000", 2024-09-16T00:09:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1543e-05 2024-09-16T00:09:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.430972+0000", 2024-09-16T00:09:16.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021622130000000001 2024-09-16T00:09:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.431013+0000", 2024-09-16T00:09:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0572999999999999e-05 2024-09-16T00:09:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.018 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:101 3.11 e57/20 3:88015c6d:::1000000025c.00000000:head v 57'1, mlcod=0'0)", 2024-09-16T00:09:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.484094+0000", 2024-09-16T00:09:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.505934023999998, 2024-09-16T00:09:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030040079999999999, 2024-09-16T00:09:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484094+0000", 2024-09-16T00:09:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484094+0000", 2024-09-16T00:09:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484094+0000", 2024-09-16T00:09:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8999999999999996e-07 2024-09-16T00:09:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484099+0000", 2024-09-16T00:09:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6589999999999998e-06 2024-09-16T00:09:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484108+0000", 2024-09-16T00:09:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1910000000000006e-06 2024-09-16T00:09:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484115+0000", 2024-09-16T00:09:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6270000000000002e-06 2024-09-16T00:09:16.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484163+0000", 2024-09-16T00:09:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8137999999999999e-05 2024-09-16T00:09:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484182+0000", 2024-09-16T00:09:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8600000000000001e-05 2024-09-16T00:09:16.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487055+0000", 2024-09-16T00:09:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873858 2024-09-16T00:09:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487098+0000", 2024-09-16T00:09:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2345000000000002e-05 2024-09-16T00:09:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.023 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:132 3.13 e57/20 3:c93e5698:::1000000027d.00000000:head v 57'7, mlcod=57'5)", 2024-09-16T00:09:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.557897+0000", 2024-09-16T00:09:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.432130609000001, 2024-09-16T00:09:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029674789999999999, 2024-09-16T00:09:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.024 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557897+0000", 2024-09-16T00:09:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557897+0000", 2024-09-16T00:09:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557898+0000", 2024-09-16T00:09:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1800000000000005e-07 2024-09-16T00:09:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557906+0000", 2024-09-16T00:09:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8390000000000007e-06 2024-09-16T00:09:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557919+0000", 2024-09-16T00:09:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2914999999999999e-05 2024-09-16T00:09:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557931+0000", 2024-09-16T00:09:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2734999999999999e-05 2024-09-16T00:09:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557982+0000", 2024-09-16T00:09:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0293000000000001e-05 2024-09-16T00:09:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557997+0000", 2024-09-16T00:09:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5160999999999999e-05 2024-09-16T00:09:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.560830+0000", 2024-09-16T00:09:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002832776 2024-09-16T00:09:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.560865+0000", 2024-09-16T00:09:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4941999999999999e-05 2024-09-16T00:09:16.029 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:243 3.24 e57/20 3:25048551:::100000002f0.00000000:head v 57'4, mlcod=57'2)", 2024-09-16T00:09:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.799371+0000", 2024-09-16T00:09:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.190656958999998, 2024-09-16T00:09:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037781889999999999, 2024-09-16T00:09:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.030 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799371+0000", 2024-09-16T00:09:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799371+0000", 2024-09-16T00:09:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799372+0000", 2024-09-16T00:09:16.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9100000000000003e-07 2024-09-16T00:09:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799379+0000", 2024-09-16T00:09:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.99e-06 2024-09-16T00:09:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799387+0000", 2024-09-16T00:09:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5790000000000004e-06 2024-09-16T00:09:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799392+0000", 2024-09-16T00:09:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5210000000000004e-06 2024-09-16T00:09:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799434+0000", 2024-09-16T00:09:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2341e-05 2024-09-16T00:09:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799447+0000", 2024-09-16T00:09:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3379e-05 2024-09-16T00:09:16.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.803117+0000", 2024-09-16T00:09:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036697169999999999 2024-09-16T00:09:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.803149+0000", 2024-09-16T00:09:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1971000000000003e-05 2024-09-16T00:09:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:319 3.1a 3:5a6e0114:::10000000350.00000000:head [write 0~3550] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.023452+0000", 2024-09-16T00:09:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.966576057000001, 2024-09-16T00:09:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597186, 2024-09-16T00:09:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 319 2024-09-16T00:09:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.035 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023452+0000", 2024-09-16T00:09:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023452+0000", 2024-09-16T00:09:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023452+0000", 2024-09-16T00:09:16.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5199999999999996e-07 2024-09-16T00:09:16.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023457+0000", 2024-09-16T00:09:16.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8289999999999998e-06 2024-09-16T00:09:16.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023467+0000", 2024-09-16T00:09:16.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8190000000000003e-06 2024-09-16T00:09:16.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023472+0000", 2024-09-16T00:09:16.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0810000000000001e-06 2024-09-16T00:09:16.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023524+0000", 2024-09-16T00:09:16.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2017e-05 2024-09-16T00:09:16.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023649+0000", 2024-09-16T00:09:16.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491999999999999 2024-09-16T00:09:16.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:16.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023772+0000", 2024-09-16T00:09:16.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012327599999999999 2024-09-16T00:09:16.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:16.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.024507+0000", 2024-09-16T00:09:16.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073499100000000005 2024-09-16T00:09:16.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:16.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.026948+0000", 2024-09-16T00:09:16.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002441033 2024-09-16T00:09:16.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.026997+0000", 2024-09-16T00:09:16.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8778e-05 2024-09-16T00:09:16.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.027049+0000", 2024-09-16T00:09:16.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690000000000001e-05 2024-09-16T00:09:16.040 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.041 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.041 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.041 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:323 3.3 3:c319d237:::10000000354.00000000:head [write 0~1622] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:16.041 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.032376+0000", 2024-09-16T00:09:16.041 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.957652033999999, 2024-09-16T00:09:16.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039255940000000001, 2024-09-16T00:09:16.041 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.041 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.041 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:16.041 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:16.042 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:16.042 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 323 2024-09-16T00:09:16.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.042 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032376+0000", 2024-09-16T00:09:16.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032376+0000", 2024-09-16T00:09:16.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032377+0000", 2024-09-16T00:09:16.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.42e-06 2024-09-16T00:09:16.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032383+0000", 2024-09-16T00:09:16.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9900000000000002e-06 2024-09-16T00:09:16.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032394+0000", 2024-09-16T00:09:16.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1216999999999999e-05 2024-09-16T00:09:16.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032400+0000", 2024-09-16T00:09:16.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8039999999999998e-06 2024-09-16T00:09:16.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032451+0000", 2024-09-16T00:09:16.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0612999999999998e-05 2024-09-16T00:09:16.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032536+0000", 2024-09-16T00:09:16.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4956000000000004e-05 2024-09-16T00:09:16.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:16.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032614+0000", 2024-09-16T00:09:16.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8716999999999999e-05 2024-09-16T00:09:16.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:16.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.033109+0000", 2024-09-16T00:09:16.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049412399999999997 2024-09-16T00:09:16.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:16.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.036222+0000", 2024-09-16T00:09:16.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031137949999999999 2024-09-16T00:09:16.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.036264+0000", 2024-09-16T00:09:16.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1572000000000001e-05 2024-09-16T00:09:16.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.036301+0000", 2024-09-16T00:09:16.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7385999999999997e-05 2024-09-16T00:09:16.047 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.047 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.047 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.047 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:324 3.29 e58/20 3:960c7fc3:::10000000355.00000000:head v 58'7, mlcod=57'5)", 2024-09-16T00:09:16.047 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.035116+0000", 2024-09-16T00:09:16.047 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.954912261, 2024-09-16T00:09:16.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029776550000000001, 2024-09-16T00:09:16.047 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.047 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.047 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.048 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035116+0000", 2024-09-16T00:09:16.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035116+0000", 2024-09-16T00:09:16.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035116+0000", 2024-09-16T00:09:16.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.91e-07 2024-09-16T00:09:16.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035119+0000", 2024-09-16T00:09:16.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.755e-06 2024-09-16T00:09:16.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035130+0000", 2024-09-16T00:09:16.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169999999999999e-05 2024-09-16T00:09:16.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035135+0000", 2024-09-16T00:09:16.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0019999999999997e-06 2024-09-16T00:09:16.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035175+0000", 2024-09-16T00:09:16.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0027999999999997e-05 2024-09-16T00:09:16.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035188+0000", 2024-09-16T00:09:16.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3386e-05 2024-09-16T00:09:16.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.038054+0000", 2024-09-16T00:09:16.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002866324 2024-09-16T00:09:16.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.038093+0000", 2024-09-16T00:09:16.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8798999999999999e-05 2024-09-16T00:09:16.052 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.052 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.052 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.052 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:392 3.f e58/20 3:f03826fe:::100000003a5.00000000:head v 58'8, mlcod=57'6)", 2024-09-16T00:09:16.052 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.307961+0000", 2024-09-16T00:09:16.052 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.682066547000002, 2024-09-16T00:09:16.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003252202, 2024-09-16T00:09:16.052 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.052 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.052 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.052 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.307961+0000", 2024-09-16T00:09:16.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.307961+0000", 2024-09-16T00:09:16.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.307962+0000", 2024-09-16T00:09:16.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5000000000000001e-07 2024-09-16T00:09:16.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.307993+0000", 2024-09-16T00:09:16.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0414000000000001e-05 2024-09-16T00:09:16.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.308003+0000", 2024-09-16T00:09:16.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0618999999999999e-05 2024-09-16T00:09:16.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.308010+0000", 2024-09-16T00:09:16.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5200000000000003e-06 2024-09-16T00:09:16.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.308062+0000", 2024-09-16T00:09:16.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.185e-05 2024-09-16T00:09:16.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.308083+0000", 2024-09-16T00:09:16.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1489e-05 2024-09-16T00:09:16.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.311178+0000", 2024-09-16T00:09:16.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030948630000000002 2024-09-16T00:09:16.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.311213+0000", 2024-09-16T00:09:16.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5497000000000003e-05 2024-09-16T00:09:16.056 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.057 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.057 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.057 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:411 3.29 e58/20 3:96aa9846:::100000003bf.00000000:head v 58'9, mlcod=58'7)", 2024-09-16T00:09:16.057 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.363807+0000", 2024-09-16T00:09:16.057 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.626221002000001, 2024-09-16T00:09:16.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032126009999999998, 2024-09-16T00:09:16.057 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.057 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.057 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.057 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363807+0000", 2024-09-16T00:09:16.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363807+0000", 2024-09-16T00:09:16.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363807+0000", 2024-09-16T00:09:16.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4899999999999995e-07 2024-09-16T00:09:16.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363812+0000", 2024-09-16T00:09:16.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4769999999999997e-06 2024-09-16T00:09:16.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363822+0000", 2024-09-16T00:09:16.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7950000000000002e-06 2024-09-16T00:09:16.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363827+0000", 2024-09-16T00:09:16.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8049999999999997e-06 2024-09-16T00:09:16.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363867+0000", 2024-09-16T00:09:16.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0694e-05 2024-09-16T00:09:16.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363880+0000", 2024-09-16T00:09:16.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3022e-05 2024-09-16T00:09:16.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.366979+0000", 2024-09-16T00:09:16.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003099161 2024-09-16T00:09:16.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.367019+0000", 2024-09-16T00:09:16.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9997999999999999e-05 2024-09-16T00:09:16.061 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.061 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.062 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.062 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:500 3.13 e58/20 3:c8c66365:::1000000042a.00000000:head v 58'14, mlcod=58'12)", 2024-09-16T00:09:16.062 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.760251+0000", 2024-09-16T00:09:16.062 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.229777201000001, 2024-09-16T00:09:16.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002913793, 2024-09-16T00:09:16.062 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.062 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.062 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.062 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760251+0000", 2024-09-16T00:09:16.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760251+0000", 2024-09-16T00:09:16.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760251+0000", 2024-09-16T00:09:16.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7400000000000003e-07 2024-09-16T00:09:16.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760257+0000", 2024-09-16T00:09:16.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0000000000000002e-06 2024-09-16T00:09:16.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760268+0000", 2024-09-16T00:09:16.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0397999999999999e-05 2024-09-16T00:09:16.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760273+0000", 2024-09-16T00:09:16.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2889999999999997e-06 2024-09-16T00:09:16.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760322+0000", 2024-09-16T00:09:16.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9193999999999997e-05 2024-09-16T00:09:16.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760343+0000", 2024-09-16T00:09:16.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1228000000000001e-05 2024-09-16T00:09:16.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.763123+0000", 2024-09-16T00:09:16.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027797189999999999 2024-09-16T00:09:16.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.763164+0000", 2024-09-16T00:09:16.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1291000000000002e-05 2024-09-16T00:09:16.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.067 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.067 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:811 3.1c e59/20 3:3b54038d:::100000005a1.00000000:head v 59'10, mlcod=59'8)", 2024-09-16T00:09:16.067 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:58.585639+0000", 2024-09-16T00:09:16.067 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.404388470000001, 2024-09-16T00:09:16.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002996163, 2024-09-16T00:09:16.068 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.068 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.068 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.068 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585639+0000", 2024-09-16T00:09:16.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585639+0000", 2024-09-16T00:09:16.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585640+0000", 2024-09-16T00:09:16.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1800000000000005e-07 2024-09-16T00:09:16.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585657+0000", 2024-09-16T00:09:16.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7249e-05 2024-09-16T00:09:16.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585669+0000", 2024-09-16T00:09:16.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1212e-05 2024-09-16T00:09:16.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585675+0000", 2024-09-16T00:09:16.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6200000000000001e-06 2024-09-16T00:09:16.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585723+0000", 2024-09-16T00:09:16.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7991000000000002e-05 2024-09-16T00:09:16.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585738+0000", 2024-09-16T00:09:16.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4578e-05 2024-09-16T00:09:16.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588596+0000", 2024-09-16T00:09:16.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028585780000000001 2024-09-16T00:09:16.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588635+0000", 2024-09-16T00:09:16.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9116999999999998e-05 2024-09-16T00:09:16.072 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.072 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.072 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.072 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:1618 3.1c e61/20 3:3b08c232:::1000000094e.00000000:head v 61'20, mlcod=61'18)", 2024-09-16T00:09:16.072 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.253213+0000", 2024-09-16T00:09:16.072 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.736814964000001, 2024-09-16T00:09:16.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025388519999999999, 2024-09-16T00:09:16.072 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.072 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.073 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.073 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253213+0000", 2024-09-16T00:09:16.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253213+0000", 2024-09-16T00:09:16.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253214+0000", 2024-09-16T00:09:16.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4600000000000004e-07 2024-09-16T00:09:16.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253231+0000", 2024-09-16T00:09:16.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6982000000000001e-05 2024-09-16T00:09:16.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253243+0000", 2024-09-16T00:09:16.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2412000000000001e-05 2024-09-16T00:09:16.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253250+0000", 2024-09-16T00:09:16.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.922e-06 2024-09-16T00:09:16.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253293+0000", 2024-09-16T00:09:16.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2823e-05 2024-09-16T00:09:16.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253309+0000", 2024-09-16T00:09:16.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6049000000000002e-05 2024-09-16T00:09:16.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255709+0000", 2024-09-16T00:09:16.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002400639 2024-09-16T00:09:16.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255752+0000", 2024-09-16T00:09:16.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2279e-05 2024-09-16T00:09:16.077 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.077 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.077 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.077 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:1691 3.a e61/46 3:518183a8:::10000000999.00000000:head v 61'30, mlcod=61'28)", 2024-09-16T00:09:16.077 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.411769+0000", 2024-09-16T00:09:16.078 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.578258858, 2024-09-16T00:09:16.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027805550000000001, 2024-09-16T00:09:16.078 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.078 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.078 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.078 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411769+0000", 2024-09-16T00:09:16.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411769+0000", 2024-09-16T00:09:16.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411770+0000", 2024-09-16T00:09:16.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3e-07 2024-09-16T00:09:16.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411787+0000", 2024-09-16T00:09:16.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7632999999999999e-05 2024-09-16T00:09:16.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411798+0000", 2024-09-16T00:09:16.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.094e-05 2024-09-16T00:09:16.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411807+0000", 2024-09-16T00:09:16.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3930000000000008e-06 2024-09-16T00:09:16.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411860+0000", 2024-09-16T00:09:16.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3477999999999999e-05 2024-09-16T00:09:16.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411879+0000", 2024-09-16T00:09:16.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9148999999999999e-05 2024-09-16T00:09:16.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.414504+0000", 2024-09-16T00:09:16.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026246759999999998 2024-09-16T00:09:16.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.414549+0000", 2024-09-16T00:09:16.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5556000000000002e-05 2024-09-16T00:09:16.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.082 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.082 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.083 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:1889 3.f e61/20 3:f2db18a4:::10000000a6a.00000000:head v 61'27, mlcod=61'25)", 2024-09-16T00:09:16.083 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.171388+0000", 2024-09-16T00:09:16.083 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.818639691, 2024-09-16T00:09:16.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00302002, 2024-09-16T00:09:16.083 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.083 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.083 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.083 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171388+0000", 2024-09-16T00:09:16.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171388+0000", 2024-09-16T00:09:16.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171389+0000", 2024-09-16T00:09:16.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0470000000000001e-06 2024-09-16T00:09:16.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171399+0000", 2024-09-16T00:09:16.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0088000000000001e-05 2024-09-16T00:09:16.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171415+0000", 2024-09-16T00:09:16.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5526e-05 2024-09-16T00:09:16.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171422+0000", 2024-09-16T00:09:16.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.764e-06 2024-09-16T00:09:16.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171488+0000", 2024-09-16T00:09:16.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6308000000000003e-05 2024-09-16T00:09:16.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171511+0000", 2024-09-16T00:09:16.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2891000000000002e-05 2024-09-16T00:09:16.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.174359+0000", 2024-09-16T00:09:16.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028479669999999999 2024-09-16T00:09:16.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.174408+0000", 2024-09-16T00:09:16.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9428999999999997e-05 2024-09-16T00:09:16.087 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.087 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.087 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.087 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:161 2.b e64/19 2:d5c7a900:::200.00000003:head v 64'5, mlcod=61'3)", 2024-09-16T00:09:16.087 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:07.541046+0000", 2024-09-16T00:09:16.087 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.4489813730000005, 2024-09-16T00:09:16.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010081302, 2024-09-16T00:09:16.088 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.088 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.088 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.088 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.541046+0000", 2024-09-16T00:09:16.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.541046+0000", 2024-09-16T00:09:16.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.541048+0000", 2024-09-16T00:09:16.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5859999999999999e-06 2024-09-16T00:09:16.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543134+0000", 2024-09-16T00:09:16.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020863589999999999 2024-09-16T00:09:16.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543155+0000", 2024-09-16T00:09:16.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0344999999999999e-05 2024-09-16T00:09:16.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543167+0000", 2024-09-16T00:09:16.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2071e-05 2024-09-16T00:09:16.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543242+0000", 2024-09-16T00:09:16.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5108e-05 2024-09-16T00:09:16.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543268+0000", 2024-09-16T00:09:16.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6471000000000001e-05 2024-09-16T00:09:16.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551087+0000", 2024-09-16T00:09:16.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0078188670000000002 2024-09-16T00:09:16.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551128+0000", 2024-09-16T00:09:16.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0494999999999998e-05 2024-09-16T00:09:16.092 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.092 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.092 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.092 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:224 2.1a e66/46 2:5e41ab69:::200.00000005:head v 66'4, mlcod=63'2)", 2024-09-16T00:09:16.092 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:12.643213+0000", 2024-09-16T00:09:16.092 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.346814577, 2024-09-16T00:09:16.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004469298, 2024-09-16T00:09:16.092 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.092 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.092 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.093 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.643213+0000", 2024-09-16T00:09:16.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.643213+0000", 2024-09-16T00:09:16.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.643215+0000", 2024-09-16T00:09:16.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.418e-06 2024-09-16T00:09:16.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.644677+0000", 2024-09-16T00:09:16.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001462124 2024-09-16T00:09:16.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.644696+0000", 2024-09-16T00:09:16.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9726000000000001e-05 2024-09-16T00:09:16.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.644707+0000", 2024-09-16T00:09:16.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0533000000000001e-05 2024-09-16T00:09:16.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.644765+0000", 2024-09-16T00:09:16.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8199999999999998e-05 2024-09-16T00:09:16.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.644784+0000", 2024-09-16T00:09:16.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9162999999999999e-05 2024-09-16T00:09:16.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.647638+0000", 2024-09-16T00:09:16.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028538769999999999 2024-09-16T00:09:16.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.647683+0000", 2024-09-16T00:09:16.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4257e-05 2024-09-16T00:09:16.096 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.096 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.096 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.096 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.097 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.097 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:16.099 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:09:16.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.075+0000 7f51bcff9640 1 -- 172.21.15.139:0/3385353640 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v69) ==== 113+0+0 (secure 0 0 0) 0x7f51a807d9a0 con 0x7f51c00c8c50 2024-09-16T00:09:16.100 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T00:09:16.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.078+0000 7f51c59f8640 1 -- 172.21.15.139:0/3385353640 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f51a006c2b0 msgr2=0x7f51a006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:16.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.078+0000 7f51c59f8640 1 --2- 172.21.15.139:0/3385353640 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f51a006c2b0 0x7f51a006e770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009560 tx=0x7f51b0009190 comp rx=0 tx=0).stop 2024-09-16T00:09:16.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.078+0000 7f51c59f8640 1 -- 172.21.15.139:0/3385353640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51c00c8c50 msgr2=0x7f51c01f1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:16.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.078+0000 7f51c59f8640 1 --2- 172.21.15.139:0/3385353640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51c00c8c50 0x7f51c01f1670 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f51a8035040 tx=0x7f51a80046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:16.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.078+0000 7f51c59f8640 1 -- 172.21.15.139:0/3385353640 shutdown_connections 2024-09-16T00:09:16.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.078+0000 7f51c59f8640 1 --2- 172.21.15.139:0/3385353640 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f51a006c2b0 0x7f51a006e770 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:16.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.078+0000 7f51c59f8640 1 --2- 172.21.15.139:0/3385353640 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f51c0152b20 0x7f51c00cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:16.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.078+0000 7f51c59f8640 1 --2- 172.21.15.139:0/3385353640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51c00c8c50 0x7f51c01f1670 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:16.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.078+0000 7f51c59f8640 1 --2- 172.21.15.139:0/3385353640 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f51c0070440 0x7f51c01e40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:16.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.078+0000 7f51c59f8640 1 -- 172.21.15.139:0/3385353640 >> 172.21.15.139:0/3385353640 conn(0x7f51c006bd00 msgr2=0x7f51c0157380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:16.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.078+0000 7f51c59f8640 1 -- 172.21.15.139:0/3385353640 shutdown_connections 2024-09-16T00:09:16.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.078+0000 7f51c59f8640 1 -- 172.21.15.139:0/3385353640 wait complete. 2024-09-16T00:09:16.102 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T00:09:16.102 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:16.097+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:16.199 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:16.197+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:16.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.246+0000 7f281de77640 1 Processor -- start 2024-09-16T00:09:16.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.246+0000 7f281de77640 1 -- start start 2024-09-16T00:09:16.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.247+0000 7f281de77640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28180c6a20 0x7f281806f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:16.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.247+0000 7f281de77640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f28180c6e00 0x7f281815bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:16.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.247+0000 7f281de77640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f281806ea70 0x7f28180c9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:16.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.247+0000 7f281de77640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f281806f520 con 0x7f281806ea70 2024-09-16T00:09:16.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.247+0000 7f281de77640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f281806f210 con 0x7f28180c6e00 2024-09-16T00:09:16.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.247+0000 7f281de77640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f281806ef20 con 0x7f28180c6a20 2024-09-16T00:09:16.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.247+0000 7f2817fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f281806ea70 0x7f28180c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:16.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.247+0000 7f2817fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f281806ea70 0x7f28180c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58832/0 (socket says 172.21.15.139:58832) 2024-09-16T00:09:16.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.247+0000 7f2817fff640 1 -- 172.21.15.139:0/3189023188 learned_addr learned my addr 172.21.15.139:0/3189023188 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:16.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.247+0000 7f2816ffd640 1 --2- 172.21.15.139:0/3189023188 >> v2:172.21.15.164:3300/0 conn(0x7f28180c6e00 0x7f281815bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:16.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.247+0000 7f28177fe640 1 --2- 172.21.15.139:0/3189023188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28180c6a20 0x7f281806f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:16.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.247+0000 7f2817fff640 1 -- 172.21.15.139:0/3189023188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28180c6a20 msgr2=0x7f281806f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:16.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.247+0000 7f2817fff640 1 --2- 172.21.15.139:0/3189023188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28180c6a20 0x7f281806f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:16.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.247+0000 7f2817fff640 1 -- 172.21.15.139:0/3189023188 >> v2:172.21.15.164:3300/0 conn(0x7f28180c6e00 msgr2=0x7f281815bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:16.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.247+0000 7f2817fff640 1 --2- 172.21.15.139:0/3189023188 >> v2:172.21.15.164:3300/0 conn(0x7f28180c6e00 0x7f281815bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:16.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.247+0000 7f2817fff640 1 -- 172.21.15.139:0/3189023188 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28180ca080 con 0x7f281806ea70 2024-09-16T00:09:16.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.248+0000 7f2817fff640 1 --2- 172.21.15.139:0/3189023188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281806ea70 0x7f28180c9530 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2818076570 tx=0x7f281002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c1926a81e22176 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:16.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.248+0000 7f28167fc640 1 -- 172.21.15.139:0/3189023188 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810036070 con 0x7f281806ea70 2024-09-16T00:09:16.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.248+0000 7f28167fc640 1 -- 172.21.15.139:0/3189023188 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2810031600 con 0x7f281806ea70 2024-09-16T00:09:16.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.248+0000 7f28167fc640 1 -- 172.21.15.139:0/3189023188 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28100319d0 con 0x7f281806ea70 2024-09-16T00:09:16.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.249+0000 7f281de77640 1 -- 172.21.15.139:0/3189023188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281806ea70 msgr2=0x7f28180c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:16.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.249+0000 7f281de77640 1 --2- 172.21.15.139:0/3189023188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281806ea70 0x7f28180c9530 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2818076570 tx=0x7f281002a730 comp rx=0 tx=0).stop 2024-09-16T00:09:16.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.249+0000 7f281de77640 1 -- 172.21.15.139:0/3189023188 shutdown_connections 2024-09-16T00:09:16.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.249+0000 7f281de77640 1 --2- 172.21.15.139:0/3189023188 >> v2:172.21.15.164:3300/0 conn(0x7f28180c6e00 0x7f281815bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:16.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.249+0000 7f281de77640 1 --2- 172.21.15.139:0/3189023188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281806ea70 0x7f28180c9530 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:16.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.249+0000 7f281de77640 1 --2- 172.21.15.139:0/3189023188 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28180c6a20 0x7f281806f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:16.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.249+0000 7f281de77640 1 -- 172.21.15.139:0/3189023188 >> 172.21.15.139:0/3189023188 conn(0x7f281806cc30 msgr2=0x7f281806d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:16.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.249+0000 7f281de77640 1 -- 172.21.15.139:0/3189023188 shutdown_connections 2024-09-16T00:09:16.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.249+0000 7f281de77640 1 -- 172.21.15.139:0/3189023188 wait complete. 2024-09-16T00:09:16.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.250+0000 7f281de77640 1 Processor -- start 2024-09-16T00:09:16.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.250+0000 7f281de77640 1 -- start start 2024-09-16T00:09:16.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f281de77640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281806ea70 0x7f28180cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:16.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f281de77640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28180c6a20 0x7f28181e33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:16.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f281de77640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28180c6e00 0x7f28181e38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:16.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f281de77640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f28180ca600 con 0x7f281806ea70 2024-09-16T00:09:16.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f281de77640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f28180ca320 con 0x7f28180c6a20 2024-09-16T00:09:16.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f281de77640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f28180ca8e0 con 0x7f28180c6e00 2024-09-16T00:09:16.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f28177fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281806ea70 0x7f28180cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:16.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f28177fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281806ea70 0x7f28180cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58834/0 (socket says 172.21.15.139:58834) 2024-09-16T00:09:16.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f28177fe640 1 -- 172.21.15.139:0/1932990406 learned_addr learned my addr 172.21.15.139:0/1932990406 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:16.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f2816ffd640 1 --2- 172.21.15.139:0/1932990406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28180c6a20 0x7f28181e33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:16.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f2817fff640 1 --2- 172.21.15.139:0/1932990406 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28180c6e00 0x7f28181e38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:16.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f28177fe640 1 -- 172.21.15.139:0/1932990406 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28180c6e00 msgr2=0x7f28181e38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:16.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f28177fe640 1 --2- 172.21.15.139:0/1932990406 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28180c6e00 0x7f28181e38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:16.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f28177fe640 1 -- 172.21.15.139:0/1932990406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28180c6a20 msgr2=0x7f28181e33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:16.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f28177fe640 1 --2- 172.21.15.139:0/1932990406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28180c6a20 0x7f28181e33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:16.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f28177fe640 1 -- 172.21.15.139:0/1932990406 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28181e1e80 con 0x7f281806ea70 2024-09-16T00:09:16.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.251+0000 7f28177fe640 1 --2- 172.21.15.139:0/1932990406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281806ea70 0x7f28180cd5f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f280800e9b0 tx=0x7f280800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:16.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.252+0000 7f27fffff640 1 -- 172.21.15.139:0/1932990406 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2808017070 con 0x7f281806ea70 2024-09-16T00:09:16.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.252+0000 7f27fffff640 1 -- 172.21.15.139:0/1932990406 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f280800c9a0 con 0x7f281806ea70 2024-09-16T00:09:16.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.252+0000 7f281de77640 1 -- 172.21.15.139:0/1932990406 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28181e2140 con 0x7f281806ea70 2024-09-16T00:09:16.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.252+0000 7f27fffff640 1 -- 172.21.15.139:0/1932990406 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280800cd90 con 0x7f281806ea70 2024-09-16T00:09:16.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.252+0000 7f281de77640 1 -- 172.21.15.139:0/1932990406 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28181e2700 con 0x7f281806ea70 2024-09-16T00:09:16.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.253+0000 7f281de77640 1 -- 172.21.15.139:0/1932990406 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281806f9b0 con 0x7f281806ea70 2024-09-16T00:09:16.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.253+0000 7f27fffff640 1 -- 172.21.15.139:0/1932990406 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f280801a070 con 0x7f281806ea70 2024-09-16T00:09:16.260 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:16.260 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:16.260 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:16.260 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:16.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.254+0000 7f27fffff640 1 --2- 172.21.15.139:0/1932990406 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f280406c2b0 0x7f280406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:16.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.254+0000 7f27fffff640 1 -- 172.21.15.139:0/1932990406 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5944+0+0 (secure 0 0 0) 0x7f280808b840 con 0x7f281806ea70 2024-09-16T00:09:16.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.254+0000 7f2816ffd640 1 --2- 172.21.15.139:0/1932990406 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f280406c2b0 0x7f280406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:16.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.254+0000 7f2816ffd640 1 --2- 172.21.15.139:0/1932990406 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f280406c2b0 0x7f280406e770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f281814ddc0 tx=0x7f280c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:16.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.258+0000 7f27fffff640 1 -- 172.21.15.139:0/1932990406 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2808059870 con 0x7f281806ea70 2024-09-16T00:09:16.264 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:09:16.300 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:16.299+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:16.401 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:16.400+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:16.420 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:16.420 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:16.421 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:16.421 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:16.421 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:16.430 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:09:16.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:16.471+0000 7f281de77640 1 -- 172.21.15.139:0/1932990406 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) -- 0x7f281814ddf0 con 0x7f281806ea70 2024-09-16T00:09:16.502 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:16.501+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:16.545 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:16.545 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:16.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:16.545 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:16.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.545 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:14 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:16.545 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:16.546 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.926113616999999, 2024-09-16T00:09:16.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029190470000000001, 2024-09-16T00:09:16.546 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.546 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.546 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.546 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:16.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:16.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615982+0000", 2024-09-16T00:09:16.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7439999999999999e-06 2024-09-16T00:09:16.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:09:16.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3743999999999999e-05 2024-09-16T00:09:16.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:16.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2255e-05 2024-09-16T00:09:16.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616029+0000", 2024-09-16T00:09:16.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1175999999999999e-05 2024-09-16T00:09:16.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616098+0000", 2024-09-16T00:09:16.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9153999999999996e-05 2024-09-16T00:09:16.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616132+0000", 2024-09-16T00:09:16.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3924000000000002e-05 2024-09-16T00:09:16.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:09:16.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027226189999999999 2024-09-16T00:09:16.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618899+0000", 2024-09-16T00:09:16.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4431000000000002e-05 2024-09-16T00:09:16.554 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.554 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.554 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.555 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:30 2.15 e22/19 2:aa448500:::500.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:09:16.555 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:09:16.556 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.923829311999995, 2024-09-16T00:09:16.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058031250000000001, 2024-09-16T00:09:16.556 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.556 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.556 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.557 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:09:16.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:09:16.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618265+0000", 2024-09-16T00:09:16.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9200000000000001e-07 2024-09-16T00:09:16.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618271+0000", 2024-09-16T00:09:16.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9580000000000004e-06 2024-09-16T00:09:16.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618279+0000", 2024-09-16T00:09:16.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2239999999999995e-06 2024-09-16T00:09:16.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618285+0000", 2024-09-16T00:09:16.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-06 2024-09-16T00:09:16.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618567+0000", 2024-09-16T00:09:16.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028204400000000001 2024-09-16T00:09:16.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:09:16.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9678999999999999e-05 2024-09-16T00:09:16.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624024+0000", 2024-09-16T00:09:16.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054376720000000002 2024-09-16T00:09:16.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624068+0000", 2024-09-16T00:09:16.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:09:16.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.565 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.566 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:20 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:16.566 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:09:16.566 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.923472947999997, 2024-09-16T00:09:16.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028808739999999998, 2024-09-16T00:09:16.566 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.567 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.567 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.567 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:09:16.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:09:16.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618622+0000", 2024-09-16T00:09:16.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1260000000000001e-06 2024-09-16T00:09:16.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618631+0000", 2024-09-16T00:09:16.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4700000000000008e-06 2024-09-16T00:09:16.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618645+0000", 2024-09-16T00:09:16.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3984e-05 2024-09-16T00:09:16.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:09:16.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2101e-05 2024-09-16T00:09:16.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618720+0000", 2024-09-16T00:09:16.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2761000000000004e-05 2024-09-16T00:09:16.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618740+0000", 2024-09-16T00:09:16.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9409e-05 2024-09-16T00:09:16.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621436+0000", 2024-09-16T00:09:16.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002696211 2024-09-16T00:09:16.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621502+0000", 2024-09-16T00:09:16.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5811999999999999e-05 2024-09-16T00:09:16.575 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.575 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.575 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.575 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:21 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:16.576 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:09:16.576 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 97.922649660000005, 2024-09-16T00:09:16.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046318540000000004, 2024-09-16T00:09:16.576 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.576 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.576 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.576 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:09:16.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:09:16.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619445+0000", 2024-09-16T00:09:16.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.083e-06 2024-09-16T00:09:16.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619455+0000", 2024-09-16T00:09:16.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5920000000000007e-06 2024-09-16T00:09:16.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619467+0000", 2024-09-16T00:09:16.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2711000000000001e-05 2024-09-16T00:09:16.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619476+0000", 2024-09-16T00:09:16.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0139999999999998e-06 2024-09-16T00:09:16.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619529+0000", 2024-09-16T00:09:16.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3078999999999998e-05 2024-09-16T00:09:16.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619548+0000", 2024-09-16T00:09:16.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8952000000000001e-05 2024-09-16T00:09:16.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624037+0000", 2024-09-16T00:09:16.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044895730000000002 2024-09-16T00:09:16.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624076+0000", 2024-09-16T00:09:16.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8850000000000002e-05 2024-09-16T00:09:16.584 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.584 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.584 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.585 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:93 3.5 3:a3a50212:::10000000254.00000000:head [write 0~6832] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:16.585 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.467788+0000", 2024-09-16T00:09:16.585 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.07430591, 2024-09-16T00:09:16.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001903882, 2024-09-16T00:09:16.585 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.585 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.585 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.586 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:16.586 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:16.586 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:16.586 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93 2024-09-16T00:09:16.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.586 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467788+0000", 2024-09-16T00:09:16.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467788+0000", 2024-09-16T00:09:16.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467789+0000", 2024-09-16T00:09:16.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4399999999999999e-07 2024-09-16T00:09:16.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467798+0000", 2024-09-16T00:09:16.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0510000000000002e-06 2024-09-16T00:09:16.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467809+0000", 2024-09-16T00:09:16.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0859000000000001e-05 2024-09-16T00:09:16.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467814+0000", 2024-09-16T00:09:16.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0409999999999997e-06 2024-09-16T00:09:16.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467860+0000", 2024-09-16T00:09:16.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6604999999999998e-05 2024-09-16T00:09:16.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467935+0000", 2024-09-16T00:09:16.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4417000000000003e-05 2024-09-16T00:09:16.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:16.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.468012+0000", 2024-09-16T00:09:16.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6971000000000006e-05 2024-09-16T00:09:16.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:16.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.468821+0000", 2024-09-16T00:09:16.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00080984099999999995 2024-09-16T00:09:16.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:16.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.469636+0000", 2024-09-16T00:09:16.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00081428499999999999 2024-09-16T00:09:16.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.469668+0000", 2024-09-16T00:09:16.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2322e-05 2024-09-16T00:09:16.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.469692+0000", 2024-09-16T00:09:16.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3745999999999999e-05 2024-09-16T00:09:16.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.596 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.597 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:249 3.28 3:14c9b427:::100000002f7.00000000:head [write 0~5746] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:16.597 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.812683+0000", 2024-09-16T00:09:16.597 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.729410422000001, 2024-09-16T00:09:16.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019975150000000001, 2024-09-16T00:09:16.597 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.597 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.598 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:16.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:16.598 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:16.598 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 249 2024-09-16T00:09:16.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.599 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812683+0000", 2024-09-16T00:09:16.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812683+0000", 2024-09-16T00:09:16.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812684+0000", 2024-09-16T00:09:16.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1100000000000004e-07 2024-09-16T00:09:16.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812708+0000", 2024-09-16T00:09:16.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4074e-05 2024-09-16T00:09:16.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812736+0000", 2024-09-16T00:09:16.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7767e-05 2024-09-16T00:09:16.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812741+0000", 2024-09-16T00:09:16.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3319999999999999e-06 2024-09-16T00:09:16.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812790+0000", 2024-09-16T00:09:16.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8294000000000002e-05 2024-09-16T00:09:16.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812881+0000", 2024-09-16T00:09:16.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1085e-05 2024-09-16T00:09:16.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:16.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812960+0000", 2024-09-16T00:09:16.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8824000000000003e-05 2024-09-16T00:09:16.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:16.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.813562+0000", 2024-09-16T00:09:16.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00060270500000000004 2024-09-16T00:09:16.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:16.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.814630+0000", 2024-09-16T00:09:16.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010679489999999999 2024-09-16T00:09:16.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.814660+0000", 2024-09-16T00:09:16.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.936e-05 2024-09-16T00:09:16.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.814681+0000", 2024-09-16T00:09:16.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1214000000000001e-05 2024-09-16T00:09:16.609 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.610 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.610 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.611 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:311 3.2a 3:548c374b:::10000000347.00000000:head [write 0~4872] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:16.611 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.002861+0000", 2024-09-16T00:09:16.611 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.539233146000001, 2024-09-16T00:09:16.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001734665, 2024-09-16T00:09:16.611 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.611 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.611 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:16.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:16.612 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:16.612 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 311 2024-09-16T00:09:16.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.612 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002861+0000", 2024-09-16T00:09:16.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002861+0000", 2024-09-16T00:09:16.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002863+0000", 2024-09-16T00:09:16.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0540000000000002e-06 2024-09-16T00:09:16.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002877+0000", 2024-09-16T00:09:16.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4677e-05 2024-09-16T00:09:16.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002896+0000", 2024-09-16T00:09:16.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9038999999999998e-05 2024-09-16T00:09:16.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002905+0000", 2024-09-16T00:09:16.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0599999999999997e-06 2024-09-16T00:09:16.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002973+0000", 2024-09-16T00:09:16.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7203e-05 2024-09-16T00:09:16.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.003111+0000", 2024-09-16T00:09:16.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013858100000000001 2024-09-16T00:09:16.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:16.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.003269+0000", 2024-09-16T00:09:16.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015809 2024-09-16T00:09:16.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:16.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.004171+0000", 2024-09-16T00:09:16.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00090205600000000004 2024-09-16T00:09:16.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:16.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.004494+0000", 2024-09-16T00:09:16.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000322126 2024-09-16T00:09:16.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.004549+0000", 2024-09-16T00:09:16.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5049000000000001e-05 2024-09-16T00:09:16.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.004595+0000", 2024-09-16T00:09:16.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6730000000000002e-05 2024-09-16T00:09:16.622 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.622 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.622 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.623 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:333 3.c 3:329df26d:::1000000035f.00000000:head [write 0~657] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:16.623 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.059938+0000", 2024-09-16T00:09:16.623 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.482155447, 2024-09-16T00:09:16.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001777437, 2024-09-16T00:09:16.623 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.623 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.624 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:16.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:16.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:16.624 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 333 2024-09-16T00:09:16.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.625 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059938+0000", 2024-09-16T00:09:16.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059938+0000", 2024-09-16T00:09:16.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059940+0000", 2024-09-16T00:09:16.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1129999999999999e-06 2024-09-16T00:09:16.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059953+0000", 2024-09-16T00:09:16.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.278e-05 2024-09-16T00:09:16.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059980+0000", 2024-09-16T00:09:16.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6533000000000002e-05 2024-09-16T00:09:16.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059994+0000", 2024-09-16T00:09:16.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4049999999999999e-05 2024-09-16T00:09:16.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.060136+0000", 2024-09-16T00:09:16.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014184399999999999 2024-09-16T00:09:16.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.060396+0000", 2024-09-16T00:09:16.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026020600000000002 2024-09-16T00:09:16.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:16.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.060540+0000", 2024-09-16T00:09:16.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143838 2024-09-16T00:09:16.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:16.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.061336+0000", 2024-09-16T00:09:16.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00079669099999999998 2024-09-16T00:09:16.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:16.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.061621+0000", 2024-09-16T00:09:16.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028509599999999998 2024-09-16T00:09:16.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.061674+0000", 2024-09-16T00:09:16.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2685000000000001e-05 2024-09-16T00:09:16.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.061716+0000", 2024-09-16T00:09:16.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1600999999999997e-05 2024-09-16T00:09:16.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.634 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.634 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.635 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:490 3.d e58/51 3:b3311d72:::1000000041b.00000000:head v 58'8, mlcod=58'6)", 2024-09-16T00:09:16.635 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.728943+0000", 2024-09-16T00:09:16.635 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.813151147999999, 2024-09-16T00:09:16.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002073142, 2024-09-16T00:09:16.635 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.636 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.636 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.636 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728943+0000", 2024-09-16T00:09:16.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728943+0000", 2024-09-16T00:09:16.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728943+0000", 2024-09-16T00:09:16.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:09:16.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728949+0000", 2024-09-16T00:09:16.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T00:09:16.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728958+0000", 2024-09-16T00:09:16.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3049999999999998e-06 2024-09-16T00:09:16.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728964+0000", 2024-09-16T00:09:16.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9699999999999996e-06 2024-09-16T00:09:16.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.729018+0000", 2024-09-16T00:09:16.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4045000000000002e-05 2024-09-16T00:09:16.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.729039+0000", 2024-09-16T00:09:16.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1098999999999999e-05 2024-09-16T00:09:16.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.730988+0000", 2024-09-16T00:09:16.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019486919999999999 2024-09-16T00:09:16.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.731016+0000", 2024-09-16T00:09:16.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7971e-05 2024-09-16T00:09:16.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.643 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.643 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.643 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:681 3.17 e59/20 3:ea55746f:::1000000050b.00000000:head v 59'12, mlcod=59'10)", 2024-09-16T00:09:16.643 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:58.265463+0000", 2024-09-16T00:09:16.643 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.276630271999998, 2024-09-16T00:09:16.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030780130000000001, 2024-09-16T00:09:16.644 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.644 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.644 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.644 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265463+0000", 2024-09-16T00:09:16.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265463+0000", 2024-09-16T00:09:16.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265464+0000", 2024-09-16T00:09:16.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7999999999999995e-07 2024-09-16T00:09:16.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265471+0000", 2024-09-16T00:09:16.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5010000000000001e-06 2024-09-16T00:09:16.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265480+0000", 2024-09-16T00:09:16.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.302e-06 2024-09-16T00:09:16.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265485+0000", 2024-09-16T00:09:16.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:09:16.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265526+0000", 2024-09-16T00:09:16.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1229000000000001e-05 2024-09-16T00:09:16.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265539+0000", 2024-09-16T00:09:16.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3083e-05 2024-09-16T00:09:16.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.268498+0000", 2024-09-16T00:09:16.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029594320000000001 2024-09-16T00:09:16.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.268542+0000", 2024-09-16T00:09:16.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3207000000000002e-05 2024-09-16T00:09:16.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.651 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.651 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.652 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:811 3.1c 3:3b54038d:::100000005a1.00000000:head [write 0~2412] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:16.652 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:58.585366+0000", 2024-09-16T00:09:16.652 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.956728241, 2024-09-16T00:09:16.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003510216, 2024-09-16T00:09:16.652 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.652 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.652 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.653 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:16.653 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:16.653 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:16.653 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 811 2024-09-16T00:09:16.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.653 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585366+0000", 2024-09-16T00:09:16.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585366+0000", 2024-09-16T00:09:16.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585366+0000", 2024-09-16T00:09:16.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5300000000000003e-07 2024-09-16T00:09:16.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585371+0000", 2024-09-16T00:09:16.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2849999999999996e-06 2024-09-16T00:09:16.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585380+0000", 2024-09-16T00:09:16.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9219999999999992e-06 2024-09-16T00:09:16.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585390+0000", 2024-09-16T00:09:16.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2960000000000004e-06 2024-09-16T00:09:16.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585436+0000", 2024-09-16T00:09:16.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5874000000000002e-05 2024-09-16T00:09:16.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585510+0000", 2024-09-16T00:09:16.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4070000000000001e-05 2024-09-16T00:09:16.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:16.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585586+0000", 2024-09-16T00:09:16.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6044999999999994e-05 2024-09-16T00:09:16.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:16.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.586070+0000", 2024-09-16T00:09:16.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048407300000000001 2024-09-16T00:09:16.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:16.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588779+0000", 2024-09-16T00:09:16.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027090339999999999 2024-09-16T00:09:16.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588834+0000", 2024-09-16T00:09:16.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5513000000000002e-05 2024-09-16T00:09:16.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588876+0000", 2024-09-16T00:09:16.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1350999999999998e-05 2024-09-16T00:09:16.662 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.663 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.663 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:1617 3.36 e61/46 3:6fb28598:::1000000094d.00000000:head v 61'29, mlcod=61'27)", 2024-09-16T00:09:16.663 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.251467+0000", 2024-09-16T00:09:16.663 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.290626586, 2024-09-16T00:09:16.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040699999999999998, 2024-09-16T00:09:16.664 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.664 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.664 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.664 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251467+0000", 2024-09-16T00:09:16.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251467+0000", 2024-09-16T00:09:16.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251468+0000", 2024-09-16T00:09:16.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6400000000000001e-07 2024-09-16T00:09:16.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251478+0000", 2024-09-16T00:09:16.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0057e-05 2024-09-16T00:09:16.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251489+0000", 2024-09-16T00:09:16.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0992000000000001e-05 2024-09-16T00:09:16.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251494+0000", 2024-09-16T00:09:16.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8999999999999997e-06 2024-09-16T00:09:16.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251537+0000", 2024-09-16T00:09:16.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3513999999999997e-05 2024-09-16T00:09:16.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251550+0000", 2024-09-16T00:09:16.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3083e-05 2024-09-16T00:09:16.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255500+0000", 2024-09-16T00:09:16.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039494090000000001 2024-09-16T00:09:16.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255537+0000", 2024-09-16T00:09:16.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280999999999998e-05 2024-09-16T00:09:16.671 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.671 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.671 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.671 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:1618 3.1c 3:3b08c232:::1000000094e.00000000:head [write 0~2553] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T00:09:16.672 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.252928+0000", 2024-09-16T00:09:16.672 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.289165444, 2024-09-16T00:09:16.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030458360000000001, 2024-09-16T00:09:16.672 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.672 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.672 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.672 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:16.673 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:16.673 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:16.673 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1618 2024-09-16T00:09:16.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.673 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252928+0000", 2024-09-16T00:09:16.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252928+0000", 2024-09-16T00:09:16.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252929+0000", 2024-09-16T00:09:16.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.13e-06 2024-09-16T00:09:16.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252939+0000", 2024-09-16T00:09:16.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8239999999999995e-06 2024-09-16T00:09:16.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252950+0000", 2024-09-16T00:09:16.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251999999999999e-05 2024-09-16T00:09:16.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252962+0000", 2024-09-16T00:09:16.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2332e-05 2024-09-16T00:09:16.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253005+0000", 2024-09-16T00:09:16.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3380000000000001e-05 2024-09-16T00:09:16.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253084+0000", 2024-09-16T00:09:16.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9097000000000005e-05 2024-09-16T00:09:16.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:16.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253162+0000", 2024-09-16T00:09:16.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7890000000000001e-05 2024-09-16T00:09:16.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:16.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255878+0000", 2024-09-16T00:09:16.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027157589999999999 2024-09-16T00:09:16.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:16.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255892+0000", 2024-09-16T00:09:16.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4329e-05 2024-09-16T00:09:16.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255940+0000", 2024-09-16T00:09:16.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7979000000000003e-05 2024-09-16T00:09:16.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255974+0000", 2024-09-16T00:09:16.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3864e-05 2024-09-16T00:09:16.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.682 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.682 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:84 2.17 e61/19 2:ee44d6a7:::10000000261.03e00000:head v 61'1, mlcod=0'0)", 2024-09-16T00:09:16.683 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.077563+0000", 2024-09-16T00:09:16.683 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.464531060000001, 2024-09-16T00:09:16.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033709450000000002, 2024-09-16T00:09:16.683 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.683 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.683 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.683 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077563+0000", 2024-09-16T00:09:16.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077563+0000", 2024-09-16T00:09:16.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077563+0000", 2024-09-16T00:09:16.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.61e-07 2024-09-16T00:09:16.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077577+0000", 2024-09-16T00:09:16.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3108000000000001e-05 2024-09-16T00:09:16.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077587+0000", 2024-09-16T00:09:16.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0273e-05 2024-09-16T00:09:16.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077593+0000", 2024-09-16T00:09:16.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5209999999999999e-06 2024-09-16T00:09:16.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077641+0000", 2024-09-16T00:09:16.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7579e-05 2024-09-16T00:09:16.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077658+0000", 2024-09-16T00:09:16.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7297999999999999e-05 2024-09-16T00:09:16.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080906+0000", 2024-09-16T00:09:16.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032481760000000002 2024-09-16T00:09:16.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080934+0000", 2024-09-16T00:09:16.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7229e-05 2024-09-16T00:09:16.690 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.690 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.690 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.691 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:80 2.12 2:4a871970:::10000000261.03600000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:16.691 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.077908+0000", 2024-09-16T00:09:16.691 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.464186083, 2024-09-16T00:09:16.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030856009999999999, 2024-09-16T00:09:16.691 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.692 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.692 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.692 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077908+0000", 2024-09-16T00:09:16.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077908+0000", 2024-09-16T00:09:16.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077909+0000", 2024-09-16T00:09:16.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5719999999999999e-06 2024-09-16T00:09:16.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077920+0000", 2024-09-16T00:09:16.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0631e-05 2024-09-16T00:09:16.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077930+0000", 2024-09-16T00:09:16.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0180000000000001e-05 2024-09-16T00:09:16.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077937+0000", 2024-09-16T00:09:16.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:09:16.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077993+0000", 2024-09-16T00:09:16.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6174999999999997e-05 2024-09-16T00:09:16.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078106+0000", 2024-09-16T00:09:16.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113133 2024-09-16T00:09:16.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:16.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078218+0000", 2024-09-16T00:09:16.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011114 2024-09-16T00:09:16.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:16.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.079121+0000", 2024-09-16T00:09:16.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00090328499999999998 2024-09-16T00:09:16.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:16.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080928+0000", 2024-09-16T00:09:16.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018072310000000001 2024-09-16T00:09:16.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080967+0000", 2024-09-16T00:09:16.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8982e-05 2024-09-16T00:09:16.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080993+0000", 2024-09-16T00:09:16.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6157e-05 2024-09-16T00:09:16.701 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.701 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.702 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.702 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:1878 3.3c 3:3f4f0e87:::10000000a5f.00000000:head [write 0~2494] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T00:09:16.702 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.078761+0000", 2024-09-16T00:09:16.702 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.463333152000001, 2024-09-16T00:09:16.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025239199999999998, 2024-09-16T00:09:16.703 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.703 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.703 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.703 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:16.703 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:16.703 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:16.703 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1878 2024-09-16T00:09:16.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.704 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078761+0000", 2024-09-16T00:09:16.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078761+0000", 2024-09-16T00:09:16.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078762+0000", 2024-09-16T00:09:16.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7600000000000006e-07 2024-09-16T00:09:16.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078767+0000", 2024-09-16T00:09:16.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5929999999999996e-06 2024-09-16T00:09:16.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078781+0000", 2024-09-16T00:09:16.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3804e-05 2024-09-16T00:09:16.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078789+0000", 2024-09-16T00:09:16.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6960000000000005e-06 2024-09-16T00:09:16.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078838+0000", 2024-09-16T00:09:16.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9416000000000003e-05 2024-09-16T00:09:16.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078940+0000", 2024-09-16T00:09:16.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101431 2024-09-16T00:09:16.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:16.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.079032+0000", 2024-09-16T00:09:16.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2559999999999995e-05 2024-09-16T00:09:16.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:16.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.079823+0000", 2024-09-16T00:09:16.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00079055300000000005 2024-09-16T00:09:16.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:16.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081201+0000", 2024-09-16T00:09:16.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013780579999999999 2024-09-16T00:09:16.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081247+0000", 2024-09-16T00:09:16.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6213e-05 2024-09-16T00:09:16.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081285+0000", 2024-09-16T00:09:16.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7620000000000002e-05 2024-09-16T00:09:16.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.713 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.713 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.713 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:2173 3.30 3:0ff9340b:::10000000bb6.00000000:head [write 0~80] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T00:09:16.713 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:03.053795+0000", 2024-09-16T00:09:16.713 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.488299119000001, 2024-09-16T00:09:16.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020696619999999999, 2024-09-16T00:09:16.714 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.714 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.714 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:16.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:16.714 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:16.714 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 2173 2024-09-16T00:09:16.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.714 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053795+0000", 2024-09-16T00:09:16.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053795+0000", 2024-09-16T00:09:16.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053796+0000", 2024-09-16T00:09:16.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.257e-06 2024-09-16T00:09:16.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053802+0000", 2024-09-16T00:09:16.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3430000000000001e-06 2024-09-16T00:09:16.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053813+0000", 2024-09-16T00:09:16.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.101e-05 2024-09-16T00:09:16.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053819+0000", 2024-09-16T00:09:16.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0750000000000003e-06 2024-09-16T00:09:16.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053864+0000", 2024-09-16T00:09:16.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4258999999999998e-05 2024-09-16T00:09:16.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053954+0000", 2024-09-16T00:09:16.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0195e-05 2024-09-16T00:09:16.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:16.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.054034+0000", 2024-09-16T00:09:16.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0319000000000001e-05 2024-09-16T00:09:16.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:16.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.054604+0000", 2024-09-16T00:09:16.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00057022099999999999 2024-09-16T00:09:16.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:16.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.055768+0000", 2024-09-16T00:09:16.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001164113 2024-09-16T00:09:16.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.055821+0000", 2024-09-16T00:09:16.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2364000000000002e-05 2024-09-16T00:09:16.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.055864+0000", 2024-09-16T00:09:16.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3506e-05 2024-09-16T00:09:16.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.720 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.721 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:2315 3.1f e62/33 3:f84a2a2d:::10000000c60.00000000:head v 62'45, mlcod=62'43)", 2024-09-16T00:09:16.721 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:03.440894+0000", 2024-09-16T00:09:16.721 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.101199959000001, 2024-09-16T00:09:16.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028251109999999999, 2024-09-16T00:09:16.721 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.721 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.721 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.721 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.440894+0000", 2024-09-16T00:09:16.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.440894+0000", 2024-09-16T00:09:16.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.440894+0000", 2024-09-16T00:09:16.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7700000000000004e-07 2024-09-16T00:09:16.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441598+0000", 2024-09-16T00:09:16.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00070308100000000004 2024-09-16T00:09:16.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441607+0000", 2024-09-16T00:09:16.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7489999999999993e-06 2024-09-16T00:09:16.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441612+0000", 2024-09-16T00:09:16.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6519999999999997e-06 2024-09-16T00:09:16.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441650+0000", 2024-09-16T00:09:16.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7867000000000002e-05 2024-09-16T00:09:16.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441663+0000", 2024-09-16T00:09:16.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2788e-05 2024-09-16T00:09:16.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.443681+0000", 2024-09-16T00:09:16.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020186599999999998 2024-09-16T00:09:16.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.443719+0000", 2024-09-16T00:09:16.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7636999999999999e-05 2024-09-16T00:09:16.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.726 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.727 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:2326 3.1c 3:39be8b5c:::10000000c6c.00000000:head [write 0~173460] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T00:09:16.727 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:03.473107+0000", 2024-09-16T00:09:16.727 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.068986675, 2024-09-16T00:09:16.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001822029, 2024-09-16T00:09:16.727 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.727 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.727 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:16.727 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:16.728 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:16.728 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 2326 2024-09-16T00:09:16.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.728 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473107+0000", 2024-09-16T00:09:16.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473107+0000", 2024-09-16T00:09:16.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473108+0000", 2024-09-16T00:09:16.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0139999999999999e-06 2024-09-16T00:09:16.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473292+0000", 2024-09-16T00:09:16.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000184072 2024-09-16T00:09:16.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473305+0000", 2024-09-16T00:09:16.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2444e-05 2024-09-16T00:09:16.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473310+0000", 2024-09-16T00:09:16.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5720000000000002e-06 2024-09-16T00:09:16.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473353+0000", 2024-09-16T00:09:16.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3041999999999999e-05 2024-09-16T00:09:16.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473455+0000", 2024-09-16T00:09:16.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101297 2024-09-16T00:09:16.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:16.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473542+0000", 2024-09-16T00:09:16.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7787999999999994e-05 2024-09-16T00:09:16.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:16.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.474469+0000", 2024-09-16T00:09:16.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00092618500000000005 2024-09-16T00:09:16.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:16.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.474868+0000", 2024-09-16T00:09:16.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00039969400000000003 2024-09-16T00:09:16.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.474900+0000", 2024-09-16T00:09:16.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1341999999999999e-05 2024-09-16T00:09:16.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.474929+0000", 2024-09-16T00:09:16.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9578999999999999e-05 2024-09-16T00:09:16.734 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.734 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.734 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.735 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:6120 3.37 e69/20 3:ef2e412a:::10000001c11.00000000:head v 69'100, mlcod=68'98)", 2024-09-16T00:09:16.735 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:16.085749+0000", 2024-09-16T00:09:16.735 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.45634453600000002, 2024-09-16T00:09:16.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020186219999999999, 2024-09-16T00:09:16.735 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:16.735 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:16.735 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:16.735 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:16.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:16.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085749+0000", 2024-09-16T00:09:16.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:16.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085749+0000", 2024-09-16T00:09:16.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:16.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:16.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085749+0000", 2024-09-16T00:09:16.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3200000000000001e-07 2024-09-16T00:09:16.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:16.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085767+0000", 2024-09-16T00:09:16.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7212000000000001e-05 2024-09-16T00:09:16.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:16.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085775+0000", 2024-09-16T00:09:16.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8590000000000005e-06 2024-09-16T00:09:16.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:16.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085780+0000", 2024-09-16T00:09:16.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.989e-06 2024-09-16T00:09:16.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:16.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085825+0000", 2024-09-16T00:09:16.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5090999999999999e-05 2024-09-16T00:09:16.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:16.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085845+0000", 2024-09-16T00:09:16.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0412999999999999e-05 2024-09-16T00:09:16.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:16.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.087730+0000", 2024-09-16T00:09:16.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001884817 2024-09-16T00:09:16.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:16.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:16.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:16.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.087768+0000", 2024-09-16T00:09:16.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8009000000000001e-05 2024-09-16T00:09:16.740 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.740 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.740 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.740 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:16.740 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:16.740 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:16.742 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:09:16.742 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:16.740+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:16.842 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:16.841+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:16.890 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:16.890 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:16.890 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:16.890 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:16.899 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:09:16.943 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:16.942+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:17.003 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:17.003 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:17.004 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:17.004 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:17.004 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:17.012 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:09:17.053 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:17.052+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:17.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.076+0000 7f27fffff640 1 -- 172.21.15.139:0/1932990406 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v70) ==== 113+0+0 (secure 0 0 0) 0x7f280805db70 con 0x7f281806ea70 2024-09-16T00:09:17.077 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T00:09:17.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.079+0000 7f281de77640 1 -- 172.21.15.139:0/1932990406 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f280406c2b0 msgr2=0x7f280406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.079+0000 7f281de77640 1 --2- 172.21.15.139:0/1932990406 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f280406c2b0 0x7f280406e770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f281814ddc0 tx=0x7f280c009190 comp rx=0 tx=0).stop 2024-09-16T00:09:17.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.079+0000 7f281de77640 1 -- 172.21.15.139:0/1932990406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281806ea70 msgr2=0x7f28180cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.079+0000 7f281de77640 1 --2- 172.21.15.139:0/1932990406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281806ea70 0x7f28180cd5f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f280800e9b0 tx=0x7f280800ee70 comp rx=0 tx=0).stop 2024-09-16T00:09:17.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.079+0000 7f281de77640 1 -- 172.21.15.139:0/1932990406 shutdown_connections 2024-09-16T00:09:17.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.079+0000 7f281de77640 1 --2- 172.21.15.139:0/1932990406 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f280406c2b0 0x7f280406e770 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.079+0000 7f281de77640 1 --2- 172.21.15.139:0/1932990406 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f28180c6e00 0x7f28181e38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.079+0000 7f281de77640 1 --2- 172.21.15.139:0/1932990406 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f28180c6a20 0x7f28181e33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.079+0000 7f281de77640 1 --2- 172.21.15.139:0/1932990406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281806ea70 0x7f28180cd5f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.079+0000 7f281de77640 1 -- 172.21.15.139:0/1932990406 >> 172.21.15.139:0/1932990406 conn(0x7f281806cc30 msgr2=0x7f281814e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:17.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.079+0000 7f281de77640 1 -- 172.21.15.139:0/1932990406 shutdown_connections 2024-09-16T00:09:17.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.079+0000 7f281de77640 1 -- 172.21.15.139:0/1932990406 wait complete. 2024-09-16T00:09:17.092 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T00:09:17.096 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T00:09:17.163 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:17.163 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:17.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:17.163 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:17.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.163 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:17.164 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:09:17.164 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.545290053000002, 2024-09-16T00:09:17.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056100969999999997, 2024-09-16T00:09:17.164 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.164 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.164 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.164 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:09:17.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:09:17.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615352+0000", 2024-09-16T00:09:17.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9859999999999999e-06 2024-09-16T00:09:17.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615363+0000", 2024-09-16T00:09:17.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1168999999999999e-05 2024-09-16T00:09:17.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615382+0000", 2024-09-16T00:09:17.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8224e-05 2024-09-16T00:09:17.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615393+0000", 2024-09-16T00:09:17.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0925e-05 2024-09-16T00:09:17.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615624+0000", 2024-09-16T00:09:17.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023159600000000001 2024-09-16T00:09:17.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615721+0000", 2024-09-16T00:09:17.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7040000000000006e-05 2024-09-16T00:09:17.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:17.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615885+0000", 2024-09-16T00:09:17.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163456 2024-09-16T00:09:17.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:17.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:09:17.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002191368 2024-09-16T00:09:17.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:17.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620870+0000", 2024-09-16T00:09:17.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941839999999999 2024-09-16T00:09:17.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620922+0000", 2024-09-16T00:09:17.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1641999999999998e-05 2024-09-16T00:09:17.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620956+0000", 2024-09-16T00:09:17.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4507e-05 2024-09-16T00:09:17.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.169 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.170 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:17.170 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:09:17.170 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.545199694000004, 2024-09-16T00:09:17.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005695739, 2024-09-16T00:09:17.170 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.170 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.170 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.170 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:09:17.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:09:17.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:09:17.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4900000000000005e-07 2024-09-16T00:09:17.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615442+0000", 2024-09-16T00:09:17.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.994e-06 2024-09-16T00:09:17.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:09:17.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1081e-05 2024-09-16T00:09:17.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615473+0000", 2024-09-16T00:09:17.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1910000000000006e-06 2024-09-16T00:09:17.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616230+0000", 2024-09-16T00:09:17.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075700400000000003 2024-09-16T00:09:17.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616305+0000", 2024-09-16T00:09:17.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4759999999999996e-05 2024-09-16T00:09:17.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:17.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616466+0000", 2024-09-16T00:09:17.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000161714 2024-09-16T00:09:17.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:17.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619821+0000", 2024-09-16T00:09:17.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033547339999999998 2024-09-16T00:09:17.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:17.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621028+0000", 2024-09-16T00:09:17.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012067530000000001 2024-09-16T00:09:17.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621087+0000", 2024-09-16T00:09:17.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9364999999999998e-05 2024-09-16T00:09:17.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621132+0000", 2024-09-16T00:09:17.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5293999999999997e-05 2024-09-16T00:09:17.175 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.175 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.175 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.175 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:17.175 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:09:17.175 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.545137462, 2024-09-16T00:09:17.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050565760000000001, 2024-09-16T00:09:17.175 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.176 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.176 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.176 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:09:17.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:09:17.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615500+0000", 2024-09-16T00:09:17.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8199999999999999e-07 2024-09-16T00:09:17.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615505+0000", 2024-09-16T00:09:17.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5280000000000003e-06 2024-09-16T00:09:17.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:17.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0317e-05 2024-09-16T00:09:17.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615523+0000", 2024-09-16T00:09:17.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4580000000000004e-06 2024-09-16T00:09:17.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615607+0000", 2024-09-16T00:09:17.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4153999999999994e-05 2024-09-16T00:09:17.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615748+0000", 2024-09-16T00:09:17.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140533 2024-09-16T00:09:17.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:17.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615911+0000", 2024-09-16T00:09:17.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163678 2024-09-16T00:09:17.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:17.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619944+0000", 2024-09-16T00:09:17.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040330909999999999 2024-09-16T00:09:17.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:17.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620471+0000", 2024-09-16T00:09:17.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052645600000000002 2024-09-16T00:09:17.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620512+0000", 2024-09-16T00:09:17.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0738000000000002e-05 2024-09-16T00:09:17.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620555+0000", 2024-09-16T00:09:17.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3841000000000003e-05 2024-09-16T00:09:17.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.183 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.183 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.184 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:17.184 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:09:17.184 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.545085354999998, 2024-09-16T00:09:17.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089162479999999999, 2024-09-16T00:09:17.184 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.184 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.184 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.184 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:09:17.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:09:17.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615552+0000", 2024-09-16T00:09:17.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:09:17.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615556+0000", 2024-09-16T00:09:17.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3610000000000003e-06 2024-09-16T00:09:17.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615566+0000", 2024-09-16T00:09:17.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6099999999999995e-06 2024-09-16T00:09:17.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615574+0000", 2024-09-16T00:09:17.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9379999999999992e-06 2024-09-16T00:09:17.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616777+0000", 2024-09-16T00:09:17.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012031640000000001 2024-09-16T00:09:17.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616882+0000", 2024-09-16T00:09:17.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105146 2024-09-16T00:09:17.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:17.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619696+0000", 2024-09-16T00:09:17.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028142179999999998 2024-09-16T00:09:17.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:17.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624234+0000", 2024-09-16T00:09:17.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045381889999999998 2024-09-16T00:09:17.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:17.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624393+0000", 2024-09-16T00:09:17.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015871200000000001 2024-09-16T00:09:17.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624434+0000", 2024-09-16T00:09:17.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0908999999999997e-05 2024-09-16T00:09:17.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624467+0000", 2024-09-16T00:09:17.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3217999999999999e-05 2024-09-16T00:09:17.189 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.189 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.189 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.189 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:17.189 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:09:17.189 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.545015558000003, 2024-09-16T00:09:17.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064898359999999997, 2024-09-16T00:09:17.189 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.190 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.190 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.190 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:09:17.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:09:17.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615622+0000", 2024-09-16T00:09:17.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.181e-06 2024-09-16T00:09:17.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615628+0000", 2024-09-16T00:09:17.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2450000000000003e-06 2024-09-16T00:09:17.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:09:17.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2554999999999999e-05 2024-09-16T00:09:17.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615650+0000", 2024-09-16T00:09:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4350000000000003e-06 2024-09-16T00:09:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616772+0000", 2024-09-16T00:09:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001122058 2024-09-16T00:09:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616889+0000", 2024-09-16T00:09:17.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116623 2024-09-16T00:09:17.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:17.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617035+0000", 2024-09-16T00:09:17.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014579799999999999 2024-09-16T00:09:17.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:17.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619829+0000", 2024-09-16T00:09:17.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941379999999998 2024-09-16T00:09:17.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:17.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622024+0000", 2024-09-16T00:09:17.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021949920000000002 2024-09-16T00:09:17.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622087+0000", 2024-09-16T00:09:17.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3048999999999999e-05 2024-09-16T00:09:17.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622111+0000", 2024-09-16T00:09:17.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3762000000000001e-05 2024-09-16T00:09:17.197 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.197 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.197 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.198 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:17.198 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:09:17.198 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.544952996999996, 2024-09-16T00:09:17.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065192160000000004, 2024-09-16T00:09:17.198 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.198 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.198 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.198 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:09:17.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:09:17.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615684+0000", 2024-09-16T00:09:17.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8299999999999995e-07 2024-09-16T00:09:17.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615691+0000", 2024-09-16T00:09:17.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.866e-06 2024-09-16T00:09:17.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615707+0000", 2024-09-16T00:09:17.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6073999999999999e-05 2024-09-16T00:09:17.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615714+0000", 2024-09-16T00:09:17.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-16T00:09:17.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617403+0000", 2024-09-16T00:09:17.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016881629999999999 2024-09-16T00:09:17.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617559+0000", 2024-09-16T00:09:17.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015669699999999999 2024-09-16T00:09:17.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:17.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:09:17.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000144165 2024-09-16T00:09:17.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:17.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621413+0000", 2024-09-16T00:09:17.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037092200000000001 2024-09-16T00:09:17.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:17.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622147+0000", 2024-09-16T00:09:17.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073468800000000005 2024-09-16T00:09:17.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622180+0000", 2024-09-16T00:09:17.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2901999999999998e-05 2024-09-16T00:09:17.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622203+0000", 2024-09-16T00:09:17.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2211000000000001e-05 2024-09-16T00:09:17.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.203 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.203 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:17.203 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:09:17.203 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.544909903000004, 2024-09-16T00:09:17.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065825270000000003, 2024-09-16T00:09:17.204 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.204 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.204 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:09:17.204 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:09:17.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:09:17.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:09:17.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7299999999999996e-07 2024-09-16T00:09:17.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615730+0000", 2024-09-16T00:09:17.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.004e-06 2024-09-16T00:09:17.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615737+0000", 2024-09-16T00:09:17.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4789999999999999e-06 2024-09-16T00:09:17.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615745+0000", 2024-09-16T00:09:17.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6699999999999994e-06 2024-09-16T00:09:17.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618115+0000", 2024-09-16T00:09:17.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023701569999999999 2024-09-16T00:09:17.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618247+0000", 2024-09-16T00:09:17.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013185000000000001 2024-09-16T00:09:17.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622309+0000", 2024-09-16T00:09:17.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040617939999999996 2024-09-16T00:09:17.207 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.207 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.207 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.207 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:17.207 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:09:17.208 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.544882654000006, 2024-09-16T00:09:17.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036685950000000002, 2024-09-16T00:09:17.208 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.208 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.208 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:09:17.208 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:09:17.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:09:17.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:09:17.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T00:09:17.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615763+0000", 2024-09-16T00:09:17.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4360000000000002e-06 2024-09-16T00:09:17.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615770+0000", 2024-09-16T00:09:17.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7540000000000006e-06 2024-09-16T00:09:17.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615803+0000", 2024-09-16T00:09:17.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3008e-05 2024-09-16T00:09:17.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615888+0000", 2024-09-16T00:09:17.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5056999999999996e-05 2024-09-16T00:09:17.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:09:17.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013059700000000001 2024-09-16T00:09:17.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619422+0000", 2024-09-16T00:09:17.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034033230000000002 2024-09-16T00:09:17.211 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.211 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.211 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.212 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:16 2.1d e22/19 2:b871830b:::605.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:17.212 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:09:17.212 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.540894064, 2024-09-16T00:09:17.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045616900000000002, 2024-09-16T00:09:17.212 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.212 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.212 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.212 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:09:17.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:09:17.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:09:17.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:09:17.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619755+0000", 2024-09-16T00:09:17.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3599999999999996e-06 2024-09-16T00:09:17.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619767+0000", 2024-09-16T00:09:17.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2805e-05 2024-09-16T00:09:17.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619787+0000", 2024-09-16T00:09:17.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9264000000000001e-05 2024-09-16T00:09:17.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619869+0000", 2024-09-16T00:09:17.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2639999999999995e-05 2024-09-16T00:09:17.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619895+0000", 2024-09-16T00:09:17.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5890000000000001e-05 2024-09-16T00:09:17.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624264+0000", 2024-09-16T00:09:17.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043690049999999996 2024-09-16T00:09:17.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624304+0000", 2024-09-16T00:09:17.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9807e-05 2024-09-16T00:09:17.216 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.216 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.216 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.216 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:17 2.1d e22/19 2:b871830b:::605.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:17.216 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:09:17.217 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.540541355000002, 2024-09-16T00:09:17.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042370910000000001, 2024-09-16T00:09:17.217 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.217 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.217 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.217 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:09:17.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:09:17.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620096+0000", 2024-09-16T00:09:17.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:09:17.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620102+0000", 2024-09-16T00:09:17.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7220000000000002e-06 2024-09-16T00:09:17.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620112+0000", 2024-09-16T00:09:17.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0156999999999999e-05 2024-09-16T00:09:17.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620120+0000", 2024-09-16T00:09:17.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2880000000000004e-06 2024-09-16T00:09:17.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620171+0000", 2024-09-16T00:09:17.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.168e-05 2024-09-16T00:09:17.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620189+0000", 2024-09-16T00:09:17.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7419e-05 2024-09-16T00:09:17.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624306+0000", 2024-09-16T00:09:17.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041175430000000004 2024-09-16T00:09:17.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624332+0000", 2024-09-16T00:09:17.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5717000000000001e-05 2024-09-16T00:09:17.220 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.220 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.220 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.221 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:09:17.221 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:09:17.221 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 98.538174538000007, 2024-09-16T00:09:17.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044651079999999998, 2024-09-16T00:09:17.221 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.221 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.221 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.221 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:09:17.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:09:17.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622463+0000", 2024-09-16T00:09:17.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5349999999999999e-06 2024-09-16T00:09:17.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622470+0000", 2024-09-16T00:09:17.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7050000000000002e-06 2024-09-16T00:09:17.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622485+0000", 2024-09-16T00:09:17.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4978e-05 2024-09-16T00:09:17.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622493+0000", 2024-09-16T00:09:17.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2139999999999996e-06 2024-09-16T00:09:17.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622546+0000", 2024-09-16T00:09:17.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:09:17.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622595+0000", 2024-09-16T00:09:17.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9112e-05 2024-09-16T00:09:17.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:17.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622677+0000", 2024-09-16T00:09:17.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2332999999999999e-05 2024-09-16T00:09:17.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:17.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.623515+0000", 2024-09-16T00:09:17.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000837615 2024-09-16T00:09:17.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:17.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626867+0000", 2024-09-16T00:09:17.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033524819999999999 2024-09-16T00:09:17.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626902+0000", 2024-09-16T00:09:17.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4310000000000002e-05 2024-09-16T00:09:17.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626927+0000", 2024-09-16T00:09:17.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5128999999999999e-05 2024-09-16T00:09:17.226 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.226 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.226 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.226 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:103 3.26 3:6519881c:::1000000025e.00000000:head [write 0~2892] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:17.227 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.487113+0000", 2024-09-16T00:09:17.227 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 20.673523389, 2024-09-16T00:09:17.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034171290000000001, 2024-09-16T00:09:17.227 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.227 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.227 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.227 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:17.227 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:17.227 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:17.227 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103 2024-09-16T00:09:17.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.227 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487113+0000", 2024-09-16T00:09:17.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487113+0000", 2024-09-16T00:09:17.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487114+0000", 2024-09-16T00:09:17.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3030000000000001e-06 2024-09-16T00:09:17.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487122+0000", 2024-09-16T00:09:17.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3889999999999999e-06 2024-09-16T00:09:17.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487133+0000", 2024-09-16T00:09:17.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1325000000000001e-05 2024-09-16T00:09:17.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487138+0000", 2024-09-16T00:09:17.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0660000000000003e-06 2024-09-16T00:09:17.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487182+0000", 2024-09-16T00:09:17.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3655999999999997e-05 2024-09-16T00:09:17.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487307+0000", 2024-09-16T00:09:17.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012541599999999999 2024-09-16T00:09:17.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:17.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487391+0000", 2024-09-16T00:09:17.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4404e-05 2024-09-16T00:09:17.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:17.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487908+0000", 2024-09-16T00:09:17.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000516608 2024-09-16T00:09:17.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:17.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.490429+0000", 2024-09-16T00:09:17.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025209339999999998 2024-09-16T00:09:17.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.490482+0000", 2024-09-16T00:09:17.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2655000000000003e-05 2024-09-16T00:09:17.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.490530+0000", 2024-09-16T00:09:17.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8372999999999999e-05 2024-09-16T00:09:17.233 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.233 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.233 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.233 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:53 2.14 2:2e2fa760:::200.00000001:head [write 20850~1000165 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e58)", 2024-09-16T00:09:17.233 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.452702+0000", 2024-09-16T00:09:17.233 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.707933840999999, 2024-09-16T00:09:17.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046625019999999998, 2024-09-16T00:09:17.233 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.234 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.234 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.234 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.452702+0000", 2024-09-16T00:09:17.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.452702+0000", 2024-09-16T00:09:17.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.452704+0000", 2024-09-16T00:09:17.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4699999999999999e-06 2024-09-16T00:09:17.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453184+0000", 2024-09-16T00:09:17.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048001000000000001 2024-09-16T00:09:17.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453199+0000", 2024-09-16T00:09:17.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4905e-05 2024-09-16T00:09:17.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453222+0000", 2024-09-16T00:09:17.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3587999999999998e-05 2024-09-16T00:09:17.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453288+0000", 2024-09-16T00:09:17.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5817999999999999e-05 2024-09-16T00:09:17.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453348+0000", 2024-09-16T00:09:17.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9716999999999998e-05 2024-09-16T00:09:17.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:17.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453440+0000", 2024-09-16T00:09:17.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2090999999999996e-05 2024-09-16T00:09:17.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:17.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.455907+0000", 2024-09-16T00:09:17.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024670730000000002 2024-09-16T00:09:17.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:17.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.457293+0000", 2024-09-16T00:09:17.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013861979999999999 2024-09-16T00:09:17.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.457326+0000", 2024-09-16T00:09:17.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2484000000000003e-05 2024-09-16T00:09:17.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.457365+0000", 2024-09-16T00:09:17.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9147999999999999e-05 2024-09-16T00:09:17.239 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.240 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.240 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:55 2.14 2:2e2fa760:::200.00000001:head [write 1021015~941162 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e59)", 2024-09-16T00:09:17.240 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:58.642546+0000", 2024-09-16T00:09:17.240 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.518090723, 2024-09-16T00:09:17.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041709410000000001, 2024-09-16T00:09:17.240 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.240 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.241 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.241 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.642546+0000", 2024-09-16T00:09:17.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.642546+0000", 2024-09-16T00:09:17.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.642546+0000", 2024-09-16T00:09:17.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5499999999999997e-07 2024-09-16T00:09:17.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643041+0000", 2024-09-16T00:09:17.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049419399999999999 2024-09-16T00:09:17.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643051+0000", 2024-09-16T00:09:17.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0229e-05 2024-09-16T00:09:17.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643056+0000", 2024-09-16T00:09:17.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.327e-06 2024-09-16T00:09:17.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643112+0000", 2024-09-16T00:09:17.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5939000000000001e-05 2024-09-16T00:09:17.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643193+0000", 2024-09-16T00:09:17.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0921999999999995e-05 2024-09-16T00:09:17.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:17.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643305+0000", 2024-09-16T00:09:17.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011162099999999999 2024-09-16T00:09:17.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:17.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.645976+0000", 2024-09-16T00:09:17.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026712609999999999 2024-09-16T00:09:17.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:17.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.646651+0000", 2024-09-16T00:09:17.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00067499200000000002 2024-09-16T00:09:17.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.646684+0000", 2024-09-16T00:09:17.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3328999999999999e-05 2024-09-16T00:09:17.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.646717+0000", 2024-09-16T00:09:17.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2271999999999999e-05 2024-09-16T00:09:17.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.245 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.246 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:1617 3.36 3:6fb28598:::1000000094d.00000000:head [write 0~2552] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T00:09:17.246 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.251180+0000", 2024-09-16T00:09:17.246 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.909456808, 2024-09-16T00:09:17.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004599601, 2024-09-16T00:09:17.246 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.246 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.246 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:17.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:17.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:17.246 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1617 2024-09-16T00:09:17.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.247 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251180+0000", 2024-09-16T00:09:17.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251180+0000", 2024-09-16T00:09:17.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251180+0000", 2024-09-16T00:09:17.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4600000000000003e-07 2024-09-16T00:09:17.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251185+0000", 2024-09-16T00:09:17.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3239999999999996e-06 2024-09-16T00:09:17.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251194+0000", 2024-09-16T00:09:17.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3049999999999998e-06 2024-09-16T00:09:17.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251199+0000", 2024-09-16T00:09:17.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2780000000000001e-06 2024-09-16T00:09:17.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251240+0000", 2024-09-16T00:09:17.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1171999999999998e-05 2024-09-16T00:09:17.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251317+0000", 2024-09-16T00:09:17.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6804999999999994e-05 2024-09-16T00:09:17.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:17.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251399+0000", 2024-09-16T00:09:17.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2101000000000006e-05 2024-09-16T00:09:17.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:17.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255497+0000", 2024-09-16T00:09:17.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040972889999999996 2024-09-16T00:09:17.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:17.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255687+0000", 2024-09-16T00:09:17.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019032899999999999 2024-09-16T00:09:17.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255739+0000", 2024-09-16T00:09:17.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2086999999999998e-05 2024-09-16T00:09:17.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255779+0000", 2024-09-16T00:09:17.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0064999999999997e-05 2024-09-16T00:09:17.252 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.252 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.252 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.252 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:1691 3.a 3:518183a8:::10000000999.00000000:head [write 0~2546] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T00:09:17.252 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.411469+0000", 2024-09-16T00:09:17.252 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.749167772, 2024-09-16T00:09:17.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032776289999999998, 2024-09-16T00:09:17.252 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.252 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.252 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.253 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:17.253 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:17.253 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:17.253 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1691 2024-09-16T00:09:17.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.253 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411469+0000", 2024-09-16T00:09:17.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411469+0000", 2024-09-16T00:09:17.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411470+0000", 2024-09-16T00:09:17.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.128e-06 2024-09-16T00:09:17.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411476+0000", 2024-09-16T00:09:17.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3720000000000001e-06 2024-09-16T00:09:17.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411488+0000", 2024-09-16T00:09:17.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2319e-05 2024-09-16T00:09:17.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411494+0000", 2024-09-16T00:09:17.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.519e-06 2024-09-16T00:09:17.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411541+0000", 2024-09-16T00:09:17.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6723e-05 2024-09-16T00:09:17.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411625+0000", 2024-09-16T00:09:17.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4127000000000003e-05 2024-09-16T00:09:17.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:17.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411704+0000", 2024-09-16T00:09:17.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9511000000000005e-05 2024-09-16T00:09:17.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:17.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.414377+0000", 2024-09-16T00:09:17.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026722999999999998 2024-09-16T00:09:17.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:17.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.414654+0000", 2024-09-16T00:09:17.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027762400000000002 2024-09-16T00:09:17.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.414704+0000", 2024-09-16T00:09:17.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9413000000000003e-05 2024-09-16T00:09:17.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.414746+0000", 2024-09-16T00:09:17.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2592999999999997e-05 2024-09-16T00:09:17.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.258 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.258 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.258 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:72 2.14 2:2e2fa760:::200.00000001:head [write 3118411~1075893 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:17.258 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.671915+0000", 2024-09-16T00:09:17.258 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.488720873, 2024-09-16T00:09:17.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049035859999999997, 2024-09-16T00:09:17.258 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.258 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.258 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.258 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.671915+0000", 2024-09-16T00:09:17.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.671915+0000", 2024-09-16T00:09:17.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.671917+0000", 2024-09-16T00:09:17.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7e-06 2024-09-16T00:09:17.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672671+0000", 2024-09-16T00:09:17.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075343199999999995 2024-09-16T00:09:17.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672686+0000", 2024-09-16T00:09:17.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4960000000000001e-05 2024-09-16T00:09:17.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672693+0000", 2024-09-16T00:09:17.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0770000000000002e-06 2024-09-16T00:09:17.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672737+0000", 2024-09-16T00:09:17.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4614999999999999e-05 2024-09-16T00:09:17.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672793+0000", 2024-09-16T00:09:17.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5785999999999997e-05 2024-09-16T00:09:17.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:17.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672870+0000", 2024-09-16T00:09:17.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7187999999999993e-05 2024-09-16T00:09:17.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:17.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.675448+0000", 2024-09-16T00:09:17.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002578033 2024-09-16T00:09:17.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:17.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.676746+0000", 2024-09-16T00:09:17.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001297896 2024-09-16T00:09:17.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.676778+0000", 2024-09-16T00:09:17.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1605e-05 2024-09-16T00:09:17.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.676819+0000", 2024-09-16T00:09:17.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1294000000000001e-05 2024-09-16T00:09:17.263 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.263 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.263 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.263 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:75 2.11 2:8fcca0a5:::10000000261.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:17.263 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.985646+0000", 2024-09-16T00:09:17.263 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.174990468000001, 2024-09-16T00:09:17.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041697660000000001, 2024-09-16T00:09:17.263 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.264 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.264 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.264 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985646+0000", 2024-09-16T00:09:17.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985646+0000", 2024-09-16T00:09:17.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985648+0000", 2024-09-16T00:09:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5739999999999998e-06 2024-09-16T00:09:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985699+0000", 2024-09-16T00:09:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0513999999999998e-05 2024-09-16T00:09:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985718+0000", 2024-09-16T00:09:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8689999999999999e-05 2024-09-16T00:09:17.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985729+0000", 2024-09-16T00:09:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1812e-05 2024-09-16T00:09:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985802+0000", 2024-09-16T00:09:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2465999999999994e-05 2024-09-16T00:09:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985912+0000", 2024-09-16T00:09:17.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109795 2024-09-16T00:09:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.986236+0000", 2024-09-16T00:09:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032388799999999999 2024-09-16T00:09:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.987505+0000", 2024-09-16T00:09:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001269788 2024-09-16T00:09:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:17.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.989708+0000", 2024-09-16T00:09:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022022579999999999 2024-09-16T00:09:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.989765+0000", 2024-09-16T00:09:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7268000000000001e-05 2024-09-16T00:09:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.989816+0000", 2024-09-16T00:09:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0713e-05 2024-09-16T00:09:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:84 2.17 2:ee44d6a7:::10000000261.03e00000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.076954+0000", 2024-09-16T00:09:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 15.083682282, 2024-09-16T00:09:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004155237, 2024-09-16T00:09:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076954+0000", 2024-09-16T00:09:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076954+0000", 2024-09-16T00:09:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076955+0000", 2024-09-16T00:09:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4199999999999996e-07 2024-09-16T00:09:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076971+0000", 2024-09-16T00:09:17.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6294999999999999e-05 2024-09-16T00:09:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076980+0000", 2024-09-16T00:09:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5599999999999999e-06 2024-09-16T00:09:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076986+0000", 2024-09-16T00:09:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9329999999999998e-06 2024-09-16T00:09:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077032+0000", 2024-09-16T00:09:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5566999999999999e-05 2024-09-16T00:09:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077143+0000", 2024-09-16T00:09:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011068 2024-09-16T00:09:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077295+0000", 2024-09-16T00:09:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152578 2024-09-16T00:09:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078459+0000", 2024-09-16T00:09:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001163645 2024-09-16T00:09:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081048+0000", 2024-09-16T00:09:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025893840000000001 2024-09-16T00:09:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081084+0000", 2024-09-16T00:09:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5834999999999998e-05 2024-09-16T00:09:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081109+0000", 2024-09-16T00:09:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5218000000000001e-05 2024-09-16T00:09:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:224 2.1a 2:5e41ab69:::200.00000005:head [write 0~1943310 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e66)", 2024-09-16T00:09:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:12.641517+0000", 2024-09-16T00:09:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.5191192039999999, 2024-09-16T00:09:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064053460000000001, 2024-09-16T00:09:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:17.274 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.641517+0000", 2024-09-16T00:09:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.641517+0000", 2024-09-16T00:09:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:17.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.641520+0000", 2024-09-16T00:09:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5660000000000001e-06 2024-09-16T00:09:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.642832+0000", 2024-09-16T00:09:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00131274 2024-09-16T00:09:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.642853+0000", 2024-09-16T00:09:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0710999999999999e-05 2024-09-16T00:09:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.642880+0000", 2024-09-16T00:09:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6693999999999999e-05 2024-09-16T00:09:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.642939+0000", 2024-09-16T00:09:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9184000000000001e-05 2024-09-16T00:09:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.643032+0000", 2024-09-16T00:09:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2583000000000004e-05 2024-09-16T00:09:17.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.643114+0000", 2024-09-16T00:09:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2554999999999998e-05 2024-09-16T00:09:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.647381+0000", 2024-09-16T00:09:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042663969999999999 2024-09-16T00:09:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.647813+0000", 2024-09-16T00:09:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043280899999999998 2024-09-16T00:09:17.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.647873+0000", 2024-09-16T00:09:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9619e-05 2024-09-16T00:09:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.647922+0000", 2024-09-16T00:09:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9487999999999998e-05 2024-09-16T00:09:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:17.279 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:17.280 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:17.280 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:09:17.281 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:17.175+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:17.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.234+0000 7feb71b66640 1 Processor -- start 2024-09-16T00:09:17.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.234+0000 7feb71b66640 1 -- start start 2024-09-16T00:09:17.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.234+0000 7feb71b66640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feb6c1528d0 0x7feb6c158630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.234+0000 7feb71b66640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb6c072f00 0x7feb6c0c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7feb71b66640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6c0702c0 con 0x7feb6c1524f0 2024-09-16T00:09:17.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7feb71b66640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7feb6c0c9ae0 con 0x7feb6c1528d0 2024-09-16T00:09:17.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7feb71b66640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7feb6c0c6c00 con 0x7feb6c072f00 2024-09-16T00:09:17.283 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:09:17.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7f996e128640 1 Processor -- start 2024-09-16T00:09:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7f996e128640 1 -- start start 2024-09-16T00:09:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7feb6bfff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb6c072f00 0x7feb6c0c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7feb6affd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feb6c1528d0 0x7feb6c158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7feb6affd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7feb6c1528d0 0x7feb6c158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:47528/0 (socket says 172.21.15.139:47528) 2024-09-16T00:09:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7feb6bfff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb6c072f00 0x7feb6c0c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:49112/0 (socket says 172.21.15.139:49112) 2024-09-16T00:09:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7feb6b7fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7feb6c1524f0 0x7feb6c0c5910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:56052/0 (socket says 172.21.15.139:56052) 2024-09-16T00:09:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7feb6affd640 1 -- 172.21.15.139:0/980896288 learned_addr learned my addr 172.21.15.139:0/980896288 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7feb6bfff640 1 -- 172.21.15.139:0/980896288 >> v2:172.21.15.164:3300/0 conn(0x7feb6c1528d0 msgr2=0x7feb6c158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7feb6bfff640 1 --2- 172.21.15.139:0/980896288 >> v2:172.21.15.164:3300/0 conn(0x7feb6c1528d0 0x7feb6c158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7feb6bfff640 1 -- 172.21.15.139:0/980896288 >> v1:172.21.15.164:6789/0 conn(0x7feb6c1524f0 legacy=0x7feb6c0c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7feb6a7fc640 1 -- 172.21.15.139:0/980896288 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3942831166 0 0) 0x7feb6c0702c0 con 0x7feb6c1524f0 2024-09-16T00:09:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7feb6bfff640 1 -- 172.21.15.139:0/980896288 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c0c8320 con 0x7feb6c072f00 2024-09-16T00:09:17.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.235+0000 7feb6affd640 1 --2- 172.21.15.139:0/980896288 >> v2:172.21.15.164:3300/0 conn(0x7feb6c1528d0 0x7feb6c158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:17.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7f996e128640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9968152b20 0x7f99680c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7f996e128640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99681587b0 0x7f9968158c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7f996e128640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9968159180 con 0x7f99680c8c50 2024-09-16T00:09:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7f996e128640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f99680c5470 con 0x7f9968152b20 2024-09-16T00:09:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7f996e128640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f99680cbba0 con 0x7f99681587b0 2024-09-16T00:09:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7f99677fe640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9968152b20 0x7f99680c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7f9966ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f99680c8c50 0x7f996806ff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41586/0 (socket says 172.21.15.139:41586) 2024-09-16T00:09:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7f9966ffd640 1 -- 172.21.15.139:0/2173633389 learned_addr learned my addr 172.21.15.139:0/2173633389 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7f9967fff640 1 --2- 172.21.15.139:0/2173633389 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99681587b0 0x7f9968158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7f99677fe640 1 -- 172.21.15.139:0/2173633389 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99681587b0 msgr2=0x7f9968158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7f99677fe640 1 --2- 172.21.15.139:0/2173633389 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99681587b0 0x7f9968158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7f99667fc640 1 -- 172.21.15.139:0/2173633389 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1537734045 0 0) 0x7f9968159180 con 0x7f99680c8c50 2024-09-16T00:09:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7f99677fe640 1 -- 172.21.15.139:0/2173633389 >> v1:172.21.15.139:6789/0 conn(0x7f99680c8c50 legacy=0x7f996806ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7f99677fe640 1 -- 172.21.15.139:0/2173633389 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99680ce310 con 0x7f9968152b20 2024-09-16T00:09:17.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7feb6bfff640 1 --2- 172.21.15.139:0/980896288 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb6c072f00 0x7feb6c0c95a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feb64009d90 tx=0x7feb6402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc585efb2a28f596 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7f9967fff640 1 --2- 172.21.15.139:0/2173633389 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99681587b0 0x7f9968158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:17.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7feb6a7fc640 1 -- 172.21.15.139:0/980896288 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64007740 con 0x7feb6c072f00 2024-09-16T00:09:17.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7feb6a7fc640 1 -- 172.21.15.139:0/980896288 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb640078e0 con 0x7feb6c072f00 2024-09-16T00:09:17.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.236+0000 7feb6a7fc640 1 -- 172.21.15.139:0/980896288 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64007cb0 con 0x7feb6c072f00 2024-09-16T00:09:17.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.237+0000 7f99677fe640 1 --2- 172.21.15.139:0/2173633389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9968152b20 0x7f99680c4f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f99580097d0 tx=0x7f995802a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3419de20cc25d148 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.237+0000 7feb71b66640 1 -- 172.21.15.139:0/980896288 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb6c072f00 msgr2=0x7feb6c0c95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.237+0000 7feb71b66640 1 --2- 172.21.15.139:0/980896288 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb6c072f00 0x7feb6c0c95a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feb64009d90 tx=0x7feb6402a660 comp rx=0 tx=0).stop 2024-09-16T00:09:17.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.237+0000 7f99667fc640 1 -- 172.21.15.139:0/2173633389 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958037070 con 0x7f9968152b20 2024-09-16T00:09:17.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.237+0000 7f99667fc640 1 -- 172.21.15.139:0/2173633389 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995802ae40 con 0x7f9968152b20 2024-09-16T00:09:17.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.237+0000 7feb71b66640 1 -- 172.21.15.139:0/980896288 shutdown_connections 2024-09-16T00:09:17.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.237+0000 7f99667fc640 1 -- 172.21.15.139:0/2173633389 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958032660 con 0x7f9968152b20 2024-09-16T00:09:17.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.237+0000 7feb71b66640 1 --2- 172.21.15.139:0/980896288 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb6c072f00 0x7feb6c0c95a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.237+0000 7feb71b66640 1 --2- 172.21.15.139:0/980896288 >> v2:172.21.15.164:3300/0 conn(0x7feb6c1528d0 0x7feb6c158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.237+0000 7feb71b66640 1 -- 172.21.15.139:0/980896288 >> 172.21.15.139:0/980896288 conn(0x7feb6c06da40 msgr2=0x7feb6c06f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:17.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.237+0000 7feb71b66640 1 -- 172.21.15.139:0/980896288 shutdown_connections 2024-09-16T00:09:17.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.237+0000 7feb71b66640 1 -- 172.21.15.139:0/980896288 wait complete. 2024-09-16T00:09:17.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.238+0000 7f996e128640 1 -- 172.21.15.139:0/2173633389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9968152b20 msgr2=0x7f99680c4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.238+0000 7f996e128640 1 --2- 172.21.15.139:0/2173633389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9968152b20 0x7f99680c4f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f99580097d0 tx=0x7f995802a130 comp rx=0 tx=0).stop 2024-09-16T00:09:17.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.238+0000 7f996e128640 1 -- 172.21.15.139:0/2173633389 shutdown_connections 2024-09-16T00:09:17.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.238+0000 7f996e128640 1 --2- 172.21.15.139:0/2173633389 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99681587b0 0x7f9968158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.238+0000 7f996e128640 1 --2- 172.21.15.139:0/2173633389 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9968152b20 0x7f99680c4f30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.238+0000 7f996e128640 1 -- 172.21.15.139:0/2173633389 >> 172.21.15.139:0/2173633389 conn(0x7f996806bd00 msgr2=0x7f9968067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:17.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.238+0000 7f996e128640 1 -- 172.21.15.139:0/2173633389 shutdown_connections 2024-09-16T00:09:17.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.238+0000 7f996e128640 1 -- 172.21.15.139:0/2173633389 wait complete. 2024-09-16T00:09:17.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.238+0000 7feb71b66640 1 Processor -- start 2024-09-16T00:09:17.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.238+0000 7feb71b66640 1 -- start start 2024-09-16T00:09:17.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.239+0000 7f996e128640 1 Processor -- start 2024-09-16T00:09:17.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.239+0000 7f996e128640 1 -- start start 2024-09-16T00:09:17.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.239+0000 7feb71b66640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb6c072f00 0x7feb6c0cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.239+0000 7feb71b66640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb6c1524f0 0x7feb6c151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.239+0000 7feb71b66640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6c1528d0 0x7feb6c1f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.239+0000 7feb71b66640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c06ffe0 con 0x7feb6c1528d0 2024-09-16T00:09:17.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.239+0000 7feb71b66640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c06fd00 con 0x7feb6c072f00 2024-09-16T00:09:17.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.239+0000 7feb71b66640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7feb6c0c8da0 con 0x7feb6c1524f0 2024-09-16T00:09:17.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.239+0000 7feb6bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6c1528d0 0x7feb6c1f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.239+0000 7feb6bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6c1528d0 0x7feb6c1f2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53874/0 (socket says 172.21.15.139:53874) 2024-09-16T00:09:17.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.239+0000 7feb6bfff640 1 -- 172.21.15.139:0/2615504394 learned_addr learned my addr 172.21.15.139:0/2615504394 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:17.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.239+0000 7feb6affd640 1 --2- 172.21.15.139:0/2615504394 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb6c1524f0 0x7feb6c151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.239+0000 7feb6b7fe640 1 --2- 172.21.15.139:0/2615504394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb6c072f00 0x7feb6c0cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7feb6bfff640 1 -- 172.21.15.139:0/2615504394 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb6c1524f0 msgr2=0x7feb6c151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7feb6bfff640 1 --2- 172.21.15.139:0/2615504394 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb6c1524f0 0x7feb6c151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7feb6bfff640 1 -- 172.21.15.139:0/2615504394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb6c072f00 msgr2=0x7feb6c0cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7feb6bfff640 1 --2- 172.21.15.139:0/2615504394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb6c072f00 0x7feb6c0cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7feb6bfff640 1 -- 172.21.15.139:0/2615504394 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c0ca130 con 0x7feb6c1528d0 2024-09-16T00:09:17.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7feb6affd640 1 --2- 172.21.15.139:0/2615504394 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb6c1524f0 0x7feb6c151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:17.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7feb6b7fe640 1 --2- 172.21.15.139:0/2615504394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb6c072f00 0x7feb6c0cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:17.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f996e128640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99680c8c50 0x7f99681e0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7feb6bfff640 1 --2- 172.21.15.139:0/2615504394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6c1528d0 0x7feb6c1f2440 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7feb64009eb0 tx=0x7feb64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f996e128640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9968152b20 0x7f99681e3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f996e128640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99681587b0 0x7f99681f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f996e128640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f99680ce5b0 con 0x7f9968152b20 2024-09-16T00:09:17.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f996e128640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f99680c3660 con 0x7f99681587b0 2024-09-16T00:09:17.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f996e128640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f99680c3940 con 0x7f99680c8c50 2024-09-16T00:09:17.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f9966ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9968152b20 0x7f99681e3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f99677fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99680c8c50 0x7f99681e0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f9966ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9968152b20 0x7f99681e3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53876/0 (socket says 172.21.15.139:53876) 2024-09-16T00:09:17.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f9966ffd640 1 -- 172.21.15.139:0/3641262673 learned_addr learned my addr 172.21.15.139:0/3641262673 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:17.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f9967fff640 1 --2- 172.21.15.139:0/3641262673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99681587b0 0x7f99681f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7feb53fff640 1 -- 172.21.15.139:0/2615504394 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64040070 con 0x7feb6c1528d0 2024-09-16T00:09:17.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7feb53fff640 1 -- 172.21.15.139:0/2615504394 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb64031c40 con 0x7feb6c1528d0 2024-09-16T00:09:17.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7feb71b66640 1 -- 172.21.15.139:0/2615504394 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c1f1c80 con 0x7feb6c1528d0 2024-09-16T00:09:17.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7feb53fff640 1 -- 172.21.15.139:0/2615504394 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64009450 con 0x7feb6c1528d0 2024-09-16T00:09:17.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7feb71b66640 1 -- 172.21.15.139:0/2615504394 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c1f1f20 con 0x7feb6c1528d0 2024-09-16T00:09:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f9966ffd640 1 -- 172.21.15.139:0/3641262673 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99680c8c50 msgr2=0x7f99681e0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f9966ffd640 1 --2- 172.21.15.139:0/3641262673 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99680c8c50 0x7f99681e0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f9966ffd640 1 -- 172.21.15.139:0/3641262673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99681587b0 msgr2=0x7f99681f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f9966ffd640 1 --2- 172.21.15.139:0/3641262673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99681587b0 0x7f99681f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f9966ffd640 1 -- 172.21.15.139:0/3641262673 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99681eec00 con 0x7f9968152b20 2024-09-16T00:09:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f99677fe640 1 --2- 172.21.15.139:0/3641262673 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99680c8c50 0x7f99681e0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.240+0000 7f9966ffd640 1 --2- 172.21.15.139:0/3641262673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9968152b20 0x7f99681e3db0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c430 tx=0x7f995c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.241+0000 7f994ffff640 1 -- 172.21.15.139:0/3641262673 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c017070 con 0x7f9968152b20 2024-09-16T00:09:17.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.241+0000 7f994ffff640 1 -- 172.21.15.139:0/3641262673 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995c005ce0 con 0x7f9968152b20 2024-09-16T00:09:17.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.241+0000 7f996e128640 1 -- 172.21.15.139:0/3641262673 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99681eeec0 con 0x7f9968152b20 2024-09-16T00:09:17.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.241+0000 7f994ffff640 1 -- 172.21.15.139:0/3641262673 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c007310 con 0x7f9968152b20 2024-09-16T00:09:17.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.241+0000 7f996e128640 1 -- 172.21.15.139:0/3641262673 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99681ef480 con 0x7f9968152b20 2024-09-16T00:09:17.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.241+0000 7feb71b66640 1 -- 172.21.15.139:0/2615504394 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c0ca4e0 con 0x7feb6c1528d0 2024-09-16T00:09:17.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.242+0000 7feb53fff640 1 -- 172.21.15.139:0/2615504394 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7feb64039040 con 0x7feb6c1528d0 2024-09-16T00:09:17.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.243+0000 7feb53fff640 1 --2- 172.21.15.139:0/2615504394 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7feb5806c300 0x7feb5806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.243+0000 7feb53fff640 1 -- 172.21.15.139:0/2615504394 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5944+0+0 (secure 0 0 0) 0x7feb640ac150 con 0x7feb6c1528d0 2024-09-16T00:09:17.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.243+0000 7f996e128640 1 -- 172.21.15.139:0/3641262673 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99680cccd0 con 0x7f9968152b20 2024-09-16T00:09:17.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.243+0000 7feb6b7fe640 1 --2- 172.21.15.139:0/2615504394 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7feb5806c300 0x7feb5806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.243+0000 7f994ffff640 1 -- 172.21.15.139:0/3641262673 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f995c01a070 con 0x7f9968152b20 2024-09-16T00:09:17.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.244+0000 7f994ffff640 1 --2- 172.21.15.139:0/3641262673 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f995406c2b0 0x7f995406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.244+0000 7feb6b7fe640 1 --2- 172.21.15.139:0/2615504394 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7feb5806c300 0x7feb5806e7c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7feb5c000f60 tx=0x7feb5c007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.244+0000 7f994ffff640 1 -- 172.21.15.139:0/3641262673 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5944+0+0 (secure 0 0 0) 0x7f995c08a750 con 0x7f9968152b20 2024-09-16T00:09:17.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.244+0000 7f99677fe640 1 --2- 172.21.15.139:0/3641262673 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f995406c2b0 0x7f995406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.244+0000 7f99677fe640 1 --2- 172.21.15.139:0/3641262673 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f995406c2b0 0x7f995406e770 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9958034000 tx=0x7f995802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.247+0000 7feb53fff640 1 -- 172.21.15.139:0/2615504394 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feb6407a2f0 con 0x7feb6c1528d0 2024-09-16T00:09:17.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.255+0000 7f994ffff640 1 -- 172.21.15.139:0/3641262673 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f99680cccd0 con 0x7f9968152b20 2024-09-16T00:09:17.304 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:17.280+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:17.382 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:17.381+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:17.391 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:17.391 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:17.391 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:17.391 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:17.400 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:09:17.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.400+0000 7f1255b7f640 1 Processor -- start 2024-09-16T00:09:17.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.400+0000 7f1255b7f640 1 -- start start 2024-09-16T00:09:17.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.401+0000 7f1255b7f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12500d0870 0x7f12500b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.401+0000 7f1255b7f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f12500b3430 0x7f12500b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.401+0000 7f1255b7f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f125015adc0 con 0x7f12500d0490 2024-09-16T00:09:17.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.401+0000 7f1255b7f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f125015b940 con 0x7f12500b3430 2024-09-16T00:09:17.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.401+0000 7f1255b7f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f12500b3bb0 con 0x7f12500d0870 2024-09-16T00:09:17.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.401+0000 7f124ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f12500b3430 0x7f12500b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.401+0000 7f124f7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f12500d0490 0x7f12500b2c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41600/0 (socket says 172.21.15.139:41600) 2024-09-16T00:09:17.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.401+0000 7f124f7fe640 1 -- 172.21.15.139:0/4239739445 learned_addr learned my addr 172.21.15.139:0/4239739445 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:17.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.401+0000 7f124effd640 1 --2- 172.21.15.139:0/4239739445 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12500d0870 0x7f12500b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.401+0000 7f124ffff640 1 -- 172.21.15.139:0/4239739445 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12500d0870 msgr2=0x7f12500b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:17.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.401+0000 7f124ffff640 1 --2- 172.21.15.139:0/4239739445 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12500d0870 0x7f12500b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.401+0000 7f124ffff640 1 -- 172.21.15.139:0/4239739445 >> v1:172.21.15.139:6789/0 conn(0x7f12500d0490 legacy=0x7f12500b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.401+0000 7f124e7fc640 1 -- 172.21.15.139:0/4239739445 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 342519910 0 0) 0x7f125015adc0 con 0x7f12500d0490 2024-09-16T00:09:17.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.401+0000 7f124ffff640 1 -- 172.21.15.139:0/4239739445 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125015b0a0 con 0x7f12500b3430 2024-09-16T00:09:17.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.402+0000 7f124ffff640 1 --2- 172.21.15.139:0/4239739445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12500b3430 0x7f12500b4580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1248009dc0 tx=0x7f124802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24461fabde7a9294 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.403+0000 7f124e7fc640 1 -- 172.21.15.139:0/4239739445 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248007740 con 0x7f12500b3430 2024-09-16T00:09:17.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.403+0000 7f124e7fc640 1 -- 172.21.15.139:0/4239739445 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12480078e0 con 0x7f12500b3430 2024-09-16T00:09:17.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.403+0000 7f124e7fc640 1 -- 172.21.15.139:0/4239739445 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248007cb0 con 0x7f12500b3430 2024-09-16T00:09:17.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.403+0000 7f1255b7f640 1 -- 172.21.15.139:0/4239739445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12500b3430 msgr2=0x7f12500b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.403+0000 7f1255b7f640 1 --2- 172.21.15.139:0/4239739445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12500b3430 0x7f12500b4580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1248009dc0 tx=0x7f124802a710 comp rx=0 tx=0).stop 2024-09-16T00:09:17.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.403+0000 7f1255b7f640 1 -- 172.21.15.139:0/4239739445 shutdown_connections 2024-09-16T00:09:17.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.403+0000 7f1255b7f640 1 --2- 172.21.15.139:0/4239739445 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12500b3430 0x7f12500b4580 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.403+0000 7f1255b7f640 1 --2- 172.21.15.139:0/4239739445 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12500d0870 0x7f12500b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.403+0000 7f1255b7f640 1 -- 172.21.15.139:0/4239739445 >> 172.21.15.139:0/4239739445 conn(0x7f125006c520 msgr2=0x7f125006c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:17.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.403+0000 7f1255b7f640 1 -- 172.21.15.139:0/4239739445 shutdown_connections 2024-09-16T00:09:17.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.403+0000 7f1255b7f640 1 -- 172.21.15.139:0/4239739445 wait complete. 2024-09-16T00:09:17.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.404+0000 7f1255b7f640 1 Processor -- start 2024-09-16T00:09:17.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.405+0000 7f1255b7f640 1 -- start start 2024-09-16T00:09:17.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.405+0000 7f1255b7f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12500b3430 0x7f12501504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.405+0000 7f1255b7f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12500d0490 0x7f125014ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.405+0000 7f1255b7f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12500d0870 0x7f12500c2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.405+0000 7f1255b7f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f125015a980 con 0x7f12500d0870 2024-09-16T00:09:17.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.405+0000 7f1255b7f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f125015b660 con 0x7f12500b3430 2024-09-16T00:09:17.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.405+0000 7f1255b7f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f125015ab00 con 0x7f12500d0490 2024-09-16T00:09:17.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.405+0000 7f124ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12500d0870 0x7f12500c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.405+0000 7f124effd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12500d0490 0x7f125014ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.405+0000 7f124ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12500d0870 0x7f12500c2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53904/0 (socket says 172.21.15.139:53904) 2024-09-16T00:09:17.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.405+0000 7f124effd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12500d0490 0x7f125014ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:49156/0 (socket says 172.21.15.139:49156) 2024-09-16T00:09:17.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.405+0000 7f124f7fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12500b3430 0x7f12501504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.405+0000 7f124ffff640 1 -- 172.21.15.139:0/1850598032 learned_addr learned my addr 172.21.15.139:0/1850598032 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:17.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.406+0000 7f124ffff640 1 -- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12500d0490 msgr2=0x7f125014ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.406+0000 7f124ffff640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12500d0490 0x7f125014ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.406+0000 7f124ffff640 1 -- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12500b3430 msgr2=0x7f12501504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.406+0000 7f124ffff640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12500b3430 0x7f12501504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.406+0000 7f124ffff640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125014f7c0 con 0x7f12500d0870 2024-09-16T00:09:17.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.406+0000 7f124f7fe640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12500b3430 0x7f12501504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:17.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.406+0000 7f124effd640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12500d0490 0x7f125014ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:17.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.406+0000 7f124ffff640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12500d0870 0x7f12500c2750 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f124802ad00 tx=0x7f1248004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.406+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124803f070 con 0x7f12500d0870 2024-09-16T00:09:17.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.406+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1248031c40 con 0x7f12500d0870 2024-09-16T00:09:17.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.406+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248009450 con 0x7f12500d0870 2024-09-16T00:09:17.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.406+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12501f0510 con 0x7f12500d0870 2024-09-16T00:09:17.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.406+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12501f1a40 con 0x7f12500d0870 2024-09-16T00:09:17.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.407+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f12500b2c40 con 0x7f12500d0870 2024-09-16T00:09:17.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.408+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1248003990 con 0x7f12500d0870 2024-09-16T00:09:17.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.408+0000 7f1237fff640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f123c06c2b0 0x7f123c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.408+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5944+0+0 (secure 0 0 0) 0x7f12480ac560 con 0x7f12500d0870 2024-09-16T00:09:17.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.409+0000 7f124f7fe640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f123c06c2b0 0x7f123c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.409+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v70) ==== 54+0+16 (secure 0 0 0) 0x7f12480aebe0 con 0x7f12500d0870 2024-09-16T00:09:17.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.409+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f12501f1db0 0x7f1250151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.409+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f12501f02c0 con 0x7f12501f1db0 2024-09-16T00:09:17.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.409+0000 7f124effd640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f12501f1db0 0x7f1250151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.409+0000 7f124f7fe640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f123c06c2b0 0x7f123c06e770 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1240000f60 tx=0x7f1240007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.409+0000 7f124effd640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f12501f1db0 0x7f1250151080 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.410+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f12501f02c0 con 0x7f12501f1db0 2024-09-16T00:09:17.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.435+0000 7feb71b66640 1 -- 172.21.15.139:0/2615504394 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7feb6c0c73e0 con 0x7feb6c1528d0 2024-09-16T00:09:17.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.442+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f12500cbce0 con 0x7f12501f1db0 2024-09-16T00:09:17.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.444+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f12500cbce0 con 0x7f12501f1db0 2024-09-16T00:09:17.445 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:17.446 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:17.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.445+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f12501f2750 0x7f12501f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.445+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f12500cba00 con 0x7f12501f2750 2024-09-16T00:09:17.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.445+0000 7f124f7fe640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f12501f2750 0x7f12501f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.445+0000 7f124f7fe640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f12501f2750 0x7f12501f3780 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.447 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.446+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f12500cba00 con 0x7f12501f2750 2024-09-16T00:09:17.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.475+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f12500cc840 con 0x7f12501f2750 2024-09-16T00:09:17.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.477+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f12500cc840 con 0x7f12501f2750 2024-09-16T00:09:17.478 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:17.478 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:17.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.477+0000 7f996e128640 1 -- 172.21.15.139:0/3641262673 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) -- 0x7f9968068530 con 0x7f9968152b20 2024-09-16T00:09:17.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.477+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f12501f41d0 0x7f1250062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.477+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f12501f45f0 con 0x7f12501f41d0 2024-09-16T00:09:17.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.477+0000 7f124effd640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f12501f41d0 0x7f1250062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.478+0000 7f124effd640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f12501f41d0 0x7f1250062630 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.478+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f12501f45f0 con 0x7f12501f41d0 2024-09-16T00:09:17.482 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:17.482+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:17.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.492+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f125014f360 con 0x7f12501f41d0 2024-09-16T00:09:17.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.493+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f125014f360 con 0x7f12501f41d0 2024-09-16T00:09:17.494 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:17.495 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:17.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.493+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f1250145630 0x7f1250145a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.493+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f125014f570 con 0x7f1250145630 2024-09-16T00:09:17.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.493+0000 7f124ffff640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f1250145630 0x7f1250145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.494+0000 7f124ffff640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f1250145630 0x7f1250145a50 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.494+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f125014f570 con 0x7f1250145630 2024-09-16T00:09:17.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.507+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f125014f570 con 0x7f1250145630 2024-09-16T00:09:17.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.509+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f125014f570 con 0x7f1250145630 2024-09-16T00:09:17.510 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:17.511 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:17.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.509+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f125002c640 0x7f125002ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.509+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f12501464a0 con 0x7f125002c640 2024-09-16T00:09:17.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.510+0000 7f124f7fe640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f125002c640 0x7f125002ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.510+0000 7f124f7fe640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f125002c640 0x7f125002ca80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.511+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f12501464a0 con 0x7f125002c640 2024-09-16T00:09:17.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.524+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f12501464a0 con 0x7f125002c640 2024-09-16T00:09:17.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.526+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f12501464a0 con 0x7f125002c640 2024-09-16T00:09:17.527 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:17.527 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:17.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.526+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f125002d4d0 0x7f1250206b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.526+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f125002d930 con 0x7f125002d4d0 2024-09-16T00:09:17.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.526+0000 7f124effd640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f125002d4d0 0x7f1250206b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.526+0000 7f124effd640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f125002d4d0 0x7f1250206b50 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.527+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f125002d930 con 0x7f125002d4d0 2024-09-16T00:09:17.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.540+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f125002db70 con 0x7f125002d4d0 2024-09-16T00:09:17.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.542+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f125002db70 con 0x7f125002d4d0 2024-09-16T00:09:17.543 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:17.544 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:17.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.543+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f12502075a0 0x7f1250209990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.543+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f125020c8e0 con 0x7f12502075a0 2024-09-16T00:09:17.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.543+0000 7f124ffff640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f12502075a0 0x7f1250209990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.543+0000 7f124ffff640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f12502075a0 0x7f1250209990 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.544+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f125020c8e0 con 0x7f12502075a0 2024-09-16T00:09:17.548 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:17.548 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:17.548 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:17.548 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:17.548 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:17.557 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:09:17.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.557+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f12501f2b50 con 0x7f12502075a0 2024-09-16T00:09:17.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.559+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f12501f2b50 con 0x7f12502075a0 2024-09-16T00:09:17.560 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:17.561 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:09:17.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.559+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f125020a430 0x7f125020a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:17.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.559+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f125020fcb0 con 0x7f125020a430 2024-09-16T00:09:17.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.559+0000 7f124f7fe640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f125020a430 0x7f125020a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:17.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.559+0000 7f124f7fe640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f125020a430 0x7f125020a8b0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:17.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.560+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f125020fcb0 con 0x7f125020a430 2024-09-16T00:09:17.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.573+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f125020af00 con 0x7f125020a430 2024-09-16T00:09:17.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1237fff640 1 -- 172.21.15.139:0/1850598032 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f125020af00 con 0x7f125020a430 2024-09-16T00:09:17.577 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:17.578 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:17.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f12501f1db0 msgr2=0x7f1250151080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f12501f1db0 0x7f1250151080 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f12501f2750 msgr2=0x7f12501f3780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f12501f2750 0x7f12501f3780 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f12501f41d0 msgr2=0x7f1250062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f12501f41d0 0x7f1250062630 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f1250145630 msgr2=0x7f1250145a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f1250145630 0x7f1250145a50 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f125002c640 msgr2=0x7f125002ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f125002c640 0x7f125002ca80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f124ffff640 1 -- 172.21.15.139:0/1850598032 reap_dead start 2024-09-16T00:09:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f125002d4d0 msgr2=0x7f1250206b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f125002d4d0 0x7f1250206b50 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f124ffff640 1 -- 172.21.15.139:0/1850598032 reap_dead start 2024-09-16T00:09:17.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f124ffff640 1 -- 172.21.15.139:0/1850598032 reap_dead start 2024-09-16T00:09:17.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f12502075a0 msgr2=0x7f1250209990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f12502075a0 0x7f1250209990 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f125020a430 msgr2=0x7f125020a8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f125020a430 0x7f125020a8b0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f123c06c2b0 msgr2=0x7f123c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f123c06c2b0 0x7f123c06e770 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1240000f60 tx=0x7f1240007d50 comp rx=0 tx=0).stop 2024-09-16T00:09:17.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12500d0870 msgr2=0x7f12500c2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:17.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12500d0870 0x7f12500c2750 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f124802ad00 tx=0x7f1248004a40 comp rx=0 tx=0).stop 2024-09-16T00:09:17.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 shutdown_connections 2024-09-16T00:09:17.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f12502075a0 0x7f1250209990 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f125020a430 0x7f125020a8b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f123c06c2b0 0x7f123c06e770 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.576+0000 7f1255b7f640 1 --2- 172.21.15.139:0/1850598032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12500d0870 0x7f12500c2750 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:17.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.577+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 >> 172.21.15.139:0/1850598032 conn(0x7f125006c520 msgr2=0x7f12500b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:17.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.577+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 shutdown_connections 2024-09-16T00:09:17.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:17.577+0000 7f1255b7f640 1 -- 172.21.15.139:0/1850598032 wait complete. 2024-09-16T00:09:17.601 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:17.600+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:17.664 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:17.664 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:17.673 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:09:17.718 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:17.717+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:17.777 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:17.777 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:17.786 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:09:17.787 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:17.784+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:17.886 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:17.885+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:17.930 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:17.931 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:17.939 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:09:17.988 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:17.987+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:18.041 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:18.042 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:18.050 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:09:18.089 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:18.087+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:18.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.090+0000 7feb53fff640 1 -- 172.21.15.139:0/2615504394 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v71) ==== 87+0+0 (secure 0 0 0) 0x7feb6c0c73e0 con 0x7feb6c1528d0 2024-09-16T00:09:18.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.090+0000 7f994ffff640 1 -- 172.21.15.139:0/3641262673 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v71) ==== 113+0+0 (secure 0 0 0) 0x7f9968068530 con 0x7f9968152b20 2024-09-16T00:09:18.091 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is set 2024-09-16T00:09:18.092 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T00:09:18.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7f996e128640 1 -- 172.21.15.139:0/3641262673 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f995406c2b0 msgr2=0x7f995406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:18.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7f996e128640 1 --2- 172.21.15.139:0/3641262673 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f995406c2b0 0x7f995406e770 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9958034000 tx=0x7f995802e000 comp rx=0 tx=0).stop 2024-09-16T00:09:18.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7f996e128640 1 -- 172.21.15.139:0/3641262673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9968152b20 msgr2=0x7f99681e3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:18.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7f996e128640 1 --2- 172.21.15.139:0/3641262673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9968152b20 0x7f99681e3db0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f995c00c430 tx=0x7f995c00c830 comp rx=0 tx=0).stop 2024-09-16T00:09:18.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7f996e128640 1 -- 172.21.15.139:0/3641262673 shutdown_connections 2024-09-16T00:09:18.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7f996e128640 1 --2- 172.21.15.139:0/3641262673 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f995406c2b0 0x7f995406e770 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7f996e128640 1 --2- 172.21.15.139:0/3641262673 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99681587b0 0x7f99681f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7f996e128640 1 --2- 172.21.15.139:0/3641262673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9968152b20 0x7f99681e3db0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7f996e128640 1 --2- 172.21.15.139:0/3641262673 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99680c8c50 0x7f99681e0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7f996e128640 1 -- 172.21.15.139:0/3641262673 >> 172.21.15.139:0/3641262673 conn(0x7f996806bd00 msgr2=0x7f9968158c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:18.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7f996e128640 1 -- 172.21.15.139:0/3641262673 shutdown_connections 2024-09-16T00:09:18.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7f996e128640 1 -- 172.21.15.139:0/3641262673 wait complete. 2024-09-16T00:09:18.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7feb71b66640 1 -- 172.21.15.139:0/2615504394 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7feb5806c300 msgr2=0x7feb5806e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:18.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7feb71b66640 1 --2- 172.21.15.139:0/2615504394 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7feb5806c300 0x7feb5806e7c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7feb5c000f60 tx=0x7feb5c007d50 comp rx=0 tx=0).stop 2024-09-16T00:09:18.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7feb71b66640 1 -- 172.21.15.139:0/2615504394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6c1528d0 msgr2=0x7feb6c1f2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:18.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7feb71b66640 1 --2- 172.21.15.139:0/2615504394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6c1528d0 0x7feb6c1f2440 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7feb64009eb0 tx=0x7feb64004a40 comp rx=0 tx=0).stop 2024-09-16T00:09:18.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7feb71b66640 1 -- 172.21.15.139:0/2615504394 shutdown_connections 2024-09-16T00:09:18.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7feb71b66640 1 --2- 172.21.15.139:0/2615504394 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7feb5806c300 0x7feb5806e7c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7feb71b66640 1 --2- 172.21.15.139:0/2615504394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6c1528d0 0x7feb6c1f2440 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7feb71b66640 1 --2- 172.21.15.139:0/2615504394 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feb6c1524f0 0x7feb6c151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7feb71b66640 1 --2- 172.21.15.139:0/2615504394 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feb6c072f00 0x7feb6c0cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.093+0000 7feb71b66640 1 -- 172.21.15.139:0/2615504394 >> 172.21.15.139:0/2615504394 conn(0x7feb6c06da40 msgr2=0x7feb6c158580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:18.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.094+0000 7feb71b66640 1 -- 172.21.15.139:0/2615504394 shutdown_connections 2024-09-16T00:09:18.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.094+0000 7feb71b66640 1 -- 172.21.15.139:0/2615504394 wait complete. 2024-09-16T00:09:18.108 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T00:09:18.192 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:18.192+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:18.204 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:18.204 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:18.213 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:09:18.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.252+0000 7f4139045640 1 Processor -- start 2024-09-16T00:09:18.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.252+0000 7f4139045640 1 -- start start 2024-09-16T00:09:18.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.253+0000 7f4139045640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4134150fb0 0x7f41340718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:18.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.253+0000 7f4139045640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4134073230 con 0x7f4134150bd0 2024-09-16T00:09:18.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.253+0000 7f4139045640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4134072020 con 0x7f413415afc0 2024-09-16T00:09:18.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.253+0000 7f4139045640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4134071df0 con 0x7f4134150fb0 2024-09-16T00:09:18.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.253+0000 7f413259c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4134150fb0 0x7f41340718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:18.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.253+0000 7f413259c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4134150fb0 0x7f41340718b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53908/0 (socket says 172.21.15.139:53908) 2024-09-16T00:09:18.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.253+0000 7f4132d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4134150bd0 0x7f4134157790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41614/0 (socket says 172.21.15.139:41614) 2024-09-16T00:09:18.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.253+0000 7f413259c640 1 -- 172.21.15.139:0/754563582 learned_addr learned my addr 172.21.15.139:0/754563582 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:18.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.253+0000 7f413259c640 1 -- 172.21.15.139:0/754563582 >> v1:172.21.15.164:6789/0 conn(0x7f413415afc0 legacy=0x7f41340c4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:18.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.253+0000 7f413259c640 1 -- 172.21.15.139:0/754563582 >> v1:172.21.15.139:6789/0 conn(0x7f4134150bd0 legacy=0x7f4134157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:18.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.253+0000 7f413259c640 1 -- 172.21.15.139:0/754563582 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41340722c0 con 0x7f4134150fb0 2024-09-16T00:09:18.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.254+0000 7f413259c640 1 --2- 172.21.15.139:0/754563582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4134150fb0 0x7f41340718b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4128009870 tx=0x7f412802b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743354abdb2c3d28 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:18.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.254+0000 7f4131d9b640 1 -- 172.21.15.139:0/754563582 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128037070 con 0x7f4134150fb0 2024-09-16T00:09:18.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.254+0000 7f4131d9b640 1 -- 172.21.15.139:0/754563582 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4128030710 con 0x7f4134150fb0 2024-09-16T00:09:18.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.254+0000 7f4131d9b640 1 -- 172.21.15.139:0/754563582 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128030ae0 con 0x7f4134150fb0 2024-09-16T00:09:18.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.255+0000 7f4139045640 1 -- 172.21.15.139:0/754563582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4134150fb0 msgr2=0x7f41340718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:18.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.255+0000 7f4139045640 1 --2- 172.21.15.139:0/754563582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4134150fb0 0x7f41340718b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4128009870 tx=0x7f412802b5e0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.255+0000 7f4139045640 1 -- 172.21.15.139:0/754563582 shutdown_connections 2024-09-16T00:09:18.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.255+0000 7f4139045640 1 --2- 172.21.15.139:0/754563582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4134150fb0 0x7f41340718b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.255+0000 7f4139045640 1 -- 172.21.15.139:0/754563582 >> 172.21.15.139:0/754563582 conn(0x7f413406da40 msgr2=0x7f413406f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:18.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.255+0000 7f4139045640 1 -- 172.21.15.139:0/754563582 shutdown_connections 2024-09-16T00:09:18.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.255+0000 7f4139045640 1 -- 172.21.15.139:0/754563582 wait complete. 2024-09-16T00:09:18.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.256+0000 7f4139045640 1 Processor -- start 2024-09-16T00:09:18.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.256+0000 7f4139045640 1 -- start start 2024-09-16T00:09:18.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.257+0000 7f4139045640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4134150bd0 0x7f41340bc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:18.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.257+0000 7f4139045640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4134150fb0 0x7f413414de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:18.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.257+0000 7f4139045640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f413415afc0 0x7f41340baf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:18.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.257+0000 7f4139045640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f413415a7c0 con 0x7f413415afc0 2024-09-16T00:09:18.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.257+0000 7f4139045640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f413415ab30 con 0x7f4134150fb0 2024-09-16T00:09:18.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.257+0000 7f4139045640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f413406fb70 con 0x7f4134150bd0 2024-09-16T00:09:18.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.257+0000 7f413359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f413415afc0 0x7f41340baf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:18.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.257+0000 7f413259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4134150fb0 0x7f413414de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:18.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.257+0000 7f413359e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f413415afc0 0x7f41340baf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53916/0 (socket says 172.21.15.139:53916) 2024-09-16T00:09:18.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.257+0000 7f413359e640 1 -- 172.21.15.139:0/4161168656 learned_addr learned my addr 172.21.15.139:0/4161168656 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:18.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.257+0000 7f4132d9d640 1 --2- 172.21.15.139:0/4161168656 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4134150bd0 0x7f41340bc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:18.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.257+0000 7f413359e640 1 -- 172.21.15.139:0/4161168656 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4134150bd0 msgr2=0x7f41340bc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:18.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.257+0000 7f413359e640 1 --2- 172.21.15.139:0/4161168656 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4134150bd0 0x7f41340bc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.257+0000 7f413359e640 1 -- 172.21.15.139:0/4161168656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4134150fb0 msgr2=0x7f413414de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:18.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.257+0000 7f413359e640 1 --2- 172.21.15.139:0/4161168656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4134150fb0 0x7f413414de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:18.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.257+0000 7f413359e640 1 -- 172.21.15.139:0/4161168656 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41340b8eb0 con 0x7f413415afc0 2024-09-16T00:09:18.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.258+0000 7f413359e640 1 --2- 172.21.15.139:0/4161168656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f413415afc0 0x7f41340baf60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f412c00c730 tx=0x7f412c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:18.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.258+0000 7f411b7fe640 1 -- 172.21.15.139:0/4161168656 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c003b90 con 0x7f413415afc0 2024-09-16T00:09:18.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.258+0000 7f411b7fe640 1 -- 172.21.15.139:0/4161168656 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f412c003d30 con 0x7f413415afc0 2024-09-16T00:09:18.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.258+0000 7f4139045640 1 -- 172.21.15.139:0/4161168656 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41340ba7a0 con 0x7f413415afc0 2024-09-16T00:09:18.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.258+0000 7f411b7fe640 1 -- 172.21.15.139:0/4161168656 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c005310 con 0x7f413415afc0 2024-09-16T00:09:18.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.258+0000 7f4139045640 1 -- 172.21.15.139:0/4161168656 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41340b9150 con 0x7f413415afc0 2024-09-16T00:09:18.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.259+0000 7f4139045640 1 -- 172.21.15.139:0/4161168656 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f413406ff00 con 0x7f413415afc0 2024-09-16T00:09:18.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.259+0000 7f411b7fe640 1 -- 172.21.15.139:0/4161168656 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f412c0056c0 con 0x7f413415afc0 2024-09-16T00:09:18.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.260+0000 7f411b7fe640 1 --2- 172.21.15.139:0/4161168656 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f412006c2e0 0x7f412006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:18.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.260+0000 7f411b7fe640 1 -- 172.21.15.139:0/4161168656 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5944+0+0 (secure 0 0 0) 0x7f412c08b440 con 0x7f413415afc0 2024-09-16T00:09:18.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.260+0000 7f4132d9d640 1 --2- 172.21.15.139:0/4161168656 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f412006c2e0 0x7f412006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:18.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.260+0000 7f4132d9d640 1 --2- 172.21.15.139:0/4161168656 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f412006c2e0 0x7f412006e7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4124007fd0 tx=0x7f4124007e00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:18.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.264+0000 7f411b7fe640 1 -- 172.21.15.139:0/4161168656 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f412c059470 con 0x7f413415afc0 2024-09-16T00:09:18.291 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:18.290+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:18.358 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:18.358 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:18.367 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:09:18.408 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:18.407+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:18.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:18.448+0000 7f4139045640 1 -- 172.21.15.139:0/4161168656 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) -- 0x7f41340b9390 con 0x7f413415afc0 2024-09-16T00:09:18.493 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:18.492+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:18.511 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:18.511 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:18.520 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:09:18.594 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:18.592+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:18.670 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:18.670 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:18.680 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:09:18.695 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:18.694+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:18.795 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:18.796 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:18.796 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:18.795+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:18.805 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:09:18.897 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:18.896+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:18.965 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:18.966 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:18.975 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:09:18.998 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:18.997+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:19.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.091+0000 7f411b7fe640 1 -- 172.21.15.139:0/4161168656 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v72) ==== 113+0+0 (secure 0 0 0) 0x7f412c05d770 con 0x7f413415afc0 2024-09-16T00:09:19.093 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T00:09:19.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.094+0000 7f4139045640 1 -- 172.21.15.139:0/4161168656 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f412006c2e0 msgr2=0x7f412006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.094+0000 7f4139045640 1 --2- 172.21.15.139:0/4161168656 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f412006c2e0 0x7f412006e7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4124007fd0 tx=0x7f4124007e00 comp rx=0 tx=0).stop 2024-09-16T00:09:19.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.094+0000 7f4139045640 1 -- 172.21.15.139:0/4161168656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f413415afc0 msgr2=0x7f41340baf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.094+0000 7f4139045640 1 --2- 172.21.15.139:0/4161168656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f413415afc0 0x7f41340baf60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f412c00c730 tx=0x7f412c00cbf0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.095+0000 7f4139045640 1 -- 172.21.15.139:0/4161168656 shutdown_connections 2024-09-16T00:09:19.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.095+0000 7f4139045640 1 --2- 172.21.15.139:0/4161168656 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f412006c2e0 0x7f412006e7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.095+0000 7f4139045640 1 --2- 172.21.15.139:0/4161168656 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f413415afc0 0x7f41340baf60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.095+0000 7f4139045640 1 --2- 172.21.15.139:0/4161168656 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4134150fb0 0x7f413414de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.095+0000 7f4139045640 1 --2- 172.21.15.139:0/4161168656 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4134150bd0 0x7f41340bc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.095+0000 7f4139045640 1 -- 172.21.15.139:0/4161168656 >> 172.21.15.139:0/4161168656 conn(0x7f413406da40 msgr2=0x7f413406f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:19.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.095+0000 7f4139045640 1 -- 172.21.15.139:0/4161168656 shutdown_connections 2024-09-16T00:09:19.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.095+0000 7f4139045640 1 -- 172.21.15.139:0/4161168656 wait complete. 2024-09-16T00:09:19.099 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:19.098+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:19.110 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T00:09:19.139 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:19.140 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:19.150 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:09:19.235 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:19.235+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:19.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.245+0000 7efe1d786640 1 Processor -- start 2024-09-16T00:09:19.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.245+0000 7efe1d786640 1 -- start start 2024-09-16T00:09:19.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.246+0000 7efe1d786640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efe180c9b00 0x7efe18154d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.246+0000 7efe1d786640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe1815b0d0 con 0x7efe1815de90 2024-09-16T00:09:19.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.246+0000 7efe1d786640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe1815b390 con 0x7efe18154960 2024-09-16T00:09:19.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.246+0000 7efe1d786640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7efe1815b670 con 0x7efe180c9b00 2024-09-16T00:09:19.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.246+0000 7efe167fc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7efe180c9b00 0x7efe18154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.246+0000 7efe16ffd640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7efe18154960 0x7efe1815a920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:56066/0 (socket says 172.21.15.139:56066) 2024-09-16T00:09:19.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.246+0000 7efe16ffd640 1 -- 172.21.15.139:0/2095993202 learned_addr learned my addr 172.21.15.139:0/2095993202 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:19.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.247+0000 7efe167fc640 1 -- 172.21.15.139:0/2095993202 >> v1:172.21.15.164:6789/0 conn(0x7efe18154960 legacy=0x7efe1815a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.247+0000 7efe167fc640 1 -- 172.21.15.139:0/2095993202 >> v1:172.21.15.139:6789/0 conn(0x7efe1815de90 legacy=0x7efe180cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.247+0000 7efe167fc640 1 -- 172.21.15.139:0/2095993202 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe180c7380 con 0x7efe180c9b00 2024-09-16T00:09:19.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.247+0000 7efe167fc640 1 --2- 172.21.15.139:0/2095993202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe180c9b00 0x7efe18154d40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0097d0 tx=0x7efe0c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d4930e2ab2d0cc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:19.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.248+0000 7efe15ffb640 1 -- 172.21.15.139:0/2095993202 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c037070 con 0x7efe180c9b00 2024-09-16T00:09:19.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.248+0000 7efe15ffb640 1 -- 172.21.15.139:0/2095993202 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe0c032440 con 0x7efe180c9b00 2024-09-16T00:09:19.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.248+0000 7efe15ffb640 1 -- 172.21.15.139:0/2095993202 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c032810 con 0x7efe180c9b00 2024-09-16T00:09:19.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.248+0000 7efe1d786640 1 -- 172.21.15.139:0/2095993202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe180c9b00 msgr2=0x7efe18154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.248+0000 7efe1d786640 1 --2- 172.21.15.139:0/2095993202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe180c9b00 0x7efe18154d40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efe0c0097d0 tx=0x7efe0c02a250 comp rx=0 tx=0).stop 2024-09-16T00:09:19.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.248+0000 7efe1d786640 1 -- 172.21.15.139:0/2095993202 shutdown_connections 2024-09-16T00:09:19.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.248+0000 7efe1d786640 1 --2- 172.21.15.139:0/2095993202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe180c9b00 0x7efe18154d40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.248+0000 7efe1d786640 1 -- 172.21.15.139:0/2095993202 >> 172.21.15.139:0/2095993202 conn(0x7efe1806bb20 msgr2=0x7efe1806bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:19.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.248+0000 7efe1d786640 1 -- 172.21.15.139:0/2095993202 shutdown_connections 2024-09-16T00:09:19.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.248+0000 7efe1d786640 1 -- 172.21.15.139:0/2095993202 wait complete. 2024-09-16T00:09:19.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.250+0000 7efe1d786640 1 Processor -- start 2024-09-16T00:09:19.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.250+0000 7efe1d786640 1 -- start start 2024-09-16T00:09:19.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.250+0000 7efe1d786640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe180c9b00 0x7efe18157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.250+0000 7efe1d786640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe18154960 0x7efe181e45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.250+0000 7efe1d786640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe1815de90 0x7efe181e4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.250+0000 7efe1d786640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efe180ccd10 con 0x7efe1815de90 2024-09-16T00:09:19.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.250+0000 7efe1d786640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe180ccb90 con 0x7efe180c9b00 2024-09-16T00:09:19.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.250+0000 7efe1d786640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7efe180c7b80 con 0x7efe18154960 2024-09-16T00:09:19.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efe16ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe180c9b00 0x7efe18157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efe177fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe1815de90 0x7efe181e4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efe16ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe180c9b00 0x7efe18157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:47580/0 (socket says 172.21.15.139:47580) 2024-09-16T00:09:19.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efe16ffd640 1 -- 172.21.15.139:0/819757937 learned_addr learned my addr 172.21.15.139:0/819757937 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:19.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efe167fc640 1 --2- 172.21.15.139:0/819757937 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe18154960 0x7efe181e45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efe16ffd640 1 -- 172.21.15.139:0/819757937 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe18154960 msgr2=0x7efe181e45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efe16ffd640 1 --2- 172.21.15.139:0/819757937 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe18154960 0x7efe181e45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efe16ffd640 1 -- 172.21.15.139:0/819757937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe1815de90 msgr2=0x7efe181e4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efe16ffd640 1 --2- 172.21.15.139:0/819757937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe1815de90 0x7efe181e4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efe16ffd640 1 -- 172.21.15.139:0/819757937 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe18153e40 con 0x7efe180c9b00 2024-09-16T00:09:19.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efe177fe640 1 --2- 172.21.15.139:0/819757937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe1815de90 0x7efe181e4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:19.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efe16ffd640 1 --2- 172.21.15.139:0/819757937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe180c9b00 0x7efe18157d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efe0800c4a0 tx=0x7efe0800c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:19.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efdff7fe640 1 -- 172.21.15.139:0/819757937 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08017070 con 0x7efe180c9b00 2024-09-16T00:09:19.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efdff7fe640 1 -- 172.21.15.139:0/819757937 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe08005ce0 con 0x7efe180c9b00 2024-09-16T00:09:19.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efdff7fe640 1 -- 172.21.15.139:0/819757937 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08007270 con 0x7efe180c9b00 2024-09-16T00:09:19.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efe1d786640 1 -- 172.21.15.139:0/819757937 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe180cb4d0 con 0x7efe180c9b00 2024-09-16T00:09:19.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.251+0000 7efe1d786640 1 -- 172.21.15.139:0/819757937 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe181e7a30 con 0x7efe180c9b00 2024-09-16T00:09:19.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.252+0000 7efe1d786640 1 -- 172.21.15.139:0/819757937 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe18154d40 con 0x7efe180c9b00 2024-09-16T00:09:19.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.253+0000 7efdff7fe640 1 -- 172.21.15.139:0/819757937 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efe0801a070 con 0x7efe180c9b00 2024-09-16T00:09:19.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.253+0000 7efdff7fe640 1 --2- 172.21.15.139:0/819757937 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efe0406c2e0 0x7efe0406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.253+0000 7efdff7fe640 1 -- 172.21.15.139:0/819757937 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5944+0+0 (secure 0 0 0) 0x7efe0808a670 con 0x7efe180c9b00 2024-09-16T00:09:19.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.253+0000 7efe167fc640 1 --2- 172.21.15.139:0/819757937 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efe0406c2e0 0x7efe0406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.254+0000 7efe167fc640 1 --2- 172.21.15.139:0/819757937 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efe0406c2e0 0x7efe0406e7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efe0c034000 tx=0x7efe0c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:19.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.258+0000 7efdff7fe640 1 -- 172.21.15.139:0/819757937 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efe08058730 con 0x7efe180c9b00 2024-09-16T00:09:19.301 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:19.300+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:19.301 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:19.301 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:19.310 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:09:19.401 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:19.400+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:19.475 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:19.476 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:19.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.477+0000 7efe1d786640 1 -- 172.21.15.139:0/819757937 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) -- 0x7efe180ca3e0 con 0x7efe180c9b00 2024-09-16T00:09:19.487 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:09:19.531 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:19.530+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:19.589 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:09:19.603 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:19.603 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:19.614 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:09:19.633 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:19.632+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:19.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.699+0000 7fb7270ba640 1 Processor -- start 2024-09-16T00:09:19.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.699+0000 7fb7270ba640 1 -- start start 2024-09-16T00:09:19.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.700+0000 7fb7270ba640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb72014f000 0x7fb72006bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.700+0000 7fb7270ba640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb7200c6700 0x7fb720159130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.700+0000 7fb7270ba640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb720067fb0 con 0x7fb720070180 2024-09-16T00:09:19.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.700+0000 7fb7270ba640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb72006dc40 con 0x7fb7200c6700 2024-09-16T00:09:19.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.700+0000 7fb7270ba640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb7200679f0 con 0x7fb72014f000 2024-09-16T00:09:19.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.700+0000 7fb725657640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb720070180 0x7fb7200c80e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41636/0 (socket says 172.21.15.139:41636) 2024-09-16T00:09:19.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.700+0000 7fb725657640 1 -- 172.21.15.139:0/3198275767 learned_addr learned my addr 172.21.15.139:0/3198275767 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:19.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.700+0000 7fb724e56640 1 --2- 172.21.15.139:0/3198275767 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb72014f000 0x7fb72006bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.700+0000 7fb71ffff640 1 --2- 172.21.15.139:0/3198275767 >> v2:172.21.15.164:3300/0 conn(0x7fb7200c6700 0x7fb720159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.700+0000 7fb71f7fe640 1 -- 172.21.15.139:0/3198275767 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1698926288 0 0) 0x7fb720067fb0 con 0x7fb720070180 2024-09-16T00:09:19.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.700+0000 7fb71f7fe640 1 -- 172.21.15.139:0/3198275767 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb710003520 con 0x7fb720070180 2024-09-16T00:09:19.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.701+0000 7fb724e56640 1 -- 172.21.15.139:0/3198275767 >> v2:172.21.15.164:3300/0 conn(0x7fb7200c6700 msgr2=0x7fb720159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:19.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.701+0000 7fb724e56640 1 --2- 172.21.15.139:0/3198275767 >> v2:172.21.15.164:3300/0 conn(0x7fb7200c6700 0x7fb720159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.701+0000 7fb724e56640 1 -- 172.21.15.139:0/3198275767 >> v1:172.21.15.139:6789/0 conn(0x7fb720070180 legacy=0x7fb7200c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.701+0000 7fb724e56640 1 -- 172.21.15.139:0/3198275767 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb720067cd0 con 0x7fb72014f000 2024-09-16T00:09:19.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.701+0000 7fb71ffff640 1 --2- 172.21.15.139:0/3198275767 >> v2:172.21.15.164:3300/0 conn(0x7fb7200c6700 0x7fb720159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:19.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.702+0000 7fb724e56640 1 --2- 172.21.15.139:0/3198275767 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb72014f000 0x7fb72006bb20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb7140097a0 tx=0x7fb71402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f342717d51548a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:19.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.702+0000 7fb71f7fe640 1 -- 172.21.15.139:0/3198275767 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714037070 con 0x7fb72014f000 2024-09-16T00:09:19.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.702+0000 7fb71f7fe640 1 -- 172.21.15.139:0/3198275767 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb714032440 con 0x7fb72014f000 2024-09-16T00:09:19.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.702+0000 7fb71f7fe640 1 -- 172.21.15.139:0/3198275767 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7140327f0 con 0x7fb72014f000 2024-09-16T00:09:19.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.702+0000 7fb7270ba640 1 -- 172.21.15.139:0/3198275767 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb72014f000 msgr2=0x7fb72006bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.702+0000 7fb7270ba640 1 --2- 172.21.15.139:0/3198275767 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb72014f000 0x7fb72006bb20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb7140097a0 tx=0x7fb71402a210 comp rx=0 tx=0).stop 2024-09-16T00:09:19.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.702+0000 7fb7270ba640 1 -- 172.21.15.139:0/3198275767 shutdown_connections 2024-09-16T00:09:19.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.702+0000 7fb7270ba640 1 --2- 172.21.15.139:0/3198275767 >> v2:172.21.15.164:3300/0 conn(0x7fb7200c6700 0x7fb720159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.702+0000 7fb7270ba640 1 --2- 172.21.15.139:0/3198275767 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb72014f000 0x7fb72006bb20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.702+0000 7fb7270ba640 1 -- 172.21.15.139:0/3198275767 >> 172.21.15.139:0/3198275767 conn(0x7fb72006f660 msgr2=0x7fb72006fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:19.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.702+0000 7fb7270ba640 1 -- 172.21.15.139:0/3198275767 shutdown_connections 2024-09-16T00:09:19.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.702+0000 7fb7270ba640 1 -- 172.21.15.139:0/3198275767 wait complete. 2024-09-16T00:09:19.705 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:19.702+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:19.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.704+0000 7fb7270ba640 1 Processor -- start 2024-09-16T00:09:19.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.704+0000 7fb7270ba640 1 -- start start 2024-09-16T00:09:19.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.705+0000 7fb7270ba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb720070180 0x7fb7200b8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.705+0000 7fb7270ba640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7200c6700 0x7fb7200729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.705+0000 7fb7270ba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72014f000 0x7fb720072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.705+0000 7fb7270ba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb720158c10 con 0x7fb72014f000 2024-09-16T00:09:19.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.705+0000 7fb7270ba640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb720158930 con 0x7fb720070180 2024-09-16T00:09:19.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.705+0000 7fb7270ba640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb72014f3e0 con 0x7fb7200c6700 2024-09-16T00:09:19.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.705+0000 7fb725657640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72014f000 0x7fb720072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.705+0000 7fb71ffff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7200c6700 0x7fb7200729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.705+0000 7fb724e56640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb720070180 0x7fb7200b8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.705+0000 7fb71ffff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7200c6700 0x7fb7200729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:49194/0 (socket says 172.21.15.139:49194) 2024-09-16T00:09:19.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.705+0000 7fb725657640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72014f000 0x7fb720072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53934/0 (socket says 172.21.15.139:53934) 2024-09-16T00:09:19.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.705+0000 7fb71ffff640 1 -- 172.21.15.139:0/2321308441 learned_addr learned my addr 172.21.15.139:0/2321308441 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:19.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.706+0000 7fb725657640 1 -- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7200c6700 msgr2=0x7fb7200729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.706+0000 7fb725657640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7200c6700 0x7fb7200729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.706+0000 7fb725657640 1 -- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb720070180 msgr2=0x7fb7200b8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.706+0000 7fb725657640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb720070180 0x7fb7200b8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.706+0000 7fb725657640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7201e58c0 con 0x7fb72014f000 2024-09-16T00:09:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.706+0000 7fb724e56640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb720070180 0x7fb7200b8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.706+0000 7fb71ffff640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7200c6700 0x7fb7200729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.706+0000 7fb725657640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72014f000 0x7fb720072f30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb71800caa0 tx=0x7fb71800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.706+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718017070 con 0x7fb72014f000 2024-09-16T00:09:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.706+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71800ab40 con 0x7fb72014f000 2024-09-16T00:09:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.706+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7201e5070 con 0x7fb72014f000 2024-09-16T00:09:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.706+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718012400 con 0x7fb72014f000 2024-09-16T00:09:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.706+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7201e3fc0 con 0x7fb72014f000 2024-09-16T00:09:19.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.707+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fb720070630 con 0x7fb72014f000 2024-09-16T00:09:19.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.708+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb71801a070 con 0x7fb72014f000 2024-09-16T00:09:19.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.708+0000 7fb71d7fa640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb70406c2b0 0x7fb70406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.708+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5944+0+0 (secure 0 0 0) 0x7fb71808b070 con 0x7fb72014f000 2024-09-16T00:09:19.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.708+0000 7fb724e56640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb70406c2b0 0x7fb70406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.709+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v72) ==== 54+0+16 (secure 0 0 0) 0x7fb71808d6f0 con 0x7fb72014f000 2024-09-16T00:09:19.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.709+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fb72014e0a0 0x7fb72014e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.709+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb72014e9c0 con 0x7fb72014e0a0 2024-09-16T00:09:19.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.709+0000 7fb724e56640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb70406c2b0 0x7fb70406e770 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb714034000 tx=0x7fb71402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:19.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.709+0000 7fb71ffff640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fb72014e0a0 0x7fb72014e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.709+0000 7fb71ffff640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fb72014e0a0 0x7fb72014e480 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:19.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.710+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb72014e9c0 con 0x7fb72014e0a0 2024-09-16T00:09:19.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.726+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb72015c940 con 0x7fb72014e0a0 2024-09-16T00:09:19.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.728+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb72015c940 con 0x7fb72014e0a0 2024-09-16T00:09:19.729 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:19.730 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:19.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.728+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fb7201f0db0 0x7fb7201f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.728+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fb7200c8350 con 0x7fb7201f0db0 2024-09-16T00:09:19.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.728+0000 7fb724e56640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fb7201f0db0 0x7fb7201f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.729+0000 7fb724e56640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fb7201f0db0 0x7fb7201f3ac0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:19.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.729+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb7200c8350 con 0x7fb7201f0db0 2024-09-16T00:09:19.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.748+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb7200c8070 con 0x7fb7201f0db0 2024-09-16T00:09:19.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.749+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb7200c8070 con 0x7fb7201f0db0 2024-09-16T00:09:19.750 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:19.751 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:19.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.749+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fb7201f4510 0x7fb720062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.749+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fb7200c7d90 con 0x7fb7201f4510 2024-09-16T00:09:19.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.750+0000 7fb71ffff640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fb7201f4510 0x7fb720062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.750+0000 7fb71ffff640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fb7201f4510 0x7fb720062630 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:19.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.751+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb7200c7d90 con 0x7fb7201f4510 2024-09-16T00:09:19.766 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:19.766 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:19.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.767+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb7200c7ab0 con 0x7fb7201f4510 2024-09-16T00:09:19.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.768+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb7200c7ab0 con 0x7fb7201f4510 2024-09-16T00:09:19.769 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:19.770 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:19.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.769+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fb7201e1970 0x7fb7201e1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.769+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fb720063440 con 0x7fb7201e1970 2024-09-16T00:09:19.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.769+0000 7fb725657640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fb7201e1970 0x7fb7201e1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.769+0000 7fb725657640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fb7201e1970 0x7fb7201e1df0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:19.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.770+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb720063440 con 0x7fb7201e1970 2024-09-16T00:09:19.777 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:09:19.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.785+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb7201e2600 con 0x7fb7201e1970 2024-09-16T00:09:19.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.787+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb7201e2600 con 0x7fb7201e1970 2024-09-16T00:09:19.788 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.787+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fb72002cb50 0x7fb72002cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.787+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fb7201e2810 con 0x7fb72002cb50 2024-09-16T00:09:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.787+0000 7fb724e56640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fb72002cb50 0x7fb72002cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.788+0000 7fb724e56640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fb72002cb50 0x7fb72002cf90 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:19.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.788+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb7201e2810 con 0x7fb72002cb50 2024-09-16T00:09:19.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.804+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb7201e2810 con 0x7fb72002cb50 2024-09-16T00:09:19.805 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:19.804+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:19.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.805+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb7201e2810 con 0x7fb72002cb50 2024-09-16T00:09:19.806 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:19.807 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:19.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.806+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fb720204e40 0x7fb720207230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.806+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fb7201e28e0 con 0x7fb720204e40 2024-09-16T00:09:19.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.806+0000 7fb71ffff640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fb720204e40 0x7fb720207230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.806+0000 7fb71ffff640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fb720204e40 0x7fb720207230 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:19.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.807+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb7201e28e0 con 0x7fb720204e40 2024-09-16T00:09:19.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.824+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb7201e28e0 con 0x7fb720204e40 2024-09-16T00:09:19.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.826+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb7201e28e0 con 0x7fb720204e40 2024-09-16T00:09:19.828 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:19.829 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:19.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.826+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fb720207c80 0x7fb72020a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.826+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fb72002d9e0 con 0x7fb720207c80 2024-09-16T00:09:19.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.826+0000 7fb725657640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fb720207c80 0x7fb72020a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.827+0000 7fb725657640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fb720207c80 0x7fb72020a070 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:19.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.828+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb72002d9e0 con 0x7fb720207c80 2024-09-16T00:09:19.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.843+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb72002d9e0 con 0x7fb720207c80 2024-09-16T00:09:19.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.845+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb72002d9e0 con 0x7fb720207c80 2024-09-16T00:09:19.846 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:09:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.845+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fb72020aac0 0x7fb72020ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.845+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fb72020fc80 con 0x7fb72020aac0 2024-09-16T00:09:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.846+0000 7fb724e56640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fb72020aac0 0x7fb72020ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.846+0000 7fb724e56640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fb72020aac0 0x7fb72020ceb0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:19.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.847+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb72020fc80 con 0x7fb72020aac0 2024-09-16T00:09:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.861+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb72020d3f0 con 0x7fb72020aac0 2024-09-16T00:09:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb71d7fa640 1 -- 172.21.15.139:0/2321308441 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb72020d3f0 con 0x7fb72020aac0 2024-09-16T00:09:19.864 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:19.865 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:19.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fb72014e0a0 msgr2=0x7fb72014e480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fb72014e0a0 0x7fb72014e480 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fb7201f0db0 msgr2=0x7fb7201f3ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fb7201f0db0 0x7fb7201f3ac0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fb7201f4510 msgr2=0x7fb720062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fb7201f4510 0x7fb720062630 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fb7201e1970 msgr2=0x7fb7201e1df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fb7201e1970 0x7fb7201e1df0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fb72002cb50 msgr2=0x7fb72002cf90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fb72002cb50 0x7fb72002cf90 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb725657640 1 -- 172.21.15.139:0/2321308441 reap_dead start 2024-09-16T00:09:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb725657640 1 -- 172.21.15.139:0/2321308441 reap_dead start 2024-09-16T00:09:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fb720204e40 msgr2=0x7fb720207230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fb720204e40 0x7fb720207230 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fb720207c80 msgr2=0x7fb72020a070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fb720207c80 0x7fb72020a070 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fb72020aac0 msgr2=0x7fb72020ceb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fb72020aac0 0x7fb72020ceb0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb70406c2b0 msgr2=0x7fb70406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb70406c2b0 0x7fb70406e770 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb714034000 tx=0x7fb71402e000 comp rx=0 tx=0).stop 2024-09-16T00:09:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72014f000 msgr2=0x7fb720072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb7270ba640 1 --2- 172.21.15.139:0/2321308441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb72014f000 0x7fb720072f30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb71800caa0 tx=0x7fb71800cea0 comp rx=0 tx=0).stop 2024-09-16T00:09:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.863+0000 7fb725657640 1 -- 172.21.15.139:0/2321308441 reap_dead start 2024-09-16T00:09:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.864+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 shutdown_connections 2024-09-16T00:09:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.864+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 >> 172.21.15.139:0/2321308441 conn(0x7fb72006f660 msgr2=0x7fb720156ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.864+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 shutdown_connections 2024-09-16T00:09:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:19.864+0000 7fb7270ba640 1 -- 172.21.15.139:0/2321308441 wait complete. 2024-09-16T00:09:19.906 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:19.905+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:19.933 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:19.933 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:19.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:19.933 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 101.31653838699999, 2024-09-16T00:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036770370000000002, 2024-09-16T00:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:09:19.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615149+0000", 2024-09-16T00:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615154+0000", 2024-09-16T00:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1510000000000003e-06 2024-09-16T00:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:19.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615163+0000", 2024-09-16T00:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4490000000000005e-06 2024-09-16T00:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615182+0000", 2024-09-16T00:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8876e-05 2024-09-16T00:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615197+0000", 2024-09-16T00:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4671000000000001e-05 2024-09-16T00:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615349+0000", 2024-09-16T00:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015258700000000001 2024-09-16T00:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000204716 2024-09-16T00:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:19.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:09:19.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017250800000000001 2024-09-16T00:09:19.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:19.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616919+0000", 2024-09-16T00:09:19.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011921799999999999 2024-09-16T00:09:19.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:19.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618728+0000", 2024-09-16T00:09:19.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018087769999999999 2024-09-16T00:09:19.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:19.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618774+0000", 2024-09-16T00:09:19.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6075999999999997e-05 2024-09-16T00:09:19.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:19.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618826+0000", 2024-09-16T00:09:19.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2046000000000003e-05 2024-09-16T00:09:19.939 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.939 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:19.939 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.939 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.1a epoch 22/19 rep_tid 2 entries 22'1 (0'0) error 2:5e41ab69:::200.00000005:head by mds.0.4:36 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-16T00:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 101.315182599, 2024-09-16T00:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023501059999999998, 2024-09-16T00:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616504+0000", 2024-09-16T00:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616506+0000", 2024-09-16T00:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4100000000000001e-06 2024-09-16T00:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617073+0000", 2024-09-16T00:09:19.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00056725899999999997 2024-09-16T00:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617099+0000", 2024-09-16T00:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5860999999999999e-05 2024-09-16T00:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617114+0000", 2024-09-16T00:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5180999999999999e-05 2024-09-16T00:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:19.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617236+0000", 2024-09-16T00:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121662 2024-09-16T00:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001618733 2024-09-16T00:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:47 3.1a 3:582f7b2d:::10000000226.00000000:head [write 0~5348] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:19.943 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.358277+0000", 2024-09-16T00:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.573410218999999, 2024-09-16T00:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003561935, 2024-09-16T00:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 47 2024-09-16T00:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358277+0000", 2024-09-16T00:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358277+0000", 2024-09-16T00:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358278+0000", 2024-09-16T00:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.404e-06 2024-09-16T00:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358290+0000", 2024-09-16T00:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1926999999999999e-05 2024-09-16T00:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358307+0000", 2024-09-16T00:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6529e-05 2024-09-16T00:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358316+0000", 2024-09-16T00:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1779999999999992e-06 2024-09-16T00:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358398+0000", 2024-09-16T00:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2056000000000002e-05 2024-09-16T00:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358488+0000", 2024-09-16T00:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9761999999999999e-05 2024-09-16T00:09:19.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.358567+0000", 2024-09-16T00:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9173999999999998e-05 2024-09-16T00:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.359485+0000", 2024-09-16T00:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00091798800000000001 2024-09-16T00:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:19.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.361743+0000", 2024-09-16T00:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002258038 2024-09-16T00:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.361784+0000", 2024-09-16T00:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1529999999999997e-05 2024-09-16T00:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.361839+0000", 2024-09-16T00:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4348999999999998e-05 2024-09-16T00:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.949 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:74 3.13 e57/20 3:c933d8fa:::10000000241.00000000:head v 57'5, mlcod=57'3)", 2024-09-16T00:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.428708+0000", 2024-09-16T00:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.502979133, 2024-09-16T00:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023048140000000001, 2024-09-16T00:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:19.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428708+0000", 2024-09-16T00:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428708+0000", 2024-09-16T00:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428709+0000", 2024-09-16T00:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7000000000000003e-07 2024-09-16T00:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428716+0000", 2024-09-16T00:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5449999999999998e-06 2024-09-16T00:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428730+0000", 2024-09-16T00:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3464e-05 2024-09-16T00:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428738+0000", 2024-09-16T00:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7919999999999993e-06 2024-09-16T00:09:19.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428788+0000", 2024-09-16T00:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0813999999999998e-05 2024-09-16T00:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.428810+0000", 2024-09-16T00:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1543e-05 2024-09-16T00:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:19.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.430972+0000", 2024-09-16T00:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021622130000000001 2024-09-16T00:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.431013+0000", 2024-09-16T00:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0572999999999999e-05 2024-09-16T00:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:101 3.11 e57/20 3:88015c6d:::1000000025c.00000000:head v 57'1, mlcod=0'0)", 2024-09-16T00:09:19.954 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.484094+0000", 2024-09-16T00:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.447593274999999, 2024-09-16T00:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030040079999999999, 2024-09-16T00:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484094+0000", 2024-09-16T00:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:19.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484094+0000", 2024-09-16T00:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484094+0000", 2024-09-16T00:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8999999999999996e-07 2024-09-16T00:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484099+0000", 2024-09-16T00:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6589999999999998e-06 2024-09-16T00:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484108+0000", 2024-09-16T00:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1910000000000006e-06 2024-09-16T00:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484115+0000", 2024-09-16T00:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6270000000000002e-06 2024-09-16T00:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:19.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484163+0000", 2024-09-16T00:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8137999999999999e-05 2024-09-16T00:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.484182+0000", 2024-09-16T00:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8600000000000001e-05 2024-09-16T00:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487055+0000", 2024-09-16T00:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873858 2024-09-16T00:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487098+0000", 2024-09-16T00:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2345000000000002e-05 2024-09-16T00:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:132 3.13 e57/20 3:c93e5698:::1000000027d.00000000:head v 57'7, mlcod=57'5)", 2024-09-16T00:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.557897+0000", 2024-09-16T00:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.373789859999999, 2024-09-16T00:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029674789999999999, 2024-09-16T00:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:19.959 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557897+0000", 2024-09-16T00:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557897+0000", 2024-09-16T00:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557898+0000", 2024-09-16T00:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1800000000000005e-07 2024-09-16T00:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557906+0000", 2024-09-16T00:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8390000000000007e-06 2024-09-16T00:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557919+0000", 2024-09-16T00:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2914999999999999e-05 2024-09-16T00:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557931+0000", 2024-09-16T00:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2734999999999999e-05 2024-09-16T00:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557982+0000", 2024-09-16T00:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0293000000000001e-05 2024-09-16T00:09:19.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:19.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.557997+0000", 2024-09-16T00:09:19.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5160999999999999e-05 2024-09-16T00:09:19.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:19.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.560830+0000", 2024-09-16T00:09:19.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002832776 2024-09-16T00:09:19.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:19.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.560865+0000", 2024-09-16T00:09:19.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4941999999999999e-05 2024-09-16T00:09:19.964 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.964 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:19.964 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.964 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:243 3.24 e57/20 3:25048551:::100000002f0.00000000:head v 57'4, mlcod=57'2)", 2024-09-16T00:09:19.964 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.799371+0000", 2024-09-16T00:09:19.964 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.132316209999999, 2024-09-16T00:09:19.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037781889999999999, 2024-09-16T00:09:19.964 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:19.964 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:19.964 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:19.965 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:19.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:19.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799371+0000", 2024-09-16T00:09:19.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:19.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799371+0000", 2024-09-16T00:09:19.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:19.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799372+0000", 2024-09-16T00:09:19.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9100000000000003e-07 2024-09-16T00:09:19.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:19.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799379+0000", 2024-09-16T00:09:19.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.99e-06 2024-09-16T00:09:19.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:19.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799387+0000", 2024-09-16T00:09:19.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5790000000000004e-06 2024-09-16T00:09:19.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:19.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799392+0000", 2024-09-16T00:09:19.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5210000000000004e-06 2024-09-16T00:09:19.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:19.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799434+0000", 2024-09-16T00:09:19.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2341e-05 2024-09-16T00:09:19.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:19.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.799447+0000", 2024-09-16T00:09:19.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3379e-05 2024-09-16T00:09:19.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:19.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.803117+0000", 2024-09-16T00:09:19.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036697169999999999 2024-09-16T00:09:19.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:19.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.803149+0000", 2024-09-16T00:09:19.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1971000000000003e-05 2024-09-16T00:09:19.968 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.968 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:19.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.969 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:319 3.1a 3:5a6e0114:::10000000350.00000000:head [write 0~3550] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:19.969 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.023452+0000", 2024-09-16T00:09:19.969 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.908235307999998, 2024-09-16T00:09:19.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597186, 2024-09-16T00:09:19.969 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:19.969 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:19.969 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:19.969 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:19.969 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:19.969 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:19.969 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 319 2024-09-16T00:09:19.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.970 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:19.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:19.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023452+0000", 2024-09-16T00:09:19.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:19.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023452+0000", 2024-09-16T00:09:19.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:19.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023452+0000", 2024-09-16T00:09:19.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5199999999999996e-07 2024-09-16T00:09:19.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:19.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023457+0000", 2024-09-16T00:09:19.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8289999999999998e-06 2024-09-16T00:09:19.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:19.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023467+0000", 2024-09-16T00:09:19.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8190000000000003e-06 2024-09-16T00:09:19.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:19.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023472+0000", 2024-09-16T00:09:19.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0810000000000001e-06 2024-09-16T00:09:19.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:19.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023524+0000", 2024-09-16T00:09:19.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2017e-05 2024-09-16T00:09:19.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:19.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023649+0000", 2024-09-16T00:09:19.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012491999999999999 2024-09-16T00:09:19.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:19.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.023772+0000", 2024-09-16T00:09:19.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012327599999999999 2024-09-16T00:09:19.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:19.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.024507+0000", 2024-09-16T00:09:19.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073499100000000005 2024-09-16T00:09:19.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:19.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.026948+0000", 2024-09-16T00:09:19.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002441033 2024-09-16T00:09:19.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:19.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.026997+0000", 2024-09-16T00:09:19.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8778e-05 2024-09-16T00:09:19.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:19.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.027049+0000", 2024-09-16T00:09:19.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1690000000000001e-05 2024-09-16T00:09:19.974 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.975 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:19.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.975 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:323 3.3 3:c319d237:::10000000354.00000000:head [write 0~1622] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:19.975 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.032376+0000", 2024-09-16T00:09:19.975 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.899311285, 2024-09-16T00:09:19.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039255940000000001, 2024-09-16T00:09:19.975 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:19.975 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:19.975 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:19.975 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:19.975 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:19.976 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:19.976 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 323 2024-09-16T00:09:19.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.976 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:19.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:19.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032376+0000", 2024-09-16T00:09:19.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:19.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032376+0000", 2024-09-16T00:09:19.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:19.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032377+0000", 2024-09-16T00:09:19.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.42e-06 2024-09-16T00:09:19.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:19.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032383+0000", 2024-09-16T00:09:19.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9900000000000002e-06 2024-09-16T00:09:19.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:19.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032394+0000", 2024-09-16T00:09:19.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1216999999999999e-05 2024-09-16T00:09:19.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:19.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032400+0000", 2024-09-16T00:09:19.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8039999999999998e-06 2024-09-16T00:09:19.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:19.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032451+0000", 2024-09-16T00:09:19.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0612999999999998e-05 2024-09-16T00:09:19.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:19.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032536+0000", 2024-09-16T00:09:19.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4956000000000004e-05 2024-09-16T00:09:19.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:19.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.032614+0000", 2024-09-16T00:09:19.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8716999999999999e-05 2024-09-16T00:09:19.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:19.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.033109+0000", 2024-09-16T00:09:19.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049412399999999997 2024-09-16T00:09:19.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:19.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.036222+0000", 2024-09-16T00:09:19.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031137949999999999 2024-09-16T00:09:19.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:19.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.036264+0000", 2024-09-16T00:09:19.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1572000000000001e-05 2024-09-16T00:09:19.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:19.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.036301+0000", 2024-09-16T00:09:19.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7385999999999997e-05 2024-09-16T00:09:19.981 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.981 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:19.981 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.981 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:324 3.29 e58/20 3:960c7fc3:::10000000355.00000000:head v 58'7, mlcod=57'5)", 2024-09-16T00:09:19.981 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.035116+0000", 2024-09-16T00:09:19.981 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.896571512000001, 2024-09-16T00:09:19.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029776550000000001, 2024-09-16T00:09:19.981 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:19.981 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:19.981 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:19.982 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:19.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:19.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035116+0000", 2024-09-16T00:09:19.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:19.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035116+0000", 2024-09-16T00:09:19.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:19.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035116+0000", 2024-09-16T00:09:19.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.91e-07 2024-09-16T00:09:19.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:19.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035119+0000", 2024-09-16T00:09:19.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.755e-06 2024-09-16T00:09:19.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:19.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035130+0000", 2024-09-16T00:09:19.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0169999999999999e-05 2024-09-16T00:09:19.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:19.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035135+0000", 2024-09-16T00:09:19.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0019999999999997e-06 2024-09-16T00:09:19.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:19.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035175+0000", 2024-09-16T00:09:19.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0027999999999997e-05 2024-09-16T00:09:19.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:19.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.035188+0000", 2024-09-16T00:09:19.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3386e-05 2024-09-16T00:09:19.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:19.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.038054+0000", 2024-09-16T00:09:19.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002866324 2024-09-16T00:09:19.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:19.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.038093+0000", 2024-09-16T00:09:19.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8798999999999999e-05 2024-09-16T00:09:19.985 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.985 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:19.985 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.986 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:392 3.f e58/20 3:f03826fe:::100000003a5.00000000:head v 58'8, mlcod=57'6)", 2024-09-16T00:09:19.986 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.307961+0000", 2024-09-16T00:09:19.986 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.623725797999999, 2024-09-16T00:09:19.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003252202, 2024-09-16T00:09:19.986 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:19.986 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:19.986 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:19.986 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:19.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:19.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.307961+0000", 2024-09-16T00:09:19.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:19.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.307961+0000", 2024-09-16T00:09:19.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:19.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.307962+0000", 2024-09-16T00:09:19.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5000000000000001e-07 2024-09-16T00:09:19.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:19.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.307993+0000", 2024-09-16T00:09:19.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0414000000000001e-05 2024-09-16T00:09:19.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:19.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.308003+0000", 2024-09-16T00:09:19.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0618999999999999e-05 2024-09-16T00:09:19.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:19.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.308010+0000", 2024-09-16T00:09:19.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5200000000000003e-06 2024-09-16T00:09:19.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:19.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.308062+0000", 2024-09-16T00:09:19.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.185e-05 2024-09-16T00:09:19.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:19.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.308083+0000", 2024-09-16T00:09:19.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1489e-05 2024-09-16T00:09:19.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:19.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.311178+0000", 2024-09-16T00:09:19.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030948630000000002 2024-09-16T00:09:19.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:19.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.311213+0000", 2024-09-16T00:09:19.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5497000000000003e-05 2024-09-16T00:09:19.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.990 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:19.990 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.991 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:411 3.29 e58/20 3:96aa9846:::100000003bf.00000000:head v 58'9, mlcod=58'7)", 2024-09-16T00:09:19.991 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.363807+0000", 2024-09-16T00:09:19.991 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.567880252999998, 2024-09-16T00:09:19.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032126009999999998, 2024-09-16T00:09:19.991 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:19.991 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:19.991 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:19.991 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:19.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:19.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363807+0000", 2024-09-16T00:09:19.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:19.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363807+0000", 2024-09-16T00:09:19.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:19.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363807+0000", 2024-09-16T00:09:19.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4899999999999995e-07 2024-09-16T00:09:19.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:19.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363812+0000", 2024-09-16T00:09:19.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4769999999999997e-06 2024-09-16T00:09:19.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:19.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363822+0000", 2024-09-16T00:09:19.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7950000000000002e-06 2024-09-16T00:09:19.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:19.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363827+0000", 2024-09-16T00:09:19.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8049999999999997e-06 2024-09-16T00:09:19.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:19.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363867+0000", 2024-09-16T00:09:19.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0694e-05 2024-09-16T00:09:19.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:19.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.363880+0000", 2024-09-16T00:09:19.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3022e-05 2024-09-16T00:09:19.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:19.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.366979+0000", 2024-09-16T00:09:19.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003099161 2024-09-16T00:09:19.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:19.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.367019+0000", 2024-09-16T00:09:19.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9997999999999999e-05 2024-09-16T00:09:19.995 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.995 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:19.995 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.995 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:500 3.13 e58/20 3:c8c66365:::1000000042a.00000000:head v 58'14, mlcod=58'12)", 2024-09-16T00:09:19.995 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.760251+0000", 2024-09-16T00:09:19.995 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.171436451999998, 2024-09-16T00:09:19.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002913793, 2024-09-16T00:09:19.995 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:19.996 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:19.996 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:19.996 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:19.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:19.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760251+0000", 2024-09-16T00:09:19.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:19.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760251+0000", 2024-09-16T00:09:19.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:19.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:19.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760251+0000", 2024-09-16T00:09:19.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7400000000000003e-07 2024-09-16T00:09:19.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:19.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760257+0000", 2024-09-16T00:09:19.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0000000000000002e-06 2024-09-16T00:09:19.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:19.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760268+0000", 2024-09-16T00:09:19.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0397999999999999e-05 2024-09-16T00:09:19.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:19.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760273+0000", 2024-09-16T00:09:19.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2889999999999997e-06 2024-09-16T00:09:19.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:19.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760322+0000", 2024-09-16T00:09:19.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9193999999999997e-05 2024-09-16T00:09:19.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:19.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.760343+0000", 2024-09-16T00:09:19.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1228000000000001e-05 2024-09-16T00:09:19.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:19.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.763123+0000", 2024-09-16T00:09:19.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027797189999999999 2024-09-16T00:09:19.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:19.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:19.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:19.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.763164+0000", 2024-09-16T00:09:19.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1291000000000002e-05 2024-09-16T00:09:19.999 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:19.999 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.000 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.000 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:811 3.1c e59/20 3:3b54038d:::100000005a1.00000000:head v 59'10, mlcod=59'8)", 2024-09-16T00:09:20.000 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:58.585639+0000", 2024-09-16T00:09:20.000 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.346047721000001, 2024-09-16T00:09:20.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002996163, 2024-09-16T00:09:20.000 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.000 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.000 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.000 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585639+0000", 2024-09-16T00:09:20.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585639+0000", 2024-09-16T00:09:20.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585640+0000", 2024-09-16T00:09:20.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1800000000000005e-07 2024-09-16T00:09:20.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585657+0000", 2024-09-16T00:09:20.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7249e-05 2024-09-16T00:09:20.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585669+0000", 2024-09-16T00:09:20.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1212e-05 2024-09-16T00:09:20.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585675+0000", 2024-09-16T00:09:20.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6200000000000001e-06 2024-09-16T00:09:20.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585723+0000", 2024-09-16T00:09:20.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7991000000000002e-05 2024-09-16T00:09:20.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585738+0000", 2024-09-16T00:09:20.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4578e-05 2024-09-16T00:09:20.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588596+0000", 2024-09-16T00:09:20.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028585780000000001 2024-09-16T00:09:20.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588635+0000", 2024-09-16T00:09:20.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9116999999999998e-05 2024-09-16T00:09:20.004 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.004 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.004 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.004 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:1618 3.1c e61/20 3:3b08c232:::1000000094e.00000000:head v 61'20, mlcod=61'18)", 2024-09-16T00:09:20.004 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.253213+0000", 2024-09-16T00:09:20.005 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.678474215000001, 2024-09-16T00:09:20.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025388519999999999, 2024-09-16T00:09:20.005 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.005 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.005 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.005 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253213+0000", 2024-09-16T00:09:20.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253213+0000", 2024-09-16T00:09:20.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253214+0000", 2024-09-16T00:09:20.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4600000000000004e-07 2024-09-16T00:09:20.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253231+0000", 2024-09-16T00:09:20.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6982000000000001e-05 2024-09-16T00:09:20.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253243+0000", 2024-09-16T00:09:20.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2412000000000001e-05 2024-09-16T00:09:20.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253250+0000", 2024-09-16T00:09:20.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.922e-06 2024-09-16T00:09:20.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253293+0000", 2024-09-16T00:09:20.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2823e-05 2024-09-16T00:09:20.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253309+0000", 2024-09-16T00:09:20.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6049000000000002e-05 2024-09-16T00:09:20.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255709+0000", 2024-09-16T00:09:20.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002400639 2024-09-16T00:09:20.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255752+0000", 2024-09-16T00:09:20.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2279e-05 2024-09-16T00:09:20.009 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.009 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.009 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.009 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:1691 3.a e61/46 3:518183a8:::10000000999.00000000:head v 61'30, mlcod=61'28)", 2024-09-16T00:09:20.009 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.411769+0000", 2024-09-16T00:09:20.009 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.519918108999999, 2024-09-16T00:09:20.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027805550000000001, 2024-09-16T00:09:20.009 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.009 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.009 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.009 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411769+0000", 2024-09-16T00:09:20.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411769+0000", 2024-09-16T00:09:20.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411770+0000", 2024-09-16T00:09:20.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3e-07 2024-09-16T00:09:20.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411787+0000", 2024-09-16T00:09:20.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7632999999999999e-05 2024-09-16T00:09:20.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411798+0000", 2024-09-16T00:09:20.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.094e-05 2024-09-16T00:09:20.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411807+0000", 2024-09-16T00:09:20.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3930000000000008e-06 2024-09-16T00:09:20.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411860+0000", 2024-09-16T00:09:20.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3477999999999999e-05 2024-09-16T00:09:20.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.411879+0000", 2024-09-16T00:09:20.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9148999999999999e-05 2024-09-16T00:09:20.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.414504+0000", 2024-09-16T00:09:20.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026246759999999998 2024-09-16T00:09:20.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.414549+0000", 2024-09-16T00:09:20.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5556000000000002e-05 2024-09-16T00:09:20.013 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.013 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.013 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.014 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:1889 3.f e61/20 3:f2db18a4:::10000000a6a.00000000:head v 61'27, mlcod=61'25)", 2024-09-16T00:09:20.014 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.171388+0000", 2024-09-16T00:09:20.014 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.760298941999999, 2024-09-16T00:09:20.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00302002, 2024-09-16T00:09:20.014 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.014 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.014 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.014 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171388+0000", 2024-09-16T00:09:20.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171388+0000", 2024-09-16T00:09:20.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171389+0000", 2024-09-16T00:09:20.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0470000000000001e-06 2024-09-16T00:09:20.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171399+0000", 2024-09-16T00:09:20.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0088000000000001e-05 2024-09-16T00:09:20.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171415+0000", 2024-09-16T00:09:20.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5526e-05 2024-09-16T00:09:20.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171422+0000", 2024-09-16T00:09:20.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.764e-06 2024-09-16T00:09:20.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171488+0000", 2024-09-16T00:09:20.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6308000000000003e-05 2024-09-16T00:09:20.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.171511+0000", 2024-09-16T00:09:20.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2891000000000002e-05 2024-09-16T00:09:20.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.174359+0000", 2024-09-16T00:09:20.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028479669999999999 2024-09-16T00:09:20.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.174408+0000", 2024-09-16T00:09:20.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9428999999999997e-05 2024-09-16T00:09:20.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.018 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.018 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:161 2.b e64/19 2:d5c7a900:::200.00000003:head v 64'5, mlcod=61'3)", 2024-09-16T00:09:20.018 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:07.541046+0000", 2024-09-16T00:09:20.018 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.390640624, 2024-09-16T00:09:20.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010081302, 2024-09-16T00:09:20.018 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.018 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.018 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.019 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.541046+0000", 2024-09-16T00:09:20.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.541046+0000", 2024-09-16T00:09:20.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.541048+0000", 2024-09-16T00:09:20.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5859999999999999e-06 2024-09-16T00:09:20.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543134+0000", 2024-09-16T00:09:20.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020863589999999999 2024-09-16T00:09:20.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543155+0000", 2024-09-16T00:09:20.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0344999999999999e-05 2024-09-16T00:09:20.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543167+0000", 2024-09-16T00:09:20.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2071e-05 2024-09-16T00:09:20.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543242+0000", 2024-09-16T00:09:20.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5108e-05 2024-09-16T00:09:20.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543268+0000", 2024-09-16T00:09:20.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6471000000000001e-05 2024-09-16T00:09:20.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551087+0000", 2024-09-16T00:09:20.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0078188670000000002 2024-09-16T00:09:20.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551128+0000", 2024-09-16T00:09:20.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0494999999999998e-05 2024-09-16T00:09:20.022 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.022 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.022 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.023 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:224 2.1a e66/46 2:5e41ab69:::200.00000005:head v 66'4, mlcod=63'2)", 2024-09-16T00:09:20.023 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:12.643213+0000", 2024-09-16T00:09:20.023 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.2884738279999999, 2024-09-16T00:09:20.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004469298, 2024-09-16T00:09:20.023 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.023 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.023 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.023 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.643213+0000", 2024-09-16T00:09:20.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.643213+0000", 2024-09-16T00:09:20.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.643215+0000", 2024-09-16T00:09:20.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.418e-06 2024-09-16T00:09:20.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.644677+0000", 2024-09-16T00:09:20.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001462124 2024-09-16T00:09:20.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.644696+0000", 2024-09-16T00:09:20.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9726000000000001e-05 2024-09-16T00:09:20.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.644707+0000", 2024-09-16T00:09:20.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0533000000000001e-05 2024-09-16T00:09:20.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.644765+0000", 2024-09-16T00:09:20.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8199999999999998e-05 2024-09-16T00:09:20.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.644784+0000", 2024-09-16T00:09:20.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9162999999999999e-05 2024-09-16T00:09:20.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.647638+0000", 2024-09-16T00:09:20.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028538769999999999 2024-09-16T00:09:20.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.647683+0000", 2024-09-16T00:09:20.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4257e-05 2024-09-16T00:09:20.027 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.027 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.027 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.027 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:6118 3.14 3:2ad8f612:::10000001c0f.00000000:head [write 0~1829] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:20.028 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:16.076799+0000", 2024-09-16T00:09:20.028 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.854887578, 2024-09-16T00:09:20.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026989879999999998, 2024-09-16T00:09:20.028 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.028 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.028 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:20.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:20.028 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:20.028 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 6118 2024-09-16T00:09:20.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.028 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.076799+0000", 2024-09-16T00:09:20.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.076799+0000", 2024-09-16T00:09:20.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.076801+0000", 2024-09-16T00:09:20.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4720000000000001e-06 2024-09-16T00:09:20.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.076809+0000", 2024-09-16T00:09:20.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8189999999999992e-06 2024-09-16T00:09:20.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.076824+0000", 2024-09-16T00:09:20.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4807000000000001e-05 2024-09-16T00:09:20.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.076831+0000", 2024-09-16T00:09:20.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1400000000000002e-06 2024-09-16T00:09:20.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.076878+0000", 2024-09-16T00:09:20.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7747999999999998e-05 2024-09-16T00:09:20.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.076979+0000", 2024-09-16T00:09:20.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010053800000000001 2024-09-16T00:09:20.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:20.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.077058+0000", 2024-09-16T00:09:20.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8652000000000005e-05 2024-09-16T00:09:20.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:20.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.077664+0000", 2024-09-16T00:09:20.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00060631300000000003 2024-09-16T00:09:20.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:20.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.079385+0000", 2024-09-16T00:09:20.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017209339999999999 2024-09-16T00:09:20.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.079450+0000", 2024-09-16T00:09:20.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5075999999999998e-05 2024-09-16T00:09:20.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.079498+0000", 2024-09-16T00:09:20.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8489000000000003e-05 2024-09-16T00:09:20.033 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.033 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.033 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.033 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.033 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.033 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:20.035 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:09:20.035 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:20.033+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:20.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.102+0000 7efdff7fe640 1 -- 172.21.15.139:0/819757937 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v73) ==== 113+0+0 (secure 0 0 0) 0x7efe0805ca30 con 0x7efe180c9b00 2024-09-16T00:09:20.103 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T00:09:20.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.104+0000 7efe1d786640 1 -- 172.21.15.139:0/819757937 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efe0406c2e0 msgr2=0x7efe0406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:20.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.104+0000 7efe1d786640 1 --2- 172.21.15.139:0/819757937 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efe0406c2e0 0x7efe0406e7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efe0c034000 tx=0x7efe0c02e000 comp rx=0 tx=0).stop 2024-09-16T00:09:20.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.104+0000 7efe1d786640 1 -- 172.21.15.139:0/819757937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe180c9b00 msgr2=0x7efe18157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:20.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.104+0000 7efe1d786640 1 --2- 172.21.15.139:0/819757937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe180c9b00 0x7efe18157d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efe0800c4a0 tx=0x7efe0800c960 comp rx=0 tx=0).stop 2024-09-16T00:09:20.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.104+0000 7efe1d786640 1 -- 172.21.15.139:0/819757937 shutdown_connections 2024-09-16T00:09:20.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.104+0000 7efe1d786640 1 --2- 172.21.15.139:0/819757937 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efe0406c2e0 0x7efe0406e7a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.104+0000 7efe1d786640 1 --2- 172.21.15.139:0/819757937 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe1815de90 0x7efe181e4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.104+0000 7efe1d786640 1 --2- 172.21.15.139:0/819757937 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe18154960 0x7efe181e45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.104+0000 7efe1d786640 1 --2- 172.21.15.139:0/819757937 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe180c9b00 0x7efe18157d80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.104+0000 7efe1d786640 1 -- 172.21.15.139:0/819757937 >> 172.21.15.139:0/819757937 conn(0x7efe1806bb20 msgr2=0x7efe1815a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:20.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.104+0000 7efe1d786640 1 -- 172.21.15.139:0/819757937 shutdown_connections 2024-09-16T00:09:20.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.105+0000 7efe1d786640 1 -- 172.21.15.139:0/819757937 wait complete. 2024-09-16T00:09:20.109 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T00:09:20.120 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T00:09:20.160 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:20.160+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:20.191 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:20.191 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:20.191 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:20.192 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:20.202 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:09:20.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.221+0000 7f12d1465640 1 Processor -- start 2024-09-16T00:09:20.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.221+0000 7f12d1465640 1 -- start start 2024-09-16T00:09:20.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.222+0000 7f12d1465640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f12cc154c20 0x7f12cc15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:20.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.222+0000 7f12d1465640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f12cc068060 0x7f12cc0c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:20.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.222+0000 7f12d1465640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12cc0ccc90 0x7f12cc0cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:20.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.222+0000 7f12d1465640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f12cc15b7c0 con 0x7f12cc068060 2024-09-16T00:09:20.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.222+0000 7f12d1465640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f12cc0cc260 con 0x7f12cc154c20 2024-09-16T00:09:20.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.222+0000 7f12d1465640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f12cc0cc4b0 con 0x7f12cc0ccc90 2024-09-16T00:09:20.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.222+0000 7f12ca7fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f12cc068060 0x7f12cc0c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:20.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.222+0000 7f12ca7fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f12cc068060 0x7f12cc0c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53938/0 (socket says 172.21.15.139:53938) 2024-09-16T00:09:20.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.222+0000 7f12ca7fc640 1 -- 172.21.15.139:0/1818139542 learned_addr learned my addr 172.21.15.139:0/1818139542 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:20.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.222+0000 7f12caffd640 1 --2- 172.21.15.139:0/1818139542 >> v2:172.21.15.164:3300/0 conn(0x7f12cc154c20 0x7f12cc15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:20.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.222+0000 7f12cb7fe640 1 --2- 172.21.15.139:0/1818139542 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12cc0ccc90 0x7f12cc0cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:20.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.223+0000 7f12ca7fc640 1 -- 172.21.15.139:0/1818139542 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12cc0ccc90 msgr2=0x7f12cc0cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:20.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.223+0000 7f12ca7fc640 1 --2- 172.21.15.139:0/1818139542 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12cc0ccc90 0x7f12cc0cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.223+0000 7f12ca7fc640 1 -- 172.21.15.139:0/1818139542 >> v2:172.21.15.164:3300/0 conn(0x7f12cc154c20 msgr2=0x7f12cc15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:20.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.223+0000 7f12ca7fc640 1 --2- 172.21.15.139:0/1818139542 >> v2:172.21.15.164:3300/0 conn(0x7f12cc154c20 0x7f12cc15b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.223+0000 7f12ca7fc640 1 -- 172.21.15.139:0/1818139542 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc0cc790 con 0x7f12cc068060 2024-09-16T00:09:20.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.223+0000 7f12ca7fc640 1 --2- 172.21.15.139:0/1818139542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12cc068060 0x7f12cc0c5120 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f12cc0d1290 tx=0x7f12b402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c2f7f9c54094d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:20.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.223+0000 7f12c9ffb640 1 -- 172.21.15.139:0/1818139542 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4004030 con 0x7f12cc068060 2024-09-16T00:09:20.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.223+0000 7f12c9ffb640 1 -- 172.21.15.139:0/1818139542 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12b40041d0 con 0x7f12cc068060 2024-09-16T00:09:20.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.223+0000 7f12c9ffb640 1 -- 172.21.15.139:0/1818139542 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4003610 con 0x7f12cc068060 2024-09-16T00:09:20.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.224+0000 7f12d1465640 1 -- 172.21.15.139:0/1818139542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12cc068060 msgr2=0x7f12cc0c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:20.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.224+0000 7f12d1465640 1 --2- 172.21.15.139:0/1818139542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12cc068060 0x7f12cc0c5120 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f12cc0d1290 tx=0x7f12b402a1c0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.224+0000 7f12d1465640 1 -- 172.21.15.139:0/1818139542 shutdown_connections 2024-09-16T00:09:20.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.224+0000 7f12d1465640 1 --2- 172.21.15.139:0/1818139542 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12cc0ccc90 0x7f12cc0cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.224+0000 7f12d1465640 1 --2- 172.21.15.139:0/1818139542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12cc068060 0x7f12cc0c5120 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.224+0000 7f12d1465640 1 --2- 172.21.15.139:0/1818139542 >> v2:172.21.15.164:3300/0 conn(0x7f12cc154c20 0x7f12cc15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.224+0000 7f12d1465640 1 -- 172.21.15.139:0/1818139542 >> 172.21.15.139:0/1818139542 conn(0x7f12cc06ca40 msgr2=0x7f12cc067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:20.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.224+0000 7f12d1465640 1 -- 172.21.15.139:0/1818139542 shutdown_connections 2024-09-16T00:09:20.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.224+0000 7f12d1465640 1 -- 172.21.15.139:0/1818139542 wait complete. 2024-09-16T00:09:20.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.225+0000 7f12d1465640 1 Processor -- start 2024-09-16T00:09:20.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.225+0000 7f12d1465640 1 -- start start 2024-09-16T00:09:20.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.226+0000 7f12d1465640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12cc068060 0x7f12cc1f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:20.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.226+0000 7f12d1465640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12cc0ccc90 0x7f12cc1f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:20.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.226+0000 7f12d1465640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12cc154c20 0x7f12cc149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:20.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.226+0000 7f12d1465640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc15e3b0 con 0x7f12cc154c20 2024-09-16T00:09:20.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.226+0000 7f12d1465640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc15e0d0 con 0x7f12cc068060 2024-09-16T00:09:20.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.226+0000 7f12d1465640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f12cc15e690 con 0x7f12cc0ccc90 2024-09-16T00:09:20.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.226+0000 7f12cb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12cc154c20 0x7f12cc149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:20.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.226+0000 7f12cb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12cc154c20 0x7f12cc149cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53954/0 (socket says 172.21.15.139:53954) 2024-09-16T00:09:20.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.226+0000 7f12cb7fe640 1 -- 172.21.15.139:0/345158618 learned_addr learned my addr 172.21.15.139:0/345158618 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:20.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.226+0000 7f12caffd640 1 --2- 172.21.15.139:0/345158618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12cc068060 0x7f12cc1f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:20.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.226+0000 7f12ca7fc640 1 --2- 172.21.15.139:0/345158618 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12cc0ccc90 0x7f12cc1f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:20.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.227+0000 7f12cb7fe640 1 -- 172.21.15.139:0/345158618 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12cc0ccc90 msgr2=0x7f12cc1f15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:20.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.227+0000 7f12cb7fe640 1 --2- 172.21.15.139:0/345158618 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12cc0ccc90 0x7f12cc1f15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.227+0000 7f12cb7fe640 1 -- 172.21.15.139:0/345158618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12cc068060 msgr2=0x7f12cc1f1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.227+0000 7f12cb7fe640 1 --2- 172.21.15.139:0/345158618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12cc068060 0x7f12cc1f1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.227+0000 7f12cb7fe640 1 -- 172.21.15.139:0/345158618 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc1f0b10 con 0x7f12cc154c20 2024-09-16T00:09:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.227+0000 7f12cb7fe640 1 --2- 172.21.15.139:0/345158618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12cc154c20 0x7f12cc149cb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00eed0 tx=0x7f12bc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.227+0000 7f12c37fe640 1 -- 172.21.15.139:0/345158618 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc018020 con 0x7f12cc154c20 2024-09-16T00:09:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.227+0000 7f12c37fe640 1 -- 172.21.15.139:0/345158618 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12bc00bd10 con 0x7f12cc154c20 2024-09-16T00:09:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.227+0000 7f12d1465640 1 -- 172.21.15.139:0/345158618 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12cc15bca0 con 0x7f12cc154c20 2024-09-16T00:09:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.227+0000 7f12c37fe640 1 -- 172.21.15.139:0/345158618 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc014640 con 0x7f12cc154c20 2024-09-16T00:09:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.227+0000 7f12d1465640 1 -- 172.21.15.139:0/345158618 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12cc1f0dd0 con 0x7f12cc154c20 2024-09-16T00:09:20.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.228+0000 7f12d1465640 1 -- 172.21.15.139:0/345158618 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12cc0686b0 con 0x7f12cc154c20 2024-09-16T00:09:20.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.228+0000 7f12c37fe640 1 -- 172.21.15.139:0/345158618 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f12bc005d10 con 0x7f12cc154c20 2024-09-16T00:09:20.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.229+0000 7f12c37fe640 1 --2- 172.21.15.139:0/345158618 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f12ac06c2b0 0x7f12ac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:20.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.229+0000 7f12c37fe640 1 -- 172.21.15.139:0/345158618 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5944+0+0 (secure 0 0 0) 0x7f12bc08b970 con 0x7f12cc154c20 2024-09-16T00:09:20.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.229+0000 7f12caffd640 1 --2- 172.21.15.139:0/345158618 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f12ac06c2b0 0x7f12ac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:20.231 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.229+0000 7f12caffd640 1 --2- 172.21.15.139:0/345158618 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f12ac06c2b0 0x7f12ac06e770 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009b40 tx=0x7f12c4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:20.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.233+0000 7f12c37fe640 1 -- 172.21.15.139:0/345158618 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f12bc022070 con 0x7f12cc154c20 2024-09-16T00:09:20.247 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:20.246+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:20.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.265+0000 7fc114277640 1 Processor -- start 2024-09-16T00:09:20.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.265+0000 7fc114277640 1 -- start start 2024-09-16T00:09:20.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.266+0000 7fc114277640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc10c068060 0x7fc10c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:20.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.266+0000 7fc114277640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc10c0ccc90 0x7fc10c0cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:20.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.266+0000 7fc114277640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc10c0cc4b0 con 0x7fc10c154c20 2024-09-16T00:09:20.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.266+0000 7fc114277640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc10c0c5830 con 0x7fc10c068060 2024-09-16T00:09:20.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.266+0000 7fc114277640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc10c15b7c0 con 0x7fc10c0ccc90 2024-09-16T00:09:20.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.266+0000 7fc111812640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc10c068060 0x7fc10c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:20.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.266+0000 7fc112013640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc10c154c20 0x7fc10c0c5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41644/0 (socket says 172.21.15.139:41644) 2024-09-16T00:09:20.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.266+0000 7fc111812640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc10c068060 0x7fc10c15b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53960/0 (socket says 172.21.15.139:53960) 2024-09-16T00:09:20.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.266+0000 7fc112013640 1 -- 172.21.15.139:0/2582991048 learned_addr learned my addr 172.21.15.139:0/2582991048 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:20.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.266+0000 7fc112814640 1 --2- 172.21.15.139:0/2582991048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc10c0ccc90 0x7fc10c0cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:20.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.266+0000 7fc111812640 1 -- 172.21.15.139:0/2582991048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc10c0ccc90 msgr2=0x7fc10c0cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:20.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.266+0000 7fc111812640 1 --2- 172.21.15.139:0/2582991048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc10c0ccc90 0x7fc10c0cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.266+0000 7fc111812640 1 -- 172.21.15.139:0/2582991048 >> v1:172.21.15.139:6789/0 conn(0x7fc10c154c20 legacy=0x7fc10c0c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:20.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.266+0000 7fc111812640 1 -- 172.21.15.139:0/2582991048 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c0cc790 con 0x7fc10c068060 2024-09-16T00:09:20.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.266+0000 7fc111011640 1 -- 172.21.15.139:0/2582991048 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 34182297 0 0) 0x7fc10c0cc4b0 con 0x7fc10c154c20 2024-09-16T00:09:20.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.266+0000 7fc112814640 1 --2- 172.21.15.139:0/2582991048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc10c0ccc90 0x7fc10c0cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:09:20.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.267+0000 7fc111812640 1 --2- 172.21.15.139:0/2582991048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10c068060 0x7fc10c15b280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009790 tx=0x7fc0fc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f90d4aedcb8dcb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:20.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.267+0000 7fc111011640 1 -- 172.21.15.139:0/2582991048 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc037070 con 0x7fc10c068060 2024-09-16T00:09:20.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.267+0000 7fc111011640 1 -- 172.21.15.139:0/2582991048 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0fc032440 con 0x7fc10c068060 2024-09-16T00:09:20.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.267+0000 7fc111011640 1 -- 172.21.15.139:0/2582991048 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc032810 con 0x7fc10c068060 2024-09-16T00:09:20.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.268+0000 7fc114277640 1 -- 172.21.15.139:0/2582991048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10c068060 msgr2=0x7fc10c15b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:20.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.268+0000 7fc114277640 1 --2- 172.21.15.139:0/2582991048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10c068060 0x7fc10c15b280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc009790 tx=0x7fc0fc02a180 comp rx=0 tx=0).stop 2024-09-16T00:09:20.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.268+0000 7fc114277640 1 -- 172.21.15.139:0/2582991048 shutdown_connections 2024-09-16T00:09:20.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.268+0000 7fc114277640 1 --2- 172.21.15.139:0/2582991048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc10c0ccc90 0x7fc10c0cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.268+0000 7fc114277640 1 --2- 172.21.15.139:0/2582991048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10c068060 0x7fc10c15b280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.268+0000 7fc114277640 1 -- 172.21.15.139:0/2582991048 >> 172.21.15.139:0/2582991048 conn(0x7fc10c06ca40 msgr2=0x7fc10c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:20.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.268+0000 7fc114277640 1 -- 172.21.15.139:0/2582991048 shutdown_connections 2024-09-16T00:09:20.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.268+0000 7fc114277640 1 -- 172.21.15.139:0/2582991048 wait complete. 2024-09-16T00:09:20.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.269+0000 7fc114277640 1 Processor -- start 2024-09-16T00:09:20.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.269+0000 7fc114277640 1 -- start start 2024-09-16T00:09:20.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc114277640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc10c068060 0x7fc10c159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:20.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc114277640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10c0ccc90 0x7fc10c150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:20.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc114277640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc10c154c20 0x7fc10c1f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:20.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc114277640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c15e0d0 con 0x7fc10c0ccc90 2024-09-16T00:09:20.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc114277640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc10c0cc260 con 0x7fc10c068060 2024-09-16T00:09:20.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc114277640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc10c15e3b0 con 0x7fc10c154c20 2024-09-16T00:09:20.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc111812640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10c0ccc90 0x7fc10c150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:20.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc111812640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10c0ccc90 0x7fc10c150400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53962/0 (socket says 172.21.15.139:53962) 2024-09-16T00:09:20.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc111812640 1 -- 172.21.15.139:0/1159031052 learned_addr learned my addr 172.21.15.139:0/1159031052 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:20.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc112814640 1 --2- 172.21.15.139:0/1159031052 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc10c154c20 0x7fc10c1f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:20.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc112013640 1 --2- 172.21.15.139:0/1159031052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc10c068060 0x7fc10c159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:20.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc111812640 1 -- 172.21.15.139:0/1159031052 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc10c154c20 msgr2=0x7fc10c1f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:20.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc111812640 1 --2- 172.21.15.139:0/1159031052 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc10c154c20 0x7fc10c1f3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc111812640 1 -- 172.21.15.139:0/1159031052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc10c068060 msgr2=0x7fc10c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:20.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc111812640 1 --2- 172.21.15.139:0/1159031052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc10c068060 0x7fc10c159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:20.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc111812640 1 -- 172.21.15.139:0/1159031052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10c1f1160 con 0x7fc10c0ccc90 2024-09-16T00:09:20.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc111812640 1 --2- 172.21.15.139:0/1159031052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10c0ccc90 0x7fc10c150400 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc000c00 tx=0x7fc0fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:20.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc102ffd640 1 -- 172.21.15.139:0/1159031052 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc042020 con 0x7fc10c0ccc90 2024-09-16T00:09:20.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc102ffd640 1 -- 172.21.15.139:0/1159031052 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0fc038040 con 0x7fc10c0ccc90 2024-09-16T00:09:20.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc114277640 1 -- 172.21.15.139:0/1159031052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10c1f1440 con 0x7fc10c0ccc90 2024-09-16T00:09:20.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc102ffd640 1 -- 172.21.15.139:0/1159031052 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc037050 con 0x7fc10c0ccc90 2024-09-16T00:09:20.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.270+0000 7fc114277640 1 -- 172.21.15.139:0/1159031052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc10c15d9a0 con 0x7fc10c0ccc90 2024-09-16T00:09:20.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.271+0000 7fc114277640 1 -- 172.21.15.139:0/1159031052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc10c15e690 con 0x7fc10c0ccc90 2024-09-16T00:09:20.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.272+0000 7fc102ffd640 1 -- 172.21.15.139:0/1159031052 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc0fc005990 con 0x7fc10c0ccc90 2024-09-16T00:09:20.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.273+0000 7fc102ffd640 1 --2- 172.21.15.139:0/1159031052 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc0f006c2e0 0x7fc0f006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:20.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.273+0000 7fc102ffd640 1 -- 172.21.15.139:0/1159031052 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5944+0+0 (secure 0 0 0) 0x7fc0fc0abbe0 con 0x7fc10c0ccc90 2024-09-16T00:09:20.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.273+0000 7fc112013640 1 --2- 172.21.15.139:0/1159031052 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc0f006c2e0 0x7fc0f006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:20.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.273+0000 7fc112013640 1 --2- 172.21.15.139:0/1159031052 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc0f006c2e0 0x7fc0f006e7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc108007fd0 tx=0x7fc108007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:20.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.281+0000 7fc102ffd640 1 -- 172.21.15.139:0/1159031052 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc0fc04c020 con 0x7fc10c0ccc90 2024-09-16T00:09:20.309 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:20.309 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:20.309 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:20.309 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:20.310 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:20.318 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:09:20.359 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:20.358+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:20.439 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:20.438+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:20.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.452+0000 7f12d1465640 1 -- 172.21.15.139:0/345158618 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f12cc0cca70 con 0x7f12cc154c20 2024-09-16T00:09:20.477 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:20.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:14 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 101.854572326, 2024-09-16T00:09:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029190470000000001, 2024-09-16T00:09:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615982+0000", 2024-09-16T00:09:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7439999999999999e-06 2024-09-16T00:09:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:09:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3743999999999999e-05 2024-09-16T00:09:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2255e-05 2024-09-16T00:09:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616029+0000", 2024-09-16T00:09:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1175999999999999e-05 2024-09-16T00:09:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616098+0000", 2024-09-16T00:09:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9153999999999996e-05 2024-09-16T00:09:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616132+0000", 2024-09-16T00:09:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3924000000000002e-05 2024-09-16T00:09:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:09:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027226189999999999 2024-09-16T00:09:20.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618899+0000", 2024-09-16T00:09:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4431000000000002e-05 2024-09-16T00:09:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:30 2.15 e22/19 2:aa448500:::500.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:09:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:09:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 101.85228802100001, 2024-09-16T00:09:20.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058031250000000001, 2024-09-16T00:09:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:09:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:09:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618265+0000", 2024-09-16T00:09:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9200000000000001e-07 2024-09-16T00:09:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618271+0000", 2024-09-16T00:09:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9580000000000004e-06 2024-09-16T00:09:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618279+0000", 2024-09-16T00:09:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2239999999999995e-06 2024-09-16T00:09:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618285+0000", 2024-09-16T00:09:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-06 2024-09-16T00:09:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618567+0000", 2024-09-16T00:09:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028204400000000001 2024-09-16T00:09:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:09:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9678999999999999e-05 2024-09-16T00:09:20.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624024+0000", 2024-09-16T00:09:20.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054376720000000002 2024-09-16T00:09:20.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624068+0000", 2024-09-16T00:09:20.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:09:20.489 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.489 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.489 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:20 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:09:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 101.85193165699999, 2024-09-16T00:09:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028808739999999998, 2024-09-16T00:09:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:09:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:09:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618622+0000", 2024-09-16T00:09:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1260000000000001e-06 2024-09-16T00:09:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618631+0000", 2024-09-16T00:09:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4700000000000008e-06 2024-09-16T00:09:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618645+0000", 2024-09-16T00:09:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3984e-05 2024-09-16T00:09:20.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:09:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2101e-05 2024-09-16T00:09:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618720+0000", 2024-09-16T00:09:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2761000000000004e-05 2024-09-16T00:09:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618740+0000", 2024-09-16T00:09:20.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9409e-05 2024-09-16T00:09:20.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621436+0000", 2024-09-16T00:09:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002696211 2024-09-16T00:09:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621502+0000", 2024-09-16T00:09:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5811999999999999e-05 2024-09-16T00:09:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:21 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:20.495 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:09:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 101.851108369, 2024-09-16T00:09:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046318540000000004, 2024-09-16T00:09:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:09:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:09:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619445+0000", 2024-09-16T00:09:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.083e-06 2024-09-16T00:09:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619455+0000", 2024-09-16T00:09:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5920000000000007e-06 2024-09-16T00:09:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619467+0000", 2024-09-16T00:09:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2711000000000001e-05 2024-09-16T00:09:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619476+0000", 2024-09-16T00:09:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0139999999999998e-06 2024-09-16T00:09:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619529+0000", 2024-09-16T00:09:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3078999999999998e-05 2024-09-16T00:09:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619548+0000", 2024-09-16T00:09:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8952000000000001e-05 2024-09-16T00:09:20.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624037+0000", 2024-09-16T00:09:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044895730000000002 2024-09-16T00:09:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624076+0000", 2024-09-16T00:09:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8850000000000002e-05 2024-09-16T00:09:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.500 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:93 3.5 3:a3a50212:::10000000254.00000000:head [write 0~6832] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.467788+0000", 2024-09-16T00:09:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.002764619000001, 2024-09-16T00:09:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001903882, 2024-09-16T00:09:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.501 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93 2024-09-16T00:09:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467788+0000", 2024-09-16T00:09:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467788+0000", 2024-09-16T00:09:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467789+0000", 2024-09-16T00:09:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4399999999999999e-07 2024-09-16T00:09:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467798+0000", 2024-09-16T00:09:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0510000000000002e-06 2024-09-16T00:09:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467809+0000", 2024-09-16T00:09:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0859000000000001e-05 2024-09-16T00:09:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467814+0000", 2024-09-16T00:09:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0409999999999997e-06 2024-09-16T00:09:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467860+0000", 2024-09-16T00:09:20.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6604999999999998e-05 2024-09-16T00:09:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467935+0000", 2024-09-16T00:09:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4417000000000003e-05 2024-09-16T00:09:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.468012+0000", 2024-09-16T00:09:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6971000000000006e-05 2024-09-16T00:09:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.468821+0000", 2024-09-16T00:09:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00080984099999999995 2024-09-16T00:09:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.469636+0000", 2024-09-16T00:09:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00081428499999999999 2024-09-16T00:09:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.469668+0000", 2024-09-16T00:09:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2322e-05 2024-09-16T00:09:20.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.469692+0000", 2024-09-16T00:09:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3745999999999999e-05 2024-09-16T00:09:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:249 3.28 3:14c9b427:::100000002f7.00000000:head [write 0~5746] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.812683+0000", 2024-09-16T00:09:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.657869130999998, 2024-09-16T00:09:20.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019975150000000001, 2024-09-16T00:09:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 249 2024-09-16T00:09:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812683+0000", 2024-09-16T00:09:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812683+0000", 2024-09-16T00:09:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812684+0000", 2024-09-16T00:09:20.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1100000000000004e-07 2024-09-16T00:09:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812708+0000", 2024-09-16T00:09:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4074e-05 2024-09-16T00:09:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812736+0000", 2024-09-16T00:09:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7767e-05 2024-09-16T00:09:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812741+0000", 2024-09-16T00:09:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3319999999999999e-06 2024-09-16T00:09:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812790+0000", 2024-09-16T00:09:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8294000000000002e-05 2024-09-16T00:09:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812881+0000", 2024-09-16T00:09:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1085e-05 2024-09-16T00:09:20.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812960+0000", 2024-09-16T00:09:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8824000000000003e-05 2024-09-16T00:09:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.813562+0000", 2024-09-16T00:09:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00060270500000000004 2024-09-16T00:09:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.814630+0000", 2024-09-16T00:09:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010679489999999999 2024-09-16T00:09:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.814660+0000", 2024-09-16T00:09:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.936e-05 2024-09-16T00:09:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.814681+0000", 2024-09-16T00:09:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1214000000000001e-05 2024-09-16T00:09:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:311 3.2a 3:548c374b:::10000000347.00000000:head [write 0~4872] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.002861+0000", 2024-09-16T00:09:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.467691854999998, 2024-09-16T00:09:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001734665, 2024-09-16T00:09:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:20.515 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 311 2024-09-16T00:09:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002861+0000", 2024-09-16T00:09:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002861+0000", 2024-09-16T00:09:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002863+0000", 2024-09-16T00:09:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0540000000000002e-06 2024-09-16T00:09:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002877+0000", 2024-09-16T00:09:20.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4677e-05 2024-09-16T00:09:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002896+0000", 2024-09-16T00:09:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9038999999999998e-05 2024-09-16T00:09:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002905+0000", 2024-09-16T00:09:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0599999999999997e-06 2024-09-16T00:09:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002973+0000", 2024-09-16T00:09:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7203e-05 2024-09-16T00:09:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.003111+0000", 2024-09-16T00:09:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013858100000000001 2024-09-16T00:09:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.003269+0000", 2024-09-16T00:09:20.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015809 2024-09-16T00:09:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.004171+0000", 2024-09-16T00:09:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00090205600000000004 2024-09-16T00:09:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.004494+0000", 2024-09-16T00:09:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000322126 2024-09-16T00:09:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.004549+0000", 2024-09-16T00:09:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5049000000000001e-05 2024-09-16T00:09:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.004595+0000", 2024-09-16T00:09:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6730000000000002e-05 2024-09-16T00:09:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.522 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:333 3.c 3:329df26d:::1000000035f.00000000:head [write 0~657] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:20.522 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.059938+0000", 2024-09-16T00:09:20.522 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.410614156000001, 2024-09-16T00:09:20.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001777437, 2024-09-16T00:09:20.522 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.522 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.522 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.522 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 333 2024-09-16T00:09:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059938+0000", 2024-09-16T00:09:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059938+0000", 2024-09-16T00:09:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059940+0000", 2024-09-16T00:09:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1129999999999999e-06 2024-09-16T00:09:20.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059953+0000", 2024-09-16T00:09:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.278e-05 2024-09-16T00:09:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059980+0000", 2024-09-16T00:09:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6533000000000002e-05 2024-09-16T00:09:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059994+0000", 2024-09-16T00:09:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4049999999999999e-05 2024-09-16T00:09:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.060136+0000", 2024-09-16T00:09:20.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014184399999999999 2024-09-16T00:09:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.060396+0000", 2024-09-16T00:09:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026020600000000002 2024-09-16T00:09:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.060540+0000", 2024-09-16T00:09:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143838 2024-09-16T00:09:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:20.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.061336+0000", 2024-09-16T00:09:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00079669099999999998 2024-09-16T00:09:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.061621+0000", 2024-09-16T00:09:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028509599999999998 2024-09-16T00:09:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.061674+0000", 2024-09-16T00:09:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2685000000000001e-05 2024-09-16T00:09:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.061716+0000", 2024-09-16T00:09:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1600999999999997e-05 2024-09-16T00:09:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:490 3.d e58/51 3:b3311d72:::1000000041b.00000000:head v 58'8, mlcod=58'6)", 2024-09-16T00:09:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.728943+0000", 2024-09-16T00:09:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.741609857, 2024-09-16T00:09:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002073142, 2024-09-16T00:09:20.529 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728943+0000", 2024-09-16T00:09:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728943+0000", 2024-09-16T00:09:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728943+0000", 2024-09-16T00:09:20.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:09:20.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728949+0000", 2024-09-16T00:09:20.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T00:09:20.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728958+0000", 2024-09-16T00:09:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3049999999999998e-06 2024-09-16T00:09:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728964+0000", 2024-09-16T00:09:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9699999999999996e-06 2024-09-16T00:09:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.729018+0000", 2024-09-16T00:09:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4045000000000002e-05 2024-09-16T00:09:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.729039+0000", 2024-09-16T00:09:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1098999999999999e-05 2024-09-16T00:09:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.730988+0000", 2024-09-16T00:09:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019486919999999999 2024-09-16T00:09:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.731016+0000", 2024-09-16T00:09:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7971e-05 2024-09-16T00:09:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:681 3.17 e59/20 3:ea55746f:::1000000050b.00000000:head v 59'12, mlcod=59'10)", 2024-09-16T00:09:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:58.265463+0000", 2024-09-16T00:09:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.205088980999999, 2024-09-16T00:09:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030780130000000001, 2024-09-16T00:09:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.534 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265463+0000", 2024-09-16T00:09:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265463+0000", 2024-09-16T00:09:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265464+0000", 2024-09-16T00:09:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7999999999999995e-07 2024-09-16T00:09:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265471+0000", 2024-09-16T00:09:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5010000000000001e-06 2024-09-16T00:09:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265480+0000", 2024-09-16T00:09:20.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.302e-06 2024-09-16T00:09:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265485+0000", 2024-09-16T00:09:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:09:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265526+0000", 2024-09-16T00:09:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1229000000000001e-05 2024-09-16T00:09:20.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265539+0000", 2024-09-16T00:09:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3083e-05 2024-09-16T00:09:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.268498+0000", 2024-09-16T00:09:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029594320000000001 2024-09-16T00:09:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.268542+0000", 2024-09-16T00:09:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3207000000000002e-05 2024-09-16T00:09:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:811 3.1c 3:3b54038d:::100000005a1.00000000:head [write 0~2412] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:58.585366+0000", 2024-09-16T00:09:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.885186950000001, 2024-09-16T00:09:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003510216, 2024-09-16T00:09:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.539 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 811 2024-09-16T00:09:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585366+0000", 2024-09-16T00:09:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585366+0000", 2024-09-16T00:09:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585366+0000", 2024-09-16T00:09:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5300000000000003e-07 2024-09-16T00:09:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585371+0000", 2024-09-16T00:09:20.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2849999999999996e-06 2024-09-16T00:09:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585380+0000", 2024-09-16T00:09:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9219999999999992e-06 2024-09-16T00:09:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585390+0000", 2024-09-16T00:09:20.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2960000000000004e-06 2024-09-16T00:09:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585436+0000", 2024-09-16T00:09:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5874000000000002e-05 2024-09-16T00:09:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585510+0000", 2024-09-16T00:09:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4070000000000001e-05 2024-09-16T00:09:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585586+0000", 2024-09-16T00:09:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6044999999999994e-05 2024-09-16T00:09:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.586070+0000", 2024-09-16T00:09:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048407300000000001 2024-09-16T00:09:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:20.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588779+0000", 2024-09-16T00:09:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027090339999999999 2024-09-16T00:09:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588834+0000", 2024-09-16T00:09:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5513000000000002e-05 2024-09-16T00:09:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588876+0000", 2024-09-16T00:09:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1350999999999998e-05 2024-09-16T00:09:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.545 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:1617 3.36 e61/46 3:6fb28598:::1000000094d.00000000:head v 61'29, mlcod=61'27)", 2024-09-16T00:09:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.251467+0000", 2024-09-16T00:09:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.219085294999999, 2024-09-16T00:09:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040699999999999998, 2024-09-16T00:09:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251467+0000", 2024-09-16T00:09:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251467+0000", 2024-09-16T00:09:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251468+0000", 2024-09-16T00:09:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6400000000000001e-07 2024-09-16T00:09:20.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251478+0000", 2024-09-16T00:09:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0057e-05 2024-09-16T00:09:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251489+0000", 2024-09-16T00:09:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0992000000000001e-05 2024-09-16T00:09:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251494+0000", 2024-09-16T00:09:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8999999999999997e-06 2024-09-16T00:09:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251537+0000", 2024-09-16T00:09:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3513999999999997e-05 2024-09-16T00:09:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251550+0000", 2024-09-16T00:09:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3083e-05 2024-09-16T00:09:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255500+0000", 2024-09-16T00:09:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039494090000000001 2024-09-16T00:09:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255537+0000", 2024-09-16T00:09:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280999999999998e-05 2024-09-16T00:09:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.550 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:1618 3.1c 3:3b08c232:::1000000094e.00000000:head [write 0~2553] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T00:09:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.252928+0000", 2024-09-16T00:09:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.217624152999999, 2024-09-16T00:09:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030458360000000001, 2024-09-16T00:09:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:20.551 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1618 2024-09-16T00:09:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252928+0000", 2024-09-16T00:09:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252928+0000", 2024-09-16T00:09:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252929+0000", 2024-09-16T00:09:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.13e-06 2024-09-16T00:09:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252939+0000", 2024-09-16T00:09:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8239999999999995e-06 2024-09-16T00:09:20.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252950+0000", 2024-09-16T00:09:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251999999999999e-05 2024-09-16T00:09:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252962+0000", 2024-09-16T00:09:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2332e-05 2024-09-16T00:09:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253005+0000", 2024-09-16T00:09:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3380000000000001e-05 2024-09-16T00:09:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253084+0000", 2024-09-16T00:09:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9097000000000005e-05 2024-09-16T00:09:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253162+0000", 2024-09-16T00:09:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7890000000000001e-05 2024-09-16T00:09:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255878+0000", 2024-09-16T00:09:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027157589999999999 2024-09-16T00:09:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255892+0000", 2024-09-16T00:09:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4329e-05 2024-09-16T00:09:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255940+0000", 2024-09-16T00:09:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7979000000000003e-05 2024-09-16T00:09:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255974+0000", 2024-09-16T00:09:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3864e-05 2024-09-16T00:09:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:84 2.17 e61/19 2:ee44d6a7:::10000000261.03e00000:head v 61'1, mlcod=0'0)", 2024-09-16T00:09:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.077563+0000", 2024-09-16T00:09:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.392989769, 2024-09-16T00:09:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033709450000000002, 2024-09-16T00:09:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077563+0000", 2024-09-16T00:09:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077563+0000", 2024-09-16T00:09:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077563+0000", 2024-09-16T00:09:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.61e-07 2024-09-16T00:09:20.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077577+0000", 2024-09-16T00:09:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3108000000000001e-05 2024-09-16T00:09:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077587+0000", 2024-09-16T00:09:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0273e-05 2024-09-16T00:09:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077593+0000", 2024-09-16T00:09:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5209999999999999e-06 2024-09-16T00:09:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077641+0000", 2024-09-16T00:09:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7579e-05 2024-09-16T00:09:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077658+0000", 2024-09-16T00:09:20.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7297999999999999e-05 2024-09-16T00:09:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080906+0000", 2024-09-16T00:09:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032481760000000002 2024-09-16T00:09:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080934+0000", 2024-09-16T00:09:20.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7229e-05 2024-09-16T00:09:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:80 2.12 2:4a871970:::10000000261.03600000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.077908+0000", 2024-09-16T00:09:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.392644791999999, 2024-09-16T00:09:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030856009999999999, 2024-09-16T00:09:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.563 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077908+0000", 2024-09-16T00:09:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077908+0000", 2024-09-16T00:09:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077909+0000", 2024-09-16T00:09:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5719999999999999e-06 2024-09-16T00:09:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077920+0000", 2024-09-16T00:09:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0631e-05 2024-09-16T00:09:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077930+0000", 2024-09-16T00:09:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0180000000000001e-05 2024-09-16T00:09:20.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077937+0000", 2024-09-16T00:09:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:09:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077993+0000", 2024-09-16T00:09:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6174999999999997e-05 2024-09-16T00:09:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078106+0000", 2024-09-16T00:09:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113133 2024-09-16T00:09:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078218+0000", 2024-09-16T00:09:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011114 2024-09-16T00:09:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.079121+0000", 2024-09-16T00:09:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00090328499999999998 2024-09-16T00:09:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080928+0000", 2024-09-16T00:09:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018072310000000001 2024-09-16T00:09:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080967+0000", 2024-09-16T00:09:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8982e-05 2024-09-16T00:09:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080993+0000", 2024-09-16T00:09:20.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6157e-05 2024-09-16T00:09:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:1878 3.3c 3:3f4f0e87:::10000000a5f.00000000:head [write 0~2494] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T00:09:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.078761+0000", 2024-09-16T00:09:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.391791861000002, 2024-09-16T00:09:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025239199999999998, 2024-09-16T00:09:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.569 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1878 2024-09-16T00:09:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078761+0000", 2024-09-16T00:09:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078761+0000", 2024-09-16T00:09:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078762+0000", 2024-09-16T00:09:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7600000000000006e-07 2024-09-16T00:09:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078767+0000", 2024-09-16T00:09:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5929999999999996e-06 2024-09-16T00:09:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078781+0000", 2024-09-16T00:09:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3804e-05 2024-09-16T00:09:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078789+0000", 2024-09-16T00:09:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6960000000000005e-06 2024-09-16T00:09:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078838+0000", 2024-09-16T00:09:20.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9416000000000003e-05 2024-09-16T00:09:20.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078940+0000", 2024-09-16T00:09:20.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101431 2024-09-16T00:09:20.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:20.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.079032+0000", 2024-09-16T00:09:20.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2559999999999995e-05 2024-09-16T00:09:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.079823+0000", 2024-09-16T00:09:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00079055300000000005 2024-09-16T00:09:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081201+0000", 2024-09-16T00:09:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013780579999999999 2024-09-16T00:09:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081247+0000", 2024-09-16T00:09:20.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6213e-05 2024-09-16T00:09:20.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081285+0000", 2024-09-16T00:09:20.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7620000000000002e-05 2024-09-16T00:09:20.575 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.575 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.575 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.576 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:2173 3.30 3:0ff9340b:::10000000bb6.00000000:head [write 0~80] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T00:09:20.576 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:03.053795+0000", 2024-09-16T00:09:20.576 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.416757828000001, 2024-09-16T00:09:20.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020696619999999999, 2024-09-16T00:09:20.576 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.576 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.576 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 2173 2024-09-16T00:09:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053795+0000", 2024-09-16T00:09:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053795+0000", 2024-09-16T00:09:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053796+0000", 2024-09-16T00:09:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.257e-06 2024-09-16T00:09:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053802+0000", 2024-09-16T00:09:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3430000000000001e-06 2024-09-16T00:09:20.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053813+0000", 2024-09-16T00:09:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.101e-05 2024-09-16T00:09:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053819+0000", 2024-09-16T00:09:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0750000000000003e-06 2024-09-16T00:09:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053864+0000", 2024-09-16T00:09:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4258999999999998e-05 2024-09-16T00:09:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053954+0000", 2024-09-16T00:09:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0195e-05 2024-09-16T00:09:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.054034+0000", 2024-09-16T00:09:20.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0319000000000001e-05 2024-09-16T00:09:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.054604+0000", 2024-09-16T00:09:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00057022099999999999 2024-09-16T00:09:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.055768+0000", 2024-09-16T00:09:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001164113 2024-09-16T00:09:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.055821+0000", 2024-09-16T00:09:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2364000000000002e-05 2024-09-16T00:09:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.055864+0000", 2024-09-16T00:09:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3506e-05 2024-09-16T00:09:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:2315 3.1f e62/33 3:f84a2a2d:::10000000c60.00000000:head v 62'45, mlcod=62'43)", 2024-09-16T00:09:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:03.440894+0000", 2024-09-16T00:09:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.029658668, 2024-09-16T00:09:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028251109999999999, 2024-09-16T00:09:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.440894+0000", 2024-09-16T00:09:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.440894+0000", 2024-09-16T00:09:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.440894+0000", 2024-09-16T00:09:20.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7700000000000004e-07 2024-09-16T00:09:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441598+0000", 2024-09-16T00:09:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00070308100000000004 2024-09-16T00:09:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441607+0000", 2024-09-16T00:09:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7489999999999993e-06 2024-09-16T00:09:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441612+0000", 2024-09-16T00:09:20.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6519999999999997e-06 2024-09-16T00:09:20.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441650+0000", 2024-09-16T00:09:20.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7867000000000002e-05 2024-09-16T00:09:20.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441663+0000", 2024-09-16T00:09:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2788e-05 2024-09-16T00:09:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.443681+0000", 2024-09-16T00:09:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020186599999999998 2024-09-16T00:09:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.443719+0000", 2024-09-16T00:09:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7636999999999999e-05 2024-09-16T00:09:20.587 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:2326 3.1c 3:39be8b5c:::10000000c6c.00000000:head [write 0~173460] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T00:09:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:03.473107+0000", 2024-09-16T00:09:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 16.997445383999999, 2024-09-16T00:09:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001822029, 2024-09-16T00:09:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.588 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 2326 2024-09-16T00:09:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473107+0000", 2024-09-16T00:09:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473107+0000", 2024-09-16T00:09:20.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473108+0000", 2024-09-16T00:09:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0139999999999999e-06 2024-09-16T00:09:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473292+0000", 2024-09-16T00:09:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000184072 2024-09-16T00:09:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473305+0000", 2024-09-16T00:09:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2444e-05 2024-09-16T00:09:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473310+0000", 2024-09-16T00:09:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5720000000000002e-06 2024-09-16T00:09:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473353+0000", 2024-09-16T00:09:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3041999999999999e-05 2024-09-16T00:09:20.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473455+0000", 2024-09-16T00:09:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101297 2024-09-16T00:09:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473542+0000", 2024-09-16T00:09:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7787999999999994e-05 2024-09-16T00:09:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.474469+0000", 2024-09-16T00:09:20.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00092618500000000005 2024-09-16T00:09:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.474868+0000", 2024-09-16T00:09:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00039969400000000003 2024-09-16T00:09:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.474900+0000", 2024-09-16T00:09:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1341999999999999e-05 2024-09-16T00:09:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.474929+0000", 2024-09-16T00:09:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9578999999999999e-05 2024-09-16T00:09:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:6120 3.37 e69/20 3:ef2e412a:::10000001c11.00000000:head v 69'100, mlcod=68'98)", 2024-09-16T00:09:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:16.085749+0000", 2024-09-16T00:09:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3848032449999996, 2024-09-16T00:09:20.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020186219999999999, 2024-09-16T00:09:20.595 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:20.595 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:20.595 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:20.595 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:20.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:20.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085749+0000", 2024-09-16T00:09:20.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:20.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085749+0000", 2024-09-16T00:09:20.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:20.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:20.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085749+0000", 2024-09-16T00:09:20.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3200000000000001e-07 2024-09-16T00:09:20.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:20.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085767+0000", 2024-09-16T00:09:20.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7212000000000001e-05 2024-09-16T00:09:20.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:20.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085775+0000", 2024-09-16T00:09:20.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8590000000000005e-06 2024-09-16T00:09:20.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:20.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085780+0000", 2024-09-16T00:09:20.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.989e-06 2024-09-16T00:09:20.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:20.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085825+0000", 2024-09-16T00:09:20.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5090999999999999e-05 2024-09-16T00:09:20.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:20.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085845+0000", 2024-09-16T00:09:20.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0412999999999999e-05 2024-09-16T00:09:20.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:20.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.087730+0000", 2024-09-16T00:09:20.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001884817 2024-09-16T00:09:20.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:20.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:20.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:20.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.087768+0000", 2024-09-16T00:09:20.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8009000000000001e-05 2024-09-16T00:09:20.600 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.600 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.600 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.600 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:20.600 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:20.600 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:20.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:20.498+0000 7fc114277640 1 -- 172.21.15.139:0/1159031052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) -- 0x7fc10c0cca70 con 0x7fc10c0ccc90 2024-09-16T00:09:20.603 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:09:20.603 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:20.600+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:20.702 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:20.701+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:20.766 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:20.766 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:20.766 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:20.766 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:20.775 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:09:20.816 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:20.815+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:20.904 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:20.903+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:20.929 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:20.929 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:20.929 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:20.929 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:20.930 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:20.938 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:09:21.005 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:21.004+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:21.092 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:21.093 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:21.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:21.093 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:21.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.093 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:21.093 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:09:21.093 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.475098742, 2024-09-16T00:09:21.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056100969999999997, 2024-09-16T00:09:21.093 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.093 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.093 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.094 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:09:21.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:09:21.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615352+0000", 2024-09-16T00:09:21.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9859999999999999e-06 2024-09-16T00:09:21.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615363+0000", 2024-09-16T00:09:21.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1168999999999999e-05 2024-09-16T00:09:21.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615382+0000", 2024-09-16T00:09:21.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8224e-05 2024-09-16T00:09:21.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615393+0000", 2024-09-16T00:09:21.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0925e-05 2024-09-16T00:09:21.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615624+0000", 2024-09-16T00:09:21.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023159600000000001 2024-09-16T00:09:21.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615721+0000", 2024-09-16T00:09:21.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7040000000000006e-05 2024-09-16T00:09:21.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:21.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615885+0000", 2024-09-16T00:09:21.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163456 2024-09-16T00:09:21.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:09:21.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002191368 2024-09-16T00:09:21.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620870+0000", 2024-09-16T00:09:21.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941839999999999 2024-09-16T00:09:21.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620922+0000", 2024-09-16T00:09:21.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1641999999999998e-05 2024-09-16T00:09:21.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620956+0000", 2024-09-16T00:09:21.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4507e-05 2024-09-16T00:09:21.098 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.099 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.099 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:21.099 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:09:21.099 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.475008383, 2024-09-16T00:09:21.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005695739, 2024-09-16T00:09:21.099 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.099 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.099 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.099 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:09:21.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:09:21.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:09:21.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4900000000000005e-07 2024-09-16T00:09:21.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615442+0000", 2024-09-16T00:09:21.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.994e-06 2024-09-16T00:09:21.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:09:21.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1081e-05 2024-09-16T00:09:21.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615473+0000", 2024-09-16T00:09:21.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1910000000000006e-06 2024-09-16T00:09:21.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616230+0000", 2024-09-16T00:09:21.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075700400000000003 2024-09-16T00:09:21.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616305+0000", 2024-09-16T00:09:21.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4759999999999996e-05 2024-09-16T00:09:21.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:21.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616466+0000", 2024-09-16T00:09:21.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000161714 2024-09-16T00:09:21.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619821+0000", 2024-09-16T00:09:21.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033547339999999998 2024-09-16T00:09:21.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621028+0000", 2024-09-16T00:09:21.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012067530000000001 2024-09-16T00:09:21.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621087+0000", 2024-09-16T00:09:21.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9364999999999998e-05 2024-09-16T00:09:21.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621132+0000", 2024-09-16T00:09:21.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5293999999999997e-05 2024-09-16T00:09:21.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.104 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.104 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.105 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:21.105 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:09:21.105 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.474946151, 2024-09-16T00:09:21.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050565760000000001, 2024-09-16T00:09:21.105 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.105 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.105 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.105 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:09:21.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:09:21.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615500+0000", 2024-09-16T00:09:21.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8199999999999999e-07 2024-09-16T00:09:21.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615505+0000", 2024-09-16T00:09:21.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5280000000000003e-06 2024-09-16T00:09:21.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:21.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0317e-05 2024-09-16T00:09:21.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615523+0000", 2024-09-16T00:09:21.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4580000000000004e-06 2024-09-16T00:09:21.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615607+0000", 2024-09-16T00:09:21.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4153999999999994e-05 2024-09-16T00:09:21.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615748+0000", 2024-09-16T00:09:21.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140533 2024-09-16T00:09:21.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:21.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615911+0000", 2024-09-16T00:09:21.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163678 2024-09-16T00:09:21.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619944+0000", 2024-09-16T00:09:21.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040330909999999999 2024-09-16T00:09:21.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620471+0000", 2024-09-16T00:09:21.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052645600000000002 2024-09-16T00:09:21.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620512+0000", 2024-09-16T00:09:21.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0738000000000002e-05 2024-09-16T00:09:21.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620555+0000", 2024-09-16T00:09:21.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3841000000000003e-05 2024-09-16T00:09:21.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.110 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.110 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.110 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:21.111 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:09:21.111 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.474894044, 2024-09-16T00:09:21.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089162479999999999, 2024-09-16T00:09:21.111 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.111 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.111 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.111 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:09:21.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:09:21.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615552+0000", 2024-09-16T00:09:21.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:09:21.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615556+0000", 2024-09-16T00:09:21.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3610000000000003e-06 2024-09-16T00:09:21.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615566+0000", 2024-09-16T00:09:21.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6099999999999995e-06 2024-09-16T00:09:21.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615574+0000", 2024-09-16T00:09:21.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9379999999999992e-06 2024-09-16T00:09:21.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616777+0000", 2024-09-16T00:09:21.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012031640000000001 2024-09-16T00:09:21.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616882+0000", 2024-09-16T00:09:21.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105146 2024-09-16T00:09:21.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:21.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619696+0000", 2024-09-16T00:09:21.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028142179999999998 2024-09-16T00:09:21.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624234+0000", 2024-09-16T00:09:21.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045381889999999998 2024-09-16T00:09:21.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624393+0000", 2024-09-16T00:09:21.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015871200000000001 2024-09-16T00:09:21.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624434+0000", 2024-09-16T00:09:21.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0908999999999997e-05 2024-09-16T00:09:21.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624467+0000", 2024-09-16T00:09:21.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3217999999999999e-05 2024-09-16T00:09:21.116 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.116 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.116 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.117 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:21.117 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:09:21.117 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.474824247, 2024-09-16T00:09:21.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064898359999999997, 2024-09-16T00:09:21.117 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.117 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.117 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.117 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:09:21.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:09:21.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615622+0000", 2024-09-16T00:09:21.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.181e-06 2024-09-16T00:09:21.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615628+0000", 2024-09-16T00:09:21.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2450000000000003e-06 2024-09-16T00:09:21.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:09:21.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2554999999999999e-05 2024-09-16T00:09:21.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615650+0000", 2024-09-16T00:09:21.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4350000000000003e-06 2024-09-16T00:09:21.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616772+0000", 2024-09-16T00:09:21.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001122058 2024-09-16T00:09:21.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616889+0000", 2024-09-16T00:09:21.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116623 2024-09-16T00:09:21.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:21.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617035+0000", 2024-09-16T00:09:21.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014579799999999999 2024-09-16T00:09:21.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619829+0000", 2024-09-16T00:09:21.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941379999999998 2024-09-16T00:09:21.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622024+0000", 2024-09-16T00:09:21.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021949920000000002 2024-09-16T00:09:21.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622087+0000", 2024-09-16T00:09:21.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3048999999999999e-05 2024-09-16T00:09:21.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622111+0000", 2024-09-16T00:09:21.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3762000000000001e-05 2024-09-16T00:09:21.123 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.123 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.123 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.124 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:21.124 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:09:21.124 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.47476168599999, 2024-09-16T00:09:21.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065192160000000004, 2024-09-16T00:09:21.124 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.124 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.124 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.124 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:09:21.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:09:21.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615684+0000", 2024-09-16T00:09:21.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8299999999999995e-07 2024-09-16T00:09:21.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615691+0000", 2024-09-16T00:09:21.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.866e-06 2024-09-16T00:09:21.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615707+0000", 2024-09-16T00:09:21.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6073999999999999e-05 2024-09-16T00:09:21.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615714+0000", 2024-09-16T00:09:21.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-16T00:09:21.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617403+0000", 2024-09-16T00:09:21.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016881629999999999 2024-09-16T00:09:21.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617559+0000", 2024-09-16T00:09:21.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015669699999999999 2024-09-16T00:09:21.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:21.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:09:21.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000144165 2024-09-16T00:09:21.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621413+0000", 2024-09-16T00:09:21.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037092200000000001 2024-09-16T00:09:21.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.129 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.129 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622147+0000", 2024-09-16T00:09:21.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073468800000000005 2024-09-16T00:09:21.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622180+0000", 2024-09-16T00:09:21.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2901999999999998e-05 2024-09-16T00:09:21.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622203+0000", 2024-09-16T00:09:21.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2211000000000001e-05 2024-09-16T00:09:21.130 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.130 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.130 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.131 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:21.131 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:09:21.131 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.474718592, 2024-09-16T00:09:21.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065825270000000003, 2024-09-16T00:09:21.131 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.131 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.131 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:09:21.131 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:09:21.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:09:21.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:09:21.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7299999999999996e-07 2024-09-16T00:09:21.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615730+0000", 2024-09-16T00:09:21.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.004e-06 2024-09-16T00:09:21.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615737+0000", 2024-09-16T00:09:21.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4789999999999999e-06 2024-09-16T00:09:21.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615745+0000", 2024-09-16T00:09:21.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6699999999999994e-06 2024-09-16T00:09:21.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618115+0000", 2024-09-16T00:09:21.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023701569999999999 2024-09-16T00:09:21.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618247+0000", 2024-09-16T00:09:21.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013185000000000001 2024-09-16T00:09:21.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622309+0000", 2024-09-16T00:09:21.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040617939999999996 2024-09-16T00:09:21.134 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.134 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.135 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.135 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:21.135 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:09:21.135 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.474691343, 2024-09-16T00:09:21.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036685950000000002, 2024-09-16T00:09:21.135 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.135 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.135 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:09:21.135 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:09:21.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:09:21.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:09:21.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T00:09:21.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615763+0000", 2024-09-16T00:09:21.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4360000000000002e-06 2024-09-16T00:09:21.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615770+0000", 2024-09-16T00:09:21.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7540000000000006e-06 2024-09-16T00:09:21.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615803+0000", 2024-09-16T00:09:21.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3008e-05 2024-09-16T00:09:21.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615888+0000", 2024-09-16T00:09:21.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5056999999999996e-05 2024-09-16T00:09:21.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:09:21.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013059700000000001 2024-09-16T00:09:21.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619422+0000", 2024-09-16T00:09:21.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034033230000000002 2024-09-16T00:09:21.139 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.139 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.139 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.139 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:16 2.1d e22/19 2:b871830b:::605.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:21.139 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:09:21.139 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.470702753, 2024-09-16T00:09:21.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045616900000000002, 2024-09-16T00:09:21.139 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.139 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.139 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.140 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:09:21.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:09:21.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:09:21.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:09:21.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619755+0000", 2024-09-16T00:09:21.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3599999999999996e-06 2024-09-16T00:09:21.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619767+0000", 2024-09-16T00:09:21.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2805e-05 2024-09-16T00:09:21.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619787+0000", 2024-09-16T00:09:21.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9264000000000001e-05 2024-09-16T00:09:21.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619869+0000", 2024-09-16T00:09:21.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2639999999999995e-05 2024-09-16T00:09:21.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619895+0000", 2024-09-16T00:09:21.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5890000000000001e-05 2024-09-16T00:09:21.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624264+0000", 2024-09-16T00:09:21.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043690049999999996 2024-09-16T00:09:21.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624304+0000", 2024-09-16T00:09:21.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9807e-05 2024-09-16T00:09:21.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.143 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.143 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.144 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:17 2.1d e22/19 2:b871830b:::605.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:21.144 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:09:21.144 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.470350044, 2024-09-16T00:09:21.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042370910000000001, 2024-09-16T00:09:21.144 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.144 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.144 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.144 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:09:21.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:09:21.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620096+0000", 2024-09-16T00:09:21.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:09:21.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620102+0000", 2024-09-16T00:09:21.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7220000000000002e-06 2024-09-16T00:09:21.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620112+0000", 2024-09-16T00:09:21.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0156999999999999e-05 2024-09-16T00:09:21.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.146 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.146 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620120+0000", 2024-09-16T00:09:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2880000000000004e-06 2024-09-16T00:09:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620171+0000", 2024-09-16T00:09:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.168e-05 2024-09-16T00:09:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620189+0000", 2024-09-16T00:09:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7419e-05 2024-09-16T00:09:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624306+0000", 2024-09-16T00:09:21.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041175430000000004 2024-09-16T00:09:21.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624332+0000", 2024-09-16T00:09:21.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5717000000000001e-05 2024-09-16T00:09:21.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.148 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.148 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.148 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:09:21.149 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:09:21.149 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 102.467983227, 2024-09-16T00:09:21.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044651079999999998, 2024-09-16T00:09:21.149 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.149 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.149 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.149 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:09:21.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:09:21.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622463+0000", 2024-09-16T00:09:21.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5349999999999999e-06 2024-09-16T00:09:21.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622470+0000", 2024-09-16T00:09:21.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7050000000000002e-06 2024-09-16T00:09:21.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622485+0000", 2024-09-16T00:09:21.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4978e-05 2024-09-16T00:09:21.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622493+0000", 2024-09-16T00:09:21.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2139999999999996e-06 2024-09-16T00:09:21.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622546+0000", 2024-09-16T00:09:21.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:09:21.152 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622595+0000", 2024-09-16T00:09:21.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9112e-05 2024-09-16T00:09:21.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:21.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622677+0000", 2024-09-16T00:09:21.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2332999999999999e-05 2024-09-16T00:09:21.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.623515+0000", 2024-09-16T00:09:21.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000837615 2024-09-16T00:09:21.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626867+0000", 2024-09-16T00:09:21.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033524819999999999 2024-09-16T00:09:21.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626902+0000", 2024-09-16T00:09:21.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4310000000000002e-05 2024-09-16T00:09:21.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626927+0000", 2024-09-16T00:09:21.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5128999999999999e-05 2024-09-16T00:09:21.155 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.155 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.155 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.155 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:103 3.26 3:6519881c:::1000000025e.00000000:head [write 0~2892] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:21.155 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.487113+0000", 2024-09-16T00:09:21.155 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.603332078000001, 2024-09-16T00:09:21.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034171290000000001, 2024-09-16T00:09:21.156 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.156 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.156 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:21.156 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:21.156 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103 2024-09-16T00:09:21.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.156 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487113+0000", 2024-09-16T00:09:21.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487113+0000", 2024-09-16T00:09:21.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487114+0000", 2024-09-16T00:09:21.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3030000000000001e-06 2024-09-16T00:09:21.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487122+0000", 2024-09-16T00:09:21.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3889999999999999e-06 2024-09-16T00:09:21.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487133+0000", 2024-09-16T00:09:21.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1325000000000001e-05 2024-09-16T00:09:21.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487138+0000", 2024-09-16T00:09:21.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0660000000000003e-06 2024-09-16T00:09:21.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487182+0000", 2024-09-16T00:09:21.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3655999999999997e-05 2024-09-16T00:09:21.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487307+0000", 2024-09-16T00:09:21.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012541599999999999 2024-09-16T00:09:21.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:21.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487391+0000", 2024-09-16T00:09:21.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4404e-05 2024-09-16T00:09:21.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487908+0000", 2024-09-16T00:09:21.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000516608 2024-09-16T00:09:21.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.490429+0000", 2024-09-16T00:09:21.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025209339999999998 2024-09-16T00:09:21.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.490482+0000", 2024-09-16T00:09:21.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2655000000000003e-05 2024-09-16T00:09:21.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.490530+0000", 2024-09-16T00:09:21.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8372999999999999e-05 2024-09-16T00:09:21.162 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.162 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.162 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.162 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:53 2.14 2:2e2fa760:::200.00000001:head [write 20850~1000165 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e58)", 2024-09-16T00:09:21.162 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.452702+0000", 2024-09-16T00:09:21.163 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.637742530000001, 2024-09-16T00:09:21.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046625019999999998, 2024-09-16T00:09:21.163 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.163 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.163 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.163 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.452702+0000", 2024-09-16T00:09:21.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.452702+0000", 2024-09-16T00:09:21.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.452704+0000", 2024-09-16T00:09:21.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4699999999999999e-06 2024-09-16T00:09:21.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453184+0000", 2024-09-16T00:09:21.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048001000000000001 2024-09-16T00:09:21.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453199+0000", 2024-09-16T00:09:21.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4905e-05 2024-09-16T00:09:21.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453222+0000", 2024-09-16T00:09:21.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3587999999999998e-05 2024-09-16T00:09:21.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453288+0000", 2024-09-16T00:09:21.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5817999999999999e-05 2024-09-16T00:09:21.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453348+0000", 2024-09-16T00:09:21.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9716999999999998e-05 2024-09-16T00:09:21.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:21.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453440+0000", 2024-09-16T00:09:21.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2090999999999996e-05 2024-09-16T00:09:21.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.455907+0000", 2024-09-16T00:09:21.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024670730000000002 2024-09-16T00:09:21.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.457293+0000", 2024-09-16T00:09:21.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013861979999999999 2024-09-16T00:09:21.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.457326+0000", 2024-09-16T00:09:21.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2484000000000003e-05 2024-09-16T00:09:21.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.457365+0000", 2024-09-16T00:09:21.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9147999999999999e-05 2024-09-16T00:09:21.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.169 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.169 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.169 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:55 2.14 2:2e2fa760:::200.00000001:head [write 1021015~941162 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e59)", 2024-09-16T00:09:21.169 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:58.642546+0000", 2024-09-16T00:09:21.169 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.447899412000002, 2024-09-16T00:09:21.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041709410000000001, 2024-09-16T00:09:21.170 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.170 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.170 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.170 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.642546+0000", 2024-09-16T00:09:21.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.642546+0000", 2024-09-16T00:09:21.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.642546+0000", 2024-09-16T00:09:21.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5499999999999997e-07 2024-09-16T00:09:21.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643041+0000", 2024-09-16T00:09:21.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049419399999999999 2024-09-16T00:09:21.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643051+0000", 2024-09-16T00:09:21.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0229e-05 2024-09-16T00:09:21.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643056+0000", 2024-09-16T00:09:21.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.327e-06 2024-09-16T00:09:21.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643112+0000", 2024-09-16T00:09:21.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5939000000000001e-05 2024-09-16T00:09:21.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643193+0000", 2024-09-16T00:09:21.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0921999999999995e-05 2024-09-16T00:09:21.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:21.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643305+0000", 2024-09-16T00:09:21.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011162099999999999 2024-09-16T00:09:21.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.645976+0000", 2024-09-16T00:09:21.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026712609999999999 2024-09-16T00:09:21.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.646651+0000", 2024-09-16T00:09:21.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00067499200000000002 2024-09-16T00:09:21.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.646684+0000", 2024-09-16T00:09:21.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3328999999999999e-05 2024-09-16T00:09:21.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.646717+0000", 2024-09-16T00:09:21.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2271999999999999e-05 2024-09-16T00:09:21.175 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.175 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.175 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.175 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:1617 3.36 3:6fb28598:::1000000094d.00000000:head [write 0~2552] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T00:09:21.175 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.251180+0000", 2024-09-16T00:09:21.175 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.839265497, 2024-09-16T00:09:21.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004599601, 2024-09-16T00:09:21.175 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.175 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.176 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.176 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.176 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:21.176 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:21.176 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1617 2024-09-16T00:09:21.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.176 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251180+0000", 2024-09-16T00:09:21.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251180+0000", 2024-09-16T00:09:21.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251180+0000", 2024-09-16T00:09:21.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4600000000000003e-07 2024-09-16T00:09:21.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251185+0000", 2024-09-16T00:09:21.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3239999999999996e-06 2024-09-16T00:09:21.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251194+0000", 2024-09-16T00:09:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3049999999999998e-06 2024-09-16T00:09:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251199+0000", 2024-09-16T00:09:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2780000000000001e-06 2024-09-16T00:09:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251240+0000", 2024-09-16T00:09:21.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1171999999999998e-05 2024-09-16T00:09:21.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251317+0000", 2024-09-16T00:09:21.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6804999999999994e-05 2024-09-16T00:09:21.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:21.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251399+0000", 2024-09-16T00:09:21.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2101000000000006e-05 2024-09-16T00:09:21.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255497+0000", 2024-09-16T00:09:21.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040972889999999996 2024-09-16T00:09:21.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255687+0000", 2024-09-16T00:09:21.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019032899999999999 2024-09-16T00:09:21.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255739+0000", 2024-09-16T00:09:21.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2086999999999998e-05 2024-09-16T00:09:21.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255779+0000", 2024-09-16T00:09:21.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0064999999999997e-05 2024-09-16T00:09:21.181 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.181 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.181 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.181 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:72 2.14 2:2e2fa760:::200.00000001:head [write 3118411~1075893 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:21.181 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.671915+0000", 2024-09-16T00:09:21.181 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.418529562, 2024-09-16T00:09:21.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049035859999999997, 2024-09-16T00:09:21.181 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.182 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.182 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.182 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.671915+0000", 2024-09-16T00:09:21.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.671915+0000", 2024-09-16T00:09:21.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.671917+0000", 2024-09-16T00:09:21.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7e-06 2024-09-16T00:09:21.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672671+0000", 2024-09-16T00:09:21.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075343199999999995 2024-09-16T00:09:21.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672686+0000", 2024-09-16T00:09:21.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4960000000000001e-05 2024-09-16T00:09:21.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672693+0000", 2024-09-16T00:09:21.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0770000000000002e-06 2024-09-16T00:09:21.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672737+0000", 2024-09-16T00:09:21.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4614999999999999e-05 2024-09-16T00:09:21.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672793+0000", 2024-09-16T00:09:21.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5785999999999997e-05 2024-09-16T00:09:21.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:21.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672870+0000", 2024-09-16T00:09:21.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7187999999999993e-05 2024-09-16T00:09:21.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.675448+0000", 2024-09-16T00:09:21.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002578033 2024-09-16T00:09:21.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.676746+0000", 2024-09-16T00:09:21.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001297896 2024-09-16T00:09:21.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.676778+0000", 2024-09-16T00:09:21.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1605e-05 2024-09-16T00:09:21.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.676819+0000", 2024-09-16T00:09:21.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1294000000000001e-05 2024-09-16T00:09:21.186 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.187 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.187 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:75 2.11 2:8fcca0a5:::10000000261.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:21.187 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.985646+0000", 2024-09-16T00:09:21.187 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.104799156999999, 2024-09-16T00:09:21.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041697660000000001, 2024-09-16T00:09:21.187 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.188 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.188 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.188 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985646+0000", 2024-09-16T00:09:21.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985646+0000", 2024-09-16T00:09:21.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985648+0000", 2024-09-16T00:09:21.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5739999999999998e-06 2024-09-16T00:09:21.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985699+0000", 2024-09-16T00:09:21.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0513999999999998e-05 2024-09-16T00:09:21.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985718+0000", 2024-09-16T00:09:21.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8689999999999999e-05 2024-09-16T00:09:21.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985729+0000", 2024-09-16T00:09:21.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1812e-05 2024-09-16T00:09:21.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985802+0000", 2024-09-16T00:09:21.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2465999999999994e-05 2024-09-16T00:09:21.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985912+0000", 2024-09-16T00:09:21.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109795 2024-09-16T00:09:21.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:21.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.986236+0000", 2024-09-16T00:09:21.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032388799999999999 2024-09-16T00:09:21.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.987505+0000", 2024-09-16T00:09:21.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001269788 2024-09-16T00:09:21.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.989708+0000", 2024-09-16T00:09:21.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022022579999999999 2024-09-16T00:09:21.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.989765+0000", 2024-09-16T00:09:21.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7268000000000001e-05 2024-09-16T00:09:21.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.989816+0000", 2024-09-16T00:09:21.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0713e-05 2024-09-16T00:09:21.194 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.194 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.194 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.194 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:84 2.17 2:ee44d6a7:::10000000261.03e00000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:21.194 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.076954+0000", 2024-09-16T00:09:21.194 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.013490971, 2024-09-16T00:09:21.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004155237, 2024-09-16T00:09:21.195 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.195 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.195 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.195 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076954+0000", 2024-09-16T00:09:21.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076954+0000", 2024-09-16T00:09:21.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076955+0000", 2024-09-16T00:09:21.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4199999999999996e-07 2024-09-16T00:09:21.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076971+0000", 2024-09-16T00:09:21.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6294999999999999e-05 2024-09-16T00:09:21.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076980+0000", 2024-09-16T00:09:21.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5599999999999999e-06 2024-09-16T00:09:21.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076986+0000", 2024-09-16T00:09:21.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9329999999999998e-06 2024-09-16T00:09:21.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077032+0000", 2024-09-16T00:09:21.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5566999999999999e-05 2024-09-16T00:09:21.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077143+0000", 2024-09-16T00:09:21.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011068 2024-09-16T00:09:21.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:21.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077295+0000", 2024-09-16T00:09:21.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152578 2024-09-16T00:09:21.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078459+0000", 2024-09-16T00:09:21.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001163645 2024-09-16T00:09:21.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081048+0000", 2024-09-16T00:09:21.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025893840000000001 2024-09-16T00:09:21.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081084+0000", 2024-09-16T00:09:21.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5834999999999998e-05 2024-09-16T00:09:21.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081109+0000", 2024-09-16T00:09:21.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5218000000000001e-05 2024-09-16T00:09:21.200 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.200 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.200 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.200 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:224 2.1a 2:5e41ab69:::200.00000005:head [write 0~1943310 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e66)", 2024-09-16T00:09:21.200 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:12.641517+0000", 2024-09-16T00:09:21.200 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.4489278930000005, 2024-09-16T00:09:21.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064053460000000001, 2024-09-16T00:09:21.201 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.201 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.201 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.201 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.641517+0000", 2024-09-16T00:09:21.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.641517+0000", 2024-09-16T00:09:21.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.641520+0000", 2024-09-16T00:09:21.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5660000000000001e-06 2024-09-16T00:09:21.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.642832+0000", 2024-09-16T00:09:21.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00131274 2024-09-16T00:09:21.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.642853+0000", 2024-09-16T00:09:21.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0710999999999999e-05 2024-09-16T00:09:21.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.642880+0000", 2024-09-16T00:09:21.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6693999999999999e-05 2024-09-16T00:09:21.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.642939+0000", 2024-09-16T00:09:21.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9184000000000001e-05 2024-09-16T00:09:21.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.643032+0000", 2024-09-16T00:09:21.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2583000000000004e-05 2024-09-16T00:09:21.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:21.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.643114+0000", 2024-09-16T00:09:21.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2554999999999998e-05 2024-09-16T00:09:21.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.647381+0000", 2024-09-16T00:09:21.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042663969999999999 2024-09-16T00:09:21.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.647813+0000", 2024-09-16T00:09:21.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043280899999999998 2024-09-16T00:09:21.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.647873+0000", 2024-09-16T00:09:21.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9619e-05 2024-09-16T00:09:21.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.647922+0000", 2024-09-16T00:09:21.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9487999999999998e-05 2024-09-16T00:09:21.208 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.208 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.208 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.208 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.208 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:327 2.1d e73/46 2:bec1251c:::200.00000007:head v 73'7, mlcod=63'5)", 2024-09-16T00:09:21.208 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:20.724139+0000", 2024-09-16T00:09:21.208 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.36630603900000003, 2024-09-16T00:09:21.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049608899999999999, 2024-09-16T00:09:21.208 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.208 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.209 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.209 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.724139+0000", 2024-09-16T00:09:21.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.724139+0000", 2024-09-16T00:09:21.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.724141+0000", 2024-09-16T00:09:21.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7850000000000001e-06 2024-09-16T00:09:21.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.725369+0000", 2024-09-16T00:09:21.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001227978 2024-09-16T00:09:21.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.725383+0000", 2024-09-16T00:09:21.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3839000000000001e-05 2024-09-16T00:09:21.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.725391+0000", 2024-09-16T00:09:21.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5939999999999994e-06 2024-09-16T00:09:21.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.725469+0000", 2024-09-16T00:09:21.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7404999999999995e-05 2024-09-16T00:09:21.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.725487+0000", 2024-09-16T00:09:21.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8766e-05 2024-09-16T00:09:21.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.729063+0000", 2024-09-16T00:09:21.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035752900000000001 2024-09-16T00:09:21.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.729100+0000", 2024-09-16T00:09:21.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7233000000000001e-05 2024-09-16T00:09:21.212 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.212 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.212 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.212 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.213 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.213 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:21.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.104+0000 7f12c37fe640 1 -- 172.21.15.139:0/345158618 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v74) ==== 97+0+0 (secure 0 0 0) 0x7f12bc059a30 con 0x7f12cc154c20 2024-09-16T00:09:21.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.104+0000 7fc102ffd640 1 -- 172.21.15.139:0/1159031052 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v74) ==== 113+0+0 (secure 0 0 0) 0x7fc0fc079ca0 con 0x7fc10c0ccc90 2024-09-16T00:09:21.215 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is set 2024-09-16T00:09:21.215 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T00:09:21.215 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:09:21.216 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:21.212+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:21.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.106+0000 7f12d1465640 1 -- 172.21.15.139:0/345158618 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f12ac06c2b0 msgr2=0x7f12ac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:21.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.106+0000 7f12d1465640 1 --2- 172.21.15.139:0/345158618 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f12ac06c2b0 0x7f12ac06e770 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009b40 tx=0x7f12c4009190 comp rx=0 tx=0).stop 2024-09-16T00:09:21.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.106+0000 7f12d1465640 1 -- 172.21.15.139:0/345158618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12cc154c20 msgr2=0x7f12cc149cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:21.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.106+0000 7f12d1465640 1 --2- 172.21.15.139:0/345158618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12cc154c20 0x7f12cc149cb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00eed0 tx=0x7f12bc00b480 comp rx=0 tx=0).stop 2024-09-16T00:09:21.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.106+0000 7f12d1465640 1 -- 172.21.15.139:0/345158618 shutdown_connections 2024-09-16T00:09:21.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7f12d1465640 1 --2- 172.21.15.139:0/345158618 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f12ac06c2b0 0x7f12ac06e770 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7f12d1465640 1 --2- 172.21.15.139:0/345158618 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f12cc154c20 0x7f12cc149cb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7f12d1465640 1 --2- 172.21.15.139:0/345158618 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f12cc0ccc90 0x7f12cc1f15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7f12d1465640 1 --2- 172.21.15.139:0/345158618 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f12cc068060 0x7f12cc1f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7f12d1465640 1 -- 172.21.15.139:0/345158618 >> 172.21.15.139:0/345158618 conn(0x7f12cc06ca40 msgr2=0x7f12cc0c6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:21.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7fc114277640 1 -- 172.21.15.139:0/1159031052 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc0f006c2e0 msgr2=0x7fc0f006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:21.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7fc114277640 1 --2- 172.21.15.139:0/1159031052 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc0f006c2e0 0x7fc0f006e7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc108007fd0 tx=0x7fc108007de0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7fc114277640 1 -- 172.21.15.139:0/1159031052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10c0ccc90 msgr2=0x7fc10c150400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:21.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7fc114277640 1 --2- 172.21.15.139:0/1159031052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10c0ccc90 0x7fc10c150400 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc000c00 tx=0x7fc0fc0046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7f12d1465640 1 -- 172.21.15.139:0/345158618 shutdown_connections 2024-09-16T00:09:21.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7f12d1465640 1 -- 172.21.15.139:0/345158618 wait complete. 2024-09-16T00:09:21.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7fc114277640 1 -- 172.21.15.139:0/1159031052 shutdown_connections 2024-09-16T00:09:21.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7fc114277640 1 --2- 172.21.15.139:0/1159031052 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc0f006c2e0 0x7fc0f006e7a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7fc114277640 1 --2- 172.21.15.139:0/1159031052 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc10c154c20 0x7fc10c1f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7fc114277640 1 --2- 172.21.15.139:0/1159031052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc10c0ccc90 0x7fc10c150400 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7fc114277640 1 --2- 172.21.15.139:0/1159031052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc10c068060 0x7fc10c159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7fc114277640 1 -- 172.21.15.139:0/1159031052 >> 172.21.15.139:0/1159031052 conn(0x7fc10c06ca40 msgr2=0x7fc10c0c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:21.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7fc114277640 1 -- 172.21.15.139:0/1159031052 shutdown_connections 2024-09-16T00:09:21.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.107+0000 7fc114277640 1 -- 172.21.15.139:0/1159031052 wait complete. 2024-09-16T00:09:21.221 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T00:09:21.221 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:09:21.315 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:21.314+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:21.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.357+0000 7f8f79e27640 1 Processor -- start 2024-09-16T00:09:21.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.357+0000 7f8f79e27640 1 -- start start 2024-09-16T00:09:21.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.358+0000 7f8f79e27640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8f740c7cb0 0x7f8f74154580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:21.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.358+0000 7f8f79e27640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f7406f150 0x7f8f74154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:21.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.358+0000 7f8f79e27640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f740c9890 con 0x7f8f7406ed70 2024-09-16T00:09:21.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.358+0000 7f8f79e27640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8f7406c540 con 0x7f8f740c7cb0 2024-09-16T00:09:21.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.358+0000 7f8f79e27640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7406c820 con 0x7f8f7406f150 2024-09-16T00:09:21.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.358+0000 7f8f72ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8f740c7cb0 0x7f8f74154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:21.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.358+0000 7f8f737fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8f7406ed70 0x7f8f7415a950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41646/0 (socket says 172.21.15.139:41646) 2024-09-16T00:09:21.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.358+0000 7f8f737fe640 1 -- 172.21.15.139:0/1051321014 learned_addr learned my addr 172.21.15.139:0/1051321014 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:21.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.358+0000 7f8f73fff640 1 --2- 172.21.15.139:0/1051321014 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f7406f150 0x7f8f74154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:21.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.358+0000 7f8f72ffd640 1 -- 172.21.15.139:0/1051321014 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f7406f150 msgr2=0x7f8f74154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:21.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.358+0000 7f8f72ffd640 1 --2- 172.21.15.139:0/1051321014 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f7406f150 0x7f8f74154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.358+0000 7f8f72ffd640 1 -- 172.21.15.139:0/1051321014 >> v1:172.21.15.139:6789/0 conn(0x7f8f7406ed70 legacy=0x7f8f7415a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:21.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.358+0000 7f8f727fc640 1 -- 172.21.15.139:0/1051321014 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 884750848 0 0) 0x7f8f740c9890 con 0x7f8f7406ed70 2024-09-16T00:09:21.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.358+0000 7f8f72ffd640 1 -- 172.21.15.139:0/1051321014 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7406cb00 con 0x7f8f740c7cb0 2024-09-16T00:09:21.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.359+0000 7f8f72ffd640 1 --2- 172.21.15.139:0/1051321014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f740c7cb0 0x7f8f74154580 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8f680097d0 tx=0x7f8f6802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2abfddb08a1265b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:21.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.359+0000 7f8f727fc640 1 -- 172.21.15.139:0/1051321014 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68037070 con 0x7f8f740c7cb0 2024-09-16T00:09:21.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.359+0000 7f8f727fc640 1 -- 172.21.15.139:0/1051321014 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f68032440 con 0x7f8f740c7cb0 2024-09-16T00:09:21.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.359+0000 7f8f727fc640 1 -- 172.21.15.139:0/1051321014 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68032810 con 0x7f8f740c7cb0 2024-09-16T00:09:21.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.360+0000 7f8f79e27640 1 -- 172.21.15.139:0/1051321014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f740c7cb0 msgr2=0x7f8f74154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:21.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.360+0000 7f8f79e27640 1 --2- 172.21.15.139:0/1051321014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f740c7cb0 0x7f8f74154580 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8f680097d0 tx=0x7f8f6802a250 comp rx=0 tx=0).stop 2024-09-16T00:09:21.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.360+0000 7f8f79e27640 1 -- 172.21.15.139:0/1051321014 shutdown_connections 2024-09-16T00:09:21.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.360+0000 7f8f79e27640 1 --2- 172.21.15.139:0/1051321014 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f7406f150 0x7f8f74154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.360+0000 7f8f79e27640 1 --2- 172.21.15.139:0/1051321014 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f740c7cb0 0x7f8f74154580 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.360+0000 7f8f79e27640 1 -- 172.21.15.139:0/1051321014 >> 172.21.15.139:0/1051321014 conn(0x7f8f74067950 msgr2=0x7f8f74067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:21.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.360+0000 7f8f79e27640 1 -- 172.21.15.139:0/1051321014 shutdown_connections 2024-09-16T00:09:21.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.360+0000 7f8f79e27640 1 -- 172.21.15.139:0/1051321014 wait complete. 2024-09-16T00:09:21.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.361+0000 7f8f79e27640 1 Processor -- start 2024-09-16T00:09:21.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.361+0000 7f8f79e27640 1 -- start start 2024-09-16T00:09:21.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f79e27640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f7406ed70 0x7f8f74149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:21.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f79e27640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f7406f150 0x7f8f741e3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:21.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f79e27640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f740c7cb0 0x7f8f74147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:21.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f79e27640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7415b060 con 0x7f8f740c7cb0 2024-09-16T00:09:21.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f79e27640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8f740c9b70 con 0x7f8f7406ed70 2024-09-16T00:09:21.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f79e27640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7415b1e0 con 0x7f8f7406f150 2024-09-16T00:09:21.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f73fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f740c7cb0 0x7f8f74147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:21.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f73fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f740c7cb0 0x7f8f74147b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53986/0 (socket says 172.21.15.139:53986) 2024-09-16T00:09:21.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f73fff640 1 -- 172.21.15.139:0/4204398940 learned_addr learned my addr 172.21.15.139:0/4204398940 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:21.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f737fe640 1 --2- 172.21.15.139:0/4204398940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f7406ed70 0x7f8f74149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:21.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f72ffd640 1 --2- 172.21.15.139:0/4204398940 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f7406f150 0x7f8f741e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:21.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f73fff640 1 -- 172.21.15.139:0/4204398940 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f7406f150 msgr2=0x7f8f741e3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:21.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f73fff640 1 --2- 172.21.15.139:0/4204398940 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f7406f150 0x7f8f741e3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f73fff640 1 -- 172.21.15.139:0/4204398940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f7406ed70 msgr2=0x7f8f74149cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:21.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f73fff640 1 --2- 172.21.15.139:0/4204398940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f7406ed70 0x7f8f74149cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f73fff640 1 -- 172.21.15.139:0/4204398940 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7414a600 con 0x7f8f740c7cb0 2024-09-16T00:09:21.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f73fff640 1 --2- 172.21.15.139:0/4204398940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f740c7cb0 0x7f8f74147b00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c004650 tx=0x7f8f6c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:21.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.362+0000 7f8f5bfff640 1 -- 172.21.15.139:0/4204398940 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c017070 con 0x7f8f740c7cb0 2024-09-16T00:09:21.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.363+0000 7f8f5bfff640 1 -- 172.21.15.139:0/4204398940 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f6c00c940 con 0x7f8f740c7cb0 2024-09-16T00:09:21.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.363+0000 7f8f79e27640 1 -- 172.21.15.139:0/4204398940 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f741efcb0 con 0x7f8f740c7cb0 2024-09-16T00:09:21.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.363+0000 7f8f5bfff640 1 -- 172.21.15.139:0/4204398940 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c00cd30 con 0x7f8f740c7cb0 2024-09-16T00:09:21.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.363+0000 7f8f79e27640 1 -- 172.21.15.139:0/4204398940 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f741f0270 con 0x7f8f740c7cb0 2024-09-16T00:09:21.367 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:21.368 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:21.368 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:21.368 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:21.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.363+0000 7f8f79e27640 1 -- 172.21.15.139:0/4204398940 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f7406bb20 con 0x7f8f740c7cb0 2024-09-16T00:09:21.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.364+0000 7f8f5bfff640 1 -- 172.21.15.139:0/4204398940 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8f6c01a070 con 0x7f8f740c7cb0 2024-09-16T00:09:21.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.365+0000 7f8f5bfff640 1 --2- 172.21.15.139:0/4204398940 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f6006c2b0 0x7f8f6006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:21.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.365+0000 7f8f5bfff640 1 -- 172.21.15.139:0/4204398940 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5944+0+0 (secure 0 0 0) 0x7f8f6c08b830 con 0x7f8f740c7cb0 2024-09-16T00:09:21.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.365+0000 7f8f737fe640 1 --2- 172.21.15.139:0/4204398940 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f6006c2b0 0x7f8f6006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:21.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.365+0000 7f8f737fe640 1 --2- 172.21.15.139:0/4204398940 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f6006c2b0 0x7f8f6006e770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8f640012e0 tx=0x7f8f64001700 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:21.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.369+0000 7f8f5bfff640 1 -- 172.21.15.139:0/4204398940 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8f6c0597d0 con 0x7f8f740c7cb0 2024-09-16T00:09:21.372 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:09:21.415 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:21.415+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:21.516 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:21.515+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:21.530 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:21.530 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:21.531 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:21.539 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:09:21.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.573+0000 7f8f79e27640 1 -- 172.21.15.139:0/4204398940 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8f740756c0 con 0x7f8f6006c2b0 2024-09-16T00:09:21.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.580+0000 7f8f5bfff640 1 -- 172.21.15.139:0/4204398940 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254099 (secure 0 0 0) 0x7f8f740756c0 con 0x7f8f6006c2b0 2024-09-16T00:09:21.585 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:09:21.586 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:09:21.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.584+0000 7f8f79e27640 1 -- 172.21.15.139:0/4204398940 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f6006c2b0 msgr2=0x7f8f6006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:21.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.584+0000 7f8f79e27640 1 --2- 172.21.15.139:0/4204398940 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f6006c2b0 0x7f8f6006e770 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8f640012e0 tx=0x7f8f64001700 comp rx=0 tx=0).stop 2024-09-16T00:09:21.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.584+0000 7f8f79e27640 1 -- 172.21.15.139:0/4204398940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f740c7cb0 msgr2=0x7f8f74147b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:21.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.584+0000 7f8f79e27640 1 --2- 172.21.15.139:0/4204398940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f740c7cb0 0x7f8f74147b00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c004650 tx=0x7f8f6c00ece0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.584+0000 7f8f79e27640 1 -- 172.21.15.139:0/4204398940 shutdown_connections 2024-09-16T00:09:21.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.584+0000 7f8f79e27640 1 --2- 172.21.15.139:0/4204398940 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f6006c2b0 0x7f8f6006e770 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.584+0000 7f8f79e27640 1 --2- 172.21.15.139:0/4204398940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f740c7cb0 0x7f8f74147b00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.584+0000 7f8f79e27640 1 --2- 172.21.15.139:0/4204398940 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8f7406f150 0x7f8f741e3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.584+0000 7f8f79e27640 1 --2- 172.21.15.139:0/4204398940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8f7406ed70 0x7f8f74149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.584+0000 7f8f79e27640 1 -- 172.21.15.139:0/4204398940 >> 172.21.15.139:0/4204398940 conn(0x7f8f74067950 msgr2=0x7f8f74152960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:21.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.584+0000 7f8f79e27640 1 -- 172.21.15.139:0/4204398940 shutdown_connections 2024-09-16T00:09:21.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.584+0000 7f8f79e27640 1 -- 172.21.15.139:0/4204398940 wait complete. 2024-09-16T00:09:21.598 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-09-16T00:09:21.108173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":65148657,"num_objects":6596,"num_object_clones":0,"num_object_copies":13192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6596,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":7103,"num_write_kb":70442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":397258,"num_keys_recovered":4,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6965,"ondisk_log_size":6965,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":195,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379436,"kb_used_data":104948,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754595284,"statfs":{"total":773094113280,"available":772705570816,"internally_reserved":0,"allocated":107466752,"data_stored":132903816,"data_compressed":15573650,"data_compressed_allocated":18968576,"data_compressed_original":78938112,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5389156,"num_objects":399,"num_object_clones":0,"num_object_copies":798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":457,"num_write_kb":5691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":280651,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.030459"},"pg_stats":[{"pgid":"3.3f","version":"69'100","reported_seq":215,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085352+0000","last_change":"2024-09-16T00:08:36.888570+0000","last_active":"2024-09-16T00:09:16.085352+0000","last_peered":"2024-09-16T00:09:16.085352+0000","last_clean":"2024-09-16T00:09:16.085352+0000","last_became_active":"2024-09-16T00:08:36.888143+0000","last_became_peered":"2024-09-16T00:08:36.888143+0000","last_unstale":"2024-09-16T00:09:16.085352+0000","last_undegraded":"2024-09-16T00:09:16.085352+0000","last_fullsized":"2024-09-16T00:09:16.085352+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:30.579681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":275759,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"72'101","reported_seq":214,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101588+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:09:20.101588+0000","last_peered":"2024-09-16T00:09:20.101588+0000","last_clean":"2024-09-16T00:09:20.101588+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:09:20.101588+0000","last_undegraded":"2024-09-16T00:09:20.101588+0000","last_fullsized":"2024-09-16T00:09:20.101588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":338694,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"73'107","reported_seq":220,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.691078+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:09:20.691078+0000","last_peered":"2024-09-16T00:09:20.691078+0000","last_clean":"2024-09-16T00:09:20.691078+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:09:20.691078+0000","last_undegraded":"2024-09-16T00:09:20.691078+0000","last_fullsized":"2024-09-16T00:09:20.691078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":613745,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107,"num_write_kb":660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"69'80","reported_seq":186,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078402+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:09:16.078402+0000","last_peered":"2024-09-16T00:09:16.078402+0000","last_clean":"2024-09-16T00:09:16.078402+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:09:16.078402+0000","last_undegraded":"2024-09-16T00:09:16.078402+0000","last_fullsized":"2024-09-16T00:09:16.078402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":321366,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"72'119","reported_seq":207,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101405+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:09:20.101405+0000","last_peered":"2024-09-16T00:09:20.101405+0000","last_clean":"2024-09-16T00:09:20.101405+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:09:20.101405+0000","last_undegraded":"2024-09-16T00:09:20.101405+0000","last_fullsized":"2024-09-16T00:09:20.101405+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1698370,"num_objects":119,"num_object_clones":0,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":1722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"72'105","reported_seq":217,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.053613+0000","last_change":"2024-09-16T00:09:16.225647+0000","last_active":"2024-09-16T00:09:20.053613+0000","last_peered":"2024-09-16T00:09:20.053613+0000","last_clean":"2024-09-16T00:09:20.053613+0000","last_became_active":"2024-09-16T00:09:16.081926+0000","last_became_peered":"2024-09-16T00:09:16.081926+0000","last_unstale":"2024-09-16T00:09:20.053613+0000","last_undegraded":"2024-09-16T00:09:20.053613+0000","last_fullsized":"2024-09-16T00:09:20.053613+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:59.896840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":300521,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":280651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"69'108","reported_seq":213,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.084873+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:09:16.084873+0000","last_peered":"2024-09-16T00:09:16.084873+0000","last_clean":"2024-09-16T00:09:16.084873+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:09:16.084873+0000","last_undegraded":"2024-09-16T00:09:16.084873+0000","last_fullsized":"2024-09-16T00:09:16.084873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":944450,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"72'86","reported_seq":172,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.984615+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:09:19.984615+0000","last_peered":"2024-09-16T00:09:19.984615+0000","last_clean":"2024-09-16T00:09:19.984615+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:09:19.984615+0000","last_undegraded":"2024-09-16T00:09:19.984615+0000","last_fullsized":"2024-09-16T00:09:19.984615+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173073,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"69'101","reported_seq":206,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.087940+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:09:16.087940+0000","last_peered":"2024-09-16T00:09:16.087940+0000","last_clean":"2024-09-16T00:09:16.087940+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:09:16.087940+0000","last_undegraded":"2024-09-16T00:09:16.087940+0000","last_fullsized":"2024-09-16T00:09:16.087940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":401581,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"69'88","reported_seq":141,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.386306+0000","last_change":"2024-09-16T00:08:36.891868+0000","last_active":"2024-09-16T00:09:16.386306+0000","last_peered":"2024-09-16T00:09:16.386306+0000","last_clean":"2024-09-16T00:09:16.386306+0000","last_became_active":"2024-09-16T00:08:36.891605+0000","last_became_peered":"2024-09-16T00:08:36.891605+0000","last_unstale":"2024-09-16T00:09:16.386306+0000","last_undegraded":"2024-09-16T00:09:16.386306+0000","last_fullsized":"2024-09-16T00:09:16.386306+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:46:00.730436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":415370,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"69'95","reported_seq":148,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.100144+0000","last_change":"2024-09-16T00:08:36.891589+0000","last_active":"2024-09-16T00:09:16.100144+0000","last_peered":"2024-09-16T00:09:16.100144+0000","last_clean":"2024-09-16T00:09:16.100144+0000","last_became_active":"2024-09-16T00:08:36.891279+0000","last_became_peered":"2024-09-16T00:08:36.891279+0000","last_unstale":"2024-09-16T00:09:16.100144+0000","last_undegraded":"2024-09-16T00:09:16.100144+0000","last_fullsized":"2024-09-16T00:09:16.100144+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318764,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95,"num_write_kb":358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"68'85","reported_seq":200,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.079669+0000","last_change":"2024-09-16T00:08:36.888576+0000","last_active":"2024-09-16T00:09:16.079669+0000","last_peered":"2024-09-16T00:09:16.079669+0000","last_clean":"2024-09-16T00:09:16.079669+0000","last_became_active":"2024-09-16T00:08:36.888301+0000","last_became_peered":"2024-09-16T00:08:36.888301+0000","last_unstale":"2024-09-16T00:09:16.079669+0000","last_undegraded":"2024-09-16T00:09:16.079669+0000","last_fullsized":"2024-09-16T00:09:16.079669+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":391399,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85,"num_write_kb":427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"73'99","reported_seq":223,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.637793+0000","last_change":"2024-09-16T00:08:36.892595+0000","last_active":"2024-09-16T00:09:20.637793+0000","last_peered":"2024-09-16T00:09:20.637793+0000","last_clean":"2024-09-16T00:09:20.637793+0000","last_became_active":"2024-09-16T00:08:36.892316+0000","last_became_peered":"2024-09-16T00:08:36.892316+0000","last_unstale":"2024-09-16T00:09:20.637793+0000","last_undegraded":"2024-09-16T00:09:20.637793+0000","last_fullsized":"2024-09-16T00:09:20.637793+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:47.480168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":479457,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99,"num_write_kb":524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"73'103","reported_seq":216,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.107205+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:09:20.107205+0000","last_peered":"2024-09-16T00:09:20.107205+0000","last_clean":"2024-09-16T00:09:20.107205+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:09:20.107205+0000","last_undegraded":"2024-09-16T00:09:20.107205+0000","last_fullsized":"2024-09-16T00:09:20.107205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4866994,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102,"num_write_kb":4810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"73'109","reported_seq":222,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.352209+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:09:20.352209+0000","last_peered":"2024-09-16T00:09:20.352209+0000","last_clean":"2024-09-16T00:09:20.352209+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:09:20.352209+0000","last_undegraded":"2024-09-16T00:09:20.352209+0000","last_fullsized":"2024-09-16T00:09:20.352209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270259,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"69'113","reported_seq":219,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.290098+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:09:16.290098+0000","last_peered":"2024-09-16T00:09:16.290098+0000","last_clean":"2024-09-16T00:09:16.290098+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:09:16.290098+0000","last_undegraded":"2024-09-16T00:09:16.290098+0000","last_fullsized":"2024-09-16T00:09:16.290098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356656,"num_objects":113,"num_object_clones":0,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"69'92","reported_seq":198,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.096996+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:09:16.096996+0000","last_peered":"2024-09-16T00:09:16.096996+0000","last_clean":"2024-09-16T00:09:16.096996+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:09:16.096996+0000","last_undegraded":"2024-09-16T00:09:16.096996+0000","last_fullsized":"2024-09-16T00:09:16.096996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262585,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92,"num_write_kb":301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"72'94","reported_seq":226,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.104985+0000","last_change":"2024-09-16T00:08:36.894430+0000","last_active":"2024-09-16T00:09:19.104985+0000","last_peered":"2024-09-16T00:09:19.104985+0000","last_clean":"2024-09-16T00:09:19.104985+0000","last_became_active":"2024-09-16T00:08:36.894138+0000","last_became_peered":"2024-09-16T00:08:36.894138+0000","last_unstale":"2024-09-16T00:09:19.104985+0000","last_undegraded":"2024-09-16T00:09:19.104985+0000","last_fullsized":"2024-09-16T00:09:19.104985+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:19.478353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":321431,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94,"num_write_kb":365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"69'89","reported_seq":259,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085229+0000","last_change":"2024-09-16T00:09:04.210763+0000","last_active":"2024-09-16T00:09:16.085229+0000","last_peered":"2024-09-16T00:09:16.085229+0000","last_clean":"2024-09-16T00:09:16.085229+0000","last_became_active":"2024-09-16T00:09:04.158723+0000","last_became_peered":"2024-09-16T00:09:04.158723+0000","last_unstale":"2024-09-16T00:09:16.085229+0000","last_undegraded":"2024-09-16T00:09:16.085229+0000","last_fullsized":"2024-09-16T00:09:16.085229+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:25:48.973773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450120,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89,"num_write_kb":483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":103497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"69'111","reported_seq":164,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.100385+0000","last_change":"2024-09-16T00:08:36.890676+0000","last_active":"2024-09-16T00:09:16.100385+0000","last_peered":"2024-09-16T00:09:16.100385+0000","last_clean":"2024-09-16T00:09:16.100385+0000","last_became_active":"2024-09-16T00:08:36.889743+0000","last_became_peered":"2024-09-16T00:08:36.889743+0000","last_unstale":"2024-09-16T00:09:16.100385+0000","last_undegraded":"2024-09-16T00:09:16.100385+0000","last_fullsized":"2024-09-16T00:09:16.100385+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:02.709496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363934,"num_objects":111,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"73'106","reported_seq":219,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.666709+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:09:20.666709+0000","last_peered":"2024-09-16T00:09:20.666709+0000","last_clean":"2024-09-16T00:09:20.666709+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:09:20.666709+0000","last_undegraded":"2024-09-16T00:09:20.666709+0000","last_fullsized":"2024-09-16T00:09:20.666709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281224,"num_objects":106,"num_object_clones":0,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"69'105","reported_seq":211,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.352279+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:09:16.352279+0000","last_peered":"2024-09-16T00:09:16.352279+0000","last_clean":"2024-09-16T00:09:16.352279+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:09:16.352279+0000","last_undegraded":"2024-09-16T00:09:16.352279+0000","last_fullsized":"2024-09-16T00:09:16.352279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":391692,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"73'108","reported_seq":222,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.705526+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:09:20.705526+0000","last_peered":"2024-09-16T00:09:20.705526+0000","last_clean":"2024-09-16T00:09:20.705526+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:09:20.705526+0000","last_undegraded":"2024-09-16T00:09:20.705526+0000","last_fullsized":"2024-09-16T00:09:20.705526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":453356,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"68'99","reported_seq":205,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.077979+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:09:16.077979+0000","last_peered":"2024-09-16T00:09:16.077979+0000","last_clean":"2024-09-16T00:09:16.077979+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:09:16.077979+0000","last_undegraded":"2024-09-16T00:09:16.077979+0000","last_fullsized":"2024-09-16T00:09:16.077979+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":414019,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99,"num_write_kb":455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"73'113","reported_seq":227,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.677908+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:09:20.677908+0000","last_peered":"2024-09-16T00:09:20.677908+0000","last_clean":"2024-09-16T00:09:20.677908+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:09:20.677908+0000","last_undegraded":"2024-09-16T00:09:20.677908+0000","last_fullsized":"2024-09-16T00:09:20.677908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316565,"num_objects":113,"num_object_clones":0,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"69'115","reported_seq":220,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.478406+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:09:16.478406+0000","last_peered":"2024-09-16T00:09:16.478406+0000","last_clean":"2024-09-16T00:09:16.478406+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:09:16.478406+0000","last_undegraded":"2024-09-16T00:09:16.478406+0000","last_fullsized":"2024-09-16T00:09:16.478406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":745081,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"73'100","reported_seq":213,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101453+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:09:20.101453+0000","last_peered":"2024-09-16T00:09:20.101453+0000","last_clean":"2024-09-16T00:09:20.101453+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:09:20.101453+0000","last_undegraded":"2024-09-16T00:09:20.101453+0000","last_fullsized":"2024-09-16T00:09:20.101453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":346243,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"73'99","reported_seq":213,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119270+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:09:20.119270+0000","last_peered":"2024-09-16T00:09:20.119270+0000","last_clean":"2024-09-16T00:09:20.119270+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:09:20.119270+0000","last_undegraded":"2024-09-16T00:09:20.119270+0000","last_fullsized":"2024-09-16T00:09:20.119270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458275,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99,"num_write_kb":503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"73'103","reported_seq":216,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.107613+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:09:20.107613+0000","last_peered":"2024-09-16T00:09:20.107613+0000","last_clean":"2024-09-16T00:09:20.107613+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:09:20.107613+0000","last_undegraded":"2024-09-16T00:09:20.107613+0000","last_fullsized":"2024-09-16T00:09:20.107613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316000,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"72'122","reported_seq":208,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.411641+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:09:19.411641+0000","last_peered":"2024-09-16T00:09:19.411641+0000","last_clean":"2024-09-16T00:09:19.411641+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:09:19.411641+0000","last_undegraded":"2024-09-16T00:09:19.411641+0000","last_fullsized":"2024-09-16T00:09:19.411641+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410431,"num_objects":122,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122,"num_write_kb":469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"72'112","reported_seq":229,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.806282+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:09:19.806282+0000","last_peered":"2024-09-16T00:09:19.806282+0000","last_clean":"2024-09-16T00:09:19.806282+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:09:19.806282+0000","last_undegraded":"2024-09-16T00:09:19.806282+0000","last_fullsized":"2024-09-16T00:09:19.806282+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":381579,"num_objects":112,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"69'105","reported_seq":158,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.372491+0000","last_change":"2024-09-16T00:08:36.894101+0000","last_active":"2024-09-16T00:09:16.372491+0000","last_peered":"2024-09-16T00:09:16.372491+0000","last_clean":"2024-09-16T00:09:16.372491+0000","last_became_active":"2024-09-16T00:08:36.893807+0000","last_became_peered":"2024-09-16T00:08:36.893807+0000","last_unstale":"2024-09-16T00:09:16.372491+0000","last_undegraded":"2024-09-16T00:09:16.372491+0000","last_fullsized":"2024-09-16T00:09:16.372491+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:40.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":958407,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"69'102","reported_seq":207,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.461457+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:09:16.461457+0000","last_peered":"2024-09-16T00:09:16.461457+0000","last_clean":"2024-09-16T00:09:16.461457+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:09:16.461457+0000","last_undegraded":"2024-09-16T00:09:16.461457+0000","last_fullsized":"2024-09-16T00:09:16.461457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":304220,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"63'62","reported_seq":178,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119631+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:09:20.119631+0000","last_peered":"2024-09-16T00:09:20.119631+0000","last_clean":"2024-09-16T00:09:20.119631+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:09:20.119631+0000","last_undegraded":"2024-09-16T00:09:20.119631+0000","last_fullsized":"2024-09-16T00:09:20.119631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62,"num_write_kb":4136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"73'94","reported_seq":207,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.488618+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:09:20.488618+0000","last_peered":"2024-09-16T00:09:20.488618+0000","last_clean":"2024-09-16T00:09:20.488618+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:09:20.488618+0000","last_undegraded":"2024-09-16T00:09:20.488618+0000","last_fullsized":"2024-09-16T00:09:20.488618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":283620,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94,"num_write_kb":326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"67'39","reported_seq":91,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.099997+0000","last_change":"2024-09-16T00:08:36.890316+0000","last_active":"2024-09-16T00:09:16.099997+0000","last_peered":"2024-09-16T00:09:16.099997+0000","last_clean":"2024-09-16T00:09:16.099997+0000","last_became_active":"2024-09-16T00:08:36.889635+0000","last_became_peered":"2024-09-16T00:08:36.889635+0000","last_unstale":"2024-09-16T00:09:16.099997+0000","last_undegraded":"2024-09-16T00:09:16.099997+0000","last_fullsized":"2024-09-16T00:09:16.099997+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:28.843883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"63'1","reported_seq":127,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119169+0000","last_change":"2024-09-16T00:08:36.888349+0000","last_active":"2024-09-16T00:09:20.119169+0000","last_peered":"2024-09-16T00:09:20.119169+0000","last_clean":"2024-09-16T00:09:20.119169+0000","last_became_active":"2024-09-16T00:08:36.888052+0000","last_became_peered":"2024-09-16T00:08:36.888052+0000","last_unstale":"2024-09-16T00:09:20.119169+0000","last_undegraded":"2024-09-16T00:09:20.119169+0000","last_fullsized":"2024-09-16T00:09:20.119169+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:49.607254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"73'123","reported_seq":237,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.556875+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:09:20.556875+0000","last_peered":"2024-09-16T00:09:20.556875+0000","last_clean":"2024-09-16T00:09:20.556875+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:09:20.556875+0000","last_undegraded":"2024-09-16T00:09:20.556875+0000","last_fullsized":"2024-09-16T00:09:20.556875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":788879,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"66'2","reported_seq":110,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078017+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:09:16.078017+0000","last_peered":"2024-09-16T00:09:16.078017+0000","last_clean":"2024-09-16T00:09:16.078017+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:09:16.078017+0000","last_undegraded":"2024-09-16T00:09:16.078017+0000","last_fullsized":"2024-09-16T00:09:16.078017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"73'115","reported_seq":228,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.429522+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:09:20.429522+0000","last_peered":"2024-09-16T00:09:20.429522+0000","last_clean":"2024-09-16T00:09:20.429522+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:09:20.429522+0000","last_undegraded":"2024-09-16T00:09:20.429522+0000","last_fullsized":"2024-09-16T00:09:20.429522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":652806,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"61'1","reported_seq":108,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.100175+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:09:16.100175+0000","last_peered":"2024-09-16T00:09:16.100175+0000","last_clean":"2024-09-16T00:09:16.100175+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:09:16.100175+0000","last_undegraded":"2024-09-16T00:09:16.100175+0000","last_fullsized":"2024-09-16T00:09:16.100175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"69'106","reported_seq":212,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.407763+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:09:16.407763+0000","last_peered":"2024-09-16T00:09:16.407763+0000","last_clean":"2024-09-16T00:09:16.407763+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:09:16.407763+0000","last_undegraded":"2024-09-16T00:09:16.407763+0000","last_fullsized":"2024-09-16T00:09:16.407763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":465077,"num_objects":106,"num_object_clones":0,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":108,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078240+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:09:16.078240+0000","last_peered":"2024-09-16T00:09:16.078240+0000","last_clean":"2024-09-16T00:09:16.078240+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:09:16.078240+0000","last_undegraded":"2024-09-16T00:09:16.078240+0000","last_fullsized":"2024-09-16T00:09:16.078240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"73'107","reported_seq":220,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.579780+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:09:20.579780+0000","last_peered":"2024-09-16T00:09:20.579780+0000","last_clean":"2024-09-16T00:09:20.579780+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:09:20.579780+0000","last_undegraded":"2024-09-16T00:09:20.579780+0000","last_fullsized":"2024-09-16T00:09:20.579780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":864794,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107,"num_write_kb":903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"71'7","reported_seq":122,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101517+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:09:20.101517+0000","last_peered":"2024-09-16T00:09:20.101517+0000","last_clean":"2024-09-16T00:09:20.101517+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:09:20.101517+0000","last_undegraded":"2024-09-16T00:09:20.101517+0000","last_fullsized":"2024-09-16T00:09:20.101517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"69'88","reported_seq":193,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.088005+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:09:16.088005+0000","last_peered":"2024-09-16T00:09:16.088005+0000","last_clean":"2024-09-16T00:09:16.088005+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:09:16.088005+0000","last_undegraded":"2024-09-16T00:09:16.088005+0000","last_fullsized":"2024-09-16T00:09:16.088005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420942,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"65'22","reported_seq":75,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.100272+0000","last_change":"2024-09-16T00:08:36.914308+0000","last_active":"2024-09-16T00:09:16.100272+0000","last_peered":"2024-09-16T00:09:16.100272+0000","last_clean":"2024-09-16T00:09:16.100272+0000","last_became_active":"2024-09-16T00:08:36.890227+0000","last_became_peered":"2024-09-16T00:08:36.890227+0000","last_unstale":"2024-09-16T00:09:16.100272+0000","last_undegraded":"2024-09-16T00:09:16.100272+0000","last_fullsized":"2024-09-16T00:09:16.100272+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:31.761195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13110,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"69'85","reported_seq":190,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.090928+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:09:16.090928+0000","last_peered":"2024-09-16T00:09:16.090928+0000","last_clean":"2024-09-16T00:09:16.090928+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:09:16.090928+0000","last_undegraded":"2024-09-16T00:09:16.090928+0000","last_fullsized":"2024-09-16T00:09:16.090928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327524,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"71'7","reported_seq":122,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.107651+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:09:20.107651+0000","last_peered":"2024-09-16T00:09:20.107651+0000","last_clean":"2024-09-16T00:09:20.107651+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:09:20.107651+0000","last_undegraded":"2024-09-16T00:09:20.107651+0000","last_fullsized":"2024-09-16T00:09:20.107651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"73'104","reported_seq":217,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.407378+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:09:20.407378+0000","last_peered":"2024-09-16T00:09:20.407378+0000","last_clean":"2024-09-16T00:09:20.407378+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:09:20.407378+0000","last_undegraded":"2024-09-16T00:09:20.407378+0000","last_fullsized":"2024-09-16T00:09:20.407378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418435,"num_objects":104,"num_object_clones":0,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"61'1","reported_seq":109,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078207+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:09:16.078207+0000","last_peered":"2024-09-16T00:09:16.078207+0000","last_clean":"2024-09-16T00:09:16.078207+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:09:16.078207+0000","last_undegraded":"2024-09-16T00:09:16.078207+0000","last_fullsized":"2024-09-16T00:09:16.078207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"69'99","reported_seq":204,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.322192+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:09:16.322192+0000","last_peered":"2024-09-16T00:09:16.322192+0000","last_clean":"2024-09-16T00:09:16.322192+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:09:16.322192+0000","last_undegraded":"2024-09-16T00:09:16.322192+0000","last_fullsized":"2024-09-16T00:09:16.322192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2645634,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99,"num_write_kb":2639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"62'3","reported_seq":56,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.099989+0000","last_change":"2024-09-16T00:08:36.889720+0000","last_active":"2024-09-16T00:09:16.099989+0000","last_peered":"2024-09-16T00:09:16.099989+0000","last_clean":"2024-09-16T00:09:16.099989+0000","last_became_active":"2024-09-16T00:08:36.889345+0000","last_became_peered":"2024-09-16T00:08:36.889345+0000","last_unstale":"2024-09-16T00:09:16.099989+0000","last_undegraded":"2024-09-16T00:09:16.099989+0000","last_fullsized":"2024-09-16T00:09:16.099989+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:03.170073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"69'97","reported_seq":150,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.414321+0000","last_change":"2024-09-16T00:08:36.896787+0000","last_active":"2024-09-16T00:09:16.414321+0000","last_peered":"2024-09-16T00:09:16.414321+0000","last_clean":"2024-09-16T00:09:16.414321+0000","last_became_active":"2024-09-16T00:08:36.896505+0000","last_became_peered":"2024-09-16T00:08:36.896505+0000","last_unstale":"2024-09-16T00:09:16.414321+0000","last_undegraded":"2024-09-16T00:09:16.414321+0000","last_fullsized":"2024-09-16T00:09:16.414321+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:55.997551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4580108,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97,"num_write_kb":4521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"63'10","reported_seq":124,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.103086+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:09:19.103086+0000","last_peered":"2024-09-16T00:09:19.103086+0000","last_clean":"2024-09-16T00:09:19.103086+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:09:19.103086+0000","last_undegraded":"2024-09-16T00:09:19.103086+0000","last_fullsized":"2024-09-16T00:09:19.103086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"69'106","reported_seq":211,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.140958+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:09:16.140958+0000","last_peered":"2024-09-16T00:09:16.140958+0000","last_clean":"2024-09-16T00:09:16.140958+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:09:16.140958+0000","last_undegraded":"2024-09-16T00:09:16.140958+0000","last_fullsized":"2024-09-16T00:09:16.140958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387466,"num_objects":106,"num_object_clones":0,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"61'2","reported_seq":109,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.079842+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:09:16.079842+0000","last_peered":"2024-09-16T00:09:16.079842+0000","last_clean":"2024-09-16T00:09:16.079842+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:09:16.079842+0000","last_undegraded":"2024-09-16T00:09:16.079842+0000","last_fullsized":"2024-09-16T00:09:16.079842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"69'85","reported_seq":190,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.363811+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:09:16.363811+0000","last_peered":"2024-09-16T00:09:16.363811+0000","last_clean":"2024-09-16T00:09:16.363811+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:09:16.363811+0000","last_undegraded":"2024-09-16T00:09:16.363811+0000","last_fullsized":"2024-09-16T00:09:16.363811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":338686,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"71'4","reported_seq":118,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.103202+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:09:19.103202+0000","last_peered":"2024-09-16T00:09:19.103202+0000","last_clean":"2024-09-16T00:09:19.103202+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:09:19.103202+0000","last_undegraded":"2024-09-16T00:09:19.103202+0000","last_fullsized":"2024-09-16T00:09:19.103202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"69'90","reported_seq":195,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085461+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:09:16.085461+0000","last_peered":"2024-09-16T00:09:16.085461+0000","last_clean":"2024-09-16T00:09:16.085461+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:09:16.085461+0000","last_undegraded":"2024-09-16T00:09:16.085461+0000","last_fullsized":"2024-09-16T00:09:16.085461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":259850,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90,"num_write_kb":297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"61'2","reported_seq":117,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101525+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:09:20.101525+0000","last_peered":"2024-09-16T00:09:20.101525+0000","last_clean":"2024-09-16T00:09:20.101525+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:09:20.101525+0000","last_undegraded":"2024-09-16T00:09:20.101525+0000","last_fullsized":"2024-09-16T00:09:20.101525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"69'105","reported_seq":211,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078544+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:09:16.078544+0000","last_peered":"2024-09-16T00:09:16.078544+0000","last_clean":"2024-09-16T00:09:16.078544+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:09:16.078544+0000","last_undegraded":"2024-09-16T00:09:16.078544+0000","last_fullsized":"2024-09-16T00:09:16.078544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":598227,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"66'1","reported_seq":115,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.103233+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:09:19.103233+0000","last_peered":"2024-09-16T00:09:19.103233+0000","last_clean":"2024-09-16T00:09:19.103233+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:09:19.103233+0000","last_undegraded":"2024-09-16T00:09:19.103233+0000","last_fullsized":"2024-09-16T00:09:19.103233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"69'106","reported_seq":221,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.383992+0000","last_change":"2024-09-16T00:08:47.953122+0000","last_active":"2024-09-16T00:09:16.383992+0000","last_peered":"2024-09-16T00:09:16.383992+0000","last_clean":"2024-09-16T00:09:16.383992+0000","last_became_active":"2024-09-16T00:08:47.952799+0000","last_became_peered":"2024-09-16T00:08:47.952799+0000","last_unstale":"2024-09-16T00:09:16.383992+0000","last_undegraded":"2024-09-16T00:09:16.383992+0000","last_fullsized":"2024-09-16T00:09:16.383992+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:03.447562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":381576,"num_objects":106,"num_object_clones":0,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":263,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.084830+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:09:16.084830+0000","last_peered":"2024-09-16T00:09:16.084830+0000","last_clean":"2024-09-16T00:09:16.084830+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:09:16.084830+0000","last_undegraded":"2024-09-16T00:09:16.084830+0000","last_fullsized":"2024-09-16T00:09:16.084830+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"66'1","reported_seq":109,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078027+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:09:16.078027+0000","last_peered":"2024-09-16T00:09:16.078027+0000","last_clean":"2024-09-16T00:09:16.078027+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:09:16.078027+0000","last_undegraded":"2024-09-16T00:09:16.078027+0000","last_fullsized":"2024-09-16T00:09:16.078027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"73'100","reported_seq":214,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.688814+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:09:20.688814+0000","last_peered":"2024-09-16T00:09:20.688814+0000","last_clean":"2024-09-16T00:09:20.688814+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:09:20.688814+0000","last_undegraded":"2024-09-16T00:09:20.688814+0000","last_fullsized":"2024-09-16T00:09:20.688814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426614,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'2","reported_seq":109,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.084756+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:09:16.084756+0000","last_peered":"2024-09-16T00:09:16.084756+0000","last_clean":"2024-09-16T00:09:16.084756+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:09:16.084756+0000","last_undegraded":"2024-09-16T00:09:16.084756+0000","last_fullsized":"2024-09-16T00:09:16.084756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"73'97","reported_seq":210,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.534875+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:09:20.534875+0000","last_peered":"2024-09-16T00:09:20.534875+0000","last_clean":"2024-09-16T00:09:20.534875+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:09:20.534875+0000","last_undegraded":"2024-09-16T00:09:20.534875+0000","last_fullsized":"2024-09-16T00:09:20.534875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":253833,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97,"num_write_kb":301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"71'1","reported_seq":115,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.103092+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:09:19.103092+0000","last_peered":"2024-09-16T00:09:19.103092+0000","last_clean":"2024-09-16T00:09:19.103092+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:09:19.103092+0000","last_undegraded":"2024-09-16T00:09:19.103092+0000","last_fullsized":"2024-09-16T00:09:19.103092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"73'115","reported_seq":228,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.511361+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:09:20.511361+0000","last_peered":"2024-09-16T00:09:20.511361+0000","last_clean":"2024-09-16T00:09:20.511361+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:09:20.511361+0000","last_undegraded":"2024-09-16T00:09:20.511361+0000","last_fullsized":"2024-09-16T00:09:20.511361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":504645,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"67'3","reported_seq":111,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078296+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:09:16.078296+0000","last_peered":"2024-09-16T00:09:16.078296+0000","last_clean":"2024-09-16T00:09:16.078296+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:09:16.078296+0000","last_undegraded":"2024-09-16T00:09:16.078296+0000","last_fullsized":"2024-09-16T00:09:16.078296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"69'97","reported_seq":202,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.367943+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:09:16.367943+0000","last_peered":"2024-09-16T00:09:16.367943+0000","last_clean":"2024-09-16T00:09:16.367943+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:09:16.367943+0000","last_undegraded":"2024-09-16T00:09:16.367943+0000","last_fullsized":"2024-09-16T00:09:16.367943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":563894,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97,"num_write_kb":603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"61'1","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101534+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:09:20.101534+0000","last_peered":"2024-09-16T00:09:20.101534+0000","last_clean":"2024-09-16T00:09:20.101534+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:09:20.101534+0000","last_undegraded":"2024-09-16T00:09:20.101534+0000","last_fullsized":"2024-09-16T00:09:20.101534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"69'92","reported_seq":198,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.301283+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:09:16.301283+0000","last_peered":"2024-09-16T00:09:16.301283+0000","last_clean":"2024-09-16T00:09:16.301283+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:09:16.301283+0000","last_undegraded":"2024-09-16T00:09:16.301283+0000","last_fullsized":"2024-09-16T00:09:16.301283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212579,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085249+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:09:16.085249+0000","last_peered":"2024-09-16T00:09:16.085249+0000","last_clean":"2024-09-16T00:09:16.085249+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:09:16.085249+0000","last_undegraded":"2024-09-16T00:09:16.085249+0000","last_fullsized":"2024-09-16T00:09:16.085249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'104","reported_seq":217,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.431593+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:09:20.431593+0000","last_peered":"2024-09-16T00:09:20.431593+0000","last_clean":"2024-09-16T00:09:20.431593+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:09:20.431593+0000","last_undegraded":"2024-09-16T00:09:20.431593+0000","last_fullsized":"2024-09-16T00:09:20.431593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":498070,"num_objects":104,"num_object_clones":0,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"73'32","reported_seq":165,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101645+0000","last_change":"2024-09-16T00:08:36.906253+0000","last_active":"2024-09-16T00:09:20.101645+0000","last_peered":"2024-09-16T00:09:20.101645+0000","last_clean":"2024-09-16T00:09:20.101645+0000","last_became_active":"2024-09-16T00:08:36.894363+0000","last_became_peered":"2024-09-16T00:08:36.894363+0000","last_unstale":"2024-09-16T00:09:20.101645+0000","last_undegraded":"2024-09-16T00:09:20.101645+0000","last_fullsized":"2024-09-16T00:09:20.101645+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:00.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84,"num_write_kb":825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"73'108","reported_seq":222,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119533+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:09:20.119533+0000","last_peered":"2024-09-16T00:09:20.119533+0000","last_clean":"2024-09-16T00:09:20.119533+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:09:20.119533+0000","last_undegraded":"2024-09-16T00:09:20.119533+0000","last_fullsized":"2024-09-16T00:09:20.119533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":601448,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"61'3","reported_seq":110,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.100204+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:09:16.100204+0000","last_peered":"2024-09-16T00:09:16.100204+0000","last_clean":"2024-09-16T00:09:16.100204+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:09:16.100204+0000","last_undegraded":"2024-09-16T00:09:16.100204+0000","last_fullsized":"2024-09-16T00:09:16.100204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"72'109","reported_seq":222,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.107621+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:09:20.107621+0000","last_peered":"2024-09-16T00:09:20.107621+0000","last_clean":"2024-09-16T00:09:20.107621+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:09:20.107621+0000","last_undegraded":"2024-09-16T00:09:20.107621+0000","last_fullsized":"2024-09-16T00:09:20.107621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":298496,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":107,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.084727+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:09:16.084727+0000","last_peered":"2024-09-16T00:09:16.084727+0000","last_clean":"2024-09-16T00:09:16.084727+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:09:16.084727+0000","last_undegraded":"2024-09-16T00:09:16.084727+0000","last_fullsized":"2024-09-16T00:09:16.084727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"69'112","reported_seq":217,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.405822+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:09:16.405822+0000","last_peered":"2024-09-16T00:09:16.405822+0000","last_clean":"2024-09-16T00:09:16.405822+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:09:16.405822+0000","last_undegraded":"2024-09-16T00:09:16.405822+0000","last_fullsized":"2024-09-16T00:09:16.405822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318386,"num_objects":112,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"61'1","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101307+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:09:20.101307+0000","last_peered":"2024-09-16T00:09:20.101307+0000","last_clean":"2024-09-16T00:09:20.101307+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:09:20.101307+0000","last_undegraded":"2024-09-16T00:09:20.101307+0000","last_fullsized":"2024-09-16T00:09:20.101307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"73'105","reported_seq":219,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.559467+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:09:20.559467+0000","last_peered":"2024-09-16T00:09:20.559467+0000","last_clean":"2024-09-16T00:09:20.559467+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:09:20.559467+0000","last_undegraded":"2024-09-16T00:09:20.559467+0000","last_fullsized":"2024-09-16T00:09:20.559467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335746,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"63'2","reported_seq":117,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101354+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:09:20.101354+0000","last_peered":"2024-09-16T00:09:20.101354+0000","last_clean":"2024-09-16T00:09:20.101354+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:09:20.101354+0000","last_undegraded":"2024-09-16T00:09:20.101354+0000","last_fullsized":"2024-09-16T00:09:20.101354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"73'109","reported_seq":223,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119590+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:09:20.119590+0000","last_peered":"2024-09-16T00:09:20.119590+0000","last_clean":"2024-09-16T00:09:20.119590+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:09:20.119590+0000","last_undegraded":"2024-09-16T00:09:20.119590+0000","last_fullsized":"2024-09-16T00:09:20.119590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":372221,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"64'6","reported_seq":113,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.084974+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:09:16.084974+0000","last_peered":"2024-09-16T00:09:16.084974+0000","last_clean":"2024-09-16T00:09:16.084974+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:09:16.084974+0000","last_undegraded":"2024-09-16T00:09:16.084974+0000","last_fullsized":"2024-09-16T00:09:16.084974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"69'95","reported_seq":148,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.125392+0000","last_change":"2024-09-16T00:08:36.890347+0000","last_active":"2024-09-16T00:09:16.125392+0000","last_peered":"2024-09-16T00:09:16.125392+0000","last_clean":"2024-09-16T00:09:16.125392+0000","last_became_active":"2024-09-16T00:08:36.890009+0000","last_became_peered":"2024-09-16T00:08:36.890009+0000","last_unstale":"2024-09-16T00:09:16.125392+0000","last_undegraded":"2024-09-16T00:09:16.125392+0000","last_fullsized":"2024-09-16T00:09:16.125392+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:24.298978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353539,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95,"num_write_kb":393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"69'87","reported_seq":193,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.334327+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:09:16.334327+0000","last_peered":"2024-09-16T00:09:16.334327+0000","last_clean":"2024-09-16T00:09:16.334327+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:09:16.334327+0000","last_undegraded":"2024-09-16T00:09:16.334327+0000","last_fullsized":"2024-09-16T00:09:16.334327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240570,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87,"num_write_kb":283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"66'3","reported_seq":110,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085003+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:09:16.085003+0000","last_peered":"2024-09-16T00:09:16.085003+0000","last_clean":"2024-09-16T00:09:16.085003+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:09:16.085003+0000","last_undegraded":"2024-09-16T00:09:16.085003+0000","last_fullsized":"2024-09-16T00:09:16.085003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"69'95","reported_seq":201,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078513+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:09:16.078513+0000","last_peered":"2024-09-16T00:09:16.078513+0000","last_clean":"2024-09-16T00:09:16.078513+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:09:16.078513+0000","last_undegraded":"2024-09-16T00:09:16.078513+0000","last_fullsized":"2024-09-16T00:09:16.078513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":479096,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95,"num_write_kb":517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"66'6","reported_seq":131,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085152+0000","last_change":"2024-09-16T00:08:36.900704+0000","last_active":"2024-09-16T00:09:16.085152+0000","last_peered":"2024-09-16T00:09:16.085152+0000","last_clean":"2024-09-16T00:09:16.085152+0000","last_became_active":"2024-09-16T00:08:36.892520+0000","last_became_peered":"2024-09-16T00:08:36.892520+0000","last_unstale":"2024-09-16T00:09:16.085152+0000","last_undegraded":"2024-09-16T00:09:16.085152+0000","last_fullsized":"2024-09-16T00:09:16.085152+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:30.056041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"72'110","reported_seq":196,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.570967+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:09:19.570967+0000","last_peered":"2024-09-16T00:09:19.570967+0000","last_clean":"2024-09-16T00:09:19.570967+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:09:19.570967+0000","last_undegraded":"2024-09-16T00:09:19.570967+0000","last_fullsized":"2024-09-16T00:09:19.570967+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878216,"num_objects":110,"num_object_clones":0,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"72'39","reported_seq":155,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119425+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:09:20.119425+0000","last_peered":"2024-09-16T00:09:20.119425+0000","last_clean":"2024-09-16T00:09:20.119425+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:09:20.119425+0000","last_undegraded":"2024-09-16T00:09:20.119425+0000","last_fullsized":"2024-09-16T00:09:20.119425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":4141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"73'107","reported_seq":231,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.450104+0000","last_change":"2024-09-16T00:08:36.889082+0000","last_active":"2024-09-16T00:09:20.450104+0000","last_peered":"2024-09-16T00:09:20.450104+0000","last_clean":"2024-09-16T00:09:20.450104+0000","last_became_active":"2024-09-16T00:08:36.888807+0000","last_became_peered":"2024-09-16T00:08:36.888807+0000","last_unstale":"2024-09-16T00:09:20.450104+0000","last_undegraded":"2024-09-16T00:09:20.450104+0000","last_fullsized":"2024-09-16T00:09:20.450104+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:22.825324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":298398,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107,"num_write_kb":346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"65'58","reported_seq":173,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.107353+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:09:20.107353+0000","last_peered":"2024-09-16T00:09:20.107353+0000","last_clean":"2024-09-16T00:09:20.107353+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:09:20.107353+0000","last_undegraded":"2024-09-16T00:09:20.107353+0000","last_fullsized":"2024-09-16T00:09:20.107353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":4138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":39390995,"num_objects":6525,"num_object_clones":0,"num_object_copies":13050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6525,"num_write_kb":41915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":384148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90755072,"data_stored":78767502,"data_compressed":7812362,"data_compressed_allocated":9633792,"data_compressed_original":28786688,"omap_allocated":0,"internal_metadata":0},"log_size":6526,"ondisk_log_size":6526,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":25167294,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":419,"num_write_kb":26111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":13110,"num_keys_recovered":4,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10543104,"data_stored":50334588,"data_compressed":7770882,"data_compressed_allocated":9338880,"data_compressed_original":49184768,"omap_allocated":0,"internal_metadata":0},"log_size":322,"ondisk_log_size":322,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509467,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50176,"kb_used_data":14828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94321664,"statfs":{"total":96636764160,"available":96585383936,"internally_reserved":0,"allocated":15183872,"data_stored":23373544,"data_compressed":2945461,"data_compressed_allocated":3616768,"data_compressed_original":16404480,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509467,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43788,"kb_used_data":12592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328052,"statfs":{"total":96636764160,"available":96591925248,"internally_reserved":0,"allocated":12894208,"data_stored":18170022,"data_compressed":2007399,"data_compressed_allocated":2420736,"data_compressed_original":10899456,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509466,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45576,"kb_used_data":10224,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326264,"statfs":{"total":96636764160,"available":96590094336,"internally_reserved":0,"allocated":10469376,"data_stored":8320139,"data_compressed":1085058,"data_compressed_allocated":1335296,"data_compressed_original":3911680,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509467,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49168,"kb_used_data":13812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322672,"statfs":{"total":96636764160,"available":96586416128,"internally_reserved":0,"allocated":14143488,"data_stored":17711202,"data_compressed":2042984,"data_compressed_allocated":2514944,"data_compressed_original":10903552,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49076,"kb_used_data":13724,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322764,"statfs":{"total":96636764160,"available":96586510336,"internally_reserved":0,"allocated":14053376,"data_stored":21050186,"data_compressed":3148104,"data_compressed_allocated":3756032,"data_compressed_original":15941632,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47316,"kb_used_data":11968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94324524,"statfs":{"total":96636764160,"available":96588312576,"internally_reserved":0,"allocated":12255232,"data_stored":10931436,"data_compressed":929701,"data_compressed_allocated":1146880,"data_compressed_original":3465216,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56276,"kb_used_data":20924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94315564,"statfs":{"total":96636764160,"available":96579137536,"internally_reserved":0,"allocated":21426176,"data_stored":26980158,"data_compressed":2504865,"data_compressed_allocated":3092480,"data_compressed_original":13815808,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38060,"kb_used_data":6876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333780,"statfs":{"total":96636764160,"available":96597790720,"internally_reserved":0,"allocated":7041024,"data_stored":6367129,"data_compressed":910078,"data_compressed_allocated":1085440,"data_compressed_original":3596288,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1748992,"data_stored":8389246,"data_compressed":1255393,"data_compressed_allocated":1544192,"data_compressed_original":8196096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2703360,"data_stored":12583584,"data_compressed":1983184,"data_compressed_allocated":2338816,"data_compressed_original":12226560,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1675264,"data_stored":8388734,"data_compressed":1280256,"data_compressed_allocated":1560576,"data_compressed_original":8286208,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1712128,"data_stored":8389280,"data_compressed":1327713,"data_compressed_allocated":1560576,"data_compressed_original":8245248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2691072,"data_stored":12583494,"data_compressed":1924336,"data_compressed_allocated":2334720,"data_compressed_original":12230656,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":18800640,"data_stored":17629859,"data_compressed":1217095,"data_compressed_allocated":1503232,"data_compressed_original":5046272,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11399168,"data_stored":10541937,"data_compressed":929701,"data_compressed_allocated":1146880,"data_compressed_original":3465216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6352896,"data_stored":6045784,"data_compressed":910078,"data_compressed_allocated":1085440,"data_compressed_original":3596288,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10821632,"data_stored":8372607,"data_compressed":1239268,"data_compressed_allocated":1511424,"data_compressed_original":3895296,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11669504,"data_stored":8957737,"data_compressed":762728,"data_compressed_allocated":954368,"data_compressed_original":2617344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9666560,"data_stored":7955318,"data_compressed":1085058,"data_compressed_allocated":1335296,"data_compressed_original":3911680,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10383360,"data_stored":9416011,"data_compressed":679686,"data_compressed_allocated":860160,"data_compressed_original":2654208,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11661312,"data_stored":9848249,"data_compressed":988748,"data_compressed_allocated":1236992,"data_compressed_original":3600384,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:09:21.602 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:09:21.639 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:21.638+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:21.660 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:21.660 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:21.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:21.661 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:21.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.661 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:21.661 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:21.661 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 118.577703574, 2024-09-16T00:09:21.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:09:21.661 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.661 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.661 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.661 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.661 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.661 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.661 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:09:21.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.662 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:21.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:21.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:09:21.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:09:21.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:09:21.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:09:21.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:09:21.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:09:21.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:09:21.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:09:21.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:09:21.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:09:21.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:09:21.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:09:21.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:09:21.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:09:21.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:09:21.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:09:21.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:09:21.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:09:21.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:09:21.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:09:21.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:09:21.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:09:21.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:09:21.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:09:21.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:09:21.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:09:21.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:09:21.667 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.667 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.668 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.668 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:21.668 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:09:21.668 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.49817194400001, 2024-09-16T00:09:21.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093071190000000008, 2024-09-16T00:09:21.668 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.668 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.668 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.668 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.668 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.668 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.669 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 106 2024-09-16T00:09:21.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.669 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:09:21.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:09:21.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160362+0000", 2024-09-16T00:09:21.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:09:21.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160438+0000", 2024-09-16T00:09:21.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6711000000000005e-05 2024-09-16T00:09:21.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160444+0000", 2024-09-16T00:09:21.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0530000000000003e-06 2024-09-16T00:09:21.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160449+0000", 2024-09-16T00:09:21.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2189999999999998e-06 2024-09-16T00:09:21.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161270+0000", 2024-09-16T00:09:21.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082099199999999999 2024-09-16T00:09:21.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161348+0000", 2024-09-16T00:09:21.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8361000000000004e-05 2024-09-16T00:09:21.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161472+0000", 2024-09-16T00:09:21.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123523 2024-09-16T00:09:21.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163461+0000", 2024-09-16T00:09:21.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019891560000000002 2024-09-16T00:09:21.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169418+0000", 2024-09-16T00:09:21.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059568490000000002 2024-09-16T00:09:21.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169570+0000", 2024-09-16T00:09:21.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152461 2024-09-16T00:09:21.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169668+0000", 2024-09-16T00:09:21.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T00:09:21.673 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.673 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.673 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.674 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:21.674 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:09:21.674 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.498076173, 2024-09-16T00:09:21.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092931679999999992, 2024-09-16T00:09:21.674 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.674 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.674 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.674 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.674 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.674 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.674 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107 2024-09-16T00:09:21.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.674 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:09:21.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:09:21.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:09:21.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0699999999999998e-07 2024-09-16T00:09:21.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160459+0000", 2024-09-16T00:09:21.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0370000000000001e-06 2024-09-16T00:09:21.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160463+0000", 2024-09-16T00:09:21.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999999e-06 2024-09-16T00:09:21.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160466+0000", 2024-09-16T00:09:21.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.644e-06 2024-09-16T00:09:21.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161873+0000", 2024-09-16T00:09:21.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014074840000000001 2024-09-16T00:09:21.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161949+0000", 2024-09-16T00:09:21.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5749999999999998e-05 2024-09-16T00:09:21.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162069+0000", 2024-09-16T00:09:21.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120257 2024-09-16T00:09:21.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163506+0000", 2024-09-16T00:09:21.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001436344 2024-09-16T00:09:21.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169704+0000", 2024-09-16T00:09:21.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061982829999999997 2024-09-16T00:09:21.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169730+0000", 2024-09-16T00:09:21.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6060000000000001e-05 2024-09-16T00:09:21.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169750+0000", 2024-09-16T00:09:21.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0092e-05 2024-09-16T00:09:21.679 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.679 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.679 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.679 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:21.679 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:09:21.679 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.49796686099999, 2024-09-16T00:09:21.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092280039999999997, 2024-09-16T00:09:21.679 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.680 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.680 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.680 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 108 2024-09-16T00:09:21.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.680 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:09:21.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:09:21.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160567+0000", 2024-09-16T00:09:21.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7300000000000001e-07 2024-09-16T00:09:21.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160569+0000", 2024-09-16T00:09:21.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5299999999999999e-06 2024-09-16T00:09:21.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160574+0000", 2024-09-16T00:09:21.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-16T00:09:21.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160577+0000", 2024-09-16T00:09:21.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.01e-06 2024-09-16T00:09:21.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162518+0000", 2024-09-16T00:09:21.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001940703 2024-09-16T00:09:21.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162571+0000", 2024-09-16T00:09:21.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2670000000000002e-05 2024-09-16T00:09:21.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162647+0000", 2024-09-16T00:09:21.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6778000000000002e-05 2024-09-16T00:09:21.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163544+0000", 2024-09-16T00:09:21.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089702599999999999 2024-09-16T00:09:21.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169754+0000", 2024-09-16T00:09:21.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062091739999999996 2024-09-16T00:09:21.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169776+0000", 2024-09-16T00:09:21.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2391999999999998e-05 2024-09-16T00:09:21.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169794+0000", 2024-09-16T00:09:21.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8318999999999999e-05 2024-09-16T00:09:21.685 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.685 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.685 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.685 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:21.685 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:09:21.685 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.49780896199999, 2024-09-16T00:09:21.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092871919999999997, 2024-09-16T00:09:21.685 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.685 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.685 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.686 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.686 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.686 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.686 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 110 2024-09-16T00:09:21.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.686 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:09:21.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:09:21.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:09:21.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:09:21.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160728+0000", 2024-09-16T00:09:21.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0620000000000001e-06 2024-09-16T00:09:21.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160731+0000", 2024-09-16T00:09:21.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6500000000000002e-06 2024-09-16T00:09:21.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160734+0000", 2024-09-16T00:09:21.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.762e-06 2024-09-16T00:09:21.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163580+0000", 2024-09-16T00:09:21.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028464549999999999 2024-09-16T00:09:21.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163657+0000", 2024-09-16T00:09:21.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6695999999999999e-05 2024-09-16T00:09:21.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163798+0000", 2024-09-16T00:09:21.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014055 2024-09-16T00:09:21.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165483+0000", 2024-09-16T00:09:21.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016853090000000001 2024-09-16T00:09:21.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169970+0000", 2024-09-16T00:09:21.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00448709 2024-09-16T00:09:21.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169994+0000", 2024-09-16T00:09:21.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3575999999999999e-05 2024-09-16T00:09:21.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170011+0000", 2024-09-16T00:09:21.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7702000000000001e-05 2024-09-16T00:09:21.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.691 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.691 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:21.691 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:09:21.691 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.497795293, 2024-09-16T00:09:21.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093174859999999998, 2024-09-16T00:09:21.691 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.691 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.691 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.691 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 111 2024-09-16T00:09:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:09:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:09:21.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:09:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1e-07 2024-09-16T00:09:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160741+0000", 2024-09-16T00:09:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6110000000000001e-06 2024-09-16T00:09:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160744+0000", 2024-09-16T00:09:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3299999999999999e-06 2024-09-16T00:09:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160746+0000", 2024-09-16T00:09:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5469999999999999e-06 2024-09-16T00:09:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164069+0000", 2024-09-16T00:09:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033222709999999999 2024-09-16T00:09:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164113+0000", 2024-09-16T00:09:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4212999999999998e-05 2024-09-16T00:09:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164187+0000", 2024-09-16T00:09:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4136999999999998e-05 2024-09-16T00:09:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166429+0000", 2024-09-16T00:09:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022423180000000001 2024-09-16T00:09:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170016+0000", 2024-09-16T00:09:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035869449999999998 2024-09-16T00:09:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170038+0000", 2024-09-16T00:09:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2053e-05 2024-09-16T00:09:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170055+0000", 2024-09-16T00:09:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6850999999999998e-05 2024-09-16T00:09:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.696 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:09:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.497781578, 2024-09-16T00:09:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093480279999999995, 2024-09-16T00:09:21.697 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.701 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 112 2024-09-16T00:09:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:09:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:09:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160752+0000", 2024-09-16T00:09:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2700000000000001e-07 2024-09-16T00:09:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160808+0000", 2024-09-16T00:09:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5797999999999997e-05 2024-09-16T00:09:21.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160814+0000", 2024-09-16T00:09:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5980000000000002e-06 2024-09-16T00:09:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160819+0000", 2024-09-16T00:09:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8960000000000002e-06 2024-09-16T00:09:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164468+0000", 2024-09-16T00:09:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003649476 2024-09-16T00:09:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164525+0000", 2024-09-16T00:09:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6227999999999998e-05 2024-09-16T00:09:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164595+0000", 2024-09-16T00:09:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759000000000005e-05 2024-09-16T00:09:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167237+0000", 2024-09-16T00:09:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026410470000000001 2024-09-16T00:09:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170060+0000", 2024-09-16T00:09:21.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028234969999999999 2024-09-16T00:09:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170081+0000", 2024-09-16T00:09:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1242000000000001e-05 2024-09-16T00:09:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170100+0000", 2024-09-16T00:09:21.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8260000000000001e-05 2024-09-16T00:09:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:09:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.49770448300001, 2024-09-16T00:09:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093139409999999992, 2024-09-16T00:09:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.708 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 113 2024-09-16T00:09:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:09:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:09:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:09:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-07 2024-09-16T00:09:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160833+0000", 2024-09-16T00:09:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.557e-06 2024-09-16T00:09:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160839+0000", 2024-09-16T00:09:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3829999999999997e-06 2024-09-16T00:09:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160844+0000", 2024-09-16T00:09:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6909999999999996e-06 2024-09-16T00:09:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165008+0000", 2024-09-16T00:09:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041642129999999999 2024-09-16T00:09:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165085+0000", 2024-09-16T00:09:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6651999999999997e-05 2024-09-16T00:09:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165205+0000", 2024-09-16T00:09:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012045900000000001 2024-09-16T00:09:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167283+0000", 2024-09-16T00:09:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020782909999999999 2024-09-16T00:09:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170104+0000", 2024-09-16T00:09:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028207950000000001 2024-09-16T00:09:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170125+0000", 2024-09-16T00:09:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0852e-05 2024-09-16T00:09:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170143+0000", 2024-09-16T00:09:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7467000000000001e-05 2024-09-16T00:09:21.714 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:09:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.497658001, 2024-09-16T00:09:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093126960000000005, 2024-09-16T00:09:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.715 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-16T00:09:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:09:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:09:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160876+0000", 2024-09-16T00:09:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5700000000000002e-07 2024-09-16T00:09:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160880+0000", 2024-09-16T00:09:21.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5539999999999999e-06 2024-09-16T00:09:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160888+0000", 2024-09-16T00:09:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.157e-06 2024-09-16T00:09:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160894+0000", 2024-09-16T00:09:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7690000000000002e-06 2024-09-16T00:09:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165519+0000", 2024-09-16T00:09:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046249539999999997 2024-09-16T00:09:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165593+0000", 2024-09-16T00:09:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4039e-05 2024-09-16T00:09:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165715+0000", 2024-09-16T00:09:21.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012131899999999999 2024-09-16T00:09:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167309+0000", 2024-09-16T00:09:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001594011 2024-09-16T00:09:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170147+0000", 2024-09-16T00:09:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028387130000000001 2024-09-16T00:09:21.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170170+0000", 2024-09-16T00:09:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2517000000000001e-05 2024-09-16T00:09:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170188+0000", 2024-09-16T00:09:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8006e-05 2024-09-16T00:09:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.721 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:09:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.49761697300001, 2024-09-16T00:09:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093140040000000007, 2024-09-16T00:09:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.722 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-16T00:09:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:09:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:09:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160917+0000", 2024-09-16T00:09:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7599999999999996e-07 2024-09-16T00:09:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160992+0000", 2024-09-16T00:09:21.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4760999999999998e-05 2024-09-16T00:09:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161000+0000", 2024-09-16T00:09:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9829999999999996e-06 2024-09-16T00:09:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161007+0000", 2024-09-16T00:09:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4909999999999999e-06 2024-09-16T00:09:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165983+0000", 2024-09-16T00:09:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049755399999999997 2024-09-16T00:09:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166027+0000", 2024-09-16T00:09:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4109000000000001e-05 2024-09-16T00:09:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166100+0000", 2024-09-16T00:09:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3360000000000002e-05 2024-09-16T00:09:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169698+0000", 2024-09-16T00:09:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597795 2024-09-16T00:09:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170192+0000", 2024-09-16T00:09:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049447300000000005 2024-09-16T00:09:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170214+0000", 2024-09-16T00:09:21.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1546e-05 2024-09-16T00:09:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170230+0000", 2024-09-16T00:09:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6169999999999999e-05 2024-09-16T00:09:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:21.729 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:09:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.497507244, 2024-09-16T00:09:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092457029999999992, 2024-09-16T00:09:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-16T00:09:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:09:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:09:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:09:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7899999999999999e-07 2024-09-16T00:09:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161029+0000", 2024-09-16T00:09:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6630000000000002e-06 2024-09-16T00:09:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161036+0000", 2024-09-16T00:09:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T00:09:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161041+0000", 2024-09-16T00:09:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:09:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166468+0000", 2024-09-16T00:09:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054266050000000001 2024-09-16T00:09:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166511+0000", 2024-09-16T00:09:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3869999999999998e-05 2024-09-16T00:09:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166606+0000", 2024-09-16T00:09:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4437000000000004e-05 2024-09-16T00:09:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169955+0000", 2024-09-16T00:09:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033494810000000001 2024-09-16T00:09:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170234+0000", 2024-09-16T00:09:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027900899999999998 2024-09-16T00:09:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170255+0000", 2024-09-16T00:09:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0537999999999999e-05 2024-09-16T00:09:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170272+0000", 2024-09-16T00:09:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6611000000000001e-05 2024-09-16T00:09:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:09:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.497482099, 2024-09-16T00:09:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093511010000000006, 2024-09-16T00:09:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-16T00:09:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:09:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:09:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:09:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.63e-07 2024-09-16T00:09:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161055+0000", 2024-09-16T00:09:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6770000000000001e-06 2024-09-16T00:09:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161063+0000", 2024-09-16T00:09:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6469999999999998e-06 2024-09-16T00:09:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161068+0000", 2024-09-16T00:09:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.746e-06 2024-09-16T00:09:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166817+0000", 2024-09-16T00:09:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057494959999999998 2024-09-16T00:09:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166893+0000", 2024-09-16T00:09:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6210000000000004e-05 2024-09-16T00:09:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167019+0000", 2024-09-16T00:09:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125585 2024-09-16T00:09:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170275+0000", 2024-09-16T00:09:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032563620000000001 2024-09-16T00:09:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170309+0000", 2024-09-16T00:09:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3865999999999997e-05 2024-09-16T00:09:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170362+0000", 2024-09-16T00:09:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3094999999999999e-05 2024-09-16T00:09:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170402+0000", 2024-09-16T00:09:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9953999999999997e-05 2024-09-16T00:09:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:09:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.04875001400001, 2024-09-16T00:09:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097364389999999995, 2024-09-16T00:09:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138 2024-09-16T00:09:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:09:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:09:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609784+0000", 2024-09-16T00:09:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T00:09:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609787+0000", 2024-09-16T00:09:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280000000000001e-06 2024-09-16T00:09:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609794+0000", 2024-09-16T00:09:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6409999999999996e-06 2024-09-16T00:09:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609799+0000", 2024-09-16T00:09:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9359999999999998e-06 2024-09-16T00:09:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615759+0000", 2024-09-16T00:09:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059599910000000004 2024-09-16T00:09:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615806+0000", 2024-09-16T00:09:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7290000000000003e-05 2024-09-16T00:09:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615877+0000", 2024-09-16T00:09:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1000000000000005e-05 2024-09-16T00:09:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618961+0000", 2024-09-16T00:09:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030841390000000001 2024-09-16T00:09:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619436+0000", 2024-09-16T00:09:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047452799999999999 2024-09-16T00:09:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619482+0000", 2024-09-16T00:09:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5689000000000003e-05 2024-09-16T00:09:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619519+0000", 2024-09-16T00:09:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7938000000000002e-05 2024-09-16T00:09:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:09:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.0487263, 2024-09-16T00:09:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011812954000000001, 2024-09-16T00:09:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139 2024-09-16T00:09:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:09:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:09:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:09:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999998e-07 2024-09-16T00:09:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609810+0000", 2024-09-16T00:09:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2100000000000002e-06 2024-09-16T00:09:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609817+0000", 2024-09-16T00:09:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.28e-06 2024-09-16T00:09:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609825+0000", 2024-09-16T00:09:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:09:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616116+0000", 2024-09-16T00:09:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006290543 2024-09-16T00:09:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616187+0000", 2024-09-16T00:09:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1830999999999997e-05 2024-09-16T00:09:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616298+0000", 2024-09-16T00:09:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011044699999999999 2024-09-16T00:09:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619559+0000", 2024-09-16T00:09:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032614710000000002 2024-09-16T00:09:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621562+0000", 2024-09-16T00:09:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020028680000000001 2024-09-16T00:09:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621597+0000", 2024-09-16T00:09:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4625000000000001e-05 2024-09-16T00:09:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621620+0000", 2024-09-16T00:09:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2864e-05 2024-09-16T00:09:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:09:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.048698504, 2024-09-16T00:09:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011828211999999999, 2024-09-16T00:09:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 140 2024-09-16T00:09:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:09:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:09:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:09:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T00:09:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961000000000003e-05 2024-09-16T00:09:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609892+0000", 2024-09-16T00:09:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0516000000000001e-05 2024-09-16T00:09:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609898+0000", 2024-09-16T00:09:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0449999999999997e-06 2024-09-16T00:09:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616797+0000", 2024-09-16T00:09:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068998840000000002 2024-09-16T00:09:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616844+0000", 2024-09-16T00:09:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6501000000000001e-05 2024-09-16T00:09:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616935+0000", 2024-09-16T00:09:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1222000000000002e-05 2024-09-16T00:09:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619808+0000", 2024-09-16T00:09:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873198 2024-09-16T00:09:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621624+0000", 2024-09-16T00:09:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001815933 2024-09-16T00:09:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621646+0000", 2024-09-16T00:09:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1622999999999999e-05 2024-09-16T00:09:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621663+0000", 2024-09-16T00:09:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6854000000000002e-05 2024-09-16T00:09:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 117.048612695, 2024-09-16T00:09:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014331501999999999, 2024-09-16T00:09:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-16T00:09:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609921+0000", 2024-09-16T00:09:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9599999999999999e-07 2024-09-16T00:09:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609925+0000", 2024-09-16T00:09:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T00:09:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609932+0000", 2024-09-16T00:09:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8240000000000002e-06 2024-09-16T00:09:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609937+0000", 2024-09-16T00:09:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1070000000000004e-06 2024-09-16T00:09:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618987+0000", 2024-09-16T00:09:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090502719999999998 2024-09-16T00:09:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619064+0000", 2024-09-16T00:09:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6868999999999999e-05 2024-09-16T00:09:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619183+0000", 2024-09-16T00:09:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011961 2024-09-16T00:09:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.620052+0000", 2024-09-16T00:09:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086822799999999997 2024-09-16T00:09:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624197+0000", 2024-09-16T00:09:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041454930000000001 2024-09-16T00:09:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624229+0000", 2024-09-16T00:09:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1758999999999999e-05 2024-09-16T00:09:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624252+0000", 2024-09-16T00:09:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:09:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:09:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.575846887, 2024-09-16T00:09:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010093276999999999, 2024-09-16T00:09:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 157 2024-09-16T00:09:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:09:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:09:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:09:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4699999999999998e-07 2024-09-16T00:09:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082688+0000", 2024-09-16T00:09:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0209999999999998e-06 2024-09-16T00:09:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082692+0000", 2024-09-16T00:09:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6890000000000002e-06 2024-09-16T00:09:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082695+0000", 2024-09-16T00:09:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5339999999999998e-06 2024-09-16T00:09:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085303+0000", 2024-09-16T00:09:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026084789999999999 2024-09-16T00:09:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085374+0000", 2024-09-16T00:09:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1317999999999997e-05 2024-09-16T00:09:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085506+0000", 2024-09-16T00:09:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013118 2024-09-16T00:09:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086928+0000", 2024-09-16T00:09:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014224559999999999 2024-09-16T00:09:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092665+0000", 2024-09-16T00:09:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057369830000000002 2024-09-16T00:09:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092727+0000", 2024-09-16T00:09:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1810000000000006e-05 2024-09-16T00:09:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092779+0000", 2024-09-16T00:09:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2559999999999998e-05 2024-09-16T00:09:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:09:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.575834719, 2024-09-16T00:09:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010176388, 2024-09-16T00:09:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 158 2024-09-16T00:09:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:09:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:09:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082699+0000", 2024-09-16T00:09:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.11e-07 2024-09-16T00:09:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082742+0000", 2024-09-16T00:09:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3699000000000003e-05 2024-09-16T00:09:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082748+0000", 2024-09-16T00:09:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8690000000000001e-06 2024-09-16T00:09:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082751+0000", 2024-09-16T00:09:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.942e-06 2024-09-16T00:09:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085881+0000", 2024-09-16T00:09:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031303450000000001 2024-09-16T00:09:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085959+0000", 2024-09-16T00:09:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7874999999999996e-05 2024-09-16T00:09:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086086+0000", 2024-09-16T00:09:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012685499999999999 2024-09-16T00:09:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087127+0000", 2024-09-16T00:09:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001041298 2024-09-16T00:09:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092790+0000", 2024-09-16T00:09:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056628 2024-09-16T00:09:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092832+0000", 2024-09-16T00:09:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2227000000000001e-05 2024-09-16T00:09:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092875+0000", 2024-09-16T00:09:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2267e-05 2024-09-16T00:09:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:09:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 116.57577668099999, 2024-09-16T00:09:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010199547, 2024-09-16T00:09:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159 2024-09-16T00:09:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:09:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:09:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082757+0000", 2024-09-16T00:09:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2500000000000001e-07 2024-09-16T00:09:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082759+0000", 2024-09-16T00:09:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-16T00:09:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082763+0000", 2024-09-16T00:09:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4889999999999997e-06 2024-09-16T00:09:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082766+0000", 2024-09-16T00:09:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:09:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086571+0000", 2024-09-16T00:09:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038053269999999998 2024-09-16T00:09:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086619+0000", 2024-09-16T00:09:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7924999999999999e-05 2024-09-16T00:09:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086693+0000", 2024-09-16T00:09:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000005e-05 2024-09-16T00:09:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087383+0000", 2024-09-16T00:09:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069008299999999997 2024-09-16T00:09:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092882+0000", 2024-09-16T00:09:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054987450000000002 2024-09-16T00:09:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092921+0000", 2024-09-16T00:09:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9388999999999998e-05 2024-09-16T00:09:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092956+0000", 2024-09-16T00:09:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4647999999999998e-05 2024-09-16T00:09:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:161 2.b 2:d5c7a900:::200.00000003:head [write 0~3935733 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e64)", 2024-09-16T00:09:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:07.538495+0000", 2024-09-16T00:09:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.120038041000001, 2024-09-16T00:09:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012844983000000001, 2024-09-16T00:09:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.538495+0000", 2024-09-16T00:09:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.538495+0000", 2024-09-16T00:09:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.538497+0000", 2024-09-16T00:09:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5149999999999999e-06 2024-09-16T00:09:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540617+0000", 2024-09-16T00:09:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021204079999999998 2024-09-16T00:09:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540640+0000", 2024-09-16T00:09:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3167e-05 2024-09-16T00:09:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540650+0000", 2024-09-16T00:09:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6099999999999995e-06 2024-09-16T00:09:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540724+0000", 2024-09-16T00:09:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4340999999999999e-05 2024-09-16T00:09:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540812+0000", 2024-09-16T00:09:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8376999999999998e-05 2024-09-16T00:09:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540957+0000", 2024-09-16T00:09:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014463 2024-09-16T00:09:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.550270+0000", 2024-09-16T00:09:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093128380000000004 2024-09-16T00:09:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551270+0000", 2024-09-16T00:09:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00100015 2024-09-16T00:09:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551309+0000", 2024-09-16T00:09:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8526999999999999e-05 2024-09-16T00:09:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551340+0000", 2024-09-16T00:09:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1420000000000001e-05 2024-09-16T00:09:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:21.798 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:21.800 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:21.798+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:21.801 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:09:21.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.750+0000 7f35826a5640 1 Processor -- start 2024-09-16T00:09:21.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.751+0000 7f35826a5640 1 -- start start 2024-09-16T00:09:21.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.752+0000 7f35826a5640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f357c06d380 0x7f357c1545f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:21.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.752+0000 7f35826a5640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f357c0c5600 0x7f357c154b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:21.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.752+0000 7f35826a5640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f357c15a8b0 con 0x7f357c06d760 2024-09-16T00:09:21.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.752+0000 7f35826a5640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f357c15b4c0 con 0x7f357c0c5600 2024-09-16T00:09:21.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.752+0000 7f35826a5640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f357c15ab90 con 0x7f357c06d380 2024-09-16T00:09:21.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.752+0000 7f3573fff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f357c06d760 0x7f357c06c310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:56076/0 (socket says 172.21.15.139:56076) 2024-09-16T00:09:21.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.752+0000 7f3573fff640 1 -- 172.21.15.139:0/1706879033 learned_addr learned my addr 172.21.15.139:0/1706879033 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:21.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.752+0000 7f357bfff640 1 --2- 172.21.15.139:0/1706879033 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f357c06d380 0x7f357c1545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:21.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.752+0000 7f3580c42640 1 --2- 172.21.15.139:0/1706879033 >> v2:172.21.15.164:3300/0 conn(0x7f357c0c5600 0x7f357c154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:21.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.752+0000 7f357b7fe640 1 -- 172.21.15.139:0/1706879033 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 205473154 0 0) 0x7f357c15a8b0 con 0x7f357c06d760 2024-09-16T00:09:21.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.752+0000 7f357b7fe640 1 -- 172.21.15.139:0/1706879033 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3568003520 con 0x7f357c06d760 2024-09-16T00:09:21.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.752+0000 7f3580c42640 1 -- 172.21.15.139:0/1706879033 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f357c06d380 msgr2=0x7f357c1545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:21.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.752+0000 7f3580c42640 1 --2- 172.21.15.139:0/1706879033 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f357c06d380 0x7f357c1545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.752+0000 7f3580c42640 1 -- 172.21.15.139:0/1706879033 >> v1:172.21.15.164:6789/0 conn(0x7f357c06d760 legacy=0x7f357c06c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:21.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.752+0000 7f3580c42640 1 -- 172.21.15.139:0/1706879033 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357c15ae70 con 0x7f357c0c5600 2024-09-16T00:09:21.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.752+0000 7f357bfff640 1 --2- 172.21.15.139:0/1706879033 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f357c06d380 0x7f357c1545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:21.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.753+0000 7f3580c42640 1 --2- 172.21.15.139:0/1706879033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357c0c5600 0x7f357c154b30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3574009d90 tx=0x7f357402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0472d4ac0c3ef39 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:21.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.753+0000 7f357b7fe640 1 -- 172.21.15.139:0/1706879033 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574007740 con 0x7f357c0c5600 2024-09-16T00:09:21.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.753+0000 7f357b7fe640 1 -- 172.21.15.139:0/1706879033 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35740078e0 con 0x7f357c0c5600 2024-09-16T00:09:21.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.753+0000 7f357b7fe640 1 -- 172.21.15.139:0/1706879033 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3574007cb0 con 0x7f357c0c5600 2024-09-16T00:09:21.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.754+0000 7f35826a5640 1 -- 172.21.15.139:0/1706879033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357c0c5600 msgr2=0x7f357c154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:21.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.754+0000 7f35826a5640 1 --2- 172.21.15.139:0/1706879033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357c0c5600 0x7f357c154b30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3574009d90 tx=0x7f357402a660 comp rx=0 tx=0).stop 2024-09-16T00:09:21.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.754+0000 7f35826a5640 1 -- 172.21.15.139:0/1706879033 shutdown_connections 2024-09-16T00:09:21.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.754+0000 7f35826a5640 1 --2- 172.21.15.139:0/1706879033 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357c0c5600 0x7f357c154b30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.754+0000 7f35826a5640 1 --2- 172.21.15.139:0/1706879033 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f357c06d380 0x7f357c1545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.754+0000 7f35826a5640 1 -- 172.21.15.139:0/1706879033 >> 172.21.15.139:0/1706879033 conn(0x7f357c067f80 msgr2=0x7f357c0d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:21.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.754+0000 7f35826a5640 1 -- 172.21.15.139:0/1706879033 shutdown_connections 2024-09-16T00:09:21.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.754+0000 7f35826a5640 1 -- 172.21.15.139:0/1706879033 wait complete. 2024-09-16T00:09:21.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.755+0000 7f35826a5640 1 Processor -- start 2024-09-16T00:09:21.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.755+0000 7f35826a5640 1 -- start start 2024-09-16T00:09:21.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.756+0000 7f35826a5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357c06d380 0x7f357c1e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:21.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.756+0000 7f35826a5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f357c06d760 0x7f357c15d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:21.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.756+0000 7f35826a5640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f357c0c5600 0x7f357c15da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:21.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.756+0000 7f35826a5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f357c06ca20 con 0x7f357c06d760 2024-09-16T00:09:21.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.756+0000 7f35826a5640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f357c06ce80 con 0x7f357c06d380 2024-09-16T00:09:21.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.756+0000 7f35826a5640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f357c06cba0 con 0x7f357c0c5600 2024-09-16T00:09:21.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f3573fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f357c06d760 0x7f357c15d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:21.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f357bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357c06d380 0x7f357c1e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:21.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f3573fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f357c06d760 0x7f357c15d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54002/0 (socket says 172.21.15.139:54002) 2024-09-16T00:09:21.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f3573fff640 1 -- 172.21.15.139:0/2464721131 learned_addr learned my addr 172.21.15.139:0/2464721131 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:21.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f357bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357c06d380 0x7f357c1e3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:47660/0 (socket says 172.21.15.139:47660) 2024-09-16T00:09:21.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f3580c42640 1 --2- 172.21.15.139:0/2464721131 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f357c0c5600 0x7f357c15da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:21.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f3573fff640 1 -- 172.21.15.139:0/2464721131 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f357c0c5600 msgr2=0x7f357c15da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:21.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f3573fff640 1 --2- 172.21.15.139:0/2464721131 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f357c0c5600 0x7f357c15da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f3573fff640 1 -- 172.21.15.139:0/2464721131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357c06d380 msgr2=0x7f357c1e3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:21.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f3573fff640 1 --2- 172.21.15.139:0/2464721131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357c06d380 0x7f357c1e3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:21.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f3573fff640 1 -- 172.21.15.139:0/2464721131 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357c1f0790 con 0x7f357c06d760 2024-09-16T00:09:21.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f357bfff640 1 --2- 172.21.15.139:0/2464721131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357c06d380 0x7f357c1e3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:21.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f3573fff640 1 --2- 172.21.15.139:0/2464721131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f357c06d760 0x7f357c15d510 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f356400c500 tx=0x7f356400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:21.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f35797fa640 1 -- 172.21.15.139:0/2464721131 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564017070 con 0x7f357c06d760 2024-09-16T00:09:21.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f35797fa640 1 -- 172.21.15.139:0/2464721131 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3564005ce0 con 0x7f357c06d760 2024-09-16T00:09:21.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f35826a5640 1 -- 172.21.15.139:0/2464721131 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f357c1f0a70 con 0x7f357c06d760 2024-09-16T00:09:21.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f35797fa640 1 -- 172.21.15.139:0/2464721131 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35640072f0 con 0x7f357c06d760 2024-09-16T00:09:21.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.757+0000 7f35826a5640 1 -- 172.21.15.139:0/2464721131 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f357c1f1030 con 0x7f357c06d760 2024-09-16T00:09:21.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.758+0000 7f35826a5640 1 -- 172.21.15.139:0/2464721131 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f357c0b6400 con 0x7f357c06d760 2024-09-16T00:09:21.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.759+0000 7f35797fa640 1 -- 172.21.15.139:0/2464721131 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f356401a070 con 0x7f357c06d760 2024-09-16T00:09:21.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.760+0000 7f35797fa640 1 --2- 172.21.15.139:0/2464721131 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f355c06c2e0 0x7f355c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:21.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.760+0000 7f35797fa640 1 -- 172.21.15.139:0/2464721131 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5944+0+0 (secure 0 0 0) 0x7f356408a7d0 con 0x7f357c06d760 2024-09-16T00:09:21.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.760+0000 7f357bfff640 1 --2- 172.21.15.139:0/2464721131 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f355c06c2e0 0x7f355c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:21.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.760+0000 7f357bfff640 1 --2- 172.21.15.139:0/2464721131 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f355c06c2e0 0x7f355c06e7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f356c009560 tx=0x7f356c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:21.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.765+0000 7f35797fa640 1 -- 172.21.15.139:0/2464721131 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f357c0b6400 con 0x7f357c06d760 2024-09-16T00:09:21.877 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:09:21.922 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:21.921+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:21.937 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:21.937 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:21.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:21.938 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:7114 3.6 3:63547153:::10000002086.00000000:head [write 0~67896 in=67896b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-16T00:09:21.938 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:21.936013+0000", 2024-09-16T00:09:21.938 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.001251952, 2024-09-16T00:09:21.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012706239999999999, 2024-09-16T00:09:21.938 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:21.938 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:21.938 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "waiting for sub ops", 2024-09-16T00:09:21.938 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:21.938 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:21.938 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:21.938 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 7114 2024-09-16T00:09:21.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:21.938 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.936013+0000", 2024-09-16T00:09:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.936013+0000", 2024-09-16T00:09:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.936014+0000", 2024-09-16T00:09:21.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.302e-06 2024-09-16T00:09:21.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:21.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:21.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:21.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.936050+0000", 2024-09-16T00:09:21.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5889999999999997e-05 2024-09-16T00:09:21.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:21.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:21.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:21.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.936062+0000", 2024-09-16T00:09:21.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2054e-05 2024-09-16T00:09:21.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:21.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:21.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:21.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.936069+0000", 2024-09-16T00:09:21.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.097e-06 2024-09-16T00:09:21.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:21.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:21.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:21.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.936137+0000", 2024-09-16T00:09:21.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7382e-05 2024-09-16T00:09:21.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:21.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:21.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:21.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.936267+0000", 2024-09-16T00:09:21.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000130109 2024-09-16T00:09:21.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:21.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:21.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:21.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.936405+0000", 2024-09-16T00:09:21.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013834900000000001 2024-09-16T00:09:21.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:21.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:21.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:21.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.937203+0000", 2024-09-16T00:09:21.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079785299999999995 2024-09-16T00:09:21.942 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:21.942 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:21.942 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:21.942 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:21.942 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:21.942 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 1 2024-09-16T00:09:21.943 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:21.949 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:09:21.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.990+0000 7f35826a5640 1 -- 172.21.15.139:0/2464721131 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f357c15a8b0 con 0x7f355c06c2e0 2024-09-16T00:09:21.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.997+0000 7fd92af91640 1 Processor -- start 2024-09-16T00:09:21.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.997+0000 7fd92af91640 1 -- start start 2024-09-16T00:09:21.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd92af91640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9240d0490 0x7fd9240b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:21.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd92af91640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd9240d0870 0x7fd9240b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:21.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd92af91640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd9240b3430 0x7fd9240b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:21.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd92af91640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd92415b940 con 0x7fd9240d0870 2024-09-16T00:09:21.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd92af91640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fd9240b3bb0 con 0x7fd9240b3430 2024-09-16T00:09:21.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd92af91640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd9240b3180 con 0x7fd9240d0490 2024-09-16T00:09:21.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd928d2d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9240d0490 0x7fd9240b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd928d2d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9240d0490 0x7fd9240b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:49292/0 (socket says 172.21.15.139:49292) 2024-09-16T00:09:22.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd928d2d640 1 -- 172.21.15.139:0/749815815 learned_addr learned my addr 172.21.15.139:0/749815815 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:22.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd92952e640 1 --2- 172.21.15.139:0/749815815 >> v2:172.21.15.164:3300/0 conn(0x7fd9240b3430 0x7fd9240b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd923fff640 1 --2- 172.21.15.139:0/749815815 >> v2:172.21.15.139:3300/0 conn(0x7fd9240d0870 0x7fd9240b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7f35797fa640 1 -- 172.21.15.139:0/2464721131 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254099 (secure 0 0 0) 0x7f357c15a8b0 con 0x7f355c06c2e0 2024-09-16T00:09:22.001 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:09:22.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd92952e640 1 -- 172.21.15.139:0/749815815 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9240d0490 msgr2=0x7fd9240b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:22.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd92952e640 1 --2- 172.21.15.139:0/749815815 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9240d0490 0x7fd9240b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd92952e640 1 -- 172.21.15.139:0/749815815 >> v2:172.21.15.139:3300/0 conn(0x7fd9240d0870 msgr2=0x7fd9240b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:22.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd92952e640 1 --2- 172.21.15.139:0/749815815 >> v2:172.21.15.139:3300/0 conn(0x7fd9240d0870 0x7fd9240b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd92952e640 1 -- 172.21.15.139:0/749815815 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd92415adc0 con 0x7fd9240b3430 2024-09-16T00:09:22.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.998+0000 7fd928d2d640 1 --2- 172.21.15.139:0/749815815 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9240d0490 0x7fd9240b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:22.005 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:09:22.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:21.999+0000 7fd92952e640 1 --2- 172.21.15.139:0/749815815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9240b3430 0x7fd9240b4580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd9240c7780 tx=0x7fd91c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34b2a25724b62b80 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.000+0000 7fd9237fe640 1 -- 172.21.15.139:0/749815815 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c007740 con 0x7fd9240b3430 2024-09-16T00:09:22.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.000+0000 7fd9237fe640 1 -- 172.21.15.139:0/749815815 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd91c0078e0 con 0x7fd9240b3430 2024-09-16T00:09:22.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.000+0000 7fd9237fe640 1 -- 172.21.15.139:0/749815815 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c007cb0 con 0x7fd9240b3430 2024-09-16T00:09:22.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.001+0000 7fd92af91640 1 -- 172.21.15.139:0/749815815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9240b3430 msgr2=0x7fd9240b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.001+0000 7fd92af91640 1 --2- 172.21.15.139:0/749815815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9240b3430 0x7fd9240b4580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd9240c7780 tx=0x7fd91c02a660 comp rx=0 tx=0).stop 2024-09-16T00:09:22.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.001+0000 7fd92af91640 1 -- 172.21.15.139:0/749815815 shutdown_connections 2024-09-16T00:09:22.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.001+0000 7fd92af91640 1 --2- 172.21.15.139:0/749815815 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9240b3430 0x7fd9240b4580 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.001+0000 7fd92af91640 1 --2- 172.21.15.139:0/749815815 >> v2:172.21.15.139:3300/0 conn(0x7fd9240d0870 0x7fd9240b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.001+0000 7fd92af91640 1 --2- 172.21.15.139:0/749815815 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9240d0490 0x7fd9240b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.001+0000 7fd92af91640 1 -- 172.21.15.139:0/749815815 >> 172.21.15.139:0/749815815 conn(0x7fd92406c520 msgr2=0x7fd92406c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:22.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.001+0000 7fd92af91640 1 -- 172.21.15.139:0/749815815 shutdown_connections 2024-09-16T00:09:22.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.001+0000 7fd92af91640 1 -- 172.21.15.139:0/749815815 wait complete. 2024-09-16T00:09:22.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.002+0000 7f35826a5640 1 -- 172.21.15.139:0/2464721131 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f355c06c2e0 msgr2=0x7f355c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.002+0000 7f35826a5640 1 --2- 172.21.15.139:0/2464721131 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f355c06c2e0 0x7f355c06e7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f356c009560 tx=0x7f356c009190 comp rx=0 tx=0).stop 2024-09-16T00:09:22.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.002+0000 7f35826a5640 1 -- 172.21.15.139:0/2464721131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f357c06d760 msgr2=0x7f357c15d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.002+0000 7f35826a5640 1 --2- 172.21.15.139:0/2464721131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f357c06d760 0x7f357c15d510 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f356400c500 tx=0x7f356400c9c0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.002+0000 7f35826a5640 1 -- 172.21.15.139:0/2464721131 shutdown_connections 2024-09-16T00:09:22.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.002+0000 7f35826a5640 1 --2- 172.21.15.139:0/2464721131 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f355c06c2e0 0x7f355c06e7a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.002+0000 7f35826a5640 1 --2- 172.21.15.139:0/2464721131 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f357c0c5600 0x7f357c15da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.002+0000 7f35826a5640 1 --2- 172.21.15.139:0/2464721131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f357c06d760 0x7f357c15d510 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.002+0000 7f35826a5640 1 --2- 172.21.15.139:0/2464721131 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f357c06d380 0x7f357c1e3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.002+0000 7f35826a5640 1 -- 172.21.15.139:0/2464721131 >> 172.21.15.139:0/2464721131 conn(0x7f357c067f80 msgr2=0x7f357c14ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:22.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.002+0000 7f35826a5640 1 -- 172.21.15.139:0/2464721131 shutdown_connections 2024-09-16T00:09:22.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.002+0000 7f35826a5640 1 -- 172.21.15.139:0/2464721131 wait complete. 2024-09-16T00:09:22.010 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:22.003+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:22.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.003+0000 7fd92af91640 1 Processor -- start 2024-09-16T00:09:22.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.003+0000 7fd92af91640 1 -- start start 2024-09-16T00:09:22.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.004+0000 7fd92af91640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9240b3430 0x7fd92415be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.004+0000 7fd92af91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9240d0490 0x7fd9240c4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.004+0000 7fd92af91640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9240d0870 0x7fd92415d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.004+0000 7fd92af91640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd92415b660 con 0x7fd9240d0490 2024-09-16T00:09:22.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.004+0000 7fd92af91640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd92415b380 con 0x7fd9240b3430 2024-09-16T00:09:22.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.004+0000 7fd92af91640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd92415a980 con 0x7fd9240d0870 2024-09-16T00:09:22.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.004+0000 7fd923fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9240d0490 0x7fd9240c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.004+0000 7fd923fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9240d0490 0x7fd9240c4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54026/0 (socket says 172.21.15.139:54026) 2024-09-16T00:09:22.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.004+0000 7fd923fff640 1 -- 172.21.15.139:0/4179807407 learned_addr learned my addr 172.21.15.139:0/4179807407 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:22.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.004+0000 7fd92952e640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9240d0870 0x7fd92415d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.005+0000 7fd928d2d640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9240b3430 0x7fd92415be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.005+0000 7fd923fff640 1 -- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9240d0870 msgr2=0x7fd92415d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.005+0000 7fd923fff640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9240d0870 0x7fd92415d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.005+0000 7fd923fff640 1 -- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9240b3430 msgr2=0x7fd92415be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.005+0000 7fd923fff640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9240b3430 0x7fd92415be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.005+0000 7fd923fff640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9240c3cb0 con 0x7fd9240d0490 2024-09-16T00:09:22.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.005+0000 7fd928d2d640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd9240b3430 0x7fd92415be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:22.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.005+0000 7fd92952e640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd9240d0870 0x7fd92415d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:22.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.005+0000 7fd923fff640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9240d0490 0x7fd9240c4970 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd91000e870 tx=0x7fd91000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.005+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910017070 con 0x7fd9240d0490 2024-09-16T00:09:22.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.005+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd91000ee60 con 0x7fd9240d0490 2024-09-16T00:09:22.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.005+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91000cb60 con 0x7fd9240d0490 2024-09-16T00:09:22.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.005+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd92415c740 con 0x7fd9240d0490 2024-09-16T00:09:22.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.005+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd92415cd00 con 0x7fd9240d0490 2024-09-16T00:09:22.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.007+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fd9240b2c40 con 0x7fd9240d0490 2024-09-16T00:09:22.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.008+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd91001a070 con 0x7fd9240d0490 2024-09-16T00:09:22.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.008+0000 7fd9217fa640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd90806c2e0 0x7fd90806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.008+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5944+0+0 (secure 0 0 0) 0x7fd91008b510 con 0x7fd9240d0490 2024-09-16T00:09:22.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.008+0000 7fd928d2d640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd90806c2e0 0x7fd90806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.008+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v74) ==== 54+0+16 (secure 0 0 0) 0x7fd91008db90 con 0x7fd9240d0490 2024-09-16T00:09:22.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.009+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fd9240c59c0 0x7fd9240c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.009+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd9240c56c0 con 0x7fd9240c59c0 2024-09-16T00:09:22.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.009+0000 7fd928d2d640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd90806c2e0 0x7fd90806e7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd918009630 tx=0x7fd918009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.009+0000 7fd92952e640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fd9240c59c0 0x7fd9240c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.009+0000 7fd92952e640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fd9240c59c0 0x7fd9240c5da0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.010+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd91c007540 con 0x7fd9240c59c0 2024-09-16T00:09:22.018 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-09-16T00:09:21.108173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":65148657,"num_objects":6596,"num_object_clones":0,"num_object_copies":13192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6596,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":7103,"num_write_kb":70442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":397258,"num_keys_recovered":4,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6965,"ondisk_log_size":6965,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":195,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379436,"kb_used_data":104948,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754595284,"statfs":{"total":773094113280,"available":772705570816,"internally_reserved":0,"allocated":107466752,"data_stored":132903816,"data_compressed":15573650,"data_compressed_allocated":18968576,"data_compressed_original":78938112,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5389156,"num_objects":399,"num_object_clones":0,"num_object_copies":798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":457,"num_write_kb":5691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":280651,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.030459"},"pg_stats":[{"pgid":"3.3f","version":"69'100","reported_seq":215,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085352+0000","last_change":"2024-09-16T00:08:36.888570+0000","last_active":"2024-09-16T00:09:16.085352+0000","last_peered":"2024-09-16T00:09:16.085352+0000","last_clean":"2024-09-16T00:09:16.085352+0000","last_became_active":"2024-09-16T00:08:36.888143+0000","last_became_peered":"2024-09-16T00:08:36.888143+0000","last_unstale":"2024-09-16T00:09:16.085352+0000","last_undegraded":"2024-09-16T00:09:16.085352+0000","last_fullsized":"2024-09-16T00:09:16.085352+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:30.579681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":275759,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"72'101","reported_seq":214,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101588+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:09:20.101588+0000","last_peered":"2024-09-16T00:09:20.101588+0000","last_clean":"2024-09-16T00:09:20.101588+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:09:20.101588+0000","last_undegraded":"2024-09-16T00:09:20.101588+0000","last_fullsized":"2024-09-16T00:09:20.101588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":338694,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"73'107","reported_seq":220,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.691078+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:09:20.691078+0000","last_peered":"2024-09-16T00:09:20.691078+0000","last_clean":"2024-09-16T00:09:20.691078+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:09:20.691078+0000","last_undegraded":"2024-09-16T00:09:20.691078+0000","last_fullsized":"2024-09-16T00:09:20.691078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":613745,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107,"num_write_kb":660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"69'80","reported_seq":186,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078402+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:09:16.078402+0000","last_peered":"2024-09-16T00:09:16.078402+0000","last_clean":"2024-09-16T00:09:16.078402+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:09:16.078402+0000","last_undegraded":"2024-09-16T00:09:16.078402+0000","last_fullsized":"2024-09-16T00:09:16.078402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":321366,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"72'119","reported_seq":207,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101405+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:09:20.101405+0000","last_peered":"2024-09-16T00:09:20.101405+0000","last_clean":"2024-09-16T00:09:20.101405+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:09:20.101405+0000","last_undegraded":"2024-09-16T00:09:20.101405+0000","last_fullsized":"2024-09-16T00:09:20.101405+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1698370,"num_objects":119,"num_object_clones":0,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":1722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"72'105","reported_seq":217,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.053613+0000","last_change":"2024-09-16T00:09:16.225647+0000","last_active":"2024-09-16T00:09:20.053613+0000","last_peered":"2024-09-16T00:09:20.053613+0000","last_clean":"2024-09-16T00:09:20.053613+0000","last_became_active":"2024-09-16T00:09:16.081926+0000","last_became_peered":"2024-09-16T00:09:16.081926+0000","last_unstale":"2024-09-16T00:09:20.053613+0000","last_undegraded":"2024-09-16T00:09:20.053613+0000","last_fullsized":"2024-09-16T00:09:20.053613+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:59.896840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":300521,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":280651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"69'108","reported_seq":213,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.084873+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:09:16.084873+0000","last_peered":"2024-09-16T00:09:16.084873+0000","last_clean":"2024-09-16T00:09:16.084873+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:09:16.084873+0000","last_undegraded":"2024-09-16T00:09:16.084873+0000","last_fullsized":"2024-09-16T00:09:16.084873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":944450,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"72'86","reported_seq":172,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.984615+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:09:19.984615+0000","last_peered":"2024-09-16T00:09:19.984615+0000","last_clean":"2024-09-16T00:09:19.984615+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:09:19.984615+0000","last_undegraded":"2024-09-16T00:09:19.984615+0000","last_fullsized":"2024-09-16T00:09:19.984615+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173073,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"69'101","reported_seq":206,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.087940+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:09:16.087940+0000","last_peered":"2024-09-16T00:09:16.087940+0000","last_clean":"2024-09-16T00:09:16.087940+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:09:16.087940+0000","last_undegraded":"2024-09-16T00:09:16.087940+0000","last_fullsized":"2024-09-16T00:09:16.087940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":401581,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"69'88","reported_seq":141,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.386306+0000","last_change":"2024-09-16T00:08:36.891868+0000","last_active":"2024-09-16T00:09:16.386306+0000","last_peered":"2024-09-16T00:09:16.386306+0000","last_clean":"2024-09-16T00:09:16.386306+0000","last_became_active":"2024-09-16T00:08:36.891605+0000","last_became_peered":"2024-09-16T00:08:36.891605+0000","last_unstale":"2024-09-16T00:09:16.386306+0000","last_undegraded":"2024-09-16T00:09:16.386306+0000","last_fullsized":"2024-09-16T00:09:16.386306+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:46:00.730436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":415370,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"69'95","reported_seq":148,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.100144+0000","last_change":"2024-09-16T00:08:36.891589+0000","last_active":"2024-09-16T00:09:16.100144+0000","last_peered":"2024-09-16T00:09:16.100144+0000","last_clean":"2024-09-16T00:09:16.100144+0000","last_became_active":"2024-09-16T00:08:36.891279+0000","last_became_peered":"2024-09-16T00:08:36.891279+0000","last_unstale":"2024-09-16T00:09:16.100144+0000","last_undegraded":"2024-09-16T00:09:16.100144+0000","last_fullsized":"2024-09-16T00:09:16.100144+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318764,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95,"num_write_kb":358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"68'85","reported_seq":200,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.079669+0000","last_change":"2024-09-16T00:08:36.888576+0000","last_active":"2024-09-16T00:09:16.079669+0000","last_peered":"2024-09-16T00:09:16.079669+0000","last_clean":"2024-09-16T00:09:16.079669+0000","last_became_active":"2024-09-16T00:08:36.888301+0000","last_became_peered":"2024-09-16T00:08:36.888301+0000","last_unstale":"2024-09-16T00:09:16.079669+0000","last_undegraded":"2024-09-16T00:09:16.079669+0000","last_fullsized":"2024-09-16T00:09:16.079669+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":391399,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85,"num_write_kb":427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"73'99","reported_seq":223,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.637793+0000","last_change":"2024-09-16T00:08:36.892595+0000","last_active":"2024-09-16T00:09:20.637793+0000","last_peered":"2024-09-16T00:09:20.637793+0000","last_clean":"2024-09-16T00:09:20.637793+0000","last_became_active":"2024-09-16T00:08:36.892316+0000","last_became_peered":"2024-09-16T00:08:36.892316+0000","last_unstale":"2024-09-16T00:09:20.637793+0000","last_undegraded":"2024-09-16T00:09:20.637793+0000","last_fullsized":"2024-09-16T00:09:20.637793+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:47.480168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":479457,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99,"num_write_kb":524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"73'103","reported_seq":216,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.107205+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:09:20.107205+0000","last_peered":"2024-09-16T00:09:20.107205+0000","last_clean":"2024-09-16T00:09:20.107205+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:09:20.107205+0000","last_undegraded":"2024-09-16T00:09:20.107205+0000","last_fullsized":"2024-09-16T00:09:20.107205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4866994,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102,"num_write_kb":4810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"73'109","reported_seq":222,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.352209+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:09:20.352209+0000","last_peered":"2024-09-16T00:09:20.352209+0000","last_clean":"2024-09-16T00:09:20.352209+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:09:20.352209+0000","last_undegraded":"2024-09-16T00:09:20.352209+0000","last_fullsized":"2024-09-16T00:09:20.352209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270259,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"69'113","reported_seq":219,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.290098+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:09:16.290098+0000","last_peered":"2024-09-16T00:09:16.290098+0000","last_clean":"2024-09-16T00:09:16.290098+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:09:16.290098+0000","last_undegraded":"2024-09-16T00:09:16.290098+0000","last_fullsized":"2024-09-16T00:09:16.290098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356656,"num_objects":113,"num_object_clones":0,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"69'92","reported_seq":198,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.096996+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:09:16.096996+0000","last_peered":"2024-09-16T00:09:16.096996+0000","last_clean":"2024-09-16T00:09:16.096996+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:09:16.096996+0000","last_undegraded":"2024-09-16T00:09:16.096996+0000","last_fullsized":"2024-09-16T00:09:16.096996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262585,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92,"num_write_kb":301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"72'94","reported_seq":226,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.104985+0000","last_change":"2024-09-16T00:08:36.894430+0000","last_active":"2024-09-16T00:09:19.104985+0000","last_peered":"2024-09-16T00:09:19.104985+0000","last_clean":"2024-09-16T00:09:19.104985+0000","last_became_active":"2024-09-16T00:08:36.894138+0000","last_became_peered":"2024-09-16T00:08:36.894138+0000","last_unstale":"2024-09-16T00:09:19.104985+0000","last_undegraded":"2024-09-16T00:09:19.104985+0000","last_fullsized":"2024-09-16T00:09:19.104985+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:19.478353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":321431,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94,"num_write_kb":365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"69'89","reported_seq":259,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085229+0000","last_change":"2024-09-16T00:09:04.210763+0000","last_active":"2024-09-16T00:09:16.085229+0000","last_peered":"2024-09-16T00:09:16.085229+0000","last_clean":"2024-09-16T00:09:16.085229+0000","last_became_active":"2024-09-16T00:09:04.158723+0000","last_became_peered":"2024-09-16T00:09:04.158723+0000","last_unstale":"2024-09-16T00:09:16.085229+0000","last_undegraded":"2024-09-16T00:09:16.085229+0000","last_fullsized":"2024-09-16T00:09:16.085229+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:25:48.973773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450120,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89,"num_write_kb":483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":103497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"69'111","reported_seq":164,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.100385+0000","last_change":"2024-09-16T00:08:36.890676+0000","last_active":"2024-09-16T00:09:16.100385+0000","last_peered":"2024-09-16T00:09:16.100385+0000","last_clean":"2024-09-16T00:09:16.100385+0000","last_became_active":"2024-09-16T00:08:36.889743+0000","last_became_peered":"2024-09-16T00:08:36.889743+0000","last_unstale":"2024-09-16T00:09:16.100385+0000","last_undegraded":"2024-09-16T00:09:16.100385+0000","last_fullsized":"2024-09-16T00:09:16.100385+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:02.709496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363934,"num_objects":111,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"73'106","reported_seq":219,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.666709+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:09:20.666709+0000","last_peered":"2024-09-16T00:09:20.666709+0000","last_clean":"2024-09-16T00:09:20.666709+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:09:20.666709+0000","last_undegraded":"2024-09-16T00:09:20.666709+0000","last_fullsized":"2024-09-16T00:09:20.666709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281224,"num_objects":106,"num_object_clones":0,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"69'105","reported_seq":211,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.352279+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:09:16.352279+0000","last_peered":"2024-09-16T00:09:16.352279+0000","last_clean":"2024-09-16T00:09:16.352279+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:09:16.352279+0000","last_undegraded":"2024-09-16T00:09:16.352279+0000","last_fullsized":"2024-09-16T00:09:16.352279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":391692,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"73'108","reported_seq":222,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.705526+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:09:20.705526+0000","last_peered":"2024-09-16T00:09:20.705526+0000","last_clean":"2024-09-16T00:09:20.705526+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:09:20.705526+0000","last_undegraded":"2024-09-16T00:09:20.705526+0000","last_fullsized":"2024-09-16T00:09:20.705526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":453356,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"68'99","reported_seq":205,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.077979+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:09:16.077979+0000","last_peered":"2024-09-16T00:09:16.077979+0000","last_clean":"2024-09-16T00:09:16.077979+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:09:16.077979+0000","last_undegraded":"2024-09-16T00:09:16.077979+0000","last_fullsized":"2024-09-16T00:09:16.077979+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":414019,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99,"num_write_kb":455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"73'113","reported_seq":227,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.677908+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:09:20.677908+0000","last_peered":"2024-09-16T00:09:20.677908+0000","last_clean":"2024-09-16T00:09:20.677908+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:09:20.677908+0000","last_undegraded":"2024-09-16T00:09:20.677908+0000","last_fullsized":"2024-09-16T00:09:20.677908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316565,"num_objects":113,"num_object_clones":0,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"69'115","reported_seq":220,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.478406+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:09:16.478406+0000","last_peered":"2024-09-16T00:09:16.478406+0000","last_clean":"2024-09-16T00:09:16.478406+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:09:16.478406+0000","last_undegraded":"2024-09-16T00:09:16.478406+0000","last_fullsized":"2024-09-16T00:09:16.478406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":745081,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"73'100","reported_seq":213,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101453+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:09:20.101453+0000","last_peered":"2024-09-16T00:09:20.101453+0000","last_clean":"2024-09-16T00:09:20.101453+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:09:20.101453+0000","last_undegraded":"2024-09-16T00:09:20.101453+0000","last_fullsized":"2024-09-16T00:09:20.101453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":346243,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"73'99","reported_seq":213,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119270+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:09:20.119270+0000","last_peered":"2024-09-16T00:09:20.119270+0000","last_clean":"2024-09-16T00:09:20.119270+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:09:20.119270+0000","last_undegraded":"2024-09-16T00:09:20.119270+0000","last_fullsized":"2024-09-16T00:09:20.119270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458275,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99,"num_write_kb":503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"73'103","reported_seq":216,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.107613+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:09:20.107613+0000","last_peered":"2024-09-16T00:09:20.107613+0000","last_clean":"2024-09-16T00:09:20.107613+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:09:20.107613+0000","last_undegraded":"2024-09-16T00:09:20.107613+0000","last_fullsized":"2024-09-16T00:09:20.107613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316000,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"72'122","reported_seq":208,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.411641+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:09:19.411641+0000","last_peered":"2024-09-16T00:09:19.411641+0000","last_clean":"2024-09-16T00:09:19.411641+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:09:19.411641+0000","last_undegraded":"2024-09-16T00:09:19.411641+0000","last_fullsized":"2024-09-16T00:09:19.411641+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410431,"num_objects":122,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122,"num_write_kb":469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"72'112","reported_seq":229,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.806282+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:09:19.806282+0000","last_peered":"2024-09-16T00:09:19.806282+0000","last_clean":"2024-09-16T00:09:19.806282+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:09:19.806282+0000","last_undegraded":"2024-09-16T00:09:19.806282+0000","last_fullsized":"2024-09-16T00:09:19.806282+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":381579,"num_objects":112,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"69'105","reported_seq":158,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.372491+0000","last_change":"2024-09-16T00:08:36.894101+0000","last_active":"2024-09-16T00:09:16.372491+0000","last_peered":"2024-09-16T00:09:16.372491+0000","last_clean":"2024-09-16T00:09:16.372491+0000","last_became_active":"2024-09-16T00:08:36.893807+0000","last_became_peered":"2024-09-16T00:08:36.893807+0000","last_unstale":"2024-09-16T00:09:16.372491+0000","last_undegraded":"2024-09-16T00:09:16.372491+0000","last_fullsized":"2024-09-16T00:09:16.372491+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:40.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":958407,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"69'102","reported_seq":207,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.461457+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:09:16.461457+0000","last_peered":"2024-09-16T00:09:16.461457+0000","last_clean":"2024-09-16T00:09:16.461457+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:09:16.461457+0000","last_undegraded":"2024-09-16T00:09:16.461457+0000","last_fullsized":"2024-09-16T00:09:16.461457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":304220,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"63'62","reported_seq":178,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119631+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:09:20.119631+0000","last_peered":"2024-09-16T00:09:20.119631+0000","last_clean":"2024-09-16T00:09:20.119631+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:09:20.119631+0000","last_undegraded":"2024-09-16T00:09:20.119631+0000","last_fullsized":"2024-09-16T00:09:20.119631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62,"num_write_kb":4136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"73'94","reported_seq":207,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.488618+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:09:20.488618+0000","last_peered":"2024-09-16T00:09:20.488618+0000","last_clean":"2024-09-16T00:09:20.488618+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:09:20.488618+0000","last_undegraded":"2024-09-16T00:09:20.488618+0000","last_fullsized":"2024-09-16T00:09:20.488618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":283620,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94,"num_write_kb":326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"67'39","reported_seq":91,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.099997+0000","last_change":"2024-09-16T00:08:36.890316+0000","last_active":"2024-09-16T00:09:16.099997+0000","last_peered":"2024-09-16T00:09:16.099997+0000","last_clean":"2024-09-16T00:09:16.099997+0000","last_became_active":"2024-09-16T00:08:36.889635+0000","last_became_peered":"2024-09-16T00:08:36.889635+0000","last_unstale":"2024-09-16T00:09:16.099997+0000","last_undegraded":"2024-09-16T00:09:16.099997+0000","last_fullsized":"2024-09-16T00:09:16.099997+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:28.843883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"63'1","reported_seq":127,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119169+0000","last_change":"2024-09-16T00:08:36.888349+0000","last_active":"2024-09-16T00:09:20.119169+0000","last_peered":"2024-09-16T00:09:20.119169+0000","last_clean":"2024-09-16T00:09:20.119169+0000","last_became_active":"2024-09-16T00:08:36.888052+0000","last_became_peered":"2024-09-16T00:08:36.888052+0000","last_unstale":"2024-09-16T00:09:20.119169+0000","last_undegraded":"2024-09-16T00:09:20.119169+0000","last_fullsized":"2024-09-16T00:09:20.119169+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:49.607254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"73'123","reported_seq":237,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.556875+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:09:20.556875+0000","last_peered":"2024-09-16T00:09:20.556875+0000","last_clean":"2024-09-16T00:09:20.556875+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:09:20.556875+0000","last_undegraded":"2024-09-16T00:09:20.556875+0000","last_fullsized":"2024-09-16T00:09:20.556875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":788879,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"66'2","reported_seq":110,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078017+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:09:16.078017+0000","last_peered":"2024-09-16T00:09:16.078017+0000","last_clean":"2024-09-16T00:09:16.078017+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:09:16.078017+0000","last_undegraded":"2024-09-16T00:09:16.078017+0000","last_fullsized":"2024-09-16T00:09:16.078017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"73'115","reported_seq":228,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.429522+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:09:20.429522+0000","last_peered":"2024-09-16T00:09:20.429522+0000","last_clean":"2024-09-16T00:09:20.429522+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:09:20.429522+0000","last_undegraded":"2024-09-16T00:09:20.429522+0000","last_fullsized":"2024-09-16T00:09:20.429522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":652806,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"61'1","reported_seq":108,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.100175+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:09:16.100175+0000","last_peered":"2024-09-16T00:09:16.100175+0000","last_clean":"2024-09-16T00:09:16.100175+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:09:16.100175+0000","last_undegraded":"2024-09-16T00:09:16.100175+0000","last_fullsized":"2024-09-16T00:09:16.100175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"69'106","reported_seq":212,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.407763+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:09:16.407763+0000","last_peered":"2024-09-16T00:09:16.407763+0000","last_clean":"2024-09-16T00:09:16.407763+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:09:16.407763+0000","last_undegraded":"2024-09-16T00:09:16.407763+0000","last_fullsized":"2024-09-16T00:09:16.407763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":465077,"num_objects":106,"num_object_clones":0,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":108,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078240+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:09:16.078240+0000","last_peered":"2024-09-16T00:09:16.078240+0000","last_clean":"2024-09-16T00:09:16.078240+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:09:16.078240+0000","last_undegraded":"2024-09-16T00:09:16.078240+0000","last_fullsized":"2024-09-16T00:09:16.078240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"73'107","reported_seq":220,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.579780+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:09:20.579780+0000","last_peered":"2024-09-16T00:09:20.579780+0000","last_clean":"2024-09-16T00:09:20.579780+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:09:20.579780+0000","last_undegraded":"2024-09-16T00:09:20.579780+0000","last_fullsized":"2024-09-16T00:09:20.579780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":864794,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107,"num_write_kb":903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"71'7","reported_seq":122,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101517+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:09:20.101517+0000","last_peered":"2024-09-16T00:09:20.101517+0000","last_clean":"2024-09-16T00:09:20.101517+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:09:20.101517+0000","last_undegraded":"2024-09-16T00:09:20.101517+0000","last_fullsized":"2024-09-16T00:09:20.101517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"69'88","reported_seq":193,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.088005+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:09:16.088005+0000","last_peered":"2024-09-16T00:09:16.088005+0000","last_clean":"2024-09-16T00:09:16.088005+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:09:16.088005+0000","last_undegraded":"2024-09-16T00:09:16.088005+0000","last_fullsized":"2024-09-16T00:09:16.088005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420942,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"65'22","reported_seq":75,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.100272+0000","last_change":"2024-09-16T00:08:36.914308+0000","last_active":"2024-09-16T00:09:16.100272+0000","last_peered":"2024-09-16T00:09:16.100272+0000","last_clean":"2024-09-16T00:09:16.100272+0000","last_became_active":"2024-09-16T00:08:36.890227+0000","last_became_peered":"2024-09-16T00:08:36.890227+0000","last_unstale":"2024-09-16T00:09:16.100272+0000","last_undegraded":"2024-09-16T00:09:16.100272+0000","last_fullsized":"2024-09-16T00:09:16.100272+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:31.761195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13110,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"69'85","reported_seq":190,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.090928+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:09:16.090928+0000","last_peered":"2024-09-16T00:09:16.090928+0000","last_clean":"2024-09-16T00:09:16.090928+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:09:16.090928+0000","last_undegraded":"2024-09-16T00:09:16.090928+0000","last_fullsized":"2024-09-16T00:09:16.090928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327524,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"71'7","reported_seq":122,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.107651+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:09:20.107651+0000","last_peered":"2024-09-16T00:09:20.107651+0000","last_clean":"2024-09-16T00:09:20.107651+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:09:20.107651+0000","last_undegraded":"2024-09-16T00:09:20.107651+0000","last_fullsized":"2024-09-16T00:09:20.107651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"73'104","reported_seq":217,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.407378+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:09:20.407378+0000","last_peered":"2024-09-16T00:09:20.407378+0000","last_clean":"2024-09-16T00:09:20.407378+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:09:20.407378+0000","last_undegraded":"2024-09-16T00:09:20.407378+0000","last_fullsized":"2024-09-16T00:09:20.407378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418435,"num_objects":104,"num_object_clones":0,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"61'1","reported_seq":109,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078207+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:09:16.078207+0000","last_peered":"2024-09-16T00:09:16.078207+0000","last_clean":"2024-09-16T00:09:16.078207+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:09:16.078207+0000","last_undegraded":"2024-09-16T00:09:16.078207+0000","last_fullsized":"2024-09-16T00:09:16.078207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"69'99","reported_seq":204,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.322192+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:09:16.322192+0000","last_peered":"2024-09-16T00:09:16.322192+0000","last_clean":"2024-09-16T00:09:16.322192+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:09:16.322192+0000","last_undegraded":"2024-09-16T00:09:16.322192+0000","last_fullsized":"2024-09-16T00:09:16.322192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2645634,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99,"num_write_kb":2639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"62'3","reported_seq":56,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.099989+0000","last_change":"2024-09-16T00:08:36.889720+0000","last_active":"2024-09-16T00:09:16.099989+0000","last_peered":"2024-09-16T00:09:16.099989+0000","last_clean":"2024-09-16T00:09:16.099989+0000","last_became_active":"2024-09-16T00:08:36.889345+0000","last_became_peered":"2024-09-16T00:08:36.889345+0000","last_unstale":"2024-09-16T00:09:16.099989+0000","last_undegraded":"2024-09-16T00:09:16.099989+0000","last_fullsized":"2024-09-16T00:09:16.099989+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:03.170073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"69'97","reported_seq":150,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.414321+0000","last_change":"2024-09-16T00:08:36.896787+0000","last_active":"2024-09-16T00:09:16.414321+0000","last_peered":"2024-09-16T00:09:16.414321+0000","last_clean":"2024-09-16T00:09:16.414321+0000","last_became_active":"2024-09-16T00:08:36.896505+0000","last_became_peered":"2024-09-16T00:08:36.896505+0000","last_unstale":"2024-09-16T00:09:16.414321+0000","last_undegraded":"2024-09-16T00:09:16.414321+0000","last_fullsized":"2024-09-16T00:09:16.414321+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:55.997551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4580108,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97,"num_write_kb":4521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"63'10","reported_seq":124,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.103086+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:09:19.103086+0000","last_peered":"2024-09-16T00:09:19.103086+0000","last_clean":"2024-09-16T00:09:19.103086+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:09:19.103086+0000","last_undegraded":"2024-09-16T00:09:19.103086+0000","last_fullsized":"2024-09-16T00:09:19.103086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"69'106","reported_seq":211,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.140958+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:09:16.140958+0000","last_peered":"2024-09-16T00:09:16.140958+0000","last_clean":"2024-09-16T00:09:16.140958+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:09:16.140958+0000","last_undegraded":"2024-09-16T00:09:16.140958+0000","last_fullsized":"2024-09-16T00:09:16.140958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387466,"num_objects":106,"num_object_clones":0,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"61'2","reported_seq":109,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.079842+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:09:16.079842+0000","last_peered":"2024-09-16T00:09:16.079842+0000","last_clean":"2024-09-16T00:09:16.079842+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:09:16.079842+0000","last_undegraded":"2024-09-16T00:09:16.079842+0000","last_fullsized":"2024-09-16T00:09:16.079842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"69'85","reported_seq":190,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.363811+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:09:16.363811+0000","last_peered":"2024-09-16T00:09:16.363811+0000","last_clean":"2024-09-16T00:09:16.363811+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:09:16.363811+0000","last_undegraded":"2024-09-16T00:09:16.363811+0000","last_fullsized":"2024-09-16T00:09:16.363811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":338686,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"71'4","reported_seq":118,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.103202+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:09:19.103202+0000","last_peered":"2024-09-16T00:09:19.103202+0000","last_clean":"2024-09-16T00:09:19.103202+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:09:19.103202+0000","last_undegraded":"2024-09-16T00:09:19.103202+0000","last_fullsized":"2024-09-16T00:09:19.103202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"69'90","reported_seq":195,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085461+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:09:16.085461+0000","last_peered":"2024-09-16T00:09:16.085461+0000","last_clean":"2024-09-16T00:09:16.085461+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:09:16.085461+0000","last_undegraded":"2024-09-16T00:09:16.085461+0000","last_fullsized":"2024-09-16T00:09:16.085461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":259850,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90,"num_write_kb":297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"61'2","reported_seq":117,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101525+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:09:20.101525+0000","last_peered":"2024-09-16T00:09:20.101525+0000","last_clean":"2024-09-16T00:09:20.101525+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:09:20.101525+0000","last_undegraded":"2024-09-16T00:09:20.101525+0000","last_fullsized":"2024-09-16T00:09:20.101525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"69'105","reported_seq":211,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078544+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:09:16.078544+0000","last_peered":"2024-09-16T00:09:16.078544+0000","last_clean":"2024-09-16T00:09:16.078544+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:09:16.078544+0000","last_undegraded":"2024-09-16T00:09:16.078544+0000","last_fullsized":"2024-09-16T00:09:16.078544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":598227,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"66'1","reported_seq":115,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.103233+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:09:19.103233+0000","last_peered":"2024-09-16T00:09:19.103233+0000","last_clean":"2024-09-16T00:09:19.103233+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:09:19.103233+0000","last_undegraded":"2024-09-16T00:09:19.103233+0000","last_fullsized":"2024-09-16T00:09:19.103233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"69'106","reported_seq":221,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.383992+0000","last_change":"2024-09-16T00:08:47.953122+0000","last_active":"2024-09-16T00:09:16.383992+0000","last_peered":"2024-09-16T00:09:16.383992+0000","last_clean":"2024-09-16T00:09:16.383992+0000","last_became_active":"2024-09-16T00:08:47.952799+0000","last_became_peered":"2024-09-16T00:08:47.952799+0000","last_unstale":"2024-09-16T00:09:16.383992+0000","last_undegraded":"2024-09-16T00:09:16.383992+0000","last_fullsized":"2024-09-16T00:09:16.383992+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:03.447562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":381576,"num_objects":106,"num_object_clones":0,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":263,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.084830+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:09:16.084830+0000","last_peered":"2024-09-16T00:09:16.084830+0000","last_clean":"2024-09-16T00:09:16.084830+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:09:16.084830+0000","last_undegraded":"2024-09-16T00:09:16.084830+0000","last_fullsized":"2024-09-16T00:09:16.084830+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"66'1","reported_seq":109,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078027+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:09:16.078027+0000","last_peered":"2024-09-16T00:09:16.078027+0000","last_clean":"2024-09-16T00:09:16.078027+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:09:16.078027+0000","last_undegraded":"2024-09-16T00:09:16.078027+0000","last_fullsized":"2024-09-16T00:09:16.078027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"73'100","reported_seq":214,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.688814+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:09:20.688814+0000","last_peered":"2024-09-16T00:09:20.688814+0000","last_clean":"2024-09-16T00:09:20.688814+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:09:20.688814+0000","last_undegraded":"2024-09-16T00:09:20.688814+0000","last_fullsized":"2024-09-16T00:09:20.688814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426614,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'2","reported_seq":109,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.084756+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:09:16.084756+0000","last_peered":"2024-09-16T00:09:16.084756+0000","last_clean":"2024-09-16T00:09:16.084756+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:09:16.084756+0000","last_undegraded":"2024-09-16T00:09:16.084756+0000","last_fullsized":"2024-09-16T00:09:16.084756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"73'97","reported_seq":210,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.534875+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:09:20.534875+0000","last_peered":"2024-09-16T00:09:20.534875+0000","last_clean":"2024-09-16T00:09:20.534875+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:09:20.534875+0000","last_undegraded":"2024-09-16T00:09:20.534875+0000","last_fullsized":"2024-09-16T00:09:20.534875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":253833,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97,"num_write_kb":301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"71'1","reported_seq":115,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.103092+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:09:19.103092+0000","last_peered":"2024-09-16T00:09:19.103092+0000","last_clean":"2024-09-16T00:09:19.103092+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:09:19.103092+0000","last_undegraded":"2024-09-16T00:09:19.103092+0000","last_fullsized":"2024-09-16T00:09:19.103092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"73'115","reported_seq":228,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.511361+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:09:20.511361+0000","last_peered":"2024-09-16T00:09:20.511361+0000","last_clean":"2024-09-16T00:09:20.511361+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:09:20.511361+0000","last_undegraded":"2024-09-16T00:09:20.511361+0000","last_fullsized":"2024-09-16T00:09:20.511361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":504645,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"67'3","reported_seq":111,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078296+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:09:16.078296+0000","last_peered":"2024-09-16T00:09:16.078296+0000","last_clean":"2024-09-16T00:09:16.078296+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:09:16.078296+0000","last_undegraded":"2024-09-16T00:09:16.078296+0000","last_fullsized":"2024-09-16T00:09:16.078296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"69'97","reported_seq":202,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.367943+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:09:16.367943+0000","last_peered":"2024-09-16T00:09:16.367943+0000","last_clean":"2024-09-16T00:09:16.367943+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:09:16.367943+0000","last_undegraded":"2024-09-16T00:09:16.367943+0000","last_fullsized":"2024-09-16T00:09:16.367943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":563894,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97,"num_write_kb":603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"61'1","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101534+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:09:20.101534+0000","last_peered":"2024-09-16T00:09:20.101534+0000","last_clean":"2024-09-16T00:09:20.101534+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:09:20.101534+0000","last_undegraded":"2024-09-16T00:09:20.101534+0000","last_fullsized":"2024-09-16T00:09:20.101534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"69'92","reported_seq":198,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.301283+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:09:16.301283+0000","last_peered":"2024-09-16T00:09:16.301283+0000","last_clean":"2024-09-16T00:09:16.301283+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:09:16.301283+0000","last_undegraded":"2024-09-16T00:09:16.301283+0000","last_fullsized":"2024-09-16T00:09:16.301283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212579,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085249+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:09:16.085249+0000","last_peered":"2024-09-16T00:09:16.085249+0000","last_clean":"2024-09-16T00:09:16.085249+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:09:16.085249+0000","last_undegraded":"2024-09-16T00:09:16.085249+0000","last_fullsized":"2024-09-16T00:09:16.085249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'104","reported_seq":217,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.431593+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:09:20.431593+0000","last_peered":"2024-09-16T00:09:20.431593+0000","last_clean":"2024-09-16T00:09:20.431593+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:09:20.431593+0000","last_undegraded":"2024-09-16T00:09:20.431593+0000","last_fullsized":"2024-09-16T00:09:20.431593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":498070,"num_objects":104,"num_object_clones":0,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"73'32","reported_seq":165,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101645+0000","last_change":"2024-09-16T00:08:36.906253+0000","last_active":"2024-09-16T00:09:20.101645+0000","last_peered":"2024-09-16T00:09:20.101645+0000","last_clean":"2024-09-16T00:09:20.101645+0000","last_became_active":"2024-09-16T00:08:36.894363+0000","last_became_peered":"2024-09-16T00:08:36.894363+0000","last_unstale":"2024-09-16T00:09:20.101645+0000","last_undegraded":"2024-09-16T00:09:20.101645+0000","last_fullsized":"2024-09-16T00:09:20.101645+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:00.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84,"num_write_kb":825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"73'108","reported_seq":222,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119533+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:09:20.119533+0000","last_peered":"2024-09-16T00:09:20.119533+0000","last_clean":"2024-09-16T00:09:20.119533+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:09:20.119533+0000","last_undegraded":"2024-09-16T00:09:20.119533+0000","last_fullsized":"2024-09-16T00:09:20.119533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":601448,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"61'3","reported_seq":110,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.100204+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:09:16.100204+0000","last_peered":"2024-09-16T00:09:16.100204+0000","last_clean":"2024-09-16T00:09:16.100204+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:09:16.100204+0000","last_undegraded":"2024-09-16T00:09:16.100204+0000","last_fullsized":"2024-09-16T00:09:16.100204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"72'109","reported_seq":222,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.107621+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:09:20.107621+0000","last_peered":"2024-09-16T00:09:20.107621+0000","last_clean":"2024-09-16T00:09:20.107621+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:09:20.107621+0000","last_undegraded":"2024-09-16T00:09:20.107621+0000","last_fullsized":"2024-09-16T00:09:20.107621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":298496,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":107,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.084727+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:09:16.084727+0000","last_peered":"2024-09-16T00:09:16.084727+0000","last_clean":"2024-09-16T00:09:16.084727+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:09:16.084727+0000","last_undegraded":"2024-09-16T00:09:16.084727+0000","last_fullsized":"2024-09-16T00:09:16.084727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"69'112","reported_seq":217,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.405822+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:09:16.405822+0000","last_peered":"2024-09-16T00:09:16.405822+0000","last_clean":"2024-09-16T00:09:16.405822+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:09:16.405822+0000","last_undegraded":"2024-09-16T00:09:16.405822+0000","last_fullsized":"2024-09-16T00:09:16.405822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318386,"num_objects":112,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"61'1","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101307+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:09:20.101307+0000","last_peered":"2024-09-16T00:09:20.101307+0000","last_clean":"2024-09-16T00:09:20.101307+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:09:20.101307+0000","last_undegraded":"2024-09-16T00:09:20.101307+0000","last_fullsized":"2024-09-16T00:09:20.101307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"73'105","reported_seq":219,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.559467+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:09:20.559467+0000","last_peered":"2024-09-16T00:09:20.559467+0000","last_clean":"2024-09-16T00:09:20.559467+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:09:20.559467+0000","last_undegraded":"2024-09-16T00:09:20.559467+0000","last_fullsized":"2024-09-16T00:09:20.559467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335746,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"63'2","reported_seq":117,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101354+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:09:20.101354+0000","last_peered":"2024-09-16T00:09:20.101354+0000","last_clean":"2024-09-16T00:09:20.101354+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:09:20.101354+0000","last_undegraded":"2024-09-16T00:09:20.101354+0000","last_fullsized":"2024-09-16T00:09:20.101354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"73'109","reported_seq":223,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119590+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:09:20.119590+0000","last_peered":"2024-09-16T00:09:20.119590+0000","last_clean":"2024-09-16T00:09:20.119590+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:09:20.119590+0000","last_undegraded":"2024-09-16T00:09:20.119590+0000","last_fullsized":"2024-09-16T00:09:20.119590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":372221,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"64'6","reported_seq":113,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.084974+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:09:16.084974+0000","last_peered":"2024-09-16T00:09:16.084974+0000","last_clean":"2024-09-16T00:09:16.084974+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:09:16.084974+0000","last_undegraded":"2024-09-16T00:09:16.084974+0000","last_fullsized":"2024-09-16T00:09:16.084974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"69'95","reported_seq":148,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.125392+0000","last_change":"2024-09-16T00:08:36.890347+0000","last_active":"2024-09-16T00:09:16.125392+0000","last_peered":"2024-09-16T00:09:16.125392+0000","last_clean":"2024-09-16T00:09:16.125392+0000","last_became_active":"2024-09-16T00:08:36.890009+0000","last_became_peered":"2024-09-16T00:08:36.890009+0000","last_unstale":"2024-09-16T00:09:16.125392+0000","last_undegraded":"2024-09-16T00:09:16.125392+0000","last_fullsized":"2024-09-16T00:09:16.125392+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:24.298978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353539,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95,"num_write_kb":393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"69'87","reported_seq":193,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.334327+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:09:16.334327+0000","last_peered":"2024-09-16T00:09:16.334327+0000","last_clean":"2024-09-16T00:09:16.334327+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:09:16.334327+0000","last_undegraded":"2024-09-16T00:09:16.334327+0000","last_fullsized":"2024-09-16T00:09:16.334327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240570,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87,"num_write_kb":283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"66'3","reported_seq":110,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085003+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:09:16.085003+0000","last_peered":"2024-09-16T00:09:16.085003+0000","last_clean":"2024-09-16T00:09:16.085003+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:09:16.085003+0000","last_undegraded":"2024-09-16T00:09:16.085003+0000","last_fullsized":"2024-09-16T00:09:16.085003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"69'95","reported_seq":201,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078513+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:09:16.078513+0000","last_peered":"2024-09-16T00:09:16.078513+0000","last_clean":"2024-09-16T00:09:16.078513+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:09:16.078513+0000","last_undegraded":"2024-09-16T00:09:16.078513+0000","last_fullsized":"2024-09-16T00:09:16.078513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":479096,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95,"num_write_kb":517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"66'6","reported_seq":131,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085152+0000","last_change":"2024-09-16T00:08:36.900704+0000","last_active":"2024-09-16T00:09:16.085152+0000","last_peered":"2024-09-16T00:09:16.085152+0000","last_clean":"2024-09-16T00:09:16.085152+0000","last_became_active":"2024-09-16T00:08:36.892520+0000","last_became_peered":"2024-09-16T00:08:36.892520+0000","last_unstale":"2024-09-16T00:09:16.085152+0000","last_undegraded":"2024-09-16T00:09:16.085152+0000","last_fullsized":"2024-09-16T00:09:16.085152+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:30.056041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"72'110","reported_seq":196,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.570967+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:09:19.570967+0000","last_peered":"2024-09-16T00:09:19.570967+0000","last_clean":"2024-09-16T00:09:19.570967+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:09:19.570967+0000","last_undegraded":"2024-09-16T00:09:19.570967+0000","last_fullsized":"2024-09-16T00:09:19.570967+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878216,"num_objects":110,"num_object_clones":0,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"72'39","reported_seq":155,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119425+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:09:20.119425+0000","last_peered":"2024-09-16T00:09:20.119425+0000","last_clean":"2024-09-16T00:09:20.119425+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:09:20.119425+0000","last_undegraded":"2024-09-16T00:09:20.119425+0000","last_fullsized":"2024-09-16T00:09:20.119425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":4141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"73'107","reported_seq":231,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.450104+0000","last_change":"2024-09-16T00:08:36.889082+0000","last_active":"2024-09-16T00:09:20.450104+0000","last_peered":"2024-09-16T00:09:20.450104+0000","last_clean":"2024-09-16T00:09:20.450104+0000","last_became_active":"2024-09-16T00:08:36.888807+0000","last_became_peered":"2024-09-16T00:08:36.888807+0000","last_unstale":"2024-09-16T00:09:20.450104+0000","last_undegraded":"2024-09-16T00:09:20.450104+0000","last_fullsized":"2024-09-16T00:09:20.450104+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:22.825324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":298398,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107,"num_write_kb":346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"65'58","reported_seq":173,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.107353+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:09:20.107353+0000","last_peered":"2024-09-16T00:09:20.107353+0000","last_clean":"2024-09-16T00:09:20.107353+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:09:20.107353+0000","last_undegraded":"2024-09-16T00:09:20.107353+0000","last_fullsized":"2024-09-16T00:09:20.107353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":4138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":39390995,"num_objects":6525,"num_object_clones":0,"num_object_copies":13050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6525,"num_write_kb":41915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":384148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90755072,"data_stored":78767502,"data_compressed":7812362,"data_compressed_allocated":9633792,"data_compressed_original":28786688,"omap_allocated":0,"internal_metadata":0},"log_size":6526,"ondisk_log_size":6526,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":25167294,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":419,"num_write_kb":26111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":13110,"num_keys_recovered":4,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10543104,"data_stored":50334588,"data_compressed":7770882,"data_compressed_allocated":9338880,"data_compressed_original":49184768,"omap_allocated":0,"internal_metadata":0},"log_size":322,"ondisk_log_size":322,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509467,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50176,"kb_used_data":14828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94321664,"statfs":{"total":96636764160,"available":96585383936,"internally_reserved":0,"allocated":15183872,"data_stored":23373544,"data_compressed":2945461,"data_compressed_allocated":3616768,"data_compressed_original":16404480,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509467,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43788,"kb_used_data":12592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328052,"statfs":{"total":96636764160,"available":96591925248,"internally_reserved":0,"allocated":12894208,"data_stored":18170022,"data_compressed":2007399,"data_compressed_allocated":2420736,"data_compressed_original":10899456,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509466,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45576,"kb_used_data":10224,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326264,"statfs":{"total":96636764160,"available":96590094336,"internally_reserved":0,"allocated":10469376,"data_stored":8320139,"data_compressed":1085058,"data_compressed_allocated":1335296,"data_compressed_original":3911680,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509467,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49168,"kb_used_data":13812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322672,"statfs":{"total":96636764160,"available":96586416128,"internally_reserved":0,"allocated":14143488,"data_stored":17711202,"data_compressed":2042984,"data_compressed_allocated":2514944,"data_compressed_original":10903552,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49076,"kb_used_data":13724,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322764,"statfs":{"total":96636764160,"available":96586510336,"internally_reserved":0,"allocated":14053376,"data_stored":21050186,"data_compressed":3148104,"data_compressed_allocated":3756032,"data_compressed_original":15941632,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47316,"kb_used_data":11968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94324524,"statfs":{"total":96636764160,"available":96588312576,"internally_reserved":0,"allocated":12255232,"data_stored":10931436,"data_compressed":929701,"data_compressed_allocated":1146880,"data_compressed_original":3465216,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56276,"kb_used_data":20924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94315564,"statfs":{"total":96636764160,"available":96579137536,"internally_reserved":0,"allocated":21426176,"data_stored":26980158,"data_compressed":2504865,"data_compressed_allocated":3092480,"data_compressed_original":13815808,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38060,"kb_used_data":6876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333780,"statfs":{"total":96636764160,"available":96597790720,"internally_reserved":0,"allocated":7041024,"data_stored":6367129,"data_compressed":910078,"data_compressed_allocated":1085440,"data_compressed_original":3596288,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1748992,"data_stored":8389246,"data_compressed":1255393,"data_compressed_allocated":1544192,"data_compressed_original":8196096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2703360,"data_stored":12583584,"data_compressed":1983184,"data_compressed_allocated":2338816,"data_compressed_original":12226560,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1675264,"data_stored":8388734,"data_compressed":1280256,"data_compressed_allocated":1560576,"data_compressed_original":8286208,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1712128,"data_stored":8389280,"data_compressed":1327713,"data_compressed_allocated":1560576,"data_compressed_original":8245248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2691072,"data_stored":12583494,"data_compressed":1924336,"data_compressed_allocated":2334720,"data_compressed_original":12230656,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":18800640,"data_stored":17629859,"data_compressed":1217095,"data_compressed_allocated":1503232,"data_compressed_original":5046272,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11399168,"data_stored":10541937,"data_compressed":929701,"data_compressed_allocated":1146880,"data_compressed_original":3465216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6352896,"data_stored":6045784,"data_compressed":910078,"data_compressed_allocated":1085440,"data_compressed_original":3596288,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10821632,"data_stored":8372607,"data_compressed":1239268,"data_compressed_allocated":1511424,"data_compressed_original":3895296,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11669504,"data_stored":8957737,"data_compressed":762728,"data_compressed_allocated":954368,"data_compressed_original":2617344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9666560,"data_stored":7955318,"data_compressed":1085058,"data_compressed_allocated":1335296,"data_compressed_original":3911680,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10383360,"data_stored":9416011,"data_compressed":679686,"data_compressed_allocated":860160,"data_compressed_original":2654208,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11661312,"data_stored":9848249,"data_compressed":988748,"data_compressed_allocated":1236992,"data_compressed_original":3600384,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:09:22.022 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T00:09:22.022 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:09:22.022 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T00:09:22.022 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T00:09:22.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.028+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd92415c390 con 0x7fd9240c59c0 2024-09-16T00:09:22.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.030+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd92415c390 con 0x7fd9240c59c0 2024-09-16T00:09:22.032 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:22.033 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:22.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.031+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fd92414a640 0x7fd92414aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.031+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fd92414af60 con 0x7fd92414a640 2024-09-16T00:09:22.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.031+0000 7fd928d2d640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fd92414a640 0x7fd92414aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.031+0000 7fd928d2d640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fd92414a640 0x7fd92414aa20 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.032+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd92414af60 con 0x7fd92414a640 2024-09-16T00:09:22.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.052+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd9240c3950 con 0x7fd92414a640 2024-09-16T00:09:22.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.054+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd9240c3950 con 0x7fd92414a640 2024-09-16T00:09:22.055 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:22.056 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:22.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.054+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fd9241f3a80 0x7fd9241f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.054+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fd9241f4420 con 0x7fd9241f3a80 2024-09-16T00:09:22.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.054+0000 7fd923fff640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fd9241f3a80 0x7fd9241f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.055+0000 7fd923fff640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fd9241f3a80 0x7fd9241f3ee0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.056+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9241f4420 con 0x7fd9241f3a80 2024-09-16T00:09:22.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.074+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd924062900 con 0x7fd9241f3a80 2024-09-16T00:09:22.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.075+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd924062900 con 0x7fd9241f3a80 2024-09-16T00:09:22.076 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:22.077 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:22.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.076+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fd9240ce260 0x7fd9240ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.076+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fd9240c4100 con 0x7fd9240ce260 2024-09-16T00:09:22.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.076+0000 7fd92952e640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fd9240ce260 0x7fd9240ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.076+0000 7fd92952e640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fd9240ce260 0x7fd9240ce6a0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.077+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9240c4100 con 0x7fd9240ce260 2024-09-16T00:09:22.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.095+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd9240c4100 con 0x7fd9240ce260 2024-09-16T00:09:22.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.098+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd9240c4100 con 0x7fd9240ce260 2024-09-16T00:09:22.099 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:22.099 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:22.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.098+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fd92402c640 0x7fd92402ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.098+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fd9240cf170 con 0x7fd92402c640 2024-09-16T00:09:22.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.098+0000 7fd928d2d640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fd92402c640 0x7fd92402ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.098+0000 7fd928d2d640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fd92402c640 0x7fd92402ca80 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.099+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd9240cf170 con 0x7fd92402c640 2024-09-16T00:09:22.105 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:22.104+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:22.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.118+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd9240cf170 con 0x7fd92402c640 2024-09-16T00:09:22.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.120+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd9240cf170 con 0x7fd92402c640 2024-09-16T00:09:22.121 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.120+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fd92402d520 0x7fd924206b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.120+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fd92402daf0 con 0x7fd92402d520 2024-09-16T00:09:22.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.120+0000 7fd923fff640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fd92402d520 0x7fd924206b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.121+0000 7fd923fff640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fd92402d520 0x7fd924206b60 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.122+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd92402daf0 con 0x7fd92402d520 2024-09-16T00:09:22.124 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:22.124 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:22.138 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:09:22.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.140+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd9242070a0 con 0x7fd92402d520 2024-09-16T00:09:22.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.142+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd9242070a0 con 0x7fd92402d520 2024-09-16T00:09:22.143 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:22.144 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:22.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.142+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fd9242077c0 0x7fd924209bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.142+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fd92420ca50 con 0x7fd9242077c0 2024-09-16T00:09:22.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.143+0000 7fd92952e640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fd9242077c0 0x7fd924209bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.143+0000 7fd92952e640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fd9242077c0 0x7fd924209bb0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.144+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd92420ca50 con 0x7fd9242077c0 2024-09-16T00:09:22.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.152+0000 7f010bffd640 1 Processor -- start 2024-09-16T00:09:22.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.152+0000 7f010bffd640 1 -- start start 2024-09-16T00:09:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.153+0000 7f010bffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01041548c0 0x7f010406eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.153+0000 7f010bffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f010406bb20 0x7f010406f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.154+0000 7f010bffd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01040c7f90 con 0x7f01040c8a30 2024-09-16T00:09:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.154+0000 7f010bffd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f01040c9820 con 0x7f010406bb20 2024-09-16T00:09:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.154+0000 7f010bffd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f01040cabd0 con 0x7f01041548c0 2024-09-16T00:09:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.154+0000 7f0109d99640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01041548c0 0x7f010406eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.154+0000 7f0109d99640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01041548c0 0x7f010406eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:49312/0 (socket says 172.21.15.139:49312) 2024-09-16T00:09:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.154+0000 7f0109d99640 1 -- 172.21.15.139:0/711148014 learned_addr learned my addr 172.21.15.139:0/711148014 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:22.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.154+0000 7f0109d99640 1 -- 172.21.15.139:0/711148014 >> v2:172.21.15.139:3300/0 conn(0x7f010406bb20 msgr2=0x7f010406f000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:09:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.154+0000 7f0109d99640 1 --2- 172.21.15.139:0/711148014 >> v2:172.21.15.139:3300/0 conn(0x7f010406bb20 0x7f010406f000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.154+0000 7f0109d99640 1 -- 172.21.15.139:0/711148014 >> v1:172.21.15.139:6789/0 conn(0x7f01040c8a30 legacy=0x7f010415a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.154+0000 7f0109d99640 1 -- 172.21.15.139:0/711148014 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01040c8250 con 0x7f01041548c0 2024-09-16T00:09:22.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.155+0000 7f0109d99640 1 --2- 172.21.15.139:0/711148014 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01041548c0 0x7f010406eac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0100009870 tx=0x7f010002a310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d82415b1718b8801 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.155+0000 7f0108d97640 1 -- 172.21.15.139:0/711148014 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100037070 con 0x7f01041548c0 2024-09-16T00:09:22.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.155+0000 7f0108d97640 1 -- 172.21.15.139:0/711148014 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0100032640 con 0x7f01041548c0 2024-09-16T00:09:22.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.155+0000 7f0108d97640 1 -- 172.21.15.139:0/711148014 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010002ae60 con 0x7f01041548c0 2024-09-16T00:09:22.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.156+0000 7f010bffd640 1 -- 172.21.15.139:0/711148014 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01041548c0 msgr2=0x7f010406eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.156+0000 7f010bffd640 1 --2- 172.21.15.139:0/711148014 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01041548c0 0x7f010406eac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0100009870 tx=0x7f010002a310 comp rx=0 tx=0).stop 2024-09-16T00:09:22.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.156+0000 7f010bffd640 1 -- 172.21.15.139:0/711148014 shutdown_connections 2024-09-16T00:09:22.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.156+0000 7f010bffd640 1 --2- 172.21.15.139:0/711148014 >> v2:172.21.15.139:3300/0 conn(0x7f010406bb20 0x7f010406f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.156+0000 7f010bffd640 1 --2- 172.21.15.139:0/711148014 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01041548c0 0x7f010406eac0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.156+0000 7f010bffd640 1 -- 172.21.15.139:0/711148014 >> 172.21.15.139:0/711148014 conn(0x7f0104067950 msgr2=0x7f0104067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:22.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.157+0000 7f010bffd640 1 -- 172.21.15.139:0/711148014 shutdown_connections 2024-09-16T00:09:22.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.157+0000 7f010bffd640 1 -- 172.21.15.139:0/711148014 wait complete. 2024-09-16T00:09:22.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.158+0000 7f010bffd640 1 Processor -- start 2024-09-16T00:09:22.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.158+0000 7f010bffd640 1 -- start start 2024-09-16T00:09:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.159+0000 7f010bffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f010406bb20 0x7f0104159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.159+0000 7f010bffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01040c8a30 0x7f010414ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.159+0000 7f010bffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01041548c0 0x7f01041f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.159+0000 7f010bffd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f01040c91d0 con 0x7f01041548c0 2024-09-16T00:09:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.159+0000 7f0109d99640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f010406bb20 0x7f0104159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.159+0000 7f0109598640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01040c8a30 0x7f010414ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.159+0000 7f0109d99640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f010406bb20 0x7f0104159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:47684/0 (socket says 172.21.15.139:47684) 2024-09-16T00:09:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.159+0000 7f0109d99640 1 -- 172.21.15.139:0/595203882 learned_addr learned my addr 172.21.15.139:0/595203882 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.159+0000 7f010bffd640 1 -- 172.21.15.139:0/595203882 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01040c8ef0 con 0x7f010406bb20 2024-09-16T00:09:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.159+0000 7f010bffd640 1 -- 172.21.15.139:0/595203882 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f01040c94b0 con 0x7f01040c8a30 2024-09-16T00:09:22.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.159+0000 7f010a59a640 1 --2- 172.21.15.139:0/595203882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01041548c0 0x7f01041f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.159+0000 7f0109d99640 1 -- 172.21.15.139:0/595203882 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01040c8a30 msgr2=0x7f010414ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.159+0000 7f0109d99640 1 --2- 172.21.15.139:0/595203882 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01040c8a30 0x7f010414ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.159+0000 7f0109d99640 1 -- 172.21.15.139:0/595203882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01041548c0 msgr2=0x7f01041f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.160+0000 7f0109d99640 1 --2- 172.21.15.139:0/595203882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01041548c0 0x7f01041f3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.160+0000 7f0109d99640 1 -- 172.21.15.139:0/595203882 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01041598a0 con 0x7f010406bb20 2024-09-16T00:09:22.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.160+0000 7f010a59a640 1 --2- 172.21.15.139:0/595203882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01041548c0 0x7f01041f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:22.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.160+0000 7f0109598640 1 --2- 172.21.15.139:0/595203882 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01040c8a30 0x7f010414ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:22.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.160+0000 7f0109d99640 1 --2- 172.21.15.139:0/595203882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f010406bb20 0x7f0104159360 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0100000c00 tx=0x7f01000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.160+0000 7f00fa7fc640 1 -- 172.21.15.139:0/595203882 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100041030 con 0x7f010406bb20 2024-09-16T00:09:22.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.160+0000 7f00fa7fc640 1 -- 172.21.15.139:0/595203882 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0100030770 con 0x7f010406bb20 2024-09-16T00:09:22.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.160+0000 7f00fa7fc640 1 -- 172.21.15.139:0/595203882 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100037040 con 0x7f010406bb20 2024-09-16T00:09:22.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.160+0000 7f010bffd640 1 -- 172.21.15.139:0/595203882 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01041e38d0 con 0x7f010406bb20 2024-09-16T00:09:22.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.160+0000 7f010bffd640 1 -- 172.21.15.139:0/595203882 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01041e2890 con 0x7f010406bb20 2024-09-16T00:09:22.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.162+0000 7f010bffd640 1 -- 172.21.15.139:0/595203882 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01040cbb80 con 0x7f010406bb20 2024-09-16T00:09:22.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.162+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd92420ca50 con 0x7fd9242077c0 2024-09-16T00:09:22.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.164+0000 7f00fa7fc640 1 -- 172.21.15.139:0/595203882 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0100044070 con 0x7f010406bb20 2024-09-16T00:09:22.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.164+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd92420ca50 con 0x7fd9242077c0 2024-09-16T00:09:22.165 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:22.165 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:09:22.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.164+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd92420a600 0x7fd92420aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.164+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fd92420fe20 con 0x7fd92420a600 2024-09-16T00:09:22.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.164+0000 7f00fa7fc640 1 --2- 172.21.15.139:0/595203882 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f00e806c2b0 0x7f00e806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.164+0000 7fd928d2d640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd92420a600 0x7fd92420aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.164+0000 7f00fa7fc640 1 -- 172.21.15.139:0/595203882 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5944+0+0 (secure 0 0 0) 0x7f01000ab850 con 0x7f010406bb20 2024-09-16T00:09:22.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.164+0000 7f0109598640 1 --2- 172.21.15.139:0/595203882 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f00e806c2b0 0x7f00e806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.164+0000 7fd928d2d640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd92420a600 0x7fd92420aa40 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.165+0000 7f0109598640 1 --2- 172.21.15.139:0/595203882 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f00e806c2b0 0x7f00e806e770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0104159dc0 tx=0x7f00f40090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.165+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd92420fe20 con 0x7fd92420a600 2024-09-16T00:09:22.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.169+0000 7f00fa7fc640 1 -- 172.21.15.139:0/595203882 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f01040cbb80 con 0x7f010406bb20 2024-09-16T00:09:22.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.182+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd92420b090 con 0x7fd92420a600 2024-09-16T00:09:22.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.184+0000 7fd9217fa640 1 -- 172.21.15.139:0/4179807407 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd92420b090 con 0x7fd92420a600 2024-09-16T00:09:22.185 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:22.187 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:22.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.184+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fd9240c59c0 msgr2=0x7fd9240c5da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.184+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fd9240c59c0 0x7fd9240c5da0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.184+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fd92414a640 msgr2=0x7fd92414aa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fd92414a640 0x7fd92414aa20 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fd9241f3a80 msgr2=0x7fd9241f3ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fd9241f3a80 0x7fd9241f3ee0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fd9240ce260 msgr2=0x7fd9240ce6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fd9240ce260 0x7fd9240ce6a0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fd92402c640 msgr2=0x7fd92402ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fd92402c640 0x7fd92402ca80 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fd92402d520 msgr2=0x7fd924206b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fd92402d520 0x7fd924206b60 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fd9242077c0 msgr2=0x7fd924209bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fd9242077c0 0x7fd924209bb0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd92420a600 msgr2=0x7fd92420aa40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd92420a600 0x7fd92420aa40 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92952e640 1 -- 172.21.15.139:0/4179807407 reap_dead start 2024-09-16T00:09:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92952e640 1 -- 172.21.15.139:0/4179807407 reap_dead start 2024-09-16T00:09:22.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd90806c2e0 msgr2=0x7fd90806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd90806c2e0 0x7fd90806e7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd918009630 tx=0x7fd918009240 comp rx=0 tx=0).stop 2024-09-16T00:09:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92952e640 1 -- 172.21.15.139:0/4179807407 reap_dead start 2024-09-16T00:09:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92952e640 1 -- 172.21.15.139:0/4179807407 reap_dead start 2024-09-16T00:09:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9240d0490 msgr2=0x7fd9240c4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9240d0490 0x7fd9240c4970 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd91000e870 tx=0x7fd91000ec70 comp rx=0 tx=0).stop 2024-09-16T00:09:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92952e640 1 -- 172.21.15.139:0/4179807407 reap_dead start 2024-09-16T00:09:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 shutdown_connections 2024-09-16T00:09:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 --2- 172.21.15.139:0/4179807407 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9240d0490 0x7fd9240c4970 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 >> 172.21.15.139:0/4179807407 conn(0x7fd92406c520 msgr2=0x7fd924158b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 shutdown_connections 2024-09-16T00:09:22.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.185+0000 7fd92af91640 1 -- 172.21.15.139:0/4179807407 wait complete. 2024-09-16T00:09:22.238 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:22.238+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:22.264 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:22.264 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:22.274 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:09:22.307 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:22.306+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:22.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.378+0000 7f010bffd640 1 -- 172.21.15.139:0/595203882 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f01040cc140 con 0x7f010406bb20 2024-09-16T00:09:22.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.380+0000 7f00fa7fc640 1 -- 172.21.15.139:0/595203882 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v74) ==== 81+0+35561 (secure 0 0 0) 0x7f01040cc140 con 0x7f010406bb20 2024-09-16T00:09:22.381 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:09:22.381 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:22.381 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 74, 2024-09-16T00:09:22.381 INFO:teuthology.orchestra.run.smithi139.stdout: "fsid": "2d3366e5-6a59-4b59-93b2-615e72ac7015", 2024-09-16T00:09:22.382 INFO:teuthology.orchestra.run.smithi139.stdout: "created": "2024-09-16T00:07:09.967833+0000", 2024-09-16T00:09:22.382 INFO:teuthology.orchestra.run.smithi139.stdout: "modified": "2024-09-16T00:09:21.103705+0000", 2024-09-16T00:09:22.382 INFO:teuthology.orchestra.run.smithi139.stdout: "last_up_change": "2024-09-16T00:07:21.049678+0000", 2024-09-16T00:09:22.382 INFO:teuthology.orchestra.run.smithi139.stdout: "last_in_change": "2024-09-16T00:07:12.025620+0000", 2024-09-16T00:09:22.382 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T00:09:22.382 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_num": 5806080, 2024-09-16T00:09:22.382 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_set": [ 2024-09-16T00:09:22.382 INFO:teuthology.orchestra.run.smithi139.stdout: "nodeep-scrub", 2024-09-16T00:09:22.382 INFO:teuthology.orchestra.run.smithi139.stdout: "noscrub", 2024-09-16T00:09:22.382 INFO:teuthology.orchestra.run.smithi139.stdout: "pglog_hardlimit", 2024-09-16T00:09:22.382 INFO:teuthology.orchestra.run.smithi139.stdout: "purged_snapdirs", 2024-09-16T00:09:22.382 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_deletes", 2024-09-16T00:09:22.382 INFO:teuthology.orchestra.run.smithi139.stdout: "sortbitwise" 2024-09-16T00:09:22.383 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:22.383 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_version": 6, 2024-09-16T00:09:22.383 INFO:teuthology.orchestra.run.smithi139.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T00:09:22.383 INFO:teuthology.orchestra.run.smithi139.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T00:09:22.383 INFO:teuthology.orchestra.run.smithi139.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T00:09:22.383 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_snapshot": "", 2024-09-16T00:09:22.383 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_max": 3, 2024-09-16T00:09:22.383 INFO:teuthology.orchestra.run.smithi139.stdout: "max_osd": 8, 2024-09-16T00:09:22.383 INFO:teuthology.orchestra.run.smithi139.stdout: "require_min_compat_client": "luminous", 2024-09-16T00:09:22.383 INFO:teuthology.orchestra.run.smithi139.stdout: "min_compat_client": "luminous", 2024-09-16T00:09:22.383 INFO:teuthology.orchestra.run.smithi139.stdout: "require_osd_release": "squid", 2024-09-16T00:09:22.383 INFO:teuthology.orchestra.run.smithi139.stdout: "allow_crimson": false, 2024-09-16T00:09:22.383 INFO:teuthology.orchestra.run.smithi139.stdout: "pools": [ 2024-09-16T00:09:22.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.384 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 1, 2024-09-16T00:09:22.384 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": ".mgr", 2024-09-16T00:09:22.384 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:22.056303+0000", 2024-09-16T00:09:22.384 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:09:22.384 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:09:22.384 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:09:22.384 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:09:22.384 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:09:22.384 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:09:22.384 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:09:22.384 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:09:22.384 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:09:22.385 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:09:22.385 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:09:22.385 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:09:22.385 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:09:22.385 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 1, 2024-09-16T00:09:22.385 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 1, 2024-09-16T00:09:22.385 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 1, 2024-09-16T00:09:22.385 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 1, 2024-09-16T00:09:22.385 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 1, 2024-09-16T00:09:22.385 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:09:22.385 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:09:22.385 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:09:22.385 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:09:22.385 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:09:22.386 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:09:22.386 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:09:22.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.386 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "18", 2024-09-16T00:09:22.386 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:09:22.386 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:09:22.386 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:09:22.386 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:09:22.386 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:09:22.386 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:09:22.386 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:09:22.386 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:09:22.386 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:09:22.387 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:09:22.387 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:09:22.387 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:09:22.387 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:09:22.387 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:09:22.387 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:09:22.387 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:09:22.387 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:09:22.387 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:09:22.387 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:09:22.387 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:09:22.387 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:09:22.387 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:09:22.387 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:09:22.388 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:09:22.388 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:09:22.388 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:09:22.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.388 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:09:22.388 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:09:22.388 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:09:22.388 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:09:22.388 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:09:22.388 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:09:22.388 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:09:22.388 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:09:22.388 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:09:22.389 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:09:22.389 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:09:22.389 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:09:22.389 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_max": 32, 2024-09-16T00:09:22.389 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 1 2024-09-16T00:09:22.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.389 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:09:22.389 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": {} 2024-09-16T00:09:22.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.389 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:09:22.389 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:09:22.389 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 8, 2024-09-16T00:09:22.389 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 8, 2024-09-16T00:09:22.390 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 0.25, 2024-09-16T00:09:22.390 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 2, 2024-09-16T00:09:22.390 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 2, 2024-09-16T00:09:22.390 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:09:22.390 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:09:22.390 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:09:22.390 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.390 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 2, 2024-09-16T00:09:22.390 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": "cephfs_metadata", 2024-09-16T00:09:22.390 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:30.768649+0000", 2024-09-16T00:09:22.390 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:09:22.390 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:09:22.391 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:09:22.391 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:09:22.391 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:09:22.391 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:09:22.391 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:09:22.391 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:09:22.391 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:09:22.391 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:09:22.391 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:09:22.391 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:09:22.391 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:09:22.391 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 32, 2024-09-16T00:09:22.391 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 32, 2024-09-16T00:09:22.391 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 32, 2024-09-16T00:09:22.392 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 32, 2024-09-16T00:09:22.392 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 32, 2024-09-16T00:09:22.392 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:09:22.392 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:09:22.392 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:09:22.392 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:09:22.392 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:09:22.392 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:09:22.392 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:09:22.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.392 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "21", 2024-09-16T00:09:22.392 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:09:22.392 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:09:22.393 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:09:22.393 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:09:22.393 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:09:22.393 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:09:22.393 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:09:22.393 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:09:22.393 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:09:22.393 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:09:22.393 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:09:22.393 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:09:22.393 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:09:22.393 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:09:22.393 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:09:22.393 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:09:22.394 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:09:22.394 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:09:22.394 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:09:22.394 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:09:22.394 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:09:22.394 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:09:22.394 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:09:22.394 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:09:22.394 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:09:22.394 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:09:22.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.394 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:09:22.394 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:09:22.394 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:09:22.395 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:09:22.395 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:09:22.395 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:09:22.395 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:09:22.395 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:09:22.395 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:09:22.395 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:09:22.395 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:09:22.395 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:09:22.395 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_bias": 4, 2024-09-16T00:09:22.395 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 16, 2024-09-16T00:09:22.395 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_priority": 5 2024-09-16T00:09:22.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.395 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:09:22.396 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs": { 2024-09-16T00:09:22.396 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": "cephfs" 2024-09-16T00:09:22.396 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.396 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:09:22.396 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:09:22.396 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 1.5, 2024-09-16T00:09:22.396 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 1.5, 2024-09-16T00:09:22.396 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 1, 2024-09-16T00:09:22.396 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 1.5, 2024-09-16T00:09:22.396 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 1.5, 2024-09-16T00:09:22.396 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:09:22.396 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:09:22.396 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:09:22.397 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.397 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 3, 2024-09-16T00:09:22.397 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": "cephfs_data", 2024-09-16T00:09:22.397 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:32.008209+0000", 2024-09-16T00:09:22.397 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:09:22.397 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:09:22.397 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:09:22.397 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:09:22.397 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:09:22.397 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:09:22.397 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:09:22.398 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:09:22.398 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:09:22.398 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:09:22.398 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:09:22.398 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:09:22.398 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:09:22.398 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 64, 2024-09-16T00:09:22.398 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 64, 2024-09-16T00:09:22.398 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 64, 2024-09-16T00:09:22.398 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 64, 2024-09-16T00:09:22.398 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 64, 2024-09-16T00:09:22.398 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:09:22.398 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:09:22.398 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:09:22.399 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:09:22.399 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:09:22.399 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:09:22.399 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:09:22.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.399 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "22", 2024-09-16T00:09:22.399 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:09:22.399 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:09:22.399 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:09:22.399 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:09:22.399 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:09:22.399 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:09:22.399 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:09:22.400 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:09:22.400 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:09:22.400 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:09:22.400 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:09:22.400 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:09:22.400 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:09:22.400 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:09:22.400 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:09:22.400 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:09:22.400 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:09:22.400 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:09:22.400 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:09:22.400 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:09:22.400 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:09:22.401 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:09:22.401 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:09:22.401 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:09:22.401 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:09:22.401 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:09:22.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.401 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:09:22.401 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:09:22.401 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:09:22.401 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:09:22.401 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:09:22.401 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:09:22.401 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:09:22.402 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:09:22.402 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:09:22.402 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:09:22.402 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:09:22.402 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:09:22.402 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 64, 2024-09-16T00:09:22.402 INFO:teuthology.orchestra.run.smithi139.stdout: "target_size_ratio": 0.89999997615814209 2024-09-16T00:09:22.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.402 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:09:22.402 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs": { 2024-09-16T00:09:22.402 INFO:teuthology.orchestra.run.smithi139.stdout: "data": "cephfs" 2024-09-16T00:09:22.402 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.402 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:09:22.403 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:09:22.403 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 1.25, 2024-09-16T00:09:22.403 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 1.25, 2024-09-16T00:09:22.403 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 1, 2024-09-16T00:09:22.403 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 1.25, 2024-09-16T00:09:22.403 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 1.25, 2024-09-16T00:09:22.403 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:09:22.403 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:09:22.403 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:09:22.403 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.403 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.403 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:22.403 INFO:teuthology.orchestra.run.smithi139.stdout: "osds": [ 2024-09-16T00:09:22.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.404 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 0, 2024-09-16T00:09:22.404 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "83699a05-4937-43c3-aefc-b669bed9bb6f", 2024-09-16T00:09:22.404 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:22.404 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:22.404 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:22.404 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:22.404 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:22.404 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:22.404 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 14, 2024-09-16T00:09:22.404 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 37, 2024-09-16T00:09:22.404 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:22.404 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:22.404 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:22.405 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.405 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.405 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6816", 2024-09-16T00:09:22.405 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:22.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.405 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.405 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6817", 2024-09-16T00:09:22.405 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:22.405 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.405 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.405 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:22.406 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.406 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.406 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6818", 2024-09-16T00:09:22.406 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:22.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.406 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.406 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6819", 2024-09-16T00:09:22.406 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:22.406 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.406 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.407 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:22.407 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.407 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.407 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6822", 2024-09-16T00:09:22.407 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:22.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.407 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.407 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6823", 2024-09-16T00:09:22.407 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:22.407 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.407 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.408 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:22.408 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.408 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.408 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6820", 2024-09-16T00:09:22.408 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:22.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.408 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.408 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6821", 2024-09-16T00:09:22.408 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:22.408 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.408 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.409 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6817/1974901438", 2024-09-16T00:09:22.409 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6819/1974901438", 2024-09-16T00:09:22.409 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6823/1974901438", 2024-09-16T00:09:22.409 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6821/1974901438", 2024-09-16T00:09:22.409 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:22.409 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:22.409 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:22.409 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.409 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 1, 2024-09-16T00:09:22.409 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "9c04991e-9695-4bbb-8011-bf613bf404b3", 2024-09-16T00:09:22.409 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:22.410 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:22.410 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:22.410 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:22.410 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:22.410 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:22.410 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 14, 2024-09-16T00:09:22.410 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:09:22.410 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:22.410 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:22.410 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:22.410 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.410 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.410 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6808", 2024-09-16T00:09:22.411 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:22.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.411 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.411 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6809", 2024-09-16T00:09:22.411 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:22.411 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.411 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.411 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:22.411 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.411 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.411 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6810", 2024-09-16T00:09:22.412 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:22.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.412 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.412 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6811", 2024-09-16T00:09:22.412 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:22.412 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.412 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.412 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:22.412 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.412 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.412 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6814", 2024-09-16T00:09:22.413 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:22.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.413 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.413 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6815", 2024-09-16T00:09:22.413 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:22.413 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.413 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.413 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:22.413 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.413 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.413 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6812", 2024-09-16T00:09:22.414 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:22.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.414 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.414 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6813", 2024-09-16T00:09:22.414 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:22.414 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.414 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.414 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6809/2232380998", 2024-09-16T00:09:22.414 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6811/2232380998", 2024-09-16T00:09:22.414 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6815/2232380998", 2024-09-16T00:09:22.414 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6813/2232380998", 2024-09-16T00:09:22.414 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:22.415 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:22.415 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:22.415 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.415 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 2, 2024-09-16T00:09:22.415 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "8048fc64-72b3-4866-ac62-6a8b2bf6f5c8", 2024-09-16T00:09:22.415 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:22.415 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:22.415 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:22.415 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:22.415 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:22.415 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:22.416 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 13, 2024-09-16T00:09:22.416 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 68, 2024-09-16T00:09:22.416 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:22.416 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:22.416 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:22.416 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.416 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.416 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6800", 2024-09-16T00:09:22.416 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:22.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.416 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.416 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6801", 2024-09-16T00:09:22.417 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:22.417 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.417 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.417 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:22.417 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.417 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.417 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6802", 2024-09-16T00:09:22.417 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:22.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.417 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.418 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6803", 2024-09-16T00:09:22.418 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:22.418 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.418 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.418 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:22.418 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.418 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.418 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6806", 2024-09-16T00:09:22.418 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:22.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.418 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.419 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6807", 2024-09-16T00:09:22.419 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:22.419 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.419 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.419 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:22.419 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.419 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.419 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6804", 2024-09-16T00:09:22.419 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:22.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.420 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.420 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6805", 2024-09-16T00:09:22.420 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:22.420 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.420 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.420 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6801/4180588905", 2024-09-16T00:09:22.420 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6803/4180588905", 2024-09-16T00:09:22.420 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6807/4180588905", 2024-09-16T00:09:22.420 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6805/4180588905", 2024-09-16T00:09:22.420 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:22.420 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:22.420 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:22.420 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.421 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 3, 2024-09-16T00:09:22.421 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "c34ac77b-354c-4884-902d-3e2a84f5eb27", 2024-09-16T00:09:22.421 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:22.421 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:22.421 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:22.421 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:22.421 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:22.421 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:22.421 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:09:22.421 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:09:22.421 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:22.422 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:22.422 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:22.422 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.422 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.422 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6826", 2024-09-16T00:09:22.422 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:22.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.422 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.422 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6827", 2024-09-16T00:09:22.422 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:22.422 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.422 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.423 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:22.423 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.423 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.423 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6828", 2024-09-16T00:09:22.423 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:22.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.423 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.423 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6829", 2024-09-16T00:09:22.423 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:22.423 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.423 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.424 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:22.424 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.424 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.424 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6832", 2024-09-16T00:09:22.424 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:22.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.424 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.424 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6833", 2024-09-16T00:09:22.424 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:22.424 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.425 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.425 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:22.425 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.425 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.425 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6830", 2024-09-16T00:09:22.425 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:22.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.425 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.425 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6831", 2024-09-16T00:09:22.425 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:22.425 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.426 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.426 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6827/3947768492", 2024-09-16T00:09:22.426 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6829/3947768492", 2024-09-16T00:09:22.426 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6833/3947768492", 2024-09-16T00:09:22.426 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6831/3947768492", 2024-09-16T00:09:22.426 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:22.426 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:22.426 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:22.426 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.426 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 4, 2024-09-16T00:09:22.426 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "ba9a7c93-9672-43e1-8c98-2878cdf7f548", 2024-09-16T00:09:22.427 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:22.427 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:22.427 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:22.427 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:22.427 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:22.427 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:22.427 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:09:22.427 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:09:22.427 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:22.427 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:22.427 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:22.427 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.427 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.428 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6816", 2024-09-16T00:09:22.428 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:22.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.428 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.428 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6817", 2024-09-16T00:09:22.428 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:22.428 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.428 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.428 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:22.428 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.429 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.429 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6818", 2024-09-16T00:09:22.429 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:22.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.429 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.429 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6819", 2024-09-16T00:09:22.429 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:22.429 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.429 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.430 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:22.430 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.430 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.430 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6822", 2024-09-16T00:09:22.430 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:22.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.430 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.430 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6823", 2024-09-16T00:09:22.430 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:22.431 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.431 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.431 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:22.431 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.431 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.431 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6820", 2024-09-16T00:09:22.431 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:22.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.431 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.431 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6821", 2024-09-16T00:09:22.432 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:22.432 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.432 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.432 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6817/642632916", 2024-09-16T00:09:22.432 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6819/642632916", 2024-09-16T00:09:22.432 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6823/642632916", 2024-09-16T00:09:22.432 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6821/642632916", 2024-09-16T00:09:22.432 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:22.433 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:22.433 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:22.433 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.433 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 5, 2024-09-16T00:09:22.433 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "46a0916b-474c-411b-84ad-86f27d4cbf53", 2024-09-16T00:09:22.433 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:22.433 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:22.433 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:22.433 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:22.433 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:22.433 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:22.433 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:09:22.434 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 37, 2024-09-16T00:09:22.434 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:22.434 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:22.434 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:22.434 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.434 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.434 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6800", 2024-09-16T00:09:22.434 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:22.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.434 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.434 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6801", 2024-09-16T00:09:22.435 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:22.435 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.435 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.435 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:22.435 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.435 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.435 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6802", 2024-09-16T00:09:22.435 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:22.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.435 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.435 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6803", 2024-09-16T00:09:22.436 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:22.436 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.436 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.436 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:22.436 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.436 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.436 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6806", 2024-09-16T00:09:22.436 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:22.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.437 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.437 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6807", 2024-09-16T00:09:22.437 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:22.437 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.437 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.437 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:22.437 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.437 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.437 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6804", 2024-09-16T00:09:22.437 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:22.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.438 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.438 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6805", 2024-09-16T00:09:22.438 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:22.438 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.438 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.438 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6801/1051778383", 2024-09-16T00:09:22.438 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6803/1051778383", 2024-09-16T00:09:22.438 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6807/1051778383", 2024-09-16T00:09:22.438 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6805/1051778383", 2024-09-16T00:09:22.438 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:22.438 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:22.438 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:22.438 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.439 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 6, 2024-09-16T00:09:22.439 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "9b6b33cc-d6d9-4a0a-8143-526073a58b93", 2024-09-16T00:09:22.439 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:22.439 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:22.439 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:22.439 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:22.439 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:22.439 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:22.439 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:09:22.439 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:09:22.439 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:22.440 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:22.440 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:22.440 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.440 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.440 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6824", 2024-09-16T00:09:22.440 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:22.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.440 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.440 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6825", 2024-09-16T00:09:22.440 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:22.440 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.440 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.441 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:22.441 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.441 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.441 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6826", 2024-09-16T00:09:22.441 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:22.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.441 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.441 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6827", 2024-09-16T00:09:22.441 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:22.441 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.442 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.442 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:22.442 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.442 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.442 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6830", 2024-09-16T00:09:22.442 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:22.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.442 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.442 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6831", 2024-09-16T00:09:22.442 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:22.442 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.443 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.443 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:22.443 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.443 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.443 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6828", 2024-09-16T00:09:22.443 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:22.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.443 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.443 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6829", 2024-09-16T00:09:22.443 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:22.443 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.444 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.444 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6825/1908495807", 2024-09-16T00:09:22.444 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6827/1908495807", 2024-09-16T00:09:22.444 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6831/1908495807", 2024-09-16T00:09:22.444 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6829/1908495807", 2024-09-16T00:09:22.444 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:22.444 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:22.444 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:22.444 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.444 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 7, 2024-09-16T00:09:22.445 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "50bcd808-3d3e-4a72-88e4-30decb53cfea", 2024-09-16T00:09:22.445 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:22.445 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:22.445 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:22.445 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:22.445 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:22.445 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:22.445 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:09:22.445 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 62, 2024-09-16T00:09:22.445 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:22.445 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:22.445 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:22.445 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.446 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.446 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6808", 2024-09-16T00:09:22.446 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:22.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.446 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.446 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6809", 2024-09-16T00:09:22.446 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:22.446 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.446 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.446 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:22.446 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.447 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.447 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6810", 2024-09-16T00:09:22.447 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:22.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.447 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.447 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6811", 2024-09-16T00:09:22.447 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:22.447 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.447 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.448 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:22.448 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.448 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.448 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6814", 2024-09-16T00:09:22.448 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:22.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.448 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.448 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6815", 2024-09-16T00:09:22.448 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:22.448 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.448 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.449 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:22.449 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:22.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.449 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:22.449 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6812", 2024-09-16T00:09:22.449 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:22.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.449 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:22.449 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6813", 2024-09-16T00:09:22.449 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:22.449 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.449 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.450 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6809/1158838808", 2024-09-16T00:09:22.450 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6811/1158838808", 2024-09-16T00:09:22.450 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6815/1158838808", 2024-09-16T00:09:22.450 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6813/1158838808", 2024-09-16T00:09:22.450 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:22.450 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:22.450 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:22.450 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.450 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.450 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:22.450 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_xinfo": [ 2024-09-16T00:09:22.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.450 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 0, 2024-09-16T00:09:22.450 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:22.451 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:22.451 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:22.451 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:22.451 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:22.451 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.843117+0000", 2024-09-16T00:09:22.451 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:22.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.451 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 1, 2024-09-16T00:09:22.451 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:22.451 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:22.451 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:22.451 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:22.452 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:22.452 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.622789+0000", 2024-09-16T00:09:22.452 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:22.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.452 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 2, 2024-09-16T00:09:22.452 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:22.452 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:22.452 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:22.452 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:22.452 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:22.452 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:17.919457+0000", 2024-09-16T00:09:22.452 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:22.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.453 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 3, 2024-09-16T00:09:22.453 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:22.453 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:22.453 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:22.453 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:22.453 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:22.453 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:19.330341+0000", 2024-09-16T00:09:22.453 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:22.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.453 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 4, 2024-09-16T00:09:22.453 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:22.453 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:22.454 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:22.454 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:22.454 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:22.454 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.815640+0000", 2024-09-16T00:09:22.454 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:22.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.454 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 5, 2024-09-16T00:09:22.454 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:22.454 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:22.454 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:22.454 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:22.454 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:22.455 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.083046+0000", 2024-09-16T00:09:22.455 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:22.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.455 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 6, 2024-09-16T00:09:22.455 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:22.455 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:22.455 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:22.455 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:22.455 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:22.455 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.551499+0000", 2024-09-16T00:09:22.455 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:22.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.456 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 7, 2024-09-16T00:09:22.456 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:22.456 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:22.456 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:22.456 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:22.456 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:22.456 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.377179+0000", 2024-09-16T00:09:22.456 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:22.456 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.456 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:22.456 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap": [ 2024-09-16T00:09:22.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.457 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.2d", 2024-09-16T00:09:22.457 INFO:teuthology.orchestra.run.smithi139.stdout: "osds": [ 2024-09-16T00:09:22.457 INFO:teuthology.orchestra.run.smithi139.stdout: 7, 2024-09-16T00:09:22.457 INFO:teuthology.orchestra.run.smithi139.stdout: 5 2024-09-16T00:09:22.457 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.457 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.457 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:22.457 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap_items": [ 2024-09-16T00:09:22.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.458 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.1f", 2024-09-16T00:09:22.458 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:22.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.458 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 0, 2024-09-16T00:09:22.458 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:09:22.458 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.458 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.458 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.21", 2024-09-16T00:09:22.458 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:22.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.458 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 5, 2024-09-16T00:09:22.458 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 1 2024-09-16T00:09:22.459 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.459 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.459 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.22", 2024-09-16T00:09:22.459 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:22.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.459 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 5, 2024-09-16T00:09:22.459 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:09:22.459 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.459 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.459 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.38", 2024-09-16T00:09:22.460 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:22.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.460 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 3, 2024-09-16T00:09:22.460 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:09:22.460 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.460 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.460 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.3a", 2024-09-16T00:09:22.460 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:22.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.460 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 5, 2024-09-16T00:09:22.460 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:09:22.461 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.461 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.461 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.3b", 2024-09-16T00:09:22.461 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:22.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:22.461 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 7, 2024-09-16T00:09:22.461 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 1 2024-09-16T00:09:22.461 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.461 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:22.461 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.461 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:22.461 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap_primaries": [], 2024-09-16T00:09:22.462 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_temp": [], 2024-09-16T00:09:22.462 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_temp": [], 2024-09-16T00:09:22.462 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklist": {}, 2024-09-16T00:09:22.462 INFO:teuthology.orchestra.run.smithi139.stdout: "range_blocklist": {}, 2024-09-16T00:09:22.462 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profiles": { 2024-09-16T00:09:22.462 INFO:teuthology.orchestra.run.smithi139.stdout: "default": { 2024-09-16T00:09:22.462 INFO:teuthology.orchestra.run.smithi139.stdout: "crush-failure-domain": "osd", 2024-09-16T00:09:22.462 INFO:teuthology.orchestra.run.smithi139.stdout: "k": "2", 2024-09-16T00:09:22.462 INFO:teuthology.orchestra.run.smithi139.stdout: "m": "1", 2024-09-16T00:09:22.462 INFO:teuthology.orchestra.run.smithi139.stdout: "plugin": "jerasure", 2024-09-16T00:09:22.462 INFO:teuthology.orchestra.run.smithi139.stdout: "technique": "reed_sol_van" 2024-09-16T00:09:22.462 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:22.462 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps_queue": [], 2024-09-16T00:09:22.462 INFO:teuthology.orchestra.run.smithi139.stdout: "new_removed_snaps": [], 2024-09-16T00:09:22.463 INFO:teuthology.orchestra.run.smithi139.stdout: "new_purged_snaps": [], 2024-09-16T00:09:22.463 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_node_flags": {}, 2024-09-16T00:09:22.463 INFO:teuthology.orchestra.run.smithi139.stdout: "device_class_flags": {}, 2024-09-16T00:09:22.463 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode": { 2024-09-16T00:09:22.463 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode_enabled": false, 2024-09-16T00:09:22.463 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_bucket_count": 0, 2024-09-16T00:09:22.463 INFO:teuthology.orchestra.run.smithi139.stdout: "degraded_stretch_mode": 0, 2024-09-16T00:09:22.463 INFO:teuthology.orchestra.run.smithi139.stdout: "recovering_stretch_mode": 0, 2024-09-16T00:09:22.463 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode_bucket": 0 2024-09-16T00:09:22.463 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:22.463 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:22.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.382+0000 7f010bffd640 1 -- 172.21.15.139:0/595203882 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f00e806c2b0 msgr2=0x7f00e806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.382+0000 7f010bffd640 1 --2- 172.21.15.139:0/595203882 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f00e806c2b0 0x7f00e806e770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0104159dc0 tx=0x7f00f40090e0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.382+0000 7f010bffd640 1 -- 172.21.15.139:0/595203882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f010406bb20 msgr2=0x7f0104159360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.382+0000 7f010bffd640 1 --2- 172.21.15.139:0/595203882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f010406bb20 0x7f0104159360 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0100000c00 tx=0x7f01000046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.382+0000 7f010bffd640 1 -- 172.21.15.139:0/595203882 shutdown_connections 2024-09-16T00:09:22.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.382+0000 7f010bffd640 1 --2- 172.21.15.139:0/595203882 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f00e806c2b0 0x7f00e806e770 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.382+0000 7f010bffd640 1 --2- 172.21.15.139:0/595203882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01041548c0 0x7f01041f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.382+0000 7f010bffd640 1 --2- 172.21.15.139:0/595203882 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01040c8a30 0x7f010414ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.382+0000 7f010bffd640 1 --2- 172.21.15.139:0/595203882 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f010406bb20 0x7f0104159360 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.382+0000 7f010bffd640 1 -- 172.21.15.139:0/595203882 >> 172.21.15.139:0/595203882 conn(0x7f0104067950 msgr2=0x7f010406eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:22.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.382+0000 7f010bffd640 1 -- 172.21.15.139:0/595203882 shutdown_connections 2024-09-16T00:09:22.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.382+0000 7f010bffd640 1 -- 172.21.15.139:0/595203882 wait complete. 2024-09-16T00:09:22.467 INFO:tasks.thrashosds.thrasher:j is {'epoch': 74, 'fsid': '2d3366e5-6a59-4b59-93b2-615e72ac7015', 'created': '2024-09-16T00:07:09.967833+0000', 'modified': '2024-09-16T00:09:21.103705+0000', 'last_up_change': '2024-09-16T00:07:21.049678+0000', 'last_in_change': '2024-09-16T00:07:12.025620+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:07:22.056303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T00:07:30.768649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T00:07:32.008209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '83699a05-4937-43c3-aefc-b669bed9bb6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6816', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6817', 'nonce': 1974901438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6818', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6819', 'nonce': 1974901438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6822', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6823', 'nonce': 1974901438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6820', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6821', 'nonce': 1974901438}]}, 'public_addr': '172.21.15.139:6817/1974901438', 'cluster_addr': '172.21.15.139:6819/1974901438', 'heartbeat_back_addr': '172.21.15.139:6823/1974901438', 'heartbeat_front_addr': '172.21.15.139:6821/1974901438', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9c04991e-9695-4bbb-8011-bf613bf404b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6808', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6809', 'nonce': 2232380998}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6810', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6811', 'nonce': 2232380998}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6814', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6815', 'nonce': 2232380998}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6812', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6813', 'nonce': 2232380998}]}, 'public_addr': '172.21.15.139:6809/2232380998', 'cluster_addr': '172.21.15.139:6811/2232380998', 'heartbeat_back_addr': '172.21.15.139:6815/2232380998', 'heartbeat_front_addr': '172.21.15.139:6813/2232380998', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8048fc64-72b3-4866-ac62-6a8b2bf6f5c8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 68, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6800', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6801', 'nonce': 4180588905}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6802', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6803', 'nonce': 4180588905}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6806', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6807', 'nonce': 4180588905}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6804', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6805', 'nonce': 4180588905}]}, 'public_addr': '172.21.15.139:6801/4180588905', 'cluster_addr': '172.21.15.139:6803/4180588905', 'heartbeat_back_addr': '172.21.15.139:6807/4180588905', 'heartbeat_front_addr': '172.21.15.139:6805/4180588905', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c34ac77b-354c-4884-902d-3e2a84f5eb27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6826', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6827', 'nonce': 3947768492}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6828', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6829', 'nonce': 3947768492}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6832', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6833', 'nonce': 3947768492}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6830', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6831', 'nonce': 3947768492}]}, 'public_addr': '172.21.15.139:6827/3947768492', 'cluster_addr': '172.21.15.139:6829/3947768492', 'heartbeat_back_addr': '172.21.15.139:6833/3947768492', 'heartbeat_front_addr': '172.21.15.139:6831/3947768492', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ba9a7c93-9672-43e1-8c98-2878cdf7f548', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6816', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6817', 'nonce': 642632916}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6818', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 642632916}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 642632916}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6821', 'nonce': 642632916}]}, 'public_addr': '172.21.15.164:6817/642632916', 'cluster_addr': '172.21.15.164:6819/642632916', 'heartbeat_back_addr': '172.21.15.164:6823/642632916', 'heartbeat_front_addr': '172.21.15.164:6821/642632916', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '46a0916b-474c-411b-84ad-86f27d4cbf53', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 1051778383}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 1051778383}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 1051778383}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 1051778383}]}, 'public_addr': '172.21.15.164:6801/1051778383', 'cluster_addr': '172.21.15.164:6803/1051778383', 'heartbeat_back_addr': '172.21.15.164:6807/1051778383', 'heartbeat_front_addr': '172.21.15.164:6805/1051778383', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b6b33cc-d6d9-4a0a-8143-526073a58b93', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6824', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 1908495807}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6827', 'nonce': 1908495807}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6830', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 1908495807}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 1908495807}]}, 'public_addr': '172.21.15.164:6825/1908495807', 'cluster_addr': '172.21.15.164:6827/1908495807', 'heartbeat_back_addr': '172.21.15.164:6831/1908495807', 'heartbeat_front_addr': '172.21.15.164:6829/1908495807', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '50bcd808-3d3e-4a72-88e4-30decb53cfea', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6809', 'nonce': 1158838808}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 1158838808}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6815', 'nonce': 1158838808}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6812', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 1158838808}]}, 'public_addr': '172.21.15.164:6809/1158838808', 'cluster_addr': '172.21.15.164:6811/1158838808', 'heartbeat_back_addr': '172.21.15.164:6815/1158838808', 'heartbeat_front_addr': '172.21.15.164:6813/1158838808', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.843117+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.622789+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:17.919457+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:19.330341+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.815640+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.083046+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.551499+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.377179+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2d', 'osds': [7, 5]}], 'pg_upmap_items': [{'pgid': '3.1f', 'mappings': [{'from': 0, 'to': 2}]}, {'pgid': '3.21', 'mappings': [{'from': 5, 'to': 1}]}, {'pgid': '3.22', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '3.38', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '3.3a', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '3.3b', 'mappings': [{'from': 7, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T00:09:22.467 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:09:22.468 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:22.468 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:22.469 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:09:22.469 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:22.464+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:22.566 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:22.566+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:22.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.608+0000 7f87679a1640 1 Processor -- start 2024-09-16T00:09:22.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.609+0000 7f87679a1640 1 -- start start 2024-09-16T00:09:22.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.609+0000 7f87679a1640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f87600c67e0 0x7f87600c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.609+0000 7f87679a1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f876006c110 0x7f876015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.609+0000 7f87679a1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f876006ca30 con 0x7f876006c560 2024-09-16T00:09:22.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.609+0000 7f87679a1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f876006cd10 con 0x7f876006c110 2024-09-16T00:09:22.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.609+0000 7f87679a1640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f87600c6bc0 con 0x7f87600c67e0 2024-09-16T00:09:22.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.610+0000 7f876573d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f87600c67e0 0x7f87600c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.610+0000 7f8765f3e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f876006c560 0x7f87600cbe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41668/0 (socket says 172.21.15.139:41668) 2024-09-16T00:09:22.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.610+0000 7f8765f3e640 1 -- 172.21.15.139:0/1545972120 learned_addr learned my addr 172.21.15.139:0/1545972120 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:22.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.610+0000 7f8764f3c640 1 --2- 172.21.15.139:0/1545972120 >> v2:172.21.15.139:3300/0 conn(0x7f876006c110 0x7f876015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.610+0000 7f876573d640 1 -- 172.21.15.139:0/1545972120 >> v2:172.21.15.139:3300/0 conn(0x7f876006c110 msgr2=0x7f876015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:22.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.610+0000 7f876573d640 1 --2- 172.21.15.139:0/1545972120 >> v2:172.21.15.139:3300/0 conn(0x7f876006c110 0x7f876015a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.610+0000 7f876573d640 1 -- 172.21.15.139:0/1545972120 >> v1:172.21.15.139:6789/0 conn(0x7f876006c560 legacy=0x7f87600cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.610+0000 7f876573d640 1 -- 172.21.15.139:0/1545972120 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876015df20 con 0x7f87600c67e0 2024-09-16T00:09:22.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.611+0000 7f876573d640 1 --2- 172.21.15.139:0/1545972120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87600c67e0 0x7f87600c7b90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f876008ded0 tx=0x7f875c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d5be175e0f0eaa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.611+0000 7f8757fff640 1 -- 172.21.15.139:0/1545972120 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c037070 con 0x7f87600c67e0 2024-09-16T00:09:22.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.611+0000 7f8757fff640 1 -- 172.21.15.139:0/1545972120 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f875c032440 con 0x7f87600c67e0 2024-09-16T00:09:22.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.611+0000 7f8757fff640 1 -- 172.21.15.139:0/1545972120 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c032810 con 0x7f87600c67e0 2024-09-16T00:09:22.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.611+0000 7f87679a1640 1 -- 172.21.15.139:0/1545972120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87600c67e0 msgr2=0x7f87600c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.611+0000 7f87679a1640 1 --2- 172.21.15.139:0/1545972120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87600c67e0 0x7f87600c7b90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f876008ded0 tx=0x7f875c02a230 comp rx=0 tx=0).stop 2024-09-16T00:09:22.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.612+0000 7f87679a1640 1 -- 172.21.15.139:0/1545972120 shutdown_connections 2024-09-16T00:09:22.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.612+0000 7f87679a1640 1 --2- 172.21.15.139:0/1545972120 >> v2:172.21.15.139:3300/0 conn(0x7f876006c110 0x7f876015a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.612+0000 7f87679a1640 1 --2- 172.21.15.139:0/1545972120 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87600c67e0 0x7f87600c7b90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.612+0000 7f87679a1640 1 -- 172.21.15.139:0/1545972120 >> 172.21.15.139:0/1545972120 conn(0x7f87600d05f0 msgr2=0x7f87600d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:22.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.612+0000 7f87679a1640 1 -- 172.21.15.139:0/1545972120 shutdown_connections 2024-09-16T00:09:22.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.612+0000 7f87679a1640 1 -- 172.21.15.139:0/1545972120 wait complete. 2024-09-16T00:09:22.615 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:22.615 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:22.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.613+0000 7f87679a1640 1 Processor -- start 2024-09-16T00:09:22.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.613+0000 7f87679a1640 1 -- start start 2024-09-16T00:09:22.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f87679a1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f876006c110 0x7f87601f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f87679a1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f876006c560 0x7f876015af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f87679a1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87600c67e0 0x7f87601f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f87679a1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f876015e4e0 con 0x7f876006c560 2024-09-16T00:09:22.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f87679a1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f876015e850 con 0x7f87600c67e0 2024-09-16T00:09:22.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f87679a1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f87600cc560 con 0x7f876006c110 2024-09-16T00:09:22.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f8764f3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f876006c560 0x7f876015af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f8764f3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f876006c560 0x7f876015af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54060/0 (socket says 172.21.15.139:54060) 2024-09-16T00:09:22.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f8765f3e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87600c67e0 0x7f87601f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f8764f3c640 1 -- 172.21.15.139:0/555476848 learned_addr learned my addr 172.21.15.139:0/555476848 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:22.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f876573d640 1 --2- 172.21.15.139:0/555476848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f876006c110 0x7f87601f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f8764f3c640 1 -- 172.21.15.139:0/555476848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f876006c110 msgr2=0x7f87601f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f8764f3c640 1 --2- 172.21.15.139:0/555476848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f876006c110 0x7f87601f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f8764f3c640 1 -- 172.21.15.139:0/555476848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87600c67e0 msgr2=0x7f87601f0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f8764f3c640 1 --2- 172.21.15.139:0/555476848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87600c67e0 0x7f87601f0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f8764f3c640 1 -- 172.21.15.139:0/555476848 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87601f4c10 con 0x7f876006c560 2024-09-16T00:09:22.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.614+0000 7f8764f3c640 1 --2- 172.21.15.139:0/555476848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f876006c560 0x7f876015af40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f875000e9b0 tx=0x7f875000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.615+0000 7f8755ffb640 1 -- 172.21.15.139:0/555476848 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750017070 con 0x7f876006c560 2024-09-16T00:09:22.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.615+0000 7f8755ffb640 1 -- 172.21.15.139:0/555476848 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f875000c9a0 con 0x7f876006c560 2024-09-16T00:09:22.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.615+0000 7f8755ffb640 1 -- 172.21.15.139:0/555476848 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875000cd90 con 0x7f876006c560 2024-09-16T00:09:22.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.615+0000 7f87679a1640 1 -- 172.21.15.139:0/555476848 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876015cba0 con 0x7f876006c560 2024-09-16T00:09:22.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.615+0000 7f87679a1640 1 -- 172.21.15.139:0/555476848 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f876015d160 con 0x7f876006c560 2024-09-16T00:09:22.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.616+0000 7f87679a1640 1 -- 172.21.15.139:0/555476848 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87600c7b90 con 0x7f876006c560 2024-09-16T00:09:22.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.616+0000 7f8755ffb640 1 -- 172.21.15.139:0/555476848 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f875001a070 con 0x7f876006c560 2024-09-16T00:09:22.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.617+0000 7f8755ffb640 1 --2- 172.21.15.139:0/555476848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f874406c2b0 0x7f874406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:22.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.617+0000 7f8755ffb640 1 -- 172.21.15.139:0/555476848 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5944+0+0 (secure 0 0 0) 0x7f875008b770 con 0x7f876006c560 2024-09-16T00:09:22.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.617+0000 7f876573d640 1 --2- 172.21.15.139:0/555476848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f874406c2b0 0x7f874406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:22.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.618+0000 7f876573d640 1 --2- 172.21.15.139:0/555476848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f874406c2b0 0x7f874406e770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f875c034000 tx=0x7f875c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:22.623 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:09:22.623 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.622+0000 7f8755ffb640 1 -- 172.21.15.139:0/555476848 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8750059830 con 0x7f876006c560 2024-09-16T00:09:22.667 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:22.666+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:22.767 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:22.767+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:22.779 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:22.779 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:22.789 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:09:22.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.824+0000 7f87679a1640 1 -- 172.21.15.139:0/555476848 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f87600cbc30 con 0x7f874406c2b0 2024-09-16T00:09:22.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.831+0000 7f8755ffb640 1 -- 172.21.15.139:0/555476848 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254099 (secure 0 0 0) 0x7f87600cbc30 con 0x7f874406c2b0 2024-09-16T00:09:22.835 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:09:22.836 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:09:22.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.835+0000 7f87679a1640 1 -- 172.21.15.139:0/555476848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f874406c2b0 msgr2=0x7f874406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.835+0000 7f87679a1640 1 --2- 172.21.15.139:0/555476848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f874406c2b0 0x7f874406e770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f875c034000 tx=0x7f875c02e000 comp rx=0 tx=0).stop 2024-09-16T00:09:22.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.835+0000 7f87679a1640 1 -- 172.21.15.139:0/555476848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f876006c560 msgr2=0x7f876015af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:22.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.835+0000 7f87679a1640 1 --2- 172.21.15.139:0/555476848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f876006c560 0x7f876015af40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f875000e9b0 tx=0x7f875000ee70 comp rx=0 tx=0).stop 2024-09-16T00:09:22.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.835+0000 7f87679a1640 1 -- 172.21.15.139:0/555476848 shutdown_connections 2024-09-16T00:09:22.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.835+0000 7f87679a1640 1 --2- 172.21.15.139:0/555476848 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f874406c2b0 0x7f874406e770 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.835+0000 7f87679a1640 1 --2- 172.21.15.139:0/555476848 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87600c67e0 0x7f87601f0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.835+0000 7f87679a1640 1 --2- 172.21.15.139:0/555476848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f876006c560 0x7f876015af40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.835+0000 7f87679a1640 1 --2- 172.21.15.139:0/555476848 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f876006c110 0x7f87601f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:22.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.835+0000 7f87679a1640 1 -- 172.21.15.139:0/555476848 >> 172.21.15.139:0/555476848 conn(0x7f87600d05f0 msgr2=0x7f87600c3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:22.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.835+0000 7f87679a1640 1 -- 172.21.15.139:0/555476848 shutdown_connections 2024-09-16T00:09:22.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:22.835+0000 7f87679a1640 1 -- 172.21.15.139:0/555476848 wait complete. 2024-09-16T00:09:22.849 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-16T00:09:22.044216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":65148657,"num_objects":6596,"num_object_clones":0,"num_object_copies":13192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6596,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":7103,"num_write_kb":70442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":397258,"num_keys_recovered":4,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6965,"ondisk_log_size":6965,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":195,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379436,"kb_used_data":104948,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754595284,"statfs":{"total":773094113280,"available":772705570816,"internally_reserved":0,"allocated":107466752,"data_stored":132903816,"data_compressed":15573650,"data_compressed_allocated":18968576,"data_compressed_original":78938112,"omap_allocated":12752,"internal_metadata":280874544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5389156,"num_objects":399,"num_object_clones":0,"num_object_copies":798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":457,"num_write_kb":5691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":280651,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000549"},"pg_stats":[{"pgid":"3.3f","version":"69'100","reported_seq":215,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085352+0000","last_change":"2024-09-16T00:08:36.888570+0000","last_active":"2024-09-16T00:09:16.085352+0000","last_peered":"2024-09-16T00:09:16.085352+0000","last_clean":"2024-09-16T00:09:16.085352+0000","last_became_active":"2024-09-16T00:08:36.888143+0000","last_became_peered":"2024-09-16T00:08:36.888143+0000","last_unstale":"2024-09-16T00:09:16.085352+0000","last_undegraded":"2024-09-16T00:09:16.085352+0000","last_fullsized":"2024-09-16T00:09:16.085352+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:30.579681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":275759,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"72'101","reported_seq":214,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101588+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:09:20.101588+0000","last_peered":"2024-09-16T00:09:20.101588+0000","last_clean":"2024-09-16T00:09:20.101588+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:09:20.101588+0000","last_undegraded":"2024-09-16T00:09:20.101588+0000","last_fullsized":"2024-09-16T00:09:20.101588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":338694,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"73'107","reported_seq":220,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.691078+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:09:20.691078+0000","last_peered":"2024-09-16T00:09:20.691078+0000","last_clean":"2024-09-16T00:09:20.691078+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:09:20.691078+0000","last_undegraded":"2024-09-16T00:09:20.691078+0000","last_fullsized":"2024-09-16T00:09:20.691078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":613745,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107,"num_write_kb":660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"69'80","reported_seq":186,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078402+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:09:16.078402+0000","last_peered":"2024-09-16T00:09:16.078402+0000","last_clean":"2024-09-16T00:09:16.078402+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:09:16.078402+0000","last_undegraded":"2024-09-16T00:09:16.078402+0000","last_fullsized":"2024-09-16T00:09:16.078402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":321366,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"72'119","reported_seq":207,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101405+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:09:20.101405+0000","last_peered":"2024-09-16T00:09:20.101405+0000","last_clean":"2024-09-16T00:09:20.101405+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:09:20.101405+0000","last_undegraded":"2024-09-16T00:09:20.101405+0000","last_fullsized":"2024-09-16T00:09:20.101405+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1698370,"num_objects":119,"num_object_clones":0,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":1722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"72'105","reported_seq":217,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.053613+0000","last_change":"2024-09-16T00:09:16.225647+0000","last_active":"2024-09-16T00:09:20.053613+0000","last_peered":"2024-09-16T00:09:20.053613+0000","last_clean":"2024-09-16T00:09:20.053613+0000","last_became_active":"2024-09-16T00:09:16.081926+0000","last_became_peered":"2024-09-16T00:09:16.081926+0000","last_unstale":"2024-09-16T00:09:20.053613+0000","last_undegraded":"2024-09-16T00:09:20.053613+0000","last_fullsized":"2024-09-16T00:09:20.053613+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:59.896840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":300521,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":280651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"69'108","reported_seq":213,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.084873+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:09:16.084873+0000","last_peered":"2024-09-16T00:09:16.084873+0000","last_clean":"2024-09-16T00:09:16.084873+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:09:16.084873+0000","last_undegraded":"2024-09-16T00:09:16.084873+0000","last_fullsized":"2024-09-16T00:09:16.084873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":944450,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"72'86","reported_seq":172,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.984615+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:09:19.984615+0000","last_peered":"2024-09-16T00:09:19.984615+0000","last_clean":"2024-09-16T00:09:19.984615+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:09:19.984615+0000","last_undegraded":"2024-09-16T00:09:19.984615+0000","last_fullsized":"2024-09-16T00:09:19.984615+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173073,"num_objects":86,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"69'101","reported_seq":206,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.087940+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:09:16.087940+0000","last_peered":"2024-09-16T00:09:16.087940+0000","last_clean":"2024-09-16T00:09:16.087940+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:09:16.087940+0000","last_undegraded":"2024-09-16T00:09:16.087940+0000","last_fullsized":"2024-09-16T00:09:16.087940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":401581,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101,"num_write_kb":445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"69'88","reported_seq":141,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.386306+0000","last_change":"2024-09-16T00:08:36.891868+0000","last_active":"2024-09-16T00:09:16.386306+0000","last_peered":"2024-09-16T00:09:16.386306+0000","last_clean":"2024-09-16T00:09:16.386306+0000","last_became_active":"2024-09-16T00:08:36.891605+0000","last_became_peered":"2024-09-16T00:08:36.891605+0000","last_unstale":"2024-09-16T00:09:16.386306+0000","last_undegraded":"2024-09-16T00:09:16.386306+0000","last_fullsized":"2024-09-16T00:09:16.386306+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:46:00.730436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":415370,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"69'95","reported_seq":148,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.100144+0000","last_change":"2024-09-16T00:08:36.891589+0000","last_active":"2024-09-16T00:09:16.100144+0000","last_peered":"2024-09-16T00:09:16.100144+0000","last_clean":"2024-09-16T00:09:16.100144+0000","last_became_active":"2024-09-16T00:08:36.891279+0000","last_became_peered":"2024-09-16T00:08:36.891279+0000","last_unstale":"2024-09-16T00:09:16.100144+0000","last_undegraded":"2024-09-16T00:09:16.100144+0000","last_fullsized":"2024-09-16T00:09:16.100144+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318764,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95,"num_write_kb":358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"68'85","reported_seq":200,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.079669+0000","last_change":"2024-09-16T00:08:36.888576+0000","last_active":"2024-09-16T00:09:16.079669+0000","last_peered":"2024-09-16T00:09:16.079669+0000","last_clean":"2024-09-16T00:09:16.079669+0000","last_became_active":"2024-09-16T00:08:36.888301+0000","last_became_peered":"2024-09-16T00:08:36.888301+0000","last_unstale":"2024-09-16T00:09:16.079669+0000","last_undegraded":"2024-09-16T00:09:16.079669+0000","last_fullsized":"2024-09-16T00:09:16.079669+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":391399,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85,"num_write_kb":427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"73'99","reported_seq":223,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.637793+0000","last_change":"2024-09-16T00:08:36.892595+0000","last_active":"2024-09-16T00:09:20.637793+0000","last_peered":"2024-09-16T00:09:20.637793+0000","last_clean":"2024-09-16T00:09:20.637793+0000","last_became_active":"2024-09-16T00:08:36.892316+0000","last_became_peered":"2024-09-16T00:08:36.892316+0000","last_unstale":"2024-09-16T00:09:20.637793+0000","last_undegraded":"2024-09-16T00:09:20.637793+0000","last_fullsized":"2024-09-16T00:09:20.637793+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:47.480168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":479457,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99,"num_write_kb":524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"73'103","reported_seq":216,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.107205+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:09:20.107205+0000","last_peered":"2024-09-16T00:09:20.107205+0000","last_clean":"2024-09-16T00:09:20.107205+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:09:20.107205+0000","last_undegraded":"2024-09-16T00:09:20.107205+0000","last_fullsized":"2024-09-16T00:09:20.107205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4866994,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102,"num_write_kb":4810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"73'109","reported_seq":222,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.352209+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:09:20.352209+0000","last_peered":"2024-09-16T00:09:20.352209+0000","last_clean":"2024-09-16T00:09:20.352209+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:09:20.352209+0000","last_undegraded":"2024-09-16T00:09:20.352209+0000","last_fullsized":"2024-09-16T00:09:20.352209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270259,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"69'113","reported_seq":219,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.290098+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:09:16.290098+0000","last_peered":"2024-09-16T00:09:16.290098+0000","last_clean":"2024-09-16T00:09:16.290098+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:09:16.290098+0000","last_undegraded":"2024-09-16T00:09:16.290098+0000","last_fullsized":"2024-09-16T00:09:16.290098+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356656,"num_objects":113,"num_object_clones":0,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"69'92","reported_seq":198,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.096996+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:09:16.096996+0000","last_peered":"2024-09-16T00:09:16.096996+0000","last_clean":"2024-09-16T00:09:16.096996+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:09:16.096996+0000","last_undegraded":"2024-09-16T00:09:16.096996+0000","last_fullsized":"2024-09-16T00:09:16.096996+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262585,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92,"num_write_kb":301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"72'94","reported_seq":226,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.104985+0000","last_change":"2024-09-16T00:08:36.894430+0000","last_active":"2024-09-16T00:09:19.104985+0000","last_peered":"2024-09-16T00:09:19.104985+0000","last_clean":"2024-09-16T00:09:19.104985+0000","last_became_active":"2024-09-16T00:08:36.894138+0000","last_became_peered":"2024-09-16T00:08:36.894138+0000","last_unstale":"2024-09-16T00:09:19.104985+0000","last_undegraded":"2024-09-16T00:09:19.104985+0000","last_fullsized":"2024-09-16T00:09:19.104985+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:19.478353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":321431,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94,"num_write_kb":365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"69'89","reported_seq":259,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085229+0000","last_change":"2024-09-16T00:09:04.210763+0000","last_active":"2024-09-16T00:09:16.085229+0000","last_peered":"2024-09-16T00:09:16.085229+0000","last_clean":"2024-09-16T00:09:16.085229+0000","last_became_active":"2024-09-16T00:09:04.158723+0000","last_became_peered":"2024-09-16T00:09:04.158723+0000","last_unstale":"2024-09-16T00:09:16.085229+0000","last_undegraded":"2024-09-16T00:09:16.085229+0000","last_fullsized":"2024-09-16T00:09:16.085229+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:25:48.973773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450120,"num_objects":89,"num_object_clones":0,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":89,"num_write_kb":483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":103497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"69'111","reported_seq":164,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.100385+0000","last_change":"2024-09-16T00:08:36.890676+0000","last_active":"2024-09-16T00:09:16.100385+0000","last_peered":"2024-09-16T00:09:16.100385+0000","last_clean":"2024-09-16T00:09:16.100385+0000","last_became_active":"2024-09-16T00:08:36.889743+0000","last_became_peered":"2024-09-16T00:08:36.889743+0000","last_unstale":"2024-09-16T00:09:16.100385+0000","last_undegraded":"2024-09-16T00:09:16.100385+0000","last_fullsized":"2024-09-16T00:09:16.100385+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:02.709496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363934,"num_objects":111,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111,"num_write_kb":417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"73'106","reported_seq":219,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.666709+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:09:20.666709+0000","last_peered":"2024-09-16T00:09:20.666709+0000","last_clean":"2024-09-16T00:09:20.666709+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:09:20.666709+0000","last_undegraded":"2024-09-16T00:09:20.666709+0000","last_fullsized":"2024-09-16T00:09:20.666709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":281224,"num_objects":106,"num_object_clones":0,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"69'105","reported_seq":211,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.352279+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:09:16.352279+0000","last_peered":"2024-09-16T00:09:16.352279+0000","last_clean":"2024-09-16T00:09:16.352279+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:09:16.352279+0000","last_undegraded":"2024-09-16T00:09:16.352279+0000","last_fullsized":"2024-09-16T00:09:16.352279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":391692,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"73'108","reported_seq":222,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.705526+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:09:20.705526+0000","last_peered":"2024-09-16T00:09:20.705526+0000","last_clean":"2024-09-16T00:09:20.705526+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:09:20.705526+0000","last_undegraded":"2024-09-16T00:09:20.705526+0000","last_fullsized":"2024-09-16T00:09:20.705526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":453356,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"68'99","reported_seq":205,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.077979+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:09:16.077979+0000","last_peered":"2024-09-16T00:09:16.077979+0000","last_clean":"2024-09-16T00:09:16.077979+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:09:16.077979+0000","last_undegraded":"2024-09-16T00:09:16.077979+0000","last_fullsized":"2024-09-16T00:09:16.077979+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":414019,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99,"num_write_kb":455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"73'113","reported_seq":227,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.677908+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:09:20.677908+0000","last_peered":"2024-09-16T00:09:20.677908+0000","last_clean":"2024-09-16T00:09:20.677908+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:09:20.677908+0000","last_undegraded":"2024-09-16T00:09:20.677908+0000","last_fullsized":"2024-09-16T00:09:20.677908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316565,"num_objects":113,"num_object_clones":0,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"69'115","reported_seq":220,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.478406+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:09:16.478406+0000","last_peered":"2024-09-16T00:09:16.478406+0000","last_clean":"2024-09-16T00:09:16.478406+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:09:16.478406+0000","last_undegraded":"2024-09-16T00:09:16.478406+0000","last_fullsized":"2024-09-16T00:09:16.478406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":745081,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"73'100","reported_seq":213,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101453+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:09:20.101453+0000","last_peered":"2024-09-16T00:09:20.101453+0000","last_clean":"2024-09-16T00:09:20.101453+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:09:20.101453+0000","last_undegraded":"2024-09-16T00:09:20.101453+0000","last_fullsized":"2024-09-16T00:09:20.101453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":346243,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"73'99","reported_seq":213,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119270+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:09:20.119270+0000","last_peered":"2024-09-16T00:09:20.119270+0000","last_clean":"2024-09-16T00:09:20.119270+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:09:20.119270+0000","last_undegraded":"2024-09-16T00:09:20.119270+0000","last_fullsized":"2024-09-16T00:09:20.119270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458275,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99,"num_write_kb":503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"73'103","reported_seq":216,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.107613+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:09:20.107613+0000","last_peered":"2024-09-16T00:09:20.107613+0000","last_clean":"2024-09-16T00:09:20.107613+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:09:20.107613+0000","last_undegraded":"2024-09-16T00:09:20.107613+0000","last_fullsized":"2024-09-16T00:09:20.107613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316000,"num_objects":103,"num_object_clones":0,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103,"num_write_kb":360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"72'122","reported_seq":208,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.411641+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:09:19.411641+0000","last_peered":"2024-09-16T00:09:19.411641+0000","last_clean":"2024-09-16T00:09:19.411641+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:09:19.411641+0000","last_undegraded":"2024-09-16T00:09:19.411641+0000","last_fullsized":"2024-09-16T00:09:19.411641+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":410431,"num_objects":122,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122,"num_write_kb":469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"72'112","reported_seq":229,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.806282+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:09:19.806282+0000","last_peered":"2024-09-16T00:09:19.806282+0000","last_clean":"2024-09-16T00:09:19.806282+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:09:19.806282+0000","last_undegraded":"2024-09-16T00:09:19.806282+0000","last_fullsized":"2024-09-16T00:09:19.806282+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":381579,"num_objects":112,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"69'105","reported_seq":158,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.372491+0000","last_change":"2024-09-16T00:08:36.894101+0000","last_active":"2024-09-16T00:09:16.372491+0000","last_peered":"2024-09-16T00:09:16.372491+0000","last_clean":"2024-09-16T00:09:16.372491+0000","last_became_active":"2024-09-16T00:08:36.893807+0000","last_became_peered":"2024-09-16T00:08:36.893807+0000","last_unstale":"2024-09-16T00:09:16.372491+0000","last_undegraded":"2024-09-16T00:09:16.372491+0000","last_fullsized":"2024-09-16T00:09:16.372491+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:40.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":958407,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"69'102","reported_seq":207,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.461457+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:09:16.461457+0000","last_peered":"2024-09-16T00:09:16.461457+0000","last_clean":"2024-09-16T00:09:16.461457+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:09:16.461457+0000","last_undegraded":"2024-09-16T00:09:16.461457+0000","last_fullsized":"2024-09-16T00:09:16.461457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":304220,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102,"num_write_kb":351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"63'62","reported_seq":178,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119631+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:09:20.119631+0000","last_peered":"2024-09-16T00:09:20.119631+0000","last_clean":"2024-09-16T00:09:20.119631+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:09:20.119631+0000","last_undegraded":"2024-09-16T00:09:20.119631+0000","last_fullsized":"2024-09-16T00:09:20.119631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62,"num_write_kb":4136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"73'94","reported_seq":207,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.488618+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:09:20.488618+0000","last_peered":"2024-09-16T00:09:20.488618+0000","last_clean":"2024-09-16T00:09:20.488618+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:09:20.488618+0000","last_undegraded":"2024-09-16T00:09:20.488618+0000","last_fullsized":"2024-09-16T00:09:20.488618+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":283620,"num_objects":94,"num_object_clones":0,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94,"num_write_kb":326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"67'39","reported_seq":91,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.099997+0000","last_change":"2024-09-16T00:08:36.890316+0000","last_active":"2024-09-16T00:09:16.099997+0000","last_peered":"2024-09-16T00:09:16.099997+0000","last_clean":"2024-09-16T00:09:16.099997+0000","last_became_active":"2024-09-16T00:08:36.889635+0000","last_became_peered":"2024-09-16T00:08:36.889635+0000","last_unstale":"2024-09-16T00:09:16.099997+0000","last_undegraded":"2024-09-16T00:09:16.099997+0000","last_fullsized":"2024-09-16T00:09:16.099997+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:28.843883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"63'1","reported_seq":127,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119169+0000","last_change":"2024-09-16T00:08:36.888349+0000","last_active":"2024-09-16T00:09:20.119169+0000","last_peered":"2024-09-16T00:09:20.119169+0000","last_clean":"2024-09-16T00:09:20.119169+0000","last_became_active":"2024-09-16T00:08:36.888052+0000","last_became_peered":"2024-09-16T00:08:36.888052+0000","last_unstale":"2024-09-16T00:09:20.119169+0000","last_undegraded":"2024-09-16T00:09:20.119169+0000","last_fullsized":"2024-09-16T00:09:20.119169+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:49.607254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"73'123","reported_seq":237,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.556875+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:09:20.556875+0000","last_peered":"2024-09-16T00:09:20.556875+0000","last_clean":"2024-09-16T00:09:20.556875+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:09:20.556875+0000","last_undegraded":"2024-09-16T00:09:20.556875+0000","last_fullsized":"2024-09-16T00:09:20.556875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":788879,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"66'2","reported_seq":110,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078017+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:09:16.078017+0000","last_peered":"2024-09-16T00:09:16.078017+0000","last_clean":"2024-09-16T00:09:16.078017+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:09:16.078017+0000","last_undegraded":"2024-09-16T00:09:16.078017+0000","last_fullsized":"2024-09-16T00:09:16.078017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"73'115","reported_seq":228,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.429522+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:09:20.429522+0000","last_peered":"2024-09-16T00:09:20.429522+0000","last_clean":"2024-09-16T00:09:20.429522+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:09:20.429522+0000","last_undegraded":"2024-09-16T00:09:20.429522+0000","last_fullsized":"2024-09-16T00:09:20.429522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":652806,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"61'1","reported_seq":108,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.100175+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:09:16.100175+0000","last_peered":"2024-09-16T00:09:16.100175+0000","last_clean":"2024-09-16T00:09:16.100175+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:09:16.100175+0000","last_undegraded":"2024-09-16T00:09:16.100175+0000","last_fullsized":"2024-09-16T00:09:16.100175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"69'106","reported_seq":212,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.407763+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:09:16.407763+0000","last_peered":"2024-09-16T00:09:16.407763+0000","last_clean":"2024-09-16T00:09:16.407763+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:09:16.407763+0000","last_undegraded":"2024-09-16T00:09:16.407763+0000","last_fullsized":"2024-09-16T00:09:16.407763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":465077,"num_objects":106,"num_object_clones":0,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":108,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078240+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:09:16.078240+0000","last_peered":"2024-09-16T00:09:16.078240+0000","last_clean":"2024-09-16T00:09:16.078240+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:09:16.078240+0000","last_undegraded":"2024-09-16T00:09:16.078240+0000","last_fullsized":"2024-09-16T00:09:16.078240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"73'107","reported_seq":220,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.579780+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:09:20.579780+0000","last_peered":"2024-09-16T00:09:20.579780+0000","last_clean":"2024-09-16T00:09:20.579780+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:09:20.579780+0000","last_undegraded":"2024-09-16T00:09:20.579780+0000","last_fullsized":"2024-09-16T00:09:20.579780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":864794,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107,"num_write_kb":903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"71'7","reported_seq":122,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101517+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:09:20.101517+0000","last_peered":"2024-09-16T00:09:20.101517+0000","last_clean":"2024-09-16T00:09:20.101517+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:09:20.101517+0000","last_undegraded":"2024-09-16T00:09:20.101517+0000","last_fullsized":"2024-09-16T00:09:20.101517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"69'88","reported_seq":193,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.088005+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:09:16.088005+0000","last_peered":"2024-09-16T00:09:16.088005+0000","last_clean":"2024-09-16T00:09:16.088005+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:09:16.088005+0000","last_undegraded":"2024-09-16T00:09:16.088005+0000","last_fullsized":"2024-09-16T00:09:16.088005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":420942,"num_objects":88,"num_object_clones":0,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88,"num_write_kb":455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"65'22","reported_seq":75,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.100272+0000","last_change":"2024-09-16T00:08:36.914308+0000","last_active":"2024-09-16T00:09:16.100272+0000","last_peered":"2024-09-16T00:09:16.100272+0000","last_clean":"2024-09-16T00:09:16.100272+0000","last_became_active":"2024-09-16T00:08:36.890227+0000","last_became_peered":"2024-09-16T00:08:36.890227+0000","last_unstale":"2024-09-16T00:09:16.100272+0000","last_undegraded":"2024-09-16T00:09:16.100272+0000","last_fullsized":"2024-09-16T00:09:16.100272+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:31.761195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13110,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"69'85","reported_seq":190,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.090928+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:09:16.090928+0000","last_peered":"2024-09-16T00:09:16.090928+0000","last_clean":"2024-09-16T00:09:16.090928+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:09:16.090928+0000","last_undegraded":"2024-09-16T00:09:16.090928+0000","last_fullsized":"2024-09-16T00:09:16.090928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327524,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85,"num_write_kb":367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"71'7","reported_seq":122,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.107651+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:09:20.107651+0000","last_peered":"2024-09-16T00:09:20.107651+0000","last_clean":"2024-09-16T00:09:20.107651+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:09:20.107651+0000","last_undegraded":"2024-09-16T00:09:20.107651+0000","last_fullsized":"2024-09-16T00:09:20.107651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"73'104","reported_seq":217,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.407378+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:09:20.407378+0000","last_peered":"2024-09-16T00:09:20.407378+0000","last_clean":"2024-09-16T00:09:20.407378+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:09:20.407378+0000","last_undegraded":"2024-09-16T00:09:20.407378+0000","last_fullsized":"2024-09-16T00:09:20.407378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":418435,"num_objects":104,"num_object_clones":0,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"61'1","reported_seq":109,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078207+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:09:16.078207+0000","last_peered":"2024-09-16T00:09:16.078207+0000","last_clean":"2024-09-16T00:09:16.078207+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:09:16.078207+0000","last_undegraded":"2024-09-16T00:09:16.078207+0000","last_fullsized":"2024-09-16T00:09:16.078207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"69'99","reported_seq":204,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.322192+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:09:16.322192+0000","last_peered":"2024-09-16T00:09:16.322192+0000","last_clean":"2024-09-16T00:09:16.322192+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:09:16.322192+0000","last_undegraded":"2024-09-16T00:09:16.322192+0000","last_fullsized":"2024-09-16T00:09:16.322192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2645634,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99,"num_write_kb":2639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"62'3","reported_seq":56,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.099989+0000","last_change":"2024-09-16T00:08:36.889720+0000","last_active":"2024-09-16T00:09:16.099989+0000","last_peered":"2024-09-16T00:09:16.099989+0000","last_clean":"2024-09-16T00:09:16.099989+0000","last_became_active":"2024-09-16T00:08:36.889345+0000","last_became_peered":"2024-09-16T00:08:36.889345+0000","last_unstale":"2024-09-16T00:09:16.099989+0000","last_undegraded":"2024-09-16T00:09:16.099989+0000","last_fullsized":"2024-09-16T00:09:16.099989+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:03.170073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"69'97","reported_seq":150,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.414321+0000","last_change":"2024-09-16T00:08:36.896787+0000","last_active":"2024-09-16T00:09:16.414321+0000","last_peered":"2024-09-16T00:09:16.414321+0000","last_clean":"2024-09-16T00:09:16.414321+0000","last_became_active":"2024-09-16T00:08:36.896505+0000","last_became_peered":"2024-09-16T00:08:36.896505+0000","last_unstale":"2024-09-16T00:09:16.414321+0000","last_undegraded":"2024-09-16T00:09:16.414321+0000","last_fullsized":"2024-09-16T00:09:16.414321+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:55.997551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4580108,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97,"num_write_kb":4521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"63'10","reported_seq":124,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.103086+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:09:19.103086+0000","last_peered":"2024-09-16T00:09:19.103086+0000","last_clean":"2024-09-16T00:09:19.103086+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:09:19.103086+0000","last_undegraded":"2024-09-16T00:09:19.103086+0000","last_fullsized":"2024-09-16T00:09:19.103086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"69'106","reported_seq":211,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.140958+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:09:16.140958+0000","last_peered":"2024-09-16T00:09:16.140958+0000","last_clean":"2024-09-16T00:09:16.140958+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:09:16.140958+0000","last_undegraded":"2024-09-16T00:09:16.140958+0000","last_fullsized":"2024-09-16T00:09:16.140958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":387466,"num_objects":106,"num_object_clones":0,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"61'2","reported_seq":109,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.079842+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:09:16.079842+0000","last_peered":"2024-09-16T00:09:16.079842+0000","last_clean":"2024-09-16T00:09:16.079842+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:09:16.079842+0000","last_undegraded":"2024-09-16T00:09:16.079842+0000","last_fullsized":"2024-09-16T00:09:16.079842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"69'85","reported_seq":190,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.363811+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:09:16.363811+0000","last_peered":"2024-09-16T00:09:16.363811+0000","last_clean":"2024-09-16T00:09:16.363811+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:09:16.363811+0000","last_undegraded":"2024-09-16T00:09:16.363811+0000","last_fullsized":"2024-09-16T00:09:16.363811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":338686,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85,"num_write_kb":369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"71'4","reported_seq":118,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.103202+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:09:19.103202+0000","last_peered":"2024-09-16T00:09:19.103202+0000","last_clean":"2024-09-16T00:09:19.103202+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:09:19.103202+0000","last_undegraded":"2024-09-16T00:09:19.103202+0000","last_fullsized":"2024-09-16T00:09:19.103202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"69'90","reported_seq":195,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085461+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:09:16.085461+0000","last_peered":"2024-09-16T00:09:16.085461+0000","last_clean":"2024-09-16T00:09:16.085461+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:09:16.085461+0000","last_undegraded":"2024-09-16T00:09:16.085461+0000","last_fullsized":"2024-09-16T00:09:16.085461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":259850,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90,"num_write_kb":297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"61'2","reported_seq":117,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101525+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:09:20.101525+0000","last_peered":"2024-09-16T00:09:20.101525+0000","last_clean":"2024-09-16T00:09:20.101525+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:09:20.101525+0000","last_undegraded":"2024-09-16T00:09:20.101525+0000","last_fullsized":"2024-09-16T00:09:20.101525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"69'105","reported_seq":211,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078544+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:09:16.078544+0000","last_peered":"2024-09-16T00:09:16.078544+0000","last_clean":"2024-09-16T00:09:16.078544+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:09:16.078544+0000","last_undegraded":"2024-09-16T00:09:16.078544+0000","last_fullsized":"2024-09-16T00:09:16.078544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":598227,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"66'1","reported_seq":115,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.103233+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:09:19.103233+0000","last_peered":"2024-09-16T00:09:19.103233+0000","last_clean":"2024-09-16T00:09:19.103233+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:09:19.103233+0000","last_undegraded":"2024-09-16T00:09:19.103233+0000","last_fullsized":"2024-09-16T00:09:19.103233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"69'106","reported_seq":221,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.383992+0000","last_change":"2024-09-16T00:08:47.953122+0000","last_active":"2024-09-16T00:09:16.383992+0000","last_peered":"2024-09-16T00:09:16.383992+0000","last_clean":"2024-09-16T00:09:16.383992+0000","last_became_active":"2024-09-16T00:08:47.952799+0000","last_became_peered":"2024-09-16T00:08:47.952799+0000","last_unstale":"2024-09-16T00:09:16.383992+0000","last_undegraded":"2024-09-16T00:09:16.383992+0000","last_fullsized":"2024-09-16T00:09:16.383992+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:03.447562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":381576,"num_objects":106,"num_object_clones":0,"num_object_copies":212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106,"num_write_kb":427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":263,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.084830+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:09:16.084830+0000","last_peered":"2024-09-16T00:09:16.084830+0000","last_clean":"2024-09-16T00:09:16.084830+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:09:16.084830+0000","last_undegraded":"2024-09-16T00:09:16.084830+0000","last_fullsized":"2024-09-16T00:09:16.084830+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"66'1","reported_seq":109,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078027+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:09:16.078027+0000","last_peered":"2024-09-16T00:09:16.078027+0000","last_clean":"2024-09-16T00:09:16.078027+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:09:16.078027+0000","last_undegraded":"2024-09-16T00:09:16.078027+0000","last_fullsized":"2024-09-16T00:09:16.078027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"73'100","reported_seq":214,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.688814+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:09:20.688814+0000","last_peered":"2024-09-16T00:09:20.688814+0000","last_clean":"2024-09-16T00:09:20.688814+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:09:20.688814+0000","last_undegraded":"2024-09-16T00:09:20.688814+0000","last_fullsized":"2024-09-16T00:09:20.688814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426614,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":100,"num_write_kb":466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"64'2","reported_seq":109,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.084756+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:09:16.084756+0000","last_peered":"2024-09-16T00:09:16.084756+0000","last_clean":"2024-09-16T00:09:16.084756+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:09:16.084756+0000","last_undegraded":"2024-09-16T00:09:16.084756+0000","last_fullsized":"2024-09-16T00:09:16.084756+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"73'97","reported_seq":210,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.534875+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:09:20.534875+0000","last_peered":"2024-09-16T00:09:20.534875+0000","last_clean":"2024-09-16T00:09:20.534875+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:09:20.534875+0000","last_undegraded":"2024-09-16T00:09:20.534875+0000","last_fullsized":"2024-09-16T00:09:20.534875+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":253833,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97,"num_write_kb":301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"71'1","reported_seq":115,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.103092+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:09:19.103092+0000","last_peered":"2024-09-16T00:09:19.103092+0000","last_clean":"2024-09-16T00:09:19.103092+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:09:19.103092+0000","last_undegraded":"2024-09-16T00:09:19.103092+0000","last_fullsized":"2024-09-16T00:09:19.103092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"73'115","reported_seq":228,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.511361+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:09:20.511361+0000","last_peered":"2024-09-16T00:09:20.511361+0000","last_clean":"2024-09-16T00:09:20.511361+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:09:20.511361+0000","last_undegraded":"2024-09-16T00:09:20.511361+0000","last_fullsized":"2024-09-16T00:09:20.511361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":504645,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"67'3","reported_seq":111,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078296+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:09:16.078296+0000","last_peered":"2024-09-16T00:09:16.078296+0000","last_clean":"2024-09-16T00:09:16.078296+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:09:16.078296+0000","last_undegraded":"2024-09-16T00:09:16.078296+0000","last_fullsized":"2024-09-16T00:09:16.078296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"69'97","reported_seq":202,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.367943+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:09:16.367943+0000","last_peered":"2024-09-16T00:09:16.367943+0000","last_clean":"2024-09-16T00:09:16.367943+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:09:16.367943+0000","last_undegraded":"2024-09-16T00:09:16.367943+0000","last_fullsized":"2024-09-16T00:09:16.367943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":563894,"num_objects":97,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97,"num_write_kb":603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"61'1","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101534+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:09:20.101534+0000","last_peered":"2024-09-16T00:09:20.101534+0000","last_clean":"2024-09-16T00:09:20.101534+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:09:20.101534+0000","last_undegraded":"2024-09-16T00:09:20.101534+0000","last_fullsized":"2024-09-16T00:09:20.101534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"69'92","reported_seq":198,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.301283+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:09:16.301283+0000","last_peered":"2024-09-16T00:09:16.301283+0000","last_clean":"2024-09-16T00:09:16.301283+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:09:16.301283+0000","last_undegraded":"2024-09-16T00:09:16.301283+0000","last_fullsized":"2024-09-16T00:09:16.301283+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":212579,"num_objects":92,"num_object_clones":0,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92,"num_write_kb":260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":107,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085249+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:09:16.085249+0000","last_peered":"2024-09-16T00:09:16.085249+0000","last_clean":"2024-09-16T00:09:16.085249+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:09:16.085249+0000","last_undegraded":"2024-09-16T00:09:16.085249+0000","last_fullsized":"2024-09-16T00:09:16.085249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'104","reported_seq":217,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.431593+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:09:20.431593+0000","last_peered":"2024-09-16T00:09:20.431593+0000","last_clean":"2024-09-16T00:09:20.431593+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:09:20.431593+0000","last_undegraded":"2024-09-16T00:09:20.431593+0000","last_fullsized":"2024-09-16T00:09:20.431593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":498070,"num_objects":104,"num_object_clones":0,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104,"num_write_kb":543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"73'32","reported_seq":165,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101645+0000","last_change":"2024-09-16T00:08:36.906253+0000","last_active":"2024-09-16T00:09:20.101645+0000","last_peered":"2024-09-16T00:09:20.101645+0000","last_clean":"2024-09-16T00:09:20.101645+0000","last_became_active":"2024-09-16T00:08:36.894363+0000","last_became_peered":"2024-09-16T00:08:36.894363+0000","last_unstale":"2024-09-16T00:09:20.101645+0000","last_undegraded":"2024-09-16T00:09:20.101645+0000","last_fullsized":"2024-09-16T00:09:20.101645+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:00.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":84,"num_write_kb":825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"73'108","reported_seq":222,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119533+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:09:20.119533+0000","last_peered":"2024-09-16T00:09:20.119533+0000","last_clean":"2024-09-16T00:09:20.119533+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:09:20.119533+0000","last_undegraded":"2024-09-16T00:09:20.119533+0000","last_fullsized":"2024-09-16T00:09:20.119533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":601448,"num_objects":108,"num_object_clones":0,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108,"num_write_kb":645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"61'3","reported_seq":110,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.100204+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:09:16.100204+0000","last_peered":"2024-09-16T00:09:16.100204+0000","last_clean":"2024-09-16T00:09:16.100204+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:09:16.100204+0000","last_undegraded":"2024-09-16T00:09:16.100204+0000","last_fullsized":"2024-09-16T00:09:16.100204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"72'109","reported_seq":222,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.107621+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:09:20.107621+0000","last_peered":"2024-09-16T00:09:20.107621+0000","last_clean":"2024-09-16T00:09:20.107621+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:09:20.107621+0000","last_undegraded":"2024-09-16T00:09:20.107621+0000","last_fullsized":"2024-09-16T00:09:20.107621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":298496,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":107,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.084727+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:09:16.084727+0000","last_peered":"2024-09-16T00:09:16.084727+0000","last_clean":"2024-09-16T00:09:16.084727+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:09:16.084727+0000","last_undegraded":"2024-09-16T00:09:16.084727+0000","last_fullsized":"2024-09-16T00:09:16.084727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"69'112","reported_seq":217,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.405822+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:09:16.405822+0000","last_peered":"2024-09-16T00:09:16.405822+0000","last_clean":"2024-09-16T00:09:16.405822+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:09:16.405822+0000","last_undegraded":"2024-09-16T00:09:16.405822+0000","last_fullsized":"2024-09-16T00:09:16.405822+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318386,"num_objects":112,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"61'1","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101307+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:09:20.101307+0000","last_peered":"2024-09-16T00:09:20.101307+0000","last_clean":"2024-09-16T00:09:20.101307+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:09:20.101307+0000","last_undegraded":"2024-09-16T00:09:20.101307+0000","last_fullsized":"2024-09-16T00:09:20.101307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"73'105","reported_seq":219,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.559467+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:09:20.559467+0000","last_peered":"2024-09-16T00:09:20.559467+0000","last_clean":"2024-09-16T00:09:20.559467+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:09:20.559467+0000","last_undegraded":"2024-09-16T00:09:20.559467+0000","last_fullsized":"2024-09-16T00:09:20.559467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335746,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":105,"num_write_kb":386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"63'2","reported_seq":117,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.101354+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:09:20.101354+0000","last_peered":"2024-09-16T00:09:20.101354+0000","last_clean":"2024-09-16T00:09:20.101354+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:09:20.101354+0000","last_undegraded":"2024-09-16T00:09:20.101354+0000","last_fullsized":"2024-09-16T00:09:20.101354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"73'109","reported_seq":223,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119590+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:09:20.119590+0000","last_peered":"2024-09-16T00:09:20.119590+0000","last_clean":"2024-09-16T00:09:20.119590+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:09:20.119590+0000","last_undegraded":"2024-09-16T00:09:20.119590+0000","last_fullsized":"2024-09-16T00:09:20.119590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":372221,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"64'6","reported_seq":113,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.084974+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:09:16.084974+0000","last_peered":"2024-09-16T00:09:16.084974+0000","last_clean":"2024-09-16T00:09:16.084974+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:09:16.084974+0000","last_undegraded":"2024-09-16T00:09:16.084974+0000","last_fullsized":"2024-09-16T00:09:16.084974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"69'95","reported_seq":148,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.125392+0000","last_change":"2024-09-16T00:08:36.890347+0000","last_active":"2024-09-16T00:09:16.125392+0000","last_peered":"2024-09-16T00:09:16.125392+0000","last_clean":"2024-09-16T00:09:16.125392+0000","last_became_active":"2024-09-16T00:08:36.890009+0000","last_became_peered":"2024-09-16T00:08:36.890009+0000","last_unstale":"2024-09-16T00:09:16.125392+0000","last_undegraded":"2024-09-16T00:09:16.125392+0000","last_fullsized":"2024-09-16T00:09:16.125392+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:24.298978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":353539,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95,"num_write_kb":393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"69'87","reported_seq":193,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.334327+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:09:16.334327+0000","last_peered":"2024-09-16T00:09:16.334327+0000","last_clean":"2024-09-16T00:09:16.334327+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:09:16.334327+0000","last_undegraded":"2024-09-16T00:09:16.334327+0000","last_fullsized":"2024-09-16T00:09:16.334327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240570,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":87,"num_write_kb":283,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"66'3","reported_seq":110,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085003+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:09:16.085003+0000","last_peered":"2024-09-16T00:09:16.085003+0000","last_clean":"2024-09-16T00:09:16.085003+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:09:16.085003+0000","last_undegraded":"2024-09-16T00:09:16.085003+0000","last_fullsized":"2024-09-16T00:09:16.085003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"69'95","reported_seq":201,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.078513+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:09:16.078513+0000","last_peered":"2024-09-16T00:09:16.078513+0000","last_clean":"2024-09-16T00:09:16.078513+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:09:16.078513+0000","last_undegraded":"2024-09-16T00:09:16.078513+0000","last_fullsized":"2024-09-16T00:09:16.078513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":479096,"num_objects":95,"num_object_clones":0,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":95,"num_write_kb":517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"66'6","reported_seq":131,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-16T00:09:16.085152+0000","last_change":"2024-09-16T00:08:36.900704+0000","last_active":"2024-09-16T00:09:16.085152+0000","last_peered":"2024-09-16T00:09:16.085152+0000","last_clean":"2024-09-16T00:09:16.085152+0000","last_became_active":"2024-09-16T00:08:36.892520+0000","last_became_peered":"2024-09-16T00:08:36.892520+0000","last_unstale":"2024-09-16T00:09:16.085152+0000","last_undegraded":"2024-09-16T00:09:16.085152+0000","last_fullsized":"2024-09-16T00:09:16.085152+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:30.056041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"72'110","reported_seq":196,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-16T00:09:19.570967+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:09:19.570967+0000","last_peered":"2024-09-16T00:09:19.570967+0000","last_clean":"2024-09-16T00:09:19.570967+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:09:19.570967+0000","last_undegraded":"2024-09-16T00:09:19.570967+0000","last_fullsized":"2024-09-16T00:09:19.570967+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878216,"num_objects":110,"num_object_clones":0,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110,"num_write_kb":920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"72'39","reported_seq":155,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.119425+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:09:20.119425+0000","last_peered":"2024-09-16T00:09:20.119425+0000","last_clean":"2024-09-16T00:09:20.119425+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:09:20.119425+0000","last_undegraded":"2024-09-16T00:09:20.119425+0000","last_fullsized":"2024-09-16T00:09:20.119425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":4141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"73'107","reported_seq":231,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.450104+0000","last_change":"2024-09-16T00:08:36.889082+0000","last_active":"2024-09-16T00:09:20.450104+0000","last_peered":"2024-09-16T00:09:20.450104+0000","last_clean":"2024-09-16T00:09:20.450104+0000","last_became_active":"2024-09-16T00:08:36.888807+0000","last_became_peered":"2024-09-16T00:08:36.888807+0000","last_unstale":"2024-09-16T00:09:20.450104+0000","last_undegraded":"2024-09-16T00:09:20.450104+0000","last_fullsized":"2024-09-16T00:09:20.450104+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:22.825324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":298398,"num_objects":107,"num_object_clones":0,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":107,"num_write_kb":346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"65'58","reported_seq":173,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-16T00:09:20.107353+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:09:20.107353+0000","last_peered":"2024-09-16T00:09:20.107353+0000","last_clean":"2024-09-16T00:09:20.107353+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:09:20.107353+0000","last_undegraded":"2024-09-16T00:09:20.107353+0000","last_fullsized":"2024-09-16T00:09:20.107353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":4138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":39390995,"num_objects":6525,"num_object_clones":0,"num_object_copies":13050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6525,"num_write_kb":41915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":384148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":90755072,"data_stored":78767502,"data_compressed":7812362,"data_compressed_allocated":9633792,"data_compressed_original":28786688,"omap_allocated":0,"internal_metadata":0},"log_size":6526,"ondisk_log_size":6526,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":25167294,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":419,"num_write_kb":26111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":13110,"num_keys_recovered":4,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10543104,"data_stored":50334588,"data_compressed":7770882,"data_compressed_allocated":9338880,"data_compressed_original":49184768,"omap_allocated":0,"internal_metadata":0},"log_size":322,"ondisk_log_size":322,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509467,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50176,"kb_used_data":14828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94321664,"statfs":{"total":96636764160,"available":96585383936,"internally_reserved":0,"allocated":15183872,"data_stored":23373544,"data_compressed":2945461,"data_compressed_allocated":3616768,"data_compressed_original":16404480,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509467,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43788,"kb_used_data":12592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328052,"statfs":{"total":96636764160,"available":96591925248,"internally_reserved":0,"allocated":12894208,"data_stored":18170022,"data_compressed":2007399,"data_compressed_allocated":2420736,"data_compressed_original":10899456,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509466,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45576,"kb_used_data":10224,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326264,"statfs":{"total":96636764160,"available":96590094336,"internally_reserved":0,"allocated":10469376,"data_stored":8320139,"data_compressed":1085058,"data_compressed_allocated":1335296,"data_compressed_original":3911680,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509467,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49168,"kb_used_data":13812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322672,"statfs":{"total":96636764160,"available":96586416128,"internally_reserved":0,"allocated":14143488,"data_stored":17711202,"data_compressed":2042984,"data_compressed_allocated":2514944,"data_compressed_original":10903552,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49076,"kb_used_data":13724,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322764,"statfs":{"total":96636764160,"available":96586510336,"internally_reserved":0,"allocated":14053376,"data_stored":21050186,"data_compressed":3148104,"data_compressed_allocated":3756032,"data_compressed_original":15941632,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47316,"kb_used_data":11968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94324524,"statfs":{"total":96636764160,"available":96588312576,"internally_reserved":0,"allocated":12255232,"data_stored":10931436,"data_compressed":929701,"data_compressed_allocated":1146880,"data_compressed_original":3465216,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56276,"kb_used_data":20924,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94315564,"statfs":{"total":96636764160,"available":96579137536,"internally_reserved":0,"allocated":21426176,"data_stored":26980158,"data_compressed":2504865,"data_compressed_allocated":3092480,"data_compressed_original":13815808,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38060,"kb_used_data":6876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333780,"statfs":{"total":96636764160,"available":96597790720,"internally_reserved":0,"allocated":7041024,"data_stored":6367129,"data_compressed":910078,"data_compressed_allocated":1085440,"data_compressed_original":3596288,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1748992,"data_stored":8389246,"data_compressed":1255393,"data_compressed_allocated":1544192,"data_compressed_original":8196096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2703360,"data_stored":12583584,"data_compressed":1983184,"data_compressed_allocated":2338816,"data_compressed_original":12226560,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1675264,"data_stored":8388734,"data_compressed":1280256,"data_compressed_allocated":1560576,"data_compressed_original":8286208,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1712128,"data_stored":8389280,"data_compressed":1327713,"data_compressed_allocated":1560576,"data_compressed_original":8245248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2691072,"data_stored":12583494,"data_compressed":1924336,"data_compressed_allocated":2334720,"data_compressed_original":12230656,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":18800640,"data_stored":17629859,"data_compressed":1217095,"data_compressed_allocated":1503232,"data_compressed_original":5046272,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11399168,"data_stored":10541937,"data_compressed":929701,"data_compressed_allocated":1146880,"data_compressed_original":3465216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6352896,"data_stored":6045784,"data_compressed":910078,"data_compressed_allocated":1085440,"data_compressed_original":3596288,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10821632,"data_stored":8372607,"data_compressed":1239268,"data_compressed_allocated":1511424,"data_compressed_original":3895296,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11669504,"data_stored":8957737,"data_compressed":762728,"data_compressed_allocated":954368,"data_compressed_original":2617344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9666560,"data_stored":7955318,"data_compressed":1085058,"data_compressed_allocated":1335296,"data_compressed_original":3911680,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":10383360,"data_stored":9416011,"data_compressed":679686,"data_compressed_allocated":860160,"data_compressed_original":2654208,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11661312,"data_stored":9848249,"data_compressed":988748,"data_compressed_allocated":1236992,"data_compressed_original":3600384,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:09:22.853 INFO:tasks.thrashosds.thrasher:Setting 3.c to [5, 3] 2024-09-16T00:09:22.853 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.c', '5', '3'] 2024-09-16T00:09:22.854 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.c 5 3 2024-09-16T00:09:22.868 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:22.867+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:22.951 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:22.951 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:22.962 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:09:23.005 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:23.005+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.017+0000 7fa1a7c17640 1 Processor -- start 2024-09-16T00:09:23.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.017+0000 7fa1a7c17640 1 -- start start 2024-09-16T00:09:23.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.018+0000 7fa1a7c17640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa1a00c6e00 0x7fa1a00ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:23.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.018+0000 7fa1a7c17640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a0158a70 con 0x7fa1a00c6a20 2024-09-16T00:09:23.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.018+0000 7fa1a7c17640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a006f5b0 con 0x7fa1a0158620 2024-09-16T00:09:23.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.018+0000 7fa1a7c17640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a00ca220 con 0x7fa1a00c6e00 2024-09-16T00:09:23.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.018+0000 7fa1a59b3640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa1a00c6a20 0x7fa1a00c7b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41678/0 (socket says 172.21.15.139:41678) 2024-09-16T00:09:23.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.018+0000 7fa1a59b3640 1 -- 172.21.15.139:0/806519981 learned_addr learned my addr 172.21.15.139:0/806519981 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:23.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.018+0000 7fa1a51b2640 1 --2- 172.21.15.139:0/806519981 >> v2:172.21.15.164:3300/0 conn(0x7fa1a00c6e00 0x7fa1a00ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:23.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.019+0000 7fa1a49b1640 1 -- 172.21.15.139:0/806519981 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1139975636 0 0) 0x7fa1a0158a70 con 0x7fa1a00c6a20 2024-09-16T00:09:23.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.019+0000 7fa1a49b1640 1 -- 172.21.15.139:0/806519981 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa18c003520 con 0x7fa1a00c6a20 2024-09-16T00:09:23.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.019+0000 7fa1a49b1640 1 -- 172.21.15.139:0/806519981 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 114619633 0 0) 0x7fa1a006f5b0 con 0x7fa1a0158620 2024-09-16T00:09:23.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.019+0000 7fa1a49b1640 1 -- 172.21.15.139:0/806519981 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1a0158a70 con 0x7fa1a0158620 2024-09-16T00:09:23.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.019+0000 7fa1a51b2640 1 -- 172.21.15.139:0/806519981 >> v1:172.21.15.164:6789/0 conn(0x7fa1a0158620 legacy=0x7fa1a00c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:23.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.019+0000 7fa1a51b2640 1 -- 172.21.15.139:0/806519981 >> v1:172.21.15.139:6789/0 conn(0x7fa1a00c6a20 legacy=0x7fa1a00c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:23.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.019+0000 7fa1a51b2640 1 -- 172.21.15.139:0/806519981 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a006f890 con 0x7fa1a00c6e00 2024-09-16T00:09:23.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.020+0000 7fa1a51b2640 1 --2- 172.21.15.139:0/806519981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a00c6e00 0x7fa1a00ca6b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa1900097a0 tx=0x7fa19002a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc158ae4e4502350 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:23.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.020+0000 7fa1a49b1640 1 -- 172.21.15.139:0/806519981 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190037070 con 0x7fa1a00c6e00 2024-09-16T00:09:23.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.020+0000 7fa1a49b1640 1 -- 172.21.15.139:0/806519981 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa190032640 con 0x7fa1a00c6e00 2024-09-16T00:09:23.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.020+0000 7fa1a49b1640 1 -- 172.21.15.139:0/806519981 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190032a10 con 0x7fa1a00c6e00 2024-09-16T00:09:23.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.021+0000 7fa1a7c17640 1 -- 172.21.15.139:0/806519981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a00c6e00 msgr2=0x7fa1a00ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:23.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.021+0000 7fa1a7c17640 1 --2- 172.21.15.139:0/806519981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a00c6e00 0x7fa1a00ca6b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa1900097a0 tx=0x7fa19002a2c0 comp rx=0 tx=0).stop 2024-09-16T00:09:23.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.021+0000 7fa1a7c17640 1 -- 172.21.15.139:0/806519981 shutdown_connections 2024-09-16T00:09:23.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.021+0000 7fa1a7c17640 1 --2- 172.21.15.139:0/806519981 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a00c6e00 0x7fa1a00ca6b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:23.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.021+0000 7fa1a7c17640 1 -- 172.21.15.139:0/806519981 >> 172.21.15.139:0/806519981 conn(0x7fa1a006cc30 msgr2=0x7fa1a006d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:23.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.021+0000 7fa1a7c17640 1 -- 172.21.15.139:0/806519981 shutdown_connections 2024-09-16T00:09:23.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.021+0000 7fa1a7c17640 1 -- 172.21.15.139:0/806519981 wait complete. 2024-09-16T00:09:23.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.022+0000 7fa1a7c17640 1 Processor -- start 2024-09-16T00:09:23.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.023+0000 7fa1a7c17640 1 -- start start 2024-09-16T00:09:23.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.023+0000 7fa1a7c17640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa1a00c6a20 0x7fa1a0149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:23.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.023+0000 7fa1a7c17640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a00c6e00 0x7fa1a01e1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:23.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.023+0000 7fa1a7c17640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1a0158620 0x7fa1a01e23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:23.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.023+0000 7fa1a7c17640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a006fe50 con 0x7fa1a0158620 2024-09-16T00:09:23.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.023+0000 7fa1a7c17640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a006edb0 con 0x7fa1a00c6e00 2024-09-16T00:09:23.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.023+0000 7fa1a7c17640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a00c9eb0 con 0x7fa1a00c6a20 2024-09-16T00:09:23.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.023+0000 7fa1a51b2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a00c6e00 0x7fa1a01e1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:23.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.023+0000 7fa1a61b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1a0158620 0x7fa1a01e23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:23.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.023+0000 7fa1a51b2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a00c6e00 0x7fa1a01e1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:47720/0 (socket says 172.21.15.139:47720) 2024-09-16T00:09:23.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.023+0000 7fa1a51b2640 1 -- 172.21.15.139:0/740091063 learned_addr learned my addr 172.21.15.139:0/740091063 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:23.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.023+0000 7fa1a59b3640 1 --2- 172.21.15.139:0/740091063 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa1a00c6a20 0x7fa1a0149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:23.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.024+0000 7fa1a51b2640 1 -- 172.21.15.139:0/740091063 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa1a00c6a20 msgr2=0x7fa1a0149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:23.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.024+0000 7fa1a51b2640 1 --2- 172.21.15.139:0/740091063 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa1a00c6a20 0x7fa1a0149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:23.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.024+0000 7fa1a51b2640 1 -- 172.21.15.139:0/740091063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1a0158620 msgr2=0x7fa1a01e23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:23.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.024+0000 7fa1a51b2640 1 --2- 172.21.15.139:0/740091063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1a0158620 0x7fa1a01e23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:23.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.024+0000 7fa1a51b2640 1 -- 172.21.15.139:0/740091063 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a014bee0 con 0x7fa1a00c6e00 2024-09-16T00:09:23.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.024+0000 7fa1a51b2640 1 --2- 172.21.15.139:0/740091063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a00c6e00 0x7fa1a01e1ea0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa19002a8b0 tx=0x7fa1900046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:23.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.024+0000 7fa1967fc640 1 -- 172.21.15.139:0/740091063 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190037040 con 0x7fa1a00c6e00 2024-09-16T00:09:23.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.024+0000 7fa1967fc640 1 -- 172.21.15.139:0/740091063 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa190032440 con 0x7fa1a00c6e00 2024-09-16T00:09:23.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.024+0000 7fa1967fc640 1 -- 172.21.15.139:0/740091063 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190030990 con 0x7fa1a00c6e00 2024-09-16T00:09:23.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.024+0000 7fa1a7c17640 1 -- 172.21.15.139:0/740091063 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a014c1c0 con 0x7fa1a00c6e00 2024-09-16T00:09:23.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.024+0000 7fa1a7c17640 1 -- 172.21.15.139:0/740091063 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a01e4710 con 0x7fa1a00c6e00 2024-09-16T00:09:23.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.025+0000 7fa1a7c17640 1 -- 172.21.15.139:0/740091063 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a01e4220 con 0x7fa1a00c6e00 2024-09-16T00:09:23.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.026+0000 7fa1967fc640 1 -- 172.21.15.139:0/740091063 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa190044020 con 0x7fa1a00c6e00 2024-09-16T00:09:23.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.026+0000 7fa1967fc640 1 --2- 172.21.15.139:0/740091063 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa18406c2b0 0x7fa18406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:23.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.027+0000 7fa1967fc640 1 -- 172.21.15.139:0/740091063 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5944+0+0 (secure 0 0 0) 0x7fa1900ab650 con 0x7fa1a00c6e00 2024-09-16T00:09:23.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.027+0000 7fa1a59b3640 1 --2- 172.21.15.139:0/740091063 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa18406c2b0 0x7fa18406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:23.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.027+0000 7fa1a59b3640 1 --2- 172.21.15.139:0/740091063 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa18406c2b0 0x7fa18406e770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa19c000f60 tx=0x7fa19c007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:23.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.031+0000 7fa1967fc640 1 -- 172.21.15.139:0/740091063 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa190079710 con 0x7fa1a00c6e00 2024-09-16T00:09:23.070 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:23.069+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:23.095 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:23.095 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:23.106 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:09:23.171 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:23.170+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:23.222 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T00:09:23.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.268+0000 7fa1a7c17640 1 -- 172.21.15.139:0/740091063 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pg-upmap", "pgid": "3.c", "id": [5, 3]} v 0) -- 0x7fa1a00ca6b0 con 0x7fa1a00c6e00 2024-09-16T00:09:23.272 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:23.271+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:23.275 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:23.275 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:23.287 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:09:23.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.340+0000 7f380bcd2640 1 Processor -- start 2024-09-16T00:09:23.344 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.341+0000 7f380bcd2640 1 -- start start 2024-09-16T00:09:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.342+0000 7f380bcd2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f38040cd2a0 0x7f38040c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.342+0000 7f380bcd2640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3804151b30 0x7f3804157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.342+0000 7f380bcd2640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f380406f990 con 0x7f38040cd680 2024-09-16T00:09:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.342+0000 7f380bcd2640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f380406ee10 con 0x7f38040cd2a0 2024-09-16T00:09:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.342+0000 7f380bcd2640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f380406f0f0 con 0x7f3804151b30 2024-09-16T00:09:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.342+0000 7f3809a6e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f38040cd2a0 0x7f38040c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:23.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.342+0000 7f3809a6e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f38040cd2a0 0x7f38040c7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54074/0 (socket says 172.21.15.139:54074) 2024-09-16T00:09:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.342+0000 7f380926d640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f38040cd680 0x7f3804151420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:56100/0 (socket says 172.21.15.139:56100) 2024-09-16T00:09:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.342+0000 7f3809a6e640 1 -- 172.21.15.139:0/737015240 learned_addr learned my addr 172.21.15.139:0/737015240 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.342+0000 7f380a26f640 1 --2- 172.21.15.139:0/737015240 >> v2:172.21.15.164:3300/0 conn(0x7f3804151b30 0x7f3804157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.342+0000 7f3809a6e640 1 -- 172.21.15.139:0/737015240 >> v2:172.21.15.164:3300/0 conn(0x7f3804151b30 msgr2=0x7f3804157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.342+0000 7f3809a6e640 1 --2- 172.21.15.139:0/737015240 >> v2:172.21.15.164:3300/0 conn(0x7f3804151b30 0x7f3804157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.342+0000 7f3809a6e640 1 -- 172.21.15.139:0/737015240 >> v1:172.21.15.164:6789/0 conn(0x7f38040cd680 legacy=0x7f3804151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.342+0000 7f3809a6e640 1 -- 172.21.15.139:0/737015240 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380406f3d0 con 0x7f38040cd2a0 2024-09-16T00:09:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.343+0000 7f3809a6e640 1 --2- 172.21.15.139:0/737015240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38040cd2a0 0x7f38040c7aa0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f38040c5fb0 tx=0x7f380002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805707f4223c77b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.343+0000 7f3808a6c640 1 -- 172.21.15.139:0/737015240 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800037070 con 0x7f38040cd2a0 2024-09-16T00:09:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.343+0000 7f3808a6c640 1 -- 172.21.15.139:0/737015240 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f380002ae50 con 0x7f38040cd2a0 2024-09-16T00:09:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.343+0000 7f3808a6c640 1 -- 172.21.15.139:0/737015240 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800032860 con 0x7f38040cd2a0 2024-09-16T00:09:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.344+0000 7f380bcd2640 1 -- 172.21.15.139:0/737015240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38040cd2a0 msgr2=0x7f38040c7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:23.347 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.344+0000 7f380bcd2640 1 --2- 172.21.15.139:0/737015240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38040cd2a0 0x7f38040c7aa0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f38040c5fb0 tx=0x7f380002a300 comp rx=0 tx=0).stop 2024-09-16T00:09:23.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.344+0000 7f380bcd2640 1 -- 172.21.15.139:0/737015240 shutdown_connections 2024-09-16T00:09:23.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.344+0000 7f380bcd2640 1 --2- 172.21.15.139:0/737015240 >> v2:172.21.15.164:3300/0 conn(0x7f3804151b30 0x7f3804157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:23.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.344+0000 7f380bcd2640 1 --2- 172.21.15.139:0/737015240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38040cd2a0 0x7f38040c7aa0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:23.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.344+0000 7f380bcd2640 1 -- 172.21.15.139:0/737015240 >> 172.21.15.139:0/737015240 conn(0x7f380406cc30 msgr2=0x7f380406d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:23.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.344+0000 7f380bcd2640 1 -- 172.21.15.139:0/737015240 shutdown_connections 2024-09-16T00:09:23.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.344+0000 7f380bcd2640 1 -- 172.21.15.139:0/737015240 wait complete. 2024-09-16T00:09:23.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.346+0000 7f380bcd2640 1 Processor -- start 2024-09-16T00:09:23.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.346+0000 7f380bcd2640 1 -- start start 2024-09-16T00:09:23.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f380bcd2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38040cd2a0 0x7f38041eec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:23.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f380bcd2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38040cd680 0x7f3804156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:23.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f380bcd2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3804151b30 0x7f38041f2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:23.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f380bcd2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f38040c72b0 con 0x7f38040cd680 2024-09-16T00:09:23.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f380bcd2640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f38041576b0 con 0x7f38040cd2a0 2024-09-16T00:09:23.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f380bcd2640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f38040c7430 con 0x7f3804151b30 2024-09-16T00:09:23.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f380926d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38040cd680 0x7f3804156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:23.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f3809a6e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38040cd2a0 0x7f38041eec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:23.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f380926d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38040cd680 0x7f3804156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54076/0 (socket says 172.21.15.139:54076) 2024-09-16T00:09:23.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f3809a6e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38040cd2a0 0x7f38041eec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:47740/0 (socket says 172.21.15.139:47740) 2024-09-16T00:09:23.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f380926d640 1 -- 172.21.15.139:0/653640834 learned_addr learned my addr 172.21.15.139:0/653640834 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:23.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f380a26f640 1 --2- 172.21.15.139:0/653640834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3804151b30 0x7f38041f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:23.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f3809a6e640 1 -- 172.21.15.139:0/653640834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3804151b30 msgr2=0x7f38041f2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:23.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f3809a6e640 1 --2- 172.21.15.139:0/653640834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3804151b30 0x7f38041f2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:23.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f3809a6e640 1 -- 172.21.15.139:0/653640834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38040cd680 msgr2=0x7f3804156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:23.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f3809a6e640 1 --2- 172.21.15.139:0/653640834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38040cd680 0x7f3804156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:23.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f3809a6e640 1 -- 172.21.15.139:0/653640834 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380414a9d0 con 0x7f38040cd2a0 2024-09-16T00:09:23.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f380926d640 1 --2- 172.21.15.139:0/653640834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38040cd680 0x7f3804156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:23.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.347+0000 7f380a26f640 1 --2- 172.21.15.139:0/653640834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3804151b30 0x7f38041f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:23.351 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.348+0000 7f3809a6e640 1 --2- 172.21.15.139:0/653640834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38040cd2a0 0x7f38041eec30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3800031000 tx=0x7f38000046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:23.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.348+0000 7f37fa7fc640 1 -- 172.21.15.139:0/653640834 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800041070 con 0x7f38040cd2a0 2024-09-16T00:09:23.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.348+0000 7f380bcd2640 1 -- 172.21.15.139:0/653640834 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380414acb0 con 0x7f38040cd2a0 2024-09-16T00:09:23.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.348+0000 7f37fa7fc640 1 -- 172.21.15.139:0/653640834 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f380003c660 con 0x7f38040cd2a0 2024-09-16T00:09:23.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.348+0000 7f380bcd2640 1 -- 172.21.15.139:0/653640834 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38041ee430 con 0x7f38040cd2a0 2024-09-16T00:09:23.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.348+0000 7f37fa7fc640 1 -- 172.21.15.139:0/653640834 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800041070 con 0x7f38040cd2a0 2024-09-16T00:09:23.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.349+0000 7f380bcd2640 1 -- 172.21.15.139:0/653640834 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38040c45a0 con 0x7f38040cd2a0 2024-09-16T00:09:23.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.350+0000 7f37fa7fc640 1 -- 172.21.15.139:0/653640834 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3800044070 con 0x7f38040cd2a0 2024-09-16T00:09:23.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.350+0000 7f37fa7fc640 1 --2- 172.21.15.139:0/653640834 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f37e806c4b0 0x7f37e806e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:23.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.350+0000 7f37fa7fc640 1 -- 172.21.15.139:0/653640834 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5944+0+0 (secure 0 0 0) 0x7f38000ab930 con 0x7f38040cd2a0 2024-09-16T00:09:23.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.350+0000 7f380926d640 1 --2- 172.21.15.139:0/653640834 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f37e806c4b0 0x7f37e806e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:23.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.351+0000 7f380926d640 1 --2- 172.21.15.139:0/653640834 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f37e806c4b0 0x7f37e806e970 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3804157650 tx=0x7f37f4007ee0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:23.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.354+0000 7f37fa7fc640 1 -- 172.21.15.139:0/653640834 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f38000799f0 con 0x7f38040cd2a0 2024-09-16T00:09:23.372 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:23.372+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:23.443 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:23.443 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:23.452 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:09:23.473 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:23.472+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:23.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:23.562+0000 7f380bcd2640 1 -- 172.21.15.139:0/653640834 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f3804070e80 con 0x7f38040cd2a0 2024-09-16T00:09:23.574 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:23.573+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:23.614 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:23.615 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:23.624 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:09:23.680 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:23.679+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:23.747 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:23.747 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:23.757 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:09:23.801 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:23.800+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:23.878 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:23.877+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:23.892 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:23.892 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:23.902 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:09:23.979 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:23.979+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:24.028 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:24.028 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:24.039 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:09:24.083 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:24.082+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:24.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.131+0000 7fa1967fc640 1 -- 172.21.15.139:0/740091063 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [5, 3]}]=0 set 3.c pg_upmap mapping to [5,3] v75) ==== 122+0+0 (secure 0 0 0) 0x7fa19007da10 con 0x7fa1a00c6e00 2024-09-16T00:09:24.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.131+0000 7f37fa7fc640 1 -- 172.21.15.139:0/653640834 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v75) ==== 91+0+0 (secure 0 0 0) 0x7f380007dcf0 con 0x7f38040cd2a0 2024-09-16T00:09:24.132 INFO:teuthology.orchestra.run.smithi139.stderr:set 3.c pg_upmap mapping to [5,3] 2024-09-16T00:09:24.132 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is unset 2024-09-16T00:09:24.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7f380bcd2640 1 -- 172.21.15.139:0/653640834 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f37e806c4b0 msgr2=0x7f37e806e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7f380bcd2640 1 --2- 172.21.15.139:0/653640834 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f37e806c4b0 0x7f37e806e970 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3804157650 tx=0x7f37f4007ee0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7f380bcd2640 1 -- 172.21.15.139:0/653640834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38040cd2a0 msgr2=0x7f38041eec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7f380bcd2640 1 --2- 172.21.15.139:0/653640834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38040cd2a0 0x7f38041eec30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3800031000 tx=0x7f38000046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7f380bcd2640 1 -- 172.21.15.139:0/653640834 shutdown_connections 2024-09-16T00:09:24.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7f380bcd2640 1 --2- 172.21.15.139:0/653640834 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f37e806c4b0 0x7f37e806e970 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7f380bcd2640 1 --2- 172.21.15.139:0/653640834 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3804151b30 0x7f38041f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7f380bcd2640 1 --2- 172.21.15.139:0/653640834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38040cd680 0x7f3804156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7f380bcd2640 1 --2- 172.21.15.139:0/653640834 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38040cd2a0 0x7f38041eec30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7f380bcd2640 1 -- 172.21.15.139:0/653640834 >> 172.21.15.139:0/653640834 conn(0x7f380406cc30 msgr2=0x7f38040cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:24.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7fa1a7c17640 1 -- 172.21.15.139:0/740091063 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa18406c2b0 msgr2=0x7fa18406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7fa1a7c17640 1 --2- 172.21.15.139:0/740091063 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa18406c2b0 0x7fa18406e770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa19c000f60 tx=0x7fa19c007d30 comp rx=0 tx=0).stop 2024-09-16T00:09:24.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7fa1a7c17640 1 -- 172.21.15.139:0/740091063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a00c6e00 msgr2=0x7fa1a01e1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7fa1a7c17640 1 --2- 172.21.15.139:0/740091063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a00c6e00 0x7fa1a01e1ea0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa19002a8b0 tx=0x7fa1900046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7f380bcd2640 1 -- 172.21.15.139:0/653640834 shutdown_connections 2024-09-16T00:09:24.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7f380bcd2640 1 -- 172.21.15.139:0/653640834 wait complete. 2024-09-16T00:09:24.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7fa1a7c17640 1 -- 172.21.15.139:0/740091063 shutdown_connections 2024-09-16T00:09:24.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7fa1a7c17640 1 --2- 172.21.15.139:0/740091063 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa18406c2b0 0x7fa18406e770 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7fa1a7c17640 1 --2- 172.21.15.139:0/740091063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1a0158620 0x7fa1a01e23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7fa1a7c17640 1 --2- 172.21.15.139:0/740091063 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1a00c6e00 0x7fa1a01e1ea0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7fa1a7c17640 1 --2- 172.21.15.139:0/740091063 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa1a00c6a20 0x7fa1a0149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7fa1a7c17640 1 -- 172.21.15.139:0/740091063 >> 172.21.15.139:0/740091063 conn(0x7fa1a006cc30 msgr2=0x7fa1a00c9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:24.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7fa1a7c17640 1 -- 172.21.15.139:0/740091063 shutdown_connections 2024-09-16T00:09:24.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.134+0000 7fa1a7c17640 1 -- 172.21.15.139:0/740091063 wait complete. 2024-09-16T00:09:24.162 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:24.163 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:24.173 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:09:24.191 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:24.190+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:24.198 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:09:24.282 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:24.281+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:24.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.304+0000 7f10d128d640 1 Processor -- start 2024-09-16T00:09:24.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.304+0000 7f10d128d640 1 -- start start 2024-09-16T00:09:24.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.304+0000 7f10d128d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f10cc1519d0 0x7f10cc0c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:24.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.304+0000 7f10d128d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10cc067ad0 con 0x7f10cc0c8a80 2024-09-16T00:09:24.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.304+0000 7f10d128d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10cc067db0 con 0x7f10cc0c58f0 2024-09-16T00:09:24.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.304+0000 7f10d128d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f10cc06d320 con 0x7f10cc1519d0 2024-09-16T00:09:24.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.305+0000 7f10cb7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f10cc0c8a80 0x7f10cc162010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41694/0 (socket says 172.21.15.139:41694) 2024-09-16T00:09:24.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.305+0000 7f10cb7fe640 1 -- 172.21.15.139:0/3997669754 learned_addr learned my addr 172.21.15.139:0/3997669754 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:24.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.305+0000 7f10caffd640 1 --2- 172.21.15.139:0/3997669754 >> v2:172.21.15.164:3300/0 conn(0x7f10cc1519d0 0x7f10cc0c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:24.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.305+0000 7f10c9ffb640 1 -- 172.21.15.139:0/3997669754 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1565020121 0 0) 0x7f10cc067ad0 con 0x7f10cc0c8a80 2024-09-16T00:09:24.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.305+0000 7f10c9ffb640 1 -- 172.21.15.139:0/3997669754 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10b4003520 con 0x7f10cc0c8a80 2024-09-16T00:09:24.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.305+0000 7f10c9ffb640 1 -- 172.21.15.139:0/3997669754 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1165307390 0 0) 0x7f10cc067db0 con 0x7f10cc0c58f0 2024-09-16T00:09:24.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.305+0000 7f10c9ffb640 1 -- 172.21.15.139:0/3997669754 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10cc067ad0 con 0x7f10cc0c58f0 2024-09-16T00:09:24.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.305+0000 7f10caffd640 1 -- 172.21.15.139:0/3997669754 >> v1:172.21.15.164:6789/0 conn(0x7f10cc0c58f0 legacy=0x7f10cc0c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.305+0000 7f10caffd640 1 -- 172.21.15.139:0/3997669754 >> v1:172.21.15.139:6789/0 conn(0x7f10cc0c8a80 legacy=0x7f10cc162010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.305+0000 7f10caffd640 1 -- 172.21.15.139:0/3997669754 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc06ccd0 con 0x7f10cc1519d0 2024-09-16T00:09:24.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.306+0000 7f10caffd640 1 --2- 172.21.15.139:0/3997669754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc1519d0 0x7f10cc0c3cd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f10c40097a0 tx=0x7f10c402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=552875cfd1fcdc13 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:24.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.306+0000 7f10c9ffb640 1 -- 172.21.15.139:0/3997669754 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4037070 con 0x7f10cc1519d0 2024-09-16T00:09:24.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.306+0000 7f10c9ffb640 1 -- 172.21.15.139:0/3997669754 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c4032640 con 0x7f10cc1519d0 2024-09-16T00:09:24.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.306+0000 7f10c9ffb640 1 -- 172.21.15.139:0/3997669754 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c4032a10 con 0x7f10cc1519d0 2024-09-16T00:09:24.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.306+0000 7f10d128d640 1 -- 172.21.15.139:0/3997669754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc1519d0 msgr2=0x7f10cc0c3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.306+0000 7f10d128d640 1 --2- 172.21.15.139:0/3997669754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc1519d0 0x7f10cc0c3cd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f10c40097a0 tx=0x7f10c402a2c0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.306+0000 7f10d128d640 1 -- 172.21.15.139:0/3997669754 shutdown_connections 2024-09-16T00:09:24.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.306+0000 7f10d128d640 1 --2- 172.21.15.139:0/3997669754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc1519d0 0x7f10cc0c3cd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.306+0000 7f10d128d640 1 -- 172.21.15.139:0/3997669754 >> 172.21.15.139:0/3997669754 conn(0x7f10cc0c92d0 msgr2=0x7f10cc0c96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:24.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.307+0000 7f10d128d640 1 -- 172.21.15.139:0/3997669754 shutdown_connections 2024-09-16T00:09:24.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.307+0000 7f10d128d640 1 -- 172.21.15.139:0/3997669754 wait complete. 2024-09-16T00:09:24.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.308+0000 7f10d128d640 1 Processor -- start 2024-09-16T00:09:24.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.308+0000 7f10d128d640 1 -- start start 2024-09-16T00:09:24.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.308+0000 7f10d128d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10cc0c58f0 0x7f10cc1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:24.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.308+0000 7f10d128d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10cc0c8a80 0x7f10cc157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:24.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10d128d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc1519d0 0x7f10cc140b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:24.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10d128d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc0c4210 con 0x7f10cc0c8a80 2024-09-16T00:09:24.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10d128d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc0c6ba0 con 0x7f10cc1519d0 2024-09-16T00:09:24.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10d128d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc151db0 con 0x7f10cc0c58f0 2024-09-16T00:09:24.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10ca7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10cc0c8a80 0x7f10cc157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:24.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10ca7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10cc0c8a80 0x7f10cc157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54082/0 (socket says 172.21.15.139:54082) 2024-09-16T00:09:24.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10ca7fc640 1 -- 172.21.15.139:0/453703106 learned_addr learned my addr 172.21.15.139:0/453703106 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:24.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10cb7fe640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc1519d0 0x7f10cc140b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:24.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10caffd640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10cc0c58f0 0x7f10cc1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:24.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10ca7fc640 1 -- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10cc0c58f0 msgr2=0x7f10cc1466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10ca7fc640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10cc0c58f0 0x7f10cc1466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10ca7fc640 1 -- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc1519d0 msgr2=0x7f10cc140b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10ca7fc640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc1519d0 0x7f10cc140b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10ca7fc640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc14eb20 con 0x7f10cc0c8a80 2024-09-16T00:09:24.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10cb7fe640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f10cc1519d0 0x7f10cc140b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:24.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10ca7fc640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10cc0c8a80 0x7f10cc157030 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f10c000c8a0 tx=0x7f10c000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:24.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0003b70 con 0x7f10cc0c8a80 2024-09-16T00:09:24.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c0003d30 con 0x7f10cc0c8a80 2024-09-16T00:09:24.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc145c00 con 0x7f10cc0c8a80 2024-09-16T00:09:24.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.309+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c000aa90 con 0x7f10cc0c8a80 2024-09-16T00:09:24.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.310+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc0750c0 con 0x7f10cc0c8a80 2024-09-16T00:09:24.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.311+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f10cc067db0 con 0x7f10cc0c8a80 2024-09-16T00:09:24.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.311+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f10c000ac30 con 0x7f10cc0c8a80 2024-09-16T00:09:24.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.312+0000 7f10b37fe640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f10b806c2b0 0x7f10b806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:24.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.312+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5944+0+0 (secure 0 0 0) 0x7f10c008b400 con 0x7f10cc0c8a80 2024-09-16T00:09:24.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.312+0000 7f10caffd640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f10b806c2b0 0x7f10b806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:24.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.312+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v75) ==== 54+0+16 (secure 0 0 0) 0x7f10c008da80 con 0x7f10cc0c8a80 2024-09-16T00:09:24.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.312+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f10cc152580 0x7f10cc1410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:24.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.312+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f10cc155350 con 0x7f10cc152580 2024-09-16T00:09:24.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.312+0000 7f10caffd640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f10b806c2b0 0x7f10b806e770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f10c4034000 tx=0x7f10c402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:24.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.313+0000 7f10cb7fe640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f10cc152580 0x7f10cc1410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:24.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.313+0000 7f10cb7fe640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f10cc152580 0x7f10cc1410b0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:24.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.313+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10cc155350 con 0x7f10cc152580 2024-09-16T00:09:24.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.331+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f10cc0c7620 con 0x7f10cc152580 2024-09-16T00:09:24.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.333+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f10cc0c7620 con 0x7f10cc152580 2024-09-16T00:09:24.334 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:24.335 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:24.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.333+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f10cc147060 0x7f10cc154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:24.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.333+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f10cc0c7340 con 0x7f10cc147060 2024-09-16T00:09:24.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.333+0000 7f10caffd640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f10cc147060 0x7f10cc154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:24.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.334+0000 7f10caffd640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f10cc147060 0x7f10cc154260 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:24.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.334+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10cc0c7340 con 0x7f10cc147060 2024-09-16T00:09:24.340 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:24.340 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:24.350 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:09:24.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.352+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f10cc146250 con 0x7f10cc147060 2024-09-16T00:09:24.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.353+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f10cc146250 con 0x7f10cc147060 2024-09-16T00:09:24.354 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:24.355 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:24.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.354+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f10cc154cb0 0x7f10cc062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:24.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.354+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f10cc152250 con 0x7f10cc154cb0 2024-09-16T00:09:24.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.354+0000 7f10ca7fc640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f10cc154cb0 0x7f10cc062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:24.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.354+0000 7f10ca7fc640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f10cc154cb0 0x7f10cc062630 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:24.356 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.355+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10cc152250 con 0x7f10cc154cb0 2024-09-16T00:09:24.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.373+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f10cc062e40 con 0x7f10cc154cb0 2024-09-16T00:09:24.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.374+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f10cc062e40 con 0x7f10cc154cb0 2024-09-16T00:09:24.375 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:24.376 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:24.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.374+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f10cc0caae0 0x7f10cc0caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:24.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.374+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f10cc146460 con 0x7f10cc0caae0 2024-09-16T00:09:24.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.375+0000 7f10cb7fe640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f10cc0caae0 0x7f10cc0caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:24.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.375+0000 7f10cb7fe640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f10cc0caae0 0x7f10cc0caec0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:24.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.376+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10cc146460 con 0x7f10cc0caae0 2024-09-16T00:09:24.383 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:24.382+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:24.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.391+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f10cc155090 con 0x7f10cc0caae0 2024-09-16T00:09:24.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.393+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f10cc155090 con 0x7f10cc0caae0 2024-09-16T00:09:24.394 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:24.394 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:24.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.393+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f10cc02cb50 0x7f10cc02cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:24.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.393+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f10cc062dc0 con 0x7f10cc02cb50 2024-09-16T00:09:24.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.393+0000 7f10caffd640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f10cc02cb50 0x7f10cc02cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:24.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.394+0000 7f10caffd640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f10cc02cb50 0x7f10cc02cf90 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:24.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.394+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10cc062dc0 con 0x7f10cc02cb50 2024-09-16T00:09:24.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.411+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f10cc0008d0 con 0x7f10cc02cb50 2024-09-16T00:09:24.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.412+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f10cc0008d0 con 0x7f10cc02cb50 2024-09-16T00:09:24.413 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:24.414 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:24.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.413+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f10cc202b70 0x7f10cc204f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:24.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.413+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f10cc0cb400 con 0x7f10cc202b70 2024-09-16T00:09:24.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.413+0000 7f10ca7fc640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f10cc202b70 0x7f10cc204f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:24.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.413+0000 7f10ca7fc640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f10cc202b70 0x7f10cc204f60 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:24.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.414+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10cc0cb400 con 0x7f10cc202b70 2024-09-16T00:09:24.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.430+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f10cc0cb400 con 0x7f10cc202b70 2024-09-16T00:09:24.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.432+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f10cc0cb400 con 0x7f10cc202b70 2024-09-16T00:09:24.433 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:24.433 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:24.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.432+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f10cc2059b0 0x7f10cc207da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:24.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.432+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f10cc02da60 con 0x7f10cc2059b0 2024-09-16T00:09:24.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.432+0000 7f10cb7fe640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f10cc2059b0 0x7f10cc207da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:24.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.433+0000 7f10cb7fe640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f10cc2059b0 0x7f10cc207da0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:24.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.433+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10cc02da60 con 0x7f10cc2059b0 2024-09-16T00:09:24.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.449+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f10cc02d9e0 con 0x7f10cc2059b0 2024-09-16T00:09:24.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.450+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f10cc02d9e0 con 0x7f10cc2059b0 2024-09-16T00:09:24.451 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:24.452 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:09:24.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.451+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f10cc2087f0 0x7f10cc20abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:24.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.451+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f10cc063550 con 0x7f10cc2087f0 2024-09-16T00:09:24.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.451+0000 7f10caffd640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f10cc2087f0 0x7f10cc20abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:24.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.451+0000 7f10caffd640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f10cc2087f0 0x7f10cc20abe0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:24.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.452+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10cc063550 con 0x7f10cc2087f0 2024-09-16T00:09:24.466 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:24.466 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:24.466 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:24.466 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:24.466 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:24.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.468+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f10cc062dc0 con 0x7f10cc2087f0 2024-09-16T00:09:24.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10b37fe640 1 -- 172.21.15.139:0/453703106 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f10cc062dc0 con 0x7f10cc2087f0 2024-09-16T00:09:24.470 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:24.472 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:24.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f10cc152580 msgr2=0x7f10cc1410b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f10cc152580 0x7f10cc1410b0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f10cc147060 msgr2=0x7f10cc154260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f10cc147060 0x7f10cc154260 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f10cc154cb0 msgr2=0x7f10cc062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f10cc154cb0 0x7f10cc062630 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f10cc0caae0 msgr2=0x7f10cc0caec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f10cc0caae0 0x7f10cc0caec0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f10cc02cb50 msgr2=0x7f10cc02cf90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f10cc02cb50 0x7f10cc02cf90 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f10cc202b70 msgr2=0x7f10cc204f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f10cc202b70 0x7f10cc204f60 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f10cc2059b0 msgr2=0x7f10cc207da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f10cc2059b0 0x7f10cc207da0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f10cc2087f0 msgr2=0x7f10cc20abe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f10cc2087f0 0x7f10cc20abe0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10cb7fe640 1 -- 172.21.15.139:0/453703106 reap_dead start 2024-09-16T00:09:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10cb7fe640 1 -- 172.21.15.139:0/453703106 reap_dead start 2024-09-16T00:09:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10cb7fe640 1 -- 172.21.15.139:0/453703106 reap_dead start 2024-09-16T00:09:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10cb7fe640 1 -- 172.21.15.139:0/453703106 reap_dead start 2024-09-16T00:09:24.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10cb7fe640 1 -- 172.21.15.139:0/453703106 reap_dead start 2024-09-16T00:09:24.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f10b806c2b0 msgr2=0x7f10b806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f10b806c2b0 0x7f10b806e770 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f10c4034000 tx=0x7f10c402e000 comp rx=0 tx=0).stop 2024-09-16T00:09:24.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10cc0c8a80 msgr2=0x7f10cc157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:24.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10cc0c8a80 0x7f10cc157030 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f10c000c8a0 tx=0x7f10c000cd60 comp rx=0 tx=0).stop 2024-09-16T00:09:24.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 shutdown_connections 2024-09-16T00:09:24.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f10b806c2b0 0x7f10b806e770 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 --2- 172.21.15.139:0/453703106 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10cc0c8a80 0x7f10cc157030 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:24.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 >> 172.21.15.139:0/453703106 conn(0x7f10cc0c92d0 msgr2=0x7f10cc147a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:24.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 shutdown_connections 2024-09-16T00:09:24.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:24.470+0000 7f10d128d640 1 -- 172.21.15.139:0/453703106 wait complete. 2024-09-16T00:09:24.475 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:09:24.519 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:24.518+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:24.584 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:24.583+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:24.594 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:24.594 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:24.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:24.594 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:24.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.594 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:14 2.15 e22/19 2:a93a17c2:::604.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:24.594 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:24.594 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.976526349, 2024-09-16T00:09:24.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029190470000000001, 2024-09-16T00:09:24.595 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.595 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.595 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.595 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:24.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:24.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615982+0000", 2024-09-16T00:09:24.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7439999999999999e-06 2024-09-16T00:09:24.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:09:24.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3743999999999999e-05 2024-09-16T00:09:24.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:24.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2255e-05 2024-09-16T00:09:24.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616029+0000", 2024-09-16T00:09:24.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1175999999999999e-05 2024-09-16T00:09:24.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616098+0000", 2024-09-16T00:09:24.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9153999999999996e-05 2024-09-16T00:09:24.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616132+0000", 2024-09-16T00:09:24.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3924000000000002e-05 2024-09-16T00:09:24.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618855+0000", 2024-09-16T00:09:24.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027226189999999999 2024-09-16T00:09:24.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618899+0000", 2024-09-16T00:09:24.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4431000000000002e-05 2024-09-16T00:09:24.599 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.599 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.599 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.599 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:30 2.15 e22/19 2:aa448500:::500.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:09:24.599 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:09:24.599 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.97424204399999, 2024-09-16T00:09:24.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058031250000000001, 2024-09-16T00:09:24.599 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.599 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.599 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.600 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:09:24.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618264+0000", 2024-09-16T00:09:24.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618265+0000", 2024-09-16T00:09:24.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9200000000000001e-07 2024-09-16T00:09:24.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618271+0000", 2024-09-16T00:09:24.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9580000000000004e-06 2024-09-16T00:09:24.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618279+0000", 2024-09-16T00:09:24.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2239999999999995e-06 2024-09-16T00:09:24.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618285+0000", 2024-09-16T00:09:24.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5199999999999997e-06 2024-09-16T00:09:24.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618567+0000", 2024-09-16T00:09:24.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028204400000000001 2024-09-16T00:09:24.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:09:24.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9678999999999999e-05 2024-09-16T00:09:24.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624024+0000", 2024-09-16T00:09:24.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054376720000000002 2024-09-16T00:09:24.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624068+0000", 2024-09-16T00:09:24.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:09:24.603 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.603 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.603 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.604 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:20 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:24.604 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:09:24.604 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.97388568, 2024-09-16T00:09:24.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028808739999999998, 2024-09-16T00:09:24.604 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.604 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.604 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.604 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:09:24.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618621+0000", 2024-09-16T00:09:24.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618622+0000", 2024-09-16T00:09:24.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1260000000000001e-06 2024-09-16T00:09:24.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618631+0000", 2024-09-16T00:09:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4700000000000008e-06 2024-09-16T00:09:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618645+0000", 2024-09-16T00:09:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3984e-05 2024-09-16T00:09:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618658+0000", 2024-09-16T00:09:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2101e-05 2024-09-16T00:09:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618720+0000", 2024-09-16T00:09:24.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2761000000000004e-05 2024-09-16T00:09:24.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618740+0000", 2024-09-16T00:09:24.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9409e-05 2024-09-16T00:09:24.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621436+0000", 2024-09-16T00:09:24.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002696211 2024-09-16T00:09:24.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621502+0000", 2024-09-16T00:09:24.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5811999999999999e-05 2024-09-16T00:09:24.608 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.608 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.608 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.608 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:21 2.1c e22/19 2:3ed09add:::607.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:24.608 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:09:24.608 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 105.973062392, 2024-09-16T00:09:24.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046318540000000004, 2024-09-16T00:09:24.608 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.608 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.609 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.609 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:09:24.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619444+0000", 2024-09-16T00:09:24.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619445+0000", 2024-09-16T00:09:24.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.083e-06 2024-09-16T00:09:24.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619455+0000", 2024-09-16T00:09:24.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5920000000000007e-06 2024-09-16T00:09:24.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619467+0000", 2024-09-16T00:09:24.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2711000000000001e-05 2024-09-16T00:09:24.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619476+0000", 2024-09-16T00:09:24.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0139999999999998e-06 2024-09-16T00:09:24.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619529+0000", 2024-09-16T00:09:24.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3078999999999998e-05 2024-09-16T00:09:24.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619548+0000", 2024-09-16T00:09:24.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8952000000000001e-05 2024-09-16T00:09:24.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624037+0000", 2024-09-16T00:09:24.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044895730000000002 2024-09-16T00:09:24.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624076+0000", 2024-09-16T00:09:24.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8850000000000002e-05 2024-09-16T00:09:24.612 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.612 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.612 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.613 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:93 3.5 3:a3a50212:::10000000254.00000000:head [write 0~6832] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:24.613 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.467788+0000", 2024-09-16T00:09:24.613 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.124718642000001, 2024-09-16T00:09:24.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001903882, 2024-09-16T00:09:24.613 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.613 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.613 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:24.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:24.613 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:24.613 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 93 2024-09-16T00:09:24.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.614 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467788+0000", 2024-09-16T00:09:24.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467788+0000", 2024-09-16T00:09:24.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467789+0000", 2024-09-16T00:09:24.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4399999999999999e-07 2024-09-16T00:09:24.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467798+0000", 2024-09-16T00:09:24.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0510000000000002e-06 2024-09-16T00:09:24.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467809+0000", 2024-09-16T00:09:24.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0859000000000001e-05 2024-09-16T00:09:24.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467814+0000", 2024-09-16T00:09:24.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0409999999999997e-06 2024-09-16T00:09:24.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467860+0000", 2024-09-16T00:09:24.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6604999999999998e-05 2024-09-16T00:09:24.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.467935+0000", 2024-09-16T00:09:24.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4417000000000003e-05 2024-09-16T00:09:24.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:24.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.468012+0000", 2024-09-16T00:09:24.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6971000000000006e-05 2024-09-16T00:09:24.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:24.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.468821+0000", 2024-09-16T00:09:24.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00080984099999999995 2024-09-16T00:09:24.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:24.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.469636+0000", 2024-09-16T00:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00081428499999999999 2024-09-16T00:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.469668+0000", 2024-09-16T00:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2322e-05 2024-09-16T00:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.469692+0000", 2024-09-16T00:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3745999999999999e-05 2024-09-16T00:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.618 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.619 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:249 3.28 3:14c9b427:::100000002f7.00000000:head [write 0~5746] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:24.619 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.812683+0000", 2024-09-16T00:09:24.619 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.779823153999999, 2024-09-16T00:09:24.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019975150000000001, 2024-09-16T00:09:24.619 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.619 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.619 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:24.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:24.619 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:24.619 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 249 2024-09-16T00:09:24.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.620 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812683+0000", 2024-09-16T00:09:24.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812683+0000", 2024-09-16T00:09:24.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812684+0000", 2024-09-16T00:09:24.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1100000000000004e-07 2024-09-16T00:09:24.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812708+0000", 2024-09-16T00:09:24.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4074e-05 2024-09-16T00:09:24.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812736+0000", 2024-09-16T00:09:24.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7767e-05 2024-09-16T00:09:24.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812741+0000", 2024-09-16T00:09:24.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3319999999999999e-06 2024-09-16T00:09:24.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812790+0000", 2024-09-16T00:09:24.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8294000000000002e-05 2024-09-16T00:09:24.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812881+0000", 2024-09-16T00:09:24.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1085e-05 2024-09-16T00:09:24.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:24.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.812960+0000", 2024-09-16T00:09:24.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8824000000000003e-05 2024-09-16T00:09:24.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:24.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.813562+0000", 2024-09-16T00:09:24.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00060270500000000004 2024-09-16T00:09:24.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:24.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.814630+0000", 2024-09-16T00:09:24.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010679489999999999 2024-09-16T00:09:24.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.814660+0000", 2024-09-16T00:09:24.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.936e-05 2024-09-16T00:09:24.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.814681+0000", 2024-09-16T00:09:24.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1214000000000001e-05 2024-09-16T00:09:24.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.624 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.625 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.625 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:311 3.2a 3:548c374b:::10000000347.00000000:head [write 0~4872] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:24.625 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.002861+0000", 2024-09-16T00:09:24.625 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.589645877999999, 2024-09-16T00:09:24.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001734665, 2024-09-16T00:09:24.625 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.625 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.625 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:24.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:24.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:24.626 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 311 2024-09-16T00:09:24.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.626 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002861+0000", 2024-09-16T00:09:24.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002861+0000", 2024-09-16T00:09:24.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002863+0000", 2024-09-16T00:09:24.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0540000000000002e-06 2024-09-16T00:09:24.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002877+0000", 2024-09-16T00:09:24.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4677e-05 2024-09-16T00:09:24.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002896+0000", 2024-09-16T00:09:24.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9038999999999998e-05 2024-09-16T00:09:24.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002905+0000", 2024-09-16T00:09:24.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0599999999999997e-06 2024-09-16T00:09:24.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.002973+0000", 2024-09-16T00:09:24.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7203e-05 2024-09-16T00:09:24.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.003111+0000", 2024-09-16T00:09:24.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013858100000000001 2024-09-16T00:09:24.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:24.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.003269+0000", 2024-09-16T00:09:24.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015809 2024-09-16T00:09:24.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:24.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.004171+0000", 2024-09-16T00:09:24.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00090205600000000004 2024-09-16T00:09:24.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:24.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.004494+0000", 2024-09-16T00:09:24.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000322126 2024-09-16T00:09:24.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.004549+0000", 2024-09-16T00:09:24.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5049000000000001e-05 2024-09-16T00:09:24.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.004595+0000", 2024-09-16T00:09:24.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6730000000000002e-05 2024-09-16T00:09:24.631 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.631 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.631 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.631 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:333 3.c 3:329df26d:::1000000035f.00000000:head [write 0~657] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:24.631 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.059938+0000", 2024-09-16T00:09:24.631 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.532568178999998, 2024-09-16T00:09:24.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001777437, 2024-09-16T00:09:24.631 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.631 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.632 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:24.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:24.632 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:24.632 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 333 2024-09-16T00:09:24.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.632 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059938+0000", 2024-09-16T00:09:24.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059938+0000", 2024-09-16T00:09:24.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059940+0000", 2024-09-16T00:09:24.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1129999999999999e-06 2024-09-16T00:09:24.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059953+0000", 2024-09-16T00:09:24.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.278e-05 2024-09-16T00:09:24.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059980+0000", 2024-09-16T00:09:24.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6533000000000002e-05 2024-09-16T00:09:24.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.059994+0000", 2024-09-16T00:09:24.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4049999999999999e-05 2024-09-16T00:09:24.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.060136+0000", 2024-09-16T00:09:24.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014184399999999999 2024-09-16T00:09:24.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.060396+0000", 2024-09-16T00:09:24.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00026020600000000002 2024-09-16T00:09:24.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:24.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.060540+0000", 2024-09-16T00:09:24.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000143838 2024-09-16T00:09:24.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:24.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.061336+0000", 2024-09-16T00:09:24.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00079669099999999998 2024-09-16T00:09:24.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:24.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.061621+0000", 2024-09-16T00:09:24.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00028509599999999998 2024-09-16T00:09:24.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.061674+0000", 2024-09-16T00:09:24.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2685000000000001e-05 2024-09-16T00:09:24.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.061716+0000", 2024-09-16T00:09:24.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1600999999999997e-05 2024-09-16T00:09:24.637 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.637 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.637 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.637 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:490 3.d e58/51 3:b3311d72:::1000000041b.00000000:head v 58'8, mlcod=58'6)", 2024-09-16T00:09:24.637 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.728943+0000", 2024-09-16T00:09:24.637 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 26.863563880000001, 2024-09-16T00:09:24.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002073142, 2024-09-16T00:09:24.638 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.638 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.638 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.638 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728943+0000", 2024-09-16T00:09:24.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728943+0000", 2024-09-16T00:09:24.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728943+0000", 2024-09-16T00:09:24.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:09:24.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728949+0000", 2024-09-16T00:09:24.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.322e-06 2024-09-16T00:09:24.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728958+0000", 2024-09-16T00:09:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3049999999999998e-06 2024-09-16T00:09:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.728964+0000", 2024-09-16T00:09:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9699999999999996e-06 2024-09-16T00:09:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.729018+0000", 2024-09-16T00:09:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4045000000000002e-05 2024-09-16T00:09:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.729039+0000", 2024-09-16T00:09:24.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1098999999999999e-05 2024-09-16T00:09:24.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.730988+0000", 2024-09-16T00:09:24.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019486919999999999 2024-09-16T00:09:24.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.731016+0000", 2024-09-16T00:09:24.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7971e-05 2024-09-16T00:09:24.641 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.642 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.642 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.642 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:681 3.17 e59/20 3:ea55746f:::1000000050b.00000000:head v 59'12, mlcod=59'10)", 2024-09-16T00:09:24.642 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:58.265463+0000", 2024-09-16T00:09:24.642 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 26.327043004, 2024-09-16T00:09:24.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030780130000000001, 2024-09-16T00:09:24.642 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.642 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.642 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.642 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265463+0000", 2024-09-16T00:09:24.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265463+0000", 2024-09-16T00:09:24.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265464+0000", 2024-09-16T00:09:24.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7999999999999995e-07 2024-09-16T00:09:24.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265471+0000", 2024-09-16T00:09:24.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5010000000000001e-06 2024-09-16T00:09:24.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265480+0000", 2024-09-16T00:09:24.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.302e-06 2024-09-16T00:09:24.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265485+0000", 2024-09-16T00:09:24.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:09:24.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265526+0000", 2024-09-16T00:09:24.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1229000000000001e-05 2024-09-16T00:09:24.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.265539+0000", 2024-09-16T00:09:24.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3083e-05 2024-09-16T00:09:24.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.268498+0000", 2024-09-16T00:09:24.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029594320000000001 2024-09-16T00:09:24.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.268542+0000", 2024-09-16T00:09:24.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3207000000000002e-05 2024-09-16T00:09:24.646 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.646 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.646 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.646 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:811 3.1c 3:3b54038d:::100000005a1.00000000:head [write 0~2412] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:24.646 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:58.585366+0000", 2024-09-16T00:09:24.646 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 26.007140972999998, 2024-09-16T00:09:24.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003510216, 2024-09-16T00:09:24.647 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.647 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.647 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.647 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:24.647 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:24.647 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:24.647 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 811 2024-09-16T00:09:24.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.647 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585366+0000", 2024-09-16T00:09:24.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585366+0000", 2024-09-16T00:09:24.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585366+0000", 2024-09-16T00:09:24.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5300000000000003e-07 2024-09-16T00:09:24.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585371+0000", 2024-09-16T00:09:24.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2849999999999996e-06 2024-09-16T00:09:24.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585380+0000", 2024-09-16T00:09:24.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9219999999999992e-06 2024-09-16T00:09:24.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585390+0000", 2024-09-16T00:09:24.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2960000000000004e-06 2024-09-16T00:09:24.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585436+0000", 2024-09-16T00:09:24.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5874000000000002e-05 2024-09-16T00:09:24.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585510+0000", 2024-09-16T00:09:24.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4070000000000001e-05 2024-09-16T00:09:24.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:24.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.585586+0000", 2024-09-16T00:09:24.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6044999999999994e-05 2024-09-16T00:09:24.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:24.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.586070+0000", 2024-09-16T00:09:24.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048407300000000001 2024-09-16T00:09:24.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:24.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588779+0000", 2024-09-16T00:09:24.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027090339999999999 2024-09-16T00:09:24.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588834+0000", 2024-09-16T00:09:24.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5513000000000002e-05 2024-09-16T00:09:24.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.588876+0000", 2024-09-16T00:09:24.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1350999999999998e-05 2024-09-16T00:09:24.652 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.652 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.652 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.652 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:1617 3.36 e61/46 3:6fb28598:::1000000094d.00000000:head v 61'29, mlcod=61'27)", 2024-09-16T00:09:24.653 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.251467+0000", 2024-09-16T00:09:24.653 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.341039318, 2024-09-16T00:09:24.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040699999999999998, 2024-09-16T00:09:24.653 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.653 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.653 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.653 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251467+0000", 2024-09-16T00:09:24.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251467+0000", 2024-09-16T00:09:24.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251468+0000", 2024-09-16T00:09:24.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6400000000000001e-07 2024-09-16T00:09:24.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251478+0000", 2024-09-16T00:09:24.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0057e-05 2024-09-16T00:09:24.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251489+0000", 2024-09-16T00:09:24.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0992000000000001e-05 2024-09-16T00:09:24.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251494+0000", 2024-09-16T00:09:24.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8999999999999997e-06 2024-09-16T00:09:24.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251537+0000", 2024-09-16T00:09:24.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3513999999999997e-05 2024-09-16T00:09:24.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251550+0000", 2024-09-16T00:09:24.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3083e-05 2024-09-16T00:09:24.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255500+0000", 2024-09-16T00:09:24.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039494090000000001 2024-09-16T00:09:24.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255537+0000", 2024-09-16T00:09:24.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280999999999998e-05 2024-09-16T00:09:24.656 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.657 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.657 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.657 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:1618 3.1c 3:3b08c232:::1000000094e.00000000:head [write 0~2553] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T00:09:24.657 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.252928+0000", 2024-09-16T00:09:24.657 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.339578176, 2024-09-16T00:09:24.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030458360000000001, 2024-09-16T00:09:24.657 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.657 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.657 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.657 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:24.657 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:24.657 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:24.658 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1618 2024-09-16T00:09:24.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.658 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252928+0000", 2024-09-16T00:09:24.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252928+0000", 2024-09-16T00:09:24.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252929+0000", 2024-09-16T00:09:24.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.13e-06 2024-09-16T00:09:24.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252939+0000", 2024-09-16T00:09:24.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8239999999999995e-06 2024-09-16T00:09:24.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252950+0000", 2024-09-16T00:09:24.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0251999999999999e-05 2024-09-16T00:09:24.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.252962+0000", 2024-09-16T00:09:24.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2332e-05 2024-09-16T00:09:24.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253005+0000", 2024-09-16T00:09:24.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3380000000000001e-05 2024-09-16T00:09:24.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253084+0000", 2024-09-16T00:09:24.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9097000000000005e-05 2024-09-16T00:09:24.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:24.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.253162+0000", 2024-09-16T00:09:24.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7890000000000001e-05 2024-09-16T00:09:24.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:24.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255878+0000", 2024-09-16T00:09:24.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027157589999999999 2024-09-16T00:09:24.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:24.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255892+0000", 2024-09-16T00:09:24.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4329e-05 2024-09-16T00:09:24.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255940+0000", 2024-09-16T00:09:24.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7979000000000003e-05 2024-09-16T00:09:24.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255974+0000", 2024-09-16T00:09:24.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3864e-05 2024-09-16T00:09:24.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.663 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.663 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.663 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:84 2.17 e61/19 2:ee44d6a7:::10000000261.03e00000:head v 61'1, mlcod=0'0)", 2024-09-16T00:09:24.663 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.077563+0000", 2024-09-16T00:09:24.663 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.514943792, 2024-09-16T00:09:24.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033709450000000002, 2024-09-16T00:09:24.663 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.664 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.664 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.664 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077563+0000", 2024-09-16T00:09:24.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077563+0000", 2024-09-16T00:09:24.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077563+0000", 2024-09-16T00:09:24.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.61e-07 2024-09-16T00:09:24.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077577+0000", 2024-09-16T00:09:24.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3108000000000001e-05 2024-09-16T00:09:24.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077587+0000", 2024-09-16T00:09:24.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0273e-05 2024-09-16T00:09:24.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077593+0000", 2024-09-16T00:09:24.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5209999999999999e-06 2024-09-16T00:09:24.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077641+0000", 2024-09-16T00:09:24.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7579e-05 2024-09-16T00:09:24.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077658+0000", 2024-09-16T00:09:24.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7297999999999999e-05 2024-09-16T00:09:24.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080906+0000", 2024-09-16T00:09:24.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032481760000000002 2024-09-16T00:09:24.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080934+0000", 2024-09-16T00:09:24.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7229e-05 2024-09-16T00:09:24.667 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.667 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.667 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.668 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:80 2.12 2:4a871970:::10000000261.03600000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:24.668 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.077908+0000", 2024-09-16T00:09:24.668 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.514598814999999, 2024-09-16T00:09:24.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030856009999999999, 2024-09-16T00:09:24.668 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.668 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.668 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.668 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077908+0000", 2024-09-16T00:09:24.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077908+0000", 2024-09-16T00:09:24.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077909+0000", 2024-09-16T00:09:24.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5719999999999999e-06 2024-09-16T00:09:24.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077920+0000", 2024-09-16T00:09:24.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0631e-05 2024-09-16T00:09:24.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077930+0000", 2024-09-16T00:09:24.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0180000000000001e-05 2024-09-16T00:09:24.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077937+0000", 2024-09-16T00:09:24.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:09:24.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077993+0000", 2024-09-16T00:09:24.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6174999999999997e-05 2024-09-16T00:09:24.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078106+0000", 2024-09-16T00:09:24.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113133 2024-09-16T00:09:24.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:24.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078218+0000", 2024-09-16T00:09:24.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011114 2024-09-16T00:09:24.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:24.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.079121+0000", 2024-09-16T00:09:24.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00090328499999999998 2024-09-16T00:09:24.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:24.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080928+0000", 2024-09-16T00:09:24.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018072310000000001 2024-09-16T00:09:24.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080967+0000", 2024-09-16T00:09:24.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8982e-05 2024-09-16T00:09:24.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.080993+0000", 2024-09-16T00:09:24.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6157e-05 2024-09-16T00:09:24.673 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.673 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.673 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.673 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:1878 3.3c 3:3f4f0e87:::10000000a5f.00000000:head [write 0~2494] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T00:09:24.673 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.078761+0000", 2024-09-16T00:09:24.674 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.513745883999999, 2024-09-16T00:09:24.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025239199999999998, 2024-09-16T00:09:24.674 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.674 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.674 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.674 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:24.674 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:24.674 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:24.674 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1878 2024-09-16T00:09:24.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.674 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078761+0000", 2024-09-16T00:09:24.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078761+0000", 2024-09-16T00:09:24.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078762+0000", 2024-09-16T00:09:24.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7600000000000006e-07 2024-09-16T00:09:24.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078767+0000", 2024-09-16T00:09:24.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5929999999999996e-06 2024-09-16T00:09:24.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078781+0000", 2024-09-16T00:09:24.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3804e-05 2024-09-16T00:09:24.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078789+0000", 2024-09-16T00:09:24.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6960000000000005e-06 2024-09-16T00:09:24.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078838+0000", 2024-09-16T00:09:24.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9416000000000003e-05 2024-09-16T00:09:24.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078940+0000", 2024-09-16T00:09:24.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101431 2024-09-16T00:09:24.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:24.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.079032+0000", 2024-09-16T00:09:24.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2559999999999995e-05 2024-09-16T00:09:24.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:24.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.079823+0000", 2024-09-16T00:09:24.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00079055300000000005 2024-09-16T00:09:24.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:24.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081201+0000", 2024-09-16T00:09:24.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013780579999999999 2024-09-16T00:09:24.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081247+0000", 2024-09-16T00:09:24.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6213e-05 2024-09-16T00:09:24.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081285+0000", 2024-09-16T00:09:24.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7620000000000002e-05 2024-09-16T00:09:24.679 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.679 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.679 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.680 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:2173 3.30 3:0ff9340b:::10000000bb6.00000000:head [write 0~80] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T00:09:24.680 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:03.053795+0000", 2024-09-16T00:09:24.680 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.538711850999999, 2024-09-16T00:09:24.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020696619999999999, 2024-09-16T00:09:24.680 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.680 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.680 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:24.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:24.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:24.680 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 2173 2024-09-16T00:09:24.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.680 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053795+0000", 2024-09-16T00:09:24.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053795+0000", 2024-09-16T00:09:24.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053796+0000", 2024-09-16T00:09:24.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.257e-06 2024-09-16T00:09:24.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053802+0000", 2024-09-16T00:09:24.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3430000000000001e-06 2024-09-16T00:09:24.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053813+0000", 2024-09-16T00:09:24.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.101e-05 2024-09-16T00:09:24.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053819+0000", 2024-09-16T00:09:24.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0750000000000003e-06 2024-09-16T00:09:24.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053864+0000", 2024-09-16T00:09:24.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4258999999999998e-05 2024-09-16T00:09:24.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.053954+0000", 2024-09-16T00:09:24.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0195e-05 2024-09-16T00:09:24.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:24.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.054034+0000", 2024-09-16T00:09:24.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0319000000000001e-05 2024-09-16T00:09:24.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:24.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.054604+0000", 2024-09-16T00:09:24.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00057022099999999999 2024-09-16T00:09:24.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:24.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.055768+0000", 2024-09-16T00:09:24.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001164113 2024-09-16T00:09:24.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.055821+0000", 2024-09-16T00:09:24.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2364000000000002e-05 2024-09-16T00:09:24.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.055864+0000", 2024-09-16T00:09:24.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3506e-05 2024-09-16T00:09:24.685 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.685 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.685 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.686 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:2315 3.1f e62/33 3:f84a2a2d:::10000000c60.00000000:head v 62'45, mlcod=62'43)", 2024-09-16T00:09:24.686 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:03.440894+0000", 2024-09-16T00:09:24.686 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.151612691, 2024-09-16T00:09:24.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028251109999999999, 2024-09-16T00:09:24.686 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.686 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.686 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.686 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.440894+0000", 2024-09-16T00:09:24.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.440894+0000", 2024-09-16T00:09:24.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.440894+0000", 2024-09-16T00:09:24.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7700000000000004e-07 2024-09-16T00:09:24.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441598+0000", 2024-09-16T00:09:24.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00070308100000000004 2024-09-16T00:09:24.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441607+0000", 2024-09-16T00:09:24.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7489999999999993e-06 2024-09-16T00:09:24.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441612+0000", 2024-09-16T00:09:24.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6519999999999997e-06 2024-09-16T00:09:24.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441650+0000", 2024-09-16T00:09:24.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7867000000000002e-05 2024-09-16T00:09:24.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.441663+0000", 2024-09-16T00:09:24.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2788e-05 2024-09-16T00:09:24.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.443681+0000", 2024-09-16T00:09:24.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020186599999999998 2024-09-16T00:09:24.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.443719+0000", 2024-09-16T00:09:24.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7636999999999999e-05 2024-09-16T00:09:24.690 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.690 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.690 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.690 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:2326 3.1c 3:39be8b5c:::10000000c6c.00000000:head [write 0~173460] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T00:09:24.690 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:03.473107+0000", 2024-09-16T00:09:24.690 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.119399407, 2024-09-16T00:09:24.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001822029, 2024-09-16T00:09:24.690 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.690 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.690 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.691 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:24.691 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:24.691 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:24.691 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 2326 2024-09-16T00:09:24.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.691 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473107+0000", 2024-09-16T00:09:24.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473107+0000", 2024-09-16T00:09:24.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473108+0000", 2024-09-16T00:09:24.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0139999999999999e-06 2024-09-16T00:09:24.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.692 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473292+0000", 2024-09-16T00:09:24.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000184072 2024-09-16T00:09:24.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.692 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473305+0000", 2024-09-16T00:09:24.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2444e-05 2024-09-16T00:09:24.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473310+0000", 2024-09-16T00:09:24.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5720000000000002e-06 2024-09-16T00:09:24.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473353+0000", 2024-09-16T00:09:24.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3041999999999999e-05 2024-09-16T00:09:24.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473455+0000", 2024-09-16T00:09:24.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101297 2024-09-16T00:09:24.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:24.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.473542+0000", 2024-09-16T00:09:24.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7787999999999994e-05 2024-09-16T00:09:24.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:24.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.474469+0000", 2024-09-16T00:09:24.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00092618500000000005 2024-09-16T00:09:24.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:24.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.474868+0000", 2024-09-16T00:09:24.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00039969400000000003 2024-09-16T00:09:24.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.474900+0000", 2024-09-16T00:09:24.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1341999999999999e-05 2024-09-16T00:09:24.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:03.474929+0000", 2024-09-16T00:09:24.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9578999999999999e-05 2024-09-16T00:09:24.696 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.696 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.696 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.696 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:6120 3.37 e69/20 3:ef2e412a:::10000001c11.00000000:head v 69'100, mlcod=68'98)", 2024-09-16T00:09:24.696 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:16.085749+0000", 2024-09-16T00:09:24.696 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.5067572679999994, 2024-09-16T00:09:24.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020186219999999999, 2024-09-16T00:09:24.696 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:24.696 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:24.696 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:24.696 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:24.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:24.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085749+0000", 2024-09-16T00:09:24.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:24.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085749+0000", 2024-09-16T00:09:24.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:24.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:24.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085749+0000", 2024-09-16T00:09:24.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3200000000000001e-07 2024-09-16T00:09:24.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:24.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085767+0000", 2024-09-16T00:09:24.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7212000000000001e-05 2024-09-16T00:09:24.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:24.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085775+0000", 2024-09-16T00:09:24.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8590000000000005e-06 2024-09-16T00:09:24.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:24.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085780+0000", 2024-09-16T00:09:24.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.989e-06 2024-09-16T00:09:24.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:24.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085825+0000", 2024-09-16T00:09:24.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5090999999999999e-05 2024-09-16T00:09:24.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:24.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.085845+0000", 2024-09-16T00:09:24.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0412999999999999e-05 2024-09-16T00:09:24.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:24.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.087730+0000", 2024-09-16T00:09:24.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001884817 2024-09-16T00:09:24.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:24.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:24.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:24.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:16.087768+0000", 2024-09-16T00:09:24.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8009000000000001e-05 2024-09-16T00:09:24.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.700 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.700 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:24.700 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:24.700 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:24.702 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:09:24.702 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:24.700+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:24.802 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:24.801+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:24.863 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:24.873 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:09:24.903 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:24.903+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:25.005 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:25.004+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:25.025 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:25.025 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:25.025 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:25.025 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:25.025 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:25.036 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:09:25.106 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:25.105+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:25.152 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:25.152 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:25.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:25.152 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:25.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.153 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:18 2.14 2:2f55791f:::606.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:25.153 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:09:25.153 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.53475799899999, 2024-09-16T00:09:25.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056100969999999997, 2024-09-16T00:09:25.153 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.153 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.153 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.153 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:09:25.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615346+0000", 2024-09-16T00:09:25.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615352+0000", 2024-09-16T00:09:25.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9859999999999999e-06 2024-09-16T00:09:25.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615363+0000", 2024-09-16T00:09:25.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1168999999999999e-05 2024-09-16T00:09:25.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615382+0000", 2024-09-16T00:09:25.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8224e-05 2024-09-16T00:09:25.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615393+0000", 2024-09-16T00:09:25.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0925e-05 2024-09-16T00:09:25.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615624+0000", 2024-09-16T00:09:25.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023159600000000001 2024-09-16T00:09:25.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615721+0000", 2024-09-16T00:09:25.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7040000000000006e-05 2024-09-16T00:09:25.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:25.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615885+0000", 2024-09-16T00:09:25.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163456 2024-09-16T00:09:25.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618076+0000", 2024-09-16T00:09:25.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002191368 2024-09-16T00:09:25.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620870+0000", 2024-09-16T00:09:25.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941839999999999 2024-09-16T00:09:25.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620922+0000", 2024-09-16T00:09:25.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1641999999999998e-05 2024-09-16T00:09:25.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620956+0000", 2024-09-16T00:09:25.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4507e-05 2024-09-16T00:09:25.160 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.160 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.161 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.161 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:19 2.14 2:2f55791f:::606.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:25.161 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:09:25.161 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.53466764, 2024-09-16T00:09:25.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005695739, 2024-09-16T00:09:25.161 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.161 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.161 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.162 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:09:25.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:09:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615437+0000", 2024-09-16T00:09:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4900000000000005e-07 2024-09-16T00:09:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.163 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615442+0000", 2024-09-16T00:09:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.994e-06 2024-09-16T00:09:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:09:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1081e-05 2024-09-16T00:09:25.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615473+0000", 2024-09-16T00:09:25.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1910000000000006e-06 2024-09-16T00:09:25.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616230+0000", 2024-09-16T00:09:25.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075700400000000003 2024-09-16T00:09:25.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616305+0000", 2024-09-16T00:09:25.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4759999999999996e-05 2024-09-16T00:09:25.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:25.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616466+0000", 2024-09-16T00:09:25.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000161714 2024-09-16T00:09:25.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619821+0000", 2024-09-16T00:09:25.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033547339999999998 2024-09-16T00:09:25.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621028+0000", 2024-09-16T00:09:25.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012067530000000001 2024-09-16T00:09:25.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621087+0000", 2024-09-16T00:09:25.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9364999999999998e-05 2024-09-16T00:09:25.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621132+0000", 2024-09-16T00:09:25.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5293999999999997e-05 2024-09-16T00:09:25.170 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.170 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.170 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.170 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:25.170 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:09:25.170 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.534605408, 2024-09-16T00:09:25.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050565760000000001, 2024-09-16T00:09:25.171 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.171 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.171 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.171 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:09:25.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615499+0000", 2024-09-16T00:09:25.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615500+0000", 2024-09-16T00:09:25.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8199999999999999e-07 2024-09-16T00:09:25.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615505+0000", 2024-09-16T00:09:25.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5280000000000003e-06 2024-09-16T00:09:25.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:25.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0317e-05 2024-09-16T00:09:25.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615523+0000", 2024-09-16T00:09:25.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4580000000000004e-06 2024-09-16T00:09:25.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615607+0000", 2024-09-16T00:09:25.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4153999999999994e-05 2024-09-16T00:09:25.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615748+0000", 2024-09-16T00:09:25.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140533 2024-09-16T00:09:25.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:25.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615911+0000", 2024-09-16T00:09:25.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000163678 2024-09-16T00:09:25.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619944+0000", 2024-09-16T00:09:25.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040330909999999999 2024-09-16T00:09:25.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620471+0000", 2024-09-16T00:09:25.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052645600000000002 2024-09-16T00:09:25.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620512+0000", 2024-09-16T00:09:25.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0738000000000002e-05 2024-09-16T00:09:25.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620555+0000", 2024-09-16T00:09:25.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3841000000000003e-05 2024-09-16T00:09:25.179 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.179 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.179 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.179 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:25.179 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:09:25.180 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.534553301, 2024-09-16T00:09:25.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089162479999999999, 2024-09-16T00:09:25.180 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.180 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.180 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.180 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:09:25.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615551+0000", 2024-09-16T00:09:25.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615552+0000", 2024-09-16T00:09:25.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:09:25.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615556+0000", 2024-09-16T00:09:25.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3610000000000003e-06 2024-09-16T00:09:25.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615566+0000", 2024-09-16T00:09:25.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6099999999999995e-06 2024-09-16T00:09:25.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615574+0000", 2024-09-16T00:09:25.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9379999999999992e-06 2024-09-16T00:09:25.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616777+0000", 2024-09-16T00:09:25.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012031640000000001 2024-09-16T00:09:25.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616882+0000", 2024-09-16T00:09:25.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105146 2024-09-16T00:09:25.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:25.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619696+0000", 2024-09-16T00:09:25.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028142179999999998 2024-09-16T00:09:25.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624234+0000", 2024-09-16T00:09:25.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045381889999999998 2024-09-16T00:09:25.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624393+0000", 2024-09-16T00:09:25.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015871200000000001 2024-09-16T00:09:25.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.186 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.186 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624434+0000", 2024-09-16T00:09:25.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0908999999999997e-05 2024-09-16T00:09:25.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624467+0000", 2024-09-16T00:09:25.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3217999999999999e-05 2024-09-16T00:09:25.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.187 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.187 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.187 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:25.188 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:09:25.188 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.53448350399999, 2024-09-16T00:09:25.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064898359999999997, 2024-09-16T00:09:25.188 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.188 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.188 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.188 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:09:25.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615621+0000", 2024-09-16T00:09:25.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615622+0000", 2024-09-16T00:09:25.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.181e-06 2024-09-16T00:09:25.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615628+0000", 2024-09-16T00:09:25.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2450000000000003e-06 2024-09-16T00:09:25.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:09:25.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2554999999999999e-05 2024-09-16T00:09:25.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615650+0000", 2024-09-16T00:09:25.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4350000000000003e-06 2024-09-16T00:09:25.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616772+0000", 2024-09-16T00:09:25.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001122058 2024-09-16T00:09:25.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616889+0000", 2024-09-16T00:09:25.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116623 2024-09-16T00:09:25.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:25.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617035+0000", 2024-09-16T00:09:25.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014579799999999999 2024-09-16T00:09:25.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619829+0000", 2024-09-16T00:09:25.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027941379999999998 2024-09-16T00:09:25.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622024+0000", 2024-09-16T00:09:25.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021949920000000002 2024-09-16T00:09:25.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622087+0000", 2024-09-16T00:09:25.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3048999999999999e-05 2024-09-16T00:09:25.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622111+0000", 2024-09-16T00:09:25.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3762000000000001e-05 2024-09-16T00:09:25.193 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.193 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.193 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.193 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:25.193 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:09:25.193 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.534420943, 2024-09-16T00:09:25.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065192160000000004, 2024-09-16T00:09:25.194 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.194 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.194 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.194 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:09:25.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615683+0000", 2024-09-16T00:09:25.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615684+0000", 2024-09-16T00:09:25.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8299999999999995e-07 2024-09-16T00:09:25.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615691+0000", 2024-09-16T00:09:25.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.866e-06 2024-09-16T00:09:25.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615707+0000", 2024-09-16T00:09:25.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6073999999999999e-05 2024-09-16T00:09:25.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615714+0000", 2024-09-16T00:09:25.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2470000000000002e-06 2024-09-16T00:09:25.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617403+0000", 2024-09-16T00:09:25.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016881629999999999 2024-09-16T00:09:25.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617559+0000", 2024-09-16T00:09:25.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015669699999999999 2024-09-16T00:09:25.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:25.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.617704+0000", 2024-09-16T00:09:25.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000144165 2024-09-16T00:09:25.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.621413+0000", 2024-09-16T00:09:25.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037092200000000001 2024-09-16T00:09:25.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622147+0000", 2024-09-16T00:09:25.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00073468800000000005 2024-09-16T00:09:25.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.198 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.198 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622180+0000", 2024-09-16T00:09:25.198 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2901999999999998e-05 2024-09-16T00:09:25.198 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.198 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.199 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.199 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622203+0000", 2024-09-16T00:09:25.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2211000000000001e-05 2024-09-16T00:09:25.199 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.199 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.199 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.199 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.199 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.199 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:25.199 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:09:25.199 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.53437784899999, 2024-09-16T00:09:25.199 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065825270000000003, 2024-09-16T00:09:25.199 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.199 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.199 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:09:25.200 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.200 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:09:25.200 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.200 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.200 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.200 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615726+0000", 2024-09-16T00:09:25.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615727+0000", 2024-09-16T00:09:25.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7299999999999996e-07 2024-09-16T00:09:25.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.201 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.201 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.201 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615730+0000", 2024-09-16T00:09:25.201 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.004e-06 2024-09-16T00:09:25.201 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615737+0000", 2024-09-16T00:09:25.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4789999999999999e-06 2024-09-16T00:09:25.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615745+0000", 2024-09-16T00:09:25.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6699999999999994e-06 2024-09-16T00:09:25.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.202 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.202 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.202 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618115+0000", 2024-09-16T00:09:25.202 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023701569999999999 2024-09-16T00:09:25.202 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.618247+0000", 2024-09-16T00:09:25.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013185000000000001 2024-09-16T00:09:25.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.203 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.203 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622309+0000", 2024-09-16T00:09:25.203 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040617939999999996 2024-09-16T00:09:25.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.203 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.203 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.203 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.203 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.204 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:36 2.1a 2:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:25.204 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:09:25.204 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.5343506, 2024-09-16T00:09:25.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036685950000000002, 2024-09-16T00:09:25.204 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.204 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.204 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:09:25.204 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.204 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.204 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:09:25.204 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.204 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.204 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:09:25.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615754+0000", 2024-09-16T00:09:25.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2e-07 2024-09-16T00:09:25.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.205 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.205 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.205 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615763+0000", 2024-09-16T00:09:25.205 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4360000000000002e-06 2024-09-16T00:09:25.205 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615770+0000", 2024-09-16T00:09:25.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7540000000000006e-06 2024-09-16T00:09:25.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615803+0000", 2024-09-16T00:09:25.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3008e-05 2024-09-16T00:09:25.206 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.206 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.206 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.206 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.615888+0000", 2024-09-16T00:09:25.206 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5056999999999996e-05 2024-09-16T00:09:25.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:09:25.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013059700000000001 2024-09-16T00:09:25.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.207 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.207 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.207 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619422+0000", 2024-09-16T00:09:25.207 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034033230000000002 2024-09-16T00:09:25.207 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.207 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.207 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.207 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.208 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:16 2.1d e22/19 2:b871830b:::605.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:25.208 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:09:25.208 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.53036201, 2024-09-16T00:09:25.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045616900000000002, 2024-09-16T00:09:25.208 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.208 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.208 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.208 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.208 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.208 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.208 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:09:25.208 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619742+0000", 2024-09-16T00:09:25.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619746+0000", 2024-09-16T00:09:25.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:09:25.209 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.209 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.209 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.209 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619755+0000", 2024-09-16T00:09:25.209 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3599999999999996e-06 2024-09-16T00:09:25.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619767+0000", 2024-09-16T00:09:25.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2805e-05 2024-09-16T00:09:25.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619787+0000", 2024-09-16T00:09:25.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9264000000000001e-05 2024-09-16T00:09:25.210 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.210 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.210 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619869+0000", 2024-09-16T00:09:25.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2639999999999995e-05 2024-09-16T00:09:25.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.619895+0000", 2024-09-16T00:09:25.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5890000000000001e-05 2024-09-16T00:09:25.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.211 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624264+0000", 2024-09-16T00:09:25.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043690049999999996 2024-09-16T00:09:25.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.211 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624304+0000", 2024-09-16T00:09:25.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9807e-05 2024-09-16T00:09:25.212 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.212 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.212 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.212 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:17 2.1d e22/19 2:b871830b:::605.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:25.212 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:09:25.212 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.53000930100001, 2024-09-16T00:09:25.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042370910000000001, 2024-09-16T00:09:25.212 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.212 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.212 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.213 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:09:25.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620095+0000", 2024-09-16T00:09:25.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620096+0000", 2024-09-16T00:09:25.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:09:25.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620102+0000", 2024-09-16T00:09:25.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7220000000000002e-06 2024-09-16T00:09:25.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620112+0000", 2024-09-16T00:09:25.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0156999999999999e-05 2024-09-16T00:09:25.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620120+0000", 2024-09-16T00:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2880000000000004e-06 2024-09-16T00:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620171+0000", 2024-09-16T00:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.168e-05 2024-09-16T00:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.620189+0000", 2024-09-16T00:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7419e-05 2024-09-16T00:09:25.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624306+0000", 2024-09-16T00:09:25.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041175430000000004 2024-09-16T00:09:25.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.624332+0000", 2024-09-16T00:09:25.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5717000000000001e-05 2024-09-16T00:09:25.216 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.216 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.216 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.217 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:38 2.14 2:2e2fa760:::200.00000001:head [write 0~940] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:09:25.217 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:09:25.217 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 106.527642484, 2024-09-16T00:09:25.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044651079999999998, 2024-09-16T00:09:25.217 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.217 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.217 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.217 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.217 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.217 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:09:25.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622462+0000", 2024-09-16T00:09:25.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622463+0000", 2024-09-16T00:09:25.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5349999999999999e-06 2024-09-16T00:09:25.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622470+0000", 2024-09-16T00:09:25.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7050000000000002e-06 2024-09-16T00:09:25.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622485+0000", 2024-09-16T00:09:25.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4978e-05 2024-09-16T00:09:25.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622493+0000", 2024-09-16T00:09:25.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2139999999999996e-06 2024-09-16T00:09:25.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622546+0000", 2024-09-16T00:09:25.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:09:25.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622595+0000", 2024-09-16T00:09:25.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9112e-05 2024-09-16T00:09:25.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:25.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.622677+0000", 2024-09-16T00:09:25.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2332999999999999e-05 2024-09-16T00:09:25.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.623515+0000", 2024-09-16T00:09:25.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000837615 2024-09-16T00:09:25.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626867+0000", 2024-09-16T00:09:25.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033524819999999999 2024-09-16T00:09:25.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626902+0000", 2024-09-16T00:09:25.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4310000000000002e-05 2024-09-16T00:09:25.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:38.626927+0000", 2024-09-16T00:09:25.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5128999999999999e-05 2024-09-16T00:09:25.222 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.222 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.222 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.222 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:103 3.26 3:6519881c:::1000000025e.00000000:head [write 0~2892] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:25.222 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:56.487113+0000", 2024-09-16T00:09:25.222 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.662991335000001, 2024-09-16T00:09:25.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034171290000000001, 2024-09-16T00:09:25.222 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.222 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.223 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:25.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:25.223 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 103 2024-09-16T00:09:25.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.223 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487113+0000", 2024-09-16T00:09:25.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487113+0000", 2024-09-16T00:09:25.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487114+0000", 2024-09-16T00:09:25.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3030000000000001e-06 2024-09-16T00:09:25.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487122+0000", 2024-09-16T00:09:25.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3889999999999999e-06 2024-09-16T00:09:25.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487133+0000", 2024-09-16T00:09:25.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1325000000000001e-05 2024-09-16T00:09:25.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487138+0000", 2024-09-16T00:09:25.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0660000000000003e-06 2024-09-16T00:09:25.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487182+0000", 2024-09-16T00:09:25.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3655999999999997e-05 2024-09-16T00:09:25.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487307+0000", 2024-09-16T00:09:25.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012541599999999999 2024-09-16T00:09:25.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:25.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487391+0000", 2024-09-16T00:09:25.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4404e-05 2024-09-16T00:09:25.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.487908+0000", 2024-09-16T00:09:25.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000516608 2024-09-16T00:09:25.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.490429+0000", 2024-09-16T00:09:25.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025209339999999998 2024-09-16T00:09:25.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.490482+0000", 2024-09-16T00:09:25.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2655000000000003e-05 2024-09-16T00:09:25.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:56.490530+0000", 2024-09-16T00:09:25.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8372999999999999e-05 2024-09-16T00:09:25.228 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.228 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.228 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.228 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:53 2.14 2:2e2fa760:::200.00000001:head [write 20850~1000165 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e58)", 2024-09-16T00:09:25.228 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:57.452702+0000", 2024-09-16T00:09:25.228 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.697401787, 2024-09-16T00:09:25.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046625019999999998, 2024-09-16T00:09:25.228 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.229 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.229 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.229 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.452702+0000", 2024-09-16T00:09:25.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.452702+0000", 2024-09-16T00:09:25.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.452704+0000", 2024-09-16T00:09:25.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4699999999999999e-06 2024-09-16T00:09:25.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453184+0000", 2024-09-16T00:09:25.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00048001000000000001 2024-09-16T00:09:25.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453199+0000", 2024-09-16T00:09:25.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4905e-05 2024-09-16T00:09:25.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453222+0000", 2024-09-16T00:09:25.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3587999999999998e-05 2024-09-16T00:09:25.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453288+0000", 2024-09-16T00:09:25.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5817999999999999e-05 2024-09-16T00:09:25.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453348+0000", 2024-09-16T00:09:25.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9716999999999998e-05 2024-09-16T00:09:25.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:25.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.453440+0000", 2024-09-16T00:09:25.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2090999999999996e-05 2024-09-16T00:09:25.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.455907+0000", 2024-09-16T00:09:25.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024670730000000002 2024-09-16T00:09:25.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.457293+0000", 2024-09-16T00:09:25.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013861979999999999 2024-09-16T00:09:25.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.457326+0000", 2024-09-16T00:09:25.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2484000000000003e-05 2024-09-16T00:09:25.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:57.457365+0000", 2024-09-16T00:09:25.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9147999999999999e-05 2024-09-16T00:09:25.233 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.233 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.233 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.234 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:55 2.14 2:2e2fa760:::200.00000001:head [write 1021015~941162 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e59)", 2024-09-16T00:09:25.234 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:08:58.642546+0000", 2024-09-16T00:09:25.234 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 26.507558669000002, 2024-09-16T00:09:25.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041709410000000001, 2024-09-16T00:09:25.234 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.234 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.234 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.234 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.642546+0000", 2024-09-16T00:09:25.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.642546+0000", 2024-09-16T00:09:25.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.642546+0000", 2024-09-16T00:09:25.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5499999999999997e-07 2024-09-16T00:09:25.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643041+0000", 2024-09-16T00:09:25.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049419399999999999 2024-09-16T00:09:25.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643051+0000", 2024-09-16T00:09:25.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0229e-05 2024-09-16T00:09:25.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643056+0000", 2024-09-16T00:09:25.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.327e-06 2024-09-16T00:09:25.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643112+0000", 2024-09-16T00:09:25.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5939000000000001e-05 2024-09-16T00:09:25.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643193+0000", 2024-09-16T00:09:25.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0921999999999995e-05 2024-09-16T00:09:25.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:25.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.643305+0000", 2024-09-16T00:09:25.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011162099999999999 2024-09-16T00:09:25.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.645976+0000", 2024-09-16T00:09:25.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026712609999999999 2024-09-16T00:09:25.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.646651+0000", 2024-09-16T00:09:25.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00067499200000000002 2024-09-16T00:09:25.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.646684+0000", 2024-09-16T00:09:25.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3328999999999999e-05 2024-09-16T00:09:25.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:08:58.646717+0000", 2024-09-16T00:09:25.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2271999999999999e-05 2024-09-16T00:09:25.239 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.239 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.239 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.239 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:1617 3.36 3:6fb28598:::1000000094d.00000000:head [write 0~2552] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-09-16T00:09:25.239 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.251180+0000", 2024-09-16T00:09:25.239 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.898924753999999, 2024-09-16T00:09:25.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004599601, 2024-09-16T00:09:25.239 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.240 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.240 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.240 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.240 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:25.240 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:25.240 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1617 2024-09-16T00:09:25.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.240 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251180+0000", 2024-09-16T00:09:25.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251180+0000", 2024-09-16T00:09:25.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251180+0000", 2024-09-16T00:09:25.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4600000000000003e-07 2024-09-16T00:09:25.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251185+0000", 2024-09-16T00:09:25.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3239999999999996e-06 2024-09-16T00:09:25.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251194+0000", 2024-09-16T00:09:25.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3049999999999998e-06 2024-09-16T00:09:25.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251199+0000", 2024-09-16T00:09:25.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2780000000000001e-06 2024-09-16T00:09:25.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251240+0000", 2024-09-16T00:09:25.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1171999999999998e-05 2024-09-16T00:09:25.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251317+0000", 2024-09-16T00:09:25.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6804999999999994e-05 2024-09-16T00:09:25.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:25.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.251399+0000", 2024-09-16T00:09:25.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2101000000000006e-05 2024-09-16T00:09:25.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255497+0000", 2024-09-16T00:09:25.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040972889999999996 2024-09-16T00:09:25.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255687+0000", 2024-09-16T00:09:25.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019032899999999999 2024-09-16T00:09:25.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255739+0000", 2024-09-16T00:09:25.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2086999999999998e-05 2024-09-16T00:09:25.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.255779+0000", 2024-09-16T00:09:25.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0064999999999997e-05 2024-09-16T00:09:25.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.245 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.245 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.245 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:72 2.14 2:2e2fa760:::200.00000001:head [write 3118411~1075893 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:25.246 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.671915+0000", 2024-09-16T00:09:25.246 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.478188819, 2024-09-16T00:09:25.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049035859999999997, 2024-09-16T00:09:25.246 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.246 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.246 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.246 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.671915+0000", 2024-09-16T00:09:25.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.671915+0000", 2024-09-16T00:09:25.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.671917+0000", 2024-09-16T00:09:25.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7e-06 2024-09-16T00:09:25.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672671+0000", 2024-09-16T00:09:25.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00075343199999999995 2024-09-16T00:09:25.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672686+0000", 2024-09-16T00:09:25.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4960000000000001e-05 2024-09-16T00:09:25.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672693+0000", 2024-09-16T00:09:25.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0770000000000002e-06 2024-09-16T00:09:25.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672737+0000", 2024-09-16T00:09:25.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4614999999999999e-05 2024-09-16T00:09:25.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672793+0000", 2024-09-16T00:09:25.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5785999999999997e-05 2024-09-16T00:09:25.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:25.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.672870+0000", 2024-09-16T00:09:25.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7187999999999993e-05 2024-09-16T00:09:25.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.675448+0000", 2024-09-16T00:09:25.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002578033 2024-09-16T00:09:25.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.676746+0000", 2024-09-16T00:09:25.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001297896 2024-09-16T00:09:25.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.676778+0000", 2024-09-16T00:09:25.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1605e-05 2024-09-16T00:09:25.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.676819+0000", 2024-09-16T00:09:25.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1294000000000001e-05 2024-09-16T00:09:25.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.251 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.251 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.251 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:75 2.11 2:8fcca0a5:::10000000261.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:25.251 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:01.985646+0000", 2024-09-16T00:09:25.251 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.164458413999998, 2024-09-16T00:09:25.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041697660000000001, 2024-09-16T00:09:25.251 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.252 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.252 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.252 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985646+0000", 2024-09-16T00:09:25.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985646+0000", 2024-09-16T00:09:25.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985648+0000", 2024-09-16T00:09:25.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5739999999999998e-06 2024-09-16T00:09:25.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985699+0000", 2024-09-16T00:09:25.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0513999999999998e-05 2024-09-16T00:09:25.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985718+0000", 2024-09-16T00:09:25.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8689999999999999e-05 2024-09-16T00:09:25.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985729+0000", 2024-09-16T00:09:25.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1812e-05 2024-09-16T00:09:25.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985802+0000", 2024-09-16T00:09:25.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2465999999999994e-05 2024-09-16T00:09:25.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.985912+0000", 2024-09-16T00:09:25.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109795 2024-09-16T00:09:25.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:25.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.986236+0000", 2024-09-16T00:09:25.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00032388799999999999 2024-09-16T00:09:25.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.987505+0000", 2024-09-16T00:09:25.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001269788 2024-09-16T00:09:25.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.989708+0000", 2024-09-16T00:09:25.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022022579999999999 2024-09-16T00:09:25.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.989765+0000", 2024-09-16T00:09:25.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7268000000000001e-05 2024-09-16T00:09:25.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:01.989816+0000", 2024-09-16T00:09:25.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0713e-05 2024-09-16T00:09:25.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.257 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.257 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:84 2.17 2:ee44d6a7:::10000000261.03e00000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:25.257 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:02.076954+0000", 2024-09-16T00:09:25.257 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.073150227999999, 2024-09-16T00:09:25.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004155237, 2024-09-16T00:09:25.257 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.257 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.257 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.257 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076954+0000", 2024-09-16T00:09:25.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076954+0000", 2024-09-16T00:09:25.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076955+0000", 2024-09-16T00:09:25.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4199999999999996e-07 2024-09-16T00:09:25.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076971+0000", 2024-09-16T00:09:25.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6294999999999999e-05 2024-09-16T00:09:25.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076980+0000", 2024-09-16T00:09:25.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5599999999999999e-06 2024-09-16T00:09:25.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.076986+0000", 2024-09-16T00:09:25.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9329999999999998e-06 2024-09-16T00:09:25.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077032+0000", 2024-09-16T00:09:25.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5566999999999999e-05 2024-09-16T00:09:25.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077143+0000", 2024-09-16T00:09:25.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011068 2024-09-16T00:09:25.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:25.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.077295+0000", 2024-09-16T00:09:25.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152578 2024-09-16T00:09:25.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.078459+0000", 2024-09-16T00:09:25.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001163645 2024-09-16T00:09:25.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081048+0000", 2024-09-16T00:09:25.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025893840000000001 2024-09-16T00:09:25.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081084+0000", 2024-09-16T00:09:25.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5834999999999998e-05 2024-09-16T00:09:25.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:02.081109+0000", 2024-09-16T00:09:25.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5218000000000001e-05 2024-09-16T00:09:25.263 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.263 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.263 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.263 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:224 2.1a 2:5e41ab69:::200.00000005:head [write 0~1943310 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e66)", 2024-09-16T00:09:25.263 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:12.641517+0000", 2024-09-16T00:09:25.263 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.50858715, 2024-09-16T00:09:25.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064053460000000001, 2024-09-16T00:09:25.263 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.263 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.263 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.264 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.641517+0000", 2024-09-16T00:09:25.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.641517+0000", 2024-09-16T00:09:25.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.641520+0000", 2024-09-16T00:09:25.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5660000000000001e-06 2024-09-16T00:09:25.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.642832+0000", 2024-09-16T00:09:25.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00131274 2024-09-16T00:09:25.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.642853+0000", 2024-09-16T00:09:25.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0710999999999999e-05 2024-09-16T00:09:25.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.642880+0000", 2024-09-16T00:09:25.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6693999999999999e-05 2024-09-16T00:09:25.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.642939+0000", 2024-09-16T00:09:25.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9184000000000001e-05 2024-09-16T00:09:25.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.643032+0000", 2024-09-16T00:09:25.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2583000000000004e-05 2024-09-16T00:09:25.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:25.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.643114+0000", 2024-09-16T00:09:25.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2554999999999998e-05 2024-09-16T00:09:25.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.647381+0000", 2024-09-16T00:09:25.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042663969999999999 2024-09-16T00:09:25.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.647813+0000", 2024-09-16T00:09:25.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00043280899999999998 2024-09-16T00:09:25.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.647873+0000", 2024-09-16T00:09:25.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9619e-05 2024-09-16T00:09:25.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:12.647922+0000", 2024-09-16T00:09:25.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9487999999999998e-05 2024-09-16T00:09:25.268 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.269 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.269 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.269 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:327 2.1d e73/46 2:bec1251c:::200.00000007:head v 73'7, mlcod=63'5)", 2024-09-16T00:09:25.269 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:20.724139+0000", 2024-09-16T00:09:25.269 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4259652960000002, 2024-09-16T00:09:25.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049608899999999999, 2024-09-16T00:09:25.269 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.269 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.269 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.269 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.724139+0000", 2024-09-16T00:09:25.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.724139+0000", 2024-09-16T00:09:25.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.724141+0000", 2024-09-16T00:09:25.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7850000000000001e-06 2024-09-16T00:09:25.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.725369+0000", 2024-09-16T00:09:25.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001227978 2024-09-16T00:09:25.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.725383+0000", 2024-09-16T00:09:25.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3839000000000001e-05 2024-09-16T00:09:25.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.725391+0000", 2024-09-16T00:09:25.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5939999999999994e-06 2024-09-16T00:09:25.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.725469+0000", 2024-09-16T00:09:25.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7404999999999995e-05 2024-09-16T00:09:25.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.725487+0000", 2024-09-16T00:09:25.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8766e-05 2024-09-16T00:09:25.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.729063+0000", 2024-09-16T00:09:25.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035752900000000001 2024-09-16T00:09:25.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:20.729100+0000", 2024-09-16T00:09:25.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7233000000000001e-05 2024-09-16T00:09:25.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.273 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.273 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.273 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:25.275 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:09:25.275 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:25.273+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:25.375 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:25.374+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:25.424 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:25.424 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:25.424 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:25.424 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:25.433 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:09:25.476 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:25.475+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:25.577 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:25.575+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:25.596 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:25.596 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:25.597 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:25.597 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:25.597 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:25.606 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:09:25.690 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:25.689+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:25.771 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:25.772 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:25.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:25.772 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:25.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.772 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:25.772 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:25.772 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 122.688545398, 2024-09-16T00:09:25.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:09:25.772 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.772 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.773 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.773 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:09:25.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.773 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:25.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:25.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:09:25.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:09:25.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:09:25.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:09:25.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:09:25.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:09:25.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:09:25.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:09:25.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:09:25.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:09:25.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:09:25.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:09:25.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:09:25.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:09:25.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:09:25.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:09:25.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:09:25.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:09:25.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:09:25.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:09:25.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:09:25.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:09:25.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:09:25.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:09:25.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:09:25.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:09:25.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:09:25.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.779 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.780 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:25.780 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:09:25.780 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.609013768, 2024-09-16T00:09:25.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093071190000000008, 2024-09-16T00:09:25.780 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.780 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.780 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.781 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.781 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 106 2024-09-16T00:09:25.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.781 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:09:25.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:09:25.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160362+0000", 2024-09-16T00:09:25.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:09:25.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160438+0000", 2024-09-16T00:09:25.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6711000000000005e-05 2024-09-16T00:09:25.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160444+0000", 2024-09-16T00:09:25.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0530000000000003e-06 2024-09-16T00:09:25.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160449+0000", 2024-09-16T00:09:25.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2189999999999998e-06 2024-09-16T00:09:25.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161270+0000", 2024-09-16T00:09:25.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082099199999999999 2024-09-16T00:09:25.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161348+0000", 2024-09-16T00:09:25.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8361000000000004e-05 2024-09-16T00:09:25.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161472+0000", 2024-09-16T00:09:25.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123523 2024-09-16T00:09:25.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163461+0000", 2024-09-16T00:09:25.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019891560000000002 2024-09-16T00:09:25.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169418+0000", 2024-09-16T00:09:25.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059568490000000002 2024-09-16T00:09:25.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169570+0000", 2024-09-16T00:09:25.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152461 2024-09-16T00:09:25.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169668+0000", 2024-09-16T00:09:25.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T00:09:25.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.786 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.786 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:25.786 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:09:25.786 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.60891799700001, 2024-09-16T00:09:25.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092931679999999992, 2024-09-16T00:09:25.786 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.786 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.786 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.786 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.787 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.787 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.787 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107 2024-09-16T00:09:25.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.787 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:09:25.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:09:25.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:09:25.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0699999999999998e-07 2024-09-16T00:09:25.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160459+0000", 2024-09-16T00:09:25.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0370000000000001e-06 2024-09-16T00:09:25.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160463+0000", 2024-09-16T00:09:25.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999999e-06 2024-09-16T00:09:25.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160466+0000", 2024-09-16T00:09:25.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.644e-06 2024-09-16T00:09:25.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161873+0000", 2024-09-16T00:09:25.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014074840000000001 2024-09-16T00:09:25.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161949+0000", 2024-09-16T00:09:25.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5749999999999998e-05 2024-09-16T00:09:25.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162069+0000", 2024-09-16T00:09:25.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120257 2024-09-16T00:09:25.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163506+0000", 2024-09-16T00:09:25.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001436344 2024-09-16T00:09:25.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169704+0000", 2024-09-16T00:09:25.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061982829999999997 2024-09-16T00:09:25.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169730+0000", 2024-09-16T00:09:25.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6060000000000001e-05 2024-09-16T00:09:25.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169750+0000", 2024-09-16T00:09:25.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0092e-05 2024-09-16T00:09:25.791 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.792 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.792 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.792 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:25.792 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:09:25.792 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.608808685, 2024-09-16T00:09:25.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092280039999999997, 2024-09-16T00:09:25.792 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.792 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.792 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.792 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.793 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.793 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 108 2024-09-16T00:09:25.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.793 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:09:25.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:09:25.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160567+0000", 2024-09-16T00:09:25.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7300000000000001e-07 2024-09-16T00:09:25.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160569+0000", 2024-09-16T00:09:25.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5299999999999999e-06 2024-09-16T00:09:25.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160574+0000", 2024-09-16T00:09:25.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-16T00:09:25.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160577+0000", 2024-09-16T00:09:25.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.01e-06 2024-09-16T00:09:25.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162518+0000", 2024-09-16T00:09:25.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001940703 2024-09-16T00:09:25.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162571+0000", 2024-09-16T00:09:25.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2670000000000002e-05 2024-09-16T00:09:25.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162647+0000", 2024-09-16T00:09:25.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6778000000000002e-05 2024-09-16T00:09:25.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163544+0000", 2024-09-16T00:09:25.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089702599999999999 2024-09-16T00:09:25.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169754+0000", 2024-09-16T00:09:25.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062091739999999996 2024-09-16T00:09:25.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169776+0000", 2024-09-16T00:09:25.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2391999999999998e-05 2024-09-16T00:09:25.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169794+0000", 2024-09-16T00:09:25.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8318999999999999e-05 2024-09-16T00:09:25.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.798 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.798 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.798 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:25.798 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:09:25.798 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.608650786, 2024-09-16T00:09:25.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092871919999999997, 2024-09-16T00:09:25.798 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.798 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.798 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.798 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.799 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.799 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.799 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 110 2024-09-16T00:09:25.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.799 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:09:25.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:09:25.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:09:25.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:09:25.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160728+0000", 2024-09-16T00:09:25.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0620000000000001e-06 2024-09-16T00:09:25.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160731+0000", 2024-09-16T00:09:25.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6500000000000002e-06 2024-09-16T00:09:25.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160734+0000", 2024-09-16T00:09:25.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.762e-06 2024-09-16T00:09:25.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163580+0000", 2024-09-16T00:09:25.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028464549999999999 2024-09-16T00:09:25.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163657+0000", 2024-09-16T00:09:25.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6695999999999999e-05 2024-09-16T00:09:25.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163798+0000", 2024-09-16T00:09:25.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014055 2024-09-16T00:09:25.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165483+0000", 2024-09-16T00:09:25.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016853090000000001 2024-09-16T00:09:25.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169970+0000", 2024-09-16T00:09:25.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00448709 2024-09-16T00:09:25.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169994+0000", 2024-09-16T00:09:25.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3575999999999999e-05 2024-09-16T00:09:25.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170011+0000", 2024-09-16T00:09:25.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7702000000000001e-05 2024-09-16T00:09:25.805 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.805 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.805 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.805 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:25.805 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:09:25.805 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.608637117, 2024-09-16T00:09:25.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093174859999999998, 2024-09-16T00:09:25.805 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.806 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.806 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.806 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.806 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.806 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.806 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 111 2024-09-16T00:09:25.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.807 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:09:25.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:09:25.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:09:25.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1e-07 2024-09-16T00:09:25.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160741+0000", 2024-09-16T00:09:25.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6110000000000001e-06 2024-09-16T00:09:25.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160744+0000", 2024-09-16T00:09:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3299999999999999e-06 2024-09-16T00:09:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160746+0000", 2024-09-16T00:09:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5469999999999999e-06 2024-09-16T00:09:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164069+0000", 2024-09-16T00:09:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033222709999999999 2024-09-16T00:09:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164113+0000", 2024-09-16T00:09:25.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4212999999999998e-05 2024-09-16T00:09:25.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164187+0000", 2024-09-16T00:09:25.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4136999999999998e-05 2024-09-16T00:09:25.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166429+0000", 2024-09-16T00:09:25.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022423180000000001 2024-09-16T00:09:25.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170016+0000", 2024-09-16T00:09:25.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035869449999999998 2024-09-16T00:09:25.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170038+0000", 2024-09-16T00:09:25.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2053e-05 2024-09-16T00:09:25.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170055+0000", 2024-09-16T00:09:25.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6850999999999998e-05 2024-09-16T00:09:25.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.812 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.812 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.812 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:25.812 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:09:25.812 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.60862340200001, 2024-09-16T00:09:25.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093480279999999995, 2024-09-16T00:09:25.813 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.813 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.813 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.813 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.813 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 112 2024-09-16T00:09:25.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.813 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:09:25.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:09:25.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160752+0000", 2024-09-16T00:09:25.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2700000000000001e-07 2024-09-16T00:09:25.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160808+0000", 2024-09-16T00:09:25.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5797999999999997e-05 2024-09-16T00:09:25.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160814+0000", 2024-09-16T00:09:25.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5980000000000002e-06 2024-09-16T00:09:25.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160819+0000", 2024-09-16T00:09:25.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8960000000000002e-06 2024-09-16T00:09:25.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164468+0000", 2024-09-16T00:09:25.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003649476 2024-09-16T00:09:25.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164525+0000", 2024-09-16T00:09:25.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6227999999999998e-05 2024-09-16T00:09:25.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164595+0000", 2024-09-16T00:09:25.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759000000000005e-05 2024-09-16T00:09:25.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167237+0000", 2024-09-16T00:09:25.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026410470000000001 2024-09-16T00:09:25.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170060+0000", 2024-09-16T00:09:25.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028234969999999999 2024-09-16T00:09:25.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170081+0000", 2024-09-16T00:09:25.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1242000000000001e-05 2024-09-16T00:09:25.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170100+0000", 2024-09-16T00:09:25.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8260000000000001e-05 2024-09-16T00:09:25.820 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.821 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.821 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:25.821 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:09:25.821 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.608546307, 2024-09-16T00:09:25.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093139409999999992, 2024-09-16T00:09:25.821 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.821 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.821 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.821 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.821 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.822 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 113 2024-09-16T00:09:25.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.822 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:09:25.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:09:25.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:09:25.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-07 2024-09-16T00:09:25.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160833+0000", 2024-09-16T00:09:25.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.557e-06 2024-09-16T00:09:25.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160839+0000", 2024-09-16T00:09:25.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3829999999999997e-06 2024-09-16T00:09:25.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160844+0000", 2024-09-16T00:09:25.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6909999999999996e-06 2024-09-16T00:09:25.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165008+0000", 2024-09-16T00:09:25.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041642129999999999 2024-09-16T00:09:25.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165085+0000", 2024-09-16T00:09:25.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6651999999999997e-05 2024-09-16T00:09:25.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165205+0000", 2024-09-16T00:09:25.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012045900000000001 2024-09-16T00:09:25.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167283+0000", 2024-09-16T00:09:25.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020782909999999999 2024-09-16T00:09:25.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170104+0000", 2024-09-16T00:09:25.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028207950000000001 2024-09-16T00:09:25.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170125+0000", 2024-09-16T00:09:25.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0852e-05 2024-09-16T00:09:25.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170143+0000", 2024-09-16T00:09:25.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7467000000000001e-05 2024-09-16T00:09:25.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.828 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.828 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:25.828 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:09:25.828 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.608499825, 2024-09-16T00:09:25.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093126960000000005, 2024-09-16T00:09:25.829 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.829 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.829 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.829 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-16T00:09:25.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.829 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:09:25.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:09:25.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160876+0000", 2024-09-16T00:09:25.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5700000000000002e-07 2024-09-16T00:09:25.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160880+0000", 2024-09-16T00:09:25.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5539999999999999e-06 2024-09-16T00:09:25.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160888+0000", 2024-09-16T00:09:25.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.157e-06 2024-09-16T00:09:25.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160894+0000", 2024-09-16T00:09:25.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7690000000000002e-06 2024-09-16T00:09:25.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165519+0000", 2024-09-16T00:09:25.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046249539999999997 2024-09-16T00:09:25.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165593+0000", 2024-09-16T00:09:25.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4039e-05 2024-09-16T00:09:25.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165715+0000", 2024-09-16T00:09:25.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012131899999999999 2024-09-16T00:09:25.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167309+0000", 2024-09-16T00:09:25.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001594011 2024-09-16T00:09:25.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170147+0000", 2024-09-16T00:09:25.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028387130000000001 2024-09-16T00:09:25.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170170+0000", 2024-09-16T00:09:25.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2517000000000001e-05 2024-09-16T00:09:25.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170188+0000", 2024-09-16T00:09:25.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8006e-05 2024-09-16T00:09:25.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.835 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.835 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.835 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:25.836 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:09:25.836 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.608458797, 2024-09-16T00:09:25.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093140040000000007, 2024-09-16T00:09:25.836 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.836 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.836 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.836 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.836 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.836 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.836 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-16T00:09:25.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.837 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:09:25.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:09:25.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160917+0000", 2024-09-16T00:09:25.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7599999999999996e-07 2024-09-16T00:09:25.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160992+0000", 2024-09-16T00:09:25.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4760999999999998e-05 2024-09-16T00:09:25.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161000+0000", 2024-09-16T00:09:25.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9829999999999996e-06 2024-09-16T00:09:25.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161007+0000", 2024-09-16T00:09:25.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4909999999999999e-06 2024-09-16T00:09:25.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165983+0000", 2024-09-16T00:09:25.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049755399999999997 2024-09-16T00:09:25.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166027+0000", 2024-09-16T00:09:25.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4109000000000001e-05 2024-09-16T00:09:25.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166100+0000", 2024-09-16T00:09:25.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3360000000000002e-05 2024-09-16T00:09:25.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169698+0000", 2024-09-16T00:09:25.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597795 2024-09-16T00:09:25.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170192+0000", 2024-09-16T00:09:25.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049447300000000005 2024-09-16T00:09:25.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170214+0000", 2024-09-16T00:09:25.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1546e-05 2024-09-16T00:09:25.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170230+0000", 2024-09-16T00:09:25.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6169999999999999e-05 2024-09-16T00:09:25.842 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.842 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.842 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.842 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:25.842 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:09:25.842 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.608349068, 2024-09-16T00:09:25.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092457029999999992, 2024-09-16T00:09:25.843 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.843 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.843 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.843 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.843 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-16T00:09:25.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.843 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:09:25.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:09:25.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:09:25.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7899999999999999e-07 2024-09-16T00:09:25.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161029+0000", 2024-09-16T00:09:25.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6630000000000002e-06 2024-09-16T00:09:25.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161036+0000", 2024-09-16T00:09:25.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T00:09:25.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161041+0000", 2024-09-16T00:09:25.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:09:25.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166468+0000", 2024-09-16T00:09:25.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054266050000000001 2024-09-16T00:09:25.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166511+0000", 2024-09-16T00:09:25.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3869999999999998e-05 2024-09-16T00:09:25.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166606+0000", 2024-09-16T00:09:25.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4437000000000004e-05 2024-09-16T00:09:25.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169955+0000", 2024-09-16T00:09:25.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033494810000000001 2024-09-16T00:09:25.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170234+0000", 2024-09-16T00:09:25.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027900899999999998 2024-09-16T00:09:25.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170255+0000", 2024-09-16T00:09:25.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0537999999999999e-05 2024-09-16T00:09:25.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170272+0000", 2024-09-16T00:09:25.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6611000000000001e-05 2024-09-16T00:09:25.849 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.850 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.850 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:25.850 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:09:25.850 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.608323923, 2024-09-16T00:09:25.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093511010000000006, 2024-09-16T00:09:25.851 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.851 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.851 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.851 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.851 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-16T00:09:25.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.852 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:09:25.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:09:25.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:09:25.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.63e-07 2024-09-16T00:09:25.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161055+0000", 2024-09-16T00:09:25.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6770000000000001e-06 2024-09-16T00:09:25.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161063+0000", 2024-09-16T00:09:25.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6469999999999998e-06 2024-09-16T00:09:25.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161068+0000", 2024-09-16T00:09:25.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.746e-06 2024-09-16T00:09:25.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166817+0000", 2024-09-16T00:09:25.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057494959999999998 2024-09-16T00:09:25.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166893+0000", 2024-09-16T00:09:25.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6210000000000004e-05 2024-09-16T00:09:25.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167019+0000", 2024-09-16T00:09:25.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125585 2024-09-16T00:09:25.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170275+0000", 2024-09-16T00:09:25.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032563620000000001 2024-09-16T00:09:25.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170309+0000", 2024-09-16T00:09:25.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3865999999999997e-05 2024-09-16T00:09:25.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170362+0000", 2024-09-16T00:09:25.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3094999999999999e-05 2024-09-16T00:09:25.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170402+0000", 2024-09-16T00:09:25.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9953999999999997e-05 2024-09-16T00:09:25.858 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.858 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.858 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.859 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:25.859 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:09:25.859 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.159591838, 2024-09-16T00:09:25.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097364389999999995, 2024-09-16T00:09:25.859 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.859 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.859 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.860 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138 2024-09-16T00:09:25.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.860 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:09:25.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:09:25.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609784+0000", 2024-09-16T00:09:25.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T00:09:25.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609787+0000", 2024-09-16T00:09:25.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280000000000001e-06 2024-09-16T00:09:25.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609794+0000", 2024-09-16T00:09:25.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6409999999999996e-06 2024-09-16T00:09:25.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609799+0000", 2024-09-16T00:09:25.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9359999999999998e-06 2024-09-16T00:09:25.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615759+0000", 2024-09-16T00:09:25.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059599910000000004 2024-09-16T00:09:25.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615806+0000", 2024-09-16T00:09:25.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7290000000000003e-05 2024-09-16T00:09:25.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615877+0000", 2024-09-16T00:09:25.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1000000000000005e-05 2024-09-16T00:09:25.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618961+0000", 2024-09-16T00:09:25.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030841390000000001 2024-09-16T00:09:25.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619436+0000", 2024-09-16T00:09:25.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047452799999999999 2024-09-16T00:09:25.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619482+0000", 2024-09-16T00:09:25.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5689000000000003e-05 2024-09-16T00:09:25.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619519+0000", 2024-09-16T00:09:25.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7938000000000002e-05 2024-09-16T00:09:25.866 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.866 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.866 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.866 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:25.867 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:09:25.867 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.159568124, 2024-09-16T00:09:25.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011812954000000001, 2024-09-16T00:09:25.867 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.867 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.867 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.867 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.867 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.867 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.867 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139 2024-09-16T00:09:25.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.867 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:09:25.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:09:25.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:09:25.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999998e-07 2024-09-16T00:09:25.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609810+0000", 2024-09-16T00:09:25.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2100000000000002e-06 2024-09-16T00:09:25.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609817+0000", 2024-09-16T00:09:25.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.28e-06 2024-09-16T00:09:25.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609825+0000", 2024-09-16T00:09:25.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:09:25.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616116+0000", 2024-09-16T00:09:25.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006290543 2024-09-16T00:09:25.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616187+0000", 2024-09-16T00:09:25.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1830999999999997e-05 2024-09-16T00:09:25.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616298+0000", 2024-09-16T00:09:25.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011044699999999999 2024-09-16T00:09:25.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619559+0000", 2024-09-16T00:09:25.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032614710000000002 2024-09-16T00:09:25.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621562+0000", 2024-09-16T00:09:25.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020028680000000001 2024-09-16T00:09:25.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621597+0000", 2024-09-16T00:09:25.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4625000000000001e-05 2024-09-16T00:09:25.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621620+0000", 2024-09-16T00:09:25.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2864e-05 2024-09-16T00:09:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:09:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.15954032800001, 2024-09-16T00:09:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011828211999999999, 2024-09-16T00:09:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 140 2024-09-16T00:09:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:09:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:09:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:09:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T00:09:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961000000000003e-05 2024-09-16T00:09:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609892+0000", 2024-09-16T00:09:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0516000000000001e-05 2024-09-16T00:09:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609898+0000", 2024-09-16T00:09:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0449999999999997e-06 2024-09-16T00:09:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616797+0000", 2024-09-16T00:09:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068998840000000002 2024-09-16T00:09:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616844+0000", 2024-09-16T00:09:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6501000000000001e-05 2024-09-16T00:09:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616935+0000", 2024-09-16T00:09:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1222000000000002e-05 2024-09-16T00:09:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619808+0000", 2024-09-16T00:09:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873198 2024-09-16T00:09:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621624+0000", 2024-09-16T00:09:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001815933 2024-09-16T00:09:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621646+0000", 2024-09-16T00:09:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1622999999999999e-05 2024-09-16T00:09:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621663+0000", 2024-09-16T00:09:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6854000000000002e-05 2024-09-16T00:09:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 121.15945451899999, 2024-09-16T00:09:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014331501999999999, 2024-09-16T00:09:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-16T00:09:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609921+0000", 2024-09-16T00:09:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9599999999999999e-07 2024-09-16T00:09:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609925+0000", 2024-09-16T00:09:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T00:09:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609932+0000", 2024-09-16T00:09:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8240000000000002e-06 2024-09-16T00:09:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609937+0000", 2024-09-16T00:09:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1070000000000004e-06 2024-09-16T00:09:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618987+0000", 2024-09-16T00:09:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090502719999999998 2024-09-16T00:09:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619064+0000", 2024-09-16T00:09:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6868999999999999e-05 2024-09-16T00:09:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619183+0000", 2024-09-16T00:09:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011961 2024-09-16T00:09:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.620052+0000", 2024-09-16T00:09:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086822799999999997 2024-09-16T00:09:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624197+0000", 2024-09-16T00:09:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041454930000000001 2024-09-16T00:09:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624229+0000", 2024-09-16T00:09:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1758999999999999e-05 2024-09-16T00:09:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624252+0000", 2024-09-16T00:09:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:09:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:09:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 120.686688711, 2024-09-16T00:09:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010093276999999999, 2024-09-16T00:09:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 157 2024-09-16T00:09:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:09:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:09:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:09:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4699999999999998e-07 2024-09-16T00:09:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082688+0000", 2024-09-16T00:09:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0209999999999998e-06 2024-09-16T00:09:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082692+0000", 2024-09-16T00:09:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6890000000000002e-06 2024-09-16T00:09:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082695+0000", 2024-09-16T00:09:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5339999999999998e-06 2024-09-16T00:09:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085303+0000", 2024-09-16T00:09:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026084789999999999 2024-09-16T00:09:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085374+0000", 2024-09-16T00:09:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1317999999999997e-05 2024-09-16T00:09:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085506+0000", 2024-09-16T00:09:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013118 2024-09-16T00:09:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086928+0000", 2024-09-16T00:09:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014224559999999999 2024-09-16T00:09:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092665+0000", 2024-09-16T00:09:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057369830000000002 2024-09-16T00:09:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092727+0000", 2024-09-16T00:09:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1810000000000006e-05 2024-09-16T00:09:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092779+0000", 2024-09-16T00:09:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2559999999999998e-05 2024-09-16T00:09:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:09:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 120.686676543, 2024-09-16T00:09:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010176388, 2024-09-16T00:09:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 158 2024-09-16T00:09:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:09:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:09:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082699+0000", 2024-09-16T00:09:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.11e-07 2024-09-16T00:09:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082742+0000", 2024-09-16T00:09:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3699000000000003e-05 2024-09-16T00:09:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082748+0000", 2024-09-16T00:09:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8690000000000001e-06 2024-09-16T00:09:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082751+0000", 2024-09-16T00:09:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.942e-06 2024-09-16T00:09:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085881+0000", 2024-09-16T00:09:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031303450000000001 2024-09-16T00:09:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085959+0000", 2024-09-16T00:09:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7874999999999996e-05 2024-09-16T00:09:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086086+0000", 2024-09-16T00:09:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012685499999999999 2024-09-16T00:09:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087127+0000", 2024-09-16T00:09:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001041298 2024-09-16T00:09:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092790+0000", 2024-09-16T00:09:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056628 2024-09-16T00:09:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092832+0000", 2024-09-16T00:09:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2227000000000001e-05 2024-09-16T00:09:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092875+0000", 2024-09-16T00:09:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2267e-05 2024-09-16T00:09:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:09:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 120.686618505, 2024-09-16T00:09:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010199547, 2024-09-16T00:09:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159 2024-09-16T00:09:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:09:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:09:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082757+0000", 2024-09-16T00:09:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2500000000000001e-07 2024-09-16T00:09:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082759+0000", 2024-09-16T00:09:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-16T00:09:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082763+0000", 2024-09-16T00:09:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4889999999999997e-06 2024-09-16T00:09:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082766+0000", 2024-09-16T00:09:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:09:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086571+0000", 2024-09-16T00:09:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038053269999999998 2024-09-16T00:09:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086619+0000", 2024-09-16T00:09:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7924999999999999e-05 2024-09-16T00:09:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086693+0000", 2024-09-16T00:09:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000005e-05 2024-09-16T00:09:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087383+0000", 2024-09-16T00:09:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069008299999999997 2024-09-16T00:09:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092882+0000", 2024-09-16T00:09:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054987450000000002 2024-09-16T00:09:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092921+0000", 2024-09-16T00:09:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9388999999999998e-05 2024-09-16T00:09:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092956+0000", 2024-09-16T00:09:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4647999999999998e-05 2024-09-16T00:09:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:161 2.b 2:d5c7a900:::200.00000003:head [write 0~3935733 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e64)", 2024-09-16T00:09:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:07.538495+0000", 2024-09-16T00:09:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.230879864999999, 2024-09-16T00:09:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012844983000000001, 2024-09-16T00:09:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.538495+0000", 2024-09-16T00:09:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:25.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.538495+0000", 2024-09-16T00:09:25.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:25.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:25.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.538497+0000", 2024-09-16T00:09:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5149999999999999e-06 2024-09-16T00:09:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540617+0000", 2024-09-16T00:09:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021204079999999998 2024-09-16T00:09:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540640+0000", 2024-09-16T00:09:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3167e-05 2024-09-16T00:09:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540650+0000", 2024-09-16T00:09:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6099999999999995e-06 2024-09-16T00:09:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540724+0000", 2024-09-16T00:09:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4340999999999999e-05 2024-09-16T00:09:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540812+0000", 2024-09-16T00:09:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8376999999999998e-05 2024-09-16T00:09:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540957+0000", 2024-09-16T00:09:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014463 2024-09-16T00:09:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.550270+0000", 2024-09-16T00:09:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093128380000000004 2024-09-16T00:09:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551270+0000", 2024-09-16T00:09:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00100015 2024-09-16T00:09:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551309+0000", 2024-09-16T00:09:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8526999999999999e-05 2024-09-16T00:09:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551340+0000", 2024-09-16T00:09:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1420000000000001e-05 2024-09-16T00:09:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:25.913 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:25.914 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:25.913+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:25.915 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:09:26.015 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:26.014+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:26.033 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:26.033 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:26.033 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:26.033 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:26.043 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:09:26.116 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:26.116+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:26.149 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T00:09:26.165 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:26.165 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:26.165 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:26.165 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:26.165 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:26.176 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:09:26.218 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:26.216+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:26.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.264+0000 7fec5db4b640 1 Processor -- start 2024-09-16T00:09:26.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.264+0000 7fec5db4b640 1 -- start start 2024-09-16T00:09:26.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.265+0000 7fec5db4b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fec580c9430 0x7fec58163560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.265+0000 7fec5db4b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec580703e0 con 0x7fec580ce810 2024-09-16T00:09:26.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.265+0000 7fec5db4b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec58070680 con 0x7fec580ce430 2024-09-16T00:09:26.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.265+0000 7fec5db4b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fec580c5750 con 0x7fec580c9430 2024-09-16T00:09:26.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.265+0000 7fec56ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fec580ce810 0x7fec580c5be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:41706/0 (socket says 172.21.15.139:41706) 2024-09-16T00:09:26.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.265+0000 7fec56ffd640 1 -- 172.21.15.139:0/3971514109 learned_addr learned my addr 172.21.15.139:0/3971514109 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:26.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.265+0000 7fec57fff640 1 --2- 172.21.15.139:0/3971514109 >> v2:172.21.15.164:3300/0 conn(0x7fec580c9430 0x7fec58163560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.266+0000 7fec567fc640 1 -- 172.21.15.139:0/3971514109 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3600780970 0 0) 0x7fec580703e0 con 0x7fec580ce810 2024-09-16T00:09:26.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.266+0000 7fec567fc640 1 -- 172.21.15.139:0/3971514109 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec40003520 con 0x7fec580ce810 2024-09-16T00:09:26.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.266+0000 7fec567fc640 1 -- 172.21.15.139:0/3971514109 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 801517835 0 0) 0x7fec58070680 con 0x7fec580ce430 2024-09-16T00:09:26.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.266+0000 7fec567fc640 1 -- 172.21.15.139:0/3971514109 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec580703e0 con 0x7fec580ce430 2024-09-16T00:09:26.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.266+0000 7fec57fff640 1 -- 172.21.15.139:0/3971514109 >> v1:172.21.15.164:6789/0 conn(0x7fec580ce430 legacy=0x7fec580b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.266+0000 7fec57fff640 1 -- 172.21.15.139:0/3971514109 >> v1:172.21.15.139:6789/0 conn(0x7fec580ce810 legacy=0x7fec580c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.266+0000 7fec57fff640 1 -- 172.21.15.139:0/3971514109 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec58070920 con 0x7fec580c9430 2024-09-16T00:09:26.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.267+0000 7fec57fff640 1 --2- 172.21.15.139:0/3971514109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec580c9430 0x7fec58163560 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fec50009d90 tx=0x7fec5002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=476dd40dd9438444 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:26.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.267+0000 7fec567fc640 1 -- 172.21.15.139:0/3971514109 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec50007740 con 0x7fec580c9430 2024-09-16T00:09:26.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.267+0000 7fec567fc640 1 -- 172.21.15.139:0/3971514109 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec500078e0 con 0x7fec580c9430 2024-09-16T00:09:26.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.267+0000 7fec567fc640 1 -- 172.21.15.139:0/3971514109 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec50007c90 con 0x7fec580c9430 2024-09-16T00:09:26.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.268+0000 7fec5db4b640 1 -- 172.21.15.139:0/3971514109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec580c9430 msgr2=0x7fec58163560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.268+0000 7fec5db4b640 1 --2- 172.21.15.139:0/3971514109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec580c9430 0x7fec58163560 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fec50009d90 tx=0x7fec5002a6f0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.268+0000 7fec5db4b640 1 -- 172.21.15.139:0/3971514109 shutdown_connections 2024-09-16T00:09:26.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.268+0000 7fec5db4b640 1 --2- 172.21.15.139:0/3971514109 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec580c9430 0x7fec58163560 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.268+0000 7fec5db4b640 1 -- 172.21.15.139:0/3971514109 >> 172.21.15.139:0/3971514109 conn(0x7fec5806c150 msgr2=0x7fec58067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:26.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.268+0000 7fec5db4b640 1 -- 172.21.15.139:0/3971514109 shutdown_connections 2024-09-16T00:09:26.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.268+0000 7fec5db4b640 1 -- 172.21.15.139:0/3971514109 wait complete. 2024-09-16T00:09:26.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.269+0000 7fec5db4b640 1 Processor -- start 2024-09-16T00:09:26.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.269+0000 7fec5db4b640 1 -- start start 2024-09-16T00:09:26.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.270+0000 7fec5db4b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec580c9430 0x7fec58156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.270+0000 7fec5db4b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec580ce430 0x7fec580c2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.270+0000 7fec5db4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec580ce810 0x7fec580c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.270+0000 7fec5db4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fec5806ffc0 con 0x7fec580ce810 2024-09-16T00:09:26.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.270+0000 7fec5db4b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fec58070ee0 con 0x7fec580ce430 2024-09-16T00:09:26.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.270+0000 7fec5db4b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fec58070140 con 0x7fec580c9430 2024-09-16T00:09:26.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.270+0000 7fec57fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec580ce810 0x7fec580c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.270+0000 7fec56ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec580ce430 0x7fec580c2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.270+0000 7fec577fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec580c9430 0x7fec58156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.270+0000 7fec56ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec580ce430 0x7fec580c2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:47770/0 (socket says 172.21.15.139:47770) 2024-09-16T00:09:26.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.270+0000 7fec56ffd640 1 -- 172.21.15.139:0/1286311527 learned_addr learned my addr 172.21.15.139:0/1286311527 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:26.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.271+0000 7fec577fe640 1 -- 172.21.15.139:0/1286311527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec580ce430 msgr2=0x7fec580c2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.271+0000 7fec577fe640 1 --2- 172.21.15.139:0/1286311527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec580ce430 0x7fec580c2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.271+0000 7fec577fe640 1 -- 172.21.15.139:0/1286311527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec580ce810 msgr2=0x7fec580c3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.271+0000 7fec577fe640 1 --2- 172.21.15.139:0/1286311527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec580ce810 0x7fec580c3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.271+0000 7fec577fe640 1 -- 172.21.15.139:0/1286311527 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec58152f60 con 0x7fec580c9430 2024-09-16T00:09:26.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.271+0000 7fec56ffd640 1 --2- 172.21.15.139:0/1286311527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec580ce430 0x7fec580c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:26.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.271+0000 7fec577fe640 1 --2- 172.21.15.139:0/1286311527 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec580c9430 0x7fec58156850 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fec4800c3c0 tx=0x7fec4800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:26.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.271+0000 7fec3ffff640 1 -- 172.21.15.139:0/1286311527 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48017070 con 0x7fec580c9430 2024-09-16T00:09:26.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.271+0000 7fec3ffff640 1 -- 172.21.15.139:0/1286311527 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec480052d0 con 0x7fec580c9430 2024-09-16T00:09:26.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.271+0000 7fec3ffff640 1 -- 172.21.15.139:0/1286311527 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48003830 con 0x7fec580c9430 2024-09-16T00:09:26.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.271+0000 7fec5db4b640 1 -- 172.21.15.139:0/1286311527 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5815d140 con 0x7fec580c9430 2024-09-16T00:09:26.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.271+0000 7fec5db4b640 1 -- 172.21.15.139:0/1286311527 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fec58152c80 con 0x7fec580c9430 2024-09-16T00:09:26.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.273+0000 7fec5db4b640 1 -- 172.21.15.139:0/1286311527 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec58072480 con 0x7fec580c9430 2024-09-16T00:09:26.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.273+0000 7fec3ffff640 1 -- 172.21.15.139:0/1286311527 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fec4801a070 con 0x7fec580c9430 2024-09-16T00:09:26.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.274+0000 7fec3ffff640 1 --2- 172.21.15.139:0/1286311527 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fec4406c2b0 0x7fec4406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.274+0000 7fec3ffff640 1 -- 172.21.15.139:0/1286311527 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(75..75 src has 1..75) ==== 5944+0+0 (secure 0 0 0) 0x7fec4808a740 con 0x7fec580c9430 2024-09-16T00:09:26.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.274+0000 7fec56ffd640 1 --2- 172.21.15.139:0/1286311527 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fec4406c2b0 0x7fec4406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.275+0000 7fec56ffd640 1 --2- 172.21.15.139:0/1286311527 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fec4406c2b0 0x7fec4406e770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fec5814a9c0 tx=0x7fec4c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:26.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.278+0000 7fec3ffff640 1 -- 172.21.15.139:0/1286311527 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fec58072480 con 0x7fec580c9430 2024-09-16T00:09:26.318 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:26.317+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:26.341 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:26.341 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:26.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:26.341 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:26.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.341 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'66)", 2024-09-16T00:09:26.341 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:09:26.341 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 122.17315594900001, 2024-09-16T00:09:26.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032925509999999999, 2024-09-16T00:09:26.342 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.342 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.342 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.342 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:09:26.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:09:26.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166177+0000", 2024-09-16T00:09:26.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:09:26.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166216+0000", 2024-09-16T00:09:26.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9008000000000003e-05 2024-09-16T00:09:26.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.344 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166226+0000", 2024-09-16T00:09:26.344 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0081999999999999e-05 2024-09-16T00:09:26.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.344 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.344 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.344 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166230+0000", 2024-09-16T00:09:26.344 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5079999999999999e-06 2024-09-16T00:09:26.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.344 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.344 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.344 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166259+0000", 2024-09-16T00:09:26.344 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8928000000000002e-05 2024-09-16T00:09:26.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166271+0000", 2024-09-16T00:09:26.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1531e-05 2024-09-16T00:09:26.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169449+0000", 2024-09-16T00:09:26.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003178143 2024-09-16T00:09:26.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169468+0000", 2024-09-16T00:09:26.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9700000000000001e-05 2024-09-16T00:09:26.346 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.346 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.346 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.346 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'66)", 2024-09-16T00:09:26.346 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:09:26.347 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 122.17269449, 2024-09-16T00:09:26.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003179374, 2024-09-16T00:09:26.347 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.347 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.347 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.347 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:09:26.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:09:26.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166638+0000", 2024-09-16T00:09:26.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2300000000000002e-07 2024-09-16T00:09:26.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166641+0000", 2024-09-16T00:09:26.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0079999999999998e-06 2024-09-16T00:09:26.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166646+0000", 2024-09-16T00:09:26.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4970000000000001e-06 2024-09-16T00:09:26.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.350 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166650+0000", 2024-09-16T00:09:26.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5729999999999999e-06 2024-09-16T00:09:26.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.350 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166682+0000", 2024-09-16T00:09:26.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2218999999999998e-05 2024-09-16T00:09:26.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.350 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166694+0000", 2024-09-16T00:09:26.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1902999999999999e-05 2024-09-16T00:09:26.351 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.351 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.351 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.351 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169784+0000", 2024-09-16T00:09:26.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003089545 2024-09-16T00:09:26.351 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.351 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.351 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.351 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169817+0000", 2024-09-16T00:09:26.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3206e-05 2024-09-16T00:09:26.351 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.351 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.351 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.351 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.352 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:117 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'79, mlcod=17'66)", 2024-09-16T00:09:26.352 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:09:26.352 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 122.17225043000001, 2024-09-16T00:09:26.352 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00275326, 2024-09-16T00:09:26.352 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.352 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.352 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.352 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.352 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.352 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:09:26.352 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167081+0000", 2024-09-16T00:09:26.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167082+0000", 2024-09-16T00:09:26.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6499999999999999e-07 2024-09-16T00:09:26.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167086+0000", 2024-09-16T00:09:26.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1559999999999997e-06 2024-09-16T00:09:26.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167096+0000", 2024-09-16T00:09:26.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8390000000000002e-06 2024-09-16T00:09:26.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167101+0000", 2024-09-16T00:09:26.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5789999999999997e-06 2024-09-16T00:09:26.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167130+0000", 2024-09-16T00:09:26.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9649e-05 2024-09-16T00:09:26.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.167142+0000", 2024-09-16T00:09:26.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1435e-05 2024-09-16T00:09:26.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169818+0000", 2024-09-16T00:09:26.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002676056 2024-09-16T00:09:26.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169835+0000", 2024-09-16T00:09:26.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6980999999999999e-05 2024-09-16T00:09:26.356 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.356 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.356 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.356 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:130 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'88, mlcod=17'85)", 2024-09-16T00:09:26.357 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:26.357 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 121.729451101, 2024-09-16T00:09:26.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002880456, 2024-09-16T00:09:26.357 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.357 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.357 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.357 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:26.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:26.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:26.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5699999999999998e-07 2024-09-16T00:09:26.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609885+0000", 2024-09-16T00:09:26.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.608e-06 2024-09-16T00:09:26.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609891+0000", 2024-09-16T00:09:26.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:09:26.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609895+0000", 2024-09-16T00:09:26.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.54e-06 2024-09-16T00:09:26.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609931+0000", 2024-09-16T00:09:26.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5893999999999999e-05 2024-09-16T00:09:26.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609945+0000", 2024-09-16T00:09:26.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4259e-05 2024-09-16T00:09:26.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612721+0000", 2024-09-16T00:09:26.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027757849999999998 2024-09-16T00:09:26.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612761+0000", 2024-09-16T00:09:26.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0638e-05 2024-09-16T00:09:26.361 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.361 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.361 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.362 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-16T00:09:26.362 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:09:26.362 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 121.729067128, 2024-09-16T00:09:26.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028016529999999999, 2024-09-16T00:09:26.362 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.362 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.362 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.362 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:09:26.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:09:26.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:09:26.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.63e-07 2024-09-16T00:09:26.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.364 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.364 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610344+0000", 2024-09-16T00:09:26.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8996000000000001e-05 2024-09-16T00:09:26.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.364 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.364 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610352+0000", 2024-09-16T00:09:26.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:09:26.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610356+0000", 2024-09-16T00:09:26.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0790000000000002e-06 2024-09-16T00:09:26.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610388+0000", 2024-09-16T00:09:26.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1541000000000002e-05 2024-09-16T00:09:26.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610400+0000", 2024-09-16T00:09:26.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2075e-05 2024-09-16T00:09:26.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613045+0000", 2024-09-16T00:09:26.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002645213 2024-09-16T00:09:26.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613066+0000", 2024-09-16T00:09:26.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1353e-05 2024-09-16T00:09:26.366 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.367 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.367 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.367 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:139 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'97, mlcod=18'87)", 2024-09-16T00:09:26.367 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:09:26.367 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 121.72298877999999, 2024-09-16T00:09:26.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030908139999999999, 2024-09-16T00:09:26.367 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.367 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.367 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.367 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:09:26.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:09:26.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616344+0000", 2024-09-16T00:09:26.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3000000000000001e-07 2024-09-16T00:09:26.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616347+0000", 2024-09-16T00:09:26.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7740000000000002e-06 2024-09-16T00:09:26.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616355+0000", 2024-09-16T00:09:26.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4699999999999996e-06 2024-09-16T00:09:26.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616358+0000", 2024-09-16T00:09:26.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5899999999999999e-06 2024-09-16T00:09:26.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616393+0000", 2024-09-16T00:09:26.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4529000000000001e-05 2024-09-16T00:09:26.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616404+0000", 2024-09-16T00:09:26.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0726e-05 2024-09-16T00:09:26.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619411+0000", 2024-09-16T00:09:26.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030076249999999999 2024-09-16T00:09:26.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619434+0000", 2024-09-16T00:09:26.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-05 2024-09-16T00:09:26.372 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.372 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.372 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.373 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:26.373 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:09:26.373 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 107.723429147, 2024-09-16T00:09:26.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061593860000000002, 2024-09-16T00:09:26.373 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.373 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.373 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.373 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:09:26.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:09:26.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615907+0000", 2024-09-16T00:09:26.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9389999999999999e-06 2024-09-16T00:09:26.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615920+0000", 2024-09-16T00:09:26.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2989999999999999e-05 2024-09-16T00:09:26.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615931+0000", 2024-09-16T00:09:26.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.164e-05 2024-09-16T00:09:26.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615939+0000", 2024-09-16T00:09:26.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8029999999999997e-06 2024-09-16T00:09:26.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616310+0000", 2024-09-16T00:09:26.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00037111400000000003 2024-09-16T00:09:26.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616406+0000", 2024-09-16T00:09:26.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5372999999999996e-05 2024-09-16T00:09:26.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:26.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616514+0000", 2024-09-16T00:09:26.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108564 2024-09-16T00:09:26.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:26.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618158+0000", 2024-09-16T00:09:26.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016442449999999999 2024-09-16T00:09:26.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:26.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621972+0000", 2024-09-16T00:09:26.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038132299999999999 2024-09-16T00:09:26.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622018+0000", 2024-09-16T00:09:26.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6239000000000002e-05 2024-09-16T00:09:26.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622062+0000", 2024-09-16T00:09:26.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4249000000000003e-05 2024-09-16T00:09:26.379 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.379 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.379 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.379 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:26.379 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:09:26.379 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 107.723375679, 2024-09-16T00:09:26.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006258914, 2024-09-16T00:09:26.380 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.380 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.380 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.380 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:09:26.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:09:26.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615957+0000", 2024-09-16T00:09:26.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1399999999999997e-07 2024-09-16T00:09:26.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615960+0000", 2024-09-16T00:09:26.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0860000000000002e-06 2024-09-16T00:09:26.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615967+0000", 2024-09-16T00:09:26.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9059999999999996e-06 2024-09-16T00:09:26.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615972+0000", 2024-09-16T00:09:26.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5679999999999999e-06 2024-09-16T00:09:26.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616717+0000", 2024-09-16T00:09:26.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074438 2024-09-16T00:09:26.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616781+0000", 2024-09-16T00:09:26.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4808999999999996e-05 2024-09-16T00:09:26.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:26.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616885+0000", 2024-09-16T00:09:26.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010404900000000001 2024-09-16T00:09:26.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:26.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:09:26.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012773870000000001 2024-09-16T00:09:26.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:26.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622116+0000", 2024-09-16T00:09:26.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039533470000000003 2024-09-16T00:09:26.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622173+0000", 2024-09-16T00:09:26.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6347000000000001e-05 2024-09-16T00:09:26.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622215+0000", 2024-09-16T00:09:26.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2521000000000002e-05 2024-09-16T00:09:26.385 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.386 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.386 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.386 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:26.386 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:26.386 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 107.72335194199999, 2024-09-16T00:09:26.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041725529999999999, 2024-09-16T00:09:26.386 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.386 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.386 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.386 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:26.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:26.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:26.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8799999999999998e-07 2024-09-16T00:09:26.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615990+0000", 2024-09-16T00:09:26.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2150000000000007e-06 2024-09-16T00:09:26.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:09:26.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7159999999999997e-06 2024-09-16T00:09:26.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616001+0000", 2024-09-16T00:09:26.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9339999999999997e-06 2024-09-16T00:09:26.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616293+0000", 2024-09-16T00:09:26.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029210899999999998 2024-09-16T00:09:26.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616401+0000", 2024-09-16T00:09:26.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108087 2024-09-16T00:09:26.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:26.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616561+0000", 2024-09-16T00:09:26.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015940600000000001 2024-09-16T00:09:26.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:26.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618146+0000", 2024-09-16T00:09:26.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015857040000000001 2024-09-16T00:09:26.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:26.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620091+0000", 2024-09-16T00:09:26.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001944837 2024-09-16T00:09:26.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620126+0000", 2024-09-16T00:09:26.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4242999999999997e-05 2024-09-16T00:09:26.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620152+0000", 2024-09-16T00:09:26.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6914e-05 2024-09-16T00:09:26.392 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.392 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.392 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.392 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:26.392 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:26.393 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 107.723313717, 2024-09-16T00:09:26.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00630628, 2024-09-16T00:09:26.393 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.393 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.393 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.393 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:26.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:26.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:09:26.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5999999999999999e-07 2024-09-16T00:09:26.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616022+0000", 2024-09-16T00:09:26.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0149999999999999e-06 2024-09-16T00:09:26.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616031+0000", 2024-09-16T00:09:26.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9770000000000006e-06 2024-09-16T00:09:26.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616036+0000", 2024-09-16T00:09:26.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2639999999999999e-06 2024-09-16T00:09:26.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617057+0000", 2024-09-16T00:09:26.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010212999999999999 2024-09-16T00:09:26.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617146+0000", 2024-09-16T00:09:26.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9234000000000007e-05 2024-09-16T00:09:26.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:26.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617232+0000", 2024-09-16T00:09:26.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5218999999999998e-05 2024-09-16T00:09:26.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:26.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619107+0000", 2024-09-16T00:09:26.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001875016 2024-09-16T00:09:26.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:26.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622259+0000", 2024-09-16T00:09:26.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031527930000000001 2024-09-16T00:09:26.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622290+0000", 2024-09-16T00:09:26.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1022999999999998e-05 2024-09-16T00:09:26.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622324+0000", 2024-09-16T00:09:26.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3979000000000001e-05 2024-09-16T00:09:26.398 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.398 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.398 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.398 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:26.398 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:09:26.399 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 107.723288415, 2024-09-16T00:09:26.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042412739999999997, 2024-09-16T00:09:26.399 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.399 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.399 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.399 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:09:26.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:09:26.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616044+0000", 2024-09-16T00:09:26.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:09:26.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616052+0000", 2024-09-16T00:09:26.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.439e-06 2024-09-16T00:09:26.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616059+0000", 2024-09-16T00:09:26.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1389999999999998e-06 2024-09-16T00:09:26.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616065+0000", 2024-09-16T00:09:26.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:09:26.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616823+0000", 2024-09-16T00:09:26.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075781199999999996 2024-09-16T00:09:26.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616918+0000", 2024-09-16T00:09:26.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5174999999999996e-05 2024-09-16T00:09:26.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:26.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617019+0000", 2024-09-16T00:09:26.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101161 2024-09-16T00:09:26.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:26.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618503+0000", 2024-09-16T00:09:26.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014836750000000001 2024-09-16T00:09:26.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:26.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620201+0000", 2024-09-16T00:09:26.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00169805 2024-09-16T00:09:26.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620255+0000", 2024-09-16T00:09:26.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4499000000000001e-05 2024-09-16T00:09:26.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620285+0000", 2024-09-16T00:09:26.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9349e-05 2024-09-16T00:09:26.404 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.404 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.405 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.405 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:26.405 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:09:26.405 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 107.72325942400001, 2024-09-16T00:09:26.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063682799999999996, 2024-09-16T00:09:26.405 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.405 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.405 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:09:26.405 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:09:26.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:09:26.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:09:26.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9900000000000001e-07 2024-09-16T00:09:26.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616079+0000", 2024-09-16T00:09:26.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9009999999999999e-06 2024-09-16T00:09:26.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616086+0000", 2024-09-16T00:09:26.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.776e-06 2024-09-16T00:09:26.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616091+0000", 2024-09-16T00:09:26.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5600000000000001e-06 2024-09-16T00:09:26.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618339+0000", 2024-09-16T00:09:26.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022476179999999998 2024-09-16T00:09:26.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:09:26.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9025999999999997e-05 2024-09-16T00:09:26.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622441+0000", 2024-09-16T00:09:26.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040029999999999996 2024-09-16T00:09:26.409 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.409 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.409 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.410 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:221 3.1b 3:dad7a251:::100000002d7.00000000:head [write 0~30412] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:26.410 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:09:26.410 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 29.590051234000001, 2024-09-16T00:09:26.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034396769999999999, 2024-09-16T00:09:26.410 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.410 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.410 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.410 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:26.410 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:26.410 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:26.410 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 221 2024-09-16T00:09:26.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.411 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:09:26.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:09:26.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749282+0000", 2024-09-16T00:09:26.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3769999999999999e-06 2024-09-16T00:09:26.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749300+0000", 2024-09-16T00:09:26.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8204999999999999e-05 2024-09-16T00:09:26.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749311+0000", 2024-09-16T00:09:26.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0599e-05 2024-09-16T00:09:26.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749318+0000", 2024-09-16T00:09:26.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9789999999999996e-06 2024-09-16T00:09:26.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749367+0000", 2024-09-16T00:09:26.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9382999999999998e-05 2024-09-16T00:09:26.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749450+0000", 2024-09-16T00:09:26.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-05 2024-09-16T00:09:26.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:26.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749526+0000", 2024-09-16T00:09:26.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5564999999999999e-05 2024-09-16T00:09:26.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:26.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.750601+0000", 2024-09-16T00:09:26.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010751199999999999 2024-09-16T00:09:26.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:26.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752665+0000", 2024-09-16T00:09:26.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00206442 2024-09-16T00:09:26.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752696+0000", 2024-09-16T00:09:26.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0476000000000001e-05 2024-09-16T00:09:26.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752720+0000", 2024-09-16T00:09:26.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4482999999999999e-05 2024-09-16T00:09:26.416 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.417 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.417 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.417 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:323 3.3 e58/20 3:c319d237:::10000000354.00000000:head v 58'3, mlcod=57'1)", 2024-09-16T00:09:26.417 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:09:26.417 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 29.306671463000001, 2024-09-16T00:09:26.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034011000000000002, 2024-09-16T00:09:26.417 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.417 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.417 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.417 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:09:26.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:09:26.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032661+0000", 2024-09-16T00:09:26.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7000000000000003e-07 2024-09-16T00:09:26.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032670+0000", 2024-09-16T00:09:26.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.602e-06 2024-09-16T00:09:26.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032682+0000", 2024-09-16T00:09:26.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1754e-05 2024-09-16T00:09:26.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032688+0000", 2024-09-16T00:09:26.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8140000000000003e-06 2024-09-16T00:09:26.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032740+0000", 2024-09-16T00:09:26.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1981000000000003e-05 2024-09-16T00:09:26.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032765+0000", 2024-09-16T00:09:26.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4097e-05 2024-09-16T00:09:26.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.036023+0000", 2024-09-16T00:09:26.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032585240000000001 2024-09-16T00:09:26.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.036061+0000", 2024-09-16T00:09:26.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8457999999999997e-05 2024-09-16T00:09:26.421 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.421 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.421 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.422 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:664 3.39 e59/20 3:9c9d3e51:::100000004f6.00000000:head v 59'7, mlcod=58'5)", 2024-09-16T00:09:26.422 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:26.422 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 28.119002303999999, 2024-09-16T00:09:26.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035253329999999999, 2024-09-16T00:09:26.422 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.422 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.422 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.422 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:26.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:26.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:26.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2399999999999996e-07 2024-09-16T00:09:26.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220338+0000", 2024-09-16T00:09:26.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0120000000000005e-06 2024-09-16T00:09:26.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220353+0000", 2024-09-16T00:09:26.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4718999999999999e-05 2024-09-16T00:09:26.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220361+0000", 2024-09-16T00:09:26.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6559999999999992e-06 2024-09-16T00:09:26.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220422+0000", 2024-09-16T00:09:26.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.071e-05 2024-09-16T00:09:26.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220446+0000", 2024-09-16T00:09:26.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4556e-05 2024-09-16T00:09:26.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.223813+0000", 2024-09-16T00:09:26.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033666439999999998 2024-09-16T00:09:26.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.223855+0000", 2024-09-16T00:09:26.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2111999999999999e-05 2024-09-16T00:09:26.427 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.427 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.427 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.427 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:681 3.17 3:ea55746f:::1000000050b.00000000:head [write 0~1071] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:26.427 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:26.427 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 28.074259339000001, 2024-09-16T00:09:26.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036588620000000001, 2024-09-16T00:09:26.427 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.427 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.427 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.427 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:26.428 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:26.428 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:26.428 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 681 2024-09-16T00:09:26.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.428 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:26.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:26.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:26.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7600000000000006e-07 2024-09-16T00:09:26.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265080+0000", 2024-09-16T00:09:26.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8759999999999999e-06 2024-09-16T00:09:26.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265094+0000", 2024-09-16T00:09:26.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.411e-05 2024-09-16T00:09:26.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265102+0000", 2024-09-16T00:09:26.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9640000000000003e-06 2024-09-16T00:09:26.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265156+0000", 2024-09-16T00:09:26.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3375000000000003e-05 2024-09-16T00:09:26.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265229+0000", 2024-09-16T00:09:26.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3147999999999998e-05 2024-09-16T00:09:26.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:26.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265304+0000", 2024-09-16T00:09:26.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4568999999999997e-05 2024-09-16T00:09:26.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:26.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265747+0000", 2024-09-16T00:09:26.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00044363099999999999 2024-09-16T00:09:26.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:26.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.268635+0000", 2024-09-16T00:09:26.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028873459999999998 2024-09-16T00:09:26.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.268688+0000", 2024-09-16T00:09:26.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3155999999999997e-05 2024-09-16T00:09:26.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.268731+0000", 2024-09-16T00:09:26.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3711000000000002e-05 2024-09-16T00:09:26.433 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.433 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.433 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.433 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:3099 3.6 3:617936f3:::10000000f78.00000000:head [write 0~1641] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T00:09:26.433 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:06.096600+0000", 2024-09-16T00:09:26.433 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 20.242732157999999, 2024-09-16T00:09:26.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031984349999999999, 2024-09-16T00:09:26.434 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.434 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.434 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.434 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:26.434 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:26.434 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:26.434 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 3099 2024-09-16T00:09:26.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.434 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096600+0000", 2024-09-16T00:09:26.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096600+0000", 2024-09-16T00:09:26.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096602+0000", 2024-09-16T00:09:26.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4889999999999998e-06 2024-09-16T00:09:26.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096612+0000", 2024-09-16T00:09:26.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5629999999999998e-06 2024-09-16T00:09:26.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096631+0000", 2024-09-16T00:09:26.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9369999999999999e-05 2024-09-16T00:09:26.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096641+0000", 2024-09-16T00:09:26.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0098999999999999e-05 2024-09-16T00:09:26.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096705+0000", 2024-09-16T00:09:26.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3551000000000002e-05 2024-09-16T00:09:26.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op must wait for map", 2024-09-16T00:09:26.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096711+0000", 2024-09-16T00:09:26.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5660000000000003e-06 2024-09-16T00:09:26.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.098401+0000", 2024-09-16T00:09:26.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016899840000000001 2024-09-16T00:09:26.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.098514+0000", 2024-09-16T00:09:26.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000112551 2024-09-16T00:09:26.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:26.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.098636+0000", 2024-09-16T00:09:26.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000122488 2024-09-16T00:09:26.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:26.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.099270+0000", 2024-09-16T00:09:26.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00063404300000000004 2024-09-16T00:09:26.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:26.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.099702+0000", 2024-09-16T00:09:26.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000431956 2024-09-16T00:09:26.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.099761+0000", 2024-09-16T00:09:26.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8421000000000002e-05 2024-09-16T00:09:26.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.099798+0000", 2024-09-16T00:09:26.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7354000000000002e-05 2024-09-16T00:09:26.441 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.441 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.441 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.441 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:219 2.1f 2:f83d65d8:::200.00000004:head [write 863658~2915703 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e65)", 2024-09-16T00:09:26.441 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:26.441 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 15.468392077000001, 2024-09-16T00:09:26.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013772627000000001, 2024-09-16T00:09:26.441 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.441 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.441 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.441 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:26.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:26.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870941+0000", 2024-09-16T00:09:26.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4e-07 2024-09-16T00:09:26.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877134+0000", 2024-09-16T00:09:26.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061936769999999999 2024-09-16T00:09:26.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877153+0000", 2024-09-16T00:09:26.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8408000000000001e-05 2024-09-16T00:09:26.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877161+0000", 2024-09-16T00:09:26.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2709999999999992e-06 2024-09-16T00:09:26.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877225+0000", 2024-09-16T00:09:26.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4190999999999996e-05 2024-09-16T00:09:26.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877289+0000", 2024-09-16T00:09:26.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4058000000000002e-05 2024-09-16T00:09:26.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:26.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877376+0000", 2024-09-16T00:09:26.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7227999999999999e-05 2024-09-16T00:09:26.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:26.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.882925+0000", 2024-09-16T00:09:26.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055481050000000002 2024-09-16T00:09:26.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:26.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884635+0000", 2024-09-16T00:09:26.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017102759999999999 2024-09-16T00:09:26.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884674+0000", 2024-09-16T00:09:26.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8914e-05 2024-09-16T00:09:26.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884712+0000", 2024-09-16T00:09:26.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658999999999997e-05 2024-09-16T00:09:26.447 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.447 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.447 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.448 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5787 3.3d 3:bc1a97d0:::10000001a99.00000000:head [write 0~2770] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:26.448 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.087043+0000", 2024-09-16T00:09:26.448 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 11.252288977999999, 2024-09-16T00:09:26.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081081320000000005, 2024-09-16T00:09:26.448 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.448 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.448 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.448 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:26.448 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:26.448 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:26.448 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5787 2024-09-16T00:09:26.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.449 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087043+0000", 2024-09-16T00:09:26.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087043+0000", 2024-09-16T00:09:26.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087045+0000", 2024-09-16T00:09:26.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:09:26.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087053+0000", 2024-09-16T00:09:26.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7669999999999992e-06 2024-09-16T00:09:26.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087069+0000", 2024-09-16T00:09:26.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5155e-05 2024-09-16T00:09:26.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087081+0000", 2024-09-16T00:09:26.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2485e-05 2024-09-16T00:09:26.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087144+0000", 2024-09-16T00:09:26.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3255000000000003e-05 2024-09-16T00:09:26.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op must wait for map", 2024-09-16T00:09:26.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087152+0000", 2024-09-16T00:09:26.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5850000000000002e-06 2024-09-16T00:09:26.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.088816+0000", 2024-09-16T00:09:26.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016638480000000001 2024-09-16T00:09:26.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.088981+0000", 2024-09-16T00:09:26.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016522600000000001 2024-09-16T00:09:26.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:26.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.089128+0000", 2024-09-16T00:09:26.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000147441 2024-09-16T00:09:26.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:26.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.092831+0000", 2024-09-16T00:09:26.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037027319999999998 2024-09-16T00:09:26.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:26.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.095037+0000", 2024-09-16T00:09:26.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022059979999999998 2024-09-16T00:09:26.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.095100+0000", 2024-09-16T00:09:26.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2401000000000001e-05 2024-09-16T00:09:26.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.095151+0000", 2024-09-16T00:09:26.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1443000000000002e-05 2024-09-16T00:09:26.455 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.455 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.455 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.455 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:276 2.1e e69/19 2:7bf7e7b5:::200.00000006:head v 69'3, mlcod=22'1)", 2024-09-16T00:09:26.455 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:16.493303+0000", 2024-09-16T00:09:26.455 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.846028488, 2024-09-16T00:09:26.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0062641140000000003, 2024-09-16T00:09:26.456 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:26.456 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:26.456 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:26.456 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:26.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:26.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493303+0000", 2024-09-16T00:09:26.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:26.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493303+0000", 2024-09-16T00:09:26.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:26.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:26.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493305+0000", 2024-09-16T00:09:26.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.573e-06 2024-09-16T00:09:26.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:26.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.495058+0000", 2024-09-16T00:09:26.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001753183 2024-09-16T00:09:26.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:26.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.495080+0000", 2024-09-16T00:09:26.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1676999999999999e-05 2024-09-16T00:09:26.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:26.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.495092+0000", 2024-09-16T00:09:26.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2279000000000001e-05 2024-09-16T00:09:26.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:26.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.495159+0000", 2024-09-16T00:09:26.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6872000000000006e-05 2024-09-16T00:09:26.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:26.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.495178+0000", 2024-09-16T00:09:26.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8963000000000001e-05 2024-09-16T00:09:26.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:26.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.499520+0000", 2024-09-16T00:09:26.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043416490000000004 2024-09-16T00:09:26.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:26.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:26.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:26.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.499567+0000", 2024-09-16T00:09:26.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7917999999999998e-05 2024-09-16T00:09:26.460 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.460 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.460 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.460 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:26.460 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:26.461 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:26.462 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:09:26.463 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:26.461+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:26.484 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:09:26.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.488+0000 7fec5db4b640 1 -- 172.21.15.139:0/1286311527 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fec580cdcc0 con 0x7fec580c9430 2024-09-16T00:09:26.563 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:26.561+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:26.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.605+0000 7f75dc4e4640 1 Processor -- start 2024-09-16T00:09:26.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.605+0000 7f75dc4e4640 1 -- start start 2024-09-16T00:09:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.606+0000 7f75dc4e4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f75d4151730 0x7f75d4072830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.606+0000 7f75dc4e4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f75d4073b70 0x7f75d4073030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.606+0000 7f75dc4e4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75d4151b10 con 0x7f75d4073570 2024-09-16T00:09:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.606+0000 7f75dc4e4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f75d40c3ad0 con 0x7f75d4151730 2024-09-16T00:09:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.606+0000 7f75dc4e4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f75d40cd8c0 con 0x7f75d4073b70 2024-09-16T00:09:26.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.606+0000 7f75da280640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f75d4151730 0x7f75d4072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.606+0000 7f75d9a7f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f75d4073b70 0x7f75d4073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.606+0000 7f75da280640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f75d4151730 0x7f75d4072830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54094/0 (socket says 172.21.15.139:54094) 2024-09-16T00:09:26.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.607+0000 7f75da280640 1 -- 172.21.15.139:0/874132214 learned_addr learned my addr 172.21.15.139:0/874132214 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:26.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.607+0000 7f75d9a7f640 1 --2- 172.21.15.139:0/874132214 >> v2:172.21.15.164:3300/0 conn(0x7f75d4073b70 0x7f75d4073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:47780/0 (socket says 172.21.15.139:47780) 2024-09-16T00:09:26.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.607+0000 7f75d9a7f640 1 -- 172.21.15.139:0/874132214 >> v2:172.21.15.139:3300/0 conn(0x7f75d4151730 msgr2=0x7f75d4072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:26.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.607+0000 7f75d9a7f640 1 --2- 172.21.15.139:0/874132214 >> v2:172.21.15.139:3300/0 conn(0x7f75d4151730 0x7f75d4072830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.607+0000 7f75d9a7f640 1 -- 172.21.15.139:0/874132214 >> v1:172.21.15.139:6789/0 conn(0x7f75d4073570 legacy=0x7f75d41573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.607+0000 7f75d9a7f640 1 -- 172.21.15.139:0/874132214 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d40c3d20 con 0x7f75d4073b70 2024-09-16T00:09:26.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.608+0000 7f75d9a7f640 1 --2- 172.21.15.139:0/874132214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75d4073b70 0x7f75d4073030 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f75d406ff10 tx=0x7f75c402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b09efaf66863711d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.608+0000 7f75d927e640 1 -- 172.21.15.139:0/874132214 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4037070 con 0x7f75d4073b70 2024-09-16T00:09:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.608+0000 7f75d927e640 1 -- 172.21.15.139:0/874132214 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c4032440 con 0x7f75d4073b70 2024-09-16T00:09:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.608+0000 7f75d927e640 1 -- 172.21.15.139:0/874132214 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4032810 con 0x7f75d4073b70 2024-09-16T00:09:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.609+0000 7f75dc4e4640 1 -- 172.21.15.139:0/874132214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75d4073b70 msgr2=0x7f75d4073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.609+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/874132214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75d4073b70 0x7f75d4073030 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f75d406ff10 tx=0x7f75c402a230 comp rx=0 tx=0).stop 2024-09-16T00:09:26.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.609+0000 7f75dc4e4640 1 -- 172.21.15.139:0/874132214 shutdown_connections 2024-09-16T00:09:26.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.609+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/874132214 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75d4073b70 0x7f75d4073030 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.609+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/874132214 >> v2:172.21.15.139:3300/0 conn(0x7f75d4151730 0x7f75d4072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.609+0000 7f75dc4e4640 1 -- 172.21.15.139:0/874132214 >> 172.21.15.139:0/874132214 conn(0x7f75d4067950 msgr2=0x7f75d4067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:26.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.609+0000 7f75dc4e4640 1 -- 172.21.15.139:0/874132214 shutdown_connections 2024-09-16T00:09:26.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.609+0000 7f75dc4e4640 1 -- 172.21.15.139:0/874132214 wait complete. 2024-09-16T00:09:26.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.611+0000 7f75dc4e4640 1 Processor -- start 2024-09-16T00:09:26.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.611+0000 7f75dc4e4640 1 -- start start 2024-09-16T00:09:26.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.612+0000 7f75dc4e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d4073570 0x7f75d40cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.612+0000 7f75dc4e4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75d4073b70 0x7f75d41eff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.612+0000 7f75dc4e4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75d4151730 0x7f75d41550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.612+0000 7f75dc4e4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f75d40cd550 con 0x7f75d4073570 2024-09-16T00:09:26.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.612+0000 7f75dc4e4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f75d40cd270 con 0x7f75d4151730 2024-09-16T00:09:26.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.612+0000 7f75dc4e4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f75d4073fa0 con 0x7f75d4073b70 2024-09-16T00:09:26.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.612+0000 7f75da280640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d4073570 0x7f75d40cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.612+0000 7f75da280640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d4073570 0x7f75d40cc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:54096/0 (socket says 172.21.15.139:54096) 2024-09-16T00:09:26.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.612+0000 7f75da280640 1 -- 172.21.15.139:0/1860682711 learned_addr learned my addr 172.21.15.139:0/1860682711 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:26.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.612+0000 7f75d9a7f640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75d4073b70 0x7f75d41eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.612+0000 7f75daa81640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75d4151730 0x7f75d41550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.613+0000 7f75da280640 1 -- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75d4073b70 msgr2=0x7f75d41eff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.613+0000 7f75da280640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75d4073b70 0x7f75d41eff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.613+0000 7f75da280640 1 -- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75d4151730 msgr2=0x7f75d41550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.613+0000 7f75da280640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75d4151730 0x7f75d41550c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.613+0000 7f75da280640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d414d7e0 con 0x7f75d4073570 2024-09-16T00:09:26.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.613+0000 7f75da280640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d4073570 0x7f75d40cc140 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f75d0010df0 tx=0x7f75d000e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:26.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.613+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d001e070 con 0x7f75d4073570 2024-09-16T00:09:26.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.613+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d0011660 con 0x7f75d4073570 2024-09-16T00:09:26.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.613+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0011a30 con 0x7f75d4073570 2024-09-16T00:09:26.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.613+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d414dac0 con 0x7f75d4073570 2024-09-16T00:09:26.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.613+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d414cf70 con 0x7f75d4073570 2024-09-16T00:09:26.619 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:26.619 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:26.619 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:26.619 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.615+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f75d406d9a0 con 0x7f75d4073570 2024-09-16T00:09:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.616+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f75d0021070 con 0x7f75d4073570 2024-09-16T00:09:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.616+0000 7f75caffd640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f75b806c2e0 0x7f75b806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.617+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5944+0+0 (secure 0 0 0) 0x7f75d008e7d0 con 0x7f75d4073570 2024-09-16T00:09:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.617+0000 7f75d9a7f640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f75b806c2e0 0x7f75b806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.617+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v75) ==== 54+0+16 (secure 0 0 0) 0x7f75d008ec60 con 0x7f75d4073570 2024-09-16T00:09:26.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.617+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f75d4155600 0x7f75d4148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.617+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75d414d270 con 0x7f75d4155600 2024-09-16T00:09:26.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.617+0000 7f75daa81640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f75d4155600 0x7f75d4148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.617+0000 7f75d9a7f640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f75b806c2e0 0x7f75b806e7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f75c4034000 tx=0x7f75c402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:26.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.618+0000 7f75daa81640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f75d4155600 0x7f75d4148910 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:26.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.618+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75d414d270 con 0x7f75d4155600 2024-09-16T00:09:26.626 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:09:26.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.638+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f75d40c2800 con 0x7f75d4155600 2024-09-16T00:09:26.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.639+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f75d40c2800 con 0x7f75d4155600 2024-09-16T00:09:26.640 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:26.641 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:26.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.640+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f75d41ec140 0x7f75d414b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.640+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f75d4149360 con 0x7f75d41ec140 2024-09-16T00:09:26.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.640+0000 7f75da280640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f75d41ec140 0x7f75d414b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.640+0000 7f75da280640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f75d41ec140 0x7f75d414b9d0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:26.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.641+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75d4149360 con 0x7f75d41ec140 2024-09-16T00:09:26.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.659+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f75d41f0490 con 0x7f75d41ec140 2024-09-16T00:09:26.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.661+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f75d41f0490 con 0x7f75d41ec140 2024-09-16T00:09:26.662 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:26.663 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:26.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.661+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f75d414c420 0x7f75d4062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.661+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f75d4150740 con 0x7f75d414c420 2024-09-16T00:09:26.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.661+0000 7f75d9a7f640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f75d414c420 0x7f75d4062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.662+0000 7f75d9a7f640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f75d414c420 0x7f75d4062630 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:26.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.662+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75d4150740 con 0x7f75d414c420 2024-09-16T00:09:26.664 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:26.663+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:26.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.681+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f75d414dc90 con 0x7f75d414c420 2024-09-16T00:09:26.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.683+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f75d414dc90 con 0x7f75d414c420 2024-09-16T00:09:26.684 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:26.685 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:26.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.684+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f75d414a750 0x7f75d414abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.684+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f75d4063440 con 0x7f75d414a750 2024-09-16T00:09:26.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.684+0000 7f75daa81640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f75d414a750 0x7f75d414abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.684+0000 7f75daa81640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f75d414a750 0x7f75d414abd0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:26.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.685+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75d4063440 con 0x7f75d414a750 2024-09-16T00:09:26.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.701+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f75d414b110 con 0x7f75d414a750 2024-09-16T00:09:26.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.703+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f75d414b110 con 0x7f75d414a750 2024-09-16T00:09:26.704 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:26.704 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:26.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.703+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f75d402cb50 0x7f75d402cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.703+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f75d414b110 con 0x7f75d402cb50 2024-09-16T00:09:26.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.703+0000 7f75da280640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f75d402cb50 0x7f75d402cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.704+0000 7f75da280640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f75d402cb50 0x7f75d402cfb0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:26.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.705+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75d414b110 con 0x7f75d402cb50 2024-09-16T00:09:26.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.721+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f75d414b110 con 0x7f75d402cb50 2024-09-16T00:09:26.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.723+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f75d414b110 con 0x7f75d402cb50 2024-09-16T00:09:26.724 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:26.725 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:26.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.724+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f75d4200a30 0x7f75d4202e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.724+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f75d414de40 con 0x7f75d4200a30 2024-09-16T00:09:26.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.724+0000 7f75d9a7f640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f75d4200a30 0x7f75d4202e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.724+0000 7f75d9a7f640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f75d4200a30 0x7f75d4202e20 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:26.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.725+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75d414de40 con 0x7f75d4200a30 2024-09-16T00:09:26.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.742+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f75d414de40 con 0x7f75d4200a30 2024-09-16T00:09:26.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.743+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f75d414de40 con 0x7f75d4200a30 2024-09-16T00:09:26.744 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.743+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f75d4203870 0x7f75d4205c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.744+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f75d402da00 con 0x7f75d4203870 2024-09-16T00:09:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.744+0000 7f75daa81640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f75d4203870 0x7f75d4205c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.744+0000 7f75daa81640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f75d4203870 0x7f75d4205c60 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:26.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.745+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75d402da00 con 0x7f75d4203870 2024-09-16T00:09:26.752 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:26.753 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:26.762 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:09:26.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.762+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f75d402da00 con 0x7f75d4203870 2024-09-16T00:09:26.765 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:26.766 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:09:26.766 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:26.764+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:26.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.764+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f75d402da00 con 0x7f75d4203870 2024-09-16T00:09:26.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.764+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f75d42066b0 0x7f75d4208aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:26.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.764+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f75d420b9f0 con 0x7f75d42066b0 2024-09-16T00:09:26.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.764+0000 7f75da280640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f75d42066b0 0x7f75d4208aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:26.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.765+0000 7f75da280640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f75d42066b0 0x7f75d4208aa0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:26.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.765+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75d420b9f0 con 0x7f75d42066b0 2024-09-16T00:09:26.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.783+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f75d40755b0 con 0x7f75d42066b0 2024-09-16T00:09:26.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75caffd640 1 -- 172.21.15.139:0/1860682711 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f75d40755b0 con 0x7f75d42066b0 2024-09-16T00:09:26.786 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:26.787 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:26.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f75d4155600 msgr2=0x7f75d4148910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f75d4155600 0x7f75d4148910 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f75d41ec140 msgr2=0x7f75d414b9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f75d41ec140 0x7f75d414b9d0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f75d414c420 msgr2=0x7f75d4062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f75d414c420 0x7f75d4062630 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f75d414a750 msgr2=0x7f75d414abd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75daa81640 1 -- 172.21.15.139:0/1860682711 reap_dead start 2024-09-16T00:09:26.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f75d414a750 0x7f75d414abd0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f75d402cb50 msgr2=0x7f75d402cfb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f75d402cb50 0x7f75d402cfb0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f75d4200a30 msgr2=0x7f75d4202e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f75d4200a30 0x7f75d4202e20 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f75d4203870 msgr2=0x7f75d4205c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f75d4203870 0x7f75d4205c60 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f75d42066b0 msgr2=0x7f75d4208aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f75d42066b0 0x7f75d4208aa0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75daa81640 1 -- 172.21.15.139:0/1860682711 reap_dead start 2024-09-16T00:09:26.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f75b806c2e0 msgr2=0x7f75b806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f75b806c2e0 0x7f75b806e7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f75c4034000 tx=0x7f75c402e000 comp rx=0 tx=0).stop 2024-09-16T00:09:26.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d4073570 msgr2=0x7f75d40cc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:26.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d4073570 0x7f75d40cc140 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f75d0010df0 tx=0x7f75d000e730 comp rx=0 tx=0).stop 2024-09-16T00:09:26.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 shutdown_connections 2024-09-16T00:09:26.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.785+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f75b806c2e0 0x7f75b806e7a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.786+0000 7f75dc4e4640 1 --2- 172.21.15.139:0/1860682711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75d4073570 0x7f75d40cc140 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:26.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.786+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 >> 172.21.15.139:0/1860682711 conn(0x7f75d4067950 msgr2=0x7f75d40c8640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:26.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.786+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 shutdown_connections 2024-09-16T00:09:26.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:26.786+0000 7f75dc4e4640 1 -- 172.21.15.139:0/1860682711 wait complete. 2024-09-16T00:09:26.866 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:26.865+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:26.887 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:26.887 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:26.897 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:09:26.967 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:26.966+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:27.017 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:27.017 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:27.027 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:09:27.071 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:27.070+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:27.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:27.135+0000 7fec3ffff640 1 -- 172.21.15.139:0/1286311527 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v76) ==== 101+0+0 (secure 0 0 0) 0x7fec48058760 con 0x7fec580c9430 2024-09-16T00:09:27.136 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is unset 2024-09-16T00:09:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:27.138+0000 7fec5db4b640 1 -- 172.21.15.139:0/1286311527 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fec4406c2b0 msgr2=0x7fec4406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:27.138+0000 7fec5db4b640 1 --2- 172.21.15.139:0/1286311527 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fec4406c2b0 0x7fec4406e770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fec5814a9c0 tx=0x7fec4c009040 comp rx=0 tx=0).stop 2024-09-16T00:09:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:27.138+0000 7fec5db4b640 1 -- 172.21.15.139:0/1286311527 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec580c9430 msgr2=0x7fec58156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:27.138+0000 7fec5db4b640 1 --2- 172.21.15.139:0/1286311527 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec580c9430 0x7fec58156850 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fec4800c3c0 tx=0x7fec4800c880 comp rx=0 tx=0).stop 2024-09-16T00:09:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:27.138+0000 7fec5db4b640 1 -- 172.21.15.139:0/1286311527 shutdown_connections 2024-09-16T00:09:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:27.138+0000 7fec5db4b640 1 --2- 172.21.15.139:0/1286311527 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fec4406c2b0 0x7fec4406e770 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:27.138+0000 7fec5db4b640 1 --2- 172.21.15.139:0/1286311527 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec580ce810 0x7fec580c3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:27.138+0000 7fec5db4b640 1 --2- 172.21.15.139:0/1286311527 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fec580ce430 0x7fec580c2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:27.138+0000 7fec5db4b640 1 --2- 172.21.15.139:0/1286311527 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fec580c9430 0x7fec58156850 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fec4800c3c0 tx=0x7fec4800c880 comp rx=0 tx=0).stop 2024-09-16T00:09:27.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:27.138+0000 7fec5db4b640 1 -- 172.21.15.139:0/1286311527 >> 172.21.15.139:0/1286311527 conn(0x7fec5806c150 msgr2=0x7fec580cabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:27.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:27.138+0000 7fec5db4b640 1 -- 172.21.15.139:0/1286311527 shutdown_connections 2024-09-16T00:09:27.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:27.138+0000 7fec5db4b640 1 -- 172.21.15.139:0/1286311527 wait complete. 2024-09-16T00:09:27.172 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:27.171+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:27.196 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:27.196 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:27.207 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:09:27.273 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:27.272+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:27.331 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:27.331 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:27.340 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:09:27.374 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:27.373+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:27.475 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:27.474+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:27.494 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:27.494 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:27.504 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:09:27.576 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:27.575+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:27.624 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:27.624 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:27.635 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:09:27.677 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:27.676+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:27.750 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:27.751 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:27.760 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:09:27.803 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:27.802+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:27.874 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:27.875 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:27.879 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:27.878+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:27.885 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:09:27.980 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:27.980+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:28.041 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:28.041 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:28.050 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:09:28.081 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:28.080+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:28.181 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:28.181 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:28.182 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:28.182+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:28.193 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:09:28.283 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:28.282+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:28.306 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:28.307 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:28.316 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:09:28.384 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:28.382+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:28.449 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:28.450 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:28.458 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:09:28.499 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:28.498+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:28.586 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:28.586+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:28.602 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:28.602 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:28.610 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:09:28.686 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:28.686+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:28.751 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:28.751 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:28.762 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:09:28.787 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:28.787+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:28.800 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:09:28.876 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:28.876 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:28.889 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:09:28.889 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:28.888+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:28.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.907+0000 7f3dee077640 1 Processor -- start 2024-09-16T00:09:28.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.907+0000 7f3dee077640 1 -- start start 2024-09-16T00:09:28.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.908+0000 7f3dee077640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3de80b4750 0x7f3de81546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:28.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.908+0000 7f3dee077640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3de815adc0 con 0x7f3de80d0870 2024-09-16T00:09:28.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.908+0000 7f3dee077640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3de815b0a0 con 0x7f3de80d0490 2024-09-16T00:09:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.908+0000 7f3dee077640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3de815b940 con 0x7f3de80b4750 2024-09-16T00:09:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.908+0000 7f3de7fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3de80b4750 0x7f3de81546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.908+0000 7f3de6ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f3de80d0870 0x7f3de80b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38092/0 (socket says 172.21.15.139:38092) 2024-09-16T00:09:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.908+0000 7f3de6ffd640 1 -- 172.21.15.139:0/2638264679 learned_addr learned my addr 172.21.15.139:0/2638264679 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.908+0000 7f3de7fff640 1 -- 172.21.15.139:0/2638264679 >> v1:172.21.15.164:6789/0 conn(0x7f3de80d0490 legacy=0x7f3de80b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.908+0000 7f3de7fff640 1 -- 172.21.15.139:0/2638264679 >> v1:172.21.15.139:6789/0 conn(0x7f3de80d0870 legacy=0x7f3de80b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.908+0000 7f3de67fc640 1 -- 172.21.15.139:0/2638264679 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1545950279 0 0) 0x7f3de815b0a0 con 0x7f3de80d0490 2024-09-16T00:09:28.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.908+0000 7f3de7fff640 1 -- 172.21.15.139:0/2638264679 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de815b380 con 0x7f3de80b4750 2024-09-16T00:09:28.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.909+0000 7f3de7fff640 1 --2- 172.21.15.139:0/2638264679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de80b4750 0x7f3de81546a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009d90 tx=0x7f3de002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f697e9090689b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:28.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.909+0000 7f3de67fc640 1 -- 172.21.15.139:0/2638264679 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0007740 con 0x7f3de80b4750 2024-09-16T00:09:28.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.909+0000 7f3de67fc640 1 -- 172.21.15.139:0/2638264679 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3de00078e0 con 0x7f3de80b4750 2024-09-16T00:09:28.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.909+0000 7f3de67fc640 1 -- 172.21.15.139:0/2638264679 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0007cb0 con 0x7f3de80b4750 2024-09-16T00:09:28.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.910+0000 7f3dee077640 1 -- 172.21.15.139:0/2638264679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de80b4750 msgr2=0x7f3de81546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:28.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.910+0000 7f3dee077640 1 --2- 172.21.15.139:0/2638264679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de80b4750 0x7f3de81546a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009d90 tx=0x7f3de002a680 comp rx=0 tx=0).stop 2024-09-16T00:09:28.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.910+0000 7f3dee077640 1 -- 172.21.15.139:0/2638264679 shutdown_connections 2024-09-16T00:09:28.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.910+0000 7f3dee077640 1 --2- 172.21.15.139:0/2638264679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de80b4750 0x7f3de81546a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:28.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.910+0000 7f3dee077640 1 -- 172.21.15.139:0/2638264679 >> 172.21.15.139:0/2638264679 conn(0x7f3de806c520 msgr2=0x7f3de806c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:28.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.910+0000 7f3dee077640 1 -- 172.21.15.139:0/2638264679 shutdown_connections 2024-09-16T00:09:28.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.910+0000 7f3dee077640 1 -- 172.21.15.139:0/2638264679 wait complete. 2024-09-16T00:09:28.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.911+0000 7f3dee077640 1 Processor -- start 2024-09-16T00:09:28.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.911+0000 7f3dee077640 1 -- start start 2024-09-16T00:09:28.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.913+0000 7f3dee077640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3de80b4750 0x7f3de81576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:28.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.913+0000 7f3dee077640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de80d0490 0x7f3de814fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:28.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.913+0000 7f3dee077640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3de80d0870 0x7f3de81f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:28.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.913+0000 7f3dee077640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3de815a980 con 0x7f3de80d0490 2024-09-16T00:09:28.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.913+0000 7f3dee077640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3de80b3bb0 con 0x7f3de80d0870 2024-09-16T00:09:28.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.913+0000 7f3dee077640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3de815ab00 con 0x7f3de80b4750 2024-09-16T00:09:28.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.913+0000 7f3de6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de80d0490 0x7f3de814fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:28.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.913+0000 7f3de6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de80d0490 0x7f3de814fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58742/0 (socket says 172.21.15.139:58742) 2024-09-16T00:09:28.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.913+0000 7f3de6ffd640 1 -- 172.21.15.139:0/3439033710 learned_addr learned my addr 172.21.15.139:0/3439033710 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:28.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.913+0000 7f3de77fe640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3de80b4750 0x7f3de81576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:28.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.913+0000 7f3de6ffd640 1 -- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3de80b4750 msgr2=0x7f3de81576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:28.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.913+0000 7f3de6ffd640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3de80b4750 0x7f3de81576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:28.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.913+0000 7f3de6ffd640 1 -- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3de80d0870 msgr2=0x7f3de81f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:28.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.913+0000 7f3de6ffd640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3de80d0870 0x7f3de81f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:28.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.913+0000 7f3de6ffd640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de81568d0 con 0x7f3de80d0490 2024-09-16T00:09:28.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.913+0000 7f3de6ffd640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de80d0490 0x7f3de814fc40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00c5a0 tx=0x7f3ddc00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:28.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.914+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc017070 con 0x7f3de80d0490 2024-09-16T00:09:28.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.914+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ddc0074e0 con 0x7f3de80d0490 2024-09-16T00:09:28.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.914+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de8155440 con 0x7f3de80d0490 2024-09-16T00:09:28.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.914+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc004a00 con 0x7f3de80d0490 2024-09-16T00:09:28.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.914+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de8155a00 con 0x7f3de80d0490 2024-09-16T00:09:28.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.915+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f3de80cb770 con 0x7f3de80d0490 2024-09-16T00:09:28.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.915+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3ddc01a070 con 0x7f3de80d0490 2024-09-16T00:09:28.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.916+0000 7f3dcffff640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3dd406c2b0 0x7f3dd406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:28.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.916+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5944+0+0 (secure 0 0 0) 0x7f3ddc08a8e0 con 0x7f3de80d0490 2024-09-16T00:09:28.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.916+0000 7f3de77fe640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3dd406c2b0 0x7f3dd406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:28.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.916+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v76) ==== 54+0+16 (secure 0 0 0) 0x7f3de80cb770 con 0x7f3de80d0490 2024-09-16T00:09:28.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.916+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f3de80c8940 0x7f3de8158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:28.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.916+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3de80b3430 con 0x7f3de80c8940 2024-09-16T00:09:28.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.917+0000 7f3de7fff640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f3de80c8940 0x7f3de8158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:28.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.917+0000 7f3de77fe640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3dd406c2b0 0x7f3dd406e770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8000f60 tx=0x7f3dd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:28.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.917+0000 7f3de7fff640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f3de80c8940 0x7f3de8158090 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:28.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.918+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3de80b3430 con 0x7f3de80c8940 2024-09-16T00:09:28.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.933+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f3de81596c0 con 0x7f3de80c8940 2024-09-16T00:09:28.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.935+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3de81596c0 con 0x7f3de80c8940 2024-09-16T00:09:28.936 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:28.937 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:28.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.936+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f3de814a510 0x7f3de814a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:28.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.936+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f3de81593e0 con 0x7f3de814a510 2024-09-16T00:09:28.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.936+0000 7f3de77fe640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f3de814a510 0x7f3de814a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:28.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.936+0000 7f3de77fe640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f3de814a510 0x7f3de814a8f0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:28.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.937+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3de81593e0 con 0x7f3de814a510 2024-09-16T00:09:28.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.953+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f3de8159100 con 0x7f3de814a510 2024-09-16T00:09:28.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.955+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3de8159100 con 0x7f3de814a510 2024-09-16T00:09:28.956 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:28.956 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:28.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.955+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f3de814ae30 0x7f3de80c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:28.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.955+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f3de80c82d0 con 0x7f3de814ae30 2024-09-16T00:09:28.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.955+0000 7f3de6ffd640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f3de814ae30 0x7f3de80c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:28.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.955+0000 7f3de6ffd640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f3de814ae30 0x7f3de80c7d90 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:28.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.956+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3de80c82d0 con 0x7f3de814ae30 2024-09-16T00:09:28.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.971+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f3de8062630 con 0x7f3de814ae30 2024-09-16T00:09:28.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.973+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3de8062630 con 0x7f3de814ae30 2024-09-16T00:09:28.974 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:28.974 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:28.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.973+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f3de8063110 0x7f3de80ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:28.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.973+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f3de8063570 con 0x7f3de8063110 2024-09-16T00:09:28.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.973+0000 7f3de7fff640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f3de8063110 0x7f3de80ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:28.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.974+0000 7f3de7fff640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f3de8063110 0x7f3de80ce0b0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:28.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.974+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3de8063570 con 0x7f3de8063110 2024-09-16T00:09:28.989 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:28.989+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:28.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.992+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f3de8063570 con 0x7f3de8063110 2024-09-16T00:09:28.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.993+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3de8063570 con 0x7f3de8063110 2024-09-16T00:09:28.994 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:28.995 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:28.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.994+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f3de80ced50 0x7f3de802c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:28.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.994+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f3de8063570 con 0x7f3de80ced50 2024-09-16T00:09:28.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.994+0000 7f3de77fe640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f3de80ced50 0x7f3de802c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:28.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.994+0000 7f3de77fe640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f3de80ced50 0x7f3de802c640 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:28.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:28.995+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3de8063570 con 0x7f3de80ced50 2024-09-16T00:09:29.006 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:29.006 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:29.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.011+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f3de8063570 con 0x7f3de80ced50 2024-09-16T00:09:29.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.013+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3de8063570 con 0x7f3de80ced50 2024-09-16T00:09:29.014 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:29.015 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:29.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.013+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f3de802d090 0x7f3de802d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.013+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f3de802db20 con 0x7f3de802d090 2024-09-16T00:09:29.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.014+0000 7f3de6ffd640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f3de802d090 0x7f3de802d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.014+0000 7f3de6ffd640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f3de802d090 0x7f3de802d510 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:29.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.015+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3de802db20 con 0x7f3de802d090 2024-09-16T00:09:29.017 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:09:29.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.032+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f3de802db20 con 0x7f3de802d090 2024-09-16T00:09:29.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.034+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3de802db20 con 0x7f3de802d090 2024-09-16T00:09:29.035 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:29.035 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:29.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.034+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f3de82073a0 0x7f3de8209790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.034+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f3de820c6c0 con 0x7f3de82073a0 2024-09-16T00:09:29.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.034+0000 7f3de7fff640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f3de82073a0 0x7f3de8209790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.035+0000 7f3de7fff640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f3de82073a0 0x7f3de8209790 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:29.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.035+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3de820c6c0 con 0x7f3de82073a0 2024-09-16T00:09:29.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.050+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f3de820c690 con 0x7f3de82073a0 2024-09-16T00:09:29.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.052+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3de820c690 con 0x7f3de82073a0 2024-09-16T00:09:29.053 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:29.054 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:09:29.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.052+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f3de820b940 0x7f3de820bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.052+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f3de820fae0 con 0x7f3de820b940 2024-09-16T00:09:29.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.052+0000 7f3de77fe640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f3de820b940 0x7f3de820bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.053+0000 7f3de77fe640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f3de820b940 0x7f3de820bda0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:29.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.054+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3de820fae0 con 0x7f3de820b940 2024-09-16T00:09:29.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.069+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f3de820e960 con 0x7f3de820b940 2024-09-16T00:09:29.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dcffff640 1 -- 172.21.15.139:0/3439033710 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3de820e960 con 0x7f3de820b940 2024-09-16T00:09:29.072 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:29.073 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:29.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f3de80c8940 msgr2=0x7f3de8158090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f3de80c8940 0x7f3de8158090 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f3de814a510 msgr2=0x7f3de814a8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f3de814a510 0x7f3de814a8f0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f3de814ae30 msgr2=0x7f3de80c7d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f3de814ae30 0x7f3de80c7d90 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f3de8063110 msgr2=0x7f3de80ce0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f3de8063110 0x7f3de80ce0b0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f3de80ced50 msgr2=0x7f3de802c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f3de80ced50 0x7f3de802c640 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3de7fff640 1 -- 172.21.15.139:0/3439033710 reap_dead start 2024-09-16T00:09:29.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3de7fff640 1 -- 172.21.15.139:0/3439033710 reap_dead start 2024-09-16T00:09:29.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f3de802d090 msgr2=0x7f3de802d510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f3de802d090 0x7f3de802d510 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f3de82073a0 msgr2=0x7f3de8209790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f3de82073a0 0x7f3de8209790 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f3de820b940 msgr2=0x7f3de820bda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f3de820b940 0x7f3de820bda0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3dd406c2b0 msgr2=0x7f3dd406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3dd406c2b0 0x7f3dd406e770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8000f60 tx=0x7f3dd8009040 comp rx=0 tx=0).stop 2024-09-16T00:09:29.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de80d0490 msgr2=0x7f3de814fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 --2- 172.21.15.139:0/3439033710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3de80d0490 0x7f3de814fc40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00c5a0 tx=0x7f3ddc00ca60 comp rx=0 tx=0).stop 2024-09-16T00:09:29.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3de7fff640 1 -- 172.21.15.139:0/3439033710 reap_dead start 2024-09-16T00:09:29.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 shutdown_connections 2024-09-16T00:09:29.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.071+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 >> 172.21.15.139:0/3439033710 conn(0x7f3de806c520 msgr2=0x7f3de80cae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:29.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.072+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 shutdown_connections 2024-09-16T00:09:29.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.072+0000 7f3dee077640 1 -- 172.21.15.139:0/3439033710 wait complete. 2024-09-16T00:09:29.090 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:29.089+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:29.140 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:29.140 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:29.140 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:29.140 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:29.149 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:09:29.150 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T00:09:29.150 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:09:29.150 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T00:09:29.150 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T00:09:29.155 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T00:09:29.192 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:29.190+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:29.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.263+0000 7f536998f640 1 Processor -- start 2024-09-16T00:09:29.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.263+0000 7f536998f640 1 -- start start 2024-09-16T00:09:29.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.264+0000 7f536998f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53640c7b90 0x7f536415a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.264+0000 7f536998f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f536415ad50 0x7f53640cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.264+0000 7f536998f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53640c7fb0 con 0x7f53640c67e0 2024-09-16T00:09:29.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.264+0000 7f536998f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f536406cd10 con 0x7f536415ad50 2024-09-16T00:09:29.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.264+0000 7f536998f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f53640c6bc0 con 0x7f53640c7b90 2024-09-16T00:09:29.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.264+0000 7f53637fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f536415ad50 0x7f53640cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.264+0000 7f53627fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53640c7b90 0x7f536415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.264+0000 7f53637fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f536415ad50 0x7f53640cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58750/0 (socket says 172.21.15.139:58750) 2024-09-16T00:09:29.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.264+0000 7f5362ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f53640c67e0 0x7f536406c110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38108/0 (socket says 172.21.15.139:38108) 2024-09-16T00:09:29.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.264+0000 7f53637fe640 1 -- 172.21.15.139:0/2756397867 learned_addr learned my addr 172.21.15.139:0/2756397867 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:29.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.265+0000 7f53637fe640 1 -- 172.21.15.139:0/2756397867 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53640c7b90 msgr2=0x7f536415a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:29.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.265+0000 7f53637fe640 1 --2- 172.21.15.139:0/2756397867 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53640c7b90 0x7f536415a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.265+0000 7f53637fe640 1 -- 172.21.15.139:0/2756397867 >> v1:172.21.15.139:6789/0 conn(0x7f53640c67e0 legacy=0x7f536406c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.266 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.265+0000 7f53637fe640 1 -- 172.21.15.139:0/2756397867 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536406ca30 con 0x7f536415ad50 2024-09-16T00:09:29.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.266+0000 7f53637fe640 1 --2- 172.21.15.139:0/2756397867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f536415ad50 0x7f53640cbe50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f535c009e50 tx=0x7f535c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c84c44aced3cfa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:29.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.266+0000 7f5361ffb640 1 -- 172.21.15.139:0/2756397867 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c036070 con 0x7f536415ad50 2024-09-16T00:09:29.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.266+0000 7f5361ffb640 1 -- 172.21.15.139:0/2756397867 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535c031600 con 0x7f536415ad50 2024-09-16T00:09:29.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.266+0000 7f5361ffb640 1 -- 172.21.15.139:0/2756397867 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c0319d0 con 0x7f536415ad50 2024-09-16T00:09:29.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.266+0000 7f536998f640 1 -- 172.21.15.139:0/2756397867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f536415ad50 msgr2=0x7f53640cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.266+0000 7f536998f640 1 --2- 172.21.15.139:0/2756397867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f536415ad50 0x7f53640cbe50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f535c009e50 tx=0x7f535c02a6c0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.267+0000 7f536998f640 1 -- 172.21.15.139:0/2756397867 shutdown_connections 2024-09-16T00:09:29.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.267+0000 7f536998f640 1 --2- 172.21.15.139:0/2756397867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f536415ad50 0x7f53640cbe50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.267+0000 7f536998f640 1 --2- 172.21.15.139:0/2756397867 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53640c7b90 0x7f536415a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.267+0000 7f536998f640 1 -- 172.21.15.139:0/2756397867 >> 172.21.15.139:0/2756397867 conn(0x7f53640d05f0 msgr2=0x7f53640d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:29.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.267+0000 7f536998f640 1 -- 172.21.15.139:0/2756397867 shutdown_connections 2024-09-16T00:09:29.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.267+0000 7f536998f640 1 -- 172.21.15.139:0/2756397867 wait complete. 2024-09-16T00:09:29.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.268+0000 7f536998f640 1 Processor -- start 2024-09-16T00:09:29.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.269+0000 7f536998f640 1 -- start start 2024-09-16T00:09:29.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f536998f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53640c67e0 0x7f53640c2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f536998f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53640c7b90 0x7f53641e4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f536998f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f536415ad50 0x7f5364154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f536998f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f536415e200 con 0x7f53640c67e0 2024-09-16T00:09:29.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f536998f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f536415e850 con 0x7f536415ad50 2024-09-16T00:09:29.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f536998f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f536415e4e0 con 0x7f53640c7b90 2024-09-16T00:09:29.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f5362ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53640c67e0 0x7f53640c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f53627fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53640c7b90 0x7f53641e4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f53627fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53640c7b90 0x7f53641e4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:43546/0 (socket says 172.21.15.139:43546) 2024-09-16T00:09:29.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f5362ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53640c67e0 0x7f53640c2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58758/0 (socket says 172.21.15.139:58758) 2024-09-16T00:09:29.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f53627fc640 1 -- 172.21.15.139:0/2272375969 learned_addr learned my addr 172.21.15.139:0/2272375969 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:29.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f53637fe640 1 --2- 172.21.15.139:0/2272375969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f536415ad50 0x7f5364154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f53627fc640 1 -- 172.21.15.139:0/2272375969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f536415ad50 msgr2=0x7f5364154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f53627fc640 1 --2- 172.21.15.139:0/2272375969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f536415ad50 0x7f5364154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f53627fc640 1 -- 172.21.15.139:0/2272375969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53640c67e0 msgr2=0x7f53640c2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f53627fc640 1 --2- 172.21.15.139:0/2272375969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53640c67e0 0x7f53640c2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f53627fc640 1 -- 172.21.15.139:0/2272375969 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53641f0560 con 0x7f53640c7b90 2024-09-16T00:09:29.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.270+0000 7f5362ffd640 1 --2- 172.21.15.139:0/2272375969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53640c67e0 0x7f53640c2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:29.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.271+0000 7f53627fc640 1 --2- 172.21.15.139:0/2272375969 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53640c7b90 0x7f53641e4e40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f535800e9b0 tx=0x7f535800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:29.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.271+0000 7f534bfff640 1 -- 172.21.15.139:0/2272375969 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358017070 con 0x7f53640c7b90 2024-09-16T00:09:29.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.271+0000 7f534bfff640 1 -- 172.21.15.139:0/2272375969 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f535800c9a0 con 0x7f53640c7b90 2024-09-16T00:09:29.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.271+0000 7f536998f640 1 -- 172.21.15.139:0/2272375969 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5364155cc0 con 0x7f53640c7b90 2024-09-16T00:09:29.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.271+0000 7f534bfff640 1 -- 172.21.15.139:0/2272375969 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535800cd70 con 0x7f53640c7b90 2024-09-16T00:09:29.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.271+0000 7f536998f640 1 -- 172.21.15.139:0/2272375969 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5364156280 con 0x7f53640c7b90 2024-09-16T00:09:29.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.272+0000 7f536998f640 1 -- 172.21.15.139:0/2272375969 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f536415a810 con 0x7f53640c7b90 2024-09-16T00:09:29.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.273+0000 7f534bfff640 1 -- 172.21.15.139:0/2272375969 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f535801a070 con 0x7f53640c7b90 2024-09-16T00:09:29.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.274+0000 7f534bfff640 1 --2- 172.21.15.139:0/2272375969 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f535006c300 0x7f535006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.274+0000 7f534bfff640 1 -- 172.21.15.139:0/2272375969 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5944+0+0 (secure 0 0 0) 0x7f535808b430 con 0x7f53640c7b90 2024-09-16T00:09:29.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.274+0000 7f5362ffd640 1 --2- 172.21.15.139:0/2272375969 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f535006c300 0x7f535006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.274+0000 7f5362ffd640 1 --2- 172.21.15.139:0/2272375969 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f535006c300 0x7f535006e7c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5354000f60 tx=0x7f5354000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:29.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.280+0000 7f534bfff640 1 -- 172.21.15.139:0/2272375969 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f53580594b0 con 0x7f53640c7b90 2024-09-16T00:09:29.291 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:29.291+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:29.308 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:29.309 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:29.309 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:29.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.313+0000 7f285fe65640 1 Processor -- start 2024-09-16T00:09:29.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.314+0000 7f285fe65640 1 -- start start 2024-09-16T00:09:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.315+0000 7f285fe65640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2858154aa0 0x7f285806bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.315+0000 7f285fe65640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f285806f360 0x7f28580cd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.315+0000 7f285fe65640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f285806f830 con 0x7f285806eae0 2024-09-16T00:09:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.315+0000 7f285fe65640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f28580c85b0 con 0x7f285806f360 2024-09-16T00:09:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.315+0000 7f285fe65640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f285806fb10 con 0x7f2858154aa0 2024-09-16T00:09:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.315+0000 7f285d400640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f285806eae0 0x7f28580c8a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38120/0 (socket says 172.21.15.139:38120) 2024-09-16T00:09:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.315+0000 7f285d400640 1 -- 172.21.15.139:0/3600426404 learned_addr learned my addr 172.21.15.139:0/3600426404 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.315+0000 7f285e402640 1 --2- 172.21.15.139:0/3600426404 >> v2:172.21.15.164:3300/0 conn(0x7f285806f360 0x7f28580cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.315+0000 7f285dc01640 1 --2- 172.21.15.139:0/3600426404 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2858154aa0 0x7f285806bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.315+0000 7f285cbff640 1 -- 172.21.15.139:0/3600426404 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3486422299 0 0) 0x7f285806f830 con 0x7f285806eae0 2024-09-16T00:09:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.315+0000 7f285cbff640 1 -- 172.21.15.139:0/3600426404 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2844003520 con 0x7f285806eae0 2024-09-16T00:09:29.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.315+0000 7f285e402640 1 -- 172.21.15.139:0/3600426404 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2858154aa0 msgr2=0x7f285806bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:29.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.315+0000 7f285e402640 1 --2- 172.21.15.139:0/3600426404 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2858154aa0 0x7f285806bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.315+0000 7f285e402640 1 -- 172.21.15.139:0/3600426404 >> v1:172.21.15.139:6789/0 conn(0x7f285806eae0 legacy=0x7f28580c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.315+0000 7f285e402640 1 -- 172.21.15.139:0/3600426404 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285806fdf0 con 0x7f285806f360 2024-09-16T00:09:29.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.316+0000 7f285dc01640 1 --2- 172.21.15.139:0/3600426404 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2858154aa0 0x7f285806bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:29.320 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:09:29.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.316+0000 7f285e402640 1 --2- 172.21.15.139:0/3600426404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f285806f360 0x7f28580cd130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2850009d90 tx=0x7f285002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28d17a687e3188e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:29.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.316+0000 7f285cbff640 1 -- 172.21.15.139:0/3600426404 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850007740 con 0x7f285806f360 2024-09-16T00:09:29.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.316+0000 7f285cbff640 1 -- 172.21.15.139:0/3600426404 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28500078e0 con 0x7f285806f360 2024-09-16T00:09:29.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.316+0000 7f285cbff640 1 -- 172.21.15.139:0/3600426404 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850007c90 con 0x7f285806f360 2024-09-16T00:09:29.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.317+0000 7f285fe65640 1 -- 172.21.15.139:0/3600426404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f285806f360 msgr2=0x7f28580cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.317+0000 7f285fe65640 1 --2- 172.21.15.139:0/3600426404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f285806f360 0x7f28580cd130 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2850009d90 tx=0x7f285002a6f0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.317+0000 7f285fe65640 1 -- 172.21.15.139:0/3600426404 shutdown_connections 2024-09-16T00:09:29.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.317+0000 7f285fe65640 1 --2- 172.21.15.139:0/3600426404 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f285806f360 0x7f28580cd130 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.317+0000 7f285fe65640 1 --2- 172.21.15.139:0/3600426404 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2858154aa0 0x7f285806bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.317+0000 7f285fe65640 1 -- 172.21.15.139:0/3600426404 >> 172.21.15.139:0/3600426404 conn(0x7f2858072b30 msgr2=0x7f2858070510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:29.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.317+0000 7f285fe65640 1 -- 172.21.15.139:0/3600426404 shutdown_connections 2024-09-16T00:09:29.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.318+0000 7f285fe65640 1 -- 172.21.15.139:0/3600426404 wait complete. 2024-09-16T00:09:29.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.319+0000 7f285fe65640 1 Processor -- start 2024-09-16T00:09:29.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.319+0000 7f285fe65640 1 -- start start 2024-09-16T00:09:29.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.320+0000 7f285fe65640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f285806eae0 0x7f2858153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.320+0000 7f285fe65640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f285806f360 0x7f2858155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.320+0000 7f285fe65640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2858154aa0 0x7f285814ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.320+0000 7f285fe65640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2858154e80 con 0x7f285806f360 2024-09-16T00:09:29.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.320+0000 7f285fe65640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f285806c1f0 con 0x7f285806eae0 2024-09-16T00:09:29.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.320+0000 7f285fe65640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f285815ddf0 con 0x7f2858154aa0 2024-09-16T00:09:29.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.320+0000 7f285dc01640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f285806eae0 0x7f2858153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.320+0000 7f285dc01640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f285806eae0 0x7f2858153420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:55288/0 (socket says 172.21.15.139:55288) 2024-09-16T00:09:29.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.320+0000 7f285dc01640 1 -- 172.21.15.139:0/2172187411 learned_addr learned my addr 172.21.15.139:0/2172187411 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:29.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.320+0000 7f285e402640 1 --2- 172.21.15.139:0/2172187411 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2858154aa0 0x7f285814ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.320+0000 7f285d400640 1 --2- 172.21.15.139:0/2172187411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f285806f360 0x7f2858155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.320+0000 7f285dc01640 1 -- 172.21.15.139:0/2172187411 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2858154aa0 msgr2=0x7f285814ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.320+0000 7f285dc01640 1 --2- 172.21.15.139:0/2172187411 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2858154aa0 0x7f285814ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.320+0000 7f285dc01640 1 -- 172.21.15.139:0/2172187411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f285806f360 msgr2=0x7f2858155f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.320+0000 7f285dc01640 1 --2- 172.21.15.139:0/2172187411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f285806f360 0x7f2858155f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.320+0000 7f285dc01640 1 -- 172.21.15.139:0/2172187411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285814d290 con 0x7f285806eae0 2024-09-16T00:09:29.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.321+0000 7f285d400640 1 --2- 172.21.15.139:0/2172187411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f285806f360 0x7f2858155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:09:29.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.321+0000 7f285dc01640 1 --2- 172.21.15.139:0/2172187411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f285806eae0 0x7f2858153420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f285400e960 tx=0x7f285400ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:29.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.321+0000 7f284a7fc640 1 -- 172.21.15.139:0/2172187411 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854017070 con 0x7f285806eae0 2024-09-16T00:09:29.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.321+0000 7f284a7fc640 1 -- 172.21.15.139:0/2172187411 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f285400ca00 con 0x7f285806eae0 2024-09-16T00:09:29.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.321+0000 7f285fe65640 1 -- 172.21.15.139:0/2172187411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285814cf90 con 0x7f285806eae0 2024-09-16T00:09:29.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.321+0000 7f284a7fc640 1 -- 172.21.15.139:0/2172187411 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285400cdd0 con 0x7f285806eae0 2024-09-16T00:09:29.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.321+0000 7f285fe65640 1 -- 172.21.15.139:0/2172187411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28581f3c20 con 0x7f285806eae0 2024-09-16T00:09:29.326 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.322+0000 7f285fe65640 1 -- 172.21.15.139:0/2172187411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f285806bcb0 con 0x7f285806eae0 2024-09-16T00:09:29.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.323+0000 7f284a7fc640 1 -- 172.21.15.139:0/2172187411 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f285401a070 con 0x7f285806eae0 2024-09-16T00:09:29.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.323+0000 7f284a7fc640 1 --2- 172.21.15.139:0/2172187411 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f283c06c2e0 0x7f283c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.323+0000 7f284a7fc640 1 -- 172.21.15.139:0/2172187411 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5944+0+0 (secure 0 0 0) 0x7f285408b3e0 con 0x7f285806eae0 2024-09-16T00:09:29.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.323+0000 7f285d400640 1 --2- 172.21.15.139:0/2172187411 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f283c06c2e0 0x7f283c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.324+0000 7f285d400640 1 --2- 172.21.15.139:0/2172187411 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f283c06c2e0 0x7f283c06e7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f284c007fd0 tx=0x7f284c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:29.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.327+0000 7f284a7fc640 1 -- 172.21.15.139:0/2172187411 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f285814cf90 con 0x7f285806eae0 2024-09-16T00:09:29.392 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:29.391+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:29.435 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:29.435 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:29.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:29.435 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:29.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.435 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:29.436 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:29.436 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 126.351989093, 2024-09-16T00:09:29.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:09:29.436 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.436 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.436 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.436 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.436 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.436 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.436 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:09:29.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.436 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:29.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:29.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:09:29.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:09:29.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:09:29.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:09:29.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:09:29.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:09:29.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:09:29.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:09:29.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:09:29.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:09:29.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:09:29.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:09:29.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:09:29.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:09:29.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:09:29.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:09:29.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:09:29.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:09:29.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:09:29.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:09:29.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:09:29.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:09:29.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:09:29.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:09:29.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:09:29.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:09:29.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:09:29.443 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.444 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.444 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:106 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:29.444 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:09:29.444 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.27245746299999, 2024-09-16T00:09:29.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093071190000000008, 2024-09-16T00:09:29.444 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.444 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.444 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.444 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.445 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 106 2024-09-16T00:09:29.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.445 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:09:29.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160361+0000", 2024-09-16T00:09:29.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160362+0000", 2024-09-16T00:09:29.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0699999999999997e-07 2024-09-16T00:09:29.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160438+0000", 2024-09-16T00:09:29.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6711000000000005e-05 2024-09-16T00:09:29.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160444+0000", 2024-09-16T00:09:29.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0530000000000003e-06 2024-09-16T00:09:29.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160449+0000", 2024-09-16T00:09:29.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2189999999999998e-06 2024-09-16T00:09:29.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161270+0000", 2024-09-16T00:09:29.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00082099199999999999 2024-09-16T00:09:29.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161348+0000", 2024-09-16T00:09:29.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8361000000000004e-05 2024-09-16T00:09:29.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161472+0000", 2024-09-16T00:09:29.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123523 2024-09-16T00:09:29.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163461+0000", 2024-09-16T00:09:29.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019891560000000002 2024-09-16T00:09:29.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169418+0000", 2024-09-16T00:09:29.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059568490000000002 2024-09-16T00:09:29.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169570+0000", 2024-09-16T00:09:29.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152461 2024-09-16T00:09:29.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169668+0000", 2024-09-16T00:09:29.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8187e-05 2024-09-16T00:09:29.451 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.451 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.451 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.451 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:107 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:29.451 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:09:29.451 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.272361692, 2024-09-16T00:09:29.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092931679999999992, 2024-09-16T00:09:29.452 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.452 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.452 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.452 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.452 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.452 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.452 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 107 2024-09-16T00:09:29.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.452 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:09:29.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:09:29.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160457+0000", 2024-09-16T00:09:29.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0699999999999998e-07 2024-09-16T00:09:29.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160459+0000", 2024-09-16T00:09:29.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0370000000000001e-06 2024-09-16T00:09:29.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160463+0000", 2024-09-16T00:09:29.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999999e-06 2024-09-16T00:09:29.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160466+0000", 2024-09-16T00:09:29.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.644e-06 2024-09-16T00:09:29.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161873+0000", 2024-09-16T00:09:29.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014074840000000001 2024-09-16T00:09:29.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161949+0000", 2024-09-16T00:09:29.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5749999999999998e-05 2024-09-16T00:09:29.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162069+0000", 2024-09-16T00:09:29.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120257 2024-09-16T00:09:29.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163506+0000", 2024-09-16T00:09:29.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001436344 2024-09-16T00:09:29.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169704+0000", 2024-09-16T00:09:29.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061982829999999997 2024-09-16T00:09:29.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169730+0000", 2024-09-16T00:09:29.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6060000000000001e-05 2024-09-16T00:09:29.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169750+0000", 2024-09-16T00:09:29.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0092e-05 2024-09-16T00:09:29.463 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.463 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.463 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.464 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:108 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:29.464 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:09:29.464 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.27225238, 2024-09-16T00:09:29.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092280039999999997, 2024-09-16T00:09:29.464 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.465 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.465 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.465 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.465 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.465 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.465 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 108 2024-09-16T00:09:29.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.466 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:09:29.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160566+0000", 2024-09-16T00:09:29.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160567+0000", 2024-09-16T00:09:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7300000000000001e-07 2024-09-16T00:09:29.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160569+0000", 2024-09-16T00:09:29.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5299999999999999e-06 2024-09-16T00:09:29.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160574+0000", 2024-09-16T00:09:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9289999999999997e-06 2024-09-16T00:09:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160577+0000", 2024-09-16T00:09:29.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.01e-06 2024-09-16T00:09:29.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162518+0000", 2024-09-16T00:09:29.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001940703 2024-09-16T00:09:29.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162571+0000", 2024-09-16T00:09:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2670000000000002e-05 2024-09-16T00:09:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.162647+0000", 2024-09-16T00:09:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6778000000000002e-05 2024-09-16T00:09:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163544+0000", 2024-09-16T00:09:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00089702599999999999 2024-09-16T00:09:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169754+0000", 2024-09-16T00:09:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062091739999999996 2024-09-16T00:09:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169776+0000", 2024-09-16T00:09:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2391999999999998e-05 2024-09-16T00:09:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169794+0000", 2024-09-16T00:09:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8318999999999999e-05 2024-09-16T00:09:29.477 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:09:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.272094481, 2024-09-16T00:09:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092871919999999997, 2024-09-16T00:09:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.478 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 110 2024-09-16T00:09:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:09:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:09:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160724+0000", 2024-09-16T00:09:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:09:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160728+0000", 2024-09-16T00:09:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0620000000000001e-06 2024-09-16T00:09:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160731+0000", 2024-09-16T00:09:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6500000000000002e-06 2024-09-16T00:09:29.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160734+0000", 2024-09-16T00:09:29.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.762e-06 2024-09-16T00:09:29.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163580+0000", 2024-09-16T00:09:29.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028464549999999999 2024-09-16T00:09:29.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163657+0000", 2024-09-16T00:09:29.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6695999999999999e-05 2024-09-16T00:09:29.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.163798+0000", 2024-09-16T00:09:29.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014055 2024-09-16T00:09:29.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165483+0000", 2024-09-16T00:09:29.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016853090000000001 2024-09-16T00:09:29.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169970+0000", 2024-09-16T00:09:29.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00448709 2024-09-16T00:09:29.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169994+0000", 2024-09-16T00:09:29.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3575999999999999e-05 2024-09-16T00:09:29.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170011+0000", 2024-09-16T00:09:29.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7702000000000001e-05 2024-09-16T00:09:29.483 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.483 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.483 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.484 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:29.484 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:09:29.484 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.272080812, 2024-09-16T00:09:29.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093174859999999998, 2024-09-16T00:09:29.484 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.484 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.484 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.484 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.484 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.484 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.484 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 111 2024-09-16T00:09:29.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.485 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:09:29.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:09:29.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160738+0000", 2024-09-16T00:09:29.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1e-07 2024-09-16T00:09:29.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160741+0000", 2024-09-16T00:09:29.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6110000000000001e-06 2024-09-16T00:09:29.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160744+0000", 2024-09-16T00:09:29.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3299999999999999e-06 2024-09-16T00:09:29.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160746+0000", 2024-09-16T00:09:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5469999999999999e-06 2024-09-16T00:09:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164069+0000", 2024-09-16T00:09:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033222709999999999 2024-09-16T00:09:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164113+0000", 2024-09-16T00:09:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4212999999999998e-05 2024-09-16T00:09:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164187+0000", 2024-09-16T00:09:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4136999999999998e-05 2024-09-16T00:09:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166429+0000", 2024-09-16T00:09:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022423180000000001 2024-09-16T00:09:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170016+0000", 2024-09-16T00:09:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035869449999999998 2024-09-16T00:09:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170038+0000", 2024-09-16T00:09:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2053e-05 2024-09-16T00:09:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170055+0000", 2024-09-16T00:09:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6850999999999998e-05 2024-09-16T00:09:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:29.489 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:09:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.272067097, 2024-09-16T00:09:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093480279999999995, 2024-09-16T00:09:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 112 2024-09-16T00:09:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:09:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160751+0000", 2024-09-16T00:09:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160752+0000", 2024-09-16T00:09:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2700000000000001e-07 2024-09-16T00:09:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160808+0000", 2024-09-16T00:09:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5797999999999997e-05 2024-09-16T00:09:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160814+0000", 2024-09-16T00:09:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5980000000000002e-06 2024-09-16T00:09:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160819+0000", 2024-09-16T00:09:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8960000000000002e-06 2024-09-16T00:09:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164468+0000", 2024-09-16T00:09:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003649476 2024-09-16T00:09:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164525+0000", 2024-09-16T00:09:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6227999999999998e-05 2024-09-16T00:09:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.164595+0000", 2024-09-16T00:09:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0759000000000005e-05 2024-09-16T00:09:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167237+0000", 2024-09-16T00:09:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026410470000000001 2024-09-16T00:09:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170060+0000", 2024-09-16T00:09:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028234969999999999 2024-09-16T00:09:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170081+0000", 2024-09-16T00:09:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1242000000000001e-05 2024-09-16T00:09:29.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170100+0000", 2024-09-16T00:09:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8260000000000001e-05 2024-09-16T00:09:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:09:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.271990002, 2024-09-16T00:09:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093139409999999992, 2024-09-16T00:09:29.495 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 113 2024-09-16T00:09:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:09:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:09:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160829+0000", 2024-09-16T00:09:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8100000000000003e-07 2024-09-16T00:09:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160833+0000", 2024-09-16T00:09:29.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.557e-06 2024-09-16T00:09:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160839+0000", 2024-09-16T00:09:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3829999999999997e-06 2024-09-16T00:09:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160844+0000", 2024-09-16T00:09:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6909999999999996e-06 2024-09-16T00:09:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165008+0000", 2024-09-16T00:09:29.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041642129999999999 2024-09-16T00:09:29.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165085+0000", 2024-09-16T00:09:29.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6651999999999997e-05 2024-09-16T00:09:29.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165205+0000", 2024-09-16T00:09:29.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012045900000000001 2024-09-16T00:09:29.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167283+0000", 2024-09-16T00:09:29.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020782909999999999 2024-09-16T00:09:29.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170104+0000", 2024-09-16T00:09:29.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028207950000000001 2024-09-16T00:09:29.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170125+0000", 2024-09-16T00:09:29.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0852e-05 2024-09-16T00:09:29.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170143+0000", 2024-09-16T00:09:29.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7467000000000001e-05 2024-09-16T00:09:29.501 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.501 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.501 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.501 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:29.501 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:09:29.501 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.27194351999999, 2024-09-16T00:09:29.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093126960000000005, 2024-09-16T00:09:29.501 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.501 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.501 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.502 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.502 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.502 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.502 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 114 2024-09-16T00:09:29.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.502 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:09:29.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160875+0000", 2024-09-16T00:09:29.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160876+0000", 2024-09-16T00:09:29.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5700000000000002e-07 2024-09-16T00:09:29.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160880+0000", 2024-09-16T00:09:29.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5539999999999999e-06 2024-09-16T00:09:29.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160888+0000", 2024-09-16T00:09:29.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.157e-06 2024-09-16T00:09:29.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160894+0000", 2024-09-16T00:09:29.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7690000000000002e-06 2024-09-16T00:09:29.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165519+0000", 2024-09-16T00:09:29.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046249539999999997 2024-09-16T00:09:29.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165593+0000", 2024-09-16T00:09:29.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4039e-05 2024-09-16T00:09:29.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165715+0000", 2024-09-16T00:09:29.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012131899999999999 2024-09-16T00:09:29.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167309+0000", 2024-09-16T00:09:29.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001594011 2024-09-16T00:09:29.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170147+0000", 2024-09-16T00:09:29.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028387130000000001 2024-09-16T00:09:29.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170170+0000", 2024-09-16T00:09:29.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2517000000000001e-05 2024-09-16T00:09:29.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170188+0000", 2024-09-16T00:09:29.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8006e-05 2024-09-16T00:09:29.506 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.507 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.507 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.507 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:29.507 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:09:29.507 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.271902492, 2024-09-16T00:09:29.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093140040000000007, 2024-09-16T00:09:29.507 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.507 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.507 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.507 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.507 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.508 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.508 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 115 2024-09-16T00:09:29.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.508 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:09:29.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160916+0000", 2024-09-16T00:09:29.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160917+0000", 2024-09-16T00:09:29.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7599999999999996e-07 2024-09-16T00:09:29.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.160992+0000", 2024-09-16T00:09:29.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4760999999999998e-05 2024-09-16T00:09:29.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161000+0000", 2024-09-16T00:09:29.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9829999999999996e-06 2024-09-16T00:09:29.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161007+0000", 2024-09-16T00:09:29.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4909999999999999e-06 2024-09-16T00:09:29.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.165983+0000", 2024-09-16T00:09:29.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049755399999999997 2024-09-16T00:09:29.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166027+0000", 2024-09-16T00:09:29.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4109000000000001e-05 2024-09-16T00:09:29.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166100+0000", 2024-09-16T00:09:29.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3360000000000002e-05 2024-09-16T00:09:29.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169698+0000", 2024-09-16T00:09:29.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003597795 2024-09-16T00:09:29.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170192+0000", 2024-09-16T00:09:29.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00049447300000000005 2024-09-16T00:09:29.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170214+0000", 2024-09-16T00:09:29.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1546e-05 2024-09-16T00:09:29.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170230+0000", 2024-09-16T00:09:29.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6169999999999999e-05 2024-09-16T00:09:29.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.512 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.513 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.513 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:29.513 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:09:29.513 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.27179276299999, 2024-09-16T00:09:29.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092457029999999992, 2024-09-16T00:09:29.513 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.513 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.513 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.513 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.513 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 116 2024-09-16T00:09:29.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.514 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:09:29.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:09:29.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161026+0000", 2024-09-16T00:09:29.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7899999999999999e-07 2024-09-16T00:09:29.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161029+0000", 2024-09-16T00:09:29.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6630000000000002e-06 2024-09-16T00:09:29.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161036+0000", 2024-09-16T00:09:29.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8669999999999996e-06 2024-09-16T00:09:29.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161041+0000", 2024-09-16T00:09:29.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:09:29.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166468+0000", 2024-09-16T00:09:29.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054266050000000001 2024-09-16T00:09:29.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166511+0000", 2024-09-16T00:09:29.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3869999999999998e-05 2024-09-16T00:09:29.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166606+0000", 2024-09-16T00:09:29.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4437000000000004e-05 2024-09-16T00:09:29.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.169955+0000", 2024-09-16T00:09:29.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033494810000000001 2024-09-16T00:09:29.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170234+0000", 2024-09-16T00:09:29.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00027900899999999998 2024-09-16T00:09:29.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170255+0000", 2024-09-16T00:09:29.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0537999999999999e-05 2024-09-16T00:09:29.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170272+0000", 2024-09-16T00:09:29.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6611000000000001e-05 2024-09-16T00:09:29.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.518 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.519 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:29.519 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:09:29.519 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 125.271767618, 2024-09-16T00:09:29.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093511010000000006, 2024-09-16T00:09:29.519 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.519 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.519 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.519 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.519 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 117 2024-09-16T00:09:29.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.519 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:09:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:09:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161051+0000", 2024-09-16T00:09:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.63e-07 2024-09-16T00:09:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161055+0000", 2024-09-16T00:09:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6770000000000001e-06 2024-09-16T00:09:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161063+0000", 2024-09-16T00:09:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6469999999999998e-06 2024-09-16T00:09:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.161068+0000", 2024-09-16T00:09:29.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.746e-06 2024-09-16T00:09:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166817+0000", 2024-09-16T00:09:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057494959999999998 2024-09-16T00:09:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.166893+0000", 2024-09-16T00:09:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6210000000000004e-05 2024-09-16T00:09:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.167019+0000", 2024-09-16T00:09:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125585 2024-09-16T00:09:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170275+0000", 2024-09-16T00:09:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032563620000000001 2024-09-16T00:09:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170309+0000", 2024-09-16T00:09:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3865999999999997e-05 2024-09-16T00:09:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170362+0000", 2024-09-16T00:09:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3094999999999999e-05 2024-09-16T00:09:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.170402+0000", 2024-09-16T00:09:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9953999999999997e-05 2024-09-16T00:09:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:09:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.823035533, 2024-09-16T00:09:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097364389999999995, 2024-09-16T00:09:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 138 2024-09-16T00:09:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:09:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609783+0000", 2024-09-16T00:09:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609784+0000", 2024-09-16T00:09:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5899999999999996e-07 2024-09-16T00:09:29.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609787+0000", 2024-09-16T00:09:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7280000000000001e-06 2024-09-16T00:09:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609794+0000", 2024-09-16T00:09:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6409999999999996e-06 2024-09-16T00:09:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609799+0000", 2024-09-16T00:09:29.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9359999999999998e-06 2024-09-16T00:09:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615759+0000", 2024-09-16T00:09:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059599910000000004 2024-09-16T00:09:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615806+0000", 2024-09-16T00:09:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7290000000000003e-05 2024-09-16T00:09:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.615877+0000", 2024-09-16T00:09:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1000000000000005e-05 2024-09-16T00:09:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618961+0000", 2024-09-16T00:09:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030841390000000001 2024-09-16T00:09:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619436+0000", 2024-09-16T00:09:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00047452799999999999 2024-09-16T00:09:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619482+0000", 2024-09-16T00:09:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5689000000000003e-05 2024-09-16T00:09:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619519+0000", 2024-09-16T00:09:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7938000000000002e-05 2024-09-16T00:09:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:29.530 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:09:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.823011819, 2024-09-16T00:09:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011812954000000001, 2024-09-16T00:09:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 139 2024-09-16T00:09:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:09:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:09:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609807+0000", 2024-09-16T00:09:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8099999999999998e-07 2024-09-16T00:09:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609810+0000", 2024-09-16T00:09:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2100000000000002e-06 2024-09-16T00:09:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609817+0000", 2024-09-16T00:09:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.28e-06 2024-09-16T00:09:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609825+0000", 2024-09-16T00:09:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4339999999999992e-06 2024-09-16T00:09:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616116+0000", 2024-09-16T00:09:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006290543 2024-09-16T00:09:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616187+0000", 2024-09-16T00:09:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1830999999999997e-05 2024-09-16T00:09:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616298+0000", 2024-09-16T00:09:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011044699999999999 2024-09-16T00:09:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619559+0000", 2024-09-16T00:09:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032614710000000002 2024-09-16T00:09:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621562+0000", 2024-09-16T00:09:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020028680000000001 2024-09-16T00:09:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621597+0000", 2024-09-16T00:09:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4625000000000001e-05 2024-09-16T00:09:29.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621620+0000", 2024-09-16T00:09:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2864e-05 2024-09-16T00:09:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:09:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.822984023, 2024-09-16T00:09:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011828211999999999, 2024-09-16T00:09:29.536 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 140 2024-09-16T00:09:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:09:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:09:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609835+0000", 2024-09-16T00:09:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.75e-07 2024-09-16T00:09:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961000000000003e-05 2024-09-16T00:09:29.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609892+0000", 2024-09-16T00:09:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0516000000000001e-05 2024-09-16T00:09:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609898+0000", 2024-09-16T00:09:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0449999999999997e-06 2024-09-16T00:09:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616797+0000", 2024-09-16T00:09:29.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068998840000000002 2024-09-16T00:09:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616844+0000", 2024-09-16T00:09:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6501000000000001e-05 2024-09-16T00:09:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.616935+0000", 2024-09-16T00:09:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1222000000000002e-05 2024-09-16T00:09:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619808+0000", 2024-09-16T00:09:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002873198 2024-09-16T00:09:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621624+0000", 2024-09-16T00:09:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001815933 2024-09-16T00:09:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621646+0000", 2024-09-16T00:09:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1622999999999999e-05 2024-09-16T00:09:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.621663+0000", 2024-09-16T00:09:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6854000000000002e-05 2024-09-16T00:09:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.82289821400001, 2024-09-16T00:09:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014331501999999999, 2024-09-16T00:09:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.542 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-16T00:09:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609921+0000", 2024-09-16T00:09:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9599999999999999e-07 2024-09-16T00:09:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609925+0000", 2024-09-16T00:09:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T00:09:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609932+0000", 2024-09-16T00:09:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8240000000000002e-06 2024-09-16T00:09:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609937+0000", 2024-09-16T00:09:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1070000000000004e-06 2024-09-16T00:09:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618987+0000", 2024-09-16T00:09:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090502719999999998 2024-09-16T00:09:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619064+0000", 2024-09-16T00:09:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6868999999999999e-05 2024-09-16T00:09:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619183+0000", 2024-09-16T00:09:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011961 2024-09-16T00:09:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.620052+0000", 2024-09-16T00:09:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086822799999999997 2024-09-16T00:09:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624197+0000", 2024-09-16T00:09:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041454930000000001 2024-09-16T00:09:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624229+0000", 2024-09-16T00:09:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1758999999999999e-05 2024-09-16T00:09:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624252+0000", 2024-09-16T00:09:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:09:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.547 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:09:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.350132406, 2024-09-16T00:09:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010093276999999999, 2024-09-16T00:09:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.548 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 157 2024-09-16T00:09:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:09:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:09:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082686+0000", 2024-09-16T00:09:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4699999999999998e-07 2024-09-16T00:09:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082688+0000", 2024-09-16T00:09:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0209999999999998e-06 2024-09-16T00:09:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082692+0000", 2024-09-16T00:09:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6890000000000002e-06 2024-09-16T00:09:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082695+0000", 2024-09-16T00:09:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5339999999999998e-06 2024-09-16T00:09:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085303+0000", 2024-09-16T00:09:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026084789999999999 2024-09-16T00:09:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085374+0000", 2024-09-16T00:09:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1317999999999997e-05 2024-09-16T00:09:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085506+0000", 2024-09-16T00:09:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013118 2024-09-16T00:09:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086928+0000", 2024-09-16T00:09:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0014224559999999999 2024-09-16T00:09:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092665+0000", 2024-09-16T00:09:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057369830000000002 2024-09-16T00:09:29.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092727+0000", 2024-09-16T00:09:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1810000000000006e-05 2024-09-16T00:09:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092779+0000", 2024-09-16T00:09:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2559999999999998e-05 2024-09-16T00:09:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:09:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.350120238, 2024-09-16T00:09:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010176388, 2024-09-16T00:09:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 158 2024-09-16T00:09:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:09:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082698+0000", 2024-09-16T00:09:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082699+0000", 2024-09-16T00:09:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.11e-07 2024-09-16T00:09:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082742+0000", 2024-09-16T00:09:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3699000000000003e-05 2024-09-16T00:09:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082748+0000", 2024-09-16T00:09:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8690000000000001e-06 2024-09-16T00:09:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082751+0000", 2024-09-16T00:09:29.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.942e-06 2024-09-16T00:09:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085881+0000", 2024-09-16T00:09:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031303450000000001 2024-09-16T00:09:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.085959+0000", 2024-09-16T00:09:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7874999999999996e-05 2024-09-16T00:09:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086086+0000", 2024-09-16T00:09:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012685499999999999 2024-09-16T00:09:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087127+0000", 2024-09-16T00:09:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001041298 2024-09-16T00:09:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092790+0000", 2024-09-16T00:09:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056628 2024-09-16T00:09:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092832+0000", 2024-09-16T00:09:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2227000000000001e-05 2024-09-16T00:09:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092875+0000", 2024-09-16T00:09:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2267e-05 2024-09-16T00:09:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:29.559 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:09:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 124.3500622, 2024-09-16T00:09:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010199547, 2024-09-16T00:09:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 159 2024-09-16T00:09:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:09:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082756+0000", 2024-09-16T00:09:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082757+0000", 2024-09-16T00:09:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2500000000000001e-07 2024-09-16T00:09:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082759+0000", 2024-09-16T00:09:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0660000000000002e-06 2024-09-16T00:09:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082763+0000", 2024-09-16T00:09:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4889999999999997e-06 2024-09-16T00:09:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.082766+0000", 2024-09-16T00:09:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:09:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086571+0000", 2024-09-16T00:09:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038053269999999998 2024-09-16T00:09:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086619+0000", 2024-09-16T00:09:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7924999999999999e-05 2024-09-16T00:09:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.086693+0000", 2024-09-16T00:09:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000005e-05 2024-09-16T00:09:29.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.087383+0000", 2024-09-16T00:09:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00069008299999999997 2024-09-16T00:09:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092882+0000", 2024-09-16T00:09:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054987450000000002 2024-09-16T00:09:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092921+0000", 2024-09-16T00:09:29.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9388999999999998e-05 2024-09-16T00:09:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:25.092956+0000", 2024-09-16T00:09:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4647999999999998e-05 2024-09-16T00:09:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:161 2.b 2:d5c7a900:::200.00000003:head [write 0~3935733 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e64)", 2024-09-16T00:09:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:07.538495+0000", 2024-09-16T00:09:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.89432356, 2024-09-16T00:09:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012844983000000001, 2024-09-16T00:09:29.565 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.538495+0000", 2024-09-16T00:09:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.538495+0000", 2024-09-16T00:09:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.538497+0000", 2024-09-16T00:09:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5149999999999999e-06 2024-09-16T00:09:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540617+0000", 2024-09-16T00:09:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021204079999999998 2024-09-16T00:09:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540640+0000", 2024-09-16T00:09:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3167e-05 2024-09-16T00:09:29.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540650+0000", 2024-09-16T00:09:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6099999999999995e-06 2024-09-16T00:09:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540724+0000", 2024-09-16T00:09:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4340999999999999e-05 2024-09-16T00:09:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540812+0000", 2024-09-16T00:09:29.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8376999999999998e-05 2024-09-16T00:09:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540957+0000", 2024-09-16T00:09:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014463 2024-09-16T00:09:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.550270+0000", 2024-09-16T00:09:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093128380000000004 2024-09-16T00:09:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:29.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551270+0000", 2024-09-16T00:09:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00100015 2024-09-16T00:09:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551309+0000", 2024-09-16T00:09:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8526999999999999e-05 2024-09-16T00:09:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551340+0000", 2024-09-16T00:09:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1420000000000001e-05 2024-09-16T00:09:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.570 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.571 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:29.571 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:29.571 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.465+0000 7f536998f640 1 -- 172.21.15.139:0/2272375969 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f536406db80 con 0x7f53640c7b90 2024-09-16T00:09:29.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.466+0000 7f534bfff640 1 -- 172.21.15.139:0/2272375969 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v76) ==== 81+0+35497 (secure 0 0 0) 0x7f535805d7b0 con 0x7f53640c7b90 2024-09-16T00:09:29.573 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:09:29.573 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:29.573 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 76, 2024-09-16T00:09:29.573 INFO:teuthology.orchestra.run.smithi139.stdout: "fsid": "2d3366e5-6a59-4b59-93b2-615e72ac7015", 2024-09-16T00:09:29.573 INFO:teuthology.orchestra.run.smithi139.stdout: "created": "2024-09-16T00:07:09.967833+0000", 2024-09-16T00:09:29.574 INFO:teuthology.orchestra.run.smithi139.stdout: "modified": "2024-09-16T00:09:27.129287+0000", 2024-09-16T00:09:29.574 INFO:teuthology.orchestra.run.smithi139.stdout: "last_up_change": "2024-09-16T00:07:21.049678+0000", 2024-09-16T00:09:29.574 INFO:teuthology.orchestra.run.smithi139.stdout: "last_in_change": "2024-09-16T00:07:12.025620+0000", 2024-09-16T00:09:29.574 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T00:09:29.574 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_num": 5799936, 2024-09-16T00:09:29.574 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_set": [ 2024-09-16T00:09:29.574 INFO:teuthology.orchestra.run.smithi139.stdout: "pglog_hardlimit", 2024-09-16T00:09:29.574 INFO:teuthology.orchestra.run.smithi139.stdout: "purged_snapdirs", 2024-09-16T00:09:29.574 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_deletes", 2024-09-16T00:09:29.574 INFO:teuthology.orchestra.run.smithi139.stdout: "sortbitwise" 2024-09-16T00:09:29.574 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:29.574 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_version": 6, 2024-09-16T00:09:29.574 INFO:teuthology.orchestra.run.smithi139.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T00:09:29.575 INFO:teuthology.orchestra.run.smithi139.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T00:09:29.575 INFO:teuthology.orchestra.run.smithi139.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T00:09:29.575 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_snapshot": "", 2024-09-16T00:09:29.575 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_max": 3, 2024-09-16T00:09:29.575 INFO:teuthology.orchestra.run.smithi139.stdout: "max_osd": 8, 2024-09-16T00:09:29.575 INFO:teuthology.orchestra.run.smithi139.stdout: "require_min_compat_client": "luminous", 2024-09-16T00:09:29.575 INFO:teuthology.orchestra.run.smithi139.stdout: "min_compat_client": "luminous", 2024-09-16T00:09:29.575 INFO:teuthology.orchestra.run.smithi139.stdout: "require_osd_release": "squid", 2024-09-16T00:09:29.575 INFO:teuthology.orchestra.run.smithi139.stdout: "allow_crimson": false, 2024-09-16T00:09:29.575 INFO:teuthology.orchestra.run.smithi139.stdout: "pools": [ 2024-09-16T00:09:29.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.575 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 1, 2024-09-16T00:09:29.575 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": ".mgr", 2024-09-16T00:09:29.575 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:22.056303+0000", 2024-09-16T00:09:29.576 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:09:29.576 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:09:29.576 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:09:29.576 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:09:29.576 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:09:29.576 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:09:29.576 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:09:29.576 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:09:29.576 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:09:29.576 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:09:29.576 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:09:29.576 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:09:29.576 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:09:29.576 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 1, 2024-09-16T00:09:29.577 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 1, 2024-09-16T00:09:29.577 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 1, 2024-09-16T00:09:29.577 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 1, 2024-09-16T00:09:29.577 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 1, 2024-09-16T00:09:29.577 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:09:29.577 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:09:29.577 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:09:29.577 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:09:29.577 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:09:29.577 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:09:29.577 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:09:29.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.577 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "18", 2024-09-16T00:09:29.577 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:09:29.578 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:09:29.578 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:09:29.578 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:09:29.578 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:09:29.578 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:09:29.578 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:09:29.578 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:09:29.578 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:09:29.578 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:09:29.578 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:09:29.578 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:09:29.578 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:09:29.578 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:09:29.578 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:09:29.579 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:09:29.579 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:09:29.579 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:09:29.579 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:09:29.579 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:09:29.579 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:09:29.579 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:09:29.579 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:09:29.579 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:09:29.579 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:09:29.579 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:09:29.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.579 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:09:29.579 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:09:29.580 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:09:29.580 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:09:29.580 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:09:29.580 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:09:29.580 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:09:29.580 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:09:29.580 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:09:29.580 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:09:29.581 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:09:29.581 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:09:29.581 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_max": 32, 2024-09-16T00:09:29.581 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 1 2024-09-16T00:09:29.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.581 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:09:29.581 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": {} 2024-09-16T00:09:29.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.581 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:09:29.582 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:09:29.582 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 8, 2024-09-16T00:09:29.582 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 8, 2024-09-16T00:09:29.582 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 0.25, 2024-09-16T00:09:29.582 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 2, 2024-09-16T00:09:29.582 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 2, 2024-09-16T00:09:29.582 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:09:29.582 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:09:29.582 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:09:29.582 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.583 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 2, 2024-09-16T00:09:29.583 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": "cephfs_metadata", 2024-09-16T00:09:29.583 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:30.768649+0000", 2024-09-16T00:09:29.583 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:09:29.583 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:09:29.583 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:09:29.583 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:09:29.583 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:09:29.583 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:09:29.583 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:09:29.583 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:09:29.584 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:09:29.584 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:09:29.584 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:09:29.584 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:09:29.584 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:09:29.584 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 32, 2024-09-16T00:09:29.584 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 32, 2024-09-16T00:09:29.584 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 32, 2024-09-16T00:09:29.584 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 32, 2024-09-16T00:09:29.584 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 32, 2024-09-16T00:09:29.584 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:09:29.584 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:09:29.584 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:09:29.585 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:09:29.585 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:09:29.585 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:09:29.585 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:09:29.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.585 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "21", 2024-09-16T00:09:29.585 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:09:29.585 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:09:29.585 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:09:29.585 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:09:29.585 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:09:29.585 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:09:29.585 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:09:29.585 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:09:29.586 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:09:29.586 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:09:29.586 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:09:29.586 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:09:29.586 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:09:29.586 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:09:29.586 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:09:29.586 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:09:29.586 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:09:29.586 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:09:29.586 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:09:29.586 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:09:29.586 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:09:29.586 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:09:29.587 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:09:29.587 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:09:29.587 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:09:29.587 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:09:29.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.587 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:09:29.587 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:09:29.587 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:09:29.587 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:09:29.587 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:09:29.587 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:09:29.587 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:09:29.587 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:09:29.587 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:09:29.587 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:09:29.588 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:09:29.588 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:09:29.588 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_bias": 4, 2024-09-16T00:09:29.588 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 16, 2024-09-16T00:09:29.588 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_priority": 5 2024-09-16T00:09:29.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.588 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:09:29.588 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs": { 2024-09-16T00:09:29.588 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": "cephfs" 2024-09-16T00:09:29.588 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.588 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:09:29.588 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:09:29.589 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 1.5, 2024-09-16T00:09:29.589 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 1.5, 2024-09-16T00:09:29.589 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 1, 2024-09-16T00:09:29.589 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 1.5, 2024-09-16T00:09:29.589 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 1.5, 2024-09-16T00:09:29.589 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:09:29.589 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:09:29.589 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:09:29.589 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.589 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 3, 2024-09-16T00:09:29.589 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": "cephfs_data", 2024-09-16T00:09:29.589 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:32.008209+0000", 2024-09-16T00:09:29.590 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:09:29.590 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:09:29.590 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:09:29.590 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:09:29.590 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:09:29.590 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:09:29.590 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:09:29.590 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:09:29.590 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:09:29.590 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:09:29.590 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:09:29.590 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:09:29.590 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:09:29.590 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 64, 2024-09-16T00:09:29.590 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 64, 2024-09-16T00:09:29.591 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 64, 2024-09-16T00:09:29.591 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 64, 2024-09-16T00:09:29.591 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 64, 2024-09-16T00:09:29.591 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:09:29.591 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:09:29.591 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:09:29.591 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:09:29.591 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:09:29.591 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:09:29.591 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:09:29.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.592 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "22", 2024-09-16T00:09:29.592 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:09:29.592 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:09:29.592 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:09:29.592 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:09:29.592 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:09:29.592 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:09:29.592 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:09:29.592 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:09:29.592 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:09:29.593 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:09:29.593 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:09:29.593 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:09:29.593 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:09:29.593 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:09:29.593 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:09:29.593 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:09:29.593 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:09:29.593 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:09:29.593 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:09:29.593 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:09:29.594 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:09:29.594 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:09:29.594 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:09:29.594 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:09:29.594 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:09:29.594 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:09:29.594 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.594 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:09:29.594 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:09:29.594 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:09:29.594 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:09:29.594 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:09:29.594 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:09:29.594 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:09:29.595 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:09:29.595 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:09:29.595 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:09:29.595 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:09:29.595 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:09:29.595 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 64, 2024-09-16T00:09:29.595 INFO:teuthology.orchestra.run.smithi139.stdout: "target_size_ratio": 0.89999997615814209 2024-09-16T00:09:29.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.595 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:09:29.595 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs": { 2024-09-16T00:09:29.595 INFO:teuthology.orchestra.run.smithi139.stdout: "data": "cephfs" 2024-09-16T00:09:29.595 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.595 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:09:29.596 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:09:29.596 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 1.25, 2024-09-16T00:09:29.596 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 1.25, 2024-09-16T00:09:29.596 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 1, 2024-09-16T00:09:29.596 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 1.25, 2024-09-16T00:09:29.596 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 1.25, 2024-09-16T00:09:29.596 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:09:29.596 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:09:29.596 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:09:29.596 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.596 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.596 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:29.596 INFO:teuthology.orchestra.run.smithi139.stdout: "osds": [ 2024-09-16T00:09:29.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.597 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 0, 2024-09-16T00:09:29.597 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "83699a05-4937-43c3-aefc-b669bed9bb6f", 2024-09-16T00:09:29.597 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:29.597 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:29.597 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:29.597 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:29.597 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:29.597 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:29.597 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 14, 2024-09-16T00:09:29.597 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 37, 2024-09-16T00:09:29.597 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:29.597 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:29.597 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:29.598 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.598 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.598 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.598 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6816", 2024-09-16T00:09:29.598 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:29.598 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.598 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.598 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.598 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6817", 2024-09-16T00:09:29.598 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:29.598 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.598 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.598 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.598 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:29.599 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.599 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.599 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6818", 2024-09-16T00:09:29.599 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:29.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.599 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.599 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6819", 2024-09-16T00:09:29.599 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:29.599 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.599 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.600 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:29.600 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.600 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.600 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6822", 2024-09-16T00:09:29.600 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:29.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.600 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.600 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6823", 2024-09-16T00:09:29.600 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:29.600 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.600 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.601 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:29.601 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.601 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.601 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6820", 2024-09-16T00:09:29.601 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:29.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.601 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.601 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6821", 2024-09-16T00:09:29.601 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:09:29.601 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.601 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.602 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6817/1974901438", 2024-09-16T00:09:29.602 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6819/1974901438", 2024-09-16T00:09:29.602 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6823/1974901438", 2024-09-16T00:09:29.602 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6821/1974901438", 2024-09-16T00:09:29.602 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:29.602 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:29.602 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:29.602 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.602 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 1, 2024-09-16T00:09:29.602 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "9c04991e-9695-4bbb-8011-bf613bf404b3", 2024-09-16T00:09:29.602 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:29.603 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:29.603 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:29.603 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:29.603 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:29.603 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:29.603 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 14, 2024-09-16T00:09:29.603 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:09:29.603 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:29.603 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:29.603 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:29.603 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.603 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.603 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6808", 2024-09-16T00:09:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6809", 2024-09-16T00:09:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6810", 2024-09-16T00:09:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6811", 2024-09-16T00:09:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6814", 2024-09-16T00:09:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6815", 2024-09-16T00:09:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6812", 2024-09-16T00:09:29.607 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:29.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.607 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.607 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6813", 2024-09-16T00:09:29.607 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:09:29.607 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.607 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.607 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6809/2232380998", 2024-09-16T00:09:29.607 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6811/2232380998", 2024-09-16T00:09:29.607 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6815/2232380998", 2024-09-16T00:09:29.607 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6813/2232380998", 2024-09-16T00:09:29.607 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:29.608 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:29.608 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:29.608 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.608 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 2, 2024-09-16T00:09:29.608 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "8048fc64-72b3-4866-ac62-6a8b2bf6f5c8", 2024-09-16T00:09:29.608 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:29.608 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:29.608 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:29.608 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:29.608 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:29.608 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:29.608 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 13, 2024-09-16T00:09:29.609 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 68, 2024-09-16T00:09:29.609 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:29.609 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:29.609 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:29.609 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.609 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.609 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6800", 2024-09-16T00:09:29.609 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:29.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.609 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.609 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6801", 2024-09-16T00:09:29.609 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:29.610 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.610 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.610 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:29.610 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.610 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.610 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6802", 2024-09-16T00:09:29.610 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:29.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.610 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.610 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6803", 2024-09-16T00:09:29.610 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:29.611 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.611 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.611 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:29.611 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.611 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.611 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6806", 2024-09-16T00:09:29.611 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:29.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.611 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.611 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6807", 2024-09-16T00:09:29.611 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:29.612 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.612 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.612 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:29.612 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.612 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.612 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6804", 2024-09-16T00:09:29.612 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:29.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.612 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.612 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6805", 2024-09-16T00:09:29.612 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:09:29.613 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.613 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.613 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6801/4180588905", 2024-09-16T00:09:29.613 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6803/4180588905", 2024-09-16T00:09:29.613 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6807/4180588905", 2024-09-16T00:09:29.613 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6805/4180588905", 2024-09-16T00:09:29.613 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:29.613 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:29.613 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:29.613 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.613 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 3, 2024-09-16T00:09:29.614 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "c34ac77b-354c-4884-902d-3e2a84f5eb27", 2024-09-16T00:09:29.614 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:29.614 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:29.614 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:29.614 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:29.614 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:29.614 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:29.614 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:09:29.614 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:09:29.614 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:29.614 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:29.614 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:29.614 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.615 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.615 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6826", 2024-09-16T00:09:29.615 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:29.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.615 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.615 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6827", 2024-09-16T00:09:29.615 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:29.615 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.615 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.615 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:29.615 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.615 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.616 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6828", 2024-09-16T00:09:29.616 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:29.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.616 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.616 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6829", 2024-09-16T00:09:29.616 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:29.616 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.616 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.616 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:29.616 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.617 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.617 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6832", 2024-09-16T00:09:29.617 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:29.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.617 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.617 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6833", 2024-09-16T00:09:29.617 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:29.617 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.617 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.617 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:29.617 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.618 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.618 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6830", 2024-09-16T00:09:29.618 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:29.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.618 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.618 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6831", 2024-09-16T00:09:29.618 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:09:29.618 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.618 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.618 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6827/3947768492", 2024-09-16T00:09:29.618 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6829/3947768492", 2024-09-16T00:09:29.619 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6833/3947768492", 2024-09-16T00:09:29.619 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6831/3947768492", 2024-09-16T00:09:29.619 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:29.619 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:29.619 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:29.619 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.619 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 4, 2024-09-16T00:09:29.619 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "ba9a7c93-9672-43e1-8c98-2878cdf7f548", 2024-09-16T00:09:29.619 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:29.619 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:29.619 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:29.619 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:29.619 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:29.620 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:29.620 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:09:29.620 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:09:29.620 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:29.620 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:29.620 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:29.620 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.620 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.620 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6816", 2024-09-16T00:09:29.620 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:29.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.620 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.621 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6817", 2024-09-16T00:09:29.621 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:29.621 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.621 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.621 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:29.621 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.621 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.621 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6818", 2024-09-16T00:09:29.621 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:29.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.622 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.622 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6819", 2024-09-16T00:09:29.622 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:29.622 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.622 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.622 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:29.622 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.622 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.622 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6822", 2024-09-16T00:09:29.622 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:29.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.623 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.623 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6823", 2024-09-16T00:09:29.623 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:29.623 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.623 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.623 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:29.623 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.623 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.623 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6820", 2024-09-16T00:09:29.623 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:29.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.624 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.624 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6821", 2024-09-16T00:09:29.624 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:09:29.624 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.624 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.624 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6817/642632916", 2024-09-16T00:09:29.624 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6819/642632916", 2024-09-16T00:09:29.624 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6823/642632916", 2024-09-16T00:09:29.624 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6821/642632916", 2024-09-16T00:09:29.624 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:29.624 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:29.624 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:29.625 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.625 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 5, 2024-09-16T00:09:29.625 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "46a0916b-474c-411b-84ad-86f27d4cbf53", 2024-09-16T00:09:29.625 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:29.625 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:29.625 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:29.625 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:29.625 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:29.625 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:29.625 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:09:29.625 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 37, 2024-09-16T00:09:29.625 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:29.626 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:29.626 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:29.626 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.626 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.626 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6800", 2024-09-16T00:09:29.626 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:29.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.626 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.626 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6801", 2024-09-16T00:09:29.626 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:29.626 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.626 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.627 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:29.627 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.627 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.627 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6802", 2024-09-16T00:09:29.627 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:29.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.627 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.627 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6803", 2024-09-16T00:09:29.627 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:29.627 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.627 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.628 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:29.628 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.628 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.628 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6806", 2024-09-16T00:09:29.628 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:29.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.628 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.628 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6807", 2024-09-16T00:09:29.628 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:29.628 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.628 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.629 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:29.629 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.629 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.629 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6804", 2024-09-16T00:09:29.629 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:29.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.629 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.629 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6805", 2024-09-16T00:09:29.629 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:09:29.629 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.629 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.630 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6801/1051778383", 2024-09-16T00:09:29.630 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6803/1051778383", 2024-09-16T00:09:29.630 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6807/1051778383", 2024-09-16T00:09:29.630 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6805/1051778383", 2024-09-16T00:09:29.630 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:29.630 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:29.630 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:29.630 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.630 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 6, 2024-09-16T00:09:29.630 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "9b6b33cc-d6d9-4a0a-8143-526073a58b93", 2024-09-16T00:09:29.630 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:29.631 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:29.631 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:29.631 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:29.631 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:29.631 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:29.631 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:09:29.631 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 46, 2024-09-16T00:09:29.631 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:29.631 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:29.631 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:29.631 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.631 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.632 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6824", 2024-09-16T00:09:29.632 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:29.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.632 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.632 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6825", 2024-09-16T00:09:29.632 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:29.632 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.632 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.632 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:29.632 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.632 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.633 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6826", 2024-09-16T00:09:29.633 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:29.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.633 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.633 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6827", 2024-09-16T00:09:29.633 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:29.633 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.633 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.633 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:29.633 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.633 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.634 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6830", 2024-09-16T00:09:29.634 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:29.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.634 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.634 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6831", 2024-09-16T00:09:29.634 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:29.634 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.634 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.634 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:29.634 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.635 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.635 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6828", 2024-09-16T00:09:29.635 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:29.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.635 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.635 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6829", 2024-09-16T00:09:29.635 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1908495807 2024-09-16T00:09:29.635 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.635 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.635 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6825/1908495807", 2024-09-16T00:09:29.635 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6827/1908495807", 2024-09-16T00:09:29.635 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6831/1908495807", 2024-09-16T00:09:29.636 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6829/1908495807", 2024-09-16T00:09:29.636 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:29.636 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:29.636 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:29.636 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.636 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 7, 2024-09-16T00:09:29.636 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "50bcd808-3d3e-4a72-88e4-30decb53cfea", 2024-09-16T00:09:29.636 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:09:29.636 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:09:29.636 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:09:29.636 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:09:29.636 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:09:29.637 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:09:29.637 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:09:29.637 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 62, 2024-09-16T00:09:29.637 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:09:29.637 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:09:29.637 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:09:29.637 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.637 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.637 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6808", 2024-09-16T00:09:29.637 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:29.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.638 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.638 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6809", 2024-09-16T00:09:29.638 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:29.638 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.638 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.638 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:09:29.638 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.638 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.638 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6810", 2024-09-16T00:09:29.638 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:29.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.639 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.639 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6811", 2024-09-16T00:09:29.639 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:29.639 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.639 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.639 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:09:29.639 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.639 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.639 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6814", 2024-09-16T00:09:29.639 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:29.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.640 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.640 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6815", 2024-09-16T00:09:29.640 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:29.640 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.640 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.640 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:09:29.640 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:09:29.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.640 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:09:29.640 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6812", 2024-09-16T00:09:29.640 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:29.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.641 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:09:29.641 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6813", 2024-09-16T00:09:29.641 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:09:29.641 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.641 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.641 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6809/1158838808", 2024-09-16T00:09:29.641 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6811/1158838808", 2024-09-16T00:09:29.641 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6815/1158838808", 2024-09-16T00:09:29.641 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6813/1158838808", 2024-09-16T00:09:29.641 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:09:29.641 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:09:29.642 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:09:29.642 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.642 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.642 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:29.642 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_xinfo": [ 2024-09-16T00:09:29.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.642 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 0, 2024-09-16T00:09:29.642 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:29.642 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:29.642 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:29.642 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:29.642 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:29.642 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.843117+0000", 2024-09-16T00:09:29.643 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:29.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.643 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 1, 2024-09-16T00:09:29.643 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:29.643 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:29.643 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:29.643 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:29.643 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:29.643 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.622789+0000", 2024-09-16T00:09:29.643 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:29.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.643 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 2, 2024-09-16T00:09:29.643 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:29.644 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:29.644 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:29.644 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:29.644 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:29.644 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:17.919457+0000", 2024-09-16T00:09:29.644 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:29.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.644 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 3, 2024-09-16T00:09:29.644 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:29.644 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:29.644 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:29.644 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:29.644 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:29.645 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:19.330341+0000", 2024-09-16T00:09:29.645 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:29.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.645 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 4, 2024-09-16T00:09:29.645 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:29.645 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:29.645 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:29.645 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:29.645 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:29.645 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.815640+0000", 2024-09-16T00:09:29.645 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:29.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.646 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 5, 2024-09-16T00:09:29.646 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:29.646 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:29.646 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:29.646 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:29.646 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:29.646 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.083046+0000", 2024-09-16T00:09:29.646 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:29.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.646 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 6, 2024-09-16T00:09:29.646 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:29.646 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:29.646 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:29.647 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:29.647 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:29.647 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.551499+0000", 2024-09-16T00:09:29.647 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:29.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.647 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 7, 2024-09-16T00:09:29.647 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:09:29.647 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:09:29.647 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:09:29.647 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:09:29.647 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:09:29.647 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.377179+0000", 2024-09-16T00:09:29.647 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:09:29.648 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.648 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:29.648 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap": [ 2024-09-16T00:09:29.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.648 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.2d", 2024-09-16T00:09:29.648 INFO:teuthology.orchestra.run.smithi139.stdout: "osds": [ 2024-09-16T00:09:29.648 INFO:teuthology.orchestra.run.smithi139.stdout: 7, 2024-09-16T00:09:29.648 INFO:teuthology.orchestra.run.smithi139.stdout: 5 2024-09-16T00:09:29.648 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.648 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.648 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:29.648 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap_items": [ 2024-09-16T00:09:29.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.649 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.1f", 2024-09-16T00:09:29.649 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:29.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.649 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 0, 2024-09-16T00:09:29.649 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:09:29.649 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.649 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.649 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.21", 2024-09-16T00:09:29.649 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:29.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.649 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 5, 2024-09-16T00:09:29.649 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 1 2024-09-16T00:09:29.650 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.650 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.650 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.22", 2024-09-16T00:09:29.650 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:29.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.650 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 5, 2024-09-16T00:09:29.650 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:09:29.650 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.650 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.650 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.38", 2024-09-16T00:09:29.651 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:29.651 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.651 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 3, 2024-09-16T00:09:29.651 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:09:29.651 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.651 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.651 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.651 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.651 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.3a", 2024-09-16T00:09:29.651 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:29.651 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.651 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 5, 2024-09-16T00:09:29.651 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:09:29.652 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.652 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.652 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.652 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.3b", 2024-09-16T00:09:29.652 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:09:29.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:29.652 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 7, 2024-09-16T00:09:29.652 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 1 2024-09-16T00:09:29.652 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.652 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:29.652 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.652 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:29.652 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap_primaries": [], 2024-09-16T00:09:29.653 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_temp": [], 2024-09-16T00:09:29.653 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_temp": [], 2024-09-16T00:09:29.653 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklist": {}, 2024-09-16T00:09:29.653 INFO:teuthology.orchestra.run.smithi139.stdout: "range_blocklist": {}, 2024-09-16T00:09:29.653 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profiles": { 2024-09-16T00:09:29.653 INFO:teuthology.orchestra.run.smithi139.stdout: "default": { 2024-09-16T00:09:29.653 INFO:teuthology.orchestra.run.smithi139.stdout: "crush-failure-domain": "osd", 2024-09-16T00:09:29.653 INFO:teuthology.orchestra.run.smithi139.stdout: "k": "2", 2024-09-16T00:09:29.653 INFO:teuthology.orchestra.run.smithi139.stdout: "m": "1", 2024-09-16T00:09:29.653 INFO:teuthology.orchestra.run.smithi139.stdout: "plugin": "jerasure", 2024-09-16T00:09:29.653 INFO:teuthology.orchestra.run.smithi139.stdout: "technique": "reed_sol_van" 2024-09-16T00:09:29.653 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:29.653 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps_queue": [], 2024-09-16T00:09:29.654 INFO:teuthology.orchestra.run.smithi139.stdout: "new_removed_snaps": [], 2024-09-16T00:09:29.654 INFO:teuthology.orchestra.run.smithi139.stdout: "new_purged_snaps": [], 2024-09-16T00:09:29.654 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_node_flags": {}, 2024-09-16T00:09:29.654 INFO:teuthology.orchestra.run.smithi139.stdout: "device_class_flags": {}, 2024-09-16T00:09:29.654 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode": { 2024-09-16T00:09:29.654 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode_enabled": false, 2024-09-16T00:09:29.654 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_bucket_count": 0, 2024-09-16T00:09:29.654 INFO:teuthology.orchestra.run.smithi139.stdout: "degraded_stretch_mode": 0, 2024-09-16T00:09:29.654 INFO:teuthology.orchestra.run.smithi139.stdout: "recovering_stretch_mode": 0, 2024-09-16T00:09:29.654 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode_bucket": 0 2024-09-16T00:09:29.654 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:29.654 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:29.655 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:09:29.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.469+0000 7f536998f640 1 -- 172.21.15.139:0/2272375969 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f535006c300 msgr2=0x7f535006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.469+0000 7f536998f640 1 --2- 172.21.15.139:0/2272375969 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f535006c300 0x7f535006e7c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5354000f60 tx=0x7f5354000bc0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.469+0000 7f536998f640 1 -- 172.21.15.139:0/2272375969 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53640c7b90 msgr2=0x7f53641e4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.469+0000 7f536998f640 1 --2- 172.21.15.139:0/2272375969 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53640c7b90 0x7f53641e4e40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f535800e9b0 tx=0x7f535800ee70 comp rx=0 tx=0).stop 2024-09-16T00:09:29.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.469+0000 7f536998f640 1 -- 172.21.15.139:0/2272375969 shutdown_connections 2024-09-16T00:09:29.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.469+0000 7f536998f640 1 --2- 172.21.15.139:0/2272375969 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f535006c300 0x7f535006e7c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.469+0000 7f536998f640 1 --2- 172.21.15.139:0/2272375969 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f536415ad50 0x7f5364154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.469+0000 7f536998f640 1 --2- 172.21.15.139:0/2272375969 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53640c7b90 0x7f53641e4e40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.469+0000 7f536998f640 1 --2- 172.21.15.139:0/2272375969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53640c67e0 0x7f53640c2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.469+0000 7f536998f640 1 -- 172.21.15.139:0/2272375969 >> 172.21.15.139:0/2272375969 conn(0x7f53640d05f0 msgr2=0x7f53640caa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:29.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.469+0000 7f536998f640 1 -- 172.21.15.139:0/2272375969 shutdown_connections 2024-09-16T00:09:29.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.469+0000 7f536998f640 1 -- 172.21.15.139:0/2272375969 wait complete. 2024-09-16T00:09:29.658 INFO:tasks.thrashosds.thrasher:j is {'epoch': 76, 'fsid': '2d3366e5-6a59-4b59-93b2-615e72ac7015', 'created': '2024-09-16T00:07:09.967833+0000', 'modified': '2024-09-16T00:09:27.129287+0000', 'last_up_change': '2024-09-16T00:07:21.049678+0000', 'last_in_change': '2024-09-16T00:07:12.025620+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:07:22.056303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T00:07:30.768649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T00:07:32.008209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '83699a05-4937-43c3-aefc-b669bed9bb6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6816', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6817', 'nonce': 1974901438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6818', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6819', 'nonce': 1974901438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6822', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6823', 'nonce': 1974901438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6820', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6821', 'nonce': 1974901438}]}, 'public_addr': '172.21.15.139:6817/1974901438', 'cluster_addr': '172.21.15.139:6819/1974901438', 'heartbeat_back_addr': '172.21.15.139:6823/1974901438', 'heartbeat_front_addr': '172.21.15.139:6821/1974901438', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9c04991e-9695-4bbb-8011-bf613bf404b3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6808', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6809', 'nonce': 2232380998}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6810', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6811', 'nonce': 2232380998}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6814', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6815', 'nonce': 2232380998}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6812', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6813', 'nonce': 2232380998}]}, 'public_addr': '172.21.15.139:6809/2232380998', 'cluster_addr': '172.21.15.139:6811/2232380998', 'heartbeat_back_addr': '172.21.15.139:6815/2232380998', 'heartbeat_front_addr': '172.21.15.139:6813/2232380998', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8048fc64-72b3-4866-ac62-6a8b2bf6f5c8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 68, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6800', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6801', 'nonce': 4180588905}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6802', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6803', 'nonce': 4180588905}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6806', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6807', 'nonce': 4180588905}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6804', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6805', 'nonce': 4180588905}]}, 'public_addr': '172.21.15.139:6801/4180588905', 'cluster_addr': '172.21.15.139:6803/4180588905', 'heartbeat_back_addr': '172.21.15.139:6807/4180588905', 'heartbeat_front_addr': '172.21.15.139:6805/4180588905', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c34ac77b-354c-4884-902d-3e2a84f5eb27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6826', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6827', 'nonce': 3947768492}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6828', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6829', 'nonce': 3947768492}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6832', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6833', 'nonce': 3947768492}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6830', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6831', 'nonce': 3947768492}]}, 'public_addr': '172.21.15.139:6827/3947768492', 'cluster_addr': '172.21.15.139:6829/3947768492', 'heartbeat_back_addr': '172.21.15.139:6833/3947768492', 'heartbeat_front_addr': '172.21.15.139:6831/3947768492', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ba9a7c93-9672-43e1-8c98-2878cdf7f548', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6816', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6817', 'nonce': 642632916}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6818', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 642632916}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 642632916}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6821', 'nonce': 642632916}]}, 'public_addr': '172.21.15.164:6817/642632916', 'cluster_addr': '172.21.15.164:6819/642632916', 'heartbeat_back_addr': '172.21.15.164:6823/642632916', 'heartbeat_front_addr': '172.21.15.164:6821/642632916', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '46a0916b-474c-411b-84ad-86f27d4cbf53', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 37, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 1051778383}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 1051778383}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 1051778383}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 1051778383}]}, 'public_addr': '172.21.15.164:6801/1051778383', 'cluster_addr': '172.21.15.164:6803/1051778383', 'heartbeat_back_addr': '172.21.15.164:6807/1051778383', 'heartbeat_front_addr': '172.21.15.164:6805/1051778383', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b6b33cc-d6d9-4a0a-8143-526073a58b93', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 46, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6824', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 1908495807}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6827', 'nonce': 1908495807}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6830', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 1908495807}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 1908495807}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 1908495807}]}, 'public_addr': '172.21.15.164:6825/1908495807', 'cluster_addr': '172.21.15.164:6827/1908495807', 'heartbeat_back_addr': '172.21.15.164:6831/1908495807', 'heartbeat_front_addr': '172.21.15.164:6829/1908495807', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '50bcd808-3d3e-4a72-88e4-30decb53cfea', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 62, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6809', 'nonce': 1158838808}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 1158838808}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6815', 'nonce': 1158838808}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6812', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 1158838808}]}, 'public_addr': '172.21.15.164:6809/1158838808', 'cluster_addr': '172.21.15.164:6811/1158838808', 'heartbeat_back_addr': '172.21.15.164:6815/1158838808', 'heartbeat_front_addr': '172.21.15.164:6813/1158838808', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.843117+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.622789+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:17.919457+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:19.330341+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.815640+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.083046+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.551499+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.377179+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.2d', 'osds': [7, 5]}], 'pg_upmap_items': [{'pgid': '3.1f', 'mappings': [{'from': 0, 'to': 2}]}, {'pgid': '3.21', 'mappings': [{'from': 5, 'to': 1}]}, {'pgid': '3.22', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '3.38', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '3.3a', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '3.3b', 'mappings': [{'from': 7, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T00:09:29.658 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:09:29.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.524+0000 7f285fe65640 1 -- 172.21.15.139:0/2172187411 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f2858070720 con 0x7f285806eae0 2024-09-16T00:09:29.659 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:29.570+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:29.699 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:29.697+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:29.774 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:29.772+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:29.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.815+0000 7f700da5c640 1 Processor -- start 2024-09-16T00:09:29.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.815+0000 7f700da5c640 1 -- start start 2024-09-16T00:09:29.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.815+0000 7f700da5c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7008154a20 0x7f700806d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.816+0000 7f700da5c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f70080c5120 0x7f70080cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.816+0000 7f700da5c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f700815b290 con 0x7f700815a7e0 2024-09-16T00:09:29.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.816+0000 7f700da5c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f70080cac00 con 0x7f70080c5120 2024-09-16T00:09:29.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.816+0000 7f700da5c640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f70080cbe70 con 0x7f7008154a20 2024-09-16T00:09:29.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.816+0000 7f70067fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f70080c5120 0x7f70080cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.816+0000 7f70067fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f70080c5120 0x7f70080cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58768/0 (socket says 172.21.15.139:58768) 2024-09-16T00:09:29.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.816+0000 7f70067fc640 1 -- 172.21.15.139:0/475852056 learned_addr learned my addr 172.21.15.139:0/475852056 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:29.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.816+0000 7f7006ffd640 1 --2- 172.21.15.139:0/475852056 >> v2:172.21.15.164:3300/0 conn(0x7f7008154a20 0x7f700806d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.816+0000 7f70067fc640 1 -- 172.21.15.139:0/475852056 >> v2:172.21.15.164:3300/0 conn(0x7f7008154a20 msgr2=0x7f700806d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:29.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.816+0000 7f70067fc640 1 --2- 172.21.15.139:0/475852056 >> v2:172.21.15.164:3300/0 conn(0x7f7008154a20 0x7f700806d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.816+0000 7f70067fc640 1 -- 172.21.15.139:0/475852056 >> v1:172.21.15.164:6789/0 conn(0x7f700815a7e0 legacy=0x7f70080cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.816+0000 7f70067fc640 1 -- 172.21.15.139:0/475852056 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700815b570 con 0x7f70080c5120 2024-09-16T00:09:29.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.817+0000 7f70067fc640 1 --2- 172.21.15.139:0/475852056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70080c5120 0x7f70080cb930 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f700808dbe0 tx=0x7f6ffc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7223318d02eac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:29.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.817+0000 7f7005ffb640 1 -- 172.21.15.139:0/475852056 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc037070 con 0x7f70080c5120 2024-09-16T00:09:29.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.817+0000 7f7005ffb640 1 -- 172.21.15.139:0/475852056 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ffc02ae50 con 0x7f70080c5120 2024-09-16T00:09:29.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.817+0000 7f7005ffb640 1 -- 172.21.15.139:0/475852056 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc032860 con 0x7f70080c5120 2024-09-16T00:09:29.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.818+0000 7f700da5c640 1 -- 172.21.15.139:0/475852056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70080c5120 msgr2=0x7f70080cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.818+0000 7f700da5c640 1 --2- 172.21.15.139:0/475852056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70080c5120 0x7f70080cb930 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f700808dbe0 tx=0x7f6ffc02a300 comp rx=0 tx=0).stop 2024-09-16T00:09:29.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.818+0000 7f700da5c640 1 -- 172.21.15.139:0/475852056 shutdown_connections 2024-09-16T00:09:29.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.818+0000 7f700da5c640 1 --2- 172.21.15.139:0/475852056 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70080c5120 0x7f70080cb930 secure :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f700808dbe0 tx=0x7f6ffc02a300 comp rx=0 tx=0).stop 2024-09-16T00:09:29.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.818+0000 7f700da5c640 1 --2- 172.21.15.139:0/475852056 >> v2:172.21.15.164:3300/0 conn(0x7f7008154a20 0x7f700806d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.818+0000 7f700da5c640 1 -- 172.21.15.139:0/475852056 >> 172.21.15.139:0/475852056 conn(0x7f7008067f80 msgr2=0x7f70080d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:29.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.818+0000 7f700da5c640 1 -- 172.21.15.139:0/475852056 shutdown_connections 2024-09-16T00:09:29.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.818+0000 7f700da5c640 1 -- 172.21.15.139:0/475852056 wait complete. 2024-09-16T00:09:29.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.820+0000 7f700da5c640 1 Processor -- start 2024-09-16T00:09:29.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.821+0000 7f700da5c640 1 -- start start 2024-09-16T00:09:29.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f700da5c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7008154a20 0x7f7008156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f700da5c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f700815a7e0 0x7f7008153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f700da5c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7008157d80 0x7f700814b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f700da5c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f70080cb410 con 0x7f7008157d80 2024-09-16T00:09:29.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f700da5c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f70080cb130 con 0x7f7008154a20 2024-09-16T00:09:29.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f700da5c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7008154e00 con 0x7f700815a7e0 2024-09-16T00:09:29.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f7006ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7008154a20 0x7f7008156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f7006ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7008154a20 0x7f7008156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:55306/0 (socket says 172.21.15.139:55306) 2024-09-16T00:09:29.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f70077fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7008157d80 0x7f700814b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f7006ffd640 1 -- 172.21.15.139:0/2070922573 learned_addr learned my addr 172.21.15.139:0/2070922573 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:29.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f70067fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f700815a7e0 0x7f7008153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f7006ffd640 1 -- 172.21.15.139:0/2070922573 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f700815a7e0 msgr2=0x7f7008153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f7006ffd640 1 --2- 172.21.15.139:0/2070922573 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f700815a7e0 0x7f7008153500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f7006ffd640 1 -- 172.21.15.139:0/2070922573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7008157d80 msgr2=0x7f700814b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:29.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f7006ffd640 1 --2- 172.21.15.139:0/2070922573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7008157d80 0x7f700814b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:29.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f7006ffd640 1 -- 172.21.15.139:0/2070922573 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70080b4680 con 0x7f7008154a20 2024-09-16T00:09:29.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.822+0000 7f70077fe640 1 --2- 172.21.15.139:0/2070922573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7008157d80 0x7f700814b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:29.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.823+0000 7f7006ffd640 1 --2- 172.21.15.139:0/2070922573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7008154a20 0x7f7008156800 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800e9e0 tx=0x7f6ff800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:29.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.823+0000 7f6feffff640 1 -- 172.21.15.139:0/2070922573 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8017070 con 0x7f7008154a20 2024-09-16T00:09:29.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.823+0000 7f6feffff640 1 -- 172.21.15.139:0/2070922573 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ff800c970 con 0x7f7008154a20 2024-09-16T00:09:29.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.823+0000 7f700da5c640 1 -- 172.21.15.139:0/2070922573 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7008155360 con 0x7f7008154a20 2024-09-16T00:09:29.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.823+0000 7f700da5c640 1 -- 172.21.15.139:0/2070922573 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7008155920 con 0x7f7008154a20 2024-09-16T00:09:29.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.823+0000 7f6feffff640 1 -- 172.21.15.139:0/2070922573 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800cd60 con 0x7f7008154a20 2024-09-16T00:09:29.827 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:29.827 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:29.827 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:29.828 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:29.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.824+0000 7f700da5c640 1 -- 172.21.15.139:0/2070922573 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f700815a450 con 0x7f7008154a20 2024-09-16T00:09:29.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.825+0000 7f6feffff640 1 -- 172.21.15.139:0/2070922573 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6ff801a070 con 0x7f7008154a20 2024-09-16T00:09:29.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.826+0000 7f6feffff640 1 --2- 172.21.15.139:0/2070922573 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6ff406c2e0 0x7f6ff406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:29.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.826+0000 7f6feffff640 1 -- 172.21.15.139:0/2070922573 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5944+0+0 (secure 0 0 0) 0x7f6ff808b7d0 con 0x7f7008154a20 2024-09-16T00:09:29.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.826+0000 7f70067fc640 1 --2- 172.21.15.139:0/2070922573 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6ff406c2e0 0x7f6ff406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:29.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.826+0000 7f70067fc640 1 --2- 172.21.15.139:0/2070922573 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6ff406c2e0 0x7f6ff406e7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc034000 tx=0x7f6ffc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:29.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:29.834+0000 7f6feffff640 1 -- 172.21.15.139:0/2070922573 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f700815a450 con 0x7f7008154a20 2024-09-16T00:09:29.836 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:09:29.880 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:29.879+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:29.964 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:29.964 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:29.964 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:29.964 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:29.964 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:29.974 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:09:30.016 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:30.014+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:30.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.049+0000 7f700da5c640 1 -- 172.21.15.139:0/2070922573 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70080cb930 con 0x7f6ff406c2e0 2024-09-16T00:09:30.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.057+0000 7f6feffff640 1 -- 172.21.15.139:0/2070922573 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254293 (secure 0 0 0) 0x7f70080cb930 con 0x7f6ff406c2e0 2024-09-16T00:09:30.062 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:09:30.064 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:09:30.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.061+0000 7f700da5c640 1 -- 172.21.15.139:0/2070922573 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6ff406c2e0 msgr2=0x7f6ff406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:30.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.061+0000 7f700da5c640 1 --2- 172.21.15.139:0/2070922573 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6ff406c2e0 0x7f6ff406e7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc034000 tx=0x7f6ffc02e000 comp rx=0 tx=0).stop 2024-09-16T00:09:30.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.061+0000 7f700da5c640 1 -- 172.21.15.139:0/2070922573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7008154a20 msgr2=0x7f7008156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:30.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.061+0000 7f700da5c640 1 --2- 172.21.15.139:0/2070922573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7008154a20 0x7f7008156800 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800e9e0 tx=0x7f6ff800eea0 comp rx=0 tx=0).stop 2024-09-16T00:09:30.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.061+0000 7f700da5c640 1 -- 172.21.15.139:0/2070922573 shutdown_connections 2024-09-16T00:09:30.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.061+0000 7f700da5c640 1 --2- 172.21.15.139:0/2070922573 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6ff406c2e0 0x7f6ff406e7a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:30.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.061+0000 7f700da5c640 1 --2- 172.21.15.139:0/2070922573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7008157d80 0x7f700814b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:30.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.061+0000 7f700da5c640 1 --2- 172.21.15.139:0/2070922573 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f700815a7e0 0x7f7008153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:30.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.061+0000 7f700da5c640 1 --2- 172.21.15.139:0/2070922573 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7008154a20 0x7f7008156800 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:30.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.061+0000 7f700da5c640 1 -- 172.21.15.139:0/2070922573 >> 172.21.15.139:0/2070922573 conn(0x7f7008067f80 msgr2=0x7f700806cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:30.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.061+0000 7f700da5c640 1 -- 172.21.15.139:0/2070922573 shutdown_connections 2024-09-16T00:09:30.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.061+0000 7f700da5c640 1 -- 172.21.15.139:0/2070922573 wait complete. 2024-09-16T00:09:30.076 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2024-09-16T00:09:30.045588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":85543791,"num_objects":8508,"num_object_clones":0,"num_object_copies":17016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8508,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":9102,"num_write_kb":91604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":135,"num_bytes_recovered":397258,"num_keys_recovered":4,"num_objects_omap":54,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8955,"ondisk_log_size":8955,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":434968,"kb_used_data":131396,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754539752,"statfs":{"total":773094113280,"available":772648706048,"internally_reserved":0,"allocated":134549504,"data_stored":173618983,"data_compressed":22090714,"data_compressed_allocated":26906624,"data_compressed_original":109580288,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20395134,"num_objects":1912,"num_object_clones":0,"num_object_copies":3824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1999,"num_write_kb":21162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001372"},"pg_stats":[{"pgid":"3.3f","version":"75'136","reported_seq":263,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.489786+0000","last_change":"2024-09-16T00:08:36.888570+0000","last_active":"2024-09-16T00:09:26.489786+0000","last_peered":"2024-09-16T00:09:26.489786+0000","last_clean":"2024-09-16T00:09:26.489786+0000","last_became_active":"2024-09-16T00:08:36.888143+0000","last_became_peered":"2024-09-16T00:08:36.888143+0000","last_unstale":"2024-09-16T00:09:26.489786+0000","last_undegraded":"2024-09-16T00:09:26.489786+0000","last_fullsized":"2024-09-16T00:09:26.489786+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:30.579681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":408338,"num_objects":136,"num_object_clones":0,"num_object_copies":272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":136,"num_write_kb":470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"75'121","reported_seq":238,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.222794+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:09:25.222794+0000","last_peered":"2024-09-16T00:09:25.222794+0000","last_clean":"2024-09-16T00:09:25.222794+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:09:25.222794+0000","last_undegraded":"2024-09-16T00:09:25.222794+0000","last_fullsized":"2024-09-16T00:09:25.222794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":498751,"num_objects":121,"num_object_clones":0,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"75'122","reported_seq":239,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.006788+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:09:26.006788+0000","last_peered":"2024-09-16T00:09:26.006788+0000","last_clean":"2024-09-16T00:09:26.006788+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:09:26.006788+0000","last_undegraded":"2024-09-16T00:09:26.006788+0000","last_fullsized":"2024-09-16T00:09:26.006788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":685241,"num_objects":122,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122,"num_write_kb":740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"75'120","reported_seq":238,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:27.032547+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:09:27.032547+0000","last_peered":"2024-09-16T00:09:27.032547+0000","last_clean":"2024-09-16T00:09:27.032547+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:09:27.032547+0000","last_undegraded":"2024-09-16T00:09:27.032547+0000","last_fullsized":"2024-09-16T00:09:27.032547+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":475163,"num_objects":120,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"75'143","reported_seq":235,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.250140+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:09:25.250140+0000","last_peered":"2024-09-16T00:09:25.250140+0000","last_clean":"2024-09-16T00:09:25.250140+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:09:25.250140+0000","last_undegraded":"2024-09-16T00:09:25.250140+0000","last_fullsized":"2024-09-16T00:09:25.250140+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1773537,"num_objects":143,"num_object_clones":0,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":1810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"75'130","reported_seq":248,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.017037+0000","last_change":"2024-09-16T00:09:16.225647+0000","last_active":"2024-09-16T00:09:25.017037+0000","last_peered":"2024-09-16T00:09:25.017037+0000","last_clean":"2024-09-16T00:09:25.017037+0000","last_became_active":"2024-09-16T00:09:16.081926+0000","last_became_peered":"2024-09-16T00:09:16.081926+0000","last_unstale":"2024-09-16T00:09:25.017037+0000","last_undegraded":"2024-09-16T00:09:25.017037+0000","last_fullsized":"2024-09-16T00:09:25.017037+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:59.896840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":445073,"num_objects":130,"num_object_clones":0,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":130,"num_write_kb":502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":94,"num_bytes_recovered":280651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"75'146","reported_seq":263,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.712531+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:09:26.712531+0000","last_peered":"2024-09-16T00:09:26.712531+0000","last_clean":"2024-09-16T00:09:26.712531+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:09:26.712531+0000","last_undegraded":"2024-09-16T00:09:26.712531+0000","last_fullsized":"2024-09-16T00:09:26.712531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1168019,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":1215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"75'109","reported_seq":201,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.980153+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:09:24.980153+0000","last_peered":"2024-09-16T00:09:24.980153+0000","last_clean":"2024-09-16T00:09:24.980153+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:09:24.980153+0000","last_undegraded":"2024-09-16T00:09:24.980153+0000","last_fullsized":"2024-09-16T00:09:24.980153+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":609510,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"75'138","reported_seq":255,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.116932+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:09:26.116932+0000","last_peered":"2024-09-16T00:09:26.116932+0000","last_clean":"2024-09-16T00:09:26.116932+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:09:26.116932+0000","last_undegraded":"2024-09-16T00:09:26.116932+0000","last_fullsized":"2024-09-16T00:09:26.116932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":635274,"num_objects":138,"num_object_clones":0,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"75'127","reported_seq":192,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.869093+0000","last_change":"2024-09-16T00:08:36.891868+0000","last_active":"2024-09-16T00:09:26.869093+0000","last_peered":"2024-09-16T00:09:26.869093+0000","last_clean":"2024-09-16T00:09:26.869093+0000","last_became_active":"2024-09-16T00:08:36.891605+0000","last_became_peered":"2024-09-16T00:08:36.891605+0000","last_unstale":"2024-09-16T00:09:26.869093+0000","last_undegraded":"2024-09-16T00:09:26.869093+0000","last_fullsized":"2024-09-16T00:09:26.869093+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:46:00.730436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554186,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"75'126","reported_seq":191,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.700951+0000","last_change":"2024-09-16T00:08:36.891589+0000","last_active":"2024-09-16T00:09:26.700951+0000","last_peered":"2024-09-16T00:09:26.700951+0000","last_clean":"2024-09-16T00:09:26.700951+0000","last_became_active":"2024-09-16T00:08:36.891279+0000","last_became_peered":"2024-09-16T00:08:36.891279+0000","last_unstale":"2024-09-16T00:09:26.700951+0000","last_undegraded":"2024-09-16T00:09:26.700951+0000","last_fullsized":"2024-09-16T00:09:26.700951+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:01.316526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":465263,"num_objects":126,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"75'120","reported_seq":247,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.860217+0000","last_change":"2024-09-16T00:08:36.888576+0000","last_active":"2024-09-16T00:09:26.860217+0000","last_peered":"2024-09-16T00:09:26.860217+0000","last_clean":"2024-09-16T00:09:26.860217+0000","last_became_active":"2024-09-16T00:08:36.888301+0000","last_became_peered":"2024-09-16T00:08:36.888301+0000","last_unstale":"2024-09-16T00:09:26.860217+0000","last_undegraded":"2024-09-16T00:09:26.860217+0000","last_fullsized":"2024-09-16T00:09:26.860217+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":478560,"num_objects":120,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"75'123","reported_seq":251,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.951300+0000","last_change":"2024-09-16T00:08:36.892595+0000","last_active":"2024-09-16T00:09:25.951300+0000","last_peered":"2024-09-16T00:09:25.951300+0000","last_clean":"2024-09-16T00:09:25.951300+0000","last_became_active":"2024-09-16T00:08:36.892316+0000","last_became_peered":"2024-09-16T00:08:36.892316+0000","last_unstale":"2024-09-16T00:09:25.951300+0000","last_undegraded":"2024-09-16T00:09:25.951300+0000","last_fullsized":"2024-09-16T00:09:25.951300+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:09:47.480168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":597033,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"75'131","reported_seq":248,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.764534+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:09:25.764534+0000","last_peered":"2024-09-16T00:09:25.764534+0000","last_clean":"2024-09-16T00:09:25.764534+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:09:25.764534+0000","last_undegraded":"2024-09-16T00:09:25.764534+0000","last_fullsized":"2024-09-16T00:09:25.764534+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5021232,"num_objects":130,"num_object_clones":0,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":130,"num_write_kb":4974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"75'132","reported_seq":249,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.194139+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:09:25.194139+0000","last_peered":"2024-09-16T00:09:25.194139+0000","last_clean":"2024-09-16T00:09:25.194139+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:09:25.194139+0000","last_undegraded":"2024-09-16T00:09:25.194139+0000","last_fullsized":"2024-09-16T00:09:25.194139+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":400360,"num_objects":132,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"75'148","reported_seq":266,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:27.048632+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:09:27.048632+0000","last_peered":"2024-09-16T00:09:27.048632+0000","last_clean":"2024-09-16T00:09:27.048632+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:09:27.048632+0000","last_undegraded":"2024-09-16T00:09:27.048632+0000","last_fullsized":"2024-09-16T00:09:27.048632+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":687269,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":148,"num_write_kb":754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"75'131","reported_seq":249,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.793614+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:09:26.793614+0000","last_peered":"2024-09-16T00:09:26.793614+0000","last_clean":"2024-09-16T00:09:26.793614+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:09:26.793614+0000","last_undegraded":"2024-09-16T00:09:26.793614+0000","last_fullsized":"2024-09-16T00:09:26.793614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":421628,"num_objects":131,"num_object_clones":0,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":131,"num_write_kb":476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"75'120","reported_seq":258,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.938405+0000","last_change":"2024-09-16T00:08:36.894430+0000","last_active":"2024-09-16T00:09:24.938405+0000","last_peered":"2024-09-16T00:09:24.938405+0000","last_clean":"2024-09-16T00:09:24.938405+0000","last_became_active":"2024-09-16T00:08:36.894138+0000","last_became_peered":"2024-09-16T00:08:36.894138+0000","last_unstale":"2024-09-16T00:09:24.938405+0000","last_undegraded":"2024-09-16T00:09:24.938405+0000","last_fullsized":"2024-09-16T00:09:24.938405+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:54:19.478353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":543629,"num_objects":120,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"75'122","reported_seq":304,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.858238+0000","last_change":"2024-09-16T00:09:04.210763+0000","last_active":"2024-09-16T00:09:26.858238+0000","last_peered":"2024-09-16T00:09:26.858238+0000","last_clean":"2024-09-16T00:09:26.858238+0000","last_became_active":"2024-09-16T00:09:04.158723+0000","last_became_peered":"2024-09-16T00:09:04.158723+0000","last_unstale":"2024-09-16T00:09:26.858238+0000","last_undegraded":"2024-09-16T00:09:26.858238+0000","last_fullsized":"2024-09-16T00:09:26.858238+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:25:48.973773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":624762,"num_objects":122,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122,"num_write_kb":672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":103497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"75'155","reported_seq":220,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.684444+0000","last_change":"2024-09-16T00:08:36.890676+0000","last_active":"2024-09-16T00:09:26.684444+0000","last_peered":"2024-09-16T00:09:26.684444+0000","last_clean":"2024-09-16T00:09:26.684444+0000","last_became_active":"2024-09-16T00:08:36.889743+0000","last_became_peered":"2024-09-16T00:08:36.889743+0000","last_unstale":"2024-09-16T00:09:26.684444+0000","last_undegraded":"2024-09-16T00:09:26.684444+0000","last_fullsized":"2024-09-16T00:09:26.684444+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:02.709496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":510862,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"75'128","reported_seq":245,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.257205+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:09:25.257205+0000","last_peered":"2024-09-16T00:09:25.257205+0000","last_clean":"2024-09-16T00:09:25.257205+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:09:25.257205+0000","last_undegraded":"2024-09-16T00:09:25.257205+0000","last_fullsized":"2024-09-16T00:09:25.257205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":378325,"num_objects":128,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"75'144","reported_seq":262,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.896465+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:09:26.896465+0000","last_peered":"2024-09-16T00:09:26.896465+0000","last_clean":"2024-09-16T00:09:26.896465+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:09:26.896465+0000","last_undegraded":"2024-09-16T00:09:26.896465+0000","last_fullsized":"2024-09-16T00:09:26.896465+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":568332,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"75'124","reported_seq":242,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.802007+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:09:25.802007+0000","last_peered":"2024-09-16T00:09:25.802007+0000","last_clean":"2024-09-16T00:09:25.802007+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:09:25.802007+0000","last_undegraded":"2024-09-16T00:09:25.802007+0000","last_fullsized":"2024-09-16T00:09:25.802007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":531695,"num_objects":124,"num_object_clones":0,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"75'142","reported_seq":260,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.885306+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:09:26.885306+0000","last_peered":"2024-09-16T00:09:26.885306+0000","last_clean":"2024-09-16T00:09:26.885306+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:09:26.885306+0000","last_undegraded":"2024-09-16T00:09:26.885306+0000","last_fullsized":"2024-09-16T00:09:26.885306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538984,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"75'145","reported_seq":263,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.924093+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:09:25.924093+0000","last_peered":"2024-09-16T00:09:25.924093+0000","last_clean":"2024-09-16T00:09:25.924093+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:09:25.924093+0000","last_undegraded":"2024-09-16T00:09:25.924093+0000","last_fullsized":"2024-09-16T00:09:25.924093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462921,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":145,"num_write_kb":528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"75'160","reported_seq":277,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.487407+0000","last_change":"2024-09-16T00:07:33.631098+0000","last_active":"2024-09-16T00:09:26.487407+0000","last_peered":"2024-09-16T00:09:26.487407+0000","last_clean":"2024-09-16T00:09:26.487407+0000","last_became_active":"2024-09-16T00:07:33.630788+0000","last_became_peered":"2024-09-16T00:07:33.630788+0000","last_unstale":"2024-09-16T00:09:26.487407+0000","last_undegraded":"2024-09-16T00:09:26.487407+0000","last_fullsized":"2024-09-16T00:09:26.487407+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1073509,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":1130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"75'126","reported_seq":243,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.803849+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:09:25.803849+0000","last_peered":"2024-09-16T00:09:25.803849+0000","last_clean":"2024-09-16T00:09:25.803849+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:09:25.803849+0000","last_undegraded":"2024-09-16T00:09:25.803849+0000","last_fullsized":"2024-09-16T00:09:25.803849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":555670,"num_objects":126,"num_object_clones":0,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":126,"num_write_kb":610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"75'121","reported_seq":239,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.030593+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:09:26.030593+0000","last_peered":"2024-09-16T00:09:26.030593+0000","last_clean":"2024-09-16T00:09:26.030593+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:09:26.030593+0000","last_undegraded":"2024-09-16T00:09:26.030593+0000","last_fullsized":"2024-09-16T00:09:26.030593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":633359,"num_objects":121,"num_object_clones":0,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"75'123","reported_seq":240,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.734797+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:09:25.734797+0000","last_peered":"2024-09-16T00:09:25.734797+0000","last_clean":"2024-09-16T00:09:25.734797+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:09:25.734797+0000","last_undegraded":"2024-09-16T00:09:25.734797+0000","last_fullsized":"2024-09-16T00:09:25.734797+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":430036,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"75'140","reported_seq":232,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.616808+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:09:24.616808+0000","last_peered":"2024-09-16T00:09:24.616808+0000","last_clean":"2024-09-16T00:09:24.616808+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:09:24.616808+0000","last_undegraded":"2024-09-16T00:09:24.616808+0000","last_fullsized":"2024-09-16T00:09:24.616808+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":587102,"num_objects":140,"num_object_clones":0,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":140,"num_write_kb":649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"75'133","reported_seq":256,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.630815+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:09:24.630815+0000","last_peered":"2024-09-16T00:09:24.630815+0000","last_clean":"2024-09-16T00:09:24.630815+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:09:24.630815+0000","last_undegraded":"2024-09-16T00:09:24.630815+0000","last_fullsized":"2024-09-16T00:09:24.630815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":491137,"num_objects":133,"num_object_clones":0,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":133,"num_write_kb":549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"75'154","reported_seq":219,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:27.005401+0000","last_change":"2024-09-16T00:08:36.894101+0000","last_active":"2024-09-16T00:09:27.005401+0000","last_peered":"2024-09-16T00:09:27.005401+0000","last_clean":"2024-09-16T00:09:27.005401+0000","last_became_active":"2024-09-16T00:08:36.893807+0000","last_became_peered":"2024-09-16T00:08:36.893807+0000","last_unstale":"2024-09-16T00:09:27.005401+0000","last_undegraded":"2024-09-16T00:09:27.005401+0000","last_fullsized":"2024-09-16T00:09:27.005401+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:30:40.760753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2727617,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":154,"num_write_kb":2740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"75'142","reported_seq":259,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.181379+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:09:26.181379+0000","last_peered":"2024-09-16T00:09:26.181379+0000","last_clean":"2024-09-16T00:09:26.181379+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:09:26.181379+0000","last_undegraded":"2024-09-16T00:09:26.181379+0000","last_fullsized":"2024-09-16T00:09:26.181379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":429982,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"63'62","reported_seq":182,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.131411+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:09:24.131411+0000","last_peered":"2024-09-16T00:09:24.131411+0000","last_clean":"2024-09-16T00:09:24.131411+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:09:24.131411+0000","last_undegraded":"2024-09-16T00:09:24.131411+0000","last_fullsized":"2024-09-16T00:09:24.131411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":62,"num_write_kb":4136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"75'112","reported_seq":229,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.716399+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:09:25.716399+0000","last_peered":"2024-09-16T00:09:25.716399+0000","last_clean":"2024-09-16T00:09:25.716399+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:09:25.716399+0000","last_undegraded":"2024-09-16T00:09:25.716399+0000","last_fullsized":"2024-09-16T00:09:25.716399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":112,"log_dups_size":0,"ondisk_log_size":112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":345071,"num_objects":112,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112,"num_write_kb":394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"71'40","reported_seq":104,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.129571+0000","last_change":"2024-09-16T00:08:36.890316+0000","last_active":"2024-09-16T00:09:24.129571+0000","last_peered":"2024-09-16T00:09:24.129571+0000","last_clean":"2024-09-16T00:09:24.129571+0000","last_became_active":"2024-09-16T00:08:36.889635+0000","last_became_peered":"2024-09-16T00:08:36.889635+0000","last_unstale":"2024-09-16T00:09:24.129571+0000","last_undegraded":"2024-09-16T00:09:24.129571+0000","last_fullsized":"2024-09-16T00:09:24.129571+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:28.843883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":4157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"63'1","reported_seq":131,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.131252+0000","last_change":"2024-09-16T00:08:36.888349+0000","last_active":"2024-09-16T00:09:24.131252+0000","last_peered":"2024-09-16T00:09:24.131252+0000","last_clean":"2024-09-16T00:09:24.131252+0000","last_became_active":"2024-09-16T00:08:36.888052+0000","last_became_peered":"2024-09-16T00:08:36.888052+0000","last_unstale":"2024-09-16T00:09:24.131252+0000","last_undegraded":"2024-09-16T00:09:24.131252+0000","last_fullsized":"2024-09-16T00:09:24.131252+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:49.607254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"75'144","reported_seq":262,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.915975+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:09:25.915975+0000","last_peered":"2024-09-16T00:09:25.915975+0000","last_clean":"2024-09-16T00:09:25.915975+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:09:25.915975+0000","last_undegraded":"2024-09-16T00:09:25.915975+0000","last_fullsized":"2024-09-16T00:09:25.915975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":900365,"num_objects":144,"num_object_clones":0,"num_object_copies":288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":144,"num_write_kb":954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'4","reported_seq":124,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.135207+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:09:24.135207+0000","last_peered":"2024-09-16T00:09:24.135207+0000","last_clean":"2024-09-16T00:09:24.135207+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:09:24.135207+0000","last_undegraded":"2024-09-16T00:09:24.135207+0000","last_fullsized":"2024-09-16T00:09:24.135207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"75'135","reported_seq":252,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.891475+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:09:25.891475+0000","last_peered":"2024-09-16T00:09:25.891475+0000","last_clean":"2024-09-16T00:09:25.891475+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:09:25.891475+0000","last_undegraded":"2024-09-16T00:09:25.891475+0000","last_fullsized":"2024-09-16T00:09:25.891475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":135,"log_dups_size":0,"ondisk_log_size":135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":725752,"num_objects":135,"num_object_clones":0,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"61'1","reported_seq":120,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.129613+0000","last_change":"2024-09-16T00:07:32.630661+0000","last_active":"2024-09-16T00:09:24.129613+0000","last_peered":"2024-09-16T00:09:24.129613+0000","last_clean":"2024-09-16T00:09:24.129613+0000","last_became_active":"2024-09-16T00:07:32.629644+0000","last_became_peered":"2024-09-16T00:07:32.629644+0000","last_unstale":"2024-09-16T00:09:24.129613+0000","last_undegraded":"2024-09-16T00:09:24.129613+0000","last_fullsized":"2024-09-16T00:09:24.129613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"75'146","reported_seq":264,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.561149+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:09:26.561149+0000","last_peered":"2024-09-16T00:09:26.561149+0000","last_clean":"2024-09-16T00:09:26.561149+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:09:26.561149+0000","last_undegraded":"2024-09-16T00:09:26.561149+0000","last_fullsized":"2024-09-16T00:09:26.561149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":627551,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":146,"num_write_kb":689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":120,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.135649+0000","last_change":"2024-09-16T00:07:32.627075+0000","last_active":"2024-09-16T00:09:24.135649+0000","last_peered":"2024-09-16T00:09:24.135649+0000","last_clean":"2024-09-16T00:09:24.135649+0000","last_became_active":"2024-09-16T00:07:32.626818+0000","last_became_peered":"2024-09-16T00:07:32.626818+0000","last_unstale":"2024-09-16T00:09:24.135649+0000","last_undegraded":"2024-09-16T00:09:24.135649+0000","last_fullsized":"2024-09-16T00:09:24.135649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"75'124","reported_seq":241,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.028262+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:09:26.028262+0000","last_peered":"2024-09-16T00:09:26.028262+0000","last_clean":"2024-09-16T00:09:26.028262+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:09:26.028262+0000","last_undegraded":"2024-09-16T00:09:26.028262+0000","last_fullsized":"2024-09-16T00:09:26.028262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981961,"num_objects":124,"num_object_clones":0,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":1026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"71'7","reported_seq":126,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.163445+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:09:24.163445+0000","last_peered":"2024-09-16T00:09:24.163445+0000","last_clean":"2024-09-16T00:09:24.163445+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:09:24.163445+0000","last_undegraded":"2024-09-16T00:09:24.163445+0000","last_fullsized":"2024-09-16T00:09:24.163445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'123","reported_seq":240,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.928961+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:09:25.928961+0000","last_peered":"2024-09-16T00:09:25.928961+0000","last_clean":"2024-09-16T00:09:25.928961+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:09:25.928961+0000","last_undegraded":"2024-09-16T00:09:25.928961+0000","last_fullsized":"2024-09-16T00:09:25.928961+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":560776,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"65'22","reported_seq":87,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.129678+0000","last_change":"2024-09-16T00:08:36.914308+0000","last_active":"2024-09-16T00:09:24.129678+0000","last_peered":"2024-09-16T00:09:24.129678+0000","last_clean":"2024-09-16T00:09:24.129678+0000","last_became_active":"2024-09-16T00:08:36.890227+0000","last_became_peered":"2024-09-16T00:08:36.890227+0000","last_unstale":"2024-09-16T00:09:24.129678+0000","last_undegraded":"2024-09-16T00:09:24.129678+0000","last_fullsized":"2024-09-16T00:09:24.129678+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:31.761195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":4105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":13110,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"75'122","reported_seq":239,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:27.011298+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:09:27.011298+0000","last_peered":"2024-09-16T00:09:27.011298+0000","last_clean":"2024-09-16T00:09:27.011298+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:09:27.011298+0000","last_undegraded":"2024-09-16T00:09:27.011298+0000","last_fullsized":"2024-09-16T00:09:27.011298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520169,"num_objects":122,"num_object_clones":0,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":122,"num_write_kb":576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"74'8","reported_seq":127,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.148336+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:09:24.148336+0000","last_peered":"2024-09-16T00:09:24.148336+0000","last_clean":"2024-09-16T00:09:24.148336+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:09:24.148336+0000","last_undegraded":"2024-09-16T00:09:24.148336+0000","last_fullsized":"2024-09-16T00:09:24.148336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"75'128","reported_seq":245,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.974368+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:09:25.974368+0000","last_peered":"2024-09-16T00:09:25.974368+0000","last_clean":"2024-09-16T00:09:25.974368+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:09:25.974368+0000","last_undegraded":"2024-09-16T00:09:25.974368+0000","last_fullsized":"2024-09-16T00:09:25.974368+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":532236,"num_objects":128,"num_object_clones":0,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":128,"num_write_kb":593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"61'1","reported_seq":121,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.135104+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:09:24.135104+0000","last_peered":"2024-09-16T00:09:24.135104+0000","last_clean":"2024-09-16T00:09:24.135104+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:09:24.135104+0000","last_undegraded":"2024-09-16T00:09:24.135104+0000","last_fullsized":"2024-09-16T00:09:24.135104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"75'137","reported_seq":254,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.874250+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:09:26.874250+0000","last_peered":"2024-09-16T00:09:26.874250+0000","last_clean":"2024-09-16T00:09:26.874250+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:09:26.874250+0000","last_undegraded":"2024-09-16T00:09:26.874250+0000","last_fullsized":"2024-09-16T00:09:26.874250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2786802,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":137,"num_write_kb":2796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"62'3","reported_seq":68,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.129420+0000","last_change":"2024-09-16T00:08:36.889720+0000","last_active":"2024-09-16T00:09:24.129420+0000","last_peered":"2024-09-16T00:09:24.129420+0000","last_clean":"2024-09-16T00:09:24.129420+0000","last_became_active":"2024-09-16T00:08:36.889345+0000","last_became_peered":"2024-09-16T00:08:36.889345+0000","last_unstale":"2024-09-16T00:09:24.129420+0000","last_undegraded":"2024-09-16T00:09:24.129420+0000","last_fullsized":"2024-09-16T00:09:24.129420+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:03.170073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"75'141","reported_seq":206,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.526831+0000","last_change":"2024-09-16T00:08:36.896787+0000","last_active":"2024-09-16T00:09:26.526831+0000","last_peered":"2024-09-16T00:09:26.526831+0000","last_clean":"2024-09-16T00:09:26.526831+0000","last_became_active":"2024-09-16T00:08:36.896505+0000","last_became_peered":"2024-09-16T00:08:36.896505+0000","last_unstale":"2024-09-16T00:09:26.526831+0000","last_undegraded":"2024-09-16T00:09:26.526831+0000","last_fullsized":"2024-09-16T00:09:26.526831+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:55.997551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4953393,"num_objects":141,"num_object_clones":0,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":141,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"63'10","reported_seq":130,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.204605+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:09:24.204605+0000","last_peered":"2024-09-16T00:09:24.204605+0000","last_clean":"2024-09-16T00:09:24.204605+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:09:24.204605+0000","last_undegraded":"2024-09-16T00:09:24.204605+0000","last_fullsized":"2024-09-16T00:09:24.204605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"75'149","reported_seq":266,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.872289+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:09:26.872289+0000","last_peered":"2024-09-16T00:09:26.872289+0000","last_clean":"2024-09-16T00:09:26.872289+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:09:26.872289+0000","last_undegraded":"2024-09-16T00:09:26.872289+0000","last_fullsized":"2024-09-16T00:09:26.872289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":624777,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":149,"num_write_kb":684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"61'2","reported_seq":121,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.161713+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:09:24.161713+0000","last_peered":"2024-09-16T00:09:24.161713+0000","last_clean":"2024-09-16T00:09:24.161713+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:09:24.161713+0000","last_undegraded":"2024-09-16T00:09:24.161713+0000","last_fullsized":"2024-09-16T00:09:24.161713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"75'123","reported_seq":240,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.570738+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:09:26.570738+0000","last_peered":"2024-09-16T00:09:26.570738+0000","last_clean":"2024-09-16T00:09:26.570738+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:09:26.570738+0000","last_undegraded":"2024-09-16T00:09:26.570738+0000","last_fullsized":"2024-09-16T00:09:26.570738+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458794,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"71'4","reported_seq":124,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.204906+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:09:24.204906+0000","last_peered":"2024-09-16T00:09:24.204906+0000","last_clean":"2024-09-16T00:09:24.204906+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:09:24.204906+0000","last_undegraded":"2024-09-16T00:09:24.204906+0000","last_fullsized":"2024-09-16T00:09:24.204906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"75'127","reported_seq":244,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.612946+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:09:26.612946+0000","last_peered":"2024-09-16T00:09:26.612946+0000","last_clean":"2024-09-16T00:09:26.612946+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:09:26.612946+0000","last_undegraded":"2024-09-16T00:09:26.612946+0000","last_fullsized":"2024-09-16T00:09:26.612946+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436289,"num_objects":127,"num_object_clones":0,"num_object_copies":254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"61'2","reported_seq":121,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.163515+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:09:24.163515+0000","last_peered":"2024-09-16T00:09:24.163515+0000","last_clean":"2024-09-16T00:09:24.163515+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:09:24.163515+0000","last_undegraded":"2024-09-16T00:09:24.163515+0000","last_fullsized":"2024-09-16T00:09:24.163515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"75'142","reported_seq":260,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.845073+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:09:26.845073+0000","last_peered":"2024-09-16T00:09:26.845073+0000","last_clean":"2024-09-16T00:09:26.845073+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:09:26.845073+0000","last_undegraded":"2024-09-16T00:09:26.845073+0000","last_fullsized":"2024-09-16T00:09:26.845073+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":726708,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"66'1","reported_seq":121,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.204868+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:09:24.204868+0000","last_peered":"2024-09-16T00:09:24.204868+0000","last_clean":"2024-09-16T00:09:24.204868+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:09:24.204868+0000","last_undegraded":"2024-09-16T00:09:24.204868+0000","last_fullsized":"2024-09-16T00:09:24.204868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"75'142","reported_seq":269,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.521411+0000","last_change":"2024-09-16T00:08:47.953122+0000","last_active":"2024-09-16T00:09:26.521411+0000","last_peered":"2024-09-16T00:09:26.521411+0000","last_clean":"2024-09-16T00:09:26.521411+0000","last_became_active":"2024-09-16T00:08:47.952799+0000","last_became_peered":"2024-09-16T00:08:47.952799+0000","last_unstale":"2024-09-16T00:09:26.521411+0000","last_undegraded":"2024-09-16T00:09:26.521411+0000","last_fullsized":"2024-09-16T00:09:26.521411+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:03.447562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":603219,"num_objects":142,"num_object_clones":0,"num_object_copies":284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":142,"num_write_kb":660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":276,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.129707+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:09:24.129707+0000","last_peered":"2024-09-16T00:09:24.129707+0000","last_clean":"2024-09-16T00:09:24.129707+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:09:24.129707+0000","last_undegraded":"2024-09-16T00:09:24.129707+0000","last_fullsized":"2024-09-16T00:09:24.129707+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"66'1","reported_seq":121,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.135030+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:09:24.135030+0000","last_peered":"2024-09-16T00:09:24.135030+0000","last_clean":"2024-09-16T00:09:24.135030+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:09:24.135030+0000","last_undegraded":"2024-09-16T00:09:24.135030+0000","last_fullsized":"2024-09-16T00:09:24.135030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"75'119","reported_seq":237,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.196464+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:09:25.196464+0000","last_peered":"2024-09-16T00:09:25.196464+0000","last_clean":"2024-09-16T00:09:25.196464+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:09:25.196464+0000","last_undegraded":"2024-09-16T00:09:25.196464+0000","last_fullsized":"2024-09-16T00:09:25.196464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":523333,"num_objects":119,"num_object_clones":0,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119,"num_write_kb":573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"72'3","reported_seq":122,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.129566+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:09:24.129566+0000","last_peered":"2024-09-16T00:09:24.129566+0000","last_clean":"2024-09-16T00:09:24.129566+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:09:24.129566+0000","last_undegraded":"2024-09-16T00:09:24.129566+0000","last_fullsized":"2024-09-16T00:09:24.129566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'109","reported_seq":226,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.212347+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:09:25.212347+0000","last_peered":"2024-09-16T00:09:25.212347+0000","last_clean":"2024-09-16T00:09:25.212347+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:09:25.212347+0000","last_undegraded":"2024-09-16T00:09:25.212347+0000","last_fullsized":"2024-09-16T00:09:25.212347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":567700,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"71'1","reported_seq":121,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.204696+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:09:24.204696+0000","last_peered":"2024-09-16T00:09:24.204696+0000","last_clean":"2024-09-16T00:09:24.204696+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:09:24.204696+0000","last_undegraded":"2024-09-16T00:09:24.204696+0000","last_fullsized":"2024-09-16T00:09:24.204696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'133","reported_seq":250,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.244907+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:09:25.244907+0000","last_peered":"2024-09-16T00:09:25.244907+0000","last_clean":"2024-09-16T00:09:25.244907+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:09:25.244907+0000","last_undegraded":"2024-09-16T00:09:25.244907+0000","last_fullsized":"2024-09-16T00:09:25.244907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":556463,"num_objects":133,"num_object_clones":0,"num_object_copies":266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":133,"num_write_kb":616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"67'3","reported_seq":123,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.135141+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:09:24.135141+0000","last_peered":"2024-09-16T00:09:24.135141+0000","last_clean":"2024-09-16T00:09:24.135141+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:09:24.135141+0000","last_undegraded":"2024-09-16T00:09:24.135141+0000","last_fullsized":"2024-09-16T00:09:24.135141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'129","reported_seq":246,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.864404+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:09:26.864404+0000","last_peered":"2024-09-16T00:09:26.864404+0000","last_clean":"2024-09-16T00:09:26.864404+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:09:26.864404+0000","last_undegraded":"2024-09-16T00:09:26.864404+0000","last_fullsized":"2024-09-16T00:09:26.864404+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":691086,"num_objects":129,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"61'1","reported_seq":120,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.163406+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:09:24.163406+0000","last_peered":"2024-09-16T00:09:24.163406+0000","last_clean":"2024-09-16T00:09:24.163406+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:09:24.163406+0000","last_undegraded":"2024-09-16T00:09:24.163406+0000","last_fullsized":"2024-09-16T00:09:24.163406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'134","reported_seq":252,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.636694+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:09:26.636694+0000","last_peered":"2024-09-16T00:09:26.636694+0000","last_clean":"2024-09-16T00:09:26.636694+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:09:26.636694+0000","last_undegraded":"2024-09-16T00:09:26.636694+0000","last_fullsized":"2024-09-16T00:09:26.636694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":401125,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":134,"num_write_kb":468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":119,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.130481+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:09:24.130481+0000","last_peered":"2024-09-16T00:09:24.130481+0000","last_clean":"2024-09-16T00:09:24.130481+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:09:24.130481+0000","last_undegraded":"2024-09-16T00:09:24.130481+0000","last_fullsized":"2024-09-16T00:09:24.130481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'120","reported_seq":237,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.905162+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:09:24.905162+0000","last_peered":"2024-09-16T00:09:24.905162+0000","last_clean":"2024-09-16T00:09:24.905162+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:09:24.905162+0000","last_undegraded":"2024-09-16T00:09:24.905162+0000","last_fullsized":"2024-09-16T00:09:24.905162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":734200,"num_objects":120,"num_object_clones":0,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"75'37","reported_seq":174,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.684810+0000","last_change":"2024-09-16T00:08:36.906253+0000","last_active":"2024-09-16T00:09:24.684810+0000","last_peered":"2024-09-16T00:09:24.684810+0000","last_clean":"2024-09-16T00:09:24.684810+0000","last_became_active":"2024-09-16T00:08:36.894363+0000","last_became_peered":"2024-09-16T00:08:36.894363+0000","last_unstale":"2024-09-16T00:09:24.684810+0000","last_undegraded":"2024-09-16T00:09:24.684810+0000","last_fullsized":"2024-09-16T00:09:24.684810+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:00.308997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98,"num_write_kb":999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":4,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"75'124","reported_seq":242,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.814780+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:09:25.814780+0000","last_peered":"2024-09-16T00:09:25.814780+0000","last_clean":"2024-09-16T00:09:25.814780+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:09:25.814780+0000","last_undegraded":"2024-09-16T00:09:25.814780+0000","last_fullsized":"2024-09-16T00:09:25.814780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":725795,"num_objects":124,"num_object_clones":0,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"61'3","reported_seq":122,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.129700+0000","last_change":"2024-09-16T00:07:32.629549+0000","last_active":"2024-09-16T00:09:24.129700+0000","last_peered":"2024-09-16T00:09:24.129700+0000","last_clean":"2024-09-16T00:09:24.129700+0000","last_became_active":"2024-09-16T00:07:32.629223+0000","last_became_peered":"2024-09-16T00:07:32.629223+0000","last_unstale":"2024-09-16T00:09:24.129700+0000","last_undegraded":"2024-09-16T00:09:24.129700+0000","last_fullsized":"2024-09-16T00:09:24.129700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'134","reported_seq":251,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.808030+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:09:25.808030+0000","last_peered":"2024-09-16T00:09:25.808030+0000","last_clean":"2024-09-16T00:09:25.808030+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:09:25.808030+0000","last_undegraded":"2024-09-16T00:09:25.808030+0000","last_fullsized":"2024-09-16T00:09:25.808030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":430405,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":134,"num_write_kb":488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"71'1","reported_seq":120,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.129433+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:09:24.129433+0000","last_peered":"2024-09-16T00:09:24.129433+0000","last_clean":"2024-09-16T00:09:24.129433+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:09:24.129433+0000","last_undegraded":"2024-09-16T00:09:24.129433+0000","last_fullsized":"2024-09-16T00:09:24.129433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"75'155","reported_seq":272,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.707743+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:09:26.707743+0000","last_peered":"2024-09-16T00:09:26.707743+0000","last_clean":"2024-09-16T00:09:26.707743+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:09:26.707743+0000","last_undegraded":"2024-09-16T00:09:26.707743+0000","last_fullsized":"2024-09-16T00:09:26.707743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":567179,"num_objects":155,"num_object_clones":0,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":155,"num_write_kb":629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"61'1","reported_seq":120,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.163316+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:09:24.163316+0000","last_peered":"2024-09-16T00:09:24.163316+0000","last_clean":"2024-09-16T00:09:24.163316+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:09:24.163316+0000","last_undegraded":"2024-09-16T00:09:24.163316+0000","last_fullsized":"2024-09-16T00:09:24.163316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"75'132","reported_seq":250,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.159402+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:09:25.159402+0000","last_peered":"2024-09-16T00:09:25.159402+0000","last_clean":"2024-09-16T00:09:25.159402+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:09:25.159402+0000","last_undegraded":"2024-09-16T00:09:25.159402+0000","last_fullsized":"2024-09-16T00:09:25.159402+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":132,"log_dups_size":0,"ondisk_log_size":132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":651529,"num_objects":132,"num_object_clones":0,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":132,"num_write_kb":708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"63'2","reported_seq":121,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.163333+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:09:24.163333+0000","last_peered":"2024-09-16T00:09:24.163333+0000","last_clean":"2024-09-16T00:09:24.163333+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:09:24.163333+0000","last_undegraded":"2024-09-16T00:09:24.163333+0000","last_fullsized":"2024-09-16T00:09:24.163333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"75'134","reported_seq":252,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.124902+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:09:25.124902+0000","last_peered":"2024-09-16T00:09:25.124902+0000","last_clean":"2024-09-16T00:09:25.124902+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:09:25.124902+0000","last_undegraded":"2024-09-16T00:09:25.124902+0000","last_fullsized":"2024-09-16T00:09:25.124902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":497339,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":134,"num_write_kb":559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"64'6","reported_seq":125,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.130321+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:09:24.130321+0000","last_peered":"2024-09-16T00:09:24.130321+0000","last_clean":"2024-09-16T00:09:24.130321+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:09:24.130321+0000","last_undegraded":"2024-09-16T00:09:24.130321+0000","last_fullsized":"2024-09-16T00:09:24.130321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"75'134","reported_seq":199,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.929466+0000","last_change":"2024-09-16T00:08:36.890347+0000","last_active":"2024-09-16T00:09:26.929466+0000","last_peered":"2024-09-16T00:09:26.929466+0000","last_clean":"2024-09-16T00:09:26.929466+0000","last_became_active":"2024-09-16T00:08:36.890009+0000","last_became_peered":"2024-09-16T00:08:36.890009+0000","last_unstale":"2024-09-16T00:09:26.929466+0000","last_undegraded":"2024-09-16T00:09:26.929466+0000","last_fullsized":"2024-09-16T00:09:26.929466+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:24.298978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":568003,"num_objects":134,"num_object_clones":0,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":134,"num_write_kb":620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"75'124","reported_seq":242,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.833217+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:09:26.833217+0000","last_peered":"2024-09-16T00:09:26.833217+0000","last_clean":"2024-09-16T00:09:26.833217+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:09:26.833217+0000","last_undegraded":"2024-09-16T00:09:26.833217+0000","last_fullsized":"2024-09-16T00:09:26.833217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":400420,"num_objects":124,"num_object_clones":0,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":124,"num_write_kb":463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"66'3","reported_seq":122,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.129773+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:09:24.129773+0000","last_peered":"2024-09-16T00:09:24.129773+0000","last_clean":"2024-09-16T00:09:24.129773+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:09:24.129773+0000","last_undegraded":"2024-09-16T00:09:24.129773+0000","last_fullsized":"2024-09-16T00:09:24.129773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"75'123","reported_seq":241,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:26.599526+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:09:26.599526+0000","last_peered":"2024-09-16T00:09:26.599526+0000","last_clean":"2024-09-16T00:09:26.599526+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:09:26.599526+0000","last_undegraded":"2024-09-16T00:09:26.599526+0000","last_fullsized":"2024-09-16T00:09:26.599526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570450,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"75'78","reported_seq":215,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.414137+0000","last_change":"2024-09-16T00:08:36.900704+0000","last_active":"2024-09-16T00:09:25.414137+0000","last_peered":"2024-09-16T00:09:25.414137+0000","last_clean":"2024-09-16T00:09:25.414137+0000","last_became_active":"2024-09-16T00:08:36.892520+0000","last_became_peered":"2024-09-16T00:08:36.892520+0000","last_unstale":"2024-09-16T00:09:25.414137+0000","last_undegraded":"2024-09-16T00:09:25.414137+0000","last_fullsized":"2024-09-16T00:09:25.414137+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:30.056041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":79,"num_write_kb":8260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"75'123","reported_seq":215,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.957709+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:09:24.957709+0000","last_peered":"2024-09-16T00:09:24.957709+0000","last_clean":"2024-09-16T00:09:24.957709+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:09:24.957709+0000","last_undegraded":"2024-09-16T00:09:24.957709+0000","last_fullsized":"2024-09-16T00:09:24.957709+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":908782,"num_objects":123,"num_object_clones":0,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":123,"num_write_kb":958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"72'39","reported_seq":159,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.131524+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:09:24.131524+0000","last_peered":"2024-09-16T00:09:24.131524+0000","last_clean":"2024-09-16T00:09:24.131524+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:09:24.131524+0000","last_undegraded":"2024-09-16T00:09:24.131524+0000","last_fullsized":"2024-09-16T00:09:24.131524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":4141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"75'129","reported_seq":257,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:25.873917+0000","last_change":"2024-09-16T00:08:36.889082+0000","last_active":"2024-09-16T00:09:25.873917+0000","last_peered":"2024-09-16T00:09:25.873917+0000","last_clean":"2024-09-16T00:09:25.873917+0000","last_became_active":"2024-09-16T00:08:36.888807+0000","last_became_peered":"2024-09-16T00:08:36.888807+0000","last_unstale":"2024-09-16T00:09:25.873917+0000","last_undegraded":"2024-09-16T00:09:25.873917+0000","last_fullsized":"2024-09-16T00:09:25.873917+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:22.825324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":407490,"num_objects":129,"num_object_clones":0,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"65'58","reported_seq":177,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-16T00:09:24.148273+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:09:24.148273+0000","last_peered":"2024-09-16T00:09:24.148273+0000","last_clean":"2024-09-16T00:09:24.148273+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:09:24.148273+0000","last_undegraded":"2024-09-16T00:09:24.148273+0000","last_fullsized":"2024-09-16T00:09:24.148273+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":4138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":51397521,"num_objects":8432,"num_object_clones":0,"num_object_copies":16864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8432,"num_write_kb":54631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":384148,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114737152,"data_stored":102891791,"data_compressed":11674515,"data_compressed_allocated":14434304,"data_compressed_original":42958848,"omap_allocated":0,"internal_metadata":0},"log_size":8433,"ondisk_log_size":8433,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":33555902,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":511,"num_write_kb":34557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":13110,"num_keys_recovered":4,"num_objects_omap":54,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":14008320,"data_stored":67111804,"data_compressed":10401512,"data_compressed_allocated":12443648,"data_compressed_original":65601536,"omap_allocated":0,"internal_metadata":0},"log_size":405,"ondisk_log_size":405,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59956,"kb_used_data":20448,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94311884,"statfs":{"total":96636764160,"available":96575369216,"internally_reserved":0,"allocated":20938752,"data_stored":34996117,"data_compressed":4842679,"data_compressed_allocated":5910528,"data_compressed_original":26202112,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52628,"kb_used_data":17280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94319212,"statfs":{"total":96636764160,"available":96582873088,"internally_reserved":0,"allocated":17694720,"data_stored":30557260,"data_compressed":3818314,"data_compressed_allocated":4595712,"data_compressed_original":21839872,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53536,"kb_used_data":14032,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94318304,"statfs":{"total":96636764160,"available":96581943296,"internally_reserved":0,"allocated":14368768,"data_stored":11648573,"data_compressed":1622989,"data_compressed_allocated":1998848,"data_compressed_original":5574656,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509469,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57244,"kb_used_data":17740,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94314596,"statfs":{"total":96636764160,"available":96578146304,"internally_reserved":0,"allocated":18165760,"data_stored":20817633,"data_compressed":2575306,"data_compressed_allocated":3182592,"data_compressed_original":12292096,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509467,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55980,"kb_used_data":16472,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94315860,"statfs":{"total":96636764160,"available":96579440640,"internally_reserved":0,"allocated":16867328,"data_stored":23656812,"data_compressed":3660183,"data_compressed_allocated":4378624,"data_compressed_original":17358848,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48908,"kb_used_data":13560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94322932,"statfs":{"total":96636764160,"available":96586682368,"internally_reserved":0,"allocated":13885440,"data_stored":12658855,"data_compressed":1193335,"data_compressed_allocated":1486848,"data_compressed_original":4526080,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63016,"kb_used_data":23512,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94308824,"statfs":{"total":96636764160,"available":96572235776,"internally_reserved":0,"allocated":24076288,"data_stored":31159940,"data_compressed":3167895,"data_compressed_allocated":3891200,"data_compressed_original":17027072,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43700,"kb_used_data":8352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328140,"statfs":{"total":96636764160,"available":96592015360,"internally_reserved":0,"allocated":8552448,"data_stored":8123793,"data_compressed":1210013,"data_compressed_allocated":1462272,"data_compressed_original":4759552,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1748992,"data_stored":8389246,"data_compressed":1255393,"data_compressed_allocated":1544192,"data_compressed_original":8196096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2703360,"data_stored":12583584,"data_compressed":1983184,"data_compressed_allocated":2338816,"data_compressed_original":12226560,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1675264,"data_stored":8388734,"data_compressed":1280256,"data_compressed_allocated":1560576,"data_compressed_original":8286208,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3444736,"data_stored":16777888,"data_compressed":2643028,"data_compressed_allocated":3112960,"data_compressed_original":16453632,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4423680,"data_stored":20972102,"data_compressed":3239651,"data_compressed_allocated":3887104,"data_compressed_original":20439040,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":21536768,"data_stored":21834464,"data_compressed":1880125,"data_compressed_allocated":2301952,"data_compressed_original":8257536,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13053952,"data_stored":12270921,"data_compressed":1193335,"data_compressed_allocated":1486848,"data_compressed_original":4526080,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7876608,"data_stored":7838583,"data_compressed":1229411,"data_compressed_allocated":1482752,"data_compressed_original":4808704,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":13504512,"data_stored":10761615,"data_compressed":1676999,"data_compressed_allocated":2039808,"data_compressed_original":5132288,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15843328,"data_stored":12180975,"data_compressed":1301064,"data_compressed_allocated":1630208,"data_compressed_original":4018176,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":13725696,"data_stored":11409463,"data_compressed":1632148,"data_compressed_allocated":2011136,"data_compressed_original":5595136,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":13402112,"data_stored":13391069,"data_compressed":1175286,"data_compressed_allocated":1482752,"data_compressed_original":5386240,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15794176,"data_stored":13204701,"data_compressed":1586147,"data_compressed_allocated":1998848,"data_compressed_original":5234688,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:09:30.080 INFO:tasks.thrashosds.thrasher:Setting 3.3a to [5, 6] 2024-09-16T00:09:30.080 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.3a', '5', '6'] 2024-09-16T00:09:30.081 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.3a 5 6 2024-09-16T00:09:30.081 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:30.075+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:30.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.137+0000 7f284a7fc640 1 -- 172.21.15.139:0/2172187411 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v77) ==== 87+0+0 (secure 0 0 0) 0x7f2854059370 con 0x7f285806eae0 2024-09-16T00:09:30.140 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is set 2024-09-16T00:09:30.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.141+0000 7f285fe65640 1 -- 172.21.15.139:0/2172187411 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f283c06c2e0 msgr2=0x7f283c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:30.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.142+0000 7f285fe65640 1 --2- 172.21.15.139:0/2172187411 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f283c06c2e0 0x7f283c06e7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f284c007fd0 tx=0x7f284c009040 comp rx=0 tx=0).stop 2024-09-16T00:09:30.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.142+0000 7f285fe65640 1 -- 172.21.15.139:0/2172187411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f285806eae0 msgr2=0x7f2858153420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:30.144 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:30.144 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:30.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:30.145 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:30.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.145 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'66)", 2024-09-16T00:09:30.145 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:09:30.145 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 125.975146642, 2024-09-16T00:09:30.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032925509999999999, 2024-09-16T00:09:30.145 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.145 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.145 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.145 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:09:30.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:09:30.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166177+0000", 2024-09-16T00:09:30.147 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:09:30.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.147 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.147 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166216+0000", 2024-09-16T00:09:30.147 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9008000000000003e-05 2024-09-16T00:09:30.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.147 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.147 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166226+0000", 2024-09-16T00:09:30.147 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0081999999999999e-05 2024-09-16T00:09:30.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166230+0000", 2024-09-16T00:09:30.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5079999999999999e-06 2024-09-16T00:09:30.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166259+0000", 2024-09-16T00:09:30.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8928000000000002e-05 2024-09-16T00:09:30.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166271+0000", 2024-09-16T00:09:30.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1531e-05 2024-09-16T00:09:30.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.149 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169449+0000", 2024-09-16T00:09:30.149 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003178143 2024-09-16T00:09:30.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.149 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169468+0000", 2024-09-16T00:09:30.149 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9700000000000001e-05 2024-09-16T00:09:30.149 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.150 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.150 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.150 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.150 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'66)", 2024-09-16T00:09:30.150 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:09:30.150 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 125.97468518300001, 2024-09-16T00:09:30.150 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003179374, 2024-09-16T00:09:30.150 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.150 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.150 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.150 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.150 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.151 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.151 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:09:30.151 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.151 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.151 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.151 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:09:30.151 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.151 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.151 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.151 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166638+0000", 2024-09-16T00:09:30.151 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2300000000000002e-07 2024-09-16T00:09:30.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.152 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.152 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.152 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166641+0000", 2024-09-16T00:09:30.152 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0079999999999998e-06 2024-09-16T00:09:30.152 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.152 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.152 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.152 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166646+0000", 2024-09-16T00:09:30.152 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4970000000000001e-06 2024-09-16T00:09:30.152 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.152 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.152 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.152 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166650+0000", 2024-09-16T00:09:30.152 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5729999999999999e-06 2024-09-16T00:09:30.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.153 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.153 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.153 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166682+0000", 2024-09-16T00:09:30.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2218999999999998e-05 2024-09-16T00:09:30.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.153 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.153 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.153 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166694+0000", 2024-09-16T00:09:30.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1902999999999999e-05 2024-09-16T00:09:30.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.153 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.153 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.153 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169784+0000", 2024-09-16T00:09:30.154 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003089545 2024-09-16T00:09:30.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.154 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.154 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.154 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169817+0000", 2024-09-16T00:09:30.154 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3206e-05 2024-09-16T00:09:30.154 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.154 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.154 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.154 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.154 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:130 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'88, mlcod=17'85)", 2024-09-16T00:09:30.154 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:30.154 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 125.531441794, 2024-09-16T00:09:30.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002880456, 2024-09-16T00:09:30.155 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.155 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.155 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.155 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:30.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:30.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.156 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.156 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.156 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609881+0000", 2024-09-16T00:09:30.156 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5699999999999998e-07 2024-09-16T00:09:30.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.156 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.156 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.156 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609885+0000", 2024-09-16T00:09:30.156 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.608e-06 2024-09-16T00:09:30.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.156 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.156 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.156 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609891+0000", 2024-09-16T00:09:30.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2750000000000001e-06 2024-09-16T00:09:30.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609895+0000", 2024-09-16T00:09:30.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.54e-06 2024-09-16T00:09:30.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609931+0000", 2024-09-16T00:09:30.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5893999999999999e-05 2024-09-16T00:09:30.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.158 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.609945+0000", 2024-09-16T00:09:30.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4259e-05 2024-09-16T00:09:30.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.158 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.158 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612721+0000", 2024-09-16T00:09:30.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027757849999999998 2024-09-16T00:09:30.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.158 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.158 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.612761+0000", 2024-09-16T00:09:30.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0638e-05 2024-09-16T00:09:30.158 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.159 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.159 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.159 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.159 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=17'85)", 2024-09-16T00:09:30.159 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:09:30.159 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 125.531057821, 2024-09-16T00:09:30.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028016529999999999, 2024-09-16T00:09:30.159 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.159 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.160 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.160 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.160 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.160 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:09:30.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.160 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.160 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:09:30.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.161 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.161 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610265+0000", 2024-09-16T00:09:30.161 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.63e-07 2024-09-16T00:09:30.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.161 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.161 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.161 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610344+0000", 2024-09-16T00:09:30.161 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8996000000000001e-05 2024-09-16T00:09:30.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.161 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.161 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.161 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610352+0000", 2024-09-16T00:09:30.161 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9330000000000001e-06 2024-09-16T00:09:30.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610356+0000", 2024-09-16T00:09:30.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0790000000000002e-06 2024-09-16T00:09:30.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610388+0000", 2024-09-16T00:09:30.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1541000000000002e-05 2024-09-16T00:09:30.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.610400+0000", 2024-09-16T00:09:30.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2075e-05 2024-09-16T00:09:30.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613045+0000", 2024-09-16T00:09:30.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002645213 2024-09-16T00:09:30.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.613066+0000", 2024-09-16T00:09:30.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1353e-05 2024-09-16T00:09:30.163 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.163 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.164 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.164 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.164 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:139 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'97, mlcod=18'87)", 2024-09-16T00:09:30.164 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:09:30.164 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 125.524979473, 2024-09-16T00:09:30.164 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030908139999999999, 2024-09-16T00:09:30.164 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.165 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.165 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.165 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.165 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.165 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:09:30.165 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.165 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616343+0000", 2024-09-16T00:09:30.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616344+0000", 2024-09-16T00:09:30.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3000000000000001e-07 2024-09-16T00:09:30.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616347+0000", 2024-09-16T00:09:30.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7740000000000002e-06 2024-09-16T00:09:30.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616355+0000", 2024-09-16T00:09:30.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4699999999999996e-06 2024-09-16T00:09:30.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616358+0000", 2024-09-16T00:09:30.168 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5899999999999999e-06 2024-09-16T00:09:30.168 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.168 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.168 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.168 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616393+0000", 2024-09-16T00:09:30.168 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4529000000000001e-05 2024-09-16T00:09:30.168 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.168 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.168 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.168 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.616404+0000", 2024-09-16T00:09:30.168 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0726e-05 2024-09-16T00:09:30.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.169 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619411+0000", 2024-09-16T00:09:30.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030076249999999999 2024-09-16T00:09:30.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.169 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.619434+0000", 2024-09-16T00:09:30.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-05 2024-09-16T00:09:30.170 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.170 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.170 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.170 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.170 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:30.170 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:09:30.170 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 111.52541984, 2024-09-16T00:09:30.170 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061593860000000002, 2024-09-16T00:09:30.170 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.170 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.171 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.171 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:09:30.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:09:30.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.172 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.172 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615907+0000", 2024-09-16T00:09:30.172 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9389999999999999e-06 2024-09-16T00:09:30.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.172 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.172 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615920+0000", 2024-09-16T00:09:30.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2989999999999999e-05 2024-09-16T00:09:30.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.173 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.173 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.173 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615931+0000", 2024-09-16T00:09:30.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.164e-05 2024-09-16T00:09:30.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.173 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.173 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.174 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615939+0000", 2024-09-16T00:09:30.174 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8029999999999997e-06 2024-09-16T00:09:30.174 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.174 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.174 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.174 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616310+0000", 2024-09-16T00:09:30.174 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00037111400000000003 2024-09-16T00:09:30.174 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.174 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.174 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616406+0000", 2024-09-16T00:09:30.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5372999999999996e-05 2024-09-16T00:09:30.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:30.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616514+0000", 2024-09-16T00:09:30.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108564 2024-09-16T00:09:30.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618158+0000", 2024-09-16T00:09:30.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016442449999999999 2024-09-16T00:09:30.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621972+0000", 2024-09-16T00:09:30.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038132299999999999 2024-09-16T00:09:30.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622018+0000", 2024-09-16T00:09:30.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6239000000000002e-05 2024-09-16T00:09:30.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622062+0000", 2024-09-16T00:09:30.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4249000000000003e-05 2024-09-16T00:09:30.177 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.177 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.177 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.142+0000 7f285fe65640 1 --2- 172.21.15.139:0/2172187411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f285806eae0 0x7f2858153420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f285400e960 tx=0x7f285400ee20 comp rx=0 tx=0).stop 2024-09-16T00:09:30.178 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:30.179 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:09:30.179 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 111.52536637199999, 2024-09-16T00:09:30.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006258914, 2024-09-16T00:09:30.179 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.179 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.179 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.179 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.179 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:09:30.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.180 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:09:30.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.180 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615957+0000", 2024-09-16T00:09:30.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1399999999999997e-07 2024-09-16T00:09:30.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615960+0000", 2024-09-16T00:09:30.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0860000000000002e-06 2024-09-16T00:09:30.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615967+0000", 2024-09-16T00:09:30.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9059999999999996e-06 2024-09-16T00:09:30.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615972+0000", 2024-09-16T00:09:30.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5679999999999999e-06 2024-09-16T00:09:30.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616717+0000", 2024-09-16T00:09:30.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074438 2024-09-16T00:09:30.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616781+0000", 2024-09-16T00:09:30.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4808999999999996e-05 2024-09-16T00:09:30.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:30.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616885+0000", 2024-09-16T00:09:30.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010404900000000001 2024-09-16T00:09:30.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:09:30.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012773870000000001 2024-09-16T00:09:30.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622116+0000", 2024-09-16T00:09:30.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039533470000000003 2024-09-16T00:09:30.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622173+0000", 2024-09-16T00:09:30.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6347000000000001e-05 2024-09-16T00:09:30.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622215+0000", 2024-09-16T00:09:30.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2521000000000002e-05 2024-09-16T00:09:30.184 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.185 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.185 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.185 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:30.185 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:30.185 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 111.525342635, 2024-09-16T00:09:30.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041725529999999999, 2024-09-16T00:09:30.185 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.185 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.185 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.185 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.186 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.186 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:30.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.186 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.186 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.186 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:30.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.186 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.186 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.186 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:30.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8799999999999998e-07 2024-09-16T00:09:30.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615990+0000", 2024-09-16T00:09:30.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2150000000000007e-06 2024-09-16T00:09:30.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:09:30.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7159999999999997e-06 2024-09-16T00:09:30.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616001+0000", 2024-09-16T00:09:30.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9339999999999997e-06 2024-09-16T00:09:30.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616293+0000", 2024-09-16T00:09:30.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029210899999999998 2024-09-16T00:09:30.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616401+0000", 2024-09-16T00:09:30.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108087 2024-09-16T00:09:30.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:30.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616561+0000", 2024-09-16T00:09:30.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015940600000000001 2024-09-16T00:09:30.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618146+0000", 2024-09-16T00:09:30.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015857040000000001 2024-09-16T00:09:30.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620091+0000", 2024-09-16T00:09:30.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001944837 2024-09-16T00:09:30.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620126+0000", 2024-09-16T00:09:30.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4242999999999997e-05 2024-09-16T00:09:30.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620152+0000", 2024-09-16T00:09:30.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6914e-05 2024-09-16T00:09:30.191 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.191 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.191 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.191 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:30.191 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:30.191 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 111.52530441, 2024-09-16T00:09:30.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00630628, 2024-09-16T00:09:30.191 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.192 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.192 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.192 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.192 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:30.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.192 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:30.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:09:30.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5999999999999999e-07 2024-09-16T00:09:30.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616022+0000", 2024-09-16T00:09:30.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0149999999999999e-06 2024-09-16T00:09:30.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616031+0000", 2024-09-16T00:09:30.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9770000000000006e-06 2024-09-16T00:09:30.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616036+0000", 2024-09-16T00:09:30.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2639999999999999e-06 2024-09-16T00:09:30.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617057+0000", 2024-09-16T00:09:30.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010212999999999999 2024-09-16T00:09:30.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617146+0000", 2024-09-16T00:09:30.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9234000000000007e-05 2024-09-16T00:09:30.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:30.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617232+0000", 2024-09-16T00:09:30.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5218999999999998e-05 2024-09-16T00:09:30.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619107+0000", 2024-09-16T00:09:30.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001875016 2024-09-16T00:09:30.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622259+0000", 2024-09-16T00:09:30.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031527930000000001 2024-09-16T00:09:30.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622290+0000", 2024-09-16T00:09:30.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1022999999999998e-05 2024-09-16T00:09:30.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622324+0000", 2024-09-16T00:09:30.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3979000000000001e-05 2024-09-16T00:09:30.197 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.197 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.197 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.197 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:30.198 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:09:30.198 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 111.52527910800001, 2024-09-16T00:09:30.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042412739999999997, 2024-09-16T00:09:30.198 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.198 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.198 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.198 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:09:30.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:09:30.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616044+0000", 2024-09-16T00:09:30.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:09:30.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616052+0000", 2024-09-16T00:09:30.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.439e-06 2024-09-16T00:09:30.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616059+0000", 2024-09-16T00:09:30.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1389999999999998e-06 2024-09-16T00:09:30.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616065+0000", 2024-09-16T00:09:30.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:09:30.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616823+0000", 2024-09-16T00:09:30.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075781199999999996 2024-09-16T00:09:30.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616918+0000", 2024-09-16T00:09:30.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5174999999999996e-05 2024-09-16T00:09:30.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:30.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617019+0000", 2024-09-16T00:09:30.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101161 2024-09-16T00:09:30.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618503+0000", 2024-09-16T00:09:30.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014836750000000001 2024-09-16T00:09:30.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620201+0000", 2024-09-16T00:09:30.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00169805 2024-09-16T00:09:30.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620255+0000", 2024-09-16T00:09:30.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4499000000000001e-05 2024-09-16T00:09:30.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620285+0000", 2024-09-16T00:09:30.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9349e-05 2024-09-16T00:09:30.205 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.205 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.205 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.205 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:30.205 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:09:30.205 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 111.525250117, 2024-09-16T00:09:30.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063682799999999996, 2024-09-16T00:09:30.206 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.206 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.206 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:09:30.206 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:09:30.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:09:30.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:09:30.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9900000000000001e-07 2024-09-16T00:09:30.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616079+0000", 2024-09-16T00:09:30.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9009999999999999e-06 2024-09-16T00:09:30.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616086+0000", 2024-09-16T00:09:30.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.776e-06 2024-09-16T00:09:30.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616091+0000", 2024-09-16T00:09:30.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5600000000000001e-06 2024-09-16T00:09:30.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618339+0000", 2024-09-16T00:09:30.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022476179999999998 2024-09-16T00:09:30.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:09:30.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9025999999999997e-05 2024-09-16T00:09:30.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622441+0000", 2024-09-16T00:09:30.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040029999999999996 2024-09-16T00:09:30.211 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.211 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.211 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.211 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:221 3.1b 3:dad7a251:::100000002d7.00000000:head [write 0~30412] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:30.211 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:09:30.211 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.392041927000001, 2024-09-16T00:09:30.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034396769999999999, 2024-09-16T00:09:30.212 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.212 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.212 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.212 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:30.212 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:30.212 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:30.212 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 221 2024-09-16T00:09:30.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.212 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.142+0000 7f285fe65640 1 -- 172.21.15.139:0/2172187411 shutdown_connections 2024-09-16T00:09:30.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.142+0000 7f285fe65640 1 --2- 172.21.15.139:0/2172187411 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f283c06c2e0 0x7f283c06e7a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:30.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.142+0000 7f285fe65640 1 --2- 172.21.15.139:0/2172187411 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2858154aa0 0x7f285814ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:30.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.142+0000 7f285fe65640 1 --2- 172.21.15.139:0/2172187411 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f285806f360 0x7f2858155f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:30.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:09:30.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:09:30.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749282+0000", 2024-09-16T00:09:30.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3769999999999999e-06 2024-09-16T00:09:30.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749300+0000", 2024-09-16T00:09:30.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8204999999999999e-05 2024-09-16T00:09:30.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749311+0000", 2024-09-16T00:09:30.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0599e-05 2024-09-16T00:09:30.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749318+0000", 2024-09-16T00:09:30.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9789999999999996e-06 2024-09-16T00:09:30.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749367+0000", 2024-09-16T00:09:30.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9382999999999998e-05 2024-09-16T00:09:30.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749450+0000", 2024-09-16T00:09:30.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-05 2024-09-16T00:09:30.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:30.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749526+0000", 2024-09-16T00:09:30.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5564999999999999e-05 2024-09-16T00:09:30.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.750601+0000", 2024-09-16T00:09:30.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010751199999999999 2024-09-16T00:09:30.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.218 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752665+0000", 2024-09-16T00:09:30.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00206442 2024-09-16T00:09:30.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.218 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752696+0000", 2024-09-16T00:09:30.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0476000000000001e-05 2024-09-16T00:09:30.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.219 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.219 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752720+0000", 2024-09-16T00:09:30.219 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4482999999999999e-05 2024-09-16T00:09:30.219 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.219 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.219 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.219 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.219 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:323 3.3 e58/20 3:c319d237:::10000000354.00000000:head v 58'3, mlcod=57'1)", 2024-09-16T00:09:30.219 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:09:30.219 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.108662156000001, 2024-09-16T00:09:30.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034011000000000002, 2024-09-16T00:09:30.220 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.220 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.220 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.220 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:09:30.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:09:30.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032661+0000", 2024-09-16T00:09:30.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7000000000000003e-07 2024-09-16T00:09:30.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032670+0000", 2024-09-16T00:09:30.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.602e-06 2024-09-16T00:09:30.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032682+0000", 2024-09-16T00:09:30.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1754e-05 2024-09-16T00:09:30.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032688+0000", 2024-09-16T00:09:30.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8140000000000003e-06 2024-09-16T00:09:30.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032740+0000", 2024-09-16T00:09:30.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1981000000000003e-05 2024-09-16T00:09:30.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032765+0000", 2024-09-16T00:09:30.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4097e-05 2024-09-16T00:09:30.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.036023+0000", 2024-09-16T00:09:30.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032585240000000001 2024-09-16T00:09:30.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.036061+0000", 2024-09-16T00:09:30.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8457999999999997e-05 2024-09-16T00:09:30.224 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.224 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.224 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.224 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:664 3.39 e59/20 3:9c9d3e51:::100000004f6.00000000:head v 59'7, mlcod=58'5)", 2024-09-16T00:09:30.224 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:30.224 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.920992996999999, 2024-09-16T00:09:30.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035253329999999999, 2024-09-16T00:09:30.224 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.224 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.224 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.225 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:30.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:30.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:30.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2399999999999996e-07 2024-09-16T00:09:30.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220338+0000", 2024-09-16T00:09:30.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0120000000000005e-06 2024-09-16T00:09:30.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220353+0000", 2024-09-16T00:09:30.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4718999999999999e-05 2024-09-16T00:09:30.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220361+0000", 2024-09-16T00:09:30.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6559999999999992e-06 2024-09-16T00:09:30.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220422+0000", 2024-09-16T00:09:30.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.071e-05 2024-09-16T00:09:30.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220446+0000", 2024-09-16T00:09:30.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4556e-05 2024-09-16T00:09:30.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.223813+0000", 2024-09-16T00:09:30.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033666439999999998 2024-09-16T00:09:30.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.223855+0000", 2024-09-16T00:09:30.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2111999999999999e-05 2024-09-16T00:09:30.228 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.228 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.228 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.229 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:681 3.17 3:ea55746f:::1000000050b.00000000:head [write 0~1071] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:30.229 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:30.229 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.876250032000002, 2024-09-16T00:09:30.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036588620000000001, 2024-09-16T00:09:30.229 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.229 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.229 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.229 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:30.229 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:30.229 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:30.229 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 681 2024-09-16T00:09:30.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.230 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:30.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:30.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:30.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7600000000000006e-07 2024-09-16T00:09:30.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265080+0000", 2024-09-16T00:09:30.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8759999999999999e-06 2024-09-16T00:09:30.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265094+0000", 2024-09-16T00:09:30.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.411e-05 2024-09-16T00:09:30.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265102+0000", 2024-09-16T00:09:30.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9640000000000003e-06 2024-09-16T00:09:30.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265156+0000", 2024-09-16T00:09:30.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3375000000000003e-05 2024-09-16T00:09:30.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265229+0000", 2024-09-16T00:09:30.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3147999999999998e-05 2024-09-16T00:09:30.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:30.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265304+0000", 2024-09-16T00:09:30.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4568999999999997e-05 2024-09-16T00:09:30.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265747+0000", 2024-09-16T00:09:30.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00044363099999999999 2024-09-16T00:09:30.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.268635+0000", 2024-09-16T00:09:30.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028873459999999998 2024-09-16T00:09:30.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.268688+0000", 2024-09-16T00:09:30.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3155999999999997e-05 2024-09-16T00:09:30.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.268731+0000", 2024-09-16T00:09:30.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3711000000000002e-05 2024-09-16T00:09:30.235 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.235 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.235 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.235 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:3099 3.6 3:617936f3:::10000000f78.00000000:head [write 0~1641] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T00:09:30.235 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:06.096600+0000", 2024-09-16T00:09:30.235 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 24.044722851, 2024-09-16T00:09:30.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031984349999999999, 2024-09-16T00:09:30.236 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.236 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.236 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.236 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:30.236 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:30.236 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:30.236 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 3099 2024-09-16T00:09:30.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.236 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096600+0000", 2024-09-16T00:09:30.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096600+0000", 2024-09-16T00:09:30.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096602+0000", 2024-09-16T00:09:30.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4889999999999998e-06 2024-09-16T00:09:30.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096612+0000", 2024-09-16T00:09:30.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5629999999999998e-06 2024-09-16T00:09:30.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096631+0000", 2024-09-16T00:09:30.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9369999999999999e-05 2024-09-16T00:09:30.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096641+0000", 2024-09-16T00:09:30.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0098999999999999e-05 2024-09-16T00:09:30.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096705+0000", 2024-09-16T00:09:30.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3551000000000002e-05 2024-09-16T00:09:30.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op must wait for map", 2024-09-16T00:09:30.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096711+0000", 2024-09-16T00:09:30.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5660000000000003e-06 2024-09-16T00:09:30.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.098401+0000", 2024-09-16T00:09:30.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016899840000000001 2024-09-16T00:09:30.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.098514+0000", 2024-09-16T00:09:30.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000112551 2024-09-16T00:09:30.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.242 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:30.242 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.098636+0000", 2024-09-16T00:09:30.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000122488 2024-09-16T00:09:30.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.242 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.242 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.099270+0000", 2024-09-16T00:09:30.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00063404300000000004 2024-09-16T00:09:30.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.099702+0000", 2024-09-16T00:09:30.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000431956 2024-09-16T00:09:30.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.099761+0000", 2024-09-16T00:09:30.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8421000000000002e-05 2024-09-16T00:09:30.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.099798+0000", 2024-09-16T00:09:30.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7354000000000002e-05 2024-09-16T00:09:30.244 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.244 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.244 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.245 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:219 2.1f 2:f83d65d8:::200.00000004:head [write 863658~2915703 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e65)", 2024-09-16T00:09:30.245 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:30.245 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.270382770000001, 2024-09-16T00:09:30.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013772627000000001, 2024-09-16T00:09:30.245 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.245 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.245 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.246 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:30.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:30.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870941+0000", 2024-09-16T00:09:30.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.142+0000 7f285fe65640 1 --2- 172.21.15.139:0/2172187411 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f285806eae0 0x7f2858153420 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:30.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.142+0000 7f285fe65640 1 -- 172.21.15.139:0/2172187411 >> 172.21.15.139:0/2172187411 conn(0x7f2858072b30 msgr2=0x7f28580cafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:30.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4e-07 2024-09-16T00:09:30.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877134+0000", 2024-09-16T00:09:30.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061936769999999999 2024-09-16T00:09:30.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877153+0000", 2024-09-16T00:09:30.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8408000000000001e-05 2024-09-16T00:09:30.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877161+0000", 2024-09-16T00:09:30.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2709999999999992e-06 2024-09-16T00:09:30.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877225+0000", 2024-09-16T00:09:30.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4190999999999996e-05 2024-09-16T00:09:30.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877289+0000", 2024-09-16T00:09:30.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4058000000000002e-05 2024-09-16T00:09:30.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:30.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877376+0000", 2024-09-16T00:09:30.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7227999999999999e-05 2024-09-16T00:09:30.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.882925+0000", 2024-09-16T00:09:30.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055481050000000002 2024-09-16T00:09:30.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884635+0000", 2024-09-16T00:09:30.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017102759999999999 2024-09-16T00:09:30.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884674+0000", 2024-09-16T00:09:30.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8914e-05 2024-09-16T00:09:30.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884712+0000", 2024-09-16T00:09:30.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658999999999997e-05 2024-09-16T00:09:30.252 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.252 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.252 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.252 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5787 3.3d 3:bc1a97d0:::10000001a99.00000000:head [write 0~2770] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:30.252 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.087043+0000", 2024-09-16T00:09:30.253 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 15.054279671, 2024-09-16T00:09:30.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081081320000000005, 2024-09-16T00:09:30.253 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.253 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.253 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.253 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:30.253 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:30.253 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:30.253 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5787 2024-09-16T00:09:30.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.253 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087043+0000", 2024-09-16T00:09:30.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087043+0000", 2024-09-16T00:09:30.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087045+0000", 2024-09-16T00:09:30.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:09:30.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087053+0000", 2024-09-16T00:09:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7669999999999992e-06 2024-09-16T00:09:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087069+0000", 2024-09-16T00:09:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5155e-05 2024-09-16T00:09:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087081+0000", 2024-09-16T00:09:30.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2485e-05 2024-09-16T00:09:30.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087144+0000", 2024-09-16T00:09:30.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3255000000000003e-05 2024-09-16T00:09:30.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op must wait for map", 2024-09-16T00:09:30.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087152+0000", 2024-09-16T00:09:30.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5850000000000002e-06 2024-09-16T00:09:30.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.088816+0000", 2024-09-16T00:09:30.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016638480000000001 2024-09-16T00:09:30.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.088981+0000", 2024-09-16T00:09:30.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016522600000000001 2024-09-16T00:09:30.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:30.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.089128+0000", 2024-09-16T00:09:30.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000147441 2024-09-16T00:09:30.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.092831+0000", 2024-09-16T00:09:30.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037027319999999998 2024-09-16T00:09:30.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.095037+0000", 2024-09-16T00:09:30.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022059979999999998 2024-09-16T00:09:30.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.095100+0000", 2024-09-16T00:09:30.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2401000000000001e-05 2024-09-16T00:09:30.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.095151+0000", 2024-09-16T00:09:30.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1443000000000002e-05 2024-09-16T00:09:30.259 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.259 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.259 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.259 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:276 2.1e e69/19 2:7bf7e7b5:::200.00000006:head v 69'3, mlcod=22'1)", 2024-09-16T00:09:30.259 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:16.493303+0000", 2024-09-16T00:09:30.260 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 13.648019181, 2024-09-16T00:09:30.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0062641140000000003, 2024-09-16T00:09:30.260 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.260 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.260 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.260 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493303+0000", 2024-09-16T00:09:30.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493303+0000", 2024-09-16T00:09:30.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493305+0000", 2024-09-16T00:09:30.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.573e-06 2024-09-16T00:09:30.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.495058+0000", 2024-09-16T00:09:30.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001753183 2024-09-16T00:09:30.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.495080+0000", 2024-09-16T00:09:30.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1676999999999999e-05 2024-09-16T00:09:30.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.495092+0000", 2024-09-16T00:09:30.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2279000000000001e-05 2024-09-16T00:09:30.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.495159+0000", 2024-09-16T00:09:30.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6872000000000006e-05 2024-09-16T00:09:30.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.495178+0000", 2024-09-16T00:09:30.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8963000000000001e-05 2024-09-16T00:09:30.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.499520+0000", 2024-09-16T00:09:30.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043416490000000004 2024-09-16T00:09:30.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.499567+0000", 2024-09-16T00:09:30.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7917999999999998e-05 2024-09-16T00:09:30.264 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.264 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.264 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.264 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:9173 3.10 e76/46 3:084cf2ca:::10000002a15.00000000:head v 76'149, mlcod=76'147)", 2024-09-16T00:09:30.264 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:29.578156+0000", 2024-09-16T00:09:30.264 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.56316690999999997, 2024-09-16T00:09:30.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027572899999999999, 2024-09-16T00:09:30.264 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.264 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.264 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.265 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.578156+0000", 2024-09-16T00:09:30.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.578156+0000", 2024-09-16T00:09:30.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.578156+0000", 2024-09-16T00:09:30.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.16e-07 2024-09-16T00:09:30.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.578164+0000", 2024-09-16T00:09:30.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1790000000000002e-06 2024-09-16T00:09:30.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.578173+0000", 2024-09-16T00:09:30.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7459999999999995e-06 2024-09-16T00:09:30.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.578178+0000", 2024-09-16T00:09:30.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6630000000000001e-06 2024-09-16T00:09:30.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.578225+0000", 2024-09-16T00:09:30.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7426000000000003e-05 2024-09-16T00:09:30.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.578244+0000", 2024-09-16T00:09:30.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8998999999999999e-05 2024-09-16T00:09:30.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.580875+0000", 2024-09-16T00:09:30.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026307230000000002 2024-09-16T00:09:30.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.580913+0000", 2024-09-16T00:09:30.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7737999999999997e-05 2024-09-16T00:09:30.268 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.268 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.268 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.269 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.269 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.269 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:30.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.142+0000 7f285fe65640 1 -- 172.21.15.139:0/2172187411 shutdown_connections 2024-09-16T00:09:30.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.142+0000 7f285fe65640 1 -- 172.21.15.139:0/2172187411 wait complete. 2024-09-16T00:09:30.270 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:09:30.276 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.237+0000 7f5263318640 1 Processor -- start 2024-09-16T00:09:30.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.237+0000 7f5263318640 1 -- start start 2024-09-16T00:09:30.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.238+0000 7f5263318640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f525c151be0 0x7f525c0c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:30.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.238+0000 7f5263318640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525c0c63c0 0x7f525c15b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:30.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.238+0000 7f5263318640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f525c15ad00 con 0x7f525c151800 2024-09-16T00:09:30.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.238+0000 7f5263318640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f525c0c5010 con 0x7f525c151be0 2024-09-16T00:09:30.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.238+0000 7f5263318640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f525c0c9c40 con 0x7f525c0c63c0 2024-09-16T00:09:30.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.238+0000 7f52608b3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f525c151be0 0x7f525c0c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:30.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.238+0000 7f52608b3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f525c151be0 0x7f525c0c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58794/0 (socket says 172.21.15.139:58794) 2024-09-16T00:09:30.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.238+0000 7f52608b3640 1 -- 172.21.15.139:0/2804368283 learned_addr learned my addr 172.21.15.139:0/2804368283 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:30.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.238+0000 7f52618b5640 1 --2- 172.21.15.139:0/2804368283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525c0c63c0 0x7f525c15b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:30.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.239+0000 7f52608b3640 1 -- 172.21.15.139:0/2804368283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525c0c63c0 msgr2=0x7f525c15b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:30.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.239+0000 7f52608b3640 1 --2- 172.21.15.139:0/2804368283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525c0c63c0 0x7f525c15b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:30.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.239+0000 7f52608b3640 1 -- 172.21.15.139:0/2804368283 >> v1:172.21.15.164:6789/0 conn(0x7f525c151800 legacy=0x7f525c0c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:30.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.239+0000 7f52608b3640 1 -- 172.21.15.139:0/2804368283 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525c0c5650 con 0x7f525c151be0 2024-09-16T00:09:30.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.239+0000 7f52608b3640 1 --2- 172.21.15.139:0/2804368283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f525c151be0 0x7f525c0c6c00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f524c00bb10 tx=0x7f524c02c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8776adcee4b079 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:30.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.239+0000 7f5253fff640 1 -- 172.21.15.139:0/2804368283 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c036020 con 0x7f525c151be0 2024-09-16T00:09:30.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.240+0000 7f5253fff640 1 -- 172.21.15.139:0/2804368283 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f524c032820 con 0x7f525c151be0 2024-09-16T00:09:30.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.240+0000 7f5253fff640 1 -- 172.21.15.139:0/2804368283 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c032bf0 con 0x7f525c151be0 2024-09-16T00:09:30.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.240+0000 7f5263318640 1 -- 172.21.15.139:0/2804368283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f525c151be0 msgr2=0x7f525c0c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:30.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.240+0000 7f5263318640 1 --2- 172.21.15.139:0/2804368283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f525c151be0 0x7f525c0c6c00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f524c00bb10 tx=0x7f524c02c920 comp rx=0 tx=0).stop 2024-09-16T00:09:30.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.240+0000 7f5263318640 1 -- 172.21.15.139:0/2804368283 shutdown_connections 2024-09-16T00:09:30.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.240+0000 7f5263318640 1 --2- 172.21.15.139:0/2804368283 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525c0c63c0 0x7f525c15b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:30.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.240+0000 7f5263318640 1 --2- 172.21.15.139:0/2804368283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f525c151be0 0x7f525c0c6c00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:30.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.240+0000 7f5263318640 1 -- 172.21.15.139:0/2804368283 >> 172.21.15.139:0/2804368283 conn(0x7f525c06da40 msgr2=0x7f525c06f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:30.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.240+0000 7f5263318640 1 -- 172.21.15.139:0/2804368283 shutdown_connections 2024-09-16T00:09:30.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.240+0000 7f5263318640 1 -- 172.21.15.139:0/2804368283 wait complete. 2024-09-16T00:09:30.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.242+0000 7f5263318640 1 Processor -- start 2024-09-16T00:09:30.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.242+0000 7f5263318640 1 -- start start 2024-09-16T00:09:30.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.242+0000 7f5263318640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f525c0c63c0 0x7f525c0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:30.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.242+0000 7f5263318640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f525c151800 0x7f525c1ed650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:30.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f5263318640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525c151be0 0x7f525c14dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:30.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f5263318640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f525c0c5c10 con 0x7f525c0c63c0 2024-09-16T00:09:30.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f5263318640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f525c15afa0 con 0x7f525c151800 2024-09-16T00:09:30.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f5263318640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f525c0c52d0 con 0x7f525c151be0 2024-09-16T00:09:30.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f52610b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f525c0c63c0 0x7f525c0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:30.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f52610b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f525c0c63c0 0x7f525c0cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58798/0 (socket says 172.21.15.139:58798) 2024-09-16T00:09:30.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f52610b4640 1 -- 172.21.15.139:0/1493934142 learned_addr learned my addr 172.21.15.139:0/1493934142 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:30.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f52608b3640 1 --2- 172.21.15.139:0/1493934142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f525c151800 0x7f525c1ed650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:30.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f52618b5640 1 --2- 172.21.15.139:0/1493934142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525c151be0 0x7f525c14dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:30.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f52610b4640 1 -- 172.21.15.139:0/1493934142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525c151be0 msgr2=0x7f525c14dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:30.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f52610b4640 1 --2- 172.21.15.139:0/1493934142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525c151be0 0x7f525c14dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:30.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f52610b4640 1 -- 172.21.15.139:0/1493934142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f525c151800 msgr2=0x7f525c1ed650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:30.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f52610b4640 1 --2- 172.21.15.139:0/1493934142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f525c151800 0x7f525c1ed650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:30.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f52610b4640 1 -- 172.21.15.139:0/1493934142 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525c1e2050 con 0x7f525c0c63c0 2024-09-16T00:09:30.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f52610b4640 1 --2- 172.21.15.139:0/1493934142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f525c0c63c0 0x7f525c0cc410 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f525800c430 tx=0x7f525800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:30.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f5251ffb640 1 -- 172.21.15.139:0/1493934142 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258017070 con 0x7f525c0c63c0 2024-09-16T00:09:30.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f5251ffb640 1 -- 172.21.15.139:0/1493934142 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52580052d0 con 0x7f525c0c63c0 2024-09-16T00:09:30.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f5251ffb640 1 -- 172.21.15.139:0/1493934142 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258003850 con 0x7f525c0c63c0 2024-09-16T00:09:30.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.243+0000 7f5263318640 1 -- 172.21.15.139:0/1493934142 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f525c1e2330 con 0x7f525c0c63c0 2024-09-16T00:09:30.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.244+0000 7f5263318640 1 -- 172.21.15.139:0/1493934142 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f525c1e28f0 con 0x7f525c0c63c0 2024-09-16T00:09:30.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.245+0000 7f5263318640 1 -- 172.21.15.139:0/1493934142 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f525c15b7a0 con 0x7f525c0c63c0 2024-09-16T00:09:30.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.245+0000 7f5251ffb640 1 -- 172.21.15.139:0/1493934142 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f525801a070 con 0x7f525c0c63c0 2024-09-16T00:09:30.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.246+0000 7f5251ffb640 1 --2- 172.21.15.139:0/1493934142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f524006c2e0 0x7f524006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:30.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.246+0000 7f5251ffb640 1 -- 172.21.15.139:0/1493934142 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5944+0+0 (secure 0 0 0) 0x7f525808a810 con 0x7f525c0c63c0 2024-09-16T00:09:30.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.246+0000 7f52608b3640 1 --2- 172.21.15.139:0/1493934142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f524006c2e0 0x7f524006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:30.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.246+0000 7f52608b3640 1 --2- 172.21.15.139:0/1493934142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f524006c2e0 0x7f524006e7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f524c037000 tx=0x7f524c03d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:30.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.251+0000 7f5251ffb640 1 -- 172.21.15.139:0/1493934142 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5258058840 con 0x7f525c0c63c0 2024-09-16T00:09:30.282 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:30.271+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:30.373 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:30.372+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:30.427 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:30.427 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:30.427 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:30.428 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:30.436 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:09:30.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:30.454+0000 7f5263318640 1 -- 172.21.15.139:0/1493934142 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pg-upmap", "pgid": "3.3a", "id": [5, 6]} v 0) -- 0x7f525c0c96f0 con 0x7f525c0c63c0 2024-09-16T00:09:30.474 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:30.473+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:30.547 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:30.547 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:30.548 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:30.548 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:30.548 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:30.557 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:09:30.598 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:30.596+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:30.677 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:30.675+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:30.720 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:30.720 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:30.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:30.720 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:30.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.720 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:14 2.15 2:a93a17c2:::604.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:30.720 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:09:30.720 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 112.102573996, 2024-09-16T00:09:30.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037514929999999998, 2024-09-16T00:09:30.720 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.721 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.721 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.721 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:09:30.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615360+0000", 2024-09-16T00:09:30.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615364+0000", 2024-09-16T00:09:30.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8430000000000003e-06 2024-09-16T00:09:30.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615371+0000", 2024-09-16T00:09:30.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.148e-06 2024-09-16T00:09:30.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615380+0000", 2024-09-16T00:09:30.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5889999999999992e-06 2024-09-16T00:09:30.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615389+0000", 2024-09-16T00:09:30.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1620000000000008e-06 2024-09-16T00:09:30.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615525+0000", 2024-09-16T00:09:30.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000136083 2024-09-16T00:09:30.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615662+0000", 2024-09-16T00:09:30.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013696100000000001 2024-09-16T00:09:30.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:30.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615799+0000", 2024-09-16T00:09:30.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013781399999999999 2024-09-16T00:09:30.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:09:30.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002363502 2024-09-16T00:09:30.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619018+0000", 2024-09-16T00:09:30.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085522199999999999 2024-09-16T00:09:30.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619068+0000", 2024-09-16T00:09:30.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9545000000000001e-05 2024-09-16T00:09:30.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619111+0000", 2024-09-16T00:09:30.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3624000000000001e-05 2024-09-16T00:09:30.726 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.727 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.727 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.727 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:30.727 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:09:30.727 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 112.102521925, 2024-09-16T00:09:30.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043656110000000001, 2024-09-16T00:09:30.727 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.727 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.728 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.728 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:09:30.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:09:30.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:09:30.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2199999999999999e-07 2024-09-16T00:09:30.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615416+0000", 2024-09-16T00:09:30.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5590000000000001e-06 2024-09-16T00:09:30.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.732 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615423+0000", 2024-09-16T00:09:30.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6250000000000001e-06 2024-09-16T00:09:30.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615426+0000", 2024-09-16T00:09:30.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2540000000000001e-06 2024-09-16T00:09:30.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616075+0000", 2024-09-16T00:09:30.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064968800000000004 2024-09-16T00:09:30.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616142+0000", 2024-09-16T00:09:30.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6203000000000003e-05 2024-09-16T00:09:30.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:30.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616247+0000", 2024-09-16T00:09:30.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010572499999999999 2024-09-16T00:09:30.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618913+0000", 2024-09-16T00:09:30.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026654339999999999 2024-09-16T00:09:30.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619712+0000", 2024-09-16T00:09:30.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079885199999999996 2024-09-16T00:09:30.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619744+0000", 2024-09-16T00:09:30.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2790000000000003e-05 2024-09-16T00:09:30.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619778+0000", 2024-09-16T00:09:30.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3059000000000003e-05 2024-09-16T00:09:30.738 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.738 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.738 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.738 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:30.738 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:09:30.738 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 112.10248584599999, 2024-09-16T00:09:30.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087673219999999993, 2024-09-16T00:09:30.738 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.738 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.739 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.739 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:09:30.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:09:30.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:09:30.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:09:30.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:09:30.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5392000000000001e-05 2024-09-16T00:09:30.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615470+0000", 2024-09-16T00:09:30.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1999999999999999e-06 2024-09-16T00:09:30.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615474+0000", 2024-09-16T00:09:30.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7359999999999999e-06 2024-09-16T00:09:30.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617614+0000", 2024-09-16T00:09:30.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021398189999999998 2024-09-16T00:09:30.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617724+0000", 2024-09-16T00:09:30.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110808 2024-09-16T00:09:30.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:30.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617856+0000", 2024-09-16T00:09:30.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131676 2024-09-16T00:09:30.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:09:30.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029629280000000001 2024-09-16T00:09:30.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624158+0000", 2024-09-16T00:09:30.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033386549999999998 2024-09-16T00:09:30.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624189+0000", 2024-09-16T00:09:30.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1508999999999999e-05 2024-09-16T00:09:30.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624215+0000", 2024-09-16T00:09:30.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6228999999999999e-05 2024-09-16T00:09:30.744 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.744 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.744 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.744 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:30.744 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:09:30.744 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 112.102451831, 2024-09-16T00:09:30.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053285279999999999, 2024-09-16T00:09:30.744 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.745 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.745 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.745 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:09:30.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:09:30.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:09:30.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5600000000000002e-07 2024-09-16T00:09:30.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615484+0000", 2024-09-16T00:09:30.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.688e-06 2024-09-16T00:09:30.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615488+0000", 2024-09-16T00:09:30.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.658e-06 2024-09-16T00:09:30.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615490+0000", 2024-09-16T00:09:30.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.728e-06 2024-09-16T00:09:30.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616441+0000", 2024-09-16T00:09:30.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00095065500000000001 2024-09-16T00:09:30.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616539+0000", 2024-09-16T00:09:30.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8335e-05 2024-09-16T00:09:30.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:30.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616631+0000", 2024-09-16T00:09:30.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1260000000000004e-05 2024-09-16T00:09:30.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619837+0000", 2024-09-16T00:09:30.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003206899 2024-09-16T00:09:30.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620725+0000", 2024-09-16T00:09:30.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088713599999999996 2024-09-16T00:09:30.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620772+0000", 2024-09-16T00:09:30.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7639999999999998e-05 2024-09-16T00:09:30.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620811+0000", 2024-09-16T00:09:30.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8272999999999998e-05 2024-09-16T00:09:30.750 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.750 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.750 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.750 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:30.750 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:09:30.750 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 112.10042527100001, 2024-09-16T00:09:30.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032610299999999998, 2024-09-16T00:09:30.750 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.750 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.751 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.751 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:09:30.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617509+0000", 2024-09-16T00:09:30.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617556+0000", 2024-09-16T00:09:30.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7431e-05 2024-09-16T00:09:30.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617847+0000", 2024-09-16T00:09:30.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029133400000000003 2024-09-16T00:09:30.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617858+0000", 2024-09-16T00:09:30.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0222e-05 2024-09-16T00:09:30.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617864+0000", 2024-09-16T00:09:30.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7619999999999998e-06 2024-09-16T00:09:30.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617909+0000", 2024-09-16T00:09:30.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4747999999999999e-05 2024-09-16T00:09:30.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617927+0000", 2024-09-16T00:09:30.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7785999999999999e-05 2024-09-16T00:09:30.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620734+0000", 2024-09-16T00:09:30.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002807305 2024-09-16T00:09:30.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620770+0000", 2024-09-16T00:09:30.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5441999999999997e-05 2024-09-16T00:09:30.754 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.754 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.755 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.755 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-16T00:09:30.755 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:09:30.755 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 112.099920395, 2024-09-16T00:09:30.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032703020000000001, 2024-09-16T00:09:30.755 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.755 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.755 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.755 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:09:30.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618013+0000", 2024-09-16T00:09:30.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618014+0000", 2024-09-16T00:09:30.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7799999999999997e-07 2024-09-16T00:09:30.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618017+0000", 2024-09-16T00:09:30.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4309999999999998e-06 2024-09-16T00:09:30.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618023+0000", 2024-09-16T00:09:30.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8490000000000002e-06 2024-09-16T00:09:30.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618028+0000", 2024-09-16T00:09:30.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3489999999999999e-06 2024-09-16T00:09:30.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618586+0000", 2024-09-16T00:09:30.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055724099999999999 2024-09-16T00:09:30.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618603+0000", 2024-09-16T00:09:30.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7486000000000002e-05 2024-09-16T00:09:30.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621235+0000", 2024-09-16T00:09:30.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002632087 2024-09-16T00:09:30.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621284+0000", 2024-09-16T00:09:30.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8581000000000001e-05 2024-09-16T00:09:30.759 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.759 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.759 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.760 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:39 2.6 2:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:09:30.760 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:09:30.760 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 107.07765598899999, 2024-09-16T00:09:30.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050072119999999996, 2024-09-16T00:09:30.760 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.760 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.760 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.760 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:09:30.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:09:30.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640280+0000", 2024-09-16T00:09:30.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5380000000000001e-06 2024-09-16T00:09:30.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640296+0000", 2024-09-16T00:09:30.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5950000000000001e-05 2024-09-16T00:09:30.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640327+0000", 2024-09-16T00:09:30.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0309999999999999e-05 2024-09-16T00:09:30.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640340+0000", 2024-09-16T00:09:30.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3773e-05 2024-09-16T00:09:30.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.763 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.763 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640422+0000", 2024-09-16T00:09:30.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1122e-05 2024-09-16T00:09:30.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.763 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.763 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640572+0000", 2024-09-16T00:09:30.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015044099999999999 2024-09-16T00:09:30.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.764 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:30.764 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640691+0000", 2024-09-16T00:09:30.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000119393 2024-09-16T00:09:30.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.764 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.764 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.764 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645006+0000", 2024-09-16T00:09:30.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043143779999999998 2024-09-16T00:09:30.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.764 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.764 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.764 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645169+0000", 2024-09-16T00:09:30.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016323299999999999 2024-09-16T00:09:30.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645227+0000", 2024-09-16T00:09:30.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8e-05 2024-09-16T00:09:30.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645285+0000", 2024-09-16T00:09:30.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8074e-05 2024-09-16T00:09:30.765 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.765 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.766 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.766 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:54 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e58)", 2024-09-16T00:09:30.766 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.652050+0000", 2024-09-16T00:09:30.766 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.065884146000002, 2024-09-16T00:09:30.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039842300000000001, 2024-09-16T00:09:30.766 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.766 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.766 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.766 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652050+0000", 2024-09-16T00:09:30.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652050+0000", 2024-09-16T00:09:30.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652052+0000", 2024-09-16T00:09:30.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.857e-06 2024-09-16T00:09:30.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652101+0000", 2024-09-16T00:09:30.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9812999999999999e-05 2024-09-16T00:09:30.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652122+0000", 2024-09-16T00:09:30.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0659e-05 2024-09-16T00:09:30.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652133+0000", 2024-09-16T00:09:30.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0977e-05 2024-09-16T00:09:30.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652198+0000", 2024-09-16T00:09:30.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5229999999999997e-05 2024-09-16T00:09:30.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.652293+0000", 2024-09-16T00:09:30.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4596e-05 2024-09-16T00:09:30.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:30.770 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.654316+0000", 2024-09-16T00:09:30.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020229699999999998 2024-09-16T00:09:30.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.770 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.770 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.655783+0000", 2024-09-16T00:09:30.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014669240000000001 2024-09-16T00:09:30.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.770 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.770 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.655974+0000", 2024-09-16T00:09:30.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000191437 2024-09-16T00:09:30.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.656008+0000", 2024-09-16T00:09:30.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3603000000000003e-05 2024-09-16T00:09:30.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.656034+0000", 2024-09-16T00:09:30.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6163999999999999e-05 2024-09-16T00:09:30.771 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.771 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.771 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.771 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:56 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e59)", 2024-09-16T00:09:30.772 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.652483+0000", 2024-09-16T00:09:30.772 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.065450951000003, 2024-09-16T00:09:30.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003414191, 2024-09-16T00:09:30.772 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.772 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.772 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.772 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652483+0000", 2024-09-16T00:09:30.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652483+0000", 2024-09-16T00:09:30.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652485+0000", 2024-09-16T00:09:30.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4059999999999999e-06 2024-09-16T00:09:30.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652527+0000", 2024-09-16T00:09:30.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1319000000000002e-05 2024-09-16T00:09:30.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652548+0000", 2024-09-16T00:09:30.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1149999999999999e-05 2024-09-16T00:09:30.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652562+0000", 2024-09-16T00:09:30.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3971000000000001e-05 2024-09-16T00:09:30.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652631+0000", 2024-09-16T00:09:30.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9411000000000003e-05 2024-09-16T00:09:30.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.775 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.775 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.652717+0000", 2024-09-16T00:09:30.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5556999999999994e-05 2024-09-16T00:09:30.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.775 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:30.775 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.654006+0000", 2024-09-16T00:09:30.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012897200000000001 2024-09-16T00:09:30.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.775 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.775 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.655280+0000", 2024-09-16T00:09:30.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012732609999999999 2024-09-16T00:09:30.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.776 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.776 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.655811+0000", 2024-09-16T00:09:30.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00053082400000000005 2024-09-16T00:09:30.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.776 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.776 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.655862+0000", 2024-09-16T00:09:30.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1672000000000003e-05 2024-09-16T00:09:30.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.655897+0000", 2024-09-16T00:09:30.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4900000000000001e-05 2024-09-16T00:09:30.777 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.777 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.777 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.777 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:85 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:30.777 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:02.658347+0000", 2024-09-16T00:09:30.777 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 28.059586825, 2024-09-16T00:09:30.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047582939999999997, 2024-09-16T00:09:30.777 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.778 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.778 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.778 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658347+0000", 2024-09-16T00:09:30.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658347+0000", 2024-09-16T00:09:30.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658349+0000", 2024-09-16T00:09:30.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7999999999999999e-06 2024-09-16T00:09:30.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658439+0000", 2024-09-16T00:09:30.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0276000000000001e-05 2024-09-16T00:09:30.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658454+0000", 2024-09-16T00:09:30.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5105e-05 2024-09-16T00:09:30.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658462+0000", 2024-09-16T00:09:30.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.525e-06 2024-09-16T00:09:30.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658512+0000", 2024-09-16T00:09:30.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9877999999999999e-05 2024-09-16T00:09:30.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658569+0000", 2024-09-16T00:09:30.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7022999999999999e-05 2024-09-16T00:09:30.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:30.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.660742+0000", 2024-09-16T00:09:30.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021734419999999998 2024-09-16T00:09:30.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.662841+0000", 2024-09-16T00:09:30.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020994120000000002 2024-09-16T00:09:30.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.782 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.782 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.782 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.663045+0000", 2024-09-16T00:09:30.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000203422 2024-09-16T00:09:30.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.782 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.782 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.782 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.663079+0000", 2024-09-16T00:09:30.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3769999999999997e-05 2024-09-16T00:09:30.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.782 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.782 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.663105+0000", 2024-09-16T00:09:30.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6641000000000001e-05 2024-09-16T00:09:30.783 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.783 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.783 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.783 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:160 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e64)", 2024-09-16T00:09:30.783 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:06.662147+0000", 2024-09-16T00:09:30.783 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 24.055786431000001, 2024-09-16T00:09:30.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034276770000000001, 2024-09-16T00:09:30.783 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.784 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.784 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.784 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.662147+0000", 2024-09-16T00:09:30.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.662147+0000", 2024-09-16T00:09:30.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.662150+0000", 2024-09-16T00:09:30.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2400000000000002e-06 2024-09-16T00:09:30.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.662188+0000", 2024-09-16T00:09:30.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8176000000000002e-05 2024-09-16T00:09:30.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.662211+0000", 2024-09-16T00:09:30.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3603000000000001e-05 2024-09-16T00:09:30.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.786 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.786 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.662242+0000", 2024-09-16T00:09:30.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0446e-05 2024-09-16T00:09:30.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.786 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.786 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.662313+0000", 2024-09-16T00:09:30.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0730999999999998e-05 2024-09-16T00:09:30.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.662413+0000", 2024-09-16T00:09:30.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000100204 2024-09-16T00:09:30.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:30.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.663804+0000", 2024-09-16T00:09:30.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013914890000000001 2024-09-16T00:09:30.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.665348+0000", 2024-09-16T00:09:30.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001543574 2024-09-16T00:09:30.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.788 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.788 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.788 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.665465+0000", 2024-09-16T00:09:30.788 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117224 2024-09-16T00:09:30.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.788 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.788 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.788 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.665521+0000", 2024-09-16T00:09:30.788 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5634999999999998e-05 2024-09-16T00:09:30.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.789 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.789 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.665575+0000", 2024-09-16T00:09:30.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4354999999999997e-05 2024-09-16T00:09:30.789 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.789 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.789 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.789 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:162 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e64)", 2024-09-16T00:09:30.789 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:07.662897+0000", 2024-09-16T00:09:30.789 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.055036519000002, 2024-09-16T00:09:30.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003663272, 2024-09-16T00:09:30.790 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.790 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.790 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.790 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:07.662897+0000", 2024-09-16T00:09:30.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:07.662897+0000", 2024-09-16T00:09:30.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:07.662900+0000", 2024-09-16T00:09:30.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3250000000000002e-06 2024-09-16T00:09:30.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:07.662956+0000", 2024-09-16T00:09:30.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6518000000000003e-05 2024-09-16T00:09:30.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:07.662975+0000", 2024-09-16T00:09:30.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9066e-05 2024-09-16T00:09:30.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:07.662987+0000", 2024-09-16T00:09:30.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1821000000000001e-05 2024-09-16T00:09:30.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:07.663058+0000", 2024-09-16T00:09:30.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1276e-05 2024-09-16T00:09:30.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:07.663126+0000", 2024-09-16T00:09:30.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7915000000000003e-05 2024-09-16T00:09:30.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:30.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:07.664122+0000", 2024-09-16T00:09:30.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00099568599999999997 2024-09-16T00:09:30.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:07.665654+0000", 2024-09-16T00:09:30.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015315820000000001 2024-09-16T00:09:30.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:07.666504+0000", 2024-09-16T00:09:30.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00085025499999999996 2024-09-16T00:09:30.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:07.666534+0000", 2024-09-16T00:09:30.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0148e-05 2024-09-16T00:09:30.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:07.666561+0000", 2024-09-16T00:09:30.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6679999999999999e-05 2024-09-16T00:09:30.794 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.795 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.795 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.795 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.795 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.795 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:223 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e65)", 2024-09-16T00:09:30.795 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:11.668302+0000", 2024-09-16T00:09:30.795 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.049632291000002, 2024-09-16T00:09:30.795 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052512100000000001, 2024-09-16T00:09:30.795 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.795 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.795 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.795 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.795 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.795 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.668302+0000", 2024-09-16T00:09:30.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.668302+0000", 2024-09-16T00:09:30.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.668303+0000", 2024-09-16T00:09:30.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9600000000000008e-07 2024-09-16T00:09:30.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.668418+0000", 2024-09-16T00:09:30.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011578099999999999 2024-09-16T00:09:30.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.668430+0000", 2024-09-16T00:09:30.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1973e-05 2024-09-16T00:09:30.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.668436+0000", 2024-09-16T00:09:30.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.682e-06 2024-09-16T00:09:30.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.668478+0000", 2024-09-16T00:09:30.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2233e-05 2024-09-16T00:09:30.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.668530+0000", 2024-09-16T00:09:30.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2054e-05 2024-09-16T00:09:30.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:30.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.670743+0000", 2024-09-16T00:09:30.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022122579999999999 2024-09-16T00:09:30.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.673248+0000", 2024-09-16T00:09:30.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025053580000000001 2024-09-16T00:09:30.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.673502+0000", 2024-09-16T00:09:30.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00025403699999999998 2024-09-16T00:09:30.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.673532+0000", 2024-09-16T00:09:30.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9878000000000001e-05 2024-09-16T00:09:30.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.800 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.673553+0000", 2024-09-16T00:09:30.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0959999999999999e-05 2024-09-16T00:09:30.800 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.800 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.800 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.800 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:274 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e68)", 2024-09-16T00:09:30.800 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.672085+0000", 2024-09-16T00:09:30.801 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 15.04584906, 2024-09-16T00:09:30.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053189819999999999, 2024-09-16T00:09:30.801 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.801 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.801 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.801 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.801 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.801 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.801 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.672085+0000", 2024-09-16T00:09:30.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.801 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.801 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.801 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.801 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.672085+0000", 2024-09-16T00:09:30.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.672086+0000", 2024-09-16T00:09:30.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1179999999999999e-06 2024-09-16T00:09:30.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.672158+0000", 2024-09-16T00:09:30.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2261000000000005e-05 2024-09-16T00:09:30.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.672171+0000", 2024-09-16T00:09:30.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2510999999999999e-05 2024-09-16T00:09:30.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.672176+0000", 2024-09-16T00:09:30.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5729999999999998e-06 2024-09-16T00:09:30.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.672236+0000", 2024-09-16T00:09:30.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9484999999999997e-05 2024-09-16T00:09:30.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.672320+0000", 2024-09-16T00:09:30.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3908999999999999e-05 2024-09-16T00:09:30.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:30.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.674880+0000", 2024-09-16T00:09:30.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025598970000000002 2024-09-16T00:09:30.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.677164+0000", 2024-09-16T00:09:30.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022845750000000001 2024-09-16T00:09:30.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.677334+0000", 2024-09-16T00:09:30.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017007400000000001 2024-09-16T00:09:30.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.677364+0000", 2024-09-16T00:09:30.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9526000000000002e-05 2024-09-16T00:09:30.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.677404+0000", 2024-09-16T00:09:30.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0052999999999998e-05 2024-09-16T00:09:30.806 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.806 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.806 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.806 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:6137 3.0 3:03c3687d:::10000001c23.00000000:head [write 0~9470] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-16T00:09:30.806 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:16.258393+0000", 2024-09-16T00:09:30.806 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.459541244, 2024-09-16T00:09:30.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041063480000000001, 2024-09-16T00:09:30.806 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.806 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.807 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.807 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:30.807 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:30.807 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:30.807 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 6137 2024-09-16T00:09:30.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.807 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.807 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.258393+0000", 2024-09-16T00:09:30.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.258393+0000", 2024-09-16T00:09:30.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.258395+0000", 2024-09-16T00:09:30.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2079999999999999e-06 2024-09-16T00:09:30.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.258412+0000", 2024-09-16T00:09:30.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7348999999999999e-05 2024-09-16T00:09:30.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.258432+0000", 2024-09-16T00:09:30.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9706000000000001e-05 2024-09-16T00:09:30.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.258441+0000", 2024-09-16T00:09:30.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5219999999999996e-06 2024-09-16T00:09:30.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.258507+0000", 2024-09-16T00:09:30.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5739000000000002e-05 2024-09-16T00:09:30.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op must wait for map", 2024-09-16T00:09:30.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.258512+0000", 2024-09-16T00:09:30.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8099999999999997e-06 2024-09-16T00:09:30.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.260964+0000", 2024-09-16T00:09:30.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024522329999999998 2024-09-16T00:09:30.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.261081+0000", 2024-09-16T00:09:30.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011697 2024-09-16T00:09:30.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:30.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.261214+0000", 2024-09-16T00:09:30.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013290700000000001 2024-09-16T00:09:30.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.261912+0000", 2024-09-16T00:09:30.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00069814400000000004 2024-09-16T00:09:30.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.262428+0000", 2024-09-16T00:09:30.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00051624500000000003 2024-09-16T00:09:30.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.262464+0000", 2024-09-16T00:09:30.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6010000000000003e-05 2024-09-16T00:09:30.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.262499+0000", 2024-09-16T00:09:30.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4505000000000003e-05 2024-09-16T00:09:30.813 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.813 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.813 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.813 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:277 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e69)", 2024-09-16T00:09:30.813 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:16.672890+0000", 2024-09-16T00:09:30.813 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.045043604, 2024-09-16T00:09:30.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039664729999999999, 2024-09-16T00:09:30.813 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.813 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.813 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.813 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.672890+0000", 2024-09-16T00:09:30.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.672890+0000", 2024-09-16T00:09:30.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.672893+0000", 2024-09-16T00:09:30.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5299999999999999e-06 2024-09-16T00:09:30.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.672941+0000", 2024-09-16T00:09:30.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7905000000000003e-05 2024-09-16T00:09:30.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.672954+0000", 2024-09-16T00:09:30.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3687e-05 2024-09-16T00:09:30.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.672961+0000", 2024-09-16T00:09:30.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8900000000000001e-06 2024-09-16T00:09:30.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.673012+0000", 2024-09-16T00:09:30.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0260000000000003e-05 2024-09-16T00:09:30.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.673095+0000", 2024-09-16T00:09:30.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3473999999999994e-05 2024-09-16T00:09:30.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:30.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.674360+0000", 2024-09-16T00:09:30.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001265064 2024-09-16T00:09:30.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.676085+0000", 2024-09-16T00:09:30.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017246970000000001 2024-09-16T00:09:30.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.676790+0000", 2024-09-16T00:09:30.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00070506900000000001 2024-09-16T00:09:30.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.818 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.818 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.676823+0000", 2024-09-16T00:09:30.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3181e-05 2024-09-16T00:09:30.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.818 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.818 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.818 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.676857+0000", 2024-09-16T00:09:30.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3716e-05 2024-09-16T00:09:30.818 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.818 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.818 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.818 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.818 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:328 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e74)", 2024-09-16T00:09:30.819 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:21.676780+0000", 2024-09-16T00:09:30.819 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.0411539360000006, 2024-09-16T00:09:30.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051768939999999996, 2024-09-16T00:09:30.819 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.819 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.819 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.819 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.819 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.819 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.676780+0000", 2024-09-16T00:09:30.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.819 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.819 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.676780+0000", 2024-09-16T00:09:30.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.676782+0000", 2024-09-16T00:09:30.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.793e-06 2024-09-16T00:09:30.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.676865+0000", 2024-09-16T00:09:30.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3602000000000004e-05 2024-09-16T00:09:30.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.676881+0000", 2024-09-16T00:09:30.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5483000000000001e-05 2024-09-16T00:09:30.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.676888+0000", 2024-09-16T00:09:30.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2080000000000003e-06 2024-09-16T00:09:30.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.676944+0000", 2024-09-16T00:09:30.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5621999999999997e-05 2024-09-16T00:09:30.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.677002+0000", 2024-09-16T00:09:30.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8449999999999998e-05 2024-09-16T00:09:30.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:30.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.678932+0000", 2024-09-16T00:09:30.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001929676 2024-09-16T00:09:30.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.681197+0000", 2024-09-16T00:09:30.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002265127 2024-09-16T00:09:30.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.681883+0000", 2024-09-16T00:09:30.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00068578900000000002 2024-09-16T00:09:30.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.681918+0000", 2024-09-16T00:09:30.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5139999999999999e-05 2024-09-16T00:09:30.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.681957+0000", 2024-09-16T00:09:30.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9004000000000001e-05 2024-09-16T00:09:30.824 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.824 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.824 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.824 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:401 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e75)", 2024-09-16T00:09:30.824 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:24.680116+0000", 2024-09-16T00:09:30.824 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.0378182819999999, 2024-09-16T00:09:30.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004726002, 2024-09-16T00:09:30.824 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.824 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.824 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.825 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.680116+0000", 2024-09-16T00:09:30.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.680116+0000", 2024-09-16T00:09:30.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.680117+0000", 2024-09-16T00:09:30.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1480000000000001e-06 2024-09-16T00:09:30.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.680181+0000", 2024-09-16T00:09:30.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4286000000000001e-05 2024-09-16T00:09:30.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.680191+0000", 2024-09-16T00:09:30.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0426e-05 2024-09-16T00:09:30.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.680197+0000", 2024-09-16T00:09:30.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2789999999999998e-06 2024-09-16T00:09:30.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.680250+0000", 2024-09-16T00:09:30.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3053999999999997e-05 2024-09-16T00:09:30.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.680334+0000", 2024-09-16T00:09:30.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3981999999999997e-05 2024-09-16T00:09:30.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:30.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.682946+0000", 2024-09-16T00:09:30.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026117950000000001 2024-09-16T00:09:30.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.684663+0000", 2024-09-16T00:09:30.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001717508 2024-09-16T00:09:30.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.684775+0000", 2024-09-16T00:09:30.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011200399999999999 2024-09-16T00:09:30.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.684809+0000", 2024-09-16T00:09:30.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4048999999999999e-05 2024-09-16T00:09:30.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.684842+0000", 2024-09-16T00:09:30.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2471000000000001e-05 2024-09-16T00:09:30.829 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.829 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.829 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.830 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:403 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e75)", 2024-09-16T00:09:30.830 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:25.681255+0000", 2024-09-16T00:09:30.830 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.0366788339999999, 2024-09-16T00:09:30.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041662790000000002, 2024-09-16T00:09:30.830 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.830 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.830 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.830 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.681255+0000", 2024-09-16T00:09:30.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.681255+0000", 2024-09-16T00:09:30.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.681256+0000", 2024-09-16T00:09:30.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.294e-06 2024-09-16T00:09:30.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.681351+0000", 2024-09-16T00:09:30.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5043000000000005e-05 2024-09-16T00:09:30.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.681364+0000", 2024-09-16T00:09:30.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2829e-05 2024-09-16T00:09:30.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.681372+0000", 2024-09-16T00:09:30.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8080000000000005e-06 2024-09-16T00:09:30.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.681412+0000", 2024-09-16T00:09:30.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0187e-05 2024-09-16T00:09:30.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.681477+0000", 2024-09-16T00:09:30.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4836000000000001e-05 2024-09-16T00:09:30.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:30.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.682659+0000", 2024-09-16T00:09:30.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011815860000000001 2024-09-16T00:09:30.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:30.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.684317+0000", 2024-09-16T00:09:30.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001658269 2024-09-16T00:09:30.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:30.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.685361+0000", 2024-09-16T00:09:30.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010441039999999999 2024-09-16T00:09:30.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.685394+0000", 2024-09-16T00:09:30.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2889000000000003e-05 2024-09-16T00:09:30.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.685421+0000", 2024-09-16T00:09:30.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7433999999999999e-05 2024-09-16T00:09:30.835 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.835 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.835 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.835 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:488 2.0 e76/19 2:02ac1934:::200.00000009:head v 76'62, mlcod=76'60)", 2024-09-16T00:09:30.835 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:29.731903+0000", 2024-09-16T00:09:30.835 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.98603043499999998, 2024-09-16T00:09:30.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041601370000000004, 2024-09-16T00:09:30.835 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:30.836 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:30.836 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:30.836 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:30.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:30.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.731903+0000", 2024-09-16T00:09:30.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:30.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.731903+0000", 2024-09-16T00:09:30.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:30.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:30.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.731904+0000", 2024-09-16T00:09:30.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0800000000000004e-07 2024-09-16T00:09:30.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:30.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.733231+0000", 2024-09-16T00:09:30.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013272780000000001 2024-09-16T00:09:30.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:30.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.733248+0000", 2024-09-16T00:09:30.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6526999999999999e-05 2024-09-16T00:09:30.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:30.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.733256+0000", 2024-09-16T00:09:30.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4579999999999993e-06 2024-09-16T00:09:30.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:30.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.733321+0000", 2024-09-16T00:09:30.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4614000000000002e-05 2024-09-16T00:09:30.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:30.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.733345+0000", 2024-09-16T00:09:30.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3808e-05 2024-09-16T00:09:30.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:30.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.735988+0000", 2024-09-16T00:09:30.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026435389999999999 2024-09-16T00:09:30.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:30.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:30.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:30.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.736064+0000", 2024-09-16T00:09:30.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5104999999999993e-05 2024-09-16T00:09:30.839 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.839 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.839 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.840 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:30.840 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:30.840 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:30.841 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:09:30.842 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:30.839+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:30.941 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:30.940+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:31.006 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:31.006 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:31.006 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:31.006 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:31.018 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:09:31.043 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:31.042+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:31.085 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:09:31.144 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:31.143+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:31.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.153+0000 7f5251ffb640 1 -- 172.21.15.139:0/1493934142 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap", "pgid": "3.3a", "id": [5, 6]}]=0 set 3.3a pg_upmap mapping to [5,6] v78) ==== 124+0+0 (secure 0 0 0) 0x7f525805cb40 con 0x7f525c0c63c0 2024-09-16T00:09:31.155 INFO:teuthology.orchestra.run.smithi139.stderr:set 3.3a pg_upmap mapping to [5,6] 2024-09-16T00:09:31.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.156+0000 7f5263318640 1 -- 172.21.15.139:0/1493934142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f524006c2e0 msgr2=0x7f524006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:31.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.156+0000 7f5263318640 1 --2- 172.21.15.139:0/1493934142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f524006c2e0 0x7f524006e7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f524c037000 tx=0x7f524c03d040 comp rx=0 tx=0).stop 2024-09-16T00:09:31.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.156+0000 7f5263318640 1 -- 172.21.15.139:0/1493934142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f525c0c63c0 msgr2=0x7f525c0cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:31.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.156+0000 7f5263318640 1 --2- 172.21.15.139:0/1493934142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f525c0c63c0 0x7f525c0cc410 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f525800c430 tx=0x7f525800c8f0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.156+0000 7f5263318640 1 -- 172.21.15.139:0/1493934142 shutdown_connections 2024-09-16T00:09:31.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.156+0000 7f5263318640 1 --2- 172.21.15.139:0/1493934142 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f524006c2e0 0x7f524006e7a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.156+0000 7f5263318640 1 --2- 172.21.15.139:0/1493934142 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525c151be0 0x7f525c14dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.156+0000 7f5263318640 1 --2- 172.21.15.139:0/1493934142 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f525c151800 0x7f525c1ed650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.156+0000 7f5263318640 1 --2- 172.21.15.139:0/1493934142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f525c0c63c0 0x7f525c0cc410 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.156+0000 7f5263318640 1 -- 172.21.15.139:0/1493934142 >> 172.21.15.139:0/1493934142 conn(0x7f525c06da40 msgr2=0x7f525c0c7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:31.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.157+0000 7f5263318640 1 -- 172.21.15.139:0/1493934142 shutdown_connections 2024-09-16T00:09:31.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.157+0000 7f5263318640 1 -- 172.21.15.139:0/1493934142 wait complete. 2024-09-16T00:09:31.184 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:31.184 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:31.184 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:31.184 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:31.184 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:31.195 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:09:31.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.221+0000 7fc1629c4640 1 Processor -- start 2024-09-16T00:09:31.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.221+0000 7fc1629c4640 1 -- start start 2024-09-16T00:09:31.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.221+0000 7fc1629c4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc15c0d0490 0x7fc15c0b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:31.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.221+0000 7fc1629c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc15c0d0870 0x7fc15c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:31.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.221+0000 7fc1629c4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc15c0b3430 0x7fc15c0b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:31.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.221+0000 7fc1629c4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc15c15b940 con 0x7fc15c0b3430 2024-09-16T00:09:31.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.222+0000 7fc1629c4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fc15c0b3bb0 con 0x7fc15c0d0490 2024-09-16T00:09:31.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.222+0000 7fc1629c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc15c0b3180 con 0x7fc15c0d0870 2024-09-16T00:09:31.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.222+0000 7fc160f61640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc15c0b3430 0x7fc15c0b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:31.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.222+0000 7fc15bfff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc15c0d0490 0x7fc15c0b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:31.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.222+0000 7fc15b7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc15c0d0870 0x7fc15c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:31.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.222+0000 7fc15bfff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc15c0d0490 0x7fc15c0b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:55326/0 (socket says 172.21.15.139:55326) 2024-09-16T00:09:31.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.222+0000 7fc15bfff640 1 -- 172.21.15.139:0/2561361454 learned_addr learned my addr 172.21.15.139:0/2561361454 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:31.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.222+0000 7fc160f61640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc15c0b3430 0x7fc15c0b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58812/0 (socket says 172.21.15.139:58812) 2024-09-16T00:09:31.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.222+0000 7fc15bfff640 1 -- 172.21.15.139:0/2561361454 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc15c0d0870 msgr2=0x7fc15c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:31.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.222+0000 7fc15bfff640 1 --2- 172.21.15.139:0/2561361454 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc15c0d0870 0x7fc15c0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.222+0000 7fc15bfff640 1 -- 172.21.15.139:0/2561361454 >> v2:172.21.15.139:3300/0 conn(0x7fc15c0b3430 msgr2=0x7fc15c0b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:31.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.222+0000 7fc15bfff640 1 --2- 172.21.15.139:0/2561361454 >> v2:172.21.15.139:3300/0 conn(0x7fc15c0b3430 0x7fc15c0b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.222+0000 7fc15bfff640 1 -- 172.21.15.139:0/2561361454 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15c15adc0 con 0x7fc15c0d0490 2024-09-16T00:09:31.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.222+0000 7fc160f61640 1 --2- 172.21.15.139:0/2561361454 >> v2:172.21.15.139:3300/0 conn(0x7fc15c0b3430 0x7fc15c0b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:31.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.222+0000 7fc15b7fe640 1 --2- 172.21.15.139:0/2561361454 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc15c0d0870 0x7fc15c0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:31.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.223+0000 7fc15bfff640 1 --2- 172.21.15.139:0/2561361454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc15c0d0490 0x7fc15c0b2c40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0cddf0 tx=0x7fc14c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8edf2ddb80fcf20b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:31.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.223+0000 7fc15affd640 1 -- 172.21.15.139:0/2561361454 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c037070 con 0x7fc15c0d0490 2024-09-16T00:09:31.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.223+0000 7fc15affd640 1 -- 172.21.15.139:0/2561361454 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc14c032440 con 0x7fc15c0d0490 2024-09-16T00:09:31.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.223+0000 7fc15affd640 1 -- 172.21.15.139:0/2561361454 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c032810 con 0x7fc15c0d0490 2024-09-16T00:09:31.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.224+0000 7fc1629c4640 1 -- 172.21.15.139:0/2561361454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc15c0d0490 msgr2=0x7fc15c0b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:31.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.224+0000 7fc1629c4640 1 --2- 172.21.15.139:0/2561361454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc15c0d0490 0x7fc15c0b2c40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0cddf0 tx=0x7fc14c02a270 comp rx=0 tx=0).stop 2024-09-16T00:09:31.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.224+0000 7fc1629c4640 1 -- 172.21.15.139:0/2561361454 shutdown_connections 2024-09-16T00:09:31.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.224+0000 7fc1629c4640 1 --2- 172.21.15.139:0/2561361454 >> v2:172.21.15.139:3300/0 conn(0x7fc15c0b3430 0x7fc15c0b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.224+0000 7fc1629c4640 1 --2- 172.21.15.139:0/2561361454 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc15c0d0870 0x7fc15c0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.224+0000 7fc1629c4640 1 --2- 172.21.15.139:0/2561361454 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc15c0d0490 0x7fc15c0b2c40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.224+0000 7fc1629c4640 1 -- 172.21.15.139:0/2561361454 >> 172.21.15.139:0/2561361454 conn(0x7fc15c06c520 msgr2=0x7fc15c06c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:31.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.224+0000 7fc1629c4640 1 -- 172.21.15.139:0/2561361454 shutdown_connections 2024-09-16T00:09:31.229 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.224+0000 7fc1629c4640 1 -- 172.21.15.139:0/2561361454 wait complete. 2024-09-16T00:09:31.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.226+0000 7fc1629c4640 1 Processor -- start 2024-09-16T00:09:31.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.226+0000 7fc1629c4640 1 -- start start 2024-09-16T00:09:31.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.227+0000 7fc1629c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc15c0b3430 0x7fc15c0cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:31.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.227+0000 7fc1629c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc15c0d0490 0x7fc15c0c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:31.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.227+0000 7fc1629c4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc15c0d0870 0x7fc15c0ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:31.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.227+0000 7fc1629c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c15b660 con 0x7fc15c0d0490 2024-09-16T00:09:31.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.227+0000 7fc1629c4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c15b380 con 0x7fc15c0d0870 2024-09-16T00:09:31.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.227+0000 7fc1629c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc15c15a980 con 0x7fc15c0b3430 2024-09-16T00:09:31.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.227+0000 7fc15bfff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc15c0b3430 0x7fc15c0cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:31.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.227+0000 7fc15bfff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc15c0b3430 0x7fc15c0cf4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:43594/0 (socket says 172.21.15.139:43594) 2024-09-16T00:09:31.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.227+0000 7fc15b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc15c0d0490 0x7fc15c0c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:31.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.227+0000 7fc15bfff640 1 -- 172.21.15.139:0/898446009 learned_addr learned my addr 172.21.15.139:0/898446009 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:31.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.227+0000 7fc160f61640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc15c0d0870 0x7fc15c0ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:31.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.228+0000 7fc15bfff640 1 -- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc15c0d0870 msgr2=0x7fc15c0ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:31.232 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.228+0000 7fc15bfff640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc15c0d0870 0x7fc15c0ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.228+0000 7fc15bfff640 1 -- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc15c0d0490 msgr2=0x7fc15c0c9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:31.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.228+0000 7fc15bfff640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc15c0d0490 0x7fc15c0c9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.228+0000 7fc15bfff640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15c15bac0 con 0x7fc15c0b3430 2024-09-16T00:09:31.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.228+0000 7fc160f61640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc15c0d0870 0x7fc15c0ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:31.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.228+0000 7fc15b7fe640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc15c0d0490 0x7fc15c0c9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:31.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.228+0000 7fc15bfff640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc15c0b3430 0x7fc15c0cf4a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc14c000c00 tx=0x7fc14c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:31.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.228+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c041020 con 0x7fc15c0b3430 2024-09-16T00:09:31.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.228+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc14c03d620 con 0x7fc15c0b3430 2024-09-16T00:09:31.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.228+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c037040 con 0x7fc15c0b3430 2024-09-16T00:09:31.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.228+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15c15bd60 con 0x7fc15c0b3430 2024-09-16T00:09:31.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.228+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc15c0c8a10 con 0x7fc15c0b3430 2024-09-16T00:09:31.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.229+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fc15c0b2c40 con 0x7fc15c0b3430 2024-09-16T00:09:31.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.230+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc14c044020 con 0x7fc15c0b3430 2024-09-16T00:09:31.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.231+0000 7fc158ff9640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc14806c2b0 0x7fc14806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:31.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.231+0000 7fc15b7fe640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc14806c2b0 0x7fc14806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:31.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.231+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(78..78 src has 1..78) ==== 5973+0+0 (secure 0 0 0) 0x7fc14c0abf60 con 0x7fc15c0b3430 2024-09-16T00:09:31.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.231+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v78) ==== 54+0+16 (secure 0 0 0) 0x7fc14c04b030 con 0x7fc15c0b3430 2024-09-16T00:09:31.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.231+0000 7fc15b7fe640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc14806c2b0 0x7fc14806e770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc15c069220 tx=0x7fc150009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:31.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.232+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fc15c0c8be0 0x7fc15c149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:31.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.232+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc15c1518a0 con 0x7fc15c0c8be0 2024-09-16T00:09:31.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.232+0000 7fc160f61640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fc15c0c8be0 0x7fc15c149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:31.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.232+0000 7fc160f61640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fc15c0c8be0 0x7fc15c149000 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:31.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.233+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc15c1518a0 con 0x7fc15c0c8be0 2024-09-16T00:09:31.245 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:31.243+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:31.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.251+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc15c15a0a0 con 0x7fc15c0c8be0 2024-09-16T00:09:31.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.253+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc15c15a0a0 con 0x7fc15c0c8be0 2024-09-16T00:09:31.255 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:31.256 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:31.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.253+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fc15c15e580 0x7fc15c149a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:31.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.253+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fc15c0cf9e0 con 0x7fc15c15e580 2024-09-16T00:09:31.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.254+0000 7fc15bfff640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fc15c15e580 0x7fc15c149a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:31.256 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.254+0000 7fc15bfff640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fc15c15e580 0x7fc15c149a50 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:31.257 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.254+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc15c0cf9e0 con 0x7fc15c15e580 2024-09-16T00:09:31.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.273+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc15c0cfd70 con 0x7fc15c15e580 2024-09-16T00:09:31.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.275+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc15c0cfd70 con 0x7fc15c15e580 2024-09-16T00:09:31.277 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:31.278 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:31.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.275+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fc15c153a50 0x7fc15c062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:31.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.275+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fc15c0cff80 con 0x7fc15c153a50 2024-09-16T00:09:31.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.275+0000 7fc15b7fe640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fc15c153a50 0x7fc15c062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:31.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.276+0000 7fc15b7fe640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fc15c153a50 0x7fc15c062630 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:31.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.277+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc15c0cff80 con 0x7fc15c153a50 2024-09-16T00:09:31.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.296+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc15c062b70 con 0x7fc15c153a50 2024-09-16T00:09:31.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.298+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc15c062b70 con 0x7fc15c153a50 2024-09-16T00:09:31.300 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:31.300 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:31.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.298+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fc15c0ce0b0 0x7fc15c0ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:31.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.298+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fc15c0ce9f0 con 0x7fc15c0ce0b0 2024-09-16T00:09:31.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.298+0000 7fc160f61640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fc15c0ce0b0 0x7fc15c0ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:31.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.299+0000 7fc160f61640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fc15c0ce0b0 0x7fc15c0ce4b0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:31.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.300+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc15c0ce9f0 con 0x7fc15c0ce0b0 2024-09-16T00:09:31.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.318+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc15c0ce9f0 con 0x7fc15c0ce0b0 2024-09-16T00:09:31.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.320+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc15c0ce9f0 con 0x7fc15c0ce0b0 2024-09-16T00:09:31.322 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:31.322 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:31.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.320+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fc15c0cee20 0x7fc15c02cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:31.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.320+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fc15c02d090 con 0x7fc15c0cee20 2024-09-16T00:09:31.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.320+0000 7fc15bfff640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fc15c0cee20 0x7fc15c02cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:31.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.321+0000 7fc15bfff640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fc15c0cee20 0x7fc15c02cb50 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:31.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.322+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc15c02d090 con 0x7fc15c0cee20 2024-09-16T00:09:31.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.338+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc15c02d090 con 0x7fc15c0cee20 2024-09-16T00:09:31.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.339+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc15c02d090 con 0x7fc15c0cee20 2024-09-16T00:09:31.342 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:31.343 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:31.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.340+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fc15c02d710 0x7fc15c206c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:31.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.340+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fc15c02db90 con 0x7fc15c02d710 2024-09-16T00:09:31.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.340+0000 7fc15b7fe640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fc15c02d710 0x7fc15c206c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:31.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.340+0000 7fc15b7fe640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fc15c02d710 0x7fc15c206c40 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:31.343 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.341+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc150028e50 con 0x7fc15c02d710 2024-09-16T00:09:31.345 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:31.344+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:31.348 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:31.348 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:31.358 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:09:31.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.357+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc15c207180 con 0x7fc15c02d710 2024-09-16T00:09:31.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.359+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc15c207180 con 0x7fc15c02d710 2024-09-16T00:09:31.361 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:31.362 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:31.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.359+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fc15c2078a0 0x7fc15c209c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:31.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.359+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fc15c20cae0 con 0x7fc15c2078a0 2024-09-16T00:09:31.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.359+0000 7fc160f61640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fc15c2078a0 0x7fc15c209c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:31.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.360+0000 7fc160f61640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fc15c2078a0 0x7fc15c209c90 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:31.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.361+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc15c20cae0 con 0x7fc15c2078a0 2024-09-16T00:09:31.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.382+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc15c20a250 con 0x7fc15c2078a0 2024-09-16T00:09:31.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.384+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc15c20a250 con 0x7fc15c2078a0 2024-09-16T00:09:31.386 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:31.387 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:09:31.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.384+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fc15c20c0d0 0x7fc15c20c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:31.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.384+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fc15c20fe80 con 0x7fc15c20c0d0 2024-09-16T00:09:31.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.385+0000 7fc15bfff640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fc15c20c0d0 0x7fc15c20c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:31.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.385+0000 7fc15bfff640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fc15c20c0d0 0x7fc15c20c510 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:31.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.386+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc15c20fe80 con 0x7fc15c20c0d0 2024-09-16T00:09:31.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.400+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc15c20a460 con 0x7fc15c20c0d0 2024-09-16T00:09:31.405 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc158ff9640 1 -- 172.21.15.139:0/898446009 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc15c20a460 con 0x7fc15c20c0d0 2024-09-16T00:09:31.405 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:31.407 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:31.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fc15c0c8be0 msgr2=0x7fc15c149000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:31.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fc15c0c8be0 0x7fc15c149000 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fc15c15e580 msgr2=0x7fc15c149a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:31.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fc15c15e580 0x7fc15c149a50 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fc15c153a50 msgr2=0x7fc15c062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:31.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fc15c153a50 0x7fc15c062630 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fc15c0ce0b0 msgr2=0x7fc15c0ce4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:31.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fc15c0ce0b0 0x7fc15c0ce4b0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc160f61640 1 -- 172.21.15.139:0/898446009 reap_dead start 2024-09-16T00:09:31.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fc15c0cee20 msgr2=0x7fc15c02cb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:31.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fc15c0cee20 0x7fc15c02cb50 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc160f61640 1 -- 172.21.15.139:0/898446009 reap_dead start 2024-09-16T00:09:31.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fc15c02d710 msgr2=0x7fc15c206c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:31.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fc15c02d710 0x7fc15c206c40 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fc15c2078a0 msgr2=0x7fc15c209c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:31.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7fc15c2078a0 0x7fc15c209c90 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fc15c20c0d0 msgr2=0x7fc15c20c510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:31.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fc15c20c0d0 0x7fc15c20c510 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc14806c2b0 msgr2=0x7fc14806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:31.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc14806c2b0 0x7fc14806e770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc15c069220 tx=0x7fc150009190 comp rx=0 tx=0).stop 2024-09-16T00:09:31.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc15c0b3430 msgr2=0x7fc15c0cf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:31.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc1629c4640 1 --2- 172.21.15.139:0/898446009 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc15c0b3430 0x7fc15c0cf4a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc14c000c00 tx=0x7fc14c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:31.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.403+0000 7fc160f61640 1 -- 172.21.15.139:0/898446009 reap_dead start 2024-09-16T00:09:31.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.404+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 shutdown_connections 2024-09-16T00:09:31.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.404+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 >> 172.21.15.139:0/898446009 conn(0x7fc15c06c520 msgr2=0x7fc15c1598c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:31.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.404+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 shutdown_connections 2024-09-16T00:09:31.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:31.404+0000 7fc1629c4640 1 -- 172.21.15.139:0/898446009 wait complete. 2024-09-16T00:09:31.457 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:31.455+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:31.470 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:31.471 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:31.482 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:09:31.548 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:31.547+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:31.647 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:31.647 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:31.649 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:31.648+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:31.657 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:09:31.750 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:31.749+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:31.783 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:31.783 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:31.793 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:09:31.851 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:31.850+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:31.925 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:31.925 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:31.937 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:09:31.976 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:31.975+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:32.053 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:32.052+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:32.097 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:32.097 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:32.108 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:09:32.192 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:32.191+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:32.216 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:09:32.218 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:09:32.259 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:32.258+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:32.269 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:32.269 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:32.273 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T00:09:32.278 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:09:32.355 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:32.354+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:32.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.388+0000 7f0da26d0640 1 Processor -- start 2024-09-16T00:09:32.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.389+0000 7f0da26d0640 1 -- start start 2024-09-16T00:09:32.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.390+0000 7f0da26d0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0d9c154c20 0x7f0d9c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:32.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.390+0000 7f0da26d0640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0d9c068060 0x7f0d9c0c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:32.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.390+0000 7f0da26d0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d9c0cc4b0 con 0x7f0d9c0ccc90 2024-09-16T00:09:32.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.390+0000 7f0da26d0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0d9c15b7c0 con 0x7f0d9c154c20 2024-09-16T00:09:32.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.390+0000 7f0da26d0640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f0d9c0cc260 con 0x7f0d9c068060 2024-09-16T00:09:32.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.390+0000 7f0d9bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0d9c154c20 0x7f0d9c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:32.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.390+0000 7f0da0c6d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f0d9c0ccc90 0x7f0d9c0cac00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38132/0 (socket says 172.21.15.139:38132) 2024-09-16T00:09:32.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.390+0000 7f0da0c6d640 1 -- 172.21.15.139:0/2509902215 learned_addr learned my addr 172.21.15.139:0/2509902215 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:32.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.390+0000 7f0d9b7fe640 1 --2- 172.21.15.139:0/2509902215 >> v2:172.21.15.164:3300/0 conn(0x7f0d9c068060 0x7f0d9c0c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:32.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.390+0000 7f0d9bfff640 1 -- 172.21.15.139:0/2509902215 >> v2:172.21.15.164:3300/0 conn(0x7f0d9c068060 msgr2=0x7f0d9c0c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:32.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.390+0000 7f0d9bfff640 1 --2- 172.21.15.139:0/2509902215 >> v2:172.21.15.164:3300/0 conn(0x7f0d9c068060 0x7f0d9c0c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:32.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.390+0000 7f0d9bfff640 1 -- 172.21.15.139:0/2509902215 >> v1:172.21.15.139:6789/0 conn(0x7f0d9c0ccc90 legacy=0x7f0d9c0cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:32.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.390+0000 7f0d9affd640 1 -- 172.21.15.139:0/2509902215 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3380778222 0 0) 0x7f0d9c0cc4b0 con 0x7f0d9c0ccc90 2024-09-16T00:09:32.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.390+0000 7f0d9bfff640 1 -- 172.21.15.139:0/2509902215 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c0cc790 con 0x7f0d9c154c20 2024-09-16T00:09:32.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.391+0000 7f0d9bfff640 1 --2- 172.21.15.139:0/2509902215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d9c154c20 0x7f0d9c15b280 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0685f0 tx=0x7f0d8c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c30de4687fad76 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:32.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.392+0000 7f0d9affd640 1 -- 172.21.15.139:0/2509902215 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c037070 con 0x7f0d9c154c20 2024-09-16T00:09:32.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.392+0000 7f0d9affd640 1 -- 172.21.15.139:0/2509902215 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8c032440 con 0x7f0d9c154c20 2024-09-16T00:09:32.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.392+0000 7f0d9affd640 1 -- 172.21.15.139:0/2509902215 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c032810 con 0x7f0d9c154c20 2024-09-16T00:09:32.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.392+0000 7f0da26d0640 1 -- 172.21.15.139:0/2509902215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d9c154c20 msgr2=0x7f0d9c15b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:32.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.392+0000 7f0da26d0640 1 --2- 172.21.15.139:0/2509902215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d9c154c20 0x7f0d9c15b280 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0685f0 tx=0x7f0d8c02a1a0 comp rx=0 tx=0).stop 2024-09-16T00:09:32.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.392+0000 7f0da26d0640 1 -- 172.21.15.139:0/2509902215 shutdown_connections 2024-09-16T00:09:32.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.392+0000 7f0da26d0640 1 --2- 172.21.15.139:0/2509902215 >> v2:172.21.15.164:3300/0 conn(0x7f0d9c068060 0x7f0d9c0c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:32.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.392+0000 7f0da26d0640 1 --2- 172.21.15.139:0/2509902215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d9c154c20 0x7f0d9c15b280 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:32.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.392+0000 7f0da26d0640 1 -- 172.21.15.139:0/2509902215 >> 172.21.15.139:0/2509902215 conn(0x7f0d9c06ca40 msgr2=0x7f0d9c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:32.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.392+0000 7f0da26d0640 1 -- 172.21.15.139:0/2509902215 shutdown_connections 2024-09-16T00:09:32.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.392+0000 7f0da26d0640 1 -- 172.21.15.139:0/2509902215 wait complete. 2024-09-16T00:09:32.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.394+0000 7f0da26d0640 1 Processor -- start 2024-09-16T00:09:32.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.394+0000 7f0da26d0640 1 -- start start 2024-09-16T00:09:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.395+0000 7f0da26d0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d9c068060 0x7f0d9c1f1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.395+0000 7f0da26d0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0d9c0ccc90 0x7f0d9c1f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.395+0000 7f0da26d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d9c154c20 0x7f0d9c1498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:32.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.395+0000 7f0da26d0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c15e3b0 con 0x7f0d9c154c20 2024-09-16T00:09:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.395+0000 7f0da26d0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c15e0d0 con 0x7f0d9c068060 2024-09-16T00:09:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.395+0000 7f0da26d0640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0d9c15e690 con 0x7f0d9c0ccc90 2024-09-16T00:09:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.395+0000 7f0da0c6d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d9c154c20 0x7f0d9c1498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.395+0000 7f0d9bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d9c068060 0x7f0d9c1f1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.395+0000 7f0da0c6d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d9c154c20 0x7f0d9c1498d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58838/0 (socket says 172.21.15.139:58838) 2024-09-16T00:09:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.395+0000 7f0d9bfff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d9c068060 0x7f0d9c1f1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:55358/0 (socket says 172.21.15.139:55358) 2024-09-16T00:09:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.395+0000 7f0da0c6d640 1 -- 172.21.15.139:0/2108605066 learned_addr learned my addr 172.21.15.139:0/2108605066 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.395+0000 7f0d9b7fe640 1 --2- 172.21.15.139:0/2108605066 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0d9c0ccc90 0x7f0d9c1f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.396+0000 7f0da0c6d640 1 -- 172.21.15.139:0/2108605066 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0d9c0ccc90 msgr2=0x7f0d9c1f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.396+0000 7f0da0c6d640 1 --2- 172.21.15.139:0/2108605066 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0d9c0ccc90 0x7f0d9c1f4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.396+0000 7f0da0c6d640 1 -- 172.21.15.139:0/2108605066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d9c068060 msgr2=0x7f0d9c1f1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.396+0000 7f0da0c6d640 1 --2- 172.21.15.139:0/2108605066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d9c068060 0x7f0d9c1f1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:32.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.396+0000 7f0da0c6d640 1 -- 172.21.15.139:0/2108605066 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c147f70 con 0x7f0d9c154c20 2024-09-16T00:09:32.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.396+0000 7f0da0c6d640 1 --2- 172.21.15.139:0/2108605066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d9c154c20 0x7f0d9c1498d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400c800 tx=0x7f0d9400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:32.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.396+0000 7f0d98ff9640 1 -- 172.21.15.139:0/2108605066 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94018070 con 0x7f0d9c154c20 2024-09-16T00:09:32.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.396+0000 7f0d98ff9640 1 -- 172.21.15.139:0/2108605066 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d94004d80 con 0x7f0d9c154c20 2024-09-16T00:09:32.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.396+0000 7f0d98ff9640 1 -- 172.21.15.139:0/2108605066 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9400a880 con 0x7f0d9c154c20 2024-09-16T00:09:32.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.396+0000 7f0da26d0640 1 -- 172.21.15.139:0/2108605066 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c1f4080 con 0x7f0d9c154c20 2024-09-16T00:09:32.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.396+0000 7f0da26d0640 1 -- 172.21.15.139:0/2108605066 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c159e30 con 0x7f0d9c154c20 2024-09-16T00:09:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.397+0000 7f0da26d0640 1 -- 172.21.15.139:0/2108605066 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d9c0686b0 con 0x7f0d9c154c20 2024-09-16T00:09:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.398+0000 7f0d98ff9640 1 -- 172.21.15.139:0/2108605066 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0d9401b070 con 0x7f0d9c154c20 2024-09-16T00:09:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.398+0000 7f0d98ff9640 1 --2- 172.21.15.139:0/2108605066 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0d8806c2b0 0x7f0d8806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.399+0000 7f0d98ff9640 1 -- 172.21.15.139:0/2108605066 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5998+0+0 (secure 0 0 0) 0x7f0d9408a960 con 0x7f0d9c154c20 2024-09-16T00:09:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.399+0000 7f0d9bfff640 1 --2- 172.21.15.139:0/2108605066 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0d8806c2b0 0x7f0d8806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:32.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.399+0000 7f0d9bfff640 1 --2- 172.21.15.139:0/2108605066 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0d8806c2b0 0x7f0d8806e770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c034000 tx=0x7f0d8c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:32.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.403+0000 7f0d98ff9640 1 -- 172.21.15.139:0/2108605066 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0d94058ab0 con 0x7f0d9c154c20 2024-09-16T00:09:32.440 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:32.440 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:32.449 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:09:32.456 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:32.454+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:32.557 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:32.556+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:32.558 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:32.559 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:32.568 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:09:32.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:32.593+0000 7f0da26d0640 1 -- 172.21.15.139:0/2108605066 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f0d9c0ca640 con 0x7f0d9c154c20 2024-09-16T00:09:32.658 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:32.656+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:32.746 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:32.746 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:32.757 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:09:32.758 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:32.756+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:32.859 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:32.858+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:32.887 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:32.888 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:32.899 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:09:32.960 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:32.959+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:33.019 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:33.019 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:33.030 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:09:33.062 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:33.060+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:33.156 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:33.156 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:33.162 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:33.161+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:33.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.164+0000 7f0d98ff9640 1 -- 172.21.15.139:0/2108605066 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v80) ==== 97+0+0 (secure 0 0 0) 0x7f0d9405cdb0 con 0x7f0d9c154c20 2024-09-16T00:09:33.166 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is set 2024-09-16T00:09:33.167 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:09:33.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.167+0000 7f0da26d0640 1 -- 172.21.15.139:0/2108605066 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0d8806c2b0 msgr2=0x7f0d8806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:33.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.167+0000 7f0da26d0640 1 --2- 172.21.15.139:0/2108605066 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0d8806c2b0 0x7f0d8806e770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c034000 tx=0x7f0d8c02e000 comp rx=0 tx=0).stop 2024-09-16T00:09:33.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.167+0000 7f0da26d0640 1 -- 172.21.15.139:0/2108605066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d9c154c20 msgr2=0x7f0d9c1498d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:33.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.167+0000 7f0da26d0640 1 --2- 172.21.15.139:0/2108605066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d9c154c20 0x7f0d9c1498d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400c800 tx=0x7f0d9400ccc0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.167+0000 7f0da26d0640 1 -- 172.21.15.139:0/2108605066 shutdown_connections 2024-09-16T00:09:33.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.167+0000 7f0da26d0640 1 --2- 172.21.15.139:0/2108605066 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0d8806c2b0 0x7f0d8806e770 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.167+0000 7f0da26d0640 1 --2- 172.21.15.139:0/2108605066 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d9c154c20 0x7f0d9c1498d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.167+0000 7f0da26d0640 1 --2- 172.21.15.139:0/2108605066 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0d9c0ccc90 0x7f0d9c1f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.167+0000 7f0da26d0640 1 --2- 172.21.15.139:0/2108605066 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d9c068060 0x7f0d9c1f1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.167+0000 7f0da26d0640 1 -- 172.21.15.139:0/2108605066 >> 172.21.15.139:0/2108605066 conn(0x7f0d9c06ca40 msgr2=0x7f0d9c0c5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:33.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.167+0000 7f0da26d0640 1 -- 172.21.15.139:0/2108605066 shutdown_connections 2024-09-16T00:09:33.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.167+0000 7f0da26d0640 1 -- 172.21.15.139:0/2108605066 wait complete. 2024-09-16T00:09:33.264 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:33.262+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:33.294 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:33.294 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:33.304 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:09:33.365 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:33.363+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:33.416 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:09:33.429 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:33.429 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:33.438 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:09:33.482 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:33.481+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:33.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.513+0000 7f9e8e7b9640 1 Processor -- start 2024-09-16T00:09:33.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.513+0000 7f9e8e7b9640 1 -- start start 2024-09-16T00:09:33.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.514+0000 7f9e8e7b9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9e880d0490 0x7f9e880b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:33.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.514+0000 7f9e8e7b9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9e880d0870 0x7f9e880b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:33.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.514+0000 7f9e8e7b9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9e880b3430 0x7f9e880b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:33.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.514+0000 7f9e8e7b9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9e8815b940 con 0x7f9e880d0490 2024-09-16T00:09:33.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.514+0000 7f9e8e7b9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f9e880b3bb0 con 0x7f9e880d0870 2024-09-16T00:09:33.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.514+0000 7f9e8e7b9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9e880b3180 con 0x7f9e880b3430 2024-09-16T00:09:33.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.515+0000 7f9e87fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9e880d0490 0x7f9e880b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:33.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.515+0000 7f9e87fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9e880d0490 0x7f9e880b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58840/0 (socket says 172.21.15.139:58840) 2024-09-16T00:09:33.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.515+0000 7f9e87fff640 1 -- 172.21.15.139:0/3547516845 learned_addr learned my addr 172.21.15.139:0/3547516845 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:33.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.515+0000 7f9e877fe640 1 --2- 172.21.15.139:0/3547516845 >> v2:172.21.15.164:3300/0 conn(0x7f9e880d0870 0x7f9e880b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:33.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.515+0000 7f9e8cd56640 1 --2- 172.21.15.139:0/3547516845 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9e880b3430 0x7f9e880b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:33.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.515+0000 7f9e87fff640 1 -- 172.21.15.139:0/3547516845 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9e880b3430 msgr2=0x7f9e880b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:33.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.515+0000 7f9e87fff640 1 --2- 172.21.15.139:0/3547516845 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9e880b3430 0x7f9e880b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.515+0000 7f9e87fff640 1 -- 172.21.15.139:0/3547516845 >> v2:172.21.15.164:3300/0 conn(0x7f9e880d0870 msgr2=0x7f9e880b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:33.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.515+0000 7f9e87fff640 1 --2- 172.21.15.139:0/3547516845 >> v2:172.21.15.164:3300/0 conn(0x7f9e880d0870 0x7f9e880b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.515+0000 7f9e87fff640 1 -- 172.21.15.139:0/3547516845 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8815adc0 con 0x7f9e880d0490 2024-09-16T00:09:33.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.516+0000 7f9e87fff640 1 --2- 172.21.15.139:0/3547516845 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e880d0490 0x7f9e880b2c40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9e880b5cc0 tx=0x7f9e7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5dd3836c5ee004 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:33.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.516+0000 7f9e86ffd640 1 -- 172.21.15.139:0/3547516845 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78037070 con 0x7f9e880d0490 2024-09-16T00:09:33.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.516+0000 7f9e86ffd640 1 -- 172.21.15.139:0/3547516845 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e78032440 con 0x7f9e880d0490 2024-09-16T00:09:33.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.516+0000 7f9e86ffd640 1 -- 172.21.15.139:0/3547516845 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78032810 con 0x7f9e880d0490 2024-09-16T00:09:33.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.517+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/3547516845 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e880d0490 msgr2=0x7f9e880b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:33.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.517+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/3547516845 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e880d0490 0x7f9e880b2c40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9e880b5cc0 tx=0x7f9e7802a270 comp rx=0 tx=0).stop 2024-09-16T00:09:33.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.517+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/3547516845 shutdown_connections 2024-09-16T00:09:33.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.517+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/3547516845 >> v2:172.21.15.164:3300/0 conn(0x7f9e880d0870 0x7f9e880b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.517+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/3547516845 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9e880b3430 0x7f9e880b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.517+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/3547516845 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e880d0490 0x7f9e880b2c40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.517+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/3547516845 >> 172.21.15.139:0/3547516845 conn(0x7f9e8806c520 msgr2=0x7f9e8806c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:33.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.517+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/3547516845 shutdown_connections 2024-09-16T00:09:33.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.517+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/3547516845 wait complete. 2024-09-16T00:09:33.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.518+0000 7f9e8e7b9640 1 Processor -- start 2024-09-16T00:09:33.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.519+0000 7f9e8e7b9640 1 -- start start 2024-09-16T00:09:33.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.519+0000 7f9e8e7b9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e880b3430 0x7f9e88157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:33.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.519+0000 7f9e8e7b9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e880d0490 0x7f9e881f1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:33.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.519+0000 7f9e8e7b9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9e880d0870 0x7f9e88148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:33.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.519+0000 7f9e8e7b9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8815b660 con 0x7f9e880b3430 2024-09-16T00:09:33.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.519+0000 7f9e8e7b9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8815b380 con 0x7f9e880d0490 2024-09-16T00:09:33.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.519+0000 7f9e8e7b9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9e8815a980 con 0x7f9e880d0870 2024-09-16T00:09:33.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.520+0000 7f9e87fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e880b3430 0x7f9e88157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:33.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.520+0000 7f9e87fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e880b3430 0x7f9e88157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58856/0 (socket says 172.21.15.139:58856) 2024-09-16T00:09:33.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.520+0000 7f9e87fff640 1 -- 172.21.15.139:0/1366038964 learned_addr learned my addr 172.21.15.139:0/1366038964 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:33.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.520+0000 7f9e8cd56640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9e880d0870 0x7f9e88148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:33.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.520+0000 7f9e877fe640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e880d0490 0x7f9e881f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:33.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.520+0000 7f9e87fff640 1 -- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9e880d0870 msgr2=0x7f9e88148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:33.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.520+0000 7f9e87fff640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9e880d0870 0x7f9e88148560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.520+0000 7f9e87fff640 1 -- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e880d0490 msgr2=0x7f9e881f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:33.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.520+0000 7f9e87fff640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9e880d0490 0x7f9e881f1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.520+0000 7f9e87fff640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e88150c00 con 0x7f9e880b3430 2024-09-16T00:09:33.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.520+0000 7f9e87fff640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e880b3430 0x7f9e88157030 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9e7802a2b0 tx=0x7f9e780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:33.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.520+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78041020 con 0x7f9e880b3430 2024-09-16T00:09:33.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.520+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e7803d620 con 0x7f9e880b3430 2024-09-16T00:09:33.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.520+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78037040 con 0x7f9e880b3430 2024-09-16T00:09:33.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.520+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e88150920 con 0x7f9e880b3430 2024-09-16T00:09:33.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.520+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e88156550 con 0x7f9e880b3430 2024-09-16T00:09:33.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.521+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f9e880b2c40 con 0x7f9e880b3430 2024-09-16T00:09:33.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.522+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9e78044020 con 0x7f9e880b3430 2024-09-16T00:09:33.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.523+0000 7f9e84ff9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9e7406c2b0 0x7f9e7406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:33.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.523+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5998+0+0 (secure 0 0 0) 0x7f9e780abad0 con 0x7f9e880b3430 2024-09-16T00:09:33.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.523+0000 7f9e877fe640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9e7406c2b0 0x7f9e7406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:33.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.523+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v80) ==== 54+0+16 (secure 0 0 0) 0x7f9e7804b030 con 0x7f9e880b3430 2024-09-16T00:09:33.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.523+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f9e88156790 0x7f9e880cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:33.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.523+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e880b4a80 con 0x7f9e88156790 2024-09-16T00:09:33.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.523+0000 7f9e877fe640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9e7406c2b0 0x7f9e7406e770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9e881f4a20 tx=0x7f9e7c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:33.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.523+0000 7f9e8cd56640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f9e88156790 0x7f9e880cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:33.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.524+0000 7f9e8cd56640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f9e88156790 0x7f9e880cb720 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:33.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.524+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e880b4a80 con 0x7f9e88156790 2024-09-16T00:09:33.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.541+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f9e88159fc0 con 0x7f9e88156790 2024-09-16T00:09:33.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.542+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9e88159fc0 con 0x7f9e88156790 2024-09-16T00:09:33.544 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:33.545 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:33.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.543+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f9e88155c70 0x7f9e881f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:33.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.543+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f9e88068f30 con 0x7f9e88155c70 2024-09-16T00:09:33.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.543+0000 7f9e87fff640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f9e88155c70 0x7f9e881f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:33.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.543+0000 7f9e87fff640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f9e88155c70 0x7f9e881f3bd0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:33.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.544+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e88068f30 con 0x7f9e88155c70 2024-09-16T00:09:33.549 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:33.549 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:33.558 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:09:33.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.563+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f9e881f4360 con 0x7f9e88155c70 2024-09-16T00:09:33.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.565+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9e881f4360 con 0x7f9e88155c70 2024-09-16T00:09:33.567 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:33.568 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:33.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.565+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f9e881f0900 0x7f9e881f0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:33.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.565+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f9e881f1220 con 0x7f9e881f0900 2024-09-16T00:09:33.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.566+0000 7f9e877fe640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f9e881f0900 0x7f9e881f0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:33.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.566+0000 7f9e877fe640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f9e881f0900 0x7f9e881f0ce0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:33.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.567+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e881f1220 con 0x7f9e881f0900 2024-09-16T00:09:33.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.586+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f9e881f1220 con 0x7f9e881f0900 2024-09-16T00:09:33.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.587+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9e881f1220 con 0x7f9e881f0900 2024-09-16T00:09:33.589 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.588+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f9e88063230 0x7f9e880ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.588+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f9e881f1220 con 0x7f9e88063230 2024-09-16T00:09:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.588+0000 7f9e8cd56640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f9e88063230 0x7f9e880ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:33.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.588+0000 7f9e8cd56640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f9e88063230 0x7f9e880ce0b0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:33.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.589+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e881f1220 con 0x7f9e88063230 2024-09-16T00:09:33.599 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:33.598+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:33.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.606+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f9e881f1220 con 0x7f9e88063230 2024-09-16T00:09:33.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.608+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9e881f1220 con 0x7f9e88063230 2024-09-16T00:09:33.609 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:33.610 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:33.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.608+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f9e880cee30 0x7f9e8802c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:33.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.608+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f9e881f1220 con 0x7f9e880cee30 2024-09-16T00:09:33.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.608+0000 7f9e87fff640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f9e880cee30 0x7f9e8802c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:33.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.608+0000 7f9e87fff640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f9e880cee30 0x7f9e8802c640 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:33.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.609+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e881f1220 con 0x7f9e880cee30 2024-09-16T00:09:33.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.629+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f9e881f1220 con 0x7f9e880cee30 2024-09-16T00:09:33.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.631+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9e881f1220 con 0x7f9e880cee30 2024-09-16T00:09:33.632 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.631+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f9e8802d0f0 0x7f9e8802d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.631+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f9e882095e0 con 0x7f9e8802d0f0 2024-09-16T00:09:33.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.631+0000 7f9e877fe640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f9e8802d0f0 0x7f9e8802d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:33.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.632+0000 7f9e877fe640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f9e8802d0f0 0x7f9e8802d550 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:33.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.633+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e882095e0 con 0x7f9e8802d0f0 2024-09-16T00:09:33.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.650+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f9e88204ce0 con 0x7f9e8802d0f0 2024-09-16T00:09:33.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.652+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9e88204ce0 con 0x7f9e8802d0f0 2024-09-16T00:09:33.654 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.653+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f9e88205360 0x7f9e88205780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.653+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f9e88205f00 con 0x7f9e88205360 2024-09-16T00:09:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.653+0000 7f9e8cd56640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f9e88205360 0x7f9e88205780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:33.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.653+0000 7f9e8cd56640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f9e88205360 0x7f9e88205780 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:33.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.654+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e88205f00 con 0x7f9e88205360 2024-09-16T00:09:33.668 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:33.667+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:33.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.673+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f9e88205f00 con 0x7f9e88205360 2024-09-16T00:09:33.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.675+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9e88205f00 con 0x7f9e88205360 2024-09-16T00:09:33.677 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:33.677 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:09:33.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.675+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9e8820a980 0x7f9e8820cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:33.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.675+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f9e8820fb40 con 0x7f9e8820a980 2024-09-16T00:09:33.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.675+0000 7f9e87fff640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9e8820a980 0x7f9e8820cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:33.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.676+0000 7f9e87fff640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9e8820a980 0x7f9e8820cd70 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:33.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.677+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e8820fb40 con 0x7f9e8820a980 2024-09-16T00:09:33.696 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:33.696 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:33.697 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:33.697 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:33.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.706+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f9e8820d2b0 con 0x7f9e8820a980 2024-09-16T00:09:33.710 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:09:33.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.708+0000 7f9e84ff9640 1 -- 172.21.15.139:0/1366038964 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9e8820d2b0 con 0x7f9e8820a980 2024-09-16T00:09:33.711 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:33.712 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:33.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f9e88156790 msgr2=0x7f9e880cb720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:33.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f9e88156790 0x7f9e880cb720 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f9e88155c70 msgr2=0x7f9e881f3bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:33.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f9e88155c70 0x7f9e881f3bd0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f9e881f0900 msgr2=0x7f9e881f0ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:33.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f9e881f0900 0x7f9e881f0ce0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f9e88063230 msgr2=0x7f9e880ce0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:33.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f9e88063230 0x7f9e880ce0b0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f9e880cee30 msgr2=0x7f9e8802c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:33.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8cd56640 1 -- 172.21.15.139:0/1366038964 reap_dead start 2024-09-16T00:09:33.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f9e880cee30 0x7f9e8802c640 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8cd56640 1 -- 172.21.15.139:0/1366038964 reap_dead start 2024-09-16T00:09:33.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f9e8802d0f0 msgr2=0x7f9e8802d550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:33.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f9e8802d0f0 0x7f9e8802d550 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f9e88205360 msgr2=0x7f9e88205780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:33.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f9e88205360 0x7f9e88205780 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9e8820a980 msgr2=0x7f9e8820cd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:33.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9e8820a980 0x7f9e8820cd70 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9e7406c2b0 msgr2=0x7f9e7406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:33.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9e7406c2b0 0x7f9e7406e770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9e881f4a20 tx=0x7f9e7c009190 comp rx=0 tx=0).stop 2024-09-16T00:09:33.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e880b3430 msgr2=0x7f9e88157030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:33.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 --2- 172.21.15.139:0/1366038964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e880b3430 0x7f9e88157030 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9e7802a2b0 tx=0x7f9e780046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:33.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8cd56640 1 -- 172.21.15.139:0/1366038964 reap_dead start 2024-09-16T00:09:33.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 shutdown_connections 2024-09-16T00:09:33.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 >> 172.21.15.139:0/1366038964 conn(0x7f9e8806c520 msgr2=0x7f9e8814a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:33.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 shutdown_connections 2024-09-16T00:09:33.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:33.709+0000 7f9e8e7b9640 1 -- 172.21.15.139:0/1366038964 wait complete. 2024-09-16T00:09:33.769 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:33.767+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:33.831 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:33.831 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:33.831 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:33.831 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:33.831 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:33.840 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:09:33.882 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:33.880+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:33.971 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:33.970+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:34.001 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:34.002 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:34.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:34.002 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:34.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.002 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:115 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'77, mlcod=17'66)", 2024-09-16T00:09:34.002 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:09:34.002 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 129.833438282, 2024-09-16T00:09:34.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032925509999999999, 2024-09-16T00:09:34.002 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.002 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.002 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.002 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:09:34.003 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166176+0000", 2024-09-16T00:09:34.003 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166177+0000", 2024-09-16T00:09:34.003 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:09:34.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.004 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166216+0000", 2024-09-16T00:09:34.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9008000000000003e-05 2024-09-16T00:09:34.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.004 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166226+0000", 2024-09-16T00:09:34.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0081999999999999e-05 2024-09-16T00:09:34.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.004 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166230+0000", 2024-09-16T00:09:34.005 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5079999999999999e-06 2024-09-16T00:09:34.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.005 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.005 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166259+0000", 2024-09-16T00:09:34.005 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8928000000000002e-05 2024-09-16T00:09:34.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.005 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.005 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166271+0000", 2024-09-16T00:09:34.005 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1531e-05 2024-09-16T00:09:34.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169449+0000", 2024-09-16T00:09:34.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003178143 2024-09-16T00:09:34.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169468+0000", 2024-09-16T00:09:34.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9700000000000001e-05 2024-09-16T00:09:34.006 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.006 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.006 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.006 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4203.0:116 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'78, mlcod=17'66)", 2024-09-16T00:09:34.007 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:09:34.007 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 129.832976823, 2024-09-16T00:09:34.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003179374, 2024-09-16T00:09:34.007 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.007 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.007 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.007 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:09:34.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166637+0000", 2024-09-16T00:09:34.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166638+0000", 2024-09-16T00:09:34.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2300000000000002e-07 2024-09-16T00:09:34.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166641+0000", 2024-09-16T00:09:34.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0079999999999998e-06 2024-09-16T00:09:34.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.009 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.009 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166646+0000", 2024-09-16T00:09:34.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4970000000000001e-06 2024-09-16T00:09:34.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.009 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.009 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166650+0000", 2024-09-16T00:09:34.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5729999999999999e-06 2024-09-16T00:09:34.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.009 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166682+0000", 2024-09-16T00:09:34.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2218999999999998e-05 2024-09-16T00:09:34.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.010 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.166694+0000", 2024-09-16T00:09:34.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1902999999999999e-05 2024-09-16T00:09:34.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.010 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169784+0000", 2024-09-16T00:09:34.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003089545 2024-09-16T00:09:34.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.011 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.011 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:24.169817+0000", 2024-09-16T00:09:34.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3206e-05 2024-09-16T00:09:34.011 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.011 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.011 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.011 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:34.011 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:09:34.011 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 115.38371148, 2024-09-16T00:09:34.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061593860000000002, 2024-09-16T00:09:34.012 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.012 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.012 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.012 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.012 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:09:34.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.012 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615903+0000", 2024-09-16T00:09:34.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615907+0000", 2024-09-16T00:09:34.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9389999999999999e-06 2024-09-16T00:09:34.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615920+0000", 2024-09-16T00:09:34.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2989999999999999e-05 2024-09-16T00:09:34.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615931+0000", 2024-09-16T00:09:34.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.164e-05 2024-09-16T00:09:34.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615939+0000", 2024-09-16T00:09:34.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8029999999999997e-06 2024-09-16T00:09:34.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616310+0000", 2024-09-16T00:09:34.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00037111400000000003 2024-09-16T00:09:34.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616406+0000", 2024-09-16T00:09:34.015 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5372999999999996e-05 2024-09-16T00:09:34.015 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.015 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.015 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:34.015 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616514+0000", 2024-09-16T00:09:34.015 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108564 2024-09-16T00:09:34.015 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.015 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.015 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.016 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618158+0000", 2024-09-16T00:09:34.016 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016442449999999999 2024-09-16T00:09:34.016 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.016 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.016 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.016 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621972+0000", 2024-09-16T00:09:34.016 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038132299999999999 2024-09-16T00:09:34.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.017 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.017 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.017 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622018+0000", 2024-09-16T00:09:34.017 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6239000000000002e-05 2024-09-16T00:09:34.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.017 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.017 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.017 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622062+0000", 2024-09-16T00:09:34.017 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4249000000000003e-05 2024-09-16T00:09:34.017 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.017 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.096 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.096 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.096 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.096 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:34.096 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:09:34.096 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 115.383658012, 2024-09-16T00:09:34.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006258914, 2024-09-16T00:09:34.097 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.097 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.097 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.097 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.097 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.097 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.097 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:09:34.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.097 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.097 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.097 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615956+0000", 2024-09-16T00:09:34.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615957+0000", 2024-09-16T00:09:34.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1399999999999997e-07 2024-09-16T00:09:34.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615960+0000", 2024-09-16T00:09:34.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0860000000000002e-06 2024-09-16T00:09:34.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.099 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615967+0000", 2024-09-16T00:09:34.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9059999999999996e-06 2024-09-16T00:09:34.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.099 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.100 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615972+0000", 2024-09-16T00:09:34.100 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5679999999999999e-06 2024-09-16T00:09:34.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.100 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616717+0000", 2024-09-16T00:09:34.100 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074438 2024-09-16T00:09:34.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616781+0000", 2024-09-16T00:09:34.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4808999999999996e-05 2024-09-16T00:09:34.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:34.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616885+0000", 2024-09-16T00:09:34.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010404900000000001 2024-09-16T00:09:34.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618163+0000", 2024-09-16T00:09:34.102 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012773870000000001 2024-09-16T00:09:34.102 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.102 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.102 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.102 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622116+0000", 2024-09-16T00:09:34.102 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039533470000000003 2024-09-16T00:09:34.102 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.102 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.102 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.102 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622173+0000", 2024-09-16T00:09:34.102 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6347000000000001e-05 2024-09-16T00:09:34.102 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.102 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.103 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.103 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622215+0000", 2024-09-16T00:09:34.103 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2521000000000002e-05 2024-09-16T00:09:34.103 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.103 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.103 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.103 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.103 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.103 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:27 2.13 2:cd5a64a3:::100.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:34.103 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:34.103 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 115.38363427500001, 2024-09-16T00:09:34.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041725529999999999, 2024-09-16T00:09:34.104 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.104 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.104 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.104 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.104 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.104 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.104 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:34.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.104 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:34.105 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.105 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615980+0000", 2024-09-16T00:09:34.105 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8799999999999998e-07 2024-09-16T00:09:34.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.106 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.106 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615990+0000", 2024-09-16T00:09:34.106 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2150000000000007e-06 2024-09-16T00:09:34.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.106 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.106 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615996+0000", 2024-09-16T00:09:34.106 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7159999999999997e-06 2024-09-16T00:09:34.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.107 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.107 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616001+0000", 2024-09-16T00:09:34.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9339999999999997e-06 2024-09-16T00:09:34.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.107 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.107 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.107 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616293+0000", 2024-09-16T00:09:34.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029210899999999998 2024-09-16T00:09:34.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.107 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.107 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.107 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616401+0000", 2024-09-16T00:09:34.108 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108087 2024-09-16T00:09:34.108 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.108 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.108 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:34.108 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616561+0000", 2024-09-16T00:09:34.108 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015940600000000001 2024-09-16T00:09:34.108 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.108 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.108 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.109 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618146+0000", 2024-09-16T00:09:34.109 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015857040000000001 2024-09-16T00:09:34.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.109 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.109 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.109 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620091+0000", 2024-09-16T00:09:34.109 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001944837 2024-09-16T00:09:34.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.109 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.109 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.109 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620126+0000", 2024-09-16T00:09:34.110 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4242999999999997e-05 2024-09-16T00:09:34.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.110 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.110 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620152+0000", 2024-09-16T00:09:34.110 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6914e-05 2024-09-16T00:09:34.110 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.110 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.110 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.110 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:34.111 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:34.111 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 115.38359604999999, 2024-09-16T00:09:34.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00630628, 2024-09-16T00:09:34.111 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.111 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.111 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.111 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.111 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.111 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.111 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:34.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.112 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616018+0000", 2024-09-16T00:09:34.112 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.112 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616019+0000", 2024-09-16T00:09:34.112 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5999999999999999e-07 2024-09-16T00:09:34.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.113 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616022+0000", 2024-09-16T00:09:34.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0149999999999999e-06 2024-09-16T00:09:34.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.113 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616031+0000", 2024-09-16T00:09:34.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9770000000000006e-06 2024-09-16T00:09:34.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.114 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616036+0000", 2024-09-16T00:09:34.114 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2639999999999999e-06 2024-09-16T00:09:34.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.114 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617057+0000", 2024-09-16T00:09:34.114 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010212999999999999 2024-09-16T00:09:34.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617146+0000", 2024-09-16T00:09:34.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9234000000000007e-05 2024-09-16T00:09:34.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:34.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617232+0000", 2024-09-16T00:09:34.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5218999999999998e-05 2024-09-16T00:09:34.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619107+0000", 2024-09-16T00:09:34.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001875016 2024-09-16T00:09:34.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622259+0000", 2024-09-16T00:09:34.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031527930000000001 2024-09-16T00:09:34.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622290+0000", 2024-09-16T00:09:34.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1022999999999998e-05 2024-09-16T00:09:34.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622324+0000", 2024-09-16T00:09:34.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3979000000000001e-05 2024-09-16T00:09:34.117 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.117 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.117 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.117 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:28 2.13 2:c91d4a1d:::mds0_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:34.117 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:09:34.118 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 115.383570748, 2024-09-16T00:09:34.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042412739999999997, 2024-09-16T00:09:34.118 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.118 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.118 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.118 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:09:34.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.119 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.119 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616043+0000", 2024-09-16T00:09:34.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.119 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.119 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616044+0000", 2024-09-16T00:09:34.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:09:34.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.119 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.119 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616052+0000", 2024-09-16T00:09:34.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.439e-06 2024-09-16T00:09:34.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616059+0000", 2024-09-16T00:09:34.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1389999999999998e-06 2024-09-16T00:09:34.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616065+0000", 2024-09-16T00:09:34.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:09:34.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616823+0000", 2024-09-16T00:09:34.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075781199999999996 2024-09-16T00:09:34.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616918+0000", 2024-09-16T00:09:34.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5174999999999996e-05 2024-09-16T00:09:34.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:34.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617019+0000", 2024-09-16T00:09:34.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101161 2024-09-16T00:09:34.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618503+0000", 2024-09-16T00:09:34.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014836750000000001 2024-09-16T00:09:34.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620201+0000", 2024-09-16T00:09:34.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00169805 2024-09-16T00:09:34.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.123 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620255+0000", 2024-09-16T00:09:34.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4499000000000001e-05 2024-09-16T00:09:34.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.123 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620285+0000", 2024-09-16T00:09:34.124 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9349e-05 2024-09-16T00:09:34.124 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.124 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.124 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.124 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:34.124 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:09:34.125 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 115.383541757, 2024-09-16T00:09:34.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063682799999999996, 2024-09-16T00:09:34.125 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.125 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.125 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:09:34.125 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.125 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:09:34.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616072+0000", 2024-09-16T00:09:34.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:09:34.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9900000000000001e-07 2024-09-16T00:09:34.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616079+0000", 2024-09-16T00:09:34.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9009999999999999e-06 2024-09-16T00:09:34.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.127 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.127 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616086+0000", 2024-09-16T00:09:34.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.776e-06 2024-09-16T00:09:34.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.127 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.127 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616091+0000", 2024-09-16T00:09:34.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5600000000000001e-06 2024-09-16T00:09:34.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618339+0000", 2024-09-16T00:09:34.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022476179999999998 2024-09-16T00:09:34.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618438+0000", 2024-09-16T00:09:34.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9025999999999997e-05 2024-09-16T00:09:34.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.129 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622441+0000", 2024-09-16T00:09:34.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040029999999999996 2024-09-16T00:09:34.129 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.129 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.129 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.129 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:221 3.1b 3:dad7a251:::100000002d7.00000000:head [write 0~30412] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:34.129 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:09:34.129 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.250333566999998, 2024-09-16T00:09:34.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034396769999999999, 2024-09-16T00:09:34.129 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.129 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.130 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.130 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:34.130 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:34.130 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:34.130 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 221 2024-09-16T00:09:34.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.130 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:09:34.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749281+0000", 2024-09-16T00:09:34.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749282+0000", 2024-09-16T00:09:34.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3769999999999999e-06 2024-09-16T00:09:34.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749300+0000", 2024-09-16T00:09:34.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8204999999999999e-05 2024-09-16T00:09:34.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749311+0000", 2024-09-16T00:09:34.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0599e-05 2024-09-16T00:09:34.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749318+0000", 2024-09-16T00:09:34.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9789999999999996e-06 2024-09-16T00:09:34.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749367+0000", 2024-09-16T00:09:34.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9382999999999998e-05 2024-09-16T00:09:34.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.133 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749450+0000", 2024-09-16T00:09:34.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-05 2024-09-16T00:09:34.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.133 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:34.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.749526+0000", 2024-09-16T00:09:34.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5564999999999999e-05 2024-09-16T00:09:34.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.750601+0000", 2024-09-16T00:09:34.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010751199999999999 2024-09-16T00:09:34.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752665+0000", 2024-09-16T00:09:34.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00206442 2024-09-16T00:09:34.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752696+0000", 2024-09-16T00:09:34.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0476000000000001e-05 2024-09-16T00:09:34.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.135 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.135 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.752720+0000", 2024-09-16T00:09:34.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4482999999999999e-05 2024-09-16T00:09:34.135 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.135 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.135 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.135 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:323 3.3 e58/20 3:c319d237:::10000000354.00000000:head v 58'3, mlcod=57'1)", 2024-09-16T00:09:34.135 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:09:34.135 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.966953795999999, 2024-09-16T00:09:34.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034011000000000002, 2024-09-16T00:09:34.135 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.136 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.136 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.136 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.136 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.136 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:09:34.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.136 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.136 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032660+0000", 2024-09-16T00:09:34.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032661+0000", 2024-09-16T00:09:34.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7000000000000003e-07 2024-09-16T00:09:34.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032670+0000", 2024-09-16T00:09:34.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.602e-06 2024-09-16T00:09:34.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032682+0000", 2024-09-16T00:09:34.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1754e-05 2024-09-16T00:09:34.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032688+0000", 2024-09-16T00:09:34.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8140000000000003e-06 2024-09-16T00:09:34.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032740+0000", 2024-09-16T00:09:34.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1981000000000003e-05 2024-09-16T00:09:34.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.032765+0000", 2024-09-16T00:09:34.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4097e-05 2024-09-16T00:09:34.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.036023+0000", 2024-09-16T00:09:34.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032585240000000001 2024-09-16T00:09:34.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.140 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.140 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.140 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.036061+0000", 2024-09-16T00:09:34.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8457999999999997e-05 2024-09-16T00:09:34.140 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.140 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.140 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.141 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:664 3.39 e59/20 3:9c9d3e51:::100000004f6.00000000:head v 59'7, mlcod=58'5)", 2024-09-16T00:09:34.141 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:34.141 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 35.779284637000003, 2024-09-16T00:09:34.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035253329999999999, 2024-09-16T00:09:34.141 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.141 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.141 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.141 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:34.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:34.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220330+0000", 2024-09-16T00:09:34.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2399999999999996e-07 2024-09-16T00:09:34.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220338+0000", 2024-09-16T00:09:34.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0120000000000005e-06 2024-09-16T00:09:34.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220353+0000", 2024-09-16T00:09:34.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4718999999999999e-05 2024-09-16T00:09:34.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220361+0000", 2024-09-16T00:09:34.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6559999999999992e-06 2024-09-16T00:09:34.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220422+0000", 2024-09-16T00:09:34.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.071e-05 2024-09-16T00:09:34.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.145 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.220446+0000", 2024-09-16T00:09:34.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4556e-05 2024-09-16T00:09:34.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.145 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.145 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.223813+0000", 2024-09-16T00:09:34.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033666439999999998 2024-09-16T00:09:34.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.223855+0000", 2024-09-16T00:09:34.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2111999999999999e-05 2024-09-16T00:09:34.146 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.146 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.146 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.146 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:681 3.17 3:ea55746f:::1000000050b.00000000:head [write 0~1071] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:34.147 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:34.147 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 35.734541671999999, 2024-09-16T00:09:34.147 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036588620000000001, 2024-09-16T00:09:34.147 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.147 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.147 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.147 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:34.147 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:34.147 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:34.147 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 681 2024-09-16T00:09:34.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.147 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:34.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:34.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265073+0000", 2024-09-16T00:09:34.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7600000000000006e-07 2024-09-16T00:09:34.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.149 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265080+0000", 2024-09-16T00:09:34.149 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8759999999999999e-06 2024-09-16T00:09:34.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.149 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265094+0000", 2024-09-16T00:09:34.149 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.411e-05 2024-09-16T00:09:34.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.149 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265102+0000", 2024-09-16T00:09:34.150 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9640000000000003e-06 2024-09-16T00:09:34.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.150 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.150 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.150 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265156+0000", 2024-09-16T00:09:34.150 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3375000000000003e-05 2024-09-16T00:09:34.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.150 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.150 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.150 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265229+0000", 2024-09-16T00:09:34.150 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3147999999999998e-05 2024-09-16T00:09:34.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.150 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.150 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:34.151 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265304+0000", 2024-09-16T00:09:34.151 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4568999999999997e-05 2024-09-16T00:09:34.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.151 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.151 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.151 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.265747+0000", 2024-09-16T00:09:34.151 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00044363099999999999 2024-09-16T00:09:34.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.151 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.151 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.151 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.268635+0000", 2024-09-16T00:09:34.151 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028873459999999998 2024-09-16T00:09:34.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.152 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.152 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.152 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.268688+0000", 2024-09-16T00:09:34.152 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3155999999999997e-05 2024-09-16T00:09:34.152 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.152 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.152 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.152 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.268731+0000", 2024-09-16T00:09:34.152 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3711000000000002e-05 2024-09-16T00:09:34.152 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.152 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.152 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.152 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.152 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.153 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:3099 3.6 3:617936f3:::10000000f78.00000000:head [write 0~1641] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-16T00:09:34.153 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:06.096600+0000", 2024-09-16T00:09:34.153 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.903014491, 2024-09-16T00:09:34.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031984349999999999, 2024-09-16T00:09:34.153 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.153 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.153 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.153 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:34.153 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:34.153 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:34.153 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 3099 2024-09-16T00:09:34.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.153 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.153 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.154 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.154 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096600+0000", 2024-09-16T00:09:34.154 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.154 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.154 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.154 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096600+0000", 2024-09-16T00:09:34.154 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.154 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.154 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.154 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096602+0000", 2024-09-16T00:09:34.154 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4889999999999998e-06 2024-09-16T00:09:34.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096612+0000", 2024-09-16T00:09:34.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5629999999999998e-06 2024-09-16T00:09:34.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096631+0000", 2024-09-16T00:09:34.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9369999999999999e-05 2024-09-16T00:09:34.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096641+0000", 2024-09-16T00:09:34.156 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0098999999999999e-05 2024-09-16T00:09:34.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.156 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.156 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.156 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096705+0000", 2024-09-16T00:09:34.156 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3551000000000002e-05 2024-09-16T00:09:34.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.156 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.156 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op must wait for map", 2024-09-16T00:09:34.156 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.096711+0000", 2024-09-16T00:09:34.156 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5660000000000003e-06 2024-09-16T00:09:34.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.156 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.098401+0000", 2024-09-16T00:09:34.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016899840000000001 2024-09-16T00:09:34.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.098514+0000", 2024-09-16T00:09:34.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000112551 2024-09-16T00:09:34.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:34.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.098636+0000", 2024-09-16T00:09:34.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000122488 2024-09-16T00:09:34.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.158 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.158 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.099270+0000", 2024-09-16T00:09:34.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00063404300000000004 2024-09-16T00:09:34.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.158 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.158 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.099702+0000", 2024-09-16T00:09:34.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000431956 2024-09-16T00:09:34.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.158 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.158 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.099761+0000", 2024-09-16T00:09:34.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8421000000000002e-05 2024-09-16T00:09:34.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.159 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.159 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.159 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:06.099798+0000", 2024-09-16T00:09:34.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7354000000000002e-05 2024-09-16T00:09:34.159 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.159 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.159 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.159 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.159 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:219 2.1f 2:f83d65d8:::200.00000004:head [write 863658~2915703 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e65)", 2024-09-16T00:09:34.159 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:34.159 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.128674409999999, 2024-09-16T00:09:34.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013772627000000001, 2024-09-16T00:09:34.160 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.160 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.160 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.160 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.160 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.160 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:34.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.160 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.160 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:34.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.161 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.161 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.161 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870941+0000", 2024-09-16T00:09:34.161 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4e-07 2024-09-16T00:09:34.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.161 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.161 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.161 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877134+0000", 2024-09-16T00:09:34.161 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061936769999999999 2024-09-16T00:09:34.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.161 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.161 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877153+0000", 2024-09-16T00:09:34.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8408000000000001e-05 2024-09-16T00:09:34.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877161+0000", 2024-09-16T00:09:34.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2709999999999992e-06 2024-09-16T00:09:34.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877225+0000", 2024-09-16T00:09:34.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4190999999999996e-05 2024-09-16T00:09:34.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877289+0000", 2024-09-16T00:09:34.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4058000000000002e-05 2024-09-16T00:09:34.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:34.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877376+0000", 2024-09-16T00:09:34.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7227999999999999e-05 2024-09-16T00:09:34.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.882925+0000", 2024-09-16T00:09:34.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055481050000000002 2024-09-16T00:09:34.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.164 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.164 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.164 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884635+0000", 2024-09-16T00:09:34.164 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017102759999999999 2024-09-16T00:09:34.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.164 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.164 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.164 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884674+0000", 2024-09-16T00:09:34.164 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8914e-05 2024-09-16T00:09:34.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.164 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.164 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.164 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884712+0000", 2024-09-16T00:09:34.165 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658999999999997e-05 2024-09-16T00:09:34.165 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.165 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.165 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.165 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5787 3.3d 3:bc1a97d0:::10000001a99.00000000:head [write 0~2770] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:34.165 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.087043+0000", 2024-09-16T00:09:34.165 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.912571311000001, 2024-09-16T00:09:34.165 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081081320000000005, 2024-09-16T00:09:34.165 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.165 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.165 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.166 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:34.166 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:34.166 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:34.166 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5787 2024-09-16T00:09:34.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.166 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087043+0000", 2024-09-16T00:09:34.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087043+0000", 2024-09-16T00:09:34.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087045+0000", 2024-09-16T00:09:34.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:09:34.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087053+0000", 2024-09-16T00:09:34.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7669999999999992e-06 2024-09-16T00:09:34.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.168 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.168 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087069+0000", 2024-09-16T00:09:34.168 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5155e-05 2024-09-16T00:09:34.168 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.168 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.168 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.168 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087081+0000", 2024-09-16T00:09:34.168 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2485e-05 2024-09-16T00:09:34.168 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.168 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.168 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.168 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087144+0000", 2024-09-16T00:09:34.168 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3255000000000003e-05 2024-09-16T00:09:34.168 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.169 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op must wait for map", 2024-09-16T00:09:34.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.087152+0000", 2024-09-16T00:09:34.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5850000000000002e-06 2024-09-16T00:09:34.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.169 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.088816+0000", 2024-09-16T00:09:34.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016638480000000001 2024-09-16T00:09:34.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.169 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.088981+0000", 2024-09-16T00:09:34.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016522600000000001 2024-09-16T00:09:34.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.170 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.170 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:34.170 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.089128+0000", 2024-09-16T00:09:34.170 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000147441 2024-09-16T00:09:34.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.170 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.170 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.170 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.092831+0000", 2024-09-16T00:09:34.170 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037027319999999998 2024-09-16T00:09:34.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.170 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.170 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.095037+0000", 2024-09-16T00:09:34.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022059979999999998 2024-09-16T00:09:34.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.095100+0000", 2024-09-16T00:09:34.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2401000000000001e-05 2024-09-16T00:09:34.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.095151+0000", 2024-09-16T00:09:34.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1443000000000002e-05 2024-09-16T00:09:34.172 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.172 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.172 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.172 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:276 2.1e e69/19 2:7bf7e7b5:::200.00000006:head v 69'3, mlcod=22'1)", 2024-09-16T00:09:34.172 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:16.493303+0000", 2024-09-16T00:09:34.172 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 17.506310821, 2024-09-16T00:09:34.172 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0062641140000000003, 2024-09-16T00:09:34.172 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.172 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.172 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.172 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.173 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.173 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493303+0000", 2024-09-16T00:09:34.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.173 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.173 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.173 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493303+0000", 2024-09-16T00:09:34.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.173 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.173 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.173 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493305+0000", 2024-09-16T00:09:34.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.573e-06 2024-09-16T00:09:34.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.174 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.174 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.174 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.495058+0000", 2024-09-16T00:09:34.174 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001753183 2024-09-16T00:09:34.174 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.174 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.174 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.174 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.495080+0000", 2024-09-16T00:09:34.174 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1676999999999999e-05 2024-09-16T00:09:34.174 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.174 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.174 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.174 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.495092+0000", 2024-09-16T00:09:34.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2279000000000001e-05 2024-09-16T00:09:34.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.495159+0000", 2024-09-16T00:09:34.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6872000000000006e-05 2024-09-16T00:09:34.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.495178+0000", 2024-09-16T00:09:34.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8963000000000001e-05 2024-09-16T00:09:34.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.499520+0000", 2024-09-16T00:09:34.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043416490000000004 2024-09-16T00:09:34.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.499567+0000", 2024-09-16T00:09:34.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7917999999999998e-05 2024-09-16T00:09:34.176 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.176 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.176 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.177 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:9371 3.3d 3:bf38d834:::10000002b02.00000000:head [write 0~22654] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T00:09:34.177 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:30.164671+0000", 2024-09-16T00:09:34.177 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.8349433149999999, 2024-09-16T00:09:34.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040030400000000002, 2024-09-16T00:09:34.177 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.177 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.177 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.177 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:34.177 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:34.177 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:34.177 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 9371 2024-09-16T00:09:34.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.177 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.164671+0000", 2024-09-16T00:09:34.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.164671+0000", 2024-09-16T00:09:34.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.164674+0000", 2024-09-16T00:09:34.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8449999999999999e-06 2024-09-16T00:09:34.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.179 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.164698+0000", 2024-09-16T00:09:34.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4312999999999999e-05 2024-09-16T00:09:34.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.179 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.164722+0000", 2024-09-16T00:09:34.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4244e-05 2024-09-16T00:09:34.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.179 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.164736+0000", 2024-09-16T00:09:34.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3455000000000001e-05 2024-09-16T00:09:34.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.180 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.164820+0000", 2024-09-16T00:09:34.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4512000000000007e-05 2024-09-16T00:09:34.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op must wait for map", 2024-09-16T00:09:34.180 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.164826+0000", 2024-09-16T00:09:34.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6289999999999998e-06 2024-09-16T00:09:34.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.167082+0000", 2024-09-16T00:09:34.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022562060000000002 2024-09-16T00:09:34.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.167222+0000", 2024-09-16T00:09:34.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014012000000000001 2024-09-16T00:09:34.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:34.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.167329+0000", 2024-09-16T00:09:34.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000106425 2024-09-16T00:09:34.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.168164+0000", 2024-09-16T00:09:34.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00083520199999999995 2024-09-16T00:09:34.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.168623+0000", 2024-09-16T00:09:34.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00045896899999999999 2024-09-16T00:09:34.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.168651+0000", 2024-09-16T00:09:34.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8300999999999999e-05 2024-09-16T00:09:34.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.168674+0000", 2024-09-16T00:09:34.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2819e-05 2024-09-16T00:09:34.183 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.183 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.183 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.183 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:9461 3.6 3:61aad86c:::10000002b6b.00000000:head [write 0~1280] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T00:09:34.183 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:30.579141+0000", 2024-09-16T00:09:34.183 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.4204732670000002, 2024-09-16T00:09:34.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031435539999999998, 2024-09-16T00:09:34.184 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.184 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.184 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.184 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:34.184 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:34.184 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:34.184 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 9461 2024-09-16T00:09:34.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.184 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.579141+0000", 2024-09-16T00:09:34.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.579141+0000", 2024-09-16T00:09:34.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.579142+0000", 2024-09-16T00:09:34.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3289999999999999e-06 2024-09-16T00:09:34.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.186 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.579147+0000", 2024-09-16T00:09:34.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9099999999999996e-06 2024-09-16T00:09:34.186 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.186 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.579158+0000", 2024-09-16T00:09:34.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0419e-05 2024-09-16T00:09:34.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.579164+0000", 2024-09-16T00:09:34.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7279999999999998e-06 2024-09-16T00:09:34.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.579211+0000", 2024-09-16T00:09:34.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6269e-05 2024-09-16T00:09:34.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.579303+0000", 2024-09-16T00:09:34.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2540000000000005e-05 2024-09-16T00:09:34.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:34.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.579390+0000", 2024-09-16T00:09:34.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7058000000000006e-05 2024-09-16T00:09:34.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.582068+0000", 2024-09-16T00:09:34.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026780049999999998 2024-09-16T00:09:34.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.582220+0000", 2024-09-16T00:09:34.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015213100000000001 2024-09-16T00:09:34.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.582255+0000", 2024-09-16T00:09:34.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4449000000000002e-05 2024-09-16T00:09:34.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:30.582284+0000", 2024-09-16T00:09:34.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9716000000000001e-05 2024-09-16T00:09:34.191 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.191 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.191 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.191 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:9671 3.1 3:81efd1de:::10000002c6d.00000000:head [write 0~9928] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T00:09:34.191 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:31.158433+0000", 2024-09-16T00:09:34.191 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 2.841181094, 2024-09-16T00:09:34.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031311590000000001, 2024-09-16T00:09:34.191 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.192 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.192 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.192 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:34.192 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:34.192 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:34.192 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 9671 2024-09-16T00:09:34.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.192 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.192 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.158433+0000", 2024-09-16T00:09:34.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.158433+0000", 2024-09-16T00:09:34.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.158436+0000", 2024-09-16T00:09:34.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9210000000000001e-06 2024-09-16T00:09:34.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.158458+0000", 2024-09-16T00:09:34.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1554999999999999e-05 2024-09-16T00:09:34.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.158483+0000", 2024-09-16T00:09:34.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5282999999999998e-05 2024-09-16T00:09:34.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.158495+0000", 2024-09-16T00:09:34.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2631e-05 2024-09-16T00:09:34.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.158582+0000", 2024-09-16T00:09:34.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6965000000000005e-05 2024-09-16T00:09:34.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.158749+0000", 2024-09-16T00:09:34.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016687700000000001 2024-09-16T00:09:34.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:34.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.158950+0000", 2024-09-16T00:09:34.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020088800000000001 2024-09-16T00:09:34.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.159749+0000", 2024-09-16T00:09:34.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079905000000000004 2024-09-16T00:09:34.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.161447+0000", 2024-09-16T00:09:34.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016981489999999999 2024-09-16T00:09:34.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.161511+0000", 2024-09-16T00:09:34.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3509000000000005e-05 2024-09-16T00:09:34.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.161564+0000", 2024-09-16T00:09:34.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3331000000000002e-05 2024-09-16T00:09:34.197 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.197 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.197 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.197 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:9672 3.3e e77/20 3:7d534130:::10000002c6e.00000000:head v 77'158, mlcod=77'156)", 2024-09-16T00:09:34.197 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:31.164369+0000", 2024-09-16T00:09:34.197 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 2.8352456579999998, 2024-09-16T00:09:34.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0060795850000000002, 2024-09-16T00:09:34.198 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.198 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.198 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.198 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.164369+0000", 2024-09-16T00:09:34.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.164369+0000", 2024-09-16T00:09:34.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.164369+0000", 2024-09-16T00:09:34.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5199999999999995e-07 2024-09-16T00:09:34.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.164378+0000", 2024-09-16T00:09:34.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4300000000000006e-06 2024-09-16T00:09:34.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.164390+0000", 2024-09-16T00:09:34.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1783999999999999e-05 2024-09-16T00:09:34.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.164398+0000", 2024-09-16T00:09:34.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6980000000000004e-06 2024-09-16T00:09:34.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.164503+0000", 2024-09-16T00:09:34.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104243 2024-09-16T00:09:34.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.164522+0000", 2024-09-16T00:09:34.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9749999999999999e-05 2024-09-16T00:09:34.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.170400+0000", 2024-09-16T00:09:34.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058780849999999999 2024-09-16T00:09:34.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.170448+0000", 2024-09-16T00:09:34.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7642999999999998e-05 2024-09-16T00:09:34.201 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.202 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.202 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.202 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.202 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.202 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:34.203 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:09:34.204 INFO:tasks.workunit.client.0.smithi139.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-09-16T00:09:34.204 INFO:tasks.workunit.client.0.smithi139.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-16T00:09:34.205 INFO:tasks.workunit.client.0.smithi139.stderr:+ make 2024-09-16T00:09:34.205 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:34.202+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:34.249 INFO:tasks.workunit.client.0.smithi139.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-09-16T00:09:34.304 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:34.303+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:34.314 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:34.314 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:34.314 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:34.314 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:34.322 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:09:34.405 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:34.404+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:34.429 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:34.429 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:34.429 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:34.429 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:34.429 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:34.437 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:09:34.506 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:34.504+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:34.541 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:34.541 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:15 2.15 2:a93a17c2:::604.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:09:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 115.92444368699999, 2024-09-16T00:09:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043656110000000001, 2024-09-16T00:09:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:09:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:09:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615412+0000", 2024-09-16T00:09:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2199999999999999e-07 2024-09-16T00:09:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615416+0000", 2024-09-16T00:09:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5590000000000001e-06 2024-09-16T00:09:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615423+0000", 2024-09-16T00:09:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6250000000000001e-06 2024-09-16T00:09:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615426+0000", 2024-09-16T00:09:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2540000000000001e-06 2024-09-16T00:09:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616075+0000", 2024-09-16T00:09:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064968800000000004 2024-09-16T00:09:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616142+0000", 2024-09-16T00:09:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6203000000000003e-05 2024-09-16T00:09:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616247+0000", 2024-09-16T00:09:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010572499999999999 2024-09-16T00:09:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618913+0000", 2024-09-16T00:09:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026654339999999999 2024-09-16T00:09:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619712+0000", 2024-09-16T00:09:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00079885199999999996 2024-09-16T00:09:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619744+0000", 2024-09-16T00:09:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2790000000000003e-05 2024-09-16T00:09:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619778+0000", 2024-09-16T00:09:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3059000000000003e-05 2024-09-16T00:09:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:09:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 115.924407608, 2024-09-16T00:09:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087673219999999993, 2024-09-16T00:09:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:09:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:09:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615448+0000", 2024-09-16T00:09:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7e-07 2024-09-16T00:09:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615464+0000", 2024-09-16T00:09:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5392000000000001e-05 2024-09-16T00:09:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615470+0000", 2024-09-16T00:09:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1999999999999999e-06 2024-09-16T00:09:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615474+0000", 2024-09-16T00:09:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7359999999999999e-06 2024-09-16T00:09:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617614+0000", 2024-09-16T00:09:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021398189999999998 2024-09-16T00:09:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617724+0000", 2024-09-16T00:09:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110808 2024-09-16T00:09:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617856+0000", 2024-09-16T00:09:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131676 2024-09-16T00:09:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620819+0000", 2024-09-16T00:09:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029629280000000001 2024-09-16T00:09:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624158+0000", 2024-09-16T00:09:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033386549999999998 2024-09-16T00:09:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624189+0000", 2024-09-16T00:09:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1508999999999999e-05 2024-09-16T00:09:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.624215+0000", 2024-09-16T00:09:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6228999999999999e-05 2024-09-16T00:09:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:09:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 115.924373593, 2024-09-16T00:09:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053285279999999999, 2024-09-16T00:09:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:09:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:09:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615482+0000", 2024-09-16T00:09:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5600000000000002e-07 2024-09-16T00:09:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615484+0000", 2024-09-16T00:09:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.688e-06 2024-09-16T00:09:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615488+0000", 2024-09-16T00:09:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.658e-06 2024-09-16T00:09:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615490+0000", 2024-09-16T00:09:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.728e-06 2024-09-16T00:09:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616441+0000", 2024-09-16T00:09:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00095065500000000001 2024-09-16T00:09:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616539+0000", 2024-09-16T00:09:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8335e-05 2024-09-16T00:09:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616631+0000", 2024-09-16T00:09:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1260000000000004e-05 2024-09-16T00:09:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619837+0000", 2024-09-16T00:09:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003206899 2024-09-16T00:09:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620725+0000", 2024-09-16T00:09:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088713599999999996 2024-09-16T00:09:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620772+0000", 2024-09-16T00:09:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7639999999999998e-05 2024-09-16T00:09:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620811+0000", 2024-09-16T00:09:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8272999999999998e-05 2024-09-16T00:09:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:39 2.6 2:654134d2:::mds0_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-16T00:09:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:09:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 110.899577751, 2024-09-16T00:09:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050072119999999996, 2024-09-16T00:09:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:09:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640278+0000", 2024-09-16T00:09:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640280+0000", 2024-09-16T00:09:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5380000000000001e-06 2024-09-16T00:09:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640296+0000", 2024-09-16T00:09:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5950000000000001e-05 2024-09-16T00:09:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640327+0000", 2024-09-16T00:09:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0309999999999999e-05 2024-09-16T00:09:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.562 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.562 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640340+0000", 2024-09-16T00:09:34.562 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3773e-05 2024-09-16T00:09:34.562 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.562 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.562 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.562 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640422+0000", 2024-09-16T00:09:34.562 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1122e-05 2024-09-16T00:09:34.562 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.562 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.562 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.562 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640572+0000", 2024-09-16T00:09:34.562 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015044099999999999 2024-09-16T00:09:34.562 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.563 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:34.563 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.640691+0000", 2024-09-16T00:09:34.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000119393 2024-09-16T00:09:34.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.563 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.563 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645006+0000", 2024-09-16T00:09:34.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043143779999999998 2024-09-16T00:09:34.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.563 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.563 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645169+0000", 2024-09-16T00:09:34.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016323299999999999 2024-09-16T00:09:34.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.564 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645227+0000", 2024-09-16T00:09:34.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8e-05 2024-09-16T00:09:34.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.564 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:43.645285+0000", 2024-09-16T00:09:34.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8074e-05 2024-09-16T00:09:34.564 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.564 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.564 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.565 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.565 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:85 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e61)", 2024-09-16T00:09:34.565 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:02.658347+0000", 2024-09-16T00:09:34.565 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.881508586999999, 2024-09-16T00:09:34.565 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047582939999999997, 2024-09-16T00:09:34.565 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.565 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.565 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.565 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.565 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.565 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.566 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658347+0000", 2024-09-16T00:09:34.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.566 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.566 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658347+0000", 2024-09-16T00:09:34.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.566 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.566 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658349+0000", 2024-09-16T00:09:34.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7999999999999999e-06 2024-09-16T00:09:34.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658439+0000", 2024-09-16T00:09:34.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0276000000000001e-05 2024-09-16T00:09:34.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658454+0000", 2024-09-16T00:09:34.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5105e-05 2024-09-16T00:09:34.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658462+0000", 2024-09-16T00:09:34.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.525e-06 2024-09-16T00:09:34.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.568 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.568 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658512+0000", 2024-09-16T00:09:34.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9877999999999999e-05 2024-09-16T00:09:34.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.568 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.568 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.658569+0000", 2024-09-16T00:09:34.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7022999999999999e-05 2024-09-16T00:09:34.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:34.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.660742+0000", 2024-09-16T00:09:34.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021734419999999998 2024-09-16T00:09:34.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.662841+0000", 2024-09-16T00:09:34.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020994120000000002 2024-09-16T00:09:34.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.663045+0000", 2024-09-16T00:09:34.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000203422 2024-09-16T00:09:34.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.663079+0000", 2024-09-16T00:09:34.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3769999999999997e-05 2024-09-16T00:09:34.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:02.663105+0000", 2024-09-16T00:09:34.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6641000000000001e-05 2024-09-16T00:09:34.570 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.570 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.570 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.571 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:223 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e65)", 2024-09-16T00:09:34.571 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:11.668302+0000", 2024-09-16T00:09:34.571 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.871554053000001, 2024-09-16T00:09:34.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052512100000000001, 2024-09-16T00:09:34.571 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.571 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.571 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.571 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.571 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.571 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.668302+0000", 2024-09-16T00:09:34.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.668302+0000", 2024-09-16T00:09:34.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.668303+0000", 2024-09-16T00:09:34.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9600000000000008e-07 2024-09-16T00:09:34.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.573 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.668418+0000", 2024-09-16T00:09:34.573 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011578099999999999 2024-09-16T00:09:34.573 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.573 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.573 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.573 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.668430+0000", 2024-09-16T00:09:34.573 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1973e-05 2024-09-16T00:09:34.573 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.573 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.573 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.573 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.668436+0000", 2024-09-16T00:09:34.573 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.682e-06 2024-09-16T00:09:34.573 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.573 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.574 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.668478+0000", 2024-09-16T00:09:34.574 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2233e-05 2024-09-16T00:09:34.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.574 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.668530+0000", 2024-09-16T00:09:34.574 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2054e-05 2024-09-16T00:09:34.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:34.574 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.670743+0000", 2024-09-16T00:09:34.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022122579999999999 2024-09-16T00:09:34.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.673248+0000", 2024-09-16T00:09:34.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025053580000000001 2024-09-16T00:09:34.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.673502+0000", 2024-09-16T00:09:34.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00025403699999999998 2024-09-16T00:09:34.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.673532+0000", 2024-09-16T00:09:34.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9878000000000001e-05 2024-09-16T00:09:34.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.576 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.576 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.576 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:11.673553+0000", 2024-09-16T00:09:34.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0959999999999999e-05 2024-09-16T00:09:34.576 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.576 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.576 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.576 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.576 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:274 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e68)", 2024-09-16T00:09:34.576 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.672085+0000", 2024-09-16T00:09:34.577 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.867770822000001, 2024-09-16T00:09:34.577 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053189819999999999, 2024-09-16T00:09:34.577 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.577 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.577 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.577 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.672085+0000", 2024-09-16T00:09:34.577 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.672085+0000", 2024-09-16T00:09:34.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.672086+0000", 2024-09-16T00:09:34.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1179999999999999e-06 2024-09-16T00:09:34.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.672158+0000", 2024-09-16T00:09:34.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2261000000000005e-05 2024-09-16T00:09:34.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.672171+0000", 2024-09-16T00:09:34.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2510999999999999e-05 2024-09-16T00:09:34.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.672176+0000", 2024-09-16T00:09:34.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5729999999999998e-06 2024-09-16T00:09:34.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.672236+0000", 2024-09-16T00:09:34.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9484999999999997e-05 2024-09-16T00:09:34.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.672320+0000", 2024-09-16T00:09:34.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3908999999999999e-05 2024-09-16T00:09:34.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:34.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.674880+0000", 2024-09-16T00:09:34.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025598970000000002 2024-09-16T00:09:34.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.677164+0000", 2024-09-16T00:09:34.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022845750000000001 2024-09-16T00:09:34.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.581 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.581 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.677334+0000", 2024-09-16T00:09:34.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017007400000000001 2024-09-16T00:09:34.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.581 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.581 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.677364+0000", 2024-09-16T00:09:34.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9526000000000002e-05 2024-09-16T00:09:34.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.581 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.581 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.677404+0000", 2024-09-16T00:09:34.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0052999999999998e-05 2024-09-16T00:09:34.582 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.582 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.582 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.582 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:328 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e74)", 2024-09-16T00:09:34.582 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:21.676780+0000", 2024-09-16T00:09:34.582 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 12.863075697999999, 2024-09-16T00:09:34.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051768939999999996, 2024-09-16T00:09:34.582 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.582 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.582 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.582 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.583 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.676780+0000", 2024-09-16T00:09:34.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.583 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.676780+0000", 2024-09-16T00:09:34.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.583 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.676782+0000", 2024-09-16T00:09:34.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.793e-06 2024-09-16T00:09:34.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.584 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.584 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.676865+0000", 2024-09-16T00:09:34.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3602000000000004e-05 2024-09-16T00:09:34.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.584 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.584 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.676881+0000", 2024-09-16T00:09:34.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5483000000000001e-05 2024-09-16T00:09:34.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.584 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.584 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.676888+0000", 2024-09-16T00:09:34.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2080000000000003e-06 2024-09-16T00:09:34.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.676944+0000", 2024-09-16T00:09:34.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5621999999999997e-05 2024-09-16T00:09:34.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.677002+0000", 2024-09-16T00:09:34.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8449999999999998e-05 2024-09-16T00:09:34.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:34.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.678932+0000", 2024-09-16T00:09:34.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001929676 2024-09-16T00:09:34.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.681197+0000", 2024-09-16T00:09:34.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002265127 2024-09-16T00:09:34.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.681883+0000", 2024-09-16T00:09:34.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00068578900000000002 2024-09-16T00:09:34.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.587 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.681918+0000", 2024-09-16T00:09:34.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5139999999999999e-05 2024-09-16T00:09:34.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.587 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.587 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:21.681957+0000", 2024-09-16T00:09:34.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9004000000000001e-05 2024-09-16T00:09:34.587 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.587 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.587 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.587 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:401 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e75)", 2024-09-16T00:09:34.587 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:24.680116+0000", 2024-09-16T00:09:34.588 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.8597400440000005, 2024-09-16T00:09:34.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004726002, 2024-09-16T00:09:34.588 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.588 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.588 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.588 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.680116+0000", 2024-09-16T00:09:34.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.680116+0000", 2024-09-16T00:09:34.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.680117+0000", 2024-09-16T00:09:34.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1480000000000001e-06 2024-09-16T00:09:34.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.680181+0000", 2024-09-16T00:09:34.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4286000000000001e-05 2024-09-16T00:09:34.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.680191+0000", 2024-09-16T00:09:34.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0426e-05 2024-09-16T00:09:34.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.590 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.680197+0000", 2024-09-16T00:09:34.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2789999999999998e-06 2024-09-16T00:09:34.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.590 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.680250+0000", 2024-09-16T00:09:34.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3053999999999997e-05 2024-09-16T00:09:34.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.680334+0000", 2024-09-16T00:09:34.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3981999999999997e-05 2024-09-16T00:09:34.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:34.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.682946+0000", 2024-09-16T00:09:34.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026117950000000001 2024-09-16T00:09:34.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.684663+0000", 2024-09-16T00:09:34.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001717508 2024-09-16T00:09:34.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.592 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.592 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.684775+0000", 2024-09-16T00:09:34.592 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011200399999999999 2024-09-16T00:09:34.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.592 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.592 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.684809+0000", 2024-09-16T00:09:34.592 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4048999999999999e-05 2024-09-16T00:09:34.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.592 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.592 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:24.684842+0000", 2024-09-16T00:09:34.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2471000000000001e-05 2024-09-16T00:09:34.593 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.593 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.593 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.593 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.593 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:403 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e75)", 2024-09-16T00:09:34.593 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:25.681255+0000", 2024-09-16T00:09:34.593 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.8586005960000005, 2024-09-16T00:09:34.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041662790000000002, 2024-09-16T00:09:34.593 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.593 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.593 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.593 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.593 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.594 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.594 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.681255+0000", 2024-09-16T00:09:34.594 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.594 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.594 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.594 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.594 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.681255+0000", 2024-09-16T00:09:34.594 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.594 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.594 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.594 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.594 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.681256+0000", 2024-09-16T00:09:34.594 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.294e-06 2024-09-16T00:09:34.594 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.595 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.595 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.595 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.681351+0000", 2024-09-16T00:09:34.595 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5043000000000005e-05 2024-09-16T00:09:34.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.595 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.595 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.595 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.681364+0000", 2024-09-16T00:09:34.595 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2829e-05 2024-09-16T00:09:34.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.595 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.595 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.595 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.681372+0000", 2024-09-16T00:09:34.595 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8080000000000005e-06 2024-09-16T00:09:34.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.596 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.596 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.596 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.681412+0000", 2024-09-16T00:09:34.596 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0187e-05 2024-09-16T00:09:34.596 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.596 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.596 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.596 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.681477+0000", 2024-09-16T00:09:34.596 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4836000000000001e-05 2024-09-16T00:09:34.596 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.596 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.596 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:34.596 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.682659+0000", 2024-09-16T00:09:34.596 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011815860000000001 2024-09-16T00:09:34.597 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.597 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.597 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.684317+0000", 2024-09-16T00:09:34.597 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001658269 2024-09-16T00:09:34.597 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.597 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.597 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.685361+0000", 2024-09-16T00:09:34.597 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010441039999999999 2024-09-16T00:09:34.597 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.597 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.597 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.685394+0000", 2024-09-16T00:09:34.598 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2889000000000003e-05 2024-09-16T00:09:34.598 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.598 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.598 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.598 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:25.685421+0000", 2024-09-16T00:09:34.598 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7433999999999999e-05 2024-09-16T00:09:34.598 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.598 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.598 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.598 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.598 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.598 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:488 2.0 e76/19 2:02ac1934:::200.00000009:head v 76'62, mlcod=76'60)", 2024-09-16T00:09:34.598 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:29.731903+0000", 2024-09-16T00:09:34.598 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.8079521969999997, 2024-09-16T00:09:34.599 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041601370000000004, 2024-09-16T00:09:34.599 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.599 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.599 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.599 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.599 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.599 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.731903+0000", 2024-09-16T00:09:34.599 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.599 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.599 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.731903+0000", 2024-09-16T00:09:34.599 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.731904+0000", 2024-09-16T00:09:34.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0800000000000004e-07 2024-09-16T00:09:34.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.733231+0000", 2024-09-16T00:09:34.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013272780000000001 2024-09-16T00:09:34.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.733248+0000", 2024-09-16T00:09:34.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6526999999999999e-05 2024-09-16T00:09:34.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.601 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.601 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.733256+0000", 2024-09-16T00:09:34.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4579999999999993e-06 2024-09-16T00:09:34.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.601 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.601 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.733321+0000", 2024-09-16T00:09:34.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4614000000000002e-05 2024-09-16T00:09:34.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.733345+0000", 2024-09-16T00:09:34.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3808e-05 2024-09-16T00:09:34.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.735988+0000", 2024-09-16T00:09:34.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026435389999999999 2024-09-16T00:09:34.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.736064+0000", 2024-09-16T00:09:34.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5104999999999993e-05 2024-09-16T00:09:34.603 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.603 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.603 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.603 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:9672 3.3e 3:7d534130:::10000002c6e.00000000:head [write 0~459] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-16T00:09:34.603 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:31.162774+0000", 2024-09-16T00:09:34.603 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.3770817470000001, 2024-09-16T00:09:34.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0079216349999999998, 2024-09-16T00:09:34.603 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.603 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.603 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.603 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:34.604 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:34.604 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:34.604 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 9672 2024-09-16T00:09:34.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.604 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.604 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.604 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.162774+0000", 2024-09-16T00:09:34.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.604 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.604 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.162774+0000", 2024-09-16T00:09:34.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.162776+0000", 2024-09-16T00:09:34.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.148e-06 2024-09-16T00:09:34.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.162787+0000", 2024-09-16T00:09:34.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0710000000000001e-05 2024-09-16T00:09:34.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.162808+0000", 2024-09-16T00:09:34.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1582000000000001e-05 2024-09-16T00:09:34.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.162820+0000", 2024-09-16T00:09:34.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1446e-05 2024-09-16T00:09:34.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.162895+0000", 2024-09-16T00:09:34.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5013999999999997e-05 2024-09-16T00:09:34.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.607 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.163050+0000", 2024-09-16T00:09:34.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015527600000000001 2024-09-16T00:09:34.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.607 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:34.607 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.163188+0000", 2024-09-16T00:09:34.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013749300000000001 2024-09-16T00:09:34.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.607 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.607 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.164139+0000", 2024-09-16T00:09:34.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00095196500000000004 2024-09-16T00:09:34.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.608 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.170610+0000", 2024-09-16T00:09:34.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0064700579999999999 2024-09-16T00:09:34.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.608 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.170647+0000", 2024-09-16T00:09:34.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7599999999999999e-05 2024-09-16T00:09:34.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.608 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:31.170695+0000", 2024-09-16T00:09:34.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8343000000000001e-05 2024-09-16T00:09:34.608 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.609 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.609 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.609 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:508 2.2 e80/19 2:4272d415:::200.0000000b:head v 80'16, mlcod=80'11)", 2024-09-16T00:09:34.609 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:33.656735+0000", 2024-09-16T00:09:34.609 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.88312108199999995, 2024-09-16T00:09:34.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057223270000000001, 2024-09-16T00:09:34.609 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.609 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.609 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.609 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.656735+0000", 2024-09-16T00:09:34.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.656735+0000", 2024-09-16T00:09:34.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.656735+0000", 2024-09-16T00:09:34.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.44e-07 2024-09-16T00:09:34.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.656741+0000", 2024-09-16T00:09:34.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.553e-06 2024-09-16T00:09:34.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.656756+0000", 2024-09-16T00:09:34.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5342e-05 2024-09-16T00:09:34.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.656763+0000", 2024-09-16T00:09:34.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6270000000000002e-06 2024-09-16T00:09:34.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.612 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.656852+0000", 2024-09-16T00:09:34.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9711999999999998e-05 2024-09-16T00:09:34.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.612 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.612 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.656870+0000", 2024-09-16T00:09:34.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7977000000000001e-05 2024-09-16T00:09:34.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.612 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.612 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.662407+0000", 2024-09-16T00:09:34.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055366779999999997 2024-09-16T00:09:34.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.613 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.613 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.662457+0000", 2024-09-16T00:09:34.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9793999999999998e-05 2024-09-16T00:09:34.613 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.613 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.613 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.613 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:509 2.2 e80/19 2:4272d415:::200.0000000b:head v 80'17, mlcod=80'12)", 2024-09-16T00:09:34.613 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:33.657235+0000", 2024-09-16T00:09:34.613 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.88262075699999998, 2024-09-16T00:09:34.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052540260000000002, 2024-09-16T00:09:34.613 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.613 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.614 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.614 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.657235+0000", 2024-09-16T00:09:34.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.657235+0000", 2024-09-16T00:09:34.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.657236+0000", 2024-09-16T00:09:34.615 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8100000000000002e-07 2024-09-16T00:09:34.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.615 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.615 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.657242+0000", 2024-09-16T00:09:34.615 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1310000000000003e-06 2024-09-16T00:09:34.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.615 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.615 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.657254+0000", 2024-09-16T00:09:34.615 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2132e-05 2024-09-16T00:09:34.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.615 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.657261+0000", 2024-09-16T00:09:34.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3810000000000001e-06 2024-09-16T00:09:34.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.657294+0000", 2024-09-16T00:09:34.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2827999999999998e-05 2024-09-16T00:09:34.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.657306+0000", 2024-09-16T00:09:34.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1484999999999999e-05 2024-09-16T00:09:34.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.662460+0000", 2024-09-16T00:09:34.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051547140000000003 2024-09-16T00:09:34.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.662489+0000", 2024-09-16T00:09:34.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8574000000000001e-05 2024-09-16T00:09:34.617 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.617 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.617 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.618 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:510 2.2 e80/19 2:4272d415:::200.0000000b:head v 80'18, mlcod=80'12)", 2024-09-16T00:09:34.618 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:33.657670+0000", 2024-09-16T00:09:34.618 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.88218563000000005, 2024-09-16T00:09:34.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0048597079999999999, 2024-09-16T00:09:34.618 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.618 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.618 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.618 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.618 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.618 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.657670+0000", 2024-09-16T00:09:34.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.618 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.619 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.657670+0000", 2024-09-16T00:09:34.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.619 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.619 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.657671+0000", 2024-09-16T00:09:34.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9699999999999995e-07 2024-09-16T00:09:34.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.619 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.619 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.657679+0000", 2024-09-16T00:09:34.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1790000000000008e-06 2024-09-16T00:09:34.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.657692+0000", 2024-09-16T00:09:34.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.322e-05 2024-09-16T00:09:34.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.657699+0000", 2024-09-16T00:09:34.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0790000000000003e-06 2024-09-16T00:09:34.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.657735+0000", 2024-09-16T00:09:34.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5398000000000002e-05 2024-09-16T00:09:34.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.657747+0000", 2024-09-16T00:09:34.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2075999999999999e-05 2024-09-16T00:09:34.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.662491+0000", 2024-09-16T00:09:34.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004744018 2024-09-16T00:09:34.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.662530+0000", 2024-09-16T00:09:34.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8940999999999999e-05 2024-09-16T00:09:34.622 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.622 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.622 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.622 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:511 2.2 e80/19 2:4272d415:::200.0000000b:head v 80'19, mlcod=80'13)", 2024-09-16T00:09:34.622 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:33.658283+0000", 2024-09-16T00:09:34.622 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.88157305799999996, 2024-09-16T00:09:34.622 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042719639999999996, 2024-09-16T00:09:34.622 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.622 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.622 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.622 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.658283+0000", 2024-09-16T00:09:34.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.658283+0000", 2024-09-16T00:09:34.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.658283+0000", 2024-09-16T00:09:34.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3200000000000004e-07 2024-09-16T00:09:34.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.624 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.658290+0000", 2024-09-16T00:09:34.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4899999999999997e-06 2024-09-16T00:09:34.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.624 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.658302+0000", 2024-09-16T00:09:34.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2367e-05 2024-09-16T00:09:34.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.624 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.658309+0000", 2024-09-16T00:09:34.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8000000000000001e-06 2024-09-16T00:09:34.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.625 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.658368+0000", 2024-09-16T00:09:34.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8798000000000002e-05 2024-09-16T00:09:34.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.625 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.658386+0000", 2024-09-16T00:09:34.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8230999999999999e-05 2024-09-16T00:09:34.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.662531+0000", 2024-09-16T00:09:34.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041452659999999999 2024-09-16T00:09:34.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.626 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.662555+0000", 2024-09-16T00:09:34.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3280000000000001e-05 2024-09-16T00:09:34.626 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.626 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.626 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.626 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:518 2.6 2:654134d2:::mds0_openfiles.0:head [omap-set-header,omap-set-vals,omap-rm-keys] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e80)", 2024-09-16T00:09:34.626 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:33.690263+0000", 2024-09-16T00:09:34.627 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.84959279600000004, 2024-09-16T00:09:34.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047251760000000002, 2024-09-16T00:09:34.627 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.627 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.627 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.627 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.627 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.690263+0000", 2024-09-16T00:09:34.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.690263+0000", 2024-09-16T00:09:34.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.690264+0000", 2024-09-16T00:09:34.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4449999999999999e-06 2024-09-16T00:09:34.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.690357+0000", 2024-09-16T00:09:34.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2712000000000003e-05 2024-09-16T00:09:34.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.629 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.690372+0000", 2024-09-16T00:09:34.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4844000000000001e-05 2024-09-16T00:09:34.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.629 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.690379+0000", 2024-09-16T00:09:34.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.435e-06 2024-09-16T00:09:34.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.629 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.690443+0000", 2024-09-16T00:09:34.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3713999999999993e-05 2024-09-16T00:09:34.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.630 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.690530+0000", 2024-09-16T00:09:34.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7206000000000005e-05 2024-09-16T00:09:34.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:34.630 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.692519+0000", 2024-09-16T00:09:34.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019890189999999999 2024-09-16T00:09:34.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:34.631 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.694775+0000", 2024-09-16T00:09:34.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022557380000000002 2024-09-16T00:09:34.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.631 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:34.631 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.694871+0000", 2024-09-16T00:09:34.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6019000000000003e-05 2024-09-16T00:09:34.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.631 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.631 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.694940+0000", 2024-09-16T00:09:34.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9158000000000004e-05 2024-09-16T00:09:34.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.632 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.632 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.694988+0000", 2024-09-16T00:09:34.632 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7886000000000002e-05 2024-09-16T00:09:34.632 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.632 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.632 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.632 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:526 2.2 e80/19 2:4272d415:::200.0000000b:head v 80'33, mlcod=80'30)", 2024-09-16T00:09:34.632 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:33.869861+0000", 2024-09-16T00:09:34.632 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.66999494000000004, 2024-09-16T00:09:34.632 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0046082060000000001, 2024-09-16T00:09:34.632 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.633 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.633 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.633 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.869861+0000", 2024-09-16T00:09:34.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.869861+0000", 2024-09-16T00:09:34.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.634 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.869861+0000", 2024-09-16T00:09:34.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6199999999999998e-07 2024-09-16T00:09:34.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.634 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.869868+0000", 2024-09-16T00:09:34.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.348e-06 2024-09-16T00:09:34.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.634 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.869876+0000", 2024-09-16T00:09:34.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0369999999999995e-06 2024-09-16T00:09:34.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.869879+0000", 2024-09-16T00:09:34.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.873e-06 2024-09-16T00:09:34.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.869906+0000", 2024-09-16T00:09:34.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6956000000000001e-05 2024-09-16T00:09:34.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.869918+0000", 2024-09-16T00:09:34.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1402e-05 2024-09-16T00:09:34.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.874442+0000", 2024-09-16T00:09:34.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045244680000000002 2024-09-16T00:09:34.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.874469+0000", 2024-09-16T00:09:34.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.656e-05 2024-09-16T00:09:34.636 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.636 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.636 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.637 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:527 2.2 e80/19 2:4272d415:::200.0000000b:head v 80'34, mlcod=80'30)", 2024-09-16T00:09:34.637 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:33.870276+0000", 2024-09-16T00:09:34.637 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.66957979099999998, 2024-09-16T00:09:34.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042096850000000003, 2024-09-16T00:09:34.637 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.637 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.637 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.637 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.870276+0000", 2024-09-16T00:09:34.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.870276+0000", 2024-09-16T00:09:34.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.870276+0000", 2024-09-16T00:09:34.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9999999999999997e-07 2024-09-16T00:09:34.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.870281+0000", 2024-09-16T00:09:34.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.561e-06 2024-09-16T00:09:34.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.870291+0000", 2024-09-16T00:09:34.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5380000000000008e-06 2024-09-16T00:09:34.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.870295+0000", 2024-09-16T00:09:34.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4610000000000001e-06 2024-09-16T00:09:34.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.870335+0000", 2024-09-16T00:09:34.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9944000000000002e-05 2024-09-16T00:09:34.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.870353+0000", 2024-09-16T00:09:34.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7609999999999999e-05 2024-09-16T00:09:34.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.874470+0000", 2024-09-16T00:09:34.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041175150000000004 2024-09-16T00:09:34.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.874485+0000", 2024-09-16T00:09:34.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5455999999999999e-05 2024-09-16T00:09:34.641 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.641 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.641 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.641 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:533 2.2 e80/19 2:4272d415:::200.0000000b:head v 80'40, mlcod=80'36)", 2024-09-16T00:09:34.641 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:33.890926+0000", 2024-09-16T00:09:34.641 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.64892925999999995, 2024-09-16T00:09:34.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050365949999999996, 2024-09-16T00:09:34.641 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:34.641 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:34.641 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:34.642 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:34.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:34.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.890926+0000", 2024-09-16T00:09:34.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:34.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.890926+0000", 2024-09-16T00:09:34.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:34.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:34.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.890927+0000", 2024-09-16T00:09:34.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4700000000000001e-07 2024-09-16T00:09:34.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:34.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.890933+0000", 2024-09-16T00:09:34.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8799999999999996e-06 2024-09-16T00:09:34.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:34.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.890943+0000", 2024-09-16T00:09:34.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0359e-05 2024-09-16T00:09:34.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:34.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.890949+0000", 2024-09-16T00:09:34.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9510000000000003e-06 2024-09-16T00:09:34.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:34.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.890997+0000", 2024-09-16T00:09:34.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7892999999999997e-05 2024-09-16T00:09:34.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:34.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.891016+0000", 2024-09-16T00:09:34.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8881000000000001e-05 2024-09-16T00:09:34.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:34.645 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.895936+0000", 2024-09-16T00:09:34.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0049201189999999997 2024-09-16T00:09:34.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:34.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:34.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:34.645 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:33.895963+0000", 2024-09-16T00:09:34.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6865000000000001e-05 2024-09-16T00:09:34.645 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.645 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.645 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.645 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:34.645 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:34.646 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:34.647 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:09:34.648 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:34.646+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:34.748 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:34.747+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:34.753 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:34.753 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:34.753 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:34.753 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:34.762 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:09:34.850 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:34.848+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:34.868 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:34.869 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:34.869 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:34.869 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:34.869 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:34.878 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:09:34.951 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:34.949+0000 7f04ec18f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 36663) UID: 0 2024-09-16T00:09:35.026 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:35.026 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:09:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 116.40900955799999, 2024-09-16T00:09:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038263220000000001, 2024-09-16T00:09:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.027 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:09:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615460+0000", 2024-09-16T00:09:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615465+0000", 2024-09-16T00:09:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2769999999999999e-06 2024-09-16T00:09:35.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615471+0000", 2024-09-16T00:09:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5729999999999996e-06 2024-09-16T00:09:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615483+0000", 2024-09-16T00:09:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1838000000000001e-05 2024-09-16T00:09:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615492+0000", 2024-09-16T00:09:35.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2410000000000001e-06 2024-09-16T00:09:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615596+0000", 2024-09-16T00:09:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104097 2024-09-16T00:09:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615688+0000", 2024-09-16T00:09:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1225999999999997e-05 2024-09-16T00:09:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:35.030 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615804+0000", 2024-09-16T00:09:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000116601 2024-09-16T00:09:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618165+0000", 2024-09-16T00:09:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023609429999999999 2024-09-16T00:09:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619191+0000", 2024-09-16T00:09:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010258509999999999 2024-09-16T00:09:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.031 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619243+0000", 2024-09-16T00:09:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2327999999999998e-05 2024-09-16T00:09:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.619287+0000", 2024-09-16T00:09:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3346999999999997e-05 2024-09-16T00:09:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.032 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 116.408954482, 2024-09-16T00:09:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0054735510000000001, 2024-09-16T00:09:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.033 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615516+0000", 2024-09-16T00:09:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6899999999999998e-07 2024-09-16T00:09:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615519+0000", 2024-09-16T00:09:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1760000000000002e-06 2024-09-16T00:09:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615526+0000", 2024-09-16T00:09:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8399999999999997e-06 2024-09-16T00:09:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615531+0000", 2024-09-16T00:09:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4450000000000004e-06 2024-09-16T00:09:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:09:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054127099999999996 2024-09-16T00:09:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616140+0000", 2024-09-16T00:09:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7230999999999994e-05 2024-09-16T00:09:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616249+0000", 2024-09-16T00:09:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109122 2024-09-16T00:09:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618646+0000", 2024-09-16T00:09:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023972279999999999 2024-09-16T00:09:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:09:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022396220000000001 2024-09-16T00:09:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620939+0000", 2024-09-16T00:09:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2735999999999998e-05 2024-09-16T00:09:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620989+0000", 2024-09-16T00:09:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0411000000000002e-05 2024-09-16T00:09:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 116.408929003, 2024-09-16T00:09:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055872789999999997, 2024-09-16T00:09:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.01e-07 2024-09-16T00:09:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615547+0000", 2024-09-16T00:09:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7899999999999996e-06 2024-09-16T00:09:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:09:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1119999999999998e-06 2024-09-16T00:09:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615560+0000", 2024-09-16T00:09:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:09:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616424+0000", 2024-09-16T00:09:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086350800000000003 2024-09-16T00:09:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:09:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9675999999999999e-05 2024-09-16T00:09:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616621+0000", 2024-09-16T00:09:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7345999999999999e-05 2024-09-16T00:09:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618654+0000", 2024-09-16T00:09:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020335269999999998 2024-09-16T00:09:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621055+0000", 2024-09-16T00:09:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024009729999999998 2024-09-16T00:09:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621100+0000", 2024-09-16T00:09:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5402999999999999e-05 2024-09-16T00:09:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621128+0000", 2024-09-16T00:09:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7736e-05 2024-09-16T00:09:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 116.408903353, 2024-09-16T00:09:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057024509999999999, 2024-09-16T00:09:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6300000000000001e-07 2024-09-16T00:09:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615569+0000", 2024-09-16T00:09:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3709999999999998e-06 2024-09-16T00:09:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615580+0000", 2024-09-16T00:09:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0448000000000001e-05 2024-09-16T00:09:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615584+0000", 2024-09-16T00:09:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6020000000000002e-06 2024-09-16T00:09:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616850+0000", 2024-09-16T00:09:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012661110000000001 2024-09-16T00:09:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616915+0000", 2024-09-16T00:09:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4820000000000006e-05 2024-09-16T00:09:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617031+0000", 2024-09-16T00:09:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115258 2024-09-16T00:09:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618659+0000", 2024-09-16T00:09:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001627975 2024-09-16T00:09:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621179+0000", 2024-09-16T00:09:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025203410000000002 2024-09-16T00:09:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621238+0000", 2024-09-16T00:09:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9323000000000001e-05 2024-09-16T00:09:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621269+0000", 2024-09-16T00:09:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0839000000000001e-05 2024-09-16T00:09:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:09:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 116.40887960400001, 2024-09-16T00:09:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058012769999999996, 2024-09-16T00:09:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:09:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:09:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615591+0000", 2024-09-16T00:09:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6199999999999999e-07 2024-09-16T00:09:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615594+0000", 2024-09-16T00:09:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8739999999999999e-06 2024-09-16T00:09:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615599+0000", 2024-09-16T00:09:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:09:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615603+0000", 2024-09-16T00:09:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8489999999999999e-06 2024-09-16T00:09:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617235+0000", 2024-09-16T00:09:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001632037 2024-09-16T00:09:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617348+0000", 2024-09-16T00:09:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113387 2024-09-16T00:09:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617458+0000", 2024-09-16T00:09:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109765 2024-09-16T00:09:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618664+0000", 2024-09-16T00:09:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012059950000000001 2024-09-16T00:09:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621311+0000", 2024-09-16T00:09:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026468469999999999 2024-09-16T00:09:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621357+0000", 2024-09-16T00:09:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6360999999999999e-05 2024-09-16T00:09:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621392+0000", 2024-09-16T00:09:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4190999999999998e-05 2024-09-16T00:09:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 116.40886098, 2024-09-16T00:09:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058737279999999999, 2024-09-16T00:09:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3799999999999999e-07 2024-09-16T00:09:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615611+0000", 2024-09-16T00:09:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-06 2024-09-16T00:09:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615626+0000", 2024-09-16T00:09:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:09:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615634+0000", 2024-09-16T00:09:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-06 2024-09-16T00:09:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617692+0000", 2024-09-16T00:09:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002057718 2024-09-16T00:09:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617768+0000", 2024-09-16T00:09:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6878999999999994e-05 2024-09-16T00:09:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:09:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019501699999999999 2024-09-16T00:09:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:09:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034357200000000002 2024-09-16T00:09:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621429+0000", 2024-09-16T00:09:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0012999999999999e-05 2024-09-16T00:09:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621459+0000", 2024-09-16T00:09:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9498000000000001e-05 2024-09-16T00:09:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621483+0000", 2024-09-16T00:09:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3957000000000001e-05 2024-09-16T00:09:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:09:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 116.4088297, 2024-09-16T00:09:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059371789999999999, 2024-09-16T00:09:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:09:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:09:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:09:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4000000000000002e-07 2024-09-16T00:09:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615643+0000", 2024-09-16T00:09:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6860000000000002e-06 2024-09-16T00:09:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615649+0000", 2024-09-16T00:09:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7610000000000004e-06 2024-09-16T00:09:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615653+0000", 2024-09-16T00:09:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0380000000000001e-06 2024-09-16T00:09:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618197+0000", 2024-09-16T00:09:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025440079999999999 2024-09-16T00:09:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618312+0000", 2024-09-16T00:09:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115083 2024-09-16T00:09:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618412+0000", 2024-09-16T00:09:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010016799999999999 2024-09-16T00:09:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621518+0000", 2024-09-16T00:09:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031058679999999999 2024-09-16T00:09:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621526+0000", 2024-09-16T00:09:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2490000000000004e-06 2024-09-16T00:09:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621549+0000", 2024-09-16T00:09:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3686999999999999e-05 2024-09-16T00:09:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621577+0000", 2024-09-16T00:09:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8191000000000002e-05 2024-09-16T00:09:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 116.40881203399999, 2024-09-16T00:09:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005989622, 2024-09-16T00:09:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3799999999999998e-07 2024-09-16T00:09:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615660+0000", 2024-09-16T00:09:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9740000000000001e-06 2024-09-16T00:09:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615666+0000", 2024-09-16T00:09:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3499999999999996e-06 2024-09-16T00:09:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615671+0000", 2024-09-16T00:09:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.13e-06 2024-09-16T00:09:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618759+0000", 2024-09-16T00:09:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030880500000000002 2024-09-16T00:09:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618832+0000", 2024-09-16T00:09:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3725000000000003e-05 2024-09-16T00:09:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618958+0000", 2024-09-16T00:09:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012587899999999999 2024-09-16T00:09:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621583+0000", 2024-09-16T00:09:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026246469999999999 2024-09-16T00:09:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621602+0000", 2024-09-16T00:09:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9137e-05 2024-09-16T00:09:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621629+0000", 2024-09-16T00:09:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6451000000000001e-05 2024-09-16T00:09:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621648+0000", 2024-09-16T00:09:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8941e-05 2024-09-16T00:09:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 116.408794415, 2024-09-16T00:09:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053141380000000004, 2024-09-16T00:09:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-07 2024-09-16T00:09:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615678+0000", 2024-09-16T00:09:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:09:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615682+0000", 2024-09-16T00:09:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6659999999999999e-06 2024-09-16T00:09:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615686+0000", 2024-09-16T00:09:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6990000000000001e-06 2024-09-16T00:09:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615736+0000", 2024-09-16T00:09:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9656e-05 2024-09-16T00:09:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615864+0000", 2024-09-16T00:09:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012836100000000001 2024-09-16T00:09:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615989+0000", 2024-09-16T00:09:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000125525 2024-09-16T00:09:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617355+0000", 2024-09-16T00:09:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001365982 2024-09-16T00:09:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620897+0000", 2024-09-16T00:09:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035419610000000002 2024-09-16T00:09:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620941+0000", 2024-09-16T00:09:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3998999999999997e-05 2024-09-16T00:09:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620990+0000", 2024-09-16T00:09:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8199999999999999e-05 2024-09-16T00:09:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 116.408767157, 2024-09-16T00:09:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006316449, 2024-09-16T00:09:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3299999999999998e-07 2024-09-16T00:09:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615705+0000", 2024-09-16T00:09:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.278e-06 2024-09-16T00:09:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615711+0000", 2024-09-16T00:09:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8119999999999996e-06 2024-09-16T00:09:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615716+0000", 2024-09-16T00:09:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4909999999999999e-06 2024-09-16T00:09:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617551+0000", 2024-09-16T00:09:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00183549 2024-09-16T00:09:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617627+0000", 2024-09-16T00:09:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6163999999999998e-05 2024-09-16T00:09:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617776+0000", 2024-09-16T00:09:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014909499999999999 2024-09-16T00:09:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621072+0000", 2024-09-16T00:09:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032955150000000002 2024-09-16T00:09:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621957+0000", 2024-09-16T00:09:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088553199999999996 2024-09-16T00:09:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621998+0000", 2024-09-16T00:09:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0815000000000001e-05 2024-09-16T00:09:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:09:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0924000000000002e-05 2024-09-16T00:09:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:584 3.22 3:47e2f9af:::10000000492.00000000:head [write 0~10710] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.989570+0000", 2024-09-16T00:09:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.034899574000001, 2024-09-16T00:09:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038063620000000002, 2024-09-16T00:09:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 584 2024-09-16T00:09:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989570+0000", 2024-09-16T00:09:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989570+0000", 2024-09-16T00:09:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989573+0000", 2024-09-16T00:09:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9909999999999998e-06 2024-09-16T00:09:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989602+0000", 2024-09-16T00:09:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8353000000000001e-05 2024-09-16T00:09:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989622+0000", 2024-09-16T00:09:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9905999999999999e-05 2024-09-16T00:09:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989631+0000", 2024-09-16T00:09:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6360000000000006e-06 2024-09-16T00:09:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989706+0000", 2024-09-16T00:09:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4493000000000007e-05 2024-09-16T00:09:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.989856+0000", 2024-09-16T00:09:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015023700000000001 2024-09-16T00:09:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.990004+0000", 2024-09-16T00:09:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000148157 2024-09-16T00:09:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.089 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.992924+0000", 2024-09-16T00:09:35.089 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029203250000000001 2024-09-16T00:09:35.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.089 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.993270+0000", 2024-09-16T00:09:35.089 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00034548899999999998 2024-09-16T00:09:35.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.993325+0000", 2024-09-16T00:09:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5337999999999998e-05 2024-09-16T00:09:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.993377+0000", 2024-09-16T00:09:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1437000000000002e-05 2024-09-16T00:09:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:714 3.22 3:453cbc5d:::10000000536.00000000:head [write 0~260] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.356475+0000", 2024-09-16T00:09:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.667995275999999, 2024-09-16T00:09:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003898926, 2024-09-16T00:09:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 714 2024-09-16T00:09:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356475+0000", 2024-09-16T00:09:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356475+0000", 2024-09-16T00:09:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356476+0000", 2024-09-16T00:09:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6750000000000001e-06 2024-09-16T00:09:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356493+0000", 2024-09-16T00:09:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6562000000000002e-05 2024-09-16T00:09:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356509+0000", 2024-09-16T00:09:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5908000000000001e-05 2024-09-16T00:09:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356516+0000", 2024-09-16T00:09:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9079999999999998e-06 2024-09-16T00:09:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356573+0000", 2024-09-16T00:09:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7243000000000001e-05 2024-09-16T00:09:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356657+0000", 2024-09-16T00:09:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3809999999999999e-05 2024-09-16T00:09:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356742+0000", 2024-09-16T00:09:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5704000000000005e-05 2024-09-16T00:09:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360000+0000", 2024-09-16T00:09:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003257916 2024-09-16T00:09:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360279+0000", 2024-09-16T00:09:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00027872500000000001 2024-09-16T00:09:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360324+0000", 2024-09-16T00:09:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5359999999999999e-05 2024-09-16T00:09:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360374+0000", 2024-09-16T00:09:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9115e-05 2024-09-16T00:09:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:766 3.1f 3:f8970b5b:::10000000571.00000000:head [write 0~1989] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.481586+0000", 2024-09-16T00:09:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.542883621000001, 2024-09-16T00:09:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038531529999999998, 2024-09-16T00:09:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 766 2024-09-16T00:09:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481586+0000", 2024-09-16T00:09:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481586+0000", 2024-09-16T00:09:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481588+0000", 2024-09-16T00:09:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.691e-06 2024-09-16T00:09:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481595+0000", 2024-09-16T00:09:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5780000000000004e-06 2024-09-16T00:09:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481605+0000", 2024-09-16T00:09:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0729e-05 2024-09-16T00:09:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481610+0000", 2024-09-16T00:09:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:09:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481663+0000", 2024-09-16T00:09:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2429999999999998e-05 2024-09-16T00:09:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481740+0000", 2024-09-16T00:09:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7526000000000003e-05 2024-09-16T00:09:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481829+0000", 2024-09-16T00:09:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8265999999999998e-05 2024-09-16T00:09:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.482578+0000", 2024-09-16T00:09:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074936600000000001 2024-09-16T00:09:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.485314+0000", 2024-09-16T00:09:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027358949999999999 2024-09-16T00:09:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.485398+0000", 2024-09-16T00:09:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4462000000000005e-05 2024-09-16T00:09:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.485439+0000", 2024-09-16T00:09:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1066999999999998e-05 2024-09-16T00:09:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:2315 3.1f 3:f84a2a2d:::10000000c60.00000000:head [write 0~648229] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T00:09:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.440012+0000", 2024-09-16T00:09:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.584457578999999, 2024-09-16T00:09:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038521150000000001, 2024-09-16T00:09:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 2315 2024-09-16T00:09:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440012+0000", 2024-09-16T00:09:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440012+0000", 2024-09-16T00:09:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440014+0000", 2024-09-16T00:09:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5689999999999999e-06 2024-09-16T00:09:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440415+0000", 2024-09-16T00:09:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00040101999999999998 2024-09-16T00:09:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440434+0000", 2024-09-16T00:09:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8833000000000001e-05 2024-09-16T00:09:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440444+0000", 2024-09-16T00:09:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0552e-05 2024-09-16T00:09:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440508+0000", 2024-09-16T00:09:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4156e-05 2024-09-16T00:09:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440610+0000", 2024-09-16T00:09:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101584 2024-09-16T00:09:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.440698+0000", 2024-09-16T00:09:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7824999999999993e-05 2024-09-16T00:09:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.443176+0000", 2024-09-16T00:09:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024776379999999999 2024-09-16T00:09:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.443789+0000", 2024-09-16T00:09:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00061301699999999999 2024-09-16T00:09:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.443820+0000", 2024-09-16T00:09:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1392e-05 2024-09-16T00:09:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.443864+0000", 2024-09-16T00:09:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4529e-05 2024-09-16T00:09:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5850 3.3a 3:5c3cfc2b:::10000001adf.00000000:head [write 0~116] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:09:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.608533223999999, 2024-09-16T00:09:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66931295800000001, 2024-09-16T00:09:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5850 2024-09-16T00:09:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:09:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:09:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415938+0000", 2024-09-16T00:09:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7099999999999999e-06 2024-09-16T00:09:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415945+0000", 2024-09-16T00:09:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:09:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415959+0000", 2024-09-16T00:09:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3545e-05 2024-09-16T00:09:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415966+0000", 2024-09-16T00:09:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6379999999999998e-06 2024-09-16T00:09:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416013+0000", 2024-09-16T00:09:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7314999999999997e-05 2024-09-16T00:09:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416020+0000", 2024-09-16T00:09:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.52e-06 2024-09-16T00:09:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082393+0000", 2024-09-16T00:09:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66637251799999997 2024-09-16T00:09:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082537+0000", 2024-09-16T00:09:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014418800000000001 2024-09-16T00:09:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082689+0000", 2024-09-16T00:09:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015198499999999999 2024-09-16T00:09:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084930+0000", 2024-09-16T00:09:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002241041 2024-09-16T00:09:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084994+0000", 2024-09-16T00:09:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3274000000000004e-05 2024-09-16T00:09:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085049+0000", 2024-09-16T00:09:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.554e-05 2024-09-16T00:09:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085250+0000", 2024-09-16T00:09:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020060799999999999 2024-09-16T00:09:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5910 3.3a 3:5ec98a62:::10000001b22.00000000:head [write 0~1274] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:09:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.461589585999999, 2024-09-16T00:09:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.52286026600000002, 2024-09-16T00:09:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5910 2024-09-16T00:09:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:09:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:09:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562881+0000", 2024-09-16T00:09:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:09:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562887+0000", 2024-09-16T00:09:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2870000000000004e-06 2024-09-16T00:09:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562897+0000", 2024-09-16T00:09:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.064e-05 2024-09-16T00:09:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562902+0000", 2024-09-16T00:09:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2340000000000002e-06 2024-09-16T00:09:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562958+0000", 2024-09-16T00:09:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5523999999999999e-05 2024-09-16T00:09:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562968+0000", 2024-09-16T00:09:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0394000000000001e-05 2024-09-16T00:09:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083461+0000", 2024-09-16T00:09:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.520492968 2024-09-16T00:09:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083645+0000", 2024-09-16T00:09:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018346399999999999 2024-09-16T00:09:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083844+0000", 2024-09-16T00:09:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000199268 2024-09-16T00:09:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085410+0000", 2024-09-16T00:09:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015662359999999999 2024-09-16T00:09:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085493+0000", 2024-09-16T00:09:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2478999999999994e-05 2024-09-16T00:09:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085563+0000", 2024-09-16T00:09:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0159000000000004e-05 2024-09-16T00:09:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085741+0000", 2024-09-16T00:09:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017764300000000001 2024-09-16T00:09:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5962 3.3a 3:5c7b4d29:::10000001b5d.00000000:head [write 0~3074] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:09:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.332052769000001, 2024-09-16T00:09:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39368737500000001, 2024-09-16T00:09:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5962 2024-09-16T00:09:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:09:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:09:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692418+0000", 2024-09-16T00:09:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3510000000000001e-06 2024-09-16T00:09:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692425+0000", 2024-09-16T00:09:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3770000000000001e-06 2024-09-16T00:09:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692440+0000", 2024-09-16T00:09:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4975e-05 2024-09-16T00:09:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692448+0000", 2024-09-16T00:09:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6279999999999994e-06 2024-09-16T00:09:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692499+0000", 2024-09-16T00:09:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0872999999999999e-05 2024-09-16T00:09:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692507+0000", 2024-09-16T00:09:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9889999999999992e-06 2024-09-16T00:09:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084221+0000", 2024-09-16T00:09:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39171379699999997 2024-09-16T00:09:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084446+0000", 2024-09-16T00:09:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022519900000000001 2024-09-16T00:09:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084672+0000", 2024-09-16T00:09:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000225405 2024-09-16T00:09:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085832+0000", 2024-09-16T00:09:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011600040000000001 2024-09-16T00:09:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085864+0000", 2024-09-16T00:09:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2592999999999998e-05 2024-09-16T00:09:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085943+0000", 2024-09-16T00:09:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8534000000000004e-05 2024-09-16T00:09:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086105+0000", 2024-09-16T00:09:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016165 2024-09-16T00:09:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5963 3.3a 3:5e4192f5:::10000001b5e.00000000:head [write 0~152] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.329988338, 2024-09-16T00:09:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39413463900000001, 2024-09-16T00:09:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5963 2024-09-16T00:09:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:09:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694486+0000", 2024-09-16T00:09:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0960000000000003e-06 2024-09-16T00:09:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694496+0000", 2024-09-16T00:09:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2189999999999993e-06 2024-09-16T00:09:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694500+0000", 2024-09-16T00:09:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9529999999999998e-06 2024-09-16T00:09:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694552+0000", 2024-09-16T00:09:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1360999999999998e-05 2024-09-16T00:09:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694561+0000", 2024-09-16T00:09:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1700000000000003e-06 2024-09-16T00:09:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086346+0000", 2024-09-16T00:09:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39178468 2024-09-16T00:09:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086480+0000", 2024-09-16T00:09:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013469299999999999 2024-09-16T00:09:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086790+0000", 2024-09-16T00:09:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030923300000000001 2024-09-16T00:09:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088361+0000", 2024-09-16T00:09:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015717050000000001 2024-09-16T00:09:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088432+0000", 2024-09-16T00:09:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0922999999999998e-05 2024-09-16T00:09:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088489+0000", 2024-09-16T00:09:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6653999999999997e-05 2024-09-16T00:09:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088616+0000", 2024-09-16T00:09:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012730099999999999 2024-09-16T00:09:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:6082 3.3a 3:5c50f65b:::10000001be7.00000000:head [write 0~2439] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:09:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.036215481999999, 2024-09-16T00:09:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.10065850799999999, 2024-09-16T00:09:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 6082 2024-09-16T00:09:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:09:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:09:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988256+0000", 2024-09-16T00:09:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.065e-06 2024-09-16T00:09:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988262+0000", 2024-09-16T00:09:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4640000000000003e-06 2024-09-16T00:09:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988272+0000", 2024-09-16T00:09:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0335e-05 2024-09-16T00:09:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988279+0000", 2024-09-16T00:09:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:09:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988324+0000", 2024-09-16T00:09:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4953000000000001e-05 2024-09-16T00:09:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988331+0000", 2024-09-16T00:09:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5989999999999998e-06 2024-09-16T00:09:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087272+0000", 2024-09-16T00:09:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.098941045000000005 2024-09-16T00:09:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087486+0000", 2024-09-16T00:09:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00021360199999999999 2024-09-16T00:09:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087740+0000", 2024-09-16T00:09:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00025456799999999999 2024-09-16T00:09:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088638+0000", 2024-09-16T00:09:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00089793900000000005 2024-09-16T00:09:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088673+0000", 2024-09-16T00:09:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5111999999999999e-05 2024-09-16T00:09:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088722+0000", 2024-09-16T00:09:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8758000000000003e-05 2024-09-16T00:09:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088913+0000", 2024-09-16T00:09:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000190953 2024-09-16T00:09:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:6684 3.3a 3:5c48fbe7:::10000001e8d.00000000:head [write 0~156] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e73)", 2024-09-16T00:09:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:20.112205+0000", 2024-09-16T00:09:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.912264829, 2024-09-16T00:09:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033887909999999999, 2024-09-16T00:09:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 6684 2024-09-16T00:09:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:20.112205+0000", 2024-09-16T00:09:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:20.112205+0000", 2024-09-16T00:09:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:20.112208+0000", 2024-09-16T00:09:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.526e-06 2024-09-16T00:09:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:20.112218+0000", 2024-09-16T00:09:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0694e-05 2024-09-16T00:09:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:20.112241+0000", 2024-09-16T00:09:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3142999999999998e-05 2024-09-16T00:09:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:20.112253+0000", 2024-09-16T00:09:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1361e-05 2024-09-16T00:09:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:20.112331+0000", 2024-09-16T00:09:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8196999999999997e-05 2024-09-16T00:09:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op must wait for map", 2024-09-16T00:09:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:20.112339+0000", 2024-09-16T00:09:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7759999999999997e-06 2024-09-16T00:09:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:20.114126+0000", 2024-09-16T00:09:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001787341 2024-09-16T00:09:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:20.114261+0000", 2024-09-16T00:09:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000135258 2024-09-16T00:09:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:20.114393+0000", 2024-09-16T00:09:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013113400000000001 2024-09-16T00:09:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:20.115146+0000", 2024-09-16T00:09:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075359000000000005 2024-09-16T00:09:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:20.115450+0000", 2024-09-16T00:09:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030404999999999999 2024-09-16T00:09:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:20.115551+0000", 2024-09-16T00:09:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000100831 2024-09-16T00:09:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:20.115594+0000", 2024-09-16T00:09:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2889999999999998e-05 2024-09-16T00:09:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.149 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:35.150 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:09:35.150 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:35.149+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:35.185 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T00:09:35.251 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:35.250+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:35.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.295+0000 7f4226e65640 1 Processor -- start 2024-09-16T00:09:35.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.296+0000 7f4226e65640 1 -- start start 2024-09-16T00:09:35.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.296+0000 7f4226e65640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4220151850 0x7f42200c6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:35.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.296+0000 7f4226e65640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42200c8d30 con 0x7f42200c7390 2024-09-16T00:09:35.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.296+0000 7f4226e65640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42200c6a90 con 0x7f4220151c30 2024-09-16T00:09:35.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.296+0000 7f4226e65640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f42200c88a0 con 0x7f4220151850 2024-09-16T00:09:35.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.297+0000 7f4225402640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f42200c7390 0x7f42200c8fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:38138/0 (socket says 172.21.15.139:38138) 2024-09-16T00:09:35.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.297+0000 7f4224c01640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4220151850 0x7f42200c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:35.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.297+0000 7f4225402640 1 -- 172.21.15.139:0/2665052226 learned_addr learned my addr 172.21.15.139:0/2665052226 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:35.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.297+0000 7f4224c01640 1 -- 172.21.15.139:0/2665052226 >> v1:172.21.15.164:6789/0 conn(0x7f4220151c30 legacy=0x7f422015ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:35.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.297+0000 7f4224c01640 1 -- 172.21.15.139:0/2665052226 >> v1:172.21.15.139:6789/0 conn(0x7f42200c7390 legacy=0x7f42200c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:35.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.297+0000 7f4224c01640 1 -- 172.21.15.139:0/2665052226 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42200c67b0 con 0x7f4220151850 2024-09-16T00:09:35.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.298+0000 7f4224c01640 1 --2- 172.21.15.139:0/2665052226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220151850 0x7f42200c6100 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f421400ba40 tx=0x7f421402c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fef5fd95978ec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:35.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.298+0000 7f421f7fe640 1 -- 172.21.15.139:0/2665052226 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214036020 con 0x7f4220151850 2024-09-16T00:09:35.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.298+0000 7f421f7fe640 1 -- 172.21.15.139:0/2665052226 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4214032870 con 0x7f4220151850 2024-09-16T00:09:35.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.298+0000 7f421f7fe640 1 -- 172.21.15.139:0/2665052226 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214032c40 con 0x7f4220151850 2024-09-16T00:09:35.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.299+0000 7f4226e65640 1 -- 172.21.15.139:0/2665052226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220151850 msgr2=0x7f42200c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:35.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.299+0000 7f4226e65640 1 --2- 172.21.15.139:0/2665052226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220151850 0x7f42200c6100 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f421400ba40 tx=0x7f421402c970 comp rx=0 tx=0).stop 2024-09-16T00:09:35.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.299+0000 7f4226e65640 1 -- 172.21.15.139:0/2665052226 shutdown_connections 2024-09-16T00:09:35.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.299+0000 7f4226e65640 1 --2- 172.21.15.139:0/2665052226 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220151850 0x7f42200c6100 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:35.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.299+0000 7f4226e65640 1 -- 172.21.15.139:0/2665052226 >> 172.21.15.139:0/2665052226 conn(0x7f422006cc30 msgr2=0x7f422006d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:35.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.299+0000 7f4226e65640 1 -- 172.21.15.139:0/2665052226 shutdown_connections 2024-09-16T00:09:35.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.299+0000 7f4226e65640 1 -- 172.21.15.139:0/2665052226 wait complete. 2024-09-16T00:09:35.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.300+0000 7f4226e65640 1 Processor -- start 2024-09-16T00:09:35.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.300+0000 7f4226e65640 1 -- start start 2024-09-16T00:09:35.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.301+0000 7f4226e65640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42200c7390 0x7f42201546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:35.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.301+0000 7f4226e65640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220151850 0x7f42200c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:35.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.301+0000 7f4226e65640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4220151c30 0x7f4220158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:35.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.301+0000 7f4226e65640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42200c8530 con 0x7f4220151850 2024-09-16T00:09:35.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.301+0000 7f4226e65640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f42200c8250 con 0x7f4220151c30 2024-09-16T00:09:35.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.301+0000 7f4226e65640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f422015b320 con 0x7f42200c7390 2024-09-16T00:09:35.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.301+0000 7f421ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220151850 0x7f42200c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:35.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.301+0000 7f421ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220151850 0x7f42200c3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58884/0 (socket says 172.21.15.139:58884) 2024-09-16T00:09:35.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.301+0000 7f421ffff640 1 -- 172.21.15.139:0/2919908995 learned_addr learned my addr 172.21.15.139:0/2919908995 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:35.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.301+0000 7f4225402640 1 --2- 172.21.15.139:0/2919908995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4220151c30 0x7f4220158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:35.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.301+0000 7f4224c01640 1 --2- 172.21.15.139:0/2919908995 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42200c7390 0x7f42201546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:35.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.302+0000 7f421ffff640 1 -- 172.21.15.139:0/2919908995 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42200c7390 msgr2=0x7f42201546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:35.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.302+0000 7f421ffff640 1 --2- 172.21.15.139:0/2919908995 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42200c7390 0x7f42201546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:35.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.302+0000 7f421ffff640 1 -- 172.21.15.139:0/2919908995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4220151c30 msgr2=0x7f4220158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:35.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.302+0000 7f421ffff640 1 --2- 172.21.15.139:0/2919908995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4220151c30 0x7f4220158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:35.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.302+0000 7f421ffff640 1 -- 172.21.15.139:0/2919908995 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4220153870 con 0x7f4220151850 2024-09-16T00:09:35.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.302+0000 7f421ffff640 1 --2- 172.21.15.139:0/2919908995 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220151850 0x7f42200c3b00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f420c00c3c0 tx=0x7f420c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:35.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.302+0000 7f421d7fa640 1 -- 172.21.15.139:0/2919908995 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c0052d0 con 0x7f4220151850 2024-09-16T00:09:35.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.302+0000 7f421d7fa640 1 -- 172.21.15.139:0/2919908995 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420c005490 con 0x7f4220151850 2024-09-16T00:09:35.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.302+0000 7f421d7fa640 1 -- 172.21.15.139:0/2919908995 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c007260 con 0x7f4220151850 2024-09-16T00:09:35.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.302+0000 7f4226e65640 1 -- 172.21.15.139:0/2919908995 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4220153b50 con 0x7f4220151850 2024-09-16T00:09:35.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.302+0000 7f4226e65640 1 -- 172.21.15.139:0/2919908995 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42201553a0 con 0x7f4220151850 2024-09-16T00:09:35.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.303+0000 7f4226e65640 1 -- 172.21.15.139:0/2919908995 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42200c45a0 con 0x7f4220151850 2024-09-16T00:09:35.308 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:35.308 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:35.308 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:35.308 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:35.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.304+0000 7f421d7fa640 1 -- 172.21.15.139:0/2919908995 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f420c007400 con 0x7f4220151850 2024-09-16T00:09:35.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.304+0000 7f421d7fa640 1 --2- 172.21.15.139:0/2919908995 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f420406c2b0 0x7f420406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:35.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.304+0000 7f421d7fa640 1 -- 172.21.15.139:0/2919908995 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5973+0+0 (secure 0 0 0) 0x7f420c08a9e0 con 0x7f4220151850 2024-09-16T00:09:35.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.304+0000 7f4224c01640 1 --2- 172.21.15.139:0/2919908995 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f420406c2b0 0x7f420406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:35.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.305+0000 7f4224c01640 1 --2- 172.21.15.139:0/2919908995 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f420406c2b0 0x7f420406e770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4214037000 tx=0x7f4214032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:35.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.313+0000 7f421d7fa640 1 -- 172.21.15.139:0/2919908995 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f420c058960 con 0x7f4220151850 2024-09-16T00:09:35.317 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:09:35.359 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:35.357+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:35.453 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:35.452+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:35.458 INFO:tasks.workunit.client.0.smithi139.stderr:++ readlink -f fsstress 2024-09-16T00:09:35.460 INFO:tasks.workunit.client.0.smithi139.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-09-16T00:09:35.460 INFO:tasks.workunit.client.0.smithi139.stderr:+ popd 2024-09-16T00:09:35.461 INFO:tasks.workunit.client.0.smithi139.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-16T00:09:35.461 INFO:tasks.workunit.client.0.smithi139.stderr:+ popd 2024-09-16T00:09:35.461 INFO:tasks.workunit.client.0.smithi139.stdout:~/cephtest/mnt.0/client.0/tmp 2024-09-16T00:09:35.462 INFO:tasks.workunit.client.0.smithi139.stderr:++ mktemp -d -p . 2024-09-16T00:09:35.475 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:35.475 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:35.475 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:35.475 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:35.475 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:35.484 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:09:35.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.511+0000 7f4226e65640 1 -- 172.21.15.139:0/2919908995 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f4220157ed0 con 0x7f4220151850 2024-09-16T00:09:35.554 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:35.553+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:35.566 INFO:tasks.workunit.client.0.smithi139.stderr:+ T=./tmp.y3b0TuueRu 2024-09-16T00:09:35.567 INFO:tasks.workunit.client.0.smithi139.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.y3b0TuueRu -l 1 -n 1000 -p 10 -v 2024-09-16T00:09:35.569 INFO:tasks.workunit.client.0.smithi139.stdout:seed = 1726989597 2024-09-16T00:09:35.572 INFO:tasks.workunit.client.0.smithi139.stdout:0/0: write - no filename 2024-09-16T00:09:35.572 INFO:tasks.workunit.client.0.smithi139.stdout:0/1: chown . 3 1 2024-09-16T00:09:35.573 INFO:tasks.workunit.client.0.smithi139.stdout:0/2: write - no filename 2024-09-16T00:09:35.573 INFO:tasks.workunit.client.0.smithi139.stdout:0/3: dwrite - no filename 2024-09-16T00:09:35.573 INFO:tasks.workunit.client.0.smithi139.stdout:0/4: write - no filename 2024-09-16T00:09:35.573 INFO:tasks.workunit.client.0.smithi139.stdout:0/5: dwrite - no filename 2024-09-16T00:09:35.573 INFO:tasks.workunit.client.0.smithi139.stdout:0/6: getdents . 0 2024-09-16T00:09:35.573 INFO:tasks.workunit.client.0.smithi139.stdout:1/0: write - no filename 2024-09-16T00:09:35.573 INFO:tasks.workunit.client.0.smithi139.stdout:1/1: readlink - no filename 2024-09-16T00:09:35.573 INFO:tasks.workunit.client.0.smithi139.stdout:1/2: read - no filename 2024-09-16T00:09:35.574 INFO:tasks.workunit.client.0.smithi139.stdout:1/3: dwrite - no filename 2024-09-16T00:09:35.574 INFO:tasks.workunit.client.0.smithi139.stdout:2/0: readlink - no filename 2024-09-16T00:09:35.575 INFO:tasks.workunit.client.0.smithi139.stdout:1/4: creat f0 x:0 0 0 2024-09-16T00:09:35.576 INFO:tasks.workunit.client.0.smithi139.stdout:3/0: dwrite - no filename 2024-09-16T00:09:35.576 INFO:tasks.workunit.client.0.smithi139.stdout:3/1: rename - no filename 2024-09-16T00:09:35.576 INFO:tasks.workunit.client.0.smithi139.stdout:3/2: rmdir - no directory 2024-09-16T00:09:35.576 INFO:tasks.workunit.client.0.smithi139.stdout:0/7: symlink l0 0 2024-09-16T00:09:35.577 INFO:tasks.workunit.client.0.smithi139.stdout:1/5: creat f1 x:0 0 0 2024-09-16T00:09:35.577 INFO:tasks.workunit.client.0.smithi139.stdout:1/6: read - f1 zero size 2024-09-16T00:09:35.578 INFO:tasks.workunit.client.0.smithi139.stdout:2/1: mknod c0 0 2024-09-16T00:09:35.578 INFO:tasks.workunit.client.0.smithi139.stdout:2/2: dwrite - no filename 2024-09-16T00:09:35.578 INFO:tasks.workunit.client.0.smithi139.stdout:2/3: dread - no filename 2024-09-16T00:09:35.579 INFO:tasks.workunit.client.0.smithi139.stdout:2/4: dwrite - no filename 2024-09-16T00:09:35.579 INFO:tasks.workunit.client.0.smithi139.stdout:4/0: fdatasync - no filename 2024-09-16T00:09:35.579 INFO:tasks.workunit.client.0.smithi139.stdout:4/1: link - no file 2024-09-16T00:09:35.579 INFO:tasks.workunit.client.0.smithi139.stdout:4/2: dread - no filename 2024-09-16T00:09:35.579 INFO:tasks.workunit.client.0.smithi139.stdout:4/3: dread - no filename 2024-09-16T00:09:35.579 INFO:tasks.workunit.client.0.smithi139.stdout:4/4: write - no filename 2024-09-16T00:09:35.579 INFO:tasks.workunit.client.0.smithi139.stdout:4/5: chown . 379917286 1 2024-09-16T00:09:35.579 INFO:tasks.workunit.client.0.smithi139.stdout:1/7: write f1 [130021,130729] 0 2024-09-16T00:09:35.579 INFO:tasks.workunit.client.0.smithi139.stdout:4/6: chown . 2 1 2024-09-16T00:09:35.579 INFO:tasks.workunit.client.0.smithi139.stdout:4/7: dwrite - no filename 2024-09-16T00:09:35.580 INFO:tasks.workunit.client.0.smithi139.stdout:4/8: rename - no filename 2024-09-16T00:09:35.580 INFO:tasks.workunit.client.0.smithi139.stdout:0/8: unlink l0 0 2024-09-16T00:09:35.580 INFO:tasks.workunit.client.0.smithi139.stdout:0/9: write - no filename 2024-09-16T00:09:35.580 INFO:tasks.workunit.client.0.smithi139.stdout:0/10: link - no file 2024-09-16T00:09:35.580 INFO:tasks.workunit.client.0.smithi139.stdout:0/11: dwrite - no filename 2024-09-16T00:09:35.580 INFO:tasks.workunit.client.0.smithi139.stdout:0/12: unlink - no file 2024-09-16T00:09:35.580 INFO:tasks.workunit.client.0.smithi139.stdout:0/13: truncate - no filename 2024-09-16T00:09:35.581 INFO:tasks.workunit.client.0.smithi139.stdout:3/3: symlink l0 0 2024-09-16T00:09:35.581 INFO:tasks.workunit.client.0.smithi139.stdout:3/4: fsync - no filename 2024-09-16T00:09:35.582 INFO:tasks.workunit.client.0.smithi139.stdout:2/5: creat f1 x:0 0 0 2024-09-16T00:09:35.582 INFO:tasks.workunit.client.0.smithi139.stdout:2/6: dread - f1 zero size 2024-09-16T00:09:35.582 INFO:tasks.workunit.client.0.smithi139.stdout:2/7: truncate f1 786733 0 2024-09-16T00:09:35.582 INFO:tasks.workunit.client.0.smithi139.stdout:2/8: readlink - no filename 2024-09-16T00:09:35.582 INFO:tasks.workunit.client.0.smithi139.stdout:1/8: rename f1 to f2 0 2024-09-16T00:09:35.583 INFO:tasks.workunit.client.0.smithi139.stdout:0/14: creat f1 x:0 0 0 2024-09-16T00:09:35.583 INFO:tasks.workunit.client.0.smithi139.stdout:6/0: truncate - no filename 2024-09-16T00:09:35.583 INFO:tasks.workunit.client.0.smithi139.stdout:3/5: mknod c1 0 2024-09-16T00:09:35.583 INFO:tasks.workunit.client.0.smithi139.stdout:3/6: write - no filename 2024-09-16T00:09:35.584 INFO:tasks.workunit.client.0.smithi139.stdout:4/9: creat f0 x:0 0 0 2024-09-16T00:09:35.585 INFO:tasks.workunit.client.0.smithi139.stdout:5/0: creat f0 x:0 0 0 2024-09-16T00:09:35.587 INFO:tasks.workunit.client.0.smithi139.stdout:1/9: write f2 [680748,61129] 0 2024-09-16T00:09:35.588 INFO:tasks.workunit.client.0.smithi139.stdout:5/1: chown f0 98253233 1 2024-09-16T00:09:35.588 INFO:tasks.workunit.client.0.smithi139.stdout:4/10: creat f1 x:0 0 0 2024-09-16T00:09:35.589 INFO:tasks.workunit.client.0.smithi139.stdout:3/7: unlink c1 0 2024-09-16T00:09:35.589 INFO:tasks.workunit.client.0.smithi139.stdout:6/1: creat f0 x:0 0 0 2024-09-16T00:09:35.589 INFO:tasks.workunit.client.0.smithi139.stdout:3/8: stat l0 0 2024-09-16T00:09:35.590 INFO:tasks.workunit.client.0.smithi139.stdout:3/9: write - no filename 2024-09-16T00:09:35.590 INFO:tasks.workunit.client.0.smithi139.stdout:3/10: write - no filename 2024-09-16T00:09:35.590 INFO:tasks.workunit.client.0.smithi139.stdout:3/11: truncate - no filename 2024-09-16T00:09:35.590 INFO:tasks.workunit.client.0.smithi139.stdout:3/12: dread - no filename 2024-09-16T00:09:35.590 INFO:tasks.workunit.client.0.smithi139.stdout:3/13: fdatasync - no filename 2024-09-16T00:09:35.590 INFO:tasks.workunit.client.0.smithi139.stdout:7/0: chown . 1815634 1 2024-09-16T00:09:35.590 INFO:tasks.workunit.client.0.smithi139.stdout:7/1: dread - no filename 2024-09-16T00:09:35.591 INFO:tasks.workunit.client.0.smithi139.stdout:5/2: unlink f0 0 2024-09-16T00:09:35.592 INFO:tasks.workunit.client.0.smithi139.stdout:1/10: symlink l3 0 2024-09-16T00:09:35.593 INFO:tasks.workunit.client.0.smithi139.stdout:6/2: symlink l1 0 2024-09-16T00:09:35.593 INFO:tasks.workunit.client.0.smithi139.stdout:8/0: rename - no filename 2024-09-16T00:09:35.593 INFO:tasks.workunit.client.0.smithi139.stdout:4/11: write f0 [236374,83016] 0 2024-09-16T00:09:35.595 INFO:tasks.workunit.client.0.smithi139.stdout:3/14: rename l0 to l2 0 2024-09-16T00:09:35.595 INFO:tasks.workunit.client.0.smithi139.stdout:3/15: write - no filename 2024-09-16T00:09:35.597 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:35.597 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:35.597 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:35.597 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:35.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.597 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:22 2.e e22/19 2:717a0223:::608.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:35.597 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:09:35.598 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 116.977942361, 2024-09-16T00:09:35.598 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042195870000000003, 2024-09-16T00:09:35.598 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.598 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.598 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.598 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.598 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.598 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.598 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:09:35.598 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.598 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.598 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.598 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.599 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616425+0000", 2024-09-16T00:09:35.599 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.599 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.599 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616426+0000", 2024-09-16T00:09:35.599 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.037e-06 2024-09-16T00:09:35.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.599 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.599 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616435+0000", 2024-09-16T00:09:35.599 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6440000000000006e-06 2024-09-16T00:09:35.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616447+0000", 2024-09-16T00:09:35.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2466000000000001e-05 2024-09-16T00:09:35.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.601 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616456+0000", 2024-09-16T00:09:35.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8799999999999997e-06 2024-09-16T00:09:35.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.601 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.601 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616517+0000", 2024-09-16T00:09:35.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1171000000000001e-05 2024-09-16T00:09:35.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.601 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.601 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616540+0000", 2024-09-16T00:09:35.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2991000000000001e-05 2024-09-16T00:09:35.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620599+0000", 2024-09-16T00:09:35.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040586140000000003 2024-09-16T00:09:35.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620645+0000", 2024-09-16T00:09:35.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5784000000000001e-05 2024-09-16T00:09:35.602 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.602 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.602 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.603 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:35.603 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:09:35.603 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 116.977852631, 2024-09-16T00:09:35.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044836249999999998, 2024-09-16T00:09:35.603 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.603 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.603 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:09:35.603 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.603 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.603 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:09:35.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.604 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.604 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616515+0000", 2024-09-16T00:09:35.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.604 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.604 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616518+0000", 2024-09-16T00:09:35.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3699999999999999e-06 2024-09-16T00:09:35.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:09:35.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1479999999999997e-06 2024-09-16T00:09:35.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616534+0000", 2024-09-16T00:09:35.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0641e-05 2024-09-16T00:09:35.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616541+0000", 2024-09-16T00:09:35.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3339999999999996e-06 2024-09-16T00:09:35.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617318+0000", 2024-09-16T00:09:35.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00077694799999999996 2024-09-16T00:09:35.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617409+0000", 2024-09-16T00:09:35.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0740000000000002e-05 2024-09-16T00:09:35.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.607 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620998+0000", 2024-09-16T00:09:35.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035894439999999998 2024-09-16T00:09:35.607 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.607 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.607 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.607 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:37 2.1e 2:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:35.607 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:09:35.607 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 116.977804728, 2024-09-16T00:09:35.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041597320000000002, 2024-09-16T00:09:35.607 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.607 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.608 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:09:35.608 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.608 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:09:35.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.608 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:09:35.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616563+0000", 2024-09-16T00:09:35.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:09:35.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616566+0000", 2024-09-16T00:09:35.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6979999999999998e-06 2024-09-16T00:09:35.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616574+0000", 2024-09-16T00:09:35.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7759999999999997e-06 2024-09-16T00:09:35.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616580+0000", 2024-09-16T00:09:35.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8130000000000001e-06 2024-09-16T00:09:35.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616800+0000", 2024-09-16T00:09:35.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022016700000000001 2024-09-16T00:09:35.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616904+0000", 2024-09-16T00:09:35.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104579 2024-09-16T00:09:35.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620722+0000", 2024-09-16T00:09:35.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003818133 2024-09-16T00:09:35.611 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.611 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.611 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.611 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:18 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'1, mlcod=0'0)", 2024-09-16T00:09:35.611 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:09:35.611 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 116.977554043, 2024-09-16T00:09:35.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038451570000000001, 2024-09-16T00:09:35.612 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.612 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.612 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.612 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.612 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.612 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:09:35.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.612 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.612 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616813+0000", 2024-09-16T00:09:35.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.613 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.613 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616814+0000", 2024-09-16T00:09:35.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9299999999999997e-07 2024-09-16T00:09:35.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.613 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.613 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616824+0000", 2024-09-16T00:09:35.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6600000000000007e-06 2024-09-16T00:09:35.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616835+0000", 2024-09-16T00:09:35.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0713000000000001e-05 2024-09-16T00:09:35.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616841+0000", 2024-09-16T00:09:35.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.934e-06 2024-09-16T00:09:35.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616948+0000", 2024-09-16T00:09:35.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010631100000000001 2024-09-16T00:09:35.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616967+0000", 2024-09-16T00:09:35.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9598000000000001e-05 2024-09-16T00:09:35.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620619+0000", 2024-09-16T00:09:35.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036515810000000001 2024-09-16T00:09:35.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620659+0000", 2024-09-16T00:09:35.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9666999999999998e-05 2024-09-16T00:09:35.617 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.617 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.617 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.618 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:19 2.14 e22/19 2:2f55791f:::606.00000000:head v 22'2, mlcod=0'0)", 2024-09-16T00:09:35.618 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:09:35.618 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 116.97750709100001, 2024-09-16T00:09:35.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038240129999999998, 2024-09-16T00:09:35.618 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.618 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.618 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.618 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.618 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.618 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:09:35.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.619 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.619 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616860+0000", 2024-09-16T00:09:35.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.619 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.619 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616861+0000", 2024-09-16T00:09:35.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0200000000000003e-07 2024-09-16T00:09:35.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.619 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616864+0000", 2024-09-16T00:09:35.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5090000000000002e-06 2024-09-16T00:09:35.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616871+0000", 2024-09-16T00:09:35.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9249999999999998e-06 2024-09-16T00:09:35.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616877+0000", 2024-09-16T00:09:35.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.981e-06 2024-09-16T00:09:35.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617168+0000", 2024-09-16T00:09:35.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029035699999999998 2024-09-16T00:09:35.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617184+0000", 2024-09-16T00:09:35.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6005999999999999e-05 2024-09-16T00:09:35.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620661+0000", 2024-09-16T00:09:35.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034775330000000001 2024-09-16T00:09:35.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.622 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.622 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620684+0000", 2024-09-16T00:09:35.622 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3300000000000001e-05 2024-09-16T00:09:35.622 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.622 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.622 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.622 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:1 2.14 e22/19 2:292cf221:::200.00000000:head v 22'3, mlcod=0'0)", 2024-09-16T00:09:35.622 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:09:35.622 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 116.977051648, 2024-09-16T00:09:35.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045415610000000004, 2024-09-16T00:09:35.623 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.623 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.623 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.623 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:09:35.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:09:35.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.624 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617316+0000", 2024-09-16T00:09:35.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8800000000000002e-07 2024-09-16T00:09:35.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.624 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617321+0000", 2024-09-16T00:09:35.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6979999999999997e-06 2024-09-16T00:09:35.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.625 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617330+0000", 2024-09-16T00:09:35.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6729999999999998e-06 2024-09-16T00:09:35.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.625 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617336+0000", 2024-09-16T00:09:35.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2110000000000003e-06 2024-09-16T00:09:35.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.625 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617394+0000", 2024-09-16T00:09:35.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8087999999999997e-05 2024-09-16T00:09:35.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.626 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617412+0000", 2024-09-16T00:09:35.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8099999999999999e-05 2024-09-16T00:09:35.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.626 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621818+0000", 2024-09-16T00:09:35.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044061209999999998 2024-09-16T00:09:35.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.626 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.627 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621857+0000", 2024-09-16T00:09:35.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9082000000000003e-05 2024-09-16T00:09:35.627 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.627 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.627 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.627 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:243 3.24 3:25048551:::100000002f0.00000000:head [write 0~1428] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e57)", 2024-09-16T00:09:35.627 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:56.799025+0000", 2024-09-16T00:09:35.627 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 38.795342292999997, 2024-09-16T00:09:35.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0042804829999999999, 2024-09-16T00:09:35.627 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.627 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.627 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.628 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:35.628 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:35.628 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:35.628 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 243 2024-09-16T00:09:35.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.628 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799025+0000", 2024-09-16T00:09:35.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.629 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799025+0000", 2024-09-16T00:09:35.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.629 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799026+0000", 2024-09-16T00:09:35.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9099999999999991e-07 2024-09-16T00:09:35.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.629 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799032+0000", 2024-09-16T00:09:35.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6049999999999997e-06 2024-09-16T00:09:35.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.630 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799044+0000", 2024-09-16T00:09:35.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1861999999999999e-05 2024-09-16T00:09:35.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.630 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799049+0000", 2024-09-16T00:09:35.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6570000000000002e-06 2024-09-16T00:09:35.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.631 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799095+0000", 2024-09-16T00:09:35.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5479999999999998e-05 2024-09-16T00:09:35.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.631 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.631 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799175+0000", 2024-09-16T00:09:35.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0181999999999999e-05 2024-09-16T00:09:35.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.631 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:35.631 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799252+0000", 2024-09-16T00:09:35.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6822000000000004e-05 2024-09-16T00:09:35.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.632 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.632 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.799855+0000", 2024-09-16T00:09:35.632 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00060294199999999997 2024-09-16T00:09:35.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.632 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.632 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803230+0000", 2024-09-16T00:09:35.632 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033752719999999999 2024-09-16T00:09:35.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.632 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803263+0000", 2024-09-16T00:09:35.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3339e-05 2024-09-16T00:09:35.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:56.803306+0000", 2024-09-16T00:09:35.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2330999999999998e-05 2024-09-16T00:09:35.633 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.633 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.633 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.633 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:324 3.29 3:960c7fc3:::10000000355.00000000:head [write 0~1506] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:35.634 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.034622+0000", 2024-09-16T00:09:35.634 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 38.559745272000001, 2024-09-16T00:09:35.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003641746, 2024-09-16T00:09:35.634 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.634 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.634 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.634 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:35.634 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:35.634 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:35.634 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 324 2024-09-16T00:09:35.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.634 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034622+0000", 2024-09-16T00:09:35.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034622+0000", 2024-09-16T00:09:35.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034623+0000", 2024-09-16T00:09:35.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1689999999999999e-06 2024-09-16T00:09:35.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034629+0000", 2024-09-16T00:09:35.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4859999999999997e-06 2024-09-16T00:09:35.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034639+0000", 2024-09-16T00:09:35.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0142999999999999e-05 2024-09-16T00:09:35.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034644+0000", 2024-09-16T00:09:35.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1159999999999998e-06 2024-09-16T00:09:35.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034693+0000", 2024-09-16T00:09:35.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9228999999999999e-05 2024-09-16T00:09:35.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034813+0000", 2024-09-16T00:09:35.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012003200000000001 2024-09-16T00:09:35.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:35.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.034940+0000", 2024-09-16T00:09:35.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012655699999999999 2024-09-16T00:09:35.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.036244+0000", 2024-09-16T00:09:35.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001304326 2024-09-16T00:09:35.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.038175+0000", 2024-09-16T00:09:35.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001930768 2024-09-16T00:09:35.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.038225+0000", 2024-09-16T00:09:35.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0046000000000002e-05 2024-09-16T00:09:35.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.038264+0000", 2024-09-16T00:09:35.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8874000000000001e-05 2024-09-16T00:09:35.639 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.639 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.639 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.639 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:411 3.29 3:96aa9846:::100000003bf.00000000:head [write 0~1161] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-16T00:09:35.640 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:57.363361+0000", 2024-09-16T00:09:35.640 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 38.231006221999998, 2024-09-16T00:09:35.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038090889999999999, 2024-09-16T00:09:35.640 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.640 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.640 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.640 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:35.640 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:35.640 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:35.640 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 411 2024-09-16T00:09:35.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.641 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363361+0000", 2024-09-16T00:09:35.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363361+0000", 2024-09-16T00:09:35.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363363+0000", 2024-09-16T00:09:35.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.421e-06 2024-09-16T00:09:35.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363369+0000", 2024-09-16T00:09:35.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7270000000000001e-06 2024-09-16T00:09:35.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363382+0000", 2024-09-16T00:09:35.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2700999999999999e-05 2024-09-16T00:09:35.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363388+0000", 2024-09-16T00:09:35.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0009999999999998e-06 2024-09-16T00:09:35.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363438+0000", 2024-09-16T00:09:35.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9789000000000001e-05 2024-09-16T00:09:35.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363562+0000", 2024-09-16T00:09:35.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000123798 2024-09-16T00:09:35.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:35.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.363639+0000", 2024-09-16T00:09:35.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7074000000000001e-05 2024-09-16T00:09:35.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.645 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.364153+0000", 2024-09-16T00:09:35.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00051393999999999995 2024-09-16T00:09:35.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.645 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.367097+0000", 2024-09-16T00:09:35.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00294414 2024-09-16T00:09:35.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.646 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.367145+0000", 2024-09-16T00:09:35.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8591999999999998e-05 2024-09-16T00:09:35.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.646 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.646 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:57.367170+0000", 2024-09-16T00:09:35.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4905999999999998e-05 2024-09-16T00:09:35.646 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.646 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.646 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.646 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:72 2.14 e61/46 2:2e2fa760:::200.00000001:head v 61'21, mlcod=61'19)", 2024-09-16T00:09:35.646 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:01.672924+0000", 2024-09-16T00:09:35.647 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.921443353999997, 2024-09-16T00:09:35.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003655784, 2024-09-16T00:09:35.647 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.647 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.647 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.647 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.672924+0000", 2024-09-16T00:09:35.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.672924+0000", 2024-09-16T00:09:35.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.672925+0000", 2024-09-16T00:09:35.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0189999999999999e-06 2024-09-16T00:09:35.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674096+0000", 2024-09-16T00:09:35.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011709660000000001 2024-09-16T00:09:35.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.650 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.651 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674112+0000", 2024-09-16T00:09:35.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6149000000000001e-05 2024-09-16T00:09:35.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.653 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.653 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674120+0000", 2024-09-16T00:09:35.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.892e-06 2024-09-16T00:09:35.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.654 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674179+0000", 2024-09-16T00:09:35.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8876999999999999e-05 2024-09-16T00:09:35.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.654 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.674197+0000", 2024-09-16T00:09:35.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8048999999999999e-05 2024-09-16T00:09:35.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.676539+0000", 2024-09-16T00:09:35.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023414830000000001 2024-09-16T00:09:35.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.655 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:01.676580+0000", 2024-09-16T00:09:35.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1349e-05 2024-09-16T00:09:35.655 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.655 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.655 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.655 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:86 2.1b 2:ddb4e4d7:::200.00000002:head [write 368603~2220227 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:35.656 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.644964+0000", 2024-09-16T00:09:35.656 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.949403280999999, 2024-09-16T00:09:35.656 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011714478, 2024-09-16T00:09:35.656 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.656 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.656 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.656 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.656 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.656 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.656 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.644964+0000", 2024-09-16T00:09:35.656 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.656 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.656 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.657 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.657 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.644964+0000", 2024-09-16T00:09:35.657 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.657 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.657 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.657 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.657 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.644966+0000", 2024-09-16T00:09:35.657 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4789999999999999e-06 2024-09-16T00:09:35.657 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.657 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.657 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.657 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.648979+0000", 2024-09-16T00:09:35.657 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040136900000000003 2024-09-16T00:09:35.657 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.658 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.648993+0000", 2024-09-16T00:09:35.658 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3129000000000001e-05 2024-09-16T00:09:35.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.658 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649000+0000", 2024-09-16T00:09:35.658 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5700000000000004e-06 2024-09-16T00:09:35.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.658 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649055+0000", 2024-09-16T00:09:35.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4914999999999999e-05 2024-09-16T00:09:35.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.659 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649131+0000", 2024-09-16T00:09:35.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6055000000000003e-05 2024-09-16T00:09:35.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.660 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.660 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:35.660 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649258+0000", 2024-09-16T00:09:35.660 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000127229 2024-09-16T00:09:35.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.662 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.662 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.662 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656218+0000", 2024-09-16T00:09:35.662 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069600469999999996 2024-09-16T00:09:35.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.662 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.662 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.662 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656602+0000", 2024-09-16T00:09:35.662 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038380700000000001 2024-09-16T00:09:35.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.662 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656638+0000", 2024-09-16T00:09:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5725000000000001e-05 2024-09-16T00:09:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656679+0000", 2024-09-16T00:09:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0831999999999998e-05 2024-09-16T00:09:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:87 2.1b 2:ddb4e4d7:::200.00000002:head [write 2588830~1400 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649020+0000", 2024-09-16T00:09:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.945347089999999, 2024-09-16T00:09:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0083707460000000001, 2024-09-16T00:09:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649020+0000", 2024-09-16T00:09:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649020+0000", 2024-09-16T00:09:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649021+0000", 2024-09-16T00:09:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6000000000000002e-07 2024-09-16T00:09:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649034+0000", 2024-09-16T00:09:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2627e-05 2024-09-16T00:09:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649045+0000", 2024-09-16T00:09:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0744000000000001e-05 2024-09-16T00:09:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649051+0000", 2024-09-16T00:09:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5459999999999997e-06 2024-09-16T00:09:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654369+0000", 2024-09-16T00:09:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053179519999999999 2024-09-16T00:09:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654435+0000", 2024-09-16T00:09:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6175999999999998e-05 2024-09-16T00:09:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654525+0000", 2024-09-16T00:09:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9856000000000002e-05 2024-09-16T00:09:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656225+0000", 2024-09-16T00:09:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001699386 2024-09-16T00:09:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657339+0000", 2024-09-16T00:09:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011145250000000001 2024-09-16T00:09:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657366+0000", 2024-09-16T00:09:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7172e-05 2024-09-16T00:09:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657391+0000", 2024-09-16T00:09:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4902e-05 2024-09-16T00:09:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:88 2.1b 2:ddb4e4d7:::200.00000002:head [write 2590230~1400 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649060+0000", 2024-09-16T00:09:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.945307287999999, 2024-09-16T00:09:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0088748730000000001, 2024-09-16T00:09:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649060+0000", 2024-09-16T00:09:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649060+0000", 2024-09-16T00:09:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649061+0000", 2024-09-16T00:09:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2300000000000001e-07 2024-09-16T00:09:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649064+0000", 2024-09-16T00:09:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3160000000000001e-06 2024-09-16T00:09:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649070+0000", 2024-09-16T00:09:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9880000000000001e-06 2024-09-16T00:09:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649074+0000", 2024-09-16T00:09:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4329999999999997e-06 2024-09-16T00:09:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654769+0000", 2024-09-16T00:09:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0056944500000000002 2024-09-16T00:09:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654875+0000", 2024-09-16T00:09:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000106183 2024-09-16T00:09:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654994+0000", 2024-09-16T00:09:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011867900000000001 2024-09-16T00:09:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657697+0000", 2024-09-16T00:09:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027033920000000002 2024-09-16T00:09:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657882+0000", 2024-09-16T00:09:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018478900000000001 2024-09-16T00:09:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657913+0000", 2024-09-16T00:09:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1306000000000001e-05 2024-09-16T00:09:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657935+0000", 2024-09-16T00:09:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1814000000000002e-05 2024-09-16T00:09:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:89 2.1b 2:ddb4e4d7:::200.00000002:head [write 2591630~1401 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.945287028999999, 2024-09-16T00:09:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011802609, 2024-09-16T00:09:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649081+0000", 2024-09-16T00:09:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7100000000000002e-07 2024-09-16T00:09:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649089+0000", 2024-09-16T00:09:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7640000000000005e-06 2024-09-16T00:09:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649094+0000", 2024-09-16T00:09:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6219999999999997e-06 2024-09-16T00:09:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649099+0000", 2024-09-16T00:09:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6970000000000001e-06 2024-09-16T00:09:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655288+0000", 2024-09-16T00:09:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061885569999999999 2024-09-16T00:09:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655365+0000", 2024-09-16T00:09:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7723000000000001e-05 2024-09-16T00:09:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655483+0000", 2024-09-16T00:09:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117895 2024-09-16T00:09:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657758+0000", 2024-09-16T00:09:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022752369999999998 2024-09-16T00:09:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660803+0000", 2024-09-16T00:09:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030441499999999998 2024-09-16T00:09:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660850+0000", 2024-09-16T00:09:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7017000000000001e-05 2024-09-16T00:09:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660883+0000", 2024-09-16T00:09:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3476000000000003e-05 2024-09-16T00:09:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:90 2.1b 2:ddb4e4d7:::200.00000002:head [write 2593031~1404 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.945249124, 2024-09-16T00:09:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011882886, 2024-09-16T00:09:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649119+0000", 2024-09-16T00:09:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:09:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649125+0000", 2024-09-16T00:09:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1650000000000003e-06 2024-09-16T00:09:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649134+0000", 2024-09-16T00:09:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5669999999999995e-06 2024-09-16T00:09:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649141+0000", 2024-09-16T00:09:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9040000000000003e-06 2024-09-16T00:09:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655796+0000", 2024-09-16T00:09:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066553649999999999 2024-09-16T00:09:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655868+0000", 2024-09-16T00:09:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2219999999999996e-05 2024-09-16T00:09:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655986+0000", 2024-09-16T00:09:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117457 2024-09-16T00:09:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657794+0000", 2024-09-16T00:09:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001807894 2024-09-16T00:09:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660890+0000", 2024-09-16T00:09:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030958449999999998 2024-09-16T00:09:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660966+0000", 2024-09-16T00:09:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6382000000000002e-05 2024-09-16T00:09:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.661001+0000", 2024-09-16T00:09:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5349000000000003e-05 2024-09-16T00:09:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:2397 3.29 3:948abdad:::10000000cb5.00000000:head [write 0~1373] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-16T00:09:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.655571+0000", 2024-09-16T00:09:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.938796838999998, 2024-09-16T00:09:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052117400000000003, 2024-09-16T00:09:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 2397 2024-09-16T00:09:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655571+0000", 2024-09-16T00:09:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655571+0000", 2024-09-16T00:09:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655573+0000", 2024-09-16T00:09:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0789999999999999e-06 2024-09-16T00:09:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655580+0000", 2024-09-16T00:09:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7859999999999999e-06 2024-09-16T00:09:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655592+0000", 2024-09-16T00:09:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2541000000000001e-05 2024-09-16T00:09:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655601+0000", 2024-09-16T00:09:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.721e-06 2024-09-16T00:09:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655663+0000", 2024-09-16T00:09:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2287999999999997e-05 2024-09-16T00:09:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655765+0000", 2024-09-16T00:09:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000102212 2024-09-16T00:09:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655893+0000", 2024-09-16T00:09:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012742499999999999 2024-09-16T00:09:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657022+0000", 2024-09-16T00:09:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011293340000000001 2024-09-16T00:09:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660686+0000", 2024-09-16T00:09:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036637409999999999 2024-09-16T00:09:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660737+0000", 2024-09-16T00:09:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1711e-05 2024-09-16T00:09:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660782+0000", 2024-09-16T00:09:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4901999999999998e-05 2024-09-16T00:09:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:276 2.1e 2:7bf7e7b5:::200.00000006:head [write 0~2705854 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e69)", 2024-09-16T00:09:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:16.488883+0000", 2024-09-16T00:09:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.105484311000001, 2024-09-16T00:09:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010883972, 2024-09-16T00:09:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.488883+0000", 2024-09-16T00:09:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.488883+0000", 2024-09-16T00:09:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.488884+0000", 2024-09-16T00:09:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1299999999999999e-07 2024-09-16T00:09:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493021+0000", 2024-09-16T00:09:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041369299999999996 2024-09-16T00:09:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493036+0000", 2024-09-16T00:09:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4592e-05 2024-09-16T00:09:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493042+0000", 2024-09-16T00:09:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6150000000000002e-06 2024-09-16T00:09:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493092+0000", 2024-09-16T00:09:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0126000000000001e-05 2024-09-16T00:09:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493140+0000", 2024-09-16T00:09:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8207000000000001e-05 2024-09-16T00:09:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493218+0000", 2024-09-16T00:09:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8030999999999999e-05 2024-09-16T00:09:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.499383+0000", 2024-09-16T00:09:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061649699999999997 2024-09-16T00:09:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.499696+0000", 2024-09-16T00:09:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00031260499999999998 2024-09-16T00:09:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.499732+0000", 2024-09-16T00:09:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5868000000000003e-05 2024-09-16T00:09:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.499767+0000", 2024-09-16T00:09:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5215000000000001e-05 2024-09-16T00:09:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:9172 3.15 e76/20 3:a8348462:::10000002a14.00000000:head v 76'136, mlcod=76'134)", 2024-09-16T00:09:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:29.576090+0000", 2024-09-16T00:09:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.0182779609999999, 2024-09-16T00:09:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004853642, 2024-09-16T00:09:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.576090+0000", 2024-09-16T00:09:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.576090+0000", 2024-09-16T00:09:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.576090+0000", 2024-09-16T00:09:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0900000000000001e-07 2024-09-16T00:09:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.576100+0000", 2024-09-16T00:09:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9129999999999998e-06 2024-09-16T00:09:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.576112+0000", 2024-09-16T00:09:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.154e-05 2024-09-16T00:09:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.576118+0000", 2024-09-16T00:09:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6409999999999996e-06 2024-09-16T00:09:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.576161+0000", 2024-09-16T00:09:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2341e-05 2024-09-16T00:09:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.576178+0000", 2024-09-16T00:09:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7299000000000002e-05 2024-09-16T00:09:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.580913+0000", 2024-09-16T00:09:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004734879 2024-09-16T00:09:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:29.580943+0000", 2024-09-16T00:09:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0320000000000001e-05 2024-09-16T00:09:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10559 3.1e 3:7831c2c5:::10000003054.00000000:head [write 0~1406019] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T00:09:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:34.029318+0000", 2024-09-16T00:09:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.5650491149999999, 2024-09-16T00:09:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0079691380000000006, 2024-09-16T00:09:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10559 2024-09-16T00:09:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.029318+0000", 2024-09-16T00:09:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.029318+0000", 2024-09-16T00:09:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.029321+0000", 2024-09-16T00:09:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6699999999999998e-06 2024-09-16T00:09:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.030077+0000", 2024-09-16T00:09:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00075637899999999995 2024-09-16T00:09:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.030097+0000", 2024-09-16T00:09:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9602999999999998e-05 2024-09-16T00:09:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.030108+0000", 2024-09-16T00:09:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0499000000000001e-05 2024-09-16T00:09:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.030177+0000", 2024-09-16T00:09:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9601e-05 2024-09-16T00:09:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.030289+0000", 2024-09-16T00:09:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000112244 2024-09-16T00:09:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.030388+0000", 2024-09-16T00:09:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8859999999999999e-05 2024-09-16T00:09:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.036016+0000", 2024-09-16T00:09:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0056272939999999997 2024-09-16T00:09:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.037209+0000", 2024-09-16T00:09:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00119355 2024-09-16T00:09:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.037242+0000", 2024-09-16T00:09:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2866e-05 2024-09-16T00:09:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.037288+0000", 2024-09-16T00:09:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5571999999999997e-05 2024-09-16T00:09:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10561 3.1e 3:79a87bfb:::10000003056.00000000:head [write 0~1813] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-16T00:09:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:34.033310+0000", 2024-09-16T00:09:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.5610579339999999, 2024-09-16T00:09:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041472530000000001, 2024-09-16T00:09:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10561 2024-09-16T00:09:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.033310+0000", 2024-09-16T00:09:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.033310+0000", 2024-09-16T00:09:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.033311+0000", 2024-09-16T00:09:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6050000000000001e-06 2024-09-16T00:09:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.033322+0000", 2024-09-16T00:09:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0930999999999999e-05 2024-09-16T00:09:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.033332+0000", 2024-09-16T00:09:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0212e-05 2024-09-16T00:09:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.033338+0000", 2024-09-16T00:09:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7259999999999999e-06 2024-09-16T00:09:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.035513+0000", 2024-09-16T00:09:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002175256 2024-09-16T00:09:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.035644+0000", 2024-09-16T00:09:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013073300000000001 2024-09-16T00:09:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.035740+0000", 2024-09-16T00:09:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5544999999999994e-05 2024-09-16T00:09:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.036205+0000", 2024-09-16T00:09:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00046494 2024-09-16T00:09:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.037371+0000", 2024-09-16T00:09:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001166411 2024-09-16T00:09:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.037419+0000", 2024-09-16T00:09:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8393000000000002e-05 2024-09-16T00:09:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:34.037457+0000", 2024-09-16T00:09:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7500999999999999e-05 2024-09-16T00:09:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:35.714 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:35.717 INFO:tasks.workunit.client.0.smithi139.stdout:1/11: write f2 [1710040,39898] 0 2024-09-16T00:09:35.717 INFO:tasks.workunit.client.0.smithi139.stdout:2/9: dwrite f1 [0,4194304] 0 2024-09-16T00:09:35.717 INFO:tasks.workunit.client.0.smithi139.stdout:6/3: truncate f0 661745 0 2024-09-16T00:09:35.717 INFO:tasks.workunit.client.0.smithi139.stdout:5/3: mkdir d1 0 2024-09-16T00:09:35.717 INFO:tasks.workunit.client.0.smithi139.stdout:0/15: dwrite f1 [0,4194304] 0 2024-09-16T00:09:35.717 INFO:tasks.workunit.client.0.smithi139.stdout:5/4: rename d1 to d1/d2 22 2024-09-16T00:09:35.718 INFO:tasks.workunit.client.0.smithi139.stdout:6/4: read f0 [197340,14875] 0 2024-09-16T00:09:35.718 INFO:tasks.workunit.client.0.smithi139.stdout:4/12: rename f0 to f2 0 2024-09-16T00:09:35.718 INFO:tasks.workunit.client.0.smithi139.stdout:8/1: creat f0 x:0 0 0 2024-09-16T00:09:35.718 INFO:tasks.workunit.client.0.smithi139.stdout:1/12: write f2 [2170911,46010] 0 2024-09-16T00:09:35.718 INFO:tasks.workunit.client.0.smithi139.stdout:0/16: stat f1 0 2024-09-16T00:09:35.718 INFO:tasks.workunit.client.0.smithi139.stdout:4/13: write f1 [644864,77948] 0 2024-09-16T00:09:35.718 INFO:tasks.workunit.client.0.smithi139.stdout:9/0: creat f0 x:0 0 0 2024-09-16T00:09:35.718 INFO:tasks.workunit.client.0.smithi139.stdout:8/2: creat f1 x:0 0 0 2024-09-16T00:09:35.718 INFO:tasks.workunit.client.0.smithi139.stdout:8/3: dread - f1 zero size 2024-09-16T00:09:35.718 INFO:tasks.workunit.client.0.smithi139.stdout:5/5: creat d1/f3 x:0 0 0 2024-09-16T00:09:35.718 INFO:tasks.workunit.client.0.smithi139.stdout:6/5: write f0 [1435960,22223] 0 2024-09-16T00:09:35.718 INFO:tasks.workunit.client.0.smithi139.stdout:0/17: creat f2 x:0 0 0 2024-09-16T00:09:35.718 INFO:tasks.workunit.client.0.smithi139.stdout:9/1: creat f1 x:0 0 0 2024-09-16T00:09:35.719 INFO:tasks.workunit.client.0.smithi139.stdout:4/14: creat f3 x:0 0 0 2024-09-16T00:09:35.719 INFO:tasks.workunit.client.0.smithi139.stdout:8/4: dread - f1 zero size 2024-09-16T00:09:35.720 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:09:35.722 INFO:tasks.workunit.client.0.smithi139.stdout:2/10: dread f1 [0,4194304] 0 2024-09-16T00:09:35.722 INFO:tasks.workunit.client.0.smithi139.stdout:0/18: chown f1 301 1 2024-09-16T00:09:35.722 INFO:tasks.workunit.client.0.smithi139.stdout:5/6: rmdir d1 39 2024-09-16T00:09:35.723 INFO:tasks.workunit.client.0.smithi139.stdout:6/6: write f0 [1392022,79143] 0 2024-09-16T00:09:35.723 INFO:tasks.workunit.client.0.smithi139.stdout:9/2: write f0 [7465,116142] 0 2024-09-16T00:09:35.723 INFO:tasks.workunit.client.0.smithi139.stdout:2/11: creat f2 x:0 0 0 2024-09-16T00:09:35.723 INFO:tasks.workunit.client.0.smithi139.stdout:9/3: creat f2 x:0 0 0 2024-09-16T00:09:35.723 INFO:tasks.workunit.client.0.smithi139.stdout:1/13: dwrite f0 [0,4194304] 0 2024-09-16T00:09:35.723 INFO:tasks.workunit.client.0.smithi139.stdout:9/4: creat f3 x:0 0 0 2024-09-16T00:09:35.723 INFO:tasks.workunit.client.0.smithi139.stdout:8/5: dwrite f0 [0,4194304] 0 2024-09-16T00:09:35.723 INFO:tasks.workunit.client.0.smithi139.stdout:1/14: dread f0 [0,4194304] 0 2024-09-16T00:09:35.723 INFO:tasks.workunit.client.0.smithi139.stdout:9/5: link f0 f4 0 2024-09-16T00:09:35.723 INFO:tasks.workunit.client.0.smithi139.stdout:8/6: rename f1 to f2 0 2024-09-16T00:09:35.723 INFO:tasks.workunit.client.0.smithi139.stdout:9/6: unlink f3 0 2024-09-16T00:09:35.723 INFO:tasks.workunit.client.0.smithi139.stdout:1/15: creat f4 x:0 0 0 2024-09-16T00:09:35.723 INFO:tasks.workunit.client.0.smithi139.stdout:8/7: rename f0 to f3 0 2024-09-16T00:09:35.724 INFO:tasks.workunit.client.0.smithi139.stdout:9/7: unlink f1 0 2024-09-16T00:09:35.724 INFO:tasks.workunit.client.0.smithi139.stdout:9/8: read - f2 zero size 2024-09-16T00:09:35.724 INFO:tasks.workunit.client.0.smithi139.stdout:9/9: chown f4 1857496973 1 2024-09-16T00:09:35.724 INFO:tasks.workunit.client.0.smithi139.stdout:9/10: dwrite f4 [0,4194304] 0 2024-09-16T00:09:35.724 INFO:tasks.workunit.client.0.smithi139.stdout:9/11: chown f4 102263477 1 2024-09-16T00:09:35.724 INFO:tasks.workunit.client.0.smithi139.stdout:9/12: rename f0 to f5 0 2024-09-16T00:09:35.724 INFO:tasks.workunit.client.0.smithi139.stdout:9/13: stat f4 0 2024-09-16T00:09:35.724 INFO:tasks.workunit.client.0.smithi139.stdout:9/14: dwrite f2 [0,4194304] 0 2024-09-16T00:09:35.724 INFO:tasks.workunit.client.0.smithi139.stdout:9/15: rmdir - no directory 2024-09-16T00:09:35.724 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:35.714+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:35.725 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:09:35.756 INFO:tasks.workunit.client.0.smithi139.stdout:0/19: fsync f1 0 2024-09-16T00:09:35.759 INFO:tasks.workunit.client.0.smithi139.stdout:0/20: creat f3 x:0 0 0 2024-09-16T00:09:35.759 INFO:tasks.workunit.client.0.smithi139.stdout:0/21: chown f3 0 1 2024-09-16T00:09:35.759 INFO:tasks.workunit.client.0.smithi139.stdout:0/22: dread - f2 zero size 2024-09-16T00:09:35.762 INFO:tasks.workunit.client.0.smithi139.stdout:6/7: getdents . 0 2024-09-16T00:09:35.762 INFO:tasks.workunit.client.0.smithi139.stdout:0/23: dread f1 [0,4194304] 0 2024-09-16T00:09:35.763 INFO:tasks.workunit.client.0.smithi139.stdout:3/16: chown l2 15093 1 2024-09-16T00:09:35.763 INFO:tasks.workunit.client.0.smithi139.stdout:3/17: chown l2 989847 1 2024-09-16T00:09:35.763 INFO:tasks.workunit.client.0.smithi139.stdout:6/8: mknod c2 0 2024-09-16T00:09:35.763 INFO:tasks.workunit.client.0.smithi139.stdout:3/18: chown l2 12 1 2024-09-16T00:09:35.764 INFO:tasks.workunit.client.0.smithi139.stdout:3/19: rmdir - no directory 2024-09-16T00:09:35.765 INFO:tasks.workunit.client.0.smithi139.stdout:6/9: creat f3 x:0 0 0 2024-09-16T00:09:35.766 INFO:tasks.workunit.client.0.smithi139.stdout:6/10: write f0 [2498473,120644] 0 2024-09-16T00:09:35.766 INFO:tasks.workunit.client.0.smithi139.stdout:3/20: rename l2 to l3 0 2024-09-16T00:09:35.766 INFO:tasks.workunit.client.0.smithi139.stdout:3/21: dwrite - no filename 2024-09-16T00:09:35.768 INFO:tasks.workunit.client.0.smithi139.stdout:0/24: link f1 f4 0 2024-09-16T00:09:35.771 INFO:tasks.workunit.client.0.smithi139.stdout:0/25: mknod c5 0 2024-09-16T00:09:35.772 INFO:tasks.workunit.client.0.smithi139.stdout:0/26: dread - f3 zero size 2024-09-16T00:09:35.774 INFO:tasks.workunit.client.0.smithi139.stdout:6/11: dwrite f3 [0,4194304] 0 2024-09-16T00:09:35.775 INFO:tasks.workunit.client.0.smithi139.stdout:0/27: mkdir d6 0 2024-09-16T00:09:35.775 INFO:tasks.workunit.client.0.smithi139.stdout:0/28: read - f2 zero size 2024-09-16T00:09:35.775 INFO:tasks.workunit.client.0.smithi139.stdout:5/7: getdents d1 0 2024-09-16T00:09:35.776 INFO:tasks.workunit.client.0.smithi139.stdout:5/8: dread - d1/f3 zero size 2024-09-16T00:09:35.776 INFO:tasks.workunit.client.0.smithi139.stdout:6/12: mknod c4 0 2024-09-16T00:09:35.778 INFO:tasks.workunit.client.0.smithi139.stdout:5/9: truncate d1/f3 602239 0 2024-09-16T00:09:35.780 INFO:tasks.workunit.client.0.smithi139.stdout:2/12: getdents . 0 2024-09-16T00:09:35.781 INFO:tasks.workunit.client.0.smithi139.stdout:5/10: mknod d1/c4 0 2024-09-16T00:09:35.781 INFO:tasks.workunit.client.0.smithi139.stdout:5/11: readlink - no filename 2024-09-16T00:09:35.783 INFO:tasks.workunit.client.0.smithi139.stdout:5/12: stat d1/c4 0 2024-09-16T00:09:35.785 INFO:tasks.workunit.client.0.smithi139.stdout:2/13: rename f1 to f3 0 2024-09-16T00:09:35.787 INFO:tasks.workunit.client.0.smithi139.stdout:2/14: read - f2 zero size 2024-09-16T00:09:35.787 INFO:tasks.workunit.client.0.smithi139.stdout:5/13: symlink d1/l5 0 2024-09-16T00:09:35.788 INFO:tasks.workunit.client.0.smithi139.stdout:8/8: rename f3 to f4 0 2024-09-16T00:09:35.790 INFO:tasks.workunit.client.0.smithi139.stdout:0/29: dwrite f1 [0,4194304] 0 2024-09-16T00:09:35.790 INFO:tasks.workunit.client.0.smithi139.stdout:5/14: readlink d1/l5 0 2024-09-16T00:09:35.790 INFO:tasks.workunit.client.0.smithi139.stdout:6/13: dwrite f0 [0,4194304] 0 2024-09-16T00:09:35.791 INFO:tasks.workunit.client.0.smithi139.stdout:5/15: chown d1/l5 0 1 2024-09-16T00:09:35.791 INFO:tasks.workunit.client.0.smithi139.stdout:2/15: symlink l4 0 2024-09-16T00:09:35.792 INFO:tasks.workunit.client.0.smithi139.stdout:8/9: mknod c5 0 2024-09-16T00:09:35.793 INFO:tasks.workunit.client.0.smithi139.stdout:6/14: creat f5 x:0 0 0 2024-09-16T00:09:35.794 INFO:tasks.workunit.client.0.smithi139.stdout:9/16: truncate f2 1317321 0 2024-09-16T00:09:35.796 INFO:tasks.workunit.client.0.smithi139.stdout:0/30: rename f4 to d6/f7 0 2024-09-16T00:09:35.798 INFO:tasks.workunit.client.0.smithi139.stdout:5/16: mkdir d1/d6 0 2024-09-16T00:09:35.799 INFO:tasks.workunit.client.0.smithi139.stdout:2/16: fdatasync f3 0 2024-09-16T00:09:35.799 INFO:tasks.workunit.client.0.smithi139.stdout:5/17: write d1/f3 [1210009,27828] 0 2024-09-16T00:09:35.801 INFO:tasks.workunit.client.0.smithi139.stdout:2/17: truncate f2 13926 0 2024-09-16T00:09:35.802 INFO:tasks.workunit.client.0.smithi139.stdout:5/18: write d1/f3 [1724091,49575] 0 2024-09-16T00:09:35.802 INFO:tasks.workunit.client.0.smithi139.stdout:6/15: unlink f5 0 2024-09-16T00:09:35.805 INFO:tasks.workunit.client.0.smithi139.stdout:0/31: dwrite f2 [0,4194304] 0 2024-09-16T00:09:35.808 INFO:tasks.workunit.client.0.smithi139.stdout:6/16: dread f3 [0,4194304] 0 2024-09-16T00:09:35.813 INFO:tasks.workunit.client.0.smithi139.stdout:5/19: dread d1/f3 [0,4194304] 0 2024-09-16T00:09:35.814 INFO:tasks.workunit.client.0.smithi139.stdout:5/20: chown d1/f3 91643937 1 2024-09-16T00:09:35.816 INFO:tasks.workunit.client.0.smithi139.stdout:6/17: dwrite f3 [0,4194304] 0 2024-09-16T00:09:35.816 INFO:tasks.workunit.client.0.smithi139.stdout:3/22: rename l3 to l4 0 2024-09-16T00:09:35.817 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:35.815+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:35.820 INFO:tasks.workunit.client.0.smithi139.stdout:8/10: link c5 c6 0 2024-09-16T00:09:35.824 INFO:tasks.workunit.client.0.smithi139.stdout:0/32: rename f1 to d6/f8 0 2024-09-16T00:09:35.825 INFO:tasks.workunit.client.0.smithi139.stdout:8/11: creat f7 x:0 0 0 2024-09-16T00:09:35.827 INFO:tasks.workunit.client.0.smithi139.stdout:0/33: symlink d6/l9 0 2024-09-16T00:09:35.828 INFO:tasks.workunit.client.0.smithi139.stdout:0/34: stat d6/f7 0 2024-09-16T00:09:35.831 INFO:tasks.workunit.client.0.smithi139.stdout:8/12: chown c6 1434112 1 2024-09-16T00:09:35.831 INFO:tasks.workunit.client.0.smithi139.stdout:5/21: rmdir d1/d6 0 2024-09-16T00:09:35.831 INFO:tasks.workunit.client.0.smithi139.stdout:6/18: dwrite f3 [0,4194304] 0 2024-09-16T00:09:35.831 INFO:tasks.workunit.client.0.smithi139.stdout:8/13: chown f2 5592 1 2024-09-16T00:09:35.831 INFO:tasks.workunit.client.0.smithi139.stdout:0/35: creat d6/fa x:0 0 0 2024-09-16T00:09:35.832 INFO:tasks.workunit.client.0.smithi139.stdout:5/22: unlink d1/f3 0 2024-09-16T00:09:35.832 INFO:tasks.workunit.client.0.smithi139.stdout:5/23: write - no filename 2024-09-16T00:09:35.832 INFO:tasks.workunit.client.0.smithi139.stdout:5/24: truncate - no filename 2024-09-16T00:09:35.832 INFO:tasks.workunit.client.0.smithi139.stdout:5/25: fdatasync - no filename 2024-09-16T00:09:35.832 INFO:tasks.workunit.client.0.smithi139.stdout:5/26: fdatasync - no filename 2024-09-16T00:09:35.832 INFO:tasks.workunit.client.0.smithi139.stdout:5/27: dwrite - no filename 2024-09-16T00:09:35.832 INFO:tasks.workunit.client.0.smithi139.stdout:5/28: dwrite - no filename 2024-09-16T00:09:35.833 INFO:tasks.workunit.client.0.smithi139.stdout:6/19: creat f6 x:0 0 0 2024-09-16T00:09:35.834 INFO:tasks.workunit.client.0.smithi139.stdout:8/14: creat f8 x:0 0 0 2024-09-16T00:09:35.835 INFO:tasks.workunit.client.0.smithi139.stdout:2/18: dwrite f3 [0,4194304] 0 2024-09-16T00:09:35.835 INFO:tasks.workunit.client.0.smithi139.stdout:0/36: mkdir d6/db 0 2024-09-16T00:09:35.837 INFO:tasks.workunit.client.0.smithi139.stdout:3/23: unlink l4 0 2024-09-16T00:09:35.837 INFO:tasks.workunit.client.0.smithi139.stdout:5/29: rename d1/c4 to d1/c7 0 2024-09-16T00:09:35.837 INFO:tasks.workunit.client.0.smithi139.stdout:3/24: write - no filename 2024-09-16T00:09:35.837 INFO:tasks.workunit.client.0.smithi139.stdout:3/25: dread - no filename 2024-09-16T00:09:35.837 INFO:tasks.workunit.client.0.smithi139.stdout:3/26: truncate - no filename 2024-09-16T00:09:35.837 INFO:tasks.workunit.client.0.smithi139.stdout:3/27: truncate - no filename 2024-09-16T00:09:35.837 INFO:tasks.workunit.client.0.smithi139.stdout:3/28: stat - no entries 2024-09-16T00:09:35.838 INFO:tasks.workunit.client.0.smithi139.stdout:5/30: chown d1/l5 4651 1 2024-09-16T00:09:35.838 INFO:tasks.workunit.client.0.smithi139.stdout:6/20: unlink f0 0 2024-09-16T00:09:35.840 INFO:tasks.workunit.client.0.smithi139.stdout:2/19: creat f5 x:0 0 0 2024-09-16T00:09:35.840 INFO:tasks.workunit.client.0.smithi139.stdout:6/21: dread - f6 zero size 2024-09-16T00:09:35.842 INFO:tasks.workunit.client.0.smithi139.stdout:8/15: creat f9 x:0 0 0 2024-09-16T00:09:35.842 INFO:tasks.workunit.client.0.smithi139.stdout:0/37: mkdir d6/dc 0 2024-09-16T00:09:35.842 INFO:tasks.workunit.client.0.smithi139.stdout:9/17: truncate f2 173033 0 2024-09-16T00:09:35.842 INFO:tasks.workunit.client.0.smithi139.stdout:9/18: readlink - no filename 2024-09-16T00:09:35.843 INFO:tasks.workunit.client.0.smithi139.stdout:5/31: rmdir d1 39 2024-09-16T00:09:35.843 INFO:tasks.workunit.client.0.smithi139.stdout:5/32: dread - no filename 2024-09-16T00:09:35.843 INFO:tasks.workunit.client.0.smithi139.stdout:8/16: dread - f8 zero size 2024-09-16T00:09:35.844 INFO:tasks.workunit.client.0.smithi139.stdout:0/38: write f3 [437316,36817] 0 2024-09-16T00:09:35.844 INFO:tasks.workunit.client.0.smithi139.stdout:6/22: creat f7 x:0 0 0 2024-09-16T00:09:35.845 INFO:tasks.workunit.client.0.smithi139.stdout:0/39: chown d6/dc 610 1 2024-09-16T00:09:35.846 INFO:tasks.workunit.client.0.smithi139.stdout:6/23: stat l1 0 2024-09-16T00:09:35.846 INFO:tasks.workunit.client.0.smithi139.stdout:5/33: readlink d1/l5 0 2024-09-16T00:09:35.847 INFO:tasks.workunit.client.0.smithi139.stdout:5/34: write - no filename 2024-09-16T00:09:35.847 INFO:tasks.workunit.client.0.smithi139.stdout:5/35: dread - no filename 2024-09-16T00:09:35.847 INFO:tasks.workunit.client.0.smithi139.stdout:5/36: write - no filename 2024-09-16T00:09:35.847 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:35.847 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:35.847 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:35.847 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:35.848 INFO:tasks.workunit.client.0.smithi139.stdout:6/24: mkdir d8 0 2024-09-16T00:09:35.851 INFO:tasks.workunit.client.0.smithi139.stdout:5/37: creat d1/f8 x:0 0 0 2024-09-16T00:09:35.851 INFO:tasks.workunit.client.0.smithi139.stdout:2/20: dwrite f3 [4194304,4194304] 0 2024-09-16T00:09:35.851 INFO:tasks.workunit.client.0.smithi139.stdout:2/21: rmdir - no directory 2024-09-16T00:09:35.852 INFO:tasks.workunit.client.0.smithi139.stdout:3/29: getdents . 0 2024-09-16T00:09:35.852 INFO:tasks.workunit.client.0.smithi139.stdout:3/30: dwrite - no filename 2024-09-16T00:09:35.852 INFO:tasks.workunit.client.0.smithi139.stdout:3/31: dwrite - no filename 2024-09-16T00:09:35.854 INFO:tasks.workunit.client.0.smithi139.stdout:6/25: mkdir d8/d9 0 2024-09-16T00:09:35.855 INFO:tasks.workunit.client.0.smithi139.stdout:2/22: write f3 [6374303,63672] 0 2024-09-16T00:09:35.856 INFO:tasks.workunit.client.0.smithi139.stdout:9/19: dwrite f5 [0,4194304] 0 2024-09-16T00:09:35.856 INFO:tasks.workunit.client.0.smithi139.stdout:8/17: fsync f9 0 2024-09-16T00:09:35.856 INFO:tasks.workunit.client.0.smithi139.stdout:5/38: truncate d1/f8 373612 0 2024-09-16T00:09:35.857 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:09:35.858 INFO:tasks.workunit.client.0.smithi139.stdout:3/32: mkdir d5 0 2024-09-16T00:09:35.858 INFO:tasks.workunit.client.0.smithi139.stdout:3/33: chown d5 30531 1 2024-09-16T00:09:35.858 INFO:tasks.workunit.client.0.smithi139.stdout:3/34: write - no filename 2024-09-16T00:09:35.858 INFO:tasks.workunit.client.0.smithi139.stdout:2/23: creat f6 x:0 0 0 2024-09-16T00:09:35.860 INFO:tasks.workunit.client.0.smithi139.stdout:2/24: write f2 [777360,66261] 0 2024-09-16T00:09:35.860 INFO:tasks.workunit.client.0.smithi139.stdout:8/18: creat fa x:0 0 0 2024-09-16T00:09:35.861 INFO:tasks.workunit.client.0.smithi139.stdout:0/40: write d6/f8 [2029544,50966] 0 2024-09-16T00:09:35.861 INFO:tasks.workunit.client.0.smithi139.stdout:5/39: fsync d1/f8 0 2024-09-16T00:09:35.862 INFO:tasks.workunit.client.0.smithi139.stdout:0/41: readlink d6/l9 0 2024-09-16T00:09:35.863 INFO:tasks.workunit.client.0.smithi139.stdout:9/20: dread f5 [0,4194304] 0 2024-09-16T00:09:35.864 INFO:tasks.workunit.client.0.smithi139.stdout:5/40: dread d1/f8 [0,4194304] 0 2024-09-16T00:09:35.865 INFO:tasks.workunit.client.0.smithi139.stdout:5/41: write d1/f8 [878150,45727] 0 2024-09-16T00:09:35.868 INFO:tasks.workunit.client.0.smithi139.stdout:3/35: symlink d5/l6 0 2024-09-16T00:09:35.868 INFO:tasks.workunit.client.0.smithi139.stdout:3/36: dwrite - no filename 2024-09-16T00:09:35.868 INFO:tasks.workunit.client.0.smithi139.stdout:3/37: write - no filename 2024-09-16T00:09:35.868 INFO:tasks.workunit.client.0.smithi139.stdout:3/38: write - no filename 2024-09-16T00:09:35.868 INFO:tasks.workunit.client.0.smithi139.stdout:3/39: dwrite - no filename 2024-09-16T00:09:35.868 INFO:tasks.workunit.client.0.smithi139.stdout:3/40: dread - no filename 2024-09-16T00:09:35.869 INFO:tasks.workunit.client.0.smithi139.stdout:6/26: link c2 d8/ca 0 2024-09-16T00:09:35.870 INFO:tasks.workunit.client.0.smithi139.stdout:5/42: mknod d1/c9 0 2024-09-16T00:09:35.872 INFO:tasks.workunit.client.0.smithi139.stdout:5/43: fdatasync d1/f8 0 2024-09-16T00:09:35.872 INFO:tasks.workunit.client.0.smithi139.stdout:0/42: mkdir d6/dc/dd 0 2024-09-16T00:09:35.873 INFO:tasks.workunit.client.0.smithi139.stdout:3/41: mkdir d5/d7 0 2024-09-16T00:09:35.873 INFO:tasks.workunit.client.0.smithi139.stdout:6/27: mkdir d8/db 0 2024-09-16T00:09:35.875 INFO:tasks.workunit.client.0.smithi139.stdout:8/19: dwrite f4 [4194304,4194304] 0 2024-09-16T00:09:35.877 INFO:tasks.workunit.client.0.smithi139.stdout:6/28: chown c2 32278122 1 2024-09-16T00:09:35.877 INFO:tasks.workunit.client.0.smithi139.stdout:8/20: dread - f8 zero size 2024-09-16T00:09:35.877 INFO:tasks.workunit.client.0.smithi139.stdout:3/42: unlink d5/l6 0 2024-09-16T00:09:35.877 INFO:tasks.workunit.client.0.smithi139.stdout:2/25: truncate f3 2236119 0 2024-09-16T00:09:35.879 INFO:tasks.workunit.client.0.smithi139.stdout:3/43: chown d5/d7 739502781 1 2024-09-16T00:09:35.879 INFO:tasks.workunit.client.0.smithi139.stdout:3/44: dwrite - no filename 2024-09-16T00:09:35.879 INFO:tasks.workunit.client.0.smithi139.stdout:3/45: readlink - no filename 2024-09-16T00:09:35.879 INFO:tasks.workunit.client.0.smithi139.stdout:3/46: write - no filename 2024-09-16T00:09:35.880 INFO:tasks.workunit.client.0.smithi139.stdout:8/21: write f9 [80048,89421] 0 2024-09-16T00:09:35.880 INFO:tasks.workunit.client.0.smithi139.stdout:3/47: chown d5 1034788278 1 2024-09-16T00:09:35.881 INFO:tasks.workunit.client.0.smithi139.stdout:3/48: read - no filename 2024-09-16T00:09:35.881 INFO:tasks.workunit.client.0.smithi139.stdout:3/49: truncate - no filename 2024-09-16T00:09:35.881 INFO:tasks.workunit.client.0.smithi139.stdout:3/50: write - no filename 2024-09-16T00:09:35.881 INFO:tasks.workunit.client.0.smithi139.stdout:9/21: dwrite f5 [0,4194304] 0 2024-09-16T00:09:35.882 INFO:tasks.workunit.client.0.smithi139.stdout:2/26: write f5 [966527,129909] 0 2024-09-16T00:09:35.884 INFO:tasks.workunit.client.0.smithi139.stdout:9/22: chown f4 1 1 2024-09-16T00:09:35.887 INFO:tasks.workunit.client.0.smithi139.stdout:2/27: chown c0 1473 1 2024-09-16T00:09:35.887 INFO:tasks.workunit.client.0.smithi139.stdout:0/43: getdents d6/dc 0 2024-09-16T00:09:35.888 INFO:tasks.workunit.client.0.smithi139.stdout:3/51: creat d5/f8 x:0 0 0 2024-09-16T00:09:35.888 INFO:tasks.workunit.client.0.smithi139.stdout:2/28: readlink l4 0 2024-09-16T00:09:35.889 INFO:tasks.workunit.client.0.smithi139.stdout:6/29: link f6 d8/d9/fc 0 2024-09-16T00:09:35.890 INFO:tasks.workunit.client.0.smithi139.stdout:5/44: dwrite d1/f8 [0,4194304] 0 2024-09-16T00:09:35.892 INFO:tasks.workunit.client.0.smithi139.stdout:2/29: unlink f5 0 2024-09-16T00:09:35.892 INFO:tasks.workunit.client.0.smithi139.stdout:3/52: write d5/f8 [592217,51092] 0 2024-09-16T00:09:35.893 INFO:tasks.workunit.client.0.smithi139.stdout:5/45: write d1/f8 [1717746,118483] 0 2024-09-16T00:09:35.896 INFO:tasks.workunit.client.0.smithi139.stdout:2/30: truncate f6 596665 0 2024-09-16T00:09:35.896 INFO:tasks.workunit.client.0.smithi139.stdout:8/22: dwrite fa [0,4194304] 0 2024-09-16T00:09:35.898 INFO:tasks.workunit.client.0.smithi139.stdout:0/44: symlink d6/dc/dd/le 0 2024-09-16T00:09:35.898 INFO:tasks.workunit.client.0.smithi139.stdout:8/23: dread - f7 zero size 2024-09-16T00:09:35.901 INFO:tasks.workunit.client.0.smithi139.stdout:9/23: dread f4 [0,4194304] 0 2024-09-16T00:09:35.902 INFO:tasks.workunit.client.0.smithi139.stdout:5/46: chown d1/c7 0 1 2024-09-16T00:09:35.902 INFO:tasks.workunit.client.0.smithi139.stdout:3/53: getdents d5/d7 0 2024-09-16T00:09:35.902 INFO:tasks.workunit.client.0.smithi139.stdout:0/45: link f2 d6/ff 0 2024-09-16T00:09:35.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.901+0000 7f87e405e640 1 Processor -- start 2024-09-16T00:09:35.903 INFO:tasks.workunit.client.0.smithi139.stdout:0/46: fdatasync f3 0 2024-09-16T00:09:35.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.901+0000 7f87e405e640 1 -- start start 2024-09-16T00:09:35.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.902+0000 7f87e405e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f87dc0aed70 0x7f87dc1514a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:35.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.902+0000 7f87e405e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f87dc0c82e0 0x7f87dc1519e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:35.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.902+0000 7f87e405e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87dc0718e0 con 0x7f87dc0c7f00 2024-09-16T00:09:35.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.902+0000 7f87e405e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f87dc0c11a0 con 0x7f87dc0c82e0 2024-09-16T00:09:35.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.902+0000 7f87e405e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f87dc151f20 con 0x7f87dc0aed70 2024-09-16T00:09:35.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.903+0000 7f87e1dfa640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f87dc0c7f00 0x7f87dc0c1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:38732/0 (socket says 172.21.15.139:38732) 2024-09-16T00:09:35.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.903+0000 7f87e1dfa640 1 -- 172.21.15.139:0/689981202 learned_addr learned my addr 172.21.15.139:0/689981202 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:35.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.903+0000 7f87e15f9640 1 --2- 172.21.15.139:0/689981202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f87dc0aed70 0x7f87dc1514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:35.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.903+0000 7f87e25fb640 1 --2- 172.21.15.139:0/689981202 >> v2:172.21.15.139:3300/0 conn(0x7f87dc0c82e0 0x7f87dc1519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:35.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.903+0000 7f87e0df8640 1 -- 172.21.15.139:0/689981202 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1554923341 0 0) 0x7f87dc0718e0 con 0x7f87dc0c7f00 2024-09-16T00:09:35.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.903+0000 7f87e15f9640 1 -- 172.21.15.139:0/689981202 >> v2:172.21.15.139:3300/0 conn(0x7f87dc0c82e0 msgr2=0x7f87dc1519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:35.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.903+0000 7f87e15f9640 1 --2- 172.21.15.139:0/689981202 >> v2:172.21.15.139:3300/0 conn(0x7f87dc0c82e0 0x7f87dc1519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:35.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.903+0000 7f87e15f9640 1 -- 172.21.15.139:0/689981202 >> v1:172.21.15.164:6789/0 conn(0x7f87dc0c7f00 legacy=0x7f87dc0c1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:35.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.903+0000 7f87e15f9640 1 -- 172.21.15.139:0/689981202 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc071b50 con 0x7f87dc0aed70 2024-09-16T00:09:35.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.903+0000 7f87e25fb640 1 --2- 172.21.15.139:0/689981202 >> v2:172.21.15.139:3300/0 conn(0x7f87dc0c82e0 0x7f87dc1519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:09:35.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.904+0000 7f87e15f9640 1 --2- 172.21.15.139:0/689981202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f87dc0aed70 0x7f87dc1514a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0097a0 tx=0x7f87cc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ce9c2e5fa4f6e63 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:35.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.904+0000 7f87e0df8640 1 -- 172.21.15.139:0/689981202 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc037070 con 0x7f87dc0aed70 2024-09-16T00:09:35.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.904+0000 7f87e0df8640 1 -- 172.21.15.139:0/689981202 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87cc032440 con 0x7f87dc0aed70 2024-09-16T00:09:35.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.904+0000 7f87e0df8640 1 -- 172.21.15.139:0/689981202 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc032810 con 0x7f87dc0aed70 2024-09-16T00:09:35.907 INFO:tasks.workunit.client.0.smithi139.stdout:0/47: symlink d6/l10 0 2024-09-16T00:09:35.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.905+0000 7f87e405e640 1 -- 172.21.15.139:0/689981202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f87dc0aed70 msgr2=0x7f87dc1514a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:35.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.905+0000 7f87e405e640 1 --2- 172.21.15.139:0/689981202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f87dc0aed70 0x7f87dc1514a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0097a0 tx=0x7f87cc02a210 comp rx=0 tx=0).stop 2024-09-16T00:09:35.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.905+0000 7f87e405e640 1 -- 172.21.15.139:0/689981202 shutdown_connections 2024-09-16T00:09:35.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.905+0000 7f87e405e640 1 --2- 172.21.15.139:0/689981202 >> v2:172.21.15.139:3300/0 conn(0x7f87dc0c82e0 0x7f87dc1519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:35.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.905+0000 7f87e405e640 1 --2- 172.21.15.139:0/689981202 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f87dc0aed70 0x7f87dc1514a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:35.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.905+0000 7f87e405e640 1 -- 172.21.15.139:0/689981202 >> 172.21.15.139:0/689981202 conn(0x7f87dc06e960 msgr2=0x7f87dc06f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:35.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.905+0000 7f87e405e640 1 -- 172.21.15.139:0/689981202 shutdown_connections 2024-09-16T00:09:35.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.905+0000 7f87e405e640 1 -- 172.21.15.139:0/689981202 wait complete. 2024-09-16T00:09:35.909 INFO:tasks.workunit.client.0.smithi139.stdout:0/48: fsync d6/fa 0 2024-09-16T00:09:35.909 INFO:tasks.workunit.client.0.smithi139.stdout:3/54: creat d5/d7/f9 x:0 0 0 2024-09-16T00:09:35.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.907+0000 7f87e405e640 1 Processor -- start 2024-09-16T00:09:35.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.907+0000 7f87e405e640 1 -- start start 2024-09-16T00:09:35.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e405e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87dc0aed70 0x7f87dc0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:35.910 INFO:tasks.workunit.client.0.smithi139.stdout:2/31: write f3 [2127511,79578] 0 2024-09-16T00:09:35.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e405e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f87dc0c7f00 0x7f87dc1533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:35.910 INFO:tasks.workunit.client.0.smithi139.stdout:0/49: fsync d6/fa 0 2024-09-16T00:09:35.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e405e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87dc0c82e0 0x7f87dc13c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:35.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e405e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc06ffa0 con 0x7f87dc0aed70 2024-09-16T00:09:35.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e405e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc072110 con 0x7f87dc0c82e0 2024-09-16T00:09:35.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e405e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f87dc0c32d0 con 0x7f87dc0c7f00 2024-09-16T00:09:35.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e1dfa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87dc0aed70 0x7f87dc0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:35.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e1dfa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87dc0aed70 0x7f87dc0cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:58890/0 (socket says 172.21.15.139:58890) 2024-09-16T00:09:35.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e1dfa640 1 -- 172.21.15.139:0/3533208476 learned_addr learned my addr 172.21.15.139:0/3533208476 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:35.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e15f9640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f87dc0c7f00 0x7f87dc1533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:35.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e25fb640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87dc0c82e0 0x7f87dc13c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:35.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e1dfa640 1 -- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f87dc0c7f00 msgr2=0x7f87dc1533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:35.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e1dfa640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f87dc0c7f00 0x7f87dc1533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:35.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e1dfa640 1 -- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87dc0c82e0 msgr2=0x7f87dc13c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:35.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e1dfa640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87dc0c82e0 0x7f87dc13c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:35.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e1dfa640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc13e160 con 0x7f87dc0aed70 2024-09-16T00:09:35.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e15f9640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f87dc0c7f00 0x7f87dc1533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:35.913 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.908+0000 7f87e25fb640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f87dc0c82e0 0x7f87dc13c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:35.913 INFO:tasks.workunit.client.0.smithi139.stdout:3/55: mkdir d5/da 0 2024-09-16T00:09:35.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.909+0000 7f87e1dfa640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87dc0aed70 0x7f87dc0cb270 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f87d800c4d0 tx=0x7f87d800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:35.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.909+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8018070 con 0x7f87dc0aed70 2024-09-16T00:09:35.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.909+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d80074e0 con 0x7f87dc0aed70 2024-09-16T00:09:35.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.909+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8004a20 con 0x7f87dc0aed70 2024-09-16T00:09:35.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.909+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87dc0cccb0 con 0x7f87dc0aed70 2024-09-16T00:09:35.914 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.909+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87dc13b790 con 0x7f87dc0aed70 2024-09-16T00:09:35.914 INFO:tasks.workunit.client.0.smithi139.stdout:5/47: dwrite d1/f8 [0,4194304] 0 2024-09-16T00:09:35.915 INFO:tasks.workunit.client.0.smithi139.stdout:9/24: write f2 [1200039,43038] 0 2024-09-16T00:09:35.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.910+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f87dc0c2720 con 0x7f87dc0aed70 2024-09-16T00:09:35.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.911+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f87d801b070 con 0x7f87dc0aed70 2024-09-16T00:09:35.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.912+0000 7f87d27fc640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f87c006c2e0 0x7f87c006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:35.915 INFO:tasks.workunit.client.0.smithi139.stdout:5/48: chown d1/l5 1130 1 2024-09-16T00:09:35.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.912+0000 7f87e15f9640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f87c006c2e0 0x7f87c006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:35.917 INFO:tasks.workunit.client.0.smithi139.stdout:9/25: fsync f4 0 2024-09-16T00:09:35.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.912+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5973+0+0 (secure 0 0 0) 0x7f87d808a860 con 0x7f87dc0aed70 2024-09-16T00:09:35.917 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.912+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v82) ==== 54+0+16 (secure 0 0 0) 0x7f87d808f240 con 0x7f87dc0aed70 2024-09-16T00:09:35.918 INFO:tasks.workunit.client.0.smithi139.stdout:3/56: rename d5/da to d5/da/db 22 2024-09-16T00:09:35.918 INFO:tasks.workunit.client.0.smithi139.stdout:9/26: stat f4 0 2024-09-16T00:09:35.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.913+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f87dc13d8d0 0x7f87dc0cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:35.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.913+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f87dc165050 con 0x7f87dc13d8d0 2024-09-16T00:09:35.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.913+0000 7f87e15f9640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f87c006c2e0 0x7f87c006e7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f87cc034000 tx=0x7f87cc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:35.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.914+0000 7f87e25fb640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f87dc13d8d0 0x7f87dc0cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:35.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.914+0000 7f87e25fb640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f87dc13d8d0 0x7f87dc0cbe90 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:35.919 INFO:tasks.workunit.client.0.smithi139.stdout:0/50: truncate f2 3830419 0 2024-09-16T00:09:35.919 INFO:tasks.workunit.client.0.smithi139.stdout:9/27: truncate f5 4263624 0 2024-09-16T00:09:35.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.916+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f87dc165050 con 0x7f87dc13d8d0 2024-09-16T00:09:35.920 INFO:tasks.workunit.client.0.smithi139.stdout:3/57: mkdir d5/d7/dc 0 2024-09-16T00:09:35.920 INFO:tasks.workunit.client.0.smithi139.stdout:2/32: truncate f6 540051 0 2024-09-16T00:09:35.921 INFO:tasks.workunit.client.0.smithi139.stdout:9/28: rename f4 to f6 0 2024-09-16T00:09:35.923 INFO:tasks.workunit.client.0.smithi139.stdout:3/58: link d5/d7/f9 d5/d7/fd 0 2024-09-16T00:09:35.923 INFO:tasks.workunit.client.0.smithi139.stdout:5/49: dwrite d1/f8 [0,4194304] 0 2024-09-16T00:09:35.924 INFO:tasks.workunit.client.0.smithi139.stdout:9/29: link f2 f7 0 2024-09-16T00:09:35.927 INFO:tasks.workunit.client.0.smithi139.stdout:9/30: rename f7 to f8 0 2024-09-16T00:09:35.927 INFO:tasks.workunit.client.0.smithi139.stdout:3/59: dread d5/f8 [0,4194304] 0 2024-09-16T00:09:35.932 INFO:tasks.workunit.client.0.smithi139.stdout:3/60: mknod d5/d7/dc/ce 0 2024-09-16T00:09:35.934 INFO:tasks.workunit.client.0.smithi139.stdout:3/61: unlink d5/d7/f9 0 2024-09-16T00:09:35.935 INFO:tasks.workunit.client.0.smithi139.stdout:9/31: rename f8 to f9 0 2024-09-16T00:09:35.936 INFO:tasks.workunit.client.0.smithi139.stdout:3/62: symlink d5/d7/dc/lf 0 2024-09-16T00:09:35.936 INFO:tasks.workunit.client.0.smithi139.stdout:5/50: truncate d1/f8 519434 0 2024-09-16T00:09:35.937 INFO:tasks.workunit.client.0.smithi139.stdout:5/51: chown d1 1 1 2024-09-16T00:09:35.938 INFO:tasks.workunit.client.0.smithi139.stdout:0/51: write d6/ff [1814280,88910] 0 2024-09-16T00:09:35.938 INFO:tasks.workunit.client.0.smithi139.stdout:3/63: rmdir d5/d7/dc 39 2024-09-16T00:09:35.938 INFO:tasks.workunit.client.0.smithi139.stdout:3/64: chown d5/f8 91259 1 2024-09-16T00:09:35.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.938+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f87dc06f7b0 con 0x7f87dc13d8d0 2024-09-16T00:09:35.940 INFO:tasks.workunit.client.0.smithi139.stdout:0/52: creat d6/dc/dd/f11 x:0 0 0 2024-09-16T00:09:35.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.940+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f87dc06f7b0 con 0x7f87dc13d8d0 2024-09-16T00:09:35.942 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:35.943 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:35.943 INFO:tasks.workunit.client.0.smithi139.stdout:3/65: symlink d5/d7/dc/l10 0 2024-09-16T00:09:35.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.941+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f87dc1f1790 0x7f87dc1f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:35.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.941+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f87dc13cbe0 con 0x7f87dc1f1790 2024-09-16T00:09:35.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.941+0000 7f87e1dfa640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f87dc1f1790 0x7f87dc1f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:35.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.941+0000 7f87e1dfa640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f87dc1f1790 0x7f87dc1f1b70 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:35.944 INFO:tasks.workunit.client.0.smithi139.stdout:0/53: dread d6/ff [0,4194304] 0 2024-09-16T00:09:35.945 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.942+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f87dc13cbe0 con 0x7f87dc1f1790 2024-09-16T00:09:35.945 INFO:tasks.workunit.client.0.smithi139.stdout:3/66: dread - d5/d7/fd zero size 2024-09-16T00:09:35.945 INFO:tasks.workunit.client.0.smithi139.stdout:3/67: stat d5/f8 0 2024-09-16T00:09:35.945 INFO:tasks.workunit.client.0.smithi139.stdout:5/52: rename d1/f8 to d1/fa 0 2024-09-16T00:09:35.946 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:35.941+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:35.949 INFO:tasks.workunit.client.0.smithi139.stdout:0/54: rename d6/l9 to d6/dc/dd/l12 0 2024-09-16T00:09:35.950 INFO:tasks.workunit.client.0.smithi139.stdout:9/32: dwrite f9 [0,4194304] 0 2024-09-16T00:09:35.952 INFO:tasks.workunit.client.0.smithi139.stdout:0/55: truncate d6/dc/dd/f11 915571 0 2024-09-16T00:09:35.952 INFO:tasks.workunit.client.0.smithi139.stdout:0/56: dread - d6/fa zero size 2024-09-16T00:09:35.959 INFO:tasks.workunit.client.0.smithi139.stdout:3/68: dread - d5/d7/fd zero size 2024-09-16T00:09:35.961 INFO:tasks.workunit.client.0.smithi139.stdout:9/33: truncate f2 1879734 0 2024-09-16T00:09:35.963 INFO:tasks.workunit.client.0.smithi139.stdout:3/69: creat d5/da/f11 x:0 0 0 2024-09-16T00:09:35.964 INFO:tasks.workunit.client.0.smithi139.stdout:0/57: truncate d6/f7 3789512 0 2024-09-16T00:09:35.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.962+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f87dc1f20b0 con 0x7f87dc1f1790 2024-09-16T00:09:35.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.964+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f87dc1f20b0 con 0x7f87dc1f1790 2024-09-16T00:09:35.966 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:35.966 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:35.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.964+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f87dc0ca490 0x7f87dc0ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:35.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.964+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f87dc0cadd0 con 0x7f87dc0ca490 2024-09-16T00:09:35.967 INFO:tasks.workunit.client.0.smithi139.stdout:0/58: rmdir d6/dc/dd 39 2024-09-16T00:09:35.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.964+0000 7f87e15f9640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f87dc0ca490 0x7f87dc0ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:35.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.965+0000 7f87e15f9640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f87dc0ca490 0x7f87dc0ca890 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:35.967 INFO:tasks.workunit.client.0.smithi139.stdout:9/34: dread f6 [0,4194304] 0 2024-09-16T00:09:35.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.965+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f87dc0cadd0 con 0x7f87dc0ca490 2024-09-16T00:09:35.968 INFO:tasks.workunit.client.0.smithi139.stdout:0/59: dread - d6/fa zero size 2024-09-16T00:09:35.968 INFO:tasks.workunit.client.0.smithi139.stdout:5/53: dwrite d1/fa [0,4194304] 0 2024-09-16T00:09:35.968 INFO:tasks.workunit.client.0.smithi139.stdout:3/70: unlink d5/d7/dc/lf 0 2024-09-16T00:09:35.970 INFO:tasks.workunit.client.0.smithi139.stdout:3/71: truncate d5/da/f11 721640 0 2024-09-16T00:09:35.973 INFO:tasks.workunit.client.0.smithi139.stdout:9/35: dread f6 [0,4194304] 0 2024-09-16T00:09:35.974 INFO:tasks.workunit.client.0.smithi139.stdout:3/72: truncate d5/d7/fd 957037 0 2024-09-16T00:09:35.977 INFO:tasks.workunit.client.0.smithi139.stdout:9/36: creat fa x:0 0 0 2024-09-16T00:09:35.977 INFO:tasks.workunit.client.0.smithi139.stdout:5/54: link d1/c7 d1/cb 0 2024-09-16T00:09:35.977 INFO:tasks.workunit.client.0.smithi139.stdout:5/55: read d1/fa [2653946,26706] 0 2024-09-16T00:09:35.979 INFO:tasks.workunit.client.0.smithi139.stdout:9/37: dread f9 [0,4194304] 0 2024-09-16T00:09:35.981 INFO:tasks.workunit.client.0.smithi139.stdout:5/56: dread d1/fa [0,4194304] 0 2024-09-16T00:09:35.981 INFO:tasks.workunit.client.0.smithi139.stdout:0/60: truncate d6/f7 2137521 0 2024-09-16T00:09:35.982 INFO:tasks.workunit.client.0.smithi139.stdout:5/57: symlink d1/lc 0 2024-09-16T00:09:35.983 INFO:tasks.workunit.client.0.smithi139.stdout:5/58: chown d1/fa 5 1 2024-09-16T00:09:35.983 INFO:tasks.workunit.client.0.smithi139.stdout:5/59: rmdir d1 39 2024-09-16T00:09:35.984 INFO:tasks.workunit.client.0.smithi139.stdout:0/61: rmdir d6/db 0 2024-09-16T00:09:35.986 INFO:tasks.workunit.client.0.smithi139.stdout:5/60: mknod d1/cd 0 2024-09-16T00:09:35.987 INFO:tasks.workunit.client.0.smithi139.stdout:5/61: mknod d1/ce 0 2024-09-16T00:09:35.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.986+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f87dc062880 con 0x7f87dc0ca490 2024-09-16T00:09:35.988 INFO:tasks.workunit.client.0.smithi139.stdout:5/62: rename d1 to d1/df 22 2024-09-16T00:09:35.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.987+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f87dc062880 con 0x7f87dc0ca490 2024-09-16T00:09:35.989 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:35.990 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:35.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.988+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f87dc063360 0x7f87dc14da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:35.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.988+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f87dc14df50 con 0x7f87dc063360 2024-09-16T00:09:35.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.988+0000 7f87e25fb640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f87dc063360 0x7f87dc14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:35.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.988+0000 7f87e25fb640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f87dc063360 0x7f87dc14da10 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:35.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:35.989+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f87dc14df50 con 0x7f87dc063360 2024-09-16T00:09:35.992 INFO:tasks.workunit.client.0.smithi139.stdout:0/62: write d6/f7 [2699223,67501] 0 2024-09-16T00:09:35.993 INFO:tasks.workunit.client.0.smithi139.stdout:5/63: truncate d1/fa 3932189 0 2024-09-16T00:09:35.995 INFO:tasks.workunit.client.0.smithi139.stdout:5/64: getdents d1 0 2024-09-16T00:09:36.001 INFO:tasks.workunit.client.0.smithi139.stdout:5/65: write d1/fa [455509,9310] 0 2024-09-16T00:09:36.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.004+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f87dc14e430 con 0x7f87dc063360 2024-09-16T00:09:36.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.005+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f87dc14e430 con 0x7f87dc063360 2024-09-16T00:09:36.007 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:36.008 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:36.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.005+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f87dc02c640 0x7f87dc02caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:36.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.005+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f87dc14df50 con 0x7f87dc02c640 2024-09-16T00:09:36.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.006+0000 7f87e1dfa640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f87dc02c640 0x7f87dc02caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:36.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.006+0000 7f87e1dfa640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f87dc02c640 0x7f87dc02caa0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:36.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.007+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f87dc14df50 con 0x7f87dc02c640 2024-09-16T00:09:36.018 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:36.017+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:36.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.020+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f87dc14e430 con 0x7f87dc02c640 2024-09-16T00:09:36.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.022+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f87dc14e430 con 0x7f87dc02c640 2024-09-16T00:09:36.024 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:36.024 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:36.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.022+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f87dc02d4f0 0x7f87dc202aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:36.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.022+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f87dc14df50 con 0x7f87dc02d4f0 2024-09-16T00:09:36.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.022+0000 7f87e15f9640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f87dc02d4f0 0x7f87dc202aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:36.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.023+0000 7f87e15f9640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f87dc02d4f0 0x7f87dc202aa0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:36.025 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:36.025 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:36.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.023+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f87dc14df50 con 0x7f87dc02d4f0 2024-09-16T00:09:36.033 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:09:36.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.037+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f87dc14df50 con 0x7f87dc02d4f0 2024-09-16T00:09:36.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.038+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f87dc14df50 con 0x7f87dc02d4f0 2024-09-16T00:09:36.040 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:36.041 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:36.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.039+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f87dc02d970 0x7f87dc205500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:36.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.039+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f87dc2082d0 con 0x7f87dc02d970 2024-09-16T00:09:36.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.039+0000 7f87e25fb640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f87dc02d970 0x7f87dc205500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:36.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.040+0000 7f87e25fb640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f87dc02d970 0x7f87dc205500 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:36.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.041+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== osd.6 v2:172.21.15.164:6824/1908495807 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f87dc2082d0 con 0x7f87dc02d970 2024-09-16T00:09:36.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.054+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f87dc205a40 con 0x7f87dc02d970 2024-09-16T00:09:36.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.056+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== osd.6 v2:172.21.15.164:6824/1908495807 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f87dc205a40 con 0x7f87dc02d970 2024-09-16T00:09:36.058 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:09:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.056+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f87dc206160 0x7f87dc2065a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.056+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f87dc20b6c0 con 0x7f87dc206160 2024-09-16T00:09:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.056+0000 7f87e1dfa640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f87dc206160 0x7f87dc2065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.057+0000 7f87e1dfa640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f87dc206160 0x7f87dc2065a0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:36.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.058+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f87dc20b6c0 con 0x7f87dc206160 2024-09-16T00:09:36.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.071+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f87dc206bf0 con 0x7f87dc206160 2024-09-16T00:09:36.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87d27fc640 1 -- 172.21.15.139:0/3533208476 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f87dc206bf0 con 0x7f87dc206160 2024-09-16T00:09:36.075 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:36.076 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:36.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f87dc13d8d0 msgr2=0x7f87dc0cbe90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:36.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f87dc13d8d0 0x7f87dc0cbe90 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:36.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f87dc1f1790 msgr2=0x7f87dc1f1b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:36.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f87dc1f1790 0x7f87dc1f1b70 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:36.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f87dc0ca490 msgr2=0x7f87dc0ca890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:36.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f87dc0ca490 0x7f87dc0ca890 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:36.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f87dc063360 msgr2=0x7f87dc14da10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:36.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f87dc063360 0x7f87dc14da10 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:36.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f87dc02c640 msgr2=0x7f87dc02caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:36.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f87dc02c640 0x7f87dc02caa0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:36.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e25fb640 1 -- 172.21.15.139:0/3533208476 reap_dead start 2024-09-16T00:09:36.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f87dc02d4f0 msgr2=0x7f87dc202aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:36.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f87dc02d4f0 0x7f87dc202aa0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:36.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e25fb640 1 -- 172.21.15.139:0/3533208476 reap_dead start 2024-09-16T00:09:36.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f87dc02d970 msgr2=0x7f87dc205500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:36.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6824/1908495807,v1:172.21.15.164:6825/1908495807] conn(0x7f87dc02d970 0x7f87dc205500 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:36.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f87dc206160 msgr2=0x7f87dc2065a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:36.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f87dc206160 0x7f87dc2065a0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:36.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e25fb640 1 -- 172.21.15.139:0/3533208476 reap_dead start 2024-09-16T00:09:36.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f87c006c2e0 msgr2=0x7f87c006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:36.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f87c006c2e0 0x7f87c006e7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f87cc034000 tx=0x7f87cc02e000 comp rx=0 tx=0).stop 2024-09-16T00:09:36.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87dc0aed70 msgr2=0x7f87dc0cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:36.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87dc0aed70 0x7f87dc0cb270 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f87d800c4d0 tx=0x7f87d800c990 comp rx=0 tx=0).stop 2024-09-16T00:09:36.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 shutdown_connections 2024-09-16T00:09:36.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f87c006c2e0 0x7f87c006e7a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:36.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 --2- 172.21.15.139:0/3533208476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87dc0aed70 0x7f87dc0cb270 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:36.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 >> 172.21.15.139:0/3533208476 conn(0x7f87dc06e960 msgr2=0x7f87dc14f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:36.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 shutdown_connections 2024-09-16T00:09:36.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.073+0000 7f87e405e640 1 -- 172.21.15.139:0/3533208476 wait complete. 2024-09-16T00:09:36.120 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:36.119+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:36.178 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:36.178 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:36.178 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T00:09:36.178 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:09:36.178 INFO:tasks.thrashosds.thrasher:Killing osd 6, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T00:09:36.179 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-16T00:09:36.179 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:09:36.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.184+0000 7f421d7fa640 1 -- 172.21.15.139:0/2919908995 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v83) ==== 91+0+0 (secure 0 0 0) 0x7f420c05cc60 con 0x7f4220151850 2024-09-16T00:09:36.186 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is unset 2024-09-16T00:09:36.187 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:09:36.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.187+0000 7f4226e65640 1 -- 172.21.15.139:0/2919908995 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f420406c2b0 msgr2=0x7f420406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:36.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.187+0000 7f4226e65640 1 --2- 172.21.15.139:0/2919908995 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f420406c2b0 0x7f420406e770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4214037000 tx=0x7f4214032000 comp rx=0 tx=0).stop 2024-09-16T00:09:36.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.187+0000 7f4226e65640 1 -- 172.21.15.139:0/2919908995 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220151850 msgr2=0x7f42200c3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:36.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.187+0000 7f4226e65640 1 --2- 172.21.15.139:0/2919908995 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220151850 0x7f42200c3b00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f420c00c3c0 tx=0x7f420c00c880 comp rx=0 tx=0).stop 2024-09-16T00:09:36.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.187+0000 7f4226e65640 1 -- 172.21.15.139:0/2919908995 shutdown_connections 2024-09-16T00:09:36.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.187+0000 7f4226e65640 1 --2- 172.21.15.139:0/2919908995 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f420406c2b0 0x7f420406e770 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:36.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.187+0000 7f4226e65640 1 --2- 172.21.15.139:0/2919908995 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4220151c30 0x7f4220158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:36.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.187+0000 7f4226e65640 1 --2- 172.21.15.139:0/2919908995 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4220151850 0x7f42200c3b00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:36.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.187+0000 7f4226e65640 1 --2- 172.21.15.139:0/2919908995 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42200c7390 0x7f42201546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:36.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.187+0000 7f4226e65640 1 -- 172.21.15.139:0/2919908995 >> 172.21.15.139:0/2919908995 conn(0x7f422006cc30 msgr2=0x7f422006f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:36.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.187+0000 7f4226e65640 1 -- 172.21.15.139:0/2919908995 shutdown_connections 2024-09-16T00:09:36.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:36.187+0000 7f4226e65640 1 -- 172.21.15.139:0/2919908995 wait complete. 2024-09-16T00:09:36.214 INFO:tasks.ceph.osd.6:Stopped 2024-09-16T00:09:36.214 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.6 2024-09-16T00:09:36.214 INFO:tasks.thrashosds.thrasher:remote for osd 6 is ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T00:09:36.214 DEBUG:teuthology.orchestra.run.smithi164:> type ceph-objectstore-tool 2024-09-16T00:09:36.219 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:36.218+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:36.269 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T00:09:36.269 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op list-pgs 2024-09-16T00:09:36.354 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:36.353+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:36.360 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:36.360 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:36.369 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:09:36.422 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:36.422+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:36.523 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:36.522+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:36.528 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:36.528 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:36.537 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:09:36.624 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:36.622+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:36.685 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:36.685 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:36.693 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:09:36.725 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:36.724+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:36.797 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:36.797 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:36.806 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:09:36.849 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:36.848+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:36.911 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:36.911 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:36.921 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:09:36.927 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:36.925+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:37.028 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:37.026+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:37.076 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:37.076 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:37.086 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:09:37.129 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:37.128+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:37.202 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:37.202 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:37.211 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:09:37.230 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:37.228+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:37.331 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:37.329+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:37.357 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:37.357 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:37.366 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:09:37.449 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:37.448+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:37.509 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:37.509 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:37.517 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:09:37.521 INFO:tasks.workunit.client.0.smithi139.stdout:7/2: sync 2024-09-16T00:09:37.521 INFO:tasks.workunit.client.0.smithi139.stdout:7/3: dread - no filename 2024-09-16T00:09:37.533 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:37.532+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:37.546 INFO:tasks.workunit.client.0.smithi139.stdout:7/4: getdents . 0 2024-09-16T00:09:37.546 INFO:tasks.workunit.client.0.smithi139.stdout:7/5: read - no filename 2024-09-16T00:09:37.546 INFO:tasks.workunit.client.0.smithi139.stdout:7/6: dwrite - no filename 2024-09-16T00:09:37.547 INFO:tasks.workunit.client.0.smithi139.stdout:7/7: write - no filename 2024-09-16T00:09:37.547 INFO:tasks.workunit.client.0.smithi139.stdout:7/8: chown . 170 1 2024-09-16T00:09:37.547 INFO:tasks.workunit.client.0.smithi139.stdout:7/9: chown . 96 1 2024-09-16T00:09:37.547 INFO:tasks.workunit.client.0.smithi139.stdout:7/10: unlink - no file 2024-09-16T00:09:37.547 INFO:tasks.workunit.client.0.smithi139.stdout:7/11: mknod c0 0 2024-09-16T00:09:37.549 INFO:tasks.workunit.client.0.smithi139.stdout:7/12: unlink c0 0 2024-09-16T00:09:37.550 INFO:tasks.workunit.client.0.smithi139.stdout:7/13: creat f1 x:0 0 0 2024-09-16T00:09:37.551 INFO:tasks.workunit.client.0.smithi139.stdout:7/14: symlink l2 0 2024-09-16T00:09:37.552 INFO:tasks.workunit.client.0.smithi139.stdout:7/15: read - f1 zero size 2024-09-16T00:09:37.552 INFO:tasks.workunit.client.0.smithi139.stdout:7/16: fdatasync f1 0 2024-09-16T00:09:37.625 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:37.625 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:37.634 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:09:37.634 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:37.632+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:37.686 INFO:tasks.workunit.client.0.smithi139.stdout:4/15: sync 2024-09-16T00:09:37.687 INFO:tasks.workunit.client.0.smithi139.stdout:4/16: dread - f3 zero size 2024-09-16T00:09:37.687 INFO:tasks.workunit.client.0.smithi139.stdout:4/17: write f3 [180034,118249] 0 2024-09-16T00:09:37.712 INFO:tasks.workunit.client.0.smithi139.stdout:1/16: sync 2024-09-16T00:09:37.712 INFO:tasks.workunit.client.0.smithi139.stdout:6/30: sync 2024-09-16T00:09:37.712 INFO:tasks.workunit.client.0.smithi139.stdout:1/17: chown f2 2 1 2024-09-16T00:09:37.713 INFO:tasks.workunit.client.0.smithi139.stdout:6/31: truncate d8/d9/fc 738844 0 2024-09-16T00:09:37.715 INFO:tasks.workunit.client.0.smithi139.stdout:6/32: mknod d8/d9/cd 0 2024-09-16T00:09:37.716 INFO:tasks.workunit.client.0.smithi139.stdout:6/33: truncate f7 324856 0 2024-09-16T00:09:37.716 INFO:tasks.workunit.client.0.smithi139.stdout:6/34: chown d8 130507936 1 2024-09-16T00:09:37.719 INFO:tasks.workunit.client.0.smithi139.stdout:1/18: dwrite f4 [0,4194304] 0 2024-09-16T00:09:37.721 INFO:tasks.workunit.client.0.smithi139.stdout:1/19: write f2 [2527941,128833] 0 2024-09-16T00:09:37.723 INFO:tasks.workunit.client.0.smithi139.stdout:1/20: write f0 [2920219,71394] 0 2024-09-16T00:09:37.725 INFO:tasks.workunit.client.0.smithi139.stdout:6/35: dwrite f7 [0,4194304] 0 2024-09-16T00:09:37.728 INFO:tasks.workunit.client.0.smithi139.stdout:6/36: creat d8/fe x:0 0 0 2024-09-16T00:09:37.732 INFO:tasks.workunit.client.0.smithi139.stdout:1/21: dwrite f2 [0,4194304] 0 2024-09-16T00:09:37.734 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:37.733+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:37.737 INFO:tasks.workunit.client.0.smithi139.stdout:1/22: dread f2 [0,4194304] 0 2024-09-16T00:09:37.740 INFO:tasks.workunit.client.0.smithi139.stdout:1/23: dread f2 [0,4194304] 0 2024-09-16T00:09:37.743 INFO:tasks.workunit.client.0.smithi139.stdout:1/24: link f0 f5 0 2024-09-16T00:09:37.744 INFO:tasks.workunit.client.0.smithi139.stdout:1/25: creat f6 x:0 0 0 2024-09-16T00:09:37.745 INFO:tasks.workunit.client.0.smithi139.stdout:1/26: mkdir d7 0 2024-09-16T00:09:37.779 INFO:tasks.workunit.client.0.smithi139.stdout:6/37: fsync d8/d9/fc 0 2024-09-16T00:09:37.781 INFO:tasks.workunit.client.0.smithi139.stdout:1/27: truncate f4 2683455 0 2024-09-16T00:09:37.785 INFO:tasks.workunit.client.0.smithi139.stdout:1/28: dread f4 [0,4194304] 0 2024-09-16T00:09:37.787 INFO:tasks.workunit.client.0.smithi139.stdout:1/29: mkdir d7/d8 0 2024-09-16T00:09:37.792 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:37.792 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:37.793 INFO:tasks.workunit.client.0.smithi139.stdout:1/30: dwrite f6 [0,4194304] 0 2024-09-16T00:09:37.795 INFO:tasks.workunit.client.0.smithi139.stdout:1/31: rename f0 to d7/d8/f9 0 2024-09-16T00:09:37.798 INFO:tasks.workunit.client.0.smithi139.stdout:1/32: write d7/d8/f9 [3281970,63068] 0 2024-09-16T00:09:37.801 INFO:tasks.workunit.client.0.smithi139.stdout:1/33: link f4 d7/d8/fa 0 2024-09-16T00:09:37.801 INFO:tasks.workunit.client.0.smithi139.stdout:2/33: sync 2024-09-16T00:09:37.801 INFO:tasks.workunit.client.0.smithi139.stdout:8/24: sync 2024-09-16T00:09:37.805 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:09:37.805 INFO:tasks.workunit.client.0.smithi139.stdout:2/34: truncate f6 1210539 0 2024-09-16T00:09:37.806 INFO:tasks.workunit.client.0.smithi139.stdout:2/35: stat f6 0 2024-09-16T00:09:37.809 INFO:tasks.workunit.client.0.smithi139.stdout:8/25: dread fa [0,4194304] 0 2024-09-16T00:09:37.809 INFO:tasks.workunit.client.0.smithi139.stdout:2/36: rename f3 to f7 0 2024-09-16T00:09:37.810 INFO:tasks.workunit.client.0.smithi139.stdout:1/34: dwrite f4 [0,4194304] 0 2024-09-16T00:09:37.813 INFO:tasks.workunit.client.0.smithi139.stdout:1/35: creat d7/d8/fb x:0 0 0 2024-09-16T00:09:37.814 INFO:tasks.workunit.client.0.smithi139.stdout:2/37: rename f7 to f8 0 2024-09-16T00:09:37.815 INFO:tasks.workunit.client.0.smithi139.stdout:2/38: chown l4 139656 1 2024-09-16T00:09:37.817 INFO:tasks.workunit.client.0.smithi139.stdout:8/26: dwrite f4 [0,4194304] 0 2024-09-16T00:09:37.819 INFO:tasks.workunit.client.0.smithi139.stdout:8/27: write f7 [170382,63210] 0 2024-09-16T00:09:37.821 INFO:tasks.workunit.client.0.smithi139.stdout:2/39: truncate f8 1993097 0 2024-09-16T00:09:37.822 INFO:tasks.workunit.client.0.smithi139.stdout:8/28: rename fa to fb 0 2024-09-16T00:09:37.823 INFO:tasks.workunit.client.0.smithi139.stdout:1/36: dwrite f2 [0,4194304] 0 2024-09-16T00:09:37.824 INFO:tasks.workunit.client.0.smithi139.stdout:1/37: write f5 [2900425,93832] 0 2024-09-16T00:09:37.831 INFO:tasks.workunit.client.0.smithi139.stdout:1/38: truncate f6 3838289 0 2024-09-16T00:09:37.835 INFO:tasks.workunit.client.0.smithi139.stdout:1/39: read f6 [3226436,62023] 0 2024-09-16T00:09:37.835 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:37.834+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:37.835 INFO:tasks.workunit.client.0.smithi139.stdout:1/40: write d7/d8/fa [1586976,76060] 0 2024-09-16T00:09:37.837 INFO:tasks.workunit.client.0.smithi139.stdout:1/41: creat d7/fc x:0 0 0 2024-09-16T00:09:37.843 INFO:tasks.workunit.client.0.smithi139.stdout:1/42: dwrite d7/d8/f9 [0,4194304] 0 2024-09-16T00:09:37.843 INFO:tasks.workunit.client.0.smithi139.stdout:1/43: stat f6 0 2024-09-16T00:09:37.906 INFO:tasks.workunit.client.0.smithi139.stdout:3/73: sync 2024-09-16T00:09:37.907 INFO:tasks.workunit.client.0.smithi139.stdout:3/74: truncate d5/f8 1235644 0 2024-09-16T00:09:37.907 INFO:tasks.workunit.client.0.smithi139.stdout:3/75: fsync d5/da/f11 0 2024-09-16T00:09:37.909 INFO:tasks.workunit.client.0.smithi139.stdout:3/76: creat d5/d7/dc/f12 x:0 0 0 2024-09-16T00:09:37.911 INFO:tasks.workunit.client.0.smithi139.stdout:3/77: dread d5/da/f11 [0,4194304] 0 2024-09-16T00:09:37.914 INFO:tasks.workunit.client.0.smithi139.stdout:3/78: dread d5/d7/fd [0,4194304] 0 2024-09-16T00:09:37.914 INFO:tasks.workunit.client.0.smithi139.stdout:3/79: chown d5 443494 1 2024-09-16T00:09:37.915 INFO:tasks.workunit.client.0.smithi139.stdout:3/80: write d5/f8 [146097,29393] 0 2024-09-16T00:09:37.919 INFO:tasks.workunit.client.0.smithi139.stdout:3/81: link d5/da/f11 d5/d7/f13 0 2024-09-16T00:09:37.920 INFO:tasks.workunit.client.0.smithi139.stdout:3/82: write d5/d7/f13 [1603630,25500] 0 2024-09-16T00:09:37.921 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:37.921 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:37.922 INFO:tasks.workunit.client.0.smithi139.stdout:3/83: symlink d5/da/l14 0 2024-09-16T00:09:37.924 INFO:tasks.workunit.client.0.smithi139.stdout:3/84: fsync d5/d7/fd 0 2024-09-16T00:09:37.926 INFO:tasks.workunit.client.0.smithi139.stdout:3/85: dread - d5/d7/dc/f12 zero size 2024-09-16T00:09:37.928 INFO:tasks.workunit.client.0.smithi139.stdout:3/86: write d5/d7/fd [261470,30368] 0 2024-09-16T00:09:37.928 INFO:tasks.workunit.client.0.smithi139.stdout:3/87: readlink d5/da/l14 0 2024-09-16T00:09:37.929 INFO:tasks.workunit.client.0.smithi139.stdout:3/88: stat d5/d7/dc/l10 0 2024-09-16T00:09:37.931 INFO:tasks.workunit.client.0.smithi139.stdout:3/89: dread d5/f8 [0,4194304] 0 2024-09-16T00:09:37.932 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:09:37.932 INFO:tasks.workunit.client.0.smithi139.stdout:3/90: read d5/da/f11 [598513,113644] 0 2024-09-16T00:09:37.933 INFO:tasks.workunit.client.0.smithi139.stdout:3/91: creat d5/da/f15 x:0 0 0 2024-09-16T00:09:37.939 INFO:tasks.workunit.client.0.smithi139.stdout:3/92: dwrite d5/d7/dc/f12 [0,4194304] 0 2024-09-16T00:09:37.940 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:37.938+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:37.940 INFO:tasks.workunit.client.0.smithi139.stdout:3/93: truncate d5/da/f15 877344 0 2024-09-16T00:09:38.034 INFO:tasks.workunit.client.0.smithi139.stdout:6/38: sync 2024-09-16T00:09:38.035 INFO:tasks.workunit.client.0.smithi139.stdout:9/38: sync 2024-09-16T00:09:38.035 INFO:tasks.workunit.client.0.smithi139.stdout:0/63: sync 2024-09-16T00:09:38.035 INFO:tasks.workunit.client.0.smithi139.stdout:6/39: fsync f6 0 2024-09-16T00:09:38.035 INFO:tasks.workunit.client.0.smithi139.stdout:9/39: fdatasync f6 0 2024-09-16T00:09:38.036 INFO:tasks.workunit.client.0.smithi139.stdout:6/40: mknod d8/cf 0 2024-09-16T00:09:38.036 INFO:tasks.workunit.client.0.smithi139.stdout:9/40: truncate f2 2286540 0 2024-09-16T00:09:38.036 INFO:tasks.workunit.client.0.smithi139.stdout:9/41: readlink - no filename 2024-09-16T00:09:38.037 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:38.035+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:38.037 INFO:tasks.workunit.client.0.smithi139.stdout:9/42: creat fb x:0 0 0 2024-09-16T00:09:38.038 INFO:tasks.workunit.client.0.smithi139.stdout:6/41: mkdir d8/d9/d10 0 2024-09-16T00:09:38.038 INFO:tasks.workunit.client.0.smithi139.stdout:9/43: write f9 [732242,90966] 0 2024-09-16T00:09:38.039 INFO:tasks.workunit.client.0.smithi139.stdout:9/44: stat f9 0 2024-09-16T00:09:38.039 INFO:tasks.workunit.client.0.smithi139.stdout:6/42: creat d8/d9/d10/f11 x:0 0 0 2024-09-16T00:09:38.041 INFO:tasks.workunit.client.0.smithi139.stdout:9/45: rename f5 to fc 0 2024-09-16T00:09:38.041 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:38.041 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:38.042 INFO:tasks.workunit.client.0.smithi139.stdout:6/43: rename f7 to d8/f12 0 2024-09-16T00:09:38.043 INFO:tasks.workunit.client.0.smithi139.stdout:6/44: truncate d8/fe 90951 0 2024-09-16T00:09:38.043 INFO:tasks.workunit.client.0.smithi139.stdout:9/46: symlink ld 0 2024-09-16T00:09:38.044 INFO:tasks.workunit.client.0.smithi139.stdout:9/47: write f2 [2149658,5804] 0 2024-09-16T00:09:38.045 INFO:tasks.workunit.client.0.smithi139.stdout:9/48: chown fc 39 1 2024-09-16T00:09:38.050 INFO:tasks.workunit.client.0.smithi139.stdout:6/45: dwrite d8/d9/fc [0,4194304] 0 2024-09-16T00:09:38.052 INFO:tasks.workunit.client.0.smithi139.stdout:6/46: rmdir d8/d9 39 2024-09-16T00:09:38.052 INFO:tasks.workunit.client.0.smithi139.stdout:9/49: dread fc [0,4194304] 0 2024-09-16T00:09:38.053 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:09:38.055 INFO:tasks.workunit.client.0.smithi139.stdout:6/47: rename d8/d9/fc to d8/d9/d10/f13 0 2024-09-16T00:09:38.059 INFO:tasks.workunit.client.0.smithi139.stdout:9/50: dwrite f6 [0,4194304] 0 2024-09-16T00:09:38.063 INFO:tasks.workunit.client.0.smithi139.stdout:9/51: dread fc [0,4194304] 0 2024-09-16T00:09:38.063 INFO:tasks.workunit.client.0.smithi139.stdout:9/52: chown ld 268390 1 2024-09-16T00:09:38.066 INFO:tasks.workunit.client.0.smithi139.stdout:9/53: link f2 fe 0 2024-09-16T00:09:38.067 INFO:tasks.workunit.client.0.smithi139.stdout:9/54: creat ff x:0 0 0 2024-09-16T00:09:38.075 INFO:tasks.workunit.client.0.smithi139.stdout:9/55: dwrite f6 [4194304,4194304] 0 2024-09-16T00:09:38.076 INFO:tasks.workunit.client.0.smithi139.stdout:9/56: write fc [8337438,81955] 0 2024-09-16T00:09:38.076 INFO:tasks.workunit.client.0.smithi139.stdout:9/57: chown f9 32333466 1 2024-09-16T00:09:38.078 INFO:tasks.workunit.client.0.smithi139.stdout:9/58: creat f10 x:0 0 0 2024-09-16T00:09:38.078 INFO:tasks.workunit.client.0.smithi139.stdout:9/59: rmdir - no directory 2024-09-16T00:09:38.079 INFO:tasks.workunit.client.0.smithi139.stdout:9/60: chown f10 1074950931 1 2024-09-16T00:09:38.079 INFO:tasks.workunit.client.0.smithi139.stdout:9/61: truncate fa 366371 0 2024-09-16T00:09:38.086 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:09:38.087 INFO:tasks.workunit.client.0.smithi139.stdout:9/62: dwrite f10 [0,4194304] 0 2024-09-16T00:09:38.137 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:38.136+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:38.167 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:38.167 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:38.176 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:09:38.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.183+0000 7fb93de7b640 1 Processor -- start 2024-09-16T00:09:38.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.183+0000 7fb93de7b640 1 -- start start 2024-09-16T00:09:38.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.184+0000 7fb93de7b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb938071660 0x7fb9380cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.184+0000 7fb93de7b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb938071a40 0x7fb93815c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.184+0000 7fb93de7b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9380c74f0 con 0x7fb9380cac90 2024-09-16T00:09:38.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.184+0000 7fb93de7b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb93815c000 con 0x7fb938071a40 2024-09-16T00:09:38.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.184+0000 7fb93de7b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb9380c77d0 con 0x7fb938071660 2024-09-16T00:09:38.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.184+0000 7fb937fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb9380cac90 0x7fb938152770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45512/0 (socket says 172.21.15.139:45512) 2024-09-16T00:09:38.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.184+0000 7fb937fff640 1 -- 172.21.15.139:0/4017313282 learned_addr learned my addr 172.21.15.139:0/4017313282 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:38.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.184+0000 7fb936ffd640 1 --2- 172.21.15.139:0/4017313282 >> v2:172.21.15.164:3300/0 conn(0x7fb938071a40 0x7fb93815c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.184+0000 7fb9377fe640 1 --2- 172.21.15.139:0/4017313282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb938071660 0x7fb9380cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.184+0000 7fb9367fc640 1 -- 172.21.15.139:0/4017313282 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1261110200 0 0) 0x7fb9380c74f0 con 0x7fb9380cac90 2024-09-16T00:09:38.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.184+0000 7fb9367fc640 1 -- 172.21.15.139:0/4017313282 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb924003520 con 0x7fb9380cac90 2024-09-16T00:09:38.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.184+0000 7fb936ffd640 1 -- 172.21.15.139:0/4017313282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb938071660 msgr2=0x7fb9380cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:38.187 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.184+0000 7fb936ffd640 1 --2- 172.21.15.139:0/4017313282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb938071660 0x7fb9380cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:38.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.184+0000 7fb936ffd640 1 -- 172.21.15.139:0/4017313282 >> v1:172.21.15.139:6789/0 conn(0x7fb9380cac90 legacy=0x7fb938152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:38.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.184+0000 7fb936ffd640 1 -- 172.21.15.139:0/4017313282 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9380c7ab0 con 0x7fb938071a40 2024-09-16T00:09:38.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.184+0000 7fb9377fe640 1 --2- 172.21.15.139:0/4017313282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb938071660 0x7fb9380cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:38.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.185+0000 7fb936ffd640 1 --2- 172.21.15.139:0/4017313282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb938071a40 0x7fb93815c490 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb9200097d0 tx=0x7fb92002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c2e770fec76630 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:38.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.186+0000 7fb9367fc640 1 -- 172.21.15.139:0/4017313282 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920037070 con 0x7fb938071a40 2024-09-16T00:09:38.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.186+0000 7fb9367fc640 1 -- 172.21.15.139:0/4017313282 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb920032440 con 0x7fb938071a40 2024-09-16T00:09:38.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.186+0000 7fb9367fc640 1 -- 172.21.15.139:0/4017313282 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920032810 con 0x7fb938071a40 2024-09-16T00:09:38.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.186+0000 7fb93de7b640 1 -- 172.21.15.139:0/4017313282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb938071a40 msgr2=0x7fb93815c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:38.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.186+0000 7fb93de7b640 1 --2- 172.21.15.139:0/4017313282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb938071a40 0x7fb93815c490 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb9200097d0 tx=0x7fb92002a1c0 comp rx=0 tx=0).stop 2024-09-16T00:09:38.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.187+0000 7fb93de7b640 1 -- 172.21.15.139:0/4017313282 shutdown_connections 2024-09-16T00:09:38.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.187+0000 7fb93de7b640 1 --2- 172.21.15.139:0/4017313282 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb938071a40 0x7fb93815c490 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:38.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.187+0000 7fb93de7b640 1 --2- 172.21.15.139:0/4017313282 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb938071660 0x7fb9380cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:38.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.187+0000 7fb93de7b640 1 -- 172.21.15.139:0/4017313282 >> 172.21.15.139:0/4017313282 conn(0x7fb938073950 msgr2=0x7fb93806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:38.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.187+0000 7fb93de7b640 1 -- 172.21.15.139:0/4017313282 shutdown_connections 2024-09-16T00:09:38.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.187+0000 7fb93de7b640 1 -- 172.21.15.139:0/4017313282 wait complete. 2024-09-16T00:09:38.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.188+0000 7fb93de7b640 1 Processor -- start 2024-09-16T00:09:38.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.188+0000 7fb93de7b640 1 -- start start 2024-09-16T00:09:38.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.189+0000 7fb93de7b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb938071660 0x7fb9380ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.189+0000 7fb93de7b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb938071a40 0x7fb9380ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.189+0000 7fb93de7b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9380cac90 0x7fb9380b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.189+0000 7fb93de7b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb9380c8ba0 con 0x7fb938071a40 2024-09-16T00:09:38.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.189+0000 7fb93de7b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb9380c8070 con 0x7fb938071660 2024-09-16T00:09:38.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.189+0000 7fb93de7b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb9380cba00 con 0x7fb9380cac90 2024-09-16T00:09:38.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.189+0000 7fb936ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb938071a40 0x7fb9380ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.189+0000 7fb9377fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb938071660 0x7fb9380ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.189+0000 7fb936ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb938071a40 0x7fb9380ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43744/0 (socket says 172.21.15.139:43744) 2024-09-16T00:09:38.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.189+0000 7fb936ffd640 1 -- 172.21.15.139:0/1153586158 learned_addr learned my addr 172.21.15.139:0/1153586158 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:38.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.189+0000 7fb937fff640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9380cac90 0x7fb9380b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.190+0000 7fb9377fe640 1 -- 172.21.15.139:0/1153586158 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9380cac90 msgr2=0x7fb9380b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:38.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.190+0000 7fb9377fe640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb9380cac90 0x7fb9380b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:38.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.190+0000 7fb9377fe640 1 -- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb938071a40 msgr2=0x7fb9380ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:38.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.190+0000 7fb9377fe640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb938071a40 0x7fb9380ace00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:38.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.190+0000 7fb9377fe640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9380ae450 con 0x7fb938071660 2024-09-16T00:09:38.193 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.190+0000 7fb9377fe640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb938071660 0x7fb9380ac8c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb928009e60 tx=0x7fb92800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:38.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.190+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928018020 con 0x7fb938071660 2024-09-16T00:09:38.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.190+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb928014680 con 0x7fb938071660 2024-09-16T00:09:38.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.190+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9380aa5f0 con 0x7fb938071660 2024-09-16T00:09:38.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.190+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928014a70 con 0x7fb938071660 2024-09-16T00:09:38.194 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.190+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9380ab110 con 0x7fb938071660 2024-09-16T00:09:38.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.191+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fb9380697a0 con 0x7fb938071660 2024-09-16T00:09:38.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.192+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb928005d10 con 0x7fb938071660 2024-09-16T00:09:38.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.192+0000 7fb92ffff640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb91806c2b0 0x7fb91806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.192+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5973+0+0 (secure 0 0 0) 0x7fb92808c180 con 0x7fb938071660 2024-09-16T00:09:38.195 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.192+0000 7fb936ffd640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb91806c2b0 0x7fb91806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.195 INFO:tasks.workunit.client.0.smithi139.stdout:1/44: sync 2024-09-16T00:09:38.196 INFO:tasks.workunit.client.0.smithi139.stdout:7/17: sync 2024-09-16T00:09:38.196 INFO:tasks.workunit.client.0.smithi139.stdout:3/94: sync 2024-09-16T00:09:38.196 INFO:tasks.workunit.client.0.smithi139.stdout:2/40: sync 2024-09-16T00:09:38.196 INFO:tasks.workunit.client.0.smithi139.stdout:8/29: sync 2024-09-16T00:09:38.196 INFO:tasks.workunit.client.0.smithi139.stdout:5/66: sync 2024-09-16T00:09:38.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.193+0000 7fb936ffd640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb91806c2b0 0x7fb91806e770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb92002a200 tx=0x7fb92002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:38.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.194+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v84) ==== 54+0+16 (secure 0 0 0) 0x7fb928022030 con 0x7fb938071660 2024-09-16T00:09:38.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.194+0000 7fb93de7b640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fb9380ac280 0x7fb9380c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.194+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9380b3600 con 0x7fb9380ac280 2024-09-16T00:09:38.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.194+0000 7fb937fff640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fb9380ac280 0x7fb9380c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.197 INFO:tasks.workunit.client.0.smithi139.stdout:2/41: creat f9 x:0 0 0 2024-09-16T00:09:38.197 INFO:tasks.workunit.client.0.smithi139.stdout:8/30: write f7 [949955,46731] 0 2024-09-16T00:09:38.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.195+0000 7fb937fff640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fb9380ac280 0x7fb9380c53d0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:38.198 INFO:tasks.workunit.client.0.smithi139.stdout:7/18: mknod c3 0 2024-09-16T00:09:38.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.195+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9380b3600 con 0x7fb9380ac280 2024-09-16T00:09:38.198 INFO:tasks.workunit.client.0.smithi139.stdout:3/95: dread d5/d7/fd [0,4194304] 0 2024-09-16T00:09:38.199 INFO:tasks.workunit.client.0.smithi139.stdout:1/45: rename f2 to d7/fd 0 2024-09-16T00:09:38.200 INFO:tasks.workunit.client.0.smithi139.stdout:1/46: chown d7/d8/fa 44100314 1 2024-09-16T00:09:38.200 INFO:tasks.workunit.client.0.smithi139.stdout:2/42: unlink l4 0 2024-09-16T00:09:38.200 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T00:09:38.201 INFO:tasks.workunit.client.0.smithi139.stdout:1/47: stat d7/d8 0 2024-09-16T00:09:38.204 INFO:tasks.workunit.client.0.smithi139.stdout:8/31: creat fc x:0 0 0 2024-09-16T00:09:38.204 INFO:tasks.workunit.client.0.smithi139.stdout:7/19: creat f4 x:0 0 0 2024-09-16T00:09:38.211 INFO:tasks.workunit.client.0.smithi139.stdout:2/43: dwrite f9 [0,4194304] 0 2024-09-16T00:09:38.211 INFO:tasks.workunit.client.0.smithi139.stdout:5/67: rename d1/cb to d1/c10 0 2024-09-16T00:09:38.212 INFO:tasks.workunit.client.0.smithi139.stdout:3/96: creat d5/f16 x:0 0 0 2024-09-16T00:09:38.214 INFO:tasks.workunit.client.0.smithi139.stdout:7/20: mkdir d5 0 2024-09-16T00:09:38.215 INFO:tasks.workunit.client.0.smithi139.stdout:4/18: dread f2 [0,4194304] 0 2024-09-16T00:09:38.217 INFO:tasks.workunit.client.0.smithi139.stdout:8/32: write fb [925538,57491] 0 2024-09-16T00:09:38.218 INFO:tasks.workunit.client.0.smithi139.stdout:2/44: creat fa x:0 0 0 2024-09-16T00:09:38.219 INFO:tasks.workunit.client.0.smithi139.stdout:7/21: chown c3 316 1 2024-09-16T00:09:38.220 INFO:tasks.workunit.client.0.smithi139.stdout:0/64: sync 2024-09-16T00:09:38.221 INFO:tasks.workunit.client.0.smithi139.stdout:7/22: dread - f1 zero size 2024-09-16T00:09:38.227 INFO:tasks.workunit.client.0.smithi139.stdout:4/19: creat f4 x:0 0 0 2024-09-16T00:09:38.227 INFO:tasks.workunit.client.0.smithi139.stdout:7/23: truncate f4 384007 0 2024-09-16T00:09:38.229 INFO:tasks.workunit.client.0.smithi139.stdout:5/68: dwrite d1/fa [4194304,4194304] 0 2024-09-16T00:09:38.230 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.228+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb9380c9e00 con 0x7fb9380ac280 2024-09-16T00:09:38.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.231+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb9380c9e00 con 0x7fb9380ac280 2024-09-16T00:09:38.233 INFO:tasks.workunit.client.0.smithi139.stdout:3/97: dwrite d5/da/f15 [0,4194304] 0 2024-09-16T00:09:38.233 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:38.234 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:38.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.232+0000 7fb93de7b640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fb9380ab350 0x7fb9381f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.234 INFO:tasks.workunit.client.0.smithi139.stdout:5/69: write d1/fa [4328310,56785] 0 2024-09-16T00:09:38.234 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.232+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fb9380c5a80 con 0x7fb9380ab350 2024-09-16T00:09:38.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.234+0000 7fb9377fe640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fb9380ab350 0x7fb9381f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.236 INFO:tasks.workunit.client.0.smithi139.stdout:3/98: dread d5/f8 [0,4194304] 0 2024-09-16T00:09:38.237 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.235+0000 7fb9377fe640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fb9380ab350 0x7fb9381f1ef0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:38.237 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:38.236+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:38.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.238+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9380c5a80 con 0x7fb9380ab350 2024-09-16T00:09:38.241 INFO:tasks.workunit.client.0.smithi139.stdout:7/24: dwrite f1 [0,4194304] 0 2024-09-16T00:09:38.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.258+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb9381f2680 con 0x7fb9380ab350 2024-09-16T00:09:38.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.260+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb9381f2680 con 0x7fb9380ab350 2024-09-16T00:09:38.262 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:38.263 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:38.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.261+0000 7fb93de7b640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fb9381ee690 0x7fb9381eea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.261+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fb9381f2680 con 0x7fb9381ee690 2024-09-16T00:09:38.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.261+0000 7fb936ffd640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fb9381ee690 0x7fb9381eea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.261+0000 7fb936ffd640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fb9381ee690 0x7fb9381eea90 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:38.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.262+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9381f2680 con 0x7fb9381ee690 2024-09-16T00:09:38.264 INFO:tasks.workunit.client.0.smithi139.stdout:2/45: mknod cb 0 2024-09-16T00:09:38.265 INFO:tasks.workunit.client.0.smithi139.stdout:4/20: creat f5 x:0 0 0 2024-09-16T00:09:38.266 INFO:tasks.workunit.client.0.smithi139.stdout:4/21: write f1 [1087403,126258] 0 2024-09-16T00:09:38.267 INFO:tasks.workunit.client.0.smithi139.stdout:8/33: link fc fd 0 2024-09-16T00:09:38.270 INFO:tasks.workunit.client.0.smithi139.stdout:3/99: creat d5/da/f17 x:0 0 0 2024-09-16T00:09:38.271 INFO:tasks.workunit.client.0.smithi139.stdout:2/46: creat fc x:0 0 0 2024-09-16T00:09:38.273 INFO:tasks.workunit.client.0.smithi139.stdout:8/34: creat fe x:0 0 0 2024-09-16T00:09:38.273 INFO:tasks.workunit.client.0.smithi139.stdout:8/35: rmdir - no directory 2024-09-16T00:09:38.274 INFO:tasks.workunit.client.0.smithi139.stdout:8/36: fsync fb 0 2024-09-16T00:09:38.274 INFO:tasks.workunit.client.0.smithi139.stdout:8/37: readlink - no filename 2024-09-16T00:09:38.275 INFO:tasks.workunit.client.0.smithi139.stdout:5/70: rename d1/c10 to d1/c11 0 2024-09-16T00:09:38.275 INFO:tasks.workunit.client.0.smithi139.stdout:7/25: creat d5/f6 x:0 0 0 2024-09-16T00:09:38.276 INFO:tasks.workunit.client.0.smithi139.stdout:3/100: mknod d5/c18 0 2024-09-16T00:09:38.278 INFO:tasks.workunit.client.0.smithi139.stdout:8/38: rename fe to ff 0 2024-09-16T00:09:38.279 INFO:tasks.workunit.client.0.smithi139.stdout:2/47: dwrite f9 [0,4194304] 0 2024-09-16T00:09:38.279 INFO:tasks.workunit.client.0.smithi139.stdout:8/39: chown f2 42 1 2024-09-16T00:09:38.280 INFO:tasks.workunit.client.0.smithi139.stdout:0/65: getdents d6/dc/dd 0 2024-09-16T00:09:38.281 INFO:tasks.workunit.client.0.smithi139.stdout:7/26: creat d5/f7 x:0 0 0 2024-09-16T00:09:38.282 INFO:tasks.workunit.client.0.smithi139.stdout:2/48: write fc [310223,63969] 0 2024-09-16T00:09:38.282 INFO:tasks.workunit.client.0.smithi139.stdout:7/27: rename d5 to d5/d8 22 2024-09-16T00:09:38.282 INFO:tasks.workunit.client.0.smithi139.stdout:3/101: mkdir d5/d7/dc/d19 0 2024-09-16T00:09:38.283 INFO:tasks.workunit.client.0.smithi139.stdout:7/28: readlink l2 0 2024-09-16T00:09:38.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.282+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb9381f2680 con 0x7fb9381ee690 2024-09-16T00:09:38.284 INFO:tasks.workunit.client.0.smithi139.stdout:5/71: rename d1/c7 to d1/c12 0 2024-09-16T00:09:38.286 INFO:tasks.workunit.client.0.smithi139.stdout:7/29: write d5/f6 [872479,128962] 0 2024-09-16T00:09:38.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.285+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb9381f2680 con 0x7fb9381ee690 2024-09-16T00:09:38.287 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:38.287 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:38.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.286+0000 7fb93de7b640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fb938063230 0x7fb9380c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.286+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fb9381f2680 con 0x7fb938063230 2024-09-16T00:09:38.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.286+0000 7fb937fff640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fb938063230 0x7fb9380c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.286+0000 7fb937fff640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fb938063230 0x7fb9380c1eb0 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:38.289 INFO:tasks.workunit.client.0.smithi139.stdout:8/40: write fd [393733,62800] 0 2024-09-16T00:09:38.289 INFO:tasks.workunit.client.0.smithi139.stdout:6/48: sync 2024-09-16T00:09:38.289 INFO:tasks.workunit.client.0.smithi139.stdout:9/63: sync 2024-09-16T00:09:38.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.287+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9381f2680 con 0x7fb938063230 2024-09-16T00:09:38.291 INFO:tasks.workunit.client.0.smithi139.stdout:2/49: creat fd x:0 0 0 2024-09-16T00:09:38.293 INFO:tasks.workunit.client.0.smithi139.stdout:7/30: creat d5/f9 x:0 0 0 2024-09-16T00:09:38.294 INFO:tasks.workunit.client.0.smithi139.stdout:8/41: mknod c10 0 2024-09-16T00:09:38.295 INFO:tasks.workunit.client.0.smithi139.stdout:0/66: dwrite d6/f7 [0,4194304] 0 2024-09-16T00:09:38.295 INFO:tasks.workunit.client.0.smithi139.stdout:9/64: mknod c11 0 2024-09-16T00:09:38.296 INFO:tasks.workunit.client.0.smithi139.stdout:7/31: write d5/f6 [1810673,24840] 0 2024-09-16T00:09:38.297 INFO:tasks.workunit.client.0.smithi139.stdout:2/50: creat fe x:0 0 0 2024-09-16T00:09:38.298 INFO:tasks.workunit.client.0.smithi139.stdout:2/51: fsync f9 0 2024-09-16T00:09:38.298 INFO:tasks.workunit.client.0.smithi139.stdout:6/49: mknod d8/c14 0 2024-09-16T00:09:38.300 INFO:tasks.workunit.client.0.smithi139.stdout:8/42: symlink l11 0 2024-09-16T00:09:38.301 INFO:tasks.workunit.client.0.smithi139.stdout:9/65: mknod c12 0 2024-09-16T00:09:38.302 INFO:tasks.workunit.client.0.smithi139.stdout:8/43: dread - f2 zero size 2024-09-16T00:09:38.302 INFO:tasks.workunit.client.0.smithi139.stdout:0/67: rename d6 to d6/dc/d13 22 2024-09-16T00:09:38.302 INFO:tasks.workunit.client.0.smithi139.stdout:6/50: dread d8/fe [0,4194304] 0 2024-09-16T00:09:38.303 INFO:tasks.workunit.client.0.smithi139.stdout:9/66: write fa [891721,121549] 0 2024-09-16T00:09:38.307 INFO:tasks.workunit.client.0.smithi139.stdout:0/68: fdatasync f2 0 2024-09-16T00:09:38.311 INFO:tasks.workunit.client.0.smithi139.stdout:7/32: dwrite d5/f7 [0,4194304] 0 2024-09-16T00:09:38.311 INFO:tasks.workunit.client.0.smithi139.stdout:6/51: creat d8/db/f15 x:0 0 0 2024-09-16T00:09:38.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.310+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb9380c26c0 con 0x7fb938063230 2024-09-16T00:09:38.312 INFO:tasks.workunit.client.0.smithi139.stdout:9/67: write f9 [749357,51464] 0 2024-09-16T00:09:38.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.313+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb9380c26c0 con 0x7fb938063230 2024-09-16T00:09:38.315 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:38.316 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:38.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.314+0000 7fb93de7b640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fb93802c640 0x7fb93802caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.314+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fb9381f2680 con 0x7fb93802c640 2024-09-16T00:09:38.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.314+0000 7fb9377fe640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fb93802c640 0x7fb93802caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.316+0000 7fb9377fe640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fb93802c640 0x7fb93802caa0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:38.318 INFO:tasks.workunit.client.0.smithi139.stdout:9/68: write f10 [3012601,55396] 0 2024-09-16T00:09:38.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.317+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9381f2680 con 0x7fb93802c640 2024-09-16T00:09:38.319 INFO:tasks.workunit.client.0.smithi139.stdout:8/44: dwrite fb [0,4194304] 0 2024-09-16T00:09:38.320 INFO:tasks.workunit.client.0.smithi139.stdout:6/52: creat d8/db/f16 x:0 0 0 2024-09-16T00:09:38.320 INFO:tasks.workunit.client.0.smithi139.stdout:8/45: fdatasync f4 0 2024-09-16T00:09:38.322 INFO:tasks.workunit.client.0.smithi139.stdout:9/69: mknod c13 0 2024-09-16T00:09:38.324 INFO:tasks.workunit.client.0.smithi139.stdout:8/46: mknod c12 0 2024-09-16T00:09:38.325 INFO:tasks.workunit.client.0.smithi139.stdout:6/53: creat d8/d9/d10/f17 x:0 0 0 2024-09-16T00:09:38.326 INFO:tasks.workunit.client.0.smithi139.stdout:0/69: link d6/l10 d6/dc/dd/l14 0 2024-09-16T00:09:38.328 INFO:tasks.workunit.client.0.smithi139.stdout:6/54: write d8/db/f16 [12671,1546] 0 2024-09-16T00:09:38.329 INFO:tasks.workunit.client.0.smithi139.stdout:9/70: creat f14 x:0 0 0 2024-09-16T00:09:38.331 INFO:tasks.workunit.client.0.smithi139.stdout:9/71: chown ld 58690 1 2024-09-16T00:09:38.333 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:38.333 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:38.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:38.333 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:38.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.333 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:38.333 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:38.333 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 119.715098114, 2024-09-16T00:09:38.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0054735510000000001, 2024-09-16T00:09:38.333 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.333 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.334 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.334 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:38.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615515+0000", 2024-09-16T00:09:38.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615516+0000", 2024-09-16T00:09:38.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6899999999999998e-07 2024-09-16T00:09:38.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615519+0000", 2024-09-16T00:09:38.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1760000000000002e-06 2024-09-16T00:09:38.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615526+0000", 2024-09-16T00:09:38.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8399999999999997e-06 2024-09-16T00:09:38.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615531+0000", 2024-09-16T00:09:38.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4450000000000004e-06 2024-09-16T00:09:38.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616073+0000", 2024-09-16T00:09:38.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00054127099999999996 2024-09-16T00:09:38.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616140+0000", 2024-09-16T00:09:38.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7230999999999994e-05 2024-09-16T00:09:38.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:38.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616249+0000", 2024-09-16T00:09:38.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109122 2024-09-16T00:09:38.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618646+0000", 2024-09-16T00:09:38.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023972279999999999 2024-09-16T00:09:38.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620886+0000", 2024-09-16T00:09:38.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022396220000000001 2024-09-16T00:09:38.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620939+0000", 2024-09-16T00:09:38.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2735999999999998e-05 2024-09-16T00:09:38.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620989+0000", 2024-09-16T00:09:38.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0411000000000002e-05 2024-09-16T00:09:38.339 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.339 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.339 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.339 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:38.339 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:38.339 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 119.715072635, 2024-09-16T00:09:38.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055872789999999997, 2024-09-16T00:09:38.339 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.339 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.339 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.339 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.340 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:38.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.340 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:38.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.340 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615541+0000", 2024-09-16T00:09:38.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.01e-07 2024-09-16T00:09:38.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615547+0000", 2024-09-16T00:09:38.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7899999999999996e-06 2024-09-16T00:09:38.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615554+0000", 2024-09-16T00:09:38.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1119999999999998e-06 2024-09-16T00:09:38.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615560+0000", 2024-09-16T00:09:38.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:09:38.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616424+0000", 2024-09-16T00:09:38.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086350800000000003 2024-09-16T00:09:38.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616523+0000", 2024-09-16T00:09:38.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9675999999999999e-05 2024-09-16T00:09:38.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:38.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616621+0000", 2024-09-16T00:09:38.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7345999999999999e-05 2024-09-16T00:09:38.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618654+0000", 2024-09-16T00:09:38.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020335269999999998 2024-09-16T00:09:38.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621055+0000", 2024-09-16T00:09:38.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024009729999999998 2024-09-16T00:09:38.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.344 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621100+0000", 2024-09-16T00:09:38.344 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5402999999999999e-05 2024-09-16T00:09:38.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.344 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.344 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.344 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621128+0000", 2024-09-16T00:09:38.344 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7736e-05 2024-09-16T00:09:38.344 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.344 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.344 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.344 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.344 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:38.345 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:38.345 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 119.715046985, 2024-09-16T00:09:38.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057024509999999999, 2024-09-16T00:09:38.345 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.345 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.345 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.345 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:38.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:38.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.346 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615567+0000", 2024-09-16T00:09:38.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6300000000000001e-07 2024-09-16T00:09:38.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.346 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615569+0000", 2024-09-16T00:09:38.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3709999999999998e-06 2024-09-16T00:09:38.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615580+0000", 2024-09-16T00:09:38.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0448000000000001e-05 2024-09-16T00:09:38.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615584+0000", 2024-09-16T00:09:38.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6020000000000002e-06 2024-09-16T00:09:38.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616850+0000", 2024-09-16T00:09:38.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012661110000000001 2024-09-16T00:09:38.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.616915+0000", 2024-09-16T00:09:38.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4820000000000006e-05 2024-09-16T00:09:38.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:38.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617031+0000", 2024-09-16T00:09:38.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115258 2024-09-16T00:09:38.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618659+0000", 2024-09-16T00:09:38.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001627975 2024-09-16T00:09:38.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621179+0000", 2024-09-16T00:09:38.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025203410000000002 2024-09-16T00:09:38.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621238+0000", 2024-09-16T00:09:38.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9323000000000001e-05 2024-09-16T00:09:38.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.350 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621269+0000", 2024-09-16T00:09:38.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0839000000000001e-05 2024-09-16T00:09:38.350 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.350 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.350 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.350 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:38.350 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:09:38.350 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 119.71502323599999, 2024-09-16T00:09:38.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058012769999999996, 2024-09-16T00:09:38.351 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.351 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.351 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.351 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.351 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.351 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.351 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:09:38.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.351 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.351 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.351 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.351 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615590+0000", 2024-09-16T00:09:38.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.351 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.352 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.352 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615591+0000", 2024-09-16T00:09:38.352 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6199999999999999e-07 2024-09-16T00:09:38.352 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.352 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.352 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615594+0000", 2024-09-16T00:09:38.352 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8739999999999999e-06 2024-09-16T00:09:38.352 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.352 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.352 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615599+0000", 2024-09-16T00:09:38.352 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.609e-06 2024-09-16T00:09:38.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615603+0000", 2024-09-16T00:09:38.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8489999999999999e-06 2024-09-16T00:09:38.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617235+0000", 2024-09-16T00:09:38.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001632037 2024-09-16T00:09:38.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617348+0000", 2024-09-16T00:09:38.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113387 2024-09-16T00:09:38.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:38.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617458+0000", 2024-09-16T00:09:38.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109765 2024-09-16T00:09:38.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618664+0000", 2024-09-16T00:09:38.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012059950000000001 2024-09-16T00:09:38.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621311+0000", 2024-09-16T00:09:38.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026468469999999999 2024-09-16T00:09:38.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621357+0000", 2024-09-16T00:09:38.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6360999999999999e-05 2024-09-16T00:09:38.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621392+0000", 2024-09-16T00:09:38.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4190999999999998e-05 2024-09-16T00:09:38.356 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.356 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.356 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.356 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:38.356 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:38.356 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 119.715004612, 2024-09-16T00:09:38.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058737279999999999, 2024-09-16T00:09:38.356 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.356 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.356 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.357 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:38.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:38.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615609+0000", 2024-09-16T00:09:38.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3799999999999999e-07 2024-09-16T00:09:38.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615611+0000", 2024-09-16T00:09:38.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2670000000000001e-06 2024-09-16T00:09:38.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615626+0000", 2024-09-16T00:09:38.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4114000000000001e-05 2024-09-16T00:09:38.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615634+0000", 2024-09-16T00:09:38.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3070000000000003e-06 2024-09-16T00:09:38.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617692+0000", 2024-09-16T00:09:38.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002057718 2024-09-16T00:09:38.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617768+0000", 2024-09-16T00:09:38.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6878999999999994e-05 2024-09-16T00:09:38.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:38.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617963+0000", 2024-09-16T00:09:38.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019501699999999999 2024-09-16T00:09:38.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621399+0000", 2024-09-16T00:09:38.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034357200000000002 2024-09-16T00:09:38.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621429+0000", 2024-09-16T00:09:38.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0012999999999999e-05 2024-09-16T00:09:38.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621459+0000", 2024-09-16T00:09:38.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9498000000000001e-05 2024-09-16T00:09:38.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621483+0000", 2024-09-16T00:09:38.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3957000000000001e-05 2024-09-16T00:09:38.361 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.361 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.361 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.362 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:38.362 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:09:38.362 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 119.714973332, 2024-09-16T00:09:38.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059371789999999999, 2024-09-16T00:09:38.362 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.362 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.362 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.362 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:09:38.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615640+0000", 2024-09-16T00:09:38.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615641+0000", 2024-09-16T00:09:38.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4000000000000002e-07 2024-09-16T00:09:38.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615643+0000", 2024-09-16T00:09:38.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6860000000000002e-06 2024-09-16T00:09:38.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.364 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.364 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615649+0000", 2024-09-16T00:09:38.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7610000000000004e-06 2024-09-16T00:09:38.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.364 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.364 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615653+0000", 2024-09-16T00:09:38.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0380000000000001e-06 2024-09-16T00:09:38.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618197+0000", 2024-09-16T00:09:38.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025440079999999999 2024-09-16T00:09:38.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618312+0000", 2024-09-16T00:09:38.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115083 2024-09-16T00:09:38.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:38.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618412+0000", 2024-09-16T00:09:38.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010016799999999999 2024-09-16T00:09:38.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621518+0000", 2024-09-16T00:09:38.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031058679999999999 2024-09-16T00:09:38.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621526+0000", 2024-09-16T00:09:38.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2490000000000004e-06 2024-09-16T00:09:38.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621549+0000", 2024-09-16T00:09:38.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3686999999999999e-05 2024-09-16T00:09:38.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621577+0000", 2024-09-16T00:09:38.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8191000000000002e-05 2024-09-16T00:09:38.367 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.367 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.367 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.367 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:38.367 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:38.367 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 119.71495566599999, 2024-09-16T00:09:38.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005989622, 2024-09-16T00:09:38.367 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.368 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.368 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.368 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:38.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:38.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615658+0000", 2024-09-16T00:09:38.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3799999999999998e-07 2024-09-16T00:09:38.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615660+0000", 2024-09-16T00:09:38.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9740000000000001e-06 2024-09-16T00:09:38.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615666+0000", 2024-09-16T00:09:38.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3499999999999996e-06 2024-09-16T00:09:38.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615671+0000", 2024-09-16T00:09:38.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.13e-06 2024-09-16T00:09:38.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618759+0000", 2024-09-16T00:09:38.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030880500000000002 2024-09-16T00:09:38.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618832+0000", 2024-09-16T00:09:38.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3725000000000003e-05 2024-09-16T00:09:38.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:38.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.618958+0000", 2024-09-16T00:09:38.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012587899999999999 2024-09-16T00:09:38.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621583+0000", 2024-09-16T00:09:38.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026246469999999999 2024-09-16T00:09:38.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621602+0000", 2024-09-16T00:09:38.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9137e-05 2024-09-16T00:09:38.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621629+0000", 2024-09-16T00:09:38.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6451000000000001e-05 2024-09-16T00:09:38.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621648+0000", 2024-09-16T00:09:38.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8941e-05 2024-09-16T00:09:38.372 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.372 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.372 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.373 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:22 2.e 2:717a0223:::608.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:38.373 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:38.373 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 119.714938047, 2024-09-16T00:09:38.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053141380000000004, 2024-09-16T00:09:38.373 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.373 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.373 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.373 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:38.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:38.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615676+0000", 2024-09-16T00:09:38.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-07 2024-09-16T00:09:38.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615678+0000", 2024-09-16T00:09:38.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:09:38.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615682+0000", 2024-09-16T00:09:38.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6659999999999999e-06 2024-09-16T00:09:38.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615686+0000", 2024-09-16T00:09:38.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6990000000000001e-06 2024-09-16T00:09:38.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615736+0000", 2024-09-16T00:09:38.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9656e-05 2024-09-16T00:09:38.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615864+0000", 2024-09-16T00:09:38.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012836100000000001 2024-09-16T00:09:38.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:38.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615989+0000", 2024-09-16T00:09:38.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000125525 2024-09-16T00:09:38.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617355+0000", 2024-09-16T00:09:38.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001365982 2024-09-16T00:09:38.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620897+0000", 2024-09-16T00:09:38.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035419610000000002 2024-09-16T00:09:38.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620941+0000", 2024-09-16T00:09:38.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3998999999999997e-05 2024-09-16T00:09:38.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.620990+0000", 2024-09-16T00:09:38.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8199999999999999e-05 2024-09-16T00:09:38.378 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.378 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.378 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.378 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:23 2.e 2:717a0223:::608.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-16T00:09:38.378 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:38.378 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 119.714910789, 2024-09-16T00:09:38.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006316449, 2024-09-16T00:09:38.378 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.378 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.379 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.379 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:38.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:38.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615703+0000", 2024-09-16T00:09:38.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3299999999999998e-07 2024-09-16T00:09:38.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615705+0000", 2024-09-16T00:09:38.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.278e-06 2024-09-16T00:09:38.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615711+0000", 2024-09-16T00:09:38.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8119999999999996e-06 2024-09-16T00:09:38.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.615716+0000", 2024-09-16T00:09:38.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4909999999999999e-06 2024-09-16T00:09:38.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617551+0000", 2024-09-16T00:09:38.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00183549 2024-09-16T00:09:38.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617627+0000", 2024-09-16T00:09:38.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6163999999999998e-05 2024-09-16T00:09:38.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:38.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.617776+0000", 2024-09-16T00:09:38.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014909499999999999 2024-09-16T00:09:38.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621072+0000", 2024-09-16T00:09:38.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032955150000000002 2024-09-16T00:09:38.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621957+0000", 2024-09-16T00:09:38.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00088553199999999996 2024-09-16T00:09:38.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.621998+0000", 2024-09-16T00:09:38.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0815000000000001e-05 2024-09-16T00:09:38.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:07:38.622019+0000", 2024-09-16T00:09:38.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0924000000000002e-05 2024-09-16T00:09:38.383 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.383 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.383 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.383 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:714 3.22 3:453cbc5d:::10000000536.00000000:head [write 0~260] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:38.383 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.356475+0000", 2024-09-16T00:09:38.384 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 39.974138908, 2024-09-16T00:09:38.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003898926, 2024-09-16T00:09:38.384 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.384 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.384 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.384 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:38.384 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:38.384 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:38.384 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 714 2024-09-16T00:09:38.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.384 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356475+0000", 2024-09-16T00:09:38.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356475+0000", 2024-09-16T00:09:38.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356476+0000", 2024-09-16T00:09:38.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6750000000000001e-06 2024-09-16T00:09:38.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356493+0000", 2024-09-16T00:09:38.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6562000000000002e-05 2024-09-16T00:09:38.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356509+0000", 2024-09-16T00:09:38.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5908000000000001e-05 2024-09-16T00:09:38.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356516+0000", 2024-09-16T00:09:38.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9079999999999998e-06 2024-09-16T00:09:38.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356573+0000", 2024-09-16T00:09:38.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7243000000000001e-05 2024-09-16T00:09:38.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356657+0000", 2024-09-16T00:09:38.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3809999999999999e-05 2024-09-16T00:09:38.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:38.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.356742+0000", 2024-09-16T00:09:38.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5704000000000005e-05 2024-09-16T00:09:38.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360000+0000", 2024-09-16T00:09:38.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003257916 2024-09-16T00:09:38.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360279+0000", 2024-09-16T00:09:38.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00027872500000000001 2024-09-16T00:09:38.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360324+0000", 2024-09-16T00:09:38.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5359999999999999e-05 2024-09-16T00:09:38.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.360374+0000", 2024-09-16T00:09:38.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9115e-05 2024-09-16T00:09:38.389 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.389 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.389 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.389 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:766 3.1f 3:f8970b5b:::10000000571.00000000:head [write 0~1989] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-16T00:09:38.390 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:08:58.481586+0000", 2024-09-16T00:09:38.390 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 39.849027253000003, 2024-09-16T00:09:38.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038531529999999998, 2024-09-16T00:09:38.390 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.390 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.390 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.390 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:38.390 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:38.390 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:38.390 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 766 2024-09-16T00:09:38.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.390 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481586+0000", 2024-09-16T00:09:38.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481586+0000", 2024-09-16T00:09:38.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481588+0000", 2024-09-16T00:09:38.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.691e-06 2024-09-16T00:09:38.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481595+0000", 2024-09-16T00:09:38.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5780000000000004e-06 2024-09-16T00:09:38.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481605+0000", 2024-09-16T00:09:38.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0729e-05 2024-09-16T00:09:38.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481610+0000", 2024-09-16T00:09:38.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1429999999999997e-06 2024-09-16T00:09:38.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481663+0000", 2024-09-16T00:09:38.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2429999999999998e-05 2024-09-16T00:09:38.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481740+0000", 2024-09-16T00:09:38.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7526000000000003e-05 2024-09-16T00:09:38.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:38.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.481829+0000", 2024-09-16T00:09:38.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8265999999999998e-05 2024-09-16T00:09:38.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.482578+0000", 2024-09-16T00:09:38.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074936600000000001 2024-09-16T00:09:38.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.485314+0000", 2024-09-16T00:09:38.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027358949999999999 2024-09-16T00:09:38.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.485398+0000", 2024-09-16T00:09:38.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4462000000000005e-05 2024-09-16T00:09:38.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:08:58.485439+0000", 2024-09-16T00:09:38.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1066999999999998e-05 2024-09-16T00:09:38.395 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.395 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.395 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.395 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5850 3.3a 3:5c3cfc2b:::10000001adf.00000000:head [write 0~116] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:38.395 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:09:38.396 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.914676856, 2024-09-16T00:09:38.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66931295800000001, 2024-09-16T00:09:38.396 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.396 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.396 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.396 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:38.396 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:38.396 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:38.396 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5850 2024-09-16T00:09:38.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.396 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:09:38.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:09:38.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415938+0000", 2024-09-16T00:09:38.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7099999999999999e-06 2024-09-16T00:09:38.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415945+0000", 2024-09-16T00:09:38.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:09:38.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415959+0000", 2024-09-16T00:09:38.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3545e-05 2024-09-16T00:09:38.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415966+0000", 2024-09-16T00:09:38.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6379999999999998e-06 2024-09-16T00:09:38.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416013+0000", 2024-09-16T00:09:38.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7314999999999997e-05 2024-09-16T00:09:38.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:38.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416020+0000", 2024-09-16T00:09:38.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.52e-06 2024-09-16T00:09:38.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082393+0000", 2024-09-16T00:09:38.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66637251799999997 2024-09-16T00:09:38.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082537+0000", 2024-09-16T00:09:38.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014418800000000001 2024-09-16T00:09:38.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:38.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082689+0000", 2024-09-16T00:09:38.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015198499999999999 2024-09-16T00:09:38.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084930+0000", 2024-09-16T00:09:38.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002241041 2024-09-16T00:09:38.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084994+0000", 2024-09-16T00:09:38.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3274000000000004e-05 2024-09-16T00:09:38.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085049+0000", 2024-09-16T00:09:38.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.554e-05 2024-09-16T00:09:38.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085250+0000", 2024-09-16T00:09:38.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020060799999999999 2024-09-16T00:09:38.402 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.402 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.402 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.402 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5910 3.3a 3:5ec98a62:::10000001b22.00000000:head [write 0~1274] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:38.402 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:09:38.402 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.767733218, 2024-09-16T00:09:38.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.52286026600000002, 2024-09-16T00:09:38.402 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.402 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.402 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.403 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:38.403 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:38.403 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:38.403 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5910 2024-09-16T00:09:38.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.403 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:09:38.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:09:38.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562881+0000", 2024-09-16T00:09:38.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:09:38.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562887+0000", 2024-09-16T00:09:38.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2870000000000004e-06 2024-09-16T00:09:38.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562897+0000", 2024-09-16T00:09:38.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.064e-05 2024-09-16T00:09:38.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562902+0000", 2024-09-16T00:09:38.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2340000000000002e-06 2024-09-16T00:09:38.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562958+0000", 2024-09-16T00:09:38.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5523999999999999e-05 2024-09-16T00:09:38.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:38.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562968+0000", 2024-09-16T00:09:38.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0394000000000001e-05 2024-09-16T00:09:38.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083461+0000", 2024-09-16T00:09:38.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.520492968 2024-09-16T00:09:38.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083645+0000", 2024-09-16T00:09:38.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018346399999999999 2024-09-16T00:09:38.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:38.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083844+0000", 2024-09-16T00:09:38.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000199268 2024-09-16T00:09:38.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085410+0000", 2024-09-16T00:09:38.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015662359999999999 2024-09-16T00:09:38.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085493+0000", 2024-09-16T00:09:38.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2478999999999994e-05 2024-09-16T00:09:38.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085563+0000", 2024-09-16T00:09:38.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0159000000000004e-05 2024-09-16T00:09:38.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085741+0000", 2024-09-16T00:09:38.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017764300000000001 2024-09-16T00:09:38.409 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.409 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.409 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.409 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5962 3.3a 3:5c7b4d29:::10000001b5d.00000000:head [write 0~3074] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:38.409 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:09:38.409 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.638196400999998, 2024-09-16T00:09:38.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39368737500000001, 2024-09-16T00:09:38.410 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.410 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.410 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.410 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:38.410 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:38.410 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:38.410 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5962 2024-09-16T00:09:38.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.410 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:09:38.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:09:38.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692418+0000", 2024-09-16T00:09:38.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3510000000000001e-06 2024-09-16T00:09:38.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692425+0000", 2024-09-16T00:09:38.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3770000000000001e-06 2024-09-16T00:09:38.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692440+0000", 2024-09-16T00:09:38.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4975e-05 2024-09-16T00:09:38.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692448+0000", 2024-09-16T00:09:38.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6279999999999994e-06 2024-09-16T00:09:38.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692499+0000", 2024-09-16T00:09:38.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0872999999999999e-05 2024-09-16T00:09:38.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:38.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692507+0000", 2024-09-16T00:09:38.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9889999999999992e-06 2024-09-16T00:09:38.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084221+0000", 2024-09-16T00:09:38.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39171379699999997 2024-09-16T00:09:38.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084446+0000", 2024-09-16T00:09:38.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022519900000000001 2024-09-16T00:09:38.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:38.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084672+0000", 2024-09-16T00:09:38.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000225405 2024-09-16T00:09:38.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085832+0000", 2024-09-16T00:09:38.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011600040000000001 2024-09-16T00:09:38.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085864+0000", 2024-09-16T00:09:38.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2592999999999998e-05 2024-09-16T00:09:38.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085943+0000", 2024-09-16T00:09:38.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8534000000000004e-05 2024-09-16T00:09:38.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086105+0000", 2024-09-16T00:09:38.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016165 2024-09-16T00:09:38.416 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.416 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.416 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.416 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5963 3.3a 3:5e4192f5:::10000001b5e.00000000:head [write 0~152] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:38.416 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:38.416 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.636131970000001, 2024-09-16T00:09:38.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39413463900000001, 2024-09-16T00:09:38.417 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.417 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.417 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.417 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:38.417 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:38.417 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:38.417 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5963 2024-09-16T00:09:38.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.417 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:38.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:38.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:38.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:09:38.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694486+0000", 2024-09-16T00:09:38.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0960000000000003e-06 2024-09-16T00:09:38.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694496+0000", 2024-09-16T00:09:38.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2189999999999993e-06 2024-09-16T00:09:38.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694500+0000", 2024-09-16T00:09:38.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9529999999999998e-06 2024-09-16T00:09:38.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694552+0000", 2024-09-16T00:09:38.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1360999999999998e-05 2024-09-16T00:09:38.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:38.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694561+0000", 2024-09-16T00:09:38.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1700000000000003e-06 2024-09-16T00:09:38.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086346+0000", 2024-09-16T00:09:38.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39178468 2024-09-16T00:09:38.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086480+0000", 2024-09-16T00:09:38.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013469299999999999 2024-09-16T00:09:38.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:38.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086790+0000", 2024-09-16T00:09:38.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030923300000000001 2024-09-16T00:09:38.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088361+0000", 2024-09-16T00:09:38.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015717050000000001 2024-09-16T00:09:38.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088432+0000", 2024-09-16T00:09:38.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0922999999999998e-05 2024-09-16T00:09:38.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088489+0000", 2024-09-16T00:09:38.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6653999999999997e-05 2024-09-16T00:09:38.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088616+0000", 2024-09-16T00:09:38.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012730099999999999 2024-09-16T00:09:38.423 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.423 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.423 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.423 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:6082 3.3a 3:5c50f65b:::10000001be7.00000000:head [write 0~2439] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:38.423 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:09:38.423 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.342359114000001, 2024-09-16T00:09:38.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.10065850799999999, 2024-09-16T00:09:38.424 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.424 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.424 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.424 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:38.424 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:38.424 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:38.424 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 6082 2024-09-16T00:09:38.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.424 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:09:38.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:09:38.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988256+0000", 2024-09-16T00:09:38.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.065e-06 2024-09-16T00:09:38.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988262+0000", 2024-09-16T00:09:38.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4640000000000003e-06 2024-09-16T00:09:38.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988272+0000", 2024-09-16T00:09:38.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0335e-05 2024-09-16T00:09:38.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988279+0000", 2024-09-16T00:09:38.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:09:38.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988324+0000", 2024-09-16T00:09:38.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4953000000000001e-05 2024-09-16T00:09:38.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:38.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988331+0000", 2024-09-16T00:09:38.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5989999999999998e-06 2024-09-16T00:09:38.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087272+0000", 2024-09-16T00:09:38.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.098941045000000005 2024-09-16T00:09:38.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087486+0000", 2024-09-16T00:09:38.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00021360199999999999 2024-09-16T00:09:38.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:38.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087740+0000", 2024-09-16T00:09:38.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00025456799999999999 2024-09-16T00:09:38.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088638+0000", 2024-09-16T00:09:38.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00089793900000000005 2024-09-16T00:09:38.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088673+0000", 2024-09-16T00:09:38.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5111999999999999e-05 2024-09-16T00:09:38.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088722+0000", 2024-09-16T00:09:38.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8758000000000003e-05 2024-09-16T00:09:38.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088913+0000", 2024-09-16T00:09:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000190953 2024-09-16T00:09:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10643 3.21 3:8453163e:::100000030b3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:09:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 2.7308829210000001, 2024-09-16T00:09:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013028809000000001, 2024-09-16T00:09:38.430 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10643 2024-09-16T00:09:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:09:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:09:38.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599734+0000", 2024-09-16T00:09:38.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-06 2024-09-16T00:09:38.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603345+0000", 2024-09-16T00:09:38.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003611628 2024-09-16T00:09:38.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603368+0000", 2024-09-16T00:09:38.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3017e-05 2024-09-16T00:09:38.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603379+0000", 2024-09-16T00:09:38.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0528e-05 2024-09-16T00:09:38.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603448+0000", 2024-09-16T00:09:38.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9080999999999998e-05 2024-09-16T00:09:38.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603623+0000", 2024-09-16T00:09:38.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017525599999999999 2024-09-16T00:09:38.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:38.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603766+0000", 2024-09-16T00:09:38.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000142977 2024-09-16T00:09:38.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.611250+0000", 2024-09-16T00:09:38.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074835240000000001 2024-09-16T00:09:38.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612617+0000", 2024-09-16T00:09:38.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001367793 2024-09-16T00:09:38.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612700+0000", 2024-09-16T00:09:38.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3123999999999999e-05 2024-09-16T00:09:38.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612759+0000", 2024-09-16T00:09:38.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8950999999999999e-05 2024-09-16T00:09:38.436 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.436 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.436 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.436 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10652 3.22 3:44e46336:::100000030c1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:38.436 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:09:38.436 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 2.7028312890000001, 2024-09-16T00:09:38.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011405732, 2024-09-16T00:09:38.437 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.437 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.437 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.437 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:38.437 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:38.437 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:38.437 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10652 2024-09-16T00:09:38.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.437 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:09:38.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:09:38.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627785+0000", 2024-09-16T00:09:38.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6970000000000002e-06 2024-09-16T00:09:38.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629610+0000", 2024-09-16T00:09:38.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018250359999999999 2024-09-16T00:09:38.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629639+0000", 2024-09-16T00:09:38.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8676000000000001e-05 2024-09-16T00:09:38.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629652+0000", 2024-09-16T00:09:38.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3572e-05 2024-09-16T00:09:38.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629735+0000", 2024-09-16T00:09:38.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.25e-05 2024-09-16T00:09:38.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629925+0000", 2024-09-16T00:09:38.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019011000000000001 2024-09-16T00:09:38.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:38.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.630075+0000", 2024-09-16T00:09:38.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000149841 2024-09-16T00:09:38.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.636470+0000", 2024-09-16T00:09:38.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063952289999999997 2024-09-16T00:09:38.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639098+0000", 2024-09-16T00:09:38.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026277980000000002 2024-09-16T00:09:38.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639154+0000", 2024-09-16T00:09:38.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6413999999999999e-05 2024-09-16T00:09:38.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639188+0000", 2024-09-16T00:09:38.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3859000000000002e-05 2024-09-16T00:09:38.442 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.442 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.442 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.443 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10670 3.21 3:8453163e:::100000030b3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:38.443 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.836377+0000", 2024-09-16T00:09:38.443 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 2.4942368510000001, 2024-09-16T00:09:38.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0082817159999999997, 2024-09-16T00:09:38.443 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.443 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.443 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.443 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:38.443 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:38.443 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:38.443 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10670 2024-09-16T00:09:38.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.443 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.836377+0000", 2024-09-16T00:09:38.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.836377+0000", 2024-09-16T00:09:38.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.836380+0000", 2024-09-16T00:09:38.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3019999999999999e-06 2024-09-16T00:09:38.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.838089+0000", 2024-09-16T00:09:38.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017086180000000001 2024-09-16T00:09:38.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.838114+0000", 2024-09-16T00:09:38.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5590000000000001e-05 2024-09-16T00:09:38.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.838127+0000", 2024-09-16T00:09:38.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2989e-05 2024-09-16T00:09:38.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.838212+0000", 2024-09-16T00:09:38.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4695000000000001e-05 2024-09-16T00:09:38.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.838307+0000", 2024-09-16T00:09:38.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5254000000000007e-05 2024-09-16T00:09:38.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:38.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.838425+0000", 2024-09-16T00:09:38.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000118121 2024-09-16T00:09:38.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.842737+0000", 2024-09-16T00:09:38.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004312209 2024-09-16T00:09:38.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.844547+0000", 2024-09-16T00:09:38.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00181 2024-09-16T00:09:38.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.844605+0000", 2024-09-16T00:09:38.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7544000000000003e-05 2024-09-16T00:09:38.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.844658+0000", 2024-09-16T00:09:38.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3393999999999997e-05 2024-09-16T00:09:38.448 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.448 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.448 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.449 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10684 3.21 3:8673bb5f:::100000030ed.00000000:head [write 592217~51092] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:38.449 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.893791+0000", 2024-09-16T00:09:38.449 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 2.436823033, 2024-09-16T00:09:38.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050536770000000003, 2024-09-16T00:09:38.449 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.449 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.449 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.449 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:38.449 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:38.449 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:38.449 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10684 2024-09-16T00:09:38.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.449 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.893791+0000", 2024-09-16T00:09:38.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.893791+0000", 2024-09-16T00:09:38.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.893794+0000", 2024-09-16T00:09:38.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2729999999999998e-06 2024-09-16T00:09:38.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.893831+0000", 2024-09-16T00:09:38.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7289000000000002e-05 2024-09-16T00:09:38.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.893870+0000", 2024-09-16T00:09:38.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8723999999999997e-05 2024-09-16T00:09:38.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.893885+0000", 2024-09-16T00:09:38.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5554e-05 2024-09-16T00:09:38.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.893977+0000", 2024-09-16T00:09:38.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1342999999999996e-05 2024-09-16T00:09:38.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.894146+0000", 2024-09-16T00:09:38.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000169352 2024-09-16T00:09:38.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:38.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.894295+0000", 2024-09-16T00:09:38.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014857299999999999 2024-09-16T00:09:38.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.895459+0000", 2024-09-16T00:09:38.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00116478 2024-09-16T00:09:38.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.898715+0000", 2024-09-16T00:09:38.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032559450000000001 2024-09-16T00:09:38.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.898792+0000", 2024-09-16T00:09:38.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6355999999999999e-05 2024-09-16T00:09:38.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.898844+0000", 2024-09-16T00:09:38.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2488000000000003e-05 2024-09-16T00:09:38.454 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.455 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.455 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.455 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.455 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.455 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:38.456 INFO:tasks.workunit.client.0.smithi139.stdout:8/47: dread fb [0,4194304] 0 2024-09-16T00:09:38.456 INFO:tasks.workunit.client.0.smithi139.stdout:6/55: dread - d8/d9/d10/f11 zero size 2024-09-16T00:09:38.456 INFO:tasks.workunit.client.0.smithi139.stdout:9/72: mknod c15 0 2024-09-16T00:09:38.456 INFO:tasks.workunit.client.0.smithi139.stdout:6/56: write f3 [2001431,54515] 0 2024-09-16T00:09:38.456 INFO:tasks.workunit.client.0.smithi139.stdout:0/70: dwrite d6/f7 [0,4194304] 0 2024-09-16T00:09:38.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.336+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb9381f2680 con 0x7fb93802c640 2024-09-16T00:09:38.457 INFO:tasks.workunit.client.0.smithi139.stdout:9/73: creat f16 x:0 0 0 2024-09-16T00:09:38.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.338+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb9381f2680 con 0x7fb93802c640 2024-09-16T00:09:38.457 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:38.458 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:38.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.338+0000 7fb93de7b640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fb93802d4f0 0x7fb938204b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.338+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fb9381eefd0 con 0x7fb93802d4f0 2024-09-16T00:09:38.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.338+0000 7fb936ffd640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fb93802d4f0 0x7fb938204b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.458 INFO:tasks.workunit.client.0.smithi139.stdout:6/57: creat d8/d9/f18 x:0 0 0 2024-09-16T00:09:38.459 INFO:tasks.workunit.client.0.smithi139.stdout:9/74: readlink ld 0 2024-09-16T00:09:38.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.340+0000 7fb936ffd640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fb93802d4f0 0x7fb938204b60 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:38.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.341+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9381eefd0 con 0x7fb93802d4f0 2024-09-16T00:09:38.460 INFO:teuthology.orchestra.run.smithi164.stdout:3.3f 2024-09-16T00:09:38.460 INFO:teuthology.orchestra.run.smithi164.stdout:3.36 2024-09-16T00:09:38.460 INFO:teuthology.orchestra.run.smithi164.stdout:3.35 2024-09-16T00:09:38.460 INFO:teuthology.orchestra.run.smithi164.stdout:3.33 2024-09-16T00:09:38.460 INFO:teuthology.orchestra.run.smithi164.stdout:3.2c 2024-09-16T00:09:38.460 INFO:teuthology.orchestra.run.smithi164.stdout:3.3a 2024-09-16T00:09:38.460 INFO:teuthology.orchestra.run.smithi164.stdout:3.20 2024-09-16T00:09:38.461 INFO:teuthology.orchestra.run.smithi164.stdout:3.1e 2024-09-16T00:09:38.461 INFO:teuthology.orchestra.run.smithi164.stdout:3.10 2024-09-16T00:09:38.461 INFO:teuthology.orchestra.run.smithi164.stdout:3.a 2024-09-16T00:09:38.461 INFO:teuthology.orchestra.run.smithi164.stdout:3.2e 2024-09-16T00:09:38.461 INFO:teuthology.orchestra.run.smithi164.stdout:2.6 2024-09-16T00:09:38.461 INFO:teuthology.orchestra.run.smithi164.stdout:2.1a 2024-09-16T00:09:38.461 INFO:teuthology.orchestra.run.smithi164.stdout:2.19 2024-09-16T00:09:38.461 INFO:teuthology.orchestra.run.smithi164.stdout:2.17 2024-09-16T00:09:38.461 INFO:teuthology.orchestra.run.smithi164.stdout:2.1d 2024-09-16T00:09:38.461 INFO:teuthology.orchestra.run.smithi164.stdout:2.16 2024-09-16T00:09:38.461 INFO:teuthology.orchestra.run.smithi164.stdout:2.7 2024-09-16T00:09:38.461 INFO:teuthology.orchestra.run.smithi164.stdout:2.14 2024-09-16T00:09:38.461 INFO:teuthology.orchestra.run.smithi164.stdout:3.34 2024-09-16T00:09:38.461 INFO:teuthology.orchestra.run.smithi164.stdout:3.26 2024-09-16T00:09:38.462 INFO:teuthology.orchestra.run.smithi164.stdout:2.11 2024-09-16T00:09:38.462 INFO:tasks.workunit.client.0.smithi139.stdout:9/75: mkdir d17 0 2024-09-16T00:09:38.463 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:09:38.464 INFO:tasks.workunit.client.0.smithi139.stdout:9/76: write fb [637838,66275] 0 2024-09-16T00:09:38.464 INFO:tasks.workunit.client.0.smithi139.stdout:4/22: truncate f2 91482 0 2024-09-16T00:09:38.464 INFO:tasks.workunit.client.0.smithi139.stdout:6/58: dread f3 [0,4194304] 0 2024-09-16T00:09:38.464 INFO:tasks.workunit.client.0.smithi139.stdout:0/71: dwrite d6/fa [0,4194304] 0 2024-09-16T00:09:38.464 INFO:tasks.workunit.client.0.smithi139.stdout:3/102: write d5/da/f11 [1497495,64951] 0 2024-09-16T00:09:38.465 INFO:tasks.workunit.client.0.smithi139.stdout:3/103: chown d5/f16 33269356 1 2024-09-16T00:09:38.465 INFO:tasks.workunit.client.0.smithi139.stdout:4/23: creat f6 x:0 0 0 2024-09-16T00:09:38.465 INFO:tasks.workunit.client.0.smithi139.stdout:5/72: dwrite d1/fa [8388608,4194304] 0 2024-09-16T00:09:38.465 INFO:tasks.workunit.client.0.smithi139.stdout:4/24: chown f5 10 1 2024-09-16T00:09:38.465 INFO:tasks.workunit.client.0.smithi139.stdout:6/59: creat d8/db/f19 x:0 0 0 2024-09-16T00:09:38.465 INFO:tasks.workunit.client.0.smithi139.stdout:7/33: write d5/f7 [4106334,117252] 0 2024-09-16T00:09:38.465 INFO:tasks.workunit.client.0.smithi139.stdout:2/52: write f8 [1198708,69114] 0 2024-09-16T00:09:38.465 INFO:tasks.workunit.client.0.smithi139.stdout:6/60: chown d8/f12 649429125 1 2024-09-16T00:09:38.465 INFO:tasks.workunit.client.0.smithi139.stdout:8/48: truncate f4 3618317 0 2024-09-16T00:09:38.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.365+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb9381eefd0 con 0x7fb93802d4f0 2024-09-16T00:09:38.466 INFO:tasks.workunit.client.0.smithi139.stdout:6/61: stat d8/d9/d10/f17 0 2024-09-16T00:09:38.466 INFO:tasks.workunit.client.0.smithi139.stdout:0/72: dwrite f2 [0,4194304] 0 2024-09-16T00:09:38.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.367+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb9381eefd0 con 0x7fb93802d4f0 2024-09-16T00:09:38.466 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:38.467 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:38.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.368+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=85}) -- 0x7fb9380c2de0 con 0x7fb938071660 2024-09-16T00:09:38.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.368+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb9381eefd0 con 0x7fb938071660 2024-09-16T00:09:38.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.368+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== mon.1 v2:172.21.15.164:3300/0 7 ==== osd_map(85..85 src has 1..85) ==== 344+0+0 (secure 0 0 0) 0x7fb9280720c0 con 0x7fb938071660 2024-09-16T00:09:38.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.368+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== mon.1 v2:172.21.15.164:3300/0 8 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb928093070 con 0x7fb938071660 2024-09-16T00:09:38.468 INFO:teuthology.orchestra.run.smithi139.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T00:09:38.468 INFO:tasks.workunit.client.0.smithi139.stdout:4/25: creat f7 x:0 0 0 2024-09-16T00:09:38.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.368+0000 7fb93de7b640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fb9382055b0 0x7fb9382079a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.368+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fb93820a840 con 0x7fb9382055b0 2024-09-16T00:09:38.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.369+0000 7fb937fff640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fb9382055b0 0x7fb9382079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.369+0000 7fb937fff640 1 --2- 172.21.15.139:0/1153586158 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fb9382055b0 0x7fb9382079a0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:38.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.370+0000 7fb649f5b640 1 Processor -- start 2024-09-16T00:09:38.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.370+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb93820a840 con 0x7fb9382055b0 2024-09-16T00:09:38.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.370+0000 7fb649f5b640 1 -- start start 2024-09-16T00:09:38.470 INFO:tasks.workunit.client.0.smithi139.stdout:5/73: dread d1/fa [0,4194304] 0 2024-09-16T00:09:38.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.371+0000 7fb649f5b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6440c57d0 0x7fb64415aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.371+0000 7fb649f5b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6440caa00 con 0x7fb644151970 2024-09-16T00:09:38.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.371+0000 7fb649f5b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6440c4720 con 0x7fb64406f010 2024-09-16T00:09:38.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.371+0000 7fb649f5b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb6440c49c0 con 0x7fb6440c57d0 2024-09-16T00:09:38.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.372+0000 7fb642ffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6440c57d0 0x7fb64415aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.372+0000 7fb642ffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6440c57d0 0x7fb64415aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:35216/0 (socket says 172.21.15.139:35216) 2024-09-16T00:09:38.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.372+0000 7fb642ffd640 1 -- 172.21.15.139:0/2010552021 learned_addr learned my addr 172.21.15.139:0/2010552021 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:38.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.372+0000 7fb642ffd640 1 -- 172.21.15.139:0/2010552021 >> v1:172.21.15.164:6789/0 conn(0x7fb64406f010 legacy=0x7fb644161dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:38.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.372+0000 7fb642ffd640 1 -- 172.21.15.139:0/2010552021 >> v1:172.21.15.139:6789/0 conn(0x7fb644151970 legacy=0x7fb6440ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:38.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.372+0000 7fb642ffd640 1 -- 172.21.15.139:0/2010552021 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64406f590 con 0x7fb6440c57d0 2024-09-16T00:09:38.471 INFO:tasks.workunit.client.0.smithi139.stdout:8/49: unlink ff 0 2024-09-16T00:09:38.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.373+0000 7fb642ffd640 1 --2- 172.21.15.139:0/2010552021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6440c57d0 0x7fb64415aa40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb6380098a0 tx=0x7fb63802a370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4d616c2738b761e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:38.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.373+0000 7fb6427fc640 1 -- 172.21.15.139:0/2010552021 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638037070 con 0x7fb6440c57d0 2024-09-16T00:09:38.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.373+0000 7fb6427fc640 1 -- 172.21.15.139:0/2010552021 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb638032640 con 0x7fb6440c57d0 2024-09-16T00:09:38.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.373+0000 7fb6427fc640 1 -- 172.21.15.139:0/2010552021 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638032a10 con 0x7fb6440c57d0 2024-09-16T00:09:38.472 INFO:tasks.workunit.client.0.smithi139.stdout:8/50: dread - f8 zero size 2024-09-16T00:09:38.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.374+0000 7fb649f5b640 1 -- 172.21.15.139:0/2010552021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6440c57d0 msgr2=0x7fb64415aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:38.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.374+0000 7fb649f5b640 1 --2- 172.21.15.139:0/2010552021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6440c57d0 0x7fb64415aa40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb6380098a0 tx=0x7fb63802a370 comp rx=0 tx=0).stop 2024-09-16T00:09:38.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.374+0000 7fb649f5b640 1 -- 172.21.15.139:0/2010552021 shutdown_connections 2024-09-16T00:09:38.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.374+0000 7fb649f5b640 1 --2- 172.21.15.139:0/2010552021 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb6440c57d0 0x7fb64415aa40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:38.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.374+0000 7fb649f5b640 1 -- 172.21.15.139:0/2010552021 >> 172.21.15.139:0/2010552021 conn(0x7fb64406cc30 msgr2=0x7fb64406d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:38.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.375+0000 7fb649f5b640 1 -- 172.21.15.139:0/2010552021 shutdown_connections 2024-09-16T00:09:38.473 INFO:tasks.workunit.client.0.smithi139.stdout:6/62: write d8/d9/d10/f13 [4413281,54590] 0 2024-09-16T00:09:38.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.375+0000 7fb649f5b640 1 -- 172.21.15.139:0/2010552021 wait complete. 2024-09-16T00:09:38.474 INFO:tasks.workunit.client.0.smithi139.stdout:7/34: dwrite f1 [0,4194304] 0 2024-09-16T00:09:38.474 INFO:tasks.workunit.client.0.smithi139.stdout:4/26: mknod c8 0 2024-09-16T00:09:38.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.376+0000 7fb649f5b640 1 Processor -- start 2024-09-16T00:09:38.475 INFO:tasks.workunit.client.0.smithi139.stdout:3/104: rename d5/da/l14 to d5/d7/l1a 0 2024-09-16T00:09:38.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.377+0000 7fb649f5b640 1 -- start start 2024-09-16T00:09:38.475 INFO:tasks.workunit.client.0.smithi139.stdout:8/51: mknod c13 0 2024-09-16T00:09:38.475 INFO:tasks.workunit.client.0.smithi139.stdout:2/53: truncate f8 1194504 0 2024-09-16T00:09:38.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.379+0000 7fb649f5b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb64406f010 0x7fb644152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.379+0000 7fb649f5b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb644151970 0x7fb6441efaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.379+0000 7fb649f5b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb644159db0 0x7fb6441f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.379+0000 7fb649f5b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb64406ec00 con 0x7fb644151970 2024-09-16T00:09:38.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.379+0000 7fb649f5b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb6440c8110 con 0x7fb64406f010 2024-09-16T00:09:38.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.379+0000 7fb649f5b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb64406fb50 con 0x7fb644159db0 2024-09-16T00:09:38.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.379+0000 7fb6437fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb64406f010 0x7fb644152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.379+0000 7fb6437fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb64406f010 0x7fb644152130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:47148/0 (socket says 172.21.15.139:47148) 2024-09-16T00:09:38.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.379+0000 7fb6437fe640 1 -- 172.21.15.139:0/705111139 learned_addr learned my addr 172.21.15.139:0/705111139 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:38.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.379+0000 7fb642ffd640 1 --2- 172.21.15.139:0/705111139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb644151970 0x7fb6441efaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.379+0000 7fb643fff640 1 --2- 172.21.15.139:0/705111139 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb644159db0 0x7fb6441f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.379+0000 7fb6437fe640 1 -- 172.21.15.139:0/705111139 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb644159db0 msgr2=0x7fb6441f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:38.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.379+0000 7fb6437fe640 1 --2- 172.21.15.139:0/705111139 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb644159db0 0x7fb6441f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:38.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.379+0000 7fb6437fe640 1 -- 172.21.15.139:0/705111139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb644151970 msgr2=0x7fb6441efaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:38.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.379+0000 7fb6437fe640 1 --2- 172.21.15.139:0/705111139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb644151970 0x7fb6441efaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:38.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.379+0000 7fb6437fe640 1 -- 172.21.15.139:0/705111139 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6441f1360 con 0x7fb64406f010 2024-09-16T00:09:38.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.380+0000 7fb6437fe640 1 --2- 172.21.15.139:0/705111139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb64406f010 0x7fb644152130 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb63400c430 tx=0x7fb63400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:38.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.380+0000 7fb62bfff640 1 -- 172.21.15.139:0/705111139 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634017070 con 0x7fb64406f010 2024-09-16T00:09:38.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.380+0000 7fb62bfff640 1 -- 172.21.15.139:0/705111139 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6340052d0 con 0x7fb64406f010 2024-09-16T00:09:38.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.380+0000 7fb62bfff640 1 -- 172.21.15.139:0/705111139 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634003830 con 0x7fb64406f010 2024-09-16T00:09:38.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.380+0000 7fb649f5b640 1 -- 172.21.15.139:0/705111139 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6441f1640 con 0x7fb64406f010 2024-09-16T00:09:38.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.380+0000 7fb649f5b640 1 -- 172.21.15.139:0/705111139 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6440cc5c0 con 0x7fb64406f010 2024-09-16T00:09:38.479 INFO:tasks.workunit.client.0.smithi139.stdout:7/35: symlink d5/la 0 2024-09-16T00:09:38.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.382+0000 7fb649f5b640 1 -- 172.21.15.139:0/705111139 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6440c71b0 con 0x7fb64406f010 2024-09-16T00:09:38.479 INFO:tasks.workunit.client.0.smithi139.stdout:4/27: mknod c9 0 2024-09-16T00:09:38.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.383+0000 7fb62bfff640 1 -- 172.21.15.139:0/705111139 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb63401a070 con 0x7fb64406f010 2024-09-16T00:09:38.480 INFO:tasks.workunit.client.0.smithi139.stdout:3/105: mknod d5/d7/dc/c1b 0 2024-09-16T00:09:38.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.383+0000 7fb62bfff640 1 --2- 172.21.15.139:0/705111139 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb63006c2b0 0x7fb63006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:38.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.384+0000 7fb62bfff640 1 -- 172.21.15.139:0/705111139 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5973+0+0 (secure 0 0 0) 0x7fb63408a930 con 0x7fb64406f010 2024-09-16T00:09:38.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.384+0000 7fb642ffd640 1 --2- 172.21.15.139:0/705111139 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb63006c2b0 0x7fb63006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:38.481 INFO:tasks.workunit.client.0.smithi139.stdout:2/54: creat ff x:0 0 0 2024-09-16T00:09:38.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.385+0000 7fb642ffd640 1 --2- 172.21.15.139:0/705111139 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb63006c2b0 0x7fb63006e770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb638034000 tx=0x7fb63802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:38.482 INFO:tasks.workunit.client.0.smithi139.stdout:7/36: chown c3 1532 1 2024-09-16T00:09:38.482 INFO:tasks.workunit.client.0.smithi139.stdout:8/52: link fb f14 0 2024-09-16T00:09:38.482 INFO:tasks.workunit.client.0.smithi139.stdout:0/73: getdents d6 0 2024-09-16T00:09:38.482 INFO:tasks.workunit.client.0.smithi139.stdout:2/55: mknod c10 0 2024-09-16T00:09:38.482 INFO:tasks.workunit.client.0.smithi139.stdout:3/106: symlink d5/l1c 0 2024-09-16T00:09:38.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.390+0000 7fb93de7b640 1 -- 172.21.15.139:0/1153586158 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb938000950 con 0x7fb9382055b0 2024-09-16T00:09:38.483 INFO:tasks.workunit.client.0.smithi139.stdout:6/63: dwrite f6 [0,4194304] 0 2024-09-16T00:09:38.483 INFO:tasks.workunit.client.0.smithi139.stdout:8/53: stat f4 0 2024-09-16T00:09:38.483 INFO:tasks.workunit.client.0.smithi139.stdout:8/54: rmdir - no directory 2024-09-16T00:09:38.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.392+0000 7fb92ffff640 1 -- 172.21.15.139:0/1153586158 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb938000950 con 0x7fb9382055b0 2024-09-16T00:09:38.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.392+0000 7fb62bfff640 1 -- 172.21.15.139:0/705111139 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb634058810 con 0x7fb64406f010 2024-09-16T00:09:38.484 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:38.484 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:38.484 INFO:tasks.workunit.client.0.smithi139.stdout:0/74: mknod d6/c15 0 2024-09-16T00:09:38.485 INFO:tasks.workunit.client.0.smithi139.stdout:8/55: unlink f2 0 2024-09-16T00:09:38.485 INFO:tasks.workunit.client.0.smithi139.stdout:5/74: dwrite d1/fa [8388608,4194304] 0 2024-09-16T00:09:38.485 INFO:tasks.workunit.client.0.smithi139.stdout:3/107: unlink d5/d7/dc/ce 0 2024-09-16T00:09:38.485 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T00:09:38.486 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T00:09:38.489 INFO:tasks.workunit.client.0.smithi139.stdout:0/75: mknod d6/dc/dd/c16 0 2024-09-16T00:09:38.489 INFO:tasks.workunit.client.0.smithi139.stdout:4/28: dwrite f1 [0,4194304] 0 2024-09-16T00:09:38.489 INFO:tasks.workunit.client.0.smithi139.stdout:6/64: rename d8/db/f16 to d8/db/f1a 0 2024-09-16T00:09:38.489 INFO:tasks.workunit.client.0.smithi139.stdout:4/29: chown f3 416 1 2024-09-16T00:09:38.489 INFO:tasks.workunit.client.0.smithi139.stdout:7/37: dread f1 [0,4194304] 0 2024-09-16T00:09:38.489 INFO:tasks.workunit.client.0.smithi139.stdout:3/108: mknod d5/d7/dc/d19/c1d 0 2024-09-16T00:09:38.490 INFO:tasks.workunit.client.0.smithi139.stdout:6/65: chown d8/db 104 1 2024-09-16T00:09:38.490 INFO:tasks.workunit.client.0.smithi139.stdout:0/76: symlink d6/dc/dd/l17 0 2024-09-16T00:09:38.490 INFO:tasks.workunit.client.0.smithi139.stdout:3/109: write d5/da/f15 [4304031,15091] 0 2024-09-16T00:09:38.490 INFO:tasks.workunit.client.0.smithi139.stdout:2/56: dwrite f9 [0,4194304] 0 2024-09-16T00:09:38.490 INFO:tasks.workunit.client.0.smithi139.stdout:7/38: creat d5/fb x:0 0 0 2024-09-16T00:09:38.490 INFO:tasks.workunit.client.0.smithi139.stdout:6/66: mknod d8/d9/c1b 0 2024-09-16T00:09:38.490 INFO:tasks.workunit.client.0.smithi139.stdout:0/77: mknod d6/dc/dd/c18 0 2024-09-16T00:09:38.490 INFO:tasks.workunit.client.0.smithi139.stdout:7/39: chown d5/f7 1 1 2024-09-16T00:09:38.490 INFO:tasks.workunit.client.0.smithi139.stdout:8/56: dwrite f7 [0,4194304] 0 2024-09-16T00:09:38.490 INFO:tasks.workunit.client.0.smithi139.stdout:3/110: fdatasync d5/d7/fd 0 2024-09-16T00:09:38.490 INFO:tasks.workunit.client.0.smithi139.stdout:3/111: readlink d5/l1c 0 2024-09-16T00:09:38.490 INFO:tasks.workunit.client.0.smithi139.stdout:0/78: creat d6/dc/dd/f19 x:0 0 0 2024-09-16T00:09:38.490 INFO:tasks.workunit.client.0.smithi139.stdout:4/30: dwrite f7 [0,4194304] 0 2024-09-16T00:09:38.490 INFO:tasks.workunit.client.0.smithi139.stdout:4/31: dread - f5 zero size 2024-09-16T00:09:38.491 INFO:tasks.workunit.client.0.smithi139.stdout:6/67: link d8/f12 d8/d9/f1c 0 2024-09-16T00:09:38.491 INFO:tasks.workunit.client.0.smithi139.stdout:5/75: truncate d1/fa 9414440 0 2024-09-16T00:09:38.491 INFO:tasks.workunit.client.0.smithi139.stdout:4/32: stat c9 0 2024-09-16T00:09:38.491 INFO:tasks.workunit.client.0.smithi139.stdout:2/57: dread f9 [0,4194304] 0 2024-09-16T00:09:38.491 INFO:tasks.workunit.client.0.smithi139.stdout:5/76: readlink d1/l5 0 2024-09-16T00:09:38.491 INFO:tasks.workunit.client.0.smithi139.stdout:4/33: fdatasync f5 0 2024-09-16T00:09:38.491 INFO:tasks.workunit.client.0.smithi139.stdout:2/58: write fc [1049353,63015] 0 2024-09-16T00:09:38.491 INFO:tasks.workunit.client.0.smithi139.stdout:7/40: dwrite d5/fb [0,4194304] 0 2024-09-16T00:09:38.491 INFO:tasks.workunit.client.0.smithi139.stdout:8/57: dread fb [0,4194304] 0 2024-09-16T00:09:38.491 INFO:tasks.workunit.client.0.smithi139.stdout:3/112: dwrite d5/f16 [0,4194304] 0 2024-09-16T00:09:38.491 INFO:tasks.workunit.client.0.smithi139.stdout:4/34: creat fa x:0 0 0 2024-09-16T00:09:38.491 INFO:tasks.workunit.client.0.smithi139.stdout:4/35: rmdir - no directory 2024-09-16T00:09:38.491 INFO:tasks.workunit.client.0.smithi139.stdout:0/79: dwrite d6/f7 [0,4194304] 0 2024-09-16T00:09:38.491 INFO:tasks.workunit.client.0.smithi139.stdout:3/113: stat d5/da/f15 0 2024-09-16T00:09:38.492 INFO:tasks.workunit.client.0.smithi139.stdout:6/68: dwrite f6 [4194304,4194304] 0 2024-09-16T00:09:38.492 INFO:tasks.workunit.client.0.smithi139.stdout:7/41: write d5/f6 [1337020,69348] 0 2024-09-16T00:09:38.492 INFO:tasks.workunit.client.0.smithi139.stdout:0/80: write f3 [963241,53918] 0 2024-09-16T00:09:38.492 INFO:tasks.workunit.client.0.smithi139.stdout:8/58: mknod c15 0 2024-09-16T00:09:38.492 INFO:tasks.workunit.client.0.smithi139.stdout:3/114: mknod d5/c1e 0 2024-09-16T00:09:38.492 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:38.458+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:38.495 INFO:tasks.workunit.client.0.smithi139.stdout:2/59: fdatasync f8 0 2024-09-16T00:09:38.495 INFO:tasks.workunit.client.0.smithi139.stdout:8/59: readlink l11 0 2024-09-16T00:09:38.495 INFO:tasks.workunit.client.0.smithi139.stdout:7/42: creat d5/fc x:0 0 0 2024-09-16T00:09:38.495 INFO:tasks.workunit.client.0.smithi139.stdout:3/115: write d5/f8 [138009,9381] 0 2024-09-16T00:09:38.495 INFO:tasks.workunit.client.0.smithi139.stdout:5/77: dread d1/fa [0,4194304] 0 2024-09-16T00:09:38.495 INFO:tasks.workunit.client.0.smithi139.stdout:8/60: write f14 [4881269,48938] 0 2024-09-16T00:09:38.495 INFO:tasks.workunit.client.0.smithi139.stdout:2/60: mkdir d11 0 2024-09-16T00:09:38.495 INFO:tasks.workunit.client.0.smithi139.stdout:3/116: truncate d5/d7/f13 2674872 0 2024-09-16T00:09:38.495 INFO:tasks.workunit.client.0.smithi139.stdout:8/61: fdatasync f7 0 2024-09-16T00:09:38.495 INFO:tasks.workunit.client.0.smithi139.stdout:0/81: mknod d6/dc/dd/c1a 0 2024-09-16T00:09:38.495 INFO:tasks.workunit.client.0.smithi139.stdout:7/43: mknod d5/cd 0 2024-09-16T00:09:38.496 INFO:tasks.workunit.client.0.smithi139.stdout:6/69: link d8/ca d8/d9/d10/c1d 0 2024-09-16T00:09:38.496 INFO:tasks.workunit.client.0.smithi139.stdout:0/82: write d6/f8 [2976443,8666] 0 2024-09-16T00:09:38.496 INFO:tasks.workunit.client.0.smithi139.stdout:5/78: mkdir d1/d13 0 2024-09-16T00:09:38.496 INFO:tasks.workunit.client.0.smithi139.stdout:3/117: rmdir d5/d7/dc/d19 39 2024-09-16T00:09:38.496 INFO:tasks.workunit.client.0.smithi139.stdout:7/44: rename c3 to d5/ce 0 2024-09-16T00:09:38.496 INFO:tasks.workunit.client.0.smithi139.stdout:6/70: symlink d8/d9/l1e 0 2024-09-16T00:09:38.496 INFO:tasks.workunit.client.0.smithi139.stdout:0/83: mknod d6/dc/dd/c1b 0 2024-09-16T00:09:38.496 INFO:tasks.workunit.client.0.smithi139.stdout:7/45: chown d5/fc 29 1 2024-09-16T00:09:38.496 INFO:tasks.workunit.client.0.smithi139.stdout:5/79: dread d1/fa [0,4194304] 0 2024-09-16T00:09:38.496 INFO:tasks.workunit.client.0.smithi139.stdout:0/84: chown d6/f8 173 1 2024-09-16T00:09:38.496 INFO:tasks.workunit.client.0.smithi139.stdout:0/85: write d6/ff [251457,13591] 0 2024-09-16T00:09:38.496 INFO:tasks.workunit.client.0.smithi139.stdout:3/118: dwrite d5/f16 [0,4194304] 0 2024-09-16T00:09:38.496 INFO:tasks.workunit.client.0.smithi139.stdout:5/80: dread d1/fa [0,4194304] 0 2024-09-16T00:09:38.511 INFO:tasks.workunit.client.0.smithi139.stdout:1/48: sync 2024-09-16T00:09:38.517 INFO:tasks.workunit.client.0.smithi139.stdout:1/49: dwrite d7/d8/f9 [0,4194304] 0 2024-09-16T00:09:38.525 INFO:tasks.workunit.client.0.smithi139.stdout:1/50: dwrite d7/d8/fb [0,4194304] 0 2024-09-16T00:09:38.561 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:38.560+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:38.604 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:38.605 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:38.605 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:38.605 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:38.608 INFO:tasks.workunit.client.0.smithi139.stdout:1/51: dwrite d7/d8/f9 [0,4194304] 0 2024-09-16T00:09:38.615 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:09:38.619 INFO:tasks.workunit.client.0.smithi139.stdout:8/62: link c15 c16 0 2024-09-16T00:09:38.619 INFO:tasks.workunit.client.0.smithi139.stdout:7/46: fsync f1 0 2024-09-16T00:09:38.622 INFO:tasks.workunit.client.0.smithi139.stdout:8/63: write f9 [262346,27187] 0 2024-09-16T00:09:38.622 INFO:tasks.workunit.client.0.smithi139.stdout:0/86: mkdir d6/d1c 0 2024-09-16T00:09:38.622 INFO:tasks.workunit.client.0.smithi139.stdout:5/81: mknod d1/c14 0 2024-09-16T00:09:38.623 INFO:tasks.workunit.client.0.smithi139.stdout:1/52: mknod d7/ce 0 2024-09-16T00:09:38.623 INFO:tasks.workunit.client.0.smithi139.stdout:8/64: dread fc [0,4194304] 0 2024-09-16T00:09:38.624 INFO:tasks.workunit.client.0.smithi139.stdout:1/53: write f4 [1697998,64046] 0 2024-09-16T00:09:38.625 INFO:tasks.workunit.client.0.smithi139.stdout:1/54: readlink l3 0 2024-09-16T00:09:38.625 INFO:tasks.workunit.client.0.smithi139.stdout:3/119: mknod d5/d7/c1f 0 2024-09-16T00:09:38.626 INFO:tasks.workunit.client.0.smithi139.stdout:5/82: rename d1/ce to d1/d13/c15 0 2024-09-16T00:09:38.628 INFO:tasks.workunit.client.0.smithi139.stdout:8/65: mkdir d17 0 2024-09-16T00:09:38.629 INFO:tasks.workunit.client.0.smithi139.stdout:0/87: symlink d6/d1c/l1d 0 2024-09-16T00:09:38.630 INFO:tasks.workunit.client.0.smithi139.stdout:8/66: write f9 [420292,127693] 0 2024-09-16T00:09:38.634 INFO:tasks.workunit.client.0.smithi139.stdout:7/47: getdents d5 0 2024-09-16T00:09:38.634 INFO:tasks.workunit.client.0.smithi139.stdout:1/55: dwrite d7/d8/fa [0,4194304] 0 2024-09-16T00:09:38.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:38.634+0000 7fb649f5b640 1 -- 172.21.15.139:0/705111139 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fb6440c8910 con 0x7fb64406f010 2024-09-16T00:09:38.636 INFO:tasks.workunit.client.0.smithi139.stdout:0/88: truncate d6/dc/dd/f19 501343 0 2024-09-16T00:09:38.636 INFO:tasks.workunit.client.0.smithi139.stdout:5/83: dread d1/fa [0,4194304] 0 2024-09-16T00:09:38.637 INFO:tasks.workunit.client.0.smithi139.stdout:3/120: rename d5/d7/dc/d19 to d5/d7/d20 0 2024-09-16T00:09:38.637 INFO:tasks.workunit.client.0.smithi139.stdout:1/56: chown d7/d8/fb 3787 1 2024-09-16T00:09:38.637 INFO:tasks.workunit.client.0.smithi139.stdout:7/48: mknod d5/cf 0 2024-09-16T00:09:38.638 INFO:tasks.workunit.client.0.smithi139.stdout:1/57: stat d7/fd 0 2024-09-16T00:09:38.639 INFO:tasks.workunit.client.0.smithi139.stdout:0/89: write d6/f8 [3082110,32763] 0 2024-09-16T00:09:38.639 INFO:tasks.workunit.client.0.smithi139.stdout:1/58: readlink l3 0 2024-09-16T00:09:38.640 INFO:tasks.workunit.client.0.smithi139.stdout:5/84: creat d1/f16 x:0 0 0 2024-09-16T00:09:38.641 INFO:tasks.workunit.client.0.smithi139.stdout:3/121: mknod d5/c21 0 2024-09-16T00:09:38.641 INFO:tasks.workunit.client.0.smithi139.stdout:1/59: chown d7/d8 3755729 1 2024-09-16T00:09:38.643 INFO:tasks.workunit.client.0.smithi139.stdout:7/49: rename d5/ce to d5/c10 0 2024-09-16T00:09:38.646 INFO:tasks.workunit.client.0.smithi139.stdout:7/50: chown d5 209016157 1 2024-09-16T00:09:38.646 INFO:tasks.workunit.client.0.smithi139.stdout:8/67: dwrite fc [0,4194304] 0 2024-09-16T00:09:38.647 INFO:tasks.workunit.client.0.smithi139.stdout:8/68: readlink l11 0 2024-09-16T00:09:38.648 INFO:tasks.workunit.client.0.smithi139.stdout:7/51: chown f1 67313299 1 2024-09-16T00:09:38.650 INFO:tasks.workunit.client.0.smithi139.stdout:9/77: sync 2024-09-16T00:09:38.651 INFO:tasks.workunit.client.0.smithi139.stdout:0/90: dread d6/f7 [0,4194304] 0 2024-09-16T00:09:38.652 INFO:tasks.workunit.client.0.smithi139.stdout:9/78: write f14 [355086,11039] 0 2024-09-16T00:09:38.656 INFO:tasks.workunit.client.0.smithi139.stdout:3/122: symlink d5/da/l22 0 2024-09-16T00:09:38.657 INFO:tasks.workunit.client.0.smithi139.stdout:5/85: dwrite d1/fa [0,4194304] 0 2024-09-16T00:09:38.659 INFO:tasks.workunit.client.0.smithi139.stdout:5/86: dread - d1/f16 zero size 2024-09-16T00:09:38.660 INFO:tasks.workunit.client.0.smithi139.stdout:7/52: dread d5/fb [0,4194304] 0 2024-09-16T00:09:38.661 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:38.660+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:38.662 INFO:tasks.workunit.client.0.smithi139.stdout:8/69: rename f14 to d17/f18 0 2024-09-16T00:09:38.662 INFO:tasks.workunit.client.0.smithi139.stdout:3/123: write d5/d7/f13 [2399745,3062] 0 2024-09-16T00:09:38.665 INFO:tasks.workunit.client.0.smithi139.stdout:1/60: link d7/d8/fa d7/ff 0 2024-09-16T00:09:38.668 INFO:tasks.workunit.client.0.smithi139.stdout:8/70: creat d17/f19 x:0 0 0 2024-09-16T00:09:38.668 INFO:tasks.workunit.client.0.smithi139.stdout:7/53: rename d5/la to d5/l11 0 2024-09-16T00:09:38.669 INFO:tasks.workunit.client.0.smithi139.stdout:0/91: dread d6/fa [0,4194304] 0 2024-09-16T00:09:38.671 INFO:tasks.workunit.client.0.smithi139.stdout:8/71: write f9 [119978,41801] 0 2024-09-16T00:09:38.671 INFO:tasks.workunit.client.0.smithi139.stdout:9/79: dwrite fc [8388608,4194304] 0 2024-09-16T00:09:38.671 INFO:tasks.workunit.client.0.smithi139.stdout:7/54: dread - d5/f9 zero size 2024-09-16T00:09:38.672 INFO:tasks.workunit.client.0.smithi139.stdout:9/80: fsync f2 0 2024-09-16T00:09:38.672 INFO:tasks.workunit.client.0.smithi139.stdout:0/92: stat d6/dc/dd/l12 0 2024-09-16T00:09:38.675 INFO:tasks.workunit.client.0.smithi139.stdout:7/55: creat d5/f12 x:0 0 0 2024-09-16T00:09:38.675 INFO:tasks.workunit.client.0.smithi139.stdout:1/61: dread f5 [0,4194304] 0 2024-09-16T00:09:38.686 INFO:tasks.workunit.client.0.smithi139.stdout:8/72: dwrite f7 [4194304,4194304] 0 2024-09-16T00:09:38.687 INFO:tasks.workunit.client.0.smithi139.stdout:7/56: link d5/f9 d5/f13 0 2024-09-16T00:09:38.688 INFO:tasks.workunit.client.0.smithi139.stdout:9/81: dwrite f2 [0,4194304] 0 2024-09-16T00:09:38.688 INFO:tasks.workunit.client.0.smithi139.stdout:0/93: dwrite d6/ff [0,4194304] 0 2024-09-16T00:09:38.689 INFO:tasks.workunit.client.0.smithi139.stdout:8/73: chown c6 4 1 2024-09-16T00:09:38.690 INFO:tasks.workunit.client.0.smithi139.stdout:1/62: dread d7/d8/fb [0,4194304] 0 2024-09-16T00:09:38.692 INFO:tasks.workunit.client.0.smithi139.stdout:7/57: unlink d5/f13 0 2024-09-16T00:09:38.692 INFO:tasks.workunit.client.0.smithi139.stdout:7/58: readlink l2 0 2024-09-16T00:09:38.694 INFO:tasks.workunit.client.0.smithi139.stdout:8/74: mkdir d17/d1a 0 2024-09-16T00:09:38.695 INFO:tasks.workunit.client.0.smithi139.stdout:0/94: dread d6/f7 [0,4194304] 0 2024-09-16T00:09:38.695 INFO:tasks.workunit.client.0.smithi139.stdout:8/75: write f8 [382734,111914] 0 2024-09-16T00:09:38.697 INFO:tasks.workunit.client.0.smithi139.stdout:1/63: write d7/d8/f9 [1161037,124822] 0 2024-09-16T00:09:38.698 INFO:tasks.workunit.client.0.smithi139.stdout:9/82: rmdir d17 0 2024-09-16T00:09:38.698 INFO:tasks.workunit.client.0.smithi139.stdout:8/76: fsync f8 0 2024-09-16T00:09:38.698 INFO:tasks.workunit.client.0.smithi139.stdout:9/83: stat f6 0 2024-09-16T00:09:38.698 INFO:tasks.workunit.client.0.smithi139.stdout:9/84: rmdir - no directory 2024-09-16T00:09:38.699 INFO:tasks.workunit.client.0.smithi139.stdout:8/77: chown c12 41 1 2024-09-16T00:09:38.701 INFO:tasks.workunit.client.0.smithi139.stdout:0/95: readlink d6/l10 0 2024-09-16T00:09:38.703 INFO:tasks.workunit.client.0.smithi139.stdout:1/64: creat d7/f10 x:0 0 0 2024-09-16T00:09:38.703 INFO:tasks.workunit.client.0.smithi139.stdout:7/59: dread d5/f6 [0,4194304] 0 2024-09-16T00:09:38.703 INFO:tasks.workunit.client.0.smithi139.stdout:9/85: unlink f9 0 2024-09-16T00:09:38.703 INFO:tasks.workunit.client.0.smithi139.stdout:9/86: dread - f16 zero size 2024-09-16T00:09:38.704 INFO:tasks.workunit.client.0.smithi139.stdout:8/78: truncate f4 4413313 0 2024-09-16T00:09:38.704 INFO:tasks.workunit.client.0.smithi139.stdout:7/60: truncate d5/f12 637965 0 2024-09-16T00:09:38.706 INFO:tasks.workunit.client.0.smithi139.stdout:0/96: dread d6/ff [0,4194304] 0 2024-09-16T00:09:38.707 INFO:tasks.workunit.client.0.smithi139.stdout:9/87: symlink l18 0 2024-09-16T00:09:38.707 INFO:tasks.workunit.client.0.smithi139.stdout:7/61: dread d5/f12 [0,4194304] 0 2024-09-16T00:09:38.707 INFO:tasks.workunit.client.0.smithi139.stdout:9/88: chown fc 906825098 1 2024-09-16T00:09:38.707 INFO:tasks.workunit.client.0.smithi139.stdout:8/79: creat d17/f1b x:0 0 0 2024-09-16T00:09:38.708 INFO:tasks.workunit.client.0.smithi139.stdout:1/65: link d7/d8/fa d7/d8/f11 0 2024-09-16T00:09:38.709 INFO:tasks.workunit.client.0.smithi139.stdout:7/62: creat d5/f14 x:0 0 0 2024-09-16T00:09:38.711 INFO:tasks.workunit.client.0.smithi139.stdout:9/89: mkdir d19 0 2024-09-16T00:09:38.713 INFO:tasks.workunit.client.0.smithi139.stdout:1/66: write d7/ff [204537,77150] 0 2024-09-16T00:09:38.713 INFO:tasks.workunit.client.0.smithi139.stdout:1/67: chown d7/d8 22771 1 2024-09-16T00:09:38.714 INFO:tasks.workunit.client.0.smithi139.stdout:7/63: creat d5/f15 x:0 0 0 2024-09-16T00:09:38.714 INFO:tasks.workunit.client.0.smithi139.stdout:8/80: creat d17/d1a/f1c x:0 0 0 2024-09-16T00:09:38.715 INFO:tasks.workunit.client.0.smithi139.stdout:1/68: write d7/d8/f11 [4796589,115403] 0 2024-09-16T00:09:38.717 INFO:tasks.workunit.client.0.smithi139.stdout:7/64: mkdir d5/d16 0 2024-09-16T00:09:38.718 INFO:tasks.workunit.client.0.smithi139.stdout:9/90: mkdir d19/d1a 0 2024-09-16T00:09:38.719 INFO:tasks.workunit.client.0.smithi139.stdout:0/97: rename d6/dc/dd/c18 to d6/dc/c1e 0 2024-09-16T00:09:38.720 INFO:tasks.workunit.client.0.smithi139.stdout:9/91: creat d19/f1b x:0 0 0 2024-09-16T00:09:38.721 INFO:tasks.workunit.client.0.smithi139.stdout:7/65: symlink d5/d16/l17 0 2024-09-16T00:09:38.722 INFO:tasks.workunit.client.0.smithi139.stdout:8/81: dread d17/f18 [0,4194304] 0 2024-09-16T00:09:38.723 INFO:tasks.workunit.client.0.smithi139.stdout:9/92: unlink ff 0 2024-09-16T00:09:38.724 INFO:tasks.workunit.client.0.smithi139.stdout:4/36: sync 2024-09-16T00:09:38.724 INFO:tasks.workunit.client.0.smithi139.stdout:0/98: write d6/f8 [2816454,61931] 0 2024-09-16T00:09:38.726 INFO:tasks.workunit.client.0.smithi139.stdout:1/69: dwrite d7/f10 [0,4194304] 0 2024-09-16T00:09:38.726 INFO:tasks.workunit.client.0.smithi139.stdout:8/82: write fc [2167296,15044] 0 2024-09-16T00:09:38.728 INFO:tasks.workunit.client.0.smithi139.stdout:4/37: mknod cb 0 2024-09-16T00:09:38.735 INFO:tasks.workunit.client.0.smithi139.stdout:1/70: truncate f6 4580973 0 2024-09-16T00:09:38.739 INFO:tasks.workunit.client.0.smithi139.stdout:8/83: rename f9 to d17/d1a/f1d 0 2024-09-16T00:09:38.739 INFO:tasks.workunit.client.0.smithi139.stdout:7/66: dwrite d5/fc [0,4194304] 0 2024-09-16T00:09:38.742 INFO:tasks.workunit.client.0.smithi139.stdout:8/84: fdatasync f8 0 2024-09-16T00:09:38.742 INFO:tasks.workunit.client.0.smithi139.stdout:9/93: dwrite fc [0,4194304] 0 2024-09-16T00:09:38.743 INFO:tasks.workunit.client.0.smithi139.stdout:8/85: chown c10 4922266 1 2024-09-16T00:09:38.743 INFO:tasks.workunit.client.0.smithi139.stdout:0/99: dread d6/f7 [0,4194304] 0 2024-09-16T00:09:38.744 INFO:tasks.workunit.client.0.smithi139.stdout:0/100: write f3 [544341,129929] 0 2024-09-16T00:09:38.745 INFO:tasks.workunit.client.0.smithi139.stdout:7/67: symlink d5/l18 0 2024-09-16T00:09:38.746 INFO:tasks.workunit.client.0.smithi139.stdout:9/94: rename c13 to d19/c1c 0 2024-09-16T00:09:38.747 INFO:tasks.workunit.client.0.smithi139.stdout:8/86: write fb [268120,114444] 0 2024-09-16T00:09:38.750 INFO:tasks.workunit.client.0.smithi139.stdout:0/101: write f2 [3402674,22693] 0 2024-09-16T00:09:38.752 INFO:tasks.workunit.client.0.smithi139.stdout:7/68: symlink d5/d16/l19 0 2024-09-16T00:09:38.753 INFO:tasks.workunit.client.0.smithi139.stdout:7/69: chown d5/d16/l17 53342782 1 2024-09-16T00:09:38.756 INFO:tasks.workunit.client.0.smithi139.stdout:8/87: rename fc to d17/d1a/f1e 0 2024-09-16T00:09:38.756 INFO:tasks.workunit.client.0.smithi139.stdout:0/102: symlink d6/l1f 0 2024-09-16T00:09:38.758 INFO:tasks.workunit.client.0.smithi139.stdout:8/88: mkdir d17/d1f 0 2024-09-16T00:09:38.759 INFO:tasks.workunit.client.0.smithi139.stdout:0/103: rmdir d6/dc/dd 39 2024-09-16T00:09:38.762 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:38.760+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:38.765 INFO:tasks.workunit.client.0.smithi139.stdout:0/104: readlink d6/dc/dd/l12 0 2024-09-16T00:09:38.770 INFO:tasks.workunit.client.0.smithi139.stdout:0/105: creat d6/d1c/f20 x:0 0 0 2024-09-16T00:09:38.774 INFO:tasks.workunit.client.0.smithi139.stdout:8/89: dwrite d17/d1a/f1d [0,4194304] 0 2024-09-16T00:09:38.774 INFO:tasks.workunit.client.0.smithi139.stdout:7/70: dwrite d5/f9 [0,4194304] 0 2024-09-16T00:09:38.775 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:38.775 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:38.775 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:38.776 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:38.776 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:38.776 INFO:tasks.workunit.client.0.smithi139.stdout:0/106: creat d6/d1c/f21 x:0 0 0 2024-09-16T00:09:38.777 INFO:tasks.workunit.client.0.smithi139.stdout:7/71: chown d5/d16/l19 706 1 2024-09-16T00:09:38.777 INFO:tasks.workunit.client.0.smithi139.stdout:7/72: fsync f4 0 2024-09-16T00:09:38.778 INFO:tasks.workunit.client.0.smithi139.stdout:0/107: write f2 [630979,38387] 0 2024-09-16T00:09:38.781 INFO:tasks.workunit.client.0.smithi139.stdout:0/108: write d6/dc/dd/f19 [1030602,23547] 0 2024-09-16T00:09:38.785 INFO:tasks.workunit.client.0.smithi139.stdout:0/109: symlink d6/dc/l22 0 2024-09-16T00:09:38.786 INFO:tasks.workunit.client.0.smithi139.stdout:0/110: fsync d6/fa 0 2024-09-16T00:09:38.787 INFO:tasks.workunit.client.0.smithi139.stdout:0/111: write d6/fa [555952,66540] 0 2024-09-16T00:09:38.789 INFO:tasks.workunit.client.0.smithi139.stdout:0/112: write d6/dc/dd/f11 [899417,36366] 0 2024-09-16T00:09:38.793 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:09:38.793 INFO:tasks.workunit.client.0.smithi139.stdout:0/113: rename d6/fa to d6/dc/f23 0 2024-09-16T00:09:38.795 INFO:tasks.workunit.client.0.smithi139.stdout:6/71: write d8/f12 [3724326,130644] 0 2024-09-16T00:09:38.795 INFO:tasks.workunit.client.0.smithi139.stdout:0/114: rename d6/d1c/f20 to d6/dc/dd/f24 0 2024-09-16T00:09:38.798 INFO:tasks.workunit.client.0.smithi139.stdout:6/72: getdents d8/db 0 2024-09-16T00:09:38.799 INFO:tasks.workunit.client.0.smithi139.stdout:5/87: stat d1/d13/c15 0 2024-09-16T00:09:38.801 INFO:tasks.workunit.client.0.smithi139.stdout:6/73: mkdir d8/d1f 0 2024-09-16T00:09:38.802 INFO:tasks.workunit.client.0.smithi139.stdout:5/88: link d1/f16 d1/d13/f17 0 2024-09-16T00:09:38.803 INFO:tasks.workunit.client.0.smithi139.stdout:5/89: rename d1 to d1/d18 22 2024-09-16T00:09:38.803 INFO:tasks.workunit.client.0.smithi139.stdout:5/90: chown d1/d13/f17 28 1 2024-09-16T00:09:38.804 INFO:tasks.workunit.client.0.smithi139.stdout:5/91: chown d1/d13 1035112 1 2024-09-16T00:09:38.804 INFO:tasks.workunit.client.0.smithi139.stdout:3/124: getdents d5/d7 0 2024-09-16T00:09:38.807 INFO:tasks.workunit.client.0.smithi139.stdout:5/92: truncate d1/d13/f17 646380 0 2024-09-16T00:09:38.809 INFO:tasks.workunit.client.0.smithi139.stdout:7/73: unlink d5/c10 0 2024-09-16T00:09:38.810 INFO:tasks.workunit.client.0.smithi139.stdout:5/93: write d1/fa [2018543,9426] 0 2024-09-16T00:09:38.812 INFO:tasks.workunit.client.0.smithi139.stdout:6/74: dwrite d8/db/f1a [0,4194304] 0 2024-09-16T00:09:38.812 INFO:tasks.workunit.client.0.smithi139.stdout:7/74: mknod d5/c1a 0 2024-09-16T00:09:38.813 INFO:tasks.workunit.client.0.smithi139.stdout:5/94: rmdir d1 39 2024-09-16T00:09:38.814 INFO:tasks.workunit.client.0.smithi139.stdout:3/125: dread d5/d7/dc/f12 [0,4194304] 0 2024-09-16T00:09:38.814 INFO:tasks.workunit.client.0.smithi139.stdout:6/75: rmdir d8/db 39 2024-09-16T00:09:38.815 INFO:tasks.workunit.client.0.smithi139.stdout:3/126: fsync d5/d7/f13 0 2024-09-16T00:09:38.817 INFO:tasks.workunit.client.0.smithi139.stdout:5/95: mknod d1/c19 0 2024-09-16T00:09:38.817 INFO:tasks.workunit.client.0.smithi139.stdout:6/76: write d8/d9/d10/f17 [651760,59612] 0 2024-09-16T00:09:38.818 INFO:tasks.workunit.client.0.smithi139.stdout:3/127: dread - d5/da/f17 zero size 2024-09-16T00:09:38.820 INFO:tasks.workunit.client.0.smithi139.stdout:3/128: write d5/da/f17 [582563,9583] 0 2024-09-16T00:09:38.820 INFO:tasks.workunit.client.0.smithi139.stdout:5/96: creat d1/f1a x:0 0 0 2024-09-16T00:09:38.827 INFO:tasks.workunit.client.0.smithi139.stdout:5/97: rename d1/c14 to d1/d13/c1b 0 2024-09-16T00:09:38.828 INFO:tasks.workunit.client.0.smithi139.stdout:6/77: getdents d8/d9 0 2024-09-16T00:09:38.828 INFO:tasks.workunit.client.0.smithi139.stdout:3/129: dread d5/d7/f13 [0,4194304] 0 2024-09-16T00:09:38.829 INFO:tasks.workunit.client.0.smithi139.stdout:5/98: dread d1/f16 [0,4194304] 0 2024-09-16T00:09:38.831 INFO:tasks.workunit.client.0.smithi139.stdout:5/99: symlink d1/d13/l1c 0 2024-09-16T00:09:38.832 INFO:tasks.workunit.client.0.smithi139.stdout:5/100: chown d1/d13/l1c 259 1 2024-09-16T00:09:38.833 INFO:tasks.workunit.client.0.smithi139.stdout:3/130: dread d5/d7/f13 [0,4194304] 0 2024-09-16T00:09:38.833 INFO:tasks.workunit.client.0.smithi139.stdout:5/101: chown d1/c19 115764224 1 2024-09-16T00:09:38.835 INFO:tasks.workunit.client.0.smithi139.stdout:3/131: rename d5/da to d5/d7/d23 0 2024-09-16T00:09:38.836 INFO:tasks.workunit.client.0.smithi139.stdout:3/132: truncate d5/f8 2174220 0 2024-09-16T00:09:38.836 INFO:tasks.workunit.client.0.smithi139.stdout:5/102: rename d1/c19 to d1/c1d 0 2024-09-16T00:09:38.839 INFO:tasks.workunit.client.0.smithi139.stdout:3/133: creat d5/d7/d20/f24 x:0 0 0 2024-09-16T00:09:38.841 INFO:tasks.workunit.client.0.smithi139.stdout:9/95: fsync f2 0 2024-09-16T00:09:38.842 INFO:tasks.workunit.client.0.smithi139.stdout:9/96: read f6 [2257218,17825] 0 2024-09-16T00:09:38.843 INFO:tasks.workunit.client.0.smithi139.stdout:9/97: chown c15 973 1 2024-09-16T00:09:38.845 INFO:tasks.workunit.client.0.smithi139.stdout:9/98: creat d19/f1d x:0 0 0 2024-09-16T00:09:38.845 INFO:tasks.workunit.client.0.smithi139.stdout:9/99: read - f16 zero size 2024-09-16T00:09:38.846 INFO:tasks.workunit.client.0.smithi139.stdout:3/134: dwrite d5/d7/d23/f15 [0,4194304] 0 2024-09-16T00:09:38.850 INFO:tasks.workunit.client.0.smithi139.stdout:8/90: write fd [5148079,117402] 0 2024-09-16T00:09:38.851 INFO:tasks.workunit.client.0.smithi139.stdout:8/91: stat fb 0 2024-09-16T00:09:38.855 INFO:tasks.workunit.client.0.smithi139.stdout:8/92: fdatasync f4 0 2024-09-16T00:09:38.855 INFO:tasks.workunit.client.0.smithi139.stdout:1/71: truncate f4 1271990 0 2024-09-16T00:09:38.857 INFO:tasks.workunit.client.0.smithi139.stdout:7/75: getdents d5/d16 0 2024-09-16T00:09:38.858 INFO:tasks.workunit.client.0.smithi139.stdout:3/135: dwrite d5/f16 [4194304,4194304] 0 2024-09-16T00:09:38.859 INFO:tasks.workunit.client.0.smithi139.stdout:8/93: symlink d17/l20 0 2024-09-16T00:09:38.859 INFO:tasks.workunit.client.0.smithi139.stdout:9/100: dwrite f16 [0,4194304] 0 2024-09-16T00:09:38.860 INFO:tasks.workunit.client.0.smithi139.stdout:1/72: creat d7/f12 x:0 0 0 2024-09-16T00:09:38.862 INFO:tasks.workunit.client.0.smithi139.stdout:9/101: fsync fe 0 2024-09-16T00:09:38.863 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:38.862+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:38.864 INFO:tasks.workunit.client.0.smithi139.stdout:0/115: rename d6/dc/f23 to d6/dc/f25 0 2024-09-16T00:09:38.864 INFO:tasks.workunit.client.0.smithi139.stdout:9/102: readlink l18 0 2024-09-16T00:09:38.864 INFO:tasks.workunit.client.0.smithi139.stdout:3/136: write d5/d7/d23/f11 [747052,72452] 0 2024-09-16T00:09:38.865 INFO:tasks.workunit.client.0.smithi139.stdout:9/103: stat f2 0 2024-09-16T00:09:38.865 INFO:tasks.workunit.client.0.smithi139.stdout:8/94: mkdir d17/d1a/d21 0 2024-09-16T00:09:38.867 INFO:tasks.workunit.client.0.smithi139.stdout:5/103: fsync d1/d13/f17 0 2024-09-16T00:09:38.869 INFO:tasks.workunit.client.0.smithi139.stdout:3/137: creat d5/d7/d20/f25 x:0 0 0 2024-09-16T00:09:38.872 INFO:tasks.workunit.client.0.smithi139.stdout:7/76: dwrite d5/f7 [0,4194304] 0 2024-09-16T00:09:38.872 INFO:tasks.workunit.client.0.smithi139.stdout:1/73: link d7/d8/fb d7/f13 0 2024-09-16T00:09:38.872 INFO:tasks.workunit.client.0.smithi139.stdout:7/77: fdatasync f4 0 2024-09-16T00:09:38.874 INFO:tasks.workunit.client.0.smithi139.stdout:5/104: mkdir d1/d13/d1e 0 2024-09-16T00:09:38.874 INFO:tasks.workunit.client.0.smithi139.stdout:7/78: readlink d5/l18 0 2024-09-16T00:09:38.875 INFO:tasks.workunit.client.0.smithi139.stdout:5/105: read d1/f16 [561571,41542] 0 2024-09-16T00:09:38.877 INFO:tasks.workunit.client.0.smithi139.stdout:8/95: link d17/f1b d17/d1a/d21/f22 0 2024-09-16T00:09:38.877 INFO:tasks.workunit.client.0.smithi139.stdout:5/106: rename d1 to d1/d1f 22 2024-09-16T00:09:38.877 INFO:tasks.workunit.client.0.smithi139.stdout:9/104: dread f6 [0,4194304] 0 2024-09-16T00:09:38.879 INFO:tasks.workunit.client.0.smithi139.stdout:5/107: read d1/fa [7698920,9906] 0 2024-09-16T00:09:38.879 INFO:tasks.workunit.client.0.smithi139.stdout:8/96: write d17/d1a/f1d [3328846,94254] 0 2024-09-16T00:09:38.879 INFO:tasks.workunit.client.0.smithi139.stdout:1/74: link d7/f13 d7/d8/f14 0 2024-09-16T00:09:38.881 INFO:tasks.workunit.client.0.smithi139.stdout:7/79: dread d5/f12 [0,4194304] 0 2024-09-16T00:09:38.882 INFO:tasks.workunit.client.0.smithi139.stdout:7/80: truncate d5/f15 910460 0 2024-09-16T00:09:38.882 INFO:tasks.workunit.client.0.smithi139.stdout:9/105: chown d19/c1c 385116788 1 2024-09-16T00:09:38.884 INFO:tasks.workunit.client.0.smithi139.stdout:7/81: write d5/f14 [835149,114140] 0 2024-09-16T00:09:38.884 INFO:tasks.workunit.client.0.smithi139.stdout:5/108: mknod d1/d13/c20 0 2024-09-16T00:09:38.889 INFO:tasks.workunit.client.0.smithi139.stdout:5/109: dread - d1/f1a zero size 2024-09-16T00:09:38.890 INFO:tasks.workunit.client.0.smithi139.stdout:7/82: truncate d5/fc 4998643 0 2024-09-16T00:09:38.890 INFO:tasks.workunit.client.0.smithi139.stdout:8/97: symlink d17/d1a/d21/l23 0 2024-09-16T00:09:38.890 INFO:tasks.workunit.client.0.smithi139.stdout:7/83: readlink d5/d16/l17 0 2024-09-16T00:09:38.890 INFO:tasks.workunit.client.0.smithi139.stdout:5/110: rmdir d1 39 2024-09-16T00:09:38.891 INFO:tasks.workunit.client.0.smithi139.stdout:8/98: mknod d17/d1f/c24 0 2024-09-16T00:09:38.891 INFO:tasks.workunit.client.0.smithi139.stdout:6/78: dwrite d8/fe [0,4194304] 0 2024-09-16T00:09:38.893 INFO:tasks.workunit.client.0.smithi139.stdout:5/111: mkdir d1/d21 0 2024-09-16T00:09:38.895 INFO:tasks.workunit.client.0.smithi139.stdout:8/99: creat d17/d1f/f25 x:0 0 0 2024-09-16T00:09:38.897 INFO:tasks.workunit.client.0.smithi139.stdout:6/79: write d8/d9/d10/f13 [8171100,62529] 0 2024-09-16T00:09:38.897 INFO:tasks.workunit.client.0.smithi139.stdout:0/116: getdents d6/dc 0 2024-09-16T00:09:38.898 INFO:tasks.workunit.client.0.smithi139.stdout:1/75: dread d7/ff [0,4194304] 0 2024-09-16T00:09:38.898 INFO:tasks.workunit.client.0.smithi139.stdout:5/112: unlink d1/d13/c20 0 2024-09-16T00:09:38.898 INFO:tasks.workunit.client.0.smithi139.stdout:2/61: sync 2024-09-16T00:09:38.899 INFO:tasks.workunit.client.0.smithi139.stdout:2/62: stat ff 0 2024-09-16T00:09:38.902 INFO:tasks.workunit.client.0.smithi139.stdout:0/117: mknod d6/dc/dd/c26 0 2024-09-16T00:09:38.903 INFO:tasks.workunit.client.0.smithi139.stdout:5/113: mkdir d1/d13/d22 0 2024-09-16T00:09:38.906 INFO:tasks.workunit.client.0.smithi139.stdout:7/84: dwrite d5/f6 [0,4194304] 0 2024-09-16T00:09:38.907 INFO:tasks.workunit.client.0.smithi139.stdout:0/118: dread - d6/dc/dd/f24 zero size 2024-09-16T00:09:38.908 INFO:tasks.workunit.client.0.smithi139.stdout:3/138: truncate d5/d7/d23/f17 87716 0 2024-09-16T00:09:38.908 INFO:tasks.workunit.client.0.smithi139.stdout:5/114: chown d1/d13/c15 3 1 2024-09-16T00:09:38.909 INFO:tasks.workunit.client.0.smithi139.stdout:8/100: getdents d17/d1a 0 2024-09-16T00:09:38.913 INFO:tasks.workunit.client.0.smithi139.stdout:2/63: link c10 d11/c12 0 2024-09-16T00:09:38.913 INFO:tasks.workunit.client.0.smithi139.stdout:8/101: dread - d17/d1a/f1c zero size 2024-09-16T00:09:38.915 INFO:tasks.workunit.client.0.smithi139.stdout:8/102: chown d17 68 1 2024-09-16T00:09:38.916 INFO:tasks.workunit.client.0.smithi139.stdout:1/76: dwrite d7/fc [0,4194304] 0 2024-09-16T00:09:38.920 INFO:tasks.workunit.client.0.smithi139.stdout:6/80: dwrite d8/db/f1a [0,4194304] 0 2024-09-16T00:09:38.920 INFO:tasks.workunit.client.0.smithi139.stdout:1/77: write d7/f10 [3661288,68552] 0 2024-09-16T00:09:38.920 INFO:tasks.workunit.client.0.smithi139.stdout:8/103: symlink d17/d1a/l26 0 2024-09-16T00:09:38.922 INFO:tasks.workunit.client.0.smithi139.stdout:8/104: dread - d17/d1a/f1c zero size 2024-09-16T00:09:38.923 INFO:tasks.workunit.client.0.smithi139.stdout:3/139: dread d5/d7/d23/f15 [0,4194304] 0 2024-09-16T00:09:38.927 INFO:tasks.workunit.client.0.smithi139.stdout:0/119: truncate d6/ff 4025178 0 2024-09-16T00:09:38.927 INFO:tasks.workunit.client.0.smithi139.stdout:5/115: link d1/d13/l1c d1/d13/d22/l23 0 2024-09-16T00:09:38.929 INFO:tasks.workunit.client.0.smithi139.stdout:2/64: getdents d11 0 2024-09-16T00:09:38.929 INFO:tasks.workunit.client.0.smithi139.stdout:6/81: getdents d8/d1f 0 2024-09-16T00:09:38.929 INFO:tasks.workunit.client.0.smithi139.stdout:8/105: dread f8 [0,4194304] 0 2024-09-16T00:09:38.934 INFO:tasks.workunit.client.0.smithi139.stdout:8/106: write f7 [7744254,9083] 0 2024-09-16T00:09:38.934 INFO:tasks.workunit.client.0.smithi139.stdout:0/120: truncate d6/dc/dd/f19 1111000 0 2024-09-16T00:09:38.934 INFO:tasks.workunit.client.0.smithi139.stdout:7/85: dwrite d5/fb [0,4194304] 0 2024-09-16T00:09:38.934 INFO:tasks.workunit.client.0.smithi139.stdout:1/78: truncate f5 1744126 0 2024-09-16T00:09:38.936 INFO:tasks.workunit.client.0.smithi139.stdout:7/86: fdatasync f4 0 2024-09-16T00:09:38.937 INFO:tasks.workunit.client.0.smithi139.stdout:5/116: dread d1/f16 [0,4194304] 0 2024-09-16T00:09:38.937 INFO:tasks.workunit.client.0.smithi139.stdout:6/82: mkdir d8/d20 0 2024-09-16T00:09:38.938 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op export --pgid 2.17 --file /var/log/ceph/exp.2.17.6 2024-09-16T00:09:38.939 INFO:tasks.workunit.client.0.smithi139.stdout:3/140: creat d5/d7/f26 x:0 0 0 2024-09-16T00:09:38.939 INFO:tasks.workunit.client.0.smithi139.stdout:2/65: dread fc [0,4194304] 0 2024-09-16T00:09:38.939 INFO:tasks.workunit.client.0.smithi139.stdout:3/141: chown d5/d7/d20/c1d 586 1 2024-09-16T00:09:38.940 INFO:tasks.workunit.client.0.smithi139.stdout:0/121: symlink d6/dc/dd/l27 0 2024-09-16T00:09:38.941 INFO:tasks.workunit.client.0.smithi139.stdout:8/107: rename c15 to d17/d1f/c27 0 2024-09-16T00:09:38.944 INFO:tasks.workunit.client.0.smithi139.stdout:5/117: creat d1/d13/f24 x:0 0 0 2024-09-16T00:09:38.945 INFO:tasks.workunit.client.0.smithi139.stdout:6/83: symlink d8/db/l21 0 2024-09-16T00:09:38.946 INFO:tasks.workunit.client.0.smithi139.stdout:2/66: creat d11/f13 x:0 0 0 2024-09-16T00:09:38.947 INFO:tasks.workunit.client.0.smithi139.stdout:3/142: mkdir d5/d7/d23/d27 0 2024-09-16T00:09:38.948 INFO:tasks.workunit.client.0.smithi139.stdout:2/67: chown d11/f13 0 1 2024-09-16T00:09:38.948 INFO:tasks.workunit.client.0.smithi139.stdout:0/122: rename d6/f7 to d6/d1c/f28 0 2024-09-16T00:09:38.950 INFO:tasks.workunit.client.0.smithi139.stdout:8/108: creat d17/d1f/f28 x:0 0 0 2024-09-16T00:09:38.950 INFO:tasks.workunit.client.0.smithi139.stdout:5/118: mkdir d1/d25 0 2024-09-16T00:09:38.951 INFO:tasks.workunit.client.0.smithi139.stdout:8/109: read fb [379765,63021] 0 2024-09-16T00:09:38.951 INFO:tasks.workunit.client.0.smithi139.stdout:6/84: dread d8/fe [0,4194304] 0 2024-09-16T00:09:38.955 INFO:tasks.workunit.client.0.smithi139.stdout:0/123: chown f2 2752682 1 2024-09-16T00:09:38.955 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:38.956 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:38.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:38.957 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:38.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.957 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:86 2.1b 2:ddb4e4d7:::200.00000002:head [write 368603~2220227 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:38.957 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.644964+0000", 2024-09-16T00:09:38.957 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 35.308086871999997, 2024-09-16T00:09:38.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011714478, 2024-09-16T00:09:38.957 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.957 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.957 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.957 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.644964+0000", 2024-09-16T00:09:38.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.644964+0000", 2024-09-16T00:09:38.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.959 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.644966+0000", 2024-09-16T00:09:38.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4789999999999999e-06 2024-09-16T00:09:38.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.959 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.648979+0000", 2024-09-16T00:09:38.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040136900000000003 2024-09-16T00:09:38.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.960 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.648993+0000", 2024-09-16T00:09:38.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3129000000000001e-05 2024-09-16T00:09:38.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.960 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649000+0000", 2024-09-16T00:09:38.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5700000000000004e-06 2024-09-16T00:09:38.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649055+0000", 2024-09-16T00:09:38.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4914999999999999e-05 2024-09-16T00:09:38.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649131+0000", 2024-09-16T00:09:38.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6055000000000003e-05 2024-09-16T00:09:38.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:38.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649258+0000", 2024-09-16T00:09:38.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000127229 2024-09-16T00:09:38.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656218+0000", 2024-09-16T00:09:38.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069600469999999996 2024-09-16T00:09:38.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656602+0000", 2024-09-16T00:09:38.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038380700000000001 2024-09-16T00:09:38.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656638+0000", 2024-09-16T00:09:38.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5725000000000001e-05 2024-09-16T00:09:38.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656679+0000", 2024-09-16T00:09:38.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0831999999999998e-05 2024-09-16T00:09:38.963 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.963 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.963 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.963 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:87 2.1b 2:ddb4e4d7:::200.00000002:head [write 2588830~1400 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:38.963 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649020+0000", 2024-09-16T00:09:38.963 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 35.304030681, 2024-09-16T00:09:38.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0083707460000000001, 2024-09-16T00:09:38.964 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.964 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.964 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.964 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649020+0000", 2024-09-16T00:09:38.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649020+0000", 2024-09-16T00:09:38.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649021+0000", 2024-09-16T00:09:38.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6000000000000002e-07 2024-09-16T00:09:38.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649034+0000", 2024-09-16T00:09:38.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2627e-05 2024-09-16T00:09:38.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649045+0000", 2024-09-16T00:09:38.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0744000000000001e-05 2024-09-16T00:09:38.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649051+0000", 2024-09-16T00:09:38.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5459999999999997e-06 2024-09-16T00:09:38.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654369+0000", 2024-09-16T00:09:38.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053179519999999999 2024-09-16T00:09:38.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654435+0000", 2024-09-16T00:09:38.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6175999999999998e-05 2024-09-16T00:09:38.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.967 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:38.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654525+0000", 2024-09-16T00:09:38.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9856000000000002e-05 2024-09-16T00:09:38.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.967 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.656225+0000", 2024-09-16T00:09:38.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001699386 2024-09-16T00:09:38.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657339+0000", 2024-09-16T00:09:38.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011145250000000001 2024-09-16T00:09:38.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657366+0000", 2024-09-16T00:09:38.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7172e-05 2024-09-16T00:09:38.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657391+0000", 2024-09-16T00:09:38.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4902e-05 2024-09-16T00:09:38.969 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.969 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.969 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.969 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:88 2.1b 2:ddb4e4d7:::200.00000002:head [write 2590230~1400 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:38.969 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649060+0000", 2024-09-16T00:09:38.969 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 35.303990878999997, 2024-09-16T00:09:38.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0088748730000000001, 2024-09-16T00:09:38.969 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.969 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.969 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.970 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649060+0000", 2024-09-16T00:09:38.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649060+0000", 2024-09-16T00:09:38.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649061+0000", 2024-09-16T00:09:38.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2300000000000001e-07 2024-09-16T00:09:38.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.971 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649064+0000", 2024-09-16T00:09:38.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3160000000000001e-06 2024-09-16T00:09:38.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.971 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649070+0000", 2024-09-16T00:09:38.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9880000000000001e-06 2024-09-16T00:09:38.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.972 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.972 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649074+0000", 2024-09-16T00:09:38.972 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4329999999999997e-06 2024-09-16T00:09:38.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.972 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.972 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654769+0000", 2024-09-16T00:09:38.972 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0056944500000000002 2024-09-16T00:09:38.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.972 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.973 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654875+0000", 2024-09-16T00:09:38.973 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000106183 2024-09-16T00:09:38.973 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.973 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.973 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:38.973 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.654994+0000", 2024-09-16T00:09:38.973 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011867900000000001 2024-09-16T00:09:38.973 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.973 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.973 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.973 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657697+0000", 2024-09-16T00:09:38.973 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027033920000000002 2024-09-16T00:09:38.973 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.973 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.974 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.974 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657882+0000", 2024-09-16T00:09:38.974 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018478900000000001 2024-09-16T00:09:38.974 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.974 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.974 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.974 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657913+0000", 2024-09-16T00:09:38.974 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1306000000000001e-05 2024-09-16T00:09:38.974 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.974 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.974 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.974 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657935+0000", 2024-09-16T00:09:38.974 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1814000000000002e-05 2024-09-16T00:09:38.975 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.975 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.975 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.975 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:89 2.1b 2:ddb4e4d7:::200.00000002:head [write 2591630~1401 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:38.975 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:38.975 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 35.303970620000001, 2024-09-16T00:09:38.975 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011802609, 2024-09-16T00:09:38.975 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.975 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.975 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.975 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.976 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.976 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:38.976 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.976 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.976 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.976 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.976 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:38.976 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.976 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.976 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.976 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.976 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649081+0000", 2024-09-16T00:09:38.976 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7100000000000002e-07 2024-09-16T00:09:38.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.977 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.977 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649089+0000", 2024-09-16T00:09:38.977 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7640000000000005e-06 2024-09-16T00:09:38.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.977 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.977 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649094+0000", 2024-09-16T00:09:38.977 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6219999999999997e-06 2024-09-16T00:09:38.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.977 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.977 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649099+0000", 2024-09-16T00:09:38.978 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6970000000000001e-06 2024-09-16T00:09:38.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.978 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.978 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.978 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655288+0000", 2024-09-16T00:09:38.978 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061885569999999999 2024-09-16T00:09:38.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.978 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.978 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.978 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655365+0000", 2024-09-16T00:09:38.978 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7723000000000001e-05 2024-09-16T00:09:38.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.978 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.979 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:38.979 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655483+0000", 2024-09-16T00:09:38.979 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117895 2024-09-16T00:09:38.979 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.979 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.979 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.979 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657758+0000", 2024-09-16T00:09:38.979 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022752369999999998 2024-09-16T00:09:38.979 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.979 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.979 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.979 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660803+0000", 2024-09-16T00:09:38.979 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030441499999999998 2024-09-16T00:09:38.979 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.980 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.980 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.980 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660850+0000", 2024-09-16T00:09:38.980 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7017000000000001e-05 2024-09-16T00:09:38.980 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.980 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.980 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.980 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660883+0000", 2024-09-16T00:09:38.980 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3476000000000003e-05 2024-09-16T00:09:38.980 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.980 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.980 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.980 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.981 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.981 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:90 2.1b 2:ddb4e4d7:::200.00000002:head [write 2593031~1404 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:38.981 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:38.981 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 35.303932715000002, 2024-09-16T00:09:38.981 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011882886, 2024-09-16T00:09:38.981 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.981 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.981 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.981 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.981 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.981 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.981 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:38.981 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.981 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.982 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.982 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:38.982 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.982 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.982 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.982 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649119+0000", 2024-09-16T00:09:38.982 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:09:38.982 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.982 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.982 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649125+0000", 2024-09-16T00:09:38.982 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1650000000000003e-06 2024-09-16T00:09:38.983 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.983 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.983 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.983 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649134+0000", 2024-09-16T00:09:38.983 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5669999999999995e-06 2024-09-16T00:09:38.983 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.983 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.983 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.983 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649141+0000", 2024-09-16T00:09:38.983 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9040000000000003e-06 2024-09-16T00:09:38.983 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.983 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.983 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.983 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655796+0000", 2024-09-16T00:09:38.984 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066553649999999999 2024-09-16T00:09:38.984 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.984 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.984 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.984 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655868+0000", 2024-09-16T00:09:38.984 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2219999999999996e-05 2024-09-16T00:09:38.984 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.984 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.984 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:38.984 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655986+0000", 2024-09-16T00:09:38.984 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117457 2024-09-16T00:09:38.984 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.984 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.985 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.985 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657794+0000", 2024-09-16T00:09:38.985 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001807894 2024-09-16T00:09:38.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.985 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.985 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.985 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660890+0000", 2024-09-16T00:09:38.985 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030958449999999998 2024-09-16T00:09:38.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.985 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.985 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.985 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660966+0000", 2024-09-16T00:09:38.985 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6382000000000002e-05 2024-09-16T00:09:38.986 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.986 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.986 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.986 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.661001+0000", 2024-09-16T00:09:38.986 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5349000000000003e-05 2024-09-16T00:09:38.986 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.986 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.986 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.986 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.986 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.986 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:276 2.1e 2:7bf7e7b5:::200.00000006:head [write 0~2705854 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e69)", 2024-09-16T00:09:38.986 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:16.488883+0000", 2024-09-16T00:09:38.986 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.464167902, 2024-09-16T00:09:38.986 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010883972, 2024-09-16T00:09:38.987 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.987 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.987 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.987 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.987 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.987 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.987 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.488883+0000", 2024-09-16T00:09:38.987 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.987 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.987 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.987 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.987 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.488883+0000", 2024-09-16T00:09:38.987 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.988 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.988 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.988 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.488884+0000", 2024-09-16T00:09:38.988 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1299999999999999e-07 2024-09-16T00:09:38.988 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.988 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.988 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493021+0000", 2024-09-16T00:09:38.988 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041369299999999996 2024-09-16T00:09:38.988 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.988 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.989 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493036+0000", 2024-09-16T00:09:38.989 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4592e-05 2024-09-16T00:09:38.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.989 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.989 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.989 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493042+0000", 2024-09-16T00:09:38.989 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6150000000000002e-06 2024-09-16T00:09:38.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.989 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.989 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.990 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493092+0000", 2024-09-16T00:09:38.990 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0126000000000001e-05 2024-09-16T00:09:38.990 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.990 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.990 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493140+0000", 2024-09-16T00:09:38.990 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8207000000000001e-05 2024-09-16T00:09:38.990 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.990 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:38.990 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.493218+0000", 2024-09-16T00:09:38.990 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8030999999999999e-05 2024-09-16T00:09:38.991 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.991 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.991 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:38.991 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.499383+0000", 2024-09-16T00:09:38.991 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061649699999999997 2024-09-16T00:09:38.991 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.991 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.991 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:38.991 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.499696+0000", 2024-09-16T00:09:38.991 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00031260499999999998 2024-09-16T00:09:38.991 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.992 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.992 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.992 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.499732+0000", 2024-09-16T00:09:38.992 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5868000000000003e-05 2024-09-16T00:09:38.992 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.992 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.993 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.993 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.499767+0000", 2024-09-16T00:09:38.993 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5215000000000001e-05 2024-09-16T00:09:38.993 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.993 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.993 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.993 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.993 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.993 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:10652 3.22 e82/33 3:44e46336:::100000030c1.00000000:head v 82'193, mlcod=80'191)", 2024-09-16T00:09:38.993 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.630207+0000", 2024-09-16T00:09:38.993 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.3228440949999998, 2024-09-16T00:09:38.993 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0086997610000000003, 2024-09-16T00:09:38.993 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.994 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.994 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.994 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.994 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.994 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.994 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.630207+0000", 2024-09-16T00:09:38.994 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.994 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.994 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.994 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.994 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.630207+0000", 2024-09-16T00:09:38.994 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.994 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.994 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.995 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.995 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.630207+0000", 2024-09-16T00:09:38.995 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7100000000000002e-07 2024-09-16T00:09:38.995 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.995 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.995 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:38.995 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.632621+0000", 2024-09-16T00:09:38.995 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024136230000000002 2024-09-16T00:09:38.995 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.995 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.995 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:38.995 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.632640+0000", 2024-09-16T00:09:38.995 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9057999999999999e-05 2024-09-16T00:09:38.995 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.996 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.996 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:38.996 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.632650+0000", 2024-09-16T00:09:38.996 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0227999999999999e-05 2024-09-16T00:09:38.996 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.996 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.996 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:38.996 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.632740+0000", 2024-09-16T00:09:38.996 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9882000000000004e-05 2024-09-16T00:09:38.996 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.996 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.996 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:38.996 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.632771+0000", 2024-09-16T00:09:38.997 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0837000000000003e-05 2024-09-16T00:09:38.997 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.997 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.997 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:38.997 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.638843+0000", 2024-09-16T00:09:38.997 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0060722659999999998 2024-09-16T00:09:38.997 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.997 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.997 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:38.997 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.638907+0000", 2024-09-16T00:09:38.997 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3396000000000001e-05 2024-09-16T00:09:38.997 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.998 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:38.998 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:38.998 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.998 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.998 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:10656 3.19 e82/20 3:9943069b:::100000030ce.00000000:head v 82'171, mlcod=80'169)", 2024-09-16T00:09:38.998 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.776651+0000", 2024-09-16T00:09:38.998 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.1764005289999999, 2024-09-16T00:09:38.998 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010774453999999999, 2024-09-16T00:09:38.998 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:38.998 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:38.998 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:38.998 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:38.998 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.998 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:38.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.776651+0000", 2024-09-16T00:09:38.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:38.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.776651+0000", 2024-09-16T00:09:38.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:38.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:38.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:38.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:38.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.776651+0000", 2024-09-16T00:09:38.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7000000000000001e-07 2024-09-16T00:09:38.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:39.000 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.778529+0000", 2024-09-16T00:09:39.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018784769999999999 2024-09-16T00:09:39.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:39.000 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.778555+0000", 2024-09-16T00:09:39.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5488999999999999e-05 2024-09-16T00:09:39.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.000 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.778567+0000", 2024-09-16T00:09:39.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1994000000000001e-05 2024-09-16T00:09:39.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.001 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:39.001 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.778644+0000", 2024-09-16T00:09:39.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7324999999999996e-05 2024-09-16T00:09:39.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.001 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:39.001 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.778668+0000", 2024-09-16T00:09:39.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4204000000000001e-05 2024-09-16T00:09:39.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.002 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:39.002 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.787371+0000", 2024-09-16T00:09:39.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087031320000000006 2024-09-16T00:09:39.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.002 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:39.002 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.787425+0000", 2024-09-16T00:09:39.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3563000000000002e-05 2024-09-16T00:09:39.002 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.002 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:39.002 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.002 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10662 3.1e 3:78e5e06c:::100000030c5.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:39.003 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.806162+0000", 2024-09-16T00:09:39.003 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.1468894770000002, 2024-09-16T00:09:39.003 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010831833000000001, 2024-09-16T00:09:39.003 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:39.003 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:39.003 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.003 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:39.003 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:39.003 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.003 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10662 2024-09-16T00:09:39.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.003 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:39.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.004 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:39.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.806162+0000", 2024-09-16T00:09:39.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.004 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:39.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.806162+0000", 2024-09-16T00:09:39.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.004 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:39.004 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.806165+0000", 2024-09-16T00:09:39.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.168e-06 2024-09-16T00:09:39.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.005 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:39.005 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.808001+0000", 2024-09-16T00:09:39.005 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001836124 2024-09-16T00:09:39.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.005 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:39.005 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.808020+0000", 2024-09-16T00:09:39.005 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9089999999999998e-05 2024-09-16T00:09:39.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.808029+0000", 2024-09-16T00:09:39.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3109999999999995e-06 2024-09-16T00:09:39.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:39.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.808090+0000", 2024-09-16T00:09:39.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0340999999999997e-05 2024-09-16T00:09:39.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:39.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.808213+0000", 2024-09-16T00:09:39.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012351699999999999 2024-09-16T00:09:39.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:39.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.808328+0000", 2024-09-16T00:09:39.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114479 2024-09-16T00:09:39.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:39.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.815679+0000", 2024-09-16T00:09:39.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073512430000000004 2024-09-16T00:09:39.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:39.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.816882+0000", 2024-09-16T00:09:39.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00120297 2024-09-16T00:09:39.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:39.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.816942+0000", 2024-09-16T00:09:39.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0154e-05 2024-09-16T00:09:39.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:39.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.816993+0000", 2024-09-16T00:09:39.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1436e-05 2024-09-16T00:09:39.008 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.008 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:39.009 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.009 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10672 3.1e 3:79cbb72e:::100000030b3.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:39.009 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.852365+0000", 2024-09-16T00:09:39.009 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.1006858780000002, 2024-09-16T00:09:39.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011606098, 2024-09-16T00:09:39.009 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:39.009 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:39.009 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.009 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:39.009 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:39.009 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.010 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10672 2024-09-16T00:09:39.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.010 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:39.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.010 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:39.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.852365+0000", 2024-09-16T00:09:39.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.010 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:39.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.852365+0000", 2024-09-16T00:09:39.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.010 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:39.011 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.852367+0000", 2024-09-16T00:09:39.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2570000000000002e-06 2024-09-16T00:09:39.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.011 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:39.011 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.853984+0000", 2024-09-16T00:09:39.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016169019999999999 2024-09-16T00:09:39.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.011 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:39.011 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.854013+0000", 2024-09-16T00:09:39.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8217000000000001e-05 2024-09-16T00:09:39.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.012 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.854027+0000", 2024-09-16T00:09:39.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4576000000000001e-05 2024-09-16T00:09:39.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.012 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:39.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.854125+0000", 2024-09-16T00:09:39.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8239999999999995e-05 2024-09-16T00:09:39.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.012 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:39.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.854288+0000", 2024-09-16T00:09:39.012 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000162647 2024-09-16T00:09:39.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:39.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.854422+0000", 2024-09-16T00:09:39.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013353599999999999 2024-09-16T00:09:39.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:39.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.860446+0000", 2024-09-16T00:09:39.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0060246190000000002 2024-09-16T00:09:39.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:39.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.863885+0000", 2024-09-16T00:09:39.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034386040000000001 2024-09-16T00:09:39.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:39.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.863931+0000", 2024-09-16T00:09:39.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6560999999999997e-05 2024-09-16T00:09:39.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:39.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.863971+0000", 2024-09-16T00:09:39.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9938999999999998e-05 2024-09-16T00:09:39.014 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.014 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:39.015 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.015 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.015 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.015 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10673 3.1e 3:79cbb72e:::100000030b3.00000001:head [write 2179999~63672] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:39.015 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.856342+0000", 2024-09-16T00:09:39.015 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.096709266, 2024-09-16T00:09:39.015 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010040749999999999, 2024-09-16T00:09:39.015 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:39.015 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:39.016 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.016 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:39.016 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:39.016 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.016 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10673 2024-09-16T00:09:39.016 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.016 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:39.016 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.016 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:39.016 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.856342+0000", 2024-09-16T00:09:39.016 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.017 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.017 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:39.017 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.856342+0000", 2024-09-16T00:09:39.017 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.017 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.017 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:39.017 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.856344+0000", 2024-09-16T00:09:39.017 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2840000000000001e-06 2024-09-16T00:09:39.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.017 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.017 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:39.018 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.856390+0000", 2024-09-16T00:09:39.018 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5804999999999999e-05 2024-09-16T00:09:39.018 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.018 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:39.018 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.856408+0000", 2024-09-16T00:09:39.018 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7600000000000001e-05 2024-09-16T00:09:39.018 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.018 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.018 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.856422+0000", 2024-09-16T00:09:39.018 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4294000000000001e-05 2024-09-16T00:09:39.018 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.019 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:39.019 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.859462+0000", 2024-09-16T00:09:39.019 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030397900000000001 2024-09-16T00:09:39.019 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.019 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.019 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:39.019 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.859655+0000", 2024-09-16T00:09:39.019 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019379 2024-09-16T00:09:39.019 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.019 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.019 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:39.019 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.859830+0000", 2024-09-16T00:09:39.019 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017428300000000001 2024-09-16T00:09:39.019 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.019 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.020 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:39.020 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.860963+0000", 2024-09-16T00:09:39.020 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011331850000000001 2024-09-16T00:09:39.020 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.020 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.020 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:39.020 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.866297+0000", 2024-09-16T00:09:39.020 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053339989999999999 2024-09-16T00:09:39.020 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.020 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.020 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:39.020 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.866339+0000", 2024-09-16T00:09:39.020 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2477e-05 2024-09-16T00:09:39.021 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.021 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.021 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:39.021 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.866383+0000", 2024-09-16T00:09:39.021 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3242999999999999e-05 2024-09-16T00:09:39.021 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.021 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:39.021 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.021 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.021 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.021 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10686 3.1e 3:7a8af247:::100000030e7.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:39.021 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.898090+0000", 2024-09-16T00:09:39.021 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.0549609520000001, 2024-09-16T00:09:39.021 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011792016000000001, 2024-09-16T00:09:39.022 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:39.022 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:39.022 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.022 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:39.022 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:39.022 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.022 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10686 2024-09-16T00:09:39.022 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.022 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:39.022 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.022 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:39.022 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.898090+0000", 2024-09-16T00:09:39.022 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.022 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.023 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.023 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:39.023 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.898090+0000", 2024-09-16T00:09:39.023 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.023 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.023 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.023 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:39.023 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.898094+0000", 2024-09-16T00:09:39.023 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4230000000000001e-06 2024-09-16T00:09:39.023 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.023 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.023 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:39.023 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.900140+0000", 2024-09-16T00:09:39.023 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020462919999999999 2024-09-16T00:09:39.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:39.024 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.900160+0000", 2024-09-16T00:09:39.024 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9709999999999999e-05 2024-09-16T00:09:39.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.024 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.900171+0000", 2024-09-16T00:09:39.024 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1172000000000001e-05 2024-09-16T00:09:39.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:39.025 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.900251+0000", 2024-09-16T00:09:39.025 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0378999999999997e-05 2024-09-16T00:09:39.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.025 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.025 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:39.025 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.900413+0000", 2024-09-16T00:09:39.025 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016207400000000001 2024-09-16T00:09:39.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.025 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.025 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:39.025 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.900537+0000", 2024-09-16T00:09:39.025 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012429300000000001 2024-09-16T00:09:39.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.025 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.026 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:39.026 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.906910+0000", 2024-09-16T00:09:39.026 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006372043 2024-09-16T00:09:39.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.026 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.026 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:39.026 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.909741+0000", 2024-09-16T00:09:39.026 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028318100000000001 2024-09-16T00:09:39.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.026 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.026 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:39.026 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.909823+0000", 2024-09-16T00:09:39.026 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1328000000000005e-05 2024-09-16T00:09:39.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.027 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:39.027 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.909882+0000", 2024-09-16T00:09:39.027 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9491999999999999e-05 2024-09-16T00:09:39.027 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.027 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:39.027 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.027 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.027 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:10689 3.25 e82/20 3:a7febee6:::100000030e3.00000000:head v 82'165, mlcod=82'163)", 2024-09-16T00:09:39.027 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.915465+0000", 2024-09-16T00:09:39.027 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.0375859080000001, 2024-09-16T00:09:39.027 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010973778, 2024-09-16T00:09:39.027 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:39.028 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:39.028 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.028 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:39.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.028 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:39.028 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.915465+0000", 2024-09-16T00:09:39.028 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.028 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:39.028 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.915465+0000", 2024-09-16T00:09:39.028 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.029 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:39.029 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.915467+0000", 2024-09-16T00:09:39.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6419999999999999e-06 2024-09-16T00:09:39.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.029 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:39.029 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.917259+0000", 2024-09-16T00:09:39.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017919609999999999 2024-09-16T00:09:39.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.029 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:39.029 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.917281+0000", 2024-09-16T00:09:39.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1725999999999998e-05 2024-09-16T00:09:39.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.030 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.917292+0000", 2024-09-16T00:09:39.030 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1732e-05 2024-09-16T00:09:39.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:39.030 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.917401+0000", 2024-09-16T00:09:39.030 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108539 2024-09-16T00:09:39.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:39.030 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.917429+0000", 2024-09-16T00:09:39.030 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8646999999999999e-05 2024-09-16T00:09:39.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.031 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:39.031 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.926372+0000", 2024-09-16T00:09:39.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0089427050000000004 2024-09-16T00:09:39.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.031 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:39.031 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.926439+0000", 2024-09-16T00:09:39.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6826e-05 2024-09-16T00:09:39.031 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.031 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:39.031 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.032 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.032 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10753 3.1e 3:7a8af247:::100000030e7.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:39.032 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.321167+0000", 2024-09-16T00:09:39.032 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.63188436400000003, 2024-09-16T00:09:39.032 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011267736, 2024-09-16T00:09:39.032 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:39.032 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:39.032 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.032 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:39.032 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:39.032 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.032 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10753 2024-09-16T00:09:39.032 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.032 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:39.033 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.033 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:39.033 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.321167+0000", 2024-09-16T00:09:39.033 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.033 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.033 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.033 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:39.033 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.321167+0000", 2024-09-16T00:09:39.033 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.033 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.033 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.033 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:39.033 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.321169+0000", 2024-09-16T00:09:39.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1569999999999998e-06 2024-09-16T00:09:39.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.034 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:39.034 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.322728+0000", 2024-09-16T00:09:39.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015590210000000001 2024-09-16T00:09:39.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.034 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:39.034 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.322749+0000", 2024-09-16T00:09:39.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0982e-05 2024-09-16T00:09:39.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.034 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.322778+0000", 2024-09-16T00:09:39.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8949999999999999e-05 2024-09-16T00:09:39.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:39.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.322863+0000", 2024-09-16T00:09:39.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4753999999999995e-05 2024-09-16T00:09:39.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:39.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.322969+0000", 2024-09-16T00:09:39.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000106149 2024-09-16T00:09:39.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:39.036 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.332280+0000", 2024-09-16T00:09:39.036 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0093111540000000003 2024-09-16T00:09:39.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.036 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.036 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:39.036 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.332341+0000", 2024-09-16T00:09:39.036 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1302000000000004e-05 2024-09-16T00:09:39.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.036 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.036 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:39.036 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.332434+0000", 2024-09-16T00:09:39.036 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3267e-05 2024-09-16T00:09:39.036 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.037 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:39.037 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.037 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:10774 3.19 e85/20 3:990d5343:::10000003135.00000000:head v 85'177, mlcod=85'175)", 2024-09-16T00:09:39.037 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.430373+0000", 2024-09-16T00:09:39.037 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.52267778300000001, 2024-09-16T00:09:39.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0088471680000000007, 2024-09-16T00:09:39.037 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:39.037 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:39.037 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.037 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:39.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.037 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:39.038 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.430373+0000", 2024-09-16T00:09:39.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.038 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:39.038 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.430373+0000", 2024-09-16T00:09:39.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.038 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:39.038 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.430376+0000", 2024-09-16T00:09:39.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4490000000000002e-06 2024-09-16T00:09:39.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.039 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:39.039 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.432781+0000", 2024-09-16T00:09:39.039 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024054380000000002 2024-09-16T00:09:39.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.039 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.039 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:39.039 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.432803+0000", 2024-09-16T00:09:39.039 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1742e-05 2024-09-16T00:09:39.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.039 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.039 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.039 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.432813+0000", 2024-09-16T00:09:39.039 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0268000000000001e-05 2024-09-16T00:09:39.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.040 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:39.040 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.432882+0000", 2024-09-16T00:09:39.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8607000000000002e-05 2024-09-16T00:09:39.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.040 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:39.040 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.432903+0000", 2024-09-16T00:09:39.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1143000000000001e-05 2024-09-16T00:09:39.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.040 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:39.040 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.439164+0000", 2024-09-16T00:09:39.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006260603 2024-09-16T00:09:39.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.041 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.041 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:39.041 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.439220+0000", 2024-09-16T00:09:39.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6918e-05 2024-09-16T00:09:39.041 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.041 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:39.041 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.041 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.041 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:10790 3.5 e85/20 3:a3c82b2e:::10000003104.00000000:head v 85'168, mlcod=80'166)", 2024-09-16T00:09:39.041 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.530969+0000", 2024-09-16T00:09:39.041 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.42208257599999999, 2024-09-16T00:09:39.042 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0085576869999999996, 2024-09-16T00:09:39.042 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:39.042 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:39.042 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.042 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:39.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.042 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:39.042 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.530969+0000", 2024-09-16T00:09:39.042 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.042 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.042 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:39.042 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.530969+0000", 2024-09-16T00:09:39.042 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:39.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.530970+0000", 2024-09-16T00:09:39.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6899999999999999e-06 2024-09-16T00:09:39.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:39.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.534801+0000", 2024-09-16T00:09:39.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0038312670000000002 2024-09-16T00:09:39.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:39.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.534821+0000", 2024-09-16T00:09:39.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9907000000000001e-05 2024-09-16T00:09:39.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.044 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.044 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.534832+0000", 2024-09-16T00:09:39.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0747999999999999e-05 2024-09-16T00:09:39.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.044 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:39.044 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.534932+0000", 2024-09-16T00:09:39.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000100154 2024-09-16T00:09:39.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:39.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.534960+0000", 2024-09-16T00:09:39.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7783000000000001e-05 2024-09-16T00:09:39.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:39.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.539475+0000", 2024-09-16T00:09:39.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045147030000000001 2024-09-16T00:09:39.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:39.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.539526+0000", 2024-09-16T00:09:39.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1434999999999998e-05 2024-09-16T00:09:39.045 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.046 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:39.046 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.046 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10798 3.33 3:cc614d18:::100000030cb.00000000:head [write 0~4194304 [2@2683455]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:39.046 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.635480+0000", 2024-09-16T00:09:39.046 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.31757096099999998, 2024-09-16T00:09:39.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01004301, 2024-09-16T00:09:39.046 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:39.046 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:39.046 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.046 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:39.046 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:39.046 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.046 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10798 2024-09-16T00:09:39.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.047 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:39.047 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.047 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:39.047 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.635480+0000", 2024-09-16T00:09:39.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.047 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.047 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:39.047 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.635480+0000", 2024-09-16T00:09:39.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.047 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.047 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:39.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.635483+0000", 2024-09-16T00:09:39.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.943e-06 2024-09-16T00:09:39.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:39.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.638480+0000", 2024-09-16T00:09:39.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029972169999999999 2024-09-16T00:09:39.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:39.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.638503+0000", 2024-09-16T00:09:39.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2646999999999998e-05 2024-09-16T00:09:39.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.049 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.049 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.638513+0000", 2024-09-16T00:09:39.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0299000000000001e-05 2024-09-16T00:09:39.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.049 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:39.049 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.638590+0000", 2024-09-16T00:09:39.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6551000000000006e-05 2024-09-16T00:09:39.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.049 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:39.049 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.638655+0000", 2024-09-16T00:09:39.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4783000000000006e-05 2024-09-16T00:09:39.050 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.050 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.050 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:39.050 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.645384+0000", 2024-09-16T00:09:39.050 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0067295599999999999 2024-09-16T00:09:39.050 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.050 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.050 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:39.050 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.645439+0000", 2024-09-16T00:09:39.050 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5146999999999999e-05 2024-09-16T00:09:39.050 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.050 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.050 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:39.050 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.645523+0000", 2024-09-16T00:09:39.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3863000000000007e-05 2024-09-16T00:09:39.051 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.051 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:39.051 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.051 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:10802 3.25 e85/20 3:a7febee6:::100000030e3.00000000:head v 85'172, mlcod=82'170)", 2024-09-16T00:09:39.051 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.661165+0000", 2024-09-16T00:09:39.051 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.29188567799999998, 2024-09-16T00:09:39.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0089610330000000002, 2024-09-16T00:09:39.051 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:39.051 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:39.051 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.051 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:39.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:39.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.661165+0000", 2024-09-16T00:09:39.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:39.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.661165+0000", 2024-09-16T00:09:39.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:39.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.661167+0000", 2024-09-16T00:09:39.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2929999999999999e-06 2024-09-16T00:09:39.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:39.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.662847+0000", 2024-09-16T00:09:39.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001679984 2024-09-16T00:09:39.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:39.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.662868+0000", 2024-09-16T00:09:39.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1624000000000002e-05 2024-09-16T00:09:39.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.662879+0000", 2024-09-16T00:09:39.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0421000000000001e-05 2024-09-16T00:09:39.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.054 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:39.054 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.662948+0000", 2024-09-16T00:09:39.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9598999999999996e-05 2024-09-16T00:09:39.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.054 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:39.054 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.662974+0000", 2024-09-16T00:09:39.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5644999999999999e-05 2024-09-16T00:09:39.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.054 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:39.055 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.670061+0000", 2024-09-16T00:09:39.055 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0070874680000000004 2024-09-16T00:09:39.055 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.055 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:39.055 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.670126+0000", 2024-09-16T00:09:39.055 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4999000000000006e-05 2024-09-16T00:09:39.055 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.055 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:39.055 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.055 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.055 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10848 3.24 3:262efa15:::1000000311f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:39.055 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.872954+0000", 2024-09-16T00:09:39.056 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.080097373999999999, 2024-09-16T00:09:39.056 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010060827, 2024-09-16T00:09:39.056 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:39.056 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:39.056 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.056 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:39.056 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:39.056 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.056 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10848 2024-09-16T00:09:39.056 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.056 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:39.056 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.056 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:39.056 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.872954+0000", 2024-09-16T00:09:39.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:39.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.872954+0000", 2024-09-16T00:09:39.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:39.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.872957+0000", 2024-09-16T00:09:39.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9620000000000002e-06 2024-09-16T00:09:39.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.058 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:39.058 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.874763+0000", 2024-09-16T00:09:39.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001806425 2024-09-16T00:09:39.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.058 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:39.058 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.874782+0000", 2024-09-16T00:09:39.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9358999999999999e-05 2024-09-16T00:09:39.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.058 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.874795+0000", 2024-09-16T00:09:39.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2879e-05 2024-09-16T00:09:39.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.059 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:39.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.874881+0000", 2024-09-16T00:09:39.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5885999999999997e-05 2024-09-16T00:09:39.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.059 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:39.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.874971+0000", 2024-09-16T00:09:39.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9839000000000005e-05 2024-09-16T00:09:39.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:39.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.875087+0000", 2024-09-16T00:09:39.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000115721 2024-09-16T00:09:39.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:39.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.880202+0000", 2024-09-16T00:09:39.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051151479999999999 2024-09-16T00:09:39.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:39.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.882899+0000", 2024-09-16T00:09:39.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026970599999999998 2024-09-16T00:09:39.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:39.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.882961+0000", 2024-09-16T00:09:39.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1948999999999999e-05 2024-09-16T00:09:39.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:39.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.883015+0000", 2024-09-16T00:09:39.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3599e-05 2024-09-16T00:09:39.061 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.061 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:39.061 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.062 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10851 3.29 3:9786b8f5:::10000003100.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:39.062 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:09:39.062 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.060268671000000003, 2024-09-16T00:09:39.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012109089999999999, 2024-09-16T00:09:39.062 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:39.062 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:39.062 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.062 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:39.062 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:39.062 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.062 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10851 2024-09-16T00:09:39.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.063 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:39.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:39.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:09:39.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:39.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:09:39.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:39.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:39.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892786+0000", 2024-09-16T00:09:39.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4819999999999999e-06 2024-09-16T00:09:39.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:39.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894302+0000", 2024-09-16T00:09:39.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015156099999999999 2024-09-16T00:09:39.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:39.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894319+0000", 2024-09-16T00:09:39.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7098000000000001e-05 2024-09-16T00:09:39.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894329+0000", 2024-09-16T00:09:39.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9180000000000006e-06 2024-09-16T00:09:39.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:39.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894394+0000", 2024-09-16T00:09:39.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5797999999999996e-05 2024-09-16T00:09:39.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:39.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894532+0000", 2024-09-16T00:09:39.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000138128 2024-09-16T00:09:39.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:39.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894633+0000", 2024-09-16T00:09:39.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010037100000000001 2024-09-16T00:09:39.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:39.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.899212+0000", 2024-09-16T00:09:39.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045789369999999999 2024-09-16T00:09:39.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:39.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904771+0000", 2024-09-16T00:09:39.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055591169999999997 2024-09-16T00:09:39.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:39.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904838+0000", 2024-09-16T00:09:39.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7124000000000003e-05 2024-09-16T00:09:39.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:39.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:39.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:39.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904892+0000", 2024-09-16T00:09:39.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3507000000000001e-05 2024-09-16T00:09:39.067 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.067 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:39.067 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.067 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:39.068 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:39.068 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:39.069 INFO:tasks.workunit.client.0.smithi139.stdout:1/79: dwrite d7/d8/fa [0,4194304] 0 2024-09-16T00:09:39.070 INFO:tasks.workunit.client.0.smithi139.stdout:8/110: mknod d17/d1a/c29 0 2024-09-16T00:09:39.070 INFO:tasks.workunit.client.0.smithi139.stdout:6/85: chown d8/d9/l1e 210673949 1 2024-09-16T00:09:39.070 INFO:tasks.workunit.client.0.smithi139.stdout:0/124: chown f3 7854776 1 2024-09-16T00:09:39.070 INFO:tasks.workunit.client.0.smithi139.stdout:6/86: chown d8/db 78896633 1 2024-09-16T00:09:39.070 INFO:tasks.workunit.client.0.smithi139.stdout:8/111: stat d17/d1a/d21 0 2024-09-16T00:09:39.070 INFO:tasks.workunit.client.0.smithi139.stdout:7/87: truncate d5/f6 850962 0 2024-09-16T00:09:39.070 INFO:tasks.workunit.client.0.smithi139.stdout:2/68: dwrite d11/f13 [0,4194304] 0 2024-09-16T00:09:39.070 INFO:tasks.workunit.client.0.smithi139.stdout:0/125: write d6/dc/dd/f24 [909065,14131] 0 2024-09-16T00:09:39.070 INFO:tasks.workunit.client.0.smithi139.stdout:3/143: link d5/d7/f13 d5/d7/f28 0 2024-09-16T00:09:39.070 INFO:tasks.workunit.client.0.smithi139.stdout:7/88: chown d5/cf 1 1 2024-09-16T00:09:39.070 INFO:tasks.workunit.client.0.smithi139.stdout:8/112: creat d17/d1a/d21/f2a x:0 0 0 2024-09-16T00:09:39.070 INFO:tasks.workunit.client.0.smithi139.stdout:2/69: write fa [837670,66880] 0 2024-09-16T00:09:39.070 INFO:tasks.workunit.client.0.smithi139.stdout:6/87: rename d8/db/f19 to d8/d9/d10/f22 0 2024-09-16T00:09:39.071 INFO:tasks.workunit.client.0.smithi139.stdout:8/113: chown d17/d1a/d21 797755610 1 2024-09-16T00:09:39.071 INFO:tasks.workunit.client.0.smithi139.stdout:3/144: mkdir d5/d29 0 2024-09-16T00:09:39.071 INFO:tasks.workunit.client.0.smithi139.stdout:7/89: write f4 [263728,76993] 0 2024-09-16T00:09:39.071 INFO:tasks.workunit.client.0.smithi139.stdout:1/80: write d7/f13 [3031467,34086] 0 2024-09-16T00:09:39.071 INFO:tasks.workunit.client.0.smithi139.stdout:0/126: creat d6/dc/f29 x:0 0 0 2024-09-16T00:09:39.071 INFO:tasks.workunit.client.0.smithi139.stdout:6/88: write d8/fe [4823597,123567] 0 2024-09-16T00:09:39.072 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:09:39.082 INFO:tasks.workunit.client.0.smithi139.stdout:8/114: creat d17/d1a/d21/f2b x:0 0 0 2024-09-16T00:09:39.082 INFO:tasks.workunit.client.0.smithi139.stdout:8/115: stat d17/d1a/d21/f2b 0 2024-09-16T00:09:39.082 INFO:tasks.workunit.client.0.smithi139.stdout:8/116: creat d17/d1f/f2c x:0 0 0 2024-09-16T00:09:39.082 INFO:tasks.workunit.client.0.smithi139.stdout:3/145: rename d5/d7/c1f to d5/d29/c2a 0 2024-09-16T00:09:39.082 INFO:tasks.workunit.client.0.smithi139.stdout:6/89: mknod d8/d1f/c23 0 2024-09-16T00:09:39.083 INFO:tasks.workunit.client.0.smithi139.stdout:7/90: getdents d5/d16 0 2024-09-16T00:09:39.083 INFO:tasks.workunit.client.0.smithi139.stdout:1/81: getdents d7 0 2024-09-16T00:09:39.083 INFO:tasks.workunit.client.0.smithi139.stdout:7/91: mknod d5/c1b 0 2024-09-16T00:09:39.083 INFO:tasks.workunit.client.0.smithi139.stdout:8/117: write d17/d1a/d21/f2a [717322,73977] 0 2024-09-16T00:09:39.083 INFO:tasks.workunit.client.0.smithi139.stdout:7/92: write d5/f9 [4397465,122240] 0 2024-09-16T00:09:39.083 INFO:tasks.workunit.client.0.smithi139.stdout:8/118: mkdir d17/d1a/d2d 0 2024-09-16T00:09:39.083 INFO:tasks.workunit.client.0.smithi139.stdout:2/70: dwrite f8 [0,4194304] 0 2024-09-16T00:09:39.083 INFO:tasks.workunit.client.0.smithi139.stdout:0/127: dwrite f2 [0,4194304] 0 2024-09-16T00:09:39.083 INFO:tasks.workunit.client.0.smithi139.stdout:1/82: dread f5 [0,4194304] 0 2024-09-16T00:09:39.083 INFO:tasks.workunit.client.0.smithi139.stdout:2/71: write fc [2101761,106833] 0 2024-09-16T00:09:39.083 INFO:tasks.workunit.client.0.smithi139.stdout:0/128: symlink d6/l2a 0 2024-09-16T00:09:39.083 INFO:tasks.workunit.client.0.smithi139.stdout:1/83: creat d7/d8/f15 x:0 0 0 2024-09-16T00:09:39.083 INFO:tasks.workunit.client.0.smithi139.stdout:2/72: dread - fd zero size 2024-09-16T00:09:39.083 INFO:tasks.workunit.client.0.smithi139.stdout:7/93: dread d5/f14 [0,4194304] 0 2024-09-16T00:09:39.084 INFO:tasks.workunit.client.0.smithi139.stdout:8/119: rename d17/d1a/d21/f2a to d17/d1a/d2d/f2e 0 2024-09-16T00:09:39.084 INFO:tasks.workunit.client.0.smithi139.stdout:0/129: fdatasync d6/dc/f29 0 2024-09-16T00:09:39.084 INFO:tasks.workunit.client.0.smithi139.stdout:2/73: write fe [971559,24038] 0 2024-09-16T00:09:39.084 INFO:tasks.workunit.client.0.smithi139.stdout:1/84: creat d7/f16 x:0 0 0 2024-09-16T00:09:39.084 INFO:tasks.workunit.client.0.smithi139.stdout:8/120: dread f8 [0,4194304] 0 2024-09-16T00:09:39.084 INFO:tasks.workunit.client.0.smithi139.stdout:0/130: chown d6/dc/dd/l14 154572 1 2024-09-16T00:09:39.084 INFO:tasks.workunit.client.0.smithi139.stdout:2/74: write f2 [337296,68436] 0 2024-09-16T00:09:39.084 INFO:tasks.workunit.client.0.smithi139.stdout:8/121: mknod d17/d1a/d21/c2f 0 2024-09-16T00:09:39.084 INFO:tasks.workunit.client.0.smithi139.stdout:2/75: symlink d11/l14 0 2024-09-16T00:09:39.084 INFO:tasks.workunit.client.0.smithi139.stdout:7/94: dread f1 [0,4194304] 0 2024-09-16T00:09:39.084 INFO:tasks.workunit.client.0.smithi139.stdout:2/76: readlink d11/l14 0 2024-09-16T00:09:39.084 INFO:tasks.workunit.client.0.smithi139.stdout:8/122: mknod d17/d1a/d2d/c30 0 2024-09-16T00:09:39.084 INFO:tasks.workunit.client.0.smithi139.stdout:1/85: dread d7/d8/fb [0,4194304] 0 2024-09-16T00:09:39.085 INFO:tasks.workunit.client.0.smithi139.stdout:1/86: chown f4 7755 1 2024-09-16T00:09:39.085 INFO:tasks.workunit.client.0.smithi139.stdout:2/77: creat d11/f15 x:0 0 0 2024-09-16T00:09:39.085 INFO:tasks.workunit.client.0.smithi139.stdout:0/131: link d6/dc/dd/l17 d6/dc/dd/l2b 0 2024-09-16T00:09:39.085 INFO:tasks.workunit.client.0.smithi139.stdout:8/123: symlink d17/l31 0 2024-09-16T00:09:39.085 INFO:tasks.workunit.client.0.smithi139.stdout:1/87: rmdir d7/d8 39 2024-09-16T00:09:39.085 INFO:tasks.workunit.client.0.smithi139.stdout:2/78: truncate d11/f15 164744 0 2024-09-16T00:09:39.085 INFO:tasks.workunit.client.0.smithi139.stdout:0/132: write d6/dc/dd/f11 [79232,73806] 0 2024-09-16T00:09:39.085 INFO:tasks.workunit.client.0.smithi139.stdout:8/124: truncate d17/d1a/d2d/f2e 1542105 0 2024-09-16T00:09:39.085 INFO:tasks.workunit.client.0.smithi139.stdout:7/95: dwrite d5/fb [4194304,4194304] 0 2024-09-16T00:09:39.085 INFO:tasks.workunit.client.0.smithi139.stdout:0/133: creat d6/dc/dd/f2c x:0 0 0 2024-09-16T00:09:39.085 INFO:tasks.workunit.client.0.smithi139.stdout:8/125: mknod d17/d1a/c32 0 2024-09-16T00:09:39.085 INFO:tasks.workunit.client.0.smithi139.stdout:2/79: dread f8 [0,4194304] 0 2024-09-16T00:09:39.085 INFO:tasks.workunit.client.0.smithi139.stdout:8/126: chown d17/f19 7 1 2024-09-16T00:09:39.085 INFO:tasks.workunit.client.0.smithi139.stdout:1/88: dread d7/f13 [0,4194304] 0 2024-09-16T00:09:39.086 INFO:tasks.workunit.client.0.smithi139.stdout:0/134: creat d6/dc/dd/f2d x:0 0 0 2024-09-16T00:09:39.086 INFO:tasks.workunit.client.0.smithi139.stdout:2/80: creat d11/f16 x:0 0 0 2024-09-16T00:09:39.086 INFO:tasks.workunit.client.0.smithi139.stdout:2/81: chown ff 110119866 1 2024-09-16T00:09:39.086 INFO:tasks.workunit.client.0.smithi139.stdout:8/127: mknod d17/d1f/c33 0 2024-09-16T00:09:39.086 INFO:tasks.workunit.client.0.smithi139.stdout:0/135: symlink d6/dc/dd/l2e 0 2024-09-16T00:09:39.086 INFO:tasks.workunit.client.0.smithi139.stdout:0/136: write d6/d1c/f21 [652634,50930] 0 2024-09-16T00:09:39.086 INFO:tasks.workunit.client.0.smithi139.stdout:1/89: symlink d7/d8/l17 0 2024-09-16T00:09:39.086 INFO:tasks.workunit.client.0.smithi139.stdout:0/137: chown d6/dc/dd/l27 278433 1 2024-09-16T00:09:39.086 INFO:tasks.workunit.client.0.smithi139.stdout:7/96: link d5/f6 d5/f1c 0 2024-09-16T00:09:39.086 INFO:tasks.workunit.client.0.smithi139.stdout:0/138: rename d6/c15 to d6/c2f 0 2024-09-16T00:09:39.086 INFO:tasks.workunit.client.0.smithi139.stdout:1/90: rename d7/d8/fa to d7/f18 0 2024-09-16T00:09:39.086 INFO:tasks.workunit.client.0.smithi139.stdout:1/91: mknod d7/d8/c19 0 2024-09-16T00:09:39.086 INFO:tasks.workunit.client.0.smithi139.stdout:8/128: write d17/f1b [46918,2272] 0 2024-09-16T00:09:39.086 INFO:tasks.workunit.client.0.smithi139.stdout:7/97: link d5/cd d5/d16/c1d 0 2024-09-16T00:09:39.087 INFO:tasks.workunit.client.0.smithi139.stdout:1/92: write d7/f10 [556460,117072] 0 2024-09-16T00:09:39.087 INFO:tasks.workunit.client.0.smithi139.stdout:0/139: getdents d6 0 2024-09-16T00:09:39.087 INFO:tasks.workunit.client.0.smithi139.stdout:8/129: unlink d17/d1a/d21/f22 0 2024-09-16T00:09:39.087 INFO:tasks.workunit.client.0.smithi139.stdout:1/93: dread d7/f13 [0,4194304] 0 2024-09-16T00:09:39.087 INFO:tasks.workunit.client.0.smithi139.stdout:7/98: rename d5/d16/c1d to d5/d16/c1e 0 2024-09-16T00:09:39.087 INFO:tasks.workunit.client.0.smithi139.stdout:1/94: write d7/fc [3722470,40541] 0 2024-09-16T00:09:39.087 INFO:tasks.workunit.client.0.smithi139.stdout:0/140: rename d6/dc/dd/l2e to d6/dc/dd/l30 0 2024-09-16T00:09:39.087 INFO:tasks.workunit.client.0.smithi139.stdout:8/130: rmdir d17/d1a 39 2024-09-16T00:09:39.087 INFO:tasks.workunit.client.0.smithi139.stdout:7/99: dread f4 [0,4194304] 0 2024-09-16T00:09:39.087 INFO:tasks.workunit.client.0.smithi139.stdout:8/131: chown d17/d1f/f25 9 1 2024-09-16T00:09:39.087 INFO:tasks.workunit.client.0.smithi139.stdout:0/141: getdents d6/dc 0 2024-09-16T00:09:39.087 INFO:tasks.workunit.client.0.smithi139.stdout:8/132: creat d17/d1a/f34 x:0 0 0 2024-09-16T00:09:39.087 INFO:tasks.workunit.client.0.smithi139.stdout:1/95: dwrite d7/fc [0,4194304] 0 2024-09-16T00:09:39.087 INFO:tasks.workunit.client.0.smithi139.stdout:0/142: creat d6/dc/f31 x:0 0 0 2024-09-16T00:09:39.088 INFO:tasks.workunit.client.0.smithi139.stdout:8/133: creat d17/d1a/f35 x:0 0 0 2024-09-16T00:09:39.088 INFO:tasks.workunit.client.0.smithi139.stdout:8/134: dread - d17/d1f/f28 zero size 2024-09-16T00:09:39.088 INFO:tasks.workunit.client.0.smithi139.stdout:1/96: write d7/d8/f14 [3600666,38802] 0 2024-09-16T00:09:39.088 INFO:tasks.workunit.client.0.smithi139.stdout:8/135: rmdir d17/d1f 39 2024-09-16T00:09:39.088 INFO:tasks.workunit.client.0.smithi139.stdout:7/100: write d5/f1c [328317,3864] 0 2024-09-16T00:09:39.088 INFO:tasks.workunit.client.0.smithi139.stdout:7/101: chown d5/c1b 51 1 2024-09-16T00:09:39.088 INFO:tasks.workunit.client.0.smithi139.stdout:1/97: unlink d7/d8/f14 0 2024-09-16T00:09:39.088 INFO:tasks.workunit.client.0.smithi139.stdout:1/98: stat d7/ce 0 2024-09-16T00:09:39.088 INFO:tasks.workunit.client.0.smithi139.stdout:1/99: readlink l3 0 2024-09-16T00:09:39.088 INFO:tasks.workunit.client.0.smithi139.stdout:7/102: creat d5/f1f x:0 0 0 2024-09-16T00:09:39.088 INFO:tasks.workunit.client.0.smithi139.stdout:0/143: truncate d6/f8 4165388 0 2024-09-16T00:09:39.088 INFO:tasks.workunit.client.0.smithi139.stdout:1/100: truncate f5 1899993 0 2024-09-16T00:09:39.088 INFO:tasks.workunit.client.0.smithi139.stdout:0/144: write f3 [291357,127098] 0 2024-09-16T00:09:39.088 INFO:tasks.workunit.client.0.smithi139.stdout:7/103: rename d5/f7 to d5/f20 0 2024-09-16T00:09:39.089 INFO:tasks.workunit.client.0.smithi139.stdout:7/104: write d5/fc [4615243,31097] 0 2024-09-16T00:09:39.089 INFO:tasks.workunit.client.0.smithi139.stdout:1/101: rmdir d7 39 2024-09-16T00:09:39.089 INFO:tasks.workunit.client.0.smithi139.stdout:8/136: getdents d17/d1f 0 2024-09-16T00:09:39.089 INFO:tasks.workunit.client.0.smithi139.stdout:8/137: stat c6 0 2024-09-16T00:09:39.089 INFO:tasks.workunit.client.0.smithi139.stdout:7/105: creat d5/f21 x:0 0 0 2024-09-16T00:09:39.089 INFO:tasks.workunit.client.0.smithi139.stdout:8/138: creat d17/d1a/d2d/f36 x:0 0 0 2024-09-16T00:09:39.089 INFO:tasks.workunit.client.0.smithi139.stdout:7/106: unlink d5/f20 0 2024-09-16T00:09:39.089 INFO:tasks.workunit.client.0.smithi139.stdout:7/107: chown d5/c1b 13360 1 2024-09-16T00:09:39.089 INFO:tasks.workunit.client.0.smithi139.stdout:1/102: link d7/f18 d7/d8/f1a 0 2024-09-16T00:09:39.089 INFO:tasks.workunit.client.0.smithi139.stdout:8/139: dwrite fd [0,4194304] 0 2024-09-16T00:09:39.089 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:39.070+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:39.091 INFO:tasks.workunit.client.0.smithi139.stdout:0/145: getdents d6/d1c 0 2024-09-16T00:09:39.091 INFO:tasks.workunit.client.0.smithi139.stdout:1/103: dread - d7/f16 zero size 2024-09-16T00:09:39.091 INFO:tasks.workunit.client.0.smithi139.stdout:7/108: mkdir d5/d16/d22 0 2024-09-16T00:09:39.091 INFO:tasks.workunit.client.0.smithi139.stdout:7/109: symlink d5/d16/l23 0 2024-09-16T00:09:39.091 INFO:tasks.workunit.client.0.smithi139.stdout:8/140: dwrite f7 [4194304,4194304] 0 2024-09-16T00:09:39.092 INFO:tasks.workunit.client.0.smithi139.stdout:0/146: dread d6/d1c/f21 [0,4194304] 0 2024-09-16T00:09:39.092 INFO:tasks.workunit.client.0.smithi139.stdout:1/104: mkdir d7/d1b 0 2024-09-16T00:09:39.092 INFO:tasks.workunit.client.0.smithi139.stdout:7/110: symlink d5/d16/l24 0 2024-09-16T00:09:39.092 INFO:tasks.workunit.client.0.smithi139.stdout:4/38: sync 2024-09-16T00:09:39.092 INFO:tasks.workunit.client.0.smithi139.stdout:1/105: rmdir d7 39 2024-09-16T00:09:39.092 INFO:tasks.workunit.client.0.smithi139.stdout:0/147: fdatasync d6/dc/dd/f2c 0 2024-09-16T00:09:39.092 INFO:tasks.workunit.client.0.smithi139.stdout:4/39: write f3 [228418,6615] 0 2024-09-16T00:09:39.092 INFO:tasks.workunit.client.0.smithi139.stdout:4/40: dread - f6 zero size 2024-09-16T00:09:39.092 INFO:tasks.workunit.client.0.smithi139.stdout:0/148: chown d6/c2f 353361 1 2024-09-16T00:09:39.092 INFO:tasks.workunit.client.0.smithi139.stdout:8/141: dwrite d17/d1a/d2d/f36 [0,4194304] 0 2024-09-16T00:09:39.174 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:39.172+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:39.184 INFO:tasks.workunit.client.0.smithi139.stdout:1/106: read d7/f13 [1449088,114922] 0 2024-09-16T00:09:39.186 INFO:tasks.workunit.client.0.smithi139.stdout:4/41: link f1 fc 0 2024-09-16T00:09:39.186 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:39.186 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:39.186 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:39.186 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:39.187 INFO:tasks.workunit.client.0.smithi139.stdout:1/107: unlink f4 0 2024-09-16T00:09:39.187 INFO:tasks.workunit.client.0.smithi139.stdout:7/111: dwrite d5/f21 [0,4194304] 0 2024-09-16T00:09:39.191 INFO:tasks.workunit.client.0.smithi139.stdout:1/108: write d7/f10 [4144681,82775] 0 2024-09-16T00:09:39.192 INFO:tasks.workunit.client.0.smithi139.stdout:8/142: rename d17/d1a/c32 to d17/d1f/c37 0 2024-09-16T00:09:39.192 INFO:tasks.workunit.client.0.smithi139.stdout:0/149: link d6/dc/dd/l27 d6/l32 0 2024-09-16T00:09:39.193 INFO:tasks.workunit.client.0.smithi139.stdout:7/112: mkdir d5/d16/d22/d25 0 2024-09-16T00:09:39.193 INFO:tasks.workunit.client.0.smithi139.stdout:1/109: creat d7/d8/f1c x:0 0 0 2024-09-16T00:09:39.193 INFO:tasks.workunit.client.0.smithi139.stdout:8/143: dread d17/f1b [0,4194304] 0 2024-09-16T00:09:39.194 INFO:tasks.workunit.client.0.smithi139.stdout:7/113: mkdir d5/d26 0 2024-09-16T00:09:39.194 INFO:tasks.workunit.client.0.smithi139.stdout:0/150: creat d6/dc/f33 x:0 0 0 2024-09-16T00:09:39.195 INFO:tasks.workunit.client.0.smithi139.stdout:0/151: stat d6/dc/dd/c26 0 2024-09-16T00:09:39.197 INFO:tasks.workunit.client.0.smithi139.stdout:8/144: rename d17/f18 to d17/d1a/d2d/f38 0 2024-09-16T00:09:39.198 INFO:tasks.workunit.client.0.smithi139.stdout:8/145: symlink d17/d1f/l39 0 2024-09-16T00:09:39.199 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:09:39.200 INFO:tasks.workunit.client.0.smithi139.stdout:8/146: unlink f8 0 2024-09-16T00:09:39.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:39.202+0000 7fb62bfff640 1 -- 172.21.15.139:0/705111139 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v86) ==== 101+0+0 (secure 0 0 0) 0x7fb63405cb10 con 0x7fb64406f010 2024-09-16T00:09:39.205 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is unset 2024-09-16T00:09:39.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:39.205+0000 7fb649f5b640 1 -- 172.21.15.139:0/705111139 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb63006c2b0 msgr2=0x7fb63006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:39.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:39.205+0000 7fb649f5b640 1 --2- 172.21.15.139:0/705111139 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb63006c2b0 0x7fb63006e770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb638034000 tx=0x7fb63802e000 comp rx=0 tx=0).stop 2024-09-16T00:09:39.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:39.205+0000 7fb649f5b640 1 -- 172.21.15.139:0/705111139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb64406f010 msgr2=0x7fb644152130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:39.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:39.205+0000 7fb649f5b640 1 --2- 172.21.15.139:0/705111139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb64406f010 0x7fb644152130 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb63400c430 tx=0x7fb63400c8f0 comp rx=0 tx=0).stop 2024-09-16T00:09:39.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:39.205+0000 7fb649f5b640 1 -- 172.21.15.139:0/705111139 shutdown_connections 2024-09-16T00:09:39.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:39.205+0000 7fb649f5b640 1 --2- 172.21.15.139:0/705111139 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb63006c2b0 0x7fb63006e770 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:39.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:39.205+0000 7fb649f5b640 1 --2- 172.21.15.139:0/705111139 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb644159db0 0x7fb6441f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:39.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:39.205+0000 7fb649f5b640 1 --2- 172.21.15.139:0/705111139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb644151970 0x7fb6441efaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:39.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:39.205+0000 7fb649f5b640 1 --2- 172.21.15.139:0/705111139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb64406f010 0x7fb644152130 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:39.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:39.205+0000 7fb649f5b640 1 -- 172.21.15.139:0/705111139 >> 172.21.15.139:0/705111139 conn(0x7fb64406cc30 msgr2=0x7fb6440c6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:39.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:39.205+0000 7fb649f5b640 1 -- 172.21.15.139:0/705111139 shutdown_connections 2024-09-16T00:09:39.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:39.205+0000 7fb649f5b640 1 -- 172.21.15.139:0/705111139 wait complete. 2024-09-16T00:09:39.210 INFO:tasks.workunit.client.0.smithi139.stdout:1/110: write f6 [4077903,12084] 0 2024-09-16T00:09:39.210 INFO:tasks.workunit.client.0.smithi139.stdout:0/152: truncate d6/dc/f25 4030202 0 2024-09-16T00:09:39.213 INFO:tasks.workunit.client.0.smithi139.stdout:8/147: dwrite d17/d1a/d2d/f2e [0,4194304] 0 2024-09-16T00:09:39.216 INFO:tasks.workunit.client.0.smithi139.stdout:1/111: rename f6 to d7/f1d 0 2024-09-16T00:09:39.218 INFO:tasks.workunit.client.0.smithi139.stdout:8/148: mkdir d17/d1f/d3a 0 2024-09-16T00:09:39.220 INFO:tasks.workunit.client.0.smithi139.stdout:8/149: chown d17/d1a/d2d/c30 2450 1 2024-09-16T00:09:39.223 INFO:tasks.workunit.client.0.smithi139.stdout:1/112: truncate f5 1120719 0 2024-09-16T00:09:39.223 INFO:tasks.workunit.client.0.smithi139.stdout:0/153: dwrite d6/ff [4194304,4194304] 0 2024-09-16T00:09:39.224 INFO:tasks.workunit.client.0.smithi139.stdout:8/150: creat d17/d1a/d21/f3b x:0 0 0 2024-09-16T00:09:39.226 INFO:tasks.workunit.client.0.smithi139.stdout:8/151: chown d17/d1a/d2d/f2e 24393201 1 2024-09-16T00:09:39.228 INFO:tasks.workunit.client.0.smithi139.stdout:1/113: dread d7/d8/f11 [0,4194304] 0 2024-09-16T00:09:39.232 INFO:tasks.workunit.client.0.smithi139.stdout:1/114: fdatasync d7/d8/f1c 0 2024-09-16T00:09:39.235 INFO:tasks.workunit.client.0.smithi139.stdout:8/152: getdents d17/d1a/d21 0 2024-09-16T00:09:39.237 INFO:tasks.workunit.client.0.smithi139.stdout:8/153: write d17/d1a/f35 [281270,6273] 0 2024-09-16T00:09:39.237 INFO:tasks.workunit.client.0.smithi139.stdout:8/154: stat d17/d1a/f1e 0 2024-09-16T00:09:39.240 INFO:tasks.workunit.client.0.smithi139.stdout:8/155: mkdir d17/d3c 0 2024-09-16T00:09:39.240 INFO:tasks.workunit.client.0.smithi139.stdout:8/156: fsync d17/d1a/d2d/f36 0 2024-09-16T00:09:39.241 INFO:tasks.workunit.client.0.smithi139.stdout:8/157: read - d17/d1a/d21/f2b zero size 2024-09-16T00:09:39.243 INFO:tasks.workunit.client.0.smithi139.stdout:8/158: mknod d17/d1a/d21/c3d 0 2024-09-16T00:09:39.243 INFO:tasks.workunit.client.0.smithi139.stdout:1/115: write f5 [590484,113762] 0 2024-09-16T00:09:39.244 INFO:tasks.workunit.client.0.smithi139.stdout:1/116: readlink l3 0 2024-09-16T00:09:39.248 INFO:tasks.workunit.client.0.smithi139.stdout:8/159: dwrite d17/d1a/f1c [0,4194304] 0 2024-09-16T00:09:39.249 INFO:tasks.workunit.client.0.smithi139.stdout:8/160: readlink d17/d1a/l26 0 2024-09-16T00:09:39.249 INFO:tasks.workunit.client.0.smithi139.stdout:8/161: chown f7 9852024 1 2024-09-16T00:09:39.254 INFO:tasks.workunit.client.0.smithi139.stdout:8/162: truncate f7 4414666 0 2024-09-16T00:09:39.255 INFO:tasks.workunit.client.0.smithi139.stdout:8/163: write d17/d1a/f1d [3119645,32285] 0 2024-09-16T00:09:39.257 INFO:tasks.workunit.client.0.smithi139.stdout:8/164: unlink d17/f1b 0 2024-09-16T00:09:39.259 INFO:tasks.workunit.client.0.smithi139.stdout:8/165: creat d17/d1a/d2d/f3e x:0 0 0 2024-09-16T00:09:39.259 INFO:tasks.workunit.client.0.smithi139.stdout:8/166: chown d17/d1a/d21/c2f 6292 1 2024-09-16T00:09:39.263 INFO:tasks.workunit.client.0.smithi139.stdout:8/167: rmdir d17/d3c 0 2024-09-16T00:09:39.265 INFO:tasks.workunit.client.0.smithi139.stdout:8/168: creat d17/d1a/d2d/f3f x:0 0 0 2024-09-16T00:09:39.267 INFO:tasks.workunit.client.0.smithi139.stdout:8/169: mkdir d17/d1a/d21/d40 0 2024-09-16T00:09:39.268 INFO:tasks.workunit.client.0.smithi139.stdout:8/170: read d17/d1a/d2d/f36 [132565,22867] 0 2024-09-16T00:09:39.270 INFO:tasks.workunit.client.0.smithi139.stdout:8/171: getdents d17/d1a/d21/d40 0 2024-09-16T00:09:39.273 INFO:tasks.workunit.client.0.smithi139.stdout:8/172: dread d17/d1a/f1e [4194304,4194304] 0 2024-09-16T00:09:39.273 INFO:tasks.workunit.client.0.smithi139.stdout:8/173: fsync d17/d1f/f25 0 2024-09-16T00:09:39.274 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:39.273+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:39.274 INFO:tasks.workunit.client.0.smithi139.stdout:8/174: fdatasync d17/d1a/d21/f2b 0 2024-09-16T00:09:39.276 INFO:tasks.workunit.client.0.smithi139.stdout:8/175: creat d17/d1a/d21/d40/f41 x:0 0 0 2024-09-16T00:09:39.280 INFO:tasks.workunit.client.0.smithi139.stdout:8/176: link d17/d1a/d21/d40/f41 d17/d1a/d21/f42 0 2024-09-16T00:09:39.280 INFO:tasks.workunit.client.0.smithi139.stdout:8/177: dread - d17/d1a/d21/f3b zero size 2024-09-16T00:09:39.282 INFO:tasks.workunit.client.0.smithi139.stdout:8/178: fdatasync d17/d1a/d21/f42 0 2024-09-16T00:09:39.291 INFO:tasks.workunit.client.0.smithi139.stdout:8/179: rename d17/d1a/d21 to d17/d1a/d43 0 2024-09-16T00:09:39.295 INFO:tasks.workunit.client.0.smithi139.stdout:8/180: link l11 d17/l44 0 2024-09-16T00:09:39.329 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:39.329 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:39.329 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:39.329 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:39.330 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:39.338 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:09:39.376 INFO:tasks.workunit.client.0.smithi139.stdout:9/106: sync 2024-09-16T00:09:39.377 INFO:tasks.workunit.client.0.smithi139.stdout:9/107: dwrite f10 [0,4194304] 0 2024-09-16T00:09:39.378 INFO:tasks.workunit.client.0.smithi139.stdout:9/108: chown fb 212667837 1 2024-09-16T00:09:39.378 INFO:tasks.workunit.client.0.smithi139.stdout:9/109: readlink ld 0 2024-09-16T00:09:39.383 INFO:tasks.workunit.client.0.smithi139.stdout:9/110: dwrite f14 [0,4194304] 0 2024-09-16T00:09:39.384 INFO:tasks.workunit.client.0.smithi139.stdout:9/111: fsync fe 0 2024-09-16T00:09:39.385 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:39.378+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:39.389 INFO:tasks.workunit.client.0.smithi139.stdout:9/112: link fc d19/d1a/f1e 0 2024-09-16T00:09:39.403 INFO:tasks.workunit.client.0.smithi139.stdout:9/113: creat d19/f1f x:0 0 0 2024-09-16T00:09:39.404 INFO:tasks.workunit.client.0.smithi139.stdout:9/114: fsync f14 0 2024-09-16T00:09:39.407 INFO:tasks.workunit.client.0.smithi139.stdout:9/115: dread fe [0,4194304] 0 2024-09-16T00:09:39.409 INFO:tasks.workunit.client.0.smithi139.stdout:9/116: truncate d19/d1a/f1e 2506451 0 2024-09-16T00:09:39.409 INFO:tasks.workunit.client.0.smithi139.stdout:9/117: mkdir d19/d1a/d20 0 2024-09-16T00:09:39.409 INFO:tasks.workunit.client.0.smithi139.stdout:9/118: fdatasync fc 0 2024-09-16T00:09:39.409 INFO:tasks.workunit.client.0.smithi139.stdout:9/119: stat d19/f1f 0 2024-09-16T00:09:39.409 INFO:tasks.workunit.client.0.smithi139.stdout:9/120: creat d19/d1a/f21 x:0 0 0 2024-09-16T00:09:39.409 INFO:tasks.workunit.client.0.smithi139.stdout:9/121: getdents d19 0 2024-09-16T00:09:39.410 INFO:tasks.workunit.client.0.smithi139.stdout:9/122: dwrite f10 [0,4194304] 0 2024-09-16T00:09:39.411 INFO:tasks.workunit.client.0.smithi139.stdout:9/123: chown fa 90698 1 2024-09-16T00:09:39.411 INFO:tasks.workunit.client.0.smithi139.stdout:9/124: dread fb [0,4194304] 0 2024-09-16T00:09:39.415 INFO:tasks.workunit.client.0.smithi139.stdout:9/125: mkdir d19/d22 0 2024-09-16T00:09:39.416 INFO:tasks.workunit.client.0.smithi139.stdout:9/126: write f14 [2288450,91501] 0 2024-09-16T00:09:39.416 INFO:tasks.workunit.client.0.smithi139.stdout:9/127: write d19/d1a/f21 [243822,7058] 0 2024-09-16T00:09:39.416 INFO:tasks.workunit.client.0.smithi139.stdout:9/128: write fa [1269543,48944] 0 2024-09-16T00:09:39.417 INFO:tasks.workunit.client.0.smithi139.stdout:9/129: chown f16 4 1 2024-09-16T00:09:39.417 INFO:tasks.workunit.client.0.smithi139.stdout:9/130: fdatasync fa 0 2024-09-16T00:09:39.418 INFO:tasks.workunit.client.0.smithi139.stdout:9/131: mkdir d19/d1a/d20/d23 0 2024-09-16T00:09:39.433 INFO:tasks.workunit.client.0.smithi139.stdout:9/132: creat d19/d1a/d20/d23/f24 x:0 0 0 2024-09-16T00:09:39.434 INFO:tasks.workunit.client.0.smithi139.stdout:9/133: rename fc to d19/d22/f25 0 2024-09-16T00:09:39.434 INFO:tasks.workunit.client.0.smithi139.stdout:9/134: truncate d19/d1a/d20/d23/f24 841252 0 2024-09-16T00:09:39.435 INFO:tasks.workunit.client.0.smithi139.stdout:9/135: creat d19/f26 x:0 0 0 2024-09-16T00:09:39.435 INFO:tasks.workunit.client.0.smithi139.stdout:9/136: write d19/f1d [179697,72358] 0 2024-09-16T00:09:39.435 INFO:tasks.workunit.client.0.smithi139.stdout:9/137: mkdir d19/d1a/d27 0 2024-09-16T00:09:39.435 INFO:tasks.workunit.client.0.smithi139.stdout:5/119: sync 2024-09-16T00:09:39.435 INFO:tasks.workunit.client.0.smithi139.stdout:3/146: sync 2024-09-16T00:09:39.436 INFO:tasks.workunit.client.0.smithi139.stdout:5/120: read - d1/f1a zero size 2024-09-16T00:09:39.436 INFO:tasks.workunit.client.0.smithi139.stdout:3/147: write d5/d7/f26 [594385,57433] 0 2024-09-16T00:09:39.437 INFO:tasks.workunit.client.0.smithi139.stdout:3/148: unlink d5/c21 0 2024-09-16T00:09:39.437 INFO:tasks.workunit.client.0.smithi139.stdout:5/121: symlink d1/d13/d1e/l26 0 2024-09-16T00:09:39.437 INFO:tasks.workunit.client.0.smithi139.stdout:3/149: dread - d5/d7/d20/f24 zero size 2024-09-16T00:09:39.438 INFO:tasks.workunit.client.0.smithi139.stdout:5/122: creat d1/f27 x:0 0 0 2024-09-16T00:09:39.439 INFO:tasks.workunit.client.0.smithi139.stdout:9/138: dwrite d19/f1d [0,4194304] 0 2024-09-16T00:09:39.439 INFO:tasks.workunit.client.0.smithi139.stdout:9/139: mknod d19/d1a/c28 0 2024-09-16T00:09:39.439 INFO:tasks.workunit.client.0.smithi139.stdout:3/150: dread d5/d7/f13 [0,4194304] 0 2024-09-16T00:09:39.439 INFO:tasks.workunit.client.0.smithi139.stdout:9/140: read f10 [1653982,58135] 0 2024-09-16T00:09:39.439 INFO:tasks.workunit.client.0.smithi139.stdout:5/123: truncate d1/f16 1592886 0 2024-09-16T00:09:39.439 INFO:tasks.workunit.client.0.smithi139.stdout:9/141: rename d19 to d19/d22/d29 22 2024-09-16T00:09:39.439 INFO:tasks.workunit.client.0.smithi139.stdout:3/151: write d5/d7/d20/f25 [248239,34793] 0 2024-09-16T00:09:39.439 INFO:tasks.workunit.client.0.smithi139.stdout:5/124: creat d1/f28 x:0 0 0 2024-09-16T00:09:39.445 INFO:tasks.workunit.client.0.smithi139.stdout:5/125: unlink d1/c1d 0 2024-09-16T00:09:39.447 INFO:tasks.workunit.client.0.smithi139.stdout:5/126: mknod d1/d25/c29 0 2024-09-16T00:09:39.447 INFO:tasks.workunit.client.0.smithi139.stdout:3/152: link d5/c1e d5/d7/dc/c2b 0 2024-09-16T00:09:39.448 INFO:tasks.workunit.client.0.smithi139.stdout:3/153: write d5/d7/d20/f24 [152834,99800] 0 2024-09-16T00:09:39.452 INFO:tasks.workunit.client.0.smithi139.stdout:3/154: fsync d5/d7/d20/f24 0 2024-09-16T00:09:39.458 INFO:tasks.workunit.client.0.smithi139.stdout:5/127: dwrite d1/f1a [0,4194304] 0 2024-09-16T00:09:39.458 INFO:tasks.workunit.client.0.smithi139.stdout:9/142: dwrite fe [0,4194304] 0 2024-09-16T00:09:39.459 INFO:tasks.workunit.client.0.smithi139.stdout:9/143: unlink f16 0 2024-09-16T00:09:39.462 INFO:tasks.workunit.client.0.smithi139.stdout:3/155: dwrite d5/f8 [0,4194304] 0 2024-09-16T00:09:39.464 INFO:tasks.workunit.client.0.smithi139.stdout:9/144: rename c11 to d19/d1a/d20/d23/c2a 0 2024-09-16T00:09:39.466 INFO:tasks.workunit.client.0.smithi139.stdout:5/128: link d1/d13/l1c d1/d13/d22/l2a 0 2024-09-16T00:09:39.467 INFO:tasks.workunit.client.0.smithi139.stdout:3/156: dread d5/d7/f13 [0,4194304] 0 2024-09-16T00:09:39.470 INFO:tasks.workunit.client.0.smithi139.stdout:3/157: symlink d5/d29/l2c 0 2024-09-16T00:09:39.471 INFO:tasks.workunit.client.0.smithi139.stdout:9/145: dread fa [0,4194304] 0 2024-09-16T00:09:39.473 INFO:tasks.workunit.client.0.smithi139.stdout:3/158: creat d5/d7/f2d x:0 0 0 2024-09-16T00:09:39.474 INFO:tasks.workunit.client.0.smithi139.stdout:5/129: dwrite d1/f27 [0,4194304] 0 2024-09-16T00:09:39.475 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:39.474+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:39.475 INFO:tasks.workunit.client.0.smithi139.stdout:9/146: getdents d19/d22 0 2024-09-16T00:09:39.478 INFO:tasks.workunit.client.0.smithi139.stdout:9/147: rename fe to d19/d1a/d20/d23/f2b 0 2024-09-16T00:09:39.479 INFO:tasks.workunit.client.0.smithi139.stdout:3/159: creat d5/d7/f2e x:0 0 0 2024-09-16T00:09:39.480 INFO:tasks.workunit.client.0.smithi139.stdout:3/160: stat d5/d7/f2e 0 2024-09-16T00:09:39.482 INFO:tasks.workunit.client.0.smithi139.stdout:6/90: sync 2024-09-16T00:09:39.482 INFO:tasks.workunit.client.0.smithi139.stdout:0/154: sync 2024-09-16T00:09:39.482 INFO:tasks.workunit.client.0.smithi139.stdout:8/181: sync 2024-09-16T00:09:39.482 INFO:tasks.workunit.client.0.smithi139.stdout:4/42: sync 2024-09-16T00:09:39.482 INFO:tasks.workunit.client.0.smithi139.stdout:1/117: sync 2024-09-16T00:09:39.482 INFO:tasks.workunit.client.0.smithi139.stdout:2/82: sync 2024-09-16T00:09:39.483 INFO:tasks.workunit.client.0.smithi139.stdout:7/114: sync 2024-09-16T00:09:39.483 INFO:tasks.workunit.client.0.smithi139.stdout:2/83: fdatasync fa 0 2024-09-16T00:09:39.484 INFO:tasks.workunit.client.0.smithi139.stdout:9/148: rmdir d19/d1a/d27 0 2024-09-16T00:09:39.485 INFO:tasks.workunit.client.0.smithi139.stdout:4/43: creat fd x:0 0 0 2024-09-16T00:09:39.497 INFO:tasks.workunit.client.0.smithi139.stdout:5/130: dwrite d1/d13/f17 [0,4194304] 0 2024-09-16T00:09:39.497 INFO:tasks.workunit.client.0.smithi139.stdout:7/115: chown d5/f9 3 1 2024-09-16T00:09:39.498 INFO:tasks.workunit.client.0.smithi139.stdout:2/84: truncate f2 1004707 0 2024-09-16T00:09:39.498 INFO:tasks.workunit.client.0.smithi139.stdout:9/149: dread - d19/f1f zero size 2024-09-16T00:09:39.499 INFO:tasks.workunit.client.0.smithi139.stdout:0/155: truncate d6/dc/f31 197508 0 2024-09-16T00:09:39.501 INFO:tasks.workunit.client.0.smithi139.stdout:5/131: write d1/f28 [662834,112405] 0 2024-09-16T00:09:39.501 INFO:tasks.workunit.client.0.smithi139.stdout:9/150: readlink l18 0 2024-09-16T00:09:39.501 INFO:tasks.workunit.client.0.smithi139.stdout:2/85: truncate fd 1026802 0 2024-09-16T00:09:39.503 INFO:tasks.workunit.client.0.smithi139.stdout:0/156: write d6/dc/f33 [748215,32375] 0 2024-09-16T00:09:39.507 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:161 2.b e64/19 2:d5c7a900:::200.00000003:head v 64'5, mlcod=61'3)", 2024-09-16T00:09:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:07.541046+0000", 2024-09-16T00:09:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.964896723999999, 2024-09-16T00:09:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010081302, 2024-09-16T00:09:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.508 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.541046+0000", 2024-09-16T00:09:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.541046+0000", 2024-09-16T00:09:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.541048+0000", 2024-09-16T00:09:39.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5859999999999999e-06 2024-09-16T00:09:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543134+0000", 2024-09-16T00:09:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020863589999999999 2024-09-16T00:09:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543155+0000", 2024-09-16T00:09:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0344999999999999e-05 2024-09-16T00:09:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543167+0000", 2024-09-16T00:09:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2071e-05 2024-09-16T00:09:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543242+0000", 2024-09-16T00:09:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5108e-05 2024-09-16T00:09:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543268+0000", 2024-09-16T00:09:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6471000000000001e-05 2024-09-16T00:09:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551087+0000", 2024-09-16T00:09:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0078188670000000002 2024-09-16T00:09:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551128+0000", 2024-09-16T00:09:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0494999999999998e-05 2024-09-16T00:09:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:10645 3.a e82/46 3:500c83b2:::100000030b4.00000000:head v 82'172, mlcod=80'170)", 2024-09-16T00:09:39.512 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.611288+0000", 2024-09-16T00:09:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.8946551129999998, 2024-09-16T00:09:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070319170000000004, 2024-09-16T00:09:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.611288+0000", 2024-09-16T00:09:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.611288+0000", 2024-09-16T00:09:39.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.611289+0000", 2024-09-16T00:09:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3519999999999999e-06 2024-09-16T00:09:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.613725+0000", 2024-09-16T00:09:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024358829999999998 2024-09-16T00:09:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.613745+0000", 2024-09-16T00:09:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9746999999999999e-05 2024-09-16T00:09:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.613755+0000", 2024-09-16T00:09:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0438999999999999e-05 2024-09-16T00:09:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.613822+0000", 2024-09-16T00:09:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6869e-05 2024-09-16T00:09:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.613848+0000", 2024-09-16T00:09:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5911999999999999e-05 2024-09-16T00:09:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.618279+0000", 2024-09-16T00:09:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044302789999999996 2024-09-16T00:09:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.618320+0000", 2024-09-16T00:09:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1436000000000001e-05 2024-09-16T00:09:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10654 3.34 3:2c63a741:::100000030c9.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.652619+0000", 2024-09-16T00:09:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.8533239699999999, 2024-09-16T00:09:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012238643, 2024-09-16T00:09:39.519 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10654 2024-09-16T00:09:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.652619+0000", 2024-09-16T00:09:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.652619+0000", 2024-09-16T00:09:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.652620+0000", 2024-09-16T00:09:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.277e-06 2024-09-16T00:09:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.659597+0000", 2024-09-16T00:09:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069766510000000004 2024-09-16T00:09:39.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.659611+0000", 2024-09-16T00:09:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3722e-05 2024-09-16T00:09:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.659620+0000", 2024-09-16T00:09:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7669999999999992e-06 2024-09-16T00:09:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.659672+0000", 2024-09-16T00:09:39.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2590000000000003e-05 2024-09-16T00:09:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.659776+0000", 2024-09-16T00:09:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103643 2024-09-16T00:09:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.659861+0000", 2024-09-16T00:09:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5513999999999995e-05 2024-09-16T00:09:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:39.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.663941+0000", 2024-09-16T00:09:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004079938 2024-09-16T00:09:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.664758+0000", 2024-09-16T00:09:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00081642800000000001 2024-09-16T00:09:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.664813+0000", 2024-09-16T00:09:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4824999999999998e-05 2024-09-16T00:09:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.664858+0000", 2024-09-16T00:09:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5287999999999997e-05 2024-09-16T00:09:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:10657 3.a e82/46 3:500c83b2:::100000030b4.00000000:head v 82'173, mlcod=82'171)", 2024-09-16T00:09:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.797248+0000", 2024-09-16T00:09:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7086946909999998, 2024-09-16T00:09:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086005219999999993, 2024-09-16T00:09:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.526 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.797248+0000", 2024-09-16T00:09:39.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.797248+0000", 2024-09-16T00:09:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.797250+0000", 2024-09-16T00:09:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5799999999999999e-06 2024-09-16T00:09:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.799187+0000", 2024-09-16T00:09:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001937002 2024-09-16T00:09:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.799213+0000", 2024-09-16T00:09:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6281000000000001e-05 2024-09-16T00:09:39.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.799226+0000", 2024-09-16T00:09:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2526e-05 2024-09-16T00:09:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.799307+0000", 2024-09-16T00:09:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1018000000000003e-05 2024-09-16T00:09:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.799334+0000", 2024-09-16T00:09:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6821e-05 2024-09-16T00:09:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.805800+0000", 2024-09-16T00:09:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006466388 2024-09-16T00:09:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.805849+0000", 2024-09-16T00:09:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8906000000000002e-05 2024-09-16T00:09:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.531 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:10658 3.1c e82/20 3:38c4c24e:::100000030bb.00000000:head v 82'154, mlcod=82'152)", 2024-09-16T00:09:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.804957+0000", 2024-09-16T00:09:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7009863620000001, 2024-09-16T00:09:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074017450000000004, 2024-09-16T00:09:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.804957+0000", 2024-09-16T00:09:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.804957+0000", 2024-09-16T00:09:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.804959+0000", 2024-09-16T00:09:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.827e-06 2024-09-16T00:09:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.807580+0000", 2024-09-16T00:09:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002621033 2024-09-16T00:09:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.807599+0000", 2024-09-16T00:09:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9409999999999999e-05 2024-09-16T00:09:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.807609+0000", 2024-09-16T00:09:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0278999999999999e-05 2024-09-16T00:09:39.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.807674+0000", 2024-09-16T00:09:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4319999999999994e-05 2024-09-16T00:09:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.807699+0000", 2024-09-16T00:09:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5148999999999999e-05 2024-09-16T00:09:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812306+0000", 2024-09-16T00:09:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004607323 2024-09-16T00:09:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812359+0000", 2024-09-16T00:09:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2404000000000002e-05 2024-09-16T00:09:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.536 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10696 3.34 3:2c63a741:::100000030c9.00000000:head [write 0~4194304 [3@173033]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.951656+0000", 2024-09-16T00:09:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.5542869750000001, 2024-09-16T00:09:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011673619, 2024-09-16T00:09:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10696 2024-09-16T00:09:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.537 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.951656+0000", 2024-09-16T00:09:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.951656+0000", 2024-09-16T00:09:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.951658+0000", 2024-09-16T00:09:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.745e-06 2024-09-16T00:09:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.957371+0000", 2024-09-16T00:09:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00571347 2024-09-16T00:09:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.957391+0000", 2024-09-16T00:09:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9222999999999999e-05 2024-09-16T00:09:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.957401+0000", 2024-09-16T00:09:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0526999999999999e-05 2024-09-16T00:09:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.957477+0000", 2024-09-16T00:09:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6167000000000004e-05 2024-09-16T00:09:39.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.957565+0000", 2024-09-16T00:09:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7472999999999994e-05 2024-09-16T00:09:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.957702+0000", 2024-09-16T00:09:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000137315 2024-09-16T00:09:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.962248+0000", 2024-09-16T00:09:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045460869999999999 2024-09-16T00:09:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.963248+0000", 2024-09-16T00:09:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00099954699999999994 2024-09-16T00:09:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.963289+0000", 2024-09-16T00:09:39.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1743000000000003e-05 2024-09-16T00:09:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.963330+0000", 2024-09-16T00:09:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0321999999999998e-05 2024-09-16T00:09:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10756 3.a 3:500c83b2:::100000030b4.00000000:head [write 0~4194304 [3@2137521]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:39.543 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.339441+0000", 2024-09-16T00:09:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.1665019189999999, 2024-09-16T00:09:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01041241, 2024-09-16T00:09:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10756 2024-09-16T00:09:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.339441+0000", 2024-09-16T00:09:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.339441+0000", 2024-09-16T00:09:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.339442+0000", 2024-09-16T00:09:39.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.204e-06 2024-09-16T00:09:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.344325+0000", 2024-09-16T00:09:39.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048825099999999996 2024-09-16T00:09:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.344347+0000", 2024-09-16T00:09:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2007000000000001e-05 2024-09-16T00:09:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.344358+0000", 2024-09-16T00:09:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0964999999999999e-05 2024-09-16T00:09:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.344436+0000", 2024-09-16T00:09:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8461999999999995e-05 2024-09-16T00:09:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.344549+0000", 2024-09-16T00:09:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000112241 2024-09-16T00:09:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.349762+0000", 2024-09-16T00:09:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052130520000000001 2024-09-16T00:09:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.349807+0000", 2024-09-16T00:09:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4953999999999997e-05 2024-09-16T00:09:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.349854+0000", 2024-09-16T00:09:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7015000000000003e-05 2024-09-16T00:09:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:10767 3.1b e85/20 3:dab1607f:::100000030fd.00000000:head v 85'146, mlcod=80'144)", 2024-09-16T00:09:39.549 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.382223+0000", 2024-09-16T00:09:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.12372059, 2024-09-16T00:09:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088283600000000004, 2024-09-16T00:09:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.382223+0000", 2024-09-16T00:09:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.382223+0000", 2024-09-16T00:09:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.382224+0000", 2024-09-16T00:09:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.418e-06 2024-09-16T00:09:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.386079+0000", 2024-09-16T00:09:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038550020000000002 2024-09-16T00:09:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.386098+0000", 2024-09-16T00:09:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8762000000000001e-05 2024-09-16T00:09:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.386109+0000", 2024-09-16T00:09:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.093e-05 2024-09-16T00:09:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.386179+0000", 2024-09-16T00:09:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0310999999999998e-05 2024-09-16T00:09:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.386208+0000", 2024-09-16T00:09:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8807e-05 2024-09-16T00:09:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.391010+0000", 2024-09-16T00:09:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048027130000000001 2024-09-16T00:09:39.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.391051+0000", 2024-09-16T00:09:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0417000000000003e-05 2024-09-16T00:09:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:10772 3.1 e85/20 3:823e197f:::10000003110.00000000:head v 85'157, mlcod=85'155)", 2024-09-16T00:09:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.418482+0000", 2024-09-16T00:09:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.0874609079999999, 2024-09-16T00:09:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0076883020000000002, 2024-09-16T00:09:39.554 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.418482+0000", 2024-09-16T00:09:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.418482+0000", 2024-09-16T00:09:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.418483+0000", 2024-09-16T00:09:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.13e-06 2024-09-16T00:09:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.422413+0000", 2024-09-16T00:09:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039296720000000004 2024-09-16T00:09:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.422440+0000", 2024-09-16T00:09:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7356999999999999e-05 2024-09-16T00:09:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.422451+0000", 2024-09-16T00:09:39.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0753e-05 2024-09-16T00:09:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.422529+0000", 2024-09-16T00:09:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8040000000000005e-05 2024-09-16T00:09:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.422556+0000", 2024-09-16T00:09:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6794000000000001e-05 2024-09-16T00:09:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.426127+0000", 2024-09-16T00:09:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035713820000000001 2024-09-16T00:09:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.426171+0000", 2024-09-16T00:09:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3173999999999997e-05 2024-09-16T00:09:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.559 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10780 3.a 3:500c83b2:::100000030b4.00000000:head [write 0~4194304 [3@2137521]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:09:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.053028173, 2024-09-16T00:09:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015526478, 2024-09-16T00:09:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10780 2024-09-16T00:09:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:09:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:09:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452917+0000", 2024-09-16T00:09:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2180000000000002e-06 2024-09-16T00:09:39.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462177+0000", 2024-09-16T00:09:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092593769999999992 2024-09-16T00:09:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462196+0000", 2024-09-16T00:09:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9745000000000002e-05 2024-09-16T00:09:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462207+0000", 2024-09-16T00:09:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0524e-05 2024-09-16T00:09:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462284+0000", 2024-09-16T00:09:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7608000000000006e-05 2024-09-16T00:09:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462376+0000", 2024-09-16T00:09:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1811999999999994e-05 2024-09-16T00:09:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468357+0000", 2024-09-16T00:09:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059806809999999998 2024-09-16T00:09:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468386+0000", 2024-09-16T00:09:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9068999999999999e-05 2024-09-16T00:09:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468441+0000", 2024-09-16T00:09:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5444e-05 2024-09-16T00:09:39.564 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10809 3.34 3:2c63a741:::100000030c9.00000000:head [write 0~4194304 [4@1879734]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.689472+0000", 2024-09-16T00:09:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.81647097000000002, 2024-09-16T00:09:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099381250000000008, 2024-09-16T00:09:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.565 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10809 2024-09-16T00:09:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.689472+0000", 2024-09-16T00:09:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.689472+0000", 2024-09-16T00:09:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.689475+0000", 2024-09-16T00:09:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.858e-06 2024-09-16T00:09:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.694467+0000", 2024-09-16T00:09:39.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049919750000000001 2024-09-16T00:09:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.694487+0000", 2024-09-16T00:09:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9692000000000001e-05 2024-09-16T00:09:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.694497+0000", 2024-09-16T00:09:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.026e-05 2024-09-16T00:09:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.694554+0000", 2024-09-16T00:09:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6582999999999997e-05 2024-09-16T00:09:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.694640+0000", 2024-09-16T00:09:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6340000000000003e-05 2024-09-16T00:09:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.699258+0000", 2024-09-16T00:09:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046182319999999999 2024-09-16T00:09:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.699326+0000", 2024-09-16T00:09:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8087e-05 2024-09-16T00:09:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.699410+0000", 2024-09-16T00:09:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4097999999999993e-05 2024-09-16T00:09:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.570 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:10826 3.11 e85/20 3:889a8549:::10000003149.00000000:head v 85'195, mlcod=82'193)", 2024-09-16T00:09:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.747306+0000", 2024-09-16T00:09:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.75863701400000005, 2024-09-16T00:09:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0078519939999999993, 2024-09-16T00:09:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.747306+0000", 2024-09-16T00:09:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.747306+0000", 2024-09-16T00:09:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.747307+0000", 2024-09-16T00:09:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3510000000000001e-06 2024-09-16T00:09:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.748876+0000", 2024-09-16T00:09:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0015683349999999999 2024-09-16T00:09:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.748898+0000", 2024-09-16T00:09:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2042999999999999e-05 2024-09-16T00:09:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.748909+0000", 2024-09-16T00:09:39.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0995000000000001e-05 2024-09-16T00:09:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.748984+0000", 2024-09-16T00:09:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4721000000000005e-05 2024-09-16T00:09:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.749007+0000", 2024-09-16T00:09:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3561e-05 2024-09-16T00:09:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.755118+0000", 2024-09-16T00:09:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061108209999999998 2024-09-16T00:09:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.755158+0000", 2024-09-16T00:09:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0166999999999997e-05 2024-09-16T00:09:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10831 3.1a 3:59b8f98c:::100000030df.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:09:39.575 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.73022103699999996, 2024-09-16T00:09:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014692843000000001, 2024-09-16T00:09:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10831 2024-09-16T00:09:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:09:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:09:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775723+0000", 2024-09-16T00:09:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.367e-06 2024-09-16T00:09:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780402+0000", 2024-09-16T00:09:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046785769999999997 2024-09-16T00:09:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780421+0000", 2024-09-16T00:09:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9179000000000001e-05 2024-09-16T00:09:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780440+0000", 2024-09-16T00:09:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9015e-05 2024-09-16T00:09:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780533+0000", 2024-09-16T00:09:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3197999999999998e-05 2024-09-16T00:09:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780618+0000", 2024-09-16T00:09:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4787e-05 2024-09-16T00:09:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780748+0000", 2024-09-16T00:09:39.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012927199999999999 2024-09-16T00:09:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.785049+0000", 2024-09-16T00:09:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043015750000000002 2024-09-16T00:09:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790312+0000", 2024-09-16T00:09:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052633319999999999 2024-09-16T00:09:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790369+0000", 2024-09-16T00:09:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6848999999999997e-05 2024-09-16T00:09:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790415+0000", 2024-09-16T00:09:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5692000000000002e-05 2024-09-16T00:09:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:10848 3.24 e85/20 3:262efa15:::1000000311f.00000000:head v 85'162, mlcod=85'160)", 2024-09-16T00:09:39.581 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.875272+0000", 2024-09-16T00:09:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.63067149600000005, 2024-09-16T00:09:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0075374509999999997, 2024-09-16T00:09:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.875272+0000", 2024-09-16T00:09:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.875272+0000", 2024-09-16T00:09:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.875273+0000", 2024-09-16T00:09:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.153e-06 2024-09-16T00:09:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.879152+0000", 2024-09-16T00:09:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038789229999999998 2024-09-16T00:09:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.879168+0000", 2024-09-16T00:09:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6342999999999999e-05 2024-09-16T00:09:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.879177+0000", 2024-09-16T00:09:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3959999999999995e-06 2024-09-16T00:09:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.879253+0000", 2024-09-16T00:09:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5844000000000001e-05 2024-09-16T00:09:39.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.879279+0000", 2024-09-16T00:09:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6029000000000001e-05 2024-09-16T00:09:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.882763+0000", 2024-09-16T00:09:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034839810000000001 2024-09-16T00:09:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.882809+0000", 2024-09-16T00:09:39.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5782000000000003e-05 2024-09-16T00:09:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:10851 3.29 e85/20 3:9786b8f5:::10000003100.00000000:head v 85'167, mlcod=80'165)", 2024-09-16T00:09:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.894948+0000", 2024-09-16T00:09:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.610994649, 2024-09-16T00:09:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0096943959999999992, 2024-09-16T00:09:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.894948+0000", 2024-09-16T00:09:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.894948+0000", 2024-09-16T00:09:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.894949+0000", 2024-09-16T00:09:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7400000000000002e-07 2024-09-16T00:09:39.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.898908+0000", 2024-09-16T00:09:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00395898 2024-09-16T00:09:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.898927+0000", 2024-09-16T00:09:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8192000000000001e-05 2024-09-16T00:09:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.898936+0000", 2024-09-16T00:09:39.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1130000000000007e-06 2024-09-16T00:09:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.899008+0000", 2024-09-16T00:09:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2170999999999997e-05 2024-09-16T00:09:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.899025+0000", 2024-09-16T00:09:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7520000000000002e-05 2024-09-16T00:09:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.904599+0000", 2024-09-16T00:09:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055733830000000003 2024-09-16T00:09:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.904643+0000", 2024-09-16T00:09:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4162999999999997e-05 2024-09-16T00:09:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:10909 3.1 e85/20 3:8396d031:::100000031ab.00000000:head v 85'158, mlcod=85'156)", 2024-09-16T00:09:39.590 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.186086+0000", 2024-09-16T00:09:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.31985667600000001, 2024-09-16T00:09:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0075187409999999998, 2024-09-16T00:09:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.186086+0000", 2024-09-16T00:09:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.186086+0000", 2024-09-16T00:09:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.186088+0000", 2024-09-16T00:09:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1069999999999999e-06 2024-09-16T00:09:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.190029+0000", 2024-09-16T00:09:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039409320000000003 2024-09-16T00:09:39.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.190050+0000", 2024-09-16T00:09:39.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1418999999999999e-05 2024-09-16T00:09:39.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.190060+0000", 2024-09-16T00:09:39.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6909999999999992e-06 2024-09-16T00:09:39.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.190120+0000", 2024-09-16T00:09:39.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0238999999999997e-05 2024-09-16T00:09:39.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.190140+0000", 2024-09-16T00:09:39.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0548999999999999e-05 2024-09-16T00:09:39.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.193539+0000", 2024-09-16T00:09:39.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003398463 2024-09-16T00:09:39.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.193605+0000", 2024-09-16T00:09:39.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6340999999999994e-05 2024-09-16T00:09:39.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.596 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.596 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10915 3.1a 3:5a6e15a2:::1000000318e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:39.596 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:09:39.596 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.290715116, 2024-09-16T00:09:39.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014599472, 2024-09-16T00:09:39.596 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.596 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.596 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:39.597 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:39.597 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.597 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10915 2024-09-16T00:09:39.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.597 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:09:39.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:09:39.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215230+0000", 2024-09-16T00:09:39.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0090000000000002e-06 2024-09-16T00:09:39.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219556+0000", 2024-09-16T00:09:39.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043259539999999999 2024-09-16T00:09:39.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219577+0000", 2024-09-16T00:09:39.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1271999999999999e-05 2024-09-16T00:09:39.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219589+0000", 2024-09-16T00:09:39.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1766000000000001e-05 2024-09-16T00:09:39.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219663+0000", 2024-09-16T00:09:39.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4196999999999995e-05 2024-09-16T00:09:39.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219733+0000", 2024-09-16T00:09:39.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9553000000000003e-05 2024-09-16T00:09:39.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:39.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219827+0000", 2024-09-16T00:09:39.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4395999999999995e-05 2024-09-16T00:09:39.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:39.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.225724+0000", 2024-09-16T00:09:39.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058968359999999999 2024-09-16T00:09:39.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:39.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229705+0000", 2024-09-16T00:09:39.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003981409 2024-09-16T00:09:39.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229772+0000", 2024-09-16T00:09:39.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6137999999999996e-05 2024-09-16T00:09:39.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229828+0000", 2024-09-16T00:09:39.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5942000000000001e-05 2024-09-16T00:09:39.602 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.602 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.602 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.602 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10926 3.a 3:506aad5d:::1000000310f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:39.602 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.372524+0000", 2024-09-16T00:09:39.602 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.13341911000000001, 2024-09-16T00:09:39.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081466779999999992, 2024-09-16T00:09:39.602 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.602 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.603 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.603 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:39.603 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:39.603 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.603 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10926 2024-09-16T00:09:39.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.603 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.372524+0000", 2024-09-16T00:09:39.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.372524+0000", 2024-09-16T00:09:39.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.372526+0000", 2024-09-16T00:09:39.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.255e-06 2024-09-16T00:09:39.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.376378+0000", 2024-09-16T00:09:39.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003851438 2024-09-16T00:09:39.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.376396+0000", 2024-09-16T00:09:39.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8748000000000001e-05 2024-09-16T00:09:39.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.376408+0000", 2024-09-16T00:09:39.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1173e-05 2024-09-16T00:09:39.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.376511+0000", 2024-09-16T00:09:39.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103303 2024-09-16T00:09:39.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.376580+0000", 2024-09-16T00:09:39.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8775000000000005e-05 2024-09-16T00:09:39.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:39.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.380518+0000", 2024-09-16T00:09:39.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039383109999999999 2024-09-16T00:09:39.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.380580+0000", 2024-09-16T00:09:39.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2193000000000006e-05 2024-09-16T00:09:39.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.380671+0000", 2024-09-16T00:09:39.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0482000000000005e-05 2024-09-16T00:09:39.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.607 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.607 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.607 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10928 3.a 3:506aad5d:::1000000310f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:39.608 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.406639+0000", 2024-09-16T00:09:39.608 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.099304348000000001, 2024-09-16T00:09:39.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01087424, 2024-09-16T00:09:39.608 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.608 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.608 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.608 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:39.608 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:39.608 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:39.608 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10928 2024-09-16T00:09:39.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.608 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.406639+0000", 2024-09-16T00:09:39.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.406639+0000", 2024-09-16T00:09:39.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.406640+0000", 2024-09-16T00:09:39.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.697e-06 2024-09-16T00:09:39.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.410903+0000", 2024-09-16T00:09:39.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042625049999999998 2024-09-16T00:09:39.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.410928+0000", 2024-09-16T00:09:39.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4669999999999999e-05 2024-09-16T00:09:39.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.410936+0000", 2024-09-16T00:09:39.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1030000000000002e-06 2024-09-16T00:09:39.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.410999+0000", 2024-09-16T00:09:39.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2911999999999997e-05 2024-09-16T00:09:39.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.411070+0000", 2024-09-16T00:09:39.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0996999999999998e-05 2024-09-16T00:09:39.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:39.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.417378+0000", 2024-09-16T00:09:39.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063086349999999999 2024-09-16T00:09:39.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.417436+0000", 2024-09-16T00:09:39.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7946999999999999e-05 2024-09-16T00:09:39.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.417513+0000", 2024-09-16T00:09:39.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6773999999999994e-05 2024-09-16T00:09:39.613 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.613 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.613 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.613 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:10938 3.27 e86/20 3:e5179c63:::10000003176.00000000:head v 86'183, mlcod=80'181)", 2024-09-16T00:09:39.613 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.434533+0000", 2024-09-16T00:09:39.613 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.071409981999999997, 2024-09-16T00:09:39.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072902460000000002, 2024-09-16T00:09:39.613 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:39.613 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:39.613 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:39.614 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:39.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:39.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.434533+0000", 2024-09-16T00:09:39.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:39.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.434533+0000", 2024-09-16T00:09:39.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:39.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:39.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.434534+0000", 2024-09-16T00:09:39.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.201e-06 2024-09-16T00:09:39.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:39.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.438413+0000", 2024-09-16T00:09:39.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038787549999999998 2024-09-16T00:09:39.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:39.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.438442+0000", 2024-09-16T00:09:39.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8932999999999999e-05 2024-09-16T00:09:39.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:39.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.438452+0000", 2024-09-16T00:09:39.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8490000000000001e-06 2024-09-16T00:09:39.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:39.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.438510+0000", 2024-09-16T00:09:39.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8561999999999999e-05 2024-09-16T00:09:39.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:39.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.438536+0000", 2024-09-16T00:09:39.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5316999999999998e-05 2024-09-16T00:09:39.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:39.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.441776+0000", 2024-09-16T00:09:39.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003240312 2024-09-16T00:09:39.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:39.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:39.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:39.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.441823+0000", 2024-09-16T00:09:39.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7317000000000001e-05 2024-09-16T00:09:39.618 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.618 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.618 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.618 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:39.618 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:39.618 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:39.621 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:09:39.621 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:39.618+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:39.624 INFO:tasks.workunit.client.0.smithi139.stdout:3/161: dwrite d5/d7/d23/f11 [0,4194304] 0 2024-09-16T00:09:39.624 INFO:tasks.workunit.client.0.smithi139.stdout:3/162: write d5/f8 [812789,72712] 0 2024-09-16T00:09:39.624 INFO:tasks.workunit.client.0.smithi139.stdout:5/132: dwrite d1/f1a [4194304,4194304] 0 2024-09-16T00:09:39.624 INFO:tasks.workunit.client.0.smithi139.stdout:2/86: dwrite ff [0,4194304] 0 2024-09-16T00:09:39.624 INFO:tasks.workunit.client.0.smithi139.stdout:5/133: dwrite d1/f16 [0,4194304] 0 2024-09-16T00:09:39.624 INFO:tasks.workunit.client.0.smithi139.stdout:5/134: truncate d1/d13/f24 482065 0 2024-09-16T00:09:39.624 INFO:tasks.workunit.client.0.smithi139.stdout:4/44: dread f2 [0,4194304] 0 2024-09-16T00:09:39.624 INFO:tasks.workunit.client.0.smithi139.stdout:6/91: chown d8/d9/d10/c1d 910314 1 2024-09-16T00:09:39.624 INFO:tasks.workunit.client.0.smithi139.stdout:7/116: creat d5/f27 x:0 0 0 2024-09-16T00:09:39.624 INFO:tasks.workunit.client.0.smithi139.stdout:6/92: write d8/db/f1a [3116744,46291] 0 2024-09-16T00:09:39.624 INFO:tasks.workunit.client.0.smithi139.stdout:9/151: write d19/d22/f25 [445591,42388] 0 2024-09-16T00:09:39.624 INFO:tasks.workunit.client.0.smithi139.stdout:9/152: dread - d19/f26 zero size 2024-09-16T00:09:39.624 INFO:tasks.workunit.client.0.smithi139.stdout:9/153: dread - d19/f1b zero size 2024-09-16T00:09:39.625 INFO:tasks.workunit.client.0.smithi139.stdout:0/157: dread d6/f8 [0,4194304] 0 2024-09-16T00:09:39.625 INFO:tasks.workunit.client.0.smithi139.stdout:0/158: dread d6/d1c/f21 [0,4194304] 0 2024-09-16T00:09:39.625 INFO:tasks.workunit.client.0.smithi139.stdout:8/182: creat d17/d1f/d3a/f45 x:0 0 0 2024-09-16T00:09:39.625 INFO:tasks.workunit.client.0.smithi139.stdout:0/159: write d6/dc/f31 [932757,97258] 0 2024-09-16T00:09:39.625 INFO:tasks.workunit.client.0.smithi139.stdout:4/45: rename fa to fe 0 2024-09-16T00:09:39.625 INFO:tasks.workunit.client.0.smithi139.stdout:3/163: mkdir d5/d7/d23/d27/d2f 0 2024-09-16T00:09:39.625 INFO:tasks.workunit.client.0.smithi139.stdout:9/154: mkdir d19/d1a/d20/d23/d2c 0 2024-09-16T00:09:39.625 INFO:tasks.workunit.client.0.smithi139.stdout:5/135: unlink d1/d13/l1c 0 2024-09-16T00:09:39.625 INFO:tasks.workunit.client.0.smithi139.stdout:0/160: dread f3 [0,4194304] 0 2024-09-16T00:09:39.625 INFO:tasks.workunit.client.0.smithi139.stdout:1/118: creat d7/d8/f1e x:0 0 0 2024-09-16T00:09:39.625 INFO:tasks.workunit.client.0.smithi139.stdout:9/155: chown d19/d1a/c28 5682108 1 2024-09-16T00:09:39.625 INFO:tasks.workunit.client.0.smithi139.stdout:1/119: fdatasync f5 0 2024-09-16T00:09:39.626 INFO:tasks.workunit.client.0.smithi139.stdout:4/46: mknod cf 0 2024-09-16T00:09:39.626 INFO:tasks.workunit.client.0.smithi139.stdout:8/183: dwrite d17/d1f/f25 [0,4194304] 0 2024-09-16T00:09:39.626 INFO:tasks.workunit.client.0.smithi139.stdout:6/93: symlink d8/d20/l24 0 2024-09-16T00:09:39.626 INFO:tasks.workunit.client.0.smithi139.stdout:7/117: rename d5/d16/c1e to d5/d16/c28 0 2024-09-16T00:09:39.626 INFO:tasks.workunit.client.0.smithi139.stdout:5/136: dwrite d1/f28 [0,4194304] 0 2024-09-16T00:09:39.626 INFO:tasks.workunit.client.0.smithi139.stdout:4/47: write fe [121458,123289] 0 2024-09-16T00:09:39.626 INFO:tasks.workunit.client.0.smithi139.stdout:4/48: rmdir - no directory 2024-09-16T00:09:39.626 INFO:tasks.workunit.client.0.smithi139.stdout:2/87: getdents d11 0 2024-09-16T00:09:39.626 INFO:tasks.workunit.client.0.smithi139.stdout:5/137: fsync d1/f16 0 2024-09-16T00:09:39.626 INFO:tasks.workunit.client.0.smithi139.stdout:3/164: mkdir d5/d7/d23/d27/d2f/d30 0 2024-09-16T00:09:39.626 INFO:tasks.workunit.client.0.smithi139.stdout:8/184: write fd [6080452,53033] 0 2024-09-16T00:09:39.626 INFO:tasks.workunit.client.0.smithi139.stdout:0/161: dwrite d6/dc/dd/f2d [0,4194304] 0 2024-09-16T00:09:39.626 INFO:tasks.workunit.client.0.smithi139.stdout:1/120: mkdir d7/d8/d1f 0 2024-09-16T00:09:39.627 INFO:tasks.workunit.client.0.smithi139.stdout:7/118: dwrite d5/f27 [0,4194304] 0 2024-09-16T00:09:39.632 INFO:tasks.workunit.client.0.smithi139.stdout:0/162: creat d6/dc/dd/f34 x:0 0 0 2024-09-16T00:09:39.633 INFO:tasks.workunit.client.0.smithi139.stdout:8/185: rmdir d17/d1a/d43/d40 39 2024-09-16T00:09:39.635 INFO:tasks.workunit.client.0.smithi139.stdout:5/138: dwrite d1/f27 [0,4194304] 0 2024-09-16T00:09:39.635 INFO:tasks.workunit.client.0.smithi139.stdout:4/49: dwrite f6 [0,4194304] 0 2024-09-16T00:09:39.636 INFO:tasks.workunit.client.0.smithi139.stdout:1/121: readlink d7/d8/l17 0 2024-09-16T00:09:39.639 INFO:tasks.workunit.client.0.smithi139.stdout:2/88: dwrite d11/f15 [0,4194304] 0 2024-09-16T00:09:39.639 INFO:tasks.workunit.client.0.smithi139.stdout:5/139: write d1/f1a [4076815,139] 0 2024-09-16T00:09:39.641 INFO:tasks.workunit.client.0.smithi139.stdout:1/122: truncate d7/d8/f1c 838479 0 2024-09-16T00:09:39.643 INFO:tasks.workunit.client.0.smithi139.stdout:0/163: write f3 [1011874,10448] 0 2024-09-16T00:09:39.644 INFO:tasks.workunit.client.0.smithi139.stdout:2/89: symlink d11/l17 0 2024-09-16T00:09:39.645 INFO:tasks.workunit.client.0.smithi139.stdout:4/50: dread f6 [0,4194304] 0 2024-09-16T00:09:39.646 INFO:tasks.workunit.client.0.smithi139.stdout:5/140: mknod d1/d13/d1e/c2b 0 2024-09-16T00:09:39.646 INFO:tasks.workunit.client.0.smithi139.stdout:1/123: dread - d7/d8/f15 zero size 2024-09-16T00:09:39.647 INFO:tasks.workunit.client.0.smithi139.stdout:0/164: creat d6/d1c/f35 x:0 0 0 2024-09-16T00:09:39.664 INFO:tasks.workunit.client.0.smithi139.stdout:4/51: write f5 [333394,50851] 0 2024-09-16T00:09:39.665 INFO:tasks.workunit.client.0.smithi139.stdout:1/124: dread - d7/f16 zero size 2024-09-16T00:09:39.669 INFO:tasks.workunit.client.0.smithi139.stdout:2/90: link f2 d11/f18 0 2024-09-16T00:09:39.670 INFO:tasks.workunit.client.0.smithi139.stdout:0/165: dread d6/dc/dd/f24 [0,4194304] 0 2024-09-16T00:09:39.673 INFO:tasks.workunit.client.0.smithi139.stdout:2/91: creat d11/f19 x:0 0 0 2024-09-16T00:09:39.675 INFO:tasks.workunit.client.0.smithi139.stdout:2/92: write f2 [134780,92374] 0 2024-09-16T00:09:39.678 INFO:tasks.workunit.client.0.smithi139.stdout:2/93: creat d11/f1a x:0 0 0 2024-09-16T00:09:39.678 INFO:tasks.workunit.client.0.smithi139.stdout:0/166: link d6/dc/f25 d6/d1c/f36 0 2024-09-16T00:09:39.680 INFO:tasks.workunit.client.0.smithi139.stdout:7/119: getdents d5 0 2024-09-16T00:09:39.680 INFO:tasks.workunit.client.0.smithi139.stdout:2/94: mknod d11/c1b 0 2024-09-16T00:09:39.681 INFO:tasks.workunit.client.0.smithi139.stdout:0/167: rmdir d6 39 2024-09-16T00:09:39.682 INFO:tasks.workunit.client.0.smithi139.stdout:2/95: mknod d11/c1c 0 2024-09-16T00:09:39.682 INFO:tasks.workunit.client.0.smithi139.stdout:4/52: dread fc [0,4194304] 0 2024-09-16T00:09:39.683 INFO:tasks.workunit.client.0.smithi139.stdout:2/96: write ff [5064419,127669] 0 2024-09-16T00:09:39.686 INFO:tasks.workunit.client.0.smithi139.stdout:7/120: symlink d5/d16/d22/d25/l29 0 2024-09-16T00:09:39.687 INFO:tasks.workunit.client.0.smithi139.stdout:0/168: fdatasync d6/dc/dd/f24 0 2024-09-16T00:09:39.688 INFO:tasks.workunit.client.0.smithi139.stdout:4/53: dwrite fe [0,4194304] 0 2024-09-16T00:09:39.690 INFO:tasks.workunit.client.0.smithi139.stdout:2/97: symlink d11/l1d 0 2024-09-16T00:09:39.694 INFO:tasks.workunit.client.0.smithi139.stdout:2/98: write d11/f1a [162555,37974] 0 2024-09-16T00:09:39.695 INFO:tasks.workunit.client.0.smithi139.stdout:7/121: dread f1 [0,4194304] 0 2024-09-16T00:09:39.698 INFO:tasks.workunit.client.0.smithi139.stdout:7/122: rename d5 to d5/d26/d2a 22 2024-09-16T00:09:39.700 INFO:tasks.workunit.client.0.smithi139.stdout:4/54: dwrite f3 [0,4194304] 0 2024-09-16T00:09:39.704 INFO:tasks.workunit.client.0.smithi139.stdout:0/169: dwrite d6/dc/dd/f11 [0,4194304] 0 2024-09-16T00:09:39.704 INFO:tasks.workunit.client.0.smithi139.stdout:7/123: truncate d5/f14 1521960 0 2024-09-16T00:09:39.707 INFO:tasks.workunit.client.0.smithi139.stdout:7/124: readlink d5/d16/l19 0 2024-09-16T00:09:39.709 INFO:tasks.workunit.client.0.smithi139.stdout:5/141: truncate d1/f27 1795901 0 2024-09-16T00:09:39.709 INFO:tasks.workunit.client.0.smithi139.stdout:9/156: dwrite f10 [0,4194304] 0 2024-09-16T00:09:39.715 INFO:tasks.workunit.client.0.smithi139.stdout:2/99: dwrite ff [4194304,4194304] 0 2024-09-16T00:09:39.719 INFO:tasks.workunit.client.0.smithi139.stdout:5/142: mkdir d1/d21/d2c 0 2024-09-16T00:09:39.719 INFO:tasks.workunit.client.0.smithi139.stdout:4/55: dread f3 [0,4194304] 0 2024-09-16T00:09:39.720 INFO:tasks.workunit.client.0.smithi139.stdout:7/125: creat d5/d26/f2b x:0 0 0 2024-09-16T00:09:39.720 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:39.719+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:39.722 INFO:tasks.workunit.client.0.smithi139.stdout:1/125: dwrite d7/f18 [4194304,4194304] 0 2024-09-16T00:09:39.722 INFO:tasks.workunit.client.0.smithi139.stdout:2/100: rename fa to d11/f1e 0 2024-09-16T00:09:39.725 INFO:tasks.workunit.client.0.smithi139.stdout:0/170: getdents d6/d1c 0 2024-09-16T00:09:39.725 INFO:tasks.workunit.client.0.smithi139.stdout:1/126: write d7/d8/f1c [1689255,108950] 0 2024-09-16T00:09:39.729 INFO:tasks.workunit.client.0.smithi139.stdout:1/127: chown f5 1 1 2024-09-16T00:09:39.729 INFO:tasks.workunit.client.0.smithi139.stdout:9/157: dwrite d19/d1a/f21 [0,4194304] 0 2024-09-16T00:09:39.733 INFO:tasks.workunit.client.0.smithi139.stdout:0/171: write d6/dc/dd/f2d [3241916,3321] 0 2024-09-16T00:09:39.733 INFO:tasks.workunit.client.0.smithi139.stdout:4/56: link cb c10 0 2024-09-16T00:09:39.733 INFO:tasks.workunit.client.0.smithi139.stdout:9/158: dread - d19/f26 zero size 2024-09-16T00:09:39.733 INFO:tasks.workunit.client.0.smithi139.stdout:2/101: dread fe [0,4194304] 0 2024-09-16T00:09:39.734 INFO:tasks.workunit.client.0.smithi139.stdout:9/159: fdatasync d19/d1a/d20/d23/f24 0 2024-09-16T00:09:39.734 INFO:tasks.workunit.client.0.smithi139.stdout:4/57: mknod c11 0 2024-09-16T00:09:39.735 INFO:tasks.workunit.client.0.smithi139.stdout:2/102: write d11/f1a [630310,89239] 0 2024-09-16T00:09:39.736 INFO:tasks.workunit.client.0.smithi139.stdout:0/172: readlink d6/dc/dd/l27 0 2024-09-16T00:09:39.736 INFO:tasks.workunit.client.0.smithi139.stdout:7/126: link d5/f12 d5/d16/d22/d25/f2c 0 2024-09-16T00:09:39.737 INFO:tasks.workunit.client.0.smithi139.stdout:2/103: chown d11/f18 0 1 2024-09-16T00:09:39.738 INFO:tasks.workunit.client.0.smithi139.stdout:0/173: write d6/d1c/f35 [233611,2476] 0 2024-09-16T00:09:39.739 INFO:tasks.workunit.client.0.smithi139.stdout:0/174: stat d6/d1c/f35 0 2024-09-16T00:09:39.740 INFO:tasks.workunit.client.0.smithi139.stdout:4/58: mkdir d12 0 2024-09-16T00:09:39.740 INFO:tasks.workunit.client.0.smithi139.stdout:4/59: readlink - no filename 2024-09-16T00:09:39.741 INFO:tasks.workunit.client.0.smithi139.stdout:0/175: write f2 [8516317,79079] 0 2024-09-16T00:09:39.742 INFO:tasks.workunit.client.0.smithi139.stdout:4/60: write f5 [458671,85297] 0 2024-09-16T00:09:39.743 INFO:tasks.workunit.client.0.smithi139.stdout:2/104: mkdir d11/d1f 0 2024-09-16T00:09:39.743 INFO:tasks.workunit.client.0.smithi139.stdout:5/143: dread d1/fa [0,4194304] 0 2024-09-16T00:09:39.744 INFO:tasks.workunit.client.0.smithi139.stdout:9/160: dwrite d19/f1d [0,4194304] 0 2024-09-16T00:09:39.745 INFO:tasks.workunit.client.0.smithi139.stdout:5/144: write d1/d13/f24 [1500882,75252] 0 2024-09-16T00:09:39.747 INFO:tasks.workunit.client.0.smithi139.stdout:0/176: chown d6/dc/f29 808814930 1 2024-09-16T00:09:39.747 INFO:tasks.workunit.client.0.smithi139.stdout:1/128: truncate d7/ff 11872 0 2024-09-16T00:09:39.749 INFO:tasks.workunit.client.0.smithi139.stdout:2/105: symlink d11/l20 0 2024-09-16T00:09:39.752 INFO:tasks.workunit.client.0.smithi139.stdout:4/61: mknod d12/c13 0 2024-09-16T00:09:39.756 INFO:tasks.workunit.client.0.smithi139.stdout:5/145: dwrite d1/f1a [0,4194304] 0 2024-09-16T00:09:39.756 INFO:tasks.workunit.client.0.smithi139.stdout:1/129: link d7/fc d7/d8/d1f/f20 0 2024-09-16T00:09:39.762 INFO:tasks.workunit.client.0.smithi139.stdout:5/146: fdatasync d1/fa 0 2024-09-16T00:09:39.763 INFO:tasks.workunit.client.0.smithi139.stdout:0/177: dwrite d6/dc/dd/f24 [0,4194304] 0 2024-09-16T00:09:39.768 INFO:tasks.workunit.client.0.smithi139.stdout:2/106: dwrite d11/f18 [0,4194304] 0 2024-09-16T00:09:39.771 INFO:tasks.workunit.client.0.smithi139.stdout:7/127: dwrite d5/f12 [0,4194304] 0 2024-09-16T00:09:39.771 INFO:tasks.workunit.client.0.smithi139.stdout:1/130: creat d7/d8/f21 x:0 0 0 2024-09-16T00:09:39.772 INFO:tasks.workunit.client.0.smithi139.stdout:4/62: dread f6 [0,4194304] 0 2024-09-16T00:09:39.775 INFO:tasks.workunit.client.0.smithi139.stdout:1/131: rename d7 to d7/d22 22 2024-09-16T00:09:39.777 INFO:tasks.workunit.client.0.smithi139.stdout:7/128: mkdir d5/d16/d22/d25/d2d 0 2024-09-16T00:09:39.778 INFO:tasks.workunit.client.0.smithi139.stdout:5/147: dread d1/f1a [4194304,4194304] 0 2024-09-16T00:09:39.779 INFO:tasks.workunit.client.0.smithi139.stdout:0/178: dread d6/dc/dd/f19 [0,4194304] 0 2024-09-16T00:09:39.780 INFO:tasks.workunit.client.0.smithi139.stdout:1/132: symlink d7/d8/l23 0 2024-09-16T00:09:39.781 INFO:tasks.workunit.client.0.smithi139.stdout:4/63: rename c10 to d12/c14 0 2024-09-16T00:09:39.782 INFO:tasks.workunit.client.0.smithi139.stdout:2/107: dwrite ff [0,4194304] 0 2024-09-16T00:09:39.783 INFO:tasks.workunit.client.0.smithi139.stdout:4/64: fsync f6 0 2024-09-16T00:09:39.787 INFO:tasks.workunit.client.0.smithi139.stdout:2/108: creat d11/d1f/f21 x:0 0 0 2024-09-16T00:09:39.787 INFO:tasks.workunit.client.0.smithi139.stdout:5/148: dread d1/f27 [0,4194304] 0 2024-09-16T00:09:39.788 INFO:tasks.workunit.client.0.smithi139.stdout:1/133: rename d7/d8/fb to d7/d8/d1f/f24 0 2024-09-16T00:09:39.789 INFO:tasks.workunit.client.0.smithi139.stdout:7/129: dread d5/fb [4194304,4194304] 0 2024-09-16T00:09:39.789 INFO:tasks.workunit.client.0.smithi139.stdout:2/109: unlink fd 0 2024-09-16T00:09:39.790 INFO:tasks.workunit.client.0.smithi139.stdout:5/149: chown d1/d13/f24 403140960 1 2024-09-16T00:09:39.791 INFO:tasks.workunit.client.0.smithi139.stdout:0/179: dread d6/f8 [0,4194304] 0 2024-09-16T00:09:39.791 INFO:tasks.workunit.client.0.smithi139.stdout:2/110: dread - d11/f19 zero size 2024-09-16T00:09:39.791 INFO:tasks.workunit.client.0.smithi139.stdout:7/130: creat d5/f2e x:0 0 0 2024-09-16T00:09:39.793 INFO:tasks.workunit.client.0.smithi139.stdout:7/131: chown d5/l11 11 1 2024-09-16T00:09:39.793 INFO:tasks.workunit.client.0.smithi139.stdout:0/180: truncate d6/dc/f29 132486 0 2024-09-16T00:09:39.794 INFO:tasks.workunit.client.0.smithi139.stdout:1/134: rename f5 to d7/f25 0 2024-09-16T00:09:39.794 INFO:tasks.workunit.client.0.smithi139.stdout:4/65: dwrite f6 [0,4194304] 0 2024-09-16T00:09:39.796 INFO:tasks.workunit.client.0.smithi139.stdout:5/150: fdatasync d1/f27 0 2024-09-16T00:09:39.796 INFO:tasks.workunit.client.0.smithi139.stdout:0/181: read - d6/dc/dd/f34 zero size 2024-09-16T00:09:39.796 INFO:tasks.workunit.client.0.smithi139.stdout:5/151: chown d1/f16 4877 1 2024-09-16T00:09:39.796 INFO:tasks.workunit.client.0.smithi139.stdout:0/182: readlink d6/l2a 0 2024-09-16T00:09:39.797 INFO:tasks.workunit.client.0.smithi139.stdout:0/183: chown d6/dc/f33 3586 1 2024-09-16T00:09:39.799 INFO:tasks.workunit.client.0.smithi139.stdout:4/66: mkdir d12/d15 0 2024-09-16T00:09:39.804 INFO:tasks.workunit.client.0.smithi139.stdout:0/184: mknod d6/dc/dd/c37 0 2024-09-16T00:09:39.804 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:39.804 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:39.804 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:39.804 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:39.805 INFO:tasks.workunit.client.0.smithi139.stdout:2/111: rename d11/d1f/f21 to d11/f22 0 2024-09-16T00:09:39.805 INFO:tasks.workunit.client.0.smithi139.stdout:1/135: write d7/f13 [848330,69393] 0 2024-09-16T00:09:39.806 INFO:tasks.workunit.client.0.smithi139.stdout:0/185: fdatasync d6/f8 0 2024-09-16T00:09:39.808 INFO:tasks.workunit.client.0.smithi139.stdout:7/132: rename d5/c1a to d5/d16/d22/d25/d2d/c2f 0 2024-09-16T00:09:39.809 INFO:tasks.workunit.client.0.smithi139.stdout:5/152: getdents d1/d25 0 2024-09-16T00:09:39.811 INFO:tasks.workunit.client.0.smithi139.stdout:7/133: dread d5/fb [0,4194304] 0 2024-09-16T00:09:39.812 INFO:tasks.workunit.client.0.smithi139.stdout:4/67: rename c8 to d12/d15/c16 0 2024-09-16T00:09:39.813 INFO:tasks.workunit.client.0.smithi139.stdout:5/153: creat d1/d25/f2d x:0 0 0 2024-09-16T00:09:39.813 INFO:tasks.workunit.client.0.smithi139.stdout:4/68: truncate f5 1520912 0 2024-09-16T00:09:39.813 INFO:tasks.workunit.client.0.smithi139.stdout:1/136: truncate d7/d8/d1f/f20 281221 0 2024-09-16T00:09:39.814 INFO:tasks.workunit.client.0.smithi139.stdout:0/186: rename d6/dc/dd/c1b to d6/d1c/c38 0 2024-09-16T00:09:39.814 INFO:tasks.workunit.client.0.smithi139.stdout:2/112: link c10 d11/c23 0 2024-09-16T00:09:39.816 INFO:tasks.workunit.client.0.smithi139.stdout:1/137: write d7/f1d [3361398,61926] 0 2024-09-16T00:09:39.817 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:09:39.817 INFO:tasks.workunit.client.0.smithi139.stdout:0/187: creat d6/dc/dd/f39 x:0 0 0 2024-09-16T00:09:39.818 INFO:tasks.workunit.client.0.smithi139.stdout:5/154: unlink d1/d13/c1b 0 2024-09-16T00:09:39.818 INFO:tasks.workunit.client.0.smithi139.stdout:2/113: dread d11/f1e [0,4194304] 0 2024-09-16T00:09:39.818 INFO:tasks.workunit.client.0.smithi139.stdout:5/155: read d1/f28 [1643916,15934] 0 2024-09-16T00:09:39.819 INFO:tasks.workunit.client.0.smithi139.stdout:7/134: link f4 d5/d16/d22/d25/f30 0 2024-09-16T00:09:39.819 INFO:tasks.workunit.client.0.smithi139.stdout:7/135: chown d5/fb 0 1 2024-09-16T00:09:39.821 INFO:tasks.workunit.client.0.smithi139.stdout:2/114: dread d11/f1a [0,4194304] 0 2024-09-16T00:09:39.821 INFO:tasks.workunit.client.0.smithi139.stdout:2/115: readlink d11/l1d 0 2024-09-16T00:09:39.824 INFO:tasks.workunit.client.0.smithi139.stdout:5/156: creat d1/d25/f2e x:0 0 0 2024-09-16T00:09:39.825 INFO:tasks.workunit.client.0.smithi139.stdout:5/157: chown d1/d13 166816091 1 2024-09-16T00:09:39.825 INFO:tasks.workunit.client.0.smithi139.stdout:4/69: write f2 [1122830,96331] 0 2024-09-16T00:09:39.830 INFO:tasks.workunit.client.0.smithi139.stdout:2/116: dwrite ff [4194304,4194304] 0 2024-09-16T00:09:39.830 INFO:tasks.workunit.client.0.smithi139.stdout:5/158: fsync d1/f28 0 2024-09-16T00:09:39.831 INFO:tasks.workunit.client.0.smithi139.stdout:5/159: stat d1/f1a 0 2024-09-16T00:09:39.832 INFO:tasks.workunit.client.0.smithi139.stdout:5/160: chown d1/d21/d2c 110 1 2024-09-16T00:09:39.832 INFO:tasks.workunit.client.0.smithi139.stdout:7/136: fsync d5/f14 0 2024-09-16T00:09:39.834 INFO:tasks.workunit.client.0.smithi139.stdout:6/94: sync 2024-09-16T00:09:39.835 INFO:tasks.workunit.client.0.smithi139.stdout:4/70: dwrite f7 [4194304,4194304] 0 2024-09-16T00:09:39.837 INFO:tasks.workunit.client.0.smithi139.stdout:2/117: dread d11/f1a [0,4194304] 0 2024-09-16T00:09:39.838 INFO:tasks.workunit.client.0.smithi139.stdout:0/188: dread d6/d1c/f36 [0,4194304] 0 2024-09-16T00:09:39.838 INFO:tasks.workunit.client.0.smithi139.stdout:6/95: mkdir d8/d25 0 2024-09-16T00:09:39.839 INFO:tasks.workunit.client.0.smithi139.stdout:7/137: creat d5/d16/d22/d25/f31 x:0 0 0 2024-09-16T00:09:39.840 INFO:tasks.workunit.client.0.smithi139.stdout:4/71: truncate f1 1999769 0 2024-09-16T00:09:39.841 INFO:tasks.workunit.client.0.smithi139.stdout:2/118: rmdir d11 39 2024-09-16T00:09:39.841 INFO:tasks.workunit.client.0.smithi139.stdout:6/96: mkdir d8/db/d26 0 2024-09-16T00:09:39.842 INFO:tasks.workunit.client.0.smithi139.stdout:0/189: chown d6/dc/dd/f34 469263 1 2024-09-16T00:09:39.842 INFO:tasks.workunit.client.0.smithi139.stdout:7/138: write d5/f1f [749366,52915] 0 2024-09-16T00:09:39.842 INFO:tasks.workunit.client.0.smithi139.stdout:4/72: truncate fd 1003942 0 2024-09-16T00:09:39.843 INFO:tasks.workunit.client.0.smithi139.stdout:5/161: dwrite d1/d13/f17 [0,4194304] 0 2024-09-16T00:09:39.845 INFO:tasks.workunit.client.0.smithi139.stdout:6/97: read d8/d9/f1c [1118982,101593] 0 2024-09-16T00:09:39.846 INFO:tasks.workunit.client.0.smithi139.stdout:2/119: creat d11/d1f/f24 x:0 0 0 2024-09-16T00:09:39.846 INFO:tasks.workunit.client.0.smithi139.stdout:2/120: stat d11/f13 0 2024-09-16T00:09:39.847 INFO:tasks.workunit.client.0.smithi139.stdout:7/139: creat d5/d26/f32 x:0 0 0 2024-09-16T00:09:39.849 INFO:tasks.workunit.client.0.smithi139.stdout:5/162: creat d1/f2f x:0 0 0 2024-09-16T00:09:39.849 INFO:tasks.workunit.client.0.smithi139.stdout:7/140: chown d5/l11 3034538 1 2024-09-16T00:09:39.851 INFO:tasks.workunit.client.0.smithi139.stdout:6/98: rename d8/d9 to d8/d20/d27 0 2024-09-16T00:09:39.852 INFO:tasks.workunit.client.0.smithi139.stdout:2/121: mknod d11/c25 0 2024-09-16T00:09:39.852 INFO:tasks.workunit.client.0.smithi139.stdout:6/99: stat d8/fe 0 2024-09-16T00:09:39.856 INFO:tasks.workunit.client.0.smithi139.stdout:5/163: rename d1/d13/d1e to d1/d25/d30 0 2024-09-16T00:09:39.856 INFO:tasks.workunit.client.0.smithi139.stdout:4/73: dread f1 [0,4194304] 0 2024-09-16T00:09:39.857 INFO:tasks.workunit.client.0.smithi139.stdout:0/190: link d6/dc/f29 d6/dc/f3a 0 2024-09-16T00:09:39.858 INFO:tasks.workunit.client.0.smithi139.stdout:7/141: dwrite d5/f21 [4194304,4194304] 0 2024-09-16T00:09:39.858 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:39.857+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:39.859 INFO:tasks.workunit.client.0.smithi139.stdout:4/74: chown f1 767 1 2024-09-16T00:09:39.861 INFO:tasks.workunit.client.0.smithi139.stdout:0/191: chown d6/dc/dd/f2c 0 1 2024-09-16T00:09:39.861 INFO:tasks.workunit.client.0.smithi139.stdout:7/142: write d5/f9 [820263,52541] 0 2024-09-16T00:09:39.862 INFO:tasks.workunit.client.0.smithi139.stdout:5/164: mknod d1/d21/d2c/c31 0 2024-09-16T00:09:39.863 INFO:tasks.workunit.client.0.smithi139.stdout:7/143: stat d5/fc 0 2024-09-16T00:09:39.865 INFO:tasks.workunit.client.0.smithi139.stdout:6/100: dwrite d8/d20/d27/d10/f11 [0,4194304] 0 2024-09-16T00:09:39.866 INFO:tasks.workunit.client.0.smithi139.stdout:5/165: mknod d1/d21/d2c/c32 0 2024-09-16T00:09:39.869 INFO:tasks.workunit.client.0.smithi139.stdout:7/144: unlink d5/f14 0 2024-09-16T00:09:39.869 INFO:tasks.workunit.client.0.smithi139.stdout:2/122: write d11/f22 [335679,2742] 0 2024-09-16T00:09:39.870 INFO:tasks.workunit.client.0.smithi139.stdout:6/101: unlink d8/d20/l24 0 2024-09-16T00:09:39.871 INFO:tasks.workunit.client.0.smithi139.stdout:5/166: mkdir d1/d25/d30/d33 0 2024-09-16T00:09:39.872 INFO:tasks.workunit.client.0.smithi139.stdout:4/75: dwrite fe [0,4194304] 0 2024-09-16T00:09:39.875 INFO:tasks.workunit.client.0.smithi139.stdout:7/145: unlink f1 0 2024-09-16T00:09:39.875 INFO:tasks.workunit.client.0.smithi139.stdout:6/102: dread d8/d20/d27/d10/f17 [0,4194304] 0 2024-09-16T00:09:39.876 INFO:tasks.workunit.client.0.smithi139.stdout:6/103: fdatasync d8/d20/d27/d10/f17 0 2024-09-16T00:09:39.878 INFO:tasks.workunit.client.0.smithi139.stdout:6/104: write d8/d20/d27/d10/f13 [7787173,26382] 0 2024-09-16T00:09:39.879 INFO:tasks.workunit.client.0.smithi139.stdout:4/76: rename f5 to d12/f17 0 2024-09-16T00:09:39.881 INFO:tasks.workunit.client.0.smithi139.stdout:7/146: mknod d5/d16/d22/d25/d2d/c33 0 2024-09-16T00:09:39.882 INFO:tasks.workunit.client.0.smithi139.stdout:0/192: dwrite d6/d1c/f28 [4194304,4194304] 0 2024-09-16T00:09:39.883 INFO:tasks.workunit.client.0.smithi139.stdout:0/193: truncate d6/dc/dd/f2c 445268 0 2024-09-16T00:09:39.886 INFO:tasks.workunit.client.0.smithi139.stdout:4/77: unlink f6 0 2024-09-16T00:09:39.888 INFO:tasks.workunit.client.0.smithi139.stdout:6/105: unlink d8/d20/d27/d10/c1d 0 2024-09-16T00:09:39.891 INFO:tasks.workunit.client.0.smithi139.stdout:7/147: creat d5/d16/f34 x:0 0 0 2024-09-16T00:09:39.891 INFO:tasks.workunit.client.0.smithi139.stdout:5/167: mknod d1/d13/d22/c34 0 2024-09-16T00:09:39.893 INFO:tasks.workunit.client.0.smithi139.stdout:7/148: write d5/f1c [312398,99329] 0 2024-09-16T00:09:39.895 INFO:tasks.workunit.client.0.smithi139.stdout:6/106: dwrite f6 [0,4194304] 0 2024-09-16T00:09:39.896 INFO:tasks.workunit.client.0.smithi139.stdout:6/107: chown l1 360153737 1 2024-09-16T00:09:39.897 INFO:tasks.workunit.client.0.smithi139.stdout:2/123: getdents d11/d1f 0 2024-09-16T00:09:39.897 INFO:tasks.workunit.client.0.smithi139.stdout:6/108: chown d8/d25 105 1 2024-09-16T00:09:39.900 INFO:tasks.workunit.client.0.smithi139.stdout:7/149: mknod d5/c35 0 2024-09-16T00:09:39.900 INFO:tasks.workunit.client.0.smithi139.stdout:5/168: dwrite d1/d13/f17 [0,4194304] 0 2024-09-16T00:09:39.903 INFO:tasks.workunit.client.0.smithi139.stdout:4/78: unlink cb 0 2024-09-16T00:09:39.905 INFO:tasks.workunit.client.0.smithi139.stdout:2/124: creat d11/f26 x:0 0 0 2024-09-16T00:09:39.908 INFO:tasks.workunit.client.0.smithi139.stdout:0/194: creat d6/dc/f3b x:0 0 0 2024-09-16T00:09:39.908 INFO:tasks.workunit.client.0.smithi139.stdout:5/169: dread d1/f27 [0,4194304] 0 2024-09-16T00:09:39.909 INFO:tasks.workunit.client.0.smithi139.stdout:2/125: chown d11/f1a 366849695 1 2024-09-16T00:09:39.911 INFO:tasks.workunit.client.0.smithi139.stdout:6/109: dwrite d8/d20/d27/f18 [0,4194304] 0 2024-09-16T00:09:39.914 INFO:tasks.workunit.client.0.smithi139.stdout:2/126: creat d11/d1f/f27 x:0 0 0 2024-09-16T00:09:39.914 INFO:tasks.workunit.client.0.smithi139.stdout:7/150: dwrite d5/f1c [0,4194304] 0 2024-09-16T00:09:39.916 INFO:tasks.workunit.client.0.smithi139.stdout:7/151: dread f4 [0,4194304] 0 2024-09-16T00:09:39.918 INFO:tasks.workunit.client.0.smithi139.stdout:5/170: truncate d1/fa 549556 0 2024-09-16T00:09:39.919 INFO:tasks.workunit.client.0.smithi139.stdout:4/79: dread f2 [0,4194304] 0 2024-09-16T00:09:39.919 INFO:tasks.workunit.client.0.smithi139.stdout:7/152: chown d5/d16/d22/d25/l29 923229 1 2024-09-16T00:09:39.920 INFO:tasks.workunit.client.0.smithi139.stdout:7/153: dread - d5/d26/f2b zero size 2024-09-16T00:09:39.921 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:39.920+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:39.921 INFO:tasks.workunit.client.0.smithi139.stdout:6/110: mkdir d8/db/d26/d28 0 2024-09-16T00:09:39.922 INFO:tasks.workunit.client.0.smithi139.stdout:2/127: dwrite ff [8388608,4194304] 0 2024-09-16T00:09:39.922 INFO:tasks.workunit.client.0.smithi139.stdout:5/171: symlink d1/l35 0 2024-09-16T00:09:39.922 INFO:tasks.workunit.client.0.smithi139.stdout:2/128: stat f2 0 2024-09-16T00:09:39.923 INFO:tasks.workunit.client.0.smithi139.stdout:2/129: dread - d11/f19 zero size 2024-09-16T00:09:39.924 INFO:tasks.workunit.client.0.smithi139.stdout:3/165: sync 2024-09-16T00:09:39.924 INFO:tasks.workunit.client.0.smithi139.stdout:8/186: sync 2024-09-16T00:09:39.925 INFO:tasks.workunit.client.0.smithi139.stdout:4/80: write f2 [410677,12325] 0 2024-09-16T00:09:39.926 INFO:tasks.workunit.client.0.smithi139.stdout:7/154: rmdir d5/d26 39 2024-09-16T00:09:39.929 INFO:tasks.workunit.client.0.smithi139.stdout:6/111: dread d8/d20/d27/f18 [0,4194304] 0 2024-09-16T00:09:39.931 INFO:tasks.workunit.client.0.smithi139.stdout:5/172: unlink d1/d13/c15 0 2024-09-16T00:09:39.932 INFO:tasks.workunit.client.0.smithi139.stdout:4/81: dwrite fd [0,4194304] 0 2024-09-16T00:09:39.933 INFO:tasks.workunit.client.0.smithi139.stdout:2/130: symlink d11/d1f/l28 0 2024-09-16T00:09:39.934 INFO:tasks.workunit.client.0.smithi139.stdout:0/195: link d6/l1f d6/dc/l3c 0 2024-09-16T00:09:39.936 INFO:tasks.workunit.client.0.smithi139.stdout:8/187: mknod d17/d1f/c46 0 2024-09-16T00:09:39.937 INFO:tasks.workunit.client.0.smithi139.stdout:0/196: chown d6/dc/dd/l27 37 1 2024-09-16T00:09:39.937 INFO:tasks.workunit.client.0.smithi139.stdout:3/166: mknod d5/d7/d23/d27/d2f/c31 0 2024-09-16T00:09:39.938 INFO:tasks.workunit.client.0.smithi139.stdout:2/131: dread d11/f18 [0,4194304] 0 2024-09-16T00:09:39.938 INFO:tasks.workunit.client.0.smithi139.stdout:7/155: creat d5/d16/d22/d25/d2d/f36 x:0 0 0 2024-09-16T00:09:39.939 INFO:tasks.workunit.client.0.smithi139.stdout:6/112: truncate d8/f12 2278138 0 2024-09-16T00:09:39.942 INFO:tasks.workunit.client.0.smithi139.stdout:5/173: creat d1/d25/f36 x:0 0 0 2024-09-16T00:09:39.944 INFO:tasks.workunit.client.0.smithi139.stdout:0/197: mknod d6/d1c/c3d 0 2024-09-16T00:09:39.945 INFO:tasks.workunit.client.0.smithi139.stdout:2/132: mkdir d11/d1f/d29 0 2024-09-16T00:09:39.945 INFO:tasks.workunit.client.0.smithi139.stdout:7/156: dwrite d5/d16/d22/d25/f31 [0,4194304] 0 2024-09-16T00:09:39.947 INFO:tasks.workunit.client.0.smithi139.stdout:7/157: write d5/fc [2661824,59649] 0 2024-09-16T00:09:39.947 INFO:tasks.workunit.client.0.smithi139.stdout:6/113: unlink l1 0 2024-09-16T00:09:39.948 INFO:tasks.workunit.client.0.smithi139.stdout:6/114: fdatasync d8/d20/d27/d10/f13 0 2024-09-16T00:09:39.952 INFO:tasks.workunit.client.0.smithi139.stdout:5/174: creat d1/d25/f37 x:0 0 0 2024-09-16T00:09:39.953 INFO:tasks.workunit.client.0.smithi139.stdout:8/188: mknod d17/d1a/d43/c47 0 2024-09-16T00:09:39.960 INFO:tasks.workunit.client.0.smithi139.stdout:4/82: creat d12/d15/f18 x:0 0 0 2024-09-16T00:09:39.960 INFO:tasks.workunit.client.0.smithi139.stdout:4/83: readlink - no filename 2024-09-16T00:09:39.960 INFO:tasks.workunit.client.0.smithi139.stdout:4/84: write fd [2394796,115817] 0 2024-09-16T00:09:39.961 INFO:tasks.workunit.client.0.smithi139.stdout:8/189: symlink d17/d1f/l48 0 2024-09-16T00:09:39.962 INFO:tasks.workunit.client.0.smithi139.stdout:2/133: rename d11/c23 to d11/d1f/d29/c2a 0 2024-09-16T00:09:39.965 INFO:tasks.workunit.client.0.smithi139.stdout:0/198: rename d6/dc/f31 to d6/dc/f3e 0 2024-09-16T00:09:39.967 INFO:tasks.workunit.client.0.smithi139.stdout:0/199: dread d6/d1c/f35 [0,4194304] 0 2024-09-16T00:09:39.969 INFO:tasks.workunit.client.0.smithi139.stdout:6/115: rename d8/d25 to d8/d29 0 2024-09-16T00:09:39.972 INFO:tasks.workunit.client.0.smithi139.stdout:8/190: creat d17/d1a/d43/d40/f49 x:0 0 0 2024-09-16T00:09:39.973 INFO:tasks.workunit.client.0.smithi139.stdout:2/134: fsync f6 0 2024-09-16T00:09:39.974 INFO:tasks.workunit.client.0.smithi139.stdout:7/158: truncate d5/f6 2433477 0 2024-09-16T00:09:39.974 INFO:tasks.workunit.client.0.smithi139.stdout:6/116: dwrite f6 [0,4194304] 0 2024-09-16T00:09:39.976 INFO:tasks.workunit.client.0.smithi139.stdout:7/159: write d5/d16/d22/d25/d2d/f36 [774848,4375] 0 2024-09-16T00:09:39.981 INFO:tasks.workunit.client.0.smithi139.stdout:7/160: dread d5/f27 [0,4194304] 0 2024-09-16T00:09:39.981 INFO:tasks.workunit.client.0.smithi139.stdout:2/135: write f6 [1435012,10564] 0 2024-09-16T00:09:39.982 INFO:tasks.workunit.client.0.smithi139.stdout:4/85: link fc d12/f19 0 2024-09-16T00:09:39.982 INFO:tasks.workunit.client.0.smithi139.stdout:4/86: chown c9 61879 1 2024-09-16T00:09:39.983 INFO:tasks.workunit.client.0.smithi139.stdout:2/136: read f8 [2722705,18343] 0 2024-09-16T00:09:39.987 INFO:tasks.workunit.client.0.smithi139.stdout:6/117: mkdir d8/db/d26/d28/d2a 0 2024-09-16T00:09:39.988 INFO:tasks.workunit.client.0.smithi139.stdout:8/191: creat d17/f4a x:0 0 0 2024-09-16T00:09:39.988 INFO:tasks.workunit.client.0.smithi139.stdout:7/161: rmdir d5/d16/d22 39 2024-09-16T00:09:39.991 INFO:tasks.workunit.client.0.smithi139.stdout:8/192: creat d17/d1a/d43/d40/f4b x:0 0 0 2024-09-16T00:09:39.992 INFO:tasks.workunit.client.0.smithi139.stdout:4/87: mknod d12/c1a 0 2024-09-16T00:09:39.992 INFO:tasks.workunit.client.0.smithi139.stdout:0/200: link d6/dc/dd/l17 d6/dc/l3f 0 2024-09-16T00:09:39.994 INFO:tasks.workunit.client.0.smithi139.stdout:2/137: dwrite f9 [0,4194304] 0 2024-09-16T00:09:39.995 INFO:tasks.workunit.client.0.smithi139.stdout:8/193: creat d17/f4c x:0 0 0 2024-09-16T00:09:39.996 INFO:tasks.workunit.client.0.smithi139.stdout:2/138: unlink f6 0 2024-09-16T00:09:39.997 INFO:tasks.workunit.client.0.smithi139.stdout:4/88: dread f1 [0,4194304] 0 2024-09-16T00:09:39.998 INFO:tasks.workunit.client.0.smithi139.stdout:2/139: mkdir d11/d1f/d29/d2b 0 2024-09-16T00:09:40.000 INFO:tasks.workunit.client.0.smithi139.stdout:2/140: write ff [3419565,108747] 0 2024-09-16T00:09:40.003 INFO:tasks.workunit.client.0.smithi139.stdout:6/118: truncate d8/db/f1a 2199396 0 2024-09-16T00:09:40.005 INFO:tasks.workunit.client.0.smithi139.stdout:8/194: dwrite d17/d1a/d43/d40/f4b [0,4194304] 0 2024-09-16T00:09:40.005 INFO:tasks.workunit.client.0.smithi139.stdout:2/141: creat d11/d1f/d29/d2b/f2c x:0 0 0 2024-09-16T00:09:40.006 INFO:tasks.workunit.client.0.smithi139.stdout:6/119: chown d8/d20/d27/d10/f11 85954 1 2024-09-16T00:09:40.007 INFO:tasks.workunit.client.0.smithi139.stdout:8/195: dread - d17/d1a/d2d/f3e zero size 2024-09-16T00:09:40.009 INFO:tasks.workunit.client.0.smithi139.stdout:6/120: link f3 d8/d20/f2b 0 2024-09-16T00:09:40.016 INFO:tasks.workunit.client.0.smithi139.stdout:4/89: dwrite f1 [0,4194304] 0 2024-09-16T00:09:40.019 INFO:tasks.workunit.client.0.smithi139.stdout:4/90: unlink d12/c14 0 2024-09-16T00:09:40.022 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:40.020+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:40.022 INFO:tasks.workunit.client.0.smithi139.stdout:2/142: dread d11/f13 [0,4194304] 0 2024-09-16T00:09:40.022 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:40.023 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:40.023 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:40.023 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:40.023 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:40.024 INFO:tasks.workunit.client.0.smithi139.stdout:8/196: dwrite d17/d1a/d2d/f38 [4194304,4194304] 0 2024-09-16T00:09:40.025 INFO:tasks.workunit.client.0.smithi139.stdout:8/197: stat d17/d1f/l39 0 2024-09-16T00:09:40.028 INFO:tasks.workunit.client.0.smithi139.stdout:4/91: dwrite fd [0,4194304] 0 2024-09-16T00:09:40.029 INFO:tasks.workunit.client.0.smithi139.stdout:8/198: mknod d17/d1a/d43/c4d 0 2024-09-16T00:09:40.032 INFO:tasks.workunit.client.0.smithi139.stdout:2/143: dread f2 [0,4194304] 0 2024-09-16T00:09:40.033 INFO:tasks.workunit.client.0.smithi139.stdout:4/92: symlink d12/l1b 0 2024-09-16T00:09:40.034 INFO:tasks.workunit.client.0.smithi139.stdout:8/199: symlink d17/d1a/d43/l4e 0 2024-09-16T00:09:40.034 INFO:tasks.workunit.client.0.smithi139.stdout:1/138: sync 2024-09-16T00:09:40.034 INFO:tasks.workunit.client.0.smithi139.stdout:9/161: sync 2024-09-16T00:09:40.035 INFO:tasks.workunit.client.0.smithi139.stdout:8/200: dread - d17/d1a/d43/f3b zero size 2024-09-16T00:09:40.035 INFO:tasks.workunit.client.0.smithi139.stdout:2/144: write f8 [307222,123331] 0 2024-09-16T00:09:40.036 INFO:tasks.workunit.client.0.smithi139.stdout:4/93: unlink f3 0 2024-09-16T00:09:40.037 INFO:tasks.workunit.client.0.smithi139.stdout:4/94: chown f4 1929 1 2024-09-16T00:09:40.037 INFO:tasks.workunit.client.0.smithi139.stdout:2/145: chown d11/c12 26662207 1 2024-09-16T00:09:40.039 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:09:40.039 INFO:tasks.workunit.client.0.smithi139.stdout:1/139: creat d7/f26 x:0 0 0 2024-09-16T00:09:40.040 INFO:tasks.workunit.client.0.smithi139.stdout:4/95: unlink d12/f19 0 2024-09-16T00:09:40.041 INFO:tasks.workunit.client.0.smithi139.stdout:9/162: truncate d19/d1a/d20/d23/f24 754474 0 2024-09-16T00:09:40.041 INFO:tasks.workunit.client.0.smithi139.stdout:1/140: write d7/d8/d1f/f24 [530721,112785] 0 2024-09-16T00:09:40.042 INFO:tasks.workunit.client.0.smithi139.stdout:2/146: creat d11/d1f/d29/d2b/f2d x:0 0 0 2024-09-16T00:09:40.043 INFO:tasks.workunit.client.0.smithi139.stdout:9/163: chown f6 1 1 2024-09-16T00:09:40.044 INFO:tasks.workunit.client.0.smithi139.stdout:8/201: dwrite d17/d1a/f35 [0,4194304] 0 2024-09-16T00:09:40.045 INFO:tasks.workunit.client.0.smithi139.stdout:9/164: chown d19/f1f 1279 1 2024-09-16T00:09:40.046 INFO:tasks.workunit.client.0.smithi139.stdout:8/202: chown d17/d1a/d43/c4d 175471654 1 2024-09-16T00:09:40.046 INFO:tasks.workunit.client.0.smithi139.stdout:1/141: mkdir d7/d8/d27 0 2024-09-16T00:09:40.048 INFO:tasks.workunit.client.0.smithi139.stdout:4/96: rename d12/c1a to d12/c1c 0 2024-09-16T00:09:40.050 INFO:tasks.workunit.client.0.smithi139.stdout:2/147: rename d11/f13 to d11/d1f/d29/d2b/f2e 0 2024-09-16T00:09:40.051 INFO:tasks.workunit.client.0.smithi139.stdout:9/165: write d19/d1a/d20/d23/f2b [2029256,31759] 0 2024-09-16T00:09:40.052 INFO:tasks.workunit.client.0.smithi139.stdout:1/142: stat d7/d8/c19 0 2024-09-16T00:09:40.052 INFO:tasks.workunit.client.0.smithi139.stdout:9/166: stat d19/d22 0 2024-09-16T00:09:40.053 INFO:tasks.workunit.client.0.smithi139.stdout:8/203: mkdir d17/d1a/d43/d40/d4f 0 2024-09-16T00:09:40.055 INFO:tasks.workunit.client.0.smithi139.stdout:8/204: dread - d17/d1f/d3a/f45 zero size 2024-09-16T00:09:40.056 INFO:tasks.workunit.client.0.smithi139.stdout:9/167: unlink d19/d1a/d20/d23/f2b 0 2024-09-16T00:09:40.057 INFO:tasks.workunit.client.0.smithi139.stdout:2/148: dwrite f9 [4194304,4194304] 0 2024-09-16T00:09:40.058 INFO:tasks.workunit.client.0.smithi139.stdout:9/168: chown d19/d1a/c28 1547 1 2024-09-16T00:09:40.059 INFO:tasks.workunit.client.0.smithi139.stdout:9/169: dread fb [0,4194304] 0 2024-09-16T00:09:40.060 INFO:tasks.workunit.client.0.smithi139.stdout:8/205: mkdir d17/d1a/d50 0 2024-09-16T00:09:40.061 INFO:tasks.workunit.client.0.smithi139.stdout:9/170: write d19/d1a/f1e [3369284,21438] 0 2024-09-16T00:09:40.061 INFO:tasks.workunit.client.0.smithi139.stdout:1/143: link d7/d8/f1c d7/f28 0 2024-09-16T00:09:40.062 INFO:tasks.workunit.client.0.smithi139.stdout:8/206: write d17/d1a/f34 [449173,8489] 0 2024-09-16T00:09:40.062 INFO:tasks.workunit.client.0.smithi139.stdout:4/97: truncate fc 204350 0 2024-09-16T00:09:40.063 INFO:tasks.workunit.client.0.smithi139.stdout:8/207: stat d17/d1a/c29 0 2024-09-16T00:09:40.066 INFO:tasks.workunit.client.0.smithi139.stdout:2/149: dread d11/d1f/d29/d2b/f2e [0,4194304] 0 2024-09-16T00:09:40.068 INFO:tasks.workunit.client.0.smithi139.stdout:4/98: symlink d12/d15/l1d 0 2024-09-16T00:09:40.068 INFO:tasks.workunit.client.0.smithi139.stdout:1/144: dread d7/d8/f1a [0,4194304] 0 2024-09-16T00:09:40.068 INFO:tasks.workunit.client.0.smithi139.stdout:8/208: rename d17/d1f/l39 to d17/d1f/d3a/l51 0 2024-09-16T00:09:40.070 INFO:tasks.workunit.client.0.smithi139.stdout:2/150: mkdir d11/d2f 0 2024-09-16T00:09:40.070 INFO:tasks.workunit.client.0.smithi139.stdout:9/171: dwrite d19/f1d [0,4194304] 0 2024-09-16T00:09:40.070 INFO:tasks.workunit.client.0.smithi139.stdout:9/172: fsync f6 0 2024-09-16T00:09:40.072 INFO:tasks.workunit.client.0.smithi139.stdout:4/99: dread fe [0,4194304] 0 2024-09-16T00:09:40.074 INFO:tasks.workunit.client.0.smithi139.stdout:9/173: mkdir d19/d1a/d20/d23/d2d 0 2024-09-16T00:09:40.074 INFO:tasks.workunit.client.0.smithi139.stdout:8/209: symlink d17/l52 0 2024-09-16T00:09:40.075 INFO:tasks.workunit.client.0.smithi139.stdout:8/210: dread - d17/d1f/d3a/f45 zero size 2024-09-16T00:09:40.076 INFO:tasks.workunit.client.0.smithi139.stdout:4/100: rmdir d12 39 2024-09-16T00:09:40.079 INFO:tasks.workunit.client.0.smithi139.stdout:4/101: chown d12/f17 137 1 2024-09-16T00:09:40.080 INFO:tasks.workunit.client.0.smithi139.stdout:9/174: dwrite d19/d1a/f1e [0,4194304] 0 2024-09-16T00:09:40.083 INFO:tasks.workunit.client.0.smithi139.stdout:8/211: rename d17/d1a/d2d/f38 to d17/d1a/f53 0 2024-09-16T00:09:40.085 INFO:tasks.workunit.client.0.smithi139.stdout:9/175: rename d19/d1a/c28 to d19/d1a/c2e 0 2024-09-16T00:09:40.088 INFO:tasks.workunit.client.0.smithi139.stdout:8/212: dwrite d17/d1a/d43/d40/f49 [0,4194304] 0 2024-09-16T00:09:40.093 INFO:tasks.workunit.client.0.smithi139.stdout:4/102: creat d12/f1e x:0 0 0 2024-09-16T00:09:40.093 INFO:tasks.workunit.client.0.smithi139.stdout:1/145: write d7/ff [946620,107264] 0 2024-09-16T00:09:40.094 INFO:tasks.workunit.client.0.smithi139.stdout:9/176: fsync f2 0 2024-09-16T00:09:40.097 INFO:tasks.workunit.client.0.smithi139.stdout:1/146: truncate d7/d8/f1e 545186 0 2024-09-16T00:09:40.098 INFO:tasks.workunit.client.0.smithi139.stdout:1/147: dread - d7/f26 zero size 2024-09-16T00:09:40.100 INFO:tasks.workunit.client.0.smithi139.stdout:4/103: rename d12/d15/l1d to d12/l1f 0 2024-09-16T00:09:40.101 INFO:tasks.workunit.client.0.smithi139.stdout:2/151: dwrite d11/f18 [4194304,4194304] 0 2024-09-16T00:09:40.101 INFO:tasks.workunit.client.0.smithi139.stdout:9/177: dread d19/d1a/f21 [0,4194304] 0 2024-09-16T00:09:40.102 INFO:tasks.workunit.client.0.smithi139.stdout:9/178: read f14 [1338010,59882] 0 2024-09-16T00:09:40.103 INFO:tasks.workunit.client.0.smithi139.stdout:9/179: chown d19/d22 11371335 1 2024-09-16T00:09:40.103 INFO:tasks.workunit.client.0.smithi139.stdout:8/213: fsync d17/d1a/d43/d40/f49 0 2024-09-16T00:09:40.105 INFO:tasks.workunit.client.0.smithi139.stdout:8/214: write d17/d1a/d43/d40/f49 [2471184,24476] 0 2024-09-16T00:09:40.106 INFO:tasks.workunit.client.0.smithi139.stdout:8/215: write d17/d1f/f28 [998040,34998] 0 2024-09-16T00:09:40.107 INFO:tasks.workunit.client.0.smithi139.stdout:4/104: rename d12/c13 to d12/d15/c20 0 2024-09-16T00:09:40.107 INFO:tasks.workunit.client.0.smithi139.stdout:8/216: truncate d17/d1f/f2c 554947 0 2024-09-16T00:09:40.112 INFO:tasks.workunit.client.0.smithi139.stdout:4/105: mkdir d12/d21 0 2024-09-16T00:09:40.114 INFO:tasks.workunit.client.0.smithi139.stdout:4/106: dread d12/f17 [0,4194304] 0 2024-09-16T00:09:40.115 INFO:tasks.workunit.client.0.smithi139.stdout:8/217: truncate d17/d1a/f1e 1221890 0 2024-09-16T00:09:40.117 INFO:tasks.workunit.client.0.smithi139.stdout:8/218: mkdir d17/d54 0 2024-09-16T00:09:40.119 INFO:tasks.workunit.client.0.smithi139.stdout:4/107: dwrite d12/f1e [0,4194304] 0 2024-09-16T00:09:40.123 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:40.121+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:40.123 INFO:tasks.workunit.client.0.smithi139.stdout:4/108: mknod d12/c22 0 2024-09-16T00:09:40.127 INFO:tasks.workunit.client.0.smithi139.stdout:4/109: dread fd [0,4194304] 0 2024-09-16T00:09:40.132 INFO:tasks.workunit.client.0.smithi139.stdout:8/219: dread d17/d1a/d2d/f2e [0,4194304] 0 2024-09-16T00:09:40.137 INFO:tasks.workunit.client.0.smithi139.stdout:3/167: sync 2024-09-16T00:09:40.142 INFO:tasks.workunit.client.0.smithi139.stdout:8/220: dwrite f7 [4194304,4194304] 0 2024-09-16T00:09:40.145 INFO:tasks.workunit.client.0.smithi139.stdout:8/221: link d17/d1a/d43/d40/f49 d17/d54/f55 0 2024-09-16T00:09:40.155 INFO:tasks.workunit.client.0.smithi139.stdout:3/168: dread d5/d7/d23/f11 [0,4194304] 0 2024-09-16T00:09:40.156 INFO:tasks.workunit.client.0.smithi139.stdout:3/169: truncate d5/d7/f2d 32384 0 2024-09-16T00:09:40.210 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:40.210 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:40.210 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:40.210 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:40.210 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.211 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10653 3.1c 3:38022ffc:::100000030c2.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:40.211 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.643883+0000", 2024-09-16T00:09:40.211 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.5643443809999997, 2024-09-16T00:09:40.211 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012764609999999999, 2024-09-16T00:09:40.211 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.211 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.211 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.211 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.211 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.211 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.211 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10653 2024-09-16T00:09:40.211 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.211 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.211 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.643883+0000", 2024-09-16T00:09:40.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.643883+0000", 2024-09-16T00:09:40.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.212 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.212 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.212 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.212 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.643884+0000", 2024-09-16T00:09:40.212 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8599999999999997e-07 2024-09-16T00:09:40.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.651117+0000", 2024-09-16T00:09:40.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072330750000000003 2024-09-16T00:09:40.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.213 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.651128+0000", 2024-09-16T00:09:40.213 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0699e-05 2024-09-16T00:09:40.213 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.213 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.213 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.651133+0000", 2024-09-16T00:09:40.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3090000000000003e-06 2024-09-16T00:09:40.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.651182+0000", 2024-09-16T00:09:40.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9313e-05 2024-09-16T00:09:40.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.214 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.214 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.651235+0000", 2024-09-16T00:09:40.214 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2864000000000001e-05 2024-09-16T00:09:40.214 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.214 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:40.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.651339+0000", 2024-09-16T00:09:40.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000104065 2024-09-16T00:09:40.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.656308+0000", 2024-09-16T00:09:40.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049685420000000003 2024-09-16T00:09:40.215 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.215 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.215 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.215 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.656549+0000", 2024-09-16T00:09:40.215 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00024080799999999999 2024-09-16T00:09:40.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.656606+0000", 2024-09-16T00:09:40.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7454000000000003e-05 2024-09-16T00:09:40.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.216 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.216 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.216 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.656648+0000", 2024-09-16T00:09:40.216 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1594999999999997e-05 2024-09-16T00:09:40.216 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.216 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.216 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.216 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.217 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.217 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10658 3.1c 3:38c4c24e:::100000030bb.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:40.217 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:09:40.217 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4168311400000002, 2024-09-16T00:09:40.217 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021232445999999999, 2024-09-16T00:09:40.217 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.217 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.217 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.217 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.217 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.217 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.217 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10658 2024-09-16T00:09:40.217 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.218 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:09:40.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:09:40.218 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.218 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.218 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.218 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.218 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791398+0000", 2024-09-16T00:09:40.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3880000000000001e-06 2024-09-16T00:09:40.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800951+0000", 2024-09-16T00:09:40.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095532110000000007 2024-09-16T00:09:40.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.219 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800966+0000", 2024-09-16T00:09:40.219 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5058999999999999e-05 2024-09-16T00:09:40.219 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.219 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.219 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800974+0000", 2024-09-16T00:09:40.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1240000000000005e-06 2024-09-16T00:09:40.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801127+0000", 2024-09-16T00:09:40.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000153103 2024-09-16T00:09:40.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.220 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.220 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.220 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801207+0000", 2024-09-16T00:09:40.220 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9569999999999999e-05 2024-09-16T00:09:40.220 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:40.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801322+0000", 2024-09-16T00:09:40.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115358 2024-09-16T00:09:40.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.805760+0000", 2024-09-16T00:09:40.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044379409999999999 2024-09-16T00:09:40.221 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.221 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.221 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812528+0000", 2024-09-16T00:09:40.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067678249999999999 2024-09-16T00:09:40.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812586+0000", 2024-09-16T00:09:40.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8164000000000001e-05 2024-09-16T00:09:40.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812629+0000", 2024-09-16T00:09:40.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2703000000000001e-05 2024-09-16T00:09:40.222 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.222 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.222 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.223 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10674 3.1c 3:38022ffc:::100000030c2.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:40.223 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.857246+0000", 2024-09-16T00:09:40.223 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.350981258, 2024-09-16T00:09:40.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013541578, 2024-09-16T00:09:40.223 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.223 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.223 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.223 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10674 2024-09-16T00:09:40.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.224 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.857246+0000", 2024-09-16T00:09:40.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.857246+0000", 2024-09-16T00:09:40.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.857247+0000", 2024-09-16T00:09:40.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2669999999999999e-06 2024-09-16T00:09:40.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.863159+0000", 2024-09-16T00:09:40.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005911085 2024-09-16T00:09:40.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.863173+0000", 2024-09-16T00:09:40.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4651000000000001e-05 2024-09-16T00:09:40.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.863181+0000", 2024-09-16T00:09:40.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2959999999999999e-06 2024-09-16T00:09:40.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.863243+0000", 2024-09-16T00:09:40.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1969000000000002e-05 2024-09-16T00:09:40.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.863330+0000", 2024-09-16T00:09:40.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6971999999999993e-05 2024-09-16T00:09:40.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:40.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.863471+0000", 2024-09-16T00:09:40.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014018599999999999 2024-09-16T00:09:40.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.869372+0000", 2024-09-16T00:09:40.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059012989999999996 2024-09-16T00:09:40.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.870723+0000", 2024-09-16T00:09:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001350672 2024-09-16T00:09:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.870758+0000", 2024-09-16T00:09:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5207999999999999e-05 2024-09-16T00:09:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.870788+0000", 2024-09-16T00:09:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9972999999999999e-05 2024-09-16T00:09:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.228 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10777 3.3c 3:3ee19ef4:::10000003114.00000000:head [write 0~3538944] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.447001+0000", 2024-09-16T00:09:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.7612268580000001, 2024-09-16T00:09:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01321407, 2024-09-16T00:09:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.229 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10777 2024-09-16T00:09:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.447001+0000", 2024-09-16T00:09:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.447001+0000", 2024-09-16T00:09:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.447001+0000", 2024-09-16T00:09:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6700000000000002e-07 2024-09-16T00:09:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.451615+0000", 2024-09-16T00:09:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004613593 2024-09-16T00:09:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.451627+0000", 2024-09-16T00:09:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1701e-05 2024-09-16T00:09:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.451633+0000", 2024-09-16T00:09:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8880000000000002e-06 2024-09-16T00:09:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.451682+0000", 2024-09-16T00:09:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9298000000000001e-05 2024-09-16T00:09:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.451803+0000", 2024-09-16T00:09:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012070600000000001 2024-09-16T00:09:40.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.451890+0000", 2024-09-16T00:09:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7613000000000003e-05 2024-09-16T00:09:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.457215+0000", 2024-09-16T00:09:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00532439 2024-09-16T00:09:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.460143+0000", 2024-09-16T00:09:40.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029279480000000001 2024-09-16T00:09:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.460189+0000", 2024-09-16T00:09:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5977999999999999e-05 2024-09-16T00:09:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.460215+0000", 2024-09-16T00:09:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6088000000000002e-05 2024-09-16T00:09:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10790 3.5 3:a3c82b2e:::10000003104.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.526361+0000", 2024-09-16T00:09:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.681866445, 2024-09-16T00:09:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013459364999999999, 2024-09-16T00:09:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10790 2024-09-16T00:09:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.235 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.526361+0000", 2024-09-16T00:09:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.526361+0000", 2024-09-16T00:09:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.526367+0000", 2024-09-16T00:09:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.336e-06 2024-09-16T00:09:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.530600+0000", 2024-09-16T00:09:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004233161 2024-09-16T00:09:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.530620+0000", 2024-09-16T00:09:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9065000000000001e-05 2024-09-16T00:09:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.530629+0000", 2024-09-16T00:09:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9789999999999999e-06 2024-09-16T00:09:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.530693+0000", 2024-09-16T00:09:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4517000000000006e-05 2024-09-16T00:09:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.530802+0000", 2024-09-16T00:09:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108671 2024-09-16T00:09:40.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.530894+0000", 2024-09-16T00:09:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2181000000000004e-05 2024-09-16T00:09:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.534179+0000", 2024-09-16T00:09:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032855100000000002 2024-09-16T00:09:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.539756+0000", 2024-09-16T00:09:40.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055762609999999999 2024-09-16T00:09:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.539790+0000", 2024-09-16T00:09:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4249999999999999e-05 2024-09-16T00:09:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.539820+0000", 2024-09-16T00:09:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0434e-05 2024-09-16T00:09:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10827 3.1c 3:38022ffc:::100000030c2.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.744026+0000", 2024-09-16T00:09:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.464201514, 2024-09-16T00:09:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013763638, 2024-09-16T00:09:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10827 2024-09-16T00:09:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.241 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.744026+0000", 2024-09-16T00:09:40.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.744026+0000", 2024-09-16T00:09:40.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.744028+0000", 2024-09-16T00:09:40.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8020000000000001e-06 2024-09-16T00:09:40.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.749946+0000", 2024-09-16T00:09:40.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059182410000000003 2024-09-16T00:09:40.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.749969+0000", 2024-09-16T00:09:40.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2881999999999999e-05 2024-09-16T00:09:40.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.749980+0000", 2024-09-16T00:09:40.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1552000000000001e-05 2024-09-16T00:09:40.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.750057+0000", 2024-09-16T00:09:40.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6471999999999996e-05 2024-09-16T00:09:40.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.750159+0000", 2024-09-16T00:09:40.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010180500000000001 2024-09-16T00:09:40.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:40.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.750320+0000", 2024-09-16T00:09:40.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016095700000000001 2024-09-16T00:09:40.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.755446+0000", 2024-09-16T00:09:40.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051264709999999996 2024-09-16T00:09:40.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.757688+0000", 2024-09-16T00:09:40.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022419670000000001 2024-09-16T00:09:40.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.757748+0000", 2024-09-16T00:09:40.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0197e-05 2024-09-16T00:09:40.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.757790+0000", 2024-09-16T00:09:40.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1291999999999997e-05 2024-09-16T00:09:40.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.246 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.247 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10854 3.16 3:6919f8ec:::10000003105.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:40.247 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.917775+0000", 2024-09-16T00:09:40.247 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.290452787, 2024-09-16T00:09:40.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012220975, 2024-09-16T00:09:40.247 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.247 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.247 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.247 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10854 2024-09-16T00:09:40.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.247 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.917775+0000", 2024-09-16T00:09:40.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.917775+0000", 2024-09-16T00:09:40.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.917777+0000", 2024-09-16T00:09:40.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.441e-06 2024-09-16T00:09:40.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.921688+0000", 2024-09-16T00:09:40.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039112560000000001 2024-09-16T00:09:40.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.921706+0000", 2024-09-16T00:09:40.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7643999999999999e-05 2024-09-16T00:09:40.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.921716+0000", 2024-09-16T00:09:40.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6390000000000004e-06 2024-09-16T00:09:40.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.921777+0000", 2024-09-16T00:09:40.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1440999999999997e-05 2024-09-16T00:09:40.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.921917+0000", 2024-09-16T00:09:40.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014033900000000001 2024-09-16T00:09:40.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:40.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.922010+0000", 2024-09-16T00:09:40.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2414e-05 2024-09-16T00:09:40.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.929843+0000", 2024-09-16T00:09:40.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0078335560000000002 2024-09-16T00:09:40.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.929907+0000", 2024-09-16T00:09:40.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3571999999999994e-05 2024-09-16T00:09:40.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.929960+0000", 2024-09-16T00:09:40.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2879e-05 2024-09-16T00:09:40.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.929996+0000", 2024-09-16T00:09:40.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5794000000000003e-05 2024-09-16T00:09:40.253 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.253 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.253 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.253 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10864 3.28 3:1699d86f:::10000003142.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:40.253 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.935769+0000", 2024-09-16T00:09:40.254 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.2724583570000001, 2024-09-16T00:09:40.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013453568000000001, 2024-09-16T00:09:40.254 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.254 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.254 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.254 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.254 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.254 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.254 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10864 2024-09-16T00:09:40.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.254 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.935769+0000", 2024-09-16T00:09:40.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.935769+0000", 2024-09-16T00:09:40.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.935770+0000", 2024-09-16T00:09:40.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0999999999999997e-07 2024-09-16T00:09:40.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.940078+0000", 2024-09-16T00:09:40.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043085509999999999 2024-09-16T00:09:40.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.940094+0000", 2024-09-16T00:09:40.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5304000000000001e-05 2024-09-16T00:09:40.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.940101+0000", 2024-09-16T00:09:40.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9920000000000001e-06 2024-09-16T00:09:40.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.940150+0000", 2024-09-16T00:09:40.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9706000000000001e-05 2024-09-16T00:09:40.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.940216+0000", 2024-09-16T00:09:40.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5555999999999994e-05 2024-09-16T00:09:40.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:40.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.940307+0000", 2024-09-16T00:09:40.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1464000000000004e-05 2024-09-16T00:09:40.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.945201+0000", 2024-09-16T00:09:40.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048940060000000002 2024-09-16T00:09:40.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.949148+0000", 2024-09-16T00:09:40.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039461899999999996 2024-09-16T00:09:40.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.949183+0000", 2024-09-16T00:09:40.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5652999999999999e-05 2024-09-16T00:09:40.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.949223+0000", 2024-09-16T00:09:40.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9335999999999997e-05 2024-09-16T00:09:40.259 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.259 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.259 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.260 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10954 3.3c 3:3c2be35d:::100000030f7.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:40.260 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.509020+0000", 2024-09-16T00:09:40.260 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.69920747900000002, 2024-09-16T00:09:40.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016515911000000001, 2024-09-16T00:09:40.260 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.260 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.260 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.260 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10954 2024-09-16T00:09:40.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.260 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.509020+0000", 2024-09-16T00:09:40.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.509020+0000", 2024-09-16T00:09:40.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.509022+0000", 2024-09-16T00:09:40.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.187e-06 2024-09-16T00:09:40.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513579+0000", 2024-09-16T00:09:40.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045566269999999997 2024-09-16T00:09:40.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513602+0000", 2024-09-16T00:09:40.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3617000000000001e-05 2024-09-16T00:09:40.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513616+0000", 2024-09-16T00:09:40.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.343e-05 2024-09-16T00:09:40.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513699+0000", 2024-09-16T00:09:40.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2719999999999994e-05 2024-09-16T00:09:40.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513821+0000", 2024-09-16T00:09:40.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001228 2024-09-16T00:09:40.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:40.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513987+0000", 2024-09-16T00:09:40.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016540200000000001 2024-09-16T00:09:40.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.520104+0000", 2024-09-16T00:09:40.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061177280000000002 2024-09-16T00:09:40.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.525469+0000", 2024-09-16T00:09:40.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053645730000000001 2024-09-16T00:09:40.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.525503+0000", 2024-09-16T00:09:40.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4378000000000002e-05 2024-09-16T00:09:40.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.525536+0000", 2024-09-16T00:09:40.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2449000000000001e-05 2024-09-16T00:09:40.265 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.265 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.265 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.266 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10956 3.30 3:0cbee1e3:::10000003173.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:40.266 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.513819+0000", 2024-09-16T00:09:40.266 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.69440828099999996, 2024-09-16T00:09:40.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015930732, 2024-09-16T00:09:40.266 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.266 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.266 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.266 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.266 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.266 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.266 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10956 2024-09-16T00:09:40.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.266 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513819+0000", 2024-09-16T00:09:40.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513819+0000", 2024-09-16T00:09:40.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513821+0000", 2024-09-16T00:09:40.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.677e-06 2024-09-16T00:09:40.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.518999+0000", 2024-09-16T00:09:40.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051783489999999996 2024-09-16T00:09:40.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.519020+0000", 2024-09-16T00:09:40.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0489999999999999e-05 2024-09-16T00:09:40.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.519033+0000", 2024-09-16T00:09:40.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.295e-05 2024-09-16T00:09:40.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.519106+0000", 2024-09-16T00:09:40.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3326999999999998e-05 2024-09-16T00:09:40.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.519243+0000", 2024-09-16T00:09:40.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013749500000000001 2024-09-16T00:09:40.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:40.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.519379+0000", 2024-09-16T00:09:40.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000136017 2024-09-16T00:09:40.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.524268+0000", 2024-09-16T00:09:40.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048883820000000001 2024-09-16T00:09:40.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.529701+0000", 2024-09-16T00:09:40.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054327259999999997 2024-09-16T00:09:40.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.529730+0000", 2024-09-16T00:09:40.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9071e-05 2024-09-16T00:09:40.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.529750+0000", 2024-09-16T00:09:40.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0248e-05 2024-09-16T00:09:40.271 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.271 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.271 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.272 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10968 3.5 3:a09cd6d9:::10000003180.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:40.272 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.602942+0000", 2024-09-16T00:09:40.272 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.60528534899999997, 2024-09-16T00:09:40.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014525788, 2024-09-16T00:09:40.272 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.272 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.272 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.272 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10968 2024-09-16T00:09:40.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.273 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.602942+0000", 2024-09-16T00:09:40.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.602942+0000", 2024-09-16T00:09:40.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.602943+0000", 2024-09-16T00:09:40.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.404e-06 2024-09-16T00:09:40.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.607434+0000", 2024-09-16T00:09:40.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044904740000000004 2024-09-16T00:09:40.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.607454+0000", 2024-09-16T00:09:40.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9966000000000002e-05 2024-09-16T00:09:40.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.607463+0000", 2024-09-16T00:09:40.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5370000000000003e-06 2024-09-16T00:09:40.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.607532+0000", 2024-09-16T00:09:40.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8996000000000001e-05 2024-09-16T00:09:40.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.607685+0000", 2024-09-16T00:09:40.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152151 2024-09-16T00:09:40.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:40.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.607827+0000", 2024-09-16T00:09:40.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014191299999999999 2024-09-16T00:09:40.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.612195+0000", 2024-09-16T00:09:40.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043685249999999998 2024-09-16T00:09:40.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.617403+0000", 2024-09-16T00:09:40.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052079819999999999 2024-09-16T00:09:40.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.617442+0000", 2024-09-16T00:09:40.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9388999999999998e-05 2024-09-16T00:09:40.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.617468+0000", 2024-09-16T00:09:40.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5451000000000001e-05 2024-09-16T00:09:40.277 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.277 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.277 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.278 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10993 3.2a 3:56ee67f2:::10000003146.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:40.278 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.689451+0000", 2024-09-16T00:09:40.278 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.51877684199999996, 2024-09-16T00:09:40.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013790253000000001, 2024-09-16T00:09:40.278 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.278 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.278 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.278 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10993 2024-09-16T00:09:40.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.278 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.689451+0000", 2024-09-16T00:09:40.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.689451+0000", 2024-09-16T00:09:40.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.689452+0000", 2024-09-16T00:09:40.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.728e-06 2024-09-16T00:09:40.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.693556+0000", 2024-09-16T00:09:40.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041039900000000001 2024-09-16T00:09:40.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.693576+0000", 2024-09-16T00:09:40.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9763999999999999e-05 2024-09-16T00:09:40.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.693589+0000", 2024-09-16T00:09:40.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2717e-05 2024-09-16T00:09:40.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.693664+0000", 2024-09-16T00:09:40.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5592000000000004e-05 2024-09-16T00:09:40.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.693734+0000", 2024-09-16T00:09:40.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9375000000000006e-05 2024-09-16T00:09:40.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:40.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.693856+0000", 2024-09-16T00:09:40.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012249299999999999 2024-09-16T00:09:40.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.699797+0000", 2024-09-16T00:09:40.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059408739999999996 2024-09-16T00:09:40.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.703159+0000", 2024-09-16T00:09:40.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033615820000000001 2024-09-16T00:09:40.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.703194+0000", 2024-09-16T00:09:40.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5760999999999999e-05 2024-09-16T00:09:40.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.703241+0000", 2024-09-16T00:09:40.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6377e-05 2024-09-16T00:09:40.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.283 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.284 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10999 3.30 3:0d2bb3e6:::1000000313b.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:40.284 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.716323+0000", 2024-09-16T00:09:40.284 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.49190429699999999, 2024-09-16T00:09:40.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014232977000000001, 2024-09-16T00:09:40.284 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.284 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.284 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.284 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.284 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.284 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.284 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10999 2024-09-16T00:09:40.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.284 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.716323+0000", 2024-09-16T00:09:40.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.716323+0000", 2024-09-16T00:09:40.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.716325+0000", 2024-09-16T00:09:40.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3850000000000001e-06 2024-09-16T00:09:40.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.720324+0000", 2024-09-16T00:09:40.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003999496 2024-09-16T00:09:40.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.720350+0000", 2024-09-16T00:09:40.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5590000000000001e-05 2024-09-16T00:09:40.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.720363+0000", 2024-09-16T00:09:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T00:09:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.720448+0000", 2024-09-16T00:09:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5053e-05 2024-09-16T00:09:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.720551+0000", 2024-09-16T00:09:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103055 2024-09-16T00:09:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.720689+0000", 2024-09-16T00:09:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000137582 2024-09-16T00:09:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.724645+0000", 2024-09-16T00:09:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039561939999999997 2024-09-16T00:09:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.730487+0000", 2024-09-16T00:09:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058420340000000003 2024-09-16T00:09:40.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.730531+0000", 2024-09-16T00:09:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3748999999999997e-05 2024-09-16T00:09:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.730556+0000", 2024-09-16T00:09:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5435999999999998e-05 2024-09-16T00:09:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11051 3.30 3:0d2bb3e6:::1000000313b.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.831338+0000", 2024-09-16T00:09:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.37688970500000002, 2024-09-16T00:09:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013649867, 2024-09-16T00:09:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11051 2024-09-16T00:09:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.831338+0000", 2024-09-16T00:09:40.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.831338+0000", 2024-09-16T00:09:40.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.831340+0000", 2024-09-16T00:09:40.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9850000000000001e-06 2024-09-16T00:09:40.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.835421+0000", 2024-09-16T00:09:40.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040808930000000004 2024-09-16T00:09:40.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.835445+0000", 2024-09-16T00:09:40.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4542e-05 2024-09-16T00:09:40.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.835452+0000", 2024-09-16T00:09:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3289999999999996e-06 2024-09-16T00:09:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.835516+0000", 2024-09-16T00:09:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3406999999999998e-05 2024-09-16T00:09:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.835577+0000", 2024-09-16T00:09:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0854999999999999e-05 2024-09-16T00:09:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:40.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.835671+0000", 2024-09-16T00:09:40.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4302999999999995e-05 2024-09-16T00:09:40.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.840227+0000", 2024-09-16T00:09:40.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045556240000000003 2024-09-16T00:09:40.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.844895+0000", 2024-09-16T00:09:40.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046682809999999998 2024-09-16T00:09:40.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.844945+0000", 2024-09-16T00:09:40.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0417999999999997e-05 2024-09-16T00:09:40.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.844988+0000", 2024-09-16T00:09:40.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2230000000000001e-05 2024-09-16T00:09:40.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.295 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11058 3.30 3:0ecbb745:::1000000310c.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.867454+0000", 2024-09-16T00:09:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.34077355399999998, 2024-09-16T00:09:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013409572, 2024-09-16T00:09:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11058 2024-09-16T00:09:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.867454+0000", 2024-09-16T00:09:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.867454+0000", 2024-09-16T00:09:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.867456+0000", 2024-09-16T00:09:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.663e-06 2024-09-16T00:09:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.871488+0000", 2024-09-16T00:09:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040322659999999996 2024-09-16T00:09:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.871513+0000", 2024-09-16T00:09:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5301000000000001e-05 2024-09-16T00:09:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.871525+0000", 2024-09-16T00:09:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.236e-05 2024-09-16T00:09:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.871601+0000", 2024-09-16T00:09:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5127000000000001e-05 2024-09-16T00:09:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.871775+0000", 2024-09-16T00:09:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017444699999999999 2024-09-16T00:09:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.871918+0000", 2024-09-16T00:09:40.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014281600000000001 2024-09-16T00:09:40.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.875688+0000", 2024-09-16T00:09:40.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037705730000000002 2024-09-16T00:09:40.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.880757+0000", 2024-09-16T00:09:40.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050689760000000002 2024-09-16T00:09:40.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.880816+0000", 2024-09-16T00:09:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8177999999999998e-05 2024-09-16T00:09:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.880863+0000", 2024-09-16T00:09:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7865000000000003e-05 2024-09-16T00:09:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11060 3.2a 3:56ee67f2:::10000003146.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:40.302 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.872925+0000", 2024-09-16T00:09:40.302 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.33530258499999999, 2024-09-16T00:09:40.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012407483, 2024-09-16T00:09:40.302 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.302 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.302 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.302 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11060 2024-09-16T00:09:40.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.302 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.872925+0000", 2024-09-16T00:09:40.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.872925+0000", 2024-09-16T00:09:40.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.872927+0000", 2024-09-16T00:09:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2989999999999999e-06 2024-09-16T00:09:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.877110+0000", 2024-09-16T00:09:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041826609999999998 2024-09-16T00:09:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.877130+0000", 2024-09-16T00:09:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0216000000000001e-05 2024-09-16T00:09:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.877141+0000", 2024-09-16T00:09:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1358999999999999e-05 2024-09-16T00:09:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.877206+0000", 2024-09-16T00:09:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5112999999999998e-05 2024-09-16T00:09:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.877272+0000", 2024-09-16T00:09:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5314000000000005e-05 2024-09-16T00:09:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.877362+0000", 2024-09-16T00:09:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0212999999999998e-05 2024-09-16T00:09:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.884581+0000", 2024-09-16T00:09:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007218983 2024-09-16T00:09:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.885226+0000", 2024-09-16T00:09:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00064522500000000003 2024-09-16T00:09:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.885286+0000", 2024-09-16T00:09:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0003000000000001e-05 2024-09-16T00:09:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.885332+0000", 2024-09-16T00:09:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6097000000000003e-05 2024-09-16T00:09:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11076 3.2f 3:f633d486:::1000000311d.00000000:head [write 0~4194304 [2@850962]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.915473+0000", 2024-09-16T00:09:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.29275457900000001, 2024-09-16T00:09:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016108857000000001, 2024-09-16T00:09:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11076 2024-09-16T00:09:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.915473+0000", 2024-09-16T00:09:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.915473+0000", 2024-09-16T00:09:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.915478+0000", 2024-09-16T00:09:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1699999999999996e-06 2024-09-16T00:09:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.921607+0000", 2024-09-16T00:09:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061287889999999999 2024-09-16T00:09:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.921632+0000", 2024-09-16T00:09:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5299999999999998e-05 2024-09-16T00:09:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.921642+0000", 2024-09-16T00:09:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6220000000000004e-06 2024-09-16T00:09:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.921717+0000", 2024-09-16T00:09:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5475000000000005e-05 2024-09-16T00:09:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.921818+0000", 2024-09-16T00:09:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100692 2024-09-16T00:09:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.921961+0000", 2024-09-16T00:09:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014336700000000001 2024-09-16T00:09:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928835+0000", 2024-09-16T00:09:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068737729999999997 2024-09-16T00:09:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.931483+0000", 2024-09-16T00:09:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026481709999999999 2024-09-16T00:09:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.931540+0000", 2024-09-16T00:09:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6793000000000003e-05 2024-09-16T00:09:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.931582+0000", 2024-09-16T00:09:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1705000000000001e-05 2024-09-16T00:09:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11080 3.3c 3:3e92f261:::100000031f3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.946892+0000", 2024-09-16T00:09:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.26133537400000001, 2024-09-16T00:09:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01243791, 2024-09-16T00:09:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11080 2024-09-16T00:09:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.946892+0000", 2024-09-16T00:09:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.946892+0000", 2024-09-16T00:09:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.946898+0000", 2024-09-16T00:09:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9649999999999996e-06 2024-09-16T00:09:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.951614+0000", 2024-09-16T00:09:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047160520000000001 2024-09-16T00:09:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.951635+0000", 2024-09-16T00:09:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1342e-05 2024-09-16T00:09:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.951646+0000", 2024-09-16T00:09:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0514e-05 2024-09-16T00:09:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.951735+0000", 2024-09-16T00:09:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9191999999999996e-05 2024-09-16T00:09:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.951883+0000", 2024-09-16T00:09:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014800999999999999 2024-09-16T00:09:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.952023+0000", 2024-09-16T00:09:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013998600000000001 2024-09-16T00:09:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.956728+0000", 2024-09-16T00:09:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047052880000000002 2024-09-16T00:09:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.959219+0000", 2024-09-16T00:09:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024910969999999998 2024-09-16T00:09:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.959281+0000", 2024-09-16T00:09:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1477999999999997e-05 2024-09-16T00:09:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.959330+0000", 2024-09-16T00:09:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8986000000000002e-05 2024-09-16T00:09:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11110 3.1c 3:38022ffc:::100000030c2.00000000:head [write 0~4194304 [2@2506451]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.081461+0000", 2024-09-16T00:09:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.12676599, 2024-09-16T00:09:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013421068, 2024-09-16T00:09:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11110 2024-09-16T00:09:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.081461+0000", 2024-09-16T00:09:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.081461+0000", 2024-09-16T00:09:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.081464+0000", 2024-09-16T00:09:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4590000000000001e-06 2024-09-16T00:09:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.085882+0000", 2024-09-16T00:09:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044185040000000002 2024-09-16T00:09:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.085909+0000", 2024-09-16T00:09:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6732000000000001e-05 2024-09-16T00:09:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.085921+0000", 2024-09-16T00:09:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.219e-05 2024-09-16T00:09:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.086029+0000", 2024-09-16T00:09:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107663 2024-09-16T00:09:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.086130+0000", 2024-09-16T00:09:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101042 2024-09-16T00:09:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.086286+0000", 2024-09-16T00:09:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000156031 2024-09-16T00:09:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.092679+0000", 2024-09-16T00:09:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006393358 2024-09-16T00:09:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094764+0000", 2024-09-16T00:09:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020850629999999998 2024-09-16T00:09:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094821+0000", 2024-09-16T00:09:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6067999999999999e-05 2024-09-16T00:09:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094882+0000", 2024-09-16T00:09:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1958000000000005e-05 2024-09-16T00:09:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11133 3.16 3:6b3ee8b4:::100000030d8.00000001:head [write 0~4194304 [2@220362]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.142991+0000", 2024-09-16T00:09:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.065235958999999996, 2024-09-16T00:09:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012088144, 2024-09-16T00:09:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11133 2024-09-16T00:09:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.142991+0000", 2024-09-16T00:09:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.142991+0000", 2024-09-16T00:09:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.142992+0000", 2024-09-16T00:09:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0299999999999998e-07 2024-09-16T00:09:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.146958+0000", 2024-09-16T00:09:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039655109999999997 2024-09-16T00:09:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.146978+0000", 2024-09-16T00:09:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0608999999999999e-05 2024-09-16T00:09:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.146987+0000", 2024-09-16T00:09:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6209999999999993e-06 2024-09-16T00:09:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.147056+0000", 2024-09-16T00:09:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9438999999999997e-05 2024-09-16T00:09:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.147121+0000", 2024-09-16T00:09:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5050999999999998e-05 2024-09-16T00:09:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.147220+0000", 2024-09-16T00:09:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8479000000000004e-05 2024-09-16T00:09:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.151827+0000", 2024-09-16T00:09:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004607501 2024-09-16T00:09:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.155010+0000", 2024-09-16T00:09:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031821810000000001 2024-09-16T00:09:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.155048+0000", 2024-09-16T00:09:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8936000000000001e-05 2024-09-16T00:09:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.155080+0000", 2024-09-16T00:09:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1112999999999998e-05 2024-09-16T00:09:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:40.332 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:40.334 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:09:40.335 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:40.332+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:40.338 INFO:tasks.workunit.client.0.smithi139.stdout:5/175: sync 2024-09-16T00:09:40.338 INFO:tasks.workunit.client.0.smithi139.stdout:5/176: rename d1/d21/d2c/c32 to d1/d25/d30/d33/c38 0 2024-09-16T00:09:40.339 INFO:tasks.workunit.client.0.smithi139.stdout:5/177: write d1/f16 [3471131,107014] 0 2024-09-16T00:09:40.339 INFO:tasks.workunit.client.0.smithi139.stdout:5/178: truncate d1/d13/f24 704077 0 2024-09-16T00:09:40.339 INFO:tasks.workunit.client.0.smithi139.stdout:5/179: link d1/d25/f2d d1/d21/f39 0 2024-09-16T00:09:40.339 INFO:tasks.workunit.client.0.smithi139.stdout:5/180: unlink d1/c9 0 2024-09-16T00:09:40.339 INFO:tasks.workunit.client.0.smithi139.stdout:5/181: creat d1/d21/f3a x:0 0 0 2024-09-16T00:09:40.339 INFO:tasks.workunit.client.0.smithi139.stdout:5/182: truncate d1/d13/f17 2674480 0 2024-09-16T00:09:40.339 INFO:tasks.workunit.client.0.smithi139.stdout:5/183: mkdir d1/d13/d3b 0 2024-09-16T00:09:40.339 INFO:tasks.workunit.client.0.smithi139.stdout:5/184: readlink d1/d13/d22/l2a 0 2024-09-16T00:09:40.339 INFO:tasks.workunit.client.0.smithi139.stdout:5/185: mkdir d1/d3c 0 2024-09-16T00:09:40.339 INFO:tasks.workunit.client.0.smithi139.stdout:5/186: mknod d1/d25/d30/d33/c3d 0 2024-09-16T00:09:40.339 INFO:tasks.workunit.client.0.smithi139.stdout:5/187: chown d1/f2f 748 1 2024-09-16T00:09:40.339 INFO:tasks.workunit.client.0.smithi139.stdout:5/188: stat d1/d21 0 2024-09-16T00:09:40.339 INFO:tasks.workunit.client.0.smithi139.stdout:5/189: write d1/d13/f17 [3010691,96265] 0 2024-09-16T00:09:40.340 INFO:tasks.workunit.client.0.smithi139.stdout:5/190: rename d1/d3c to d1/d13/d3b/d3e 0 2024-09-16T00:09:40.340 INFO:tasks.workunit.client.0.smithi139.stdout:5/191: dwrite d1/d25/f2e [0,4194304] 0 2024-09-16T00:09:40.340 INFO:tasks.workunit.client.0.smithi139.stdout:7/162: sync 2024-09-16T00:09:40.340 INFO:tasks.workunit.client.0.smithi139.stdout:0/201: sync 2024-09-16T00:09:40.340 INFO:tasks.workunit.client.0.smithi139.stdout:6/121: sync 2024-09-16T00:09:40.340 INFO:tasks.workunit.client.0.smithi139.stdout:5/192: rename d1/d21/d2c to d1/d13/d3f 0 2024-09-16T00:09:40.340 INFO:tasks.workunit.client.0.smithi139.stdout:0/202: read d6/f8 [5375046,102289] 0 2024-09-16T00:09:40.340 INFO:tasks.workunit.client.0.smithi139.stdout:0/203: write d6/dc/dd/f34 [795892,54096] 0 2024-09-16T00:09:40.340 INFO:tasks.workunit.client.0.smithi139.stdout:0/204: read d6/dc/dd/f24 [1853665,51989] 0 2024-09-16T00:09:40.340 INFO:tasks.workunit.client.0.smithi139.stdout:7/163: unlink d5/f27 0 2024-09-16T00:09:40.340 INFO:tasks.workunit.client.0.smithi139.stdout:6/122: chown d8/db/f1a 7 1 2024-09-16T00:09:40.340 INFO:tasks.workunit.client.0.smithi139.stdout:5/193: dread d1/d13/f17 [0,4194304] 0 2024-09-16T00:09:40.340 INFO:tasks.workunit.client.0.smithi139.stdout:0/205: chown d6/dc/l3f 2779 1 2024-09-16T00:09:40.340 INFO:tasks.workunit.client.0.smithi139.stdout:6/123: creat d8/db/f2c x:0 0 0 2024-09-16T00:09:40.341 INFO:tasks.workunit.client.0.smithi139.stdout:7/164: mkdir d5/d37 0 2024-09-16T00:09:40.341 INFO:tasks.workunit.client.0.smithi139.stdout:5/194: symlink d1/d25/d30/l40 0 2024-09-16T00:09:40.341 INFO:tasks.workunit.client.0.smithi139.stdout:0/206: creat d6/dc/dd/f40 x:0 0 0 2024-09-16T00:09:40.341 INFO:tasks.workunit.client.0.smithi139.stdout:5/195: symlink d1/d21/l41 0 2024-09-16T00:09:40.341 INFO:tasks.workunit.client.0.smithi139.stdout:7/165: creat d5/d26/f38 x:0 0 0 2024-09-16T00:09:40.341 INFO:tasks.workunit.client.0.smithi139.stdout:0/207: dwrite d6/dc/f33 [0,4194304] 0 2024-09-16T00:09:40.341 INFO:tasks.workunit.client.0.smithi139.stdout:5/196: mknod d1/d21/c42 0 2024-09-16T00:09:40.341 INFO:tasks.workunit.client.0.smithi139.stdout:5/197: dread - d1/d21/f3a zero size 2024-09-16T00:09:40.341 INFO:tasks.workunit.client.0.smithi139.stdout:0/208: mkdir d6/d1c/d41 0 2024-09-16T00:09:40.341 INFO:tasks.workunit.client.0.smithi139.stdout:6/124: link d8/d20/d27/f1c d8/d29/f2d 0 2024-09-16T00:09:40.341 INFO:tasks.workunit.client.0.smithi139.stdout:0/209: write d6/dc/f3b [952912,77834] 0 2024-09-16T00:09:40.341 INFO:tasks.workunit.client.0.smithi139.stdout:1/148: sync 2024-09-16T00:09:40.341 INFO:tasks.workunit.client.0.smithi139.stdout:9/180: sync 2024-09-16T00:09:40.341 INFO:tasks.workunit.client.0.smithi139.stdout:4/110: sync 2024-09-16T00:09:40.342 INFO:tasks.workunit.client.0.smithi139.stdout:3/170: sync 2024-09-16T00:09:40.342 INFO:tasks.workunit.client.0.smithi139.stdout:2/152: sync 2024-09-16T00:09:40.342 INFO:tasks.workunit.client.0.smithi139.stdout:8/222: sync 2024-09-16T00:09:40.342 INFO:tasks.workunit.client.0.smithi139.stdout:8/223: chown d17/d1a/d43/d40 7834128 1 2024-09-16T00:09:40.342 INFO:tasks.workunit.client.0.smithi139.stdout:2/153: mkdir d11/d30 0 2024-09-16T00:09:40.342 INFO:tasks.workunit.client.0.smithi139.stdout:2/154: readlink d11/l20 0 2024-09-16T00:09:40.342 INFO:tasks.workunit.client.0.smithi139.stdout:1/149: dwrite d7/d8/f1e [0,4194304] 0 2024-09-16T00:09:40.342 INFO:tasks.workunit.client.0.smithi139.stdout:3/171: rename d5/d7/d23/f11 to d5/d29/f32 0 2024-09-16T00:09:40.342 INFO:tasks.workunit.client.0.smithi139.stdout:1/150: rename d7/d1b to d7/d1b/d29 22 2024-09-16T00:09:40.342 INFO:tasks.workunit.client.0.smithi139.stdout:7/166: link d5/d16/l17 d5/d16/d22/d25/d2d/l39 0 2024-09-16T00:09:40.342 INFO:tasks.workunit.client.0.smithi139.stdout:5/198: write d1/d21/f39 [740515,43587] 0 2024-09-16T00:09:40.343 INFO:tasks.workunit.client.0.smithi139.stdout:4/111: unlink fc 0 2024-09-16T00:09:40.343 INFO:tasks.workunit.client.0.smithi139.stdout:1/151: fdatasync d7/f12 0 2024-09-16T00:09:40.343 INFO:tasks.workunit.client.0.smithi139.stdout:9/181: mkdir d19/d1a/d20/d23/d2d/d2f 0 2024-09-16T00:09:40.344 INFO:tasks.workunit.client.0.smithi139.stdout:6/125: write d8/d29/f2d [82548,708] 0 2024-09-16T00:09:40.346 INFO:tasks.workunit.client.0.smithi139.stdout:3/172: dread d5/d7/f26 [0,4194304] 0 2024-09-16T00:09:40.346 INFO:tasks.workunit.client.0.smithi139.stdout:2/155: dwrite f2 [4194304,4194304] 0 2024-09-16T00:09:40.346 INFO:tasks.workunit.client.0.smithi139.stdout:5/199: unlink d1/d25/f2d 0 2024-09-16T00:09:40.347 INFO:tasks.workunit.client.0.smithi139.stdout:9/182: chown f6 3147808 1 2024-09-16T00:09:40.348 INFO:tasks.workunit.client.0.smithi139.stdout:5/200: readlink d1/lc 0 2024-09-16T00:09:40.349 INFO:tasks.workunit.client.0.smithi139.stdout:1/152: write d7/d8/d1f/f24 [1812210,36091] 0 2024-09-16T00:09:40.349 INFO:tasks.workunit.client.0.smithi139.stdout:7/167: creat d5/d16/d22/d25/d2d/f3a x:0 0 0 2024-09-16T00:09:40.352 INFO:tasks.workunit.client.0.smithi139.stdout:6/126: fsync d8/d20/d27/d10/f13 0 2024-09-16T00:09:40.352 INFO:tasks.workunit.client.0.smithi139.stdout:4/112: link d12/f17 d12/f23 0 2024-09-16T00:09:40.352 INFO:tasks.workunit.client.0.smithi139.stdout:3/173: chown d5/c1e 52707698 1 2024-09-16T00:09:40.353 INFO:tasks.workunit.client.0.smithi139.stdout:0/210: dwrite d6/d1c/f35 [0,4194304] 0 2024-09-16T00:09:40.353 INFO:tasks.workunit.client.0.smithi139.stdout:2/156: read - d11/d1f/d29/d2b/f2d zero size 2024-09-16T00:09:40.356 INFO:tasks.workunit.client.0.smithi139.stdout:2/157: stat d11/c25 0 2024-09-16T00:09:40.356 INFO:tasks.workunit.client.0.smithi139.stdout:1/153: write d7/d8/f1e [1134062,58980] 0 2024-09-16T00:09:40.356 INFO:tasks.workunit.client.0.smithi139.stdout:5/201: mkdir d1/d25/d30/d33/d43 0 2024-09-16T00:09:40.357 INFO:tasks.workunit.client.0.smithi139.stdout:0/211: stat d6/c2f 0 2024-09-16T00:09:40.357 INFO:tasks.workunit.client.0.smithi139.stdout:3/174: chown d5/c1e 210 1 2024-09-16T00:09:40.358 INFO:tasks.workunit.client.0.smithi139.stdout:2/158: stat cb 0 2024-09-16T00:09:40.360 INFO:tasks.workunit.client.0.smithi139.stdout:8/224: dwrite f4 [0,4194304] 0 2024-09-16T00:09:40.360 INFO:tasks.workunit.client.0.smithi139.stdout:5/202: chown d1/cd 2 1 2024-09-16T00:09:40.361 INFO:tasks.workunit.client.0.smithi139.stdout:2/159: read d11/f18 [1488027,99544] 0 2024-09-16T00:09:40.365 INFO:tasks.workunit.client.0.smithi139.stdout:0/212: mkdir d6/d42 0 2024-09-16T00:09:40.367 INFO:tasks.workunit.client.0.smithi139.stdout:2/160: chown d11/f18 100054 1 2024-09-16T00:09:40.368 INFO:tasks.workunit.client.0.smithi139.stdout:7/168: link d5/d26/f2b d5/d16/d22/d25/d2d/f3b 0 2024-09-16T00:09:40.369 INFO:tasks.workunit.client.0.smithi139.stdout:9/183: getdents d19/d1a/d20/d23 0 2024-09-16T00:09:40.370 INFO:tasks.workunit.client.0.smithi139.stdout:2/161: readlink d11/l17 0 2024-09-16T00:09:40.371 INFO:tasks.workunit.client.0.smithi139.stdout:4/113: truncate fd 1878993 0 2024-09-16T00:09:40.372 INFO:tasks.workunit.client.0.smithi139.stdout:5/203: dread d1/fa [0,4194304] 0 2024-09-16T00:09:40.374 INFO:tasks.workunit.client.0.smithi139.stdout:6/127: dwrite d8/db/f2c [0,4194304] 0 2024-09-16T00:09:40.381 INFO:tasks.workunit.client.0.smithi139.stdout:3/175: dwrite d5/d7/d20/f24 [0,4194304] 0 2024-09-16T00:09:40.382 INFO:tasks.workunit.client.0.smithi139.stdout:1/154: dwrite d7/f12 [0,4194304] 0 2024-09-16T00:09:40.382 INFO:tasks.workunit.client.0.smithi139.stdout:7/169: write d5/d16/d22/d25/d2d/f36 [490138,86312] 0 2024-09-16T00:09:40.385 INFO:tasks.workunit.client.0.smithi139.stdout:2/162: rename d11/f18 to d11/d1f/d29/d2b/f31 0 2024-09-16T00:09:40.387 INFO:tasks.workunit.client.0.smithi139.stdout:5/204: creat d1/d13/d3b/f44 x:0 0 0 2024-09-16T00:09:40.387 INFO:tasks.workunit.client.0.smithi139.stdout:0/213: creat d6/dc/f43 x:0 0 0 2024-09-16T00:09:40.389 INFO:tasks.workunit.client.0.smithi139.stdout:8/225: dwrite d17/d1a/d43/f2b [0,4194304] 0 2024-09-16T00:09:40.389 INFO:tasks.workunit.client.0.smithi139.stdout:3/176: fsync d5/d29/f32 0 2024-09-16T00:09:40.389 INFO:tasks.workunit.client.0.smithi139.stdout:7/170: write d5/f1f [652923,98338] 0 2024-09-16T00:09:40.390 INFO:tasks.workunit.client.0.smithi139.stdout:1/155: mknod d7/d8/d1f/c2a 0 2024-09-16T00:09:40.392 INFO:tasks.workunit.client.0.smithi139.stdout:4/114: rename d12/f17 to d12/d21/f24 0 2024-09-16T00:09:40.393 INFO:tasks.workunit.client.0.smithi139.stdout:6/128: truncate f6 2260901 0 2024-09-16T00:09:40.394 INFO:tasks.workunit.client.0.smithi139.stdout:3/177: read d5/d7/f13 [346436,107712] 0 2024-09-16T00:09:40.394 INFO:tasks.workunit.client.0.smithi139.stdout:8/226: stat d17/d1a/d2d/f2e 0 2024-09-16T00:09:40.396 INFO:tasks.workunit.client.0.smithi139.stdout:0/214: fdatasync d6/d1c/f28 0 2024-09-16T00:09:40.398 INFO:tasks.workunit.client.0.smithi139.stdout:8/227: stat d17/d1a/d43/c47 0 2024-09-16T00:09:40.398 INFO:tasks.workunit.client.0.smithi139.stdout:9/184: dwrite f14 [0,4194304] 0 2024-09-16T00:09:40.400 INFO:tasks.workunit.client.0.smithi139.stdout:8/228: readlink d17/d1a/d43/l4e 0 2024-09-16T00:09:40.402 INFO:tasks.workunit.client.0.smithi139.stdout:3/178: write d5/d7/dc/f12 [2125277,123821] 0 2024-09-16T00:09:40.403 INFO:tasks.workunit.client.0.smithi139.stdout:1/156: mknod d7/d1b/c2b 0 2024-09-16T00:09:40.404 INFO:tasks.workunit.client.0.smithi139.stdout:6/129: creat d8/d20/d27/d10/f2e x:0 0 0 2024-09-16T00:09:40.404 INFO:tasks.workunit.client.0.smithi139.stdout:7/171: dwrite d5/f1f [0,4194304] 0 2024-09-16T00:09:40.405 INFO:tasks.workunit.client.0.smithi139.stdout:8/229: read d17/d1a/d43/d40/f4b [448225,87976] 0 2024-09-16T00:09:40.407 INFO:tasks.workunit.client.0.smithi139.stdout:2/163: dread f2 [0,4194304] 0 2024-09-16T00:09:40.407 INFO:tasks.workunit.client.0.smithi139.stdout:3/179: rename d5/d7 to d5/d7/d23/d27/d33 22 2024-09-16T00:09:40.407 INFO:tasks.workunit.client.0.smithi139.stdout:9/185: creat d19/d1a/d20/d23/f30 x:0 0 0 2024-09-16T00:09:40.407 INFO:tasks.workunit.client.0.smithi139.stdout:6/130: dread - d8/d20/d27/d10/f2e zero size 2024-09-16T00:09:40.407 INFO:tasks.workunit.client.0.smithi139.stdout:0/215: write d6/dc/f3e [535086,59069] 0 2024-09-16T00:09:40.409 INFO:tasks.workunit.client.0.smithi139.stdout:2/164: write d11/d1f/f24 [602542,20084] 0 2024-09-16T00:09:40.409 INFO:tasks.workunit.client.0.smithi139.stdout:9/186: write d19/f1d [5197303,118535] 0 2024-09-16T00:09:40.409 INFO:tasks.workunit.client.0.smithi139.stdout:0/216: stat d6/dc/f33 0 2024-09-16T00:09:40.410 INFO:tasks.workunit.client.0.smithi139.stdout:4/115: rename d12/f1e to d12/f25 0 2024-09-16T00:09:40.411 INFO:tasks.workunit.client.0.smithi139.stdout:5/205: getdents d1 0 2024-09-16T00:09:40.412 INFO:tasks.workunit.client.0.smithi139.stdout:0/217: readlink d6/dc/dd/l12 0 2024-09-16T00:09:40.412 INFO:tasks.workunit.client.0.smithi139.stdout:5/206: chown d1/f2f 1398 1 2024-09-16T00:09:40.416 INFO:tasks.workunit.client.0.smithi139.stdout:3/180: mknod d5/d7/d23/d27/d2f/c34 0 2024-09-16T00:09:40.418 INFO:tasks.workunit.client.0.smithi139.stdout:3/181: read d5/d7/d23/f15 [1516584,64753] 0 2024-09-16T00:09:40.419 INFO:tasks.workunit.client.0.smithi139.stdout:3/182: fdatasync d5/d7/d20/f24 0 2024-09-16T00:09:40.419 INFO:tasks.workunit.client.0.smithi139.stdout:9/187: creat d19/f31 x:0 0 0 2024-09-16T00:09:40.421 INFO:tasks.workunit.client.0.smithi139.stdout:2/165: mkdir d11/d32 0 2024-09-16T00:09:40.421 INFO:tasks.workunit.client.0.smithi139.stdout:2/166: readlink d11/l17 0 2024-09-16T00:09:40.421 INFO:tasks.workunit.client.0.smithi139.stdout:2/167: fsync d11/d1f/f27 0 2024-09-16T00:09:40.421 INFO:tasks.workunit.client.0.smithi139.stdout:1/157: truncate d7/fc 1149286 0 2024-09-16T00:09:40.422 INFO:tasks.workunit.client.0.smithi139.stdout:7/172: fsync d5/f1c 0 2024-09-16T00:09:40.424 INFO:tasks.workunit.client.0.smithi139.stdout:5/207: rmdir d1/d13/d22 39 2024-09-16T00:09:40.426 INFO:tasks.workunit.client.0.smithi139.stdout:0/218: rename d6/dc to d6/d1c/d41/d44 0 2024-09-16T00:09:40.427 INFO:tasks.workunit.client.0.smithi139.stdout:3/183: rmdir d5/d7/d23/d27 39 2024-09-16T00:09:40.428 INFO:tasks.workunit.client.0.smithi139.stdout:3/184: stat d5/d29/l2c 0 2024-09-16T00:09:40.428 INFO:tasks.workunit.client.0.smithi139.stdout:9/188: mkdir d19/d22/d32 0 2024-09-16T00:09:40.429 INFO:tasks.workunit.client.0.smithi139.stdout:9/189: chown d19/f26 98 1 2024-09-16T00:09:40.432 INFO:tasks.workunit.client.0.smithi139.stdout:4/116: link d12/d15/f18 d12/d15/f26 0 2024-09-16T00:09:40.433 INFO:tasks.workunit.client.0.smithi139.stdout:5/208: unlink d1/d25/f37 0 2024-09-16T00:09:40.433 INFO:tasks.workunit.client.0.smithi139.stdout:5/209: stat d1/d25/d30/l26 0 2024-09-16T00:09:40.434 INFO:tasks.workunit.client.0.smithi139.stdout:8/230: creat d17/f56 x:0 0 0 2024-09-16T00:09:40.434 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:40.433+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:40.434 INFO:tasks.workunit.client.0.smithi139.stdout:0/219: stat d6/l1f 0 2024-09-16T00:09:40.435 INFO:tasks.workunit.client.0.smithi139.stdout:7/173: rename d5/c35 to d5/d16/d22/c3c 0 2024-09-16T00:09:40.435 INFO:tasks.workunit.client.0.smithi139.stdout:6/131: link d8/d20/d27/cd d8/db/d26/d28/c2f 0 2024-09-16T00:09:40.436 INFO:tasks.workunit.client.0.smithi139.stdout:6/132: chown d8/db/f2c 322491 1 2024-09-16T00:09:40.436 INFO:tasks.workunit.client.0.smithi139.stdout:6/133: chown d8/d20/d27/cd 18 1 2024-09-16T00:09:40.437 INFO:tasks.workunit.client.0.smithi139.stdout:9/190: rmdir d19/d1a/d20/d23/d2d 39 2024-09-16T00:09:40.438 INFO:tasks.workunit.client.0.smithi139.stdout:9/191: chown d19/f1f 125551 1 2024-09-16T00:09:40.439 INFO:tasks.workunit.client.0.smithi139.stdout:4/117: rmdir d12/d21 39 2024-09-16T00:09:40.441 INFO:tasks.workunit.client.0.smithi139.stdout:7/174: unlink d5/d26/f38 0 2024-09-16T00:09:40.441 INFO:tasks.workunit.client.0.smithi139.stdout:7/175: fdatasync d5/f15 0 2024-09-16T00:09:40.442 INFO:tasks.workunit.client.0.smithi139.stdout:7/176: readlink d5/d16/l19 0 2024-09-16T00:09:40.443 INFO:tasks.workunit.client.0.smithi139.stdout:0/220: dread d6/d1c/d41/d44/dd/f24 [0,4194304] 0 2024-09-16T00:09:40.444 INFO:tasks.workunit.client.0.smithi139.stdout:7/177: write d5/d26/f32 [186092,25316] 0 2024-09-16T00:09:40.445 INFO:tasks.workunit.client.0.smithi139.stdout:6/134: dread d8/d29/f2d [0,4194304] 0 2024-09-16T00:09:40.445 INFO:tasks.workunit.client.0.smithi139.stdout:6/135: stat d8/c14 0 2024-09-16T00:09:40.445 INFO:tasks.workunit.client.0.smithi139.stdout:6/136: chown d8/cf 44 1 2024-09-16T00:09:40.446 INFO:tasks.workunit.client.0.smithi139.stdout:2/168: truncate f9 6531972 0 2024-09-16T00:09:40.447 INFO:tasks.workunit.client.0.smithi139.stdout:6/137: dread - d8/d20/d27/d10/f2e zero size 2024-09-16T00:09:40.449 INFO:tasks.workunit.client.0.smithi139.stdout:3/185: truncate d5/f8 1395527 0 2024-09-16T00:09:40.450 INFO:tasks.workunit.client.0.smithi139.stdout:1/158: dwrite d7/d8/d1f/f20 [0,4194304] 0 2024-09-16T00:09:40.453 INFO:tasks.workunit.client.0.smithi139.stdout:5/210: symlink d1/d13/d22/l45 0 2024-09-16T00:09:40.454 INFO:tasks.workunit.client.0.smithi139.stdout:5/211: fsync d1/d13/d3b/f44 0 2024-09-16T00:09:40.455 INFO:tasks.workunit.client.0.smithi139.stdout:9/192: rename d19/d22/f25 to d19/d1a/d20/d23/f33 0 2024-09-16T00:09:40.456 INFO:tasks.workunit.client.0.smithi139.stdout:9/193: truncate d19/f26 791677 0 2024-09-16T00:09:40.457 INFO:tasks.workunit.client.0.smithi139.stdout:7/178: mkdir d5/d16/d3d 0 2024-09-16T00:09:40.459 INFO:tasks.workunit.client.0.smithi139.stdout:3/186: write d5/d7/fd [1551373,105661] 0 2024-09-16T00:09:40.460 INFO:tasks.workunit.client.0.smithi139.stdout:8/231: getdents d17/d1a/d43 0 2024-09-16T00:09:40.463 INFO:tasks.workunit.client.0.smithi139.stdout:2/169: creat d11/d2f/f33 x:0 0 0 2024-09-16T00:09:40.464 INFO:tasks.workunit.client.0.smithi139.stdout:0/221: dwrite d6/d1c/d41/d44/dd/f19 [0,4194304] 0 2024-09-16T00:09:40.466 INFO:tasks.workunit.client.0.smithi139.stdout:5/212: mknod d1/d25/c46 0 2024-09-16T00:09:40.468 INFO:tasks.workunit.client.0.smithi139.stdout:4/118: rename d12/c22 to d12/d15/c27 0 2024-09-16T00:09:40.469 INFO:tasks.workunit.client.0.smithi139.stdout:4/119: dread - d12/d15/f18 zero size 2024-09-16T00:09:40.471 INFO:tasks.workunit.client.0.smithi139.stdout:0/222: dwrite d6/d1c/d41/d44/f3b [0,4194304] 0 2024-09-16T00:09:40.473 INFO:tasks.workunit.client.0.smithi139.stdout:6/138: link d8/db/d26/d28/c2f d8/d20/d27/c30 0 2024-09-16T00:09:40.474 INFO:tasks.workunit.client.0.smithi139.stdout:1/159: dwrite d7/f28 [0,4194304] 0 2024-09-16T00:09:40.479 INFO:tasks.workunit.client.0.smithi139.stdout:0/223: mkdir d6/d1c/d45 0 2024-09-16T00:09:40.482 INFO:tasks.workunit.client.0.smithi139.stdout:6/139: dwrite d8/db/f2c [0,4194304] 0 2024-09-16T00:09:40.482 INFO:tasks.workunit.client.0.smithi139.stdout:5/213: link d1/d13/d3b/f44 d1/d13/d3b/f47 0 2024-09-16T00:09:40.482 INFO:tasks.workunit.client.0.smithi139.stdout:0/224: write d6/d1c/d41/d44/dd/f19 [2461051,5003] 0 2024-09-16T00:09:40.484 INFO:tasks.workunit.client.0.smithi139.stdout:4/120: mknod d12/d21/c28 0 2024-09-16T00:09:40.486 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:09:40.487 INFO:tasks.workunit.client.0.smithi139.stdout:1/160: creat d7/d8/d27/f2c x:0 0 0 2024-09-16T00:09:40.487 INFO:tasks.workunit.client.0.smithi139.stdout:1/161: dread - d7/d8/f15 zero size 2024-09-16T00:09:40.489 INFO:tasks.workunit.client.0.smithi139.stdout:8/232: getdents d17/d1f/d3a 0 2024-09-16T00:09:40.490 INFO:tasks.workunit.client.0.smithi139.stdout:2/170: getdents d11 0 2024-09-16T00:09:40.491 INFO:tasks.workunit.client.0.smithi139.stdout:7/179: rename d5/f15 to d5/d16/f3e 0 2024-09-16T00:09:40.491 INFO:tasks.workunit.client.0.smithi139.stdout:1/162: write d7/ff [925684,128166] 0 2024-09-16T00:09:40.493 INFO:tasks.workunit.client.0.smithi139.stdout:0/225: creat d6/d1c/d41/d44/dd/f46 x:0 0 0 2024-09-16T00:09:40.494 INFO:tasks.workunit.client.0.smithi139.stdout:5/214: write d1/d13/d3b/f47 [134682,119915] 0 2024-09-16T00:09:40.502 INFO:tasks.workunit.client.0.smithi139.stdout:4/121: chown d12/d15/c27 119 1 2024-09-16T00:09:40.503 INFO:tasks.workunit.client.0.smithi139.stdout:2/171: creat d11/d1f/d29/d2b/f34 x:0 0 0 2024-09-16T00:09:40.503 INFO:tasks.workunit.client.0.smithi139.stdout:5/215: read d1/d13/f17 [369676,88400] 0 2024-09-16T00:09:40.503 INFO:tasks.workunit.client.0.smithi139.stdout:9/194: rename d19/d1a/d20/d23/f24 to d19/d1a/d20/d23/d2c/f34 0 2024-09-16T00:09:40.505 INFO:tasks.workunit.client.0.smithi139.stdout:8/233: dread d17/d54/f55 [0,4194304] 0 2024-09-16T00:09:40.505 INFO:tasks.workunit.client.0.smithi139.stdout:2/172: symlink d11/d1f/l35 0 2024-09-16T00:09:40.505 INFO:tasks.workunit.client.0.smithi139.stdout:6/140: dread f3 [0,4194304] 0 2024-09-16T00:09:40.506 INFO:tasks.workunit.client.0.smithi139.stdout:7/180: mknod d5/d37/c3f 0 2024-09-16T00:09:40.508 INFO:tasks.workunit.client.0.smithi139.stdout:1/163: rename d7/ce to d7/d8/d27/c2d 0 2024-09-16T00:09:40.509 INFO:tasks.workunit.client.0.smithi139.stdout:0/226: dread d6/d1c/d41/d44/f3e [0,4194304] 0 2024-09-16T00:09:40.510 INFO:tasks.workunit.client.0.smithi139.stdout:1/164: chown d7/f13 21 1 2024-09-16T00:09:40.511 INFO:tasks.workunit.client.0.smithi139.stdout:5/216: dwrite d1/d13/d3b/f47 [0,4194304] 0 2024-09-16T00:09:40.512 INFO:tasks.workunit.client.0.smithi139.stdout:8/234: mkdir d17/d1a/d57 0 2024-09-16T00:09:40.513 INFO:tasks.workunit.client.0.smithi139.stdout:7/181: fdatasync d5/f1c 0 2024-09-16T00:09:40.514 INFO:tasks.workunit.client.0.smithi139.stdout:6/141: creat d8/db/d26/d28/f31 x:0 0 0 2024-09-16T00:09:40.515 INFO:tasks.workunit.client.0.smithi139.stdout:9/195: mkdir d19/d1a/d20/d23/d2d/d35 0 2024-09-16T00:09:40.516 INFO:tasks.workunit.client.0.smithi139.stdout:2/173: creat d11/d30/f36 x:0 0 0 2024-09-16T00:09:40.519 INFO:tasks.workunit.client.0.smithi139.stdout:1/165: unlink d7/f1d 0 2024-09-16T00:09:40.522 INFO:tasks.workunit.client.0.smithi139.stdout:5/217: creat d1/f48 x:0 0 0 2024-09-16T00:09:40.523 INFO:tasks.workunit.client.0.smithi139.stdout:5/218: chown d1/l35 1968 1 2024-09-16T00:09:40.525 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:40.525 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:40.525 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:40.525 INFO:tasks.workunit.client.0.smithi139.stdout:7/182: rename d5/f2e to d5/d37/f40 0 2024-09-16T00:09:40.526 INFO:tasks.workunit.client.0.smithi139.stdout:5/219: dread d1/d13/f17 [0,4194304] 0 2024-09-16T00:09:40.527 INFO:tasks.workunit.client.0.smithi139.stdout:5/220: truncate d1/d25/f36 822536 0 2024-09-16T00:09:40.527 INFO:tasks.workunit.client.0.smithi139.stdout:0/227: creat d6/d1c/d45/f47 x:0 0 0 2024-09-16T00:09:40.528 INFO:tasks.workunit.client.0.smithi139.stdout:8/235: unlink c16 0 2024-09-16T00:09:40.529 INFO:tasks.workunit.client.0.smithi139.stdout:4/122: getdents d12/d21 0 2024-09-16T00:09:40.529 INFO:tasks.workunit.client.0.smithi139.stdout:9/196: symlink d19/d22/d32/l36 0 2024-09-16T00:09:40.530 INFO:tasks.workunit.client.0.smithi139.stdout:9/197: write d19/f1b [273836,101300] 0 2024-09-16T00:09:40.531 INFO:tasks.workunit.client.0.smithi139.stdout:1/166: write d7/d8/f9 [301224,79364] 0 2024-09-16T00:09:40.531 INFO:tasks.workunit.client.0.smithi139.stdout:9/198: chown d19/d1a/f21 385663 1 2024-09-16T00:09:40.532 INFO:tasks.workunit.client.0.smithi139.stdout:1/167: write d7/f26 [831447,45870] 0 2024-09-16T00:09:40.532 INFO:tasks.workunit.client.0.smithi139.stdout:9/199: read d19/d1a/d20/d23/f33 [1061860,65211] 0 2024-09-16T00:09:40.533 INFO:tasks.workunit.client.0.smithi139.stdout:8/236: creat d17/d1f/d3a/f58 x:0 0 0 2024-09-16T00:09:40.534 INFO:tasks.workunit.client.0.smithi139.stdout:7/183: mknod d5/c41 0 2024-09-16T00:09:40.534 INFO:tasks.workunit.client.0.smithi139.stdout:1/168: write d7/d8/d1f/f24 [2615906,124822] 0 2024-09-16T00:09:40.535 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:40.533+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:40.535 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:09:40.536 INFO:tasks.workunit.client.0.smithi139.stdout:5/221: mkdir d1/d21/d49 0 2024-09-16T00:09:40.536 INFO:tasks.workunit.client.0.smithi139.stdout:0/228: creat d6/d42/f48 x:0 0 0 2024-09-16T00:09:40.537 INFO:tasks.workunit.client.0.smithi139.stdout:7/184: write d5/d16/d22/d25/f2c [2772270,30542] 0 2024-09-16T00:09:40.538 INFO:tasks.workunit.client.0.smithi139.stdout:0/229: chown d6/d1c/d45 67679219 1 2024-09-16T00:09:40.540 INFO:tasks.workunit.client.0.smithi139.stdout:8/237: creat d17/d1a/f59 x:0 0 0 2024-09-16T00:09:40.541 INFO:tasks.workunit.client.0.smithi139.stdout:4/123: rename d12/d15/c27 to d12/c29 0 2024-09-16T00:09:40.542 INFO:tasks.workunit.client.0.smithi139.stdout:8/238: write f7 [7527384,23659] 0 2024-09-16T00:09:40.543 INFO:tasks.workunit.client.0.smithi139.stdout:7/185: mkdir d5/d26/d42 0 2024-09-16T00:09:40.543 INFO:tasks.workunit.client.0.smithi139.stdout:9/200: dwrite d19/f1b [0,4194304] 0 2024-09-16T00:09:40.544 INFO:tasks.workunit.client.0.smithi139.stdout:0/230: write d6/d1c/d41/d44/dd/f2d [1292680,19916] 0 2024-09-16T00:09:40.544 INFO:tasks.workunit.client.0.smithi139.stdout:2/174: truncate fc 2058066 0 2024-09-16T00:09:40.548 INFO:tasks.workunit.client.0.smithi139.stdout:7/186: chown d5/d37/f40 16517918 1 2024-09-16T00:09:40.548 INFO:tasks.workunit.client.0.smithi139.stdout:1/169: dwrite d7/d8/d27/f2c [0,4194304] 0 2024-09-16T00:09:40.549 INFO:tasks.workunit.client.0.smithi139.stdout:8/239: symlink d17/d1a/d2d/l5a 0 2024-09-16T00:09:40.549 INFO:tasks.workunit.client.0.smithi139.stdout:2/175: chown f2 171708 1 2024-09-16T00:09:40.550 INFO:tasks.workunit.client.0.smithi139.stdout:0/231: mkdir d6/d1c/d45/d49 0 2024-09-16T00:09:40.550 INFO:tasks.workunit.client.0.smithi139.stdout:8/240: fdatasync d17/f19 0 2024-09-16T00:09:40.553 INFO:tasks.workunit.client.0.smithi139.stdout:9/201: dread d19/d1a/d20/d23/d2c/f34 [0,4194304] 0 2024-09-16T00:09:40.553 INFO:tasks.workunit.client.0.smithi139.stdout:0/232: truncate d6/d1c/d41/d44/dd/f19 4933164 0 2024-09-16T00:09:40.554 INFO:tasks.workunit.client.0.smithi139.stdout:4/124: link f4 d12/f2a 0 2024-09-16T00:09:40.559 INFO:tasks.workunit.client.0.smithi139.stdout:0/233: dread d6/d1c/f35 [0,4194304] 0 2024-09-16T00:09:40.560 INFO:tasks.workunit.client.0.smithi139.stdout:4/125: rmdir d12/d21 39 2024-09-16T00:09:40.560 INFO:tasks.workunit.client.0.smithi139.stdout:7/187: write f4 [416461,121630] 0 2024-09-16T00:09:40.561 INFO:tasks.workunit.client.0.smithi139.stdout:1/170: symlink d7/l2e 0 2024-09-16T00:09:40.563 INFO:tasks.workunit.client.0.smithi139.stdout:9/202: getdents d19/d1a/d20/d23/d2d/d2f 0 2024-09-16T00:09:40.563 INFO:tasks.workunit.client.0.smithi139.stdout:4/126: truncate d12/f2a 131689 0 2024-09-16T00:09:40.564 INFO:tasks.workunit.client.0.smithi139.stdout:0/234: symlink d6/l4a 0 2024-09-16T00:09:40.564 INFO:tasks.workunit.client.0.smithi139.stdout:9/203: chown d19/d1a/d20/d23/f33 17567 1 2024-09-16T00:09:40.565 INFO:tasks.workunit.client.0.smithi139.stdout:7/188: mknod d5/d26/c43 0 2024-09-16T00:09:40.566 INFO:tasks.workunit.client.0.smithi139.stdout:8/241: dread d17/d1a/d2d/f36 [0,4194304] 0 2024-09-16T00:09:40.567 INFO:tasks.workunit.client.0.smithi139.stdout:1/171: mkdir d7/d8/d27/d2f 0 2024-09-16T00:09:40.571 INFO:tasks.workunit.client.0.smithi139.stdout:2/176: link f9 d11/f37 0 2024-09-16T00:09:40.573 INFO:tasks.workunit.client.0.smithi139.stdout:9/204: dwrite d19/f26 [0,4194304] 0 2024-09-16T00:09:40.574 INFO:tasks.workunit.client.0.smithi139.stdout:2/177: write d11/d1f/f27 [132474,26060] 0 2024-09-16T00:09:40.575 INFO:tasks.workunit.client.0.smithi139.stdout:8/242: mknod d17/c5b 0 2024-09-16T00:09:40.577 INFO:tasks.workunit.client.0.smithi139.stdout:9/205: stat d19/d1a/d20/d23/d2d/d2f 0 2024-09-16T00:09:40.578 INFO:tasks.workunit.client.0.smithi139.stdout:1/172: dwrite d7/d8/d1f/f24 [0,4194304] 0 2024-09-16T00:09:40.579 INFO:tasks.workunit.client.0.smithi139.stdout:4/127: write d12/d21/f24 [81665,36025] 0 2024-09-16T00:09:40.580 INFO:tasks.workunit.client.0.smithi139.stdout:2/178: chown d11/d1f/d29/d2b/f34 2381047 1 2024-09-16T00:09:40.580 INFO:tasks.workunit.client.0.smithi139.stdout:7/189: write d5/d26/f2b [19145,78212] 0 2024-09-16T00:09:40.582 INFO:tasks.workunit.client.0.smithi139.stdout:1/173: rmdir d7 39 2024-09-16T00:09:40.582 INFO:tasks.workunit.client.0.smithi139.stdout:2/179: chown ff 1019750430 1 2024-09-16T00:09:40.583 INFO:tasks.workunit.client.0.smithi139.stdout:8/243: write d17/d1a/d2d/f2e [1748321,27612] 0 2024-09-16T00:09:40.584 INFO:tasks.workunit.client.0.smithi139.stdout:4/128: chown f1 0 1 2024-09-16T00:09:40.586 INFO:tasks.workunit.client.0.smithi139.stdout:7/190: mkdir d5/d16/d3d/d44 0 2024-09-16T00:09:40.588 INFO:tasks.workunit.client.0.smithi139.stdout:2/180: unlink d11/l1d 0 2024-09-16T00:09:40.589 INFO:tasks.workunit.client.0.smithi139.stdout:0/235: dwrite d6/d1c/f28 [4194304,4194304] 0 2024-09-16T00:09:40.589 INFO:tasks.workunit.client.0.smithi139.stdout:7/191: write f4 [1043738,2761] 0 2024-09-16T00:09:40.590 INFO:tasks.workunit.client.0.smithi139.stdout:9/206: truncate d19/f1d 2042472 0 2024-09-16T00:09:40.593 INFO:tasks.workunit.client.0.smithi139.stdout:9/207: dread - d19/f31 zero size 2024-09-16T00:09:40.594 INFO:tasks.workunit.client.0.smithi139.stdout:2/181: write fe [1362432,34839] 0 2024-09-16T00:09:40.594 INFO:tasks.workunit.client.0.smithi139.stdout:1/174: dread d7/d8/f1c [0,4194304] 0 2024-09-16T00:09:40.597 INFO:tasks.workunit.client.0.smithi139.stdout:7/192: write d5/d16/f3e [927914,81166] 0 2024-09-16T00:09:40.598 INFO:tasks.workunit.client.0.smithi139.stdout:4/129: rename d12/d15 to d12/d2b 0 2024-09-16T00:09:40.600 INFO:tasks.workunit.client.0.smithi139.stdout:2/182: creat d11/f38 x:0 0 0 2024-09-16T00:09:40.601 INFO:tasks.workunit.client.0.smithi139.stdout:8/244: link d17/l52 d17/d1a/d43/d40/d4f/l5c 0 2024-09-16T00:09:40.601 INFO:tasks.workunit.client.0.smithi139.stdout:8/245: chown d17 57 1 2024-09-16T00:09:40.603 INFO:tasks.workunit.client.0.smithi139.stdout:1/175: rename d7/d8/f15 to d7/d8/d27/d2f/f30 0 2024-09-16T00:09:40.603 INFO:tasks.workunit.client.0.smithi139.stdout:1/176: readlink d7/l2e 0 2024-09-16T00:09:40.604 INFO:tasks.workunit.client.0.smithi139.stdout:7/193: unlink d5/d16/d22/d25/f2c 0 2024-09-16T00:09:40.605 INFO:tasks.workunit.client.0.smithi139.stdout:4/130: readlink d12/l1f 0 2024-09-16T00:09:40.606 INFO:tasks.workunit.client.0.smithi139.stdout:0/236: dwrite d6/d1c/f36 [0,4194304] 0 2024-09-16T00:09:40.606 INFO:tasks.workunit.client.0.smithi139.stdout:9/208: link fb d19/d22/f37 0 2024-09-16T00:09:40.608 INFO:tasks.workunit.client.0.smithi139.stdout:2/183: mkdir d11/d30/d39 0 2024-09-16T00:09:40.608 INFO:tasks.workunit.client.0.smithi139.stdout:1/177: mknod d7/d1b/c31 0 2024-09-16T00:09:40.609 INFO:tasks.workunit.client.0.smithi139.stdout:8/246: creat d17/d1a/d43/d40/f5d x:0 0 0 2024-09-16T00:09:40.609 INFO:tasks.workunit.client.0.smithi139.stdout:1/178: readlink d7/l2e 0 2024-09-16T00:09:40.610 INFO:tasks.workunit.client.0.smithi139.stdout:4/131: chown fd 1256205 1 2024-09-16T00:09:40.610 INFO:tasks.workunit.client.0.smithi139.stdout:8/247: creat d17/d1a/f5e x:0 0 0 2024-09-16T00:09:40.611 INFO:tasks.workunit.client.0.smithi139.stdout:1/179: write d7/d8/f9 [1411317,31677] 0 2024-09-16T00:09:40.611 INFO:tasks.workunit.client.0.smithi139.stdout:2/184: rename cb to d11/d1f/c3a 0 2024-09-16T00:09:40.613 INFO:tasks.workunit.client.0.smithi139.stdout:0/237: symlink d6/d1c/d41/d44/l4b 0 2024-09-16T00:09:40.613 INFO:tasks.workunit.client.0.smithi139.stdout:4/132: rmdir d12 39 2024-09-16T00:09:40.614 INFO:tasks.workunit.client.0.smithi139.stdout:4/133: dread f4 [0,4194304] 0 2024-09-16T00:09:40.615 INFO:tasks.workunit.client.0.smithi139.stdout:1/180: creat d7/d8/d27/f32 x:0 0 0 2024-09-16T00:09:40.616 INFO:tasks.workunit.client.0.smithi139.stdout:7/194: link d5/d16/l19 d5/d16/d22/d25/d2d/l45 0 2024-09-16T00:09:40.617 INFO:tasks.workunit.client.0.smithi139.stdout:1/181: dread - d7/d8/d27/f32 zero size 2024-09-16T00:09:40.619 INFO:tasks.workunit.client.0.smithi139.stdout:2/185: rmdir d11/d30/d39 0 2024-09-16T00:09:40.619 INFO:tasks.workunit.client.0.smithi139.stdout:4/134: write fe [916386,76449] 0 2024-09-16T00:09:40.620 INFO:tasks.workunit.client.0.smithi139.stdout:1/182: fsync d7/d8/d1f/f20 0 2024-09-16T00:09:40.621 INFO:tasks.workunit.client.0.smithi139.stdout:7/195: dread f4 [0,4194304] 0 2024-09-16T00:09:40.622 INFO:tasks.workunit.client.0.smithi139.stdout:8/248: dwrite fb [4194304,4194304] 0 2024-09-16T00:09:40.625 INFO:tasks.workunit.client.0.smithi139.stdout:7/196: mkdir d5/d26/d46 0 2024-09-16T00:09:40.626 INFO:tasks.workunit.client.0.smithi139.stdout:1/183: creat d7/d8/d27/d2f/f33 x:0 0 0 2024-09-16T00:09:40.627 INFO:tasks.workunit.client.0.smithi139.stdout:7/197: chown d5/d16/l24 1874110 1 2024-09-16T00:09:40.631 INFO:tasks.workunit.client.0.smithi139.stdout:1/184: unlink d7/d8/f9 0 2024-09-16T00:09:40.633 INFO:tasks.workunit.client.0.smithi139.stdout:7/198: fsync d5/d16/d22/d25/f30 0 2024-09-16T00:09:40.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.632+0000 7f45f964c640 1 Processor -- start 2024-09-16T00:09:40.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.632+0000 7f45f964c640 1 -- start start 2024-09-16T00:09:40.634 INFO:tasks.workunit.client.0.smithi139.stdout:8/249: dwrite fb [0,4194304] 0 2024-09-16T00:09:40.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.633+0000 7f45f964c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f45f40c9280 0x7f45f40c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:40.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.633+0000 7f45f964c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45f40c9730 con 0x7f45f4151b80 2024-09-16T00:09:40.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.633+0000 7f45f964c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45f40c1610 con 0x7f45f41517a0 2024-09-16T00:09:40.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.633+0000 7f45f964c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f45f40737f0 con 0x7f45f40c9280 2024-09-16T00:09:40.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.633+0000 7f45f27fc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f45f4151b80 0x7f45f40c2dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45536/0 (socket says 172.21.15.139:45536) 2024-09-16T00:09:40.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.633+0000 7f45f27fc640 1 -- 172.21.15.139:0/3489197769 learned_addr learned my addr 172.21.15.139:0/3489197769 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:40.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.634+0000 7f45f37fe640 1 --2- 172.21.15.139:0/3489197769 >> v2:172.21.15.139:3300/0 conn(0x7f45f40c9280 0x7f45f40c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:40.636 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:40.634+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:40.637 INFO:tasks.workunit.client.0.smithi139.stdout:1/185: write d7/d8/f11 [1879741,99394] 0 2024-09-16T00:09:40.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.634+0000 7f45f1ffb640 1 -- 172.21.15.139:0/3489197769 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 900843729 0 0) 0x7f45f40c9730 con 0x7f45f4151b80 2024-09-16T00:09:40.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.634+0000 7f45f37fe640 1 -- 172.21.15.139:0/3489197769 >> v1:172.21.15.164:6789/0 conn(0x7f45f41517a0 legacy=0x7f45f40c1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:40.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.634+0000 7f45f37fe640 1 -- 172.21.15.139:0/3489197769 >> v1:172.21.15.139:6789/0 conn(0x7f45f4151b80 legacy=0x7f45f40c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:40.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.634+0000 7f45f37fe640 1 -- 172.21.15.139:0/3489197769 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f40c18f0 con 0x7f45f40c9280 2024-09-16T00:09:40.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.635+0000 7f45f37fe640 1 --2- 172.21.15.139:0/3489197769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f40c9280 0x7f45f40c7b90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009d90 tx=0x7f45ec02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6ac41dc5396e04 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:40.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.635+0000 7f45f1ffb640 1 -- 172.21.15.139:0/3489197769 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec037070 con 0x7f45f40c9280 2024-09-16T00:09:40.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.635+0000 7f45f1ffb640 1 -- 172.21.15.139:0/3489197769 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45ec007740 con 0x7f45f40c9280 2024-09-16T00:09:40.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.635+0000 7f45f1ffb640 1 -- 172.21.15.139:0/3489197769 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec007b10 con 0x7f45f40c9280 2024-09-16T00:09:40.638 INFO:tasks.workunit.client.0.smithi139.stdout:7/199: creat d5/d37/f47 x:0 0 0 2024-09-16T00:09:40.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.636+0000 7f45f964c640 1 -- 172.21.15.139:0/3489197769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f40c9280 msgr2=0x7f45f40c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:40.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.636+0000 7f45f964c640 1 --2- 172.21.15.139:0/3489197769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f40c9280 0x7f45f40c7b90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009d90 tx=0x7f45ec02a780 comp rx=0 tx=0).stop 2024-09-16T00:09:40.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.636+0000 7f45f964c640 1 -- 172.21.15.139:0/3489197769 shutdown_connections 2024-09-16T00:09:40.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.636+0000 7f45f964c640 1 --2- 172.21.15.139:0/3489197769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f40c9280 0x7f45f40c7b90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:40.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.636+0000 7f45f964c640 1 -- 172.21.15.139:0/3489197769 >> 172.21.15.139:0/3489197769 conn(0x7f45f406e960 msgr2=0x7f45f406f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:40.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.636+0000 7f45f964c640 1 -- 172.21.15.139:0/3489197769 shutdown_connections 2024-09-16T00:09:40.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.637+0000 7f45f964c640 1 -- 172.21.15.139:0/3489197769 wait complete. 2024-09-16T00:09:40.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.639+0000 7f45f964c640 1 Processor -- start 2024-09-16T00:09:40.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.639+0000 7f45f964c640 1 -- start start 2024-09-16T00:09:40.641 INFO:tasks.workunit.client.0.smithi139.stdout:4/135: dread f7 [4194304,4194304] 0 2024-09-16T00:09:40.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.640+0000 7f45f964c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45f40c9280 0x7f45f414c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:40.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.640+0000 7f45f964c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45f41517a0 0x7f45f41f1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:40.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.640+0000 7f45f964c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f4151b80 0x7f45f41f1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:40.642 INFO:tasks.workunit.client.0.smithi139.stdout:8/250: getdents d17/d1a 0 2024-09-16T00:09:40.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.640+0000 7f45f964c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f45f40aec50 con 0x7f45f4151b80 2024-09-16T00:09:40.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.640+0000 7f45f37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f4151b80 0x7f45f41f1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:40.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.640+0000 7f45f2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45f40c9280 0x7f45f414c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:40.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.640+0000 7f45f37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f4151b80 0x7f45f41f1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43770/0 (socket says 172.21.15.139:43770) 2024-09-16T00:09:40.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.640+0000 7f45f2ffd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45f40c9280 0x7f45f414c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:47162/0 (socket says 172.21.15.139:47162) 2024-09-16T00:09:40.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.640+0000 7f45f37fe640 1 -- 172.21.15.139:0/1238094052 learned_addr learned my addr 172.21.15.139:0/1238094052 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:40.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.640+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45f40c13c0 con 0x7f45f40c9280 2024-09-16T00:09:40.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.640+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f45f40aedd0 con 0x7f45f41517a0 2024-09-16T00:09:40.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.640+0000 7f45f27fc640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45f41517a0 0x7f45f41f1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:40.644 INFO:tasks.workunit.client.0.smithi139.stdout:1/186: link d7/d8/f1e d7/d8/d27/f34 0 2024-09-16T00:09:40.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.641+0000 7f45f2ffd640 1 -- 172.21.15.139:0/1238094052 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45f41517a0 msgr2=0x7f45f41f1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:40.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.641+0000 7f45f2ffd640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45f41517a0 0x7f45f41f1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:40.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.641+0000 7f45f2ffd640 1 -- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f4151b80 msgr2=0x7f45f41f1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:40.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.641+0000 7f45f2ffd640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f4151b80 0x7f45f41f1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:40.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.641+0000 7f45f2ffd640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f4152c10 con 0x7f45f40c9280 2024-09-16T00:09:40.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.641+0000 7f45f27fc640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f45f41517a0 0x7f45f41f1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:40.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.641+0000 7f45f2ffd640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f45f40c9280 0x7f45f414c250 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f45e400c3f0 tx=0x7f45e400c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:40.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.641+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4017070 con 0x7f45f40c9280 2024-09-16T00:09:40.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.641+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f415fc00 con 0x7f45f40c9280 2024-09-16T00:09:40.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.641+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e40052d0 con 0x7f45f40c9280 2024-09-16T00:09:40.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.641+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4003830 con 0x7f45f40c9280 2024-09-16T00:09:40.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.642+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f41f0af0 con 0x7f45f40c9280 2024-09-16T00:09:40.647 INFO:tasks.workunit.client.0.smithi139.stdout:2/186: dwrite d11/f1a [0,4194304] 0 2024-09-16T00:09:40.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.643+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f45f40c2c70 con 0x7f45f40c9280 2024-09-16T00:09:40.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.643+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f45e401a070 con 0x7f45f40c9280 2024-09-16T00:09:40.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.644+0000 7f45eb7fe640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f45dc06c2b0 0x7f45dc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:40.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.644+0000 7f45f27fc640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f45dc06c2b0 0x7f45dc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:40.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.644+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5973+0+0 (secure 0 0 0) 0x7f45e408a790 con 0x7f45f40c9280 2024-09-16T00:09:40.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.644+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v86) ==== 54+0+16 (secure 0 0 0) 0x7f45f40c2c70 con 0x7f45f40c9280 2024-09-16T00:09:40.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.645+0000 7f45f27fc640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f45dc06c2b0 0x7f45dc06e770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f45f4155430 tx=0x7f45e0007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:40.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.645+0000 7f45f964c640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f45f415ebb0 0x7f45f414abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:40.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.645+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45f41f0cf0 con 0x7f45f415ebb0 2024-09-16T00:09:40.649 INFO:tasks.workunit.client.0.smithi139.stdout:4/136: creat d12/d21/f2c x:0 0 0 2024-09-16T00:09:40.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.645+0000 7f45f37fe640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f45f415ebb0 0x7f45f414abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:40.649 INFO:tasks.workunit.client.0.smithi139.stdout:1/187: mknod d7/d8/c35 0 2024-09-16T00:09:40.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.646+0000 7f45f37fe640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f45f415ebb0 0x7f45f414abd0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:40.650 INFO:tasks.workunit.client.0.smithi139.stdout:1/188: fdatasync d7/f18 0 2024-09-16T00:09:40.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.647+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45f41f0cf0 con 0x7f45f415ebb0 2024-09-16T00:09:40.652 INFO:tasks.workunit.client.0.smithi139.stdout:1/189: write d7/fd [4385587,118364] 0 2024-09-16T00:09:40.655 INFO:tasks.workunit.client.0.smithi139.stdout:8/251: link d17/d1a/c29 d17/d1a/d57/c5f 0 2024-09-16T00:09:40.656 INFO:tasks.workunit.client.0.smithi139.stdout:8/252: fsync d17/d1a/d2d/f36 0 2024-09-16T00:09:40.657 INFO:tasks.workunit.client.0.smithi139.stdout:8/253: readlink d17/d1a/d43/l23 0 2024-09-16T00:09:40.658 INFO:tasks.workunit.client.0.smithi139.stdout:8/254: mknod d17/d1a/d2d/c60 0 2024-09-16T00:09:40.659 INFO:tasks.workunit.client.0.smithi139.stdout:8/255: write d17/d1f/f25 [1558441,78251] 0 2024-09-16T00:09:40.659 INFO:tasks.workunit.client.0.smithi139.stdout:8/256: chown fb 1307 1 2024-09-16T00:09:40.660 INFO:tasks.workunit.client.0.smithi139.stdout:8/257: stat d17 0 2024-09-16T00:09:40.662 INFO:tasks.workunit.client.0.smithi139.stdout:4/137: truncate f7 5014003 0 2024-09-16T00:09:40.664 INFO:tasks.workunit.client.0.smithi139.stdout:1/190: chown d7/d1b 1297 1 2024-09-16T00:09:40.666 INFO:tasks.workunit.client.0.smithi139.stdout:8/258: dwrite d17/d1f/f28 [0,4194304] 0 2024-09-16T00:09:40.667 INFO:tasks.workunit.client.0.smithi139.stdout:1/191: write d7/d8/f11 [2283432,72706] 0 2024-09-16T00:09:40.668 INFO:tasks.workunit.client.0.smithi139.stdout:4/138: rename d12/l1f to d12/l2d 0 2024-09-16T00:09:40.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.667+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f45f4073f40 con 0x7f45f415ebb0 2024-09-16T00:09:40.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.669+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f45f4073f40 con 0x7f45f415ebb0 2024-09-16T00:09:40.671 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:40.672 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:40.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.670+0000 7f45f964c640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f45f415f0e0 0x7f45f40c5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:40.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.670+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f45f414b620 con 0x7f45f415f0e0 2024-09-16T00:09:40.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.670+0000 7f45f2ffd640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f45f415f0e0 0x7f45f40c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:40.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.671+0000 7f45f2ffd640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f45f415f0e0 0x7f45f40c5750 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:40.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.671+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45f414b620 con 0x7f45f415f0e0 2024-09-16T00:09:40.673 INFO:tasks.workunit.client.0.smithi139.stdout:8/259: getdents d17/d1a 0 2024-09-16T00:09:40.673 INFO:tasks.workunit.client.0.smithi139.stdout:1/192: dread d7/d8/d27/f34 [0,4194304] 0 2024-09-16T00:09:40.680 INFO:tasks.workunit.client.0.smithi139.stdout:4/139: write fd [1923569,77562] 0 2024-09-16T00:09:40.682 INFO:tasks.workunit.client.0.smithi139.stdout:4/140: creat d12/d2b/f2e x:0 0 0 2024-09-16T00:09:40.683 INFO:tasks.workunit.client.0.smithi139.stdout:8/260: truncate d17/d1f/f25 223828 0 2024-09-16T00:09:40.684 INFO:tasks.workunit.client.0.smithi139.stdout:8/261: chown d17/d1a/d2d/f3e 1710762873 1 2024-09-16T00:09:40.687 INFO:tasks.workunit.client.0.smithi139.stdout:8/262: unlink d17/d1a/f59 0 2024-09-16T00:09:40.689 INFO:tasks.workunit.client.0.smithi139.stdout:4/141: rename d12/d2b/c20 to d12/c2f 0 2024-09-16T00:09:40.689 INFO:tasks.workunit.client.0.smithi139.stdout:1/193: dread d7/fd [0,4194304] 0 2024-09-16T00:09:40.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.690+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f45f414f1f0 con 0x7f45f415f0e0 2024-09-16T00:09:40.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.692+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f45f414f1f0 con 0x7f45f415f0e0 2024-09-16T00:09:40.694 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:40.694 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:40.694 INFO:tasks.workunit.client.0.smithi139.stdout:4/142: link d12/d21/f2c d12/d2b/f30 0 2024-09-16T00:09:40.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.692+0000 7f45f964c640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f45f40c61a0 0x7f45f4062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:40.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.692+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f45f40c65a0 con 0x7f45f40c61a0 2024-09-16T00:09:40.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.692+0000 7f45f27fc640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f45f40c61a0 0x7f45f4062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:40.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.693+0000 7f45f27fc640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f45f40c61a0 0x7f45f4062630 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:40.696 INFO:tasks.workunit.client.0.smithi139.stdout:4/143: truncate d12/d21/f24 1723338 0 2024-09-16T00:09:40.696 INFO:tasks.workunit.client.0.smithi139.stdout:1/194: creat d7/d8/f36 x:0 0 0 2024-09-16T00:09:40.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.693+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45e0001c10 con 0x7f45f40c61a0 2024-09-16T00:09:40.696 INFO:tasks.workunit.client.0.smithi139.stdout:4/144: write fe [968043,121113] 0 2024-09-16T00:09:40.697 INFO:tasks.workunit.client.0.smithi139.stdout:1/195: chown d7/f25 13901450 1 2024-09-16T00:09:40.699 INFO:tasks.workunit.client.0.smithi139.stdout:1/196: chown d7/f12 0 1 2024-09-16T00:09:40.701 INFO:tasks.workunit.client.0.smithi139.stdout:1/197: creat d7/d8/d1f/f37 x:0 0 0 2024-09-16T00:09:40.704 INFO:tasks.workunit.client.0.smithi139.stdout:1/198: rmdir d7/d8/d1f 39 2024-09-16T00:09:40.704 INFO:tasks.workunit.client.0.smithi139.stdout:4/145: write d12/d2b/f18 [811130,99276] 0 2024-09-16T00:09:40.706 INFO:tasks.workunit.client.0.smithi139.stdout:8/263: dwrite d17/d1a/f1e [0,4194304] 0 2024-09-16T00:09:40.707 INFO:tasks.workunit.client.0.smithi139.stdout:1/199: write d7/f26 [940000,13822] 0 2024-09-16T00:09:40.708 INFO:tasks.workunit.client.0.smithi139.stdout:8/264: chown d17/d1f/l48 22919685 1 2024-09-16T00:09:40.708 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:40.708 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:40.708 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:40.708 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:40.708 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:40.709 INFO:tasks.workunit.client.0.smithi139.stdout:8/265: write fd [1965988,34167] 0 2024-09-16T00:09:40.710 INFO:tasks.workunit.client.0.smithi139.stdout:4/146: dread d12/d2b/f26 [0,4194304] 0 2024-09-16T00:09:40.711 INFO:tasks.workunit.client.0.smithi139.stdout:8/266: chown d17/d1a/d43/l4e 1900 1 2024-09-16T00:09:40.713 INFO:tasks.workunit.client.0.smithi139.stdout:4/147: chown d12/c2f 7572690 1 2024-09-16T00:09:40.714 INFO:tasks.workunit.client.0.smithi139.stdout:8/267: mknod d17/c61 0 2024-09-16T00:09:40.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.713+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f45f4150b70 con 0x7f45f40c61a0 2024-09-16T00:09:40.715 INFO:tasks.workunit.client.0.smithi139.stdout:8/268: write d17/d1a/d2d/f2e [3611143,52448] 0 2024-09-16T00:09:40.716 INFO:tasks.workunit.client.0.smithi139.stdout:1/200: dwrite d7/d8/f1c [0,4194304] 0 2024-09-16T00:09:40.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.716+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f45f4150b70 con 0x7f45f40c61a0 2024-09-16T00:09:40.718 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:40.718 INFO:tasks.workunit.client.0.smithi139.stdout:8/269: readlink l11 0 2024-09-16T00:09:40.719 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:40.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.716+0000 7f45f964c640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f45f4063080 0x7f45f40c3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:40.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.716+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f45f4063500 con 0x7f45f4063080 2024-09-16T00:09:40.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.716+0000 7f45f37fe640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f45f4063080 0x7f45f40c3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:40.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.717+0000 7f45f37fe640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f45f4063080 0x7f45f40c3580 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:40.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.717+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45f4063500 con 0x7f45f4063080 2024-09-16T00:09:40.721 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:09:40.722 INFO:tasks.workunit.client.0.smithi139.stdout:8/270: write d17/d54/f55 [2846007,114358] 0 2024-09-16T00:09:40.725 INFO:tasks.workunit.client.0.smithi139.stdout:1/201: rename d7/fc to d7/d8/d1f/f38 0 2024-09-16T00:09:40.727 INFO:tasks.workunit.client.0.smithi139.stdout:4/148: dwrite d12/f2a [0,4194304] 0 2024-09-16T00:09:40.729 INFO:tasks.workunit.client.0.smithi139.stdout:8/271: creat d17/f62 x:0 0 0 2024-09-16T00:09:40.732 INFO:tasks.workunit.client.0.smithi139.stdout:1/202: chown d7/d8/l23 74935900 1 2024-09-16T00:09:40.734 INFO:tasks.workunit.client.0.smithi139.stdout:3/187: sync 2024-09-16T00:09:40.734 INFO:tasks.workunit.client.0.smithi139.stdout:8/272: getdents d17/d1a/d43/d40 0 2024-09-16T00:09:40.735 INFO:tasks.workunit.client.0.smithi139.stdout:1/203: dread - d7/d8/d27/d2f/f30 zero size 2024-09-16T00:09:40.735 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:40.734+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:40.735 INFO:tasks.workunit.client.0.smithi139.stdout:4/149: dwrite f4 [0,4194304] 0 2024-09-16T00:09:40.736 INFO:tasks.workunit.client.0.smithi139.stdout:1/204: chown d7/d8/d1f/f38 1616799 1 2024-09-16T00:09:40.736 INFO:tasks.workunit.client.0.smithi139.stdout:1/205: readlink d7/d8/l17 0 2024-09-16T00:09:40.739 INFO:tasks.workunit.client.0.smithi139.stdout:3/188: mkdir d5/d7/d23/d35 0 2024-09-16T00:09:40.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.738+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f45f40c3ac0 con 0x7f45f4063080 2024-09-16T00:09:40.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.740+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f45f40c3ac0 con 0x7f45f4063080 2024-09-16T00:09:40.742 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:40.743 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:40.743 INFO:tasks.workunit.client.0.smithi139.stdout:1/206: link d7/d8/d1f/f24 d7/d8/d1f/f39 0 2024-09-16T00:09:40.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.741+0000 7f45f964c640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f45f40c41e0 0x7f45f402c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:40.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.741+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f45f4206a00 con 0x7f45f40c41e0 2024-09-16T00:09:40.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.741+0000 7f45f2ffd640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f45f40c41e0 0x7f45f402c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:40.744 INFO:tasks.workunit.client.0.smithi139.stdout:1/207: chown d7/d8/d27/d2f/f33 0 1 2024-09-16T00:09:40.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.741+0000 7f45f2ffd640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f45f40c41e0 0x7f45f402c640 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:40.744 INFO:tasks.workunit.client.0.smithi139.stdout:4/150: creat d12/f31 x:0 0 0 2024-09-16T00:09:40.744 INFO:tasks.workunit.client.0.smithi139.stdout:8/273: truncate fd 2176119 0 2024-09-16T00:09:40.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.742+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45f4206a00 con 0x7f45f40c41e0 2024-09-16T00:09:40.745 INFO:tasks.workunit.client.0.smithi139.stdout:8/274: write d17/d1a/d43/d40/f5d [873183,51167] 0 2024-09-16T00:09:40.746 INFO:tasks.workunit.client.0.smithi139.stdout:4/151: readlink d12/l2d 0 2024-09-16T00:09:40.746 INFO:tasks.workunit.client.0.smithi139.stdout:8/275: truncate d17/d1a/d2d/f3e 199800 0 2024-09-16T00:09:40.749 INFO:tasks.workunit.client.0.smithi139.stdout:8/276: mknod d17/d1a/d43/d40/c63 0 2024-09-16T00:09:40.751 INFO:tasks.workunit.client.0.smithi139.stdout:4/152: link d12/d21/f2c d12/d21/f32 0 2024-09-16T00:09:40.751 INFO:tasks.workunit.client.0.smithi139.stdout:8/277: write d17/f4c [544304,78207] 0 2024-09-16T00:09:40.755 INFO:tasks.workunit.client.0.smithi139.stdout:3/189: dwrite d5/d7/d23/f17 [0,4194304] 0 2024-09-16T00:09:40.756 INFO:tasks.workunit.client.0.smithi139.stdout:4/153: creat d12/f33 x:0 0 0 2024-09-16T00:09:40.759 INFO:tasks.workunit.client.0.smithi139.stdout:8/278: getdents d17/d1a/d57 0 2024-09-16T00:09:40.760 INFO:tasks.workunit.client.0.smithi139.stdout:3/190: creat d5/d7/d23/d27/d2f/d30/f36 x:0 0 0 2024-09-16T00:09:40.762 INFO:tasks.workunit.client.0.smithi139.stdout:4/154: dwrite d12/f23 [0,4194304] 0 2024-09-16T00:09:40.762 INFO:tasks.workunit.client.0.smithi139.stdout:8/279: symlink d17/d1a/d43/d40/d4f/l64 0 2024-09-16T00:09:40.762 INFO:tasks.workunit.client.0.smithi139.stdout:4/155: chown d12 15 1 2024-09-16T00:09:40.763 INFO:tasks.workunit.client.0.smithi139.stdout:8/280: stat d17/d1a/d2d/c30 0 2024-09-16T00:09:40.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.762+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f45f402cb80 con 0x7f45f40c41e0 2024-09-16T00:09:40.765 INFO:tasks.workunit.client.0.smithi139.stdout:4/156: creat d12/d2b/f34 x:0 0 0 2024-09-16T00:09:40.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.764+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f45f402cb80 con 0x7f45f40c41e0 2024-09-16T00:09:40.767 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:40.767 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:40.767 INFO:tasks.workunit.client.0.smithi139.stdout:4/157: dread - d12/d2b/f34 zero size 2024-09-16T00:09:40.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.764+0000 7f45f964c640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f45f402d2a0 0x7f45f402d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:40.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.765+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f45f4209b90 con 0x7f45f402d2a0 2024-09-16T00:09:40.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.765+0000 7f45f27fc640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f45f402d2a0 0x7f45f402d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:40.769 INFO:tasks.workunit.client.0.smithi139.stdout:8/281: creat d17/d1a/d43/d40/d4f/f65 x:0 0 0 2024-09-16T00:09:40.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.765+0000 7f45f27fc640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f45f402d2a0 0x7f45f402d720 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:40.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.766+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45f4209b90 con 0x7f45f402d2a0 2024-09-16T00:09:40.771 INFO:tasks.workunit.client.0.smithi139.stdout:8/282: unlink c13 0 2024-09-16T00:09:40.771 INFO:tasks.workunit.client.0.smithi139.stdout:3/191: dwrite d5/d7/dc/f12 [4194304,4194304] 0 2024-09-16T00:09:40.771 INFO:tasks.workunit.client.0.smithi139.stdout:8/283: fsync d17/d1a/f53 0 2024-09-16T00:09:40.772 INFO:tasks.workunit.client.0.smithi139.stdout:8/284: write f4 [634313,7851] 0 2024-09-16T00:09:40.775 INFO:tasks.workunit.client.0.smithi139.stdout:3/192: getdents d5/d7/d20 0 2024-09-16T00:09:40.775 INFO:tasks.workunit.client.0.smithi139.stdout:8/285: mknod d17/d1a/d57/c66 0 2024-09-16T00:09:40.776 INFO:tasks.workunit.client.0.smithi139.stdout:8/286: stat d17/l20 0 2024-09-16T00:09:40.777 INFO:tasks.workunit.client.0.smithi139.stdout:4/158: dwrite d12/f31 [0,4194304] 0 2024-09-16T00:09:40.778 INFO:tasks.workunit.client.0.smithi139.stdout:8/287: write d17/d1a/d2d/f3f [717305,38581] 0 2024-09-16T00:09:40.779 INFO:tasks.workunit.client.0.smithi139.stdout:4/159: creat d12/d2b/f35 x:0 0 0 2024-09-16T00:09:40.782 INFO:tasks.workunit.client.0.smithi139.stdout:8/288: truncate d17/d1a/f1d 4690064 0 2024-09-16T00:09:40.783 INFO:tasks.workunit.client.0.smithi139.stdout:3/193: rename d5/d29/f32 to d5/d7/f37 0 2024-09-16T00:09:40.785 INFO:tasks.workunit.client.0.smithi139.stdout:8/289: write d17/d1a/d43/d40/f49 [1890136,125923] 0 2024-09-16T00:09:40.786 INFO:tasks.workunit.client.0.smithi139.stdout:4/160: dread f4 [0,4194304] 0 2024-09-16T00:09:40.787 INFO:tasks.workunit.client.0.smithi139.stdout:8/290: write d17/d54/f55 [4892302,80233] 0 2024-09-16T00:09:40.788 INFO:tasks.workunit.client.0.smithi139.stdout:8/291: truncate d17/f4a 352222 0 2024-09-16T00:09:40.789 INFO:tasks.workunit.client.0.smithi139.stdout:3/194: fsync d5/d7/f28 0 2024-09-16T00:09:40.792 INFO:tasks.workunit.client.0.smithi139.stdout:8/292: getdents d17 0 2024-09-16T00:09:40.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.791+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f45f4207300 con 0x7f45f402d2a0 2024-09-16T00:09:40.794 INFO:tasks.workunit.client.0.smithi139.stdout:4/161: dwrite fe [0,4194304] 0 2024-09-16T00:09:40.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.793+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f45f4207300 con 0x7f45f402d2a0 2024-09-16T00:09:40.795 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.793+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=87}) -- 0x7f45f4209b90 con 0x7f45f40c9280 2024-09-16T00:09:40.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.793+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f45f4209d60 con 0x7f45f40c9280 2024-09-16T00:09:40.796 INFO:tasks.workunit.client.0.smithi139.stdout:4/162: dread - d12/d2b/f35 zero size 2024-09-16T00:09:40.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.794+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f45e405caa0 con 0x7f45f40c9280 2024-09-16T00:09:40.796 INFO:tasks.workunit.client.0.smithi139.stdout:8/293: symlink d17/d1a/d50/l67 0 2024-09-16T00:09:40.797 INFO:teuthology.orchestra.run.smithi139.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T00:09:40.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.794+0000 7f45f964c640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f45f4207bd0 0x7f45f4208040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:40.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.794+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f45f42098c0 con 0x7f45f4207bd0 2024-09-16T00:09:40.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.795+0000 7f45f37fe640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f45f4207bd0 0x7f45f4208040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:40.798 INFO:tasks.workunit.client.0.smithi139.stdout:4/163: readlink d12/l2d 0 2024-09-16T00:09:40.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.795+0000 7f45f37fe640 1 --2- 172.21.15.139:0/1238094052 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f45f4207bd0 0x7f45f4208040 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:40.799 INFO:tasks.workunit.client.0.smithi139.stdout:3/195: dwrite d5/d7/fd [0,4194304] 0 2024-09-16T00:09:40.799 INFO:tasks.workunit.client.0.smithi139.stdout:8/294: write d17/d1a/d43/d40/d4f/f65 [991722,83643] 0 2024-09-16T00:09:40.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.796+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45f42098c0 con 0x7f45f4207bd0 2024-09-16T00:09:40.799 INFO:tasks.workunit.client.0.smithi139.stdout:4/164: dread d12/d2b/f26 [0,4194304] 0 2024-09-16T00:09:40.803 INFO:tasks.workunit.client.0.smithi139.stdout:4/165: unlink d12/c1c 0 2024-09-16T00:09:40.804 INFO:tasks.workunit.client.0.smithi139.stdout:3/196: dread d5/d7/f2d [0,4194304] 0 2024-09-16T00:09:40.806 INFO:tasks.workunit.client.0.smithi139.stdout:8/295: dwrite d17/d1a/d2d/f3e [0,4194304] 0 2024-09-16T00:09:40.807 INFO:tasks.workunit.client.0.smithi139.stdout:3/197: mkdir d5/d7/d23/d27/d38 0 2024-09-16T00:09:40.811 INFO:tasks.workunit.client.0.smithi139.stdout:8/296: dread d17/d1a/f53 [4194304,4194304] 0 2024-09-16T00:09:40.812 INFO:tasks.workunit.client.0.smithi139.stdout:8/297: write d17/d1a/d43/d40/d4f/f65 [1032490,36579] 0 2024-09-16T00:09:40.813 INFO:tasks.workunit.client.0.smithi139.stdout:3/198: read d5/f16 [8350101,35814] 0 2024-09-16T00:09:40.820 INFO:tasks.workunit.client.0.smithi139.stdout:8/298: fdatasync d17/d1a/d2d/f3e 0 2024-09-16T00:09:40.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.820+0000 7f45f964c640 1 -- 172.21.15.139:0/1238094052 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f45f40008d0 con 0x7f45f4207bd0 2024-09-16T00:09:40.822 INFO:tasks.workunit.client.0.smithi139.stdout:4/166: dwrite d12/d21/f2c [0,4194304] 0 2024-09-16T00:09:40.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:40.821+0000 7f45eb7fe640 1 -- 172.21.15.139:0/1238094052 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f45f40008d0 con 0x7f45f4207bd0 2024-09-16T00:09:40.823 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:40.824 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:40.824 INFO:tasks.workunit.client.0.smithi139.stdout:8/299: mkdir d17/d54/d68 0 2024-09-16T00:09:40.824 INFO:tasks.workunit.client.0.smithi139.stdout:4/167: rmdir d12/d2b 39 2024-09-16T00:09:40.825 INFO:tasks.workunit.client.0.smithi139.stdout:8/300: readlink d17/l20 0 2024-09-16T00:09:40.825 INFO:tasks.workunit.client.0.smithi139.stdout:3/199: dwrite d5/f8 [0,4194304] 0 2024-09-16T00:09:40.827 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T00:09:40.828 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T00:09:40.828 INFO:tasks.workunit.client.0.smithi139.stdout:3/200: write d5/d7/d20/f25 [747933,98799] 0 2024-09-16T00:09:40.829 INFO:tasks.workunit.client.0.smithi139.stdout:8/301: creat d17/d54/f69 x:0 0 0 2024-09-16T00:09:40.831 INFO:tasks.workunit.client.0.smithi139.stdout:3/201: write d5/d7/f28 [2510654,97202] 0 2024-09-16T00:09:40.833 INFO:tasks.workunit.client.0.smithi139.stdout:8/302: rmdir d17/d1f 39 2024-09-16T00:09:40.834 INFO:tasks.workunit.client.0.smithi139.stdout:3/202: write d5/d7/d23/f17 [4326059,124898] 0 2024-09-16T00:09:40.836 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:40.835+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:40.838 INFO:tasks.workunit.client.0.smithi139.stdout:4/168: dwrite f1 [0,4194304] 0 2024-09-16T00:09:40.839 INFO:tasks.workunit.client.0.smithi139.stdout:8/303: rmdir d17/d54/d68 0 2024-09-16T00:09:40.841 INFO:tasks.workunit.client.0.smithi139.stdout:8/304: rmdir d17/d1a 39 2024-09-16T00:09:40.843 INFO:tasks.workunit.client.0.smithi139.stdout:3/203: dwrite d5/d7/d20/f25 [0,4194304] 0 2024-09-16T00:09:40.844 INFO:tasks.workunit.client.0.smithi139.stdout:3/204: stat d5/d7/d23/d27/d2f/d30/f36 0 2024-09-16T00:09:40.848 INFO:tasks.workunit.client.0.smithi139.stdout:3/205: creat d5/d29/f39 x:0 0 0 2024-09-16T00:09:40.849 INFO:tasks.workunit.client.0.smithi139.stdout:4/169: truncate d12/f2a 3552306 0 2024-09-16T00:09:40.849 INFO:tasks.workunit.client.0.smithi139.stdout:8/305: dread d17/d1a/d43/d40/f4b [0,4194304] 0 2024-09-16T00:09:40.850 INFO:tasks.workunit.client.0.smithi139.stdout:4/170: stat d12/d2b/f2e 0 2024-09-16T00:09:40.852 INFO:tasks.workunit.client.0.smithi139.stdout:3/206: truncate d5/d7/d23/d27/d2f/d30/f36 123707 0 2024-09-16T00:09:40.852 INFO:tasks.workunit.client.0.smithi139.stdout:8/306: getdents d17/d1a/d43 0 2024-09-16T00:09:40.853 INFO:tasks.workunit.client.0.smithi139.stdout:3/207: mknod d5/d7/c3a 0 2024-09-16T00:09:40.854 INFO:tasks.workunit.client.0.smithi139.stdout:8/307: creat d17/d1a/d57/f6a x:0 0 0 2024-09-16T00:09:40.855 INFO:tasks.workunit.client.0.smithi139.stdout:3/208: getdents d5/d7/d23/d27 0 2024-09-16T00:09:40.857 INFO:tasks.workunit.client.0.smithi139.stdout:3/209: write d5/d7/d23/d27/d2f/d30/f36 [28048,12652] 0 2024-09-16T00:09:40.859 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:40.859 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:40.860 INFO:tasks.workunit.client.0.smithi139.stdout:8/308: dwrite d17/d1a/d2d/f36 [0,4194304] 0 2024-09-16T00:09:40.865 INFO:tasks.workunit.client.0.smithi139.stdout:3/210: dwrite d5/d29/f39 [0,4194304] 0 2024-09-16T00:09:40.868 INFO:tasks.workunit.client.0.smithi139.stdout:3/211: creat d5/d7/d23/d35/f3b x:0 0 0 2024-09-16T00:09:40.870 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:09:40.873 INFO:tasks.workunit.client.0.smithi139.stdout:3/212: creat d5/d7/dc/f3c x:0 0 0 2024-09-16T00:09:40.873 INFO:tasks.workunit.client.0.smithi139.stdout:8/309: truncate d17/d1f/f28 3721706 0 2024-09-16T00:09:40.874 INFO:tasks.workunit.client.0.smithi139.stdout:8/310: readlink d17/l31 0 2024-09-16T00:09:40.874 INFO:tasks.workunit.client.0.smithi139.stdout:3/213: mknod d5/d7/c3d 0 2024-09-16T00:09:40.876 INFO:tasks.workunit.client.0.smithi139.stdout:3/214: mknod d5/d7/d23/d27/d2f/d30/c3e 0 2024-09-16T00:09:40.881 INFO:tasks.workunit.client.0.smithi139.stdout:8/311: write d17/d1a/f1e [2971767,28837] 0 2024-09-16T00:09:40.883 INFO:tasks.workunit.client.0.smithi139.stdout:3/215: dwrite d5/d7/d23/d35/f3b [0,4194304] 0 2024-09-16T00:09:40.884 INFO:tasks.workunit.client.0.smithi139.stdout:3/216: chown d5/d7/f28 332025721 1 2024-09-16T00:09:40.885 INFO:tasks.workunit.client.0.smithi139.stdout:3/217: chown d5/d7/c3a 0 1 2024-09-16T00:09:40.886 INFO:tasks.workunit.client.0.smithi139.stdout:8/312: dread d17/d1a/d2d/f36 [0,4194304] 0 2024-09-16T00:09:40.889 INFO:tasks.workunit.client.0.smithi139.stdout:3/218: rename d5/d29/l2c to d5/d7/d23/d27/d38/l3f 0 2024-09-16T00:09:40.890 INFO:tasks.workunit.client.0.smithi139.stdout:8/313: rename d17/d1a/d2d/f36 to d17/d1a/d2d/f6b 0 2024-09-16T00:09:40.891 INFO:tasks.workunit.client.0.smithi139.stdout:3/219: mkdir d5/d7/d23/d27/d2f/d30/d40 0 2024-09-16T00:09:40.893 INFO:tasks.workunit.client.0.smithi139.stdout:8/314: dread d17/f4a [0,4194304] 0 2024-09-16T00:09:40.893 INFO:tasks.workunit.client.0.smithi139.stdout:3/220: dread d5/d7/d23/d27/d2f/d30/f36 [0,4194304] 0 2024-09-16T00:09:40.896 INFO:tasks.workunit.client.0.smithi139.stdout:8/315: rename d17/f4c to d17/d1a/d43/f6c 0 2024-09-16T00:09:40.901 INFO:tasks.workunit.client.0.smithi139.stdout:3/221: rename d5/d29 to d5/d7/d23/d27/d2f/d30/d41 0 2024-09-16T00:09:40.901 INFO:tasks.workunit.client.0.smithi139.stdout:3/222: stat d5/d7/d23/d27/d2f/d30/c3e 0 2024-09-16T00:09:40.903 INFO:tasks.workunit.client.0.smithi139.stdout:3/223: creat d5/d7/d23/d27/d2f/d30/f42 x:0 0 0 2024-09-16T00:09:40.910 INFO:tasks.workunit.client.0.smithi139.stdout:3/224: dwrite d5/d7/d23/d27/d2f/d30/f36 [0,4194304] 0 2024-09-16T00:09:40.912 INFO:tasks.workunit.client.0.smithi139.stdout:3/225: getdents d5/d7/d23/d27/d2f/d30/d40 0 2024-09-16T00:09:40.917 INFO:tasks.workunit.client.0.smithi139.stdout:3/226: link d5/d7/c3d d5/d7/dc/c43 0 2024-09-16T00:09:40.920 INFO:tasks.workunit.client.0.smithi139.stdout:3/227: link d5/d7/d23/d27/d2f/c34 d5/d7/d23/c44 0 2024-09-16T00:09:40.921 INFO:tasks.workunit.client.0.smithi139.stdout:3/228: creat d5/d7/d23/d35/f45 x:0 0 0 2024-09-16T00:09:40.922 INFO:tasks.workunit.client.0.smithi139.stdout:3/229: read d5/d7/f26 [622105,49778] 0 2024-09-16T00:09:40.924 INFO:tasks.workunit.client.0.smithi139.stdout:3/230: creat d5/d7/d23/d35/f46 x:0 0 0 2024-09-16T00:09:40.927 INFO:tasks.workunit.client.0.smithi139.stdout:3/231: rename d5/d7/d20 to d5/d7/d23/d27/d2f/d30/d47 0 2024-09-16T00:09:40.930 INFO:tasks.workunit.client.0.smithi139.stdout:3/232: rename d5/d7/d23/d27/d38 to d5/d7/d23/d48 0 2024-09-16T00:09:40.931 INFO:tasks.workunit.client.0.smithi139.stdout:3/233: creat d5/d7/f49 x:0 0 0 2024-09-16T00:09:40.935 INFO:tasks.workunit.client.0.smithi139.stdout:3/234: link d5/f8 d5/d7/d23/d27/d2f/d30/d47/f4a 0 2024-09-16T00:09:40.936 INFO:tasks.workunit.client.0.smithi139.stdout:3/235: rmdir d5/d7/d23 39 2024-09-16T00:09:40.936 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:40.935+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:40.946 INFO:tasks.workunit.client.0.smithi139.stdout:3/236: truncate d5/d7/f26 607241 0 2024-09-16T00:09:40.949 INFO:tasks.workunit.client.0.smithi139.stdout:3/237: mkdir d5/d7/d23/d27/d2f/d30/d40/d4b 0 2024-09-16T00:09:40.950 INFO:tasks.workunit.client.0.smithi139.stdout:3/238: mkdir d5/d7/d23/d27/d2f/d30/d4c 0 2024-09-16T00:09:40.952 INFO:tasks.workunit.client.0.smithi139.stdout:3/239: getdents d5/d7/dc 0 2024-09-16T00:09:40.960 INFO:teuthology.orchestra.run.smithi164.stderr:Exporting 2.17 info 2.17( v 76'3 (0'0,76'3] local-lis/les=19/20 n=3 ec=19/19 lis/c=19/19 les/c/f=20/20/0 sis=19) 2024-09-16T00:09:40.965 INFO:teuthology.orchestra.run.smithi164.stderr:Read #2:e8f2ea03:::10000002282.03600000:head# 2024-09-16T00:09:40.965 INFO:teuthology.orchestra.run.smithi164.stderr:Read #2:ed8e7cdc:::1000000260f.03000000:head# 2024-09-16T00:09:40.966 INFO:teuthology.orchestra.run.smithi164.stderr:Read #2:ee44d6a7:::10000000261.03e00000:head# 2024-09-16T00:09:40.966 INFO:teuthology.orchestra.run.smithi164.stderr:Export successful 2024-09-16T00:09:40.988 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:40.988 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:40.997 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:09:41.012 INFO:tasks.workunit.client.0.smithi139.stdout:5/222: sync 2024-09-16T00:09:41.012 INFO:tasks.workunit.client.0.smithi139.stdout:6/142: sync 2024-09-16T00:09:41.012 INFO:tasks.workunit.client.0.smithi139.stdout:3/240: dread d5/d7/f28 [0,4194304] 0 2024-09-16T00:09:41.014 INFO:tasks.workunit.client.0.smithi139.stdout:6/143: fsync d8/d20/f2b 0 2024-09-16T00:09:41.014 INFO:tasks.workunit.client.0.smithi139.stdout:3/241: mkdir d5/d7/d23/d27/d2f/d30/d4d 0 2024-09-16T00:09:41.017 INFO:tasks.workunit.client.0.smithi139.stdout:3/242: write d5/d7/f28 [4610273,109021] 0 2024-09-16T00:09:41.018 INFO:tasks.workunit.client.0.smithi139.stdout:5/223: write d1/fa [756888,87442] 0 2024-09-16T00:09:41.021 INFO:tasks.workunit.client.0.smithi139.stdout:5/224: chown d1/d25/f36 99 1 2024-09-16T00:09:41.022 INFO:tasks.workunit.client.0.smithi139.stdout:6/144: dwrite d8/db/f15 [0,4194304] 0 2024-09-16T00:09:41.023 INFO:tasks.workunit.client.0.smithi139.stdout:3/243: creat d5/d7/d23/d27/d2f/d30/f4e x:0 0 0 2024-09-16T00:09:41.023 INFO:tasks.workunit.client.0.smithi139.stdout:3/244: stat d5/d7/f2d 0 2024-09-16T00:09:41.024 INFO:tasks.workunit.client.0.smithi139.stdout:6/145: stat d8/d20/d27/d10/f22 0 2024-09-16T00:09:41.027 INFO:tasks.workunit.client.0.smithi139.stdout:6/146: creat d8/d29/f32 x:0 0 0 2024-09-16T00:09:41.030 INFO:tasks.workunit.client.0.smithi139.stdout:3/245: rmdir d5/d7/d23/d27/d2f/d30/d40/d4b 0 2024-09-16T00:09:41.030 INFO:tasks.workunit.client.0.smithi139.stdout:5/225: dwrite d1/d25/f36 [0,4194304] 0 2024-09-16T00:09:41.033 INFO:tasks.workunit.client.0.smithi139.stdout:5/226: creat d1/f4a x:0 0 0 2024-09-16T00:09:41.034 INFO:tasks.workunit.client.0.smithi139.stdout:6/147: getdents d8/d20/d27 0 2024-09-16T00:09:41.036 INFO:tasks.workunit.client.0.smithi139.stdout:6/148: write d8/db/f15 [1805755,17259] 0 2024-09-16T00:09:41.037 INFO:tasks.workunit.client.0.smithi139.stdout:3/246: mknod d5/d7/d23/d27/d2f/d30/d4c/c4f 0 2024-09-16T00:09:41.037 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:41.036+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:41.040 INFO:tasks.workunit.client.0.smithi139.stdout:6/149: symlink d8/d20/d27/d10/l33 0 2024-09-16T00:09:41.041 INFO:tasks.workunit.client.0.smithi139.stdout:3/247: mkdir d5/d7/d23/d27/d2f/d30/d4c/d50 0 2024-09-16T00:09:41.046 INFO:tasks.workunit.client.0.smithi139.stdout:5/227: dwrite d1/d25/f36 [4194304,4194304] 0 2024-09-16T00:09:41.053 INFO:tasks.workunit.client.0.smithi139.stdout:6/150: dwrite d8/db/d26/d28/f31 [0,4194304] 0 2024-09-16T00:09:41.056 INFO:tasks.workunit.client.0.smithi139.stdout:6/151: symlink d8/d20/d27/l34 0 2024-09-16T00:09:41.059 INFO:tasks.workunit.client.0.smithi139.stdout:3/248: dread d5/d7/d23/f15 [0,4194304] 0 2024-09-16T00:09:41.060 INFO:tasks.workunit.client.0.smithi139.stdout:5/228: dwrite d1/d13/d3b/f47 [0,4194304] 0 2024-09-16T00:09:41.063 INFO:tasks.workunit.client.0.smithi139.stdout:5/229: dread - d1/d21/f3a zero size 2024-09-16T00:09:41.069 INFO:tasks.workunit.client.0.smithi139.stdout:6/152: dread f6 [0,4194304] 0 2024-09-16T00:09:41.070 INFO:tasks.workunit.client.0.smithi139.stdout:3/249: dwrite d5/d7/d23/f15 [0,4194304] 0 2024-09-16T00:09:41.072 INFO:tasks.workunit.client.0.smithi139.stdout:6/153: read d8/db/f1a [1794409,7589] 0 2024-09-16T00:09:41.074 INFO:tasks.workunit.client.0.smithi139.stdout:5/230: dwrite d1/d13/f17 [0,4194304] 0 2024-09-16T00:09:41.078 INFO:tasks.workunit.client.0.smithi139.stdout:6/154: mknod d8/db/d26/d28/c35 0 2024-09-16T00:09:41.079 INFO:tasks.workunit.client.0.smithi139.stdout:5/231: rmdir d1/d13/d3f 39 2024-09-16T00:09:41.082 INFO:tasks.workunit.client.0.smithi139.stdout:5/232: mknod d1/c4b 0 2024-09-16T00:09:41.083 INFO:tasks.workunit.client.0.smithi139.stdout:3/250: dwrite d5/d7/dc/f3c [0,4194304] 0 2024-09-16T00:09:41.083 INFO:tasks.workunit.client.0.smithi139.stdout:6/155: creat d8/d29/f36 x:0 0 0 2024-09-16T00:09:41.083 INFO:tasks.workunit.client.0.smithi139.stdout:6/156: readlink d8/d20/d27/l1e 0 2024-09-16T00:09:41.084 INFO:tasks.workunit.client.0.smithi139.stdout:6/157: chown d8/db/d26/d28/d2a 4 1 2024-09-16T00:09:41.085 INFO:tasks.workunit.client.0.smithi139.stdout:5/233: creat d1/d13/d3b/f4c x:0 0 0 2024-09-16T00:09:41.087 INFO:tasks.workunit.client.0.smithi139.stdout:6/158: creat d8/d20/d27/d10/f37 x:0 0 0 2024-09-16T00:09:41.088 INFO:tasks.workunit.client.0.smithi139.stdout:5/234: creat d1/d25/d30/d33/d43/f4d x:0 0 0 2024-09-16T00:09:41.091 INFO:tasks.workunit.client.0.smithi139.stdout:6/159: unlink d8/d20/d27/d10/f17 0 2024-09-16T00:09:41.091 INFO:tasks.workunit.client.0.smithi139.stdout:3/251: dwrite d5/d7/d23/f17 [4194304,4194304] 0 2024-09-16T00:09:41.092 INFO:tasks.workunit.client.0.smithi139.stdout:6/160: stat d8/d20/d27/d10/f22 0 2024-09-16T00:09:41.093 INFO:tasks.workunit.client.0.smithi139.stdout:5/235: dread d1/fa [0,4194304] 0 2024-09-16T00:09:41.094 INFO:tasks.workunit.client.0.smithi139.stdout:3/252: symlink d5/d7/d23/d48/l51 0 2024-09-16T00:09:41.095 INFO:tasks.workunit.client.0.smithi139.stdout:3/253: dread - d5/d7/f49 zero size 2024-09-16T00:09:41.095 INFO:tasks.workunit.client.0.smithi139.stdout:5/236: write d1/f1a [7579681,23984] 0 2024-09-16T00:09:41.096 INFO:tasks.workunit.client.0.smithi139.stdout:6/161: link d8/d29/f2d d8/d20/f38 0 2024-09-16T00:09:41.097 INFO:tasks.workunit.client.0.smithi139.stdout:5/237: creat d1/d13/d3b/d3e/f4e x:0 0 0 2024-09-16T00:09:41.099 INFO:tasks.workunit.client.0.smithi139.stdout:3/254: dread d5/d7/f13 [0,4194304] 0 2024-09-16T00:09:41.100 INFO:tasks.workunit.client.0.smithi139.stdout:5/238: mkdir d1/d13/d3b/d3e/d4f 0 2024-09-16T00:09:41.101 INFO:tasks.workunit.client.0.smithi139.stdout:3/255: chown d5/d7/d23/d35/f45 0 1 2024-09-16T00:09:41.101 INFO:tasks.workunit.client.0.smithi139.stdout:9/209: sync 2024-09-16T00:09:41.101 INFO:tasks.workunit.client.0.smithi139.stdout:0/238: sync 2024-09-16T00:09:41.101 INFO:tasks.workunit.client.0.smithi139.stdout:7/200: sync 2024-09-16T00:09:41.102 INFO:tasks.workunit.client.0.smithi139.stdout:3/256: chown d5/d7/d23/d27 375957 1 2024-09-16T00:09:41.102 INFO:tasks.workunit.client.0.smithi139.stdout:5/239: creat d1/d25/d30/d33/f50 x:0 0 0 2024-09-16T00:09:41.103 INFO:tasks.workunit.client.0.smithi139.stdout:9/210: creat d19/f38 x:0 0 0 2024-09-16T00:09:41.104 INFO:tasks.workunit.client.0.smithi139.stdout:7/201: mkdir d5/d26/d48 0 2024-09-16T00:09:41.106 INFO:tasks.workunit.client.0.smithi139.stdout:3/257: symlink d5/d7/d23/d27/d2f/d30/l52 0 2024-09-16T00:09:41.109 INFO:tasks.workunit.client.0.smithi139.stdout:9/211: rename d19/d22/f37 to d19/d1a/f39 0 2024-09-16T00:09:41.110 INFO:tasks.workunit.client.0.smithi139.stdout:5/240: creat d1/d13/d3b/d3e/d4f/f51 x:0 0 0 2024-09-16T00:09:41.110 INFO:tasks.workunit.client.0.smithi139.stdout:0/239: dwrite d6/d1c/d41/d44/f3e [0,4194304] 0 2024-09-16T00:09:41.112 INFO:tasks.workunit.client.0.smithi139.stdout:6/162: truncate d8/db/f2c 3453828 0 2024-09-16T00:09:41.112 INFO:tasks.workunit.client.0.smithi139.stdout:0/240: chown d6/l1f 48441 1 2024-09-16T00:09:41.113 INFO:tasks.workunit.client.0.smithi139.stdout:5/241: mkdir d1/d25/d30/d33/d43/d52 0 2024-09-16T00:09:41.113 INFO:tasks.workunit.client.0.smithi139.stdout:0/241: write d6/d1c/d41/d44/f3e [2656991,19425] 0 2024-09-16T00:09:41.114 INFO:tasks.workunit.client.0.smithi139.stdout:0/242: write d6/d1c/d41/d44/f3e [1826522,117837] 0 2024-09-16T00:09:41.115 INFO:tasks.workunit.client.0.smithi139.stdout:0/243: write d6/d1c/d41/d44/dd/f40 [1000766,84384] 0 2024-09-16T00:09:41.121 INFO:tasks.workunit.client.0.smithi139.stdout:7/202: write d5/f1c [24380,99465] 0 2024-09-16T00:09:41.121 INFO:tasks.workunit.client.0.smithi139.stdout:6/163: creat d8/f39 x:0 0 0 2024-09-16T00:09:41.121 INFO:tasks.workunit.client.0.smithi139.stdout:0/244: read d6/f8 [285029,5359] 0 2024-09-16T00:09:41.123 INFO:tasks.workunit.client.0.smithi139.stdout:0/245: write d6/d1c/d41/d44/dd/f19 [1537057,28895] 0 2024-09-16T00:09:41.123 INFO:tasks.workunit.client.0.smithi139.stdout:6/164: rename d8/d20/d27/c30 to d8/d20/c3a 0 2024-09-16T00:09:41.124 INFO:tasks.workunit.client.0.smithi139.stdout:6/165: readlink d8/d20/d27/l1e 0 2024-09-16T00:09:41.125 INFO:tasks.workunit.client.0.smithi139.stdout:7/203: symlink d5/l49 0 2024-09-16T00:09:41.125 INFO:tasks.workunit.client.0.smithi139.stdout:0/246: dread d6/d1c/d41/d44/f29 [0,4194304] 0 2024-09-16T00:09:41.127 INFO:tasks.workunit.client.0.smithi139.stdout:5/242: getdents d1/d13 0 2024-09-16T00:09:41.127 INFO:tasks.workunit.client.0.smithi139.stdout:5/243: chown d1/d13/d22 0 1 2024-09-16T00:09:41.127 INFO:tasks.workunit.client.0.smithi139.stdout:0/247: read d6/f8 [377515,65851] 0 2024-09-16T00:09:41.128 INFO:tasks.workunit.client.0.smithi139.stdout:5/244: read - d1/d13/d3b/d3e/f4e zero size 2024-09-16T00:09:41.128 INFO:tasks.workunit.client.0.smithi139.stdout:6/166: rename d8/d20/f38 to d8/d20/d27/d10/f3b 0 2024-09-16T00:09:41.128 INFO:tasks.workunit.client.0.smithi139.stdout:5/245: stat d1/d21 0 2024-09-16T00:09:41.129 INFO:tasks.workunit.client.0.smithi139.stdout:7/204: creat d5/d26/d46/f4a x:0 0 0 2024-09-16T00:09:41.130 INFO:tasks.workunit.client.0.smithi139.stdout:6/167: mknod d8/db/d26/d28/c3c 0 2024-09-16T00:09:41.131 INFO:tasks.workunit.client.0.smithi139.stdout:7/205: write d5/f1f [2910116,129512] 0 2024-09-16T00:09:41.133 INFO:tasks.workunit.client.0.smithi139.stdout:5/246: getdents d1/d13/d3f 0 2024-09-16T00:09:41.134 INFO:tasks.workunit.client.0.smithi139.stdout:0/248: dwrite d6/d1c/d41/d44/f33 [0,4194304] 0 2024-09-16T00:09:41.134 INFO:tasks.workunit.client.0.smithi139.stdout:6/168: fdatasync d8/d20/d27/d10/f13 0 2024-09-16T00:09:41.135 INFO:tasks.workunit.client.0.smithi139.stdout:7/206: chown d5/d16/d22/d25/d2d/c2f 1 1 2024-09-16T00:09:41.136 INFO:tasks.workunit.client.0.smithi139.stdout:0/249: fsync d6/f8 0 2024-09-16T00:09:41.137 INFO:tasks.workunit.client.0.smithi139.stdout:7/207: dread - d5/d26/d46/f4a zero size 2024-09-16T00:09:41.137 INFO:tasks.workunit.client.0.smithi139.stdout:5/247: rename d1/d13/d3b/d3e/d4f to d1/d21/d49/d53 0 2024-09-16T00:09:41.138 INFO:tasks.workunit.client.0.smithi139.stdout:5/248: chown d1/d13/d3b/f44 915412 1 2024-09-16T00:09:41.138 INFO:tasks.workunit.client.0.smithi139.stdout:0/250: creat d6/d1c/f4c x:0 0 0 2024-09-16T00:09:41.140 INFO:tasks.workunit.client.0.smithi139.stdout:0/251: dread d6/d1c/d41/d44/f29 [0,4194304] 0 2024-09-16T00:09:41.145 INFO:tasks.workunit.client.0.smithi139.stdout:0/252: dwrite f3 [0,4194304] 0 2024-09-16T00:09:41.146 INFO:tasks.workunit.client.0.smithi139.stdout:0/253: chown d6/d42/f48 170941 1 2024-09-16T00:09:41.149 INFO:tasks.workunit.client.0.smithi139.stdout:0/254: rename d6/d1c/d41/d44/dd/f34 to d6/d1c/f4d 0 2024-09-16T00:09:41.167 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:41.167 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:41.176 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:09:41.227 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T00:09:41.250 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:41.136+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:41.251 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:41.249+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:41.257 INFO:tasks.workunit.client.0.smithi139.stdout:5/249: truncate d1/f1a 5001260 0 2024-09-16T00:09:41.258 INFO:tasks.workunit.client.0.smithi139.stdout:0/255: write d6/d1c/d41/d44/dd/f24 [4878387,9371] 0 2024-09-16T00:09:41.262 INFO:tasks.workunit.client.0.smithi139.stdout:0/256: stat d6/d1c/d41/d44/dd/f2d 0 2024-09-16T00:09:41.265 INFO:tasks.workunit.client.0.smithi139.stdout:6/169: dwrite d8/d20/d27/d10/f13 [0,4194304] 0 2024-09-16T00:09:41.266 INFO:tasks.workunit.client.0.smithi139.stdout:7/208: dwrite d5/f12 [0,4194304] 0 2024-09-16T00:09:41.268 INFO:tasks.workunit.client.0.smithi139.stdout:5/250: dread d1/d13/f17 [0,4194304] 0 2024-09-16T00:09:41.269 INFO:tasks.workunit.client.0.smithi139.stdout:0/257: unlink d6/d1c/d41/d44/l3c 0 2024-09-16T00:09:41.270 INFO:tasks.workunit.client.0.smithi139.stdout:5/251: fdatasync d1/d13/d3b/f47 0 2024-09-16T00:09:41.271 INFO:tasks.workunit.client.0.smithi139.stdout:6/170: creat d8/d29/f3d x:0 0 0 2024-09-16T00:09:41.273 INFO:tasks.workunit.client.0.smithi139.stdout:0/258: stat d6/d1c/d41/d44/dd/l17 0 2024-09-16T00:09:41.277 INFO:tasks.workunit.client.0.smithi139.stdout:0/259: write d6/d1c/d41/d44/dd/f19 [4597776,67196] 0 2024-09-16T00:09:41.281 INFO:tasks.workunit.client.0.smithi139.stdout:6/171: link d8/d20/d27/c1b d8/d20/d27/c3e 0 2024-09-16T00:09:41.281 INFO:tasks.workunit.client.0.smithi139.stdout:7/209: dwrite d5/d16/d22/d25/d2d/f36 [0,4194304] 0 2024-09-16T00:09:41.283 INFO:tasks.workunit.client.0.smithi139.stdout:0/260: chown d6/d1c/d41/d44/f3e 9654 1 2024-09-16T00:09:41.285 INFO:tasks.workunit.client.0.smithi139.stdout:7/210: rmdir d5 39 2024-09-16T00:09:41.286 INFO:tasks.workunit.client.0.smithi139.stdout:0/261: readlink d6/l2a 0 2024-09-16T00:09:41.289 INFO:tasks.workunit.client.0.smithi139.stdout:5/252: dwrite d1/f48 [0,4194304] 0 2024-09-16T00:09:41.292 INFO:tasks.workunit.client.0.smithi139.stdout:5/253: fsync d1/d13/d3b/f44 0 2024-09-16T00:09:41.293 INFO:tasks.workunit.client.0.smithi139.stdout:6/172: write d8/d29/f2d [1181132,43142] 0 2024-09-16T00:09:41.298 INFO:tasks.workunit.client.0.smithi139.stdout:2/187: sync 2024-09-16T00:09:41.300 INFO:tasks.workunit.client.0.smithi139.stdout:7/211: dwrite d5/f1f [0,4194304] 0 2024-09-16T00:09:41.302 INFO:tasks.workunit.client.0.smithi139.stdout:0/262: dwrite d6/d1c/d41/d44/dd/f11 [0,4194304] 0 2024-09-16T00:09:41.303 INFO:tasks.workunit.client.0.smithi139.stdout:6/173: truncate d8/d20/d27/d10/f37 457704 0 2024-09-16T00:09:41.305 INFO:tasks.workunit.client.0.smithi139.stdout:7/212: write d5/d26/f32 [902324,10729] 0 2024-09-16T00:09:41.309 INFO:tasks.workunit.client.0.smithi139.stdout:0/263: unlink d6/d1c/f28 0 2024-09-16T00:09:41.314 INFO:tasks.workunit.client.0.smithi139.stdout:0/264: chown d6/l2a 268050233 1 2024-09-16T00:09:41.315 INFO:tasks.workunit.client.0.smithi139.stdout:5/254: dwrite d1/f27 [0,4194304] 0 2024-09-16T00:09:41.316 INFO:tasks.workunit.client.0.smithi139.stdout:2/188: dwrite d11/f1a [0,4194304] 0 2024-09-16T00:09:41.317 INFO:tasks.workunit.client.0.smithi139.stdout:7/213: creat d5/f4b x:0 0 0 2024-09-16T00:09:41.319 INFO:tasks.workunit.client.0.smithi139.stdout:6/174: truncate d8/d29/f2d 718998 0 2024-09-16T00:09:41.320 INFO:tasks.workunit.client.0.smithi139.stdout:0/265: creat d6/d1c/d41/f4e x:0 0 0 2024-09-16T00:09:41.320 INFO:tasks.workunit.client.0.smithi139.stdout:7/214: readlink d5/d16/l24 0 2024-09-16T00:09:41.321 INFO:tasks.workunit.client.0.smithi139.stdout:2/189: symlink d11/l3b 0 2024-09-16T00:09:41.325 INFO:tasks.workunit.client.0.smithi139.stdout:5/255: dread d1/f48 [0,4194304] 0 2024-09-16T00:09:41.326 INFO:tasks.workunit.client.0.smithi139.stdout:0/266: creat d6/d1c/d45/d49/f4f x:0 0 0 2024-09-16T00:09:41.328 INFO:tasks.workunit.client.0.smithi139.stdout:7/215: dread d5/d16/f3e [0,4194304] 0 2024-09-16T00:09:41.328 INFO:tasks.workunit.client.0.smithi139.stdout:0/267: chown d6/d42/f48 12612671 1 2024-09-16T00:09:41.331 INFO:tasks.workunit.client.0.smithi139.stdout:2/190: dwrite d11/d30/f36 [0,4194304] 0 2024-09-16T00:09:41.331 INFO:tasks.workunit.client.0.smithi139.stdout:7/216: write d5/d26/d46/f4a [332347,13345] 0 2024-09-16T00:09:41.331 INFO:tasks.workunit.client.0.smithi139.stdout:0/268: chown d6/d1c/d41/d44/dd/c16 11 1 2024-09-16T00:09:41.334 INFO:tasks.workunit.client.0.smithi139.stdout:2/191: write d11/d1f/f24 [436535,42555] 0 2024-09-16T00:09:41.335 INFO:tasks.workunit.client.0.smithi139.stdout:7/217: link d5/fb d5/d26/f4c 0 2024-09-16T00:09:41.336 INFO:tasks.workunit.client.0.smithi139.stdout:7/218: chown f4 9767708 1 2024-09-16T00:09:41.336 INFO:tasks.workunit.client.0.smithi139.stdout:7/219: chown d5/cf 866652052 1 2024-09-16T00:09:41.339 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:41.337+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:41.339 INFO:tasks.workunit.client.0.smithi139.stdout:0/269: dread d6/d1c/d41/d44/f33 [0,4194304] 0 2024-09-16T00:09:41.339 INFO:tasks.workunit.client.0.smithi139.stdout:2/192: dread d11/f1a [0,4194304] 0 2024-09-16T00:09:41.340 INFO:tasks.workunit.client.0.smithi139.stdout:7/220: symlink d5/d26/d42/l4d 0 2024-09-16T00:09:41.341 INFO:tasks.workunit.client.0.smithi139.stdout:7/221: fdatasync d5/d37/f47 0 2024-09-16T00:09:41.341 INFO:tasks.workunit.client.0.smithi139.stdout:0/270: creat d6/d1c/d45/f50 x:0 0 0 2024-09-16T00:09:41.342 INFO:tasks.workunit.client.0.smithi139.stdout:2/193: symlink d11/d32/l3c 0 2024-09-16T00:09:41.344 INFO:tasks.workunit.client.0.smithi139.stdout:6/175: dread d8/d20/d27/f18 [0,4194304] 0 2024-09-16T00:09:41.346 INFO:tasks.workunit.client.0.smithi139.stdout:2/194: getdents d11/d1f/d29 0 2024-09-16T00:09:41.348 INFO:tasks.workunit.client.0.smithi139.stdout:6/176: creat d8/db/d26/d28/f3f x:0 0 0 2024-09-16T00:09:41.348 INFO:tasks.workunit.client.0.smithi139.stdout:2/195: mknod d11/d1f/d29/d2b/c3d 0 2024-09-16T00:09:41.349 INFO:tasks.workunit.client.0.smithi139.stdout:0/271: dwrite f2 [0,4194304] 0 2024-09-16T00:09:41.351 INFO:tasks.workunit.client.0.smithi139.stdout:2/196: write d11/d1f/d29/d2b/f2d [537186,91158] 0 2024-09-16T00:09:41.352 INFO:tasks.workunit.client.0.smithi139.stdout:0/272: truncate d6/d1c/d45/f50 285983 0 2024-09-16T00:09:41.352 INFO:tasks.workunit.client.0.smithi139.stdout:6/177: rename d8/d20/d27/d10/f11 to d8/db/d26/d28/f40 0 2024-09-16T00:09:41.353 INFO:tasks.workunit.client.0.smithi139.stdout:2/197: rmdir d11 39 2024-09-16T00:09:41.354 INFO:tasks.workunit.client.0.smithi139.stdout:2/198: stat d11/f1e 0 2024-09-16T00:09:41.355 INFO:tasks.workunit.client.0.smithi139.stdout:2/199: readlink d11/l3b 0 2024-09-16T00:09:41.355 INFO:tasks.workunit.client.0.smithi139.stdout:6/178: rename d8/d20/d27/d10/f22 to d8/d1f/f41 0 2024-09-16T00:09:41.358 INFO:tasks.workunit.client.0.smithi139.stdout:2/200: truncate d11/f37 1059700 0 2024-09-16T00:09:41.358 INFO:tasks.workunit.client.0.smithi139.stdout:2/201: readlink d11/l14 0 2024-09-16T00:09:41.358 INFO:tasks.workunit.client.0.smithi139.stdout:6/179: mkdir d8/db/d26/d28/d2a/d42 0 2024-09-16T00:09:41.359 INFO:tasks.workunit.client.0.smithi139.stdout:2/202: mknod d11/c3e 0 2024-09-16T00:09:41.359 INFO:tasks.workunit.client.0.smithi139.stdout:6/180: stat d8/d20/d27/d10/l33 0 2024-09-16T00:09:41.360 INFO:tasks.workunit.client.0.smithi139.stdout:2/203: symlink d11/l3f 0 2024-09-16T00:09:41.361 INFO:tasks.workunit.client.0.smithi139.stdout:2/204: write ff [4605256,10524] 0 2024-09-16T00:09:41.363 INFO:tasks.workunit.client.0.smithi139.stdout:2/205: symlink d11/d2f/l40 0 2024-09-16T00:09:41.363 INFO:tasks.workunit.client.0.smithi139.stdout:2/206: stat d11/l3f 0 2024-09-16T00:09:41.364 INFO:tasks.workunit.client.0.smithi139.stdout:2/207: creat d11/f41 x:0 0 0 2024-09-16T00:09:41.370 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:41.370 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:41.379 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:09:41.392 INFO:tasks.workunit.client.0.smithi139.stdout:5/256: truncate d1/f27 1785463 0 2024-09-16T00:09:41.394 INFO:tasks.workunit.client.0.smithi139.stdout:5/257: fdatasync d1/d13/f17 0 2024-09-16T00:09:41.396 INFO:tasks.workunit.client.0.smithi139.stdout:5/258: rename d1/c4b to d1/d21/d49/c54 0 2024-09-16T00:09:41.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.397+0000 7f4b12837640 1 Processor -- start 2024-09-16T00:09:41.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.397+0000 7f4b12837640 1 -- start start 2024-09-16T00:09:41.399 INFO:tasks.workunit.client.0.smithi139.stdout:5/259: creat d1/d25/d30/f55 x:0 0 0 2024-09-16T00:09:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.398+0000 7f4b12837640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4b0c0c7fb0 0x7f4b0c1661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.398+0000 7f4b12837640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b0c0c9800 con 0x7f4b0c154bc0 2024-09-16T00:09:41.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.398+0000 7f4b12837640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b0c06bbc0 con 0x7f4b0c0c79c0 2024-09-16T00:09:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.398+0000 7f4b12837640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f4b0c06be60 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.398+0000 7f4b0bfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4b0c154bc0 0x7f4b0c15b230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45538/0 (socket says 172.21.15.139:45538) 2024-09-16T00:09:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.398+0000 7f4b0bfff640 1 -- 172.21.15.139:0/1717556218 learned_addr learned my addr 172.21.15.139:0/1717556218 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.398+0000 7f4b10dd4640 1 --2- 172.21.15.139:0/1717556218 >> v2:172.21.15.164:3300/0 conn(0x7f4b0c0c7fb0 0x7f4b0c1661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.398+0000 7f4b0b7fe640 1 -- 172.21.15.139:0/1717556218 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3451194067 0 0) 0x7f4b0c0c9800 con 0x7f4b0c154bc0 2024-09-16T00:09:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.398+0000 7f4b0b7fe640 1 -- 172.21.15.139:0/1717556218 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4af8003520 con 0x7f4b0c154bc0 2024-09-16T00:09:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.398+0000 7f4b10dd4640 1 -- 172.21.15.139:0/1717556218 >> v1:172.21.15.164:6789/0 conn(0x7f4b0c0c79c0 legacy=0x7f4b0c0c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.398+0000 7f4b10dd4640 1 -- 172.21.15.139:0/1717556218 >> v1:172.21.15.139:6789/0 conn(0x7f4b0c154bc0 legacy=0x7f4b0c15b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:41.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.398+0000 7f4b10dd4640 1 -- 172.21.15.139:0/1717556218 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0c0c9520 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.402 INFO:tasks.workunit.client.0.smithi139.stdout:0/273: truncate d6/d1c/d41/d44/f25 1371260 0 2024-09-16T00:09:41.402 INFO:tasks.workunit.client.0.smithi139.stdout:5/260: truncate d1/d13/d3b/f4c 193654 0 2024-09-16T00:09:41.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.399+0000 7f4b10dd4640 1 --2- 172.21.15.139:0/1717556218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b0c0c7fb0 0x7f4b0c1661a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009d90 tx=0x7f4b0402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22b78053efb6798b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:41.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.400+0000 7f4b0b7fe640 1 -- 172.21.15.139:0/1717556218 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04007740 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.400+0000 7f4b0b7fe640 1 -- 172.21.15.139:0/1717556218 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b040078e0 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.400+0000 7f4b0b7fe640 1 -- 172.21.15.139:0/1717556218 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04007c90 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.403 INFO:tasks.workunit.client.0.smithi139.stdout:5/261: chown d1/f28 8996515 1 2024-09-16T00:09:41.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.401+0000 7f4b12837640 1 -- 172.21.15.139:0/1717556218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b0c0c7fb0 msgr2=0x7f4b0c1661a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:41.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.401+0000 7f4b12837640 1 --2- 172.21.15.139:0/1717556218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b0c0c7fb0 0x7f4b0c1661a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009d90 tx=0x7f4b0402a6f0 comp rx=0 tx=0).stop 2024-09-16T00:09:41.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.401+0000 7f4b12837640 1 -- 172.21.15.139:0/1717556218 shutdown_connections 2024-09-16T00:09:41.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.401+0000 7f4b12837640 1 --2- 172.21.15.139:0/1717556218 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b0c0c7fb0 0x7f4b0c1661a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:41.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.401+0000 7f4b12837640 1 -- 172.21.15.139:0/1717556218 >> 172.21.15.139:0/1717556218 conn(0x7f4b0c070460 msgr2=0x7f4b0c070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:41.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.401+0000 7f4b12837640 1 -- 172.21.15.139:0/1717556218 shutdown_connections 2024-09-16T00:09:41.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.401+0000 7f4b12837640 1 -- 172.21.15.139:0/1717556218 wait complete. 2024-09-16T00:09:41.405 INFO:tasks.workunit.client.0.smithi139.stdout:5/262: stat d1/d25/d30/d33/d43/d52 0 2024-09-16T00:09:41.405 INFO:tasks.workunit.client.0.smithi139.stdout:0/274: mknod d6/d1c/d41/d44/c51 0 2024-09-16T00:09:41.405 INFO:tasks.workunit.client.0.smithi139.stdout:6/181: write d8/d20/d27/d10/f3b [1039478,31101] 0 2024-09-16T00:09:41.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.403+0000 7f4b12837640 1 Processor -- start 2024-09-16T00:09:41.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.403+0000 7f4b12837640 1 -- start start 2024-09-16T00:09:41.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b12837640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4b0c0c79c0 0x7f4b0c0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:41.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b12837640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b0c0c7fb0 0x7f4b0c14a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:41.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b12837640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b0c154bc0 0x7f4b0c14f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:41.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b12837640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c154fa0 con 0x7f4b0c154bc0 2024-09-16T00:09:41.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b12837640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c0c9220 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.406 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b12837640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4b0c0c8e50 con 0x7f4b0c0c79c0 2024-09-16T00:09:41.407 INFO:tasks.workunit.client.0.smithi139.stdout:5/263: stat d1/d21/d49/d53/f51 0 2024-09-16T00:09:41.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b10dd4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b0c154bc0 0x7f4b0c14f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:41.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b03fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b0c0c7fb0 0x7f4b0c14a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:41.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b10dd4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b0c154bc0 0x7f4b0c14f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43778/0 (socket says 172.21.15.139:43778) 2024-09-16T00:09:41.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b10dd4640 1 -- 172.21.15.139:0/4104911171 learned_addr learned my addr 172.21.15.139:0/4104911171 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:41.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b03fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b0c0c7fb0 0x7f4b0c14a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:47184/0 (socket says 172.21.15.139:47184) 2024-09-16T00:09:41.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b0bfff640 1 --2- 172.21.15.139:0/4104911171 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4b0c0c79c0 0x7f4b0c0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:41.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b03fff640 1 -- 172.21.15.139:0/4104911171 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4b0c0c79c0 msgr2=0x7f4b0c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:41.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b03fff640 1 --2- 172.21.15.139:0/4104911171 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4b0c0c79c0 0x7f4b0c0cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:41.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b03fff640 1 -- 172.21.15.139:0/4104911171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b0c154bc0 msgr2=0x7f4b0c14f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:41.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b03fff640 1 --2- 172.21.15.139:0/4104911171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b0c154bc0 0x7f4b0c14f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:41.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.404+0000 7f4b03fff640 1 -- 172.21.15.139:0/4104911171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0c0cd2b0 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.405+0000 7f4b03fff640 1 --2- 172.21.15.139:0/4104911171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b0c0c7fb0 0x7f4b0c14a1c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4af400c3c0 tx=0x7f4af400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:41.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.405+0000 7f4b097fa640 1 -- 172.21.15.139:0/4104911171 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af40052d0 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.405+0000 7f4b097fa640 1 -- 172.21.15.139:0/4104911171 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4af4005490 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.405+0000 7f4b12837640 1 -- 172.21.15.139:0/4104911171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0c0cdd80 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.405+0000 7f4b097fa640 1 -- 172.21.15.139:0/4104911171 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af4007260 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.405+0000 7f4b12837640 1 -- 172.21.15.139:0/4104911171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b0c15a000 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.411 INFO:tasks.workunit.client.0.smithi139.stdout:0/275: link f3 d6/f52 0 2024-09-16T00:09:41.411 INFO:tasks.workunit.client.0.smithi139.stdout:5/264: read d1/f28 [1680476,22481] 0 2024-09-16T00:09:41.411 INFO:tasks.workunit.client.0.smithi139.stdout:6/182: creat d8/db/d26/d28/d2a/f43 x:0 0 0 2024-09-16T00:09:41.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.407+0000 7f4b12837640 1 -- 172.21.15.139:0/4104911171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b0c06fb50 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.407+0000 7f4b097fa640 1 -- 172.21.15.139:0/4104911171 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4af4007400 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.409+0000 7f4b097fa640 1 --2- 172.21.15.139:0/4104911171 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4aec06c2b0 0x7f4aec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:41.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.409+0000 7f4b097fa640 1 -- 172.21.15.139:0/4104911171 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5973+0+0 (secure 0 0 0) 0x7f4af408a750 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.409+0000 7f4b0bfff640 1 --2- 172.21.15.139:0/4104911171 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4aec06c2b0 0x7f4aec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:41.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.409+0000 7f4b0bfff640 1 --2- 172.21.15.139:0/4104911171 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4aec06c2b0 0x7f4aec06e770 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0011e0 tx=0x7f4afc007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:41.413 INFO:tasks.workunit.client.0.smithi139.stdout:5/265: link d1/d13/d22/l2a d1/d25/d30/d33/l56 0 2024-09-16T00:09:41.417 INFO:tasks.workunit.client.0.smithi139.stdout:2/208: dwrite d11/f37 [0,4194304] 0 2024-09-16T00:09:41.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.415+0000 7f4b097fa640 1 -- 172.21.15.139:0/4104911171 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4b0c06fb50 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.420 INFO:tasks.workunit.client.0.smithi139.stdout:2/209: creat d11/d30/f42 x:0 0 0 2024-09-16T00:09:41.422 INFO:tasks.workunit.client.0.smithi139.stdout:5/266: dwrite d1/d25/f2e [4194304,4194304] 0 2024-09-16T00:09:41.430 INFO:tasks.workunit.client.0.smithi139.stdout:2/210: dwrite d11/f22 [0,4194304] 0 2024-09-16T00:09:41.431 INFO:tasks.workunit.client.0.smithi139.stdout:5/267: dwrite d1/f28 [0,4194304] 0 2024-09-16T00:09:41.431 INFO:tasks.workunit.client.0.smithi139.stdout:2/211: write fe [592623,64464] 0 2024-09-16T00:09:41.432 INFO:tasks.workunit.client.0.smithi139.stdout:2/212: stat d11/f1e 0 2024-09-16T00:09:41.436 INFO:tasks.workunit.client.0.smithi139.stdout:1/208: sync 2024-09-16T00:09:41.438 INFO:tasks.workunit.client.0.smithi139.stdout:2/213: read d11/d1f/d29/d2b/f2e [2980509,26610] 0 2024-09-16T00:09:41.440 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:41.439+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:41.443 INFO:tasks.workunit.client.0.smithi139.stdout:1/209: unlink d7/fd 0 2024-09-16T00:09:41.444 INFO:tasks.workunit.client.0.smithi139.stdout:1/210: chown d7/d8 1872518 1 2024-09-16T00:09:41.445 INFO:tasks.workunit.client.0.smithi139.stdout:1/211: stat d7/d8/f1a 0 2024-09-16T00:09:41.447 INFO:tasks.workunit.client.0.smithi139.stdout:2/214: getdents d11/d30 0 2024-09-16T00:09:41.448 INFO:tasks.workunit.client.0.smithi139.stdout:2/215: write ff [3183359,75286] 0 2024-09-16T00:09:41.449 INFO:tasks.workunit.client.0.smithi139.stdout:1/212: rename d7/d8/f1c to d7/d1b/f3a 0 2024-09-16T00:09:41.452 INFO:tasks.workunit.client.0.smithi139.stdout:2/216: creat d11/d32/f43 x:0 0 0 2024-09-16T00:09:41.453 INFO:tasks.workunit.client.0.smithi139.stdout:1/213: symlink d7/d8/d1f/l3b 0 2024-09-16T00:09:41.455 INFO:tasks.workunit.client.0.smithi139.stdout:5/268: dwrite d1/f1a [0,4194304] 0 2024-09-16T00:09:41.456 INFO:tasks.workunit.client.0.smithi139.stdout:2/217: creat d11/d30/f44 x:0 0 0 2024-09-16T00:09:41.458 INFO:tasks.workunit.client.0.smithi139.stdout:1/214: rename d7/d8/d1f/f24 to d7/d1b/f3c 0 2024-09-16T00:09:41.459 INFO:tasks.workunit.client.0.smithi139.stdout:2/218: mkdir d11/d45 0 2024-09-16T00:09:41.460 INFO:tasks.workunit.client.0.smithi139.stdout:5/269: symlink d1/d25/d30/d33/d43/l57 0 2024-09-16T00:09:41.460 INFO:tasks.workunit.client.0.smithi139.stdout:2/219: stat d11/d1f/d29/d2b/f2c 0 2024-09-16T00:09:41.460 INFO:tasks.workunit.client.0.smithi139.stdout:1/215: mknod d7/d1b/c3d 0 2024-09-16T00:09:41.463 INFO:tasks.workunit.client.0.smithi139.stdout:1/216: write d7/f26 [1215933,52351] 0 2024-09-16T00:09:41.464 INFO:tasks.workunit.client.0.smithi139.stdout:1/217: dread - d7/d8/d27/f32 zero size 2024-09-16T00:09:41.464 INFO:tasks.workunit.client.0.smithi139.stdout:2/220: rename d11/d1f/l35 to d11/l46 0 2024-09-16T00:09:41.465 INFO:tasks.workunit.client.0.smithi139.stdout:2/221: readlink d11/l17 0 2024-09-16T00:09:41.465 INFO:tasks.workunit.client.0.smithi139.stdout:5/270: link d1/f27 d1/d25/d30/f58 0 2024-09-16T00:09:41.465 INFO:tasks.workunit.client.0.smithi139.stdout:2/222: dread - d11/d1f/d29/d2b/f34 zero size 2024-09-16T00:09:41.468 INFO:tasks.workunit.client.0.smithi139.stdout:2/223: mknod d11/d30/c47 0 2024-09-16T00:09:41.469 INFO:tasks.workunit.client.0.smithi139.stdout:2/224: mknod d11/d2f/c48 0 2024-09-16T00:09:41.472 INFO:tasks.workunit.client.0.smithi139.stdout:2/225: link c0 d11/d1f/c49 0 2024-09-16T00:09:41.472 INFO:tasks.workunit.client.0.smithi139.stdout:2/226: write d11/d30/f36 [3395424,93838] 0 2024-09-16T00:09:41.474 INFO:tasks.workunit.client.0.smithi139.stdout:2/227: read d11/f1e [505598,19579] 0 2024-09-16T00:09:41.476 INFO:tasks.workunit.client.0.smithi139.stdout:2/228: unlink d11/f37 0 2024-09-16T00:09:41.479 INFO:tasks.workunit.client.0.smithi139.stdout:5/271: write d1/d21/f39 [1805854,54167] 0 2024-09-16T00:09:41.479 INFO:tasks.workunit.client.0.smithi139.stdout:2/229: link c10 d11/d1f/d29/c4a 0 2024-09-16T00:09:41.482 INFO:tasks.workunit.client.0.smithi139.stdout:2/230: write d11/f16 [105122,115377] 0 2024-09-16T00:09:41.484 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --force --op remove --pgid 2.17 2024-09-16T00:09:41.485 INFO:tasks.workunit.client.0.smithi139.stdout:1/218: dwrite d7/d8/d27/f34 [4194304,4194304] 0 2024-09-16T00:09:41.486 INFO:tasks.workunit.client.0.smithi139.stdout:2/231: symlink d11/d45/l4b 0 2024-09-16T00:09:41.488 INFO:tasks.workunit.client.0.smithi139.stdout:2/232: read d11/f22 [211914,83180] 0 2024-09-16T00:09:41.492 INFO:tasks.workunit.client.0.smithi139.stdout:5/272: dwrite d1/d25/f2e [4194304,4194304] 0 2024-09-16T00:09:41.494 INFO:tasks.workunit.client.0.smithi139.stdout:1/219: rename d7/d8/d1f/l3b to d7/d8/l3e 0 2024-09-16T00:09:41.495 INFO:tasks.workunit.client.0.smithi139.stdout:5/273: mknod d1/d13/d3f/c59 0 2024-09-16T00:09:41.498 INFO:tasks.workunit.client.0.smithi139.stdout:5/274: write d1/f16 [5043093,70799] 0 2024-09-16T00:09:41.501 INFO:tasks.workunit.client.0.smithi139.stdout:1/220: link d7/d8/d1f/f37 d7/f3f 0 2024-09-16T00:09:41.501 INFO:tasks.workunit.client.0.smithi139.stdout:5/275: mkdir d1/d25/d30/d33/d43/d5a 0 2024-09-16T00:09:41.505 INFO:tasks.workunit.client.0.smithi139.stdout:1/221: write d7/f12 [4228425,100128] 0 2024-09-16T00:09:41.507 INFO:tasks.workunit.client.0.smithi139.stdout:2/233: dread f8 [0,4194304] 0 2024-09-16T00:09:41.508 INFO:tasks.workunit.client.0.smithi139.stdout:1/222: mkdir d7/d8/d40 0 2024-09-16T00:09:41.510 INFO:tasks.workunit.client.0.smithi139.stdout:5/276: truncate d1/d13/d3b/f47 1956370 0 2024-09-16T00:09:41.511 INFO:tasks.workunit.client.0.smithi139.stdout:2/234: creat d11/d30/f4c x:0 0 0 2024-09-16T00:09:41.511 INFO:tasks.workunit.client.0.smithi139.stdout:2/235: readlink d11/d2f/l40 0 2024-09-16T00:09:41.511 INFO:tasks.workunit.client.0.smithi139.stdout:2/236: readlink d11/d32/l3c 0 2024-09-16T00:09:41.512 INFO:tasks.workunit.client.0.smithi139.stdout:1/223: unlink d7/d8/d1f/f39 0 2024-09-16T00:09:41.512 INFO:tasks.workunit.client.0.smithi139.stdout:2/237: fsync d11/d1f/f27 0 2024-09-16T00:09:41.513 INFO:tasks.workunit.client.0.smithi139.stdout:5/277: mkdir d1/d25/d30/d33/d5b 0 2024-09-16T00:09:41.517 INFO:tasks.workunit.client.0.smithi139.stdout:2/238: link d11/d1f/l28 d11/d1f/d29/d2b/l4d 0 2024-09-16T00:09:41.520 INFO:tasks.workunit.client.0.smithi139.stdout:1/224: dwrite d7/f26 [0,4194304] 0 2024-09-16T00:09:41.521 INFO:tasks.workunit.client.0.smithi139.stdout:5/278: dwrite d1/d21/f39 [0,4194304] 0 2024-09-16T00:09:41.522 INFO:tasks.workunit.client.0.smithi139.stdout:2/239: mknod d11/d1f/d29/d2b/c4e 0 2024-09-16T00:09:41.523 INFO:tasks.workunit.client.0.smithi139.stdout:5/279: chown d1/d25/d30/d33/d43/f4d 394420367 1 2024-09-16T00:09:41.523 INFO:tasks.workunit.client.0.smithi139.stdout:2/240: chown d11/d32/l3c 6 1 2024-09-16T00:09:41.525 INFO:tasks.workunit.client.0.smithi139.stdout:5/280: creat d1/d25/f5c x:0 0 0 2024-09-16T00:09:41.526 INFO:tasks.workunit.client.0.smithi139.stdout:2/241: mkdir d11/d1f/d29/d4f 0 2024-09-16T00:09:41.528 INFO:tasks.workunit.client.0.smithi139.stdout:2/242: mkdir d11/d1f/d50 0 2024-09-16T00:09:41.529 INFO:tasks.workunit.client.0.smithi139.stdout:1/225: dwrite d7/d8/d27/f34 [0,4194304] 0 2024-09-16T00:09:41.531 INFO:tasks.workunit.client.0.smithi139.stdout:2/243: symlink d11/d1f/l51 0 2024-09-16T00:09:41.532 INFO:tasks.workunit.client.0.smithi139.stdout:2/244: read - d11/d1f/d29/d2b/f34 zero size 2024-09-16T00:09:41.533 INFO:tasks.workunit.client.0.smithi139.stdout:1/226: creat d7/d8/d27/d2f/f41 x:0 0 0 2024-09-16T00:09:41.536 INFO:tasks.workunit.client.0.smithi139.stdout:1/227: dread - d7/d8/f36 zero size 2024-09-16T00:09:41.536 INFO:tasks.workunit.client.0.smithi139.stdout:5/281: write d1/d13/f24 [1430524,76198] 0 2024-09-16T00:09:41.537 INFO:tasks.workunit.client.0.smithi139.stdout:2/245: getdents d11/d30 0 2024-09-16T00:09:41.542 INFO:tasks.workunit.client.0.smithi139.stdout:1/228: write d7/f10 [5228809,47051] 0 2024-09-16T00:09:41.542 INFO:tasks.workunit.client.0.smithi139.stdout:2/246: mknod d11/c52 0 2024-09-16T00:09:41.542 INFO:tasks.workunit.client.0.smithi139.stdout:5/282: symlink d1/d13/d22/l5d 0 2024-09-16T00:09:41.542 INFO:tasks.workunit.client.0.smithi139.stdout:2/247: dread - d11/d30/f42 zero size 2024-09-16T00:09:41.542 INFO:tasks.workunit.client.0.smithi139.stdout:1/229: symlink d7/d8/d1f/l42 0 2024-09-16T00:09:41.545 INFO:tasks.workunit.client.0.smithi139.stdout:5/283: rmdir d1/d21/d49/d53 39 2024-09-16T00:09:41.545 INFO:tasks.workunit.client.0.smithi139.stdout:1/230: fdatasync d7/f26 0 2024-09-16T00:09:41.545 INFO:tasks.workunit.client.0.smithi139.stdout:2/248: truncate d11/d30/f42 675279 0 2024-09-16T00:09:41.546 INFO:tasks.workunit.client.0.smithi139.stdout:1/231: write d7/d8/f36 [407969,21505] 0 2024-09-16T00:09:41.547 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:41.543+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:41.548 INFO:tasks.workunit.client.0.smithi139.stdout:5/284: mkdir d1/d13/d3f/d5e 0 2024-09-16T00:09:41.548 INFO:tasks.workunit.client.0.smithi139.stdout:2/249: creat d11/f53 x:0 0 0 2024-09-16T00:09:41.550 INFO:tasks.workunit.client.0.smithi139.stdout:2/250: creat d11/d2f/f54 x:0 0 0 2024-09-16T00:09:41.551 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:41.551 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:41.551 INFO:tasks.workunit.client.0.smithi139.stdout:5/285: getdents d1/d25/d30/d33/d43/d52 0 2024-09-16T00:09:41.552 INFO:tasks.workunit.client.0.smithi139.stdout:5/286: write d1/d21/f3a [206826,87010] 0 2024-09-16T00:09:41.552 INFO:tasks.workunit.client.0.smithi139.stdout:1/232: dread d7/d8/f11 [0,4194304] 0 2024-09-16T00:09:41.552 INFO:tasks.workunit.client.0.smithi139.stdout:2/251: mknod d11/d1f/d29/d2b/c55 0 2024-09-16T00:09:41.553 INFO:tasks.workunit.client.0.smithi139.stdout:1/233: fdatasync d7/f10 0 2024-09-16T00:09:41.554 INFO:tasks.workunit.client.0.smithi139.stdout:2/252: creat d11/d30/f56 x:0 0 0 2024-09-16T00:09:41.556 INFO:tasks.workunit.client.0.smithi139.stdout:2/253: write d11/f1e [565009,24872] 0 2024-09-16T00:09:41.556 INFO:tasks.workunit.client.0.smithi139.stdout:1/234: symlink d7/d8/d27/d2f/l43 0 2024-09-16T00:09:41.558 INFO:tasks.workunit.client.0.smithi139.stdout:1/235: symlink d7/d8/l44 0 2024-09-16T00:09:41.559 INFO:tasks.workunit.client.0.smithi139.stdout:5/287: dwrite d1/f48 [0,4194304] 0 2024-09-16T00:09:41.564 INFO:tasks.workunit.client.0.smithi139.stdout:5/288: rename d1/d21/f3a to d1/d13/d3b/d3e/f5f 0 2024-09-16T00:09:41.565 INFO:tasks.workunit.client.0.smithi139.stdout:2/254: dwrite d11/d2f/f33 [0,4194304] 0 2024-09-16T00:09:41.567 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:09:41.567 INFO:tasks.workunit.client.0.smithi139.stdout:5/289: read - d1/d25/d30/f55 zero size 2024-09-16T00:09:41.568 INFO:tasks.workunit.client.0.smithi139.stdout:2/255: truncate d11/d30/f56 506050 0 2024-09-16T00:09:41.568 INFO:tasks.workunit.client.0.smithi139.stdout:5/290: chown d1/d13/d22/l45 10888 1 2024-09-16T00:09:41.569 INFO:tasks.workunit.client.0.smithi139.stdout:2/256: write d11/d1f/d29/d2b/f34 [696018,28918] 0 2024-09-16T00:09:41.571 INFO:tasks.workunit.client.0.smithi139.stdout:1/236: truncate d7/d8/f1e 2823004 0 2024-09-16T00:09:41.571 INFO:tasks.workunit.client.0.smithi139.stdout:2/257: dread d11/d30/f56 [0,4194304] 0 2024-09-16T00:09:41.574 INFO:tasks.workunit.client.0.smithi139.stdout:2/258: rename d11/d1f to d11/d30/d57 0 2024-09-16T00:09:41.575 INFO:tasks.workunit.client.0.smithi139.stdout:2/259: write d11/d30/f42 [1141569,116808] 0 2024-09-16T00:09:41.575 INFO:tasks.workunit.client.0.smithi139.stdout:1/237: rename d7/d8/l23 to d7/d8/l45 0 2024-09-16T00:09:41.577 INFO:tasks.workunit.client.0.smithi139.stdout:1/238: write d7/d8/f21 [918398,92108] 0 2024-09-16T00:09:41.577 INFO:tasks.workunit.client.0.smithi139.stdout:1/239: readlink d7/d8/d1f/l42 0 2024-09-16T00:09:41.580 INFO:tasks.workunit.client.0.smithi139.stdout:1/240: mkdir d7/d8/d40/d46 0 2024-09-16T00:09:41.580 INFO:tasks.workunit.client.0.smithi139.stdout:2/260: dread d11/d30/d57/d29/d2b/f2d [0,4194304] 0 2024-09-16T00:09:41.581 INFO:tasks.workunit.client.0.smithi139.stdout:2/261: read - d11/f38 zero size 2024-09-16T00:09:41.582 INFO:tasks.workunit.client.0.smithi139.stdout:1/241: mkdir d7/d8/d40/d47 0 2024-09-16T00:09:41.582 INFO:tasks.workunit.client.0.smithi139.stdout:1/242: truncate d7/f3f 439407 0 2024-09-16T00:09:41.584 INFO:tasks.workunit.client.0.smithi139.stdout:1/243: dread - d7/d8/d27/f32 zero size 2024-09-16T00:09:41.585 INFO:tasks.workunit.client.0.smithi139.stdout:3/258: sync 2024-09-16T00:09:41.585 INFO:tasks.workunit.client.0.smithi139.stdout:8/316: sync 2024-09-16T00:09:41.585 INFO:tasks.workunit.client.0.smithi139.stdout:4/171: sync 2024-09-16T00:09:41.586 INFO:tasks.workunit.client.0.smithi139.stdout:1/244: chown d7/l2e 172429 1 2024-09-16T00:09:41.587 INFO:tasks.workunit.client.0.smithi139.stdout:1/245: dread - d7/d8/d27/f32 zero size 2024-09-16T00:09:41.588 INFO:tasks.workunit.client.0.smithi139.stdout:4/172: symlink d12/d2b/l36 0 2024-09-16T00:09:41.589 INFO:tasks.workunit.client.0.smithi139.stdout:2/262: dwrite d11/d30/d57/d29/d2b/f2d [0,4194304] 0 2024-09-16T00:09:41.590 INFO:tasks.workunit.client.0.smithi139.stdout:2/263: fdatasync d11/d30/f36 0 2024-09-16T00:09:41.590 INFO:tasks.workunit.client.0.smithi139.stdout:1/246: mkdir d7/d8/d40/d48 0 2024-09-16T00:09:41.595 INFO:tasks.workunit.client.0.smithi139.stdout:2/264: mknod d11/d30/d57/d29/c58 0 2024-09-16T00:09:41.598 INFO:tasks.workunit.client.0.smithi139.stdout:2/265: creat d11/d30/d57/f59 x:0 0 0 2024-09-16T00:09:41.601 INFO:tasks.workunit.client.0.smithi139.stdout:3/259: dwrite d5/f8 [0,4194304] 0 2024-09-16T00:09:41.602 INFO:tasks.workunit.client.0.smithi139.stdout:1/247: dwrite d7/d8/d1f/f38 [0,4194304] 0 2024-09-16T00:09:41.608 INFO:tasks.workunit.client.0.smithi139.stdout:1/248: symlink d7/d8/d27/l49 0 2024-09-16T00:09:41.608 INFO:tasks.workunit.client.0.smithi139.stdout:2/266: rename d11/d30/d57/l28 to d11/d30/d57/d50/l5a 0 2024-09-16T00:09:41.609 INFO:tasks.workunit.client.0.smithi139.stdout:2/267: chown d11/d30/d57 384 1 2024-09-16T00:09:41.611 INFO:tasks.workunit.client.0.smithi139.stdout:2/268: dread - d11/d30/d57/d29/d2b/f2c zero size 2024-09-16T00:09:41.611 INFO:tasks.workunit.client.0.smithi139.stdout:1/249: mkdir d7/d8/d40/d48/d4a 0 2024-09-16T00:09:41.613 INFO:tasks.workunit.client.0.smithi139.stdout:3/260: rename d5/d7/f26 to d5/f53 0 2024-09-16T00:09:41.613 INFO:tasks.workunit.client.0.smithi139.stdout:2/269: unlink d11/c52 0 2024-09-16T00:09:41.615 INFO:tasks.workunit.client.0.smithi139.stdout:2/270: write d11/d30/d57/f27 [424842,79652] 0 2024-09-16T00:09:41.615 INFO:tasks.workunit.client.0.smithi139.stdout:1/250: creat d7/d8/d40/d48/d4a/f4b x:0 0 0 2024-09-16T00:09:41.616 INFO:tasks.workunit.client.0.smithi139.stdout:2/271: write d11/d30/f44 [50674,105800] 0 2024-09-16T00:09:41.616 INFO:tasks.workunit.client.0.smithi139.stdout:3/261: creat d5/d7/d23/d27/d2f/d30/d4c/d50/f54 x:0 0 0 2024-09-16T00:09:41.620 INFO:tasks.workunit.client.0.smithi139.stdout:3/262: mknod d5/d7/d23/d27/d2f/c55 0 2024-09-16T00:09:41.623 INFO:tasks.workunit.client.0.smithi139.stdout:1/251: dwrite d7/f12 [0,4194304] 0 2024-09-16T00:09:41.624 INFO:tasks.workunit.client.0.smithi139.stdout:3/263: creat d5/d7/d23/d48/f56 x:0 0 0 2024-09-16T00:09:41.625 INFO:tasks.workunit.client.0.smithi139.stdout:2/272: dwrite d11/d30/d57/f24 [0,4194304] 0 2024-09-16T00:09:41.627 INFO:tasks.workunit.client.0.smithi139.stdout:3/264: write d5/d7/f2e [310359,86109] 0 2024-09-16T00:09:41.631 INFO:tasks.workunit.client.0.smithi139.stdout:2/273: write d11/f1a [1622794,77584] 0 2024-09-16T00:09:41.636 INFO:tasks.workunit.client.0.smithi139.stdout:1/252: dwrite d7/f12 [0,4194304] 0 2024-09-16T00:09:41.638 INFO:tasks.workunit.client.0.smithi139.stdout:2/274: unlink d11/l3b 0 2024-09-16T00:09:41.639 INFO:tasks.workunit.client.0.smithi139.stdout:1/253: write d7/d8/d27/d2f/f41 [371650,122708] 0 2024-09-16T00:09:41.640 INFO:tasks.workunit.client.0.smithi139.stdout:1/254: write d7/f10 [1174249,17901] 0 2024-09-16T00:09:41.641 INFO:tasks.workunit.client.0.smithi139.stdout:2/275: symlink d11/d2f/l5b 0 2024-09-16T00:09:41.642 INFO:tasks.workunit.client.0.smithi139.stdout:3/265: getdents d5/d7/d23/d27/d2f/d30/d47 0 2024-09-16T00:09:41.644 INFO:tasks.workunit.client.0.smithi139.stdout:1/255: creat d7/d8/d40/d48/d4a/f4c x:0 0 0 2024-09-16T00:09:41.645 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:41.644+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:41.646 INFO:tasks.workunit.client.0.smithi139.stdout:3/266: symlink d5/l57 0 2024-09-16T00:09:41.648 INFO:tasks.workunit.client.0.smithi139.stdout:3/267: stat d5/d7/d23/d27 0 2024-09-16T00:09:41.650 INFO:tasks.workunit.client.0.smithi139.stdout:1/256: write d7/d8/f11 [1098121,64154] 0 2024-09-16T00:09:41.653 INFO:tasks.workunit.client.0.smithi139.stdout:2/276: dwrite d11/f1e [0,4194304] 0 2024-09-16T00:09:41.653 INFO:tasks.workunit.client.0.smithi139.stdout:3/268: mknod d5/d7/d23/d27/d2f/d30/d4c/c58 0 2024-09-16T00:09:41.654 INFO:tasks.workunit.client.0.smithi139.stdout:2/277: chown d11/c1c 1 1 2024-09-16T00:09:41.654 INFO:tasks.workunit.client.0.smithi139.stdout:1/257: rename l3 to d7/d8/d27/l4d 0 2024-09-16T00:09:41.656 INFO:tasks.workunit.client.0.smithi139.stdout:3/269: creat d5/d7/d23/d27/d2f/d30/d4c/d50/f59 x:0 0 0 2024-09-16T00:09:41.657 INFO:tasks.workunit.client.0.smithi139.stdout:1/258: mkdir d7/d8/d4e 0 2024-09-16T00:09:41.659 INFO:tasks.workunit.client.0.smithi139.stdout:3/270: mknod d5/d7/d23/d27/d2f/d30/d4d/c5a 0 2024-09-16T00:09:41.659 INFO:tasks.workunit.client.0.smithi139.stdout:2/278: dread d11/f1a [0,4194304] 0 2024-09-16T00:09:41.660 INFO:tasks.workunit.client.0.smithi139.stdout:2/279: dread - d11/d32/f43 zero size 2024-09-16T00:09:41.661 INFO:tasks.workunit.client.0.smithi139.stdout:2/280: write d11/f15 [4037730,81976] 0 2024-09-16T00:09:41.661 INFO:tasks.workunit.client.0.smithi139.stdout:3/271: symlink d5/d7/d23/d27/d2f/l5b 0 2024-09-16T00:09:41.662 INFO:tasks.workunit.client.0.smithi139.stdout:2/281: stat d11/d30/d57/d29/d2b 0 2024-09-16T00:09:41.668 INFO:tasks.workunit.client.0.smithi139.stdout:3/272: link d5/d7/d23/d48/l3f d5/d7/d23/d27/d2f/d30/d41/l5c 0 2024-09-16T00:09:41.671 INFO:tasks.workunit.client.0.smithi139.stdout:3/273: link d5/d7/c3a d5/d7/d23/d27/d2f/d30/c5d 0 2024-09-16T00:09:41.673 INFO:tasks.workunit.client.0.smithi139.stdout:1/259: dread d7/d1b/f3a [0,4194304] 0 2024-09-16T00:09:41.674 INFO:tasks.workunit.client.0.smithi139.stdout:3/274: symlink d5/d7/d23/d27/d2f/d30/d4c/d50/l5e 0 2024-09-16T00:09:41.680 INFO:tasks.workunit.client.0.smithi139.stdout:3/275: dwrite d5/d7/d23/d27/d2f/d30/f36 [0,4194304] 0 2024-09-16T00:09:41.684 INFO:tasks.workunit.client.0.smithi139.stdout:3/276: rename d5/d7/d23/d27/d2f/d30/d40 to d5/d5f 0 2024-09-16T00:09:41.687 INFO:tasks.workunit.client.0.smithi139.stdout:3/277: creat d5/d7/d23/d27/d2f/d30/d4d/f60 x:0 0 0 2024-09-16T00:09:41.690 INFO:tasks.workunit.client.0.smithi139.stdout:1/260: dwrite d7/d1b/f3c [0,4194304] 0 2024-09-16T00:09:41.690 INFO:tasks.workunit.client.0.smithi139.stdout:3/278: creat d5/d7/d23/d35/f61 x:0 0 0 2024-09-16T00:09:41.691 INFO:tasks.workunit.client.0.smithi139.stdout:1/261: readlink d7/d8/d27/d2f/l43 0 2024-09-16T00:09:41.691 INFO:tasks.workunit.client.0.smithi139.stdout:3/279: write d5/d7/dc/f12 [4390953,68494] 0 2024-09-16T00:09:41.693 INFO:tasks.workunit.client.0.smithi139.stdout:3/280: write d5/d7/d23/d35/f61 [1000524,73257] 0 2024-09-16T00:09:41.695 INFO:tasks.workunit.client.0.smithi139.stdout:3/281: chown d5/d7/d23/d27/d2f/c31 693293198 1 2024-09-16T00:09:41.699 INFO:tasks.workunit.client.0.smithi139.stdout:1/262: dwrite d7/d8/d27/f32 [0,4194304] 0 2024-09-16T00:09:41.705 INFO:tasks.workunit.client.0.smithi139.stdout:3/282: dwrite d5/d7/d23/d27/d2f/d30/d47/f25 [0,4194304] 0 2024-09-16T00:09:41.706 INFO:tasks.workunit.client.0.smithi139.stdout:1/263: link d7/d8/l45 d7/d8/d27/d2f/l4f 0 2024-09-16T00:09:41.709 INFO:tasks.workunit.client.0.smithi139.stdout:3/283: write d5/d7/f13 [1009928,129006] 0 2024-09-16T00:09:41.711 INFO:tasks.workunit.client.0.smithi139.stdout:3/284: dread - d5/d7/d23/d27/d2f/d30/f42 zero size 2024-09-16T00:09:41.712 INFO:tasks.workunit.client.0.smithi139.stdout:1/264: unlink d7/d8/d1f/c2a 0 2024-09-16T00:09:41.712 INFO:tasks.workunit.client.0.smithi139.stdout:3/285: fsync d5/f8 0 2024-09-16T00:09:41.714 INFO:tasks.workunit.client.0.smithi139.stdout:3/286: write d5/d7/d23/f15 [5075779,83017] 0 2024-09-16T00:09:41.717 INFO:tasks.workunit.client.0.smithi139.stdout:3/287: dread - d5/d7/f49 zero size 2024-09-16T00:09:41.720 INFO:tasks.workunit.client.0.smithi139.stdout:3/288: symlink d5/l62 0 2024-09-16T00:09:41.721 INFO:tasks.workunit.client.0.smithi139.stdout:1/265: dwrite d7/f26 [0,4194304] 0 2024-09-16T00:09:41.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:41.722+0000 7f4b12837640 1 -- 172.21.15.139:0/4104911171 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f4b0c0c6980 con 0x7f4b0c0c7fb0 2024-09-16T00:09:41.724 INFO:tasks.workunit.client.0.smithi139.stdout:1/266: chown d7/d8/f1e 122671582 1 2024-09-16T00:09:41.725 INFO:tasks.workunit.client.0.smithi139.stdout:1/267: readlink d7/d8/d1f/l42 0 2024-09-16T00:09:41.727 INFO:tasks.workunit.client.0.smithi139.stdout:3/289: dwrite d5/d7/d23/d48/f56 [0,4194304] 0 2024-09-16T00:09:41.728 INFO:tasks.workunit.client.0.smithi139.stdout:1/268: creat d7/d8/d40/d47/f50 x:0 0 0 2024-09-16T00:09:41.732 INFO:tasks.workunit.client.0.smithi139.stdout:1/269: dread d7/f26 [0,4194304] 0 2024-09-16T00:09:41.734 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:41.734 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:41.735 INFO:tasks.workunit.client.0.smithi139.stdout:1/270: dread d7/f3f [0,4194304] 0 2024-09-16T00:09:41.737 INFO:tasks.workunit.client.0.smithi139.stdout:1/271: rmdir d7/d8/d40/d47 39 2024-09-16T00:09:41.738 INFO:tasks.workunit.client.0.smithi139.stdout:1/272: mknod d7/d8/d27/d2f/c51 0 2024-09-16T00:09:41.740 INFO:tasks.workunit.client.0.smithi139.stdout:1/273: unlink d7/d8/d27/f32 0 2024-09-16T00:09:41.745 INFO:tasks.workunit.client.0.smithi139.stdout:1/274: symlink d7/d8/d40/d48/d4a/l52 0 2024-09-16T00:09:41.745 INFO:tasks.workunit.client.0.smithi139.stdout:3/290: dread d5/f16 [4194304,4194304] 0 2024-09-16T00:09:41.745 INFO:tasks.workunit.client.0.smithi139.stdout:1/275: truncate d7/d8/d40/d48/d4a/f4b 299174 0 2024-09-16T00:09:41.746 INFO:tasks.workunit.client.0.smithi139.stdout:3/291: creat d5/d7/dc/f63 x:0 0 0 2024-09-16T00:09:41.747 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:09:41.747 INFO:tasks.workunit.client.0.smithi139.stdout:1/276: mkdir d7/d8/d40/d48/d53 0 2024-09-16T00:09:41.747 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:41.744+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:41.748 INFO:tasks.workunit.client.0.smithi139.stdout:3/292: rename d5/d7/d23/d27/d2f/d30/d47/f24 to d5/d7/d23/d27/d2f/d30/d47/f64 0 2024-09-16T00:09:41.750 INFO:tasks.workunit.client.0.smithi139.stdout:3/293: mkdir d5/d7/d23/d48/d65 0 2024-09-16T00:09:41.750 INFO:tasks.workunit.client.0.smithi139.stdout:1/277: dwrite d7/d1b/f3c [0,4194304] 0 2024-09-16T00:09:41.752 INFO:tasks.workunit.client.0.smithi139.stdout:3/294: readlink d5/d7/d23/d27/d2f/d30/l52 0 2024-09-16T00:09:41.754 INFO:tasks.workunit.client.0.smithi139.stdout:1/278: dread d7/d8/d27/f34 [0,4194304] 0 2024-09-16T00:09:41.754 INFO:tasks.workunit.client.0.smithi139.stdout:3/295: mknod d5/d7/d23/c66 0 2024-09-16T00:09:41.755 INFO:tasks.workunit.client.0.smithi139.stdout:1/279: chown d7/d1b/f3a 3132523 1 2024-09-16T00:09:41.756 INFO:tasks.workunit.client.0.smithi139.stdout:3/296: rename d5/d7/d23/d27/d2f/d30/d4d to d5/d7/d23/d27/d2f/d30/d4d/d67 22 2024-09-16T00:09:41.763 INFO:tasks.workunit.client.0.smithi139.stdout:3/297: dread d5/d7/d23/d48/f56 [0,4194304] 0 2024-09-16T00:09:41.764 INFO:tasks.workunit.client.0.smithi139.stdout:1/280: dwrite d7/f28 [4194304,4194304] 0 2024-09-16T00:09:41.766 INFO:tasks.workunit.client.0.smithi139.stdout:1/281: chown d7/d8/d27/f34 15 1 2024-09-16T00:09:41.768 INFO:tasks.workunit.client.0.smithi139.stdout:3/298: rename d5/d7/d23/d48/d65 to d5/d7/d23/d27/d2f/d30/d41/d68 0 2024-09-16T00:09:41.768 INFO:tasks.workunit.client.0.smithi139.stdout:1/282: fdatasync d7/f26 0 2024-09-16T00:09:41.769 INFO:tasks.workunit.client.0.smithi139.stdout:1/283: chown d7/d8/d40/d48/d4a/f4b 0 1 2024-09-16T00:09:41.770 INFO:tasks.workunit.client.0.smithi139.stdout:3/299: creat d5/d7/d23/d27/d2f/d30/d4c/f69 x:0 0 0 2024-09-16T00:09:41.770 INFO:tasks.workunit.client.0.smithi139.stdout:3/300: fsync d5/d7/f37 0 2024-09-16T00:09:41.771 INFO:tasks.workunit.client.0.smithi139.stdout:1/284: creat d7/d1b/f54 x:0 0 0 2024-09-16T00:09:41.777 INFO:tasks.workunit.client.0.smithi139.stdout:1/285: dwrite d7/f12 [0,4194304] 0 2024-09-16T00:09:41.778 INFO:tasks.workunit.client.0.smithi139.stdout:3/301: dwrite d5/d7/d23/d27/d2f/d30/d4c/d50/f54 [0,4194304] 0 2024-09-16T00:09:41.779 INFO:tasks.workunit.client.0.smithi139.stdout:3/302: chown d5/d7/f49 417 1 2024-09-16T00:09:41.780 INFO:tasks.workunit.client.0.smithi139.stdout:7/222: sync 2024-09-16T00:09:41.780 INFO:tasks.workunit.client.0.smithi139.stdout:0/276: sync 2024-09-16T00:09:41.780 INFO:tasks.workunit.client.0.smithi139.stdout:9/212: sync 2024-09-16T00:09:41.780 INFO:tasks.workunit.client.0.smithi139.stdout:6/183: sync 2024-09-16T00:09:41.787 INFO:tasks.workunit.client.0.smithi139.stdout:9/213: mkdir d19/d1a/d20/d23/d2c/d3a 0 2024-09-16T00:09:41.788 INFO:tasks.workunit.client.0.smithi139.stdout:1/286: dwrite d7/d1b/f3a [0,4194304] 0 2024-09-16T00:09:41.789 INFO:tasks.workunit.client.0.smithi139.stdout:0/277: creat d6/d1c/d45/d49/f53 x:0 0 0 2024-09-16T00:09:41.790 INFO:tasks.workunit.client.0.smithi139.stdout:3/303: rename d5/d7/d23/d48/f56 to d5/d7/d23/d35/f6a 0 2024-09-16T00:09:41.791 INFO:tasks.workunit.client.0.smithi139.stdout:7/223: link f4 d5/d16/d22/f4e 0 2024-09-16T00:09:41.794 INFO:tasks.workunit.client.0.smithi139.stdout:9/214: symlink d19/d1a/d20/d23/d2c/d3a/l3b 0 2024-09-16T00:09:41.794 INFO:tasks.workunit.client.0.smithi139.stdout:3/304: chown d5/d7/d23/d27/d2f/d30/f36 2101 1 2024-09-16T00:09:41.796 INFO:tasks.workunit.client.0.smithi139.stdout:6/184: rename d8/d20/d27/c3e to d8/d1f/c44 0 2024-09-16T00:09:41.797 INFO:tasks.workunit.client.0.smithi139.stdout:9/215: mknod d19/d1a/d20/d23/d2d/c3c 0 2024-09-16T00:09:41.798 INFO:tasks.workunit.client.0.smithi139.stdout:6/185: chown d8/db/d26/d28/d2a 129875 1 2024-09-16T00:09:41.798 INFO:tasks.workunit.client.0.smithi139.stdout:0/278: getdents d6/d1c/d45 0 2024-09-16T00:09:41.798 INFO:tasks.workunit.client.0.smithi139.stdout:1/287: dwrite d7/d8/f1a [0,4194304] 0 2024-09-16T00:09:41.799 INFO:tasks.workunit.client.0.smithi139.stdout:3/305: creat d5/d7/d23/d27/d2f/d30/d41/f6b x:0 0 0 2024-09-16T00:09:41.800 INFO:tasks.workunit.client.0.smithi139.stdout:7/224: rename d5/f1c to d5/d16/d3d/d44/f4f 0 2024-09-16T00:09:41.800 INFO:tasks.workunit.client.0.smithi139.stdout:6/186: fdatasync d8/db/d26/d28/f3f 0 2024-09-16T00:09:41.801 INFO:tasks.workunit.client.0.smithi139.stdout:0/279: symlink d6/d1c/d45/l54 0 2024-09-16T00:09:41.802 INFO:tasks.workunit.client.0.smithi139.stdout:9/216: mkdir d19/d1a/d20/d23/d2d/d35/d3d 0 2024-09-16T00:09:41.802 INFO:tasks.workunit.client.0.smithi139.stdout:6/187: readlink d8/d20/d27/d10/l33 0 2024-09-16T00:09:41.802 INFO:tasks.workunit.client.0.smithi139.stdout:1/288: stat d7/d8/d40/d47 0 2024-09-16T00:09:41.803 INFO:tasks.workunit.client.0.smithi139.stdout:3/306: mknod d5/d5f/c6c 0 2024-09-16T00:09:41.804 INFO:tasks.workunit.client.0.smithi139.stdout:6/188: rmdir d8/db/d26/d28 39 2024-09-16T00:09:41.805 INFO:tasks.workunit.client.0.smithi139.stdout:0/280: dread d6/d1c/d41/d44/dd/f2c [0,4194304] 0 2024-09-16T00:09:41.806 INFO:tasks.workunit.client.0.smithi139.stdout:1/289: dread - d7/d8/d40/d47/f50 zero size 2024-09-16T00:09:41.807 INFO:tasks.workunit.client.0.smithi139.stdout:6/189: fsync d8/d29/f32 0 2024-09-16T00:09:41.807 INFO:tasks.workunit.client.0.smithi139.stdout:3/307: write d5/d7/d23/d27/d2f/d30/d47/f25 [5122097,20033] 0 2024-09-16T00:09:41.808 INFO:tasks.workunit.client.0.smithi139.stdout:7/225: getdents d5/d16 0 2024-09-16T00:09:41.810 INFO:tasks.workunit.client.0.smithi139.stdout:1/290: symlink d7/d8/d1f/l55 0 2024-09-16T00:09:41.810 INFO:tasks.workunit.client.0.smithi139.stdout:6/190: fsync d8/d1f/f41 0 2024-09-16T00:09:41.811 INFO:tasks.workunit.client.0.smithi139.stdout:3/308: stat d5/d7/dc 0 2024-09-16T00:09:41.812 INFO:tasks.workunit.client.0.smithi139.stdout:1/291: chown d7/d8/d27/d2f/f33 11859 1 2024-09-16T00:09:41.816 INFO:tasks.workunit.client.0.smithi139.stdout:7/226: rename d5/d16/d3d to d5/d16/d22/d25/d2d/d50 0 2024-09-16T00:09:41.817 INFO:tasks.workunit.client.0.smithi139.stdout:0/281: dwrite d6/d1c/d41/d44/dd/f39 [0,4194304] 0 2024-09-16T00:09:41.817 INFO:tasks.workunit.client.0.smithi139.stdout:1/292: stat d7/d8/d27/c2d 0 2024-09-16T00:09:41.820 INFO:tasks.workunit.client.0.smithi139.stdout:9/217: dread d19/f1b [0,4194304] 0 2024-09-16T00:09:41.821 INFO:tasks.workunit.client.0.smithi139.stdout:9/218: chown d19/f26 13863126 1 2024-09-16T00:09:41.822 INFO:tasks.workunit.client.0.smithi139.stdout:9/219: stat f14 0 2024-09-16T00:09:41.823 INFO:tasks.workunit.client.0.smithi139.stdout:0/282: mknod d6/d1c/d41/c55 0 2024-09-16T00:09:41.824 INFO:tasks.workunit.client.0.smithi139.stdout:0/283: chown d6/l32 968 1 2024-09-16T00:09:41.825 INFO:tasks.workunit.client.0.smithi139.stdout:1/293: creat d7/d8/d40/d47/f56 x:0 0 0 2024-09-16T00:09:41.825 INFO:tasks.workunit.client.0.smithi139.stdout:1/294: chown d7 13525655 1 2024-09-16T00:09:41.826 INFO:tasks.workunit.client.0.smithi139.stdout:3/309: truncate d5/d7/dc/f12 7347051 0 2024-09-16T00:09:41.827 INFO:tasks.workunit.client.0.smithi139.stdout:1/295: truncate d7/d8/d40/d48/d4a/f4c 103531 0 2024-09-16T00:09:41.828 INFO:tasks.workunit.client.0.smithi139.stdout:6/191: rename d8/db/d26/d28/d2a to d8/db/d26/d45 0 2024-09-16T00:09:41.828 INFO:tasks.workunit.client.0.smithi139.stdout:3/310: truncate d5/d7/d23/d27/d2f/d30/d4c/d50/f59 851747 0 2024-09-16T00:09:41.829 INFO:tasks.workunit.client.0.smithi139.stdout:1/296: truncate d7/d8/d27/d2f/f33 760142 0 2024-09-16T00:09:41.830 INFO:tasks.workunit.client.0.smithi139.stdout:7/227: dread - d5/d16/d22/d25/d2d/f3a zero size 2024-09-16T00:09:41.830 INFO:tasks.workunit.client.0.smithi139.stdout:1/297: read d7/f3f [282564,111223] 0 2024-09-16T00:09:41.832 INFO:tasks.workunit.client.0.smithi139.stdout:3/311: mkdir d5/d7/d23/d27/d2f/d30/d4c/d6d 0 2024-09-16T00:09:41.833 INFO:tasks.workunit.client.0.smithi139.stdout:7/228: chown d5/d16/d22/d25/d2d/f3b 2783013 1 2024-09-16T00:09:41.834 INFO:tasks.workunit.client.0.smithi139.stdout:1/298: creat d7/d8/d27/f57 x:0 0 0 2024-09-16T00:09:41.834 INFO:tasks.workunit.client.0.smithi139.stdout:3/312: dread - d5/d7/d23/d35/f45 zero size 2024-09-16T00:09:41.838 INFO:tasks.workunit.client.0.smithi139.stdout:6/192: dwrite d8/d20/f2b [4194304,4194304] 0 2024-09-16T00:09:41.838 INFO:tasks.workunit.client.0.smithi139.stdout:9/220: dread d19/d1a/f1e [0,4194304] 0 2024-09-16T00:09:41.839 INFO:tasks.workunit.client.0.smithi139.stdout:9/221: fdatasync f10 0 2024-09-16T00:09:41.840 INFO:tasks.workunit.client.0.smithi139.stdout:3/313: unlink d5/d7/dc/l10 0 2024-09-16T00:09:41.841 INFO:tasks.workunit.client.0.smithi139.stdout:1/299: dread d7/f28 [4194304,4194304] 0 2024-09-16T00:09:41.842 INFO:tasks.workunit.client.0.smithi139.stdout:6/193: creat d8/db/d26/d45/f46 x:0 0 0 2024-09-16T00:09:41.842 INFO:tasks.workunit.client.0.smithi139.stdout:1/300: rmdir d7/d8/d1f 39 2024-09-16T00:09:41.845 INFO:tasks.workunit.client.0.smithi139.stdout:6/194: creat d8/db/d26/d28/f47 x:0 0 0 2024-09-16T00:09:41.846 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:41.845+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:41.847 INFO:tasks.workunit.client.0.smithi139.stdout:3/314: symlink d5/d7/d23/d48/l6e 0 2024-09-16T00:09:41.849 INFO:tasks.workunit.client.0.smithi139.stdout:1/301: rmdir d7/d8/d27/d2f 39 2024-09-16T00:09:41.850 INFO:tasks.workunit.client.0.smithi139.stdout:9/222: getdents d19/d1a/d20 0 2024-09-16T00:09:41.851 INFO:tasks.workunit.client.0.smithi139.stdout:1/302: chown d7/d1b/c3d 1 1 2024-09-16T00:09:41.851 INFO:tasks.workunit.client.0.smithi139.stdout:3/315: mkdir d5/d7/d23/d27/d2f/d30/d4c/d6f 0 2024-09-16T00:09:41.851 INFO:tasks.workunit.client.0.smithi139.stdout:9/223: truncate fb 1474657 0 2024-09-16T00:09:41.852 INFO:tasks.workunit.client.0.smithi139.stdout:7/229: dread d5/f6 [0,4194304] 0 2024-09-16T00:09:41.853 INFO:tasks.workunit.client.0.smithi139.stdout:9/224: write f6 [601734,79493] 0 2024-09-16T00:09:41.853 INFO:tasks.workunit.client.0.smithi139.stdout:3/316: chown d5/d7/c3d 2267 1 2024-09-16T00:09:41.854 INFO:tasks.workunit.client.0.smithi139.stdout:9/225: chown d19/f38 38969573 1 2024-09-16T00:09:41.855 INFO:tasks.workunit.client.0.smithi139.stdout:3/317: rmdir d5/d7/d23/d48 39 2024-09-16T00:09:41.856 INFO:tasks.workunit.client.0.smithi139.stdout:7/230: dread f4 [0,4194304] 0 2024-09-16T00:09:41.858 INFO:tasks.workunit.client.0.smithi139.stdout:7/231: symlink d5/d26/l51 0 2024-09-16T00:09:41.860 INFO:tasks.workunit.client.0.smithi139.stdout:7/232: write f4 [169314,37603] 0 2024-09-16T00:09:41.860 INFO:tasks.workunit.client.0.smithi139.stdout:9/226: dwrite f6 [0,4194304] 0 2024-09-16T00:09:41.862 INFO:tasks.workunit.client.0.smithi139.stdout:9/227: dread - d19/f31 zero size 2024-09-16T00:09:41.864 INFO:tasks.workunit.client.0.smithi139.stdout:7/233: mknod d5/c52 0 2024-09-16T00:09:41.865 INFO:tasks.workunit.client.0.smithi139.stdout:3/318: dwrite d5/d7/f2d [0,4194304] 0 2024-09-16T00:09:41.866 INFO:tasks.workunit.client.0.smithi139.stdout:9/228: creat d19/d1a/d20/d23/f3e x:0 0 0 2024-09-16T00:09:41.868 INFO:tasks.workunit.client.0.smithi139.stdout:7/234: creat d5/d16/d22/d25/d2d/d50/f53 x:0 0 0 2024-09-16T00:09:41.869 INFO:tasks.workunit.client.0.smithi139.stdout:9/229: creat d19/d1a/f3f x:0 0 0 2024-09-16T00:09:41.871 INFO:tasks.workunit.client.0.smithi139.stdout:3/319: creat d5/d7/d23/d35/f70 x:0 0 0 2024-09-16T00:09:41.871 INFO:tasks.workunit.client.0.smithi139.stdout:3/320: fdatasync d5/f8 0 2024-09-16T00:09:41.872 INFO:tasks.workunit.client.0.smithi139.stdout:7/235: mknod d5/d16/d22/d25/d2d/c54 0 2024-09-16T00:09:41.872 INFO:tasks.workunit.client.0.smithi139.stdout:3/321: rmdir d5/d7/d23/d27/d2f/d30/d4d 39 2024-09-16T00:09:41.875 INFO:tasks.workunit.client.0.smithi139.stdout:3/322: creat d5/d7/d23/d27/d2f/d30/d41/f71 x:0 0 0 2024-09-16T00:09:41.876 INFO:tasks.workunit.client.0.smithi139.stdout:7/236: dwrite d5/f1f [0,4194304] 0 2024-09-16T00:09:41.877 INFO:tasks.workunit.client.0.smithi139.stdout:7/237: chown d5/d16/d22/d25 57782329 1 2024-09-16T00:09:41.877 INFO:tasks.workunit.client.0.smithi139.stdout:3/323: dread - d5/d7/dc/f63 zero size 2024-09-16T00:09:41.877 INFO:tasks.workunit.client.0.smithi139.stdout:7/238: readlink d5/d26/d42/l4d 0 2024-09-16T00:09:41.881 INFO:tasks.workunit.client.0.smithi139.stdout:7/239: rename d5/d26/d48 to d5/d16/d22/d25/d2d/d50/d44/d55 0 2024-09-16T00:09:41.881 INFO:tasks.workunit.client.0.smithi139.stdout:7/240: chown d5/fc 11 1 2024-09-16T00:09:41.885 INFO:tasks.workunit.client.0.smithi139.stdout:7/241: rename d5/d26/l51 to d5/d16/d22/d25/l56 0 2024-09-16T00:09:41.886 INFO:tasks.workunit.client.0.smithi139.stdout:7/242: write d5/d26/f2b [934177,124592] 0 2024-09-16T00:09:41.889 INFO:tasks.workunit.client.0.smithi139.stdout:9/230: dread f2 [0,4194304] 0 2024-09-16T00:09:41.889 INFO:tasks.workunit.client.0.smithi139.stdout:7/243: truncate d5/d16/d22/d25/f31 5144762 0 2024-09-16T00:09:41.890 INFO:tasks.workunit.client.0.smithi139.stdout:9/231: write d19/f1b [4854512,123836] 0 2024-09-16T00:09:41.891 INFO:tasks.workunit.client.0.smithi139.stdout:9/232: stat ld 0 2024-09-16T00:09:41.891 INFO:tasks.workunit.client.0.smithi139.stdout:7/244: dread d5/d16/d22/d25/f30 [0,4194304] 0 2024-09-16T00:09:41.892 INFO:tasks.workunit.client.0.smithi139.stdout:7/245: write d5/d26/f2b [184235,69099] 0 2024-09-16T00:09:41.894 INFO:tasks.workunit.client.0.smithi139.stdout:7/246: truncate d5/d26/f2b 1732826 0 2024-09-16T00:09:41.897 INFO:tasks.workunit.client.0.smithi139.stdout:3/324: dread d5/d7/dc/f12 [0,4194304] 0 2024-09-16T00:09:41.902 INFO:tasks.workunit.client.0.smithi139.stdout:7/247: dwrite d5/d16/f34 [0,4194304] 0 2024-09-16T00:09:41.905 INFO:tasks.workunit.client.0.smithi139.stdout:7/248: unlink d5/f4b 0 2024-09-16T00:09:41.906 INFO:tasks.workunit.client.0.smithi139.stdout:7/249: dread - d5/d37/f40 zero size 2024-09-16T00:09:41.908 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:41.908 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:41.908 INFO:tasks.workunit.client.0.smithi139.stdout:9/233: dread f10 [0,4194304] 0 2024-09-16T00:09:41.908 INFO:tasks.workunit.client.0.smithi139.stdout:7/250: chown d5/cd 18698 1 2024-09-16T00:09:41.913 INFO:tasks.workunit.client.0.smithi139.stdout:7/251: getdents d5/d16/d22 0 2024-09-16T00:09:41.914 INFO:tasks.workunit.client.0.smithi139.stdout:7/252: write d5/d37/f40 [108156,66456] 0 2024-09-16T00:09:41.915 INFO:tasks.workunit.client.0.smithi139.stdout:7/253: chown d5/d16/d22/d25/f31 220428153 1 2024-09-16T00:09:41.918 INFO:tasks.workunit.client.0.smithi139.stdout:7/254: creat d5/d16/d22/d25/d2d/d50/d44/f57 x:0 0 0 2024-09-16T00:09:41.919 INFO:tasks.workunit.client.0.smithi139.stdout:9/234: dwrite d19/f1d [0,4194304] 0 2024-09-16T00:09:41.920 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:09:41.921 INFO:tasks.workunit.client.0.smithi139.stdout:9/235: mknod d19/d1a/d20/d23/d2d/d2f/c40 0 2024-09-16T00:09:41.924 INFO:tasks.workunit.client.0.smithi139.stdout:9/236: read - d19/d1a/d20/d23/f3e zero size 2024-09-16T00:09:41.926 INFO:tasks.workunit.client.0.smithi139.stdout:7/255: dwrite d5/f6 [0,4194304] 0 2024-09-16T00:09:41.927 INFO:tasks.workunit.client.0.smithi139.stdout:7/256: truncate d5/f1f 4723657 0 2024-09-16T00:09:41.933 INFO:tasks.workunit.client.0.smithi139.stdout:9/237: dwrite fb [0,4194304] 0 2024-09-16T00:09:41.934 INFO:tasks.workunit.client.0.smithi139.stdout:9/238: fdatasync d19/f1f 0 2024-09-16T00:09:41.934 INFO:tasks.workunit.client.0.smithi139.stdout:7/257: dwrite d5/f1f [0,4194304] 0 2024-09-16T00:09:41.936 INFO:tasks.workunit.client.0.smithi139.stdout:9/239: write d19/f26 [1894154,21444] 0 2024-09-16T00:09:41.939 INFO:tasks.workunit.client.0.smithi139.stdout:9/240: mkdir d19/d1a/d20/d23/d2d/d41 0 2024-09-16T00:09:41.939 INFO:tasks.workunit.client.0.smithi139.stdout:7/258: getdents d5/d16/d22/d25/d2d/d50/d44/d55 0 2024-09-16T00:09:41.942 INFO:tasks.workunit.client.0.smithi139.stdout:7/259: mknod d5/d16/d22/d25/d2d/c58 0 2024-09-16T00:09:41.944 INFO:tasks.workunit.client.0.smithi139.stdout:9/241: dwrite d19/f1f [0,4194304] 0 2024-09-16T00:09:41.950 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:41.946+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:41.951 INFO:tasks.workunit.client.0.smithi139.stdout:7/260: dread d5/f12 [0,4194304] 0 2024-09-16T00:09:41.951 INFO:tasks.workunit.client.0.smithi139.stdout:7/261: creat d5/f59 x:0 0 0 2024-09-16T00:09:41.952 INFO:tasks.workunit.client.0.smithi139.stdout:7/262: chown d5/d26/d46/f4a 29854890 1 2024-09-16T00:09:41.952 INFO:tasks.workunit.client.0.smithi139.stdout:9/242: truncate d19/d1a/f1e 2681478 0 2024-09-16T00:09:41.953 INFO:tasks.workunit.client.0.smithi139.stdout:7/263: chown d5/d16/d22/d25/d2d/c58 161 1 2024-09-16T00:09:41.954 INFO:tasks.workunit.client.0.smithi139.stdout:7/264: readlink d5/d16/l24 0 2024-09-16T00:09:41.955 INFO:tasks.workunit.client.0.smithi139.stdout:7/265: fdatasync d5/d16/d22/d25/d2d/f3b 0 2024-09-16T00:09:41.955 INFO:tasks.workunit.client.0.smithi139.stdout:9/243: rename d19/d1a/d20/d23/d2c to d19/d1a/d20/d23/d2d/d35/d42 0 2024-09-16T00:09:41.956 INFO:tasks.workunit.client.0.smithi139.stdout:7/266: write d5/d16/d22/f4e [31313,23875] 0 2024-09-16T00:09:41.957 INFO:tasks.workunit.client.0.smithi139.stdout:9/244: truncate f2 389960 0 2024-09-16T00:09:41.958 INFO:tasks.workunit.client.0.smithi139.stdout:7/267: mknod d5/d37/c5a 0 2024-09-16T00:09:41.959 INFO:tasks.workunit.client.0.smithi139.stdout:7/268: fsync d5/d16/d22/d25/f31 0 2024-09-16T00:09:41.959 INFO:tasks.workunit.client.0.smithi139.stdout:9/245: mknod d19/d1a/d20/d23/d2d/d41/c43 0 2024-09-16T00:09:41.961 INFO:tasks.workunit.client.0.smithi139.stdout:7/269: creat d5/d26/d42/f5b x:0 0 0 2024-09-16T00:09:41.963 INFO:tasks.workunit.client.0.smithi139.stdout:9/246: unlink d19/d1a/d20/d23/f30 0 2024-09-16T00:09:41.967 INFO:tasks.workunit.client.0.smithi139.stdout:7/270: dwrite d5/d37/f47 [0,4194304] 0 2024-09-16T00:09:41.969 INFO:tasks.workunit.client.0.smithi139.stdout:9/247: unlink d19/d1a/c2e 0 2024-09-16T00:09:41.970 INFO:tasks.workunit.client.0.smithi139.stdout:7/271: symlink d5/d26/l5c 0 2024-09-16T00:09:41.971 INFO:tasks.workunit.client.0.smithi139.stdout:7/272: mkdir d5/d26/d5d 0 2024-09-16T00:09:41.974 INFO:tasks.workunit.client.0.smithi139.stdout:7/273: symlink d5/d26/d5d/l5e 0 2024-09-16T00:09:41.975 INFO:tasks.workunit.client.0.smithi139.stdout:7/274: write d5/f21 [678313,127800] 0 2024-09-16T00:09:41.978 INFO:tasks.workunit.client.0.smithi139.stdout:9/248: unlink f6 0 2024-09-16T00:09:41.981 INFO:tasks.workunit.client.0.smithi139.stdout:7/275: rename d5/d16/d22/d25/d2d/c58 to d5/d16/d22/d25/d2d/d50/d44/d55/c5f 0 2024-09-16T00:09:41.982 INFO:tasks.workunit.client.0.smithi139.stdout:7/276: write d5/d16/d22/d25/d2d/f3b [323859,22520] 0 2024-09-16T00:09:41.983 INFO:tasks.workunit.client.0.smithi139.stdout:9/249: dread d19/f1f [0,4194304] 0 2024-09-16T00:09:41.983 INFO:tasks.workunit.client.0.smithi139.stdout:5/291: sync 2024-09-16T00:09:41.985 INFO:tasks.workunit.client.0.smithi139.stdout:9/250: mkdir d19/d1a/d20/d23/d2d/d35/d44 0 2024-09-16T00:09:41.987 INFO:tasks.workunit.client.0.smithi139.stdout:9/251: mkdir d19/d1a/d20/d23/d2d/d35/d42/d3a/d45 0 2024-09-16T00:09:41.988 INFO:tasks.workunit.client.0.smithi139.stdout:7/277: truncate d5/d16/d22/d25/d2d/f3b 1513594 0 2024-09-16T00:09:41.989 INFO:tasks.workunit.client.0.smithi139.stdout:9/252: symlink d19/d1a/d20/d23/d2d/d35/d3d/l46 0 2024-09-16T00:09:41.990 INFO:tasks.workunit.client.0.smithi139.stdout:7/278: creat d5/d16/d22/d25/d2d/d50/d44/f60 x:0 0 0 2024-09-16T00:09:41.990 INFO:tasks.workunit.client.0.smithi139.stdout:9/253: read d19/d1a/d20/d23/d2d/d35/d42/f34 [716941,71881] 0 2024-09-16T00:09:41.991 INFO:tasks.workunit.client.0.smithi139.stdout:9/254: truncate d19/f26 5077928 0 2024-09-16T00:09:41.996 INFO:tasks.workunit.client.0.smithi139.stdout:7/279: dwrite d5/d16/d22/f4e [0,4194304] 0 2024-09-16T00:09:41.997 INFO:tasks.workunit.client.0.smithi139.stdout:9/255: dwrite f14 [4194304,4194304] 0 2024-09-16T00:09:42.000 INFO:tasks.workunit.client.0.smithi139.stdout:9/256: dread - d19/d1a/f3f zero size 2024-09-16T00:09:42.003 INFO:tasks.workunit.client.0.smithi139.stdout:9/257: rmdir d19/d1a/d20/d23/d2d/d35/d42/d3a 39 2024-09-16T00:09:42.004 INFO:tasks.workunit.client.0.smithi139.stdout:7/280: dwrite d5/f9 [4194304,4194304] 0 2024-09-16T00:09:42.005 INFO:tasks.workunit.client.0.smithi139.stdout:7/281: chown d5/l11 255 1 2024-09-16T00:09:42.008 INFO:tasks.workunit.client.0.smithi139.stdout:9/258: truncate fa 797508 0 2024-09-16T00:09:42.014 INFO:tasks.workunit.client.0.smithi139.stdout:9/259: link c15 d19/d1a/d20/d23/d2d/d41/c47 0 2024-09-16T00:09:42.015 INFO:tasks.workunit.client.0.smithi139.stdout:7/282: dwrite d5/d16/d22/d25/d2d/d50/d44/f57 [0,4194304] 0 2024-09-16T00:09:42.020 INFO:tasks.workunit.client.0.smithi139.stdout:9/260: creat d19/d1a/d20/d23/d2d/d35/d42/d3a/f48 x:0 0 0 2024-09-16T00:09:42.021 INFO:tasks.workunit.client.0.smithi139.stdout:9/261: chown f2 238447830 1 2024-09-16T00:09:42.022 INFO:tasks.workunit.client.0.smithi139.stdout:7/283: getdents d5/d16/d22/d25/d2d/d50/d44 0 2024-09-16T00:09:42.023 INFO:tasks.workunit.client.0.smithi139.stdout:7/284: stat d5/d16/d22/d25/f30 0 2024-09-16T00:09:42.024 INFO:tasks.workunit.client.0.smithi139.stdout:7/285: write d5/d37/f40 [467363,38921] 0 2024-09-16T00:09:42.024 INFO:tasks.workunit.client.0.smithi139.stdout:9/262: creat d19/d1a/d20/d23/d2d/d2f/f49 x:0 0 0 2024-09-16T00:09:42.029 INFO:tasks.workunit.client.0.smithi139.stdout:9/263: link d19/d1a/d20/d23/d2d/d2f/c40 d19/d1a/c4a 0 2024-09-16T00:09:42.030 INFO:tasks.workunit.client.0.smithi139.stdout:9/264: chown d19/d1a/d20/d23/d2d/d35/d42/d3a/f48 126402712 1 2024-09-16T00:09:42.031 INFO:tasks.workunit.client.0.smithi139.stdout:7/286: truncate d5/d37/f47 1655200 0 2024-09-16T00:09:42.032 INFO:tasks.workunit.client.0.smithi139.stdout:7/287: rmdir d5/d26/d5d 39 2024-09-16T00:09:42.033 INFO:tasks.workunit.client.0.smithi139.stdout:9/265: getdents d19/d1a/d20/d23/d2d/d35 0 2024-09-16T00:09:42.034 INFO:tasks.workunit.client.0.smithi139.stdout:7/288: stat d5/d37/f47 0 2024-09-16T00:09:42.037 INFO:tasks.workunit.client.0.smithi139.stdout:9/266: rename d19/d1a/d20/d23/d2d/d41/c47 to d19/d1a/d20/d23/d2d/c4b 0 2024-09-16T00:09:42.039 INFO:tasks.workunit.client.0.smithi139.stdout:9/267: unlink d19/d1a/d20/d23/d2d/c3c 0 2024-09-16T00:09:42.040 INFO:tasks.workunit.client.0.smithi139.stdout:7/289: rename d5/d16/d22/d25/d2d/d50/d44/d55/c5f to d5/d37/c61 0 2024-09-16T00:09:42.041 INFO:tasks.workunit.client.0.smithi139.stdout:9/268: unlink d19/f1f 0 2024-09-16T00:09:42.043 INFO:tasks.workunit.client.0.smithi139.stdout:9/269: symlink d19/d1a/d20/d23/d2d/d35/d42/l4c 0 2024-09-16T00:09:42.044 INFO:tasks.workunit.client.0.smithi139.stdout:9/270: read d19/d1a/d20/d23/f33 [1124,39930] 0 2024-09-16T00:09:42.046 INFO:tasks.workunit.client.0.smithi139.stdout:9/271: creat d19/d1a/d20/d23/f4d x:0 0 0 2024-09-16T00:09:42.047 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:42.047+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:42.051 INFO:tasks.workunit.client.0.smithi139.stdout:9/272: dwrite d19/f26 [0,4194304] 0 2024-09-16T00:09:42.053 INFO:tasks.workunit.client.0.smithi139.stdout:9/273: mkdir d19/d22/d32/d4e 0 2024-09-16T00:09:42.055 INFO:tasks.workunit.client.0.smithi139.stdout:9/274: chown d19/d1a/d20/d23/d2d/d35/d42/d3a/l3b 236983 1 2024-09-16T00:09:42.057 INFO:tasks.workunit.client.0.smithi139.stdout:9/275: symlink d19/d22/d32/d4e/l4f 0 2024-09-16T00:09:42.059 INFO:tasks.workunit.client.0.smithi139.stdout:9/276: creat d19/d1a/f50 x:0 0 0 2024-09-16T00:09:42.060 INFO:tasks.workunit.client.0.smithi139.stdout:9/277: write d19/d1a/d20/d23/d2d/d2f/f49 [735091,87245] 0 2024-09-16T00:09:42.063 INFO:tasks.workunit.client.0.smithi139.stdout:9/278: dread d19/d1a/f1e [0,4194304] 0 2024-09-16T00:09:42.065 INFO:tasks.workunit.client.0.smithi139.stdout:9/279: mknod d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/c51 0 2024-09-16T00:09:42.065 INFO:tasks.workunit.client.0.smithi139.stdout:9/280: fsync fb 0 2024-09-16T00:09:42.066 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:42.066 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:42.066 INFO:tasks.workunit.client.0.smithi139.stdout:9/281: stat d19/d22/d32/l36 0 2024-09-16T00:09:42.067 INFO:tasks.workunit.client.0.smithi139.stdout:9/282: chown d19/d1a/f50 124240646 1 2024-09-16T00:09:42.068 INFO:tasks.workunit.client.0.smithi139.stdout:9/283: mknod d19/d22/c52 0 2024-09-16T00:09:42.070 INFO:tasks.workunit.client.0.smithi139.stdout:9/284: mknod d19/d22/c53 0 2024-09-16T00:09:42.074 INFO:tasks.workunit.client.0.smithi139.stdout:9/285: getdents d19/d22/d32 0 2024-09-16T00:09:42.075 INFO:tasks.workunit.client.0.smithi139.stdout:9/286: readlink d19/d1a/d20/d23/d2d/d35/d42/l4c 0 2024-09-16T00:09:42.076 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:09:42.077 INFO:tasks.workunit.client.0.smithi139.stdout:9/287: rename f10 to d19/d1a/d20/d23/d2d/d35/d3d/f54 0 2024-09-16T00:09:42.083 INFO:tasks.workunit.client.0.smithi139.stdout:9/288: dwrite d19/f38 [0,4194304] 0 2024-09-16T00:09:42.083 INFO:tasks.workunit.client.0.smithi139.stdout:9/289: chown d19/d1a 8704 1 2024-09-16T00:09:42.087 INFO:tasks.workunit.client.0.smithi139.stdout:9/290: dread d19/f1d [0,4194304] 0 2024-09-16T00:09:42.094 INFO:tasks.workunit.client.0.smithi139.stdout:9/291: truncate d19/f1d 2254636 0 2024-09-16T00:09:42.095 INFO:tasks.workunit.client.0.smithi139.stdout:9/292: truncate fb 4737879 0 2024-09-16T00:09:42.100 INFO:tasks.workunit.client.0.smithi139.stdout:9/293: dwrite d19/d1a/f39 [4194304,4194304] 0 2024-09-16T00:09:42.104 INFO:tasks.workunit.client.0.smithi139.stdout:9/294: creat d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/f55 x:0 0 0 2024-09-16T00:09:42.104 INFO:tasks.workunit.client.0.smithi139.stdout:9/295: read d19/d1a/f1e [381772,54022] 0 2024-09-16T00:09:42.105 INFO:tasks.workunit.client.0.smithi139.stdout:9/296: dread - d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/f55 zero size 2024-09-16T00:09:42.109 INFO:tasks.workunit.client.0.smithi139.stdout:9/297: dread d19/d1a/d20/d23/d2d/d35/d3d/f54 [0,4194304] 0 2024-09-16T00:09:42.112 INFO:tasks.workunit.client.0.smithi139.stdout:9/298: rename d19/f1b to d19/d1a/d20/d23/d2d/d35/d42/d3a/f56 0 2024-09-16T00:09:42.116 INFO:tasks.workunit.client.0.smithi139.stdout:9/299: dread d19/d1a/d20/d23/d2d/d2f/f49 [0,4194304] 0 2024-09-16T00:09:42.117 INFO:tasks.workunit.client.0.smithi139.stdout:9/300: creat d19/d22/f57 x:0 0 0 2024-09-16T00:09:42.119 INFO:tasks.workunit.client.0.smithi139.stdout:9/301: dread d19/d1a/d20/d23/d2d/d35/d42/f34 [0,4194304] 0 2024-09-16T00:09:42.119 INFO:tasks.workunit.client.0.smithi139.stdout:9/302: fsync d19/d1a/f39 0 2024-09-16T00:09:42.122 INFO:tasks.workunit.client.0.smithi139.stdout:9/303: creat d19/d1a/d20/d23/d2d/d35/f58 x:0 0 0 2024-09-16T00:09:42.124 INFO:tasks.workunit.client.0.smithi139.stdout:9/304: rmdir d19/d1a/d20/d23/d2d/d35/d42/d3a 39 2024-09-16T00:09:42.126 INFO:tasks.workunit.client.0.smithi139.stdout:9/305: mkdir d19/d1a/d20/d59 0 2024-09-16T00:09:42.130 INFO:tasks.workunit.client.0.smithi139.stdout:9/306: rename d19/f31 to d19/d1a/d20/d23/d2d/d35/d3d/f5a 0 2024-09-16T00:09:42.136 INFO:tasks.workunit.client.0.smithi139.stdout:9/307: truncate d19/f26 815045 0 2024-09-16T00:09:42.139 INFO:tasks.workunit.client.0.smithi139.stdout:9/308: mknod d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/c5b 0 2024-09-16T00:09:42.141 INFO:tasks.workunit.client.0.smithi139.stdout:9/309: mknod d19/d1a/d20/d23/c5c 0 2024-09-16T00:09:42.141 INFO:tasks.workunit.client.0.smithi139.stdout:9/310: chown d19/d1a/d20/d23/d2d/d35/d42 241574 1 2024-09-16T00:09:42.144 INFO:tasks.workunit.client.0.smithi139.stdout:9/311: rename d19/d1a/d20/d23/d2d/d2f/f49 to d19/d1a/d20/d59/f5d 0 2024-09-16T00:09:42.149 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:42.147+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:42.149 INFO:tasks.workunit.client.0.smithi139.stdout:9/312: rename d19/d1a/d20/d23/d2d/d35/f58 to d19/d1a/d20/d23/d2d/d35/d42/f5e 0 2024-09-16T00:09:42.155 INFO:tasks.workunit.client.0.smithi139.stdout:9/313: rename c12 to d19/d1a/d20/d23/d2d/d35/d42/d3a/c5f 0 2024-09-16T00:09:42.155 INFO:tasks.workunit.client.0.smithi139.stdout:9/314: mkdir d19/d1a/d20/d23/d2d/d35/d60 0 2024-09-16T00:09:42.155 INFO:tasks.workunit.client.0.smithi139.stdout:9/315: getdents d19/d1a/d20/d23/d2d/d35/d60 0 2024-09-16T00:09:42.159 INFO:tasks.workunit.client.0.smithi139.stdout:8/317: sync 2024-09-16T00:09:42.159 INFO:tasks.workunit.client.0.smithi139.stdout:4/173: sync 2024-09-16T00:09:42.160 INFO:tasks.workunit.client.0.smithi139.stdout:8/318: chown d17/d1a/d43/f3b 0 1 2024-09-16T00:09:42.161 INFO:tasks.workunit.client.0.smithi139.stdout:4/174: mknod d12/d21/c37 0 2024-09-16T00:09:42.162 INFO:tasks.workunit.client.0.smithi139.stdout:4/175: fdatasync f2 0 2024-09-16T00:09:42.163 INFO:tasks.workunit.client.0.smithi139.stdout:8/319: mknod d17/d1f/d3a/c6d 0 2024-09-16T00:09:42.165 INFO:tasks.workunit.client.0.smithi139.stdout:8/320: read d17/f4a [205082,97949] 0 2024-09-16T00:09:42.166 INFO:tasks.workunit.client.0.smithi139.stdout:4/176: creat d12/f38 x:0 0 0 2024-09-16T00:09:42.167 INFO:tasks.workunit.client.0.smithi139.stdout:8/321: unlink d17/d1a/d43/c2f 0 2024-09-16T00:09:42.168 INFO:tasks.workunit.client.0.smithi139.stdout:8/322: fdatasync f7 0 2024-09-16T00:09:42.168 INFO:tasks.workunit.client.0.smithi139.stdout:4/177: symlink d12/d21/l39 0 2024-09-16T00:09:42.169 INFO:tasks.workunit.client.0.smithi139.stdout:8/323: write d17/d1a/d43/d40/f41 [880905,70091] 0 2024-09-16T00:09:42.171 INFO:tasks.workunit.client.0.smithi139.stdout:8/324: creat d17/d54/f6e x:0 0 0 2024-09-16T00:09:42.174 INFO:tasks.workunit.client.0.smithi139.stdout:8/325: creat d17/d1f/d3a/f6f x:0 0 0 2024-09-16T00:09:42.181 INFO:tasks.workunit.client.0.smithi139.stdout:4/178: truncate d12/f31 135329 0 2024-09-16T00:09:42.183 INFO:tasks.workunit.client.0.smithi139.stdout:4/179: creat d12/d2b/f3a x:0 0 0 2024-09-16T00:09:42.184 INFO:tasks.workunit.client.0.smithi139.stdout:4/180: mknod d12/d2b/c3b 0 2024-09-16T00:09:42.186 INFO:tasks.workunit.client.0.smithi139.stdout:4/181: write d12/d21/f32 [4389846,73715] 0 2024-09-16T00:09:42.187 INFO:tasks.workunit.client.0.smithi139.stdout:4/182: mknod d12/d2b/c3c 0 2024-09-16T00:09:42.188 INFO:tasks.workunit.client.0.smithi139.stdout:8/326: dread d17/d54/f55 [0,4194304] 0 2024-09-16T00:09:42.190 INFO:tasks.workunit.client.0.smithi139.stdout:4/183: symlink d12/d2b/l3d 0 2024-09-16T00:09:42.190 INFO:tasks.workunit.client.0.smithi139.stdout:8/327: write d17/d1a/d43/f42 [1141862,10658] 0 2024-09-16T00:09:42.191 INFO:tasks.workunit.client.0.smithi139.stdout:8/328: chown d17/d1a/d43/d40/f4b 4250355 1 2024-09-16T00:09:42.193 INFO:tasks.workunit.client.0.smithi139.stdout:8/329: fsync d17/d54/f69 0 2024-09-16T00:09:42.194 INFO:tasks.workunit.client.0.smithi139.stdout:8/330: chown l11 12 1 2024-09-16T00:09:42.196 INFO:tasks.workunit.client.0.smithi139.stdout:8/331: creat d17/f70 x:0 0 0 2024-09-16T00:09:42.196 INFO:tasks.workunit.client.0.smithi139.stdout:4/184: dwrite d12/d21/f24 [0,4194304] 0 2024-09-16T00:09:42.198 INFO:tasks.workunit.client.0.smithi139.stdout:4/185: mknod d12/d2b/c3e 0 2024-09-16T00:09:42.198 INFO:tasks.workunit.client.0.smithi139.stdout:8/332: creat d17/d1a/d57/f71 x:0 0 0 2024-09-16T00:09:42.199 INFO:tasks.workunit.client.0.smithi139.stdout:4/186: write d12/d2b/f26 [256059,78145] 0 2024-09-16T00:09:42.203 INFO:tasks.workunit.client.0.smithi139.stdout:4/187: creat d12/f3f x:0 0 0 2024-09-16T00:09:42.206 INFO:tasks.workunit.client.0.smithi139.stdout:8/333: dwrite fb [4194304,4194304] 0 2024-09-16T00:09:42.207 INFO:tasks.workunit.client.0.smithi139.stdout:8/334: readlink d17/d1a/d43/d40/d4f/l5c 0 2024-09-16T00:09:42.210 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:42.210 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:42.210 INFO:tasks.workunit.client.0.smithi139.stdout:8/335: truncate d17/d1f/f25 310360 0 2024-09-16T00:09:42.212 INFO:tasks.workunit.client.0.smithi139.stdout:8/336: readlink d17/l20 0 2024-09-16T00:09:42.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.216+0000 7f4b097fa640 1 -- 172.21.15.139:0/4104911171 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v87) ==== 87+0+0 (secure 0 0 0) 0x7f4af40587f0 con 0x7f4b0c0c7fb0 2024-09-16T00:09:42.218 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is set 2024-09-16T00:09:42.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.219+0000 7f4b12837640 1 -- 172.21.15.139:0/4104911171 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4aec06c2b0 msgr2=0x7f4aec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:42.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.219+0000 7f4b12837640 1 --2- 172.21.15.139:0/4104911171 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4aec06c2b0 0x7f4aec06e770 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0011e0 tx=0x7f4afc007d30 comp rx=0 tx=0).stop 2024-09-16T00:09:42.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.219+0000 7f4b12837640 1 -- 172.21.15.139:0/4104911171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b0c0c7fb0 msgr2=0x7f4b0c14a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:42.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.219+0000 7f4b12837640 1 --2- 172.21.15.139:0/4104911171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b0c0c7fb0 0x7f4b0c14a1c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4af400c3c0 tx=0x7f4af400c880 comp rx=0 tx=0).stop 2024-09-16T00:09:42.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.219+0000 7f4b12837640 1 -- 172.21.15.139:0/4104911171 shutdown_connections 2024-09-16T00:09:42.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.219+0000 7f4b12837640 1 --2- 172.21.15.139:0/4104911171 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4aec06c2b0 0x7f4aec06e770 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:42.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.219+0000 7f4b12837640 1 --2- 172.21.15.139:0/4104911171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b0c154bc0 0x7f4b0c14f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:42.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.219+0000 7f4b12837640 1 --2- 172.21.15.139:0/4104911171 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4b0c0c7fb0 0x7f4b0c14a1c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:42.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.219+0000 7f4b12837640 1 --2- 172.21.15.139:0/4104911171 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4b0c0c79c0 0x7f4b0c0cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:42.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.219+0000 7f4b12837640 1 -- 172.21.15.139:0/4104911171 >> 172.21.15.139:0/4104911171 conn(0x7f4b0c070460 msgr2=0x7f4b0c0c9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:42.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.219+0000 7f4b12837640 1 -- 172.21.15.139:0/4104911171 shutdown_connections 2024-09-16T00:09:42.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.219+0000 7f4b12837640 1 -- 172.21.15.139:0/4104911171 wait complete. 2024-09-16T00:09:42.223 INFO:tasks.workunit.client.0.smithi139.stdout:4/188: dread d12/d21/f32 [0,4194304] 0 2024-09-16T00:09:42.224 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:09:42.224 INFO:tasks.workunit.client.0.smithi139.stdout:4/189: stat d12/d2b/l36 0 2024-09-16T00:09:42.225 INFO:tasks.workunit.client.0.smithi139.stdout:4/190: creat d12/f40 x:0 0 0 2024-09-16T00:09:42.226 INFO:tasks.workunit.client.0.smithi139.stdout:4/191: write d12/f3f [363537,25286] 0 2024-09-16T00:09:42.227 INFO:tasks.workunit.client.0.smithi139.stdout:4/192: read - d12/d2b/f34 zero size 2024-09-16T00:09:42.234 INFO:tasks.workunit.client.0.smithi139.stdout:4/193: dwrite d12/f3f [0,4194304] 0 2024-09-16T00:09:42.235 INFO:tasks.workunit.client.0.smithi139.stdout:4/194: write d12/d2b/f3a [851374,14059] 0 2024-09-16T00:09:42.245 INFO:tasks.workunit.client.0.smithi139.stdout:4/195: dread d12/f2a [0,4194304] 0 2024-09-16T00:09:42.246 INFO:tasks.workunit.client.0.smithi139.stdout:4/196: write d12/d2b/f18 [165106,78571] 0 2024-09-16T00:09:42.248 INFO:tasks.workunit.client.0.smithi139.stdout:4/197: symlink d12/d2b/l41 0 2024-09-16T00:09:42.250 INFO:tasks.workunit.client.0.smithi139.stdout:4/198: read f7 [4215280,72680] 0 2024-09-16T00:09:42.251 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:42.249+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:42.251 INFO:tasks.workunit.client.0.smithi139.stdout:4/199: stat d12/f38 0 2024-09-16T00:09:42.252 INFO:tasks.workunit.client.0.smithi139.stdout:4/200: creat d12/f42 x:0 0 0 2024-09-16T00:09:42.256 INFO:tasks.workunit.client.0.smithi139.stdout:4/201: rename fe to d12/f43 0 2024-09-16T00:09:42.257 INFO:tasks.workunit.client.0.smithi139.stdout:4/202: rmdir d12/d21 39 2024-09-16T00:09:42.267 INFO:tasks.workunit.client.0.smithi139.stdout:4/203: dread d12/f25 [0,4194304] 0 2024-09-16T00:09:42.272 INFO:tasks.workunit.client.0.smithi139.stdout:4/204: dwrite d12/f38 [0,4194304] 0 2024-09-16T00:09:42.273 INFO:tasks.workunit.client.0.smithi139.stdout:4/205: chown d12/l2d 24 1 2024-09-16T00:09:42.274 INFO:tasks.workunit.client.0.smithi139.stdout:4/206: write d12/d2b/f35 [727448,31830] 0 2024-09-16T00:09:42.279 INFO:tasks.workunit.client.0.smithi139.stdout:4/207: rename d12/f33 to d12/f44 0 2024-09-16T00:09:42.282 INFO:tasks.workunit.client.0.smithi139.stdout:4/208: creat d12/d21/f45 x:0 0 0 2024-09-16T00:09:42.288 INFO:tasks.workunit.client.0.smithi139.stdout:4/209: dwrite d12/d2b/f3a [0,4194304] 0 2024-09-16T00:09:42.291 INFO:tasks.workunit.client.0.smithi139.stdout:0/284: sync 2024-09-16T00:09:42.291 INFO:tasks.workunit.client.0.smithi139.stdout:2/282: sync 2024-09-16T00:09:42.293 INFO:tasks.workunit.client.0.smithi139.stdout:0/285: truncate d6/d42/f48 945856 0 2024-09-16T00:09:42.293 INFO:tasks.workunit.client.0.smithi139.stdout:2/283: write d11/d30/f42 [2174750,35733] 0 2024-09-16T00:09:42.295 INFO:tasks.workunit.client.0.smithi139.stdout:0/286: chown d6/d1c/d41/d44/dd/f39 2 1 2024-09-16T00:09:42.295 INFO:tasks.workunit.client.0.smithi139.stdout:2/284: creat d11/d30/d57/d29/f5c x:0 0 0 2024-09-16T00:09:42.296 INFO:tasks.workunit.client.0.smithi139.stdout:2/285: stat d11/d2f/l40 0 2024-09-16T00:09:42.296 INFO:tasks.workunit.client.0.smithi139.stdout:4/210: dread fd [0,4194304] 0 2024-09-16T00:09:42.298 INFO:tasks.workunit.client.0.smithi139.stdout:0/287: mkdir d6/d1c/d41/d44/dd/d56 0 2024-09-16T00:09:42.300 INFO:tasks.workunit.client.0.smithi139.stdout:0/288: chown d6/d1c/d45/d49 210233 1 2024-09-16T00:09:42.300 INFO:tasks.workunit.client.0.smithi139.stdout:4/211: mknod d12/c46 0 2024-09-16T00:09:42.304 INFO:tasks.workunit.client.0.smithi139.stdout:2/286: dwrite d11/d32/f43 [0,4194304] 0 2024-09-16T00:09:42.304 INFO:tasks.workunit.client.0.smithi139.stdout:0/289: getdents d6/d1c/d41 0 2024-09-16T00:09:42.306 INFO:tasks.workunit.client.0.smithi139.stdout:2/287: mkdir d11/d30/d57/d5d 0 2024-09-16T00:09:42.308 INFO:tasks.workunit.client.0.smithi139.stdout:2/288: write d11/d30/d57/d29/d2b/f34 [1202168,127523] 0 2024-09-16T00:09:42.312 INFO:tasks.workunit.client.0.smithi139.stdout:4/212: rename d12/d21 to d12/d47 0 2024-09-16T00:09:42.315 INFO:tasks.workunit.client.0.smithi139.stdout:2/289: dwrite d11/d30/d57/d29/f5c [0,4194304] 0 2024-09-16T00:09:42.320 INFO:tasks.workunit.client.0.smithi139.stdout:4/213: truncate fd 614893 0 2024-09-16T00:09:42.321 INFO:tasks.workunit.client.0.smithi139.stdout:2/290: getdents d11/d30/d57/d29 0 2024-09-16T00:09:42.322 INFO:tasks.workunit.client.0.smithi139.stdout:0/290: dwrite d6/f52 [0,4194304] 0 2024-09-16T00:09:42.323 INFO:tasks.workunit.client.0.smithi139.stdout:4/214: chown d12/c29 330161822 1 2024-09-16T00:09:42.323 INFO:tasks.workunit.client.0.smithi139.stdout:0/291: chown d6/d1c/f4c 63403 1 2024-09-16T00:09:42.326 INFO:tasks.workunit.client.0.smithi139.stdout:4/215: write d12/d2b/f30 [3839806,107163] 0 2024-09-16T00:09:42.332 INFO:tasks.workunit.client.0.smithi139.stdout:2/291: dwrite f8 [0,4194304] 0 2024-09-16T00:09:42.333 INFO:tasks.workunit.client.0.smithi139.stdout:0/292: dwrite d6/d1c/d45/f47 [0,4194304] 0 2024-09-16T00:09:42.335 INFO:tasks.workunit.client.0.smithi139.stdout:0/293: fsync d6/d1c/d45/d49/f4f 0 2024-09-16T00:09:42.341 INFO:tasks.workunit.client.0.smithi139.stdout:0/294: write d6/d1c/f35 [4485721,118302] 0 2024-09-16T00:09:42.346 INFO:tasks.workunit.client.0.smithi139.stdout:0/295: creat d6/d1c/d41/d44/dd/d56/f57 x:0 0 0 2024-09-16T00:09:42.347 INFO:tasks.workunit.client.0.smithi139.stdout:4/216: dread d12/f43 [0,4194304] 0 2024-09-16T00:09:42.349 INFO:tasks.workunit.client.0.smithi139.stdout:0/296: mkdir d6/d1c/d41/d44/d58 0 2024-09-16T00:09:42.351 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:42.350+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:42.356 INFO:tasks.workunit.client.0.smithi139.stdout:4/217: write d12/f2a [1377255,88903] 0 2024-09-16T00:09:42.356 INFO:tasks.workunit.client.0.smithi139.stdout:4/218: write d12/f42 [188081,13392] 0 2024-09-16T00:09:42.356 INFO:tasks.workunit.client.0.smithi139.stdout:0/297: truncate d6/d1c/d41/d44/f33 107788 0 2024-09-16T00:09:42.357 INFO:tasks.workunit.client.0.smithi139.stdout:4/219: link cf d12/d2b/c48 0 2024-09-16T00:09:42.358 INFO:tasks.workunit.client.0.smithi139.stdout:4/220: chown f1 938996 1 2024-09-16T00:09:42.359 INFO:tasks.workunit.client.0.smithi139.stdout:4/221: mkdir d12/d47/d49 0 2024-09-16T00:09:42.363 INFO:tasks.workunit.client.0.smithi139.stdout:0/298: truncate d6/d1c/d41/d44/dd/f2d 1921265 0 2024-09-16T00:09:42.364 INFO:tasks.workunit.client.0.smithi139.stdout:0/299: creat d6/d1c/f59 x:0 0 0 2024-09-16T00:09:42.365 INFO:tasks.workunit.client.0.smithi139.stdout:0/300: fsync d6/d1c/d45/f50 0 2024-09-16T00:09:42.365 INFO:tasks.workunit.client.0.smithi139.stdout:4/222: truncate d12/f25 712266 0 2024-09-16T00:09:42.367 INFO:tasks.workunit.client.0.smithi139.stdout:0/301: mkdir d6/d5a 0 2024-09-16T00:09:42.367 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:42.367 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:42.368 INFO:tasks.workunit.client.0.smithi139.stdout:4/223: mkdir d12/d4a 0 2024-09-16T00:09:42.370 INFO:tasks.workunit.client.0.smithi139.stdout:4/224: symlink d12/d47/d49/l4b 0 2024-09-16T00:09:42.370 INFO:tasks.workunit.client.0.smithi139.stdout:0/302: link d6/d1c/d41/d44/c51 d6/d1c/d41/d44/dd/c5b 0 2024-09-16T00:09:42.372 INFO:tasks.workunit.client.0.smithi139.stdout:4/225: creat d12/f4c x:0 0 0 2024-09-16T00:09:42.372 INFO:tasks.workunit.client.0.smithi139.stdout:0/303: dread d6/d1c/d41/d44/f29 [0,4194304] 0 2024-09-16T00:09:42.373 INFO:tasks.workunit.client.0.smithi139.stdout:4/226: chown d12/d47/f2c 230 1 2024-09-16T00:09:42.374 INFO:tasks.workunit.client.0.smithi139.stdout:0/304: dread - d6/d1c/d45/d49/f4f zero size 2024-09-16T00:09:42.374 INFO:tasks.workunit.client.0.smithi139.stdout:4/227: fdatasync d12/d2b/f2e 0 2024-09-16T00:09:42.375 INFO:tasks.workunit.client.0.smithi139.stdout:0/305: stat d6/d1c/d41/d44/dd/f24 0 2024-09-16T00:09:42.375 INFO:tasks.workunit.client.0.smithi139.stdout:4/228: chown d12/d2b/l36 39068 1 2024-09-16T00:09:42.375 INFO:tasks.workunit.client.0.smithi139.stdout:4/229: chown f1 141 1 2024-09-16T00:09:42.376 INFO:tasks.workunit.client.0.smithi139.stdout:4/230: mkdir d12/d4a/d4d 0 2024-09-16T00:09:42.378 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:09:42.385 INFO:tasks.workunit.client.0.smithi139.stdout:0/306: getdents d6/d1c/d41/d44/dd 0 2024-09-16T00:09:42.388 INFO:tasks.workunit.client.0.smithi139.stdout:0/307: symlink d6/d1c/d45/d49/l5c 0 2024-09-16T00:09:42.388 INFO:tasks.workunit.client.0.smithi139.stdout:0/308: read - d6/d1c/f59 zero size 2024-09-16T00:09:42.394 INFO:tasks.workunit.client.0.smithi139.stdout:0/309: dread d6/d1c/f36 [0,4194304] 0 2024-09-16T00:09:42.394 INFO:tasks.workunit.client.0.smithi139.stdout:0/310: stat d6/d1c/d41/d44/dd/l12 0 2024-09-16T00:09:42.396 INFO:tasks.workunit.client.0.smithi139.stdout:0/311: mknod d6/d1c/d45/c5d 0 2024-09-16T00:09:42.402 INFO:tasks.workunit.client.0.smithi139.stdout:0/312: dwrite d6/d1c/f4c [0,4194304] 0 2024-09-16T00:09:42.404 INFO:tasks.workunit.client.0.smithi139.stdout:0/313: creat d6/d1c/d45/f5e x:0 0 0 2024-09-16T00:09:42.411 INFO:tasks.workunit.client.0.smithi139.stdout:0/314: dwrite d6/d1c/d41/d44/dd/f40 [0,4194304] 0 2024-09-16T00:09:42.414 INFO:tasks.workunit.client.0.smithi139.stdout:0/315: creat d6/d1c/d41/d44/d58/f5f x:0 0 0 2024-09-16T00:09:42.415 INFO:tasks.workunit.client.0.smithi139.stdout:0/316: read - d6/d1c/d41/d44/f43 zero size 2024-09-16T00:09:42.416 INFO:tasks.workunit.client.0.smithi139.stdout:0/317: dread d6/d1c/d41/d44/dd/f2c [0,4194304] 0 2024-09-16T00:09:42.452 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:42.451+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:42.460 INFO:tasks.workunit.client.0.smithi139.stdout:6/195: sync 2024-09-16T00:09:42.461 INFO:tasks.workunit.client.0.smithi139.stdout:6/196: rmdir d8/db/d26/d28 39 2024-09-16T00:09:42.462 INFO:tasks.workunit.client.0.smithi139.stdout:6/197: write d8/d20/d27/d10/f2e [684496,65341] 0 2024-09-16T00:09:42.465 INFO:tasks.workunit.client.0.smithi139.stdout:6/198: truncate d8/db/f2c 3354642 0 2024-09-16T00:09:42.467 INFO:tasks.workunit.client.0.smithi139.stdout:6/199: symlink d8/db/d26/d45/l48 0 2024-09-16T00:09:42.469 INFO:tasks.workunit.client.0.smithi139.stdout:6/200: creat d8/d29/f49 x:0 0 0 2024-09-16T00:09:42.470 INFO:tasks.workunit.client.0.smithi139.stdout:6/201: creat d8/d20/f4a x:0 0 0 2024-09-16T00:09:42.473 INFO:tasks.workunit.client.0.smithi139.stdout:6/202: unlink f6 0 2024-09-16T00:09:42.475 INFO:tasks.workunit.client.0.smithi139.stdout:6/203: truncate d8/db/d26/d28/f47 114159 0 2024-09-16T00:09:42.475 INFO:tasks.workunit.client.0.smithi139.stdout:6/204: chown d8/db/d26/d28/f40 332 1 2024-09-16T00:09:42.478 INFO:tasks.workunit.client.0.smithi139.stdout:6/205: rename d8/db/f1a to d8/db/d26/f4b 0 2024-09-16T00:09:42.483 INFO:tasks.workunit.client.0.smithi139.stdout:6/206: dwrite d8/db/d26/d28/f47 [0,4194304] 0 2024-09-16T00:09:42.484 INFO:tasks.workunit.client.0.smithi139.stdout:6/207: readlink d8/d20/d27/l1e 0 2024-09-16T00:09:42.485 INFO:tasks.workunit.client.0.smithi139.stdout:6/208: chown d8/d20/d27/d10/f37 9646949 1 2024-09-16T00:09:42.486 INFO:tasks.workunit.client.0.smithi139.stdout:6/209: fsync d8/db/d26/d28/f3f 0 2024-09-16T00:09:42.487 INFO:tasks.workunit.client.0.smithi139.stdout:6/210: chown d8/fe 32423874 1 2024-09-16T00:09:42.490 INFO:tasks.workunit.client.0.smithi139.stdout:6/211: symlink d8/d20/l4c 0 2024-09-16T00:09:42.495 INFO:tasks.workunit.client.0.smithi139.stdout:6/212: dwrite d8/d20/d27/d10/f3b [0,4194304] 0 2024-09-16T00:09:42.502 INFO:tasks.workunit.client.0.smithi139.stdout:6/213: stat d8/db/f2c 0 2024-09-16T00:09:42.503 INFO:tasks.workunit.client.0.smithi139.stdout:6/214: truncate d8/d20/f4a 139564 0 2024-09-16T00:09:42.505 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:42.505 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:42.507 INFO:tasks.workunit.client.0.smithi139.stdout:6/215: mkdir d8/d20/d4d 0 2024-09-16T00:09:42.509 INFO:tasks.workunit.client.0.smithi139.stdout:6/216: mknod d8/d20/d27/c4e 0 2024-09-16T00:09:42.509 INFO:tasks.workunit.client.0.smithi139.stdout:6/217: read - d8/db/d26/d45/f43 zero size 2024-09-16T00:09:42.511 INFO:tasks.workunit.client.0.smithi139.stdout:6/218: chown d8/ca 398 1 2024-09-16T00:09:42.511 INFO:tasks.workunit.client.0.smithi139.stdout:6/219: dread - d8/f39 zero size 2024-09-16T00:09:42.511 INFO:tasks.workunit.client.0.smithi139.stdout:6/220: fsync d8/db/f15 0 2024-09-16T00:09:42.512 INFO:tasks.workunit.client.0.smithi139.stdout:6/221: chown d8/db/d26/d45/f46 25319 1 2024-09-16T00:09:42.515 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:09:42.517 INFO:tasks.workunit.client.0.smithi139.stdout:6/222: link d8/d20/d27/f1c d8/db/d26/d28/f4f 0 2024-09-16T00:09:42.517 INFO:tasks.workunit.client.0.smithi139.stdout:6/223: readlink d8/d20/l4c 0 2024-09-16T00:09:42.520 INFO:tasks.workunit.client.0.smithi139.stdout:6/224: dread d8/d20/d27/f1c [0,4194304] 0 2024-09-16T00:09:42.521 INFO:tasks.workunit.client.0.smithi139.stdout:6/225: creat d8/d29/f50 x:0 0 0 2024-09-16T00:09:42.523 INFO:tasks.workunit.client.0.smithi139.stdout:6/226: write d8/db/d26/d28/f4f [14016,88502] 0 2024-09-16T00:09:42.525 INFO:tasks.workunit.client.0.smithi139.stdout:6/227: chown d8/db/d26/d28/c2f 11033 1 2024-09-16T00:09:42.527 INFO:tasks.workunit.client.0.smithi139.stdout:6/228: fsync d8/db/f2c 0 2024-09-16T00:09:42.531 INFO:tasks.workunit.client.0.smithi139.stdout:6/229: rename d8/db/l21 to d8/d20/d27/d10/l51 0 2024-09-16T00:09:42.532 INFO:tasks.workunit.client.0.smithi139.stdout:6/230: write d8/db/d26/d28/f3f [897065,99840] 0 2024-09-16T00:09:42.534 INFO:tasks.workunit.client.0.smithi139.stdout:6/231: mknod d8/db/d26/d45/d42/c52 0 2024-09-16T00:09:42.544 INFO:tasks.workunit.client.0.smithi139.stdout:6/232: dread d8/fe [0,4194304] 0 2024-09-16T00:09:42.546 INFO:tasks.workunit.client.0.smithi139.stdout:6/233: creat d8/d20/d27/f53 x:0 0 0 2024-09-16T00:09:42.547 INFO:tasks.workunit.client.0.smithi139.stdout:6/234: write d8/db/d26/d28/f40 [3063467,455] 0 2024-09-16T00:09:42.549 INFO:tasks.workunit.client.0.smithi139.stdout:6/235: write d8/db/f2c [2869988,37255] 0 2024-09-16T00:09:42.553 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:42.553+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:42.555 INFO:tasks.workunit.client.0.smithi139.stdout:6/236: dwrite d8/fe [0,4194304] 0 2024-09-16T00:09:42.562 INFO:tasks.workunit.client.0.smithi139.stdout:6/237: creat d8/db/d26/f54 x:0 0 0 2024-09-16T00:09:42.562 INFO:tasks.workunit.client.0.smithi139.stdout:6/238: rename d8/d20/d27/cd to d8/d29/c55 0 2024-09-16T00:09:42.562 INFO:tasks.workunit.client.0.smithi139.stdout:6/239: mkdir d8/d56 0 2024-09-16T00:09:42.563 INFO:tasks.workunit.client.0.smithi139.stdout:6/240: stat d8/db/f2c 0 2024-09-16T00:09:42.563 INFO:tasks.workunit.client.0.smithi139.stdout:6/241: chown d8/db/d26/d45/f46 1731 1 2024-09-16T00:09:42.565 INFO:tasks.workunit.client.0.smithi139.stdout:6/242: symlink d8/db/d26/d45/d42/l57 0 2024-09-16T00:09:42.567 INFO:tasks.workunit.client.0.smithi139.stdout:6/243: getdents d8/db/d26/d45 0 2024-09-16T00:09:42.567 INFO:tasks.workunit.client.0.smithi139.stdout:6/244: read - d8/db/d26/d45/f43 zero size 2024-09-16T00:09:42.573 INFO:tasks.workunit.client.0.smithi139.stdout:6/245: dwrite d8/db/d26/d28/f47 [0,4194304] 0 2024-09-16T00:09:42.574 INFO:tasks.workunit.client.0.smithi139.stdout:1/303: sync 2024-09-16T00:09:42.574 INFO:tasks.workunit.client.0.smithi139.stdout:3/325: sync 2024-09-16T00:09:42.575 INFO:tasks.workunit.client.0.smithi139.stdout:3/326: dread - d5/d7/d23/d27/d2f/d30/f4e zero size 2024-09-16T00:09:42.576 INFO:tasks.workunit.client.0.smithi139.stdout:1/304: unlink d7/d1b/f54 0 2024-09-16T00:09:42.577 INFO:tasks.workunit.client.0.smithi139.stdout:6/246: link d8/d1f/f41 d8/d20/d27/f58 0 2024-09-16T00:09:42.577 INFO:tasks.workunit.client.0.smithi139.stdout:1/305: fdatasync d7/d8/f1a 0 2024-09-16T00:09:42.578 INFO:tasks.workunit.client.0.smithi139.stdout:3/327: creat d5/d7/d23/d27/d2f/f72 x:0 0 0 2024-09-16T00:09:42.580 INFO:tasks.workunit.client.0.smithi139.stdout:6/247: mkdir d8/db/d26/d45/d59 0 2024-09-16T00:09:42.582 INFO:tasks.workunit.client.0.smithi139.stdout:6/248: mkdir d8/db/d26/d28/d5a 0 2024-09-16T00:09:42.585 INFO:tasks.workunit.client.0.smithi139.stdout:1/306: dwrite d7/d8/f36 [0,4194304] 0 2024-09-16T00:09:42.586 INFO:tasks.workunit.client.0.smithi139.stdout:3/328: rename d5/d7/d23/d27/d2f/d30/d41/l5c to d5/d7/l73 0 2024-09-16T00:09:42.588 INFO:tasks.workunit.client.0.smithi139.stdout:1/307: fsync d7/f25 0 2024-09-16T00:09:42.589 INFO:tasks.workunit.client.0.smithi139.stdout:6/249: rename d8/d20/d27/l1e to d8/d1f/l5b 0 2024-09-16T00:09:42.590 INFO:tasks.workunit.client.0.smithi139.stdout:3/329: creat d5/d7/d23/d27/d2f/f74 x:0 0 0 2024-09-16T00:09:42.592 INFO:tasks.workunit.client.0.smithi139.stdout:3/330: rmdir d5/d7 39 2024-09-16T00:09:42.596 INFO:tasks.workunit.client.0.smithi139.stdout:1/308: creat d7/f58 x:0 0 0 2024-09-16T00:09:42.597 INFO:tasks.workunit.client.0.smithi139.stdout:3/331: write d5/d7/d23/d27/d2f/d30/d4d/f60 [925067,62806] 0 2024-09-16T00:09:42.597 INFO:tasks.workunit.client.0.smithi139.stdout:3/332: readlink d5/l1c 0 2024-09-16T00:09:42.600 INFO:tasks.workunit.client.0.smithi139.stdout:1/309: rename d7/d8/d27/d2f/f30 to d7/d8/d27/f59 0 2024-09-16T00:09:42.601 INFO:tasks.workunit.client.0.smithi139.stdout:3/333: mkdir d5/d7/d23/d27/d2f/d30/d4c/d50/d75 0 2024-09-16T00:09:42.602 INFO:tasks.workunit.client.0.smithi139.stdout:1/310: mknod d7/d8/d40/d48/c5a 0 2024-09-16T00:09:42.603 INFO:tasks.workunit.client.0.smithi139.stdout:3/334: mkdir d5/d7/d23/d27/d2f/d30/d4c/d6d/d76 0 2024-09-16T00:09:42.604 INFO:tasks.workunit.client.0.smithi139.stdout:1/311: rename d7/d8/d40/d47/f56 to d7/d8/d40/d46/f5b 0 2024-09-16T00:09:42.606 INFO:tasks.workunit.client.0.smithi139.stdout:1/312: creat d7/d8/d27/d2f/f5c x:0 0 0 2024-09-16T00:09:42.607 INFO:tasks.workunit.client.0.smithi139.stdout:1/313: read d7/d8/f36 [521988,121116] 0 2024-09-16T00:09:42.608 INFO:tasks.workunit.client.0.smithi139.stdout:1/314: write d7/d8/f11 [547555,22981] 0 2024-09-16T00:09:42.615 INFO:tasks.workunit.client.0.smithi139.stdout:1/315: dwrite d7/f18 [4194304,4194304] 0 2024-09-16T00:09:42.616 INFO:tasks.workunit.client.0.smithi139.stdout:1/316: dread - d7/d8/d27/f59 zero size 2024-09-16T00:09:42.619 INFO:tasks.workunit.client.0.smithi139.stdout:1/317: dread d7/d8/d1f/f37 [0,4194304] 0 2024-09-16T00:09:42.619 INFO:tasks.workunit.client.0.smithi139.stdout:1/318: dread - d7/d8/d27/f57 zero size 2024-09-16T00:09:42.633 INFO:tasks.workunit.client.0.smithi139.stdout:1/319: dread d7/f26 [0,4194304] 0 2024-09-16T00:09:42.635 INFO:tasks.workunit.client.0.smithi139.stdout:1/320: write d7/d8/d40/d48/d4a/f4b [105089,105283] 0 2024-09-16T00:09:42.636 INFO:tasks.workunit.client.0.smithi139.stdout:1/321: rename d7/ff to d7/d8/d1f/f5d 0 2024-09-16T00:09:42.642 INFO:tasks.workunit.client.0.smithi139.stdout:1/322: dwrite d7/d8/d27/d2f/f5c [0,4194304] 0 2024-09-16T00:09:42.647 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:42.648 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:42.649 INFO:tasks.workunit.client.0.smithi139.stdout:1/323: dwrite d7/d1b/f3a [4194304,4194304] 0 2024-09-16T00:09:42.652 INFO:tasks.workunit.client.0.smithi139.stdout:1/324: unlink d7/d8/d27/l49 0 2024-09-16T00:09:42.654 INFO:tasks.workunit.client.0.smithi139.stdout:1/325: creat d7/d8/d40/d47/f5e x:0 0 0 2024-09-16T00:09:42.654 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:42.653+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:42.656 INFO:tasks.workunit.client.0.smithi139.stdout:1/326: symlink d7/d8/d40/d46/l5f 0 2024-09-16T00:09:42.658 INFO:tasks.workunit.client.0.smithi139.stdout:1/327: unlink d7/d8/d27/d2f/f33 0 2024-09-16T00:09:42.660 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:09:42.661 INFO:tasks.workunit.client.0.smithi139.stdout:1/328: symlink d7/l60 0 2024-09-16T00:09:42.663 INFO:tasks.workunit.client.0.smithi139.stdout:1/329: dread - d7/d8/d40/d47/f50 zero size 2024-09-16T00:09:42.663 INFO:tasks.workunit.client.0.smithi139.stdout:1/330: chown d7/d8/d40/d48 186216045 1 2024-09-16T00:09:42.665 INFO:tasks.workunit.client.0.smithi139.stdout:1/331: rename d7/d8/d4e to d7/d8/d40/d48/d53/d61 0 2024-09-16T00:09:42.667 INFO:tasks.workunit.client.0.smithi139.stdout:1/332: symlink d7/d8/d40/l62 0 2024-09-16T00:09:42.679 INFO:tasks.workunit.client.0.smithi139.stdout:1/333: dread d7/d8/d1f/f20 [0,4194304] 0 2024-09-16T00:09:42.680 INFO:tasks.workunit.client.0.smithi139.stdout:1/334: write d7/d8/d40/d48/d4a/f4b [245456,33073] 0 2024-09-16T00:09:42.681 INFO:tasks.workunit.client.0.smithi139.stdout:1/335: truncate d7/d8/d40/d47/f50 331816 0 2024-09-16T00:09:42.702 INFO:tasks.workunit.client.0.smithi139.stdout:5/292: sync 2024-09-16T00:09:42.707 INFO:tasks.workunit.client.0.smithi139.stdout:5/293: dwrite d1/d13/f24 [0,4194304] 0 2024-09-16T00:09:42.710 INFO:tasks.workunit.client.0.smithi139.stdout:5/294: creat d1/d25/d30/f60 x:0 0 0 2024-09-16T00:09:42.710 INFO:tasks.workunit.client.0.smithi139.stdout:5/295: fdatasync d1/f48 0 2024-09-16T00:09:42.713 INFO:tasks.workunit.client.0.smithi139.stdout:5/296: rename d1/d25/d30/f60 to d1/d13/d3f/d5e/f61 0 2024-09-16T00:09:42.715 INFO:tasks.workunit.client.0.smithi139.stdout:5/297: mknod d1/d21/c62 0 2024-09-16T00:09:42.719 INFO:tasks.workunit.client.0.smithi139.stdout:5/298: symlink d1/d25/d30/d33/d43/d5a/l63 0 2024-09-16T00:09:42.721 INFO:tasks.workunit.client.0.smithi139.stdout:5/299: mkdir d1/d25/d30/d64 0 2024-09-16T00:09:42.730 INFO:tasks.workunit.client.0.smithi139.stdout:5/300: dread d1/f27 [0,4194304] 0 2024-09-16T00:09:42.732 INFO:tasks.workunit.client.0.smithi139.stdout:5/301: symlink d1/d13/d3b/d3e/l65 0 2024-09-16T00:09:42.737 INFO:tasks.workunit.client.0.smithi139.stdout:5/302: dwrite d1/d13/f17 [0,4194304] 0 2024-09-16T00:09:42.737 INFO:tasks.workunit.client.0.smithi139.stdout:5/303: fdatasync d1/d25/f5c 0 2024-09-16T00:09:42.738 INFO:tasks.workunit.client.0.smithi139.stdout:5/304: dread - d1/d25/f5c zero size 2024-09-16T00:09:42.745 INFO:tasks.workunit.client.0.smithi139.stdout:5/305: dwrite d1/d13/f17 [4194304,4194304] 0 2024-09-16T00:09:42.748 INFO:tasks.workunit.client.0.smithi139.stdout:5/306: mknod d1/d21/c66 0 2024-09-16T00:09:42.751 INFO:tasks.workunit.client.0.smithi139.stdout:5/307: unlink d1/d13/d3b/d3e/f4e 0 2024-09-16T00:09:42.752 INFO:tasks.workunit.client.0.smithi139.stdout:5/308: symlink d1/d13/d3f/d5e/l67 0 2024-09-16T00:09:42.755 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:42.755+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:42.766 INFO:tasks.workunit.client.0.smithi139.stdout:5/309: dread d1/f1a [0,4194304] 0 2024-09-16T00:09:42.766 INFO:tasks.workunit.client.0.smithi139.stdout:5/310: dread d1/f16 [0,4194304] 0 2024-09-16T00:09:42.767 INFO:tasks.workunit.client.0.smithi139.stdout:5/311: readlink d1/d25/d30/d33/d43/l57 0 2024-09-16T00:09:42.767 INFO:tasks.workunit.client.0.smithi139.stdout:5/312: read d1/d13/f17 [2257811,70134] 0 2024-09-16T00:09:42.769 INFO:tasks.workunit.client.0.smithi139.stdout:5/313: write d1/d21/d49/d53/f51 [1022087,31581] 0 2024-09-16T00:09:42.769 INFO:tasks.workunit.client.0.smithi139.stdout:5/314: fdatasync d1/d25/d30/f55 0 2024-09-16T00:09:42.771 INFO:tasks.workunit.client.0.smithi139.stdout:5/315: fdatasync d1/d13/d3b/f47 0 2024-09-16T00:09:42.773 INFO:tasks.workunit.client.0.smithi139.stdout:5/316: mkdir d1/d25/d30/d33/d68 0 2024-09-16T00:09:42.776 INFO:tasks.workunit.client.0.smithi139.stdout:5/317: dread d1/d13/f24 [0,4194304] 0 2024-09-16T00:09:42.782 INFO:tasks.workunit.client.0.smithi139.stdout:5/318: dwrite d1/d13/f24 [0,4194304] 0 2024-09-16T00:09:42.783 INFO:tasks.workunit.client.0.smithi139.stdout:5/319: creat d1/d13/d3b/d3e/f69 x:0 0 0 2024-09-16T00:09:42.783 INFO:tasks.workunit.client.0.smithi139.stdout:5/320: stat d1/d25/d30/d33/d5b 0 2024-09-16T00:09:42.785 INFO:tasks.workunit.client.0.smithi139.stdout:5/321: mkdir d1/d21/d49/d53/d6a 0 2024-09-16T00:09:42.788 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:42.789 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:42.789 INFO:tasks.workunit.client.0.smithi139.stdout:5/322: dwrite d1/f4a [0,4194304] 0 2024-09-16T00:09:42.792 INFO:tasks.workunit.client.0.smithi139.stdout:5/323: fsync d1/f27 0 2024-09-16T00:09:42.793 INFO:tasks.workunit.client.0.smithi139.stdout:5/324: truncate d1/f2f 659588 0 2024-09-16T00:09:42.797 INFO:tasks.workunit.client.0.smithi139.stdout:5/325: dwrite d1/f48 [0,4194304] 0 2024-09-16T00:09:42.801 INFO:tasks.workunit.client.0.smithi139.stdout:5/326: creat d1/d13/d22/f6b x:0 0 0 2024-09-16T00:09:42.802 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:09:42.803 INFO:tasks.workunit.client.0.smithi139.stdout:5/327: creat d1/d25/f6c x:0 0 0 2024-09-16T00:09:42.804 INFO:tasks.workunit.client.0.smithi139.stdout:5/328: unlink d1/d25/d30/f58 0 2024-09-16T00:09:42.806 INFO:tasks.workunit.client.0.smithi139.stdout:5/329: mknod d1/d13/d3b/d3e/c6d 0 2024-09-16T00:09:42.807 INFO:tasks.workunit.client.0.smithi139.stdout:5/330: creat d1/d13/d3b/d3e/f6e x:0 0 0 2024-09-16T00:09:42.814 INFO:tasks.workunit.client.0.smithi139.stdout:5/331: truncate d1/d21/d49/d53/f51 434325 0 2024-09-16T00:09:42.817 INFO:tasks.workunit.client.0.smithi139.stdout:5/332: dread d1/fa [0,4194304] 0 2024-09-16T00:09:42.817 INFO:tasks.workunit.client.0.smithi139.stdout:7/290: sync 2024-09-16T00:09:42.817 INFO:tasks.workunit.client.0.smithi139.stdout:0/318: sync 2024-09-16T00:09:42.817 INFO:tasks.workunit.client.0.smithi139.stdout:9/316: sync 2024-09-16T00:09:42.817 INFO:tasks.workunit.client.0.smithi139.stdout:8/337: sync 2024-09-16T00:09:42.818 INFO:tasks.workunit.client.0.smithi139.stdout:2/292: sync 2024-09-16T00:09:42.818 INFO:tasks.workunit.client.0.smithi139.stdout:4/231: sync 2024-09-16T00:09:42.819 INFO:tasks.workunit.client.0.smithi139.stdout:8/338: fsync d17/d1a/f1e 0 2024-09-16T00:09:42.819 INFO:tasks.workunit.client.0.smithi139.stdout:2/293: dread - d11/d2f/f54 zero size 2024-09-16T00:09:42.819 INFO:tasks.workunit.client.0.smithi139.stdout:5/333: dread d1/d13/d3b/f4c [0,4194304] 0 2024-09-16T00:09:42.819 INFO:tasks.workunit.client.0.smithi139.stdout:4/232: mknod d12/d4a/c4e 0 2024-09-16T00:09:42.820 INFO:tasks.workunit.client.0.smithi139.stdout:7/291: truncate d5/d26/f32 1224317 0 2024-09-16T00:09:42.821 INFO:tasks.workunit.client.0.smithi139.stdout:5/334: stat d1/d21/d49/d53 0 2024-09-16T00:09:42.821 INFO:tasks.workunit.client.0.smithi139.stdout:9/317: creat d19/d1a/d20/d23/d2d/d35/f61 x:0 0 0 2024-09-16T00:09:42.822 INFO:tasks.workunit.client.0.smithi139.stdout:2/294: unlink d11/d30/d57/d29/d2b/f2e 0 2024-09-16T00:09:42.824 INFO:tasks.workunit.client.0.smithi139.stdout:5/335: symlink d1/d13/d3f/d5e/l6f 0 2024-09-16T00:09:42.825 INFO:tasks.workunit.client.0.smithi139.stdout:8/339: link c10 d17/c72 0 2024-09-16T00:09:42.827 INFO:tasks.workunit.client.0.smithi139.stdout:2/295: dwrite ff [0,4194304] 0 2024-09-16T00:09:42.827 INFO:tasks.workunit.client.0.smithi139.stdout:7/292: creat d5/d26/f62 x:0 0 0 2024-09-16T00:09:42.828 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:09:42.829 INFO:tasks.workunit.client.0.smithi139.stdout:9/318: creat d19/d1a/d20/d23/d2d/d35/d60/f62 x:0 0 0 2024-09-16T00:09:42.831 INFO:tasks.workunit.client.0.smithi139.stdout:8/340: creat d17/d1a/f73 x:0 0 0 2024-09-16T00:09:42.833 INFO:tasks.workunit.client.0.smithi139.stdout:2/296: read fc [319127,82317] 0 2024-09-16T00:09:42.833 INFO:tasks.workunit.client.0.smithi139.stdout:7/293: mkdir d5/d16/d22/d25/d63 0 2024-09-16T00:09:42.834 INFO:tasks.workunit.client.0.smithi139.stdout:4/233: truncate f4 1169732 0 2024-09-16T00:09:42.835 INFO:tasks.workunit.client.0.smithi139.stdout:9/319: rmdir d19/d1a 39 2024-09-16T00:09:42.836 INFO:tasks.workunit.client.0.smithi139.stdout:5/336: dwrite d1/f16 [0,4194304] 0 2024-09-16T00:09:42.836 INFO:tasks.workunit.client.0.smithi139.stdout:5/337: stat d1/d25/c29 0 2024-09-16T00:09:42.837 INFO:tasks.workunit.client.0.smithi139.stdout:8/341: rename d17/d1a/d50/l67 to d17/d1a/d2d/l74 0 2024-09-16T00:09:42.839 INFO:tasks.workunit.client.0.smithi139.stdout:2/297: unlink d11/d30/d57/f24 0 2024-09-16T00:09:42.842 INFO:tasks.workunit.client.0.smithi139.stdout:7/294: unlink d5/d16/d22/d25/d2d/d50/f53 0 2024-09-16T00:09:42.843 INFO:tasks.workunit.client.0.smithi139.stdout:5/338: mknod d1/d13/d3b/c70 0 2024-09-16T00:09:42.845 INFO:tasks.workunit.client.0.smithi139.stdout:5/339: fsync d1/d13/f24 0 2024-09-16T00:09:42.847 INFO:tasks.workunit.client.0.smithi139.stdout:8/342: dwrite d17/d1f/f2c [0,4194304] 0 2024-09-16T00:09:42.849 INFO:tasks.workunit.client.0.smithi139.stdout:2/298: dread fc [0,4194304] 0 2024-09-16T00:09:42.849 INFO:tasks.workunit.client.0.smithi139.stdout:7/295: rmdir d5/d16/d22 39 2024-09-16T00:09:42.850 INFO:tasks.workunit.client.0.smithi139.stdout:7/296: stat d5/cf 0 2024-09-16T00:09:42.853 INFO:tasks.workunit.client.0.smithi139.stdout:9/320: write f2 [1302774,5495] 0 2024-09-16T00:09:42.854 INFO:tasks.workunit.client.0.smithi139.stdout:5/340: rename d1/d13/d22/l5d to d1/d25/d30/d64/l71 0 2024-09-16T00:09:42.856 INFO:tasks.workunit.client.0.smithi139.stdout:8/343: creat d17/d1a/d43/d40/d4f/f75 x:0 0 0 2024-09-16T00:09:42.857 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:42.856+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:42.857 INFO:tasks.workunit.client.0.smithi139.stdout:8/344: chown d17/d1f/c24 7841630 1 2024-09-16T00:09:42.857 INFO:tasks.workunit.client.0.smithi139.stdout:4/234: creat d12/f4f x:0 0 0 2024-09-16T00:09:42.858 INFO:tasks.workunit.client.0.smithi139.stdout:4/235: write d12/d47/f2c [364811,94830] 0 2024-09-16T00:09:42.860 INFO:tasks.workunit.client.0.smithi139.stdout:2/299: rename d11/l17 to d11/d30/d57/d29/d2b/l5e 0 2024-09-16T00:09:42.861 INFO:tasks.workunit.client.0.smithi139.stdout:4/236: readlink d12/d47/d49/l4b 0 2024-09-16T00:09:42.863 INFO:tasks.workunit.client.0.smithi139.stdout:5/341: fsync d1/d21/d49/d53/f51 0 2024-09-16T00:09:42.864 INFO:tasks.workunit.client.0.smithi139.stdout:9/321: creat d19/d1a/d20/d23/d2d/d35/d3d/f63 x:0 0 0 2024-09-16T00:09:42.864 INFO:tasks.workunit.client.0.smithi139.stdout:2/300: rmdir d11/d30/d57 39 2024-09-16T00:09:42.865 INFO:tasks.workunit.client.0.smithi139.stdout:9/322: readlink ld 0 2024-09-16T00:09:42.865 INFO:tasks.workunit.client.0.smithi139.stdout:8/345: dwrite d17/d1a/d2d/f2e [4194304,4194304] 0 2024-09-16T00:09:42.865 INFO:tasks.workunit.client.0.smithi139.stdout:4/237: write d12/f23 [2034559,11886] 0 2024-09-16T00:09:42.868 INFO:tasks.workunit.client.0.smithi139.stdout:5/342: mknod d1/c72 0 2024-09-16T00:09:42.870 INFO:tasks.workunit.client.0.smithi139.stdout:2/301: write d11/d2f/f54 [438150,6995] 0 2024-09-16T00:09:42.870 INFO:tasks.workunit.client.0.smithi139.stdout:5/343: stat d1/d13/d3f 0 2024-09-16T00:09:42.871 INFO:tasks.workunit.client.0.smithi139.stdout:7/297: dread d5/d16/f34 [0,4194304] 0 2024-09-16T00:09:42.872 INFO:tasks.workunit.client.0.smithi139.stdout:4/238: symlink d12/l50 0 2024-09-16T00:09:42.872 INFO:tasks.workunit.client.0.smithi139.stdout:7/298: read - d5/d26/f62 zero size 2024-09-16T00:09:42.873 INFO:tasks.workunit.client.0.smithi139.stdout:8/346: symlink d17/d1a/d43/d40/d4f/l76 0 2024-09-16T00:09:42.874 INFO:tasks.workunit.client.0.smithi139.stdout:8/347: dread - d17/f70 zero size 2024-09-16T00:09:42.876 INFO:tasks.workunit.client.0.smithi139.stdout:4/239: mknod d12/d2b/c51 0 2024-09-16T00:09:42.877 INFO:tasks.workunit.client.0.smithi139.stdout:4/240: write d12/d47/f24 [4966300,33201] 0 2024-09-16T00:09:42.879 INFO:tasks.workunit.client.0.smithi139.stdout:2/302: mknod d11/d30/d57/d29/c5f 0 2024-09-16T00:09:42.880 INFO:tasks.workunit.client.0.smithi139.stdout:5/344: truncate d1/d21/d49/d53/f51 272436 0 2024-09-16T00:09:42.880 INFO:tasks.workunit.client.0.smithi139.stdout:2/303: chown d11/f19 13 1 2024-09-16T00:09:42.882 INFO:tasks.workunit.client.0.smithi139.stdout:4/241: mknod d12/d47/c52 0 2024-09-16T00:09:42.882 INFO:tasks.workunit.client.0.smithi139.stdout:9/323: truncate f14 2118592 0 2024-09-16T00:09:42.882 INFO:tasks.workunit.client.0.smithi139.stdout:4/242: read d12/d47/f2c [1028189,26863] 0 2024-09-16T00:09:42.884 INFO:tasks.workunit.client.0.smithi139.stdout:7/299: symlink d5/d16/d22/d25/d2d/d50/l64 0 2024-09-16T00:09:42.885 INFO:tasks.workunit.client.0.smithi139.stdout:5/345: mkdir d1/d25/d73 0 2024-09-16T00:09:42.886 INFO:tasks.workunit.client.0.smithi139.stdout:2/304: symlink d11/d30/d57/l60 0 2024-09-16T00:09:42.889 INFO:tasks.workunit.client.0.smithi139.stdout:4/243: dwrite d12/f42 [0,4194304] 0 2024-09-16T00:09:42.891 INFO:tasks.workunit.client.0.smithi139.stdout:8/348: getdents d17/d1a/d50 0 2024-09-16T00:09:42.892 INFO:tasks.workunit.client.0.smithi139.stdout:2/305: creat d11/d2f/f61 x:0 0 0 2024-09-16T00:09:42.894 INFO:tasks.workunit.client.0.smithi139.stdout:9/324: rename d19/d1a/d20/d23/d2d/c4b to d19/d1a/d20/d23/d2d/d35/d3d/c64 0 2024-09-16T00:09:42.896 INFO:tasks.workunit.client.0.smithi139.stdout:9/325: chown d19/d1a/d20/d23/d2d/d35/d3d/l46 12654 1 2024-09-16T00:09:42.897 INFO:tasks.workunit.client.0.smithi139.stdout:5/346: creat d1/d13/d22/f74 x:0 0 0 2024-09-16T00:09:42.900 INFO:tasks.workunit.client.0.smithi139.stdout:4/244: rename d12/f2a to d12/d47/d49/f53 0 2024-09-16T00:09:42.902 INFO:tasks.workunit.client.0.smithi139.stdout:9/326: mknod d19/d1a/d20/d23/d2d/d2f/c65 0 2024-09-16T00:09:42.904 INFO:tasks.workunit.client.0.smithi139.stdout:4/245: dread d12/d2b/f18 [0,4194304] 0 2024-09-16T00:09:42.905 INFO:tasks.workunit.client.0.smithi139.stdout:7/300: link d5/d16/d22/d25/d2d/l45 d5/l65 0 2024-09-16T00:09:42.906 INFO:tasks.workunit.client.0.smithi139.stdout:9/327: creat d19/d1a/d20/d23/d2d/d35/f66 x:0 0 0 2024-09-16T00:09:42.906 INFO:tasks.workunit.client.0.smithi139.stdout:9/328: readlink ld 0 2024-09-16T00:09:42.908 INFO:tasks.workunit.client.0.smithi139.stdout:9/329: write d19/d1a/d20/d23/d2d/d35/d60/f62 [386417,96180] 0 2024-09-16T00:09:42.909 INFO:tasks.workunit.client.0.smithi139.stdout:2/306: rename d11/d30/d57/d29/c2a to d11/d30/d57/c62 0 2024-09-16T00:09:42.911 INFO:tasks.workunit.client.0.smithi139.stdout:4/246: creat d12/d4a/f54 x:0 0 0 2024-09-16T00:09:42.911 INFO:tasks.workunit.client.0.smithi139.stdout:4/247: fdatasync d12/f42 0 2024-09-16T00:09:42.914 INFO:tasks.workunit.client.0.smithi139.stdout:9/330: mkdir d19/d22/d32/d4e/d67 0 2024-09-16T00:09:42.918 INFO:tasks.workunit.client.0.smithi139.stdout:4/248: unlink d12/f38 0 2024-09-16T00:09:42.919 INFO:tasks.workunit.client.0.smithi139.stdout:5/347: link d1/d21/l41 d1/d13/l75 0 2024-09-16T00:09:42.920 INFO:tasks.workunit.client.0.smithi139.stdout:2/307: mknod d11/d30/d57/d5d/c63 0 2024-09-16T00:09:42.921 INFO:tasks.workunit.client.0.smithi139.stdout:2/308: write d11/d30/d57/f27 [82831,36626] 0 2024-09-16T00:09:42.922 INFO:tasks.workunit.client.0.smithi139.stdout:7/301: link d5/d26/f4c d5/d26/d42/f66 0 2024-09-16T00:09:42.924 INFO:tasks.workunit.client.0.smithi139.stdout:3/335: sync 2024-09-16T00:09:42.925 INFO:tasks.workunit.client.0.smithi139.stdout:1/336: sync 2024-09-16T00:09:42.925 INFO:tasks.workunit.client.0.smithi139.stdout:6/250: sync 2024-09-16T00:09:42.926 INFO:tasks.workunit.client.0.smithi139.stdout:3/336: write d5/d7/d23/d27/d2f/d30/d4d/f60 [1469933,110746] 0 2024-09-16T00:09:42.927 INFO:tasks.workunit.client.0.smithi139.stdout:5/348: mknod d1/d21/d49/d53/c76 0 2024-09-16T00:09:42.932 INFO:tasks.workunit.client.0.smithi139.stdout:4/249: creat d12/d4a/d4d/f55 x:0 0 0 2024-09-16T00:09:42.934 INFO:tasks.workunit.client.0.smithi139.stdout:1/337: mkdir d7/d8/d27/d2f/d63 0 2024-09-16T00:09:42.936 INFO:tasks.workunit.client.0.smithi139.stdout:2/309: stat d11/d30/d57/d29/c4a 0 2024-09-16T00:09:42.939 INFO:tasks.workunit.client.0.smithi139.stdout:4/250: rmdir d12/d2b 39 2024-09-16T00:09:42.940 INFO:tasks.workunit.client.0.smithi139.stdout:6/251: write d8/d20/d27/d10/f37 [421524,65868] 0 2024-09-16T00:09:42.943 INFO:tasks.workunit.client.0.smithi139.stdout:4/251: symlink d12/d47/l56 0 2024-09-16T00:09:42.944 INFO:tasks.workunit.client.0.smithi139.stdout:6/252: creat d8/d1f/f5c x:0 0 0 2024-09-16T00:09:42.946 INFO:tasks.workunit.client.0.smithi139.stdout:3/337: rename d5/d7/d23/d35/f6a to d5/d7/d23/f77 0 2024-09-16T00:09:42.947 INFO:tasks.workunit.client.0.smithi139.stdout:5/349: getdents d1/d25/d30/d33 0 2024-09-16T00:09:42.949 INFO:tasks.workunit.client.0.smithi139.stdout:5/350: write d1/d25/f36 [9364859,82935] 0 2024-09-16T00:09:42.949 INFO:tasks.workunit.client.0.smithi139.stdout:6/253: readlink d8/d20/d27/d10/l33 0 2024-09-16T00:09:42.949 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:42.949 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:42.950 INFO:tasks.workunit.client.0.smithi139.stdout:2/310: rename d11/d32/f43 to d11/f64 0 2024-09-16T00:09:42.952 INFO:tasks.workunit.client.0.smithi139.stdout:7/302: write d5/d26/f2b [2050824,82233] 0 2024-09-16T00:09:42.954 INFO:tasks.workunit.client.0.smithi139.stdout:6/254: unlink d8/d20/d27/f53 0 2024-09-16T00:09:42.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.953+0000 7f401a54b640 1 Processor -- start 2024-09-16T00:09:42.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.953+0000 7f401a54b640 1 -- start start 2024-09-16T00:09:42.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.954+0000 7f401a54b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40140b4750 0x7f40141546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:42.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.954+0000 7f401a54b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f401415adc0 con 0x7f40140d0490 2024-09-16T00:09:42.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.954+0000 7f401a54b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f401415b0a0 con 0x7f40140d0870 2024-09-16T00:09:42.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.954+0000 7f401a54b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f401415b940 con 0x7f40140b4750 2024-09-16T00:09:42.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.955+0000 7f4013fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f40140d0490 0x7f40140b2c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45542/0 (socket says 172.21.15.139:45542) 2024-09-16T00:09:42.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.955+0000 7f4013fff640 1 -- 172.21.15.139:0/1490058585 learned_addr learned my addr 172.21.15.139:0/1490058585 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:42.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.955+0000 7f4018ae8640 1 --2- 172.21.15.139:0/1490058585 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40140b4750 0x7f40141546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:42.957 INFO:tasks.workunit.client.0.smithi139.stdout:3/338: rmdir d5/d7/d23/d48 39 2024-09-16T00:09:42.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.955+0000 7f4012ffd640 1 -- 172.21.15.139:0/1490058585 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1418162556 0 0) 0x7f401415adc0 con 0x7f40140d0490 2024-09-16T00:09:42.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.955+0000 7f4012ffd640 1 -- 172.21.15.139:0/1490058585 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ffc003520 con 0x7f40140d0490 2024-09-16T00:09:42.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.955+0000 7f4012ffd640 1 -- 172.21.15.139:0/1490058585 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3271838976 0 0) 0x7f401415b0a0 con 0x7f40140d0870 2024-09-16T00:09:42.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.955+0000 7f4012ffd640 1 -- 172.21.15.139:0/1490058585 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f401415adc0 con 0x7f40140d0870 2024-09-16T00:09:42.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.955+0000 7f4018ae8640 1 -- 172.21.15.139:0/1490058585 >> v1:172.21.15.164:6789/0 conn(0x7f40140d0870 legacy=0x7f40140b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:42.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.955+0000 7f4018ae8640 1 -- 172.21.15.139:0/1490058585 >> v1:172.21.15.139:6789/0 conn(0x7f40140d0490 legacy=0x7f40140b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:42.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.955+0000 7f4018ae8640 1 -- 172.21.15.139:0/1490058585 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401415b380 con 0x7f40140b4750 2024-09-16T00:09:42.958 INFO:tasks.workunit.client.0.smithi139.stdout:5/351: truncate d1/d13/d3b/f44 2909230 0 2024-09-16T00:09:42.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.956+0000 7f4018ae8640 1 --2- 172.21.15.139:0/1490058585 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40140b4750 0x7f40141546a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f400c009dc0 tx=0x7f400c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c726231914ff4cfe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:42.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.957+0000 7f4012ffd640 1 -- 172.21.15.139:0/1490058585 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c007740 con 0x7f40140b4750 2024-09-16T00:09:42.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.957+0000 7f4012ffd640 1 -- 172.21.15.139:0/1490058585 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f400c0078e0 con 0x7f40140b4750 2024-09-16T00:09:42.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.957+0000 7f4012ffd640 1 -- 172.21.15.139:0/1490058585 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c007c90 con 0x7f40140b4750 2024-09-16T00:09:42.959 INFO:tasks.workunit.client.0.smithi139.stdout:9/331: dwrite d19/f1d [0,4194304] 0 2024-09-16T00:09:42.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.957+0000 7f401a54b640 1 -- 172.21.15.139:0/1490058585 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40140b4750 msgr2=0x7f40141546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:42.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.957+0000 7f401a54b640 1 --2- 172.21.15.139:0/1490058585 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40140b4750 0x7f40141546a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f400c009dc0 tx=0x7f400c02a710 comp rx=0 tx=0).stop 2024-09-16T00:09:42.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.958+0000 7f401a54b640 1 -- 172.21.15.139:0/1490058585 shutdown_connections 2024-09-16T00:09:42.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.958+0000 7f401a54b640 1 --2- 172.21.15.139:0/1490058585 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40140b4750 0x7f40141546a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:42.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.958+0000 7f401a54b640 1 -- 172.21.15.139:0/1490058585 >> 172.21.15.139:0/1490058585 conn(0x7f401406c520 msgr2=0x7f401406c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:42.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.958+0000 7f401a54b640 1 -- 172.21.15.139:0/1490058585 shutdown_connections 2024-09-16T00:09:42.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.958+0000 7f401a54b640 1 -- 172.21.15.139:0/1490058585 wait complete. 2024-09-16T00:09:42.961 INFO:tasks.workunit.client.0.smithi139.stdout:3/339: readlink d5/d7/d23/l22 0 2024-09-16T00:09:42.962 INFO:tasks.workunit.client.0.smithi139.stdout:5/352: chown d1/l35 2930608 1 2024-09-16T00:09:42.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.959+0000 7f401a54b640 1 Processor -- start 2024-09-16T00:09:42.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.960+0000 7f401a54b640 1 -- start start 2024-09-16T00:09:42.962 INFO:tasks.workunit.client.0.smithi139.stdout:2/311: rename f8 to d11/d30/d57/d29/d2b/f65 0 2024-09-16T00:09:42.963 INFO:tasks.workunit.client.0.smithi139.stdout:6/255: rmdir d8/db/d26/d45 39 2024-09-16T00:09:42.963 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:42.960+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:42.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.961+0000 7f401a54b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40140b4750 0x7f40141f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:42.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.961+0000 7f401a54b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40140d0490 0x7f401414f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:42.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.961+0000 7f401a54b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40140d0870 0x7f40140c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:42.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.961+0000 7f401a54b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f401415a980 con 0x7f40140b4750 2024-09-16T00:09:42.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.961+0000 7f401a54b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f40140b3bb0 con 0x7f40140d0490 2024-09-16T00:09:42.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.961+0000 7f401a54b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f401415ab00 con 0x7f40140d0870 2024-09-16T00:09:42.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.961+0000 7f4013fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40140b4750 0x7f40141f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:42.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.961+0000 7f4018ae8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40140d0870 0x7f40140c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:42.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.961+0000 7f4013fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40140b4750 0x7f40141f08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43786/0 (socket says 172.21.15.139:43786) 2024-09-16T00:09:42.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.961+0000 7f4013fff640 1 -- 172.21.15.139:0/827921768 learned_addr learned my addr 172.21.15.139:0/827921768 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:42.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.961+0000 7f40137fe640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40140d0490 0x7f401414f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:42.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.961+0000 7f4018ae8640 1 -- 172.21.15.139:0/827921768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40140d0490 msgr2=0x7f401414f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:42.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.961+0000 7f4018ae8640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f40140d0490 0x7f401414f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:42.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.961+0000 7f4018ae8640 1 -- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40140b4750 msgr2=0x7f40141f08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:42.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.961+0000 7f4018ae8640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40140b4750 0x7f40141f08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:42.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.961+0000 7f4018ae8640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401414e1b0 con 0x7f40140d0870 2024-09-16T00:09:42.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.962+0000 7f4018ae8640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f40140d0870 0x7f40140c97c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f400c031000 tx=0x7f400c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:42.970 INFO:tasks.workunit.client.0.smithi139.stdout:7/303: creat d5/d16/d22/d25/d2d/d50/f67 x:0 0 0 2024-09-16T00:09:42.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.962+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c009450 con 0x7f40140d0870 2024-09-16T00:09:42.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.962+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40141f0f70 con 0x7f40140d0870 2024-09-16T00:09:42.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.962+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f400c004240 con 0x7f40140d0870 2024-09-16T00:09:42.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.962+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c004630 con 0x7f40140d0870 2024-09-16T00:09:42.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.962+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f40141f1530 con 0x7f40140d0870 2024-09-16T00:09:42.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.964+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f40140b3430 con 0x7f40140d0870 2024-09-16T00:09:42.972 INFO:tasks.workunit.client.0.smithi139.stdout:2/312: fsync d11/f53 0 2024-09-16T00:09:42.974 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:09:42.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.965+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f400c039040 con 0x7f40140d0870 2024-09-16T00:09:42.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.965+0000 7f4010ff9640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f400006c2e0 0x7f400006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:42.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.965+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(87..87 src has 1..87) ==== 5973+0+0 (secure 0 0 0) 0x7f400c0acad0 con 0x7f40140d0870 2024-09-16T00:09:42.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.965+0000 7f4013fff640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f400006c2e0 0x7f400006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:42.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.965+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v87) ==== 54+0+16 (secure 0 0 0) 0x7f401414e1b0 con 0x7f40140d0870 2024-09-16T00:09:42.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.966+0000 7f401a54b640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f40141f3e30 0x7f40140c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:42.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.966+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40140c9460 con 0x7f40141f3e30 2024-09-16T00:09:42.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.966+0000 7f40137fe640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f40141f3e30 0x7f40140c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:42.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.967+0000 7f4013fff640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f400006c2e0 0x7f400006e7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f40040011e0 tx=0x7f4004009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:42.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.967+0000 7f40137fe640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f40141f3e30 0x7f40140c8f20 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:42.976 INFO:tasks.workunit.client.0.smithi139.stdout:9/332: write d19/d1a/f21 [3098843,117709] 0 2024-09-16T00:09:42.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.967+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f40140c9460 con 0x7f40141f3e30 2024-09-16T00:09:42.977 INFO:tasks.workunit.client.0.smithi139.stdout:4/252: dwrite d12/d2b/f18 [0,4194304] 0 2024-09-16T00:09:42.977 INFO:tasks.workunit.client.0.smithi139.stdout:1/338: dwrite d7/d8/f36 [0,4194304] 0 2024-09-16T00:09:42.977 INFO:tasks.workunit.client.0.smithi139.stdout:5/353: write d1/d25/d30/d33/f50 [341278,13274] 0 2024-09-16T00:09:42.977 INFO:tasks.workunit.client.0.smithi139.stdout:7/304: creat d5/d26/d46/f68 x:0 0 0 2024-09-16T00:09:42.977 INFO:tasks.workunit.client.0.smithi139.stdout:1/339: dread - d7/d8/d27/f59 zero size 2024-09-16T00:09:42.977 INFO:tasks.workunit.client.0.smithi139.stdout:7/305: fdatasync d5/d16/d22/f4e 0 2024-09-16T00:09:42.977 INFO:tasks.workunit.client.0.smithi139.stdout:4/253: unlink d12/c29 0 2024-09-16T00:09:42.978 INFO:tasks.workunit.client.0.smithi139.stdout:5/354: creat d1/d13/d3f/f77 x:0 0 0 2024-09-16T00:09:42.978 INFO:tasks.workunit.client.0.smithi139.stdout:6/256: truncate d8/d20/d27/d10/f13 4164132 0 2024-09-16T00:09:42.979 INFO:tasks.workunit.client.0.smithi139.stdout:7/306: readlink d5/l18 0 2024-09-16T00:09:42.979 INFO:tasks.workunit.client.0.smithi139.stdout:1/340: mknod d7/d8/d40/d47/c64 0 2024-09-16T00:09:42.981 INFO:tasks.workunit.client.0.smithi139.stdout:4/254: chown d12/f3f 301847 1 2024-09-16T00:09:42.983 INFO:tasks.workunit.client.0.smithi139.stdout:9/333: rename d19/d1a/d20/d23/d2d/d41/c43 to d19/d1a/d20/d59/c68 0 2024-09-16T00:09:42.984 INFO:tasks.workunit.client.0.smithi139.stdout:7/307: mknod d5/d16/d22/d25/d2d/c69 0 2024-09-16T00:09:42.986 INFO:tasks.workunit.client.0.smithi139.stdout:1/341: dread d7/d8/d40/d48/d4a/f4b [0,4194304] 0 2024-09-16T00:09:42.986 INFO:tasks.workunit.client.0.smithi139.stdout:2/313: getdents d11/d32 0 2024-09-16T00:09:42.988 INFO:tasks.workunit.client.0.smithi139.stdout:5/355: fsync d1/d13/d3b/f44 0 2024-09-16T00:09:42.990 INFO:tasks.workunit.client.0.smithi139.stdout:3/340: dread d5/d7/d23/d27/d2f/d30/d47/f25 [0,4194304] 0 2024-09-16T00:09:42.991 INFO:tasks.workunit.client.0.smithi139.stdout:6/257: link d8/d20/d27/l34 d8/d1f/l5d 0 2024-09-16T00:09:42.991 INFO:tasks.workunit.client.0.smithi139.stdout:7/308: read - d5/d16/d22/d25/d2d/d50/d44/f60 zero size 2024-09-16T00:09:42.992 INFO:tasks.workunit.client.0.smithi139.stdout:4/255: dread d12/d2b/f26 [0,4194304] 0 2024-09-16T00:09:42.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.991+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4014151ed0 con 0x7f40141f3e30 2024-09-16T00:09:42.992 INFO:tasks.workunit.client.0.smithi139.stdout:1/342: creat d7/d8/d40/d48/f65 x:0 0 0 2024-09-16T00:09:42.993 INFO:tasks.workunit.client.0.smithi139.stdout:6/258: write d8/f39 [655493,50069] 0 2024-09-16T00:09:42.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.993+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4014151ed0 con 0x7f40141f3e30 2024-09-16T00:09:42.994 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:42.994 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:42.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.993+0000 7f401a54b640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f40141564e0 0x7f40141568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:42.995 INFO:tasks.workunit.client.0.smithi139.stdout:2/314: rename d11/d2f/f54 to d11/d30/d57/d29/f66 0 2024-09-16T00:09:42.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.993+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f40140cb170 con 0x7f40141564e0 2024-09-16T00:09:42.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.993+0000 7f4013fff640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f40141564e0 0x7f40141568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:42.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.994+0000 7f4013fff640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f40141564e0 0x7f40141568c0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:42.996 INFO:tasks.workunit.client.0.smithi139.stdout:9/334: symlink d19/d1a/d20/d23/d2d/d35/d42/d3a/l69 0 2024-09-16T00:09:42.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:42.994+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f40140cb170 con 0x7f40141564e0 2024-09-16T00:09:42.997 INFO:tasks.workunit.client.0.smithi139.stdout:5/356: write d1/d13/d3f/d5e/f61 [299337,70938] 0 2024-09-16T00:09:42.998 INFO:tasks.workunit.client.0.smithi139.stdout:4/256: mkdir d12/d47/d57 0 2024-09-16T00:09:42.998 INFO:tasks.workunit.client.0.smithi139.stdout:5/357: readlink d1/lc 0 2024-09-16T00:09:43.000 INFO:tasks.workunit.client.0.smithi139.stdout:2/315: mknod d11/d30/d57/d5d/c67 0 2024-09-16T00:09:43.001 INFO:tasks.workunit.client.0.smithi139.stdout:7/309: dwrite d5/d16/d22/d25/d2d/d50/d44/f60 [0,4194304] 0 2024-09-16T00:09:43.006 INFO:tasks.workunit.client.0.smithi139.stdout:3/341: creat d5/d7/f78 x:0 0 0 2024-09-16T00:09:43.006 INFO:tasks.workunit.client.0.smithi139.stdout:1/343: write d7/f25 [1337137,38859] 0 2024-09-16T00:09:43.006 INFO:tasks.workunit.client.0.smithi139.stdout:3/342: read - d5/d7/f49 zero size 2024-09-16T00:09:43.008 INFO:tasks.workunit.client.0.smithi139.stdout:9/335: mknod d19/d1a/d20/d23/d2d/d35/d42/c6a 0 2024-09-16T00:09:43.009 INFO:tasks.workunit.client.0.smithi139.stdout:4/257: symlink d12/l58 0 2024-09-16T00:09:43.010 INFO:tasks.workunit.client.0.smithi139.stdout:2/316: mknod d11/d30/d57/d29/d2b/c68 0 2024-09-16T00:09:43.011 INFO:tasks.workunit.client.0.smithi139.stdout:7/310: mknod d5/d16/d22/d25/d2d/c6a 0 2024-09-16T00:09:43.013 INFO:tasks.workunit.client.0.smithi139.stdout:6/259: creat d8/db/d26/d45/d59/f5e x:0 0 0 2024-09-16T00:09:43.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.013+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f40140b2f60 con 0x7f40141564e0 2024-09-16T00:09:43.014 INFO:tasks.workunit.client.0.smithi139.stdout:1/344: truncate d7/f16 856135 0 2024-09-16T00:09:43.014 INFO:tasks.workunit.client.0.smithi139.stdout:6/260: dread - d8/db/d26/d45/f46 zero size 2024-09-16T00:09:43.015 INFO:tasks.workunit.client.0.smithi139.stdout:3/343: rename d5/d7/d23/d27/d2f/d30/f36 to d5/d7/d23/d27/d2f/d30/d4c/d6d/f79 0 2024-09-16T00:09:43.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.014+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f40140b2f60 con 0x7f40141564e0 2024-09-16T00:09:43.016 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:43.017 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:43.017 INFO:tasks.workunit.client.0.smithi139.stdout:6/261: chown d8/db/d26/d45/f43 3 1 2024-09-16T00:09:43.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.015+0000 7f401a54b640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f4014156e00 0x7f40140c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:43.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.015+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f40140b2c80 con 0x7f4014156e00 2024-09-16T00:09:43.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.015+0000 7f40137fe640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f4014156e00 0x7f40140c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:43.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.015+0000 7f40137fe640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f4014156e00 0x7f40140c7d70 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:43.019 INFO:tasks.workunit.client.0.smithi139.stdout:4/258: unlink d12/d2b/f2e 0 2024-09-16T00:09:43.019 INFO:tasks.workunit.client.0.smithi139.stdout:9/336: read f14 [1729116,63465] 0 2024-09-16T00:09:43.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.016+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f40140b2c80 con 0x7f4014156e00 2024-09-16T00:09:43.020 INFO:tasks.workunit.client.0.smithi139.stdout:7/311: mkdir d5/d26/d46/d6b 0 2024-09-16T00:09:43.021 INFO:tasks.workunit.client.0.smithi139.stdout:3/344: creat d5/d7/d23/d27/d2f/f7a x:0 0 0 2024-09-16T00:09:43.021 INFO:tasks.workunit.client.0.smithi139.stdout:6/262: rmdir d8/db/d26 39 2024-09-16T00:09:43.021 INFO:tasks.workunit.client.0.smithi139.stdout:5/358: getdents d1/d25 0 2024-09-16T00:09:43.021 INFO:tasks.workunit.client.0.smithi139.stdout:6/263: fdatasync d8/d20/f2b 0 2024-09-16T00:09:43.021 INFO:tasks.workunit.client.0.smithi139.stdout:9/337: symlink d19/d22/d32/l6b 0 2024-09-16T00:09:43.022 INFO:tasks.workunit.client.0.smithi139.stdout:2/317: rename f9 to d11/d30/d57/d50/f69 0 2024-09-16T00:09:43.024 INFO:tasks.workunit.client.0.smithi139.stdout:4/259: link f1 d12/d47/d57/f59 0 2024-09-16T00:09:43.024 INFO:tasks.workunit.client.0.smithi139.stdout:3/345: mknod d5/d7/d23/d27/c7b 0 2024-09-16T00:09:43.024 INFO:tasks.workunit.client.0.smithi139.stdout:0/319: sync 2024-09-16T00:09:43.025 INFO:tasks.workunit.client.0.smithi139.stdout:1/345: creat d7/d8/d40/f66 x:0 0 0 2024-09-16T00:09:43.027 INFO:tasks.workunit.client.0.smithi139.stdout:7/312: link d5/d16/f34 d5/d16/d22/d25/d2d/d50/d44/d55/f6c 0 2024-09-16T00:09:43.028 INFO:tasks.workunit.client.0.smithi139.stdout:3/346: read d5/d7/d23/d35/f3b [782388,8443] 0 2024-09-16T00:09:43.028 INFO:tasks.workunit.client.0.smithi139.stdout:2/318: creat d11/d30/d57/d50/f6a x:0 0 0 2024-09-16T00:09:43.029 INFO:tasks.workunit.client.0.smithi139.stdout:4/260: unlink d12/f44 0 2024-09-16T00:09:43.030 INFO:tasks.workunit.client.0.smithi139.stdout:5/359: dwrite d1/d21/f39 [0,4194304] 0 2024-09-16T00:09:43.031 INFO:tasks.workunit.client.0.smithi139.stdout:6/264: symlink d8/db/d26/d45/l5f 0 2024-09-16T00:09:43.032 INFO:tasks.workunit.client.0.smithi139.stdout:9/338: dwrite d19/d1a/d20/d23/d2d/d35/d3d/f63 [0,4194304] 0 2024-09-16T00:09:43.033 INFO:tasks.workunit.client.0.smithi139.stdout:5/360: stat d1/d13/d3b/d3e 0 2024-09-16T00:09:43.034 INFO:tasks.workunit.client.0.smithi139.stdout:2/319: fsync d11/d30/d57/d29/d2b/f2c 0 2024-09-16T00:09:43.035 INFO:tasks.workunit.client.0.smithi139.stdout:0/320: creat d6/d1c/d41/d44/d58/f60 x:0 0 0 2024-09-16T00:09:43.036 INFO:tasks.workunit.client.0.smithi139.stdout:3/347: stat d5/d7/d23/d27/d2f/d30/d4c/d50/l5e 0 2024-09-16T00:09:43.036 INFO:tasks.workunit.client.0.smithi139.stdout:1/346: mkdir d7/d8/d40/d48/d67 0 2024-09-16T00:09:43.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.038+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f40140c8580 con 0x7f4014156e00 2024-09-16T00:09:43.039 INFO:tasks.workunit.client.0.smithi139.stdout:7/313: unlink d5/d16/l19 0 2024-09-16T00:09:43.040 INFO:tasks.workunit.client.0.smithi139.stdout:4/261: creat d12/d2b/f5a x:0 0 0 2024-09-16T00:09:43.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.041+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f40140c8580 con 0x7f4014156e00 2024-09-16T00:09:43.042 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:43.042 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:43.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.041+0000 7f401a54b640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f4014062f00 0x7f4014152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:43.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.041+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f4014063320 con 0x7f4014062f00 2024-09-16T00:09:43.043 INFO:tasks.workunit.client.0.smithi139.stdout:0/321: creat d6/d1c/f61 x:0 0 0 2024-09-16T00:09:43.043 INFO:tasks.workunit.client.0.smithi139.stdout:3/348: fdatasync d5/d7/f49 0 2024-09-16T00:09:43.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.042+0000 7f4018ae8640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f4014062f00 0x7f4014152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:43.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.042+0000 7f4018ae8640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f4014062f00 0x7f4014152130 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:43.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.043+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4014063320 con 0x7f4014062f00 2024-09-16T00:09:43.051 INFO:tasks.workunit.client.0.smithi139.stdout:6/265: dwrite d8/d29/f32 [0,4194304] 0 2024-09-16T00:09:43.051 INFO:tasks.workunit.client.0.smithi139.stdout:3/349: read - d5/d7/d23/d27/d2f/d30/d4c/f69 zero size 2024-09-16T00:09:43.054 INFO:tasks.workunit.client.0.smithi139.stdout:9/339: write d19/d1a/d20/d23/d2d/d35/d3d/f5a [608893,16083] 0 2024-09-16T00:09:43.055 INFO:tasks.workunit.client.0.smithi139.stdout:2/320: dwrite d11/d30/f4c [0,4194304] 0 2024-09-16T00:09:43.059 INFO:tasks.workunit.client.0.smithi139.stdout:5/361: mkdir d1/d25/d30/d33/d5b/d78 0 2024-09-16T00:09:43.060 INFO:tasks.workunit.client.0.smithi139.stdout:4/262: dread d12/d2b/f26 [0,4194304] 0 2024-09-16T00:09:43.062 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:43.061+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:43.063 INFO:tasks.workunit.client.0.smithi139.stdout:1/347: dread d7/f10 [0,4194304] 0 2024-09-16T00:09:43.063 INFO:tasks.workunit.client.0.smithi139.stdout:5/362: dread - d1/d13/d22/f6b zero size 2024-09-16T00:09:43.063 INFO:tasks.workunit.client.0.smithi139.stdout:2/321: unlink d11/d30/d57/d5d/c63 0 2024-09-16T00:09:43.063 INFO:tasks.workunit.client.0.smithi139.stdout:6/266: fdatasync d8/d20/d27/d10/f13 0 2024-09-16T00:09:43.065 INFO:tasks.workunit.client.0.smithi139.stdout:4/263: mknod d12/d4a/d4d/c5b 0 2024-09-16T00:09:43.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.067+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4014152670 con 0x7f4014062f00 2024-09-16T00:09:43.069 INFO:tasks.workunit.client.0.smithi139.stdout:1/348: readlink d7/d8/d27/d2f/l4f 0 2024-09-16T00:09:43.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.069+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4014152670 con 0x7f4014062f00 2024-09-16T00:09:43.070 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:43.072 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:43.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.069+0000 7f401a54b640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f4014152b80 0x7f401402c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:43.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.069+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f4014152fa0 con 0x7f4014152b80 2024-09-16T00:09:43.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.069+0000 7f4013fff640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f4014152b80 0x7f401402c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:43.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.070+0000 7f4013fff640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f4014152b80 0x7f401402c640 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:43.073 INFO:tasks.workunit.client.0.smithi139.stdout:9/340: creat d19/f6c x:0 0 0 2024-09-16T00:09:43.073 INFO:tasks.workunit.client.0.smithi139.stdout:2/322: dwrite d11/d30/f42 [0,4194304] 0 2024-09-16T00:09:43.073 INFO:tasks.workunit.client.0.smithi139.stdout:7/314: truncate d5/d16/d22/d25/d2d/d50/d44/f57 1788232 0 2024-09-16T00:09:43.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.071+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4014152fa0 con 0x7f4014152b80 2024-09-16T00:09:43.075 INFO:tasks.workunit.client.0.smithi139.stdout:0/322: getdents d6/d1c/d41/d44 0 2024-09-16T00:09:43.075 INFO:tasks.workunit.client.0.smithi139.stdout:9/341: stat d19/d22/d32/d4e/l4f 0 2024-09-16T00:09:43.075 INFO:tasks.workunit.client.0.smithi139.stdout:4/264: creat d12/d47/f5c x:0 0 0 2024-09-16T00:09:43.075 INFO:tasks.workunit.client.0.smithi139.stdout:5/363: getdents d1/d25/d73 0 2024-09-16T00:09:43.077 INFO:tasks.workunit.client.0.smithi139.stdout:9/342: creat d19/d1a/d20/d23/d2d/d35/d42/d3a/f6d x:0 0 0 2024-09-16T00:09:43.078 INFO:tasks.workunit.client.0.smithi139.stdout:5/364: unlink d1/d13/d22/f74 0 2024-09-16T00:09:43.079 INFO:tasks.workunit.client.0.smithi139.stdout:0/323: symlink d6/d5a/l62 0 2024-09-16T00:09:43.080 INFO:tasks.workunit.client.0.smithi139.stdout:7/315: mkdir d5/d16/d6d 0 2024-09-16T00:09:43.082 INFO:tasks.workunit.client.0.smithi139.stdout:4/265: mknod d12/c5d 0 2024-09-16T00:09:43.082 INFO:tasks.workunit.client.0.smithi139.stdout:6/267: link d8/db/d26/f4b d8/d20/d4d/f60 0 2024-09-16T00:09:43.082 INFO:tasks.workunit.client.0.smithi139.stdout:6/268: chown f3 5769 1 2024-09-16T00:09:43.083 INFO:tasks.workunit.client.0.smithi139.stdout:1/349: rmdir d7/d8/d27/d2f/d63 0 2024-09-16T00:09:43.083 INFO:tasks.workunit.client.0.smithi139.stdout:9/343: creat d19/d1a/d20/d23/d2d/d35/d42/d3a/f6e x:0 0 0 2024-09-16T00:09:43.084 INFO:tasks.workunit.client.0.smithi139.stdout:1/350: truncate d7/d8/d27/f57 733238 0 2024-09-16T00:09:43.085 INFO:tasks.workunit.client.0.smithi139.stdout:5/365: rename d1/d25/d30/d33/d43/d5a to d1/d25/d73/d79 0 2024-09-16T00:09:43.085 INFO:tasks.workunit.client.0.smithi139.stdout:0/324: mkdir d6/d1c/d41/d63 0 2024-09-16T00:09:43.086 INFO:tasks.workunit.client.0.smithi139.stdout:2/323: link d11/d45/l4b d11/d30/d57/d29/d2b/l6b 0 2024-09-16T00:09:43.087 INFO:tasks.workunit.client.0.smithi139.stdout:7/316: creat d5/d16/d6d/f6e x:0 0 0 2024-09-16T00:09:43.089 INFO:tasks.workunit.client.0.smithi139.stdout:2/324: readlink d11/d30/d57/l60 0 2024-09-16T00:09:43.089 INFO:tasks.workunit.client.0.smithi139.stdout:4/266: dwrite d12/d2b/f34 [0,4194304] 0 2024-09-16T00:09:43.089 INFO:tasks.workunit.client.0.smithi139.stdout:0/325: creat d6/f64 x:0 0 0 2024-09-16T00:09:43.090 INFO:tasks.workunit.client.0.smithi139.stdout:9/344: truncate d19/d1a/d20/d23/f4d 785267 0 2024-09-16T00:09:43.091 INFO:tasks.workunit.client.0.smithi139.stdout:1/351: mkdir d7/d8/d27/d68 0 2024-09-16T00:09:43.092 INFO:tasks.workunit.client.0.smithi139.stdout:1/352: chown d7/d8/f11 1989239 1 2024-09-16T00:09:43.092 INFO:tasks.workunit.client.0.smithi139.stdout:0/326: readlink d6/l1f 0 2024-09-16T00:09:43.093 INFO:tasks.workunit.client.0.smithi139.stdout:7/317: rename d5/f9 to d5/d16/d22/f6f 0 2024-09-16T00:09:43.094 INFO:tasks.workunit.client.0.smithi139.stdout:7/318: write d5/d26/d42/f5b [253614,62114] 0 2024-09-16T00:09:43.095 INFO:tasks.workunit.client.0.smithi139.stdout:7/319: fdatasync d5/d16/d22/f4e 0 2024-09-16T00:09:43.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.094+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4014152fa0 con 0x7f4014152b80 2024-09-16T00:09:43.096 INFO:tasks.workunit.client.0.smithi139.stdout:4/267: truncate f7 3643826 0 2024-09-16T00:09:43.096 INFO:tasks.workunit.client.0.smithi139.stdout:4/268: chown d12/d47/d57 134787 1 2024-09-16T00:09:43.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.097+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4014152fa0 con 0x7f4014152b80 2024-09-16T00:09:43.098 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:43.099 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:43.099 INFO:tasks.workunit.client.0.smithi139.stdout:5/366: truncate d1/d21/d49/d53/f51 326495 0 2024-09-16T00:09:43.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.098+0000 7f401a54b640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f401402d090 0x7f401402d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:43.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.098+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f401402da30 con 0x7f401402d090 2024-09-16T00:09:43.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.098+0000 7f40137fe640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f401402d090 0x7f401402d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:43.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.098+0000 7f40137fe640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f401402d090 0x7f401402d4f0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:43.100 INFO:tasks.workunit.client.0.smithi139.stdout:9/345: creat d19/d1a/d20/d23/d2d/d41/f6f x:0 0 0 2024-09-16T00:09:43.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.099+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f401402da30 con 0x7f401402d090 2024-09-16T00:09:43.101 INFO:tasks.workunit.client.0.smithi139.stdout:0/327: creat d6/d5a/f65 x:0 0 0 2024-09-16T00:09:43.102 INFO:tasks.workunit.client.0.smithi139.stdout:1/353: mkdir d7/d8/d40/d48/d69 0 2024-09-16T00:09:43.105 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:43.105 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:43.106 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:43.107 INFO:tasks.workunit.client.0.smithi139.stdout:9/346: dwrite f2 [0,4194304] 0 2024-09-16T00:09:43.112 INFO:tasks.workunit.client.0.smithi139.stdout:0/328: creat d6/d1c/d41/d63/f66 x:0 0 0 2024-09-16T00:09:43.113 INFO:tasks.workunit.client.0.smithi139.stdout:2/325: link d11/d30/f56 d11/d30/f6c 0 2024-09-16T00:09:43.114 INFO:tasks.workunit.client.0.smithi139.stdout:2/326: chown d11/d30/d57/d50 1 1 2024-09-16T00:09:43.115 INFO:tasks.workunit.client.0.smithi139.stdout:2/327: chown d11/d30/d57/d29/d2b/f34 554 1 2024-09-16T00:09:43.115 INFO:tasks.workunit.client.0.smithi139.stdout:9/347: mknod d19/d1a/d20/d23/d2d/d35/d60/c70 0 2024-09-16T00:09:43.116 INFO:tasks.workunit.client.0.smithi139.stdout:5/367: dread d1/d21/d49/d53/f51 [0,4194304] 0 2024-09-16T00:09:43.116 INFO:tasks.workunit.client.0.smithi139.stdout:9/348: read d19/f1d [2349989,100993] 0 2024-09-16T00:09:43.117 INFO:tasks.workunit.client.0.smithi139.stdout:9/349: fdatasync fb 0 2024-09-16T00:09:43.118 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:09:43.119 INFO:tasks.workunit.client.0.smithi139.stdout:1/354: link d7/d8/f21 d7/d8/d40/d47/f6a 0 2024-09-16T00:09:43.119 INFO:tasks.workunit.client.0.smithi139.stdout:1/355: chown d7/d8/d27/d2f/f5c 42223872 1 2024-09-16T00:09:43.120 INFO:tasks.workunit.client.0.smithi139.stdout:1/356: chown d7/d8/d27/d68 4760353 1 2024-09-16T00:09:43.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.119+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f401402da30 con 0x7f401402d090 2024-09-16T00:09:43.120 INFO:tasks.workunit.client.0.smithi139.stdout:0/329: rename d6/d1c/f4d to d6/d1c/d41/d44/dd/f67 0 2024-09-16T00:09:43.121 INFO:tasks.workunit.client.0.smithi139.stdout:7/320: write d5/d16/f34 [3789580,55408] 0 2024-09-16T00:09:43.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.121+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f401402da30 con 0x7f401402d090 2024-09-16T00:09:43.122 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:43.123 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:43.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.121+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=88}) -- 0x7f4014209a50 con 0x7f40140d0870 2024-09-16T00:09:43.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.121+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4014209c20 con 0x7f40140d0870 2024-09-16T00:09:43.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.121+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f400c049040 con 0x7f40140d0870 2024-09-16T00:09:43.123 INFO:teuthology.orchestra.run.smithi139.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T00:09:43.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.121+0000 7f401a54b640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f4014207a90 0x7f4014207ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:43.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.122+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f4014209780 con 0x7f4014207a90 2024-09-16T00:09:43.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.122+0000 7f4018ae8640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f4014207a90 0x7f4014207ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:43.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.122+0000 7f4018ae8640 1 --2- 172.21.15.139:0/827921768 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f4014207a90 0x7f4014207ed0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:43.124 INFO:tasks.workunit.client.0.smithi139.stdout:5/368: mkdir d1/d25/d30/d33/d5b/d7a 0 2024-09-16T00:09:43.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.123+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4014209780 con 0x7f4014207a90 2024-09-16T00:09:43.125 INFO:tasks.workunit.client.0.smithi139.stdout:0/330: unlink d6/d42/f48 0 2024-09-16T00:09:43.126 INFO:tasks.workunit.client.0.smithi139.stdout:9/350: dwrite d19/d1a/f50 [0,4194304] 0 2024-09-16T00:09:43.126 INFO:tasks.workunit.client.0.smithi139.stdout:5/369: truncate d1/d25/f6c 519923 0 2024-09-16T00:09:43.127 INFO:tasks.workunit.client.0.smithi139.stdout:2/328: link d11/f16 d11/d32/f6d 0 2024-09-16T00:09:43.130 INFO:tasks.workunit.client.0.smithi139.stdout:9/351: fdatasync d19/d1a/d20/d23/f3e 0 2024-09-16T00:09:43.130 INFO:tasks.workunit.client.0.smithi139.stdout:5/370: write d1/d25/d30/d33/f50 [113943,127480] 0 2024-09-16T00:09:43.131 INFO:tasks.workunit.client.0.smithi139.stdout:4/269: dwrite d12/d2b/f30 [0,4194304] 0 2024-09-16T00:09:43.133 INFO:tasks.workunit.client.0.smithi139.stdout:4/270: read d12/d47/f2c [3184450,30639] 0 2024-09-16T00:09:43.134 INFO:tasks.workunit.client.0.smithi139.stdout:0/331: creat d6/d1c/d41/d44/d58/f68 x:0 0 0 2024-09-16T00:09:43.134 INFO:tasks.workunit.client.0.smithi139.stdout:9/352: dread - d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/f55 zero size 2024-09-16T00:09:43.135 INFO:tasks.workunit.client.0.smithi139.stdout:1/357: getdents d7/d8/d40/d48/d53/d61 0 2024-09-16T00:09:43.135 INFO:tasks.workunit.client.0.smithi139.stdout:0/332: readlink d6/l1f 0 2024-09-16T00:09:43.135 INFO:tasks.workunit.client.0.smithi139.stdout:9/353: chown d19/d1a 3 1 2024-09-16T00:09:43.136 INFO:tasks.workunit.client.0.smithi139.stdout:2/329: mkdir d11/d2f/d6e 0 2024-09-16T00:09:43.136 INFO:tasks.workunit.client.0.smithi139.stdout:0/333: fsync d6/d1c/f4c 0 2024-09-16T00:09:43.137 INFO:tasks.workunit.client.0.smithi139.stdout:9/354: dread - d19/d22/f57 zero size 2024-09-16T00:09:43.138 INFO:tasks.workunit.client.0.smithi139.stdout:2/330: chown d11/f15 1782 1 2024-09-16T00:09:43.138 INFO:tasks.workunit.client.0.smithi139.stdout:7/321: dread - d5/d16/d22/d25/d2d/f3a zero size 2024-09-16T00:09:43.138 INFO:tasks.workunit.client.0.smithi139.stdout:0/334: write d6/d5a/f65 [212791,15121] 0 2024-09-16T00:09:43.140 INFO:tasks.workunit.client.0.smithi139.stdout:9/355: chown d19/d1a/d20/d59/f5d 43215 1 2024-09-16T00:09:43.141 INFO:tasks.workunit.client.0.smithi139.stdout:5/371: rename d1/d25/d30/d33/d5b to d1/d21/d49/d53/d6a/d7b 0 2024-09-16T00:09:43.142 INFO:tasks.workunit.client.0.smithi139.stdout:5/372: read d1/f4a [3516532,15537] 0 2024-09-16T00:09:43.142 INFO:tasks.workunit.client.0.smithi139.stdout:5/373: fdatasync d1/f28 0 2024-09-16T00:09:43.143 INFO:tasks.workunit.client.0.smithi139.stdout:5/374: fdatasync d1/d13/d3f/d5e/f61 0 2024-09-16T00:09:43.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.144+0000 7f401a54b640 1 -- 172.21.15.139:0/827921768 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f40142096d0 con 0x7f4014207a90 2024-09-16T00:09:43.145 INFO:tasks.workunit.client.0.smithi139.stdout:7/322: write d5/d16/d22/d25/d2d/f3b [2403503,28856] 0 2024-09-16T00:09:43.146 INFO:tasks.workunit.client.0.smithi139.stdout:0/335: symlink d6/d1c/l69 0 2024-09-16T00:09:43.147 INFO:tasks.workunit.client.0.smithi139.stdout:0/336: write d6/f52 [4609582,3234] 0 2024-09-16T00:09:43.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:43.146+0000 7f4010ff9640 1 -- 172.21.15.139:0/827921768 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f40142096d0 con 0x7f4014207a90 2024-09-16T00:09:43.147 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:43.148 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:43.148 INFO:tasks.workunit.client.0.smithi139.stdout:9/356: rename d19/d22 to d19/d1a/d20/d23/d2d/d41/d71 0 2024-09-16T00:09:43.151 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T00:09:43.152 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T00:09:43.153 INFO:tasks.workunit.client.0.smithi139.stdout:2/331: mknod d11/d30/c6f 0 2024-09-16T00:09:43.154 INFO:tasks.workunit.client.0.smithi139.stdout:5/375: truncate d1/d13/d3b/f4c 1042601 0 2024-09-16T00:09:43.155 INFO:tasks.workunit.client.0.smithi139.stdout:2/332: write d11/d30/d57/d29/d2b/f34 [983682,120443] 0 2024-09-16T00:09:43.155 INFO:tasks.workunit.client.0.smithi139.stdout:8/349: sync 2024-09-16T00:09:43.157 INFO:tasks.workunit.client.0.smithi139.stdout:5/376: write d1/d13/f17 [5181070,57648] 0 2024-09-16T00:09:43.157 INFO:tasks.workunit.client.0.smithi139.stdout:9/357: dread f14 [0,4194304] 0 2024-09-16T00:09:43.157 INFO:tasks.workunit.client.0.smithi139.stdout:0/337: unlink d6/d1c/d41/d44/dd/c1a 0 2024-09-16T00:09:43.162 INFO:tasks.workunit.client.0.smithi139.stdout:7/323: truncate d5/d16/d22/d25/f30 1171070 0 2024-09-16T00:09:43.163 INFO:tasks.workunit.client.0.smithi139.stdout:4/271: dwrite d12/d47/d49/f53 [0,4194304] 0 2024-09-16T00:09:43.163 INFO:tasks.workunit.client.0.smithi139.stdout:8/350: rename l11 to d17/d1a/d50/l77 0 2024-09-16T00:09:43.163 INFO:tasks.workunit.client.0.smithi139.stdout:4/272: stat d12/d2b/l36 0 2024-09-16T00:09:43.164 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:43.162+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:43.164 INFO:tasks.workunit.client.0.smithi139.stdout:8/351: chown d17/d1a/d57/f71 23315708 1 2024-09-16T00:09:43.167 INFO:tasks.workunit.client.0.smithi139.stdout:1/358: getdents d7/d8/d40/d48/d4a 0 2024-09-16T00:09:43.168 INFO:tasks.workunit.client.0.smithi139.stdout:0/338: creat d6/d1c/d41/f6a x:0 0 0 2024-09-16T00:09:43.169 INFO:tasks.workunit.client.0.smithi139.stdout:0/339: dread - d6/f64 zero size 2024-09-16T00:09:43.169 INFO:tasks.workunit.client.0.smithi139.stdout:4/273: dread d12/d2b/f30 [0,4194304] 0 2024-09-16T00:09:43.169 INFO:tasks.workunit.client.0.smithi139.stdout:1/359: read d7/d8/d40/d48/d4a/f4c [75114,70220] 0 2024-09-16T00:09:43.173 INFO:tasks.workunit.client.0.smithi139.stdout:8/352: mknod d17/d1f/d3a/c78 0 2024-09-16T00:09:43.176 INFO:tasks.workunit.client.0.smithi139.stdout:9/358: dwrite f14 [0,4194304] 0 2024-09-16T00:09:43.179 INFO:tasks.workunit.client.0.smithi139.stdout:5/377: write d1/f27 [771136,114394] 0 2024-09-16T00:09:43.179 INFO:tasks.workunit.client.0.smithi139.stdout:0/340: rmdir d6/d1c/d45 39 2024-09-16T00:09:43.183 INFO:tasks.workunit.client.0.smithi139.stdout:2/333: getdents d11/d30/d57 0 2024-09-16T00:09:43.186 INFO:tasks.workunit.client.0.smithi139.stdout:0/341: mkdir d6/d1c/d41/d44/d6b 0 2024-09-16T00:09:43.187 INFO:tasks.workunit.client.0.smithi139.stdout:5/378: creat d1/d25/d30/d33/f7c x:0 0 0 2024-09-16T00:09:43.187 INFO:tasks.workunit.client.0.smithi139.stdout:9/359: mknod d19/d1a/d20/d23/c72 0 2024-09-16T00:09:43.188 INFO:teuthology.orchestra.run.smithi164.stdout: marking collection for removal 2024-09-16T00:09:43.189 INFO:teuthology.orchestra.run.smithi164.stdout:setting '_remove' omap key 2024-09-16T00:09:43.189 INFO:teuthology.orchestra.run.smithi164.stdout:finish_remove_pgs 2.17_head removing 2.17 2024-09-16T00:09:43.190 INFO:tasks.workunit.client.0.smithi139.stdout:2/334: write d11/d30/d57/d29/f66 [879253,99743] 0 2024-09-16T00:09:43.190 INFO:tasks.workunit.client.0.smithi139.stdout:7/324: dwrite d5/d16/d22/f4e [0,4194304] 0 2024-09-16T00:09:43.190 INFO:tasks.workunit.client.0.smithi139.stdout:5/379: read d1/f4a [3402536,8831] 0 2024-09-16T00:09:43.190 INFO:teuthology.orchestra.run.smithi164.stdout:Remove successful 2024-09-16T00:09:43.193 INFO:tasks.workunit.client.0.smithi139.stdout:0/342: getdents d6/d1c/d41 0 2024-09-16T00:09:43.194 INFO:tasks.workunit.client.0.smithi139.stdout:8/353: dread fb [0,4194304] 0 2024-09-16T00:09:43.197 INFO:tasks.workunit.client.0.smithi139.stdout:7/325: mkdir d5/d37/d70 0 2024-09-16T00:09:43.197 INFO:tasks.workunit.client.0.smithi139.stdout:5/380: creat d1/d13/d3b/f7d x:0 0 0 2024-09-16T00:09:43.198 INFO:tasks.workunit.client.0.smithi139.stdout:1/360: dwrite d7/d8/f1e [0,4194304] 0 2024-09-16T00:09:43.198 INFO:tasks.workunit.client.0.smithi139.stdout:2/335: mknod d11/d30/c70 0 2024-09-16T00:09:43.201 INFO:tasks.workunit.client.0.smithi139.stdout:7/326: chown d5/d16/d22 7 1 2024-09-16T00:09:43.201 INFO:tasks.workunit.client.0.smithi139.stdout:8/354: creat d17/d1a/d57/f79 x:0 0 0 2024-09-16T00:09:43.202 INFO:tasks.workunit.client.0.smithi139.stdout:4/274: dwrite fd [0,4194304] 0 2024-09-16T00:09:43.204 INFO:tasks.workunit.client.0.smithi139.stdout:9/360: truncate f14 885159 0 2024-09-16T00:09:43.205 INFO:tasks.workunit.client.0.smithi139.stdout:4/275: readlink d12/d47/l56 0 2024-09-16T00:09:43.208 INFO:tasks.workunit.client.0.smithi139.stdout:8/355: mkdir d17/d1a/d7a 0 2024-09-16T00:09:43.212 INFO:tasks.workunit.client.0.smithi139.stdout:0/343: creat d6/d1c/d41/d44/d6b/f6c x:0 0 0 2024-09-16T00:09:43.214 INFO:tasks.workunit.client.0.smithi139.stdout:4/276: getdents d12/d47/d49 0 2024-09-16T00:09:43.215 INFO:tasks.workunit.client.0.smithi139.stdout:2/336: truncate d11/d30/d57/d29/d2b/f31 1809632 0 2024-09-16T00:09:43.217 INFO:tasks.workunit.client.0.smithi139.stdout:9/361: truncate d19/d1a/d20/d23/d2d/d35/f66 376297 0 2024-09-16T00:09:43.218 INFO:tasks.workunit.client.0.smithi139.stdout:0/344: fdatasync d6/d1c/d41/d44/dd/f11 0 2024-09-16T00:09:43.218 INFO:tasks.workunit.client.0.smithi139.stdout:5/381: dwrite d1/f16 [8388608,4194304] 0 2024-09-16T00:09:43.218 INFO:tasks.workunit.client.0.smithi139.stdout:7/327: dread f4 [0,4194304] 0 2024-09-16T00:09:43.219 INFO:tasks.workunit.client.0.smithi139.stdout:1/361: dwrite d7/d8/f11 [0,4194304] 0 2024-09-16T00:09:43.220 INFO:tasks.workunit.client.0.smithi139.stdout:5/382: chown d1/d21/d49/d53/d6a 185974796 1 2024-09-16T00:09:43.222 INFO:tasks.workunit.client.0.smithi139.stdout:4/277: getdents d12/d47/d57 0 2024-09-16T00:09:43.222 INFO:tasks.workunit.client.0.smithi139.stdout:7/328: mkdir d5/d16/d22/d71 0 2024-09-16T00:09:43.223 INFO:tasks.workunit.client.0.smithi139.stdout:9/362: readlink d19/d1a/d20/d23/d2d/d35/d42/d3a/l3b 0 2024-09-16T00:09:43.223 INFO:tasks.workunit.client.0.smithi139.stdout:0/345: fdatasync d6/d1c/d45/d49/f4f 0 2024-09-16T00:09:43.225 INFO:tasks.workunit.client.0.smithi139.stdout:7/329: chown d5/d16/d22/d71 1 1 2024-09-16T00:09:43.226 INFO:tasks.workunit.client.0.smithi139.stdout:4/278: mknod d12/d2b/c5e 0 2024-09-16T00:09:43.226 INFO:tasks.workunit.client.0.smithi139.stdout:9/363: creat d19/d1a/f73 x:0 0 0 2024-09-16T00:09:43.227 INFO:tasks.workunit.client.0.smithi139.stdout:2/337: dwrite d11/d30/d57/d29/f66 [0,4194304] 0 2024-09-16T00:09:43.227 INFO:tasks.workunit.client.0.smithi139.stdout:1/362: symlink d7/d8/d40/d48/d67/l6b 0 2024-09-16T00:09:43.228 INFO:tasks.workunit.client.0.smithi139.stdout:4/279: write f4 [391045,114401] 0 2024-09-16T00:09:43.228 INFO:tasks.workunit.client.0.smithi139.stdout:5/383: creat d1/d21/d49/d53/d6a/d7b/f7e x:0 0 0 2024-09-16T00:09:43.229 INFO:tasks.workunit.client.0.smithi139.stdout:7/330: creat d5/d16/d6d/f72 x:0 0 0 2024-09-16T00:09:43.231 INFO:tasks.workunit.client.0.smithi139.stdout:0/346: rename d6/d1c/d41/d44/dd/l14 to d6/d5a/l6d 0 2024-09-16T00:09:43.231 INFO:tasks.workunit.client.0.smithi139.stdout:1/363: dread d7/f10 [4194304,4194304] 0 2024-09-16T00:09:43.244 INFO:tasks.workunit.client.0.smithi139.stdout:9/364: rename d19/d1a/d20/d23/c2a to d19/d1a/d20/d23/c74 0 2024-09-16T00:09:43.244 INFO:tasks.workunit.client.0.smithi139.stdout:4/280: dwrite d12/f40 [0,4194304] 0 2024-09-16T00:09:43.245 INFO:tasks.workunit.client.0.smithi139.stdout:1/364: mkdir d7/d8/d27/d68/d6c 0 2024-09-16T00:09:43.250 INFO:tasks.workunit.client.0.smithi139.stdout:5/384: dwrite d1/d13/d3b/d3e/f6e [0,4194304] 0 2024-09-16T00:09:43.251 INFO:tasks.workunit.client.0.smithi139.stdout:2/338: dwrite d11/d30/d57/d29/d2b/f65 [0,4194304] 0 2024-09-16T00:09:43.252 INFO:tasks.workunit.client.0.smithi139.stdout:7/331: dwrite d5/d16/d22/f4e [4194304,4194304] 0 2024-09-16T00:09:43.253 INFO:tasks.workunit.client.0.smithi139.stdout:4/281: unlink d12/d2b/l36 0 2024-09-16T00:09:43.254 INFO:tasks.workunit.client.0.smithi139.stdout:0/347: link d6/d1c/d41/d44/d58/f5f d6/d1c/d41/d44/dd/f6e 0 2024-09-16T00:09:43.256 INFO:tasks.workunit.client.0.smithi139.stdout:9/365: write d19/d1a/f1e [1794989,72602] 0 2024-09-16T00:09:43.256 INFO:tasks.workunit.client.0.smithi139.stdout:1/365: symlink d7/d8/d27/d2f/l6d 0 2024-09-16T00:09:43.257 INFO:tasks.workunit.client.0.smithi139.stdout:5/385: rmdir d1/d13/d3f 39 2024-09-16T00:09:43.257 INFO:tasks.workunit.client.0.smithi139.stdout:7/332: symlink d5/d16/d6d/l73 0 2024-09-16T00:09:43.258 INFO:tasks.workunit.client.0.smithi139.stdout:0/348: dread - d6/d1c/d41/d44/dd/f46 zero size 2024-09-16T00:09:43.258 INFO:tasks.workunit.client.0.smithi139.stdout:4/282: unlink d12/d2b/c3b 0 2024-09-16T00:09:43.261 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:43.261 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:161 2.b e64/19 2:d5c7a900:::200.00000003:head v 64'5, mlcod=61'3)", 2024-09-16T00:09:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:07.541046+0000", 2024-09-16T00:09:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 35.71763198, 2024-09-16T00:09:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010081302, 2024-09-16T00:09:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.541046+0000", 2024-09-16T00:09:43.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.541046+0000", 2024-09-16T00:09:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.541048+0000", 2024-09-16T00:09:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5859999999999999e-06 2024-09-16T00:09:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543134+0000", 2024-09-16T00:09:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020863589999999999 2024-09-16T00:09:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543155+0000", 2024-09-16T00:09:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0344999999999999e-05 2024-09-16T00:09:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543167+0000", 2024-09-16T00:09:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2071e-05 2024-09-16T00:09:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543242+0000", 2024-09-16T00:09:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5108e-05 2024-09-16T00:09:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.543268+0000", 2024-09-16T00:09:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6471000000000001e-05 2024-09-16T00:09:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551087+0000", 2024-09-16T00:09:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0078188670000000002 2024-09-16T00:09:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551128+0000", 2024-09-16T00:09:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0494999999999998e-05 2024-09-16T00:09:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.267 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10654 3.34 3:2c63a741:::100000030c9.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.652619+0000", 2024-09-16T00:09:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.6060592260000002, 2024-09-16T00:09:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012238643, 2024-09-16T00:09:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.268 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10654 2024-09-16T00:09:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.652619+0000", 2024-09-16T00:09:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.652619+0000", 2024-09-16T00:09:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.652620+0000", 2024-09-16T00:09:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.277e-06 2024-09-16T00:09:43.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.659597+0000", 2024-09-16T00:09:43.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069766510000000004 2024-09-16T00:09:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.659611+0000", 2024-09-16T00:09:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3722e-05 2024-09-16T00:09:43.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.659620+0000", 2024-09-16T00:09:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7669999999999992e-06 2024-09-16T00:09:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.659672+0000", 2024-09-16T00:09:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2590000000000003e-05 2024-09-16T00:09:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.659776+0000", 2024-09-16T00:09:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103643 2024-09-16T00:09:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.659861+0000", 2024-09-16T00:09:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5513999999999995e-05 2024-09-16T00:09:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.663941+0000", 2024-09-16T00:09:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004079938 2024-09-16T00:09:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.664758+0000", 2024-09-16T00:09:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00081642800000000001 2024-09-16T00:09:43.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.664813+0000", 2024-09-16T00:09:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4824999999999998e-05 2024-09-16T00:09:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.664858+0000", 2024-09-16T00:09:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5287999999999997e-05 2024-09-16T00:09:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.277 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10696 3.34 3:2c63a741:::100000030c9.00000000:head [write 0~4194304 [3@173033]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.951656+0000", 2024-09-16T00:09:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.3070222310000004, 2024-09-16T00:09:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011673619, 2024-09-16T00:09:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10696 2024-09-16T00:09:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.278 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.951656+0000", 2024-09-16T00:09:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.951656+0000", 2024-09-16T00:09:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.951658+0000", 2024-09-16T00:09:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.745e-06 2024-09-16T00:09:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.957371+0000", 2024-09-16T00:09:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00571347 2024-09-16T00:09:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.957391+0000", 2024-09-16T00:09:43.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9222999999999999e-05 2024-09-16T00:09:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.957401+0000", 2024-09-16T00:09:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0526999999999999e-05 2024-09-16T00:09:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.957477+0000", 2024-09-16T00:09:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6167000000000004e-05 2024-09-16T00:09:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.957565+0000", 2024-09-16T00:09:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7472999999999994e-05 2024-09-16T00:09:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 6", 2024-09-16T00:09:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.957702+0000", 2024-09-16T00:09:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000137315 2024-09-16T00:09:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.962248+0000", 2024-09-16T00:09:43.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045460869999999999 2024-09-16T00:09:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.963248+0000", 2024-09-16T00:09:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00099954699999999994 2024-09-16T00:09:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.963289+0000", 2024-09-16T00:09:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1743000000000003e-05 2024-09-16T00:09:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.963330+0000", 2024-09-16T00:09:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0321999999999998e-05 2024-09-16T00:09:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10756 3.a 3:500c83b2:::100000030b4.00000000:head [write 0~4194304 [3@2137521]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.339441+0000", 2024-09-16T00:09:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.9192371750000001, 2024-09-16T00:09:43.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01041241, 2024-09-16T00:09:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10756 2024-09-16T00:09:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.339441+0000", 2024-09-16T00:09:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.339441+0000", 2024-09-16T00:09:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.339442+0000", 2024-09-16T00:09:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.204e-06 2024-09-16T00:09:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.344325+0000", 2024-09-16T00:09:43.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048825099999999996 2024-09-16T00:09:43.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.344347+0000", 2024-09-16T00:09:43.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2007000000000001e-05 2024-09-16T00:09:43.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.344358+0000", 2024-09-16T00:09:43.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0964999999999999e-05 2024-09-16T00:09:43.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.344436+0000", 2024-09-16T00:09:43.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8461999999999995e-05 2024-09-16T00:09:43.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.344549+0000", 2024-09-16T00:09:43.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000112241 2024-09-16T00:09:43.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.349762+0000", 2024-09-16T00:09:43.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052130520000000001 2024-09-16T00:09:43.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.349807+0000", 2024-09-16T00:09:43.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4953999999999997e-05 2024-09-16T00:09:43.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.349854+0000", 2024-09-16T00:09:43.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7015000000000003e-05 2024-09-16T00:09:43.289 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.289 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.289 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.290 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10780 3.a 3:500c83b2:::100000030b4.00000000:head [write 0~4194304 [3@2137521]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:43.290 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:09:43.290 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.8057634289999998, 2024-09-16T00:09:43.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015526478, 2024-09-16T00:09:43.290 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.290 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.290 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.290 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10780 2024-09-16T00:09:43.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.290 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:09:43.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:09:43.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452917+0000", 2024-09-16T00:09:43.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2180000000000002e-06 2024-09-16T00:09:43.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462177+0000", 2024-09-16T00:09:43.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092593769999999992 2024-09-16T00:09:43.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462196+0000", 2024-09-16T00:09:43.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9745000000000002e-05 2024-09-16T00:09:43.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462207+0000", 2024-09-16T00:09:43.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0524e-05 2024-09-16T00:09:43.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462284+0000", 2024-09-16T00:09:43.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7608000000000006e-05 2024-09-16T00:09:43.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462376+0000", 2024-09-16T00:09:43.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1811999999999994e-05 2024-09-16T00:09:43.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468357+0000", 2024-09-16T00:09:43.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059806809999999998 2024-09-16T00:09:43.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468386+0000", 2024-09-16T00:09:43.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9068999999999999e-05 2024-09-16T00:09:43.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468441+0000", 2024-09-16T00:09:43.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5444e-05 2024-09-16T00:09:43.294 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.295 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.295 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10809 3.34 3:2c63a741:::100000030c9.00000000:head [write 0~4194304 [4@1879734]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:43.295 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.689472+0000", 2024-09-16T00:09:43.295 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.5692062260000004, 2024-09-16T00:09:43.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099381250000000008, 2024-09-16T00:09:43.295 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.295 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.295 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.295 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.296 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.296 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10809 2024-09-16T00:09:43.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.296 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.689472+0000", 2024-09-16T00:09:43.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.689472+0000", 2024-09-16T00:09:43.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.689475+0000", 2024-09-16T00:09:43.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.858e-06 2024-09-16T00:09:43.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.694467+0000", 2024-09-16T00:09:43.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049919750000000001 2024-09-16T00:09:43.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.694487+0000", 2024-09-16T00:09:43.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9692000000000001e-05 2024-09-16T00:09:43.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.694497+0000", 2024-09-16T00:09:43.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.026e-05 2024-09-16T00:09:43.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.694554+0000", 2024-09-16T00:09:43.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6582999999999997e-05 2024-09-16T00:09:43.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.694640+0000", 2024-09-16T00:09:43.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6340000000000003e-05 2024-09-16T00:09:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.699258+0000", 2024-09-16T00:09:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046182319999999999 2024-09-16T00:09:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.699326+0000", 2024-09-16T00:09:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8087e-05 2024-09-16T00:09:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.699410+0000", 2024-09-16T00:09:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4097999999999993e-05 2024-09-16T00:09:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10831 3.1a 3:59b8f98c:::100000030df.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:09:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.482956293, 2024-09-16T00:09:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014692843000000001, 2024-09-16T00:09:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10831 2024-09-16T00:09:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:09:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:09:43.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775723+0000", 2024-09-16T00:09:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.367e-06 2024-09-16T00:09:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780402+0000", 2024-09-16T00:09:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046785769999999997 2024-09-16T00:09:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780421+0000", 2024-09-16T00:09:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9179000000000001e-05 2024-09-16T00:09:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780440+0000", 2024-09-16T00:09:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9015e-05 2024-09-16T00:09:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780533+0000", 2024-09-16T00:09:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3197999999999998e-05 2024-09-16T00:09:43.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780618+0000", 2024-09-16T00:09:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4787e-05 2024-09-16T00:09:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780748+0000", 2024-09-16T00:09:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012927199999999999 2024-09-16T00:09:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.785049+0000", 2024-09-16T00:09:43.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043015750000000002 2024-09-16T00:09:43.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790312+0000", 2024-09-16T00:09:43.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052633319999999999 2024-09-16T00:09:43.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790369+0000", 2024-09-16T00:09:43.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6848999999999997e-05 2024-09-16T00:09:43.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790415+0000", 2024-09-16T00:09:43.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5692000000000002e-05 2024-09-16T00:09:43.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.307 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.307 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10915 3.1a 3:5a6e15a2:::1000000318e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:43.307 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:09:43.307 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0434503719999997, 2024-09-16T00:09:43.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014599472, 2024-09-16T00:09:43.307 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.307 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.307 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.308 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.308 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10915 2024-09-16T00:09:43.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.308 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:09:43.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:09:43.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215230+0000", 2024-09-16T00:09:43.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0090000000000002e-06 2024-09-16T00:09:43.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219556+0000", 2024-09-16T00:09:43.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043259539999999999 2024-09-16T00:09:43.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219577+0000", 2024-09-16T00:09:43.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1271999999999999e-05 2024-09-16T00:09:43.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219589+0000", 2024-09-16T00:09:43.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1766000000000001e-05 2024-09-16T00:09:43.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219663+0000", 2024-09-16T00:09:43.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4196999999999995e-05 2024-09-16T00:09:43.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219733+0000", 2024-09-16T00:09:43.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9553000000000003e-05 2024-09-16T00:09:43.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:43.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219827+0000", 2024-09-16T00:09:43.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4395999999999995e-05 2024-09-16T00:09:43.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.225724+0000", 2024-09-16T00:09:43.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058968359999999999 2024-09-16T00:09:43.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229705+0000", 2024-09-16T00:09:43.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003981409 2024-09-16T00:09:43.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229772+0000", 2024-09-16T00:09:43.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6137999999999996e-05 2024-09-16T00:09:43.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229828+0000", 2024-09-16T00:09:43.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5942000000000001e-05 2024-09-16T00:09:43.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.313 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.313 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10928 3.a 3:506aad5d:::1000000310f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:43.313 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.406639+0000", 2024-09-16T00:09:43.314 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.8520396039999998, 2024-09-16T00:09:43.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01087424, 2024-09-16T00:09:43.314 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.314 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.314 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.314 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.314 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10928 2024-09-16T00:09:43.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.314 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.406639+0000", 2024-09-16T00:09:43.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.406639+0000", 2024-09-16T00:09:43.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.406640+0000", 2024-09-16T00:09:43.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.697e-06 2024-09-16T00:09:43.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.410903+0000", 2024-09-16T00:09:43.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042625049999999998 2024-09-16T00:09:43.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.410928+0000", 2024-09-16T00:09:43.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4669999999999999e-05 2024-09-16T00:09:43.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.410936+0000", 2024-09-16T00:09:43.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1030000000000002e-06 2024-09-16T00:09:43.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.410999+0000", 2024-09-16T00:09:43.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2911999999999997e-05 2024-09-16T00:09:43.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.411070+0000", 2024-09-16T00:09:43.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0996999999999998e-05 2024-09-16T00:09:43.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.417378+0000", 2024-09-16T00:09:43.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063086349999999999 2024-09-16T00:09:43.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.417436+0000", 2024-09-16T00:09:43.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7946999999999999e-05 2024-09-16T00:09:43.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.417513+0000", 2024-09-16T00:09:43.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6773999999999994e-05 2024-09-16T00:09:43.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.321 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.321 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.321 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10942 3.34 3:2c63a741:::100000030c9.00000000:head [write 0~4194304 [4@1879734]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:43.321 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.457569+0000", 2024-09-16T00:09:43.322 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.801109082, 2024-09-16T00:09:43.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013282484000000001, 2024-09-16T00:09:43.322 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.322 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.322 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.322 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.323 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.323 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10942 2024-09-16T00:09:43.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.323 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.457569+0000", 2024-09-16T00:09:43.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.457569+0000", 2024-09-16T00:09:43.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.457571+0000", 2024-09-16T00:09:43.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6449999999999999e-06 2024-09-16T00:09:43.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.465713+0000", 2024-09-16T00:09:43.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081418250000000001 2024-09-16T00:09:43.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.465732+0000", 2024-09-16T00:09:43.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8725000000000001e-05 2024-09-16T00:09:43.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.465739+0000", 2024-09-16T00:09:43.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5789999999999998e-06 2024-09-16T00:09:43.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.465819+0000", 2024-09-16T00:09:43.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0332000000000003e-05 2024-09-16T00:09:43.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.465896+0000", 2024-09-16T00:09:43.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6281999999999999e-05 2024-09-16T00:09:43.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.470765+0000", 2024-09-16T00:09:43.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048688239999999999 2024-09-16T00:09:43.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.470800+0000", 2024-09-16T00:09:43.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5673000000000002e-05 2024-09-16T00:09:43.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.470852+0000", 2024-09-16T00:09:43.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1598999999999998e-05 2024-09-16T00:09:43.328 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.328 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.328 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.328 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:10941 3.1 e86/20 3:8246fad2:::10000003173.00000000:head v 86'159, mlcod=85'157)", 2024-09-16T00:09:43.328 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.458071+0000", 2024-09-16T00:09:43.329 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.8006073570000001, 2024-09-16T00:09:43.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010310779000000001, 2024-09-16T00:09:43.329 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.329 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.329 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.329 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.458071+0000", 2024-09-16T00:09:43.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.458071+0000", 2024-09-16T00:09:43.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.458072+0000", 2024-09-16T00:09:43.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.108e-06 2024-09-16T00:09:43.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.464755+0000", 2024-09-16T00:09:43.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066831460000000001 2024-09-16T00:09:43.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.464780+0000", 2024-09-16T00:09:43.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.457e-05 2024-09-16T00:09:43.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.464792+0000", 2024-09-16T00:09:43.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2495e-05 2024-09-16T00:09:43.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.464868+0000", 2024-09-16T00:09:43.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6019000000000005e-05 2024-09-16T00:09:43.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.464893+0000", 2024-09-16T00:09:43.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5026000000000001e-05 2024-09-16T00:09:43.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.468326+0000", 2024-09-16T00:09:43.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034325979999999998 2024-09-16T00:09:43.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.468382+0000", 2024-09-16T00:09:43.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5816999999999998e-05 2024-09-16T00:09:43.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.333 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.333 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10975 3.a 3:521a33ef:::10000003133.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:43.333 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.637144+0000", 2024-09-16T00:09:43.333 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.621534794, 2024-09-16T00:09:43.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010692317999999999, 2024-09-16T00:09:43.333 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.333 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.333 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.333 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.334 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.334 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10975 2024-09-16T00:09:43.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.334 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.637144+0000", 2024-09-16T00:09:43.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.637144+0000", 2024-09-16T00:09:43.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.637146+0000", 2024-09-16T00:09:43.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.345e-06 2024-09-16T00:09:43.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.643638+0000", 2024-09-16T00:09:43.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064925060000000003 2024-09-16T00:09:43.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.643651+0000", 2024-09-16T00:09:43.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2526e-05 2024-09-16T00:09:43.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.643657+0000", 2024-09-16T00:09:43.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1240000000000002e-06 2024-09-16T00:09:43.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.643710+0000", 2024-09-16T00:09:43.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2487000000000001e-05 2024-09-16T00:09:43.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.643812+0000", 2024-09-16T00:09:43.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102607 2024-09-16T00:09:43.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.647721+0000", 2024-09-16T00:09:43.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003908473 2024-09-16T00:09:43.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.647759+0000", 2024-09-16T00:09:43.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8272000000000002e-05 2024-09-16T00:09:43.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.647836+0000", 2024-09-16T00:09:43.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6977999999999994e-05 2024-09-16T00:09:43.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.338 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.338 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10998 3.a 3:506aad5d:::1000000310f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:43.338 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.711361+0000", 2024-09-16T00:09:43.338 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.5473173629999999, 2024-09-16T00:09:43.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099876419999999997, 2024-09-16T00:09:43.339 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.339 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.339 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.339 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.339 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10998 2024-09-16T00:09:43.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.339 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.711361+0000", 2024-09-16T00:09:43.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.711361+0000", 2024-09-16T00:09:43.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.711374+0000", 2024-09-16T00:09:43.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2789000000000001e-05 2024-09-16T00:09:43.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.716256+0000", 2024-09-16T00:09:43.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048822919999999999 2024-09-16T00:09:43.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.716279+0000", 2024-09-16T00:09:43.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2447999999999999e-05 2024-09-16T00:09:43.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.716289+0000", 2024-09-16T00:09:43.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0342e-05 2024-09-16T00:09:43.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.716355+0000", 2024-09-16T00:09:43.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6320000000000002e-05 2024-09-16T00:09:43.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.716422+0000", 2024-09-16T00:09:43.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6787999999999998e-05 2024-09-16T00:09:43.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.721182+0000", 2024-09-16T00:09:43.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004759887 2024-09-16T00:09:43.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.721251+0000", 2024-09-16T00:09:43.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8967999999999994e-05 2024-09-16T00:09:43.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.721349+0000", 2024-09-16T00:09:43.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7807999999999996e-05 2024-09-16T00:09:43.343 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.343 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.343 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.343 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11074 3.14 3:2834b7ee:::10000003131.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:43.343 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:09:43.344 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.3459102490000001, 2024-09-16T00:09:43.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015588350000000001, 2024-09-16T00:09:43.344 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.344 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.344 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.344 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11074 2024-09-16T00:09:43.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.344 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:09:43.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:09:43.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912769+0000", 2024-09-16T00:09:43.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4600000000000004e-07 2024-09-16T00:09:43.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918264+0000", 2024-09-16T00:09:43.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054954890000000001 2024-09-16T00:09:43.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918281+0000", 2024-09-16T00:09:43.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6160000000000001e-05 2024-09-16T00:09:43.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918289+0000", 2024-09-16T00:09:43.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1650000000000006e-06 2024-09-16T00:09:43.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918386+0000", 2024-09-16T00:09:43.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7288999999999996e-05 2024-09-16T00:09:43.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918535+0000", 2024-09-16T00:09:43.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014876200000000001 2024-09-16T00:09:43.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:43.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918639+0000", 2024-09-16T00:09:43.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010470400000000001 2024-09-16T00:09:43.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.924703+0000", 2024-09-16T00:09:43.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060639759999999996 2024-09-16T00:09:43.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928246+0000", 2024-09-16T00:09:43.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035421720000000001 2024-09-16T00:09:43.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928305+0000", 2024-09-16T00:09:43.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9014000000000001e-05 2024-09-16T00:09:43.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928357+0000", 2024-09-16T00:09:43.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1873000000000003e-05 2024-09-16T00:09:43.349 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.349 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.349 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.350 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:11109 3.27 e86/20 3:e5179c63:::10000003176.00000000:head v 86'185, mlcod=86'183)", 2024-09-16T00:09:43.350 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.073187+0000", 2024-09-16T00:09:43.350 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.185490959, 2024-09-16T00:09:43.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097254690000000005, 2024-09-16T00:09:43.350 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.350 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.350 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.350 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.073187+0000", 2024-09-16T00:09:43.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.073187+0000", 2024-09-16T00:09:43.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.073188+0000", 2024-09-16T00:09:43.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7400000000000002e-07 2024-09-16T00:09:43.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.077008+0000", 2024-09-16T00:09:43.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038198429999999998 2024-09-16T00:09:43.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.077022+0000", 2024-09-16T00:09:43.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4358e-05 2024-09-16T00:09:43.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.077030+0000", 2024-09-16T00:09:43.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4529999999999996e-06 2024-09-16T00:09:43.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.077099+0000", 2024-09-16T00:09:43.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8714999999999995e-05 2024-09-16T00:09:43.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.077125+0000", 2024-09-16T00:09:43.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6832999999999999e-05 2024-09-16T00:09:43.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.082866+0000", 2024-09-16T00:09:43.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057409050000000001 2024-09-16T00:09:43.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.082913+0000", 2024-09-16T00:09:43.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6588000000000002e-05 2024-09-16T00:09:43.354 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.354 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.354 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.354 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11111 3.9 3:93301cc4:::10000003210.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:43.354 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:09:43.354 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.169172047, 2024-09-16T00:09:43.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014336099999999999, 2024-09-16T00:09:43.354 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.354 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.355 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.355 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.355 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11111 2024-09-16T00:09:43.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.355 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:09:43.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:09:43.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089520+0000", 2024-09-16T00:09:43.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3903000000000001e-05 2024-09-16T00:09:43.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094463+0000", 2024-09-16T00:09:43.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049423000000000002 2024-09-16T00:09:43.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094494+0000", 2024-09-16T00:09:43.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1029999999999999e-05 2024-09-16T00:09:43.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094503+0000", 2024-09-16T00:09:43.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0180000000000007e-06 2024-09-16T00:09:43.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094573+0000", 2024-09-16T00:09:43.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0457000000000007e-05 2024-09-16T00:09:43.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094737+0000", 2024-09-16T00:09:43.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016408699999999999 2024-09-16T00:09:43.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:43.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094897+0000", 2024-09-16T00:09:43.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015938899999999999 2024-09-16T00:09:43.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.102326+0000", 2024-09-16T00:09:43.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074291230000000002 2024-09-16T00:09:43.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103723+0000", 2024-09-16T00:09:43.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001397255 2024-09-16T00:09:43.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103786+0000", 2024-09-16T00:09:43.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3489000000000001e-05 2024-09-16T00:09:43.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103842+0000", 2024-09-16T00:09:43.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6048999999999998e-05 2024-09-16T00:09:43.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.360 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.360 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11152 3.14 3:2b654b97:::100000031da.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:43.360 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.355158+0000", 2024-09-16T00:09:43.361 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.9035202440000001, 2024-09-16T00:09:43.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010522013, 2024-09-16T00:09:43.361 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.361 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.361 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.361 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.361 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11152 2024-09-16T00:09:43.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.361 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.355158+0000", 2024-09-16T00:09:43.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.355158+0000", 2024-09-16T00:09:43.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.355160+0000", 2024-09-16T00:09:43.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3459999999999999e-06 2024-09-16T00:09:43.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.359409+0000", 2024-09-16T00:09:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042498919999999999 2024-09-16T00:09:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.359429+0000", 2024-09-16T00:09:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0097000000000001e-05 2024-09-16T00:09:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.359441+0000", 2024-09-16T00:09:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1113e-05 2024-09-16T00:09:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.359510+0000", 2024-09-16T00:09:43.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9704999999999998e-05 2024-09-16T00:09:43.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.359604+0000", 2024-09-16T00:09:43.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3449999999999995e-05 2024-09-16T00:09:43.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:43.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.359742+0000", 2024-09-16T00:09:43.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013862100000000001 2024-09-16T00:09:43.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.365420+0000", 2024-09-16T00:09:43.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056777190000000003 2024-09-16T00:09:43.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.365581+0000", 2024-09-16T00:09:43.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000161395 2024-09-16T00:09:43.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.365632+0000", 2024-09-16T00:09:43.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0865000000000002e-05 2024-09-16T00:09:43.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.365680+0000", 2024-09-16T00:09:43.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7809999999999998e-05 2024-09-16T00:09:43.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.366 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.366 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:11162 3.11 e86/20 3:88de54a5:::10000003191.00000000:head v 86'199, mlcod=86'197)", 2024-09-16T00:09:43.366 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.396872+0000", 2024-09-16T00:09:43.367 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.8618062960000001, 2024-09-16T00:09:43.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010108260000000001, 2024-09-16T00:09:43.367 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.367 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.367 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.367 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.396872+0000", 2024-09-16T00:09:43.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.396872+0000", 2024-09-16T00:09:43.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.396873+0000", 2024-09-16T00:09:43.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9899999999999999e-07 2024-09-16T00:09:43.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.398548+0000", 2024-09-16T00:09:43.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016751960000000001 2024-09-16T00:09:43.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.398570+0000", 2024-09-16T00:09:43.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1614e-05 2024-09-16T00:09:43.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.398582+0000", 2024-09-16T00:09:43.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1912999999999999e-05 2024-09-16T00:09:43.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.398655+0000", 2024-09-16T00:09:43.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3190999999999998e-05 2024-09-16T00:09:43.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.398682+0000", 2024-09-16T00:09:43.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.709e-05 2024-09-16T00:09:43.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.406921+0000", 2024-09-16T00:09:43.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082386549999999992 2024-09-16T00:09:43.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.406980+0000", 2024-09-16T00:09:43.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9701999999999999e-05 2024-09-16T00:09:43.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.371 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.371 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.371 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11181 3.14 3:2b981f79:::10000003145.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:43.371 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.465034+0000", 2024-09-16T00:09:43.371 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 2.793643989, 2024-09-16T00:09:43.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01065118, 2024-09-16T00:09:43.371 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.371 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.371 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.371 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.371 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.371 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.372 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11181 2024-09-16T00:09:43.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.372 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.465034+0000", 2024-09-16T00:09:43.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.465034+0000", 2024-09-16T00:09:43.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.465036+0000", 2024-09-16T00:09:43.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2869999999999999e-06 2024-09-16T00:09:43.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.469763+0000", 2024-09-16T00:09:43.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047272529999999998 2024-09-16T00:09:43.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.469783+0000", 2024-09-16T00:09:43.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9661e-05 2024-09-16T00:09:43.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.469795+0000", 2024-09-16T00:09:43.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2309e-05 2024-09-16T00:09:43.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.469905+0000", 2024-09-16T00:09:43.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010978499999999999 2024-09-16T00:09:43.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.470014+0000", 2024-09-16T00:09:43.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010971699999999999 2024-09-16T00:09:43.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:43.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.470174+0000", 2024-09-16T00:09:43.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015976200000000001 2024-09-16T00:09:43.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.474627+0000", 2024-09-16T00:09:43.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044532590000000002 2024-09-16T00:09:43.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.475566+0000", 2024-09-16T00:09:43.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00093897700000000002 2024-09-16T00:09:43.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.475629+0000", 2024-09-16T00:09:43.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2408000000000003e-05 2024-09-16T00:09:43.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.475686+0000", 2024-09-16T00:09:43.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6762000000000003e-05 2024-09-16T00:09:43.377 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.377 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.377 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.378 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11781 3.a 3:516d17a9:::100000031cd.00000000:head [write 0~4194304 [3@614893]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:43.378 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.203256+0000", 2024-09-16T00:09:43.378 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.055422759000000002, 2024-09-16T00:09:43.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010184565, 2024-09-16T00:09:43.378 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.378 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.378 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.378 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11781 2024-09-16T00:09:43.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.378 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.203256+0000", 2024-09-16T00:09:43.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.203256+0000", 2024-09-16T00:09:43.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.203257+0000", 2024-09-16T00:09:43.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2950000000000001e-06 2024-09-16T00:09:43.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.207378+0000", 2024-09-16T00:09:43.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041205640000000002 2024-09-16T00:09:43.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.207401+0000", 2024-09-16T00:09:43.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3187e-05 2024-09-16T00:09:43.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.207410+0000", 2024-09-16T00:09:43.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5589999999999994e-06 2024-09-16T00:09:43.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.207487+0000", 2024-09-16T00:09:43.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6623000000000001e-05 2024-09-16T00:09:43.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.207561+0000", 2024-09-16T00:09:43.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3868000000000005e-05 2024-09-16T00:09:43.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.213281+0000", 2024-09-16T00:09:43.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057199479999999999 2024-09-16T00:09:43.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.213342+0000", 2024-09-16T00:09:43.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1221000000000002e-05 2024-09-16T00:09:43.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.213440+0000", 2024-09-16T00:09:43.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8300000000000004e-05 2024-09-16T00:09:43.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.383 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.383 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.383 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.383 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:43.385 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:09:43.386 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:43.383+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:43.390 INFO:tasks.workunit.client.0.smithi139.stdout:7/333: dread - d5/d26/d46/f68 zero size 2024-09-16T00:09:43.390 INFO:tasks.workunit.client.0.smithi139.stdout:4/283: rmdir d12 39 2024-09-16T00:09:43.390 INFO:tasks.workunit.client.0.smithi139.stdout:2/339: write d11/d30/d57/d29/d2b/f65 [396319,112592] 0 2024-09-16T00:09:43.390 INFO:tasks.workunit.client.0.smithi139.stdout:5/386: write d1/d13/f17 [551689,112131] 0 2024-09-16T00:09:43.390 INFO:tasks.workunit.client.0.smithi139.stdout:1/366: symlink d7/d8/d1f/l6e 0 2024-09-16T00:09:43.390 INFO:tasks.workunit.client.0.smithi139.stdout:7/334: read f4 [6450721,15439] 0 2024-09-16T00:09:43.390 INFO:tasks.workunit.client.0.smithi139.stdout:0/349: creat d6/d5a/f6f x:0 0 0 2024-09-16T00:09:43.390 INFO:tasks.workunit.client.0.smithi139.stdout:7/335: read d5/d26/f32 [1116249,57378] 0 2024-09-16T00:09:43.390 INFO:tasks.workunit.client.0.smithi139.stdout:4/284: write d12/d4a/f54 [64985,85441] 0 2024-09-16T00:09:43.390 INFO:tasks.workunit.client.0.smithi139.stdout:4/285: fsync d12/d47/f45 0 2024-09-16T00:09:43.390 INFO:tasks.workunit.client.0.smithi139.stdout:9/366: dwrite d19/d1a/d20/d23/d2d/d35/d42/d3a/f48 [0,4194304] 0 2024-09-16T00:09:43.391 INFO:tasks.workunit.client.0.smithi139.stdout:4/286: fdatasync d12/d47/f45 0 2024-09-16T00:09:43.391 INFO:tasks.workunit.client.0.smithi139.stdout:0/350: read d6/d1c/d41/d44/f33 [102135,48936] 0 2024-09-16T00:09:43.391 INFO:tasks.workunit.client.0.smithi139.stdout:4/287: chown d12/d47/c28 0 1 2024-09-16T00:09:43.391 INFO:tasks.workunit.client.0.smithi139.stdout:1/367: creat d7/d8/f6f x:0 0 0 2024-09-16T00:09:43.391 INFO:tasks.workunit.client.0.smithi139.stdout:2/340: write d11/d30/d57/d29/f66 [4907009,121724] 0 2024-09-16T00:09:43.391 INFO:tasks.workunit.client.0.smithi139.stdout:7/336: mkdir d5/d26/d74 0 2024-09-16T00:09:43.391 INFO:tasks.workunit.client.0.smithi139.stdout:1/368: chown d7/d8/d40/d47/f50 179 1 2024-09-16T00:09:43.391 INFO:tasks.workunit.client.0.smithi139.stdout:2/341: write d11/f19 [1030924,76773] 0 2024-09-16T00:09:43.391 INFO:tasks.workunit.client.0.smithi139.stdout:0/351: rename d6/d1c/d45/d49 to d6/d5a/d70 0 2024-09-16T00:09:43.391 INFO:tasks.workunit.client.0.smithi139.stdout:5/387: creat d1/d21/d49/f7f x:0 0 0 2024-09-16T00:09:43.391 INFO:tasks.workunit.client.0.smithi139.stdout:4/288: creat d12/d47/d57/f5f x:0 0 0 2024-09-16T00:09:43.391 INFO:tasks.workunit.client.0.smithi139.stdout:0/352: chown d6/d1c/d41/d44/dd/l2b 28 1 2024-09-16T00:09:43.391 INFO:tasks.workunit.client.0.smithi139.stdout:4/289: write d12/d2b/f5a [932625,26298] 0 2024-09-16T00:09:43.391 INFO:tasks.workunit.client.0.smithi139.stdout:2/342: getdents d11/d30 0 2024-09-16T00:09:43.391 INFO:tasks.workunit.client.0.smithi139.stdout:1/369: getdents d7/d8/d40/d48/d67 0 2024-09-16T00:09:43.392 INFO:tasks.workunit.client.0.smithi139.stdout:4/290: write d12/d4a/f54 [493241,127010] 0 2024-09-16T00:09:43.392 INFO:tasks.workunit.client.0.smithi139.stdout:9/367: dread d19/d1a/d20/d23/d2d/d35/d3d/f54 [0,4194304] 0 2024-09-16T00:09:43.392 INFO:tasks.workunit.client.0.smithi139.stdout:7/337: rename d5/d16/d22/d25/d2d/d50/d44/f4f to d5/d26/f75 0 2024-09-16T00:09:43.392 INFO:tasks.workunit.client.0.smithi139.stdout:2/343: truncate d11/d30/f56 1244496 0 2024-09-16T00:09:43.392 INFO:tasks.workunit.client.0.smithi139.stdout:5/388: dwrite d1/d25/d30/d33/f7c [0,4194304] 0 2024-09-16T00:09:43.392 INFO:tasks.workunit.client.0.smithi139.stdout:0/353: dwrite d6/d5a/d70/f53 [0,4194304] 0 2024-09-16T00:09:43.392 INFO:tasks.workunit.client.0.smithi139.stdout:2/344: write d11/d2f/f33 [2849796,16053] 0 2024-09-16T00:09:43.392 INFO:tasks.workunit.client.0.smithi139.stdout:9/368: unlink d19/d1a/d20/d23/d2d/d35/d42/c6a 0 2024-09-16T00:09:43.392 INFO:tasks.workunit.client.0.smithi139.stdout:1/370: write d7/d8/d1f/f37 [757100,44485] 0 2024-09-16T00:09:43.392 INFO:tasks.workunit.client.0.smithi139.stdout:5/389: write d1/f4a [4842980,108562] 0 2024-09-16T00:09:43.392 INFO:tasks.workunit.client.0.smithi139.stdout:7/338: rename d5/d26/f62 to d5/d16/d22/d25/f76 0 2024-09-16T00:09:43.393 INFO:tasks.workunit.client.0.smithi139.stdout:2/345: chown d11/d30/c47 21 1 2024-09-16T00:09:43.393 INFO:tasks.workunit.client.0.smithi139.stdout:4/291: dwrite f2 [0,4194304] 0 2024-09-16T00:09:43.393 INFO:tasks.workunit.client.0.smithi139.stdout:9/369: mknod d19/d1a/d20/c75 0 2024-09-16T00:09:43.393 INFO:tasks.workunit.client.0.smithi139.stdout:4/292: stat d12/d2b/f26 0 2024-09-16T00:09:43.393 INFO:tasks.workunit.client.0.smithi139.stdout:1/371: creat d7/d8/d40/d46/f70 x:0 0 0 2024-09-16T00:09:43.393 INFO:tasks.workunit.client.0.smithi139.stdout:1/372: chown d7/f28 10019 1 2024-09-16T00:09:43.393 INFO:tasks.workunit.client.0.smithi139.stdout:9/370: creat d19/d1a/d20/d23/d2d/d41/d71/d32/f76 x:0 0 0 2024-09-16T00:09:43.393 INFO:tasks.workunit.client.0.smithi139.stdout:7/339: symlink d5/l77 0 2024-09-16T00:09:43.393 INFO:tasks.workunit.client.0.smithi139.stdout:0/354: creat d6/d1c/d41/d44/f71 x:0 0 0 2024-09-16T00:09:43.393 INFO:tasks.workunit.client.0.smithi139.stdout:2/346: rename d11/d30/d57/d50/l5a to d11/d32/l71 0 2024-09-16T00:09:43.393 INFO:tasks.workunit.client.0.smithi139.stdout:1/373: mknod d7/d8/d40/d48/d53/c71 0 2024-09-16T00:09:43.393 INFO:tasks.workunit.client.0.smithi139.stdout:5/390: creat d1/d13/d22/f80 x:0 0 0 2024-09-16T00:09:43.393 INFO:tasks.workunit.client.0.smithi139.stdout:9/371: creat d19/d1a/d20/d59/f77 x:0 0 0 2024-09-16T00:09:43.393 INFO:tasks.workunit.client.0.smithi139.stdout:4/293: dread d12/d47/f24 [0,4194304] 0 2024-09-16T00:09:43.394 INFO:tasks.workunit.client.0.smithi139.stdout:4/294: readlink d12/l58 0 2024-09-16T00:09:43.394 INFO:tasks.workunit.client.0.smithi139.stdout:3/350: sync 2024-09-16T00:09:43.420 INFO:tasks.workunit.client.0.smithi139.stdout:1/374: dread d7/d8/d27/d2f/f41 [0,4194304] 0 2024-09-16T00:09:43.420 INFO:tasks.workunit.client.0.smithi139.stdout:4/295: dread - d12/f4f zero size 2024-09-16T00:09:43.421 INFO:tasks.workunit.client.0.smithi139.stdout:5/391: creat d1/d21/d49/f81 x:0 0 0 2024-09-16T00:09:43.421 INFO:tasks.workunit.client.0.smithi139.stdout:9/372: chown d19/d1a/d20/d23/d2d/d35/d42/f5e 15406728 1 2024-09-16T00:09:43.423 INFO:tasks.workunit.client.0.smithi139.stdout:3/351: readlink d5/d7/d23/d48/l51 0 2024-09-16T00:09:43.423 INFO:tasks.workunit.client.0.smithi139.stdout:5/392: chown d1/d13/d22 1109620 1 2024-09-16T00:09:43.424 INFO:tasks.workunit.client.0.smithi139.stdout:4/296: write d12/f40 [4072456,28963] 0 2024-09-16T00:09:43.424 INFO:tasks.workunit.client.0.smithi139.stdout:1/375: mknod d7/d8/d27/c72 0 2024-09-16T00:09:43.426 INFO:tasks.workunit.client.0.smithi139.stdout:9/373: write d19/d1a/f3f [239666,42643] 0 2024-09-16T00:09:43.428 INFO:tasks.workunit.client.0.smithi139.stdout:5/393: chown d1/d13/d3f/d5e 1 1 2024-09-16T00:09:43.429 INFO:tasks.workunit.client.0.smithi139.stdout:1/376: write d7/d1b/f3c [1920084,95893] 0 2024-09-16T00:09:43.430 INFO:tasks.workunit.client.0.smithi139.stdout:2/347: dwrite d11/f1a [0,4194304] 0 2024-09-16T00:09:43.430 INFO:tasks.workunit.client.0.smithi139.stdout:4/297: fsync d12/d2b/f26 0 2024-09-16T00:09:43.430 INFO:tasks.workunit.client.0.smithi139.stdout:5/394: read d1/f4a [1266668,2349] 0 2024-09-16T00:09:43.431 INFO:tasks.workunit.client.0.smithi139.stdout:0/355: dwrite d6/d1c/d41/d44/dd/f11 [4194304,4194304] 0 2024-09-16T00:09:43.432 INFO:tasks.workunit.client.0.smithi139.stdout:3/352: mknod d5/d7/d23/d27/d2f/d30/c7c 0 2024-09-16T00:09:43.435 INFO:tasks.workunit.client.0.smithi139.stdout:5/395: stat d1/c11 0 2024-09-16T00:09:43.438 INFO:tasks.workunit.client.0.smithi139.stdout:4/298: mkdir d12/d47/d49/d60 0 2024-09-16T00:09:43.441 INFO:tasks.workunit.client.0.smithi139.stdout:2/348: unlink d11/d2f/c48 0 2024-09-16T00:09:43.443 INFO:tasks.workunit.client.0.smithi139.stdout:1/377: creat d7/d8/d27/d2f/f73 x:0 0 0 2024-09-16T00:09:43.445 INFO:tasks.workunit.client.0.smithi139.stdout:1/378: write d7/f18 [9382523,62557] 0 2024-09-16T00:09:43.446 INFO:tasks.workunit.client.0.smithi139.stdout:9/374: rename d19/d1a/d20/d59/f77 to d19/d1a/d20/d23/d2d/d41/d71/d32/d4e/f78 0 2024-09-16T00:09:43.447 INFO:tasks.workunit.client.0.smithi139.stdout:1/379: write d7/d8/d1f/f37 [40415,6518] 0 2024-09-16T00:09:43.447 INFO:tasks.workunit.client.0.smithi139.stdout:4/299: fdatasync d12/f43 0 2024-09-16T00:09:43.448 INFO:tasks.workunit.client.0.smithi139.stdout:5/396: chown d1/d21/d49/c54 67743 1 2024-09-16T00:09:43.448 INFO:tasks.workunit.client.0.smithi139.stdout:2/349: dwrite d11/f15 [4194304,4194304] 0 2024-09-16T00:09:43.450 INFO:tasks.workunit.client.0.smithi139.stdout:3/353: creat d5/d7/f7d x:0 0 0 2024-09-16T00:09:43.450 INFO:tasks.workunit.client.0.smithi139.stdout:4/300: stat d12/d47/f45 0 2024-09-16T00:09:43.451 INFO:tasks.workunit.client.0.smithi139.stdout:5/397: symlink d1/l82 0 2024-09-16T00:09:43.451 INFO:tasks.workunit.client.0.smithi139.stdout:1/380: write d7/f12 [1899471,118128] 0 2024-09-16T00:09:43.454 INFO:tasks.workunit.client.0.smithi139.stdout:4/301: mknod d12/d4a/c61 0 2024-09-16T00:09:43.455 INFO:tasks.workunit.client.0.smithi139.stdout:9/375: truncate d19/d1a/d20/d23/d2d/d41/d71/d32/f76 849150 0 2024-09-16T00:09:43.456 INFO:tasks.workunit.client.0.smithi139.stdout:3/354: creat d5/d7/d23/d27/d2f/d30/d4c/d6d/f7e x:0 0 0 2024-09-16T00:09:43.457 INFO:tasks.workunit.client.0.smithi139.stdout:2/350: truncate d11/d30/d57/d29/d2b/f31 374259 0 2024-09-16T00:09:43.457 INFO:tasks.workunit.client.0.smithi139.stdout:5/398: rename d1/d21/d49/d53 to d1/d21/d49/d53/d6a/d7b/d78/d83 22 2024-09-16T00:09:43.460 INFO:tasks.workunit.client.0.smithi139.stdout:4/302: write d12/f23 [2188205,40944] 0 2024-09-16T00:09:43.461 INFO:tasks.workunit.client.0.smithi139.stdout:9/376: chown d19/d1a/d20/d23/d2d/d35/d44 92586 1 2024-09-16T00:09:43.463 INFO:tasks.workunit.client.0.smithi139.stdout:5/399: chown d1/d13/d3b/d3e/f5f 0 1 2024-09-16T00:09:43.463 INFO:tasks.workunit.client.0.smithi139.stdout:2/351: chown d11/d30/d57/d29/c4a 118873207 1 2024-09-16T00:09:43.465 INFO:tasks.workunit.client.0.smithi139.stdout:0/356: dwrite d6/d1c/d41/d44/f29 [0,4194304] 0 2024-09-16T00:09:43.465 INFO:tasks.workunit.client.0.smithi139.stdout:3/355: creat d5/d7/d23/d27/d2f/d30/d41/f7f x:0 0 0 2024-09-16T00:09:43.466 INFO:tasks.workunit.client.0.smithi139.stdout:4/303: rename d12/d47/f24 to d12/d47/d57/f62 0 2024-09-16T00:09:43.468 INFO:tasks.workunit.client.0.smithi139.stdout:5/400: symlink d1/d25/d30/d64/l84 0 2024-09-16T00:09:43.470 INFO:tasks.workunit.client.0.smithi139.stdout:0/357: creat d6/d1c/d41/f72 x:0 0 0 2024-09-16T00:09:43.471 INFO:tasks.workunit.client.0.smithi139.stdout:9/377: rename d19/d1a/d20/d59 to d19/d1a/d20/d23/d2d/d41/d71/d32/d79 0 2024-09-16T00:09:43.471 INFO:tasks.workunit.client.0.smithi139.stdout:0/358: dread - d6/d1c/d41/d44/dd/f6e zero size 2024-09-16T00:09:43.472 INFO:tasks.workunit.client.0.smithi139.stdout:4/304: creat d12/d4a/f63 x:0 0 0 2024-09-16T00:09:43.473 INFO:tasks.workunit.client.0.smithi139.stdout:2/352: dwrite d11/f64 [0,4194304] 0 2024-09-16T00:09:43.474 INFO:tasks.workunit.client.0.smithi139.stdout:5/401: symlink d1/d21/d49/d53/d6a/d7b/d7a/l85 0 2024-09-16T00:09:43.477 INFO:tasks.workunit.client.0.smithi139.stdout:4/305: symlink d12/d4a/d4d/l64 0 2024-09-16T00:09:43.478 INFO:tasks.workunit.client.0.smithi139.stdout:9/378: mkdir d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/d7a 0 2024-09-16T00:09:43.480 INFO:tasks.workunit.client.0.smithi139.stdout:5/402: symlink d1/d25/d73/l86 0 2024-09-16T00:09:43.482 INFO:tasks.workunit.client.0.smithi139.stdout:5/403: dread d1/d25/f6c [0,4194304] 0 2024-09-16T00:09:43.483 INFO:tasks.workunit.client.0.smithi139.stdout:5/404: write d1/d13/d3b/f7d [141469,125349] 0 2024-09-16T00:09:43.483 INFO:tasks.workunit.client.0.smithi139.stdout:3/356: rename d5/d7/d23/d35/f70 to d5/d7/d23/d27/d2f/d30/f80 0 2024-09-16T00:09:43.485 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:43.484+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:43.486 INFO:tasks.workunit.client.0.smithi139.stdout:9/379: mknod d19/d1a/d20/d23/d2d/d41/c7b 0 2024-09-16T00:09:43.488 INFO:tasks.workunit.client.0.smithi139.stdout:0/359: write d6/f8 [7955912,42357] 0 2024-09-16T00:09:43.489 INFO:tasks.workunit.client.0.smithi139.stdout:0/360: write d6/d5a/f6f [1012862,31105] 0 2024-09-16T00:09:43.489 INFO:tasks.workunit.client.0.smithi139.stdout:4/306: mknod d12/c65 0 2024-09-16T00:09:43.490 INFO:tasks.workunit.client.0.smithi139.stdout:3/357: rename d5/d7/d23 to d5/d7/d23/d27/d2f/d30/d81 22 2024-09-16T00:09:43.490 INFO:tasks.workunit.client.0.smithi139.stdout:0/361: dread d6/d1c/d45/f50 [0,4194304] 0 2024-09-16T00:09:43.491 INFO:tasks.workunit.client.0.smithi139.stdout:9/380: creat d19/d1a/d20/d23/d2d/d41/f7c x:0 0 0 2024-09-16T00:09:43.493 INFO:tasks.workunit.client.0.smithi139.stdout:5/405: mkdir d1/d21/d49/d87 0 2024-09-16T00:09:43.494 INFO:tasks.workunit.client.0.smithi139.stdout:2/353: getdents d11/d30/d57/d50 0 2024-09-16T00:09:43.495 INFO:tasks.workunit.client.0.smithi139.stdout:3/358: write d5/d7/d23/d27/d2f/d30/d47/f25 [4121615,115288] 0 2024-09-16T00:09:43.496 INFO:tasks.workunit.client.0.smithi139.stdout:3/359: readlink d5/d7/d23/l22 0 2024-09-16T00:09:43.497 INFO:tasks.workunit.client.0.smithi139.stdout:9/381: mknod d19/d1a/d20/d23/d2d/d2f/c7d 0 2024-09-16T00:09:43.497 INFO:tasks.workunit.client.0.smithi139.stdout:9/382: stat d19 0 2024-09-16T00:09:43.498 INFO:tasks.workunit.client.0.smithi139.stdout:9/383: write d19/d1a/d20/d23/d2d/d35/d42/d3a/f6d [948916,41947] 0 2024-09-16T00:09:43.499 INFO:tasks.workunit.client.0.smithi139.stdout:5/406: rename d1/d25/d30/d64 to d1/d13/d3f/d88 0 2024-09-16T00:09:43.499 INFO:tasks.workunit.client.0.smithi139.stdout:5/407: chown d1/d21/c62 34 1 2024-09-16T00:09:43.500 INFO:tasks.workunit.client.0.smithi139.stdout:4/307: creat d12/d47/d49/d60/f66 x:0 0 0 2024-09-16T00:09:43.501 INFO:tasks.workunit.client.0.smithi139.stdout:0/362: truncate d6/d1c/d41/d44/dd/f2d 1518798 0 2024-09-16T00:09:43.503 INFO:tasks.workunit.client.0.smithi139.stdout:3/360: mknod d5/d7/d23/d27/d2f/d30/d4c/d6d/c82 0 2024-09-16T00:09:43.504 INFO:tasks.workunit.client.0.smithi139.stdout:9/384: creat d19/d1a/d20/d23/d2d/d35/d3d/f7e x:0 0 0 2024-09-16T00:09:43.504 INFO:tasks.workunit.client.0.smithi139.stdout:3/361: dread - d5/d7/d23/d35/f45 zero size 2024-09-16T00:09:43.506 INFO:tasks.workunit.client.0.smithi139.stdout:5/408: dwrite d1/d13/d3b/d3e/f69 [0,4194304] 0 2024-09-16T00:09:43.512 INFO:tasks.workunit.client.0.smithi139.stdout:3/362: write d5/d7/dc/f63 [858014,21458] 0 2024-09-16T00:09:43.512 INFO:tasks.workunit.client.0.smithi139.stdout:2/354: link d11/d30/d57/d29/d2b/c55 d11/d32/c72 0 2024-09-16T00:09:43.512 INFO:tasks.workunit.client.0.smithi139.stdout:0/363: chown d6/d1c/d41/d44/f33 7 1 2024-09-16T00:09:43.514 INFO:tasks.workunit.client.0.smithi139.stdout:4/308: dread d12/f31 [0,4194304] 0 2024-09-16T00:09:43.516 INFO:tasks.workunit.client.0.smithi139.stdout:0/364: stat d6/d1c/d41/d44/c51 0 2024-09-16T00:09:43.519 INFO:tasks.workunit.client.0.smithi139.stdout:3/363: creat d5/d7/d23/d27/f83 x:0 0 0 2024-09-16T00:09:43.521 INFO:tasks.workunit.client.0.smithi139.stdout:5/409: dwrite d1/d21/d49/f81 [0,4194304] 0 2024-09-16T00:09:43.521 INFO:tasks.workunit.client.0.smithi139.stdout:2/355: dread d11/d30/d57/d29/d2b/f65 [0,4194304] 0 2024-09-16T00:09:43.522 INFO:tasks.workunit.client.0.smithi139.stdout:9/385: rmdir d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/d7a 0 2024-09-16T00:09:43.523 INFO:tasks.workunit.client.0.smithi139.stdout:2/356: chown d11/d30/d57/c3a 3391 1 2024-09-16T00:09:43.525 INFO:tasks.workunit.client.0.smithi139.stdout:5/410: write d1/d13/d3b/f7d [693390,33829] 0 2024-09-16T00:09:43.525 INFO:tasks.workunit.client.0.smithi139.stdout:3/364: mkdir d5/d7/dc/d84 0 2024-09-16T00:09:43.526 INFO:tasks.workunit.client.0.smithi139.stdout:9/386: fsync d19/d1a/d20/d23/d2d/d35/d42/d3a/f6e 0 2024-09-16T00:09:43.527 INFO:tasks.workunit.client.0.smithi139.stdout:2/357: mkdir d11/d30/d57/d5d/d73 0 2024-09-16T00:09:43.527 INFO:tasks.workunit.client.0.smithi139.stdout:3/365: rmdir d5/d7/d23/d48 39 2024-09-16T00:09:43.528 INFO:tasks.workunit.client.0.smithi139.stdout:2/358: readlink d11/d30/d57/l51 0 2024-09-16T00:09:43.530 INFO:tasks.workunit.client.0.smithi139.stdout:5/411: rename d1/d25/d30/d33/d43/l57 to d1/d21/d49/d53/d6a/l89 0 2024-09-16T00:09:43.530 INFO:tasks.workunit.client.0.smithi139.stdout:3/366: dread d5/d7/d23/d27/d2f/d30/d4c/d50/f59 [0,4194304] 0 2024-09-16T00:09:43.531 INFO:tasks.workunit.client.0.smithi139.stdout:3/367: write d5/d7/d23/d27/d2f/d30/f42 [271946,49444] 0 2024-09-16T00:09:43.535 INFO:tasks.workunit.client.0.smithi139.stdout:0/365: dread d6/ff [4194304,4194304] 0 2024-09-16T00:09:43.535 INFO:tasks.workunit.client.0.smithi139.stdout:5/412: rmdir d1/d21/d49/d53/d6a/d7b 39 2024-09-16T00:09:43.536 INFO:tasks.workunit.client.0.smithi139.stdout:0/366: fdatasync d6/d5a/f65 0 2024-09-16T00:09:43.537 INFO:tasks.workunit.client.0.smithi139.stdout:4/309: truncate fd 3441299 0 2024-09-16T00:09:43.542 INFO:tasks.workunit.client.0.smithi139.stdout:9/387: write d19/d1a/d20/d23/d2d/d35/d42/f34 [759671,78921] 0 2024-09-16T00:09:43.542 INFO:tasks.workunit.client.0.smithi139.stdout:0/367: symlink d6/d42/l73 0 2024-09-16T00:09:43.545 INFO:tasks.workunit.client.0.smithi139.stdout:0/368: mknod d6/d42/c74 0 2024-09-16T00:09:43.545 INFO:tasks.workunit.client.0.smithi139.stdout:3/368: dread d5/d7/d23/f15 [0,4194304] 0 2024-09-16T00:09:43.546 INFO:tasks.workunit.client.0.smithi139.stdout:0/369: readlink d6/d5a/l62 0 2024-09-16T00:09:43.546 INFO:tasks.workunit.client.0.smithi139.stdout:2/359: getdents d11/d30/d57/d29/d2b 0 2024-09-16T00:09:43.547 INFO:tasks.workunit.client.0.smithi139.stdout:6/269: sync 2024-09-16T00:09:43.548 INFO:tasks.workunit.client.0.smithi139.stdout:9/388: rename d19/d1a/d20/c75 to d19/d1a/d20/c7f 0 2024-09-16T00:09:43.548 INFO:tasks.workunit.client.0.smithi139.stdout:0/370: dread - d6/d1c/d41/f72 zero size 2024-09-16T00:09:43.550 INFO:tasks.workunit.client.0.smithi139.stdout:6/270: write d8/f12 [4943833,38548] 0 2024-09-16T00:09:43.550 INFO:tasks.workunit.client.0.smithi139.stdout:0/371: read f3 [4163273,122629] 0 2024-09-16T00:09:43.550 INFO:tasks.workunit.client.0.smithi139.stdout:3/369: rename d5/l62 to d5/d7/dc/l85 0 2024-09-16T00:09:43.553 INFO:tasks.workunit.client.0.smithi139.stdout:6/271: dread - d8/d29/f49 zero size 2024-09-16T00:09:43.553 INFO:tasks.workunit.client.0.smithi139.stdout:0/372: fsync d6/d1c/d41/f6a 0 2024-09-16T00:09:43.555 INFO:tasks.workunit.client.0.smithi139.stdout:4/310: dread f1 [0,4194304] 0 2024-09-16T00:09:43.556 INFO:tasks.workunit.client.0.smithi139.stdout:6/272: write d8/d29/f49 [369892,16932] 0 2024-09-16T00:09:43.558 INFO:tasks.workunit.client.0.smithi139.stdout:9/389: link d19/d1a/d20/d23/d2d/d41/d71/c53 d19/c80 0 2024-09-16T00:09:43.559 INFO:tasks.workunit.client.0.smithi139.stdout:3/370: truncate d5/d7/f2d 2734030 0 2024-09-16T00:09:43.562 INFO:tasks.workunit.client.0.smithi139.stdout:4/311: getdents d12/d47 0 2024-09-16T00:09:43.562 INFO:tasks.workunit.client.0.smithi139.stdout:5/413: dwrite d1/d13/d3b/f44 [0,4194304] 0 2024-09-16T00:09:43.565 INFO:tasks.workunit.client.0.smithi139.stdout:2/360: dread d11/f22 [0,4194304] 0 2024-09-16T00:09:43.566 INFO:tasks.workunit.client.0.smithi139.stdout:4/312: write d12/f3f [3201474,18823] 0 2024-09-16T00:09:43.567 INFO:tasks.workunit.client.0.smithi139.stdout:9/390: getdents d19/d1a/d20/d23 0 2024-09-16T00:09:43.567 INFO:tasks.workunit.client.0.smithi139.stdout:3/371: write d5/d7/d23/d27/d2f/d30/d41/f39 [4706730,43706] 0 2024-09-16T00:09:43.568 INFO:tasks.workunit.client.0.smithi139.stdout:5/414: fsync d1/d25/d30/d33/f50 0 2024-09-16T00:09:43.569 INFO:tasks.workunit.client.0.smithi139.stdout:0/373: dwrite d6/d5a/f6f [0,4194304] 0 2024-09-16T00:09:43.572 INFO:tasks.workunit.client.0.smithi139.stdout:9/391: stat d19/d1a/d20/d23/d2d/d35/d3d 0 2024-09-16T00:09:43.575 INFO:tasks.workunit.client.0.smithi139.stdout:4/313: mknod d12/d4a/d4d/c67 0 2024-09-16T00:09:43.575 INFO:tasks.workunit.client.0.smithi139.stdout:6/273: dwrite d8/db/d26/d28/f3f [0,4194304] 0 2024-09-16T00:09:43.576 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:1512 2.18 2:1e8917d7:::200.0000000e:head [write 0~1350 [fadvise_dontneed] in=1350b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e87)", 2024-09-16T00:09:43.576 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.575154+0000", 2024-09-16T00:09:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.001012045, 2024-09-16T00:09:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010380299999999999, 2024-09-16T00:09:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "waiting for sub ops", 2024-09-16T00:09:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.575154+0000", 2024-09-16T00:09:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.575154+0000", 2024-09-16T00:09:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.575155+0000", 2024-09-16T00:09:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6199999999999997e-07 2024-09-16T00:09:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.575161+0000", 2024-09-16T00:09:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5169999999999999e-06 2024-09-16T00:09:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.575171+0000", 2024-09-16T00:09:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0756e-05 2024-09-16T00:09:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.575177+0000", 2024-09-16T00:09:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3510000000000001e-06 2024-09-16T00:09:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.575223+0000", 2024-09-16T00:09:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6140000000000002e-05 2024-09-16T00:09:43.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.575303+0000", 2024-09-16T00:09:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0481000000000004e-05 2024-09-16T00:09:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.575445+0000", 2024-09-16T00:09:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014132000000000001 2024-09-16T00:09:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.580 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-16T00:09:43.581 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 1 2024-09-16T00:09:43.581 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:43.581 INFO:tasks.workunit.client.0.smithi139.stdout:0/374: write d6/d1c/d45/f5e [755701,69509] 0 2024-09-16T00:09:43.581 INFO:tasks.workunit.client.0.smithi139.stdout:3/372: symlink d5/d7/d23/d27/d2f/d30/d4d/l86 0 2024-09-16T00:09:43.581 INFO:tasks.workunit.client.0.smithi139.stdout:5/415: dread d1/f1a [4194304,4194304] 0 2024-09-16T00:09:43.582 INFO:tasks.workunit.client.0.smithi139.stdout:0/375: fdatasync d6/d1c/d41/d63/f66 0 2024-09-16T00:09:43.584 INFO:tasks.workunit.client.0.smithi139.stdout:2/361: dwrite d11/d30/d57/f27 [0,4194304] 0 2024-09-16T00:09:43.584 INFO:tasks.workunit.client.0.smithi139.stdout:5/416: mkdir d1/d25/d8a 0 2024-09-16T00:09:43.585 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:43.584+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:43.586 INFO:tasks.workunit.client.0.smithi139.stdout:9/392: symlink d19/d1a/d20/d23/d2d/d35/d42/l81 0 2024-09-16T00:09:43.588 INFO:tasks.workunit.client.0.smithi139.stdout:3/373: creat d5/d7/d23/d27/d2f/d30/d4c/d50/f87 x:0 0 0 2024-09-16T00:09:43.589 INFO:tasks.workunit.client.0.smithi139.stdout:5/417: write d1/d21/d49/f7f [783221,101641] 0 2024-09-16T00:09:43.590 INFO:tasks.workunit.client.0.smithi139.stdout:0/376: fsync d6/d1c/f36 0 2024-09-16T00:09:43.591 INFO:tasks.workunit.client.0.smithi139.stdout:2/362: mkdir d11/d30/d57/d5d/d73/d74 0 2024-09-16T00:09:43.591 INFO:tasks.workunit.client.0.smithi139.stdout:4/314: dwrite d12/d2b/f35 [0,4194304] 0 2024-09-16T00:09:43.594 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:09:43.595 INFO:tasks.workunit.client.0.smithi139.stdout:0/377: dread - d6/d5a/d70/f4f zero size 2024-09-16T00:09:43.595 INFO:tasks.workunit.client.0.smithi139.stdout:0/378: write d6/d5a/d70/f4f [1032758,107930] 0 2024-09-16T00:09:43.596 INFO:tasks.workunit.client.0.smithi139.stdout:3/374: truncate d5/d7/d23/d27/d2f/d30/d47/f64 4856216 0 2024-09-16T00:09:43.597 INFO:tasks.workunit.client.0.smithi139.stdout:4/315: mknod d12/d4a/d4d/c68 0 2024-09-16T00:09:43.597 INFO:tasks.workunit.client.0.smithi139.stdout:2/363: mknod d11/d30/d57/d50/c75 0 2024-09-16T00:09:43.598 INFO:tasks.workunit.client.0.smithi139.stdout:6/274: dread d8/d20/d27/d10/f13 [0,4194304] 0 2024-09-16T00:09:43.599 INFO:tasks.workunit.client.0.smithi139.stdout:2/364: truncate d11/d30/d57/f27 4990036 0 2024-09-16T00:09:43.600 INFO:tasks.workunit.client.0.smithi139.stdout:2/365: stat d11/d30 0 2024-09-16T00:09:43.601 INFO:tasks.workunit.client.0.smithi139.stdout:0/379: mkdir d6/d1c/d41/d44/dd/d56/d75 0 2024-09-16T00:09:43.601 INFO:tasks.workunit.client.0.smithi139.stdout:3/375: write d5/f16 [2959553,75035] 0 2024-09-16T00:09:43.601 INFO:tasks.workunit.client.0.smithi139.stdout:8/356: sync 2024-09-16T00:09:43.601 INFO:tasks.workunit.client.0.smithi139.stdout:4/316: mkdir d12/d4a/d4d/d69 0 2024-09-16T00:09:43.602 INFO:tasks.workunit.client.0.smithi139.stdout:9/393: link d19/d1a/d20/c7f d19/d1a/d20/d23/c82 0 2024-09-16T00:09:43.602 INFO:tasks.workunit.client.0.smithi139.stdout:6/275: truncate d8/db/d26/d45/d59/f5e 837708 0 2024-09-16T00:09:43.603 INFO:tasks.workunit.client.0.smithi139.stdout:5/418: creat d1/d21/d49/d53/d6a/f8b x:0 0 0 2024-09-16T00:09:43.604 INFO:tasks.workunit.client.0.smithi139.stdout:0/380: mkdir d6/d1c/d41/d76 0 2024-09-16T00:09:43.605 INFO:tasks.workunit.client.0.smithi139.stdout:2/366: rename d11/d30/d57/d29/c5f to d11/d2f/d6e/c76 0 2024-09-16T00:09:43.606 INFO:tasks.workunit.client.0.smithi139.stdout:4/317: symlink d12/d47/d57/l6a 0 2024-09-16T00:09:43.606 INFO:tasks.workunit.client.0.smithi139.stdout:4/318: stat d12/d4a/d4d/l64 0 2024-09-16T00:09:43.607 INFO:tasks.workunit.client.0.smithi139.stdout:3/376: mknod d5/d7/d23/d48/c88 0 2024-09-16T00:09:43.608 INFO:tasks.workunit.client.0.smithi139.stdout:5/419: chown d1/d13/d3b/f4c 7 1 2024-09-16T00:09:43.609 INFO:tasks.workunit.client.0.smithi139.stdout:4/319: dread d12/d2b/f5a [0,4194304] 0 2024-09-16T00:09:43.611 INFO:tasks.workunit.client.0.smithi139.stdout:0/381: creat d6/d1c/d41/d44/dd/d56/f77 x:0 0 0 2024-09-16T00:09:43.616 INFO:tasks.workunit.client.0.smithi139.stdout:3/377: dread d5/f53 [0,4194304] 0 2024-09-16T00:09:43.616 INFO:tasks.workunit.client.0.smithi139.stdout:5/420: creat d1/d25/d73/f8c x:0 0 0 2024-09-16T00:09:43.617 INFO:tasks.workunit.client.0.smithi139.stdout:8/357: rmdir d17/d1a/d7a 0 2024-09-16T00:09:43.619 INFO:tasks.workunit.client.0.smithi139.stdout:4/320: creat d12/d2b/f6b x:0 0 0 2024-09-16T00:09:43.619 INFO:tasks.workunit.client.0.smithi139.stdout:6/276: write d8/d20/d27/f58 [253949,38039] 0 2024-09-16T00:09:43.620 INFO:tasks.workunit.client.0.smithi139.stdout:0/382: mkdir d6/d1c/d41/d78 0 2024-09-16T00:09:43.621 INFO:tasks.workunit.client.0.smithi139.stdout:2/367: fdatasync d11/d30/d57/d29/d2b/f31 0 2024-09-16T00:09:43.624 INFO:tasks.workunit.client.0.smithi139.stdout:5/421: dwrite d1/f27 [0,4194304] 0 2024-09-16T00:09:43.625 INFO:tasks.workunit.client.0.smithi139.stdout:2/368: write d11/d30/d57/d29/d2b/f34 [1237388,68178] 0 2024-09-16T00:09:43.627 INFO:tasks.workunit.client.0.smithi139.stdout:5/422: write d1/d13/d3f/d5e/f61 [842195,43258] 0 2024-09-16T00:09:43.630 INFO:tasks.workunit.client.0.smithi139.stdout:9/394: symlink d19/d1a/d20/d23/d2d/d35/d42/l83 0 2024-09-16T00:09:43.632 INFO:tasks.workunit.client.0.smithi139.stdout:0/383: dread d6/d1c/d41/d44/f29 [0,4194304] 0 2024-09-16T00:09:43.633 INFO:tasks.workunit.client.0.smithi139.stdout:5/423: mkdir d1/d25/d73/d79/d8d 0 2024-09-16T00:09:43.634 INFO:tasks.workunit.client.0.smithi139.stdout:8/358: truncate d17/d1a/f53 1130298 0 2024-09-16T00:09:43.635 INFO:tasks.workunit.client.0.smithi139.stdout:4/321: dwrite d12/d2b/f30 [0,4194304] 0 2024-09-16T00:09:43.635 INFO:tasks.workunit.client.0.smithi139.stdout:5/424: stat d1/d13/d3b 0 2024-09-16T00:09:43.635 INFO:tasks.workunit.client.0.smithi139.stdout:6/277: rename d8/d20/d27/f1c to d8/db/d26/f61 0 2024-09-16T00:09:43.636 INFO:tasks.workunit.client.0.smithi139.stdout:0/384: write d6/d1c/f61 [847961,107766] 0 2024-09-16T00:09:43.636 INFO:tasks.workunit.client.0.smithi139.stdout:3/378: link d5/d7/d23/d27/d2f/c34 d5/d7/dc/d84/c89 0 2024-09-16T00:09:43.637 INFO:tasks.workunit.client.0.smithi139.stdout:3/379: chown d5/d7 1753791 1 2024-09-16T00:09:43.637 INFO:tasks.workunit.client.0.smithi139.stdout:2/369: dread d11/d30/d57/d29/d2b/f65 [0,4194304] 0 2024-09-16T00:09:43.638 INFO:tasks.workunit.client.0.smithi139.stdout:4/322: creat d12/d47/f6c x:0 0 0 2024-09-16T00:09:43.639 INFO:tasks.workunit.client.0.smithi139.stdout:8/359: dread - d17/d1a/d43/d40/d4f/f75 zero size 2024-09-16T00:09:43.640 INFO:tasks.workunit.client.0.smithi139.stdout:2/370: mknod d11/d32/c77 0 2024-09-16T00:09:43.642 INFO:tasks.workunit.client.0.smithi139.stdout:2/371: read - d11/d2f/f61 zero size 2024-09-16T00:09:43.643 INFO:tasks.workunit.client.0.smithi139.stdout:8/360: creat d17/d1a/d57/f7b x:0 0 0 2024-09-16T00:09:43.643 INFO:tasks.workunit.client.0.smithi139.stdout:0/385: mknod d6/d1c/d41/d44/dd/d56/d75/c79 0 2024-09-16T00:09:43.644 INFO:tasks.workunit.client.0.smithi139.stdout:5/425: dread d1/f4a [0,4194304] 0 2024-09-16T00:09:43.645 INFO:tasks.workunit.client.0.smithi139.stdout:6/278: dwrite d8/d20/f4a [0,4194304] 0 2024-09-16T00:09:43.656 INFO:tasks.workunit.client.0.smithi139.stdout:6/279: dread d8/db/d26/f61 [0,4194304] 0 2024-09-16T00:09:43.660 INFO:tasks.workunit.client.0.smithi139.stdout:0/386: mkdir d6/d1c/d45/d7a 0 2024-09-16T00:09:43.663 INFO:tasks.workunit.client.0.smithi139.stdout:8/361: mknod d17/d1a/d43/d40/d4f/c7c 0 2024-09-16T00:09:43.663 INFO:tasks.workunit.client.0.smithi139.stdout:8/362: read - d17/d54/f6e zero size 2024-09-16T00:09:43.666 INFO:tasks.workunit.client.0.smithi139.stdout:4/323: getdents d12/d2b 0 2024-09-16T00:09:43.668 INFO:tasks.workunit.client.0.smithi139.stdout:6/280: creat d8/d20/f62 x:0 0 0 2024-09-16T00:09:43.672 INFO:tasks.workunit.client.0.smithi139.stdout:8/363: mknod d17/d1a/d43/d40/c7d 0 2024-09-16T00:09:43.674 INFO:tasks.workunit.client.0.smithi139.stdout:4/324: symlink d12/d4a/l6d 0 2024-09-16T00:09:43.679 INFO:tasks.workunit.client.0.smithi139.stdout:4/325: dwrite d12/d4a/f63 [0,4194304] 0 2024-09-16T00:09:43.680 INFO:tasks.workunit.client.0.smithi139.stdout:8/364: symlink d17/d1a/d57/l7e 0 2024-09-16T00:09:43.680 INFO:tasks.workunit.client.0.smithi139.stdout:5/426: creat d1/d21/d49/d53/d6a/d7b/d78/f8e x:0 0 0 2024-09-16T00:09:43.682 INFO:tasks.workunit.client.0.smithi139.stdout:8/365: chown d17/d1a/d43/d40/f41 16 1 2024-09-16T00:09:43.684 INFO:tasks.workunit.client.0.smithi139.stdout:6/281: dread d8/d20/d27/d10/f3b [0,4194304] 0 2024-09-16T00:09:43.686 INFO:tasks.workunit.client.0.smithi139.stdout:5/427: symlink d1/d21/d49/d53/d6a/d7b/l8f 0 2024-09-16T00:09:43.686 INFO:tasks.workunit.client.0.smithi139.stdout:8/366: write d17/d1a/f73 [1048166,30969] 0 2024-09-16T00:09:43.686 INFO:tasks.workunit.client.0.smithi139.stdout:6/282: chown d8/db 9238301 1 2024-09-16T00:09:43.687 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:43.685+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:43.687 INFO:tasks.workunit.client.0.smithi139.stdout:0/387: dwrite d6/d1c/d41/d44/f3a [0,4194304] 0 2024-09-16T00:09:43.689 INFO:tasks.workunit.client.0.smithi139.stdout:5/428: symlink d1/d13/d3f/l90 0 2024-09-16T00:09:43.690 INFO:tasks.workunit.client.0.smithi139.stdout:0/388: chown d6/d1c/d41/d44/dd/c26 6773 1 2024-09-16T00:09:43.691 INFO:tasks.workunit.client.0.smithi139.stdout:3/380: creat d5/d7/dc/f8a x:0 0 0 2024-09-16T00:09:43.691 INFO:tasks.workunit.client.0.smithi139.stdout:4/326: dwrite d12/d2b/f30 [4194304,4194304] 0 2024-09-16T00:09:43.693 INFO:tasks.workunit.client.0.smithi139.stdout:8/367: symlink d17/d1a/d43/d40/d4f/l7f 0 2024-09-16T00:09:43.693 INFO:tasks.workunit.client.0.smithi139.stdout:6/283: symlink d8/db/d26/d45/d42/l63 0 2024-09-16T00:09:43.697 INFO:tasks.workunit.client.0.smithi139.stdout:0/389: unlink d6/d5a/l62 0 2024-09-16T00:09:43.699 INFO:tasks.workunit.client.0.smithi139.stdout:4/327: creat d12/d47/d57/f6e x:0 0 0 2024-09-16T00:09:43.699 INFO:tasks.workunit.client.0.smithi139.stdout:2/372: dwrite d11/f16 [0,4194304] 0 2024-09-16T00:09:43.701 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.17.6 2024-09-16T00:09:43.702 INFO:tasks.workunit.client.0.smithi139.stdout:3/381: creat d5/d7/d23/d27/d2f/d30/d47/f8b x:0 0 0 2024-09-16T00:09:43.707 INFO:tasks.workunit.client.0.smithi139.stdout:0/390: mkdir d6/d5a/d70/d7b 0 2024-09-16T00:09:43.707 INFO:tasks.workunit.client.0.smithi139.stdout:4/328: dwrite d12/f4c [0,4194304] 0 2024-09-16T00:09:43.709 INFO:tasks.workunit.client.0.smithi139.stdout:2/373: rename d11/c25 to d11/d30/d57/c78 0 2024-09-16T00:09:43.711 INFO:tasks.workunit.client.0.smithi139.stdout:3/382: creat d5/d7/d23/d27/d2f/d30/d4d/f8c x:0 0 0 2024-09-16T00:09:43.714 INFO:tasks.workunit.client.0.smithi139.stdout:6/284: fsync d8/d20/d4d/f60 0 2024-09-16T00:09:43.717 INFO:tasks.workunit.client.0.smithi139.stdout:4/329: dwrite f2 [4194304,4194304] 0 2024-09-16T00:09:43.719 INFO:tasks.workunit.client.0.smithi139.stdout:0/391: dwrite d6/d1c/d45/f5e [0,4194304] 0 2024-09-16T00:09:43.722 INFO:tasks.workunit.client.0.smithi139.stdout:8/368: getdents d17/d1a/d50 0 2024-09-16T00:09:43.722 INFO:tasks.workunit.client.0.smithi139.stdout:2/374: dwrite d11/d30/d57/d29/f5c [0,4194304] 0 2024-09-16T00:09:43.723 INFO:tasks.workunit.client.0.smithi139.stdout:6/285: creat d8/d20/f64 x:0 0 0 2024-09-16T00:09:43.725 INFO:tasks.workunit.client.0.smithi139.stdout:6/286: write d8/db/f15 [1101366,34050] 0 2024-09-16T00:09:43.729 INFO:tasks.workunit.client.0.smithi139.stdout:0/392: rename d6/d1c/d45/d7a to d6/d1c/d41/d7c 0 2024-09-16T00:09:43.730 INFO:tasks.workunit.client.0.smithi139.stdout:0/393: chown d6/d1c/d41/d44/dd/f46 44720401 1 2024-09-16T00:09:43.732 INFO:tasks.workunit.client.0.smithi139.stdout:8/369: creat d17/d54/f80 x:0 0 0 2024-09-16T00:09:43.732 INFO:tasks.workunit.client.0.smithi139.stdout:2/375: rmdir d11 39 2024-09-16T00:09:43.735 INFO:tasks.workunit.client.0.smithi139.stdout:4/330: truncate d12/f43 1105818 0 2024-09-16T00:09:43.735 INFO:tasks.workunit.client.0.smithi139.stdout:4/331: write d12/f4c [3477518,12004] 0 2024-09-16T00:09:43.736 INFO:tasks.workunit.client.0.smithi139.stdout:3/383: rmdir d5/d7/d23/d27/d2f/d30/d4c/d6f 0 2024-09-16T00:09:43.737 INFO:tasks.workunit.client.0.smithi139.stdout:3/384: readlink d5/d7/d23/d27/d2f/d30/d4d/l86 0 2024-09-16T00:09:43.739 INFO:tasks.workunit.client.0.smithi139.stdout:8/370: rename d17/d1a/f1e to d17/d1f/f81 0 2024-09-16T00:09:43.750 INFO:tasks.workunit.client.0.smithi139.stdout:8/371: getdents d17/d1a/d43/d40 0 2024-09-16T00:09:43.750 INFO:tasks.workunit.client.0.smithi139.stdout:8/372: readlink d17/d1a/d43/d40/d4f/l5c 0 2024-09-16T00:09:43.751 INFO:tasks.workunit.client.0.smithi139.stdout:8/373: write d17/d54/f55 [1163713,21274] 0 2024-09-16T00:09:43.752 INFO:tasks.workunit.client.0.smithi139.stdout:8/374: readlink d17/d1a/l26 0 2024-09-16T00:09:43.757 INFO:tasks.workunit.client.0.smithi139.stdout:8/375: dwrite d17/d54/f55 [4194304,4194304] 0 2024-09-16T00:09:43.759 INFO:tasks.workunit.client.0.smithi139.stdout:4/332: write d12/d2b/f5a [368227,24479] 0 2024-09-16T00:09:43.764 INFO:tasks.workunit.client.0.smithi139.stdout:4/333: write d12/d47/d57/f59 [5118703,9544] 0 2024-09-16T00:09:43.765 INFO:tasks.workunit.client.0.smithi139.stdout:2/376: dwrite fc [0,4194304] 0 2024-09-16T00:09:43.766 INFO:tasks.workunit.client.0.smithi139.stdout:3/385: dwrite d5/d7/d23/d27/d2f/d30/d4c/d50/f59 [0,4194304] 0 2024-09-16T00:09:43.767 INFO:tasks.workunit.client.0.smithi139.stdout:4/334: fsync d12/f31 0 2024-09-16T00:09:43.769 INFO:tasks.workunit.client.0.smithi139.stdout:2/377: chown d11/l3f 895346625 1 2024-09-16T00:09:43.774 INFO:tasks.workunit.client.0.smithi139.stdout:2/378: truncate d11/d30/d57/d29/d2b/f31 548591 0 2024-09-16T00:09:43.775 INFO:tasks.workunit.client.0.smithi139.stdout:3/386: dwrite d5/f8 [0,4194304] 0 2024-09-16T00:09:43.776 INFO:tasks.workunit.client.0.smithi139.stdout:7/340: sync 2024-09-16T00:09:43.776 INFO:tasks.workunit.client.0.smithi139.stdout:1/381: sync 2024-09-16T00:09:43.778 INFO:tasks.workunit.client.0.smithi139.stdout:7/341: write d5/d16/f34 [2040366,43170] 0 2024-09-16T00:09:43.778 INFO:tasks.workunit.client.0.smithi139.stdout:2/379: creat d11/d30/d57/d5d/d73/f79 x:0 0 0 2024-09-16T00:09:43.778 INFO:tasks.workunit.client.0.smithi139.stdout:1/382: dread - d7/d8/d40/d46/f70 zero size 2024-09-16T00:09:43.779 INFO:tasks.workunit.client.0.smithi139.stdout:3/387: creat d5/d7/d23/d27/f8d x:0 0 0 2024-09-16T00:09:43.781 INFO:tasks.workunit.client.0.smithi139.stdout:2/380: truncate fe 2408687 0 2024-09-16T00:09:43.786 INFO:tasks.workunit.client.0.smithi139.stdout:1/383: mkdir d7/d8/d40/d48/d4a/d74 0 2024-09-16T00:09:43.786 INFO:tasks.workunit.client.0.smithi139.stdout:8/376: dread d17/d1a/d2d/f3e [0,4194304] 0 2024-09-16T00:09:43.786 INFO:tasks.workunit.client.0.smithi139.stdout:3/388: stat d5/d7/d23/d27/d2f/d30/d4d/f60 0 2024-09-16T00:09:43.786 INFO:tasks.workunit.client.0.smithi139.stdout:7/342: mknod d5/d26/d5d/c78 0 2024-09-16T00:09:43.786 INFO:tasks.workunit.client.0.smithi139.stdout:1/384: readlink d7/d8/d40/d46/l5f 0 2024-09-16T00:09:43.786 INFO:tasks.workunit.client.0.smithi139.stdout:7/343: write d5/d26/f2b [3222718,53821] 0 2024-09-16T00:09:43.787 INFO:tasks.workunit.client.0.smithi139.stdout:8/377: chown d17/d1a/d2d/f3e 7489 1 2024-09-16T00:09:43.787 INFO:tasks.workunit.client.0.smithi139.stdout:3/389: mknod d5/d7/d23/d27/c8e 0 2024-09-16T00:09:43.787 INFO:tasks.workunit.client.0.smithi139.stdout:2/381: getdents d11/d2f/d6e 0 2024-09-16T00:09:43.787 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:43.787 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:43.788 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:43.788 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:43.788 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:43.788 INFO:tasks.workunit.client.0.smithi139.stdout:7/344: chown d5/d16/d22/d25/d2d/d50/l64 3145741 1 2024-09-16T00:09:43.788 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:43.787+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:43.789 INFO:tasks.workunit.client.0.smithi139.stdout:1/385: read d7/f13 [151592,22381] 0 2024-09-16T00:09:43.789 INFO:tasks.workunit.client.0.smithi139.stdout:8/378: creat d17/d1a/d57/f82 x:0 0 0 2024-09-16T00:09:43.789 INFO:tasks.workunit.client.0.smithi139.stdout:8/379: dread - d17/d54/f69 zero size 2024-09-16T00:09:43.789 INFO:tasks.workunit.client.0.smithi139.stdout:7/345: write d5/d26/d46/f68 [810029,85280] 0 2024-09-16T00:09:43.789 INFO:tasks.workunit.client.0.smithi139.stdout:1/386: dread - d7/d8/d40/d48/f65 zero size 2024-09-16T00:09:43.790 INFO:tasks.workunit.client.0.smithi139.stdout:2/382: creat d11/d30/d57/d29/d4f/f7a x:0 0 0 2024-09-16T00:09:43.791 INFO:tasks.workunit.client.0.smithi139.stdout:3/390: creat d5/d7/d23/f8f x:0 0 0 2024-09-16T00:09:43.791 INFO:tasks.workunit.client.0.smithi139.stdout:2/383: dread - d11/d2f/f61 zero size 2024-09-16T00:09:43.793 INFO:tasks.workunit.client.0.smithi139.stdout:2/384: mknod d11/c7b 0 2024-09-16T00:09:43.794 INFO:tasks.workunit.client.0.smithi139.stdout:1/387: creat d7/d8/d27/f75 x:0 0 0 2024-09-16T00:09:43.794 INFO:tasks.workunit.client.0.smithi139.stdout:2/385: chown d11/c7b 6616 1 2024-09-16T00:09:43.794 INFO:tasks.workunit.client.0.smithi139.stdout:3/391: symlink d5/d7/d23/d27/d2f/d30/d47/l90 0 2024-09-16T00:09:43.795 INFO:tasks.workunit.client.0.smithi139.stdout:8/380: dread d17/d1a/d43/f42 [0,4194304] 0 2024-09-16T00:09:43.799 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:09:43.815 INFO:tasks.workunit.client.0.smithi139.stdout:8/381: rmdir d17/d1a/d50 39 2024-09-16T00:09:43.815 INFO:tasks.workunit.client.0.smithi139.stdout:3/392: read d5/d7/d23/f17 [4157162,95682] 0 2024-09-16T00:09:43.815 INFO:tasks.workunit.client.0.smithi139.stdout:2/386: link d11/f26 d11/d45/f7c 0 2024-09-16T00:09:43.815 INFO:tasks.workunit.client.0.smithi139.stdout:7/346: dread d5/d26/f32 [0,4194304] 0 2024-09-16T00:09:43.816 INFO:tasks.workunit.client.0.smithi139.stdout:1/388: read d7/f13 [1995034,35828] 0 2024-09-16T00:09:43.817 INFO:tasks.workunit.client.0.smithi139.stdout:3/393: rename d5 to d5/d7/d23/d91 22 2024-09-16T00:09:43.817 INFO:tasks.workunit.client.0.smithi139.stdout:2/387: symlink d11/d30/l7d 0 2024-09-16T00:09:43.818 INFO:tasks.workunit.client.0.smithi139.stdout:1/389: truncate d7/d8/d27/f59 248589 0 2024-09-16T00:09:43.818 INFO:tasks.workunit.client.0.smithi139.stdout:7/347: getdents d5/d26/d74 0 2024-09-16T00:09:43.821 INFO:tasks.workunit.client.0.smithi139.stdout:7/348: creat d5/d37/f79 x:0 0 0 2024-09-16T00:09:43.821 INFO:tasks.workunit.client.0.smithi139.stdout:2/388: read d11/d30/d57/d29/d2b/f34 [1060759,45512] 0 2024-09-16T00:09:43.825 INFO:tasks.workunit.client.0.smithi139.stdout:1/390: rename d7/d8/d40/d48/d4a/f4b to d7/d8/d1f/f76 0 2024-09-16T00:09:43.825 INFO:tasks.workunit.client.0.smithi139.stdout:8/382: dwrite d17/d1a/d43/f6c [0,4194304] 0 2024-09-16T00:09:43.826 INFO:tasks.workunit.client.0.smithi139.stdout:7/349: creat d5/d16/d22/d25/f7a x:0 0 0 2024-09-16T00:09:43.827 INFO:tasks.workunit.client.0.smithi139.stdout:8/383: readlink d17/d1a/d43/l23 0 2024-09-16T00:09:43.827 INFO:tasks.workunit.client.0.smithi139.stdout:3/394: dwrite d5/d7/f28 [0,4194304] 0 2024-09-16T00:09:43.828 INFO:tasks.workunit.client.0.smithi139.stdout:2/389: mknod d11/d2f/d6e/c7e 0 2024-09-16T00:09:43.831 INFO:tasks.workunit.client.0.smithi139.stdout:8/384: unlink d17/d1a/d57/f6a 0 2024-09-16T00:09:43.833 INFO:tasks.workunit.client.0.smithi139.stdout:2/390: mkdir d11/d2f/d6e/d7f 0 2024-09-16T00:09:43.835 INFO:tasks.workunit.client.0.smithi139.stdout:1/391: dwrite d7/d8/f6f [0,4194304] 0 2024-09-16T00:09:43.837 INFO:tasks.workunit.client.0.smithi139.stdout:7/350: write d5/d37/f47 [327482,75725] 0 2024-09-16T00:09:43.840 INFO:tasks.workunit.client.0.smithi139.stdout:3/395: dread d5/d7/d23/d27/d2f/d30/d47/f25 [0,4194304] 0 2024-09-16T00:09:43.840 INFO:tasks.workunit.client.0.smithi139.stdout:7/351: write d5/d16/d22/d25/d2d/d50/d44/d55/f6c [587105,2538] 0 2024-09-16T00:09:43.840 INFO:tasks.workunit.client.0.smithi139.stdout:2/391: creat d11/d2f/d6e/f80 x:0 0 0 2024-09-16T00:09:43.841 INFO:tasks.workunit.client.0.smithi139.stdout:7/352: write d5/f59 [155185,10147] 0 2024-09-16T00:09:43.842 INFO:tasks.workunit.client.0.smithi139.stdout:8/385: getdents d17/d1a/d43/d40 0 2024-09-16T00:09:43.843 INFO:tasks.workunit.client.0.smithi139.stdout:3/396: dread - d5/d7/d23/d27/d2f/d30/d4c/f69 zero size 2024-09-16T00:09:43.844 INFO:tasks.workunit.client.0.smithi139.stdout:2/392: chown d11/d30/d57/d29/d2b/f31 1828355 1 2024-09-16T00:09:43.845 INFO:tasks.workunit.client.0.smithi139.stdout:1/392: getdents d7/d8/d40/d47 0 2024-09-16T00:09:43.846 INFO:tasks.workunit.client.0.smithi139.stdout:8/386: chown d17/c5b 346 1 2024-09-16T00:09:43.846 INFO:tasks.workunit.client.0.smithi139.stdout:7/353: creat d5/d37/f7b x:0 0 0 2024-09-16T00:09:43.847 INFO:tasks.workunit.client.0.smithi139.stdout:3/397: mkdir d5/d7/d23/d27/d2f/d30/d4c/d92 0 2024-09-16T00:09:43.848 INFO:tasks.workunit.client.0.smithi139.stdout:3/398: read - d5/d7/dc/f8a zero size 2024-09-16T00:09:43.850 INFO:tasks.workunit.client.0.smithi139.stdout:8/387: creat d17/f83 x:0 0 0 2024-09-16T00:09:43.851 INFO:tasks.workunit.client.0.smithi139.stdout:2/393: getdents d11/d45 0 2024-09-16T00:09:43.852 INFO:tasks.workunit.client.0.smithi139.stdout:2/394: dread - d11/f41 zero size 2024-09-16T00:09:43.852 INFO:tasks.workunit.client.0.smithi139.stdout:3/399: mknod d5/d7/d23/c93 0 2024-09-16T00:09:43.853 INFO:tasks.workunit.client.0.smithi139.stdout:1/393: truncate d7/d8/d1f/f5d 5229770 0 2024-09-16T00:09:43.853 INFO:tasks.workunit.client.0.smithi139.stdout:1/394: chown d7/d8/d27/c2d 3786 1 2024-09-16T00:09:43.854 INFO:tasks.workunit.client.0.smithi139.stdout:8/388: rename d17/d1a/d2d/c30 to d17/d1a/d43/c84 0 2024-09-16T00:09:43.857 INFO:tasks.workunit.client.0.smithi139.stdout:8/389: truncate d17/d1a/d43/f2b 5217175 0 2024-09-16T00:09:43.858 INFO:tasks.workunit.client.0.smithi139.stdout:1/395: symlink d7/d8/d40/d48/d4a/d74/l77 0 2024-09-16T00:09:43.858 INFO:tasks.workunit.client.0.smithi139.stdout:1/396: readlink d7/d8/d40/l62 0 2024-09-16T00:09:43.859 INFO:tasks.workunit.client.0.smithi139.stdout:2/395: dwrite d11/f1a [0,4194304] 0 2024-09-16T00:09:43.860 INFO:tasks.workunit.client.0.smithi139.stdout:7/354: dread d5/d16/d22/d25/f30 [0,4194304] 0 2024-09-16T00:09:43.862 INFO:tasks.workunit.client.0.smithi139.stdout:1/397: creat d7/d8/d40/d47/f78 x:0 0 0 2024-09-16T00:09:43.863 INFO:tasks.workunit.client.0.smithi139.stdout:7/355: creat d5/d16/d6d/f7c x:0 0 0 2024-09-16T00:09:43.864 INFO:tasks.workunit.client.0.smithi139.stdout:7/356: chown d5/d26/d5d/l5e 126650011 1 2024-09-16T00:09:43.867 INFO:tasks.workunit.client.0.smithi139.stdout:7/357: mknod d5/d26/d42/c7d 0 2024-09-16T00:09:43.871 INFO:tasks.workunit.client.0.smithi139.stdout:1/398: link d7/d8/f1e d7/d8/d27/f79 0 2024-09-16T00:09:43.871 INFO:tasks.workunit.client.0.smithi139.stdout:2/396: write f2 [1407843,130167] 0 2024-09-16T00:09:43.873 INFO:tasks.workunit.client.0.smithi139.stdout:7/358: symlink d5/l7e 0 2024-09-16T00:09:43.873 INFO:tasks.workunit.client.0.smithi139.stdout:8/390: dread d17/d1a/f1d [0,4194304] 0 2024-09-16T00:09:43.878 INFO:tasks.workunit.client.0.smithi139.stdout:2/397: dread d11/d30/f6c [0,4194304] 0 2024-09-16T00:09:43.880 INFO:tasks.workunit.client.0.smithi139.stdout:8/391: getdents d17/d1a/d57 0 2024-09-16T00:09:43.880 INFO:tasks.workunit.client.0.smithi139.stdout:7/359: dwrite d5/f1f [4194304,4194304] 0 2024-09-16T00:09:43.882 INFO:tasks.workunit.client.0.smithi139.stdout:7/360: write d5/d26/d42/f5b [455131,9562] 0 2024-09-16T00:09:43.886 INFO:tasks.workunit.client.0.smithi139.stdout:2/398: dread d11/d30/d57/f27 [0,4194304] 0 2024-09-16T00:09:43.887 INFO:tasks.workunit.client.0.smithi139.stdout:8/392: dwrite d17/d1a/d57/f82 [0,4194304] 0 2024-09-16T00:09:43.888 INFO:tasks.workunit.client.0.smithi139.stdout:8/393: chown d17/d1a/d2d 2757772 1 2024-09-16T00:09:43.889 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:43.887+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:43.891 INFO:tasks.workunit.client.0.smithi139.stdout:1/399: dread d7/d8/d27/f2c [0,4194304] 0 2024-09-16T00:09:43.895 INFO:tasks.workunit.client.0.smithi139.stdout:2/399: mkdir d11/d30/d57/d5d/d73/d74/d81 0 2024-09-16T00:09:43.897 INFO:tasks.workunit.client.0.smithi139.stdout:7/361: truncate d5/d16/d22/d25/d2d/d50/d44/d55/f6c 2473481 0 2024-09-16T00:09:43.898 INFO:tasks.workunit.client.0.smithi139.stdout:2/400: unlink d11/d30/f6c 0 2024-09-16T00:09:43.898 INFO:tasks.workunit.client.0.smithi139.stdout:1/400: creat d7/d8/d27/d68/d6c/f7a x:0 0 0 2024-09-16T00:09:43.899 INFO:tasks.workunit.client.0.smithi139.stdout:7/362: mkdir d5/d16/d6d/d7f 0 2024-09-16T00:09:43.902 INFO:tasks.workunit.client.0.smithi139.stdout:7/363: creat d5/d26/d42/f80 x:0 0 0 2024-09-16T00:09:43.902 INFO:tasks.workunit.client.0.smithi139.stdout:7/364: chown d5/d16/d22/d25/f76 40 1 2024-09-16T00:09:43.903 INFO:tasks.workunit.client.0.smithi139.stdout:2/401: truncate d11/d30/d57/d50/f69 1146612 0 2024-09-16T00:09:43.904 INFO:tasks.workunit.client.0.smithi139.stdout:7/365: fdatasync d5/f6 0 2024-09-16T00:09:43.905 INFO:tasks.workunit.client.0.smithi139.stdout:2/402: symlink d11/d30/d57/d29/d4f/l82 0 2024-09-16T00:09:43.906 INFO:tasks.workunit.client.0.smithi139.stdout:8/394: dread d17/d1a/d2d/f2e [0,4194304] 0 2024-09-16T00:09:43.907 INFO:tasks.workunit.client.0.smithi139.stdout:7/366: mkdir d5/d16/d22/d25/d2d/d81 0 2024-09-16T00:09:43.908 INFO:tasks.workunit.client.0.smithi139.stdout:2/403: creat d11/d30/d57/f83 x:0 0 0 2024-09-16T00:09:43.908 INFO:tasks.workunit.client.0.smithi139.stdout:2/404: dread - d11/d30/d57/f59 zero size 2024-09-16T00:09:43.934 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:43.934 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:43.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:43.934 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:43.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.934 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10658 3.1c 3:38c4c24e:::100000030bb.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:43.934 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:09:43.935 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.1400558069999995, 2024-09-16T00:09:43.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021232445999999999, 2024-09-16T00:09:43.935 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.935 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.935 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.935 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.935 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.935 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.935 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10658 2024-09-16T00:09:43.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.936 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:09:43.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:09:43.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791398+0000", 2024-09-16T00:09:43.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3880000000000001e-06 2024-09-16T00:09:43.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800951+0000", 2024-09-16T00:09:43.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095532110000000007 2024-09-16T00:09:43.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800966+0000", 2024-09-16T00:09:43.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5058999999999999e-05 2024-09-16T00:09:43.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800974+0000", 2024-09-16T00:09:43.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1240000000000005e-06 2024-09-16T00:09:43.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801127+0000", 2024-09-16T00:09:43.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000153103 2024-09-16T00:09:43.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801207+0000", 2024-09-16T00:09:43.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9569999999999999e-05 2024-09-16T00:09:43.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:43.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801322+0000", 2024-09-16T00:09:43.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115358 2024-09-16T00:09:43.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.805760+0000", 2024-09-16T00:09:43.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044379409999999999 2024-09-16T00:09:43.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812528+0000", 2024-09-16T00:09:43.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067678249999999999 2024-09-16T00:09:43.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812586+0000", 2024-09-16T00:09:43.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8164000000000001e-05 2024-09-16T00:09:43.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812629+0000", 2024-09-16T00:09:43.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2703000000000001e-05 2024-09-16T00:09:43.943 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.943 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.943 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.943 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10674 3.1c 3:38022ffc:::100000030c2.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:43.943 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.857246+0000", 2024-09-16T00:09:43.943 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.0742059249999993, 2024-09-16T00:09:43.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013541578, 2024-09-16T00:09:43.944 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.944 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.944 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.944 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10674 2024-09-16T00:09:43.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.944 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.857246+0000", 2024-09-16T00:09:43.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.857246+0000", 2024-09-16T00:09:43.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.857247+0000", 2024-09-16T00:09:43.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2669999999999999e-06 2024-09-16T00:09:43.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.863159+0000", 2024-09-16T00:09:43.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005911085 2024-09-16T00:09:43.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.863173+0000", 2024-09-16T00:09:43.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4651000000000001e-05 2024-09-16T00:09:43.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.863181+0000", 2024-09-16T00:09:43.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2959999999999999e-06 2024-09-16T00:09:43.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.863243+0000", 2024-09-16T00:09:43.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1969000000000002e-05 2024-09-16T00:09:43.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.863330+0000", 2024-09-16T00:09:43.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6971999999999993e-05 2024-09-16T00:09:43.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:43.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.863471+0000", 2024-09-16T00:09:43.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014018599999999999 2024-09-16T00:09:43.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.869372+0000", 2024-09-16T00:09:43.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059012989999999996 2024-09-16T00:09:43.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.870723+0000", 2024-09-16T00:09:43.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001350672 2024-09-16T00:09:43.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.870758+0000", 2024-09-16T00:09:43.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5207999999999999e-05 2024-09-16T00:09:43.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.870788+0000", 2024-09-16T00:09:43.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9972999999999999e-05 2024-09-16T00:09:43.952 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.952 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.952 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.952 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10790 3.5 3:a3c82b2e:::10000003104.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:43.952 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.526361+0000", 2024-09-16T00:09:43.952 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.405091112, 2024-09-16T00:09:43.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013459364999999999, 2024-09-16T00:09:43.953 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.953 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.953 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.953 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10790 2024-09-16T00:09:43.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.953 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.526361+0000", 2024-09-16T00:09:43.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.526361+0000", 2024-09-16T00:09:43.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.526367+0000", 2024-09-16T00:09:43.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.336e-06 2024-09-16T00:09:43.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.530600+0000", 2024-09-16T00:09:43.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004233161 2024-09-16T00:09:43.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.530620+0000", 2024-09-16T00:09:43.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9065000000000001e-05 2024-09-16T00:09:43.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.530629+0000", 2024-09-16T00:09:43.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9789999999999999e-06 2024-09-16T00:09:43.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.530693+0000", 2024-09-16T00:09:43.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4517000000000006e-05 2024-09-16T00:09:43.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.530802+0000", 2024-09-16T00:09:43.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108671 2024-09-16T00:09:43.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:43.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.530894+0000", 2024-09-16T00:09:43.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2181000000000004e-05 2024-09-16T00:09:43.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.534179+0000", 2024-09-16T00:09:43.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032855100000000002 2024-09-16T00:09:43.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.539756+0000", 2024-09-16T00:09:43.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055762609999999999 2024-09-16T00:09:43.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.539790+0000", 2024-09-16T00:09:43.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4249999999999999e-05 2024-09-16T00:09:43.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.539820+0000", 2024-09-16T00:09:43.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0434e-05 2024-09-16T00:09:43.959 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.959 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.959 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.960 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10827 3.1c 3:38022ffc:::100000030c2.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:43.960 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.744026+0000", 2024-09-16T00:09:43.960 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.1874261810000002, 2024-09-16T00:09:43.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013763638, 2024-09-16T00:09:43.960 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.960 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.960 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.960 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.960 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.961 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.961 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10827 2024-09-16T00:09:43.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.961 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.744026+0000", 2024-09-16T00:09:43.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.744026+0000", 2024-09-16T00:09:43.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.744028+0000", 2024-09-16T00:09:43.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8020000000000001e-06 2024-09-16T00:09:43.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.749946+0000", 2024-09-16T00:09:43.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059182410000000003 2024-09-16T00:09:43.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.749969+0000", 2024-09-16T00:09:43.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2881999999999999e-05 2024-09-16T00:09:43.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.749980+0000", 2024-09-16T00:09:43.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1552000000000001e-05 2024-09-16T00:09:43.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.750057+0000", 2024-09-16T00:09:43.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6471999999999996e-05 2024-09-16T00:09:43.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.750159+0000", 2024-09-16T00:09:43.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010180500000000001 2024-09-16T00:09:43.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:43.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.750320+0000", 2024-09-16T00:09:43.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016095700000000001 2024-09-16T00:09:43.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.755446+0000", 2024-09-16T00:09:43.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051264709999999996 2024-09-16T00:09:43.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.757688+0000", 2024-09-16T00:09:43.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022419670000000001 2024-09-16T00:09:43.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.757748+0000", 2024-09-16T00:09:43.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0197e-05 2024-09-16T00:09:43.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.757790+0000", 2024-09-16T00:09:43.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1291999999999997e-05 2024-09-16T00:09:43.966 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.966 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.966 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.966 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10954 3.3c 3:3c2be35d:::100000030f7.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:43.966 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.509020+0000", 2024-09-16T00:09:43.966 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4224321460000002, 2024-09-16T00:09:43.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016515911000000001, 2024-09-16T00:09:43.966 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.966 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.966 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.966 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.966 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.967 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10954 2024-09-16T00:09:43.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.967 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.509020+0000", 2024-09-16T00:09:43.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.509020+0000", 2024-09-16T00:09:43.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.509022+0000", 2024-09-16T00:09:43.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.187e-06 2024-09-16T00:09:43.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513579+0000", 2024-09-16T00:09:43.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045566269999999997 2024-09-16T00:09:43.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513602+0000", 2024-09-16T00:09:43.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3617000000000001e-05 2024-09-16T00:09:43.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513616+0000", 2024-09-16T00:09:43.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.343e-05 2024-09-16T00:09:43.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513699+0000", 2024-09-16T00:09:43.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2719999999999994e-05 2024-09-16T00:09:43.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513821+0000", 2024-09-16T00:09:43.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001228 2024-09-16T00:09:43.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:43.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513987+0000", 2024-09-16T00:09:43.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016540200000000001 2024-09-16T00:09:43.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.520104+0000", 2024-09-16T00:09:43.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061177280000000002 2024-09-16T00:09:43.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.525469+0000", 2024-09-16T00:09:43.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053645730000000001 2024-09-16T00:09:43.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.525503+0000", 2024-09-16T00:09:43.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4378000000000002e-05 2024-09-16T00:09:43.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.525536+0000", 2024-09-16T00:09:43.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2449000000000001e-05 2024-09-16T00:09:43.972 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.972 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.972 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.972 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10956 3.30 3:0cbee1e3:::10000003173.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:43.972 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.513819+0000", 2024-09-16T00:09:43.972 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4176329479999996, 2024-09-16T00:09:43.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015930732, 2024-09-16T00:09:43.972 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.972 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.972 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.972 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.973 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.973 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.973 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10956 2024-09-16T00:09:43.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.973 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513819+0000", 2024-09-16T00:09:43.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513819+0000", 2024-09-16T00:09:43.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.513821+0000", 2024-09-16T00:09:43.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.677e-06 2024-09-16T00:09:43.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.518999+0000", 2024-09-16T00:09:43.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051783489999999996 2024-09-16T00:09:43.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.519020+0000", 2024-09-16T00:09:43.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0489999999999999e-05 2024-09-16T00:09:43.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.519033+0000", 2024-09-16T00:09:43.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.295e-05 2024-09-16T00:09:43.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.519106+0000", 2024-09-16T00:09:43.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3326999999999998e-05 2024-09-16T00:09:43.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.519243+0000", 2024-09-16T00:09:43.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013749500000000001 2024-09-16T00:09:43.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:43.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.519379+0000", 2024-09-16T00:09:43.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000136017 2024-09-16T00:09:43.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.524268+0000", 2024-09-16T00:09:43.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048883820000000001 2024-09-16T00:09:43.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.529701+0000", 2024-09-16T00:09:43.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054327259999999997 2024-09-16T00:09:43.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.529730+0000", 2024-09-16T00:09:43.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9071e-05 2024-09-16T00:09:43.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.529750+0000", 2024-09-16T00:09:43.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0248e-05 2024-09-16T00:09:43.978 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.978 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.978 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.978 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10968 3.5 3:a09cd6d9:::10000003180.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:43.978 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.602942+0000", 2024-09-16T00:09:43.978 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.3285100160000001, 2024-09-16T00:09:43.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014525788, 2024-09-16T00:09:43.979 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.979 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.979 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.979 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10968 2024-09-16T00:09:43.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.979 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.602942+0000", 2024-09-16T00:09:43.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.602942+0000", 2024-09-16T00:09:43.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.602943+0000", 2024-09-16T00:09:43.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.404e-06 2024-09-16T00:09:43.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.607434+0000", 2024-09-16T00:09:43.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044904740000000004 2024-09-16T00:09:43.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.607454+0000", 2024-09-16T00:09:43.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9966000000000002e-05 2024-09-16T00:09:43.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.607463+0000", 2024-09-16T00:09:43.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5370000000000003e-06 2024-09-16T00:09:43.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.607532+0000", 2024-09-16T00:09:43.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8996000000000001e-05 2024-09-16T00:09:43.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.607685+0000", 2024-09-16T00:09:43.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000152151 2024-09-16T00:09:43.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:43.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.607827+0000", 2024-09-16T00:09:43.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014191299999999999 2024-09-16T00:09:43.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.612195+0000", 2024-09-16T00:09:43.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043685249999999998 2024-09-16T00:09:43.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.617403+0000", 2024-09-16T00:09:43.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052079819999999999 2024-09-16T00:09:43.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.617442+0000", 2024-09-16T00:09:43.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9388999999999998e-05 2024-09-16T00:09:43.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.617468+0000", 2024-09-16T00:09:43.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5451000000000001e-05 2024-09-16T00:09:43.984 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.984 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.984 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.984 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10993 3.2a 3:56ee67f2:::10000003146.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:43.984 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.689451+0000", 2024-09-16T00:09:43.984 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.2420015089999996, 2024-09-16T00:09:43.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013790253000000001, 2024-09-16T00:09:43.985 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.985 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.985 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.985 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.985 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.985 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.988 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10993 2024-09-16T00:09:43.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.988 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.689451+0000", 2024-09-16T00:09:43.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.689451+0000", 2024-09-16T00:09:43.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.689452+0000", 2024-09-16T00:09:43.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.728e-06 2024-09-16T00:09:43.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.693556+0000", 2024-09-16T00:09:43.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041039900000000001 2024-09-16T00:09:43.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.693576+0000", 2024-09-16T00:09:43.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9763999999999999e-05 2024-09-16T00:09:43.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.693589+0000", 2024-09-16T00:09:43.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2717e-05 2024-09-16T00:09:43.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.693664+0000", 2024-09-16T00:09:43.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5592000000000004e-05 2024-09-16T00:09:43.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.693734+0000", 2024-09-16T00:09:43.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9375000000000006e-05 2024-09-16T00:09:43.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:43.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.693856+0000", 2024-09-16T00:09:43.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012249299999999999 2024-09-16T00:09:43.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.699797+0000", 2024-09-16T00:09:43.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059408739999999996 2024-09-16T00:09:43.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.703159+0000", 2024-09-16T00:09:43.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033615820000000001 2024-09-16T00:09:43.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.703194+0000", 2024-09-16T00:09:43.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5760999999999999e-05 2024-09-16T00:09:43.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.703241+0000", 2024-09-16T00:09:43.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6377e-05 2024-09-16T00:09:43.993 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.993 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:43.993 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.994 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10999 3.30 3:0d2bb3e6:::1000000313b.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:43.994 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.716323+0000", 2024-09-16T00:09:43.994 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.2151289639999998, 2024-09-16T00:09:43.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014232977000000001, 2024-09-16T00:09:43.994 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:43.994 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:43.994 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:43.994 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:43.994 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:43.994 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:43.994 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10999 2024-09-16T00:09:43.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.995 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:43.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:43.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.716323+0000", 2024-09-16T00:09:43.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:43.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.716323+0000", 2024-09-16T00:09:43.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:43.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:43.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.716325+0000", 2024-09-16T00:09:43.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3850000000000001e-06 2024-09-16T00:09:43.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:43.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.720324+0000", 2024-09-16T00:09:43.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003999496 2024-09-16T00:09:43.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:43.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.720350+0000", 2024-09-16T00:09:43.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5590000000000001e-05 2024-09-16T00:09:43.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:43.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.720363+0000", 2024-09-16T00:09:43.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3403e-05 2024-09-16T00:09:43.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:43.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.720448+0000", 2024-09-16T00:09:43.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5053e-05 2024-09-16T00:09:43.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:43.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.720551+0000", 2024-09-16T00:09:43.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103055 2024-09-16T00:09:43.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:43.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.720689+0000", 2024-09-16T00:09:43.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000137582 2024-09-16T00:09:43.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:43.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.724645+0000", 2024-09-16T00:09:43.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039561939999999997 2024-09-16T00:09:43.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:43.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.730487+0000", 2024-09-16T00:09:43.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058420340000000003 2024-09-16T00:09:43.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:43.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.730531+0000", 2024-09-16T00:09:43.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3748999999999997e-05 2024-09-16T00:09:43.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:43.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:43.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:43.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.730556+0000", 2024-09-16T00:09:43.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5435999999999998e-05 2024-09-16T00:09:43.999 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:43.999 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.000 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.000 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11051 3.30 3:0d2bb3e6:::1000000313b.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:44.000 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.831338+0000", 2024-09-16T00:09:44.000 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1001143720000002, 2024-09-16T00:09:44.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013649867, 2024-09-16T00:09:44.000 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.000 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.000 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.000 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.000 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.000 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.001 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11051 2024-09-16T00:09:44.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.001 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.831338+0000", 2024-09-16T00:09:44.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.831338+0000", 2024-09-16T00:09:44.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.831340+0000", 2024-09-16T00:09:44.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9850000000000001e-06 2024-09-16T00:09:44.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.835421+0000", 2024-09-16T00:09:44.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040808930000000004 2024-09-16T00:09:44.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.835445+0000", 2024-09-16T00:09:44.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4542e-05 2024-09-16T00:09:44.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.835452+0000", 2024-09-16T00:09:44.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3289999999999996e-06 2024-09-16T00:09:44.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.835516+0000", 2024-09-16T00:09:44.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3406999999999998e-05 2024-09-16T00:09:44.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.835577+0000", 2024-09-16T00:09:44.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0854999999999999e-05 2024-09-16T00:09:44.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:44.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.835671+0000", 2024-09-16T00:09:44.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4302999999999995e-05 2024-09-16T00:09:44.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.840227+0000", 2024-09-16T00:09:44.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045556240000000003 2024-09-16T00:09:44.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.844895+0000", 2024-09-16T00:09:44.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046682809999999998 2024-09-16T00:09:44.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.844945+0000", 2024-09-16T00:09:44.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0417999999999997e-05 2024-09-16T00:09:44.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.844988+0000", 2024-09-16T00:09:44.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2230000000000001e-05 2024-09-16T00:09:44.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.006 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.006 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.006 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11076 3.2f 3:f633d486:::1000000311d.00000000:head [write 0~4194304 [2@850962]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:44.006 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.915473+0000", 2024-09-16T00:09:44.006 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.0159792459999997, 2024-09-16T00:09:44.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016108857000000001, 2024-09-16T00:09:44.006 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.006 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.006 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.007 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.007 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11076 2024-09-16T00:09:44.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.007 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.915473+0000", 2024-09-16T00:09:44.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.915473+0000", 2024-09-16T00:09:44.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.915478+0000", 2024-09-16T00:09:44.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1699999999999996e-06 2024-09-16T00:09:44.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.921607+0000", 2024-09-16T00:09:44.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061287889999999999 2024-09-16T00:09:44.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.921632+0000", 2024-09-16T00:09:44.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5299999999999998e-05 2024-09-16T00:09:44.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.921642+0000", 2024-09-16T00:09:44.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6220000000000004e-06 2024-09-16T00:09:44.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.921717+0000", 2024-09-16T00:09:44.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5475000000000005e-05 2024-09-16T00:09:44.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.921818+0000", 2024-09-16T00:09:44.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000100692 2024-09-16T00:09:44.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:44.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.921961+0000", 2024-09-16T00:09:44.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014336700000000001 2024-09-16T00:09:44.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928835+0000", 2024-09-16T00:09:44.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068737729999999997 2024-09-16T00:09:44.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.931483+0000", 2024-09-16T00:09:44.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026481709999999999 2024-09-16T00:09:44.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.931540+0000", 2024-09-16T00:09:44.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6793000000000003e-05 2024-09-16T00:09:44.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.931582+0000", 2024-09-16T00:09:44.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1705000000000001e-05 2024-09-16T00:09:44.012 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.012 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.012 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.012 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11208 3.5 3:a2a10960:::1000000324a.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:44.012 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.550298+0000", 2024-09-16T00:09:44.012 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.3811536850000001, 2024-09-16T00:09:44.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013851664, 2024-09-16T00:09:44.012 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.012 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.012 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.012 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.013 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.013 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.013 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11208 2024-09-16T00:09:44.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.013 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.550298+0000", 2024-09-16T00:09:44.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.550298+0000", 2024-09-16T00:09:44.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.550299+0000", 2024-09-16T00:09:44.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.099e-06 2024-09-16T00:09:44.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.554685+0000", 2024-09-16T00:09:44.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043850419999999996 2024-09-16T00:09:44.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.554699+0000", 2024-09-16T00:09:44.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4450000000000001e-05 2024-09-16T00:09:44.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.554710+0000", 2024-09-16T00:09:44.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0878e-05 2024-09-16T00:09:44.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.554767+0000", 2024-09-16T00:09:44.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6861000000000003e-05 2024-09-16T00:09:44.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.554893+0000", 2024-09-16T00:09:44.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012634699999999999 2024-09-16T00:09:44.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:44.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.555037+0000", 2024-09-16T00:09:44.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000144253 2024-09-16T00:09:44.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.559587+0000", 2024-09-16T00:09:44.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045497940000000002 2024-09-16T00:09:44.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.564076+0000", 2024-09-16T00:09:44.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044890859999999998 2024-09-16T00:09:44.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.564111+0000", 2024-09-16T00:09:44.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5200000000000002e-05 2024-09-16T00:09:44.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.564150+0000", 2024-09-16T00:09:44.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8654e-05 2024-09-16T00:09:44.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.018 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.018 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11219 3.5 3:a3c82b2e:::10000003104.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:44.018 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.579601+0000", 2024-09-16T00:09:44.018 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.351850722, 2024-09-16T00:09:44.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015583845000000001, 2024-09-16T00:09:44.018 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.018 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.018 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.019 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11219 2024-09-16T00:09:44.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.019 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.579601+0000", 2024-09-16T00:09:44.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.579601+0000", 2024-09-16T00:09:44.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.579603+0000", 2024-09-16T00:09:44.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.782e-06 2024-09-16T00:09:44.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.584456+0000", 2024-09-16T00:09:44.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048530379999999996 2024-09-16T00:09:44.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.584476+0000", 2024-09-16T00:09:44.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9797999999999999e-05 2024-09-16T00:09:44.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.584486+0000", 2024-09-16T00:09:44.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7019999999999996e-06 2024-09-16T00:09:44.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.584547+0000", 2024-09-16T00:09:44.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1138999999999999e-05 2024-09-16T00:09:44.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.584627+0000", 2024-09-16T00:09:44.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9721999999999993e-05 2024-09-16T00:09:44.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:44.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.584750+0000", 2024-09-16T00:09:44.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012373399999999999 2024-09-16T00:09:44.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.591045+0000", 2024-09-16T00:09:44.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062944519999999999 2024-09-16T00:09:44.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.595077+0000", 2024-09-16T00:09:44.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040327109999999996 2024-09-16T00:09:44.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.595136+0000", 2024-09-16T00:09:44.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9011000000000001e-05 2024-09-16T00:09:44.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.595185+0000", 2024-09-16T00:09:44.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8755999999999999e-05 2024-09-16T00:09:44.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.024 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.024 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11274 3.5 3:a39318f5:::10000003106.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:44.024 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.770419+0000", 2024-09-16T00:09:44.024 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.1610328120000002, 2024-09-16T00:09:44.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016402152, 2024-09-16T00:09:44.024 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.024 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.024 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.024 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.025 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.025 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11274 2024-09-16T00:09:44.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.025 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.770419+0000", 2024-09-16T00:09:44.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.770419+0000", 2024-09-16T00:09:44.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.770420+0000", 2024-09-16T00:09:44.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0699999999999996e-07 2024-09-16T00:09:44.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.775020+0000", 2024-09-16T00:09:44.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004599433 2024-09-16T00:09:44.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.775043+0000", 2024-09-16T00:09:44.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:09:44.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.775053+0000", 2024-09-16T00:09:44.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0806e-05 2024-09-16T00:09:44.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.775136+0000", 2024-09-16T00:09:44.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3041000000000007e-05 2024-09-16T00:09:44.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.775294+0000", 2024-09-16T00:09:44.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015752100000000001 2024-09-16T00:09:44.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:44.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.775450+0000", 2024-09-16T00:09:44.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015631400000000001 2024-09-16T00:09:44.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.779198+0000", 2024-09-16T00:09:44.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037480590000000002 2024-09-16T00:09:44.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.786706+0000", 2024-09-16T00:09:44.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007507863 2024-09-16T00:09:44.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.786764+0000", 2024-09-16T00:09:44.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7624000000000003e-05 2024-09-16T00:09:44.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.786821+0000", 2024-09-16T00:09:44.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7562999999999998e-05 2024-09-16T00:09:44.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.030 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.030 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.030 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11752 3.30 3:0d01f22c:::1000000332e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:44.030 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.127099+0000", 2024-09-16T00:09:44.030 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.80435348600000001, 2024-09-16T00:09:44.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013651125, 2024-09-16T00:09:44.030 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.030 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.030 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.030 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.031 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.031 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11752 2024-09-16T00:09:44.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.031 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.127099+0000", 2024-09-16T00:09:44.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.127099+0000", 2024-09-16T00:09:44.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.127100+0000", 2024-09-16T00:09:44.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1680000000000001e-06 2024-09-16T00:09:44.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.032 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.131276+0000", 2024-09-16T00:09:44.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041766329999999999 2024-09-16T00:09:44.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.032 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.131296+0000", 2024-09-16T00:09:44.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0007e-05 2024-09-16T00:09:44.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.131308+0000", 2024-09-16T00:09:44.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1138000000000001e-05 2024-09-16T00:09:44.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.131388+0000", 2024-09-16T00:09:44.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0660000000000004e-05 2024-09-16T00:09:44.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.131572+0000", 2024-09-16T00:09:44.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018373200000000001 2024-09-16T00:09:44.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:44.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.131745+0000", 2024-09-16T00:09:44.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000172831 2024-09-16T00:09:44.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.136187+0000", 2024-09-16T00:09:44.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044419560000000004 2024-09-16T00:09:44.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.140635+0000", 2024-09-16T00:09:44.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044480559999999997 2024-09-16T00:09:44.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.140697+0000", 2024-09-16T00:09:44.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2241000000000003e-05 2024-09-16T00:09:44.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.140750+0000", 2024-09-16T00:09:44.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2703e-05 2024-09-16T00:09:44.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.036 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.036 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.036 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11831 3.16 3:69180767:::100000030f5.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:44.036 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.433667+0000", 2024-09-16T00:09:44.036 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.49778544800000002, 2024-09-16T00:09:44.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014292518000000001, 2024-09-16T00:09:44.036 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.036 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.036 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.036 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.037 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.037 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11831 2024-09-16T00:09:44.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.037 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.433667+0000", 2024-09-16T00:09:44.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.433667+0000", 2024-09-16T00:09:44.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.433668+0000", 2024-09-16T00:09:44.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.747e-06 2024-09-16T00:09:44.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.439244+0000", 2024-09-16T00:09:44.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005575867 2024-09-16T00:09:44.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.439273+0000", 2024-09-16T00:09:44.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9e-05 2024-09-16T00:09:44.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.439286+0000", 2024-09-16T00:09:44.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2771e-05 2024-09-16T00:09:44.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.439371+0000", 2024-09-16T00:09:44.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4610000000000005e-05 2024-09-16T00:09:44.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.039 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.439539+0000", 2024-09-16T00:09:44.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016804 2024-09-16T00:09:44.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:44.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.439688+0000", 2024-09-16T00:09:44.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000149801 2024-09-16T00:09:44.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.447263+0000", 2024-09-16T00:09:44.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007574229 2024-09-16T00:09:44.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.447878+0000", 2024-09-16T00:09:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00061571599999999998 2024-09-16T00:09:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.447923+0000", 2024-09-16T00:09:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4932999999999998e-05 2024-09-16T00:09:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.447959+0000", 2024-09-16T00:09:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5803999999999998e-05 2024-09-16T00:09:44.041 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.042 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.042 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.042 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:1539 2.18 2:1e8917d7:::200.0000000e:head [write 150960~1398 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e87)", 2024-09-16T00:09:44.042 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.653465+0000", 2024-09-16T00:09:44.042 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.27798677799999999, 2024-09-16T00:09:44.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013498066, 2024-09-16T00:09:44.042 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.042 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.042 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.042 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.653465+0000", 2024-09-16T00:09:44.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.653465+0000", 2024-09-16T00:09:44.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.653466+0000", 2024-09-16T00:09:44.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7499999999999999e-07 2024-09-16T00:09:44.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.653471+0000", 2024-09-16T00:09:44.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2739999999999998e-06 2024-09-16T00:09:44.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.653482+0000", 2024-09-16T00:09:44.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0195e-05 2024-09-16T00:09:44.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.653489+0000", 2024-09-16T00:09:44.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0829999999999998e-06 2024-09-16T00:09:44.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.659980+0000", 2024-09-16T00:09:44.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00649133 2024-09-16T00:09:44.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.660043+0000", 2024-09-16T00:09:44.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3186000000000001e-05 2024-09-16T00:09:44.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:44.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.660143+0000", 2024-09-16T00:09:44.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9872999999999997e-05 2024-09-16T00:09:44.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.666868+0000", 2024-09-16T00:09:44.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067244979999999998 2024-09-16T00:09:44.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.666877+0000", 2024-09-16T00:09:44.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2250000000000006e-06 2024-09-16T00:09:44.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.666924+0000", 2024-09-16T00:09:44.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7500000000000003e-05 2024-09-16T00:09:44.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.666963+0000", 2024-09-16T00:09:44.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9026999999999998e-05 2024-09-16T00:09:44.047 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.047 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.047 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.047 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11935 3.30 3:0e442c74:::1000000326e.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:44.048 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.692322+0000", 2024-09-16T00:09:44.048 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.23913002799999999, 2024-09-16T00:09:44.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014515112, 2024-09-16T00:09:44.048 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.048 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.048 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.048 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.048 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11935 2024-09-16T00:09:44.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.048 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.692322+0000", 2024-09-16T00:09:44.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.692322+0000", 2024-09-16T00:09:44.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.692323+0000", 2024-09-16T00:09:44.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1689999999999999e-06 2024-09-16T00:09:44.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.698154+0000", 2024-09-16T00:09:44.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058302830000000003 2024-09-16T00:09:44.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.698174+0000", 2024-09-16T00:09:44.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0971e-05 2024-09-16T00:09:44.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.698187+0000", 2024-09-16T00:09:44.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.275e-05 2024-09-16T00:09:44.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.698418+0000", 2024-09-16T00:09:44.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00023032700000000001 2024-09-16T00:09:44.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.698521+0000", 2024-09-16T00:09:44.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103099 2024-09-16T00:09:44.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:44.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.698661+0000", 2024-09-16T00:09:44.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000140037 2024-09-16T00:09:44.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705220+0000", 2024-09-16T00:09:44.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0065590529999999996 2024-09-16T00:09:44.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.706749+0000", 2024-09-16T00:09:44.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0015289629999999999 2024-09-16T00:09:44.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.706799+0000", 2024-09-16T00:09:44.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0331000000000003e-05 2024-09-16T00:09:44.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.706837+0000", 2024-09-16T00:09:44.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8129e-05 2024-09-16T00:09:44.053 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.053 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.053 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.053 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11981 3.c 3:33faa5a0:::10000003215.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:44.053 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.826554+0000", 2024-09-16T00:09:44.054 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.104898005, 2024-09-16T00:09:44.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017074514999999998, 2024-09-16T00:09:44.054 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.054 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.054 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.054 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.054 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11981 2024-09-16T00:09:44.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.054 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.826554+0000", 2024-09-16T00:09:44.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.826554+0000", 2024-09-16T00:09:44.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.826555+0000", 2024-09-16T00:09:44.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.33e-06 2024-09-16T00:09:44.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.831028+0000", 2024-09-16T00:09:44.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044725900000000002 2024-09-16T00:09:44.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.831041+0000", 2024-09-16T00:09:44.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2941e-05 2024-09-16T00:09:44.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.831048+0000", 2024-09-16T00:09:44.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6800000000000004e-06 2024-09-16T00:09:44.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.831108+0000", 2024-09-16T00:09:44.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0207000000000001e-05 2024-09-16T00:09:44.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.831200+0000", 2024-09-16T00:09:44.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2478999999999993e-05 2024-09-16T00:09:44.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:44.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.831344+0000", 2024-09-16T00:09:44.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014351199999999999 2024-09-16T00:09:44.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.841581+0000", 2024-09-16T00:09:44.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010236723 2024-09-16T00:09:44.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.843525+0000", 2024-09-16T00:09:44.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019441739999999999 2024-09-16T00:09:44.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.843587+0000", 2024-09-16T00:09:44.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2347000000000005e-05 2024-09-16T00:09:44.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.059 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.843629+0000", 2024-09-16T00:09:44.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1532000000000001e-05 2024-09-16T00:09:44.059 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.059 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.059 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.059 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.059 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11982 3.3c 3:3c2be35d:::100000030f7.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:44.059 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.831068+0000", 2024-09-16T00:09:44.059 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.100383818, 2024-09-16T00:09:44.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015244723999999999, 2024-09-16T00:09:44.060 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.060 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.060 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.060 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.060 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.060 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.060 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11982 2024-09-16T00:09:44.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.060 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.831068+0000", 2024-09-16T00:09:44.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.831068+0000", 2024-09-16T00:09:44.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.831069+0000", 2024-09-16T00:09:44.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2600000000000001e-07 2024-09-16T00:09:44.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.835408+0000", 2024-09-16T00:09:44.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043384449999999998 2024-09-16T00:09:44.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.835432+0000", 2024-09-16T00:09:44.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4700999999999999e-05 2024-09-16T00:09:44.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.835446+0000", 2024-09-16T00:09:44.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3645999999999999e-05 2024-09-16T00:09:44.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.835521+0000", 2024-09-16T00:09:44.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5087999999999997e-05 2024-09-16T00:09:44.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.835625+0000", 2024-09-16T00:09:44.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000103996 2024-09-16T00:09:44.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:44.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.835774+0000", 2024-09-16T00:09:44.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014938000000000001 2024-09-16T00:09:44.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.842026+0000", 2024-09-16T00:09:44.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062514099999999998 2024-09-16T00:09:44.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.846253+0000", 2024-09-16T00:09:44.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042273889999999998 2024-09-16T00:09:44.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.846285+0000", 2024-09-16T00:09:44.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1976e-05 2024-09-16T00:09:44.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.846313+0000", 2024-09-16T00:09:44.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7767e-05 2024-09-16T00:09:44.065 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.065 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.065 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.065 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.065 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.065 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:44.067 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:09:44.073 INFO:tasks.workunit.client.0.smithi139.stdout:9/395: sync 2024-09-16T00:09:44.073 INFO:tasks.workunit.client.0.smithi139.stdout:1/401: write d7/d8/d1f/f5d [3765317,24966] 0 2024-09-16T00:09:44.074 INFO:tasks.workunit.client.0.smithi139.stdout:7/367: read d5/d26/d42/f66 [5240448,51628] 0 2024-09-16T00:09:44.074 INFO:tasks.workunit.client.0.smithi139.stdout:2/405: creat d11/d2f/d6e/f84 x:0 0 0 2024-09-16T00:09:44.074 INFO:tasks.workunit.client.0.smithi139.stdout:1/402: chown d7/d8/l3e 286670 1 2024-09-16T00:09:44.074 INFO:tasks.workunit.client.0.smithi139.stdout:2/406: fdatasync d11/f1e 0 2024-09-16T00:09:44.074 INFO:tasks.workunit.client.0.smithi139.stdout:9/396: creat d19/d1a/d20/d23/d2d/d41/d71/d32/f84 x:0 0 0 2024-09-16T00:09:44.074 INFO:tasks.workunit.client.0.smithi139.stdout:8/395: dwrite d17/d1a/d2d/f2e [4194304,4194304] 0 2024-09-16T00:09:44.074 INFO:tasks.workunit.client.0.smithi139.stdout:7/368: creat d5/d16/d22/d25/d2d/d81/f82 x:0 0 0 2024-09-16T00:09:44.074 INFO:tasks.workunit.client.0.smithi139.stdout:1/403: symlink d7/l7b 0 2024-09-16T00:09:44.074 INFO:tasks.workunit.client.0.smithi139.stdout:9/397: symlink d19/d1a/d20/d23/d2d/d2f/l85 0 2024-09-16T00:09:44.074 INFO:tasks.workunit.client.0.smithi139.stdout:8/396: creat d17/d1a/d2d/f85 x:0 0 0 2024-09-16T00:09:44.074 INFO:tasks.workunit.client.0.smithi139.stdout:9/398: stat d19/d1a/d20/d23/d2d 0 2024-09-16T00:09:44.074 INFO:tasks.workunit.client.0.smithi139.stdout:1/404: mkdir d7/d8/d27/d68/d7c 0 2024-09-16T00:09:44.074 INFO:tasks.workunit.client.0.smithi139.stdout:1/405: chown d7/f16 17 1 2024-09-16T00:09:44.074 INFO:tasks.workunit.client.0.smithi139.stdout:9/399: dread - d19/d1a/d20/d23/d2d/d41/f6f zero size 2024-09-16T00:09:44.075 INFO:tasks.workunit.client.0.smithi139.stdout:2/407: dwrite d11/d30/d57/d29/d2b/f31 [0,4194304] 0 2024-09-16T00:09:44.075 INFO:tasks.workunit.client.0.smithi139.stdout:8/397: mkdir d17/d1a/d50/d86 0 2024-09-16T00:09:44.075 INFO:tasks.workunit.client.0.smithi139.stdout:1/406: rmdir d7/d8/d27/d2f 39 2024-09-16T00:09:44.075 INFO:tasks.workunit.client.0.smithi139.stdout:7/369: link d5/d16/d22/d25/f30 d5/d37/f83 0 2024-09-16T00:09:44.075 INFO:tasks.workunit.client.0.smithi139.stdout:2/408: creat d11/d30/d57/d50/f85 x:0 0 0 2024-09-16T00:09:44.075 INFO:tasks.workunit.client.0.smithi139.stdout:8/398: mknod d17/d54/c87 0 2024-09-16T00:09:44.075 INFO:tasks.workunit.client.0.smithi139.stdout:1/407: mkdir d7/d8/d40/d48/d67/d7d 0 2024-09-16T00:09:44.075 INFO:tasks.workunit.client.0.smithi139.stdout:7/370: creat d5/d16/d22/d25/d2d/d50/d44/f84 x:0 0 0 2024-09-16T00:09:44.075 INFO:tasks.workunit.client.0.smithi139.stdout:2/409: unlink c0 0 2024-09-16T00:09:44.075 INFO:tasks.workunit.client.0.smithi139.stdout:7/371: chown d5/d37/f83 148 1 2024-09-16T00:09:44.075 INFO:tasks.workunit.client.0.smithi139.stdout:7/372: link d5/d37/f47 d5/f85 0 2024-09-16T00:09:44.075 INFO:tasks.workunit.client.0.smithi139.stdout:2/410: dwrite d11/d30/f44 [0,4194304] 0 2024-09-16T00:09:44.075 INFO:tasks.workunit.client.0.smithi139.stdout:2/411: chown d11/d30/d57/d5d/d73/f79 119197 1 2024-09-16T00:09:44.075 INFO:tasks.workunit.client.0.smithi139.stdout:9/400: dread d19/d1a/d20/d23/f33 [0,4194304] 0 2024-09-16T00:09:44.076 INFO:tasks.workunit.client.0.smithi139.stdout:7/373: rename d5/d16/d22/d25/d2d/f3b to d5/d37/f86 0 2024-09-16T00:09:44.076 INFO:tasks.workunit.client.0.smithi139.stdout:9/401: rename d19/d1a/d20/d23/d2d/d35/d3d to d19/d86 0 2024-09-16T00:09:44.076 INFO:tasks.workunit.client.0.smithi139.stdout:2/412: getdents d11/d30/d57/d29 0 2024-09-16T00:09:44.076 INFO:tasks.workunit.client.0.smithi139.stdout:7/374: dwrite d5/d37/f47 [0,4194304] 0 2024-09-16T00:09:44.076 INFO:tasks.workunit.client.0.smithi139.stdout:9/402: rmdir d19/d1a/d20/d23/d2d/d41/d71/d32 39 2024-09-16T00:09:44.076 INFO:tasks.workunit.client.0.smithi139.stdout:2/413: mkdir d11/d30/d57/d29/d86 0 2024-09-16T00:09:44.076 INFO:tasks.workunit.client.0.smithi139.stdout:9/403: mknod d19/d1a/d20/d23/d2d/c87 0 2024-09-16T00:09:44.076 INFO:tasks.workunit.client.0.smithi139.stdout:2/414: stat d11/d32/c72 0 2024-09-16T00:09:44.076 INFO:tasks.workunit.client.0.smithi139.stdout:9/404: mknod d19/d1a/d20/d23/c88 0 2024-09-16T00:09:44.076 INFO:tasks.workunit.client.0.smithi139.stdout:9/405: readlink ld 0 2024-09-16T00:09:44.076 INFO:tasks.workunit.client.0.smithi139.stdout:7/375: dwrite d5/d37/f86 [0,4194304] 0 2024-09-16T00:09:44.076 INFO:tasks.workunit.client.0.smithi139.stdout:7/376: stat d5/d16/d22/d25/d2d/d50/d44/f84 0 2024-09-16T00:09:44.076 INFO:tasks.workunit.client.0.smithi139.stdout:2/415: creat d11/d30/d57/d5d/d73/d74/d81/f87 x:0 0 0 2024-09-16T00:09:44.077 INFO:tasks.workunit.client.0.smithi139.stdout:2/416: symlink d11/d30/l88 0 2024-09-16T00:09:44.077 INFO:tasks.workunit.client.0.smithi139.stdout:2/417: creat d11/d30/d57/d5d/d73/d74/d81/f89 x:0 0 0 2024-09-16T00:09:44.077 INFO:tasks.workunit.client.0.smithi139.stdout:9/406: dread d19/d1a/d20/d23/d2d/d41/d71/d32/d79/f5d [0,4194304] 0 2024-09-16T00:09:44.077 INFO:tasks.workunit.client.0.smithi139.stdout:2/418: mknod d11/d30/d57/d50/c8a 0 2024-09-16T00:09:44.077 INFO:tasks.workunit.client.0.smithi139.stdout:2/419: rename d11/d2f/d6e/c76 to d11/d45/c8b 0 2024-09-16T00:09:44.077 INFO:tasks.workunit.client.0.smithi139.stdout:7/377: dread d5/d16/d22/d25/f31 [0,4194304] 0 2024-09-16T00:09:44.077 INFO:tasks.workunit.client.0.smithi139.stdout:2/420: mknod d11/d45/c8c 0 2024-09-16T00:09:44.077 INFO:tasks.workunit.client.0.smithi139.stdout:9/407: dread d19/d86/f5a [0,4194304] 0 2024-09-16T00:09:44.077 INFO:tasks.workunit.client.0.smithi139.stdout:2/421: write d11/d30/d57/d29/d2b/f31 [78421,74743] 0 2024-09-16T00:09:44.077 INFO:tasks.workunit.client.0.smithi139.stdout:9/408: write d19/d1a/d20/d23/d2d/d35/d42/d3a/f6d [68157,15534] 0 2024-09-16T00:09:44.077 INFO:tasks.workunit.client.0.smithi139.stdout:9/409: chown d19/d86/c64 10773 1 2024-09-16T00:09:44.077 INFO:tasks.workunit.client.0.smithi139.stdout:9/410: fsync d19/f38 0 2024-09-16T00:09:44.077 INFO:tasks.workunit.client.0.smithi139.stdout:2/422: rename c10 to d11/d30/d57/d5d/c8d 0 2024-09-16T00:09:44.077 INFO:tasks.workunit.client.0.smithi139.stdout:7/378: dwrite d5/d16/d22/d25/d2d/f3a [0,4194304] 0 2024-09-16T00:09:44.078 INFO:tasks.workunit.client.0.smithi139.stdout:9/411: chown d19/d1a/d20/d23/d2d/d41/d71/d32/d79/f5d 71453224 1 2024-09-16T00:09:44.078 INFO:tasks.workunit.client.0.smithi139.stdout:7/379: rmdir d5/d26/d46 39 2024-09-16T00:09:44.078 INFO:tasks.workunit.client.0.smithi139.stdout:9/412: rmdir d19/d1a/d20/d23/d2d/d35/d42/d3a/d45 39 2024-09-16T00:09:44.078 INFO:tasks.workunit.client.0.smithi139.stdout:7/380: mkdir d5/d16/d87 0 2024-09-16T00:09:44.078 INFO:tasks.workunit.client.0.smithi139.stdout:7/381: chown d5/d16/d22/d25/d2d/d50/d44/d55 67 1 2024-09-16T00:09:44.078 INFO:tasks.workunit.client.0.smithi139.stdout:9/413: dread fa [0,4194304] 0 2024-09-16T00:09:44.078 INFO:tasks.workunit.client.0.smithi139.stdout:9/414: getdents d19/d1a/d20/d23/d2d/d41/d71/d32/d79 0 2024-09-16T00:09:44.078 INFO:tasks.workunit.client.0.smithi139.stdout:9/415: fdatasync d19/d1a/f50 0 2024-09-16T00:09:44.078 INFO:tasks.workunit.client.0.smithi139.stdout:7/382: dwrite d5/d37/f40 [0,4194304] 0 2024-09-16T00:09:44.078 INFO:tasks.workunit.client.0.smithi139.stdout:7/383: rename d5/d16 to d5/d16/d6d/d88 22 2024-09-16T00:09:44.078 INFO:tasks.workunit.client.0.smithi139.stdout:9/416: getdents d19/d1a/d20/d23/d2d/d35/d42 0 2024-09-16T00:09:44.078 INFO:tasks.workunit.client.0.smithi139.stdout:7/384: rename d5/d37/f83 to d5/d16/d22/d25/d2d/d81/f89 0 2024-09-16T00:09:44.078 INFO:tasks.workunit.client.0.smithi139.stdout:9/417: mkdir d19/d1a/d20/d89 0 2024-09-16T00:09:44.078 INFO:tasks.workunit.client.0.smithi139.stdout:7/385: dwrite d5/d37/f7b [0,4194304] 0 2024-09-16T00:09:44.079 INFO:tasks.workunit.client.0.smithi139.stdout:9/418: dwrite d19/f6c [0,4194304] 0 2024-09-16T00:09:44.079 INFO:tasks.workunit.client.0.smithi139.stdout:9/419: dread - d19/d86/f7e zero size 2024-09-16T00:09:44.079 INFO:tasks.workunit.client.0.smithi139.stdout:7/386: truncate f4 7831145 0 2024-09-16T00:09:44.079 INFO:tasks.workunit.client.0.smithi139.stdout:9/420: read d19/d86/f54 [22830,14961] 0 2024-09-16T00:09:44.079 INFO:tasks.workunit.client.0.smithi139.stdout:9/421: creat d19/d1a/d20/d23/d2d/d41/d71/d32/d4e/f8a x:0 0 0 2024-09-16T00:09:44.079 INFO:tasks.workunit.client.0.smithi139.stdout:9/422: mknod d19/c8b 0 2024-09-16T00:09:44.079 INFO:tasks.workunit.client.0.smithi139.stdout:9/423: unlink d19/d86/f7e 0 2024-09-16T00:09:44.079 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:44.066+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:44.080 INFO:tasks.workunit.client.0.smithi139.stdout:7/387: write d5/d16/d22/d25/d2d/d50/d44/d55/f6c [2074521,103760] 0 2024-09-16T00:09:44.080 INFO:tasks.workunit.client.0.smithi139.stdout:9/424: fsync d19/d1a/d20/d23/d2d/d35/d42/d3a/f6e 0 2024-09-16T00:09:44.080 INFO:tasks.workunit.client.0.smithi139.stdout:9/425: mknod d19/d1a/d20/d23/d2d/d41/d71/d32/d4e/d67/c8c 0 2024-09-16T00:09:44.080 INFO:tasks.workunit.client.0.smithi139.stdout:7/388: dwrite d5/d37/f40 [0,4194304] 0 2024-09-16T00:09:44.080 INFO:tasks.workunit.client.0.smithi139.stdout:9/426: chown d19/d1a/d20/d23/d2d/d41/d71 4817 1 2024-09-16T00:09:44.081 INFO:tasks.workunit.client.0.smithi139.stdout:9/427: dwrite d19/d86/f54 [0,4194304] 0 2024-09-16T00:09:44.087 INFO:tasks.workunit.client.0.smithi139.stdout:9/428: truncate d19/d1a/f21 175167 0 2024-09-16T00:09:44.088 INFO:tasks.workunit.client.0.smithi139.stdout:9/429: chown d19/d1a/d20/d23/d2d/d2f 3300195 1 2024-09-16T00:09:44.088 INFO:tasks.workunit.client.0.smithi139.stdout:7/389: dread d5/f6 [0,4194304] 0 2024-09-16T00:09:44.090 INFO:tasks.workunit.client.0.smithi139.stdout:7/390: rename d5/d26/d5d/c78 to d5/d16/d22/c8a 0 2024-09-16T00:09:44.091 INFO:tasks.workunit.client.0.smithi139.stdout:7/391: truncate d5/d37/f79 493135 0 2024-09-16T00:09:44.092 INFO:tasks.workunit.client.0.smithi139.stdout:5/429: sync 2024-09-16T00:09:44.093 INFO:tasks.workunit.client.0.smithi139.stdout:7/392: stat d5/d37/c61 0 2024-09-16T00:09:44.094 INFO:tasks.workunit.client.0.smithi139.stdout:9/430: dwrite d19/d1a/d20/d23/d2d/d35/f66 [0,4194304] 0 2024-09-16T00:09:44.094 INFO:tasks.workunit.client.0.smithi139.stdout:5/430: read d1/d13/f17 [3762517,73941] 0 2024-09-16T00:09:44.097 INFO:tasks.workunit.client.0.smithi139.stdout:9/431: truncate d19/d1a/d20/d23/d2d/d35/d42/d3a/f6e 325700 0 2024-09-16T00:09:44.098 INFO:tasks.workunit.client.0.smithi139.stdout:7/393: rename d5/d26/d5d/l5e to d5/d26/d74/l8b 0 2024-09-16T00:09:44.098 INFO:tasks.workunit.client.0.smithi139.stdout:9/432: readlink d19/d1a/d20/d23/d2d/d35/d42/l4c 0 2024-09-16T00:09:44.100 INFO:tasks.workunit.client.0.smithi139.stdout:5/431: creat d1/d21/d49/d87/f91 x:0 0 0 2024-09-16T00:09:44.100 INFO:tasks.workunit.client.0.smithi139.stdout:9/433: write d19/d1a/d20/d23/d2d/d35/d60/f62 [350204,96420] 0 2024-09-16T00:09:44.101 INFO:tasks.workunit.client.0.smithi139.stdout:7/394: mknod d5/d16/d6d/d7f/c8c 0 2024-09-16T00:09:44.102 INFO:tasks.workunit.client.0.smithi139.stdout:5/432: mknod d1/d13/d3f/c92 0 2024-09-16T00:09:44.103 INFO:tasks.workunit.client.0.smithi139.stdout:5/433: mkdir d1/d25/d73/d93 0 2024-09-16T00:09:44.105 INFO:tasks.workunit.client.0.smithi139.stdout:5/434: mkdir d1/d13/d94 0 2024-09-16T00:09:44.106 INFO:tasks.workunit.client.0.smithi139.stdout:5/435: stat d1/d13/d22 0 2024-09-16T00:09:44.107 INFO:tasks.workunit.client.0.smithi139.stdout:5/436: stat d1/d13/d3f/d5e/l6f 0 2024-09-16T00:09:44.107 INFO:tasks.workunit.client.0.smithi139.stdout:9/434: dwrite d19/d1a/d20/d23/d2d/d41/d71/d32/d79/f5d [0,4194304] 0 2024-09-16T00:09:44.109 INFO:tasks.workunit.client.0.smithi139.stdout:7/395: truncate d5/d16/d22/f6f 4343923 0 2024-09-16T00:09:44.111 INFO:tasks.workunit.client.0.smithi139.stdout:9/435: mknod d19/d1a/d20/d23/c8d 0 2024-09-16T00:09:44.111 INFO:tasks.workunit.client.0.smithi139.stdout:9/436: readlink l18 0 2024-09-16T00:09:44.112 INFO:tasks.workunit.client.0.smithi139.stdout:7/396: readlink d5/d16/l17 0 2024-09-16T00:09:44.114 INFO:tasks.workunit.client.0.smithi139.stdout:9/437: rmdir d19/d1a/d20/d23/d2d/d2f 39 2024-09-16T00:09:44.115 INFO:tasks.workunit.client.0.smithi139.stdout:7/397: symlink d5/d26/d46/l8d 0 2024-09-16T00:09:44.121 INFO:tasks.workunit.client.0.smithi139.stdout:9/438: dwrite d19/d1a/d20/d23/d2d/d41/d71/d32/f84 [0,4194304] 0 2024-09-16T00:09:44.122 INFO:tasks.workunit.client.0.smithi139.stdout:7/398: dwrite d5/f1f [8388608,4194304] 0 2024-09-16T00:09:44.126 INFO:tasks.workunit.client.0.smithi139.stdout:7/399: creat d5/d26/d5d/f8e x:0 0 0 2024-09-16T00:09:44.128 INFO:tasks.workunit.client.0.smithi139.stdout:9/439: mknod d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/c8e 0 2024-09-16T00:09:44.129 INFO:tasks.workunit.client.0.smithi139.stdout:9/440: read fa [463422,78436] 0 2024-09-16T00:09:44.130 INFO:tasks.workunit.client.0.smithi139.stdout:7/400: rename d5/d16/d6d/f6e to d5/d26/d42/f8f 0 2024-09-16T00:09:44.132 INFO:tasks.workunit.client.0.smithi139.stdout:9/441: stat d19/d1a/d20/d23/f33 0 2024-09-16T00:09:44.132 INFO:tasks.workunit.client.0.smithi139.stdout:7/401: mkdir d5/d16/d22/d25/d2d/d50/d44/d55/d90 0 2024-09-16T00:09:44.133 INFO:tasks.workunit.client.0.smithi139.stdout:9/442: rmdir d19/d1a/d20/d23/d2d/d41/d71/d32/d4e/d67 39 2024-09-16T00:09:44.135 INFO:tasks.workunit.client.0.smithi139.stdout:7/402: creat d5/d16/d22/d25/d63/f91 x:0 0 0 2024-09-16T00:09:44.137 INFO:tasks.workunit.client.0.smithi139.stdout:7/403: symlink d5/d16/d22/d25/d2d/d81/l92 0 2024-09-16T00:09:44.138 INFO:tasks.workunit.client.0.smithi139.stdout:9/443: dread d19/d1a/d20/d23/d2d/d41/d71/d32/d79/f5d [0,4194304] 0 2024-09-16T00:09:44.141 INFO:tasks.workunit.client.0.smithi139.stdout:7/404: link d5/d16/d22/d25/f76 d5/d16/d6d/d7f/f93 0 2024-09-16T00:09:44.142 INFO:tasks.workunit.client.0.smithi139.stdout:9/444: dwrite d19/d86/f63 [0,4194304] 0 2024-09-16T00:09:44.142 INFO:tasks.workunit.client.0.smithi139.stdout:7/405: stat d5/d37/f47 0 2024-09-16T00:09:44.143 INFO:tasks.workunit.client.0.smithi139.stdout:9/445: chown d19/d1a/d20/d23/c72 93743 1 2024-09-16T00:09:44.146 INFO:tasks.workunit.client.0.smithi139.stdout:9/446: creat d19/d1a/f8f x:0 0 0 2024-09-16T00:09:44.149 INFO:tasks.workunit.client.0.smithi139.stdout:9/447: read d19/d1a/f21 [140498,45710] 0 2024-09-16T00:09:44.149 INFO:tasks.workunit.client.0.smithi139.stdout:7/406: dwrite d5/d16/d22/d25/d2d/f3a [0,4194304] 0 2024-09-16T00:09:44.150 INFO:tasks.workunit.client.0.smithi139.stdout:7/407: chown d5/c41 10609 1 2024-09-16T00:09:44.151 INFO:tasks.workunit.client.0.smithi139.stdout:7/408: chown d5/c52 6981585 1 2024-09-16T00:09:44.154 INFO:tasks.workunit.client.0.smithi139.stdout:9/448: creat d19/d1a/d20/d23/d2d/f90 x:0 0 0 2024-09-16T00:09:44.156 INFO:tasks.workunit.client.0.smithi139.stdout:7/409: truncate d5/d16/d22/f4e 7903055 0 2024-09-16T00:09:44.157 INFO:tasks.workunit.client.0.smithi139.stdout:9/449: creat d19/d1a/d20/d23/d2d/d41/d71/d32/d79/f91 x:0 0 0 2024-09-16T00:09:44.157 INFO:tasks.workunit.client.0.smithi139.stdout:7/410: mknod d5/d16/d87/c94 0 2024-09-16T00:09:44.160 INFO:tasks.workunit.client.0.smithi139.stdout:9/450: creat d19/d1a/d20/d23/d2d/f92 x:0 0 0 2024-09-16T00:09:44.160 INFO:tasks.workunit.client.0.smithi139.stdout:7/411: getdents d5/d26/d46/d6b 0 2024-09-16T00:09:44.162 INFO:tasks.workunit.client.0.smithi139.stdout:9/451: mkdir d19/d1a/d20/d89/d93 0 2024-09-16T00:09:44.163 INFO:tasks.workunit.client.0.smithi139.stdout:7/412: dread d5/d26/f75 [0,4194304] 0 2024-09-16T00:09:44.165 INFO:tasks.workunit.client.0.smithi139.stdout:7/413: creat d5/d26/d5d/f95 x:0 0 0 2024-09-16T00:09:44.166 INFO:tasks.workunit.client.0.smithi139.stdout:7/414: fdatasync d5/d16/d22/d25/f7a 0 2024-09-16T00:09:44.166 INFO:tasks.workunit.client.0.smithi139.stdout:9/452: rename d19/d1a/d20/d23/d2d/d41/d71/d32/d4e/d67/c8c to d19/d1a/d20/d23/d2d/d35/d44/c94 0 2024-09-16T00:09:44.167 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:44.166+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:44.169 INFO:tasks.workunit.client.0.smithi139.stdout:7/415: rename d5/d16/d6d/d7f/c8c to d5/d16/d22/d25/d2d/d81/c96 0 2024-09-16T00:09:44.169 INFO:tasks.workunit.client.0.smithi139.stdout:9/453: symlink d19/d1a/d20/d23/d2d/d41/d71/d32/d79/l95 0 2024-09-16T00:09:44.172 INFO:tasks.workunit.client.0.smithi139.stdout:7/416: rename d5/d16/d6d/f7c to d5/d16/d22/d25/d63/f97 0 2024-09-16T00:09:44.172 INFO:tasks.workunit.client.0.smithi139.stdout:9/454: write d19/d1a/d20/d23/d2d/d41/d71/d32/d4e/f78 [134207,45433] 0 2024-09-16T00:09:44.174 INFO:tasks.workunit.client.0.smithi139.stdout:7/417: getdents d5/d16/d22/d25/d2d/d50/d44 0 2024-09-16T00:09:44.176 INFO:tasks.workunit.client.0.smithi139.stdout:9/455: link d19/d1a/d20/d23/d2d/d41/d71/f57 d19/d1a/d20/d23/d2d/d2f/f96 0 2024-09-16T00:09:44.177 INFO:tasks.workunit.client.0.smithi139.stdout:7/418: dread d5/fc [4194304,4194304] 0 2024-09-16T00:09:44.177 INFO:tasks.workunit.client.0.smithi139.stdout:9/456: mknod d19/d86/c97 0 2024-09-16T00:09:44.178 INFO:tasks.workunit.client.0.smithi139.stdout:7/419: dread - d5/d16/d22/d25/d63/f91 zero size 2024-09-16T00:09:44.179 INFO:tasks.workunit.client.0.smithi139.stdout:9/457: mknod d19/d1a/d20/d23/d2d/d41/d71/d32/c98 0 2024-09-16T00:09:44.180 INFO:tasks.workunit.client.0.smithi139.stdout:7/420: creat d5/d26/d5d/f98 x:0 0 0 2024-09-16T00:09:44.184 INFO:tasks.workunit.client.0.smithi139.stdout:7/421: link d5/c41 d5/d26/c99 0 2024-09-16T00:09:44.185 INFO:tasks.workunit.client.0.smithi139.stdout:9/458: dwrite d19/d1a/d20/d23/d2d/d35/d42/d3a/f48 [0,4194304] 0 2024-09-16T00:09:44.186 INFO:tasks.workunit.client.0.smithi139.stdout:7/422: write d5/d16/d22/d25/d2d/d50/f67 [119034,123986] 0 2024-09-16T00:09:44.186 INFO:tasks.workunit.client.0.smithi139.stdout:7/423: chown d5/d26/d46 233820 1 2024-09-16T00:09:44.188 INFO:tasks.workunit.client.0.smithi139.stdout:7/424: truncate d5/d26/d46/f4a 603411 0 2024-09-16T00:09:44.194 INFO:tasks.workunit.client.0.smithi139.stdout:7/425: getdents d5/d16 0 2024-09-16T00:09:44.197 INFO:tasks.workunit.client.0.smithi139.stdout:7/426: creat d5/d26/f9a x:0 0 0 2024-09-16T00:09:44.198 INFO:tasks.workunit.client.0.smithi139.stdout:7/427: write d5/d26/f9a [195811,87171] 0 2024-09-16T00:09:44.199 INFO:tasks.workunit.client.0.smithi139.stdout:9/459: truncate d19/f6c 447939 0 2024-09-16T00:09:44.200 INFO:tasks.workunit.client.0.smithi139.stdout:9/460: fdatasync d19/d1a/d20/d23/d2d/d41/f6f 0 2024-09-16T00:09:44.201 INFO:tasks.workunit.client.0.smithi139.stdout:7/428: dread d5/d26/d42/f5b [0,4194304] 0 2024-09-16T00:09:44.202 INFO:tasks.workunit.client.0.smithi139.stdout:9/461: creat d19/d1a/d20/d23/d2d/f99 x:0 0 0 2024-09-16T00:09:44.203 INFO:tasks.workunit.client.0.smithi139.stdout:7/429: unlink d5/l18 0 2024-09-16T00:09:44.203 INFO:tasks.workunit.client.0.smithi139.stdout:9/462: write d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/f55 [43761,42641] 0 2024-09-16T00:09:44.205 INFO:tasks.workunit.client.0.smithi139.stdout:9/463: mkdir d19/d1a/d20/d23/d2d/d9a 0 2024-09-16T00:09:44.207 INFO:tasks.workunit.client.0.smithi139.stdout:9/464: symlink d19/d1a/d20/d23/d2d/d41/d71/d32/d4e/l9b 0 2024-09-16T00:09:44.208 INFO:tasks.workunit.client.0.smithi139.stdout:9/465: stat d19/d1a 0 2024-09-16T00:09:44.209 INFO:tasks.workunit.client.0.smithi139.stdout:6/287: sync 2024-09-16T00:09:44.209 INFO:tasks.workunit.client.0.smithi139.stdout:0/394: sync 2024-09-16T00:09:44.210 INFO:tasks.workunit.client.0.smithi139.stdout:6/288: write d8/d20/f62 [660070,58218] 0 2024-09-16T00:09:44.211 INFO:tasks.workunit.client.0.smithi139.stdout:0/395: write d6/f52 [1465021,111054] 0 2024-09-16T00:09:44.214 INFO:tasks.workunit.client.0.smithi139.stdout:0/396: creat d6/d1c/d41/d78/f7d x:0 0 0 2024-09-16T00:09:44.214 INFO:tasks.workunit.client.0.smithi139.stdout:0/397: chown d6/d42/l73 1079 1 2024-09-16T00:09:44.215 INFO:tasks.workunit.client.0.smithi139.stdout:0/398: chown d6/d5a/d70/f53 671 1 2024-09-16T00:09:44.219 INFO:tasks.workunit.client.0.smithi139.stdout:0/399: dread d6/d5a/d70/f4f [0,4194304] 0 2024-09-16T00:09:44.221 INFO:tasks.workunit.client.0.smithi139.stdout:9/466: dwrite d19/d1a/f21 [0,4194304] 0 2024-09-16T00:09:44.224 INFO:tasks.workunit.client.0.smithi139.stdout:9/467: write d19/d1a/d20/d23/f33 [2015154,107663] 0 2024-09-16T00:09:44.225 INFO:tasks.workunit.client.0.smithi139.stdout:0/400: link d6/d1c/d41/d44/c1e d6/d5a/d70/d7b/c7e 0 2024-09-16T00:09:44.226 INFO:tasks.workunit.client.0.smithi139.stdout:0/401: write d6/d1c/d45/f5e [2293282,86339] 0 2024-09-16T00:09:44.227 INFO:tasks.workunit.client.0.smithi139.stdout:9/468: mknod d19/d1a/d20/d23/d2d/d9a/c9c 0 2024-09-16T00:09:44.228 INFO:tasks.workunit.client.0.smithi139.stdout:0/402: mkdir d6/d5a/d70/d7f 0 2024-09-16T00:09:44.229 INFO:tasks.workunit.client.0.smithi139.stdout:0/403: stat d6/d1c/d41/d44/dd/l30 0 2024-09-16T00:09:44.230 INFO:tasks.workunit.client.0.smithi139.stdout:9/469: creat d19/d1a/d20/d89/f9d x:0 0 0 2024-09-16T00:09:44.233 INFO:tasks.workunit.client.0.smithi139.stdout:9/470: mknod d19/d1a/d20/d23/d2d/d35/d42/d3a/c9e 0 2024-09-16T00:09:44.234 INFO:tasks.workunit.client.0.smithi139.stdout:9/471: truncate d19/d1a/d20/d89/f9d 8235 0 2024-09-16T00:09:44.234 INFO:tasks.workunit.client.0.smithi139.stdout:0/404: truncate d6/d1c/f21 305836 0 2024-09-16T00:09:44.236 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T00:09:44.236 INFO:tasks.workunit.client.0.smithi139.stdout:0/405: symlink d6/d1c/d41/d63/l80 0 2024-09-16T00:09:44.238 INFO:tasks.workunit.client.0.smithi139.stdout:0/406: mkdir d6/d5a/d81 0 2024-09-16T00:09:44.239 INFO:tasks.workunit.client.0.smithi139.stdout:9/472: write fa [1563257,116164] 0 2024-09-16T00:09:44.239 INFO:tasks.workunit.client.0.smithi139.stdout:0/407: truncate d6/d1c/d41/f4e 530113 0 2024-09-16T00:09:44.240 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:44.240 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:44.241 INFO:tasks.workunit.client.0.smithi139.stdout:0/408: mkdir d6/d1c/d41/d63/d82 0 2024-09-16T00:09:44.242 INFO:tasks.workunit.client.0.smithi139.stdout:9/473: dread d19/d1a/f1e [0,4194304] 0 2024-09-16T00:09:44.242 INFO:tasks.workunit.client.0.smithi139.stdout:0/409: chown d6/l4a 2 1 2024-09-16T00:09:44.243 INFO:tasks.workunit.client.0.smithi139.stdout:0/410: write d6/d1c/d41/d44/f3a [1773091,7957] 0 2024-09-16T00:09:44.244 INFO:tasks.workunit.client.0.smithi139.stdout:9/474: mkdir d19/d1a/d20/d23/d2d/d41/d9f 0 2024-09-16T00:09:44.246 INFO:tasks.workunit.client.0.smithi139.stdout:0/411: creat d6/d1c/d41/d63/d82/f83 x:0 0 0 2024-09-16T00:09:44.246 INFO:tasks.workunit.client.0.smithi139.stdout:9/475: write d19/d1a/d20/d23/d2d/d35/d42/f34 [1479813,6652] 0 2024-09-16T00:09:44.248 INFO:tasks.workunit.client.0.smithi139.stdout:0/412: unlink d6/d1c/d41/d44/c51 0 2024-09-16T00:09:44.250 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:09:44.250 INFO:tasks.workunit.client.0.smithi139.stdout:9/476: creat d19/d1a/d20/d23/d2d/fa0 x:0 0 0 2024-09-16T00:09:44.253 INFO:tasks.workunit.client.0.smithi139.stdout:9/477: creat d19/d1a/d20/d23/d2d/d9a/fa1 x:0 0 0 2024-09-16T00:09:44.254 INFO:tasks.workunit.client.0.smithi139.stdout:9/478: symlink d19/d1a/d20/la2 0 2024-09-16T00:09:44.257 INFO:tasks.workunit.client.0.smithi139.stdout:0/413: write d6/d1c/d41/d44/dd/f2c [762196,61204] 0 2024-09-16T00:09:44.260 INFO:tasks.workunit.client.0.smithi139.stdout:9/479: dwrite d19/d86/f5a [0,4194304] 0 2024-09-16T00:09:44.261 INFO:tasks.workunit.client.0.smithi139.stdout:0/414: truncate d6/d1c/d41/d63/d82/f83 932566 0 2024-09-16T00:09:44.261 INFO:tasks.workunit.client.0.smithi139.stdout:9/480: stat d19/d1a/d20/d23/d2d/d41/d71/d32/d4e/d67 0 2024-09-16T00:09:44.263 INFO:tasks.workunit.client.0.smithi139.stdout:9/481: stat d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/c8e 0 2024-09-16T00:09:44.266 INFO:tasks.workunit.client.0.smithi139.stdout:9/482: mkdir d19/d1a/d20/d23/d2d/d41/d71/da3 0 2024-09-16T00:09:44.267 INFO:tasks.workunit.client.0.smithi139.stdout:0/415: symlink d6/d1c/d41/d44/dd/l84 0 2024-09-16T00:09:44.268 INFO:tasks.workunit.client.0.smithi139.stdout:0/416: read d6/d1c/d41/d44/f3a [374561,105966] 0 2024-09-16T00:09:44.268 INFO:tasks.workunit.client.0.smithi139.stdout:9/483: write d19/d1a/d20/d23/d2d/d41/d71/d32/d79/f5d [4783741,36900] 0 2024-09-16T00:09:44.271 INFO:tasks.workunit.client.0.smithi139.stdout:0/417: rmdir d6/d5a/d70/d7f 0 2024-09-16T00:09:44.272 INFO:tasks.workunit.client.0.smithi139.stdout:0/418: dread - d6/d1c/d41/d44/d6b/f6c zero size 2024-09-16T00:09:44.278 INFO:tasks.workunit.client.0.smithi139.stdout:0/419: dwrite d6/d1c/d41/d44/dd/f67 [0,4194304] 0 2024-09-16T00:09:44.281 INFO:tasks.workunit.client.0.smithi139.stdout:0/420: creat d6/d1c/d41/d63/d82/f85 x:0 0 0 2024-09-16T00:09:44.294 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:44.293+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:44.295 INFO:tasks.workunit.client.0.smithi139.stdout:0/421: dread d6/f8 [0,4194304] 0 2024-09-16T00:09:44.297 INFO:tasks.workunit.client.0.smithi139.stdout:0/422: fsync d6/d1c/f21 0 2024-09-16T00:09:44.299 INFO:tasks.workunit.client.0.smithi139.stdout:0/423: mkdir d6/d1c/d41/d7c/d86 0 2024-09-16T00:09:44.300 INFO:tasks.workunit.client.0.smithi139.stdout:0/424: read - d6/d1c/d41/d44/dd/f46 zero size 2024-09-16T00:09:44.303 INFO:tasks.workunit.client.0.smithi139.stdout:0/425: creat d6/d1c/d45/f87 x:0 0 0 2024-09-16T00:09:44.304 INFO:tasks.workunit.client.0.smithi139.stdout:0/426: rmdir d6/d42 39 2024-09-16T00:09:44.305 INFO:tasks.workunit.client.0.smithi139.stdout:0/427: fdatasync d6/d5a/f6f 0 2024-09-16T00:09:44.305 INFO:tasks.workunit.client.0.smithi139.stdout:0/428: stat d6/d1c/d41/d44 0 2024-09-16T00:09:44.305 INFO:tasks.workunit.client.0.smithi139.stdout:0/429: write d6/d1c/d41/d78/f7d [837463,43964] 0 2024-09-16T00:09:44.306 INFO:tasks.workunit.client.0.smithi139.stdout:0/430: getdents d6/d1c/d41/d76 0 2024-09-16T00:09:44.309 INFO:tasks.workunit.client.0.smithi139.stdout:0/431: dread d6/d1c/d41/d44/f29 [0,4194304] 0 2024-09-16T00:09:44.312 INFO:tasks.workunit.client.0.smithi139.stdout:0/432: getdents d6/d1c/d41/d63 0 2024-09-16T00:09:44.314 INFO:tasks.workunit.client.0.smithi139.stdout:0/433: creat d6/d1c/d41/d44/d58/f88 x:0 0 0 2024-09-16T00:09:44.316 INFO:tasks.workunit.client.0.smithi139.stdout:0/434: unlink d6/d42/c74 0 2024-09-16T00:09:44.319 INFO:tasks.workunit.client.0.smithi139.stdout:0/435: link d6/d1c/d41/d44/dd/f40 d6/d1c/d41/d63/f89 0 2024-09-16T00:09:44.321 INFO:tasks.workunit.client.0.smithi139.stdout:0/436: chown d6/d1c/d41/d44/l22 30 1 2024-09-16T00:09:44.322 INFO:tasks.workunit.client.0.smithi139.stdout:0/437: chown d6/d1c/d41/d44/dd/l2b 31 1 2024-09-16T00:09:44.324 INFO:tasks.workunit.client.0.smithi139.stdout:0/438: mkdir d6/d1c/d41/d44/dd/d8a 0 2024-09-16T00:09:44.325 INFO:tasks.workunit.client.0.smithi139.stdout:0/439: truncate d6/d1c/d41/d44/d58/f60 860918 0 2024-09-16T00:09:44.326 INFO:tasks.workunit.client.0.smithi139.stdout:0/440: read - d6/d1c/d41/d44/d58/f5f zero size 2024-09-16T00:09:44.327 INFO:tasks.workunit.client.0.smithi139.stdout:0/441: mknod d6/d1c/d41/d7c/d86/c8b 0 2024-09-16T00:09:44.329 INFO:tasks.workunit.client.0.smithi139.stdout:0/442: creat d6/d1c/f8c x:0 0 0 2024-09-16T00:09:44.330 INFO:tasks.workunit.client.0.smithi139.stdout:0/443: rmdir d6/d1c/d41/d63/d82 39 2024-09-16T00:09:44.331 INFO:tasks.workunit.client.0.smithi139.stdout:0/444: rmdir d6/d5a 39 2024-09-16T00:09:44.332 INFO:tasks.workunit.client.0.smithi139.stdout:0/445: fsync d6/d1c/d41/f6a 0 2024-09-16T00:09:44.334 INFO:tasks.workunit.client.0.smithi139.stdout:0/446: creat d6/d1c/f8d x:0 0 0 2024-09-16T00:09:44.336 INFO:tasks.workunit.client.0.smithi139.stdout:0/447: creat d6/d1c/d45/f8e x:0 0 0 2024-09-16T00:09:44.336 INFO:tasks.workunit.client.0.smithi139.stdout:0/448: write d6/d1c/d45/f5e [1129699,49241] 0 2024-09-16T00:09:44.338 INFO:tasks.workunit.client.0.smithi139.stdout:0/449: mknod d6/d42/c8f 0 2024-09-16T00:09:44.339 INFO:tasks.workunit.client.0.smithi139.stdout:0/450: chown d6/d1c/d41/d44/dd/l30 67547 1 2024-09-16T00:09:44.344 INFO:tasks.workunit.client.0.smithi139.stdout:0/451: dwrite d6/d1c/d41/d44/f3b [0,4194304] 0 2024-09-16T00:09:44.349 INFO:tasks.workunit.client.0.smithi139.stdout:0/452: unlink d6/l32 0 2024-09-16T00:09:44.351 INFO:tasks.workunit.client.0.smithi139.stdout:0/453: mknod d6/d5a/c90 0 2024-09-16T00:09:44.352 INFO:tasks.workunit.client.0.smithi139.stdout:0/454: chown d6/d5a/d70/l5c 10960 1 2024-09-16T00:09:44.353 INFO:tasks.workunit.client.0.smithi139.stdout:0/455: write d6/d1c/d41/d78/f7d [716443,63205] 0 2024-09-16T00:09:44.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.356+0000 7fdc55f8d640 1 Processor -- start 2024-09-16T00:09:44.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.356+0000 7fdc55f8d640 1 -- start start 2024-09-16T00:09:44.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.357+0000 7fdc55f8d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc50154bc0 0x7fdc5015b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:44.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.357+0000 7fdc55f8d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdc500c79c0 0x7fdc500c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:44.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.357+0000 7fdc55f8d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc500c9800 con 0x7fdc500c7fb0 2024-09-16T00:09:44.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.357+0000 7fdc55f8d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdc5006bbc0 con 0x7fdc500c79c0 2024-09-16T00:09:44.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.357+0000 7fdc55f8d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5006be60 con 0x7fdc50154bc0 2024-09-16T00:09:44.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.357+0000 7fdc47fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdc500c79c0 0x7fdc500c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:44.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.357+0000 7fdc4ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fdc500c7fb0 0x7fdc5015de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45556/0 (socket says 172.21.15.139:45556) 2024-09-16T00:09:44.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.357+0000 7fdc4ffff640 1 -- 172.21.15.139:0/3468748799 learned_addr learned my addr 172.21.15.139:0/3468748799 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:44.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.357+0000 7fdc4f7fe640 1 --2- 172.21.15.139:0/3468748799 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc50154bc0 0x7fdc5015b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:44.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.358+0000 7fdc47fff640 1 -- 172.21.15.139:0/3468748799 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc50154bc0 msgr2=0x7fdc5015b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:44.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.358+0000 7fdc47fff640 1 --2- 172.21.15.139:0/3468748799 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc50154bc0 0x7fdc5015b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:44.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.358+0000 7fdc47fff640 1 -- 172.21.15.139:0/3468748799 >> v1:172.21.15.139:6789/0 conn(0x7fdc500c7fb0 legacy=0x7fdc5015de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:44.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.358+0000 7fdc4effd640 1 -- 172.21.15.139:0/3468748799 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2829649868 0 0) 0x7fdc500c9800 con 0x7fdc500c7fb0 2024-09-16T00:09:44.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.358+0000 7fdc47fff640 1 -- 172.21.15.139:0/3468748799 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc500c9520 con 0x7fdc500c79c0 2024-09-16T00:09:44.362 INFO:tasks.workunit.client.0.smithi139.stdout:0/456: dwrite d6/d1c/f8d [0,4194304] 0 2024-09-16T00:09:44.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.358+0000 7fdc47fff640 1 --2- 172.21.15.139:0/3468748799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc500c79c0 0x7fdc500c8740 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdc380097a0 tx=0x7fdc3802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c18823dc9e8f0f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:44.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.359+0000 7fdc4effd640 1 -- 172.21.15.139:0/3468748799 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38037070 con 0x7fdc500c79c0 2024-09-16T00:09:44.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.359+0000 7fdc4effd640 1 -- 172.21.15.139:0/3468748799 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc38032440 con 0x7fdc500c79c0 2024-09-16T00:09:44.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.359+0000 7fdc4effd640 1 -- 172.21.15.139:0/3468748799 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38032810 con 0x7fdc500c79c0 2024-09-16T00:09:44.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.360+0000 7fdc55f8d640 1 -- 172.21.15.139:0/3468748799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc500c79c0 msgr2=0x7fdc500c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:44.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.360+0000 7fdc55f8d640 1 --2- 172.21.15.139:0/3468748799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc500c79c0 0x7fdc500c8740 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdc380097a0 tx=0x7fdc3802a180 comp rx=0 tx=0).stop 2024-09-16T00:09:44.363 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.360+0000 7fdc55f8d640 1 -- 172.21.15.139:0/3468748799 shutdown_connections 2024-09-16T00:09:44.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.360+0000 7fdc55f8d640 1 --2- 172.21.15.139:0/3468748799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc500c79c0 0x7fdc500c8740 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:44.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.360+0000 7fdc55f8d640 1 --2- 172.21.15.139:0/3468748799 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc50154bc0 0x7fdc5015b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:44.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.360+0000 7fdc55f8d640 1 -- 172.21.15.139:0/3468748799 >> 172.21.15.139:0/3468748799 conn(0x7fdc50070460 msgr2=0x7fdc50070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:44.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.360+0000 7fdc55f8d640 1 -- 172.21.15.139:0/3468748799 shutdown_connections 2024-09-16T00:09:44.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.360+0000 7fdc55f8d640 1 -- 172.21.15.139:0/3468748799 wait complete. 2024-09-16T00:09:44.364 INFO:tasks.workunit.client.0.smithi139.stdout:0/457: write d6/d1c/d41/d44/dd/d56/f77 [850851,124095] 0 2024-09-16T00:09:44.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.361+0000 7fdc55f8d640 1 Processor -- start 2024-09-16T00:09:44.365 INFO:tasks.workunit.client.0.smithi139.stdout:0/458: fsync d6/d5a/d70/f53 0 2024-09-16T00:09:44.365 INFO:tasks.workunit.client.0.smithi139.stdout:0/459: write d6/d1c/d45/f47 [4144016,45726] 0 2024-09-16T00:09:44.368 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:44.368+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:44.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.362+0000 7fdc55f8d640 1 -- start start 2024-09-16T00:09:44.369 INFO:tasks.workunit.client.0.smithi139.stdout:4/335: sync 2024-09-16T00:09:44.370 INFO:tasks.workunit.client.0.smithi139.stdout:3/400: sync 2024-09-16T00:09:44.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc55f8d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc500c79c0 0x7fdc50148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:44.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc55f8d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc500c7fb0 0x7fdc500ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:44.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc55f8d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc50154bc0 0x7fdc50158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:44.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc55f8d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdc50154fa0 con 0x7fdc50154bc0 2024-09-16T00:09:44.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc55f8d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdc500c9220 con 0x7fdc500c7fb0 2024-09-16T00:09:44.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc55f8d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5015b770 con 0x7fdc500c79c0 2024-09-16T00:09:44.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc500c7fb0 0x7fdc500ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:44.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc4f7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc500c79c0 0x7fdc50148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:44.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc47fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc500c7fb0 0x7fdc500ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:47208/0 (socket says 172.21.15.139:47208) 2024-09-16T00:09:44.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc4ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc50154bc0 0x7fdc50158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:44.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc4f7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc500c79c0 0x7fdc50148be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:35282/0 (socket says 172.21.15.139:35282) 2024-09-16T00:09:44.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc47fff640 1 -- 172.21.15.139:0/2943811187 learned_addr learned my addr 172.21.15.139:0/2943811187 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:44.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc47fff640 1 -- 172.21.15.139:0/2943811187 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc500c79c0 msgr2=0x7fdc50148be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:44.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc47fff640 1 --2- 172.21.15.139:0/2943811187 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc500c79c0 0x7fdc50148be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:44.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc47fff640 1 -- 172.21.15.139:0/2943811187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc50154bc0 msgr2=0x7fdc50158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:44.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc47fff640 1 --2- 172.21.15.139:0/2943811187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc50154bc0 0x7fdc50158890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:44.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc47fff640 1 -- 172.21.15.139:0/2943811187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc500cc0b0 con 0x7fdc500c7fb0 2024-09-16T00:09:44.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.363+0000 7fdc4ffff640 1 --2- 172.21.15.139:0/2943811187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc50154bc0 0x7fdc50158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:44.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.364+0000 7fdc47fff640 1 --2- 172.21.15.139:0/2943811187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc500c7fb0 0x7fdc500ccdc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdc380098c0 tx=0x7fdc380046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:44.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.364+0000 7fdc4cff9640 1 -- 172.21.15.139:0/2943811187 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc38037040 con 0x7fdc500c7fb0 2024-09-16T00:09:44.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.364+0000 7fdc4cff9640 1 -- 172.21.15.139:0/2943811187 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc38030500 con 0x7fdc500c7fb0 2024-09-16T00:09:44.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.364+0000 7fdc55f8d640 1 -- 172.21.15.139:0/2943811187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc500cc370 con 0x7fdc500c7fb0 2024-09-16T00:09:44.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.364+0000 7fdc4cff9640 1 -- 172.21.15.139:0/2943811187 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc380308d0 con 0x7fdc500c7fb0 2024-09-16T00:09:44.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.364+0000 7fdc55f8d640 1 -- 172.21.15.139:0/2943811187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc501f22d0 con 0x7fdc500c7fb0 2024-09-16T00:09:44.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.365+0000 7fdc55f8d640 1 -- 172.21.15.139:0/2943811187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5006dc90 con 0x7fdc500c7fb0 2024-09-16T00:09:44.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.366+0000 7fdc4cff9640 1 -- 172.21.15.139:0/2943811187 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdc38044020 con 0x7fdc500c7fb0 2024-09-16T00:09:44.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.366+0000 7fdc4cff9640 1 --2- 172.21.15.139:0/2943811187 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdc3006c2b0 0x7fdc3006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:44.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.366+0000 7fdc4cff9640 1 -- 172.21.15.139:0/2943811187 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5973+0+0 (secure 0 0 0) 0x7fdc380ab5b0 con 0x7fdc500c7fb0 2024-09-16T00:09:44.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.367+0000 7fdc4f7fe640 1 --2- 172.21.15.139:0/2943811187 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdc3006c2b0 0x7fdc3006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:44.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.367+0000 7fdc4f7fe640 1 --2- 172.21.15.139:0/2943811187 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdc3006c2b0 0x7fdc3006e770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009aa0 tx=0x7fdc40009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:44.375 INFO:tasks.workunit.client.0.smithi139.stdout:4/336: readlink d12/d4a/l6d 0 2024-09-16T00:09:44.375 INFO:tasks.workunit.client.0.smithi139.stdout:3/401: write d5/d7/d23/d27/f83 [487096,63857] 0 2024-09-16T00:09:44.376 INFO:tasks.workunit.client.0.smithi139.stdout:4/337: symlink d12/d4a/l6f 0 2024-09-16T00:09:44.376 INFO:tasks.workunit.client.0.smithi139.stdout:0/460: dwrite d6/d1c/d41/d44/dd/f2d [0,4194304] 0 2024-09-16T00:09:44.376 INFO:tasks.workunit.client.0.smithi139.stdout:3/402: creat d5/d7/d23/f94 x:0 0 0 2024-09-16T00:09:44.377 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:44.377 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:44.377 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:44.377 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:44.377 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:44.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.375+0000 7fdc4cff9640 1 -- 172.21.15.139:0/2943811187 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdc5006dc90 con 0x7fdc500c7fb0 2024-09-16T00:09:44.377 INFO:tasks.workunit.client.0.smithi139.stdout:4/338: rename d12/d47/d49/l4b to d12/l70 0 2024-09-16T00:09:44.379 INFO:tasks.workunit.client.0.smithi139.stdout:0/461: link d6/d1c/f36 d6/d1c/d41/d7c/d86/f91 0 2024-09-16T00:09:44.381 INFO:tasks.workunit.client.0.smithi139.stdout:3/403: dread d5/f8 [0,4194304] 0 2024-09-16T00:09:44.381 INFO:tasks.workunit.client.0.smithi139.stdout:0/462: write d6/d1c/d45/f8e [755901,103081] 0 2024-09-16T00:09:44.384 INFO:tasks.workunit.client.0.smithi139.stdout:0/463: readlink d6/d1c/d41/d44/dd/le 0 2024-09-16T00:09:44.384 INFO:tasks.workunit.client.0.smithi139.stdout:3/404: mkdir d5/d7/d23/d27/d2f/d30/d4d/d95 0 2024-09-16T00:09:44.386 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:09:44.386 INFO:tasks.workunit.client.0.smithi139.stdout:3/405: unlink d5/d7/d23/d27/d2f/f72 0 2024-09-16T00:09:44.388 INFO:tasks.workunit.client.0.smithi139.stdout:0/464: link f2 d6/d1c/d41/d44/dd/d8a/f92 0 2024-09-16T00:09:44.389 INFO:tasks.workunit.client.0.smithi139.stdout:3/406: symlink d5/d7/d23/d48/l96 0 2024-09-16T00:09:44.390 INFO:tasks.workunit.client.0.smithi139.stdout:3/407: mknod d5/d7/d23/d27/d2f/d30/d41/c97 0 2024-09-16T00:09:44.392 INFO:tasks.workunit.client.0.smithi139.stdout:4/339: dread d12/d47/f2c [4194304,4194304] 0 2024-09-16T00:09:44.393 INFO:tasks.workunit.client.0.smithi139.stdout:3/408: creat d5/d7/d23/d27/d2f/d30/d4c/d6d/d76/f98 x:0 0 0 2024-09-16T00:09:44.394 INFO:tasks.workunit.client.0.smithi139.stdout:3/409: stat d5/d7/dc/c1b 0 2024-09-16T00:09:44.395 INFO:tasks.workunit.client.0.smithi139.stdout:3/410: chown d5/d7/d23/d27/c8e 2904 1 2024-09-16T00:09:44.395 INFO:tasks.workunit.client.0.smithi139.stdout:4/340: dread d12/f23 [4194304,4194304] 0 2024-09-16T00:09:44.397 INFO:tasks.workunit.client.0.smithi139.stdout:3/411: write d5/d7/d23/d27/d2f/d30/d47/f25 [821008,102817] 0 2024-09-16T00:09:44.400 INFO:tasks.workunit.client.0.smithi139.stdout:3/412: symlink d5/d7/d23/d27/d2f/d30/d4c/d6d/d76/l99 0 2024-09-16T00:09:44.402 INFO:tasks.workunit.client.0.smithi139.stdout:3/413: mkdir d5/d7/d23/d48/d9a 0 2024-09-16T00:09:44.403 INFO:tasks.workunit.client.0.smithi139.stdout:3/414: write d5/d7/d23/d35/f45 [241203,120000] 0 2024-09-16T00:09:44.404 INFO:tasks.workunit.client.0.smithi139.stdout:3/415: chown d5/d7 2056955 1 2024-09-16T00:09:44.407 INFO:tasks.workunit.client.0.smithi139.stdout:4/341: dread d12/d4a/f63 [0,4194304] 0 2024-09-16T00:09:44.407 INFO:tasks.workunit.client.0.smithi139.stdout:3/416: dread d5/d7/f37 [0,4194304] 0 2024-09-16T00:09:44.409 INFO:tasks.workunit.client.0.smithi139.stdout:3/417: symlink d5/d7/d23/d27/d2f/l9b 0 2024-09-16T00:09:44.411 INFO:tasks.workunit.client.0.smithi139.stdout:3/418: mkdir d5/d7/dc/d9c 0 2024-09-16T00:09:44.413 INFO:tasks.workunit.client.0.smithi139.stdout:4/342: dwrite d12/d47/d57/f62 [4194304,4194304] 0 2024-09-16T00:09:44.414 INFO:tasks.workunit.client.0.smithi139.stdout:4/343: symlink d12/d47/d57/l71 0 2024-09-16T00:09:44.415 INFO:tasks.workunit.client.0.smithi139.stdout:3/419: rmdir d5/d7/d23/d27/d2f/d30/d4c/d92 0 2024-09-16T00:09:44.417 INFO:tasks.workunit.client.0.smithi139.stdout:4/344: creat d12/d4a/d4d/d69/f72 x:0 0 0 2024-09-16T00:09:44.418 INFO:tasks.workunit.client.0.smithi139.stdout:3/420: creat d5/d7/dc/d9c/f9d x:0 0 0 2024-09-16T00:09:44.419 INFO:tasks.workunit.client.0.smithi139.stdout:4/345: rename f1 to d12/d2b/f73 0 2024-09-16T00:09:44.420 INFO:tasks.workunit.client.0.smithi139.stdout:4/346: chown d12/d4a/d4d 6 1 2024-09-16T00:09:44.423 INFO:tasks.workunit.client.0.smithi139.stdout:4/347: link f7 d12/d47/d49/d60/f74 0 2024-09-16T00:09:44.425 INFO:tasks.workunit.client.0.smithi139.stdout:4/348: symlink d12/l75 0 2024-09-16T00:09:44.425 INFO:tasks.workunit.client.0.smithi139.stdout:4/349: write d12/d47/d57/f6e [191345,125463] 0 2024-09-16T00:09:44.426 INFO:tasks.workunit.client.0.smithi139.stdout:4/350: stat d12/d2b/f73 0 2024-09-16T00:09:44.427 INFO:tasks.workunit.client.0.smithi139.stdout:4/351: stat d12/d2b/c48 0 2024-09-16T00:09:44.430 INFO:tasks.workunit.client.0.smithi139.stdout:3/421: write d5/d7/fd [4860471,8039] 0 2024-09-16T00:09:44.432 INFO:tasks.workunit.client.0.smithi139.stdout:4/352: dwrite d12/d2b/f5a [0,4194304] 0 2024-09-16T00:09:44.433 INFO:tasks.workunit.client.0.smithi139.stdout:3/422: creat d5/d7/d23/d27/d2f/d30/d4c/d6d/f9e x:0 0 0 2024-09-16T00:09:44.435 INFO:tasks.workunit.client.0.smithi139.stdout:4/353: unlink d12/f3f 0 2024-09-16T00:09:44.435 INFO:tasks.workunit.client.0.smithi139.stdout:4/354: readlink d12/d47/l39 0 2024-09-16T00:09:44.437 INFO:tasks.workunit.client.0.smithi139.stdout:4/355: write d12/d47/d57/f6e [1081029,1040] 0 2024-09-16T00:09:44.439 INFO:tasks.workunit.client.0.smithi139.stdout:4/356: creat d12/d47/d49/d60/f76 x:0 0 0 2024-09-16T00:09:44.441 INFO:tasks.workunit.client.0.smithi139.stdout:3/423: dwrite d5/d7/d23/d27/d2f/d30/d4c/d50/f54 [0,4194304] 0 2024-09-16T00:09:44.442 INFO:tasks.workunit.client.0.smithi139.stdout:4/357: rename d12/d2b/f18 to d12/d4a/f77 0 2024-09-16T00:09:44.443 INFO:tasks.workunit.client.0.smithi139.stdout:3/424: write d5/d7/d23/d27/d2f/d30/f42 [1271802,18079] 0 2024-09-16T00:09:44.445 INFO:tasks.workunit.client.0.smithi139.stdout:3/425: dread - d5/d7/d23/d27/d2f/d30/d4c/d6d/d76/f98 zero size 2024-09-16T00:09:44.445 INFO:tasks.workunit.client.0.smithi139.stdout:4/358: creat d12/d2b/f78 x:0 0 0 2024-09-16T00:09:44.446 INFO:tasks.workunit.client.0.smithi139.stdout:3/426: rmdir d5/d7/dc 39 2024-09-16T00:09:44.448 INFO:tasks.workunit.client.0.smithi139.stdout:4/359: creat d12/d4a/d4d/f79 x:0 0 0 2024-09-16T00:09:44.449 INFO:tasks.workunit.client.0.smithi139.stdout:3/427: rename d5/d7/d23/c66 to d5/d7/d23/d27/d2f/d30/d47/c9f 0 2024-09-16T00:09:44.450 INFO:tasks.workunit.client.0.smithi139.stdout:3/428: write d5/d7/d23/d27/d2f/d30/d41/f7f [32240,46303] 0 2024-09-16T00:09:44.452 INFO:tasks.workunit.client.0.smithi139.stdout:4/360: rename d12/l2d to d12/d4a/d4d/l7a 0 2024-09-16T00:09:44.453 INFO:tasks.workunit.client.0.smithi139.stdout:4/361: mkdir d12/d47/d49/d7b 0 2024-09-16T00:09:44.457 INFO:tasks.workunit.client.0.smithi139.stdout:3/429: dwrite d5/d7/d23/d27/d2f/d30/f80 [0,4194304] 0 2024-09-16T00:09:44.459 INFO:tasks.workunit.client.0.smithi139.stdout:3/430: read d5/d7/d23/d35/f45 [244088,25917] 0 2024-09-16T00:09:44.459 INFO:tasks.workunit.client.0.smithi139.stdout:4/362: dwrite d12/f40 [0,4194304] 0 2024-09-16T00:09:44.463 INFO:tasks.workunit.client.0.smithi139.stdout:4/363: dread d12/f31 [0,4194304] 0 2024-09-16T00:09:44.465 INFO:tasks.workunit.client.0.smithi139.stdout:4/364: truncate d12/d2b/f6b 605744 0 2024-09-16T00:09:44.468 INFO:tasks.workunit.client.0.smithi139.stdout:4/365: getdents d12/d47/d49/d7b 0 2024-09-16T00:09:44.469 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:44.468+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:44.469 INFO:tasks.workunit.client.0.smithi139.stdout:4/366: mknod d12/d47/d49/d60/c7c 0 2024-09-16T00:09:44.470 INFO:tasks.workunit.client.0.smithi139.stdout:3/431: link d5/d7/dc/c43 d5/d7/dc/d9c/ca0 0 2024-09-16T00:09:44.475 INFO:tasks.workunit.client.0.smithi139.stdout:4/367: dwrite d12/d2b/f78 [0,4194304] 0 2024-09-16T00:09:44.476 INFO:tasks.workunit.client.0.smithi139.stdout:4/368: chown d12/d47/d49/d60/f66 6406 1 2024-09-16T00:09:44.479 INFO:tasks.workunit.client.0.smithi139.stdout:4/369: rename d12/d47/c37 to d12/d47/d49/d60/c7d 0 2024-09-16T00:09:44.484 INFO:tasks.workunit.client.0.smithi139.stdout:4/370: dwrite d12/d4a/d4d/d69/f72 [0,4194304] 0 2024-09-16T00:09:44.487 INFO:tasks.workunit.client.0.smithi139.stdout:4/371: unlink d12/f4c 0 2024-09-16T00:09:44.490 INFO:tasks.workunit.client.0.smithi139.stdout:8/399: sync 2024-09-16T00:09:44.490 INFO:tasks.workunit.client.0.smithi139.stdout:4/372: link d12/f43 d12/d47/d49/f7e 0 2024-09-16T00:09:44.491 INFO:tasks.workunit.client.0.smithi139.stdout:8/400: chown d17/d1a/d43/f42 42 1 2024-09-16T00:09:44.493 INFO:tasks.workunit.client.0.smithi139.stdout:4/373: rename d12/d47/d49/d7b to d12/d47/d57/d7f 0 2024-09-16T00:09:44.493 INFO:tasks.workunit.client.0.smithi139.stdout:8/401: mkdir d17/d1f/d88 0 2024-09-16T00:09:44.494 INFO:tasks.workunit.client.0.smithi139.stdout:4/374: truncate d12/d4a/d4d/f55 716719 0 2024-09-16T00:09:44.495 INFO:tasks.workunit.client.0.smithi139.stdout:8/402: truncate d17/d1a/d57/f71 781862 0 2024-09-16T00:09:44.495 INFO:tasks.workunit.client.0.smithi139.stdout:4/375: write d12/d47/d49/f53 [2915699,23474] 0 2024-09-16T00:09:44.495 INFO:tasks.workunit.client.0.smithi139.stdout:8/403: stat d17/d1a/d57/f82 0 2024-09-16T00:09:44.496 INFO:tasks.workunit.client.0.smithi139.stdout:8/404: fdatasync f7 0 2024-09-16T00:09:44.496 INFO:tasks.workunit.client.0.smithi139.stdout:8/405: fsync d17/d1a/f35 0 2024-09-16T00:09:44.497 INFO:tasks.workunit.client.0.smithi139.stdout:4/376: fsync f7 0 2024-09-16T00:09:44.499 INFO:tasks.workunit.client.0.smithi139.stdout:8/406: mkdir d17/d89 0 2024-09-16T00:09:44.499 INFO:tasks.workunit.client.0.smithi139.stdout:4/377: dread d12/d47/f32 [4194304,4194304] 0 2024-09-16T00:09:44.500 INFO:tasks.workunit.client.0.smithi139.stdout:8/407: dread - d17/d1a/d2d/f85 zero size 2024-09-16T00:09:44.500 INFO:tasks.workunit.client.0.smithi139.stdout:4/378: fdatasync d12/d4a/d4d/d69/f72 0 2024-09-16T00:09:44.501 INFO:tasks.workunit.client.0.smithi139.stdout:8/408: chown d17/d1f/d3a/c78 14 1 2024-09-16T00:09:44.501 INFO:tasks.workunit.client.0.smithi139.stdout:4/379: write d12/d2b/f78 [965166,120627] 0 2024-09-16T00:09:44.504 INFO:tasks.workunit.client.0.smithi139.stdout:8/409: symlink d17/d1a/d2d/l8a 0 2024-09-16T00:09:44.504 INFO:tasks.workunit.client.0.smithi139.stdout:8/410: chown d17/d1a/d43/d40/d4f/l5c 62500976 1 2024-09-16T00:09:44.518 INFO:tasks.workunit.client.0.smithi139.stdout:4/380: truncate d12/d2b/f78 1195778 0 2024-09-16T00:09:44.522 INFO:tasks.workunit.client.0.smithi139.stdout:8/411: dwrite fb [0,4194304] 0 2024-09-16T00:09:44.525 INFO:tasks.workunit.client.0.smithi139.stdout:8/412: creat d17/d1a/d57/f8b x:0 0 0 2024-09-16T00:09:44.526 INFO:tasks.workunit.client.0.smithi139.stdout:4/381: dwrite d12/d47/d57/f59 [4194304,4194304] 0 2024-09-16T00:09:44.527 INFO:tasks.workunit.client.0.smithi139.stdout:8/413: readlink d17/d1a/d50/l77 0 2024-09-16T00:09:44.528 INFO:tasks.workunit.client.0.smithi139.stdout:8/414: write d17/f83 [414982,7072] 0 2024-09-16T00:09:44.530 INFO:tasks.workunit.client.0.smithi139.stdout:4/382: mkdir d12/d80 0 2024-09-16T00:09:44.531 INFO:tasks.workunit.client.0.smithi139.stdout:8/415: symlink d17/d1a/d2d/l8c 0 2024-09-16T00:09:44.531 INFO:tasks.workunit.client.0.smithi139.stdout:4/383: fdatasync d12/d47/f45 0 2024-09-16T00:09:44.534 INFO:tasks.workunit.client.0.smithi139.stdout:8/416: creat d17/d1a/d43/d40/f8d x:0 0 0 2024-09-16T00:09:44.537 INFO:tasks.workunit.client.0.smithi139.stdout:8/417: getdents d17 0 2024-09-16T00:09:44.540 INFO:tasks.workunit.client.0.smithi139.stdout:8/418: creat d17/d1f/d88/f8e x:0 0 0 2024-09-16T00:09:44.541 INFO:tasks.workunit.client.0.smithi139.stdout:8/419: stat d17/d1f/l48 0 2024-09-16T00:09:44.541 INFO:tasks.workunit.client.0.smithi139.stdout:4/384: write d12/d47/d49/f7e [983132,39647] 0 2024-09-16T00:09:44.541 INFO:tasks.workunit.client.0.smithi139.stdout:4/385: stat d12/d47 0 2024-09-16T00:09:44.543 INFO:tasks.workunit.client.0.smithi139.stdout:4/386: write d12/d2b/f78 [844248,10517] 0 2024-09-16T00:09:44.544 INFO:tasks.workunit.client.0.smithi139.stdout:8/420: creat d17/d89/f8f x:0 0 0 2024-09-16T00:09:44.544 INFO:tasks.workunit.client.0.smithi139.stdout:4/387: readlink d12/d4a/l6d 0 2024-09-16T00:09:44.544 INFO:tasks.workunit.client.0.smithi139.stdout:4/388: fdatasync d12/f4f 0 2024-09-16T00:09:44.548 INFO:tasks.workunit.client.0.smithi139.stdout:4/389: getdents d12/d4a/d4d 0 2024-09-16T00:09:44.550 INFO:tasks.workunit.client.0.smithi139.stdout:8/421: dwrite d17/d1a/f73 [0,4194304] 0 2024-09-16T00:09:44.551 INFO:tasks.workunit.client.0.smithi139.stdout:4/390: dread d12/d4a/d4d/f55 [0,4194304] 0 2024-09-16T00:09:44.552 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:44.552 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:44.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:44.552 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:44.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.552 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:44.552 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:44.552 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 141.469481433, 2024-09-16T00:09:44.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:09:44.553 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.553 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.553 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:44.553 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:44.553 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:09:44.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.553 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:44.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:44.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:09:44.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:09:44.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:09:44.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:09:44.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:09:44.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:09:44.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:09:44.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:09:44.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:09:44.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:09:44.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:09:44.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:09:44.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:09:44.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:09:44.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:09:44.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:09:44.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:09:44.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:44.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:09:44.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:09:44.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:09:44.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:09:44.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:09:44.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:09:44.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:09:44.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:09:44.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:09:44.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:09:44.559 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.559 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.559 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.559 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:44.559 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:44.559 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 139.940390554, 2024-09-16T00:09:44.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014331501999999999, 2024-09-16T00:09:44.559 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.560 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.560 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.560 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.560 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:44.560 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:44.560 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-16T00:09:44.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.560 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:44.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:44.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609921+0000", 2024-09-16T00:09:44.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9599999999999999e-07 2024-09-16T00:09:44.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609925+0000", 2024-09-16T00:09:44.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T00:09:44.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609932+0000", 2024-09-16T00:09:44.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8240000000000002e-06 2024-09-16T00:09:44.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609937+0000", 2024-09-16T00:09:44.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1070000000000004e-06 2024-09-16T00:09:44.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618987+0000", 2024-09-16T00:09:44.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090502719999999998 2024-09-16T00:09:44.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619064+0000", 2024-09-16T00:09:44.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6868999999999999e-05 2024-09-16T00:09:44.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:44.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619183+0000", 2024-09-16T00:09:44.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011961 2024-09-16T00:09:44.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.620052+0000", 2024-09-16T00:09:44.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086822799999999997 2024-09-16T00:09:44.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624197+0000", 2024-09-16T00:09:44.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041454930000000001 2024-09-16T00:09:44.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624229+0000", 2024-09-16T00:09:44.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1758999999999999e-05 2024-09-16T00:09:44.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624252+0000", 2024-09-16T00:09:44.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:09:44.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.565 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.565 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.565 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:161 2.b 2:d5c7a900:::200.00000003:head [write 0~3935733 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e64)", 2024-09-16T00:09:44.565 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:07.538495+0000", 2024-09-16T00:09:44.565 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.011815900000002, 2024-09-16T00:09:44.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012844983000000001, 2024-09-16T00:09:44.566 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.566 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.566 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.566 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.538495+0000", 2024-09-16T00:09:44.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.538495+0000", 2024-09-16T00:09:44.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.538497+0000", 2024-09-16T00:09:44.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5149999999999999e-06 2024-09-16T00:09:44.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540617+0000", 2024-09-16T00:09:44.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021204079999999998 2024-09-16T00:09:44.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540640+0000", 2024-09-16T00:09:44.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3167e-05 2024-09-16T00:09:44.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540650+0000", 2024-09-16T00:09:44.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6099999999999995e-06 2024-09-16T00:09:44.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540724+0000", 2024-09-16T00:09:44.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4340999999999999e-05 2024-09-16T00:09:44.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540812+0000", 2024-09-16T00:09:44.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8376999999999998e-05 2024-09-16T00:09:44.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:44.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.540957+0000", 2024-09-16T00:09:44.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014463 2024-09-16T00:09:44.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.550270+0000", 2024-09-16T00:09:44.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093128380000000004 2024-09-16T00:09:44.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551270+0000", 2024-09-16T00:09:44.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00100015 2024-09-16T00:09:44.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551309+0000", 2024-09-16T00:09:44.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8526999999999999e-05 2024-09-16T00:09:44.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:07.551340+0000", 2024-09-16T00:09:44.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1420000000000001e-05 2024-09-16T00:09:44.571 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.571 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.571 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.571 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10651 3.39 3:9d97a463:::100000030ab.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:44.571 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.621786+0000", 2024-09-16T00:09:44.571 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.9285251970000008, 2024-09-16T00:09:44.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015164901999999999, 2024-09-16T00:09:44.571 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.571 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.571 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.571 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.572 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.572 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.572 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10651 2024-09-16T00:09:44.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.572 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.621786+0000", 2024-09-16T00:09:44.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.621786+0000", 2024-09-16T00:09:44.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.621788+0000", 2024-09-16T00:09:44.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7999999999999999e-06 2024-09-16T00:09:44.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.626753+0000", 2024-09-16T00:09:44.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004965667 2024-09-16T00:09:44.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.626771+0000", 2024-09-16T00:09:44.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7816999999999999e-05 2024-09-16T00:09:44.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.626778+0000", 2024-09-16T00:09:44.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3869999999999997e-06 2024-09-16T00:09:44.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.626843+0000", 2024-09-16T00:09:44.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4969999999999996e-05 2024-09-16T00:09:44.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.627002+0000", 2024-09-16T00:09:44.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000158441 2024-09-16T00:09:44.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:44.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.627147+0000", 2024-09-16T00:09:44.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014563999999999999 2024-09-16T00:09:44.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.634047+0000", 2024-09-16T00:09:44.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006899222 2024-09-16T00:09:44.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.636883+0000", 2024-09-16T00:09:44.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028367890000000002 2024-09-16T00:09:44.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.636918+0000", 2024-09-16T00:09:44.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4397999999999998e-05 2024-09-16T00:09:44.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.636951+0000", 2024-09-16T00:09:44.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2771000000000002e-05 2024-09-16T00:09:44.577 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.577 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.577 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.577 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10770 3.15 3:aa8f498f:::100000030ba.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:44.577 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.404839+0000", 2024-09-16T00:09:44.577 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 6.1454720280000004, 2024-09-16T00:09:44.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013644351000000001, 2024-09-16T00:09:44.578 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.578 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.578 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.578 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.578 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.578 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.578 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10770 2024-09-16T00:09:44.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.578 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.578 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.578 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.404839+0000", 2024-09-16T00:09:44.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.404839+0000", 2024-09-16T00:09:44.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.404840+0000", 2024-09-16T00:09:44.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5710000000000001e-06 2024-09-16T00:09:44.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.409487+0000", 2024-09-16T00:09:44.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046467380000000001 2024-09-16T00:09:44.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.409500+0000", 2024-09-16T00:09:44.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3246e-05 2024-09-16T00:09:44.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.409511+0000", 2024-09-16T00:09:44.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0852999999999999e-05 2024-09-16T00:09:44.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.409577+0000", 2024-09-16T00:09:44.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5429e-05 2024-09-16T00:09:44.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.409675+0000", 2024-09-16T00:09:44.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7808999999999998e-05 2024-09-16T00:09:44.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:44.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.409770+0000", 2024-09-16T00:09:44.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5110000000000002e-05 2024-09-16T00:09:44.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.414205+0000", 2024-09-16T00:09:44.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044357420000000003 2024-09-16T00:09:44.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.418378+0000", 2024-09-16T00:09:44.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041728989999999999 2024-09-16T00:09:44.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.418441+0000", 2024-09-16T00:09:44.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2438000000000001e-05 2024-09-16T00:09:44.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.418483+0000", 2024-09-16T00:09:44.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2515999999999997e-05 2024-09-16T00:09:44.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.583 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.583 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.583 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10773 3.39 3:9d207bac:::100000030d8.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:44.583 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.423389+0000", 2024-09-16T00:09:44.583 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 6.1269217830000002, 2024-09-16T00:09:44.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014464233999999999, 2024-09-16T00:09:44.584 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.584 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.584 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.584 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.584 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10773 2024-09-16T00:09:44.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.584 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.584 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.584 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.423389+0000", 2024-09-16T00:09:44.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.423389+0000", 2024-09-16T00:09:44.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.423391+0000", 2024-09-16T00:09:44.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.406e-06 2024-09-16T00:09:44.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.427961+0000", 2024-09-16T00:09:44.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045706740000000003 2024-09-16T00:09:44.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.427974+0000", 2024-09-16T00:09:44.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2388000000000001e-05 2024-09-16T00:09:44.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.427980+0000", 2024-09-16T00:09:44.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3559999999999998e-06 2024-09-16T00:09:44.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.428035+0000", 2024-09-16T00:09:44.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5096999999999998e-05 2024-09-16T00:09:44.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.428105+0000", 2024-09-16T00:09:44.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9528000000000002e-05 2024-09-16T00:09:44.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:44.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.428183+0000", 2024-09-16T00:09:44.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8619000000000001e-05 2024-09-16T00:09:44.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.432734+0000", 2024-09-16T00:09:44.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00455092 2024-09-16T00:09:44.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.437748+0000", 2024-09-16T00:09:44.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050134979999999999 2024-09-16T00:09:44.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.437804+0000", 2024-09-16T00:09:44.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6576000000000002e-05 2024-09-16T00:09:44.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.437853+0000", 2024-09-16T00:09:44.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9172000000000003e-05 2024-09-16T00:09:44.589 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.589 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.589 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.589 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10789 3.39 3:9d97a463:::100000030ab.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:44.589 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.518624+0000", 2024-09-16T00:09:44.590 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 6.0316867639999998, 2024-09-16T00:09:44.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012988437, 2024-09-16T00:09:44.590 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.590 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.590 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.590 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.590 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.590 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.590 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10789 2024-09-16T00:09:44.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.590 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.518624+0000", 2024-09-16T00:09:44.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.518624+0000", 2024-09-16T00:09:44.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.518626+0000", 2024-09-16T00:09:44.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.103e-06 2024-09-16T00:09:44.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.522374+0000", 2024-09-16T00:09:44.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037479240000000001 2024-09-16T00:09:44.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.522390+0000", 2024-09-16T00:09:44.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5600999999999999e-05 2024-09-16T00:09:44.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.522400+0000", 2024-09-16T00:09:44.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0225e-05 2024-09-16T00:09:44.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.522476+0000", 2024-09-16T00:09:44.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6014999999999996e-05 2024-09-16T00:09:44.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.522603+0000", 2024-09-16T00:09:44.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012704700000000001 2024-09-16T00:09:44.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:44.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.522779+0000", 2024-09-16T00:09:44.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017634700000000001 2024-09-16T00:09:44.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.528131+0000", 2024-09-16T00:09:44.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053515189999999999 2024-09-16T00:09:44.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.531491+0000", 2024-09-16T00:09:44.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033599139999999999 2024-09-16T00:09:44.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.531553+0000", 2024-09-16T00:09:44.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2438000000000001e-05 2024-09-16T00:09:44.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.531613+0000", 2024-09-16T00:09:44.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9304e-05 2024-09-16T00:09:44.595 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.595 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.595 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.595 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10791 3.39 3:9d97a463:::100000030ab.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:44.596 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.609371+0000", 2024-09-16T00:09:44.596 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.9409398549999999, 2024-09-16T00:09:44.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012992976999999999, 2024-09-16T00:09:44.596 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.596 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.596 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.596 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.596 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10791 2024-09-16T00:09:44.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.596 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.609371+0000", 2024-09-16T00:09:44.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.609371+0000", 2024-09-16T00:09:44.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.609373+0000", 2024-09-16T00:09:44.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.131e-06 2024-09-16T00:09:44.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.613395+0000", 2024-09-16T00:09:44.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040214919999999998 2024-09-16T00:09:44.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.613415+0000", 2024-09-16T00:09:44.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9996e-05 2024-09-16T00:09:44.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.613428+0000", 2024-09-16T00:09:44.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3601000000000001e-05 2024-09-16T00:09:44.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.613491+0000", 2024-09-16T00:09:44.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2713999999999996e-05 2024-09-16T00:09:44.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.613558+0000", 2024-09-16T00:09:44.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7477000000000005e-05 2024-09-16T00:09:44.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:44.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.613648+0000", 2024-09-16T00:09:44.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9449000000000004e-05 2024-09-16T00:09:44.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.617164+0000", 2024-09-16T00:09:44.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035162240000000001 2024-09-16T00:09:44.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.622291+0000", 2024-09-16T00:09:44.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051271759999999998 2024-09-16T00:09:44.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.622326+0000", 2024-09-16T00:09:44.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4647000000000002e-05 2024-09-16T00:09:44.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.622364+0000", 2024-09-16T00:09:44.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.807e-05 2024-09-16T00:09:44.601 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.601 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.601 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.601 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10826 3.11 3:889a8549:::10000003149.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:44.601 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.741353+0000", 2024-09-16T00:09:44.602 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.8089576889999996, 2024-09-16T00:09:44.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014025037000000001, 2024-09-16T00:09:44.602 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.602 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.602 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.602 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.602 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10826 2024-09-16T00:09:44.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.604 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.741353+0000", 2024-09-16T00:09:44.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.741353+0000", 2024-09-16T00:09:44.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.741354+0000", 2024-09-16T00:09:44.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2440000000000001e-06 2024-09-16T00:09:44.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.746855+0000", 2024-09-16T00:09:44.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055001750000000004 2024-09-16T00:09:44.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.746867+0000", 2024-09-16T00:09:44.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2132e-05 2024-09-16T00:09:44.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.746873+0000", 2024-09-16T00:09:44.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0530000000000003e-06 2024-09-16T00:09:44.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.746921+0000", 2024-09-16T00:09:44.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8018e-05 2024-09-16T00:09:44.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.747092+0000", 2024-09-16T00:09:44.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017092100000000001 2024-09-16T00:09:44.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:44.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.747197+0000", 2024-09-16T00:09:44.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105571 2024-09-16T00:09:44.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.754346+0000", 2024-09-16T00:09:44.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071485439999999997 2024-09-16T00:09:44.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.755286+0000", 2024-09-16T00:09:44.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00093984899999999998 2024-09-16T00:09:44.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.755340+0000", 2024-09-16T00:09:44.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4397000000000001e-05 2024-09-16T00:09:44.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.755378+0000", 2024-09-16T00:09:44.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8133000000000002e-05 2024-09-16T00:09:44.610 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.610 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.610 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.610 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10916 3.d 3:b1fb403d:::100000030c5.00000001:head [write 0~4194304 [3@0]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:44.610 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.223310+0000", 2024-09-16T00:09:44.610 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.3270011100000003, 2024-09-16T00:09:44.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013318058000000001, 2024-09-16T00:09:44.610 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.610 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.611 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.611 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.611 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.611 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.611 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10916 2024-09-16T00:09:44.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.611 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.223310+0000", 2024-09-16T00:09:44.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.223310+0000", 2024-09-16T00:09:44.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.223311+0000", 2024-09-16T00:09:44.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1999999999999998e-07 2024-09-16T00:09:44.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.228616+0000", 2024-09-16T00:09:44.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053051590000000003 2024-09-16T00:09:44.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.228634+0000", 2024-09-16T00:09:44.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7859999999999998e-05 2024-09-16T00:09:44.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.228643+0000", 2024-09-16T00:09:44.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.952e-06 2024-09-16T00:09:44.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.228704+0000", 2024-09-16T00:09:44.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0915000000000002e-05 2024-09-16T00:09:44.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.228824+0000", 2024-09-16T00:09:44.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120579 2024-09-16T00:09:44.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:44.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.228939+0000", 2024-09-16T00:09:44.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0001147 2024-09-16T00:09:44.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.235970+0000", 2024-09-16T00:09:44.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070311189999999997 2024-09-16T00:09:44.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.236567+0000", 2024-09-16T00:09:44.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00059735099999999998 2024-09-16T00:09:44.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.236602+0000", 2024-09-16T00:09:44.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5083000000000003e-05 2024-09-16T00:09:44.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.236628+0000", 2024-09-16T00:09:44.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.552e-05 2024-09-16T00:09:44.616 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.616 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.616 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.617 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11098 3.15 3:aa8f498f:::100000030ba.00000000:head [write 0~4194304 [2@1999769]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:44.617 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.016861+0000", 2024-09-16T00:09:44.617 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.5334501750000005, 2024-09-16T00:09:44.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018826408999999999, 2024-09-16T00:09:44.617 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.617 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.617 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.617 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.617 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.617 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.617 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11098 2024-09-16T00:09:44.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.618 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.016861+0000", 2024-09-16T00:09:44.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.016861+0000", 2024-09-16T00:09:44.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.016863+0000", 2024-09-16T00:09:44.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5390000000000002e-06 2024-09-16T00:09:44.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.021057+0000", 2024-09-16T00:09:44.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041934279999999999 2024-09-16T00:09:44.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.021081+0000", 2024-09-16T00:09:44.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4332e-05 2024-09-16T00:09:44.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.021092+0000", 2024-09-16T00:09:44.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0559e-05 2024-09-16T00:09:44.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.021212+0000", 2024-09-16T00:09:44.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120377 2024-09-16T00:09:44.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.021284+0000", 2024-09-16T00:09:44.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2311999999999995e-05 2024-09-16T00:09:44.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:44.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.021384+0000", 2024-09-16T00:09:44.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9418000000000003e-05 2024-09-16T00:09:44.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.026734+0000", 2024-09-16T00:09:44.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053502660000000002 2024-09-16T00:09:44.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.035567+0000", 2024-09-16T00:09:44.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088331299999999998 2024-09-16T00:09:44.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.035626+0000", 2024-09-16T00:09:44.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9290000000000003e-05 2024-09-16T00:09:44.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.035687+0000", 2024-09-16T00:09:44.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0757999999999997e-05 2024-09-16T00:09:44.622 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.622 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.622 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.623 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11162 3.11 3:88de54a5:::10000003191.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:44.623 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.391787+0000", 2024-09-16T00:09:44.623 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.1585241079999999, 2024-09-16T00:09:44.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015917648999999999, 2024-09-16T00:09:44.623 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.623 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.623 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.623 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.623 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.623 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.623 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11162 2024-09-16T00:09:44.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.623 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.391787+0000", 2024-09-16T00:09:44.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.391787+0000", 2024-09-16T00:09:44.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.391789+0000", 2024-09-16T00:09:44.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-16T00:09:44.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.396253+0000", 2024-09-16T00:09:44.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044639450000000004 2024-09-16T00:09:44.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.396277+0000", 2024-09-16T00:09:44.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4105999999999999e-05 2024-09-16T00:09:44.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.396289+0000", 2024-09-16T00:09:44.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1232e-05 2024-09-16T00:09:44.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.396443+0000", 2024-09-16T00:09:44.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000154155 2024-09-16T00:09:44.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.396624+0000", 2024-09-16T00:09:44.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018098300000000001 2024-09-16T00:09:44.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:44.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.396779+0000", 2024-09-16T00:09:44.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015505799999999999 2024-09-16T00:09:44.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.401976+0000", 2024-09-16T00:09:44.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051975179999999999 2024-09-16T00:09:44.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.407575+0000", 2024-09-16T00:09:44.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055986910000000003 2024-09-16T00:09:44.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.407643+0000", 2024-09-16T00:09:44.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8355000000000005e-05 2024-09-16T00:09:44.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.407704+0000", 2024-09-16T00:09:44.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1033000000000003e-05 2024-09-16T00:09:44.629 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.629 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.629 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.629 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11163 3.d 3:b08debfe:::1000000312e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:44.629 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.399573+0000", 2024-09-16T00:09:44.629 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.150738219, 2024-09-16T00:09:44.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014239757, 2024-09-16T00:09:44.629 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.629 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.629 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.629 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.630 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.630 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11163 2024-09-16T00:09:44.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.630 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.399573+0000", 2024-09-16T00:09:44.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.399573+0000", 2024-09-16T00:09:44.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.399575+0000", 2024-09-16T00:09:44.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0590000000000001e-06 2024-09-16T00:09:44.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.403675+0000", 2024-09-16T00:09:44.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041001750000000002 2024-09-16T00:09:44.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.403689+0000", 2024-09-16T00:09:44.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4551e-05 2024-09-16T00:09:44.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.403698+0000", 2024-09-16T00:09:44.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4950000000000008e-06 2024-09-16T00:09:44.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.403767+0000", 2024-09-16T00:09:44.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8674e-05 2024-09-16T00:09:44.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.403868+0000", 2024-09-16T00:09:44.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101659 2024-09-16T00:09:44.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:44.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.404014+0000", 2024-09-16T00:09:44.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000145396 2024-09-16T00:09:44.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.411247+0000", 2024-09-16T00:09:44.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072330019999999997 2024-09-16T00:09:44.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.413705+0000", 2024-09-16T00:09:44.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024581569999999999 2024-09-16T00:09:44.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.413764+0000", 2024-09-16T00:09:44.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9005999999999997e-05 2024-09-16T00:09:44.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.413812+0000", 2024-09-16T00:09:44.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8582999999999999e-05 2024-09-16T00:09:44.635 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.635 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.635 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.635 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11281 3.f 3:f35949a2:::100000030f0.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:44.635 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.798967+0000", 2024-09-16T00:09:44.635 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7513438450000001, 2024-09-16T00:09:44.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014241894999999999, 2024-09-16T00:09:44.635 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.635 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.635 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.636 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.636 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.636 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.636 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11281 2024-09-16T00:09:44.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.636 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.798967+0000", 2024-09-16T00:09:44.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.798967+0000", 2024-09-16T00:09:44.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.798968+0000", 2024-09-16T00:09:44.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2810000000000001e-06 2024-09-16T00:09:44.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.803629+0000", 2024-09-16T00:09:44.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004660628 2024-09-16T00:09:44.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.803648+0000", 2024-09-16T00:09:44.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8654000000000001e-05 2024-09-16T00:09:44.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.803658+0000", 2024-09-16T00:09:44.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9490000000000008e-06 2024-09-16T00:09:44.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.803727+0000", 2024-09-16T00:09:44.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9727000000000005e-05 2024-09-16T00:09:44.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.803850+0000", 2024-09-16T00:09:44.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012224600000000001 2024-09-16T00:09:44.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:44.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.804048+0000", 2024-09-16T00:09:44.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019871799999999999 2024-09-16T00:09:44.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.812750+0000", 2024-09-16T00:09:44.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0087017009999999992 2024-09-16T00:09:44.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.813111+0000", 2024-09-16T00:09:44.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036071399999999999 2024-09-16T00:09:44.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.813162+0000", 2024-09-16T00:09:44.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1319000000000001e-05 2024-09-16T00:09:44.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.813209+0000", 2024-09-16T00:09:44.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6958e-05 2024-09-16T00:09:44.641 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.641 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.641 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.641 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11284 3.15 3:ab224bc6:::100000031b9.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:44.641 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.807120+0000", 2024-09-16T00:09:44.641 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.743191275, 2024-09-16T00:09:44.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013001497000000001, 2024-09-16T00:09:44.641 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.642 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.642 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.642 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.642 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.642 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.642 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11284 2024-09-16T00:09:44.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.642 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.807120+0000", 2024-09-16T00:09:44.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.807120+0000", 2024-09-16T00:09:44.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.807122+0000", 2024-09-16T00:09:44.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9149999999999999e-06 2024-09-16T00:09:44.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.811562+0000", 2024-09-16T00:09:44.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044405859999999998 2024-09-16T00:09:44.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.811578+0000", 2024-09-16T00:09:44.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6143000000000001e-05 2024-09-16T00:09:44.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.811588+0000", 2024-09-16T00:09:44.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6690000000000001e-06 2024-09-16T00:09:44.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.811654+0000", 2024-09-16T00:09:44.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6357000000000002e-05 2024-09-16T00:09:44.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.811787+0000", 2024-09-16T00:09:44.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013305700000000001 2024-09-16T00:09:44.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:44.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.811929+0000", 2024-09-16T00:09:44.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014127000000000001 2024-09-16T00:09:44.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.816818+0000", 2024-09-16T00:09:44.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048894029999999996 2024-09-16T00:09:44.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.820047+0000", 2024-09-16T00:09:44.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032289879999999999 2024-09-16T00:09:44.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.820084+0000", 2024-09-16T00:09:44.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6990000000000003e-05 2024-09-16T00:09:44.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.820121+0000", 2024-09-16T00:09:44.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7119000000000002e-05 2024-09-16T00:09:44.647 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.647 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.647 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.647 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11830 3.d 3:b3bd9509:::100000031dc.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:44.647 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.431258+0000", 2024-09-16T00:09:44.647 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 1.11905261, 2024-09-16T00:09:44.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015383539999999999, 2024-09-16T00:09:44.647 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.647 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.648 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.648 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.648 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.648 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.648 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11830 2024-09-16T00:09:44.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.648 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.431258+0000", 2024-09-16T00:09:44.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.431258+0000", 2024-09-16T00:09:44.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.431261+0000", 2024-09-16T00:09:44.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2400000000000002e-06 2024-09-16T00:09:44.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.436543+0000", 2024-09-16T00:09:44.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052820740000000003 2024-09-16T00:09:44.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.436566+0000", 2024-09-16T00:09:44.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3536999999999998e-05 2024-09-16T00:09:44.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.436578+0000", 2024-09-16T00:09:44.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.216e-05 2024-09-16T00:09:44.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.436658+0000", 2024-09-16T00:09:44.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9558e-05 2024-09-16T00:09:44.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.436763+0000", 2024-09-16T00:09:44.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010471000000000001 2024-09-16T00:09:44.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:44.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.436923+0000", 2024-09-16T00:09:44.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016087600000000001 2024-09-16T00:09:44.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.443272+0000", 2024-09-16T00:09:44.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063481420000000002 2024-09-16T00:09:44.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.446535+0000", 2024-09-16T00:09:44.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032632440000000002 2024-09-16T00:09:44.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.446596+0000", 2024-09-16T00:09:44.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1001000000000001e-05 2024-09-16T00:09:44.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.446642+0000", 2024-09-16T00:09:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5998000000000002e-05 2024-09-16T00:09:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11894 3.11 3:8929e5dc:::100000032b6.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.577585+0000", 2024-09-16T00:09:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.97272623199999997, 2024-09-16T00:09:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016935823999999999, 2024-09-16T00:09:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.653 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11894 2024-09-16T00:09:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.577585+0000", 2024-09-16T00:09:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.577585+0000", 2024-09-16T00:09:44.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.577588+0000", 2024-09-16T00:09:44.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0060000000000001e-06 2024-09-16T00:09:44.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.582079+0000", 2024-09-16T00:09:44.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044913380000000001 2024-09-16T00:09:44.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.582103+0000", 2024-09-16T00:09:44.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3575e-05 2024-09-16T00:09:44.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.582114+0000", 2024-09-16T00:09:44.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1766000000000001e-05 2024-09-16T00:09:44.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.582261+0000", 2024-09-16T00:09:44.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146596 2024-09-16T00:09:44.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.582354+0000", 2024-09-16T00:09:44.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2677999999999996e-05 2024-09-16T00:09:44.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:44.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.582501+0000", 2024-09-16T00:09:44.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000147773 2024-09-16T00:09:44.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.590250+0000", 2024-09-16T00:09:44.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077488160000000004 2024-09-16T00:09:44.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.594405+0000", 2024-09-16T00:09:44.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041547420000000003 2024-09-16T00:09:44.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.594470+0000", 2024-09-16T00:09:44.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4729999999999999e-05 2024-09-16T00:09:44.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.594521+0000", 2024-09-16T00:09:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0803999999999997e-05 2024-09-16T00:09:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11901 3.39 3:9f6e58bb:::10000003202.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.585459+0000", 2024-09-16T00:09:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.96485168499999996, 2024-09-16T00:09:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014913165000000001, 2024-09-16T00:09:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.659 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11901 2024-09-16T00:09:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.585459+0000", 2024-09-16T00:09:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.585459+0000", 2024-09-16T00:09:44.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.585460+0000", 2024-09-16T00:09:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.215e-06 2024-09-16T00:09:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.590006+0000", 2024-09-16T00:09:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004545398 2024-09-16T00:09:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.590020+0000", 2024-09-16T00:09:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3946e-05 2024-09-16T00:09:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.590028+0000", 2024-09-16T00:09:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1710000000000002e-06 2024-09-16T00:09:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.590091+0000", 2024-09-16T00:09:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2552999999999996e-05 2024-09-16T00:09:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.590171+0000", 2024-09-16T00:09:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0654000000000004e-05 2024-09-16T00:09:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.590280+0000", 2024-09-16T00:09:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108823 2024-09-16T00:09:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.594955+0000", 2024-09-16T00:09:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046746510000000002 2024-09-16T00:09:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.600250+0000", 2024-09-16T00:09:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052952279999999999 2024-09-16T00:09:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.600316+0000", 2024-09-16T00:09:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6107999999999998e-05 2024-09-16T00:09:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.600372+0000", 2024-09-16T00:09:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6418000000000001e-05 2024-09-16T00:09:44.664 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11940 3.2d 3:b522f437:::1000000335a.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.719979+0000", 2024-09-16T00:09:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.83033194499999996, 2024-09-16T00:09:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018463778, 2024-09-16T00:09:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.665 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11940 2024-09-16T00:09:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.719979+0000", 2024-09-16T00:09:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.719979+0000", 2024-09-16T00:09:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.719981+0000", 2024-09-16T00:09:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0940000000000002e-06 2024-09-16T00:09:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.724176+0000", 2024-09-16T00:09:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041950909999999998 2024-09-16T00:09:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.724194+0000", 2024-09-16T00:09:44.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-16T00:09:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.724204+0000", 2024-09-16T00:09:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0431000000000001e-05 2024-09-16T00:09:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.724280+0000", 2024-09-16T00:09:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6099000000000004e-05 2024-09-16T00:09:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.724383+0000", 2024-09-16T00:09:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102598 2024-09-16T00:09:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.724548+0000", 2024-09-16T00:09:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000164857 2024-09-16T00:09:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.737989+0000", 2024-09-16T00:09:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013440821 2024-09-16T00:09:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.738344+0000", 2024-09-16T00:09:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035568299999999998 2024-09-16T00:09:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.738402+0000", 2024-09-16T00:09:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7552000000000001e-05 2024-09-16T00:09:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.738443+0000", 2024-09-16T00:09:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0893000000000003e-05 2024-09-16T00:09:44.670 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.671 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.671 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.671 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12000 3.d 3:b3bd9509:::100000031dc.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:44.671 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.860231+0000", 2024-09-16T00:09:44.671 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.69008004300000003, 2024-09-16T00:09:44.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013111581000000001, 2024-09-16T00:09:44.671 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:44.671 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:44.671 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:44.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:44.671 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:44.672 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:44.672 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12000 2024-09-16T00:09:44.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.672 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:44.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:44.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.860231+0000", 2024-09-16T00:09:44.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:44.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.860231+0000", 2024-09-16T00:09:44.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:44.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:44.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.860233+0000", 2024-09-16T00:09:44.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.688e-06 2024-09-16T00:09:44.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:44.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.864516+0000", 2024-09-16T00:09:44.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042832249999999999 2024-09-16T00:09:44.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:44.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.864539+0000", 2024-09-16T00:09:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2897000000000001e-05 2024-09-16T00:09:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.864549+0000", 2024-09-16T00:09:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0301e-05 2024-09-16T00:09:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.864639+0000", 2024-09-16T00:09:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0310999999999996e-05 2024-09-16T00:09:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:44.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.864727+0000", 2024-09-16T00:09:44.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8096999999999994e-05 2024-09-16T00:09:44.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:44.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.864863+0000", 2024-09-16T00:09:44.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013540199999999999 2024-09-16T00:09:44.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:44.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.870785+0000", 2024-09-16T00:09:44.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005921745 2024-09-16T00:09:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.873250+0000", 2024-09-16T00:09:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024653159999999999 2024-09-16T00:09:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.873299+0000", 2024-09-16T00:09:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9631e-05 2024-09-16T00:09:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:44.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.873342+0000", 2024-09-16T00:09:44.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2967999999999999e-05 2024-09-16T00:09:44.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.677 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.677 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:44.677 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:44.677 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:44.679 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:09:44.681 INFO:tasks.workunit.client.0.smithi139.stdout:8/422: symlink d17/d1f/d88/l90 0 2024-09-16T00:09:44.681 INFO:tasks.workunit.client.0.smithi139.stdout:4/391: creat d12/d4a/d4d/f81 x:0 0 0 2024-09-16T00:09:44.681 INFO:tasks.workunit.client.0.smithi139.stdout:4/392: mknod d12/c82 0 2024-09-16T00:09:44.681 INFO:tasks.workunit.client.0.smithi139.stdout:4/393: dread d12/d2b/f6b [0,4194304] 0 2024-09-16T00:09:44.681 INFO:tasks.workunit.client.0.smithi139.stdout:8/423: getdents d17 0 2024-09-16T00:09:44.681 INFO:tasks.workunit.client.0.smithi139.stdout:8/424: write d17/d1a/d2d/f6b [451140,18417] 0 2024-09-16T00:09:44.681 INFO:tasks.workunit.client.0.smithi139.stdout:8/425: symlink d17/d1a/d50/l91 0 2024-09-16T00:09:44.681 INFO:tasks.workunit.client.0.smithi139.stdout:8/426: mkdir d17/d1a/d50/d86/d92 0 2024-09-16T00:09:44.681 INFO:tasks.workunit.client.0.smithi139.stdout:8/427: rename d17/d1a/d43/l4e to d17/d1a/d43/d40/l93 0 2024-09-16T00:09:44.681 INFO:tasks.workunit.client.0.smithi139.stdout:8/428: creat d17/d1f/f94 x:0 0 0 2024-09-16T00:09:44.681 INFO:tasks.workunit.client.0.smithi139.stdout:1/408: sync 2024-09-16T00:09:44.682 INFO:tasks.workunit.client.0.smithi139.stdout:2/423: sync 2024-09-16T00:09:44.682 INFO:tasks.workunit.client.0.smithi139.stdout:1/409: fdatasync d7/f3f 0 2024-09-16T00:09:44.682 INFO:tasks.workunit.client.0.smithi139.stdout:2/424: creat d11/d30/d57/d5d/d73/f8e x:0 0 0 2024-09-16T00:09:44.682 INFO:tasks.workunit.client.0.smithi139.stdout:1/410: creat d7/d8/d40/d48/d4a/d74/f7e x:0 0 0 2024-09-16T00:09:44.682 INFO:tasks.workunit.client.0.smithi139.stdout:2/425: symlink d11/d30/d57/d5d/l8f 0 2024-09-16T00:09:44.682 INFO:tasks.workunit.client.0.smithi139.stdout:8/429: dread d17/d1a/d43/d40/f49 [4194304,4194304] 0 2024-09-16T00:09:44.682 INFO:tasks.workunit.client.0.smithi139.stdout:8/430: dread - d17/d54/f80 zero size 2024-09-16T00:09:44.682 INFO:tasks.workunit.client.0.smithi139.stdout:2/426: dread d11/d30/f56 [0,4194304] 0 2024-09-16T00:09:44.682 INFO:tasks.workunit.client.0.smithi139.stdout:1/411: dwrite d7/d8/f11 [0,4194304] 0 2024-09-16T00:09:44.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:44.617+0000 7fdc55f8d640 1 -- 172.21.15.139:0/2943811187 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fdc500c72c0 con 0x7fdc500c7fb0 2024-09-16T00:09:44.686 INFO:tasks.workunit.client.0.smithi139.stdout:2/427: symlink d11/d30/d57/d5d/d73/d74/d81/l90 0 2024-09-16T00:09:44.686 INFO:tasks.workunit.client.0.smithi139.stdout:2/428: readlink d11/d2f/l40 0 2024-09-16T00:09:44.686 INFO:tasks.workunit.client.0.smithi139.stdout:8/431: link d17/d1f/d3a/l51 d17/d1f/d88/l95 0 2024-09-16T00:09:44.686 INFO:tasks.workunit.client.0.smithi139.stdout:1/412: dwrite d7/d8/d27/f2c [0,4194304] 0 2024-09-16T00:09:44.686 INFO:tasks.workunit.client.0.smithi139.stdout:8/432: dread d17/d1a/d43/d40/f41 [0,4194304] 0 2024-09-16T00:09:44.686 INFO:tasks.workunit.client.0.smithi139.stdout:8/433: stat d17/d1a/d43 0 2024-09-16T00:09:44.686 INFO:tasks.workunit.client.0.smithi139.stdout:8/434: symlink d17/d1a/l96 0 2024-09-16T00:09:44.687 INFO:tasks.workunit.client.0.smithi139.stdout:2/429: dwrite d11/d30/f44 [4194304,4194304] 0 2024-09-16T00:09:44.687 INFO:tasks.workunit.client.0.smithi139.stdout:8/435: rename d17/f62 to d17/d1a/d50/d86/f97 0 2024-09-16T00:09:44.687 INFO:tasks.workunit.client.0.smithi139.stdout:8/436: chown d17/f19 528137407 1 2024-09-16T00:09:44.687 INFO:tasks.workunit.client.0.smithi139.stdout:2/430: unlink d11/d30/d57/d50/f85 0 2024-09-16T00:09:44.687 INFO:tasks.workunit.client.0.smithi139.stdout:8/437: fdatasync d17/d54/f69 0 2024-09-16T00:09:44.687 INFO:tasks.workunit.client.0.smithi139.stdout:2/431: chown d11/d30/d57/d29/d2b/f2c 41373550 1 2024-09-16T00:09:44.687 INFO:tasks.workunit.client.0.smithi139.stdout:2/432: fdatasync d11/d2f/f33 0 2024-09-16T00:09:44.687 INFO:tasks.workunit.client.0.smithi139.stdout:2/433: symlink d11/d30/d57/d5d/l91 0 2024-09-16T00:09:44.687 INFO:tasks.workunit.client.0.smithi139.stdout:8/438: getdents d17/d54 0 2024-09-16T00:09:44.687 INFO:tasks.workunit.client.0.smithi139.stdout:2/434: write d11/d30/d57/d29/d2b/f31 [1938646,54438] 0 2024-09-16T00:09:44.687 INFO:tasks.workunit.client.0.smithi139.stdout:8/439: link d17/d1a/d43/d40/f8d d17/d1a/d57/f98 0 2024-09-16T00:09:44.687 INFO:tasks.workunit.client.0.smithi139.stdout:8/440: chown d17/d1a/d43/f2b 86 1 2024-09-16T00:09:44.687 INFO:tasks.workunit.client.0.smithi139.stdout:8/441: write d17/f19 [314801,100895] 0 2024-09-16T00:09:44.687 INFO:tasks.workunit.client.0.smithi139.stdout:8/442: write d17/d1a/f34 [1059604,114924] 0 2024-09-16T00:09:44.688 INFO:tasks.workunit.client.0.smithi139.stdout:8/443: creat d17/f99 x:0 0 0 2024-09-16T00:09:44.688 INFO:tasks.workunit.client.0.smithi139.stdout:8/444: symlink d17/d1f/d3a/l9a 0 2024-09-16T00:09:44.688 INFO:tasks.workunit.client.0.smithi139.stdout:2/435: getdents d11/d30/d57/d5d 0 2024-09-16T00:09:44.688 INFO:tasks.workunit.client.0.smithi139.stdout:8/445: creat d17/d1a/d50/d86/d92/f9b x:0 0 0 2024-09-16T00:09:44.688 INFO:tasks.workunit.client.0.smithi139.stdout:8/446: rmdir d17/d1f/d88 39 2024-09-16T00:09:44.688 INFO:tasks.workunit.client.0.smithi139.stdout:8/447: unlink d17/d54/f6e 0 2024-09-16T00:09:44.688 INFO:tasks.workunit.client.0.smithi139.stdout:2/436: dwrite d11/d30/d57/f59 [0,4194304] 0 2024-09-16T00:09:44.688 INFO:tasks.workunit.client.0.smithi139.stdout:2/437: fdatasync d11/d30/d57/d5d/d73/f79 0 2024-09-16T00:09:44.688 INFO:tasks.workunit.client.0.smithi139.stdout:8/448: truncate f4 204484 0 2024-09-16T00:09:44.688 INFO:tasks.workunit.client.0.smithi139.stdout:2/438: link d11/d30/f56 d11/d30/f92 0 2024-09-16T00:09:44.688 INFO:tasks.workunit.client.0.smithi139.stdout:8/449: symlink d17/d1f/l9c 0 2024-09-16T00:09:44.688 INFO:tasks.workunit.client.0.smithi139.stdout:8/450: dread d17/d1a/d43/d40/f41 [0,4194304] 0 2024-09-16T00:09:44.688 INFO:tasks.workunit.client.0.smithi139.stdout:2/439: dwrite d11/f1e [0,4194304] 0 2024-09-16T00:09:44.688 INFO:tasks.workunit.client.0.smithi139.stdout:8/451: mkdir d17/d1a/d50/d86/d9d 0 2024-09-16T00:09:44.689 INFO:tasks.workunit.client.0.smithi139.stdout:2/440: mknod d11/d2f/d6e/d7f/c93 0 2024-09-16T00:09:44.689 INFO:tasks.workunit.client.0.smithi139.stdout:2/441: mknod d11/d30/d57/d5d/c94 0 2024-09-16T00:09:44.689 INFO:tasks.workunit.client.0.smithi139.stdout:2/442: write d11/f15 [2395276,60809] 0 2024-09-16T00:09:44.689 INFO:tasks.workunit.client.0.smithi139.stdout:2/443: chown d11/d30/d57/d5d/c67 338 1 2024-09-16T00:09:44.689 INFO:tasks.workunit.client.0.smithi139.stdout:2/444: mknod d11/d45/c95 0 2024-09-16T00:09:44.689 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:44.677+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:44.690 INFO:tasks.workunit.client.0.smithi139.stdout:8/452: write d17/d1a/d43/d40/f8d [542841,14547] 0 2024-09-16T00:09:44.690 INFO:tasks.workunit.client.0.smithi139.stdout:2/445: dwrite d11/d30/d57/d29/f5c [0,4194304] 0 2024-09-16T00:09:44.690 INFO:tasks.workunit.client.0.smithi139.stdout:8/453: dwrite d17/d1a/d57/f7b [0,4194304] 0 2024-09-16T00:09:44.691 INFO:tasks.workunit.client.0.smithi139.stdout:8/454: dread - d17/d54/f80 zero size 2024-09-16T00:09:44.694 INFO:tasks.workunit.client.0.smithi139.stdout:8/455: chown d17/d1f/f28 27 1 2024-09-16T00:09:44.696 INFO:tasks.workunit.client.0.smithi139.stdout:2/446: dread d11/f15 [4194304,4194304] 0 2024-09-16T00:09:44.700 INFO:tasks.workunit.client.0.smithi139.stdout:2/447: rename d11/d32/c72 to d11/d30/d57/d5d/d73/c96 0 2024-09-16T00:09:44.701 INFO:tasks.workunit.client.0.smithi139.stdout:8/456: link d17/d1a/d2d/c60 d17/d1a/d50/d86/d9d/c9e 0 2024-09-16T00:09:44.702 INFO:tasks.workunit.client.0.smithi139.stdout:2/448: creat d11/d30/d57/d5d/d73/d74/d81/f97 x:0 0 0 2024-09-16T00:09:44.704 INFO:tasks.workunit.client.0.smithi139.stdout:8/457: mkdir d17/d1a/d50/d86/d9d/d9f 0 2024-09-16T00:09:44.708 INFO:tasks.workunit.client.0.smithi139.stdout:2/449: getdents d11/d32 0 2024-09-16T00:09:44.710 INFO:tasks.workunit.client.0.smithi139.stdout:8/458: getdents d17/d1a/d50/d86/d9d 0 2024-09-16T00:09:44.711 INFO:tasks.workunit.client.0.smithi139.stdout:2/450: dread d11/d30/f92 [0,4194304] 0 2024-09-16T00:09:44.713 INFO:tasks.workunit.client.0.smithi139.stdout:8/459: mknod d17/d1a/ca0 0 2024-09-16T00:09:44.723 INFO:tasks.workunit.client.0.smithi139.stdout:8/460: truncate d17/d1a/f1d 4539779 0 2024-09-16T00:09:44.724 INFO:tasks.workunit.client.0.smithi139.stdout:2/451: dread d11/f64 [0,4194304] 0 2024-09-16T00:09:44.724 INFO:tasks.workunit.client.0.smithi139.stdout:8/461: mknod d17/d1a/d43/d40/ca1 0 2024-09-16T00:09:44.726 INFO:tasks.workunit.client.0.smithi139.stdout:2/452: creat d11/d30/d57/d29/d2b/f98 x:0 0 0 2024-09-16T00:09:44.726 INFO:tasks.workunit.client.0.smithi139.stdout:8/462: mknod d17/d1f/d88/ca2 0 2024-09-16T00:09:44.727 INFO:tasks.workunit.client.0.smithi139.stdout:2/453: readlink d11/d30/d57/d29/d4f/l82 0 2024-09-16T00:09:44.728 INFO:tasks.workunit.client.0.smithi139.stdout:2/454: readlink d11/d30/d57/l51 0 2024-09-16T00:09:44.728 INFO:tasks.workunit.client.0.smithi139.stdout:8/463: creat d17/d1a/d43/d40/d4f/fa3 x:0 0 0 2024-09-16T00:09:44.730 INFO:tasks.workunit.client.0.smithi139.stdout:8/464: creat d17/d1a/d50/fa4 x:0 0 0 2024-09-16T00:09:44.734 INFO:tasks.workunit.client.0.smithi139.stdout:8/465: unlink d17/d1a/f34 0 2024-09-16T00:09:44.735 INFO:tasks.workunit.client.0.smithi139.stdout:8/466: chown d17/d1f/f81 1764881974 1 2024-09-16T00:09:44.738 INFO:tasks.workunit.client.0.smithi139.stdout:8/467: creat d17/d1a/fa5 x:0 0 0 2024-09-16T00:09:44.741 INFO:tasks.workunit.client.0.smithi139.stdout:8/468: rename d17/d1f/d88/f8e to d17/d1f/d3a/fa6 0 2024-09-16T00:09:44.742 INFO:tasks.workunit.client.0.smithi139.stdout:2/455: dread fc [0,4194304] 0 2024-09-16T00:09:44.744 INFO:tasks.workunit.client.0.smithi139.stdout:2/456: creat d11/d30/d57/d5d/d73/d74/d81/f99 x:0 0 0 2024-09-16T00:09:44.745 INFO:tasks.workunit.client.0.smithi139.stdout:2/457: stat d11/d30/d57/d5d/d73/d74/d81 0 2024-09-16T00:09:44.745 INFO:tasks.workunit.client.0.smithi139.stdout:8/469: link d17/d1a/d57/f8b d17/d1a/d43/d40/d4f/fa7 0 2024-09-16T00:09:44.755 INFO:tasks.workunit.client.0.smithi139.stdout:2/458: dread d11/d30/f42 [0,4194304] 0 2024-09-16T00:09:44.757 INFO:tasks.workunit.client.0.smithi139.stdout:2/459: creat d11/d2f/d6e/f9a x:0 0 0 2024-09-16T00:09:44.758 INFO:tasks.workunit.client.0.smithi139.stdout:8/470: dwrite f4 [0,4194304] 0 2024-09-16T00:09:44.763 INFO:tasks.workunit.client.0.smithi139.stdout:2/460: rename d11/d30/d57/c49 to d11/d30/d57/d29/d86/c9b 0 2024-09-16T00:09:44.763 INFO:tasks.workunit.client.0.smithi139.stdout:8/471: dread d17/d1a/d57/f98 [0,4194304] 0 2024-09-16T00:09:44.763 INFO:tasks.workunit.client.0.smithi139.stdout:8/472: readlink d17/d1a/d2d/l8c 0 2024-09-16T00:09:44.765 INFO:tasks.workunit.client.0.smithi139.stdout:6/289: sync 2024-09-16T00:09:44.766 INFO:tasks.workunit.client.0.smithi139.stdout:7/430: sync 2024-09-16T00:09:44.766 INFO:tasks.workunit.client.0.smithi139.stdout:5/437: sync 2024-09-16T00:09:44.767 INFO:tasks.workunit.client.0.smithi139.stdout:7/431: read d5/f85 [2169930,8795] 0 2024-09-16T00:09:44.769 INFO:tasks.workunit.client.0.smithi139.stdout:7/432: read d5/d26/d46/f4a [491282,60043] 0 2024-09-16T00:09:44.769 INFO:tasks.workunit.client.0.smithi139.stdout:5/438: write d1/d13/d3b/d3e/f6e [1678046,9585] 0 2024-09-16T00:09:44.770 INFO:tasks.workunit.client.0.smithi139.stdout:6/290: dread d8/db/d26/d28/f4f [4194304,4194304] 0 2024-09-16T00:09:44.771 INFO:tasks.workunit.client.0.smithi139.stdout:5/439: chown d1/d25/d30/l40 6 1 2024-09-16T00:09:44.771 INFO:tasks.workunit.client.0.smithi139.stdout:2/461: truncate d11/f64 1286417 0 2024-09-16T00:09:44.772 INFO:tasks.workunit.client.0.smithi139.stdout:8/473: truncate d17/d1a/d43/f6c 3263266 0 2024-09-16T00:09:44.772 INFO:tasks.workunit.client.0.smithi139.stdout:6/291: dread - d8/d29/f36 zero size 2024-09-16T00:09:44.773 INFO:tasks.workunit.client.0.smithi139.stdout:7/433: write d5/d16/d22/d25/d2d/d50/d44/f84 [640799,46430] 0 2024-09-16T00:09:44.773 INFO:tasks.workunit.client.0.smithi139.stdout:8/474: readlink d17/l31 0 2024-09-16T00:09:44.773 INFO:tasks.workunit.client.0.smithi139.stdout:5/440: chown d1/d21/d49/d53/d6a/d7b/d7a 0 1 2024-09-16T00:09:44.774 INFO:tasks.workunit.client.0.smithi139.stdout:8/475: fdatasync d17/d54/f80 0 2024-09-16T00:09:44.775 INFO:tasks.workunit.client.0.smithi139.stdout:6/292: write d8/db/d26/d45/f46 [582987,37794] 0 2024-09-16T00:09:44.775 INFO:tasks.workunit.client.0.smithi139.stdout:7/434: chown d5/d16/d22/d25/d63/f91 5 1 2024-09-16T00:09:44.777 INFO:tasks.workunit.client.0.smithi139.stdout:8/476: readlink d17/d1a/d2d/l74 0 2024-09-16T00:09:44.779 INFO:tasks.workunit.client.0.smithi139.stdout:5/441: dread d1/fa [0,4194304] 0 2024-09-16T00:09:44.779 INFO:tasks.workunit.client.0.smithi139.stdout:8/477: chown d17/d1a/d50 26884461 1 2024-09-16T00:09:44.779 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:44.778+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:44.779 INFO:tasks.workunit.client.0.smithi139.stdout:6/293: truncate d8/d20/f64 933788 0 2024-09-16T00:09:44.781 INFO:tasks.workunit.client.0.smithi139.stdout:2/462: dwrite d11/d30/f36 [0,4194304] 0 2024-09-16T00:09:44.784 INFO:tasks.workunit.client.0.smithi139.stdout:2/463: fsync d11/d30/d57/d5d/d73/f8e 0 2024-09-16T00:09:44.784 INFO:tasks.workunit.client.0.smithi139.stdout:8/478: dread d17/f83 [0,4194304] 0 2024-09-16T00:09:44.786 INFO:tasks.workunit.client.0.smithi139.stdout:7/435: dwrite d5/d26/d42/f8f [0,4194304] 0 2024-09-16T00:09:44.786 INFO:tasks.workunit.client.0.smithi139.stdout:5/442: rename d1/d25/d30/d33/d43/d52 to d1/d13/d95 0 2024-09-16T00:09:44.787 INFO:tasks.workunit.client.0.smithi139.stdout:8/479: write d17/d1a/d50/d86/d92/f9b [350181,29863] 0 2024-09-16T00:09:44.791 INFO:tasks.workunit.client.0.smithi139.stdout:5/443: write d1/d13/d3f/f77 [457560,34036] 0 2024-09-16T00:09:44.791 INFO:tasks.workunit.client.0.smithi139.stdout:6/294: rename d8/d20/d27/d10 to d8/db/d26/d45/d65 0 2024-09-16T00:09:44.796 INFO:tasks.workunit.client.0.smithi139.stdout:2/464: dwrite d11/f16 [0,4194304] 0 2024-09-16T00:09:44.797 INFO:tasks.workunit.client.0.smithi139.stdout:0/465: sync 2024-09-16T00:09:44.797 INFO:tasks.workunit.client.0.smithi139.stdout:4/394: sync 2024-09-16T00:09:44.797 INFO:tasks.workunit.client.0.smithi139.stdout:3/432: sync 2024-09-16T00:09:44.797 INFO:tasks.workunit.client.0.smithi139.stdout:9/484: sync 2024-09-16T00:09:44.797 INFO:tasks.workunit.client.0.smithi139.stdout:6/295: read - d8/db/d26/f54 zero size 2024-09-16T00:09:44.798 INFO:tasks.workunit.client.0.smithi139.stdout:8/480: dread d17/d1a/d43/d40/f41 [0,4194304] 0 2024-09-16T00:09:44.800 INFO:tasks.workunit.client.0.smithi139.stdout:2/465: stat d11/d30/d57/d5d/c94 0 2024-09-16T00:09:44.800 INFO:tasks.workunit.client.0.smithi139.stdout:3/433: rmdir d5 39 2024-09-16T00:09:44.800 INFO:tasks.workunit.client.0.smithi139.stdout:9/485: chown d19/d1a/d20/d23/c88 28 1 2024-09-16T00:09:44.801 INFO:tasks.workunit.client.0.smithi139.stdout:4/395: mknod d12/d47/d57/d7f/c83 0 2024-09-16T00:09:44.802 INFO:tasks.workunit.client.0.smithi139.stdout:8/481: stat d17/d1a/d43/c4d 0 2024-09-16T00:09:44.802 INFO:tasks.workunit.client.0.smithi139.stdout:9/486: write fa [1318684,51988] 0 2024-09-16T00:09:44.803 INFO:tasks.workunit.client.0.smithi139.stdout:6/296: readlink d8/d20/d27/l34 0 2024-09-16T00:09:44.805 INFO:tasks.workunit.client.0.smithi139.stdout:4/396: write d12/f4f [161751,41046] 0 2024-09-16T00:09:44.807 INFO:tasks.workunit.client.0.smithi139.stdout:7/436: write d5/d16/d22/d25/d2d/d81/f89 [5365865,72080] 0 2024-09-16T00:09:44.807 INFO:tasks.workunit.client.0.smithi139.stdout:8/482: rmdir d17/d1f 39 2024-09-16T00:09:44.807 INFO:tasks.workunit.client.0.smithi139.stdout:6/297: readlink d8/d1f/l5d 0 2024-09-16T00:09:44.812 INFO:tasks.workunit.client.0.smithi139.stdout:2/466: fdatasync d11/d30/d57/f27 0 2024-09-16T00:09:44.813 INFO:tasks.workunit.client.0.smithi139.stdout:4/397: fdatasync d12/d2b/f6b 0 2024-09-16T00:09:44.813 INFO:tasks.workunit.client.0.smithi139.stdout:3/434: chown d5/d7/d23/d27/d2f/l9b 4 1 2024-09-16T00:09:44.813 INFO:tasks.workunit.client.0.smithi139.stdout:7/437: read d5/d37/f86 [3308788,128020] 0 2024-09-16T00:09:44.814 INFO:tasks.workunit.client.0.smithi139.stdout:2/467: read d11/d30/f42 [1086238,111947] 0 2024-09-16T00:09:44.815 INFO:tasks.workunit.client.0.smithi139.stdout:0/466: dwrite d6/d1c/d41/d44/f71 [0,4194304] 0 2024-09-16T00:09:44.817 INFO:tasks.workunit.client.0.smithi139.stdout:4/398: mknod d12/d4a/d4d/d69/c84 0 2024-09-16T00:09:44.818 INFO:tasks.workunit.client.0.smithi139.stdout:5/444: dwrite d1/f4a [0,4194304] 0 2024-09-16T00:09:44.819 INFO:tasks.workunit.client.0.smithi139.stdout:3/435: mkdir d5/d7/d23/d27/d2f/d30/d4c/da1 0 2024-09-16T00:09:44.821 INFO:tasks.workunit.client.0.smithi139.stdout:7/438: creat d5/d16/d22/d25/f9b x:0 0 0 2024-09-16T00:09:44.822 INFO:tasks.workunit.client.0.smithi139.stdout:9/487: rename d19/d1a/d20/d23/f4d to d19/d1a/d20/d23/d2d/fa4 0 2024-09-16T00:09:44.822 INFO:tasks.workunit.client.0.smithi139.stdout:2/468: mkdir d11/d30/d57/d29/d4f/d9c 0 2024-09-16T00:09:44.823 INFO:tasks.workunit.client.0.smithi139.stdout:5/445: write d1/d13/d3b/d3e/f6e [5100743,1751] 0 2024-09-16T00:09:44.824 INFO:tasks.workunit.client.0.smithi139.stdout:3/436: fdatasync d5/d7/dc/f8a 0 2024-09-16T00:09:44.826 INFO:tasks.workunit.client.0.smithi139.stdout:7/439: write d5/f12 [3627795,48300] 0 2024-09-16T00:09:44.828 INFO:tasks.workunit.client.0.smithi139.stdout:9/488: creat d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/fa5 x:0 0 0 2024-09-16T00:09:44.828 INFO:tasks.workunit.client.0.smithi139.stdout:0/467: dread d6/d1c/d41/d44/f71 [0,4194304] 0 2024-09-16T00:09:44.830 INFO:tasks.workunit.client.0.smithi139.stdout:2/469: dread d11/d30/d57/f59 [0,4194304] 0 2024-09-16T00:09:44.831 INFO:tasks.workunit.client.0.smithi139.stdout:9/489: dread - d19/d1a/d20/d23/d2d/d41/d71/d32/d4e/f8a zero size 2024-09-16T00:09:44.834 INFO:tasks.workunit.client.0.smithi139.stdout:7/440: symlink d5/d16/d22/l9c 0 2024-09-16T00:09:44.835 INFO:tasks.workunit.client.0.smithi139.stdout:8/483: rename d17/d1a/d43/d40/d4f/fa3 to d17/fa8 0 2024-09-16T00:09:44.836 INFO:tasks.workunit.client.0.smithi139.stdout:0/468: creat d6/d1c/d41/d7c/d86/f93 x:0 0 0 2024-09-16T00:09:44.837 INFO:tasks.workunit.client.0.smithi139.stdout:6/298: truncate d8/db/d26/d28/f31 3169284 0 2024-09-16T00:09:44.840 INFO:tasks.workunit.client.0.smithi139.stdout:4/399: rename d12/d2b/c16 to d12/d47/d49/c85 0 2024-09-16T00:09:44.841 INFO:tasks.workunit.client.0.smithi139.stdout:8/484: write d17/d1a/d2d/f3e [2726198,94354] 0 2024-09-16T00:09:44.841 INFO:tasks.workunit.client.0.smithi139.stdout:6/299: fdatasync d8/d20/f4a 0 2024-09-16T00:09:44.842 INFO:tasks.workunit.client.0.smithi139.stdout:2/470: dwrite fe [0,4194304] 0 2024-09-16T00:09:44.842 INFO:tasks.workunit.client.0.smithi139.stdout:5/446: truncate d1/f48 1848616 0 2024-09-16T00:09:44.843 INFO:tasks.workunit.client.0.smithi139.stdout:6/300: fdatasync f3 0 2024-09-16T00:09:44.844 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:44.844 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:44.845 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:44.845 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:44.845 INFO:tasks.workunit.client.0.smithi139.stdout:9/490: dwrite d19/d1a/d20/d23/d2d/d41/f7c [0,4194304] 0 2024-09-16T00:09:44.850 INFO:tasks.workunit.client.0.smithi139.stdout:8/485: dread - d17/d1a/d57/f79 zero size 2024-09-16T00:09:44.854 INFO:tasks.workunit.client.0.smithi139.stdout:7/441: dread f4 [0,4194304] 0 2024-09-16T00:09:44.857 INFO:tasks.workunit.client.0.smithi139.stdout:9/491: fdatasync d19/d1a/d20/d23/d2d/d35/d42/d3a/f6d 0 2024-09-16T00:09:44.859 INFO:tasks.workunit.client.0.smithi139.stdout:8/486: write d17/fa8 [969970,125882] 0 2024-09-16T00:09:44.859 INFO:tasks.workunit.client.0.smithi139.stdout:5/447: creat d1/d25/d73/d79/d8d/f96 x:0 0 0 2024-09-16T00:09:44.859 INFO:tasks.workunit.client.0.smithi139.stdout:4/400: dwrite d12/d2b/f73 [0,4194304] 0 2024-09-16T00:09:44.861 INFO:tasks.workunit.client.0.smithi139.stdout:2/471: getdents d11/d30 0 2024-09-16T00:09:44.862 INFO:tasks.workunit.client.0.smithi139.stdout:6/301: creat d8/db/d26/f66 x:0 0 0 2024-09-16T00:09:44.862 INFO:tasks.workunit.client.0.smithi139.stdout:5/448: chown d1/d13/d95 75 1 2024-09-16T00:09:44.865 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:09:44.866 INFO:tasks.workunit.client.0.smithi139.stdout:4/401: dread - d12/d47/d49/d60/f76 zero size 2024-09-16T00:09:44.866 INFO:tasks.workunit.client.0.smithi139.stdout:8/487: readlink d17/d1a/d43/d40/d4f/l5c 0 2024-09-16T00:09:44.866 INFO:tasks.workunit.client.0.smithi139.stdout:7/442: truncate d5/d16/d22/d25/f31 418707 0 2024-09-16T00:09:44.866 INFO:tasks.workunit.client.0.smithi139.stdout:2/472: write d11/d30/d57/d29/f5c [3570080,27657] 0 2024-09-16T00:09:44.866 INFO:tasks.workunit.client.0.smithi139.stdout:5/449: fsync d1/d25/d30/f55 0 2024-09-16T00:09:44.867 INFO:tasks.workunit.client.0.smithi139.stdout:6/302: mknod d8/db/c67 0 2024-09-16T00:09:44.870 INFO:tasks.workunit.client.0.smithi139.stdout:2/473: chown d11/d45/c8b 29 1 2024-09-16T00:09:44.870 INFO:tasks.workunit.client.0.smithi139.stdout:4/402: rename d12/f40 to d12/d4a/d4d/f86 0 2024-09-16T00:09:44.871 INFO:tasks.workunit.client.0.smithi139.stdout:0/469: dread d6/d1c/d41/d44/f3e [0,4194304] 0 2024-09-16T00:09:44.872 INFO:tasks.workunit.client.0.smithi139.stdout:8/488: creat d17/d1a/d2d/fa9 x:0 0 0 2024-09-16T00:09:44.874 INFO:tasks.workunit.client.0.smithi139.stdout:2/474: chown d11/d30/l7d 504 1 2024-09-16T00:09:44.876 INFO:tasks.workunit.client.0.smithi139.stdout:5/450: creat d1/d13/f97 x:0 0 0 2024-09-16T00:09:44.877 INFO:tasks.workunit.client.0.smithi139.stdout:9/492: dwrite d19/d1a/d20/d23/d2d/f90 [0,4194304] 0 2024-09-16T00:09:44.877 INFO:tasks.workunit.client.0.smithi139.stdout:7/443: link d5/l77 d5/d16/l9d 0 2024-09-16T00:09:44.879 INFO:tasks.workunit.client.0.smithi139.stdout:4/403: unlink fd 0 2024-09-16T00:09:44.879 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:44.879+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:44.881 INFO:tasks.workunit.client.0.smithi139.stdout:8/489: mknod d17/d1a/d57/caa 0 2024-09-16T00:09:44.881 INFO:tasks.workunit.client.0.smithi139.stdout:0/470: symlink d6/d1c/d41/d44/dd/d56/l94 0 2024-09-16T00:09:44.883 INFO:tasks.workunit.client.0.smithi139.stdout:6/303: dwrite d8/d1f/f41 [0,4194304] 0 2024-09-16T00:09:44.884 INFO:tasks.workunit.client.0.smithi139.stdout:5/451: write d1/d13/d3b/d3e/f5f [273851,9748] 0 2024-09-16T00:09:44.885 INFO:tasks.workunit.client.0.smithi139.stdout:9/493: mknod d19/d1a/d20/d23/d2d/d9a/ca6 0 2024-09-16T00:09:44.885 INFO:tasks.workunit.client.0.smithi139.stdout:0/471: chown d6/d5a/d70/f53 6 1 2024-09-16T00:09:44.887 INFO:tasks.workunit.client.0.smithi139.stdout:5/452: chown d1/d25/d30/d33/c38 3209025 1 2024-09-16T00:09:44.889 INFO:tasks.workunit.client.0.smithi139.stdout:8/490: rename d17/d1f/c33 to d17/d1f/d3a/cab 0 2024-09-16T00:09:44.894 INFO:tasks.workunit.client.0.smithi139.stdout:0/472: creat d6/d1c/d41/d76/f95 x:0 0 0 2024-09-16T00:09:44.895 INFO:tasks.workunit.client.0.smithi139.stdout:2/475: truncate d11/d30/d57/d29/f66 4324607 0 2024-09-16T00:09:44.896 INFO:tasks.workunit.client.0.smithi139.stdout:0/473: readlink d6/d1c/l69 0 2024-09-16T00:09:44.896 INFO:tasks.workunit.client.0.smithi139.stdout:4/404: dwrite d12/d2b/f26 [0,4194304] 0 2024-09-16T00:09:44.896 INFO:tasks.workunit.client.0.smithi139.stdout:7/444: dread d5/d26/f75 [0,4194304] 0 2024-09-16T00:09:44.898 INFO:tasks.workunit.client.0.smithi139.stdout:2/476: mknod d11/c9d 0 2024-09-16T00:09:44.898 INFO:tasks.workunit.client.0.smithi139.stdout:4/405: fsync d12/d47/d49/d60/f76 0 2024-09-16T00:09:44.899 INFO:tasks.workunit.client.0.smithi139.stdout:5/453: truncate d1/d13/d3b/f44 3221424 0 2024-09-16T00:09:44.899 INFO:tasks.workunit.client.0.smithi139.stdout:2/477: write d11/f38 [845083,103297] 0 2024-09-16T00:09:44.901 INFO:tasks.workunit.client.0.smithi139.stdout:4/406: symlink d12/l87 0 2024-09-16T00:09:44.906 INFO:tasks.workunit.client.0.smithi139.stdout:9/494: dwrite d19/f26 [0,4194304] 0 2024-09-16T00:09:44.906 INFO:tasks.workunit.client.0.smithi139.stdout:7/445: fdatasync d5/d16/d22/d25/f76 0 2024-09-16T00:09:44.906 INFO:tasks.workunit.client.0.smithi139.stdout:2/478: dread d11/f64 [0,4194304] 0 2024-09-16T00:09:44.909 INFO:tasks.workunit.client.0.smithi139.stdout:9/495: dread - d19/d1a/d20/d23/d2d/d35/d42/f5e zero size 2024-09-16T00:09:44.910 INFO:tasks.workunit.client.0.smithi139.stdout:5/454: getdents d1/d13/d3b 0 2024-09-16T00:09:44.912 INFO:tasks.workunit.client.0.smithi139.stdout:5/455: truncate d1/fa 1355552 0 2024-09-16T00:09:44.912 INFO:tasks.workunit.client.0.smithi139.stdout:2/479: mkdir d11/d30/d57/d29/d4f/d9e 0 2024-09-16T00:09:44.913 INFO:tasks.workunit.client.0.smithi139.stdout:4/407: link d12/d2b/c3e d12/d47/d49/c88 0 2024-09-16T00:09:44.915 INFO:tasks.workunit.client.0.smithi139.stdout:4/408: rmdir d12 39 2024-09-16T00:09:44.916 INFO:tasks.workunit.client.0.smithi139.stdout:9/496: mknod d19/d1a/d20/d23/d2d/d41/d71/da3/ca7 0 2024-09-16T00:09:44.917 INFO:tasks.workunit.client.0.smithi139.stdout:5/456: creat d1/d21/d49/f98 x:0 0 0 2024-09-16T00:09:44.918 INFO:tasks.workunit.client.0.smithi139.stdout:7/446: getdents d5/d16/d22/d25/d2d/d81 0 2024-09-16T00:09:44.922 INFO:tasks.workunit.client.0.smithi139.stdout:7/447: write d5/d16/d22/d25/d2d/d50/d44/f84 [1566809,129519] 0 2024-09-16T00:09:44.923 INFO:tasks.workunit.client.0.smithi139.stdout:4/409: dwrite d12/f42 [4194304,4194304] 0 2024-09-16T00:09:44.926 INFO:tasks.workunit.client.0.smithi139.stdout:5/457: rename d1/d21/d49/d53/d6a/d7b/d7a to d1/d21/d49/d53/d6a/d7b/d99 0 2024-09-16T00:09:44.926 INFO:tasks.workunit.client.0.smithi139.stdout:4/410: truncate d12/d2b/f6b 819912 0 2024-09-16T00:09:44.930 INFO:tasks.workunit.client.0.smithi139.stdout:4/411: creat d12/d4a/d4d/f89 x:0 0 0 2024-09-16T00:09:44.930 INFO:tasks.workunit.client.0.smithi139.stdout:1/413: sync 2024-09-16T00:09:44.932 INFO:tasks.workunit.client.0.smithi139.stdout:9/497: dread d19/d1a/f39 [0,4194304] 0 2024-09-16T00:09:44.932 INFO:tasks.workunit.client.0.smithi139.stdout:5/458: unlink d1/d25/c46 0 2024-09-16T00:09:44.932 INFO:tasks.workunit.client.0.smithi139.stdout:1/414: dread d7/d8/d27/f57 [0,4194304] 0 2024-09-16T00:09:44.933 INFO:tasks.workunit.client.0.smithi139.stdout:5/459: read d1/d25/d30/d33/f50 [150247,45450] 0 2024-09-16T00:09:44.935 INFO:tasks.workunit.client.0.smithi139.stdout:4/412: creat d12/d4a/d4d/d69/f8a x:0 0 0 2024-09-16T00:09:44.938 INFO:tasks.workunit.client.0.smithi139.stdout:5/460: symlink d1/d21/d49/d53/d6a/d7b/d99/l9a 0 2024-09-16T00:09:44.940 INFO:tasks.workunit.client.0.smithi139.stdout:9/498: rename d19/d86/f54 to d19/d1a/d20/d23/d2d/d41/d71/d32/d4e/d67/fa8 0 2024-09-16T00:09:44.941 INFO:tasks.workunit.client.0.smithi139.stdout:7/448: dread d5/d16/f34 [0,4194304] 0 2024-09-16T00:09:44.942 INFO:tasks.workunit.client.0.smithi139.stdout:9/499: write d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/f55 [1016448,101623] 0 2024-09-16T00:09:44.943 INFO:tasks.workunit.client.0.smithi139.stdout:4/413: write d12/d2b/f30 [4639019,75727] 0 2024-09-16T00:09:44.944 INFO:tasks.workunit.client.0.smithi139.stdout:1/415: getdents d7/d8/d40/d48/d4a 0 2024-09-16T00:09:44.945 INFO:tasks.workunit.client.0.smithi139.stdout:7/449: creat d5/d26/d46/f9e x:0 0 0 2024-09-16T00:09:44.946 INFO:tasks.workunit.client.0.smithi139.stdout:4/414: write d12/d2b/f30 [4871698,94830] 0 2024-09-16T00:09:44.946 INFO:tasks.workunit.client.0.smithi139.stdout:9/500: readlink d19/d1a/d20/d23/d2d/d41/d71/d32/l6b 0 2024-09-16T00:09:44.947 INFO:tasks.workunit.client.0.smithi139.stdout:1/416: chown d7/d8/d1f/f38 33 1 2024-09-16T00:09:44.949 INFO:tasks.workunit.client.0.smithi139.stdout:7/450: unlink d5/d16/l23 0 2024-09-16T00:09:44.949 INFO:tasks.workunit.client.0.smithi139.stdout:9/501: rmdir d19/d1a/d20/d89 39 2024-09-16T00:09:44.951 INFO:tasks.workunit.client.0.smithi139.stdout:5/461: dwrite d1/d13/f17 [8388608,4194304] 0 2024-09-16T00:09:44.954 INFO:tasks.workunit.client.0.smithi139.stdout:5/462: readlink d1/l35 0 2024-09-16T00:09:44.955 INFO:tasks.workunit.client.0.smithi139.stdout:9/502: symlink d19/d1a/d20/d23/d2d/d41/la9 0 2024-09-16T00:09:44.957 INFO:tasks.workunit.client.0.smithi139.stdout:7/451: truncate d5/d16/d22/d25/f76 84476 0 2024-09-16T00:09:44.957 INFO:tasks.workunit.client.0.smithi139.stdout:9/503: chown l18 42093060 1 2024-09-16T00:09:44.959 INFO:tasks.workunit.client.0.smithi139.stdout:5/463: fdatasync d1/d25/f6c 0 2024-09-16T00:09:44.959 INFO:tasks.workunit.client.0.smithi139.stdout:4/415: dwrite d12/d4a/d4d/d69/f8a [0,4194304] 0 2024-09-16T00:09:44.959 INFO:tasks.workunit.client.0.smithi139.stdout:1/417: dwrite d7/d8/d40/d47/f78 [0,4194304] 0 2024-09-16T00:09:44.960 INFO:tasks.workunit.client.0.smithi139.stdout:4/416: stat d12/d47/f45 0 2024-09-16T00:09:44.961 INFO:tasks.workunit.client.0.smithi139.stdout:7/452: unlink d5/d26/d46/f4a 0 2024-09-16T00:09:44.961 INFO:tasks.workunit.client.0.smithi139.stdout:9/504: fsync d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/fa5 0 2024-09-16T00:09:44.963 INFO:tasks.workunit.client.0.smithi139.stdout:1/418: creat d7/d8/d40/d48/d4a/d74/f7f x:0 0 0 2024-09-16T00:09:44.965 INFO:tasks.workunit.client.0.smithi139.stdout:9/505: readlink d19/d1a/d20/d23/d2d/d35/d42/l81 0 2024-09-16T00:09:44.965 INFO:tasks.workunit.client.0.smithi139.stdout:1/419: chown d7/d8/d27/c2d 14 1 2024-09-16T00:09:44.965 INFO:tasks.workunit.client.0.smithi139.stdout:7/453: rename d5/fb to d5/d37/d70/f9f 0 2024-09-16T00:09:44.967 INFO:tasks.workunit.client.0.smithi139.stdout:1/420: chown d7/d8/d40/d48/d53/c71 16403 1 2024-09-16T00:09:44.967 INFO:tasks.workunit.client.0.smithi139.stdout:7/454: chown d5/d26/d46/f9e 31717059 1 2024-09-16T00:09:44.968 INFO:tasks.workunit.client.0.smithi139.stdout:5/464: link d1/f27 d1/d21/d49/d53/d6a/f9b 0 2024-09-16T00:09:44.969 INFO:tasks.workunit.client.0.smithi139.stdout:9/506: creat d19/d1a/d20/d23/d2d/d9a/faa x:0 0 0 2024-09-16T00:09:44.970 INFO:tasks.workunit.client.0.smithi139.stdout:5/465: chown d1/d21/d49/d53/d6a/d7b/d99/l9a 58434 1 2024-09-16T00:09:44.970 INFO:tasks.workunit.client.0.smithi139.stdout:1/421: mknod d7/d8/d27/d68/c80 0 2024-09-16T00:09:44.970 INFO:tasks.workunit.client.0.smithi139.stdout:7/455: rmdir d5/d16/d22/d25/d63 39 2024-09-16T00:09:44.972 INFO:tasks.workunit.client.0.smithi139.stdout:1/422: read d7/d8/f1e [1558094,108189] 0 2024-09-16T00:09:44.972 INFO:tasks.workunit.client.0.smithi139.stdout:7/456: write d5/d37/f79 [404100,17338] 0 2024-09-16T00:09:44.973 INFO:tasks.workunit.client.0.smithi139.stdout:1/423: write d7/d8/d40/d47/f5e [126943,84648] 0 2024-09-16T00:09:44.973 INFO:tasks.workunit.client.0.smithi139.stdout:5/466: creat d1/d25/d30/d33/d43/f9c x:0 0 0 2024-09-16T00:09:44.974 INFO:tasks.workunit.client.0.smithi139.stdout:9/507: symlink d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/lab 0 2024-09-16T00:09:44.980 INFO:tasks.workunit.client.0.smithi139.stdout:5/467: dwrite d1/d13/d3b/f7d [0,4194304] 0 2024-09-16T00:09:44.980 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:44.979+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:44.981 INFO:tasks.workunit.client.0.smithi139.stdout:9/508: dread d19/d1a/d20/d23/d2d/d41/f7c [0,4194304] 0 2024-09-16T00:09:44.982 INFO:tasks.workunit.client.0.smithi139.stdout:7/457: unlink d5/d16/l17 0 2024-09-16T00:09:44.983 INFO:tasks.workunit.client.0.smithi139.stdout:5/468: mknod d1/d13/d3f/d5e/c9d 0 2024-09-16T00:09:44.983 INFO:tasks.workunit.client.0.smithi139.stdout:9/509: truncate d19/d1a/d20/d23/d2d/d41/d71/d32/d4e/f8a 208038 0 2024-09-16T00:09:44.984 INFO:tasks.workunit.client.0.smithi139.stdout:7/458: write d5/d16/d22/d25/d2d/f3a [227519,91630] 0 2024-09-16T00:09:44.985 INFO:tasks.workunit.client.0.smithi139.stdout:1/424: rename d7/d8/d40/d46/f70 to d7/f81 0 2024-09-16T00:09:44.987 INFO:tasks.workunit.client.0.smithi139.stdout:1/425: chown d7/d8/d40/f66 70 1 2024-09-16T00:09:44.987 INFO:tasks.workunit.client.0.smithi139.stdout:9/510: dread - d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/fa5 zero size 2024-09-16T00:09:44.987 INFO:tasks.workunit.client.0.smithi139.stdout:7/459: chown d5/d16/d22/d25/l56 1547 1 2024-09-16T00:09:44.988 INFO:tasks.workunit.client.0.smithi139.stdout:1/426: fsync d7/d8/d27/f2c 0 2024-09-16T00:09:44.989 INFO:tasks.workunit.client.0.smithi139.stdout:5/469: link d1/d21/d49/f81 d1/d25/d73/d79/d8d/f9e 0 2024-09-16T00:09:44.989 INFO:tasks.workunit.client.0.smithi139.stdout:1/427: chown d7/d8/d40/d48/d67/l6b 946 1 2024-09-16T00:09:44.991 INFO:tasks.workunit.client.0.smithi139.stdout:5/470: write d1/d25/f5c [717184,43935] 0 2024-09-16T00:09:44.991 INFO:tasks.workunit.client.0.smithi139.stdout:9/511: creat d19/d1a/d20/d23/d2d/d9a/fac x:0 0 0 2024-09-16T00:09:44.991 INFO:tasks.workunit.client.0.smithi139.stdout:7/460: unlink d5/d16/d22/l9c 0 2024-09-16T00:09:44.993 INFO:tasks.workunit.client.0.smithi139.stdout:1/428: mkdir d7/d8/d40/d48/d4a/d82 0 2024-09-16T00:09:44.993 INFO:tasks.workunit.client.0.smithi139.stdout:7/461: write d5/d37/f40 [4641597,120688] 0 2024-09-16T00:09:44.993 INFO:tasks.workunit.client.0.smithi139.stdout:1/429: write d7/f28 [3268353,18499] 0 2024-09-16T00:09:45.000 INFO:tasks.workunit.client.0.smithi139.stdout:5/471: rename d1/d25 to d1/d13/d3f/d9f 0 2024-09-16T00:09:45.006 INFO:tasks.workunit.client.0.smithi139.stdout:1/430: getdents d7/d8/d27/d2f 0 2024-09-16T00:09:45.006 INFO:tasks.workunit.client.0.smithi139.stdout:9/512: truncate d19/d1a/d20/d23/d2d/fa4 409734 0 2024-09-16T00:09:45.007 INFO:tasks.workunit.client.0.smithi139.stdout:7/462: write d5/d16/d22/f6f [5105392,49124] 0 2024-09-16T00:09:45.007 INFO:tasks.workunit.client.0.smithi139.stdout:1/431: chown d7/d8/d40/d48/d4a/d82 54984437 1 2024-09-16T00:09:45.009 INFO:tasks.workunit.client.0.smithi139.stdout:5/472: write d1/d13/d3f/d9f/f6c [1153981,23601] 0 2024-09-16T00:09:45.011 INFO:tasks.workunit.client.0.smithi139.stdout:7/463: mknod d5/d16/d22/d25/ca0 0 2024-09-16T00:09:45.011 INFO:tasks.workunit.client.0.smithi139.stdout:5/473: mknod d1/d21/d49/d87/ca0 0 2024-09-16T00:09:45.013 INFO:tasks.workunit.client.0.smithi139.stdout:5/474: fdatasync d1/d13/d3b/d3e/f5f 0 2024-09-16T00:09:45.016 INFO:tasks.workunit.client.0.smithi139.stdout:9/513: dwrite d19/d1a/d20/d23/d2d/d41/f6f [0,4194304] 0 2024-09-16T00:09:45.020 INFO:tasks.workunit.client.0.smithi139.stdout:7/464: creat d5/d16/d22/d25/d2d/d81/fa1 x:0 0 0 2024-09-16T00:09:45.021 INFO:tasks.workunit.client.0.smithi139.stdout:1/432: dwrite d7/d8/d27/f34 [4194304,4194304] 0 2024-09-16T00:09:45.023 INFO:tasks.workunit.client.0.smithi139.stdout:9/514: creat d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/fad x:0 0 0 2024-09-16T00:09:45.025 INFO:tasks.workunit.client.0.smithi139.stdout:1/433: mkdir d7/d8/d27/d68/d7c/d83 0 2024-09-16T00:09:45.026 INFO:tasks.workunit.client.0.smithi139.stdout:9/515: unlink d19/d1a/d20/d23/d2d/d41/f6f 0 2024-09-16T00:09:45.027 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:45.028 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:45.028 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:45.028 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:45.028 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:45.028 INFO:tasks.workunit.client.0.smithi139.stdout:5/475: dread d1/f1a [0,4194304] 0 2024-09-16T00:09:45.029 INFO:tasks.workunit.client.0.smithi139.stdout:5/476: stat d1/d13/d3b/d3e/f69 0 2024-09-16T00:09:45.030 INFO:tasks.workunit.client.0.smithi139.stdout:9/516: write d19/d1a/d20/d23/d2d/d41/f7c [1956943,16301] 0 2024-09-16T00:09:45.031 INFO:tasks.workunit.client.0.smithi139.stdout:5/477: dread - d1/d21/d49/f98 zero size 2024-09-16T00:09:45.032 INFO:tasks.workunit.client.0.smithi139.stdout:5/478: read - d1/d21/d49/f98 zero size 2024-09-16T00:09:45.033 INFO:tasks.workunit.client.0.smithi139.stdout:1/434: rename d7/d8/d40/f66 to d7/d8/d40/d48/d4a/d74/f84 0 2024-09-16T00:09:45.034 INFO:tasks.workunit.client.0.smithi139.stdout:9/517: dread d19/d1a/d20/d89/f9d [0,4194304] 0 2024-09-16T00:09:45.037 INFO:tasks.workunit.client.0.smithi139.stdout:5/479: dread d1/f4a [0,4194304] 0 2024-09-16T00:09:45.037 INFO:tasks.workunit.client.0.smithi139.stdout:9/518: mknod d19/d1a/d20/d23/d2d/d35/d42/d3a/cae 0 2024-09-16T00:09:45.039 INFO:tasks.workunit.client.0.smithi139.stdout:9/519: dread - d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/fa5 zero size 2024-09-16T00:09:45.040 INFO:tasks.workunit.client.0.smithi139.stdout:5/480: mkdir d1/d21/d49/d53/d6a/d7b/da1 0 2024-09-16T00:09:45.040 INFO:tasks.workunit.client.0.smithi139.stdout:9/520: readlink ld 0 2024-09-16T00:09:45.041 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:09:45.045 INFO:tasks.workunit.client.0.smithi139.stdout:9/521: link d19/d1a/d20/d23/d2d/d41/d71/d32/d4e/d67/fa8 d19/d1a/faf 0 2024-09-16T00:09:45.045 INFO:tasks.workunit.client.0.smithi139.stdout:1/435: dread d7/d8/d27/f79 [0,4194304] 0 2024-09-16T00:09:45.045 INFO:tasks.workunit.client.0.smithi139.stdout:5/481: link d1/d13/d3f/l90 d1/d13/d3f/d5e/la2 0 2024-09-16T00:09:45.047 INFO:tasks.workunit.client.0.smithi139.stdout:1/436: write d7/f18 [3669791,19497] 0 2024-09-16T00:09:45.047 INFO:tasks.workunit.client.0.smithi139.stdout:5/482: truncate d1/d13/d22/f6b 585938 0 2024-09-16T00:09:45.047 INFO:tasks.workunit.client.0.smithi139.stdout:1/437: readlink d7/l7b 0 2024-09-16T00:09:45.048 INFO:tasks.workunit.client.0.smithi139.stdout:5/483: dread - d1/d21/d49/d53/d6a/f8b zero size 2024-09-16T00:09:45.049 INFO:tasks.workunit.client.0.smithi139.stdout:1/438: getdents d7/d8/d27/d68/d7c/d83 0 2024-09-16T00:09:45.050 INFO:tasks.workunit.client.0.smithi139.stdout:1/439: read d7/f18 [417459,46878] 0 2024-09-16T00:09:45.051 INFO:tasks.workunit.client.0.smithi139.stdout:1/440: chown d7/d8/d27/f57 4176437 1 2024-09-16T00:09:45.051 INFO:tasks.workunit.client.0.smithi139.stdout:5/484: truncate d1/d13/d3f/d9f/d73/d79/d8d/f9e 3972879 0 2024-09-16T00:09:45.053 INFO:tasks.workunit.client.0.smithi139.stdout:1/441: rename d7/d8/d40/d48/d4a/d74/f84 to d7/d8/d40/d48/d53/f85 0 2024-09-16T00:09:45.055 INFO:tasks.workunit.client.0.smithi139.stdout:1/442: dread - d7/d8/d40/d48/d53/f85 zero size 2024-09-16T00:09:45.057 INFO:tasks.workunit.client.0.smithi139.stdout:9/522: write d19/d1a/d20/d23/d2d/d2f/f96 [752304,85325] 0 2024-09-16T00:09:45.058 INFO:tasks.workunit.client.0.smithi139.stdout:5/485: dwrite d1/d13/d3f/d9f/f5c [0,4194304] 0 2024-09-16T00:09:45.061 INFO:tasks.workunit.client.0.smithi139.stdout:5/486: mknod d1/d13/d3f/d9f/d73/d93/ca3 0 2024-09-16T00:09:45.062 INFO:tasks.workunit.client.0.smithi139.stdout:9/523: rename f2 to d19/d1a/d20/d23/d2d/d41/d71/fb0 0 2024-09-16T00:09:45.062 INFO:tasks.workunit.client.0.smithi139.stdout:5/487: chown d1/d13/d3f/d9f/d30/d33/f7c 17252 1 2024-09-16T00:09:45.066 INFO:tasks.workunit.client.0.smithi139.stdout:5/488: dread d1/f1a [0,4194304] 0 2024-09-16T00:09:45.067 INFO:tasks.workunit.client.0.smithi139.stdout:9/524: link d19/d1a/d20/d23/d2d/d35/d42/d3a/f6e d19/d1a/d20/d23/d2d/d35/fb1 0 2024-09-16T00:09:45.071 INFO:tasks.workunit.client.0.smithi139.stdout:9/525: rename d19/d1a/d20/d23/d2d/d9a/c9c to d19/d1a/d20/d23/d2d/d41/d9f/cb2 0 2024-09-16T00:09:45.072 INFO:tasks.workunit.client.0.smithi139.stdout:5/489: getdents d1/d21/d49/d53/d6a 0 2024-09-16T00:09:45.072 INFO:tasks.workunit.client.0.smithi139.stdout:9/526: readlink d19/d1a/d20/d23/d2d/d35/d42/l4c 0 2024-09-16T00:09:45.073 INFO:tasks.workunit.client.0.smithi139.stdout:5/490: chown d1/d13/d3f/d9f/d30/d33/d68 23184 1 2024-09-16T00:09:45.074 INFO:tasks.workunit.client.0.smithi139.stdout:5/491: creat d1/d13/d3f/d9f/fa4 x:0 0 0 2024-09-16T00:09:45.077 INFO:tasks.workunit.client.0.smithi139.stdout:5/492: unlink d1/c12 0 2024-09-16T00:09:45.078 INFO:tasks.workunit.client.0.smithi139.stdout:5/493: fdatasync d1/d21/d49/d53/d6a/d7b/d78/f8e 0 2024-09-16T00:09:45.079 INFO:tasks.workunit.client.0.smithi139.stdout:9/527: dwrite d19/d1a/d20/d23/d2d/d35/d42/d3a/f48 [0,4194304] 0 2024-09-16T00:09:45.081 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:45.081+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:45.082 INFO:tasks.workunit.client.0.smithi139.stdout:9/528: write d19/d1a/d20/d23/d2d/d41/d71/d32/d79/f91 [936762,101779] 0 2024-09-16T00:09:45.082 INFO:tasks.workunit.client.0.smithi139.stdout:5/494: mkdir d1/d13/d3f/d9f/d73/d79/da5 0 2024-09-16T00:09:45.084 INFO:tasks.workunit.client.0.smithi139.stdout:5/495: truncate d1/d13/f97 783375 0 2024-09-16T00:09:45.087 INFO:tasks.workunit.client.0.smithi139.stdout:5/496: dread d1/d13/d3b/f7d [0,4194304] 0 2024-09-16T00:09:45.088 INFO:tasks.workunit.client.0.smithi139.stdout:5/497: mknod d1/d13/d3f/ca6 0 2024-09-16T00:09:45.098 INFO:tasks.workunit.client.0.smithi139.stdout:5/498: write d1/d13/d3f/d9f/d73/d79/d8d/f9e [1552268,79767] 0 2024-09-16T00:09:45.101 INFO:tasks.workunit.client.0.smithi139.stdout:9/529: dread d19/d86/f5a [0,4194304] 0 2024-09-16T00:09:45.103 INFO:tasks.workunit.client.0.smithi139.stdout:9/530: write d19/d1a/d20/d23/d2d/d35/f61 [713185,22209] 0 2024-09-16T00:09:45.103 INFO:tasks.workunit.client.0.smithi139.stdout:5/499: fdatasync d1/d13/d3b/f4c 0 2024-09-16T00:09:45.107 INFO:tasks.workunit.client.0.smithi139.stdout:9/531: rename d19/d1a/d20/d23/d2d/fa0 to d19/d1a/d20/d23/d2d/d35/d60/fb3 0 2024-09-16T00:09:45.110 INFO:tasks.workunit.client.0.smithi139.stdout:9/532: unlink d19/d1a/d20/d23/d2d/d41/la9 0 2024-09-16T00:09:45.111 INFO:tasks.workunit.client.0.smithi139.stdout:9/533: readlink d19/d1a/d20/d23/d2d/d35/d42/l81 0 2024-09-16T00:09:45.112 INFO:tasks.workunit.client.0.smithi139.stdout:9/534: readlink d19/d1a/d20/d23/d2d/d41/d71/d32/l6b 0 2024-09-16T00:09:45.112 INFO:tasks.workunit.client.0.smithi139.stdout:3/437: sync 2024-09-16T00:09:45.112 INFO:tasks.workunit.client.0.smithi139.stdout:5/500: rename d1/f2f to d1/d21/d49/d53/d6a/fa7 0 2024-09-16T00:09:45.113 INFO:tasks.workunit.client.0.smithi139.stdout:3/438: chown d5/d7/d23/f77 0 1 2024-09-16T00:09:45.113 INFO:tasks.workunit.client.0.smithi139.stdout:3/439: dread - d5/d7/d23/d27/d2f/d30/d41/f6b zero size 2024-09-16T00:09:45.114 INFO:tasks.workunit.client.0.smithi139.stdout:9/535: creat d19/d1a/d20/d23/d2d/d41/d71/d32/fb4 x:0 0 0 2024-09-16T00:09:45.114 INFO:tasks.workunit.client.0.smithi139.stdout:3/440: chown d5/d7/d23/d27/c8e 67 1 2024-09-16T00:09:45.116 INFO:tasks.workunit.client.0.smithi139.stdout:3/441: stat d5/d7/l1a 0 2024-09-16T00:09:45.118 INFO:tasks.workunit.client.0.smithi139.stdout:9/536: mkdir d19/d1a/d20/d23/d2d/d41/db5 0 2024-09-16T00:09:45.119 INFO:tasks.workunit.client.0.smithi139.stdout:3/442: rename d5/d7/d23/d27/d2f/d30/d4c to d5/d7/d23/d27/d2f/d30/d4c/da1/da2 22 2024-09-16T00:09:45.119 INFO:tasks.workunit.client.0.smithi139.stdout:9/537: dread - d19/d1a/f8f zero size 2024-09-16T00:09:45.122 INFO:tasks.workunit.client.0.smithi139.stdout:3/443: unlink d5/d7/d23/d27/d2f/d30/d4c/d50/f59 0 2024-09-16T00:09:45.124 INFO:tasks.workunit.client.0.smithi139.stdout:5/501: rename d1/fa to d1/d21/fa8 0 2024-09-16T00:09:45.127 INFO:tasks.workunit.client.0.smithi139.stdout:3/444: link d5/d7/d23/d35/f45 d5/d7/dc/d9c/fa3 0 2024-09-16T00:09:45.129 INFO:tasks.workunit.client.0.smithi139.stdout:3/445: chown d5/d7/dc/l85 30 1 2024-09-16T00:09:45.130 INFO:tasks.workunit.client.0.smithi139.stdout:9/538: dread d19/d1a/d20/d23/d2d/d35/f66 [0,4194304] 0 2024-09-16T00:09:45.131 INFO:tasks.workunit.client.0.smithi139.stdout:3/446: symlink d5/d7/d23/la4 0 2024-09-16T00:09:45.133 INFO:tasks.workunit.client.0.smithi139.stdout:3/447: unlink d5/d7/f2e 0 2024-09-16T00:09:45.135 INFO:tasks.workunit.client.0.smithi139.stdout:3/448: mknod d5/d7/d23/d48/ca5 0 2024-09-16T00:09:45.136 INFO:tasks.workunit.client.0.smithi139.stdout:9/539: creat d19/d1a/d20/d23/d2d/d41/d71/d32/d4e/fb6 x:0 0 0 2024-09-16T00:09:45.137 INFO:tasks.workunit.client.0.smithi139.stdout:3/449: symlink d5/la6 0 2024-09-16T00:09:45.138 INFO:tasks.workunit.client.0.smithi139.stdout:9/540: fsync d19/d1a/d20/d23/d2d/d41/d71/fb0 0 2024-09-16T00:09:45.140 INFO:tasks.workunit.client.0.smithi139.stdout:9/541: mknod d19/d1a/cb7 0 2024-09-16T00:09:45.141 INFO:tasks.workunit.client.0.smithi139.stdout:5/502: dread d1/f27 [0,4194304] 0 2024-09-16T00:09:45.143 INFO:tasks.workunit.client.0.smithi139.stdout:9/542: rmdir d19/d1a/d20/d23/d2d/d41/d71/d32/d4e 39 2024-09-16T00:09:45.144 INFO:tasks.workunit.client.0.smithi139.stdout:3/450: rename d5/d7/d23/d48 to d5/d7/d23/d35/da7 0 2024-09-16T00:09:45.145 INFO:tasks.workunit.client.0.smithi139.stdout:9/543: fsync d19/d1a/d20/d23/d2d/d41/d71/d32/d4e/f78 0 2024-09-16T00:09:45.146 INFO:tasks.workunit.client.0.smithi139.stdout:5/503: mkdir d1/d13/d22/da9 0 2024-09-16T00:09:45.147 INFO:tasks.workunit.client.0.smithi139.stdout:9/544: creat d19/d1a/d20/d89/fb8 x:0 0 0 2024-09-16T00:09:45.149 INFO:tasks.workunit.client.0.smithi139.stdout:3/451: getdents d5/d7/d23/d27 0 2024-09-16T00:09:45.149 INFO:tasks.workunit.client.0.smithi139.stdout:9/545: dread - d19/d1a/d20/d23/d2d/d35/d60/fb3 zero size 2024-09-16T00:09:45.150 INFO:tasks.workunit.client.0.smithi139.stdout:5/504: read d1/d21/d49/d53/f51 [245214,34237] 0 2024-09-16T00:09:45.150 INFO:tasks.workunit.client.0.smithi139.stdout:3/452: dread - d5/d7/f7d zero size 2024-09-16T00:09:45.152 INFO:tasks.workunit.client.0.smithi139.stdout:5/505: creat d1/d21/d49/d53/faa x:0 0 0 2024-09-16T00:09:45.152 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:09:45.157 INFO:tasks.workunit.client.0.smithi139.stdout:9/546: rename d19/d1a/d20/d23/d2d/d41/d71/d32/c98 to d19/cb9 0 2024-09-16T00:09:45.157 INFO:tasks.workunit.client.0.smithi139.stdout:5/506: truncate d1/d13/d3f/d9f/d30/d33/f50 1290822 0 2024-09-16T00:09:45.158 INFO:tasks.workunit.client.0.smithi139.stdout:9/547: dread - d19/d1a/d20/d23/d2d/f92 zero size 2024-09-16T00:09:45.158 INFO:tasks.workunit.client.0.smithi139.stdout:3/453: dwrite d5/d7/dc/f8a [0,4194304] 0 2024-09-16T00:09:45.160 INFO:tasks.workunit.client.0.smithi139.stdout:5/507: write d1/d13/d3b/f7d [4715634,21680] 0 2024-09-16T00:09:45.161 INFO:tasks.workunit.client.0.smithi139.stdout:9/548: symlink d19/d1a/d20/d23/d2d/d41/db5/lba 0 2024-09-16T00:09:45.163 INFO:tasks.workunit.client.0.smithi139.stdout:9/549: dread - d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/fa5 zero size 2024-09-16T00:09:45.165 INFO:tasks.workunit.client.0.smithi139.stdout:5/508: creat d1/d21/d49/d53/d6a/d7b/d99/fab x:0 0 0 2024-09-16T00:09:45.166 INFO:tasks.workunit.client.0.smithi139.stdout:9/550: creat d19/d1a/d20/d89/d93/fbb x:0 0 0 2024-09-16T00:09:45.167 INFO:tasks.workunit.client.0.smithi139.stdout:9/551: write d19/d1a/f50 [1174372,90667] 0 2024-09-16T00:09:45.168 INFO:tasks.workunit.client.0.smithi139.stdout:5/509: creat d1/d21/d49/d53/d6a/d7b/d99/fac x:0 0 0 2024-09-16T00:09:45.170 INFO:tasks.workunit.client.0.smithi139.stdout:9/552: symlink d19/d1a/d20/d23/d2d/d35/d44/lbc 0 2024-09-16T00:09:45.172 INFO:tasks.workunit.client.0.smithi139.stdout:9/553: mkdir d19/d1a/d20/d89/dbd 0 2024-09-16T00:09:45.173 INFO:tasks.workunit.client.0.smithi139.stdout:5/510: dread d1/f1a [0,4194304] 0 2024-09-16T00:09:45.175 INFO:tasks.workunit.client.0.smithi139.stdout:5/511: read d1/d13/d3f/d9f/f36 [3388209,21498] 0 2024-09-16T00:09:45.175 INFO:tasks.workunit.client.0.smithi139.stdout:5/512: fdatasync d1/d13/d3f/d9f/f5c 0 2024-09-16T00:09:45.179 INFO:tasks.workunit.client.0.smithi139.stdout:9/554: write d19/d1a/faf [3701481,43379] 0 2024-09-16T00:09:45.180 INFO:tasks.workunit.client.0.smithi139.stdout:9/555: stat d19/d1a/d20/d23/d2d/d35/d60 0 2024-09-16T00:09:45.182 INFO:tasks.workunit.client.0.smithi139.stdout:9/556: mkdir d19/d1a/d20/dbe 0 2024-09-16T00:09:45.183 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:45.182+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:45.185 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:45.185 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:45.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:45.185 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:45.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.185 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:219 2.1f 2:f83d65d8:::200.00000004:head [write 863658~2915703 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e65)", 2024-09-16T00:09:45.185 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:45.186 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 34.311289057000003, 2024-09-16T00:09:45.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013772627000000001, 2024-09-16T00:09:45.186 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.186 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.186 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.186 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.186 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:45.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:45.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870941+0000", 2024-09-16T00:09:45.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4e-07 2024-09-16T00:09:45.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877134+0000", 2024-09-16T00:09:45.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061936769999999999 2024-09-16T00:09:45.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877153+0000", 2024-09-16T00:09:45.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8408000000000001e-05 2024-09-16T00:09:45.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877161+0000", 2024-09-16T00:09:45.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2709999999999992e-06 2024-09-16T00:09:45.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877225+0000", 2024-09-16T00:09:45.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4190999999999996e-05 2024-09-16T00:09:45.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877289+0000", 2024-09-16T00:09:45.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4058000000000002e-05 2024-09-16T00:09:45.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:45.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877376+0000", 2024-09-16T00:09:45.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7227999999999999e-05 2024-09-16T00:09:45.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.882925+0000", 2024-09-16T00:09:45.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055481050000000002 2024-09-16T00:09:45.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884635+0000", 2024-09-16T00:09:45.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017102759999999999 2024-09-16T00:09:45.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.192 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884674+0000", 2024-09-16T00:09:45.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8914e-05 2024-09-16T00:09:45.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.192 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884712+0000", 2024-09-16T00:09:45.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658999999999997e-05 2024-09-16T00:09:45.192 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.192 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.192 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.193 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10742 3.1 3:823e197f:::10000003110.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:45.193 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.281167+0000", 2024-09-16T00:09:45.193 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.9010621050000003, 2024-09-16T00:09:45.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010663346000000001, 2024-09-16T00:09:45.193 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.193 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.193 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.193 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.193 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.193 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.193 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10742 2024-09-16T00:09:45.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.194 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.281167+0000", 2024-09-16T00:09:45.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.281167+0000", 2024-09-16T00:09:45.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.281168+0000", 2024-09-16T00:09:45.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7120000000000001e-06 2024-09-16T00:09:45.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.282581+0000", 2024-09-16T00:09:45.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014120999999999999 2024-09-16T00:09:45.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.282602+0000", 2024-09-16T00:09:45.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1310000000000001e-05 2024-09-16T00:09:45.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.282615+0000", 2024-09-16T00:09:45.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.33e-05 2024-09-16T00:09:45.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.282713+0000", 2024-09-16T00:09:45.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8306000000000004e-05 2024-09-16T00:09:45.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.282852+0000", 2024-09-16T00:09:45.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013900100000000001 2024-09-16T00:09:45.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:45.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.283022+0000", 2024-09-16T00:09:45.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000169677 2024-09-16T00:09:45.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.289309+0000", 2024-09-16T00:09:45.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0062870519999999996 2024-09-16T00:09:45.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.291694+0000", 2024-09-16T00:09:45.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023847199999999999 2024-09-16T00:09:45.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.291772+0000", 2024-09-16T00:09:45.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7947000000000004e-05 2024-09-16T00:09:45.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.291830+0000", 2024-09-16T00:09:45.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8220999999999997e-05 2024-09-16T00:09:45.199 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.199 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.199 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.199 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10767 3.1b 3:dab1607f:::100000030fd.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:45.199 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.379273+0000", 2024-09-16T00:09:45.199 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.8029563250000002, 2024-09-16T00:09:45.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012051947, 2024-09-16T00:09:45.199 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.199 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.200 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.200 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.200 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.200 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.200 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10767 2024-09-16T00:09:45.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.200 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.379273+0000", 2024-09-16T00:09:45.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.379273+0000", 2024-09-16T00:09:45.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.379275+0000", 2024-09-16T00:09:45.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6230000000000001e-06 2024-09-16T00:09:45.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381583+0000", 2024-09-16T00:09:45.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023079939999999998 2024-09-16T00:09:45.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381617+0000", 2024-09-16T00:09:45.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4227999999999998e-05 2024-09-16T00:09:45.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381633+0000", 2024-09-16T00:09:45.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5642e-05 2024-09-16T00:09:45.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381739+0000", 2024-09-16T00:09:45.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010638500000000001 2024-09-16T00:09:45.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381850+0000", 2024-09-16T00:09:45.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110164 2024-09-16T00:09:45.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:45.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.382033+0000", 2024-09-16T00:09:45.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018355699999999999 2024-09-16T00:09:45.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.388313+0000", 2024-09-16T00:09:45.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006279588 2024-09-16T00:09:45.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.391197+0000", 2024-09-16T00:09:45.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028841130000000002 2024-09-16T00:09:45.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.391263+0000", 2024-09-16T00:09:45.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6105000000000005e-05 2024-09-16T00:09:45.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.205 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.391324+0000", 2024-09-16T00:09:45.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1548000000000001e-05 2024-09-16T00:09:45.205 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.205 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.205 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.205 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10768 3.2b 3:d738455c:::100000030db.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:45.205 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:09:45.206 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.7881038839999999, 2024-09-16T00:09:45.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012411195999999999, 2024-09-16T00:09:45.206 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.206 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.206 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.206 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.206 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.206 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.206 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10768 2024-09-16T00:09:45.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.206 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:09:45.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:09:45.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394129+0000", 2024-09-16T00:09:45.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4100000000000001e-06 2024-09-16T00:09:45.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395723+0000", 2024-09-16T00:09:45.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015938409999999999 2024-09-16T00:09:45.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395745+0000", 2024-09-16T00:09:45.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1611e-05 2024-09-16T00:09:45.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395782+0000", 2024-09-16T00:09:45.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7292000000000001e-05 2024-09-16T00:09:45.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395856+0000", 2024-09-16T00:09:45.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3675000000000002e-05 2024-09-16T00:09:45.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395924+0000", 2024-09-16T00:09:45.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8646999999999995e-05 2024-09-16T00:09:45.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:45.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.396069+0000", 2024-09-16T00:09:45.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014455199999999999 2024-09-16T00:09:45.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.403755+0000", 2024-09-16T00:09:45.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0076863160000000003 2024-09-16T00:09:45.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406407+0000", 2024-09-16T00:09:45.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026516220000000002 2024-09-16T00:09:45.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406477+0000", 2024-09-16T00:09:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0204999999999996e-05 2024-09-16T00:09:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406536+0000", 2024-09-16T00:09:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9024999999999998e-05 2024-09-16T00:09:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10772 3.1 3:823e197f:::10000003110.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.415400+0000", 2024-09-16T00:09:45.211 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.7668290429999995, 2024-09-16T00:09:45.212 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011076012999999999, 2024-09-16T00:09:45.212 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.212 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.212 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.212 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.212 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.212 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.212 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10772 2024-09-16T00:09:45.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.212 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.212 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.212 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.415400+0000", 2024-09-16T00:09:45.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.415400+0000", 2024-09-16T00:09:45.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.415402+0000", 2024-09-16T00:09:45.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.481e-06 2024-09-16T00:09:45.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.417895+0000", 2024-09-16T00:09:45.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002492247 2024-09-16T00:09:45.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.417917+0000", 2024-09-16T00:09:45.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1977999999999999e-05 2024-09-16T00:09:45.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.417925+0000", 2024-09-16T00:09:45.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7220000000000005e-06 2024-09-16T00:09:45.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.418000+0000", 2024-09-16T00:09:45.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4715000000000005e-05 2024-09-16T00:09:45.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.418137+0000", 2024-09-16T00:09:45.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013744699999999999 2024-09-16T00:09:45.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:45.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.418295+0000", 2024-09-16T00:09:45.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000157622 2024-09-16T00:09:45.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.424462+0000", 2024-09-16T00:09:45.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061665369999999997 2024-09-16T00:09:45.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.426370+0000", 2024-09-16T00:09:45.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001908217 2024-09-16T00:09:45.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.426424+0000", 2024-09-16T00:09:45.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4693e-05 2024-09-16T00:09:45.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.426476+0000", 2024-09-16T00:09:45.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1354000000000003e-05 2024-09-16T00:09:45.217 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.218 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.218 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.218 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10909 3.1 3:8396d031:::100000031ab.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:45.218 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.183024+0000", 2024-09-16T00:09:45.218 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.9992050389999996, 2024-09-16T00:09:45.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010857537, 2024-09-16T00:09:45.218 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.218 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.219 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.219 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.219 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.219 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.219 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10909 2024-09-16T00:09:45.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.219 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.219 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.219 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.219 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.183024+0000", 2024-09-16T00:09:45.219 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.183024+0000", 2024-09-16T00:09:45.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.183026+0000", 2024-09-16T00:09:45.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4839999999999998e-06 2024-09-16T00:09:45.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.185491+0000", 2024-09-16T00:09:45.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024650710000000001 2024-09-16T00:09:45.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.185511+0000", 2024-09-16T00:09:45.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9519999999999999e-05 2024-09-16T00:09:45.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.185519+0000", 2024-09-16T00:09:45.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2719999999999997e-06 2024-09-16T00:09:45.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.185591+0000", 2024-09-16T00:09:45.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1571999999999998e-05 2024-09-16T00:09:45.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.185773+0000", 2024-09-16T00:09:45.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000181962 2024-09-16T00:09:45.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:45.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.185927+0000", 2024-09-16T00:09:45.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000153948 2024-09-16T00:09:45.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.192576+0000", 2024-09-16T00:09:45.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066497550000000002 2024-09-16T00:09:45.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.193764+0000", 2024-09-16T00:09:45.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011879410000000001 2024-09-16T00:09:45.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.193830+0000", 2024-09-16T00:09:45.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5903999999999998e-05 2024-09-16T00:09:45.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.229 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.229 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.193881+0000", 2024-09-16T00:09:45.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1107999999999999e-05 2024-09-16T00:09:45.230 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.230 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.230 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.230 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10941 3.1 3:8246fad2:::10000003173.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:45.231 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:09:45.231 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.726350547, 2024-09-16T00:09:45.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01286782, 2024-09-16T00:09:45.231 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.231 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.232 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.232 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.232 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.232 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.232 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10941 2024-09-16T00:09:45.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.233 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:09:45.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:09:45.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.235 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455881+0000", 2024-09-16T00:09:45.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7269999999999999e-06 2024-09-16T00:09:45.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.236 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457462+0000", 2024-09-16T00:09:45.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015811690000000001 2024-09-16T00:09:45.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457487+0000", 2024-09-16T00:09:45.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4607e-05 2024-09-16T00:09:45.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457497+0000", 2024-09-16T00:09:45.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9939999999999995e-06 2024-09-16T00:09:45.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457571+0000", 2024-09-16T00:09:45.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3867000000000003e-05 2024-09-16T00:09:45.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457689+0000", 2024-09-16T00:09:45.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000118814 2024-09-16T00:09:45.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:45.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457804+0000", 2024-09-16T00:09:45.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114702 2024-09-16T00:09:45.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.463422+0000", 2024-09-16T00:09:45.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0056179239999999998 2024-09-16T00:09:45.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468641+0000", 2024-09-16T00:09:45.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052192109999999996 2024-09-16T00:09:45.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.242 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.242 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468703+0000", 2024-09-16T00:09:45.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1268999999999999e-05 2024-09-16T00:09:45.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.242 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468746+0000", 2024-09-16T00:09:45.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:09:45.243 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.243 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.243 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.244 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11068 3.2b 3:d738455c:::100000030db.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:45.244 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.896637+0000", 2024-09-16T00:09:45.244 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.2855916189999999, 2024-09-16T00:09:45.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010717340000000001, 2024-09-16T00:09:45.244 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.244 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.245 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.245 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.245 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.245 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.245 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11068 2024-09-16T00:09:45.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.245 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.896637+0000", 2024-09-16T00:09:45.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.896637+0000", 2024-09-16T00:09:45.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.896640+0000", 2024-09-16T00:09:45.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7250000000000002e-06 2024-09-16T00:09:45.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.897830+0000", 2024-09-16T00:09:45.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001190088 2024-09-16T00:09:45.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.897857+0000", 2024-09-16T00:09:45.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6991e-05 2024-09-16T00:09:45.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.897869+0000", 2024-09-16T00:09:45.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2336e-05 2024-09-16T00:09:45.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.897936+0000", 2024-09-16T00:09:45.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6278999999999993e-05 2024-09-16T00:09:45.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.898018+0000", 2024-09-16T00:09:45.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2013000000000002e-05 2024-09-16T00:09:45.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:45.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.898164+0000", 2024-09-16T00:09:45.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000146417 2024-09-16T00:09:45.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.903866+0000", 2024-09-16T00:09:45.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057015750000000004 2024-09-16T00:09:45.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.907250+0000", 2024-09-16T00:09:45.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033840929999999999 2024-09-16T00:09:45.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.907303+0000", 2024-09-16T00:09:45.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3476000000000001e-05 2024-09-16T00:09:45.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.907355+0000", 2024-09-16T00:09:45.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1347000000000002e-05 2024-09-16T00:09:45.253 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.253 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.253 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.254 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11085 3.2b 3:d738455c:::100000030db.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:45.254 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.975328+0000", 2024-09-16T00:09:45.254 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.2069011740000004, 2024-09-16T00:09:45.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011239193999999999, 2024-09-16T00:09:45.254 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.254 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.255 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.255 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.255 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.255 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.255 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11085 2024-09-16T00:09:45.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.255 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.975328+0000", 2024-09-16T00:09:45.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.975328+0000", 2024-09-16T00:09:45.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.975330+0000", 2024-09-16T00:09:45.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9010000000000001e-06 2024-09-16T00:09:45.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.976617+0000", 2024-09-16T00:09:45.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001287531 2024-09-16T00:09:45.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.976634+0000", 2024-09-16T00:09:45.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7153999999999999e-05 2024-09-16T00:09:45.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.976643+0000", 2024-09-16T00:09:45.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7630000000000006e-06 2024-09-16T00:09:45.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.976702+0000", 2024-09-16T00:09:45.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8662000000000002e-05 2024-09-16T00:09:45.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.976794+0000", 2024-09-16T00:09:45.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2244999999999995e-05 2024-09-16T00:09:45.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:45.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.976948+0000", 2024-09-16T00:09:45.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015442599999999999 2024-09-16T00:09:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.981794+0000", 2024-09-16T00:09:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0048452679999999998 2024-09-16T00:09:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.986502+0000", 2024-09-16T00:09:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047087609999999997 2024-09-16T00:09:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.986532+0000", 2024-09-16T00:09:45.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9635999999999999e-05 2024-09-16T00:09:45.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.986567+0000", 2024-09-16T00:09:45.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4847e-05 2024-09-16T00:09:45.260 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.260 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.260 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.260 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11089 3.1 3:823e197f:::10000003110.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:45.260 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.995768+0000", 2024-09-16T00:09:45.261 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.1864608270000003, 2024-09-16T00:09:45.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011209723, 2024-09-16T00:09:45.261 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.261 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.261 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.261 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.261 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.261 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.261 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11089 2024-09-16T00:09:45.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.261 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.995768+0000", 2024-09-16T00:09:45.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.995768+0000", 2024-09-16T00:09:45.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.995770+0000", 2024-09-16T00:09:45.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.725e-06 2024-09-16T00:09:45.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.997100+0000", 2024-09-16T00:09:45.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013299410000000001 2024-09-16T00:09:45.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.997122+0000", 2024-09-16T00:09:45.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2140000000000001e-05 2024-09-16T00:09:45.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.997132+0000", 2024-09-16T00:09:45.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0495e-05 2024-09-16T00:09:45.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.997209+0000", 2024-09-16T00:09:45.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6786000000000006e-05 2024-09-16T00:09:45.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.997284+0000", 2024-09-16T00:09:45.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5086999999999994e-05 2024-09-16T00:09:45.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:45.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.997399+0000", 2024-09-16T00:09:45.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114883 2024-09-16T00:09:45.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.005091+0000", 2024-09-16T00:09:45.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0076915250000000003 2024-09-16T00:09:45.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.006850+0000", 2024-09-16T00:09:45.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017591919999999999 2024-09-16T00:09:45.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.006919+0000", 2024-09-16T00:09:45.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9368000000000004e-05 2024-09-16T00:09:45.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.006978+0000", 2024-09-16T00:09:45.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8581e-05 2024-09-16T00:09:45.267 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.267 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.267 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.267 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11739 3.17 3:e9f3a865:::10000003294.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:45.267 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.052491+0000", 2024-09-16T00:09:45.268 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 2.1297379680000001, 2024-09-16T00:09:45.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011149121, 2024-09-16T00:09:45.268 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.268 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.268 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.268 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.268 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.268 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.268 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11739 2024-09-16T00:09:45.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.268 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.052491+0000", 2024-09-16T00:09:45.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.052491+0000", 2024-09-16T00:09:45.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.052494+0000", 2024-09-16T00:09:45.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0750000000000002e-06 2024-09-16T00:09:45.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.053984+0000", 2024-09-16T00:09:45.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001490395 2024-09-16T00:09:45.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.053999+0000", 2024-09-16T00:09:45.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4606e-05 2024-09-16T00:09:45.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.054007+0000", 2024-09-16T00:09:45.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4999999999999999e-06 2024-09-16T00:09:45.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.054072+0000", 2024-09-16T00:09:45.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4974999999999994e-05 2024-09-16T00:09:45.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.054230+0000", 2024-09-16T00:09:45.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015757600000000001 2024-09-16T00:09:45.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:45.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.054392+0000", 2024-09-16T00:09:45.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016178199999999999 2024-09-16T00:09:45.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.062316+0000", 2024-09-16T00:09:45.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0079242099999999992 2024-09-16T00:09:45.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.063541+0000", 2024-09-16T00:09:45.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001225114 2024-09-16T00:09:45.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.063598+0000", 2024-09-16T00:09:45.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6511000000000002e-05 2024-09-16T00:09:45.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.063640+0000", 2024-09-16T00:09:45.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2376999999999998e-05 2024-09-16T00:09:45.274 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.274 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.274 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.274 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11742 3.1b 3:d847de26:::100000032cd.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:45.274 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:09:45.274 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 2.1258589620000001, 2024-09-16T00:09:45.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012791903, 2024-09-16T00:09:45.275 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.275 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.275 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.275 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.275 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.275 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.275 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11742 2024-09-16T00:09:45.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.275 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:09:45.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.276 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.276 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:09:45.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.276 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.276 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056373+0000", 2024-09-16T00:09:45.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9799999999999998e-06 2024-09-16T00:09:45.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058604+0000", 2024-09-16T00:09:45.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022316020000000001 2024-09-16T00:09:45.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058630+0000", 2024-09-16T00:09:45.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5425000000000001e-05 2024-09-16T00:09:45.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058643+0000", 2024-09-16T00:09:45.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3361e-05 2024-09-16T00:09:45.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058731+0000", 2024-09-16T00:09:45.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7270999999999998e-05 2024-09-16T00:09:45.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058894+0000", 2024-09-16T00:09:45.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016376800000000001 2024-09-16T00:09:45.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:45.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.059050+0000", 2024-09-16T00:09:45.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015603700000000001 2024-09-16T00:09:45.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.066364+0000", 2024-09-16T00:09:45.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073138639999999998 2024-09-16T00:09:45.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069025+0000", 2024-09-16T00:09:45.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026607660000000002 2024-09-16T00:09:45.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069083+0000", 2024-09-16T00:09:45.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7680999999999999e-05 2024-09-16T00:09:45.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069162+0000", 2024-09-16T00:09:45.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9147999999999995e-05 2024-09-16T00:09:45.281 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.281 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.281 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.281 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11780 3.23 3:c752d055:::100000031d2.00000000:head [write 0~4194304 [2@2823004]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:45.281 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.200354+0000", 2024-09-16T00:09:45.281 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.981874744, 2024-09-16T00:09:45.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011318407000000001, 2024-09-16T00:09:45.282 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.282 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.282 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.282 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.282 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.282 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.283 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11780 2024-09-16T00:09:45.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.283 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.200354+0000", 2024-09-16T00:09:45.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.200354+0000", 2024-09-16T00:09:45.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.200356+0000", 2024-09-16T00:09:45.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.057e-06 2024-09-16T00:09:45.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.202142+0000", 2024-09-16T00:09:45.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017862939999999999 2024-09-16T00:09:45.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.202166+0000", 2024-09-16T00:09:45.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3339000000000001e-05 2024-09-16T00:09:45.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.202176+0000", 2024-09-16T00:09:45.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0658e-05 2024-09-16T00:09:45.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.202253+0000", 2024-09-16T00:09:45.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6743999999999996e-05 2024-09-16T00:09:45.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.202363+0000", 2024-09-16T00:09:45.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010973500000000001 2024-09-16T00:09:45.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:45.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.202524+0000", 2024-09-16T00:09:45.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000160963 2024-09-16T00:09:45.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.210136+0000", 2024-09-16T00:09:45.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0076117390000000002 2024-09-16T00:09:45.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.288 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.211557+0000", 2024-09-16T00:09:45.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014213520000000001 2024-09-16T00:09:45.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.211624+0000", 2024-09-16T00:09:45.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7119999999999994e-05 2024-09-16T00:09:45.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.211673+0000", 2024-09-16T00:09:45.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8405999999999997e-05 2024-09-16T00:09:45.290 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.290 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.290 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.291 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11837 3.1b 3:d94fa8da:::10000003190.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:45.291 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.465632+0000", 2024-09-16T00:09:45.291 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.716597114, 2024-09-16T00:09:45.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011434082, 2024-09-16T00:09:45.292 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.292 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.292 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.292 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.292 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.293 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.293 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11837 2024-09-16T00:09:45.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.293 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.465632+0000", 2024-09-16T00:09:45.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.465632+0000", 2024-09-16T00:09:45.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.465634+0000", 2024-09-16T00:09:45.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9180000000000001e-06 2024-09-16T00:09:45.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.467891+0000", 2024-09-16T00:09:45.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022575049999999999 2024-09-16T00:09:45.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.467919+0000", 2024-09-16T00:09:45.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8119999999999998e-05 2024-09-16T00:09:45.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.467933+0000", 2024-09-16T00:09:45.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3968000000000001e-05 2024-09-16T00:09:45.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.468012+0000", 2024-09-16T00:09:45.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8874000000000004e-05 2024-09-16T00:09:45.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.300 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.300 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.468174+0000", 2024-09-16T00:09:45.300 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000161972 2024-09-16T00:09:45.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:45.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.468319+0000", 2024-09-16T00:09:45.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000145117 2024-09-16T00:09:45.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.472853+0000", 2024-09-16T00:09:45.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045335490000000004 2024-09-16T00:09:45.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.303 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.476990+0000", 2024-09-16T00:09:45.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041376629999999998 2024-09-16T00:09:45.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.304 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.477025+0000", 2024-09-16T00:09:45.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4975999999999999e-05 2024-09-16T00:09:45.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.304 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.304 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.477066+0000", 2024-09-16T00:09:45.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0420000000000003e-05 2024-09-16T00:09:45.305 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.305 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.305 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.306 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11934 3.1b 3:d94fa8da:::10000003190.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:45.306 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:09:45.306 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.4938156870000001, 2024-09-16T00:09:45.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.017110325999999999, 2024-09-16T00:09:45.307 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.307 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.307 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.307 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.307 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.308 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.308 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11934 2024-09-16T00:09:45.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.308 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:09:45.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:09:45.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688416+0000", 2024-09-16T00:09:45.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.875e-06 2024-09-16T00:09:45.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689633+0000", 2024-09-16T00:09:45.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001216997 2024-09-16T00:09:45.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689655+0000", 2024-09-16T00:09:45.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2285000000000001e-05 2024-09-16T00:09:45.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689665+0000", 2024-09-16T00:09:45.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7089999999999997e-06 2024-09-16T00:09:45.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689739+0000", 2024-09-16T00:09:45.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3973999999999993e-05 2024-09-16T00:09:45.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689898+0000", 2024-09-16T00:09:45.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015941100000000001 2024-09-16T00:09:45.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:45.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.690062+0000", 2024-09-16T00:09:45.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000163633 2024-09-16T00:09:45.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.695224+0000", 2024-09-16T00:09:45.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051620210000000001 2024-09-16T00:09:45.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705434+0000", 2024-09-16T00:09:45.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010209701 2024-09-16T00:09:45.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705488+0000", 2024-09-16T00:09:45.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3974000000000002e-05 2024-09-16T00:09:45.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705523+0000", 2024-09-16T00:09:45.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5746e-05 2024-09-16T00:09:45.313 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.314 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.314 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.314 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11941 3.12 3:4a0e6682:::1000000334b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:45.314 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:09:45.314 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.458798174, 2024-09-16T00:09:45.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01458448, 2024-09-16T00:09:45.314 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.314 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.314 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.314 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.314 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.314 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.315 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11941 2024-09-16T00:09:45.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.315 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:09:45.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:09:45.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723433+0000", 2024-09-16T00:09:45.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3410000000000001e-06 2024-09-16T00:09:45.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724966+0000", 2024-09-16T00:09:45.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001533242 2024-09-16T00:09:45.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724983+0000", 2024-09-16T00:09:45.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7156999999999999e-05 2024-09-16T00:09:45.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724995+0000", 2024-09-16T00:09:45.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2089999999999999e-05 2024-09-16T00:09:45.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725062+0000", 2024-09-16T00:09:45.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6026999999999997e-05 2024-09-16T00:09:45.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725159+0000", 2024-09-16T00:09:45.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7374000000000007e-05 2024-09-16T00:09:45.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725289+0000", 2024-09-16T00:09:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012988 2024-09-16T00:09:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.731944+0000", 2024-09-16T00:09:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066547439999999998 2024-09-16T00:09:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.737956+0000", 2024-09-16T00:09:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0060121630000000001 2024-09-16T00:09:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.737985+0000", 2024-09-16T00:09:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9352999999999998e-05 2024-09-16T00:09:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.738015+0000", 2024-09-16T00:09:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0108999999999999e-05 2024-09-16T00:09:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11954 3.6 3:625e0e90:::100000032ef.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:09:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.4143466069999999, 2024-09-16T00:09:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012484808999999999, 2024-09-16T00:09:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11954 2024-09-16T00:09:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:09:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:09:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767886+0000", 2024-09-16T00:09:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6650000000000001e-06 2024-09-16T00:09:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770082+0000", 2024-09-16T00:09:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021964979999999999 2024-09-16T00:09:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770104+0000", 2024-09-16T00:09:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1671999999999999e-05 2024-09-16T00:09:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770114+0000", 2024-09-16T00:09:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0336000000000001e-05 2024-09-16T00:09:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770197+0000", 2024-09-16T00:09:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2997000000000005e-05 2024-09-16T00:09:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770324+0000", 2024-09-16T00:09:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012680100000000001 2024-09-16T00:09:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770497+0000", 2024-09-16T00:09:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000172776 2024-09-16T00:09:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.776330+0000", 2024-09-16T00:09:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058330939999999996 2024-09-16T00:09:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780252+0000", 2024-09-16T00:09:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039215689999999997 2024-09-16T00:09:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780317+0000", 2024-09-16T00:09:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4948000000000002e-05 2024-09-16T00:09:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780367+0000", 2024-09-16T00:09:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0453e-05 2024-09-16T00:09:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12154 3.2b 3:d78f40a6:::100000033bb.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.433567+0000", 2024-09-16T00:09:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.74866138100000001, 2024-09-16T00:09:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011046830000000001, 2024-09-16T00:09:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12154 2024-09-16T00:09:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.433567+0000", 2024-09-16T00:09:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.433567+0000", 2024-09-16T00:09:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.433570+0000", 2024-09-16T00:09:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4650000000000001e-06 2024-09-16T00:09:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.434882+0000", 2024-09-16T00:09:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013122380000000001 2024-09-16T00:09:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.434908+0000", 2024-09-16T00:09:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5616999999999999e-05 2024-09-16T00:09:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.434922+0000", 2024-09-16T00:09:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3753e-05 2024-09-16T00:09:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.435003+0000", 2024-09-16T00:09:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1217000000000005e-05 2024-09-16T00:09:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.435094+0000", 2024-09-16T00:09:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1090999999999999e-05 2024-09-16T00:09:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.435243+0000", 2024-09-16T00:09:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014871599999999999 2024-09-16T00:09:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.441369+0000", 2024-09-16T00:09:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061262790000000001 2024-09-16T00:09:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.444505+0000", 2024-09-16T00:09:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031357049999999999 2024-09-16T00:09:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.444566+0000", 2024-09-16T00:09:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1826000000000001e-05 2024-09-16T00:09:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.444614+0000", 2024-09-16T00:09:45.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7923000000000002e-05 2024-09-16T00:09:45.333 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.333 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.333 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.333 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12198 3.12 3:4a0e6682:::1000000334b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:45.333 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.683089+0000", 2024-09-16T00:09:45.334 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.49913955700000001, 2024-09-16T00:09:45.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011990244000000001, 2024-09-16T00:09:45.334 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.334 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.334 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.334 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.334 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.334 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.334 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12198 2024-09-16T00:09:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.683089+0000", 2024-09-16T00:09:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.683089+0000", 2024-09-16T00:09:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.683092+0000", 2024-09-16T00:09:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.931e-06 2024-09-16T00:09:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684715+0000", 2024-09-16T00:09:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00162269 2024-09-16T00:09:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684734+0000", 2024-09-16T00:09:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9545999999999999e-05 2024-09-16T00:09:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684745+0000", 2024-09-16T00:09:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0131e-05 2024-09-16T00:09:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684814+0000", 2024-09-16T00:09:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9740000000000007e-05 2024-09-16T00:09:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684871+0000", 2024-09-16T00:09:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6215999999999998e-05 2024-09-16T00:09:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684980+0000", 2024-09-16T00:09:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109713 2024-09-16T00:09:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.690215+0000", 2024-09-16T00:09:45.341 INFO:tasks.workunit.client.0.smithi139.stdout:9/557: read - d19/d1a/d20/d23/d2d/d41/d71/d32/fb4 zero size 2024-09-16T00:09:45.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052349049999999998 2024-09-16T00:09:45.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.694970+0000", 2024-09-16T00:09:45.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047552649999999998 2024-09-16T00:09:45.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.695030+0000", 2024-09-16T00:09:45.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9086999999999999e-05 2024-09-16T00:09:45.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.695080+0000", 2024-09-16T00:09:45.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0019999999999999e-05 2024-09-16T00:09:45.343 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.343 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.344 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.344 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.344 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12348 3.1b 3:db04998a:::100000032d0.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:45.344 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.059532+0000", 2024-09-16T00:09:45.344 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.122697229, 2024-09-16T00:09:45.344 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011465410000000001, 2024-09-16T00:09:45.344 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:45.345 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:45.345 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:45.345 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:45.345 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:45.345 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:45.345 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12348 2024-09-16T00:09:45.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.345 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.059532+0000", 2024-09-16T00:09:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.059532+0000", 2024-09-16T00:09:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:45.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:45.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.059534+0000", 2024-09-16T00:09:45.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8710000000000001e-06 2024-09-16T00:09:45.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:45.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.060900+0000", 2024-09-16T00:09:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013656219999999999 2024-09-16T00:09:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.060919+0000", 2024-09-16T00:09:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.878e-05 2024-09-16T00:09:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.060930+0000", 2024-09-16T00:09:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1204e-05 2024-09-16T00:09:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.060993+0000", 2024-09-16T00:09:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2899999999999997e-05 2024-09-16T00:09:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.061063+0000", 2024-09-16T00:09:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9737e-05 2024-09-16T00:09:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.061165+0000", 2024-09-16T00:09:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010232499999999999 2024-09-16T00:09:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.066355+0000", 2024-09-16T00:09:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051903139999999997 2024-09-16T00:09:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.070925+0000", 2024-09-16T00:09:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045696510000000001 2024-09-16T00:09:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.070964+0000", 2024-09-16T00:09:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9026000000000002e-05 2024-09-16T00:09:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.070997+0000", 2024-09-16T00:09:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2979999999999999e-05 2024-09-16T00:09:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.352 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:45.352 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:45.352 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:45.353 INFO:tasks.workunit.client.0.smithi139.stdout:9/558: chown d19/d1a/d20/d23/d2d/d41/d71/d32/d79/f5d 13 1 2024-09-16T00:09:45.353 INFO:tasks.workunit.client.0.smithi139.stdout:9/559: write d19/d1a/d20/d23/d2d/d35/d42/f34 [74870,95905] 0 2024-09-16T00:09:45.353 INFO:tasks.workunit.client.0.smithi139.stdout:5/513: dread d1/d13/d3f/d9f/f36 [0,4194304] 0 2024-09-16T00:09:45.353 INFO:tasks.workunit.client.0.smithi139.stdout:9/560: rmdir d19/d1a/d20/d23/d2d/d35/d42/d3a 39 2024-09-16T00:09:45.353 INFO:tasks.workunit.client.0.smithi139.stdout:5/514: rmdir d1/d13/d3f/d9f/d73/d93 39 2024-09-16T00:09:45.353 INFO:tasks.workunit.client.0.smithi139.stdout:5/515: dread - d1/d21/d49/d87/f91 zero size 2024-09-16T00:09:45.354 INFO:tasks.workunit.client.0.smithi139.stdout:5/516: write d1/d13/d3f/d9f/d73/f8c [105303,117990] 0 2024-09-16T00:09:45.354 INFO:tasks.workunit.client.0.smithi139.stdout:9/561: dread d19/d1a/f39 [0,4194304] 0 2024-09-16T00:09:45.354 INFO:tasks.workunit.client.0.smithi139.stdout:9/562: chown d19/d86 55 1 2024-09-16T00:09:45.354 INFO:tasks.workunit.client.0.smithi139.stdout:5/517: creat d1/fad x:0 0 0 2024-09-16T00:09:45.354 INFO:tasks.workunit.client.0.smithi139.stdout:9/563: write d19/d1a/d20/d23/d2d/f90 [3218726,102794] 0 2024-09-16T00:09:45.355 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:09:45.356 INFO:tasks.workunit.client.0.smithi139.stdout:5/518: read d1/d13/d3f/d9f/f2e [4463051,60202] 0 2024-09-16T00:09:45.356 INFO:tasks.workunit.client.0.smithi139.stdout:9/564: chown d19/d1a/d20/d23/d2d/d41/d71/d32/d79/f5d 0 1 2024-09-16T00:09:45.357 INFO:tasks.workunit.client.0.smithi139.stdout:9/565: read d19/d1a/d20/d23/d2d/d41/d71/d32/d79/f5d [1523878,48445] 0 2024-09-16T00:09:45.357 INFO:tasks.workunit.client.0.smithi139.stdout:9/566: chown d19/d1a/d20/d23/d2d/d41/d71/d32/fb4 39803 1 2024-09-16T00:09:45.357 INFO:tasks.workunit.client.0.smithi139.stdout:9/567: stat d19/d1a/faf 0 2024-09-16T00:09:45.357 INFO:tasks.workunit.client.0.smithi139.stdout:5/519: creat d1/d21/d49/d53/d6a/d7b/d99/fae x:0 0 0 2024-09-16T00:09:45.357 INFO:tasks.workunit.client.0.smithi139.stdout:5/520: rename d1/lc to d1/d13/d3f/laf 0 2024-09-16T00:09:45.357 INFO:tasks.workunit.client.0.smithi139.stdout:9/568: dwrite fb [0,4194304] 0 2024-09-16T00:09:45.357 INFO:tasks.workunit.client.0.smithi139.stdout:5/521: fsync d1/d21/d49/d53/d6a/f9b 0 2024-09-16T00:09:45.357 INFO:tasks.workunit.client.0.smithi139.stdout:9/569: fsync d19/d1a/d20/d89/f9d 0 2024-09-16T00:09:45.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.216+0000 7fdc4cff9640 1 -- 172.21.15.139:0/2943811187 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v88) ==== 97+0+0 (secure 0 0 0) 0x7fdc38079650 con 0x7fdc500c7fb0 2024-09-16T00:09:45.358 INFO:tasks.workunit.client.0.smithi139.stdout:9/570: symlink d19/d1a/d20/d23/d2d/d41/d71/lbf 0 2024-09-16T00:09:45.358 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is set 2024-09-16T00:09:45.358 INFO:tasks.workunit.client.0.smithi139.stdout:5/522: dwrite d1/d21/d49/f81 [0,4194304] 0 2024-09-16T00:09:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.219+0000 7fdc55f8d640 1 -- 172.21.15.139:0/2943811187 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdc3006c2b0 msgr2=0x7fdc3006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.219+0000 7fdc55f8d640 1 --2- 172.21.15.139:0/2943811187 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdc3006c2b0 0x7fdc3006e770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009aa0 tx=0x7fdc40009190 comp rx=0 tx=0).stop 2024-09-16T00:09:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.219+0000 7fdc55f8d640 1 -- 172.21.15.139:0/2943811187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc500c7fb0 msgr2=0x7fdc500ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.219+0000 7fdc55f8d640 1 --2- 172.21.15.139:0/2943811187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc500c7fb0 0x7fdc500ccdc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdc380098c0 tx=0x7fdc380046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.219+0000 7fdc55f8d640 1 -- 172.21.15.139:0/2943811187 shutdown_connections 2024-09-16T00:09:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.219+0000 7fdc55f8d640 1 --2- 172.21.15.139:0/2943811187 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdc3006c2b0 0x7fdc3006e770 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.219+0000 7fdc55f8d640 1 --2- 172.21.15.139:0/2943811187 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc50154bc0 0x7fdc50158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:45.359 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.219+0000 7fdc55f8d640 1 --2- 172.21.15.139:0/2943811187 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdc500c7fb0 0x7fdc500ccdc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:45.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.219+0000 7fdc55f8d640 1 --2- 172.21.15.139:0/2943811187 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdc500c79c0 0x7fdc50148be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:45.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.219+0000 7fdc55f8d640 1 -- 172.21.15.139:0/2943811187 >> 172.21.15.139:0/2943811187 conn(0x7fdc50070460 msgr2=0x7fdc5014def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:45.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.219+0000 7fdc55f8d640 1 -- 172.21.15.139:0/2943811187 shutdown_connections 2024-09-16T00:09:45.360 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.219+0000 7fdc55f8d640 1 -- 172.21.15.139:0/2943811187 wait complete. 2024-09-16T00:09:45.361 INFO:tasks.workunit.client.0.smithi139.stdout:9/571: mkdir d19/d1a/d20/d23/d2d/d41/d71/da3/dc0 0 2024-09-16T00:09:45.361 INFO:tasks.workunit.client.0.smithi139.stdout:5/523: write d1/d13/f17 [4569483,29352] 0 2024-09-16T00:09:45.361 INFO:tasks.workunit.client.0.smithi139.stdout:9/572: chown d19/d1a/f8f 666 1 2024-09-16T00:09:45.361 INFO:tasks.workunit.client.0.smithi139.stdout:5/524: write d1/f27 [679341,9296] 0 2024-09-16T00:09:45.361 INFO:tasks.workunit.client.0.smithi139.stdout:5/525: chown d1/d21/d49/d53/d6a 21314797 1 2024-09-16T00:09:45.361 INFO:tasks.workunit.client.0.smithi139.stdout:9/573: symlink d19/d1a/d20/d23/d2d/d41/d71/d32/d4e/lc1 0 2024-09-16T00:09:45.361 INFO:tasks.workunit.client.0.smithi139.stdout:5/526: mkdir d1/d21/d49/db0 0 2024-09-16T00:09:45.361 INFO:tasks.workunit.client.0.smithi139.stdout:5/527: fsync d1/d21/d49/d53/d6a/fa7 0 2024-09-16T00:09:45.361 INFO:tasks.workunit.client.0.smithi139.stdout:5/528: readlink d1/d21/d49/d53/d6a/l89 0 2024-09-16T00:09:45.361 INFO:tasks.workunit.client.0.smithi139.stdout:9/574: dread d19/d1a/d20/d23/d2d/d2f/f96 [0,4194304] 0 2024-09-16T00:09:45.365 INFO:tasks.workunit.client.0.smithi139.stdout:9/575: symlink d19/d1a/d20/d23/d2d/d2f/lc2 0 2024-09-16T00:09:45.365 INFO:tasks.workunit.client.0.smithi139.stdout:5/529: dwrite d1/d21/d49/d53/d6a/d7b/d78/f8e [0,4194304] 0 2024-09-16T00:09:45.366 INFO:tasks.workunit.client.0.smithi139.stdout:5/530: write d1/d21/d49/d53/d6a/d7b/d78/f8e [2274362,10371] 0 2024-09-16T00:09:45.366 INFO:tasks.workunit.client.0.smithi139.stdout:9/576: dwrite d19/f26 [0,4194304] 0 2024-09-16T00:09:45.366 INFO:tasks.workunit.client.0.smithi139.stdout:5/531: dwrite d1/d13/f97 [0,4194304] 0 2024-09-16T00:09:45.366 INFO:tasks.workunit.client.0.smithi139.stdout:9/577: dwrite d19/d1a/d20/d23/f3e [0,4194304] 0 2024-09-16T00:09:45.366 INFO:tasks.workunit.client.0.smithi139.stdout:5/532: write d1/d13/d3b/d3e/f69 [2559197,58900] 0 2024-09-16T00:09:45.366 INFO:tasks.workunit.client.0.smithi139.stdout:9/578: mknod d19/d1a/d20/d23/d2d/d41/d9f/cc3 0 2024-09-16T00:09:45.366 INFO:tasks.workunit.client.0.smithi139.stdout:5/533: creat d1/d13/d3f/d9f/d73/d79/d8d/fb1 x:0 0 0 2024-09-16T00:09:45.366 INFO:tasks.workunit.client.0.smithi139.stdout:6/304: sync 2024-09-16T00:09:45.366 INFO:tasks.workunit.client.0.smithi139.stdout:9/579: symlink d19/d1a/d20/d23/d2d/d41/d71/da3/dc0/lc4 0 2024-09-16T00:09:45.366 INFO:tasks.workunit.client.0.smithi139.stdout:6/305: mkdir d8/d20/d68 0 2024-09-16T00:09:45.366 INFO:tasks.workunit.client.0.smithi139.stdout:6/306: chown d8/d29 5444440 1 2024-09-16T00:09:45.366 INFO:tasks.workunit.client.0.smithi139.stdout:5/534: unlink d1/d13/d3f/d9f/d73/d79/d8d/fb1 0 2024-09-16T00:09:45.366 INFO:tasks.workunit.client.0.smithi139.stdout:9/580: mknod d19/d1a/d20/d23/d2d/d41/d9f/cc5 0 2024-09-16T00:09:45.366 INFO:tasks.workunit.client.0.smithi139.stdout:5/535: truncate d1/d21/d49/d53/d6a/f8b 86174 0 2024-09-16T00:09:45.366 INFO:tasks.workunit.client.0.smithi139.stdout:5/536: unlink d1/d13/d3b/f7d 0 2024-09-16T00:09:45.367 INFO:tasks.workunit.client.0.smithi139.stdout:6/307: link d8/db/d26/d28/c2f d8/d20/d68/c69 0 2024-09-16T00:09:45.367 INFO:tasks.workunit.client.0.smithi139.stdout:6/308: mknod d8/db/d26/d45/d42/c6a 0 2024-09-16T00:09:45.367 INFO:tasks.workunit.client.0.smithi139.stdout:5/537: mkdir d1/d21/d49/db2 0 2024-09-16T00:09:45.367 INFO:tasks.workunit.client.0.smithi139.stdout:6/309: fsync d8/d20/d4d/f60 0 2024-09-16T00:09:45.367 INFO:tasks.workunit.client.0.smithi139.stdout:6/310: stat f3 0 2024-09-16T00:09:45.367 INFO:tasks.workunit.client.0.smithi139.stdout:6/311: dread d8/d20/f62 [0,4194304] 0 2024-09-16T00:09:45.367 INFO:tasks.workunit.client.0.smithi139.stdout:5/538: dwrite d1/d13/d3f/d9f/d30/d33/f50 [0,4194304] 0 2024-09-16T00:09:45.367 INFO:tasks.workunit.client.0.smithi139.stdout:5/539: truncate d1/d21/d49/d53/d6a/f8b 593661 0 2024-09-16T00:09:45.367 INFO:tasks.workunit.client.0.smithi139.stdout:9/581: truncate d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/f55 634994 0 2024-09-16T00:09:45.367 INFO:tasks.workunit.client.0.smithi139.stdout:5/540: fdatasync d1/d13/f24 0 2024-09-16T00:09:45.367 INFO:tasks.workunit.client.0.smithi139.stdout:6/312: creat d8/db/d26/d45/d59/f6b x:0 0 0 2024-09-16T00:09:45.367 INFO:tasks.workunit.client.0.smithi139.stdout:9/582: write d19/d1a/f50 [3625501,90318] 0 2024-09-16T00:09:45.367 INFO:tasks.workunit.client.0.smithi139.stdout:6/313: chown d8/cf 11171 1 2024-09-16T00:09:45.367 INFO:tasks.workunit.client.0.smithi139.stdout:6/314: stat c2 0 2024-09-16T00:09:45.368 INFO:tasks.workunit.client.0.smithi139.stdout:5/541: mknod d1/cb3 0 2024-09-16T00:09:45.368 INFO:tasks.workunit.client.0.smithi139.stdout:9/583: symlink d19/d86/lc6 0 2024-09-16T00:09:45.368 INFO:tasks.workunit.client.0.smithi139.stdout:6/315: write d8/db/d26/d45/f43 [345725,129678] 0 2024-09-16T00:09:45.368 INFO:tasks.workunit.client.0.smithi139.stdout:9/584: chown d19/d1a/d20/d23/c82 208010 1 2024-09-16T00:09:45.368 INFO:tasks.workunit.client.0.smithi139.stdout:5/542: symlink d1/d21/d49/d53/d6a/d7b/d99/lb4 0 2024-09-16T00:09:45.368 INFO:tasks.workunit.client.0.smithi139.stdout:9/585: dread d19/d1a/d20/d23/d2d/d41/d71/d32/f76 [0,4194304] 0 2024-09-16T00:09:45.368 INFO:tasks.workunit.client.0.smithi139.stdout:6/316: dwrite f3 [4194304,4194304] 0 2024-09-16T00:09:45.368 INFO:tasks.workunit.client.0.smithi139.stdout:5/543: dread d1/d13/d3f/d5e/f61 [0,4194304] 0 2024-09-16T00:09:45.368 INFO:tasks.workunit.client.0.smithi139.stdout:9/586: chown d19/d1a/d20/d23/d2d/d35/d42/d3a/d45 76 1 2024-09-16T00:09:45.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.294+0000 7f933d477640 1 Processor -- start 2024-09-16T00:09:45.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.294+0000 7f933d477640 1 -- start start 2024-09-16T00:09:45.369 INFO:tasks.workunit.client.0.smithi139.stdout:5/544: symlink d1/d13/d3f/lb5 0 2024-09-16T00:09:45.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.295+0000 7f933d477640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9338152970 0x7f9338158f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:45.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.295+0000 7f933d477640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f93380c64d0 0x7f9338162e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:45.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.295+0000 7f933d477640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93380c6a30 con 0x7f93381599f0 2024-09-16T00:09:45.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.295+0000 7f933d477640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9338068040 con 0x7f93380c64d0 2024-09-16T00:09:45.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.295+0000 7f933d477640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f9338067a80 con 0x7f9338152970 2024-09-16T00:09:45.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.295+0000 7f93377fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f93380c64d0 0x7f9338162e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:45.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.295+0000 7f93377fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f93380c64d0 0x7f9338162e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43816/0 (socket says 172.21.15.139:43816) 2024-09-16T00:09:45.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.295+0000 7f93377fe640 1 -- 172.21.15.139:0/3177290053 learned_addr learned my addr 172.21.15.139:0/3177290053 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:45.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.295+0000 7f9336ffd640 1 --2- 172.21.15.139:0/3177290053 >> v2:172.21.15.164:3300/0 conn(0x7f9338152970 0x7f9338158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:45.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.295+0000 7f93377fe640 1 -- 172.21.15.139:0/3177290053 >> v2:172.21.15.164:3300/0 conn(0x7f9338152970 msgr2=0x7f9338158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:45.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.295+0000 7f93377fe640 1 --2- 172.21.15.139:0/3177290053 >> v2:172.21.15.164:3300/0 conn(0x7f9338152970 0x7f9338158f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:45.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.295+0000 7f93377fe640 1 -- 172.21.15.139:0/3177290053 >> v1:172.21.15.164:6789/0 conn(0x7f93381599f0 legacy=0x7f93380c44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:45.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.295+0000 7f93377fe640 1 -- 172.21.15.139:0/3177290053 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9338067d60 con 0x7f93380c64d0 2024-09-16T00:09:45.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.295+0000 7f9336ffd640 1 --2- 172.21.15.139:0/3177290053 >> v2:172.21.15.164:3300/0 conn(0x7f9338152970 0x7f9338158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:09:45.371 INFO:tasks.workunit.client.0.smithi139.stdout:9/587: write d19/d1a/d20/d23/f33 [2178809,22803] 0 2024-09-16T00:09:45.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.296+0000 7f93377fe640 1 --2- 172.21.15.139:0/3177290053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c64d0 0x7f9338162e80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9338075c40 tx=0x7f933002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa02af7f2498cbf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:45.371 INFO:tasks.workunit.client.0.smithi139.stdout:5/545: read d1/d13/f17 [4891345,81132] 0 2024-09-16T00:09:45.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.296+0000 7f93367fc640 1 -- 172.21.15.139:0/3177290053 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330036070 con 0x7f93380c64d0 2024-09-16T00:09:45.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.297+0000 7f93367fc640 1 -- 172.21.15.139:0/3177290053 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9330031600 con 0x7f93380c64d0 2024-09-16T00:09:45.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.297+0000 7f93367fc640 1 -- 172.21.15.139:0/3177290053 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93300319d0 con 0x7f93380c64d0 2024-09-16T00:09:45.372 INFO:tasks.workunit.client.0.smithi139.stdout:6/317: read d8/db/d26/d45/f46 [143072,41529] 0 2024-09-16T00:09:45.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.297+0000 7f933d477640 1 -- 172.21.15.139:0/3177290053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c64d0 msgr2=0x7f9338162e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:45.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.297+0000 7f933d477640 1 --2- 172.21.15.139:0/3177290053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c64d0 0x7f9338162e80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9338075c40 tx=0x7f933002a6f0 comp rx=0 tx=0).stop 2024-09-16T00:09:45.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.297+0000 7f933d477640 1 -- 172.21.15.139:0/3177290053 shutdown_connections 2024-09-16T00:09:45.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.298+0000 7f933d477640 1 --2- 172.21.15.139:0/3177290053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c64d0 0x7f9338162e80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:45.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.298+0000 7f933d477640 1 --2- 172.21.15.139:0/3177290053 >> v2:172.21.15.164:3300/0 conn(0x7f9338152970 0x7f9338158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:45.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.298+0000 7f933d477640 1 -- 172.21.15.139:0/3177290053 >> 172.21.15.139:0/3177290053 conn(0x7f933806f5d0 msgr2=0x7f933806f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:45.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.298+0000 7f933d477640 1 -- 172.21.15.139:0/3177290053 shutdown_connections 2024-09-16T00:09:45.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.298+0000 7f933d477640 1 -- 172.21.15.139:0/3177290053 wait complete. 2024-09-16T00:09:45.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.299+0000 7f933d477640 1 Processor -- start 2024-09-16T00:09:45.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.299+0000 7f933d477640 1 -- start start 2024-09-16T00:09:45.374 INFO:tasks.workunit.client.0.smithi139.stdout:5/546: mknod d1/cb6 0 2024-09-16T00:09:45.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.300+0000 7f933d477640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c64d0 0x7f93381e54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:45.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.300+0000 7f933d477640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9338152970 0x7f9338145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:45.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.300+0000 7f933d477640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93381599f0 0x7f9338145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:45.375 INFO:tasks.workunit.client.0.smithi139.stdout:9/588: mkdir d19/d1a/d20/d23/d2d/d41/d71/dc7 0 2024-09-16T00:09:45.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.300+0000 7f933d477640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9338159450 con 0x7f93380c64d0 2024-09-16T00:09:45.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.300+0000 7f933d477640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f93381589f0 con 0x7f93381599f0 2024-09-16T00:09:45.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.300+0000 7f933d477640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f93380700f0 con 0x7f9338152970 2024-09-16T00:09:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.300+0000 7f9336ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c64d0 0x7f93381e54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.300+0000 7f932e7fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9338152970 0x7f9338145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.300+0000 7f9336ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c64d0 0x7f93381e54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43830/0 (socket says 172.21.15.139:43830) 2024-09-16T00:09:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.300+0000 7f9336ffd640 1 -- 172.21.15.139:0/2715716558 learned_addr learned my addr 172.21.15.139:0/2715716558 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.300+0000 7f93377fe640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93381599f0 0x7f9338145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.301+0000 7f932e7fc640 1 -- 172.21.15.139:0/2715716558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93381599f0 msgr2=0x7f9338145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.301+0000 7f932e7fc640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f93381599f0 0x7f9338145940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.301+0000 7f932e7fc640 1 -- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c64d0 msgr2=0x7f93381e54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.301+0000 7f932e7fc640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c64d0 0x7f93381e54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.301+0000 7f932e7fc640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933814b260 con 0x7f9338152970 2024-09-16T00:09:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.301+0000 7f9336ffd640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93380c64d0 0x7f93381e54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:45.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.301+0000 7f932e7fc640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9338152970 0x7f9338145400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f932000b410 tx=0x7f932000b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:45.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.301+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320015030 con 0x7f9338152970 2024-09-16T00:09:45.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.302+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9320004020 con 0x7f9338152970 2024-09-16T00:09:45.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.302+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320004450 con 0x7f9338152970 2024-09-16T00:09:45.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.302+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f933814b540 con 0x7f9338152970 2024-09-16T00:09:45.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.302+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f933814d880 con 0x7f9338152970 2024-09-16T00:09:45.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.304+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f933814d390 con 0x7f9338152970 2024-09-16T00:09:45.378 INFO:tasks.workunit.client.0.smithi139.stdout:9/589: read d19/d1a/f1e [2197636,68813] 0 2024-09-16T00:09:45.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.304+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9320018070 con 0x7f9338152970 2024-09-16T00:09:45.379 INFO:tasks.workunit.client.0.smithi139.stdout:6/318: dwrite d8/db/d26/d28/f40 [0,4194304] 0 2024-09-16T00:09:45.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.305+0000 7f932ffff640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f931806c330 0x7f931806e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:45.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.305+0000 7f9336ffd640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f931806c330 0x7f931806e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:45.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.305+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(88..88 src has 1..88) ==== 5973+0+0 (secure 0 0 0) 0x7f932008ad90 con 0x7f9338152970 2024-09-16T00:09:45.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.306+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v88) ==== 54+0+16 (secure 0 0 0) 0x7f932008d460 con 0x7f9338152970 2024-09-16T00:09:45.380 INFO:tasks.workunit.client.0.smithi139.stdout:6/319: fsync d8/d1f/f5c 0 2024-09-16T00:09:45.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.306+0000 7f9336ffd640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f931806c330 0x7f931806e7f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f932800f240 tx=0x7f9328008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:45.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.306+0000 7f933d477640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f933806bc70 0x7f93381f04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:45.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.306+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93380c42c0 con 0x7f933806bc70 2024-09-16T00:09:45.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.306+0000 7f93377fe640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f933806bc70 0x7f93381f04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:45.381 INFO:tasks.workunit.client.0.smithi139.stdout:9/590: write d19/d1a/d20/d23/d2d/d9a/fac [937808,5537] 0 2024-09-16T00:09:45.381 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.307+0000 7f93377fe640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f933806bc70 0x7f93381f04d0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:45.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.308+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93380c42c0 con 0x7f933806bc70 2024-09-16T00:09:45.383 INFO:tasks.workunit.client.0.smithi139.stdout:6/320: fsync d8/db/d26/d28/f3f 0 2024-09-16T00:09:45.383 INFO:tasks.workunit.client.0.smithi139.stdout:6/321: symlink d8/db/d26/d28/l6c 0 2024-09-16T00:09:45.383 INFO:tasks.workunit.client.0.smithi139.stdout:6/322: mkdir d8/db/d26/d28/d6d 0 2024-09-16T00:09:45.383 INFO:tasks.workunit.client.0.smithi139.stdout:6/323: chown f3 6 1 2024-09-16T00:09:45.383 INFO:tasks.workunit.client.0.smithi139.stdout:6/324: getdents d8 0 2024-09-16T00:09:45.383 INFO:tasks.workunit.client.0.smithi139.stdout:5/547: dread d1/d13/f24 [0,4194304] 0 2024-09-16T00:09:45.383 INFO:tasks.workunit.client.0.smithi139.stdout:6/325: mkdir d8/db/d26/d45/d65/d6e 0 2024-09-16T00:09:45.383 INFO:tasks.workunit.client.0.smithi139.stdout:5/548: mkdir d1/d21/d49/d53/d6a/d7b/da1/db7 0 2024-09-16T00:09:45.384 INFO:tasks.workunit.client.0.smithi139.stdout:5/549: chown d1/d13/d3f/d9f/d73/d79 0 1 2024-09-16T00:09:45.384 INFO:tasks.workunit.client.0.smithi139.stdout:6/326: unlink d8/db/d26/d45/d42/c52 0 2024-09-16T00:09:45.384 INFO:tasks.workunit.client.0.smithi139.stdout:9/591: write d19/f1d [1348370,101624] 0 2024-09-16T00:09:45.384 INFO:tasks.workunit.client.0.smithi139.stdout:9/592: chown d19/d1a/d20/d23/d2d/d41/d71 2325604 1 2024-09-16T00:09:45.384 INFO:tasks.workunit.client.0.smithi139.stdout:6/327: rename d8/db/d26/d28/f47 to d8/f6f 0 2024-09-16T00:09:45.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.328+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f933806c950 con 0x7f933806bc70 2024-09-16T00:09:45.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.330+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f933806c950 con 0x7f933806bc70 2024-09-16T00:09:45.385 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:45.385 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:45.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.330+0000 7f933d477640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f933806c050 0x7f93381f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:45.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.330+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f93380cc2e0 con 0x7f933806c050 2024-09-16T00:09:45.386 INFO:tasks.workunit.client.0.smithi139.stdout:9/593: mkdir d19/d1a/d20/d23/d2d/d35/d42/dc8 0 2024-09-16T00:09:45.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.330+0000 7f9336ffd640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f933806c050 0x7f93381f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:45.387 INFO:tasks.workunit.client.0.smithi139.stdout:9/594: fsync fa 0 2024-09-16T00:09:45.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.330+0000 7f9336ffd640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f933806c050 0x7f93381f3660 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:45.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.331+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93380cc2e0 con 0x7f933806c050 2024-09-16T00:09:45.388 INFO:tasks.workunit.client.0.smithi139.stdout:6/328: getdents d8/d56 0 2024-09-16T00:09:45.388 INFO:tasks.workunit.client.0.smithi139.stdout:6/329: creat d8/db/d26/d28/d6d/f70 x:0 0 0 2024-09-16T00:09:45.388 INFO:tasks.workunit.client.0.smithi139.stdout:9/595: dwrite f14 [0,4194304] 0 2024-09-16T00:09:45.389 INFO:tasks.workunit.client.0.smithi139.stdout:6/330: creat d8/db/d26/d28/d6d/f71 x:0 0 0 2024-09-16T00:09:45.389 INFO:tasks.workunit.client.0.smithi139.stdout:9/596: creat d19/d1a/d20/d23/d2d/d41/d71/da3/fc9 x:0 0 0 2024-09-16T00:09:45.389 INFO:tasks.workunit.client.0.smithi139.stdout:6/331: symlink d8/db/d26/l72 0 2024-09-16T00:09:45.389 INFO:tasks.workunit.client.0.smithi139.stdout:6/332: dread - d8/d29/f50 zero size 2024-09-16T00:09:45.389 INFO:tasks.workunit.client.0.smithi139.stdout:6/333: creat d8/d56/f73 x:0 0 0 2024-09-16T00:09:45.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.350+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f93381e5a30 con 0x7f933806c050 2024-09-16T00:09:45.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.351+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f93381e5a30 con 0x7f933806c050 2024-09-16T00:09:45.390 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.352+0000 7f933d477640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f93381f40b0 0x7f9338062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.352+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f93381f4490 con 0x7f93381f40b0 2024-09-16T00:09:45.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.352+0000 7f932e7fc640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f93381f40b0 0x7f9338062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:45.391 INFO:tasks.workunit.client.0.smithi139.stdout:9/597: rename d19/d1a/d20/d23/d2d/d41 to d19/d1a/d20/d23/dca 0 2024-09-16T00:09:45.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.352+0000 7f932e7fc640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f93381f40b0 0x7f9338062630 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:45.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.353+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93381f4490 con 0x7f93381f40b0 2024-09-16T00:09:45.392 INFO:tasks.workunit.client.0.smithi139.stdout:9/598: chown d19/d1a/d20/d23/dca/db5/lba 739112 1 2024-09-16T00:09:45.392 INFO:tasks.workunit.client.0.smithi139.stdout:9/599: dread d19/d1a/d20/d23/d2d/d35/fb1 [0,4194304] 0 2024-09-16T00:09:45.392 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:45.356+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:45.393 INFO:tasks.workunit.client.0.smithi139.stdout:6/334: dread f3 [0,4194304] 0 2024-09-16T00:09:45.393 INFO:tasks.workunit.client.0.smithi139.stdout:6/335: mkdir d8/db/d26/d28/d74 0 2024-09-16T00:09:45.394 INFO:tasks.workunit.client.0.smithi139.stdout:9/600: mkdir d19/d1a/d20/d23/d2d/d35/d42/d3a/dcb 0 2024-09-16T00:09:45.394 INFO:tasks.workunit.client.0.smithi139.stdout:6/336: dread d8/d20/f64 [0,4194304] 0 2024-09-16T00:09:45.394 INFO:tasks.workunit.client.0.smithi139.stdout:6/337: write d8/d20/f4a [55460,73347] 0 2024-09-16T00:09:45.394 INFO:tasks.workunit.client.0.smithi139.stdout:9/601: rename d19/d1a/d20/d23/dca/d71/d32/d4e/d67/fa8 to d19/d1a/d20/d23/d2d/d9a/fcc 0 2024-09-16T00:09:45.394 INFO:tasks.workunit.client.0.smithi139.stdout:6/338: unlink d8/db/d26/d28/d6d/f70 0 2024-09-16T00:09:45.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.370+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9338062dc0 con 0x7f93381f40b0 2024-09-16T00:09:45.394 INFO:tasks.workunit.client.0.smithi139.stdout:6/339: link d8/d29/f2d d8/db/d26/f75 0 2024-09-16T00:09:45.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.371+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9338062dc0 con 0x7f93381f40b0 2024-09-16T00:09:45.395 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:45.396 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:45.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.371+0000 7f933d477640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f9338063390 0x7f9338151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:45.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.371+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f9338062dc0 con 0x7f9338063390 2024-09-16T00:09:45.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.372+0000 7f93377fe640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f9338063390 0x7f9338151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:45.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.372+0000 7f93377fe640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f9338063390 0x7f9338151140 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:45.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.373+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9330045e50 con 0x7f9338063390 2024-09-16T00:09:45.397 INFO:tasks.workunit.client.0.smithi139.stdout:9/602: dwrite d19/d1a/d20/d23/d2d/d35/fb1 [0,4194304] 0 2024-09-16T00:09:45.397 INFO:tasks.workunit.client.0.smithi139.stdout:9/603: chown d19/d1a/d20/d23/d2d/d35/d60 18 1 2024-09-16T00:09:45.397 INFO:tasks.workunit.client.0.smithi139.stdout:9/604: write d19/d1a/d20/d23/d2d/d35/d60/f62 [1309342,102163] 0 2024-09-16T00:09:45.398 INFO:tasks.workunit.client.0.smithi139.stdout:6/340: dread d8/f6f [0,4194304] 0 2024-09-16T00:09:45.398 INFO:tasks.workunit.client.0.smithi139.stdout:9/605: read d19/d1a/d20/d23/dca/d71/fb0 [2167947,45297] 0 2024-09-16T00:09:45.398 INFO:tasks.workunit.client.0.smithi139.stdout:6/341: creat d8/db/d26/d45/d42/f76 x:0 0 0 2024-09-16T00:09:45.398 INFO:tasks.workunit.client.0.smithi139.stdout:6/342: symlink d8/d20/l77 0 2024-09-16T00:09:45.398 INFO:tasks.workunit.client.0.smithi139.stdout:6/343: chown d8/fe 110 1 2024-09-16T00:09:45.398 INFO:tasks.workunit.client.0.smithi139.stdout:9/606: symlink d19/d1a/d20/d23/dca/d71/d32/d4e/d67/lcd 0 2024-09-16T00:09:45.398 INFO:tasks.workunit.client.0.smithi139.stdout:6/344: symlink d8/db/l78 0 2024-09-16T00:09:45.398 INFO:tasks.workunit.client.0.smithi139.stdout:9/607: creat d19/d1a/d20/d23/dca/d71/da3/fce x:0 0 0 2024-09-16T00:09:45.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.393+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f93381518d0 con 0x7f9338063390 2024-09-16T00:09:45.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.395+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f93381518d0 con 0x7f9338063390 2024-09-16T00:09:45.399 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:45.400 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:45.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.395+0000 7f933d477640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f933802cb50 0x7f933802cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:45.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.395+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f93381518d0 con 0x7f933802cb50 2024-09-16T00:09:45.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.396+0000 7f9336ffd640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f933802cb50 0x7f933802cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:45.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.396+0000 7f9336ffd640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f933802cb50 0x7f933802cfb0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:45.400 INFO:tasks.workunit.client.0.smithi139.stdout:9/608: dread d19/d1a/d20/d23/d2d/f90 [0,4194304] 0 2024-09-16T00:09:45.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.397+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93381518d0 con 0x7f933802cb50 2024-09-16T00:09:45.401 INFO:tasks.workunit.client.0.smithi139.stdout:6/345: dread d8/db/d26/f61 [0,4194304] 0 2024-09-16T00:09:45.401 INFO:tasks.workunit.client.0.smithi139.stdout:9/609: creat d19/d1a/d20/d23/d2d/d35/d42/d3a/fcf x:0 0 0 2024-09-16T00:09:45.402 INFO:tasks.workunit.client.0.smithi139.stdout:6/346: getdents d8/d20/d68 0 2024-09-16T00:09:45.403 INFO:tasks.workunit.client.0.smithi139.stdout:9/610: dread - d19/d1a/d20/d89/d93/fbb zero size 2024-09-16T00:09:45.404 INFO:tasks.workunit.client.0.smithi139.stdout:9/611: write d19/d1a/d20/d23/d2d/f92 [62352,86639] 0 2024-09-16T00:09:45.404 INFO:tasks.workunit.client.0.smithi139.stdout:6/347: unlink d8/db/d26/d28/f4f 0 2024-09-16T00:09:45.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.415+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f93381518d0 con 0x7f933802cb50 2024-09-16T00:09:45.416 INFO:tasks.workunit.client.0.smithi139.stdout:9/612: dread d19/d1a/f50 [0,4194304] 0 2024-09-16T00:09:45.417 INFO:tasks.workunit.client.0.smithi139.stdout:9/613: readlink d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/lab 0 2024-09-16T00:09:45.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.416+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f93381518d0 con 0x7f933802cb50 2024-09-16T00:09:45.418 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:45.418 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:45.419 INFO:tasks.workunit.client.0.smithi139.stdout:9/614: chown d19/f26 195192014 1 2024-09-16T00:09:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.417+0000 7f933d477640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f9338204a00 0x7f9338206df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.417+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f933802da00 con 0x7f9338204a00 2024-09-16T00:09:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.417+0000 7f932e7fc640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f9338204a00 0x7f9338206df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:45.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.417+0000 7f932e7fc640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f9338204a00 0x7f9338206df0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:45.420 INFO:tasks.workunit.client.0.smithi139.stdout:9/615: dread - d19/d1a/d20/d23/d2d/d35/d60/fb3 zero size 2024-09-16T00:09:45.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.418+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f933802da00 con 0x7f9338204a00 2024-09-16T00:09:45.421 INFO:tasks.workunit.client.0.smithi139.stdout:9/616: mknod d19/d1a/d20/d23/dca/db5/cd0 0 2024-09-16T00:09:45.422 INFO:tasks.workunit.client.0.smithi139.stdout:9/617: write d19/d1a/faf [3283810,61741] 0 2024-09-16T00:09:45.425 INFO:tasks.workunit.client.0.smithi139.stdout:9/618: link d19/d1a/d20/d23/dca/d71/lbf d19/d1a/d20/d23/d2d/d35/d44/ld1 0 2024-09-16T00:09:45.426 INFO:tasks.workunit.client.0.smithi139.stdout:4/417: sync 2024-09-16T00:09:45.426 INFO:tasks.workunit.client.0.smithi139.stdout:0/474: sync 2024-09-16T00:09:45.426 INFO:tasks.workunit.client.0.smithi139.stdout:2/480: sync 2024-09-16T00:09:45.426 INFO:tasks.workunit.client.0.smithi139.stdout:8/491: sync 2024-09-16T00:09:45.429 INFO:tasks.workunit.client.0.smithi139.stdout:9/619: symlink d19/d1a/d20/d23/d2d/d35/d42/dc8/ld2 0 2024-09-16T00:09:45.429 INFO:tasks.workunit.client.0.smithi139.stdout:2/481: dread - d11/d30/d57/d50/f6a zero size 2024-09-16T00:09:45.430 INFO:tasks.workunit.client.0.smithi139.stdout:8/492: write d17/d1a/d43/f2b [1447095,54890] 0 2024-09-16T00:09:45.430 INFO:tasks.workunit.client.0.smithi139.stdout:0/475: stat d6/d5a/l6d 0 2024-09-16T00:09:45.432 INFO:tasks.workunit.client.0.smithi139.stdout:0/476: creat d6/d1c/d41/f96 x:0 0 0 2024-09-16T00:09:45.432 INFO:tasks.workunit.client.0.smithi139.stdout:9/620: rmdir d19/d1a/d20/d23/dca/d71/d32/d79 39 2024-09-16T00:09:45.432 INFO:tasks.workunit.client.0.smithi139.stdout:8/493: read d17/d1a/d57/f98 [431903,117722] 0 2024-09-16T00:09:45.437 INFO:tasks.workunit.client.0.smithi139.stdout:0/477: dread - d6/d1c/d41/d44/d58/f68 zero size 2024-09-16T00:09:45.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.437+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f933802da00 con 0x7f9338204a00 2024-09-16T00:09:45.439 INFO:tasks.workunit.client.0.smithi139.stdout:9/621: write d19/d1a/d20/d23/d2d/d35/f66 [351284,116071] 0 2024-09-16T00:09:45.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.438+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f933802da00 con 0x7f9338204a00 2024-09-16T00:09:45.440 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:45.440 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:45.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.439+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=89}) -- 0x7f933802da00 con 0x7f9338152970 2024-09-16T00:09:45.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.439+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9338209c20 con 0x7f9338152970 2024-09-16T00:09:45.441 INFO:tasks.workunit.client.0.smithi139.stdout:2/482: dwrite d11/f1a [4194304,4194304] 0 2024-09-16T00:09:45.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.439+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f932007fa50 con 0x7f9338152970 2024-09-16T00:09:45.441 INFO:teuthology.orchestra.run.smithi139.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T00:09:45.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.440+0000 7f933d477640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9338207c00 0x7f9338208070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:45.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.440+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f9338209960 con 0x7f9338207c00 2024-09-16T00:09:45.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.440+0000 7f93377fe640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9338207c00 0x7f9338208070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:45.442 INFO:tasks.workunit.client.0.smithi139.stdout:0/478: unlink d6/f8 0 2024-09-16T00:09:45.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.440+0000 7f93377fe640 1 --2- 172.21.15.139:0/2715716558 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f9338207c00 0x7f9338208070 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:45.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.441+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9338209960 con 0x7f9338207c00 2024-09-16T00:09:45.444 INFO:tasks.workunit.client.0.smithi139.stdout:2/483: stat d11/d30/d57/d29/f5c 0 2024-09-16T00:09:45.445 INFO:tasks.workunit.client.0.smithi139.stdout:9/622: creat d19/d1a/d20/d89/d93/fd3 x:0 0 0 2024-09-16T00:09:45.445 INFO:tasks.workunit.client.0.smithi139.stdout:4/418: dread d12/d4a/d4d/d69/f72 [0,4194304] 0 2024-09-16T00:09:45.446 INFO:tasks.workunit.client.0.smithi139.stdout:8/494: dwrite d17/d1a/d2d/f85 [0,4194304] 0 2024-09-16T00:09:45.447 INFO:tasks.workunit.client.0.smithi139.stdout:4/419: chown d12/d2b/c3c 31 1 2024-09-16T00:09:45.449 INFO:tasks.workunit.client.0.smithi139.stdout:2/484: write d11/d30/d57/f83 [668359,106431] 0 2024-09-16T00:09:45.450 INFO:tasks.workunit.client.0.smithi139.stdout:2/485: stat d11/d30/d57/d5d/d73 0 2024-09-16T00:09:45.450 INFO:tasks.workunit.client.0.smithi139.stdout:9/623: mknod d19/d1a/d20/d23/cd4 0 2024-09-16T00:09:45.451 INFO:tasks.workunit.client.0.smithi139.stdout:0/479: write d6/d1c/d41/d44/f29 [3082071,78485] 0 2024-09-16T00:09:45.452 INFO:tasks.workunit.client.0.smithi139.stdout:8/495: chown d17/d1a/d43/c84 82134739 1 2024-09-16T00:09:45.453 INFO:tasks.workunit.client.0.smithi139.stdout:4/420: symlink d12/d47/d57/d7f/l8b 0 2024-09-16T00:09:45.453 INFO:tasks.workunit.client.0.smithi139.stdout:2/486: rmdir d11/d30/d57/d29/d4f 39 2024-09-16T00:09:45.454 INFO:tasks.workunit.client.0.smithi139.stdout:9/624: fsync d19/d1a/d20/d23/dca/d71/d32/fb4 0 2024-09-16T00:09:45.455 INFO:tasks.workunit.client.0.smithi139.stdout:8/496: chown d17/l44 397986354 1 2024-09-16T00:09:45.455 INFO:tasks.workunit.client.0.smithi139.stdout:4/421: mkdir d12/d2b/d8c 0 2024-09-16T00:09:45.455 INFO:tasks.workunit.client.0.smithi139.stdout:0/480: read d6/d1c/d41/d44/f33 [15802,88434] 0 2024-09-16T00:09:45.456 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:45.456+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:45.458 INFO:tasks.workunit.client.0.smithi139.stdout:2/487: rename d11/c1c to d11/d30/d57/d5d/d73/d74/c9f 0 2024-09-16T00:09:45.458 INFO:tasks.workunit.client.0.smithi139.stdout:8/497: write d17/d1a/d43/d40/d4f/f65 [668998,66097] 0 2024-09-16T00:09:45.458 INFO:tasks.workunit.client.0.smithi139.stdout:0/481: mknod d6/d42/c97 0 2024-09-16T00:09:45.460 INFO:tasks.workunit.client.0.smithi139.stdout:4/422: creat d12/f8d x:0 0 0 2024-09-16T00:09:45.460 INFO:tasks.workunit.client.0.smithi139.stdout:8/498: mkdir d17/d1a/d2d/dac 0 2024-09-16T00:09:45.461 INFO:tasks.workunit.client.0.smithi139.stdout:2/488: mknod d11/d2f/d6e/d7f/ca0 0 2024-09-16T00:09:45.462 INFO:tasks.workunit.client.0.smithi139.stdout:0/482: creat d6/d1c/d41/d44/dd/d56/d75/f98 x:0 0 0 2024-09-16T00:09:45.463 INFO:tasks.workunit.client.0.smithi139.stdout:4/423: rmdir d12/d47/d57/d7f 39 2024-09-16T00:09:45.463 INFO:tasks.workunit.client.0.smithi139.stdout:8/499: chown d17/d1a/d43/d40/f8d 4792 1 2024-09-16T00:09:45.464 INFO:tasks.workunit.client.0.smithi139.stdout:9/625: dwrite d19/d1a/d20/d23/d2d/d9a/fac [0,4194304] 0 2024-09-16T00:09:45.464 INFO:tasks.workunit.client.0.smithi139.stdout:2/489: truncate d11/d2f/d6e/f80 677778 0 2024-09-16T00:09:45.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.464+0000 7f933d477640 1 -- 172.21.15.139:0/2715716558 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9338209930 con 0x7f9338207c00 2024-09-16T00:09:45.466 INFO:tasks.workunit.client.0.smithi139.stdout:8/500: mkdir d17/d1f/d3a/dad 0 2024-09-16T00:09:45.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:45.466+0000 7f932ffff640 1 -- 172.21.15.139:0/2715716558 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9338209930 con 0x7f9338207c00 2024-09-16T00:09:45.468 INFO:tasks.workunit.client.0.smithi139.stdout:4/424: symlink d12/d47/d49/d60/l8e 0 2024-09-16T00:09:45.468 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:45.468 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:45.469 INFO:tasks.workunit.client.0.smithi139.stdout:0/483: creat d6/d1c/d41/d63/d82/f99 x:0 0 0 2024-09-16T00:09:45.469 INFO:tasks.workunit.client.0.smithi139.stdout:2/490: mknod d11/d30/d57/d5d/ca1 0 2024-09-16T00:09:45.471 INFO:tasks.workunit.client.0.smithi139.stdout:0/484: stat d6/d1c/d41/d44/f3b 0 2024-09-16T00:09:45.472 INFO:tasks.workunit.client.0.smithi139.stdout:8/501: truncate f7 1746998 0 2024-09-16T00:09:45.473 INFO:tasks.workunit.client.0.smithi139.stdout:4/425: truncate d12/d47/d49/d60/f74 944224 0 2024-09-16T00:09:45.474 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T00:09:45.474 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T00:09:45.474 INFO:tasks.workunit.client.0.smithi139.stdout:2/491: dread d11/d2f/d6e/f80 [0,4194304] 0 2024-09-16T00:09:45.475 INFO:tasks.workunit.client.0.smithi139.stdout:2/492: write d11/f16 [375064,45593] 0 2024-09-16T00:09:45.476 INFO:tasks.workunit.client.0.smithi139.stdout:8/502: mknod d17/d1a/d43/d40/cae 0 2024-09-16T00:09:45.478 INFO:tasks.workunit.client.0.smithi139.stdout:2/493: write d11/f53 [783958,76564] 0 2024-09-16T00:09:45.482 INFO:tasks.workunit.client.0.smithi139.stdout:8/503: mkdir d17/d1a/d50/d86/d9d/daf 0 2024-09-16T00:09:45.482 INFO:tasks.workunit.client.0.smithi139.stdout:0/485: dwrite d6/d1c/f61 [0,4194304] 0 2024-09-16T00:09:45.482 INFO:tasks.workunit.client.0.smithi139.stdout:2/494: write d11/d30/d57/d5d/d73/f79 [781403,54427] 0 2024-09-16T00:09:45.484 INFO:tasks.workunit.client.0.smithi139.stdout:9/626: dread d19/d86/f63 [0,4194304] 0 2024-09-16T00:09:45.487 INFO:tasks.workunit.client.0.smithi139.stdout:8/504: mkdir d17/d1f/d3a/db0 0 2024-09-16T00:09:45.487 INFO:tasks.workunit.client.0.smithi139.stdout:4/426: write f7 [1556352,120025] 0 2024-09-16T00:09:45.489 INFO:tasks.workunit.client.0.smithi139.stdout:0/486: symlink d6/d1c/d41/l9a 0 2024-09-16T00:09:45.490 INFO:tasks.workunit.client.0.smithi139.stdout:2/495: rename d11/d30/d57/f83 to d11/d30/d57/d29/d4f/fa2 0 2024-09-16T00:09:45.492 INFO:tasks.workunit.client.0.smithi139.stdout:8/505: symlink d17/d1a/lb1 0 2024-09-16T00:09:45.492 INFO:tasks.workunit.client.0.smithi139.stdout:2/496: write d11/d30/d57/d5d/d73/f8e [652015,71298] 0 2024-09-16T00:09:45.493 INFO:tasks.workunit.client.0.smithi139.stdout:4/427: dread d12/f4f [0,4194304] 0 2024-09-16T00:09:45.493 INFO:tasks.workunit.client.0.smithi139.stdout:8/506: read d17/f83 [264537,29659] 0 2024-09-16T00:09:45.494 INFO:tasks.workunit.client.0.smithi139.stdout:4/428: readlink d12/l75 0 2024-09-16T00:09:45.494 INFO:tasks.workunit.client.0.smithi139.stdout:0/487: link d6/d1c/d45/c5d d6/d1c/d41/d44/d6b/c9b 0 2024-09-16T00:09:45.495 INFO:tasks.workunit.client.0.smithi139.stdout:0/488: chown d6/d1c/d41/d78 18854 1 2024-09-16T00:09:45.496 INFO:tasks.workunit.client.0.smithi139.stdout:2/497: creat d11/d30/d57/d5d/d73/fa3 x:0 0 0 2024-09-16T00:09:45.497 INFO:tasks.workunit.client.0.smithi139.stdout:4/429: creat d12/d47/d49/d60/f8f x:0 0 0 2024-09-16T00:09:45.499 INFO:tasks.workunit.client.0.smithi139.stdout:2/498: creat d11/d45/fa4 x:0 0 0 2024-09-16T00:09:45.500 INFO:tasks.workunit.client.0.smithi139.stdout:4/430: rmdir d12/d2b 39 2024-09-16T00:09:45.502 INFO:tasks.workunit.client.0.smithi139.stdout:4/431: write d12/d47/f2c [6702072,15178] 0 2024-09-16T00:09:45.502 INFO:tasks.workunit.client.0.smithi139.stdout:2/499: truncate d11/d30/d57/f27 4470542 0 2024-09-16T00:09:45.503 INFO:tasks.workunit.client.0.smithi139.stdout:4/432: readlink d12/d4a/l6d 0 2024-09-16T00:09:45.503 INFO:tasks.workunit.client.0.smithi139.stdout:4/433: stat d12/c2f 0 2024-09-16T00:09:45.505 INFO:tasks.workunit.client.0.smithi139.stdout:2/500: write d11/d30/d57/d29/d2b/f98 [481030,78043] 0 2024-09-16T00:09:45.505 INFO:tasks.workunit.client.0.smithi139.stdout:4/434: mknod d12/d80/c90 0 2024-09-16T00:09:45.506 INFO:tasks.workunit.client.0.smithi139.stdout:8/507: dread d17/d1a/f73 [0,4194304] 0 2024-09-16T00:09:45.507 INFO:tasks.workunit.client.0.smithi139.stdout:4/435: write d12/d4a/d4d/f79 [698800,86909] 0 2024-09-16T00:09:45.508 INFO:tasks.workunit.client.0.smithi139.stdout:8/508: write d17/d1a/f35 [4386369,25982] 0 2024-09-16T00:09:45.510 INFO:tasks.workunit.client.0.smithi139.stdout:4/436: truncate d12/d4a/d4d/f81 881399 0 2024-09-16T00:09:45.511 INFO:tasks.workunit.client.0.smithi139.stdout:2/501: rename d11/d30/d57/d5d/d73/d74 to d11/d30/d57/d29/da5 0 2024-09-16T00:09:45.512 INFO:tasks.workunit.client.0.smithi139.stdout:0/489: dread d6/d1c/d41/d44/dd/f39 [0,4194304] 0 2024-09-16T00:09:45.512 INFO:tasks.workunit.client.0.smithi139.stdout:4/437: write d12/d4a/d4d/f79 [797442,63787] 0 2024-09-16T00:09:45.521 INFO:tasks.workunit.client.0.smithi139.stdout:4/438: fdatasync d12/d2b/f6b 0 2024-09-16T00:09:45.522 INFO:tasks.workunit.client.0.smithi139.stdout:2/502: dread d11/f1a [4194304,4194304] 0 2024-09-16T00:09:45.524 INFO:tasks.workunit.client.0.smithi139.stdout:8/509: dread d17/d1a/d2d/f3e [0,4194304] 0 2024-09-16T00:09:45.524 INFO:tasks.workunit.client.0.smithi139.stdout:4/439: write d12/d47/d49/d60/f76 [471369,46683] 0 2024-09-16T00:09:45.525 INFO:tasks.workunit.client.0.smithi139.stdout:0/490: dwrite d6/d1c/d41/d44/d58/f88 [0,4194304] 0 2024-09-16T00:09:45.527 INFO:tasks.workunit.client.0.smithi139.stdout:2/503: truncate d11/d30/d57/d29/d2b/f2c 467619 0 2024-09-16T00:09:45.527 INFO:tasks.workunit.client.0.smithi139.stdout:4/440: read - d12/d47/d57/f5f zero size 2024-09-16T00:09:45.530 INFO:tasks.workunit.client.0.smithi139.stdout:0/491: truncate d6/d1c/d41/d44/dd/f24 5427544 0 2024-09-16T00:09:45.531 INFO:tasks.workunit.client.0.smithi139.stdout:2/504: mknod d11/d2f/ca6 0 2024-09-16T00:09:45.531 INFO:tasks.workunit.client.0.smithi139.stdout:0/492: write d6/f64 [218610,129538] 0 2024-09-16T00:09:45.532 INFO:tasks.workunit.client.0.smithi139.stdout:4/441: symlink d12/d47/d57/d7f/l91 0 2024-09-16T00:09:45.532 INFO:tasks.workunit.client.0.smithi139.stdout:8/510: getdents d17/d1a/d50/d86/d92 0 2024-09-16T00:09:45.534 INFO:tasks.workunit.client.0.smithi139.stdout:0/493: mknod d6/d1c/c9c 0 2024-09-16T00:09:45.535 INFO:tasks.workunit.client.0.smithi139.stdout:0/494: chown d6/d1c/d41/d44/dd/f39 457 1 2024-09-16T00:09:45.535 INFO:tasks.workunit.client.0.smithi139.stdout:0/495: fsync d6/d1c/d45/f50 0 2024-09-16T00:09:45.536 INFO:tasks.workunit.client.0.smithi139.stdout:4/442: creat d12/d2b/d8c/f92 x:0 0 0 2024-09-16T00:09:45.536 INFO:tasks.workunit.client.0.smithi139.stdout:8/511: mknod d17/d1a/d50/d86/d9d/daf/cb2 0 2024-09-16T00:09:45.537 INFO:tasks.workunit.client.0.smithi139.stdout:8/512: chown d17/d1f/d3a 9694 1 2024-09-16T00:09:45.537 INFO:tasks.workunit.client.0.smithi139.stdout:4/443: rmdir d12/d2b/d8c 39 2024-09-16T00:09:45.539 INFO:tasks.workunit.client.0.smithi139.stdout:4/444: mknod d12/d47/d49/d60/c93 0 2024-09-16T00:09:45.540 INFO:tasks.workunit.client.0.smithi139.stdout:2/505: truncate d11/d30/d57/d5d/d73/f8e 707101 0 2024-09-16T00:09:45.541 INFO:tasks.workunit.client.0.smithi139.stdout:8/513: link d17/d1a/fa5 d17/d1a/d2d/dac/fb3 0 2024-09-16T00:09:45.542 INFO:tasks.workunit.client.0.smithi139.stdout:2/506: creat d11/d30/d57/d29/d2b/fa7 x:0 0 0 2024-09-16T00:09:45.543 INFO:tasks.workunit.client.0.smithi139.stdout:0/496: dwrite d6/d1c/d41/d44/dd/d56/f57 [0,4194304] 0 2024-09-16T00:09:45.543 INFO:tasks.workunit.client.0.smithi139.stdout:8/514: chown d17/d1a/d50/d86/d9d 180 1 2024-09-16T00:09:45.544 INFO:tasks.workunit.client.0.smithi139.stdout:0/497: write d6/d1c/d41/f6a [114859,2183] 0 2024-09-16T00:09:45.546 INFO:tasks.workunit.client.0.smithi139.stdout:2/507: creat d11/d32/fa8 x:0 0 0 2024-09-16T00:09:45.547 INFO:tasks.workunit.client.0.smithi139.stdout:0/498: symlink d6/d5a/d70/l9d 0 2024-09-16T00:09:45.548 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:45.548 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:45.549 INFO:tasks.workunit.client.0.smithi139.stdout:2/508: symlink d11/d30/d57/d29/d4f/d9c/la9 0 2024-09-16T00:09:45.550 INFO:tasks.workunit.client.0.smithi139.stdout:4/445: truncate d12/d47/d57/f59 1825342 0 2024-09-16T00:09:45.552 INFO:tasks.workunit.client.0.smithi139.stdout:4/446: fdatasync d12/d4a/f54 0 2024-09-16T00:09:45.553 INFO:tasks.workunit.client.0.smithi139.stdout:8/515: dwrite d17/d1a/d43/d40/f49 [0,4194304] 0 2024-09-16T00:09:45.555 INFO:tasks.workunit.client.0.smithi139.stdout:2/509: rename d11/d30/d57/d29/d4f/d9e to d11/d30/d57/d29/da5/daa 0 2024-09-16T00:09:45.555 INFO:tasks.workunit.client.0.smithi139.stdout:8/516: truncate d17/d89/f8f 165852 0 2024-09-16T00:09:45.556 INFO:tasks.workunit.client.0.smithi139.stdout:2/510: fsync d11/f19 0 2024-09-16T00:09:45.556 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:45.556+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:45.557 INFO:tasks.workunit.client.0.smithi139.stdout:8/517: unlink d17/f19 0 2024-09-16T00:09:45.558 INFO:tasks.workunit.client.0.smithi139.stdout:8/518: write d17/d54/f69 [122604,63166] 0 2024-09-16T00:09:45.559 INFO:tasks.workunit.client.0.smithi139.stdout:4/447: getdents d12/d4a/d4d 0 2024-09-16T00:09:45.561 INFO:tasks.workunit.client.0.smithi139.stdout:4/448: chown d12/f23 105160921 1 2024-09-16T00:09:45.562 INFO:tasks.workunit.client.0.smithi139.stdout:2/511: rename d11/d30/d57/d29/d2b/f34 to d11/d30/d57/d29/d4f/d9c/fab 0 2024-09-16T00:09:45.563 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:09:45.565 INFO:tasks.workunit.client.0.smithi139.stdout:0/499: dread d6/d1c/d41/d44/f29 [0,4194304] 0 2024-09-16T00:09:45.565 INFO:tasks.workunit.client.0.smithi139.stdout:8/519: rename d17/d1a/d50/fa4 to d17/d1a/d43/d40/fb4 0 2024-09-16T00:09:45.566 INFO:tasks.workunit.client.0.smithi139.stdout:2/512: creat d11/d30/d57/d5d/fac x:0 0 0 2024-09-16T00:09:45.568 INFO:tasks.workunit.client.0.smithi139.stdout:8/520: creat d17/d1f/d3a/dad/fb5 x:0 0 0 2024-09-16T00:09:45.571 INFO:tasks.workunit.client.0.smithi139.stdout:0/500: mkdir d6/d1c/d41/d44/dd/d8a/d9e 0 2024-09-16T00:09:45.572 INFO:tasks.workunit.client.0.smithi139.stdout:0/501: write d6/d1c/d45/f50 [926911,34974] 0 2024-09-16T00:09:45.574 INFO:tasks.workunit.client.0.smithi139.stdout:2/513: link d11/d2f/d6e/d7f/ca0 d11/d30/cad 0 2024-09-16T00:09:45.576 INFO:tasks.workunit.client.0.smithi139.stdout:2/514: write d11/d30/d57/d5d/d73/fa3 [854635,115551] 0 2024-09-16T00:09:45.576 INFO:tasks.workunit.client.0.smithi139.stdout:8/521: link d17/d1a/f73 d17/d54/fb6 0 2024-09-16T00:09:45.577 INFO:tasks.workunit.client.0.smithi139.stdout:0/502: fdatasync d6/d1c/d41/d44/dd/d8a/f92 0 2024-09-16T00:09:45.578 INFO:tasks.workunit.client.0.smithi139.stdout:4/449: dread d12/d47/d57/f59 [0,4194304] 0 2024-09-16T00:09:45.579 INFO:tasks.workunit.client.0.smithi139.stdout:0/503: chown d6/d1c/d41/d44/dd/le 7 1 2024-09-16T00:09:45.580 INFO:tasks.workunit.client.0.smithi139.stdout:8/522: rename d17/d1a/d43 to d17/d1f/d3a/db7 0 2024-09-16T00:09:45.581 INFO:tasks.workunit.client.0.smithi139.stdout:8/523: stat d17/d1f/d3a/db7/d40/fb4 0 2024-09-16T00:09:45.582 INFO:tasks.workunit.client.0.smithi139.stdout:0/504: readlink d6/d1c/d41/d44/l3f 0 2024-09-16T00:09:45.583 INFO:tasks.workunit.client.0.smithi139.stdout:2/515: rename d11/d2f/d6e/f9a to d11/d30/d57/d29/d4f/d9c/fae 0 2024-09-16T00:09:45.584 INFO:tasks.workunit.client.0.smithi139.stdout:8/524: creat d17/d89/fb8 x:0 0 0 2024-09-16T00:09:45.585 INFO:tasks.workunit.client.0.smithi139.stdout:2/516: read d11/d30/d57/d29/f5c [3216344,7957] 0 2024-09-16T00:09:45.585 INFO:tasks.workunit.client.0.smithi139.stdout:0/505: symlink d6/d5a/l9f 0 2024-09-16T00:09:45.589 INFO:tasks.workunit.client.0.smithi139.stdout:8/525: getdents d17/d1a/d50/d86/d92 0 2024-09-16T00:09:45.590 INFO:tasks.workunit.client.0.smithi139.stdout:4/450: rename d12/d47/f2c to d12/d2b/f94 0 2024-09-16T00:09:45.591 INFO:tasks.workunit.client.0.smithi139.stdout:8/526: rmdir d17/d1a/d2d/dac 39 2024-09-16T00:09:45.593 INFO:tasks.workunit.client.0.smithi139.stdout:0/506: getdents d6/d1c/d45 0 2024-09-16T00:09:45.593 INFO:tasks.workunit.client.0.smithi139.stdout:0/507: readlink d6/l1f 0 2024-09-16T00:09:45.593 INFO:tasks.workunit.client.0.smithi139.stdout:2/517: getdents d11/d30/d57/d29/d86 0 2024-09-16T00:09:45.594 INFO:tasks.workunit.client.0.smithi139.stdout:8/527: creat d17/d1f/d3a/dad/fb9 x:0 0 0 2024-09-16T00:09:45.595 INFO:tasks.workunit.client.0.smithi139.stdout:2/518: chown d11/d30/d57/c62 13631 1 2024-09-16T00:09:45.596 INFO:tasks.workunit.client.0.smithi139.stdout:8/528: symlink d17/d89/lba 0 2024-09-16T00:09:45.596 INFO:tasks.workunit.client.0.smithi139.stdout:4/451: dread d12/d2b/f78 [0,4194304] 0 2024-09-16T00:09:45.597 INFO:tasks.workunit.client.0.smithi139.stdout:0/508: creat d6/d1c/d41/d44/dd/fa0 x:0 0 0 2024-09-16T00:09:45.597 INFO:tasks.workunit.client.0.smithi139.stdout:2/519: chown d11/l46 3582928 1 2024-09-16T00:09:45.598 INFO:tasks.workunit.client.0.smithi139.stdout:4/452: creat d12/d80/f95 x:0 0 0 2024-09-16T00:09:45.598 INFO:tasks.workunit.client.0.smithi139.stdout:2/520: chown d11/f64 10327 1 2024-09-16T00:09:45.600 INFO:tasks.workunit.client.0.smithi139.stdout:0/509: fsync d6/d1c/d41/d44/f25 0 2024-09-16T00:09:45.601 INFO:tasks.workunit.client.0.smithi139.stdout:4/453: unlink d12/d4a/d4d/c67 0 2024-09-16T00:09:45.601 INFO:tasks.workunit.client.0.smithi139.stdout:0/510: readlink d6/d1c/d41/l9a 0 2024-09-16T00:09:45.603 INFO:tasks.workunit.client.0.smithi139.stdout:8/529: dwrite d17/d54/f69 [0,4194304] 0 2024-09-16T00:09:45.604 INFO:tasks.workunit.client.0.smithi139.stdout:2/521: rename fe to d11/d30/d57/d29/d2b/faf 0 2024-09-16T00:09:45.604 INFO:tasks.workunit.client.0.smithi139.stdout:4/454: symlink d12/d47/d49/l96 0 2024-09-16T00:09:45.608 INFO:tasks.workunit.client.0.smithi139.stdout:8/530: creat d17/d89/fbb x:0 0 0 2024-09-16T00:09:45.611 INFO:tasks.workunit.client.0.smithi139.stdout:0/511: dwrite d6/d1c/d41/d63/d82/f83 [0,4194304] 0 2024-09-16T00:09:45.614 INFO:tasks.workunit.client.0.smithi139.stdout:2/522: rename d11/d32/f6d to d11/d30/d57/d29/fb0 0 2024-09-16T00:09:45.616 INFO:tasks.workunit.client.0.smithi139.stdout:4/455: dwrite d12/d47/d49/d60/f76 [0,4194304] 0 2024-09-16T00:09:45.616 INFO:tasks.workunit.client.0.smithi139.stdout:0/512: mknod d6/d1c/d41/d78/ca1 0 2024-09-16T00:09:45.618 INFO:tasks.workunit.client.0.smithi139.stdout:8/531: creat d17/d1a/fbc x:0 0 0 2024-09-16T00:09:45.620 INFO:tasks.workunit.client.0.smithi139.stdout:0/513: truncate d6/d1c/d41/d44/dd/d8a/f92 1747008 0 2024-09-16T00:09:45.622 INFO:tasks.workunit.client.0.smithi139.stdout:2/523: symlink d11/d30/d57/d29/lb1 0 2024-09-16T00:09:45.625 INFO:tasks.workunit.client.0.smithi139.stdout:2/524: fsync d11/d30/f56 0 2024-09-16T00:09:45.629 INFO:tasks.workunit.client.0.smithi139.stdout:4/456: link d12/d4a/c4e d12/d47/d57/c97 0 2024-09-16T00:09:45.632 INFO:tasks.workunit.client.0.smithi139.stdout:2/525: dwrite d11/d30/d57/d50/f6a [0,4194304] 0 2024-09-16T00:09:45.632 INFO:tasks.workunit.client.0.smithi139.stdout:4/457: readlink d12/d47/d49/d60/l8e 0 2024-09-16T00:09:45.633 INFO:tasks.workunit.client.0.smithi139.stdout:3/454: sync 2024-09-16T00:09:45.633 INFO:tasks.workunit.client.0.smithi139.stdout:1/443: sync 2024-09-16T00:09:45.633 INFO:tasks.workunit.client.0.smithi139.stdout:7/465: sync 2024-09-16T00:09:45.633 INFO:tasks.workunit.client.0.smithi139.stdout:0/514: link d6/d1c/d41/d44/dd/l17 d6/d1c/d41/d44/d58/la2 0 2024-09-16T00:09:45.634 INFO:tasks.workunit.client.0.smithi139.stdout:3/455: readlink d5/l57 0 2024-09-16T00:09:45.634 INFO:tasks.workunit.client.0.smithi139.stdout:8/532: write d17/d1f/d3a/db7/f6c [43344,119144] 0 2024-09-16T00:09:45.635 INFO:tasks.workunit.client.0.smithi139.stdout:0/515: chown d6/d1c/d41/d44/dd/f40 0 1 2024-09-16T00:09:45.636 INFO:tasks.workunit.client.0.smithi139.stdout:8/533: chown d17/d1f 15317781 1 2024-09-16T00:09:45.636 INFO:tasks.workunit.client.0.smithi139.stdout:4/458: write d12/d4a/f63 [4071373,119901] 0 2024-09-16T00:09:45.637 INFO:tasks.workunit.client.0.smithi139.stdout:2/526: write d11/f15 [2737913,3942] 0 2024-09-16T00:09:45.640 INFO:tasks.workunit.client.0.smithi139.stdout:1/444: rmdir d7/d8/d40/d48/d69 0 2024-09-16T00:09:45.640 INFO:tasks.workunit.client.0.smithi139.stdout:8/534: dread - d17/d1a/d2d/dac/fb3 zero size 2024-09-16T00:09:45.641 INFO:tasks.workunit.client.0.smithi139.stdout:2/527: stat d11/d30/c70 0 2024-09-16T00:09:45.641 INFO:tasks.workunit.client.0.smithi139.stdout:0/516: read f3 [1986416,91742] 0 2024-09-16T00:09:45.644 INFO:tasks.workunit.client.0.smithi139.stdout:8/535: write d17/d1a/d57/f79 [944085,76727] 0 2024-09-16T00:09:45.644 INFO:tasks.workunit.client.0.smithi139.stdout:8/536: readlink d17/l20 0 2024-09-16T00:09:45.645 INFO:tasks.workunit.client.0.smithi139.stdout:2/528: creat d11/d30/d57/d29/d4f/fb2 x:0 0 0 2024-09-16T00:09:45.646 INFO:tasks.workunit.client.0.smithi139.stdout:1/445: mknod d7/d8/d27/d68/d7c/c86 0 2024-09-16T00:09:45.647 INFO:tasks.workunit.client.0.smithi139.stdout:7/466: dwrite d5/d16/f34 [0,4194304] 0 2024-09-16T00:09:45.649 INFO:tasks.workunit.client.0.smithi139.stdout:1/446: dread - d7/d8/d40/d48/d53/f85 zero size 2024-09-16T00:09:45.649 INFO:tasks.workunit.client.0.smithi139.stdout:4/459: truncate d12/d2b/f34 2196906 0 2024-09-16T00:09:45.651 INFO:tasks.workunit.client.0.smithi139.stdout:2/529: dread d11/d30/f92 [0,4194304] 0 2024-09-16T00:09:45.651 INFO:tasks.workunit.client.0.smithi139.stdout:8/537: write d17/f56 [570429,40136] 0 2024-09-16T00:09:45.652 INFO:tasks.workunit.client.0.smithi139.stdout:7/467: write d5/d37/f86 [2831455,50266] 0 2024-09-16T00:09:45.653 INFO:tasks.workunit.client.0.smithi139.stdout:0/517: dwrite d6/d1c/d45/f47 [0,4194304] 0 2024-09-16T00:09:45.653 INFO:teuthology.orchestra.run.smithi164.stdout:get_pg_num_history pg_num_history pg_num_history(e82 pg_nums {1={16=1},2={19=32},3={20=64}} deleted_pools ) 2024-09-16T00:09:45.653 INFO:teuthology.orchestra.run.smithi164.stdout:Importing pgid 2.17 2024-09-16T00:09:45.653 INFO:tasks.workunit.client.0.smithi139.stdout:1/447: dread - d7/d8/d40/d48/d4a/d74/f7e zero size 2024-09-16T00:09:45.654 INFO:teuthology.orchestra.run.smithi164.stdout:Write #2:e8f2ea03:::10000002282.03600000:head# 2024-09-16T00:09:45.654 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 0=[]:{} 2024-09-16T00:09:45.654 INFO:teuthology.orchestra.run.smithi164.stdout:Write #2:ed8e7cdc:::1000000260f.03000000:head# 2024-09-16T00:09:45.654 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 0=[]:{} 2024-09-16T00:09:45.655 INFO:tasks.workunit.client.0.smithi139.stdout:4/460: symlink d12/d47/d57/d7f/l98 0 2024-09-16T00:09:45.655 INFO:teuthology.orchestra.run.smithi164.stdout:Write #2:ee44d6a7:::10000000261.03e00000:head# 2024-09-16T00:09:45.655 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 0=[]:{} 2024-09-16T00:09:45.655 INFO:teuthology.orchestra.run.smithi164.stdout:write_pg epoch 76 info 2.17( v 76'3 (0'0,76'3] local-lis/les=19/20 n=3 ec=19/19 lis/c=19/19 les/c/f=20/20/0 sis=19) 2024-09-16T00:09:45.655 INFO:teuthology.orchestra.run.smithi164.stdout:Import successful 2024-09-16T00:09:45.657 INFO:tasks.workunit.client.0.smithi139.stdout:7/468: write d5/d37/f7b [2462939,79063] 0 2024-09-16T00:09:45.657 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:45.656+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:45.658 INFO:tasks.workunit.client.0.smithi139.stdout:4/461: dread - d12/d47/f5c zero size 2024-09-16T00:09:45.661 INFO:tasks.workunit.client.0.smithi139.stdout:0/518: truncate d6/d1c/d41/d44/f3b 4509186 0 2024-09-16T00:09:45.662 INFO:tasks.workunit.client.0.smithi139.stdout:7/469: write d5/f1f [5948237,52163] 0 2024-09-16T00:09:45.663 INFO:tasks.workunit.client.0.smithi139.stdout:2/530: dwrite d11/f53 [0,4194304] 0 2024-09-16T00:09:45.664 INFO:tasks.workunit.client.0.smithi139.stdout:1/448: truncate d7/d8/d40/d48/d53/f85 322910 0 2024-09-16T00:09:45.665 INFO:tasks.workunit.client.0.smithi139.stdout:8/538: truncate d17/d1a/d2d/f2e 5561677 0 2024-09-16T00:09:45.670 INFO:tasks.workunit.client.0.smithi139.stdout:4/462: rename d12/f23 to d12/d2b/f99 0 2024-09-16T00:09:45.674 INFO:tasks.workunit.client.0.smithi139.stdout:0/519: unlink d6/d1c/d41/d44/dd/l2b 0 2024-09-16T00:09:45.674 INFO:tasks.workunit.client.0.smithi139.stdout:2/531: truncate d11/f19 617207 0 2024-09-16T00:09:45.676 INFO:tasks.workunit.client.0.smithi139.stdout:1/449: rename d7/d8/d40/d48/d4a/f4c to d7/d8/d40/d48/d53/d61/f87 0 2024-09-16T00:09:45.677 INFO:tasks.workunit.client.0.smithi139.stdout:0/520: truncate d6/d5a/d70/f4f 2131128 0 2024-09-16T00:09:45.679 INFO:tasks.workunit.client.0.smithi139.stdout:4/463: unlink d12/d47/d49/d60/c7c 0 2024-09-16T00:09:45.681 INFO:tasks.workunit.client.0.smithi139.stdout:0/521: write d6/d1c/d45/f50 [1986557,16014] 0 2024-09-16T00:09:45.681 INFO:tasks.workunit.client.0.smithi139.stdout:8/539: dwrite d17/d1a/d2d/f3e [0,4194304] 0 2024-09-16T00:09:45.685 INFO:tasks.workunit.client.0.smithi139.stdout:7/470: dread d5/d26/f2b [0,4194304] 0 2024-09-16T00:09:45.691 INFO:tasks.workunit.client.0.smithi139.stdout:4/464: getdents d12/d4a/d4d 0 2024-09-16T00:09:45.691 INFO:tasks.workunit.client.0.smithi139.stdout:2/532: dwrite d11/d30/d57/d29/d2b/faf [0,4194304] 0 2024-09-16T00:09:45.692 INFO:tasks.workunit.client.0.smithi139.stdout:7/471: unlink d5/d16/l9d 0 2024-09-16T00:09:45.695 INFO:tasks.workunit.client.0.smithi139.stdout:8/540: link d17/d1a/d50/d86/d9d/daf/cb2 d17/d1a/d50/d86/d9d/cbd 0 2024-09-16T00:09:45.697 INFO:tasks.workunit.client.0.smithi139.stdout:2/533: unlink d11/d30/d57/d29/d2b/l6b 0 2024-09-16T00:09:45.697 INFO:tasks.workunit.client.0.smithi139.stdout:4/465: dread d12/d4a/d4d/f55 [0,4194304] 0 2024-09-16T00:09:45.699 INFO:tasks.workunit.client.0.smithi139.stdout:1/450: dread d7/d8/d27/d2f/f5c [0,4194304] 0 2024-09-16T00:09:45.699 INFO:tasks.workunit.client.0.smithi139.stdout:7/472: symlink d5/la2 0 2024-09-16T00:09:45.699 INFO:tasks.workunit.client.0.smithi139.stdout:8/541: creat d17/d54/fbe x:0 0 0 2024-09-16T00:09:45.701 INFO:tasks.workunit.client.0.smithi139.stdout:4/466: write d12/d47/f32 [8853564,71553] 0 2024-09-16T00:09:45.702 INFO:tasks.workunit.client.0.smithi139.stdout:2/534: creat d11/d30/d57/d29/d4f/d9c/fb3 x:0 0 0 2024-09-16T00:09:45.705 INFO:tasks.workunit.client.0.smithi139.stdout:0/522: dread d6/d1c/d41/d63/f89 [0,4194304] 0 2024-09-16T00:09:45.705 INFO:tasks.workunit.client.0.smithi139.stdout:1/451: creat d7/d8/d40/d48/d67/f88 x:0 0 0 2024-09-16T00:09:45.706 INFO:tasks.workunit.client.0.smithi139.stdout:8/542: truncate d17/d1f/d3a/db7/f42 1250373 0 2024-09-16T00:09:45.706 INFO:tasks.workunit.client.0.smithi139.stdout:4/467: creat d12/d80/f9a x:0 0 0 2024-09-16T00:09:45.707 INFO:tasks.workunit.client.0.smithi139.stdout:0/523: chown d6/d1c/f21 68 1 2024-09-16T00:09:45.707 INFO:tasks.workunit.client.0.smithi139.stdout:7/473: dread d5/d16/d22/d25/d2d/d50/f67 [0,4194304] 0 2024-09-16T00:09:45.708 INFO:tasks.workunit.client.0.smithi139.stdout:8/543: chown d17/d1a/d57/f7b 791 1 2024-09-16T00:09:45.708 INFO:tasks.workunit.client.0.smithi139.stdout:2/535: creat d11/d30/d57/d5d/fb4 x:0 0 0 2024-09-16T00:09:45.709 INFO:tasks.workunit.client.0.smithi139.stdout:7/474: write d5/d37/f79 [1195600,75682] 0 2024-09-16T00:09:45.709 INFO:tasks.workunit.client.0.smithi139.stdout:7/475: readlink d5/l11 0 2024-09-16T00:09:45.710 INFO:tasks.workunit.client.0.smithi139.stdout:0/524: fdatasync d6/d1c/d41/d44/f29 0 2024-09-16T00:09:45.711 INFO:tasks.workunit.client.0.smithi139.stdout:8/544: truncate d17/d1a/d50/d86/f97 432065 0 2024-09-16T00:09:45.713 INFO:tasks.workunit.client.0.smithi139.stdout:7/476: mknod d5/d37/ca3 0 2024-09-16T00:09:45.716 INFO:tasks.workunit.client.0.smithi139.stdout:4/468: creat d12/d47/d57/f9b x:0 0 0 2024-09-16T00:09:45.717 INFO:tasks.workunit.client.0.smithi139.stdout:8/545: dwrite d17/d1f/d3a/f58 [0,4194304] 0 2024-09-16T00:09:45.717 INFO:tasks.workunit.client.0.smithi139.stdout:0/525: symlink d6/d1c/d45/la3 0 2024-09-16T00:09:45.718 INFO:tasks.workunit.client.0.smithi139.stdout:8/546: chown d17/d1a/d2d/l8a 8296438 1 2024-09-16T00:09:45.723 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:45.723 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:45.725 INFO:tasks.workunit.client.0.smithi139.stdout:8/547: unlink d17/d1f/f94 0 2024-09-16T00:09:45.726 INFO:tasks.workunit.client.0.smithi139.stdout:4/469: creat d12/d2b/d8c/f9c x:0 0 0 2024-09-16T00:09:45.727 INFO:tasks.workunit.client.0.smithi139.stdout:0/526: creat d6/d5a/d81/fa4 x:0 0 0 2024-09-16T00:09:45.728 INFO:tasks.workunit.client.0.smithi139.stdout:2/536: getdents d11/d45 0 2024-09-16T00:09:45.733 INFO:tasks.workunit.client.0.smithi139.stdout:2/537: creat d11/d2f/d6e/d7f/fb5 x:0 0 0 2024-09-16T00:09:45.733 INFO:tasks.workunit.client.0.smithi139.stdout:1/452: dwrite d7/f13 [4194304,4194304] 0 2024-09-16T00:09:45.734 INFO:tasks.workunit.client.0.smithi139.stdout:1/453: stat d7/d8/d40/d46/f5b 0 2024-09-16T00:09:45.736 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:09:45.738 INFO:tasks.workunit.client.0.smithi139.stdout:0/527: getdents d6/d1c 0 2024-09-16T00:09:45.738 INFO:tasks.workunit.client.0.smithi139.stdout:4/470: getdents d12/d47/d49/d60 0 2024-09-16T00:09:45.738 INFO:tasks.workunit.client.0.smithi139.stdout:1/454: dread d7/d8/d40/d47/f5e [0,4194304] 0 2024-09-16T00:09:45.740 INFO:tasks.workunit.client.0.smithi139.stdout:2/538: rename d11/d32/l3c to d11/d30/d57/d29/d4f/d9c/lb6 0 2024-09-16T00:09:45.740 INFO:tasks.workunit.client.0.smithi139.stdout:0/528: fsync d6/d1c/d41/d78/f7d 0 2024-09-16T00:09:45.741 INFO:tasks.workunit.client.0.smithi139.stdout:4/471: creat d12/d4a/d4d/f9d x:0 0 0 2024-09-16T00:09:45.743 INFO:tasks.workunit.client.0.smithi139.stdout:1/455: mkdir d7/d8/d40/d46/d89 0 2024-09-16T00:09:45.744 INFO:tasks.workunit.client.0.smithi139.stdout:8/548: write d17/d1f/d3a/db7/d40/f4b [1337007,85889] 0 2024-09-16T00:09:45.746 INFO:tasks.workunit.client.0.smithi139.stdout:1/456: mknod d7/d8/d40/d46/c8a 0 2024-09-16T00:09:45.746 INFO:tasks.workunit.client.0.smithi139.stdout:0/529: getdents d6/d1c/d41/d44/dd/d8a/d9e 0 2024-09-16T00:09:45.747 INFO:tasks.workunit.client.0.smithi139.stdout:2/539: dread d11/d30/f42 [0,4194304] 0 2024-09-16T00:09:45.749 INFO:tasks.workunit.client.0.smithi139.stdout:2/540: rmdir d11/d30/d57 39 2024-09-16T00:09:45.750 INFO:tasks.workunit.client.0.smithi139.stdout:4/472: write d12/d2b/f73 [593890,25428] 0 2024-09-16T00:09:45.754 INFO:tasks.workunit.client.0.smithi139.stdout:4/473: chown d12/l87 22107 1 2024-09-16T00:09:45.755 INFO:tasks.workunit.client.0.smithi139.stdout:2/541: symlink d11/d30/d57/d29/d2b/lb7 0 2024-09-16T00:09:45.756 INFO:tasks.workunit.client.0.smithi139.stdout:4/474: mkdir d12/d47/d57/d7f/d9e 0 2024-09-16T00:09:45.757 INFO:tasks.workunit.client.0.smithi139.stdout:1/457: dwrite d7/d8/d40/d46/f5b [0,4194304] 0 2024-09-16T00:09:45.758 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:45.757+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:45.763 INFO:tasks.workunit.client.0.smithi139.stdout:0/530: dread d6/d5a/f6f [0,4194304] 0 2024-09-16T00:09:45.765 INFO:tasks.workunit.client.0.smithi139.stdout:2/542: getdents d11/d30/d57/d29/d2b 0 2024-09-16T00:09:45.766 INFO:tasks.workunit.client.0.smithi139.stdout:8/549: dread d17/d1a/f1c [0,4194304] 0 2024-09-16T00:09:45.767 INFO:tasks.workunit.client.0.smithi139.stdout:0/531: dread d6/d5a/f65 [0,4194304] 0 2024-09-16T00:09:45.768 INFO:tasks.workunit.client.0.smithi139.stdout:2/543: truncate d11/d30/d57/f59 4566391 0 2024-09-16T00:09:45.768 INFO:tasks.workunit.client.0.smithi139.stdout:8/550: symlink d17/d1f/d3a/dad/lbf 0 2024-09-16T00:09:45.769 INFO:tasks.workunit.client.0.smithi139.stdout:0/532: read d6/d1c/d41/f6a [100180,2217] 0 2024-09-16T00:09:45.770 INFO:tasks.workunit.client.0.smithi139.stdout:1/458: dread d7/d8/f1a [4194304,4194304] 0 2024-09-16T00:09:45.770 INFO:tasks.workunit.client.0.smithi139.stdout:2/544: creat d11/d45/fb8 x:0 0 0 2024-09-16T00:09:45.771 INFO:tasks.workunit.client.0.smithi139.stdout:8/551: fsync d17/d1a/f1c 0 2024-09-16T00:09:45.773 INFO:tasks.workunit.client.0.smithi139.stdout:0/533: unlink d6/d1c/d45/f47 0 2024-09-16T00:09:45.774 INFO:tasks.workunit.client.0.smithi139.stdout:0/534: truncate d6/d1c/f8c 464046 0 2024-09-16T00:09:45.777 INFO:tasks.workunit.client.0.smithi139.stdout:4/475: dread d12/d2b/f30 [0,4194304] 0 2024-09-16T00:09:45.778 INFO:tasks.workunit.client.0.smithi139.stdout:8/552: write d17/d1f/d3a/db7/f42 [411624,45079] 0 2024-09-16T00:09:45.779 INFO:tasks.workunit.client.0.smithi139.stdout:0/535: mknod d6/d1c/d41/d44/dd/d56/d75/ca5 0 2024-09-16T00:09:45.780 INFO:tasks.workunit.client.0.smithi139.stdout:2/545: write d11/d30/d57/d29/d2b/f2d [4868830,47113] 0 2024-09-16T00:09:45.781 INFO:tasks.workunit.client.0.smithi139.stdout:8/553: rmdir d17/d1a/d57 39 2024-09-16T00:09:45.781 INFO:tasks.workunit.client.0.smithi139.stdout:4/476: mknod d12/d2b/d8c/c9f 0 2024-09-16T00:09:45.783 INFO:tasks.workunit.client.0.smithi139.stdout:8/554: chown d17/d1a/d50/l91 517068218 1 2024-09-16T00:09:45.784 INFO:tasks.workunit.client.0.smithi139.stdout:0/536: creat d6/d1c/d41/d44/dd/d56/d75/fa6 x:0 0 0 2024-09-16T00:09:45.785 INFO:tasks.workunit.client.0.smithi139.stdout:4/477: creat d12/d47/d49/fa0 x:0 0 0 2024-09-16T00:09:45.785 INFO:tasks.workunit.client.0.smithi139.stdout:8/555: chown d17/d1f/d3a/db7/c47 660 1 2024-09-16T00:09:45.785 INFO:tasks.workunit.client.0.smithi139.stdout:1/459: dwrite d7/d8/f21 [0,4194304] 0 2024-09-16T00:09:45.854 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:45.854 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:45.858 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:45.858+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:45.862 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:09:45.863 INFO:tasks.workunit.client.0.smithi139.stdout:1/460: readlink d7/d8/d1f/l55 0 2024-09-16T00:09:45.864 INFO:tasks.workunit.client.0.smithi139.stdout:8/556: unlink d17/d1f/d3a/db7/c3d 0 2024-09-16T00:09:45.866 INFO:tasks.workunit.client.0.smithi139.stdout:5/550: sync 2024-09-16T00:09:45.866 INFO:tasks.workunit.client.0.smithi139.stdout:0/537: truncate d6/d1c/d41/d44/d6b/f6c 759899 0 2024-09-16T00:09:45.866 INFO:tasks.workunit.client.0.smithi139.stdout:2/546: rename d11/d30/d57/d29/d2b/c55 to d11/d30/cb9 0 2024-09-16T00:09:45.869 INFO:tasks.workunit.client.0.smithi139.stdout:2/547: stat d11/f41 0 2024-09-16T00:09:45.870 INFO:tasks.workunit.client.0.smithi139.stdout:1/461: write d7/d8/d27/f34 [867112,48843] 0 2024-09-16T00:09:45.871 INFO:tasks.workunit.client.0.smithi139.stdout:8/557: mknod d17/d1a/d2d/dac/cc0 0 2024-09-16T00:09:45.872 INFO:tasks.workunit.client.0.smithi139.stdout:5/551: creat d1/d13/d3f/fb8 x:0 0 0 2024-09-16T00:09:45.872 INFO:tasks.workunit.client.0.smithi139.stdout:2/548: chown d11/d30/d57/d29/da5 5672 1 2024-09-16T00:09:45.873 INFO:tasks.workunit.client.0.smithi139.stdout:0/538: fsync d6/d1c/d41/d44/d58/f5f 0 2024-09-16T00:09:45.874 INFO:tasks.workunit.client.0.smithi139.stdout:4/478: dwrite d12/d47/f45 [0,4194304] 0 2024-09-16T00:09:45.876 INFO:tasks.workunit.client.0.smithi139.stdout:5/552: read - d1/d21/d49/d87/f91 zero size 2024-09-16T00:09:45.877 INFO:tasks.workunit.client.0.smithi139.stdout:1/462: mkdir d7/d8/d1f/d8b 0 2024-09-16T00:09:45.878 INFO:tasks.workunit.client.0.smithi139.stdout:8/558: unlink d17/d1f/d3a/db7/c4d 0 2024-09-16T00:09:45.881 INFO:tasks.workunit.client.0.smithi139.stdout:5/553: mknod d1/d13/d3b/cb9 0 2024-09-16T00:09:45.882 INFO:tasks.workunit.client.0.smithi139.stdout:4/479: dwrite f2 [4194304,4194304] 0 2024-09-16T00:09:45.883 INFO:tasks.workunit.client.0.smithi139.stdout:8/559: symlink d17/d54/lc1 0 2024-09-16T00:09:45.884 INFO:tasks.workunit.client.0.smithi139.stdout:4/480: read - d12/d80/f9a zero size 2024-09-16T00:09:45.887 INFO:tasks.workunit.client.0.smithi139.stdout:2/549: rename d11/d30/d57/l60 to d11/d30/d57/d29/da5/daa/lba 0 2024-09-16T00:09:45.888 INFO:tasks.workunit.client.0.smithi139.stdout:8/560: mknod d17/d89/cc2 0 2024-09-16T00:09:45.889 INFO:tasks.workunit.client.0.smithi139.stdout:5/554: getdents d1/d13/d3f/d9f/d8a 0 2024-09-16T00:09:45.889 INFO:tasks.workunit.client.0.smithi139.stdout:0/539: getdents d6/d5a/d70/d7b 0 2024-09-16T00:09:45.891 INFO:tasks.workunit.client.0.smithi139.stdout:4/481: dread d12/d4a/f54 [0,4194304] 0 2024-09-16T00:09:45.891 INFO:tasks.workunit.client.0.smithi139.stdout:1/463: dwrite d7/d8/d1f/f76 [0,4194304] 0 2024-09-16T00:09:45.891 INFO:tasks.workunit.client.0.smithi139.stdout:2/550: rmdir d11 39 2024-09-16T00:09:45.893 INFO:tasks.workunit.client.0.smithi139.stdout:5/555: dread - d1/d21/d49/f98 zero size 2024-09-16T00:09:45.893 INFO:tasks.workunit.client.0.smithi139.stdout:1/464: stat d7/f13 0 2024-09-16T00:09:45.894 INFO:tasks.workunit.client.0.smithi139.stdout:8/561: mknod d17/d1a/d50/d86/d9d/daf/cc3 0 2024-09-16T00:09:45.894 INFO:tasks.workunit.client.0.smithi139.stdout:2/551: stat d11/d30/l7d 0 2024-09-16T00:09:45.895 INFO:tasks.workunit.client.0.smithi139.stdout:4/482: chown d12/d4a/d4d/f9d 121 1 2024-09-16T00:09:45.895 INFO:tasks.workunit.client.0.smithi139.stdout:0/540: readlink d6/d1c/d41/d44/dd/l27 0 2024-09-16T00:09:45.896 INFO:tasks.workunit.client.0.smithi139.stdout:8/562: fdatasync d17/d1a/d2d/f3e 0 2024-09-16T00:09:45.896 INFO:tasks.workunit.client.0.smithi139.stdout:4/483: rmdir d12/d4a 39 2024-09-16T00:09:45.897 INFO:tasks.workunit.client.0.smithi139.stdout:2/552: dread - d11/d30/d57/d29/da5/d81/f89 zero size 2024-09-16T00:09:45.898 INFO:tasks.workunit.client.0.smithi139.stdout:1/465: creat d7/d8/d40/d48/d4a/f8c x:0 0 0 2024-09-16T00:09:45.899 INFO:tasks.workunit.client.0.smithi139.stdout:5/556: creat d1/d21/d49/db2/fba x:0 0 0 2024-09-16T00:09:45.900 INFO:tasks.workunit.client.0.smithi139.stdout:0/541: symlink d6/d5a/d81/la7 0 2024-09-16T00:09:45.902 INFO:tasks.workunit.client.0.smithi139.stdout:5/557: rmdir d1/d13 39 2024-09-16T00:09:45.904 INFO:tasks.workunit.client.0.smithi139.stdout:2/553: rename d11/d30/cad to d11/d2f/d6e/cbb 0 2024-09-16T00:09:45.905 INFO:tasks.workunit.client.0.smithi139.stdout:0/542: chown d6/d1c/d41/d44/dd/f11 201 1 2024-09-16T00:09:45.906 INFO:tasks.workunit.client.0.smithi139.stdout:4/484: dwrite d12/d47/d49/d60/f8f [0,4194304] 0 2024-09-16T00:09:45.906 INFO:tasks.workunit.client.0.smithi139.stdout:5/558: write d1/d13/f24 [3977464,97390] 0 2024-09-16T00:09:45.908 INFO:tasks.workunit.client.0.smithi139.stdout:2/554: fsync d11/d30/d57/d29/d4f/d9c/fae 0 2024-09-16T00:09:45.910 INFO:tasks.workunit.client.0.smithi139.stdout:4/485: symlink d12/d2b/la1 0 2024-09-16T00:09:45.910 INFO:tasks.workunit.client.0.smithi139.stdout:0/543: symlink d6/d1c/d41/d44/dd/la8 0 2024-09-16T00:09:45.911 INFO:tasks.workunit.client.0.smithi139.stdout:2/555: write d11/d30/d57/d5d/d73/f79 [1394299,109477] 0 2024-09-16T00:09:45.914 INFO:tasks.workunit.client.0.smithi139.stdout:8/563: dread fb [0,4194304] 0 2024-09-16T00:09:45.915 INFO:tasks.workunit.client.0.smithi139.stdout:4/486: creat d12/d47/d57/d7f/d9e/fa2 x:0 0 0 2024-09-16T00:09:45.916 INFO:tasks.workunit.client.0.smithi139.stdout:5/559: link d1/d21/d49/d53/d6a/d7b/d99/lb4 d1/d21/d49/d87/lbb 0 2024-09-16T00:09:45.917 INFO:tasks.workunit.client.0.smithi139.stdout:0/544: truncate d6/d1c/d41/d44/d58/f60 1448858 0 2024-09-16T00:09:45.919 INFO:tasks.workunit.client.0.smithi139.stdout:4/487: write d12/d4a/f77 [1814730,85687] 0 2024-09-16T00:09:45.920 INFO:tasks.workunit.client.0.smithi139.stdout:0/545: write d6/d1c/d41/d44/d58/f88 [2737501,35443] 0 2024-09-16T00:09:45.925 INFO:tasks.workunit.client.0.smithi139.stdout:2/556: dwrite d11/d30/d57/f59 [0,4194304] 0 2024-09-16T00:09:45.925 INFO:tasks.workunit.client.0.smithi139.stdout:4/488: unlink d12/d2b/f5a 0 2024-09-16T00:09:45.926 INFO:tasks.workunit.client.0.smithi139.stdout:8/564: dwrite d17/d1f/d3a/db7/d40/f49 [8388608,4194304] 0 2024-09-16T00:09:45.927 INFO:tasks.workunit.client.0.smithi139.stdout:4/489: chown d12/d47/d49/d60/f66 140319 1 2024-09-16T00:09:45.928 INFO:tasks.workunit.client.0.smithi139.stdout:4/490: readlink d12/d47/d57/d7f/l98 0 2024-09-16T00:09:45.929 INFO:tasks.workunit.client.0.smithi139.stdout:8/565: symlink d17/d1a/d50/lc4 0 2024-09-16T00:09:45.930 INFO:tasks.workunit.client.0.smithi139.stdout:8/566: readlink d17/d1f/d3a/l9a 0 2024-09-16T00:09:45.934 INFO:tasks.workunit.client.0.smithi139.stdout:4/491: dread d12/d2b/f94 [0,4194304] 0 2024-09-16T00:09:45.934 INFO:tasks.workunit.client.0.smithi139.stdout:5/560: dread d1/d21/d49/f81 [0,4194304] 0 2024-09-16T00:09:45.935 INFO:tasks.workunit.client.0.smithi139.stdout:2/557: write d11/f64 [156157,87744] 0 2024-09-16T00:09:45.936 INFO:tasks.workunit.client.0.smithi139.stdout:4/492: chown d12/c2f 437705 1 2024-09-16T00:09:45.937 INFO:tasks.workunit.client.0.smithi139.stdout:5/561: truncate d1/d13/d3f/d9f/f6c 2211698 0 2024-09-16T00:09:45.938 INFO:tasks.workunit.client.0.smithi139.stdout:4/493: creat d12/d47/d57/d7f/fa3 x:0 0 0 2024-09-16T00:09:45.939 INFO:tasks.workunit.client.0.smithi139.stdout:5/562: symlink d1/d13/d3f/d9f/d73/d79/da5/lbc 0 2024-09-16T00:09:45.941 INFO:tasks.workunit.client.0.smithi139.stdout:5/563: truncate d1/d21/d49/f7f 1823131 0 2024-09-16T00:09:45.941 INFO:tasks.workunit.client.0.smithi139.stdout:8/567: truncate d17/d1f/f81 1364610 0 2024-09-16T00:09:45.942 INFO:tasks.workunit.client.0.smithi139.stdout:8/568: chown d17/d1a/d2d/dac 6356 1 2024-09-16T00:09:45.943 INFO:tasks.workunit.client.0.smithi139.stdout:4/494: dread d12/d47/d49/d60/f8f [0,4194304] 0 2024-09-16T00:09:45.943 INFO:tasks.workunit.client.0.smithi139.stdout:8/569: write d17/d1f/d3a/db7/f2b [1522603,48560] 0 2024-09-16T00:09:45.943 INFO:tasks.workunit.client.0.smithi139.stdout:5/564: symlink d1/d21/d49/d53/d6a/d7b/lbd 0 2024-09-16T00:09:45.946 INFO:tasks.workunit.client.0.smithi139.stdout:5/565: unlink d1/d13/d3f/d9f/d73/d79/d8d/f96 0 2024-09-16T00:09:45.948 INFO:tasks.workunit.client.0.smithi139.stdout:4/495: dread d12/d4a/d4d/d69/f72 [0,4194304] 0 2024-09-16T00:09:45.948 INFO:tasks.workunit.client.0.smithi139.stdout:8/570: dread d17/d1f/d3a/db7/d40/f41 [0,4194304] 0 2024-09-16T00:09:45.948 INFO:tasks.workunit.client.0.smithi139.stdout:5/566: symlink d1/d13/d3f/d88/lbe 0 2024-09-16T00:09:45.950 INFO:tasks.workunit.client.0.smithi139.stdout:4/496: symlink d12/d2b/d8c/la4 0 2024-09-16T00:09:45.953 INFO:tasks.workunit.client.0.smithi139.stdout:4/497: creat d12/d47/fa5 x:0 0 0 2024-09-16T00:09:45.954 INFO:tasks.workunit.client.0.smithi139.stdout:5/567: truncate d1/d21/d49/d53/d6a/fa7 548751 0 2024-09-16T00:09:45.957 INFO:tasks.workunit.client.0.smithi139.stdout:5/568: symlink d1/d21/d49/d53/d6a/d7b/da1/db7/lbf 0 2024-09-16T00:09:45.959 INFO:tasks.workunit.client.0.smithi139.stdout:5/569: write d1/d21/d49/f98 [255152,86422] 0 2024-09-16T00:09:45.959 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:45.958+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:45.960 INFO:tasks.workunit.client.0.smithi139.stdout:4/498: write d12/d4a/d4d/f55 [213759,113485] 0 2024-09-16T00:09:45.963 INFO:tasks.workunit.client.0.smithi139.stdout:4/499: creat d12/d47/fa6 x:0 0 0 2024-09-16T00:09:45.964 INFO:tasks.workunit.client.0.smithi139.stdout:8/571: dwrite d17/d1a/fa5 [0,4194304] 0 2024-09-16T00:09:45.970 INFO:tasks.workunit.client.0.smithi139.stdout:4/500: dwrite d12/d4a/d4d/d69/f8a [4194304,4194304] 0 2024-09-16T00:09:45.973 INFO:tasks.workunit.client.0.smithi139.stdout:4/501: mknod d12/d47/ca7 0 2024-09-16T00:09:45.973 INFO:tasks.workunit.client.0.smithi139.stdout:8/572: truncate d17/d1a/f53 570396 0 2024-09-16T00:09:45.974 INFO:tasks.workunit.client.0.smithi139.stdout:4/502: stat c11 0 2024-09-16T00:09:45.974 INFO:tasks.workunit.client.0.smithi139.stdout:8/573: dread - d17/d1a/f5e zero size 2024-09-16T00:09:45.977 INFO:tasks.workunit.client.0.smithi139.stdout:5/570: dread d1/d13/d3b/f44 [0,4194304] 0 2024-09-16T00:09:45.981 INFO:tasks.workunit.client.0.smithi139.stdout:8/574: write d17/d1f/f25 [261957,104710] 0 2024-09-16T00:09:45.982 INFO:tasks.workunit.client.0.smithi139.stdout:4/503: dwrite d12/d47/f6c [0,4194304] 0 2024-09-16T00:09:45.982 INFO:tasks.workunit.client.0.smithi139.stdout:5/571: write d1/f4a [4836374,38394] 0 2024-09-16T00:09:45.983 INFO:tasks.workunit.client.0.smithi139.stdout:8/575: write f4 [3054698,25776] 0 2024-09-16T00:09:45.986 INFO:tasks.workunit.client.0.smithi139.stdout:8/576: fdatasync d17/d1f/d3a/dad/fb5 0 2024-09-16T00:09:45.990 INFO:tasks.workunit.client.0.smithi139.stdout:8/577: truncate d17/d1a/f1c 3098227 0 2024-09-16T00:09:45.993 INFO:tasks.workunit.client.0.smithi139.stdout:5/572: rename d1/d21/d49/d53/d6a/d7b/d99/lb4 to d1/d21/d49/d87/lc0 0 2024-09-16T00:09:45.993 INFO:tasks.workunit.client.0.smithi139.stdout:4/504: dwrite d12/d4a/d4d/f81 [0,4194304] 0 2024-09-16T00:09:45.999 INFO:tasks.workunit.client.0.smithi139.stdout:5/573: rmdir d1/d13/d95 0 2024-09-16T00:09:46.003 INFO:tasks.workunit.client.0.smithi139.stdout:5/574: read d1/d13/d3f/d9f/d73/d79/d8d/f9e [1013068,54026] 0 2024-09-16T00:09:46.003 INFO:tasks.workunit.client.0.smithi139.stdout:8/578: dwrite d17/d1a/d57/f7b [0,4194304] 0 2024-09-16T00:09:46.004 INFO:tasks.workunit.client.0.smithi139.stdout:4/505: dread d12/d4a/f63 [0,4194304] 0 2024-09-16T00:09:46.006 INFO:tasks.workunit.client.0.smithi139.stdout:4/506: write d12/d47/d49/d60/f8f [4777203,97654] 0 2024-09-16T00:09:46.007 INFO:tasks.workunit.client.0.smithi139.stdout:5/575: rename d1/d21/d49/d53/d6a/d7b/da1/db7/lbf to d1/d13/d3f/d9f/lc1 0 2024-09-16T00:09:46.008 INFO:tasks.workunit.client.0.smithi139.stdout:4/507: mkdir d12/d47/d57/d7f/d9e/da8 0 2024-09-16T00:09:46.009 INFO:tasks.workunit.client.0.smithi139.stdout:4/508: write d12/d47/d49/f53 [215739,111436] 0 2024-09-16T00:09:46.013 INFO:tasks.workunit.client.0.smithi139.stdout:5/576: getdents d1/d21/d49/d53 0 2024-09-16T00:09:46.014 INFO:tasks.workunit.client.0.smithi139.stdout:8/579: write f7 [1264547,25500] 0 2024-09-16T00:09:46.015 INFO:tasks.workunit.client.0.smithi139.stdout:5/577: mkdir d1/d13/d3f/d5e/dc2 0 2024-09-16T00:09:46.017 INFO:tasks.workunit.client.0.smithi139.stdout:8/580: rmdir d17/d1f/d3a/db7/d40/d4f 39 2024-09-16T00:09:46.017 INFO:tasks.workunit.client.0.smithi139.stdout:4/509: dwrite d12/d4a/d4d/f86 [0,4194304] 0 2024-09-16T00:09:46.018 INFO:tasks.workunit.client.0.smithi139.stdout:8/581: write f7 [1522752,30479] 0 2024-09-16T00:09:46.020 INFO:tasks.workunit.client.0.smithi139.stdout:4/510: mkdir d12/d4a/d4d/da9 0 2024-09-16T00:09:46.021 INFO:tasks.workunit.client.0.smithi139.stdout:8/582: creat d17/d1f/d3a/dad/fc5 x:0 0 0 2024-09-16T00:09:46.023 INFO:tasks.workunit.client.0.smithi139.stdout:4/511: write d12/d2b/f35 [2306264,51400] 0 2024-09-16T00:09:46.025 INFO:tasks.workunit.client.0.smithi139.stdout:5/578: dwrite d1/f1a [0,4194304] 0 2024-09-16T00:09:46.026 INFO:tasks.workunit.client.0.smithi139.stdout:4/512: write d12/d4a/d4d/d69/f72 [3612429,92900] 0 2024-09-16T00:09:46.027 INFO:tasks.workunit.client.0.smithi139.stdout:4/513: fdatasync d12/d47/fa5 0 2024-09-16T00:09:46.028 INFO:tasks.workunit.client.0.smithi139.stdout:8/583: rename d17/d1a/d50/d86/d9d/c9e to d17/d1f/d3a/db7/cc6 0 2024-09-16T00:09:46.028 INFO:tasks.workunit.client.0.smithi139.stdout:4/514: chown d12/d47/l56 1763785984 1 2024-09-16T00:09:46.029 INFO:tasks.workunit.client.0.smithi139.stdout:5/579: mkdir d1/dc3 0 2024-09-16T00:09:46.032 INFO:tasks.workunit.client.0.smithi139.stdout:8/584: creat d17/d1a/d50/d86/d9d/daf/fc7 x:0 0 0 2024-09-16T00:09:46.032 INFO:tasks.workunit.client.0.smithi139.stdout:5/580: mknod d1/d21/d49/d53/d6a/d7b/cc4 0 2024-09-16T00:09:46.032 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:46.032 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:46.034 INFO:tasks.workunit.client.0.smithi139.stdout:8/585: fsync d17/d1f/d3a/db7/f3b 0 2024-09-16T00:09:46.035 INFO:tasks.workunit.client.0.smithi139.stdout:4/515: dwrite d12/d4a/d4d/f86 [0,4194304] 0 2024-09-16T00:09:46.036 INFO:tasks.workunit.client.0.smithi139.stdout:4/516: fsync d12/d4a/d4d/f9d 0 2024-09-16T00:09:46.040 INFO:tasks.workunit.client.0.smithi139.stdout:8/586: fsync d17/f4a 0 2024-09-16T00:09:46.040 INFO:tasks.workunit.client.0.smithi139.stdout:4/517: getdents d12/d4a/d4d/da9 0 2024-09-16T00:09:46.041 INFO:tasks.workunit.client.0.smithi139.stdout:7/477: sync 2024-09-16T00:09:46.041 INFO:tasks.workunit.client.0.smithi139.stdout:6/348: sync 2024-09-16T00:09:46.041 INFO:tasks.workunit.client.0.smithi139.stdout:9/627: sync 2024-09-16T00:09:46.041 INFO:tasks.workunit.client.0.smithi139.stdout:3/456: sync 2024-09-16T00:09:46.042 INFO:tasks.workunit.client.0.smithi139.stdout:6/349: fdatasync d8/d20/f4a 0 2024-09-16T00:09:46.044 INFO:tasks.workunit.client.0.smithi139.stdout:4/518: creat d12/d4a/d4d/da9/faa x:0 0 0 2024-09-16T00:09:46.046 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:09:46.047 INFO:tasks.workunit.client.0.smithi139.stdout:8/587: creat d17/d1f/d3a/db7/d40/fc8 x:0 0 0 2024-09-16T00:09:46.047 INFO:tasks.workunit.client.0.smithi139.stdout:5/581: truncate d1/d21/d49/f7f 312045 0 2024-09-16T00:09:46.049 INFO:tasks.workunit.client.0.smithi139.stdout:8/588: chown d17/d1a/d57/l7e 42 1 2024-09-16T00:09:46.049 INFO:tasks.workunit.client.0.smithi139.stdout:6/350: creat d8/d56/f79 x:0 0 0 2024-09-16T00:09:46.049 INFO:tasks.workunit.client.0.smithi139.stdout:7/478: mkdir d5/d16/d22/d25/d2d/da4 0 2024-09-16T00:09:46.050 INFO:tasks.workunit.client.0.smithi139.stdout:8/589: readlink d17/d1a/d50/l91 0 2024-09-16T00:09:46.050 INFO:tasks.workunit.client.0.smithi139.stdout:9/628: creat d19/fd5 x:0 0 0 2024-09-16T00:09:46.052 INFO:tasks.workunit.client.0.smithi139.stdout:6/351: creat d8/d56/f7a x:0 0 0 2024-09-16T00:09:46.052 INFO:tasks.workunit.client.0.smithi139.stdout:4/519: dwrite d12/d80/f9a [0,4194304] 0 2024-09-16T00:09:46.053 INFO:tasks.workunit.client.0.smithi139.stdout:7/479: creat d5/d26/d42/fa5 x:0 0 0 2024-09-16T00:09:46.053 INFO:tasks.workunit.client.0.smithi139.stdout:8/590: write d17/d1a/d2d/f6b [4370630,5321] 0 2024-09-16T00:09:46.060 INFO:tasks.workunit.client.0.smithi139.stdout:9/629: rename d19/d1a/f39 to d19/d1a/d20/d23/d2d/d35/d42/dc8/fd6 0 2024-09-16T00:09:46.060 INFO:tasks.workunit.client.0.smithi139.stdout:5/582: dread d1/d21/d49/f7f [0,4194304] 0 2024-09-16T00:09:46.060 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:46.059+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:46.061 INFO:tasks.workunit.client.0.smithi139.stdout:3/457: truncate d5/f16 2948411 0 2024-09-16T00:09:46.062 INFO:tasks.workunit.client.0.smithi139.stdout:6/352: link d8/d56/f7a d8/f7b 0 2024-09-16T00:09:46.065 INFO:tasks.workunit.client.0.smithi139.stdout:7/480: dwrite d5/d37/f86 [0,4194304] 0 2024-09-16T00:09:46.066 INFO:tasks.workunit.client.0.smithi139.stdout:3/458: write d5/d7/d23/d27/d2f/d30/d4c/d6d/d76/f98 [73951,68298] 0 2024-09-16T00:09:46.066 INFO:tasks.workunit.client.0.smithi139.stdout:6/353: chown d8/db/d26/d45/f43 4 1 2024-09-16T00:09:46.069 INFO:tasks.workunit.client.0.smithi139.stdout:7/481: rmdir d5/d16/d22/d25 39 2024-09-16T00:09:46.071 INFO:tasks.workunit.client.0.smithi139.stdout:8/591: getdents d17/d1f/d3a/db7/d40/d4f 0 2024-09-16T00:09:46.072 INFO:tasks.workunit.client.0.smithi139.stdout:5/583: rename d1/f16 to d1/fc5 0 2024-09-16T00:09:46.074 INFO:tasks.workunit.client.0.smithi139.stdout:6/354: write d8/d20/d27/f18 [4375622,49256] 0 2024-09-16T00:09:46.074 INFO:tasks.workunit.client.0.smithi139.stdout:9/630: link d19/d1a/d20/d23/cd4 d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/cd7 0 2024-09-16T00:09:46.074 INFO:tasks.workunit.client.0.smithi139.stdout:7/482: dread d5/fc [4194304,4194304] 0 2024-09-16T00:09:46.075 INFO:tasks.workunit.client.0.smithi139.stdout:8/592: symlink d17/lc9 0 2024-09-16T00:09:46.075 INFO:tasks.workunit.client.0.smithi139.stdout:5/584: symlink d1/d13/d3f/d9f/d73/d79/lc6 0 2024-09-16T00:09:46.076 INFO:tasks.workunit.client.0.smithi139.stdout:6/355: chown d8/db/d26/f61 261711 1 2024-09-16T00:09:46.077 INFO:tasks.workunit.client.0.smithi139.stdout:8/593: fdatasync d17/d1f/d3a/f45 0 2024-09-16T00:09:46.078 INFO:tasks.workunit.client.0.smithi139.stdout:7/483: creat d5/fa6 x:0 0 0 2024-09-16T00:09:46.079 INFO:tasks.workunit.client.0.smithi139.stdout:5/585: creat d1/d13/d3b/d3e/fc7 x:0 0 0 2024-09-16T00:09:46.080 INFO:tasks.workunit.client.0.smithi139.stdout:6/356: write d8/fe [1013090,5175] 0 2024-09-16T00:09:46.083 INFO:tasks.workunit.client.0.smithi139.stdout:6/357: rmdir d8/d20/d4d 39 2024-09-16T00:09:46.083 INFO:tasks.workunit.client.0.smithi139.stdout:7/484: chown d5/c41 1447733505 1 2024-09-16T00:09:46.084 INFO:tasks.workunit.client.0.smithi139.stdout:9/631: dwrite d19/d1a/faf [0,4194304] 0 2024-09-16T00:09:46.085 INFO:tasks.workunit.client.0.smithi139.stdout:8/594: getdents d17/d1f/d88 0 2024-09-16T00:09:46.087 INFO:tasks.workunit.client.0.smithi139.stdout:6/358: mknod d8/d20/c7c 0 2024-09-16T00:09:46.087 INFO:tasks.workunit.client.0.smithi139.stdout:8/595: stat d17/d1f 0 2024-09-16T00:09:46.088 INFO:tasks.workunit.client.0.smithi139.stdout:6/359: fsync d8/d56/f79 0 2024-09-16T00:09:46.089 INFO:tasks.workunit.client.0.smithi139.stdout:7/485: unlink d5/d16/d22/d25/d2d/d81/l92 0 2024-09-16T00:09:46.090 INFO:tasks.workunit.client.0.smithi139.stdout:9/632: dread d19/d1a/d20/d23/dca/d71/d32/d4e/f78 [0,4194304] 0 2024-09-16T00:09:46.091 INFO:tasks.workunit.client.0.smithi139.stdout:8/596: mknod d17/d54/cca 0 2024-09-16T00:09:46.092 INFO:tasks.workunit.client.0.smithi139.stdout:7/486: mkdir d5/d26/d46/d6b/da7 0 2024-09-16T00:09:46.093 INFO:tasks.workunit.client.0.smithi139.stdout:5/586: truncate d1/d21/d49/d53/d6a/f9b 3064196 0 2024-09-16T00:09:46.093 INFO:tasks.workunit.client.0.smithi139.stdout:8/597: read d17/d1f/d3a/db7/d40/f4b [758728,4103] 0 2024-09-16T00:09:46.094 INFO:tasks.workunit.client.0.smithi139.stdout:7/487: fsync d5/d26/d46/f68 0 2024-09-16T00:09:46.094 INFO:tasks.workunit.client.0.smithi139.stdout:9/633: symlink d19/d1a/d20/d23/dca/d71/da3/ld8 0 2024-09-16T00:09:46.095 INFO:tasks.workunit.client.0.smithi139.stdout:5/587: symlink d1/dc3/lc8 0 2024-09-16T00:09:46.096 INFO:tasks.workunit.client.0.smithi139.stdout:6/360: dread f3 [0,4194304] 0 2024-09-16T00:09:46.097 INFO:tasks.workunit.client.0.smithi139.stdout:9/634: fdatasync d19/d1a/d20/d23/dca/d71/fb0 0 2024-09-16T00:09:46.098 INFO:tasks.workunit.client.0.smithi139.stdout:8/598: unlink d17/d1f/d3a/db7/d40/f49 0 2024-09-16T00:09:46.101 INFO:tasks.workunit.client.0.smithi139.stdout:5/588: unlink d1/d13/d3f/d5e/l67 0 2024-09-16T00:09:46.104 INFO:tasks.workunit.client.0.smithi139.stdout:7/488: creat d5/d16/d22/d25/d2d/fa8 x:0 0 0 2024-09-16T00:09:46.105 INFO:tasks.workunit.client.0.smithi139.stdout:9/635: dwrite d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/fad [0,4194304] 0 2024-09-16T00:09:46.106 INFO:tasks.workunit.client.0.smithi139.stdout:5/589: mknod d1/d21/d49/d53/d6a/d7b/d78/cc9 0 2024-09-16T00:09:46.110 INFO:tasks.workunit.client.0.smithi139.stdout:8/599: rename d17/d1f/c27 to d17/d1a/d2d/ccb 0 2024-09-16T00:09:46.111 INFO:tasks.workunit.client.0.smithi139.stdout:6/361: dwrite d8/d20/f62 [0,4194304] 0 2024-09-16T00:09:46.113 INFO:tasks.workunit.client.0.smithi139.stdout:8/600: read d17/d54/f69 [2982833,106849] 0 2024-09-16T00:09:46.113 INFO:tasks.workunit.client.0.smithi139.stdout:9/636: write d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/fad [2338349,60751] 0 2024-09-16T00:09:46.119 INFO:tasks.workunit.client.0.smithi139.stdout:9/637: chown d19/d1a/d20/d23/dca/f7c 6392 1 2024-09-16T00:09:46.121 INFO:tasks.workunit.client.0.smithi139.stdout:7/489: dwrite d5/d26/d5d/f95 [0,4194304] 0 2024-09-16T00:09:46.121 INFO:tasks.workunit.client.0.smithi139.stdout:8/601: read d17/d1f/d3a/db7/d40/d4f/f65 [561080,66322] 0 2024-09-16T00:09:46.122 INFO:tasks.workunit.client.0.smithi139.stdout:6/362: write d8/d29/f49 [452392,55541] 0 2024-09-16T00:09:46.123 INFO:tasks.workunit.client.0.smithi139.stdout:5/590: dwrite d1/d13/d3f/d9f/d30/d33/d43/f9c [0,4194304] 0 2024-09-16T00:09:46.128 INFO:tasks.workunit.client.0.smithi139.stdout:9/638: mkdir d19/d1a/d20/d23/dca/d9f/dd9 0 2024-09-16T00:09:46.130 INFO:tasks.workunit.client.0.smithi139.stdout:5/591: mkdir d1/d21/d49/d53/d6a/d7b/d78/dca 0 2024-09-16T00:09:46.130 INFO:tasks.workunit.client.0.smithi139.stdout:7/490: mknod d5/d16/d6d/d7f/ca9 0 2024-09-16T00:09:46.131 INFO:tasks.workunit.client.0.smithi139.stdout:6/363: mknod d8/d20/c7d 0 2024-09-16T00:09:46.133 INFO:tasks.workunit.client.0.smithi139.stdout:5/592: mknod d1/d21/d49/d53/d6a/d7b/da1/ccb 0 2024-09-16T00:09:46.134 INFO:tasks.workunit.client.0.smithi139.stdout:9/639: truncate fa 1590803 0 2024-09-16T00:09:46.140 INFO:tasks.workunit.client.0.smithi139.stdout:6/364: dwrite d8/d29/f3d [0,4194304] 0 2024-09-16T00:09:46.140 INFO:tasks.workunit.client.0.smithi139.stdout:7/491: link d5/d16/c28 d5/d16/d22/caa 0 2024-09-16T00:09:46.144 INFO:tasks.workunit.client.0.smithi139.stdout:7/492: unlink d5/d26/d46/l8d 0 2024-09-16T00:09:46.144 INFO:tasks.workunit.client.0.smithi139.stdout:9/640: dwrite d19/f1d [0,4194304] 0 2024-09-16T00:09:46.145 INFO:tasks.workunit.client.0.smithi139.stdout:6/365: truncate d8/db/d26/d45/d65/f37 1388583 0 2024-09-16T00:09:46.146 INFO:tasks.workunit.client.0.smithi139.stdout:5/593: dwrite d1/d21/d49/d53/d6a/d7b/d78/f8e [0,4194304] 0 2024-09-16T00:09:46.147 INFO:tasks.workunit.client.0.smithi139.stdout:6/366: fdatasync d8/d20/d27/f58 0 2024-09-16T00:09:46.148 INFO:tasks.workunit.client.0.smithi139.stdout:6/367: readlink d8/db/d26/d28/l6c 0 2024-09-16T00:09:46.150 INFO:tasks.workunit.client.0.smithi139.stdout:5/594: write d1/fc5 [12266933,36518] 0 2024-09-16T00:09:46.151 INFO:tasks.workunit.client.0.smithi139.stdout:7/493: mkdir d5/d16/d22/d25/d2d/dab 0 2024-09-16T00:09:46.152 INFO:tasks.workunit.client.0.smithi139.stdout:6/368: creat d8/db/d26/d28/d6d/f7e x:0 0 0 2024-09-16T00:09:46.153 INFO:tasks.workunit.client.0.smithi139.stdout:9/641: mknod d19/d1a/d20/d23/dca/d71/d32/d79/cda 0 2024-09-16T00:09:46.154 INFO:tasks.workunit.client.0.smithi139.stdout:5/595: stat d1/d13/d3f/d5e/la2 0 2024-09-16T00:09:46.157 INFO:tasks.workunit.client.0.smithi139.stdout:5/596: write d1/d21/d49/d53/d6a/d7b/d99/fae [5060,131039] 0 2024-09-16T00:09:46.157 INFO:tasks.workunit.client.0.smithi139.stdout:9/642: unlink d19/d1a/f21 0 2024-09-16T00:09:46.159 INFO:tasks.workunit.client.0.smithi139.stdout:7/494: truncate d5/f1f 5196300 0 2024-09-16T00:09:46.161 INFO:tasks.workunit.client.0.smithi139.stdout:5/597: symlink d1/d13/d22/lcc 0 2024-09-16T00:09:46.161 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:46.160+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:46.162 INFO:tasks.workunit.client.0.smithi139.stdout:6/369: dwrite d8/db/d26/d45/d59/f6b [0,4194304] 0 2024-09-16T00:09:46.162 INFO:tasks.workunit.client.0.smithi139.stdout:5/598: chown d1/d21/c66 20 1 2024-09-16T00:09:46.162 INFO:tasks.workunit.client.0.smithi139.stdout:9/643: creat d19/d1a/d20/d89/d93/fdb x:0 0 0 2024-09-16T00:09:46.164 INFO:tasks.workunit.client.0.smithi139.stdout:9/644: read d19/d1a/f50 [907562,23604] 0 2024-09-16T00:09:46.165 INFO:tasks.workunit.client.0.smithi139.stdout:7/495: unlink d5/d16/d6d/d7f/f93 0 2024-09-16T00:09:46.165 INFO:tasks.workunit.client.0.smithi139.stdout:9/645: chown d19/d1a/d20/d23/dca/d71/d32/d4e/d67 218378 1 2024-09-16T00:09:46.166 INFO:tasks.workunit.client.0.smithi139.stdout:6/370: symlink d8/db/d26/l7f 0 2024-09-16T00:09:46.167 INFO:tasks.workunit.client.0.smithi139.stdout:6/371: fsync d8/d20/f62 0 2024-09-16T00:09:46.167 INFO:tasks.workunit.client.0.smithi139.stdout:9/646: read d19/d1a/d20/d23/dca/d71/d32/d79/f91 [145953,6923] 0 2024-09-16T00:09:46.168 INFO:tasks.workunit.client.0.smithi139.stdout:5/599: rename d1/d13/d3b/cb9 to d1/d21/d49/d53/d6a/d7b/d78/dca/ccd 0 2024-09-16T00:09:46.170 INFO:tasks.workunit.client.0.smithi139.stdout:5/600: truncate d1/d21/d49/d87/f91 907989 0 2024-09-16T00:09:46.172 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /var/log/ceph/exp.2.17.6 2024-09-16T00:09:46.176 INFO:tasks.workunit.client.0.smithi139.stdout:7/496: rename d5/d16/d22/caa to d5/d16/d6d/d7f/cac 0 2024-09-16T00:09:46.179 INFO:tasks.workunit.client.0.smithi139.stdout:5/601: creat d1/d13/d3f/d5e/dc2/fce x:0 0 0 2024-09-16T00:09:46.180 INFO:tasks.workunit.client.0.smithi139.stdout:7/497: dread - d5/d16/d22/d25/d63/f91 zero size 2024-09-16T00:09:46.182 INFO:tasks.workunit.client.0.smithi139.stdout:6/372: truncate d8/db/f2c 3089819 0 2024-09-16T00:09:46.182 INFO:tasks.workunit.client.0.smithi139.stdout:7/498: chown d5/d16/d22/d25/d2d/d50/d44/d55/d90 37929304 1 2024-09-16T00:09:46.184 INFO:tasks.workunit.client.0.smithi139.stdout:7/499: read d5/d26/d5d/f95 [3720225,37673] 0 2024-09-16T00:09:46.186 INFO:tasks.workunit.client.0.smithi139.stdout:5/602: getdents d1/d21/d49/d53/d6a/d7b/d78 0 2024-09-16T00:09:46.186 INFO:tasks.workunit.client.0.smithi139.stdout:9/647: dwrite d19/d1a/d20/d89/f9d [0,4194304] 0 2024-09-16T00:09:46.186 INFO:tasks.workunit.client.0.smithi139.stdout:7/500: chown d5/d26/d42/c7d 111142350 1 2024-09-16T00:09:46.187 INFO:tasks.workunit.client.0.smithi139.stdout:6/373: dread d8/f39 [0,4194304] 0 2024-09-16T00:09:46.192 INFO:tasks.workunit.client.0.smithi139.stdout:5/603: unlink d1/d21/d49/d53/d6a/d7b/d78/cc9 0 2024-09-16T00:09:46.193 INFO:tasks.workunit.client.0.smithi139.stdout:5/604: fdatasync d1/d13/d3f/d9f/f6c 0 2024-09-16T00:09:46.196 INFO:tasks.workunit.client.0.smithi139.stdout:6/374: getdents d8/d20 0 2024-09-16T00:09:46.196 INFO:tasks.workunit.client.0.smithi139.stdout:9/648: write fa [2081919,94926] 0 2024-09-16T00:09:46.197 INFO:tasks.workunit.client.0.smithi139.stdout:5/605: creat d1/d21/d49/d53/d6a/d7b/d78/dca/fcf x:0 0 0 2024-09-16T00:09:46.198 INFO:tasks.workunit.client.0.smithi139.stdout:6/375: mknod d8/db/d26/d28/d6d/c80 0 2024-09-16T00:09:46.199 INFO:tasks.workunit.client.0.smithi139.stdout:9/649: mknod d19/d1a/d20/d23/d2d/d35/d60/cdc 0 2024-09-16T00:09:46.204 INFO:tasks.workunit.client.0.smithi139.stdout:9/650: creat d19/d1a/d20/d23/d2d/d35/d42/d3a/dcb/fdd x:0 0 0 2024-09-16T00:09:46.205 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:46.205 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:46.206 INFO:tasks.workunit.client.0.smithi139.stdout:7/501: dread d5/f1f [0,4194304] 0 2024-09-16T00:09:46.206 INFO:tasks.workunit.client.0.smithi139.stdout:5/606: dwrite d1/d13/d3f/d9f/fa4 [0,4194304] 0 2024-09-16T00:09:46.208 INFO:tasks.workunit.client.0.smithi139.stdout:6/376: dwrite d8/d56/f79 [0,4194304] 0 2024-09-16T00:09:46.209 INFO:tasks.workunit.client.0.smithi139.stdout:9/651: symlink d19/d1a/d20/lde 0 2024-09-16T00:09:46.211 INFO:tasks.workunit.client.0.smithi139.stdout:7/502: chown d5/d26/d42/f66 1669 1 2024-09-16T00:09:46.214 INFO:tasks.workunit.client.0.smithi139.stdout:5/607: mkdir d1/d13/d3f/d9f/d73/dd0 0 2024-09-16T00:09:46.214 INFO:tasks.workunit.client.0.smithi139.stdout:7/503: stat d5/d16/d22/d25/f31 0 2024-09-16T00:09:46.216 INFO:tasks.workunit.client.0.smithi139.stdout:6/377: creat d8/db/f81 x:0 0 0 2024-09-16T00:09:46.216 INFO:tasks.workunit.client.0.smithi139.stdout:7/504: stat d5/d26/d5d/f8e 0 2024-09-16T00:09:46.217 INFO:tasks.workunit.client.0.smithi139.stdout:5/608: write d1/d21/d49/f98 [403409,103810] 0 2024-09-16T00:09:46.219 INFO:tasks.workunit.client.0.smithi139.stdout:6/378: chown d8/db/d26/d28/d5a 92117365 1 2024-09-16T00:09:46.219 INFO:tasks.workunit.client.0.smithi139.stdout:7/505: truncate d5/d16/f34 4954562 0 2024-09-16T00:09:46.223 INFO:tasks.workunit.client.0.smithi139.stdout:5/609: creat d1/d21/d49/fd1 x:0 0 0 2024-09-16T00:09:46.224 INFO:tasks.workunit.client.0.smithi139.stdout:1/466: sync 2024-09-16T00:09:46.226 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:09:46.227 INFO:tasks.workunit.client.0.smithi139.stdout:9/652: dwrite d19/d86/f5a [0,4194304] 0 2024-09-16T00:09:46.228 INFO:tasks.workunit.client.0.smithi139.stdout:1/467: fsync d7/d8/d27/f2c 0 2024-09-16T00:09:46.228 INFO:tasks.workunit.client.0.smithi139.stdout:9/653: write d19/d1a/d20/d23/dca/d71/da3/fc9 [550411,58573] 0 2024-09-16T00:09:46.228 INFO:tasks.workunit.client.0.smithi139.stdout:7/506: dread d5/f1f [0,4194304] 0 2024-09-16T00:09:46.231 INFO:tasks.workunit.client.0.smithi139.stdout:6/379: link d8/db/l78 d8/db/d26/d28/l82 0 2024-09-16T00:09:46.231 INFO:tasks.workunit.client.0.smithi139.stdout:1/468: mkdir d7/d8/d40/d46/d8d 0 2024-09-16T00:09:46.233 INFO:tasks.workunit.client.0.smithi139.stdout:9/654: write d19/d1a/d20/d23/d2d/d35/d60/fb3 [642281,32563] 0 2024-09-16T00:09:46.234 INFO:tasks.workunit.client.0.smithi139.stdout:9/655: fdatasync d19/d1a/d20/d89/d93/fbb 0 2024-09-16T00:09:46.234 INFO:tasks.workunit.client.0.smithi139.stdout:1/469: dread d7/d8/d27/d2f/f5c [0,4194304] 0 2024-09-16T00:09:46.236 INFO:tasks.workunit.client.0.smithi139.stdout:9/656: mknod d19/d1a/d20/d23/d2d/d9a/cdf 0 2024-09-16T00:09:46.286 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:46.286+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:46.328 INFO:tasks.workunit.client.0.smithi139.stdout:7/507: write d5/d26/d42/f66 [8888848,47959] 0 2024-09-16T00:09:46.328 INFO:tasks.workunit.client.0.smithi139.stdout:9/657: creat d19/d86/fe0 x:0 0 0 2024-09-16T00:09:46.330 INFO:tasks.workunit.client.0.smithi139.stdout:1/470: dread d7/d8/d40/d47/f6a [0,4194304] 0 2024-09-16T00:09:46.330 INFO:tasks.workunit.client.0.smithi139.stdout:6/380: getdents d8/d1f 0 2024-09-16T00:09:46.332 INFO:tasks.workunit.client.0.smithi139.stdout:1/471: read - d7/d8/d40/d48/d4a/f8c zero size 2024-09-16T00:09:46.332 INFO:tasks.workunit.client.0.smithi139.stdout:7/508: rename d5/f6 to d5/d26/d74/fad 0 2024-09-16T00:09:46.333 INFO:tasks.workunit.client.0.smithi139.stdout:5/610: dwrite d1/d13/d3b/f44 [0,4194304] 0 2024-09-16T00:09:46.334 INFO:tasks.workunit.client.0.smithi139.stdout:7/509: readlink d5/d16/d6d/l73 0 2024-09-16T00:09:46.334 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:46.334 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:46.335 INFO:tasks.workunit.client.0.smithi139.stdout:5/611: write d1/d13/d3b/f47 [3970976,64877] 0 2024-09-16T00:09:46.336 INFO:tasks.workunit.client.0.smithi139.stdout:6/381: rename d8/db to d8/d29/d83 0 2024-09-16T00:09:46.338 INFO:tasks.workunit.client.0.smithi139.stdout:6/382: chown d8/f6f 166 1 2024-09-16T00:09:46.340 INFO:tasks.workunit.client.0.smithi139.stdout:7/510: unlink d5/d16/d22/d25/d2d/l45 0 2024-09-16T00:09:46.343 INFO:tasks.workunit.client.0.smithi139.stdout:6/383: symlink d8/d56/l84 0 2024-09-16T00:09:46.345 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:09:46.346 INFO:tasks.workunit.client.0.smithi139.stdout:7/511: mkdir d5/d26/d42/dae 0 2024-09-16T00:09:46.346 INFO:tasks.workunit.client.0.smithi139.stdout:6/384: mkdir d8/d29/d83/d26/d45/d85 0 2024-09-16T00:09:46.347 INFO:tasks.workunit.client.0.smithi139.stdout:5/612: dread d1/d13/f24 [0,4194304] 0 2024-09-16T00:09:46.348 INFO:tasks.workunit.client.0.smithi139.stdout:7/512: creat d5/d16/d22/d25/faf x:0 0 0 2024-09-16T00:09:46.351 INFO:tasks.workunit.client.0.smithi139.stdout:7/513: rename d5/d37/c61 to d5/d16/d22/d25/d2d/da4/cb0 0 2024-09-16T00:09:46.353 INFO:tasks.workunit.client.0.smithi139.stdout:5/613: dwrite d1/d13/d3b/f44 [0,4194304] 0 2024-09-16T00:09:46.357 INFO:tasks.workunit.client.0.smithi139.stdout:6/385: write d8/d29/d83/d26/d45/d65/f13 [4112183,67531] 0 2024-09-16T00:09:46.357 INFO:tasks.workunit.client.0.smithi139.stdout:7/514: truncate d5/d16/d22/d25/d2d/fa8 897911 0 2024-09-16T00:09:46.360 INFO:tasks.workunit.client.0.smithi139.stdout:6/386: read d8/d29/d83/d26/d45/d65/f13 [1168243,5156] 0 2024-09-16T00:09:46.362 INFO:tasks.workunit.client.0.smithi139.stdout:7/515: write d5/d26/f4c [8795666,57874] 0 2024-09-16T00:09:46.362 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:46.362+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:46.363 INFO:tasks.workunit.client.0.smithi139.stdout:5/614: dwrite d1/d13/f24 [4194304,4194304] 0 2024-09-16T00:09:46.364 INFO:tasks.workunit.client.0.smithi139.stdout:6/387: creat d8/d29/d83/d26/d28/d5a/f86 x:0 0 0 2024-09-16T00:09:46.367 INFO:tasks.workunit.client.0.smithi139.stdout:7/516: mkdir d5/d16/d22/db1 0 2024-09-16T00:09:46.379 INFO:tasks.workunit.client.0.smithi139.stdout:6/388: dwrite d8/d56/f7a [0,4194304] 0 2024-09-16T00:09:46.379 INFO:tasks.workunit.client.0.smithi139.stdout:7/517: dwrite d5/f1f [0,4194304] 0 2024-09-16T00:09:46.380 INFO:tasks.workunit.client.0.smithi139.stdout:5/615: dread d1/d13/f97 [0,4194304] 0 2024-09-16T00:09:46.381 INFO:tasks.workunit.client.0.smithi139.stdout:7/518: chown d5/d16/d22/d25/d2d/d50/l64 53442 1 2024-09-16T00:09:46.383 INFO:tasks.workunit.client.0.smithi139.stdout:5/616: fdatasync d1/d21/d49/d53/d6a/d7b/d78/dca/fcf 0 2024-09-16T00:09:46.388 INFO:tasks.workunit.client.0.smithi139.stdout:6/389: dwrite d8/d29/d83/d26/d28/d6d/f7e [0,4194304] 0 2024-09-16T00:09:46.389 INFO:tasks.workunit.client.0.smithi139.stdout:5/617: creat d1/d21/d49/d53/d6a/fd2 x:0 0 0 2024-09-16T00:09:46.392 INFO:tasks.workunit.client.0.smithi139.stdout:6/390: truncate d8/d29/d83/d26/d45/d59/f5e 1498296 0 2024-09-16T00:09:46.395 INFO:tasks.workunit.client.0.smithi139.stdout:5/618: link d1/d13/d3f/d9f/f5c d1/d13/d3f/fd3 0 2024-09-16T00:09:46.397 INFO:tasks.workunit.client.0.smithi139.stdout:6/391: mknod d8/d29/d83/d26/d45/d59/c87 0 2024-09-16T00:09:46.399 INFO:tasks.workunit.client.0.smithi139.stdout:5/619: getdents d1/d21/d49/db2 0 2024-09-16T00:09:46.400 INFO:tasks.workunit.client.0.smithi139.stdout:6/392: mknod d8/d29/d83/d26/c88 0 2024-09-16T00:09:46.402 INFO:tasks.workunit.client.0.smithi139.stdout:7/519: dread d5/fc [0,4194304] 0 2024-09-16T00:09:46.404 INFO:tasks.workunit.client.0.smithi139.stdout:7/520: rename d5/d26/c99 to d5/d16/d6d/cb2 0 2024-09-16T00:09:46.406 INFO:tasks.workunit.client.0.smithi139.stdout:7/521: chown d5/d26/d42/f66 372033 1 2024-09-16T00:09:46.408 INFO:tasks.workunit.client.0.smithi139.stdout:6/393: dwrite d8/d29/d83/d26/d45/f43 [0,4194304] 0 2024-09-16T00:09:46.408 INFO:tasks.workunit.client.0.smithi139.stdout:5/620: dwrite d1/d21/d49/d53/d6a/d7b/d78/dca/fcf [0,4194304] 0 2024-09-16T00:09:46.410 INFO:tasks.workunit.client.0.smithi139.stdout:6/394: stat d8/d29/d83/d26/d45/d42/c6a 0 2024-09-16T00:09:46.410 INFO:tasks.workunit.client.0.smithi139.stdout:5/621: chown d1/d21/d49/d87/f91 481 1 2024-09-16T00:09:46.411 INFO:tasks.workunit.client.0.smithi139.stdout:7/522: rename d5/cd to d5/d26/d46/cb3 0 2024-09-16T00:09:46.414 INFO:tasks.workunit.client.0.smithi139.stdout:6/395: creat d8/d29/d83/d26/d28/d6d/f89 x:0 0 0 2024-09-16T00:09:46.414 INFO:tasks.workunit.client.0.smithi139.stdout:6/396: stat d8/d29/d83/d26/f54 0 2024-09-16T00:09:46.414 INFO:tasks.workunit.client.0.smithi139.stdout:5/622: mkdir d1/d13/d3f/d9f/d30/dd4 0 2024-09-16T00:09:46.416 INFO:tasks.workunit.client.0.smithi139.stdout:6/397: read - d8/d29/d83/d26/d45/d42/f76 zero size 2024-09-16T00:09:46.417 INFO:tasks.workunit.client.0.smithi139.stdout:5/623: mkdir d1/d13/d22/dd5 0 2024-09-16T00:09:46.418 INFO:tasks.workunit.client.0.smithi139.stdout:0/546: sync 2024-09-16T00:09:46.419 INFO:tasks.workunit.client.0.smithi139.stdout:7/523: dwrite d5/d26/d5d/f98 [0,4194304] 0 2024-09-16T00:09:46.421 INFO:tasks.workunit.client.0.smithi139.stdout:6/398: creat d8/d29/d83/d26/d45/f8a x:0 0 0 2024-09-16T00:09:46.422 INFO:tasks.workunit.client.0.smithi139.stdout:5/624: mkdir d1/d13/d3f/d9f/d30/dd6 0 2024-09-16T00:09:46.423 INFO:tasks.workunit.client.0.smithi139.stdout:5/625: write d1/d13/f17 [11177804,63450] 0 2024-09-16T00:09:46.425 INFO:tasks.workunit.client.0.smithi139.stdout:5/626: write d1/d13/d3f/d5e/dc2/fce [573238,122350] 0 2024-09-16T00:09:46.427 INFO:tasks.workunit.client.0.smithi139.stdout:6/399: mkdir d8/d29/d83/d26/d45/d65/d8b 0 2024-09-16T00:09:46.428 INFO:tasks.workunit.client.0.smithi139.stdout:6/400: readlink d8/d29/d83/d26/d45/l5f 0 2024-09-16T00:09:46.428 INFO:tasks.workunit.client.0.smithi139.stdout:5/627: stat d1/d13/d22/l23 0 2024-09-16T00:09:46.429 INFO:tasks.workunit.client.0.smithi139.stdout:7/524: getdents d5/d16/d22/d25/d2d/d50/d44/d55 0 2024-09-16T00:09:46.429 INFO:tasks.workunit.client.0.smithi139.stdout:0/547: link d6/d1c/d41/d44/dd/c26 d6/d1c/d41/d76/ca9 0 2024-09-16T00:09:46.431 INFO:tasks.workunit.client.0.smithi139.stdout:5/628: creat d1/d21/d49/d53/d6a/d7b/fd7 x:0 0 0 2024-09-16T00:09:46.432 INFO:tasks.workunit.client.0.smithi139.stdout:0/548: rmdir d6/d1c/d41/d63/d82 39 2024-09-16T00:09:46.434 INFO:tasks.workunit.client.0.smithi139.stdout:0/549: truncate d6/d1c/d41/d7c/d86/f93 714630 0 2024-09-16T00:09:46.435 INFO:tasks.workunit.client.0.smithi139.stdout:7/525: rename d5/d16/d22/d25/d2d/l39 to d5/d16/d22/d25/d2d/dab/lb4 0 2024-09-16T00:09:46.437 INFO:tasks.workunit.client.0.smithi139.stdout:0/550: creat d6/d1c/d41/d76/faa x:0 0 0 2024-09-16T00:09:46.437 INFO:tasks.workunit.client.0.smithi139.stdout:7/526: mknod d5/d26/d46/d6b/cb5 0 2024-09-16T00:09:46.439 INFO:tasks.workunit.client.0.smithi139.stdout:7/527: symlink d5/d26/d46/d6b/lb6 0 2024-09-16T00:09:46.441 INFO:tasks.workunit.client.0.smithi139.stdout:0/551: symlink d6/d1c/d41/d44/dd/d8a/d9e/lab 0 2024-09-16T00:09:46.442 INFO:tasks.workunit.client.0.smithi139.stdout:6/401: truncate d8/d29/d83/d26/d45/d65/f37 172527 0 2024-09-16T00:09:46.444 INFO:tasks.workunit.client.0.smithi139.stdout:5/629: truncate d1/f4a 4412183 0 2024-09-16T00:09:46.449 INFO:tasks.workunit.client.0.smithi139.stdout:0/552: write d6/d1c/f36 [1906376,57871] 0 2024-09-16T00:09:46.449 INFO:tasks.workunit.client.0.smithi139.stdout:5/630: creat d1/d21/d49/d53/d6a/d7b/fd8 x:0 0 0 2024-09-16T00:09:46.450 INFO:tasks.workunit.client.0.smithi139.stdout:7/528: dwrite d5/d16/d22/d25/d2d/f3a [4194304,4194304] 0 2024-09-16T00:09:46.452 INFO:tasks.workunit.client.0.smithi139.stdout:6/402: dwrite d8/d29/d83/f81 [0,4194304] 0 2024-09-16T00:09:46.452 INFO:tasks.workunit.client.0.smithi139.stdout:0/553: write d6/d1c/d41/d44/dd/fa0 [273158,96262] 0 2024-09-16T00:09:46.453 INFO:tasks.workunit.client.0.smithi139.stdout:5/631: readlink d1/d13/d3f/d9f/d73/d79/da5/lbc 0 2024-09-16T00:09:46.455 INFO:tasks.workunit.client.0.smithi139.stdout:7/529: stat d5/d37/c3f 0 2024-09-16T00:09:46.461 INFO:tasks.workunit.client.0.smithi139.stdout:5/632: creat d1/d13/fd9 x:0 0 0 2024-09-16T00:09:46.461 INFO:tasks.workunit.client.0.smithi139.stdout:7/530: mknod d5/d16/d22/d25/d2d/dab/cb7 0 2024-09-16T00:09:46.463 INFO:tasks.workunit.client.0.smithi139.stdout:0/554: dwrite d6/d1c/f61 [0,4194304] 0 2024-09-16T00:09:46.463 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:46.462+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:46.464 INFO:tasks.workunit.client.0.smithi139.stdout:7/531: mknod d5/d37/cb8 0 2024-09-16T00:09:46.468 INFO:tasks.workunit.client.0.smithi139.stdout:6/403: dread d8/d29/d83/d26/d45/d65/f37 [0,4194304] 0 2024-09-16T00:09:46.473 INFO:tasks.workunit.client.0.smithi139.stdout:6/404: readlink d8/d29/d83/d26/d45/d42/l63 0 2024-09-16T00:09:46.473 INFO:tasks.workunit.client.0.smithi139.stdout:5/633: dwrite d1/d13/d22/f6b [0,4194304] 0 2024-09-16T00:09:46.475 INFO:tasks.workunit.client.0.smithi139.stdout:0/555: dwrite d6/d1c/d41/d44/f43 [0,4194304] 0 2024-09-16T00:09:46.475 INFO:tasks.workunit.client.0.smithi139.stdout:5/634: chown d1/d21/d49/d53/d6a/d7b/d99/l9a 170230656 1 2024-09-16T00:09:46.478 INFO:tasks.workunit.client.0.smithi139.stdout:5/635: symlink d1/d21/d49/d53/d6a/d7b/da1/lda 0 2024-09-16T00:09:46.481 INFO:tasks.workunit.client.0.smithi139.stdout:5/636: rmdir d1/d13/d3f/d88 39 2024-09-16T00:09:46.484 INFO:tasks.workunit.client.0.smithi139.stdout:5/637: creat d1/d13/d3f/d9f/d73/d93/fdb x:0 0 0 2024-09-16T00:09:46.485 INFO:tasks.workunit.client.0.smithi139.stdout:6/405: dread d8/d29/d83/d26/f4b [0,4194304] 0 2024-09-16T00:09:46.485 INFO:tasks.workunit.client.0.smithi139.stdout:5/638: chown d1/d21/c66 1419530 1 2024-09-16T00:09:46.486 INFO:tasks.workunit.client.0.smithi139.stdout:5/639: read d1/d13/d3f/d9f/d73/f8c [167714,44737] 0 2024-09-16T00:09:46.487 INFO:tasks.workunit.client.0.smithi139.stdout:6/406: unlink d8/d20/f4a 0 2024-09-16T00:09:46.492 INFO:tasks.workunit.client.0.smithi139.stdout:5/640: dwrite d1/d13/d3f/fb8 [0,4194304] 0 2024-09-16T00:09:46.492 INFO:tasks.workunit.client.0.smithi139.stdout:6/407: link d8/d29/d83/d26/d28/f3f d8/f8c 0 2024-09-16T00:09:46.493 INFO:tasks.workunit.client.0.smithi139.stdout:5/641: dread - d1/d13/d3f/d9f/d73/d93/fdb zero size 2024-09-16T00:09:46.497 INFO:tasks.workunit.client.0.smithi139.stdout:5/642: creat d1/d13/fdc x:0 0 0 2024-09-16T00:09:46.500 INFO:tasks.workunit.client.0.smithi139.stdout:6/408: dwrite d8/d29/d83/d26/d45/f46 [0,4194304] 0 2024-09-16T00:09:46.505 INFO:tasks.workunit.client.0.smithi139.stdout:5/643: truncate d1/d13/d22/f6b 299050 0 2024-09-16T00:09:46.506 INFO:tasks.workunit.client.0.smithi139.stdout:5/644: write d1/d21/f39 [902122,3476] 0 2024-09-16T00:09:46.507 INFO:tasks.workunit.client.0.smithi139.stdout:6/409: dwrite d8/d29/d83/d26/f54 [0,4194304] 0 2024-09-16T00:09:46.507 INFO:tasks.workunit.client.0.smithi139.stdout:5/645: fdatasync d1/d13/d3b/f47 0 2024-09-16T00:09:46.510 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:46.510 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:46.512 INFO:tasks.workunit.client.0.smithi139.stdout:6/410: creat d8/f8d x:0 0 0 2024-09-16T00:09:46.513 INFO:tasks.workunit.client.0.smithi139.stdout:5/646: creat d1/fdd x:0 0 0 2024-09-16T00:09:46.513 INFO:tasks.workunit.client.0.smithi139.stdout:6/411: chown d8/d29/d83/d26/d28/d6d/f71 1 1 2024-09-16T00:09:46.514 INFO:tasks.workunit.client.0.smithi139.stdout:5/647: chown d1/d13/d3f/ca6 117816834 1 2024-09-16T00:09:46.518 INFO:tasks.workunit.client.0.smithi139.stdout:6/412: rename d8/d29/d83/d26/d28/d6d/f7e to d8/d20/d4d/f8e 0 2024-09-16T00:09:46.519 INFO:tasks.workunit.client.0.smithi139.stdout:5/648: dwrite d1/d13/d3b/d3e/f5f [0,4194304] 0 2024-09-16T00:09:46.523 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:09:46.527 INFO:tasks.workunit.client.0.smithi139.stdout:5/649: mkdir d1/d21/dde 0 2024-09-16T00:09:46.528 INFO:tasks.workunit.client.0.smithi139.stdout:6/413: write d8/d29/d83/d26/d28/f3f [72428,104153] 0 2024-09-16T00:09:46.530 INFO:tasks.workunit.client.0.smithi139.stdout:6/414: dread d8/d20/f64 [0,4194304] 0 2024-09-16T00:09:46.531 INFO:tasks.workunit.client.0.smithi139.stdout:6/415: dread d8/f39 [0,4194304] 0 2024-09-16T00:09:46.536 INFO:tasks.workunit.client.0.smithi139.stdout:6/416: truncate d8/d29/f2d 1989826 0 2024-09-16T00:09:46.540 INFO:tasks.workunit.client.0.smithi139.stdout:5/650: dwrite d1/d13/d3f/d9f/d73/f8c [0,4194304] 0 2024-09-16T00:09:46.541 INFO:tasks.workunit.client.0.smithi139.stdout:5/651: dread - d1/d21/d49/fd1 zero size 2024-09-16T00:09:46.549 INFO:tasks.workunit.client.0.smithi139.stdout:5/652: dwrite d1/d21/d49/fd1 [0,4194304] 0 2024-09-16T00:09:46.552 INFO:tasks.workunit.client.0.smithi139.stdout:5/653: creat d1/d21/d49/db2/fdf x:0 0 0 2024-09-16T00:09:46.552 INFO:tasks.workunit.client.0.smithi139.stdout:6/417: dread d8/d29/d83/d26/d28/f3f [0,4194304] 0 2024-09-16T00:09:46.553 INFO:tasks.workunit.client.0.smithi139.stdout:6/418: dread - d8/d29/d83/d26/d45/d42/f76 zero size 2024-09-16T00:09:46.554 INFO:tasks.workunit.client.0.smithi139.stdout:5/654: creat d1/d13/d3f/d9f/d73/d93/fe0 x:0 0 0 2024-09-16T00:09:46.555 INFO:tasks.workunit.client.0.smithi139.stdout:6/419: creat d8/d29/f8f x:0 0 0 2024-09-16T00:09:46.559 INFO:tasks.workunit.client.0.smithi139.stdout:5/655: dwrite d1/d21/d49/db2/fba [0,4194304] 0 2024-09-16T00:09:46.560 INFO:tasks.workunit.client.0.smithi139.stdout:5/656: chown d1/d13/d3f/ca6 457 1 2024-09-16T00:09:46.564 INFO:tasks.workunit.client.0.smithi139.stdout:5/657: link d1/c72 d1/d21/d49/db0/ce1 0 2024-09-16T00:09:46.564 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:46.564+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:46.566 INFO:tasks.workunit.client.0.smithi139.stdout:5/658: mknod d1/d13/d3f/ce2 0 2024-09-16T00:09:46.574 INFO:tasks.workunit.client.0.smithi139.stdout:5/659: dwrite d1/d21/d49/f7f [0,4194304] 0 2024-09-16T00:09:46.580 INFO:tasks.workunit.client.0.smithi139.stdout:2/558: sync 2024-09-16T00:09:46.584 INFO:tasks.workunit.client.0.smithi139.stdout:5/660: dwrite d1/f48 [0,4194304] 0 2024-09-16T00:09:46.586 INFO:tasks.workunit.client.0.smithi139.stdout:5/661: mknod d1/d21/d49/db2/ce3 0 2024-09-16T00:09:46.589 INFO:tasks.workunit.client.0.smithi139.stdout:2/559: dwrite d11/d30/d57/d29/da5/d81/f97 [0,4194304] 0 2024-09-16T00:09:46.591 INFO:tasks.workunit.client.0.smithi139.stdout:5/662: rename d1/f27 to d1/d13/d3f/d88/fe4 0 2024-09-16T00:09:46.594 INFO:tasks.workunit.client.0.smithi139.stdout:5/663: symlink d1/d13/d3f/le5 0 2024-09-16T00:09:46.648 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:46.648 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:46.658 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:09:46.665 INFO:tasks.workunit.client.0.smithi139.stdout:2/560: truncate d11/d30/d57/d29/d2b/faf 1048424 0 2024-09-16T00:09:46.665 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:46.664+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:46.667 INFO:tasks.workunit.client.0.smithi139.stdout:2/561: unlink d11/d30/c47 0 2024-09-16T00:09:46.667 INFO:tasks.workunit.client.0.smithi139.stdout:2/562: chown d11/l20 5620 1 2024-09-16T00:09:46.668 INFO:tasks.workunit.client.0.smithi139.stdout:5/664: rename d1/d13/d3f/d9f/f2e to d1/d13/d3f/fe6 0 2024-09-16T00:09:46.670 INFO:tasks.workunit.client.0.smithi139.stdout:5/665: mknod d1/d13/d22/dd5/ce7 0 2024-09-16T00:09:46.671 INFO:tasks.workunit.client.0.smithi139.stdout:5/666: write d1/d13/f24 [5986440,100939] 0 2024-09-16T00:09:46.672 INFO:tasks.workunit.client.0.smithi139.stdout:5/667: chown d1/d13/d3f/ca6 7025786 1 2024-09-16T00:09:46.673 INFO:tasks.workunit.client.0.smithi139.stdout:2/563: dwrite d11/f16 [0,4194304] 0 2024-09-16T00:09:46.677 INFO:tasks.workunit.client.0.smithi139.stdout:2/564: truncate d11/d30/d57/d5d/fb4 543500 0 2024-09-16T00:09:46.679 INFO:tasks.workunit.client.0.smithi139.stdout:2/565: symlink d11/d30/d57/d29/d2b/lbc 0 2024-09-16T00:09:46.680 INFO:tasks.workunit.client.0.smithi139.stdout:5/668: dwrite d1/d21/d49/fd1 [0,4194304] 0 2024-09-16T00:09:46.682 INFO:tasks.workunit.client.0.smithi139.stdout:2/566: symlink d11/d30/d57/d29/d2b/lbd 0 2024-09-16T00:09:46.685 INFO:tasks.workunit.client.0.smithi139.stdout:5/669: mkdir d1/d13/d22/de8 0 2024-09-16T00:09:46.685 INFO:tasks.workunit.client.0.smithi139.stdout:2/567: creat d11/d30/d57/d29/da5/fbe x:0 0 0 2024-09-16T00:09:46.686 INFO:tasks.workunit.client.0.smithi139.stdout:5/670: chown d1/d13/d22/f80 0 1 2024-09-16T00:09:46.686 INFO:tasks.workunit.client.0.smithi139.stdout:5/671: readlink d1/d13/d22/l45 0 2024-09-16T00:09:46.687 INFO:tasks.workunit.client.0.smithi139.stdout:2/568: mkdir d11/d30/dbf 0 2024-09-16T00:09:46.689 INFO:tasks.workunit.client.0.smithi139.stdout:2/569: mknod d11/d30/d57/cc0 0 2024-09-16T00:09:46.692 INFO:tasks.workunit.client.0.smithi139.stdout:2/570: creat d11/d30/d57/fc1 x:0 0 0 2024-09-16T00:09:46.693 INFO:tasks.workunit.client.0.smithi139.stdout:5/672: dwrite d1/d21/d49/d53/d6a/d7b/d99/fac [0,4194304] 0 2024-09-16T00:09:46.696 INFO:tasks.workunit.client.0.smithi139.stdout:5/673: getdents d1/d13/d3f/d9f/d30/dd4 0 2024-09-16T00:09:46.698 INFO:tasks.workunit.client.0.smithi139.stdout:5/674: creat d1/fe9 x:0 0 0 2024-09-16T00:09:46.700 INFO:tasks.workunit.client.0.smithi139.stdout:5/675: write d1/d13/d3f/d9f/d73/d93/fe0 [679564,36934] 0 2024-09-16T00:09:46.702 INFO:tasks.workunit.client.0.smithi139.stdout:5/676: creat d1/d13/d3f/d9f/d30/fea x:0 0 0 2024-09-16T00:09:46.703 INFO:tasks.workunit.client.0.smithi139.stdout:5/677: chown d1/f1a 95840 1 2024-09-16T00:09:46.706 INFO:tasks.workunit.client.0.smithi139.stdout:5/678: getdents d1/d21 0 2024-09-16T00:09:46.707 INFO:tasks.workunit.client.0.smithi139.stdout:2/571: dread d11/f1a [0,4194304] 0 2024-09-16T00:09:46.709 INFO:tasks.workunit.client.0.smithi139.stdout:2/572: creat d11/d30/d57/d50/fc2 x:0 0 0 2024-09-16T00:09:46.712 INFO:tasks.workunit.client.0.smithi139.stdout:2/573: mkdir d11/d30/d57/d5d/d73/dc3 0 2024-09-16T00:09:46.713 INFO:tasks.workunit.client.0.smithi139.stdout:5/679: dwrite d1/d21/d49/d53/d6a/d7b/fd8 [0,4194304] 0 2024-09-16T00:09:46.713 INFO:tasks.workunit.client.0.smithi139.stdout:2/574: chown d11/d30/d57/d29/d4f/d9c/la9 189359036 1 2024-09-16T00:09:46.714 INFO:tasks.workunit.client.0.smithi139.stdout:2/575: fsync d11/d32/fa8 0 2024-09-16T00:09:46.716 INFO:tasks.workunit.client.0.smithi139.stdout:2/576: readlink d11/d30/d57/d29/d2b/lbc 0 2024-09-16T00:09:46.717 INFO:tasks.workunit.client.0.smithi139.stdout:5/680: creat d1/d21/d49/d53/d6a/d7b/d99/feb x:0 0 0 2024-09-16T00:09:46.718 INFO:tasks.workunit.client.0.smithi139.stdout:4/520: sync 2024-09-16T00:09:46.720 INFO:tasks.workunit.client.0.smithi139.stdout:5/681: write d1/d13/d3f/d9f/d30/f55 [192160,121727] 0 2024-09-16T00:09:46.729 INFO:tasks.workunit.client.0.smithi139.stdout:2/577: dwrite d11/d2f/d6e/d7f/fb5 [0,4194304] 0 2024-09-16T00:09:46.729 INFO:tasks.workunit.client.0.smithi139.stdout:5/682: write d1/d13/d22/f6b [799579,52884] 0 2024-09-16T00:09:46.732 INFO:tasks.workunit.client.0.smithi139.stdout:5/683: chown d1/d13/d3f/d9f/d30/c2b 119950 1 2024-09-16T00:09:46.735 INFO:tasks.workunit.client.0.smithi139.stdout:5/684: mknod d1/d13/d3f/d88/cec 0 2024-09-16T00:09:46.737 INFO:tasks.workunit.client.0.smithi139.stdout:2/578: dwrite d11/f15 [8388608,4194304] 0 2024-09-16T00:09:46.738 INFO:tasks.workunit.client.0.smithi139.stdout:4/521: dread d12/d80/f9a [0,4194304] 0 2024-09-16T00:09:46.749 INFO:tasks.workunit.client.0.smithi139.stdout:2/579: dwrite d11/d45/fa4 [0,4194304] 0 2024-09-16T00:09:46.750 INFO:tasks.workunit.client.0.smithi139.stdout:4/522: dwrite d12/d80/f95 [0,4194304] 0 2024-09-16T00:09:46.750 INFO:tasks.workunit.client.0.smithi139.stdout:5/685: dwrite d1/d13/d3f/d9f/f36 [0,4194304] 0 2024-09-16T00:09:46.753 INFO:tasks.workunit.client.0.smithi139.stdout:4/523: rmdir d12/d47/d49 39 2024-09-16T00:09:46.754 INFO:tasks.workunit.client.0.smithi139.stdout:5/686: symlink d1/d13/d3f/d9f/d73/d93/led 0 2024-09-16T00:09:46.757 INFO:tasks.workunit.client.0.smithi139.stdout:2/580: creat d11/d30/d57/d29/fc4 x:0 0 0 2024-09-16T00:09:46.758 INFO:tasks.workunit.client.0.smithi139.stdout:5/687: creat d1/d21/d49/d53/d6a/d7b/da1/db7/fee x:0 0 0 2024-09-16T00:09:46.760 INFO:tasks.workunit.client.0.smithi139.stdout:5/688: write d1/d13/d3f/d9f/d73/d93/fdb [855635,86417] 0 2024-09-16T00:09:46.762 INFO:tasks.workunit.client.0.smithi139.stdout:2/581: rename d11/f53 to d11/d30/d57/d29/da5/d81/fc5 0 2024-09-16T00:09:46.764 INFO:tasks.workunit.client.0.smithi139.stdout:5/689: unlink d1/d13/d3f/d9f/d30/d33/f50 0 2024-09-16T00:09:46.766 INFO:tasks.workunit.client.0.smithi139.stdout:2/582: mkdir d11/d30/d57/d29/da5/dc6 0 2024-09-16T00:09:46.766 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:46.765+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:46.771 INFO:tasks.workunit.client.0.smithi139.stdout:5/690: creat d1/d13/d3f/d9f/d73/fef x:0 0 0 2024-09-16T00:09:46.773 INFO:tasks.workunit.client.0.smithi139.stdout:5/691: symlink d1/d13/d3f/d9f/d30/dd6/lf0 0 2024-09-16T00:09:46.774 INFO:tasks.workunit.client.0.smithi139.stdout:2/583: write d11/d30/f92 [1025238,79994] 0 2024-09-16T00:09:46.776 INFO:tasks.workunit.client.0.smithi139.stdout:2/584: write d11/d30/d57/d29/d4f/fb2 [73418,102026] 0 2024-09-16T00:09:46.777 INFO:tasks.workunit.client.0.smithi139.stdout:2/585: write d11/d30/d57/d29/da5/d81/f99 [728042,15811] 0 2024-09-16T00:09:46.778 INFO:tasks.workunit.client.0.smithi139.stdout:2/586: chown d11/d32/c77 7985 1 2024-09-16T00:09:46.780 INFO:tasks.workunit.client.0.smithi139.stdout:5/692: dread d1/d13/d3f/d9f/f6c [0,4194304] 0 2024-09-16T00:09:46.781 INFO:tasks.workunit.client.0.smithi139.stdout:5/693: read d1/d13/d22/f6b [302778,73509] 0 2024-09-16T00:09:46.781 INFO:tasks.workunit.client.0.smithi139.stdout:2/587: unlink d11/d30/d57/d29/d4f/d9c/fae 0 2024-09-16T00:09:46.785 INFO:tasks.workunit.client.0.smithi139.stdout:5/694: dread d1/d13/d3f/d9f/f36 [0,4194304] 0 2024-09-16T00:09:46.788 INFO:tasks.workunit.client.0.smithi139.stdout:5/695: creat d1/d21/d49/d53/d6a/d7b/da1/db7/ff1 x:0 0 0 2024-09-16T00:09:46.789 INFO:tasks.workunit.client.0.smithi139.stdout:2/588: truncate d11/d30/f42 2796487 0 2024-09-16T00:09:46.790 INFO:tasks.workunit.client.0.smithi139.stdout:2/589: chown d11/d30/d57/d29/da5/d81/fc5 0 1 2024-09-16T00:09:46.790 INFO:tasks.workunit.client.0.smithi139.stdout:5/696: creat d1/d21/d49/d53/d6a/d7b/d78/ff2 x:0 0 0 2024-09-16T00:09:46.794 INFO:tasks.workunit.client.0.smithi139.stdout:5/697: link d1/d13/d3f/d88/cec d1/d13/d3f/d88/cf3 0 2024-09-16T00:09:46.801 INFO:tasks.workunit.client.0.smithi139.stdout:5/698: dwrite d1/fdd [0,4194304] 0 2024-09-16T00:09:46.801 INFO:tasks.workunit.client.0.smithi139.stdout:2/590: dread d11/d30/f42 [0,4194304] 0 2024-09-16T00:09:46.805 INFO:tasks.workunit.client.0.smithi139.stdout:5/699: creat d1/d13/ff4 x:0 0 0 2024-09-16T00:09:46.805 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:46.805 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:46.806 INFO:tasks.workunit.client.0.smithi139.stdout:2/591: creat d11/d30/d57/d5d/d73/fc7 x:0 0 0 2024-09-16T00:09:46.806 INFO:tasks.workunit.client.0.smithi139.stdout:5/700: symlink d1/d13/d3f/lf5 0 2024-09-16T00:09:46.810 INFO:tasks.workunit.client.0.smithi139.stdout:2/592: dread d11/d30/f92 [0,4194304] 0 2024-09-16T00:09:46.811 INFO:tasks.workunit.client.0.smithi139.stdout:2/593: fdatasync d11/d30/d57/d29/fc4 0 2024-09-16T00:09:46.812 INFO:tasks.workunit.client.0.smithi139.stdout:5/701: dwrite d1/fe9 [0,4194304] 0 2024-09-16T00:09:46.816 INFO:tasks.workunit.client.0.smithi139.stdout:5/702: chown d1/d13/d3f/d88/cec 574 1 2024-09-16T00:09:46.816 INFO:tasks.workunit.client.0.smithi139.stdout:2/594: symlink d11/d30/d57/d5d/d73/dc3/lc8 0 2024-09-16T00:09:46.817 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:09:46.818 INFO:tasks.workunit.client.0.smithi139.stdout:5/703: mkdir d1/df6 0 2024-09-16T00:09:46.818 INFO:tasks.workunit.client.0.smithi139.stdout:2/595: mknod d11/d30/d57/d29/da5/cc9 0 2024-09-16T00:09:46.819 INFO:tasks.workunit.client.0.smithi139.stdout:5/704: symlink d1/dc3/lf7 0 2024-09-16T00:09:46.821 INFO:tasks.workunit.client.0.smithi139.stdout:2/596: unlink d11/d30/d57/d29/da5/d81/l90 0 2024-09-16T00:09:46.823 INFO:tasks.workunit.client.0.smithi139.stdout:5/705: rename d1/d13/d3b/d3e/c6d to d1/d13/d3f/d5e/cf8 0 2024-09-16T00:09:46.826 INFO:tasks.workunit.client.0.smithi139.stdout:2/597: link d11/d30/f56 d11/d30/d57/d29/d4f/d9c/fca 0 2024-09-16T00:09:46.827 INFO:tasks.workunit.client.0.smithi139.stdout:5/706: unlink d1/d13/d3f/laf 0 2024-09-16T00:09:46.830 INFO:tasks.workunit.client.0.smithi139.stdout:5/707: creat d1/d21/d49/d53/d6a/d7b/d78/dca/ff9 x:0 0 0 2024-09-16T00:09:46.831 INFO:tasks.workunit.client.0.smithi139.stdout:2/598: truncate d11/d30/d57/d5d/d73/f8e 232222 0 2024-09-16T00:09:46.832 INFO:tasks.workunit.client.0.smithi139.stdout:2/599: stat d11/d30/cb9 0 2024-09-16T00:09:46.833 INFO:tasks.workunit.client.0.smithi139.stdout:2/600: fdatasync d11/f22 0 2024-09-16T00:09:46.835 INFO:tasks.workunit.client.0.smithi139.stdout:5/708: dwrite d1/d13/d3b/f47 [0,4194304] 0 2024-09-16T00:09:46.837 INFO:tasks.workunit.client.0.smithi139.stdout:2/601: mkdir d11/d30/d57/d50/dcb 0 2024-09-16T00:09:46.838 INFO:tasks.workunit.client.0.smithi139.stdout:5/709: mknod d1/d13/d3f/d9f/d73/d79/d8d/cfa 0 2024-09-16T00:09:46.838 INFO:tasks.workunit.client.0.smithi139.stdout:2/602: mkdir d11/d2f/dcc 0 2024-09-16T00:09:46.840 INFO:tasks.workunit.client.0.smithi139.stdout:2/603: write d11/d30/d57/d29/da5/d81/f89 [831860,50792] 0 2024-09-16T00:09:46.858 INFO:tasks.workunit.client.0.smithi139.stdout:2/604: dread d11/d30/d57/d29/d2b/f31 [0,4194304] 0 2024-09-16T00:09:46.860 INFO:tasks.workunit.client.0.smithi139.stdout:2/605: rename d11/d30/d57/d50/fc2 to d11/d32/fcd 0 2024-09-16T00:09:46.862 INFO:tasks.workunit.client.0.smithi139.stdout:2/606: mkdir d11/d30/dce 0 2024-09-16T00:09:46.863 INFO:tasks.workunit.client.0.smithi139.stdout:2/607: getdents d11/d2f/dcc 0 2024-09-16T00:09:46.863 INFO:tasks.workunit.client.0.smithi139.stdout:2/608: write d11/d32/fa8 [907662,130212] 0 2024-09-16T00:09:46.865 INFO:tasks.workunit.client.0.smithi139.stdout:2/609: unlink d11/d30/d57/d29/d4f/l82 0 2024-09-16T00:09:46.866 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:46.866+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:46.867 INFO:tasks.workunit.client.0.smithi139.stdout:2/610: symlink d11/d30/d57/d29/da5/lcf 0 2024-09-16T00:09:46.873 INFO:tasks.workunit.client.0.smithi139.stdout:3/459: sync 2024-09-16T00:09:46.877 INFO:tasks.workunit.client.0.smithi139.stdout:2/611: dread d11/d30/d57/d29/f66 [0,4194304] 0 2024-09-16T00:09:46.878 INFO:tasks.workunit.client.0.smithi139.stdout:2/612: fdatasync d11/d30/d57/d29/d2b/f65 0 2024-09-16T00:09:46.884 INFO:tasks.workunit.client.0.smithi139.stdout:2/613: dwrite d11/f41 [0,4194304] 0 2024-09-16T00:09:46.884 INFO:tasks.workunit.client.0.smithi139.stdout:3/460: dread d5/d7/f2d [0,4194304] 0 2024-09-16T00:09:46.885 INFO:tasks.workunit.client.0.smithi139.stdout:3/461: chown d5/f53 6458 1 2024-09-16T00:09:46.886 INFO:tasks.workunit.client.0.smithi139.stdout:2/614: write d11/d30/d57/d29/d4f/fa2 [298888,6660] 0 2024-09-16T00:09:46.886 INFO:tasks.workunit.client.0.smithi139.stdout:3/462: fdatasync d5/d7/d23/d27/d2f/d30/d41/f7f 0 2024-09-16T00:09:46.889 INFO:tasks.workunit.client.0.smithi139.stdout:2/615: symlink d11/d30/d57/d5d/d73/ld0 0 2024-09-16T00:09:46.890 INFO:tasks.workunit.client.0.smithi139.stdout:3/463: symlink d5/d7/d23/d35/da7/la8 0 2024-09-16T00:09:46.890 INFO:tasks.workunit.client.0.smithi139.stdout:3/464: dread - d5/d7/f49 zero size 2024-09-16T00:09:46.891 INFO:tasks.workunit.client.0.smithi139.stdout:2/616: getdents d11/d30/dbf 0 2024-09-16T00:09:46.892 INFO:tasks.workunit.client.0.smithi139.stdout:3/465: fdatasync d5/d7/d23/d27/d2f/d30/d4c/d50/f87 0 2024-09-16T00:09:46.895 INFO:tasks.workunit.client.0.smithi139.stdout:2/617: link d11/d2f/d6e/c7e d11/d30/d57/cd1 0 2024-09-16T00:09:46.903 INFO:tasks.workunit.client.0.smithi139.stdout:2/618: dwrite d11/d30/f92 [0,4194304] 0 2024-09-16T00:09:46.910 INFO:tasks.workunit.client.0.smithi139.stdout:2/619: link d11/d30/d57/cd1 d11/d2f/d6e/cd2 0 2024-09-16T00:09:46.912 INFO:tasks.workunit.client.0.smithi139.stdout:2/620: chown d11/d2f/d6e/cd2 145 1 2024-09-16T00:09:46.914 INFO:tasks.workunit.client.0.smithi139.stdout:2/621: mknod d11/d30/d57/d29/cd3 0 2024-09-16T00:09:46.915 INFO:tasks.workunit.client.0.smithi139.stdout:2/622: truncate d11/d2f/d6e/f84 82121 0 2024-09-16T00:09:46.917 INFO:tasks.workunit.client.0.smithi139.stdout:2/623: dread d11/d30/d57/d29/d4f/fb2 [0,4194304] 0 2024-09-16T00:09:46.917 INFO:tasks.workunit.client.0.smithi139.stdout:2/624: readlink d11/d30/l88 0 2024-09-16T00:09:46.921 INFO:tasks.workunit.client.0.smithi139.stdout:2/625: rename d11/d30/d57/d50/f69 to d11/d30/d57/d5d/d73/fd4 0 2024-09-16T00:09:46.924 INFO:tasks.workunit.client.0.smithi139.stdout:2/626: creat d11/d30/fd5 x:0 0 0 2024-09-16T00:09:46.924 INFO:tasks.workunit.client.0.smithi139.stdout:2/627: chown d11/d2f/ca6 479 1 2024-09-16T00:09:46.925 INFO:tasks.workunit.client.0.smithi139.stdout:2/628: write d11/f1e [4356255,97804] 0 2024-09-16T00:09:46.930 INFO:tasks.workunit.client.0.smithi139.stdout:2/629: dwrite d11/d30/f56 [0,4194304] 0 2024-09-16T00:09:46.934 INFO:tasks.workunit.client.0.smithi139.stdout:2/630: rename d11/d30/d57/c3a to d11/d30/d57/d29/da5/d81/cd6 0 2024-09-16T00:09:46.934 INFO:tasks.workunit.client.0.smithi139.stdout:2/631: chown d11/d30/d57/cc0 825 1 2024-09-16T00:09:46.936 INFO:tasks.workunit.client.0.smithi139.stdout:2/632: read d11/f19 [171756,92824] 0 2024-09-16T00:09:46.938 INFO:tasks.workunit.client.0.smithi139.stdout:2/633: mknod d11/d2f/dcc/cd7 0 2024-09-16T00:09:46.940 INFO:tasks.workunit.client.0.smithi139.stdout:2/634: mknod d11/d30/d57/d5d/d73/dc3/cd8 0 2024-09-16T00:09:46.941 INFO:tasks.workunit.client.0.smithi139.stdout:2/635: mknod d11/d30/d57/d50/cd9 0 2024-09-16T00:09:46.942 INFO:tasks.workunit.client.0.smithi139.stdout:2/636: mknod d11/d30/d57/d29/d4f/cda 0 2024-09-16T00:09:46.944 INFO:tasks.workunit.client.0.smithi139.stdout:2/637: mkdir d11/d30/d57/d29/da5/dc6/ddb 0 2024-09-16T00:09:46.945 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:46.946 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:46.948 INFO:tasks.workunit.client.0.smithi139.stdout:2/638: dwrite d11/d2f/f33 [0,4194304] 0 2024-09-16T00:09:46.950 INFO:tasks.workunit.client.0.smithi139.stdout:2/639: creat d11/d30/d57/d29/da5/fdc x:0 0 0 2024-09-16T00:09:46.953 INFO:tasks.workunit.client.0.smithi139.stdout:2/640: link d11/d30/f36 d11/fdd 0 2024-09-16T00:09:46.955 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:09:46.955 INFO:tasks.workunit.client.0.smithi139.stdout:2/641: mkdir d11/d45/dde 0 2024-09-16T00:09:46.957 INFO:tasks.workunit.client.0.smithi139.stdout:2/642: rename d11/d32/fcd to d11/d30/d57/d5d/d73/fdf 0 2024-09-16T00:09:46.959 INFO:tasks.workunit.client.0.smithi139.stdout:2/643: truncate d11/d30/d57/d29/f5c 4538250 0 2024-09-16T00:09:46.961 INFO:tasks.workunit.client.0.smithi139.stdout:2/644: symlink d11/d30/d57/d29/da5/dc6/ddb/le0 0 2024-09-16T00:09:46.962 INFO:tasks.workunit.client.0.smithi139.stdout:2/645: mkdir d11/d30/d57/d29/da5/daa/de1 0 2024-09-16T00:09:46.964 INFO:tasks.workunit.client.0.smithi139.stdout:2/646: creat d11/d30/dce/fe2 x:0 0 0 2024-09-16T00:09:46.965 INFO:tasks.workunit.client.0.smithi139.stdout:2/647: chown d11/d32/fa8 47 1 2024-09-16T00:09:46.967 INFO:tasks.workunit.client.0.smithi139.stdout:2/648: read d11/d30/d57/d29/da5/d81/f99 [678004,84246] 0 2024-09-16T00:09:46.967 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:46.966+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:46.972 INFO:tasks.workunit.client.0.smithi139.stdout:2/649: dwrite d11/d30/d57/d29/d2b/f98 [0,4194304] 0 2024-09-16T00:09:46.975 INFO:tasks.workunit.client.0.smithi139.stdout:2/650: symlink d11/d30/d57/le3 0 2024-09-16T00:09:46.982 INFO:tasks.workunit.client.0.smithi139.stdout:2/651: dread d11/f22 [0,4194304] 0 2024-09-16T00:09:46.992 INFO:tasks.workunit.client.0.smithi139.stdout:2/652: write d11/d30/d57/d29/d2b/faf [1034249,92439] 0 2024-09-16T00:09:46.994 INFO:tasks.workunit.client.0.smithi139.stdout:2/653: read d11/d30/d57/f59 [1619388,93600] 0 2024-09-16T00:09:46.996 INFO:tasks.workunit.client.0.smithi139.stdout:2/654: symlink d11/d30/dbf/le4 0 2024-09-16T00:09:47.005 INFO:tasks.workunit.client.0.smithi139.stdout:2/655: dwrite d11/d30/d57/d5d/d73/f8e [0,4194304] 0 2024-09-16T00:09:47.016 INFO:tasks.workunit.client.0.smithi139.stdout:2/656: dread d11/d30/d57/d29/f5c [0,4194304] 0 2024-09-16T00:09:47.018 INFO:tasks.workunit.client.0.smithi139.stdout:2/657: rmdir d11/d30/d57/d29/da5/dc6 39 2024-09-16T00:09:47.019 INFO:tasks.workunit.client.0.smithi139.stdout:2/658: mkdir d11/d2f/d6e/de5 0 2024-09-16T00:09:47.020 INFO:tasks.workunit.client.0.smithi139.stdout:2/659: truncate d11/d30/fd5 83411 0 2024-09-16T00:09:47.021 INFO:tasks.workunit.client.0.smithi139.stdout:2/660: mknod d11/d30/d57/d50/ce6 0 2024-09-16T00:09:47.023 INFO:tasks.workunit.client.0.smithi139.stdout:2/661: link d11/c1b d11/d45/dde/ce7 0 2024-09-16T00:09:47.023 INFO:tasks.workunit.client.0.smithi139.stdout:2/662: chown d11/d2f/d6e/de5 9796 1 2024-09-16T00:09:47.025 INFO:tasks.workunit.client.0.smithi139.stdout:2/663: write fc [1260200,106688] 0 2024-09-16T00:09:47.026 INFO:tasks.workunit.client.0.smithi139.stdout:2/664: dread d11/d30/d57/d29/d4f/fb2 [0,4194304] 0 2024-09-16T00:09:47.027 INFO:tasks.workunit.client.0.smithi139.stdout:2/665: read d11/f64 [517660,49538] 0 2024-09-16T00:09:47.028 INFO:tasks.workunit.client.0.smithi139.stdout:4/524: sync 2024-09-16T00:09:47.028 INFO:tasks.workunit.client.0.smithi139.stdout:0/556: sync 2024-09-16T00:09:47.028 INFO:tasks.workunit.client.0.smithi139.stdout:8/602: sync 2024-09-16T00:09:47.028 INFO:tasks.workunit.client.0.smithi139.stdout:1/472: sync 2024-09-16T00:09:47.028 INFO:tasks.workunit.client.0.smithi139.stdout:7/532: sync 2024-09-16T00:09:47.028 INFO:tasks.workunit.client.0.smithi139.stdout:6/420: sync 2024-09-16T00:09:47.028 INFO:tasks.workunit.client.0.smithi139.stdout:9/658: sync 2024-09-16T00:09:47.030 INFO:tasks.workunit.client.0.smithi139.stdout:9/659: fdatasync d19/d1a/f3f 0 2024-09-16T00:09:47.030 INFO:tasks.workunit.client.0.smithi139.stdout:6/421: dread - d8/d29/f36 zero size 2024-09-16T00:09:47.031 INFO:tasks.workunit.client.0.smithi139.stdout:8/603: symlink d17/d1a/d50/lcc 0 2024-09-16T00:09:47.032 INFO:tasks.workunit.client.0.smithi139.stdout:7/533: creat d5/d16/d22/d25/d63/fb9 x:0 0 0 2024-09-16T00:09:47.034 INFO:tasks.workunit.client.0.smithi139.stdout:4/525: getdents d12/d4a 0 2024-09-16T00:09:47.034 INFO:tasks.workunit.client.0.smithi139.stdout:1/473: read d7/d8/d40/d48/d53/d61/f87 [63817,38510] 0 2024-09-16T00:09:47.035 INFO:tasks.workunit.client.0.smithi139.stdout:6/422: mknod d8/c90 0 2024-09-16T00:09:47.035 INFO:tasks.workunit.client.0.smithi139.stdout:9/660: dread - d19/d1a/d20/d23/dca/d71/da3/fce zero size 2024-09-16T00:09:47.035 INFO:tasks.workunit.client.0.smithi139.stdout:8/604: dread - d17/d1f/d3a/db7/d40/fb4 zero size 2024-09-16T00:09:47.037 INFO:tasks.workunit.client.0.smithi139.stdout:2/666: dwrite d11/d45/fb8 [0,4194304] 0 2024-09-16T00:09:47.038 INFO:tasks.workunit.client.0.smithi139.stdout:7/534: dread d5/d16/d22/d25/d2d/fa8 [0,4194304] 0 2024-09-16T00:09:47.039 INFO:tasks.workunit.client.0.smithi139.stdout:4/526: symlink d12/lab 0 2024-09-16T00:09:47.039 INFO:tasks.workunit.client.0.smithi139.stdout:1/474: write d7/f10 [5252590,114043] 0 2024-09-16T00:09:47.039 INFO:tasks.workunit.client.0.smithi139.stdout:8/605: read - d17/d1f/d3a/db7/d40/fb4 zero size 2024-09-16T00:09:47.040 INFO:tasks.workunit.client.0.smithi139.stdout:0/557: getdents d6/d1c/d41/d44/dd/d8a 0 2024-09-16T00:09:47.042 INFO:tasks.workunit.client.0.smithi139.stdout:9/661: mkdir d19/d1a/d20/d23/dca/d71/dc7/de1 0 2024-09-16T00:09:47.043 INFO:tasks.workunit.client.0.smithi139.stdout:2/667: mkdir d11/d30/d57/d5d/d73/dc3/de8 0 2024-09-16T00:09:47.045 INFO:tasks.workunit.client.0.smithi139.stdout:1/475: mknod d7/d8/d40/d48/d53/d61/c8e 0 2024-09-16T00:09:47.046 INFO:tasks.workunit.client.0.smithi139.stdout:7/535: dread d5/d16/d22/d25/d2d/fa8 [0,4194304] 0 2024-09-16T00:09:47.046 INFO:tasks.workunit.client.0.smithi139.stdout:0/558: creat d6/d1c/d41/d44/dd/d56/fac x:0 0 0 2024-09-16T00:09:47.047 INFO:tasks.workunit.client.0.smithi139.stdout:9/662: dread - d19/d1a/d20/d23/d2d/d35/d42/d3a/fcf zero size 2024-09-16T00:09:47.047 INFO:tasks.workunit.client.0.smithi139.stdout:4/527: mkdir d12/d47/d49/d60/dac 0 2024-09-16T00:09:47.048 INFO:tasks.workunit.client.0.smithi139.stdout:2/668: creat d11/d30/d57/d29/da5/d81/fe9 x:0 0 0 2024-09-16T00:09:47.050 INFO:tasks.workunit.client.0.smithi139.stdout:0/559: mkdir d6/d1c/d41/d7c/dad 0 2024-09-16T00:09:47.051 INFO:tasks.workunit.client.0.smithi139.stdout:7/536: symlink d5/d16/d22/lba 0 2024-09-16T00:09:47.051 INFO:tasks.workunit.client.0.smithi139.stdout:8/606: dwrite d17/d1a/d2d/f3e [0,4194304] 0 2024-09-16T00:09:47.052 INFO:tasks.workunit.client.0.smithi139.stdout:6/423: truncate d8/d29/d83/d26/d28/f40 701005 0 2024-09-16T00:09:47.053 INFO:tasks.workunit.client.0.smithi139.stdout:4/528: symlink d12/d4a/d4d/da9/lad 0 2024-09-16T00:09:47.053 INFO:tasks.workunit.client.0.smithi139.stdout:1/476: rename d7/d8/d40/d46/c8a to d7/d8/d40/d48/d4a/d74/c8f 0 2024-09-16T00:09:47.055 INFO:tasks.workunit.client.0.smithi139.stdout:8/607: mkdir d17/d1f/d3a/dcd 0 2024-09-16T00:09:47.056 INFO:tasks.workunit.client.0.smithi139.stdout:8/608: dread - d17/d1f/d3a/f45 zero size 2024-09-16T00:09:47.061 INFO:tasks.workunit.client.0.smithi139.stdout:1/477: write d7/d8/d40/d46/f5b [2724096,806] 0 2024-09-16T00:09:47.061 INFO:tasks.workunit.client.0.smithi139.stdout:0/560: creat d6/d1c/d41/d44/fae x:0 0 0 2024-09-16T00:09:47.061 INFO:tasks.workunit.client.0.smithi139.stdout:8/609: symlink d17/d1a/d50/d86/d9d/lce 0 2024-09-16T00:09:47.061 INFO:tasks.workunit.client.0.smithi139.stdout:5/710: sync 2024-09-16T00:09:47.061 INFO:tasks.workunit.client.0.smithi139.stdout:3/466: sync 2024-09-16T00:09:47.062 INFO:tasks.workunit.client.0.smithi139.stdout:4/529: symlink d12/d47/d57/d7f/d9e/da8/lae 0 2024-09-16T00:09:47.063 INFO:tasks.workunit.client.0.smithi139.stdout:7/537: dread d5/d16/d22/d25/d2d/d50/d44/f57 [0,4194304] 0 2024-09-16T00:09:47.063 INFO:tasks.workunit.client.0.smithi139.stdout:9/663: dread d19/d1a/d20/d23/f33 [0,4194304] 0 2024-09-16T00:09:47.064 INFO:tasks.workunit.client.0.smithi139.stdout:6/424: creat d8/d29/d83/d26/f91 x:0 0 0 2024-09-16T00:09:47.064 INFO:tasks.workunit.client.0.smithi139.stdout:0/561: creat d6/d1c/d41/d44/d58/faf x:0 0 0 2024-09-16T00:09:47.065 INFO:tasks.workunit.client.0.smithi139.stdout:1/478: write d7/d8/d27/f79 [6515351,71459] 0 2024-09-16T00:09:47.065 INFO:tasks.workunit.client.0.smithi139.stdout:8/610: chown d17/d1f/d3a/db7/d40/d4f/l64 4 1 2024-09-16T00:09:47.066 INFO:tasks.workunit.client.0.smithi139.stdout:2/669: getdents d11/d30/d57/d29/da5/dc6 0 2024-09-16T00:09:47.068 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:47.066+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:47.068 INFO:tasks.workunit.client.0.smithi139.stdout:3/467: unlink d5/d7/d23/d27/c7b 0 2024-09-16T00:09:47.068 INFO:tasks.workunit.client.0.smithi139.stdout:9/664: write d19/d1a/d20/d23/d2d/d9a/fac [3387815,107283] 0 2024-09-16T00:09:47.069 INFO:tasks.workunit.client.0.smithi139.stdout:2/670: write d11/f16 [767490,44470] 0 2024-09-16T00:09:47.075 INFO:tasks.workunit.client.0.smithi139.stdout:0/562: dwrite d6/d1c/d41/d44/d6b/f6c [0,4194304] 0 2024-09-16T00:09:47.076 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:47.076 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:47.077 INFO:tasks.workunit.client.0.smithi139.stdout:5/711: rename d1/d21/d49/d53/d6a/d7b/da1/ccb to d1/d21/dde/cfb 0 2024-09-16T00:09:47.078 INFO:tasks.workunit.client.0.smithi139.stdout:6/425: write d8/d20/f2b [793162,82729] 0 2024-09-16T00:09:47.079 INFO:tasks.workunit.client.0.smithi139.stdout:3/468: symlink d5/d5f/la9 0 2024-09-16T00:09:47.081 INFO:tasks.workunit.client.0.smithi139.stdout:2/671: mkdir d11/d2f/dcc/dea 0 2024-09-16T00:09:47.081 INFO:tasks.workunit.client.0.smithi139.stdout:6/426: write d8/d29/d83/d26/d45/d42/f76 [352390,55212] 0 2024-09-16T00:09:47.082 INFO:tasks.workunit.client.0.smithi139.stdout:2/672: read ff [953384,93981] 0 2024-09-16T00:09:47.083 INFO:tasks.workunit.client.0.smithi139.stdout:7/538: write d5/d16/d22/d25/d2d/d50/d44/f57 [1552948,96407] 0 2024-09-16T00:09:47.086 INFO:tasks.workunit.client.0.smithi139.stdout:8/611: mknod d17/d1f/d3a/db0/ccf 0 2024-09-16T00:09:47.086 INFO:tasks.workunit.client.0.smithi139.stdout:4/530: creat d12/d47/faf x:0 0 0 2024-09-16T00:09:47.087 INFO:tasks.workunit.client.0.smithi139.stdout:4/531: fdatasync d12/d47/d49/d60/f76 0 2024-09-16T00:09:47.088 INFO:tasks.workunit.client.0.smithi139.stdout:3/469: unlink d5/d7/d23/d27/d2f/f74 0 2024-09-16T00:09:47.089 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:09:47.091 INFO:tasks.workunit.client.0.smithi139.stdout:4/532: dread d12/f31 [0,4194304] 0 2024-09-16T00:09:47.091 INFO:tasks.workunit.client.0.smithi139.stdout:4/533: readlink d12/d47/l56 0 2024-09-16T00:09:47.091 INFO:tasks.workunit.client.0.smithi139.stdout:4/534: fdatasync d12/d47/d57/f6e 0 2024-09-16T00:09:47.091 INFO:tasks.workunit.client.0.smithi139.stdout:2/673: rmdir d11/d30/dce 39 2024-09-16T00:09:47.092 INFO:tasks.workunit.client.0.smithi139.stdout:0/563: fdatasync d6/d1c/d41/d44/dd/d8a/f92 0 2024-09-16T00:09:47.094 INFO:tasks.workunit.client.0.smithi139.stdout:5/712: link d1/d21/d49/d53/d6a/d7b/d78/dca/fcf d1/d13/d3f/d9f/d30/d33/d43/ffc 0 2024-09-16T00:09:47.096 INFO:tasks.workunit.client.0.smithi139.stdout:6/427: creat d8/d20/d68/f92 x:0 0 0 2024-09-16T00:09:47.096 INFO:tasks.workunit.client.0.smithi139.stdout:1/479: link d7/d1b/c3d d7/d8/d40/d46/d89/c90 0 2024-09-16T00:09:47.097 INFO:tasks.workunit.client.0.smithi139.stdout:3/470: mknod d5/d7/d23/d27/d2f/d30/d47/caa 0 2024-09-16T00:09:47.097 INFO:tasks.workunit.client.0.smithi139.stdout:9/665: creat d19/d1a/d20/d23/d2d/d35/d42/d3a/fe2 x:0 0 0 2024-09-16T00:09:47.098 INFO:tasks.workunit.client.0.smithi139.stdout:3/471: read - d5/d7/dc/d9c/f9d zero size 2024-09-16T00:09:47.099 INFO:tasks.workunit.client.0.smithi139.stdout:2/674: mkdir d11/d2f/deb 0 2024-09-16T00:09:47.100 INFO:tasks.workunit.client.0.smithi139.stdout:8/612: write d17/d54/fbe [1026477,32817] 0 2024-09-16T00:09:47.102 INFO:tasks.workunit.client.0.smithi139.stdout:7/539: rename d5/d16/d22/d25/d2d/f36 to d5/d16/d22/d25/d2d/d50/d44/fbb 0 2024-09-16T00:09:47.102 INFO:tasks.workunit.client.0.smithi139.stdout:8/613: write d17/d1a/d2d/fa9 [272916,112468] 0 2024-09-16T00:09:47.105 INFO:tasks.workunit.client.0.smithi139.stdout:5/713: dread d1/fc5 [0,4194304] 0 2024-09-16T00:09:47.105 INFO:tasks.workunit.client.0.smithi139.stdout:6/428: mknod d8/d56/c93 0 2024-09-16T00:09:47.106 INFO:tasks.workunit.client.0.smithi139.stdout:1/480: mkdir d7/d8/d40/d46/d91 0 2024-09-16T00:09:47.107 INFO:tasks.workunit.client.0.smithi139.stdout:3/472: mkdir d5/d7/d23/d27/dab 0 2024-09-16T00:09:47.107 INFO:tasks.workunit.client.0.smithi139.stdout:1/481: truncate d7/d8/d40/d47/f50 1188347 0 2024-09-16T00:09:47.108 INFO:tasks.workunit.client.0.smithi139.stdout:9/666: symlink d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/le3 0 2024-09-16T00:09:47.108 INFO:tasks.workunit.client.0.smithi139.stdout:3/473: chown d5/d7/d23/d27/d2f/d30/c3e 10845251 1 2024-09-16T00:09:47.130 INFO:tasks.workunit.client.0.smithi139.stdout:4/535: rename d12/d47/d57/d7f/d9e/da8 to d12/d4a/d4d/db0 0 2024-09-16T00:09:47.133 INFO:tasks.workunit.client.0.smithi139.stdout:6/429: symlink d8/d1f/l94 0 2024-09-16T00:09:47.133 INFO:tasks.workunit.client.0.smithi139.stdout:3/474: chown d5/d7/l73 218824 1 2024-09-16T00:09:47.133 INFO:tasks.workunit.client.0.smithi139.stdout:0/564: link d6/d1c/d41/d44/f3b d6/d1c/d41/d44/dd/fb0 0 2024-09-16T00:09:47.142 INFO:tasks.workunit.client.0.smithi139.stdout:5/714: symlink d1/d13/d3f/d9f/d30/dd4/lfd 0 2024-09-16T00:09:47.144 INFO:tasks.workunit.client.0.smithi139.stdout:7/540: creat d5/fbc x:0 0 0 2024-09-16T00:09:47.144 INFO:tasks.workunit.client.0.smithi139.stdout:9/667: creat d19/d1a/d20/d23/dca/d9f/dd9/fe4 x:0 0 0 2024-09-16T00:09:47.145 INFO:tasks.workunit.client.0.smithi139.stdout:6/430: read d8/d20/d27/f58 [3904599,7385] 0 2024-09-16T00:09:47.145 INFO:tasks.workunit.client.0.smithi139.stdout:0/565: symlink d6/d5a/d70/lb1 0 2024-09-16T00:09:47.146 INFO:tasks.workunit.client.0.smithi139.stdout:1/482: rename d7/d8/d40/d48/d67/l6b to d7/d8/l92 0 2024-09-16T00:09:47.146 INFO:tasks.workunit.client.0.smithi139.stdout:5/715: chown d1/d21/d49/db2/fba 1 1 2024-09-16T00:09:47.147 INFO:tasks.workunit.client.0.smithi139.stdout:4/536: link d12/d2b/l41 d12/d47/d49/d60/dac/lb1 0 2024-09-16T00:09:47.147 INFO:tasks.workunit.client.0.smithi139.stdout:3/475: rename d5/d7/d23/d27 to d5/d7/d23/d27/d2f/d30/d47/dac 22 2024-09-16T00:09:47.148 INFO:tasks.workunit.client.0.smithi139.stdout:2/675: dwrite d11/d30/f36 [0,4194304] 0 2024-09-16T00:09:47.151 INFO:tasks.workunit.client.0.smithi139.stdout:9/668: rename d19/d1a/d20/d23/c72 to d19/d86/ce5 0 2024-09-16T00:09:47.151 INFO:tasks.workunit.client.0.smithi139.stdout:0/566: write d6/d1c/d41/d63/d82/f85 [807233,29423] 0 2024-09-16T00:09:47.154 INFO:tasks.workunit.client.0.smithi139.stdout:7/541: getdents d5/d16/d22/d25 0 2024-09-16T00:09:47.155 INFO:tasks.workunit.client.0.smithi139.stdout:6/431: rename d8/f8d to d8/d29/d83/d26/d28/f95 0 2024-09-16T00:09:47.155 INFO:tasks.workunit.client.0.smithi139.stdout:7/542: fdatasync d5/d26/d42/f80 0 2024-09-16T00:09:47.156 INFO:tasks.workunit.client.0.smithi139.stdout:2/676: symlink d11/d45/dde/lec 0 2024-09-16T00:09:47.158 INFO:tasks.workunit.client.0.smithi139.stdout:3/476: rmdir d5/d7/d23/d27/d2f/d30/d4d/d95 0 2024-09-16T00:09:47.159 INFO:tasks.workunit.client.0.smithi139.stdout:1/483: dwrite d7/f28 [8388608,4194304] 0 2024-09-16T00:09:47.161 INFO:tasks.workunit.client.0.smithi139.stdout:7/543: dread - d5/d16/d22/d25/d2d/d81/f82 zero size 2024-09-16T00:09:47.164 INFO:tasks.workunit.client.0.smithi139.stdout:9/669: unlink d19/d1a/d20/d23/d2d/d35/d42/f5e 0 2024-09-16T00:09:47.169 INFO:tasks.workunit.client.0.smithi139.stdout:5/716: dread d1/f1a [0,4194304] 0 2024-09-16T00:09:47.170 INFO:tasks.workunit.client.0.smithi139.stdout:4/537: rename d12/d4a to d12/d47/db2 0 2024-09-16T00:09:47.170 INFO:tasks.workunit.client.0.smithi139.stdout:1/484: symlink d7/d8/d40/d46/d91/l93 0 2024-09-16T00:09:47.171 INFO:tasks.workunit.client.0.smithi139.stdout:9/670: fsync d19/d1a/d20/d23/dca/d71/d32/d4e/f8a 0 2024-09-16T00:09:47.171 INFO:tasks.workunit.client.0.smithi139.stdout:0/567: dwrite d6/d5a/d70/f4f [0,4194304] 0 2024-09-16T00:09:47.174 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:47.173+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:47.178 INFO:tasks.workunit.client.0.smithi139.stdout:3/477: dwrite d5/d7/d23/d27/d2f/d30/d41/f6b [0,4194304] 0 2024-09-16T00:09:47.178 INFO:tasks.workunit.client.0.smithi139.stdout:2/677: rename d11/d32/fa8 to d11/d2f/fed 0 2024-09-16T00:09:47.181 INFO:tasks.workunit.client.0.smithi139.stdout:7/544: link d5/fbc d5/d16/d22/d25/d2d/d50/fbd 0 2024-09-16T00:09:47.184 INFO:tasks.workunit.client.0.smithi139.stdout:6/432: link d8/d29/d83/d26/d45/d59/c87 d8/d29/d83/d26/d45/d65/d6e/c96 0 2024-09-16T00:09:47.187 INFO:tasks.workunit.client.0.smithi139.stdout:4/538: dwrite d12/d47/f32 [0,4194304] 0 2024-09-16T00:09:47.191 INFO:tasks.workunit.client.0.smithi139.stdout:3/478: write d5/d7/d23/d27/d2f/d30/d4c/d6d/f79 [3162397,44473] 0 2024-09-16T00:09:47.191 INFO:tasks.workunit.client.0.smithi139.stdout:5/717: mknod d1/d13/d94/cfe 0 2024-09-16T00:09:47.192 INFO:tasks.workunit.client.0.smithi139.stdout:1/485: dread d7/f12 [0,4194304] 0 2024-09-16T00:09:47.193 INFO:tasks.workunit.client.0.smithi139.stdout:5/718: chown d1/d13/d3f/d9f/d73/d93/led 119 1 2024-09-16T00:09:47.194 INFO:tasks.workunit.client.0.smithi139.stdout:7/545: symlink d5/d16/d22/d25/d2d/da4/lbe 0 2024-09-16T00:09:47.197 INFO:tasks.workunit.client.0.smithi139.stdout:1/486: dread d7/d8/f1a [4194304,4194304] 0 2024-09-16T00:09:47.198 INFO:tasks.workunit.client.0.smithi139.stdout:4/539: write d12/d80/f9a [3670618,113912] 0 2024-09-16T00:09:47.198 INFO:tasks.workunit.client.0.smithi139.stdout:2/678: mkdir d11/d2f/dcc/dea/dee 0 2024-09-16T00:09:47.199 INFO:tasks.workunit.client.0.smithi139.stdout:5/719: creat d1/d21/d49/d87/fff x:0 0 0 2024-09-16T00:09:47.200 INFO:tasks.workunit.client.0.smithi139.stdout:9/671: creat d19/d1a/d20/d23/d2d/d35/d42/d3a/fe6 x:0 0 0 2024-09-16T00:09:47.201 INFO:tasks.workunit.client.0.smithi139.stdout:7/546: rename d5/d26/d42/l4d to d5/d26/d46/d6b/da7/lbf 0 2024-09-16T00:09:47.201 INFO:tasks.workunit.client.0.smithi139.stdout:0/568: link d6/d5a/d70/l9d d6/d1c/d41/d63/d82/lb2 0 2024-09-16T00:09:47.202 INFO:tasks.workunit.client.0.smithi139.stdout:4/540: truncate d12/d47/db2/f54 1468641 0 2024-09-16T00:09:47.205 INFO:tasks.workunit.client.0.smithi139.stdout:1/487: chown d7/d8/d27/l4d 18720314 1 2024-09-16T00:09:47.205 INFO:tasks.workunit.client.0.smithi139.stdout:4/541: dread - d12/d47/d57/f5f zero size 2024-09-16T00:09:47.206 INFO:tasks.workunit.client.0.smithi139.stdout:1/488: read d7/f12 [3623412,5388] 0 2024-09-16T00:09:47.207 INFO:tasks.workunit.client.0.smithi139.stdout:3/479: dread d5/d7/d23/d27/d2f/d30/d4c/d50/f54 [0,4194304] 0 2024-09-16T00:09:47.208 INFO:tasks.workunit.client.0.smithi139.stdout:5/720: rename d1/d13/d22/l23 to d1/d21/d49/d53/d6a/d7b/d78/l100 0 2024-09-16T00:09:47.210 INFO:tasks.workunit.client.0.smithi139.stdout:3/480: write d5/d7/d23/d27/d2f/f7a [88803,69423] 0 2024-09-16T00:09:47.210 INFO:tasks.workunit.client.0.smithi139.stdout:9/672: mkdir d19/d1a/d20/d23/d2d/d35/d42/de7 0 2024-09-16T00:09:47.211 INFO:tasks.workunit.client.0.smithi139.stdout:0/569: creat d6/d1c/d41/d44/dd/d8a/fb3 x:0 0 0 2024-09-16T00:09:47.213 INFO:tasks.workunit.client.0.smithi139.stdout:1/489: chown d7/d8/l92 3 1 2024-09-16T00:09:47.214 INFO:tasks.workunit.client.0.smithi139.stdout:0/570: chown d6/d5a/d70/f53 6 1 2024-09-16T00:09:47.215 INFO:tasks.workunit.client.0.smithi139.stdout:4/542: dwrite d12/f42 [0,4194304] 0 2024-09-16T00:09:47.215 INFO:tasks.workunit.client.0.smithi139.stdout:7/547: rename d5/f12 to d5/d16/d22/d25/d2d/dab/fc0 0 2024-09-16T00:09:47.217 INFO:tasks.workunit.client.0.smithi139.stdout:2/679: dread d11/d30/d57/d50/f6a [0,4194304] 0 2024-09-16T00:09:47.217 INFO:tasks.workunit.client.0.smithi139.stdout:5/721: creat d1/d13/d22/da9/f101 x:0 0 0 2024-09-16T00:09:47.219 INFO:tasks.workunit.client.0.smithi139.stdout:0/571: stat d6/d1c/d41/d44/d58 0 2024-09-16T00:09:47.221 INFO:tasks.workunit.client.0.smithi139.stdout:1/490: creat d7/d8/d40/d48/d67/d7d/f94 x:0 0 0 2024-09-16T00:09:47.223 INFO:tasks.workunit.client.0.smithi139.stdout:4/543: creat d12/d47/fb3 x:0 0 0 2024-09-16T00:09:47.224 INFO:tasks.workunit.client.0.smithi139.stdout:1/491: readlink d7/d8/l3e 0 2024-09-16T00:09:47.224 INFO:tasks.workunit.client.0.smithi139.stdout:2/680: mknod d11/d30/d57/d5d/cef 0 2024-09-16T00:09:47.226 INFO:tasks.workunit.client.0.smithi139.stdout:7/548: mknod d5/d16/d22/d25/d2d/dab/cc1 0 2024-09-16T00:09:47.226 INFO:tasks.workunit.client.0.smithi139.stdout:6/433: dwrite d8/d29/d83/d26/d28/f40 [0,4194304] 0 2024-09-16T00:09:47.226 INFO:tasks.workunit.client.0.smithi139.stdout:8/614: sync 2024-09-16T00:09:47.228 INFO:tasks.workunit.client.0.smithi139.stdout:3/481: dwrite d5/d7/d23/f17 [0,4194304] 0 2024-09-16T00:09:47.229 INFO:tasks.workunit.client.0.smithi139.stdout:8/615: chown d17/f99 29 1 2024-09-16T00:09:47.229 INFO:tasks.workunit.client.0.smithi139.stdout:6/434: chown d8/d29/d83/d26/d45/d85 0 1 2024-09-16T00:09:47.237 INFO:tasks.workunit.client.0.smithi139.stdout:8/616: read - d17/d1a/d50/d86/d9d/daf/fc7 zero size 2024-09-16T00:09:47.237 INFO:tasks.workunit.client.0.smithi139.stdout:5/722: getdents d1/d21/dde 0 2024-09-16T00:09:47.243 INFO:tasks.workunit.client.0.smithi139.stdout:9/673: dread d19/d1a/d20/d23/d2d/d9a/fcc [0,4194304] 0 2024-09-16T00:09:47.243 INFO:tasks.workunit.client.0.smithi139.stdout:7/549: rename d5/d16/d22/d25/d2d to d5/d16/d6d/d7f/dc2 0 2024-09-16T00:09:47.244 INFO:tasks.workunit.client.0.smithi139.stdout:3/482: dread d5/d7/f28 [4194304,4194304] 0 2024-09-16T00:09:47.244 INFO:tasks.workunit.client.0.smithi139.stdout:0/572: dwrite d6/d1c/d41/d44/dd/f39 [0,4194304] 0 2024-09-16T00:09:47.244 INFO:tasks.workunit.client.0.smithi139.stdout:5/723: fsync d1/d13/d3f/d9f/d30/d33/d43/f9c 0 2024-09-16T00:09:47.245 INFO:tasks.workunit.client.0.smithi139.stdout:1/492: write d7/d8/d40/d47/f5e [773821,23438] 0 2024-09-16T00:09:47.246 INFO:tasks.workunit.client.0.smithi139.stdout:2/681: link d11/d30/c70 d11/d30/d57/cf0 0 2024-09-16T00:09:47.246 INFO:tasks.workunit.client.0.smithi139.stdout:6/435: dread d8/d20/d4d/f60 [0,4194304] 0 2024-09-16T00:09:47.247 INFO:tasks.workunit.client.0.smithi139.stdout:8/617: write d17/d1a/d57/f98 [530936,130369] 0 2024-09-16T00:09:47.249 INFO:tasks.workunit.client.0.smithi139.stdout:4/544: dread d12/d2b/f3a [0,4194304] 0 2024-09-16T00:09:47.250 INFO:tasks.workunit.client.0.smithi139.stdout:5/724: dread - d1/d13/d22/da9/f101 zero size 2024-09-16T00:09:47.250 INFO:tasks.workunit.client.0.smithi139.stdout:3/483: chown d5/d7/d23/d35/da7/c88 1524496152 1 2024-09-16T00:09:47.251 INFO:tasks.workunit.client.0.smithi139.stdout:7/550: chown d5/d16/d6d/d7f/dc2/d50/f67 0 1 2024-09-16T00:09:47.251 INFO:tasks.workunit.client.0.smithi139.stdout:1/493: read - d7/d8/d40/d48/d4a/d74/f7e zero size 2024-09-16T00:09:47.251 INFO:tasks.workunit.client.0.smithi139.stdout:8/618: write d17/d1a/fbc [3560,86812] 0 2024-09-16T00:09:47.252 INFO:tasks.workunit.client.0.smithi139.stdout:0/573: rename d6/d1c/d45 to d6/d42/db4 0 2024-09-16T00:09:47.254 INFO:tasks.workunit.client.0.smithi139.stdout:2/682: mknod d11/d2f/dcc/cf1 0 2024-09-16T00:09:47.256 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:47.256 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:47.257 INFO:tasks.workunit.client.0.smithi139.stdout:4/545: creat d12/d47/db2/d4d/d69/fb4 x:0 0 0 2024-09-16T00:09:47.260 INFO:tasks.workunit.client.0.smithi139.stdout:3/484: dread d5/d7/d23/d27/f83 [0,4194304] 0 2024-09-16T00:09:47.260 INFO:tasks.workunit.client.0.smithi139.stdout:6/436: link d8/d29/f49 d8/d20/d27/f97 0 2024-09-16T00:09:47.260 INFO:tasks.workunit.client.0.smithi139.stdout:4/546: read - d12/d47/faf zero size 2024-09-16T00:09:47.262 INFO:tasks.workunit.client.0.smithi139.stdout:2/683: write d11/d30/d57/d29/d2b/fa7 [13817,24400] 0 2024-09-16T00:09:47.262 INFO:tasks.workunit.client.0.smithi139.stdout:1/494: rmdir d7/d8/d40/d48/d67/d7d 39 2024-09-16T00:09:47.263 INFO:tasks.workunit.client.0.smithi139.stdout:8/619: symlink d17/d1f/d3a/db7/d40/ld0 0 2024-09-16T00:09:47.264 INFO:tasks.workunit.client.0.smithi139.stdout:9/674: dwrite d19/d1a/d20/d23/dca/d71/da3/fce [0,4194304] 0 2024-09-16T00:09:47.265 INFO:tasks.workunit.client.0.smithi139.stdout:0/574: read - d6/d1c/d41/f72 zero size 2024-09-16T00:09:47.267 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:09:47.268 INFO:tasks.workunit.client.0.smithi139.stdout:2/684: write d11/fdd [943075,78846] 0 2024-09-16T00:09:47.268 INFO:tasks.workunit.client.0.smithi139.stdout:5/725: dwrite d1/d13/ff4 [0,4194304] 0 2024-09-16T00:09:47.269 INFO:tasks.workunit.client.0.smithi139.stdout:3/485: fdatasync d5/d7/d23/d35/f46 0 2024-09-16T00:09:47.269 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:47.269+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:47.271 INFO:tasks.workunit.client.0.smithi139.stdout:5/726: chown d1/d13/f24 138427031 1 2024-09-16T00:09:47.272 INFO:tasks.workunit.client.0.smithi139.stdout:1/495: truncate d7/d8/d40/d48/d4a/d74/f7e 682810 0 2024-09-16T00:09:47.273 INFO:tasks.workunit.client.0.smithi139.stdout:0/575: write d6/d1c/d41/d44/dd/f39 [4872842,103761] 0 2024-09-16T00:09:47.274 INFO:tasks.workunit.client.0.smithi139.stdout:4/547: creat d12/d47/d57/fb5 x:0 0 0 2024-09-16T00:09:47.276 INFO:tasks.workunit.client.0.smithi139.stdout:3/486: symlink d5/lad 0 2024-09-16T00:09:47.278 INFO:tasks.workunit.client.0.smithi139.stdout:6/437: write d8/d29/d83/d26/f61 [2236735,28513] 0 2024-09-16T00:09:47.280 INFO:tasks.workunit.client.0.smithi139.stdout:5/727: chown d1/d21/d49/d53/d6a/d7b/d78/f8e 93 1 2024-09-16T00:09:47.280 INFO:tasks.workunit.client.0.smithi139.stdout:8/620: getdents d17/d1a/d50/d86 0 2024-09-16T00:09:47.281 INFO:tasks.workunit.client.0.smithi139.stdout:0/576: creat d6/d1c/d41/d7c/fb5 x:0 0 0 2024-09-16T00:09:47.281 INFO:tasks.workunit.client.0.smithi139.stdout:4/548: write d12/d47/db2/f77 [5166677,80250] 0 2024-09-16T00:09:47.283 INFO:tasks.workunit.client.0.smithi139.stdout:7/551: dread d5/d26/d42/f66 [4194304,4194304] 0 2024-09-16T00:09:47.286 INFO:tasks.workunit.client.0.smithi139.stdout:4/549: dread - d12/d47/faf zero size 2024-09-16T00:09:47.287 INFO:tasks.workunit.client.0.smithi139.stdout:6/438: readlink d8/d29/d83/d26/d28/l6c 0 2024-09-16T00:09:47.288 INFO:tasks.workunit.client.0.smithi139.stdout:3/487: write d5/d7/d23/d27/d2f/d30/f42 [1531596,42783] 0 2024-09-16T00:09:47.288 INFO:tasks.workunit.client.0.smithi139.stdout:0/577: write d6/d42/db4/f8e [90913,8827] 0 2024-09-16T00:09:47.289 INFO:tasks.workunit.client.0.smithi139.stdout:2/685: dwrite d11/d30/d57/d29/da5/d81/f87 [0,4194304] 0 2024-09-16T00:09:47.290 INFO:tasks.workunit.client.0.smithi139.stdout:6/439: chown d8/d1f 10623001 1 2024-09-16T00:09:47.290 INFO:tasks.workunit.client.0.smithi139.stdout:9/675: dread d19/f26 [0,4194304] 0 2024-09-16T00:09:47.292 INFO:tasks.workunit.client.0.smithi139.stdout:4/550: read - d12/d2b/d8c/f92 zero size 2024-09-16T00:09:47.292 INFO:tasks.workunit.client.0.smithi139.stdout:7/552: mknod d5/d16/d6d/d7f/dc2/d81/cc3 0 2024-09-16T00:09:47.294 INFO:tasks.workunit.client.0.smithi139.stdout:1/496: dwrite d7/d8/d40/d48/f65 [0,4194304] 0 2024-09-16T00:09:47.298 INFO:tasks.workunit.client.0.smithi139.stdout:0/578: dread - d6/d1c/d41/d7c/fb5 zero size 2024-09-16T00:09:47.298 INFO:tasks.workunit.client.0.smithi139.stdout:4/551: dread - d12/d2b/d8c/f9c zero size 2024-09-16T00:09:47.298 INFO:tasks.workunit.client.0.smithi139.stdout:7/553: readlink d5/l7e 0 2024-09-16T00:09:47.301 INFO:tasks.workunit.client.0.smithi139.stdout:2/686: stat d11/d30/d57/d29/da5/daa/de1 0 2024-09-16T00:09:47.303 INFO:tasks.workunit.client.0.smithi139.stdout:5/728: dwrite d1/d13/d3f/f77 [0,4194304] 0 2024-09-16T00:09:47.305 INFO:tasks.workunit.client.0.smithi139.stdout:3/488: rename d5/d7/d23/d27/d2f/d30/d41/c97 to d5/d7/d23/cae 0 2024-09-16T00:09:47.305 INFO:tasks.workunit.client.0.smithi139.stdout:8/621: dwrite d17/d1f/d3a/dad/fb9 [0,4194304] 0 2024-09-16T00:09:47.307 INFO:tasks.workunit.client.0.smithi139.stdout:8/622: write d17/d54/fbe [2068197,38076] 0 2024-09-16T00:09:47.307 INFO:tasks.workunit.client.0.smithi139.stdout:3/489: chown d5/d7/d23/d27/d2f/d30/d4c/d50/d75 343 1 2024-09-16T00:09:47.309 INFO:tasks.workunit.client.0.smithi139.stdout:8/623: readlink d17/d1a/l26 0 2024-09-16T00:09:47.309 INFO:tasks.workunit.client.0.smithi139.stdout:4/552: creat d12/d47/db2/d4d/da9/fb6 x:0 0 0 2024-09-16T00:09:47.309 INFO:tasks.workunit.client.0.smithi139.stdout:2/687: creat d11/d30/dbf/ff2 x:0 0 0 2024-09-16T00:09:47.310 INFO:tasks.workunit.client.0.smithi139.stdout:6/440: link d8/d29/d83/f2c d8/d29/d83/f98 0 2024-09-16T00:09:47.310 INFO:tasks.workunit.client.0.smithi139.stdout:7/554: write d5/d16/d22/d25/d63/f91 [853562,121131] 0 2024-09-16T00:09:47.311 INFO:tasks.workunit.client.0.smithi139.stdout:4/553: fsync d12/d47/db2/d4d/f81 0 2024-09-16T00:09:47.311 INFO:tasks.workunit.client.0.smithi139.stdout:8/624: rmdir d17/d1f/d3a/db0 39 2024-09-16T00:09:47.312 INFO:tasks.workunit.client.0.smithi139.stdout:9/676: getdents d19/d1a/d20/d23/d2d/d35/d42 0 2024-09-16T00:09:47.315 INFO:tasks.workunit.client.0.smithi139.stdout:4/554: mknod d12/d47/d57/d7f/cb7 0 2024-09-16T00:09:47.316 INFO:tasks.workunit.client.0.smithi139.stdout:9/677: write d19/d1a/d20/d23/d2d/d35/d60/fb3 [1502215,124275] 0 2024-09-16T00:09:47.320 INFO:tasks.workunit.client.0.smithi139.stdout:2/688: dread d11/d30/d57/d5d/d73/f8e [0,4194304] 0 2024-09-16T00:09:47.321 INFO:tasks.workunit.client.0.smithi139.stdout:9/678: unlink d19/d86/fe0 0 2024-09-16T00:09:47.323 INFO:tasks.workunit.client.0.smithi139.stdout:8/625: dread d17/d1f/d3a/db7/d40/d4f/f65 [0,4194304] 0 2024-09-16T00:09:47.323 INFO:tasks.workunit.client.0.smithi139.stdout:4/555: rename d12/f8d to d12/d2b/d8c/fb8 0 2024-09-16T00:09:47.324 INFO:tasks.workunit.client.0.smithi139.stdout:7/555: dwrite d5/d26/d42/f5b [0,4194304] 0 2024-09-16T00:09:47.330 INFO:tasks.workunit.client.0.smithi139.stdout:3/490: dread d5/d7/dc/f12 [0,4194304] 0 2024-09-16T00:09:47.331 INFO:tasks.workunit.client.0.smithi139.stdout:4/556: rename d12/d47/d49/l96 to d12/d47/d57/d7f/lb9 0 2024-09-16T00:09:47.333 INFO:tasks.workunit.client.0.smithi139.stdout:2/689: dwrite d11/d30/d57/d5d/fb4 [0,4194304] 0 2024-09-16T00:09:47.333 INFO:tasks.workunit.client.0.smithi139.stdout:6/441: truncate d8/d29/d83/d26/d45/d65/f13 548207 0 2024-09-16T00:09:47.337 INFO:tasks.workunit.client.0.smithi139.stdout:8/626: creat d17/d1f/d3a/db0/fd1 x:0 0 0 2024-09-16T00:09:47.337 INFO:tasks.workunit.client.0.smithi139.stdout:9/679: dwrite d19/d1a/d20/d23/d2d/d35/d42/d3a/fcf [0,4194304] 0 2024-09-16T00:09:47.339 INFO:tasks.workunit.client.0.smithi139.stdout:9/680: chown d19/d1a/d20/d23/d2d/d35/fb1 77 1 2024-09-16T00:09:47.344 INFO:tasks.workunit.client.0.smithi139.stdout:4/557: unlink d12/d47/d57/c97 0 2024-09-16T00:09:47.345 INFO:tasks.workunit.client.0.smithi139.stdout:4/558: chown d12/d47/faf 0 1 2024-09-16T00:09:47.346 INFO:tasks.workunit.client.0.smithi139.stdout:6/442: mknod d8/c99 0 2024-09-16T00:09:47.350 INFO:tasks.workunit.client.0.smithi139.stdout:8/627: rename d17/d1f/d3a/db7/f6c to d17/d1a/d50/fd2 0 2024-09-16T00:09:47.355 INFO:tasks.workunit.client.0.smithi139.stdout:3/491: dread d5/d7/d23/d27/d2f/d30/d4c/d6d/f79 [0,4194304] 0 2024-09-16T00:09:47.357 INFO:tasks.workunit.client.0.smithi139.stdout:2/690: dread d11/f15 [0,4194304] 0 2024-09-16T00:09:47.359 INFO:tasks.workunit.client.0.smithi139.stdout:7/556: dwrite d5/d16/d22/d25/f76 [0,4194304] 0 2024-09-16T00:09:47.359 INFO:tasks.workunit.client.0.smithi139.stdout:4/559: write d12/f25 [1341311,3712] 0 2024-09-16T00:09:47.361 INFO:tasks.workunit.client.0.smithi139.stdout:6/443: dwrite d8/d29/d83/d26/d45/d59/f6b [0,4194304] 0 2024-09-16T00:09:47.361 INFO:tasks.workunit.client.0.smithi139.stdout:3/492: chown d5/d7/d23/f17 682799117 1 2024-09-16T00:09:47.363 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T00:09:47.363 INFO:tasks.workunit.client.0.smithi139.stdout:2/691: mkdir d11/d30/d57/d29/d86/df3 0 2024-09-16T00:09:47.366 INFO:tasks.workunit.client.0.smithi139.stdout:4/560: symlink d12/d47/d49/lba 0 2024-09-16T00:09:47.366 INFO:tasks.workunit.client.0.smithi139.stdout:6/444: write d8/d29/d83/d26/d45/f43 [704041,9379] 0 2024-09-16T00:09:47.367 INFO:tasks.workunit.client.0.smithi139.stdout:4/561: fdatasync f7 0 2024-09-16T00:09:47.368 INFO:tasks.workunit.client.0.smithi139.stdout:7/557: rename d5/d16/d6d/cb2 to d5/d26/d74/cc4 0 2024-09-16T00:09:47.369 INFO:tasks.workunit.client.0.smithi139.stdout:8/628: dwrite d17/d1a/d57/f71 [0,4194304] 0 2024-09-16T00:09:47.370 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:47.369+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:47.370 INFO:tasks.workunit.client.0.smithi139.stdout:3/493: chown d5/d7/d23/d27/d2f/d30/d4c/d50/f54 59271576 1 2024-09-16T00:09:47.374 INFO:tasks.workunit.client.0.smithi139.stdout:4/562: read f7 [307846,123833] 0 2024-09-16T00:09:47.374 INFO:tasks.workunit.client.0.smithi139.stdout:8/629: fsync d17/d1a/d50/fd2 0 2024-09-16T00:09:47.375 INFO:tasks.workunit.client.0.smithi139.stdout:3/494: write d5/d7/f49 [730556,76162] 0 2024-09-16T00:09:47.378 INFO:tasks.workunit.client.0.smithi139.stdout:4/563: write d12/d47/d57/f6e [1734935,62282] 0 2024-09-16T00:09:47.381 INFO:tasks.workunit.client.0.smithi139.stdout:2/692: dwrite d11/d30/d57/d5d/fb4 [0,4194304] 0 2024-09-16T00:09:47.383 INFO:tasks.workunit.client.0.smithi139.stdout:6/445: rename d8/ca to d8/d56/c9a 0 2024-09-16T00:09:47.383 INFO:tasks.workunit.client.0.smithi139.stdout:3/495: unlink d5/d7/d23/d27/d2f/d30/d41/f6b 0 2024-09-16T00:09:47.386 INFO:tasks.workunit.client.0.smithi139.stdout:2/693: readlink d11/d30/d57/d29/da5/daa/lba 0 2024-09-16T00:09:47.387 INFO:tasks.workunit.client.0.smithi139.stdout:6/446: creat d8/d29/d83/d26/d28/d6d/f9b x:0 0 0 2024-09-16T00:09:47.390 INFO:tasks.workunit.client.0.smithi139.stdout:7/558: getdents d5/d16/d6d/d7f/dc2/d50 0 2024-09-16T00:09:47.391 INFO:tasks.workunit.client.0.smithi139.stdout:2/694: unlink d11/d30/d57/d29/d4f/d9c/la9 0 2024-09-16T00:09:47.393 INFO:tasks.workunit.client.0.smithi139.stdout:4/564: dwrite d12/d2b/d8c/fb8 [0,4194304] 0 2024-09-16T00:09:47.396 INFO:tasks.workunit.client.0.smithi139.stdout:2/695: write d11/d30/d57/d50/f6a [1960149,110800] 0 2024-09-16T00:09:47.397 INFO:tasks.workunit.client.0.smithi139.stdout:4/565: read - d12/d47/d57/d7f/fa3 zero size 2024-09-16T00:09:47.399 INFO:tasks.workunit.client.0.smithi139.stdout:7/559: rename d5/d16/d6d/d7f/dc2/da4 to d5/d26/d46/d6b/dc5 0 2024-09-16T00:09:47.400 INFO:tasks.workunit.client.0.smithi139.stdout:8/630: dread d17/d1f/f2c [0,4194304] 0 2024-09-16T00:09:47.401 INFO:tasks.workunit.client.0.smithi139.stdout:3/496: dread d5/d7/d23/d27/d2f/d30/d47/f25 [0,4194304] 0 2024-09-16T00:09:47.403 INFO:tasks.workunit.client.0.smithi139.stdout:7/560: mkdir d5/d16/dc6 0 2024-09-16T00:09:47.405 INFO:tasks.workunit.client.0.smithi139.stdout:3/497: symlink d5/d7/d23/d27/d2f/d30/d41/laf 0 2024-09-16T00:09:47.405 INFO:tasks.workunit.client.0.smithi139.stdout:8/631: dread d17/d1a/d50/d86/d92/f9b [0,4194304] 0 2024-09-16T00:09:47.406 INFO:tasks.workunit.client.0.smithi139.stdout:8/632: chown d17 20856 1 2024-09-16T00:09:47.407 INFO:tasks.workunit.client.0.smithi139.stdout:2/696: dwrite d11/d30/d57/d29/d2b/f65 [0,4194304] 0 2024-09-16T00:09:47.408 INFO:tasks.workunit.client.0.smithi139.stdout:3/498: write d5/d7/d23/f15 [2152240,51885] 0 2024-09-16T00:09:47.410 INFO:tasks.workunit.client.0.smithi139.stdout:2/697: rmdir d11/d30/d57/d29/da5 39 2024-09-16T00:09:47.410 INFO:tasks.workunit.client.0.smithi139.stdout:2/698: stat d11/d30/d57/d29/d2b/lbc 0 2024-09-16T00:09:47.410 INFO:tasks.workunit.client.0.smithi139.stdout:2/699: stat d11/f26 0 2024-09-16T00:09:47.412 INFO:tasks.workunit.client.0.smithi139.stdout:2/700: dread - d11/d30/dbf/ff2 zero size 2024-09-16T00:09:47.413 INFO:tasks.workunit.client.0.smithi139.stdout:3/499: write d5/d7/f37 [175066,119389] 0 2024-09-16T00:09:47.413 INFO:tasks.workunit.client.0.smithi139.stdout:8/633: write d17/d1f/d3a/db7/d40/d4f/f65 [1388010,50927] 0 2024-09-16T00:09:47.417 INFO:tasks.workunit.client.0.smithi139.stdout:3/500: mkdir d5/d7/dc/d9c/db0 0 2024-09-16T00:09:47.418 INFO:tasks.workunit.client.0.smithi139.stdout:2/701: creat d11/d30/d57/d50/ff4 x:0 0 0 2024-09-16T00:09:47.419 INFO:tasks.workunit.client.0.smithi139.stdout:8/634: creat d17/d1f/fd3 x:0 0 0 2024-09-16T00:09:47.421 INFO:tasks.workunit.client.0.smithi139.stdout:8/635: fsync d17/d1a/d57/f7b 0 2024-09-16T00:09:47.421 INFO:tasks.workunit.client.0.smithi139.stdout:7/561: dwrite d5/fbc [0,4194304] 0 2024-09-16T00:09:47.426 INFO:tasks.workunit.client.0.smithi139.stdout:7/562: creat d5/d16/d22/fc7 x:0 0 0 2024-09-16T00:09:47.428 INFO:tasks.workunit.client.0.smithi139.stdout:2/702: dwrite d11/f1a [4194304,4194304] 0 2024-09-16T00:09:47.430 INFO:tasks.workunit.client.0.smithi139.stdout:2/703: dread - d11/d30/d57/d5d/d73/fc7 zero size 2024-09-16T00:09:47.432 INFO:tasks.workunit.client.0.smithi139.stdout:8/636: dwrite d17/d1f/d3a/db7/d40/d4f/f65 [0,4194304] 0 2024-09-16T00:09:47.432 INFO:tasks.workunit.client.0.smithi139.stdout:7/563: rename d5/d16/d6d/d7f/dc2/dab/cc1 to d5/d26/d42/cc8 0 2024-09-16T00:09:47.433 INFO:tasks.workunit.client.0.smithi139.stdout:7/564: write d5/d16/d22/d25/f76 [4178133,6681] 0 2024-09-16T00:09:47.437 INFO:tasks.workunit.client.0.smithi139.stdout:7/565: symlink d5/d26/d42/dae/lc9 0 2024-09-16T00:09:47.437 INFO:tasks.workunit.client.0.smithi139.stdout:2/704: dread d11/d30/d57/d5d/d73/f8e [0,4194304] 0 2024-09-16T00:09:47.439 INFO:tasks.workunit.client.0.smithi139.stdout:8/637: read d17/d1a/f1d [3378723,107838] 0 2024-09-16T00:09:47.439 INFO:tasks.workunit.client.0.smithi139.stdout:7/566: symlink d5/d26/d42/lca 0 2024-09-16T00:09:47.440 INFO:tasks.workunit.client.0.smithi139.stdout:2/705: dread - d11/d30/d57/d29/da5/fbe zero size 2024-09-16T00:09:47.442 INFO:tasks.workunit.client.0.smithi139.stdout:8/638: unlink c5 0 2024-09-16T00:09:47.443 INFO:tasks.workunit.client.0.smithi139.stdout:7/567: creat d5/d16/d6d/d7f/dc2/d81/fcb x:0 0 0 2024-09-16T00:09:47.444 INFO:tasks.workunit.client.0.smithi139.stdout:7/568: read - d5/d16/d22/fc7 zero size 2024-09-16T00:09:47.444 INFO:tasks.workunit.client.0.smithi139.stdout:2/706: fdatasync d11/f22 0 2024-09-16T00:09:47.446 INFO:tasks.workunit.client.0.smithi139.stdout:8/639: mknod d17/d1a/cd4 0 2024-09-16T00:09:47.447 INFO:tasks.workunit.client.0.smithi139.stdout:8/640: truncate d17/d1a/d2d/dac/fb3 5242243 0 2024-09-16T00:09:47.448 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:47.449 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:47.454 INFO:tasks.workunit.client.0.smithi139.stdout:2/707: dread d11/d2f/f33 [0,4194304] 0 2024-09-16T00:09:47.455 INFO:tasks.workunit.client.0.smithi139.stdout:8/641: getdents d17/d1f 0 2024-09-16T00:09:47.455 INFO:tasks.workunit.client.0.smithi139.stdout:7/569: truncate d5/d37/f7b 1277514 0 2024-09-16T00:09:47.457 INFO:tasks.workunit.client.0.smithi139.stdout:2/708: mknod d11/d2f/dcc/dea/cf5 0 2024-09-16T00:09:47.459 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:09:47.459 INFO:tasks.workunit.client.0.smithi139.stdout:8/642: creat d17/d1f/fd5 x:0 0 0 2024-09-16T00:09:47.460 INFO:tasks.workunit.client.0.smithi139.stdout:7/570: mkdir d5/d16/dc6/dcc 0 2024-09-16T00:09:47.460 INFO:tasks.workunit.client.0.smithi139.stdout:2/709: write d11/d30/d57/d5d/d73/f8e [4396909,4072] 0 2024-09-16T00:09:47.460 INFO:tasks.workunit.client.0.smithi139.stdout:8/643: mkdir d17/d1a/d50/d86/d92/dd6 0 2024-09-16T00:09:47.460 INFO:tasks.workunit.client.0.smithi139.stdout:7/571: write d5/d26/d42/f8f [1986173,5154] 0 2024-09-16T00:09:47.462 INFO:tasks.workunit.client.0.smithi139.stdout:8/644: dread - d17/d54/f80 zero size 2024-09-16T00:09:47.464 INFO:tasks.workunit.client.0.smithi139.stdout:2/710: mknod d11/d2f/d6e/de5/cf6 0 2024-09-16T00:09:47.468 INFO:tasks.workunit.client.0.smithi139.stdout:7/572: dwrite d5/d26/d5d/f8e [0,4194304] 0 2024-09-16T00:09:47.469 INFO:tasks.workunit.client.0.smithi139.stdout:8/645: dread - d17/d1f/d3a/db7/d40/fb4 zero size 2024-09-16T00:09:47.469 INFO:tasks.workunit.client.0.smithi139.stdout:8/646: write d17/d1f/fd5 [739661,47081] 0 2024-09-16T00:09:47.469 INFO:tasks.workunit.client.0.smithi139.stdout:2/711: link d11/d2f/fed d11/d2f/deb/ff7 0 2024-09-16T00:09:47.470 INFO:tasks.workunit.client.0.smithi139.stdout:7/573: truncate d5/d16/d22/fc7 298018 0 2024-09-16T00:09:47.475 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:09:47.475 INFO:tasks.workunit.client.0.smithi139.stdout:7/574: rmdir d5/d16/dc6/dcc 0 2024-09-16T00:09:47.475 INFO:tasks.workunit.client.0.smithi139.stdout:8/647: dread d17/d1f/f2c [0,4194304] 0 2024-09-16T00:09:47.478 INFO:tasks.workunit.client.0.smithi139.stdout:8/648: dread - d17/d1f/fd3 zero size 2024-09-16T00:09:47.478 INFO:tasks.workunit.client.0.smithi139.stdout:7/575: truncate d5/d16/d22/d25/faf 1001098 0 2024-09-16T00:09:47.481 INFO:tasks.workunit.client.0.smithi139.stdout:7/576: creat d5/d16/d6d/d7f/dc2/dab/fcd x:0 0 0 2024-09-16T00:09:47.482 INFO:tasks.workunit.client.0.smithi139.stdout:8/649: creat d17/d1a/d50/d86/d9d/d9f/fd7 x:0 0 0 2024-09-16T00:09:47.483 INFO:tasks.workunit.client.0.smithi139.stdout:2/712: dwrite d11/d30/d57/f27 [0,4194304] 0 2024-09-16T00:09:47.485 INFO:tasks.workunit.client.0.smithi139.stdout:7/577: getdents d5/d16/d22/d71 0 2024-09-16T00:09:47.486 INFO:tasks.workunit.client.0.smithi139.stdout:2/713: unlink d11/d2f/d6e/d7f/fb5 0 2024-09-16T00:09:47.487 INFO:tasks.workunit.client.0.smithi139.stdout:2/714: write d11/d30/f92 [2209681,54723] 0 2024-09-16T00:09:47.489 INFO:tasks.workunit.client.0.smithi139.stdout:2/715: write d11/d30/d57/d29/da5/fbe [424209,50102] 0 2024-09-16T00:09:47.490 INFO:tasks.workunit.client.0.smithi139.stdout:8/650: link d17/d1f/d3a/db0/ccf d17/d1a/d2d/dac/cd8 0 2024-09-16T00:09:47.495 INFO:tasks.workunit.client.0.smithi139.stdout:2/716: dread d11/d30/d57/d5d/fb4 [0,4194304] 0 2024-09-16T00:09:47.502 INFO:tasks.workunit.client.0.smithi139.stdout:7/578: write d5/d16/f3e [668652,88722] 0 2024-09-16T00:09:47.503 INFO:tasks.workunit.client.0.smithi139.stdout:8/651: dwrite f7 [0,4194304] 0 2024-09-16T00:09:47.503 INFO:tasks.workunit.client.0.smithi139.stdout:2/717: mknod d11/d30/d57/d5d/d73/dc3/cf8 0 2024-09-16T00:09:47.503 INFO:tasks.workunit.client.0.smithi139.stdout:7/579: write d5/d26/d42/f5b [1697581,96048] 0 2024-09-16T00:09:47.503 INFO:tasks.workunit.client.0.smithi139.stdout:2/718: chown d11/d45 12 1 2024-09-16T00:09:47.503 INFO:tasks.workunit.client.0.smithi139.stdout:1/497: sync 2024-09-16T00:09:47.505 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:47.503+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:47.506 INFO:tasks.workunit.client.0.smithi139.stdout:7/580: unlink d5/d16/d22/d25/f30 0 2024-09-16T00:09:47.507 INFO:tasks.workunit.client.0.smithi139.stdout:1/498: creat d7/d8/d40/d46/d8d/f95 x:0 0 0 2024-09-16T00:09:47.508 INFO:tasks.workunit.client.0.smithi139.stdout:2/719: write d11/f19 [798174,36522] 0 2024-09-16T00:09:47.509 INFO:tasks.workunit.client.0.smithi139.stdout:8/652: write d17/d1a/f73 [505513,5460] 0 2024-09-16T00:09:47.509 INFO:tasks.workunit.client.0.smithi139.stdout:1/499: readlink d7/d8/d40/d46/l5f 0 2024-09-16T00:09:47.512 INFO:tasks.workunit.client.0.smithi139.stdout:8/653: chown d17/d1a/d57/f71 2 1 2024-09-16T00:09:47.515 INFO:tasks.workunit.client.0.smithi139.stdout:7/581: write d5/f85 [2623891,2642] 0 2024-09-16T00:09:47.515 INFO:tasks.workunit.client.0.smithi139.stdout:1/500: symlink d7/d8/d40/d48/d67/d7d/l96 0 2024-09-16T00:09:47.517 INFO:tasks.workunit.client.0.smithi139.stdout:8/654: creat d17/d1f/fd9 x:0 0 0 2024-09-16T00:09:47.518 INFO:tasks.workunit.client.0.smithi139.stdout:2/720: dwrite d11/d30/d57/d29/d2b/f2d [0,4194304] 0 2024-09-16T00:09:47.519 INFO:tasks.workunit.client.0.smithi139.stdout:8/655: dread - d17/d1a/f5e zero size 2024-09-16T00:09:47.519 INFO:tasks.workunit.client.0.smithi139.stdout:1/501: write d7/d8/d40/d48/d4a/f8c [447402,94703] 0 2024-09-16T00:09:47.519 INFO:tasks.workunit.client.0.smithi139.stdout:7/582: mknod d5/d16/d6d/d7f/dc2/d50/d44/d55/cce 0 2024-09-16T00:09:47.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.521+0000 7fa5697ed640 1 Processor -- start 2024-09-16T00:09:47.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.521+0000 7fa5697ed640 1 -- start start 2024-09-16T00:09:47.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.522+0000 7fa5697ed640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa56415f7c0 0x7fa5640c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.523 INFO:tasks.workunit.client.0.smithi139.stdout:1/502: truncate d7/d8/d40/d46/d8d/f95 398766 0 2024-09-16T00:09:47.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.522+0000 7fa5697ed640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa56415fba0 0x7fa5640c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.522+0000 7fa5697ed640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa56406ccd0 con 0x7fa56406e8f0 2024-09-16T00:09:47.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.522+0000 7fa5697ed640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa564070580 con 0x7fa56415fba0 2024-09-16T00:09:47.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.522+0000 7fa5697ed640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fa5640682f0 con 0x7fa56415f7c0 2024-09-16T00:09:47.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.522+0000 7fa5627fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa56415fba0 0x7fa5640c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.523+0000 7fa5627fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa56415fba0 0x7fa5640c4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:37532/0 (socket says 172.21.15.139:37532) 2024-09-16T00:09:47.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.523+0000 7fa5627fc640 1 -- 172.21.15.139:0/3795332242 learned_addr learned my addr 172.21.15.139:0/3795332242 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:47.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.523+0000 7fa562ffd640 1 --2- 172.21.15.139:0/3795332242 >> v2:172.21.15.164:3300/0 conn(0x7fa56415f7c0 0x7fa5640c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.523+0000 7fa5627fc640 1 -- 172.21.15.139:0/3795332242 >> v2:172.21.15.164:3300/0 conn(0x7fa56415f7c0 msgr2=0x7fa5640c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:47.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.523+0000 7fa5627fc640 1 --2- 172.21.15.139:0/3795332242 >> v2:172.21.15.164:3300/0 conn(0x7fa56415f7c0 0x7fa5640c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:47.527 INFO:tasks.workunit.client.0.smithi139.stdout:8/656: symlink d17/d1a/d2d/lda 0 2024-09-16T00:09:47.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.523+0000 7fa5627fc640 1 -- 172.21.15.139:0/3795332242 >> v1:172.21.15.164:6789/0 conn(0x7fa56406e8f0 legacy=0x7fa5640ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:47.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.523+0000 7fa5627fc640 1 -- 172.21.15.139:0/3795332242 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56406cf70 con 0x7fa56415fba0 2024-09-16T00:09:47.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.524+0000 7fa5627fc640 1 --2- 172.21.15.139:0/3795332242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa56415fba0 0x7fa5640c4cf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa5640cf2d0 tx=0x7fa55802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc102f9f4542cc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:47.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.524+0000 7fa561ffb640 1 -- 172.21.15.139:0/3795332242 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558037070 con 0x7fa56415fba0 2024-09-16T00:09:47.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.524+0000 7fa561ffb640 1 -- 172.21.15.139:0/3795332242 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa558032440 con 0x7fa56415fba0 2024-09-16T00:09:47.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.524+0000 7fa561ffb640 1 -- 172.21.15.139:0/3795332242 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558032810 con 0x7fa56415fba0 2024-09-16T00:09:47.530 INFO:tasks.workunit.client.0.smithi139.stdout:7/583: creat d5/d26/fcf x:0 0 0 2024-09-16T00:09:47.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.525+0000 7fa5697ed640 1 -- 172.21.15.139:0/3795332242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa56415fba0 msgr2=0x7fa5640c4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:47.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.525+0000 7fa5697ed640 1 --2- 172.21.15.139:0/3795332242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa56415fba0 0x7fa5640c4cf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa5640cf2d0 tx=0x7fa55802a2c0 comp rx=0 tx=0).stop 2024-09-16T00:09:47.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.525+0000 7fa5697ed640 1 -- 172.21.15.139:0/3795332242 shutdown_connections 2024-09-16T00:09:47.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.525+0000 7fa5697ed640 1 --2- 172.21.15.139:0/3795332242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa56415fba0 0x7fa5640c4cf0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:47.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.525+0000 7fa5697ed640 1 --2- 172.21.15.139:0/3795332242 >> v2:172.21.15.164:3300/0 conn(0x7fa56415f7c0 0x7fa5640c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:47.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.525+0000 7fa5697ed640 1 -- 172.21.15.139:0/3795332242 >> 172.21.15.139:0/3795332242 conn(0x7fa56406bd00 msgr2=0x7fa564067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:47.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.525+0000 7fa5697ed640 1 -- 172.21.15.139:0/3795332242 shutdown_connections 2024-09-16T00:09:47.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.525+0000 7fa5697ed640 1 -- 172.21.15.139:0/3795332242 wait complete. 2024-09-16T00:09:47.532 INFO:tasks.workunit.client.0.smithi139.stdout:1/503: fdatasync d7/d8/d27/d2f/f41 0 2024-09-16T00:09:47.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.527+0000 7fa5697ed640 1 Processor -- start 2024-09-16T00:09:47.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.527+0000 7fa5697ed640 1 -- start start 2024-09-16T00:09:47.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.528+0000 7fa5697ed640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa56406e8f0 0x7fa564158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.528+0000 7fa5697ed640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa56415f7c0 0x7fa5641f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.528+0000 7fa5697ed640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa56415fba0 0x7fa5641609f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.528+0000 7fa5697ed640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa5640c3c80 con 0x7fa56415fba0 2024-09-16T00:09:47.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.528+0000 7fa562ffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa56406e8f0 0x7fa564158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.528+0000 7fa562ffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa56406e8f0 0x7fa564158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:55758/0 (socket says 172.21.15.139:55758) 2024-09-16T00:09:47.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.528+0000 7fa562ffd640 1 -- 172.21.15.139:0/2715215125 learned_addr learned my addr 172.21.15.139:0/2715215125 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:47.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.528+0000 7fa5637fe640 1 --2- 172.21.15.139:0/2715215125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa56415fba0 0x7fa5641609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.528+0000 7fa5627fc640 1 --2- 172.21.15.139:0/2715215125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa56415f7c0 0x7fa5641f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.528+0000 7fa5697ed640 1 -- 172.21.15.139:0/2715215125 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa564070ff0 con 0x7fa56415f7c0 2024-09-16T00:09:47.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.528+0000 7fa5697ed640 1 -- 172.21.15.139:0/2715215125 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa564070c60 con 0x7fa56406e8f0 2024-09-16T00:09:47.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.529+0000 7fa5637fe640 1 -- 172.21.15.139:0/2715215125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa56406e8f0 msgr2=0x7fa564158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:47.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.529+0000 7fa5637fe640 1 --2- 172.21.15.139:0/2715215125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa56406e8f0 0x7fa564158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:47.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.529+0000 7fa5637fe640 1 -- 172.21.15.139:0/2715215125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa56415f7c0 msgr2=0x7fa5641f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:47.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.529+0000 7fa5637fe640 1 --2- 172.21.15.139:0/2715215125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa56415f7c0 0x7fa5641f8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:47.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.529+0000 7fa5637fe640 1 -- 172.21.15.139:0/2715215125 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa564155490 con 0x7fa56415fba0 2024-09-16T00:09:47.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.529+0000 7fa562ffd640 1 --2- 172.21.15.139:0/2715215125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa56406e8f0 0x7fa564158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:47.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.529+0000 7fa5637fe640 1 --2- 172.21.15.139:0/2715215125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa56415fba0 0x7fa5641609f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00be30 tx=0x7fa55c00bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:47.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.529+0000 7fa54b7fe640 1 -- 172.21.15.139:0/2715215125 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c019070 con 0x7fa56415fba0 2024-09-16T00:09:47.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.529+0000 7fa54b7fe640 1 -- 172.21.15.139:0/2715215125 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa55c014440 con 0x7fa56415fba0 2024-09-16T00:09:47.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.529+0000 7fa54b7fe640 1 -- 172.21.15.139:0/2715215125 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c014830 con 0x7fa56415fba0 2024-09-16T00:09:47.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.529+0000 7fa5697ed640 1 -- 172.21.15.139:0/2715215125 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5641f8d00 con 0x7fa56415fba0 2024-09-16T00:09:47.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.530+0000 7fa5697ed640 1 -- 172.21.15.139:0/2715215125 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5640c5a60 con 0x7fa56415fba0 2024-09-16T00:09:47.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.532+0000 7fa54b7fe640 1 -- 172.21.15.139:0/2715215125 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa55c01c070 con 0x7fa56415fba0 2024-09-16T00:09:47.537 INFO:tasks.workunit.client.0.smithi139.stdout:1/504: readlink d7/l60 0 2024-09-16T00:09:47.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.532+0000 7fa5697ed640 1 -- 172.21.15.139:0/2715215125 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5640c42e0 con 0x7fa56415fba0 2024-09-16T00:09:47.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.532+0000 7fa54b7fe640 1 --2- 172.21.15.139:0/2715215125 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa55006c2b0 0x7fa55006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.532+0000 7fa54b7fe640 1 -- 172.21.15.139:0/2715215125 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5973+0+0 (secure 0 0 0) 0x7fa55c08b340 con 0x7fa56415fba0 2024-09-16T00:09:47.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.533+0000 7fa562ffd640 1 --2- 172.21.15.139:0/2715215125 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa55006c2b0 0x7fa55006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.533+0000 7fa562ffd640 1 --2- 172.21.15.139:0/2715215125 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa55006c2b0 0x7fa55006e770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa554006fd0 tx=0x7fa554008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:47.539 INFO:tasks.workunit.client.0.smithi139.stdout:2/721: dwrite d11/d30/d57/f59 [4194304,4194304] 0 2024-09-16T00:09:47.539 INFO:tasks.workunit.client.0.smithi139.stdout:7/584: creat d5/d16/d6d/fd0 x:0 0 0 2024-09-16T00:09:47.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.539+0000 7fa54b7fe640 1 -- 172.21.15.139:0/2715215125 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa5640c42e0 con 0x7fa56415fba0 2024-09-16T00:09:47.541 INFO:tasks.workunit.client.0.smithi139.stdout:2/722: creat d11/d30/d57/d5d/ff9 x:0 0 0 2024-09-16T00:09:47.542 INFO:tasks.workunit.client.0.smithi139.stdout:7/585: mknod d5/d16/d87/cd1 0 2024-09-16T00:09:47.542 INFO:tasks.workunit.client.0.smithi139.stdout:8/657: dwrite d17/d1f/d3a/dad/fb9 [0,4194304] 0 2024-09-16T00:09:47.545 INFO:tasks.workunit.client.0.smithi139.stdout:1/505: dwrite d7/d8/d40/d47/f6a [0,4194304] 0 2024-09-16T00:09:47.546 INFO:tasks.workunit.client.0.smithi139.stdout:2/723: mknod d11/d30/d57/d5d/d73/cfa 0 2024-09-16T00:09:47.548 INFO:tasks.workunit.client.0.smithi139.stdout:1/506: write d7/d8/d27/f75 [227921,49329] 0 2024-09-16T00:09:47.548 INFO:tasks.workunit.client.0.smithi139.stdout:8/658: creat d17/d1a/d50/fdb x:0 0 0 2024-09-16T00:09:47.550 INFO:tasks.workunit.client.0.smithi139.stdout:2/724: unlink d11/l3f 0 2024-09-16T00:09:47.551 INFO:tasks.workunit.client.0.smithi139.stdout:7/586: dread d5/fbc [0,4194304] 0 2024-09-16T00:09:47.553 INFO:tasks.workunit.client.0.smithi139.stdout:1/507: write d7/d8/d27/d2f/f73 [349350,9242] 0 2024-09-16T00:09:47.554 INFO:tasks.workunit.client.0.smithi139.stdout:2/725: readlink d11/d30/d57/d5d/l8f 0 2024-09-16T00:09:47.554 INFO:tasks.workunit.client.0.smithi139.stdout:8/659: creat d17/d1f/d3a/db7/d40/d4f/fdc x:0 0 0 2024-09-16T00:09:47.556 INFO:tasks.workunit.client.0.smithi139.stdout:7/587: symlink d5/d16/d6d/d7f/ld2 0 2024-09-16T00:09:47.557 INFO:tasks.workunit.client.0.smithi139.stdout:2/726: mkdir d11/d2f/dfb 0 2024-09-16T00:09:47.558 INFO:tasks.workunit.client.0.smithi139.stdout:8/660: mknod d17/d1f/d3a/cdd 0 2024-09-16T00:09:47.559 INFO:tasks.workunit.client.0.smithi139.stdout:2/727: write fc [916452,19380] 0 2024-09-16T00:09:47.560 INFO:tasks.workunit.client.0.smithi139.stdout:7/588: dread - d5/d26/d46/f9e zero size 2024-09-16T00:09:47.562 INFO:tasks.workunit.client.0.smithi139.stdout:8/661: readlink d17/d1f/l9c 0 2024-09-16T00:09:47.562 INFO:tasks.workunit.client.0.smithi139.stdout:2/728: creat d11/d30/d57/d29/da5/daa/ffc x:0 0 0 2024-09-16T00:09:47.563 INFO:tasks.workunit.client.0.smithi139.stdout:7/589: chown d5/d26/d42/f80 116031223 1 2024-09-16T00:09:47.564 INFO:tasks.workunit.client.0.smithi139.stdout:8/662: creat d17/d1a/d57/fde x:0 0 0 2024-09-16T00:09:47.566 INFO:tasks.workunit.client.0.smithi139.stdout:2/729: dread - d11/d30/d57/d29/d4f/d9c/fb3 zero size 2024-09-16T00:09:47.567 INFO:tasks.workunit.client.0.smithi139.stdout:7/590: creat d5/d26/d46/d6b/dc5/fd3 x:0 0 0 2024-09-16T00:09:47.568 INFO:tasks.workunit.client.0.smithi139.stdout:2/730: chown d11/f19 94 1 2024-09-16T00:09:47.569 INFO:tasks.workunit.client.0.smithi139.stdout:1/508: dread d7/d8/f11 [0,4194304] 0 2024-09-16T00:09:47.570 INFO:tasks.workunit.client.0.smithi139.stdout:7/591: truncate d5/d26/d74/fad 4715047 0 2024-09-16T00:09:47.571 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:47.569+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:47.575 INFO:tasks.workunit.client.0.smithi139.stdout:7/592: write d5/d16/d22/d25/d63/f97 [126524,77867] 0 2024-09-16T00:09:47.576 INFO:tasks.workunit.client.0.smithi139.stdout:8/663: dwrite d17/d1f/d3a/db7/d40/fc8 [0,4194304] 0 2024-09-16T00:09:47.579 INFO:tasks.workunit.client.0.smithi139.stdout:7/593: symlink d5/d26/d46/ld4 0 2024-09-16T00:09:47.581 INFO:tasks.workunit.client.0.smithi139.stdout:2/731: dread d11/d30/d57/d50/f6a [0,4194304] 0 2024-09-16T00:09:47.581 INFO:tasks.workunit.client.0.smithi139.stdout:2/732: readlink d11/l14 0 2024-09-16T00:09:47.582 INFO:tasks.workunit.client.0.smithi139.stdout:2/733: chown d11/d2f/l5b 27219 1 2024-09-16T00:09:47.584 INFO:tasks.workunit.client.0.smithi139.stdout:8/664: write d17/d1f/d3a/db7/d40/f41 [1596737,72330] 0 2024-09-16T00:09:47.585 INFO:tasks.workunit.client.0.smithi139.stdout:2/734: creat d11/d30/d57/d29/da5/daa/ffd x:0 0 0 2024-09-16T00:09:47.587 INFO:tasks.workunit.client.0.smithi139.stdout:1/509: dread d7/d8/d1f/f38 [0,4194304] 0 2024-09-16T00:09:47.587 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:47.587 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:47.588 INFO:tasks.workunit.client.0.smithi139.stdout:2/735: mkdir d11/d30/dbf/dfe 0 2024-09-16T00:09:47.589 INFO:tasks.workunit.client.0.smithi139.stdout:8/665: mkdir d17/d1a/d50/d86/d9d/daf/ddf 0 2024-09-16T00:09:47.589 INFO:tasks.workunit.client.0.smithi139.stdout:1/510: write d7/d8/d27/d2f/f73 [1092668,75286] 0 2024-09-16T00:09:47.589 INFO:tasks.workunit.client.0.smithi139.stdout:7/594: getdents d5/d26/d46 0 2024-09-16T00:09:47.591 INFO:tasks.workunit.client.0.smithi139.stdout:1/511: write d7/f26 [474531,111110] 0 2024-09-16T00:09:47.592 INFO:tasks.workunit.client.0.smithi139.stdout:7/595: readlink d5/d16/d6d/d7f/ld2 0 2024-09-16T00:09:47.592 INFO:tasks.workunit.client.0.smithi139.stdout:2/736: symlink d11/d30/d57/d5d/d73/lff 0 2024-09-16T00:09:47.593 INFO:tasks.workunit.client.0.smithi139.stdout:1/512: read d7/f12 [4306631,86589] 0 2024-09-16T00:09:47.595 INFO:tasks.workunit.client.0.smithi139.stdout:8/666: rename d17/d1a/d57/f98 to d17/d1f/d3a/dcd/fe0 0 2024-09-16T00:09:47.595 INFO:tasks.workunit.client.0.smithi139.stdout:7/596: write d5/d16/d6d/d7f/dc2/dab/fcd [111946,34464] 0 2024-09-16T00:09:47.596 INFO:tasks.workunit.client.0.smithi139.stdout:8/667: chown d17/d1f/d3a/l9a 121573947 1 2024-09-16T00:09:47.597 INFO:tasks.workunit.client.0.smithi139.stdout:2/737: rename d11/d30/d57/d29/d2b/c3d to d11/d2f/deb/c100 0 2024-09-16T00:09:47.598 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:09:47.599 INFO:tasks.workunit.client.0.smithi139.stdout:8/668: truncate d17/d1a/d50/d86/d92/f9b 1108947 0 2024-09-16T00:09:47.600 INFO:tasks.workunit.client.0.smithi139.stdout:1/513: link d7/d8/d27/l4d d7/d8/d40/d46/l97 0 2024-09-16T00:09:47.600 INFO:tasks.workunit.client.0.smithi139.stdout:7/597: link d5/d26/d5d/f95 d5/d16/d6d/d7f/dc2/d50/d44/d55/fd5 0 2024-09-16T00:09:47.601 INFO:tasks.workunit.client.0.smithi139.stdout:2/738: symlink d11/d30/d57/d5d/d73/dc3/de8/l101 0 2024-09-16T00:09:47.602 INFO:tasks.workunit.client.0.smithi139.stdout:1/514: write d7/d8/d27/f59 [469333,102296] 0 2024-09-16T00:09:47.603 INFO:tasks.workunit.client.0.smithi139.stdout:8/669: creat d17/d1a/d2d/fe1 x:0 0 0 2024-09-16T00:09:47.603 INFO:tasks.workunit.client.0.smithi139.stdout:7/598: write d5/d16/d6d/d7f/dc2/d50/d44/d55/f6c [5311022,3616] 0 2024-09-16T00:09:47.606 INFO:tasks.workunit.client.0.smithi139.stdout:1/515: creat d7/d8/d40/d48/d4a/f98 x:0 0 0 2024-09-16T00:09:47.606 INFO:tasks.workunit.client.0.smithi139.stdout:2/739: truncate d11/d30/d57/d29/da5/d81/fe9 760218 0 2024-09-16T00:09:47.607 INFO:tasks.workunit.client.0.smithi139.stdout:8/670: mkdir d17/d1f/d3a/de2 0 2024-09-16T00:09:47.607 INFO:tasks.workunit.client.0.smithi139.stdout:7/599: rmdir d5/d16/d22/d25/d63 39 2024-09-16T00:09:47.608 INFO:tasks.workunit.client.0.smithi139.stdout:8/671: chown d17/d1f/d3a/f45 4108692 1 2024-09-16T00:09:47.609 INFO:tasks.workunit.client.0.smithi139.stdout:1/516: truncate d7/d8/d40/d48/d4a/d74/f7f 856115 0 2024-09-16T00:09:47.610 INFO:tasks.workunit.client.0.smithi139.stdout:7/600: mkdir d5/d26/d46/d6b/dd6 0 2024-09-16T00:09:47.612 INFO:tasks.workunit.client.0.smithi139.stdout:2/740: creat d11/d30/d57/d29/f102 x:0 0 0 2024-09-16T00:09:47.613 INFO:tasks.workunit.client.0.smithi139.stdout:8/672: dread d17/d1a/d57/f79 [0,4194304] 0 2024-09-16T00:09:47.614 INFO:tasks.workunit.client.0.smithi139.stdout:7/601: symlink d5/d26/d5d/ld7 0 2024-09-16T00:09:47.615 INFO:tasks.workunit.client.0.smithi139.stdout:2/741: write d11/d30/d57/d5d/d73/f8e [2107378,60132] 0 2024-09-16T00:09:47.616 INFO:tasks.workunit.client.0.smithi139.stdout:7/602: mknod d5/d37/d70/cd8 0 2024-09-16T00:09:47.617 INFO:tasks.workunit.client.0.smithi139.stdout:2/742: mknod d11/d30/d57/d29/d4f/c103 0 2024-09-16T00:09:47.618 INFO:tasks.workunit.client.0.smithi139.stdout:7/603: mknod d5/d26/d46/d6b/dc5/cd9 0 2024-09-16T00:09:47.621 INFO:tasks.workunit.client.0.smithi139.stdout:7/604: mkdir d5/d16/dda 0 2024-09-16T00:09:47.623 INFO:tasks.workunit.client.0.smithi139.stdout:2/743: getdents d11 0 2024-09-16T00:09:47.623 INFO:tasks.workunit.client.0.smithi139.stdout:1/517: dread d7/d8/f1e [4194304,4194304] 0 2024-09-16T00:09:47.624 INFO:tasks.workunit.client.0.smithi139.stdout:7/605: mkdir d5/d26/d46/d6b/da7/ddb 0 2024-09-16T00:09:47.625 INFO:tasks.workunit.client.0.smithi139.stdout:1/518: rmdir d7/d8/d40 39 2024-09-16T00:09:47.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.625+0000 7facfa149640 1 Processor -- start 2024-09-16T00:09:47.626 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.625+0000 7facfa149640 1 -- start start 2024-09-16T00:09:47.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.626+0000 7facfa149640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7facf40c7880 0x7facf415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.626+0000 7facfa149640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facf4073e50 con 0x7facf40c4480 2024-09-16T00:09:47.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.626+0000 7facfa149640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facf40c5f80 con 0x7facf40c8e90 2024-09-16T00:09:47.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.626+0000 7facfa149640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7facf4073890 con 0x7facf40c7880 2024-09-16T00:09:47.627 INFO:tasks.workunit.client.0.smithi139.stdout:7/606: mknod d5/d26/d42/dae/cdc 0 2024-09-16T00:09:47.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.626+0000 7facf3fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7facf40c4480 0x7facf40c6a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59258/0 (socket says 172.21.15.139:59258) 2024-09-16T00:09:47.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.626+0000 7facf3fff640 1 -- 172.21.15.139:0/833189928 learned_addr learned my addr 172.21.15.139:0/833189928 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:47.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.626+0000 7facf2ffd640 1 --2- 172.21.15.139:0/833189928 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7facf40c7880 0x7facf415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.627+0000 7facf27fc640 1 -- 172.21.15.139:0/833189928 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1540650735 0 0) 0x7facf40c5f80 con 0x7facf40c8e90 2024-09-16T00:09:47.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.627+0000 7facf27fc640 1 -- 172.21.15.139:0/833189928 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facdc003520 con 0x7facf40c8e90 2024-09-16T00:09:47.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.627+0000 7facf27fc640 1 -- 172.21.15.139:0/833189928 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4166000863 0 0) 0x7facf4073e50 con 0x7facf40c4480 2024-09-16T00:09:47.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.627+0000 7facf27fc640 1 -- 172.21.15.139:0/833189928 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facf40c5f80 con 0x7facf40c4480 2024-09-16T00:09:47.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.627+0000 7facf2ffd640 1 -- 172.21.15.139:0/833189928 >> v1:172.21.15.164:6789/0 conn(0x7facf40c8e90 legacy=0x7facf40c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:47.629 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.627+0000 7facf2ffd640 1 -- 172.21.15.139:0/833189928 >> v1:172.21.15.139:6789/0 conn(0x7facf40c4480 legacy=0x7facf40c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:47.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.627+0000 7facf2ffd640 1 -- 172.21.15.139:0/833189928 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf4073b70 con 0x7facf40c7880 2024-09-16T00:09:47.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.628+0000 7facf2ffd640 1 --2- 172.21.15.139:0/833189928 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7facf40c7880 0x7facf415a2e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7face80097d0 tx=0x7face802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6add40041bf58d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:47.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.628+0000 7facf27fc640 1 -- 172.21.15.139:0/833189928 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8037070 con 0x7facf40c7880 2024-09-16T00:09:47.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.628+0000 7facf27fc640 1 -- 172.21.15.139:0/833189928 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face8032440 con 0x7facf40c7880 2024-09-16T00:09:47.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.628+0000 7facf27fc640 1 -- 172.21.15.139:0/833189928 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80327f0 con 0x7facf40c7880 2024-09-16T00:09:47.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.629+0000 7facfa149640 1 -- 172.21.15.139:0/833189928 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7facf40c7880 msgr2=0x7facf415a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:47.630 INFO:tasks.workunit.client.0.smithi139.stdout:2/744: getdents d11/d45 0 2024-09-16T00:09:47.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.629+0000 7facfa149640 1 --2- 172.21.15.139:0/833189928 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7facf40c7880 0x7facf415a2e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7face80097d0 tx=0x7face802a210 comp rx=0 tx=0).stop 2024-09-16T00:09:47.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.629+0000 7facfa149640 1 -- 172.21.15.139:0/833189928 shutdown_connections 2024-09-16T00:09:47.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.629+0000 7facfa149640 1 --2- 172.21.15.139:0/833189928 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7facf40c7880 0x7facf415a2e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:47.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.629+0000 7facfa149640 1 -- 172.21.15.139:0/833189928 >> 172.21.15.139:0/833189928 conn(0x7facf40c9a80 msgr2=0x7facf40c9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:47.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.629+0000 7facfa149640 1 -- 172.21.15.139:0/833189928 shutdown_connections 2024-09-16T00:09:47.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.629+0000 7facfa149640 1 -- 172.21.15.139:0/833189928 wait complete. 2024-09-16T00:09:47.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.631+0000 7facfa149640 1 Processor -- start 2024-09-16T00:09:47.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.631+0000 7facfa149640 1 -- start start 2024-09-16T00:09:47.633 INFO:tasks.workunit.client.0.smithi139.stdout:2/745: mknod d11/d2f/d6e/de5/c104 0 2024-09-16T00:09:47.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.632+0000 7facfa149640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facf40c4480 0x7facf40c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.632+0000 7facfa149640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf40c7880 0x7facf41f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.632+0000 7facfa149640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7facf40c8e90 0x7facf406d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.632+0000 7facfa149640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7facf4074140 con 0x7facf40c7880 2024-09-16T00:09:47.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.632+0000 7facfa149640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7facf40c59c0 con 0x7facf40c4480 2024-09-16T00:09:47.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.632+0000 7facfa149640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7facf40c7130 con 0x7facf40c8e90 2024-09-16T00:09:47.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.632+0000 7facf2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf40c7880 0x7facf41f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.632+0000 7facf37fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facf40c4480 0x7facf40c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.632+0000 7facf3fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7facf40c8e90 0x7facf406d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.632+0000 7facf2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf40c7880 0x7facf41f4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:37542/0 (socket says 172.21.15.139:37542) 2024-09-16T00:09:47.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.632+0000 7facf2ffd640 1 -- 172.21.15.139:0/1517968749 learned_addr learned my addr 172.21.15.139:0/1517968749 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:47.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.632+0000 7facf2ffd640 1 -- 172.21.15.139:0/1517968749 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7facf40c8e90 msgr2=0x7facf406d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:47.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.632+0000 7facf2ffd640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7facf40c8e90 0x7facf406d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:47.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.632+0000 7facf2ffd640 1 -- 172.21.15.139:0/1517968749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facf40c4480 msgr2=0x7facf40c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:47.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.632+0000 7facf2ffd640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facf40c4480 0x7facf40c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:47.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.632+0000 7facf2ffd640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf40c19a0 con 0x7facf40c7880 2024-09-16T00:09:47.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.633+0000 7facf3fff640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7facf40c8e90 0x7facf406d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:47.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.633+0000 7facf37fe640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7facf40c4480 0x7facf40c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:47.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.633+0000 7facf2ffd640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facf40c7880 0x7facf41f4e20 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7face80097d0 tx=0x7face80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:47.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.633+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8037050 con 0x7facf40c7880 2024-09-16T00:09:47.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.633+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face8038040 con 0x7facf40c7880 2024-09-16T00:09:47.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.633+0000 7facfa149640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf41f8840 con 0x7facf40c7880 2024-09-16T00:09:47.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.633+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8004450 con 0x7facf40c7880 2024-09-16T00:09:47.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.633+0000 7facfa149640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf40c16c0 con 0x7facf40c7880 2024-09-16T00:09:47.637 INFO:tasks.workunit.client.0.smithi139.stdout:2/746: creat d11/d45/f105 x:0 0 0 2024-09-16T00:09:47.638 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.634+0000 7facfa149640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7facf40c6a20 con 0x7facf40c7880 2024-09-16T00:09:47.638 INFO:tasks.workunit.client.0.smithi139.stdout:2/747: stat d11/d30/fd5 0 2024-09-16T00:09:47.638 INFO:tasks.workunit.client.0.smithi139.stdout:7/607: dwrite d5/d16/d22/f6f [0,4194304] 0 2024-09-16T00:09:47.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.635+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7face803c6d0 con 0x7facf40c7880 2024-09-16T00:09:47.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.635+0000 7facdbfff640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7face006c2e0 0x7face006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.636+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5973+0+0 (secure 0 0 0) 0x7face80abd20 con 0x7facf40c7880 2024-09-16T00:09:47.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.636+0000 7facf37fe640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7face006c2e0 0x7face006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.636+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v88) ==== 54+0+16 (secure 0 0 0) 0x7face8079d40 con 0x7facf40c7880 2024-09-16T00:09:47.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.636+0000 7facf37fe640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7face006c2e0 0x7face006e7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7face4001010 tx=0x7face4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:47.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.636+0000 7facd9ffb640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7facf41488c0 0x7facf4155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.636+0000 7facd9ffb640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7facf4075530 con 0x7facf41488c0 2024-09-16T00:09:47.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.636+0000 7facf3fff640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7facf41488c0 0x7facf4155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.637+0000 7facf3fff640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7facf41488c0 0x7facf4155d50 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:47.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.638+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facf4075530 con 0x7facf41488c0 2024-09-16T00:09:47.642 INFO:tasks.workunit.client.0.smithi139.stdout:2/748: symlink d11/d30/d57/d5d/d73/dc3/de8/l106 0 2024-09-16T00:09:47.642 INFO:tasks.workunit.client.0.smithi139.stdout:7/608: fsync d5/fa6 0 2024-09-16T00:09:47.645 INFO:tasks.workunit.client.0.smithi139.stdout:2/749: fsync d11/d2f/f33 0 2024-09-16T00:09:47.645 INFO:tasks.workunit.client.0.smithi139.stdout:1/519: dwrite d7/f18 [4194304,4194304] 0 2024-09-16T00:09:47.650 INFO:tasks.workunit.client.0.smithi139.stdout:0/579: sync 2024-09-16T00:09:47.650 INFO:tasks.workunit.client.0.smithi139.stdout:1/520: write d7/d8/d27/f79 [859819,9988] 0 2024-09-16T00:09:47.651 INFO:tasks.workunit.client.0.smithi139.stdout:2/750: write d11/d30/dce/fe2 [854424,112736] 0 2024-09-16T00:09:47.653 INFO:tasks.workunit.client.0.smithi139.stdout:0/580: fdatasync d6/d42/db4/f50 0 2024-09-16T00:09:47.654 INFO:tasks.workunit.client.0.smithi139.stdout:1/521: write d7/d8/f36 [3095926,127552] 0 2024-09-16T00:09:47.657 INFO:tasks.workunit.client.0.smithi139.stdout:2/751: mknod d11/d30/d57/d29/da5/daa/c107 0 2024-09-16T00:09:47.658 INFO:tasks.workunit.client.0.smithi139.stdout:7/609: dwrite d5/d26/d74/fad [4194304,4194304] 0 2024-09-16T00:09:47.662 INFO:tasks.workunit.client.0.smithi139.stdout:2/752: chown d11/d2f/d6e/f80 0 1 2024-09-16T00:09:47.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.662+0000 7facd9ffb640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7facf41f8490 con 0x7facf41488c0 2024-09-16T00:09:47.663 INFO:tasks.workunit.client.0.smithi139.stdout:7/610: symlink d5/d26/d46/d6b/da7/ldd 0 2024-09-16T00:09:47.664 INFO:tasks.workunit.client.0.smithi139.stdout:1/522: dwrite d7/d8/f1a [0,4194304] 0 2024-09-16T00:09:47.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.665+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7facf41f8490 con 0x7facf41488c0 2024-09-16T00:09:47.666 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:47.666 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:47.667 INFO:tasks.workunit.client.0.smithi139.stdout:2/753: write d11/d30/d57/d29/d2b/f2c [391489,95561] 0 2024-09-16T00:09:47.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.666+0000 7facd9ffb640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7facf41f4510 0x7facf40bf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.666+0000 7facd9ffb640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7facf4156bc0 con 0x7facf41f4510 2024-09-16T00:09:47.668 INFO:tasks.workunit.client.0.smithi139.stdout:7/611: symlink d5/d16/d87/lde 0 2024-09-16T00:09:47.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.666+0000 7facf37fe640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7facf41f4510 0x7facf40bf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.667+0000 7facf37fe640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7facf41f4510 0x7facf40bf5a0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:47.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.668+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facf4156bc0 con 0x7facf41f4510 2024-09-16T00:09:47.673 INFO:tasks.workunit.client.0.smithi139.stdout:7/612: dread - d5/d16/d6d/d7f/dc2/d81/fa1 zero size 2024-09-16T00:09:47.673 INFO:tasks.workunit.client.0.smithi139.stdout:1/523: creat d7/d8/d27/d68/d7c/f99 x:0 0 0 2024-09-16T00:09:47.673 INFO:tasks.workunit.client.0.smithi139.stdout:7/613: fsync d5/d26/d42/f80 0 2024-09-16T00:09:47.673 INFO:tasks.workunit.client.0.smithi139.stdout:2/754: unlink d11/d30/d57/d29/c58 0 2024-09-16T00:09:47.673 INFO:tasks.workunit.client.0.smithi139.stdout:1/524: write d7/f3f [1228321,67885] 0 2024-09-16T00:09:47.673 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:47.672+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:47.674 INFO:tasks.workunit.client.0.smithi139.stdout:7/614: fdatasync d5/d16/d22/d25/d63/f97 0 2024-09-16T00:09:47.675 INFO:tasks.workunit.client.0.smithi139.stdout:2/755: mkdir d11/d30/d57/d29/d86/df3/d108 0 2024-09-16T00:09:47.675 INFO:tasks.workunit.client.0.smithi139.stdout:0/581: dread d6/d1c/d41/d44/dd/f24 [0,4194304] 0 2024-09-16T00:09:47.675 INFO:tasks.workunit.client.0.smithi139.stdout:7/615: readlink d5/d26/d5d/ld7 0 2024-09-16T00:09:47.677 INFO:tasks.workunit.client.0.smithi139.stdout:2/756: chown d11/f1a 1148785779 1 2024-09-16T00:09:47.679 INFO:tasks.workunit.client.0.smithi139.stdout:0/582: creat d6/d1c/d41/d44/dd/d8a/d9e/fb6 x:0 0 0 2024-09-16T00:09:47.680 INFO:tasks.workunit.client.0.smithi139.stdout:7/616: dread d5/d37/f40 [4194304,4194304] 0 2024-09-16T00:09:47.681 INFO:tasks.workunit.client.0.smithi139.stdout:0/583: symlink d6/d5a/lb7 0 2024-09-16T00:09:47.681 INFO:tasks.workunit.client.0.smithi139.stdout:2/757: dread d11/f1a [4194304,4194304] 0 2024-09-16T00:09:47.682 INFO:tasks.workunit.client.0.smithi139.stdout:1/525: write d7/d8/d27/f57 [1182163,21956] 0 2024-09-16T00:09:47.683 INFO:tasks.workunit.client.0.smithi139.stdout:7/617: mknod d5/cdf 0 2024-09-16T00:09:47.684 INFO:tasks.workunit.client.0.smithi139.stdout:0/584: write d6/d1c/d41/d44/dd/d56/f77 [307997,73994] 0 2024-09-16T00:09:47.687 INFO:tasks.workunit.client.0.smithi139.stdout:7/618: write d5/d26/d46/d6b/dc5/fd3 [119705,28421] 0 2024-09-16T00:09:47.687 INFO:tasks.workunit.client.0.smithi139.stdout:2/758: dread d11/f19 [0,4194304] 0 2024-09-16T00:09:47.688 INFO:tasks.workunit.client.0.smithi139.stdout:0/585: truncate d6/d1c/d41/d76/faa 563952 0 2024-09-16T00:09:47.689 INFO:tasks.workunit.client.0.smithi139.stdout:7/619: fdatasync d5/d16/d6d/d7f/dc2/d50/d44/f57 0 2024-09-16T00:09:47.692 INFO:tasks.workunit.client.0.smithi139.stdout:0/586: fdatasync d6/d1c/d41/d44/f71 0 2024-09-16T00:09:47.693 INFO:tasks.workunit.client.0.smithi139.stdout:1/526: dwrite d7/d8/d27/d2f/f41 [0,4194304] 0 2024-09-16T00:09:47.698 INFO:tasks.workunit.client.0.smithi139.stdout:0/587: chown d6/d1c/d41/d44/l4b 1031420 1 2024-09-16T00:09:47.699 INFO:tasks.workunit.client.0.smithi139.stdout:2/759: dwrite d11/d30/d57/d29/d2b/f2c [0,4194304] 0 2024-09-16T00:09:47.701 INFO:tasks.workunit.client.0.smithi139.stdout:7/620: dread d5/d16/d22/d25/f76 [0,4194304] 0 2024-09-16T00:09:47.703 INFO:tasks.workunit.client.0.smithi139.stdout:2/760: mknod d11/d2f/d6e/c109 0 2024-09-16T00:09:47.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.703+0000 7facd9ffb640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7facf40c1f40 con 0x7facf41f4510 2024-09-16T00:09:47.706 INFO:tasks.workunit.client.0.smithi139.stdout:0/588: truncate d6/d5a/f65 14041 0 2024-09-16T00:09:47.706 INFO:tasks.workunit.client.0.smithi139.stdout:7/621: fsync d5/d26/d42/f66 0 2024-09-16T00:09:47.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.706+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7facf40c1f40 con 0x7facf41f4510 2024-09-16T00:09:47.707 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:47.708 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:47.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.707+0000 7facd9ffb640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7facf4062630 0x7facf4062a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.707+0000 7facd9ffb640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7facf40c03b0 con 0x7facf4062630 2024-09-16T00:09:47.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.707+0000 7facf2ffd640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7facf4062630 0x7facf4062a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.708+0000 7facf2ffd640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7facf4062630 0x7facf4062a70 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:47.709 INFO:tasks.workunit.client.0.smithi139.stdout:1/527: dwrite d7/d8/d40/d48/d4a/f8c [0,4194304] 0 2024-09-16T00:09:47.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.709+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facf40c03b0 con 0x7facf4062630 2024-09-16T00:09:47.710 INFO:tasks.workunit.client.0.smithi139.stdout:2/761: link d11/c12 d11/d30/d57/d5d/d73/dc3/de8/c10a 0 2024-09-16T00:09:47.714 INFO:tasks.workunit.client.0.smithi139.stdout:7/622: mkdir d5/d16/d6d/d7f/dc2/d50/d44/d55/de0 0 2024-09-16T00:09:47.715 INFO:tasks.workunit.client.0.smithi139.stdout:1/528: symlink d7/d8/l9a 0 2024-09-16T00:09:47.716 INFO:tasks.workunit.client.0.smithi139.stdout:7/623: symlink d5/d26/d42/dae/le1 0 2024-09-16T00:09:47.727 INFO:tasks.workunit.client.0.smithi139.stdout:7/624: dwrite d5/d16/d6d/d7f/dc2/d81/f82 [0,4194304] 0 2024-09-16T00:09:47.729 INFO:tasks.workunit.client.0.smithi139.stdout:7/625: read d5/d16/d22/d25/f76 [4154383,36418] 0 2024-09-16T00:09:47.736 INFO:tasks.workunit.client.0.smithi139.stdout:2/762: dread d11/f41 [0,4194304] 0 2024-09-16T00:09:47.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.735+0000 7facd9ffb640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7facf4063200 con 0x7facf4062630 2024-09-16T00:09:47.736 INFO:tasks.workunit.client.0.smithi139.stdout:7/626: creat d5/d26/d46/d6b/da7/fe2 x:0 0 0 2024-09-16T00:09:47.738 INFO:tasks.workunit.client.0.smithi139.stdout:1/529: dread d7/d8/d40/d46/f5b [0,4194304] 0 2024-09-16T00:09:47.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.737+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7facf4063200 con 0x7facf4062630 2024-09-16T00:09:47.739 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:47.740 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:47.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.738+0000 7facd9ffb640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7facf406cad0 0x7facf406ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.738+0000 7facd9ffb640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7facf406d3f0 con 0x7facf406cad0 2024-09-16T00:09:47.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.738+0000 7facf3fff640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7facf406cad0 0x7facf406ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.738+0000 7facf3fff640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7facf406cad0 0x7facf406ceb0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:47.740 INFO:tasks.workunit.client.0.smithi139.stdout:7/627: symlink d5/d16/d6d/d7f/dc2/dab/le3 0 2024-09-16T00:09:47.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.739+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facf406d3f0 con 0x7facf406cad0 2024-09-16T00:09:47.743 INFO:tasks.workunit.client.0.smithi139.stdout:7/628: mkdir d5/d26/d46/de4 0 2024-09-16T00:09:47.743 INFO:tasks.workunit.client.0.smithi139.stdout:7/629: write d5/f85 [775749,103916] 0 2024-09-16T00:09:47.746 INFO:tasks.workunit.client.0.smithi139.stdout:7/630: creat d5/d16/dda/fe5 x:0 0 0 2024-09-16T00:09:47.747 INFO:tasks.workunit.client.0.smithi139.stdout:2/763: dread d11/d45/fb8 [0,4194304] 0 2024-09-16T00:09:47.748 INFO:tasks.workunit.client.0.smithi139.stdout:7/631: truncate d5/d16/d22/d25/d63/fb9 306773 0 2024-09-16T00:09:47.749 INFO:tasks.workunit.client.0.smithi139.stdout:2/764: write d11/d30/f42 [3618285,114996] 0 2024-09-16T00:09:47.750 INFO:tasks.workunit.client.0.smithi139.stdout:7/632: creat d5/d37/fe6 x:0 0 0 2024-09-16T00:09:47.750 INFO:tasks.workunit.client.0.smithi139.stdout:2/765: write d11/f16 [3744748,69596] 0 2024-09-16T00:09:47.751 INFO:tasks.workunit.client.0.smithi139.stdout:2/766: stat d11/d30/dce 0 2024-09-16T00:09:47.752 INFO:tasks.workunit.client.0.smithi139.stdout:7/633: creat d5/d16/d6d/d7f/dc2/dab/fe7 x:0 0 0 2024-09-16T00:09:47.754 INFO:tasks.workunit.client.0.smithi139.stdout:7/634: mknod d5/d26/d46/d6b/dc5/ce8 0 2024-09-16T00:09:47.755 INFO:tasks.workunit.client.0.smithi139.stdout:7/635: stat d5/d26/d42/dae/le1 0 2024-09-16T00:09:47.757 INFO:tasks.workunit.client.0.smithi139.stdout:2/767: dwrite d11/d30/d57/d29/d2b/f65 [0,4194304] 0 2024-09-16T00:09:47.757 INFO:tasks.workunit.client.0.smithi139.stdout:7/636: chown d5/d16/d6d/d7f/dc2/d50/d44/d55/de0 4 1 2024-09-16T00:09:47.757 INFO:tasks.workunit.client.0.smithi139.stdout:2/768: chown d11/d2f/d6e/d7f 53574007 1 2024-09-16T00:09:47.758 INFO:tasks.workunit.client.0.smithi139.stdout:2/769: chown d11/d30/d57/d29/d2b/lbd 3934 1 2024-09-16T00:09:47.760 INFO:tasks.workunit.client.0.smithi139.stdout:7/637: symlink d5/d16/d87/le9 0 2024-09-16T00:09:47.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.759+0000 7facd9ffb640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7facf42072d0 con 0x7facf406cad0 2024-09-16T00:09:47.761 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:47.761 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:47.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.763+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7facf42072d0 con 0x7facf406cad0 2024-09-16T00:09:47.764 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:47.765 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:47.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.764+0000 7facd9ffb640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7facf42079f0 0x7facf4209de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.764+0000 7facd9ffb640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7facf406d3f0 con 0x7facf42079f0 2024-09-16T00:09:47.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.764+0000 7facf37fe640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7facf42079f0 0x7facf4209de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.765+0000 7facf37fe640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7facf42079f0 0x7facf4209de0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:47.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.766+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facf406d3f0 con 0x7facf42079f0 2024-09-16T00:09:47.767 INFO:tasks.workunit.client.0.smithi139.stdout:7/638: dread d5/d16/d6d/d7f/dc2/d50/d44/d55/f6c [4194304,4194304] 0 2024-09-16T00:09:47.768 INFO:tasks.workunit.client.0.smithi139.stdout:2/770: dwrite d11/d30/d57/d29/f102 [0,4194304] 0 2024-09-16T00:09:47.770 INFO:tasks.workunit.client.0.smithi139.stdout:2/771: mkdir d11/d2f/deb/d10b 0 2024-09-16T00:09:47.773 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:09:47.774 INFO:tasks.workunit.client.0.smithi139.stdout:7/639: truncate d5/d16/d22/d25/f31 828008 0 2024-09-16T00:09:47.774 INFO:tasks.workunit.client.0.smithi139.stdout:7/640: write d5/d26/d42/f66 [4087209,74414] 0 2024-09-16T00:09:47.774 INFO:tasks.workunit.client.0.smithi139.stdout:7/641: chown d5/d26/d42/dae 31513 1 2024-09-16T00:09:47.775 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:47.773+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:47.775 INFO:tasks.workunit.client.0.smithi139.stdout:2/772: link d11/d2f/f61 d11/d30/d57/d29/da5/d81/f10c 0 2024-09-16T00:09:47.776 INFO:tasks.workunit.client.0.smithi139.stdout:7/642: mkdir d5/d26/d46/dea 0 2024-09-16T00:09:47.777 INFO:tasks.workunit.client.0.smithi139.stdout:2/773: unlink d11/f26 0 2024-09-16T00:09:47.778 INFO:tasks.workunit.client.0.smithi139.stdout:7/643: creat d5/d16/d87/feb x:0 0 0 2024-09-16T00:09:47.780 INFO:tasks.workunit.client.0.smithi139.stdout:7/644: creat d5/d16/dda/fec x:0 0 0 2024-09-16T00:09:47.785 INFO:tasks.workunit.client.0.smithi139.stdout:7/645: symlink d5/d16/d6d/d7f/dc2/d50/d44/d55/de0/led 0 2024-09-16T00:09:47.785 INFO:tasks.workunit.client.0.smithi139.stdout:2/774: dwrite d11/d30/d57/d5d/d73/fc7 [0,4194304] 0 2024-09-16T00:09:47.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.785+0000 7facd9ffb640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7facf406d3f0 con 0x7facf42079f0 2024-09-16T00:09:47.787 INFO:tasks.workunit.client.0.smithi139.stdout:7/646: readlink d5/d16/l24 0 2024-09-16T00:09:47.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.787+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7facf406d3f0 con 0x7facf42079f0 2024-09-16T00:09:47.788 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.788+0000 7facd9ffb640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7facf420a830 0x7facf420cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.788+0000 7facd9ffb640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7facf420f9f0 con 0x7facf420a830 2024-09-16T00:09:47.789 INFO:tasks.workunit.client.0.smithi139.stdout:7/647: chown d5/c1b 14 1 2024-09-16T00:09:47.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.789+0000 7facf2ffd640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7facf420a830 0x7facf420cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.790+0000 7facf2ffd640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7facf420a830 0x7facf420cc20 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:47.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.792+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facf420f9f0 con 0x7facf420a830 2024-09-16T00:09:47.793 INFO:tasks.workunit.client.0.smithi139.stdout:7/648: unlink d5/d16/d6d/d7f/dc2/dab/fc0 0 2024-09-16T00:09:47.793 INFO:tasks.workunit.client.0.smithi139.stdout:2/775: dwrite d11/d30/d57/d29/da5/fdc [0,4194304] 0 2024-09-16T00:09:47.794 INFO:tasks.workunit.client.0.smithi139.stdout:7/649: readlink d5/d16/d87/le9 0 2024-09-16T00:09:47.794 INFO:tasks.workunit.client.0.smithi139.stdout:7/650: stat d5/d16/d87/le9 0 2024-09-16T00:09:47.795 INFO:tasks.workunit.client.0.smithi139.stdout:2/776: write d11/d30/d57/d29/d2b/f65 [2945157,32214] 0 2024-09-16T00:09:47.796 INFO:tasks.workunit.client.0.smithi139.stdout:2/777: readlink d11/l46 0 2024-09-16T00:09:47.798 INFO:tasks.workunit.client.0.smithi139.stdout:7/651: rmdir d5/d26/d46 39 2024-09-16T00:09:47.803 INFO:tasks.workunit.client.0.smithi139.stdout:2/778: dwrite d11/d30/f4c [0,4194304] 0 2024-09-16T00:09:47.805 INFO:tasks.workunit.client.0.smithi139.stdout:7/652: link d5/d16/d6d/d7f/dc2/d81/f82 d5/d16/d22/d25/d63/fee 0 2024-09-16T00:09:47.807 INFO:tasks.workunit.client.0.smithi139.stdout:2/779: rename d11/d30/d57/d29/da5/d81/f87 to d11/d2f/d6e/d7f/f10d 0 2024-09-16T00:09:47.810 INFO:tasks.workunit.client.0.smithi139.stdout:7/653: dread d5/d16/d22/d25/f76 [0,4194304] 0 2024-09-16T00:09:47.815 INFO:tasks.workunit.client.0.smithi139.stdout:2/780: link d11/d30/d57/d5d/d73/dc3/de8/l106 d11/d2f/d6e/de5/l10e 0 2024-09-16T00:09:47.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.815+0000 7facd9ffb640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7facf420d160 con 0x7facf420a830 2024-09-16T00:09:47.816 INFO:tasks.workunit.client.0.smithi139.stdout:3/501: sync 2024-09-16T00:09:47.817 INFO:tasks.workunit.client.0.smithi139.stdout:6/447: sync 2024-09-16T00:09:47.817 INFO:tasks.workunit.client.0.smithi139.stdout:5/729: sync 2024-09-16T00:09:47.817 INFO:tasks.workunit.client.0.smithi139.stdout:4/566: sync 2024-09-16T00:09:47.817 INFO:tasks.workunit.client.0.smithi139.stdout:9/681: sync 2024-09-16T00:09:47.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.817+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7facf420d160 con 0x7facf420a830 2024-09-16T00:09:47.818 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:47.819 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:47.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.817+0000 7facd9ffb640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=89}) -- 0x7facf420f9f0 con 0x7facf40c7880 2024-09-16T00:09:47.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.817+0000 7facd9ffb640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7facf420fbc0 con 0x7facf40c7880 2024-09-16T00:09:47.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.818+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7face8090050 con 0x7facf40c7880 2024-09-16T00:09:47.819 INFO:teuthology.orchestra.run.smithi139.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T00:09:47.819 INFO:tasks.workunit.client.0.smithi139.stdout:7/654: write d5/d16/d22/d25/f31 [272130,10132] 0 2024-09-16T00:09:47.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.818+0000 7facd9ffb640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7facf420da30 0x7facf420de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:47.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.818+0000 7facd9ffb640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7facf420f5d0 con 0x7facf420da30 2024-09-16T00:09:47.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.818+0000 7facf3fff640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7facf420da30 0x7facf420de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:47.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.818+0000 7facf3fff640 1 --2- 172.21.15.139:0/1517968749 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7facf420da30 0x7facf420de60 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:47.822 INFO:tasks.workunit.client.0.smithi139.stdout:7/655: stat d5/f1f 0 2024-09-16T00:09:47.822 INFO:tasks.workunit.client.0.smithi139.stdout:5/730: read d1/d13/d3f/fd3 [1260735,110356] 0 2024-09-16T00:09:47.822 INFO:tasks.workunit.client.0.smithi139.stdout:4/567: creat d12/d47/db2/d4d/fbb x:0 0 0 2024-09-16T00:09:47.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.819+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facf420f5d0 con 0x7facf420da30 2024-09-16T00:09:47.823 INFO:tasks.workunit.client.0.smithi139.stdout:6/448: read d8/d29/d83/d26/d45/d65/f2e [732608,15680] 0 2024-09-16T00:09:47.823 INFO:tasks.workunit.client.0.smithi139.stdout:3/502: creat d5/d7/d23/d27/d2f/d30/d4d/fb1 x:0 0 0 2024-09-16T00:09:47.823 INFO:tasks.workunit.client.0.smithi139.stdout:2/781: getdents d11/d2f/d6e/de5 0 2024-09-16T00:09:47.825 INFO:tasks.workunit.client.0.smithi139.stdout:7/656: write d5/d16/d22/d25/f9b [770608,45256] 0 2024-09-16T00:09:47.825 INFO:tasks.workunit.client.0.smithi139.stdout:5/731: dread d1/d21/d49/d53/d6a/f8b [0,4194304] 0 2024-09-16T00:09:47.826 INFO:tasks.workunit.client.0.smithi139.stdout:3/503: chown d5/d7/d23/d35/f45 235039430 1 2024-09-16T00:09:47.827 INFO:tasks.workunit.client.0.smithi139.stdout:9/682: write d19/d1a/d20/d23/dca/d71/d32/f76 [540568,122] 0 2024-09-16T00:09:47.829 INFO:tasks.workunit.client.0.smithi139.stdout:7/657: chown d5/d16/d6d/d7f/dc2/d50/f67 371533324 1 2024-09-16T00:09:47.830 INFO:tasks.workunit.client.0.smithi139.stdout:5/732: write d1/d13/d3b/d3e/f69 [4858881,37950] 0 2024-09-16T00:09:47.831 INFO:tasks.workunit.client.0.smithi139.stdout:3/504: write d5/d7/d23/d27/d2f/d30/d4d/fb1 [533941,49981] 0 2024-09-16T00:09:47.831 INFO:tasks.workunit.client.0.smithi139.stdout:2/782: link d11/d30/d57/d5d/d73/c96 d11/d30/d57/d29/d86/c10f 0 2024-09-16T00:09:47.832 INFO:tasks.workunit.client.0.smithi139.stdout:5/733: rename d1 to d1/d13/d3f/d9f/d73/d93/d102 22 2024-09-16T00:09:47.832 INFO:tasks.workunit.client.0.smithi139.stdout:7/658: rmdir d5/d16/dda 39 2024-09-16T00:09:47.833 INFO:tasks.workunit.client.0.smithi139.stdout:6/449: write d8/d29/d83/d26/d45/d65/f2e [1698070,24119] 0 2024-09-16T00:09:47.833 INFO:tasks.workunit.client.0.smithi139.stdout:4/568: truncate d12/d80/f9a 4177949 0 2024-09-16T00:09:47.835 INFO:tasks.workunit.client.0.smithi139.stdout:9/683: creat d19/d1a/d20/d23/dca/d9f/dd9/fe8 x:0 0 0 2024-09-16T00:09:47.835 INFO:tasks.workunit.client.0.smithi139.stdout:5/734: dread - d1/d21/d49/d53/d6a/d7b/d99/feb zero size 2024-09-16T00:09:47.836 INFO:tasks.workunit.client.0.smithi139.stdout:7/659: read d5/d37/d70/f9f [6451100,110213] 0 2024-09-16T00:09:47.837 INFO:tasks.workunit.client.0.smithi139.stdout:4/569: write d12/d47/db2/d4d/f89 [429689,36937] 0 2024-09-16T00:09:47.838 INFO:tasks.workunit.client.0.smithi139.stdout:6/450: symlink d8/d29/d83/d26/d45/d42/l9c 0 2024-09-16T00:09:47.839 INFO:tasks.workunit.client.0.smithi139.stdout:3/505: truncate d5/d7/d23/d27/d2f/d30/d4d/f8c 944692 0 2024-09-16T00:09:47.839 INFO:tasks.workunit.client.0.smithi139.stdout:9/684: mknod d19/ce9 0 2024-09-16T00:09:47.840 INFO:tasks.workunit.client.0.smithi139.stdout:2/783: write d11/d45/fb8 [2125705,58786] 0 2024-09-16T00:09:47.840 INFO:tasks.workunit.client.0.smithi139.stdout:5/735: creat d1/d21/d49/db2/f103 x:0 0 0 2024-09-16T00:09:47.841 INFO:tasks.workunit.client.0.smithi139.stdout:7/660: symlink d5/d26/d42/lef 0 2024-09-16T00:09:47.841 INFO:tasks.workunit.client.0.smithi139.stdout:4/570: readlink d12/d47/db2/d4d/db0/lae 0 2024-09-16T00:09:47.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.842+0000 7facd9ffb640 1 -- 172.21.15.139:0/1517968749 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7facf420e3a0 con 0x7facf420da30 2024-09-16T00:09:47.843 INFO:tasks.workunit.client.0.smithi139.stdout:4/571: stat d12/d2b 0 2024-09-16T00:09:47.843 INFO:tasks.workunit.client.0.smithi139.stdout:7/661: rmdir d5/d16/d6d 39 2024-09-16T00:09:47.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.843+0000 7facdbfff640 1 -- 172.21.15.139:0/1517968749 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7facf420e3a0 con 0x7facf420da30 2024-09-16T00:09:47.844 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:47.845 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:47.845 INFO:tasks.workunit.client.0.smithi139.stdout:4/572: fsync d12/d2b/f94 0 2024-09-16T00:09:47.845 INFO:tasks.workunit.client.0.smithi139.stdout:2/784: write d11/d30/f4c [728386,29665] 0 2024-09-16T00:09:47.846 INFO:tasks.workunit.client.0.smithi139.stdout:5/736: unlink d1/d21/d49/db2/ce3 0 2024-09-16T00:09:47.846 INFO:tasks.workunit.client.0.smithi139.stdout:4/573: dread - d12/d47/fa6 zero size 2024-09-16T00:09:47.847 INFO:tasks.workunit.client.0.smithi139.stdout:3/506: mkdir d5/d7/d23/d27/dab/db2 0 2024-09-16T00:09:47.850 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T00:09:47.850 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T00:09:47.851 INFO:tasks.workunit.client.0.smithi139.stdout:9/685: creat d19/d1a/d20/d23/d2d/d35/d42/fea x:0 0 0 2024-09-16T00:09:47.851 INFO:tasks.workunit.client.0.smithi139.stdout:2/785: unlink d11/d2f/d6e/d7f/c93 0 2024-09-16T00:09:47.851 INFO:tasks.workunit.client.0.smithi139.stdout:6/451: dread d8/d29/d83/d26/f75 [0,4194304] 0 2024-09-16T00:09:47.852 INFO:tasks.workunit.client.0.smithi139.stdout:5/737: creat d1/d13/d94/f104 x:0 0 0 2024-09-16T00:09:47.852 INFO:tasks.workunit.client.0.smithi139.stdout:4/574: rmdir d12/d47/d49/d60 39 2024-09-16T00:09:47.855 INFO:tasks.workunit.client.0.smithi139.stdout:6/452: write d8/f12 [2738480,30869] 0 2024-09-16T00:09:47.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:47.855+0000 7fa5697ed640 1 -- 172.21.15.139:0/2715215125 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fa56406d250 con 0x7fa56415fba0 2024-09-16T00:09:47.856 INFO:tasks.workunit.client.0.smithi139.stdout:3/507: dread d5/d7/d23/d27/d2f/d30/d4c/d6d/d76/f98 [0,4194304] 0 2024-09-16T00:09:47.857 INFO:tasks.workunit.client.0.smithi139.stdout:2/786: rename d11/d30/d57 to d11/d30/d110 0 2024-09-16T00:09:47.857 INFO:tasks.workunit.client.0.smithi139.stdout:4/575: truncate d12/f31 58131 0 2024-09-16T00:09:47.859 INFO:tasks.workunit.client.0.smithi139.stdout:4/576: stat d12/d47/db2/f54 0 2024-09-16T00:09:47.860 INFO:tasks.workunit.client.0.smithi139.stdout:5/738: truncate d1/d13/d3f/d9f/d73/f8c 5194419 0 2024-09-16T00:09:47.860 INFO:tasks.workunit.client.0.smithi139.stdout:2/787: chown d11/d30/d110/d29/d2b/c68 13 1 2024-09-16T00:09:47.862 INFO:tasks.workunit.client.0.smithi139.stdout:3/508: truncate d5/d7/d23/f77 3424142 0 2024-09-16T00:09:47.862 INFO:tasks.workunit.client.0.smithi139.stdout:7/662: dread d5/d16/d22/f4e [0,4194304] 0 2024-09-16T00:09:47.863 INFO:tasks.workunit.client.0.smithi139.stdout:4/577: write d12/d47/db2/d4d/fbb [313239,102316] 0 2024-09-16T00:09:47.864 INFO:tasks.workunit.client.0.smithi139.stdout:2/788: mkdir d11/d30/d110/d5d/d73/d111 0 2024-09-16T00:09:47.865 INFO:tasks.workunit.client.0.smithi139.stdout:7/663: write d5/d37/fe6 [1041557,29586] 0 2024-09-16T00:09:47.868 INFO:tasks.workunit.client.0.smithi139.stdout:9/686: dread d19/d1a/d20/d23/d2d/d9a/fcc [0,4194304] 0 2024-09-16T00:09:47.868 INFO:tasks.workunit.client.0.smithi139.stdout:4/578: truncate d12/d47/db2/d4d/d69/f72 4896022 0 2024-09-16T00:09:47.868 INFO:tasks.workunit.client.0.smithi139.stdout:3/509: mkdir d5/d7/d23/d27/d2f/d30/d4c/d6d/db3 0 2024-09-16T00:09:47.869 INFO:tasks.workunit.client.0.smithi139.stdout:2/789: creat d11/d32/f112 x:0 0 0 2024-09-16T00:09:47.871 INFO:tasks.workunit.client.0.smithi139.stdout:4/579: write d12/d47/db2/d4d/f89 [476027,118154] 0 2024-09-16T00:09:47.871 INFO:tasks.workunit.client.0.smithi139.stdout:7/664: symlink d5/d26/d46/d6b/da7/ddb/lf0 0 2024-09-16T00:09:47.872 INFO:tasks.workunit.client.0.smithi139.stdout:5/739: dwrite d1/d13/d3f/d9f/d30/d33/f7c [0,4194304] 0 2024-09-16T00:09:47.873 INFO:tasks.workunit.client.0.smithi139.stdout:2/790: stat d11/d30/d110/d29/da5/d81/fc5 0 2024-09-16T00:09:47.876 INFO:tasks.workunit.client.0.smithi139.stdout:6/453: truncate d8/d29/d83/f15 3983515 0 2024-09-16T00:09:47.877 INFO:tasks.workunit.client.0.smithi139.stdout:3/510: rename d5/d7 to d5/d7/d23/d27/d2f/d30/d4c/d6d/d76/db4 22 2024-09-16T00:09:47.877 INFO:tasks.workunit.client.0.smithi139.stdout:4/580: chown d12/d2b/l3d 7255 1 2024-09-16T00:09:47.877 INFO:tasks.workunit.client.0.smithi139.stdout:6/454: fdatasync d8/d29/d83/f81 0 2024-09-16T00:09:47.878 INFO:tasks.workunit.client.0.smithi139.stdout:5/740: creat d1/d21/d49/d87/f105 x:0 0 0 2024-09-16T00:09:47.878 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:47.877+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:47.878 INFO:tasks.workunit.client.0.smithi139.stdout:4/581: read d12/d47/d49/f53 [2205828,52449] 0 2024-09-16T00:09:47.880 INFO:tasks.workunit.client.0.smithi139.stdout:2/791: unlink d11/d30/d110/d29/da5/daa/ffd 0 2024-09-16T00:09:47.883 INFO:tasks.workunit.client.0.smithi139.stdout:6/455: mknod d8/d20/d4d/c9d 0 2024-09-16T00:09:47.885 INFO:tasks.workunit.client.0.smithi139.stdout:6/456: read d8/d20/d4d/f8e [460202,59377] 0 2024-09-16T00:09:47.885 INFO:tasks.workunit.client.0.smithi139.stdout:9/687: getdents d19/d1a/d20/d23/dca/d9f 0 2024-09-16T00:09:47.887 INFO:tasks.workunit.client.0.smithi139.stdout:6/457: read f3 [7183012,129352] 0 2024-09-16T00:09:47.887 INFO:tasks.workunit.client.0.smithi139.stdout:6/458: chown d8/f8c 953091434 1 2024-09-16T00:09:47.887 INFO:tasks.workunit.client.0.smithi139.stdout:4/582: rename d12/d47/d57/l6a to d12/d47/d57/d7f/lbc 0 2024-09-16T00:09:47.888 INFO:tasks.workunit.client.0.smithi139.stdout:2/792: creat d11/d2f/d6e/d7f/f113 x:0 0 0 2024-09-16T00:09:47.891 INFO:tasks.workunit.client.0.smithi139.stdout:3/511: link d5/d7/f78 d5/d7/d23/d27/d2f/d30/d41/fb5 0 2024-09-16T00:09:47.892 INFO:tasks.workunit.client.0.smithi139.stdout:9/688: rename d19/d1a/f3f to d19/d1a/d20/d23/dca/d71/dc7/feb 0 2024-09-16T00:09:47.893 INFO:tasks.workunit.client.0.smithi139.stdout:6/459: mkdir d8/d29/d83/d26/d28/d9e 0 2024-09-16T00:09:47.894 INFO:tasks.workunit.client.0.smithi139.stdout:4/583: fdatasync d12/d47/d57/f62 0 2024-09-16T00:09:47.896 INFO:tasks.workunit.client.0.smithi139.stdout:4/584: read d12/f4f [50345,44209] 0 2024-09-16T00:09:47.896 INFO:tasks.workunit.client.0.smithi139.stdout:2/793: rename d11/d30/d110/d29/da5/dc6 to d11/d30/d110/d50/d114 0 2024-09-16T00:09:47.897 INFO:tasks.workunit.client.0.smithi139.stdout:7/665: truncate d5/d37/f79 496213 0 2024-09-16T00:09:47.898 INFO:tasks.workunit.client.0.smithi139.stdout:3/512: dwrite d5/d7/d23/d27/d2f/d30/d41/f7f [0,4194304] 0 2024-09-16T00:09:47.899 INFO:tasks.workunit.client.0.smithi139.stdout:6/460: chown d8/d20/c7c 7 1 2024-09-16T00:09:47.899 INFO:tasks.workunit.client.0.smithi139.stdout:9/689: rename d19/d1a/d20/d23/d2d/d9a/cdf to d19/d1a/d20/d23/d2d/d35/d60/cec 0 2024-09-16T00:09:47.900 INFO:tasks.workunit.client.0.smithi139.stdout:2/794: write d11/d30/d110/d29/d2b/f65 [1611983,79215] 0 2024-09-16T00:09:47.901 INFO:tasks.workunit.client.0.smithi139.stdout:4/585: mkdir d12/d47/dbd 0 2024-09-16T00:09:47.903 INFO:tasks.workunit.client.0.smithi139.stdout:3/513: write d5/d7/d23/d27/d2f/d30/d47/f4a [2562223,85305] 0 2024-09-16T00:09:47.903 INFO:tasks.workunit.client.0.smithi139.stdout:6/461: mknod d8/d29/d83/d26/d45/d42/c9f 0 2024-09-16T00:09:47.903 INFO:tasks.workunit.client.0.smithi139.stdout:4/586: fdatasync d12/d47/f6c 0 2024-09-16T00:09:47.906 INFO:tasks.workunit.client.0.smithi139.stdout:2/795: readlink d11/d2f/d6e/de5/l10e 0 2024-09-16T00:09:47.909 INFO:tasks.workunit.client.0.smithi139.stdout:4/587: unlink d12/d47/fa6 0 2024-09-16T00:09:47.910 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:47.910 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:47.910 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:47.911 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:47.911 INFO:tasks.workunit.client.0.smithi139.stdout:9/690: dwrite d19/f26 [4194304,4194304] 0 2024-09-16T00:09:47.915 INFO:tasks.workunit.client.0.smithi139.stdout:3/514: creat d5/d7/d23/d27/d2f/d30/d4c/d6d/db3/fb6 x:0 0 0 2024-09-16T00:09:47.916 INFO:tasks.workunit.client.0.smithi139.stdout:7/666: dwrite d5/d16/d6d/d7f/dc2/d50/f67 [0,4194304] 0 2024-09-16T00:09:47.917 INFO:tasks.workunit.client.0.smithi139.stdout:6/462: dwrite d8/d29/d83/f81 [0,4194304] 0 2024-09-16T00:09:47.922 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:09:47.923 INFO:tasks.workunit.client.0.smithi139.stdout:2/796: write d11/d2f/d6e/f80 [1331799,63636] 0 2024-09-16T00:09:47.923 INFO:tasks.workunit.client.0.smithi139.stdout:6/463: fsync d8/d29/d83/d26/f4b 0 2024-09-16T00:09:47.923 INFO:tasks.workunit.client.0.smithi139.stdout:4/588: creat d12/d47/d49/fbe x:0 0 0 2024-09-16T00:09:47.925 INFO:tasks.workunit.client.0.smithi139.stdout:9/691: creat d19/d1a/d20/d23/dca/d71/dc7/de1/fed x:0 0 0 2024-09-16T00:09:47.928 INFO:tasks.workunit.client.0.smithi139.stdout:7/667: link d5/d16/d22/d25/faf d5/d16/d22/ff1 0 2024-09-16T00:09:47.929 INFO:tasks.workunit.client.0.smithi139.stdout:9/692: dread - d19/d1a/f8f zero size 2024-09-16T00:09:47.930 INFO:tasks.workunit.client.0.smithi139.stdout:4/589: dread d12/d47/db2/d4d/fbb [0,4194304] 0 2024-09-16T00:09:47.930 INFO:tasks.workunit.client.0.smithi139.stdout:6/464: symlink d8/d29/d83/d26/d28/d6d/la0 0 2024-09-16T00:09:47.931 INFO:tasks.workunit.client.0.smithi139.stdout:7/668: chown d5/d16/d6d/d7f/dc2/d50/d44/d55/d90 46 1 2024-09-16T00:09:47.934 INFO:tasks.workunit.client.0.smithi139.stdout:6/465: read d8/d29/d83/d26/d28/f31 [430317,10971] 0 2024-09-16T00:09:47.937 INFO:tasks.workunit.client.0.smithi139.stdout:7/669: mknod d5/d16/d22/d25/cf2 0 2024-09-16T00:09:47.938 INFO:tasks.workunit.client.0.smithi139.stdout:4/590: rmdir d12/d47/dbd 0 2024-09-16T00:09:47.939 INFO:tasks.workunit.client.0.smithi139.stdout:3/515: dwrite d5/f16 [0,4194304] 0 2024-09-16T00:09:47.940 INFO:tasks.workunit.client.0.smithi139.stdout:7/670: read - d5/d16/d6d/d7f/dc2/d81/fa1 zero size 2024-09-16T00:09:47.941 INFO:tasks.workunit.client.0.smithi139.stdout:3/516: readlink d5/d7/l73 0 2024-09-16T00:09:47.941 INFO:tasks.workunit.client.0.smithi139.stdout:4/591: chown d12/d2b/d8c/f9c 2019801 1 2024-09-16T00:09:47.942 INFO:tasks.workunit.client.0.smithi139.stdout:2/797: dwrite d11/d30/d110/d5d/d73/fd4 [0,4194304] 0 2024-09-16T00:09:47.943 INFO:tasks.workunit.client.0.smithi139.stdout:2/798: stat d11/d2f/d6e/de5/cf6 0 2024-09-16T00:09:47.944 INFO:tasks.workunit.client.0.smithi139.stdout:9/693: rename d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/f55 to d19/d1a/d20/d23/dca/d9f/dd9/fee 0 2024-09-16T00:09:47.945 INFO:tasks.workunit.client.0.smithi139.stdout:3/517: rmdir d5/d7/dc/d9c 39 2024-09-16T00:09:47.946 INFO:tasks.workunit.client.0.smithi139.stdout:3/518: readlink d5/d7/d23/la4 0 2024-09-16T00:09:47.947 INFO:tasks.workunit.client.0.smithi139.stdout:4/592: symlink d12/d47/d57/d7f/lbf 0 2024-09-16T00:09:47.949 INFO:tasks.workunit.client.0.smithi139.stdout:9/694: stat d19/d1a/d20/d23/d2d/d35/d42/dc8/ld2 0 2024-09-16T00:09:47.953 INFO:tasks.workunit.client.0.smithi139.stdout:3/519: read d5/d7/dc/d9c/fa3 [76200,103486] 0 2024-09-16T00:09:47.955 INFO:tasks.workunit.client.0.smithi139.stdout:9/695: dwrite d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/fad [0,4194304] 0 2024-09-16T00:09:47.956 INFO:tasks.workunit.client.0.smithi139.stdout:2/799: link d11/d30/f42 d11/d2f/dcc/dea/dee/f115 0 2024-09-16T00:09:47.958 INFO:tasks.workunit.client.0.smithi139.stdout:9/696: dread - d19/d1a/d20/d23/d2d/d35/d42/fea zero size 2024-09-16T00:09:47.959 INFO:tasks.workunit.client.0.smithi139.stdout:4/593: creat d12/d47/d49/fc0 x:0 0 0 2024-09-16T00:09:47.960 INFO:tasks.workunit.client.0.smithi139.stdout:4/594: chown d12/d47/f32 239540247 1 2024-09-16T00:09:47.960 INFO:tasks.workunit.client.0.smithi139.stdout:3/520: creat d5/d7/d23/d27/d2f/d30/d4c/d50/d75/fb7 x:0 0 0 2024-09-16T00:09:47.962 INFO:tasks.workunit.client.0.smithi139.stdout:7/671: dwrite d5/f21 [4194304,4194304] 0 2024-09-16T00:09:47.962 INFO:tasks.workunit.client.0.smithi139.stdout:9/697: read d19/d1a/d20/d23/d2d/d35/d42/d3a/f6e [2793339,119573] 0 2024-09-16T00:09:47.969 INFO:tasks.workunit.client.0.smithi139.stdout:7/672: symlink d5/d26/d46/de4/lf3 0 2024-09-16T00:09:47.970 INFO:tasks.workunit.client.0.smithi139.stdout:9/698: rmdir d19/d1a/d20/d23/d2d/d35 39 2024-09-16T00:09:47.973 INFO:tasks.workunit.client.0.smithi139.stdout:4/595: dwrite f7 [0,4194304] 0 2024-09-16T00:09:47.974 INFO:tasks.workunit.client.0.smithi139.stdout:7/673: mkdir d5/d26/d46/d6b/dd6/df4 0 2024-09-16T00:09:47.977 INFO:tasks.workunit.client.0.smithi139.stdout:4/596: write d12/d47/db2/f77 [941980,107877] 0 2024-09-16T00:09:47.978 INFO:tasks.workunit.client.0.smithi139.stdout:3/521: dwrite d5/d7/d23/d27/d2f/d30/d4d/f8c [0,4194304] 0 2024-09-16T00:09:47.978 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:47.977+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:47.981 INFO:tasks.workunit.client.0.smithi139.stdout:7/674: write d5/d16/dda/fe5 [1016334,120278] 0 2024-09-16T00:09:47.983 INFO:tasks.workunit.client.0.smithi139.stdout:2/800: dread d11/d30/f36 [0,4194304] 0 2024-09-16T00:09:47.984 INFO:tasks.workunit.client.0.smithi139.stdout:4/597: truncate d12/d47/d57/fb5 205632 0 2024-09-16T00:09:47.984 INFO:tasks.workunit.client.0.smithi139.stdout:7/675: mkdir d5/d16/d6d/d7f/df5 0 2024-09-16T00:09:47.985 INFO:tasks.workunit.client.0.smithi139.stdout:3/522: symlink d5/d7/dc/d84/lb8 0 2024-09-16T00:09:47.986 INFO:tasks.workunit.client.0.smithi139.stdout:4/598: rename d12/d47/db2 to d12/d47/db2/d4d/dc1 22 2024-09-16T00:09:47.986 INFO:tasks.workunit.client.0.smithi139.stdout:2/801: mkdir d11/d30/dbf/d116 0 2024-09-16T00:09:47.987 INFO:tasks.workunit.client.0.smithi139.stdout:7/676: symlink d5/d26/d46/de4/lf6 0 2024-09-16T00:09:47.988 INFO:tasks.workunit.client.0.smithi139.stdout:2/802: write d11/d2f/d6e/f84 [10977,82679] 0 2024-09-16T00:09:47.988 INFO:tasks.workunit.client.0.smithi139.stdout:4/599: creat d12/d47/db2/fc2 x:0 0 0 2024-09-16T00:09:47.989 INFO:tasks.workunit.client.0.smithi139.stdout:3/523: mkdir d5/d7/d23/d27/d2f/d30/d41/d68/db9 0 2024-09-16T00:09:47.991 INFO:tasks.workunit.client.0.smithi139.stdout:4/600: truncate d12/d47/d57/f5f 183301 0 2024-09-16T00:09:47.992 INFO:tasks.workunit.client.0.smithi139.stdout:9/699: dread d19/d1a/d20/d23/d2d/f90 [0,4194304] 0 2024-09-16T00:09:47.993 INFO:tasks.workunit.client.0.smithi139.stdout:2/803: rename d11/d30/d110/d29/da5/daa/ffc to d11/d30/d110/d5d/d73/dc3/f117 0 2024-09-16T00:09:47.996 INFO:tasks.workunit.client.0.smithi139.stdout:9/700: rmdir d19/d1a/d20/d23/dca/d71/dc7 39 2024-09-16T00:09:47.997 INFO:tasks.workunit.client.0.smithi139.stdout:9/701: chown d19/f1d 921206 1 2024-09-16T00:09:47.998 INFO:tasks.workunit.client.0.smithi139.stdout:4/601: mkdir d12/d47/d49/d60/dc3 0 2024-09-16T00:09:47.998 INFO:tasks.workunit.client.0.smithi139.stdout:7/677: write d5/d26/f32 [1347141,72407] 0 2024-09-16T00:09:47.999 INFO:tasks.workunit.client.0.smithi139.stdout:2/804: mkdir d11/d30/dbf/dfe/d118 0 2024-09-16T00:09:48.000 INFO:tasks.workunit.client.0.smithi139.stdout:4/602: fsync d12/d47/d49/d60/f8f 0 2024-09-16T00:09:48.001 INFO:tasks.workunit.client.0.smithi139.stdout:3/524: dread d5/f8 [0,4194304] 0 2024-09-16T00:09:48.003 INFO:tasks.workunit.client.0.smithi139.stdout:3/525: write d5/f16 [5160289,30447] 0 2024-09-16T00:09:48.003 INFO:tasks.workunit.client.0.smithi139.stdout:7/678: creat d5/d26/d46/d6b/dd6/df4/ff7 x:0 0 0 2024-09-16T00:09:48.004 INFO:tasks.workunit.client.0.smithi139.stdout:2/805: dread d11/d30/dce/fe2 [0,4194304] 0 2024-09-16T00:09:48.007 INFO:tasks.workunit.client.0.smithi139.stdout:3/526: mkdir d5/d7/d23/d27/d2f/d30/d47/dba 0 2024-09-16T00:09:48.009 INFO:tasks.workunit.client.0.smithi139.stdout:7/679: mkdir d5/d26/d46/dea/df8 0 2024-09-16T00:09:48.010 INFO:tasks.workunit.client.0.smithi139.stdout:7/680: readlink d5/d26/d5d/ld7 0 2024-09-16T00:09:48.010 INFO:tasks.workunit.client.0.smithi139.stdout:9/702: dread d19/d1a/d20/d23/dca/d71/da3/fce [0,4194304] 0 2024-09-16T00:09:48.011 INFO:tasks.workunit.client.0.smithi139.stdout:3/527: mkdir d5/d7/d23/d27/d2f/d30/d4c/dbb 0 2024-09-16T00:09:48.012 INFO:tasks.workunit.client.0.smithi139.stdout:4/603: link d12/lab d12/d2b/d8c/lc4 0 2024-09-16T00:09:48.014 INFO:tasks.workunit.client.0.smithi139.stdout:9/703: mkdir d19/d1a/d20/d89/dbd/def 0 2024-09-16T00:09:48.018 INFO:tasks.workunit.client.0.smithi139.stdout:7/681: dwrite d5/d26/d42/fa5 [0,4194304] 0 2024-09-16T00:09:48.021 INFO:tasks.workunit.client.0.smithi139.stdout:9/704: rename d19/d1a/d20/d23/d2d/d9a/fa1 to d19/d1a/ff0 0 2024-09-16T00:09:48.022 INFO:tasks.workunit.client.0.smithi139.stdout:7/682: fdatasync d5/d16/d6d/d7f/dc2/dab/fcd 0 2024-09-16T00:09:48.022 INFO:tasks.workunit.client.0.smithi139.stdout:4/604: dwrite d12/d2b/f73 [0,4194304] 0 2024-09-16T00:09:48.025 INFO:tasks.workunit.client.0.smithi139.stdout:9/705: chown d19/d1a/d20/d23/dca/db5/cd0 2 1 2024-09-16T00:09:48.025 INFO:tasks.workunit.client.0.smithi139.stdout:3/528: dwrite d5/d7/d23/d27/d2f/d30/d4c/d50/f54 [0,4194304] 0 2024-09-16T00:09:48.027 INFO:tasks.workunit.client.0.smithi139.stdout:4/605: symlink d12/d47/d49/lc5 0 2024-09-16T00:09:48.030 INFO:tasks.workunit.client.0.smithi139.stdout:7/683: read d5/d37/f79 [467429,79058] 0 2024-09-16T00:09:48.032 INFO:tasks.workunit.client.0.smithi139.stdout:7/684: write d5/d16/d6d/f72 [175241,85161] 0 2024-09-16T00:09:48.036 INFO:tasks.workunit.client.0.smithi139.stdout:9/706: dwrite d19/d1a/d20/d23/dca/d71/f57 [0,4194304] 0 2024-09-16T00:09:48.038 INFO:tasks.workunit.client.0.smithi139.stdout:4/606: truncate d12/d80/f95 4173566 0 2024-09-16T00:09:48.041 INFO:tasks.workunit.client.0.smithi139.stdout:8/673: sync 2024-09-16T00:09:48.042 INFO:tasks.workunit.client.0.smithi139.stdout:4/607: symlink d12/d47/db2/d4d/da9/lc6 0 2024-09-16T00:09:48.042 INFO:tasks.workunit.client.0.smithi139.stdout:3/529: dwrite d5/d7/d23/d35/f61 [0,4194304] 0 2024-09-16T00:09:48.044 INFO:tasks.workunit.client.0.smithi139.stdout:7/685: getdents d5/d26/d46/dea 0 2024-09-16T00:09:48.046 INFO:tasks.workunit.client.0.smithi139.stdout:9/707: dread d19/d1a/d20/d23/dca/d9f/dd9/fee [0,4194304] 0 2024-09-16T00:09:48.046 INFO:tasks.workunit.client.0.smithi139.stdout:8/674: mknod d17/d1a/d50/d86/d9d/d9f/ce3 0 2024-09-16T00:09:48.047 INFO:tasks.workunit.client.0.smithi139.stdout:7/686: creat d5/d26/d46/ff9 x:0 0 0 2024-09-16T00:09:48.048 INFO:tasks.workunit.client.0.smithi139.stdout:4/608: creat d12/d47/d49/fc7 x:0 0 0 2024-09-16T00:09:48.048 INFO:tasks.workunit.client.0.smithi139.stdout:3/530: mkdir d5/d7/d23/d27/d2f/d30/d4d/dbc 0 2024-09-16T00:09:48.050 INFO:tasks.workunit.client.0.smithi139.stdout:4/609: chown d12/d47/d49/d60/f76 648068282 1 2024-09-16T00:09:48.050 INFO:tasks.workunit.client.0.smithi139.stdout:8/675: mknod d17/d1a/d50/d86/d9d/daf/ddf/ce4 0 2024-09-16T00:09:48.052 INFO:tasks.workunit.client.0.smithi139.stdout:9/708: getdents d19/d1a/d20/d23/dca/d71/d32/d4e 0 2024-09-16T00:09:48.052 INFO:tasks.workunit.client.0.smithi139.stdout:8/676: dread - d17/d1a/d57/f8b zero size 2024-09-16T00:09:48.053 INFO:tasks.workunit.client.0.smithi139.stdout:4/610: mknod d12/d47/db2/d4d/cc8 0 2024-09-16T00:09:48.055 INFO:tasks.workunit.client.0.smithi139.stdout:8/677: chown d17/d1f/d3a/dad 6 1 2024-09-16T00:09:48.056 INFO:tasks.workunit.client.0.smithi139.stdout:7/687: dread d5/d16/d6d/d7f/dc2/d50/d44/f57 [0,4194304] 0 2024-09-16T00:09:48.056 INFO:tasks.workunit.client.0.smithi139.stdout:4/611: unlink d12/d47/fa5 0 2024-09-16T00:09:48.057 INFO:tasks.workunit.client.0.smithi139.stdout:8/678: readlink d17/d1a/d2d/l8c 0 2024-09-16T00:09:48.058 INFO:tasks.workunit.client.0.smithi139.stdout:7/688: chown d5/d16/d22/fc7 1946548323 1 2024-09-16T00:09:48.059 INFO:tasks.workunit.client.0.smithi139.stdout:9/709: mkdir d19/d1a/d20/d23/d2d/d35/d42/dc8/df1 0 2024-09-16T00:09:48.059 INFO:tasks.workunit.client.0.smithi139.stdout:3/531: dwrite d5/d7/d23/d27/d2f/d30/d4c/f69 [0,4194304] 0 2024-09-16T00:09:48.060 INFO:tasks.workunit.client.0.smithi139.stdout:8/679: write d17/f83 [49434,12424] 0 2024-09-16T00:09:48.062 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:48.062 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:48.062 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:48.062 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:48.063 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:48.063 INFO:tasks.workunit.client.0.smithi139.stdout:7/689: creat d5/d26/d46/d6b/dc5/ffa x:0 0 0 2024-09-16T00:09:48.063 INFO:tasks.workunit.client.0.smithi139.stdout:8/680: rmdir d17/d1f/d3a/db0 39 2024-09-16T00:09:48.064 INFO:tasks.workunit.client.0.smithi139.stdout:3/532: mkdir d5/d7/d23/d27/d2f/d30/d41/d68/dbd 0 2024-09-16T00:09:48.066 INFO:tasks.workunit.client.0.smithi139.stdout:4/612: dwrite d12/d47/db2/d4d/f89 [0,4194304] 0 2024-09-16T00:09:48.069 INFO:tasks.workunit.client.0.smithi139.stdout:7/690: dread d5/d26/f75 [4194304,4194304] 0 2024-09-16T00:09:48.072 INFO:tasks.workunit.client.0.smithi139.stdout:7/691: read - d5/d26/d46/d6b/da7/fe2 zero size 2024-09-16T00:09:48.072 INFO:tasks.workunit.client.0.smithi139.stdout:4/613: unlink d12/d47/d57/f5f 0 2024-09-16T00:09:48.073 INFO:tasks.workunit.client.0.smithi139.stdout:8/681: rename d17/l44 to d17/le5 0 2024-09-16T00:09:48.075 INFO:tasks.workunit.client.0.smithi139.stdout:7/692: stat d5/d16/d6d/d7f/dc2/d50/f67 0 2024-09-16T00:09:48.075 INFO:tasks.workunit.client.0.smithi139.stdout:4/614: symlink d12/d47/lc9 0 2024-09-16T00:09:48.076 INFO:tasks.workunit.client.0.smithi139.stdout:3/533: link d5/d7/d23/d27/d2f/d30/c5d d5/d7/d23/d27/d2f/d30/d4c/cbe 0 2024-09-16T00:09:48.077 INFO:tasks.workunit.client.0.smithi139.stdout:8/682: mknod d17/d1f/d3a/db7/ce6 0 2024-09-16T00:09:48.077 INFO:tasks.workunit.client.0.smithi139.stdout:9/710: dwrite d19/d1a/d20/d23/dca/d9f/dd9/fee [0,4194304] 0 2024-09-16T00:09:48.078 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:09:48.079 INFO:tasks.workunit.client.0.smithi139.stdout:9/711: truncate d19/fd5 240070 0 2024-09-16T00:09:48.079 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:48.078+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:48.081 INFO:tasks.workunit.client.0.smithi139.stdout:7/693: creat d5/d16/d22/ffb x:0 0 0 2024-09-16T00:09:48.081 INFO:tasks.workunit.client.0.smithi139.stdout:4/615: dread d12/d2b/f6b [0,4194304] 0 2024-09-16T00:09:48.081 INFO:tasks.workunit.client.0.smithi139.stdout:3/534: creat d5/d7/fbf x:0 0 0 2024-09-16T00:09:48.084 INFO:tasks.workunit.client.0.smithi139.stdout:9/712: getdents d19/d1a/d20/d23/d2d/d35/d42/de7 0 2024-09-16T00:09:48.084 INFO:tasks.workunit.client.0.smithi139.stdout:7/694: chown d5/d26/d46/d6b/da7/ldd 2822303 1 2024-09-16T00:09:48.084 INFO:tasks.workunit.client.0.smithi139.stdout:8/683: dread d17/d1f/f25 [0,4194304] 0 2024-09-16T00:09:48.086 INFO:tasks.workunit.client.0.smithi139.stdout:3/535: rename d5/d7/d23/d27/d2f/d30 to d5/d7/d23/d27/d2f/d30/d4c/d6d/dc0 22 2024-09-16T00:09:48.086 INFO:tasks.workunit.client.0.smithi139.stdout:8/684: rmdir d17/d1f/d3a 39 2024-09-16T00:09:48.087 INFO:tasks.workunit.client.0.smithi139.stdout:4/616: getdents d12/d47/d49/d60 0 2024-09-16T00:09:48.087 INFO:tasks.workunit.client.0.smithi139.stdout:9/713: mknod d19/d1a/d20/d23/cf2 0 2024-09-16T00:09:48.087 INFO:tasks.workunit.client.0.smithi139.stdout:3/536: write d5/d7/d23/f17 [4172794,84774] 0 2024-09-16T00:09:48.088 INFO:tasks.workunit.client.0.smithi139.stdout:8/685: symlink d17/d89/le7 0 2024-09-16T00:09:48.089 INFO:tasks.workunit.client.0.smithi139.stdout:4/617: write d12/d47/db2/d4d/f81 [2150416,5800] 0 2024-09-16T00:09:48.090 INFO:tasks.workunit.client.0.smithi139.stdout:7/695: truncate d5/d16/d6d/d7f/dc2/d50/d44/f57 1376595 0 2024-09-16T00:09:48.091 INFO:tasks.workunit.client.0.smithi139.stdout:9/714: chown d19/d1a/d20/d23/dca/d71/d32/d4e/f78 272895 1 2024-09-16T00:09:48.091 INFO:tasks.workunit.client.0.smithi139.stdout:4/618: chown d12/d47/d49/d60/l8e 968 1 2024-09-16T00:09:48.091 INFO:tasks.workunit.client.0.smithi139.stdout:3/537: rename d5/d7 to d5/d7/d23/d27/d2f/d30/d4c/dbb/dc1 22 2024-09-16T00:09:48.091 INFO:tasks.workunit.client.0.smithi139.stdout:8/686: mknod d17/d1a/d57/ce8 0 2024-09-16T00:09:48.095 INFO:tasks.workunit.client.0.smithi139.stdout:7/696: rename d5/d26/d46/d6b/dc5/ce8 to d5/d16/d6d/cfc 0 2024-09-16T00:09:48.097 INFO:tasks.workunit.client.0.smithi139.stdout:9/715: mkdir d19/d1a/d20/d23/d2d/d35/d42/d3a/df3 0 2024-09-16T00:09:48.099 INFO:tasks.workunit.client.0.smithi139.stdout:4/619: link d12/d2b/f73 d12/d47/d57/fca 0 2024-09-16T00:09:48.099 INFO:tasks.workunit.client.0.smithi139.stdout:7/697: fdatasync d5/d16/d22/d25/f7a 0 2024-09-16T00:09:48.101 INFO:tasks.workunit.client.0.smithi139.stdout:8/687: link d17/d1f/d3a/db7/d40/ld0 d17/d1f/d3a/dcd/le9 0 2024-09-16T00:09:48.102 INFO:tasks.workunit.client.0.smithi139.stdout:8/688: fdatasync d17/d89/f8f 0 2024-09-16T00:09:48.104 INFO:tasks.workunit.client.0.smithi139.stdout:3/538: rename d5/d7/d23/d27/d2f/d30/d47/dba to d5/d7/dc2 0 2024-09-16T00:09:48.106 INFO:tasks.workunit.client.0.smithi139.stdout:7/698: unlink d5/d26/d46/d6b/da7/ddb/lf0 0 2024-09-16T00:09:48.107 INFO:tasks.workunit.client.0.smithi139.stdout:8/689: dwrite d17/d1a/d2d/f3e [0,4194304] 0 2024-09-16T00:09:48.107 INFO:tasks.workunit.client.0.smithi139.stdout:4/620: creat d12/d47/d49/d60/dc3/fcb x:0 0 0 2024-09-16T00:09:48.109 INFO:tasks.workunit.client.0.smithi139.stdout:7/699: write d5/d37/f47 [2721831,24860] 0 2024-09-16T00:09:48.109 INFO:tasks.workunit.client.0.smithi139.stdout:3/539: rmdir d5/d7/d23/d27/d2f/d30 39 2024-09-16T00:09:48.114 INFO:tasks.workunit.client.0.smithi139.stdout:4/621: symlink d12/d47/db2/lcc 0 2024-09-16T00:09:48.115 INFO:tasks.workunit.client.0.smithi139.stdout:4/622: read d12/d47/db2/f54 [1447843,24131] 0 2024-09-16T00:09:48.116 INFO:tasks.workunit.client.0.smithi139.stdout:9/716: truncate d19/d1a/d20/d23/d2d/d35/d42/d3a/f48 3770982 0 2024-09-16T00:09:48.117 INFO:tasks.workunit.client.0.smithi139.stdout:7/700: mkdir d5/d26/d74/dfd 0 2024-09-16T00:09:48.119 INFO:tasks.workunit.client.0.smithi139.stdout:3/540: chown d5/d7/d23/d27/d2f/d30/d47/caa 689552787 1 2024-09-16T00:09:48.120 INFO:tasks.workunit.client.0.smithi139.stdout:9/717: truncate d19/d1a/d20/d23/d2d/d35/d42/d3a/fe6 805146 0 2024-09-16T00:09:48.122 INFO:tasks.workunit.client.0.smithi139.stdout:3/541: unlink d5/d7/c3a 0 2024-09-16T00:09:48.123 INFO:tasks.workunit.client.0.smithi139.stdout:9/718: unlink d19/d1a/d20/d23/dca/d9f/dd9/fe4 0 2024-09-16T00:09:48.125 INFO:tasks.workunit.client.0.smithi139.stdout:7/701: link d5/d26/l5c d5/d16/d6d/d7f/dc2/d50/d44/d55/lfe 0 2024-09-16T00:09:48.126 INFO:tasks.workunit.client.0.smithi139.stdout:4/623: getdents d12/d47/db2/d4d/da9 0 2024-09-16T00:09:48.126 INFO:tasks.workunit.client.0.smithi139.stdout:9/719: chown d19/d1a/d20/d23/dca/d9f/cc3 818 1 2024-09-16T00:09:48.126 INFO:tasks.workunit.client.0.smithi139.stdout:8/690: dwrite d17/d1f/d3a/db7/d40/d4f/fa7 [0,4194304] 0 2024-09-16T00:09:48.127 INFO:tasks.workunit.client.0.smithi139.stdout:3/542: write d5/d7/d23/d27/d2f/d30/d4c/d50/d75/fb7 [702774,83730] 0 2024-09-16T00:09:48.128 INFO:tasks.workunit.client.0.smithi139.stdout:4/624: chown d12/d47/db2/d4d/l64 826756 1 2024-09-16T00:09:48.129 INFO:tasks.workunit.client.0.smithi139.stdout:4/625: stat c9 0 2024-09-16T00:09:48.130 INFO:tasks.workunit.client.0.smithi139.stdout:3/543: write d5/d7/d23/d27/d2f/f7a [223140,42537] 0 2024-09-16T00:09:48.131 INFO:tasks.workunit.client.0.smithi139.stdout:8/691: mknod d17/d1a/d50/d86/d9d/cea 0 2024-09-16T00:09:48.132 INFO:tasks.workunit.client.0.smithi139.stdout:7/702: symlink d5/d26/d46/dea/df8/lff 0 2024-09-16T00:09:48.133 INFO:tasks.workunit.client.0.smithi139.stdout:3/544: stat d5/d7/d23/d27/d2f/d30/f80 0 2024-09-16T00:09:48.133 INFO:tasks.workunit.client.0.smithi139.stdout:9/720: dread d19/f6c [0,4194304] 0 2024-09-16T00:09:48.136 INFO:tasks.workunit.client.0.smithi139.stdout:7/703: truncate d5/d26/d46/f9e 208093 0 2024-09-16T00:09:48.136 INFO:tasks.workunit.client.0.smithi139.stdout:3/545: read d5/d7/d23/d35/f45 [192892,27282] 0 2024-09-16T00:09:48.137 INFO:tasks.workunit.client.0.smithi139.stdout:8/692: truncate d17/d1f/d3a/dcd/fe0 1700744 0 2024-09-16T00:09:48.139 INFO:tasks.workunit.client.0.smithi139.stdout:4/626: dwrite d12/f42 [4194304,4194304] 0 2024-09-16T00:09:48.139 INFO:tasks.workunit.client.0.smithi139.stdout:8/693: chown d17/d1f/d3a/db7/d40/d4f/l5c 388 1 2024-09-16T00:09:48.141 INFO:tasks.workunit.client.0.smithi139.stdout:7/704: unlink d5/d16/f3e 0 2024-09-16T00:09:48.142 INFO:tasks.workunit.client.0.smithi139.stdout:8/694: write d17/d1f/d3a/db7/f42 [1229070,72743] 0 2024-09-16T00:09:48.144 INFO:tasks.workunit.client.0.smithi139.stdout:8/695: read - d17/d1a/d2d/fe1 zero size 2024-09-16T00:09:48.145 INFO:tasks.workunit.client.0.smithi139.stdout:9/721: dread d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/fad [0,4194304] 0 2024-09-16T00:09:48.146 INFO:tasks.workunit.client.0.smithi139.stdout:7/705: chown d5/d16/d6d/d7f/dc2/d50/d44/d55/f6c 11295 1 2024-09-16T00:09:48.148 INFO:tasks.workunit.client.0.smithi139.stdout:4/627: dwrite d12/f43 [0,4194304] 0 2024-09-16T00:09:48.150 INFO:tasks.workunit.client.0.smithi139.stdout:9/722: mkdir d19/d1a/d20/d23/d2d/d35/d44/df4 0 2024-09-16T00:09:48.153 INFO:tasks.workunit.client.0.smithi139.stdout:3/546: getdents d5/d7/d23/d27/d2f/d30/d4d/dbc 0 2024-09-16T00:09:48.155 INFO:tasks.workunit.client.0.smithi139.stdout:8/696: dwrite d17/d1a/d57/f8b [0,4194304] 0 2024-09-16T00:09:48.158 INFO:tasks.workunit.client.0.smithi139.stdout:4/628: dwrite d12/d47/d57/fb5 [0,4194304] 0 2024-09-16T00:09:48.159 INFO:tasks.workunit.client.0.smithi139.stdout:3/547: dread - d5/d7/d23/d27/d2f/d30/d4c/d6d/db3/fb6 zero size 2024-09-16T00:09:48.161 INFO:tasks.workunit.client.0.smithi139.stdout:8/697: unlink d17/d1a/d57/f7b 0 2024-09-16T00:09:48.161 INFO:tasks.workunit.client.0.smithi139.stdout:4/629: chown d12/d47/f45 1668329 1 2024-09-16T00:09:48.163 INFO:tasks.workunit.client.0.smithi139.stdout:7/706: truncate d5/d16/d22/d25/f76 1662451 0 2024-09-16T00:09:48.165 INFO:tasks.workunit.client.0.smithi139.stdout:9/723: dwrite d19/d1a/d20/d23/d2d/d35/f66 [0,4194304] 0 2024-09-16T00:09:48.166 INFO:tasks.workunit.client.0.smithi139.stdout:8/698: mknod d17/d1f/ceb 0 2024-09-16T00:09:48.167 INFO:tasks.workunit.client.0.smithi139.stdout:4/630: write d12/d47/d57/f62 [2655916,89701] 0 2024-09-16T00:09:48.169 INFO:tasks.workunit.client.0.smithi139.stdout:7/707: symlink d5/d16/d22/d25/d63/l100 0 2024-09-16T00:09:48.170 INFO:tasks.workunit.client.0.smithi139.stdout:4/631: chown d12/d47 344009347 1 2024-09-16T00:09:48.170 INFO:tasks.workunit.client.0.smithi139.stdout:8/699: creat d17/d1a/d50/fec x:0 0 0 2024-09-16T00:09:48.171 INFO:tasks.workunit.client.0.smithi139.stdout:7/708: write d5/d26/f2b [2750299,5868] 0 2024-09-16T00:09:48.173 INFO:tasks.workunit.client.0.smithi139.stdout:3/548: link d5/d7/d23/d27/d2f/d30/c5d d5/d7/dc/d84/cc3 0 2024-09-16T00:09:48.174 INFO:tasks.workunit.client.0.smithi139.stdout:3/549: chown d5/d7/d23/d27/d2f/d30/d4c 20 1 2024-09-16T00:09:48.174 INFO:tasks.workunit.client.0.smithi139.stdout:4/632: write d12/d47/db2/d4d/fbb [225794,89659] 0 2024-09-16T00:09:48.175 INFO:tasks.workunit.client.0.smithi139.stdout:8/700: creat d17/d54/fed x:0 0 0 2024-09-16T00:09:48.179 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:48.178+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:48.180 INFO:tasks.workunit.client.0.smithi139.stdout:3/550: dwrite d5/d7/f28 [0,4194304] 0 2024-09-16T00:09:48.180 INFO:tasks.workunit.client.0.smithi139.stdout:9/724: write d19/d1a/d20/d23/d2d/fa4 [484558,10652] 0 2024-09-16T00:09:48.182 INFO:tasks.workunit.client.0.smithi139.stdout:3/551: readlink d5/d7/d23/d27/d2f/l9b 0 2024-09-16T00:09:48.183 INFO:tasks.workunit.client.0.smithi139.stdout:4/633: dwrite d12/f4f [0,4194304] 0 2024-09-16T00:09:48.183 INFO:tasks.workunit.client.0.smithi139.stdout:9/725: symlink d19/d1a/d20/d89/lf5 0 2024-09-16T00:09:48.184 INFO:tasks.workunit.client.0.smithi139.stdout:3/552: chown d5/d7/dc/d9c/f9d 125587 1 2024-09-16T00:09:48.185 INFO:tasks.workunit.client.0.smithi139.stdout:4/634: chown d12/d47/c28 7 1 2024-09-16T00:09:48.185 INFO:tasks.workunit.client.0.smithi139.stdout:4/635: stat d12 0 2024-09-16T00:09:48.192 INFO:tasks.workunit.client.0.smithi139.stdout:3/553: symlink d5/d7/d23/d27/d2f/d30/d41/d68/dbd/lc4 0 2024-09-16T00:09:48.195 INFO:tasks.workunit.client.0.smithi139.stdout:9/726: dwrite d19/d1a/d20/d23/d2d/d2f/f96 [4194304,4194304] 0 2024-09-16T00:09:48.197 INFO:tasks.workunit.client.0.smithi139.stdout:3/554: creat d5/d7/d23/d27/d2f/d30/d4d/dbc/fc5 x:0 0 0 2024-09-16T00:09:48.198 INFO:tasks.workunit.client.0.smithi139.stdout:9/727: mkdir d19/d1a/d20/d23/dca/db5/df6 0 2024-09-16T00:09:48.199 INFO:tasks.workunit.client.0.smithi139.stdout:9/728: write d19/d86/f5a [4678312,50260] 0 2024-09-16T00:09:48.199 INFO:tasks.workunit.client.0.smithi139.stdout:3/555: chown d5/d7/d23/d27/d2f/d30/d4c/d50/d75/fb7 23995 1 2024-09-16T00:09:48.201 INFO:tasks.workunit.client.0.smithi139.stdout:4/636: dread d12/d47/f6c [0,4194304] 0 2024-09-16T00:09:48.203 INFO:tasks.workunit.client.0.smithi139.stdout:9/729: symlink d19/d1a/d20/d23/d2d/d35/d44/lf7 0 2024-09-16T00:09:48.205 INFO:tasks.workunit.client.0.smithi139.stdout:3/556: rmdir d5/d7/d23/d27/dab/db2 0 2024-09-16T00:09:48.206 INFO:tasks.workunit.client.0.smithi139.stdout:3/557: dread - d5/d7/d23/d35/f46 zero size 2024-09-16T00:09:48.206 INFO:tasks.workunit.client.0.smithi139.stdout:9/730: unlink d19/d1a/d20/d23/d2d/d2f/c7d 0 2024-09-16T00:09:48.208 INFO:tasks.workunit.client.0.smithi139.stdout:4/637: write d12/d2b/f73 [5221153,16209] 0 2024-09-16T00:09:48.209 INFO:tasks.workunit.client.0.smithi139.stdout:9/731: mknod d19/d1a/d20/d23/dca/d71/d32/cf8 0 2024-09-16T00:09:48.211 INFO:tasks.workunit.client.0.smithi139.stdout:4/638: write d12/d47/db2/d4d/f79 [1287482,56345] 0 2024-09-16T00:09:48.212 INFO:tasks.workunit.client.0.smithi139.stdout:9/732: stat d19/d1a/d20/d23/d2d/d35/d44/c94 0 2024-09-16T00:09:48.213 INFO:tasks.workunit.client.0.smithi139.stdout:3/558: dwrite d5/d7/d23/d27/d2f/d30/d4c/d6d/db3/fb6 [0,4194304] 0 2024-09-16T00:09:48.214 INFO:tasks.workunit.client.0.smithi139.stdout:4/639: creat d12/d47/db2/d4d/da9/fcd x:0 0 0 2024-09-16T00:09:48.215 INFO:tasks.workunit.client.0.smithi139.stdout:9/733: symlink d19/d1a/d20/d23/dca/d71/d32/d4e/lf9 0 2024-09-16T00:09:48.216 INFO:tasks.workunit.client.0.smithi139.stdout:3/559: mkdir d5/d7/d23/d27/d2f/d30/d4d/dc6 0 2024-09-16T00:09:48.217 INFO:tasks.workunit.client.0.smithi139.stdout:9/734: dread - d19/d1a/d20/d89/d93/fdb zero size 2024-09-16T00:09:48.217 INFO:tasks.workunit.client.0.smithi139.stdout:3/560: readlink d5/d7/d23/d35/da7/la8 0 2024-09-16T00:09:48.222 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:48.222 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:48.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:48.222 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:48.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.222 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:09:48.222 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:48.222 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 145.137525754, 2024-09-16T00:09:48.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:09:48.223 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.223 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.223 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:48.223 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:48.224 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:09:48.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.224 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:48.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:09:48.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:09:48.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:09:48.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:09:48.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:09:48.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:09:48.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:09:48.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:09:48.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:09:48.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:09:48.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:09:48.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:09:48.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:09:48.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:09:48.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:09:48.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:09:48.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:09:48.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:09:48.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:48.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:09:48.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:09:48.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:09:48.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:09:48.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:09:48.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:09:48.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:09:48.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:09:48.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:09:48.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:09:48.232 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.232 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.232 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.233 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-16T00:09:48.233 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:48.233 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 143.608434875, 2024-09-16T00:09:48.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014331501999999999, 2024-09-16T00:09:48.233 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.233 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.233 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.233 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.233 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:09:48.234 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:09:48.234 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 141 2024-09-16T00:09:48.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.234 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:48.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609920+0000", 2024-09-16T00:09:48.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609921+0000", 2024-09-16T00:09:48.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9599999999999999e-07 2024-09-16T00:09:48.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609925+0000", 2024-09-16T00:09:48.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7230000000000002e-06 2024-09-16T00:09:48.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609932+0000", 2024-09-16T00:09:48.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8240000000000002e-06 2024-09-16T00:09:48.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.609937+0000", 2024-09-16T00:09:48.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1070000000000004e-06 2024-09-16T00:09:48.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.618987+0000", 2024-09-16T00:09:48.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090502719999999998 2024-09-16T00:09:48.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619064+0000", 2024-09-16T00:09:48.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6868999999999999e-05 2024-09-16T00:09:48.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:48.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.619183+0000", 2024-09-16T00:09:48.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011961 2024-09-16T00:09:48.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.620052+0000", 2024-09-16T00:09:48.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00086822799999999997 2024-09-16T00:09:48.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624197+0000", 2024-09-16T00:09:48.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041454930000000001 2024-09-16T00:09:48.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624229+0000", 2024-09-16T00:09:48.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1758999999999999e-05 2024-09-16T00:09:48.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:24.624252+0000", 2024-09-16T00:09:48.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3020999999999999e-05 2024-09-16T00:09:48.240 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.241 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.241 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10651 3.39 3:9d97a463:::100000030ab.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:48.241 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.621786+0000", 2024-09-16T00:09:48.241 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.596569518000001, 2024-09-16T00:09:48.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015164901999999999, 2024-09-16T00:09:48.241 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.241 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.241 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.242 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.242 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.242 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.242 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10651 2024-09-16T00:09:48.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.242 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.242 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.242 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.621786+0000", 2024-09-16T00:09:48.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.621786+0000", 2024-09-16T00:09:48.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.621788+0000", 2024-09-16T00:09:48.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7999999999999999e-06 2024-09-16T00:09:48.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.626753+0000", 2024-09-16T00:09:48.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004965667 2024-09-16T00:09:48.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.626771+0000", 2024-09-16T00:09:48.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7816999999999999e-05 2024-09-16T00:09:48.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.626778+0000", 2024-09-16T00:09:48.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3869999999999997e-06 2024-09-16T00:09:48.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.626843+0000", 2024-09-16T00:09:48.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4969999999999996e-05 2024-09-16T00:09:48.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.627002+0000", 2024-09-16T00:09:48.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000158441 2024-09-16T00:09:48.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:48.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.627147+0000", 2024-09-16T00:09:48.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014563999999999999 2024-09-16T00:09:48.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.246 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.246 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.634047+0000", 2024-09-16T00:09:48.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006899222 2024-09-16T00:09:48.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.636883+0000", 2024-09-16T00:09:48.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028367890000000002 2024-09-16T00:09:48.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.636918+0000", 2024-09-16T00:09:48.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4397999999999998e-05 2024-09-16T00:09:48.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.636951+0000", 2024-09-16T00:09:48.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2771000000000002e-05 2024-09-16T00:09:48.248 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.248 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.248 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.248 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10770 3.15 3:aa8f498f:::100000030ba.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:48.248 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.404839+0000", 2024-09-16T00:09:48.249 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.8135163490000004, 2024-09-16T00:09:48.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013644351000000001, 2024-09-16T00:09:48.249 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.249 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.249 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.249 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.249 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.249 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.249 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10770 2024-09-16T00:09:48.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.249 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.404839+0000", 2024-09-16T00:09:48.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.404839+0000", 2024-09-16T00:09:48.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.404840+0000", 2024-09-16T00:09:48.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5710000000000001e-06 2024-09-16T00:09:48.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.409487+0000", 2024-09-16T00:09:48.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046467380000000001 2024-09-16T00:09:48.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.409500+0000", 2024-09-16T00:09:48.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3246e-05 2024-09-16T00:09:48.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.409511+0000", 2024-09-16T00:09:48.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0852999999999999e-05 2024-09-16T00:09:48.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.409577+0000", 2024-09-16T00:09:48.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5429e-05 2024-09-16T00:09:48.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.409675+0000", 2024-09-16T00:09:48.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7808999999999998e-05 2024-09-16T00:09:48.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:48.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.409770+0000", 2024-09-16T00:09:48.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5110000000000002e-05 2024-09-16T00:09:48.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.414205+0000", 2024-09-16T00:09:48.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044357420000000003 2024-09-16T00:09:48.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.418378+0000", 2024-09-16T00:09:48.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041728989999999999 2024-09-16T00:09:48.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.418441+0000", 2024-09-16T00:09:48.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2438000000000001e-05 2024-09-16T00:09:48.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.418483+0000", 2024-09-16T00:09:48.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2515999999999997e-05 2024-09-16T00:09:48.255 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.255 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.255 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.256 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10773 3.39 3:9d207bac:::100000030d8.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:48.256 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.423389+0000", 2024-09-16T00:09:48.256 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.7949661040000002, 2024-09-16T00:09:48.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014464233999999999, 2024-09-16T00:09:48.256 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.256 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.256 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.256 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.256 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.256 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.256 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10773 2024-09-16T00:09:48.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.257 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.423389+0000", 2024-09-16T00:09:48.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.423389+0000", 2024-09-16T00:09:48.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.423391+0000", 2024-09-16T00:09:48.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.406e-06 2024-09-16T00:09:48.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.427961+0000", 2024-09-16T00:09:48.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045706740000000003 2024-09-16T00:09:48.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.427974+0000", 2024-09-16T00:09:48.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2388000000000001e-05 2024-09-16T00:09:48.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.427980+0000", 2024-09-16T00:09:48.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3559999999999998e-06 2024-09-16T00:09:48.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.428035+0000", 2024-09-16T00:09:48.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5096999999999998e-05 2024-09-16T00:09:48.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.428105+0000", 2024-09-16T00:09:48.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9528000000000002e-05 2024-09-16T00:09:48.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:48.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.428183+0000", 2024-09-16T00:09:48.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8619000000000001e-05 2024-09-16T00:09:48.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.432734+0000", 2024-09-16T00:09:48.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00455092 2024-09-16T00:09:48.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.437748+0000", 2024-09-16T00:09:48.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050134979999999999 2024-09-16T00:09:48.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.437804+0000", 2024-09-16T00:09:48.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6576000000000002e-05 2024-09-16T00:09:48.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.437853+0000", 2024-09-16T00:09:48.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9172000000000003e-05 2024-09-16T00:09:48.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.262 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.262 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.263 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10826 3.11 3:889a8549:::10000003149.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:48.263 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.741353+0000", 2024-09-16T00:09:48.263 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.4770020099999996, 2024-09-16T00:09:48.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014025037000000001, 2024-09-16T00:09:48.263 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.263 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.263 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.263 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.264 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.264 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10826 2024-09-16T00:09:48.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.264 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.741353+0000", 2024-09-16T00:09:48.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.741353+0000", 2024-09-16T00:09:48.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.741354+0000", 2024-09-16T00:09:48.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2440000000000001e-06 2024-09-16T00:09:48.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.746855+0000", 2024-09-16T00:09:48.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055001750000000004 2024-09-16T00:09:48.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.746867+0000", 2024-09-16T00:09:48.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2132e-05 2024-09-16T00:09:48.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.746873+0000", 2024-09-16T00:09:48.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0530000000000003e-06 2024-09-16T00:09:48.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.746921+0000", 2024-09-16T00:09:48.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8018e-05 2024-09-16T00:09:48.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.747092+0000", 2024-09-16T00:09:48.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00017092100000000001 2024-09-16T00:09:48.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:48.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.747197+0000", 2024-09-16T00:09:48.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105571 2024-09-16T00:09:48.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.754346+0000", 2024-09-16T00:09:48.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071485439999999997 2024-09-16T00:09:48.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.755286+0000", 2024-09-16T00:09:48.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00093984899999999998 2024-09-16T00:09:48.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.755340+0000", 2024-09-16T00:09:48.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4397000000000001e-05 2024-09-16T00:09:48.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.755378+0000", 2024-09-16T00:09:48.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8133000000000002e-05 2024-09-16T00:09:48.270 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.270 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.270 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.270 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11098 3.15 3:aa8f498f:::100000030ba.00000000:head [write 0~4194304 [2@1999769]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:48.271 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.016861+0000", 2024-09-16T00:09:48.271 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.2014944960000005, 2024-09-16T00:09:48.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018826408999999999, 2024-09-16T00:09:48.271 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.271 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.271 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.271 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.271 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.272 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.272 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11098 2024-09-16T00:09:48.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.272 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.016861+0000", 2024-09-16T00:09:48.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.016861+0000", 2024-09-16T00:09:48.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.016863+0000", 2024-09-16T00:09:48.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5390000000000002e-06 2024-09-16T00:09:48.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.021057+0000", 2024-09-16T00:09:48.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041934279999999999 2024-09-16T00:09:48.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.021081+0000", 2024-09-16T00:09:48.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4332e-05 2024-09-16T00:09:48.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.021092+0000", 2024-09-16T00:09:48.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0559e-05 2024-09-16T00:09:48.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.021212+0000", 2024-09-16T00:09:48.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120377 2024-09-16T00:09:48.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.021284+0000", 2024-09-16T00:09:48.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2311999999999995e-05 2024-09-16T00:09:48.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:48.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.021384+0000", 2024-09-16T00:09:48.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9418000000000003e-05 2024-09-16T00:09:48.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.026734+0000", 2024-09-16T00:09:48.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0053502660000000002 2024-09-16T00:09:48.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.035567+0000", 2024-09-16T00:09:48.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088331299999999998 2024-09-16T00:09:48.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.035626+0000", 2024-09-16T00:09:48.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9290000000000003e-05 2024-09-16T00:09:48.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.035687+0000", 2024-09-16T00:09:48.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0757999999999997e-05 2024-09-16T00:09:48.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.278 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.278 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11162 3.11 3:88de54a5:::10000003191.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:48.278 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.391787+0000", 2024-09-16T00:09:48.278 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8265684289999999, 2024-09-16T00:09:48.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015917648999999999, 2024-09-16T00:09:48.279 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.279 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.279 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.279 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.279 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11162 2024-09-16T00:09:48.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.279 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.391787+0000", 2024-09-16T00:09:48.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.391787+0000", 2024-09-16T00:09:48.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.391789+0000", 2024-09-16T00:09:48.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5730000000000002e-06 2024-09-16T00:09:48.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.396253+0000", 2024-09-16T00:09:48.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044639450000000004 2024-09-16T00:09:48.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.396277+0000", 2024-09-16T00:09:48.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4105999999999999e-05 2024-09-16T00:09:48.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.396289+0000", 2024-09-16T00:09:48.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1232e-05 2024-09-16T00:09:48.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.396443+0000", 2024-09-16T00:09:48.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000154155 2024-09-16T00:09:48.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.396624+0000", 2024-09-16T00:09:48.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018098300000000001 2024-09-16T00:09:48.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:48.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.396779+0000", 2024-09-16T00:09:48.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015505799999999999 2024-09-16T00:09:48.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.401976+0000", 2024-09-16T00:09:48.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051975179999999999 2024-09-16T00:09:48.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.407575+0000", 2024-09-16T00:09:48.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055986910000000003 2024-09-16T00:09:48.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.407643+0000", 2024-09-16T00:09:48.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8355000000000005e-05 2024-09-16T00:09:48.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.407704+0000", 2024-09-16T00:09:48.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1033000000000003e-05 2024-09-16T00:09:48.285 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.285 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.288 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.289 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11163 3.d 3:b08debfe:::1000000312e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:48.289 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.399573+0000", 2024-09-16T00:09:48.289 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8187825399999999, 2024-09-16T00:09:48.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014239757, 2024-09-16T00:09:48.289 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.289 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.289 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.290 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.290 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11163 2024-09-16T00:09:48.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.290 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.399573+0000", 2024-09-16T00:09:48.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.399573+0000", 2024-09-16T00:09:48.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.399575+0000", 2024-09-16T00:09:48.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0590000000000001e-06 2024-09-16T00:09:48.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.403675+0000", 2024-09-16T00:09:48.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041001750000000002 2024-09-16T00:09:48.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.403689+0000", 2024-09-16T00:09:48.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4551e-05 2024-09-16T00:09:48.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.403698+0000", 2024-09-16T00:09:48.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4950000000000008e-06 2024-09-16T00:09:48.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.403767+0000", 2024-09-16T00:09:48.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8674e-05 2024-09-16T00:09:48.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.403868+0000", 2024-09-16T00:09:48.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101659 2024-09-16T00:09:48.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:48.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.404014+0000", 2024-09-16T00:09:48.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000145396 2024-09-16T00:09:48.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.411247+0000", 2024-09-16T00:09:48.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0072330019999999997 2024-09-16T00:09:48.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.413705+0000", 2024-09-16T00:09:48.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024581569999999999 2024-09-16T00:09:48.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.413764+0000", 2024-09-16T00:09:48.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9005999999999997e-05 2024-09-16T00:09:48.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.413812+0000", 2024-09-16T00:09:48.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8582999999999999e-05 2024-09-16T00:09:48.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.297 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.297 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11281 3.f 3:f35949a2:::100000030f0.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:48.297 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.798967+0000", 2024-09-16T00:09:48.297 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.4193881660000001, 2024-09-16T00:09:48.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014241894999999999, 2024-09-16T00:09:48.297 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.297 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.297 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.298 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.298 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.298 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.298 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11281 2024-09-16T00:09:48.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.298 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.798967+0000", 2024-09-16T00:09:48.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.798967+0000", 2024-09-16T00:09:48.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.798968+0000", 2024-09-16T00:09:48.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2810000000000001e-06 2024-09-16T00:09:48.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.803629+0000", 2024-09-16T00:09:48.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004660628 2024-09-16T00:09:48.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.803648+0000", 2024-09-16T00:09:48.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8654000000000001e-05 2024-09-16T00:09:48.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.803658+0000", 2024-09-16T00:09:48.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9490000000000008e-06 2024-09-16T00:09:48.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.803727+0000", 2024-09-16T00:09:48.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9727000000000005e-05 2024-09-16T00:09:48.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.803850+0000", 2024-09-16T00:09:48.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012224600000000001 2024-09-16T00:09:48.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:48.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.804048+0000", 2024-09-16T00:09:48.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00019871799999999999 2024-09-16T00:09:48.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.812750+0000", 2024-09-16T00:09:48.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0087017009999999992 2024-09-16T00:09:48.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.813111+0000", 2024-09-16T00:09:48.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036071399999999999 2024-09-16T00:09:48.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.813162+0000", 2024-09-16T00:09:48.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1319000000000001e-05 2024-09-16T00:09:48.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.813209+0000", 2024-09-16T00:09:48.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6958e-05 2024-09-16T00:09:48.303 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.303 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.303 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.303 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11830 3.d 3:b3bd9509:::100000031dc.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.303 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.431258+0000", 2024-09-16T00:09:48.304 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.7870969309999998, 2024-09-16T00:09:48.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015383539999999999, 2024-09-16T00:09:48.304 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.304 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.304 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.304 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.304 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.304 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.304 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11830 2024-09-16T00:09:48.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.304 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.431258+0000", 2024-09-16T00:09:48.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.431258+0000", 2024-09-16T00:09:48.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.431261+0000", 2024-09-16T00:09:48.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2400000000000002e-06 2024-09-16T00:09:48.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.436543+0000", 2024-09-16T00:09:48.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052820740000000003 2024-09-16T00:09:48.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.436566+0000", 2024-09-16T00:09:48.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3536999999999998e-05 2024-09-16T00:09:48.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.436578+0000", 2024-09-16T00:09:48.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.216e-05 2024-09-16T00:09:48.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.436658+0000", 2024-09-16T00:09:48.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9558e-05 2024-09-16T00:09:48.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.436763+0000", 2024-09-16T00:09:48.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010471000000000001 2024-09-16T00:09:48.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:48.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.436923+0000", 2024-09-16T00:09:48.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016087600000000001 2024-09-16T00:09:48.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.443272+0000", 2024-09-16T00:09:48.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063481420000000002 2024-09-16T00:09:48.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.446535+0000", 2024-09-16T00:09:48.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032632440000000002 2024-09-16T00:09:48.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.446596+0000", 2024-09-16T00:09:48.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1001000000000001e-05 2024-09-16T00:09:48.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.446642+0000", 2024-09-16T00:09:48.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5998000000000002e-05 2024-09-16T00:09:48.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.310 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.311 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11894 3.11 3:8929e5dc:::100000032b6.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.311 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.577585+0000", 2024-09-16T00:09:48.311 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.6407705530000003, 2024-09-16T00:09:48.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016935823999999999, 2024-09-16T00:09:48.311 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.311 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.311 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.311 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.311 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11894 2024-09-16T00:09:48.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.312 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.577585+0000", 2024-09-16T00:09:48.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.577585+0000", 2024-09-16T00:09:48.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.577588+0000", 2024-09-16T00:09:48.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0060000000000001e-06 2024-09-16T00:09:48.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.582079+0000", 2024-09-16T00:09:48.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044913380000000001 2024-09-16T00:09:48.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.582103+0000", 2024-09-16T00:09:48.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3575e-05 2024-09-16T00:09:48.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.582114+0000", 2024-09-16T00:09:48.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1766000000000001e-05 2024-09-16T00:09:48.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.582261+0000", 2024-09-16T00:09:48.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000146596 2024-09-16T00:09:48.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.582354+0000", 2024-09-16T00:09:48.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2677999999999996e-05 2024-09-16T00:09:48.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:48.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.582501+0000", 2024-09-16T00:09:48.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000147773 2024-09-16T00:09:48.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.590250+0000", 2024-09-16T00:09:48.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077488160000000004 2024-09-16T00:09:48.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.594405+0000", 2024-09-16T00:09:48.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041547420000000003 2024-09-16T00:09:48.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.594470+0000", 2024-09-16T00:09:48.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4729999999999999e-05 2024-09-16T00:09:48.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.594521+0000", 2024-09-16T00:09:48.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0803999999999997e-05 2024-09-16T00:09:48.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.317 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.317 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11901 3.39 3:9f6e58bb:::10000003202.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.317 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.585459+0000", 2024-09-16T00:09:48.318 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.6328960060000002, 2024-09-16T00:09:48.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014913165000000001, 2024-09-16T00:09:48.318 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.318 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.318 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.318 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.318 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11901 2024-09-16T00:09:48.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.318 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.585459+0000", 2024-09-16T00:09:48.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.585459+0000", 2024-09-16T00:09:48.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.585460+0000", 2024-09-16T00:09:48.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.215e-06 2024-09-16T00:09:48.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.590006+0000", 2024-09-16T00:09:48.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004545398 2024-09-16T00:09:48.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.590020+0000", 2024-09-16T00:09:48.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3946e-05 2024-09-16T00:09:48.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.590028+0000", 2024-09-16T00:09:48.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1710000000000002e-06 2024-09-16T00:09:48.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.590091+0000", 2024-09-16T00:09:48.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2552999999999996e-05 2024-09-16T00:09:48.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.590171+0000", 2024-09-16T00:09:48.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0654000000000004e-05 2024-09-16T00:09:48.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:48.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.590280+0000", 2024-09-16T00:09:48.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000108823 2024-09-16T00:09:48.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.594955+0000", 2024-09-16T00:09:48.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046746510000000002 2024-09-16T00:09:48.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.600250+0000", 2024-09-16T00:09:48.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052952279999999999 2024-09-16T00:09:48.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.600316+0000", 2024-09-16T00:09:48.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6107999999999998e-05 2024-09-16T00:09:48.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.600372+0000", 2024-09-16T00:09:48.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6418000000000001e-05 2024-09-16T00:09:48.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.323 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.324 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11940 3.2d 3:b522f437:::1000000335a.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.324 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.719979+0000", 2024-09-16T00:09:48.324 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4983762660000002, 2024-09-16T00:09:48.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018463778, 2024-09-16T00:09:48.324 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.324 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.324 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.324 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.325 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.325 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11940 2024-09-16T00:09:48.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.325 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.719979+0000", 2024-09-16T00:09:48.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.719979+0000", 2024-09-16T00:09:48.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.719981+0000", 2024-09-16T00:09:48.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0940000000000002e-06 2024-09-16T00:09:48.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.724176+0000", 2024-09-16T00:09:48.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041950909999999998 2024-09-16T00:09:48.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.724194+0000", 2024-09-16T00:09:48.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7659000000000002e-05 2024-09-16T00:09:48.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.724204+0000", 2024-09-16T00:09:48.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0431000000000001e-05 2024-09-16T00:09:48.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.724280+0000", 2024-09-16T00:09:48.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6099000000000004e-05 2024-09-16T00:09:48.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.724383+0000", 2024-09-16T00:09:48.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102598 2024-09-16T00:09:48.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:48.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.724548+0000", 2024-09-16T00:09:48.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000164857 2024-09-16T00:09:48.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.737989+0000", 2024-09-16T00:09:48.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013440821 2024-09-16T00:09:48.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.738344+0000", 2024-09-16T00:09:48.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035568299999999998 2024-09-16T00:09:48.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.738402+0000", 2024-09-16T00:09:48.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7552000000000001e-05 2024-09-16T00:09:48.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.738443+0000", 2024-09-16T00:09:48.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0893000000000003e-05 2024-09-16T00:09:48.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.330 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.330 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.331 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12195 3.13 3:cb7758dd:::10000003116.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.331 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.669809+0000", 2024-09-16T00:09:48.331 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.5485461069999999, 2024-09-16T00:09:48.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013549189999999999, 2024-09-16T00:09:48.331 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.331 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.331 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.331 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.331 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12195 2024-09-16T00:09:48.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.332 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.669809+0000", 2024-09-16T00:09:48.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.669809+0000", 2024-09-16T00:09:48.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.669812+0000", 2024-09-16T00:09:48.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3599999999999999e-06 2024-09-16T00:09:48.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.673952+0000", 2024-09-16T00:09:48.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004140375 2024-09-16T00:09:48.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.673973+0000", 2024-09-16T00:09:48.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.09e-05 2024-09-16T00:09:48.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.673984+0000", 2024-09-16T00:09:48.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1701e-05 2024-09-16T00:09:48.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.674086+0000", 2024-09-16T00:09:48.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000101675 2024-09-16T00:09:48.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.674206+0000", 2024-09-16T00:09:48.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011946399999999999 2024-09-16T00:09:48.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:48.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.674321+0000", 2024-09-16T00:09:48.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115618 2024-09-16T00:09:48.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.681970+0000", 2024-09-16T00:09:48.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0076483779999999999 2024-09-16T00:09:48.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.683240+0000", 2024-09-16T00:09:48.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012704000000000001 2024-09-16T00:09:48.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.683303+0000", 2024-09-16T00:09:48.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3157000000000005e-05 2024-09-16T00:09:48.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.683358+0000", 2024-09-16T00:09:48.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5161999999999998e-05 2024-09-16T00:09:48.337 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.337 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.337 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.337 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12259 3.39 3:9e4b9d23:::100000033f2.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.337 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.816808+0000", 2024-09-16T00:09:48.337 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4015468449999999, 2024-09-16T00:09:48.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015648237999999998, 2024-09-16T00:09:48.337 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.338 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.338 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.338 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.338 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.338 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.338 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12259 2024-09-16T00:09:48.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.338 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.816808+0000", 2024-09-16T00:09:48.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.816808+0000", 2024-09-16T00:09:48.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.816811+0000", 2024-09-16T00:09:48.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1009999999999999e-06 2024-09-16T00:09:48.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.821462+0000", 2024-09-16T00:09:48.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046518080000000003 2024-09-16T00:09:48.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.821485+0000", 2024-09-16T00:09:48.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2858999999999999e-05 2024-09-16T00:09:48.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.821496+0000", 2024-09-16T00:09:48.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0677e-05 2024-09-16T00:09:48.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.821561+0000", 2024-09-16T00:09:48.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5584000000000001e-05 2024-09-16T00:09:48.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.821703+0000", 2024-09-16T00:09:48.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014150299999999999 2024-09-16T00:09:48.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:48.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.821844+0000", 2024-09-16T00:09:48.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014085000000000001 2024-09-16T00:09:48.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.829407+0000", 2024-09-16T00:09:48.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0075634580000000003 2024-09-16T00:09:48.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.832340+0000", 2024-09-16T00:09:48.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029328179999999998 2024-09-16T00:09:48.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.832402+0000", 2024-09-16T00:09:48.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2024000000000001e-05 2024-09-16T00:09:48.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.832457+0000", 2024-09-16T00:09:48.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4555999999999997e-05 2024-09-16T00:09:48.343 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.343 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.343 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.343 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12280 3.15 3:aa8f498f:::100000030ba.00000000:head [write 0~4194304 [3@204350]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.343 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.860811+0000", 2024-09-16T00:09:48.343 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.3575437949999998, 2024-09-16T00:09:48.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014461777, 2024-09-16T00:09:48.344 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.344 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.344 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.344 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.344 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12280 2024-09-16T00:09:48.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.344 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.860811+0000", 2024-09-16T00:09:48.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.860811+0000", 2024-09-16T00:09:48.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.860813+0000", 2024-09-16T00:09:48.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8300000000000001e-06 2024-09-16T00:09:48.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.865448+0000", 2024-09-16T00:09:48.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046346850000000004 2024-09-16T00:09:48.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.865465+0000", 2024-09-16T00:09:48.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7322000000000001e-05 2024-09-16T00:09:48.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.865475+0000", 2024-09-16T00:09:48.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.679e-06 2024-09-16T00:09:48.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.865546+0000", 2024-09-16T00:09:48.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1058999999999998e-05 2024-09-16T00:09:48.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.865629+0000", 2024-09-16T00:09:48.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3387000000000006e-05 2024-09-16T00:09:48.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:48.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.865708+0000", 2024-09-16T00:09:48.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8144000000000003e-05 2024-09-16T00:09:48.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.870969+0000", 2024-09-16T00:09:48.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052609650000000003 2024-09-16T00:09:48.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.875169+0000", 2024-09-16T00:09:48.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042007260000000001 2024-09-16T00:09:48.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.875227+0000", 2024-09-16T00:09:48.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7309000000000003e-05 2024-09-16T00:09:48.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.875273+0000", 2024-09-16T00:09:48.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6671000000000001e-05 2024-09-16T00:09:48.349 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.349 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.349 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.349 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12390 3.13 3:c88da5ba:::1000000310a.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.350 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:45.209456+0000", 2024-09-16T00:09:48.350 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.0088991190000001, 2024-09-16T00:09:48.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015856953, 2024-09-16T00:09:48.350 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.350 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.350 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.350 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.350 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.350 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.350 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12390 2024-09-16T00:09:48.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.350 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:45.209456+0000", 2024-09-16T00:09:48.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:45.209456+0000", 2024-09-16T00:09:48.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:45.209459+0000", 2024-09-16T00:09:48.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3769999999999999e-06 2024-09-16T00:09:48.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:45.215015+0000", 2024-09-16T00:09:48.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055569549999999997 2024-09-16T00:09:48.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:45.215042+0000", 2024-09-16T00:09:48.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6188999999999999e-05 2024-09-16T00:09:48.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:45.215050+0000", 2024-09-16T00:09:48.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6289999999999999e-06 2024-09-16T00:09:48.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:45.215132+0000", 2024-09-16T00:09:48.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1850999999999999e-05 2024-09-16T00:09:48.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:45.215232+0000", 2024-09-16T00:09:48.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9603000000000002e-05 2024-09-16T00:09:48.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:48.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:45.215374+0000", 2024-09-16T00:09:48.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014274099999999999 2024-09-16T00:09:48.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:45.220473+0000", 2024-09-16T00:09:48.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050987589999999996 2024-09-16T00:09:48.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:45.225242+0000", 2024-09-16T00:09:48.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047692660000000003 2024-09-16T00:09:48.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:45.225281+0000", 2024-09-16T00:09:48.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8028000000000003e-05 2024-09-16T00:09:48.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:45.225313+0000", 2024-09-16T00:09:48.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2555000000000003e-05 2024-09-16T00:09:48.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.356 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.356 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13366 3.15 3:aa8f498f:::100000030ba.00000000:head [write 0~4194304 [4@1825342]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:48.356 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.023585+0000", 2024-09-16T00:09:48.356 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.194770045, 2024-09-16T00:09:48.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019483191, 2024-09-16T00:09:48.357 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.357 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.357 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.357 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.357 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13366 2024-09-16T00:09:48.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.357 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.023585+0000", 2024-09-16T00:09:48.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.023585+0000", 2024-09-16T00:09:48.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.023587+0000", 2024-09-16T00:09:48.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.57e-06 2024-09-16T00:09:48.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.029577+0000", 2024-09-16T00:09:48.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059906860000000003 2024-09-16T00:09:48.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.029593+0000", 2024-09-16T00:09:48.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5586999999999998e-05 2024-09-16T00:09:48.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.029602+0000", 2024-09-16T00:09:48.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1079999999999999e-06 2024-09-16T00:09:48.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.029677+0000", 2024-09-16T00:09:48.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4373000000000001e-05 2024-09-16T00:09:48.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.029754+0000", 2024-09-16T00:09:48.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7829000000000003e-05 2024-09-16T00:09:48.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:48.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.029862+0000", 2024-09-16T00:09:48.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107859 2024-09-16T00:09:48.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.035992+0000", 2024-09-16T00:09:48.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061298860000000002 2024-09-16T00:09:48.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.042997+0000", 2024-09-16T00:09:48.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070053340000000002 2024-09-16T00:09:48.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.043031+0000", 2024-09-16T00:09:48.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4026999999999998e-05 2024-09-16T00:09:48.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.043068+0000", 2024-09-16T00:09:48.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6931999999999998e-05 2024-09-16T00:09:48.364 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.364 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.364 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.364 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13392 3.15 3:ab224bc6:::100000031b9.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:48.364 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.108056+0000", 2024-09-16T00:09:48.364 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.110298757, 2024-09-16T00:09:48.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013439225000000001, 2024-09-16T00:09:48.364 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:48.364 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:48.364 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:48.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:48.365 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.365 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13392 2024-09-16T00:09:48.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.365 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:48.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:48.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.108056+0000", 2024-09-16T00:09:48.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:48.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.108056+0000", 2024-09-16T00:09:48.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:48.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:48.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.108058+0000", 2024-09-16T00:09:48.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2270000000000001e-06 2024-09-16T00:09:48.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:48.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.112198+0000", 2024-09-16T00:09:48.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041397889999999996 2024-09-16T00:09:48.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:48.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.112218+0000", 2024-09-16T00:09:48.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0699999999999998e-05 2024-09-16T00:09:48.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.112229+0000", 2024-09-16T00:09:48.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1235e-05 2024-09-16T00:09:48.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:48.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.112295+0000", 2024-09-16T00:09:48.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6035e-05 2024-09-16T00:09:48.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:48.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.112393+0000", 2024-09-16T00:09:48.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7113000000000003e-05 2024-09-16T00:09:48.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:48.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.112546+0000", 2024-09-16T00:09:48.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000153614 2024-09-16T00:09:48.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:48.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.117685+0000", 2024-09-16T00:09:48.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051384890000000004 2024-09-16T00:09:48.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.121382+0000", 2024-09-16T00:09:48.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036975630000000001 2024-09-16T00:09:48.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:48.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.121447+0000", 2024-09-16T00:09:48.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5232000000000002e-05 2024-09-16T00:09:48.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:48.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:48.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:48.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.121496+0000", 2024-09-16T00:09:48.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8228e-05 2024-09-16T00:09:48.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.370 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.370 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:48.370 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:48.370 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:48.227+0000 7fa54b7fe640 1 -- 172.21.15.139:0/2715215125 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v89) ==== 91+0+0 (secure 0 0 0) 0x7fa55c059350 con 0x7fa56415fba0 2024-09-16T00:09:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is unset 2024-09-16T00:09:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:48.231+0000 7fa5697ed640 1 -- 172.21.15.139:0/2715215125 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa55006c2b0 msgr2=0x7fa55006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:48.231+0000 7fa5697ed640 1 --2- 172.21.15.139:0/2715215125 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa55006c2b0 0x7fa55006e770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa554006fd0 tx=0x7fa554008040 comp rx=0 tx=0).stop 2024-09-16T00:09:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:48.231+0000 7fa5697ed640 1 -- 172.21.15.139:0/2715215125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa56415fba0 msgr2=0x7fa5641609f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:48.231+0000 7fa5697ed640 1 --2- 172.21.15.139:0/2715215125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa56415fba0 0x7fa5641609f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00be30 tx=0x7fa55c00bf20 comp rx=0 tx=0).stop 2024-09-16T00:09:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:48.231+0000 7fa5697ed640 1 -- 172.21.15.139:0/2715215125 shutdown_connections 2024-09-16T00:09:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:48.231+0000 7fa5697ed640 1 --2- 172.21.15.139:0/2715215125 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa55006c2b0 0x7fa55006e770 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:48.231+0000 7fa5697ed640 1 --2- 172.21.15.139:0/2715215125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa56415fba0 0x7fa5641609f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:48.231+0000 7fa5697ed640 1 --2- 172.21.15.139:0/2715215125 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa56415f7c0 0x7fa5641f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:48.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:48.231+0000 7fa5697ed640 1 --2- 172.21.15.139:0/2715215125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa56406e8f0 0x7fa564158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:48.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:48.231+0000 7fa5697ed640 1 -- 172.21.15.139:0/2715215125 >> 172.21.15.139:0/2715215125 conn(0x7fa56406bd00 msgr2=0x7fa5640ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:48.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:48.231+0000 7fa5697ed640 1 -- 172.21.15.139:0/2715215125 shutdown_connections 2024-09-16T00:09:48.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:48.231+0000 7fa5697ed640 1 -- 172.21.15.139:0/2715215125 wait complete. 2024-09-16T00:09:48.374 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:09:48.375 INFO:tasks.workunit.client.0.smithi139.stdout:3/561: dwrite d5/d7/f78 [0,4194304] 0 2024-09-16T00:09:48.375 INFO:tasks.workunit.client.0.smithi139.stdout:9/735: dwrite d19/d1a/d20/d23/dca/d71/d32/d79/f91 [0,4194304] 0 2024-09-16T00:09:48.375 INFO:tasks.workunit.client.0.smithi139.stdout:9/736: readlink d19/d1a/d20/d23/dca/d71/d32/d4e/l4f 0 2024-09-16T00:09:48.376 INFO:tasks.workunit.client.0.smithi139.stdout:3/562: mknod d5/d7/d23/d27/d2f/d30/d41/d68/cc7 0 2024-09-16T00:09:48.376 INFO:tasks.workunit.client.0.smithi139.stdout:3/563: rmdir d5/d7/d23/d27/d2f/d30/d41/d68/dbd 39 2024-09-16T00:09:48.376 INFO:tasks.workunit.client.0.smithi139.stdout:3/564: write d5/f16 [5444689,51824] 0 2024-09-16T00:09:48.376 INFO:tasks.workunit.client.0.smithi139.stdout:9/737: getdents d19/d1a/d20/d23/d2d/d35/d42/d3a/dcb 0 2024-09-16T00:09:48.376 INFO:tasks.workunit.client.0.smithi139.stdout:3/565: symlink d5/d5f/lc8 0 2024-09-16T00:09:48.381 INFO:tasks.workunit.client.0.smithi139.stdout:3/566: rename d5/d7/d23/d27/d2f/d30/d47/f8b to d5/d7/d23/fc9 0 2024-09-16T00:09:48.381 INFO:tasks.workunit.client.0.smithi139.stdout:9/738: dwrite d19/d1a/d20/d23/dca/d71/d32/d4e/f78 [0,4194304] 0 2024-09-16T00:09:48.381 INFO:tasks.workunit.client.0.smithi139.stdout:0/589: sync 2024-09-16T00:09:48.381 INFO:tasks.workunit.client.0.smithi139.stdout:1/530: sync 2024-09-16T00:09:48.381 INFO:tasks.workunit.client.0.smithi139.stdout:9/739: chown d19/d1a/d20/d23/dca/d71/dc7 16 1 2024-09-16T00:09:48.381 INFO:tasks.workunit.client.0.smithi139.stdout:9/740: symlink d19/d1a/d20/d23/dca/d71/da3/lfa 0 2024-09-16T00:09:48.381 INFO:tasks.workunit.client.0.smithi139.stdout:1/531: link d7/d8/d27/d2f/f73 d7/f9b 0 2024-09-16T00:09:48.381 INFO:tasks.workunit.client.0.smithi139.stdout:3/567: truncate d5/d7/d23/d27/d2f/d30/f80 2178057 0 2024-09-16T00:09:48.381 INFO:tasks.workunit.client.0.smithi139.stdout:1/532: symlink d7/d8/d40/d48/l9c 0 2024-09-16T00:09:48.381 INFO:tasks.workunit.client.0.smithi139.stdout:3/568: stat d5/d7/d23/cae 0 2024-09-16T00:09:48.382 INFO:tasks.workunit.client.0.smithi139.stdout:9/741: dwrite d19/d1a/d20/d23/d2d/d35/d42/fea [0,4194304] 0 2024-09-16T00:09:48.382 INFO:tasks.workunit.client.0.smithi139.stdout:1/533: symlink d7/d8/d27/d68/l9d 0 2024-09-16T00:09:48.382 INFO:tasks.workunit.client.0.smithi139.stdout:3/569: mkdir d5/d7/d23/d27/d2f/d30/d4c/d50/dca 0 2024-09-16T00:09:48.382 INFO:tasks.workunit.client.0.smithi139.stdout:9/742: chown d19/d1a/d20/d23/dca/d71/d32/d4e/d67/lcd 57087 1 2024-09-16T00:09:48.382 INFO:tasks.workunit.client.0.smithi139.stdout:3/570: dread - d5/d7/d23/fc9 zero size 2024-09-16T00:09:48.382 INFO:tasks.workunit.client.0.smithi139.stdout:1/534: fsync d7/f9b 0 2024-09-16T00:09:48.382 INFO:tasks.workunit.client.0.smithi139.stdout:1/535: chown d7/d8/d27/d2f 13099 1 2024-09-16T00:09:48.382 INFO:tasks.workunit.client.0.smithi139.stdout:3/571: creat d5/d7/d23/d35/fcb x:0 0 0 2024-09-16T00:09:48.382 INFO:tasks.workunit.client.0.smithi139.stdout:3/572: rmdir d5/d7/d23/d27/d2f/d30/d41/d68/dbd 39 2024-09-16T00:09:48.382 INFO:tasks.workunit.client.0.smithi139.stdout:9/743: dwrite d19/f6c [0,4194304] 0 2024-09-16T00:09:48.382 INFO:tasks.workunit.client.0.smithi139.stdout:3/573: link d5/d7/d23/d27/d2f/d30/d4c/d50/f87 d5/d7/d23/d27/d2f/d30/d4c/fcc 0 2024-09-16T00:09:48.382 INFO:tasks.workunit.client.0.smithi139.stdout:3/574: readlink d5/d7/d23/d35/da7/l3f 0 2024-09-16T00:09:48.382 INFO:tasks.workunit.client.0.smithi139.stdout:9/744: creat d19/d1a/d20/d23/dca/db5/df6/ffb x:0 0 0 2024-09-16T00:09:48.383 INFO:tasks.workunit.client.0.smithi139.stdout:3/575: dread - d5/d7/d23/d27/d2f/d30/d4c/d6d/f9e zero size 2024-09-16T00:09:48.383 INFO:tasks.workunit.client.0.smithi139.stdout:1/536: dread d7/d8/f6f [0,4194304] 0 2024-09-16T00:09:48.383 INFO:tasks.workunit.client.0.smithi139.stdout:9/745: link d19/d1a/d20/d23/d2d/d35/d42/fea d19/d86/ffc 0 2024-09-16T00:09:48.383 INFO:tasks.workunit.client.0.smithi139.stdout:1/537: unlink d7/d8/l44 0 2024-09-16T00:09:48.383 INFO:tasks.workunit.client.0.smithi139.stdout:9/746: mkdir d19/d1a/d20/d89/d93/dfd 0 2024-09-16T00:09:48.383 INFO:tasks.workunit.client.0.smithi139.stdout:3/576: rename d5/d7/d23/f77 to d5/d7/d23/d27/d2f/d30/d4c/d50/fcd 0 2024-09-16T00:09:48.383 INFO:tasks.workunit.client.0.smithi139.stdout:9/747: dwrite d19/d1a/d20/d23/dca/db5/df6/ffb [0,4194304] 0 2024-09-16T00:09:48.383 INFO:tasks.workunit.client.0.smithi139.stdout:1/538: truncate d7/d8/d40/d48/d4a/d74/f7e 186970 0 2024-09-16T00:09:48.383 INFO:tasks.workunit.client.0.smithi139.stdout:3/577: dwrite d5/d7/d23/d27/d2f/d30/d4d/f60 [0,4194304] 0 2024-09-16T00:09:48.384 INFO:tasks.workunit.client.0.smithi139.stdout:9/748: read d19/d1a/d20/d23/d2d/d9a/fac [2440776,22841] 0 2024-09-16T00:09:48.384 INFO:tasks.workunit.client.0.smithi139.stdout:3/578: creat d5/d7/d23/d27/d2f/d30/d4d/dbc/fce x:0 0 0 2024-09-16T00:09:48.384 INFO:tasks.workunit.client.0.smithi139.stdout:3/579: readlink d5/d7/d23/d35/da7/l51 0 2024-09-16T00:09:48.384 INFO:tasks.workunit.client.0.smithi139.stdout:9/749: fdatasync d19/d1a/d20/d23/dca/d71/d32/d4e/fb6 0 2024-09-16T00:09:48.384 INFO:tasks.workunit.client.0.smithi139.stdout:1/539: truncate d7/f26 2003307 0 2024-09-16T00:09:48.384 INFO:tasks.workunit.client.0.smithi139.stdout:3/580: rename d5/d7/d23/d27/d2f/d30 to d5/d7/d23/d27/d2f/d30/d41/dcf 22 2024-09-16T00:09:48.384 INFO:tasks.workunit.client.0.smithi139.stdout:9/750: creat d19/d1a/d20/d23/ffe x:0 0 0 2024-09-16T00:09:48.384 INFO:tasks.workunit.client.0.smithi139.stdout:1/540: write d7/d8/d1f/f38 [4211228,47168] 0 2024-09-16T00:09:48.384 INFO:tasks.workunit.client.0.smithi139.stdout:9/751: unlink d19/d1a/d20/d23/cf2 0 2024-09-16T00:09:48.384 INFO:tasks.workunit.client.0.smithi139.stdout:9/752: chown d19/d1a/f1e 45834 1 2024-09-16T00:09:48.384 INFO:tasks.workunit.client.0.smithi139.stdout:1/541: getdents d7/d8/d40 0 2024-09-16T00:09:48.384 INFO:tasks.workunit.client.0.smithi139.stdout:9/753: creat d19/d1a/d20/dbe/fff x:0 0 0 2024-09-16T00:09:48.384 INFO:tasks.workunit.client.0.smithi139.stdout:9/754: symlink d19/d1a/d20/d89/dbd/l100 0 2024-09-16T00:09:48.385 INFO:tasks.workunit.client.0.smithi139.stdout:9/755: link d19/d1a/d20/d23/dca/d71/d32/d79/cda d19/d1a/d20/d23/dca/d9f/dd9/c101 0 2024-09-16T00:09:48.385 INFO:tasks.workunit.client.0.smithi139.stdout:9/756: write d19/d1a/f1e [2362223,4906] 0 2024-09-16T00:09:48.385 INFO:tasks.workunit.client.0.smithi139.stdout:1/542: dread d7/d8/f11 [0,4194304] 0 2024-09-16T00:09:48.385 INFO:tasks.workunit.client.0.smithi139.stdout:9/757: mknod d19/d1a/d20/d23/d2d/d35/d44/df4/c102 0 2024-09-16T00:09:48.385 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:48.371+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:48.386 INFO:tasks.workunit.client.0.smithi139.stdout:1/543: creat d7/d8/d40/f9e x:0 0 0 2024-09-16T00:09:48.386 INFO:tasks.workunit.client.0.smithi139.stdout:9/758: getdents d19 0 2024-09-16T00:09:48.386 INFO:tasks.workunit.client.0.smithi139.stdout:9/759: truncate d19/d1a/d20/d23/ffe 32682 0 2024-09-16T00:09:48.386 INFO:tasks.workunit.client.0.smithi139.stdout:1/544: dwrite d7/d8/d27/f59 [0,4194304] 0 2024-09-16T00:09:48.386 INFO:tasks.workunit.client.0.smithi139.stdout:9/760: unlink d19/d1a/d20/d23/d2d/d35/d42/l81 0 2024-09-16T00:09:48.391 INFO:tasks.workunit.client.0.smithi139.stdout:1/545: dread d7/f13 [0,4194304] 0 2024-09-16T00:09:48.391 INFO:tasks.workunit.client.0.smithi139.stdout:9/761: dread d19/d1a/d20/d23/d2d/d9a/fac [0,4194304] 0 2024-09-16T00:09:48.394 INFO:tasks.workunit.client.0.smithi139.stdout:9/762: mkdir d19/d1a/d20/d23/d2d/d35/d44/df4/d103 0 2024-09-16T00:09:48.398 INFO:tasks.workunit.client.0.smithi139.stdout:1/546: fsync d7/f26 0 2024-09-16T00:09:48.400 INFO:tasks.workunit.client.0.smithi139.stdout:9/763: dread d19/d1a/d20/d23/d2d/d2f/f96 [4194304,4194304] 0 2024-09-16T00:09:48.401 INFO:tasks.workunit.client.0.smithi139.stdout:9/764: write d19/d1a/d20/d23/d2d/d35/f66 [1638030,8823] 0 2024-09-16T00:09:48.404 INFO:tasks.workunit.client.0.smithi139.stdout:1/547: dread d7/f9b [0,4194304] 0 2024-09-16T00:09:48.405 INFO:tasks.workunit.client.0.smithi139.stdout:9/765: unlink d19/d1a/d20/d23/c82 0 2024-09-16T00:09:48.409 INFO:tasks.workunit.client.0.smithi139.stdout:1/548: creat d7/d8/d40/d48/d4a/d74/f9f x:0 0 0 2024-09-16T00:09:48.412 INFO:tasks.workunit.client.0.smithi139.stdout:1/549: unlink d7/d8/d40/f9e 0 2024-09-16T00:09:48.415 INFO:tasks.workunit.client.0.smithi139.stdout:1/550: creat d7/d8/fa0 x:0 0 0 2024-09-16T00:09:48.417 INFO:tasks.workunit.client.0.smithi139.stdout:9/766: dread d19/f38 [0,4194304] 0 2024-09-16T00:09:48.421 INFO:tasks.workunit.client.0.smithi139.stdout:9/767: link d19/c8b d19/d1a/d20/d23/dca/d71/d32/d79/c104 0 2024-09-16T00:09:48.421 INFO:tasks.workunit.client.0.smithi139.stdout:9/768: chown d19/d1a/d20/d89/d93/dfd 145834 1 2024-09-16T00:09:48.424 INFO:tasks.workunit.client.0.smithi139.stdout:9/769: symlink d19/d1a/d20/d89/d93/dfd/l105 0 2024-09-16T00:09:48.428 INFO:tasks.workunit.client.0.smithi139.stdout:9/770: rename d19/d1a/d20/d23/dca/d71/d32/d4e/d67 to d19/d1a/d20/d23/d2d/d106 0 2024-09-16T00:09:48.434 INFO:tasks.workunit.client.0.smithi139.stdout:9/771: dwrite d19/d1a/d20/d89/d93/fbb [0,4194304] 0 2024-09-16T00:09:48.435 INFO:tasks.workunit.client.0.smithi139.stdout:9/772: readlink d19/d1a/d20/d23/d2d/d2f/lc2 0 2024-09-16T00:09:48.436 INFO:tasks.workunit.client.0.smithi139.stdout:9/773: chown d19/d1a/d20/d23/d2d/d35/d42/d3a/fe6 968868 1 2024-09-16T00:09:48.438 INFO:tasks.workunit.client.0.smithi139.stdout:9/774: rmdir d19/d1a/d20/d89/d93 39 2024-09-16T00:09:48.442 INFO:tasks.workunit.client.0.smithi139.stdout:9/775: creat d19/d1a/d20/d23/d2d/d35/d42/de7/f107 x:0 0 0 2024-09-16T00:09:48.447 INFO:tasks.workunit.client.0.smithi139.stdout:9/776: truncate d19/d1a/d20/d23/d2d/d2f/f96 8304922 0 2024-09-16T00:09:48.449 INFO:tasks.workunit.client.0.smithi139.stdout:9/777: symlink d19/d1a/l108 0 2024-09-16T00:09:48.449 INFO:tasks.workunit.client.0.smithi139.stdout:9/778: write d19/d1a/d20/d23/d2d/f92 [1166288,38421] 0 2024-09-16T00:09:48.452 INFO:tasks.workunit.client.0.smithi139.stdout:9/779: creat d19/d1a/d20/d23/d2d/d106/f109 x:0 0 0 2024-09-16T00:09:48.453 INFO:tasks.workunit.client.0.smithi139.stdout:9/780: dread - d19/d1a/d20/d23/d2d/d35/d42/de7/f107 zero size 2024-09-16T00:09:48.455 INFO:tasks.workunit.client.0.smithi139.stdout:9/781: creat d19/d1a/d20/d23/d2d/d35/f10a x:0 0 0 2024-09-16T00:09:48.466 INFO:tasks.workunit.client.0.smithi139.stdout:9/782: dwrite d19/d1a/d20/d23/d2d/d2f/f96 [4194304,4194304] 0 2024-09-16T00:09:48.469 INFO:tasks.workunit.client.0.smithi139.stdout:9/783: creat d19/d1a/d20/d23/d2d/d106/f10b x:0 0 0 2024-09-16T00:09:48.469 INFO:tasks.workunit.client.0.smithi139.stdout:9/784: read d19/d86/ffc [1776642,124202] 0 2024-09-16T00:09:48.471 INFO:tasks.workunit.client.0.smithi139.stdout:9/785: read - d19/d1a/d20/d23/dca/d9f/dd9/fe8 zero size 2024-09-16T00:09:48.472 INFO:tasks.workunit.client.0.smithi139.stdout:9/786: read d19/d1a/f50 [2746446,2340] 0 2024-09-16T00:09:48.473 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:48.471+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:48.478 INFO:tasks.workunit.client.0.smithi139.stdout:9/787: dwrite d19/d1a/d20/d23/d2d/d35/d42/d3a/f6e [0,4194304] 0 2024-09-16T00:09:48.485 INFO:tasks.workunit.client.0.smithi139.stdout:9/788: write d19/d1a/d20/d23/d2d/d35/d42/d3a/f48 [2104389,60753] 0 2024-09-16T00:09:48.487 INFO:tasks.workunit.client.0.smithi139.stdout:9/789: mknod d19/d1a/d20/d23/dca/d71/d32/c10c 0 2024-09-16T00:09:48.489 INFO:tasks.workunit.client.0.smithi139.stdout:9/790: mkdir d19/d1a/d20/d23/dca/db5/d10d 0 2024-09-16T00:09:48.493 INFO:tasks.workunit.client.0.smithi139.stdout:9/791: truncate d19/f26 8846121 0 2024-09-16T00:09:48.493 INFO:tasks.workunit.client.0.smithi139.stdout:9/792: write d19/d1a/d20/d23/d2d/d35/f66 [2416369,74929] 0 2024-09-16T00:09:48.493 INFO:tasks.workunit.client.0.smithi139.stdout:9/793: stat d19/d1a/d20/d23/d2d/fa4 0 2024-09-16T00:09:48.496 INFO:tasks.workunit.client.0.smithi139.stdout:9/794: truncate d19/f6c 3959954 0 2024-09-16T00:09:48.500 INFO:tasks.workunit.client.0.smithi139.stdout:9/795: truncate fb 2223915 0 2024-09-16T00:09:48.504 INFO:tasks.workunit.client.0.smithi139.stdout:9/796: link d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/c8e d19/d1a/d20/d23/dca/d71/dc7/c10e 0 2024-09-16T00:09:48.505 INFO:tasks.workunit.client.0.smithi139.stdout:9/797: fdatasync d19/d1a/d20/d23/d2d/f92 0 2024-09-16T00:09:48.506 INFO:tasks.workunit.client.0.smithi139.stdout:9/798: chown d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/cd7 2452 1 2024-09-16T00:09:48.507 INFO:tasks.workunit.client.0.smithi139.stdout:9/799: chown d19/d1a/d20/d23/d2d 0 1 2024-09-16T00:09:48.510 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:48.510 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:48.510 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.510 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2328 3.3 e89/20 3:c1fbdc70:::1000000310a.00000001:head v 89'183, mlcod=88'181)", 2024-09-16T00:09:48.510 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.510169+0000", 2024-09-16T00:09:48.511 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.000278638, 2024-09-16T00:09:48.511 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00029338999999999998, 2024-09-16T00:09:48.511 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.511 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.511 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:09:48.511 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.511 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.511 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.511 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.510169+0000", 2024-09-16T00:09:48.511 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.511 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.511 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.512 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.510169+0000", 2024-09-16T00:09:48.512 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.512 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.512 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.512 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.512 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.510169+0000", 2024-09-16T00:09:48.512 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7199999999999998e-07 2024-09-16T00:09:48.512 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.512 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.512 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.512 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.510175+0000", 2024-09-16T00:09:48.512 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3879999999999999e-06 2024-09-16T00:09:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.510183+0000", 2024-09-16T00:09:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3040000000000005e-06 2024-09-16T00:09:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.510188+0000", 2024-09-16T00:09:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9740000000000001e-06 2024-09-16T00:09:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.513 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.514 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.510233+0000", 2024-09-16T00:09:48.514 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5340000000000003e-05 2024-09-16T00:09:48.514 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.514 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.514 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.514 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.510253+0000", 2024-09-16T00:09:48.514 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9655000000000001e-05 2024-09-16T00:09:48.514 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.514 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.514 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.514 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.514 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:48.515 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 1 2024-09-16T00:09:48.515 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:48.515 INFO:tasks.workunit.client.0.smithi139.stdout:9/800: dread d19/f6c [0,4194304] 0 2024-09-16T00:09:48.515 INFO:tasks.workunit.client.0.smithi139.stdout:9/801: truncate d19/f26 9068937 0 2024-09-16T00:09:48.520 INFO:tasks.workunit.client.0.smithi139.stdout:9/802: dwrite d19/d1a/d20/d23/d2d/d35/f61 [0,4194304] 0 2024-09-16T00:09:48.522 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:09:48.523 INFO:tasks.workunit.client.0.smithi139.stdout:9/803: creat d19/d1a/d20/d23/dca/d71/d32/f10f x:0 0 0 2024-09-16T00:09:48.525 INFO:tasks.workunit.client.0.smithi139.stdout:9/804: mkdir d19/d1a/d20/d23/d2d/d35/d42/d3a/df3/d110 0 2024-09-16T00:09:48.529 INFO:tasks.workunit.client.0.smithi139.stdout:9/805: rename d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/fa5 to d19/d1a/d20/d23/dca/d71/da3/f111 0 2024-09-16T00:09:48.530 INFO:tasks.workunit.client.0.smithi139.stdout:9/806: chown d19/d1a/d20/d23/dca/d71/d32/d4e 4141933 1 2024-09-16T00:09:48.538 INFO:tasks.workunit.client.0.smithi139.stdout:9/807: fsync d19/d1a/d20/d23/d2d/d9a/fac 0 2024-09-16T00:09:48.541 INFO:tasks.workunit.client.0.smithi139.stdout:9/808: rmdir d19/d1a/d20/d23/d2d/d35/d44/df4/d103 0 2024-09-16T00:09:48.562 INFO:tasks.workunit.client.0.smithi139.stdout:5/741: sync 2024-09-16T00:09:48.562 INFO:tasks.workunit.client.0.smithi139.stdout:6/466: sync 2024-09-16T00:09:48.562 INFO:tasks.workunit.client.0.smithi139.stdout:5/742: dread - d1/d13/d22/da9/f101 zero size 2024-09-16T00:09:48.564 INFO:tasks.workunit.client.0.smithi139.stdout:6/467: read d8/d20/f2b [2510683,107183] 0 2024-09-16T00:09:48.566 INFO:tasks.workunit.client.0.smithi139.stdout:6/468: rename d8/c14 to d8/d29/d83/d26/d45/d59/ca1 0 2024-09-16T00:09:48.567 INFO:tasks.workunit.client.0.smithi139.stdout:6/469: fsync d8/d1f/f5c 0 2024-09-16T00:09:48.567 INFO:tasks.workunit.client.0.smithi139.stdout:6/470: readlink d8/d29/d83/d26/d28/d6d/la0 0 2024-09-16T00:09:48.572 INFO:tasks.workunit.client.0.smithi139.stdout:6/471: dwrite d8/d29/d83/d26/f66 [0,4194304] 0 2024-09-16T00:09:48.574 INFO:tasks.workunit.client.0.smithi139.stdout:6/472: write d8/d1f/f5c [338121,67633] 0 2024-09-16T00:09:48.574 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:48.573+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:48.576 INFO:tasks.workunit.client.0.smithi139.stdout:6/473: symlink d8/d29/d83/d26/d28/la2 0 2024-09-16T00:09:48.577 INFO:tasks.workunit.client.0.smithi139.stdout:6/474: stat d8/d20/d27/c4e 0 2024-09-16T00:09:48.579 INFO:tasks.workunit.client.0.smithi139.stdout:6/475: mknod d8/d29/d83/d26/d45/d85/ca3 0 2024-09-16T00:09:48.581 INFO:tasks.workunit.client.0.smithi139.stdout:6/476: mkdir d8/da4 0 2024-09-16T00:09:48.584 INFO:tasks.workunit.client.0.smithi139.stdout:6/477: truncate d8/d29/d83/f98 3101305 0 2024-09-16T00:09:48.589 INFO:tasks.workunit.client.0.smithi139.stdout:6/478: dwrite d8/d29/d83/d26/d45/d65/f2e [0,4194304] 0 2024-09-16T00:09:48.596 INFO:tasks.workunit.client.0.smithi139.stdout:6/479: read d8/d29/d83/f98 [638101,124636] 0 2024-09-16T00:09:48.598 INFO:tasks.workunit.client.0.smithi139.stdout:6/480: write d8/d29/d83/d26/d28/f3f [4465252,103592] 0 2024-09-16T00:09:48.602 INFO:tasks.workunit.client.0.smithi139.stdout:6/481: link d8/d29/d83/d26/d45/f43 d8/d29/d83/d26/fa5 0 2024-09-16T00:09:48.607 INFO:tasks.workunit.client.0.smithi139.stdout:6/482: write d8/d29/d83/d26/d45/d65/f13 [23363,44078] 0 2024-09-16T00:09:48.609 INFO:tasks.workunit.client.0.smithi139.stdout:6/483: mknod d8/d29/d83/ca6 0 2024-09-16T00:09:48.610 INFO:tasks.workunit.client.0.smithi139.stdout:6/484: chown d8/d29/d83/d26/d45/d59/c87 426258 1 2024-09-16T00:09:48.612 INFO:tasks.workunit.client.0.smithi139.stdout:6/485: rename d8/d29/f2d to d8/d29/d83/d26/d28/d5a/fa7 0 2024-09-16T00:09:48.613 INFO:tasks.workunit.client.0.smithi139.stdout:6/486: readlink d8/d20/l4c 0 2024-09-16T00:09:48.615 INFO:tasks.workunit.client.0.smithi139.stdout:6/487: mkdir d8/d29/d83/d26/d28/d9e/da8 0 2024-09-16T00:09:48.615 INFO:tasks.workunit.client.0.smithi139.stdout:6/488: readlink d8/d1f/l94 0 2024-09-16T00:09:48.616 INFO:tasks.workunit.client.0.smithi139.stdout:6/489: chown d8/d29/d83/f98 3780 1 2024-09-16T00:09:48.621 INFO:tasks.workunit.client.0.smithi139.stdout:6/490: dwrite d8/d29/d83/d26/d28/d6d/f89 [0,4194304] 0 2024-09-16T00:09:48.626 INFO:tasks.workunit.client.0.smithi139.stdout:6/491: truncate d8/d29/d83/f81 1580502 0 2024-09-16T00:09:48.628 INFO:tasks.workunit.client.0.smithi139.stdout:6/492: readlink d8/d29/d83/l78 0 2024-09-16T00:09:48.631 INFO:tasks.workunit.client.0.smithi139.stdout:6/493: truncate d8/d29/d83/d26/d45/f43 1593123 0 2024-09-16T00:09:48.632 INFO:tasks.workunit.client.0.smithi139.stdout:2/806: sync 2024-09-16T00:09:48.634 INFO:tasks.workunit.client.0.smithi139.stdout:6/494: rename d8/d29/d83/d26/d28/l82 to d8/d29/d83/d26/d28/d6d/la9 0 2024-09-16T00:09:48.635 INFO:tasks.workunit.client.0.smithi139.stdout:6/495: chown d8/d29/d83/d26/d28/d74 178 1 2024-09-16T00:09:48.645 INFO:tasks.workunit.client.0.smithi139.stdout:6/496: dwrite d8/d29/f49 [0,4194304] 0 2024-09-16T00:09:48.646 INFO:tasks.workunit.client.0.smithi139.stdout:2/807: dread d11/d30/f44 [4194304,4194304] 0 2024-09-16T00:09:48.647 INFO:tasks.workunit.client.0.smithi139.stdout:6/497: fsync d8/d29/d83/d26/d45/d42/f76 0 2024-09-16T00:09:48.651 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:48.651 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:48.651 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:48.651 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:48.651 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:48.662 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:09:48.674 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:48.673+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:48.713 INFO:tasks.workunit.client.0.smithi139.stdout:6/498: unlink d8/d29/d83/d26/d28/d6d/f9b 0 2024-09-16T00:09:48.719 INFO:tasks.workunit.client.0.smithi139.stdout:2/808: dwrite d11/d30/d110/d29/f66 [4194304,4194304] 0 2024-09-16T00:09:48.720 INFO:tasks.workunit.client.0.smithi139.stdout:6/499: link d8/d29/d83/d26/c88 d8/d1f/caa 0 2024-09-16T00:09:48.726 INFO:tasks.workunit.client.0.smithi139.stdout:2/809: rename d11/d30/f36 to d11/d30/dbf/f119 0 2024-09-16T00:09:48.726 INFO:tasks.workunit.client.0.smithi139.stdout:6/500: link d8/d29/d83/d26/d28/c2f d8/d29/d83/d26/d28/d74/cab 0 2024-09-16T00:09:48.728 INFO:tasks.workunit.client.0.smithi139.stdout:2/810: unlink d11/d30/dbf/ff2 0 2024-09-16T00:09:48.730 INFO:tasks.workunit.client.0.smithi139.stdout:6/501: symlink d8/d29/d83/d26/d45/d65/lac 0 2024-09-16T00:09:48.730 INFO:tasks.workunit.client.0.smithi139.stdout:2/811: read d11/d30/d110/d5d/d73/f8e [2468298,45967] 0 2024-09-16T00:09:48.731 INFO:tasks.workunit.client.0.smithi139.stdout:2/812: readlink d11/d30/d110/le3 0 2024-09-16T00:09:48.733 INFO:tasks.workunit.client.0.smithi139.stdout:2/813: write d11/f38 [1583876,79558] 0 2024-09-16T00:09:48.734 INFO:tasks.workunit.client.0.smithi139.stdout:2/814: fdatasync d11/d2f/d6e/f84 0 2024-09-16T00:09:48.735 INFO:tasks.workunit.client.0.smithi139.stdout:6/502: creat d8/d29/d83/d26/d45/d85/fad x:0 0 0 2024-09-16T00:09:48.737 INFO:tasks.workunit.client.0.smithi139.stdout:6/503: creat d8/d29/d83/d26/d45/d85/fae x:0 0 0 2024-09-16T00:09:48.737 INFO:tasks.workunit.client.0.smithi139.stdout:2/815: mknod d11/d30/dbf/d116/c11a 0 2024-09-16T00:09:48.739 INFO:tasks.workunit.client.0.smithi139.stdout:6/504: unlink d8/d29/d83/f98 0 2024-09-16T00:09:48.739 INFO:tasks.workunit.client.0.smithi139.stdout:2/816: mknod d11/d2f/deb/c11b 0 2024-09-16T00:09:48.740 INFO:tasks.workunit.client.0.smithi139.stdout:2/817: dread - d11/d32/f112 zero size 2024-09-16T00:09:48.740 INFO:tasks.workunit.client.0.smithi139.stdout:6/505: rmdir d8/d29/d83 39 2024-09-16T00:09:48.741 INFO:tasks.workunit.client.0.smithi139.stdout:2/818: rename d11/d30/d110/d50 to d11/d30/d110/d50/d114/ddb/d11c 22 2024-09-16T00:09:48.742 INFO:tasks.workunit.client.0.smithi139.stdout:2/819: dread - d11/d30/d110/d29/d4f/f7a zero size 2024-09-16T00:09:48.742 INFO:tasks.workunit.client.0.smithi139.stdout:6/506: read d8/d29/d83/d26/d45/f46 [102129,38844] 0 2024-09-16T00:09:48.746 INFO:tasks.workunit.client.0.smithi139.stdout:2/820: creat d11/d30/d110/d50/dcb/f11d x:0 0 0 2024-09-16T00:09:48.755 INFO:tasks.workunit.client.0.smithi139.stdout:6/507: dwrite d8/d29/d83/f81 [0,4194304] 0 2024-09-16T00:09:48.757 INFO:tasks.workunit.client.0.smithi139.stdout:6/508: write d8/d29/f36 [422101,49903] 0 2024-09-16T00:09:48.758 INFO:tasks.workunit.client.0.smithi139.stdout:6/509: stat d8/d29/d83/d26/d45/d42/f76 0 2024-09-16T00:09:48.760 INFO:tasks.workunit.client.0.smithi139.stdout:2/821: dread d11/d30/d110/d29/d2b/f2c [0,4194304] 0 2024-09-16T00:09:48.760 INFO:tasks.workunit.client.0.smithi139.stdout:6/510: symlink d8/d20/laf 0 2024-09-16T00:09:48.762 INFO:tasks.workunit.client.0.smithi139.stdout:6/511: mkdir d8/d20/d4d/db0 0 2024-09-16T00:09:48.764 INFO:tasks.workunit.client.0.smithi139.stdout:6/512: symlink d8/d29/d83/d26/d28/d9e/lb1 0 2024-09-16T00:09:48.765 INFO:tasks.workunit.client.0.smithi139.stdout:6/513: truncate d8/d29/d83/d26/f91 749781 0 2024-09-16T00:09:48.769 INFO:tasks.workunit.client.0.smithi139.stdout:2/822: dread d11/f1a [0,4194304] 0 2024-09-16T00:09:48.772 INFO:tasks.workunit.client.0.smithi139.stdout:7/709: sync 2024-09-16T00:09:48.774 INFO:tasks.workunit.client.0.smithi139.stdout:7/710: write d5/d26/d46/d6b/da7/fe2 [979917,75151] 0 2024-09-16T00:09:48.775 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:48.774+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:48.775 INFO:tasks.workunit.client.0.smithi139.stdout:7/711: fsync d5/d16/d6d/fd0 0 2024-09-16T00:09:48.776 INFO:tasks.workunit.client.0.smithi139.stdout:2/823: dread d11/d30/f42 [0,4194304] 0 2024-09-16T00:09:48.778 INFO:tasks.workunit.client.0.smithi139.stdout:2/824: unlink d11/l14 0 2024-09-16T00:09:48.782 INFO:tasks.workunit.client.0.smithi139.stdout:7/712: dwrite d5/fbc [0,4194304] 0 2024-09-16T00:09:48.784 INFO:tasks.workunit.client.0.smithi139.stdout:2/825: dwrite d11/f16 [0,4194304] 0 2024-09-16T00:09:48.788 INFO:tasks.workunit.client.0.smithi139.stdout:7/713: creat d5/d16/d6d/d7f/dc2/f101 x:0 0 0 2024-09-16T00:09:48.789 INFO:tasks.workunit.client.0.smithi139.stdout:2/826: creat d11/d30/d110/d5d/d73/dc3/de8/f11e x:0 0 0 2024-09-16T00:09:48.791 INFO:tasks.workunit.client.0.smithi139.stdout:7/714: mknod d5/d16/d6d/d7f/dc2/d50/d44/d55/d90/c102 0 2024-09-16T00:09:48.792 INFO:tasks.workunit.client.0.smithi139.stdout:7/715: write d5/f59 [30387,82935] 0 2024-09-16T00:09:48.793 INFO:tasks.workunit.client.0.smithi139.stdout:7/716: fsync d5/d16/d6d/d7f/dc2/d81/fa1 0 2024-09-16T00:09:48.795 INFO:tasks.workunit.client.0.smithi139.stdout:7/717: symlink d5/d16/d22/l103 0 2024-09-16T00:09:48.801 INFO:tasks.workunit.client.0.smithi139.stdout:7/718: write d5/d37/f79 [107324,72479] 0 2024-09-16T00:09:48.803 INFO:tasks.workunit.client.0.smithi139.stdout:7/719: creat d5/d16/d6d/d7f/dc2/d50/f104 x:0 0 0 2024-09-16T00:09:48.803 INFO:tasks.workunit.client.0.smithi139.stdout:7/720: fsync d5/d26/d46/f68 0 2024-09-16T00:09:48.806 INFO:tasks.workunit.client.0.smithi139.stdout:7/721: rename d5/d16/d22/d25/f9b to d5/d26/d46/d6b/da7/f105 0 2024-09-16T00:09:48.807 INFO:tasks.workunit.client.0.smithi139.stdout:7/722: rmdir d5/d16/d22/d25 39 2024-09-16T00:09:48.813 INFO:tasks.workunit.client.0.smithi139.stdout:7/723: dwrite d5/f21 [4194304,4194304] 0 2024-09-16T00:09:48.817 INFO:tasks.workunit.client.0.smithi139.stdout:7/724: link d5/d16/d87/cd1 d5/d16/d6d/d7f/dc2/d50/d44/d55/de0/c106 0 2024-09-16T00:09:48.818 INFO:tasks.workunit.client.0.smithi139.stdout:7/725: mknod d5/d37/d70/c107 0 2024-09-16T00:09:48.821 INFO:tasks.workunit.client.0.smithi139.stdout:7/726: read d5/d16/d22/d25/d63/fee [3835332,97068] 0 2024-09-16T00:09:48.823 INFO:tasks.workunit.client.0.smithi139.stdout:7/727: unlink d5/d26/f9a 0 2024-09-16T00:09:48.827 INFO:tasks.workunit.client.0.smithi139.stdout:7/728: rmdir d5/d16/d22/db1 0 2024-09-16T00:09:48.829 INFO:tasks.workunit.client.0.smithi139.stdout:7/729: creat d5/d16/d6d/d7f/dc2/d50/d44/d55/d90/f108 x:0 0 0 2024-09-16T00:09:48.830 INFO:tasks.workunit.client.0.smithi139.stdout:7/730: symlink d5/d16/d87/l109 0 2024-09-16T00:09:48.831 INFO:tasks.workunit.client.0.smithi139.stdout:7/731: dread - d5/d16/d87/feb zero size 2024-09-16T00:09:48.832 INFO:tasks.workunit.client.0.smithi139.stdout:7/732: symlink d5/d26/d46/dea/l10a 0 2024-09-16T00:09:48.834 INFO:tasks.workunit.client.0.smithi139.stdout:7/733: dread d5/d16/d22/d25/faf [0,4194304] 0 2024-09-16T00:09:48.835 INFO:tasks.workunit.client.0.smithi139.stdout:7/734: read - d5/d26/d46/d6b/dd6/df4/ff7 zero size 2024-09-16T00:09:48.837 INFO:tasks.workunit.client.0.smithi139.stdout:7/735: dread d5/d16/d6d/d7f/dc2/d50/d44/f57 [0,4194304] 0 2024-09-16T00:09:48.839 INFO:tasks.workunit.client.0.smithi139.stdout:7/736: mknod d5/d16/d6d/d7f/dc2/d50/d44/d55/d90/c10b 0 2024-09-16T00:09:48.844 INFO:tasks.workunit.client.0.smithi139.stdout:7/737: dwrite d5/d16/d6d/d7f/dc2/d81/fa1 [0,4194304] 0 2024-09-16T00:09:48.847 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:48.847 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:48.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:48.847 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:48.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.847 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:219 2.1f 2:f83d65d8:::200.00000004:head [write 863658~2915703 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e65)", 2024-09-16T00:09:48.847 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:48.847 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.973548371, 2024-09-16T00:09:48.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013772627000000001, 2024-09-16T00:09:48.848 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.848 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.848 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.848 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:48.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:48.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870941+0000", 2024-09-16T00:09:48.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4e-07 2024-09-16T00:09:48.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877134+0000", 2024-09-16T00:09:48.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061936769999999999 2024-09-16T00:09:48.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877153+0000", 2024-09-16T00:09:48.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8408000000000001e-05 2024-09-16T00:09:48.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877161+0000", 2024-09-16T00:09:48.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2709999999999992e-06 2024-09-16T00:09:48.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877225+0000", 2024-09-16T00:09:48.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4190999999999996e-05 2024-09-16T00:09:48.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877289+0000", 2024-09-16T00:09:48.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4058000000000002e-05 2024-09-16T00:09:48.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:48.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877376+0000", 2024-09-16T00:09:48.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7227999999999999e-05 2024-09-16T00:09:48.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.882925+0000", 2024-09-16T00:09:48.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055481050000000002 2024-09-16T00:09:48.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884635+0000", 2024-09-16T00:09:48.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017102759999999999 2024-09-16T00:09:48.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884674+0000", 2024-09-16T00:09:48.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8914e-05 2024-09-16T00:09:48.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884712+0000", 2024-09-16T00:09:48.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658999999999997e-05 2024-09-16T00:09:48.853 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.853 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.853 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.853 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10767 3.1b 3:dab1607f:::100000030fd.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:48.853 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.379273+0000", 2024-09-16T00:09:48.853 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 10.465215639, 2024-09-16T00:09:48.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012051947, 2024-09-16T00:09:48.853 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.853 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.854 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.854 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.854 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.854 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.854 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10767 2024-09-16T00:09:48.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.854 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.379273+0000", 2024-09-16T00:09:48.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.379273+0000", 2024-09-16T00:09:48.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.379275+0000", 2024-09-16T00:09:48.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6230000000000001e-06 2024-09-16T00:09:48.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381583+0000", 2024-09-16T00:09:48.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023079939999999998 2024-09-16T00:09:48.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381617+0000", 2024-09-16T00:09:48.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4227999999999998e-05 2024-09-16T00:09:48.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381633+0000", 2024-09-16T00:09:48.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5642e-05 2024-09-16T00:09:48.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381739+0000", 2024-09-16T00:09:48.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010638500000000001 2024-09-16T00:09:48.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381850+0000", 2024-09-16T00:09:48.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110164 2024-09-16T00:09:48.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:48.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.382033+0000", 2024-09-16T00:09:48.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018355699999999999 2024-09-16T00:09:48.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.388313+0000", 2024-09-16T00:09:48.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006279588 2024-09-16T00:09:48.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.391197+0000", 2024-09-16T00:09:48.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028841130000000002 2024-09-16T00:09:48.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.391263+0000", 2024-09-16T00:09:48.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6105000000000005e-05 2024-09-16T00:09:48.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.391324+0000", 2024-09-16T00:09:48.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1548000000000001e-05 2024-09-16T00:09:48.859 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.859 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.859 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.859 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10768 3.2b 3:d738455c:::100000030db.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:48.859 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:09:48.859 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 10.450363198, 2024-09-16T00:09:48.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012411195999999999, 2024-09-16T00:09:48.860 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.860 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.860 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.860 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.860 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.860 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.860 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10768 2024-09-16T00:09:48.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.860 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:09:48.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:09:48.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394129+0000", 2024-09-16T00:09:48.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4100000000000001e-06 2024-09-16T00:09:48.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395723+0000", 2024-09-16T00:09:48.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015938409999999999 2024-09-16T00:09:48.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395745+0000", 2024-09-16T00:09:48.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1611e-05 2024-09-16T00:09:48.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395782+0000", 2024-09-16T00:09:48.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7292000000000001e-05 2024-09-16T00:09:48.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395856+0000", 2024-09-16T00:09:48.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3675000000000002e-05 2024-09-16T00:09:48.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395924+0000", 2024-09-16T00:09:48.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8646999999999995e-05 2024-09-16T00:09:48.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:48.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.396069+0000", 2024-09-16T00:09:48.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014455199999999999 2024-09-16T00:09:48.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.864 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.864 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.403755+0000", 2024-09-16T00:09:48.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0076863160000000003 2024-09-16T00:09:48.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.864 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.864 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406407+0000", 2024-09-16T00:09:48.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026516220000000002 2024-09-16T00:09:48.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.864 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406477+0000", 2024-09-16T00:09:48.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0204999999999996e-05 2024-09-16T00:09:48.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406536+0000", 2024-09-16T00:09:48.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9024999999999998e-05 2024-09-16T00:09:48.865 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.865 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.865 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.865 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10772 3.1 3:823e197f:::10000003110.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:48.866 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.415400+0000", 2024-09-16T00:09:48.866 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 10.429088356999999, 2024-09-16T00:09:48.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011076012999999999, 2024-09-16T00:09:48.866 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.866 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.866 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.866 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.866 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.866 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.866 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10772 2024-09-16T00:09:48.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.866 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.415400+0000", 2024-09-16T00:09:48.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.415400+0000", 2024-09-16T00:09:48.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.415402+0000", 2024-09-16T00:09:48.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.481e-06 2024-09-16T00:09:48.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.868 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.417895+0000", 2024-09-16T00:09:48.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002492247 2024-09-16T00:09:48.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.868 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.417917+0000", 2024-09-16T00:09:48.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1977999999999999e-05 2024-09-16T00:09:48.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.417925+0000", 2024-09-16T00:09:48.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7220000000000005e-06 2024-09-16T00:09:48.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.418000+0000", 2024-09-16T00:09:48.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4715000000000005e-05 2024-09-16T00:09:48.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.418137+0000", 2024-09-16T00:09:48.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013744699999999999 2024-09-16T00:09:48.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:48.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.418295+0000", 2024-09-16T00:09:48.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000157622 2024-09-16T00:09:48.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.424462+0000", 2024-09-16T00:09:48.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061665369999999997 2024-09-16T00:09:48.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.426370+0000", 2024-09-16T00:09:48.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001908217 2024-09-16T00:09:48.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.426424+0000", 2024-09-16T00:09:48.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4693e-05 2024-09-16T00:09:48.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.426476+0000", 2024-09-16T00:09:48.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1354000000000003e-05 2024-09-16T00:09:48.872 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.872 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.872 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.872 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10909 3.1 3:8396d031:::100000031ab.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:48.872 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.183024+0000", 2024-09-16T00:09:48.872 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.6614643529999995, 2024-09-16T00:09:48.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010857537, 2024-09-16T00:09:48.872 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.872 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.872 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.872 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.872 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.873 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.873 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10909 2024-09-16T00:09:48.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.873 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.183024+0000", 2024-09-16T00:09:48.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.183024+0000", 2024-09-16T00:09:48.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.183026+0000", 2024-09-16T00:09:48.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4839999999999998e-06 2024-09-16T00:09:48.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.185491+0000", 2024-09-16T00:09:48.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024650710000000001 2024-09-16T00:09:48.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.185511+0000", 2024-09-16T00:09:48.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9519999999999999e-05 2024-09-16T00:09:48.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.185519+0000", 2024-09-16T00:09:48.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2719999999999997e-06 2024-09-16T00:09:48.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.185591+0000", 2024-09-16T00:09:48.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1571999999999998e-05 2024-09-16T00:09:48.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.185773+0000", 2024-09-16T00:09:48.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000181962 2024-09-16T00:09:48.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.876 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:48.876 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.185927+0000", 2024-09-16T00:09:48.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000153948 2024-09-16T00:09:48.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.876 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.876 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.192576+0000", 2024-09-16T00:09:48.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066497550000000002 2024-09-16T00:09:48.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.877 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.877 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.193764+0000", 2024-09-16T00:09:48.877 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011879410000000001 2024-09-16T00:09:48.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.877 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.877 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.193830+0000", 2024-09-16T00:09:48.877 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5903999999999998e-05 2024-09-16T00:09:48.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.877 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.877 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.193881+0000", 2024-09-16T00:09:48.877 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1107999999999999e-05 2024-09-16T00:09:48.878 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.878 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.878 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.878 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10941 3.1 3:8246fad2:::10000003173.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:48.878 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:09:48.878 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.3886098610000008, 2024-09-16T00:09:48.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01286782, 2024-09-16T00:09:48.878 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.878 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.878 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.878 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.878 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.879 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.879 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10941 2024-09-16T00:09:48.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.879 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:09:48.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:09:48.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455881+0000", 2024-09-16T00:09:48.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7269999999999999e-06 2024-09-16T00:09:48.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457462+0000", 2024-09-16T00:09:48.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015811690000000001 2024-09-16T00:09:48.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457487+0000", 2024-09-16T00:09:48.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4607e-05 2024-09-16T00:09:48.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457497+0000", 2024-09-16T00:09:48.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9939999999999995e-06 2024-09-16T00:09:48.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457571+0000", 2024-09-16T00:09:48.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3867000000000003e-05 2024-09-16T00:09:48.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457689+0000", 2024-09-16T00:09:48.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000118814 2024-09-16T00:09:48.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:48.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457804+0000", 2024-09-16T00:09:48.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114702 2024-09-16T00:09:48.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.463422+0000", 2024-09-16T00:09:48.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0056179239999999998 2024-09-16T00:09:48.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468641+0000", 2024-09-16T00:09:48.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052192109999999996 2024-09-16T00:09:48.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468703+0000", 2024-09-16T00:09:48.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1268999999999999e-05 2024-09-16T00:09:48.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468746+0000", 2024-09-16T00:09:48.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:09:48.884 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.884 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.884 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.884 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11085 3.2b 3:d738455c:::100000030db.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:48.884 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.975328+0000", 2024-09-16T00:09:48.884 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.8691604880000003, 2024-09-16T00:09:48.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011239193999999999, 2024-09-16T00:09:48.884 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.884 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.884 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.885 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.885 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.885 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.885 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11085 2024-09-16T00:09:48.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.885 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.975328+0000", 2024-09-16T00:09:48.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.975328+0000", 2024-09-16T00:09:48.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.975330+0000", 2024-09-16T00:09:48.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9010000000000001e-06 2024-09-16T00:09:48.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.976617+0000", 2024-09-16T00:09:48.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001287531 2024-09-16T00:09:48.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.976634+0000", 2024-09-16T00:09:48.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7153999999999999e-05 2024-09-16T00:09:48.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.976643+0000", 2024-09-16T00:09:48.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7630000000000006e-06 2024-09-16T00:09:48.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.976702+0000", 2024-09-16T00:09:48.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8662000000000002e-05 2024-09-16T00:09:48.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.976794+0000", 2024-09-16T00:09:48.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2244999999999995e-05 2024-09-16T00:09:48.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:48.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.976948+0000", 2024-09-16T00:09:48.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015442599999999999 2024-09-16T00:09:48.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.981794+0000", 2024-09-16T00:09:48.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0048452679999999998 2024-09-16T00:09:48.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.986502+0000", 2024-09-16T00:09:48.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047087609999999997 2024-09-16T00:09:48.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.986532+0000", 2024-09-16T00:09:48.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9635999999999999e-05 2024-09-16T00:09:48.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.986567+0000", 2024-09-16T00:09:48.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4847e-05 2024-09-16T00:09:48.890 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.890 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.890 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.890 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11089 3.1 3:823e197f:::10000003110.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:48.890 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.995768+0000", 2024-09-16T00:09:48.890 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.8487201409999994, 2024-09-16T00:09:48.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011209723, 2024-09-16T00:09:48.891 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.891 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.891 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.891 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.891 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.891 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.891 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11089 2024-09-16T00:09:48.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.891 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.995768+0000", 2024-09-16T00:09:48.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.892 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.995768+0000", 2024-09-16T00:09:48.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.892 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.995770+0000", 2024-09-16T00:09:48.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.725e-06 2024-09-16T00:09:48.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.997100+0000", 2024-09-16T00:09:48.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013299410000000001 2024-09-16T00:09:48.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.997122+0000", 2024-09-16T00:09:48.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2140000000000001e-05 2024-09-16T00:09:48.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.997132+0000", 2024-09-16T00:09:48.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0495e-05 2024-09-16T00:09:48.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.997209+0000", 2024-09-16T00:09:48.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6786000000000006e-05 2024-09-16T00:09:48.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.997284+0000", 2024-09-16T00:09:48.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5086999999999994e-05 2024-09-16T00:09:48.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:48.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.997399+0000", 2024-09-16T00:09:48.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114883 2024-09-16T00:09:48.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.005091+0000", 2024-09-16T00:09:48.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0076915250000000003 2024-09-16T00:09:48.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.006850+0000", 2024-09-16T00:09:48.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017591919999999999 2024-09-16T00:09:48.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.896 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.006919+0000", 2024-09-16T00:09:48.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9368000000000004e-05 2024-09-16T00:09:48.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.896 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.006978+0000", 2024-09-16T00:09:48.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8581e-05 2024-09-16T00:09:48.896 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.896 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.896 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.897 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11739 3.17 3:e9f3a865:::10000003294.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.897 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.052491+0000", 2024-09-16T00:09:48.897 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.7919972819999996, 2024-09-16T00:09:48.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011149121, 2024-09-16T00:09:48.897 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.897 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.897 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.897 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.897 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.897 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.897 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11739 2024-09-16T00:09:48.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.897 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.052491+0000", 2024-09-16T00:09:48.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.052491+0000", 2024-09-16T00:09:48.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.052494+0000", 2024-09-16T00:09:48.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0750000000000002e-06 2024-09-16T00:09:48.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.053984+0000", 2024-09-16T00:09:48.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001490395 2024-09-16T00:09:48.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.053999+0000", 2024-09-16T00:09:48.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4606e-05 2024-09-16T00:09:48.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.054007+0000", 2024-09-16T00:09:48.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4999999999999999e-06 2024-09-16T00:09:48.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.054072+0000", 2024-09-16T00:09:48.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4974999999999994e-05 2024-09-16T00:09:48.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.054230+0000", 2024-09-16T00:09:48.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015757600000000001 2024-09-16T00:09:48.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:48.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.054392+0000", 2024-09-16T00:09:48.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016178199999999999 2024-09-16T00:09:48.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.062316+0000", 2024-09-16T00:09:48.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0079242099999999992 2024-09-16T00:09:48.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.063541+0000", 2024-09-16T00:09:48.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001225114 2024-09-16T00:09:48.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.063598+0000", 2024-09-16T00:09:48.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6511000000000002e-05 2024-09-16T00:09:48.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.063640+0000", 2024-09-16T00:09:48.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2376999999999998e-05 2024-09-16T00:09:48.902 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.902 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.903 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.903 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11742 3.1b 3:d847de26:::100000032cd.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.903 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:09:48.903 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.7881182759999996, 2024-09-16T00:09:48.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012791903, 2024-09-16T00:09:48.903 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.903 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.903 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.903 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.903 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.903 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.903 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11742 2024-09-16T00:09:48.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.904 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:09:48.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:09:48.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056373+0000", 2024-09-16T00:09:48.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9799999999999998e-06 2024-09-16T00:09:48.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058604+0000", 2024-09-16T00:09:48.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022316020000000001 2024-09-16T00:09:48.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058630+0000", 2024-09-16T00:09:48.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5425000000000001e-05 2024-09-16T00:09:48.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058643+0000", 2024-09-16T00:09:48.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3361e-05 2024-09-16T00:09:48.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058731+0000", 2024-09-16T00:09:48.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7270999999999998e-05 2024-09-16T00:09:48.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058894+0000", 2024-09-16T00:09:48.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016376800000000001 2024-09-16T00:09:48.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:48.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.059050+0000", 2024-09-16T00:09:48.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015603700000000001 2024-09-16T00:09:48.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.066364+0000", 2024-09-16T00:09:48.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073138639999999998 2024-09-16T00:09:48.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069025+0000", 2024-09-16T00:09:48.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026607660000000002 2024-09-16T00:09:48.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069083+0000", 2024-09-16T00:09:48.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7680999999999999e-05 2024-09-16T00:09:48.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069162+0000", 2024-09-16T00:09:48.909 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9147999999999995e-05 2024-09-16T00:09:48.909 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.909 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.909 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.909 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.909 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11780 3.23 3:c752d055:::100000031d2.00000000:head [write 0~4194304 [2@2823004]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.909 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.200354+0000", 2024-09-16T00:09:48.909 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.6441340579999997, 2024-09-16T00:09:48.909 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011318407000000001, 2024-09-16T00:09:48.909 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.909 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.909 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.909 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.910 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.910 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.910 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11780 2024-09-16T00:09:48.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.910 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.910 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.910 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.200354+0000", 2024-09-16T00:09:48.910 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.910 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.910 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.200354+0000", 2024-09-16T00:09:48.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.200356+0000", 2024-09-16T00:09:48.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.057e-06 2024-09-16T00:09:48.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.202142+0000", 2024-09-16T00:09:48.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017862939999999999 2024-09-16T00:09:48.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.202166+0000", 2024-09-16T00:09:48.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3339000000000001e-05 2024-09-16T00:09:48.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.202176+0000", 2024-09-16T00:09:48.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0658e-05 2024-09-16T00:09:48.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.202253+0000", 2024-09-16T00:09:48.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6743999999999996e-05 2024-09-16T00:09:48.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.202363+0000", 2024-09-16T00:09:48.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010973500000000001 2024-09-16T00:09:48.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:48.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.202524+0000", 2024-09-16T00:09:48.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000160963 2024-09-16T00:09:48.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.210136+0000", 2024-09-16T00:09:48.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0076117390000000002 2024-09-16T00:09:48.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.211557+0000", 2024-09-16T00:09:48.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014213520000000001 2024-09-16T00:09:48.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.211624+0000", 2024-09-16T00:09:48.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7119999999999994e-05 2024-09-16T00:09:48.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.915 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.915 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.211673+0000", 2024-09-16T00:09:48.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8405999999999997e-05 2024-09-16T00:09:48.915 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.915 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.915 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.915 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11837 3.1b 3:d94fa8da:::10000003190.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.915 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.465632+0000", 2024-09-16T00:09:48.915 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.3788564279999997, 2024-09-16T00:09:48.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011434082, 2024-09-16T00:09:48.915 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.916 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.916 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.916 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.916 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.916 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.916 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11837 2024-09-16T00:09:48.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.916 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.465632+0000", 2024-09-16T00:09:48.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.465632+0000", 2024-09-16T00:09:48.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.465634+0000", 2024-09-16T00:09:48.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9180000000000001e-06 2024-09-16T00:09:48.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.467891+0000", 2024-09-16T00:09:48.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022575049999999999 2024-09-16T00:09:48.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.467919+0000", 2024-09-16T00:09:48.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8119999999999998e-05 2024-09-16T00:09:48.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.467933+0000", 2024-09-16T00:09:48.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3968000000000001e-05 2024-09-16T00:09:48.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.468012+0000", 2024-09-16T00:09:48.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8874000000000004e-05 2024-09-16T00:09:48.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.468174+0000", 2024-09-16T00:09:48.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000161972 2024-09-16T00:09:48.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:48.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.468319+0000", 2024-09-16T00:09:48.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000145117 2024-09-16T00:09:48.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.472853+0000", 2024-09-16T00:09:48.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045335490000000004 2024-09-16T00:09:48.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.476990+0000", 2024-09-16T00:09:48.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041376629999999998 2024-09-16T00:09:48.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.477025+0000", 2024-09-16T00:09:48.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4975999999999999e-05 2024-09-16T00:09:48.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.921 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.477066+0000", 2024-09-16T00:09:48.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0420000000000003e-05 2024-09-16T00:09:48.921 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.921 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.921 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.921 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11934 3.1b 3:d94fa8da:::10000003190.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.921 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:09:48.922 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.1560750009999996, 2024-09-16T00:09:48.922 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.017110325999999999, 2024-09-16T00:09:48.922 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.922 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.922 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.922 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.922 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.922 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.922 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11934 2024-09-16T00:09:48.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.922 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.922 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.923 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:09:48.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.923 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.923 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:09:48.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.923 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.923 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688416+0000", 2024-09-16T00:09:48.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.875e-06 2024-09-16T00:09:48.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689633+0000", 2024-09-16T00:09:48.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001216997 2024-09-16T00:09:48.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689655+0000", 2024-09-16T00:09:48.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2285000000000001e-05 2024-09-16T00:09:48.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689665+0000", 2024-09-16T00:09:48.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7089999999999997e-06 2024-09-16T00:09:48.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689739+0000", 2024-09-16T00:09:48.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3973999999999993e-05 2024-09-16T00:09:48.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689898+0000", 2024-09-16T00:09:48.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015941100000000001 2024-09-16T00:09:48.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:48.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.690062+0000", 2024-09-16T00:09:48.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000163633 2024-09-16T00:09:48.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.695224+0000", 2024-09-16T00:09:48.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051620210000000001 2024-09-16T00:09:48.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705434+0000", 2024-09-16T00:09:48.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010209701 2024-09-16T00:09:48.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705488+0000", 2024-09-16T00:09:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3974000000000002e-05 2024-09-16T00:09:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705523+0000", 2024-09-16T00:09:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5746e-05 2024-09-16T00:09:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.928 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11941 3.12 3:4a0e6682:::1000000334b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.928 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:09:48.928 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.1210574879999999, 2024-09-16T00:09:48.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01458448, 2024-09-16T00:09:48.928 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.928 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.928 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.928 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.928 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.928 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.928 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11941 2024-09-16T00:09:48.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.929 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:09:48.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:09:48.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723433+0000", 2024-09-16T00:09:48.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3410000000000001e-06 2024-09-16T00:09:48.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724966+0000", 2024-09-16T00:09:48.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001533242 2024-09-16T00:09:48.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724983+0000", 2024-09-16T00:09:48.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7156999999999999e-05 2024-09-16T00:09:48.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724995+0000", 2024-09-16T00:09:48.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2089999999999999e-05 2024-09-16T00:09:48.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725062+0000", 2024-09-16T00:09:48.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6026999999999997e-05 2024-09-16T00:09:48.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725159+0000", 2024-09-16T00:09:48.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7374000000000007e-05 2024-09-16T00:09:48.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:48.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725289+0000", 2024-09-16T00:09:48.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012988 2024-09-16T00:09:48.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.731944+0000", 2024-09-16T00:09:48.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066547439999999998 2024-09-16T00:09:48.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.737956+0000", 2024-09-16T00:09:48.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0060121630000000001 2024-09-16T00:09:48.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.737985+0000", 2024-09-16T00:09:48.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9352999999999998e-05 2024-09-16T00:09:48.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.738015+0000", 2024-09-16T00:09:48.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0108999999999999e-05 2024-09-16T00:09:48.934 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.934 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.934 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.934 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11954 3.6 3:625e0e90:::100000032ef.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.934 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:09:48.934 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.0766059209999996, 2024-09-16T00:09:48.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012484808999999999, 2024-09-16T00:09:48.934 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.934 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.934 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.934 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.935 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.935 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.935 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11954 2024-09-16T00:09:48.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.935 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:09:48.935 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:09:48.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.936 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.936 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767886+0000", 2024-09-16T00:09:48.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6650000000000001e-06 2024-09-16T00:09:48.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.936 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.936 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770082+0000", 2024-09-16T00:09:48.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021964979999999999 2024-09-16T00:09:48.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770104+0000", 2024-09-16T00:09:48.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1671999999999999e-05 2024-09-16T00:09:48.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770114+0000", 2024-09-16T00:09:48.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0336000000000001e-05 2024-09-16T00:09:48.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770197+0000", 2024-09-16T00:09:48.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2997000000000005e-05 2024-09-16T00:09:48.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770324+0000", 2024-09-16T00:09:48.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012680100000000001 2024-09-16T00:09:48.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:48.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770497+0000", 2024-09-16T00:09:48.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000172776 2024-09-16T00:09:48.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.776330+0000", 2024-09-16T00:09:48.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058330939999999996 2024-09-16T00:09:48.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780252+0000", 2024-09-16T00:09:48.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039215689999999997 2024-09-16T00:09:48.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780317+0000", 2024-09-16T00:09:48.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4948000000000002e-05 2024-09-16T00:09:48.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780367+0000", 2024-09-16T00:09:48.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0453e-05 2024-09-16T00:09:48.940 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.940 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.940 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.940 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12154 3.2b 3:d78f40a6:::100000033bb.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.940 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.433567+0000", 2024-09-16T00:09:48.940 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.4109206949999997, 2024-09-16T00:09:48.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011046830000000001, 2024-09-16T00:09:48.940 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.941 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.941 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.941 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.941 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.941 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.941 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12154 2024-09-16T00:09:48.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.941 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.433567+0000", 2024-09-16T00:09:48.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.433567+0000", 2024-09-16T00:09:48.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.433570+0000", 2024-09-16T00:09:48.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4650000000000001e-06 2024-09-16T00:09:48.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.434882+0000", 2024-09-16T00:09:48.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013122380000000001 2024-09-16T00:09:48.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.943 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.943 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.434908+0000", 2024-09-16T00:09:48.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5616999999999999e-05 2024-09-16T00:09:48.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.943 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.943 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.434922+0000", 2024-09-16T00:09:48.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3753e-05 2024-09-16T00:09:48.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.435003+0000", 2024-09-16T00:09:48.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1217000000000005e-05 2024-09-16T00:09:48.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.435094+0000", 2024-09-16T00:09:48.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1090999999999999e-05 2024-09-16T00:09:48.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:48.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.435243+0000", 2024-09-16T00:09:48.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014871599999999999 2024-09-16T00:09:48.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.441369+0000", 2024-09-16T00:09:48.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061262790000000001 2024-09-16T00:09:48.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.444505+0000", 2024-09-16T00:09:48.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031357049999999999 2024-09-16T00:09:48.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.444566+0000", 2024-09-16T00:09:48.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1826000000000001e-05 2024-09-16T00:09:48.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.444614+0000", 2024-09-16T00:09:48.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7923000000000002e-05 2024-09-16T00:09:48.946 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.946 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.946 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.947 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12198 3.12 3:4a0e6682:::1000000334b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.947 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.683089+0000", 2024-09-16T00:09:48.947 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.1613988710000003, 2024-09-16T00:09:48.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011990244000000001, 2024-09-16T00:09:48.947 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.947 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.947 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.947 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.947 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.947 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.947 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12198 2024-09-16T00:09:48.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.947 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.948 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.683089+0000", 2024-09-16T00:09:48.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.948 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.683089+0000", 2024-09-16T00:09:48.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.948 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.683092+0000", 2024-09-16T00:09:48.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.931e-06 2024-09-16T00:09:48.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.949 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.949 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684715+0000", 2024-09-16T00:09:48.949 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00162269 2024-09-16T00:09:48.949 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.949 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.949 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684734+0000", 2024-09-16T00:09:48.949 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9545999999999999e-05 2024-09-16T00:09:48.949 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.949 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.949 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684745+0000", 2024-09-16T00:09:48.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0131e-05 2024-09-16T00:09:48.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684814+0000", 2024-09-16T00:09:48.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9740000000000007e-05 2024-09-16T00:09:48.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684871+0000", 2024-09-16T00:09:48.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6215999999999998e-05 2024-09-16T00:09:48.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:48.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684980+0000", 2024-09-16T00:09:48.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109713 2024-09-16T00:09:48.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.690215+0000", 2024-09-16T00:09:48.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052349049999999998 2024-09-16T00:09:48.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.694970+0000", 2024-09-16T00:09:48.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047552649999999998 2024-09-16T00:09:48.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.952 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.695030+0000", 2024-09-16T00:09:48.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9086999999999999e-05 2024-09-16T00:09:48.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.952 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.695080+0000", 2024-09-16T00:09:48.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0019999999999999e-05 2024-09-16T00:09:48.952 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.952 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.952 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.953 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.953 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12348 3.1b 3:db04998a:::100000032d0.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:48.953 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.059532+0000", 2024-09-16T00:09:48.953 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.7849565429999998, 2024-09-16T00:09:48.953 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011465410000000001, 2024-09-16T00:09:48.953 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.953 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.953 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.953 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.953 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.953 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.954 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12348 2024-09-16T00:09:48.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.954 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.954 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.954 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.059532+0000", 2024-09-16T00:09:48.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.954 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.954 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.059532+0000", 2024-09-16T00:09:48.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.059534+0000", 2024-09-16T00:09:48.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8710000000000001e-06 2024-09-16T00:09:48.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.060900+0000", 2024-09-16T00:09:48.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013656219999999999 2024-09-16T00:09:48.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.060919+0000", 2024-09-16T00:09:48.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.878e-05 2024-09-16T00:09:48.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.956 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.060930+0000", 2024-09-16T00:09:48.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1204e-05 2024-09-16T00:09:48.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.956 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.060993+0000", 2024-09-16T00:09:48.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2899999999999997e-05 2024-09-16T00:09:48.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.061063+0000", 2024-09-16T00:09:48.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9737e-05 2024-09-16T00:09:48.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:48.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.061165+0000", 2024-09-16T00:09:48.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010232499999999999 2024-09-16T00:09:48.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.066355+0000", 2024-09-16T00:09:48.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051903139999999997 2024-09-16T00:09:48.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.070925+0000", 2024-09-16T00:09:48.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045696510000000001 2024-09-16T00:09:48.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.070964+0000", 2024-09-16T00:09:48.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9026000000000002e-05 2024-09-16T00:09:48.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.959 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.070997+0000", 2024-09-16T00:09:48.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2979999999999999e-05 2024-09-16T00:09:48.959 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.959 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.959 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.959 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13263 3.1b 3:d9746e52:::1000000361c.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:48.959 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.785956+0000", 2024-09-16T00:09:48.959 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.0585319310000001, 2024-09-16T00:09:48.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011930029, 2024-09-16T00:09:48.959 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.960 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.960 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.960 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.960 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.960 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.960 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13263 2024-09-16T00:09:48.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.960 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.960 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.785956+0000", 2024-09-16T00:09:48.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.785956+0000", 2024-09-16T00:09:48.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.785959+0000", 2024-09-16T00:09:48.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6089999999999999e-06 2024-09-16T00:09:48.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787198+0000", 2024-09-16T00:09:48.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012395889999999999 2024-09-16T00:09:48.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787225+0000", 2024-09-16T00:09:48.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6101999999999998e-05 2024-09-16T00:09:48.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787239+0000", 2024-09-16T00:09:48.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4384e-05 2024-09-16T00:09:48.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787329+0000", 2024-09-16T00:09:48.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0555000000000003e-05 2024-09-16T00:09:48.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787503+0000", 2024-09-16T00:09:48.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017311599999999999 2024-09-16T00:09:48.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:48.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787676+0000", 2024-09-16T00:09:48.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017352900000000001 2024-09-16T00:09:48.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.793377+0000", 2024-09-16T00:09:48.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057008409999999999 2024-09-16T00:09:48.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.797765+0000", 2024-09-16T00:09:48.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004388042 2024-09-16T00:09:48.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.797831+0000", 2024-09-16T00:09:48.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.614e-05 2024-09-16T00:09:48.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.797886+0000", 2024-09-16T00:09:48.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5121999999999998e-05 2024-09-16T00:09:48.965 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.965 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.965 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.965 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13315 3.6 3:620fe43b:::100000031c3.00000001:head [write 0~4194304 [2@0]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:48.965 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.912270+0000", 2024-09-16T00:09:48.965 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.93221790699999996, 2024-09-16T00:09:48.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011126637, 2024-09-16T00:09:48.966 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:48.966 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:48.966 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:48.966 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:48.966 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:48.966 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:48.966 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13315 2024-09-16T00:09:48.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.966 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:48.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:48.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.912270+0000", 2024-09-16T00:09:48.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.967 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:48.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.912270+0000", 2024-09-16T00:09:48.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:48.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.967 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:48.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.912272+0000", 2024-09-16T00:09:48.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0099999999999998e-06 2024-09-16T00:09:48.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.967 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:48.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.913870+0000", 2024-09-16T00:09:48.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001598017 2024-09-16T00:09:48.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:48.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.913885+0000", 2024-09-16T00:09:48.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4273999999999999e-05 2024-09-16T00:09:48.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:48.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.913894+0000", 2024-09-16T00:09:48.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5840000000000001e-06 2024-09-16T00:09:48.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:48.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.913950+0000", 2024-09-16T00:09:48.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6363000000000002e-05 2024-09-16T00:09:48.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:48.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.914036+0000", 2024-09-16T00:09:48.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5245000000000001e-05 2024-09-16T00:09:48.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:48.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.914178+0000", 2024-09-16T00:09:48.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014270600000000001 2024-09-16T00:09:48.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:48.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.921155+0000", 2024-09-16T00:09:48.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069768110000000003 2024-09-16T00:09:48.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:48.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.923260+0000", 2024-09-16T00:09:48.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002104992 2024-09-16T00:09:48.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:48.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.923331+0000", 2024-09-16T00:09:48.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0324e-05 2024-09-16T00:09:48.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:48.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:48.971 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:48.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.923397+0000", 2024-09-16T00:09:48.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6310999999999996e-05 2024-09-16T00:09:48.971 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.971 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.971 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.971 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:48.971 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:48.971 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:48.972 INFO:tasks.workunit.client.0.smithi139.stdout:7/738: mkdir d5/d26/d46/d6b/dd6/d10c 0 2024-09-16T00:09:48.972 INFO:tasks.workunit.client.0.smithi139.stdout:7/739: link d5/d16/d6d/d7f/dc2/dab/fcd d5/d16/d6d/d7f/dc2/d50/d44/d55/d90/f10d 0 2024-09-16T00:09:48.973 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:09:48.979 INFO:tasks.workunit.client.0.smithi139.stdout:4/640: sync 2024-09-16T00:09:48.979 INFO:tasks.workunit.client.0.smithi139.stdout:0/590: sync 2024-09-16T00:09:48.979 INFO:tasks.workunit.client.0.smithi139.stdout:8/701: sync 2024-09-16T00:09:48.979 INFO:tasks.workunit.client.0.smithi139.stdout:4/641: chown c11 17516 1 2024-09-16T00:09:48.979 INFO:tasks.workunit.client.0.smithi139.stdout:4/642: fdatasync d12/d47/d49/d60/dc3/fcb 0 2024-09-16T00:09:48.979 INFO:tasks.workunit.client.0.smithi139.stdout:4/643: write d12/d47/db2/fc2 [994970,57219] 0 2024-09-16T00:09:48.979 INFO:tasks.workunit.client.0.smithi139.stdout:4/644: stat d12/d47/d49/d60/f66 0 2024-09-16T00:09:48.979 INFO:tasks.workunit.client.0.smithi139.stdout:4/645: read - d12/d47/d49/d60/dc3/fcb zero size 2024-09-16T00:09:48.979 INFO:tasks.workunit.client.0.smithi139.stdout:4/646: chown d12/d47/db2 566343814 1 2024-09-16T00:09:48.979 INFO:tasks.workunit.client.0.smithi139.stdout:0/591: write d6/d1c/d41/d44/f33 [710963,7261] 0 2024-09-16T00:09:48.979 INFO:tasks.workunit.client.0.smithi139.stdout:8/702: dread d17/d1f/d3a/db7/d40/f5d [0,4194304] 0 2024-09-16T00:09:48.979 INFO:tasks.workunit.client.0.smithi139.stdout:0/592: symlink d6/d1c/d41/d44/d58/lb8 0 2024-09-16T00:09:48.980 INFO:tasks.workunit.client.0.smithi139.stdout:0/593: symlink d6/d1c/d41/d44/d6b/lb9 0 2024-09-16T00:09:48.980 INFO:tasks.workunit.client.0.smithi139.stdout:4/647: truncate d12/d47/d57/f62 7113830 0 2024-09-16T00:09:48.980 INFO:tasks.workunit.client.0.smithi139.stdout:4/648: rename d12/l1b to d12/d47/db2/d4d/d69/lce 0 2024-09-16T00:09:48.980 INFO:tasks.workunit.client.0.smithi139.stdout:4/649: readlink d12/d47/db2/d4d/da9/lc6 0 2024-09-16T00:09:48.980 INFO:tasks.workunit.client.0.smithi139.stdout:4/650: write d12/d47/faf [427516,97086] 0 2024-09-16T00:09:48.980 INFO:tasks.workunit.client.0.smithi139.stdout:4/651: write d12/d2b/f78 [1315830,995] 0 2024-09-16T00:09:48.980 INFO:tasks.workunit.client.0.smithi139.stdout:8/703: dread d17/d1a/fa5 [0,4194304] 0 2024-09-16T00:09:48.980 INFO:tasks.workunit.client.0.smithi139.stdout:0/594: dwrite d6/d1c/f35 [4194304,4194304] 0 2024-09-16T00:09:48.980 INFO:tasks.workunit.client.0.smithi139.stdout:8/704: read - d17/d1f/d3a/db7/d40/d4f/f75 zero size 2024-09-16T00:09:48.980 INFO:tasks.workunit.client.0.smithi139.stdout:0/595: creat d6/d5a/d70/d7b/fba x:0 0 0 2024-09-16T00:09:48.980 INFO:tasks.workunit.client.0.smithi139.stdout:4/652: dread d12/d47/db2/f54 [0,4194304] 0 2024-09-16T00:09:48.980 INFO:tasks.workunit.client.0.smithi139.stdout:0/596: creat d6/d1c/d41/fbb x:0 0 0 2024-09-16T00:09:48.980 INFO:tasks.workunit.client.0.smithi139.stdout:8/705: dwrite d17/d1a/f73 [4194304,4194304] 0 2024-09-16T00:09:48.980 INFO:tasks.workunit.client.0.smithi139.stdout:0/597: write d6/d1c/f8d [4712752,74338] 0 2024-09-16T00:09:48.981 INFO:tasks.workunit.client.0.smithi139.stdout:0/598: mknod d6/d1c/d41/d44/d58/cbc 0 2024-09-16T00:09:48.981 INFO:tasks.workunit.client.0.smithi139.stdout:4/653: write d12/d80/f95 [4431034,111909] 0 2024-09-16T00:09:48.981 INFO:tasks.workunit.client.0.smithi139.stdout:0/599: mkdir d6/d1c/d41/d44/dd/d56/dbd 0 2024-09-16T00:09:48.981 INFO:tasks.workunit.client.0.smithi139.stdout:8/706: dwrite d17/d1f/d3a/db7/d40/d4f/f75 [0,4194304] 0 2024-09-16T00:09:48.981 INFO:tasks.workunit.client.0.smithi139.stdout:4/654: unlink d12/c82 0 2024-09-16T00:09:48.981 INFO:tasks.workunit.client.0.smithi139.stdout:0/600: write d6/d1c/d41/d44/dd/d56/f57 [3657167,87315] 0 2024-09-16T00:09:48.981 INFO:tasks.workunit.client.0.smithi139.stdout:8/707: unlink d17/d1a/d2d/l8c 0 2024-09-16T00:09:48.981 INFO:tasks.workunit.client.0.smithi139.stdout:0/601: creat d6/d1c/d41/d63/d82/fbe x:0 0 0 2024-09-16T00:09:48.981 INFO:tasks.workunit.client.0.smithi139.stdout:4/655: creat d12/d47/d57/fcf x:0 0 0 2024-09-16T00:09:48.981 INFO:tasks.workunit.client.0.smithi139.stdout:0/602: dread d6/f64 [0,4194304] 0 2024-09-16T00:09:48.981 INFO:tasks.workunit.client.0.smithi139.stdout:8/708: dwrite d17/d1a/d2d/f3e [4194304,4194304] 0 2024-09-16T00:09:48.981 INFO:tasks.workunit.client.0.smithi139.stdout:0/603: rmdir d6/d1c/d41/d44/dd 39 2024-09-16T00:09:48.981 INFO:tasks.workunit.client.0.smithi139.stdout:0/604: chown d6/d42/db4/f8e 8373144 1 2024-09-16T00:09:48.982 INFO:tasks.workunit.client.0.smithi139.stdout:8/709: mkdir d17/d1a/d50/dee 0 2024-09-16T00:09:48.982 INFO:tasks.workunit.client.0.smithi139.stdout:8/710: write d17/d1f/d3a/db7/d40/d4f/f65 [2161307,2550] 0 2024-09-16T00:09:48.982 INFO:tasks.workunit.client.0.smithi139.stdout:4/656: link d12/d47/db2/d4d/f55 d12/d47/d49/d60/dc3/fd0 0 2024-09-16T00:09:48.982 INFO:tasks.workunit.client.0.smithi139.stdout:0/605: unlink d6/d1c/d41/d44/d58/la2 0 2024-09-16T00:09:48.982 INFO:tasks.workunit.client.0.smithi139.stdout:4/657: mkdir d12/d47/db2/d4d/dd1 0 2024-09-16T00:09:48.982 INFO:tasks.workunit.client.0.smithi139.stdout:0/606: chown d6/d1c/f21 33809 1 2024-09-16T00:09:48.982 INFO:tasks.workunit.client.0.smithi139.stdout:4/658: creat d12/d47/db2/d4d/da9/fd2 x:0 0 0 2024-09-16T00:09:48.982 INFO:tasks.workunit.client.0.smithi139.stdout:4/659: write d12/d47/d57/fb5 [4428370,1061] 0 2024-09-16T00:09:48.982 INFO:tasks.workunit.client.0.smithi139.stdout:8/711: mkdir d17/d1a/d50/d86/d92/dd6/def 0 2024-09-16T00:09:48.982 INFO:tasks.workunit.client.0.smithi139.stdout:0/607: mknod d6/d5a/cbf 0 2024-09-16T00:09:48.982 INFO:tasks.workunit.client.0.smithi139.stdout:4/660: truncate d12/d47/d49/fa0 454045 0 2024-09-16T00:09:48.982 INFO:tasks.workunit.client.0.smithi139.stdout:0/608: stat d6/d1c/f4c 0 2024-09-16T00:09:48.982 INFO:tasks.workunit.client.0.smithi139.stdout:8/712: dread d17/d1f/d3a/db7/f42 [0,4194304] 0 2024-09-16T00:09:48.983 INFO:tasks.workunit.client.0.smithi139.stdout:4/661: creat d12/d47/d49/d60/dac/fd3 x:0 0 0 2024-09-16T00:09:48.983 INFO:tasks.workunit.client.0.smithi139.stdout:4/662: chown d12/d47/db2/d4d/f9d 6 1 2024-09-16T00:09:48.983 INFO:tasks.workunit.client.0.smithi139.stdout:0/609: mknod d6/d1c/d41/d44/dd/d56/d75/cc0 0 2024-09-16T00:09:48.983 INFO:tasks.workunit.client.0.smithi139.stdout:0/610: creat d6/d1c/fc1 x:0 0 0 2024-09-16T00:09:48.983 INFO:tasks.workunit.client.0.smithi139.stdout:4/663: link d12/d47/d49/d60/c7d d12/d47/d49/cd4 0 2024-09-16T00:09:48.983 INFO:tasks.workunit.client.0.smithi139.stdout:8/713: dwrite f4 [0,4194304] 0 2024-09-16T00:09:48.984 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:48.974+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:49.018 INFO:tasks.workunit.client.0.smithi139.stdout:1/551: sync 2024-09-16T00:09:49.018 INFO:tasks.workunit.client.0.smithi139.stdout:3/581: sync 2024-09-16T00:09:49.019 INFO:tasks.workunit.client.0.smithi139.stdout:4/664: creat d12/d47/d57/fd5 x:0 0 0 2024-09-16T00:09:49.019 INFO:tasks.workunit.client.0.smithi139.stdout:8/714: write d17/d1a/d2d/dac/fb3 [2048669,13779] 0 2024-09-16T00:09:49.020 INFO:tasks.workunit.client.0.smithi139.stdout:0/611: dwrite d6/d42/db4/f50 [0,4194304] 0 2024-09-16T00:09:49.023 INFO:tasks.workunit.client.0.smithi139.stdout:3/582: symlink d5/d7/d23/d27/d2f/d30/d41/ld0 0 2024-09-16T00:09:49.025 INFO:tasks.workunit.client.0.smithi139.stdout:4/665: mkdir d12/d47/db2/d4d/dd6 0 2024-09-16T00:09:49.025 INFO:tasks.workunit.client.0.smithi139.stdout:0/612: read d6/d1c/d41/d44/dd/f67 [1473340,25708] 0 2024-09-16T00:09:49.026 INFO:tasks.workunit.client.0.smithi139.stdout:4/666: write d12/f42 [7065712,91439] 0 2024-09-16T00:09:49.028 INFO:tasks.workunit.client.0.smithi139.stdout:1/552: dwrite d7/d8/f21 [0,4194304] 0 2024-09-16T00:09:49.034 INFO:tasks.workunit.client.0.smithi139.stdout:3/583: mknod d5/d7/d23/d27/d2f/d30/d4c/da1/cd1 0 2024-09-16T00:09:49.034 INFO:tasks.workunit.client.0.smithi139.stdout:0/613: chown d6/d1c/d41/d44/l3f 2127396 1 2024-09-16T00:09:49.035 INFO:tasks.workunit.client.0.smithi139.stdout:8/715: dwrite d17/d1f/d3a/db7/f42 [0,4194304] 0 2024-09-16T00:09:49.037 INFO:tasks.workunit.client.0.smithi139.stdout:8/716: fdatasync d17/d54/fed 0 2024-09-16T00:09:49.039 INFO:tasks.workunit.client.0.smithi139.stdout:0/614: write d6/d1c/d41/d44/f25 [2477729,129750] 0 2024-09-16T00:09:49.043 INFO:tasks.workunit.client.0.smithi139.stdout:4/667: getdents d12/d2b/d8c 0 2024-09-16T00:09:49.043 INFO:tasks.workunit.client.0.smithi139.stdout:3/584: write d5/d7/d23/d27/d2f/d30/d4c/d50/f87 [672869,93933] 0 2024-09-16T00:09:49.044 INFO:tasks.workunit.client.0.smithi139.stdout:3/585: fsync d5/d7/fd 0 2024-09-16T00:09:49.045 INFO:tasks.workunit.client.0.smithi139.stdout:8/717: creat d17/d1f/d3a/db7/d40/ff0 x:0 0 0 2024-09-16T00:09:49.047 INFO:tasks.workunit.client.0.smithi139.stdout:0/615: write d6/d1c/d41/d44/d58/f88 [2962867,58346] 0 2024-09-16T00:09:49.049 INFO:tasks.workunit.client.0.smithi139.stdout:4/668: getdents d12/d47/db2/d4d/db0 0 2024-09-16T00:09:49.049 INFO:tasks.workunit.client.0.smithi139.stdout:1/553: dwrite d7/f13 [4194304,4194304] 0 2024-09-16T00:09:49.050 INFO:tasks.workunit.client.0.smithi139.stdout:0/616: creat d6/d5a/d81/fc2 x:0 0 0 2024-09-16T00:09:49.053 INFO:tasks.workunit.client.0.smithi139.stdout:8/718: getdents d17/d1a/d2d 0 2024-09-16T00:09:49.054 INFO:tasks.workunit.client.0.smithi139.stdout:1/554: dread d7/f9b [0,4194304] 0 2024-09-16T00:09:49.054 INFO:tasks.workunit.client.0.smithi139.stdout:3/586: truncate d5/d7/d23/d27/d2f/d30/f4e 881848 0 2024-09-16T00:09:49.054 INFO:tasks.workunit.client.0.smithi139.stdout:0/617: fsync d6/d42/db4/f8e 0 2024-09-16T00:09:49.056 INFO:tasks.workunit.client.0.smithi139.stdout:1/555: chown d7/d8/d27/d68/d7c/d83 42 1 2024-09-16T00:09:49.056 INFO:tasks.workunit.client.0.smithi139.stdout:8/719: fsync d17/f70 0 2024-09-16T00:09:49.058 INFO:tasks.workunit.client.0.smithi139.stdout:0/618: read d6/d1c/d41/d44/dd/f67 [3789781,69310] 0 2024-09-16T00:09:49.060 INFO:tasks.workunit.client.0.smithi139.stdout:1/556: creat d7/d8/d1f/fa1 x:0 0 0 2024-09-16T00:09:49.061 INFO:tasks.workunit.client.0.smithi139.stdout:8/720: creat d17/d1a/d50/d86/d9d/daf/ff1 x:0 0 0 2024-09-16T00:09:49.062 INFO:tasks.workunit.client.0.smithi139.stdout:3/587: dwrite d5/d7/fd [0,4194304] 0 2024-09-16T00:09:49.064 INFO:tasks.workunit.client.0.smithi139.stdout:0/619: creat d6/d1c/d41/d44/dd/d8a/d9e/fc3 x:0 0 0 2024-09-16T00:09:49.065 INFO:tasks.workunit.client.0.smithi139.stdout:4/669: write d12/d47/d57/f62 [7351566,99709] 0 2024-09-16T00:09:49.066 INFO:tasks.workunit.client.0.smithi139.stdout:8/721: mkdir d17/d89/df2 0 2024-09-16T00:09:49.068 INFO:tasks.workunit.client.0.smithi139.stdout:1/557: dread d7/f3f [0,4194304] 0 2024-09-16T00:09:49.068 INFO:tasks.workunit.client.0.smithi139.stdout:3/588: read d5/d7/dc/f63 [454478,106423] 0 2024-09-16T00:09:49.069 INFO:tasks.workunit.client.0.smithi139.stdout:0/620: creat d6/d1c/d41/d44/dd/d56/fc4 x:0 0 0 2024-09-16T00:09:49.070 INFO:tasks.workunit.client.0.smithi139.stdout:8/722: truncate d17/d54/f80 121749 0 2024-09-16T00:09:49.071 INFO:tasks.workunit.client.0.smithi139.stdout:3/589: chown d5/d7/d23/d27/d2f/d30/d41 123594577 1 2024-09-16T00:09:49.072 INFO:tasks.workunit.client.0.smithi139.stdout:0/621: mkdir d6/d5a/d81/dc5 0 2024-09-16T00:09:49.074 INFO:tasks.workunit.client.0.smithi139.stdout:4/670: dwrite d12/d47/f6c [4194304,4194304] 0 2024-09-16T00:09:49.075 INFO:tasks.workunit.client.0.smithi139.stdout:4/671: stat d12/d47/f32 0 2024-09-16T00:09:49.075 INFO:tasks.workunit.client.0.smithi139.stdout:8/723: read d17/d1f/f28 [1462249,14572] 0 2024-09-16T00:09:49.076 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:49.075+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:49.077 INFO:tasks.workunit.client.0.smithi139.stdout:0/622: write d6/d1c/d41/d44/f3a [1097401,121481] 0 2024-09-16T00:09:49.082 INFO:tasks.workunit.client.0.smithi139.stdout:0/623: mknod d6/d1c/d41/cc6 0 2024-09-16T00:09:49.083 INFO:tasks.workunit.client.0.smithi139.stdout:8/724: dread fb [0,4194304] 0 2024-09-16T00:09:49.083 INFO:tasks.workunit.client.0.smithi139.stdout:1/558: dread d7/d8/f1e [0,4194304] 0 2024-09-16T00:09:49.085 INFO:tasks.workunit.client.0.smithi139.stdout:0/624: read - d6/d1c/d41/d63/d82/f99 zero size 2024-09-16T00:09:49.085 INFO:tasks.workunit.client.0.smithi139.stdout:4/672: dwrite d12/d47/db2/d4d/f86 [0,4194304] 0 2024-09-16T00:09:49.086 INFO:tasks.workunit.client.0.smithi139.stdout:8/725: read - d17/d1a/d50/d86/d9d/daf/fc7 zero size 2024-09-16T00:09:49.087 INFO:tasks.workunit.client.0.smithi139.stdout:1/559: creat d7/d8/d27/d2f/fa2 x:0 0 0 2024-09-16T00:09:49.088 INFO:tasks.workunit.client.0.smithi139.stdout:8/726: write d17/d1a/d2d/fa9 [248939,13004] 0 2024-09-16T00:09:49.090 INFO:tasks.workunit.client.0.smithi139.stdout:4/673: creat d12/d2b/d8c/fd7 x:0 0 0 2024-09-16T00:09:49.091 INFO:tasks.workunit.client.0.smithi139.stdout:8/727: chown d17/d1f/c24 12422032 1 2024-09-16T00:09:49.091 INFO:tasks.workunit.client.0.smithi139.stdout:1/560: write d7/d8/d1f/fa1 [290694,86547] 0 2024-09-16T00:09:49.091 INFO:tasks.workunit.client.0.smithi139.stdout:0/625: write d6/d1c/d41/d44/f3b [3212646,101889] 0 2024-09-16T00:09:49.094 INFO:tasks.workunit.client.0.smithi139.stdout:8/728: rmdir d17/d89 39 2024-09-16T00:09:49.095 INFO:tasks.workunit.client.0.smithi139.stdout:0/626: creat d6/d1c/d41/d63/d82/fc7 x:0 0 0 2024-09-16T00:09:49.096 INFO:tasks.workunit.client.0.smithi139.stdout:4/674: rename d12/d2b/c3e to d12/d47/d57/cd8 0 2024-09-16T00:09:49.097 INFO:tasks.workunit.client.0.smithi139.stdout:8/729: creat d17/d1a/d2d/ff3 x:0 0 0 2024-09-16T00:09:49.099 INFO:tasks.workunit.client.0.smithi139.stdout:8/730: mknod d17/d1a/d50/d86/d9d/daf/cf4 0 2024-09-16T00:09:49.104 INFO:tasks.workunit.client.0.smithi139.stdout:8/731: unlink d17/d1a/ca0 0 2024-09-16T00:09:49.106 INFO:tasks.workunit.client.0.smithi139.stdout:4/675: dwrite d12/d47/db2/d4d/fbb [0,4194304] 0 2024-09-16T00:09:49.107 INFO:tasks.workunit.client.0.smithi139.stdout:1/561: dwrite d7/f58 [0,4194304] 0 2024-09-16T00:09:49.112 INFO:tasks.workunit.client.0.smithi139.stdout:0/627: dread d6/d1c/d41/d44/f3b [0,4194304] 0 2024-09-16T00:09:49.113 INFO:tasks.workunit.client.0.smithi139.stdout:0/628: fsync f3 0 2024-09-16T00:09:49.115 INFO:tasks.workunit.client.0.smithi139.stdout:1/562: dread d7/d8/d40/d46/d8d/f95 [0,4194304] 0 2024-09-16T00:09:49.115 INFO:tasks.workunit.client.0.smithi139.stdout:0/629: stat d6/d42/c97 0 2024-09-16T00:09:49.116 INFO:tasks.workunit.client.0.smithi139.stdout:0/630: chown d6/d1c/d41/d44/dd/d56/f57 2 1 2024-09-16T00:09:49.117 INFO:tasks.workunit.client.0.smithi139.stdout:4/676: rename d12/d47/db2/d4d/da9/fb6 to d12/d47/d57/fd9 0 2024-09-16T00:09:49.117 INFO:tasks.workunit.client.0.smithi139.stdout:0/631: chown d6/d1c/d41/d44/dd/d56/d75/cc0 234544509 1 2024-09-16T00:09:49.117 INFO:tasks.workunit.client.0.smithi139.stdout:8/732: truncate d17/d1a/d57/f71 289229 0 2024-09-16T00:09:49.118 INFO:tasks.workunit.client.0.smithi139.stdout:0/632: stat d6/d1c/d41/d44/dd/l30 0 2024-09-16T00:09:49.120 INFO:tasks.workunit.client.0.smithi139.stdout:8/733: write d17/d1a/d50/d86/d92/f9b [1610351,95417] 0 2024-09-16T00:09:49.121 INFO:tasks.workunit.client.0.smithi139.stdout:0/633: creat d6/d5a/d70/fc8 x:0 0 0 2024-09-16T00:09:49.122 INFO:tasks.workunit.client.0.smithi139.stdout:1/563: mknod d7/d8/ca3 0 2024-09-16T00:09:49.123 INFO:tasks.workunit.client.0.smithi139.stdout:1/564: write d7/d8/d1f/f20 [1428093,71596] 0 2024-09-16T00:09:49.124 INFO:tasks.workunit.client.0.smithi139.stdout:4/677: dwrite d12/d2b/d8c/fd7 [0,4194304] 0 2024-09-16T00:09:49.128 INFO:tasks.workunit.client.0.smithi139.stdout:1/565: fsync d7/d8/d40/d48/f65 0 2024-09-16T00:09:49.129 INFO:tasks.workunit.client.0.smithi139.stdout:8/734: dread d17/d1a/d2d/fa9 [0,4194304] 0 2024-09-16T00:09:49.129 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:49.129 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:49.129 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:49.130 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:49.131 INFO:tasks.workunit.client.0.smithi139.stdout:4/678: getdents d12/d47/db2/d4d/dd6 0 2024-09-16T00:09:49.133 INFO:tasks.workunit.client.0.smithi139.stdout:0/634: dwrite d6/d1c/f35 [4194304,4194304] 0 2024-09-16T00:09:49.133 INFO:tasks.workunit.client.0.smithi139.stdout:1/566: getdents d7/d8/d40/d48/d4a/d82 0 2024-09-16T00:09:49.135 INFO:tasks.workunit.client.0.smithi139.stdout:4/679: unlink d12/l70 0 2024-09-16T00:09:49.138 INFO:tasks.workunit.client.0.smithi139.stdout:8/735: rename d17/d1f/c37 to d17/d1a/d50/d86/d9d/daf/ddf/cf5 0 2024-09-16T00:09:49.139 INFO:tasks.workunit.client.0.smithi139.stdout:4/680: readlink d12/d47/d57/d7f/lb9 0 2024-09-16T00:09:49.140 INFO:tasks.workunit.client.0.smithi139.stdout:8/736: chown d17/d54/cca 179056 1 2024-09-16T00:09:49.140 INFO:tasks.workunit.client.0.smithi139.stdout:1/567: mkdir d7/d8/d27/d68/d7c/d83/da4 0 2024-09-16T00:09:49.141 INFO:tasks.workunit.client.0.smithi139.stdout:4/681: write d12/d47/db2/d4d/da9/fd2 [26097,6348] 0 2024-09-16T00:09:49.143 INFO:tasks.workunit.client.0.smithi139.stdout:8/737: write d17/d1f/d3a/db7/d40/fb4 [874396,46669] 0 2024-09-16T00:09:49.144 INFO:tasks.workunit.client.0.smithi139.stdout:8/738: chown d17/d1a/f1d 3 1 2024-09-16T00:09:49.145 INFO:tasks.workunit.client.0.smithi139.stdout:1/568: rmdir d7/d8/d27/d68/d7c/d83/da4 0 2024-09-16T00:09:49.147 INFO:tasks.workunit.client.0.smithi139.stdout:0/635: dread d6/d5a/d70/f4f [0,4194304] 0 2024-09-16T00:09:49.148 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:09:49.148 INFO:tasks.workunit.client.0.smithi139.stdout:8/739: creat d17/d1a/d50/d86/d92/dd6/def/ff6 x:0 0 0 2024-09-16T00:09:49.149 INFO:tasks.workunit.client.0.smithi139.stdout:0/636: chown d6/d5a/d81/fa4 58775 1 2024-09-16T00:09:49.149 INFO:tasks.workunit.client.0.smithi139.stdout:0/637: readlink d6/d1c/d41/d44/dd/l30 0 2024-09-16T00:09:49.151 INFO:tasks.workunit.client.0.smithi139.stdout:8/740: creat d17/d89/ff7 x:0 0 0 2024-09-16T00:09:49.151 INFO:tasks.workunit.client.0.smithi139.stdout:0/638: dread d6/d1c/f35 [4194304,4194304] 0 2024-09-16T00:09:49.153 INFO:tasks.workunit.client.0.smithi139.stdout:0/639: dread - d6/d1c/d41/d44/d58/faf zero size 2024-09-16T00:09:49.154 INFO:tasks.workunit.client.0.smithi139.stdout:0/640: write d6/d1c/d41/d44/dd/d56/fc4 [5361,89353] 0 2024-09-16T00:09:49.154 INFO:tasks.workunit.client.0.smithi139.stdout:8/741: creat d17/ff8 x:0 0 0 2024-09-16T00:09:49.154 INFO:tasks.workunit.client.0.smithi139.stdout:1/569: truncate d7/d8/d1f/f37 40042 0 2024-09-16T00:09:49.155 INFO:tasks.workunit.client.0.smithi139.stdout:8/742: chown d17/d54/cca 398 1 2024-09-16T00:09:49.159 INFO:tasks.workunit.client.0.smithi139.stdout:1/570: link d7/d8/f1a d7/d1b/fa5 0 2024-09-16T00:09:49.161 INFO:tasks.workunit.client.0.smithi139.stdout:8/743: dwrite d17/d1f/d3a/db7/d40/fc8 [0,4194304] 0 2024-09-16T00:09:49.162 INFO:tasks.workunit.client.0.smithi139.stdout:0/641: link d6/d1c/f8d d6/fc9 0 2024-09-16T00:09:49.162 INFO:tasks.workunit.client.0.smithi139.stdout:8/744: write d17/f99 [1028842,55387] 0 2024-09-16T00:09:49.164 INFO:tasks.workunit.client.0.smithi139.stdout:8/745: fsync d17/d1f/f28 0 2024-09-16T00:09:49.165 INFO:tasks.workunit.client.0.smithi139.stdout:0/642: mknod d6/d1c/d41/d44/dd/d8a/d9e/cca 0 2024-09-16T00:09:49.166 INFO:tasks.workunit.client.0.smithi139.stdout:8/746: chown d17/d1f/d3a/db7/c47 1 1 2024-09-16T00:09:49.168 INFO:tasks.workunit.client.0.smithi139.stdout:0/643: mkdir d6/d1c/d41/d44/dd/d8a/d9e/dcb 0 2024-09-16T00:09:49.170 INFO:tasks.workunit.client.0.smithi139.stdout:5/743: sync 2024-09-16T00:09:49.170 INFO:tasks.workunit.client.0.smithi139.stdout:9/809: sync 2024-09-16T00:09:49.172 INFO:tasks.workunit.client.0.smithi139.stdout:8/747: creat d17/d1f/d3a/db7/d40/ff9 x:0 0 0 2024-09-16T00:09:49.173 INFO:tasks.workunit.client.0.smithi139.stdout:9/810: chown d19/d1a/d20/d23/d2d/d2f/l85 0 1 2024-09-16T00:09:49.175 INFO:tasks.workunit.client.0.smithi139.stdout:8/748: dread d17/d1a/d50/d86/f97 [0,4194304] 0 2024-09-16T00:09:49.176 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:49.175+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:49.178 INFO:tasks.workunit.client.0.smithi139.stdout:8/749: mknod d17/d89/cfa 0 2024-09-16T00:09:49.181 INFO:tasks.workunit.client.0.smithi139.stdout:5/744: dwrite d1/d21/d49/d53/d6a/d7b/d99/feb [0,4194304] 0 2024-09-16T00:09:49.182 INFO:tasks.workunit.client.0.smithi139.stdout:9/811: dread d19/d1a/d20/d23/d2d/d35/fb1 [0,4194304] 0 2024-09-16T00:09:49.251 INFO:tasks.workunit.client.0.smithi139.stdout:0/644: dread d6/d1c/d41/d44/dd/d56/f57 [0,4194304] 0 2024-09-16T00:09:49.251 INFO:tasks.workunit.client.0.smithi139.stdout:5/745: dread - d1/fad zero size 2024-09-16T00:09:49.252 INFO:tasks.workunit.client.0.smithi139.stdout:9/812: read d19/d1a/f50 [3001161,109499] 0 2024-09-16T00:09:49.252 INFO:tasks.workunit.client.0.smithi139.stdout:0/645: chown d6/d1c/f4c 24231354 1 2024-09-16T00:09:49.253 INFO:tasks.workunit.client.0.smithi139.stdout:0/646: fdatasync d6/d1c/fc1 0 2024-09-16T00:09:49.253 INFO:tasks.workunit.client.0.smithi139.stdout:5/746: read d1/d21/d49/fd1 [2858160,40212] 0 2024-09-16T00:09:49.255 INFO:tasks.workunit.client.0.smithi139.stdout:9/813: chown d19/d1a/d20/d23/d2d/d35/d42/dc8/fd6 0 1 2024-09-16T00:09:49.256 INFO:tasks.workunit.client.0.smithi139.stdout:5/747: write d1/d13/f97 [3598166,16845] 0 2024-09-16T00:09:49.257 INFO:tasks.workunit.client.0.smithi139.stdout:9/814: readlink d19/d1a/d20/d23/dca/d71/d32/d79/l95 0 2024-09-16T00:09:49.258 INFO:tasks.workunit.client.0.smithi139.stdout:0/647: dread d6/d1c/d41/d44/f33 [0,4194304] 0 2024-09-16T00:09:49.259 INFO:tasks.workunit.client.0.smithi139.stdout:5/748: mkdir d1/d13/d3f/d9f/d106 0 2024-09-16T00:09:49.259 INFO:tasks.workunit.client.0.smithi139.stdout:0/648: fdatasync d6/d1c/d41/d63/d82/fc7 0 2024-09-16T00:09:49.260 INFO:tasks.workunit.client.0.smithi139.stdout:9/815: truncate d19/d1a/d20/d23/d2d/d9a/faa 555196 0 2024-09-16T00:09:49.262 INFO:tasks.workunit.client.0.smithi139.stdout:5/749: rename d1/d13/d3f/lf5 to d1/d21/l107 0 2024-09-16T00:09:49.263 INFO:tasks.workunit.client.0.smithi139.stdout:8/750: dread d17/d1f/d3a/db7/d40/d4f/f65 [0,4194304] 0 2024-09-16T00:09:49.263 INFO:tasks.workunit.client.0.smithi139.stdout:9/816: stat d19/d1a/d20/d23/dca/d71/d32/d4e/f8a 0 2024-09-16T00:09:49.266 INFO:tasks.workunit.client.0.smithi139.stdout:9/817: unlink d19/d1a/d20/dbe/fff 0 2024-09-16T00:09:49.267 INFO:tasks.workunit.client.0.smithi139.stdout:0/649: dwrite d6/d1c/d41/d44/f29 [0,4194304] 0 2024-09-16T00:09:49.270 INFO:tasks.workunit.client.0.smithi139.stdout:9/818: symlink d19/d1a/d20/d23/d2d/d9a/l112 0 2024-09-16T00:09:49.272 INFO:tasks.workunit.client.0.smithi139.stdout:0/650: symlink d6/d1c/d41/d63/d82/lcc 0 2024-09-16T00:09:49.273 INFO:tasks.workunit.client.0.smithi139.stdout:8/751: dwrite d17/d1f/f25 [0,4194304] 0 2024-09-16T00:09:49.275 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:49.275 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:49.275 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:49.275 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:49.276 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:49.277 INFO:tasks.workunit.client.0.smithi139.stdout:9/819: truncate d19/d1a/d20/d23/dca/d71/d32/d4e/f8a 1090211 0 2024-09-16T00:09:49.277 INFO:tasks.workunit.client.0.smithi139.stdout:0/651: creat d6/d5a/d70/fcd x:0 0 0 2024-09-16T00:09:49.277 INFO:tasks.workunit.client.0.smithi139.stdout:9/820: stat d19/d1a/d20/d23/d2d/d35/fb1 0 2024-09-16T00:09:49.278 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:49.277+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:49.279 INFO:tasks.workunit.client.0.smithi139.stdout:8/752: creat d17/d1a/d50/d86/d92/dd6/ffb x:0 0 0 2024-09-16T00:09:49.281 INFO:tasks.workunit.client.0.smithi139.stdout:9/821: symlink d19/d1a/d20/d23/d2d/d2f/l113 0 2024-09-16T00:09:49.282 INFO:tasks.workunit.client.0.smithi139.stdout:0/652: mkdir d6/d1c/d41/d44/dce 0 2024-09-16T00:09:49.284 INFO:tasks.workunit.client.0.smithi139.stdout:0/653: write d6/d1c/d41/d44/fae [977916,30316] 0 2024-09-16T00:09:49.285 INFO:tasks.workunit.client.0.smithi139.stdout:8/753: getdents d17/d1f/d3a/db7/d40 0 2024-09-16T00:09:49.286 INFO:tasks.workunit.client.0.smithi139.stdout:8/754: chown fb 27647 1 2024-09-16T00:09:49.288 INFO:tasks.workunit.client.0.smithi139.stdout:5/750: dread d1/d13/d3f/fd3 [0,4194304] 0 2024-09-16T00:09:49.289 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:09:49.289 INFO:tasks.workunit.client.0.smithi139.stdout:0/654: readlink d6/d1c/d41/d63/d82/lb2 0 2024-09-16T00:09:49.290 INFO:tasks.workunit.client.0.smithi139.stdout:8/755: mknod d17/d89/df2/cfc 0 2024-09-16T00:09:49.292 INFO:tasks.workunit.client.0.smithi139.stdout:9/822: dwrite d19/d1a/d20/d23/dca/f7c [0,4194304] 0 2024-09-16T00:09:49.293 INFO:tasks.workunit.client.0.smithi139.stdout:9/823: write d19/d1a/f8f [510633,113269] 0 2024-09-16T00:09:49.294 INFO:tasks.workunit.client.0.smithi139.stdout:8/756: creat d17/d89/df2/ffd x:0 0 0 2024-09-16T00:09:49.297 INFO:tasks.workunit.client.0.smithi139.stdout:5/751: dwrite d1/d21/d49/db2/fba [4194304,4194304] 0 2024-09-16T00:09:49.300 INFO:tasks.workunit.client.0.smithi139.stdout:0/655: write d6/d1c/d41/d44/dd/d8a/f92 [899011,122867] 0 2024-09-16T00:09:49.304 INFO:tasks.workunit.client.0.smithi139.stdout:9/824: dread d19/d1a/d20/d23/d2d/d35/d60/fb3 [0,4194304] 0 2024-09-16T00:09:49.306 INFO:tasks.workunit.client.0.smithi139.stdout:0/656: read d6/d1c/d41/d44/dd/f2c [451937,121013] 0 2024-09-16T00:09:49.309 INFO:tasks.workunit.client.0.smithi139.stdout:6/514: sync 2024-09-16T00:09:49.309 INFO:tasks.workunit.client.0.smithi139.stdout:2/827: sync 2024-09-16T00:09:49.310 INFO:tasks.workunit.client.0.smithi139.stdout:8/757: dwrite d17/d54/f69 [0,4194304] 0 2024-09-16T00:09:49.310 INFO:tasks.workunit.client.0.smithi139.stdout:0/657: creat d6/d1c/d41/d44/dd/d56/d75/fcf x:0 0 0 2024-09-16T00:09:49.313 INFO:tasks.workunit.client.0.smithi139.stdout:0/658: dread - d6/d1c/d41/d44/d58/faf zero size 2024-09-16T00:09:49.313 INFO:tasks.workunit.client.0.smithi139.stdout:9/825: link d19/d1a/d20/d23/dca/d71/d32/d4e/f8a d19/d1a/d20/d23/d2d/d35/d44/f114 0 2024-09-16T00:09:49.314 INFO:tasks.workunit.client.0.smithi139.stdout:6/515: creat d8/d29/d83/d26/d28/d5a/fb2 x:0 0 0 2024-09-16T00:09:49.316 INFO:tasks.workunit.client.0.smithi139.stdout:9/826: readlink d19/d1a/d20/d23/dca/d71/d32/l6b 0 2024-09-16T00:09:49.317 INFO:tasks.workunit.client.0.smithi139.stdout:8/758: creat d17/d1a/ffe x:0 0 0 2024-09-16T00:09:49.319 INFO:tasks.workunit.client.0.smithi139.stdout:2/828: dwrite d11/d2f/d6e/f84 [0,4194304] 0 2024-09-16T00:09:49.324 INFO:tasks.workunit.client.0.smithi139.stdout:8/759: symlink d17/d1f/d3a/db0/lff 0 2024-09-16T00:09:49.325 INFO:tasks.workunit.client.0.smithi139.stdout:9/827: mkdir d19/d1a/d20/d23/dca/d71/dc7/de1/d115 0 2024-09-16T00:09:49.328 INFO:tasks.workunit.client.0.smithi139.stdout:6/516: dwrite d8/d29/d83/d26/d28/d6d/f89 [4194304,4194304] 0 2024-09-16T00:09:49.329 INFO:tasks.workunit.client.0.smithi139.stdout:9/828: fdatasync fb 0 2024-09-16T00:09:49.330 INFO:tasks.workunit.client.0.smithi139.stdout:0/659: link d6/d1c/d41/d44/dd/f6e d6/d1c/d41/d44/dd/fd0 0 2024-09-16T00:09:49.333 INFO:tasks.workunit.client.0.smithi139.stdout:2/829: dwrite d11/d30/d110/f59 [0,4194304] 0 2024-09-16T00:09:49.335 INFO:tasks.workunit.client.0.smithi139.stdout:0/660: dread - d6/d1c/d41/f96 zero size 2024-09-16T00:09:49.338 INFO:tasks.workunit.client.0.smithi139.stdout:6/517: read d8/d29/d83/d26/f54 [207487,39773] 0 2024-09-16T00:09:49.338 INFO:tasks.workunit.client.0.smithi139.stdout:2/830: stat d11/d30/d110/d5d/d73/fa3 0 2024-09-16T00:09:49.338 INFO:tasks.workunit.client.0.smithi139.stdout:8/760: dwrite d17/ff8 [0,4194304] 0 2024-09-16T00:09:49.341 INFO:tasks.workunit.client.0.smithi139.stdout:9/829: creat d19/d1a/d20/d23/d2d/d35/d42/f116 x:0 0 0 2024-09-16T00:09:49.342 INFO:tasks.workunit.client.0.smithi139.stdout:2/831: write d11/d30/d110/f59 [3824581,85016] 0 2024-09-16T00:09:49.345 INFO:tasks.workunit.client.0.smithi139.stdout:2/832: write d11/f38 [497919,79332] 0 2024-09-16T00:09:49.345 INFO:tasks.workunit.client.0.smithi139.stdout:2/833: chown d11/d2f/f61 5 1 2024-09-16T00:09:49.351 INFO:tasks.workunit.client.0.smithi139.stdout:9/830: link d19/d1a/d20/d23/d2d/d35/d42/d3a/fe6 d19/d1a/d20/d23/dca/db5/d10d/f117 0 2024-09-16T00:09:49.352 INFO:tasks.workunit.client.0.smithi139.stdout:0/661: write d6/d1c/d41/d44/d58/f5f [430018,71706] 0 2024-09-16T00:09:49.353 INFO:tasks.workunit.client.0.smithi139.stdout:8/761: truncate d17/d54/f69 3284727 0 2024-09-16T00:09:49.353 INFO:tasks.workunit.client.0.smithi139.stdout:9/831: write d19/d1a/f8f [1057415,19493] 0 2024-09-16T00:09:49.353 INFO:tasks.workunit.client.0.smithi139.stdout:2/834: truncate d11/f1e 4284134 0 2024-09-16T00:09:49.355 INFO:tasks.workunit.client.0.smithi139.stdout:0/662: read d6/d1c/d41/d44/dd/fd0 [438874,37906] 0 2024-09-16T00:09:49.355 INFO:tasks.workunit.client.0.smithi139.stdout:8/762: dread - d17/d1f/d3a/db7/d40/ff9 zero size 2024-09-16T00:09:49.356 INFO:tasks.workunit.client.0.smithi139.stdout:9/832: chown d19/d1a/d20/d23/dca/d71/lbf 77 1 2024-09-16T00:09:49.357 INFO:tasks.workunit.client.0.smithi139.stdout:8/763: write d17/d1a/fa5 [1616646,15681] 0 2024-09-16T00:09:49.358 INFO:tasks.workunit.client.0.smithi139.stdout:0/663: creat d6/fd1 x:0 0 0 2024-09-16T00:09:49.361 INFO:tasks.workunit.client.0.smithi139.stdout:8/764: rename d17/d1a/fbc to d17/d1f/d88/f100 0 2024-09-16T00:09:49.362 INFO:tasks.workunit.client.0.smithi139.stdout:9/833: truncate d19/d1a/d20/d23/d2d/d35/d42/fea 4865286 0 2024-09-16T00:09:49.363 INFO:tasks.workunit.client.0.smithi139.stdout:0/664: creat d6/d1c/d41/d7c/dad/fd2 x:0 0 0 2024-09-16T00:09:49.363 INFO:tasks.workunit.client.0.smithi139.stdout:8/765: mkdir d17/d1a/d50/d86/d9d/d9f/d101 0 2024-09-16T00:09:49.366 INFO:tasks.workunit.client.0.smithi139.stdout:9/834: mknod d19/d1a/d20/d89/d93/c118 0 2024-09-16T00:09:49.367 INFO:tasks.workunit.client.0.smithi139.stdout:2/835: dread d11/d30/d110/d29/d2b/f65 [0,4194304] 0 2024-09-16T00:09:49.367 INFO:tasks.workunit.client.0.smithi139.stdout:8/766: unlink d17/d54/lc1 0 2024-09-16T00:09:49.367 INFO:tasks.workunit.client.0.smithi139.stdout:0/665: getdents d6/d1c/d41/d44/dd/d56/dbd 0 2024-09-16T00:09:49.368 INFO:tasks.workunit.client.0.smithi139.stdout:0/666: chown d6/d42/c8f 62 1 2024-09-16T00:09:49.370 INFO:tasks.workunit.client.0.smithi139.stdout:8/767: truncate d17/d1f/d3a/f6f 752551 0 2024-09-16T00:09:49.371 INFO:tasks.workunit.client.0.smithi139.stdout:8/768: dread - d17/d89/ff7 zero size 2024-09-16T00:09:49.371 INFO:tasks.workunit.client.0.smithi139.stdout:8/769: dread - d17/d1f/d3a/db7/d40/d4f/fdc zero size 2024-09-16T00:09:49.371 INFO:tasks.workunit.client.0.smithi139.stdout:2/836: rename d11/d30/d110/f59 to d11/d30/d110/d29/d86/df3/f11f 0 2024-09-16T00:09:49.372 INFO:tasks.workunit.client.0.smithi139.stdout:0/667: mknod d6/d1c/d41/d63/d82/cd3 0 2024-09-16T00:09:49.377 INFO:tasks.workunit.client.0.smithi139.stdout:2/837: creat d11/d30/d110/d50/f120 x:0 0 0 2024-09-16T00:09:49.377 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:49.376+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:49.377 INFO:tasks.workunit.client.0.smithi139.stdout:0/668: write d6/d1c/d41/d44/f3b [65756,71817] 0 2024-09-16T00:09:49.380 INFO:tasks.workunit.client.0.smithi139.stdout:0/669: write d6/d1c/d41/d44/d58/f68 [337410,58111] 0 2024-09-16T00:09:49.381 INFO:tasks.workunit.client.0.smithi139.stdout:0/670: read - d6/d1c/d41/d63/d82/f99 zero size 2024-09-16T00:09:49.387 INFO:tasks.workunit.client.0.smithi139.stdout:8/770: dread d17/d1a/d57/f8b [0,4194304] 0 2024-09-16T00:09:49.387 INFO:tasks.workunit.client.0.smithi139.stdout:9/835: dwrite d19/d1a/d20/d23/d2d/d35/d44/f114 [0,4194304] 0 2024-09-16T00:09:49.388 INFO:tasks.workunit.client.0.smithi139.stdout:0/671: write d6/d1c/f8d [1541448,69951] 0 2024-09-16T00:09:49.390 INFO:tasks.workunit.client.0.smithi139.stdout:9/836: fdatasync d19/d1a/d20/d23/d2d/d9a/faa 0 2024-09-16T00:09:49.391 INFO:tasks.workunit.client.0.smithi139.stdout:0/672: write d6/d1c/d41/d7c/dad/fd2 [1031393,32170] 0 2024-09-16T00:09:49.395 INFO:tasks.workunit.client.0.smithi139.stdout:8/771: rename d17/d1f/d3a/db7/d40/d4f/l76 to d17/l102 0 2024-09-16T00:09:49.396 INFO:tasks.workunit.client.0.smithi139.stdout:0/673: mknod d6/d1c/d41/d44/dd/d8a/cd4 0 2024-09-16T00:09:49.397 INFO:tasks.workunit.client.0.smithi139.stdout:8/772: truncate d17/f83 1378605 0 2024-09-16T00:09:49.399 INFO:tasks.workunit.client.0.smithi139.stdout:0/674: creat d6/d1c/d41/d44/d58/fd5 x:0 0 0 2024-09-16T00:09:49.400 INFO:tasks.workunit.client.0.smithi139.stdout:9/837: dwrite d19/d1a/d20/d23/d2d/d35/d42/d3a/f48 [0,4194304] 0 2024-09-16T00:09:49.406 INFO:tasks.workunit.client.0.smithi139.stdout:0/675: unlink d6/d1c/d41/d76/ca9 0 2024-09-16T00:09:49.407 INFO:tasks.workunit.client.0.smithi139.stdout:8/773: dwrite d17/d1a/d2d/ff3 [0,4194304] 0 2024-09-16T00:09:49.408 INFO:tasks.workunit.client.0.smithi139.stdout:2/838: dread d11/f1e [0,4194304] 0 2024-09-16T00:09:49.409 INFO:tasks.workunit.client.0.smithi139.stdout:8/774: read - d17/d1a/d50/d86/d9d/d9f/fd7 zero size 2024-09-16T00:09:49.410 INFO:tasks.workunit.client.0.smithi139.stdout:0/676: creat d6/d1c/d41/d7c/d86/fd6 x:0 0 0 2024-09-16T00:09:49.412 INFO:tasks.workunit.client.0.smithi139.stdout:9/838: dread d19/d1a/d20/d23/d2d/d35/d42/d3a/f48 [0,4194304] 0 2024-09-16T00:09:49.412 INFO:tasks.workunit.client.0.smithi139.stdout:2/839: chown d11/d30/d110/d29/d4f/d9c/fb3 1409768802 1 2024-09-16T00:09:49.413 INFO:tasks.workunit.client.0.smithi139.stdout:2/840: fdatasync d11/d30/d110/d29/da5/fbe 0 2024-09-16T00:09:49.414 INFO:tasks.workunit.client.0.smithi139.stdout:8/775: creat d17/d1f/d3a/db0/f103 x:0 0 0 2024-09-16T00:09:49.416 INFO:tasks.workunit.client.0.smithi139.stdout:2/841: write d11/d30/d110/d29/d2b/f2d [1197577,87404] 0 2024-09-16T00:09:49.416 INFO:tasks.workunit.client.0.smithi139.stdout:8/776: unlink d17/d1f/fd9 0 2024-09-16T00:09:49.418 INFO:tasks.workunit.client.0.smithi139.stdout:9/839: getdents d19/d1a/d20/d23/dca/d9f 0 2024-09-16T00:09:49.418 INFO:tasks.workunit.client.0.smithi139.stdout:2/842: chown d11/d30/d110/d29/da5/d81 1815 1 2024-09-16T00:09:49.418 INFO:tasks.workunit.client.0.smithi139.stdout:2/843: chown ff 108 1 2024-09-16T00:09:49.418 INFO:tasks.workunit.client.0.smithi139.stdout:0/677: link d6/d1c/d41/d44/d58/f5f d6/d5a/fd7 0 2024-09-16T00:09:49.419 INFO:tasks.workunit.client.0.smithi139.stdout:8/777: creat d17/d1f/d3a/db7/d40/f104 x:0 0 0 2024-09-16T00:09:49.420 INFO:tasks.workunit.client.0.smithi139.stdout:9/840: mknod d19/d1a/d20/d89/d93/c119 0 2024-09-16T00:09:49.420 INFO:tasks.workunit.client.0.smithi139.stdout:0/678: truncate d6/f64 1020300 0 2024-09-16T00:09:49.421 INFO:tasks.workunit.client.0.smithi139.stdout:2/844: mknod d11/d30/d110/d5d/d73/d111/c121 0 2024-09-16T00:09:49.421 INFO:tasks.workunit.client.0.smithi139.stdout:0/679: dread - d6/d1c/d41/d7c/d86/fd6 zero size 2024-09-16T00:09:49.422 INFO:tasks.workunit.client.0.smithi139.stdout:0/680: dread - d6/d42/db4/f87 zero size 2024-09-16T00:09:49.423 INFO:tasks.workunit.client.0.smithi139.stdout:9/841: mkdir d19/d1a/d20/d23/d2d/d35/d44/df4/d11a 0 2024-09-16T00:09:49.423 INFO:tasks.workunit.client.0.smithi139.stdout:2/845: rmdir d11/d2f/dcc/dea 39 2024-09-16T00:09:49.424 INFO:tasks.workunit.client.0.smithi139.stdout:8/778: getdents d17/d1a/d50/d86/d92/dd6 0 2024-09-16T00:09:49.425 INFO:tasks.workunit.client.0.smithi139.stdout:9/842: write d19/d1a/d20/d23/d2d/d2f/f96 [177240,125220] 0 2024-09-16T00:09:49.425 INFO:tasks.workunit.client.0.smithi139.stdout:0/681: creat d6/d1c/d41/d44/dce/fd8 x:0 0 0 2024-09-16T00:09:49.426 INFO:tasks.workunit.client.0.smithi139.stdout:8/779: mknod d17/d1f/d3a/db7/c105 0 2024-09-16T00:09:49.427 INFO:tasks.workunit.client.0.smithi139.stdout:2/846: link f2 d11/d30/d110/d29/d86/df3/f122 0 2024-09-16T00:09:49.428 INFO:tasks.workunit.client.0.smithi139.stdout:8/780: stat d17/d1f/d3a/db7/f3b 0 2024-09-16T00:09:49.430 INFO:tasks.workunit.client.0.smithi139.stdout:9/843: creat d19/d1a/d20/d23/d2d/d35/d42/d3a/dcb/f11b x:0 0 0 2024-09-16T00:09:49.433 INFO:tasks.workunit.client.0.smithi139.stdout:0/682: dwrite d6/d1c/d41/d44/f3e [0,4194304] 0 2024-09-16T00:09:49.434 INFO:tasks.workunit.client.0.smithi139.stdout:9/844: stat d19/d1a/d20/d23/dca/d71/d32/d79/c104 0 2024-09-16T00:09:49.434 INFO:tasks.workunit.client.0.smithi139.stdout:8/781: dread - d17/d1f/d3a/db7/d40/ff0 zero size 2024-09-16T00:09:49.436 INFO:tasks.workunit.client.0.smithi139.stdout:2/847: rmdir d11/d30/dbf/dfe/d118 0 2024-09-16T00:09:49.441 INFO:tasks.workunit.client.0.smithi139.stdout:7/740: sync 2024-09-16T00:09:49.441 INFO:tasks.workunit.client.0.smithi139.stdout:4/682: sync 2024-09-16T00:09:49.441 INFO:tasks.workunit.client.0.smithi139.stdout:3/590: sync 2024-09-16T00:09:49.441 INFO:tasks.workunit.client.0.smithi139.stdout:0/683: dwrite d6/f52 [0,4194304] 0 2024-09-16T00:09:49.442 INFO:tasks.workunit.client.0.smithi139.stdout:2/848: creat d11/d30/d110/d29/d86/f123 x:0 0 0 2024-09-16T00:09:49.447 INFO:tasks.workunit.client.0.smithi139.stdout:2/849: rmdir d11/d45 39 2024-09-16T00:09:49.447 INFO:tasks.workunit.client.0.smithi139.stdout:0/684: dread - d6/d1c/d41/d7c/fb5 zero size 2024-09-16T00:09:49.447 INFO:tasks.workunit.client.0.smithi139.stdout:9/845: dwrite d19/d1a/d20/d23/dca/d71/d32/f10f [0,4194304] 0 2024-09-16T00:09:49.448 INFO:tasks.workunit.client.0.smithi139.stdout:4/683: mknod d12/d47/d49/d60/cda 0 2024-09-16T00:09:49.451 INFO:tasks.workunit.client.0.smithi139.stdout:7/741: link d5/d26/d42/f80 d5/d16/dc6/f10e 0 2024-09-16T00:09:49.453 INFO:tasks.workunit.client.0.smithi139.stdout:4/684: creat d12/d47/d49/d60/dac/fdb x:0 0 0 2024-09-16T00:09:49.454 INFO:tasks.workunit.client.0.smithi139.stdout:8/782: dwrite d17/d1a/d57/f8b [4194304,4194304] 0 2024-09-16T00:09:49.456 INFO:tasks.workunit.client.0.smithi139.stdout:0/685: write d6/d1c/d41/d44/dd/fa0 [987645,27972] 0 2024-09-16T00:09:49.459 INFO:tasks.workunit.client.0.smithi139.stdout:8/783: stat d17/d54 0 2024-09-16T00:09:49.460 INFO:tasks.workunit.client.0.smithi139.stdout:0/686: readlink d6/l4a 0 2024-09-16T00:09:49.460 INFO:tasks.workunit.client.0.smithi139.stdout:3/591: truncate d5/d7/d23/d27/d2f/d30/d4c/d50/f87 985027 0 2024-09-16T00:09:49.461 INFO:tasks.workunit.client.0.smithi139.stdout:3/592: readlink d5/lad 0 2024-09-16T00:09:49.464 INFO:tasks.workunit.client.0.smithi139.stdout:8/784: mknod d17/d1a/d50/c106 0 2024-09-16T00:09:49.466 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:49.466 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:49.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:49.466 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:49.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.466 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10656 3.19 3:9943069b:::100000030ce.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:49.466 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.774845+0000", 2024-09-16T00:09:49.467 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 13.688053536, 2024-09-16T00:09:49.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012894586, 2024-09-16T00:09:49.467 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.467 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.467 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.467 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.467 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.467 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.467 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10656 2024-09-16T00:09:49.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.467 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.774845+0000", 2024-09-16T00:09:49.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.774845+0000", 2024-09-16T00:09:49.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.774846+0000", 2024-09-16T00:09:49.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.469e-06 2024-09-16T00:09:49.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.776187+0000", 2024-09-16T00:09:49.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001341021 2024-09-16T00:09:49.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.776206+0000", 2024-09-16T00:09:49.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8975e-05 2024-09-16T00:09:49.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.776215+0000", 2024-09-16T00:09:49.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0499999999999997e-06 2024-09-16T00:09:49.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.776279+0000", 2024-09-16T00:09:49.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3874999999999994e-05 2024-09-16T00:09:49.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.776408+0000", 2024-09-16T00:09:49.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012841600000000001 2024-09-16T00:09:49.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:49.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.776544+0000", 2024-09-16T00:09:49.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013610900000000001 2024-09-16T00:09:49.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.784791+0000", 2024-09-16T00:09:49.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0082469290000000001 2024-09-16T00:09:49.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.787622+0000", 2024-09-16T00:09:49.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028308679999999998 2024-09-16T00:09:49.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.787684+0000", 2024-09-16T00:09:49.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2000000000000003e-05 2024-09-16T00:09:49.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.787740+0000", 2024-09-16T00:09:49.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5874000000000001e-05 2024-09-16T00:09:49.472 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.472 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.472 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.473 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10683 3.25 3:a7febee6:::100000030e3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:49.473 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.891383+0000", 2024-09-16T00:09:49.473 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 13.571515895999999, 2024-09-16T00:09:49.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010708538, 2024-09-16T00:09:49.473 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.473 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.473 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.473 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.473 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.473 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.473 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10683 2024-09-16T00:09:49.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.474 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.891383+0000", 2024-09-16T00:09:49.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.891383+0000", 2024-09-16T00:09:49.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.891385+0000", 2024-09-16T00:09:49.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4339999999999999e-06 2024-09-16T00:09:49.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.893985+0000", 2024-09-16T00:09:49.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025995860000000001 2024-09-16T00:09:49.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.894005+0000", 2024-09-16T00:09:49.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0834000000000001e-05 2024-09-16T00:09:49.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.894016+0000", 2024-09-16T00:09:49.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0584000000000001e-05 2024-09-16T00:09:49.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.894086+0000", 2024-09-16T00:09:49.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0309999999999996e-05 2024-09-16T00:09:49.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.894172+0000", 2024-09-16T00:09:49.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5593000000000005e-05 2024-09-16T00:09:49.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:49.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.894295+0000", 2024-09-16T00:09:49.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012262500000000001 2024-09-16T00:09:49.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.901229+0000", 2024-09-16T00:09:49.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069343740000000001 2024-09-16T00:09:49.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.901908+0000", 2024-09-16T00:09:49.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00067909600000000002 2024-09-16T00:09:49.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.902016+0000", 2024-09-16T00:09:49.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000107532 2024-09-16T00:09:49.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.902091+0000", 2024-09-16T00:09:49.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5569999999999996e-05 2024-09-16T00:09:49.479 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.479 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.479 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.479 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10689 3.25 3:a7febee6:::100000030e3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:49.479 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.913087+0000", 2024-09-16T00:09:49.479 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 13.549811233, 2024-09-16T00:09:49.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.019706497, 2024-09-16T00:09:49.479 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.479 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.479 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.480 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.480 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.480 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.480 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10689 2024-09-16T00:09:49.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.480 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.913087+0000", 2024-09-16T00:09:49.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.913087+0000", 2024-09-16T00:09:49.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.913089+0000", 2024-09-16T00:09:49.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1789999999999998e-06 2024-09-16T00:09:49.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.914935+0000", 2024-09-16T00:09:49.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001845469 2024-09-16T00:09:49.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.914958+0000", 2024-09-16T00:09:49.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2895e-05 2024-09-16T00:09:49.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.914967+0000", 2024-09-16T00:09:49.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5079999999999994e-06 2024-09-16T00:09:49.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.915031+0000", 2024-09-16T00:09:49.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4144000000000001e-05 2024-09-16T00:09:49.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.915096+0000", 2024-09-16T00:09:49.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4842e-05 2024-09-16T00:09:49.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:49.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.915191+0000", 2024-09-16T00:09:49.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4209999999999994e-05 2024-09-16T00:09:49.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.921193+0000", 2024-09-16T00:09:49.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0060020760000000003 2024-09-16T00:09:49.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.932664+0000", 2024-09-16T00:09:49.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011471093999999999 2024-09-16T00:09:49.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.932735+0000", 2024-09-16T00:09:49.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1660000000000002e-05 2024-09-16T00:09:49.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.486 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.486 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.932794+0000", 2024-09-16T00:09:49.486 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.842e-05 2024-09-16T00:09:49.486 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.486 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.486 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.486 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10690 3.25 3:a7febee6:::100000030e3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:49.486 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.924498+0000", 2024-09-16T00:09:49.486 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 13.538400571, 2024-09-16T00:09:49.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013019637000000001, 2024-09-16T00:09:49.487 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.487 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.487 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.487 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.487 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.487 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.487 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10690 2024-09-16T00:09:49.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.487 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.487 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.924498+0000", 2024-09-16T00:09:49.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.924498+0000", 2024-09-16T00:09:49.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.924502+0000", 2024-09-16T00:09:49.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5899999999999999e-06 2024-09-16T00:09:49.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.926464+0000", 2024-09-16T00:09:49.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019621349999999998 2024-09-16T00:09:49.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.926477+0000", 2024-09-16T00:09:49.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3104e-05 2024-09-16T00:09:49.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.926487+0000", 2024-09-16T00:09:49.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9420000000000007e-06 2024-09-16T00:09:49.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.926550+0000", 2024-09-16T00:09:49.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3584999999999995e-05 2024-09-16T00:09:49.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.926643+0000", 2024-09-16T00:09:49.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2604999999999999e-05 2024-09-16T00:09:49.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:49.491 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.926806+0000", 2024-09-16T00:09:49.491 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016308300000000001 2024-09-16T00:09:49.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.491 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.491 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.935095+0000", 2024-09-16T00:09:49.491 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0082889179999999993 2024-09-16T00:09:49.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.491 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.491 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.937429+0000", 2024-09-16T00:09:49.491 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023340589999999999 2024-09-16T00:09:49.492 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.492 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.492 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.937480+0000", 2024-09-16T00:09:49.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0594999999999999e-05 2024-09-16T00:09:49.492 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.492 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.492 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.937518+0000", 2024-09-16T00:09:49.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8021000000000001e-05 2024-09-16T00:09:49.492 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.492 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.492 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.493 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10774 3.19 3:990d5343:::10000003135.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:49.493 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.427206+0000", 2024-09-16T00:09:49.493 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 11.03569214, 2024-09-16T00:09:49.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012269067, 2024-09-16T00:09:49.493 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.493 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.493 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.493 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.493 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.493 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.493 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10774 2024-09-16T00:09:49.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.494 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.427206+0000", 2024-09-16T00:09:49.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.427206+0000", 2024-09-16T00:09:49.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.427209+0000", 2024-09-16T00:09:49.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3829999999999999e-06 2024-09-16T00:09:49.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.429890+0000", 2024-09-16T00:09:49.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026809999999999998 2024-09-16T00:09:49.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.429919+0000", 2024-09-16T00:09:49.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8778000000000001e-05 2024-09-16T00:09:49.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.496 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.496 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.429935+0000", 2024-09-16T00:09:49.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6739999999999999e-05 2024-09-16T00:09:49.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.496 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.496 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.430016+0000", 2024-09-16T00:09:49.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0766000000000006e-05 2024-09-16T00:09:49.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.497 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.430167+0000", 2024-09-16T00:09:49.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015089500000000001 2024-09-16T00:09:49.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:49.497 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.430272+0000", 2024-09-16T00:09:49.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000105447 2024-09-16T00:09:49.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.497 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.435798+0000", 2024-09-16T00:09:49.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055258140000000004 2024-09-16T00:09:49.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.498 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.498 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.498 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.439372+0000", 2024-09-16T00:09:49.498 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003573906 2024-09-16T00:09:49.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.498 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.498 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.498 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.439428+0000", 2024-09-16T00:09:49.498 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5826999999999999e-05 2024-09-16T00:09:49.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.498 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.498 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.499 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.439475+0000", 2024-09-16T00:09:49.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7511e-05 2024-09-16T00:09:49.499 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.499 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.499 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.499 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10802 3.25 3:a7febee6:::100000030e3.00000000:head [write 0~4194304 [4@4194304]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:49.499 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.658867+0000", 2024-09-16T00:09:49.499 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 10.804031573, 2024-09-16T00:09:49.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011536737, 2024-09-16T00:09:49.499 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.499 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.499 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.500 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.500 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.500 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.500 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10802 2024-09-16T00:09:49.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.500 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.500 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.500 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.658867+0000", 2024-09-16T00:09:49.500 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.501 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.658867+0000", 2024-09-16T00:09:49.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.501 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.658869+0000", 2024-09-16T00:09:49.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1940000000000001e-06 2024-09-16T00:09:49.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.501 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.660719+0000", 2024-09-16T00:09:49.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018499090000000001 2024-09-16T00:09:49.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.502 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.502 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.660732+0000", 2024-09-16T00:09:49.502 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3162999999999999e-05 2024-09-16T00:09:49.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.502 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.502 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.660742+0000", 2024-09-16T00:09:49.502 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4159999999999993e-06 2024-09-16T00:09:49.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.502 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.502 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.660828+0000", 2024-09-16T00:09:49.503 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6822000000000003e-05 2024-09-16T00:09:49.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.503 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.503 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.660938+0000", 2024-09-16T00:09:49.503 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109554 2024-09-16T00:09:49.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.503 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:49.503 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.661042+0000", 2024-09-16T00:09:49.503 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104312 2024-09-16T00:09:49.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.503 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.504 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.666676+0000", 2024-09-16T00:09:49.504 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0056336850000000003 2024-09-16T00:09:49.504 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.504 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.504 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.504 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.670301+0000", 2024-09-16T00:09:49.504 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036246170000000001 2024-09-16T00:09:49.504 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.504 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.504 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.504 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.670352+0000", 2024-09-16T00:09:49.504 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1022e-05 2024-09-16T00:09:49.504 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.505 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.505 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.670404+0000", 2024-09-16T00:09:49.505 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2043000000000003e-05 2024-09-16T00:09:49.505 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.505 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.505 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.505 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10824 3.3b 3:deb39c3c:::1000000315d.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:49.505 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.727887+0000", 2024-09-16T00:09:49.505 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 10.735011051000001, 2024-09-16T00:09:49.505 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011016697000000001, 2024-09-16T00:09:49.505 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.506 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.506 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.506 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.506 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.506 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.506 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10824 2024-09-16T00:09:49.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.506 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.506 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.506 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.506 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.727887+0000", 2024-09-16T00:09:49.506 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.507 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.507 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.727887+0000", 2024-09-16T00:09:49.507 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.507 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.507 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.507 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.727890+0000", 2024-09-16T00:09:49.507 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2340000000000001e-06 2024-09-16T00:09:49.507 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.507 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.507 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.729535+0000", 2024-09-16T00:09:49.507 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001645234 2024-09-16T00:09:49.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.508 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.508 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.729568+0000", 2024-09-16T00:09:49.508 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3188999999999997e-05 2024-09-16T00:09:49.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.508 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.508 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.729582+0000", 2024-09-16T00:09:49.508 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3583e-05 2024-09-16T00:09:49.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.509 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.509 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.729717+0000", 2024-09-16T00:09:49.509 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000135053 2024-09-16T00:09:49.509 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.509 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.509 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.509 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.729938+0000", 2024-09-16T00:09:49.509 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022084399999999999 2024-09-16T00:09:49.509 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.509 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.509 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:49.509 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.730079+0000", 2024-09-16T00:09:49.509 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000141142 2024-09-16T00:09:49.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.510 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.510 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.510 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.737071+0000", 2024-09-16T00:09:49.510 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069920750000000004 2024-09-16T00:09:49.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.510 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.510 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.510 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.738789+0000", 2024-09-16T00:09:49.510 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017177469999999999 2024-09-16T00:09:49.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.510 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.510 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.510 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.738850+0000", 2024-09-16T00:09:49.511 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1582999999999996e-05 2024-09-16T00:09:49.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.511 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.511 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.511 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.738904+0000", 2024-09-16T00:09:49.511 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4012999999999999e-05 2024-09-16T00:09:49.511 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.511 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.511 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.511 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.511 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10855 3.0 3:01e3c502:::1000000312a.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:49.511 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.921201+0000", 2024-09-16T00:09:49.512 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 10.541697039000001, 2024-09-16T00:09:49.512 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010818772000000001, 2024-09-16T00:09:49.512 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.512 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.512 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.512 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.512 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.512 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.512 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10855 2024-09-16T00:09:49.512 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.512 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.512 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.513 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.513 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.921201+0000", 2024-09-16T00:09:49.513 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.513 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.513 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.513 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.513 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.921201+0000", 2024-09-16T00:09:49.513 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.513 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.516 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.516 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.516 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.921204+0000", 2024-09-16T00:09:49.516 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1720000000000001e-06 2024-09-16T00:09:49.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.517 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.517 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.517 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.923866+0000", 2024-09-16T00:09:49.517 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026621119999999999 2024-09-16T00:09:49.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.517 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.517 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.517 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.923879+0000", 2024-09-16T00:09:49.517 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3380999999999999e-05 2024-09-16T00:09:49.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.517 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.517 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.518 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.923886+0000", 2024-09-16T00:09:49.518 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3320000000000003e-06 2024-09-16T00:09:49.518 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.518 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.518 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.518 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.923941+0000", 2024-09-16T00:09:49.518 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4536000000000001e-05 2024-09-16T00:09:49.518 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.518 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.518 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.518 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.924015+0000", 2024-09-16T00:09:49.518 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4035999999999994e-05 2024-09-16T00:09:49.518 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.519 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.519 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:49.519 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.924154+0000", 2024-09-16T00:09:49.519 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013893 2024-09-16T00:09:49.519 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.519 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.519 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.519 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.931363+0000", 2024-09-16T00:09:49.519 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0072087200000000001 2024-09-16T00:09:49.519 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.519 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.519 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.519 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.931917+0000", 2024-09-16T00:09:49.520 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000554398 2024-09-16T00:09:49.520 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.520 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.520 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.520 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.931972+0000", 2024-09-16T00:09:49.520 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4475999999999998e-05 2024-09-16T00:09:49.520 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.520 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.520 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.520 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.932020+0000", 2024-09-16T00:09:49.520 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8678999999999999e-05 2024-09-16T00:09:49.520 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.520 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.521 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.521 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.521 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.521 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11077 3.31 3:8fcbaef8:::1000000313b.00000002:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:49.521 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.923139+0000", 2024-09-16T00:09:49.521 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.5397594869999995, 2024-09-16T00:09:49.521 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011010447, 2024-09-16T00:09:49.521 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.521 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.521 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.521 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.521 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.521 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.522 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11077 2024-09-16T00:09:49.522 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.522 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.522 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.522 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.522 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.923139+0000", 2024-09-16T00:09:49.522 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.522 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.522 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.522 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.522 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.923139+0000", 2024-09-16T00:09:49.523 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.523 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.523 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.523 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.523 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.923142+0000", 2024-09-16T00:09:49.523 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6139999999999999e-06 2024-09-16T00:09:49.523 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.523 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.523 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.523 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.925489+0000", 2024-09-16T00:09:49.523 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023476479999999999 2024-09-16T00:09:49.523 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.523 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.523 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.524 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.925512+0000", 2024-09-16T00:09:49.524 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3175000000000001e-05 2024-09-16T00:09:49.524 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.524 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.524 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.524 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.925523+0000", 2024-09-16T00:09:49.524 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0428000000000001e-05 2024-09-16T00:09:49.524 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.524 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.524 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.524 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.925584+0000", 2024-09-16T00:09:49.524 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1253000000000005e-05 2024-09-16T00:09:49.524 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.525 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.525 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.525 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.925746+0000", 2024-09-16T00:09:49.525 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016229100000000001 2024-09-16T00:09:49.525 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.525 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.525 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:49.525 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.925891+0000", 2024-09-16T00:09:49.525 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000144563 2024-09-16T00:09:49.525 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.525 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.525 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.525 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.932369+0000", 2024-09-16T00:09:49.526 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0064779520000000004 2024-09-16T00:09:49.526 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.526 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.526 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.526 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.934068+0000", 2024-09-16T00:09:49.526 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016991949999999999 2024-09-16T00:09:49.526 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.526 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.526 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.526 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.934108+0000", 2024-09-16T00:09:49.526 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.994e-05 2024-09-16T00:09:49.526 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.526 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.527 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.527 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.934149+0000", 2024-09-16T00:09:49.527 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1387999999999997e-05 2024-09-16T00:09:49.527 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.527 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.527 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.527 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.527 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.527 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11182 3.19 3:9bf8f08b:::10000003201.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:49.527 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.472632+0000", 2024-09-16T00:09:49.527 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.9902661770000005, 2024-09-16T00:09:49.527 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010576481, 2024-09-16T00:09:49.527 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.528 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.528 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.528 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.528 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.528 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.528 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11182 2024-09-16T00:09:49.528 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.528 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.528 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.528 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.528 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.472632+0000", 2024-09-16T00:09:49.528 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.528 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.529 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.529 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.529 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.472632+0000", 2024-09-16T00:09:49.529 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.529 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.529 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.529 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.529 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.472635+0000", 2024-09-16T00:09:49.529 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4269999999999998e-06 2024-09-16T00:09:49.529 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.529 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.529 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.529 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.475362+0000", 2024-09-16T00:09:49.530 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002727377 2024-09-16T00:09:49.530 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.530 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.530 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.530 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.475393+0000", 2024-09-16T00:09:49.530 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0409e-05 2024-09-16T00:09:49.530 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.530 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.530 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.530 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.475407+0000", 2024-09-16T00:09:49.530 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4749000000000001e-05 2024-09-16T00:09:49.530 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.530 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.531 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.531 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.475489+0000", 2024-09-16T00:09:49.531 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1299999999999997e-05 2024-09-16T00:09:49.531 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.531 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.531 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.531 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.475566+0000", 2024-09-16T00:09:49.531 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7721999999999999e-05 2024-09-16T00:09:49.531 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.531 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.531 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:49.531 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.475741+0000", 2024-09-16T00:09:49.531 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017492899999999999 2024-09-16T00:09:49.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.532 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.532 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.532 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.482904+0000", 2024-09-16T00:09:49.532 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0071631179999999996 2024-09-16T00:09:49.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.532 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.532 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.532 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.483145+0000", 2024-09-16T00:09:49.532 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00024059799999999999 2024-09-16T00:09:49.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.532 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.532 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.483178+0000", 2024-09-16T00:09:49.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3447e-05 2024-09-16T00:09:49.533 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.533 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.533 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.483209+0000", 2024-09-16T00:09:49.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0405000000000001e-05 2024-09-16T00:09:49.533 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.533 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.534 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.534 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.534 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11938 3.3e 3:7fb917e8:::100000030b7.00000001:head [write 0~4063232 [2@0]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:49.534 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.718248+0000", 2024-09-16T00:09:49.534 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.7446502150000001, 2024-09-16T00:09:49.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011465517999999999, 2024-09-16T00:09:49.534 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.534 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.534 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.534 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.534 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.534 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.535 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11938 2024-09-16T00:09:49.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.535 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.535 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.535 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.535 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.718248+0000", 2024-09-16T00:09:49.535 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.535 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.535 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.535 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.718248+0000", 2024-09-16T00:09:49.535 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.536 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.536 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.536 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.718251+0000", 2024-09-16T00:09:49.536 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8509999999999999e-06 2024-09-16T00:09:49.536 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.536 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.536 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.536 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.719886+0000", 2024-09-16T00:09:49.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001634967 2024-09-16T00:09:49.537 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.537 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.537 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.537 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.719916+0000", 2024-09-16T00:09:49.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0258000000000001e-05 2024-09-16T00:09:49.537 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.537 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.537 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.537 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.719930+0000", 2024-09-16T00:09:49.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3379e-05 2024-09-16T00:09:49.537 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.537 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.538 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.538 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.720023+0000", 2024-09-16T00:09:49.538 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3121000000000005e-05 2024-09-16T00:09:49.538 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.538 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.538 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.538 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.720200+0000", 2024-09-16T00:09:49.538 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017721600000000001 2024-09-16T00:09:49.538 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.538 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.538 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:49.538 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.720361+0000", 2024-09-16T00:09:49.538 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016081900000000001 2024-09-16T00:09:49.538 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.539 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.539 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.539 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.727099+0000", 2024-09-16T00:09:49.539 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0067381419999999999 2024-09-16T00:09:49.539 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.539 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.539 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.539 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.729630+0000", 2024-09-16T00:09:49.539 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025311890000000001 2024-09-16T00:09:49.539 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.539 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.539 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.539 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.729667+0000", 2024-09-16T00:09:49.540 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6300000000000001e-05 2024-09-16T00:09:49.540 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.540 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.540 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.540 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.729714+0000", 2024-09-16T00:09:49.540 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7275999999999999e-05 2024-09-16T00:09:49.540 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.540 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.540 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.540 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.540 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.540 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11953 3.1a e87/20 3:580e2503:::1000000311b.00000000:head v 87'185, mlcod=86'183)", 2024-09-16T00:09:49.540 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.771482+0000", 2024-09-16T00:09:49.541 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.6914163599999998, 2024-09-16T00:09:49.541 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011212730000000001, 2024-09-16T00:09:49.541 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.541 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.541 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.541 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.541 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.541 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.541 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.771482+0000", 2024-09-16T00:09:49.541 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.541 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.541 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.541 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.541 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.771482+0000", 2024-09-16T00:09:49.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.542 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.542 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.771486+0000", 2024-09-16T00:09:49.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6820000000000001e-06 2024-09-16T00:09:49.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.542 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.542 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.777946+0000", 2024-09-16T00:09:49.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0064603530000000003 2024-09-16T00:09:49.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.542 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.543 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.777975+0000", 2024-09-16T00:09:49.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8580000000000001e-05 2024-09-16T00:09:49.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.543 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.543 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.777995+0000", 2024-09-16T00:09:49.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0136000000000002e-05 2024-09-16T00:09:49.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.543 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.543 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.778100+0000", 2024-09-16T00:09:49.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104893 2024-09-16T00:09:49.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.778135+0000", 2024-09-16T00:09:49.544 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5293000000000002e-05 2024-09-16T00:09:49.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.782643+0000", 2024-09-16T00:09:49.544 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045082059999999998 2024-09-16T00:09:49.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.782695+0000", 2024-09-16T00:09:49.545 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1586999999999999e-05 2024-09-16T00:09:49.545 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.545 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.545 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.545 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.545 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12173 3.31 3:8f5d6475:::10000003388.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:49.545 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.551110+0000", 2024-09-16T00:09:49.545 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.9117884619999996, 2024-09-16T00:09:49.545 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010486555, 2024-09-16T00:09:49.545 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.545 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.545 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.545 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.546 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.546 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.546 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12173 2024-09-16T00:09:49.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.546 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.546 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.546 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.551110+0000", 2024-09-16T00:09:49.546 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.546 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.546 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.551110+0000", 2024-09-16T00:09:49.547 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.547 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.547 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.547 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.547 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.551111+0000", 2024-09-16T00:09:49.547 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1960000000000001e-06 2024-09-16T00:09:49.547 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.547 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.547 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.547 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.552498+0000", 2024-09-16T00:09:49.547 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00138627 2024-09-16T00:09:49.547 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.547 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.547 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.548 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.552517+0000", 2024-09-16T00:09:49.548 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9573999999999999e-05 2024-09-16T00:09:49.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.548 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.548 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.552526+0000", 2024-09-16T00:09:49.548 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2019999999999993e-06 2024-09-16T00:09:49.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.548 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.548 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.552596+0000", 2024-09-16T00:09:49.548 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9795999999999994e-05 2024-09-16T00:09:49.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.549 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.552692+0000", 2024-09-16T00:09:49.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5480000000000001e-05 2024-09-16T00:09:49.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:49.549 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.552849+0000", 2024-09-16T00:09:49.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015705299999999999 2024-09-16T00:09:49.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.549 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.556444+0000", 2024-09-16T00:09:49.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035955010000000001 2024-09-16T00:09:49.550 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.550 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.561461+0000", 2024-09-16T00:09:49.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050172070000000001 2024-09-16T00:09:49.550 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.550 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.561523+0000", 2024-09-16T00:09:49.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1441999999999999e-05 2024-09-16T00:09:49.550 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.550 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.551 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.561597+0000", 2024-09-16T00:09:49.551 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3833999999999998e-05 2024-09-16T00:09:49.551 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.551 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.551 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.551 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.551 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:12245 3.1a e87/20 3:599a1cd1:::10000003252.00000000:head v 87'188, mlcod=87'186)", 2024-09-16T00:09:49.551 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.786850+0000", 2024-09-16T00:09:49.551 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.676048465, 2024-09-16T00:09:49.551 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01255644, 2024-09-16T00:09:49.551 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.551 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.551 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.552 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.552 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.552 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.552 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.786850+0000", 2024-09-16T00:09:49.552 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.552 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.552 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.552 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.552 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.786850+0000", 2024-09-16T00:09:49.552 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.552 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.552 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.552 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.786852+0000", 2024-09-16T00:09:49.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9659999999999999e-06 2024-09-16T00:09:49.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.553 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.791923+0000", 2024-09-16T00:09:49.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050705460000000004 2024-09-16T00:09:49.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.553 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.791948+0000", 2024-09-16T00:09:49.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5860999999999999e-05 2024-09-16T00:09:49.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.554 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.791964+0000", 2024-09-16T00:09:49.554 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5131e-05 2024-09-16T00:09:49.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.554 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.554 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.792054+0000", 2024-09-16T00:09:49.554 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0694999999999998e-05 2024-09-16T00:09:49.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.554 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.554 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.792085+0000", 2024-09-16T00:09:49.554 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0846999999999998e-05 2024-09-16T00:09:49.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.555 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.555 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.555 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.799315+0000", 2024-09-16T00:09:49.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0072297780000000001 2024-09-16T00:09:49.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.555 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.555 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.555 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.799406+0000", 2024-09-16T00:09:49.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1615999999999998e-05 2024-09-16T00:09:49.555 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.555 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.555 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.556 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.556 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12248 3.3b 3:de8bdadd:::100000033c7.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:49.556 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.787748+0000", 2024-09-16T00:09:49.556 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.6751509340000004, 2024-09-16T00:09:49.556 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011787646000000001, 2024-09-16T00:09:49.556 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.556 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.556 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.556 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.556 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.556 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.556 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12248 2024-09-16T00:09:49.556 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.557 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.787748+0000", 2024-09-16T00:09:49.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.787748+0000", 2024-09-16T00:09:49.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.787755+0000", 2024-09-16T00:09:49.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7049999999999999e-06 2024-09-16T00:09:49.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.558 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.789668+0000", 2024-09-16T00:09:49.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019122830000000001 2024-09-16T00:09:49.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.558 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.789692+0000", 2024-09-16T00:09:49.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4397999999999999e-05 2024-09-16T00:09:49.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.789705+0000", 2024-09-16T00:09:49.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3379e-05 2024-09-16T00:09:49.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.559 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.789788+0000", 2024-09-16T00:09:49.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2674000000000002e-05 2024-09-16T00:09:49.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.559 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.789929+0000", 2024-09-16T00:09:49.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000140638 2024-09-16T00:09:49.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:49.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.790077+0000", 2024-09-16T00:09:49.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014853199999999999 2024-09-16T00:09:49.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.798739+0000", 2024-09-16T00:09:49.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0086621660000000007 2024-09-16T00:09:49.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.799426+0000", 2024-09-16T00:09:49.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00068666699999999996 2024-09-16T00:09:49.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.561 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.799479+0000", 2024-09-16T00:09:49.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2784999999999997e-05 2024-09-16T00:09:49.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.561 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.799535+0000", 2024-09-16T00:09:49.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6419000000000003e-05 2024-09-16T00:09:49.561 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.561 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.562 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.562 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.562 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.562 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12269 3.3b 3:dc3ad4b0:::10000003125.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:49.562 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.844085+0000", 2024-09-16T00:09:49.562 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.6188133330000003, 2024-09-16T00:09:49.562 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011272122000000001, 2024-09-16T00:09:49.562 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.562 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.562 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.562 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.562 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.562 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.563 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12269 2024-09-16T00:09:49.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.563 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.563 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.563 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.844085+0000", 2024-09-16T00:09:49.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.563 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.563 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.844085+0000", 2024-09-16T00:09:49.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.844088+0000", 2024-09-16T00:09:49.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7769999999999998e-06 2024-09-16T00:09:49.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.564 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.846085+0000", 2024-09-16T00:09:49.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019974979999999999 2024-09-16T00:09:49.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.564 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.846124+0000", 2024-09-16T00:09:49.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8049000000000001e-05 2024-09-16T00:09:49.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.565 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.565 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.565 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.846141+0000", 2024-09-16T00:09:49.565 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7343e-05 2024-09-16T00:09:49.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.565 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.565 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.565 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.846235+0000", 2024-09-16T00:09:49.565 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4219e-05 2024-09-16T00:09:49.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.565 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.565 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.566 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.846349+0000", 2024-09-16T00:09:49.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113986 2024-09-16T00:09:49.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.566 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:49.566 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.846505+0000", 2024-09-16T00:09:49.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015587700000000001 2024-09-16T00:09:49.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.566 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.566 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.854748+0000", 2024-09-16T00:09:49.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0082434489999999999 2024-09-16T00:09:49.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.855238+0000", 2024-09-16T00:09:49.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000489808 2024-09-16T00:09:49.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.855305+0000", 2024-09-16T00:09:49.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6334000000000006e-05 2024-09-16T00:09:49.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.855357+0000", 2024-09-16T00:09:49.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2781999999999997e-05 2024-09-16T00:09:49.568 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.568 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.568 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.568 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12334 3.19 3:9a91b182:::100000031d2.00000001:head [write 0~4194304 [2@0]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:49.568 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.021880+0000", 2024-09-16T00:09:49.568 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.4410182850000002, 2024-09-16T00:09:49.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011231692, 2024-09-16T00:09:49.568 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.568 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.568 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.568 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.569 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.569 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.569 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12334 2024-09-16T00:09:49.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.569 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.021880+0000", 2024-09-16T00:09:49.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.021880+0000", 2024-09-16T00:09:49.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.021883+0000", 2024-09-16T00:09:49.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1829999999999998e-06 2024-09-16T00:09:49.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.024208+0000", 2024-09-16T00:09:49.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023246880000000001 2024-09-16T00:09:49.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.571 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.571 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.024249+0000", 2024-09-16T00:09:49.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1328999999999997e-05 2024-09-16T00:09:49.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.571 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.571 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.024270+0000", 2024-09-16T00:09:49.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0295000000000001e-05 2024-09-16T00:09:49.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.571 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.571 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.024363+0000", 2024-09-16T00:09:49.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3026e-05 2024-09-16T00:09:49.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.024469+0000", 2024-09-16T00:09:49.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010646599999999999 2024-09-16T00:09:49.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:49.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.024626+0000", 2024-09-16T00:09:49.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000156465 2024-09-16T00:09:49.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.573 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.573 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.030328+0000", 2024-09-16T00:09:49.573 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057018599999999996 2024-09-16T00:09:49.573 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.573 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.573 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.573 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.033005+0000", 2024-09-16T00:09:49.573 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026773159999999999 2024-09-16T00:09:49.573 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.573 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.573 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.573 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.033063+0000", 2024-09-16T00:09:49.573 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7911000000000002e-05 2024-09-16T00:09:49.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.574 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.033112+0000", 2024-09-16T00:09:49.574 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9153000000000002e-05 2024-09-16T00:09:49.574 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.574 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.574 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.574 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13316 3.25 3:a7ae0919:::100000033a3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:49.574 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.918020+0000", 2024-09-16T00:09:49.574 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.5448786239999999, 2024-09-16T00:09:49.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012689604, 2024-09-16T00:09:49.575 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.575 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.575 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.575 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.575 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.575 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.575 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13316 2024-09-16T00:09:49.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.575 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918020+0000", 2024-09-16T00:09:49.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.576 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.576 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.576 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918020+0000", 2024-09-16T00:09:49.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.576 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.576 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.576 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918023+0000", 2024-09-16T00:09:49.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0730000000000001e-06 2024-09-16T00:09:49.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.576 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920995+0000", 2024-09-16T00:09:49.577 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029723079999999999 2024-09-16T00:09:49.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.921016+0000", 2024-09-16T00:09:49.577 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1219000000000001e-05 2024-09-16T00:09:49.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.921033+0000", 2024-09-16T00:09:49.577 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6359999999999999e-05 2024-09-16T00:09:49.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.921229+0000", 2024-09-16T00:09:49.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019620600000000001 2024-09-16T00:09:49.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.921312+0000", 2024-09-16T00:09:49.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2718000000000003e-05 2024-09-16T00:09:49.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:49.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.921417+0000", 2024-09-16T00:09:49.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000105089 2024-09-16T00:09:49.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929440+0000", 2024-09-16T00:09:49.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0080232789999999995 2024-09-16T00:09:49.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.930603+0000", 2024-09-16T00:09:49.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001162426 2024-09-16T00:09:49.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.930659+0000", 2024-09-16T00:09:49.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.647e-05 2024-09-16T00:09:49.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.930709+0000", 2024-09-16T00:09:49.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0455999999999999e-05 2024-09-16T00:09:49.580 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.580 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.580 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.581 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13335 3.19 3:990d5343:::10000003135.00000000:head [write 0~4194304 [4@944224]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:49.581 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.974375+0000", 2024-09-16T00:09:49.581 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.488523998, 2024-09-16T00:09:49.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011540745999999999, 2024-09-16T00:09:49.581 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.581 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.581 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.581 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.581 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.581 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.581 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13335 2024-09-16T00:09:49.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.581 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.582 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.582 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.974375+0000", 2024-09-16T00:09:49.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.582 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.582 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.974375+0000", 2024-09-16T00:09:49.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.582 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.582 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.974376+0000", 2024-09-16T00:09:49.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:09:49.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.583 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.976458+0000", 2024-09-16T00:09:49.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020820830000000002 2024-09-16T00:09:49.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.583 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.976480+0000", 2024-09-16T00:09:49.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2005999999999999e-05 2024-09-16T00:09:49.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.584 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.976489+0000", 2024-09-16T00:09:49.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6889999999999993e-06 2024-09-16T00:09:49.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.584 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.584 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.976556+0000", 2024-09-16T00:09:49.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7183999999999999e-05 2024-09-16T00:09:49.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.584 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.584 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.976652+0000", 2024-09-16T00:09:49.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5838e-05 2024-09-16T00:09:49.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:49.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.976819+0000", 2024-09-16T00:09:49.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016737499999999999 2024-09-16T00:09:49.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.983064+0000", 2024-09-16T00:09:49.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0062443359999999996 2024-09-16T00:09:49.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.985782+0000", 2024-09-16T00:09:49.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027184940000000001 2024-09-16T00:09:49.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.985863+0000", 2024-09-16T00:09:49.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0496999999999999e-05 2024-09-16T00:09:49.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.985915+0000", 2024-09-16T00:09:49.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2448000000000003e-05 2024-09-16T00:09:49.586 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.586 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.586 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.587 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13436 3.0 3:016a09a9:::100000033bf.00000000:head [write 0~4194304 [2@447939]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:49.587 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.327809+0000", 2024-09-16T00:09:49.587 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.135089545, 2024-09-16T00:09:49.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010540610000000001, 2024-09-16T00:09:49.587 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:49.587 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:49.587 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:49.587 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:49.587 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:49.587 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:49.587 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13436 2024-09-16T00:09:49.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.588 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:49.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:49.588 INFO:tasks.workunit.client.0.smithi139.stdout:9/846: link d19/d1a/d20/d23/d2d/fa4 d19/d1a/d20/d23/d2d/d2f/f11c 0 2024-09-16T00:09:49.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.327809+0000", 2024-09-16T00:09:49.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:49.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.327809+0000", 2024-09-16T00:09:49.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:49.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:49.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.327811+0000", 2024-09-16T00:09:49.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.306e-06 2024-09-16T00:09:49.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.590 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:49.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.329879+0000", 2024-09-16T00:09:49.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020679499999999998 2024-09-16T00:09:49.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.590 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:49.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.329903+0000", 2024-09-16T00:09:49.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3283000000000001e-05 2024-09-16T00:09:49.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.590 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:49.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.329948+0000", 2024-09-16T00:09:49.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5738000000000002e-05 2024-09-16T00:09:49.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:49.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.330034+0000", 2024-09-16T00:09:49.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5290999999999994e-05 2024-09-16T00:09:49.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:49.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.330128+0000", 2024-09-16T00:09:49.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4863000000000003e-05 2024-09-16T00:09:49.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:49.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.330282+0000", 2024-09-16T00:09:49.592 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015395 2024-09-16T00:09:49.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.592 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:49.592 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.334785+0000", 2024-09-16T00:09:49.592 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045022789999999997 2024-09-16T00:09:49.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.592 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:49.592 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.338238+0000", 2024-09-16T00:09:49.592 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034532479999999999 2024-09-16T00:09:49.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.592 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:49.593 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.338298+0000", 2024-09-16T00:09:49.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0102000000000002e-05 2024-09-16T00:09:49.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:49.593 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:49.593 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:49.593 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.338350+0000", 2024-09-16T00:09:49.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1600000000000001e-05 2024-09-16T00:09:49.593 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.593 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.593 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.593 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:49.593 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:49.593 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:49.595 INFO:tasks.workunit.client.0.smithi139.stdout:0/687: creat d6/d1c/d41/d78/fd9 x:0 0 0 2024-09-16T00:09:49.595 INFO:tasks.workunit.client.0.smithi139.stdout:8/785: rmdir d17/d1a/d50/d86 39 2024-09-16T00:09:49.595 INFO:tasks.workunit.client.0.smithi139.stdout:2/850: dwrite d11/d30/d110/d29/da5/d81/f99 [0,4194304] 0 2024-09-16T00:09:49.595 INFO:tasks.workunit.client.0.smithi139.stdout:8/786: chown d17/d54/fbe 1547723396 1 2024-09-16T00:09:49.595 INFO:tasks.workunit.client.0.smithi139.stdout:4/685: truncate d12/d2b/d8c/fd7 3575100 0 2024-09-16T00:09:49.595 INFO:tasks.workunit.client.0.smithi139.stdout:3/593: rename d5/d7/dc/d84/c89 to d5/d7/d23/d27/d2f/d30/d4c/d6d/db3/cd2 0 2024-09-16T00:09:49.595 INFO:tasks.workunit.client.0.smithi139.stdout:9/847: truncate f14 2850378 0 2024-09-16T00:09:49.595 INFO:tasks.workunit.client.0.smithi139.stdout:7/742: dread d5/d26/d42/fa5 [0,4194304] 0 2024-09-16T00:09:49.595 INFO:tasks.workunit.client.0.smithi139.stdout:2/851: dread d11/d2f/d6e/f84 [0,4194304] 0 2024-09-16T00:09:49.595 INFO:tasks.workunit.client.0.smithi139.stdout:8/787: unlink d17/d1f/d3a/db7/d40/d4f/f65 0 2024-09-16T00:09:49.595 INFO:tasks.workunit.client.0.smithi139.stdout:3/594: creat d5/d7/d23/d27/d2f/d30/d4c/da1/fd3 x:0 0 0 2024-09-16T00:09:49.597 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:09:49.604 INFO:tasks.workunit.client.0.smithi139.stdout:0/688: dwrite d6/d1c/d41/d44/dd/d8a/d9e/fc3 [0,4194304] 0 2024-09-16T00:09:49.605 INFO:tasks.workunit.client.0.smithi139.stdout:4/686: creat d12/fdc x:0 0 0 2024-09-16T00:09:49.605 INFO:tasks.workunit.client.0.smithi139.stdout:7/743: fsync d5/d16/d6d/d7f/dc2/d81/f82 0 2024-09-16T00:09:49.605 INFO:tasks.workunit.client.0.smithi139.stdout:8/788: fsync d17/d1a/d2d/dac/fb3 0 2024-09-16T00:09:49.605 INFO:tasks.workunit.client.0.smithi139.stdout:3/595: dwrite d5/d7/d23/d27/f8d [0,4194304] 0 2024-09-16T00:09:49.605 INFO:tasks.workunit.client.0.smithi139.stdout:2/852: write d11/d45/f105 [266968,37757] 0 2024-09-16T00:09:49.605 INFO:tasks.workunit.client.0.smithi139.stdout:4/687: symlink d12/d47/db2/d4d/da9/ldd 0 2024-09-16T00:09:49.605 INFO:tasks.workunit.client.0.smithi139.stdout:3/596: chown d5/d7/d23/d27/dab 12990882 1 2024-09-16T00:09:49.605 INFO:tasks.workunit.client.0.smithi139.stdout:2/853: fsync d11/d30/d110/f27 0 2024-09-16T00:09:49.605 INFO:tasks.workunit.client.0.smithi139.stdout:8/789: dread d17/d1a/d57/f79 [0,4194304] 0 2024-09-16T00:09:49.605 INFO:tasks.workunit.client.0.smithi139.stdout:7/744: stat d5/d16/d6d/d7f/dc2/c6a 0 2024-09-16T00:09:49.605 INFO:tasks.workunit.client.0.smithi139.stdout:8/790: read - d17/d89/fbb zero size 2024-09-16T00:09:49.605 INFO:tasks.workunit.client.0.smithi139.stdout:3/597: write d5/d7/d23/d27/d2f/d30/f42 [131975,29711] 0 2024-09-16T00:09:49.605 INFO:tasks.workunit.client.0.smithi139.stdout:4/688: symlink d12/lde 0 2024-09-16T00:09:49.606 INFO:tasks.workunit.client.0.smithi139.stdout:9/848: dwrite d19/d1a/d20/d23/d2d/d35/d42/d3a/fe6 [0,4194304] 0 2024-09-16T00:09:49.606 INFO:tasks.workunit.client.0.smithi139.stdout:0/689: link d6/d1c/l1d d6/d1c/d41/d44/d6b/lda 0 2024-09-16T00:09:49.606 INFO:tasks.workunit.client.0.smithi139.stdout:0/690: readlink d6/d1c/d41/d44/dd/la8 0 2024-09-16T00:09:49.606 INFO:tasks.workunit.client.0.smithi139.stdout:3/598: read d5/d7/f78 [148306,112908] 0 2024-09-16T00:09:49.606 INFO:tasks.workunit.client.0.smithi139.stdout:2/854: dwrite f2 [0,4194304] 0 2024-09-16T00:09:49.606 INFO:tasks.workunit.client.0.smithi139.stdout:3/599: read - d5/d7/d23/d35/f46 zero size 2024-09-16T00:09:49.606 INFO:tasks.workunit.client.0.smithi139.stdout:4/689: mkdir d12/d47/db2/d4d/dd1/ddf 0 2024-09-16T00:09:49.606 INFO:tasks.workunit.client.0.smithi139.stdout:7/745: rename d5/d16/d6d/d7f/dc2/d50/d44/d55/d90/c10b to d5/d16/d22/c10f 0 2024-09-16T00:09:49.606 INFO:tasks.workunit.client.0.smithi139.stdout:8/791: rename d17/d1f/d3a/dad/fb9 to d17/d89/df2/f107 0 2024-09-16T00:09:49.606 INFO:tasks.workunit.client.0.smithi139.stdout:2/855: dwrite f2 [0,4194304] 0 2024-09-16T00:09:49.606 INFO:tasks.workunit.client.0.smithi139.stdout:9/849: dwrite d19/d1a/d20/d23/d2d/fa4 [0,4194304] 0 2024-09-16T00:09:49.606 INFO:tasks.workunit.client.0.smithi139.stdout:7/746: unlink d5/d16/dda/fe5 0 2024-09-16T00:09:49.606 INFO:tasks.workunit.client.0.smithi139.stdout:8/792: write d17/d1a/d2d/f85 [3189018,43521] 0 2024-09-16T00:09:49.606 INFO:tasks.workunit.client.0.smithi139.stdout:9/850: dwrite d19/d1a/d20/d23/d2d/d35/f66 [0,4194304] 0 2024-09-16T00:09:49.607 INFO:tasks.workunit.client.0.smithi139.stdout:4/690: truncate d12/d47/f45 1846695 0 2024-09-16T00:09:49.607 INFO:tasks.workunit.client.0.smithi139.stdout:1/571: sync 2024-09-16T00:09:49.607 INFO:tasks.workunit.client.0.smithi139.stdout:0/691: dwrite d6/d1c/d41/d44/dd/f6e [0,4194304] 0 2024-09-16T00:09:49.607 INFO:tasks.workunit.client.0.smithi139.stdout:2/856: symlink d11/d30/d110/l124 0 2024-09-16T00:09:49.607 INFO:tasks.workunit.client.0.smithi139.stdout:8/793: creat d17/d1a/d57/f108 x:0 0 0 2024-09-16T00:09:49.607 INFO:tasks.workunit.client.0.smithi139.stdout:4/691: readlink d12/d47/db2/lcc 0 2024-09-16T00:09:49.607 INFO:tasks.workunit.client.0.smithi139.stdout:3/600: dread d5/d7/d23/f17 [4194304,4194304] 0 2024-09-16T00:09:49.607 INFO:tasks.workunit.client.0.smithi139.stdout:0/692: write d6/d1c/fc1 [771305,28069] 0 2024-09-16T00:09:49.607 INFO:tasks.workunit.client.0.smithi139.stdout:8/794: stat d17/d1f/d3a/c6d 0 2024-09-16T00:09:49.607 INFO:tasks.workunit.client.0.smithi139.stdout:4/692: readlink d12/d47/db2/l6f 0 2024-09-16T00:09:49.607 INFO:tasks.workunit.client.0.smithi139.stdout:7/747: dwrite d5/d26/d46/d6b/dd6/df4/ff7 [0,4194304] 0 2024-09-16T00:09:49.607 INFO:tasks.workunit.client.0.smithi139.stdout:1/572: creat d7/d8/d1f/fa6 x:0 0 0 2024-09-16T00:09:49.607 INFO:tasks.workunit.client.0.smithi139.stdout:9/851: mknod d19/d1a/d20/d23/dca/d71/d32/d4e/c11d 0 2024-09-16T00:09:49.607 INFO:tasks.workunit.client.0.smithi139.stdout:8/795: chown d17/d1a/d50/lc4 2514269 1 2024-09-16T00:09:49.608 INFO:tasks.workunit.client.0.smithi139.stdout:1/573: fdatasync d7/d8/d40/d48/d4a/f8c 0 2024-09-16T00:09:49.608 INFO:tasks.workunit.client.0.smithi139.stdout:9/852: dread - d19/d1a/d20/d23/dca/d71/d32/fb4 zero size 2024-09-16T00:09:49.608 INFO:tasks.workunit.client.0.smithi139.stdout:3/601: mkdir d5/d7/d23/d35/dd4 0 2024-09-16T00:09:49.608 INFO:tasks.workunit.client.0.smithi139.stdout:4/693: unlink d12/d2b/f99 0 2024-09-16T00:09:49.608 INFO:tasks.workunit.client.0.smithi139.stdout:8/796: readlink d17/l102 0 2024-09-16T00:09:49.608 INFO:tasks.workunit.client.0.smithi139.stdout:0/693: dread d6/d1c/d41/d44/dd/d8a/f92 [0,4194304] 0 2024-09-16T00:09:49.608 INFO:tasks.workunit.client.0.smithi139.stdout:2/857: rename d11/d30/d110/d5d/d73/dc3/de8 to d11/d30/dbf/d116/d125 0 2024-09-16T00:09:49.608 INFO:tasks.workunit.client.0.smithi139.stdout:7/748: dwrite d5/d16/d6d/d7f/dc2/d50/f104 [0,4194304] 0 2024-09-16T00:09:49.608 INFO:tasks.workunit.client.0.smithi139.stdout:4/694: write d12/d47/d57/f6e [2089707,106362] 0 2024-09-16T00:09:49.608 INFO:tasks.workunit.client.0.smithi139.stdout:9/853: mknod d19/d1a/d20/d89/d93/dfd/c11e 0 2024-09-16T00:09:49.608 INFO:tasks.workunit.client.0.smithi139.stdout:8/797: read d17/d1a/fa5 [1322938,105215] 0 2024-09-16T00:09:49.608 INFO:tasks.workunit.client.0.smithi139.stdout:2/858: write d11/d30/d110/d50/ff4 [933230,16856] 0 2024-09-16T00:09:49.608 INFO:tasks.workunit.client.0.smithi139.stdout:7/749: chown d5/d26/d46/d6b/da7/f105 8649613 1 2024-09-16T00:09:49.608 INFO:tasks.workunit.client.0.smithi139.stdout:4/695: write d12/d47/db2/d4d/f86 [574404,1567] 0 2024-09-16T00:09:49.609 INFO:tasks.workunit.client.0.smithi139.stdout:1/574: creat d7/d8/d40/fa7 x:0 0 0 2024-09-16T00:09:49.609 INFO:tasks.workunit.client.0.smithi139.stdout:8/798: unlink d17/d1f/d3a/c78 0 2024-09-16T00:09:49.609 INFO:tasks.workunit.client.0.smithi139.stdout:7/750: truncate d5/d16/f34 5340550 0 2024-09-16T00:09:49.609 INFO:tasks.workunit.client.0.smithi139.stdout:3/602: rename d5/d7/d23/d27/d2f/d30/d4c/cbe to d5/d7/d23/cd5 0 2024-09-16T00:09:49.609 INFO:tasks.workunit.client.0.smithi139.stdout:4/696: mknod d12/d47/db2/d4d/dd6/ce0 0 2024-09-16T00:09:49.609 INFO:tasks.workunit.client.0.smithi139.stdout:8/799: dread - d17/d1f/d3a/db7/d40/d4f/fdc zero size 2024-09-16T00:09:49.609 INFO:tasks.workunit.client.0.smithi139.stdout:7/751: mknod d5/d16/d6d/d7f/dc2/dab/c110 0 2024-09-16T00:09:49.609 INFO:tasks.workunit.client.0.smithi139.stdout:9/854: dread d19/d1a/d20/d23/d2d/d35/fb1 [0,4194304] 0 2024-09-16T00:09:49.609 INFO:tasks.workunit.client.0.smithi139.stdout:1/575: dwrite d7/d8/d40/d48/d4a/f8c [0,4194304] 0 2024-09-16T00:09:49.609 INFO:tasks.workunit.client.0.smithi139.stdout:3/603: symlink d5/d7/d23/d35/dd4/ld6 0 2024-09-16T00:09:49.609 INFO:tasks.workunit.client.0.smithi139.stdout:8/800: dread d17/d1a/d50/d86/f97 [0,4194304] 0 2024-09-16T00:09:49.609 INFO:tasks.workunit.client.0.smithi139.stdout:2/859: dwrite d11/d30/d110/d29/da5/d81/f99 [0,4194304] 0 2024-09-16T00:09:49.609 INFO:tasks.workunit.client.0.smithi139.stdout:1/576: mknod d7/d8/d1f/ca8 0 2024-09-16T00:09:49.610 INFO:tasks.workunit.client.0.smithi139.stdout:4/697: dwrite d12/d2b/f35 [0,4194304] 0 2024-09-16T00:09:49.610 INFO:tasks.workunit.client.0.smithi139.stdout:9/855: dread fa [0,4194304] 0 2024-09-16T00:09:49.610 INFO:tasks.workunit.client.0.smithi139.stdout:8/801: mknod d17/d1f/d88/c109 0 2024-09-16T00:09:49.610 INFO:tasks.workunit.client.0.smithi139.stdout:1/577: creat d7/d8/d27/d68/fa9 x:0 0 0 2024-09-16T00:09:49.610 INFO:tasks.workunit.client.0.smithi139.stdout:9/856: dread - d19/d1a/d20/d23/dca/d9f/dd9/fe8 zero size 2024-09-16T00:09:49.610 INFO:tasks.workunit.client.0.smithi139.stdout:8/802: write d17/d1a/fa5 [4999795,105377] 0 2024-09-16T00:09:49.610 INFO:tasks.workunit.client.0.smithi139.stdout:3/604: read d5/d7/d23/d27/d2f/d30/d47/f64 [285997,15012] 0 2024-09-16T00:09:49.610 INFO:tasks.workunit.client.0.smithi139.stdout:8/803: chown d17/d1a/d50/d86/d9d/cbd 42336 1 2024-09-16T00:09:49.610 INFO:tasks.workunit.client.0.smithi139.stdout:4/698: dwrite d12/d47/d49/d60/dac/fdb [0,4194304] 0 2024-09-16T00:09:49.610 INFO:tasks.workunit.client.0.smithi139.stdout:2/860: write d11/f22 [2015049,95900] 0 2024-09-16T00:09:49.610 INFO:tasks.workunit.client.0.smithi139.stdout:3/605: write d5/d7/d23/f17 [9008715,58853] 0 2024-09-16T00:09:49.610 INFO:tasks.workunit.client.0.smithi139.stdout:8/804: mkdir d17/d1f/d3a/d10a 0 2024-09-16T00:09:49.610 INFO:tasks.workunit.client.0.smithi139.stdout:9/857: truncate d19/d1a/d20/d23/d2d/d35/d42/d3a/fe2 844392 0 2024-09-16T00:09:49.610 INFO:tasks.workunit.client.0.smithi139.stdout:4/699: chown d12/lab 2959 1 2024-09-16T00:09:49.611 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:49.597+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:49.612 INFO:tasks.workunit.client.0.smithi139.stdout:2/861: creat d11/f126 x:0 0 0 2024-09-16T00:09:49.612 INFO:tasks.workunit.client.0.smithi139.stdout:3/606: creat d5/d7/dc2/fd7 x:0 0 0 2024-09-16T00:09:49.612 INFO:tasks.workunit.client.0.smithi139.stdout:8/805: write d17/d1a/d50/d86/d92/dd6/ffb [462077,56106] 0 2024-09-16T00:09:49.612 INFO:tasks.workunit.client.0.smithi139.stdout:1/578: dwrite d7/d8/d40/d48/d53/f85 [0,4194304] 0 2024-09-16T00:09:49.612 INFO:tasks.workunit.client.0.smithi139.stdout:9/858: write d19/d1a/d20/d23/dca/d71/d32/d4e/fb6 [407285,18052] 0 2024-09-16T00:09:49.612 INFO:tasks.workunit.client.0.smithi139.stdout:9/859: write d19/d1a/d20/d23/f33 [2476862,119879] 0 2024-09-16T00:09:49.612 INFO:tasks.workunit.client.0.smithi139.stdout:4/700: truncate d12/d47/d49/d60/dc3/fd0 448594 0 2024-09-16T00:09:49.615 INFO:tasks.workunit.client.0.smithi139.stdout:2/862: truncate d11/d30/d110/d5d/d73/fd4 182273 0 2024-09-16T00:09:49.622 INFO:tasks.workunit.client.0.smithi139.stdout:8/806: dwrite d17/d1f/d3a/f6f [0,4194304] 0 2024-09-16T00:09:49.622 INFO:tasks.workunit.client.0.smithi139.stdout:2/863: fsync d11/d30/d110/d29/d4f/d9c/fab 0 2024-09-16T00:09:49.623 INFO:tasks.workunit.client.0.smithi139.stdout:4/701: rename d12/d47/db2/d4d/dd1/ddf to d12/d47/db2/d4d/da9/de1 0 2024-09-16T00:09:49.626 INFO:tasks.workunit.client.0.smithi139.stdout:1/579: dwrite d7/d8/d27/f34 [4194304,4194304] 0 2024-09-16T00:09:49.627 INFO:tasks.workunit.client.0.smithi139.stdout:3/607: dread d5/d7/d23/d27/d2f/d30/d4c/d50/f54 [0,4194304] 0 2024-09-16T00:09:49.628 INFO:tasks.workunit.client.0.smithi139.stdout:8/807: mknod d17/d89/df2/c10b 0 2024-09-16T00:09:49.630 INFO:tasks.workunit.client.0.smithi139.stdout:4/702: getdents d12/d47/db2/d4d/db0 0 2024-09-16T00:09:49.633 INFO:tasks.workunit.client.0.smithi139.stdout:8/808: symlink d17/d1f/d3a/d10a/l10c 0 2024-09-16T00:09:49.633 INFO:tasks.workunit.client.0.smithi139.stdout:9/860: dread d19/f26 [4194304,4194304] 0 2024-09-16T00:09:49.634 INFO:tasks.workunit.client.0.smithi139.stdout:1/580: getdents d7/d8/d40/d48/d4a 0 2024-09-16T00:09:49.636 INFO:tasks.workunit.client.0.smithi139.stdout:8/809: chown c10 30108 1 2024-09-16T00:09:49.638 INFO:tasks.workunit.client.0.smithi139.stdout:3/608: dread d5/d7/dc/f63 [0,4194304] 0 2024-09-16T00:09:49.638 INFO:tasks.workunit.client.0.smithi139.stdout:4/703: getdents d12/d47 0 2024-09-16T00:09:49.643 INFO:tasks.workunit.client.0.smithi139.stdout:3/609: getdents d5 0 2024-09-16T00:09:49.643 INFO:tasks.workunit.client.0.smithi139.stdout:4/704: mkdir d12/d47/d49/d60/dc3/de2 0 2024-09-16T00:09:49.648 INFO:tasks.workunit.client.0.smithi139.stdout:4/705: mknod d12/ce3 0 2024-09-16T00:09:49.648 INFO:tasks.workunit.client.0.smithi139.stdout:3/610: chown d5/d7/dc/f3c 1 1 2024-09-16T00:09:49.649 INFO:tasks.workunit.client.0.smithi139.stdout:1/581: dwrite d7/d8/f36 [0,4194304] 0 2024-09-16T00:09:49.651 INFO:tasks.workunit.client.0.smithi139.stdout:9/861: dwrite d19/d1a/d20/d23/d2d/d106/f10b [0,4194304] 0 2024-09-16T00:09:49.652 INFO:tasks.workunit.client.0.smithi139.stdout:8/810: dwrite d17/d1f/d88/f100 [0,4194304] 0 2024-09-16T00:09:49.653 INFO:tasks.workunit.client.0.smithi139.stdout:3/611: readlink d5/d7/d23/d27/d2f/d30/d4c/d6d/d76/l99 0 2024-09-16T00:09:49.660 INFO:tasks.workunit.client.0.smithi139.stdout:3/612: symlink d5/d7/d23/d27/d2f/d30/d41/ld8 0 2024-09-16T00:09:49.661 INFO:tasks.workunit.client.0.smithi139.stdout:8/811: read d17/d1a/d2d/f3f [542266,11309] 0 2024-09-16T00:09:49.662 INFO:tasks.workunit.client.0.smithi139.stdout:4/706: dread d12/d47/db2/fc2 [0,4194304] 0 2024-09-16T00:09:49.667 INFO:tasks.workunit.client.0.smithi139.stdout:1/582: dwrite d7/d8/f6f [0,4194304] 0 2024-09-16T00:09:49.668 INFO:tasks.workunit.client.0.smithi139.stdout:9/862: dwrite d19/d1a/d20/d23/d2d/d35/d42/f116 [0,4194304] 0 2024-09-16T00:09:49.670 INFO:tasks.workunit.client.0.smithi139.stdout:8/812: rename d17/d89/df2/cfc to d17/d1a/d50/c10d 0 2024-09-16T00:09:49.670 INFO:tasks.workunit.client.0.smithi139.stdout:9/863: write d19/d1a/d20/d23/d2d/d2f/f11c [338531,53648] 0 2024-09-16T00:09:49.670 INFO:tasks.workunit.client.0.smithi139.stdout:1/583: mknod d7/d8/d40/d46/caa 0 2024-09-16T00:09:49.672 INFO:tasks.workunit.client.0.smithi139.stdout:3/613: getdents d5/d7/d23/d27/d2f/d30/d4c 0 2024-09-16T00:09:49.673 INFO:tasks.workunit.client.0.smithi139.stdout:8/813: dread - d17/d1a/ffe zero size 2024-09-16T00:09:49.673 INFO:tasks.workunit.client.0.smithi139.stdout:4/707: rename d12/d47/d57 to d12/d47/db2/d4d/de4 0 2024-09-16T00:09:49.675 INFO:tasks.workunit.client.0.smithi139.stdout:5/752: sync 2024-09-16T00:09:49.675 INFO:tasks.workunit.client.0.smithi139.stdout:6/518: sync 2024-09-16T00:09:49.675 INFO:tasks.workunit.client.0.smithi139.stdout:9/864: write d19/d1a/d20/d23/d2d/d106/f109 [710444,48492] 0 2024-09-16T00:09:49.675 INFO:tasks.workunit.client.0.smithi139.stdout:1/584: mknod d7/d8/d40/d47/cab 0 2024-09-16T00:09:49.676 INFO:tasks.workunit.client.0.smithi139.stdout:8/814: write d17/d1a/d50/fdb [277976,124376] 0 2024-09-16T00:09:49.677 INFO:tasks.workunit.client.0.smithi139.stdout:9/865: read d19/f26 [8528511,127082] 0 2024-09-16T00:09:49.678 INFO:tasks.workunit.client.0.smithi139.stdout:3/614: creat d5/d7/d23/d35/da7/d9a/fd9 x:0 0 0 2024-09-16T00:09:49.678 INFO:tasks.workunit.client.0.smithi139.stdout:5/753: chown d1/d13/d3f/d9f/d30/dd4 1 1 2024-09-16T00:09:49.679 INFO:tasks.workunit.client.0.smithi139.stdout:6/519: chown d8/d29/d83/d26/d28/f40 5205 1 2024-09-16T00:09:49.680 INFO:tasks.workunit.client.0.smithi139.stdout:1/585: symlink d7/lac 0 2024-09-16T00:09:49.681 INFO:tasks.workunit.client.0.smithi139.stdout:5/754: stat d1/d13/d3f/c92 0 2024-09-16T00:09:49.682 INFO:tasks.workunit.client.0.smithi139.stdout:4/708: mknod d12/d47/db2/d4d/da9/ce5 0 2024-09-16T00:09:49.683 INFO:tasks.workunit.client.0.smithi139.stdout:6/520: rmdir d8/d29/d83/d26/d45/d65/d6e 39 2024-09-16T00:09:49.685 INFO:tasks.workunit.client.0.smithi139.stdout:6/521: chown d8/d29 12427 1 2024-09-16T00:09:49.685 INFO:tasks.workunit.client.0.smithi139.stdout:1/586: write d7/d8/d40/d46/f5b [4847673,81197] 0 2024-09-16T00:09:49.686 INFO:tasks.workunit.client.0.smithi139.stdout:3/615: write d5/d7/d23/d27/d2f/d30/d4c/d6d/f7e [876631,96262] 0 2024-09-16T00:09:49.687 INFO:tasks.workunit.client.0.smithi139.stdout:9/866: dwrite d19/f1d [4194304,4194304] 0 2024-09-16T00:09:49.688 INFO:tasks.workunit.client.0.smithi139.stdout:4/709: symlink d12/d47/d49/d60/le6 0 2024-09-16T00:09:49.688 INFO:tasks.workunit.client.0.smithi139.stdout:5/755: link d1/d13/d22/f6b d1/d21/d49/db2/f108 0 2024-09-16T00:09:49.692 INFO:tasks.workunit.client.0.smithi139.stdout:6/522: fsync d8/d29/d83/d26/d45/f43 0 2024-09-16T00:09:49.693 INFO:tasks.workunit.client.0.smithi139.stdout:3/616: symlink d5/d7/d23/d27/d2f/d30/d47/lda 0 2024-09-16T00:09:49.695 INFO:tasks.workunit.client.0.smithi139.stdout:1/587: link d7/lac d7/d8/d40/d46/lad 0 2024-09-16T00:09:49.697 INFO:tasks.workunit.client.0.smithi139.stdout:9/867: chown d19/d1a/d20/d23/d2d/d35/d42/l4c 5287726 1 2024-09-16T00:09:49.697 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:49.697+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:49.700 INFO:tasks.workunit.client.0.smithi139.stdout:3/617: chown d5/d7/d23/d27/d2f/d30/d4d/dbc/fce 174283 1 2024-09-16T00:09:49.701 INFO:tasks.workunit.client.0.smithi139.stdout:1/588: fdatasync d7/d8/d40/d48/d53/d61/f87 0 2024-09-16T00:09:49.703 INFO:tasks.workunit.client.0.smithi139.stdout:9/868: symlink d19/d1a/d20/d23/d2d/d35/l11f 0 2024-09-16T00:09:49.703 INFO:tasks.workunit.client.0.smithi139.stdout:6/523: link d8/d20/d27/l34 d8/d29/d83/d26/d45/d65/d6e/lb3 0 2024-09-16T00:09:49.704 INFO:tasks.workunit.client.0.smithi139.stdout:5/756: dwrite d1/d13/d3f/d5e/f61 [0,4194304] 0 2024-09-16T00:09:49.704 INFO:tasks.workunit.client.0.smithi139.stdout:9/869: chown d19/ce9 1556 1 2024-09-16T00:09:49.705 INFO:tasks.workunit.client.0.smithi139.stdout:3/618: write d5/d7/d23/d27/d2f/f7a [163212,124849] 0 2024-09-16T00:09:49.708 INFO:tasks.workunit.client.0.smithi139.stdout:6/524: write d8/d29/d83/f81 [4982102,109056] 0 2024-09-16T00:09:49.709 INFO:tasks.workunit.client.0.smithi139.stdout:4/710: dread d12/d47/db2/d4d/de4/fb5 [0,4194304] 0 2024-09-16T00:09:49.710 INFO:tasks.workunit.client.0.smithi139.stdout:1/589: mkdir d7/d8/d40/d47/dae 0 2024-09-16T00:09:49.712 INFO:tasks.workunit.client.0.smithi139.stdout:5/757: readlink d1/d21/d49/d87/lbb 0 2024-09-16T00:09:49.713 INFO:tasks.workunit.client.0.smithi139.stdout:1/590: chown d7/d8/ca3 11 1 2024-09-16T00:09:49.713 INFO:tasks.workunit.client.0.smithi139.stdout:3/619: read d5/d7/d23/d27/d2f/d30/d41/f7f [602782,12304] 0 2024-09-16T00:09:49.716 INFO:tasks.workunit.client.0.smithi139.stdout:5/758: creat d1/d13/d3f/d88/f109 x:0 0 0 2024-09-16T00:09:49.721 INFO:tasks.workunit.client.0.smithi139.stdout:3/620: write d5/d7/dc/f3c [632070,34902] 0 2024-09-16T00:09:49.723 INFO:tasks.workunit.client.0.smithi139.stdout:5/759: fdatasync d1/d21/d49/d53/d6a/d7b/d78/dca/fcf 0 2024-09-16T00:09:49.730 INFO:tasks.workunit.client.0.smithi139.stdout:4/711: dwrite d12/d47/db2/d4d/de4/d7f/fa3 [0,4194304] 0 2024-09-16T00:09:49.730 INFO:tasks.workunit.client.0.smithi139.stdout:9/870: dread d19/d1a/d20/d23/dca/d9f/dd9/fee [0,4194304] 0 2024-09-16T00:09:49.730 INFO:tasks.workunit.client.0.smithi139.stdout:1/591: dwrite d7/d8/f36 [0,4194304] 0 2024-09-16T00:09:49.732 INFO:tasks.workunit.client.0.smithi139.stdout:3/621: fdatasync d5/d7/d23/d27/d2f/d30/d4c/d6d/f79 0 2024-09-16T00:09:49.733 INFO:tasks.workunit.client.0.smithi139.stdout:3/622: chown d5/d7/d23/d27/d2f/d30/d4d/dbc/fc5 8903 1 2024-09-16T00:09:49.734 INFO:tasks.workunit.client.0.smithi139.stdout:1/592: unlink d7/d1b/f3c 0 2024-09-16T00:09:49.736 INFO:tasks.workunit.client.0.smithi139.stdout:3/623: fdatasync d5/d7/dc/d9c/f9d 0 2024-09-16T00:09:49.739 INFO:tasks.workunit.client.0.smithi139.stdout:5/760: rename d1/d13/d3f/d88/l84 to d1/d13/d22/l10a 0 2024-09-16T00:09:49.739 INFO:tasks.workunit.client.0.smithi139.stdout:1/593: creat d7/d8/d1f/faf x:0 0 0 2024-09-16T00:09:49.740 INFO:tasks.workunit.client.0.smithi139.stdout:5/761: write d1/d21/d49/db2/fdf [733483,2768] 0 2024-09-16T00:09:49.740 INFO:tasks.workunit.client.0.smithi139.stdout:4/712: getdents d12/d47/d49/d60/dc3/de2 0 2024-09-16T00:09:49.741 INFO:tasks.workunit.client.0.smithi139.stdout:3/624: unlink d5/d7/d23/d27/d2f/l9b 0 2024-09-16T00:09:49.742 INFO:tasks.workunit.client.0.smithi139.stdout:1/594: rmdir d7/d8/d27/d68 39 2024-09-16T00:09:49.742 INFO:tasks.workunit.client.0.smithi139.stdout:3/625: write d5/d7/f28 [3999424,55952] 0 2024-09-16T00:09:49.743 INFO:tasks.workunit.client.0.smithi139.stdout:5/762: dread - d1/d13/d3f/d9f/d30/d33/d43/f4d zero size 2024-09-16T00:09:49.744 INFO:tasks.workunit.client.0.smithi139.stdout:1/595: fsync d7/d8/d40/d48/d4a/d74/f9f 0 2024-09-16T00:09:49.744 INFO:tasks.workunit.client.0.smithi139.stdout:4/713: getdents d12/d47/db2/d4d/db0 0 2024-09-16T00:09:49.745 INFO:tasks.workunit.client.0.smithi139.stdout:5/763: write d1/d13/d3f/d9f/fa4 [501750,118348] 0 2024-09-16T00:09:49.746 INFO:tasks.workunit.client.0.smithi139.stdout:4/714: stat d12/d2b/f34 0 2024-09-16T00:09:49.747 INFO:tasks.workunit.client.0.smithi139.stdout:3/626: unlink d5/d7/d23/d27/d2f/d30/d4c/d50/d75/fb7 0 2024-09-16T00:09:49.749 INFO:tasks.workunit.client.0.smithi139.stdout:4/715: truncate d12/d47/d49/fc0 743888 0 2024-09-16T00:09:49.751 INFO:tasks.workunit.client.0.smithi139.stdout:5/764: dread d1/d21/d49/d53/d6a/f8b [0,4194304] 0 2024-09-16T00:09:49.752 INFO:tasks.workunit.client.0.smithi139.stdout:4/716: truncate d12/d47/fb3 716075 0 2024-09-16T00:09:49.753 INFO:tasks.workunit.client.0.smithi139.stdout:4/717: chown d12/d47/d49/d60/dac 10389 1 2024-09-16T00:09:49.755 INFO:tasks.workunit.client.0.smithi139.stdout:5/765: mkdir d1/d13/d3f/d9f/d73/d79/da5/d10b 0 2024-09-16T00:09:49.756 INFO:tasks.workunit.client.0.smithi139.stdout:9/871: dread d19/d1a/d20/d23/d2d/d35/d42/fea [0,4194304] 0 2024-09-16T00:09:49.756 INFO:tasks.workunit.client.0.smithi139.stdout:5/766: truncate d1/fad 479027 0 2024-09-16T00:09:49.758 INFO:tasks.workunit.client.0.smithi139.stdout:1/596: dread d7/f10 [0,4194304] 0 2024-09-16T00:09:49.758 INFO:tasks.workunit.client.0.smithi139.stdout:4/718: rename d12/d47/d49/f53 to d12/d47/db2/d4d/fe7 0 2024-09-16T00:09:49.759 INFO:tasks.workunit.client.0.smithi139.stdout:9/872: write d19/d1a/d20/d23/d2d/d35/d42/d3a/f6d [18277,23165] 0 2024-09-16T00:09:49.759 INFO:tasks.workunit.client.0.smithi139.stdout:5/767: mkdir d1/d13/d3f/d5e/dc2/d10c 0 2024-09-16T00:09:49.761 INFO:tasks.workunit.client.0.smithi139.stdout:4/719: write d12/d47/db2/d4d/de4/f6e [1919076,21289] 0 2024-09-16T00:09:49.762 INFO:tasks.workunit.client.0.smithi139.stdout:9/873: symlink d19/d1a/d20/d23/d2d/d35/d44/l120 0 2024-09-16T00:09:49.762 INFO:tasks.workunit.client.0.smithi139.stdout:3/627: write d5/d7/dc/f12 [7961173,63760] 0 2024-09-16T00:09:49.764 INFO:tasks.workunit.client.0.smithi139.stdout:9/874: write d19/d1a/d20/d23/d2d/d2f/f96 [1779160,61205] 0 2024-09-16T00:09:49.767 INFO:tasks.workunit.client.0.smithi139.stdout:9/875: chown d19/d1a/d20/d23/d2d/d35/d42/d3a/df3 14274 1 2024-09-16T00:09:49.768 INFO:tasks.workunit.client.0.smithi139.stdout:5/768: getdents d1/d13/d3f/d5e 0 2024-09-16T00:09:49.769 INFO:tasks.workunit.client.0.smithi139.stdout:4/720: link d12/lab d12/d47/d49/d60/le8 0 2024-09-16T00:09:49.769 INFO:tasks.workunit.client.0.smithi139.stdout:1/597: chown d7/d1b/f3a 89 1 2024-09-16T00:09:49.772 INFO:tasks.workunit.client.0.smithi139.stdout:3/628: dwrite d5/d7/fbf [0,4194304] 0 2024-09-16T00:09:49.772 INFO:tasks.workunit.client.0.smithi139.stdout:5/769: read d1/d13/d3b/d3e/f6e [1339005,54197] 0 2024-09-16T00:09:49.773 INFO:tasks.workunit.client.0.smithi139.stdout:4/721: chown d12/d47/db2/d4d/de4/f62 1151947 1 2024-09-16T00:09:49.775 INFO:tasks.workunit.client.0.smithi139.stdout:9/876: write f14 [2011404,63743] 0 2024-09-16T00:09:49.777 INFO:tasks.workunit.client.0.smithi139.stdout:3/629: fsync d5/d7/d23/d27/d2f/d30/d47/f4a 0 2024-09-16T00:09:49.777 INFO:tasks.workunit.client.0.smithi139.stdout:5/770: rename d1/d13/d3f/d9f/d30 to d1/d13/d22/de8/d10d 0 2024-09-16T00:09:49.778 INFO:tasks.workunit.client.0.smithi139.stdout:1/598: creat d7/d8/d27/fb0 x:0 0 0 2024-09-16T00:09:49.780 INFO:tasks.workunit.client.0.smithi139.stdout:5/771: write d1/d13/d3f/fd3 [2800115,78545] 0 2024-09-16T00:09:49.782 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:49.782 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:49.782 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:49.782 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:49.782 INFO:tasks.workunit.client.0.smithi139.stdout:9/877: unlink d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/le3 0 2024-09-16T00:09:49.782 INFO:tasks.workunit.client.0.smithi139.stdout:4/722: dwrite d12/d47/db2/d4d/de4/fcf [0,4194304] 0 2024-09-16T00:09:49.785 INFO:tasks.workunit.client.0.smithi139.stdout:5/772: mknod d1/d13/d3f/d88/c10e 0 2024-09-16T00:09:49.786 INFO:tasks.workunit.client.0.smithi139.stdout:5/773: chown d1/d13/d3f/d5e/dc2/d10c 1094251 1 2024-09-16T00:09:49.787 INFO:tasks.workunit.client.0.smithi139.stdout:4/723: rmdir d12/d2b/d8c 39 2024-09-16T00:09:49.787 INFO:tasks.workunit.client.0.smithi139.stdout:3/630: write d5/d7/d23/d27/d2f/d30/d4c/d50/fcd [693919,130897] 0 2024-09-16T00:09:49.788 INFO:tasks.workunit.client.0.smithi139.stdout:9/878: creat d19/d1a/d20/d23/dca/d71/dc7/de1/f121 x:0 0 0 2024-09-16T00:09:49.789 INFO:tasks.workunit.client.0.smithi139.stdout:4/724: unlink d12/d47/db2/d4d/c5b 0 2024-09-16T00:09:49.791 INFO:tasks.workunit.client.0.smithi139.stdout:1/599: getdents d7/d8/d40/d48/d53 0 2024-09-16T00:09:49.791 INFO:tasks.workunit.client.0.smithi139.stdout:4/725: chown d12/d2b 169727276 1 2024-09-16T00:09:49.793 INFO:tasks.workunit.client.0.smithi139.stdout:1/600: write d7/d8/d27/fb0 [861434,70292] 0 2024-09-16T00:09:49.794 INFO:tasks.workunit.client.0.smithi139.stdout:5/774: dwrite d1/d21/d49/d53/d6a/d7b/d99/fab [0,4194304] 0 2024-09-16T00:09:49.796 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:09:49.797 INFO:tasks.workunit.client.0.smithi139.stdout:1/601: chown d7/d8/d40/d48/f65 212 1 2024-09-16T00:09:49.800 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:49.799+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:49.800 INFO:tasks.workunit.client.0.smithi139.stdout:9/879: dwrite d19/d1a/d20/d23/d2d/fa4 [0,4194304] 0 2024-09-16T00:09:49.804 INFO:tasks.workunit.client.0.smithi139.stdout:1/602: fsync d7/f18 0 2024-09-16T00:09:49.807 INFO:tasks.workunit.client.0.smithi139.stdout:9/880: mknod d19/d1a/d20/d23/dca/d71/c122 0 2024-09-16T00:09:49.810 INFO:tasks.workunit.client.0.smithi139.stdout:4/726: dread d12/d2b/d8c/fd7 [0,4194304] 0 2024-09-16T00:09:49.812 INFO:tasks.workunit.client.0.smithi139.stdout:9/881: write d19/d1a/d20/d23/d2d/f99 [272431,11297] 0 2024-09-16T00:09:49.813 INFO:tasks.workunit.client.0.smithi139.stdout:4/727: mknod d12/d47/ce9 0 2024-09-16T00:09:49.814 INFO:tasks.workunit.client.0.smithi139.stdout:5/775: dread d1/d13/ff4 [0,4194304] 0 2024-09-16T00:09:49.815 INFO:tasks.workunit.client.0.smithi139.stdout:9/882: symlink d19/d1a/d20/d23/dca/d71/dc7/de1/l123 0 2024-09-16T00:09:49.816 INFO:tasks.workunit.client.0.smithi139.stdout:4/728: creat d12/d47/d49/d60/fea x:0 0 0 2024-09-16T00:09:49.819 INFO:tasks.workunit.client.0.smithi139.stdout:4/729: write d12/d47/d49/fbe [722514,25210] 0 2024-09-16T00:09:49.821 INFO:tasks.workunit.client.0.smithi139.stdout:1/603: dread d7/d8/d1f/f76 [0,4194304] 0 2024-09-16T00:09:49.824 INFO:tasks.workunit.client.0.smithi139.stdout:1/604: write d7/d8/f6f [2940884,16008] 0 2024-09-16T00:09:49.825 INFO:tasks.workunit.client.0.smithi139.stdout:4/730: creat d12/d47/db2/d4d/de4/d7f/feb x:0 0 0 2024-09-16T00:09:49.826 INFO:tasks.workunit.client.0.smithi139.stdout:9/883: dwrite d19/d1a/d20/d23/dca/d71/d32/d79/f91 [0,4194304] 0 2024-09-16T00:09:49.830 INFO:tasks.workunit.client.0.smithi139.stdout:1/605: rename d7/d8/d27/f75 to d7/d8/d27/d68/d6c/fb1 0 2024-09-16T00:09:49.834 INFO:tasks.workunit.client.0.smithi139.stdout:5/776: dread d1/d21/d49/f7f [0,4194304] 0 2024-09-16T00:09:49.834 INFO:tasks.workunit.client.0.smithi139.stdout:4/731: dread d12/f25 [0,4194304] 0 2024-09-16T00:09:49.834 INFO:tasks.workunit.client.0.smithi139.stdout:9/884: rename d19/d1a/d20/d23/d2d/d35/d60/cdc to d19/d1a/d20/d23/dca/d71/d32/d4e/c124 0 2024-09-16T00:09:49.836 INFO:tasks.workunit.client.0.smithi139.stdout:4/732: write d12/d47/db2/f77 [427104,65418] 0 2024-09-16T00:09:49.837 INFO:tasks.workunit.client.0.smithi139.stdout:5/777: creat d1/d13/d3f/d88/f10f x:0 0 0 2024-09-16T00:09:49.838 INFO:tasks.workunit.client.0.smithi139.stdout:9/885: fsync d19/d1a/d20/d23/d2d/f90 0 2024-09-16T00:09:49.839 INFO:tasks.workunit.client.0.smithi139.stdout:1/606: write d7/f16 [1602376,2196] 0 2024-09-16T00:09:49.840 INFO:tasks.workunit.client.0.smithi139.stdout:5/778: mknod d1/df6/c110 0 2024-09-16T00:09:49.841 INFO:tasks.workunit.client.0.smithi139.stdout:5/779: chown d1/d21/d49/d53/d6a/d7b 14401793 1 2024-09-16T00:09:49.841 INFO:tasks.workunit.client.0.smithi139.stdout:4/733: link d12/d47/db2/f63 d12/d47/d49/d60/dc3/fec 0 2024-09-16T00:09:49.842 INFO:tasks.workunit.client.0.smithi139.stdout:1/607: symlink d7/d8/d27/d68/d7c/d83/lb2 0 2024-09-16T00:09:49.843 INFO:tasks.workunit.client.0.smithi139.stdout:4/734: write d12/d47/db2/f77 [884754,11616] 0 2024-09-16T00:09:49.843 INFO:tasks.workunit.client.0.smithi139.stdout:9/886: creat d19/d1a/d20/d89/f125 x:0 0 0 2024-09-16T00:09:49.845 INFO:tasks.workunit.client.0.smithi139.stdout:5/780: rename d1/d13/fdc to d1/d13/d22/de8/d10d/dd4/f111 0 2024-09-16T00:09:49.848 INFO:tasks.workunit.client.0.smithi139.stdout:4/735: getdents d12/d47/db2/d4d/dd6 0 2024-09-16T00:09:49.850 INFO:tasks.workunit.client.0.smithi139.stdout:5/781: rmdir d1/d13/d3f/d9f/d106 0 2024-09-16T00:09:49.850 INFO:tasks.workunit.client.0.smithi139.stdout:9/887: dwrite d19/d1a/d20/d23/f3e [4194304,4194304] 0 2024-09-16T00:09:49.852 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:09:49.853 INFO:tasks.workunit.client.0.smithi139.stdout:5/782: creat d1/d13/d3b/d3e/f112 x:0 0 0 2024-09-16T00:09:49.855 INFO:tasks.workunit.client.0.smithi139.stdout:4/736: fsync d12/d47/db2/f63 0 2024-09-16T00:09:49.855 INFO:tasks.workunit.client.0.smithi139.stdout:9/888: unlink d19/d1a/d20/d89/d93/dfd/c11e 0 2024-09-16T00:09:49.856 INFO:tasks.workunit.client.0.smithi139.stdout:9/889: readlink d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/lab 0 2024-09-16T00:09:49.857 INFO:tasks.workunit.client.0.smithi139.stdout:5/783: fdatasync d1/d21/d49/fd1 0 2024-09-16T00:09:49.859 INFO:tasks.workunit.client.0.smithi139.stdout:4/737: symlink d12/d47/db2/d4d/led 0 2024-09-16T00:09:49.861 INFO:tasks.workunit.client.0.smithi139.stdout:1/608: write d7/f26 [801979,48518] 0 2024-09-16T00:09:49.864 INFO:tasks.workunit.client.0.smithi139.stdout:9/890: creat d19/d1a/d20/d23/dca/d71/dc7/de1/d115/f126 x:0 0 0 2024-09-16T00:09:49.867 INFO:tasks.workunit.client.0.smithi139.stdout:1/609: truncate d7/d1b/fa5 3618072 0 2024-09-16T00:09:49.872 INFO:tasks.workunit.client.0.smithi139.stdout:4/738: write d12/d80/f9a [3136865,1451] 0 2024-09-16T00:09:49.874 INFO:tasks.workunit.client.0.smithi139.stdout:9/891: dwrite d19/d1a/d20/d23/d2d/d35/d44/f114 [0,4194304] 0 2024-09-16T00:09:49.875 INFO:tasks.workunit.client.0.smithi139.stdout:1/610: getdents d7/d8/d27/d2f 0 2024-09-16T00:09:49.876 INFO:tasks.workunit.client.0.smithi139.stdout:5/784: dread d1/d13/d3f/fb8 [0,4194304] 0 2024-09-16T00:09:49.877 INFO:tasks.workunit.client.0.smithi139.stdout:1/611: dread - d7/d8/d40/fa7 zero size 2024-09-16T00:09:49.878 INFO:tasks.workunit.client.0.smithi139.stdout:4/739: creat d12/d2b/d8c/fee x:0 0 0 2024-09-16T00:09:49.879 INFO:tasks.workunit.client.0.smithi139.stdout:9/892: unlink d19/d1a/d20/d23/dca/d71/da3/f111 0 2024-09-16T00:09:49.881 INFO:tasks.workunit.client.0.smithi139.stdout:4/740: creat d12/d47/db2/d4d/dd1/fef x:0 0 0 2024-09-16T00:09:49.881 INFO:tasks.workunit.client.0.smithi139.stdout:9/893: chown d19/d1a/d20/d23/dca/d71/d32/d4e/f8a 1064 1 2024-09-16T00:09:49.882 INFO:tasks.workunit.client.0.smithi139.stdout:4/741: write d12/d47/db2/d4d/f89 [2400970,4991] 0 2024-09-16T00:09:49.883 INFO:tasks.workunit.client.0.smithi139.stdout:5/785: rename d1/d13/d3f/d9f/d73/d79/lc6 to d1/d21/d49/d53/d6a/d7b/da1/l113 0 2024-09-16T00:09:49.885 INFO:tasks.workunit.client.0.smithi139.stdout:4/742: chown d12/d47/f32 390561601 1 2024-09-16T00:09:49.888 INFO:tasks.workunit.client.0.smithi139.stdout:4/743: write d12/d47/d49/d60/dac/fdb [2297031,65251] 0 2024-09-16T00:09:49.893 INFO:tasks.workunit.client.0.smithi139.stdout:4/744: symlink d12/d47/db2/d4d/d69/lf0 0 2024-09-16T00:09:49.893 INFO:tasks.workunit.client.0.smithi139.stdout:9/894: dwrite d19/f26 [4194304,4194304] 0 2024-09-16T00:09:49.893 INFO:tasks.workunit.client.0.smithi139.stdout:5/786: dwrite d1/d13/ff4 [0,4194304] 0 2024-09-16T00:09:49.899 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:49.898+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:49.902 INFO:tasks.workunit.client.0.smithi139.stdout:5/787: rename d1/d13/d3f/d9f/d73/fef to d1/d21/d49/d53/d6a/d7b/f114 0 2024-09-16T00:09:49.907 INFO:tasks.workunit.client.0.smithi139.stdout:9/895: dwrite d19/d1a/d20/d23/dca/d71/dc7/de1/f121 [0,4194304] 0 2024-09-16T00:09:49.909 INFO:tasks.workunit.client.0.smithi139.stdout:9/896: chown d19/d1a/d20/d23/d2d/d9a/faa 423936573 1 2024-09-16T00:09:49.914 INFO:tasks.workunit.client.0.smithi139.stdout:4/745: dread d12/d47/db2/d4d/fbb [0,4194304] 0 2024-09-16T00:09:49.916 INFO:tasks.workunit.client.0.smithi139.stdout:4/746: truncate d12/d2b/f35 5194087 0 2024-09-16T00:09:49.917 INFO:tasks.workunit.client.0.smithi139.stdout:5/788: dread d1/d13/d3f/d88/fe4 [0,4194304] 0 2024-09-16T00:09:49.920 INFO:tasks.workunit.client.0.smithi139.stdout:4/747: read d12/d47/db2/d4d/f81 [3282777,39622] 0 2024-09-16T00:09:49.925 INFO:tasks.workunit.client.0.smithi139.stdout:9/897: dwrite d19/d1a/d20/d23/d2d/d35/d42/fea [4194304,4194304] 0 2024-09-16T00:09:49.925 INFO:tasks.workunit.client.0.smithi139.stdout:9/898: dread - d19/d1a/ff0 zero size 2024-09-16T00:09:49.925 INFO:tasks.workunit.client.0.smithi139.stdout:4/748: mkdir d12/d47/d49/df1 0 2024-09-16T00:09:49.926 INFO:tasks.workunit.client.0.smithi139.stdout:5/789: link d1/d21/d49/d53/d6a/d7b/d99/l85 d1/d13/l115 0 2024-09-16T00:09:49.928 INFO:tasks.workunit.client.0.smithi139.stdout:5/790: truncate d1/d13/fd9 71755 0 2024-09-16T00:09:49.930 INFO:tasks.workunit.client.0.smithi139.stdout:5/791: fdatasync d1/d21/d49/d53/d6a/d7b/fd7 0 2024-09-16T00:09:49.932 INFO:tasks.workunit.client.0.smithi139.stdout:4/749: write d12/d47/db2/d4d/fe7 [2466384,121524] 0 2024-09-16T00:09:49.932 INFO:tasks.workunit.client.0.smithi139.stdout:9/899: dwrite d19/d1a/d20/d23/d2d/f99 [0,4194304] 0 2024-09-16T00:09:49.935 INFO:tasks.workunit.client.0.smithi139.stdout:4/750: rmdir d12/d47/db2/d4d/dd1 39 2024-09-16T00:09:49.940 INFO:tasks.workunit.client.0.smithi139.stdout:4/751: creat d12/d2b/ff2 x:0 0 0 2024-09-16T00:09:49.941 INFO:tasks.workunit.client.0.smithi139.stdout:9/900: link d19/d1a/d20/d23/d2d/d35/d60/c70 d19/d1a/d20/d23/d2d/d35/d44/df4/c127 0 2024-09-16T00:09:49.942 INFO:tasks.workunit.client.0.smithi139.stdout:4/752: mknod d12/d47/db2/d4d/de4/d7f/cf3 0 2024-09-16T00:09:49.943 INFO:tasks.workunit.client.0.smithi139.stdout:9/901: creat d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/f128 x:0 0 0 2024-09-16T00:09:49.944 INFO:tasks.workunit.client.0.smithi139.stdout:4/753: creat d12/d2b/d8c/ff4 x:0 0 0 2024-09-16T00:09:49.945 INFO:tasks.workunit.client.0.smithi139.stdout:9/902: mkdir d19/d1a/d20/d23/d2d/d35/d44/d129 0 2024-09-16T00:09:49.948 INFO:tasks.workunit.client.0.smithi139.stdout:4/754: dread d12/d2b/f35 [0,4194304] 0 2024-09-16T00:09:49.949 INFO:tasks.workunit.client.0.smithi139.stdout:9/903: link d19/d1a/d20/d23/dca/d71/da3/fce d19/d1a/d20/d23/f12a 0 2024-09-16T00:09:49.951 INFO:tasks.workunit.client.0.smithi139.stdout:9/904: symlink d19/d1a/d20/d23/d2d/d106/l12b 0 2024-09-16T00:09:49.953 INFO:tasks.workunit.client.0.smithi139.stdout:9/905: write d19/d1a/d20/d23/dca/d71/d32/f76 [1488578,33508] 0 2024-09-16T00:09:49.959 INFO:tasks.workunit.client.0.smithi139.stdout:4/755: dread d12/d2b/d8c/fb8 [0,4194304] 0 2024-09-16T00:09:49.962 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:49.962 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:49.962 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:49.962 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:49.962 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:49.963 INFO:tasks.workunit.client.0.smithi139.stdout:4/756: rename d12/d47/d49/d60/dc3/fd0 to d12/d47/db2/d4d/dd6/ff5 0 2024-09-16T00:09:49.965 INFO:tasks.workunit.client.0.smithi139.stdout:9/906: dread d19/f6c [0,4194304] 0 2024-09-16T00:09:49.967 INFO:tasks.workunit.client.0.smithi139.stdout:9/907: chown d19/d1a/d20/d23/dca/d71/da3/ca7 89190 1 2024-09-16T00:09:49.968 INFO:tasks.workunit.client.0.smithi139.stdout:4/757: dread d12/d47/db2/d4d/de4/fcf [0,4194304] 0 2024-09-16T00:09:49.970 INFO:tasks.workunit.client.0.smithi139.stdout:9/908: truncate d19/d1a/d20/d23/dca/d71/d32/f84 4572631 0 2024-09-16T00:09:49.973 INFO:tasks.workunit.client.0.smithi139.stdout:4/758: dwrite d12/d47/f32 [0,4194304] 0 2024-09-16T00:09:49.975 INFO:tasks.workunit.client.0.smithi139.stdout:9/909: link d19/d1a/ff0 d19/d1a/d20/d23/d2d/d35/d42/de7/f12c 0 2024-09-16T00:09:49.976 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:09:49.978 INFO:tasks.workunit.client.0.smithi139.stdout:9/910: rename d19/d1a/d20/d23/d2d/d9a/fcc to d19/d1a/d20/d23/d2d/d35/d42/dc8/df1/f12d 0 2024-09-16T00:09:49.981 INFO:tasks.workunit.client.0.smithi139.stdout:4/759: dwrite d12/f25 [0,4194304] 0 2024-09-16T00:09:49.982 INFO:tasks.workunit.client.0.smithi139.stdout:9/911: truncate d19/d1a/d20/d23/d2d/d35/d42/de7/f12c 872405 0 2024-09-16T00:09:49.983 INFO:tasks.workunit.client.0.smithi139.stdout:4/760: dread - d12/d2b/d8c/ff4 zero size 2024-09-16T00:09:49.984 INFO:tasks.workunit.client.0.smithi139.stdout:0/694: sync 2024-09-16T00:09:49.986 INFO:tasks.workunit.client.0.smithi139.stdout:9/912: symlink d19/d1a/d20/d89/l12e 0 2024-09-16T00:09:49.988 INFO:tasks.workunit.client.0.smithi139.stdout:4/761: symlink d12/d47/d49/d60/dac/lf6 0 2024-09-16T00:09:49.988 INFO:tasks.workunit.client.0.smithi139.stdout:0/695: unlink d6/d1c/d41/d44/d58/f68 0 2024-09-16T00:09:49.989 INFO:tasks.workunit.client.0.smithi139.stdout:4/762: chown d12/d47/d49/fc7 25631 1 2024-09-16T00:09:49.990 INFO:tasks.workunit.client.0.smithi139.stdout:9/913: symlink d19/d1a/d20/d23/dca/d71/dc7/l12f 0 2024-09-16T00:09:49.991 INFO:tasks.workunit.client.0.smithi139.stdout:0/696: symlink d6/d1c/d41/d44/dd/d8a/d9e/ldb 0 2024-09-16T00:09:49.992 INFO:tasks.workunit.client.0.smithi139.stdout:0/697: fdatasync d6/d5a/fd7 0 2024-09-16T00:09:49.995 INFO:tasks.workunit.client.0.smithi139.stdout:0/698: creat d6/d1c/d41/d44/dd/d8a/d9e/fdc x:0 0 0 2024-09-16T00:09:49.996 INFO:tasks.workunit.client.0.smithi139.stdout:0/699: creat d6/d1c/d41/d76/fdd x:0 0 0 2024-09-16T00:09:50.000 INFO:tasks.workunit.client.0.smithi139.stdout:4/763: write d12/d47/d49/d60/f66 [255139,120565] 0 2024-09-16T00:09:50.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.000+0000 7f9903981640 1 Processor -- start 2024-09-16T00:09:50.001 INFO:tasks.workunit.client.0.smithi139.stdout:9/914: truncate d19/f1d 1037057 0 2024-09-16T00:09:50.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.000+0000 7f9903981640 1 -- start start 2024-09-16T00:09:50.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.002+0000 7f9903981640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f98fc0b25f0 0x7f98fc1624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.002+0000 7f9903981640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98fc162a20 0x7f98fc169810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.002+0000 7f9903981640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98fc0c8350 con 0x7f98fc15c850 2024-09-16T00:09:50.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.002+0000 7f9903981640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f98fc0c8120 con 0x7f98fc0b25f0 2024-09-16T00:09:50.003 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.002+0000 7f9903981640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f98fc15cc30 con 0x7f98fc162a20 2024-09-16T00:09:50.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.002+0000 7f990171d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f98fc15c850 0x7f98fc06d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59272/0 (socket says 172.21.15.139:59272) 2024-09-16T00:09:50.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.002+0000 7f990171d640 1 -- 172.21.15.139:0/261909011 learned_addr learned my addr 172.21.15.139:0/261909011 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:50.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.002+0000 7f9900f1c640 1 --2- 172.21.15.139:0/261909011 >> v2:172.21.15.164:3300/0 conn(0x7f98fc0b25f0 0x7f98fc1624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.002+0000 7f9901f1e640 1 --2- 172.21.15.139:0/261909011 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98fc162a20 0x7f98fc169810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.002+0000 7f98f3fff640 1 -- 172.21.15.139:0/261909011 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3170959056 0 0) 0x7f98fc0c8350 con 0x7f98fc15c850 2024-09-16T00:09:50.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.002+0000 7f98f3fff640 1 -- 172.21.15.139:0/261909011 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98e8003520 con 0x7f98fc15c850 2024-09-16T00:09:50.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.002+0000 7f9900f1c640 1 -- 172.21.15.139:0/261909011 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98fc162a20 msgr2=0x7f98fc169810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:50.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.002+0000 7f9900f1c640 1 --2- 172.21.15.139:0/261909011 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98fc162a20 0x7f98fc169810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:50.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.002+0000 7f9900f1c640 1 -- 172.21.15.139:0/261909011 >> v1:172.21.15.139:6789/0 conn(0x7f98fc15c850 legacy=0x7f98fc06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:50.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.002+0000 7f9900f1c640 1 -- 172.21.15.139:0/261909011 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98fc0c8ea0 con 0x7f98fc0b25f0 2024-09-16T00:09:50.005 INFO:tasks.workunit.client.0.smithi139.stdout:4/764: write d12/d47/db2/d4d/dd1/fef [884253,43294] 0 2024-09-16T00:09:50.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.003+0000 7f9900f1c640 1 --2- 172.21.15.139:0/261909011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98fc0b25f0 0x7f98fc1624e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0097a0 tx=0x7f98ec02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc25168c9d461e10 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:50.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.003+0000 7f98f3fff640 1 -- 172.21.15.139:0/261909011 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec037070 con 0x7f98fc0b25f0 2024-09-16T00:09:50.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.003+0000 7f98f3fff640 1 -- 172.21.15.139:0/261909011 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ec032440 con 0x7f98fc0b25f0 2024-09-16T00:09:50.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.003+0000 7f98f3fff640 1 -- 172.21.15.139:0/261909011 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec032810 con 0x7f98fc0b25f0 2024-09-16T00:09:50.006 INFO:tasks.workunit.client.0.smithi139.stdout:0/700: dwrite d6/d1c/d41/d44/d58/f88 [0,4194304] 0 2024-09-16T00:09:50.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.004+0000 7f9903981640 1 -- 172.21.15.139:0/261909011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98fc0b25f0 msgr2=0x7f98fc1624e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:50.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.004+0000 7f9903981640 1 --2- 172.21.15.139:0/261909011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98fc0b25f0 0x7f98fc1624e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0097a0 tx=0x7f98ec02a210 comp rx=0 tx=0).stop 2024-09-16T00:09:50.006 INFO:tasks.workunit.client.0.smithi139.stdout:9/915: write d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/f128 [552609,50256] 0 2024-09-16T00:09:50.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.005+0000 7f9903981640 1 -- 172.21.15.139:0/261909011 shutdown_connections 2024-09-16T00:09:50.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.005+0000 7f9903981640 1 --2- 172.21.15.139:0/261909011 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98fc162a20 0x7f98fc169810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:50.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.005+0000 7f9903981640 1 --2- 172.21.15.139:0/261909011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98fc0b25f0 0x7f98fc1624e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:50.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.005+0000 7f9903981640 1 -- 172.21.15.139:0/261909011 >> 172.21.15.139:0/261909011 conn(0x7f98fc0c9a80 msgr2=0x7f98fc0c9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:50.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.005+0000 7f9903981640 1 -- 172.21.15.139:0/261909011 shutdown_connections 2024-09-16T00:09:50.007 INFO:tasks.workunit.client.0.smithi139.stdout:0/701: stat d6/d1c/d41/d63/d82/lcc 0 2024-09-16T00:09:50.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.006+0000 7f9903981640 1 -- 172.21.15.139:0/261909011 wait complete. 2024-09-16T00:09:50.008 INFO:tasks.workunit.client.0.smithi139.stdout:4/765: write d12/d47/db2/d4d/d69/fb4 [266900,71035] 0 2024-09-16T00:09:50.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.007+0000 7f9903981640 1 Processor -- start 2024-09-16T00:09:50.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.008+0000 7f9903981640 1 -- start start 2024-09-16T00:09:50.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.009+0000 7f9903981640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98fc0b25f0 0x7f98fc1e8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.009+0000 7f9903981640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98fc15c850 0x7f98fc155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.009+0000 7f9903981640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98fc162a20 0x7f98fc1f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.009+0000 7f9903981640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc162eb0 con 0x7f98fc162a20 2024-09-16T00:09:50.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.009+0000 7f9903981640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc0c8c50 con 0x7f98fc15c850 2024-09-16T00:09:50.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.009+0000 7f9903981640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f98fc06d900 con 0x7f98fc0b25f0 2024-09-16T00:09:50.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.009+0000 7f9901f1e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98fc162a20 0x7f98fc1f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.009+0000 7f9901f1e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98fc162a20 0x7f98fc1f50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:37554/0 (socket says 172.21.15.139:37554) 2024-09-16T00:09:50.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.009+0000 7f9901f1e640 1 -- 172.21.15.139:0/4003791905 learned_addr learned my addr 172.21.15.139:0/4003791905 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:50.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.009+0000 7f9900f1c640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98fc15c850 0x7f98fc155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.010+0000 7f990171d640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98fc0b25f0 0x7f98fc1e8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.010+0000 7f9901f1e640 1 -- 172.21.15.139:0/4003791905 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98fc0b25f0 msgr2=0x7f98fc1e8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:50.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.010+0000 7f9901f1e640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98fc0b25f0 0x7f98fc1e8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:50.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.010+0000 7f9901f1e640 1 -- 172.21.15.139:0/4003791905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98fc15c850 msgr2=0x7f98fc155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:50.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.010+0000 7f9901f1e640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f98fc15c850 0x7f98fc155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:50.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.010+0000 7f9901f1e640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98fc1f8a30 con 0x7f98fc162a20 2024-09-16T00:09:50.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.010+0000 7f9901f1e640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98fc162a20 0x7f98fc1f50d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f98f400efc0 tx=0x7f98f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:50.013 INFO:tasks.workunit.client.0.smithi139.stdout:0/702: getdents d6/d5a/d81 0 2024-09-16T00:09:50.013 INFO:tasks.workunit.client.0.smithi139.stdout:4/766: read d12/d47/db2/d4d/d69/f8a [2834822,91892] 0 2024-09-16T00:09:50.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.010+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4018020 con 0x7f98fc162a20 2024-09-16T00:09:50.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.010+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f4014480 con 0x7f98fc162a20 2024-09-16T00:09:50.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.010+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98fc1f8ca0 con 0x7f98fc162a20 2024-09-16T00:09:50.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.011+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4014870 con 0x7f98fc162a20 2024-09-16T00:09:50.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.011+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98fc155f60 con 0x7f98fc162a20 2024-09-16T00:09:50.014 INFO:tasks.workunit.client.0.smithi139.stdout:0/703: stat d6/ff 0 2024-09-16T00:09:50.015 INFO:tasks.workunit.client.0.smithi139.stdout:9/916: rename d19/d1a/d20/d23/dca/d71/lbf to d19/d1a/d20/d23/dca/d71/d32/d79/l130 0 2024-09-16T00:09:50.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.012+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f98fc0b32a0 con 0x7f98fc162a20 2024-09-16T00:09:50.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.013+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f98f4005d10 con 0x7f98fc162a20 2024-09-16T00:09:50.015 INFO:tasks.workunit.client.0.smithi139.stdout:0/704: chown d6/d1c/d41/d44/f3b 3 1 2024-09-16T00:09:50.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.013+0000 7f98f1ffb640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f98e006c2b0 0x7f98e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.013+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5973+0+0 (secure 0 0 0) 0x7f98f408bf60 con 0x7f98fc162a20 2024-09-16T00:09:50.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.013+0000 7f990171d640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f98e006c2b0 0x7f98e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.014+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v89) ==== 54+0+16 (secure 0 0 0) 0x7f98f4022070 con 0x7f98fc162a20 2024-09-16T00:09:50.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.014+0000 7f990171d640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f98e006c2b0 0x7f98e006e770 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f98f8005880 tx=0x7f98f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:50.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.014+0000 7f9903981640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f98fc1f5610 0x7f98fc15ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.014+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98fc1f4a10 con 0x7f98fc1f5610 2024-09-16T00:09:50.017 INFO:tasks.workunit.client.0.smithi139.stdout:0/705: creat d6/d1c/d41/d44/d6b/fde x:0 0 0 2024-09-16T00:09:50.018 INFO:tasks.workunit.client.0.smithi139.stdout:9/917: rename d19/d1a/d20/d23/dca/d9f/cc5 to d19/d1a/d20/c131 0 2024-09-16T00:09:50.019 INFO:tasks.workunit.client.0.smithi139.stdout:4/767: dwrite d12/d80/f9a [4194304,4194304] 0 2024-09-16T00:09:50.021 INFO:tasks.workunit.client.0.smithi139.stdout:0/706: truncate d6/d1c/d41/d63/f66 496299 0 2024-09-16T00:09:50.022 INFO:tasks.workunit.client.0.smithi139.stdout:0/707: dread - d6/d5a/d70/d7b/fba zero size 2024-09-16T00:09:50.026 INFO:tasks.workunit.client.0.smithi139.stdout:4/768: dwrite d12/d47/d49/d60/dc3/fcb [0,4194304] 0 2024-09-16T00:09:50.027 INFO:tasks.workunit.client.0.smithi139.stdout:4/769: readlink d12/d2b/d8c/lc4 0 2024-09-16T00:09:50.030 INFO:tasks.workunit.client.0.smithi139.stdout:4/770: unlink d12/d47/db2/c4e 0 2024-09-16T00:09:50.063 INFO:tasks.workunit.client.0.smithi139.stdout:7/752: sync 2024-09-16T00:09:50.063 INFO:tasks.workunit.client.0.smithi139.stdout:8/815: sync 2024-09-16T00:09:50.063 INFO:tasks.workunit.client.0.smithi139.stdout:2/864: sync 2024-09-16T00:09:50.063 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:50.000+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:50.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.063+0000 7f9900f1c640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f98fc1f5610 0x7f98fc15ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.064+0000 7f9900f1c640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f98fc1f5610 0x7f98fc15ac10 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:50.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.065+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98fc1f4a10 con 0x7f98fc1f5610 2024-09-16T00:09:50.067 INFO:tasks.workunit.client.0.smithi139.stdout:7/753: symlink d5/d16/d6d/d7f/dc2/d50/d44/d55/l111 0 2024-09-16T00:09:50.068 INFO:tasks.workunit.client.0.smithi139.stdout:8/816: rmdir d17/d1a/d50/d86/d92 39 2024-09-16T00:09:50.069 INFO:tasks.workunit.client.0.smithi139.stdout:2/865: mknod d11/d30/dbf/c127 0 2024-09-16T00:09:50.069 INFO:tasks.workunit.client.0.smithi139.stdout:7/754: readlink d5/l49 0 2024-09-16T00:09:50.071 INFO:tasks.workunit.client.0.smithi139.stdout:8/817: dread - d17/d1a/f5e zero size 2024-09-16T00:09:50.073 INFO:tasks.workunit.client.0.smithi139.stdout:0/708: fsync d6/d1c/d41/d76/fdd 0 2024-09-16T00:09:50.073 INFO:tasks.workunit.client.0.smithi139.stdout:8/818: chown d17/d1a/f35 40588 1 2024-09-16T00:09:50.074 INFO:tasks.workunit.client.0.smithi139.stdout:7/755: write d5/d16/d6d/fd0 [612523,33409] 0 2024-09-16T00:09:50.075 INFO:tasks.workunit.client.0.smithi139.stdout:9/918: truncate d19/d1a/d20/d23/d2d/d2f/f96 6086282 0 2024-09-16T00:09:50.076 INFO:tasks.workunit.client.0.smithi139.stdout:2/866: mknod d11/d30/d110/d50/d114/c128 0 2024-09-16T00:09:50.078 INFO:tasks.workunit.client.0.smithi139.stdout:9/919: read d19/d86/f5a [2956872,103031] 0 2024-09-16T00:09:50.080 INFO:tasks.workunit.client.0.smithi139.stdout:4/771: dread d12/d47/db2/f63 [0,4194304] 0 2024-09-16T00:09:50.080 INFO:tasks.workunit.client.0.smithi139.stdout:7/756: mknod d5/d16/d22/d25/d63/c112 0 2024-09-16T00:09:50.081 INFO:tasks.workunit.client.0.smithi139.stdout:2/867: creat d11/d30/dbf/d116/d125/f129 x:0 0 0 2024-09-16T00:09:50.082 INFO:tasks.workunit.client.0.smithi139.stdout:7/757: fsync d5/d16/d6d/d7f/dc2/d50/f104 0 2024-09-16T00:09:50.084 INFO:tasks.workunit.client.0.smithi139.stdout:9/920: truncate d19/d1a/d20/d23/dca/d71/da3/fce 4712151 0 2024-09-16T00:09:50.085 INFO:tasks.workunit.client.0.smithi139.stdout:4/772: rename d12/d2b/la1 to d12/d47/db2/d4d/de4/d7f/lf7 0 2024-09-16T00:09:50.087 INFO:tasks.workunit.client.0.smithi139.stdout:0/709: dread d6/d1c/d41/d63/f89 [0,4194304] 0 2024-09-16T00:09:50.089 INFO:tasks.workunit.client.0.smithi139.stdout:2/868: rename d11/d30/dbf/dfe to d11/d30/d110/d29/d4f/d12a 0 2024-09-16T00:09:50.090 INFO:tasks.workunit.client.0.smithi139.stdout:9/921: creat d19/d1a/d20/d23/dca/d71/dc7/f132 x:0 0 0 2024-09-16T00:09:50.091 INFO:tasks.workunit.client.0.smithi139.stdout:4/773: creat d12/d47/db2/d4d/d69/ff8 x:0 0 0 2024-09-16T00:09:50.092 INFO:tasks.workunit.client.0.smithi139.stdout:2/869: read - d11/d30/dbf/d116/d125/f11e zero size 2024-09-16T00:09:50.093 INFO:tasks.workunit.client.0.smithi139.stdout:7/758: dwrite d5/d37/f79 [0,4194304] 0 2024-09-16T00:09:50.094 INFO:tasks.workunit.client.0.smithi139.stdout:8/819: dread d17/d89/df2/f107 [0,4194304] 0 2024-09-16T00:09:50.096 INFO:tasks.workunit.client.0.smithi139.stdout:9/922: creat d19/d1a/d20/d23/d2d/d35/f133 x:0 0 0 2024-09-16T00:09:50.097 INFO:tasks.workunit.client.0.smithi139.stdout:4/774: mknod d12/d47/db2/d4d/dd6/cf9 0 2024-09-16T00:09:50.097 INFO:tasks.workunit.client.0.smithi139.stdout:8/820: write d17/d1a/f5e [91352,126359] 0 2024-09-16T00:09:50.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.098+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98fc0c8a30 con 0x7f98fc1f5610 2024-09-16T00:09:50.099 INFO:tasks.workunit.client.0.smithi139.stdout:2/870: symlink d11/d30/d110/l12b 0 2024-09-16T00:09:50.099 INFO:tasks.workunit.client.0.smithi139.stdout:7/759: write d5/d16/d6d/d7f/dc2/d50/fbd [5083531,52606] 0 2024-09-16T00:09:50.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.100+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98fc0c8a30 con 0x7f98fc1f5610 2024-09-16T00:09:50.101 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:50.102 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:50.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.100+0000 7f9903981640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f98fc0c65a0 0x7f98fc0c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.101+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f98fc0c6ec0 con 0x7f98fc0c65a0 2024-09-16T00:09:50.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.101+0000 7f990171d640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f98fc0c65a0 0x7f98fc0c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.102 INFO:tasks.workunit.client.0.smithi139.stdout:8/821: write d17/d1f/d3a/db7/d40/f41 [1159457,121243] 0 2024-09-16T00:09:50.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.101+0000 7f990171d640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f98fc0c65a0 0x7f98fc0c6980 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:50.103 INFO:tasks.workunit.client.0.smithi139.stdout:7/760: symlink d5/d37/d70/l113 0 2024-09-16T00:09:50.103 INFO:tasks.workunit.client.0.smithi139.stdout:8/822: chown d17/d1a/d50 2011 1 2024-09-16T00:09:50.103 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:50.101+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:50.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.103+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98fc0c6ec0 con 0x7f98fc0c65a0 2024-09-16T00:09:50.104 INFO:tasks.workunit.client.0.smithi139.stdout:9/923: dread d19/d1a/d20/d23/dca/d71/d32/d4e/fb6 [0,4194304] 0 2024-09-16T00:09:50.104 INFO:tasks.workunit.client.0.smithi139.stdout:7/761: chown d5/d16/d22/fc7 41 1 2024-09-16T00:09:50.106 INFO:tasks.workunit.client.0.smithi139.stdout:2/871: symlink d11/d30/d110/d5d/d73/dc3/l12c 0 2024-09-16T00:09:50.107 INFO:tasks.workunit.client.0.smithi139.stdout:9/924: read - d19/d1a/f73 zero size 2024-09-16T00:09:50.109 INFO:tasks.workunit.client.0.smithi139.stdout:9/925: chown d19/d1a/d20/d23/c88 3537 1 2024-09-16T00:09:50.110 INFO:tasks.workunit.client.0.smithi139.stdout:8/823: mkdir d17/d1f/d10e 0 2024-09-16T00:09:50.110 INFO:tasks.workunit.client.0.smithi139.stdout:4/775: dread d12/d47/db2/d4d/de4/f6e [0,4194304] 0 2024-09-16T00:09:50.110 INFO:tasks.workunit.client.0.smithi139.stdout:2/872: mkdir d11/d12d 0 2024-09-16T00:09:50.111 INFO:tasks.workunit.client.0.smithi139.stdout:9/926: read d19/f6c [879879,94311] 0 2024-09-16T00:09:50.112 INFO:tasks.workunit.client.0.smithi139.stdout:4/776: stat d12/d47/db2/d4d/dd1 0 2024-09-16T00:09:50.112 INFO:tasks.workunit.client.0.smithi139.stdout:7/762: truncate d5/d16/d22/d25/f76 1790487 0 2024-09-16T00:09:50.113 INFO:tasks.workunit.client.0.smithi139.stdout:8/824: dread d17/f4a [0,4194304] 0 2024-09-16T00:09:50.113 INFO:tasks.workunit.client.0.smithi139.stdout:7/763: readlink d5/d26/d42/lef 0 2024-09-16T00:09:50.114 INFO:tasks.workunit.client.0.smithi139.stdout:4/777: creat d12/d47/db2/d4d/de4/d7f/d9e/ffa x:0 0 0 2024-09-16T00:09:50.116 INFO:tasks.workunit.client.0.smithi139.stdout:2/873: dread d11/d30/d110/d29/da5/fbe [0,4194304] 0 2024-09-16T00:09:50.116 INFO:tasks.workunit.client.0.smithi139.stdout:8/825: mknod d17/d1f/d10e/c10f 0 2024-09-16T00:09:50.118 INFO:tasks.workunit.client.0.smithi139.stdout:0/710: dread d6/d1c/d41/d44/f71 [0,4194304] 0 2024-09-16T00:09:50.118 INFO:tasks.workunit.client.0.smithi139.stdout:4/778: symlink d12/d47/db2/d4d/db0/lfb 0 2024-09-16T00:09:50.121 INFO:tasks.workunit.client.0.smithi139.stdout:0/711: symlink d6/d42/db4/ldf 0 2024-09-16T00:09:50.123 INFO:tasks.workunit.client.0.smithi139.stdout:8/826: rename d17/d1a/f1d to d17/d1a/d50/d86/d92/f110 0 2024-09-16T00:09:50.123 INFO:tasks.workunit.client.0.smithi139.stdout:2/874: dread d11/d2f/deb/ff7 [0,4194304] 0 2024-09-16T00:09:50.124 INFO:tasks.workunit.client.0.smithi139.stdout:7/764: dwrite d5/d16/d6d/d7f/dc2/d50/fbd [0,4194304] 0 2024-09-16T00:09:50.124 INFO:tasks.workunit.client.0.smithi139.stdout:0/712: write d6/d1c/fc1 [1176382,121316] 0 2024-09-16T00:09:50.127 INFO:tasks.workunit.client.0.smithi139.stdout:8/827: write d17/d1f/d3a/dad/fc5 [658183,65303] 0 2024-09-16T00:09:50.127 INFO:tasks.workunit.client.0.smithi139.stdout:7/765: write d5/d16/d87/feb [633245,15674] 0 2024-09-16T00:09:50.127 INFO:tasks.workunit.client.0.smithi139.stdout:0/713: chown d6/d1c/d41/d63/d82/lb2 1 1 2024-09-16T00:09:50.129 INFO:tasks.workunit.client.0.smithi139.stdout:4/779: getdents d12/d47/d49 0 2024-09-16T00:09:50.129 INFO:tasks.workunit.client.0.smithi139.stdout:2/875: write d11/d30/d110/d50/f6a [3784383,79047] 0 2024-09-16T00:09:50.131 INFO:tasks.workunit.client.0.smithi139.stdout:7/766: chown d5/cf 92161793 1 2024-09-16T00:09:50.132 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:50.132 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:50.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:50.133 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:50.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.133 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5850 3.3a 3:5c3cfc2b:::10000001adf.00000000:head [write 0~116] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:50.133 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:09:50.133 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 34.713612542, 2024-09-16T00:09:50.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66931295800000001, 2024-09-16T00:09:50.133 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.133 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.133 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.133 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.133 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.134 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.134 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5850 2024-09-16T00:09:50.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.134 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:09:50.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:09:50.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.135 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.135 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415938+0000", 2024-09-16T00:09:50.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7099999999999999e-06 2024-09-16T00:09:50.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.135 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.135 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415945+0000", 2024-09-16T00:09:50.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:09:50.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.135 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.136 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415959+0000", 2024-09-16T00:09:50.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3545e-05 2024-09-16T00:09:50.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.136 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.136 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415966+0000", 2024-09-16T00:09:50.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6379999999999998e-06 2024-09-16T00:09:50.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.136 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.136 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416013+0000", 2024-09-16T00:09:50.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7314999999999997e-05 2024-09-16T00:09:50.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:50.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416020+0000", 2024-09-16T00:09:50.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.52e-06 2024-09-16T00:09:50.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082393+0000", 2024-09-16T00:09:50.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66637251799999997 2024-09-16T00:09:50.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082537+0000", 2024-09-16T00:09:50.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014418800000000001 2024-09-16T00:09:50.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:50.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082689+0000", 2024-09-16T00:09:50.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015198499999999999 2024-09-16T00:09:50.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084930+0000", 2024-09-16T00:09:50.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002241041 2024-09-16T00:09:50.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084994+0000", 2024-09-16T00:09:50.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3274000000000004e-05 2024-09-16T00:09:50.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085049+0000", 2024-09-16T00:09:50.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.554e-05 2024-09-16T00:09:50.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085250+0000", 2024-09-16T00:09:50.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020060799999999999 2024-09-16T00:09:50.139 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.139 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.139 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.140 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.140 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5910 3.3a 3:5ec98a62:::10000001b22.00000000:head [write 0~1274] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:50.140 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:09:50.140 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 34.566668903999997, 2024-09-16T00:09:50.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.52286026600000002, 2024-09-16T00:09:50.140 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.140 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.140 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.140 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.140 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.140 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.141 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5910 2024-09-16T00:09:50.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.141 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:09:50.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:09:50.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562881+0000", 2024-09-16T00:09:50.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:09:50.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562887+0000", 2024-09-16T00:09:50.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2870000000000004e-06 2024-09-16T00:09:50.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562897+0000", 2024-09-16T00:09:50.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.064e-05 2024-09-16T00:09:50.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562902+0000", 2024-09-16T00:09:50.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2340000000000002e-06 2024-09-16T00:09:50.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562958+0000", 2024-09-16T00:09:50.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5523999999999999e-05 2024-09-16T00:09:50.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:50.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562968+0000", 2024-09-16T00:09:50.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0394000000000001e-05 2024-09-16T00:09:50.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083461+0000", 2024-09-16T00:09:50.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.520492968 2024-09-16T00:09:50.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083645+0000", 2024-09-16T00:09:50.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018346399999999999 2024-09-16T00:09:50.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.145 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:50.145 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083844+0000", 2024-09-16T00:09:50.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000199268 2024-09-16T00:09:50.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.145 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.145 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085410+0000", 2024-09-16T00:09:50.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015662359999999999 2024-09-16T00:09:50.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085493+0000", 2024-09-16T00:09:50.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2478999999999994e-05 2024-09-16T00:09:50.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085563+0000", 2024-09-16T00:09:50.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0159000000000004e-05 2024-09-16T00:09:50.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085741+0000", 2024-09-16T00:09:50.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017764300000000001 2024-09-16T00:09:50.146 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.147 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.147 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.147 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5962 3.3a 3:5c7b4d29:::10000001b5d.00000000:head [write 0~3074] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:50.147 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:09:50.147 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 34.437132087000002, 2024-09-16T00:09:50.147 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39368737500000001, 2024-09-16T00:09:50.147 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.147 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.147 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.147 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.148 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.148 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.148 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5962 2024-09-16T00:09:50.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.148 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:09:50.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:09:50.149 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.149 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692418+0000", 2024-09-16T00:09:50.149 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3510000000000001e-06 2024-09-16T00:09:50.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.149 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692425+0000", 2024-09-16T00:09:50.149 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3770000000000001e-06 2024-09-16T00:09:50.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.150 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.150 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692440+0000", 2024-09-16T00:09:50.150 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4975e-05 2024-09-16T00:09:50.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.150 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.150 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.150 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692448+0000", 2024-09-16T00:09:50.150 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6279999999999994e-06 2024-09-16T00:09:50.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.150 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.150 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.150 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692499+0000", 2024-09-16T00:09:50.150 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0872999999999999e-05 2024-09-16T00:09:50.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.151 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.151 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:50.151 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692507+0000", 2024-09-16T00:09:50.151 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9889999999999992e-06 2024-09-16T00:09:50.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.151 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.151 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.151 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084221+0000", 2024-09-16T00:09:50.151 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39171379699999997 2024-09-16T00:09:50.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.151 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.151 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.151 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084446+0000", 2024-09-16T00:09:50.151 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022519900000000001 2024-09-16T00:09:50.152 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.152 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.152 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:50.152 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084672+0000", 2024-09-16T00:09:50.152 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000225405 2024-09-16T00:09:50.152 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.152 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.152 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.152 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085832+0000", 2024-09-16T00:09:50.152 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011600040000000001 2024-09-16T00:09:50.152 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.152 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.152 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.153 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085864+0000", 2024-09-16T00:09:50.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2592999999999998e-05 2024-09-16T00:09:50.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.153 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.153 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.153 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085943+0000", 2024-09-16T00:09:50.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8534000000000004e-05 2024-09-16T00:09:50.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.153 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.153 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.153 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086105+0000", 2024-09-16T00:09:50.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016165 2024-09-16T00:09:50.153 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.153 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.154 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.154 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.154 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5963 3.3a 3:5e4192f5:::10000001b5e.00000000:head [write 0~152] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:50.154 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:50.154 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 34.435067656000001, 2024-09-16T00:09:50.154 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39413463900000001, 2024-09-16T00:09:50.154 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.154 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.154 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.154 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.154 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.154 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.154 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5963 2024-09-16T00:09:50.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.155 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:50.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:50.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.156 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.156 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:50.156 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:09:50.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.156 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.156 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.156 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694486+0000", 2024-09-16T00:09:50.156 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0960000000000003e-06 2024-09-16T00:09:50.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.156 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.156 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.156 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694496+0000", 2024-09-16T00:09:50.156 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2189999999999993e-06 2024-09-16T00:09:50.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694500+0000", 2024-09-16T00:09:50.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9529999999999998e-06 2024-09-16T00:09:50.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694552+0000", 2024-09-16T00:09:50.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1360999999999998e-05 2024-09-16T00:09:50.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:50.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694561+0000", 2024-09-16T00:09:50.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1700000000000003e-06 2024-09-16T00:09:50.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.158 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.158 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086346+0000", 2024-09-16T00:09:50.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39178468 2024-09-16T00:09:50.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.158 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.158 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086480+0000", 2024-09-16T00:09:50.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013469299999999999 2024-09-16T00:09:50.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.158 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:50.159 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086790+0000", 2024-09-16T00:09:50.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030923300000000001 2024-09-16T00:09:50.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.159 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.159 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.159 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088361+0000", 2024-09-16T00:09:50.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015717050000000001 2024-09-16T00:09:50.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.159 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.159 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.159 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088432+0000", 2024-09-16T00:09:50.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0922999999999998e-05 2024-09-16T00:09:50.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.160 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.160 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088489+0000", 2024-09-16T00:09:50.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6653999999999997e-05 2024-09-16T00:09:50.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.160 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.160 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088616+0000", 2024-09-16T00:09:50.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012730099999999999 2024-09-16T00:09:50.160 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.160 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.160 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.161 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.161 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:6082 3.3a 3:5c50f65b:::10000001be7.00000000:head [write 0~2439] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:50.161 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:09:50.161 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 34.141294799999997, 2024-09-16T00:09:50.161 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.10065850799999999, 2024-09-16T00:09:50.161 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.161 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.161 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.161 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.161 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.161 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.161 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 6082 2024-09-16T00:09:50.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.162 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:09:50.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:09:50.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988256+0000", 2024-09-16T00:09:50.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.065e-06 2024-09-16T00:09:50.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988262+0000", 2024-09-16T00:09:50.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4640000000000003e-06 2024-09-16T00:09:50.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988272+0000", 2024-09-16T00:09:50.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0335e-05 2024-09-16T00:09:50.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.164 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.164 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.164 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988279+0000", 2024-09-16T00:09:50.164 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:09:50.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.164 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.164 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.164 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988324+0000", 2024-09-16T00:09:50.164 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4953000000000001e-05 2024-09-16T00:09:50.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.164 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.164 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:50.165 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988331+0000", 2024-09-16T00:09:50.165 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5989999999999998e-06 2024-09-16T00:09:50.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.165 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.165 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087272+0000", 2024-09-16T00:09:50.165 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.098941045000000005 2024-09-16T00:09:50.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.165 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.165 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087486+0000", 2024-09-16T00:09:50.165 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00021360199999999999 2024-09-16T00:09:50.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:50.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087740+0000", 2024-09-16T00:09:50.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00025456799999999999 2024-09-16T00:09:50.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088638+0000", 2024-09-16T00:09:50.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00089793900000000005 2024-09-16T00:09:50.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088673+0000", 2024-09-16T00:09:50.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5111999999999999e-05 2024-09-16T00:09:50.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088722+0000", 2024-09-16T00:09:50.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8758000000000003e-05 2024-09-16T00:09:50.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088913+0000", 2024-09-16T00:09:50.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000190953 2024-09-16T00:09:50.167 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.167 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.167 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.167 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10643 3.21 3:8453163e:::100000030b3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:50.168 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:09:50.168 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.529818606999999, 2024-09-16T00:09:50.168 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013028809000000001, 2024-09-16T00:09:50.168 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.168 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.168 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.168 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.168 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.168 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.168 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10643 2024-09-16T00:09:50.168 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.168 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.168 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.169 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:09:50.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.169 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:09:50.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.169 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599734+0000", 2024-09-16T00:09:50.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-06 2024-09-16T00:09:50.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.170 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.170 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.170 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603345+0000", 2024-09-16T00:09:50.170 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003611628 2024-09-16T00:09:50.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.170 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.170 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.170 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603368+0000", 2024-09-16T00:09:50.170 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3017e-05 2024-09-16T00:09:50.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.170 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.170 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603379+0000", 2024-09-16T00:09:50.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0528e-05 2024-09-16T00:09:50.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603448+0000", 2024-09-16T00:09:50.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9080999999999998e-05 2024-09-16T00:09:50.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603623+0000", 2024-09-16T00:09:50.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017525599999999999 2024-09-16T00:09:50.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.172 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:50.172 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603766+0000", 2024-09-16T00:09:50.172 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000142977 2024-09-16T00:09:50.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.172 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.172 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.611250+0000", 2024-09-16T00:09:50.172 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074835240000000001 2024-09-16T00:09:50.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.173 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.173 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612617+0000", 2024-09-16T00:09:50.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001367793 2024-09-16T00:09:50.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.173 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.173 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.173 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612700+0000", 2024-09-16T00:09:50.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3123999999999999e-05 2024-09-16T00:09:50.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.173 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.173 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.173 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612759+0000", 2024-09-16T00:09:50.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8950999999999999e-05 2024-09-16T00:09:50.174 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.174 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.174 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.174 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.174 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.174 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10652 3.22 3:44e46336:::100000030c1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:50.174 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:09:50.174 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.501766975000001, 2024-09-16T00:09:50.174 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011405732, 2024-09-16T00:09:50.174 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.174 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.174 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.174 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.174 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.175 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.175 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10652 2024-09-16T00:09:50.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.175 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:09:50.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:09:50.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627785+0000", 2024-09-16T00:09:50.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6970000000000002e-06 2024-09-16T00:09:50.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629610+0000", 2024-09-16T00:09:50.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018250359999999999 2024-09-16T00:09:50.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629639+0000", 2024-09-16T00:09:50.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8676000000000001e-05 2024-09-16T00:09:50.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629652+0000", 2024-09-16T00:09:50.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3572e-05 2024-09-16T00:09:50.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629735+0000", 2024-09-16T00:09:50.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.25e-05 2024-09-16T00:09:50.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629925+0000", 2024-09-16T00:09:50.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019011000000000001 2024-09-16T00:09:50.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:50.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.630075+0000", 2024-09-16T00:09:50.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000149841 2024-09-16T00:09:50.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.636470+0000", 2024-09-16T00:09:50.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063952289999999997 2024-09-16T00:09:50.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.179 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639098+0000", 2024-09-16T00:09:50.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026277980000000002 2024-09-16T00:09:50.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.179 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639154+0000", 2024-09-16T00:09:50.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6413999999999999e-05 2024-09-16T00:09:50.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.180 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639188+0000", 2024-09-16T00:09:50.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3859000000000002e-05 2024-09-16T00:09:50.180 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.180 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.180 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.180 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10951 3.22 3:44fbf03a:::10000003157.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:50.180 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.497985+0000", 2024-09-16T00:09:50.180 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 10.631564389999999, 2024-09-16T00:09:50.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010873974999999999, 2024-09-16T00:09:50.181 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.181 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.181 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.181 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.181 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.181 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.181 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10951 2024-09-16T00:09:50.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.181 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.497985+0000", 2024-09-16T00:09:50.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.497985+0000", 2024-09-16T00:09:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.497988+0000", 2024-09-16T00:09:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9749999999999999e-06 2024-09-16T00:09:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.499736+0000", 2024-09-16T00:09:50.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001747845 2024-09-16T00:09:50.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.499771+0000", 2024-09-16T00:09:50.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5199e-05 2024-09-16T00:09:50.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.499785+0000", 2024-09-16T00:09:50.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.364e-05 2024-09-16T00:09:50.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.499893+0000", 2024-09-16T00:09:50.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108348 2024-09-16T00:09:50.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.500019+0000", 2024-09-16T00:09:50.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012591699999999999 2024-09-16T00:09:50.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:50.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.500150+0000", 2024-09-16T00:09:50.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131163 2024-09-16T00:09:50.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.504740+0000", 2024-09-16T00:09:50.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045898140000000002 2024-09-16T00:09:50.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.508724+0000", 2024-09-16T00:09:50.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039840220000000003 2024-09-16T00:09:50.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.186 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.186 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.508794+0000", 2024-09-16T00:09:50.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9793000000000001e-05 2024-09-16T00:09:50.186 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.186 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.186 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.508859+0000", 2024-09-16T00:09:50.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5259000000000007e-05 2024-09-16T00:09:50.186 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.186 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.186 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.186 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.187 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11217 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:50.187 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:09:50.187 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.5552699539999999, 2024-09-16T00:09:50.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01231253, 2024-09-16T00:09:50.187 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.187 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.187 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.187 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.187 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.187 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.187 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11217 2024-09-16T00:09:50.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.188 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:09:50.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:09:50.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574282+0000", 2024-09-16T00:09:50.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8499999999999998e-06 2024-09-16T00:09:50.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575804+0000", 2024-09-16T00:09:50.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001521859 2024-09-16T00:09:50.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575826+0000", 2024-09-16T00:09:50.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2059999999999999e-05 2024-09-16T00:09:50.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575837+0000", 2024-09-16T00:09:50.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1313e-05 2024-09-16T00:09:50.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575920+0000", 2024-09-16T00:09:50.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2918999999999997e-05 2024-09-16T00:09:50.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576088+0000", 2024-09-16T00:09:50.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016803799999999999 2024-09-16T00:09:50.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:50.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576233+0000", 2024-09-16T00:09:50.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014508199999999999 2024-09-16T00:09:50.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.580500+0000", 2024-09-16T00:09:50.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004266644 2024-09-16T00:09:50.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586479+0000", 2024-09-16T00:09:50.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059791820000000004 2024-09-16T00:09:50.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.192 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586541+0000", 2024-09-16T00:09:50.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1463000000000004e-05 2024-09-16T00:09:50.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.192 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586592+0000", 2024-09-16T00:09:50.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1119999999999998e-05 2024-09-16T00:09:50.192 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.192 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.192 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.193 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11767 3.38 3:1fa50e65:::10000003117.00000000:head [write 0~4194304 [3@1169732]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:50.193 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:09:50.193 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.965865419, 2024-09-16T00:09:50.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011555678999999999, 2024-09-16T00:09:50.193 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.193 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.193 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.193 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.193 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.193 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.193 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11767 2024-09-16T00:09:50.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.194 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:09:50.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:09:50.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163686+0000", 2024-09-16T00:09:50.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3259999999999998e-06 2024-09-16T00:09:50.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165053+0000", 2024-09-16T00:09:50.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013665859999999999 2024-09-16T00:09:50.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165075+0000", 2024-09-16T00:09:50.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2206999999999999e-05 2024-09-16T00:09:50.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165084+0000", 2024-09-16T00:09:50.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0710000000000001e-06 2024-09-16T00:09:50.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165157+0000", 2024-09-16T00:09:50.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2619000000000004e-05 2024-09-16T00:09:50.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165249+0000", 2024-09-16T00:09:50.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2517999999999997e-05 2024-09-16T00:09:50.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:50.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165360+0000", 2024-09-16T00:09:50.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011134899999999999 2024-09-16T00:09:50.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.169320+0000", 2024-09-16T00:09:50.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039593040000000003 2024-09-16T00:09:50.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175135+0000", 2024-09-16T00:09:50.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005814891 2024-09-16T00:09:50.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175199+0000", 2024-09-16T00:09:50.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4474999999999995e-05 2024-09-16T00:09:50.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175239+0000", 2024-09-16T00:09:50.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0333000000000002e-05 2024-09-16T00:09:50.198 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.199 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.199 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.199 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11789 3.21 3:8453163e:::100000030b3.00000000:head [write 0~3801088 [4@1194504]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:50.199 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:09:50.199 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.8781773230000001, 2024-09-16T00:09:50.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011097602, 2024-09-16T00:09:50.199 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.199 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.199 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.199 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.199 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.200 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.200 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11789 2024-09-16T00:09:50.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.200 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:09:50.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:09:50.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251373+0000", 2024-09-16T00:09:50.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.544e-06 2024-09-16T00:09:50.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252710+0000", 2024-09-16T00:09:50.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013362929999999999 2024-09-16T00:09:50.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252730+0000", 2024-09-16T00:09:50.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0579999999999999e-05 2024-09-16T00:09:50.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252741+0000", 2024-09-16T00:09:50.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0505e-05 2024-09-16T00:09:50.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252820+0000", 2024-09-16T00:09:50.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9098999999999996e-05 2024-09-16T00:09:50.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252931+0000", 2024-09-16T00:09:50.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011073600000000001 2024-09-16T00:09:50.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:50.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.253101+0000", 2024-09-16T00:09:50.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017038000000000001 2024-09-16T00:09:50.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.261813+0000", 2024-09-16T00:09:50.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087119469999999994 2024-09-16T00:09:50.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262338+0000", 2024-09-16T00:09:50.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00052493799999999997 2024-09-16T00:09:50.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262430+0000", 2024-09-16T00:09:50.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1626999999999995e-05 2024-09-16T00:09:50.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.205 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262470+0000", 2024-09-16T00:09:50.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9953000000000002e-05 2024-09-16T00:09:50.205 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.205 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.205 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.205 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11890 3.38 3:1e014616:::100000033e9.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:50.205 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:09:50.205 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.5582122409999997, 2024-09-16T00:09:50.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011833791999999999, 2024-09-16T00:09:50.206 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.206 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.206 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.206 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.206 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.206 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.206 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11890 2024-09-16T00:09:50.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.206 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:09:50.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:09:50.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571340+0000", 2024-09-16T00:09:50.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.36e-06 2024-09-16T00:09:50.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572866+0000", 2024-09-16T00:09:50.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001525549 2024-09-16T00:09:50.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572884+0000", 2024-09-16T00:09:50.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7796999999999999e-05 2024-09-16T00:09:50.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572897+0000", 2024-09-16T00:09:50.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3567000000000001e-05 2024-09-16T00:09:50.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572970+0000", 2024-09-16T00:09:50.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2751999999999997e-05 2024-09-16T00:09:50.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573053+0000", 2024-09-16T00:09:50.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3230000000000001e-05 2024-09-16T00:09:50.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:50.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573168+0000", 2024-09-16T00:09:50.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114536 2024-09-16T00:09:50.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.577923+0000", 2024-09-16T00:09:50.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047548720000000003 2024-09-16T00:09:50.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583081+0000", 2024-09-16T00:09:50.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051587630000000002 2024-09-16T00:09:50.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583137+0000", 2024-09-16T00:09:50.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5259000000000001e-05 2024-09-16T00:09:50.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583171+0000", 2024-09-16T00:09:50.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4106999999999997e-05 2024-09-16T00:09:50.211 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.211 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.211 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.212 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11956 3.21 3:8673bb5f:::100000030ed.00000000:head [write 0~4194304 [2@1395527]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:50.212 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.776259+0000", 2024-09-16T00:09:50.212 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.3532906640000002, 2024-09-16T00:09:50.212 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010483716000000001, 2024-09-16T00:09:50.212 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.212 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.212 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.212 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.212 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.212 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.212 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11956 2024-09-16T00:09:50.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.213 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.776259+0000", 2024-09-16T00:09:50.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.776259+0000", 2024-09-16T00:09:50.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.776262+0000", 2024-09-16T00:09:50.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1130000000000001e-06 2024-09-16T00:09:50.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.778195+0000", 2024-09-16T00:09:50.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019330630000000001 2024-09-16T00:09:50.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.778214+0000", 2024-09-16T00:09:50.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9663000000000001e-05 2024-09-16T00:09:50.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.778225+0000", 2024-09-16T00:09:50.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0190000000000001e-05 2024-09-16T00:09:50.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.778280+0000", 2024-09-16T00:09:50.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5566000000000003e-05 2024-09-16T00:09:50.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.778377+0000", 2024-09-16T00:09:50.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6343999999999998e-05 2024-09-16T00:09:50.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:50.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.778504+0000", 2024-09-16T00:09:50.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012727699999999999 2024-09-16T00:09:50.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.784281+0000", 2024-09-16T00:09:50.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057773549999999996 2024-09-16T00:09:50.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.786638+0000", 2024-09-16T00:09:50.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023566139999999999 2024-09-16T00:09:50.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.786692+0000", 2024-09-16T00:09:50.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3947999999999999e-05 2024-09-16T00:09:50.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.786742+0000", 2024-09-16T00:09:50.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0583e-05 2024-09-16T00:09:50.217 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.217 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.218 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.218 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:12037 3.9 e87/20 3:928db521:::1000000326d.00000000:head v 87'199, mlcod=87'197)", 2024-09-16T00:09:50.218 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:09:50.218 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.1270222900000002, 2024-09-16T00:09:50.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012236153, 2024-09-16T00:09:50.218 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.218 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.218 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.218 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.219 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.219 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:09:50.219 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.219 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.219 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.219 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:09:50.219 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.219 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.219 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.219 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002529+0000", 2024-09-16T00:09:50.219 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4889999999999998e-06 2024-09-16T00:09:50.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007064+0000", 2024-09-16T00:09:50.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045346220000000003 2024-09-16T00:09:50.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007093+0000", 2024-09-16T00:09:50.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9009e-05 2024-09-16T00:09:50.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007109+0000", 2024-09-16T00:09:50.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5846e-05 2024-09-16T00:09:50.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007196+0000", 2024-09-16T00:09:50.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7507000000000001e-05 2024-09-16T00:09:50.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007227+0000", 2024-09-16T00:09:50.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0916e-05 2024-09-16T00:09:50.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014658+0000", 2024-09-16T00:09:50.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074305090000000001 2024-09-16T00:09:50.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014763+0000", 2024-09-16T00:09:50.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010525499999999999 2024-09-16T00:09:50.222 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.222 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.222 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.223 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12160 3.38 3:1d9b8036:::10000003347.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:50.223 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.461114+0000", 2024-09-16T00:09:50.223 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.6684353679999999, 2024-09-16T00:09:50.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010438151, 2024-09-16T00:09:50.223 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.223 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.223 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.223 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.223 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.223 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.223 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12160 2024-09-16T00:09:50.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.223 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.461114+0000", 2024-09-16T00:09:50.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.461114+0000", 2024-09-16T00:09:50.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.461119+0000", 2024-09-16T00:09:50.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1399999999999999e-06 2024-09-16T00:09:50.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.462954+0000", 2024-09-16T00:09:50.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018347839999999999 2024-09-16T00:09:50.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.462979+0000", 2024-09-16T00:09:50.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4927999999999999e-05 2024-09-16T00:09:50.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.462989+0000", 2024-09-16T00:09:50.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0263e-05 2024-09-16T00:09:50.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.463073+0000", 2024-09-16T00:09:50.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4332999999999994e-05 2024-09-16T00:09:50.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.463164+0000", 2024-09-16T00:09:50.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0600000000000007e-05 2024-09-16T00:09:50.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:50.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.463320+0000", 2024-09-16T00:09:50.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015587500000000001 2024-09-16T00:09:50.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.468317+0000", 2024-09-16T00:09:50.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0049971410000000001 2024-09-16T00:09:50.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.471472+0000", 2024-09-16T00:09:50.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031552630000000002 2024-09-16T00:09:50.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.471511+0000", 2024-09-16T00:09:50.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8364000000000001e-05 2024-09-16T00:09:50.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.229 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.229 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.471552+0000", 2024-09-16T00:09:50.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1459999999999999e-05 2024-09-16T00:09:50.229 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.229 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.229 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.229 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12398 3.2e 3:77a11d08:::10000003432.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:50.230 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:09:50.230 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.8883435070000001, 2024-09-16T00:09:50.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012092999, 2024-09-16T00:09:50.230 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.230 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.230 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.230 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.230 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.230 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.230 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12398 2024-09-16T00:09:50.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.230 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:09:50.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:09:50.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241209+0000", 2024-09-16T00:09:50.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2710000000000001e-06 2024-09-16T00:09:50.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243840+0000", 2024-09-16T00:09:50.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026307919999999999 2024-09-16T00:09:50.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243872+0000", 2024-09-16T00:09:50.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2699e-05 2024-09-16T00:09:50.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243887+0000", 2024-09-16T00:09:50.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4348000000000001e-05 2024-09-16T00:09:50.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243972+0000", 2024-09-16T00:09:50.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5358999999999994e-05 2024-09-16T00:09:50.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244132+0000", 2024-09-16T00:09:50.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015972499999999999 2024-09-16T00:09:50.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253153+0000", 2024-09-16T00:09:50.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0090210280000000004 2024-09-16T00:09:50.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253213+0000", 2024-09-16T00:09:50.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0300999999999997e-05 2024-09-16T00:09:50.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253299+0000", 2024-09-16T00:09:50.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5476000000000006e-05 2024-09-16T00:09:50.235 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.235 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.235 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.235 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12400 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304 [2@815045]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:50.235 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:09:50.235 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.8853057729999998, 2024-09-16T00:09:50.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011140066000000001, 2024-09-16T00:09:50.235 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.236 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.236 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.236 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.236 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.236 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.236 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12400 2024-09-16T00:09:50.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.236 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.236 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:09:50.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:09:50.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244245+0000", 2024-09-16T00:09:50.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.248e-06 2024-09-16T00:09:50.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246467+0000", 2024-09-16T00:09:50.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022217949999999999 2024-09-16T00:09:50.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246496+0000", 2024-09-16T00:09:50.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9484000000000001e-05 2024-09-16T00:09:50.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246509+0000", 2024-09-16T00:09:50.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3390000000000001e-05 2024-09-16T00:09:50.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246586+0000", 2024-09-16T00:09:50.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6692000000000004e-05 2024-09-16T00:09:50.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246680+0000", 2024-09-16T00:09:50.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3468999999999996e-05 2024-09-16T00:09:50.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:50.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246850+0000", 2024-09-16T00:09:50.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170206 2024-09-16T00:09:50.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253478+0000", 2024-09-16T00:09:50.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066287530000000002 2024-09-16T00:09:50.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255318+0000", 2024-09-16T00:09:50.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001839292 2024-09-16T00:09:50.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255354+0000", 2024-09-16T00:09:50.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5964999999999999e-05 2024-09-16T00:09:50.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255384+0000", 2024-09-16T00:09:50.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9771999999999999e-05 2024-09-16T00:09:50.241 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.241 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.241 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.242 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13317 3.38 3:1cda8244:::100000035dc.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:50.242 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:09:50.242 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 2.2109512470000001, 2024-09-16T00:09:50.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011377063, 2024-09-16T00:09:50.242 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.242 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.242 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.242 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.242 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.242 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.242 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13317 2024-09-16T00:09:50.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.242 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:09:50.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:09:50.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918602+0000", 2024-09-16T00:09:50.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8160000000000004e-06 2024-09-16T00:09:50.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920364+0000", 2024-09-16T00:09:50.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017623630000000001 2024-09-16T00:09:50.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920391+0000", 2024-09-16T00:09:50.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7248000000000001e-05 2024-09-16T00:09:50.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920405+0000", 2024-09-16T00:09:50.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3969e-05 2024-09-16T00:09:50.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920486+0000", 2024-09-16T00:09:50.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0737999999999998e-05 2024-09-16T00:09:50.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920594+0000", 2024-09-16T00:09:50.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108269 2024-09-16T00:09:50.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:50.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920730+0000", 2024-09-16T00:09:50.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000135229 2024-09-16T00:09:50.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.928847+0000", 2024-09-16T00:09:50.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081178299999999995 2024-09-16T00:09:50.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929862+0000", 2024-09-16T00:09:50.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010147540000000001 2024-09-16T00:09:50.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929923+0000", 2024-09-16T00:09:50.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0452999999999999e-05 2024-09-16T00:09:50.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929975+0000", 2024-09-16T00:09:50.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2394e-05 2024-09-16T00:09:50.247 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.247 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.247 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.248 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:13326 3.9 e88/20 3:92f796cf:::1000000350f.00000000:head v 88'214, mlcod=88'212)", 2024-09-16T00:09:50.248 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:09:50.248 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 2.1684304559999998, 2024-09-16T00:09:50.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01100597, 2024-09-16T00:09:50.248 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.248 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.248 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.248 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:09:50.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:09:50.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961121+0000", 2024-09-16T00:09:50.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.815e-06 2024-09-16T00:09:50.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965747+0000", 2024-09-16T00:09:50.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0046260570000000003 2024-09-16T00:09:50.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965780+0000", 2024-09-16T00:09:50.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2851000000000001e-05 2024-09-16T00:09:50.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965788+0000", 2024-09-16T00:09:50.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0180000000000001e-06 2024-09-16T00:09:50.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965859+0000", 2024-09-16T00:09:50.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1318999999999999e-05 2024-09-16T00:09:50.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965886+0000", 2024-09-16T00:09:50.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7509999999999999e-05 2024-09-16T00:09:50.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.972066+0000", 2024-09-16T00:09:50.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061800759999999996 2024-09-16T00:09:50.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.972125+0000", 2024-09-16T00:09:50.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8323999999999999e-05 2024-09-16T00:09:50.252 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.252 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.252 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.252 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13942 3.22 3:44ed4521:::100000036d5.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:50.252 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:09:50.252 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.20706514400000001, 2024-09-16T00:09:50.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011341061, 2024-09-16T00:09:50.253 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:50.253 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:50.253 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:50.253 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:50.253 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:50.253 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:50.253 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13942 2024-09-16T00:09:50.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.253 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:50.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:50.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:09:50.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:50.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:09:50.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:50.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:50.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922487+0000", 2024-09-16T00:09:50.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0180000000000002e-06 2024-09-16T00:09:50.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:50.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923720+0000", 2024-09-16T00:09:50.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012333470000000001 2024-09-16T00:09:50.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:50.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923749+0000", 2024-09-16T00:09:50.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8748999999999999e-05 2024-09-16T00:09:50.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:50.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923770+0000", 2024-09-16T00:09:50.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1189999999999999e-05 2024-09-16T00:09:50.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:50.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923868+0000", 2024-09-16T00:09:50.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7675000000000002e-05 2024-09-16T00:09:50.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:50.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.924047+0000", 2024-09-16T00:09:50.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017935100000000001 2024-09-16T00:09:50.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:50.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.924223+0000", 2024-09-16T00:09:50.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000175661 2024-09-16T00:09:50.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:50.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.931544+0000", 2024-09-16T00:09:50.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073204589999999996 2024-09-16T00:09:50.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:50.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933692+0000", 2024-09-16T00:09:50.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021488760000000001 2024-09-16T00:09:50.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:50.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933774+0000", 2024-09-16T00:09:50.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1855999999999997e-05 2024-09-16T00:09:50.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:50.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:50.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:50.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933825+0000", 2024-09-16T00:09:50.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0878999999999999e-05 2024-09-16T00:09:50.258 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.258 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.258 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.258 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:50.258 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:50.258 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:50.259 INFO:tasks.workunit.client.0.smithi139.stdout:4/780: write d12/d47/d49/d60/dac/fdb [4636588,8132] 0 2024-09-16T00:09:50.259 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.132+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98fc0c7790 con 0x7f98fc0c65a0 2024-09-16T00:09:50.259 INFO:tasks.workunit.client.0.smithi139.stdout:2/876: creat d11/d30/d110/d29/d4f/d9c/f12e x:0 0 0 2024-09-16T00:09:50.260 INFO:tasks.workunit.client.0.smithi139.stdout:0/714: symlink d6/d1c/d41/d44/dd/le0 0 2024-09-16T00:09:50.260 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.135+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98fc0c7790 con 0x7f98fc0c65a0 2024-09-16T00:09:50.260 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:50.261 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:50.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.135+0000 7f9903981640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f98fc150200 0x7f98fc150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.135+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f98fc15b440 con 0x7f98fc150200 2024-09-16T00:09:50.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.136+0000 7f9900f1c640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f98fc150200 0x7f98fc150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.261 INFO:tasks.workunit.client.0.smithi139.stdout:4/781: write d12/d47/db2/d4d/d69/f8a [8355213,42668] 0 2024-09-16T00:09:50.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.136+0000 7f9900f1c640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f98fc150200 0x7f98fc150620 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:50.262 INFO:tasks.workunit.client.0.smithi139.stdout:2/877: creat d11/d2f/d6e/de5/f12f x:0 0 0 2024-09-16T00:09:50.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.137+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98fc15b440 con 0x7f98fc150200 2024-09-16T00:09:50.263 INFO:tasks.workunit.client.0.smithi139.stdout:8/828: link d17/d1a/d57/f71 d17/d1a/f111 0 2024-09-16T00:09:50.263 INFO:tasks.workunit.client.0.smithi139.stdout:4/782: creat d12/d47/d49/ffc x:0 0 0 2024-09-16T00:09:50.263 INFO:tasks.workunit.client.0.smithi139.stdout:0/715: truncate d6/d5a/f65 303256 0 2024-09-16T00:09:50.264 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:09:50.265 INFO:tasks.workunit.client.0.smithi139.stdout:2/878: rename d11/d32/f112 to d11/d2f/dcc/f130 0 2024-09-16T00:09:50.265 INFO:tasks.workunit.client.0.smithi139.stdout:4/783: creat d12/d47/db2/d4d/de4/ffd x:0 0 0 2024-09-16T00:09:50.266 INFO:tasks.workunit.client.0.smithi139.stdout:2/879: write d11/d30/d110/d50/f6a [3982388,69521] 0 2024-09-16T00:09:50.266 INFO:tasks.workunit.client.0.smithi139.stdout:7/767: dread d5/f21 [0,4194304] 0 2024-09-16T00:09:50.266 INFO:tasks.workunit.client.0.smithi139.stdout:8/829: dread d17/d54/fb6 [0,4194304] 0 2024-09-16T00:09:50.266 INFO:tasks.workunit.client.0.smithi139.stdout:0/716: dwrite d6/d1c/d41/d44/f3a [0,4194304] 0 2024-09-16T00:09:50.266 INFO:tasks.workunit.client.0.smithi139.stdout:2/880: fsync d11/d45/f7c 0 2024-09-16T00:09:50.266 INFO:tasks.workunit.client.0.smithi139.stdout:0/717: chown d6/d1c/d41/d44/d6b 746 1 2024-09-16T00:09:50.266 INFO:tasks.workunit.client.0.smithi139.stdout:7/768: creat d5/d26/d46/dea/df8/f114 x:0 0 0 2024-09-16T00:09:50.266 INFO:tasks.workunit.client.0.smithi139.stdout:2/881: chown d11/c7b 2859 1 2024-09-16T00:09:50.266 INFO:tasks.workunit.client.0.smithi139.stdout:7/769: dread - d5/d16/d22/d25/f7a zero size 2024-09-16T00:09:50.266 INFO:tasks.workunit.client.0.smithi139.stdout:4/784: write d12/d47/d49/f7e [4373379,96387] 0 2024-09-16T00:09:50.266 INFO:tasks.workunit.client.0.smithi139.stdout:2/882: write d11/d30/d110/d5d/d73/f79 [403310,72201] 0 2024-09-16T00:09:50.266 INFO:tasks.workunit.client.0.smithi139.stdout:8/830: link d17/d89/ff7 d17/d1f/d3a/db7/d40/f112 0 2024-09-16T00:09:50.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.162+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98fc062630 con 0x7f98fc150200 2024-09-16T00:09:50.267 INFO:tasks.workunit.client.0.smithi139.stdout:4/785: rmdir d12/d47/db2 39 2024-09-16T00:09:50.267 INFO:tasks.workunit.client.0.smithi139.stdout:0/718: getdents d6/d1c/d41/d78 0 2024-09-16T00:09:50.267 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.164+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98fc062630 con 0x7f98fc150200 2024-09-16T00:09:50.268 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:50.268 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:50.268 INFO:tasks.workunit.client.0.smithi139.stdout:7/770: unlink d5/d16/d22/d25/l29 0 2024-09-16T00:09:50.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.165+0000 7f9903981640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f98fc063110 0x7f98fc0cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.165+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f98fc062630 con 0x7f98fc063110 2024-09-16T00:09:50.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.165+0000 7f9901f1e640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f98fc063110 0x7f98fc0cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.165+0000 7f9901f1e640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f98fc063110 0x7f98fc0cbcc0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:50.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.166+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98fc062630 con 0x7f98fc063110 2024-09-16T00:09:50.270 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:50.262+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:50.271 INFO:tasks.workunit.client.0.smithi139.stdout:2/883: mkdir d11/d30/d110/d5d/d131 0 2024-09-16T00:09:50.271 INFO:tasks.workunit.client.0.smithi139.stdout:4/786: readlink d12/d47/db2/d4d/de4/d7f/l8b 0 2024-09-16T00:09:50.271 INFO:tasks.workunit.client.0.smithi139.stdout:0/719: mkdir d6/d1c/d41/d44/dd/d56/d75/de1 0 2024-09-16T00:09:50.272 INFO:tasks.workunit.client.0.smithi139.stdout:8/831: creat d17/d1a/d50/d86/f113 x:0 0 0 2024-09-16T00:09:50.272 INFO:tasks.workunit.client.0.smithi139.stdout:7/771: link d5/d26/d46/d6b/da7/fe2 d5/d26/d5d/f115 0 2024-09-16T00:09:50.272 INFO:tasks.workunit.client.0.smithi139.stdout:4/787: symlink d12/d47/db2/d4d/d69/lfe 0 2024-09-16T00:09:50.272 INFO:tasks.workunit.client.0.smithi139.stdout:0/720: symlink d6/d1c/d41/d78/le2 0 2024-09-16T00:09:50.272 INFO:tasks.workunit.client.0.smithi139.stdout:4/788: mkdir d12/d2b/d8c/dff 0 2024-09-16T00:09:50.272 INFO:tasks.workunit.client.0.smithi139.stdout:2/884: dwrite f2 [4194304,4194304] 0 2024-09-16T00:09:50.272 INFO:tasks.workunit.client.0.smithi139.stdout:0/721: unlink d6/d1c/d41/d44/d58/f88 0 2024-09-16T00:09:50.272 INFO:tasks.workunit.client.0.smithi139.stdout:2/885: symlink d11/d2f/deb/l132 0 2024-09-16T00:09:50.272 INFO:tasks.workunit.client.0.smithi139.stdout:4/789: write d12/d2b/f3a [2559334,68763] 0 2024-09-16T00:09:50.272 INFO:tasks.workunit.client.0.smithi139.stdout:7/772: dwrite d5/d16/f34 [0,4194304] 0 2024-09-16T00:09:50.272 INFO:tasks.workunit.client.0.smithi139.stdout:7/773: stat d5/d16/f34 0 2024-09-16T00:09:50.272 INFO:tasks.workunit.client.0.smithi139.stdout:2/886: unlink d11/d2f/d6e/de5/c104 0 2024-09-16T00:09:50.272 INFO:tasks.workunit.client.0.smithi139.stdout:7/774: dread d5/d16/d22/d25/faf [0,4194304] 0 2024-09-16T00:09:50.273 INFO:tasks.workunit.client.0.smithi139.stdout:2/887: truncate ff 11088428 0 2024-09-16T00:09:50.273 INFO:tasks.workunit.client.0.smithi139.stdout:4/790: read d12/d47/faf [500271,72556] 0 2024-09-16T00:09:50.273 INFO:tasks.workunit.client.0.smithi139.stdout:2/888: write d11/d30/fd5 [281846,22273] 0 2024-09-16T00:09:50.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.186+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98fc062630 con 0x7f98fc063110 2024-09-16T00:09:50.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.188+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98fc062630 con 0x7f98fc063110 2024-09-16T00:09:50.273 INFO:tasks.workunit.client.0.smithi139.stdout:4/791: read d12/d47/d49/d60/dac/fdb [3635424,41593] 0 2024-09-16T00:09:50.274 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:50.274 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:50.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.188+0000 7f9903981640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f98fc0cc9c0 0x7f98fc02c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.188+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f98fc062630 con 0x7f98fc0cc9c0 2024-09-16T00:09:50.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.188+0000 7f990171d640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f98fc0cc9c0 0x7f98fc02c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.189+0000 7f990171d640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f98fc0cc9c0 0x7f98fc02c640 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:50.275 INFO:tasks.workunit.client.0.smithi139.stdout:4/792: write f4 [4530368,124535] 0 2024-09-16T00:09:50.275 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.189+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98fc062630 con 0x7f98fc0cc9c0 2024-09-16T00:09:50.276 INFO:tasks.workunit.client.0.smithi139.stdout:4/793: rename d12/f43 to d12/d47/db2/f100 0 2024-09-16T00:09:50.276 INFO:tasks.workunit.client.0.smithi139.stdout:4/794: mknod d12/d47/db2/d4d/de4/c101 0 2024-09-16T00:09:50.276 INFO:tasks.workunit.client.0.smithi139.stdout:4/795: creat d12/d2b/d8c/dff/f102 x:0 0 0 2024-09-16T00:09:50.276 INFO:tasks.workunit.client.0.smithi139.stdout:4/796: write d12/d47/db2/d4d/d69/f72 [2480558,89356] 0 2024-09-16T00:09:50.277 INFO:tasks.workunit.client.0.smithi139.stdout:4/797: write d12/d47/db2/d4d/de4/d7f/feb [33248,76061] 0 2024-09-16T00:09:50.277 INFO:tasks.workunit.client.0.smithi139.stdout:2/889: dread d11/d30/d110/d29/fb0 [0,4194304] 0 2024-09-16T00:09:50.277 INFO:tasks.workunit.client.0.smithi139.stdout:2/890: write d11/f1a [5458184,118891] 0 2024-09-16T00:09:50.277 INFO:tasks.workunit.client.0.smithi139.stdout:4/798: truncate d12/d2b/f34 930234 0 2024-09-16T00:09:50.277 INFO:tasks.workunit.client.0.smithi139.stdout:4/799: mknod d12/d47/db2/c103 0 2024-09-16T00:09:50.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.206+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98fc062630 con 0x7f98fc0cc9c0 2024-09-16T00:09:50.277 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.207+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98fc062630 con 0x7f98fc0cc9c0 2024-09-16T00:09:50.278 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:50.278 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:50.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.208+0000 7f9903981640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f98fc02d090 0x7f98fc02d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.208+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f98fc063570 con 0x7f98fc02d090 2024-09-16T00:09:50.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.208+0000 7f9900f1c640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f98fc02d090 0x7f98fc02d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.208+0000 7f9900f1c640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f98fc02d090 0x7f98fc02d4f0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:50.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.209+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98fc063570 con 0x7f98fc02d090 2024-09-16T00:09:50.279 INFO:tasks.workunit.client.0.smithi139.stdout:4/800: dwrite d12/d47/faf [0,4194304] 0 2024-09-16T00:09:50.279 INFO:tasks.workunit.client.0.smithi139.stdout:4/801: rename c9 to d12/d2b/d8c/c104 0 2024-09-16T00:09:50.280 INFO:tasks.workunit.client.0.smithi139.stdout:4/802: chown d12/d2b/f73 82 1 2024-09-16T00:09:50.280 INFO:tasks.workunit.client.0.smithi139.stdout:4/803: write d12/d47/db2/d4d/de4/f6e [744116,97440] 0 2024-09-16T00:09:50.280 INFO:tasks.workunit.client.0.smithi139.stdout:4/804: mknod d12/d47/db2/d4d/de4/d7f/c105 0 2024-09-16T00:09:50.280 INFO:tasks.workunit.client.0.smithi139.stdout:4/805: dwrite d12/d47/db2/d4d/de4/fcf [0,4194304] 0 2024-09-16T00:09:50.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.224+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98fc063570 con 0x7f98fc02d090 2024-09-16T00:09:50.280 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:50.281 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:50.281 INFO:teuthology.orchestra.run.smithi139.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T00:09:50.281 INFO:tasks.workunit.client.0.smithi139.stdout:6/525: sync 2024-09-16T00:09:50.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.226+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98fc063570 con 0x7f98fc02d090 2024-09-16T00:09:50.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.226+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=90}) -- 0x7f98fc063570 con 0x7f98fc162a20 2024-09-16T00:09:50.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.226+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98fc02da30 con 0x7f98fc162a20 2024-09-16T00:09:50.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.227+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f98f4059810 con 0x7f98fc162a20 2024-09-16T00:09:50.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.227+0000 7f9903981640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f98fc20bdf0 0x7f98fc20e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.227+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f98fc211140 con 0x7f98fc20bdf0 2024-09-16T00:09:50.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.227+0000 7f9901f1e640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f98fc20bdf0 0x7f98fc20e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.228+0000 7f9901f1e640 1 --2- 172.21.15.139:0/4003791905 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f98fc20bdf0 0x7f98fc20e1e0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:50.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.229+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98fc211140 con 0x7f98fc20bdf0 2024-09-16T00:09:50.283 INFO:tasks.workunit.client.0.smithi139.stdout:6/526: readlink d8/d29/d83/d26/d28/d6d/la0 0 2024-09-16T00:09:50.283 INFO:tasks.workunit.client.0.smithi139.stdout:4/806: dwrite d12/d47/d49/d60/f74 [0,4194304] 0 2024-09-16T00:09:50.283 INFO:tasks.workunit.client.0.smithi139.stdout:6/527: write d8/f7b [3320885,80384] 0 2024-09-16T00:09:50.283 INFO:tasks.workunit.client.0.smithi139.stdout:4/807: truncate d12/d47/db2/d4d/f79 1538795 0 2024-09-16T00:09:50.283 INFO:tasks.workunit.client.0.smithi139.stdout:6/528: mknod d8/d29/d83/d26/d28/d6d/cb4 0 2024-09-16T00:09:50.284 INFO:tasks.workunit.client.0.smithi139.stdout:4/808: dread d12/d47/db2/d4d/f79 [0,4194304] 0 2024-09-16T00:09:50.284 INFO:tasks.workunit.client.0.smithi139.stdout:4/809: write d12/d47/db2/f77 [310562,6214] 0 2024-09-16T00:09:50.284 INFO:tasks.workunit.client.0.smithi139.stdout:6/529: dwrite d8/d29/d83/d26/f75 [0,4194304] 0 2024-09-16T00:09:50.284 INFO:tasks.workunit.client.0.smithi139.stdout:6/530: symlink d8/lb5 0 2024-09-16T00:09:50.284 INFO:tasks.workunit.client.0.smithi139.stdout:4/810: rename d12/d47/d49/lba to d12/d47/db2/d4d/da9/l106 0 2024-09-16T00:09:50.284 INFO:tasks.workunit.client.0.smithi139.stdout:4/811: write d12/f4f [4660683,87222] 0 2024-09-16T00:09:50.284 INFO:tasks.workunit.client.0.smithi139.stdout:6/531: creat d8/d29/d83/d26/d28/d6d/fb6 x:0 0 0 2024-09-16T00:09:50.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.253+0000 7f9903981640 1 -- 172.21.15.139:0/4003791905 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f98fc211110 con 0x7f98fc20bdf0 2024-09-16T00:09:50.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.255+0000 7f98f1ffb640 1 -- 172.21.15.139:0/4003791905 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f98fc211110 con 0x7f98fc20bdf0 2024-09-16T00:09:50.285 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:50.285 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:50.286 INFO:tasks.workunit.client.0.smithi139.stdout:4/812: rename d12/d2b/d8c/fee to d12/d47/db2/d4d/f107 0 2024-09-16T00:09:50.286 INFO:tasks.workunit.client.0.smithi139.stdout:4/813: chown d12/d47/d49/d60/dc3/fcb 136302 1 2024-09-16T00:09:50.286 INFO:tasks.workunit.client.0.smithi139.stdout:6/532: dwrite d8/d29/d83/d26/d28/d5a/fa7 [0,4194304] 0 2024-09-16T00:09:50.286 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T00:09:50.287 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T00:09:50.288 INFO:tasks.workunit.client.0.smithi139.stdout:4/814: getdents d12/d47/db2/d4d/de4/d7f/d9e 0 2024-09-16T00:09:50.289 INFO:tasks.workunit.client.0.smithi139.stdout:4/815: mknod d12/d80/c108 0 2024-09-16T00:09:50.289 INFO:tasks.workunit.client.0.smithi139.stdout:6/533: dwrite d8/d29/d83/d26/d45/d65/f37 [0,4194304] 0 2024-09-16T00:09:50.289 INFO:tasks.workunit.client.0.smithi139.stdout:4/816: dread d12/d2b/f30 [0,4194304] 0 2024-09-16T00:09:50.289 INFO:tasks.workunit.client.0.smithi139.stdout:4/817: mkdir d12/d2b/d8c/dff/d109 0 2024-09-16T00:09:50.289 INFO:tasks.workunit.client.0.smithi139.stdout:6/534: symlink d8/d29/d83/d26/d28/d9e/da8/lb7 0 2024-09-16T00:09:50.289 INFO:tasks.workunit.client.0.smithi139.stdout:6/535: dread - d8/d29/d83/d26/d28/d6d/f71 zero size 2024-09-16T00:09:50.289 INFO:tasks.workunit.client.0.smithi139.stdout:4/818: unlink d12/d47/f6c 0 2024-09-16T00:09:50.289 INFO:tasks.workunit.client.0.smithi139.stdout:6/536: creat d8/d29/d83/d26/d45/d59/fb8 x:0 0 0 2024-09-16T00:09:50.289 INFO:tasks.workunit.client.0.smithi139.stdout:4/819: write d12/d47/d49/d60/dc3/fcb [1530062,24124] 0 2024-09-16T00:09:50.289 INFO:tasks.workunit.client.0.smithi139.stdout:6/537: mkdir d8/d29/db9 0 2024-09-16T00:09:50.289 INFO:tasks.workunit.client.0.smithi139.stdout:4/820: mknod d12/d47/d49/d60/dc3/c10a 0 2024-09-16T00:09:50.289 INFO:tasks.workunit.client.0.smithi139.stdout:6/538: mkdir d8/d56/dba 0 2024-09-16T00:09:50.289 INFO:tasks.workunit.client.0.smithi139.stdout:6/539: fsync d8/d29/d83/d26/f61 0 2024-09-16T00:09:50.290 INFO:tasks.workunit.client.0.smithi139.stdout:4/821: creat d12/d47/db2/d4d/db0/f10b x:0 0 0 2024-09-16T00:09:50.290 INFO:tasks.workunit.client.0.smithi139.stdout:4/822: truncate d12/d47/fb3 852750 0 2024-09-16T00:09:50.290 INFO:tasks.workunit.client.0.smithi139.stdout:6/540: mknod d8/d20/d27/cbb 0 2024-09-16T00:09:50.290 INFO:tasks.workunit.client.0.smithi139.stdout:4/823: mkdir d12/d2b/d8c/d10c 0 2024-09-16T00:09:50.290 INFO:tasks.workunit.client.0.smithi139.stdout:6/541: mknod d8/da4/cbc 0 2024-09-16T00:09:50.290 INFO:tasks.workunit.client.0.smithi139.stdout:4/824: dwrite d12/d47/db2/d4d/d69/ff8 [0,4194304] 0 2024-09-16T00:09:50.292 INFO:tasks.workunit.client.0.smithi139.stdout:6/542: write d8/d29/f32 [2861211,82979] 0 2024-09-16T00:09:50.292 INFO:tasks.workunit.client.0.smithi139.stdout:4/825: chown d12/d47/f5c 193 1 2024-09-16T00:09:50.295 INFO:tasks.workunit.client.0.smithi139.stdout:6/543: write d8/d29/d83/d26/f54 [204041,85289] 0 2024-09-16T00:09:50.298 INFO:tasks.workunit.client.0.smithi139.stdout:4/826: write d12/d47/db2/d4d/de4/f62 [333490,97634] 0 2024-09-16T00:09:50.299 INFO:tasks.workunit.client.0.smithi139.stdout:6/544: symlink d8/d29/db9/lbd 0 2024-09-16T00:09:50.299 INFO:tasks.workunit.client.0.smithi139.stdout:4/827: chown d12/d2b/f34 5823045 1 2024-09-16T00:09:50.301 INFO:tasks.workunit.client.0.smithi139.stdout:4/828: mknod d12/d2b/d8c/dff/c10d 0 2024-09-16T00:09:50.302 INFO:tasks.workunit.client.0.smithi139.stdout:6/545: symlink d8/d29/d83/d26/d28/d74/lbe 0 2024-09-16T00:09:50.304 INFO:tasks.workunit.client.0.smithi139.stdout:4/829: read d12/d47/db2/d4d/d69/fb4 [147741,119538] 0 2024-09-16T00:09:50.306 INFO:tasks.workunit.client.0.smithi139.stdout:4/830: symlink d12/d47/db2/l10e 0 2024-09-16T00:09:50.308 INFO:tasks.workunit.client.0.smithi139.stdout:6/546: dwrite d8/f7b [4194304,4194304] 0 2024-09-16T00:09:50.310 INFO:tasks.workunit.client.0.smithi139.stdout:4/831: rename d12/d47/db2/d4d/da9/ldd to d12/d47/d49/d60/dac/l10f 0 2024-09-16T00:09:50.310 INFO:tasks.workunit.client.0.smithi139.stdout:6/547: read d8/d29/d83/d26/d45/d65/f13 [166646,2346] 0 2024-09-16T00:09:50.312 INFO:tasks.workunit.client.0.smithi139.stdout:4/832: rmdir d12/d47/db2/d4d/d69 39 2024-09-16T00:09:50.313 INFO:tasks.workunit.client.0.smithi139.stdout:6/548: creat d8/d20/d4d/fbf x:0 0 0 2024-09-16T00:09:50.314 INFO:tasks.workunit.client.0.smithi139.stdout:6/549: write d8/d29/d83/d26/f75 [289018,35054] 0 2024-09-16T00:09:50.316 INFO:tasks.workunit.client.0.smithi139.stdout:6/550: unlink d8/d1f/f41 0 2024-09-16T00:09:50.319 INFO:tasks.workunit.client.0.smithi139.stdout:6/551: mkdir d8/d29/d83/d26/d45/d42/dc0 0 2024-09-16T00:09:50.320 INFO:tasks.workunit.client.0.smithi139.stdout:6/552: readlink d8/d1f/l94 0 2024-09-16T00:09:50.320 INFO:tasks.workunit.client.0.smithi139.stdout:4/833: dread d12/d47/f45 [0,4194304] 0 2024-09-16T00:09:50.322 INFO:tasks.workunit.client.0.smithi139.stdout:6/553: creat d8/d20/d4d/db0/fc1 x:0 0 0 2024-09-16T00:09:50.323 INFO:tasks.workunit.client.0.smithi139.stdout:4/834: creat d12/d47/db2/d4d/de4/d7f/f110 x:0 0 0 2024-09-16T00:09:50.325 INFO:tasks.workunit.client.0.smithi139.stdout:6/554: symlink d8/d29/d83/d26/d45/d65/d6e/lc2 0 2024-09-16T00:09:50.327 INFO:tasks.workunit.client.0.smithi139.stdout:6/555: mkdir d8/d29/d83/d26/d45/dc3 0 2024-09-16T00:09:50.328 INFO:tasks.workunit.client.0.smithi139.stdout:6/556: write d8/d1f/f5c [148635,118078] 0 2024-09-16T00:09:50.329 INFO:tasks.workunit.client.0.smithi139.stdout:6/557: stat d8/d20/laf 0 2024-09-16T00:09:50.329 INFO:tasks.workunit.client.0.smithi139.stdout:4/835: dwrite d12/d47/d49/f7e [0,4194304] 0 2024-09-16T00:09:50.334 INFO:tasks.workunit.client.0.smithi139.stdout:4/836: rename cf to d12/d2b/c111 0 2024-09-16T00:09:50.337 INFO:tasks.workunit.client.0.smithi139.stdout:6/558: rename d8/d29/d83/d26/d28/d74/cab to d8/d29/d83/d26/d28/d9e/da8/cc4 0 2024-09-16T00:09:50.339 INFO:tasks.workunit.client.0.smithi139.stdout:4/837: dwrite d12/d47/db2/f100 [4194304,4194304] 0 2024-09-16T00:09:50.342 INFO:tasks.workunit.client.0.smithi139.stdout:6/559: getdents d8/d29/d83/d26/d45/d65 0 2024-09-16T00:09:50.343 INFO:tasks.workunit.client.0.smithi139.stdout:6/560: mknod d8/d20/d27/cc5 0 2024-09-16T00:09:50.360 INFO:tasks.workunit.client.0.smithi139.stdout:6/561: write d8/d29/d83/d26/fa5 [272015,35851] 0 2024-09-16T00:09:50.363 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:50.363+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:50.366 INFO:tasks.workunit.client.0.smithi139.stdout:6/562: dwrite d8/d20/d4d/db0/fc1 [0,4194304] 0 2024-09-16T00:09:50.369 INFO:tasks.workunit.client.0.smithi139.stdout:6/563: creat d8/d29/d83/d26/d45/d42/dc0/fc6 x:0 0 0 2024-09-16T00:09:50.371 INFO:tasks.workunit.client.0.smithi139.stdout:6/564: symlink d8/d29/d83/d26/d28/d9e/da8/lc7 0 2024-09-16T00:09:50.371 INFO:tasks.workunit.client.0.smithi139.stdout:6/565: chown d8/d29/d83/d26/d45/d59 17189397 1 2024-09-16T00:09:50.372 INFO:tasks.workunit.client.0.smithi139.stdout:6/566: readlink d8/d56/l84 0 2024-09-16T00:09:50.373 INFO:tasks.workunit.client.0.smithi139.stdout:6/567: unlink c4 0 2024-09-16T00:09:50.373 INFO:tasks.workunit.client.0.smithi139.stdout:6/568: chown d8/d29/f36 999260 1 2024-09-16T00:09:50.377 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T00:09:50.378 INFO:tasks.workunit.client.0.smithi139.stdout:6/569: rename d8/d29/d83/d26/d45/d59/f6b to d8/d29/d83/d26/d28/d74/fc8 0 2024-09-16T00:09:50.380 INFO:tasks.workunit.client.0.smithi139.stdout:6/570: mkdir d8/d1f/dc9 0 2024-09-16T00:09:50.382 INFO:tasks.workunit.client.0.smithi139.stdout:6/571: unlink d8/d29/c55 0 2024-09-16T00:09:50.387 INFO:tasks.workunit.client.0.smithi139.stdout:6/572: link d8/d29/d83/d26/d28/d74/lbe d8/d29/d83/d26/d45/lca 0 2024-09-16T00:09:50.389 INFO:tasks.workunit.client.0.smithi139.stdout:6/573: creat d8/d29/d83/d26/d45/d65/d6e/fcb x:0 0 0 2024-09-16T00:09:50.393 INFO:tasks.workunit.client.0.smithi139.stdout:6/574: rename d8/d29/d83/d26/d28/d6d/fb6 to d8/d29/d83/d26/d28/d9e/da8/fcc 0 2024-09-16T00:09:50.394 INFO:tasks.workunit.client.0.smithi139.stdout:6/575: rmdir d8/d29/d83/d26/d45/d59 39 2024-09-16T00:09:50.396 INFO:tasks.workunit.client.0.smithi139.stdout:6/576: unlink d8/d29/d83/d26/d28/d6d/la0 0 2024-09-16T00:09:50.396 INFO:tasks.workunit.client.0.smithi139.stdout:6/577: readlink d8/d20/laf 0 2024-09-16T00:09:50.398 INFO:tasks.workunit.client.0.smithi139.stdout:6/578: write d8/d29/d83/d26/d28/f95 [20626,91442] 0 2024-09-16T00:09:50.399 INFO:tasks.workunit.client.0.smithi139.stdout:6/579: rmdir d8/d29/d83/d26/d45 39 2024-09-16T00:09:50.401 INFO:tasks.workunit.client.0.smithi139.stdout:6/580: creat d8/d29/d83/d26/d45/d42/fcd x:0 0 0 2024-09-16T00:09:50.403 INFO:tasks.workunit.client.0.smithi139.stdout:6/581: creat d8/d29/d83/d26/d28/fce x:0 0 0 2024-09-16T00:09:50.411 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:50.412 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:50.416 INFO:tasks.workunit.client.0.smithi139.stdout:3/631: sync 2024-09-16T00:09:50.420 INFO:tasks.workunit.client.0.smithi139.stdout:3/632: creat d5/d7/dc/d9c/db0/fdb x:0 0 0 2024-09-16T00:09:50.422 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:09:50.423 INFO:tasks.workunit.client.0.smithi139.stdout:3/633: unlink d5/d7/f2d 0 2024-09-16T00:09:50.432 INFO:tasks.workunit.client.0.smithi139.stdout:6/582: dwrite d8/d20/d27/f58 [0,4194304] 0 2024-09-16T00:09:50.437 INFO:tasks.workunit.client.0.smithi139.stdout:6/583: getdents d8/d29/d83/d26/d45/d59 0 2024-09-16T00:09:50.439 INFO:tasks.workunit.client.0.smithi139.stdout:6/584: chown d8/d20 81 1 2024-09-16T00:09:50.443 INFO:tasks.workunit.client.0.smithi139.stdout:6/585: creat d8/d20/d27/fcf x:0 0 0 2024-09-16T00:09:50.445 INFO:tasks.workunit.client.0.smithi139.stdout:6/586: creat d8/d56/fd0 x:0 0 0 2024-09-16T00:09:50.446 INFO:tasks.workunit.client.0.smithi139.stdout:6/587: fsync d8/d20/f62 0 2024-09-16T00:09:50.447 INFO:tasks.workunit.client.0.smithi139.stdout:6/588: read d8/d29/f3d [2912719,27286] 0 2024-09-16T00:09:50.449 INFO:tasks.workunit.client.0.smithi139.stdout:6/589: fsync d8/d29/d83/d26/f91 0 2024-09-16T00:09:50.451 INFO:tasks.workunit.client.0.smithi139.stdout:6/590: mkdir d8/d20/dd1 0 2024-09-16T00:09:50.453 INFO:tasks.workunit.client.0.smithi139.stdout:6/591: mknod d8/d29/d83/d26/d28/d74/cd2 0 2024-09-16T00:09:50.454 INFO:tasks.workunit.client.0.smithi139.stdout:6/592: chown d8/d29/d83/d26/d28/d5a 6933 1 2024-09-16T00:09:50.456 INFO:tasks.workunit.client.0.smithi139.stdout:6/593: unlink d8/d1f/l94 0 2024-09-16T00:09:50.458 INFO:tasks.workunit.client.0.smithi139.stdout:6/594: mkdir d8/d29/d83/d26/d28/d6d/dd3 0 2024-09-16T00:09:50.461 INFO:tasks.workunit.client.0.smithi139.stdout:6/595: rename d8/d29/d83/d26/d45/d59/f5e to d8/d29/d83/d26/d45/d65/d6e/fd4 0 2024-09-16T00:09:50.464 INFO:tasks.workunit.client.0.smithi139.stdout:6/596: truncate d8/d20/d68/f92 144959 0 2024-09-16T00:09:50.464 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:50.463+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:50.468 INFO:tasks.workunit.client.0.smithi139.stdout:6/597: dwrite d8/fe [0,4194304] 0 2024-09-16T00:09:50.475 INFO:tasks.workunit.client.0.smithi139.stdout:6/598: truncate d8/d29/d83/d26/f54 466624 0 2024-09-16T00:09:50.478 INFO:tasks.workunit.client.0.smithi139.stdout:6/599: creat d8/d29/d83/d26/d45/fd5 x:0 0 0 2024-09-16T00:09:50.485 INFO:tasks.workunit.client.0.smithi139.stdout:6/600: dwrite d8/d29/d83/d26/d28/d9e/da8/fcc [0,4194304] 0 2024-09-16T00:09:50.489 INFO:tasks.workunit.client.0.smithi139.stdout:6/601: creat d8/d20/d4d/db0/fd6 x:0 0 0 2024-09-16T00:09:50.490 INFO:tasks.workunit.client.0.smithi139.stdout:6/602: dread - d8/d29/d83/d26/d45/fd5 zero size 2024-09-16T00:09:50.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.500+0000 7f7d40830640 1 Processor -- start 2024-09-16T00:09:50.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.500+0000 7f7d40830640 1 -- start start 2024-09-16T00:09:50.501 INFO:tasks.workunit.client.0.smithi139.stdout:6/603: dread d8/d29/d83/d26/f66 [0,4194304] 0 2024-09-16T00:09:50.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.501+0000 7f7d40830640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7d3c0c63c0 0x7f7d3c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.501+0000 7f7d40830640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7d3c0c67a0 0x7f7d3c0c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.501+0000 7f7d40830640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d3c0c76b0 con 0x7f7d3c0c7220 2024-09-16T00:09:50.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.501+0000 7f7d40830640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7d3c06ed80 con 0x7f7d3c0c63c0 2024-09-16T00:09:50.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.501+0000 7f7d40830640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7d3c0c90f0 con 0x7f7d3c0c67a0 2024-09-16T00:09:50.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.501+0000 7f7d3a59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7d3c0c63c0 0x7f7d3c06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.501+0000 7f7d3a59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7d3c0c63c0 0x7f7d3c06f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:37566/0 (socket says 172.21.15.139:37566) 2024-09-16T00:09:50.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.501+0000 7f7d3a59c640 1 -- 172.21.15.139:0/3299638829 learned_addr learned my addr 172.21.15.139:0/3299638829 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:50.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.501+0000 7f7d39d9b640 1 --2- 172.21.15.139:0/3299638829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7d3c0c67a0 0x7f7d3c0c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.501+0000 7f7d3a59c640 1 -- 172.21.15.139:0/3299638829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7d3c0c67a0 msgr2=0x7f7d3c0c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:50.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.501+0000 7f7d3a59c640 1 --2- 172.21.15.139:0/3299638829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7d3c0c67a0 0x7f7d3c0c46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:50.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.501+0000 7f7d3a59c640 1 -- 172.21.15.139:0/3299638829 >> v1:172.21.15.164:6789/0 conn(0x7f7d3c0c7220 legacy=0x7f7d3c151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:50.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.501+0000 7f7d3a59c640 1 -- 172.21.15.139:0/3299638829 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c0c93d0 con 0x7f7d3c0c63c0 2024-09-16T00:09:50.504 INFO:tasks.workunit.client.0.smithi139.stdout:6/604: mknod d8/d20/d68/cd7 0 2024-09-16T00:09:50.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.502+0000 7f7d3a59c640 1 --2- 172.21.15.139:0/3299638829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d3c0c63c0 0x7f7d3c06f930 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009870 tx=0x7f7d2c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcaf39b135305723 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:50.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.502+0000 7f7d3959a640 1 -- 172.21.15.139:0/3299638829 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c037070 con 0x7f7d3c0c63c0 2024-09-16T00:09:50.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.503+0000 7f7d3959a640 1 -- 172.21.15.139:0/3299638829 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d2c02ae50 con 0x7f7d3c0c63c0 2024-09-16T00:09:50.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.503+0000 7f7d3959a640 1 -- 172.21.15.139:0/3299638829 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c032860 con 0x7f7d3c0c63c0 2024-09-16T00:09:50.505 INFO:tasks.workunit.client.0.smithi139.stdout:6/605: fdatasync d8/d29/f32 0 2024-09-16T00:09:50.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.503+0000 7f7d40830640 1 -- 172.21.15.139:0/3299638829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d3c0c63c0 msgr2=0x7f7d3c06f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:50.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.503+0000 7f7d40830640 1 --2- 172.21.15.139:0/3299638829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d3c0c63c0 0x7f7d3c06f930 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c009870 tx=0x7f7d2c02a300 comp rx=0 tx=0).stop 2024-09-16T00:09:50.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.504+0000 7f7d40830640 1 -- 172.21.15.139:0/3299638829 shutdown_connections 2024-09-16T00:09:50.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.504+0000 7f7d40830640 1 --2- 172.21.15.139:0/3299638829 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7d3c0c67a0 0x7f7d3c0c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:50.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.504+0000 7f7d40830640 1 --2- 172.21.15.139:0/3299638829 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d3c0c63c0 0x7f7d3c06f930 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:50.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.504+0000 7f7d40830640 1 -- 172.21.15.139:0/3299638829 >> 172.21.15.139:0/3299638829 conn(0x7f7d3c072b30 msgr2=0x7f7d3c070510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:50.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.504+0000 7f7d40830640 1 -- 172.21.15.139:0/3299638829 shutdown_connections 2024-09-16T00:09:50.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.504+0000 7f7d40830640 1 -- 172.21.15.139:0/3299638829 wait complete. 2024-09-16T00:09:50.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.505+0000 7f7d40830640 1 Processor -- start 2024-09-16T00:09:50.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.505+0000 7f7d40830640 1 -- start start 2024-09-16T00:09:50.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d40830640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d3c0c63c0 0x7f7d3c1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d40830640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d3c0c67a0 0x7f7d3c14df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d40830640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7d3c0c7220 0x7f7d3c1f0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d40830640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c06f130 con 0x7f7d3c0c67a0 2024-09-16T00:09:50.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d40830640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c0c9990 con 0x7f7d3c0c63c0 2024-09-16T00:09:50.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d40830640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7d3c06f410 con 0x7f7d3c0c7220 2024-09-16T00:09:50.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d39d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d3c0c67a0 0x7f7d3c14df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d3a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d3c0c63c0 0x7f7d3c1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d39d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d3c0c67a0 0x7f7d3c14df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:37572/0 (socket says 172.21.15.139:37572) 2024-09-16T00:09:50.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d3a59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d3c0c63c0 0x7f7d3c1567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:57090/0 (socket says 172.21.15.139:57090) 2024-09-16T00:09:50.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d39d9b640 1 -- 172.21.15.139:0/2533402169 learned_addr learned my addr 172.21.15.139:0/2533402169 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:50.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d3ad9d640 1 --2- 172.21.15.139:0/2533402169 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7d3c0c7220 0x7f7d3c1f0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d39d9b640 1 -- 172.21.15.139:0/2533402169 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7d3c0c7220 msgr2=0x7f7d3c1f0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:50.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d39d9b640 1 --2- 172.21.15.139:0/2533402169 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7d3c0c7220 0x7f7d3c1f0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:50.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d39d9b640 1 -- 172.21.15.139:0/2533402169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d3c0c63c0 msgr2=0x7f7d3c1567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:50.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d39d9b640 1 --2- 172.21.15.139:0/2533402169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d3c0c63c0 0x7f7d3c1567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:50.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.506+0000 7f7d39d9b640 1 -- 172.21.15.139:0/2533402169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c156d20 con 0x7f7d3c0c67a0 2024-09-16T00:09:50.509 INFO:tasks.workunit.client.0.smithi139.stdout:6/606: link d8/d29/d83/d26/d45/d42/l63 d8/d29/d83/d26/d28/d5a/ld8 0 2024-09-16T00:09:50.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.507+0000 7f7d39d9b640 1 --2- 172.21.15.139:0/2533402169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d3c0c67a0 0x7f7d3c14df50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000d830 tx=0x7f7d3000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:50.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.507+0000 7f7d2affd640 1 -- 172.21.15.139:0/2533402169 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3000b6c0 con 0x7f7d3c0c67a0 2024-09-16T00:09:50.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.507+0000 7f7d2affd640 1 -- 172.21.15.139:0/2533402169 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d30019070 con 0x7f7d3c0c67a0 2024-09-16T00:09:50.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.507+0000 7f7d40830640 1 -- 172.21.15.139:0/2533402169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3c156fb0 con 0x7f7d3c0c67a0 2024-09-16T00:09:50.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.507+0000 7f7d2affd640 1 -- 172.21.15.139:0/2533402169 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3000ba80 con 0x7f7d3c0c67a0 2024-09-16T00:09:50.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.507+0000 7f7d40830640 1 -- 172.21.15.139:0/2533402169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3c158080 con 0x7f7d3c0c67a0 2024-09-16T00:09:50.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.508+0000 7f7d40830640 1 -- 172.21.15.139:0/2533402169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d3c0cd560 con 0x7f7d3c0c67a0 2024-09-16T00:09:50.511 INFO:tasks.workunit.client.0.smithi139.stdout:6/607: chown d8/d29/d83/d26/d45/d85/fad 331 1 2024-09-16T00:09:50.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.508+0000 7f7d2affd640 1 -- 172.21.15.139:0/2533402169 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7d3000bc20 con 0x7f7d3c0c67a0 2024-09-16T00:09:50.512 INFO:tasks.workunit.client.0.smithi139.stdout:6/608: dread - d8/d29/f8f zero size 2024-09-16T00:09:50.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.509+0000 7f7d2affd640 1 --2- 172.21.15.139:0/2533402169 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7d1c06c2b0 0x7f7d1c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:50.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.509+0000 7f7d2affd640 1 -- 172.21.15.139:0/2533402169 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5973+0+0 (secure 0 0 0) 0x7f7d3008b860 con 0x7f7d3c0c67a0 2024-09-16T00:09:50.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.509+0000 7f7d3a59c640 1 --2- 172.21.15.139:0/2533402169 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7d1c06c2b0 0x7f7d1c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:50.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.509+0000 7f7d3a59c640 1 --2- 172.21.15.139:0/2533402169 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7d1c06c2b0 0x7f7d1c06e770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c034000 tx=0x7f7d2c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:50.513 INFO:tasks.workunit.client.0.smithi139.stdout:6/609: fsync d8/d29/d83/d26/d45/d85/fad 0 2024-09-16T00:09:50.513 INFO:tasks.workunit.client.0.smithi139.stdout:6/610: creat d8/d29/d83/d26/d45/d42/dc0/fd9 x:0 0 0 2024-09-16T00:09:50.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.514+0000 7f7d2affd640 1 -- 172.21.15.139:0/2533402169 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7d3c0cd560 con 0x7f7d3c0c67a0 2024-09-16T00:09:50.518 INFO:tasks.workunit.client.0.smithi139.stdout:6/611: dwrite d8/d56/f79 [0,4194304] 0 2024-09-16T00:09:50.519 INFO:tasks.workunit.client.0.smithi139.stdout:6/612: truncate f3 1150896 0 2024-09-16T00:09:50.522 INFO:tasks.workunit.client.0.smithi139.stdout:6/613: write d8/d29/d83/d26/d45/d59/fb8 [164978,3575] 0 2024-09-16T00:09:50.526 INFO:tasks.workunit.client.0.smithi139.stdout:6/614: unlink d8/d29/d83/d26/d45/d42/l57 0 2024-09-16T00:09:50.529 INFO:tasks.workunit.client.0.smithi139.stdout:6/615: dread d8/f12 [0,4194304] 0 2024-09-16T00:09:50.531 INFO:tasks.workunit.client.0.smithi139.stdout:6/616: creat d8/d20/d4d/db0/fda x:0 0 0 2024-09-16T00:09:50.533 INFO:tasks.workunit.client.0.smithi139.stdout:6/617: symlink d8/d29/d83/d26/d45/d65/d8b/ldb 0 2024-09-16T00:09:50.534 INFO:tasks.workunit.client.0.smithi139.stdout:6/618: rmdir d8/d20 39 2024-09-16T00:09:50.537 INFO:tasks.workunit.client.0.smithi139.stdout:6/619: dread - d8/d29/d83/d26/d45/f8a zero size 2024-09-16T00:09:50.537 INFO:tasks.workunit.client.0.smithi139.stdout:6/620: mkdir d8/d20/dd1/ddc 0 2024-09-16T00:09:50.550 INFO:tasks.workunit.client.0.smithi139.stdout:6/621: dread d8/d29/d83/d26/d45/f46 [0,4194304] 0 2024-09-16T00:09:50.550 INFO:tasks.workunit.client.0.smithi139.stdout:6/622: chown d8/d29/f32 11 1 2024-09-16T00:09:50.553 INFO:tasks.workunit.client.0.smithi139.stdout:6/623: mkdir d8/d29/d83/d26/d28/ddd 0 2024-09-16T00:09:50.554 INFO:tasks.workunit.client.0.smithi139.stdout:6/624: symlink d8/d1f/dc9/lde 0 2024-09-16T00:09:50.560 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:50.560 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:50.565 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:50.564+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:50.566 INFO:tasks.workunit.client.0.smithi139.stdout:6/625: dwrite d8/d20/d4d/f60 [0,4194304] 0 2024-09-16T00:09:50.567 INFO:tasks.workunit.client.0.smithi139.stdout:6/626: fsync d8/d29/d83/d26/d28/fce 0 2024-09-16T00:09:50.567 INFO:tasks.workunit.client.0.smithi139.stdout:6/627: readlink d8/d29/d83/d26/d45/l5f 0 2024-09-16T00:09:50.570 INFO:tasks.workunit.client.0.smithi139.stdout:6/628: fsync d8/d29/f50 0 2024-09-16T00:09:50.573 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:09:50.586 INFO:tasks.workunit.client.0.smithi139.stdout:1/612: sync 2024-09-16T00:09:50.586 INFO:tasks.workunit.client.0.smithi139.stdout:5/792: sync 2024-09-16T00:09:50.586 INFO:tasks.workunit.client.0.smithi139.stdout:9/927: sync 2024-09-16T00:09:50.600 INFO:tasks.workunit.client.0.smithi139.stdout:1/613: mknod d7/d8/d40/d48/d67/cb3 0 2024-09-16T00:09:50.603 INFO:tasks.workunit.client.0.smithi139.stdout:9/928: rename d19/d1a/d20/d23/dca/d71/d32/l36 to d19/d1a/d20/d89/dbd/def/l134 0 2024-09-16T00:09:50.605 INFO:tasks.workunit.client.0.smithi139.stdout:6/629: dwrite d8/d20/d4d/f60 [0,4194304] 0 2024-09-16T00:09:50.608 INFO:tasks.workunit.client.0.smithi139.stdout:5/793: dwrite d1/d13/d3f/d88/f109 [0,4194304] 0 2024-09-16T00:09:50.609 INFO:tasks.workunit.client.0.smithi139.stdout:9/929: dread - d19/d1a/d20/d23/d2d/d35/d42/d3a/dcb/fdd zero size 2024-09-16T00:09:50.615 INFO:tasks.workunit.client.0.smithi139.stdout:9/930: write d19/d1a/d20/d23/d2d/f99 [196955,17325] 0 2024-09-16T00:09:50.616 INFO:tasks.workunit.client.0.smithi139.stdout:6/630: creat d8/d20/dd1/ddc/fdf x:0 0 0 2024-09-16T00:09:50.621 INFO:tasks.workunit.client.0.smithi139.stdout:1/614: dwrite d7/f12 [0,4194304] 0 2024-09-16T00:09:50.622 INFO:tasks.workunit.client.0.smithi139.stdout:5/794: rename d1/d13/d22/de8/d10d/d33/f7c to d1/d13/d3f/d9f/d73/d79/da5/d10b/f116 0 2024-09-16T00:09:50.623 INFO:tasks.workunit.client.0.smithi139.stdout:6/631: getdents d8/d29/d83/d26/d28/d74 0 2024-09-16T00:09:50.627 INFO:tasks.workunit.client.0.smithi139.stdout:5/795: unlink d1/d13/d3f/d9f/d73/d93/led 0 2024-09-16T00:09:50.628 INFO:tasks.workunit.client.0.smithi139.stdout:1/615: fdatasync d7/d8/d40/d48/d4a/d74/f7e 0 2024-09-16T00:09:50.629 INFO:tasks.workunit.client.0.smithi139.stdout:6/632: mknod d8/d20/d68/ce0 0 2024-09-16T00:09:50.629 INFO:tasks.workunit.client.0.smithi139.stdout:9/931: dwrite d19/d1a/d20/d23/d2d/d35/f133 [0,4194304] 0 2024-09-16T00:09:50.633 INFO:tasks.workunit.client.0.smithi139.stdout:5/796: fsync d1/d13/d3b/f4c 0 2024-09-16T00:09:50.633 INFO:tasks.workunit.client.0.smithi139.stdout:9/932: write d19/d1a/d20/d23/dca/d71/dc7/de1/fed [289673,20568] 0 2024-09-16T00:09:50.637 INFO:tasks.workunit.client.0.smithi139.stdout:5/797: dread d1/d21/d49/d53/d6a/f8b [0,4194304] 0 2024-09-16T00:09:50.637 INFO:tasks.workunit.client.0.smithi139.stdout:9/933: symlink d19/d1a/d20/d23/d2d/d35/d44/df4/d11a/l135 0 2024-09-16T00:09:50.638 INFO:tasks.workunit.client.0.smithi139.stdout:6/633: truncate d8/d29/d83/d26/f75 1509845 0 2024-09-16T00:09:50.639 INFO:tasks.workunit.client.0.smithi139.stdout:9/934: write d19/d1a/d20/d23/d2d/d35/f133 [4315985,43529] 0 2024-09-16T00:09:50.641 INFO:tasks.workunit.client.0.smithi139.stdout:9/935: fsync d19/d86/f5a 0 2024-09-16T00:09:50.641 INFO:tasks.workunit.client.0.smithi139.stdout:6/634: chown d8/d29/d83/d26/f54 0 1 2024-09-16T00:09:50.642 INFO:tasks.workunit.client.0.smithi139.stdout:6/635: stat d8/d29/d83/d26/d28/d9e/da8 0 2024-09-16T00:09:50.645 INFO:tasks.workunit.client.0.smithi139.stdout:5/798: truncate d1/d13/f17 454664 0 2024-09-16T00:09:50.652 INFO:tasks.workunit.client.0.smithi139.stdout:5/799: rename d1/d21/d49/d53/d6a/d7b/d99/l9a to d1/d21/d49/d53/d6a/d7b/d99/l117 0 2024-09-16T00:09:50.654 INFO:tasks.workunit.client.0.smithi139.stdout:6/636: write d8/d29/d83/d26/f75 [1184432,44608] 0 2024-09-16T00:09:50.655 INFO:tasks.workunit.client.0.smithi139.stdout:9/936: dwrite d19/d1a/ff0 [0,4194304] 0 2024-09-16T00:09:50.657 INFO:tasks.workunit.client.0.smithi139.stdout:9/937: readlink d19/d86/lc6 0 2024-09-16T00:09:50.658 INFO:tasks.workunit.client.0.smithi139.stdout:6/637: symlink d8/d29/d83/d26/d28/d5a/le1 0 2024-09-16T00:09:50.659 INFO:tasks.workunit.client.0.smithi139.stdout:9/938: stat d19/d1a/d20/d89/d93/dfd/l105 0 2024-09-16T00:09:50.662 INFO:tasks.workunit.client.0.smithi139.stdout:6/638: dread d8/d20/f64 [0,4194304] 0 2024-09-16T00:09:50.665 INFO:tasks.workunit.client.0.smithi139.stdout:5/800: truncate d1/d13/d3f/d9f/d73/d79/d8d/f9e 2111935 0 2024-09-16T00:09:50.666 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:50.665+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:50.666 INFO:tasks.workunit.client.0.smithi139.stdout:5/801: readlink d1/d13/d22/de8/d10d/dd4/lfd 0 2024-09-16T00:09:50.667 INFO:tasks.workunit.client.0.smithi139.stdout:9/939: rename d19/d1a/d20/d89/d93/c118 to d19/d1a/d20/d23/d2d/d35/d42/d3a/c136 0 2024-09-16T00:09:50.668 INFO:tasks.workunit.client.0.smithi139.stdout:6/639: link d8/d20/d27/f18 d8/d29/d83/d26/d45/d85/fe2 0 2024-09-16T00:09:50.669 INFO:tasks.workunit.client.0.smithi139.stdout:5/802: mknod d1/d21/d49/d53/d6a/d7b/d78/dca/c118 0 2024-09-16T00:09:50.670 INFO:tasks.workunit.client.0.smithi139.stdout:6/640: write d8/d29/d83/d26/d28/f40 [3919851,19500] 0 2024-09-16T00:09:50.670 INFO:tasks.workunit.client.0.smithi139.stdout:9/940: truncate d19/d1a/d20/d23/dca/d71/dc7/de1/fed 653430 0 2024-09-16T00:09:50.673 INFO:tasks.workunit.client.0.smithi139.stdout:9/941: write d19/d1a/d20/d89/f125 [751396,46248] 0 2024-09-16T00:09:50.677 INFO:tasks.workunit.client.0.smithi139.stdout:9/942: chown d19/d1a/d20/d23/d2d/d9a/fac 3 1 2024-09-16T00:09:50.679 INFO:tasks.workunit.client.0.smithi139.stdout:6/641: dwrite d8/d20/d68/f92 [0,4194304] 0 2024-09-16T00:09:50.680 INFO:tasks.workunit.client.0.smithi139.stdout:9/943: chown d19/d1a/d20/d23/d2d/d35/d42/l4c 1 1 2024-09-16T00:09:50.681 INFO:tasks.workunit.client.0.smithi139.stdout:5/803: write d1/d13/d3f/d9f/d73/d79/d8d/f9e [1454969,55967] 0 2024-09-16T00:09:50.682 INFO:tasks.workunit.client.0.smithi139.stdout:5/804: readlink d1/dc3/lf7 0 2024-09-16T00:09:50.686 INFO:tasks.workunit.client.0.smithi139.stdout:9/944: rename d19/d1a/d20/d23/dca/d71/d32/d4e to d19/d1a/d20/d23/dca/d9f/d137 0 2024-09-16T00:09:50.688 INFO:tasks.workunit.client.0.smithi139.stdout:5/805: rename d1/d13/d3f/d9f/d73/d79/da5/d10b/f116 to d1/d13/d22/dd5/f119 0 2024-09-16T00:09:50.689 INFO:tasks.workunit.client.0.smithi139.stdout:6/642: link d8/d29/d83/d26/d45/d65/d6e/lb3 d8/d29/d83/d26/d45/le3 0 2024-09-16T00:09:50.691 INFO:tasks.workunit.client.0.smithi139.stdout:6/643: write d8/d20/d4d/db0/fd6 [436386,37042] 0 2024-09-16T00:09:50.692 INFO:tasks.workunit.client.0.smithi139.stdout:9/945: getdents d19/d1a/d20/d23/dca/d71/da3 0 2024-09-16T00:09:50.692 INFO:tasks.workunit.client.0.smithi139.stdout:5/806: dread d1/d13/d3f/d88/fe4 [0,4194304] 0 2024-09-16T00:09:50.696 INFO:tasks.workunit.client.0.smithi139.stdout:9/946: mknod d19/d1a/d20/d23/d2d/d35/d42/d3a/dcb/c138 0 2024-09-16T00:09:50.696 INFO:tasks.workunit.client.0.smithi139.stdout:6/644: stat d8/d29/d83/d26/d45/d65/d6e/lb3 0 2024-09-16T00:09:50.698 INFO:tasks.workunit.client.0.smithi139.stdout:6/645: truncate d8/d29/d83/d26/d45/fd5 816062 0 2024-09-16T00:09:50.699 INFO:tasks.workunit.client.0.smithi139.stdout:9/947: symlink d19/d1a/d20/d23/dca/d71/d32/d79/l139 0 2024-09-16T00:09:50.700 INFO:tasks.workunit.client.0.smithi139.stdout:6/646: dread d8/d29/d83/d26/f61 [0,4194304] 0 2024-09-16T00:09:50.702 INFO:tasks.workunit.client.0.smithi139.stdout:9/948: creat d19/d1a/d20/d23/d2d/d35/d44/d129/f13a x:0 0 0 2024-09-16T00:09:50.704 INFO:tasks.workunit.client.0.smithi139.stdout:6/647: mkdir d8/d29/d83/d26/d28/d6d/dd3/de4 0 2024-09-16T00:09:50.705 INFO:tasks.workunit.client.0.smithi139.stdout:5/807: dwrite d1/d21/fa8 [0,4194304] 0 2024-09-16T00:09:50.708 INFO:tasks.workunit.client.0.smithi139.stdout:6/648: mkdir d8/d29/d83/d26/d28/d6d/dd3/de5 0 2024-09-16T00:09:50.710 INFO:tasks.workunit.client.0.smithi139.stdout:6/649: fdatasync d8/d29/d83/d26/d45/d59/fb8 0 2024-09-16T00:09:50.711 INFO:tasks.workunit.client.0.smithi139.stdout:6/650: stat d8/d20/dd1/ddc 0 2024-09-16T00:09:50.712 INFO:tasks.workunit.client.0.smithi139.stdout:9/949: write d19/d1a/d20/d89/d93/fd3 [870767,107662] 0 2024-09-16T00:09:50.713 INFO:tasks.workunit.client.0.smithi139.stdout:5/808: link d1/d13/d22/c34 d1/d13/c11a 0 2024-09-16T00:09:50.714 INFO:tasks.workunit.client.0.smithi139.stdout:5/809: chown d1/d13/d3f 17 1 2024-09-16T00:09:50.714 INFO:tasks.workunit.client.0.smithi139.stdout:6/651: symlink d8/d29/d83/d26/d45/d65/le6 0 2024-09-16T00:09:50.715 INFO:tasks.workunit.client.0.smithi139.stdout:9/950: dread d19/fd5 [0,4194304] 0 2024-09-16T00:09:50.716 INFO:tasks.workunit.client.0.smithi139.stdout:5/810: dread - d1/d21/d49/d53/d6a/d7b/fd7 zero size 2024-09-16T00:09:50.717 INFO:tasks.workunit.client.0.smithi139.stdout:6/652: mkdir d8/d29/d83/d26/d28/d6d/de7 0 2024-09-16T00:09:50.718 INFO:tasks.workunit.client.0.smithi139.stdout:9/951: write d19/d1a/d20/d23/d2d/d35/d44/f114 [4509779,72680] 0 2024-09-16T00:09:50.718 INFO:tasks.workunit.client.0.smithi139.stdout:5/811: mkdir d1/d21/d49/d53/d6a/d7b/d99/d11b 0 2024-09-16T00:09:50.720 INFO:tasks.workunit.client.0.smithi139.stdout:6/653: mkdir d8/d29/db9/de8 0 2024-09-16T00:09:50.721 INFO:tasks.workunit.client.0.smithi139.stdout:9/952: mknod d19/d1a/d20/d89/d93/c13b 0 2024-09-16T00:09:50.722 INFO:tasks.workunit.client.0.smithi139.stdout:6/654: creat d8/d20/d4d/db0/fe9 x:0 0 0 2024-09-16T00:09:50.722 INFO:tasks.workunit.client.0.smithi139.stdout:9/953: chown d19/d1a/d20/d89 15 1 2024-09-16T00:09:50.726 INFO:tasks.workunit.client.0.smithi139.stdout:9/954: rmdir d19/d1a/d20/d23/d2d/d35/d44/df4 39 2024-09-16T00:09:50.728 INFO:tasks.workunit.client.0.smithi139.stdout:6/655: rmdir d8/d29/d83/d26/d45/dc3 0 2024-09-16T00:09:50.729 INFO:tasks.workunit.client.0.smithi139.stdout:9/955: link d19/d1a/d20/d23/d2d/d35/d42/d3a/d45/cd7 d19/d1a/d20/d23/d2d/d35/d42/d3a/dcb/c13c 0 2024-09-16T00:09:50.730 INFO:tasks.workunit.client.0.smithi139.stdout:9/956: write d19/d86/f5a [2281883,33639] 0 2024-09-16T00:09:50.735 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:50.736 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:50.740 INFO:tasks.workunit.client.0.smithi139.stdout:9/957: write d19/d1a/f73 [407647,32887] 0 2024-09-16T00:09:50.746 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:09:50.747 INFO:tasks.workunit.client.0.smithi139.stdout:9/958: dwrite fa [0,4194304] 0 2024-09-16T00:09:50.748 INFO:tasks.workunit.client.0.smithi139.stdout:9/959: truncate d19/d1a/d20/d23/dca/d71/dc7/f132 661290 0 2024-09-16T00:09:50.759 INFO:tasks.workunit.client.0.smithi139.stdout:9/960: dread d19/d1a/d20/d89/f9d [0,4194304] 0 2024-09-16T00:09:50.761 INFO:tasks.workunit.client.0.smithi139.stdout:9/961: creat d19/d1a/d20/d89/dbd/f13d x:0 0 0 2024-09-16T00:09:50.762 INFO:tasks.workunit.client.0.smithi139.stdout:9/962: chown d19/d1a/d20/d23/d2d/d35/d42/f116 2786 1 2024-09-16T00:09:50.763 INFO:tasks.workunit.client.0.smithi139.stdout:9/963: write d19/d1a/d20/d23/dca/d9f/d137/f78 [1569214,40430] 0 2024-09-16T00:09:50.764 INFO:tasks.workunit.client.0.smithi139.stdout:9/964: readlink d19/d1a/d20/d23/dca/d9f/d137/lf9 0 2024-09-16T00:09:50.767 INFO:tasks.workunit.client.0.smithi139.stdout:9/965: mknod d19/d1a/d20/d23/d2d/d106/c13e 0 2024-09-16T00:09:50.769 INFO:tasks.workunit.client.0.smithi139.stdout:9/966: write d19/d1a/d20/d23/dca/d9f/d137/fb6 [46135,33698] 0 2024-09-16T00:09:50.771 INFO:tasks.workunit.client.0.smithi139.stdout:8/832: sync 2024-09-16T00:09:50.774 INFO:tasks.workunit.client.0.smithi139.stdout:8/833: creat d17/d1f/d3a/db7/d40/d4f/f114 x:0 0 0 2024-09-16T00:09:50.774 INFO:tasks.workunit.client.0.smithi139.stdout:8/834: chown f4 259112 1 2024-09-16T00:09:50.774 INFO:tasks.workunit.client.0.smithi139.stdout:8/835: fdatasync d17/d1a/d50/fec 0 2024-09-16T00:09:50.777 INFO:tasks.workunit.client.0.smithi139.stdout:9/967: dread d19/f38 [0,4194304] 0 2024-09-16T00:09:50.777 INFO:tasks.workunit.client.0.smithi139.stdout:8/836: getdents d17/d89/df2 0 2024-09-16T00:09:50.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:50.781+0000 7f7d40830640 1 -- 172.21.15.139:0/2533402169 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f7d3c0c6f30 con 0x7f7d3c0c67a0 2024-09-16T00:09:50.782 INFO:tasks.workunit.client.0.smithi139.stdout:9/968: unlink d19/d1a/d20/d23/d2d/d35/d42/d3a/c5f 0 2024-09-16T00:09:50.783 INFO:tasks.workunit.client.0.smithi139.stdout:8/837: dwrite f4 [0,4194304] 0 2024-09-16T00:09:50.784 INFO:tasks.workunit.client.0.smithi139.stdout:9/969: read d19/d1a/d20/d23/d2d/d106/f109 [479484,112380] 0 2024-09-16T00:09:50.786 INFO:tasks.workunit.client.0.smithi139.stdout:8/838: write d17/d1f/f2c [1528512,20173] 0 2024-09-16T00:09:50.790 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:50.790+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:50.795 INFO:tasks.workunit.client.0.smithi139.stdout:9/970: rename d19/d1a/d20/d23/d2d/d35/d42 to d19/d1a/d20/d23/d2d/d35/d13f 0 2024-09-16T00:09:50.795 INFO:tasks.workunit.client.0.smithi139.stdout:8/839: write d17/d1a/f53 [1479559,5843] 0 2024-09-16T00:09:50.797 INFO:tasks.workunit.client.0.smithi139.stdout:9/971: creat d19/d1a/f140 x:0 0 0 2024-09-16T00:09:50.798 INFO:tasks.workunit.client.0.smithi139.stdout:8/840: fdatasync d17/d1f/d3a/db7/f3b 0 2024-09-16T00:09:50.800 INFO:tasks.workunit.client.0.smithi139.stdout:9/972: creat d19/d1a/d20/dbe/f141 x:0 0 0 2024-09-16T00:09:50.801 INFO:tasks.workunit.client.0.smithi139.stdout:9/973: write d19/d1a/d20/d23/d2d/d35/d13f/de7/f107 [651742,15904] 0 2024-09-16T00:09:50.805 INFO:tasks.workunit.client.0.smithi139.stdout:9/974: creat d19/d1a/d20/d23/d2d/f142 x:0 0 0 2024-09-16T00:09:50.806 INFO:tasks.workunit.client.0.smithi139.stdout:8/841: write d17/d1a/f111 [517662,79645] 0 2024-09-16T00:09:50.809 INFO:tasks.workunit.client.0.smithi139.stdout:8/842: mkdir d17/d1a/d50/d86/d9d/daf/d115 0 2024-09-16T00:09:50.810 INFO:tasks.workunit.client.0.smithi139.stdout:8/843: truncate d17/f83 1447804 0 2024-09-16T00:09:50.811 INFO:tasks.workunit.client.0.smithi139.stdout:9/975: dwrite d19/d1a/d20/d23/d2d/fa4 [0,4194304] 0 2024-09-16T00:09:50.812 INFO:tasks.workunit.client.0.smithi139.stdout:8/844: creat d17/d1f/d3a/dcd/f116 x:0 0 0 2024-09-16T00:09:50.813 INFO:tasks.workunit.client.0.smithi139.stdout:8/845: write d17/f83 [195680,78736] 0 2024-09-16T00:09:50.813 INFO:tasks.workunit.client.0.smithi139.stdout:9/976: rmdir d19/d1a/d20/d23/dca/d71/dc7 39 2024-09-16T00:09:50.815 INFO:tasks.workunit.client.0.smithi139.stdout:8/846: truncate d17/d1f/d3a/db7/d40/d4f/f114 696954 0 2024-09-16T00:09:50.815 INFO:tasks.workunit.client.0.smithi139.stdout:9/977: mknod d19/d1a/d20/d89/c143 0 2024-09-16T00:09:50.816 INFO:tasks.workunit.client.0.smithi139.stdout:9/978: write d19/d1a/d20/d23/d2d/d35/d13f/d3a/fcf [328551,38221] 0 2024-09-16T00:09:50.817 INFO:tasks.workunit.client.0.smithi139.stdout:8/847: creat d17/d1a/d2d/f117 x:0 0 0 2024-09-16T00:09:50.819 INFO:tasks.workunit.client.0.smithi139.stdout:8/848: symlink d17/d89/df2/l118 0 2024-09-16T00:09:50.819 INFO:tasks.workunit.client.0.smithi139.stdout:9/979: mkdir d19/d1a/d20/d23/d2d/d35/d44/df4/d144 0 2024-09-16T00:09:50.822 INFO:tasks.workunit.client.0.smithi139.stdout:8/849: rename d17/d1a/d50/d86/d9d/lce to d17/d1a/d50/l119 0 2024-09-16T00:09:50.823 INFO:tasks.workunit.client.0.smithi139.stdout:8/850: readlink d17/d1a/d50/lc4 0 2024-09-16T00:09:50.825 INFO:tasks.workunit.client.0.smithi139.stdout:9/980: rename d19/d1a/d20/d23/dca/d71/d32/f84 to d19/d1a/d20/d23/d2d/d2f/f145 0 2024-09-16T00:09:50.825 INFO:tasks.workunit.client.0.smithi139.stdout:8/851: symlink d17/d1a/d50/d86/d9d/d9f/l11a 0 2024-09-16T00:09:50.829 INFO:tasks.workunit.client.0.smithi139.stdout:8/852: link d17/d1f/d88/c109 d17/d1f/d3a/d10a/c11b 0 2024-09-16T00:09:50.829 INFO:tasks.workunit.client.0.smithi139.stdout:8/853: chown d17/d1f/d3a/f6f 1 1 2024-09-16T00:09:50.830 INFO:tasks.workunit.client.0.smithi139.stdout:8/854: read - d17/d1a/d50/d86/d9d/daf/fc7 zero size 2024-09-16T00:09:50.830 INFO:tasks.workunit.client.0.smithi139.stdout:8/855: write d17/ff8 [375921,9171] 0 2024-09-16T00:09:50.831 INFO:tasks.workunit.client.0.smithi139.stdout:9/981: truncate d19/f26 2019066 0 2024-09-16T00:09:50.833 INFO:tasks.workunit.client.0.smithi139.stdout:9/982: symlink d19/d1a/d20/d23/dca/d71/da3/l146 0 2024-09-16T00:09:50.834 INFO:tasks.workunit.client.0.smithi139.stdout:9/983: readlink d19/d1a/d20/d23/dca/d71/d32/d79/l95 0 2024-09-16T00:09:50.834 INFO:tasks.workunit.client.0.smithi139.stdout:8/856: mknod d17/d1a/d50/d86/d9d/daf/c11c 0 2024-09-16T00:09:50.836 INFO:tasks.workunit.client.0.smithi139.stdout:9/984: readlink d19/d1a/d20/d23/d2d/d35/d13f/d3a/d45/lab 0 2024-09-16T00:09:50.837 INFO:tasks.workunit.client.0.smithi139.stdout:9/985: write d19/d1a/d20/d23/dca/d71/d32/fb4 [802294,90152] 0 2024-09-16T00:09:50.837 INFO:tasks.workunit.client.0.smithi139.stdout:8/857: mkdir d17/d1a/d50/d86/d9d/daf/d115/d11d 0 2024-09-16T00:09:50.839 INFO:tasks.workunit.client.0.smithi139.stdout:9/986: symlink d19/d1a/d20/d89/dbd/l147 0 2024-09-16T00:09:50.840 INFO:tasks.workunit.client.0.smithi139.stdout:8/858: creat d17/d1f/d3a/db7/f11e x:0 0 0 2024-09-16T00:09:50.842 INFO:tasks.workunit.client.0.smithi139.stdout:8/859: unlink d17/d1f/d3a/cdd 0 2024-09-16T00:09:50.845 INFO:tasks.workunit.client.0.smithi139.stdout:9/987: dwrite d19/d1a/d20/d23/f33 [0,4194304] 0 2024-09-16T00:09:50.849 INFO:tasks.workunit.client.0.smithi139.stdout:9/988: rename d19/d1a/d20/d23/d2d/d2f/f145 to d19/d1a/d20/d23/d2d/d35/d13f/d3a/df3/d110/f148 0 2024-09-16T00:09:50.851 INFO:tasks.workunit.client.0.smithi139.stdout:8/860: dwrite d17/d1f/d3a/db7/f3b [0,4194304] 0 2024-09-16T00:09:50.852 INFO:tasks.workunit.client.0.smithi139.stdout:9/989: truncate d19/d1a/d20/d89/d93/fdb 268857 0 2024-09-16T00:09:50.856 INFO:tasks.workunit.client.0.smithi139.stdout:8/861: symlink d17/d1a/d50/d86/d9d/d9f/d101/l11f 0 2024-09-16T00:09:50.858 INFO:tasks.workunit.client.0.smithi139.stdout:9/990: dwrite d19/d1a/d20/d23/d2d/d35/f61 [0,4194304] 0 2024-09-16T00:09:50.862 INFO:tasks.workunit.client.0.smithi139.stdout:9/991: dread - d19/d1a/d20/d23/d2d/d35/d13f/d3a/dcb/f11b zero size 2024-09-16T00:09:50.862 INFO:tasks.workunit.client.0.smithi139.stdout:8/862: rename d17/d1a/d50/d86/d92/dd6/def to d17/d1f/d3a/db0/d120 0 2024-09-16T00:09:50.868 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:50.867+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:50.868 INFO:tasks.workunit.client.0.smithi139.stdout:8/863: dwrite d17/d1f/d3a/db7/d40/d4f/fdc [0,4194304] 0 2024-09-16T00:09:50.870 INFO:tasks.workunit.client.0.smithi139.stdout:8/864: stat d17/d1f/d3a/db7/cc6 0 2024-09-16T00:09:50.873 INFO:tasks.workunit.client.0.smithi139.stdout:8/865: creat d17/d1f/d10e/f121 x:0 0 0 2024-09-16T00:09:50.875 INFO:tasks.workunit.client.0.smithi139.stdout:9/992: dwrite d19/d1a/d20/d23/d2d/d106/f109 [0,4194304] 0 2024-09-16T00:09:50.879 INFO:tasks.workunit.client.0.smithi139.stdout:8/866: link d17/d1f/d3a/db7/f3b d17/d1a/f122 0 2024-09-16T00:09:50.880 INFO:tasks.workunit.client.0.smithi139.stdout:8/867: chown d17/d1f/f25 462629 1 2024-09-16T00:09:50.880 INFO:tasks.workunit.client.0.smithi139.stdout:8/868: chown d17/d1f/d10e 50 1 2024-09-16T00:09:50.883 INFO:tasks.workunit.client.0.smithi139.stdout:8/869: creat d17/d1f/d10e/f123 x:0 0 0 2024-09-16T00:09:50.884 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:50.884 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:50.885 INFO:tasks.workunit.client.0.smithi139.stdout:9/993: dwrite d19/d1a/d20/d23/dca/d9f/dd9/fee [0,4194304] 0 2024-09-16T00:09:50.885 INFO:tasks.workunit.client.0.smithi139.stdout:8/870: truncate d17/d1a/d2d/ff3 4808499 0 2024-09-16T00:09:50.885 INFO:tasks.workunit.client.0.smithi139.stdout:8/871: chown d17/d1f/d3a/db7/d40 75791129 1 2024-09-16T00:09:50.885 INFO:tasks.workunit.client.0.smithi139.stdout:9/994: fdatasync d19/d1a/d20/d23/dca/d9f/d137/f78 0 2024-09-16T00:09:50.886 INFO:tasks.workunit.client.0.smithi139.stdout:9/995: chown d19/d1a/d20/d23/d2d/d9a 38 1 2024-09-16T00:09:50.893 INFO:tasks.workunit.client.0.smithi139.stdout:8/872: dwrite d17/f83 [0,4194304] 0 2024-09-16T00:09:50.894 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:09:50.896 INFO:tasks.workunit.client.0.smithi139.stdout:8/873: mknod d17/d1f/d3a/dcd/c124 0 2024-09-16T00:09:50.897 INFO:tasks.workunit.client.0.smithi139.stdout:8/874: write d17/d1a/d57/f71 [1272237,97983] 0 2024-09-16T00:09:50.898 INFO:tasks.workunit.client.0.smithi139.stdout:9/996: truncate d19/d1a/d20/d23/d2d/d35/d13f/fea 5402119 0 2024-09-16T00:09:50.899 INFO:tasks.workunit.client.0.smithi139.stdout:8/875: rmdir d17/d1a/d50/d86/d9d/d9f/d101 39 2024-09-16T00:09:50.899 INFO:tasks.workunit.client.0.smithi139.stdout:9/997: read d19/d1a/d20/d23/dca/d9f/d137/fb6 [266490,11520] 0 2024-09-16T00:09:50.901 INFO:tasks.workunit.client.0.smithi139.stdout:8/876: fsync d17/d1a/d2d/fe1 0 2024-09-16T00:09:50.901 INFO:tasks.workunit.client.0.smithi139.stdout:9/998: mknod d19/d1a/d20/d89/d93/dfd/c149 0 2024-09-16T00:09:50.904 INFO:tasks.workunit.client.0.smithi139.stdout:9/999: mknod d19/d1a/d20/d23/d2d/d35/d13f/de7/c14a 0 2024-09-16T00:09:50.906 INFO:tasks.workunit.client.0.smithi139.stdout:8/877: dwrite d17/d1a/ffe [0,4194304] 0 2024-09-16T00:09:50.917 INFO:tasks.workunit.client.0.smithi139.stdout:8/878: dread d17/d54/f55 [4194304,4194304] 0 2024-09-16T00:09:50.918 INFO:tasks.workunit.client.0.smithi139.stdout:8/879: write d17/d1f/d3a/db7/f2b [1947996,115086] 0 2024-09-16T00:09:50.919 INFO:tasks.workunit.client.0.smithi139.stdout:8/880: read d17/d1f/f28 [2652356,52188] 0 2024-09-16T00:09:50.924 INFO:tasks.workunit.client.0.smithi139.stdout:8/881: dwrite d17/d1f/d3a/db7/f2b [0,4194304] 0 2024-09-16T00:09:50.940 INFO:tasks.workunit.client.0.smithi139.stdout:8/882: dread d17/d1f/d3a/f58 [0,4194304] 0 2024-09-16T00:09:50.945 INFO:tasks.workunit.client.0.smithi139.stdout:8/883: dwrite d17/d1f/d3a/db7/d40/fc8 [0,4194304] 0 2024-09-16T00:09:50.946 INFO:tasks.workunit.client.0.smithi139.stdout:8/884: write d17/f56 [875304,69274] 0 2024-09-16T00:09:50.946 INFO:tasks.workunit.client.0.smithi139.stdout:8/885: readlink d17/d1f/d3a/l9a 0 2024-09-16T00:09:50.948 INFO:tasks.workunit.client.0.smithi139.stdout:8/886: creat d17/d1a/d57/f125 x:0 0 0 2024-09-16T00:09:50.949 INFO:tasks.workunit.client.0.smithi139.stdout:7/775: sync 2024-09-16T00:09:50.949 INFO:tasks.workunit.client.0.smithi139.stdout:0/722: sync 2024-09-16T00:09:50.949 INFO:tasks.workunit.client.0.smithi139.stdout:8/887: stat d17/d1a/d50/d86/d92/dd6 0 2024-09-16T00:09:50.950 INFO:tasks.workunit.client.0.smithi139.stdout:7/776: chown d5/d26/d46/d6b/cb5 235 1 2024-09-16T00:09:50.952 INFO:tasks.workunit.client.0.smithi139.stdout:0/723: truncate d6/d1c/d41/d44/dce/fd8 683854 0 2024-09-16T00:09:50.953 INFO:tasks.workunit.client.0.smithi139.stdout:7/777: chown d5/d16/d6d/d7f/dc2/fa8 60536 1 2024-09-16T00:09:50.955 INFO:tasks.workunit.client.0.smithi139.stdout:7/778: write d5/d26/d46/d6b/da7/f105 [795991,32517] 0 2024-09-16T00:09:50.958 INFO:tasks.workunit.client.0.smithi139.stdout:8/888: dread d17/d1a/f73 [0,4194304] 0 2024-09-16T00:09:50.959 INFO:tasks.workunit.client.0.smithi139.stdout:7/779: creat d5/d16/d22/d25/f116 x:0 0 0 2024-09-16T00:09:50.960 INFO:tasks.workunit.client.0.smithi139.stdout:8/889: symlink d17/d1f/d3a/db7/l126 0 2024-09-16T00:09:50.963 INFO:tasks.workunit.client.0.smithi139.stdout:0/724: dread d6/d1c/d41/d44/dd/fb0 [0,4194304] 0 2024-09-16T00:09:50.963 INFO:tasks.workunit.client.0.smithi139.stdout:7/780: rename d5/d26/d46/ff9 to d5/d16/d22/d71/f117 0 2024-09-16T00:09:50.964 INFO:tasks.workunit.client.0.smithi139.stdout:8/890: mknod d17/d1f/d3a/dad/c127 0 2024-09-16T00:09:50.968 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:50.968+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:50.969 INFO:tasks.workunit.client.0.smithi139.stdout:0/725: dwrite d6/d1c/d41/d44/d58/faf [0,4194304] 0 2024-09-16T00:09:50.970 INFO:tasks.workunit.client.0.smithi139.stdout:7/781: truncate d5/d26/f2b 1100363 0 2024-09-16T00:09:50.972 INFO:tasks.workunit.client.0.smithi139.stdout:7/782: mkdir d5/d37/d118 0 2024-09-16T00:09:50.972 INFO:tasks.workunit.client.0.smithi139.stdout:0/726: symlink d6/le3 0 2024-09-16T00:09:50.973 INFO:tasks.workunit.client.0.smithi139.stdout:0/727: fdatasync d6/d1c/d41/d78/fd9 0 2024-09-16T00:09:50.973 INFO:tasks.workunit.client.0.smithi139.stdout:7/783: unlink d5/d16/d6d/d7f/dc2/d50/f67 0 2024-09-16T00:09:50.974 INFO:tasks.workunit.client.0.smithi139.stdout:0/728: chown d6/d1c/d41/d76/fdd 231197902 1 2024-09-16T00:09:50.978 INFO:tasks.workunit.client.0.smithi139.stdout:0/729: link d6/d1c/d41/d44/dd/fd0 d6/d1c/d41/d7c/fe4 0 2024-09-16T00:09:50.978 INFO:tasks.workunit.client.0.smithi139.stdout:7/784: dwrite d5/d26/d42/f5b [0,4194304] 0 2024-09-16T00:09:50.979 INFO:tasks.workunit.client.0.smithi139.stdout:0/730: chown d6/fd1 96782 1 2024-09-16T00:09:50.982 INFO:tasks.workunit.client.0.smithi139.stdout:7/785: mkdir d5/d26/d46/d6b/dd6/d119 0 2024-09-16T00:09:50.983 INFO:tasks.workunit.client.0.smithi139.stdout:0/731: mknod d6/d1c/d41/d44/ce5 0 2024-09-16T00:09:50.984 INFO:tasks.workunit.client.0.smithi139.stdout:7/786: read d5/d26/d46/d6b/da7/f105 [213485,123771] 0 2024-09-16T00:09:50.986 INFO:tasks.workunit.client.0.smithi139.stdout:7/787: creat d5/d26/d42/f11a x:0 0 0 2024-09-16T00:09:50.987 INFO:tasks.workunit.client.0.smithi139.stdout:7/788: stat d5/c1b 0 2024-09-16T00:09:50.990 INFO:tasks.workunit.client.0.smithi139.stdout:0/732: dwrite d6/d1c/d41/fbb [0,4194304] 0 2024-09-16T00:09:50.993 INFO:tasks.workunit.client.0.smithi139.stdout:7/789: rename d5/d16/d87/lde to d5/d16/d22/l11b 0 2024-09-16T00:09:50.995 INFO:tasks.workunit.client.0.smithi139.stdout:7/790: symlink d5/d26/d46/d6b/da7/ddb/l11c 0 2024-09-16T00:09:50.996 INFO:tasks.workunit.client.0.smithi139.stdout:0/733: link d6/d1c/d41/cc6 d6/d1c/d41/d44/dce/ce6 0 2024-09-16T00:09:50.997 INFO:tasks.workunit.client.0.smithi139.stdout:7/791: write d5/d26/d46/dea/df8/f114 [448659,114933] 0 2024-09-16T00:09:50.998 INFO:tasks.workunit.client.0.smithi139.stdout:0/734: mkdir d6/d5a/de7 0 2024-09-16T00:09:50.998 INFO:tasks.workunit.client.0.smithi139.stdout:0/735: fsync d6/d1c/d41/d76/fdd 0 2024-09-16T00:09:50.999 INFO:tasks.workunit.client.0.smithi139.stdout:0/736: chown d6/ff 0 1 2024-09-16T00:09:50.999 INFO:tasks.workunit.client.0.smithi139.stdout:7/792: creat d5/d26/d46/dea/df8/f11d x:0 0 0 2024-09-16T00:09:51.005 INFO:tasks.workunit.client.0.smithi139.stdout:0/737: dwrite d6/d1c/d41/d44/dd/d8a/d9e/fb6 [0,4194304] 0 2024-09-16T00:09:51.007 INFO:tasks.workunit.client.0.smithi139.stdout:0/738: chown d6/d1c/d41/d7c/fe4 750418390 1 2024-09-16T00:09:51.009 INFO:tasks.workunit.client.0.smithi139.stdout:0/739: symlink d6/d1c/d41/d63/le8 0 2024-09-16T00:09:51.015 INFO:tasks.workunit.client.0.smithi139.stdout:0/740: dwrite d6/d1c/d41/d63/d82/f85 [0,4194304] 0 2024-09-16T00:09:51.018 INFO:tasks.workunit.client.0.smithi139.stdout:0/741: creat d6/d1c/d41/d44/dd/d56/d75/fe9 x:0 0 0 2024-09-16T00:09:51.020 INFO:tasks.workunit.client.0.smithi139.stdout:0/742: write d6/d1c/d41/d44/dd/d56/d75/fe9 [965434,53102] 0 2024-09-16T00:09:51.021 INFO:tasks.workunit.client.0.smithi139.stdout:7/793: dread d5/d16/d6d/d7f/dc2/d81/fa1 [0,4194304] 0 2024-09-16T00:09:51.022 INFO:tasks.workunit.client.0.smithi139.stdout:0/743: symlink d6/d1c/d41/d7c/lea 0 2024-09-16T00:09:51.024 INFO:tasks.workunit.client.0.smithi139.stdout:7/794: mknod d5/d26/d46/d6b/dd6/d119/c11e 0 2024-09-16T00:09:51.028 INFO:tasks.workunit.client.0.smithi139.stdout:7/795: dwrite d5/fc [4194304,4194304] 0 2024-09-16T00:09:51.030 INFO:tasks.workunit.client.0.smithi139.stdout:7/796: mknod d5/d26/d46/d6b/dd6/df4/c11f 0 2024-09-16T00:09:51.036 INFO:tasks.workunit.client.0.smithi139.stdout:7/797: write d5/d37/f86 [1904457,96723] 0 2024-09-16T00:09:51.038 INFO:tasks.workunit.client.0.smithi139.stdout:7/798: fdatasync d5/d26/d46/d6b/da7/fe2 0 2024-09-16T00:09:51.040 INFO:tasks.workunit.client.0.smithi139.stdout:7/799: creat d5/d16/dc6/f120 x:0 0 0 2024-09-16T00:09:51.041 INFO:tasks.workunit.client.0.smithi139.stdout:7/800: truncate d5/d26/d46/dea/df8/f114 928482 0 2024-09-16T00:09:51.042 INFO:tasks.workunit.client.0.smithi139.stdout:7/801: fsync d5/d16/d22/f6f 0 2024-09-16T00:09:51.047 INFO:tasks.workunit.client.0.smithi139.stdout:7/802: write d5/d16/d6d/d7f/dc2/d50/d44/fbb [772181,111761] 0 2024-09-16T00:09:51.050 INFO:tasks.workunit.client.0.smithi139.stdout:7/803: chown d5/d16/d6d/d7f/dc2/f101 525860369 1 2024-09-16T00:09:51.050 INFO:tasks.workunit.client.0.smithi139.stdout:7/804: chown d5/d16/l24 817175 1 2024-09-16T00:09:51.053 INFO:tasks.workunit.client.0.smithi139.stdout:7/805: link d5/d26/d46/d6b/dc5/cb0 d5/d26/d46/d6b/dd6/df4/c121 0 2024-09-16T00:09:51.057 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:51.057 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:51.061 INFO:tasks.workunit.client.0.smithi139.stdout:7/806: dwrite d5/d16/d22/d25/d63/fee [4194304,4194304] 0 2024-09-16T00:09:51.064 INFO:tasks.workunit.client.0.smithi139.stdout:7/807: fdatasync d5/d16/d6d/d7f/dc2/d50/d44/d55/d90/f10d 0 2024-09-16T00:09:51.065 INFO:tasks.workunit.client.0.smithi139.stdout:7/808: write d5/d16/d87/feb [334038,10544] 0 2024-09-16T00:09:51.067 INFO:tasks.workunit.client.0.smithi139.stdout:7/809: mknod d5/d26/d46/d6b/dc5/c122 0 2024-09-16T00:09:51.068 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:09:51.070 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:51.069+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:51.071 INFO:tasks.workunit.client.0.smithi139.stdout:7/810: rename d5/d16/d6d/d7f/dc2/f101 to d5/d16/d6d/d7f/df5/f123 0 2024-09-16T00:09:51.101 INFO:tasks.workunit.client.0.smithi139.stdout:2/891: sync 2024-09-16T00:09:51.106 INFO:tasks.workunit.client.0.smithi139.stdout:2/892: dwrite d11/d30/dbf/d116/d125/f129 [0,4194304] 0 2024-09-16T00:09:51.109 INFO:tasks.workunit.client.0.smithi139.stdout:2/893: dread d11/d30/d110/d29/d4f/fb2 [0,4194304] 0 2024-09-16T00:09:51.110 INFO:tasks.workunit.client.0.smithi139.stdout:2/894: fdatasync d11/d30/d110/d29/d86/df3/f122 0 2024-09-16T00:09:51.116 INFO:tasks.workunit.client.0.smithi139.stdout:2/895: write d11/f1e [5028020,10535] 0 2024-09-16T00:09:51.118 INFO:tasks.workunit.client.0.smithi139.stdout:2/896: mknod d11/d2f/dcc/dea/c133 0 2024-09-16T00:09:51.119 INFO:tasks.workunit.client.0.smithi139.stdout:2/897: read - d11/d2f/f61 zero size 2024-09-16T00:09:51.121 INFO:tasks.workunit.client.0.smithi139.stdout:2/898: getdents d11/d30/d110/d29/d86/df3/d108 0 2024-09-16T00:09:51.122 INFO:tasks.workunit.client.0.smithi139.stdout:2/899: symlink d11/d2f/dcc/l134 0 2024-09-16T00:09:51.123 INFO:tasks.workunit.client.0.smithi139.stdout:2/900: chown d11/d30/d110/d50/dcb/f11d 38959091 1 2024-09-16T00:09:51.123 INFO:tasks.workunit.client.0.smithi139.stdout:2/901: readlink d11/d30/d110/d29/d2b/lbd 0 2024-09-16T00:09:51.125 INFO:tasks.workunit.client.0.smithi139.stdout:2/902: mknod d11/d45/c135 0 2024-09-16T00:09:51.125 INFO:tasks.workunit.client.0.smithi139.stdout:2/903: chown d11/f41 41582892 1 2024-09-16T00:09:51.128 INFO:tasks.workunit.client.0.smithi139.stdout:2/904: link d11/d30/d110/d29/d2b/fa7 d11/d30/d110/d5d/d73/d111/f136 0 2024-09-16T00:09:51.130 INFO:tasks.workunit.client.0.smithi139.stdout:2/905: read d11/d30/d110/d5d/d73/d111/f136 [9900,57756] 0 2024-09-16T00:09:51.131 INFO:tasks.workunit.client.0.smithi139.stdout:2/906: fdatasync d11/d30/d110/d29/da5/d81/fe9 0 2024-09-16T00:09:51.131 INFO:tasks.workunit.client.0.smithi139.stdout:2/907: stat d11/d30/d110/d29/d4f/cda 0 2024-09-16T00:09:51.135 INFO:tasks.workunit.client.0.smithi139.stdout:2/908: dread d11/d30/d110/d5d/d73/fa3 [0,4194304] 0 2024-09-16T00:09:51.170 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:51.170+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:51.179 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:51.179 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:51.188 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:09:51.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:51.240+0000 7f7d2affd640 1 -- 172.21.15.139:0/2533402169 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v90) ==== 101+0+0 (secure 0 0 0) 0x7f7d3c0c6f30 con 0x7f7d3c0c67a0 2024-09-16T00:09:51.241 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is unset 2024-09-16T00:09:51.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:51.242+0000 7f7d40830640 1 -- 172.21.15.139:0/2533402169 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7d1c06c2b0 msgr2=0x7f7d1c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:51.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:51.242+0000 7f7d40830640 1 --2- 172.21.15.139:0/2533402169 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7d1c06c2b0 0x7f7d1c06e770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c034000 tx=0x7f7d2c02e000 comp rx=0 tx=0).stop 2024-09-16T00:09:51.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:51.243+0000 7f7d40830640 1 -- 172.21.15.139:0/2533402169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d3c0c67a0 msgr2=0x7f7d3c14df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:51.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:51.243+0000 7f7d40830640 1 --2- 172.21.15.139:0/2533402169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d3c0c67a0 0x7f7d3c14df50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000d830 tx=0x7f7d3000dcf0 comp rx=0 tx=0).stop 2024-09-16T00:09:51.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:51.243+0000 7f7d40830640 1 -- 172.21.15.139:0/2533402169 shutdown_connections 2024-09-16T00:09:51.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:51.243+0000 7f7d40830640 1 --2- 172.21.15.139:0/2533402169 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7d1c06c2b0 0x7f7d1c06e770 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:51.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:51.243+0000 7f7d40830640 1 --2- 172.21.15.139:0/2533402169 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7d3c0c7220 0x7f7d3c1f0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:51.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:51.243+0000 7f7d40830640 1 --2- 172.21.15.139:0/2533402169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d3c0c67a0 0x7f7d3c14df50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:51.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:51.243+0000 7f7d40830640 1 --2- 172.21.15.139:0/2533402169 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7d3c0c63c0 0x7f7d3c1567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:51.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:51.243+0000 7f7d40830640 1 -- 172.21.15.139:0/2533402169 >> 172.21.15.139:0/2533402169 conn(0x7f7d3c072b30 msgr2=0x7f7d3c0c5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:51.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:51.243+0000 7f7d40830640 1 -- 172.21.15.139:0/2533402169 shutdown_connections 2024-09-16T00:09:51.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:51.243+0000 7f7d40830640 1 -- 172.21.15.139:0/2533402169 wait complete. 2024-09-16T00:09:51.245 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2024-09-16T00:09:51.245 INFO:tasks.thrashosds.thrasher:Reviving osd 6 2024-09-16T00:09:51.245 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-16T00:09:51.245 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-16T00:09:51.259 INFO:tasks.workunit.client.0.smithi139.stdout:4/838: sync 2024-09-16T00:09:51.259 INFO:tasks.workunit.client.0.smithi139.stdout:3/634: sync 2024-09-16T00:09:51.260 INFO:tasks.workunit.client.0.smithi139.stdout:3/635: write d5/d7/d23/d27/d2f/d30/d4d/dbc/fc5 [1002054,33558] 0 2024-09-16T00:09:51.264 INFO:tasks.workunit.client.0.smithi139.stdout:3/636: mkdir d5/d7/d23/d27/d2f/d30/d4c/d6d/db3/ddc 0 2024-09-16T00:09:51.265 INFO:tasks.workunit.client.0.smithi139.stdout:4/839: dwrite d12/d47/d49/d60/f76 [0,4194304] 0 2024-09-16T00:09:51.267 INFO:tasks.workunit.client.0.smithi139.stdout:4/840: creat d12/d47/d49/f112 x:0 0 0 2024-09-16T00:09:51.269 INFO:tasks.workunit.client.0.smithi139.stdout:3/637: dread d5/d7/d23/d27/d2f/f7a [0,4194304] 0 2024-09-16T00:09:51.271 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:51.270+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:51.272 INFO:tasks.ceph.osd.6:Started 2024-09-16T00:09:51.273 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:09:51.276 INFO:tasks.workunit.client.0.smithi139.stdout:4/841: dwrite d12/d47/db2/d4d/d69/fb4 [0,4194304] 0 2024-09-16T00:09:51.281 INFO:tasks.workunit.client.0.smithi139.stdout:4/842: mknod d12/d47/db2/d4d/de4/c113 0 2024-09-16T00:09:51.283 INFO:tasks.workunit.client.0.smithi139.stdout:3/638: truncate d5/d7/d23/d27/d2f/d30/d4c/d50/fcd 400381 0 2024-09-16T00:09:51.287 INFO:tasks.workunit.client.0.smithi139.stdout:4/843: creat d12/d47/f114 x:0 0 0 2024-09-16T00:09:51.287 INFO:tasks.workunit.client.0.smithi139.stdout:3/639: truncate d5/d7/d23/d27/d2f/d30/d41/f71 734891 0 2024-09-16T00:09:51.288 INFO:tasks.workunit.client.0.smithi139.stdout:4/844: write d12/d47/db2/d4d/de4/fca [2389438,67141] 0 2024-09-16T00:09:51.289 INFO:tasks.workunit.client.0.smithi139.stdout:4/845: readlink d12/d47/d49/d60/l8e 0 2024-09-16T00:09:51.290 INFO:tasks.workunit.client.0.smithi139.stdout:4/846: creat d12/d47/db2/d4d/dd6/f115 x:0 0 0 2024-09-16T00:09:51.292 INFO:tasks.workunit.client.0.smithi139.stdout:3/640: getdents d5/d7/d23/d27/d2f/d30/d41/d68 0 2024-09-16T00:09:51.345 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:51.345 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:51.354 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:09:51.373 INFO:tasks.workunit.client.0.smithi139.stdout:7/811: sync 2024-09-16T00:09:51.373 INFO:tasks.workunit.client.0.smithi139.stdout:5/812: sync 2024-09-16T00:09:51.373 INFO:tasks.workunit.client.0.smithi139.stdout:2/909: sync 2024-09-16T00:09:51.373 INFO:tasks.workunit.client.0.smithi139.stdout:0/744: sync 2024-09-16T00:09:51.373 INFO:tasks.workunit.client.0.smithi139.stdout:8/891: sync 2024-09-16T00:09:51.373 INFO:tasks.workunit.client.0.smithi139.stdout:6/656: sync 2024-09-16T00:09:51.373 INFO:tasks.workunit.client.0.smithi139.stdout:1/616: sync 2024-09-16T00:09:51.374 INFO:tasks.workunit.client.0.smithi139.stdout:5/813: stat d1/d13/d22/de8/d10d 0 2024-09-16T00:09:51.375 INFO:tasks.workunit.client.0.smithi139.stdout:0/745: mkdir d6/d5a/deb 0 2024-09-16T00:09:51.375 INFO:tasks.workunit.client.0.smithi139.stdout:1/617: fdatasync d7/d8/d40/d47/f5e 0 2024-09-16T00:09:51.375 INFO:tasks.workunit.client.0.smithi139.stdout:0/746: chown d6/d42/db4 1 1 2024-09-16T00:09:51.376 INFO:tasks.workunit.client.0.smithi139.stdout:5/814: write d1/d21/f39 [808555,76571] 0 2024-09-16T00:09:51.376 INFO:tasks.workunit.client.0.smithi139.stdout:2/910: symlink d11/d30/l137 0 2024-09-16T00:09:51.377 INFO:tasks.workunit.client.0.smithi139.stdout:7/812: fsync d5/d16/d22/d25/faf 0 2024-09-16T00:09:51.377 INFO:tasks.workunit.client.0.smithi139.stdout:5/815: readlink d1/d21/d49/d87/lbb 0 2024-09-16T00:09:51.378 INFO:tasks.workunit.client.0.smithi139.stdout:5/816: fdatasync d1/d21/d49/db2/fdf 0 2024-09-16T00:09:51.379 INFO:tasks.workunit.client.0.smithi139.stdout:5/817: dread - d1/d13/d22/da9/f101 zero size 2024-09-16T00:09:51.380 INFO:teuthology.orchestra.run.smithi164.stderr:no valid command found; 10 closest matches: 2024-09-16T00:09:51.381 INFO:teuthology.orchestra.run.smithi164.stderr:0 2024-09-16T00:09:51.381 INFO:teuthology.orchestra.run.smithi164.stderr:1 2024-09-16T00:09:51.381 INFO:teuthology.orchestra.run.smithi164.stderr:2 2024-09-16T00:09:51.381 INFO:teuthology.orchestra.run.smithi164.stderr:abort 2024-09-16T00:09:51.381 INFO:teuthology.orchestra.run.smithi164.stderr:assert 2024-09-16T00:09:51.381 INFO:teuthology.orchestra.run.smithi164.stderr:config diff 2024-09-16T00:09:51.381 INFO:teuthology.orchestra.run.smithi164.stderr:config diff get 2024-09-16T00:09:51.381 INFO:teuthology.orchestra.run.smithi164.stderr:config get 2024-09-16T00:09:51.381 INFO:teuthology.orchestra.run.smithi164.stderr:config help [] 2024-09-16T00:09:51.381 INFO:teuthology.orchestra.run.smithi164.stderr:config set ... 2024-09-16T00:09:51.381 INFO:teuthology.orchestra.run.smithi164.stderr:admin_socket: invalid command 2024-09-16T00:09:51.382 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:51.380+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:51.383 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T00:09:51.384 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2024-09-16T00:09:51.384 INFO:tasks.workunit.client.0.smithi139.stdout:1/618: dread d7/d8/f11 [0,4194304] 0 2024-09-16T00:09:51.385 INFO:tasks.workunit.client.0.smithi139.stdout:0/747: mknod d6/d1c/cec 0 2024-09-16T00:09:51.390 INFO:tasks.workunit.client.0.smithi139.stdout:8/892: getdents d17/d89 0 2024-09-16T00:09:51.391 INFO:tasks.workunit.client.0.smithi139.stdout:0/748: dwrite d6/d1c/d41/d7c/dad/fd2 [0,4194304] 0 2024-09-16T00:09:51.392 INFO:tasks.workunit.client.0.smithi139.stdout:1/619: fdatasync d7/d8/d40/d46/d8d/f95 0 2024-09-16T00:09:51.393 INFO:tasks.workunit.client.0.smithi139.stdout:6/657: link d8/d29/d83/d26/d45/d42/c9f d8/d29/d83/d26/d28/d6d/dd3/de4/cea 0 2024-09-16T00:09:51.394 INFO:tasks.workunit.client.0.smithi139.stdout:8/893: mkdir d17/d1f/d3a/db0/d128 0 2024-09-16T00:09:51.396 INFO:tasks.workunit.client.0.smithi139.stdout:5/818: dread d1/d13/f17 [0,4194304] 0 2024-09-16T00:09:51.398 INFO:tasks.workunit.client.0.smithi139.stdout:8/894: chown d17/d1a/d57/f79 22 1 2024-09-16T00:09:51.400 INFO:tasks.workunit.client.0.smithi139.stdout:0/749: creat d6/d1c/d41/d78/fed x:0 0 0 2024-09-16T00:09:51.401 INFO:tasks.workunit.client.0.smithi139.stdout:2/911: dwrite d11/d30/d110/d5d/d73/d111/f136 [0,4194304] 0 2024-09-16T00:09:51.403 INFO:tasks.workunit.client.0.smithi139.stdout:1/620: dwrite d7/d8/d27/f34 [4194304,4194304] 0 2024-09-16T00:09:51.403 INFO:tasks.workunit.client.0.smithi139.stdout:6/658: write d8/d20/d27/f97 [4389457,47717] 0 2024-09-16T00:09:51.404 INFO:tasks.workunit.client.0.smithi139.stdout:1/621: stat d7/d8/c35 0 2024-09-16T00:09:51.404 INFO:tasks.workunit.client.0.smithi139.stdout:8/895: symlink d17/d1f/d3a/db7/d40/l129 0 2024-09-16T00:09:51.406 INFO:tasks.workunit.client.0.smithi139.stdout:6/659: stat d8/d29/d83/d26/d45/d42/f76 0 2024-09-16T00:09:51.407 INFO:tasks.workunit.client.0.smithi139.stdout:5/819: rename d1/d13/d22/de8/d10d/d33/d43/f9c to d1/d21/d49/d87/f11c 0 2024-09-16T00:09:51.410 INFO:tasks.workunit.client.0.smithi139.stdout:8/896: unlink fb 0 2024-09-16T00:09:51.413 INFO:tasks.workunit.client.0.smithi139.stdout:1/622: creat d7/d8/d27/d68/fb4 x:0 0 0 2024-09-16T00:09:51.413 INFO:tasks.workunit.client.0.smithi139.stdout:6/660: rmdir d8/d1f 39 2024-09-16T00:09:51.415 INFO:tasks.workunit.client.0.smithi139.stdout:0/750: write d6/d1c/f4c [4893619,73121] 0 2024-09-16T00:09:51.415 INFO:tasks.workunit.client.0.smithi139.stdout:6/661: write d8/d29/d83/d26/f4b [64545,58136] 0 2024-09-16T00:09:51.415 INFO:tasks.workunit.client.0.smithi139.stdout:1/623: read d7/d8/d40/d48/d53/f85 [1182040,24215] 0 2024-09-16T00:09:51.416 INFO:tasks.workunit.client.0.smithi139.stdout:8/897: creat d17/d1f/d3a/dad/f12a x:0 0 0 2024-09-16T00:09:51.420 INFO:tasks.workunit.client.0.smithi139.stdout:0/751: write d6/d1c/d41/d44/dd/fb0 [2499314,118463] 0 2024-09-16T00:09:51.421 INFO:tasks.workunit.client.0.smithi139.stdout:2/912: getdents d11/d30/d110/d29/d4f 0 2024-09-16T00:09:51.421 INFO:tasks.workunit.client.0.smithi139.stdout:0/752: dread - d6/d5a/d70/fcd zero size 2024-09-16T00:09:51.423 INFO:tasks.workunit.client.0.smithi139.stdout:0/753: chown d6/d1c/d41/d78/f7d 289 1 2024-09-16T00:09:51.425 INFO:tasks.workunit.client.0.smithi139.stdout:1/624: fdatasync d7/f81 0 2024-09-16T00:09:51.427 INFO:tasks.workunit.client.0.smithi139.stdout:2/913: dwrite d11/d30/d110/d29/d2b/f98 [0,4194304] 0 2024-09-16T00:09:51.430 INFO:tasks.workunit.client.0.smithi139.stdout:0/754: mknod d6/d1c/d41/d44/d6b/cee 0 2024-09-16T00:09:51.432 INFO:tasks.workunit.client.0.smithi139.stdout:5/820: rename d1/c72 to d1/c11d 0 2024-09-16T00:09:51.433 INFO:tasks.workunit.client.0.smithi139.stdout:8/898: mkdir d17/d1a/d50/dee/d12b 0 2024-09-16T00:09:51.435 INFO:tasks.workunit.client.0.smithi139.stdout:6/662: write d8/d29/d83/d26/d45/d65/f3b [1459882,24733] 0 2024-09-16T00:09:51.440 INFO:tasks.workunit.client.0.smithi139.stdout:5/821: rename d1/d13/d22/de8 to d1/d13/d3f/d9f/d73/d79/da5/d10b/d11e 0 2024-09-16T00:09:51.441 INFO:tasks.workunit.client.0.smithi139.stdout:1/625: truncate d7/f3f 377111 0 2024-09-16T00:09:51.443 INFO:tasks.workunit.client.0.smithi139.stdout:4/847: sync 2024-09-16T00:09:51.444 INFO:tasks.workunit.client.0.smithi139.stdout:8/899: fdatasync d17/d1a/f111 0 2024-09-16T00:09:51.445 INFO:tasks.workunit.client.0.smithi139.stdout:5/822: creat d1/d13/d3f/d9f/f11f x:0 0 0 2024-09-16T00:09:51.447 INFO:tasks.workunit.client.0.smithi139.stdout:6/663: dread d8/d29/f49 [0,4194304] 0 2024-09-16T00:09:51.448 INFO:tasks.workunit.client.0.smithi139.stdout:2/914: dwrite d11/d30/d110/d29/f5c [0,4194304] 0 2024-09-16T00:09:51.449 INFO:tasks.workunit.client.0.smithi139.stdout:0/755: getdents d6/d5a/d70 0 2024-09-16T00:09:51.450 INFO:tasks.workunit.client.0.smithi139.stdout:8/900: symlink d17/d1f/d3a/db0/l12c 0 2024-09-16T00:09:51.450 INFO:tasks.workunit.client.0.smithi139.stdout:6/664: chown d8/d29/d83/d26/d28/d74/cd2 2 1 2024-09-16T00:09:51.452 INFO:tasks.workunit.client.0.smithi139.stdout:4/848: rename d12/d47/db2/d4d/d69/f8a to d12/d47/db2/d4d/d69/f116 0 2024-09-16T00:09:51.453 INFO:tasks.workunit.client.0.smithi139.stdout:1/626: link d7/d8/l92 d7/d8/d40/d48/lb5 0 2024-09-16T00:09:51.454 INFO:tasks.workunit.client.0.smithi139.stdout:5/823: dwrite d1/d21/f39 [0,4194304] 0 2024-09-16T00:09:51.455 INFO:tasks.workunit.client.0.smithi139.stdout:2/915: unlink d11/d2f/ca6 0 2024-09-16T00:09:51.458 INFO:tasks.workunit.client.0.smithi139.stdout:1/627: rmdir d7/d8 39 2024-09-16T00:09:51.460 INFO:tasks.workunit.client.0.smithi139.stdout:4/849: chown d12/f31 2767427 1 2024-09-16T00:09:51.460 INFO:tasks.workunit.client.0.smithi139.stdout:0/756: dwrite d6/d1c/d41/d63/d82/fc7 [0,4194304] 0 2024-09-16T00:09:51.460 INFO:tasks.workunit.client.0.smithi139.stdout:5/824: dread - d1/d13/d3f/d9f/d73/d79/da5/d10b/d11e/d10d/d33/d43/f4d zero size 2024-09-16T00:09:51.461 INFO:tasks.workunit.client.0.smithi139.stdout:6/665: getdents d8/d29/d83/d26/d28/ddd 0 2024-09-16T00:09:51.463 INFO:tasks.workunit.client.0.smithi139.stdout:5/825: fsync d1/d21/f39 0 2024-09-16T00:09:51.464 INFO:tasks.workunit.client.0.smithi139.stdout:4/850: symlink d12/d2b/d8c/l117 0 2024-09-16T00:09:51.465 INFO:tasks.workunit.client.0.smithi139.stdout:0/757: chown d6/d1c/d41/d44/d6b/c9b 543 1 2024-09-16T00:09:51.466 INFO:tasks.workunit.client.0.smithi139.stdout:0/758: write d6/d1c/d41/d44/d58/fd5 [14698,20023] 0 2024-09-16T00:09:51.469 INFO:tasks.workunit.client.0.smithi139.stdout:6/666: readlink d8/d20/d27/l34 0 2024-09-16T00:09:51.471 INFO:tasks.workunit.client.0.smithi139.stdout:5/826: creat d1/d21/d49/d53/d6a/d7b/d99/f120 x:0 0 0 2024-09-16T00:09:51.471 INFO:tasks.workunit.client.0.smithi139.stdout:6/667: dread d8/d29/d83/d26/d28/f95 [0,4194304] 0 2024-09-16T00:09:51.472 INFO:tasks.workunit.client.0.smithi139.stdout:0/759: dread d6/d1c/fc1 [0,4194304] 0 2024-09-16T00:09:51.473 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:51.472+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:51.474 INFO:tasks.workunit.client.0.smithi139.stdout:4/851: dread d12/d47/d49/fc0 [0,4194304] 0 2024-09-16T00:09:51.474 INFO:tasks.workunit.client.0.smithi139.stdout:2/916: truncate d11/d30/d110/d29/d4f/d9c/fab 507903 0 2024-09-16T00:09:51.476 INFO:tasks.workunit.client.0.smithi139.stdout:2/917: readlink d11/d30/d110/d5d/l91 0 2024-09-16T00:09:51.476 INFO:tasks.workunit.client.0.smithi139.stdout:4/852: dread - d12/d47/db2/d4d/db0/f10b zero size 2024-09-16T00:09:51.479 INFO:tasks.workunit.client.0.smithi139.stdout:0/760: rmdir d6/d1c/d41/d44/dd/d8a 39 2024-09-16T00:09:51.480 INFO:tasks.workunit.client.0.smithi139.stdout:8/901: getdents d17/d1a/d50/d86 0 2024-09-16T00:09:51.482 INFO:tasks.workunit.client.0.smithi139.stdout:2/918: dwrite fc [4194304,4194304] 0 2024-09-16T00:09:51.482 INFO:tasks.workunit.client.0.smithi139.stdout:0/761: write d6/d1c/d41/d7c/d86/fd6 [257172,104346] 0 2024-09-16T00:09:51.482 INFO:tasks.workunit.client.0.smithi139.stdout:1/628: symlink d7/d8/d40/lb6 0 2024-09-16T00:09:51.483 INFO:tasks.workunit.client.0.smithi139.stdout:5/827: dread d1/d21/d49/d87/f11c [0,4194304] 0 2024-09-16T00:09:51.484 INFO:tasks.workunit.client.0.smithi139.stdout:4/853: stat d12/d47/d49/c85 0 2024-09-16T00:09:51.484 INFO:tasks.workunit.client.0.smithi139.stdout:0/762: chown d6/d5a/l6d 702052011 1 2024-09-16T00:09:51.485 INFO:tasks.workunit.client.0.smithi139.stdout:2/919: fsync d11/d30/d110/d29/da5/d81/f99 0 2024-09-16T00:09:51.485 INFO:tasks.workunit.client.0.smithi139.stdout:6/668: mknod d8/d29/d83/d26/d45/ceb 0 2024-09-16T00:09:51.486 INFO:tasks.workunit.client.0.smithi139.stdout:4/854: fsync d12/d47/d49/f7e 0 2024-09-16T00:09:51.486 INFO:tasks.workunit.client.0.smithi139.stdout:5/828: dread - d1/d21/d49/d87/fff zero size 2024-09-16T00:09:51.487 INFO:tasks.workunit.client.0.smithi139.stdout:1/629: fdatasync d7/d8/d1f/f76 0 2024-09-16T00:09:51.487 INFO:tasks.workunit.client.0.smithi139.stdout:8/902: chown d17/d1a/ffe 1614 1 2024-09-16T00:09:51.488 INFO:tasks.workunit.client.0.smithi139.stdout:2/920: write d11/d30/d110/d29/d2b/f31 [2394517,36091] 0 2024-09-16T00:09:51.489 INFO:tasks.workunit.client.0.smithi139.stdout:5/829: write d1/d13/d3f/fd3 [152889,10373] 0 2024-09-16T00:09:51.489 INFO:tasks.workunit.client.0.smithi139.stdout:0/763: creat d6/d1c/d41/d44/dd/d56/d75/fef x:0 0 0 2024-09-16T00:09:51.491 INFO:tasks.workunit.client.0.smithi139.stdout:6/669: truncate d8/d29/d83/d26/d45/d42/fcd 708792 0 2024-09-16T00:09:51.492 INFO:tasks.workunit.client.0.smithi139.stdout:4/855: creat d12/d47/db2/d4d/da9/f118 x:0 0 0 2024-09-16T00:09:51.493 INFO:tasks.workunit.client.0.smithi139.stdout:8/903: creat d17/d1a/d50/d86/d9d/daf/f12d x:0 0 0 2024-09-16T00:09:51.494 INFO:tasks.workunit.client.0.smithi139.stdout:2/921: fdatasync d11/d30/d110/d29/da5/d81/f89 0 2024-09-16T00:09:51.494 INFO:tasks.workunit.client.0.smithi139.stdout:0/764: unlink d6/d1c/d41/d44/dd/le 0 2024-09-16T00:09:51.495 INFO:tasks.workunit.client.0.smithi139.stdout:5/830: creat d1/d21/d49/d53/d6a/d7b/f121 x:0 0 0 2024-09-16T00:09:51.497 INFO:tasks.workunit.client.0.smithi139.stdout:4/856: mknod d12/d2b/c119 0 2024-09-16T00:09:51.497 INFO:tasks.workunit.client.0.smithi139.stdout:6/670: read d8/d29/d83/d26/d45/d42/fcd [170432,129471] 0 2024-09-16T00:09:51.499 INFO:tasks.workunit.client.0.smithi139.stdout:2/922: mknod d11/d30/d110/c138 0 2024-09-16T00:09:51.500 INFO:tasks.workunit.client.0.smithi139.stdout:0/765: symlink d6/d5a/de7/lf0 0 2024-09-16T00:09:51.500 INFO:tasks.workunit.client.0.smithi139.stdout:1/630: dwrite d7/d8/d40/d48/d4a/f98 [0,4194304] 0 2024-09-16T00:09:51.501 INFO:tasks.workunit.client.0.smithi139.stdout:4/857: chown d12/d47/db2/d4d/da9/de1 2 1 2024-09-16T00:09:51.501 INFO:tasks.workunit.client.0.smithi139.stdout:6/671: readlink d8/d1f/l5d 0 2024-09-16T00:09:51.502 INFO:tasks.workunit.client.0.smithi139.stdout:2/923: fsync d11/f16 0 2024-09-16T00:09:51.503 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:51.504 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:51.504 INFO:tasks.workunit.client.0.smithi139.stdout:0/766: write d6/d1c/d41/d44/f25 [1891160,75091] 0 2024-09-16T00:09:51.504 INFO:tasks.workunit.client.0.smithi139.stdout:5/831: creat d1/d13/d3f/d9f/d73/d79/da5/d10b/d11e/d10d/d33/f122 x:0 0 0 2024-09-16T00:09:51.505 INFO:tasks.workunit.client.0.smithi139.stdout:8/904: rename d17/d1a/d57/c5f to d17/d1a/d50/dee/c12e 0 2024-09-16T00:09:51.506 INFO:tasks.workunit.client.0.smithi139.stdout:1/631: truncate d7/d8/d27/d2f/f5c 4719479 0 2024-09-16T00:09:51.506 INFO:tasks.workunit.client.0.smithi139.stdout:6/672: stat d8/d29/d83/d26/d28/d9e/da8/lb7 0 2024-09-16T00:09:51.507 INFO:tasks.workunit.client.0.smithi139.stdout:4/858: mknod d12/d47/d49/d60/dc3/c11a 0 2024-09-16T00:09:51.507 INFO:tasks.workunit.client.0.smithi139.stdout:0/767: fsync d6/d1c/d41/d7c/d86/f91 0 2024-09-16T00:09:51.509 INFO:tasks.workunit.client.0.smithi139.stdout:0/768: fsync d6/d1c/d41/d7c/d86/fd6 0 2024-09-16T00:09:51.509 INFO:tasks.workunit.client.0.smithi139.stdout:8/905: truncate d17/d1f/d3a/db0/fd1 200647 0 2024-09-16T00:09:51.509 INFO:tasks.workunit.client.0.smithi139.stdout:5/832: rename d1/fc5 to d1/d21/f123 0 2024-09-16T00:09:51.510 INFO:tasks.workunit.client.0.smithi139.stdout:2/924: dread d11/d2f/fed [0,4194304] 0 2024-09-16T00:09:51.511 INFO:tasks.workunit.client.0.smithi139.stdout:3/641: sync 2024-09-16T00:09:51.514 INFO:tasks.workunit.client.0.smithi139.stdout:8/906: dwrite d17/f99 [0,4194304] 0 2024-09-16T00:09:51.515 INFO:tasks.workunit.client.0.smithi139.stdout:4/859: creat d12/d47/db2/d4d/db0/f11b x:0 0 0 2024-09-16T00:09:51.516 INFO:tasks.workunit.client.0.smithi139.stdout:8/907: chown d17/d1f/d3a/db7/d40/fc8 4093085 1 2024-09-16T00:09:51.517 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:09:51.518 INFO:tasks.workunit.client.0.smithi139.stdout:0/769: fdatasync d6/d1c/f61 0 2024-09-16T00:09:51.520 INFO:tasks.workunit.client.0.smithi139.stdout:5/833: chown d1/c11d 53 1 2024-09-16T00:09:51.520 INFO:tasks.workunit.client.0.smithi139.stdout:6/673: truncate f3 1332800 0 2024-09-16T00:09:51.522 INFO:tasks.workunit.client.0.smithi139.stdout:8/908: creat d17/d1a/d50/d86/f12f x:0 0 0 2024-09-16T00:09:51.524 INFO:tasks.workunit.client.0.smithi139.stdout:2/925: unlink d11/d30/d110/cd1 0 2024-09-16T00:09:51.525 INFO:tasks.workunit.client.0.smithi139.stdout:3/642: dread d5/d7/d23/d27/d2f/d30/d4c/d6d/db3/fb6 [0,4194304] 0 2024-09-16T00:09:51.526 INFO:tasks.workunit.client.0.smithi139.stdout:0/770: dread d6/d1c/d41/d44/dd/fb0 [0,4194304] 0 2024-09-16T00:09:51.529 INFO:tasks.workunit.client.0.smithi139.stdout:4/860: dread d12/d47/db2/d4d/f81 [0,4194304] 0 2024-09-16T00:09:51.530 INFO:tasks.workunit.client.0.smithi139.stdout:8/909: creat d17/d1f/d3a/db7/f130 x:0 0 0 2024-09-16T00:09:51.530 INFO:tasks.workunit.client.0.smithi139.stdout:1/632: getdents d7/d8/d27/d68/d7c 0 2024-09-16T00:09:51.531 INFO:tasks.workunit.client.0.smithi139.stdout:8/910: write d17/d1f/d3a/db7/d40/d4f/f75 [4191322,10716] 0 2024-09-16T00:09:51.532 INFO:tasks.workunit.client.0.smithi139.stdout:2/926: creat d11/d30/d110/d29/d86/df3/f139 x:0 0 0 2024-09-16T00:09:51.533 INFO:tasks.workunit.client.0.smithi139.stdout:8/911: chown d17/d1a/d57/f8b 10 1 2024-09-16T00:09:51.533 INFO:tasks.workunit.client.0.smithi139.stdout:3/643: rmdir d5/d7/d23/d27/d2f 39 2024-09-16T00:09:51.534 INFO:tasks.workunit.client.0.smithi139.stdout:2/927: read d11/f1e [2462958,101273] 0 2024-09-16T00:09:51.534 INFO:tasks.workunit.client.0.smithi139.stdout:6/674: symlink d8/d20/lec 0 2024-09-16T00:09:51.536 INFO:tasks.workunit.client.0.smithi139.stdout:5/834: write d1/d21/d49/d53/d6a/d7b/f7e [281678,120811] 0 2024-09-16T00:09:51.537 INFO:tasks.workunit.client.0.smithi139.stdout:6/675: write d8/d29/f32 [1756314,24377] 0 2024-09-16T00:09:51.537 INFO:tasks.workunit.client.0.smithi139.stdout:3/644: dread d5/d7/d23/d27/f83 [0,4194304] 0 2024-09-16T00:09:51.538 INFO:tasks.workunit.client.0.smithi139.stdout:2/928: mknod d11/d30/d110/d5d/d73/c13a 0 2024-09-16T00:09:51.540 INFO:tasks.workunit.client.0.smithi139.stdout:6/676: rename d8/d29/d83/d26 to d8/d29/d83/d26/d28/d9e/ded 22 2024-09-16T00:09:51.540 INFO:tasks.workunit.client.0.smithi139.stdout:2/929: write d11/d30/f4c [2395428,96904] 0 2024-09-16T00:09:51.542 INFO:tasks.workunit.client.0.smithi139.stdout:1/633: dread d7/f18 [0,4194304] 0 2024-09-16T00:09:51.543 INFO:tasks.workunit.client.0.smithi139.stdout:3/645: fsync d5/d7/d23/d27/d2f/d30/f42 0 2024-09-16T00:09:51.543 INFO:tasks.workunit.client.0.smithi139.stdout:1/634: stat d7/d8/d27 0 2024-09-16T00:09:51.544 INFO:tasks.workunit.client.0.smithi139.stdout:4/861: creat d12/d47/db2/d4d/de4/f11c x:0 0 0 2024-09-16T00:09:51.545 INFO:tasks.workunit.client.0.smithi139.stdout:0/771: write d6/d1c/d41/d44/dd/f19 [4009496,21915] 0 2024-09-16T00:09:51.546 INFO:tasks.workunit.client.0.smithi139.stdout:6/677: creat d8/d20/d4d/fee x:0 0 0 2024-09-16T00:09:51.547 INFO:tasks.workunit.client.0.smithi139.stdout:4/862: read f4 [4500410,63750] 0 2024-09-16T00:09:51.548 INFO:tasks.workunit.client.0.smithi139.stdout:8/912: getdents d17/d1f/d10e 0 2024-09-16T00:09:51.548 INFO:tasks.workunit.client.0.smithi139.stdout:1/635: truncate d7/d8/d27/d68/d7c/f99 575077 0 2024-09-16T00:09:51.553 INFO:tasks.workunit.client.0.smithi139.stdout:7/813: sync 2024-09-16T00:09:51.557 INFO:tasks.workunit.client.0.smithi139.stdout:3/646: rename d5/l1c to d5/d7/d23/d27/d2f/d30/d41/ldd 0 2024-09-16T00:09:51.557 INFO:tasks.workunit.client.0.smithi139.stdout:5/835: dwrite d1/d21/d49/d87/f11c [0,4194304] 0 2024-09-16T00:09:51.557 INFO:tasks.workunit.client.0.smithi139.stdout:6/678: chown d8/d29/d83/d26/d45/d85/fad 69154 1 2024-09-16T00:09:51.557 INFO:tasks.workunit.client.0.smithi139.stdout:0/772: mkdir d6/d5a/de7/df1 0 2024-09-16T00:09:51.558 INFO:tasks.workunit.client.0.smithi139.stdout:7/814: dread - d5/fa6 zero size 2024-09-16T00:09:51.560 INFO:tasks.workunit.client.0.smithi139.stdout:6/679: write d8/d20/d27/fcf [533774,97938] 0 2024-09-16T00:09:51.561 INFO:tasks.workunit.client.0.smithi139.stdout:1/636: dread d7/d8/d40/d48/d4a/d74/f7e [0,4194304] 0 2024-09-16T00:09:51.565 INFO:tasks.workunit.client.0.smithi139.stdout:2/930: dwrite d11/d2f/fed [0,4194304] 0 2024-09-16T00:09:51.570 INFO:tasks.workunit.client.0.smithi139.stdout:2/931: dwrite d11/d30/d110/f27 [0,4194304] 0 2024-09-16T00:09:51.573 INFO:tasks.workunit.client.0.smithi139.stdout:4/863: creat d12/d2b/d8c/dff/f11d x:0 0 0 2024-09-16T00:09:51.573 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:51.572+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:51.574 INFO:tasks.workunit.client.0.smithi139.stdout:4/864: write d12/d47/d49/f112 [650577,107649] 0 2024-09-16T00:09:51.574 INFO:tasks.workunit.client.0.smithi139.stdout:7/815: symlink d5/d26/d46/d6b/dd6/d119/l124 0 2024-09-16T00:09:51.575 INFO:tasks.workunit.client.0.smithi139.stdout:0/773: dread - d6/d1c/d41/d76/f95 zero size 2024-09-16T00:09:51.577 INFO:tasks.workunit.client.0.smithi139.stdout:6/680: creat d8/d29/d83/d26/d45/d85/fef x:0 0 0 2024-09-16T00:09:51.578 INFO:tasks.workunit.client.0.smithi139.stdout:2/932: unlink d11/d30/dbf/le4 0 2024-09-16T00:09:51.579 INFO:tasks.workunit.client.0.smithi139.stdout:8/913: symlink d17/d1f/d3a/db7/d40/l131 0 2024-09-16T00:09:51.581 INFO:tasks.workunit.client.0.smithi139.stdout:1/637: mknod d7/d8/d40/d46/d89/cb7 0 2024-09-16T00:09:51.586 INFO:tasks.workunit.client.0.smithi139.stdout:0/774: rename d6/d1c/d41/d7c/dad to d6/d1c/d41/d78/df2 0 2024-09-16T00:09:51.587 INFO:tasks.workunit.client.0.smithi139.stdout:6/681: dwrite d8/d29/d83/d26/d28/f40 [4194304,4194304] 0 2024-09-16T00:09:51.588 INFO:tasks.workunit.client.0.smithi139.stdout:2/933: mknod d11/d2f/c13b 0 2024-09-16T00:09:51.588 INFO:tasks.workunit.client.0.smithi139.stdout:5/836: creat d1/d13/d22/f124 x:0 0 0 2024-09-16T00:09:51.589 INFO:tasks.workunit.client.0.smithi139.stdout:6/682: chown d8/da4 8654 1 2024-09-16T00:09:51.590 INFO:tasks.workunit.client.0.smithi139.stdout:8/914: read - d17/d1f/d3a/db7/d40/ff0 zero size 2024-09-16T00:09:51.590 INFO:tasks.workunit.client.0.smithi139.stdout:7/816: dread d5/d26/d42/f8f [0,4194304] 0 2024-09-16T00:09:51.591 INFO:tasks.workunit.client.0.smithi139.stdout:4/865: rename d12/d47/db2/d4d/d69/ff8 to d12/d47/db2/d4d/dd6/f11e 0 2024-09-16T00:09:51.593 INFO:tasks.workunit.client.0.smithi139.stdout:0/775: creat d6/d1c/d41/ff3 x:0 0 0 2024-09-16T00:09:51.593 INFO:tasks.workunit.client.0.smithi139.stdout:1/638: dwrite d7/d8/d27/d68/fa9 [0,4194304] 0 2024-09-16T00:09:51.595 INFO:tasks.workunit.client.0.smithi139.stdout:2/934: creat d11/d30/dbf/d116/d125/f13c x:0 0 0 2024-09-16T00:09:51.597 INFO:tasks.workunit.client.0.smithi139.stdout:6/683: creat d8/d20/d4d/ff0 x:0 0 0 2024-09-16T00:09:51.597 INFO:tasks.workunit.client.0.smithi139.stdout:5/837: write d1/d13/d3f/fb8 [4152517,109980] 0 2024-09-16T00:09:51.600 INFO:tasks.workunit.client.0.smithi139.stdout:3/647: chown d5/d7/d23/d27/d2f/d30/d47/c9f 1 1 2024-09-16T00:09:51.602 INFO:tasks.workunit.client.0.smithi139.stdout:1/639: mkdir d7/d8/d40/d48/d4a/db8 0 2024-09-16T00:09:51.605 INFO:tasks.workunit.client.0.smithi139.stdout:6/684: truncate d8/d56/f73 482732 0 2024-09-16T00:09:51.607 INFO:tasks.workunit.client.0.smithi139.stdout:5/838: mknod d1/d13/d3f/d9f/d73/d79/da5/d10b/d11e/c125 0 2024-09-16T00:09:51.607 INFO:tasks.workunit.client.0.smithi139.stdout:6/685: truncate d8/d20/d4d/fee 882288 0 2024-09-16T00:09:51.608 INFO:tasks.workunit.client.0.smithi139.stdout:6/686: dread - d8/d29/d83/d26/d28/fce zero size 2024-09-16T00:09:51.608 INFO:tasks.workunit.client.0.smithi139.stdout:7/817: fsync d5/d16/d22/d25/f76 0 2024-09-16T00:09:51.611 INFO:tasks.workunit.client.0.smithi139.stdout:2/935: rename d11/d45/c135 to d11/d45/dde/c13d 0 2024-09-16T00:09:51.612 INFO:tasks.workunit.client.0.smithi139.stdout:8/915: creat d17/d54/f132 x:0 0 0 2024-09-16T00:09:51.614 INFO:tasks.workunit.client.0.smithi139.stdout:4/866: dread f2 [0,4194304] 0 2024-09-16T00:09:51.615 INFO:tasks.workunit.client.0.smithi139.stdout:6/687: readlink d8/d29/d83/d26/d28/d5a/ld8 0 2024-09-16T00:09:51.618 INFO:tasks.workunit.client.0.smithi139.stdout:2/936: dread d11/d2f/fed [0,4194304] 0 2024-09-16T00:09:51.618 INFO:tasks.workunit.client.0.smithi139.stdout:5/839: mkdir d1/d13/d126 0 2024-09-16T00:09:51.619 INFO:tasks.workunit.client.0.smithi139.stdout:1/640: dread d7/f26 [0,4194304] 0 2024-09-16T00:09:51.622 INFO:tasks.workunit.client.0.smithi139.stdout:0/776: getdents d6/d1c/d41/d7c/d86 0 2024-09-16T00:09:51.624 INFO:tasks.workunit.client.0.smithi139.stdout:0/777: read d6/d1c/d41/d44/f3e [3964177,13865] 0 2024-09-16T00:09:51.624 INFO:tasks.workunit.client.0.smithi139.stdout:3/648: link d5/d7/d23/d27/d2f/d30/d4c/d50/fcd d5/d7/d23/d35/fde 0 2024-09-16T00:09:51.625 INFO:tasks.workunit.client.0.smithi139.stdout:8/916: mknod d17/d1f/d3a/de2/c133 0 2024-09-16T00:09:51.626 INFO:tasks.workunit.client.0.smithi139.stdout:0/778: write d6/d1c/d41/d44/dd/d56/d75/fef [418495,85427] 0 2024-09-16T00:09:51.626 INFO:tasks.workunit.client.0.smithi139.stdout:5/840: fdatasync d1/d21/d49/d53/d6a/f9b 0 2024-09-16T00:09:51.628 INFO:tasks.workunit.client.0.smithi139.stdout:2/937: unlink d11/d30/d110/d5d/d73/dc3/f117 0 2024-09-16T00:09:51.628 INFO:tasks.workunit.client.0.smithi139.stdout:5/841: dread - d1/d21/d49/d53/d6a/d7b/da1/db7/ff1 zero size 2024-09-16T00:09:51.630 INFO:tasks.workunit.client.0.smithi139.stdout:4/867: dread d12/d47/db2/d4d/de4/f59 [0,4194304] 0 2024-09-16T00:09:51.630 INFO:tasks.workunit.client.0.smithi139.stdout:6/688: mknod d8/d29/cf1 0 2024-09-16T00:09:51.633 INFO:tasks.workunit.client.0.smithi139.stdout:8/917: unlink d17/d1f/d3a/db7/d40/d4f/l5c 0 2024-09-16T00:09:51.635 INFO:tasks.workunit.client.0.smithi139.stdout:2/938: symlink d11/d30/d110/d5d/l13e 0 2024-09-16T00:09:51.636 INFO:tasks.workunit.client.0.smithi139.stdout:5/842: stat d1/d13/d22/c34 0 2024-09-16T00:09:51.637 INFO:tasks.workunit.client.0.smithi139.stdout:3/649: dwrite d5/d7/d23/d27/d2f/d30/d41/fb5 [0,4194304] 0 2024-09-16T00:09:51.637 INFO:tasks.workunit.client.0.smithi139.stdout:2/939: read d11/f15 [8125244,18211] 0 2024-09-16T00:09:51.639 INFO:tasks.workunit.client.0.smithi139.stdout:4/868: creat d12/d47/db2/d4d/d69/f11f x:0 0 0 2024-09-16T00:09:51.640 INFO:tasks.workunit.client.0.smithi139.stdout:0/779: symlink d6/d1c/d41/d44/dd/d8a/lf4 0 2024-09-16T00:09:51.641 INFO:tasks.workunit.client.0.smithi139.stdout:3/650: symlink d5/d7/d23/d35/da7/ldf 0 2024-09-16T00:09:51.642 INFO:tasks.workunit.client.0.smithi139.stdout:5/843: truncate d1/d21/d49/d53/d6a/f8b 824231 0 2024-09-16T00:09:51.644 INFO:tasks.workunit.client.0.smithi139.stdout:4/869: mknod d12/d47/db2/d4d/da9/c120 0 2024-09-16T00:09:51.651 INFO:tasks.workunit.client.0.smithi139.stdout:2/940: truncate d11/d30/d110/d29/da5/d81/f89 741290 0 2024-09-16T00:09:51.651 INFO:tasks.workunit.client.0.smithi139.stdout:7/818: dwrite d5/d16/d6d/d7f/dc2/d50/d44/f84 [0,4194304] 0 2024-09-16T00:09:51.654 INFO:tasks.workunit.client.0.smithi139.stdout:8/918: rename d17/d1a/d57/fde to d17/d1a/d50/d86/d9d/d9f/d101/f134 0 2024-09-16T00:09:51.655 INFO:tasks.workunit.client.0.smithi139.stdout:6/689: dread d8/f6f [0,4194304] 0 2024-09-16T00:09:51.657 INFO:tasks.workunit.client.0.smithi139.stdout:3/651: dwrite d5/d7/f49 [0,4194304] 0 2024-09-16T00:09:51.657 INFO:tasks.workunit.client.0.smithi139.stdout:4/870: symlink d12/d2b/l121 0 2024-09-16T00:09:51.659 INFO:tasks.workunit.client.0.smithi139.stdout:5/844: creat d1/d13/d3f/d9f/d73/dd0/f127 x:0 0 0 2024-09-16T00:09:51.660 INFO:tasks.workunit.client.0.smithi139.stdout:1/641: dwrite d7/d8/d40/d48/d53/f85 [4194304,4194304] 0 2024-09-16T00:09:51.660 INFO:tasks.workunit.client.0.smithi139.stdout:7/819: truncate d5/d16/d6d/fd0 1216978 0 2024-09-16T00:09:51.660 INFO:tasks.workunit.client.0.smithi139.stdout:3/652: readlink d5/d7/d23/l22 0 2024-09-16T00:09:51.664 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:51.664 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:51.664 INFO:tasks.workunit.client.0.smithi139.stdout:0/780: dread d6/d1c/d41/d44/dd/f39 [0,4194304] 0 2024-09-16T00:09:51.668 INFO:tasks.workunit.client.0.smithi139.stdout:1/642: stat d7/d8/d40/l62 0 2024-09-16T00:09:51.672 INFO:tasks.workunit.client.0.smithi139.stdout:6/690: write d8/f39 [703707,21752] 0 2024-09-16T00:09:51.674 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:51.674+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:51.675 INFO:tasks.workunit.client.0.smithi139.stdout:1/643: fsync d7/d8/d27/d2f/f73 0 2024-09-16T00:09:51.676 INFO:tasks.workunit.client.0.smithi139.stdout:5/845: write d1/d13/d3f/d5e/dc2/fce [1497055,120161] 0 2024-09-16T00:09:51.676 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:09:51.677 INFO:tasks.workunit.client.0.smithi139.stdout:3/653: truncate d5/d7/f28 236869 0 2024-09-16T00:09:51.679 INFO:tasks.workunit.client.0.smithi139.stdout:6/691: readlink d8/d29/d83/d26/d45/d65/d6e/lc2 0 2024-09-16T00:09:51.679 INFO:tasks.workunit.client.0.smithi139.stdout:1/644: chown d7/d8/d40/d48/d4a/db8 417826 1 2024-09-16T00:09:51.681 INFO:tasks.workunit.client.0.smithi139.stdout:2/941: dwrite ff [0,4194304] 0 2024-09-16T00:09:51.682 INFO:tasks.workunit.client.0.smithi139.stdout:5/846: dread - d1/d21/d49/d53/d6a/d7b/d78/dca/ff9 zero size 2024-09-16T00:09:51.682 INFO:tasks.workunit.client.0.smithi139.stdout:4/871: truncate d12/d47/d49/d60/dac/fdb 1996565 0 2024-09-16T00:09:51.684 INFO:tasks.workunit.client.0.smithi139.stdout:8/919: dwrite d17/d1f/d3a/dad/fb5 [0,4194304] 0 2024-09-16T00:09:51.686 INFO:tasks.workunit.client.0.smithi139.stdout:4/872: chown d12/d2b/d8c/fd7 1626157 1 2024-09-16T00:09:51.686 INFO:tasks.workunit.client.0.smithi139.stdout:8/920: dread - d17/d1a/d50/d86/d9d/d9f/fd7 zero size 2024-09-16T00:09:51.689 INFO:tasks.workunit.client.0.smithi139.stdout:1/645: symlink d7/d8/d27/d2f/lb9 0 2024-09-16T00:09:51.694 INFO:tasks.workunit.client.0.smithi139.stdout:0/781: dread d6/d1c/d41/d7c/fe4 [0,4194304] 0 2024-09-16T00:09:51.698 INFO:tasks.workunit.client.0.smithi139.stdout:1/646: fdatasync d7/d8/d40/d46/d8d/f95 0 2024-09-16T00:09:51.699 INFO:tasks.workunit.client.0.smithi139.stdout:3/654: rename d5/d7/d23/d35/da7/d9a/fd9 to d5/d7/d23/fe0 0 2024-09-16T00:09:51.699 INFO:tasks.workunit.client.0.smithi139.stdout:0/782: chown d6/d1c/d41/d44/f33 1607 1 2024-09-16T00:09:51.699 INFO:tasks.workunit.client.0.smithi139.stdout:8/921: getdents d17/d89 0 2024-09-16T00:09:51.701 INFO:tasks.workunit.client.0.smithi139.stdout:2/942: truncate d11/d30/d110/d29/d4f/d9c/fca 1645097 0 2024-09-16T00:09:51.701 INFO:tasks.workunit.client.0.smithi139.stdout:1/647: readlink d7/d8/l45 0 2024-09-16T00:09:51.701 INFO:tasks.workunit.client.0.smithi139.stdout:8/922: rmdir d17 39 2024-09-16T00:09:51.702 INFO:tasks.workunit.client.0.smithi139.stdout:4/873: dwrite d12/d2b/f3a [4194304,4194304] 0 2024-09-16T00:09:51.705 INFO:tasks.workunit.client.0.smithi139.stdout:0/783: fsync d6/d1c/d41/d44/d58/fd5 0 2024-09-16T00:09:51.707 INFO:tasks.workunit.client.0.smithi139.stdout:3/655: truncate d5/d7/d23/d35/f3b 393152 0 2024-09-16T00:09:51.708 INFO:tasks.workunit.client.0.smithi139.stdout:4/874: readlink d12/d2b/l3d 0 2024-09-16T00:09:51.708 INFO:tasks.workunit.client.0.smithi139.stdout:6/692: dread d8/d20/d4d/f8e [0,4194304] 0 2024-09-16T00:09:51.710 INFO:tasks.workunit.client.0.smithi139.stdout:0/784: mknod d6/d5a/cf5 0 2024-09-16T00:09:51.710 INFO:tasks.workunit.client.0.smithi139.stdout:5/847: dread d1/f28 [0,4194304] 0 2024-09-16T00:09:51.711 INFO:tasks.workunit.client.0.smithi139.stdout:4/875: mknod d12/d47/db2/d4d/de4/d7f/c122 0 2024-09-16T00:09:51.712 INFO:tasks.workunit.client.0.smithi139.stdout:8/923: write f7 [4802130,31449] 0 2024-09-16T00:09:51.713 INFO:tasks.workunit.client.0.smithi139.stdout:1/648: getdents d7/d8/d40/d47 0 2024-09-16T00:09:51.714 INFO:tasks.workunit.client.0.smithi139.stdout:0/785: write d6/d1c/d41/d44/dd/d56/fc4 [451233,79766] 0 2024-09-16T00:09:51.714 INFO:tasks.workunit.client.0.smithi139.stdout:6/693: rename d8/d20/d4d/fbf to d8/d29/d83/d26/d45/d85/ff2 0 2024-09-16T00:09:51.715 INFO:tasks.workunit.client.0.smithi139.stdout:1/649: write d7/d8/d27/d68/fa9 [3964441,107414] 0 2024-09-16T00:09:51.718 INFO:tasks.workunit.client.0.smithi139.stdout:5/848: rmdir d1/d13/d3f/d9f/d73/d79/da5/d10b/d11e/d10d/dd6 39 2024-09-16T00:09:51.722 INFO:tasks.workunit.client.0.smithi139.stdout:0/786: unlink d6/d1c/d41/d44/dd/d56/l94 0 2024-09-16T00:09:51.723 INFO:tasks.workunit.client.0.smithi139.stdout:0/787: write d6/d1c/d41/d44/dd/d8a/d9e/fc3 [3412796,58302] 0 2024-09-16T00:09:51.725 INFO:tasks.workunit.client.0.smithi139.stdout:6/694: mkdir d8/d29/d83/d26/d45/d42/df3 0 2024-09-16T00:09:51.728 INFO:tasks.workunit.client.0.smithi139.stdout:1/650: creat d7/d8/d27/d2f/fba x:0 0 0 2024-09-16T00:09:51.729 INFO:tasks.workunit.client.0.smithi139.stdout:0/788: dread d6/d1c/d41/d44/dd/d8a/f92 [0,4194304] 0 2024-09-16T00:09:51.729 INFO:tasks.workunit.client.0.smithi139.stdout:5/849: mkdir d1/d13/d22/dd5/d128 0 2024-09-16T00:09:51.729 INFO:tasks.workunit.client.0.smithi139.stdout:1/651: fdatasync d7/d8/d40/d48/d53/f85 0 2024-09-16T00:09:51.731 INFO:tasks.workunit.client.0.smithi139.stdout:6/695: mkdir d8/d20/d68/df4 0 2024-09-16T00:09:51.735 INFO:tasks.workunit.client.0.smithi139.stdout:5/850: dwrite d1/d13/d3f/d9f/d73/d79/da5/d10b/d11e/d10d/fea [0,4194304] 0 2024-09-16T00:09:51.736 INFO:tasks.workunit.client.0.smithi139.stdout:8/924: creat d17/d1a/d50/d86/d9d/d9f/f135 x:0 0 0 2024-09-16T00:09:51.739 INFO:tasks.workunit.client.0.smithi139.stdout:4/876: getdents d12/d47/db2/d4d/de4/d7f 0 2024-09-16T00:09:51.739 INFO:tasks.workunit.client.0.smithi139.stdout:5/851: truncate d1/d21/d49/d53/d6a/d7b/da1/db7/ff1 223045 0 2024-09-16T00:09:51.740 INFO:tasks.workunit.client.0.smithi139.stdout:1/652: symlink d7/d8/lbb 0 2024-09-16T00:09:51.740 INFO:tasks.workunit.client.0.smithi139.stdout:0/789: symlink d6/d1c/d41/d44/dd/d56/dbd/lf6 0 2024-09-16T00:09:51.740 INFO:tasks.workunit.client.0.smithi139.stdout:4/877: readlink d12/d47/db2/d4d/d69/lfe 0 2024-09-16T00:09:51.747 INFO:tasks.workunit.client.0.smithi139.stdout:0/790: dread d6/d1c/d41/d44/dd/f2c [0,4194304] 0 2024-09-16T00:09:51.747 INFO:tasks.workunit.client.0.smithi139.stdout:1/653: read d7/d1b/f3a [1141131,4854] 0 2024-09-16T00:09:51.747 INFO:tasks.workunit.client.0.smithi139.stdout:5/852: symlink d1/d21/d49/d53/d6a/d7b/l129 0 2024-09-16T00:09:51.748 INFO:tasks.workunit.client.0.smithi139.stdout:8/925: creat d17/d1a/d50/d86/d9d/d9f/f136 x:0 0 0 2024-09-16T00:09:51.749 INFO:tasks.workunit.client.0.smithi139.stdout:4/878: dread d12/d2b/f34 [0,4194304] 0 2024-09-16T00:09:51.750 INFO:tasks.workunit.client.0.smithi139.stdout:0/791: dread - d6/d1c/d41/d44/dd/d56/d75/fa6 zero size 2024-09-16T00:09:51.752 INFO:tasks.workunit.client.0.smithi139.stdout:8/926: creat d17/d1a/d50/dee/f137 x:0 0 0 2024-09-16T00:09:51.753 INFO:tasks.workunit.client.0.smithi139.stdout:1/654: rename d7/d8/d40/d48/d4a/d74/f7f to d7/d8/d40/d48/d53/fbc 0 2024-09-16T00:09:51.753 INFO:tasks.workunit.client.0.smithi139.stdout:0/792: rmdir d6 39 2024-09-16T00:09:51.753 INFO:tasks.workunit.client.0.smithi139.stdout:4/879: symlink d12/d2b/d8c/l123 0 2024-09-16T00:09:51.754 INFO:tasks.workunit.client.0.smithi139.stdout:1/655: chown d7/d8/d40/d48/d4a/d74/f9f 3503070 1 2024-09-16T00:09:51.755 INFO:tasks.workunit.client.0.smithi139.stdout:4/880: dread - d12/d47/db2/d4d/d69/f11f zero size 2024-09-16T00:09:51.756 INFO:tasks.workunit.client.0.smithi139.stdout:8/927: mknod d17/d1a/d50/d86/d9d/daf/c138 0 2024-09-16T00:09:51.757 INFO:tasks.workunit.client.0.smithi139.stdout:1/656: symlink d7/d8/d40/d47/lbd 0 2024-09-16T00:09:51.757 INFO:tasks.workunit.client.0.smithi139.stdout:8/928: readlink d17/d1f/d3a/db7/d40/l131 0 2024-09-16T00:09:51.758 INFO:tasks.workunit.client.0.smithi139.stdout:4/881: creat d12/d80/f124 x:0 0 0 2024-09-16T00:09:51.761 INFO:tasks.workunit.client.0.smithi139.stdout:4/882: creat d12/d80/f125 x:0 0 0 2024-09-16T00:09:51.762 INFO:tasks.workunit.client.0.smithi139.stdout:5/853: dread d1/d13/d3b/d3e/f6e [0,4194304] 0 2024-09-16T00:09:51.762 INFO:tasks.workunit.client.0.smithi139.stdout:8/929: stat d17/d1a/d2d/l74 0 2024-09-16T00:09:51.766 INFO:tasks.workunit.client.0.smithi139.stdout:0/793: dwrite d6/d1c/d41/d44/dce/fd8 [0,4194304] 0 2024-09-16T00:09:51.768 INFO:tasks.workunit.client.0.smithi139.stdout:1/657: dwrite d7/d8/d40/d47/f5e [0,4194304] 0 2024-09-16T00:09:51.768 INFO:tasks.workunit.client.0.smithi139.stdout:5/854: rename d1/d21/c66 to d1/d13/d22/da9/c12a 0 2024-09-16T00:09:51.768 INFO:tasks.workunit.client.0.smithi139.stdout:4/883: mkdir d12/d47/d49/d60/dc3/de2/d126 0 2024-09-16T00:09:51.771 INFO:tasks.workunit.client.0.smithi139.stdout:0/794: truncate d6/d1c/f61 3735803 0 2024-09-16T00:09:51.773 INFO:tasks.workunit.client.0.smithi139.stdout:5/855: unlink d1/d13/d3f/d88/lbe 0 2024-09-16T00:09:51.775 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:51.774+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:51.775 INFO:tasks.workunit.client.0.smithi139.stdout:4/884: symlink d12/d47/d49/d60/dc3/de2/l127 0 2024-09-16T00:09:51.778 INFO:tasks.workunit.client.0.smithi139.stdout:8/930: dwrite d17/d1a/d50/d86/d9d/d9f/d101/f134 [0,4194304] 0 2024-09-16T00:09:51.778 INFO:tasks.workunit.client.0.smithi139.stdout:5/856: symlink d1/d21/d49/d53/d6a/d7b/d78/l12b 0 2024-09-16T00:09:51.780 INFO:tasks.workunit.client.0.smithi139.stdout:0/795: rename d6/d5a/deb to d6/d1c/df7 0 2024-09-16T00:09:51.781 INFO:tasks.workunit.client.0.smithi139.stdout:8/931: chown d17/d1a/d50/d86/d9d/d9f/d101/l11f 68 1 2024-09-16T00:09:51.782 INFO:tasks.workunit.client.0.smithi139.stdout:4/885: chown d12/d47/d49/d60/dac/l10f 975 1 2024-09-16T00:09:51.783 INFO:tasks.workunit.client.0.smithi139.stdout:5/857: rename d1/d21/d49/d53/d6a/d7b/da1/db7/ff1 to d1/d13/d94/f12c 0 2024-09-16T00:09:51.785 INFO:tasks.workunit.client.0.smithi139.stdout:4/886: mkdir d12/d47/d128 0 2024-09-16T00:09:51.786 INFO:tasks.workunit.client.0.smithi139.stdout:8/932: mkdir d17/d1a/d50/d86/d9d/daf/d115/d139 0 2024-09-16T00:09:51.786 INFO:tasks.workunit.client.0.smithi139.stdout:1/658: dwrite d7/d8/d27/f2c [0,4194304] 0 2024-09-16T00:09:51.787 INFO:tasks.workunit.client.0.smithi139.stdout:1/659: readlink d7/d8/l9a 0 2024-09-16T00:09:51.788 INFO:tasks.workunit.client.0.smithi139.stdout:5/858: creat d1/d21/d49/d53/d6a/d7b/f12d x:0 0 0 2024-09-16T00:09:51.791 INFO:tasks.workunit.client.0.smithi139.stdout:1/660: chown d7/f28 0 1 2024-09-16T00:09:51.792 INFO:tasks.workunit.client.0.smithi139.stdout:5/859: rmdir d1/d21/d49/d53/d6a/d7b/da1 39 2024-09-16T00:09:51.794 INFO:tasks.workunit.client.0.smithi139.stdout:8/933: creat d17/f13a x:0 0 0 2024-09-16T00:09:51.795 INFO:tasks.workunit.client.0.smithi139.stdout:1/661: rmdir d7/d8/d40/d46/d8d 39 2024-09-16T00:09:51.795 INFO:tasks.workunit.client.0.smithi139.stdout:4/887: dwrite d12/d47/db2/fc2 [0,4194304] 0 2024-09-16T00:09:51.796 INFO:tasks.workunit.client.0.smithi139.stdout:5/860: creat d1/dc3/f12e x:0 0 0 2024-09-16T00:09:51.797 INFO:tasks.workunit.client.0.smithi139.stdout:8/934: write d17/d1f/d3a/db0/f103 [79599,103717] 0 2024-09-16T00:09:51.798 INFO:tasks.workunit.client.0.smithi139.stdout:1/662: creat d7/d8/d27/d68/d6c/fbe x:0 0 0 2024-09-16T00:09:51.799 INFO:tasks.workunit.client.0.smithi139.stdout:1/663: chown d7/d8/d27/f59 3103 1 2024-09-16T00:09:51.801 INFO:tasks.workunit.client.0.smithi139.stdout:4/888: unlink d12/d47/db2/d4d/f81 0 2024-09-16T00:09:51.804 INFO:tasks.workunit.client.0.smithi139.stdout:8/935: rename d17/d1f/f28 to d17/d89/df2/f13b 0 2024-09-16T00:09:51.805 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:51.805 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:51.806 INFO:tasks.workunit.client.0.smithi139.stdout:1/664: mkdir d7/d8/d40/d48/d67/dbf 0 2024-09-16T00:09:51.808 INFO:tasks.workunit.client.0.smithi139.stdout:5/861: rename d1/d21/d49/db2/f103 to d1/d13/d3f/d9f/d73/d79/d8d/f12f 0 2024-09-16T00:09:51.810 INFO:tasks.workunit.client.0.smithi139.stdout:5/862: readlink d1/d13/d3f/d9f/d73/d79/da5/d10b/d11e/d10d/l40 0 2024-09-16T00:09:51.811 INFO:tasks.workunit.client.0.smithi139.stdout:1/665: dwrite d7/d8/d1f/f76 [0,4194304] 0 2024-09-16T00:09:51.813 INFO:tasks.workunit.client.0.smithi139.stdout:8/936: mknod d17/d1a/d50/dee/d12b/c13c 0 2024-09-16T00:09:51.815 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:09:51.816 INFO:tasks.workunit.client.0.smithi139.stdout:1/666: creat d7/d8/d27/d2f/fc0 x:0 0 0 2024-09-16T00:09:51.816 INFO:tasks.workunit.client.0.smithi139.stdout:5/863: mknod d1/d21/d49/d53/d6a/d7b/d99/d11b/c130 0 2024-09-16T00:09:51.818 INFO:tasks.workunit.client.0.smithi139.stdout:1/667: fdatasync d7/d8/d27/d68/fb4 0 2024-09-16T00:09:51.819 INFO:tasks.workunit.client.0.smithi139.stdout:4/889: dwrite d12/d47/db2/d4d/f9d [0,4194304] 0 2024-09-16T00:09:51.824 INFO:tasks.workunit.client.0.smithi139.stdout:1/668: creat d7/d8/d1f/fc1 x:0 0 0 2024-09-16T00:09:51.824 INFO:tasks.workunit.client.0.smithi139.stdout:5/864: symlink d1/d21/d49/d53/d6a/d7b/l131 0 2024-09-16T00:09:51.824 INFO:tasks.workunit.client.0.smithi139.stdout:8/937: dwrite d17/d1f/d3a/db7/d40/fb4 [0,4194304] 0 2024-09-16T00:09:51.826 INFO:tasks.workunit.client.0.smithi139.stdout:1/669: truncate d7/d8/d1f/fa1 1145078 0 2024-09-16T00:09:51.826 INFO:tasks.workunit.client.0.smithi139.stdout:5/865: mkdir d1/d13/d3f/d9f/d132 0 2024-09-16T00:09:51.831 INFO:tasks.workunit.client.0.smithi139.stdout:4/890: link d12/d47/ca7 d12/d47/d49/d60/dac/c129 0 2024-09-16T00:09:51.832 INFO:tasks.workunit.client.0.smithi139.stdout:5/866: dwrite d1/d21/d49/d53/d6a/d7b/d99/fac [4194304,4194304] 0 2024-09-16T00:09:51.833 INFO:tasks.workunit.client.0.smithi139.stdout:8/938: link d17/d1f/d3a/db7/d40/cae d17/d89/c13d 0 2024-09-16T00:09:51.835 INFO:tasks.workunit.client.0.smithi139.stdout:8/939: truncate d17/d1f/d3a/db7/d40/f104 896475 0 2024-09-16T00:09:51.836 INFO:tasks.workunit.client.0.smithi139.stdout:5/867: getdents d1/d21 0 2024-09-16T00:09:51.836 INFO:tasks.workunit.client.0.smithi139.stdout:8/940: fsync d17/d1f/d3a/db7/f2b 0 2024-09-16T00:09:51.839 INFO:tasks.workunit.client.0.smithi139.stdout:5/868: mknod d1/d13/d3f/d88/c133 0 2024-09-16T00:09:51.842 INFO:tasks.workunit.client.0.smithi139.stdout:5/869: stat d1/d13/d3f/d9f/d73/d79/da5/d10b 0 2024-09-16T00:09:51.842 INFO:tasks.workunit.client.0.smithi139.stdout:1/670: dwrite d7/d8/f1a [0,4194304] 0 2024-09-16T00:09:51.844 INFO:tasks.workunit.client.0.smithi139.stdout:5/870: chown d1/d13/d3f/d88/l71 488 1 2024-09-16T00:09:51.845 INFO:tasks.workunit.client.0.smithi139.stdout:1/671: creat d7/d8/d27/d68/d7c/fc2 x:0 0 0 2024-09-16T00:09:51.845 INFO:tasks.workunit.client.0.smithi139.stdout:5/871: readlink d1/d13/l75 0 2024-09-16T00:09:51.846 INFO:tasks.workunit.client.0.smithi139.stdout:1/672: dread - d7/d8/d1f/faf zero size 2024-09-16T00:09:51.846 INFO:tasks.workunit.client.0.smithi139.stdout:5/872: chown d1/d21/d49/d53/d6a 13310289 1 2024-09-16T00:09:51.848 INFO:tasks.workunit.client.0.smithi139.stdout:4/891: dread d12/d2b/f94 [0,4194304] 0 2024-09-16T00:09:51.849 INFO:tasks.workunit.client.0.smithi139.stdout:5/873: mknod d1/d13/d3f/d9f/d73/c134 0 2024-09-16T00:09:51.851 INFO:tasks.workunit.client.0.smithi139.stdout:1/673: getdents d7/d8/d40/d46/d89 0 2024-09-16T00:09:51.853 INFO:tasks.workunit.client.0.smithi139.stdout:5/874: creat d1/d13/d3f/d9f/d132/f135 x:0 0 0 2024-09-16T00:09:51.854 INFO:tasks.workunit.client.0.smithi139.stdout:1/674: mkdir d7/d8/d40/d48/d67/d7d/dc3 0 2024-09-16T00:09:51.855 INFO:tasks.workunit.client.0.smithi139.stdout:4/892: link d12/d47/d49/d60/dc3/fec d12/d47/d49/f12a 0 2024-09-16T00:09:51.856 INFO:tasks.workunit.client.0.smithi139.stdout:8/941: dread d17/d1a/d50/d86/d92/f110 [0,4194304] 0 2024-09-16T00:09:51.857 INFO:tasks.workunit.client.0.smithi139.stdout:1/675: fsync d7/d8/d40/d48/d53/d61/f87 0 2024-09-16T00:09:51.857 INFO:tasks.workunit.client.0.smithi139.stdout:5/875: symlink d1/d13/d3f/d9f/d73/d79/da5/d10b/d11e/d10d/d33/l136 0 2024-09-16T00:09:51.859 INFO:tasks.workunit.client.0.smithi139.stdout:1/676: readlink d7/l60 0 2024-09-16T00:09:51.859 INFO:tasks.workunit.client.0.smithi139.stdout:8/942: mkdir d17/d1f/d13e 0 2024-09-16T00:09:51.860 INFO:tasks.workunit.client.0.smithi139.stdout:1/677: readlink d7/d8/l17 0 2024-09-16T00:09:51.863 INFO:tasks.workunit.client.0.smithi139.stdout:4/893: dwrite d12/d80/f9a [4194304,4194304] 0 2024-09-16T00:09:51.863 INFO:tasks.workunit.client.0.smithi139.stdout:1/678: read d7/d8/f1e [1371231,129234] 0 2024-09-16T00:09:51.864 INFO:tasks.workunit.client.0.smithi139.stdout:8/943: creat d17/d1f/d3a/d10a/f13f x:0 0 0 2024-09-16T00:09:51.866 INFO:tasks.workunit.client.0.smithi139.stdout:8/944: write d17/d1a/d57/f71 [2408246,116241] 0 2024-09-16T00:09:51.869 INFO:tasks.workunit.client.0.smithi139.stdout:1/679: dread d7/f18 [0,4194304] 0 2024-09-16T00:09:51.870 INFO:tasks.workunit.client.0.smithi139.stdout:8/945: symlink d17/d1a/d50/d86/d9d/daf/d115/l140 0 2024-09-16T00:09:51.871 INFO:tasks.workunit.client.0.smithi139.stdout:1/680: write d7/d8/d1f/faf [261782,8504] 0 2024-09-16T00:09:51.872 INFO:tasks.workunit.client.0.smithi139.stdout:8/946: creat d17/d1a/d50/dee/f141 x:0 0 0 2024-09-16T00:09:51.873 INFO:tasks.workunit.client.0.smithi139.stdout:1/681: creat d7/d1b/fc4 x:0 0 0 2024-09-16T00:09:51.874 INFO:tasks.workunit.client.0.smithi139.stdout:1/682: chown d7/d8/d40/d47/f50 463535 1 2024-09-16T00:09:51.874 INFO:tasks.workunit.client.0.smithi139.stdout:8/947: chown d17/l52 75 1 2024-09-16T00:09:51.875 INFO:tasks.workunit.client.0.smithi139.stdout:5/876: truncate d1/d13/d3f/d5e/dc2/fce 1531430 0 2024-09-16T00:09:51.875 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:51.875+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:51.876 INFO:tasks.workunit.client.0.smithi139.stdout:1/683: creat d7/d8/d40/d47/dae/fc5 x:0 0 0 2024-09-16T00:09:51.877 INFO:tasks.workunit.client.0.smithi139.stdout:8/948: mknod d17/d1f/d3a/db7/d40/c142 0 2024-09-16T00:09:51.878 INFO:tasks.workunit.client.0.smithi139.stdout:8/949: chown d17/d1f/d3a/db7/d40/d4f/f114 0 1 2024-09-16T00:09:51.881 INFO:tasks.workunit.client.0.smithi139.stdout:8/950: creat d17/d1a/d50/d86/d9d/daf/d115/d11d/f143 x:0 0 0 2024-09-16T00:09:51.885 INFO:tasks.workunit.client.0.smithi139.stdout:5/877: truncate d1/d13/d3f/fb8 2097051 0 2024-09-16T00:09:51.886 INFO:tasks.workunit.client.0.smithi139.stdout:1/684: dread d7/d8/d1f/f38 [0,4194304] 0 2024-09-16T00:09:51.891 INFO:tasks.workunit.client.0.smithi139.stdout:8/951: dread d17/d1f/d3a/db7/f42 [0,4194304] 0 2024-09-16T00:09:51.892 INFO:tasks.workunit.client.0.smithi139.stdout:8/952: fsync d17/d1f/d3a/d10a/f13f 0 2024-09-16T00:09:51.893 INFO:tasks.workunit.client.0.smithi139.stdout:8/953: dread - d17/d1a/d50/fec zero size 2024-09-16T00:09:51.901 INFO:tasks.workunit.client.0.smithi139.stdout:8/954: dread d17/ff8 [0,4194304] 0 2024-09-16T00:09:51.904 INFO:tasks.workunit.client.0.smithi139.stdout:8/955: truncate d17/d1a/d50/d86/d92/f110 5222460 0 2024-09-16T00:09:51.906 INFO:tasks.workunit.client.0.smithi139.stdout:8/956: read d17/d1f/d3a/db7/f3b [1190868,129399] 0 2024-09-16T00:09:51.910 INFO:tasks.workunit.client.0.smithi139.stdout:8/957: creat d17/d1f/d3a/db0/f144 x:0 0 0 2024-09-16T00:09:51.911 INFO:tasks.workunit.client.0.smithi139.stdout:8/958: creat d17/d1a/d50/dee/d12b/f145 x:0 0 0 2024-09-16T00:09:51.914 INFO:tasks.workunit.client.0.smithi139.stdout:8/959: link d17/d1f/d3a/db7/c105 d17/d1f/d3a/db0/c146 0 2024-09-16T00:09:51.915 INFO:tasks.workunit.client.0.smithi139.stdout:8/960: symlink d17/d1f/d3a/db7/l147 0 2024-09-16T00:09:51.916 INFO:tasks.workunit.client.0.smithi139.stdout:8/961: fsync d17/d1f/d3a/db0/d120/ff6 0 2024-09-16T00:09:51.918 INFO:tasks.workunit.client.0.smithi139.stdout:8/962: unlink d17/d1f/d3a/db7/c47 0 2024-09-16T00:09:51.939 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:51.939 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:51.948 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:09:51.989 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:51.989+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:52.011 INFO:tasks.workunit.client.0.smithi139.stdout:7/820: sync 2024-09-16T00:09:52.011 INFO:tasks.workunit.client.0.smithi139.stdout:7/821: readlink d5/d37/d70/l113 0 2024-09-16T00:09:52.012 INFO:tasks.workunit.client.0.smithi139.stdout:7/822: readlink d5/l11 0 2024-09-16T00:09:52.013 INFO:tasks.workunit.client.0.smithi139.stdout:7/823: fsync d5/d26/d5d/f8e 0 2024-09-16T00:09:52.015 INFO:tasks.workunit.client.0.smithi139.stdout:7/824: dread d5/d16/d6d/d7f/dc2/fa8 [0,4194304] 0 2024-09-16T00:09:52.018 INFO:tasks.workunit.client.0.smithi139.stdout:7/825: symlink d5/d16/d6d/d7f/dc2/d50/d44/d55/d90/l125 0 2024-09-16T00:09:52.019 INFO:tasks.workunit.client.0.smithi139.stdout:7/826: read - d5/d26/d46/dea/df8/f11d zero size 2024-09-16T00:09:52.020 INFO:tasks.workunit.client.0.smithi139.stdout:7/827: chown d5/d16/d6d/d7f/cac 20381 1 2024-09-16T00:09:52.024 INFO:tasks.workunit.client.0.smithi139.stdout:7/828: dread d5/d26/d46/d6b/da7/fe2 [0,4194304] 0 2024-09-16T00:09:52.026 INFO:tasks.workunit.client.0.smithi139.stdout:7/829: mkdir d5/d26/d46/dea/d126 0 2024-09-16T00:09:52.028 INFO:tasks.workunit.client.0.smithi139.stdout:7/830: creat d5/d37/d118/f127 x:0 0 0 2024-09-16T00:09:52.030 INFO:tasks.workunit.client.0.smithi139.stdout:7/831: symlink d5/d37/l128 0 2024-09-16T00:09:52.031 INFO:tasks.workunit.client.0.smithi139.stdout:7/832: chown d5/d26/d42/f80 31718559 1 2024-09-16T00:09:52.031 INFO:tasks.workunit.client.0.smithi139.stdout:7/833: dread - d5/d16/d22/ffb zero size 2024-09-16T00:09:52.032 INFO:tasks.workunit.client.0.smithi139.stdout:7/834: write d5/d16/d22/ffb [233610,105965] 0 2024-09-16T00:09:52.033 INFO:tasks.workunit.client.0.smithi139.stdout:7/835: fsync d5/d37/d70/f9f 0 2024-09-16T00:09:52.034 INFO:tasks.workunit.client.0.smithi139.stdout:7/836: fdatasync d5/d37/d118/f127 0 2024-09-16T00:09:52.039 INFO:tasks.workunit.client.0.smithi139.stdout:7/837: write d5/d16/d22/d25/faf [539577,4051] 0 2024-09-16T00:09:52.040 INFO:tasks.workunit.client.0.smithi139.stdout:7/838: readlink d5/d16/d22/l103 0 2024-09-16T00:09:52.046 INFO:tasks.workunit.client.0.smithi139.stdout:7/839: dread d5/d16/d22/f4e [4194304,4194304] 0 2024-09-16T00:09:52.046 INFO:tasks.workunit.client.0.smithi139.stdout:7/840: chown d5/d37/ca3 0 1 2024-09-16T00:09:52.047 INFO:tasks.workunit.client.0.smithi139.stdout:7/841: stat d5/d37 0 2024-09-16T00:09:52.047 INFO:tasks.workunit.client.0.smithi139.stdout:7/842: fsync d5/d16/d22/fc7 0 2024-09-16T00:09:52.049 INFO:tasks.workunit.client.0.smithi139.stdout:7/843: link d5/l7e d5/d16/dda/l129 0 2024-09-16T00:09:52.051 INFO:tasks.workunit.client.0.smithi139.stdout:7/844: mkdir d5/d26/d46/d6b/dc5/d12a 0 2024-09-16T00:09:52.060 INFO:tasks.workunit.client.0.smithi139.stdout:7/845: dwrite d5/d16/d6d/d7f/dc2/d50/d44/d55/fd5 [0,4194304] 0 2024-09-16T00:09:52.061 INFO:tasks.workunit.client.0.smithi139.stdout:7/846: readlink d5/l77 0 2024-09-16T00:09:52.064 INFO:tasks.workunit.client.0.smithi139.stdout:7/847: rmdir d5/d16/d6d/d7f 39 2024-09-16T00:09:52.064 INFO:tasks.workunit.client.0.smithi139.stdout:7/848: chown d5/d26/d42/dae/lc9 54455 1 2024-09-16T00:09:52.067 INFO:tasks.workunit.client.0.smithi139.stdout:7/849: truncate d5/d26/d5d/f98 162690 0 2024-09-16T00:09:52.070 INFO:tasks.workunit.client.0.smithi139.stdout:7/850: fdatasync d5/d16/d6d/d7f/dc2/d50/d44/d55/d90/f10d 0 2024-09-16T00:09:52.075 INFO:tasks.workunit.client.0.smithi139.stdout:7/851: rename d5/d16/d22/l103 to d5/d16/d22/l12b 0 2024-09-16T00:09:52.077 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:52.077+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:52.077 INFO:tasks.workunit.client.0.smithi139.stdout:7/852: creat d5/d26/d46/d6b/f12c x:0 0 0 2024-09-16T00:09:52.082 INFO:tasks.workunit.client.0.smithi139.stdout:7/853: dwrite d5/d16/d6d/d7f/dc2/d50/fbd [0,4194304] 0 2024-09-16T00:09:52.098 INFO:tasks.workunit.client.0.smithi139.stdout:7/854: dread d5/d26/d74/fad [4194304,4194304] 0 2024-09-16T00:09:52.100 INFO:tasks.workunit.client.0.smithi139.stdout:7/855: mkdir d5/d26/d46/dea/d12d 0 2024-09-16T00:09:52.101 INFO:tasks.workunit.client.0.smithi139.stdout:7/856: write d5/d16/d6d/d7f/dc2/d81/fcb [66521,101155] 0 2024-09-16T00:09:52.102 INFO:tasks.workunit.client.0.smithi139.stdout:7/857: chown d5/d16/d6d/d7f/dc2/dab/fcd 14610 1 2024-09-16T00:09:52.102 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:52.102 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:52.104 INFO:tasks.workunit.client.0.smithi139.stdout:7/858: mknod d5/d16/d6d/d7f/dc2/d50/d44/d55/de0/c12e 0 2024-09-16T00:09:52.106 INFO:tasks.workunit.client.0.smithi139.stdout:7/859: mknod d5/c12f 0 2024-09-16T00:09:52.107 INFO:tasks.workunit.client.0.smithi139.stdout:7/860: read d5/d16/d6d/d7f/dc2/dab/fcd [84729,26338] 0 2024-09-16T00:09:52.109 INFO:tasks.workunit.client.0.smithi139.stdout:7/861: symlink d5/d16/d6d/d7f/dc2/d50/d44/d55/d90/l130 0 2024-09-16T00:09:52.110 INFO:tasks.workunit.client.0.smithi139.stdout:7/862: mkdir d5/d37/d70/d131 0 2024-09-16T00:09:52.111 INFO:tasks.workunit.client.0.smithi139.stdout:7/863: write d5/f59 [335373,40015] 0 2024-09-16T00:09:52.112 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:09:52.113 INFO:tasks.workunit.client.0.smithi139.stdout:3/656: sync 2024-09-16T00:09:52.113 INFO:tasks.workunit.client.0.smithi139.stdout:2/943: sync 2024-09-16T00:09:52.113 INFO:tasks.workunit.client.0.smithi139.stdout:2/944: dread - d11/d2f/f61 zero size 2024-09-16T00:09:52.115 INFO:tasks.workunit.client.0.smithi139.stdout:2/945: chown d11/d30/d110/d29/da5/daa/c107 16 1 2024-09-16T00:09:52.115 INFO:tasks.workunit.client.0.smithi139.stdout:2/946: fsync d11/d30/d110/fc1 0 2024-09-16T00:09:52.115 INFO:tasks.workunit.client.0.smithi139.stdout:3/657: unlink d5/d7/dc/d84/cc3 0 2024-09-16T00:09:52.115 INFO:tasks.workunit.client.0.smithi139.stdout:2/947: dread - d11/d2f/dcc/f130 zero size 2024-09-16T00:09:52.116 INFO:tasks.workunit.client.0.smithi139.stdout:3/658: dread - d5/d7/d23/fc9 zero size 2024-09-16T00:09:52.118 INFO:tasks.workunit.client.0.smithi139.stdout:2/948: chown d11/d30/d110/d29/c4a 7367 1 2024-09-16T00:09:52.121 INFO:tasks.workunit.client.0.smithi139.stdout:2/949: mknod d11/d30/d110/d29/da5/daa/de1/c13f 0 2024-09-16T00:09:52.121 INFO:tasks.workunit.client.0.smithi139.stdout:3/659: dwrite d5/f16 [0,4194304] 0 2024-09-16T00:09:52.123 INFO:tasks.workunit.client.0.smithi139.stdout:7/864: truncate d5/d16/d22/d25/d63/fb9 129929 0 2024-09-16T00:09:52.125 INFO:tasks.workunit.client.0.smithi139.stdout:2/950: creat d11/d30/d110/d50/f140 x:0 0 0 2024-09-16T00:09:52.125 INFO:tasks.workunit.client.0.smithi139.stdout:7/865: fdatasync d5/d16/d6d/d7f/dc2/d81/fcb 0 2024-09-16T00:09:52.125 INFO:tasks.workunit.client.0.smithi139.stdout:2/951: fdatasync ff 0 2024-09-16T00:09:52.129 INFO:tasks.workunit.client.0.smithi139.stdout:6/696: sync 2024-09-16T00:09:52.129 INFO:tasks.workunit.client.0.smithi139.stdout:5/878: sync 2024-09-16T00:09:52.129 INFO:tasks.workunit.client.0.smithi139.stdout:1/685: sync 2024-09-16T00:09:52.129 INFO:tasks.workunit.client.0.smithi139.stdout:8/963: sync 2024-09-16T00:09:52.129 INFO:tasks.workunit.client.0.smithi139.stdout:4/894: sync 2024-09-16T00:09:52.129 INFO:tasks.workunit.client.0.smithi139.stdout:0/796: sync 2024-09-16T00:09:52.134 INFO:tasks.workunit.client.0.smithi139.stdout:6/697: mknod d8/d20/cf5 0 2024-09-16T00:09:52.136 INFO:tasks.workunit.client.0.smithi139.stdout:7/866: write d5/d26/d46/d6b/da7/f105 [1742688,43725] 0 2024-09-16T00:09:52.138 INFO:tasks.workunit.client.0.smithi139.stdout:6/698: chown d8/d29/d83/c67 880 1 2024-09-16T00:09:52.138 INFO:tasks.workunit.client.0.smithi139.stdout:4/895: rename d12/d47/d128 to d12/d47/db2/d4d/dd1/d12b 0 2024-09-16T00:09:52.139 INFO:tasks.workunit.client.0.smithi139.stdout:5/879: truncate d1/d21/f39 644675 0 2024-09-16T00:09:52.140 INFO:tasks.workunit.client.0.smithi139.stdout:1/686: mkdir d7/d8/d40/d48/d4a/db8/dc6 0 2024-09-16T00:09:52.141 INFO:tasks.workunit.client.0.smithi139.stdout:2/952: dwrite d11/d30/d110/d29/d4f/fb2 [0,4194304] 0 2024-09-16T00:09:52.144 INFO:tasks.workunit.client.0.smithi139.stdout:7/867: chown d5/d26/d42/cc8 48 1 2024-09-16T00:09:52.144 INFO:tasks.workunit.client.0.smithi139.stdout:5/880: rmdir d1/df6 39 2024-09-16T00:09:52.145 INFO:tasks.workunit.client.0.smithi139.stdout:0/797: link d6/d5a/cf5 d6/d1c/d41/d44/dd/d56/cf8 0 2024-09-16T00:09:52.145 INFO:tasks.workunit.client.0.smithi139.stdout:3/660: dread d5/d7/d23/d27/d2f/d30/d4d/f8c [0,4194304] 0 2024-09-16T00:09:52.146 INFO:tasks.workunit.client.0.smithi139.stdout:4/896: mkdir d12/d47/db2/d4d/de4/d7f/d12c 0 2024-09-16T00:09:52.148 INFO:tasks.workunit.client.0.smithi139.stdout:1/687: mknod d7/d8/d40/d48/d53/cc7 0 2024-09-16T00:09:52.149 INFO:tasks.workunit.client.0.smithi139.stdout:5/881: dread - d1/d13/d3f/d88/f10f zero size 2024-09-16T00:09:52.151 INFO:tasks.workunit.client.0.smithi139.stdout:8/964: dwrite d17/d1f/d3a/db7/f42 [4194304,4194304] 0 2024-09-16T00:09:52.152 INFO:tasks.workunit.client.0.smithi139.stdout:4/897: creat d12/d80/f12d x:0 0 0 2024-09-16T00:09:52.154 INFO:tasks.workunit.client.0.smithi139.stdout:3/661: read - d5/d7/d23/f94 zero size 2024-09-16T00:09:52.160 INFO:tasks.workunit.client.0.smithi139.stdout:0/798: mknod d6/d1c/d41/d63/cf9 0 2024-09-16T00:09:52.161 INFO:tasks.workunit.client.0.smithi139.stdout:7/868: getdents d5/d37/d118 0 2024-09-16T00:09:52.162 INFO:tasks.workunit.client.0.smithi139.stdout:5/882: symlink d1/d13/l137 0 2024-09-16T00:09:52.163 INFO:tasks.workunit.client.0.smithi139.stdout:2/953: dwrite d11/d2f/deb/ff7 [0,4194304] 0 2024-09-16T00:09:52.164 INFO:tasks.workunit.client.0.smithi139.stdout:1/688: mkdir d7/d8/d40/d48/d53/d61/dc8 0 2024-09-16T00:09:52.164 INFO:tasks.workunit.client.0.smithi139.stdout:8/965: mknod d17/d1a/d50/d86/d9d/d9f/d101/c148 0 2024-09-16T00:09:52.164 INFO:tasks.workunit.client.0.smithi139.stdout:6/699: dwrite d8/d29/d83/d26/d45/d85/ff2 [0,4194304] 0 2024-09-16T00:09:52.164 INFO:tasks.workunit.client.0.smithi139.stdout:4/898: creat d12/d47/d49/d60/dc3/de2/f12e x:0 0 0 2024-09-16T00:09:52.165 INFO:tasks.workunit.client.0.smithi139.stdout:3/662: mkdir d5/d7/d23/d27/d2f/d30/d4c/d50/d75/de1 0 2024-09-16T00:09:52.168 INFO:tasks.workunit.client.0.smithi139.stdout:0/799: symlink d6/d1c/d41/d44/dd/d8a/lfa 0 2024-09-16T00:09:52.169 INFO:tasks.workunit.client.0.smithi139.stdout:7/869: write d5/d37/f40 [4661497,71421] 0 2024-09-16T00:09:52.169 INFO:tasks.workunit.client.0.smithi139.stdout:6/700: write d8/d29/d83/d26/d45/d42/dc0/fd9 [757911,96572] 0 2024-09-16T00:09:52.172 INFO:tasks.workunit.client.0.smithi139.stdout:2/954: creat d11/d2f/dcc/dea/dee/f141 x:0 0 0 2024-09-16T00:09:52.173 INFO:tasks.workunit.client.0.smithi139.stdout:0/800: write d6/d1c/d41/d7c/d86/f91 [2510623,116065] 0 2024-09-16T00:09:52.175 INFO:tasks.workunit.client.0.smithi139.stdout:3/663: mkdir d5/d7/d23/d27/d2f/d30/d4c/dbb/de2 0 2024-09-16T00:09:52.176 INFO:tasks.workunit.client.0.smithi139.stdout:1/689: write d7/d8/d40/d48/d67/f88 [850429,58336] 0 2024-09-16T00:09:52.177 INFO:tasks.workunit.client.0.smithi139.stdout:6/701: mknod d8/cf6 0 2024-09-16T00:09:52.178 INFO:tasks.workunit.client.0.smithi139.stdout:7/870: mknod d5/d16/d6d/d7f/dc2/dab/c132 0 2024-09-16T00:09:52.178 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:52.178+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:52.180 INFO:tasks.workunit.client.0.smithi139.stdout:4/899: rename d12/d47/db2/d4d/db0 to d12/d47/d49/d60/dac/d12f 0 2024-09-16T00:09:52.181 INFO:tasks.workunit.client.0.smithi139.stdout:8/966: getdents d17/d1a/d50 0 2024-09-16T00:09:52.183 INFO:tasks.workunit.client.0.smithi139.stdout:0/801: mknod d6/d1c/cfb 0 2024-09-16T00:09:52.184 INFO:tasks.workunit.client.0.smithi139.stdout:5/883: link d1/d21/d49/d53/d6a/d7b/da1/db7/fee d1/d13/d3f/d9f/d73/f138 0 2024-09-16T00:09:52.184 INFO:tasks.workunit.client.0.smithi139.stdout:3/664: truncate d5/d7/dc/d9c/fa3 55147 0 2024-09-16T00:09:52.184 INFO:tasks.workunit.client.0.smithi139.stdout:2/955: dwrite d11/d30/d110/d50/f140 [0,4194304] 0 2024-09-16T00:09:52.185 INFO:tasks.workunit.client.0.smithi139.stdout:6/702: rename d8/d20/d68/cd7 to d8/d29/d83/d26/d28/ddd/cf7 0 2024-09-16T00:09:52.186 INFO:tasks.workunit.client.0.smithi139.stdout:5/884: chown d1/dc3/lf7 7 1 2024-09-16T00:09:52.188 INFO:tasks.workunit.client.0.smithi139.stdout:4/900: write d12/d2b/d8c/fb8 [565285,6633] 0 2024-09-16T00:09:52.190 INFO:tasks.workunit.client.0.smithi139.stdout:0/802: creat d6/d1c/df7/ffc x:0 0 0 2024-09-16T00:09:52.191 INFO:tasks.workunit.client.0.smithi139.stdout:1/690: rename d7/d8/d27/d2f/l4f to d7/d8/d27/d68/lc9 0 2024-09-16T00:09:52.191 INFO:tasks.workunit.client.0.smithi139.stdout:2/956: rmdir d11/d30/d110/d29/d4f/d9c 39 2024-09-16T00:09:52.192 INFO:tasks.workunit.client.0.smithi139.stdout:3/665: unlink d5/d7/d23/d27/d2f/d30/d41/ld0 0 2024-09-16T00:09:52.193 INFO:tasks.workunit.client.0.smithi139.stdout:5/885: rmdir d1/d13/d3f/d9f/d73/d79/da5/d10b/d11e 39 2024-09-16T00:09:52.194 INFO:tasks.workunit.client.0.smithi139.stdout:6/703: dread d8/d20/d4d/fee [0,4194304] 0 2024-09-16T00:09:52.194 INFO:tasks.workunit.client.0.smithi139.stdout:4/901: mknod d12/d47/d49/d60/dac/c130 0 2024-09-16T00:09:52.195 INFO:tasks.workunit.client.0.smithi139.stdout:8/967: rename d17/d1a/d50/fec to d17/d1f/d3a/db0/d120/f149 0 2024-09-16T00:09:52.196 INFO:tasks.workunit.client.0.smithi139.stdout:0/803: read - d6/d5a/d70/fcd zero size 2024-09-16T00:09:52.197 INFO:tasks.workunit.client.0.smithi139.stdout:2/957: unlink d11/d30/d110/l12b 0 2024-09-16T00:09:52.197 INFO:tasks.workunit.client.0.smithi139.stdout:3/666: dread - d5/d7/d23/d35/fcb zero size 2024-09-16T00:09:52.198 INFO:tasks.workunit.client.0.smithi139.stdout:4/902: dread - d12/d47/f5c zero size 2024-09-16T00:09:52.200 INFO:tasks.workunit.client.0.smithi139.stdout:5/886: mkdir d1/d13/d3f/d139 0 2024-09-16T00:09:52.200 INFO:tasks.workunit.client.0.smithi139.stdout:6/704: chown d8/d29/d83/d26/d45/d42/f76 117260697 1 2024-09-16T00:09:52.202 INFO:tasks.workunit.client.0.smithi139.stdout:7/871: dwrite d5/d26/d42/f80 [0,4194304] 0 2024-09-16T00:09:52.203 INFO:tasks.workunit.client.0.smithi139.stdout:0/804: write d6/d1c/d41/d44/dd/fd0 [1600656,128563] 0 2024-09-16T00:09:52.206 INFO:tasks.workunit.client.0.smithi139.stdout:3/667: readlink d5/d7/d23/d27/d2f/d30/d47/lda 0 2024-09-16T00:09:52.206 INFO:tasks.workunit.client.0.smithi139.stdout:1/691: getdents d7/d8/d40/d48 0 2024-09-16T00:09:52.208 INFO:tasks.workunit.client.0.smithi139.stdout:8/968: dread d17/f56 [0,4194304] 0 2024-09-16T00:09:52.208 INFO:tasks.workunit.client.0.smithi139.stdout:5/887: truncate d1/d13/d3f/fd3 4735227 0 2024-09-16T00:09:52.209 INFO:tasks.workunit.client.0.smithi139.stdout:7/872: mknod d5/d26/c133 0 2024-09-16T00:09:52.209 INFO:tasks.workunit.client.0.smithi139.stdout:8/969: stat d17/d89 0 2024-09-16T00:09:52.211 INFO:tasks.workunit.client.0.smithi139.stdout:5/888: fdatasync d1/d13/d22/da9/f101 0 2024-09-16T00:09:52.212 INFO:tasks.workunit.client.0.smithi139.stdout:4/903: dread d12/d2b/f30 [0,4194304] 0 2024-09-16T00:09:52.213 INFO:tasks.workunit.client.0.smithi139.stdout:3/668: dread d5/d7/f28 [0,4194304] 0 2024-09-16T00:09:52.213 INFO:tasks.workunit.client.0.smithi139.stdout:0/805: rename d6/d1c/d41/d78/ca1 to d6/d1c/d41/d44/dd/d56/dbd/cfd 0 2024-09-16T00:09:52.214 INFO:tasks.workunit.client.0.smithi139.stdout:8/970: chown d17/d1f/d3a/db7/d40/d4f/f75 26 1 2024-09-16T00:09:52.214 INFO:tasks.workunit.client.0.smithi139.stdout:7/873: mknod d5/d16/d6d/d7f/df5/c134 0 2024-09-16T00:09:52.215 INFO:tasks.workunit.client.0.smithi139.stdout:5/889: stat d1/d21/d49/d53/d6a/d7b/d99 0 2024-09-16T00:09:52.216 INFO:tasks.workunit.client.0.smithi139.stdout:6/705: write d8/d29/f50 [896931,78218] 0 2024-09-16T00:09:52.216 INFO:tasks.workunit.client.0.smithi139.stdout:4/904: chown d12/d47/d49/f112 14333 1 2024-09-16T00:09:52.217 INFO:tasks.workunit.client.0.smithi139.stdout:0/806: write d6/d1c/f8d [1547548,53068] 0 2024-09-16T00:09:52.220 INFO:tasks.workunit.client.0.smithi139.stdout:2/958: dread d11/f1a [0,4194304] 0 2024-09-16T00:09:52.221 INFO:tasks.workunit.client.0.smithi139.stdout:3/669: symlink d5/d7/d23/d35/dd4/le3 0 2024-09-16T00:09:52.222 INFO:tasks.workunit.client.0.smithi139.stdout:0/807: chown d6/d5a/d70/f53 72 1 2024-09-16T00:09:52.223 INFO:tasks.workunit.client.0.smithi139.stdout:1/692: dwrite d7/d8/d1f/fa6 [0,4194304] 0 2024-09-16T00:09:52.224 INFO:tasks.workunit.client.0.smithi139.stdout:7/874: mknod d5/d16/d6d/d7f/dc2/d50/d44/c135 0 2024-09-16T00:09:52.225 INFO:tasks.workunit.client.0.smithi139.stdout:7/875: stat d5/d16/d6d 0 2024-09-16T00:09:52.226 INFO:tasks.workunit.client.0.smithi139.stdout:4/905: mknod d12/d47/db2/d4d/c131 0 2024-09-16T00:09:52.227 INFO:tasks.workunit.client.0.smithi139.stdout:2/959: fdatasync d11/d30/d110/d5d/fac 0 2024-09-16T00:09:52.228 INFO:tasks.workunit.client.0.smithi139.stdout:4/906: fsync d12/d47/db2/d4d/de4/f11c 0 2024-09-16T00:09:52.230 INFO:tasks.workunit.client.0.smithi139.stdout:6/706: creat d8/d29/d83/d26/d28/d6d/dd3/de5/ff8 x:0 0 0 2024-09-16T00:09:52.231 INFO:tasks.workunit.client.0.smithi139.stdout:0/808: mknod d6/d5a/de7/cfe 0 2024-09-16T00:09:52.232 INFO:tasks.workunit.client.0.smithi139.stdout:4/907: write d12/d47/d49/d60/dac/d12f/f10b [157916,82182] 0 2024-09-16T00:09:52.234 INFO:tasks.workunit.client.0.smithi139.stdout:1/693: creat d7/d8/d27/d68/d7c/d83/fca x:0 0 0 2024-09-16T00:09:52.234 INFO:tasks.workunit.client.0.smithi139.stdout:7/876: rmdir d5/d16/d6d/d7f/dc2/d50/d44/d55 39 2024-09-16T00:09:52.237 INFO:tasks.workunit.client.0.smithi139.stdout:5/890: dread d1/d13/d22/dd5/f119 [0,4194304] 0 2024-09-16T00:09:52.238 INFO:tasks.workunit.client.0.smithi139.stdout:2/960: dwrite d11/d30/d110/d5d/ff9 [0,4194304] 0 2024-09-16T00:09:52.239 INFO:tasks.workunit.client.0.smithi139.stdout:3/670: rename d5/d7/d23/d27/d2f/d30/d4c/d50 to d5/d7/d23/de4 0 2024-09-16T00:09:52.243 INFO:tasks.workunit.client.0.smithi139.stdout:5/891: write d1/d13/d94/f104 [1008418,4625] 0 2024-09-16T00:09:52.245 INFO:tasks.workunit.client.0.smithi139.stdout:8/971: rename d17/d1f/d3a/db7/d40/f104 to d17/d1a/d50/dee/d12b/f14a 0 2024-09-16T00:09:52.247 INFO:tasks.workunit.client.0.smithi139.stdout:2/961: symlink d11/d2f/deb/l142 0 2024-09-16T00:09:52.247 INFO:tasks.workunit.client.0.smithi139.stdout:3/671: write d5/d7/dc2/fd7 [981827,121785] 0 2024-09-16T00:09:52.248 INFO:tasks.workunit.client.0.smithi139.stdout:0/809: getdents d6/d5a 0 2024-09-16T00:09:52.253 INFO:tasks.workunit.client.0.smithi139.stdout:4/908: dwrite d12/d47/d49/ffc [0,4194304] 0 2024-09-16T00:09:52.253 INFO:tasks.workunit.client.0.smithi139.stdout:6/707: link d8/d1f/c44 d8/d29/d83/d26/d45/d42/dc0/cf9 0 2024-09-16T00:09:52.253 INFO:tasks.workunit.client.0.smithi139.stdout:1/694: write d7/d8/d40/d46/d8d/f95 [915312,31583] 0 2024-09-16T00:09:52.254 INFO:tasks.workunit.client.0.smithi139.stdout:2/962: creat d11/d32/f143 x:0 0 0 2024-09-16T00:09:52.257 INFO:tasks.workunit.client.0.smithi139.stdout:0/810: mknod d6/d5a/d70/cff 0 2024-09-16T00:09:52.258 INFO:tasks.workunit.client.0.smithi139.stdout:1/695: dread - d7/d8/fa0 zero size 2024-09-16T00:09:52.259 INFO:tasks.workunit.client.0.smithi139.stdout:5/892: symlink d1/d13/d3f/d9f/d8a/l13a 0 2024-09-16T00:09:52.259 INFO:tasks.workunit.client.0.smithi139.stdout:3/672: chown d5/d7/d23/de4/f54 268700 1 2024-09-16T00:09:52.262 INFO:tasks.workunit.client.0.smithi139.stdout:2/963: rmdir d11/d30/d110/d29/d2b 39 2024-09-16T00:09:52.262 INFO:tasks.workunit.client.0.smithi139.stdout:7/877: dread d5/d16/d6d/d7f/dc2/d50/d44/d55/f6c [4194304,4194304] 0 2024-09-16T00:09:52.263 INFO:tasks.workunit.client.0.smithi139.stdout:0/811: fsync d6/d1c/d41/d44/dd/f6e 0 2024-09-16T00:09:52.263 INFO:tasks.workunit.client.0.smithi139.stdout:6/708: mkdir d8/dfa 0 2024-09-16T00:09:52.264 INFO:tasks.workunit.client.0.smithi139.stdout:8/972: dwrite d17/d54/fbe [0,4194304] 0 2024-09-16T00:09:52.265 INFO:tasks.workunit.client.0.smithi139.stdout:2/964: chown d11/f19 2012673 1 2024-09-16T00:09:52.268 INFO:tasks.workunit.client.0.smithi139.stdout:5/893: creat d1/d13/d3f/d9f/d132/f13b x:0 0 0 2024-09-16T00:09:52.268 INFO:tasks.workunit.client.0.smithi139.stdout:1/696: read d7/d8/d40/d48/d53/d61/f87 [31451,35372] 0 2024-09-16T00:09:52.276 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:52.276 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:52.279 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:52.278+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:52.285 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:09:52.287 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:09:52.296 INFO:tasks.workunit.client.0.smithi139.stdout:0/812: chown d6/d1c/d41/d44/dd/d56/d75/ca5 1651 1 2024-09-16T00:09:52.297 INFO:tasks.workunit.client.0.smithi139.stdout:5/894: chown d1/d21/c62 65103033 1 2024-09-16T00:09:52.297 INFO:tasks.workunit.client.0.smithi139.stdout:4/909: dwrite d12/d47/db2/d4d/f9d [0,4194304] 0 2024-09-16T00:09:52.298 INFO:tasks.workunit.client.0.smithi139.stdout:8/973: truncate d17/d1f/d3a/dcd/f116 501221 0 2024-09-16T00:09:52.298 INFO:tasks.workunit.client.0.smithi139.stdout:2/965: read d11/d30/d110/d5d/d73/f8e [3870310,13563] 0 2024-09-16T00:09:52.299 INFO:tasks.workunit.client.0.smithi139.stdout:0/813: stat d6/d1c/d41/d78/df2 0 2024-09-16T00:09:52.299 INFO:tasks.workunit.client.0.smithi139.stdout:8/974: write d17/d1f/d3a/db0/f144 [343884,63450] 0 2024-09-16T00:09:52.300 INFO:tasks.workunit.client.0.smithi139.stdout:1/697: rename d7/d8/f6f to d7/d8/d1f/fcb 0 2024-09-16T00:09:52.300 INFO:tasks.workunit.client.0.smithi139.stdout:0/814: chown d6/d5a/de7/lf0 12 1 2024-09-16T00:09:52.302 INFO:tasks.workunit.client.0.smithi139.stdout:4/910: unlink d12/c5d 0 2024-09-16T00:09:52.302 INFO:tasks.workunit.client.0.smithi139.stdout:0/815: chown d6/d1c/d41/d44/dd/d8a/lf4 16 1 2024-09-16T00:09:52.303 INFO:tasks.workunit.client.0.smithi139.stdout:2/966: symlink d11/d30/d110/d50/d114/ddb/l144 0 2024-09-16T00:09:52.304 INFO:tasks.workunit.client.0.smithi139.stdout:4/911: chown f4 855007 1 2024-09-16T00:09:52.307 INFO:tasks.workunit.client.0.smithi139.stdout:6/709: rename d8/d29/d83/d26/d45/d85/fae to d8/d29/d83/d26/d28/d9e/da8/ffb 0 2024-09-16T00:09:52.308 INFO:tasks.workunit.client.0.smithi139.stdout:0/816: mkdir d6/d1c/df7/d100 0 2024-09-16T00:09:52.311 INFO:tasks.workunit.client.0.smithi139.stdout:5/895: getdents d1/d13/d3f/d5e/dc2/d10c 0 2024-09-16T00:09:52.311 INFO:tasks.workunit.client.0.smithi139.stdout:2/967: stat d11/d30/d110/d29/d2b/l5e 0 2024-09-16T00:09:52.312 INFO:tasks.workunit.client.0.smithi139.stdout:0/817: readlink d6/d1c/d41/d44/d6b/lb9 0 2024-09-16T00:09:52.313 INFO:tasks.workunit.client.0.smithi139.stdout:8/975: dwrite d17/d1a/d50/dee/f141 [0,4194304] 0 2024-09-16T00:09:52.314 INFO:tasks.workunit.client.0.smithi139.stdout:1/698: dwrite d7/d8/d40/d48/d67/d7d/f94 [0,4194304] 0 2024-09-16T00:09:52.316 INFO:tasks.workunit.client.0.smithi139.stdout:2/968: write d11/d30/d110/d29/fc4 [899087,62109] 0 2024-09-16T00:09:52.316 INFO:tasks.workunit.client.0.smithi139.stdout:3/673: truncate d5/d7/d23/d27/d2f/d30/d4d/dbc/fc5 15978 0 2024-09-16T00:09:52.317 INFO:tasks.workunit.client.0.smithi139.stdout:0/818: write d6/d5a/d70/d7b/fba [816083,64264] 0 2024-09-16T00:09:52.317 INFO:tasks.workunit.client.0.smithi139.stdout:1/699: write d7/d8/d40/d48/d4a/f98 [4927777,24406] 0 2024-09-16T00:09:52.318 INFO:tasks.workunit.client.0.smithi139.stdout:5/896: creat d1/d21/d49/db2/f13c x:0 0 0 2024-09-16T00:09:52.322 INFO:tasks.workunit.client.0.smithi139.stdout:8/976: fdatasync d17/d1a/d2d/f6b 0 2024-09-16T00:09:52.323 INFO:tasks.workunit.client.0.smithi139.stdout:0/819: unlink d6/d1c/fc1 0 2024-09-16T00:09:52.323 INFO:tasks.workunit.client.0.smithi139.stdout:3/674: write d5/d7/d23/d27/d2f/d30/d4c/d6d/f7e [1374371,4362] 0 2024-09-16T00:09:52.324 INFO:tasks.workunit.client.0.smithi139.stdout:6/710: getdents d8/d29/d83/d26/d45/d59 0 2024-09-16T00:09:52.325 INFO:tasks.workunit.client.0.smithi139.stdout:8/977: fdatasync d17/d1f/d3a/db7/d40/fb4 0 2024-09-16T00:09:52.327 INFO:tasks.workunit.client.0.smithi139.stdout:1/700: mkdir d7/d8/d40/d48/dcc 0 2024-09-16T00:09:52.328 INFO:tasks.workunit.client.0.smithi139.stdout:2/969: dwrite f2 [8388608,4194304] 0 2024-09-16T00:09:52.329 INFO:tasks.workunit.client.0.smithi139.stdout:0/820: chown d6/d1c/d41/d44/d6b/fde 4810158 1 2024-09-16T00:09:52.334 INFO:tasks.workunit.client.0.smithi139.stdout:5/897: dread - d1/d13/d3f/d9f/d73/d79/da5/d10b/d11e/d10d/d33/f122 zero size 2024-09-16T00:09:52.335 INFO:tasks.workunit.client.0.smithi139.stdout:2/970: rmdir d11/d30/dbf/d116 39 2024-09-16T00:09:52.341 INFO:tasks.workunit.client.0.smithi139.stdout:6/711: dwrite d8/d29/d83/f81 [4194304,4194304] 0 2024-09-16T00:09:52.344 INFO:tasks.workunit.client.0.smithi139.stdout:2/971: creat d11/d30/d110/d50/d114/f145 x:0 0 0 2024-09-16T00:09:52.344 INFO:tasks.workunit.client.0.smithi139.stdout:6/712: write d8/f12 [412435,77168] 0 2024-09-16T00:09:52.346 INFO:tasks.workunit.client.0.smithi139.stdout:8/978: truncate d17/d1f/d3a/db7/d40/fb4 1547070 0 2024-09-16T00:09:52.351 INFO:tasks.workunit.client.0.smithi139.stdout:1/701: dwrite d7/d8/d27/d68/d6c/fb1 [0,4194304] 0 2024-09-16T00:09:52.352 INFO:tasks.workunit.client.0.smithi139.stdout:2/972: chown d11/d30/d110/c62 1491 1 2024-09-16T00:09:52.352 INFO:tasks.workunit.client.0.smithi139.stdout:3/675: dread d5/d7/d23/d27/d2f/d30/d47/f4a [0,4194304] 0 2024-09-16T00:09:52.354 INFO:tasks.workunit.client.0.smithi139.stdout:5/898: rename d1/d21/d49/d53/d6a/d7b/lbd to d1/d21/d49/d87/l13d 0 2024-09-16T00:09:52.354 INFO:tasks.workunit.client.0.smithi139.stdout:3/676: chown d5/d7/d23 1188 1 2024-09-16T00:09:52.354 INFO:tasks.workunit.client.0.smithi139.stdout:6/713: readlink d8/d20/l4c 0 2024-09-16T00:09:52.355 INFO:tasks.workunit.client.0.smithi139.stdout:0/821: dread d6/d1c/f61 [0,4194304] 0 2024-09-16T00:09:52.355 INFO:tasks.workunit.client.0.smithi139.stdout:1/702: symlink d7/d8/d27/d68/d7c/d83/lcd 0 2024-09-16T00:09:52.355 INFO:tasks.workunit.client.0.smithi139.stdout:2/973: mknod d11/d2f/deb/c146 0 2024-09-16T00:09:52.357 INFO:tasks.workunit.client.0.smithi139.stdout:8/979: write d17/d1a/d50/d86/d9d/daf/fc7 [700729,46883] 0 2024-09-16T00:09:52.359 INFO:tasks.workunit.client.0.smithi139.stdout:1/703: chown d7/d8/d27/d2f/f73 195882 1 2024-09-16T00:09:52.359 INFO:tasks.workunit.client.0.smithi139.stdout:0/822: readlink d6/d1c/d41/d44/dd/d8a/d9e/ldb 0 2024-09-16T00:09:52.360 INFO:tasks.workunit.client.0.smithi139.stdout:6/714: creat d8/d29/d83/d26/d28/d6d/dd3/ffc x:0 0 0 2024-09-16T00:09:52.361 INFO:tasks.workunit.client.0.smithi139.stdout:8/980: readlink d17/d1f/d3a/db7/l23 0 2024-09-16T00:09:52.363 INFO:tasks.workunit.client.0.smithi139.stdout:2/974: creat d11/d30/d110/d29/d4f/f147 x:0 0 0 2024-09-16T00:09:52.364 INFO:tasks.workunit.client.0.smithi139.stdout:0/823: write d6/d1c/d41/d44/f71 [2110709,9960] 0 2024-09-16T00:09:52.366 INFO:tasks.workunit.client.0.smithi139.stdout:8/981: chown d17/d1f/d3a/dcd 3785634 1 2024-09-16T00:09:52.367 INFO:tasks.workunit.client.0.smithi139.stdout:3/677: rename d5/d7/d23/d27/c8e to d5/d7/d23/d27/d2f/d30/d4c/dbb/de2/ce5 0 2024-09-16T00:09:52.368 INFO:tasks.workunit.client.0.smithi139.stdout:5/899: dwrite d1/fe9 [0,4194304] 0 2024-09-16T00:09:52.372 INFO:tasks.workunit.client.0.smithi139.stdout:8/982: chown d17/d1a/f1c 61177243 1 2024-09-16T00:09:52.372 INFO:tasks.workunit.client.0.smithi139.stdout:6/715: mknod d8/d29/d83/d26/d45/d42/df3/cfd 0 2024-09-16T00:09:52.372 INFO:tasks.workunit.client.0.smithi139.stdout:3/678: dread - d5/d7/d23/d27/d2f/d30/d4d/dbc/fce zero size 2024-09-16T00:09:52.372 INFO:tasks.workunit.client.0.smithi139.stdout:0/824: rmdir d6/d1c/d41/d76 39 2024-09-16T00:09:52.375 INFO:tasks.workunit.client.0.smithi139.stdout:5/900: creat d1/d13/d3b/d3e/f13e x:0 0 0 2024-09-16T00:09:52.377 INFO:tasks.workunit.client.0.smithi139.stdout:6/716: mknod d8/d29/cfe 0 2024-09-16T00:09:52.377 INFO:tasks.workunit.client.0.smithi139.stdout:1/704: dwrite d7/d8/d40/d47/f6a [0,4194304] 0 2024-09-16T00:09:52.381 INFO:tasks.workunit.client.0.smithi139.stdout:0/825: write d6/d1c/d41/d44/dd/d8a/d9e/fc3 [2767159,42667] 0 2024-09-16T00:09:52.382 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:52.379+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:52.384 INFO:tasks.workunit.client.0.smithi139.stdout:2/975: dwrite d11/f1e [0,4194304] 0 2024-09-16T00:09:52.384 INFO:tasks.workunit.client.0.smithi139.stdout:3/679: write d5/d7/d23/d27/d2f/d30/d4c/d6d/f79 [3722877,3903] 0 2024-09-16T00:09:52.385 INFO:tasks.workunit.client.0.smithi139.stdout:6/717: rmdir d8/d29/d83/d26/d45 39 2024-09-16T00:09:52.389 INFO:tasks.workunit.client.0.smithi139.stdout:5/901: write d1/d21/d49/f98 [597591,71255] 0 2024-09-16T00:09:52.390 INFO:tasks.workunit.client.0.smithi139.stdout:3/680: creat d5/d7/d23/d27/d2f/d30/d41/d68/fe6 x:0 0 0 2024-09-16T00:09:52.390 INFO:tasks.workunit.client.0.smithi139.stdout:1/705: chown d7/d8/l92 867 1 2024-09-16T00:09:52.393 INFO:tasks.workunit.client.0.smithi139.stdout:0/826: link d6/d1c/d41/d44/dd/fa0 d6/d1c/d41/d76/f101 0 2024-09-16T00:09:52.394 INFO:tasks.workunit.client.0.smithi139.stdout:3/681: fdatasync d5/f53 0 2024-09-16T00:09:52.396 INFO:tasks.workunit.client.0.smithi139.stdout:5/902: rename d1/d21/d49/d53/d6a/d7b/d99/fae to d1/d13/d3f/d9f/d73/dd0/f13f 0 2024-09-16T00:09:52.396 INFO:tasks.workunit.client.0.smithi139.stdout:0/827: symlink d6/d42/l102 0 2024-09-16T00:09:52.398 INFO:tasks.workunit.client.0.smithi139.stdout:2/976: write d11/d30/d110/d29/d4f/d9c/fca [992122,92677] 0 2024-09-16T00:09:52.399 INFO:tasks.workunit.client.0.smithi139.stdout:0/828: rename d6/d5a/d81 to d6/d5a/d81/dc5/d103 22 2024-09-16T00:09:52.403 INFO:tasks.workunit.client.0.smithi139.stdout:5/903: rename d1/d21/d49/d53/d6a/f9b to d1/d13/d3f/d9f/d73/d79/da5/d10b/d11e/d10d/dd4/f140 0 2024-09-16T00:09:52.406 INFO:tasks.workunit.client.0.smithi139.stdout:3/682: dread d5/d7/f49 [0,4194304] 0 2024-09-16T00:09:52.406 INFO:tasks.workunit.client.0.smithi139.stdout:0/829: getdents d6/d1c/d41/d78/df2 0 2024-09-16T00:09:52.408 INFO:tasks.workunit.client.0.smithi139.stdout:5/904: fdatasync d1/d13/f17 0 2024-09-16T00:09:52.408 INFO:tasks.workunit.client.0.smithi139.stdout:2/977: truncate d11/d30/d110/d50/f140 1266938 0 2024-09-16T00:09:52.409 INFO:tasks.workunit.client.0.smithi139.stdout:2/978: read - d11/d2f/dcc/f130 zero size 2024-09-16T00:09:52.415 INFO:tasks.workunit.client.0.smithi139.stdout:2/979: dread - d11/d30/dbf/d116/d125/f11e zero size 2024-09-16T00:09:52.417 INFO:tasks.workunit.client.0.smithi139.stdout:2/980: dread - d11/d45/f7c zero size 2024-09-16T00:09:52.422 INFO:tasks.workunit.client.0.smithi139.stdout:5/905: dwrite d1/d13/d3f/d9f/d73/d79/d8d/f9e [0,4194304] 0 2024-09-16T00:09:52.423 INFO:tasks.workunit.client.0.smithi139.stdout:3/683: dwrite d5/d7/d23/fc9 [0,4194304] 0 2024-09-16T00:09:52.424 INFO:tasks.workunit.client.0.smithi139.stdout:2/981: mknod d11/d45/c148 0 2024-09-16T00:09:52.424 INFO:tasks.workunit.client.0.smithi139.stdout:0/830: dwrite d6/d5a/f65 [0,4194304] 0 2024-09-16T00:09:52.426 INFO:tasks.workunit.client.0.smithi139.stdout:3/684: stat d5/d7/d23/d27/d2f/d30/d41 0 2024-09-16T00:09:52.430 INFO:tasks.workunit.client.0.smithi139.stdout:0/831: symlink d6/d42/l104 0 2024-09-16T00:09:52.431 INFO:tasks.workunit.client.0.smithi139.stdout:2/982: creat d11/d2f/deb/d10b/f149 x:0 0 0 2024-09-16T00:09:52.434 INFO:tasks.workunit.client.0.smithi139.stdout:2/983: creat d11/d2f/f14a x:0 0 0 2024-09-16T00:09:52.434 INFO:tasks.workunit.client.0.smithi139.stdout:0/832: getdents d6/d1c/df7/d100 0 2024-09-16T00:09:52.435 INFO:tasks.workunit.client.0.smithi139.stdout:0/833: chown d6/d1c/d41/d7c/d86/c8b 525 1 2024-09-16T00:09:52.437 INFO:tasks.workunit.client.0.smithi139.stdout:0/834: dread - d6/d1c/d41/d63/d82/fbe zero size 2024-09-16T00:09:52.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.458+0000 7f5af9caf640 1 Processor -- start 2024-09-16T00:09:52.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.458+0000 7f5af9caf640 1 -- start start 2024-09-16T00:09:52.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.459+0000 7f5af9caf640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5af406da30 0x7f5af40cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:52.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.459+0000 7f5af9caf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5af40c8800 con 0x7f5af4073bf0 2024-09-16T00:09:52.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.459+0000 7f5af9caf640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5af40c7bd0 con 0x7f5af40b2170 2024-09-16T00:09:52.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.459+0000 7f5af9caf640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5af40c7980 con 0x7f5af406da30 2024-09-16T00:09:52.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.459+0000 7f5af3fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5af4073bf0 0x7f5af41651b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59276/0 (socket says 172.21.15.139:59276) 2024-09-16T00:09:52.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.459+0000 7f5af3fff640 1 -- 172.21.15.139:0/3070068224 learned_addr learned my addr 172.21.15.139:0/3070068224 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:52.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.459+0000 7f5af2ffd640 1 --2- 172.21.15.139:0/3070068224 >> v2:172.21.15.164:3300/0 conn(0x7f5af406da30 0x7f5af40cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:52.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.459+0000 7f5af27fc640 1 -- 172.21.15.139:0/3070068224 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 893686646 0 0) 0x7f5af40c8800 con 0x7f5af4073bf0 2024-09-16T00:09:52.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.459+0000 7f5af27fc640 1 -- 172.21.15.139:0/3070068224 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5adc003520 con 0x7f5af4073bf0 2024-09-16T00:09:52.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af27fc640 1 -- 172.21.15.139:0/3070068224 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 487561747 0 0) 0x7f5adc003520 con 0x7f5af4073bf0 2024-09-16T00:09:52.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af27fc640 1 -- 172.21.15.139:0/3070068224 >> v2:172.21.15.164:3300/0 conn(0x7f5af406da30 msgr2=0x7f5af40cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:52.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af27fc640 1 --2- 172.21.15.139:0/3070068224 >> v2:172.21.15.164:3300/0 conn(0x7f5af406da30 0x7f5af40cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:52.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af27fc640 1 -- 172.21.15.139:0/3070068224 >> v1:172.21.15.164:6789/0 conn(0x7f5af40b2170 legacy=0x7f5af415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:52.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af27fc640 1 -- 172.21.15.139:0/3070068224 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af40c7eb0 con 0x7f5af4073bf0 2024-09-16T00:09:52.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af27fc640 1 -- 172.21.15.139:0/3070068224 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7f5aec001a10 con 0x7f5af4073bf0 2024-09-16T00:09:52.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af27fc640 1 -- 172.21.15.139:0/3070068224 >> v1:172.21.15.139:6789/0 conn(0x7f5af4073bf0 legacy=0x7f5af41651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:52.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af27fc640 1 --2- 172.21.15.139:0/3070068224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5adc0039b0 0x7f5adc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:52.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af27fc640 1 --2- 172.21.15.139:0/3070068224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5adc004800 0x7f5adc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:52.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af27fc640 1 --2- 172.21.15.139:0/3070068224 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5adc0051c0 0x7f5adc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:52.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af27fc640 1 -- 172.21.15.139:0/3070068224 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5af40c7980 con 0x7f5adc0039b0 2024-09-16T00:09:52.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af27fc640 1 -- 172.21.15.139:0/3070068224 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5adc00c060 con 0x7f5adc004800 2024-09-16T00:09:52.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af27fc640 1 -- 172.21.15.139:0/3070068224 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5adc00c260 con 0x7f5adc0051c0 2024-09-16T00:09:52.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af2ffd640 1 --2- 172.21.15.139:0/3070068224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5adc004800 0x7f5adc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:52.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af37fe640 1 --2- 172.21.15.139:0/3070068224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5adc0039b0 0x7f5adc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:52.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af3fff640 1 --2- 172.21.15.139:0/3070068224 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5adc0051c0 0x7f5adc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:52.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af37fe640 1 -- 172.21.15.139:0/3070068224 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5adc0051c0 msgr2=0x7f5adc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:52.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af37fe640 1 --2- 172.21.15.139:0/3070068224 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5adc0051c0 0x7f5adc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:52.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af37fe640 1 -- 172.21.15.139:0/3070068224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5adc004800 msgr2=0x7f5adc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:52.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af37fe640 1 --2- 172.21.15.139:0/3070068224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5adc004800 0x7f5adc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:52.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af37fe640 1 -- 172.21.15.139:0/3070068224 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5adc003520 con 0x7f5adc0039b0 2024-09-16T00:09:52.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af3fff640 1 -- 172.21.15.139:0/3070068224 reap_dead start 2024-09-16T00:09:52.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.460+0000 7f5af2ffd640 1 --2- 172.21.15.139:0/3070068224 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5adc004800 0x7f5adc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:52.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.461+0000 7f5af37fe640 1 --2- 172.21.15.139:0/3070068224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5adc0039b0 0x7f5adc003dd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4001750 tx=0x7f5adc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29957bcc038fdaec server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:52.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.461+0000 7f5af27fc640 1 -- 172.21.15.139:0/3070068224 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4001370 con 0x7f5adc0039b0 2024-09-16T00:09:52.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.461+0000 7f5af27fc640 1 -- 172.21.15.139:0/3070068224 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ae40052d0 con 0x7f5adc0039b0 2024-09-16T00:09:52.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.461+0000 7f5af27fc640 1 -- 172.21.15.139:0/3070068224 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae402abe0 con 0x7f5adc0039b0 2024-09-16T00:09:52.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.462+0000 7f5af9caf640 1 -- 172.21.15.139:0/3070068224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5adc0039b0 msgr2=0x7f5adc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:52.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.462+0000 7f5af9caf640 1 --2- 172.21.15.139:0/3070068224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5adc0039b0 0x7f5adc003dd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4001750 tx=0x7f5adc005830 comp rx=0 tx=0).stop 2024-09-16T00:09:52.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.462+0000 7f5af9caf640 1 -- 172.21.15.139:0/3070068224 shutdown_connections 2024-09-16T00:09:52.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.462+0000 7f5af9caf640 1 --2- 172.21.15.139:0/3070068224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5adc0039b0 0x7f5adc003dd0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:52.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.462+0000 7f5af9caf640 1 -- 172.21.15.139:0/3070068224 >> 172.21.15.139:0/3070068224 conn(0x7f5af4067950 msgr2=0x7f5af4067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:52.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.462+0000 7f5af9caf640 1 -- 172.21.15.139:0/3070068224 shutdown_connections 2024-09-16T00:09:52.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.462+0000 7f5af9caf640 1 -- 172.21.15.139:0/3070068224 wait complete. 2024-09-16T00:09:52.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.463+0000 7f5af9caf640 1 Processor -- start 2024-09-16T00:09:52.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.463+0000 7f5af9caf640 1 -- start start 2024-09-16T00:09:52.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af9caf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5adc0039b0 0x7f5af41f04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:52.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af9caf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af4156eb0 0x7f5af415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:52.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af9caf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5af4154920 0x7f5af414db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:52.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af9caf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5af40c4020 con 0x7f5adc0039b0 2024-09-16T00:09:52.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af9caf640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5af4073480 con 0x7f5af4156eb0 2024-09-16T00:09:52.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af9caf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5af40c41a0 con 0x7f5af4154920 2024-09-16T00:09:52.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5adc0039b0 0x7f5af41f04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:52.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af37fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5adc0039b0 0x7f5af41f04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:37582/0 (socket says 172.21.15.139:37582) 2024-09-16T00:09:52.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af37fe640 1 -- 172.21.15.139:0/3197580778 learned_addr learned my addr 172.21.15.139:0/3197580778 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:52.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af3fff640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5af4154920 0x7f5af414db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:52.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af2ffd640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af4156eb0 0x7f5af415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:52.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af37fe640 1 -- 172.21.15.139:0/3197580778 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5af4154920 msgr2=0x7f5af414db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:52.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af37fe640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5af4154920 0x7f5af414db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:52.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af37fe640 1 -- 172.21.15.139:0/3197580778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af4156eb0 msgr2=0x7f5af415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:52.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af37fe640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af4156eb0 0x7f5af415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:52.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af37fe640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af41550f0 con 0x7f5adc0039b0 2024-09-16T00:09:52.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af2ffd640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5af4156eb0 0x7f5af415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:09:52.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af37fe640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5adc0039b0 0x7f5af41f04c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4005040 tx=0x7f5ae4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:52.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae403f020 con 0x7f5adc0039b0 2024-09-16T00:09:52.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ae4005480 con 0x7f5adc0039b0 2024-09-16T00:09:52.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af41553d0 con 0x7f5adc0039b0 2024-09-16T00:09:52.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4007a30 con 0x7f5adc0039b0 2024-09-16T00:09:52.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.464+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af4155990 con 0x7f5adc0039b0 2024-09-16T00:09:52.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.465+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f5af40b1660 con 0x7f5adc0039b0 2024-09-16T00:09:52.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.466+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5ae4042020 con 0x7f5adc0039b0 2024-09-16T00:09:52.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.466+0000 7f5adbfff640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5ae006c2b0 0x7f5ae006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:52.470 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:52.470 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:52.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.466+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5973+0+0 (secure 0 0 0) 0x7f5ae40ab0b0 con 0x7f5adc0039b0 2024-09-16T00:09:52.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.466+0000 7f5af2ffd640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5ae006c2b0 0x7f5ae006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:52.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.466+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v90) ==== 54+0+16 (secure 0 0 0) 0x7f5ae4049030 con 0x7f5adc0039b0 2024-09-16T00:09:52.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.467+0000 7f5af9caf640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5af415e670 0x7f5af414c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:52.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.467+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5af4074070 con 0x7f5af415e670 2024-09-16T00:09:52.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.467+0000 7f5af3fff640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5af415e670 0x7f5af414c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:52.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.467+0000 7f5af2ffd640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5ae006c2b0 0x7f5ae006e770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5af41572b0 tx=0x7f5ae8016040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:52.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.467+0000 7f5af3fff640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f5af415e670 0x7f5af414c740 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:52.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.468+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5af4074070 con 0x7f5af415e670 2024-09-16T00:09:52.479 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:09:52.486 INFO:tasks.workunit.client.0.smithi139.stdout:7/878: sync 2024-09-16T00:09:52.486 INFO:tasks.workunit.client.0.smithi139.stdout:4/912: sync 2024-09-16T00:09:52.486 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:52.485+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:52.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.486+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5af40c2650 con 0x7f5af415e670 2024-09-16T00:09:52.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.488+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5af40c2650 con 0x7f5af415e670 2024-09-16T00:09:52.489 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:52.489 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:52.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.488+0000 7f5af9caf640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5af415ea50 0x7f5af41ec4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:52.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.488+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f5af406d230 con 0x7f5af415ea50 2024-09-16T00:09:52.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.488+0000 7f5af37fe640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5af415ea50 0x7f5af41ec4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:52.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.488+0000 7f5af37fe640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5af415ea50 0x7f5af41ec4e0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:52.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.489+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5af406d230 con 0x7f5af415ea50 2024-09-16T00:09:52.497 INFO:tasks.workunit.client.0.smithi139.stdout:4/913: unlink d12/d47/db2/d4d/da9/lad 0 2024-09-16T00:09:52.497 INFO:tasks.workunit.client.0.smithi139.stdout:5/906: dwrite d1/d13/d3f/fb8 [0,4194304] 0 2024-09-16T00:09:52.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.506+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5af4157500 con 0x7f5af415ea50 2024-09-16T00:09:52.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.508+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5af4157500 con 0x7f5af415ea50 2024-09-16T00:09:52.509 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:52.510 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:52.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.508+0000 7f5af9caf640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5af41ed0a0 0x7f5af4062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:52.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.508+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f5af4062b70 con 0x7f5af41ed0a0 2024-09-16T00:09:52.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.509+0000 7f5af2ffd640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5af41ed0a0 0x7f5af4062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:52.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.509+0000 7f5af2ffd640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f5af41ed0a0 0x7f5af4062630 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:52.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.510+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5af4062b70 con 0x7f5af41ed0a0 2024-09-16T00:09:52.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.523+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5af4063020 con 0x7f5af41ed0a0 2024-09-16T00:09:52.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.525+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5af4063020 con 0x7f5af41ed0a0 2024-09-16T00:09:52.526 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:52.526 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:52.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.525+0000 7f5af9caf640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f5af40c9030 0x7f5af40c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:52.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.525+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f5af40635f0 con 0x7f5af40c9030 2024-09-16T00:09:52.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.525+0000 7f5af3fff640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f5af40c9030 0x7f5af40c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:52.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.525+0000 7f5af3fff640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f5af40c9030 0x7f5af40c9490 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:52.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.526+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5aec005cd0 con 0x7f5af40c9030 2024-09-16T00:09:52.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.540+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5af4062fa0 con 0x7f5af40c9030 2024-09-16T00:09:52.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.541+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5af4062fa0 con 0x7f5af40c9030 2024-09-16T00:09:52.542 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:52.543 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:52.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.541+0000 7f5af9caf640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5af402cb50 0x7f5af402cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:52.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.541+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f5af402d490 con 0x7f5af402cb50 2024-09-16T00:09:52.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.542+0000 7f5af37fe640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5af402cb50 0x7f5af402cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:52.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.542+0000 7f5af37fe640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5af402cb50 0x7f5af402cf50 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:52.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.543+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5af402d490 con 0x7f5af402cb50 2024-09-16T00:09:52.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.556+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5af402d490 con 0x7f5af402cb50 2024-09-16T00:09:52.558 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.558+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5af402d490 con 0x7f5af402cb50 2024-09-16T00:09:52.559 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:52.559 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:52.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.558+0000 7f5af9caf640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5af4200e90 0x7f5af4203280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:52.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.558+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f5af402db10 con 0x7f5af4200e90 2024-09-16T00:09:52.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.558+0000 7f5af2ffd640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5af4200e90 0x7f5af4203280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:52.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.559+0000 7f5af2ffd640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f5af4200e90 0x7f5af4203280 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:52.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.560+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5af402db10 con 0x7f5af4200e90 2024-09-16T00:09:52.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.573+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5af402db10 con 0x7f5af4200e90 2024-09-16T00:09:52.575 INFO:tasks.workunit.client.0.smithi139.stdout:2/984: dwrite d11/d30/d110/d29/da5/fbe [0,4194304] 0 2024-09-16T00:09:52.575 INFO:tasks.workunit.client.0.smithi139.stdout:0/835: rename d6/d5a/d70/l5c to d6/l105 0 2024-09-16T00:09:52.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.575+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5af402db10 con 0x7f5af4200e90 2024-09-16T00:09:52.576 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:52.577 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:52.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.575+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=91}) -- 0x7f5af402db10 con 0x7f5adc0039b0 2024-09-16T00:09:52.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.575+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5af4206050 con 0x7f5adc0039b0 2024-09-16T00:09:52.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.576+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f5ae40b2070 con 0x7f5adc0039b0 2024-09-16T00:09:52.577 INFO:teuthology.orchestra.run.smithi139.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T00:09:52.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.576+0000 7f5af9caf640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5af4204090 0x7f5af42044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:52.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.576+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f5af4205b80 con 0x7f5af4204090 2024-09-16T00:09:52.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.576+0000 7f5af3fff640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5af4204090 0x7f5af42044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:52.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.576+0000 7f5af3fff640 1 --2- 172.21.15.139:0/3197580778 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f5af4204090 0x7f5af42044c0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:52.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.577+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5af4205b80 con 0x7f5af4204090 2024-09-16T00:09:52.581 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:52.581+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:52.585 INFO:tasks.workunit.client.0.smithi139.stdout:4/914: chown d12/d2b/l3d 0 1 2024-09-16T00:09:52.586 INFO:tasks.workunit.client.0.smithi139.stdout:4/915: read d12/d47/db2/f54 [80225,37389] 0 2024-09-16T00:09:52.588 INFO:tasks.workunit.client.0.smithi139.stdout:4/916: readlink d12/d47/db2/d4d/de4/d7f/l91 0 2024-09-16T00:09:52.589 INFO:tasks.workunit.client.0.smithi139.stdout:4/917: stat d12/d47/d49/d60/dac/d12f/lfb 0 2024-09-16T00:09:52.591 INFO:tasks.workunit.client.0.smithi139.stdout:4/918: write d12/d47/db2/d4d/de4/d7f/f110 [552324,115382] 0 2024-09-16T00:09:52.595 INFO:tasks.workunit.client.0.smithi139.stdout:2/985: chown d11/d2f/d6e/c7e 3 1 2024-09-16T00:09:52.596 INFO:tasks.workunit.client.0.smithi139.stdout:2/986: chown d11/d30/d110/d29/d2b/c68 62474 1 2024-09-16T00:09:52.597 INFO:tasks.workunit.client.0.smithi139.stdout:4/919: write d12/d47/db2/d4d/fbb [3901750,35354] 0 2024-09-16T00:09:52.597 INFO:tasks.workunit.client.0.smithi139.stdout:7/879: link d5/d16/d6d/d7f/dc2/dab/c132 d5/d26/d46/de4/c136 0 2024-09-16T00:09:52.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.597+0000 7f5af9caf640 1 -- 172.21.15.139:0/3197580778 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5af40c99d0 con 0x7f5af4204090 2024-09-16T00:09:52.598 INFO:tasks.workunit.client.0.smithi139.stdout:4/920: readlink d12/l87 0 2024-09-16T00:09:52.599 INFO:tasks.workunit.client.0.smithi139.stdout:7/880: truncate d5/d26/d42/f80 4463579 0 2024-09-16T00:09:52.600 INFO:tasks.workunit.client.0.smithi139.stdout:0/836: getdents d6/d1c/d41/d44/dd/d8a 0 2024-09-16T00:09:52.601 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:52.601 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:52.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:52.601 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:52.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.601 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:219 2.1f 2:f83d65d8:::200.00000004:head [write 863658~2915703 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e65)", 2024-09-16T00:09:52.601 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:52.602 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.727946693, 2024-09-16T00:09:52.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013772627000000001, 2024-09-16T00:09:52.602 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.602 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.602 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.602 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:52.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.603 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:09:52.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.603 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.603 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870941+0000", 2024-09-16T00:09:52.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4e-07 2024-09-16T00:09:52.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.603 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.603 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877134+0000", 2024-09-16T00:09:52.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061936769999999999 2024-09-16T00:09:52.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.603 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.604 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877153+0000", 2024-09-16T00:09:52.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8408000000000001e-05 2024-09-16T00:09:52.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.604 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.604 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877161+0000", 2024-09-16T00:09:52.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2709999999999992e-06 2024-09-16T00:09:52.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.604 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.604 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877225+0000", 2024-09-16T00:09:52.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4190999999999996e-05 2024-09-16T00:09:52.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877289+0000", 2024-09-16T00:09:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4058000000000002e-05 2024-09-16T00:09:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877376+0000", 2024-09-16T00:09:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7227999999999999e-05 2024-09-16T00:09:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.882925+0000", 2024-09-16T00:09:52.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055481050000000002 2024-09-16T00:09:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884635+0000", 2024-09-16T00:09:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017102759999999999 2024-09-16T00:09:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884674+0000", 2024-09-16T00:09:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8914e-05 2024-09-16T00:09:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884712+0000", 2024-09-16T00:09:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658999999999997e-05 2024-09-16T00:09:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10767 3.1b 3:dab1607f:::100000030fd.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.379273+0000", 2024-09-16T00:09:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.219613961, 2024-09-16T00:09:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012051947, 2024-09-16T00:09:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.607 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10767 2024-09-16T00:09:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.379273+0000", 2024-09-16T00:09:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.379273+0000", 2024-09-16T00:09:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.379275+0000", 2024-09-16T00:09:52.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6230000000000001e-06 2024-09-16T00:09:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381583+0000", 2024-09-16T00:09:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023079939999999998 2024-09-16T00:09:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381617+0000", 2024-09-16T00:09:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4227999999999998e-05 2024-09-16T00:09:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381633+0000", 2024-09-16T00:09:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5642e-05 2024-09-16T00:09:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381739+0000", 2024-09-16T00:09:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010638500000000001 2024-09-16T00:09:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381850+0000", 2024-09-16T00:09:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110164 2024-09-16T00:09:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.382033+0000", 2024-09-16T00:09:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018355699999999999 2024-09-16T00:09:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.388313+0000", 2024-09-16T00:09:52.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006279588 2024-09-16T00:09:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.391197+0000", 2024-09-16T00:09:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028841130000000002 2024-09-16T00:09:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.391263+0000", 2024-09-16T00:09:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6105000000000005e-05 2024-09-16T00:09:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.391324+0000", 2024-09-16T00:09:52.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1548000000000001e-05 2024-09-16T00:09:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10768 3.2b 3:d738455c:::100000030db.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:09:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.20476152, 2024-09-16T00:09:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012411195999999999, 2024-09-16T00:09:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.614 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10768 2024-09-16T00:09:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:09:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.615 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:09:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394129+0000", 2024-09-16T00:09:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4100000000000001e-06 2024-09-16T00:09:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395723+0000", 2024-09-16T00:09:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015938409999999999 2024-09-16T00:09:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395745+0000", 2024-09-16T00:09:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1611e-05 2024-09-16T00:09:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395782+0000", 2024-09-16T00:09:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7292000000000001e-05 2024-09-16T00:09:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395856+0000", 2024-09-16T00:09:52.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3675000000000002e-05 2024-09-16T00:09:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395924+0000", 2024-09-16T00:09:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8646999999999995e-05 2024-09-16T00:09:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.396069+0000", 2024-09-16T00:09:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014455199999999999 2024-09-16T00:09:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.618 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.403755+0000", 2024-09-16T00:09:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0076863160000000003 2024-09-16T00:09:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406407+0000", 2024-09-16T00:09:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026516220000000002 2024-09-16T00:09:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406477+0000", 2024-09-16T00:09:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0204999999999996e-05 2024-09-16T00:09:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406536+0000", 2024-09-16T00:09:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9024999999999998e-05 2024-09-16T00:09:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10941 3.1 3:8246fad2:::10000003173.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:09:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 13.143008182999999, 2024-09-16T00:09:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01286782, 2024-09-16T00:09:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.620 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10941 2024-09-16T00:09:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:09:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:09:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455881+0000", 2024-09-16T00:09:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7269999999999999e-06 2024-09-16T00:09:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457462+0000", 2024-09-16T00:09:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015811690000000001 2024-09-16T00:09:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457487+0000", 2024-09-16T00:09:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4607e-05 2024-09-16T00:09:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457497+0000", 2024-09-16T00:09:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9939999999999995e-06 2024-09-16T00:09:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457571+0000", 2024-09-16T00:09:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3867000000000003e-05 2024-09-16T00:09:52.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457689+0000", 2024-09-16T00:09:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000118814 2024-09-16T00:09:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457804+0000", 2024-09-16T00:09:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114702 2024-09-16T00:09:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.463422+0000", 2024-09-16T00:09:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0056179239999999998 2024-09-16T00:09:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468641+0000", 2024-09-16T00:09:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052192109999999996 2024-09-16T00:09:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468703+0000", 2024-09-16T00:09:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1268999999999999e-05 2024-09-16T00:09:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468746+0000", 2024-09-16T00:09:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:09:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11742 3.1b 3:d847de26:::100000032cd.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:09:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.5425165980000006, 2024-09-16T00:09:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012791903, 2024-09-16T00:09:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.626 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11742 2024-09-16T00:09:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:09:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:09:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056373+0000", 2024-09-16T00:09:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9799999999999998e-06 2024-09-16T00:09:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058604+0000", 2024-09-16T00:09:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022316020000000001 2024-09-16T00:09:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058630+0000", 2024-09-16T00:09:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5425000000000001e-05 2024-09-16T00:09:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058643+0000", 2024-09-16T00:09:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3361e-05 2024-09-16T00:09:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058731+0000", 2024-09-16T00:09:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7270999999999998e-05 2024-09-16T00:09:52.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058894+0000", 2024-09-16T00:09:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016376800000000001 2024-09-16T00:09:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.059050+0000", 2024-09-16T00:09:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015603700000000001 2024-09-16T00:09:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.066364+0000", 2024-09-16T00:09:52.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073138639999999998 2024-09-16T00:09:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069025+0000", 2024-09-16T00:09:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026607660000000002 2024-09-16T00:09:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069083+0000", 2024-09-16T00:09:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7680999999999999e-05 2024-09-16T00:09:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.631 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069162+0000", 2024-09-16T00:09:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9147999999999995e-05 2024-09-16T00:09:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11934 3.1b 3:d94fa8da:::10000003190.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:09:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.9104733229999997, 2024-09-16T00:09:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.017110325999999999, 2024-09-16T00:09:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.632 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11934 2024-09-16T00:09:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:09:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:09:52.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688416+0000", 2024-09-16T00:09:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.875e-06 2024-09-16T00:09:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689633+0000", 2024-09-16T00:09:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001216997 2024-09-16T00:09:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.634 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689655+0000", 2024-09-16T00:09:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2285000000000001e-05 2024-09-16T00:09:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689665+0000", 2024-09-16T00:09:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7089999999999997e-06 2024-09-16T00:09:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689739+0000", 2024-09-16T00:09:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3973999999999993e-05 2024-09-16T00:09:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689898+0000", 2024-09-16T00:09:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015941100000000001 2024-09-16T00:09:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.690062+0000", 2024-09-16T00:09:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000163633 2024-09-16T00:09:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.695224+0000", 2024-09-16T00:09:52.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051620210000000001 2024-09-16T00:09:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705434+0000", 2024-09-16T00:09:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010209701 2024-09-16T00:09:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705488+0000", 2024-09-16T00:09:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3974000000000002e-05 2024-09-16T00:09:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705523+0000", 2024-09-16T00:09:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5746e-05 2024-09-16T00:09:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11941 3.12 3:4a0e6682:::1000000334b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:09:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.8754558100000001, 2024-09-16T00:09:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01458448, 2024-09-16T00:09:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.638 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11941 2024-09-16T00:09:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:09:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:09:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723433+0000", 2024-09-16T00:09:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3410000000000001e-06 2024-09-16T00:09:52.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724966+0000", 2024-09-16T00:09:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001533242 2024-09-16T00:09:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724983+0000", 2024-09-16T00:09:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7156999999999999e-05 2024-09-16T00:09:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.641 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724995+0000", 2024-09-16T00:09:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2089999999999999e-05 2024-09-16T00:09:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725062+0000", 2024-09-16T00:09:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6026999999999997e-05 2024-09-16T00:09:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725159+0000", 2024-09-16T00:09:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7374000000000007e-05 2024-09-16T00:09:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725289+0000", 2024-09-16T00:09:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012988 2024-09-16T00:09:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.731944+0000", 2024-09-16T00:09:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066547439999999998 2024-09-16T00:09:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.737956+0000", 2024-09-16T00:09:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0060121630000000001 2024-09-16T00:09:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.737985+0000", 2024-09-16T00:09:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9352999999999998e-05 2024-09-16T00:09:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.738015+0000", 2024-09-16T00:09:52.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0108999999999999e-05 2024-09-16T00:09:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11954 3.6 3:625e0e90:::100000032ef.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:09:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.8310042430000006, 2024-09-16T00:09:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012484808999999999, 2024-09-16T00:09:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.645 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11954 2024-09-16T00:09:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:09:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.646 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:09:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767886+0000", 2024-09-16T00:09:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6650000000000001e-06 2024-09-16T00:09:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770082+0000", 2024-09-16T00:09:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021964979999999999 2024-09-16T00:09:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770104+0000", 2024-09-16T00:09:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1671999999999999e-05 2024-09-16T00:09:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770114+0000", 2024-09-16T00:09:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0336000000000001e-05 2024-09-16T00:09:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770197+0000", 2024-09-16T00:09:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2997000000000005e-05 2024-09-16T00:09:52.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770324+0000", 2024-09-16T00:09:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012680100000000001 2024-09-16T00:09:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770497+0000", 2024-09-16T00:09:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000172776 2024-09-16T00:09:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.776330+0000", 2024-09-16T00:09:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058330939999999996 2024-09-16T00:09:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780252+0000", 2024-09-16T00:09:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039215689999999997 2024-09-16T00:09:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780317+0000", 2024-09-16T00:09:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4948000000000002e-05 2024-09-16T00:09:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.650 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780367+0000", 2024-09-16T00:09:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0453e-05 2024-09-16T00:09:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12198 3.12 3:4a0e6682:::1000000334b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.683089+0000", 2024-09-16T00:09:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 7.9157971929999995, 2024-09-16T00:09:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011990244000000001, 2024-09-16T00:09:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.651 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12198 2024-09-16T00:09:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.683089+0000", 2024-09-16T00:09:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.652 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.683089+0000", 2024-09-16T00:09:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.683092+0000", 2024-09-16T00:09:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.931e-06 2024-09-16T00:09:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684715+0000", 2024-09-16T00:09:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00162269 2024-09-16T00:09:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.653 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684734+0000", 2024-09-16T00:09:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9545999999999999e-05 2024-09-16T00:09:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684745+0000", 2024-09-16T00:09:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0131e-05 2024-09-16T00:09:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684814+0000", 2024-09-16T00:09:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9740000000000007e-05 2024-09-16T00:09:52.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.655 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684871+0000", 2024-09-16T00:09:52.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6215999999999998e-05 2024-09-16T00:09:52.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.655 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:52.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684980+0000", 2024-09-16T00:09:52.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109713 2024-09-16T00:09:52.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.655 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.690215+0000", 2024-09-16T00:09:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052349049999999998 2024-09-16T00:09:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.694970+0000", 2024-09-16T00:09:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047552649999999998 2024-09-16T00:09:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.695030+0000", 2024-09-16T00:09:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9086999999999999e-05 2024-09-16T00:09:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.656 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.695080+0000", 2024-09-16T00:09:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0019999999999999e-05 2024-09-16T00:09:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13263 3.1b 3:d9746e52:::1000000361c.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.785956+0000", 2024-09-16T00:09:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.8129302530000002, 2024-09-16T00:09:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011930029, 2024-09-16T00:09:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.657 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13263 2024-09-16T00:09:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.785956+0000", 2024-09-16T00:09:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.785956+0000", 2024-09-16T00:09:52.658 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.785959+0000", 2024-09-16T00:09:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6089999999999999e-06 2024-09-16T00:09:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787198+0000", 2024-09-16T00:09:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012395889999999999 2024-09-16T00:09:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787225+0000", 2024-09-16T00:09:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6101999999999998e-05 2024-09-16T00:09:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787239+0000", 2024-09-16T00:09:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4384e-05 2024-09-16T00:09:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787329+0000", 2024-09-16T00:09:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0555000000000003e-05 2024-09-16T00:09:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.660 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787503+0000", 2024-09-16T00:09:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017311599999999999 2024-09-16T00:09:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787676+0000", 2024-09-16T00:09:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017352900000000001 2024-09-16T00:09:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.793377+0000", 2024-09-16T00:09:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057008409999999999 2024-09-16T00:09:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.661 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.797765+0000", 2024-09-16T00:09:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004388042 2024-09-16T00:09:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.797831+0000", 2024-09-16T00:09:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.614e-05 2024-09-16T00:09:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.797886+0000", 2024-09-16T00:09:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5121999999999998e-05 2024-09-16T00:09:52.662 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13618 3.1 3:80dd39f3:::1000000336c.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.108808+0000", 2024-09-16T00:09:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.490078413, 2024-09-16T00:09:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013244522, 2024-09-16T00:09:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.663 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13618 2024-09-16T00:09:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.108808+0000", 2024-09-16T00:09:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.108808+0000", 2024-09-16T00:09:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.108812+0000", 2024-09-16T00:09:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4640000000000002e-06 2024-09-16T00:09:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.110957+0000", 2024-09-16T00:09:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002145185 2024-09-16T00:09:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.665 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.110974+0000", 2024-09-16T00:09:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7108999999999998e-05 2024-09-16T00:09:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.110982+0000", 2024-09-16T00:09:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6330000000000002e-06 2024-09-16T00:09:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.111130+0000", 2024-09-16T00:09:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014738899999999999 2024-09-16T00:09:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.111299+0000", 2024-09-16T00:09:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016908400000000001 2024-09-16T00:09:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.111470+0000", 2024-09-16T00:09:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170667 2024-09-16T00:09:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.117141+0000", 2024-09-16T00:09:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005670993 2024-09-16T00:09:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.121945+0000", 2024-09-16T00:09:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0048042370000000003 2024-09-16T00:09:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.121998+0000", 2024-09-16T00:09:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2802e-05 2024-09-16T00:09:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.122053+0000", 2024-09-16T00:09:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4959e-05 2024-09-16T00:09:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.668 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13628 3.12 3:491d0769:::100000031da.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.134341+0000", 2024-09-16T00:09:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.4645451129999998, 2024-09-16T00:09:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011817984, 2024-09-16T00:09:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.669 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13628 2024-09-16T00:09:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.134341+0000", 2024-09-16T00:09:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.134341+0000", 2024-09-16T00:09:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.134344+0000", 2024-09-16T00:09:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.785e-06 2024-09-16T00:09:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.135856+0000", 2024-09-16T00:09:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015116610000000001 2024-09-16T00:09:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.135878+0000", 2024-09-16T00:09:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2603999999999999e-05 2024-09-16T00:09:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.135890+0000", 2024-09-16T00:09:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1372000000000001e-05 2024-09-16T00:09:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.135958+0000", 2024-09-16T00:09:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7792000000000004e-05 2024-09-16T00:09:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.136023+0000", 2024-09-16T00:09:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5239000000000003e-05 2024-09-16T00:09:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.136147+0000", 2024-09-16T00:09:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012371600000000001 2024-09-16T00:09:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.141510+0000", 2024-09-16T00:09:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053634800000000003 2024-09-16T00:09:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.146037+0000", 2024-09-16T00:09:52.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045272130000000004 2024-09-16T00:09:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.146105+0000", 2024-09-16T00:09:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8038000000000001e-05 2024-09-16T00:09:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.146159+0000", 2024-09-16T00:09:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4083999999999999e-05 2024-09-16T00:09:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13687 3.32 3:4e600466:::1000000377a.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.340141+0000", 2024-09-16T00:09:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.2587454899999999, 2024-09-16T00:09:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012781707999999999, 2024-09-16T00:09:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13687 2024-09-16T00:09:52.675 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.340141+0000", 2024-09-16T00:09:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.340141+0000", 2024-09-16T00:09:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.340143+0000", 2024-09-16T00:09:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9520000000000001e-06 2024-09-16T00:09:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343159+0000", 2024-09-16T00:09:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030156290000000001 2024-09-16T00:09:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343178+0000", 2024-09-16T00:09:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9219999999999999e-05 2024-09-16T00:09:52.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343212+0000", 2024-09-16T00:09:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3745999999999998e-05 2024-09-16T00:09:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343293+0000", 2024-09-16T00:09:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1074000000000003e-05 2024-09-16T00:09:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343430+0000", 2024-09-16T00:09:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013718700000000001 2024-09-16T00:09:52.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343589+0000", 2024-09-16T00:09:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000158806 2024-09-16T00:09:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.352468+0000", 2024-09-16T00:09:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0088798209999999995 2024-09-16T00:09:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.352835+0000", 2024-09-16T00:09:52.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00036653400000000001 2024-09-16T00:09:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.352884+0000", 2024-09-16T00:09:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8881000000000002e-05 2024-09-16T00:09:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.352923+0000", 2024-09-16T00:09:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8858e-05 2024-09-16T00:09:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13722 3.6 3:639728a8:::100000030cc.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.442783+0000", 2024-09-16T00:09:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.1561036470000001, 2024-09-16T00:09:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013776286, 2024-09-16T00:09:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13722 2024-09-16T00:09:52.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.442783+0000", 2024-09-16T00:09:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.442783+0000", 2024-09-16T00:09:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.442785+0000", 2024-09-16T00:09:52.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9369999999999998e-06 2024-09-16T00:09:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444337+0000", 2024-09-16T00:09:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015527169999999999 2024-09-16T00:09:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444362+0000", 2024-09-16T00:09:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4513999999999999e-05 2024-09-16T00:09:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444375+0000", 2024-09-16T00:09:52.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3052e-05 2024-09-16T00:09:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444451+0000", 2024-09-16T00:09:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5814000000000003e-05 2024-09-16T00:09:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444566+0000", 2024-09-16T00:09:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011508199999999999 2024-09-16T00:09:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444727+0000", 2024-09-16T00:09:52.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000160743 2024-09-16T00:09:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.451504+0000", 2024-09-16T00:09:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006777773 2024-09-16T00:09:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.456446+0000", 2024-09-16T00:09:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004941834 2024-09-16T00:09:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.456510+0000", 2024-09-16T00:09:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3880999999999994e-05 2024-09-16T00:09:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.456559+0000", 2024-09-16T00:09:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8939e-05 2024-09-16T00:09:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13773 3.10 3:0a21c9f4:::100000034e7.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.578694+0000", 2024-09-16T00:09:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.0201927130000001, 2024-09-16T00:09:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012227249000000001, 2024-09-16T00:09:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13773 2024-09-16T00:09:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.578694+0000", 2024-09-16T00:09:52.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.578694+0000", 2024-09-16T00:09:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.578698+0000", 2024-09-16T00:09:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8700000000000002e-06 2024-09-16T00:09:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.581979+0000", 2024-09-16T00:09:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003281455 2024-09-16T00:09:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.581994+0000", 2024-09-16T00:09:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4861000000000001e-05 2024-09-16T00:09:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.582003+0000", 2024-09-16T00:09:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0399999999999998e-06 2024-09-16T00:09:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.582070+0000", 2024-09-16T00:09:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7189999999999999e-05 2024-09-16T00:09:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.582158+0000", 2024-09-16T00:09:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7843000000000006e-05 2024-09-16T00:09:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.590796+0000", 2024-09-16T00:09:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0086376820000000007 2024-09-16T00:09:52.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.590842+0000", 2024-09-16T00:09:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6779000000000001e-05 2024-09-16T00:09:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.590921+0000", 2024-09-16T00:09:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8529000000000007e-05 2024-09-16T00:09:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.691 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13876 3.17 3:e847251e:::10000003522.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.796002+0000", 2024-09-16T00:09:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 2.8028841399999997, 2024-09-16T00:09:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013015385000000001, 2024-09-16T00:09:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13876 2024-09-16T00:09:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.796002+0000", 2024-09-16T00:09:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.796002+0000", 2024-09-16T00:09:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.796004+0000", 2024-09-16T00:09:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1090000000000001e-06 2024-09-16T00:09:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797545+0000", 2024-09-16T00:09:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015402219999999999 2024-09-16T00:09:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797570+0000", 2024-09-16T00:09:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4905999999999998e-05 2024-09-16T00:09:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797580+0000", 2024-09-16T00:09:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0886e-05 2024-09-16T00:09:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797652+0000", 2024-09-16T00:09:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1050000000000006e-05 2024-09-16T00:09:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797814+0000", 2024-09-16T00:09:52.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000162578 2024-09-16T00:09:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797965+0000", 2024-09-16T00:09:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015092099999999999 2024-09-16T00:09:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.804256+0000", 2024-09-16T00:09:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0062911349999999998 2024-09-16T00:09:52.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.808943+0000", 2024-09-16T00:09:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0046868200000000004 2024-09-16T00:09:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.808985+0000", 2024-09-16T00:09:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1659000000000002e-05 2024-09-16T00:09:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.809018+0000", 2024-09-16T00:09:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3099000000000003e-05 2024-09-16T00:09:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13926 3.6 3:620fe43b:::100000031c3.00000001:head [write 0~4194304 [2@0]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.894056+0000", 2024-09-16T00:09:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 2.7048301380000002, 2024-09-16T00:09:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011575439999999999, 2024-09-16T00:09:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.698 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13926 2024-09-16T00:09:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.894056+0000", 2024-09-16T00:09:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.894056+0000", 2024-09-16T00:09:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.894059+0000", 2024-09-16T00:09:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0060000000000001e-06 2024-09-16T00:09:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.896006+0000", 2024-09-16T00:09:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019468860000000001 2024-09-16T00:09:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.896046+0000", 2024-09-16T00:09:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9792999999999997e-05 2024-09-16T00:09:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.896063+0000", 2024-09-16T00:09:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6660999999999999e-05 2024-09-16T00:09:52.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.896233+0000", 2024-09-16T00:09:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017058899999999999 2024-09-16T00:09:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.896472+0000", 2024-09-16T00:09:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00023886400000000001 2024-09-16T00:09:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.896794+0000", 2024-09-16T00:09:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00032206500000000001 2024-09-16T00:09:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.905005+0000", 2024-09-16T00:09:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.008210874 2024-09-16T00:09:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.905557+0000", 2024-09-16T00:09:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055212100000000004 2024-09-16T00:09:52.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.905597+0000", 2024-09-16T00:09:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9756999999999999e-05 2024-09-16T00:09:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.905632+0000", 2024-09-16T00:09:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4823999999999997e-05 2024-09-16T00:09:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.704 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13963 3.2b 3:d4ebc4d4:::100000034a5.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:50.005950+0000", 2024-09-16T00:09:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 2.5929369060000003, 2024-09-16T00:09:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.062103750999999999, 2024-09-16T00:09:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.705 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13963 2024-09-16T00:09:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.005950+0000", 2024-09-16T00:09:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.005950+0000", 2024-09-16T00:09:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.005952+0000", 2024-09-16T00:09:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.255e-06 2024-09-16T00:09:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008027+0000", 2024-09-16T00:09:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002075019 2024-09-16T00:09:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008051+0000", 2024-09-16T00:09:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3906000000000001e-05 2024-09-16T00:09:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008063+0000", 2024-09-16T00:09:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2564e-05 2024-09-16T00:09:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008137+0000", 2024-09-16T00:09:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3946e-05 2024-09-16T00:09:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008234+0000", 2024-09-16T00:09:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6497999999999997e-05 2024-09-16T00:09:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:52.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008377+0000", 2024-09-16T00:09:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014313300000000001 2024-09-16T00:09:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.067062+0000", 2024-09-16T00:09:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058685043999999999 2024-09-16T00:09:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.067963+0000", 2024-09-16T00:09:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000901413 2024-09-16T00:09:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.068013+0000", 2024-09-16T00:09:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9997000000000003e-05 2024-09-16T00:09:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.068053+0000", 2024-09-16T00:09:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9975999999999998e-05 2024-09-16T00:09:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.711 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2683 2.8 e89/19 2:17051619:::200.0000000f:head v 89'470, mlcod=89'468)", 2024-09-16T00:09:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:50.011143+0000", 2024-09-16T00:09:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 2.5877432570000001, 2024-09-16T00:09:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.056951953, 2024-09-16T00:09:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.011143+0000", 2024-09-16T00:09:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.011143+0000", 2024-09-16T00:09:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.011145+0000", 2024-09-16T00:09:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2869999999999999e-06 2024-09-16T00:09:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.011161+0000", 2024-09-16T00:09:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6920999999999999e-05 2024-09-16T00:09:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063382+0000", 2024-09-16T00:09:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.052221072 2024-09-16T00:09:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063394+0000", 2024-09-16T00:09:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1212999999999999e-05 2024-09-16T00:09:52.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063454+0000", 2024-09-16T00:09:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9852999999999998e-05 2024-09-16T00:09:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063477+0000", 2024-09-16T00:09:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3638999999999998e-05 2024-09-16T00:09:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.068064+0000", 2024-09-16T00:09:52.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045870599999999996 2024-09-16T00:09:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.068095+0000", 2024-09-16T00:09:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0908000000000003e-05 2024-09-16T00:09:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13967 3.32 3:4d814898:::10000003587.00000001:head [write 0~4194304 [2@0]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:50.020209+0000", 2024-09-16T00:09:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 2.5786775789999998, 2024-09-16T00:09:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.050981571000000003, 2024-09-16T00:09:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13967 2024-09-16T00:09:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.020209+0000", 2024-09-16T00:09:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.020209+0000", 2024-09-16T00:09:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.020212+0000", 2024-09-16T00:09:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.802e-06 2024-09-16T00:09:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.021676+0000", 2024-09-16T00:09:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014645909999999999 2024-09-16T00:09:52.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063378+0000", 2024-09-16T00:09:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.041701657000000003 2024-09-16T00:09:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063395+0000", 2024-09-16T00:09:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7469000000000002e-05 2024-09-16T00:09:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063457+0000", 2024-09-16T00:09:52.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1450000000000003e-05 2024-09-16T00:09:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063621+0000", 2024-09-16T00:09:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000164026 2024-09-16T00:09:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063777+0000", 2024-09-16T00:09:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015628899999999999 2024-09-16T00:09:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:52.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.070303+0000", 2024-09-16T00:09:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0065262860000000001 2024-09-16T00:09:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.071105+0000", 2024-09-16T00:09:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000801423 2024-09-16T00:09:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.071154+0000", 2024-09-16T00:09:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8949000000000002e-05 2024-09-16T00:09:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.071190+0000", 2024-09-16T00:09:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6628999999999998e-05 2024-09-16T00:09:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.722 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.722 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:52.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:52.600+0000 7f5adbfff640 1 -- 172.21.15.139:0/3197580778 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5af40c99d0 con 0x7f5af4204090 2024-09-16T00:09:52.723 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:52.723 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:52.723 INFO:tasks.workunit.client.0.smithi139.stdout:4/921: fdatasync f2 0 2024-09-16T00:09:52.724 INFO:tasks.workunit.client.0.smithi139.stdout:2/987: symlink d11/d30/d110/d29/d86/df3/d108/l14b 0 2024-09-16T00:09:52.724 INFO:tasks.workunit.client.0.smithi139.stdout:0/837: creat d6/d1c/d41/d44/dd/d8a/d9e/f106 x:0 0 0 2024-09-16T00:09:52.724 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T00:09:52.725 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T00:09:52.725 INFO:tasks.workunit.client.0.smithi139.stdout:2/988: chown d11/d2f/d6e/de5/f12f 0 1 2024-09-16T00:09:52.726 INFO:tasks.workunit.client.0.smithi139.stdout:7/881: dwrite d5/d16/d6d/d7f/dc2/d50/fbd [0,4194304] 0 2024-09-16T00:09:52.726 INFO:tasks.workunit.client.0.smithi139.stdout:2/989: mkdir d11/d2f/dcc/dea/dee/d14c 0 2024-09-16T00:09:52.726 INFO:tasks.workunit.client.0.smithi139.stdout:2/990: dread - d11/d30/d110/d50/dcb/f11d zero size 2024-09-16T00:09:52.726 INFO:tasks.workunit.client.0.smithi139.stdout:4/922: truncate d12/d47/f32 7623763 0 2024-09-16T00:09:52.726 INFO:tasks.workunit.client.0.smithi139.stdout:0/838: dwrite d6/d5a/fd7 [0,4194304] 0 2024-09-16T00:09:52.727 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:09:52.731 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:52.726+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:52.732 INFO:tasks.workunit.client.0.smithi139.stdout:0/839: write d6/d5a/fd7 [1249031,102730] 0 2024-09-16T00:09:52.732 INFO:tasks.workunit.client.0.smithi139.stdout:0/840: dread - d6/d5a/d81/fc2 zero size 2024-09-16T00:09:52.732 INFO:tasks.workunit.client.0.smithi139.stdout:5/907: dwrite d1/d13/d3f/d9f/d73/d79/da5/d10b/d11e/d10d/d33/d43/ffc [0,4194304] 0 2024-09-16T00:09:52.732 INFO:tasks.workunit.client.0.smithi139.stdout:4/923: link d12/d47/db2/lcc d12/d47/db2/d4d/d69/l132 0 2024-09-16T00:09:52.732 INFO:tasks.workunit.client.0.smithi139.stdout:0/841: creat d6/d42/db4/f107 x:0 0 0 2024-09-16T00:09:52.732 INFO:tasks.workunit.client.0.smithi139.stdout:5/908: readlink d1/d13/d3f/d9f/d73/d79/da5/lbc 0 2024-09-16T00:09:52.732 INFO:tasks.workunit.client.0.smithi139.stdout:5/909: write d1/d21/d49/d87/f105 [519487,37163] 0 2024-09-16T00:09:52.732 INFO:tasks.workunit.client.0.smithi139.stdout:7/882: dread d5/d37/d70/f9f [4194304,4194304] 0 2024-09-16T00:09:52.733 INFO:tasks.workunit.client.0.smithi139.stdout:5/910: readlink d1/d13/d3f/d9f/d73/d79/da5/lbc 0 2024-09-16T00:09:52.733 INFO:tasks.workunit.client.0.smithi139.stdout:4/924: fdatasync d12/d2b/f30 0 2024-09-16T00:09:52.733 INFO:tasks.workunit.client.0.smithi139.stdout:4/925: chown d12/d47/db2/f100 1 1 2024-09-16T00:09:52.733 INFO:tasks.workunit.client.0.smithi139.stdout:0/842: mknod d6/d1c/d41/d63/c108 0 2024-09-16T00:09:52.733 INFO:tasks.workunit.client.0.smithi139.stdout:7/883: mkdir d5/d26/d74/dfd/d137 0 2024-09-16T00:09:52.733 INFO:tasks.workunit.client.0.smithi139.stdout:4/926: truncate d12/d47/db2/d4d/dd1/fef 1004276 0 2024-09-16T00:09:52.733 INFO:tasks.workunit.client.0.smithi139.stdout:0/843: truncate d6/d1c/d41/d63/d82/f99 656445 0 2024-09-16T00:09:52.733 INFO:tasks.workunit.client.0.smithi139.stdout:5/911: dread d1/d13/d3b/f4c [0,4194304] 0 2024-09-16T00:09:52.733 INFO:tasks.workunit.client.0.smithi139.stdout:2/991: dread d11/d30/d110/d5d/d73/fc7 [0,4194304] 0 2024-09-16T00:09:52.733 INFO:tasks.workunit.client.0.smithi139.stdout:7/884: rmdir d5/d16/d6d/d7f/dc2 39 2024-09-16T00:09:52.733 INFO:tasks.workunit.client.0.smithi139.stdout:7/885: fsync d5/f59 0 2024-09-16T00:09:52.733 INFO:tasks.workunit.client.0.smithi139.stdout:0/844: mknod d6/d1c/d41/d44/d6b/c109 0 2024-09-16T00:09:52.733 INFO:tasks.workunit.client.0.smithi139.stdout:2/992: write d11/d30/dce/fe2 [1096896,51137] 0 2024-09-16T00:09:52.734 INFO:tasks.workunit.client.0.smithi139.stdout:7/886: fsync d5/d16/d22/d25/d63/f97 0 2024-09-16T00:09:52.734 INFO:tasks.workunit.client.0.smithi139.stdout:4/927: creat d12/d47/d49/f133 x:0 0 0 2024-09-16T00:09:52.734 INFO:tasks.workunit.client.0.smithi139.stdout:8/983: sync 2024-09-16T00:09:52.734 INFO:tasks.workunit.client.0.smithi139.stdout:0/845: truncate d6/d1c/d41/d44/dd/d8a/d9e/fdc 295001 0 2024-09-16T00:09:52.734 INFO:tasks.workunit.client.0.smithi139.stdout:8/984: unlink d17/d1f/d3a/db7/ce6 0 2024-09-16T00:09:52.734 INFO:tasks.workunit.client.0.smithi139.stdout:8/985: truncate d17/ff8 5110433 0 2024-09-16T00:09:52.734 INFO:tasks.workunit.client.0.smithi139.stdout:5/912: truncate d1/d21/d49/f7f 2251188 0 2024-09-16T00:09:52.734 INFO:tasks.workunit.client.0.smithi139.stdout:8/986: symlink d17/d1a/d50/d86/d92/l14b 0 2024-09-16T00:09:52.734 INFO:tasks.workunit.client.0.smithi139.stdout:7/887: rename d5/c52 to d5/d16/d6d/d7f/dc2/d50/d44/d55/d90/c138 0 2024-09-16T00:09:52.734 INFO:tasks.workunit.client.0.smithi139.stdout:0/846: dwrite d6/d1c/d41/d44/dd/f40 [0,4194304] 0 2024-09-16T00:09:52.734 INFO:tasks.workunit.client.0.smithi139.stdout:4/928: getdents d12/d47/d49/d60/dac 0 2024-09-16T00:09:52.734 INFO:tasks.workunit.client.0.smithi139.stdout:0/847: fsync d6/f52 0 2024-09-16T00:09:52.734 INFO:tasks.workunit.client.0.smithi139.stdout:2/993: dread d11/d30/dbf/d116/d125/f129 [0,4194304] 0 2024-09-16T00:09:52.734 INFO:tasks.workunit.client.0.smithi139.stdout:8/987: readlink d17/d1a/d2d/l8a 0 2024-09-16T00:09:52.734 INFO:tasks.workunit.client.0.smithi139.stdout:8/988: write d17/d54/fed [289943,93436] 0 2024-09-16T00:09:52.735 INFO:tasks.workunit.client.0.smithi139.stdout:8/989: symlink d17/d1f/d3a/db0/l14c 0 2024-09-16T00:09:52.735 INFO:tasks.workunit.client.0.smithi139.stdout:8/990: fdatasync d17/f13a 0 2024-09-16T00:09:52.735 INFO:tasks.workunit.client.0.smithi139.stdout:2/994: mknod d11/d30/d110/d29/da5/c14d 0 2024-09-16T00:09:52.735 INFO:tasks.workunit.client.0.smithi139.stdout:5/913: dread d1/d21/d49/d87/f11c [0,4194304] 0 2024-09-16T00:09:52.735 INFO:tasks.workunit.client.0.smithi139.stdout:7/888: rename d5/fbc to d5/d16/f139 0 2024-09-16T00:09:52.735 INFO:tasks.workunit.client.0.smithi139.stdout:8/991: symlink d17/d1f/d3a/dcd/l14d 0 2024-09-16T00:09:52.735 INFO:tasks.workunit.client.0.smithi139.stdout:2/995: mkdir d11/d2f/deb/d14e 0 2024-09-16T00:09:52.735 INFO:tasks.workunit.client.0.smithi139.stdout:7/889: creat d5/d26/d46/d6b/dd6/f13a x:0 0 0 2024-09-16T00:09:52.735 INFO:tasks.workunit.client.0.smithi139.stdout:5/914: creat d1/d21/d49/d53/d6a/d7b/d99/d11b/f141 x:0 0 0 2024-09-16T00:09:52.735 INFO:tasks.workunit.client.0.smithi139.stdout:5/915: dread - d1/d13/d3f/d9f/f11f zero size 2024-09-16T00:09:52.735 INFO:tasks.workunit.client.0.smithi139.stdout:2/996: mkdir d11/d30/d110/d29/d2b/d14f 0 2024-09-16T00:09:52.735 INFO:tasks.workunit.client.0.smithi139.stdout:8/992: getdents d17/d1f/d13e 0 2024-09-16T00:09:52.735 INFO:tasks.workunit.client.0.smithi139.stdout:5/916: unlink d1/d13/d3f/d9f/d73/d79/d8d/f9e 0 2024-09-16T00:09:52.735 INFO:tasks.workunit.client.0.smithi139.stdout:2/997: chown d11/d30/d110/d29/d4f/d9c/fca 25471576 1 2024-09-16T00:09:52.736 INFO:tasks.workunit.client.0.smithi139.stdout:4/929: dread d12/d47/d49/f7e [0,4194304] 0 2024-09-16T00:09:52.736 INFO:tasks.workunit.client.0.smithi139.stdout:8/993: write d17/d1f/d3a/db7/d40/d4f/f114 [1270240,60224] 0 2024-09-16T00:09:52.736 INFO:tasks.workunit.client.0.smithi139.stdout:0/848: write d6/d1c/d41/d63/f66 [198750,49100] 0 2024-09-16T00:09:52.736 INFO:tasks.workunit.client.0.smithi139.stdout:4/930: dread - d12/d2b/d8c/dff/f102 zero size 2024-09-16T00:09:52.736 INFO:tasks.workunit.client.0.smithi139.stdout:8/994: creat d17/d1a/d50/d86/d9d/d9f/d101/f14e x:0 0 0 2024-09-16T00:09:52.782 INFO:tasks.workunit.client.0.smithi139.stdout:5/917: creat d1/d13/d3f/d9f/d73/f142 x:0 0 0 2024-09-16T00:09:52.784 INFO:tasks.workunit.client.0.smithi139.stdout:4/931: dwrite d12/f25 [4194304,4194304] 0 2024-09-16T00:09:52.784 INFO:tasks.workunit.client.0.smithi139.stdout:3/685: sync 2024-09-16T00:09:52.784 INFO:tasks.workunit.client.0.smithi139.stdout:1/706: sync 2024-09-16T00:09:52.784 INFO:tasks.workunit.client.0.smithi139.stdout:6/718: sync 2024-09-16T00:09:52.785 INFO:tasks.workunit.client.0.smithi139.stdout:8/995: read d17/d1f/d3a/db7/d40/d4f/f75 [3106515,60844] 0 2024-09-16T00:09:52.788 INFO:tasks.workunit.client.0.smithi139.stdout:5/918: symlink d1/d21/l143 0 2024-09-16T00:09:52.793 INFO:tasks.workunit.client.0.smithi139.stdout:1/707: symlink d7/d8/d40/d48/d67/d7d/dc3/lce 0 2024-09-16T00:09:52.795 INFO:tasks.workunit.client.0.smithi139.stdout:0/849: dwrite d6/d1c/f59 [0,4194304] 0 2024-09-16T00:09:52.799 INFO:tasks.workunit.client.0.smithi139.stdout:8/996: truncate d17/d54/fbe 833549 0 2024-09-16T00:09:52.805 INFO:tasks.workunit.client.0.smithi139.stdout:4/932: dwrite d12/f4f [0,4194304] 0 2024-09-16T00:09:52.807 INFO:tasks.workunit.client.0.smithi139.stdout:3/686: dwrite d5/d7/d23/d27/d2f/d30/d41/f7f [0,4194304] 0 2024-09-16T00:09:52.812 INFO:tasks.workunit.client.0.smithi139.stdout:8/997: write d17/d1a/d50/dee/f141 [4729238,45591] 0 2024-09-16T00:09:52.813 INFO:tasks.workunit.client.0.smithi139.stdout:6/719: dwrite d8/d29/d83/f15 [0,4194304] 0 2024-09-16T00:09:52.814 INFO:tasks.workunit.client.0.smithi139.stdout:0/850: truncate d6/ff 63571 0 2024-09-16T00:09:52.815 INFO:tasks.workunit.client.0.smithi139.stdout:4/933: rename d12/d47/d49/d60/fea to d12/d2b/d8c/f134 0 2024-09-16T00:09:52.817 INFO:tasks.workunit.client.0.smithi139.stdout:3/687: rmdir d5/d7/d23/d27/d2f/d30/d4c/d6d/db3 39 2024-09-16T00:09:52.818 INFO:tasks.workunit.client.0.smithi139.stdout:5/919: dwrite d1/d13/d3f/d9f/d73/d79/da5/d10b/d11e/d10d/fea [0,4194304] 0 2024-09-16T00:09:52.821 INFO:tasks.workunit.client.0.smithi139.stdout:1/708: link d7/d8/d40/d48/d67/d7d/dc3/lce d7/d8/d40/d48/d67/dbf/lcf 0 2024-09-16T00:09:52.821 INFO:tasks.workunit.client.0.smithi139.stdout:4/934: unlink d12/d47/db2/d4d/de4/l71 0 2024-09-16T00:09:52.823 INFO:tasks.workunit.client.0.smithi139.stdout:3/688: write d5/d7/d23/d27/d2f/d30/d41/f7f [3160108,106362] 0 2024-09-16T00:09:52.825 INFO:tasks.workunit.client.0.smithi139.stdout:6/720: mkdir d8/d29/d83/d26/d45/d42/dc0/dff 0 2024-09-16T00:09:52.825 INFO:tasks.workunit.client.0.smithi139.stdout:5/920: write d1/d13/d3f/d9f/d73/d79/da5/d10b/d11e/d10d/d33/f122 [706967,8944] 0 2024-09-16T00:09:52.826 INFO:tasks.workunit.client.0.smithi139.stdout:4/935: fdatasync d12/d47/d49/ffc 0 2024-09-16T00:09:52.826 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:52.826+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:52.827 INFO:tasks.workunit.client.0.smithi139.stdout:3/689: chown d5/d7/d23/d27/d2f/d30/d4c/dbb 1938 1 2024-09-16T00:09:52.828 INFO:tasks.workunit.client.0.smithi139.stdout:1/709: write d7/d8/d27/d68/d7c/d83/fca [813531,100775] 0 2024-09-16T00:09:52.831 INFO:tasks.workunit.client.0.smithi139.stdout:0/851: link d6/d1c/d41/d44/dd/d56/d75/cc0 d6/d1c/d41/d44/dd/d8a/d9e/dcb/c10a 0 2024-09-16T00:09:52.833 INFO:tasks.workunit.client.0.smithi139.stdout:4/936: creat d12/d47/db2/d4d/dd6/f135 x:0 0 0 2024-09-16T00:09:52.834 INFO:tasks.workunit.client.0.smithi139.stdout:5/921: symlink d1/d21/d49/d53/d6a/d7b/l144 0 2024-09-16T00:09:52.835 INFO:tasks.workunit.client.0.smithi139.stdout:1/710: creat d7/d8/d40/d48/d4a/db8/fd0 x:0 0 0 2024-09-16T00:09:52.836 INFO:tasks.workunit.client.0.smithi139.stdout:0/852: chown d6/d1c/d41/d44/dd/d56/d75/fcf 844 1 2024-09-16T00:09:52.838 INFO:tasks.workunit.client.0.smithi139.stdout:8/998: dwrite d17/d1f/d3a/f45 [0,4194304] 0 2024-09-16T00:09:52.839 INFO:tasks.workunit.client.0.smithi139.stdout:5/922: mkdir d1/d13/d22/da9/d145 0 2024-09-16T00:09:52.842 INFO:tasks.workunit.client.0.smithi139.stdout:0/853: write d6/d5a/d70/fc8 [557063,80754] 0 2024-09-16T00:09:52.843 INFO:tasks.workunit.client.0.smithi139.stdout:5/923: readlink d1/d13/d3b/d3e/l65 0 2024-09-16T00:09:52.845 INFO:tasks.workunit.client.0.smithi139.stdout:6/721: dwrite d8/d29/f3d [0,4194304] 0 2024-09-16T00:09:52.845 INFO:tasks.workunit.client.0.smithi139.stdout:8/999: creat d17/d1a/d50/d86/d9d/f14f x:0 0 0 2024-09-16T00:09:52.845 INFO:tasks.workunit.client.0.smithi139.stdout:1/711: symlink d7/d8/d27/d68/ld1 0 2024-09-16T00:09:52.847 INFO:tasks.workunit.client.0.smithi139.stdout:5/924: chown d1/dc3/f12e 4156 1 2024-09-16T00:09:52.847 INFO:tasks.workunit.client.0.smithi139.stdout:0/854: write d6/d1c/d41/d44/dce/fd8 [1869147,68546] 0 2024-09-16T00:09:52.849 INFO:tasks.workunit.client.0.smithi139.stdout:3/690: dwrite d5/d7/d23/fe0 [0,4194304] 0 2024-09-16T00:09:52.851 INFO:tasks.workunit.client.0.smithi139.stdout:5/925: creat d1/d13/d94/f146 x:0 0 0 2024-09-16T00:09:52.851 INFO:tasks.workunit.client.0.smithi139.stdout:6/722: mknod d8/d29/d83/d26/d28/d6d/c100 0 2024-09-16T00:09:52.853 INFO:tasks.workunit.client.0.smithi139.stdout:1/712: mknod d7/d8/d40/d46/d8d/cd2 0 2024-09-16T00:09:52.854 INFO:tasks.workunit.client.0.smithi139.stdout:0/855: write d6/d5a/d70/f4f [1221178,3989] 0 2024-09-16T00:09:52.855 INFO:tasks.workunit.client.0.smithi139.stdout:5/926: fsync d1/f48 0 2024-09-16T00:09:52.857 INFO:tasks.workunit.client.0.smithi139.stdout:3/691: creat d5/d5f/fe7 x:0 0 0 2024-09-16T00:09:52.858 INFO:tasks.workunit.client.0.smithi139.stdout:1/713: mkdir d7/d8/d40/dd3 0 2024-09-16T00:09:52.863 INFO:tasks.workunit.client.0.smithi139.stdout:0/856: creat d6/d1c/d41/d44/dd/d8a/d9e/dcb/f10b x:0 0 0 2024-09-16T00:09:52.864 INFO:tasks.workunit.client.0.smithi139.stdout:1/714: rename d7/d8/d27/d2f to d7/d8/d40/d48/d53/d61/dd4 0 2024-09-16T00:09:52.865 INFO:tasks.workunit.client.0.smithi139.stdout:5/927: dwrite d1/d21/d49/d87/f11c [0,4194304] 0 2024-09-16T00:09:52.866 INFO:tasks.workunit.client.0.smithi139.stdout:0/857: unlink d6/d42/db4/c5d 0 2024-09-16T00:09:52.866 INFO:tasks.workunit.client.0.smithi139.stdout:3/692: write d5/d7/d23/de4/fcd [39859,51925] 0 2024-09-16T00:09:52.868 INFO:tasks.workunit.client.0.smithi139.stdout:1/715: creat d7/d8/d40/d48/d67/dbf/fd5 x:0 0 0 2024-09-16T00:09:52.868 INFO:tasks.workunit.client.0.smithi139.stdout:6/723: dread d8/d20/d4d/f60 [0,4194304] 0 2024-09-16T00:09:52.870 INFO:tasks.workunit.client.0.smithi139.stdout:0/858: creat d6/d5a/de7/df1/f10c x:0 0 0 2024-09-16T00:09:52.871 INFO:tasks.workunit.client.0.smithi139.stdout:1/716: mknod d7/d8/d40/d48/d4a/db8/cd6 0 2024-09-16T00:09:52.873 INFO:tasks.workunit.client.0.smithi139.stdout:3/693: rename d5/d7/d23/d35 to d5/d7/d23/de4/de8 0 2024-09-16T00:09:52.874 INFO:tasks.workunit.client.0.smithi139.stdout:0/859: rmdir d6/d1c/d41/d7c 39 2024-09-16T00:09:52.876 INFO:tasks.workunit.client.0.smithi139.stdout:6/724: unlink d8/d29/d83/d26/d45/d65/f3b 0 2024-09-16T00:09:52.877 INFO:tasks.workunit.client.0.smithi139.stdout:1/717: unlink d7/d8/d1f/fa6 0 2024-09-16T00:09:52.880 INFO:tasks.workunit.client.0.smithi139.stdout:5/928: dwrite d1/f4a [0,4194304] 0 2024-09-16T00:09:52.880 INFO:tasks.workunit.client.0.smithi139.stdout:6/725: dread d8/d20/d4d/db0/fd6 [0,4194304] 0 2024-09-16T00:09:52.882 INFO:tasks.workunit.client.0.smithi139.stdout:0/860: dread d6/d1c/d41/d7c/d86/f93 [0,4194304] 0 2024-09-16T00:09:52.883 INFO:tasks.workunit.client.0.smithi139.stdout:3/694: rename d5/d7/d23/d27/d2f/d30/d41 to d5/d7/d23/d27/d2f/d30/d41/d68/db9/de9 22 2024-09-16T00:09:52.886 INFO:tasks.workunit.client.0.smithi139.stdout:1/718: fsync d7/d8/d1f/f37 0 2024-09-16T00:09:52.889 INFO:tasks.workunit.client.0.smithi139.stdout:6/726: symlink d8/d20/d68/df4/l101 0 2024-09-16T00:09:52.890 INFO:tasks.workunit.client.0.smithi139.stdout:1/719: dread d7/d8/d27/fb0 [0,4194304] 0 2024-09-16T00:09:52.891 INFO:tasks.workunit.client.0.smithi139.stdout:5/929: link d1/fe9 d1/d21/f147 0 2024-09-16T00:09:52.893 INFO:tasks.workunit.client.0.smithi139.stdout:5/930: truncate d1/d21/f147 4650765 0 2024-09-16T00:09:52.896 INFO:tasks.workunit.client.0.smithi139.stdout:5/931: readlink d1/d13/d3f/d9f/d73/d79/da5/d10b/d11e/d10d/dd4/lfd 0 2024-09-16T00:09:52.897 INFO:tasks.workunit.client.0.smithi139.stdout:1/720: link d7/d8/d27/d68/d7c/d83/fca d7/d8/d40/d48/d4a/d82/fd7 0 2024-09-16T00:09:52.898 INFO:tasks.workunit.client.0.smithi139.stdout:0/861: truncate d6/d1c/d41/d63/f66 64128 0 2024-09-16T00:09:52.901 INFO:tasks.workunit.client.0.smithi139.stdout:3/695: dwrite d5/d7/f13 [0,4194304] 0 2024-09-16T00:09:52.901 INFO:tasks.workunit.client.0.smithi139.stdout:1/721: fsync d7/d8/d1f/f38 0 2024-09-16T00:09:52.903 INFO:tasks.workunit.client.0.smithi139.stdout:5/932: symlink d1/d13/d3f/d9f/d73/d93/l148 0 2024-09-16T00:09:52.903 INFO:tasks.workunit.client.0.smithi139.stdout:6/727: dread d8/d56/f79 [0,4194304] 0 2024-09-16T00:09:52.903 INFO:tasks.workunit.client.0.smithi139.stdout:1/722: stat d7/d8/d40/l62 0 2024-09-16T00:09:52.905 INFO:tasks.workunit.client.0.smithi139.stdout:0/862: symlink d6/d1c/d41/d44/l10d 0 2024-09-16T00:09:52.906 INFO:tasks.workunit.client.0.smithi139.stdout:3/696: write d5/d7/f37 [4312301,12207] 0 2024-09-16T00:09:52.907 INFO:tasks.workunit.client.0.smithi139.stdout:5/933: unlink d1/d21/d49/d53/d6a/d7b/f12d 0 2024-09-16T00:09:52.910 INFO:tasks.workunit.client.0.smithi139.stdout:5/934: write d1/d21/d49/d87/f11c [157789,89995] 0 2024-09-16T00:09:52.911 INFO:tasks.workunit.client.0.smithi139.stdout:0/863: write d6/d1c/d41/d44/fae [609483,18156] 0 2024-09-16T00:09:52.913 INFO:tasks.workunit.client.0.smithi139.stdout:0/864: rmdir d6/d5a/de7 39 2024-09-16T00:09:52.916 INFO:tasks.workunit.client.0.smithi139.stdout:1/723: dwrite d7/d8/d1f/f38 [0,4194304] 0 2024-09-16T00:09:52.917 INFO:tasks.workunit.client.0.smithi139.stdout:0/865: write d6/d1c/d41/f96 [108104,53167] 0 2024-09-16T00:09:52.917 INFO:tasks.workunit.client.0.smithi139.stdout:6/728: dwrite d8/d20/d27/f97 [0,4194304] 0 2024-09-16T00:09:52.920 INFO:tasks.workunit.client.0.smithi139.stdout:6/729: write d8/d20/d27/f97 [3398173,63367] 0 2024-09-16T00:09:52.921 INFO:tasks.workunit.client.0.smithi139.stdout:5/935: rename d1/d21/d49/d53/d6a/d7b/d99/f120 to d1/d21/d49/d53/d6a/d7b/d99/d11b/f149 0 2024-09-16T00:09:52.924 INFO:tasks.workunit.client.0.smithi139.stdout:5/936: write d1/dc3/f12e [84035,63768] 0 2024-09-16T00:09:52.925 INFO:tasks.workunit.client.0.smithi139.stdout:1/724: getdents d7/d8/d40/d48/dcc 0 2024-09-16T00:09:52.926 INFO:tasks.workunit.client.0.smithi139.stdout:6/730: dread d8/d29/f36 [0,4194304] 0 2024-09-16T00:09:52.926 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:52.926+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:52.927 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:52.927 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:52.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.927 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14715 3.3b 3:de17bf99:::1000000335d.00000000:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:52.928 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.918859+0000", 2024-09-16T00:09:52.928 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0082822599999999996, 2024-09-16T00:09:52.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0083124670000000005, 2024-09-16T00:09:52.928 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.928 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.928 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "waiting for sub ops", 2024-09-16T00:09:52.928 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.928 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.928 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.928 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14715 2024-09-16T00:09:52.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.928 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.918859+0000", 2024-09-16T00:09:52.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.918859+0000", 2024-09-16T00:09:52.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.918862+0000", 2024-09-16T00:09:52.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5030000000000002e-06 2024-09-16T00:09:52.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921460+0000", 2024-09-16T00:09:52.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025977579999999999 2024-09-16T00:09:52.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921479+0000", 2024-09-16T00:09:52.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9196000000000001e-05 2024-09-16T00:09:52.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921488+0000", 2024-09-16T00:09:52.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2900000000000002e-06 2024-09-16T00:09:52.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921580+0000", 2024-09-16T00:09:52.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2539000000000003e-05 2024-09-16T00:09:52.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921678+0000", 2024-09-16T00:09:52.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8288999999999994e-05 2024-09-16T00:09:52.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:52.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921839+0000", 2024-09-16T00:09:52.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000160469 2024-09-16T00:09:52.932 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.932 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.932 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.932 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14717 3.3b 3:de17bf99:::1000000335d.00000000:head [write 3398173~63367 in=63367b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:52.932 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.921511+0000", 2024-09-16T00:09:52.932 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0056298420000000004, 2024-09-16T00:09:52.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005747416, 2024-09-16T00:09:52.932 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.932 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.932 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "waiting for sub ops", 2024-09-16T00:09:52.932 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:52.933 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:52.933 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:52.933 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14717 2024-09-16T00:09:52.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.933 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921511+0000", 2024-09-16T00:09:52.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921511+0000", 2024-09-16T00:09:52.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.934 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921513+0000", 2024-09-16T00:09:52.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7320000000000001e-06 2024-09-16T00:09:52.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.934 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921536+0000", 2024-09-16T00:09:52.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2751e-05 2024-09-16T00:09:52.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.934 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921544+0000", 2024-09-16T00:09:52.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4430000000000003e-06 2024-09-16T00:09:52.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921550+0000", 2024-09-16T00:09:52.935 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0650000000000004e-06 2024-09-16T00:09:52.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.926887+0000", 2024-09-16T00:09:52.935 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005336426 2024-09-16T00:09:52.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.936 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.926980+0000", 2024-09-16T00:09:52.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2984000000000003e-05 2024-09-16T00:09:52.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.936 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:52.936 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.927129+0000", 2024-09-16T00:09:52.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014929900000000001 2024-09-16T00:09:52.936 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.936 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.936 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.936 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3061 2.d 2:b02ffd62:::200.00000010:head [write 2610387~18796 [fadvise_dontneed] in=18796b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:09:52.936 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.922968+0000", 2024-09-16T00:09:52.937 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.0041732799999999997, 2024-09-16T00:09:52.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0043700739999999998, 2024-09-16T00:09:52.937 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:52.937 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:52.937 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "waiting for sub ops", 2024-09-16T00:09:52.937 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:52.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:52.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.922968+0000", 2024-09-16T00:09:52.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:52.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.922968+0000", 2024-09-16T00:09:52.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:52.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:52.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.922971+0000", 2024-09-16T00:09:52.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0510000000000001e-06 2024-09-16T00:09:52.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:52.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.923000+0000", 2024-09-16T00:09:52.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9428e-05 2024-09-16T00:09:52.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:52.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.923021+0000", 2024-09-16T00:09:52.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0322e-05 2024-09-16T00:09:52.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:52.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.923036+0000", 2024-09-16T00:09:52.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4999e-05 2024-09-16T00:09:52.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:52.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.923124+0000", 2024-09-16T00:09:52.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7967000000000007e-05 2024-09-16T00:09:52.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:52.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.923253+0000", 2024-09-16T00:09:52.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012960599999999999 2024-09-16T00:09:52.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:52.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.923417+0000", 2024-09-16T00:09:52.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016368700000000001 2024-09-16T00:09:52.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:52.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:52.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:52.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.925336+0000", 2024-09-16T00:09:52.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019189330000000001 2024-09-16T00:09:52.941 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.941 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:52.941 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.941 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:52.941 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:09:52.941 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 3 2024-09-16T00:09:52.941 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:52.942 INFO:tasks.workunit.client.0.smithi139.stdout:5/937: creat d1/d13/d22/dd5/f14a x:0 0 0 2024-09-16T00:09:52.942 INFO:tasks.workunit.client.0.smithi139.stdout:1/725: creat d7/d8/d40/d48/d53/d61/dd4/fd8 x:0 0 0 2024-09-16T00:09:52.942 INFO:tasks.workunit.client.0.smithi139.stdout:0/866: write d6/d1c/d41/d63/d82/f83 [981107,27811] 0 2024-09-16T00:09:52.942 INFO:tasks.workunit.client.0.smithi139.stdout:6/731: creat d8/d29/d83/d26/d28/d5a/f102 x:0 0 0 2024-09-16T00:09:52.942 INFO:tasks.workunit.client.0.smithi139.stdout:3/697: dread d5/d7/d23/d27/d2f/d30/d47/f64 [0,4194304] 0 2024-09-16T00:09:52.942 INFO:tasks.workunit.client.0.smithi139.stdout:0/867: rename d6/d5a/de7 to d6/d1c/d41/d78/df2/d10e 0 2024-09-16T00:09:52.942 INFO:tasks.workunit.client.0.smithi139.stdout:5/938: dwrite d1/d21/d49/d53/d6a/d7b/f7e [0,4194304] 0 2024-09-16T00:09:52.943 INFO:tasks.workunit.client.0.smithi139.stdout:6/732: creat d8/d29/d83/d26/d28/d6d/de7/f103 x:0 0 0 2024-09-16T00:09:52.943 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:09:52.944 INFO:tasks.workunit.client.0.smithi139.stdout:3/698: read d5/d7/dc/f63 [527947,83124] 0 2024-09-16T00:09:52.944 INFO:tasks.workunit.client.0.smithi139.stdout:5/939: creat d1/d13/d3f/d5e/f14b x:0 0 0 2024-09-16T00:09:52.944 INFO:tasks.workunit.client.0.smithi139.stdout:1/726: dwrite d7/d8/d1f/fc1 [0,4194304] 0 2024-09-16T00:09:52.945 INFO:tasks.workunit.client.0.smithi139.stdout:0/868: mknod d6/d1c/d41/d44/dd/d56/dbd/c10f 0 2024-09-16T00:09:52.947 INFO:tasks.workunit.client.0.smithi139.stdout:5/940: read d1/f1a [1679397,127854] 0 2024-09-16T00:09:52.947 INFO:tasks.workunit.client.0.smithi139.stdout:6/733: truncate d8/d29/d83/d26/d28/f95 1049725 0 2024-09-16T00:09:52.947 INFO:tasks.workunit.client.0.smithi139.stdout:3/699: symlink d5/d7/d23/d27/d2f/d30/d4c/dbb/lea 0 2024-09-16T00:09:52.947 INFO:tasks.workunit.client.0.smithi139.stdout:5/941: chown d1/d21/f123 98059 1 2024-09-16T00:09:52.947 INFO:tasks.workunit.client.0.smithi139.stdout:1/727: mknod d7/d8/d40/d48/d4a/db8/cd9 0 2024-09-16T00:09:52.948 INFO:tasks.workunit.client.0.smithi139.stdout:0/869: creat d6/d1c/d41/d44/dce/f110 x:0 0 0 2024-09-16T00:09:52.949 INFO:tasks.workunit.client.0.smithi139.stdout:1/728: chown d7/d1b/fa5 371804 1 2024-09-16T00:09:52.949 INFO:tasks.workunit.client.0.smithi139.stdout:6/734: creat d8/d29/d83/d26/d28/d5a/f104 x:0 0 0 2024-09-16T00:09:52.949 INFO:tasks.workunit.client.0.smithi139.stdout:0/870: readlink d6/d1c/d41/d63/l80 0 2024-09-16T00:09:52.952 INFO:tasks.workunit.client.0.smithi139.stdout:5/942: creat d1/d13/d3f/d9f/d73/d79/da5/d10b/d11e/f14c x:0 0 0 2024-09-16T00:09:52.953 INFO:tasks.workunit.client.0.smithi139.stdout:3/700: creat d5/d7/d23/d27/d2f/d30/d41/d68/db9/feb x:0 0 0 2024-09-16T00:09:52.954 INFO:tasks.workunit.client.0.smithi139.stdout:1/729: mkdir d7/d8/d40/d48/dcc/dda 0 2024-09-16T00:09:52.955 INFO:tasks.workunit.client.0.smithi139.stdout:6/735: getdents d8/d56 0 2024-09-16T00:09:52.958 INFO:tasks.workunit.client.0.smithi139.stdout:3/701: write d5/d7/d23/d27/d2f/d30/d4d/dbc/fce [1011898,86363] 0 2024-09-16T00:09:52.959 INFO:tasks.workunit.client.0.smithi139.stdout:0/871: dwrite d6/d1c/d41/d44/dd/fd0 [0,4194304] 0 2024-09-16T00:09:52.959 INFO:tasks.workunit.client.0.smithi139.stdout:6/736: mkdir d8/da4/d105 0 2024-09-16T00:09:52.959 INFO:tasks.workunit.client.0.smithi139.stdout:7/890: sync 2024-09-16T00:09:52.961 INFO:tasks.workunit.client.0.smithi139.stdout:0/872: stat d6/d1c/f61 0 2024-09-16T00:09:52.963 INFO:tasks.workunit.client.0.smithi139.stdout:6/737: unlink d8/cf6 0 2024-09-16T00:09:52.963 INFO:tasks.workunit.client.0.smithi139.stdout:3/702: mkdir d5/d7/dc2/dec 0 2024-09-16T00:09:52.966 INFO:tasks.workunit.client.0.smithi139.stdout:5/943: rename d1/d13/d3f/d9f/d73 to d1/d13/d3f/d14d 0 2024-09-16T00:09:52.968 INFO:tasks.workunit.client.0.smithi139.stdout:1/730: dwrite d7/d8/d1f/f38 [4194304,4194304] 0 2024-09-16T00:09:52.968 INFO:tasks.workunit.client.0.smithi139.stdout:0/873: creat d6/d1c/d41/d44/dd/d56/d75/f111 x:0 0 0 2024-09-16T00:09:52.969 INFO:tasks.workunit.client.0.smithi139.stdout:5/944: chown d1/d21/d49/d53/d6a/d7b/l129 170 1 2024-09-16T00:09:52.971 INFO:tasks.workunit.client.0.smithi139.stdout:5/945: fsync d1/d13/d3f/d9f/f5c 0 2024-09-16T00:09:52.971 INFO:tasks.workunit.client.0.smithi139.stdout:7/891: rename d5/d26/l5c to d5/d26/d74/dfd/l13b 0 2024-09-16T00:09:52.972 INFO:tasks.workunit.client.0.smithi139.stdout:3/703: symlink d5/d7/d23/d27/d2f/d30/d4c/dbb/de2/led 0 2024-09-16T00:09:52.972 INFO:tasks.workunit.client.0.smithi139.stdout:0/874: symlink d6/d5a/d70/d7b/l112 0 2024-09-16T00:09:52.975 INFO:tasks.workunit.client.0.smithi139.stdout:6/738: rename d8/d29/d83/d26/f66 to d8/d20/d4d/db0/f106 0 2024-09-16T00:09:52.975 INFO:tasks.workunit.client.0.smithi139.stdout:7/892: chown d5/d26/d46/d6b/dd6/df4/ff7 108 1 2024-09-16T00:09:52.976 INFO:tasks.workunit.client.0.smithi139.stdout:0/875: stat d6/d1c/d41/d63/f66 0 2024-09-16T00:09:52.978 INFO:tasks.workunit.client.0.smithi139.stdout:3/704: fsync d5/d7/d23/d27/d2f/d30/d47/f4a 0 2024-09-16T00:09:52.980 INFO:tasks.workunit.client.0.smithi139.stdout:7/893: creat d5/d26/d46/d6b/dd6/f13c x:0 0 0 2024-09-16T00:09:52.980 INFO:tasks.workunit.client.0.smithi139.stdout:5/946: dwrite d1/d21/d49/d87/f11c [0,4194304] 0 2024-09-16T00:09:52.985 INFO:tasks.workunit.client.0.smithi139.stdout:0/876: getdents d6/d1c/d41/d44/dd/d56/d75/de1 0 2024-09-16T00:09:52.985 INFO:tasks.workunit.client.0.smithi139.stdout:7/894: fsync d5/d37/f47 0 2024-09-16T00:09:52.985 INFO:tasks.workunit.client.0.smithi139.stdout:6/739: truncate d8/d29/d83/d26/d45/d59/fb8 478389 0 2024-09-16T00:09:52.985 INFO:tasks.workunit.client.0.smithi139.stdout:5/947: chown d1/d13/d3f/d88/cec 2 1 2024-09-16T00:09:52.985 INFO:tasks.workunit.client.0.smithi139.stdout:3/705: creat d5/d7/d23/d27/d2f/d30/d4c/dbb/fee x:0 0 0 2024-09-16T00:09:52.985 INFO:tasks.workunit.client.0.smithi139.stdout:1/731: truncate d7/d8/d40/d48/d53/d61/dd4/f5c 4571887 0 2024-09-16T00:09:52.986 INFO:tasks.workunit.client.0.smithi139.stdout:6/740: chown d8/d29/f36 9 1 2024-09-16T00:09:52.986 INFO:tasks.workunit.client.0.smithi139.stdout:7/895: rename d5/d16/f139 to d5/d26/d74/dfd/f13d 0 2024-09-16T00:09:52.987 INFO:tasks.workunit.client.0.smithi139.stdout:5/948: write d1/d13/d3b/d3e/f112 [667936,38402] 0 2024-09-16T00:09:52.990 INFO:tasks.workunit.client.0.smithi139.stdout:6/741: fsync d8/d29/d83/d26/d28/d6d/dd3/de5/ff8 0 2024-09-16T00:09:52.991 INFO:tasks.workunit.client.0.smithi139.stdout:3/706: symlink d5/d7/d23/d27/d2f/d30/d4c/dbb/lef 0 2024-09-16T00:09:52.992 INFO:tasks.workunit.client.0.smithi139.stdout:0/877: write d6/d1c/d41/d44/d58/f60 [1870157,116369] 0 2024-09-16T00:09:52.994 INFO:tasks.workunit.client.0.smithi139.stdout:3/707: readlink d5/d7/d23/l22 0 2024-09-16T00:09:52.995 INFO:tasks.workunit.client.0.smithi139.stdout:0/878: read f3 [4149086,97727] 0 2024-09-16T00:09:52.995 INFO:tasks.workunit.client.0.smithi139.stdout:1/732: rename d7/d8/d40/d48/d53/d61/dd4 to d7/d8/d40/d48/d4a/d74/ddb 0 2024-09-16T00:09:52.997 INFO:tasks.workunit.client.0.smithi139.stdout:6/742: mkdir d8/d29/d83/d26/d45/d42/dc0/dff/d107 0 2024-09-16T00:09:52.998 INFO:tasks.workunit.client.0.smithi139.stdout:7/896: dwrite d5/d26/d46/f68 [0,4194304] 0 2024-09-16T00:09:52.999 INFO:tasks.workunit.client.0.smithi139.stdout:1/733: stat d7/d8/d27/d68/d6c/f7a 0 2024-09-16T00:09:53.001 INFO:tasks.workunit.client.0.smithi139.stdout:0/879: mknod d6/d1c/c113 0 2024-09-16T00:09:53.001 INFO:tasks.workunit.client.0.smithi139.stdout:0/880: readlink d6/d5a/d81/la7 0 2024-09-16T00:09:53.002 INFO:tasks.workunit.client.0.smithi139.stdout:7/897: creat d5/d37/d70/f13e x:0 0 0 2024-09-16T00:09:53.003 INFO:tasks.workunit.client.0.smithi139.stdout:5/949: truncate d1/d21/d49/d53/d6a/d7b/f7e 148641 0 2024-09-16T00:09:53.004 INFO:tasks.workunit.client.0.smithi139.stdout:1/734: fsync d7/d1b/f3a 0 2024-09-16T00:09:53.005 INFO:tasks.workunit.client.0.smithi139.stdout:6/743: creat d8/d29/d83/d26/d45/d42/f108 x:0 0 0 2024-09-16T00:09:53.006 INFO:tasks.workunit.client.0.smithi139.stdout:3/708: mkdir d5/d7/d23/d27/d2f/d30/d4d/dc6/df0 0 2024-09-16T00:09:53.007 INFO:tasks.workunit.client.0.smithi139.stdout:1/735: write d7/d8/d27/d68/d6c/fbe [749293,4058] 0 2024-09-16T00:09:53.009 INFO:tasks.workunit.client.0.smithi139.stdout:7/898: dread d5/d16/d22/d25/f31 [0,4194304] 0 2024-09-16T00:09:53.010 INFO:tasks.workunit.client.0.smithi139.stdout:3/709: dread - d5/d7/d23/de4/de8/fcb zero size 2024-09-16T00:09:53.011 INFO:tasks.workunit.client.0.smithi139.stdout:6/744: creat d8/d29/d83/d26/d28/f109 x:0 0 0 2024-09-16T00:09:53.011 INFO:tasks.workunit.client.0.smithi139.stdout:7/899: chown d5/d16/d87/l109 8 1 2024-09-16T00:09:53.013 INFO:tasks.workunit.client.0.smithi139.stdout:0/881: rename d6/d1c/d41/d7c/d86/c8b to d6/d1c/d41/d44/d6b/c114 0 2024-09-16T00:09:53.013 INFO:tasks.workunit.client.0.smithi139.stdout:7/900: dread - d5/d16/d22/d71/f117 zero size 2024-09-16T00:09:53.014 INFO:tasks.workunit.client.0.smithi139.stdout:5/950: dread d1/d13/d3f/d14d/d79/da5/d10b/d11e/d10d/fea [0,4194304] 0 2024-09-16T00:09:53.014 INFO:tasks.workunit.client.0.smithi139.stdout:6/745: dread - d8/d29/d83/d26/d28/d5a/f86 zero size 2024-09-16T00:09:53.015 INFO:tasks.workunit.client.0.smithi139.stdout:0/882: fsync d6/d1c/d41/f96 0 2024-09-16T00:09:53.016 INFO:tasks.workunit.client.0.smithi139.stdout:5/951: stat d1/d21/d49/d53/d6a/d7b/d99/fac 0 2024-09-16T00:09:53.017 INFO:tasks.workunit.client.0.smithi139.stdout:0/883: chown d6/d1c/d41/d78/df2/d10e/cfe 1166 1 2024-09-16T00:09:53.017 INFO:tasks.workunit.client.0.smithi139.stdout:7/901: creat d5/d16/d6d/d7f/df5/f13f x:0 0 0 2024-09-16T00:09:53.017 INFO:tasks.workunit.client.0.smithi139.stdout:5/952: chown d1/d21/d49/db2/fba 41881 1 2024-09-16T00:09:53.018 INFO:tasks.workunit.client.0.smithi139.stdout:3/710: link d5/d7/d23/d27/d2f/c34 d5/d7/d23/de4/de8/cf1 0 2024-09-16T00:09:53.019 INFO:tasks.workunit.client.0.smithi139.stdout:7/902: creat d5/d16/d6d/d7f/dc2/d81/f140 x:0 0 0 2024-09-16T00:09:53.019 INFO:tasks.workunit.client.0.smithi139.stdout:0/884: mknod d6/d1c/d41/d63/c115 0 2024-09-16T00:09:53.021 INFO:tasks.workunit.client.0.smithi139.stdout:7/903: readlink d5/d26/d74/dfd/l13b 0 2024-09-16T00:09:53.022 INFO:tasks.workunit.client.0.smithi139.stdout:3/711: dread d5/d7/f13 [0,4194304] 0 2024-09-16T00:09:53.022 INFO:tasks.workunit.client.0.smithi139.stdout:3/712: fsync d5/d5f/fe7 0 2024-09-16T00:09:53.024 INFO:tasks.workunit.client.0.smithi139.stdout:3/713: fsync d5/d7/d23/d27/d2f/d30/d4c/d6d/f79 0 2024-09-16T00:09:53.024 INFO:tasks.workunit.client.0.smithi139.stdout:0/885: link d6/d1c/d41/d44/dd/d8a/fb3 d6/d1c/d41/f116 0 2024-09-16T00:09:53.027 INFO:tasks.workunit.client.0.smithi139.stdout:3/714: mknod d5/d7/d23/d27/d2f/d30/d4c/d6d/db3/cf2 0 2024-09-16T00:09:53.027 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:53.027+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:53.029 INFO:tasks.workunit.client.0.smithi139.stdout:3/715: rmdir d5/d7/d23/d27/d2f/d30/d4c/da1 39 2024-09-16T00:09:53.031 INFO:tasks.workunit.client.0.smithi139.stdout:3/716: mkdir d5/d7/dc/d9c/db0/df3 0 2024-09-16T00:09:53.032 INFO:tasks.workunit.client.0.smithi139.stdout:0/886: dread d6/d1c/d41/d44/dd/fb0 [0,4194304] 0 2024-09-16T00:09:53.035 INFO:tasks.workunit.client.0.smithi139.stdout:3/717: rmdir d5/d7/d23/d27/d2f/d30/d4d/dc6/df0 0 2024-09-16T00:09:53.036 INFO:tasks.workunit.client.0.smithi139.stdout:0/887: dread d6/d1c/d41/d44/dd/f2c [0,4194304] 0 2024-09-16T00:09:53.036 INFO:tasks.workunit.client.0.smithi139.stdout:0/888: dread - d6/d1c/d41/d7c/fb5 zero size 2024-09-16T00:09:53.037 INFO:tasks.workunit.client.0.smithi139.stdout:3/718: symlink d5/d7/d23/d27/d2f/d30/d4c/d6d/d76/lf4 0 2024-09-16T00:09:53.038 INFO:tasks.workunit.client.0.smithi139.stdout:3/719: mknod d5/d7/dc/d9c/cf5 0 2024-09-16T00:09:53.041 INFO:tasks.workunit.client.0.smithi139.stdout:3/720: rename d5/d7/dc/f63 to d5/d7/d23/d27/d2f/d30/d4c/dbb/ff6 0 2024-09-16T00:09:53.042 INFO:tasks.workunit.client.0.smithi139.stdout:3/721: chown d5/d7/d23/d27/d2f/d30/d4d/fb1 116288420 1 2024-09-16T00:09:53.044 INFO:tasks.workunit.client.0.smithi139.stdout:3/722: mkdir d5/d7/d23/d27/d2f/d30/d4c/d6d/db3/ddc/df7 0 2024-09-16T00:09:53.047 INFO:tasks.workunit.client.0.smithi139.stdout:3/723: getdents d5/d7/d23/d27/d2f/d30/d4c/dbb/de2 0 2024-09-16T00:09:53.049 INFO:tasks.workunit.client.0.smithi139.stdout:3/724: creat d5/d7/ff8 x:0 0 0 2024-09-16T00:09:53.054 INFO:tasks.workunit.client.0.smithi139.stdout:3/725: dwrite d5/d7/dc2/fd7 [0,4194304] 0 2024-09-16T00:09:53.061 INFO:tasks.workunit.client.0.smithi139.stdout:3/726: dwrite d5/d7/d23/fe0 [0,4194304] 0 2024-09-16T00:09:53.065 INFO:tasks.workunit.client.0.smithi139.stdout:3/727: dread d5/f8 [0,4194304] 0 2024-09-16T00:09:53.067 INFO:tasks.workunit.client.0.smithi139.stdout:3/728: symlink d5/d7/lf9 0 2024-09-16T00:09:53.068 INFO:tasks.workunit.client.0.smithi139.stdout:3/729: getdents d5/d7/d23/de4/dca 0 2024-09-16T00:09:53.077 INFO:tasks.workunit.client.0.smithi139.stdout:3/730: dread d5/d7/d23/d27/d2f/d30/d4d/f8c [0,4194304] 0 2024-09-16T00:09:53.080 INFO:tasks.workunit.client.0.smithi139.stdout:3/731: dread d5/d7/f37 [0,4194304] 0 2024-09-16T00:09:53.082 INFO:tasks.workunit.client.0.smithi139.stdout:3/732: fdatasync d5/f53 0 2024-09-16T00:09:53.083 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:53.084 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:53.084 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:53.084 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:53.084 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:53.090 INFO:tasks.workunit.client.0.smithi139.stdout:3/733: dwrite d5/d7/d23/de4/de8/f45 [0,4194304] 0 2024-09-16T00:09:53.091 INFO:tasks.workunit.client.0.smithi139.stdout:3/734: readlink d5/d7/d23/d27/d2f/d30/d4d/l86 0 2024-09-16T00:09:53.093 INFO:tasks.workunit.client.0.smithi139.stdout:3/735: symlink d5/d7/d23/d27/d2f/d30/d4d/dc6/lfa 0 2024-09-16T00:09:53.096 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:09:53.096 INFO:tasks.workunit.client.0.smithi139.stdout:3/736: mkdir d5/d7/d23/d27/d2f/d30/d4c/d6d/d76/dfb 0 2024-09-16T00:09:53.097 INFO:tasks.workunit.client.0.smithi139.stdout:3/737: write d5/d7/d23/de4/f87 [1471288,42676] 0 2024-09-16T00:09:53.100 INFO:tasks.workunit.client.0.smithi139.stdout:3/738: rename d5/d7/dc/c43 to d5/d7/dc/d9c/cfc 0 2024-09-16T00:09:53.103 INFO:tasks.workunit.client.0.smithi139.stdout:3/739: unlink d5/d7/d23/d27/d2f/d30/f4e 0 2024-09-16T00:09:53.105 INFO:tasks.workunit.client.0.smithi139.stdout:3/740: read d5/d7/d23/d27/d2f/d30/d4c/d6d/f79 [3881118,6129] 0 2024-09-16T00:09:53.107 INFO:tasks.workunit.client.0.smithi139.stdout:3/741: creat d5/d7/d23/d27/d2f/d30/d4c/dbb/de2/ffd x:0 0 0 2024-09-16T00:09:53.109 INFO:tasks.workunit.client.0.smithi139.stdout:3/742: unlink d5/d7/dc/f8a 0 2024-09-16T00:09:53.112 INFO:tasks.workunit.client.0.smithi139.stdout:3/743: mkdir d5/d7/d23/d27/d2f/d30/d4d/dc6/dfe 0 2024-09-16T00:09:53.122 INFO:tasks.workunit.client.0.smithi139.stdout:3/744: dread d5/d7/d23/de4/de8/f61 [0,4194304] 0 2024-09-16T00:09:53.124 INFO:tasks.workunit.client.0.smithi139.stdout:3/745: chown d5/d7/d23/d27/d2f/d30/d4c/da1/fd3 170 1 2024-09-16T00:09:53.128 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:53.127+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:53.128 INFO:tasks.workunit.client.0.smithi139.stdout:3/746: dwrite d5/d7/d23/d27/d2f/d30/d4c/d6d/f79 [0,4194304] 0 2024-09-16T00:09:53.131 INFO:tasks.workunit.client.0.smithi139.stdout:3/747: write d5/d7/d23/d27/d2f/f7a [1319753,68188] 0 2024-09-16T00:09:53.134 INFO:tasks.workunit.client.0.smithi139.stdout:3/748: dread d5/d7/d23/d27/d2f/d30/d4c/d6d/d76/f98 [0,4194304] 0 2024-09-16T00:09:53.136 INFO:tasks.workunit.client.0.smithi139.stdout:3/749: unlink d5/d7/dc/f3c 0 2024-09-16T00:09:53.137 INFO:tasks.workunit.client.0.smithi139.stdout:2/998: sync 2024-09-16T00:09:53.140 INFO:tasks.workunit.client.0.smithi139.stdout:3/750: unlink d5/d7/d23/d27/d2f/d30/c3e 0 2024-09-16T00:09:53.144 INFO:tasks.workunit.client.0.smithi139.stdout:3/751: rmdir d5/d7/d23/d27/d2f/d30/d4c/d6d/db3/ddc/df7 0 2024-09-16T00:09:53.146 INFO:tasks.workunit.client.0.smithi139.stdout:2/999: truncate d11/d30/d110/d29/f66 4587056 0 2024-09-16T00:09:53.147 INFO:tasks.workunit.client.0.smithi139.stdout:3/752: dread d5/d7/d23/de4/de8/f61 [0,4194304] 0 2024-09-16T00:09:53.148 INFO:tasks.workunit.client.0.smithi139.stdout:3/753: fdatasync d5/d7/d23/d27/d2f/f7a 0 2024-09-16T00:09:53.150 INFO:tasks.workunit.client.0.smithi139.stdout:3/754: mknod d5/d7/d23/d27/d2f/d30/d4c/d6d/db3/ddc/cff 0 2024-09-16T00:09:53.152 INFO:tasks.workunit.client.0.smithi139.stdout:3/755: getdents d5/d7/dc/d9c/db0/df3 0 2024-09-16T00:09:53.153 INFO:tasks.workunit.client.0.smithi139.stdout:3/756: creat d5/d7/d23/de4/dca/f100 x:0 0 0 2024-09-16T00:09:53.156 INFO:tasks.workunit.client.0.smithi139.stdout:3/757: dread d5/d7/d23/de4/fcd [0,4194304] 0 2024-09-16T00:09:53.158 INFO:tasks.workunit.client.0.smithi139.stdout:3/758: truncate d5/d7/d23/de4/de8/f61 5112852 0 2024-09-16T00:09:53.166 INFO:tasks.workunit.client.0.smithi139.stdout:3/759: rename d5/d7/d23/d27 to d5/d7/d23/de4/de8/da7/d9a/d101 0 2024-09-16T00:09:53.168 INFO:tasks.workunit.client.0.smithi139.stdout:3/760: mknod d5/d7/d23/de4/d75/de1/c102 0 2024-09-16T00:09:53.172 INFO:tasks.workunit.client.0.smithi139.stdout:3/761: dread d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4d/fb1 [0,4194304] 0 2024-09-16T00:09:53.174 INFO:tasks.workunit.client.0.smithi139.stdout:3/762: creat d5/d7/d23/de4/de8/da7/d9a/f103 x:0 0 0 2024-09-16T00:09:53.175 INFO:tasks.workunit.client.0.smithi139.stdout:3/763: getdents d5/d7/dc/d9c/db0/df3 0 2024-09-16T00:09:53.181 INFO:tasks.workunit.client.0.smithi139.stdout:3/764: dwrite d5/d7/f28 [0,4194304] 0 2024-09-16T00:09:53.184 INFO:tasks.workunit.client.0.smithi139.stdout:3/765: chown d5/la6 2 1 2024-09-16T00:09:53.188 INFO:tasks.workunit.client.0.smithi139.stdout:3/766: rename d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4d/l86 to d5/d5f/l104 0 2024-09-16T00:09:53.195 INFO:tasks.workunit.client.0.smithi139.stdout:3/767: link d5/d7/d23/cd5 d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4d/dc6/dfe/c105 0 2024-09-16T00:09:53.197 INFO:tasks.workunit.client.0.smithi139.stdout:3/768: truncate d5/d7/d23/de4/de8/f3b 767717 0 2024-09-16T00:09:53.199 INFO:tasks.workunit.client.0.smithi139.stdout:3/769: chown d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/f7f 19144069 1 2024-09-16T00:09:53.206 INFO:tasks.workunit.client.0.smithi139.stdout:3/770: dwrite d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d47/f4a [0,4194304] 0 2024-09-16T00:09:53.209 INFO:tasks.workunit.client.0.smithi139.stdout:3/771: dread d5/d7/f13 [0,4194304] 0 2024-09-16T00:09:53.214 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:53.213+0000 7f5a07e2c640 -1 Falling back to public interface 2024-09-16T00:09:53.222 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:53.223 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:53.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:53.223 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:53.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.223 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10656 3.19 3:9943069b:::100000030ce.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:53.223 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.774845+0000", 2024-09-16T00:09:53.223 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 17.445635868, 2024-09-16T00:09:53.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012894586, 2024-09-16T00:09:53.223 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.223 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.223 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.223 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.223 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.223 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.224 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10656 2024-09-16T00:09:53.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.224 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.774845+0000", 2024-09-16T00:09:53.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.774845+0000", 2024-09-16T00:09:53.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.774846+0000", 2024-09-16T00:09:53.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.469e-06 2024-09-16T00:09:53.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.776187+0000", 2024-09-16T00:09:53.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001341021 2024-09-16T00:09:53.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.776206+0000", 2024-09-16T00:09:53.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8975e-05 2024-09-16T00:09:53.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.776215+0000", 2024-09-16T00:09:53.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0499999999999997e-06 2024-09-16T00:09:53.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.776279+0000", 2024-09-16T00:09:53.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3874999999999994e-05 2024-09-16T00:09:53.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.776408+0000", 2024-09-16T00:09:53.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012841600000000001 2024-09-16T00:09:53.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:53.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.776544+0000", 2024-09-16T00:09:53.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013610900000000001 2024-09-16T00:09:53.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.784791+0000", 2024-09-16T00:09:53.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0082469290000000001 2024-09-16T00:09:53.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.787622+0000", 2024-09-16T00:09:53.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028308679999999998 2024-09-16T00:09:53.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.787684+0000", 2024-09-16T00:09:53.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2000000000000003e-05 2024-09-16T00:09:53.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.787740+0000", 2024-09-16T00:09:53.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5874000000000001e-05 2024-09-16T00:09:53.228 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.228 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.229 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.229 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10689 3.25 3:a7febee6:::100000030e3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:53.229 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.913087+0000", 2024-09-16T00:09:53.229 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 17.307393565000002, 2024-09-16T00:09:53.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.019706497, 2024-09-16T00:09:53.229 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.229 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.229 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.229 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.229 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.229 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.229 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10689 2024-09-16T00:09:53.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.230 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.913087+0000", 2024-09-16T00:09:53.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.913087+0000", 2024-09-16T00:09:53.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.913089+0000", 2024-09-16T00:09:53.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1789999999999998e-06 2024-09-16T00:09:53.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.914935+0000", 2024-09-16T00:09:53.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001845469 2024-09-16T00:09:53.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.914958+0000", 2024-09-16T00:09:53.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2895e-05 2024-09-16T00:09:53.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.914967+0000", 2024-09-16T00:09:53.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5079999999999994e-06 2024-09-16T00:09:53.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.915031+0000", 2024-09-16T00:09:53.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4144000000000001e-05 2024-09-16T00:09:53.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.915096+0000", 2024-09-16T00:09:53.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4842e-05 2024-09-16T00:09:53.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:53.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.915191+0000", 2024-09-16T00:09:53.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4209999999999994e-05 2024-09-16T00:09:53.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.921193+0000", 2024-09-16T00:09:53.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0060020760000000003 2024-09-16T00:09:53.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.932664+0000", 2024-09-16T00:09:53.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011471093999999999 2024-09-16T00:09:53.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.932735+0000", 2024-09-16T00:09:53.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1660000000000002e-05 2024-09-16T00:09:53.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.235 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.932794+0000", 2024-09-16T00:09:53.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.842e-05 2024-09-16T00:09:53.235 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.235 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.235 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.235 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10690 3.25 3:a7febee6:::100000030e3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:53.235 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.924498+0000", 2024-09-16T00:09:53.235 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 17.295982902999999, 2024-09-16T00:09:53.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013019637000000001, 2024-09-16T00:09:53.235 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.235 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.236 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.236 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.236 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.236 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.236 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10690 2024-09-16T00:09:53.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.236 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.236 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.924498+0000", 2024-09-16T00:09:53.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.236 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.924498+0000", 2024-09-16T00:09:53.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.924502+0000", 2024-09-16T00:09:53.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5899999999999999e-06 2024-09-16T00:09:53.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.926464+0000", 2024-09-16T00:09:53.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019621349999999998 2024-09-16T00:09:53.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.926477+0000", 2024-09-16T00:09:53.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3104e-05 2024-09-16T00:09:53.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.926487+0000", 2024-09-16T00:09:53.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9420000000000007e-06 2024-09-16T00:09:53.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.926550+0000", 2024-09-16T00:09:53.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3584999999999995e-05 2024-09-16T00:09:53.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.926643+0000", 2024-09-16T00:09:53.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2604999999999999e-05 2024-09-16T00:09:53.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:53.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.926806+0000", 2024-09-16T00:09:53.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016308300000000001 2024-09-16T00:09:53.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.935095+0000", 2024-09-16T00:09:53.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0082889179999999993 2024-09-16T00:09:53.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.937429+0000", 2024-09-16T00:09:53.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023340589999999999 2024-09-16T00:09:53.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.937480+0000", 2024-09-16T00:09:53.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0594999999999999e-05 2024-09-16T00:09:53.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.937518+0000", 2024-09-16T00:09:53.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8021000000000001e-05 2024-09-16T00:09:53.241 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.241 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.241 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.241 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10774 3.19 3:990d5343:::10000003135.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:53.241 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.427206+0000", 2024-09-16T00:09:53.241 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.793274472, 2024-09-16T00:09:53.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012269067, 2024-09-16T00:09:53.242 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.242 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.242 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.242 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.242 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.242 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.242 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10774 2024-09-16T00:09:53.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.242 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.242 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.242 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.427206+0000", 2024-09-16T00:09:53.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.427206+0000", 2024-09-16T00:09:53.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.427209+0000", 2024-09-16T00:09:53.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3829999999999999e-06 2024-09-16T00:09:53.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.429890+0000", 2024-09-16T00:09:53.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026809999999999998 2024-09-16T00:09:53.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.429919+0000", 2024-09-16T00:09:53.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8778000000000001e-05 2024-09-16T00:09:53.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.429935+0000", 2024-09-16T00:09:53.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6739999999999999e-05 2024-09-16T00:09:53.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.430016+0000", 2024-09-16T00:09:53.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0766000000000006e-05 2024-09-16T00:09:53.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.430167+0000", 2024-09-16T00:09:53.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015089500000000001 2024-09-16T00:09:53.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:53.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.430272+0000", 2024-09-16T00:09:53.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000105447 2024-09-16T00:09:53.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.435798+0000", 2024-09-16T00:09:53.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055258140000000004 2024-09-16T00:09:53.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.439372+0000", 2024-09-16T00:09:53.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003573906 2024-09-16T00:09:53.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.439428+0000", 2024-09-16T00:09:53.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5826999999999999e-05 2024-09-16T00:09:53.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.439475+0000", 2024-09-16T00:09:53.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7511e-05 2024-09-16T00:09:53.248 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.248 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.248 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.248 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10802 3.25 3:a7febee6:::100000030e3.00000000:head [write 0~4194304 [4@4194304]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:53.248 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.658867+0000", 2024-09-16T00:09:53.249 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.561613905, 2024-09-16T00:09:53.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011536737, 2024-09-16T00:09:53.249 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.249 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.249 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.249 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.249 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.249 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.249 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10802 2024-09-16T00:09:53.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.249 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.658867+0000", 2024-09-16T00:09:53.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.658867+0000", 2024-09-16T00:09:53.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.658869+0000", 2024-09-16T00:09:53.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1940000000000001e-06 2024-09-16T00:09:53.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.660719+0000", 2024-09-16T00:09:53.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018499090000000001 2024-09-16T00:09:53.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.660732+0000", 2024-09-16T00:09:53.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3162999999999999e-05 2024-09-16T00:09:53.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.660742+0000", 2024-09-16T00:09:53.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4159999999999993e-06 2024-09-16T00:09:53.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.660828+0000", 2024-09-16T00:09:53.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6822000000000003e-05 2024-09-16T00:09:53.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.660938+0000", 2024-09-16T00:09:53.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109554 2024-09-16T00:09:53.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:53.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.661042+0000", 2024-09-16T00:09:53.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104312 2024-09-16T00:09:53.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.666676+0000", 2024-09-16T00:09:53.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0056336850000000003 2024-09-16T00:09:53.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.670301+0000", 2024-09-16T00:09:53.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0036246170000000001 2024-09-16T00:09:53.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.670352+0000", 2024-09-16T00:09:53.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1022e-05 2024-09-16T00:09:53.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.670404+0000", 2024-09-16T00:09:53.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2043000000000003e-05 2024-09-16T00:09:53.254 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.254 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.254 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.255 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10824 3.3b 3:deb39c3c:::1000000315d.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:53.255 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.727887+0000", 2024-09-16T00:09:53.255 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.492593382999999, 2024-09-16T00:09:53.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011016697000000001, 2024-09-16T00:09:53.255 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.255 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.255 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.255 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.255 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.255 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.255 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10824 2024-09-16T00:09:53.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.255 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.727887+0000", 2024-09-16T00:09:53.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.727887+0000", 2024-09-16T00:09:53.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.727890+0000", 2024-09-16T00:09:53.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2340000000000001e-06 2024-09-16T00:09:53.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.729535+0000", 2024-09-16T00:09:53.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001645234 2024-09-16T00:09:53.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.729568+0000", 2024-09-16T00:09:53.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3188999999999997e-05 2024-09-16T00:09:53.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.729582+0000", 2024-09-16T00:09:53.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3583e-05 2024-09-16T00:09:53.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.729717+0000", 2024-09-16T00:09:53.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000135053 2024-09-16T00:09:53.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.729938+0000", 2024-09-16T00:09:53.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022084399999999999 2024-09-16T00:09:53.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:53.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.730079+0000", 2024-09-16T00:09:53.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000141142 2024-09-16T00:09:53.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.737071+0000", 2024-09-16T00:09:53.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069920750000000004 2024-09-16T00:09:53.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.738789+0000", 2024-09-16T00:09:53.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017177469999999999 2024-09-16T00:09:53.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.738850+0000", 2024-09-16T00:09:53.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1582999999999996e-05 2024-09-16T00:09:53.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.738904+0000", 2024-09-16T00:09:53.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4012999999999999e-05 2024-09-16T00:09:53.260 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.260 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.260 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.261 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11938 3.3e 3:7fb917e8:::100000030b7.00000001:head [write 0~4063232 [2@0]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:53.261 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.718248+0000", 2024-09-16T00:09:53.261 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.5022325470000002, 2024-09-16T00:09:53.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011465517999999999, 2024-09-16T00:09:53.261 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.261 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.261 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.261 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.261 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.261 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.261 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11938 2024-09-16T00:09:53.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.261 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.718248+0000", 2024-09-16T00:09:53.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.718248+0000", 2024-09-16T00:09:53.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.718251+0000", 2024-09-16T00:09:53.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8509999999999999e-06 2024-09-16T00:09:53.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.719886+0000", 2024-09-16T00:09:53.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001634967 2024-09-16T00:09:53.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.719916+0000", 2024-09-16T00:09:53.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0258000000000001e-05 2024-09-16T00:09:53.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.719930+0000", 2024-09-16T00:09:53.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3379e-05 2024-09-16T00:09:53.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.720023+0000", 2024-09-16T00:09:53.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3121000000000005e-05 2024-09-16T00:09:53.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.720200+0000", 2024-09-16T00:09:53.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017721600000000001 2024-09-16T00:09:53.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:53.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.720361+0000", 2024-09-16T00:09:53.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016081900000000001 2024-09-16T00:09:53.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.727099+0000", 2024-09-16T00:09:53.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0067381419999999999 2024-09-16T00:09:53.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.729630+0000", 2024-09-16T00:09:53.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025311890000000001 2024-09-16T00:09:53.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.729667+0000", 2024-09-16T00:09:53.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6300000000000001e-05 2024-09-16T00:09:53.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.729714+0000", 2024-09-16T00:09:53.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7275999999999999e-05 2024-09-16T00:09:53.266 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.266 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.266 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.267 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11953 3.1a e87/20 3:580e2503:::1000000311b.00000000:head v 87'185, mlcod=86'183)", 2024-09-16T00:09:53.267 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.771482+0000", 2024-09-16T00:09:53.267 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.448998692, 2024-09-16T00:09:53.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011212730000000001, 2024-09-16T00:09:53.267 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.267 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.267 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.267 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.771482+0000", 2024-09-16T00:09:53.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.771482+0000", 2024-09-16T00:09:53.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.771486+0000", 2024-09-16T00:09:53.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6820000000000001e-06 2024-09-16T00:09:53.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.777946+0000", 2024-09-16T00:09:53.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0064603530000000003 2024-09-16T00:09:53.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.777975+0000", 2024-09-16T00:09:53.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8580000000000001e-05 2024-09-16T00:09:53.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.777995+0000", 2024-09-16T00:09:53.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0136000000000002e-05 2024-09-16T00:09:53.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.778100+0000", 2024-09-16T00:09:53.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000104893 2024-09-16T00:09:53.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.778135+0000", 2024-09-16T00:09:53.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5293000000000002e-05 2024-09-16T00:09:53.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.782643+0000", 2024-09-16T00:09:53.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045082059999999998 2024-09-16T00:09:53.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.782695+0000", 2024-09-16T00:09:53.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1586999999999999e-05 2024-09-16T00:09:53.271 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.271 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.271 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.271 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:12245 3.1a e87/20 3:599a1cd1:::10000003252.00000000:head v 87'188, mlcod=87'186)", 2024-09-16T00:09:53.271 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.786850+0000", 2024-09-16T00:09:53.271 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.4336307969999993, 2024-09-16T00:09:53.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01255644, 2024-09-16T00:09:53.272 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.272 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.272 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.272 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.786850+0000", 2024-09-16T00:09:53.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.786850+0000", 2024-09-16T00:09:53.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.786852+0000", 2024-09-16T00:09:53.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9659999999999999e-06 2024-09-16T00:09:53.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.791923+0000", 2024-09-16T00:09:53.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050705460000000004 2024-09-16T00:09:53.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.791948+0000", 2024-09-16T00:09:53.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5860999999999999e-05 2024-09-16T00:09:53.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.791964+0000", 2024-09-16T00:09:53.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5131e-05 2024-09-16T00:09:53.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.792054+0000", 2024-09-16T00:09:53.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0694999999999998e-05 2024-09-16T00:09:53.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.792085+0000", 2024-09-16T00:09:53.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0846999999999998e-05 2024-09-16T00:09:53.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.799315+0000", 2024-09-16T00:09:53.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0072297780000000001 2024-09-16T00:09:53.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.799406+0000", 2024-09-16T00:09:53.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1615999999999998e-05 2024-09-16T00:09:53.276 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.276 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.276 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.276 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12248 3.3b 3:de8bdadd:::100000033c7.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:53.276 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.787748+0000", 2024-09-16T00:09:53.276 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.4327332659999996, 2024-09-16T00:09:53.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011787646000000001, 2024-09-16T00:09:53.276 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.276 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.276 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.276 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.276 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.277 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.277 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12248 2024-09-16T00:09:53.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.277 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.787748+0000", 2024-09-16T00:09:53.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.787748+0000", 2024-09-16T00:09:53.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.787755+0000", 2024-09-16T00:09:53.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7049999999999999e-06 2024-09-16T00:09:53.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.789668+0000", 2024-09-16T00:09:53.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019122830000000001 2024-09-16T00:09:53.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.789692+0000", 2024-09-16T00:09:53.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4397999999999999e-05 2024-09-16T00:09:53.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.789705+0000", 2024-09-16T00:09:53.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3379e-05 2024-09-16T00:09:53.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.789788+0000", 2024-09-16T00:09:53.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2674000000000002e-05 2024-09-16T00:09:53.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.789929+0000", 2024-09-16T00:09:53.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000140638 2024-09-16T00:09:53.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:53.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.790077+0000", 2024-09-16T00:09:53.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014853199999999999 2024-09-16T00:09:53.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.798739+0000", 2024-09-16T00:09:53.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0086621660000000007 2024-09-16T00:09:53.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.799426+0000", 2024-09-16T00:09:53.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00068666699999999996 2024-09-16T00:09:53.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.799479+0000", 2024-09-16T00:09:53.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2784999999999997e-05 2024-09-16T00:09:53.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.799535+0000", 2024-09-16T00:09:53.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6419000000000003e-05 2024-09-16T00:09:53.282 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.282 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.282 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.282 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.282 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12269 3.3b 3:dc3ad4b0:::10000003125.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:53.282 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.844085+0000", 2024-09-16T00:09:53.282 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.3763956650000004, 2024-09-16T00:09:53.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011272122000000001, 2024-09-16T00:09:53.282 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.282 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.282 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.282 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.282 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.283 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.283 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12269 2024-09-16T00:09:53.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.283 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.844085+0000", 2024-09-16T00:09:53.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.844085+0000", 2024-09-16T00:09:53.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.844088+0000", 2024-09-16T00:09:53.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7769999999999998e-06 2024-09-16T00:09:53.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.846085+0000", 2024-09-16T00:09:53.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019974979999999999 2024-09-16T00:09:53.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.846124+0000", 2024-09-16T00:09:53.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8049000000000001e-05 2024-09-16T00:09:53.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.846141+0000", 2024-09-16T00:09:53.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7343e-05 2024-09-16T00:09:53.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.846235+0000", 2024-09-16T00:09:53.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4219e-05 2024-09-16T00:09:53.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.846349+0000", 2024-09-16T00:09:53.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113986 2024-09-16T00:09:53.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:53.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.846505+0000", 2024-09-16T00:09:53.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015587700000000001 2024-09-16T00:09:53.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.854748+0000", 2024-09-16T00:09:53.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0082434489999999999 2024-09-16T00:09:53.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.855238+0000", 2024-09-16T00:09:53.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000489808 2024-09-16T00:09:53.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.855305+0000", 2024-09-16T00:09:53.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6334000000000006e-05 2024-09-16T00:09:53.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.855357+0000", 2024-09-16T00:09:53.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2781999999999997e-05 2024-09-16T00:09:53.288 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.288 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.288 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.288 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12334 3.19 3:9a91b182:::100000031d2.00000001:head [write 0~4194304 [2@0]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:53.288 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.021880+0000", 2024-09-16T00:09:53.288 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.1986006170000003, 2024-09-16T00:09:53.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011231692, 2024-09-16T00:09:53.288 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.288 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.288 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.288 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.288 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.289 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.289 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12334 2024-09-16T00:09:53.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.289 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.021880+0000", 2024-09-16T00:09:53.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.021880+0000", 2024-09-16T00:09:53.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.021883+0000", 2024-09-16T00:09:53.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1829999999999998e-06 2024-09-16T00:09:53.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.024208+0000", 2024-09-16T00:09:53.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023246880000000001 2024-09-16T00:09:53.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.024249+0000", 2024-09-16T00:09:53.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1328999999999997e-05 2024-09-16T00:09:53.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.024270+0000", 2024-09-16T00:09:53.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0295000000000001e-05 2024-09-16T00:09:53.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.024363+0000", 2024-09-16T00:09:53.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3026e-05 2024-09-16T00:09:53.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.024469+0000", 2024-09-16T00:09:53.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010646599999999999 2024-09-16T00:09:53.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:53.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.024626+0000", 2024-09-16T00:09:53.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000156465 2024-09-16T00:09:53.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.030328+0000", 2024-09-16T00:09:53.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057018599999999996 2024-09-16T00:09:53.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.033005+0000", 2024-09-16T00:09:53.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026773159999999999 2024-09-16T00:09:53.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.033063+0000", 2024-09-16T00:09:53.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7911000000000002e-05 2024-09-16T00:09:53.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.033112+0000", 2024-09-16T00:09:53.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9153000000000002e-05 2024-09-16T00:09:53.293 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.293 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.294 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.294 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13316 3.25 3:a7ae0919:::100000033a3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:53.294 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.918020+0000", 2024-09-16T00:09:53.294 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.302460956, 2024-09-16T00:09:53.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012689604, 2024-09-16T00:09:53.294 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.294 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.294 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.294 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.294 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.294 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.294 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13316 2024-09-16T00:09:53.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.295 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918020+0000", 2024-09-16T00:09:53.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918020+0000", 2024-09-16T00:09:53.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918023+0000", 2024-09-16T00:09:53.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0730000000000001e-06 2024-09-16T00:09:53.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920995+0000", 2024-09-16T00:09:53.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0029723079999999999 2024-09-16T00:09:53.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.921016+0000", 2024-09-16T00:09:53.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1219000000000001e-05 2024-09-16T00:09:53.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.921033+0000", 2024-09-16T00:09:53.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6359999999999999e-05 2024-09-16T00:09:53.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.921229+0000", 2024-09-16T00:09:53.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019620600000000001 2024-09-16T00:09:53.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.921312+0000", 2024-09-16T00:09:53.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2718000000000003e-05 2024-09-16T00:09:53.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:53.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.921417+0000", 2024-09-16T00:09:53.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000105089 2024-09-16T00:09:53.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929440+0000", 2024-09-16T00:09:53.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0080232789999999995 2024-09-16T00:09:53.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.930603+0000", 2024-09-16T00:09:53.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001162426 2024-09-16T00:09:53.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.930659+0000", 2024-09-16T00:09:53.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.647e-05 2024-09-16T00:09:53.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.930709+0000", 2024-09-16T00:09:53.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0455999999999999e-05 2024-09-16T00:09:53.299 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.299 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.300 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.300 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13335 3.19 3:990d5343:::10000003135.00000000:head [write 0~4194304 [4@944224]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:53.300 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.974375+0000", 2024-09-16T00:09:53.300 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.2461063299999999, 2024-09-16T00:09:53.300 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011540745999999999, 2024-09-16T00:09:53.300 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.300 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.300 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.300 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.300 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.300 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.300 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13335 2024-09-16T00:09:53.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.301 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.974375+0000", 2024-09-16T00:09:53.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.974375+0000", 2024-09-16T00:09:53.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.974376+0000", 2024-09-16T00:09:53.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.796e-06 2024-09-16T00:09:53.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.976458+0000", 2024-09-16T00:09:53.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020820830000000002 2024-09-16T00:09:53.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.976480+0000", 2024-09-16T00:09:53.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2005999999999999e-05 2024-09-16T00:09:53.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.303 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.976489+0000", 2024-09-16T00:09:53.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6889999999999993e-06 2024-09-16T00:09:53.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.303 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.976556+0000", 2024-09-16T00:09:53.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7183999999999999e-05 2024-09-16T00:09:53.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.303 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.976652+0000", 2024-09-16T00:09:53.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5838e-05 2024-09-16T00:09:53.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.304 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:53.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.976819+0000", 2024-09-16T00:09:53.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016737499999999999 2024-09-16T00:09:53.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.304 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.304 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.983064+0000", 2024-09-16T00:09:53.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0062443359999999996 2024-09-16T00:09:53.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.304 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.304 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.985782+0000", 2024-09-16T00:09:53.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027184940000000001 2024-09-16T00:09:53.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.985863+0000", 2024-09-16T00:09:53.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0496999999999999e-05 2024-09-16T00:09:53.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.985915+0000", 2024-09-16T00:09:53.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2448000000000003e-05 2024-09-16T00:09:53.305 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.305 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.305 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.306 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13718 3.4 3:2337fd76:::10000003773.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:53.306 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.408371+0000", 2024-09-16T00:09:53.306 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.8121097530000001, 2024-09-16T00:09:53.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011504019000000001, 2024-09-16T00:09:53.306 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.306 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.306 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.306 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.306 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.306 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.306 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13718 2024-09-16T00:09:53.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.307 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.408371+0000", 2024-09-16T00:09:53.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.408371+0000", 2024-09-16T00:09:53.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.408374+0000", 2024-09-16T00:09:53.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4789999999999999e-06 2024-09-16T00:09:53.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.410176+0000", 2024-09-16T00:09:53.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018021840000000001 2024-09-16T00:09:53.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.410204+0000", 2024-09-16T00:09:53.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8206999999999999e-05 2024-09-16T00:09:53.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.410216+0000", 2024-09-16T00:09:53.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1602e-05 2024-09-16T00:09:53.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.410319+0000", 2024-09-16T00:09:53.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000103729 2024-09-16T00:09:53.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.410501+0000", 2024-09-16T00:09:53.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018194899999999999 2024-09-16T00:09:53.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:53.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.410671+0000", 2024-09-16T00:09:53.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170006 2024-09-16T00:09:53.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.415622+0000", 2024-09-16T00:09:53.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0049505369999999996 2024-09-16T00:09:53.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.419762+0000", 2024-09-16T00:09:53.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041400509999999996 2024-09-16T00:09:53.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.419818+0000", 2024-09-16T00:09:53.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6285000000000001e-05 2024-09-16T00:09:53.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.419875+0000", 2024-09-16T00:09:53.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6990000000000002e-05 2024-09-16T00:09:53.311 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.311 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.311 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.312 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:13722 3.6 e89/20 3:639728a8:::100000030cc.00000000:head v 89'219, mlcod=89'217)", 2024-09-16T00:09:53.312 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.444869+0000", 2024-09-16T00:09:53.312 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.7756114190000001, 2024-09-16T00:09:53.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011405108000000001, 2024-09-16T00:09:53.312 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.312 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.312 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.312 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444869+0000", 2024-09-16T00:09:53.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444869+0000", 2024-09-16T00:09:53.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444871+0000", 2024-09-16T00:09:53.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8759999999999999e-06 2024-09-16T00:09:53.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.446420+0000", 2024-09-16T00:09:53.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001549188 2024-09-16T00:09:53.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.446446+0000", 2024-09-16T00:09:53.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5579999999999999e-05 2024-09-16T00:09:53.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.446461+0000", 2024-09-16T00:09:53.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.49e-05 2024-09-16T00:09:53.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.446573+0000", 2024-09-16T00:09:53.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011165100000000001 2024-09-16T00:09:53.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.446599+0000", 2024-09-16T00:09:53.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6794000000000001e-05 2024-09-16T00:09:53.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.456214+0000", 2024-09-16T00:09:53.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0096142840000000007 2024-09-16T00:09:53.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.456275+0000", 2024-09-16T00:09:53.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0835000000000003e-05 2024-09-16T00:09:53.316 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.316 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.316 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.316 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13800 3.31 3:8f9519f5:::1000000333e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:53.316 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.623029+0000", 2024-09-16T00:09:53.316 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.5974521500000001, 2024-09-16T00:09:53.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014897562, 2024-09-16T00:09:53.317 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.317 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.317 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.317 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.317 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.317 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.317 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13800 2024-09-16T00:09:53.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.317 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.623029+0000", 2024-09-16T00:09:53.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.623029+0000", 2024-09-16T00:09:53.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.623030+0000", 2024-09-16T00:09:53.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.508e-06 2024-09-16T00:09:53.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.624839+0000", 2024-09-16T00:09:53.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018091470000000001 2024-09-16T00:09:53.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.624855+0000", 2024-09-16T00:09:53.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5526e-05 2024-09-16T00:09:53.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.624864+0000", 2024-09-16T00:09:53.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9889999999999998e-06 2024-09-16T00:09:53.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.624929+0000", 2024-09-16T00:09:53.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5432999999999995e-05 2024-09-16T00:09:53.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.625074+0000", 2024-09-16T00:09:53.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014447799999999999 2024-09-16T00:09:53.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:53.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.625219+0000", 2024-09-16T00:09:53.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000144728 2024-09-16T00:09:53.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.632675+0000", 2024-09-16T00:09:53.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074561819999999996 2024-09-16T00:09:53.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.637821+0000", 2024-09-16T00:09:53.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005145826 2024-09-16T00:09:53.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.637875+0000", 2024-09-16T00:09:53.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4409e-05 2024-09-16T00:09:53.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.637926+0000", 2024-09-16T00:09:53.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1335999999999998e-05 2024-09-16T00:09:53.322 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.322 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.322 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.322 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13802 3.19 3:9a91b182:::100000031d2.00000001:head [write 0~4194304 [2@0]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:53.322 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.627377+0000", 2024-09-16T00:09:53.322 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.5931034080000002, 2024-09-16T00:09:53.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011996005000000001, 2024-09-16T00:09:53.322 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.322 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.323 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.323 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.323 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.323 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.323 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13802 2024-09-16T00:09:53.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.323 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.627377+0000", 2024-09-16T00:09:53.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.627377+0000", 2024-09-16T00:09:53.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.627380+0000", 2024-09-16T00:09:53.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1100000000000001e-06 2024-09-16T00:09:53.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.629935+0000", 2024-09-16T00:09:53.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025558930000000001 2024-09-16T00:09:53.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.629955+0000", 2024-09-16T00:09:53.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.91e-05 2024-09-16T00:09:53.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.629972+0000", 2024-09-16T00:09:53.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7030000000000001e-05 2024-09-16T00:09:53.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.630038+0000", 2024-09-16T00:09:53.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6346000000000005e-05 2024-09-16T00:09:53.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.630319+0000", 2024-09-16T00:09:53.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000281203 2024-09-16T00:09:53.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:53.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.630446+0000", 2024-09-16T00:09:53.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012678900000000001 2024-09-16T00:09:53.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.638524+0000", 2024-09-16T00:09:53.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0080782010000000001 2024-09-16T00:09:53.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.639272+0000", 2024-09-16T00:09:53.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074833199999999999 2024-09-16T00:09:53.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.639316+0000", 2024-09-16T00:09:53.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3231e-05 2024-09-16T00:09:53.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.639373+0000", 2024-09-16T00:09:53.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7769999999999997e-05 2024-09-16T00:09:53.328 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.328 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.328 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.328 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13960 3.4 3:22be2a9e:::10000003224.00000000:head [write 0~4194304 [2@712266]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:53.328 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.982883+0000", 2024-09-16T00:09:53.328 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.2375983189999999, 2024-09-16T00:09:53.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011965287, 2024-09-16T00:09:53.328 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.329 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.329 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.329 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.329 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.329 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.329 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13960 2024-09-16T00:09:53.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.329 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.982883+0000", 2024-09-16T00:09:53.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.982883+0000", 2024-09-16T00:09:53.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.982886+0000", 2024-09-16T00:09:53.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1099999999999999e-06 2024-09-16T00:09:53.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.985563+0000", 2024-09-16T00:09:53.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026772720000000001 2024-09-16T00:09:53.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.985585+0000", 2024-09-16T00:09:53.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2007000000000001e-05 2024-09-16T00:09:53.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.985595+0000", 2024-09-16T00:09:53.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8989999999999996e-06 2024-09-16T00:09:53.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.985660+0000", 2024-09-16T00:09:53.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5259999999999995e-05 2024-09-16T00:09:53.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.985728+0000", 2024-09-16T00:09:53.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8402e-05 2024-09-16T00:09:53.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:53.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.985857+0000", 2024-09-16T00:09:53.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012845099999999999 2024-09-16T00:09:53.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.994664+0000", 2024-09-16T00:09:53.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0088070289999999992 2024-09-16T00:09:53.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.333 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.994729+0000", 2024-09-16T00:09:53.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5295000000000004e-05 2024-09-16T00:09:53.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.333 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.994806+0000", 2024-09-16T00:09:53.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6618000000000004e-05 2024-09-16T00:09:53.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.994848+0000", 2024-09-16T00:09:53.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1943999999999997e-05 2024-09-16T00:09:53.334 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.334 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.334 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.334 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14715 3.3b 3:de17bf99:::1000000335d.00000000:head [write 0~4194304 in=4194304b] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:53.334 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.918859+0000", 2024-09-16T00:09:53.334 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.30162199000000001, 2024-09-16T00:09:53.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011044022000000001, 2024-09-16T00:09:53.334 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.334 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.335 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.335 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.335 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.335 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.335 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14715 2024-09-16T00:09:53.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.335 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.918859+0000", 2024-09-16T00:09:53.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.918859+0000", 2024-09-16T00:09:53.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.918862+0000", 2024-09-16T00:09:53.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5030000000000002e-06 2024-09-16T00:09:53.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921460+0000", 2024-09-16T00:09:53.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025977579999999999 2024-09-16T00:09:53.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921479+0000", 2024-09-16T00:09:53.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9196000000000001e-05 2024-09-16T00:09:53.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921488+0000", 2024-09-16T00:09:53.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2900000000000002e-06 2024-09-16T00:09:53.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921580+0000", 2024-09-16T00:09:53.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2539000000000003e-05 2024-09-16T00:09:53.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921678+0000", 2024-09-16T00:09:53.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8288999999999994e-05 2024-09-16T00:09:53.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:53.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.921839+0000", 2024-09-16T00:09:53.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000160469 2024-09-16T00:09:53.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.927822+0000", 2024-09-16T00:09:53.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059831789999999999 2024-09-16T00:09:53.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.929806+0000", 2024-09-16T00:09:53.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019843769999999998 2024-09-16T00:09:53.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.929860+0000", 2024-09-16T00:09:53.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3171000000000003e-05 2024-09-16T00:09:53.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.340 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.929903+0000", 2024-09-16T00:09:53.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3251000000000003e-05 2024-09-16T00:09:53.340 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.340 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.340 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.340 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.340 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.340 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:53.341 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T00:09:53.342 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:09:53.345 INFO:tasks.workunit.client.0.smithi139.stdout:4/937: sync 2024-09-16T00:09:53.345 INFO:tasks.workunit.client.0.smithi139.stdout:4/938: write d12/d2b/f3a [1580294,104829] 0 2024-09-16T00:09:53.345 INFO:tasks.workunit.client.0.smithi139.stdout:4/939: dread d12/d47/d49/f7e [0,4194304] 0 2024-09-16T00:09:53.346 INFO:tasks.workunit.client.0.smithi139.stdout:4/940: rmdir d12 39 2024-09-16T00:09:53.346 INFO:tasks.workunit.client.0.smithi139.stdout:4/941: symlink d12/d47/db2/d4d/da9/l136 0 2024-09-16T00:09:53.346 INFO:tasks.workunit.client.0.smithi139.stdout:4/942: truncate d12/d47/d49/d60/dac/fd3 344237 0 2024-09-16T00:09:53.346 INFO:tasks.workunit.client.0.smithi139.stdout:4/943: read - d12/d47/db2/d4d/de4/f11c zero size 2024-09-16T00:09:53.346 INFO:tasks.workunit.client.0.smithi139.stdout:4/944: fsync d12/d47/f32 0 2024-09-16T00:09:53.346 INFO:tasks.workunit.client.0.smithi139.stdout:4/945: read f2 [1015302,43372] 0 2024-09-16T00:09:53.346 INFO:tasks.workunit.client.0.smithi139.stdout:4/946: write d12/d47/db2/d4d/de4/ffd [357017,41540] 0 2024-09-16T00:09:53.346 INFO:tasks.workunit.client.0.smithi139.stdout:4/947: rmdir d12/d47/d49/d60 39 2024-09-16T00:09:53.346 INFO:tasks.workunit.client.0.smithi139.stdout:4/948: read d12/d2b/f3a [1270081,53985] 0 2024-09-16T00:09:53.346 INFO:tasks.workunit.client.0.smithi139.stdout:4/949: symlink d12/d47/d49/df1/l137 0 2024-09-16T00:09:53.346 INFO:tasks.workunit.client.0.smithi139.stdout:4/950: mkdir d12/d47/d49/d60/d138 0 2024-09-16T00:09:53.346 INFO:tasks.workunit.client.0.smithi139.stdout:4/951: stat d12/d47/d49/d60/f76 0 2024-09-16T00:09:53.346 INFO:tasks.workunit.client.0.smithi139.stdout:4/952: dwrite d12/d80/f95 [0,4194304] 0 2024-09-16T00:09:53.347 INFO:tasks.workunit.client.0.smithi139.stdout:4/953: mknod d12/d47/db2/c139 0 2024-09-16T00:09:53.347 INFO:tasks.workunit.client.0.smithi139.stdout:4/954: fdatasync d12/d47/d49/ffc 0 2024-09-16T00:09:53.347 INFO:tasks.workunit.client.0.smithi139.stdout:4/955: unlink d12/f42 0 2024-09-16T00:09:53.347 INFO:tasks.workunit.client.0.smithi139.stdout:4/956: getdents d12/d47/d49/d60/dc3/de2/d126 0 2024-09-16T00:09:53.347 INFO:tasks.workunit.client.0.smithi139.stdout:4/957: link d12/d2b/c3c d12/d47/db2/d4d/c13a 0 2024-09-16T00:09:53.347 INFO:tasks.workunit.client.0.smithi139.stdout:4/958: symlink d12/d47/d49/d60/dac/l13b 0 2024-09-16T00:09:53.347 INFO:tasks.workunit.client.0.smithi139.stdout:4/959: chown c11 510755 1 2024-09-16T00:09:53.347 INFO:tasks.workunit.client.0.smithi139.stdout:4/960: truncate d12/d47/db2/d4d/da9/fd2 123734 0 2024-09-16T00:09:53.347 INFO:tasks.workunit.client.0.smithi139.stdout:4/961: chown d12/c2f 238734168 1 2024-09-16T00:09:53.347 INFO:tasks.workunit.client.0.smithi139.stdout:4/962: fdatasync d12/d47/db2/d4d/dd1/fef 0 2024-09-16T00:09:53.347 INFO:tasks.workunit.client.0.smithi139.stdout:4/963: write d12/d47/db2/d4d/d69/fb4 [2264229,51864] 0 2024-09-16T00:09:53.347 INFO:tasks.workunit.client.0.smithi139.stdout:4/964: dwrite d12/d2b/f26 [0,4194304] 0 2024-09-16T00:09:53.347 INFO:tasks.workunit.client.0.smithi139.stdout:4/965: write d12/d47/db2/d4d/d69/f72 [2864260,87675] 0 2024-09-16T00:09:53.347 INFO:tasks.workunit.client.0.smithi139.stdout:4/966: symlink d12/d47/db2/d4d/dd6/l13c 0 2024-09-16T00:09:53.348 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:53.341+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:53.426 INFO:tasks.workunit.client.0.smithi139.stdout:1/736: sync 2024-09-16T00:09:53.426 INFO:tasks.workunit.client.0.smithi139.stdout:6/746: sync 2024-09-16T00:09:53.428 INFO:tasks.workunit.client.0.smithi139.stdout:6/747: read d8/d29/f32 [1362250,25061] 0 2024-09-16T00:09:53.429 INFO:tasks.workunit.client.0.smithi139.stdout:6/748: read d8/f39 [552746,86479] 0 2024-09-16T00:09:53.432 INFO:tasks.workunit.client.0.smithi139.stdout:6/749: mkdir d8/d29/d83/d26/d28/d9e/d10a 0 2024-09-16T00:09:53.435 INFO:tasks.workunit.client.0.smithi139.stdout:6/750: rename d8/d20/l77 to d8/d20/d68/l10b 0 2024-09-16T00:09:53.438 INFO:tasks.workunit.client.0.smithi139.stdout:6/751: rename d8/d29/d83/d26/d45/d65/d6e/c96 to d8/d29/d83/d26/d28/d9e/c10c 0 2024-09-16T00:09:53.439 INFO:tasks.workunit.client.0.smithi139.stdout:1/737: dread d7/d8/d27/f79 [4194304,4194304] 0 2024-09-16T00:09:53.441 INFO:tasks.workunit.client.0.smithi139.stdout:1/738: symlink d7/d8/d27/d68/d6c/ldc 0 2024-09-16T00:09:53.442 INFO:tasks.workunit.client.0.smithi139.stdout:6/752: rename d8/f6f to d8/d29/d83/d26/d45/d65/d8b/f10d 0 2024-09-16T00:09:53.442 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:53.442+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:53.443 INFO:tasks.workunit.client.0.smithi139.stdout:1/739: unlink d7/f3f 0 2024-09-16T00:09:53.444 INFO:tasks.workunit.client.0.smithi139.stdout:6/753: write d8/d29/f36 [992824,28353] 0 2024-09-16T00:09:53.445 INFO:tasks.workunit.client.0.smithi139.stdout:6/754: fdatasync d8/d29/d83/d26/d28/d6d/dd3/de5/ff8 0 2024-09-16T00:09:53.445 INFO:tasks.workunit.client.0.smithi139.stdout:6/755: stat d8/da4/d105 0 2024-09-16T00:09:53.447 INFO:tasks.workunit.client.0.smithi139.stdout:1/740: dread d7/d8/d40/d47/f50 [0,4194304] 0 2024-09-16T00:09:53.448 INFO:tasks.workunit.client.0.smithi139.stdout:6/756: mkdir d8/d56/d10e 0 2024-09-16T00:09:53.449 INFO:tasks.workunit.client.0.smithi139.stdout:1/741: mknod d7/d8/d40/d48/d67/d7d/cdd 0 2024-09-16T00:09:53.451 INFO:tasks.workunit.client.0.smithi139.stdout:6/757: getdents d8/d29/d83 0 2024-09-16T00:09:53.453 INFO:tasks.workunit.client.0.smithi139.stdout:6/758: mkdir d8/d29/db9/d10f 0 2024-09-16T00:09:53.455 INFO:tasks.workunit.client.0.smithi139.stdout:6/759: symlink d8/da4/d105/l110 0 2024-09-16T00:09:53.457 INFO:tasks.workunit.client.0.smithi139.stdout:6/760: symlink d8/d29/d83/d26/d45/d65/d8b/l111 0 2024-09-16T00:09:53.459 INFO:tasks.workunit.client.0.smithi139.stdout:6/761: mkdir d8/d29/db9/d10f/d112 0 2024-09-16T00:09:53.461 INFO:tasks.workunit.client.0.smithi139.stdout:6/762: creat d8/d20/dd1/f113 x:0 0 0 2024-09-16T00:09:53.462 INFO:tasks.workunit.client.0.smithi139.stdout:6/763: write d8/d29/f3d [5224762,70986] 0 2024-09-16T00:09:53.464 INFO:tasks.workunit.client.0.smithi139.stdout:1/742: write d7/f25 [1980899,31067] 0 2024-09-16T00:09:53.467 INFO:tasks.workunit.client.0.smithi139.stdout:1/743: creat d7/d8/d40/d47/dae/fde x:0 0 0 2024-09-16T00:09:53.468 INFO:tasks.workunit.client.0.smithi139.stdout:6/764: dwrite d8/f7b [4194304,4194304] 0 2024-09-16T00:09:53.470 INFO:tasks.workunit.client.0.smithi139.stdout:1/744: mknod d7/d8/d40/d48/d4a/d82/cdf 0 2024-09-16T00:09:53.470 INFO:tasks.workunit.client.0.smithi139.stdout:6/765: write d8/d20/d4d/db0/fc1 [466925,75732] 0 2024-09-16T00:09:53.473 INFO:tasks.workunit.client.0.smithi139.stdout:1/745: truncate d7/d8/d27/d68/fb4 44934 0 2024-09-16T00:09:53.474 INFO:tasks.workunit.client.0.smithi139.stdout:1/746: write d7/d8/d40/d48/d4a/d74/f9f [923650,20709] 0 2024-09-16T00:09:53.477 INFO:tasks.workunit.client.0.smithi139.stdout:1/747: fdatasync d7/f26 0 2024-09-16T00:09:53.477 INFO:tasks.workunit.client.0.smithi139.stdout:6/766: rename d8/d20/c3a to d8/d20/d27/c114 0 2024-09-16T00:09:53.478 INFO:tasks.workunit.client.0.smithi139.stdout:1/748: dread - d7/f81 zero size 2024-09-16T00:09:53.479 INFO:tasks.workunit.client.0.smithi139.stdout:6/767: unlink d8/cf 0 2024-09-16T00:09:53.481 INFO:tasks.workunit.client.0.smithi139.stdout:1/749: write d7/f16 [38679,39852] 0 2024-09-16T00:09:53.482 INFO:tasks.workunit.client.0.smithi139.stdout:6/768: mknod d8/d29/d83/d26/d28/d6d/dd3/de5/c115 0 2024-09-16T00:09:53.483 INFO:tasks.workunit.client.0.smithi139.stdout:1/750: rmdir d7/d8/d40/d48/d53/d61 39 2024-09-16T00:09:53.484 INFO:tasks.workunit.client.0.smithi139.stdout:6/769: fsync d8/d29/d83/f2c 0 2024-09-16T00:09:53.487 INFO:tasks.workunit.client.0.smithi139.stdout:6/770: creat d8/d29/d83/d26/d28/d6d/f116 x:0 0 0 2024-09-16T00:09:53.490 INFO:tasks.workunit.client.0.smithi139.stdout:6/771: fsync d8/d29/d83/d26/f75 0 2024-09-16T00:09:53.493 INFO:tasks.workunit.client.0.smithi139.stdout:6/772: getdents d8/da4/d105 0 2024-09-16T00:09:53.494 INFO:tasks.workunit.client.0.smithi139.stdout:6/773: chown d8/d20/d27/fcf 2130780574 1 2024-09-16T00:09:53.495 INFO:tasks.workunit.client.0.smithi139.stdout:1/751: dwrite d7/f13 [4194304,4194304] 0 2024-09-16T00:09:53.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.499+0000 7fdb89c47640 1 Processor -- start 2024-09-16T00:09:53.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.499+0000 7fdb89c47640 1 -- start start 2024-09-16T00:09:53.501 INFO:tasks.workunit.client.0.smithi139.stdout:5/953: sync 2024-09-16T00:09:53.501 INFO:tasks.workunit.client.0.smithi139.stdout:7/904: sync 2024-09-16T00:09:53.501 INFO:tasks.workunit.client.0.smithi139.stdout:0/889: sync 2024-09-16T00:09:53.501 INFO:tasks.workunit.client.0.smithi139.stdout:3/772: sync 2024-09-16T00:09:53.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.500+0000 7fdb89c47640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdb84151850 0x7fdb840c6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:53.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.500+0000 7fdb89c47640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdb840c7390 0x7fdb840c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:53.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.500+0000 7fdb89c47640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb840c8d30 con 0x7fdb84151c30 2024-09-16T00:09:53.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.500+0000 7fdb89c47640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdb840c88a0 con 0x7fdb84151850 2024-09-16T00:09:53.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.500+0000 7fdb89c47640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fdb840c6a90 con 0x7fdb840c7390 2024-09-16T00:09:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.501+0000 7fdb837fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdb84151850 0x7fdb840c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.501+0000 7fdb837fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdb84151850 0x7fdb840c6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:37596/0 (socket says 172.21.15.139:37596) 2024-09-16T00:09:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.501+0000 7fdb837fe640 1 -- 172.21.15.139:0/1441437250 learned_addr learned my addr 172.21.15.139:0/1441437250 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.501+0000 7fdb83fff640 1 --2- 172.21.15.139:0/1441437250 >> v2:172.21.15.164:3300/0 conn(0x7fdb840c7390 0x7fdb840c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.501+0000 7fdb837fe640 1 -- 172.21.15.139:0/1441437250 >> v2:172.21.15.164:3300/0 conn(0x7fdb840c7390 msgr2=0x7fdb840c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.501+0000 7fdb837fe640 1 --2- 172.21.15.139:0/1441437250 >> v2:172.21.15.164:3300/0 conn(0x7fdb840c7390 0x7fdb840c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.501+0000 7fdb837fe640 1 -- 172.21.15.139:0/1441437250 >> v1:172.21.15.164:6789/0 conn(0x7fdb84151c30 legacy=0x7fdb8415ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.501+0000 7fdb837fe640 1 -- 172.21.15.139:0/1441437250 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb840c67b0 con 0x7fdb84151850 2024-09-16T00:09:53.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.501+0000 7fdb83fff640 1 --2- 172.21.15.139:0/1441437250 >> v2:172.21.15.164:3300/0 conn(0x7fdb840c7390 0x7fdb840c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.502+0000 7fdb837fe640 1 --2- 172.21.15.139:0/1441437250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb84151850 0x7fdb840c6100 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdb8406de60 tx=0x7fdb7402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823c9666267bfc9e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.502+0000 7fdb827fc640 1 -- 172.21.15.139:0/1441437250 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb74037070 con 0x7fdb84151850 2024-09-16T00:09:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.502+0000 7fdb827fc640 1 -- 172.21.15.139:0/1441437250 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb7402ae50 con 0x7fdb84151850 2024-09-16T00:09:53.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.503+0000 7fdb827fc640 1 -- 172.21.15.139:0/1441437250 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb74032860 con 0x7fdb84151850 2024-09-16T00:09:53.504 INFO:tasks.workunit.client.0.smithi139.stdout:6/774: dwrite d8/d29/f32 [0,4194304] 0 2024-09-16T00:09:53.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.504+0000 7fdb89c47640 1 -- 172.21.15.139:0/1441437250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb84151850 msgr2=0x7fdb840c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:53.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.504+0000 7fdb89c47640 1 --2- 172.21.15.139:0/1441437250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb84151850 0x7fdb840c6100 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdb8406de60 tx=0x7fdb7402a300 comp rx=0 tx=0).stop 2024-09-16T00:09:53.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.504+0000 7fdb89c47640 1 -- 172.21.15.139:0/1441437250 shutdown_connections 2024-09-16T00:09:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.504+0000 7fdb89c47640 1 --2- 172.21.15.139:0/1441437250 >> v2:172.21.15.164:3300/0 conn(0x7fdb840c7390 0x7fdb840c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.504+0000 7fdb89c47640 1 --2- 172.21.15.139:0/1441437250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb84151850 0x7fdb840c6100 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.504+0000 7fdb89c47640 1 -- 172.21.15.139:0/1441437250 >> 172.21.15.139:0/1441437250 conn(0x7fdb8406cc30 msgr2=0x7fdb8406d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.504+0000 7fdb89c47640 1 -- 172.21.15.139:0/1441437250 shutdown_connections 2024-09-16T00:09:53.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.504+0000 7fdb89c47640 1 -- 172.21.15.139:0/1441437250 wait complete. 2024-09-16T00:09:53.506 INFO:tasks.workunit.client.0.smithi139.stdout:7/905: unlink d5/d37/c3f 0 2024-09-16T00:09:53.506 INFO:tasks.workunit.client.0.smithi139.stdout:6/775: stat d8/d29/d83/d26 0 2024-09-16T00:09:53.506 INFO:tasks.workunit.client.0.smithi139.stdout:3/773: mknod d5/d7/d23/de4/d75/de1/c106 0 2024-09-16T00:09:53.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.506+0000 7fdb89c47640 1 Processor -- start 2024-09-16T00:09:53.507 INFO:tasks.workunit.client.0.smithi139.stdout:7/906: truncate d5/f85 4634110 0 2024-09-16T00:09:53.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.506+0000 7fdb89c47640 1 -- start start 2024-09-16T00:09:53.508 INFO:tasks.workunit.client.0.smithi139.stdout:5/954: rmdir d1/d13/d126 0 2024-09-16T00:09:53.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.507+0000 7fdb89c47640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb840c7390 0x7fdb84150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:53.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.507+0000 7fdb89c47640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb84151c30 0x7fdb84157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:53.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.507+0000 7fdb89c47640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdb84157e50 0x7fdb8414a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:53.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.507+0000 7fdb89c47640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdb840c8530 con 0x7fdb84151c30 2024-09-16T00:09:53.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.507+0000 7fdb89c47640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdb840c8250 con 0x7fdb840c7390 2024-09-16T00:09:53.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.507+0000 7fdb89c47640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdb8415b320 con 0x7fdb84157e50 2024-09-16T00:09:53.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.508+0000 7fdb83fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdb84157e50 0x7fdb8414a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:53.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.508+0000 7fdb83fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdb84157e50 0x7fdb8414a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:55836/0 (socket says 172.21.15.139:55836) 2024-09-16T00:09:53.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.508+0000 7fdb83fff640 1 -- 172.21.15.139:0/330968976 learned_addr learned my addr 172.21.15.139:0/330968976 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:53.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.509+0000 7fdb837fe640 1 --2- 172.21.15.139:0/330968976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb840c7390 0x7fdb84150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:53.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.510+0000 7fdb82ffd640 1 --2- 172.21.15.139:0/330968976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb84151c30 0x7fdb84157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:53.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.510+0000 7fdb83fff640 1 -- 172.21.15.139:0/330968976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb840c7390 msgr2=0x7fdb84150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:53.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.510+0000 7fdb83fff640 1 --2- 172.21.15.139:0/330968976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb840c7390 0x7fdb84150440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:53.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.510+0000 7fdb83fff640 1 -- 172.21.15.139:0/330968976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb84151c30 msgr2=0x7fdb84157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:53.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.511+0000 7fdb83fff640 1 --2- 172.21.15.139:0/330968976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb84151c30 0x7fdb84157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:53.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.511+0000 7fdb83fff640 1 -- 172.21.15.139:0/330968976 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb841541b0 con 0x7fdb84157e50 2024-09-16T00:09:53.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.511+0000 7fdb83fff640 1 --2- 172.21.15.139:0/330968976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdb84157e50 0x7fdb8414a920 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009eb0 tx=0x7fdb7c010820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:53.512 INFO:tasks.workunit.client.0.smithi139.stdout:3/774: write d5/d7/d23/de4/de8/f45 [1431368,12796] 0 2024-09-16T00:09:53.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.512+0000 7fdb6bfff640 1 -- 172.21.15.139:0/330968976 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c003990 con 0x7fdb84157e50 2024-09-16T00:09:53.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.512+0000 7fdb6bfff640 1 -- 172.21.15.139:0/330968976 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb7c003b30 con 0x7fdb84157e50 2024-09-16T00:09:53.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.512+0000 7fdb89c47640 1 -- 172.21.15.139:0/330968976 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb841538d0 con 0x7fdb84157e50 2024-09-16T00:09:53.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.512+0000 7fdb89c47640 1 -- 172.21.15.139:0/330968976 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb841526f0 con 0x7fdb84157e50 2024-09-16T00:09:53.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.512+0000 7fdb6bfff640 1 -- 172.21.15.139:0/330968976 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c004020 con 0x7fdb84157e50 2024-09-16T00:09:53.514 INFO:tasks.workunit.client.0.smithi139.stdout:5/955: creat d1/d13/d3f/d5e/f14e x:0 0 0 2024-09-16T00:09:53.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.514+0000 7fdb89c47640 1 -- 172.21.15.139:0/330968976 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb840c45a0 con 0x7fdb84157e50 2024-09-16T00:09:53.515 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:53.515 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:53.515 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:53.515 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:53.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.515+0000 7fdb6bfff640 1 -- 172.21.15.139:0/330968976 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdb7c021020 con 0x7fdb84157e50 2024-09-16T00:09:53.516 INFO:tasks.workunit.client.0.smithi139.stdout:0/890: truncate d6/d1c/d41/d44/dd/fd0 2556691 0 2024-09-16T00:09:53.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.516+0000 7fdb6bfff640 1 --2- 172.21.15.139:0/330968976 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdb7006c300 0x7fdb7006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:53.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.516+0000 7fdb6bfff640 1 -- 172.21.15.139:0/330968976 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(90..90 src has 1..90) ==== 5973+0+0 (secure 0 0 0) 0x7fdb7c08fc90 con 0x7fdb84157e50 2024-09-16T00:09:53.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.516+0000 7fdb837fe640 1 --2- 172.21.15.139:0/330968976 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdb7006c300 0x7fdb7006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:53.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.516+0000 7fdb837fe640 1 --2- 172.21.15.139:0/330968976 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdb7006c300 0x7fdb7006e7c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdb74034000 tx=0x7fdb7402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:53.519 INFO:tasks.workunit.client.0.smithi139.stdout:1/752: dread d7/f58 [0,4194304] 0 2024-09-16T00:09:53.521 INFO:tasks.workunit.client.0.smithi139.stdout:6/776: dwrite d8/d29/d83/d26/d45/d65/f2e [0,4194304] 0 2024-09-16T00:09:53.521 INFO:tasks.workunit.client.0.smithi139.stdout:7/907: dwrite d5/d26/d5d/f8e [4194304,4194304] 0 2024-09-16T00:09:53.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.521+0000 7fdb6bfff640 1 -- 172.21.15.139:0/330968976 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdb7c05e3d0 con 0x7fdb84157e50 2024-09-16T00:09:53.524 INFO:tasks.workunit.client.0.smithi139.stdout:3/775: creat d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/d76/f107 x:0 0 0 2024-09-16T00:09:53.525 INFO:tasks.workunit.client.0.smithi139.stdout:0/891: symlink d6/d1c/d41/d44/dd/d8a/d9e/dcb/l117 0 2024-09-16T00:09:53.525 INFO:tasks.workunit.client.0.smithi139.stdout:1/753: mknod d7/d8/d1f/ce0 0 2024-09-16T00:09:53.527 INFO:tasks.workunit.client.0.smithi139.stdout:1/754: chown d7/d8/d40/d47/dae/fc5 2050545 1 2024-09-16T00:09:53.528 INFO:tasks.workunit.client.0.smithi139.stdout:7/908: creat d5/d16/d6d/d7f/dc2/d50/d44/d55/d90/f141 x:0 0 0 2024-09-16T00:09:53.531 INFO:tasks.workunit.client.0.smithi139.stdout:6/777: mkdir d8/d20/d27/d117 0 2024-09-16T00:09:53.532 INFO:tasks.workunit.client.0.smithi139.stdout:3/776: rmdir d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d 39 2024-09-16T00:09:53.533 INFO:tasks.workunit.client.0.smithi139.stdout:1/755: write d7/d8/d40/d46/d8d/f95 [436867,54842] 0 2024-09-16T00:09:53.533 INFO:tasks.workunit.client.0.smithi139.stdout:7/909: truncate d5/d26/d46/d6b/dd6/f13a 312735 0 2024-09-16T00:09:53.535 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:09:53.537 INFO:tasks.workunit.client.0.smithi139.stdout:7/910: chown d5/d16/dc6/f10e 218373735 1 2024-09-16T00:09:53.537 INFO:tasks.workunit.client.0.smithi139.stdout:1/756: chown d7/d8/d40/d48/d4a/d74/f9f 344788146 1 2024-09-16T00:09:53.538 INFO:tasks.workunit.client.0.smithi139.stdout:0/892: truncate d6/d1c/d41/d44/dd/d56/d75/f98 480472 0 2024-09-16T00:09:53.539 INFO:tasks.workunit.client.0.smithi139.stdout:3/777: rename d5/d7/dc/d9c/db0 to d5/d7/dc/d108 0 2024-09-16T00:09:53.540 INFO:tasks.workunit.client.0.smithi139.stdout:5/956: dwrite d1/d21/d49/d53/d6a/d7b/d78/f8e [0,4194304] 0 2024-09-16T00:09:53.543 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:53.542+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:53.543 INFO:tasks.workunit.client.0.smithi139.stdout:0/893: symlink d6/d1c/d41/d78/df2/d10e/df1/l118 0 2024-09-16T00:09:53.546 INFO:tasks.workunit.client.0.smithi139.stdout:7/911: link d5/d16/d6d/d7f/dc2/d50/d44/d55/d90/f108 d5/d16/d6d/d7f/df5/f142 0 2024-09-16T00:09:53.547 INFO:tasks.workunit.client.0.smithi139.stdout:3/778: fdatasync d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d47/f64 0 2024-09-16T00:09:53.548 INFO:tasks.workunit.client.0.smithi139.stdout:1/757: write d7/d8/d40/d47/f50 [2050590,67956] 0 2024-09-16T00:09:53.549 INFO:tasks.workunit.client.0.smithi139.stdout:5/957: write d1/d21/d49/d53/faa [387444,113292] 0 2024-09-16T00:09:53.550 INFO:tasks.workunit.client.0.smithi139.stdout:7/912: mkdir d5/d16/dda/d143 0 2024-09-16T00:09:53.550 INFO:tasks.workunit.client.0.smithi139.stdout:0/894: dread d6/ff [0,4194304] 0 2024-09-16T00:09:53.553 INFO:tasks.workunit.client.0.smithi139.stdout:7/913: fsync d5/d16/dda/fec 0 2024-09-16T00:09:53.555 INFO:tasks.workunit.client.0.smithi139.stdout:0/895: creat d6/d1c/d41/d44/dd/d8a/d9e/dcb/f119 x:0 0 0 2024-09-16T00:09:53.555 INFO:tasks.workunit.client.0.smithi139.stdout:3/779: dwrite d5/d7/fd [4194304,4194304] 0 2024-09-16T00:09:53.563 INFO:tasks.workunit.client.0.smithi139.stdout:3/780: symlink d5/d7/dc/d108/df3/l109 0 2024-09-16T00:09:53.566 INFO:tasks.workunit.client.0.smithi139.stdout:3/781: write d5/f16 [5600223,105710] 0 2024-09-16T00:09:53.569 INFO:tasks.workunit.client.0.smithi139.stdout:3/782: dread - d5/d7/d23/de4/de8/fcb zero size 2024-09-16T00:09:53.569 INFO:tasks.workunit.client.0.smithi139.stdout:7/914: dwrite d5/d16/d6d/d7f/dc2/f3a [0,4194304] 0 2024-09-16T00:09:53.570 INFO:tasks.workunit.client.0.smithi139.stdout:0/896: dwrite d6/d1c/d41/d44/dd/fd0 [0,4194304] 0 2024-09-16T00:09:53.571 INFO:tasks.workunit.client.0.smithi139.stdout:7/915: dread - d5/d16/dc6/f120 zero size 2024-09-16T00:09:53.572 INFO:tasks.workunit.client.0.smithi139.stdout:3/783: fsync d5/d7/d23/de4/de8/da7/d9a/f103 0 2024-09-16T00:09:53.575 INFO:tasks.workunit.client.0.smithi139.stdout:7/916: fdatasync d5/d26/d74/fad 0 2024-09-16T00:09:53.579 INFO:tasks.workunit.client.0.smithi139.stdout:7/917: mknod d5/d26/c144 0 2024-09-16T00:09:53.580 INFO:tasks.workunit.client.0.smithi139.stdout:3/784: creat d5/d7/d23/de4/de8/da7/d9a/f10a x:0 0 0 2024-09-16T00:09:53.581 INFO:tasks.workunit.client.0.smithi139.stdout:7/918: dread - d5/fa6 zero size 2024-09-16T00:09:53.591 INFO:tasks.workunit.client.0.smithi139.stdout:0/897: dwrite d6/d1c/d41/f6a [0,4194304] 0 2024-09-16T00:09:53.591 INFO:tasks.workunit.client.0.smithi139.stdout:3/785: dread d5/f53 [0,4194304] 0 2024-09-16T00:09:53.593 INFO:tasks.workunit.client.0.smithi139.stdout:0/898: write d6/d1c/d41/d63/d82/f99 [369954,55420] 0 2024-09-16T00:09:53.597 INFO:tasks.workunit.client.0.smithi139.stdout:0/899: fdatasync d6/d1c/f4c 0 2024-09-16T00:09:53.597 INFO:tasks.workunit.client.0.smithi139.stdout:0/900: read d6/d1c/d41/d44/d58/fd5 [25889,15568] 0 2024-09-16T00:09:53.603 INFO:tasks.workunit.client.0.smithi139.stdout:7/919: mkdir d5/d26/d46/d6b/dd6/d145 0 2024-09-16T00:09:53.605 INFO:tasks.workunit.client.0.smithi139.stdout:0/901: rename d6/d1c/d41/l9a to d6/d1c/d41/d44/dce/l11a 0 2024-09-16T00:09:53.606 INFO:tasks.workunit.client.0.smithi139.stdout:7/920: write d5/d16/d6d/d7f/dc2/dab/fe7 [173160,8270] 0 2024-09-16T00:09:53.607 INFO:tasks.workunit.client.0.smithi139.stdout:3/786: mkdir d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/db3/d10b 0 2024-09-16T00:09:53.609 INFO:tasks.workunit.client.0.smithi139.stdout:0/902: getdents d6/d1c/d41/d78/df2/d10e 0 2024-09-16T00:09:53.613 INFO:tasks.workunit.client.0.smithi139.stdout:3/787: rename d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d47/c9f to d5/d5f/c10c 0 2024-09-16T00:09:53.613 INFO:tasks.workunit.client.0.smithi139.stdout:7/921: dwrite f4 [8388608,4194304] 0 2024-09-16T00:09:53.614 INFO:tasks.workunit.client.0.smithi139.stdout:0/903: symlink d6/d1c/d41/d44/dd/d56/d75/de1/l11b 0 2024-09-16T00:09:53.614 INFO:tasks.workunit.client.0.smithi139.stdout:3/788: chown d5/d7/d23/de4/de8/da7/l3f 1601289 1 2024-09-16T00:09:53.619 INFO:tasks.workunit.client.0.smithi139.stdout:3/789: rename d5/d7/d23/f17 to d5/d7/dc/d9c/f10d 0 2024-09-16T00:09:53.622 INFO:tasks.workunit.client.0.smithi139.stdout:0/904: rename d6/f52 to d6/d1c/d41/d44/f11c 0 2024-09-16T00:09:53.623 INFO:tasks.workunit.client.0.smithi139.stdout:3/790: creat d5/d7/dc/d84/f10e x:0 0 0 2024-09-16T00:09:53.627 INFO:tasks.workunit.client.0.smithi139.stdout:0/905: rename d6/d1c/d41/d7c/d86/f93 to d6/d1c/d41/d44/dd/f11d 0 2024-09-16T00:09:53.628 INFO:tasks.workunit.client.0.smithi139.stdout:3/791: mknod d5/d7/d23/de4/de8/da7/d9a/d101/c10f 0 2024-09-16T00:09:53.630 INFO:tasks.workunit.client.0.smithi139.stdout:0/906: write d6/d1c/d41/d44/dd/f2c [222392,8277] 0 2024-09-16T00:09:53.632 INFO:tasks.workunit.client.0.smithi139.stdout:7/922: dread d5/d16/d6d/d7f/dc2/d50/fbd [0,4194304] 0 2024-09-16T00:09:53.633 INFO:tasks.workunit.client.0.smithi139.stdout:3/792: truncate d5/d7/d23/de4/de8/da7/d9a/f10a 892962 0 2024-09-16T00:09:53.636 INFO:tasks.workunit.client.0.smithi139.stdout:7/923: mknod d5/d26/d46/dea/d126/c146 0 2024-09-16T00:09:53.638 INFO:tasks.workunit.client.0.smithi139.stdout:0/907: dwrite d6/d1c/d41/d63/d82/f83 [4194304,4194304] 0 2024-09-16T00:09:53.640 INFO:tasks.workunit.client.0.smithi139.stdout:4/967: sync 2024-09-16T00:09:53.640 INFO:tasks.workunit.client.0.smithi139.stdout:3/793: creat d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/da1/f110 x:0 0 0 2024-09-16T00:09:53.643 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:53.643+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:53.643 INFO:tasks.workunit.client.0.smithi139.stdout:3/794: truncate d5/d7/d23/de4/de8/da7/d9a/d101/f8d 5231402 0 2024-09-16T00:09:53.644 INFO:tasks.workunit.client.0.smithi139.stdout:4/968: mknod d12/d47/db2/d4d/dd6/c13d 0 2024-09-16T00:09:53.644 INFO:tasks.workunit.client.0.smithi139.stdout:0/908: dread d6/d1c/f8c [0,4194304] 0 2024-09-16T00:09:53.645 INFO:tasks.workunit.client.0.smithi139.stdout:0/909: chown d6/d5a/f6f 20448146 1 2024-09-16T00:09:53.646 INFO:tasks.workunit.client.0.smithi139.stdout:4/969: write d12/d47/db2/d4d/f89 [4715380,36137] 0 2024-09-16T00:09:53.671 INFO:tasks.workunit.client.0.smithi139.stdout:0/910: read - d6/d1c/d41/f116 zero size 2024-09-16T00:09:53.673 INFO:tasks.workunit.client.0.smithi139.stdout:0/911: read d6/d1c/f8d [2202652,607] 0 2024-09-16T00:09:53.677 INFO:tasks.workunit.client.0.smithi139.stdout:7/924: rmdir d5/d26/d46/d6b/dc5/d12a 0 2024-09-16T00:09:53.682 INFO:tasks.workunit.client.0.smithi139.stdout:7/925: dread d5/d16/d6d/fd0 [0,4194304] 0 2024-09-16T00:09:53.683 INFO:tasks.workunit.client.0.smithi139.stdout:0/912: creat d6/d1c/d41/d78/f11e x:0 0 0 2024-09-16T00:09:53.685 INFO:tasks.workunit.client.0.smithi139.stdout:3/795: link d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/ld8 d5/d7/d23/de4/de8/da7/l111 0 2024-09-16T00:09:53.686 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:53.686 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:53.686 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:53.686 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:53.686 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:53.687 INFO:tasks.workunit.client.0.smithi139.stdout:0/913: fdatasync d6/f64 0 2024-09-16T00:09:53.690 INFO:tasks.workunit.client.0.smithi139.stdout:7/926: dwrite d5/d16/d22/ffb [0,4194304] 0 2024-09-16T00:09:53.693 INFO:tasks.workunit.client.0.smithi139.stdout:3/796: mkdir d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d112 0 2024-09-16T00:09:53.701 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:09:53.702 INFO:tasks.workunit.client.0.smithi139.stdout:3/797: symlink d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/d76/dfb/l113 0 2024-09-16T00:09:53.704 INFO:tasks.workunit.client.0.smithi139.stdout:0/914: getdents d6/d1c/d41/d44/dd/d8a/d9e 0 2024-09-16T00:09:53.705 INFO:tasks.workunit.client.0.smithi139.stdout:0/915: write d6/d1c/f8d [4448574,31158] 0 2024-09-16T00:09:53.707 INFO:tasks.workunit.client.0.smithi139.stdout:3/798: creat d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/db9/f114 x:0 0 0 2024-09-16T00:09:53.710 INFO:tasks.workunit.client.0.smithi139.stdout:3/799: dread - d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/db9/feb zero size 2024-09-16T00:09:53.711 INFO:tasks.workunit.client.0.smithi139.stdout:4/970: mknod d12/d47/db2/d4d/de4/d7f/c13e 0 2024-09-16T00:09:53.713 INFO:tasks.workunit.client.0.smithi139.stdout:7/927: dread d5/d37/f47 [0,4194304] 0 2024-09-16T00:09:53.715 INFO:tasks.workunit.client.0.smithi139.stdout:4/971: creat d12/d47/d49/d60/dc3/de2/f13f x:0 0 0 2024-09-16T00:09:53.717 INFO:tasks.workunit.client.0.smithi139.stdout:4/972: truncate f7 4801535 0 2024-09-16T00:09:53.719 INFO:tasks.workunit.client.0.smithi139.stdout:7/928: unlink d5/d16/d6d/d7f/dc2/c33 0 2024-09-16T00:09:53.721 INFO:tasks.workunit.client.0.smithi139.stdout:7/929: write d5/d16/d22/d25/d63/fee [2686279,77120] 0 2024-09-16T00:09:53.722 INFO:tasks.workunit.client.0.smithi139.stdout:3/800: dwrite d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/fe6 [0,4194304] 0 2024-09-16T00:09:53.723 INFO:tasks.workunit.client.0.smithi139.stdout:3/801: stat d5/d7/d23/de4/de8/da7 0 2024-09-16T00:09:53.727 INFO:tasks.workunit.client.0.smithi139.stdout:7/930: mknod d5/d26/d74/dfd/d137/c147 0 2024-09-16T00:09:53.729 INFO:tasks.workunit.client.0.smithi139.stdout:3/802: truncate d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/d76/f107 964991 0 2024-09-16T00:09:53.734 INFO:tasks.workunit.client.0.smithi139.stdout:7/931: getdents d5/d26/d42/dae 0 2024-09-16T00:09:53.740 INFO:tasks.workunit.client.0.smithi139.stdout:0/916: link d6/d1c/d41/d44/d6b/c114 d6/d1c/d41/d63/c11f 0 2024-09-16T00:09:53.742 INFO:tasks.workunit.client.0.smithi139.stdout:0/917: mkdir d6/d42/d120 0 2024-09-16T00:09:53.744 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:53.744+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:53.745 INFO:tasks.workunit.client.0.smithi139.stdout:0/918: fsync d6/d1c/d41/d44/dd/f39 0 2024-09-16T00:09:53.746 INFO:tasks.workunit.client.0.smithi139.stdout:7/932: dwrite d5/d26/d46/d6b/dd6/f13a [0,4194304] 0 2024-09-16T00:09:53.746 INFO:tasks.workunit.client.0.smithi139.stdout:0/919: readlink d6/le3 0 2024-09-16T00:09:53.751 INFO:tasks.workunit.client.0.smithi139.stdout:7/933: rename d5/d26/d46/d6b/dd6/df4/c11f to d5/d26/d5d/c148 0 2024-09-16T00:09:53.752 INFO:tasks.workunit.client.0.smithi139.stdout:0/920: creat d6/d1c/d41/d78/df2/d10e/df1/f121 x:0 0 0 2024-09-16T00:09:53.753 INFO:tasks.workunit.client.0.smithi139.stdout:0/921: chown d6/d5a/d70 0 1 2024-09-16T00:09:53.753 INFO:tasks.workunit.client.0.smithi139.stdout:7/934: creat d5/d37/f149 x:0 0 0 2024-09-16T00:09:53.755 INFO:tasks.workunit.client.0.smithi139.stdout:7/935: fsync d5/f85 0 2024-09-16T00:09:53.755 INFO:tasks.workunit.client.0.smithi139.stdout:7/936: chown d5 1070811195 1 2024-09-16T00:09:53.756 INFO:tasks.workunit.client.0.smithi139.stdout:0/922: symlink d6/d1c/d41/d78/df2/d10e/df1/l122 0 2024-09-16T00:09:53.757 INFO:tasks.workunit.client.0.smithi139.stdout:3/803: write d5/d7/dc2/fd7 [4601320,102288] 0 2024-09-16T00:09:53.759 INFO:tasks.workunit.client.0.smithi139.stdout:3/804: write d5/d7/fd [1661295,125170] 0 2024-09-16T00:09:53.760 INFO:tasks.workunit.client.0.smithi139.stdout:7/937: dread d5/d26/d46/d6b/dd6/f13a [0,4194304] 0 2024-09-16T00:09:53.761 INFO:tasks.workunit.client.0.smithi139.stdout:4/973: creat d12/d47/db2/d4d/f140 x:0 0 0 2024-09-16T00:09:53.761 INFO:tasks.workunit.client.0.smithi139.stdout:4/974: chown d12/c65 6104503 1 2024-09-16T00:09:53.762 INFO:tasks.workunit.client.0.smithi139.stdout:3/805: fsync d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/fcc 0 2024-09-16T00:09:53.764 INFO:tasks.workunit.client.0.smithi139.stdout:6/778: sync 2024-09-16T00:09:53.765 INFO:tasks.workunit.client.0.smithi139.stdout:7/938: creat d5/d26/d46/d6b/dd6/df4/f14a x:0 0 0 2024-09-16T00:09:53.766 INFO:tasks.workunit.client.0.smithi139.stdout:4/975: write d12/d2b/d8c/dff/f11d [633527,4122] 0 2024-09-16T00:09:53.767 INFO:tasks.workunit.client.0.smithi139.stdout:3/806: creat d5/d7/d23/de4/d75/f115 x:0 0 0 2024-09-16T00:09:53.768 INFO:tasks.workunit.client.0.smithi139.stdout:7/939: write d5/d16/d22/d25/d63/fee [1392380,60992] 0 2024-09-16T00:09:53.768 INFO:tasks.workunit.client.0.smithi139.stdout:6/779: write d8/d29/d83/d26/f4b [2870321,89667] 0 2024-09-16T00:09:53.770 INFO:tasks.workunit.client.0.smithi139.stdout:3/807: write d5/d7/fbf [1420811,107081] 0 2024-09-16T00:09:53.771 INFO:tasks.workunit.client.0.smithi139.stdout:4/976: dread d12/d47/fb3 [0,4194304] 0 2024-09-16T00:09:53.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:53.806+0000 7fdb89c47640 1 -- 172.21.15.139:0/330968976 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7fdb8406bea0 con 0x7fdb84157e50 2024-09-16T00:09:53.811 INFO:tasks.workunit.client.0.smithi139.stdout:7/940: unlink d5/d16/d6d/d7f/dc2/c2f 0 2024-09-16T00:09:53.812 INFO:tasks.workunit.client.0.smithi139.stdout:4/977: creat d12/d47/db2/d4d/d69/f141 x:0 0 0 2024-09-16T00:09:53.812 INFO:tasks.workunit.client.0.smithi139.stdout:7/941: fsync d5/d37/d118/f127 0 2024-09-16T00:09:53.814 INFO:tasks.workunit.client.0.smithi139.stdout:4/978: readlink d12/d47/db2/d4d/de4/d7f/lb9 0 2024-09-16T00:09:53.815 INFO:tasks.workunit.client.0.smithi139.stdout:3/808: write d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4d/f60 [2176351,80423] 0 2024-09-16T00:09:53.816 INFO:tasks.workunit.client.0.smithi139.stdout:6/780: dwrite d8/d20/d4d/f8e [4194304,4194304] 0 2024-09-16T00:09:53.816 INFO:tasks.workunit.client.0.smithi139.stdout:7/942: truncate d5/d37/d70/f13e 631831 0 2024-09-16T00:09:53.820 INFO:tasks.workunit.client.0.smithi139.stdout:6/781: truncate d8/f39 1183457 0 2024-09-16T00:09:53.822 INFO:tasks.workunit.client.0.smithi139.stdout:7/943: symlink d5/d26/d74/dfd/l14b 0 2024-09-16T00:09:53.824 INFO:tasks.workunit.client.0.smithi139.stdout:4/979: dwrite d12/d2b/ff2 [0,4194304] 0 2024-09-16T00:09:53.825 INFO:tasks.workunit.client.0.smithi139.stdout:7/944: write d5/d16/d22/d25/d63/f97 [853019,84243] 0 2024-09-16T00:09:53.826 INFO:tasks.workunit.client.0.smithi139.stdout:3/809: dread d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/f79 [0,4194304] 0 2024-09-16T00:09:53.827 INFO:tasks.workunit.client.0.smithi139.stdout:6/782: rename d8/f7b to d8/d29/d83/d26/d45/d85/f118 0 2024-09-16T00:09:53.831 INFO:tasks.workunit.client.0.smithi139.stdout:7/945: rename d5/d16/d22/d25/f76 to d5/d26/d5d/f14c 0 2024-09-16T00:09:53.835 INFO:tasks.workunit.client.0.smithi139.stdout:7/946: truncate d5/d16/d6d/d7f/dc2/d81/fcb 257415 0 2024-09-16T00:09:53.837 INFO:tasks.workunit.client.0.smithi139.stdout:7/947: write d5/d26/d46/d6b/dd6/df4/f14a [960737,15910] 0 2024-09-16T00:09:53.838 INFO:tasks.workunit.client.0.smithi139.stdout:4/980: dread f2 [0,4194304] 0 2024-09-16T00:09:53.841 INFO:tasks.workunit.client.0.smithi139.stdout:7/948: creat d5/f14d x:0 0 0 2024-09-16T00:09:53.841 INFO:tasks.workunit.client.0.smithi139.stdout:4/981: truncate d12/d47/d49/f112 1154400 0 2024-09-16T00:09:53.841 INFO:tasks.workunit.client.0.smithi139.stdout:3/810: dread d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/f42 [0,4194304] 0 2024-09-16T00:09:53.843 INFO:tasks.workunit.client.0.smithi139.stdout:7/949: rmdir d5/d26/d46/dea 39 2024-09-16T00:09:53.843 INFO:tasks.workunit.client.0.smithi139.stdout:4/982: write d12/d47/d49/d60/dc3/de2/f13f [525356,119686] 0 2024-09-16T00:09:53.845 INFO:tasks.workunit.client.0.smithi139.stdout:3/811: mknod d5/d7/d23/de4/de8/da7/d9a/d101/c116 0 2024-09-16T00:09:53.845 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:53.845+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:53.846 INFO:tasks.workunit.client.0.smithi139.stdout:7/950: dread d5/d16/d6d/d7f/dc2/d81/fcb [0,4194304] 0 2024-09-16T00:09:53.846 INFO:tasks.workunit.client.0.smithi139.stdout:6/783: dread d8/d29/d83/d26/d45/d85/ff2 [0,4194304] 0 2024-09-16T00:09:53.847 INFO:tasks.workunit.client.0.smithi139.stdout:4/983: getdents d12/d47/d49/d60/dac/d12f 0 2024-09-16T00:09:53.849 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:53.849 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:53.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:53.850 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:53.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.850 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5850 3.3a 3:5c3cfc2b:::10000001adf.00000000:head [write 0~116] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:53.850 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:09:53.850 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 38.431384387000001, 2024-09-16T00:09:53.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66931295800000001, 2024-09-16T00:09:53.850 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.850 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.850 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.850 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.850 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.850 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.850 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5850 2024-09-16T00:09:53.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.851 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:09:53.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:09:53.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415938+0000", 2024-09-16T00:09:53.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7099999999999999e-06 2024-09-16T00:09:53.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415945+0000", 2024-09-16T00:09:53.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:09:53.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415959+0000", 2024-09-16T00:09:53.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3545e-05 2024-09-16T00:09:53.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415966+0000", 2024-09-16T00:09:53.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6379999999999998e-06 2024-09-16T00:09:53.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416013+0000", 2024-09-16T00:09:53.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7314999999999997e-05 2024-09-16T00:09:53.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:53.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416020+0000", 2024-09-16T00:09:53.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.52e-06 2024-09-16T00:09:53.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082393+0000", 2024-09-16T00:09:53.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66637251799999997 2024-09-16T00:09:53.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082537+0000", 2024-09-16T00:09:53.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014418800000000001 2024-09-16T00:09:53.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:53.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082689+0000", 2024-09-16T00:09:53.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015198499999999999 2024-09-16T00:09:53.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084930+0000", 2024-09-16T00:09:53.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002241041 2024-09-16T00:09:53.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084994+0000", 2024-09-16T00:09:53.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3274000000000004e-05 2024-09-16T00:09:53.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085049+0000", 2024-09-16T00:09:53.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.554e-05 2024-09-16T00:09:53.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085250+0000", 2024-09-16T00:09:53.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020060799999999999 2024-09-16T00:09:53.856 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.856 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.856 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.856 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5910 3.3a 3:5ec98a62:::10000001b22.00000000:head [write 0~1274] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:53.856 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:09:53.857 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 38.284440748999998, 2024-09-16T00:09:53.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.52286026600000002, 2024-09-16T00:09:53.857 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.857 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.857 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.857 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.857 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.857 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.857 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5910 2024-09-16T00:09:53.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.857 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:09:53.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:09:53.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562881+0000", 2024-09-16T00:09:53.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:09:53.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562887+0000", 2024-09-16T00:09:53.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2870000000000004e-06 2024-09-16T00:09:53.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562897+0000", 2024-09-16T00:09:53.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.064e-05 2024-09-16T00:09:53.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562902+0000", 2024-09-16T00:09:53.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2340000000000002e-06 2024-09-16T00:09:53.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562958+0000", 2024-09-16T00:09:53.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5523999999999999e-05 2024-09-16T00:09:53.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:53.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562968+0000", 2024-09-16T00:09:53.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0394000000000001e-05 2024-09-16T00:09:53.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083461+0000", 2024-09-16T00:09:53.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.520492968 2024-09-16T00:09:53.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083645+0000", 2024-09-16T00:09:53.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018346399999999999 2024-09-16T00:09:53.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:53.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083844+0000", 2024-09-16T00:09:53.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000199268 2024-09-16T00:09:53.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085410+0000", 2024-09-16T00:09:53.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015662359999999999 2024-09-16T00:09:53.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085493+0000", 2024-09-16T00:09:53.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2478999999999994e-05 2024-09-16T00:09:53.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085563+0000", 2024-09-16T00:09:53.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0159000000000004e-05 2024-09-16T00:09:53.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085741+0000", 2024-09-16T00:09:53.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017764300000000001 2024-09-16T00:09:53.863 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.863 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.863 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.863 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5962 3.3a 3:5c7b4d29:::10000001b5d.00000000:head [write 0~3074] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:53.863 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:09:53.863 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 38.154903932000003, 2024-09-16T00:09:53.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39368737500000001, 2024-09-16T00:09:53.863 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.863 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.864 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.864 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.864 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.864 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.864 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5962 2024-09-16T00:09:53.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.864 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.864 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.864 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:09:53.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:09:53.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692418+0000", 2024-09-16T00:09:53.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3510000000000001e-06 2024-09-16T00:09:53.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692425+0000", 2024-09-16T00:09:53.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3770000000000001e-06 2024-09-16T00:09:53.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692440+0000", 2024-09-16T00:09:53.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4975e-05 2024-09-16T00:09:53.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692448+0000", 2024-09-16T00:09:53.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6279999999999994e-06 2024-09-16T00:09:53.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692499+0000", 2024-09-16T00:09:53.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0872999999999999e-05 2024-09-16T00:09:53.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:53.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692507+0000", 2024-09-16T00:09:53.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9889999999999992e-06 2024-09-16T00:09:53.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084221+0000", 2024-09-16T00:09:53.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39171379699999997 2024-09-16T00:09:53.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084446+0000", 2024-09-16T00:09:53.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022519900000000001 2024-09-16T00:09:53.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.868 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:53.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084672+0000", 2024-09-16T00:09:53.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000225405 2024-09-16T00:09:53.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.868 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085832+0000", 2024-09-16T00:09:53.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011600040000000001 2024-09-16T00:09:53.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.868 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085864+0000", 2024-09-16T00:09:53.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2592999999999998e-05 2024-09-16T00:09:53.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085943+0000", 2024-09-16T00:09:53.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8534000000000004e-05 2024-09-16T00:09:53.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086105+0000", 2024-09-16T00:09:53.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016165 2024-09-16T00:09:53.869 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.869 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.870 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.870 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5963 3.3a 3:5e4192f5:::10000001b5e.00000000:head [write 0~152] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:53.870 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:53.870 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 38.152839501000003, 2024-09-16T00:09:53.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39413463900000001, 2024-09-16T00:09:53.870 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.870 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.870 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.870 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.870 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.870 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.870 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5963 2024-09-16T00:09:53.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.871 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:53.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:53.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:53.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:09:53.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.872 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694486+0000", 2024-09-16T00:09:53.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0960000000000003e-06 2024-09-16T00:09:53.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.872 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694496+0000", 2024-09-16T00:09:53.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2189999999999993e-06 2024-09-16T00:09:53.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694500+0000", 2024-09-16T00:09:53.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9529999999999998e-06 2024-09-16T00:09:53.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694552+0000", 2024-09-16T00:09:53.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1360999999999998e-05 2024-09-16T00:09:53.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:53.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694561+0000", 2024-09-16T00:09:53.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1700000000000003e-06 2024-09-16T00:09:53.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086346+0000", 2024-09-16T00:09:53.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39178468 2024-09-16T00:09:53.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086480+0000", 2024-09-16T00:09:53.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013469299999999999 2024-09-16T00:09:53.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:53.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086790+0000", 2024-09-16T00:09:53.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030923300000000001 2024-09-16T00:09:53.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088361+0000", 2024-09-16T00:09:53.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015717050000000001 2024-09-16T00:09:53.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088432+0000", 2024-09-16T00:09:53.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0922999999999998e-05 2024-09-16T00:09:53.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.876 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088489+0000", 2024-09-16T00:09:53.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6653999999999997e-05 2024-09-16T00:09:53.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.876 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.876 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088616+0000", 2024-09-16T00:09:53.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012730099999999999 2024-09-16T00:09:53.876 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.876 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.876 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.876 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:6082 3.3a 3:5c50f65b:::10000001be7.00000000:head [write 0~2439] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:53.877 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:09:53.877 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.859066644999999, 2024-09-16T00:09:53.877 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.10065850799999999, 2024-09-16T00:09:53.877 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.877 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.877 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.877 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.877 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.877 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.877 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 6082 2024-09-16T00:09:53.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.877 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:09:53.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:09:53.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988256+0000", 2024-09-16T00:09:53.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.065e-06 2024-09-16T00:09:53.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988262+0000", 2024-09-16T00:09:53.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4640000000000003e-06 2024-09-16T00:09:53.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988272+0000", 2024-09-16T00:09:53.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0335e-05 2024-09-16T00:09:53.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988279+0000", 2024-09-16T00:09:53.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:09:53.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988324+0000", 2024-09-16T00:09:53.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4953000000000001e-05 2024-09-16T00:09:53.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:53.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988331+0000", 2024-09-16T00:09:53.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5989999999999998e-06 2024-09-16T00:09:53.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087272+0000", 2024-09-16T00:09:53.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.098941045000000005 2024-09-16T00:09:53.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087486+0000", 2024-09-16T00:09:53.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00021360199999999999 2024-09-16T00:09:53.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:53.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087740+0000", 2024-09-16T00:09:53.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00025456799999999999 2024-09-16T00:09:53.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088638+0000", 2024-09-16T00:09:53.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00089793900000000005 2024-09-16T00:09:53.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088673+0000", 2024-09-16T00:09:53.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5111999999999999e-05 2024-09-16T00:09:53.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088722+0000", 2024-09-16T00:09:53.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8758000000000003e-05 2024-09-16T00:09:53.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088913+0000", 2024-09-16T00:09:53.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000190953 2024-09-16T00:09:53.883 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.883 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.883 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.883 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10643 3.21 3:8453163e:::100000030b3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:53.883 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:09:53.883 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.247590452000001, 2024-09-16T00:09:53.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013028809000000001, 2024-09-16T00:09:53.884 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.884 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.884 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.884 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.884 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.884 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.884 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10643 2024-09-16T00:09:53.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.884 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.884 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:09:53.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:09:53.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599734+0000", 2024-09-16T00:09:53.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-06 2024-09-16T00:09:53.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603345+0000", 2024-09-16T00:09:53.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003611628 2024-09-16T00:09:53.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603368+0000", 2024-09-16T00:09:53.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3017e-05 2024-09-16T00:09:53.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603379+0000", 2024-09-16T00:09:53.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0528e-05 2024-09-16T00:09:53.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603448+0000", 2024-09-16T00:09:53.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9080999999999998e-05 2024-09-16T00:09:53.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603623+0000", 2024-09-16T00:09:53.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017525599999999999 2024-09-16T00:09:53.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:53.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603766+0000", 2024-09-16T00:09:53.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000142977 2024-09-16T00:09:53.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.611250+0000", 2024-09-16T00:09:53.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074835240000000001 2024-09-16T00:09:53.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612617+0000", 2024-09-16T00:09:53.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001367793 2024-09-16T00:09:53.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612700+0000", 2024-09-16T00:09:53.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3123999999999999e-05 2024-09-16T00:09:53.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612759+0000", 2024-09-16T00:09:53.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8950999999999999e-05 2024-09-16T00:09:53.889 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.889 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.889 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.889 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10652 3.22 3:44e46336:::100000030c1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:53.889 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:09:53.889 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.21953882, 2024-09-16T00:09:53.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011405732, 2024-09-16T00:09:53.890 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.890 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.890 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.890 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.890 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.890 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.890 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10652 2024-09-16T00:09:53.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.890 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:09:53.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:09:53.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627785+0000", 2024-09-16T00:09:53.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6970000000000002e-06 2024-09-16T00:09:53.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629610+0000", 2024-09-16T00:09:53.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018250359999999999 2024-09-16T00:09:53.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.892 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629639+0000", 2024-09-16T00:09:53.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8676000000000001e-05 2024-09-16T00:09:53.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.892 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629652+0000", 2024-09-16T00:09:53.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3572e-05 2024-09-16T00:09:53.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629735+0000", 2024-09-16T00:09:53.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.25e-05 2024-09-16T00:09:53.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629925+0000", 2024-09-16T00:09:53.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019011000000000001 2024-09-16T00:09:53.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:53.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.630075+0000", 2024-09-16T00:09:53.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000149841 2024-09-16T00:09:53.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.636470+0000", 2024-09-16T00:09:53.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063952289999999997 2024-09-16T00:09:53.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639098+0000", 2024-09-16T00:09:53.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026277980000000002 2024-09-16T00:09:53.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639154+0000", 2024-09-16T00:09:53.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6413999999999999e-05 2024-09-16T00:09:53.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639188+0000", 2024-09-16T00:09:53.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3859000000000002e-05 2024-09-16T00:09:53.895 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.895 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.895 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.896 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10951 3.22 3:44fbf03a:::10000003157.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:53.896 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.497985+0000", 2024-09-16T00:09:53.896 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.349336234999999, 2024-09-16T00:09:53.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010873974999999999, 2024-09-16T00:09:53.896 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.896 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.896 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.896 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.896 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.896 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.896 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10951 2024-09-16T00:09:53.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.896 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.897 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.497985+0000", 2024-09-16T00:09:53.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.897 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.497985+0000", 2024-09-16T00:09:53.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.897 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.497988+0000", 2024-09-16T00:09:53.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9749999999999999e-06 2024-09-16T00:09:53.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.499736+0000", 2024-09-16T00:09:53.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001747845 2024-09-16T00:09:53.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.499771+0000", 2024-09-16T00:09:53.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5199e-05 2024-09-16T00:09:53.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.499785+0000", 2024-09-16T00:09:53.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.364e-05 2024-09-16T00:09:53.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.499893+0000", 2024-09-16T00:09:53.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108348 2024-09-16T00:09:53.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.500019+0000", 2024-09-16T00:09:53.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012591699999999999 2024-09-16T00:09:53.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:53.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.500150+0000", 2024-09-16T00:09:53.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000131163 2024-09-16T00:09:53.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.504740+0000", 2024-09-16T00:09:53.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045898140000000002 2024-09-16T00:09:53.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.508724+0000", 2024-09-16T00:09:53.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039840220000000003 2024-09-16T00:09:53.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.508794+0000", 2024-09-16T00:09:53.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9793000000000001e-05 2024-09-16T00:09:53.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.508859+0000", 2024-09-16T00:09:53.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5259000000000007e-05 2024-09-16T00:09:53.901 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.901 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.901 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.902 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11217 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:53.902 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:09:53.902 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 13.273041799, 2024-09-16T00:09:53.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01231253, 2024-09-16T00:09:53.902 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.902 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.902 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.902 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.902 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.902 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.902 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11217 2024-09-16T00:09:53.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.903 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:09:53.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:09:53.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574282+0000", 2024-09-16T00:09:53.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8499999999999998e-06 2024-09-16T00:09:53.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575804+0000", 2024-09-16T00:09:53.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001521859 2024-09-16T00:09:53.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575826+0000", 2024-09-16T00:09:53.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2059999999999999e-05 2024-09-16T00:09:53.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575837+0000", 2024-09-16T00:09:53.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1313e-05 2024-09-16T00:09:53.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575920+0000", 2024-09-16T00:09:53.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2918999999999997e-05 2024-09-16T00:09:53.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576088+0000", 2024-09-16T00:09:53.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016803799999999999 2024-09-16T00:09:53.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:53.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576233+0000", 2024-09-16T00:09:53.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014508199999999999 2024-09-16T00:09:53.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.580500+0000", 2024-09-16T00:09:53.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004266644 2024-09-16T00:09:53.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586479+0000", 2024-09-16T00:09:53.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059791820000000004 2024-09-16T00:09:53.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586541+0000", 2024-09-16T00:09:53.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1463000000000004e-05 2024-09-16T00:09:53.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586592+0000", 2024-09-16T00:09:53.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1119999999999998e-05 2024-09-16T00:09:53.907 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.908 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.908 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.908 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11767 3.38 3:1fa50e65:::10000003117.00000000:head [write 0~4194304 [3@1169732]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:53.908 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:09:53.908 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 10.683637264, 2024-09-16T00:09:53.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011555678999999999, 2024-09-16T00:09:53.908 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.908 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.908 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.908 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.908 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.909 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.909 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11767 2024-09-16T00:09:53.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.909 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.909 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.909 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.909 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:09:53.909 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.909 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.909 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.909 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:09:53.909 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.910 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.910 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163686+0000", 2024-09-16T00:09:53.910 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3259999999999998e-06 2024-09-16T00:09:53.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.910 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.910 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165053+0000", 2024-09-16T00:09:53.910 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013665859999999999 2024-09-16T00:09:53.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.910 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.910 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165075+0000", 2024-09-16T00:09:53.910 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2206999999999999e-05 2024-09-16T00:09:53.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165084+0000", 2024-09-16T00:09:53.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0710000000000001e-06 2024-09-16T00:09:53.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165157+0000", 2024-09-16T00:09:53.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2619000000000004e-05 2024-09-16T00:09:53.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165249+0000", 2024-09-16T00:09:53.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2517999999999997e-05 2024-09-16T00:09:53.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:53.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165360+0000", 2024-09-16T00:09:53.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011134899999999999 2024-09-16T00:09:53.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.169320+0000", 2024-09-16T00:09:53.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039593040000000003 2024-09-16T00:09:53.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175135+0000", 2024-09-16T00:09:53.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005814891 2024-09-16T00:09:53.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175199+0000", 2024-09-16T00:09:53.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4474999999999995e-05 2024-09-16T00:09:53.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175239+0000", 2024-09-16T00:09:53.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0333000000000002e-05 2024-09-16T00:09:53.913 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.913 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.914 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.914 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11789 3.21 3:8453163e:::100000030b3.00000000:head [write 0~3801088 [4@1194504]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:53.914 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:09:53.914 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 10.595949168000001, 2024-09-16T00:09:53.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011097602, 2024-09-16T00:09:53.914 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.914 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.914 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.914 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.914 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.914 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.914 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11789 2024-09-16T00:09:53.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.915 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.915 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.915 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:09:53.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.915 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.915 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:09:53.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.915 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251373+0000", 2024-09-16T00:09:53.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.544e-06 2024-09-16T00:09:53.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252710+0000", 2024-09-16T00:09:53.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013362929999999999 2024-09-16T00:09:53.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252730+0000", 2024-09-16T00:09:53.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0579999999999999e-05 2024-09-16T00:09:53.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252741+0000", 2024-09-16T00:09:53.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0505e-05 2024-09-16T00:09:53.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252820+0000", 2024-09-16T00:09:53.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9098999999999996e-05 2024-09-16T00:09:53.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252931+0000", 2024-09-16T00:09:53.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011073600000000001 2024-09-16T00:09:53.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:53.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.253101+0000", 2024-09-16T00:09:53.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017038000000000001 2024-09-16T00:09:53.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.261813+0000", 2024-09-16T00:09:53.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087119469999999994 2024-09-16T00:09:53.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262338+0000", 2024-09-16T00:09:53.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00052493799999999997 2024-09-16T00:09:53.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262430+0000", 2024-09-16T00:09:53.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1626999999999995e-05 2024-09-16T00:09:53.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262470+0000", 2024-09-16T00:09:53.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9953000000000002e-05 2024-09-16T00:09:53.919 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.919 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.919 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.920 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11890 3.38 3:1e014616:::100000033e9.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:53.920 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:09:53.920 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 10.275984085999999, 2024-09-16T00:09:53.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011833791999999999, 2024-09-16T00:09:53.920 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.920 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.920 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.920 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.920 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.920 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.920 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11890 2024-09-16T00:09:53.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.920 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.921 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:09:53.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.921 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:09:53.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.921 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571340+0000", 2024-09-16T00:09:53.922 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.36e-06 2024-09-16T00:09:53.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.922 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.922 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572866+0000", 2024-09-16T00:09:53.922 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001525549 2024-09-16T00:09:53.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.922 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.922 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572884+0000", 2024-09-16T00:09:53.922 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7796999999999999e-05 2024-09-16T00:09:53.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.922 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.923 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572897+0000", 2024-09-16T00:09:53.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3567000000000001e-05 2024-09-16T00:09:53.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.923 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.923 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572970+0000", 2024-09-16T00:09:53.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2751999999999997e-05 2024-09-16T00:09:53.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.923 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.923 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573053+0000", 2024-09-16T00:09:53.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3230000000000001e-05 2024-09-16T00:09:53.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:53.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573168+0000", 2024-09-16T00:09:53.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114536 2024-09-16T00:09:53.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.577923+0000", 2024-09-16T00:09:53.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047548720000000003 2024-09-16T00:09:53.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583081+0000", 2024-09-16T00:09:53.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051587630000000002 2024-09-16T00:09:53.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583137+0000", 2024-09-16T00:09:53.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5259000000000001e-05 2024-09-16T00:09:53.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583171+0000", 2024-09-16T00:09:53.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4106999999999997e-05 2024-09-16T00:09:53.925 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.925 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.925 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.926 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:12037 3.9 e87/20 3:928db521:::1000000326d.00000000:head v 87'199, mlcod=87'197)", 2024-09-16T00:09:53.926 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:09:53.926 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.8447941350000008, 2024-09-16T00:09:53.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012236153, 2024-09-16T00:09:53.926 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.926 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.926 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.926 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:09:53.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.927 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.927 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:09:53.927 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.927 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.927 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002529+0000", 2024-09-16T00:09:53.927 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4889999999999998e-06 2024-09-16T00:09:53.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.927 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.927 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007064+0000", 2024-09-16T00:09:53.927 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045346220000000003 2024-09-16T00:09:53.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007093+0000", 2024-09-16T00:09:53.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9009e-05 2024-09-16T00:09:53.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007109+0000", 2024-09-16T00:09:53.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5846e-05 2024-09-16T00:09:53.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007196+0000", 2024-09-16T00:09:53.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7507000000000001e-05 2024-09-16T00:09:53.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007227+0000", 2024-09-16T00:09:53.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0916e-05 2024-09-16T00:09:53.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014658+0000", 2024-09-16T00:09:53.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074305090000000001 2024-09-16T00:09:53.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014763+0000", 2024-09-16T00:09:53.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010525499999999999 2024-09-16T00:09:53.930 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.930 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.930 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.930 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12398 3.2e 3:77a11d08:::10000003432.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:53.930 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:09:53.930 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.6061153519999998, 2024-09-16T00:09:53.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012092999, 2024-09-16T00:09:53.930 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.930 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.930 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.931 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.931 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.931 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.931 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12398 2024-09-16T00:09:53.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.931 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:09:53.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:09:53.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241209+0000", 2024-09-16T00:09:53.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2710000000000001e-06 2024-09-16T00:09:53.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243840+0000", 2024-09-16T00:09:53.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026307919999999999 2024-09-16T00:09:53.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243872+0000", 2024-09-16T00:09:53.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2699e-05 2024-09-16T00:09:53.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243887+0000", 2024-09-16T00:09:53.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4348000000000001e-05 2024-09-16T00:09:53.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243972+0000", 2024-09-16T00:09:53.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5358999999999994e-05 2024-09-16T00:09:53.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.934 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244132+0000", 2024-09-16T00:09:53.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015972499999999999 2024-09-16T00:09:53.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.934 INFO:tasks.workunit.client.0.smithi139.stdout:4/984: mkdir d12/d142 0 2024-09-16T00:09:53.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253153+0000", 2024-09-16T00:09:53.935 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0090210280000000004 2024-09-16T00:09:53.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253213+0000", 2024-09-16T00:09:53.935 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0300999999999997e-05 2024-09-16T00:09:53.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.936 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.936 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253299+0000", 2024-09-16T00:09:53.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5476000000000006e-05 2024-09-16T00:09:53.936 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.936 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.936 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.936 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12400 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304 [2@815045]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:53.936 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:09:53.936 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.6030776180000004, 2024-09-16T00:09:53.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011140066000000001, 2024-09-16T00:09:53.936 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.937 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.937 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.937 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.937 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.937 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.937 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12400 2024-09-16T00:09:53.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.937 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:09:53.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:09:53.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244245+0000", 2024-09-16T00:09:53.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.248e-06 2024-09-16T00:09:53.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246467+0000", 2024-09-16T00:09:53.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022217949999999999 2024-09-16T00:09:53.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246496+0000", 2024-09-16T00:09:53.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9484000000000001e-05 2024-09-16T00:09:53.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246509+0000", 2024-09-16T00:09:53.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3390000000000001e-05 2024-09-16T00:09:53.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246586+0000", 2024-09-16T00:09:53.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6692000000000004e-05 2024-09-16T00:09:53.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246680+0000", 2024-09-16T00:09:53.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3468999999999996e-05 2024-09-16T00:09:53.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:53.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246850+0000", 2024-09-16T00:09:53.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170206 2024-09-16T00:09:53.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253478+0000", 2024-09-16T00:09:53.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066287530000000002 2024-09-16T00:09:53.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255318+0000", 2024-09-16T00:09:53.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001839292 2024-09-16T00:09:53.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255354+0000", 2024-09-16T00:09:53.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5964999999999999e-05 2024-09-16T00:09:53.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255384+0000", 2024-09-16T00:09:53.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9771999999999999e-05 2024-09-16T00:09:53.942 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.942 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.942 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.942 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13317 3.38 3:1cda8244:::100000035dc.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:53.942 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:09:53.942 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.9287230920000002, 2024-09-16T00:09:53.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011377063, 2024-09-16T00:09:53.943 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.943 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.943 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.943 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.943 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.943 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.943 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13317 2024-09-16T00:09:53.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.943 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.943 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.943 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:09:53.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:09:53.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918602+0000", 2024-09-16T00:09:53.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8160000000000004e-06 2024-09-16T00:09:53.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920364+0000", 2024-09-16T00:09:53.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017623630000000001 2024-09-16T00:09:53.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920391+0000", 2024-09-16T00:09:53.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7248000000000001e-05 2024-09-16T00:09:53.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920405+0000", 2024-09-16T00:09:53.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3969e-05 2024-09-16T00:09:53.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920486+0000", 2024-09-16T00:09:53.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0737999999999998e-05 2024-09-16T00:09:53.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920594+0000", 2024-09-16T00:09:53.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108269 2024-09-16T00:09:53.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:53.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920730+0000", 2024-09-16T00:09:53.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000135229 2024-09-16T00:09:53.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.928847+0000", 2024-09-16T00:09:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081178299999999995 2024-09-16T00:09:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929862+0000", 2024-09-16T00:09:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010147540000000001 2024-09-16T00:09:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929923+0000", 2024-09-16T00:09:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0452999999999999e-05 2024-09-16T00:09:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.948 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929975+0000", 2024-09-16T00:09:53.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2394e-05 2024-09-16T00:09:53.948 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.948 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.948 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.948 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:13326 3.9 e88/20 3:92f796cf:::1000000350f.00000000:head v 88'214, mlcod=88'212)", 2024-09-16T00:09:53.948 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:09:53.948 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.886202301, 2024-09-16T00:09:53.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01100597, 2024-09-16T00:09:53.948 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.949 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.949 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.949 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.949 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.949 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:09:53.949 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.949 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.949 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.949 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:09:53.949 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.949 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961121+0000", 2024-09-16T00:09:53.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.815e-06 2024-09-16T00:09:53.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965747+0000", 2024-09-16T00:09:53.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0046260570000000003 2024-09-16T00:09:53.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965780+0000", 2024-09-16T00:09:53.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2851000000000001e-05 2024-09-16T00:09:53.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965788+0000", 2024-09-16T00:09:53.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0180000000000001e-06 2024-09-16T00:09:53.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965859+0000", 2024-09-16T00:09:53.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1318999999999999e-05 2024-09-16T00:09:53.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965886+0000", 2024-09-16T00:09:53.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7509999999999999e-05 2024-09-16T00:09:53.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.952 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.972066+0000", 2024-09-16T00:09:53.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061800759999999996 2024-09-16T00:09:53.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.952 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.972125+0000", 2024-09-16T00:09:53.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8323999999999999e-05 2024-09-16T00:09:53.952 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.952 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.952 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.953 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.953 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13942 3.22 3:44ed4521:::100000036d5.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:53.953 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:09:53.953 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.9248369890000001, 2024-09-16T00:09:53.953 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011341061, 2024-09-16T00:09:53.953 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.953 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.953 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.953 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.953 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.953 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.953 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13942 2024-09-16T00:09:53.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.954 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.954 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.954 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:09:53.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.954 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.954 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:09:53.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.954 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.954 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922487+0000", 2024-09-16T00:09:53.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0180000000000002e-06 2024-09-16T00:09:53.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923720+0000", 2024-09-16T00:09:53.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012333470000000001 2024-09-16T00:09:53.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923749+0000", 2024-09-16T00:09:53.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8748999999999999e-05 2024-09-16T00:09:53.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923770+0000", 2024-09-16T00:09:53.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1189999999999999e-05 2024-09-16T00:09:53.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.956 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923868+0000", 2024-09-16T00:09:53.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7675000000000002e-05 2024-09-16T00:09:53.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.956 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.924047+0000", 2024-09-16T00:09:53.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017935100000000001 2024-09-16T00:09:53.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:53.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.924223+0000", 2024-09-16T00:09:53.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000175661 2024-09-16T00:09:53.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.931544+0000", 2024-09-16T00:09:53.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073204589999999996 2024-09-16T00:09:53.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933692+0000", 2024-09-16T00:09:53.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021488760000000001 2024-09-16T00:09:53.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933774+0000", 2024-09-16T00:09:53.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1855999999999997e-05 2024-09-16T00:09:53.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933825+0000", 2024-09-16T00:09:53.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0878999999999999e-05 2024-09-16T00:09:53.958 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.958 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.958 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.959 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14784 3.21 3:8673bb5f:::100000030ed.00000000:head [write 0~4194304 [2@1395527]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:53.959 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.206931+0000", 2024-09-16T00:09:53.959 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.64039042499999999, 2024-09-16T00:09:53.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010901582999999999, 2024-09-16T00:09:53.959 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.959 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.959 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.959 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.959 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.959 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.959 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14784 2024-09-16T00:09:53.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.960 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.960 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.206931+0000", 2024-09-16T00:09:53.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.960 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.206931+0000", 2024-09-16T00:09:53.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.206934+0000", 2024-09-16T00:09:53.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8499999999999998e-06 2024-09-16T00:09:53.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.209164+0000", 2024-09-16T00:09:53.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022303959999999999 2024-09-16T00:09:53.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.209186+0000", 2024-09-16T00:09:53.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1917999999999999e-05 2024-09-16T00:09:53.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.209196+0000", 2024-09-16T00:09:53.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0305e-05 2024-09-16T00:09:53.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.209268+0000", 2024-09-16T00:09:53.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1487000000000002e-05 2024-09-16T00:09:53.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.209337+0000", 2024-09-16T00:09:53.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9776000000000004e-05 2024-09-16T00:09:53.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:53.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.209428+0000", 2024-09-16T00:09:53.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0620999999999999e-05 2024-09-16T00:09:53.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.215620+0000", 2024-09-16T00:09:53.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006191612 2024-09-16T00:09:53.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.217721+0000", 2024-09-16T00:09:53.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021018209999999998 2024-09-16T00:09:53.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.217783+0000", 2024-09-16T00:09:53.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.177e-05 2024-09-16T00:09:53.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.217832+0000", 2024-09-16T00:09:53.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9026999999999997e-05 2024-09-16T00:09:53.964 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.964 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.964 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.965 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14867 3.1f 3:faedf583:::1000000393b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:53.965 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:09:53.965 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.100347253, 2024-09-16T00:09:53.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011128677999999999, 2024-09-16T00:09:53.965 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:53.965 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:53.965 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:53.965 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:53.965 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:53.965 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:53.965 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14867 2024-09-16T00:09:53.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.965 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:53.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:53.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:09:53.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:53.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:09:53.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:53.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:53.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746976+0000", 2024-09-16T00:09:53.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1169999999999998e-06 2024-09-16T00:09:53.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.967 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:53.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748348+0000", 2024-09-16T00:09:53.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013721860000000001 2024-09-16T00:09:53.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.967 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:53.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748371+0000", 2024-09-16T00:09:53.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2938999999999998e-05 2024-09-16T00:09:53.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.967 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:53.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748381+0000", 2024-09-16T00:09:53.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5300000000000002e-06 2024-09-16T00:09:53.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:53.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748458+0000", 2024-09-16T00:09:53.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7389000000000001e-05 2024-09-16T00:09:53.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:53.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748626+0000", 2024-09-16T00:09:53.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016757499999999999 2024-09-16T00:09:53.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:53.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748813+0000", 2024-09-16T00:09:53.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018743399999999999 2024-09-16T00:09:53.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:53.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.753998+0000", 2024-09-16T00:09:53.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051851140000000002 2024-09-16T00:09:53.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:53.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.757990+0000", 2024-09-16T00:09:53.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039914169999999997 2024-09-16T00:09:53.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:53.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.758054+0000", 2024-09-16T00:09:53.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4513999999999999e-05 2024-09-16T00:09:53.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:53.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:53.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:53.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.758103+0000", 2024-09-16T00:09:53.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8463e-05 2024-09-16T00:09:53.970 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.970 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.970 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.970 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:53.970 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:53.971 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:53.972 INFO:tasks.workunit.client.0.smithi139.stdout:4/985: chown d12/d47/d49/d60 16744967 1 2024-09-16T00:09:53.972 INFO:tasks.workunit.client.0.smithi139.stdout:3/812: rename d5/d7/d23/de4/de8/da7/d9a/d101/f8d to d5/d7/d23/de4/de8/da7/d9a/d101/f117 0 2024-09-16T00:09:53.972 INFO:tasks.workunit.client.0.smithi139.stdout:7/951: dwrite d5/d26/d46/d6b/da7/f105 [0,4194304] 0 2024-09-16T00:09:53.972 INFO:tasks.workunit.client.0.smithi139.stdout:4/986: creat d12/d2b/d8c/dff/d109/f143 x:0 0 0 2024-09-16T00:09:53.972 INFO:tasks.workunit.client.0.smithi139.stdout:4/987: chown d12/d47/db2/d4d/de4/d7f/d9e/ffa 62014744 1 2024-09-16T00:09:53.972 INFO:tasks.workunit.client.0.smithi139.stdout:3/813: symlink d5/d7/dc2/dec/l118 0 2024-09-16T00:09:53.972 INFO:tasks.workunit.client.0.smithi139.stdout:4/988: write d12/d47/db2/fc2 [3922485,104374] 0 2024-09-16T00:09:53.972 INFO:tasks.workunit.client.0.smithi139.stdout:4/989: unlink d12/d47/f5c 0 2024-09-16T00:09:53.972 INFO:tasks.workunit.client.0.smithi139.stdout:6/784: dread d8/d29/d83/d26/d45/d85/fe2 [0,4194304] 0 2024-09-16T00:09:53.972 INFO:tasks.workunit.client.0.smithi139.stdout:3/814: rmdir d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/db9 39 2024-09-16T00:09:53.973 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:09:53.981 INFO:tasks.workunit.client.0.smithi139.stdout:6/785: chown d8/d20/d4d/db0/fd6 929 1 2024-09-16T00:09:53.981 INFO:tasks.workunit.client.0.smithi139.stdout:4/990: creat d12/d2b/d8c/dff/d109/f144 x:0 0 0 2024-09-16T00:09:53.981 INFO:tasks.workunit.client.0.smithi139.stdout:4/991: fdatasync d12/d47/faf 0 2024-09-16T00:09:53.981 INFO:tasks.workunit.client.0.smithi139.stdout:3/815: mkdir d5/d7/dc2/d119 0 2024-09-16T00:09:53.981 INFO:tasks.workunit.client.0.smithi139.stdout:6/786: symlink d8/d29/d83/d26/d28/l119 0 2024-09-16T00:09:53.981 INFO:tasks.workunit.client.0.smithi139.stdout:7/952: dwrite d5/d16/d6d/d7f/dc2/d81/fcb [0,4194304] 0 2024-09-16T00:09:53.981 INFO:tasks.workunit.client.0.smithi139.stdout:3/816: write d5/d7/f37 [3119304,25348] 0 2024-09-16T00:09:53.981 INFO:tasks.workunit.client.0.smithi139.stdout:4/992: chown d12/d47/d49/d60/dac/d12f/lae 59357 1 2024-09-16T00:09:53.981 INFO:tasks.workunit.client.0.smithi139.stdout:6/787: dread d8/d20/f64 [0,4194304] 0 2024-09-16T00:09:53.981 INFO:tasks.workunit.client.0.smithi139.stdout:6/788: dread d8/d20/d27/f97 [0,4194304] 0 2024-09-16T00:09:53.981 INFO:tasks.workunit.client.0.smithi139.stdout:7/953: truncate d5/d16/d22/d25/faf 790388 0 2024-09-16T00:09:53.982 INFO:tasks.workunit.client.0.smithi139.stdout:4/993: dwrite d12/f31 [0,4194304] 0 2024-09-16T00:09:53.982 INFO:tasks.workunit.client.0.smithi139.stdout:3/817: truncate d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d47/f25 3111526 0 2024-09-16T00:09:53.982 INFO:tasks.workunit.client.0.smithi139.stdout:6/789: chown d8/d29/d83/d26/d45/d65/l33 1722416245 1 2024-09-16T00:09:53.982 INFO:tasks.workunit.client.0.smithi139.stdout:7/954: truncate d5/d16/d22/d71/f117 1025411 0 2024-09-16T00:09:53.982 INFO:tasks.workunit.client.0.smithi139.stdout:4/994: mknod d12/d2b/d8c/dff/d109/c145 0 2024-09-16T00:09:53.982 INFO:tasks.workunit.client.0.smithi139.stdout:6/790: mknod d8/d56/c11a 0 2024-09-16T00:09:53.982 INFO:tasks.workunit.client.0.smithi139.stdout:6/791: fdatasync d8/d29/d83/d26/d28/d6d/f116 0 2024-09-16T00:09:53.982 INFO:tasks.workunit.client.0.smithi139.stdout:7/955: creat d5/d16/d6d/d7f/dc2/d50/d44/d55/d90/f14e x:0 0 0 2024-09-16T00:09:53.982 INFO:tasks.workunit.client.0.smithi139.stdout:4/995: dwrite d12/d2b/ff2 [4194304,4194304] 0 2024-09-16T00:09:53.982 INFO:tasks.workunit.client.0.smithi139.stdout:4/996: write d12/d47/db2/d4d/de4/fd5 [120222,4855] 0 2024-09-16T00:09:53.982 INFO:tasks.workunit.client.0.smithi139.stdout:6/792: write d8/d29/d83/d26/d28/d5a/f86 [280056,18118] 0 2024-09-16T00:09:53.982 INFO:tasks.workunit.client.0.smithi139.stdout:7/956: getdents d5/d16/d6d/d7f/dc2/d50/d44/d55/de0 0 2024-09-16T00:09:53.982 INFO:tasks.workunit.client.0.smithi139.stdout:7/957: fdatasync d5/d26/f2b 0 2024-09-16T00:09:53.982 INFO:tasks.workunit.client.0.smithi139.stdout:4/997: rename d12/d2b/c3c to d12/d47/d49/d60/dac/d12f/c146 0 2024-09-16T00:09:53.983 INFO:tasks.workunit.client.0.smithi139.stdout:6/793: truncate d8/d29/d83/d26/d28/d6d/f71 669242 0 2024-09-16T00:09:53.983 INFO:tasks.workunit.client.0.smithi139.stdout:7/958: fsync d5/d16/d6d/d7f/df5/f123 0 2024-09-16T00:09:53.983 INFO:tasks.workunit.client.0.smithi139.stdout:4/998: dread - d12/d80/f125 zero size 2024-09-16T00:09:53.983 INFO:tasks.workunit.client.0.smithi139.stdout:6/794: mknod d8/d29/db9/de8/c11b 0 2024-09-16T00:09:53.983 INFO:tasks.workunit.client.0.smithi139.stdout:7/959: rmdir d5/d16/d6d/d7f/dc2/d50/d44/d55/d90 39 2024-09-16T00:09:53.983 INFO:tasks.workunit.client.0.smithi139.stdout:4/999: creat d12/d47/d49/d60/dac/f147 x:0 0 0 2024-09-16T00:09:53.983 INFO:tasks.workunit.client.0.smithi139.stdout:3/818: dread d5/d7/d23/f15 [0,4194304] 0 2024-09-16T00:09:53.983 INFO:tasks.workunit.client.0.smithi139.stdout:6/795: mkdir d8/d29/d83/d26/d45/d42/d11c 0 2024-09-16T00:09:53.983 INFO:tasks.workunit.client.0.smithi139.stdout:3/819: chown d5/d7/d23/la4 24 1 2024-09-16T00:09:53.983 INFO:tasks.workunit.client.0.smithi139.stdout:7/960: truncate d5/d26/f4c 1181858 0 2024-09-16T00:09:53.983 INFO:tasks.workunit.client.0.smithi139.stdout:6/796: mknod d8/d29/d83/d26/d45/d42/d11c/c11d 0 2024-09-16T00:09:53.983 INFO:tasks.workunit.client.0.smithi139.stdout:7/961: rmdir d5/d26/d46/de4 39 2024-09-16T00:09:53.983 INFO:tasks.workunit.client.0.smithi139.stdout:7/962: stat d5/d26/d46/d6b/dd6/d145 0 2024-09-16T00:09:53.983 INFO:tasks.workunit.client.0.smithi139.stdout:7/963: write d5/d16/d6d/d7f/dc2/dab/fe7 [203123,41129] 0 2024-09-16T00:09:53.984 INFO:tasks.workunit.client.0.smithi139.stdout:3/820: creat d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/f11a x:0 0 0 2024-09-16T00:09:53.984 INFO:tasks.workunit.client.0.smithi139.stdout:7/964: mknod d5/d26/d46/d6b/da7/ddb/c14f 0 2024-09-16T00:09:53.984 INFO:tasks.workunit.client.0.smithi139.stdout:3/821: creat d5/d7/d23/de4/de8/da7/d9a/f11b x:0 0 0 2024-09-16T00:09:53.984 INFO:tasks.workunit.client.0.smithi139.stdout:6/797: write d8/d29/d83/d26/d45/d65/d8b/f10d [4162317,60163] 0 2024-09-16T00:09:53.984 INFO:tasks.workunit.client.0.smithi139.stdout:3/822: write d5/d7/dc/f12 [4103724,121273] 0 2024-09-16T00:09:53.984 INFO:tasks.workunit.client.0.smithi139.stdout:7/965: creat d5/d26/d46/d6b/dd6/d10c/f150 x:0 0 0 2024-09-16T00:09:53.984 INFO:tasks.workunit.client.0.smithi139.stdout:7/966: write d5/d26/f2b [2465292,71200] 0 2024-09-16T00:09:53.984 INFO:tasks.workunit.client.0.smithi139.stdout:6/798: mkdir d8/d29/d83/d26/d45/d42/d11c/d11e 0 2024-09-16T00:09:53.984 INFO:tasks.workunit.client.0.smithi139.stdout:6/799: stat d8/d56/dba 0 2024-09-16T00:09:53.984 INFO:tasks.workunit.client.0.smithi139.stdout:3/823: rmdir d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4d/dc6/dfe 39 2024-09-16T00:09:53.984 INFO:tasks.workunit.client.0.smithi139.stdout:6/800: stat d8/d29/d83/d26/d28/d6d/f116 0 2024-09-16T00:09:53.984 INFO:tasks.workunit.client.0.smithi139.stdout:7/967: mknod d5/d26/d74/c151 0 2024-09-16T00:09:53.984 INFO:tasks.workunit.client.0.smithi139.stdout:6/801: symlink d8/dfa/l11f 0 2024-09-16T00:09:53.985 INFO:tasks.workunit.client.0.smithi139.stdout:3/824: link d5/d7/d23/de4/de8/da7/ldf d5/d7/dc/l11c 0 2024-09-16T00:09:53.985 INFO:tasks.workunit.client.0.smithi139.stdout:7/968: dwrite d5/f59 [0,4194304] 0 2024-09-16T00:09:53.985 INFO:tasks.workunit.client.0.smithi139.stdout:3/825: getdents d5/d7/d23/de4/de8 0 2024-09-16T00:09:53.985 INFO:tasks.workunit.client.0.smithi139.stdout:7/969: mknod d5/d26/d74/c152 0 2024-09-16T00:09:53.985 INFO:tasks.workunit.client.0.smithi139.stdout:3/826: write d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/f7f [4429161,131034] 0 2024-09-16T00:09:53.985 INFO:tasks.workunit.client.0.smithi139.stdout:6/802: truncate d8/d29/d83/d26/d45/d85/f118 6086839 0 2024-09-16T00:09:53.985 INFO:tasks.workunit.client.0.smithi139.stdout:3/827: mkdir d5/d7/d23/de4/d75/d11d 0 2024-09-16T00:09:53.985 INFO:tasks.workunit.client.0.smithi139.stdout:3/828: unlink d5/d7/d23/c93 0 2024-09-16T00:09:53.985 INFO:tasks.workunit.client.0.smithi139.stdout:7/970: dwrite d5/d16/d6d/d7f/dc2/dab/fe7 [0,4194304] 0 2024-09-16T00:09:53.985 INFO:tasks.workunit.client.0.smithi139.stdout:5/958: sync 2024-09-16T00:09:53.985 INFO:tasks.workunit.client.0.smithi139.stdout:1/758: sync 2024-09-16T00:09:53.985 INFO:tasks.workunit.client.0.smithi139.stdout:7/971: getdents d5/d16/d87 0 2024-09-16T00:09:53.985 INFO:tasks.workunit.client.0.smithi139.stdout:1/759: unlink d7/d8/d1f/l6e 0 2024-09-16T00:09:53.985 INFO:tasks.workunit.client.0.smithi139.stdout:7/972: chown d5/d16/d6d/d7f/dc2/d81/f140 4 1 2024-09-16T00:09:53.986 INFO:tasks.workunit.client.0.smithi139.stdout:1/760: symlink d7/d8/d27/d68/le1 0 2024-09-16T00:09:53.986 INFO:tasks.workunit.client.0.smithi139.stdout:7/973: read d5/d16/d22/fc7 [251998,67055] 0 2024-09-16T00:09:53.986 INFO:tasks.workunit.client.0.smithi139.stdout:6/803: dread d8/d29/d83/d26/d28/f40 [4194304,4194304] 0 2024-09-16T00:09:53.986 INFO:tasks.workunit.client.0.smithi139.stdout:7/974: stat d5/d26/d46/d6b/dd6/d119 0 2024-09-16T00:09:53.986 INFO:tasks.workunit.client.0.smithi139.stdout:3/829: dread d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4d/f60 [0,4194304] 0 2024-09-16T00:09:53.986 INFO:tasks.workunit.client.0.smithi139.stdout:5/959: dwrite d1/d21/d49/d53/d6a/d7b/da1/db7/fee [0,4194304] 0 2024-09-16T00:09:53.986 INFO:tasks.workunit.client.0.smithi139.stdout:7/975: read d5/d26/d46/d6b/dd6/f13a [1837316,75116] 0 2024-09-16T00:09:53.986 INFO:tasks.workunit.client.0.smithi139.stdout:5/960: dread - d1/d13/d22/f80 zero size 2024-09-16T00:09:53.986 INFO:tasks.workunit.client.0.smithi139.stdout:6/804: truncate d8/d29/d83/d26/d45/d85/fe2 4557032 0 2024-09-16T00:09:53.986 INFO:tasks.workunit.client.0.smithi139.stdout:3/830: write d5/d7/d23/de4/de8/f45 [7234,1577] 0 2024-09-16T00:09:53.986 INFO:tasks.workunit.client.0.smithi139.stdout:1/761: dwrite d7/d8/d40/d48/d67/dbf/fd5 [0,4194304] 0 2024-09-16T00:09:53.986 INFO:tasks.workunit.client.0.smithi139.stdout:7/976: dwrite d5/d26/f2b [0,4194304] 0 2024-09-16T00:09:53.986 INFO:tasks.workunit.client.0.smithi139.stdout:3/831: mknod d5/d7/d23/de4/de8/da7/d9a/d101/c11e 0 2024-09-16T00:09:53.986 INFO:tasks.workunit.client.0.smithi139.stdout:6/805: link d8/d20/d4d/f8e d8/d29/d83/d26/d45/d65/d8b/f120 0 2024-09-16T00:09:53.987 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:53.974+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:53.988 INFO:tasks.workunit.client.0.smithi139.stdout:6/806: readlink d8/d56/l84 0 2024-09-16T00:09:53.988 INFO:tasks.workunit.client.0.smithi139.stdout:5/961: dwrite d1/d21/d49/db2/fba [0,4194304] 0 2024-09-16T00:09:53.988 INFO:tasks.workunit.client.0.smithi139.stdout:3/832: write d5/d7/dc/d108/fdb [969356,80507] 0 2024-09-16T00:09:53.988 INFO:tasks.workunit.client.0.smithi139.stdout:1/762: write d7/d8/d27/d68/d6c/f7a [235129,122165] 0 2024-09-16T00:09:53.988 INFO:tasks.workunit.client.0.smithi139.stdout:5/962: creat d1/d13/d22/dd5/f14f x:0 0 0 2024-09-16T00:09:53.988 INFO:tasks.workunit.client.0.smithi139.stdout:6/807: dread - d8/d29/d83/d26/d28/d5a/f102 zero size 2024-09-16T00:09:53.988 INFO:tasks.workunit.client.0.smithi139.stdout:1/763: write d7/d8/d40/d48/f65 [2160801,80843] 0 2024-09-16T00:09:53.988 INFO:tasks.workunit.client.0.smithi139.stdout:5/963: chown d1/d13/d3f/d9f/d132 722303 1 2024-09-16T00:09:53.988 INFO:tasks.workunit.client.0.smithi139.stdout:3/833: dread - d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/db9/feb zero size 2024-09-16T00:09:53.989 INFO:tasks.workunit.client.0.smithi139.stdout:1/764: mkdir d7/d8/d40/d48/d4a/d82/de2 0 2024-09-16T00:09:53.990 INFO:tasks.workunit.client.0.smithi139.stdout:7/977: dwrite d5/d26/d42/f8f [4194304,4194304] 0 2024-09-16T00:09:53.993 INFO:tasks.workunit.client.0.smithi139.stdout:6/808: dread d8/d20/d4d/f8e [4194304,4194304] 0 2024-09-16T00:09:53.993 INFO:tasks.workunit.client.0.smithi139.stdout:5/964: rename d1/d13/d3f/d5e/cf8 to d1/d13/d3f/c150 0 2024-09-16T00:09:53.994 INFO:tasks.workunit.client.0.smithi139.stdout:1/765: mkdir d7/d8/d40/dd3/de3 0 2024-09-16T00:09:53.994 INFO:tasks.workunit.client.0.smithi139.stdout:0/923: sync 2024-09-16T00:09:53.995 INFO:tasks.workunit.client.0.smithi139.stdout:1/766: write d7/d8/d27/d68/d7c/fc2 [905101,69158] 0 2024-09-16T00:09:53.996 INFO:tasks.workunit.client.0.smithi139.stdout:7/978: read - d5/d16/d6d/d7f/dc2/d50/d44/d55/d90/f14e zero size 2024-09-16T00:09:53.998 INFO:tasks.workunit.client.0.smithi139.stdout:6/809: mknod d8/d29/d83/d26/d45/d59/c121 0 2024-09-16T00:09:53.999 INFO:tasks.workunit.client.0.smithi139.stdout:5/965: mknod d1/d21/d49/d53/d6a/c151 0 2024-09-16T00:09:54.000 INFO:tasks.workunit.client.0.smithi139.stdout:0/924: rename d6/d1c/d41/d44/dd/la8 to d6/d42/d120/l123 0 2024-09-16T00:09:54.002 INFO:tasks.workunit.client.0.smithi139.stdout:6/810: symlink d8/d29/d83/d26/d28/d6d/dd3/l122 0 2024-09-16T00:09:54.002 INFO:tasks.workunit.client.0.smithi139.stdout:0/925: truncate d6/d1c/d41/d63/d82/fbe 431031 0 2024-09-16T00:09:54.003 INFO:tasks.workunit.client.0.smithi139.stdout:5/966: creat d1/d13/d3f/d9f/f152 x:0 0 0 2024-09-16T00:09:54.003 INFO:tasks.workunit.client.0.smithi139.stdout:0/926: readlink d6/d1c/d41/d44/d58/lb8 0 2024-09-16T00:09:54.003 INFO:tasks.workunit.client.0.smithi139.stdout:3/834: getdents d5/d7/d23/de4/de8/da7/d9a/d101/dab 0 2024-09-16T00:09:54.004 INFO:tasks.workunit.client.0.smithi139.stdout:0/927: chown d6/d1c/d41/d44/dd/f40 6815 1 2024-09-16T00:09:54.007 INFO:tasks.workunit.client.0.smithi139.stdout:1/767: write d7/d8/d40/d48/d4a/d74/ddb/f73 [662782,121479] 0 2024-09-16T00:09:54.007 INFO:tasks.workunit.client.0.smithi139.stdout:0/928: rmdir d6/d5a 39 2024-09-16T00:09:54.008 INFO:tasks.workunit.client.0.smithi139.stdout:1/768: chown d7/d8/d40/d48/dcc/dda 41681204 1 2024-09-16T00:09:54.008 INFO:tasks.workunit.client.0.smithi139.stdout:6/811: mknod d8/d29/d83/d26/d45/c123 0 2024-09-16T00:09:54.009 INFO:tasks.workunit.client.0.smithi139.stdout:1/769: chown d7/d8/d40/d48/d53/c71 309590524 1 2024-09-16T00:09:54.010 INFO:tasks.workunit.client.0.smithi139.stdout:7/979: rename d5/d26/d46/d6b to d5/d16/d6d/d153 0 2024-09-16T00:09:54.011 INFO:tasks.workunit.client.0.smithi139.stdout:5/967: write d1/fdd [113176,73980] 0 2024-09-16T00:09:54.012 INFO:tasks.workunit.client.0.smithi139.stdout:1/770: mknod d7/d8/d40/d47/ce4 0 2024-09-16T00:09:54.013 INFO:tasks.workunit.client.0.smithi139.stdout:1/771: write d7/d8/d40/d48/d53/f85 [8276648,92285] 0 2024-09-16T00:09:54.070 INFO:tasks.workunit.client.0.smithi139.stdout:6/812: rename d8/d1f/caa to d8/d20/d27/d117/c124 0 2024-09-16T00:09:54.071 INFO:tasks.workunit.client.0.smithi139.stdout:7/980: rmdir d5/d37/d70/d131 0 2024-09-16T00:09:54.072 INFO:tasks.workunit.client.0.smithi139.stdout:1/772: creat d7/d8/d27/d68/fe5 x:0 0 0 2024-09-16T00:09:54.072 INFO:tasks.workunit.client.0.smithi139.stdout:6/813: read d8/d29/d83/d26/d28/f31 [2315959,15811] 0 2024-09-16T00:09:54.073 INFO:tasks.workunit.client.0.smithi139.stdout:5/968: rename d1/d13/d22/f80 to d1/d13/d3f/d5e/dc2/d10c/f153 0 2024-09-16T00:09:54.075 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:54.075+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:54.099 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:54.099 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:54.099 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:54.099 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:54.108 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:09:54.116 INFO:tasks.workunit.client.0.smithi139.stdout:5/969: chown d1/d13/d3b/f44 104 1 2024-09-16T00:09:54.116 INFO:tasks.workunit.client.0.smithi139.stdout:3/835: dwrite d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d47/f25 [0,4194304] 0 2024-09-16T00:09:54.116 INFO:tasks.workunit.client.0.smithi139.stdout:1/773: write d7/d8/d27/d68/d6c/fb1 [1813388,80127] 0 2024-09-16T00:09:54.121 INFO:tasks.workunit.client.0.smithi139.stdout:5/970: creat d1/d13/d22/da9/f154 x:0 0 0 2024-09-16T00:09:54.122 INFO:tasks.workunit.client.0.smithi139.stdout:1/774: mknod d7/d8/d40/d48/dcc/ce6 0 2024-09-16T00:09:54.123 INFO:tasks.workunit.client.0.smithi139.stdout:7/981: dread d5/d26/f2b [0,4194304] 0 2024-09-16T00:09:54.123 INFO:tasks.workunit.client.0.smithi139.stdout:6/814: dwrite d8/d20/f62 [4194304,4194304] 0 2024-09-16T00:09:54.125 INFO:tasks.workunit.client.0.smithi139.stdout:5/971: rmdir d1/d13/d3f/d88 39 2024-09-16T00:09:54.127 INFO:tasks.workunit.client.0.smithi139.stdout:6/815: rmdir d8/da4 39 2024-09-16T00:09:54.129 INFO:tasks.workunit.client.0.smithi139.stdout:5/972: write d1/d21/d49/d53/d6a/d7b/d78/f8e [2261409,77077] 0 2024-09-16T00:09:54.129 INFO:tasks.workunit.client.0.smithi139.stdout:6/816: fsync d8/d20/d4d/f8e 0 2024-09-16T00:09:54.131 INFO:tasks.workunit.client.0.smithi139.stdout:6/817: chown d8/d29/d83/d26/d28/l6c 0 1 2024-09-16T00:09:54.132 INFO:tasks.workunit.client.0.smithi139.stdout:1/775: dread d7/f26 [0,4194304] 0 2024-09-16T00:09:54.133 INFO:tasks.workunit.client.0.smithi139.stdout:6/818: fsync d8/d29/d83/d26/d45/f46 0 2024-09-16T00:09:54.134 INFO:tasks.workunit.client.0.smithi139.stdout:7/982: link d5/d26/d74/cc4 d5/d16/d22/d25/c154 0 2024-09-16T00:09:54.135 INFO:tasks.workunit.client.0.smithi139.stdout:1/776: mkdir d7/d8/d40/d48/dcc/de7 0 2024-09-16T00:09:54.136 INFO:tasks.workunit.client.0.smithi139.stdout:3/836: dread d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d47/f64 [0,4194304] 0 2024-09-16T00:09:54.136 INFO:tasks.workunit.client.0.smithi139.stdout:7/983: unlink d5/d37/f86 0 2024-09-16T00:09:54.137 INFO:tasks.workunit.client.0.smithi139.stdout:1/777: mknod d7/ce8 0 2024-09-16T00:09:54.139 INFO:tasks.workunit.client.0.smithi139.stdout:3/837: readlink d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/d76/lf4 0 2024-09-16T00:09:54.142 INFO:tasks.workunit.client.0.smithi139.stdout:6/819: dwrite d8/d20/d27/f97 [4194304,4194304] 0 2024-09-16T00:09:54.144 INFO:tasks.workunit.client.0.smithi139.stdout:3/838: write d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/fe6 [3812703,79843] 0 2024-09-16T00:09:54.146 INFO:tasks.workunit.client.0.smithi139.stdout:1/778: dwrite d7/d8/d40/d48/f65 [0,4194304] 0 2024-09-16T00:09:54.147 INFO:tasks.workunit.client.0.smithi139.stdout:6/820: unlink d8/d29/d83/d26/d28/d6d/dd3/de5/c115 0 2024-09-16T00:09:54.151 INFO:tasks.workunit.client.0.smithi139.stdout:3/839: unlink d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/c58 0 2024-09-16T00:09:54.152 INFO:tasks.workunit.client.0.smithi139.stdout:1/779: mkdir d7/d8/d40/d48/d53/d61/de9 0 2024-09-16T00:09:54.153 INFO:tasks.workunit.client.0.smithi139.stdout:3/840: write d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/f7f [5089745,63216] 0 2024-09-16T00:09:54.154 INFO:tasks.workunit.client.0.smithi139.stdout:1/780: read - d7/d8/d40/d48/d4a/d74/ddb/fd8 zero size 2024-09-16T00:09:54.158 INFO:tasks.workunit.client.0.smithi139.stdout:1/781: mknod d7/d8/d40/dd3/de3/cea 0 2024-09-16T00:09:54.158 INFO:tasks.workunit.client.0.smithi139.stdout:3/841: mkdir d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/db3/d10b/d11f 0 2024-09-16T00:09:54.160 INFO:tasks.workunit.client.0.smithi139.stdout:3/842: symlink d5/d5f/l120 0 2024-09-16T00:09:54.163 INFO:tasks.workunit.client.0.smithi139.stdout:1/782: dread d7/f9b [0,4194304] 0 2024-09-16T00:09:54.167 INFO:tasks.workunit.client.0.smithi139.stdout:6/821: dread d8/d29/d83/d26/d45/d85/f118 [0,4194304] 0 2024-09-16T00:09:54.169 INFO:tasks.workunit.client.0.smithi139.stdout:6/822: rmdir d8/d29/d83/d26/d28 39 2024-09-16T00:09:54.172 INFO:tasks.workunit.client.0.smithi139.stdout:6/823: dread d8/d29/d83/d26/d28/d6d/f71 [0,4194304] 0 2024-09-16T00:09:54.176 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:54.176+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:54.184 INFO:tasks.workunit.client.0.smithi139.stdout:6/824: dread d8/d29/f3d [0,4194304] 0 2024-09-16T00:09:54.188 INFO:tasks.workunit.client.0.smithi139.stdout:6/825: dread d8/d20/d4d/db0/fd6 [0,4194304] 0 2024-09-16T00:09:54.193 INFO:tasks.workunit.client.0.smithi139.stdout:6/826: dwrite d8/d29/d83/d26/d45/d42/f108 [0,4194304] 0 2024-09-16T00:09:54.199 INFO:tasks.workunit.client.0.smithi139.stdout:6/827: rename d8/da4/d105/l110 to d8/d29/d83/d26/d28/l125 0 2024-09-16T00:09:54.200 INFO:tasks.workunit.client.0.smithi139.stdout:6/828: chown d8/f39 402750472 1 2024-09-16T00:09:54.203 INFO:tasks.workunit.client.0.smithi139.stdout:6/829: rename d8/d29/f50 to d8/d29/d83/d26/f126 0 2024-09-16T00:09:54.205 INFO:tasks.workunit.client.0.smithi139.stdout:6/830: creat d8/d29/d83/d26/d28/d74/f127 x:0 0 0 2024-09-16T00:09:54.206 INFO:tasks.workunit.client.0.smithi139.stdout:6/831: rmdir d8/d29/d83/d26/d45/d65 39 2024-09-16T00:09:54.208 INFO:tasks.workunit.client.0.smithi139.stdout:6/832: fdatasync d8/d20/f2b 0 2024-09-16T00:09:54.212 INFO:tasks.workunit.client.0.smithi139.stdout:6/833: rename d8/d29/d83/d26/d28/d6d/dd3/l122 to d8/d29/d83/d26/d28/d9e/l128 0 2024-09-16T00:09:54.217 INFO:tasks.workunit.client.0.smithi139.stdout:6/834: dwrite d8/d56/fd0 [0,4194304] 0 2024-09-16T00:09:54.218 INFO:tasks.workunit.client.0.smithi139.stdout:6/835: symlink d8/d29/d83/d26/d28/d6d/de7/l129 0 2024-09-16T00:09:54.219 INFO:tasks.workunit.client.0.smithi139.stdout:6/836: chown d8/d1f/dc9/lde 0 1 2024-09-16T00:09:54.229 INFO:tasks.workunit.client.0.smithi139.stdout:3/843: dread d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d47/f4a [0,4194304] 0 2024-09-16T00:09:54.230 INFO:tasks.workunit.client.0.smithi139.stdout:1/783: dread d7/d8/d40/d48/d4a/d74/ddb/f5c [0,4194304] 0 2024-09-16T00:09:54.232 INFO:tasks.workunit.client.0.smithi139.stdout:6/837: dread d8/d29/d83/d26/d45/f46 [0,4194304] 0 2024-09-16T00:09:54.232 INFO:tasks.workunit.client.0.smithi139.stdout:3/844: mknod d5/d7/d23/de4/dca/c121 0 2024-09-16T00:09:54.233 INFO:tasks.workunit.client.0.smithi139.stdout:1/784: dread d7/f26 [0,4194304] 0 2024-09-16T00:09:54.235 INFO:tasks.workunit.client.0.smithi139.stdout:3/845: mkdir d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/d76/d122 0 2024-09-16T00:09:54.238 INFO:tasks.workunit.client.0.smithi139.stdout:1/785: rename d7/d8/d1f/f5d to d7/d8/d40/feb 0 2024-09-16T00:09:54.238 INFO:tasks.workunit.client.0.smithi139.stdout:3/846: stat d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/f39 0 2024-09-16T00:09:54.239 INFO:tasks.workunit.client.0.smithi139.stdout:6/838: dwrite d8/d29/d83/d26/d28/d6d/f89 [0,4194304] 0 2024-09-16T00:09:54.242 INFO:tasks.workunit.client.0.smithi139.stdout:6/839: write d8/d29/d83/d26/d45/fd5 [898736,8481] 0 2024-09-16T00:09:54.245 INFO:tasks.workunit.client.0.smithi139.stdout:1/786: dread d7/d8/d27/fb0 [0,4194304] 0 2024-09-16T00:09:54.250 INFO:tasks.workunit.client.0.smithi139.stdout:6/840: symlink d8/d29/d83/d26/d28/d9e/d10a/l12a 0 2024-09-16T00:09:54.251 INFO:tasks.workunit.client.0.smithi139.stdout:3/847: dwrite d5/d7/dc/d84/f10e [0,4194304] 0 2024-09-16T00:09:54.252 INFO:tasks.workunit.client.0.smithi139.stdout:6/841: chown d8/d20/d4d/f8e 23 1 2024-09-16T00:09:54.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.252+0000 7fdb6bfff640 1 -- 172.21.15.139:0/330968976 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v91) ==== 87+0+0 (secure 0 0 0) 0x7fdb7c05dcd0 con 0x7fdb84157e50 2024-09-16T00:09:54.253 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is set 2024-09-16T00:09:54.254 INFO:tasks.workunit.client.0.smithi139.stdout:6/842: read d8/d20/d27/f97 [5439223,90610] 0 2024-09-16T00:09:54.256 INFO:tasks.workunit.client.0.smithi139.stdout:3/848: creat d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/d76/d122/f123 x:0 0 0 2024-09-16T00:09:54.257 INFO:tasks.workunit.client.0.smithi139.stdout:6/843: truncate d8/d20/d4d/db0/fda 1039192 0 2024-09-16T00:09:54.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.256+0000 7fdb89c47640 1 -- 172.21.15.139:0/330968976 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdb7006c300 msgr2=0x7fdb7006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:54.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.256+0000 7fdb89c47640 1 --2- 172.21.15.139:0/330968976 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdb7006c300 0x7fdb7006e7c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdb74034000 tx=0x7fdb7402e000 comp rx=0 tx=0).stop 2024-09-16T00:09:54.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.256+0000 7fdb89c47640 1 -- 172.21.15.139:0/330968976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdb84157e50 msgr2=0x7fdb8414a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:54.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.256+0000 7fdb89c47640 1 --2- 172.21.15.139:0/330968976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdb84157e50 0x7fdb8414a920 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009eb0 tx=0x7fdb7c010820 comp rx=0 tx=0).stop 2024-09-16T00:09:54.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.257+0000 7fdb89c47640 1 -- 172.21.15.139:0/330968976 shutdown_connections 2024-09-16T00:09:54.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.257+0000 7fdb89c47640 1 --2- 172.21.15.139:0/330968976 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdb7006c300 0x7fdb7006e7c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:54.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.257+0000 7fdb89c47640 1 --2- 172.21.15.139:0/330968976 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdb84157e50 0x7fdb8414a920 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:54.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.257+0000 7fdb89c47640 1 --2- 172.21.15.139:0/330968976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb84151c30 0x7fdb84157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:54.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.257+0000 7fdb89c47640 1 --2- 172.21.15.139:0/330968976 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdb840c7390 0x7fdb84150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:54.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.257+0000 7fdb89c47640 1 -- 172.21.15.139:0/330968976 >> 172.21.15.139:0/330968976 conn(0x7fdb8406cc30 msgr2=0x7fdb8414e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:54.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.257+0000 7fdb89c47640 1 -- 172.21.15.139:0/330968976 shutdown_connections 2024-09-16T00:09:54.258 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.257+0000 7fdb89c47640 1 -- 172.21.15.139:0/330968976 wait complete. 2024-09-16T00:09:54.259 INFO:tasks.workunit.client.0.smithi139.stdout:6/844: chown d8/d20/d27/fcf 3534 1 2024-09-16T00:09:54.261 INFO:tasks.workunit.client.0.smithi139.stdout:3/849: symlink d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/d76/d122/l124 0 2024-09-16T00:09:54.261 INFO:tasks.workunit.client.0.smithi139.stdout:6/845: creat d8/d29/d83/d26/d28/d74/f12b x:0 0 0 2024-09-16T00:09:54.263 INFO:tasks.workunit.client.0.smithi139.stdout:6/846: write d8/d20/d4d/db0/fe9 [203530,3115] 0 2024-09-16T00:09:54.267 INFO:tasks.workunit.client.0.smithi139.stdout:6/847: fdatasync d8/d29/d83/d26/d45/d65/d8b/f120 0 2024-09-16T00:09:54.268 INFO:tasks.workunit.client.0.smithi139.stdout:3/850: link d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d47/f4a d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4d/f125 0 2024-09-16T00:09:54.269 INFO:tasks.workunit.client.0.smithi139.stdout:3/851: chown d5/la6 1 1 2024-09-16T00:09:54.269 INFO:tasks.workunit.client.0.smithi139.stdout:0/929: sync 2024-09-16T00:09:54.272 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:54.272 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:54.272 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:54.272 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:54.272 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:54.273 INFO:tasks.workunit.client.0.smithi139.stdout:6/848: dread f3 [0,4194304] 0 2024-09-16T00:09:54.273 INFO:tasks.workunit.client.0.smithi139.stdout:3/852: creat d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/f126 x:0 0 0 2024-09-16T00:09:54.280 INFO:tasks.workunit.client.0.smithi139.stdout:6/849: dwrite d8/d20/d4d/f60 [4194304,4194304] 0 2024-09-16T00:09:54.280 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:54.279+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:54.282 INFO:tasks.workunit.client.0.smithi139.stdout:6/850: creat d8/d29/db9/de8/f12c x:0 0 0 2024-09-16T00:09:54.285 INFO:tasks.workunit.client.0.smithi139.stdout:3/853: dwrite d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/f69 [0,4194304] 0 2024-09-16T00:09:54.286 INFO:tasks.workunit.client.0.smithi139.stdout:0/930: dread d6/d1c/d41/d44/d6b/f6c [0,4194304] 0 2024-09-16T00:09:54.287 INFO:tasks.workunit.client.0.smithi139.stdout:3/854: chown d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68 4219 1 2024-09-16T00:09:54.289 INFO:tasks.workunit.client.0.smithi139.stdout:6/851: truncate d8/d29/d83/d26/f61 852261 0 2024-09-16T00:09:54.290 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:09:54.291 INFO:tasks.workunit.client.0.smithi139.stdout:0/931: fsync d6/d1c/d41/d76/faa 0 2024-09-16T00:09:54.292 INFO:tasks.workunit.client.0.smithi139.stdout:0/932: write d6/d1c/d41/d7c/fe4 [3929028,98895] 0 2024-09-16T00:09:54.292 INFO:tasks.workunit.client.0.smithi139.stdout:6/852: symlink d8/d20/dd1/ddc/l12d 0 2024-09-16T00:09:54.294 INFO:tasks.workunit.client.0.smithi139.stdout:0/933: readlink d6/d1c/d41/d44/l10d 0 2024-09-16T00:09:54.294 INFO:tasks.workunit.client.0.smithi139.stdout:6/853: dread - d8/d29/d83/d26/d28/d6d/dd3/de5/ff8 zero size 2024-09-16T00:09:54.299 INFO:tasks.workunit.client.0.smithi139.stdout:6/854: getdents d8 0 2024-09-16T00:09:54.300 INFO:tasks.workunit.client.0.smithi139.stdout:3/855: dread d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/dbb/ff6 [0,4194304] 0 2024-09-16T00:09:54.301 INFO:tasks.workunit.client.0.smithi139.stdout:0/934: dwrite d6/d1c/d41/d63/d82/fc7 [0,4194304] 0 2024-09-16T00:09:54.302 INFO:tasks.workunit.client.0.smithi139.stdout:6/855: chown d8/d29/d83/d26/d28/d9e/da8/fcc 346 1 2024-09-16T00:09:54.305 INFO:tasks.workunit.client.0.smithi139.stdout:0/935: creat d6/d1c/d41/d7c/f124 x:0 0 0 2024-09-16T00:09:54.308 INFO:tasks.workunit.client.0.smithi139.stdout:3/856: mknod d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d112/c127 0 2024-09-16T00:09:54.309 INFO:tasks.workunit.client.0.smithi139.stdout:6/856: write d8/d29/d83/d26/d28/d6d/f71 [958387,50125] 0 2024-09-16T00:09:54.311 INFO:tasks.workunit.client.0.smithi139.stdout:3/857: unlink d5/d5f/lc8 0 2024-09-16T00:09:54.313 INFO:tasks.workunit.client.0.smithi139.stdout:6/857: chown d8/d29/d83/d26/d45/d65/d8b 3754 1 2024-09-16T00:09:54.313 INFO:tasks.workunit.client.0.smithi139.stdout:0/936: dwrite d6/d1c/d41/d44/dd/d56/d75/f111 [0,4194304] 0 2024-09-16T00:09:54.315 INFO:tasks.workunit.client.0.smithi139.stdout:3/858: truncate d5/d7/d23/de4/de8/da7/d9a/f11b 760728 0 2024-09-16T00:09:54.315 INFO:tasks.workunit.client.0.smithi139.stdout:6/858: symlink d8/d56/dba/l12e 0 2024-09-16T00:09:54.317 INFO:tasks.workunit.client.0.smithi139.stdout:3/859: creat d5/d7/dc/d84/f128 x:0 0 0 2024-09-16T00:09:54.318 INFO:tasks.workunit.client.0.smithi139.stdout:0/937: mkdir d6/d1c/d41/d44/d6b/d125 0 2024-09-16T00:09:54.320 INFO:tasks.workunit.client.0.smithi139.stdout:3/860: creat d5/d7/dc/d108/f129 x:0 0 0 2024-09-16T00:09:54.321 INFO:tasks.workunit.client.0.smithi139.stdout:6/859: link d8/d20/d4d/c9d d8/d29/d83/c12f 0 2024-09-16T00:09:54.323 INFO:tasks.workunit.client.0.smithi139.stdout:6/860: write d8/d29/d83/d26/d45/d65/f2e [1451276,63760] 0 2024-09-16T00:09:54.324 INFO:tasks.workunit.client.0.smithi139.stdout:0/938: creat d6/d1c/d41/d44/dd/f126 x:0 0 0 2024-09-16T00:09:54.326 INFO:tasks.workunit.client.0.smithi139.stdout:6/861: truncate d8/d56/f79 4637627 0 2024-09-16T00:09:54.330 INFO:tasks.workunit.client.0.smithi139.stdout:3/861: dwrite d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/f126 [0,4194304] 0 2024-09-16T00:09:54.332 INFO:tasks.workunit.client.0.smithi139.stdout:6/862: rename d8/dfa to d8/d29/d83/d26/d28/d5a/d130 0 2024-09-16T00:09:54.333 INFO:tasks.workunit.client.0.smithi139.stdout:0/939: dwrite d6/d1c/d41/d44/d58/f5f [0,4194304] 0 2024-09-16T00:09:54.335 INFO:tasks.workunit.client.0.smithi139.stdout:3/862: write d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/da1/fd3 [271159,47921] 0 2024-09-16T00:09:54.336 INFO:tasks.workunit.client.0.smithi139.stdout:0/940: write d6/d42/db4/f8e [809160,7266] 0 2024-09-16T00:09:54.337 INFO:tasks.workunit.client.0.smithi139.stdout:0/941: chown d6/d42/db4/f8e 0 1 2024-09-16T00:09:54.341 INFO:tasks.workunit.client.0.smithi139.stdout:0/942: write d6/d1c/d41/d44/fae [1028149,98336] 0 2024-09-16T00:09:54.341 INFO:tasks.workunit.client.0.smithi139.stdout:3/863: dread - d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/f11a zero size 2024-09-16T00:09:54.342 INFO:tasks.workunit.client.0.smithi139.stdout:6/863: rmdir d8/d29/d83/d26/d45/d42/dc0 39 2024-09-16T00:09:54.343 INFO:tasks.workunit.client.0.smithi139.stdout:0/943: chown d6/d5a/d81 2020 1 2024-09-16T00:09:54.344 INFO:tasks.workunit.client.0.smithi139.stdout:3/864: dread - d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/da1/f110 zero size 2024-09-16T00:09:54.345 INFO:tasks.workunit.client.0.smithi139.stdout:0/944: dread - d6/d1c/d41/d44/dd/f126 zero size 2024-09-16T00:09:54.345 INFO:tasks.workunit.client.0.smithi139.stdout:6/864: symlink d8/d29/d83/d26/d28/d6d/de7/l131 0 2024-09-16T00:09:54.346 INFO:tasks.workunit.client.0.smithi139.stdout:0/945: dread - d6/d1c/d41/d76/f95 zero size 2024-09-16T00:09:54.347 INFO:tasks.workunit.client.0.smithi139.stdout:0/946: read d6/d5a/fd7 [3871015,108261] 0 2024-09-16T00:09:54.348 INFO:tasks.workunit.client.0.smithi139.stdout:6/865: read d8/d29/d83/d26/d45/d42/dc0/fd9 [686518,23957] 0 2024-09-16T00:09:54.349 INFO:tasks.workunit.client.0.smithi139.stdout:6/866: dread - d8/d29/d83/d26/d28/d6d/dd3/de5/ff8 zero size 2024-09-16T00:09:54.351 INFO:tasks.workunit.client.0.smithi139.stdout:0/947: link d6/d1c/d41/d76/f101 d6/d1c/d41/d44/dd/d8a/d9e/dcb/f127 0 2024-09-16T00:09:54.352 INFO:tasks.workunit.client.0.smithi139.stdout:0/948: fsync d6/d1c/d41/d44/f25 0 2024-09-16T00:09:54.355 INFO:tasks.workunit.client.0.smithi139.stdout:6/867: dwrite d8/d20/d27/f18 [0,4194304] 0 2024-09-16T00:09:54.363 INFO:tasks.workunit.client.0.smithi139.stdout:6/868: dwrite d8/d29/d83/d26/d28/d9e/da8/fcc [4194304,4194304] 0 2024-09-16T00:09:54.367 INFO:tasks.workunit.client.0.smithi139.stdout:6/869: unlink d8/d29/d83/d26/d45/d59/c87 0 2024-09-16T00:09:54.368 INFO:tasks.workunit.client.0.smithi139.stdout:0/949: dread d6/d1c/d41/d63/f89 [0,4194304] 0 2024-09-16T00:09:54.369 INFO:tasks.workunit.client.0.smithi139.stdout:6/870: chown d8/fe 1097 1 2024-09-16T00:09:54.370 INFO:tasks.workunit.client.0.smithi139.stdout:0/950: creat d6/d1c/d41/d7c/f128 x:0 0 0 2024-09-16T00:09:54.370 INFO:tasks.workunit.client.0.smithi139.stdout:6/871: chown d8/d29/d83/d26/d45/d65/d6e/lb3 7415 1 2024-09-16T00:09:54.371 INFO:tasks.workunit.client.0.smithi139.stdout:6/872: chown d8/d20/d4d/db0/f106 596563 1 2024-09-16T00:09:54.372 INFO:tasks.workunit.client.0.smithi139.stdout:0/951: truncate d6/d1c/d41/d44/dd/d8a/d9e/dcb/f119 215396 0 2024-09-16T00:09:54.376 INFO:tasks.workunit.client.0.smithi139.stdout:6/873: dwrite d8/d20/f62 [0,4194304] 0 2024-09-16T00:09:54.378 INFO:tasks.workunit.client.0.smithi139.stdout:6/874: fdatasync d8/d20/d27/fcf 0 2024-09-16T00:09:54.378 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:54.378+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:54.381 INFO:tasks.workunit.client.0.smithi139.stdout:6/875: creat d8/d29/d83/d26/d28/d5a/f132 x:0 0 0 2024-09-16T00:09:54.382 INFO:tasks.workunit.client.0.smithi139.stdout:0/952: dread d6/d1c/f61 [0,4194304] 0 2024-09-16T00:09:54.383 INFO:tasks.workunit.client.0.smithi139.stdout:6/876: symlink d8/l133 0 2024-09-16T00:09:54.384 INFO:tasks.workunit.client.0.smithi139.stdout:0/953: chown d6/d1c/d41/d44/dd/d56/d75/cc0 460090 1 2024-09-16T00:09:54.385 INFO:tasks.workunit.client.0.smithi139.stdout:0/954: write d6/d1c/d41/d44/d58/faf [3990916,107659] 0 2024-09-16T00:09:54.387 INFO:tasks.workunit.client.0.smithi139.stdout:6/877: creat d8/d29/d83/d26/d45/d59/f134 x:0 0 0 2024-09-16T00:09:54.394 INFO:tasks.workunit.client.0.smithi139.stdout:6/878: getdents d8/d29/d83/d26/d45/d65 0 2024-09-16T00:09:54.394 INFO:tasks.workunit.client.0.smithi139.stdout:0/955: truncate d6/d42/db4/f50 3112261 0 2024-09-16T00:09:54.395 INFO:tasks.workunit.client.0.smithi139.stdout:6/879: dread - d8/d29/d83/d26/d45/d42/dc0/fc6 zero size 2024-09-16T00:09:54.396 INFO:tasks.workunit.client.0.smithi139.stdout:0/956: readlink d6/d1c/d41/d44/d6b/lda 0 2024-09-16T00:09:54.398 INFO:tasks.workunit.client.0.smithi139.stdout:0/957: creat d6/d1c/d41/d7c/f129 x:0 0 0 2024-09-16T00:09:54.400 INFO:tasks.workunit.client.0.smithi139.stdout:0/958: read d6/d5a/d70/f4f [416490,33979] 0 2024-09-16T00:09:54.403 INFO:tasks.workunit.client.0.smithi139.stdout:0/959: symlink d6/d1c/d41/d63/l12a 0 2024-09-16T00:09:54.404 INFO:tasks.workunit.client.0.smithi139.stdout:6/880: dwrite d8/d29/d83/d26/f61 [0,4194304] 0 2024-09-16T00:09:54.431 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:54.431 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:54.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:54.431 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:54.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.431 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:89 2.1b 2:ddb4e4d7:::200.00000002:head [write 2591630~1401 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:54.431 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:54.431 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 50.779841482999998, 2024-09-16T00:09:54.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011802609, 2024-09-16T00:09:54.432 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.432 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.432 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.432 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:54.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:54.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649081+0000", 2024-09-16T00:09:54.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7100000000000002e-07 2024-09-16T00:09:54.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649089+0000", 2024-09-16T00:09:54.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7640000000000005e-06 2024-09-16T00:09:54.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649094+0000", 2024-09-16T00:09:54.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6219999999999997e-06 2024-09-16T00:09:54.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649099+0000", 2024-09-16T00:09:54.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6970000000000001e-06 2024-09-16T00:09:54.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655288+0000", 2024-09-16T00:09:54.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061885569999999999 2024-09-16T00:09:54.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655365+0000", 2024-09-16T00:09:54.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7723000000000001e-05 2024-09-16T00:09:54.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:54.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655483+0000", 2024-09-16T00:09:54.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117895 2024-09-16T00:09:54.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657758+0000", 2024-09-16T00:09:54.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022752369999999998 2024-09-16T00:09:54.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660803+0000", 2024-09-16T00:09:54.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030441499999999998 2024-09-16T00:09:54.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660850+0000", 2024-09-16T00:09:54.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7017000000000001e-05 2024-09-16T00:09:54.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660883+0000", 2024-09-16T00:09:54.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3476000000000003e-05 2024-09-16T00:09:54.437 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.437 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.437 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.437 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:90 2.1b 2:ddb4e4d7:::200.00000002:head [write 2593031~1404 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:54.438 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:54.438 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 50.779803577999999, 2024-09-16T00:09:54.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011882886, 2024-09-16T00:09:54.438 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.438 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.438 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.438 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:54.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:54.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649119+0000", 2024-09-16T00:09:54.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:09:54.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649125+0000", 2024-09-16T00:09:54.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1650000000000003e-06 2024-09-16T00:09:54.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649134+0000", 2024-09-16T00:09:54.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5669999999999995e-06 2024-09-16T00:09:54.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649141+0000", 2024-09-16T00:09:54.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9040000000000003e-06 2024-09-16T00:09:54.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655796+0000", 2024-09-16T00:09:54.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066553649999999999 2024-09-16T00:09:54.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655868+0000", 2024-09-16T00:09:54.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2219999999999996e-05 2024-09-16T00:09:54.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:54.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655986+0000", 2024-09-16T00:09:54.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117457 2024-09-16T00:09:54.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657794+0000", 2024-09-16T00:09:54.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001807894 2024-09-16T00:09:54.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660890+0000", 2024-09-16T00:09:54.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030958449999999998 2024-09-16T00:09:54.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660966+0000", 2024-09-16T00:09:54.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6382000000000002e-05 2024-09-16T00:09:54.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.661001+0000", 2024-09-16T00:09:54.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5349000000000003e-05 2024-09-16T00:09:54.443 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.443 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.443 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.444 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10851 3.29 3:9786b8f5:::10000003100.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:54.444 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:09:54.444 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 15.536139534, 2024-09-16T00:09:54.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012109089999999999, 2024-09-16T00:09:54.444 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.444 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.444 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.444 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:54.444 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:54.445 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:54.445 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10851 2024-09-16T00:09:54.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.445 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:09:54.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:09:54.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892786+0000", 2024-09-16T00:09:54.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4819999999999999e-06 2024-09-16T00:09:54.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894302+0000", 2024-09-16T00:09:54.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015156099999999999 2024-09-16T00:09:54.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894319+0000", 2024-09-16T00:09:54.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7098000000000001e-05 2024-09-16T00:09:54.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894329+0000", 2024-09-16T00:09:54.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9180000000000006e-06 2024-09-16T00:09:54.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894394+0000", 2024-09-16T00:09:54.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5797999999999996e-05 2024-09-16T00:09:54.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894532+0000", 2024-09-16T00:09:54.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000138128 2024-09-16T00:09:54.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:54.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894633+0000", 2024-09-16T00:09:54.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010037100000000001 2024-09-16T00:09:54.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.899212+0000", 2024-09-16T00:09:54.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045789369999999999 2024-09-16T00:09:54.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904771+0000", 2024-09-16T00:09:54.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055591169999999997 2024-09-16T00:09:54.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904838+0000", 2024-09-16T00:09:54.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7124000000000003e-05 2024-09-16T00:09:54.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904892+0000", 2024-09-16T00:09:54.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3507000000000001e-05 2024-09-16T00:09:54.451 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.451 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.451 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.451 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10957 3.24 3:2748b9d7:::1000000313b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:54.452 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:09:54.452 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.914235181, 2024-09-16T00:09:54.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012163525, 2024-09-16T00:09:54.452 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.452 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.452 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.452 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:54.452 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:54.453 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:54.453 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10957 2024-09-16T00:09:54.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.453 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:09:54.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:09:54.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514690+0000", 2024-09-16T00:09:54.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.382e-06 2024-09-16T00:09:54.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516204+0000", 2024-09-16T00:09:54.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001513831 2024-09-16T00:09:54.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516232+0000", 2024-09-16T00:09:54.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7657999999999998e-05 2024-09-16T00:09:54.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516246+0000", 2024-09-16T00:09:54.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3953e-05 2024-09-16T00:09:54.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516374+0000", 2024-09-16T00:09:54.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012882 2024-09-16T00:09:54.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516588+0000", 2024-09-16T00:09:54.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000213439 2024-09-16T00:09:54.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:54.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516748+0000", 2024-09-16T00:09:54.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015971900000000001 2024-09-16T00:09:54.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.524102+0000", 2024-09-16T00:09:54.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073542499999999997 2024-09-16T00:09:54.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526714+0000", 2024-09-16T00:09:54.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026122910000000001 2024-09-16T00:09:54.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526791+0000", 2024-09-16T00:09:54.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6997999999999997e-05 2024-09-16T00:09:54.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526850+0000", 2024-09-16T00:09:54.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9184000000000001e-05 2024-09-16T00:09:54.458 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.458 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.458 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.458 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11098 3.15 e86/20 3:aa8f498f:::100000030ba.00000000:head v 86'170, mlcod=86'168)", 2024-09-16T00:09:54.459 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:09:54.459 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.407452015000001, 2024-09-16T00:09:54.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013832641999999999, 2024-09-16T00:09:54.459 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.459 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.459 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.459 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:09:54.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:09:54.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021472+0000", 2024-09-16T00:09:54.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9489999999999999e-06 2024-09-16T00:09:54.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027628+0000", 2024-09-16T00:09:54.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061563859999999998 2024-09-16T00:09:54.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027645+0000", 2024-09-16T00:09:54.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7108e-05 2024-09-16T00:09:54.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027655+0000", 2024-09-16T00:09:54.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7999999999999993e-06 2024-09-16T00:09:54.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027712+0000", 2024-09-16T00:09:54.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6826000000000001e-05 2024-09-16T00:09:54.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027738+0000", 2024-09-16T00:09:54.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5601000000000001e-05 2024-09-16T00:09:54.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.035237+0000", 2024-09-16T00:09:54.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074998699999999996 2024-09-16T00:09:54.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.463 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.035303+0000", 2024-09-16T00:09:54.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5102000000000001e-05 2024-09-16T00:09:54.463 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.463 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.463 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.463 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11099 3.b 3:d2f5b389:::100000030e7.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:54.463 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:09:54.463 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.4034072, 2024-09-16T00:09:54.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012707456000000001, 2024-09-16T00:09:54.463 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.464 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.464 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.464 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:54.464 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:54.464 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:54.464 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11099 2024-09-16T00:09:54.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.464 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:09:54.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:09:54.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025517+0000", 2024-09-16T00:09:54.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:09:54.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026955+0000", 2024-09-16T00:09:54.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001437219 2024-09-16T00:09:54.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026982+0000", 2024-09-16T00:09:54.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7523999999999999e-05 2024-09-16T00:09:54.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026996+0000", 2024-09-16T00:09:54.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4379000000000001e-05 2024-09-16T00:09:54.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027076+0000", 2024-09-16T00:09:54.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9933999999999997e-05 2024-09-16T00:09:54.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027185+0000", 2024-09-16T00:09:54.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010875899999999999 2024-09-16T00:09:54.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:54.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027380+0000", 2024-09-16T00:09:54.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019477700000000001 2024-09-16T00:09:54.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038072+0000", 2024-09-16T00:09:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010692234 2024-09-16T00:09:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038115+0000", 2024-09-16T00:09:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3291999999999998e-05 2024-09-16T00:09:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038172+0000", 2024-09-16T00:09:54.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6981000000000002e-05 2024-09-16T00:09:54.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038222+0000", 2024-09-16T00:09:54.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9737000000000002e-05 2024-09-16T00:09:54.469 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.469 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.469 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.469 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11109 3.27 3:e5179c63:::10000003176.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:54.469 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:09:54.469 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.357942917000001, 2024-09-16T00:09:54.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012250990999999999, 2024-09-16T00:09:54.470 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.470 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.470 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.470 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:54.470 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:54.470 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:54.470 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11109 2024-09-16T00:09:54.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.470 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:09:54.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:09:54.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070981+0000", 2024-09-16T00:09:54.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.289e-06 2024-09-16T00:09:54.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072616+0000", 2024-09-16T00:09:54.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016347569999999999 2024-09-16T00:09:54.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072638+0000", 2024-09-16T00:09:54.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1512999999999999e-05 2024-09-16T00:09:54.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072649+0000", 2024-09-16T00:09:54.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1626e-05 2024-09-16T00:09:54.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072775+0000", 2024-09-16T00:09:54.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012553900000000001 2024-09-16T00:09:54.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072856+0000", 2024-09-16T00:09:54.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0882000000000003e-05 2024-09-16T00:09:54.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:54.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072990+0000", 2024-09-16T00:09:54.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013483599999999999 2024-09-16T00:09:54.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.078354+0000", 2024-09-16T00:09:54.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053639500000000001 2024-09-16T00:09:54.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083150+0000", 2024-09-16T00:09:54.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004795517 2024-09-16T00:09:54.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083186+0000", 2024-09-16T00:09:54.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6019000000000002e-05 2024-09-16T00:09:54.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083230+0000", 2024-09-16T00:09:54.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4063000000000001e-05 2024-09-16T00:09:54.475 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.476 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.476 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.476 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11234 3.b 3:d2f5b389:::100000030e7.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:54.476 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:09:54.476 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 13.805327149, 2024-09-16T00:09:54.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.015217661, 2024-09-16T00:09:54.476 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.476 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.476 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.476 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:54.476 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:54.477 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:54.477 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11234 2024-09-16T00:09:54.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.477 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:09:54.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:09:54.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623598+0000", 2024-09-16T00:09:54.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1580000000000001e-06 2024-09-16T00:09:54.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626844+0000", 2024-09-16T00:09:54.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032459670000000002 2024-09-16T00:09:54.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626868+0000", 2024-09-16T00:09:54.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3584e-05 2024-09-16T00:09:54.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626879+0000", 2024-09-16T00:09:54.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1545999999999999e-05 2024-09-16T00:09:54.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626950+0000", 2024-09-16T00:09:54.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0784999999999994e-05 2024-09-16T00:09:54.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.627051+0000", 2024-09-16T00:09:54.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010155899999999999 2024-09-16T00:09:54.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:54.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.627178+0000", 2024-09-16T00:09:54.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012614899999999999 2024-09-16T00:09:54.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.635220+0000", 2024-09-16T00:09:54.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0080421490000000002 2024-09-16T00:09:54.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638699+0000", 2024-09-16T00:09:54.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034797500000000002 2024-09-16T00:09:54.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638760+0000", 2024-09-16T00:09:54.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0319000000000003e-05 2024-09-16T00:09:54.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638812+0000", 2024-09-16T00:09:54.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:09:54.483 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.483 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.483 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.483 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11777 3.b 3:d342b4ff:::10000003113.00000000:head [write 0~4194304 [2@1171070]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:54.483 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:09:54.484 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 11.237687741, 2024-09-16T00:09:54.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013079176, 2024-09-16T00:09:54.484 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.484 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.484 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.484 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:54.484 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:54.484 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:54.484 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11777 2024-09-16T00:09:54.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.484 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:09:54.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:09:54.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.486 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.486 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191237+0000", 2024-09-16T00:09:54.486 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3369999999999998e-06 2024-09-16T00:09:54.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.486 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.487 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192825+0000", 2024-09-16T00:09:54.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015887189999999999 2024-09-16T00:09:54.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.487 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192846+0000", 2024-09-16T00:09:54.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0670999999999999e-05 2024-09-16T00:09:54.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192858+0000", 2024-09-16T00:09:54.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.223e-05 2024-09-16T00:09:54.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192957+0000", 2024-09-16T00:09:54.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8416000000000001e-05 2024-09-16T00:09:54.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.193031+0000", 2024-09-16T00:09:54.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4381999999999994e-05 2024-09-16T00:09:54.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:54.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.193164+0000", 2024-09-16T00:09:54.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013349100000000001 2024-09-16T00:09:54.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204144+0000", 2024-09-16T00:09:54.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010979917 2024-09-16T00:09:54.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204204+0000", 2024-09-16T00:09:54.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0025999999999998e-05 2024-09-16T00:09:54.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204265+0000", 2024-09-16T00:09:54.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0645e-05 2024-09-16T00:09:54.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204313+0000", 2024-09-16T00:09:54.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8341999999999999e-05 2024-09-16T00:09:54.491 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.491 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.491 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.491 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11981 3.c e87/20 3:33faa5a0:::10000003215.00000000:head v 87'186, mlcod=87'184)", 2024-09-16T00:09:54.491 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:09:54.491 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 10.597452918, 2024-09-16T00:09:54.491 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011828697000000001, 2024-09-16T00:09:54.491 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.491 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.491 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.491 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.492 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.492 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:09:54.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.492 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.492 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.492 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:09:54.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.492 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.492 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.492 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831471+0000", 2024-09-16T00:09:54.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6220000000000001e-06 2024-09-16T00:09:54.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.493 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.493 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836584+0000", 2024-09-16T00:09:54.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051133510000000004 2024-09-16T00:09:54.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.493 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.493 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836606+0000", 2024-09-16T00:09:54.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2079e-05 2024-09-16T00:09:54.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.493 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836624+0000", 2024-09-16T00:09:54.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8162e-05 2024-09-16T00:09:54.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836704+0000", 2024-09-16T00:09:54.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9867e-05 2024-09-16T00:09:54.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836730+0000", 2024-09-16T00:09:54.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6285999999999999e-05 2024-09-16T00:09:54.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.843247+0000", 2024-09-16T00:09:54.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0065165650000000002 2024-09-16T00:09:54.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.843298+0000", 2024-09-16T00:09:54.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0764999999999999e-05 2024-09-16T00:09:54.495 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.495 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.495 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.496 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13365 3.2 3:4298088d:::100000035be.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:54.496 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:09:54.496 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.4097509390000003, 2024-09-16T00:09:54.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014036442999999999, 2024-09-16T00:09:54.496 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.496 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.496 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.496 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:54.496 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:54.496 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:54.496 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13365 2024-09-16T00:09:54.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.497 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.497 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:09:54.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.497 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:09:54.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.498 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019174+0000", 2024-09-16T00:09:54.498 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1250000000000001e-06 2024-09-16T00:09:54.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.498 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.498 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.498 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020613+0000", 2024-09-16T00:09:54.498 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014391930000000001 2024-09-16T00:09:54.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.498 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.498 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.498 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020638+0000", 2024-09-16T00:09:54.498 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4657000000000001e-05 2024-09-16T00:09:54.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.499 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020651+0000", 2024-09-16T00:09:54.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3451e-05 2024-09-16T00:09:54.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.499 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020744+0000", 2024-09-16T00:09:54.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3010000000000006e-05 2024-09-16T00:09:54.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.500 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020912+0000", 2024-09-16T00:09:54.500 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016763800000000001 2024-09-16T00:09:54.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.500 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:54.500 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.021043+0000", 2024-09-16T00:09:54.500 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013099800000000001 2024-09-16T00:09:54.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.500 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.500 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026194+0000", 2024-09-16T00:09:54.500 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051510879999999998 2024-09-16T00:09:54.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.501 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033132+0000", 2024-09-16T00:09:54.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069377279999999998 2024-09-16T00:09:54.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.501 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033169+0000", 2024-09-16T00:09:54.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6897000000000003e-05 2024-09-16T00:09:54.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.501 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033207+0000", 2024-09-16T00:09:54.502 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658000000000002e-05 2024-09-16T00:09:54.502 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.502 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.502 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.502 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13368 3.27 3:e75b72a1:::100000032e8.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:54.502 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:09:54.502 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.402385056, 2024-09-16T00:09:54.502 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013615175, 2024-09-16T00:09:54.502 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.502 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.502 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.503 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:54.503 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:54.503 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:54.503 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13368 2024-09-16T00:09:54.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.503 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.503 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.503 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:09:54.503 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.503 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.504 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:09:54.504 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.504 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.504 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.504 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.504 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026539+0000", 2024-09-16T00:09:54.504 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.798e-06 2024-09-16T00:09:54.504 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.504 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.504 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.504 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027825+0000", 2024-09-16T00:09:54.504 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012861839999999999 2024-09-16T00:09:54.504 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.505 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.505 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027850+0000", 2024-09-16T00:09:54.505 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5048000000000001e-05 2024-09-16T00:09:54.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.505 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.505 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027865+0000", 2024-09-16T00:09:54.505 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4939e-05 2024-09-16T00:09:54.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.505 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.506 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027944+0000", 2024-09-16T00:09:54.506 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8881999999999995e-05 2024-09-16T00:09:54.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.506 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.506 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.506 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.028043+0000", 2024-09-16T00:09:54.506 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9135000000000005e-05 2024-09-16T00:09:54.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.506 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.506 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:54.506 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.028158+0000", 2024-09-16T00:09:54.506 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011527999999999999 2024-09-16T00:09:54.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.506 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.507 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.507 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.039292+0000", 2024-09-16T00:09:54.507 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011133568999999999 2024-09-16T00:09:54.507 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.507 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.507 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.039948+0000", 2024-09-16T00:09:54.507 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00065625700000000004 2024-09-16T00:09:54.507 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.507 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.507 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.040061+0000", 2024-09-16T00:09:54.507 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011284400000000001 2024-09-16T00:09:54.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.508 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.508 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.040152+0000", 2024-09-16T00:09:54.508 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1238999999999998e-05 2024-09-16T00:09:54.508 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.508 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.508 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.508 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:13366 3.15 e88/20 3:aa8f498f:::100000030ba.00000000:head v 88'196, mlcod=88'194)", 2024-09-16T00:09:54.508 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:09:54.508 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.398959981, 2024-09-16T00:09:54.508 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012822817, 2024-09-16T00:09:54.509 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.509 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.509 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.509 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.509 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.509 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.509 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:09:54.509 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.509 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.509 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.509 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.509 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:09:54.509 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.510 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.510 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.510 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029963+0000", 2024-09-16T00:09:54.510 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4270000000000001e-06 2024-09-16T00:09:54.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.510 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.510 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.511 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034078+0000", 2024-09-16T00:09:54.511 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041148360000000002 2024-09-16T00:09:54.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.511 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.511 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.511 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034094+0000", 2024-09-16T00:09:54.511 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.535e-05 2024-09-16T00:09:54.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.511 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.511 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.511 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034103+0000", 2024-09-16T00:09:54.511 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:09:54.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.512 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.512 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.512 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.036791+0000", 2024-09-16T00:09:54.512 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026878340000000001 2024-09-16T00:09:54.512 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.512 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.512 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.512 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.036818+0000", 2024-09-16T00:09:54.512 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6885000000000001e-05 2024-09-16T00:09:54.513 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.513 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.513 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.513 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.042713+0000", 2024-09-16T00:09:54.513 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058950399999999998 2024-09-16T00:09:54.513 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.513 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.513 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.513 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.042785+0000", 2024-09-16T00:09:54.513 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1927000000000004e-05 2024-09-16T00:09:54.513 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.513 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.513 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.514 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.514 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.514 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13405 3.27 3:e708eda4:::100000034c4.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:54.514 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:09:54.514 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 6.2727582680000005, 2024-09-16T00:09:54.514 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012003333, 2024-09-16T00:09:54.514 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.514 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.514 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.514 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:54.514 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:54.515 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:54.515 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13405 2024-09-16T00:09:54.515 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.515 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.515 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.515 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.515 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:09:54.515 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.515 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.515 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.516 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.516 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:09:54.516 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.516 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.516 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.516 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.516 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156166+0000", 2024-09-16T00:09:54.516 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6230000000000001e-06 2024-09-16T00:09:54.516 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.516 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.516 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.516 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157868+0000", 2024-09-16T00:09:54.516 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017017 2024-09-16T00:09:54.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.517 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.517 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.517 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157891+0000", 2024-09-16T00:09:54.517 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2931000000000001e-05 2024-09-16T00:09:54.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.517 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.517 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.517 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157904+0000", 2024-09-16T00:09:54.517 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2763e-05 2024-09-16T00:09:54.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.517 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.517 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.517 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157988+0000", 2024-09-16T00:09:54.518 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3849000000000003e-05 2024-09-16T00:09:54.518 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.518 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.518 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.518 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.158083+0000", 2024-09-16T00:09:54.518 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5939000000000004e-05 2024-09-16T00:09:54.518 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.518 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.518 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:54.518 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.158194+0000", 2024-09-16T00:09:54.518 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110963 2024-09-16T00:09:54.518 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.518 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.518 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.519 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.164635+0000", 2024-09-16T00:09:54.519 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0064405570000000004 2024-09-16T00:09:54.519 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.519 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.519 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.519 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168088+0000", 2024-09-16T00:09:54.519 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034534639999999998 2024-09-16T00:09:54.519 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.519 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.519 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.519 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168140+0000", 2024-09-16T00:09:54.519 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1041000000000001e-05 2024-09-16T00:09:54.519 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.520 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.520 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.520 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168167+0000", 2024-09-16T00:09:54.520 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7503000000000001e-05 2024-09-16T00:09:54.520 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.520 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.520 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.520 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.520 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.521 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13685 3.b 3:d26c1580:::100000032e4.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:54.521 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:09:54.521 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.0940713029999998, 2024-09-16T00:09:54.521 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014868124999999999, 2024-09-16T00:09:54.521 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.521 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.521 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.521 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:54.521 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:54.521 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:54.521 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13685 2024-09-16T00:09:54.521 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.521 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.522 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.522 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.522 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:09:54.522 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.522 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.522 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.522 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.522 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:09:54.522 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.522 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.523 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.523 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.523 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334854+0000", 2024-09-16T00:09:54.523 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0000000000000001e-06 2024-09-16T00:09:54.523 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.523 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.523 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.523 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337155+0000", 2024-09-16T00:09:54.523 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023016870000000002 2024-09-16T00:09:54.524 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.524 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.524 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.524 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337178+0000", 2024-09-16T00:09:54.524 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2898e-05 2024-09-16T00:09:54.524 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.524 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.524 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.524 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337190+0000", 2024-09-16T00:09:54.525 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1817999999999999e-05 2024-09-16T00:09:54.525 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.525 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.525 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.525 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337301+0000", 2024-09-16T00:09:54.525 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110951 2024-09-16T00:09:54.525 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.525 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.525 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.526 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337403+0000", 2024-09-16T00:09:54.526 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000102118 2024-09-16T00:09:54.526 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.526 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.526 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:54.526 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337545+0000", 2024-09-16T00:09:54.526 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014167199999999999 2024-09-16T00:09:54.526 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.526 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.526 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.527 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.346336+0000", 2024-09-16T00:09:54.527 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087911450000000002 2024-09-16T00:09:54.527 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.527 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.527 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.527 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349641+0000", 2024-09-16T00:09:54.527 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033047570000000002 2024-09-16T00:09:54.527 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.527 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.527 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.527 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349680+0000", 2024-09-16T00:09:54.527 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9054999999999998e-05 2024-09-16T00:09:54.527 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.528 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.528 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.528 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349719+0000", 2024-09-16T00:09:54.528 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9023999999999998e-05 2024-09-16T00:09:54.528 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.528 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.528 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.528 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.528 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.528 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14624 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:54.528 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:09:54.528 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.8143121020000001, 2024-09-16T00:09:54.528 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012123376, 2024-09-16T00:09:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14624 2024-09-16T00:09:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:09:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.529 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:09:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614614+0000", 2024-09-16T00:09:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9500000000000003e-06 2024-09-16T00:09:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.530 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616480+0000", 2024-09-16T00:09:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018663169999999999 2024-09-16T00:09:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616506+0000", 2024-09-16T00:09:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6205999999999999e-05 2024-09-16T00:09:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616517+0000", 2024-09-16T00:09:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0399e-05 2024-09-16T00:09:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.531 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.532 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616595+0000", 2024-09-16T00:09:54.532 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8170999999999994e-05 2024-09-16T00:09:54.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.532 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.532 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.532 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616683+0000", 2024-09-16T00:09:54.532 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8578999999999993e-05 2024-09-16T00:09:54.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.532 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.532 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:54.532 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616825+0000", 2024-09-16T00:09:54.532 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014186299999999999 2024-09-16T00:09:54.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.532 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.533 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.624948+0000", 2024-09-16T00:09:54.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081230420000000005 2024-09-16T00:09:54.533 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.533 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.533 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626617+0000", 2024-09-16T00:09:54.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016681980000000001 2024-09-16T00:09:54.533 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.533 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.533 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626678+0000", 2024-09-16T00:09:54.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1362e-05 2024-09-16T00:09:54.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.534 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.534 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.534 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626733+0000", 2024-09-16T00:09:54.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5288999999999999e-05 2024-09-16T00:09:54.534 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.534 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.534 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.534 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.534 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14672 3.29 3:9641a85f:::1000000360e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:54.534 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:09:54.534 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.609854925, 2024-09-16T00:09:54.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012258025000000001, 2024-09-16T00:09:54.535 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.535 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.535 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.535 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:54.535 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:54.535 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:54.535 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14672 2024-09-16T00:09:54.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.535 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.535 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.535 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.535 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:09:54.535 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.536 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.536 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.536 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.536 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:09:54.536 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.536 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.536 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.536 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.536 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819070+0000", 2024-09-16T00:09:54.536 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0589999999999998e-06 2024-09-16T00:09:54.536 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.536 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.536 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.536 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821701+0000", 2024-09-16T00:09:54.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026309100000000002 2024-09-16T00:09:54.537 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.537 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.537 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.537 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821727+0000", 2024-09-16T00:09:54.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6372000000000001e-05 2024-09-16T00:09:54.537 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.537 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.537 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.537 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821738+0000", 2024-09-16T00:09:54.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0759e-05 2024-09-16T00:09:54.537 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.537 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.538 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.538 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821821+0000", 2024-09-16T00:09:54.538 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2854000000000003e-05 2024-09-16T00:09:54.538 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.538 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.538 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.538 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821920+0000", 2024-09-16T00:09:54.538 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8900999999999994e-05 2024-09-16T00:09:54.538 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.538 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.538 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:54.538 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.822084+0000", 2024-09-16T00:09:54.538 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016432799999999999 2024-09-16T00:09:54.538 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.539 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.539 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.539 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.829148+0000", 2024-09-16T00:09:54.539 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0070641460000000003 2024-09-16T00:09:54.539 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.539 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.539 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.539 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831242+0000", 2024-09-16T00:09:54.539 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002093293 2024-09-16T00:09:54.539 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.539 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.539 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.539 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831280+0000", 2024-09-16T00:09:54.540 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8707000000000001e-05 2024-09-16T00:09:54.540 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.540 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.540 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.540 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831325+0000", 2024-09-16T00:09:54.540 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4696e-05 2024-09-16T00:09:54.540 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.540 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.540 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.540 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.540 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.540 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14829 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304 [2@2556691]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:54.540 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:09:54.540 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.85733550700000005, 2024-09-16T00:09:54.541 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014399683, 2024-09-16T00:09:54.541 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.541 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.541 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.541 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:54.541 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:54.541 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:54.541 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14829 2024-09-16T00:09:54.541 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.541 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.541 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.541 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.541 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:09:54.541 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.542 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.542 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:09:54.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.542 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.542 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571590+0000", 2024-09-16T00:09:54.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:09:54.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.542 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.542 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573133+0000", 2024-09-16T00:09:54.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015422109999999999 2024-09-16T00:09:54.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.543 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.543 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573161+0000", 2024-09-16T00:09:54.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8045e-05 2024-09-16T00:09:54.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.543 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.543 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573174+0000", 2024-09-16T00:09:54.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3193999999999999e-05 2024-09-16T00:09:54.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573365+0000", 2024-09-16T00:09:54.544 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019096400000000001 2024-09-16T00:09:54.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573467+0000", 2024-09-16T00:09:54.544 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101789 2024-09-16T00:09:54.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:54.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573610+0000", 2024-09-16T00:09:54.544 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143404 2024-09-16T00:09:54.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.545 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.545 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.545 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.583614+0000", 2024-09-16T00:09:54.545 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010004422000000001 2024-09-16T00:09:54.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.545 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.545 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.545 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585556+0000", 2024-09-16T00:09:54.545 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019414009999999999 2024-09-16T00:09:54.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.545 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.545 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585939+0000", 2024-09-16T00:09:54.546 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038340999999999999 2024-09-16T00:09:54.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.546 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.546 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585986+0000", 2024-09-16T00:09:54.546 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6924e-05 2024-09-16T00:09:54.546 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.546 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.546 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.546 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.546 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14914 3.b 3:d23f6103:::10000003277.00000000:head [write 0~4194304 [3@58131]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:54.547 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:09:54.547 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.54613173599999998, 2024-09-16T00:09:54.547 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013007654, 2024-09-16T00:09:54.547 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.547 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.547 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.547 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:54.547 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:54.547 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:54.547 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14914 2024-09-16T00:09:54.547 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.548 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.548 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.548 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:09:54.548 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.548 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.548 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:09:54.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.549 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882793+0000", 2024-09-16T00:09:54.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.65e-06 2024-09-16T00:09:54.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.549 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885193+0000", 2024-09-16T00:09:54.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024003449999999999 2024-09-16T00:09:54.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.550 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885214+0000", 2024-09-16T00:09:54.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0398e-05 2024-09-16T00:09:54.550 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.550 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885224+0000", 2024-09-16T00:09:54.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0253e-05 2024-09-16T00:09:54.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.551 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.551 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.551 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885299+0000", 2024-09-16T00:09:54.551 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5434999999999999e-05 2024-09-16T00:09:54.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.551 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.551 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.551 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885389+0000", 2024-09-16T00:09:54.551 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9209999999999995e-05 2024-09-16T00:09:54.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.551 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.552 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:54.552 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885485+0000", 2024-09-16T00:09:54.552 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6218999999999995e-05 2024-09-16T00:09:54.552 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.552 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.552 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.552 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.893457+0000", 2024-09-16T00:09:54.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0079722009999999999 2024-09-16T00:09:54.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.553 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895681+0000", 2024-09-16T00:09:54.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022241449999999999 2024-09-16T00:09:54.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.553 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895732+0000", 2024-09-16T00:09:54.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1269e-05 2024-09-16T00:09:54.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.554 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895798+0000", 2024-09-16T00:09:54.554 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5529000000000002e-05 2024-09-16T00:09:54.554 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.554 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.554 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.554 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.554 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15058 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304 [2@2556691]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-16T00:09:54.555 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:09:54.555 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.093763141999999994, 2024-09-16T00:09:54.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012275842, 2024-09-16T00:09:54.555 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:54.555 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:54.555 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:54.555 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:54.555 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:54.555 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:54.555 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15058 2024-09-16T00:09:54.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.556 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:54.556 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.556 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:54.556 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:09:54.556 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.556 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.556 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.556 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:54.556 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:09:54.556 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:54.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:54.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335161+0000", 2024-09-16T00:09:54.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.367e-06 2024-09-16T00:09:54.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:54.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337545+0000", 2024-09-16T00:09:54.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023839040000000001 2024-09-16T00:09:54.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:54.558 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337572+0000", 2024-09-16T00:09:54.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.673e-05 2024-09-16T00:09:54.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:54.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337583+0000", 2024-09-16T00:09:54.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1048e-05 2024-09-16T00:09:54.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.559 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:54.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337660+0000", 2024-09-16T00:09:54.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7459999999999994e-05 2024-09-16T00:09:54.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:54.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337770+0000", 2024-09-16T00:09:54.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109634 2024-09-16T00:09:54.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:54.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337926+0000", 2024-09-16T00:09:54.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015601999999999999 2024-09-16T00:09:54.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:54.561 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.345689+0000", 2024-09-16T00:09:54.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0077625630000000001 2024-09-16T00:09:54.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:54.561 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347324+0000", 2024-09-16T00:09:54.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016351479999999999 2024-09-16T00:09:54.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:54.561 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347387+0000", 2024-09-16T00:09:54.562 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3199000000000003e-05 2024-09-16T00:09:54.562 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:54.562 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:54.562 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:54.562 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347435+0000", 2024-09-16T00:09:54.562 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7769000000000003e-05 2024-09-16T00:09:54.562 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.562 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.562 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.562 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:54.562 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:54.562 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:54.564 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:09:54.568 INFO:tasks.workunit.client.0.smithi139.stdout:5/973: sync 2024-09-16T00:09:54.568 INFO:tasks.workunit.client.0.smithi139.stdout:5/974: write d1/d13/d3f/d14d/f142 [829672,81790] 0 2024-09-16T00:09:54.568 INFO:tasks.workunit.client.0.smithi139.stdout:0/960: rename d6/d1c/d41/d44/dd/c5b to d6/d1c/d41/d44/d6b/d125/c12b 0 2024-09-16T00:09:54.568 INFO:tasks.workunit.client.0.smithi139.stdout:0/961: readlink d6/d1c/d41/d63/l12a 0 2024-09-16T00:09:54.568 INFO:tasks.workunit.client.0.smithi139.stdout:6/881: getdents d8/d29/d83 0 2024-09-16T00:09:54.568 INFO:tasks.workunit.client.0.smithi139.stdout:0/962: chown d6/d1c/d41/d44/dd/d56/fc4 73080247 1 2024-09-16T00:09:54.568 INFO:tasks.workunit.client.0.smithi139.stdout:0/963: write d6/d1c/d41/f6a [4457513,18047] 0 2024-09-16T00:09:54.568 INFO:tasks.workunit.client.0.smithi139.stdout:6/882: write d8/d29/d83/d26/d45/d59/fb8 [1124241,59653] 0 2024-09-16T00:09:54.568 INFO:tasks.workunit.client.0.smithi139.stdout:0/964: symlink d6/d1c/df7/l12c 0 2024-09-16T00:09:54.569 INFO:tasks.workunit.client.0.smithi139.stdout:5/975: dwrite d1/d13/d3f/d88/f10f [0,4194304] 0 2024-09-16T00:09:54.569 INFO:tasks.workunit.client.0.smithi139.stdout:0/965: mknod d6/d5a/d70/d7b/c12d 0 2024-09-16T00:09:54.569 INFO:tasks.workunit.client.0.smithi139.stdout:0/966: write d6/d1c/d41/d63/d82/f83 [4085685,95611] 0 2024-09-16T00:09:54.569 INFO:tasks.workunit.client.0.smithi139.stdout:5/976: unlink d1/d13/d3f/ce2 0 2024-09-16T00:09:54.569 INFO:tasks.workunit.client.0.smithi139.stdout:0/967: mkdir d6/d1c/df7/d12e 0 2024-09-16T00:09:54.569 INFO:tasks.workunit.client.0.smithi139.stdout:5/977: read d1/d13/d3f/d14d/d79/da5/d10b/d11e/d10d/f55 [63406,123618] 0 2024-09-16T00:09:54.569 INFO:tasks.workunit.client.0.smithi139.stdout:5/978: chown d1/d13/d3f/d14d 1426025 1 2024-09-16T00:09:54.569 INFO:tasks.workunit.client.0.smithi139.stdout:0/968: link d6/d1c/d41/d63/l80 d6/d1c/df7/d100/l12f 0 2024-09-16T00:09:54.569 INFO:tasks.workunit.client.0.smithi139.stdout:5/979: creat d1/d21/d49/d53/d6a/d7b/da1/f155 x:0 0 0 2024-09-16T00:09:54.569 INFO:tasks.workunit.client.0.smithi139.stdout:0/969: read - d6/d1c/d41/d44/dd/d56/fac zero size 2024-09-16T00:09:54.569 INFO:tasks.workunit.client.0.smithi139.stdout:0/970: dread - d6/d1c/d41/d44/dd/f46 zero size 2024-09-16T00:09:54.569 INFO:tasks.workunit.client.0.smithi139.stdout:0/971: rename d6/d1c/d41/d44/d58/f60 to d6/d1c/d41/d76/f130 0 2024-09-16T00:09:54.569 INFO:tasks.workunit.client.0.smithi139.stdout:0/972: stat d6/d5a/d70/d7b 0 2024-09-16T00:09:54.569 INFO:tasks.workunit.client.0.smithi139.stdout:0/973: chown d6/f64 216630 1 2024-09-16T00:09:54.570 INFO:tasks.workunit.client.0.smithi139.stdout:5/980: dwrite d1/d21/d49/d53/d6a/d7b/d78/ff2 [0,4194304] 0 2024-09-16T00:09:54.570 INFO:tasks.workunit.client.0.smithi139.stdout:0/974: dwrite d6/d1c/f4c [4194304,4194304] 0 2024-09-16T00:09:54.570 INFO:tasks.workunit.client.0.smithi139.stdout:5/981: unlink d1/d21/d49/d53/d6a/d7b/da1/db7/fee 0 2024-09-16T00:09:54.570 INFO:tasks.workunit.client.0.smithi139.stdout:0/975: write d6/d1c/d41/d76/f95 [76949,47074] 0 2024-09-16T00:09:54.570 INFO:tasks.workunit.client.0.smithi139.stdout:0/976: creat d6/d1c/d41/d78/df2/f131 x:0 0 0 2024-09-16T00:09:54.570 INFO:tasks.workunit.client.0.smithi139.stdout:0/977: truncate d6/d1c/d41/d44/f3e 4718015 0 2024-09-16T00:09:54.570 INFO:tasks.workunit.client.0.smithi139.stdout:5/982: dread d1/d21/fa8 [0,4194304] 0 2024-09-16T00:09:54.570 INFO:tasks.workunit.client.0.smithi139.stdout:5/983: fdatasync d1/d13/d22/dd5/f14f 0 2024-09-16T00:09:54.570 INFO:tasks.workunit.client.0.smithi139.stdout:0/978: getdents d6/d1c/d41/d76 0 2024-09-16T00:09:54.570 INFO:tasks.workunit.client.0.smithi139.stdout:0/979: read - d6/d1c/d41/d44/d6b/fde zero size 2024-09-16T00:09:54.570 INFO:tasks.workunit.client.0.smithi139.stdout:5/984: fdatasync d1/d21/d49/d53/d6a/fd2 0 2024-09-16T00:09:54.570 INFO:tasks.workunit.client.0.smithi139.stdout:5/985: truncate d1/d13/d3f/d14d/dd0/f127 46095 0 2024-09-16T00:09:54.570 INFO:tasks.workunit.client.0.smithi139.stdout:5/986: fdatasync d1/d21/d49/f7f 0 2024-09-16T00:09:54.571 INFO:tasks.workunit.client.0.smithi139.stdout:0/980: dwrite d6/d1c/d41/d44/dd/d56/fac [0,4194304] 0 2024-09-16T00:09:54.571 INFO:tasks.workunit.client.0.smithi139.stdout:0/981: rmdir d6/d1c/d41/d78 39 2024-09-16T00:09:54.571 INFO:tasks.workunit.client.0.smithi139.stdout:5/987: truncate d1/d21/d49/f98 363952 0 2024-09-16T00:09:54.571 INFO:tasks.workunit.client.0.smithi139.stdout:0/982: creat d6/d1c/df7/f132 x:0 0 0 2024-09-16T00:09:54.571 INFO:tasks.workunit.client.0.smithi139.stdout:0/983: write d6/d1c/d41/f72 [48788,36484] 0 2024-09-16T00:09:54.571 INFO:tasks.workunit.client.0.smithi139.stdout:5/988: dwrite d1/d21/d49/d53/d6a/f8b [0,4194304] 0 2024-09-16T00:09:54.571 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:54.563+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:54.572 INFO:tasks.workunit.client.0.smithi139.stdout:5/989: truncate d1/d13/d3f/d88/f10f 4962682 0 2024-09-16T00:09:54.572 INFO:tasks.workunit.client.0.smithi139.stdout:5/990: symlink d1/d13/d3f/d5e/l156 0 2024-09-16T00:09:54.572 INFO:tasks.workunit.client.0.smithi139.stdout:5/991: rename d1/d13/d3f/d14d/d79/da5 to d1/d13/d3f/d14d/dd0/d157 0 2024-09-16T00:09:54.572 INFO:tasks.workunit.client.0.smithi139.stdout:5/992: mknod d1/d13/d3b/c158 0 2024-09-16T00:09:54.574 INFO:tasks.workunit.client.0.smithi139.stdout:5/993: symlink d1/d21/l159 0 2024-09-16T00:09:54.575 INFO:tasks.workunit.client.0.smithi139.stdout:0/984: dread d6/d1c/d41/d44/dd/f67 [0,4194304] 0 2024-09-16T00:09:54.579 INFO:tasks.workunit.client.0.smithi139.stdout:0/985: link d6/d5a/d70/d7b/l112 d6/d1c/d41/d44/l133 0 2024-09-16T00:09:54.580 INFO:tasks.workunit.client.0.smithi139.stdout:0/986: readlink d6/d1c/d41/d44/l10d 0 2024-09-16T00:09:54.580 INFO:tasks.workunit.client.0.smithi139.stdout:5/994: truncate d1/f4a 2627718 0 2024-09-16T00:09:54.586 INFO:tasks.workunit.client.0.smithi139.stdout:0/987: dwrite d6/f64 [0,4194304] 0 2024-09-16T00:09:54.587 INFO:tasks.workunit.client.0.smithi139.stdout:0/988: fdatasync d6/d1c/d41/fbb 0 2024-09-16T00:09:54.588 INFO:tasks.workunit.client.0.smithi139.stdout:0/989: readlink d6/d1c/d41/d44/d6b/lda 0 2024-09-16T00:09:54.588 INFO:tasks.workunit.client.0.smithi139.stdout:0/990: chown d6/d1c/d41/d44/dd/f11 2027624183 1 2024-09-16T00:09:54.597 INFO:tasks.workunit.client.0.smithi139.stdout:5/995: dread d1/d21/d49/d53/d6a/d7b/d99/fac [0,4194304] 0 2024-09-16T00:09:54.613 INFO:tasks.workunit.client.0.smithi139.stdout:0/991: mknod d6/d1c/d41/d44/dd/d56/dbd/c134 0 2024-09-16T00:09:54.617 INFO:tasks.workunit.client.0.smithi139.stdout:5/996: fsync d1/d21/d49/d53/d6a/d7b/d99/feb 0 2024-09-16T00:09:54.617 INFO:tasks.workunit.client.0.smithi139.stdout:0/992: mkdir d6/d5a/d70/d135 0 2024-09-16T00:09:54.617 INFO:tasks.workunit.client.0.smithi139.stdout:5/997: mknod d1/d13/d3f/d5e/dc2/d10c/c15a 0 2024-09-16T00:09:54.618 INFO:tasks.workunit.client.0.smithi139.stdout:0/993: creat d6/d5a/d70/d135/f136 x:0 0 0 2024-09-16T00:09:54.621 INFO:tasks.workunit.client.0.smithi139.stdout:5/998: truncate d1/d13/d3f/d14d/f138 1552352 0 2024-09-16T00:09:54.622 INFO:tasks.workunit.client.0.smithi139.stdout:7/984: sync 2024-09-16T00:09:54.622 INFO:tasks.workunit.client.0.smithi139.stdout:5/999: dread d1/d13/fd9 [0,4194304] 0 2024-09-16T00:09:54.623 INFO:tasks.workunit.client.0.smithi139.stdout:7/985: write d5/d16/d22/d25/d63/f97 [1102414,113520] 0 2024-09-16T00:09:54.625 INFO:tasks.workunit.client.0.smithi139.stdout:7/986: chown d5/d26/d74/cc4 467273064 1 2024-09-16T00:09:54.626 INFO:tasks.workunit.client.0.smithi139.stdout:7/987: chown d5/d37/d70/f13e 14 1 2024-09-16T00:09:54.631 INFO:tasks.workunit.client.0.smithi139.stdout:0/994: dread d6/d1c/f4c [0,4194304] 0 2024-09-16T00:09:54.631 INFO:tasks.workunit.client.0.smithi139.stdout:0/995: chown c5 30523221 1 2024-09-16T00:09:54.632 INFO:tasks.workunit.client.0.smithi139.stdout:0/996: chown d6/d5a/d70/l9d 116820392 1 2024-09-16T00:09:54.637 INFO:tasks.workunit.client.0.smithi139.stdout:7/988: dread d5/d16/d22/f6f [0,4194304] 0 2024-09-16T00:09:54.640 INFO:tasks.workunit.client.0.smithi139.stdout:7/989: mkdir d5/d16/d6d/d7f/dc2/d155 0 2024-09-16T00:09:54.643 INFO:tasks.workunit.client.0.smithi139.stdout:0/997: dread d6/d1c/d41/fbb [0,4194304] 0 2024-09-16T00:09:54.645 INFO:tasks.workunit.client.0.smithi139.stdout:7/990: dwrite d5/d16/d6d/d153/dd6/df4/f14a [0,4194304] 0 2024-09-16T00:09:54.646 INFO:tasks.workunit.client.0.smithi139.stdout:7/991: chown d5 12926828 1 2024-09-16T00:09:54.650 INFO:tasks.workunit.client.0.smithi139.stdout:7/992: unlink d5/d16/d6d/d153/dc5/cd9 0 2024-09-16T00:09:54.652 INFO:tasks.workunit.client.0.smithi139.stdout:7/993: chown d5/d16/d22/d25/d63/fb9 0 1 2024-09-16T00:09:54.654 INFO:tasks.workunit.client.0.smithi139.stdout:7/994: creat d5/d37/d118/f156 x:0 0 0 2024-09-16T00:09:54.655 INFO:tasks.workunit.client.0.smithi139.stdout:7/995: write d5/d16/d22/f6f [260990,1194] 0 2024-09-16T00:09:54.658 INFO:tasks.workunit.client.0.smithi139.stdout:0/998: dwrite d6/d1c/d41/d44/dd/fd0 [0,4194304] 0 2024-09-16T00:09:54.659 INFO:tasks.workunit.client.0.smithi139.stdout:7/996: rename d5/d16/d6d/d7f/dc2/d50/d44/d55/f6c to d5/d16/d87/f157 0 2024-09-16T00:09:54.662 INFO:tasks.workunit.client.0.smithi139.stdout:7/997: creat d5/d26/d46/dea/f158 x:0 0 0 2024-09-16T00:09:54.664 INFO:tasks.workunit.client.0.smithi139.stdout:0/999: write d6/d5a/d70/f53 [2658499,114690] 0 2024-09-16T00:09:54.664 INFO:tasks.workunit.client.0.smithi139.stdout:7/998: mkdir d5/d26/d46/dea/d126/d159 0 2024-09-16T00:09:54.664 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:54.664+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:54.676 INFO:tasks.workunit.client.0.smithi139.stdout:7/999: dread d5/d16/d6d/d7f/dc2/d50/d44/f84 [0,4194304] 0 2024-09-16T00:09:54.694 INFO:tasks.workunit.client.0.smithi139.stdout:1/787: sync 2024-09-16T00:09:54.694 INFO:tasks.workunit.client.0.smithi139.stdout:3/865: sync 2024-09-16T00:09:54.695 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:54.695 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:54.695 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:54.695 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:54.698 INFO:tasks.workunit.client.0.smithi139.stdout:1/788: write d7/d8/d27/d68/d7c/d83/fca [1382898,112132] 0 2024-09-16T00:09:54.700 INFO:tasks.workunit.client.0.smithi139.stdout:1/789: unlink d7/d8/d40/lb6 0 2024-09-16T00:09:54.701 INFO:tasks.workunit.client.0.smithi139.stdout:1/790: chown d7/d8/d40/d46/d89/c90 11590665 1 2024-09-16T00:09:54.702 INFO:tasks.workunit.client.0.smithi139.stdout:1/791: chown d7/d8/d40/d48/d67/d7d/dc3 1353 1 2024-09-16T00:09:54.703 INFO:tasks.workunit.client.0.smithi139.stdout:1/792: chown d7/d8/d40/d47/dae/fde 1734774 1 2024-09-16T00:09:54.705 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:09:54.706 INFO:tasks.workunit.client.0.smithi139.stdout:1/793: link d7/d8/l92 d7/d8/d1f/lec 0 2024-09-16T00:09:54.707 INFO:tasks.workunit.client.0.smithi139.stdout:3/866: dread d5/d7/f13 [0,4194304] 0 2024-09-16T00:09:54.708 INFO:tasks.workunit.client.0.smithi139.stdout:1/794: mkdir d7/d8/d27/d68/d7c/ded 0 2024-09-16T00:09:54.708 INFO:tasks.workunit.client.0.smithi139.stdout:3/867: fdatasync d5/d7/d23/de4/de8/da7/d9a/d101/d2f/f7a 0 2024-09-16T00:09:54.709 INFO:tasks.workunit.client.0.smithi139.stdout:1/795: truncate d7/d8/fa0 678331 0 2024-09-16T00:09:54.711 INFO:tasks.workunit.client.0.smithi139.stdout:3/868: rename d5/d7/d23/cae to d5/d7/d23/de4/de8/dd4/c12a 0 2024-09-16T00:09:54.712 INFO:tasks.workunit.client.0.smithi139.stdout:3/869: fdatasync d5/d7/d23/f94 0 2024-09-16T00:09:54.713 INFO:tasks.workunit.client.0.smithi139.stdout:1/796: getdents d7/d8 0 2024-09-16T00:09:54.714 INFO:tasks.workunit.client.0.smithi139.stdout:1/797: stat d7/d8/fa0 0 2024-09-16T00:09:54.714 INFO:tasks.workunit.client.0.smithi139.stdout:3/870: chown d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4d/dc6 680362999 1 2024-09-16T00:09:54.715 INFO:tasks.workunit.client.0.smithi139.stdout:1/798: fsync d7/d8/d27/fb0 0 2024-09-16T00:09:54.718 INFO:tasks.workunit.client.0.smithi139.stdout:1/799: creat d7/d8/d27/d68/d7c/d83/fee x:0 0 0 2024-09-16T00:09:54.719 INFO:tasks.workunit.client.0.smithi139.stdout:3/871: fdatasync d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/f80 0 2024-09-16T00:09:54.720 INFO:tasks.workunit.client.0.smithi139.stdout:3/872: stat d5/d7/d23/de4/dca/c121 0 2024-09-16T00:09:54.723 INFO:tasks.workunit.client.0.smithi139.stdout:3/873: rename d5/d7/d23/de4/d75/de1 to d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d12b 0 2024-09-16T00:09:54.725 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:09:54.726 INFO:tasks.workunit.client.0.smithi139.stdout:3/874: symlink d5/d7/dc/d9c/l12c 0 2024-09-16T00:09:54.726 INFO:tasks.workunit.client.0.smithi139.stdout:1/800: dwrite d7/d8/d40/d48/d53/d61/f87 [0,4194304] 0 2024-09-16T00:09:54.729 INFO:tasks.workunit.client.0.smithi139.stdout:1/801: truncate d7/d8/d40/d47/dae/fc5 104609 0 2024-09-16T00:09:54.734 INFO:tasks.workunit.client.0.smithi139.stdout:3/875: dwrite d5/d7/d23/de4/de8/da7/d9a/f103 [0,4194304] 0 2024-09-16T00:09:54.735 INFO:tasks.workunit.client.0.smithi139.stdout:6/883: sync 2024-09-16T00:09:54.737 INFO:tasks.workunit.client.0.smithi139.stdout:3/876: unlink d5/d5f/la9 0 2024-09-16T00:09:54.742 INFO:tasks.workunit.client.0.smithi139.stdout:1/802: truncate d7/d8/d1f/fcb 1240594 0 2024-09-16T00:09:54.742 INFO:tasks.workunit.client.0.smithi139.stdout:3/877: link d5/d5f/l104 d5/d7/dc2/l12d 0 2024-09-16T00:09:54.745 INFO:tasks.workunit.client.0.smithi139.stdout:3/878: mknod d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/db9/c12e 0 2024-09-16T00:09:54.746 INFO:tasks.workunit.client.0.smithi139.stdout:3/879: stat d5/d7/d23/de4/de8/f45 0 2024-09-16T00:09:54.748 INFO:tasks.workunit.client.0.smithi139.stdout:1/803: dwrite d7/d8/f11 [0,4194304] 0 2024-09-16T00:09:54.753 INFO:tasks.workunit.client.0.smithi139.stdout:1/804: rename d7/d8/d40/d48/d67/d7d/dc3/lce to d7/d8/d40/d48/lef 0 2024-09-16T00:09:54.754 INFO:tasks.workunit.client.0.smithi139.stdout:3/880: link d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/d76/dfb/l113 d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/d76/l12f 0 2024-09-16T00:09:54.755 INFO:tasks.workunit.client.0.smithi139.stdout:1/805: write d7/d8/d40/d48/d4a/d82/fd7 [226809,7610] 0 2024-09-16T00:09:54.756 INFO:tasks.workunit.client.0.smithi139.stdout:1/806: symlink d7/d8/d27/d68/lf0 0 2024-09-16T00:09:54.758 INFO:tasks.workunit.client.0.smithi139.stdout:3/881: dread d5/d7/d23/de4/de8/fde [0,4194304] 0 2024-09-16T00:09:54.759 INFO:tasks.workunit.client.0.smithi139.stdout:1/807: unlink d7/d8/d40/d48/l9c 0 2024-09-16T00:09:54.759 INFO:tasks.workunit.client.0.smithi139.stdout:1/808: chown d7/d8/d40/d48/f65 25737 1 2024-09-16T00:09:54.760 INFO:tasks.workunit.client.0.smithi139.stdout:3/882: rmdir d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68 39 2024-09-16T00:09:54.761 INFO:tasks.workunit.client.0.smithi139.stdout:3/883: chown d5/d7/d23/de4/de8/da7/d9a/f11b 45579961 1 2024-09-16T00:09:54.761 INFO:tasks.workunit.client.0.smithi139.stdout:1/809: read d7/d8/d27/d68/d6c/fb1 [977910,123011] 0 2024-09-16T00:09:54.762 INFO:tasks.workunit.client.0.smithi139.stdout:3/884: truncate d5/d7/f7d 620240 0 2024-09-16T00:09:54.763 INFO:tasks.workunit.client.0.smithi139.stdout:3/885: readlink d5/l57 0 2024-09-16T00:09:54.764 INFO:tasks.workunit.client.0.smithi139.stdout:3/886: dread d5/d7/d23/de4/de8/fde [0,4194304] 0 2024-09-16T00:09:54.765 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:54.764+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:54.766 INFO:tasks.workunit.client.0.smithi139.stdout:3/887: rmdir d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4d 39 2024-09-16T00:09:54.767 INFO:tasks.workunit.client.0.smithi139.stdout:3/888: chown d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/da1/fd3 7 1 2024-09-16T00:09:54.769 INFO:tasks.workunit.client.0.smithi139.stdout:3/889: mknod d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/d76/d122/c130 0 2024-09-16T00:09:54.771 INFO:tasks.workunit.client.0.smithi139.stdout:3/890: mknod d5/d7/d23/c131 0 2024-09-16T00:09:54.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.830+0000 7f42cb027640 1 Processor -- start 2024-09-16T00:09:54.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.830+0000 7f42cb027640 1 -- start start 2024-09-16T00:09:54.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.831+0000 7f42cb027640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f42c406e9d0 0x7f42c415cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:54.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.831+0000 7f42cb027640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c4071a40 0x7f42c415d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:54.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.831+0000 7f42cb027640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42c406ccd0 con 0x7f42c4071660 2024-09-16T00:09:54.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.831+0000 7f42cb027640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f42c406d2e0 con 0x7f42c406e9d0 2024-09-16T00:09:54.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.831+0000 7f42cb027640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f42c40ce910 con 0x7f42c4071a40 2024-09-16T00:09:54.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.831+0000 7f42c3fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f42c406e9d0 0x7f42c415cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:54.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.831+0000 7f42c8dc3640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f42c4071660 0x7f42c40c8900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59290/0 (socket says 172.21.15.139:59290) 2024-09-16T00:09:54.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.831+0000 7f42c8dc3640 1 -- 172.21.15.139:0/469454329 learned_addr learned my addr 172.21.15.139:0/469454329 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:54.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.831+0000 7f42c95c4640 1 --2- 172.21.15.139:0/469454329 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c4071a40 0x7f42c415d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:54.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.831+0000 7f42c3fff640 1 -- 172.21.15.139:0/469454329 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c4071a40 msgr2=0x7f42c415d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:54.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.831+0000 7f42c3fff640 1 --2- 172.21.15.139:0/469454329 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c4071a40 0x7f42c415d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:54.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.831+0000 7f42c3fff640 1 -- 172.21.15.139:0/469454329 >> v1:172.21.15.139:6789/0 conn(0x7f42c4071660 legacy=0x7f42c40c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:54.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.831+0000 7f42c3fff640 1 -- 172.21.15.139:0/469454329 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c40cebf0 con 0x7f42c406e9d0 2024-09-16T00:09:54.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.831+0000 7f42c37fe640 1 -- 172.21.15.139:0/469454329 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4139606907 0 0) 0x7f42c406ccd0 con 0x7f42c4071660 2024-09-16T00:09:54.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.831+0000 7f42c95c4640 1 --2- 172.21.15.139:0/469454329 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c4071a40 0x7f42c415d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:09:54.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.832+0000 7f42c3fff640 1 --2- 172.21.15.139:0/469454329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c406e9d0 0x7f42c415cc90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009d90 tx=0x7f42b402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cccaedad0a55df8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:54.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.832+0000 7f42c37fe640 1 -- 172.21.15.139:0/469454329 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4007740 con 0x7f42c406e9d0 2024-09-16T00:09:54.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.832+0000 7f42c37fe640 1 -- 172.21.15.139:0/469454329 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b40078e0 con 0x7f42c406e9d0 2024-09-16T00:09:54.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.832+0000 7f42c37fe640 1 -- 172.21.15.139:0/469454329 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4007cb0 con 0x7f42c406e9d0 2024-09-16T00:09:54.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.833+0000 7f42cb027640 1 -- 172.21.15.139:0/469454329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c406e9d0 msgr2=0x7f42c415cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:54.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.833+0000 7f42cb027640 1 --2- 172.21.15.139:0/469454329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c406e9d0 0x7f42c415cc90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009d90 tx=0x7f42b402a6f0 comp rx=0 tx=0).stop 2024-09-16T00:09:54.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.833+0000 7f42cb027640 1 -- 172.21.15.139:0/469454329 shutdown_connections 2024-09-16T00:09:54.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.833+0000 7f42cb027640 1 --2- 172.21.15.139:0/469454329 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c4071a40 0x7f42c415d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:54.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.833+0000 7f42cb027640 1 --2- 172.21.15.139:0/469454329 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c406e9d0 0x7f42c415cc90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:54.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.833+0000 7f42cb027640 1 -- 172.21.15.139:0/469454329 >> 172.21.15.139:0/469454329 conn(0x7f42c406fb00 msgr2=0x7f42c40738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:54.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.833+0000 7f42cb027640 1 -- 172.21.15.139:0/469454329 shutdown_connections 2024-09-16T00:09:54.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.833+0000 7f42cb027640 1 -- 172.21.15.139:0/469454329 wait complete. 2024-09-16T00:09:54.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.834+0000 7f42cb027640 1 Processor -- start 2024-09-16T00:09:54.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.834+0000 7f42cb027640 1 -- start start 2024-09-16T00:09:54.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42cb027640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c406e9d0 0x7f42c41e23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:54.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42cb027640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c4071660 0x7f42c41e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:54.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42cb027640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c4071a40 0x7f42c40b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:54.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42cb027640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42c40ce140 con 0x7f42c4071a40 2024-09-16T00:09:54.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42cb027640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f42c40ceed0 con 0x7f42c406e9d0 2024-09-16T00:09:54.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42cb027640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f42c40ce2c0 con 0x7f42c4071660 2024-09-16T00:09:54.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42c95c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c4071a40 0x7f42c40b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:54.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42c95c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c4071a40 0x7f42c40b20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:37628/0 (socket says 172.21.15.139:37628) 2024-09-16T00:09:54.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42c95c4640 1 -- 172.21.15.139:0/543489819 learned_addr learned my addr 172.21.15.139:0/543489819 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:54.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42c3fff640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c4071660 0x7f42c41e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:54.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42c8dc3640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c406e9d0 0x7f42c41e23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:54.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42c95c4640 1 -- 172.21.15.139:0/543489819 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c4071660 msgr2=0x7f42c41e5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:54.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42c95c4640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c4071660 0x7f42c41e5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:54.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42c95c4640 1 -- 172.21.15.139:0/543489819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c406e9d0 msgr2=0x7f42c41e23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:54.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42c95c4640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c406e9d0 0x7f42c41e23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:54.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42c95c4640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c40b2bb0 con 0x7f42c4071a40 2024-09-16T00:09:54.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42c95c4640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c4071a40 0x7f42c40b20d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00e610 tx=0x7f42ac00ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:54.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.835+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00f040 con 0x7f42c4071a40 2024-09-16T00:09:54.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.836+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42ac004230 con 0x7f42c4071a40 2024-09-16T00:09:54.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.836+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c41f0fe0 con 0x7f42c4071a40 2024-09-16T00:09:54.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.836+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac003610 con 0x7f42c4071a40 2024-09-16T00:09:54.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.836+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c41e1bf0 con 0x7f42c4071a40 2024-09-16T00:09:54.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.836+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f42c40c8900 con 0x7f42c4071a40 2024-09-16T00:09:54.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.837+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f42ac0037b0 con 0x7f42c4071a40 2024-09-16T00:09:54.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.838+0000 7f42c17fa640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f42a406c2b0 0x7f42a406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:54.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.838+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5973+0+0 (secure 0 0 0) 0x7f42ac08b0b0 con 0x7f42c4071a40 2024-09-16T00:09:54.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.838+0000 7f42c8dc3640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f42a406c2b0 0x7f42a406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:54.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.838+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v91) ==== 54+0+16 (secure 0 0 0) 0x7f42ac017020 con 0x7f42c4071a40 2024-09-16T00:09:54.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.838+0000 7f42cb027640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f42c41e1e30 0x7f42c4159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:54.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.838+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42c40cd980 con 0x7f42c41e1e30 2024-09-16T00:09:54.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.838+0000 7f42c8dc3640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f42a406c2b0 0x7f42a406e770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f42bc000f60 tx=0x7f42bc007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:54.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.838+0000 7f42c3fff640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f42c41e1e30 0x7f42c4159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:54.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.838+0000 7f42c3fff640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f42c41e1e30 0x7f42c4159df0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:54.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.839+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42c40cd980 con 0x7f42c41e1e30 2024-09-16T00:09:54.852 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:54.852 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:54.852 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:54.852 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:54.852 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:54.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.853+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f42c40cd6a0 con 0x7f42c41e1e30 2024-09-16T00:09:54.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.854+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f42c40cd6a0 con 0x7f42c41e1e30 2024-09-16T00:09:54.855 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:54.856 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:54.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.854+0000 7f42cb027640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f42c40cb110 0x7f42c41f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:54.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.854+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f42c415a840 con 0x7f42c40cb110 2024-09-16T00:09:54.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.855+0000 7f42c8dc3640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f42c40cb110 0x7f42c41f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:54.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.855+0000 7f42c8dc3640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f42c40cb110 0x7f42c41f3a30 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:54.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.855+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42c415a840 con 0x7f42c40cb110 2024-09-16T00:09:54.861 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:09:54.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.870+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f42c41e63f0 con 0x7f42c40cb110 2024-09-16T00:09:54.871 INFO:tasks.workunit.client.0.smithi139.stdout:6/884: sync 2024-09-16T00:09:54.872 INFO:tasks.workunit.client.0.smithi139.stdout:6/885: dread - d8/d29/d83/d26/d28/f109 zero size 2024-09-16T00:09:54.872 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.871+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f42c41e63f0 con 0x7f42c40cb110 2024-09-16T00:09:54.872 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:54.873 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:54.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.872+0000 7f42cb027640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f42c40b2610 0x7f42c41f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:54.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.872+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f42c41f1aa0 con 0x7f42c40b2610 2024-09-16T00:09:54.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.872+0000 7f42c3fff640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f42c40b2610 0x7f42c41f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:54.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.872+0000 7f42c3fff640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f42c40b2610 0x7f42c41f4480 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:54.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.873+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42c41f1aa0 con 0x7f42c40b2610 2024-09-16T00:09:54.874 INFO:tasks.workunit.client.0.smithi139.stdout:6/886: fdatasync d8/f39 0 2024-09-16T00:09:54.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.886+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f42c4062900 con 0x7f42c40b2610 2024-09-16T00:09:54.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.888+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f42c4062900 con 0x7f42c40b2610 2024-09-16T00:09:54.889 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:54.889 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:54.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.888+0000 7f42cb027640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f42c40631d0 0x7f42c40c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:54.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.888+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f42c40635f0 con 0x7f42c40631d0 2024-09-16T00:09:54.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.888+0000 7f42c95c4640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f42c40631d0 0x7f42c40c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:54.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.888+0000 7f42c95c4640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f42c40631d0 0x7f42c40c9110 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:54.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.889+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42ac05d450 con 0x7f42c40631d0 2024-09-16T00:09:54.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.902+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f42c40c9920 con 0x7f42c40631d0 2024-09-16T00:09:54.904 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:54.904+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:54.904 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.904+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f42c40c9920 con 0x7f42c40631d0 2024-09-16T00:09:54.905 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:54.905 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:54.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.904+0000 7f42cb027640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f42c402c640 0x7f42c402ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:54.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.904+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f42c41f1aa0 con 0x7f42c402c640 2024-09-16T00:09:54.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.904+0000 7f42c8dc3640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f42c402c640 0x7f42c402ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:54.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.904+0000 7f42c8dc3640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f42c402c640 0x7f42c402ca60 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:54.906 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.905+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42c41f1aa0 con 0x7f42c402c640 2024-09-16T00:09:54.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.918+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f42c41f1aa0 con 0x7f42c402c640 2024-09-16T00:09:54.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.921+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f42c41f1aa0 con 0x7f42c402c640 2024-09-16T00:09:54.922 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:54.923 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:54.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.921+0000 7f42cb027640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f42c402d4b0 0x7f42c4206dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:54.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.921+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f42c402d910 con 0x7f42c402d4b0 2024-09-16T00:09:54.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.922+0000 7f42c3fff640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f42c402d4b0 0x7f42c4206dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:54.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.922+0000 7f42c3fff640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f42c402d4b0 0x7f42c4206dc0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:54.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.923+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42c402d910 con 0x7f42c402d4b0 2024-09-16T00:09:54.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.936+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f42c402d910 con 0x7f42c402d4b0 2024-09-16T00:09:54.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.938+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f42c402d910 con 0x7f42c402d4b0 2024-09-16T00:09:54.939 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:54.940 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:54.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.938+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=92}) -- 0x7f42c402d910 con 0x7f42c4071a40 2024-09-16T00:09:54.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.938+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f42c402db20 con 0x7f42c4071a40 2024-09-16T00:09:54.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.938+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f42ac020040 con 0x7f42c4071a40 2024-09-16T00:09:54.940 INFO:teuthology.orchestra.run.smithi139.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T00:09:54.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.938+0000 7f42cb027640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f42c4207980 0x7f42c4209d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:54.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.938+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f42c420cd90 con 0x7f42c4207980 2024-09-16T00:09:54.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.939+0000 7f42c95c4640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f42c4207980 0x7f42c4209d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:54.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.939+0000 7f42c95c4640 1 --2- 172.21.15.139:0/543489819 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f42c4207980 0x7f42c4209d70 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:54.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.940+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42c420cd90 con 0x7f42c4207980 2024-09-16T00:09:54.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.952+0000 7f42cb027640 1 -- 172.21.15.139:0/543489819 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f42c420cd60 con 0x7f42c4207980 2024-09-16T00:09:54.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:54.954+0000 7f42c17fa640 1 -- 172.21.15.139:0/543489819 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f42c420cd60 con 0x7f42c4207980 2024-09-16T00:09:54.956 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:54.956 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:54.958 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T00:09:54.959 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T00:09:54.968 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:54.968 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:54.969 INFO:tasks.workunit.client.0.smithi139.stdout:3/891: sync 2024-09-16T00:09:54.970 INFO:tasks.workunit.client.0.smithi139.stdout:1/810: sync 2024-09-16T00:09:54.970 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:54.969+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:54.972 INFO:tasks.workunit.client.0.smithi139.stdout:1/811: creat d7/d8/d40/d48/d4a/d74/ddb/ff1 x:0 0 0 2024-09-16T00:09:54.973 INFO:tasks.workunit.client.0.smithi139.stdout:3/892: write d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/f126 [2248214,62468] 0 2024-09-16T00:09:54.976 INFO:tasks.workunit.client.0.smithi139.stdout:3/893: stat d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4d/f125 0 2024-09-16T00:09:54.977 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:09:54.982 INFO:tasks.workunit.client.0.smithi139.stdout:1/812: dread d7/d8/d27/f79 [0,4194304] 0 2024-09-16T00:09:54.983 INFO:tasks.workunit.client.0.smithi139.stdout:3/894: dwrite d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/fb5 [4194304,4194304] 0 2024-09-16T00:09:54.984 INFO:tasks.workunit.client.0.smithi139.stdout:1/813: write d7/d8/d40/d46/d8d/f95 [1112791,13047] 0 2024-09-16T00:09:54.987 INFO:tasks.workunit.client.0.smithi139.stdout:1/814: unlink d7/d8/d40/d47/c64 0 2024-09-16T00:09:54.988 INFO:tasks.workunit.client.0.smithi139.stdout:1/815: write d7/f18 [2232989,77555] 0 2024-09-16T00:09:54.989 INFO:tasks.workunit.client.0.smithi139.stdout:1/816: chown d7/d8/d27/d68/d7c/d83 1451427 1 2024-09-16T00:09:54.989 INFO:tasks.workunit.client.0.smithi139.stdout:1/817: write d7/f16 [1748431,66488] 0 2024-09-16T00:09:54.991 INFO:tasks.workunit.client.0.smithi139.stdout:1/818: fdatasync d7/d8/d27/d68/d7c/d83/fca 0 2024-09-16T00:09:54.992 INFO:tasks.workunit.client.0.smithi139.stdout:1/819: chown d7/d8/d40/d46/caa 1999458 1 2024-09-16T00:09:54.992 INFO:tasks.workunit.client.0.smithi139.stdout:1/820: chown d7/d8/d40/d46/d8d/f95 29 1 2024-09-16T00:09:54.995 INFO:tasks.workunit.client.0.smithi139.stdout:1/821: getdents d7/d8/d27/d68/d7c 0 2024-09-16T00:09:54.996 INFO:tasks.workunit.client.0.smithi139.stdout:1/822: write d7/f12 [3868053,84776] 0 2024-09-16T00:09:54.999 INFO:tasks.workunit.client.0.smithi139.stdout:1/823: symlink d7/d8/d40/dd3/de3/lf2 0 2024-09-16T00:09:55.003 INFO:tasks.workunit.client.0.smithi139.stdout:1/824: dwrite d7/d8/f21 [4194304,4194304] 0 2024-09-16T00:09:55.006 INFO:tasks.workunit.client.0.smithi139.stdout:1/825: creat d7/d8/d40/d47/dae/ff3 x:0 0 0 2024-09-16T00:09:55.007 INFO:tasks.workunit.client.0.smithi139.stdout:1/826: write d7/d8/d40/d48/d4a/d74/ddb/ff1 [621917,63492] 0 2024-09-16T00:09:55.009 INFO:tasks.workunit.client.0.smithi139.stdout:1/827: write d7/d8/d27/d68/fe5 [58718,52746] 0 2024-09-16T00:09:55.011 INFO:tasks.workunit.client.0.smithi139.stdout:1/828: symlink d7/d8/d40/d48/lf4 0 2024-09-16T00:09:55.012 INFO:tasks.workunit.client.0.smithi139.stdout:1/829: mkdir d7/d8/d27/d68/d7c/d83/df5 0 2024-09-16T00:09:55.015 INFO:tasks.workunit.client.0.smithi139.stdout:1/830: rename d7/d8/d40/d48/d4a/d82/fd7 to d7/d8/d40/d48/d4a/d74/ff6 0 2024-09-16T00:09:55.016 INFO:tasks.workunit.client.0.smithi139.stdout:1/831: dread - d7/d8/d40/d48/d4a/db8/fd0 zero size 2024-09-16T00:09:55.017 INFO:tasks.workunit.client.0.smithi139.stdout:1/832: mknod d7/d8/d27/d68/cf7 0 2024-09-16T00:09:55.020 INFO:tasks.workunit.client.0.smithi139.stdout:1/833: rename d7/l7b to d7/d8/d40/d48/dcc/dda/lf8 0 2024-09-16T00:09:55.021 INFO:tasks.workunit.client.0.smithi139.stdout:1/834: mkdir d7/d8/d27/d68/d7c/d83/df9 0 2024-09-16T00:09:55.023 INFO:tasks.workunit.client.0.smithi139.stdout:1/835: symlink d7/d1b/lfa 0 2024-09-16T00:09:55.023 INFO:tasks.workunit.client.0.smithi139.stdout:1/836: fdatasync d7/d8/d1f/f76 0 2024-09-16T00:09:55.028 INFO:tasks.workunit.client.0.smithi139.stdout:1/837: dwrite d7/d8/d40/d48/d4a/f98 [0,4194304] 0 2024-09-16T00:09:55.029 INFO:tasks.workunit.client.0.smithi139.stdout:1/838: rmdir d7/d8/d40/d48/d53/d61 39 2024-09-16T00:09:55.032 INFO:tasks.workunit.client.0.smithi139.stdout:1/839: creat d7/d8/d40/d48/d67/dbf/ffb x:0 0 0 2024-09-16T00:09:55.034 INFO:tasks.workunit.client.0.smithi139.stdout:1/840: mkdir d7/d8/d40/d48/d53/d61/dfc 0 2024-09-16T00:09:55.036 INFO:tasks.workunit.client.0.smithi139.stdout:1/841: mkdir d7/d8/d40/dd3/de3/dfd 0 2024-09-16T00:09:55.036 INFO:tasks.workunit.client.0.smithi139.stdout:1/842: stat d7/d8/d27/d68/d6c/fb1 0 2024-09-16T00:09:55.038 INFO:tasks.workunit.client.0.smithi139.stdout:1/843: mknod d7/d8/d40/d48/d4a/db8/dc6/cfe 0 2024-09-16T00:09:55.040 INFO:tasks.workunit.client.0.smithi139.stdout:1/844: creat d7/d8/d40/d48/d4a/d82/fff x:0 0 0 2024-09-16T00:09:55.045 INFO:tasks.workunit.client.0.smithi139.stdout:1/845: dwrite d7/d8/d40/d48/d4a/db8/fd0 [0,4194304] 0 2024-09-16T00:09:55.046 INFO:tasks.workunit.client.0.smithi139.stdout:1/846: write d7/d8/d1f/faf [647442,130132] 0 2024-09-16T00:09:55.053 INFO:tasks.workunit.client.0.smithi139.stdout:1/847: dwrite d7/d8/d1f/fc1 [0,4194304] 0 2024-09-16T00:09:55.053 INFO:tasks.workunit.client.0.smithi139.stdout:1/848: chown d7/d8/d27/d68 1183629 1 2024-09-16T00:09:55.056 INFO:tasks.workunit.client.0.smithi139.stdout:6/887: sync 2024-09-16T00:09:55.057 INFO:tasks.workunit.client.0.smithi139.stdout:1/849: creat d7/d8/d27/d68/d7c/ded/f100 x:0 0 0 2024-09-16T00:09:55.058 INFO:tasks.workunit.client.0.smithi139.stdout:1/850: chown d7/f26 17099 1 2024-09-16T00:09:55.062 INFO:tasks.workunit.client.0.smithi139.stdout:1/851: write d7/d8/d27/f57 [542596,28759] 0 2024-09-16T00:09:55.062 INFO:tasks.workunit.client.0.smithi139.stdout:6/888: dwrite d8/d29/f32 [4194304,4194304] 0 2024-09-16T00:09:55.068 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:55.067+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:55.070 INFO:tasks.workunit.client.0.smithi139.stdout:1/852: dwrite d7/d8/d1f/f20 [4194304,4194304] 0 2024-09-16T00:09:55.073 INFO:tasks.workunit.client.0.smithi139.stdout:1/853: creat d7/d8/d40/d48/d53/d61/dfc/f101 x:0 0 0 2024-09-16T00:09:55.073 INFO:tasks.workunit.client.0.smithi139.stdout:6/889: dread d8/d20/d4d/f60 [0,4194304] 0 2024-09-16T00:09:55.074 INFO:tasks.workunit.client.0.smithi139.stdout:1/854: write d7/d8/f1a [747637,119446] 0 2024-09-16T00:09:55.075 INFO:tasks.workunit.client.0.smithi139.stdout:6/890: dread - d8/d29/d83/d26/d45/d85/fef zero size 2024-09-16T00:09:55.076 INFO:tasks.workunit.client.0.smithi139.stdout:1/855: symlink d7/d8/d40/d46/d89/l102 0 2024-09-16T00:09:55.077 INFO:tasks.workunit.client.0.smithi139.stdout:6/891: mkdir d8/d29/d83/d26/d28/d5a/d135 0 2024-09-16T00:09:55.077 INFO:tasks.workunit.client.0.smithi139.stdout:1/856: write d7/d8/d40/d48/d67/dbf/ffb [711701,15698] 0 2024-09-16T00:09:55.081 INFO:tasks.workunit.client.0.smithi139.stdout:1/857: rename d7/d8/d27/d68/d7c/d83/fee to d7/d8/d40/d48/d4a/d74/ddb/f103 0 2024-09-16T00:09:55.084 INFO:tasks.workunit.client.0.smithi139.stdout:1/858: unlink d7/d8/d40/d47/ce4 0 2024-09-16T00:09:55.086 INFO:tasks.workunit.client.0.smithi139.stdout:1/859: unlink d7/d8/d27/l4d 0 2024-09-16T00:09:55.088 INFO:tasks.workunit.client.0.smithi139.stdout:6/892: dwrite d8/d29/f49 [8388608,4194304] 0 2024-09-16T00:09:55.088 INFO:tasks.workunit.client.0.smithi139.stdout:3/895: sync 2024-09-16T00:09:55.089 INFO:tasks.workunit.client.0.smithi139.stdout:1/860: rename d7/d8/c35 to d7/d8/d40/d46/d8d/c104 0 2024-09-16T00:09:55.091 INFO:tasks.workunit.client.0.smithi139.stdout:6/893: creat d8/d29/d83/d26/d45/d42/df3/f136 x:0 0 0 2024-09-16T00:09:55.095 INFO:tasks.workunit.client.0.smithi139.stdout:3/896: dwrite d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/f7f [0,4194304] 0 2024-09-16T00:09:55.096 INFO:tasks.workunit.client.0.smithi139.stdout:6/894: dread d8/d29/d83/d26/d45/d42/dc0/fd9 [0,4194304] 0 2024-09-16T00:09:55.099 INFO:tasks.workunit.client.0.smithi139.stdout:6/895: chown d8/f12 227 1 2024-09-16T00:09:55.103 INFO:tasks.workunit.client.0.smithi139.stdout:6/896: dread d8/d20/d4d/db0/fd6 [0,4194304] 0 2024-09-16T00:09:55.105 INFO:tasks.workunit.client.0.smithi139.stdout:1/861: link d7/d8/d40/d48/d4a/d74/ddb/f41 d7/d8/d40/d48/d4a/d74/ddb/f105 0 2024-09-16T00:09:55.106 INFO:tasks.workunit.client.0.smithi139.stdout:3/897: dread d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4d/f8c [0,4194304] 0 2024-09-16T00:09:55.107 INFO:tasks.workunit.client.0.smithi139.stdout:6/897: mkdir d8/d137 0 2024-09-16T00:09:55.108 INFO:tasks.workunit.client.0.smithi139.stdout:1/862: symlink d7/d8/d40/d48/dcc/dda/l106 0 2024-09-16T00:09:55.108 INFO:tasks.workunit.client.0.smithi139.stdout:3/898: stat d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d 0 2024-09-16T00:09:55.113 INFO:tasks.workunit.client.0.smithi139.stdout:1/863: truncate d7/d8/f1a 2027556 0 2024-09-16T00:09:55.115 INFO:tasks.workunit.client.0.smithi139.stdout:3/899: dwrite d5/d7/f28 [4194304,4194304] 0 2024-09-16T00:09:55.117 INFO:tasks.workunit.client.0.smithi139.stdout:3/900: mkdir d5/d132 0 2024-09-16T00:09:55.118 INFO:tasks.workunit.client.0.smithi139.stdout:6/898: dwrite d8/d20/f64 [0,4194304] 0 2024-09-16T00:09:55.123 INFO:tasks.workunit.client.0.smithi139.stdout:1/864: truncate d7/d8/d27/f79 4987010 0 2024-09-16T00:09:55.127 INFO:tasks.workunit.client.0.smithi139.stdout:6/899: dwrite d8/d29/f32 [0,4194304] 0 2024-09-16T00:09:55.129 INFO:tasks.workunit.client.0.smithi139.stdout:6/900: rmdir d8/d29/db9 39 2024-09-16T00:09:55.132 INFO:tasks.workunit.client.0.smithi139.stdout:1/865: write d7/d8/d27/f59 [5001957,123195] 0 2024-09-16T00:09:55.133 INFO:tasks.workunit.client.0.smithi139.stdout:3/901: dwrite d5/d7/d23/f15 [0,4194304] 0 2024-09-16T00:09:55.134 INFO:tasks.workunit.client.0.smithi139.stdout:6/901: write d8/d29/d83/d26/d45/d59/fb8 [146994,63333] 0 2024-09-16T00:09:55.136 INFO:tasks.workunit.client.0.smithi139.stdout:1/866: mkdir d7/d8/d40/d48/dcc/dda/d107 0 2024-09-16T00:09:55.140 INFO:tasks.workunit.client.0.smithi139.stdout:6/902: rename d8/d29/d83/d26/d28/d9e/da8/ffb to d8/d29/d83/d26/d45/d65/f138 0 2024-09-16T00:09:55.141 INFO:tasks.workunit.client.0.smithi139.stdout:6/903: fdatasync d8/d20/f64 0 2024-09-16T00:09:55.144 INFO:tasks.workunit.client.0.smithi139.stdout:1/867: dwrite d7/d8/d1f/faf [0,4194304] 0 2024-09-16T00:09:55.145 INFO:tasks.workunit.client.0.smithi139.stdout:6/904: symlink d8/d29/d83/d26/d28/d9e/da8/l139 0 2024-09-16T00:09:55.145 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:55.145 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:55.149 INFO:tasks.workunit.client.0.smithi139.stdout:1/868: mkdir d7/d8/d40/d48/d67/d108 0 2024-09-16T00:09:55.150 INFO:tasks.workunit.client.0.smithi139.stdout:1/869: stat d7/d8/d40/d48/d67/cb3 0 2024-09-16T00:09:55.150 INFO:tasks.workunit.client.0.smithi139.stdout:3/902: dread d5/d7/fd [4194304,4194304] 0 2024-09-16T00:09:55.151 INFO:tasks.workunit.client.0.smithi139.stdout:6/905: getdents d8/d20/d4d 0 2024-09-16T00:09:55.152 INFO:tasks.workunit.client.0.smithi139.stdout:6/906: write d8/d20/d27/f58 [1474650,68334] 0 2024-09-16T00:09:55.152 INFO:tasks.workunit.client.0.smithi139.stdout:3/903: chown d5/d7/d23/de4/de8/da7/d9a/d101/d2f/c55 133764220 1 2024-09-16T00:09:55.154 INFO:tasks.workunit.client.0.smithi139.stdout:6/907: chown d8/d29/db9/d10f/d112 6996658 1 2024-09-16T00:09:55.156 INFO:tasks.workunit.client.0.smithi139.stdout:3/904: write d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4d/dbc/fce [400315,64342] 0 2024-09-16T00:09:55.157 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:09:55.158 INFO:tasks.workunit.client.0.smithi139.stdout:6/908: mkdir d8/d29/db9/de8/d13a 0 2024-09-16T00:09:55.158 INFO:tasks.workunit.client.0.smithi139.stdout:3/905: dread - d5/d7/dc/d84/f128 zero size 2024-09-16T00:09:55.159 INFO:tasks.workunit.client.0.smithi139.stdout:1/870: dwrite d7/d8/d40/d48/d4a/d74/ff6 [0,4194304] 0 2024-09-16T00:09:55.163 INFO:tasks.workunit.client.0.smithi139.stdout:3/906: creat d5/d7/d23/de4/de8/dd4/f133 x:0 0 0 2024-09-16T00:09:55.163 INFO:tasks.workunit.client.0.smithi139.stdout:1/871: fdatasync d7/f58 0 2024-09-16T00:09:55.165 INFO:tasks.workunit.client.0.smithi139.stdout:1/872: dread - d7/d8/d40/d48/d4a/d74/ddb/fba zero size 2024-09-16T00:09:55.165 INFO:tasks.workunit.client.0.smithi139.stdout:6/909: rename d8/d29/d83/d26/d45/d85/fad to d8/da4/f13b 0 2024-09-16T00:09:55.169 INFO:tasks.workunit.client.0.smithi139.stdout:1/873: mknod d7/d8/d40/d48/d67/d108/c109 0 2024-09-16T00:09:55.171 INFO:tasks.workunit.client.0.smithi139.stdout:3/907: dwrite d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/dbb/fee [0,4194304] 0 2024-09-16T00:09:55.171 INFO:tasks.workunit.client.0.smithi139.stdout:1/874: rmdir d7/d8/d40/d48/d4a 39 2024-09-16T00:09:55.175 INFO:tasks.workunit.client.0.smithi139.stdout:6/910: dwrite d8/d29/d83/d26/d28/d74/fc8 [0,4194304] 0 2024-09-16T00:09:55.175 INFO:tasks.workunit.client.0.smithi139.stdout:1/875: truncate d7/f18 1354123 0 2024-09-16T00:09:55.176 INFO:tasks.workunit.client.0.smithi139.stdout:6/911: read d8/d29/d83/d26/f61 [1525989,8925] 0 2024-09-16T00:09:55.180 INFO:tasks.workunit.client.0.smithi139.stdout:1/876: dwrite d7/f13 [4194304,4194304] 0 2024-09-16T00:09:55.181 INFO:tasks.workunit.client.0.smithi139.stdout:1/877: chown d7/f16 4 1 2024-09-16T00:09:55.183 INFO:tasks.workunit.client.0.smithi139.stdout:3/908: write d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/f9e [426812,80908] 0 2024-09-16T00:09:55.191 INFO:tasks.workunit.client.0.smithi139.stdout:3/909: dwrite d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/f9e [0,4194304] 0 2024-09-16T00:09:55.192 INFO:tasks.workunit.client.0.smithi139.stdout:6/912: dread d8/d29/d83/d26/d45/d85/f118 [0,4194304] 0 2024-09-16T00:09:55.196 INFO:tasks.workunit.client.0.smithi139.stdout:3/910: symlink d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4d/dbc/l134 0 2024-09-16T00:09:55.196 INFO:tasks.workunit.client.0.smithi139.stdout:6/913: rmdir d8/d29/d83/d26/d45/d65 39 2024-09-16T00:09:55.198 INFO:tasks.workunit.client.0.smithi139.stdout:6/914: write d8/d29/d83/d26/d45/f43 [1302060,125334] 0 2024-09-16T00:09:55.201 INFO:tasks.workunit.client.0.smithi139.stdout:6/915: rename d8/d1f to d8/d29/d83/d26/d28/d5a/d135/d13c 0 2024-09-16T00:09:55.202 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:55.202+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:55.204 INFO:tasks.workunit.client.0.smithi139.stdout:3/911: link d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/db3/cd2 d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/db9/c135 0 2024-09-16T00:09:55.204 INFO:tasks.workunit.client.0.smithi139.stdout:6/916: creat d8/d29/d83/d26/d28/d6d/f13d x:0 0 0 2024-09-16T00:09:55.206 INFO:tasks.workunit.client.0.smithi139.stdout:3/912: chown d5/d7/d23/de4/de8/da7/d9a/d101/f83 103762847 1 2024-09-16T00:09:55.208 INFO:tasks.workunit.client.0.smithi139.stdout:6/917: creat d8/d29/d83/d26/d45/f13e x:0 0 0 2024-09-16T00:09:55.208 INFO:tasks.workunit.client.0.smithi139.stdout:3/913: fsync d5/d7/f49 0 2024-09-16T00:09:55.209 INFO:tasks.workunit.client.0.smithi139.stdout:3/914: chown d5/d7/dc/d9c/fa3 2014763 1 2024-09-16T00:09:55.209 INFO:tasks.workunit.client.0.smithi139.stdout:6/918: write d8/d29/d83/d26/d28/d6d/de7/f103 [295955,113056] 0 2024-09-16T00:09:55.210 INFO:tasks.workunit.client.0.smithi139.stdout:1/878: dread d7/d8/d27/d68/d6c/fb1 [0,4194304] 0 2024-09-16T00:09:55.211 INFO:tasks.workunit.client.0.smithi139.stdout:3/915: fdatasync d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/fcc 0 2024-09-16T00:09:55.212 INFO:tasks.workunit.client.0.smithi139.stdout:6/919: symlink d8/d29/d83/d26/d28/d6d/dd3/l13f 0 2024-09-16T00:09:55.212 INFO:tasks.workunit.client.0.smithi139.stdout:1/879: chown d7/d8/d27/d68/lc9 5 1 2024-09-16T00:09:55.213 INFO:tasks.workunit.client.0.smithi139.stdout:3/916: write d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/db9/f114 [443058,38092] 0 2024-09-16T00:09:55.214 INFO:tasks.workunit.client.0.smithi139.stdout:1/880: write d7/d8/d27/d68/d7c/fc2 [594562,120747] 0 2024-09-16T00:09:55.215 INFO:tasks.workunit.client.0.smithi139.stdout:1/881: write d7/d8/d1f/f20 [2720717,75898] 0 2024-09-16T00:09:55.217 INFO:tasks.workunit.client.0.smithi139.stdout:3/917: symlink d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d12b/l136 0 2024-09-16T00:09:55.218 INFO:tasks.workunit.client.0.smithi139.stdout:1/882: chown d7/d8/c19 16304 1 2024-09-16T00:09:55.220 INFO:tasks.workunit.client.0.smithi139.stdout:3/918: truncate d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/dbb/de2/ffd 465955 0 2024-09-16T00:09:55.221 INFO:tasks.workunit.client.0.smithi139.stdout:1/883: mkdir d7/d8/d40/d48/d4a/d74/d10a 0 2024-09-16T00:09:55.223 INFO:tasks.workunit.client.0.smithi139.stdout:3/919: mkdir d5/d7/dc2/d137 0 2024-09-16T00:09:55.224 INFO:tasks.workunit.client.0.smithi139.stdout:3/920: readlink d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d47/l90 0 2024-09-16T00:09:55.229 INFO:tasks.workunit.client.0.smithi139.stdout:1/884: creat d7/d8/d40/d48/d4a/d82/f10b x:0 0 0 2024-09-16T00:09:55.229 INFO:tasks.workunit.client.0.smithi139.stdout:3/921: mknod d5/d7/d23/de4/de8/da7/c138 0 2024-09-16T00:09:55.229 INFO:tasks.workunit.client.0.smithi139.stdout:1/885: rmdir d7/d8/d27/d68 39 2024-09-16T00:09:55.230 INFO:tasks.workunit.client.0.smithi139.stdout:1/886: write d7/d8/d40/d48/d4a/d82/fff [190742,19379] 0 2024-09-16T00:09:55.231 INFO:tasks.workunit.client.0.smithi139.stdout:3/922: dread d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4d/dbc/fc5 [0,4194304] 0 2024-09-16T00:09:55.232 INFO:tasks.workunit.client.0.smithi139.stdout:6/920: truncate d8/d20/d4d/f60 811442 0 2024-09-16T00:09:55.234 INFO:tasks.workunit.client.0.smithi139.stdout:3/923: fdatasync d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4d/f125 0 2024-09-16T00:09:55.239 INFO:tasks.workunit.client.0.smithi139.stdout:3/924: link d5/d7/d23/de4/de8/f3b d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d112/f139 0 2024-09-16T00:09:55.269 INFO:tasks.workunit.client.0.smithi139.stdout:6/921: dread d8/d29/d83/f15 [0,4194304] 0 2024-09-16T00:09:55.269 INFO:tasks.workunit.client.0.smithi139.stdout:6/922: chown d8/d29/d83/d26/d45/d85 539524 1 2024-09-16T00:09:55.269 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:55.268+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:55.270 INFO:tasks.workunit.client.0.smithi139.stdout:6/923: chown d8/d29/d83/d26/d28/d6d/dd3/de5 98217 1 2024-09-16T00:09:55.270 INFO:tasks.workunit.client.0.smithi139.stdout:6/924: stat d8/d29/db9/d10f 0 2024-09-16T00:09:55.271 INFO:tasks.workunit.client.0.smithi139.stdout:3/925: creat d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4d/dc6/f13a x:0 0 0 2024-09-16T00:09:55.279 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:55.280 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:55.289 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:09:55.290 INFO:tasks.workunit.client.0.smithi139.stdout:3/926: write d5/d7/d23/de4/de8/da7/d9a/d101/f117 [4026994,1447] 0 2024-09-16T00:09:55.290 INFO:tasks.workunit.client.0.smithi139.stdout:6/925: mkdir d8/d29/d83/d26/d45/d65/d6e/d140 0 2024-09-16T00:09:55.293 INFO:tasks.workunit.client.0.smithi139.stdout:3/927: unlink d5/d7/dc2/dec/l118 0 2024-09-16T00:09:55.294 INFO:tasks.workunit.client.0.smithi139.stdout:6/926: truncate d8/d29/d83/d26/d45/d65/d8b/f120 6559428 0 2024-09-16T00:09:55.296 INFO:tasks.workunit.client.0.smithi139.stdout:6/927: chown d8/d29/d83/d26/d28/d9e/da8/cc4 512099 1 2024-09-16T00:09:55.297 INFO:tasks.workunit.client.0.smithi139.stdout:6/928: readlink d8/d29/d83/d26/d45/lca 0 2024-09-16T00:09:55.297 INFO:tasks.workunit.client.0.smithi139.stdout:6/929: fdatasync d8/d29/d83/d26/d28/d5a/f102 0 2024-09-16T00:09:55.298 INFO:tasks.workunit.client.0.smithi139.stdout:6/930: readlink d8/d29/d83/d26/d28/d5a/le1 0 2024-09-16T00:09:55.299 INFO:tasks.workunit.client.0.smithi139.stdout:6/931: readlink d8/d29/d83/d26/d45/l5f 0 2024-09-16T00:09:55.299 INFO:tasks.workunit.client.0.smithi139.stdout:3/928: link d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/d76/lf4 d5/d7/d23/de4/d75/d11d/l13b 0 2024-09-16T00:09:55.299 INFO:tasks.workunit.client.0.smithi139.stdout:6/932: chown d8/d29/d83/d26/d45/d85 45 1 2024-09-16T00:09:55.300 INFO:tasks.workunit.client.0.smithi139.stdout:3/929: write d5/d7/d23/de4/de8/fde [229761,113750] 0 2024-09-16T00:09:55.301 INFO:tasks.workunit.client.0.smithi139.stdout:6/933: mknod d8/d20/d68/df4/c141 0 2024-09-16T00:09:55.303 INFO:tasks.workunit.client.0.smithi139.stdout:6/934: symlink d8/da4/d105/l142 0 2024-09-16T00:09:55.304 INFO:tasks.workunit.client.0.smithi139.stdout:6/935: chown d8/d29/d83/d26/d45/d42/dc0 441732 1 2024-09-16T00:09:55.306 INFO:tasks.workunit.client.0.smithi139.stdout:3/930: dwrite d5/d7/dc/d108/f129 [0,4194304] 0 2024-09-16T00:09:55.306 INFO:tasks.workunit.client.0.smithi139.stdout:6/936: dread - d8/d29/d83/d26/d28/d6d/dd3/de5/ff8 zero size 2024-09-16T00:09:55.309 INFO:tasks.workunit.client.0.smithi139.stdout:3/931: stat d5/d7/dc/d9c/cfc 0 2024-09-16T00:09:55.311 INFO:tasks.workunit.client.0.smithi139.stdout:3/932: read d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/fe6 [1716331,62012] 0 2024-09-16T00:09:55.311 INFO:tasks.workunit.client.0.smithi139.stdout:6/937: unlink d8/d29/d83/f81 0 2024-09-16T00:09:55.315 INFO:tasks.workunit.client.0.smithi139.stdout:6/938: unlink d8/d20/d27/fcf 0 2024-09-16T00:09:55.316 INFO:tasks.workunit.client.0.smithi139.stdout:6/939: dread - d8/d29/f8f zero size 2024-09-16T00:09:55.316 INFO:tasks.workunit.client.0.smithi139.stdout:6/940: stat d8/d29/d83/d26/d28/d5a 0 2024-09-16T00:09:55.318 INFO:tasks.workunit.client.0.smithi139.stdout:3/933: symlink d5/d7/d23/de4/d75/d11d/l13c 0 2024-09-16T00:09:55.319 INFO:tasks.workunit.client.0.smithi139.stdout:6/941: write d8/fe [2188757,37452] 0 2024-09-16T00:09:55.321 INFO:tasks.workunit.client.0.smithi139.stdout:6/942: mknod d8/d29/db9/c143 0 2024-09-16T00:09:55.322 INFO:tasks.workunit.client.0.smithi139.stdout:3/934: creat d5/d7/d23/de4/de8/da7/d9a/d101/d2f/f13d x:0 0 0 2024-09-16T00:09:55.322 INFO:tasks.workunit.client.0.smithi139.stdout:6/943: fsync d8/d29/d83/d26/d45/d65/f138 0 2024-09-16T00:09:55.323 INFO:tasks.workunit.client.0.smithi139.stdout:3/935: chown d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/fb5 427765 1 2024-09-16T00:09:55.324 INFO:tasks.workunit.client.0.smithi139.stdout:6/944: unlink d8/d29/d83/d26/d28/l119 0 2024-09-16T00:09:55.326 INFO:tasks.workunit.client.0.smithi139.stdout:6/945: creat d8/d29/db9/f144 x:0 0 0 2024-09-16T00:09:55.329 INFO:tasks.workunit.client.0.smithi139.stdout:6/946: mkdir d8/d29/d83/d26/d28/d5a/d130/d145 0 2024-09-16T00:09:55.329 INFO:tasks.workunit.client.0.smithi139.stdout:6/947: fsync d8/d29/d83/d26/d45/f8a 0 2024-09-16T00:09:55.332 INFO:tasks.workunit.client.0.smithi139.stdout:3/936: write d5/f53 [458315,74181] 0 2024-09-16T00:09:55.333 INFO:tasks.workunit.client.0.smithi139.stdout:6/948: rename d8/d20/d27/c4e to d8/d20/c146 0 2024-09-16T00:09:55.334 INFO:tasks.workunit.client.0.smithi139.stdout:6/949: fsync d8/d29/d83/d26/d28/f109 0 2024-09-16T00:09:55.336 INFO:tasks.workunit.client.0.smithi139.stdout:6/950: read d8/d20/d27/f18 [1915692,98406] 0 2024-09-16T00:09:55.338 INFO:tasks.workunit.client.0.smithi139.stdout:6/951: creat d8/d29/d83/d26/d45/f147 x:0 0 0 2024-09-16T00:09:55.339 INFO:tasks.workunit.client.0.smithi139.stdout:3/937: truncate d5/d7/d23/de4/de8/f61 387381 0 2024-09-16T00:09:55.344 INFO:tasks.workunit.client.0.smithi139.stdout:6/952: write d8/d29/d83/d26/d45/d85/f118 [5431720,127312] 0 2024-09-16T00:09:55.345 INFO:tasks.workunit.client.0.smithi139.stdout:6/953: dread - d8/d29/d83/d26/d28/d6d/f13d zero size 2024-09-16T00:09:55.350 INFO:tasks.workunit.client.0.smithi139.stdout:6/954: dwrite d8/d29/d83/d26/d45/f46 [0,4194304] 0 2024-09-16T00:09:55.356 INFO:tasks.workunit.client.0.smithi139.stdout:6/955: truncate d8/d29/d83/d26/d28/d9e/da8/fcc 4064597 0 2024-09-16T00:09:55.360 INFO:tasks.workunit.client.0.smithi139.stdout:6/956: creat d8/d29/d83/d26/f148 x:0 0 0 2024-09-16T00:09:55.361 INFO:tasks.workunit.client.0.smithi139.stdout:6/957: write d8/d20/d27/f18 [3640606,86983] 0 2024-09-16T00:09:55.364 INFO:tasks.workunit.client.0.smithi139.stdout:6/958: unlink d8/d29/d83/d26/d28/d5a/d135/d13c/l5b 0 2024-09-16T00:09:55.365 INFO:tasks.workunit.client.0.smithi139.stdout:6/959: write d8/d29/d83/d26/f126 [585965,91781] 0 2024-09-16T00:09:55.366 INFO:tasks.workunit.client.0.smithi139.stdout:6/960: write d8/d29/d83/d26/d28/d6d/dd3/ffc [753516,108183] 0 2024-09-16T00:09:55.367 INFO:tasks.workunit.client.0.smithi139.stdout:6/961: write d8/d20/dd1/ddc/fdf [225711,75385] 0 2024-09-16T00:09:55.370 INFO:tasks.workunit.client.0.smithi139.stdout:6/962: rename d8/d20/d27/cbb to d8/d29/d83/d26/d45/d85/c149 0 2024-09-16T00:09:55.372 INFO:tasks.workunit.client.0.smithi139.stdout:6/963: mkdir d8/d20/dd1/ddc/d14a 0 2024-09-16T00:09:55.372 INFO:tasks.workunit.client.0.smithi139.stdout:6/964: fsync d8/d20/d4d/db0/fe9 0 2024-09-16T00:09:55.374 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:55.374+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:55.382 INFO:tasks.workunit.client.0.smithi139.stdout:6/965: dwrite d8/d29/d83/f2c [0,4194304] 0 2024-09-16T00:09:55.384 INFO:tasks.workunit.client.0.smithi139.stdout:6/966: creat d8/d29/d83/d26/d45/d42/df3/f14b x:0 0 0 2024-09-16T00:09:55.386 INFO:tasks.workunit.client.0.smithi139.stdout:6/967: write d8/d29/d83/d26/d28/d6d/f71 [1562755,13002] 0 2024-09-16T00:09:55.388 INFO:tasks.workunit.client.0.smithi139.stdout:6/968: write d8/d29/d83/d26/d28/d6d/f71 [75416,93813] 0 2024-09-16T00:09:55.391 INFO:tasks.workunit.client.0.smithi139.stdout:6/969: creat d8/d29/db9/de8/d13a/f14c x:0 0 0 2024-09-16T00:09:55.391 INFO:tasks.workunit.client.0.smithi139.stdout:6/970: mknod d8/da4/c14d 0 2024-09-16T00:09:55.393 INFO:tasks.workunit.client.0.smithi139.stdout:6/971: creat d8/d20/d27/d117/f14e x:0 0 0 2024-09-16T00:09:55.402 INFO:tasks.workunit.client.0.smithi139.stdout:6/972: dread d8/d29/d83/d26/d45/d65/f2e [0,4194304] 0 2024-09-16T00:09:55.405 INFO:tasks.workunit.client.0.smithi139.stdout:6/973: mknod d8/d56/d10e/c14f 0 2024-09-16T00:09:55.410 INFO:tasks.workunit.client.0.smithi139.stdout:6/974: dread d8/d56/f79 [0,4194304] 0 2024-09-16T00:09:55.413 INFO:tasks.workunit.client.0.smithi139.stdout:6/975: rename d8/d29/d83/d26/d28/d6d/c80 to d8/d29/d83/d26/d28/d5a/d135/c150 0 2024-09-16T00:09:55.414 INFO:tasks.workunit.client.0.smithi139.stdout:6/976: mknod d8/d29/d83/d26/d28/d9e/c151 0 2024-09-16T00:09:55.420 INFO:tasks.workunit.client.0.smithi139.stdout:6/977: rename d8/d29/d83/d26/d28/d9e/da8/fcc to d8/d20/d27/d117/f152 0 2024-09-16T00:09:55.420 INFO:tasks.workunit.client.0.smithi139.stdout:6/978: readlink d8/d29/d83/d26/l7f 0 2024-09-16T00:09:55.423 INFO:tasks.workunit.client.0.smithi139.stdout:6/979: getdents d8/d29/d83/d26/d45/d65/d6e/d140 0 2024-09-16T00:09:55.427 INFO:tasks.workunit.client.0.smithi139.stdout:6/980: dwrite d8/d29/d83/d26/f91 [0,4194304] 0 2024-09-16T00:09:55.433 INFO:tasks.workunit.client.0.smithi139.stdout:6/981: write d8/d29/d83/d26/d28/d5a/fa7 [2716715,20317] 0 2024-09-16T00:09:55.433 INFO:tasks.workunit.client.0.smithi139.stdout:6/982: dread - d8/d29/db9/de8/d13a/f14c zero size 2024-09-16T00:09:55.436 INFO:tasks.workunit.client.0.smithi139.stdout:6/983: creat d8/d29/d83/d26/d28/d9e/f153 x:0 0 0 2024-09-16T00:09:55.438 INFO:tasks.workunit.client.0.smithi139.stdout:6/984: creat d8/d29/d83/d26/d45/d42/d11c/f154 x:0 0 0 2024-09-16T00:09:55.445 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:55.445 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:55.454 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:09:55.454 INFO:tasks.workunit.client.0.smithi139.stdout:1/887: sync 2024-09-16T00:09:55.457 INFO:tasks.workunit.client.0.smithi139.stdout:1/888: creat d7/d8/d40/d48/d4a/d82/de2/f10c x:0 0 0 2024-09-16T00:09:55.459 INFO:tasks.workunit.client.0.smithi139.stdout:1/889: rename d7/d8/d40/d48/d4a/d74/ddb to d7/d8/d40/d48/d67/d108/d10d 0 2024-09-16T00:09:55.461 INFO:tasks.workunit.client.0.smithi139.stdout:1/890: fdatasync d7/d8/d27/d68/d7c/d83/fca 0 2024-09-16T00:09:55.464 INFO:tasks.workunit.client.0.smithi139.stdout:1/891: dread d7/d8/d1f/faf [0,4194304] 0 2024-09-16T00:09:55.466 INFO:tasks.workunit.client.0.smithi139.stdout:1/892: fsync d7/f10 0 2024-09-16T00:09:55.467 INFO:tasks.workunit.client.0.smithi139.stdout:1/893: chown d7/d8/d27/d68/l9d 11609 1 2024-09-16T00:09:55.470 INFO:tasks.workunit.client.0.smithi139.stdout:1/894: unlink d7/d8/d40/d46/l5f 0 2024-09-16T00:09:55.471 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:55.470+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:55.471 INFO:tasks.workunit.client.0.smithi139.stdout:1/895: write d7/d8/d27/d68/fa9 [423255,57330] 0 2024-09-16T00:09:55.472 INFO:tasks.workunit.client.0.smithi139.stdout:1/896: truncate d7/d8/d40/d48/d53/d61/dfc/f101 267535 0 2024-09-16T00:09:55.473 INFO:tasks.workunit.client.0.smithi139.stdout:1/897: readlink d7/d8/d40/dd3/de3/lf2 0 2024-09-16T00:09:55.478 INFO:tasks.workunit.client.0.smithi139.stdout:1/898: dwrite d7/d8/d40/d46/d8d/f95 [0,4194304] 0 2024-09-16T00:09:55.481 INFO:tasks.workunit.client.0.smithi139.stdout:1/899: dread d7/d8/d40/d48/d53/fbc [0,4194304] 0 2024-09-16T00:09:55.482 INFO:tasks.workunit.client.0.smithi139.stdout:1/900: rmdir d7/d8/d40/d48/d67/d108/d10d 39 2024-09-16T00:09:55.484 INFO:tasks.workunit.client.0.smithi139.stdout:1/901: write d7/d8/d40/d48/d53/d61/dfc/f101 [863516,22454] 0 2024-09-16T00:09:55.485 INFO:tasks.workunit.client.0.smithi139.stdout:1/902: read d7/d8/d40/d48/d4a/f98 [3247745,98121] 0 2024-09-16T00:09:55.487 INFO:tasks.workunit.client.0.smithi139.stdout:1/903: dread d7/d8/fa0 [0,4194304] 0 2024-09-16T00:09:55.489 INFO:tasks.workunit.client.0.smithi139.stdout:1/904: chown d7/d8/d40/d48/d67/d108/d10d/f5c 14 1 2024-09-16T00:09:55.494 INFO:tasks.workunit.client.0.smithi139.stdout:1/905: write d7/d8/d40/d48/d67/d108/d10d/f5c [578177,30182] 0 2024-09-16T00:09:55.497 INFO:tasks.workunit.client.0.smithi139.stdout:1/906: rename d7/d8/d40/d48/d4a/d74/f7e to d7/d8/f10e 0 2024-09-16T00:09:55.498 INFO:tasks.workunit.client.0.smithi139.stdout:1/907: readlink d7/d8/l9a 0 2024-09-16T00:09:55.500 INFO:tasks.workunit.client.0.smithi139.stdout:1/908: symlink d7/d8/d40/d48/d67/d7d/dc3/l10f 0 2024-09-16T00:09:55.502 INFO:tasks.workunit.client.0.smithi139.stdout:1/909: mkdir d7/d8/d40/d46/d91/d110 0 2024-09-16T00:09:55.503 INFO:tasks.workunit.client.0.smithi139.stdout:1/910: write d7/d8/d1f/f20 [7100591,108892] 0 2024-09-16T00:09:55.509 INFO:tasks.workunit.client.0.smithi139.stdout:1/911: write d7/d8/d40/feb [202223,14130] 0 2024-09-16T00:09:55.510 INFO:tasks.workunit.client.0.smithi139.stdout:1/912: fdatasync d7/d8/d27/d68/d6c/fbe 0 2024-09-16T00:09:55.522 INFO:tasks.workunit.client.0.smithi139.stdout:1/913: dread d7/d8/d40/d48/d67/dbf/fd5 [0,4194304] 0 2024-09-16T00:09:55.524 INFO:tasks.workunit.client.0.smithi139.stdout:1/914: getdents d7/d8/d40/d48/dcc/dda/d107 0 2024-09-16T00:09:55.532 INFO:tasks.workunit.client.0.smithi139.stdout:1/915: dwrite d7/d8/d27/fb0 [0,4194304] 0 2024-09-16T00:09:55.537 INFO:tasks.workunit.client.0.smithi139.stdout:1/916: dread d7/d8/f21 [4194304,4194304] 0 2024-09-16T00:09:55.538 INFO:tasks.workunit.client.0.smithi139.stdout:1/917: mknod d7/d1b/c111 0 2024-09-16T00:09:55.540 INFO:tasks.workunit.client.0.smithi139.stdout:1/918: creat d7/d8/d40/d48/d67/d7d/f112 x:0 0 0 2024-09-16T00:09:55.541 INFO:tasks.workunit.client.0.smithi139.stdout:1/919: stat d7/d8/d1f/lec 0 2024-09-16T00:09:55.546 INFO:tasks.workunit.client.0.smithi139.stdout:1/920: dread d7/d8/d27/f57 [0,4194304] 0 2024-09-16T00:09:55.547 INFO:tasks.workunit.client.0.smithi139.stdout:1/921: write d7/d8/d40/d48/d4a/db8/fd0 [3505359,59302] 0 2024-09-16T00:09:55.551 INFO:tasks.workunit.client.0.smithi139.stdout:1/922: rename d7/d8/d27/d68/d7c/ded/f100 to d7/d8/d40/d48/f113 0 2024-09-16T00:09:55.553 INFO:tasks.workunit.client.0.smithi139.stdout:1/923: creat d7/d8/d40/d48/dcc/de7/f114 x:0 0 0 2024-09-16T00:09:55.553 INFO:tasks.workunit.client.0.smithi139.stdout:3/938: sync 2024-09-16T00:09:55.555 INFO:tasks.workunit.client.0.smithi139.stdout:1/924: creat d7/d8/f115 x:0 0 0 2024-09-16T00:09:55.558 INFO:tasks.workunit.client.0.smithi139.stdout:1/925: dread d7/d8/d40/d48/d4a/db8/fd0 [0,4194304] 0 2024-09-16T00:09:55.559 INFO:tasks.workunit.client.0.smithi139.stdout:1/926: write d7/d8/d27/fb0 [2053182,43168] 0 2024-09-16T00:09:55.562 INFO:tasks.workunit.client.0.smithi139.stdout:3/939: dwrite d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d47/f4a [0,4194304] 0 2024-09-16T00:09:55.566 INFO:tasks.workunit.client.0.smithi139.stdout:1/927: dwrite d7/d8/d40/d48/d4a/db8/fd0 [0,4194304] 0 2024-09-16T00:09:55.571 INFO:tasks.workunit.client.0.smithi139.stdout:1/928: getdents d7/d8/d40/dd3 0 2024-09-16T00:09:55.571 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:55.570+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:55.572 INFO:tasks.workunit.client.0.smithi139.stdout:1/929: truncate d7/d8/d40/d47/dae/fc5 812366 0 2024-09-16T00:09:55.575 INFO:tasks.workunit.client.0.smithi139.stdout:1/930: creat d7/d8/d40/d48/d4a/d74/f116 x:0 0 0 2024-09-16T00:09:55.576 INFO:tasks.workunit.client.0.smithi139.stdout:1/931: read d7/d8/f10e [12401,112713] 0 2024-09-16T00:09:55.578 INFO:tasks.workunit.client.0.smithi139.stdout:1/932: rmdir d7/d8/d40/d46/d91 39 2024-09-16T00:09:55.578 INFO:tasks.workunit.client.0.smithi139.stdout:1/933: stat d7/f25 0 2024-09-16T00:09:55.579 INFO:tasks.workunit.client.0.smithi139.stdout:1/934: read d7/d8/d1f/f37 [100465,124795] 0 2024-09-16T00:09:55.580 INFO:tasks.workunit.client.0.smithi139.stdout:1/935: dread - d7/d1b/fc4 zero size 2024-09-16T00:09:55.582 INFO:tasks.workunit.client.0.smithi139.stdout:1/936: mkdir d7/d8/d40/d48/d67/d108/d117 0 2024-09-16T00:09:55.582 INFO:tasks.workunit.client.0.smithi139.stdout:1/937: write d7/f12 [1094008,68757] 0 2024-09-16T00:09:55.583 INFO:tasks.workunit.client.0.smithi139.stdout:1/938: dread - d7/d8/d40/d48/d67/d108/d10d/f103 zero size 2024-09-16T00:09:55.584 INFO:tasks.workunit.client.0.smithi139.stdout:1/939: write d7/d8/d40/d47/dae/fde [115303,103373] 0 2024-09-16T00:09:55.586 INFO:tasks.workunit.client.0.smithi139.stdout:1/940: mknod d7/d8/d40/dd3/de3/c118 0 2024-09-16T00:09:55.587 INFO:tasks.workunit.client.0.smithi139.stdout:1/941: mknod d7/d8/d40/d48/d4a/d82/de2/c119 0 2024-09-16T00:09:55.590 INFO:tasks.workunit.client.0.smithi139.stdout:1/942: rmdir d7/d8/d40/dd3/de3/dfd 0 2024-09-16T00:09:55.591 INFO:tasks.workunit.client.0.smithi139.stdout:1/943: read d7/d8/d1f/fa1 [1057090,2881] 0 2024-09-16T00:09:55.593 INFO:tasks.workunit.client.0.smithi139.stdout:1/944: mkdir d7/d8/d40/d48/d53/d61/dc8/d11a 0 2024-09-16T00:09:55.598 INFO:tasks.workunit.client.0.smithi139.stdout:1/945: fdatasync d7/f18 0 2024-09-16T00:09:55.610 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:55.610 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:55.619 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:09:55.620 INFO:tasks.workunit.client.0.smithi139.stdout:1/946: rename d7/d8/d40/d47/f50 to d7/d8/d40/d48/d67/d108/d117/f11b 0 2024-09-16T00:09:55.622 INFO:tasks.workunit.client.0.smithi139.stdout:1/947: rename d7/d8/f1e to d7/d1b/f11c 0 2024-09-16T00:09:55.624 INFO:tasks.workunit.client.0.smithi139.stdout:1/948: symlink d7/d8/d1f/l11d 0 2024-09-16T00:09:55.625 INFO:tasks.workunit.client.0.smithi139.stdout:1/949: rmdir d7/d8/d40/d48/d4a/db8/dc6 39 2024-09-16T00:09:55.626 INFO:tasks.workunit.client.0.smithi139.stdout:1/950: chown d7/d8/d40/d48/lf4 351873405 1 2024-09-16T00:09:55.627 INFO:tasks.workunit.client.0.smithi139.stdout:1/951: mknod d7/d8/d40/dd3/de3/c11e 0 2024-09-16T00:09:55.629 INFO:tasks.workunit.client.0.smithi139.stdout:1/952: symlink d7/d8/d1f/l11f 0 2024-09-16T00:09:55.630 INFO:tasks.workunit.client.0.smithi139.stdout:1/953: dread d7/d8/d40/feb [0,4194304] 0 2024-09-16T00:09:55.631 INFO:tasks.workunit.client.0.smithi139.stdout:1/954: chown d7/d8/d40/d48/d4a/l52 874222680 1 2024-09-16T00:09:55.637 INFO:tasks.workunit.client.0.smithi139.stdout:1/955: rename d7/d8/f1a to d7/d8/d40/d48/dcc/dda/f120 0 2024-09-16T00:09:55.637 INFO:tasks.workunit.client.0.smithi139.stdout:1/956: fsync d7/d1b/fc4 0 2024-09-16T00:09:55.637 INFO:tasks.workunit.client.0.smithi139.stdout:6/985: sync 2024-09-16T00:09:55.637 INFO:tasks.workunit.client.0.smithi139.stdout:1/957: mknod d7/d8/d27/d68/d7c/ded/c121 0 2024-09-16T00:09:55.638 INFO:tasks.workunit.client.0.smithi139.stdout:1/958: read d7/f18 [510506,129521] 0 2024-09-16T00:09:55.638 INFO:tasks.workunit.client.0.smithi139.stdout:1/959: fdatasync d7/f25 0 2024-09-16T00:09:55.639 INFO:tasks.workunit.client.0.smithi139.stdout:1/960: dread d7/d8/d40/feb [0,4194304] 0 2024-09-16T00:09:55.655 INFO:tasks.workunit.client.0.smithi139.stdout:1/961: truncate d7/d8/d40/d48/dcc/dda/f120 541593 0 2024-09-16T00:09:55.656 INFO:tasks.workunit.client.0.smithi139.stdout:1/962: write d7/d8/d27/d68/d6c/f7a [1029822,79823] 0 2024-09-16T00:09:55.657 INFO:tasks.workunit.client.0.smithi139.stdout:1/963: write d7/d8/d40/d48/dcc/de7/f114 [3599,59852] 0 2024-09-16T00:09:55.664 INFO:tasks.workunit.client.0.smithi139.stdout:1/964: dwrite d7/d8/d40/d48/d67/d108/d10d/f103 [0,4194304] 0 2024-09-16T00:09:55.672 INFO:tasks.workunit.client.0.smithi139.stdout:1/965: truncate d7/d8/d40/d48/d67/dbf/fd5 3657281 0 2024-09-16T00:09:55.672 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:55.671+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:55.673 INFO:tasks.workunit.client.0.smithi139.stdout:1/966: creat d7/d8/d40/d47/dae/f122 x:0 0 0 2024-09-16T00:09:55.678 INFO:tasks.workunit.client.0.smithi139.stdout:1/967: dwrite d7/d8/d40/d48/dcc/de7/f114 [0,4194304] 0 2024-09-16T00:09:55.682 INFO:tasks.workunit.client.0.smithi139.stdout:1/968: link d7/d8/d1f/l11f d7/d8/d40/d48/d53/d61/de9/l123 0 2024-09-16T00:09:55.687 INFO:tasks.workunit.client.0.smithi139.stdout:1/969: dwrite d7/d8/f115 [0,4194304] 0 2024-09-16T00:09:55.690 INFO:tasks.workunit.client.0.smithi139.stdout:1/970: mknod d7/d8/d27/d68/d6c/c124 0 2024-09-16T00:09:55.698 INFO:tasks.workunit.client.0.smithi139.stdout:1/971: dwrite d7/d8/d40/d48/d4a/f8c [0,4194304] 0 2024-09-16T00:09:55.699 INFO:tasks.workunit.client.0.smithi139.stdout:1/972: write d7/d8/d40/d48/d4a/db8/fd0 [4618810,100055] 0 2024-09-16T00:09:55.700 INFO:tasks.workunit.client.0.smithi139.stdout:1/973: chown d7/d8/fa0 2115000 1 2024-09-16T00:09:55.702 INFO:tasks.workunit.client.0.smithi139.stdout:1/974: creat d7/d1b/f125 x:0 0 0 2024-09-16T00:09:55.708 INFO:tasks.workunit.client.0.smithi139.stdout:1/975: dwrite d7/d8/d27/f57 [0,4194304] 0 2024-09-16T00:09:55.711 INFO:tasks.workunit.client.0.smithi139.stdout:1/976: creat d7/d8/d27/d68/d7c/ded/f126 x:0 0 0 2024-09-16T00:09:55.713 INFO:tasks.workunit.client.0.smithi139.stdout:1/977: read d7/d8/d40/d47/f5e [673286,107371] 0 2024-09-16T00:09:55.715 INFO:tasks.workunit.client.0.smithi139.stdout:1/978: creat d7/d8/d40/d48/d53/f127 x:0 0 0 2024-09-16T00:09:55.716 INFO:tasks.workunit.client.0.smithi139.stdout:1/979: write d7/d8/d1f/faf [4363542,41189] 0 2024-09-16T00:09:55.718 INFO:tasks.workunit.client.0.smithi139.stdout:1/980: mkdir d7/d8/d40/d48/dcc/dda/d128 0 2024-09-16T00:09:55.720 INFO:tasks.workunit.client.0.smithi139.stdout:1/981: mkdir d7/d1b/d129 0 2024-09-16T00:09:55.722 INFO:tasks.workunit.client.0.smithi139.stdout:1/982: rename d7/d8/d40/d48/dcc/de7 to d7/d8/d40/dd3/d12a 0 2024-09-16T00:09:55.725 INFO:tasks.workunit.client.0.smithi139.stdout:1/983: rename d7/d8/c19 to d7/d8/d40/dd3/de3/c12b 0 2024-09-16T00:09:55.727 INFO:tasks.workunit.client.0.smithi139.stdout:1/984: mkdir d7/d8/d27/d68/d7c/ded/d12c 0 2024-09-16T00:09:55.731 INFO:tasks.workunit.client.0.smithi139.stdout:1/985: dread d7/d8/d40/d48/d53/d61/f87 [0,4194304] 0 2024-09-16T00:09:55.731 INFO:tasks.workunit.client.0.smithi139.stdout:3/940: sync 2024-09-16T00:09:55.732 INFO:tasks.workunit.client.0.smithi139.stdout:1/986: fsync d7/d1b/f125 0 2024-09-16T00:09:55.732 INFO:tasks.workunit.client.0.smithi139.stdout:3/941: write d5/d7/f49 [1502234,4372] 0 2024-09-16T00:09:55.733 INFO:tasks.workunit.client.0.smithi139.stdout:1/987: readlink d7/d8/d40/d48/dcc/dda/l106 0 2024-09-16T00:09:55.735 INFO:tasks.workunit.client.0.smithi139.stdout:1/988: write d7/d8/d40/d48/d4a/f8c [2573631,17205] 0 2024-09-16T00:09:55.736 INFO:tasks.workunit.client.0.smithi139.stdout:3/942: symlink d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/l13e 0 2024-09-16T00:09:55.737 INFO:tasks.workunit.client.0.smithi139.stdout:1/989: creat d7/d8/d27/d68/d7c/ded/d12c/f12d x:0 0 0 2024-09-16T00:09:55.740 INFO:tasks.workunit.client.0.smithi139.stdout:3/943: creat d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/f13f x:0 0 0 2024-09-16T00:09:55.740 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:55.740 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:55.747 INFO:tasks.workunit.client.0.smithi139.stdout:1/990: dwrite d7/d1b/f11c [0,4194304] 0 2024-09-16T00:09:55.749 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:09:55.753 INFO:tasks.workunit.client.0.smithi139.stdout:1/991: dwrite d7/d8/d1f/faf [0,4194304] 0 2024-09-16T00:09:55.773 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:55.773+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:55.782 INFO:tasks.workunit.client.0.smithi139.stdout:1/992: dwrite d7/d8/d40/d48/d4a/d74/f9f [0,4194304] 0 2024-09-16T00:09:55.785 INFO:tasks.workunit.client.0.smithi139.stdout:1/993: unlink d7/d8/d27/d68/d7c/f99 0 2024-09-16T00:09:55.793 INFO:tasks.workunit.client.0.smithi139.stdout:1/994: dread d7/f25 [0,4194304] 0 2024-09-16T00:09:55.801 INFO:tasks.workunit.client.0.smithi139.stdout:1/995: dwrite d7/f18 [0,4194304] 0 2024-09-16T00:09:55.802 INFO:tasks.workunit.client.0.smithi139.stdout:1/996: write d7/d8/d27/d68/fb4 [358676,78193] 0 2024-09-16T00:09:55.804 INFO:tasks.workunit.client.0.smithi139.stdout:1/997: creat d7/d8/d40/d48/d4a/d74/f12e x:0 0 0 2024-09-16T00:09:55.809 INFO:tasks.workunit.client.0.smithi139.stdout:1/998: rename d7/d8/d40/d48/d67/d108/d10d/f105 to d7/d8/d40/d48/dcc/f12f 0 2024-09-16T00:09:55.810 INFO:tasks.workunit.client.0.smithi139.stdout:1/999: chown d7/d8/d40/d46/f5b 4167770 1 2024-09-16T00:09:55.835 INFO:tasks.workunit.client.0.smithi139.stdout:6/986: sync 2024-09-16T00:09:55.836 INFO:tasks.workunit.client.0.smithi139.stdout:6/987: stat d8/d20/c7d 0 2024-09-16T00:09:55.837 INFO:tasks.workunit.client.0.smithi139.stdout:6/988: rmdir d8 39 2024-09-16T00:09:55.838 INFO:tasks.workunit.client.0.smithi139.stdout:6/989: chown d8/d29/d83/d26/d45/d85/ca3 2788359 1 2024-09-16T00:09:55.841 INFO:tasks.workunit.client.0.smithi139.stdout:6/990: getdents d8/d29/d83/d26/d45/d65/d8b 0 2024-09-16T00:09:55.844 INFO:tasks.workunit.client.0.smithi139.stdout:6/991: creat d8/d29/db9/d10f/d112/f155 x:0 0 0 2024-09-16T00:09:55.845 INFO:tasks.workunit.client.0.smithi139.stdout:6/992: dread - d8/d29/d83/d26/d45/d59/f134 zero size 2024-09-16T00:09:55.850 INFO:tasks.workunit.client.0.smithi139.stdout:6/993: dwrite d8/d29/db9/de8/d13a/f14c [0,4194304] 0 2024-09-16T00:09:55.856 INFO:tasks.workunit.client.0.smithi139.stdout:6/994: getdents d8/d29/d83/d26/d28/d6d/dd3/de4 0 2024-09-16T00:09:55.856 INFO:tasks.workunit.client.0.smithi139.stdout:6/995: write d8/d29/db9/d10f/d112/f155 [984547,50953] 0 2024-09-16T00:09:55.858 INFO:tasks.workunit.client.0.smithi139.stdout:6/996: fdatasync d8/d20/d4d/fee 0 2024-09-16T00:09:55.859 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:55.859 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:55.860 INFO:tasks.workunit.client.0.smithi139.stdout:6/997: truncate d8/d29/d83/d26/d28/d74/f12b 683535 0 2024-09-16T00:09:55.861 INFO:tasks.workunit.client.0.smithi139.stdout:6/998: dread - d8/d29/d83/d26/d45/d42/df3/f136 zero size 2024-09-16T00:09:55.862 INFO:tasks.workunit.client.0.smithi139.stdout:6/999: fdatasync d8/d56/f73 0 2024-09-16T00:09:55.868 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:09:55.874 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:55.874+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:55.880 INFO:tasks.workunit.client.0.smithi139.stdout:3/944: sync 2024-09-16T00:09:55.883 INFO:tasks.workunit.client.0.smithi139.stdout:3/945: creat d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/f140 x:0 0 0 2024-09-16T00:09:55.888 INFO:tasks.workunit.client.0.smithi139.stdout:3/946: dwrite d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/f9e [0,4194304] 0 2024-09-16T00:09:55.892 INFO:tasks.workunit.client.0.smithi139.stdout:3/947: creat d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/dbb/de2/f141 x:0 0 0 2024-09-16T00:09:55.897 INFO:tasks.workunit.client.0.smithi139.stdout:3/948: dread d5/f8 [0,4194304] 0 2024-09-16T00:09:55.901 INFO:tasks.workunit.client.0.smithi139.stdout:3/949: creat d5/d7/d23/de4/de8/da7/d9a/f142 x:0 0 0 2024-09-16T00:09:55.902 INFO:tasks.workunit.client.0.smithi139.stdout:3/950: rmdir d5/d7/dc/d108/df3 39 2024-09-16T00:09:55.905 INFO:tasks.workunit.client.0.smithi139.stdout:3/951: rename d5/d7/dc/d9c/f9d to d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/db9/f143 0 2024-09-16T00:09:55.906 INFO:tasks.workunit.client.0.smithi139.stdout:3/952: chown d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d47/c1d 2589 1 2024-09-16T00:09:55.910 INFO:tasks.workunit.client.0.smithi139.stdout:3/953: truncate d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/f42 1346038 0 2024-09-16T00:09:55.911 INFO:tasks.workunit.client.0.smithi139.stdout:3/954: write d5/f8 [542462,43616] 0 2024-09-16T00:09:55.916 INFO:tasks.workunit.client.0.smithi139.stdout:3/955: dwrite d5/f8 [0,4194304] 0 2024-09-16T00:09:55.918 INFO:tasks.workunit.client.0.smithi139.stdout:3/956: write d5/d7/d23/de4/de8/fde [468444,89322] 0 2024-09-16T00:09:55.921 INFO:tasks.workunit.client.0.smithi139.stdout:3/957: write d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/f11a [541708,46705] 0 2024-09-16T00:09:55.928 INFO:tasks.workunit.client.0.smithi139.stdout:3/958: dwrite d5/d7/dc/d108/f129 [0,4194304] 0 2024-09-16T00:09:55.930 INFO:tasks.workunit.client.0.smithi139.stdout:3/959: truncate d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/f79 4931123 0 2024-09-16T00:09:55.931 INFO:tasks.workunit.client.0.smithi139.stdout:3/960: write d5/d7/d23/de4/f87 [2247502,58798] 0 2024-09-16T00:09:55.933 INFO:tasks.workunit.client.0.smithi139.stdout:3/961: fsync d5/d7/d23/f8f 0 2024-09-16T00:09:55.935 INFO:tasks.workunit.client.0.smithi139.stdout:3/962: chown d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d47/caa 83 1 2024-09-16T00:09:55.938 INFO:tasks.workunit.client.0.smithi139.stdout:3/963: truncate d5/d7/d23/f8f 588519 0 2024-09-16T00:09:55.939 INFO:tasks.workunit.client.0.smithi139.stdout:3/964: stat d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/db3/fb6 0 2024-09-16T00:09:55.941 INFO:tasks.workunit.client.0.smithi139.stdout:3/965: unlink d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/f126 0 2024-09-16T00:09:55.943 INFO:tasks.workunit.client.0.smithi139.stdout:3/966: write d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/fcc [3284266,114924] 0 2024-09-16T00:09:55.946 INFO:tasks.workunit.client.0.smithi139.stdout:3/967: rmdir d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/d76 39 2024-09-16T00:09:55.948 INFO:tasks.workunit.client.0.smithi139.stdout:3/968: creat d5/d7/f144 x:0 0 0 2024-09-16T00:09:55.950 INFO:tasks.workunit.client.0.smithi139.stdout:3/969: write d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/db9/f114 [1382269,3690] 0 2024-09-16T00:09:55.952 INFO:tasks.workunit.client.0.smithi139.stdout:3/970: mkdir d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d12b/d145 0 2024-09-16T00:09:55.954 INFO:tasks.workunit.client.0.smithi139.stdout:3/971: mkdir d5/d7/d23/de4/d146 0 2024-09-16T00:09:55.958 INFO:tasks.workunit.client.0.smithi139.stdout:3/972: dread d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/fb5 [4194304,4194304] 0 2024-09-16T00:09:55.960 INFO:tasks.workunit.client.0.smithi139.stdout:3/973: dread d5/d7/d23/de4/de8/da7/d9a/f11b [0,4194304] 0 2024-09-16T00:09:55.961 INFO:tasks.workunit.client.0.smithi139.stdout:3/974: symlink d5/d7/dc/d9c/l147 0 2024-09-16T00:09:55.962 INFO:tasks.workunit.client.0.smithi139.stdout:3/975: rmdir d5/d7/d23/de4/de8/da7/d9a 39 2024-09-16T00:09:55.965 INFO:tasks.workunit.client.0.smithi139.stdout:3/976: getdents d5/d7/d23/de4/de8/da7 0 2024-09-16T00:09:55.967 INFO:tasks.workunit.client.0.smithi139.stdout:3/977: mkdir d5/d7/d23/de4/de8/da7/d9a/d101/dab/d148 0 2024-09-16T00:09:55.968 INFO:tasks.workunit.client.0.smithi139.stdout:3/978: chown d5/d7/d23/de4/de8/fde 11725 1 2024-09-16T00:09:55.968 INFO:tasks.workunit.client.0.smithi139.stdout:3/979: readlink d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/laf 0 2024-09-16T00:09:55.969 INFO:tasks.workunit.client.0.smithi139.stdout:3/980: stat d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/db3/d10b/d11f 0 2024-09-16T00:09:55.971 INFO:tasks.workunit.client.0.smithi139.stdout:3/981: creat d5/d7/d23/de4/d75/f149 x:0 0 0 2024-09-16T00:09:55.976 INFO:tasks.workunit.client.0.smithi139.stdout:3/982: dwrite d5/d7/dc/d84/f10e [0,4194304] 0 2024-09-16T00:09:55.976 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:55.975+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:55.977 INFO:tasks.workunit.client.0.smithi139.stdout:3/983: chown d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/fe6 14603 1 2024-09-16T00:09:55.983 INFO:tasks.workunit.client.0.smithi139.stdout:3/984: dwrite d5/d7/d23/de4/f87 [0,4194304] 0 2024-09-16T00:09:55.989 INFO:tasks.workunit.client.0.smithi139.stdout:3/985: link d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/da1/fd3 d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/db3/d10b/d11f/f14a 0 2024-09-16T00:09:55.991 INFO:tasks.workunit.client.0.smithi139.stdout:3/986: write d5/d7/dc/f12 [4934687,7905] 0 2024-09-16T00:09:55.996 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:55.996 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:55.997 INFO:tasks.workunit.client.0.smithi139.stdout:3/987: dwrite d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/f7f [4194304,4194304] 0 2024-09-16T00:09:56.002 INFO:tasks.workunit.client.0.smithi139.stdout:3/988: symlink d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/l14b 0 2024-09-16T00:09:56.006 INFO:tasks.workunit.client.0.smithi139.stdout:3/989: mknod d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d41/d68/c14c 0 2024-09-16T00:09:56.007 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:09:56.009 INFO:tasks.workunit.client.0.smithi139.stdout:3/990: dread d5/d7/f13 [4194304,4194304] 0 2024-09-16T00:09:56.012 INFO:tasks.workunit.client.0.smithi139.stdout:3/991: unlink d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d47/c1d 0 2024-09-16T00:09:56.013 INFO:tasks.workunit.client.0.smithi139.stdout:3/992: write d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/f140 [429971,80722] 0 2024-09-16T00:09:56.015 INFO:tasks.workunit.client.0.smithi139.stdout:3/993: truncate d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/dbb/fee 4794361 0 2024-09-16T00:09:56.016 INFO:tasks.workunit.client.0.smithi139.stdout:3/994: write d5/d7/ff8 [646151,88356] 0 2024-09-16T00:09:56.018 INFO:tasks.workunit.client.0.smithi139.stdout:3/995: chown d5/d7/d23/de4/de8/f3b 704852212 1 2024-09-16T00:09:56.019 INFO:tasks.workunit.client.0.smithi139.stdout:3/996: read d5/d7/f28 [3320038,116549] 0 2024-09-16T00:09:56.022 INFO:tasks.workunit.client.0.smithi139.stdout:3/997: unlink d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/f140 0 2024-09-16T00:09:56.024 INFO:tasks.workunit.client.0.smithi139.stdout:3/998: readlink d5/d7/d23/de4/de8/da7/d9a/d101/d2f/d30/d4c/d6d/d76/l99 0 2024-09-16T00:09:56.032 INFO:tasks.workunit.client.0.smithi139.stdout:3/999: dwrite d5/d7/d23/f94 [0,4194304] 0 2024-09-16T00:09:56.036 INFO:tasks.workunit.client.0.smithi139.stderr:+ rm -rf -- ./tmp.y3b0TuueRu 2024-09-16T00:09:56.076 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:56.076+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:56.121 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:56.121 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:56.130 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:09:56.214 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:56.214+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:56.275 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T00:09:56.277 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:56.277+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:56.280 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:56.280 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:56.290 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:09:56.378 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:56.378+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:56.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.383+0000 7f25ed60e640 1 Processor -- start 2024-09-16T00:09:56.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.383+0000 7f25ed60e640 1 -- start start 2024-09-16T00:09:56.385 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:09:56.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.384+0000 7f25ed60e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f25e80c7d30 0x7f25e806cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:56.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.384+0000 7f25ed60e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25e806bcc0 con 0x7f25e8154da0 2024-09-16T00:09:56.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.384+0000 7f25ed60e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25e806bfa0 con 0x7f25e80c8110 2024-09-16T00:09:56.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.384+0000 7f25ed60e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f25e806c8d0 con 0x7f25e80c7d30 2024-09-16T00:09:56.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.384+0000 7f25e77fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f25e8154da0 0x7f25e8165180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59296/0 (socket says 172.21.15.139:59296) 2024-09-16T00:09:56.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.384+0000 7f25e77fe640 1 -- 172.21.15.139:0/3404065294 learned_addr learned my addr 172.21.15.139:0/3404065294 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:56.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.384+0000 7f25e6ffd640 1 --2- 172.21.15.139:0/3404065294 >> v2:172.21.15.164:3300/0 conn(0x7f25e80c7d30 0x7f25e806cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:56.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.385+0000 7f25e5ffb640 1 -- 172.21.15.139:0/3404065294 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1407646977 0 0) 0x7f25e806bcc0 con 0x7f25e8154da0 2024-09-16T00:09:56.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.385+0000 7f25e5ffb640 1 -- 172.21.15.139:0/3404065294 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25d0003520 con 0x7f25e8154da0 2024-09-16T00:09:56.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.385+0000 7f25e5ffb640 1 -- 172.21.15.139:0/3404065294 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1385397055 0 0) 0x7f25e806bfa0 con 0x7f25e80c8110 2024-09-16T00:09:56.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.385+0000 7f25e5ffb640 1 -- 172.21.15.139:0/3404065294 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25e806bcc0 con 0x7f25e80c8110 2024-09-16T00:09:56.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.385+0000 7f25e6ffd640 1 -- 172.21.15.139:0/3404065294 >> v1:172.21.15.164:6789/0 conn(0x7f25e80c8110 legacy=0x7f25e8154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:56.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.385+0000 7f25e6ffd640 1 -- 172.21.15.139:0/3404065294 >> v1:172.21.15.139:6789/0 conn(0x7f25e8154da0 legacy=0x7f25e8165180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:56.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.385+0000 7f25e6ffd640 1 -- 172.21.15.139:0/3404065294 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e806c280 con 0x7f25e80c7d30 2024-09-16T00:09:56.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.385+0000 7f25e6ffd640 1 --2- 172.21.15.139:0/3404065294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25e80c7d30 0x7f25e806cd60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f25e806ca50 tx=0x7f25d802a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b79119544fea17d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:56.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.386+0000 7f25e5ffb640 1 -- 172.21.15.139:0/3404065294 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8037070 con 0x7f25e80c7d30 2024-09-16T00:09:56.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.386+0000 7f25e5ffb640 1 -- 172.21.15.139:0/3404065294 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d8032640 con 0x7f25e80c7d30 2024-09-16T00:09:56.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.386+0000 7f25e5ffb640 1 -- 172.21.15.139:0/3404065294 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d80329f0 con 0x7f25e80c7d30 2024-09-16T00:09:56.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.386+0000 7f25ed60e640 1 -- 172.21.15.139:0/3404065294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25e80c7d30 msgr2=0x7f25e806cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:56.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.386+0000 7f25ed60e640 1 --2- 172.21.15.139:0/3404065294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25e80c7d30 0x7f25e806cd60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f25e806ca50 tx=0x7f25d802a2c0 comp rx=0 tx=0).stop 2024-09-16T00:09:56.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.386+0000 7f25ed60e640 1 -- 172.21.15.139:0/3404065294 shutdown_connections 2024-09-16T00:09:56.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.386+0000 7f25ed60e640 1 --2- 172.21.15.139:0/3404065294 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25e80c7d30 0x7f25e806cd60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:56.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.386+0000 7f25ed60e640 1 -- 172.21.15.139:0/3404065294 >> 172.21.15.139:0/3404065294 conn(0x7f25e80c91d0 msgr2=0x7f25e80c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:56.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.386+0000 7f25ed60e640 1 -- 172.21.15.139:0/3404065294 shutdown_connections 2024-09-16T00:09:56.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.387+0000 7f25ed60e640 1 -- 172.21.15.139:0/3404065294 wait complete. 2024-09-16T00:09:56.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.388+0000 7f25ed60e640 1 Processor -- start 2024-09-16T00:09:56.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.388+0000 7f25ed60e640 1 -- start start 2024-09-16T00:09:56.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25ed60e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25e80c7d30 0x7f25e815cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:56.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25ed60e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25e80c8110 0x7f25e81e4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:56.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25ed60e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25e8154da0 0x7f25e81e5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:56.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25ed60e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f25e806d2a0 con 0x7f25e80c8110 2024-09-16T00:09:56.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25ed60e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f25e80c9cf0 con 0x7f25e8154da0 2024-09-16T00:09:56.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25ed60e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f25e806ea00 con 0x7f25e80c7d30 2024-09-16T00:09:56.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25e67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25e80c8110 0x7f25e81e4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:56.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25e67fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25e80c8110 0x7f25e81e4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:37636/0 (socket says 172.21.15.139:37636) 2024-09-16T00:09:56.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25e67fc640 1 -- 172.21.15.139:0/4153538241 learned_addr learned my addr 172.21.15.139:0/4153538241 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:56.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25e77fe640 1 --2- 172.21.15.139:0/4153538241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25e8154da0 0x7f25e81e5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:56.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25e6ffd640 1 --2- 172.21.15.139:0/4153538241 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25e80c7d30 0x7f25e815cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:56.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25e67fc640 1 -- 172.21.15.139:0/4153538241 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25e80c7d30 msgr2=0x7f25e815cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:56.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25e67fc640 1 --2- 172.21.15.139:0/4153538241 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25e80c7d30 0x7f25e815cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:56.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25e67fc640 1 -- 172.21.15.139:0/4153538241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25e8154da0 msgr2=0x7f25e81e5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:56.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25e67fc640 1 --2- 172.21.15.139:0/4153538241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25e8154da0 0x7f25e81e5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:56.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25e67fc640 1 -- 172.21.15.139:0/4153538241 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e81f0b90 con 0x7f25e80c8110 2024-09-16T00:09:56.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25e67fc640 1 --2- 172.21.15.139:0/4153538241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25e80c8110 0x7f25e81e4f40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00c490 tx=0x7f25dc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:56.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25cf7fe640 1 -- 172.21.15.139:0/4153538241 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc017070 con 0x7f25e80c8110 2024-09-16T00:09:56.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25cf7fe640 1 -- 172.21.15.139:0/4153538241 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25dc0052d0 con 0x7f25e80c8110 2024-09-16T00:09:56.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.389+0000 7f25ed60e640 1 -- 172.21.15.139:0/4153538241 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e81f0e70 con 0x7f25e80c8110 2024-09-16T00:09:56.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.390+0000 7f25cf7fe640 1 -- 172.21.15.139:0/4153538241 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc003830 con 0x7f25e80c8110 2024-09-16T00:09:56.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.390+0000 7f25ed60e640 1 -- 172.21.15.139:0/4153538241 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e81f1430 con 0x7f25e80c8110 2024-09-16T00:09:56.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.390+0000 7f25ed60e640 1 -- 172.21.15.139:0/4153538241 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e806f0b0 con 0x7f25e80c8110 2024-09-16T00:09:56.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.391+0000 7f25cf7fe640 1 -- 172.21.15.139:0/4153538241 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f25dc01a070 con 0x7f25e80c8110 2024-09-16T00:09:56.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.392+0000 7f25cf7fe640 1 --2- 172.21.15.139:0/4153538241 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f25d406c2b0 0x7f25d406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:56.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.392+0000 7f25cf7fe640 1 -- 172.21.15.139:0/4153538241 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5973+0+0 (secure 0 0 0) 0x7f25dc08a9b0 con 0x7f25e80c8110 2024-09-16T00:09:56.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.392+0000 7f25e6ffd640 1 --2- 172.21.15.139:0/4153538241 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f25d406c2b0 0x7f25d406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:56.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.392+0000 7f25e6ffd640 1 --2- 172.21.15.139:0/4153538241 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f25d406c2b0 0x7f25d406e770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f25d8034000 tx=0x7f25d802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:56.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.396+0000 7f25cf7fe640 1 -- 172.21.15.139:0/4153538241 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f25e806f0b0 con 0x7f25e80c8110 2024-09-16T00:09:56.446 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:56.446 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:56.456 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:09:56.479 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:56.479+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:56.493 INFO:teuthology.orchestra.run.smithi164.stderr:no valid command found; 10 closest matches: 2024-09-16T00:09:56.493 INFO:teuthology.orchestra.run.smithi164.stderr:0 2024-09-16T00:09:56.493 INFO:teuthology.orchestra.run.smithi164.stderr:1 2024-09-16T00:09:56.493 INFO:teuthology.orchestra.run.smithi164.stderr:2 2024-09-16T00:09:56.494 INFO:teuthology.orchestra.run.smithi164.stderr:abort 2024-09-16T00:09:56.494 INFO:teuthology.orchestra.run.smithi164.stderr:assert 2024-09-16T00:09:56.494 INFO:teuthology.orchestra.run.smithi164.stderr:bluefs debug_inject_read_zeros 2024-09-16T00:09:56.494 INFO:teuthology.orchestra.run.smithi164.stderr:bluefs files list 2024-09-16T00:09:56.494 INFO:teuthology.orchestra.run.smithi164.stderr:bluefs stats 2024-09-16T00:09:56.494 INFO:teuthology.orchestra.run.smithi164.stderr:bluestore allocator dump block 2024-09-16T00:09:56.494 INFO:teuthology.orchestra.run.smithi164.stderr:bluestore allocator fragmentation block 2024-09-16T00:09:56.494 INFO:teuthology.orchestra.run.smithi164.stderr:admin_socket: invalid command 2024-09-16T00:09:56.496 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T00:09:56.496 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2024-09-16T00:09:56.563 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:56.563 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:56.572 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:09:56.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:56.599+0000 7f25ed60e640 1 -- 172.21.15.139:0/4153538241 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f25e814fe70 con 0x7f25e80c8110 2024-09-16T00:09:56.600 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:56.599+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:56.681 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:56.681+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:56.683 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:56.684 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:56.693 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:09:56.782 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:56.782+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:56.859 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:56.859 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:56.869 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:09:56.911 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:56.910+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:56.959 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:09:57.003 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:57.003+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:57.021 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:57.021 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:57.031 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:09:57.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.062+0000 7fdac6a96640 1 Processor -- start 2024-09-16T00:09:57.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.062+0000 7fdac6a96640 1 -- start start 2024-09-16T00:09:57.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.063+0000 7fdac6a96640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdac006cea0 0x7fdac006dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:57.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.063+0000 7fdac6a96640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdac0151cf0 0x7fdac006d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:57.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.063+0000 7fdac6a96640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdac0158520 con 0x7fdac006d280 2024-09-16T00:09:57.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.063+0000 7fdac6a96640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdac00caa30 con 0x7fdac006cea0 2024-09-16T00:09:57.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.063+0000 7fdac6a96640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fdac0072a70 con 0x7fdac0151cf0 2024-09-16T00:09:57.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.063+0000 7fdac4832640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdac006cea0 0x7fdac006dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:57.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.063+0000 7fdac5033640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fdac0151cf0 0x7fdac006d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:57.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.063+0000 7fdabffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fdac006d280 0x7fdac01515e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:59300/0 (socket says 172.21.15.139:59300) 2024-09-16T00:09:57.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.063+0000 7fdabffff640 1 -- 172.21.15.139:0/3612517916 learned_addr learned my addr 172.21.15.139:0/3612517916 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:57.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.063+0000 7fdac4832640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdac006cea0 0x7fdac006dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:37642/0 (socket says 172.21.15.139:37642) 2024-09-16T00:09:57.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.064+0000 7fdac5033640 1 -- 172.21.15.139:0/3612517916 >> v2:172.21.15.139:3300/0 conn(0x7fdac006cea0 msgr2=0x7fdac006dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:09:57.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.064+0000 7fdac5033640 1 --2- 172.21.15.139:0/3612517916 >> v2:172.21.15.139:3300/0 conn(0x7fdac006cea0 0x7fdac006dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:57.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.063+0000 7fdabf7fe640 1 -- 172.21.15.139:0/3612517916 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 884377031 0 0) 0x7fdac0158520 con 0x7fdac006d280 2024-09-16T00:09:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.064+0000 7fdac5033640 1 -- 172.21.15.139:0/3612517916 >> v1:172.21.15.139:6789/0 conn(0x7fdac006d280 legacy=0x7fdac01515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.064+0000 7fdac5033640 1 -- 172.21.15.139:0/3612517916 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac0073dc0 con 0x7fdac0151cf0 2024-09-16T00:09:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.064+0000 7fdac5033640 1 --2- 172.21.15.139:0/3612517916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0151cf0 0x7fdac006d6b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdac00c9a60 tx=0x7fdab802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49752f24b3009fff server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.064+0000 7fdabf7fe640 1 -- 172.21.15.139:0/3612517916 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8007740 con 0x7fdac0151cf0 2024-09-16T00:09:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.064+0000 7fdabf7fe640 1 -- 172.21.15.139:0/3612517916 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdab80078e0 con 0x7fdac0151cf0 2024-09-16T00:09:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.064+0000 7fdabf7fe640 1 -- 172.21.15.139:0/3612517916 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8007cb0 con 0x7fdac0151cf0 2024-09-16T00:09:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.065+0000 7fdac6a96640 1 -- 172.21.15.139:0/3612517916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0151cf0 msgr2=0x7fdac006d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.065+0000 7fdac6a96640 1 --2- 172.21.15.139:0/3612517916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0151cf0 0x7fdac006d6b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdac00c9a60 tx=0x7fdab802a710 comp rx=0 tx=0).stop 2024-09-16T00:09:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.065+0000 7fdac6a96640 1 -- 172.21.15.139:0/3612517916 shutdown_connections 2024-09-16T00:09:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.065+0000 7fdac6a96640 1 --2- 172.21.15.139:0/3612517916 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0151cf0 0x7fdac006d6b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.065+0000 7fdac6a96640 1 --2- 172.21.15.139:0/3612517916 >> v2:172.21.15.139:3300/0 conn(0x7fdac006cea0 0x7fdac006dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.065+0000 7fdac6a96640 1 -- 172.21.15.139:0/3612517916 >> 172.21.15.139:0/3612517916 conn(0x7fdac00c9af0 msgr2=0x7fdac00c9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.065+0000 7fdac6a96640 1 -- 172.21.15.139:0/3612517916 shutdown_connections 2024-09-16T00:09:57.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.065+0000 7fdac6a96640 1 -- 172.21.15.139:0/3612517916 wait complete. 2024-09-16T00:09:57.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.066+0000 7fdac6a96640 1 Processor -- start 2024-09-16T00:09:57.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.067+0000 7fdac6a96640 1 -- start start 2024-09-16T00:09:57.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.067+0000 7fdac6a96640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdac006cea0 0x7fdac0156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:57.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.067+0000 7fdac6a96640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdac006d280 0x7fdac0153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:57.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.067+0000 7fdac6a96640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0151cf0 0x7fdac014e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:57.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.067+0000 7fdac6a96640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01581e0 con 0x7fdac006cea0 2024-09-16T00:09:57.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.067+0000 7fdac6a96640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdac0072700 con 0x7fdac0151cf0 2024-09-16T00:09:57.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.067+0000 7fdac6a96640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdac0072f00 con 0x7fdac006d280 2024-09-16T00:09:57.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdac4832640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdac006cea0 0x7fdac0156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:57.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdac5033640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0151cf0 0x7fdac014e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:57.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdac4832640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdac006cea0 0x7fdac0156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:37656/0 (socket says 172.21.15.139:37656) 2024-09-16T00:09:57.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdac5033640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0151cf0 0x7fdac014e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:57164/0 (socket says 172.21.15.139:57164) 2024-09-16T00:09:57.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdac4832640 1 -- 172.21.15.139:0/2740265313 learned_addr learned my addr 172.21.15.139:0/2740265313 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:57.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdabffff640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdac006d280 0x7fdac0153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:57.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdac4832640 1 -- 172.21.15.139:0/2740265313 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdac006d280 msgr2=0x7fdac0153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:57.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdac4832640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdac006d280 0x7fdac0153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:57.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdac4832640 1 -- 172.21.15.139:0/2740265313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0151cf0 msgr2=0x7fdac014e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:57.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdac4832640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0151cf0 0x7fdac014e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:57.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdac4832640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac0154960 con 0x7fdac006cea0 2024-09-16T00:09:57.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdac5033640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdac0151cf0 0x7fdac014e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:57.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdac4832640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdac006cea0 0x7fdac0156cc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdab400eb70 tx=0x7fdab400d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:57.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4010640 con 0x7fdac006cea0 2024-09-16T00:09:57.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdab40107e0 con 0x7fdac006cea0 2024-09-16T00:09:57.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac01f0b00 con 0x7fdac006cea0 2024-09-16T00:09:57.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4010bd0 con 0x7fdac006cea0 2024-09-16T00:09:57.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.068+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac0153370 con 0x7fdac006cea0 2024-09-16T00:09:57.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.069+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fdac006d6b0 con 0x7fdac006cea0 2024-09-16T00:09:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.070+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdab4018400 con 0x7fdac006cea0 2024-09-16T00:09:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.071+0000 7fdabd7fa640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdaa406c2b0 0x7fdaa406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.071+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5973+0+0 (secure 0 0 0) 0x7fdab408bdc0 con 0x7fdac006cea0 2024-09-16T00:09:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.071+0000 7fdabffff640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdaa406c2b0 0x7fdaa406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.071+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v91) ==== 54+0+16 (secure 0 0 0) 0x7fdab4014030 con 0x7fdac006cea0 2024-09-16T00:09:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.071+0000 7fdac6a96640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fdac0153570 0x7fdac00c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.071+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdac00cb1e0 con 0x7fdac0153570 2024-09-16T00:09:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.071+0000 7fdac5033640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fdac0153570 0x7fdac00c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.071+0000 7fdabffff640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdaa406c2b0 0x7fdaa406e770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdac01505a0 tx=0x7fdaac007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:57.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.071+0000 7fdac5033640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fdac0153570 0x7fdac00c76e0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:57.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.072+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdac00cb1e0 con 0x7fdac0153570 2024-09-16T00:09:57.086 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:57.085+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:57.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.087+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdac00c4180 con 0x7fdac0153570 2024-09-16T00:09:57.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.088+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdac00c4180 con 0x7fdac0153570 2024-09-16T00:09:57.089 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:57.090 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:57.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.089+0000 7fdac6a96640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fdac01540b0 0x7fdac01f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:57.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.089+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fdac00c7c20 con 0x7fdac01540b0 2024-09-16T00:09:57.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.089+0000 7fdac4832640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fdac01540b0 0x7fdac01f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:57.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.089+0000 7fdac4832640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fdac01540b0 0x7fdac01f3a60 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:57.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.090+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdac00c7c20 con 0x7fdac01540b0 2024-09-16T00:09:57.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.105+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdac0157200 con 0x7fdac01540b0 2024-09-16T00:09:57.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.107+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdac0157200 con 0x7fdac01540b0 2024-09-16T00:09:57.108 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:57.108 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:57.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.107+0000 7fdac6a96640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fdac0062630 0x7fdac0062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:57.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.107+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fdac01578c0 con 0x7fdac0062630 2024-09-16T00:09:57.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.107+0000 7fdabffff640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fdac0062630 0x7fdac0062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:57.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.108+0000 7fdabffff640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fdac0062630 0x7fdac0062a30 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:57.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.109+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdac01578c0 con 0x7fdac0062630 2024-09-16T00:09:57.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.123+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdac0154fa0 con 0x7fdac0062630 2024-09-16T00:09:57.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.125+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdac0154fa0 con 0x7fdac0062630 2024-09-16T00:09:57.126 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:57.126 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:57.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.125+0000 7fdac6a96640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fdac0063330 0x7fdac01558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:57.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.125+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fdac01551b0 con 0x7fdac0063330 2024-09-16T00:09:57.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.125+0000 7fdac5033640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fdac0063330 0x7fdac01558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:57.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.125+0000 7fdac5033640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fdac0063330 0x7fdac01558d0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:57.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.126+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdab8045ac0 con 0x7fdac0063330 2024-09-16T00:09:57.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.142+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdac0156060 con 0x7fdac0063330 2024-09-16T00:09:57.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.143+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdac0156060 con 0x7fdac0063330 2024-09-16T00:09:57.144 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.144+0000 7fdac6a96640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fdac002cb50 0x7fdac002cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.144+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fdac01561d0 con 0x7fdac002cb50 2024-09-16T00:09:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.144+0000 7fdac4832640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fdac002cb50 0x7fdac002cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.144+0000 7fdac4832640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fdac002cb50 0x7fdac002cfb0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:57.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.145+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdac01561d0 con 0x7fdac002cb50 2024-09-16T00:09:57.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.159+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdac01561d0 con 0x7fdac002cb50 2024-09-16T00:09:57.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.161+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdac01561d0 con 0x7fdac002cb50 2024-09-16T00:09:57.162 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:57.162 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:57.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.161+0000 7fdac6a96640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fdac02048f0 0x7fdac0206ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:57.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.161+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fdac0156340 con 0x7fdac02048f0 2024-09-16T00:09:57.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.161+0000 7fdabffff640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fdac02048f0 0x7fdac0206ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:57.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.162+0000 7fdabffff640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fdac02048f0 0x7fdac0206ce0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:57.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.162+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdac0156340 con 0x7fdac02048f0 2024-09-16T00:09:57.178 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:57.178 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:57.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:57.179 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:57.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.179 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5850 3.3a 3:5c3cfc2b:::10000001adf.00000000:head [write 0~116] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:57.179 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:09:57.179 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.760679019999998, 2024-09-16T00:09:57.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66931295800000001, 2024-09-16T00:09:57.179 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.179 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.179 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.179 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.179 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.179 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.179 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5850 2024-09-16T00:09:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:09:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:09:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415938+0000", 2024-09-16T00:09:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7099999999999999e-06 2024-09-16T00:09:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415945+0000", 2024-09-16T00:09:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:09:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415959+0000", 2024-09-16T00:09:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3545e-05 2024-09-16T00:09:57.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415966+0000", 2024-09-16T00:09:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6379999999999998e-06 2024-09-16T00:09:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416013+0000", 2024-09-16T00:09:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7314999999999997e-05 2024-09-16T00:09:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416020+0000", 2024-09-16T00:09:57.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.52e-06 2024-09-16T00:09:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082393+0000", 2024-09-16T00:09:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66637251799999997 2024-09-16T00:09:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082537+0000", 2024-09-16T00:09:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014418800000000001 2024-09-16T00:09:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:57.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082689+0000", 2024-09-16T00:09:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015198499999999999 2024-09-16T00:09:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084930+0000", 2024-09-16T00:09:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002241041 2024-09-16T00:09:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084994+0000", 2024-09-16T00:09:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3274000000000004e-05 2024-09-16T00:09:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085049+0000", 2024-09-16T00:09:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.554e-05 2024-09-16T00:09:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085250+0000", 2024-09-16T00:09:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020060799999999999 2024-09-16T00:09:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5910 3.3a 3:5ec98a62:::10000001b22.00000000:head [write 0~1274] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:57.185 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:09:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.613735382000002, 2024-09-16T00:09:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.52286026600000002, 2024-09-16T00:09:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5910 2024-09-16T00:09:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.186 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562881+0000", 2024-09-16T00:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562887+0000", 2024-09-16T00:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2870000000000004e-06 2024-09-16T00:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562897+0000", 2024-09-16T00:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.064e-05 2024-09-16T00:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562902+0000", 2024-09-16T00:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2340000000000002e-06 2024-09-16T00:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562958+0000", 2024-09-16T00:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5523999999999999e-05 2024-09-16T00:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562968+0000", 2024-09-16T00:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0394000000000001e-05 2024-09-16T00:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083461+0000", 2024-09-16T00:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.520492968 2024-09-16T00:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083645+0000", 2024-09-16T00:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018346399999999999 2024-09-16T00:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083844+0000", 2024-09-16T00:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000199268 2024-09-16T00:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085410+0000", 2024-09-16T00:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015662359999999999 2024-09-16T00:09:57.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085493+0000", 2024-09-16T00:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2478999999999994e-05 2024-09-16T00:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085563+0000", 2024-09-16T00:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0159000000000004e-05 2024-09-16T00:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085741+0000", 2024-09-16T00:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017764300000000001 2024-09-16T00:09:57.191 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5962 3.3a 3:5c7b4d29:::10000001b5d.00000000:head [write 0~3074] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.484198565, 2024-09-16T00:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39368737500000001, 2024-09-16T00:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.192 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5962 2024-09-16T00:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:09:57.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692418+0000", 2024-09-16T00:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3510000000000001e-06 2024-09-16T00:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692425+0000", 2024-09-16T00:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3770000000000001e-06 2024-09-16T00:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692440+0000", 2024-09-16T00:09:57.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4975e-05 2024-09-16T00:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692448+0000", 2024-09-16T00:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6279999999999994e-06 2024-09-16T00:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692499+0000", 2024-09-16T00:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0872999999999999e-05 2024-09-16T00:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:57.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692507+0000", 2024-09-16T00:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9889999999999992e-06 2024-09-16T00:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084221+0000", 2024-09-16T00:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39171379699999997 2024-09-16T00:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084446+0000", 2024-09-16T00:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022519900000000001 2024-09-16T00:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084672+0000", 2024-09-16T00:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000225405 2024-09-16T00:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085832+0000", 2024-09-16T00:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011600040000000001 2024-09-16T00:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085864+0000", 2024-09-16T00:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2592999999999998e-05 2024-09-16T00:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085943+0000", 2024-09-16T00:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8534000000000004e-05 2024-09-16T00:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086105+0000", 2024-09-16T00:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016165 2024-09-16T00:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.198 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5963 3.3a 3:5e4192f5:::10000001b5e.00000000:head [write 0~152] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.482134133999999, 2024-09-16T00:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39413463900000001, 2024-09-16T00:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5963 2024-09-16T00:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694486+0000", 2024-09-16T00:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0960000000000003e-06 2024-09-16T00:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694496+0000", 2024-09-16T00:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2189999999999993e-06 2024-09-16T00:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694500+0000", 2024-09-16T00:09:57.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9529999999999998e-06 2024-09-16T00:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694552+0000", 2024-09-16T00:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1360999999999998e-05 2024-09-16T00:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694561+0000", 2024-09-16T00:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1700000000000003e-06 2024-09-16T00:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086346+0000", 2024-09-16T00:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39178468 2024-09-16T00:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086480+0000", 2024-09-16T00:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013469299999999999 2024-09-16T00:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086790+0000", 2024-09-16T00:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030923300000000001 2024-09-16T00:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.203 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088361+0000", 2024-09-16T00:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015717050000000001 2024-09-16T00:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088432+0000", 2024-09-16T00:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0922999999999998e-05 2024-09-16T00:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088489+0000", 2024-09-16T00:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6653999999999997e-05 2024-09-16T00:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088616+0000", 2024-09-16T00:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012730099999999999 2024-09-16T00:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:6082 3.3a 3:5c50f65b:::10000001be7.00000000:head [write 0~2439] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.188361278000002, 2024-09-16T00:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.10065850799999999, 2024-09-16T00:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.205 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 6082 2024-09-16T00:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988256+0000", 2024-09-16T00:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.065e-06 2024-09-16T00:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988262+0000", 2024-09-16T00:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4640000000000003e-06 2024-09-16T00:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988272+0000", 2024-09-16T00:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0335e-05 2024-09-16T00:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988279+0000", 2024-09-16T00:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988324+0000", 2024-09-16T00:09:57.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4953000000000001e-05 2024-09-16T00:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988331+0000", 2024-09-16T00:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5989999999999998e-06 2024-09-16T00:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087272+0000", 2024-09-16T00:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.098941045000000005 2024-09-16T00:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.209 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087486+0000", 2024-09-16T00:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00021360199999999999 2024-09-16T00:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087740+0000", 2024-09-16T00:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00025456799999999999 2024-09-16T00:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088638+0000", 2024-09-16T00:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00089793900000000005 2024-09-16T00:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088673+0000", 2024-09-16T00:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5111999999999999e-05 2024-09-16T00:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088722+0000", 2024-09-16T00:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8758000000000003e-05 2024-09-16T00:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088913+0000", 2024-09-16T00:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000190953 2024-09-16T00:09:57.211 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10643 3.21 3:8453163e:::100000030b3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 21.576885085000001, 2024-09-16T00:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013028809000000001, 2024-09-16T00:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.212 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10643 2024-09-16T00:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599734+0000", 2024-09-16T00:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-06 2024-09-16T00:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603345+0000", 2024-09-16T00:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003611628 2024-09-16T00:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603368+0000", 2024-09-16T00:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3017e-05 2024-09-16T00:09:57.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603379+0000", 2024-09-16T00:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0528e-05 2024-09-16T00:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603448+0000", 2024-09-16T00:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9080999999999998e-05 2024-09-16T00:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603623+0000", 2024-09-16T00:09:57.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017525599999999999 2024-09-16T00:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603766+0000", 2024-09-16T00:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000142977 2024-09-16T00:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.611250+0000", 2024-09-16T00:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074835240000000001 2024-09-16T00:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612617+0000", 2024-09-16T00:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001367793 2024-09-16T00:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612700+0000", 2024-09-16T00:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3123999999999999e-05 2024-09-16T00:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612759+0000", 2024-09-16T00:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8950999999999999e-05 2024-09-16T00:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.217 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10652 3.22 3:44e46336:::100000030c1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 21.548833453, 2024-09-16T00:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011405732, 2024-09-16T00:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10652 2024-09-16T00:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.218 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627785+0000", 2024-09-16T00:09:57.219 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6970000000000002e-06 2024-09-16T00:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629610+0000", 2024-09-16T00:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018250359999999999 2024-09-16T00:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629639+0000", 2024-09-16T00:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8676000000000001e-05 2024-09-16T00:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629652+0000", 2024-09-16T00:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3572e-05 2024-09-16T00:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629735+0000", 2024-09-16T00:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.25e-05 2024-09-16T00:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629925+0000", 2024-09-16T00:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019011000000000001 2024-09-16T00:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.630075+0000", 2024-09-16T00:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000149841 2024-09-16T00:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.636470+0000", 2024-09-16T00:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063952289999999997 2024-09-16T00:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639098+0000", 2024-09-16T00:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026277980000000002 2024-09-16T00:09:57.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639154+0000", 2024-09-16T00:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6413999999999999e-05 2024-09-16T00:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639188+0000", 2024-09-16T00:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3859000000000002e-05 2024-09-16T00:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11217 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 16.602336432000001, 2024-09-16T00:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01231253, 2024-09-16T00:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11217 2024-09-16T00:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574282+0000", 2024-09-16T00:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8499999999999998e-06 2024-09-16T00:09:57.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575804+0000", 2024-09-16T00:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001521859 2024-09-16T00:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575826+0000", 2024-09-16T00:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2059999999999999e-05 2024-09-16T00:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575837+0000", 2024-09-16T00:09:57.226 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1313e-05 2024-09-16T00:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575920+0000", 2024-09-16T00:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2918999999999997e-05 2024-09-16T00:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576088+0000", 2024-09-16T00:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016803799999999999 2024-09-16T00:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576233+0000", 2024-09-16T00:09:57.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014508199999999999 2024-09-16T00:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.580500+0000", 2024-09-16T00:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004266644 2024-09-16T00:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586479+0000", 2024-09-16T00:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059791820000000004 2024-09-16T00:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586541+0000", 2024-09-16T00:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1463000000000004e-05 2024-09-16T00:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586592+0000", 2024-09-16T00:09:57.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1119999999999998e-05 2024-09-16T00:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11767 3.38 3:1fa50e65:::10000003117.00000000:head [write 0~4194304 [3@1169732]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.012931897, 2024-09-16T00:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011555678999999999, 2024-09-16T00:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.230 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11767 2024-09-16T00:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163686+0000", 2024-09-16T00:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3259999999999998e-06 2024-09-16T00:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165053+0000", 2024-09-16T00:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013665859999999999 2024-09-16T00:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165075+0000", 2024-09-16T00:09:57.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2206999999999999e-05 2024-09-16T00:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165084+0000", 2024-09-16T00:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0710000000000001e-06 2024-09-16T00:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165157+0000", 2024-09-16T00:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2619000000000004e-05 2024-09-16T00:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165249+0000", 2024-09-16T00:09:57.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2517999999999997e-05 2024-09-16T00:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165360+0000", 2024-09-16T00:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011134899999999999 2024-09-16T00:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.169320+0000", 2024-09-16T00:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039593040000000003 2024-09-16T00:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175135+0000", 2024-09-16T00:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005814891 2024-09-16T00:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175199+0000", 2024-09-16T00:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4474999999999995e-05 2024-09-16T00:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175239+0000", 2024-09-16T00:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0333000000000002e-05 2024-09-16T00:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.235 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11789 3.21 3:8453163e:::100000030b3.00000000:head [write 0~3801088 [4@1194504]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:09:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 13.925243801000001, 2024-09-16T00:09:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011097602, 2024-09-16T00:09:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.236 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11789 2024-09-16T00:09:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:09:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:09:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251373+0000", 2024-09-16T00:09:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.544e-06 2024-09-16T00:09:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252710+0000", 2024-09-16T00:09:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013362929999999999 2024-09-16T00:09:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252730+0000", 2024-09-16T00:09:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0579999999999999e-05 2024-09-16T00:09:57.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252741+0000", 2024-09-16T00:09:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0505e-05 2024-09-16T00:09:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252820+0000", 2024-09-16T00:09:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9098999999999996e-05 2024-09-16T00:09:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252931+0000", 2024-09-16T00:09:57.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011073600000000001 2024-09-16T00:09:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.253101+0000", 2024-09-16T00:09:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017038000000000001 2024-09-16T00:09:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.261813+0000", 2024-09-16T00:09:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087119469999999994 2024-09-16T00:09:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262338+0000", 2024-09-16T00:09:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00052493799999999997 2024-09-16T00:09:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262430+0000", 2024-09-16T00:09:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1626999999999995e-05 2024-09-16T00:09:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262470+0000", 2024-09-16T00:09:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9953000000000002e-05 2024-09-16T00:09:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.241 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11890 3.38 3:1e014616:::100000033e9.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:09:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 13.605278718999999, 2024-09-16T00:09:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011833791999999999, 2024-09-16T00:09:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11890 2024-09-16T00:09:57.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:09:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:09:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571340+0000", 2024-09-16T00:09:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.36e-06 2024-09-16T00:09:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572866+0000", 2024-09-16T00:09:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001525549 2024-09-16T00:09:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572884+0000", 2024-09-16T00:09:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7796999999999999e-05 2024-09-16T00:09:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572897+0000", 2024-09-16T00:09:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3567000000000001e-05 2024-09-16T00:09:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572970+0000", 2024-09-16T00:09:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2751999999999997e-05 2024-09-16T00:09:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573053+0000", 2024-09-16T00:09:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3230000000000001e-05 2024-09-16T00:09:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573168+0000", 2024-09-16T00:09:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114536 2024-09-16T00:09:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.577923+0000", 2024-09-16T00:09:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047548720000000003 2024-09-16T00:09:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583081+0000", 2024-09-16T00:09:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051587630000000002 2024-09-16T00:09:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583137+0000", 2024-09-16T00:09:57.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5259000000000001e-05 2024-09-16T00:09:57.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583171+0000", 2024-09-16T00:09:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4106999999999997e-05 2024-09-16T00:09:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:12037 3.9 e87/20 3:928db521:::1000000326d.00000000:head v 87'199, mlcod=87'197)", 2024-09-16T00:09:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:09:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 13.174088768000001, 2024-09-16T00:09:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012236153, 2024-09-16T00:09:57.249 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:09:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:09:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002529+0000", 2024-09-16T00:09:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4889999999999998e-06 2024-09-16T00:09:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007064+0000", 2024-09-16T00:09:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045346220000000003 2024-09-16T00:09:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007093+0000", 2024-09-16T00:09:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9009e-05 2024-09-16T00:09:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007109+0000", 2024-09-16T00:09:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5846e-05 2024-09-16T00:09:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007196+0000", 2024-09-16T00:09:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7507000000000001e-05 2024-09-16T00:09:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007227+0000", 2024-09-16T00:09:57.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0916e-05 2024-09-16T00:09:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014658+0000", 2024-09-16T00:09:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074305090000000001 2024-09-16T00:09:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014763+0000", 2024-09-16T00:09:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010525499999999999 2024-09-16T00:09:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12398 3.2e 3:77a11d08:::10000003432.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:09:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 11.935409985, 2024-09-16T00:09:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012092999, 2024-09-16T00:09:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12398 2024-09-16T00:09:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:09:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:09:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241209+0000", 2024-09-16T00:09:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2710000000000001e-06 2024-09-16T00:09:57.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243840+0000", 2024-09-16T00:09:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026307919999999999 2024-09-16T00:09:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243872+0000", 2024-09-16T00:09:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2699e-05 2024-09-16T00:09:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.256 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243887+0000", 2024-09-16T00:09:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4348000000000001e-05 2024-09-16T00:09:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243972+0000", 2024-09-16T00:09:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5358999999999994e-05 2024-09-16T00:09:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244132+0000", 2024-09-16T00:09:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015972499999999999 2024-09-16T00:09:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253153+0000", 2024-09-16T00:09:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0090210280000000004 2024-09-16T00:09:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253213+0000", 2024-09-16T00:09:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0300999999999997e-05 2024-09-16T00:09:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253299+0000", 2024-09-16T00:09:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5476000000000006e-05 2024-09-16T00:09:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.258 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12400 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304 [2@815045]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:09:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 11.932372251, 2024-09-16T00:09:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011140066000000001, 2024-09-16T00:09:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12400 2024-09-16T00:09:57.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:09:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:09:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244245+0000", 2024-09-16T00:09:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.248e-06 2024-09-16T00:09:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246467+0000", 2024-09-16T00:09:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022217949999999999 2024-09-16T00:09:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246496+0000", 2024-09-16T00:09:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9484000000000001e-05 2024-09-16T00:09:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246509+0000", 2024-09-16T00:09:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3390000000000001e-05 2024-09-16T00:09:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246586+0000", 2024-09-16T00:09:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6692000000000004e-05 2024-09-16T00:09:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246680+0000", 2024-09-16T00:09:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3468999999999996e-05 2024-09-16T00:09:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:57.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246850+0000", 2024-09-16T00:09:57.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170206 2024-09-16T00:09:57.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253478+0000", 2024-09-16T00:09:57.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066287530000000002 2024-09-16T00:09:57.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.177+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdac002da00 con 0x7fdac02048f0 2024-09-16T00:09:57.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255318+0000", 2024-09-16T00:09:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001839292 2024-09-16T00:09:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255354+0000", 2024-09-16T00:09:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5964999999999999e-05 2024-09-16T00:09:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255384+0000", 2024-09-16T00:09:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9771999999999999e-05 2024-09-16T00:09:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.265 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13317 3.38 3:1cda8244:::100000035dc.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:09:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.2580177250000002, 2024-09-16T00:09:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011377063, 2024-09-16T00:09:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.266 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13317 2024-09-16T00:09:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:09:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:09:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918602+0000", 2024-09-16T00:09:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8160000000000004e-06 2024-09-16T00:09:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920364+0000", 2024-09-16T00:09:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017623630000000001 2024-09-16T00:09:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920391+0000", 2024-09-16T00:09:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7248000000000001e-05 2024-09-16T00:09:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920405+0000", 2024-09-16T00:09:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3969e-05 2024-09-16T00:09:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920486+0000", 2024-09-16T00:09:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0737999999999998e-05 2024-09-16T00:09:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920594+0000", 2024-09-16T00:09:57.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108269 2024-09-16T00:09:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920730+0000", 2024-09-16T00:09:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000135229 2024-09-16T00:09:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.928847+0000", 2024-09-16T00:09:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081178299999999995 2024-09-16T00:09:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929862+0000", 2024-09-16T00:09:57.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010147540000000001 2024-09-16T00:09:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929923+0000", 2024-09-16T00:09:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0452999999999999e-05 2024-09-16T00:09:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929975+0000", 2024-09-16T00:09:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2394e-05 2024-09-16T00:09:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:13326 3.9 e88/20 3:92f796cf:::1000000350f.00000000:head v 88'214, mlcod=88'212)", 2024-09-16T00:09:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:09:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.2154969340000008, 2024-09-16T00:09:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01100597, 2024-09-16T00:09:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:09:57.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:09:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961121+0000", 2024-09-16T00:09:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.815e-06 2024-09-16T00:09:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965747+0000", 2024-09-16T00:09:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0046260570000000003 2024-09-16T00:09:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965780+0000", 2024-09-16T00:09:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2851000000000001e-05 2024-09-16T00:09:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965788+0000", 2024-09-16T00:09:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0180000000000001e-06 2024-09-16T00:09:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965859+0000", 2024-09-16T00:09:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1318999999999999e-05 2024-09-16T00:09:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965886+0000", 2024-09-16T00:09:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7509999999999999e-05 2024-09-16T00:09:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.972066+0000", 2024-09-16T00:09:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061800759999999996 2024-09-16T00:09:57.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.972125+0000", 2024-09-16T00:09:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8323999999999999e-05 2024-09-16T00:09:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13942 3.22 3:44ed4521:::100000036d5.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:09:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 7.2541316220000001, 2024-09-16T00:09:57.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011341061, 2024-09-16T00:09:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13942 2024-09-16T00:09:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:09:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:09:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922487+0000", 2024-09-16T00:09:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0180000000000002e-06 2024-09-16T00:09:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923720+0000", 2024-09-16T00:09:57.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012333470000000001 2024-09-16T00:09:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923749+0000", 2024-09-16T00:09:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8748999999999999e-05 2024-09-16T00:09:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923770+0000", 2024-09-16T00:09:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1189999999999999e-05 2024-09-16T00:09:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923868+0000", 2024-09-16T00:09:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7675000000000002e-05 2024-09-16T00:09:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.924047+0000", 2024-09-16T00:09:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017935100000000001 2024-09-16T00:09:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:09:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.924223+0000", 2024-09-16T00:09:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000175661 2024-09-16T00:09:57.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.931544+0000", 2024-09-16T00:09:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073204589999999996 2024-09-16T00:09:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933692+0000", 2024-09-16T00:09:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021488760000000001 2024-09-16T00:09:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933774+0000", 2024-09-16T00:09:57.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1855999999999997e-05 2024-09-16T00:09:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933825+0000", 2024-09-16T00:09:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0878999999999999e-05 2024-09-16T00:09:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14867 3.1f 3:faedf583:::1000000393b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:09:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.4296418859999998, 2024-09-16T00:09:57.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011128677999999999, 2024-09-16T00:09:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14867 2024-09-16T00:09:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:09:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:09:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746976+0000", 2024-09-16T00:09:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1169999999999998e-06 2024-09-16T00:09:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748348+0000", 2024-09-16T00:09:57.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013721860000000001 2024-09-16T00:09:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748371+0000", 2024-09-16T00:09:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2938999999999998e-05 2024-09-16T00:09:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748381+0000", 2024-09-16T00:09:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5300000000000002e-06 2024-09-16T00:09:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748458+0000", 2024-09-16T00:09:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7389000000000001e-05 2024-09-16T00:09:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748626+0000", 2024-09-16T00:09:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016757499999999999 2024-09-16T00:09:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748813+0000", 2024-09-16T00:09:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018743399999999999 2024-09-16T00:09:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.753998+0000", 2024-09-16T00:09:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051851140000000002 2024-09-16T00:09:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.757990+0000", 2024-09-16T00:09:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039914169999999997 2024-09-16T00:09:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.758054+0000", 2024-09-16T00:09:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4513999999999999e-05 2024-09-16T00:09:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.758103+0000", 2024-09-16T00:09:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8463e-05 2024-09-16T00:09:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15003 3.21 3:8673bb5f:::100000030ed.00000000:head [read 2097152~2097152 [2@1395527]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:09:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.008219891, 2024-09-16T00:09:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058433321000000003, 2024-09-16T00:09:57.288 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:09:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15003 2024-09-16T00:09:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:09:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:09:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168397+0000", 2024-09-16T00:09:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3900000000000004e-07 2024-09-16T00:09:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168401+0000", 2024-09-16T00:09:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4050000000000002e-06 2024-09-16T00:09:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168411+0000", 2024-09-16T00:09:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6539999999999994e-06 2024-09-16T00:09:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168417+0000", 2024-09-16T00:09:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6719999999999998e-06 2024-09-16T00:09:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168467+0000", 2024-09-16T00:09:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9815999999999999e-05 2024-09-16T00:09:57.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168535+0000", 2024-09-16T00:09:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7669999999999994e-05 2024-09-16T00:09:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.226829+0000", 2024-09-16T00:09:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058294264999999998 2024-09-16T00:09:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15005 3.1f 3:f8460f10:::10000003370.00000000:head [read 1048576~1048576 [2@4194304]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:09:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.0065391789999998, 2024-09-16T00:09:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.055844088, 2024-09-16T00:09:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:09:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15005 2024-09-16T00:09:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.293 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:09:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:09:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170078+0000", 2024-09-16T00:09:57.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.235e-06 2024-09-16T00:09:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170084+0000", 2024-09-16T00:09:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9739999999999999e-06 2024-09-16T00:09:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225526+0000", 2024-09-16T00:09:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.055441777999999997 2024-09-16T00:09:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225542+0000", 2024-09-16T00:09:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6606999999999999e-05 2024-09-16T00:09:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225668+0000", 2024-09-16T00:09:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012631 2024-09-16T00:09:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225744+0000", 2024-09-16T00:09:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5794e-05 2024-09-16T00:09:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225921+0000", 2024-09-16T00:09:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017639000000000001 2024-09-16T00:09:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.297 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.297 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:57.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.179+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdac002da00 con 0x7fdac02048f0 2024-09-16T00:09:57.298 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:57.299 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:57.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.179+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=92}) -- 0x7fdac0209ab0 con 0x7fdac006cea0 2024-09-16T00:09:57.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.179+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdac002da00 con 0x7fdac006cea0 2024-09-16T00:09:57.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.179+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fdab40597c0 con 0x7fdac006cea0 2024-09-16T00:09:57.299 INFO:teuthology.orchestra.run.smithi139.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-09-16T00:09:57.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.179+0000 7fdac6a96640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fdac0207af0 0x7fdac0207f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:57.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.179+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fdac0209790 con 0x7fdac0207af0 2024-09-16T00:09:57.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.180+0000 7fdac5033640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fdac0207af0 0x7fdac0207f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:57.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.180+0000 7fdac5033640 1 --2- 172.21.15.139:0/2740265313 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fdac0207af0 0x7fdac0207f40 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:57.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.181+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdac0209790 con 0x7fdac0207af0 2024-09-16T00:09:57.300 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:57.298+0000 7f5a07e2c640 -1 osd.6 82 log_to_monitors true 2024-09-16T00:09:57.301 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:09:57.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.195+0000 7fdac6a96640 1 -- 172.21.15.139:0/2740265313 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdac02096e0 con 0x7fdac0207af0 2024-09-16T00:09:57.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.197+0000 7fdabd7fa640 1 -- 172.21.15.139:0/2740265313 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdac02096e0 con 0x7fdac0207af0 2024-09-16T00:09:57.302 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:57.303 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T00:09:57.303 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:57.303 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T00:09:57.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.251+0000 7f25cf7fe640 1 -- 172.21.15.139:0/4153538241 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v92) ==== 97+0+0 (secure 0 0 0) 0x7f25e814fe70 con 0x7f25e80c8110 2024-09-16T00:09:57.304 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is set 2024-09-16T00:09:57.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.253+0000 7f25ed60e640 1 -- 172.21.15.139:0/4153538241 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f25d406c2b0 msgr2=0x7f25d406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:57.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.253+0000 7f25ed60e640 1 --2- 172.21.15.139:0/4153538241 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f25d406c2b0 0x7f25d406e770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f25d8034000 tx=0x7f25d802e000 comp rx=0 tx=0).stop 2024-09-16T00:09:57.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.253+0000 7f25ed60e640 1 -- 172.21.15.139:0/4153538241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25e80c8110 msgr2=0x7f25e81e4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:57.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.253+0000 7f25ed60e640 1 --2- 172.21.15.139:0/4153538241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25e80c8110 0x7f25e81e4f40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00c490 tx=0x7f25dc00c950 comp rx=0 tx=0).stop 2024-09-16T00:09:57.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.254+0000 7f25ed60e640 1 -- 172.21.15.139:0/4153538241 shutdown_connections 2024-09-16T00:09:57.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.254+0000 7f25ed60e640 1 --2- 172.21.15.139:0/4153538241 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f25d406c2b0 0x7f25d406e770 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:57.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.254+0000 7f25ed60e640 1 --2- 172.21.15.139:0/4153538241 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f25e8154da0 0x7f25e81e5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:57.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.254+0000 7f25ed60e640 1 --2- 172.21.15.139:0/4153538241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25e80c8110 0x7f25e81e4f40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:57.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.254+0000 7f25ed60e640 1 --2- 172.21.15.139:0/4153538241 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f25e80c7d30 0x7f25e815cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:57.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.254+0000 7f25ed60e640 1 -- 172.21.15.139:0/4153538241 >> 172.21.15.139:0/4153538241 conn(0x7f25e80c91d0 msgr2=0x7f25e80ca6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:57.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.254+0000 7f25ed60e640 1 -- 172.21.15.139:0/4153538241 shutdown_connections 2024-09-16T00:09:57.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:57.254+0000 7f25ed60e640 1 -- 172.21.15.139:0/4153538241 wait complete. 2024-09-16T00:09:57.306 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:57.300+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:57.402 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:57.401+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:57.453 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:57.453 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:57.453 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:09:57.453 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:57.463 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:09:57.505 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:57.504+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:57.604 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:57.604+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:57.617 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:57.617 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:09:57.617 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:09:57.617 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:09:57.618 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:57.626 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:09:57.705 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:57.705+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:57.742 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:09:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:09:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:09:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:09:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:89 2.1b 2:ddb4e4d7:::200.00000002:head [write 2591630~1401 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 54.091156564999999, 2024-09-16T00:09:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011802609, 2024-09-16T00:09:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.743 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:09:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649081+0000", 2024-09-16T00:09:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7100000000000002e-07 2024-09-16T00:09:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649089+0000", 2024-09-16T00:09:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7640000000000005e-06 2024-09-16T00:09:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649094+0000", 2024-09-16T00:09:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6219999999999997e-06 2024-09-16T00:09:57.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649099+0000", 2024-09-16T00:09:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6970000000000001e-06 2024-09-16T00:09:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655288+0000", 2024-09-16T00:09:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061885569999999999 2024-09-16T00:09:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655365+0000", 2024-09-16T00:09:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7723000000000001e-05 2024-09-16T00:09:57.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655483+0000", 2024-09-16T00:09:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117895 2024-09-16T00:09:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657758+0000", 2024-09-16T00:09:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022752369999999998 2024-09-16T00:09:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660803+0000", 2024-09-16T00:09:57.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030441499999999998 2024-09-16T00:09:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660850+0000", 2024-09-16T00:09:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7017000000000001e-05 2024-09-16T00:09:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660883+0000", 2024-09-16T00:09:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3476000000000003e-05 2024-09-16T00:09:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.748 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:90 2.1b 2:ddb4e4d7:::200.00000002:head [write 2593031~1404 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:09:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 54.091118659999999, 2024-09-16T00:09:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011882886, 2024-09-16T00:09:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:09:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649119+0000", 2024-09-16T00:09:57.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:09:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649125+0000", 2024-09-16T00:09:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1650000000000003e-06 2024-09-16T00:09:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649134+0000", 2024-09-16T00:09:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5669999999999995e-06 2024-09-16T00:09:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649141+0000", 2024-09-16T00:09:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9040000000000003e-06 2024-09-16T00:09:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655796+0000", 2024-09-16T00:09:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066553649999999999 2024-09-16T00:09:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655868+0000", 2024-09-16T00:09:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2219999999999996e-05 2024-09-16T00:09:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655986+0000", 2024-09-16T00:09:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117457 2024-09-16T00:09:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657794+0000", 2024-09-16T00:09:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001807894 2024-09-16T00:09:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660890+0000", 2024-09-16T00:09:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030958449999999998 2024-09-16T00:09:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660966+0000", 2024-09-16T00:09:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6382000000000002e-05 2024-09-16T00:09:57.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.661001+0000", 2024-09-16T00:09:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5349000000000003e-05 2024-09-16T00:09:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10851 3.29 3:9786b8f5:::10000003100.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:09:57.755 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.847454616, 2024-09-16T00:09:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012109089999999999, 2024-09-16T00:09:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10851 2024-09-16T00:09:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:09:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:09:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892786+0000", 2024-09-16T00:09:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4819999999999999e-06 2024-09-16T00:09:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894302+0000", 2024-09-16T00:09:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015156099999999999 2024-09-16T00:09:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894319+0000", 2024-09-16T00:09:57.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7098000000000001e-05 2024-09-16T00:09:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894329+0000", 2024-09-16T00:09:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9180000000000006e-06 2024-09-16T00:09:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894394+0000", 2024-09-16T00:09:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5797999999999996e-05 2024-09-16T00:09:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894532+0000", 2024-09-16T00:09:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000138128 2024-09-16T00:09:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894633+0000", 2024-09-16T00:09:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010037100000000001 2024-09-16T00:09:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.899212+0000", 2024-09-16T00:09:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045789369999999999 2024-09-16T00:09:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904771+0000", 2024-09-16T00:09:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055591169999999997 2024-09-16T00:09:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904838+0000", 2024-09-16T00:09:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7124000000000003e-05 2024-09-16T00:09:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904892+0000", 2024-09-16T00:09:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3507000000000001e-05 2024-09-16T00:09:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.762 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10957 3.24 3:2748b9d7:::1000000313b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:09:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.225550262999999, 2024-09-16T00:09:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012163525, 2024-09-16T00:09:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10957 2024-09-16T00:09:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.763 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:09:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:09:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514690+0000", 2024-09-16T00:09:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.382e-06 2024-09-16T00:09:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.764 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516204+0000", 2024-09-16T00:09:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001513831 2024-09-16T00:09:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516232+0000", 2024-09-16T00:09:57.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7657999999999998e-05 2024-09-16T00:09:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516246+0000", 2024-09-16T00:09:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3953e-05 2024-09-16T00:09:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516374+0000", 2024-09-16T00:09:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012882 2024-09-16T00:09:57.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516588+0000", 2024-09-16T00:09:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000213439 2024-09-16T00:09:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516748+0000", 2024-09-16T00:09:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015971900000000001 2024-09-16T00:09:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.524102+0000", 2024-09-16T00:09:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073542499999999997 2024-09-16T00:09:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526714+0000", 2024-09-16T00:09:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026122910000000001 2024-09-16T00:09:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526791+0000", 2024-09-16T00:09:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6997999999999997e-05 2024-09-16T00:09:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.770 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526850+0000", 2024-09-16T00:09:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9184000000000001e-05 2024-09-16T00:09:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11098 3.15 e86/20 3:aa8f498f:::100000030ba.00000000:head v 86'170, mlcod=86'168)", 2024-09-16T00:09:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:09:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 17.718767097000001, 2024-09-16T00:09:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013832641999999999, 2024-09-16T00:09:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.771 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:09:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:09:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021472+0000", 2024-09-16T00:09:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9489999999999999e-06 2024-09-16T00:09:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027628+0000", 2024-09-16T00:09:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061563859999999998 2024-09-16T00:09:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027645+0000", 2024-09-16T00:09:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7108e-05 2024-09-16T00:09:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027655+0000", 2024-09-16T00:09:57.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7999999999999993e-06 2024-09-16T00:09:57.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027712+0000", 2024-09-16T00:09:57.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6826000000000001e-05 2024-09-16T00:09:57.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027738+0000", 2024-09-16T00:09:57.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5601000000000001e-05 2024-09-16T00:09:57.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.775 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.775 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.035237+0000", 2024-09-16T00:09:57.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074998699999999996 2024-09-16T00:09:57.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.775 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.775 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.035303+0000", 2024-09-16T00:09:57.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5102000000000001e-05 2024-09-16T00:09:57.775 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.775 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.775 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.776 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11099 3.b 3:d2f5b389:::100000030e7.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:57.776 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:09:57.776 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 17.714722282, 2024-09-16T00:09:57.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012707456000000001, 2024-09-16T00:09:57.776 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.776 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.776 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.776 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.776 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.776 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.776 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11099 2024-09-16T00:09:57.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.777 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:09:57.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:09:57.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025517+0000", 2024-09-16T00:09:57.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:09:57.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026955+0000", 2024-09-16T00:09:57.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001437219 2024-09-16T00:09:57.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026982+0000", 2024-09-16T00:09:57.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7523999999999999e-05 2024-09-16T00:09:57.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026996+0000", 2024-09-16T00:09:57.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4379000000000001e-05 2024-09-16T00:09:57.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027076+0000", 2024-09-16T00:09:57.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9933999999999997e-05 2024-09-16T00:09:57.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027185+0000", 2024-09-16T00:09:57.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010875899999999999 2024-09-16T00:09:57.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:57.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027380+0000", 2024-09-16T00:09:57.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019477700000000001 2024-09-16T00:09:57.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038072+0000", 2024-09-16T00:09:57.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010692234 2024-09-16T00:09:57.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038115+0000", 2024-09-16T00:09:57.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3291999999999998e-05 2024-09-16T00:09:57.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038172+0000", 2024-09-16T00:09:57.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6981000000000002e-05 2024-09-16T00:09:57.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038222+0000", 2024-09-16T00:09:57.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9737000000000002e-05 2024-09-16T00:09:57.782 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.782 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.782 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.782 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.782 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11109 3.27 3:e5179c63:::10000003176.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:57.782 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:09:57.782 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 17.669257998999999, 2024-09-16T00:09:57.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012250990999999999, 2024-09-16T00:09:57.782 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.782 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.782 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.782 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.782 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.783 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.783 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11109 2024-09-16T00:09:57.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.783 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:09:57.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:09:57.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070981+0000", 2024-09-16T00:09:57.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.289e-06 2024-09-16T00:09:57.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072616+0000", 2024-09-16T00:09:57.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016347569999999999 2024-09-16T00:09:57.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072638+0000", 2024-09-16T00:09:57.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1512999999999999e-05 2024-09-16T00:09:57.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072649+0000", 2024-09-16T00:09:57.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1626e-05 2024-09-16T00:09:57.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072775+0000", 2024-09-16T00:09:57.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012553900000000001 2024-09-16T00:09:57.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.786 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.786 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072856+0000", 2024-09-16T00:09:57.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0882000000000003e-05 2024-09-16T00:09:57.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.786 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:57.786 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072990+0000", 2024-09-16T00:09:57.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013483599999999999 2024-09-16T00:09:57.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.786 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.786 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.078354+0000", 2024-09-16T00:09:57.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053639500000000001 2024-09-16T00:09:57.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083150+0000", 2024-09-16T00:09:57.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004795517 2024-09-16T00:09:57.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083186+0000", 2024-09-16T00:09:57.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6019000000000002e-05 2024-09-16T00:09:57.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.788 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083230+0000", 2024-09-16T00:09:57.788 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4063000000000001e-05 2024-09-16T00:09:57.788 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.788 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.788 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.788 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.788 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11234 3.b 3:d2f5b389:::100000030e7.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:57.788 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:09:57.788 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 17.116642231, 2024-09-16T00:09:57.788 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.015217661, 2024-09-16T00:09:57.788 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.788 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.789 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.789 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.789 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.789 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.789 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11234 2024-09-16T00:09:57.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.789 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.789 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.789 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:09:57.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:09:57.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623598+0000", 2024-09-16T00:09:57.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1580000000000001e-06 2024-09-16T00:09:57.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626844+0000", 2024-09-16T00:09:57.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032459670000000002 2024-09-16T00:09:57.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626868+0000", 2024-09-16T00:09:57.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3584e-05 2024-09-16T00:09:57.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626879+0000", 2024-09-16T00:09:57.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1545999999999999e-05 2024-09-16T00:09:57.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626950+0000", 2024-09-16T00:09:57.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0784999999999994e-05 2024-09-16T00:09:57.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.627051+0000", 2024-09-16T00:09:57.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010155899999999999 2024-09-16T00:09:57.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:57.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.627178+0000", 2024-09-16T00:09:57.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012614899999999999 2024-09-16T00:09:57.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.635220+0000", 2024-09-16T00:09:57.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0080421490000000002 2024-09-16T00:09:57.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638699+0000", 2024-09-16T00:09:57.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034797500000000002 2024-09-16T00:09:57.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638760+0000", 2024-09-16T00:09:57.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0319000000000003e-05 2024-09-16T00:09:57.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638812+0000", 2024-09-16T00:09:57.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:09:57.794 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.794 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.795 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.795 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.795 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.795 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11777 3.b 3:d342b4ff:::10000003113.00000000:head [write 0~4194304 [2@1171070]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:57.795 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:09:57.795 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.549002823, 2024-09-16T00:09:57.795 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013079176, 2024-09-16T00:09:57.795 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.795 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.795 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.795 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.795 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.795 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.796 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11777 2024-09-16T00:09:57.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.796 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:09:57.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:09:57.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191237+0000", 2024-09-16T00:09:57.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3369999999999998e-06 2024-09-16T00:09:57.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192825+0000", 2024-09-16T00:09:57.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015887189999999999 2024-09-16T00:09:57.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192846+0000", 2024-09-16T00:09:57.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0670999999999999e-05 2024-09-16T00:09:57.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192858+0000", 2024-09-16T00:09:57.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.223e-05 2024-09-16T00:09:57.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192957+0000", 2024-09-16T00:09:57.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8416000000000001e-05 2024-09-16T00:09:57.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.193031+0000", 2024-09-16T00:09:57.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4381999999999994e-05 2024-09-16T00:09:57.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:57.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.193164+0000", 2024-09-16T00:09:57.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013349100000000001 2024-09-16T00:09:57.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204144+0000", 2024-09-16T00:09:57.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010979917 2024-09-16T00:09:57.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.800 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204204+0000", 2024-09-16T00:09:57.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0025999999999998e-05 2024-09-16T00:09:57.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.800 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204265+0000", 2024-09-16T00:09:57.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0645e-05 2024-09-16T00:09:57.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.801 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.801 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204313+0000", 2024-09-16T00:09:57.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8341999999999999e-05 2024-09-16T00:09:57.801 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.801 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.801 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.801 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.801 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.801 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11981 3.c e87/20 3:33faa5a0:::10000003215.00000000:head v 87'186, mlcod=87'184)", 2024-09-16T00:09:57.801 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:09:57.801 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 13.908768, 2024-09-16T00:09:57.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011828697000000001, 2024-09-16T00:09:57.801 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.802 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.802 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.802 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:09:57.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:09:57.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831471+0000", 2024-09-16T00:09:57.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6220000000000001e-06 2024-09-16T00:09:57.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836584+0000", 2024-09-16T00:09:57.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051133510000000004 2024-09-16T00:09:57.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836606+0000", 2024-09-16T00:09:57.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2079e-05 2024-09-16T00:09:57.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836624+0000", 2024-09-16T00:09:57.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8162e-05 2024-09-16T00:09:57.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836704+0000", 2024-09-16T00:09:57.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9867e-05 2024-09-16T00:09:57.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836730+0000", 2024-09-16T00:09:57.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6285999999999999e-05 2024-09-16T00:09:57.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.843247+0000", 2024-09-16T00:09:57.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0065165650000000002 2024-09-16T00:09:57.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.843298+0000", 2024-09-16T00:09:57.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0764999999999999e-05 2024-09-16T00:09:57.806 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.806 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.806 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.806 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13365 3.2 3:4298088d:::100000035be.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:57.806 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:09:57.806 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.7210660210000004, 2024-09-16T00:09:57.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014036442999999999, 2024-09-16T00:09:57.807 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.807 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.807 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.807 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.807 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.807 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.807 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13365 2024-09-16T00:09:57.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.807 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.807 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:09:57.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:09:57.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019174+0000", 2024-09-16T00:09:57.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1250000000000001e-06 2024-09-16T00:09:57.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020613+0000", 2024-09-16T00:09:57.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014391930000000001 2024-09-16T00:09:57.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020638+0000", 2024-09-16T00:09:57.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4657000000000001e-05 2024-09-16T00:09:57.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020651+0000", 2024-09-16T00:09:57.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3451e-05 2024-09-16T00:09:57.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020744+0000", 2024-09-16T00:09:57.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3010000000000006e-05 2024-09-16T00:09:57.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020912+0000", 2024-09-16T00:09:57.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016763800000000001 2024-09-16T00:09:57.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:09:57.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.021043+0000", 2024-09-16T00:09:57.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013099800000000001 2024-09-16T00:09:57.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026194+0000", 2024-09-16T00:09:57.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051510879999999998 2024-09-16T00:09:57.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033132+0000", 2024-09-16T00:09:57.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069377279999999998 2024-09-16T00:09:57.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033169+0000", 2024-09-16T00:09:57.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6897000000000003e-05 2024-09-16T00:09:57.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033207+0000", 2024-09-16T00:09:57.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658000000000002e-05 2024-09-16T00:09:57.812 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.812 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.812 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.813 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13368 3.27 3:e75b72a1:::100000032e8.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:57.813 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:09:57.813 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.7137001380000001, 2024-09-16T00:09:57.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013615175, 2024-09-16T00:09:57.813 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.813 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.813 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.813 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.813 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.813 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.813 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13368 2024-09-16T00:09:57.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.813 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:09:57.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:09:57.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026539+0000", 2024-09-16T00:09:57.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.798e-06 2024-09-16T00:09:57.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027825+0000", 2024-09-16T00:09:57.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012861839999999999 2024-09-16T00:09:57.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027850+0000", 2024-09-16T00:09:57.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5048000000000001e-05 2024-09-16T00:09:57.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027865+0000", 2024-09-16T00:09:57.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4939e-05 2024-09-16T00:09:57.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027944+0000", 2024-09-16T00:09:57.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8881999999999995e-05 2024-09-16T00:09:57.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.028043+0000", 2024-09-16T00:09:57.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9135000000000005e-05 2024-09-16T00:09:57.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:57.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.028158+0000", 2024-09-16T00:09:57.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011527999999999999 2024-09-16T00:09:57.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.039292+0000", 2024-09-16T00:09:57.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011133568999999999 2024-09-16T00:09:57.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.039948+0000", 2024-09-16T00:09:57.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00065625700000000004 2024-09-16T00:09:57.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.818 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.818 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.040061+0000", 2024-09-16T00:09:57.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011284400000000001 2024-09-16T00:09:57.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.818 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.818 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.818 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.040152+0000", 2024-09-16T00:09:57.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1238999999999998e-05 2024-09-16T00:09:57.818 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.818 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.818 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.818 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:13366 3.15 e88/20 3:aa8f498f:::100000030ba.00000000:head v 88'196, mlcod=88'194)", 2024-09-16T00:09:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:09:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.7102750629999992, 2024-09-16T00:09:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012822817, 2024-09-16T00:09:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:09:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.819 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:09:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029963+0000", 2024-09-16T00:09:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4270000000000001e-06 2024-09-16T00:09:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034078+0000", 2024-09-16T00:09:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041148360000000002 2024-09-16T00:09:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034094+0000", 2024-09-16T00:09:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.535e-05 2024-09-16T00:09:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034103+0000", 2024-09-16T00:09:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:09:57.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.036791+0000", 2024-09-16T00:09:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026878340000000001 2024-09-16T00:09:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.036818+0000", 2024-09-16T00:09:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6885000000000001e-05 2024-09-16T00:09:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.042713+0000", 2024-09-16T00:09:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058950399999999998 2024-09-16T00:09:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.042785+0000", 2024-09-16T00:09:57.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1927000000000004e-05 2024-09-16T00:09:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13405 3.27 3:e708eda4:::100000034c4.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:09:57.824 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 9.5840733500000006, 2024-09-16T00:09:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012003333, 2024-09-16T00:09:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13405 2024-09-16T00:09:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:09:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:09:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156166+0000", 2024-09-16T00:09:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6230000000000001e-06 2024-09-16T00:09:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157868+0000", 2024-09-16T00:09:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017017 2024-09-16T00:09:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157891+0000", 2024-09-16T00:09:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2931000000000001e-05 2024-09-16T00:09:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157904+0000", 2024-09-16T00:09:57.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2763e-05 2024-09-16T00:09:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157988+0000", 2024-09-16T00:09:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3849000000000003e-05 2024-09-16T00:09:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.158083+0000", 2024-09-16T00:09:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5939000000000004e-05 2024-09-16T00:09:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:57.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.158194+0000", 2024-09-16T00:09:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110963 2024-09-16T00:09:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.164635+0000", 2024-09-16T00:09:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0064405570000000004 2024-09-16T00:09:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168088+0000", 2024-09-16T00:09:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034534639999999998 2024-09-16T00:09:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168140+0000", 2024-09-16T00:09:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1041000000000001e-05 2024-09-16T00:09:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168167+0000", 2024-09-16T00:09:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7503000000000001e-05 2024-09-16T00:09:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13685 3.b 3:d26c1580:::100000032e4.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:57.830 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:09:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.4053863849999999, 2024-09-16T00:09:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014868124999999999, 2024-09-16T00:09:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13685 2024-09-16T00:09:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:09:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:09:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334854+0000", 2024-09-16T00:09:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0000000000000001e-06 2024-09-16T00:09:57.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337155+0000", 2024-09-16T00:09:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023016870000000002 2024-09-16T00:09:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337178+0000", 2024-09-16T00:09:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2898e-05 2024-09-16T00:09:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337190+0000", 2024-09-16T00:09:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1817999999999999e-05 2024-09-16T00:09:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337301+0000", 2024-09-16T00:09:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110951 2024-09-16T00:09:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337403+0000", 2024-09-16T00:09:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000102118 2024-09-16T00:09:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:57.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337545+0000", 2024-09-16T00:09:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014167199999999999 2024-09-16T00:09:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.346336+0000", 2024-09-16T00:09:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087911450000000002 2024-09-16T00:09:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349641+0000", 2024-09-16T00:09:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033047570000000002 2024-09-16T00:09:57.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349680+0000", 2024-09-16T00:09:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9054999999999998e-05 2024-09-16T00:09:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349719+0000", 2024-09-16T00:09:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9023999999999998e-05 2024-09-16T00:09:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14624 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:09:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 5.1256271839999998, 2024-09-16T00:09:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012123376, 2024-09-16T00:09:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14624 2024-09-16T00:09:57.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:09:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:09:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614614+0000", 2024-09-16T00:09:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9500000000000003e-06 2024-09-16T00:09:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616480+0000", 2024-09-16T00:09:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018663169999999999 2024-09-16T00:09:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616506+0000", 2024-09-16T00:09:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6205999999999999e-05 2024-09-16T00:09:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616517+0000", 2024-09-16T00:09:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0399e-05 2024-09-16T00:09:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616595+0000", 2024-09-16T00:09:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8170999999999994e-05 2024-09-16T00:09:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616683+0000", 2024-09-16T00:09:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8578999999999993e-05 2024-09-16T00:09:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616825+0000", 2024-09-16T00:09:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014186299999999999 2024-09-16T00:09:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.624948+0000", 2024-09-16T00:09:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081230420000000005 2024-09-16T00:09:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626617+0000", 2024-09-16T00:09:57.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016681980000000001 2024-09-16T00:09:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626678+0000", 2024-09-16T00:09:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1362e-05 2024-09-16T00:09:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626733+0000", 2024-09-16T00:09:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5288999999999999e-05 2024-09-16T00:09:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14672 3.29 3:9641a85f:::1000000360e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:09:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.9211700069999997, 2024-09-16T00:09:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012258025000000001, 2024-09-16T00:09:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.843 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14672 2024-09-16T00:09:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:09:57.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:09:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819070+0000", 2024-09-16T00:09:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0589999999999998e-06 2024-09-16T00:09:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821701+0000", 2024-09-16T00:09:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026309100000000002 2024-09-16T00:09:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821727+0000", 2024-09-16T00:09:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6372000000000001e-05 2024-09-16T00:09:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821738+0000", 2024-09-16T00:09:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0759e-05 2024-09-16T00:09:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821821+0000", 2024-09-16T00:09:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2854000000000003e-05 2024-09-16T00:09:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821920+0000", 2024-09-16T00:09:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8900999999999994e-05 2024-09-16T00:09:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.822084+0000", 2024-09-16T00:09:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016432799999999999 2024-09-16T00:09:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.829148+0000", 2024-09-16T00:09:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0070641460000000003 2024-09-16T00:09:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831242+0000", 2024-09-16T00:09:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002093293 2024-09-16T00:09:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831280+0000", 2024-09-16T00:09:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8707000000000001e-05 2024-09-16T00:09:57.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831325+0000", 2024-09-16T00:09:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4696e-05 2024-09-16T00:09:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14829 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304 [2@2556691]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:09:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.1686505890000003, 2024-09-16T00:09:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014399683, 2024-09-16T00:09:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14829 2024-09-16T00:09:57.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:09:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:09:57.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571590+0000", 2024-09-16T00:09:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:09:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573133+0000", 2024-09-16T00:09:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015422109999999999 2024-09-16T00:09:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573161+0000", 2024-09-16T00:09:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8045e-05 2024-09-16T00:09:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573174+0000", 2024-09-16T00:09:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3193999999999999e-05 2024-09-16T00:09:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573365+0000", 2024-09-16T00:09:57.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019096400000000001 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573467+0000", 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101789 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573610+0000", 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143404 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.583614+0000", 2024-09-16T00:09:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010004422000000001 2024-09-16T00:09:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585556+0000", 2024-09-16T00:09:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019414009999999999 2024-09-16T00:09:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585939+0000", 2024-09-16T00:09:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038340999999999999 2024-09-16T00:09:57.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585986+0000", 2024-09-16T00:09:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6924e-05 2024-09-16T00:09:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14914 3.b 3:d23f6103:::10000003277.00000000:head [write 0~4194304 [3@58131]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:09:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.8574468180000001, 2024-09-16T00:09:57.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013007654, 2024-09-16T00:09:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14914 2024-09-16T00:09:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:09:57.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:09:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882793+0000", 2024-09-16T00:09:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.65e-06 2024-09-16T00:09:57.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885193+0000", 2024-09-16T00:09:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024003449999999999 2024-09-16T00:09:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885214+0000", 2024-09-16T00:09:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0398e-05 2024-09-16T00:09:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885224+0000", 2024-09-16T00:09:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0253e-05 2024-09-16T00:09:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885299+0000", 2024-09-16T00:09:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5434999999999999e-05 2024-09-16T00:09:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885389+0000", 2024-09-16T00:09:57.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9209999999999995e-05 2024-09-16T00:09:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885485+0000", 2024-09-16T00:09:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6218999999999995e-05 2024-09-16T00:09:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.893457+0000", 2024-09-16T00:09:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0079722009999999999 2024-09-16T00:09:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895681+0000", 2024-09-16T00:09:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022241449999999999 2024-09-16T00:09:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895732+0000", 2024-09-16T00:09:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1269e-05 2024-09-16T00:09:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895798+0000", 2024-09-16T00:09:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5529000000000002e-05 2024-09-16T00:09:57.863 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15058 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304 [2@2556691]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-16T00:09:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:09:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 3.4050782239999999, 2024-09-16T00:09:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012275842, 2024-09-16T00:09:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:09:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:09:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:09:57.864 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:09:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15058 2024-09-16T00:09:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:09:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:09:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:09:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:09:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:09:57.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:09:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:09:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335161+0000", 2024-09-16T00:09:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.367e-06 2024-09-16T00:09:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:09:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337545+0000", 2024-09-16T00:09:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023839040000000001 2024-09-16T00:09:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:09:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337572+0000", 2024-09-16T00:09:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.673e-05 2024-09-16T00:09:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:09:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337583+0000", 2024-09-16T00:09:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1048e-05 2024-09-16T00:09:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:09:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337660+0000", 2024-09-16T00:09:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7459999999999994e-05 2024-09-16T00:09:57.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:09:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337770+0000", 2024-09-16T00:09:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109634 2024-09-16T00:09:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337926+0000", 2024-09-16T00:09:57.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015601999999999999 2024-09-16T00:09:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.345689+0000", 2024-09-16T00:09:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0077625630000000001 2024-09-16T00:09:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:09:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347324+0000", 2024-09-16T00:09:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016351479999999999 2024-09-16T00:09:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:09:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347387+0000", 2024-09-16T00:09:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3199000000000003e-05 2024-09-16T00:09:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:09:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:09:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:09:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347435+0000", 2024-09-16T00:09:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7769000000000003e-05 2024-09-16T00:09:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:09:57.870 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:09:57.870 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:09:57.872 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:09:57.873 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:57.871+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:57.973 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:57.973+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:57.983 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:57.984 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:57.984 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:57.984 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:57.993 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:09:58.077 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:58.077+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:58.156 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:58.156 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:58.157 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:58.157 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:58.157 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:58.167 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:09:58.174 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:09:58.174+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:09:58.273 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:09:58.272+0000 7f59fb25a640 -1 osd.6 82 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:09:58.276 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:58.276+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:58.340 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:58.340 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:58.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:58.340 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:58.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.340 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10780 3.a 3:500c83b2:::100000030b4.00000000:head [write 0~4194304 [3@2137521]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:58.341 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:09:58.341 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.884643481000001, 2024-09-16T00:09:58.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015526478, 2024-09-16T00:09:58.341 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.341 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.341 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.341 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.341 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10780 2024-09-16T00:09:58.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.341 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:09:58.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:09:58.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452917+0000", 2024-09-16T00:09:58.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2180000000000002e-06 2024-09-16T00:09:58.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462177+0000", 2024-09-16T00:09:58.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092593769999999992 2024-09-16T00:09:58.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462196+0000", 2024-09-16T00:09:58.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9745000000000002e-05 2024-09-16T00:09:58.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462207+0000", 2024-09-16T00:09:58.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0524e-05 2024-09-16T00:09:58.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462284+0000", 2024-09-16T00:09:58.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7608000000000006e-05 2024-09-16T00:09:58.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462376+0000", 2024-09-16T00:09:58.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1811999999999994e-05 2024-09-16T00:09:58.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468357+0000", 2024-09-16T00:09:58.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059806809999999998 2024-09-16T00:09:58.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468386+0000", 2024-09-16T00:09:58.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9068999999999999e-05 2024-09-16T00:09:58.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468441+0000", 2024-09-16T00:09:58.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5444e-05 2024-09-16T00:09:58.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.346 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.346 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10831 3.1a 3:59b8f98c:::100000030df.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:09:58.346 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:09:58.347 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.561836345, 2024-09-16T00:09:58.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014692843000000001, 2024-09-16T00:09:58.347 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.347 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.347 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.347 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.347 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10831 2024-09-16T00:09:58.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.348 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:09:58.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:09:58.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775723+0000", 2024-09-16T00:09:58.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.367e-06 2024-09-16T00:09:58.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780402+0000", 2024-09-16T00:09:58.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046785769999999997 2024-09-16T00:09:58.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780421+0000", 2024-09-16T00:09:58.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9179000000000001e-05 2024-09-16T00:09:58.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780440+0000", 2024-09-16T00:09:58.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9015e-05 2024-09-16T00:09:58.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780533+0000", 2024-09-16T00:09:58.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3197999999999998e-05 2024-09-16T00:09:58.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780618+0000", 2024-09-16T00:09:58.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4787e-05 2024-09-16T00:09:58.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:58.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780748+0000", 2024-09-16T00:09:58.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012927199999999999 2024-09-16T00:09:58.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.785049+0000", 2024-09-16T00:09:58.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043015750000000002 2024-09-16T00:09:58.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790312+0000", 2024-09-16T00:09:58.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052633319999999999 2024-09-16T00:09:58.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790369+0000", 2024-09-16T00:09:58.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6848999999999997e-05 2024-09-16T00:09:58.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790415+0000", 2024-09-16T00:09:58.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5692000000000002e-05 2024-09-16T00:09:58.353 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.353 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.353 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.353 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10915 3.1a 3:5a6e15a2:::1000000318e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:58.353 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:09:58.353 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 19.122330424000001, 2024-09-16T00:09:58.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014599472, 2024-09-16T00:09:58.353 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.353 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.353 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.353 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.354 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.354 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.354 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10915 2024-09-16T00:09:58.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.354 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:09:58.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:09:58.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215230+0000", 2024-09-16T00:09:58.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0090000000000002e-06 2024-09-16T00:09:58.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219556+0000", 2024-09-16T00:09:58.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043259539999999999 2024-09-16T00:09:58.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219577+0000", 2024-09-16T00:09:58.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1271999999999999e-05 2024-09-16T00:09:58.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219589+0000", 2024-09-16T00:09:58.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1766000000000001e-05 2024-09-16T00:09:58.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219663+0000", 2024-09-16T00:09:58.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4196999999999995e-05 2024-09-16T00:09:58.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219733+0000", 2024-09-16T00:09:58.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9553000000000003e-05 2024-09-16T00:09:58.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:58.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219827+0000", 2024-09-16T00:09:58.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4395999999999995e-05 2024-09-16T00:09:58.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.225724+0000", 2024-09-16T00:09:58.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058968359999999999 2024-09-16T00:09:58.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229705+0000", 2024-09-16T00:09:58.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003981409 2024-09-16T00:09:58.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229772+0000", 2024-09-16T00:09:58.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6137999999999996e-05 2024-09-16T00:09:58.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229828+0000", 2024-09-16T00:09:58.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5942000000000001e-05 2024-09-16T00:09:58.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.359 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.359 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.359 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11074 3.14 3:2834b7ee:::10000003131.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:58.359 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:09:58.359 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.424790301000002, 2024-09-16T00:09:58.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015588350000000001, 2024-09-16T00:09:58.359 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.360 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.360 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.360 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.360 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11074 2024-09-16T00:09:58.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.360 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:09:58.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:09:58.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912769+0000", 2024-09-16T00:09:58.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4600000000000004e-07 2024-09-16T00:09:58.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918264+0000", 2024-09-16T00:09:58.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054954890000000001 2024-09-16T00:09:58.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918281+0000", 2024-09-16T00:09:58.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6160000000000001e-05 2024-09-16T00:09:58.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918289+0000", 2024-09-16T00:09:58.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1650000000000006e-06 2024-09-16T00:09:58.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918386+0000", 2024-09-16T00:09:58.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7288999999999996e-05 2024-09-16T00:09:58.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918535+0000", 2024-09-16T00:09:58.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014876200000000001 2024-09-16T00:09:58.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:09:58.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918639+0000", 2024-09-16T00:09:58.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010470400000000001 2024-09-16T00:09:58.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.924703+0000", 2024-09-16T00:09:58.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060639759999999996 2024-09-16T00:09:58.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928246+0000", 2024-09-16T00:09:58.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035421720000000001 2024-09-16T00:09:58.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928305+0000", 2024-09-16T00:09:58.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9014000000000001e-05 2024-09-16T00:09:58.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928357+0000", 2024-09-16T00:09:58.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1873000000000003e-05 2024-09-16T00:09:58.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.366 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.366 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.366 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11111 3.9 3:93301cc4:::10000003210.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:09:58.366 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:09:58.366 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.248052098999999, 2024-09-16T00:09:58.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014336099999999999, 2024-09-16T00:09:58.366 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.366 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.366 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.366 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11111 2024-09-16T00:09:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:09:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:09:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089520+0000", 2024-09-16T00:09:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3903000000000001e-05 2024-09-16T00:09:58.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094463+0000", 2024-09-16T00:09:58.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049423000000000002 2024-09-16T00:09:58.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094494+0000", 2024-09-16T00:09:58.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1029999999999999e-05 2024-09-16T00:09:58.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094503+0000", 2024-09-16T00:09:58.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0180000000000007e-06 2024-09-16T00:09:58.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094573+0000", 2024-09-16T00:09:58.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0457000000000007e-05 2024-09-16T00:09:58.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094737+0000", 2024-09-16T00:09:58.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016408699999999999 2024-09-16T00:09:58.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:58.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094897+0000", 2024-09-16T00:09:58.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015938899999999999 2024-09-16T00:09:58.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.102326+0000", 2024-09-16T00:09:58.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074291230000000002 2024-09-16T00:09:58.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103723+0000", 2024-09-16T00:09:58.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001397255 2024-09-16T00:09:58.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103786+0000", 2024-09-16T00:09:58.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3489000000000001e-05 2024-09-16T00:09:58.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103842+0000", 2024-09-16T00:09:58.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6048999999999998e-05 2024-09-16T00:09:58.373 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.373 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.373 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.373 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:11934 3.1b e87/20 3:d94fa8da:::10000003190.00000000:head v 87'152, mlcod=87'150)", 2024-09-16T00:09:58.373 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:09:58.373 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.647307402999999, 2024-09-16T00:09:58.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014984934, 2024-09-16T00:09:58.373 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.373 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.374 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.374 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:09:58.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:09:58.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690252+0000", 2024-09-16T00:09:58.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8899999999999998e-07 2024-09-16T00:09:58.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696026+0000", 2024-09-16T00:09:58.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057745219999999998 2024-09-16T00:09:58.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696039+0000", 2024-09-16T00:09:58.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2611e-05 2024-09-16T00:09:58.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696047+0000", 2024-09-16T00:09:58.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5399999999999998e-06 2024-09-16T00:09:58.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696123+0000", 2024-09-16T00:09:58.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6349999999999999e-05 2024-09-16T00:09:58.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696144+0000", 2024-09-16T00:09:58.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1512999999999999e-05 2024-09-16T00:09:58.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705188+0000", 2024-09-16T00:09:58.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090438459999999995 2024-09-16T00:09:58.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705236+0000", 2024-09-16T00:09:58.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7663000000000001e-05 2024-09-16T00:09:58.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.378 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.378 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11936 3.3 3:c0886fb9:::100000031a1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:58.378 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:09:58.378 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.637162351000001, 2024-09-16T00:09:58.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015336379000000001, 2024-09-16T00:09:58.378 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.378 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.378 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.378 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.379 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11936 2024-09-16T00:09:58.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.379 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:09:58.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:09:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700399+0000", 2024-09-16T00:09:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6029999999999999e-06 2024-09-16T00:09:58.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704706+0000", 2024-09-16T00:09:58.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043070529999999999 2024-09-16T00:09:58.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704720+0000", 2024-09-16T00:09:58.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4629e-05 2024-09-16T00:09:58.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704729+0000", 2024-09-16T00:09:58.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6009999999999995e-06 2024-09-16T00:09:58.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704805+0000", 2024-09-16T00:09:58.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6015999999999999e-05 2024-09-16T00:09:58.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704866+0000", 2024-09-16T00:09:58.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0878999999999998e-05 2024-09-16T00:09:58.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:58.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704990+0000", 2024-09-16T00:09:58.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000124058 2024-09-16T00:09:58.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.714314+0000", 2024-09-16T00:09:58.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009324321 2024-09-16T00:09:58.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715632+0000", 2024-09-16T00:09:58.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001318121 2024-09-16T00:09:58.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715687+0000", 2024-09-16T00:09:58.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4135999999999998e-05 2024-09-16T00:09:58.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715732+0000", 2024-09-16T00:09:58.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961999999999998e-05 2024-09-16T00:09:58.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.386 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.386 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.386 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11953 3.1a 3:580e2503:::1000000311b.00000000:head [write 0~4194304 [2@2058066]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:58.386 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:09:58.387 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.570823197999999, 2024-09-16T00:09:58.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016299973999999998, 2024-09-16T00:09:58.387 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.387 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.387 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.388 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.388 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11953 2024-09-16T00:09:58.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.389 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:09:58.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:09:58.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766741+0000", 2024-09-16T00:09:58.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2939999999999996e-06 2024-09-16T00:09:58.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771165+0000", 2024-09-16T00:09:58.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044248229999999996 2024-09-16T00:09:58.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771181+0000", 2024-09-16T00:09:58.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5243e-05 2024-09-16T00:09:58.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771188+0000", 2024-09-16T00:09:58.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6429999999999995e-06 2024-09-16T00:09:58.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771238+0000", 2024-09-16T00:09:58.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9567999999999997e-05 2024-09-16T00:09:58.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771293+0000", 2024-09-16T00:09:58.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5207000000000002e-05 2024-09-16T00:09:58.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:58.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771380+0000", 2024-09-16T00:09:58.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6716000000000001e-05 2024-09-16T00:09:58.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.776193+0000", 2024-09-16T00:09:58.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048133350000000002 2024-09-16T00:09:58.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.782931+0000", 2024-09-16T00:09:58.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067376550000000004 2024-09-16T00:09:58.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.782987+0000", 2024-09-16T00:09:58.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6443000000000002e-05 2024-09-16T00:09:58.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.783035+0000", 2024-09-16T00:09:58.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8047000000000003e-05 2024-09-16T00:09:58.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.397 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.397 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12037 3.9 3:928db521:::1000000326d.00000000:head [write 0~4194304 [2@1655200]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:58.397 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:09:58.397 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.339615589999999, 2024-09-16T00:09:58.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017082494, 2024-09-16T00:09:58.397 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.397 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.397 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.398 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.398 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.398 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.398 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12037 2024-09-16T00:09:58.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.398 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:09:58.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:09:58.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997944+0000", 2024-09-16T00:09:58.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.517e-06 2024-09-16T00:09:58.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002177+0000", 2024-09-16T00:09:58.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042329300000000002 2024-09-16T00:09:58.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002197+0000", 2024-09-16T00:09:58.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9769e-05 2024-09-16T00:09:58.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002207+0000", 2024-09-16T00:09:58.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0178e-05 2024-09-16T00:09:58.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002266+0000", 2024-09-16T00:09:58.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9215999999999997e-05 2024-09-16T00:09:58.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002340+0000", 2024-09-16T00:09:58.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3798000000000001e-05 2024-09-16T00:09:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002454+0000", 2024-09-16T00:09:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113551 2024-09-16T00:09:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.006270+0000", 2024-09-16T00:09:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038163839999999999 2024-09-16T00:09:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.014957+0000", 2024-09-16T00:09:58.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086869119999999998 2024-09-16T00:09:58.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.014996+0000", 2024-09-16T00:09:58.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8878000000000003e-05 2024-09-16T00:09:58.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.015025+0000", 2024-09-16T00:09:58.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9360999999999999e-05 2024-09-16T00:09:58.404 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.404 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.404 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.404 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12245 3.1a 3:599a1cd1:::10000003252.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:58.404 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:09:58.404 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.554918173000001, 2024-09-16T00:09:58.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017131203000000001, 2024-09-16T00:09:58.405 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.405 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.405 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.405 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.405 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.405 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.405 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12245 2024-09-16T00:09:58.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.405 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:09:58.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:09:58.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782642+0000", 2024-09-16T00:09:58.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.311e-06 2024-09-16T00:09:58.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786551+0000", 2024-09-16T00:09:58.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039095229999999998 2024-09-16T00:09:58.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786566+0000", 2024-09-16T00:09:58.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4724e-05 2024-09-16T00:09:58.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786575+0000", 2024-09-16T00:09:58.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8449999999999992e-06 2024-09-16T00:09:58.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786636+0000", 2024-09-16T00:09:58.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.143e-05 2024-09-16T00:09:58.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786699+0000", 2024-09-16T00:09:58.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3218000000000004e-05 2024-09-16T00:09:58.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:58.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786796+0000", 2024-09-16T00:09:58.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6347000000000004e-05 2024-09-16T00:09:58.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.792369+0000", 2024-09-16T00:09:58.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055729780000000001 2024-09-16T00:09:58.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799706+0000", 2024-09-16T00:09:58.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073374349999999998 2024-09-16T00:09:58.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799739+0000", 2024-09-16T00:09:58.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2833999999999998e-05 2024-09-16T00:09:58.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799771+0000", 2024-09-16T00:09:58.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2558000000000002e-05 2024-09-16T00:09:58.410 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.410 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.410 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.411 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12292 3.3 3:c00779fd:::1000000320e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:09:58.411 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:09:58.411 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.440528076, 2024-09-16T00:09:58.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013828353999999999, 2024-09-16T00:09:58.411 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.411 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.411 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.411 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.412 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12292 2024-09-16T00:09:58.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.412 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:09:58.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:09:58.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897039+0000", 2024-09-16T00:09:58.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3299999999999999e-06 2024-09-16T00:09:58.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900830+0000", 2024-09-16T00:09:58.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037915340000000001 2024-09-16T00:09:58.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900850+0000", 2024-09-16T00:09:58.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9400000000000001e-05 2024-09-16T00:09:58.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900860+0000", 2024-09-16T00:09:58.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0609e-05 2024-09-16T00:09:58.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900928+0000", 2024-09-16T00:09:58.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7570000000000005e-05 2024-09-16T00:09:58.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.901027+0000", 2024-09-16T00:09:58.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9456000000000005e-05 2024-09-16T00:09:58.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:58.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.901168+0000", 2024-09-16T00:09:58.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014082799999999999 2024-09-16T00:09:58.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.906325+0000", 2024-09-16T00:09:58.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005156702 2024-09-16T00:09:58.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910796+0000", 2024-09-16T00:09:58.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044714350000000002 2024-09-16T00:09:58.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910829+0000", 2024-09-16T00:09:58.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3229000000000003e-05 2024-09-16T00:09:58.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910859+0000", 2024-09-16T00:09:58.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9261e-05 2024-09-16T00:09:58.416 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.417 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.417 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.417 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13326 3.9 3:92f796cf:::1000000350f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:09:58.417 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:09:58.417 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.380837794, 2024-09-16T00:09:58.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015673229, 2024-09-16T00:09:58.417 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.417 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.417 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.417 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.418 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.418 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.418 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13326 2024-09-16T00:09:58.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.418 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:09:58.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:09:58.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956722+0000", 2024-09-16T00:09:58.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3200000000000001e-06 2024-09-16T00:09:58.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960803+0000", 2024-09-16T00:09:58.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040807029999999998 2024-09-16T00:09:58.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960820+0000", 2024-09-16T00:09:58.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7478000000000001e-05 2024-09-16T00:09:58.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960830+0000", 2024-09-16T00:09:58.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:09:58.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960903+0000", 2024-09-16T00:09:58.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3057000000000002e-05 2024-09-16T00:09:58.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960970+0000", 2024-09-16T00:09:58.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7063000000000005e-05 2024-09-16T00:09:58.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:58.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.961067+0000", 2024-09-16T00:09:58.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6906999999999999e-05 2024-09-16T00:09:58.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.967945+0000", 2024-09-16T00:09:58.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068787359999999999 2024-09-16T00:09:58.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972328+0000", 2024-09-16T00:09:58.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043826170000000001 2024-09-16T00:09:58.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972363+0000", 2024-09-16T00:09:58.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4486000000000002e-05 2024-09-16T00:09:58.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972394+0000", 2024-09-16T00:09:58.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1343999999999997e-05 2024-09-16T00:09:58.423 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.423 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.423 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.423 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13429 3.1a 3:58dcd24a:::100000036d5.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:58.423 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:09:58.423 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 10.026267777999999, 2024-09-16T00:09:58.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014237671, 2024-09-16T00:09:58.423 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.423 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.423 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.423 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.424 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.424 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.424 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13429 2024-09-16T00:09:58.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.424 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:09:58.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:09:58.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311292+0000", 2024-09-16T00:09:58.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4890000000000001e-06 2024-09-16T00:09:58.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316378+0000", 2024-09-16T00:09:58.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050859290000000003 2024-09-16T00:09:58.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316399+0000", 2024-09-16T00:09:58.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1345e-05 2024-09-16T00:09:58.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316410+0000", 2024-09-16T00:09:58.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0295999999999999e-05 2024-09-16T00:09:58.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316493+0000", 2024-09-16T00:09:58.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3533000000000001e-05 2024-09-16T00:09:58.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316622+0000", 2024-09-16T00:09:58.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012887900000000001 2024-09-16T00:09:58.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:58.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316725+0000", 2024-09-16T00:09:58.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010295599999999999 2024-09-16T00:09:58.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.320948+0000", 2024-09-16T00:09:58.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042229820000000001 2024-09-16T00:09:58.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325415+0000", 2024-09-16T00:09:58.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044673849999999999 2024-09-16T00:09:58.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325487+0000", 2024-09-16T00:09:58.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1087999999999994e-05 2024-09-16T00:09:58.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325528+0000", 2024-09-16T00:09:58.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1789000000000003e-05 2024-09-16T00:09:58.429 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.429 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.429 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.429 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13527 3.3 3:c0886fb9:::100000031a1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:58.429 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:09:58.429 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.5521076889999996, 2024-09-16T00:09:58.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015187236, 2024-09-16T00:09:58.429 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.430 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.430 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.430 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.430 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.430 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.430 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13527 2024-09-16T00:09:58.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.430 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:09:58.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:09:58.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785453+0000", 2024-09-16T00:09:58.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1689999999999999e-06 2024-09-16T00:09:58.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791191+0000", 2024-09-16T00:09:58.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057378339999999998 2024-09-16T00:09:58.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791217+0000", 2024-09-16T00:09:58.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6480999999999999e-05 2024-09-16T00:09:58.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791233+0000", 2024-09-16T00:09:58.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5841e-05 2024-09-16T00:09:58.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791319+0000", 2024-09-16T00:09:58.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5527999999999998e-05 2024-09-16T00:09:58.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791413+0000", 2024-09-16T00:09:58.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4685000000000006e-05 2024-09-16T00:09:58.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:58.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791563+0000", 2024-09-16T00:09:58.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000149624 2024-09-16T00:09:58.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.795749+0000", 2024-09-16T00:09:58.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004186286 2024-09-16T00:09:58.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800577+0000", 2024-09-16T00:09:58.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048273040000000001 2024-09-16T00:09:58.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800611+0000", 2024-09-16T00:09:58.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4270000000000002e-05 2024-09-16T00:09:58.435 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.435 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.435 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.435 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800638+0000", 2024-09-16T00:09:58.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7214000000000001e-05 2024-09-16T00:09:58.435 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.435 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.436 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.436 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13724 3.9 3:90613e6c:::100000034c4.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:58.436 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:09:58.436 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8820359440000001, 2024-09-16T00:09:58.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014039800999999999, 2024-09-16T00:09:58.436 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.436 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.436 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.437 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.437 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.437 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.437 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13724 2024-09-16T00:09:58.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.437 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:09:58.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:09:58.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455524+0000", 2024-09-16T00:09:58.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.437e-06 2024-09-16T00:09:58.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459592+0000", 2024-09-16T00:09:58.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040678729999999996 2024-09-16T00:09:58.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459610+0000", 2024-09-16T00:09:58.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7688000000000001e-05 2024-09-16T00:09:58.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459617+0000", 2024-09-16T00:09:58.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.374e-06 2024-09-16T00:09:58.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459678+0000", 2024-09-16T00:09:58.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1346999999999994e-05 2024-09-16T00:09:58.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459824+0000", 2024-09-16T00:09:58.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014586699999999999 2024-09-16T00:09:58.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:58.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459908+0000", 2024-09-16T00:09:58.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3521000000000002e-05 2024-09-16T00:09:58.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.464227+0000", 2024-09-16T00:09:58.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043197060000000004 2024-09-16T00:09:58.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469499+0000", 2024-09-16T00:09:58.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005271838 2024-09-16T00:09:58.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469534+0000", 2024-09-16T00:09:58.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4712000000000002e-05 2024-09-16T00:09:58.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469562+0000", 2024-09-16T00:09:58.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8438000000000001e-05 2024-09-16T00:09:58.442 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.442 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.442 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.443 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13741 3.3 3:c1377c76:::10000003664.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:09:58.443 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:09:58.443 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.8348659989999998, 2024-09-16T00:09:58.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016142843, 2024-09-16T00:09:58.443 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.443 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.443 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.443 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.443 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.443 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.443 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13741 2024-09-16T00:09:58.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.443 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:09:58.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:09:58.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.444 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.444 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502694+0000", 2024-09-16T00:09:58.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0449999999999999e-06 2024-09-16T00:09:58.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507229+0000", 2024-09-16T00:09:58.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045357410000000003 2024-09-16T00:09:58.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507248+0000", 2024-09-16T00:09:58.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9127000000000002e-05 2024-09-16T00:09:58.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507259+0000", 2024-09-16T00:09:58.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0448999999999999e-05 2024-09-16T00:09:58.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507357+0000", 2024-09-16T00:09:58.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8400000000000007e-05 2024-09-16T00:09:58.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507514+0000", 2024-09-16T00:09:58.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000157195 2024-09-16T00:09:58.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:58.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507617+0000", 2024-09-16T00:09:58.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010262199999999999 2024-09-16T00:09:58.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.514047+0000", 2024-09-16T00:09:58.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006430018 2024-09-16T00:09:58.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518743+0000", 2024-09-16T00:09:58.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046963040000000001 2024-09-16T00:09:58.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518796+0000", 2024-09-16T00:09:58.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2305000000000002e-05 2024-09-16T00:09:58.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518835+0000", 2024-09-16T00:09:58.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9637e-05 2024-09-16T00:09:58.448 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.448 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.448 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.449 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14633 3.9 3:9170e82c:::100000035d6.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:58.449 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:09:58.449 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.7154857430000003, 2024-09-16T00:09:58.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014466105, 2024-09-16T00:09:58.449 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.449 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.449 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.449 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.449 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14633 2024-09-16T00:09:58.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.450 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:09:58.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:09:58.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622074+0000", 2024-09-16T00:09:58.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.761e-06 2024-09-16T00:09:58.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626712+0000", 2024-09-16T00:09:58.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004637263 2024-09-16T00:09:58.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626724+0000", 2024-09-16T00:09:58.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2479e-05 2024-09-16T00:09:58.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626730+0000", 2024-09-16T00:09:58.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7309999999999998e-06 2024-09-16T00:09:58.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626791+0000", 2024-09-16T00:09:58.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0854999999999999e-05 2024-09-16T00:09:58.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626857+0000", 2024-09-16T00:09:58.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6549000000000003e-05 2024-09-16T00:09:58.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:58.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626943+0000", 2024-09-16T00:09:58.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6141000000000001e-05 2024-09-16T00:09:58.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.630854+0000", 2024-09-16T00:09:58.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039100790000000003 2024-09-16T00:09:58.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636468+0000", 2024-09-16T00:09:58.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056140570000000004 2024-09-16T00:09:58.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636500+0000", 2024-09-16T00:09:58.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2441999999999999e-05 2024-09-16T00:09:58.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636539+0000", 2024-09-16T00:09:58.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8748000000000002e-05 2024-09-16T00:09:58.455 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.455 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.455 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.455 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14667 3.9 3:9379a4af:::10000003352.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:58.455 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:09:58.455 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 5.5404737370000001, 2024-09-16T00:09:58.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014580585, 2024-09-16T00:09:58.455 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.455 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.455 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.455 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.455 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.456 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.456 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14667 2024-09-16T00:09:58.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.456 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:09:58.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:09:58.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797086+0000", 2024-09-16T00:09:58.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.362e-06 2024-09-16T00:09:58.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801415+0000", 2024-09-16T00:09:58.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043288449999999996 2024-09-16T00:09:58.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801435+0000", 2024-09-16T00:09:58.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0305999999999998e-05 2024-09-16T00:09:58.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801442+0000", 2024-09-16T00:09:58.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6290000000000004e-06 2024-09-16T00:09:58.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801497+0000", 2024-09-16T00:09:58.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5111999999999997e-05 2024-09-16T00:09:58.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801639+0000", 2024-09-16T00:09:58.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014168000000000001 2024-09-16T00:09:58.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:09:58.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801788+0000", 2024-09-16T00:09:58.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014963300000000001 2024-09-16T00:09:58.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.805407+0000", 2024-09-16T00:09:58.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036186119999999998 2024-09-16T00:09:58.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811584+0000", 2024-09-16T00:09:58.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061772889999999999 2024-09-16T00:09:58.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811635+0000", 2024-09-16T00:09:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0453e-05 2024-09-16T00:09:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811665+0000", 2024-09-16T00:09:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0664000000000003e-05 2024-09-16T00:09:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.464 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14951 3.1a 3:59aee80a:::1000000394f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:09:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:09:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.370435842, 2024-09-16T00:09:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015993941000000001, 2024-09-16T00:09:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.465 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14951 2024-09-16T00:09:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:09:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:09:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967124+0000", 2024-09-16T00:09:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.86e-06 2024-09-16T00:09:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971258+0000", 2024-09-16T00:09:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041334780000000003 2024-09-16T00:09:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971274+0000", 2024-09-16T00:09:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6201e-05 2024-09-16T00:09:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971284+0000", 2024-09-16T00:09:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4779999999999997e-06 2024-09-16T00:09:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971384+0000", 2024-09-16T00:09:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9878999999999997e-05 2024-09-16T00:09:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971518+0000", 2024-09-16T00:09:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013459400000000001 2024-09-16T00:09:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:09:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971604+0000", 2024-09-16T00:09:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6215e-05 2024-09-16T00:09:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.976673+0000", 2024-09-16T00:09:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050685920000000002 2024-09-16T00:09:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983043+0000", 2024-09-16T00:09:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063698410000000002 2024-09-16T00:09:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983082+0000", 2024-09-16T00:09:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9261999999999998e-05 2024-09-16T00:09:58.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983117+0000", 2024-09-16T00:09:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4541e-05 2024-09-16T00:09:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:15152 3.3 3:c388c7e2:::100000032ec.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-16T00:09:58.472 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:09:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.6098897320000001, 2024-09-16T00:09:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014603532000000001, 2024-09-16T00:09:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15152 2024-09-16T00:09:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:09:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:09:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727674+0000", 2024-09-16T00:09:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7699999999999998e-06 2024-09-16T00:09:58.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732274+0000", 2024-09-16T00:09:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045994299999999998 2024-09-16T00:09:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732286+0000", 2024-09-16T00:09:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2016e-05 2024-09-16T00:09:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732294+0000", 2024-09-16T00:09:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7049999999999999e-06 2024-09-16T00:09:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732359+0000", 2024-09-16T00:09:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5296000000000006e-05 2024-09-16T00:09:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732501+0000", 2024-09-16T00:09:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000141731 2024-09-16T00:09:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:09:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732629+0000", 2024-09-16T00:09:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012841299999999999 2024-09-16T00:09:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.737511+0000", 2024-09-16T00:09:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048822520000000001 2024-09-16T00:09:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742172+0000", 2024-09-16T00:09:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046610610000000002 2024-09-16T00:09:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742227+0000", 2024-09-16T00:09:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4497999999999999e-05 2024-09-16T00:09:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742272+0000", 2024-09-16T00:09:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5359999999999999e-05 2024-09-16T00:09:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.481 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.482 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.482 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:58.484 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:09:58.485 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:58.483+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:58.584 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:58.584+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:58.636 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:58.636 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:58.636 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:58.636 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:58.645 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:09:58.685 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:09:58.685+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:09:58.786 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:58.786+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:58.815 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:58.816 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:58.816 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:58.816 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:58.816 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:58.824 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:09:58.909 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:58.909+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:58.976 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:58.977 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:09:58.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:09:58.977 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:09:58.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.977 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10658 3.1c 3:38c4c24e:::100000030bb.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:09:58.978 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:09:58.978 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.182929173000002, 2024-09-16T00:09:58.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021232445999999999, 2024-09-16T00:09:58.978 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.978 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.979 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:09:58.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:09:58.979 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:09:58.979 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10658 2024-09-16T00:09:58.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.980 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:09:58.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:09:58.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791398+0000", 2024-09-16T00:09:58.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3880000000000001e-06 2024-09-16T00:09:58.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800951+0000", 2024-09-16T00:09:58.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095532110000000007 2024-09-16T00:09:58.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800966+0000", 2024-09-16T00:09:58.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5058999999999999e-05 2024-09-16T00:09:58.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800974+0000", 2024-09-16T00:09:58.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1240000000000005e-06 2024-09-16T00:09:58.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801127+0000", 2024-09-16T00:09:58.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000153103 2024-09-16T00:09:58.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801207+0000", 2024-09-16T00:09:58.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9569999999999999e-05 2024-09-16T00:09:58.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:58.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801322+0000", 2024-09-16T00:09:58.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115358 2024-09-16T00:09:58.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.805760+0000", 2024-09-16T00:09:58.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044379409999999999 2024-09-16T00:09:58.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812528+0000", 2024-09-16T00:09:58.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067678249999999999 2024-09-16T00:09:58.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812586+0000", 2024-09-16T00:09:58.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8164000000000001e-05 2024-09-16T00:09:58.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812629+0000", 2024-09-16T00:09:58.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2703000000000001e-05 2024-09-16T00:09:58.986 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.986 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.986 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.986 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4122 2.18 2:1cc859d9:::200.00000012:head [write 798310~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:58.986 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:09:58.986 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.31538949300000002, 2024-09-16T00:09:58.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019267580999999999, 2024-09-16T00:09:58.986 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.986 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.986 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.987 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:09:58.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:09:58.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658937+0000", 2024-09-16T00:09:58.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7700000000000004e-07 2024-09-16T00:09:58.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658942+0000", 2024-09-16T00:09:58.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.48e-06 2024-09-16T00:09:58.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658952+0000", 2024-09-16T00:09:58.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9469999999999998e-06 2024-09-16T00:09:58.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658959+0000", 2024-09-16T00:09:58.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4400000000000002e-06 2024-09-16T00:09:58.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670507+0000", 2024-09-16T00:09:58.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011548159000000001 2024-09-16T00:09:58.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670579+0000", 2024-09-16T00:09:58.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2082999999999994e-05 2024-09-16T00:09:58.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:58.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670686+0000", 2024-09-16T00:09:58.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107069 2024-09-16T00:09:58.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678001+0000", 2024-09-16T00:09:58.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073148780000000004 2024-09-16T00:09:58.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678104+0000", 2024-09-16T00:09:58.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010290599999999999 2024-09-16T00:09:58.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678163+0000", 2024-09-16T00:09:58.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8859e-05 2024-09-16T00:09:58.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678204+0000", 2024-09-16T00:09:58.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0982999999999997e-05 2024-09-16T00:09:58.992 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.992 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.992 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.992 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4123 2.18 2:1cc859d9:::200.00000012:head [write 799709~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:58.992 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:09:58.992 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.31482031199999999, 2024-09-16T00:09:58.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018857569000000001, 2024-09-16T00:09:58.993 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.993 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.993 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.993 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:09:58.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:58.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:09:58.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:58.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659506+0000", 2024-09-16T00:09:58.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.78e-07 2024-09-16T00:09:58.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:58.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659512+0000", 2024-09-16T00:09:58.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0100000000000001e-06 2024-09-16T00:09:58.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:58.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659523+0000", 2024-09-16T00:09:58.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-16T00:09:58.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:58.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659531+0000", 2024-09-16T00:09:58.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2010000000000002e-06 2024-09-16T00:09:58.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:58.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671124+0000", 2024-09-16T00:09:58.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011593696000000001 2024-09-16T00:09:58.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:58.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671171+0000", 2024-09-16T00:09:58.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6974999999999997e-05 2024-09-16T00:09:58.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:58.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671243+0000", 2024-09-16T00:09:58.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2131000000000004e-05 2024-09-16T00:09:58.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:58.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678008+0000", 2024-09-16T00:09:58.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067642329999999997 2024-09-16T00:09:58.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:58.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678261+0000", 2024-09-16T00:09:58.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025378999999999999 2024-09-16T00:09:58.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:58.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678320+0000", 2024-09-16T00:09:58.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8452000000000002e-05 2024-09-16T00:09:58.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.998 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:58.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678363+0000", 2024-09-16T00:09:58.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2831999999999999e-05 2024-09-16T00:09:58.998 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.998 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:58.998 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:58.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:58.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.999 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4126 2.18 2:1cc859d9:::200.00000012:head [write 806443~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:58.999 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:09:58.999 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.31342252599999998, 2024-09-16T00:09:58.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019331159000000001, 2024-09-16T00:09:58.999 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:58.999 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:58.999 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:58.999 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:58.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:58.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:58.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:09:58.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:58.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:09:59.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660904+0000", 2024-09-16T00:09:59.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6800000000000009e-07 2024-09-16T00:09:59.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660909+0000", 2024-09-16T00:09:59.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2730000000000002e-06 2024-09-16T00:09:59.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660919+0000", 2024-09-16T00:09:59.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4960000000000002e-06 2024-09-16T00:09:59.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660923+0000", 2024-09-16T00:09:59.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6260000000000003e-06 2024-09-16T00:09:59.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672698+0000", 2024-09-16T00:09:59.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011774573999999999 2024-09-16T00:09:59.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672784+0000", 2024-09-16T00:09:59.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6074000000000003e-05 2024-09-16T00:09:59.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672897+0000", 2024-09-16T00:09:59.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011267199999999999 2024-09-16T00:09:59.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678026+0000", 2024-09-16T00:09:59.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051297089999999997 2024-09-16T00:09:59.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680147+0000", 2024-09-16T00:09:59.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021209409999999999 2024-09-16T00:09:59.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680197+0000", 2024-09-16T00:09:59.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0068000000000002e-05 2024-09-16T00:09:59.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680234+0000", 2024-09-16T00:09:59.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6758000000000003e-05 2024-09-16T00:09:59.004 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.004 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.004 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.005 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4127 2.18 2:1cc859d9:::200.00000012:head [write 808688~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:59.005 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:09:59.005 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.31314067699999998, 2024-09-16T00:09:59.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019226493000000001, 2024-09-16T00:09:59.005 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:59.005 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:59.005 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:59.005 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:59.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:59.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:09:59.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:09:59.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661186+0000", 2024-09-16T00:09:59.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2399999999999996e-07 2024-09-16T00:09:59.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661192+0000", 2024-09-16T00:09:59.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9259999999999997e-06 2024-09-16T00:09:59.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661203+0000", 2024-09-16T00:09:59.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1174e-05 2024-09-16T00:09:59.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661213+0000", 2024-09-16T00:09:59.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0579e-05 2024-09-16T00:09:59.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673408+0000", 2024-09-16T00:09:59.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012194442 2024-09-16T00:09:59.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673482+0000", 2024-09-16T00:09:59.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4258000000000006e-05 2024-09-16T00:09:59.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673592+0000", 2024-09-16T00:09:59.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109733 2024-09-16T00:09:59.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678033+0000", 2024-09-16T00:09:59.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044411629999999997 2024-09-16T00:09:59.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680287+0000", 2024-09-16T00:09:59.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022538469999999998 2024-09-16T00:09:59.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680367+0000", 2024-09-16T00:09:59.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9808999999999994e-05 2024-09-16T00:09:59.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680411+0000", 2024-09-16T00:09:59.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4638000000000002e-05 2024-09-16T00:09:59.011 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.011 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.011 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.011 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4128 2.18 2:1cc859d9:::200.00000012:head [write 810933~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:59.011 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:09:59.011 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.31272154600000002, 2024-09-16T00:09:59.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018982046999999998, 2024-09-16T00:09:59.012 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:59.012 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:59.012 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:59.012 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:59.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:59.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:09:59.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:09:59.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661605+0000", 2024-09-16T00:09:59.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.031e-06 2024-09-16T00:09:59.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661610+0000", 2024-09-16T00:09:59.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5010000000000004e-06 2024-09-16T00:09:59.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661623+0000", 2024-09-16T00:09:59.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2286000000000001e-05 2024-09-16T00:09:59.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661630+0000", 2024-09-16T00:09:59.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000002e-06 2024-09-16T00:09:59.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674011+0000", 2024-09-16T00:09:59.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012380923 2024-09-16T00:09:59.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674092+0000", 2024-09-16T00:09:59.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1267999999999995e-05 2024-09-16T00:09:59.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674216+0000", 2024-09-16T00:09:59.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012331700000000001 2024-09-16T00:09:59.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678039+0000", 2024-09-16T00:09:59.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038232489999999999 2024-09-16T00:09:59.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680493+0000", 2024-09-16T00:09:59.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024543429999999999 2024-09-16T00:09:59.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680546+0000", 2024-09-16T00:09:59.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3239000000000003e-05 2024-09-16T00:09:59.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680586+0000", 2024-09-16T00:09:59.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9496999999999998e-05 2024-09-16T00:09:59.017 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.018 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.018 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4129 2.18 2:1cc859d9:::200.00000012:head [write 813178~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:59.018 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:09:59.018 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.31236439500000002, 2024-09-16T00:09:59.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018785932000000002, 2024-09-16T00:09:59.018 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:59.018 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:59.018 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:59.018 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:59.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:59.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:09:59.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:09:59.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661962+0000", 2024-09-16T00:09:59.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-07 2024-09-16T00:09:59.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661966+0000", 2024-09-16T00:09:59.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-06 2024-09-16T00:09:59.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.662015+0000", 2024-09-16T00:09:59.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8364e-05 2024-09-16T00:09:59.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.662023+0000", 2024-09-16T00:09:59.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2090000000000005e-06 2024-09-16T00:09:59.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674744+0000", 2024-09-16T00:09:59.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012721423000000001 2024-09-16T00:09:59.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674825+0000", 2024-09-16T00:09:59.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1036000000000001e-05 2024-09-16T00:09:59.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674946+0000", 2024-09-16T00:09:59.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120727 2024-09-16T00:09:59.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678045+0000", 2024-09-16T00:09:59.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030993280000000002 2024-09-16T00:09:59.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680646+0000", 2024-09-16T00:09:59.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026005300000000002 2024-09-16T00:09:59.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680705+0000", 2024-09-16T00:09:59.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9141000000000002e-05 2024-09-16T00:09:59.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.025 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.025 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680747+0000", 2024-09-16T00:09:59.025 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2064000000000002e-05 2024-09-16T00:09:59.025 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.025 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.025 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.025 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.025 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.025 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4184 2.18 2:1cc859d9:::200.00000012:head [write 927308~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:59.026 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:09:59.026 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.26831579999999999, 2024-09-16T00:09:59.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023503514999999999, 2024-09-16T00:09:59.026 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:59.026 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:59.026 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:59.026 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:59.026 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.026 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:59.026 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:09:59.026 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.026 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:09:59.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.027 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.027 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.027 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.027 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706011+0000", 2024-09-16T00:09:59.027 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8400000000000003e-07 2024-09-16T00:09:59.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706016+0000", 2024-09-16T00:09:59.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7479999999999999e-06 2024-09-16T00:09:59.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.028 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706028+0000", 2024-09-16T00:09:59.028 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1477e-05 2024-09-16T00:09:59.028 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.028 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.028 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706035+0000", 2024-09-16T00:09:59.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1509999999999998e-06 2024-09-16T00:09:59.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.719820+0000", 2024-09-16T00:09:59.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013785168 2024-09-16T00:09:59.029 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.029 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.029 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.029 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.719888+0000", 2024-09-16T00:09:59.029 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7631000000000004e-05 2024-09-16T00:09:59.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720003+0000", 2024-09-16T00:09:59.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114849 2024-09-16T00:09:59.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725150+0000", 2024-09-16T00:09:59.030 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051469970000000004 2024-09-16T00:09:59.030 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.030 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.030 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.030 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729386+0000", 2024-09-16T00:09:59.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042368529999999996 2024-09-16T00:09:59.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729461+0000", 2024-09-16T00:09:59.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4935e-05 2024-09-16T00:09:59.031 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.031 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.031 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.031 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729513+0000", 2024-09-16T00:09:59.031 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1822e-05 2024-09-16T00:09:59.031 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.032 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.032 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.032 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.032 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4185 2.18 2:1cc859d9:::200.00000012:head [write 929217~3264 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:59.032 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:09:59.032 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.26730962000000003, 2024-09-16T00:09:59.032 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023998430000000001, 2024-09-16T00:09:59.032 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:59.032 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:59.032 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:59.032 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:59.032 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:59.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:09:59.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.033 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:09:59.033 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.033 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.033 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.033 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707017+0000", 2024-09-16T00:09:59.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9999999999999996e-07 2024-09-16T00:09:59.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707023+0000", 2024-09-16T00:09:59.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1789999999999996e-06 2024-09-16T00:09:59.034 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.034 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.034 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.034 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707035+0000", 2024-09-16T00:09:59.034 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2106000000000001e-05 2024-09-16T00:09:59.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707042+0000", 2024-09-16T00:09:59.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2960000000000001e-06 2024-09-16T00:09:59.035 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.035 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.035 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.035 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720503+0000", 2024-09-16T00:09:59.035 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013460952999999999 2024-09-16T00:09:59.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720583+0000", 2024-09-16T00:09:59.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0171000000000002e-05 2024-09-16T00:09:59.036 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.036 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.036 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.036 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720668+0000", 2024-09-16T00:09:59.036 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4128999999999994e-05 2024-09-16T00:09:59.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725193+0000", 2024-09-16T00:09:59.037 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045257700000000001 2024-09-16T00:09:59.037 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.037 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.037 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.730924+0000", 2024-09-16T00:09:59.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057307690000000001 2024-09-16T00:09:59.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.730972+0000", 2024-09-16T00:09:59.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7756000000000002e-05 2024-09-16T00:09:59.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.038 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.038 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731014+0000", 2024-09-16T00:09:59.038 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2401000000000003e-05 2024-09-16T00:09:59.038 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.038 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.039 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.039 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.039 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4186 2.18 2:1cc859d9:::200.00000012:head [write 932481~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:59.039 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:09:59.039 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.26672146299999999, 2024-09-16T00:09:59.039 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023544934999999999, 2024-09-16T00:09:59.039 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:59.039 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:59.039 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:59.039 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:59.039 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.039 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:59.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:09:59.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.040 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.040 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.040 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:09:59.040 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.040 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.041 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.041 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.041 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707605+0000", 2024-09-16T00:09:59.041 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1500000000000003e-07 2024-09-16T00:09:59.041 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.042 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.042 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707610+0000", 2024-09-16T00:09:59.042 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7049999999999998e-06 2024-09-16T00:09:59.042 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.042 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707618+0000", 2024-09-16T00:09:59.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6149999999999997e-06 2024-09-16T00:09:59.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707623+0000", 2024-09-16T00:09:59.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0660000000000003e-06 2024-09-16T00:09:59.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.043 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.043 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.043 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721143+0000", 2024-09-16T00:09:59.043 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01351976 2024-09-16T00:09:59.043 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721221+0000", 2024-09-16T00:09:59.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7918999999999997e-05 2024-09-16T00:09:59.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721335+0000", 2024-09-16T00:09:59.044 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113781 2024-09-16T00:09:59.044 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.044 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.044 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.044 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725236+0000", 2024-09-16T00:09:59.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039010379999999999 2024-09-16T00:09:59.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731025+0000", 2024-09-16T00:09:59.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057896659999999997 2024-09-16T00:09:59.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.045 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.045 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.045 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731091+0000", 2024-09-16T00:09:59.045 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.567e-05 2024-09-16T00:09:59.045 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.046 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.046 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731149+0000", 2024-09-16T00:09:59.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7800000000000002e-05 2024-09-16T00:09:59.046 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.046 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.046 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.046 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.046 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.046 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4187 2.18 2:1cc859d9:::200.00000012:head [write 934390~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:59.046 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:09:59.046 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.26586933400000001, 2024-09-16T00:09:59.046 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022820340000000001, 2024-09-16T00:09:59.047 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:59.047 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:59.047 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:59.047 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:59.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:59.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:09:59.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.047 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.047 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.047 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.047 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:09:59.047 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.048 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708457+0000", 2024-09-16T00:09:59.048 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6899999999999996e-07 2024-09-16T00:09:59.048 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.048 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.048 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708462+0000", 2024-09-16T00:09:59.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3090000000000003e-06 2024-09-16T00:09:59.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708472+0000", 2024-09-16T00:09:59.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7499999999999998e-06 2024-09-16T00:09:59.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.049 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.049 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.049 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708479+0000", 2024-09-16T00:09:59.049 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1189999999999999e-06 2024-09-16T00:09:59.049 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721895+0000", 2024-09-16T00:09:59.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013415664000000001 2024-09-16T00:09:59.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721941+0000", 2024-09-16T00:09:59.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6031e-05 2024-09-16T00:09:59.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.050 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.050 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.050 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722007+0000", 2024-09-16T00:09:59.050 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6503999999999999e-05 2024-09-16T00:09:59.050 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725277+0000", 2024-09-16T00:09:59.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003269554 2024-09-16T00:09:59.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.051 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731158+0000", 2024-09-16T00:09:59.051 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058813750000000003 2024-09-16T00:09:59.051 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.051 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.051 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731240+0000", 2024-09-16T00:09:59.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1815000000000002e-05 2024-09-16T00:09:59.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.052 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.052 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731276+0000", 2024-09-16T00:09:59.052 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6449999999999998e-05 2024-09-16T00:09:59.052 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.052 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.052 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.052 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.052 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.052 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4188 2.18 2:1cc859d9:::200.00000012:head [write 936299~2419 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:59.053 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:09:59.053 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.26544839100000001, 2024-09-16T00:09:59.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022518565000000001, 2024-09-16T00:09:59.053 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:59.053 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:59.053 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:59.053 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:59.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.053 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:59.053 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:09:59.053 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.053 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.053 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:09:59.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708878+0000", 2024-09-16T00:09:59.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8800000000000001e-07 2024-09-16T00:09:59.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.054 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.054 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708884+0000", 2024-09-16T00:09:59.054 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9800000000000003e-06 2024-09-16T00:09:59.054 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.054 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708895+0000", 2024-09-16T00:09:59.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1416e-05 2024-09-16T00:09:59.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708903+0000", 2024-09-16T00:09:59.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4449999999999999e-06 2024-09-16T00:09:59.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.055 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.055 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.055 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722327+0000", 2024-09-16T00:09:59.055 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013424681000000001 2024-09-16T00:09:59.055 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722408+0000", 2024-09-16T00:09:59.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0092000000000005e-05 2024-09-16T00:09:59.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722538+0000", 2024-09-16T00:09:59.056 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013046600000000001 2024-09-16T00:09:59.056 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.056 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.056 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.056 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726665+0000", 2024-09-16T00:09:59.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041267420000000001 2024-09-16T00:09:59.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731284+0000", 2024-09-16T00:09:59.057 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046195469999999999 2024-09-16T00:09:59.057 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.057 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.057 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.057 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731358+0000", 2024-09-16T00:09:59.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4002000000000001e-05 2024-09-16T00:09:59.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.058 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.058 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.058 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731396+0000", 2024-09-16T00:09:59.058 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7305999999999998e-05 2024-09-16T00:09:59.058 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.058 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.058 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.058 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.059 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4189 2.18 2:1cc859d9:::200.00000012:head [write 938718~1910 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:59.059 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:09:59.059 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.26454127500000002, 2024-09-16T00:09:59.059 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021703626, 2024-09-16T00:09:59.059 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:59.059 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:59.059 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:59.059 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:59.059 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.059 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:59.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:09:59.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:09:59.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.060 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.060 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.060 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709785+0000", 2024-09-16T00:09:59.060 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.043e-06 2024-09-16T00:09:59.060 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709791+0000", 2024-09-16T00:09:59.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9499999999999998e-06 2024-09-16T00:09:59.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.061 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.061 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.061 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709804+0000", 2024-09-16T00:09:59.061 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2958e-05 2024-09-16T00:09:59.061 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.062 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.062 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709812+0000", 2024-09-16T00:09:59.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9869999999999999e-06 2024-09-16T00:09:59.062 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723058+0000", 2024-09-16T00:09:59.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013245497 2024-09-16T00:09:59.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723131+0000", 2024-09-16T00:09:59.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3879000000000002e-05 2024-09-16T00:09:59.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723241+0000", 2024-09-16T00:09:59.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109435 2024-09-16T00:09:59.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726708+0000", 2024-09-16T00:09:59.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034671340000000002 2024-09-16T00:09:59.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731403+0000", 2024-09-16T00:09:59.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046953979999999999 2024-09-16T00:09:59.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731454+0000", 2024-09-16T00:09:59.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0747e-05 2024-09-16T00:09:59.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731488+0000", 2024-09-16T00:09:59.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3597999999999999e-05 2024-09-16T00:09:59.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.067 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.067 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.067 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4190 2.18 2:1cc859d9:::200.00000012:head [write 940628~2420 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:59.067 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:09:59.067 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.26406911, 2024-09-16T00:09:59.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021346812999999999, 2024-09-16T00:09:59.067 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:59.067 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:59.067 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:59.067 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:59.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:59.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:09:59.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:09:59.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710257+0000", 2024-09-16T00:09:59.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8100000000000001e-07 2024-09-16T00:09:59.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710263+0000", 2024-09-16T00:09:59.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8050000000000003e-06 2024-09-16T00:09:59.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710273+0000", 2024-09-16T00:09:59.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0005e-05 2024-09-16T00:09:59.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.070 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710280+0000", 2024-09-16T00:09:59.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3819999999999998e-06 2024-09-16T00:09:59.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.070 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723653+0000", 2024-09-16T00:09:59.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013372866000000001 2024-09-16T00:09:59.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.071 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.071 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.071 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723733+0000", 2024-09-16T00:09:59.071 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9396999999999999e-05 2024-09-16T00:09:59.071 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723852+0000", 2024-09-16T00:09:59.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011919899999999999 2024-09-16T00:09:59.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726752+0000", 2024-09-16T00:09:59.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028999659999999999 2024-09-16T00:09:59.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731495+0000", 2024-09-16T00:09:59.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047432209999999997 2024-09-16T00:09:59.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731568+0000", 2024-09-16T00:09:59.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2452000000000004e-05 2024-09-16T00:09:59.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731603+0000", 2024-09-16T00:09:59.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5639000000000002e-05 2024-09-16T00:09:59.074 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.074 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.074 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.074 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4192 2.18 2:1cc859d9:::200.00000012:head [write 943048~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:59.074 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:09:59.074 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.26333626999999998, 2024-09-16T00:09:59.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021688433999999999, 2024-09-16T00:09:59.074 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:59.074 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:59.074 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:59.075 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:59.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:59.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:09:59.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:09:59.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710990+0000", 2024-09-16T00:09:59.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1999999999999998e-07 2024-09-16T00:09:59.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710996+0000", 2024-09-16T00:09:59.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:09:59.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711007+0000", 2024-09-16T00:09:59.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1471000000000001e-05 2024-09-16T00:09:59.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711014+0000", 2024-09-16T00:09:59.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7780000000000002e-06 2024-09-16T00:09:59.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724471+0000", 2024-09-16T00:09:59.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013457435 2024-09-16T00:09:59.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724547+0000", 2024-09-16T00:09:59.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5383999999999995e-05 2024-09-16T00:09:59.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724663+0000", 2024-09-16T00:09:59.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116348 2024-09-16T00:09:59.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731612+0000", 2024-09-16T00:09:59.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069491409999999998 2024-09-16T00:09:59.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732573+0000", 2024-09-16T00:09:59.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0009611 2024-09-16T00:09:59.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732638+0000", 2024-09-16T00:09:59.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919000000000006e-05 2024-09-16T00:09:59.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732678+0000", 2024-09-16T00:09:59.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9230999999999998e-05 2024-09-16T00:09:59.079 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.079 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.079 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.080 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4193 2.18 2:1cc859d9:::200.00000012:head [write 944447~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:59.080 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:09:59.080 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.26279056200000001, 2024-09-16T00:09:59.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021286851999999998, 2024-09-16T00:09:59.080 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:59.080 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:59.080 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:59.080 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:59.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:59.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:09:59.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:09:59.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711536+0000", 2024-09-16T00:09:59.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0189999999999999e-06 2024-09-16T00:09:59.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711542+0000", 2024-09-16T00:09:59.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7169999999999996e-06 2024-09-16T00:09:59.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711554+0000", 2024-09-16T00:09:59.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2089000000000001e-05 2024-09-16T00:09:59.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711559+0000", 2024-09-16T00:09:59.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3739999999999997e-06 2024-09-16T00:09:59.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725311+0000", 2024-09-16T00:09:59.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013751559 2024-09-16T00:09:59.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725390+0000", 2024-09-16T00:09:59.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9779999999999998e-05 2024-09-16T00:09:59.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725509+0000", 2024-09-16T00:09:59.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118973 2024-09-16T00:09:59.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731617+0000", 2024-09-16T00:09:59.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061079000000000003 2024-09-16T00:09:59.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732762+0000", 2024-09-16T00:09:59.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011449769999999999 2024-09-16T00:09:59.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732796+0000", 2024-09-16T00:09:59.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3587999999999998e-05 2024-09-16T00:09:59.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732822+0000", 2024-09-16T00:09:59.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5876000000000001e-05 2024-09-16T00:09:59.087 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.087 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.087 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.087 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4194 2.18 2:1cc859d9:::200.00000012:head [write 946356~2929 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:59.087 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:09:59.087 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.26181553099999999, 2024-09-16T00:09:59.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020453244999999998, 2024-09-16T00:09:59.087 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:59.087 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:59.087 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:59.087 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:59.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:59.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:09:59.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:09:59.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712511+0000", 2024-09-16T00:09:59.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0999999999999997e-07 2024-09-16T00:09:59.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712517+0000", 2024-09-16T00:09:59.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8370000000000002e-06 2024-09-16T00:09:59.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712528+0000", 2024-09-16T00:09:59.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1528e-05 2024-09-16T00:09:59.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712536+0000", 2024-09-16T00:09:59.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3939999999999998e-06 2024-09-16T00:09:59.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726004+0000", 2024-09-16T00:09:59.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013468177 2024-09-16T00:09:59.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726085+0000", 2024-09-16T00:09:59.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1626999999999996e-05 2024-09-16T00:09:59.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726205+0000", 2024-09-16T00:09:59.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119751 2024-09-16T00:09:59.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731622+0000", 2024-09-16T00:09:59.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054165419999999999 2024-09-16T00:09:59.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732867+0000", 2024-09-16T00:09:59.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001244855 2024-09-16T00:09:59.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732924+0000", 2024-09-16T00:09:59.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7395999999999997e-05 2024-09-16T00:09:59.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732963+0000", 2024-09-16T00:09:59.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9227999999999998e-05 2024-09-16T00:09:59.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.093 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.093 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4195 2.18 2:1cc859d9:::200.00000012:head [write 949285~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:59.093 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:09:59.093 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.26076387099999998, 2024-09-16T00:09:59.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019498099000000001, 2024-09-16T00:09:59.093 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:59.093 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:59.093 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:59.094 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:59.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:59.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:09:59.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:09:59.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713563+0000", 2024-09-16T00:09:59.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T00:09:59.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713569+0000", 2024-09-16T00:09:59.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8710000000000002e-06 2024-09-16T00:09:59.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713582+0000", 2024-09-16T00:09:59.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3134e-05 2024-09-16T00:09:59.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713591+0000", 2024-09-16T00:09:59.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9889999999999998e-06 2024-09-16T00:09:59.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726786+0000", 2024-09-16T00:09:59.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013194991999999999 2024-09-16T00:09:59.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726856+0000", 2024-09-16T00:09:59.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0017000000000004e-05 2024-09-16T00:09:59.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726967+0000", 2024-09-16T00:09:59.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111815 2024-09-16T00:09:59.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732530+0000", 2024-09-16T00:09:59.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055624940000000003 2024-09-16T00:09:59.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733001+0000", 2024-09-16T00:09:59.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000470817 2024-09-16T00:09:59.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733030+0000", 2024-09-16T00:09:59.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9051e-05 2024-09-16T00:09:59.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733060+0000", 2024-09-16T00:09:59.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9819e-05 2024-09-16T00:09:59.098 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.098 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.098 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.099 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4215 2.18 2:1cc859d9:::200.00000012:head [write 991028~1910 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:59.099 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:09:59.099 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.24106816, 2024-09-16T00:09:59.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019416676000000001, 2024-09-16T00:09:59.099 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:59.099 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:59.099 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:59.099 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:59.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:59.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:09:59.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:09:59.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733258+0000", 2024-09-16T00:09:59.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:09:59.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733264+0000", 2024-09-16T00:09:59.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4090000000000002e-06 2024-09-16T00:09:59.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733273+0000", 2024-09-16T00:09:59.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4769999999999992e-06 2024-09-16T00:09:59.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733280+0000", 2024-09-16T00:09:59.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2989999999999999e-06 2024-09-16T00:09:59.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.742966+0000", 2024-09-16T00:09:59.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009685585 2024-09-16T00:09:59.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743046+0000", 2024-09-16T00:09:59.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0469999999999994e-05 2024-09-16T00:09:59.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743165+0000", 2024-09-16T00:09:59.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118847 2024-09-16T00:09:59.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.746968+0000", 2024-09-16T00:09:59.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038025099999999998 2024-09-16T00:09:59.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752559+0000", 2024-09-16T00:09:59.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055911290000000002 2024-09-16T00:09:59.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752634+0000", 2024-09-16T00:09:59.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5102e-05 2024-09-16T00:09:59.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752674+0000", 2024-09-16T00:09:59.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9878e-05 2024-09-16T00:09:59.108 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.109 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.109 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.109 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4216 2.18 2:1cc859d9:::200.00000012:head [write 992938~2755 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:09:59.109 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:09:59.109 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.24052225199999999, 2024-09-16T00:09:59.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019035976, 2024-09-16T00:09:59.109 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:09:59.109 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:09:59.109 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:09:59.109 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:09:59.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:09:59.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:09:59.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:09:59.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:09:59.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:09:59.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:09:59.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733804+0000", 2024-09-16T00:09:59.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2299999999999999e-07 2024-09-16T00:09:59.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:09:59.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733810+0000", 2024-09-16T00:09:59.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7259999999999999e-06 2024-09-16T00:09:59.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:09:59.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733822+0000", 2024-09-16T00:09:59.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2204e-05 2024-09-16T00:09:59.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:09:59.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733829+0000", 2024-09-16T00:09:59.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3889999999999999e-06 2024-09-16T00:09:59.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:09:59.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743675+0000", 2024-09-16T00:09:59.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098457990000000006 2024-09-16T00:09:59.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:09:59.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743752+0000", 2024-09-16T00:09:59.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6279000000000006e-05 2024-09-16T00:09:59.112 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:09:59.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743867+0000", 2024-09-16T00:09:59.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115319 2024-09-16T00:09:59.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:09:59.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.746971+0000", 2024-09-16T00:09:59.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031042399999999999 2024-09-16T00:09:59.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:09:59.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752723+0000", 2024-09-16T00:09:59.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057518329999999996 2024-09-16T00:09:59.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:09:59.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752804+0000", 2024-09-16T00:09:59.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0822999999999995e-05 2024-09-16T00:09:59.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:09:59.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:09:59.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:09:59.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752839+0000", 2024-09-16T00:09:59.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5441000000000002e-05 2024-09-16T00:09:59.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.117 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:09:59.117 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:09:59.117 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:59.119 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:09:59.119 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:09:59.118+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:09:59.220 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:59.219+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:59.287 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:59.287 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:59.287 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:09:59.287 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:59.296 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:09:59.305 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:09:59.306 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T00:09:59.339 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:59.339+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:59.406 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:09:59.406 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:09:59.407 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:09:59.407 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:09:59.407 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:09:59.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.406+0000 7f612f19d640 1 Processor -- start 2024-09-16T00:09:59.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.406+0000 7f612f19d640 1 -- start start 2024-09-16T00:09:59.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.407+0000 7f612f19d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61280c2cb0 0x7f6128073e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.407+0000 7f612f19d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61280c2210 con 0x7f61281515e0 2024-09-16T00:09:59.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.407+0000 7f612f19d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61280716c0 con 0x7f61280c1de0 2024-09-16T00:09:59.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.407+0000 7f612f19d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f61280b22b0 con 0x7f61280c2cb0 2024-09-16T00:09:59.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.407+0000 7f612e19b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f61281515e0 0x7f61280b1ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:58120/0 (socket says 172.21.15.139:58120) 2024-09-16T00:09:59.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.407+0000 7f612e19b640 1 -- 172.21.15.139:0/2026827376 learned_addr learned my addr 172.21.15.139:0/2026827376 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:59.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.407+0000 7f612d99a640 1 --2- 172.21.15.139:0/2026827376 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61280c2cb0 0x7f6128073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.407+0000 7f612d199640 1 -- 172.21.15.139:0/2026827376 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 745836524 0 0) 0x7f61280c2210 con 0x7f61281515e0 2024-09-16T00:09:59.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.407+0000 7f612d199640 1 -- 172.21.15.139:0/2026827376 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6114003520 con 0x7f61281515e0 2024-09-16T00:09:59.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.407+0000 7f612d199640 1 -- 172.21.15.139:0/2026827376 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2841791760 0 0) 0x7f61280716c0 con 0x7f61280c1de0 2024-09-16T00:09:59.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.407+0000 7f612d199640 1 -- 172.21.15.139:0/2026827376 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61280c2210 con 0x7f61280c1de0 2024-09-16T00:09:59.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.407+0000 7f612d99a640 1 -- 172.21.15.139:0/2026827376 >> v1:172.21.15.164:6789/0 conn(0x7f61280c1de0 legacy=0x7f61280c0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.407+0000 7f612d99a640 1 -- 172.21.15.139:0/2026827376 >> v1:172.21.15.139:6789/0 conn(0x7f61281515e0 legacy=0x7f61280b1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.407+0000 7f612d99a640 1 -- 172.21.15.139:0/2026827376 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61280719a0 con 0x7f61280c2cb0 2024-09-16T00:09:59.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.408+0000 7f612d99a640 1 --2- 172.21.15.139:0/2026827376 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61280c2cb0 0x7f6128073e80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f61180097d0 tx=0x7f611802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d35f4297d8a0713e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:59.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.408+0000 7f612d199640 1 -- 172.21.15.139:0/2026827376 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118037070 con 0x7f61280c2cb0 2024-09-16T00:09:59.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.408+0000 7f612d199640 1 -- 172.21.15.139:0/2026827376 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6118032440 con 0x7f61280c2cb0 2024-09-16T00:09:59.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.408+0000 7f612d199640 1 -- 172.21.15.139:0/2026827376 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61180327f0 con 0x7f61280c2cb0 2024-09-16T00:09:59.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.409+0000 7f612f19d640 1 -- 172.21.15.139:0/2026827376 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61280c2cb0 msgr2=0x7f6128073e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.409+0000 7f612f19d640 1 --2- 172.21.15.139:0/2026827376 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61280c2cb0 0x7f6128073e80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f61180097d0 tx=0x7f611802a210 comp rx=0 tx=0).stop 2024-09-16T00:09:59.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.409+0000 7f612f19d640 1 -- 172.21.15.139:0/2026827376 shutdown_connections 2024-09-16T00:09:59.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.409+0000 7f612f19d640 1 --2- 172.21.15.139:0/2026827376 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61280c2cb0 0x7f6128073e80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:59.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.409+0000 7f612f19d640 1 -- 172.21.15.139:0/2026827376 >> 172.21.15.139:0/2026827376 conn(0x7f61280c8990 msgr2=0x7f61280c8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:59.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.409+0000 7f612f19d640 1 -- 172.21.15.139:0/2026827376 shutdown_connections 2024-09-16T00:09:59.413 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.409+0000 7f612f19d640 1 -- 172.21.15.139:0/2026827376 wait complete. 2024-09-16T00:09:59.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.409+0000 7fc76822e640 1 Processor -- start 2024-09-16T00:09:59.414 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.410+0000 7fc76822e640 1 -- start start 2024-09-16T00:09:59.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.410+0000 7f612f19d640 1 Processor -- start 2024-09-16T00:09:59.415 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.410+0000 7f612f19d640 1 -- start start 2024-09-16T00:09:59.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7fc76822e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7600c8c50 0x7fc7600c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.416 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7fc76822e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7600ce310 con 0x7fc760152b20 2024-09-16T00:09:59.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7fc76822e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7600c3380 con 0x7fc7601587b0 2024-09-16T00:09:59.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7fc76822e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc7600c5470 con 0x7fc7600c8c50 2024-09-16T00:09:59.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7fc765fca640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc760152b20 0x7fc76006ff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:58130/0 (socket says 172.21.15.139:58130) 2024-09-16T00:09:59.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7fc765fca640 1 -- 172.21.15.139:0/1375540038 learned_addr learned my addr 172.21.15.139:0/1375540038 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:09:59.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7fc7657c9640 1 --2- 172.21.15.139:0/1375540038 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7600c8c50 0x7fc7600c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7fc764fc8640 1 -- 172.21.15.139:0/1375540038 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4080425873 0 0) 0x7fc7600ce310 con 0x7fc760152b20 2024-09-16T00:09:59.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7fc764fc8640 1 -- 172.21.15.139:0/1375540038 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc74c003520 con 0x7fc760152b20 2024-09-16T00:09:59.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7f612f19d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61280c1de0 0x7f6128156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7fc764fc8640 1 -- 172.21.15.139:0/1375540038 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2479701986 0 0) 0x7fc7600c3380 con 0x7fc7601587b0 2024-09-16T00:09:59.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7f612f19d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61280c2cb0 0x7f61281f0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7f612f19d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61281515e0 0x7f61280c5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7fc764fc8640 1 -- 172.21.15.139:0/1375540038 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7600ce310 con 0x7fc7601587b0 2024-09-16T00:09:59.419 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7f612f19d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f61280c98d0 con 0x7f61281515e0 2024-09-16T00:09:59.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7f612f19d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f61281519c0 con 0x7f61280c2cb0 2024-09-16T00:09:59.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7f612f19d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6128071f60 con 0x7f61280c1de0 2024-09-16T00:09:59.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7fc7657c9640 1 -- 172.21.15.139:0/1375540038 >> v1:172.21.15.164:6789/0 conn(0x7fc7601587b0 legacy=0x7fc760162f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7fc7657c9640 1 -- 172.21.15.139:0/1375540038 >> v1:172.21.15.139:6789/0 conn(0x7fc760152b20 legacy=0x7fc76006ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7fc7657c9640 1 -- 172.21.15.139:0/1375540038 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7600c3660 con 0x7fc7600c8c50 2024-09-16T00:09:59.422 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:09:59.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7f612d99a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61280c2cb0 0x7f61281f0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7f612e19b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61280c1de0 0x7f6128156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7f612e99c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61281515e0 0x7f61280c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7f612d99a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61280c2cb0 0x7f61281f0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:49754/0 (socket says 172.21.15.139:49754) 2024-09-16T00:09:59.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.411+0000 7f612d99a640 1 -- 172.21.15.139:0/2066816360 learned_addr learned my addr 172.21.15.139:0/2066816360 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:59.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.412+0000 7f612d99a640 1 -- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61280c1de0 msgr2=0x7f6128156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.412+0000 7f612d99a640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61280c1de0 0x7f6128156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:59.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.412+0000 7f612d99a640 1 -- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61281515e0 msgr2=0x7f61280c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.412+0000 7f612d99a640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61281515e0 0x7f61280c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:59.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.412+0000 7f612d99a640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61280c51d0 con 0x7f61280c2cb0 2024-09-16T00:09:59.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.412+0000 7f612e99c640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61281515e0 0x7f61280c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:09:59.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.412+0000 7f612d99a640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61280c2cb0 0x7f61281f0880 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f61180098f0 tx=0x7f61180046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:59.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.412+0000 7fc7657c9640 1 --2- 172.21.15.139:0/1375540038 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7600c8c50 0x7fc7600c4f30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc7500097d0 tx=0x7fc75002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f7252a5b7584ee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:59.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.412+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118037040 con 0x7f61280c2cb0 2024-09-16T00:09:59.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.412+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6118042070 con 0x7f61280c2cb0 2024-09-16T00:09:59.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.412+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118032810 con 0x7f61280c2cb0 2024-09-16T00:09:59.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.412+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6128157050 con 0x7f61280c2cb0 2024-09-16T00:09:59.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.412+0000 7fc764fc8640 1 -- 172.21.15.139:0/1375540038 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750037070 con 0x7fc7600c8c50 2024-09-16T00:09:59.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.412+0000 7fc764fc8640 1 -- 172.21.15.139:0/1375540038 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc750032440 con 0x7fc7600c8c50 2024-09-16T00:09:59.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.412+0000 7fc764fc8640 1 -- 172.21.15.139:0/1375540038 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750032810 con 0x7fc7600c8c50 2024-09-16T00:09:59.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.412+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612814d480 con 0x7f61280c2cb0 2024-09-16T00:09:59.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.413+0000 7fc76822e640 1 -- 172.21.15.139:0/1375540038 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7600c8c50 msgr2=0x7fc7600c4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.413+0000 7fc76822e640 1 --2- 172.21.15.139:0/1375540038 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7600c8c50 0x7fc7600c4f30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc7500097d0 tx=0x7fc75002a180 comp rx=0 tx=0).stop 2024-09-16T00:09:59.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.413+0000 7fc76822e640 1 -- 172.21.15.139:0/1375540038 shutdown_connections 2024-09-16T00:09:59.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.413+0000 7fc76822e640 1 --2- 172.21.15.139:0/1375540038 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc7600c8c50 0x7fc7600c4f30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:59.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.413+0000 7fc76822e640 1 -- 172.21.15.139:0/1375540038 >> 172.21.15.139:0/1375540038 conn(0x7fc76006bd00 msgr2=0x7fc760067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:59.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.413+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f6128073960 con 0x7f61280c2cb0 2024-09-16T00:09:59.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.413+0000 7fc76822e640 1 -- 172.21.15.139:0/1375540038 shutdown_connections 2024-09-16T00:09:59.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.413+0000 7fc76822e640 1 -- 172.21.15.139:0/1375540038 wait complete. 2024-09-16T00:09:59.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.414+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f61180329b0 con 0x7f61280c2cb0 2024-09-16T00:09:59.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.414+0000 7f611effd640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f610c06c2b0 0x7f610c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.428 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.414+0000 7fc76822e640 1 Processor -- start 2024-09-16T00:09:59.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.414+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5973+0+0 (secure 0 0 0) 0x7f61180abf00 con 0x7f61280c2cb0 2024-09-16T00:09:59.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.414+0000 7f612e19b640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f610c06c2b0 0x7f610c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.414+0000 7fc76822e640 1 -- start start 2024-09-16T00:09:59.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.414+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v94) ==== 54+0+16 (secure 0 0 0) 0x7f61180ac2b0 con 0x7f61280c2cb0 2024-09-16T00:09:59.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.415+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f61280c53a0 0x7f61280c3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.415+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f612806e640 con 0x7f61280c53a0 2024-09-16T00:09:59.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.415+0000 7f612e19b640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f610c06c2b0 0x7f610c06e770 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6124000f60 tx=0x7f6124007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:59.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.415+0000 7f612e99c640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f61280c53a0 0x7f61280c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.415+0000 7f612e99c640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f61280c53a0 0x7f61280c3420 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:59.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.415+0000 7fc76822e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7600c8c50 0x7fc7601f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.415+0000 7fc76822e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc760152b20 0x7fc76006e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.415+0000 7fc76822e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7601587b0 0x7fc76014f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.415+0000 7fc76822e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc7600c3940 con 0x7fc7600c8c50 2024-09-16T00:09:59.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.415+0000 7fc76822e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc7600cbba0 con 0x7fc7601587b0 2024-09-16T00:09:59.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.415+0000 7fc76822e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc760070610 con 0x7fc760152b20 2024-09-16T00:09:59.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.415+0000 7fc765fca640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7600c8c50 0x7fc7601f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.415+0000 7fc765fca640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7600c8c50 0x7fc7601f8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55820/0 (socket says 172.21.15.139:55820) 2024-09-16T00:09:59.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.415+0000 7fc765fca640 1 -- 172.21.15.139:0/582136466 learned_addr learned my addr 172.21.15.139:0/582136466 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:09:59.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.416+0000 7fc7657c9640 1 --2- 172.21.15.139:0/582136466 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc760152b20 0x7fc76006e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.416+0000 7fc7667cb640 1 --2- 172.21.15.139:0/582136466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7601587b0 0x7fc76014f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.433 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.416+0000 7fc765fca640 1 -- 172.21.15.139:0/582136466 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc760152b20 msgr2=0x7fc76006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.416+0000 7fc765fca640 1 --2- 172.21.15.139:0/582136466 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc760152b20 0x7fc76006e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:59.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.416+0000 7fc765fca640 1 -- 172.21.15.139:0/582136466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7601587b0 msgr2=0x7fc76014f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.416+0000 7fc765fca640 1 --2- 172.21.15.139:0/582136466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7601587b0 0x7fc76014f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:59.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.416+0000 7fc765fca640 1 -- 172.21.15.139:0/582136466 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76006d140 con 0x7fc7600c8c50 2024-09-16T00:09:59.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.416+0000 7fc765fca640 1 --2- 172.21.15.139:0/582136466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7600c8c50 0x7fc7601f8440 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00c490 tx=0x7fc75c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:59.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.416+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f612806e640 con 0x7f61280c53a0 2024-09-16T00:09:59.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.416+0000 7fc7567fc640 1 -- 172.21.15.139:0/582136466 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c017070 con 0x7fc7600c8c50 2024-09-16T00:09:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.416+0000 7fc7567fc640 1 -- 172.21.15.139:0/582136466 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc75c005d00 con 0x7fc7600c8c50 2024-09-16T00:09:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.416+0000 7fc7567fc640 1 -- 172.21.15.139:0/582136466 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c007310 con 0x7fc7600c8c50 2024-09-16T00:09:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.416+0000 7fc76822e640 1 -- 172.21.15.139:0/582136466 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76006e1d0 con 0x7fc7600c8c50 2024-09-16T00:09:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.416+0000 7fc76822e640 1 -- 172.21.15.139:0/582136466 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76006cc30 con 0x7fc7600c8c50 2024-09-16T00:09:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.417+0000 7fc76822e640 1 -- 172.21.15.139:0/582136466 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc760069240 con 0x7fc7600c8c50 2024-09-16T00:09:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.418+0000 7fc7567fc640 1 -- 172.21.15.139:0/582136466 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc75c01a070 con 0x7fc7600c8c50 2024-09-16T00:09:59.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.418+0000 7fc7567fc640 1 --2- 172.21.15.139:0/582136466 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc74406c2e0 0x7fc74406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.418+0000 7fc7567fc640 1 -- 172.21.15.139:0/582136466 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5973+0+0 (secure 0 0 0) 0x7fc75c08a870 con 0x7fc7600c8c50 2024-09-16T00:09:59.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.418+0000 7fc7657c9640 1 --2- 172.21.15.139:0/582136466 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc74406c2e0 0x7fc74406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.419+0000 7fc7657c9640 1 --2- 172.21.15.139:0/582136466 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc74406c2e0 0x7fc74406e7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc750034000 tx=0x7fc75002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:59.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.422+0000 7fc7567fc640 1 -- 172.21.15.139:0/582136466 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc75c058870 con 0x7fc7600c8c50 2024-09-16T00:09:59.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.432+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f61280c8620 con 0x7f61280c53a0 2024-09-16T00:09:59.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.433+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f61280c8620 con 0x7f61280c53a0 2024-09-16T00:09:59.439 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:09:59.440 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:09:59.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.433+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f612814d650 0x7f612814a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.433+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f6128157950 con 0x7f612814d650 2024-09-16T00:09:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.434+0000 7f612e19b640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f612814d650 0x7f612814a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.434+0000 7f612e19b640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f612814d650 0x7f612814a770 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:59.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.435+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6128157950 con 0x7f612814d650 2024-09-16T00:09:59.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.450+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6128157670 con 0x7f612814d650 2024-09-16T00:09:59.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.452+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6128157670 con 0x7f612814d650 2024-09-16T00:09:59.453 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:09:59.453 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:09:59.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.452+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f6128154ec0 0x7f612814bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.452+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f612814c2d0 con 0x7f6128154ec0 2024-09-16T00:09:59.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.452+0000 7f612d99a640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f6128154ec0 0x7f612814bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.453+0000 7f612d99a640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f6128154ec0 0x7f612814bd90 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:59.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.454+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f612814c2d0 con 0x7f6128154ec0 2024-09-16T00:09:59.464 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:59.463+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:09:59.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.468+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6128062630 con 0x7f6128154ec0 2024-09-16T00:09:59.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.471+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6128062630 con 0x7f6128154ec0 2024-09-16T00:09:59.472 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:09:59.474 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:09:59.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.471+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f6128063110 0x7f612814ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.471+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f6128063550 con 0x7f6128063110 2024-09-16T00:09:59.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.471+0000 7f612e99c640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f6128063110 0x7f612814ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.471+0000 7f612e99c640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f6128063110 0x7f612814ee00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:59.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.472+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6128063550 con 0x7f6128063110 2024-09-16T00:09:59.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.486+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f612814f590 con 0x7f6128063110 2024-09-16T00:09:59.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.488+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f612814f590 con 0x7f6128063110 2024-09-16T00:09:59.489 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:09:59.489 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:09:59.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.488+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f612802c640 0x7f612802ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.488+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f612814fc10 con 0x7f612802c640 2024-09-16T00:09:59.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.488+0000 7f612e19b640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f612802c640 0x7f612802ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.489+0000 7f612e19b640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f612802c640 0x7f612802ca80 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:59.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.490+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f612814fc10 con 0x7f612802c640 2024-09-16T00:09:59.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.506+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f612814fc10 con 0x7f612802c640 2024-09-16T00:09:59.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.508+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f612814fc10 con 0x7f612802c640 2024-09-16T00:09:59.509 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:09:59.509 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:09:59.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.508+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f612802d4d0 0x7f61282071e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.508+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f612802d930 con 0x7f612802d4d0 2024-09-16T00:09:59.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.509+0000 7f612d99a640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f612802d4d0 0x7f61282071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.510+0000 7f612d99a640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f612802d4d0 0x7f61282071e0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:59.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.511+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f612802d930 con 0x7f612802d4d0 2024-09-16T00:09:59.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.525+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f612802db70 con 0x7f612802d4d0 2024-09-16T00:09:59.527 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:59.526+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:59.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.527+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f612802db70 con 0x7f612802d4d0 2024-09-16T00:09:59.528 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:09:59.528 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:09:59.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.527+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f6128207c30 0x7f612820a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.527+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f612802daa0 con 0x7f6128207c30 2024-09-16T00:09:59.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.527+0000 7f612e99c640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f6128207c30 0x7f612820a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.528+0000 7f612e99c640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f6128207c30 0x7f612820a020 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:59.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.529+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f612802daa0 con 0x7f6128207c30 2024-09-16T00:09:59.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.543+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f612802daa0 con 0x7f6128207c30 2024-09-16T00:09:59.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.545+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' ) ==== 503+0+3 (crc 0 0 0) 0x7f612802daa0 con 0x7f6128207c30 2024-09-16T00:09:59.546 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:09:59.546 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-16T00:09:59.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.545+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f612820aa70 0x7f612820d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:09:59.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.545+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f6128210010 con 0x7f612820aa70 2024-09-16T00:09:59.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.545+0000 7f612e19b640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f612820aa70 0x7f612820d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:09:59.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.545+0000 7f612e19b640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f612820aa70 0x7f612820d240 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:09:59.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.546+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6128210010 con 0x7f612820aa70 2024-09-16T00:09:59.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.561+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f612820d780 con 0x7f612820aa70 2024-09-16T00:09:59.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.562+0000 7f611effd640 1 -- 172.21.15.139:0/2066816360 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f612820d780 con 0x7f612820aa70 2024-09-16T00:09:59.563 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:09:59.564 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:09:59.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f61280c53a0 msgr2=0x7f61280c3420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f61280c53a0 0x7f61280c3420 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:59.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f612814d650 msgr2=0x7f612814a770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f612814d650 0x7f612814a770 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:59.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f6128154ec0 msgr2=0x7f612814bd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f6128154ec0 0x7f612814bd90 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:59.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f6128063110 msgr2=0x7f612814ee00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f6128063110 0x7f612814ee00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:59.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f612802c640 msgr2=0x7f612802ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f612802c640 0x7f612802ca80 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:59.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f612802d4d0 msgr2=0x7f61282071e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f612802d4d0 0x7f61282071e0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:59.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f6128207c30 msgr2=0x7f612820a020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f6128207c30 0x7f612820a020 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:59.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f612820aa70 msgr2=0x7f612820d240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f612820aa70 0x7f612820d240 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:09:59.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612e99c640 1 -- 172.21.15.139:0/2066816360 reap_dead start 2024-09-16T00:09:59.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f610c06c2b0 msgr2=0x7f610c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612e99c640 1 -- 172.21.15.139:0/2066816360 reap_dead start 2024-09-16T00:09:59.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f610c06c2b0 0x7f610c06e770 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6124000f60 tx=0x7f6124007d50 comp rx=0 tx=0).stop 2024-09-16T00:09:59.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61280c2cb0 msgr2=0x7f61281f0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:09:59.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 --2- 172.21.15.139:0/2066816360 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61280c2cb0 0x7f61281f0880 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f61180098f0 tx=0x7f61180046c0 comp rx=0 tx=0).stop 2024-09-16T00:09:59.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612e99c640 1 -- 172.21.15.139:0/2066816360 reap_dead start 2024-09-16T00:09:59.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612e99c640 1 -- 172.21.15.139:0/2066816360 reap_dead start 2024-09-16T00:09:59.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612e99c640 1 -- 172.21.15.139:0/2066816360 reap_dead start 2024-09-16T00:09:59.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 shutdown_connections 2024-09-16T00:09:59.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 >> 172.21.15.139:0/2066816360 conn(0x7f61280c8990 msgr2=0x7f61280c71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:09:59.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 shutdown_connections 2024-09-16T00:09:59.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.563+0000 7f612f19d640 1 -- 172.21.15.139:0/2066816360 wait complete. 2024-09-16T00:09:59.588 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:59.588 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:59.599 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:09:59.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:09:59.628+0000 7fc76822e640 1 -- 172.21.15.139:0/582136466 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fc7600b06c0 con 0x7fc7600c8c50 2024-09-16T00:09:59.630 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:09:59.629+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:09:59.723 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:59.723 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:59.733 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:09:59.729+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:09:59.734 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:09:59.830 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:09:59.830+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:09:59.892 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:09:59.892 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:09:59.902 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:09:59.935 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:09:59.934+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:00.013 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:00.013 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:00.022 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:10:00.084 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:00.083+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:00.139 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:00.138+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:00.143 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:00.143 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:00.152 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:10:00.244 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:00.243+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:00.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:00.267+0000 7fc7567fc640 1 -- 172.21.15.139:0/582136466 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v95) ==== 91+0+0 (secure 0 0 0) 0x7fc75c05cb70 con 0x7fc7600c8c50 2024-09-16T00:10:00.269 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is unset 2024-09-16T00:10:00.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:00.270+0000 7fc76822e640 1 -- 172.21.15.139:0/582136466 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc74406c2e0 msgr2=0x7fc74406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:00.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:00.270+0000 7fc76822e640 1 --2- 172.21.15.139:0/582136466 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc74406c2e0 0x7fc74406e7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc750034000 tx=0x7fc75002e000 comp rx=0 tx=0).stop 2024-09-16T00:10:00.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:00.270+0000 7fc76822e640 1 -- 172.21.15.139:0/582136466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7600c8c50 msgr2=0x7fc7601f8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:00.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:00.270+0000 7fc76822e640 1 --2- 172.21.15.139:0/582136466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7600c8c50 0x7fc7601f8440 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00c490 tx=0x7fc75c00c950 comp rx=0 tx=0).stop 2024-09-16T00:10:00.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:00.271+0000 7fc76822e640 1 -- 172.21.15.139:0/582136466 shutdown_connections 2024-09-16T00:10:00.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:00.271+0000 7fc76822e640 1 --2- 172.21.15.139:0/582136466 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc74406c2e0 0x7fc74406e7a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:00.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:00.271+0000 7fc76822e640 1 --2- 172.21.15.139:0/582136466 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc7601587b0 0x7fc76014f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:00.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:00.271+0000 7fc76822e640 1 --2- 172.21.15.139:0/582136466 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc760152b20 0x7fc76006e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:00.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:00.271+0000 7fc76822e640 1 --2- 172.21.15.139:0/582136466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7600c8c50 0x7fc7601f8440 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:00.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:00.271+0000 7fc76822e640 1 -- 172.21.15.139:0/582136466 >> 172.21.15.139:0/582136466 conn(0x7fc76006bd00 msgr2=0x7fc7600cc450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:00.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:00.271+0000 7fc76822e640 1 -- 172.21.15.139:0/582136466 shutdown_connections 2024-09-16T00:10:00.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:00.271+0000 7fc76822e640 1 -- 172.21.15.139:0/582136466 wait complete. 2024-09-16T00:10:00.328 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:00.329 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:00.345 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:10:00.345 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:00.344+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:00.445 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:00.445+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:00.512 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:00.512 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:00.522 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:10:00.568 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:00.567+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:00.632 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:00.633 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:00.642 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:10:00.684 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:00.683+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:00.750 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:00.749+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:00.796 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:00.797 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:00.812 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:10:00.855 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:00.854+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:00.953 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:00.953+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:00.966 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:00.966 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:00.975 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:10:01.061 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:01.060+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:01.130 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:01.131 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:01.140 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:10:01.182 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:01.181+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:01.258 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:01.258+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:01.291 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:01.291 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:01.300 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:10:01.360 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:01.359+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:01.454 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:01.454 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:01.460 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:01.460+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:01.463 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:10:01.498 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:10:01.562 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:01.561+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:01.576 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:10:01.602 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:01.602 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:01.615 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:10:01.654 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:01.654 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:01.663 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:01.662+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:01.664 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T00:10:01.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.677+0000 7f2e2aa77640 1 Processor -- start 2024-09-16T00:10:01.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.677+0000 7f2e2aa77640 1 -- start start 2024-09-16T00:10:01.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.678+0000 7f2e2aa77640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e240b2170 0x7f2e2415a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.678+0000 7f2e2aa77640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2e24073bf0 0x7f2e241651b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.678+0000 7f2e2aa77640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e240c8800 con 0x7f2e2406da30 2024-09-16T00:10:01.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.678+0000 7f2e2aa77640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f2e240c7980 con 0x7f2e24073bf0 2024-09-16T00:10:01.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.678+0000 7f2e2aa77640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2e24073480 con 0x7f2e240b2170 2024-09-16T00:10:01.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.678+0000 7f2e23fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2e2406da30 0x7f2e240cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:58138/0 (socket says 172.21.15.139:58138) 2024-09-16T00:10:01.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.678+0000 7f2e23fff640 1 -- 172.21.15.139:0/3072187340 learned_addr learned my addr 172.21.15.139:0/3072187340 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:01.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.679+0000 7f2e28813640 1 --2- 172.21.15.139:0/3072187340 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e240b2170 0x7f2e2415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.679+0000 7f2e29014640 1 --2- 172.21.15.139:0/3072187340 >> v2:172.21.15.164:3300/0 conn(0x7f2e24073bf0 0x7f2e241651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.679+0000 7f2e237fe640 1 -- 172.21.15.139:0/3072187340 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1309328471 0 0) 0x7f2e240c8800 con 0x7f2e2406da30 2024-09-16T00:10:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.679+0000 7f2e237fe640 1 -- 172.21.15.139:0/3072187340 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e14003520 con 0x7f2e2406da30 2024-09-16T00:10:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.679+0000 7f2e29014640 1 -- 172.21.15.139:0/3072187340 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e240b2170 msgr2=0x7f2e2415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.679+0000 7f2e29014640 1 --2- 172.21.15.139:0/3072187340 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e240b2170 0x7f2e2415a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.679+0000 7f2e29014640 1 -- 172.21.15.139:0/3072187340 >> v1:172.21.15.139:6789/0 conn(0x7f2e2406da30 legacy=0x7f2e240cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.684 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.679+0000 7f2e29014640 1 -- 172.21.15.139:0/3072187340 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e240c7bd0 con 0x7f2e24073bf0 2024-09-16T00:10:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.679+0000 7f2e28813640 1 --2- 172.21.15.139:0/3072187340 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e240b2170 0x7f2e2415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.680+0000 7f2e29014640 1 --2- 172.21.15.139:0/3072187340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e24073bf0 0x7f2e241651b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009d90 tx=0x7f2e1c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0a4d9524adf8044 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.680+0000 7f2e237fe640 1 -- 172.21.15.139:0/3072187340 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c007740 con 0x7f2e24073bf0 2024-09-16T00:10:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.680+0000 7f2e237fe640 1 -- 172.21.15.139:0/3072187340 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e1c0078e0 con 0x7f2e24073bf0 2024-09-16T00:10:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.680+0000 7f2e237fe640 1 -- 172.21.15.139:0/3072187340 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c007cb0 con 0x7f2e24073bf0 2024-09-16T00:10:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.681+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3072187340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e24073bf0 msgr2=0x7f2e241651b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.681+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3072187340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e24073bf0 0x7f2e241651b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009d90 tx=0x7f2e1c02a660 comp rx=0 tx=0).stop 2024-09-16T00:10:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.681+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3072187340 shutdown_connections 2024-09-16T00:10:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.681+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3072187340 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e24073bf0 0x7f2e241651b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.681+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3072187340 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e240b2170 0x7f2e2415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.681+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3072187340 >> 172.21.15.139:0/3072187340 conn(0x7f2e24067950 msgr2=0x7f2e24067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.681+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3072187340 shutdown_connections 2024-09-16T00:10:01.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.681+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3072187340 wait complete. 2024-09-16T00:10:01.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.682+0000 7f2e2aa77640 1 Processor -- start 2024-09-16T00:10:01.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.682+0000 7f2e2aa77640 1 -- start start 2024-09-16T00:10:01.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e2aa77640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e2406da30 0x7f2e241f1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e2aa77640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e24073bf0 0x7f2e240cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e2aa77640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e240b2170 0x7f2e241f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e2aa77640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2415ae50 con 0x7f2e2406da30 2024-09-16T00:10:01.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e2aa77640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2e240736d0 con 0x7f2e240b2170 2024-09-16T00:10:01.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e2aa77640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2e240c4020 con 0x7f2e24073bf0 2024-09-16T00:10:01.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e28813640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e2406da30 0x7f2e241f1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e28813640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e2406da30 0x7f2e241f1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55826/0 (socket says 172.21.15.139:55826) 2024-09-16T00:10:01.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e28813640 1 -- 172.21.15.139:0/3425160915 learned_addr learned my addr 172.21.15.139:0/3425160915 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:01.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e29014640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e240b2170 0x7f2e241f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e23fff640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e24073bf0 0x7f2e240cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e29014640 1 -- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e24073bf0 msgr2=0x7f2e240cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e29014640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e24073bf0 0x7f2e240cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e29014640 1 -- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e2406da30 msgr2=0x7f2e241f1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e29014640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e2406da30 0x7f2e241f1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e29014640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e241f1590 con 0x7f2e240b2170 2024-09-16T00:10:01.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e23fff640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e24073bf0 0x7f2e240cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:01.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.683+0000 7f2e28813640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e2406da30 0x7f2e241f1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:01.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.684+0000 7f2e29014640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e240b2170 0x7f2e241f3b70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c031000 tx=0x7f2e1c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:01.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.684+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c040070 con 0x7f2e240b2170 2024-09-16T00:10:01.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.684+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e1c031c00 con 0x7f2e240b2170 2024-09-16T00:10:01.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.684+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e2415da20 con 0x7f2e240b2170 2024-09-16T00:10:01.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.684+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c009450 con 0x7f2e240b2170 2024-09-16T00:10:01.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.684+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e2415e4a0 con 0x7f2e240b2170 2024-09-16T00:10:01.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.685+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f2e2406d250 con 0x7f2e240b2170 2024-09-16T00:10:01.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.686+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2e1c039040 con 0x7f2e240b2170 2024-09-16T00:10:01.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.686+0000 7f2e217fa640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2e0806c2b0 0x7f2e0806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.686+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5973+0+0 (secure 0 0 0) 0x7f2e1c0ad1b0 con 0x7f2e240b2170 2024-09-16T00:10:01.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.686+0000 7f2e28813640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2e0806c2b0 0x7f2e0806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.686+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v95) ==== 54+0+16 (secure 0 0 0) 0x7f2e1c053020 con 0x7f2e240b2170 2024-09-16T00:10:01.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.687+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f2e241f40b0 0x7f2e241f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.687+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e241f49d0 con 0x7f2e241f40b0 2024-09-16T00:10:01.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.687+0000 7f2e28813640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2e0806c2b0 0x7f2e0806e770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009560 tx=0x7f2e18009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:01.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.687+0000 7f2e23fff640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f2e241f40b0 0x7f2e241f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.687+0000 7f2e23fff640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f2e241f40b0 0x7f2e241f4490 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:01.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.688+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e241f49d0 con 0x7f2e241f40b0 2024-09-16T00:10:01.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.703+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2e24074070 con 0x7f2e241f40b0 2024-09-16T00:10:01.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.705+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2e24074070 con 0x7f2e241f40b0 2024-09-16T00:10:01.705 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:01.706 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:01.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.705+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f2e2415d170 0x7f2e24062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.705+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f2e2415e140 con 0x7f2e2415d170 2024-09-16T00:10:01.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.705+0000 7f2e28813640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f2e2415d170 0x7f2e24062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.705+0000 7f2e28813640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f2e2415d170 0x7f2e24062630 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:01.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.706+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e2415e140 con 0x7f2e2415d170 2024-09-16T00:10:01.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.721+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2e2415e8e0 con 0x7f2e2415d170 2024-09-16T00:10:01.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.723+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2e2415e8e0 con 0x7f2e2415d170 2024-09-16T00:10:01.724 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:01.725 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:01.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.723+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f2e24063080 0x7f2e240c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.723+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f2e240c8390 con 0x7f2e24063080 2024-09-16T00:10:01.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.723+0000 7f2e23fff640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f2e24063080 0x7f2e240c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.724+0000 7f2e23fff640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f2e24063080 0x7f2e240c8b20 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:01.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.725+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e240c8390 con 0x7f2e24063080 2024-09-16T00:10:01.726 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:01.726 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:01.736 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:10:01.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.740+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2e240c9060 con 0x7f2e24063080 2024-09-16T00:10:01.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.741+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2e240c9060 con 0x7f2e24063080 2024-09-16T00:10:01.742 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:01.743 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:01.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.742+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f2e2402c640 0x7f2e2402caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.742+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f2e240c9060 con 0x7f2e2402c640 2024-09-16T00:10:01.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.742+0000 7f2e29014640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f2e2402c640 0x7f2e2402caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.742+0000 7f2e29014640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f2e2402c640 0x7f2e2402caa0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:01.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.743+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e240c9060 con 0x7f2e2402c640 2024-09-16T00:10:01.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.757+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2e240c9060 con 0x7f2e2402c640 2024-09-16T00:10:01.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.758+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2e240c9060 con 0x7f2e2402c640 2024-09-16T00:10:01.759 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:01.760 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:01.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.758+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f2e24153820 0x7f2e24153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.759+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f2e240c9060 con 0x7f2e24153820 2024-09-16T00:10:01.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.759+0000 7f2e28813640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f2e24153820 0x7f2e24153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.759+0000 7f2e28813640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f2e24153820 0x7f2e24153c80 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:01.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.760+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e240c9060 con 0x7f2e24153820 2024-09-16T00:10:01.764 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:01.763+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:01.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.773+0000 7f08e094b640 1 Processor -- start 2024-09-16T00:10:01.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.774+0000 7f08e094b640 1 -- start start 2024-09-16T00:10:01.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.774+0000 7f08e094b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f08dc15da90 0x7f08dc15de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.774+0000 7f08e094b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08dc06ccd0 con 0x7f08dc0c8b50 2024-09-16T00:10:01.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.774+0000 7f08e094b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08dc0ca720 con 0x7f08dc070d90 2024-09-16T00:10:01.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.774+0000 7f08e094b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f08dc15e3b0 con 0x7f08dc15da90 2024-09-16T00:10:01.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.775+0000 7f08d9d9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f08dc0c8b50 0x7f08dc070410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:58150/0 (socket says 172.21.15.139:58150) 2024-09-16T00:10:01.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.775+0000 7f08d9d9b640 1 -- 172.21.15.139:0/333171146 learned_addr learned my addr 172.21.15.139:0/333171146 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:01.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.775+0000 7f08da59c640 1 --2- 172.21.15.139:0/333171146 >> v2:172.21.15.164:3300/0 conn(0x7f08dc15da90 0x7f08dc15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.775+0000 7f08d959a640 1 -- 172.21.15.139:0/333171146 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 940498548 0 0) 0x7f08dc06ccd0 con 0x7f08dc0c8b50 2024-09-16T00:10:01.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.775+0000 7f08d959a640 1 -- 172.21.15.139:0/333171146 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08c4003520 con 0x7f08dc0c8b50 2024-09-16T00:10:01.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.775+0000 7f08da59c640 1 -- 172.21.15.139:0/333171146 >> v1:172.21.15.164:6789/0 conn(0x7f08dc070d90 legacy=0x7f08dc0ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.775+0000 7f08d959a640 1 -- 172.21.15.139:0/333171146 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2467385736 0 0) 0x7f08dc0ca720 con 0x7f08dc070d90 2024-09-16T00:10:01.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.775+0000 7f08da59c640 1 -- 172.21.15.139:0/333171146 >> v1:172.21.15.139:6789/0 conn(0x7f08dc0c8b50 legacy=0x7f08dc070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.775+0000 7f08da59c640 1 -- 172.21.15.139:0/333171146 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc06cf40 con 0x7f08dc15da90 2024-09-16T00:10:01.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.776+0000 7f08da59c640 1 --2- 172.21.15.139:0/333171146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08dc15da90 0x7f08dc15de70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0097a0 tx=0x7f08cc02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79ab21f23dd8aca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:01.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.776+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2e240c9060 con 0x7f2e24153820 2024-09-16T00:10:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.776+0000 7f08d959a640 1 -- 172.21.15.139:0/333171146 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc037070 con 0x7f08dc15da90 2024-09-16T00:10:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.776+0000 7f08d959a640 1 -- 172.21.15.139:0/333171146 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08cc032440 con 0x7f08dc15da90 2024-09-16T00:10:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.776+0000 7f08d959a640 1 -- 172.21.15.139:0/333171146 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0327f0 con 0x7f08dc15da90 2024-09-16T00:10:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.777+0000 7f08e094b640 1 -- 172.21.15.139:0/333171146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08dc15da90 msgr2=0x7f08dc15de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.777+0000 7f08e094b640 1 --2- 172.21.15.139:0/333171146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08dc15da90 0x7f08dc15de70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0097a0 tx=0x7f08cc02a230 comp rx=0 tx=0).stop 2024-09-16T00:10:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.777+0000 7f08e094b640 1 -- 172.21.15.139:0/333171146 shutdown_connections 2024-09-16T00:10:01.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.777+0000 7f08e094b640 1 --2- 172.21.15.139:0/333171146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08dc15da90 0x7f08dc15de70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.777+0000 7f08e094b640 1 -- 172.21.15.139:0/333171146 >> 172.21.15.139:0/333171146 conn(0x7f08dc06c150 msgr2=0x7f08dc067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.777+0000 7f08e094b640 1 -- 172.21.15.139:0/333171146 shutdown_connections 2024-09-16T00:10:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.777+0000 7f08e094b640 1 -- 172.21.15.139:0/333171146 wait complete. 2024-09-16T00:10:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.778+0000 7f08e094b640 1 Processor -- start 2024-09-16T00:10:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.778+0000 7f08e094b640 1 -- start start 2024-09-16T00:10:01.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.778+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2e240c9060 con 0x7f2e24153820 2024-09-16T00:10:01.781 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:01.781 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:01.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.778+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f2e241546d0 0x7f2e24154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.778+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f2e242097d0 con 0x7f2e241546d0 2024-09-16T00:10:01.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f2e23fff640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f2e241546d0 0x7f2e24154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08e094b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08dc070d90 0x7f08dc1f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08e094b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08dc0c8b50 0x7f08dc1e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08e094b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08dc15da90 0x7f08dc153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08e094b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc0ce8d0 con 0x7f08dc15da90 2024-09-16T00:10:01.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08e094b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc0caa00 con 0x7f08dc070d90 2024-09-16T00:10:01.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08e094b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f08dc0cea50 con 0x7f08dc0c8b50 2024-09-16T00:10:01.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08dad9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08dc15da90 0x7f08dc153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08da59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08dc070d90 0x7f08dc1f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08d9d9b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08dc0c8b50 0x7f08dc1e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08da59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08dc070d90 0x7f08dc1f3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:49808/0 (socket says 172.21.15.139:49808) 2024-09-16T00:10:01.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08da59c640 1 -- 172.21.15.139:0/636834703 learned_addr learned my addr 172.21.15.139:0/636834703 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:01.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f2e23fff640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f2e241546d0 0x7f2e24154b10 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:01.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08dad9d640 1 -- 172.21.15.139:0/636834703 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08dc0c8b50 msgr2=0x7f08dc1e65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08dad9d640 1 --2- 172.21.15.139:0/636834703 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08dc0c8b50 0x7f08dc1e65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08dad9d640 1 -- 172.21.15.139:0/636834703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08dc070d90 msgr2=0x7f08dc1f3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08dad9d640 1 --2- 172.21.15.139:0/636834703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08dc070d90 0x7f08dc1f3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08dad9d640 1 -- 172.21.15.139:0/636834703 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc1f45f0 con 0x7f08dc15da90 2024-09-16T00:10:01.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08da59c640 1 --2- 172.21.15.139:0/636834703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08dc070d90 0x7f08dc1f3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:01.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.779+0000 7f08dad9d640 1 --2- 172.21.15.139:0/636834703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08dc15da90 0x7f08dc153000 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0c1bc0 tx=0x7f08d400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:01.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.780+0000 7f08caffd640 1 -- 172.21.15.139:0/636834703 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4003b70 con 0x7f08dc15da90 2024-09-16T00:10:01.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.780+0000 7f08caffd640 1 -- 172.21.15.139:0/636834703 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d4003d30 con 0x7f08dc15da90 2024-09-16T00:10:01.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.780+0000 7f08caffd640 1 -- 172.21.15.139:0/636834703 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d40052b0 con 0x7f08dc15da90 2024-09-16T00:10:01.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.780+0000 7f08e094b640 1 -- 172.21.15.139:0/636834703 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08dc1f48b0 con 0x7f08dc15da90 2024-09-16T00:10:01.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.780+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e242097d0 con 0x7f2e241546d0 2024-09-16T00:10:01.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.780+0000 7f08e094b640 1 -- 172.21.15.139:0/636834703 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08dc1e4fb0 con 0x7f08dc15da90 2024-09-16T00:10:01.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.781+0000 7f08e094b640 1 -- 172.21.15.139:0/636834703 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08dc15a6e0 con 0x7f08dc15da90 2024-09-16T00:10:01.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.782+0000 7f08caffd640 1 -- 172.21.15.139:0/636834703 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f08d4005450 con 0x7f08dc15da90 2024-09-16T00:10:01.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.782+0000 7f08caffd640 1 --2- 172.21.15.139:0/636834703 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08bc06c2e0 0x7f08bc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.782+0000 7f08caffd640 1 -- 172.21.15.139:0/636834703 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5973+0+0 (secure 0 0 0) 0x7f08d408b340 con 0x7f08dc15da90 2024-09-16T00:10:01.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.782+0000 7f08da59c640 1 --2- 172.21.15.139:0/636834703 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08bc06c2e0 0x7f08bc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.783+0000 7f08da59c640 1 --2- 172.21.15.139:0/636834703 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08bc06c2e0 0x7f08bc06e7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0097a0 tx=0x7f08cc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:01.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.787+0000 7f08caffd640 1 -- 172.21.15.139:0/636834703 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f08dc15a6e0 con 0x7f08dc15da90 2024-09-16T00:10:01.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.796+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2e24206f40 con 0x7f2e241546d0 2024-09-16T00:10:01.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.798+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2e24206f40 con 0x7f2e241546d0 2024-09-16T00:10:01.798 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:01.799 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:01.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.798+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f2e24207660 0x7f2e24209a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.798+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f2e2420c930 con 0x7f2e24207660 2024-09-16T00:10:01.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.798+0000 7f2e29014640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f2e24207660 0x7f2e24209a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.798+0000 7f2e29014640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f2e24207660 0x7f2e24209a50 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:01.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.799+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e2420c930 con 0x7f2e24207660 2024-09-16T00:10:01.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.813+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2e2402d2b0 con 0x7f2e24207660 2024-09-16T00:10:01.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.816+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2e2402d2b0 con 0x7f2e24207660 2024-09-16T00:10:01.817 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:01.817 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:01.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.816+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f2e2420a4a0 0x7f2e2420a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:01.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.816+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f2e2420c570 con 0x7f2e2420a4a0 2024-09-16T00:10:01.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.816+0000 7f2e28813640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f2e2420a4a0 0x7f2e2420a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:01.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.817+0000 7f2e28813640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f2e2420a4a0 0x7f2e2420a920 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:01.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.818+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e2420c570 con 0x7f2e2420a4a0 2024-09-16T00:10:01.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.832+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2e2420ae60 con 0x7f2e2420a4a0 2024-09-16T00:10:01.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e217fa640 1 -- 172.21.15.139:0/3425160915 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2e2420ae60 con 0x7f2e2420a4a0 2024-09-16T00:10:01.835 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:10:01.836 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:10:01.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f2e241f40b0 msgr2=0x7f2e241f4490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f2e241f40b0 0x7f2e241f4490 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f2e2415d170 msgr2=0x7f2e24062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f2e2415d170 0x7f2e24062630 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f2e24063080 msgr2=0x7f2e240c8b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f2e24063080 0x7f2e240c8b20 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f2e2402c640 msgr2=0x7f2e2402caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f2e2402c640 0x7f2e2402caa0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f2e24153820 msgr2=0x7f2e24153c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f2e24153820 0x7f2e24153c80 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e29014640 1 -- 172.21.15.139:0/3425160915 reap_dead start 2024-09-16T00:10:01.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f2e241546d0 msgr2=0x7f2e24154b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f2e241546d0 0x7f2e24154b10 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e29014640 1 -- 172.21.15.139:0/3425160915 reap_dead start 2024-09-16T00:10:01.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e29014640 1 -- 172.21.15.139:0/3425160915 reap_dead start 2024-09-16T00:10:01.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f2e24207660 msgr2=0x7f2e24209a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f2e24207660 0x7f2e24209a50 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f2e2420a4a0 msgr2=0x7f2e2420a920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f2e2420a4a0 0x7f2e2420a920 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2e0806c2b0 msgr2=0x7f2e0806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2e0806c2b0 0x7f2e0806e770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009560 tx=0x7f2e18009190 comp rx=0 tx=0).stop 2024-09-16T00:10:01.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e240b2170 msgr2=0x7f2e241f3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:01.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.834+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e240b2170 0x7f2e241f3b70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c031000 tx=0x7f2e1c004a40 comp rx=0 tx=0).stop 2024-09-16T00:10:01.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.835+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 shutdown_connections 2024-09-16T00:10:01.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.835+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f2e24207660 0x7f2e24209a50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.835+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f2e2420a4a0 0x7f2e2420a920 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.835+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2e0806c2b0 0x7f2e0806e770 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.835+0000 7f2e2aa77640 1 --2- 172.21.15.139:0/3425160915 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e240b2170 0x7f2e241f3b70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:01.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.835+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 >> 172.21.15.139:0/3425160915 conn(0x7f2e24067950 msgr2=0x7f2e240ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:01.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.835+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 shutdown_connections 2024-09-16T00:10:01.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.835+0000 7f2e2aa77640 1 -- 172.21.15.139:0/3425160915 wait complete. 2024-09-16T00:10:01.849 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:01.849 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:01.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:01.849 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:01.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.850 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10780 3.a 3:500c83b2:::100000030b4.00000000:head [write 0~4194304 [3@2137521]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:01.850 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:01.850 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.393756741000001, 2024-09-16T00:10:01.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015526478, 2024-09-16T00:10:01.850 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.850 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.850 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.850 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10780 2024-09-16T00:10:01.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.850 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:01.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:01.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452917+0000", 2024-09-16T00:10:01.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2180000000000002e-06 2024-09-16T00:10:01.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462177+0000", 2024-09-16T00:10:01.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092593769999999992 2024-09-16T00:10:01.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462196+0000", 2024-09-16T00:10:01.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9745000000000002e-05 2024-09-16T00:10:01.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462207+0000", 2024-09-16T00:10:01.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0524e-05 2024-09-16T00:10:01.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462284+0000", 2024-09-16T00:10:01.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7608000000000006e-05 2024-09-16T00:10:01.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462376+0000", 2024-09-16T00:10:01.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1811999999999994e-05 2024-09-16T00:10:01.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468357+0000", 2024-09-16T00:10:01.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059806809999999998 2024-09-16T00:10:01.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468386+0000", 2024-09-16T00:10:01.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9068999999999999e-05 2024-09-16T00:10:01.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468441+0000", 2024-09-16T00:10:01.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5444e-05 2024-09-16T00:10:01.854 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.854 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.855 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.855 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10831 3.1a 3:59b8f98c:::100000030df.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:01.855 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:01.855 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.070949604999999, 2024-09-16T00:10:01.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014692843000000001, 2024-09-16T00:10:01.855 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.855 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.855 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.855 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.855 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.855 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.855 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10831 2024-09-16T00:10:01.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.856 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:01.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:01.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775723+0000", 2024-09-16T00:10:01.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.367e-06 2024-09-16T00:10:01.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780402+0000", 2024-09-16T00:10:01.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046785769999999997 2024-09-16T00:10:01.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780421+0000", 2024-09-16T00:10:01.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9179000000000001e-05 2024-09-16T00:10:01.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780440+0000", 2024-09-16T00:10:01.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9015e-05 2024-09-16T00:10:01.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780533+0000", 2024-09-16T00:10:01.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3197999999999998e-05 2024-09-16T00:10:01.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780618+0000", 2024-09-16T00:10:01.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4787e-05 2024-09-16T00:10:01.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:01.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780748+0000", 2024-09-16T00:10:01.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012927199999999999 2024-09-16T00:10:01.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.785049+0000", 2024-09-16T00:10:01.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043015750000000002 2024-09-16T00:10:01.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790312+0000", 2024-09-16T00:10:01.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052633319999999999 2024-09-16T00:10:01.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790369+0000", 2024-09-16T00:10:01.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6848999999999997e-05 2024-09-16T00:10:01.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790415+0000", 2024-09-16T00:10:01.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5692000000000002e-05 2024-09-16T00:10:01.861 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.861 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.861 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.861 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10915 3.1a 3:5a6e15a2:::1000000318e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:01.861 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:01.861 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.631443684000001, 2024-09-16T00:10:01.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014599472, 2024-09-16T00:10:01.862 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.862 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.862 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.862 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.862 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10915 2024-09-16T00:10:01.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.862 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:01.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:01.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215230+0000", 2024-09-16T00:10:01.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0090000000000002e-06 2024-09-16T00:10:01.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219556+0000", 2024-09-16T00:10:01.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043259539999999999 2024-09-16T00:10:01.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219577+0000", 2024-09-16T00:10:01.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1271999999999999e-05 2024-09-16T00:10:01.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219589+0000", 2024-09-16T00:10:01.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1766000000000001e-05 2024-09-16T00:10:01.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219663+0000", 2024-09-16T00:10:01.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4196999999999995e-05 2024-09-16T00:10:01.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219733+0000", 2024-09-16T00:10:01.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9553000000000003e-05 2024-09-16T00:10:01.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:01.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219827+0000", 2024-09-16T00:10:01.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4395999999999995e-05 2024-09-16T00:10:01.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.225724+0000", 2024-09-16T00:10:01.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058968359999999999 2024-09-16T00:10:01.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229705+0000", 2024-09-16T00:10:01.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003981409 2024-09-16T00:10:01.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229772+0000", 2024-09-16T00:10:01.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6137999999999996e-05 2024-09-16T00:10:01.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229828+0000", 2024-09-16T00:10:01.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5942000000000001e-05 2024-09-16T00:10:01.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.867 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.867 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11074 3.14 3:2834b7ee:::10000003131.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:01.867 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:01.867 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.933903561000001, 2024-09-16T00:10:01.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015588350000000001, 2024-09-16T00:10:01.868 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.868 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.868 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.868 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.868 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.868 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.868 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11074 2024-09-16T00:10:01.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.868 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:01.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:01.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912769+0000", 2024-09-16T00:10:01.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4600000000000004e-07 2024-09-16T00:10:01.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918264+0000", 2024-09-16T00:10:01.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054954890000000001 2024-09-16T00:10:01.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918281+0000", 2024-09-16T00:10:01.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6160000000000001e-05 2024-09-16T00:10:01.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918289+0000", 2024-09-16T00:10:01.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1650000000000006e-06 2024-09-16T00:10:01.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918386+0000", 2024-09-16T00:10:01.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7288999999999996e-05 2024-09-16T00:10:01.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918535+0000", 2024-09-16T00:10:01.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014876200000000001 2024-09-16T00:10:01.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:01.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918639+0000", 2024-09-16T00:10:01.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010470400000000001 2024-09-16T00:10:01.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.924703+0000", 2024-09-16T00:10:01.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060639759999999996 2024-09-16T00:10:01.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928246+0000", 2024-09-16T00:10:01.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035421720000000001 2024-09-16T00:10:01.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928305+0000", 2024-09-16T00:10:01.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9014000000000001e-05 2024-09-16T00:10:01.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928357+0000", 2024-09-16T00:10:01.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1873000000000003e-05 2024-09-16T00:10:01.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.873 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.873 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11111 3.9 3:93301cc4:::10000003210.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:01.873 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:01.873 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.757165358999998, 2024-09-16T00:10:01.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014336099999999999, 2024-09-16T00:10:01.873 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.874 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.874 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.874 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11111 2024-09-16T00:10:01.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.874 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:01.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:01.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089520+0000", 2024-09-16T00:10:01.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3903000000000001e-05 2024-09-16T00:10:01.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094463+0000", 2024-09-16T00:10:01.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049423000000000002 2024-09-16T00:10:01.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094494+0000", 2024-09-16T00:10:01.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1029999999999999e-05 2024-09-16T00:10:01.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094503+0000", 2024-09-16T00:10:01.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0180000000000007e-06 2024-09-16T00:10:01.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094573+0000", 2024-09-16T00:10:01.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0457000000000007e-05 2024-09-16T00:10:01.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094737+0000", 2024-09-16T00:10:01.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016408699999999999 2024-09-16T00:10:01.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:01.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094897+0000", 2024-09-16T00:10:01.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015938899999999999 2024-09-16T00:10:01.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.102326+0000", 2024-09-16T00:10:01.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074291230000000002 2024-09-16T00:10:01.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103723+0000", 2024-09-16T00:10:01.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001397255 2024-09-16T00:10:01.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103786+0000", 2024-09-16T00:10:01.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3489000000000001e-05 2024-09-16T00:10:01.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103842+0000", 2024-09-16T00:10:01.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6048999999999998e-05 2024-09-16T00:10:01.879 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.879 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.880 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.880 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:11934 3.1b e87/20 3:d94fa8da:::10000003190.00000000:head v 87'152, mlcod=87'150)", 2024-09-16T00:10:01.880 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:01.880 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.156420662999999, 2024-09-16T00:10:01.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014984934, 2024-09-16T00:10:01.880 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.880 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.880 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.880 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:01.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:01.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690252+0000", 2024-09-16T00:10:01.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8899999999999998e-07 2024-09-16T00:10:01.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696026+0000", 2024-09-16T00:10:01.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057745219999999998 2024-09-16T00:10:01.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696039+0000", 2024-09-16T00:10:01.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2611e-05 2024-09-16T00:10:01.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696047+0000", 2024-09-16T00:10:01.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5399999999999998e-06 2024-09-16T00:10:01.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696123+0000", 2024-09-16T00:10:01.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6349999999999999e-05 2024-09-16T00:10:01.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696144+0000", 2024-09-16T00:10:01.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1512999999999999e-05 2024-09-16T00:10:01.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705188+0000", 2024-09-16T00:10:01.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090438459999999995 2024-09-16T00:10:01.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705236+0000", 2024-09-16T00:10:01.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7663000000000001e-05 2024-09-16T00:10:01.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.884 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.884 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11936 3.3 3:c0886fb9:::100000031a1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:01.885 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:01.885 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.146275611, 2024-09-16T00:10:01.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015336379000000001, 2024-09-16T00:10:01.885 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.885 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.885 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.885 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.885 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.885 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.885 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11936 2024-09-16T00:10:01.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.885 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:01.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:01.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700399+0000", 2024-09-16T00:10:01.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6029999999999999e-06 2024-09-16T00:10:01.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704706+0000", 2024-09-16T00:10:01.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043070529999999999 2024-09-16T00:10:01.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704720+0000", 2024-09-16T00:10:01.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4629e-05 2024-09-16T00:10:01.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704729+0000", 2024-09-16T00:10:01.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6009999999999995e-06 2024-09-16T00:10:01.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704805+0000", 2024-09-16T00:10:01.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6015999999999999e-05 2024-09-16T00:10:01.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704866+0000", 2024-09-16T00:10:01.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0878999999999998e-05 2024-09-16T00:10:01.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:01.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704990+0000", 2024-09-16T00:10:01.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000124058 2024-09-16T00:10:01.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.714314+0000", 2024-09-16T00:10:01.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009324321 2024-09-16T00:10:01.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715632+0000", 2024-09-16T00:10:01.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001318121 2024-09-16T00:10:01.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715687+0000", 2024-09-16T00:10:01.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4135999999999998e-05 2024-09-16T00:10:01.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715732+0000", 2024-09-16T00:10:01.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961999999999998e-05 2024-09-16T00:10:01.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.890 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.891 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11953 3.1a 3:580e2503:::1000000311b.00000000:head [write 0~4194304 [2@2058066]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:01.891 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:01.891 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.079936457999999, 2024-09-16T00:10:01.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016299973999999998, 2024-09-16T00:10:01.891 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.891 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.891 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.891 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11953 2024-09-16T00:10:01.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.891 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:01.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:01.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766741+0000", 2024-09-16T00:10:01.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2939999999999996e-06 2024-09-16T00:10:01.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771165+0000", 2024-09-16T00:10:01.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044248229999999996 2024-09-16T00:10:01.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771181+0000", 2024-09-16T00:10:01.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5243e-05 2024-09-16T00:10:01.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771188+0000", 2024-09-16T00:10:01.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6429999999999995e-06 2024-09-16T00:10:01.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771238+0000", 2024-09-16T00:10:01.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9567999999999997e-05 2024-09-16T00:10:01.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771293+0000", 2024-09-16T00:10:01.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5207000000000002e-05 2024-09-16T00:10:01.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:01.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771380+0000", 2024-09-16T00:10:01.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6716000000000001e-05 2024-09-16T00:10:01.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.776193+0000", 2024-09-16T00:10:01.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048133350000000002 2024-09-16T00:10:01.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.782931+0000", 2024-09-16T00:10:01.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067376550000000004 2024-09-16T00:10:01.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.782987+0000", 2024-09-16T00:10:01.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6443000000000002e-05 2024-09-16T00:10:01.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.783035+0000", 2024-09-16T00:10:01.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8047000000000003e-05 2024-09-16T00:10:01.896 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.896 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.896 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.896 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12037 3.9 3:928db521:::1000000326d.00000000:head [write 0~4194304 [2@1655200]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:01.897 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:01.897 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.848728850000001, 2024-09-16T00:10:01.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017082494, 2024-09-16T00:10:01.897 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.897 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.897 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.897 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12037 2024-09-16T00:10:01.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.897 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:01.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:01.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997944+0000", 2024-09-16T00:10:01.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.517e-06 2024-09-16T00:10:01.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002177+0000", 2024-09-16T00:10:01.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042329300000000002 2024-09-16T00:10:01.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002197+0000", 2024-09-16T00:10:01.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9769e-05 2024-09-16T00:10:01.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002207+0000", 2024-09-16T00:10:01.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0178e-05 2024-09-16T00:10:01.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002266+0000", 2024-09-16T00:10:01.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9215999999999997e-05 2024-09-16T00:10:01.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002340+0000", 2024-09-16T00:10:01.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3798000000000001e-05 2024-09-16T00:10:01.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:01.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002454+0000", 2024-09-16T00:10:01.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113551 2024-09-16T00:10:01.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.006270+0000", 2024-09-16T00:10:01.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038163839999999999 2024-09-16T00:10:01.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.014957+0000", 2024-09-16T00:10:01.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086869119999999998 2024-09-16T00:10:01.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.014996+0000", 2024-09-16T00:10:01.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8878000000000003e-05 2024-09-16T00:10:01.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.015025+0000", 2024-09-16T00:10:01.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9360999999999999e-05 2024-09-16T00:10:01.902 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.902 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.902 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.902 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12245 3.1a 3:599a1cd1:::10000003252.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:01.902 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:01.902 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 17.064031433, 2024-09-16T00:10:01.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017131203000000001, 2024-09-16T00:10:01.903 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.903 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.903 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.903 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.903 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.903 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.903 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12245 2024-09-16T00:10:01.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.903 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:01.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:01.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782642+0000", 2024-09-16T00:10:01.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.311e-06 2024-09-16T00:10:01.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786551+0000", 2024-09-16T00:10:01.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039095229999999998 2024-09-16T00:10:01.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786566+0000", 2024-09-16T00:10:01.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4724e-05 2024-09-16T00:10:01.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786575+0000", 2024-09-16T00:10:01.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8449999999999992e-06 2024-09-16T00:10:01.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786636+0000", 2024-09-16T00:10:01.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.143e-05 2024-09-16T00:10:01.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786699+0000", 2024-09-16T00:10:01.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3218000000000004e-05 2024-09-16T00:10:01.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:01.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786796+0000", 2024-09-16T00:10:01.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6347000000000004e-05 2024-09-16T00:10:01.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.792369+0000", 2024-09-16T00:10:01.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055729780000000001 2024-09-16T00:10:01.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799706+0000", 2024-09-16T00:10:01.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073374349999999998 2024-09-16T00:10:01.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799739+0000", 2024-09-16T00:10:01.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2833999999999998e-05 2024-09-16T00:10:01.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799771+0000", 2024-09-16T00:10:01.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2558000000000002e-05 2024-09-16T00:10:01.908 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.908 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.908 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.908 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12292 3.3 3:c00779fd:::1000000320e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:01.908 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:01.908 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 16.949641335999999, 2024-09-16T00:10:01.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013828353999999999, 2024-09-16T00:10:01.909 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.909 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.909 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.909 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.909 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.909 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.909 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12292 2024-09-16T00:10:01.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.909 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:01.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:01.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897039+0000", 2024-09-16T00:10:01.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3299999999999999e-06 2024-09-16T00:10:01.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900830+0000", 2024-09-16T00:10:01.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037915340000000001 2024-09-16T00:10:01.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900850+0000", 2024-09-16T00:10:01.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9400000000000001e-05 2024-09-16T00:10:01.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900860+0000", 2024-09-16T00:10:01.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0609e-05 2024-09-16T00:10:01.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900928+0000", 2024-09-16T00:10:01.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7570000000000005e-05 2024-09-16T00:10:01.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.901027+0000", 2024-09-16T00:10:01.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9456000000000005e-05 2024-09-16T00:10:01.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:01.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.901168+0000", 2024-09-16T00:10:01.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014082799999999999 2024-09-16T00:10:01.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.906325+0000", 2024-09-16T00:10:01.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005156702 2024-09-16T00:10:01.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910796+0000", 2024-09-16T00:10:01.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044714350000000002 2024-09-16T00:10:01.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910829+0000", 2024-09-16T00:10:01.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3229000000000003e-05 2024-09-16T00:10:01.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910859+0000", 2024-09-16T00:10:01.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9261e-05 2024-09-16T00:10:01.914 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.914 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.914 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.914 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13326 3.9 3:92f796cf:::1000000350f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:01.914 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:01.914 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.889951054000001, 2024-09-16T00:10:01.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015673229, 2024-09-16T00:10:01.914 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.914 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.914 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.915 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13326 2024-09-16T00:10:01.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.915 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:01.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:01.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956722+0000", 2024-09-16T00:10:01.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3200000000000001e-06 2024-09-16T00:10:01.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960803+0000", 2024-09-16T00:10:01.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040807029999999998 2024-09-16T00:10:01.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960820+0000", 2024-09-16T00:10:01.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7478000000000001e-05 2024-09-16T00:10:01.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960830+0000", 2024-09-16T00:10:01.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:10:01.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960903+0000", 2024-09-16T00:10:01.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3057000000000002e-05 2024-09-16T00:10:01.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960970+0000", 2024-09-16T00:10:01.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7063000000000005e-05 2024-09-16T00:10:01.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:01.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.961067+0000", 2024-09-16T00:10:01.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6906999999999999e-05 2024-09-16T00:10:01.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.967945+0000", 2024-09-16T00:10:01.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068787359999999999 2024-09-16T00:10:01.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972328+0000", 2024-09-16T00:10:01.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043826170000000001 2024-09-16T00:10:01.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972363+0000", 2024-09-16T00:10:01.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4486000000000002e-05 2024-09-16T00:10:01.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972394+0000", 2024-09-16T00:10:01.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1343999999999997e-05 2024-09-16T00:10:01.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.920 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.920 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13429 3.1a 3:58dcd24a:::100000036d5.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:01.920 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:01.920 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.535381038000001, 2024-09-16T00:10:01.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014237671, 2024-09-16T00:10:01.920 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.920 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.920 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.920 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.920 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.921 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13429 2024-09-16T00:10:01.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.921 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:01.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:01.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311292+0000", 2024-09-16T00:10:01.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4890000000000001e-06 2024-09-16T00:10:01.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316378+0000", 2024-09-16T00:10:01.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050859290000000003 2024-09-16T00:10:01.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316399+0000", 2024-09-16T00:10:01.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1345e-05 2024-09-16T00:10:01.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316410+0000", 2024-09-16T00:10:01.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0295999999999999e-05 2024-09-16T00:10:01.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316493+0000", 2024-09-16T00:10:01.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3533000000000001e-05 2024-09-16T00:10:01.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316622+0000", 2024-09-16T00:10:01.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012887900000000001 2024-09-16T00:10:01.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:01.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316725+0000", 2024-09-16T00:10:01.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010295599999999999 2024-09-16T00:10:01.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.320948+0000", 2024-09-16T00:10:01.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042229820000000001 2024-09-16T00:10:01.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325415+0000", 2024-09-16T00:10:01.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044673849999999999 2024-09-16T00:10:01.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325487+0000", 2024-09-16T00:10:01.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1087999999999994e-05 2024-09-16T00:10:01.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325528+0000", 2024-09-16T00:10:01.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1789000000000003e-05 2024-09-16T00:10:01.925 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.925 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.926 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.926 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13527 3.3 3:c0886fb9:::100000031a1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:01.926 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:01.926 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.061220949000001, 2024-09-16T00:10:01.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015187236, 2024-09-16T00:10:01.926 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.926 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.926 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.926 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.926 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13527 2024-09-16T00:10:01.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.927 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:01.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:01.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785453+0000", 2024-09-16T00:10:01.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1689999999999999e-06 2024-09-16T00:10:01.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791191+0000", 2024-09-16T00:10:01.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057378339999999998 2024-09-16T00:10:01.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791217+0000", 2024-09-16T00:10:01.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6480999999999999e-05 2024-09-16T00:10:01.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791233+0000", 2024-09-16T00:10:01.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5841e-05 2024-09-16T00:10:01.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791319+0000", 2024-09-16T00:10:01.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5527999999999998e-05 2024-09-16T00:10:01.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791413+0000", 2024-09-16T00:10:01.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4685000000000006e-05 2024-09-16T00:10:01.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:01.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791563+0000", 2024-09-16T00:10:01.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000149624 2024-09-16T00:10:01.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.795749+0000", 2024-09-16T00:10:01.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004186286 2024-09-16T00:10:01.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800577+0000", 2024-09-16T00:10:01.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048273040000000001 2024-09-16T00:10:01.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800611+0000", 2024-09-16T00:10:01.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4270000000000002e-05 2024-09-16T00:10:01.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800638+0000", 2024-09-16T00:10:01.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7214000000000001e-05 2024-09-16T00:10:01.931 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.931 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.931 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.932 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13724 3.9 3:90613e6c:::100000034c4.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:01.932 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:01.932 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.391149204, 2024-09-16T00:10:01.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014039800999999999, 2024-09-16T00:10:01.932 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.932 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.932 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.932 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.932 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.932 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.932 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13724 2024-09-16T00:10:01.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.932 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:01.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:01.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455524+0000", 2024-09-16T00:10:01.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.437e-06 2024-09-16T00:10:01.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459592+0000", 2024-09-16T00:10:01.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040678729999999996 2024-09-16T00:10:01.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459610+0000", 2024-09-16T00:10:01.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7688000000000001e-05 2024-09-16T00:10:01.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459617+0000", 2024-09-16T00:10:01.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.374e-06 2024-09-16T00:10:01.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459678+0000", 2024-09-16T00:10:01.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1346999999999994e-05 2024-09-16T00:10:01.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459824+0000", 2024-09-16T00:10:01.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014586699999999999 2024-09-16T00:10:01.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:01.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459908+0000", 2024-09-16T00:10:01.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3521000000000002e-05 2024-09-16T00:10:01.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.464227+0000", 2024-09-16T00:10:01.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043197060000000004 2024-09-16T00:10:01.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469499+0000", 2024-09-16T00:10:01.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005271838 2024-09-16T00:10:01.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469534+0000", 2024-09-16T00:10:01.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4712000000000002e-05 2024-09-16T00:10:01.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469562+0000", 2024-09-16T00:10:01.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8438000000000001e-05 2024-09-16T00:10:01.937 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.937 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.937 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.938 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13741 3.3 3:c1377c76:::10000003664.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:01.938 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:01.938 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 12.343979258999999, 2024-09-16T00:10:01.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016142843, 2024-09-16T00:10:01.938 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.938 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.938 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.938 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.938 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.938 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.938 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13741 2024-09-16T00:10:01.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.938 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:01.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:01.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502694+0000", 2024-09-16T00:10:01.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0449999999999999e-06 2024-09-16T00:10:01.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507229+0000", 2024-09-16T00:10:01.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045357410000000003 2024-09-16T00:10:01.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507248+0000", 2024-09-16T00:10:01.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9127000000000002e-05 2024-09-16T00:10:01.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507259+0000", 2024-09-16T00:10:01.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0448999999999999e-05 2024-09-16T00:10:01.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507357+0000", 2024-09-16T00:10:01.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8400000000000007e-05 2024-09-16T00:10:01.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507514+0000", 2024-09-16T00:10:01.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000157195 2024-09-16T00:10:01.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:01.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507617+0000", 2024-09-16T00:10:01.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010262199999999999 2024-09-16T00:10:01.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.514047+0000", 2024-09-16T00:10:01.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006430018 2024-09-16T00:10:01.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518743+0000", 2024-09-16T00:10:01.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046963040000000001 2024-09-16T00:10:01.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518796+0000", 2024-09-16T00:10:01.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2305000000000002e-05 2024-09-16T00:10:01.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518835+0000", 2024-09-16T00:10:01.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9637e-05 2024-09-16T00:10:01.943 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.943 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.943 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.943 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14633 3.9 3:9170e82c:::100000035d6.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:01.943 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:01.943 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.2245990029999998, 2024-09-16T00:10:01.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014466105, 2024-09-16T00:10:01.944 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.944 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.944 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.944 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.944 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14633 2024-09-16T00:10:01.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.944 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:01.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:01.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622074+0000", 2024-09-16T00:10:01.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.761e-06 2024-09-16T00:10:01.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626712+0000", 2024-09-16T00:10:01.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004637263 2024-09-16T00:10:01.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626724+0000", 2024-09-16T00:10:01.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2479e-05 2024-09-16T00:10:01.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626730+0000", 2024-09-16T00:10:01.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7309999999999998e-06 2024-09-16T00:10:01.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626791+0000", 2024-09-16T00:10:01.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0854999999999999e-05 2024-09-16T00:10:01.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626857+0000", 2024-09-16T00:10:01.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6549000000000003e-05 2024-09-16T00:10:01.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:01.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626943+0000", 2024-09-16T00:10:01.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6141000000000001e-05 2024-09-16T00:10:01.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.630854+0000", 2024-09-16T00:10:01.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039100790000000003 2024-09-16T00:10:01.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636468+0000", 2024-09-16T00:10:01.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056140570000000004 2024-09-16T00:10:01.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636500+0000", 2024-09-16T00:10:01.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2441999999999999e-05 2024-09-16T00:10:01.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636539+0000", 2024-09-16T00:10:01.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8748000000000002e-05 2024-09-16T00:10:01.949 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.949 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.949 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.949 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14667 3.9 3:9379a4af:::10000003352.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:01.949 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:01.949 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 9.0495869970000005, 2024-09-16T00:10:01.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014580585, 2024-09-16T00:10:01.949 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.950 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.950 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.950 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.950 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14667 2024-09-16T00:10:01.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.950 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:01.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:01.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797086+0000", 2024-09-16T00:10:01.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.362e-06 2024-09-16T00:10:01.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801415+0000", 2024-09-16T00:10:01.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043288449999999996 2024-09-16T00:10:01.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801435+0000", 2024-09-16T00:10:01.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0305999999999998e-05 2024-09-16T00:10:01.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801442+0000", 2024-09-16T00:10:01.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6290000000000004e-06 2024-09-16T00:10:01.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801497+0000", 2024-09-16T00:10:01.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5111999999999997e-05 2024-09-16T00:10:01.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801639+0000", 2024-09-16T00:10:01.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014168000000000001 2024-09-16T00:10:01.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:01.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801788+0000", 2024-09-16T00:10:01.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014963300000000001 2024-09-16T00:10:01.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.805407+0000", 2024-09-16T00:10:01.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036186119999999998 2024-09-16T00:10:01.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811584+0000", 2024-09-16T00:10:01.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061772889999999999 2024-09-16T00:10:01.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811635+0000", 2024-09-16T00:10:01.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0453e-05 2024-09-16T00:10:01.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811665+0000", 2024-09-16T00:10:01.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0664000000000003e-05 2024-09-16T00:10:01.955 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.955 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.955 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.955 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14951 3.1a 3:59aee80a:::1000000394f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:01.955 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:01.955 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.8795491020000004, 2024-09-16T00:10:01.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015993941000000001, 2024-09-16T00:10:01.955 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.955 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.956 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.956 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.956 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14951 2024-09-16T00:10:01.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.956 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:01.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:01.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967124+0000", 2024-09-16T00:10:01.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.86e-06 2024-09-16T00:10:01.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971258+0000", 2024-09-16T00:10:01.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041334780000000003 2024-09-16T00:10:01.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971274+0000", 2024-09-16T00:10:01.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6201e-05 2024-09-16T00:10:01.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971284+0000", 2024-09-16T00:10:01.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4779999999999997e-06 2024-09-16T00:10:01.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971384+0000", 2024-09-16T00:10:01.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9878999999999997e-05 2024-09-16T00:10:01.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971518+0000", 2024-09-16T00:10:01.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013459400000000001 2024-09-16T00:10:01.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:01.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971604+0000", 2024-09-16T00:10:01.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6215e-05 2024-09-16T00:10:01.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.976673+0000", 2024-09-16T00:10:01.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050685920000000002 2024-09-16T00:10:01.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983043+0000", 2024-09-16T00:10:01.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063698410000000002 2024-09-16T00:10:01.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983082+0000", 2024-09-16T00:10:01.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9261999999999998e-05 2024-09-16T00:10:01.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983117+0000", 2024-09-16T00:10:01.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4541e-05 2024-09-16T00:10:01.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.961 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.961 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.961 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:15152 3.3 3:c388c7e2:::100000032ec.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-16T00:10:01.961 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:01.961 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.1190029920000004, 2024-09-16T00:10:01.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014603532000000001, 2024-09-16T00:10:01.961 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:01.961 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:01.961 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:01.961 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:01.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:01.962 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:01.962 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15152 2024-09-16T00:10:01.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.962 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:01.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:01.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:01.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:01.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:01.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:01.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:01.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727674+0000", 2024-09-16T00:10:01.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7699999999999998e-06 2024-09-16T00:10:01.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:01.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732274+0000", 2024-09-16T00:10:01.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045994299999999998 2024-09-16T00:10:01.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:01.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732286+0000", 2024-09-16T00:10:01.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2016e-05 2024-09-16T00:10:01.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:01.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732294+0000", 2024-09-16T00:10:01.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7049999999999999e-06 2024-09-16T00:10:01.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:01.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732359+0000", 2024-09-16T00:10:01.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5296000000000006e-05 2024-09-16T00:10:01.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:01.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732501+0000", 2024-09-16T00:10:01.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000141731 2024-09-16T00:10:01.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:01.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732629+0000", 2024-09-16T00:10:01.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012841299999999999 2024-09-16T00:10:01.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:01.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.737511+0000", 2024-09-16T00:10:01.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048822520000000001 2024-09-16T00:10:01.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:01.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742172+0000", 2024-09-16T00:10:01.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046610610000000002 2024-09-16T00:10:01.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:01.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742227+0000", 2024-09-16T00:10:01.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4497999999999999e-05 2024-09-16T00:10:01.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:01.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:01.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:01.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742272+0000", 2024-09-16T00:10:01.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5359999999999999e-05 2024-09-16T00:10:01.966 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.966 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.967 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.967 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:01.967 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:01.967 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:01.968 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:10:01.969 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:01.968+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:01.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:01.991+0000 7f08e094b640 1 -- 172.21.15.139:0/636834703 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) -- 0x7f08dc0ca720 con 0x7f08dc15da90 2024-09-16T00:10:02.069 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:02.069+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:02.115 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:02.115 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:02.115 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:02.115 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:02.124 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:10:02.169 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:02.169+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:02.270 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:02.270+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:02.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.270+0000 7f08caffd640 1 -- 172.21.15.139:0/636834703 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v96) ==== 113+0+0 (secure 0 0 0) 0x7f08dc0ca720 con 0x7f08dc15da90 2024-09-16T00:10:02.271 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T00:10:02.272 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:02.272 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:02.272 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:02.272 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:02.272 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:02.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.272+0000 7f08e094b640 1 -- 172.21.15.139:0/636834703 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08bc06c2e0 msgr2=0x7f08bc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:02.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.272+0000 7f08e094b640 1 --2- 172.21.15.139:0/636834703 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08bc06c2e0 0x7f08bc06e7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0097a0 tx=0x7f08cc02e000 comp rx=0 tx=0).stop 2024-09-16T00:10:02.273 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.272+0000 7f08e094b640 1 -- 172.21.15.139:0/636834703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08dc15da90 msgr2=0x7f08dc153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:02.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.272+0000 7f08e094b640 1 --2- 172.21.15.139:0/636834703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08dc15da90 0x7f08dc153000 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0c1bc0 tx=0x7f08d400cb30 comp rx=0 tx=0).stop 2024-09-16T00:10:02.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.272+0000 7f08e094b640 1 -- 172.21.15.139:0/636834703 shutdown_connections 2024-09-16T00:10:02.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.272+0000 7f08e094b640 1 --2- 172.21.15.139:0/636834703 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08bc06c2e0 0x7f08bc06e7a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:02.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.272+0000 7f08e094b640 1 --2- 172.21.15.139:0/636834703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08dc15da90 0x7f08dc153000 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:02.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.272+0000 7f08e094b640 1 --2- 172.21.15.139:0/636834703 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08dc0c8b50 0x7f08dc1e65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:02.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.272+0000 7f08e094b640 1 --2- 172.21.15.139:0/636834703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f08dc070d90 0x7f08dc1f3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:02.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.272+0000 7f08e094b640 1 -- 172.21.15.139:0/636834703 >> 172.21.15.139:0/636834703 conn(0x7f08dc06c150 msgr2=0x7f08dc159fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:02.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.272+0000 7f08e094b640 1 -- 172.21.15.139:0/636834703 shutdown_connections 2024-09-16T00:10:02.274 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.272+0000 7f08e094b640 1 -- 172.21.15.139:0/636834703 wait complete. 2024-09-16T00:10:02.284 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:10:02.285 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T00:10:02.293 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T00:10:02.370 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:02.370+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:02.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.435+0000 7f5e95add640 1 Processor -- start 2024-09-16T00:10:02.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.435+0000 7f5e95add640 1 -- start start 2024-09-16T00:10:02.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.436+0000 7f5e95add640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5e9015b280 0x7f5e90166190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:02.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.436+0000 7f5e95add640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e900cc4b0 con 0x7f5e90154c20 2024-09-16T00:10:02.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.436+0000 7f5e95add640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e900cc790 con 0x7f5e90068060 2024-09-16T00:10:02.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.436+0000 7f5e95add640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5e900c5830 con 0x7f5e9015b280 2024-09-16T00:10:02.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.436+0000 7f5e8f7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5e90154c20 0x7f5e900c5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:58162/0 (socket says 172.21.15.139:58162) 2024-09-16T00:10:02.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.436+0000 7f5e8f7fe640 1 -- 172.21.15.139:0/2514231304 learned_addr learned my addr 172.21.15.139:0/2514231304 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:02.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.436+0000 7f5e8ffff640 1 --2- 172.21.15.139:0/2514231304 >> v2:172.21.15.164:3300/0 conn(0x7f5e9015b280 0x7f5e90166190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:02.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.436+0000 7f5e8e7fc640 1 -- 172.21.15.139:0/2514231304 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2588375687 0 0) 0x7f5e900cc4b0 con 0x7f5e90154c20 2024-09-16T00:10:02.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.436+0000 7f5e8e7fc640 1 -- 172.21.15.139:0/2514231304 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e78003520 con 0x7f5e90154c20 2024-09-16T00:10:02.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.436+0000 7f5e8ffff640 1 -- 172.21.15.139:0/2514231304 >> v1:172.21.15.164:6789/0 conn(0x7f5e90068060 legacy=0x7f5e900ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:02.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.436+0000 7f5e8ffff640 1 -- 172.21.15.139:0/2514231304 >> v1:172.21.15.139:6789/0 conn(0x7f5e90154c20 legacy=0x7f5e900c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:02.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.437+0000 7f5e8ffff640 1 -- 172.21.15.139:0/2514231304 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9015ddf0 con 0x7f5e9015b280 2024-09-16T00:10:02.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.437+0000 7fc4153cf640 1 Processor -- start 2024-09-16T00:10:02.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.437+0000 7fc4153cf640 1 -- start start 2024-09-16T00:10:02.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.438+0000 7f5e8ffff640 1 --2- 172.21.15.139:0/2514231304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e9015b280 0x7f5e90166190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009d90 tx=0x7f5e8802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ece00acc1e4d4814 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:02.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.438+0000 7f5e8e7fc640 1 -- 172.21.15.139:0/2514231304 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e88007740 con 0x7f5e9015b280 2024-09-16T00:10:02.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.438+0000 7f5e8e7fc640 1 -- 172.21.15.139:0/2514231304 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e880078e0 con 0x7f5e9015b280 2024-09-16T00:10:02.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.438+0000 7f5e8e7fc640 1 -- 172.21.15.139:0/2514231304 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e88007c90 con 0x7f5e9015b280 2024-09-16T00:10:02.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.438+0000 7fc4153cf640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fc41006d080 0x7fc4100c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:02.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.438+0000 7fc4153cf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4100c64a0 0x7fc4100730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:02.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.438+0000 7fc4153cf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc410159660 con 0x7fc4100c60c0 2024-09-16T00:10:02.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.438+0000 7fc4153cf640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc410159140 con 0x7fc4100c64a0 2024-09-16T00:10:02.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.438+0000 7fc4153cf640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fc410072360 con 0x7fc41006d080 2024-09-16T00:10:02.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.438+0000 7fc40effd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc4100c60c0 0x7fc410152590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:58178/0 (socket says 172.21.15.139:58178) 2024-09-16T00:10:02.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.438+0000 7fc40effd640 1 -- 172.21.15.139:0/3947918887 learned_addr learned my addr 172.21.15.139:0/3947918887 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:02.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.438+0000 7fc40f7fe640 1 --2- 172.21.15.139:0/3947918887 >> v2:172.21.15.139:3300/0 conn(0x7fc4100c64a0 0x7fc4100730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:02.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.438+0000 7fc407fff640 1 --2- 172.21.15.139:0/3947918887 >> v2:172.21.15.164:3300/0 conn(0x7fc41006d080 0x7fc4100c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:02.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.438+0000 7fc40e7fc640 1 -- 172.21.15.139:0/3947918887 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4285414081 0 0) 0x7fc410159660 con 0x7fc4100c60c0 2024-09-16T00:10:02.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.438+0000 7fc40e7fc640 1 -- 172.21.15.139:0/3947918887 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3fc003520 con 0x7fc4100c60c0 2024-09-16T00:10:02.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.439+0000 7fc40f7fe640 1 -- 172.21.15.139:0/3947918887 >> v2:172.21.15.164:3300/0 conn(0x7fc41006d080 msgr2=0x7fc4100c7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:02.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.439+0000 7fc40f7fe640 1 --2- 172.21.15.139:0/3947918887 >> v2:172.21.15.164:3300/0 conn(0x7fc41006d080 0x7fc4100c7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:02.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.439+0000 7fc40f7fe640 1 -- 172.21.15.139:0/3947918887 >> v1:172.21.15.139:6789/0 conn(0x7fc4100c60c0 legacy=0x7fc410152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:02.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.439+0000 7f5e95add640 1 -- 172.21.15.139:0/2514231304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e9015b280 msgr2=0x7f5e90166190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:02.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.439+0000 7fc40f7fe640 1 -- 172.21.15.139:0/3947918887 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc410067a80 con 0x7fc4100c64a0 2024-09-16T00:10:02.444 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:02.444 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:02.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:02.444 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:02.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.445 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10658 3.1c 3:38c4c24e:::100000030bb.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:02.445 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:02.445 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 26.646953006, 2024-09-16T00:10:02.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021232445999999999, 2024-09-16T00:10:02.445 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.445 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.445 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:02.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:02.445 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:02.445 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10658 2024-09-16T00:10:02.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.446 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:02.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:02.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791398+0000", 2024-09-16T00:10:02.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3880000000000001e-06 2024-09-16T00:10:02.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800951+0000", 2024-09-16T00:10:02.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095532110000000007 2024-09-16T00:10:02.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800966+0000", 2024-09-16T00:10:02.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5058999999999999e-05 2024-09-16T00:10:02.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800974+0000", 2024-09-16T00:10:02.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1240000000000005e-06 2024-09-16T00:10:02.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801127+0000", 2024-09-16T00:10:02.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000153103 2024-09-16T00:10:02.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801207+0000", 2024-09-16T00:10:02.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9569999999999999e-05 2024-09-16T00:10:02.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801322+0000", 2024-09-16T00:10:02.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115358 2024-09-16T00:10:02.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.805760+0000", 2024-09-16T00:10:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044379409999999999 2024-09-16T00:10:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812528+0000", 2024-09-16T00:10:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067678249999999999 2024-09-16T00:10:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812586+0000", 2024-09-16T00:10:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8164000000000001e-05 2024-09-16T00:10:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812629+0000", 2024-09-16T00:10:02.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2703000000000001e-05 2024-09-16T00:10:02.451 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.451 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.451 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.451 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4122 2.18 2:1cc859d9:::200.00000012:head [write 798310~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.451 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:02.451 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7794133260000002, 2024-09-16T00:10:02.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019267580999999999, 2024-09-16T00:10:02.451 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.452 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.452 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.452 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:02.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:02.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658937+0000", 2024-09-16T00:10:02.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7700000000000004e-07 2024-09-16T00:10:02.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658942+0000", 2024-09-16T00:10:02.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.48e-06 2024-09-16T00:10:02.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658952+0000", 2024-09-16T00:10:02.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9469999999999998e-06 2024-09-16T00:10:02.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658959+0000", 2024-09-16T00:10:02.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4400000000000002e-06 2024-09-16T00:10:02.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670507+0000", 2024-09-16T00:10:02.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011548159000000001 2024-09-16T00:10:02.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670579+0000", 2024-09-16T00:10:02.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2082999999999994e-05 2024-09-16T00:10:02.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670686+0000", 2024-09-16T00:10:02.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107069 2024-09-16T00:10:02.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678001+0000", 2024-09-16T00:10:02.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073148780000000004 2024-09-16T00:10:02.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678104+0000", 2024-09-16T00:10:02.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010290599999999999 2024-09-16T00:10:02.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678163+0000", 2024-09-16T00:10:02.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8859e-05 2024-09-16T00:10:02.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678204+0000", 2024-09-16T00:10:02.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0982999999999997e-05 2024-09-16T00:10:02.457 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.457 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.457 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.457 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4123 2.18 2:1cc859d9:::200.00000012:head [write 799709~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.457 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:02.458 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7788441449999999, 2024-09-16T00:10:02.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018857569000000001, 2024-09-16T00:10:02.458 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.458 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.458 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.458 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:02.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:02.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659506+0000", 2024-09-16T00:10:02.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.78e-07 2024-09-16T00:10:02.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659512+0000", 2024-09-16T00:10:02.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0100000000000001e-06 2024-09-16T00:10:02.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659523+0000", 2024-09-16T00:10:02.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-16T00:10:02.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659531+0000", 2024-09-16T00:10:02.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2010000000000002e-06 2024-09-16T00:10:02.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671124+0000", 2024-09-16T00:10:02.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011593696000000001 2024-09-16T00:10:02.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671171+0000", 2024-09-16T00:10:02.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6974999999999997e-05 2024-09-16T00:10:02.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671243+0000", 2024-09-16T00:10:02.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2131000000000004e-05 2024-09-16T00:10:02.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678008+0000", 2024-09-16T00:10:02.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067642329999999997 2024-09-16T00:10:02.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678261+0000", 2024-09-16T00:10:02.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025378999999999999 2024-09-16T00:10:02.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678320+0000", 2024-09-16T00:10:02.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8452000000000002e-05 2024-09-16T00:10:02.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678363+0000", 2024-09-16T00:10:02.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2831999999999999e-05 2024-09-16T00:10:02.464 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.465 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.465 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.465 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4126 2.18 2:1cc859d9:::200.00000012:head [write 806443~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.465 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:02.465 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7774463589999998, 2024-09-16T00:10:02.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019331159000000001, 2024-09-16T00:10:02.465 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.465 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.465 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.465 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:02.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:02.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660904+0000", 2024-09-16T00:10:02.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6800000000000009e-07 2024-09-16T00:10:02.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660909+0000", 2024-09-16T00:10:02.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2730000000000002e-06 2024-09-16T00:10:02.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660919+0000", 2024-09-16T00:10:02.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4960000000000002e-06 2024-09-16T00:10:02.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660923+0000", 2024-09-16T00:10:02.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6260000000000003e-06 2024-09-16T00:10:02.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672698+0000", 2024-09-16T00:10:02.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011774573999999999 2024-09-16T00:10:02.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672784+0000", 2024-09-16T00:10:02.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6074000000000003e-05 2024-09-16T00:10:02.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672897+0000", 2024-09-16T00:10:02.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011267199999999999 2024-09-16T00:10:02.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678026+0000", 2024-09-16T00:10:02.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051297089999999997 2024-09-16T00:10:02.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.469 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680147+0000", 2024-09-16T00:10:02.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021209409999999999 2024-09-16T00:10:02.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680197+0000", 2024-09-16T00:10:02.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0068000000000002e-05 2024-09-16T00:10:02.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680234+0000", 2024-09-16T00:10:02.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6758000000000003e-05 2024-09-16T00:10:02.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.470 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.470 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.471 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4127 2.18 2:1cc859d9:::200.00000012:head [write 808688~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.471 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:02.471 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.77716451, 2024-09-16T00:10:02.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019226493000000001, 2024-09-16T00:10:02.471 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.471 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.471 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.471 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:02.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:02.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661186+0000", 2024-09-16T00:10:02.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2399999999999996e-07 2024-09-16T00:10:02.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661192+0000", 2024-09-16T00:10:02.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9259999999999997e-06 2024-09-16T00:10:02.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661203+0000", 2024-09-16T00:10:02.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1174e-05 2024-09-16T00:10:02.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661213+0000", 2024-09-16T00:10:02.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0579e-05 2024-09-16T00:10:02.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673408+0000", 2024-09-16T00:10:02.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012194442 2024-09-16T00:10:02.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673482+0000", 2024-09-16T00:10:02.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4258000000000006e-05 2024-09-16T00:10:02.474 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673592+0000", 2024-09-16T00:10:02.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109733 2024-09-16T00:10:02.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.475 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678033+0000", 2024-09-16T00:10:02.475 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044411629999999997 2024-09-16T00:10:02.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.475 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680287+0000", 2024-09-16T00:10:02.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022538469999999998 2024-09-16T00:10:02.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680367+0000", 2024-09-16T00:10:02.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9808999999999994e-05 2024-09-16T00:10:02.476 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.476 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.476 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680411+0000", 2024-09-16T00:10:02.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4638000000000002e-05 2024-09-16T00:10:02.477 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.477 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.477 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.477 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4128 2.18 2:1cc859d9:::200.00000012:head [write 810933~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.477 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:02.477 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7767453789999998, 2024-09-16T00:10:02.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018982046999999998, 2024-09-16T00:10:02.477 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.477 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.478 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.478 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:02.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:02.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661605+0000", 2024-09-16T00:10:02.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.031e-06 2024-09-16T00:10:02.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661610+0000", 2024-09-16T00:10:02.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5010000000000004e-06 2024-09-16T00:10:02.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661623+0000", 2024-09-16T00:10:02.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2286000000000001e-05 2024-09-16T00:10:02.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661630+0000", 2024-09-16T00:10:02.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000002e-06 2024-09-16T00:10:02.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674011+0000", 2024-09-16T00:10:02.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012380923 2024-09-16T00:10:02.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674092+0000", 2024-09-16T00:10:02.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1267999999999995e-05 2024-09-16T00:10:02.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674216+0000", 2024-09-16T00:10:02.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012331700000000001 2024-09-16T00:10:02.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678039+0000", 2024-09-16T00:10:02.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038232489999999999 2024-09-16T00:10:02.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680493+0000", 2024-09-16T00:10:02.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024543429999999999 2024-09-16T00:10:02.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680546+0000", 2024-09-16T00:10:02.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3239000000000003e-05 2024-09-16T00:10:02.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680586+0000", 2024-09-16T00:10:02.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9496999999999998e-05 2024-09-16T00:10:02.482 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.483 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.483 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.483 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4129 2.18 2:1cc859d9:::200.00000012:head [write 813178~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.483 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:02.483 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7763882280000001, 2024-09-16T00:10:02.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018785932000000002, 2024-09-16T00:10:02.483 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.483 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.483 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.483 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:02.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:02.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661962+0000", 2024-09-16T00:10:02.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-07 2024-09-16T00:10:02.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661966+0000", 2024-09-16T00:10:02.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-06 2024-09-16T00:10:02.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.662015+0000", 2024-09-16T00:10:02.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8364e-05 2024-09-16T00:10:02.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.662023+0000", 2024-09-16T00:10:02.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2090000000000005e-06 2024-09-16T00:10:02.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674744+0000", 2024-09-16T00:10:02.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012721423000000001 2024-09-16T00:10:02.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674825+0000", 2024-09-16T00:10:02.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1036000000000001e-05 2024-09-16T00:10:02.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674946+0000", 2024-09-16T00:10:02.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120727 2024-09-16T00:10:02.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678045+0000", 2024-09-16T00:10:02.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030993280000000002 2024-09-16T00:10:02.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.488 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.488 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680646+0000", 2024-09-16T00:10:02.488 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026005300000000002 2024-09-16T00:10:02.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680705+0000", 2024-09-16T00:10:02.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9141000000000002e-05 2024-09-16T00:10:02.489 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.489 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.489 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.489 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680747+0000", 2024-09-16T00:10:02.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2064000000000002e-05 2024-09-16T00:10:02.489 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.489 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.489 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.490 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4184 2.18 2:1cc859d9:::200.00000012:head [write 927308~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.490 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:02.490 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.732339633, 2024-09-16T00:10:02.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023503514999999999, 2024-09-16T00:10:02.490 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.490 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.490 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.490 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:02.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:02.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706011+0000", 2024-09-16T00:10:02.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8400000000000003e-07 2024-09-16T00:10:02.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706016+0000", 2024-09-16T00:10:02.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7479999999999999e-06 2024-09-16T00:10:02.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706028+0000", 2024-09-16T00:10:02.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1477e-05 2024-09-16T00:10:02.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706035+0000", 2024-09-16T00:10:02.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1509999999999998e-06 2024-09-16T00:10:02.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.719820+0000", 2024-09-16T00:10:02.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013785168 2024-09-16T00:10:02.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.719888+0000", 2024-09-16T00:10:02.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7631000000000004e-05 2024-09-16T00:10:02.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720003+0000", 2024-09-16T00:10:02.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114849 2024-09-16T00:10:02.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725150+0000", 2024-09-16T00:10:02.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051469970000000004 2024-09-16T00:10:02.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729386+0000", 2024-09-16T00:10:02.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042368529999999996 2024-09-16T00:10:02.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.495 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729461+0000", 2024-09-16T00:10:02.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4935e-05 2024-09-16T00:10:02.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729513+0000", 2024-09-16T00:10:02.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1822e-05 2024-09-16T00:10:02.496 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.496 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.496 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.496 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4185 2.18 2:1cc859d9:::200.00000012:head [write 929217~3264 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.497 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:02.497 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.731333453, 2024-09-16T00:10:02.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023998430000000001, 2024-09-16T00:10:02.497 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.497 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.497 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.498 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:02.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:02.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707017+0000", 2024-09-16T00:10:02.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9999999999999996e-07 2024-09-16T00:10:02.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707023+0000", 2024-09-16T00:10:02.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1789999999999996e-06 2024-09-16T00:10:02.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707035+0000", 2024-09-16T00:10:02.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2106000000000001e-05 2024-09-16T00:10:02.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707042+0000", 2024-09-16T00:10:02.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2960000000000001e-06 2024-09-16T00:10:02.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720503+0000", 2024-09-16T00:10:02.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013460952999999999 2024-09-16T00:10:02.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720583+0000", 2024-09-16T00:10:02.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0171000000000002e-05 2024-09-16T00:10:02.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720668+0000", 2024-09-16T00:10:02.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4128999999999994e-05 2024-09-16T00:10:02.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.501 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725193+0000", 2024-09-16T00:10:02.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045257700000000001 2024-09-16T00:10:02.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.730924+0000", 2024-09-16T00:10:02.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057307690000000001 2024-09-16T00:10:02.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.730972+0000", 2024-09-16T00:10:02.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7756000000000002e-05 2024-09-16T00:10:02.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731014+0000", 2024-09-16T00:10:02.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2401000000000003e-05 2024-09-16T00:10:02.502 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.503 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.503 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4186 2.18 2:1cc859d9:::200.00000012:head [write 932481~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.503 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:02.503 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7307452960000003, 2024-09-16T00:10:02.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023544934999999999, 2024-09-16T00:10:02.503 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.503 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.503 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.503 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:02.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:02.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707605+0000", 2024-09-16T00:10:02.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1500000000000003e-07 2024-09-16T00:10:02.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707610+0000", 2024-09-16T00:10:02.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7049999999999998e-06 2024-09-16T00:10:02.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707618+0000", 2024-09-16T00:10:02.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6149999999999997e-06 2024-09-16T00:10:02.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707623+0000", 2024-09-16T00:10:02.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0660000000000003e-06 2024-09-16T00:10:02.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721143+0000", 2024-09-16T00:10:02.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01351976 2024-09-16T00:10:02.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721221+0000", 2024-09-16T00:10:02.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7918999999999997e-05 2024-09-16T00:10:02.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721335+0000", 2024-09-16T00:10:02.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113781 2024-09-16T00:10:02.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725236+0000", 2024-09-16T00:10:02.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039010379999999999 2024-09-16T00:10:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731025+0000", 2024-09-16T00:10:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057896659999999997 2024-09-16T00:10:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731091+0000", 2024-09-16T00:10:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.567e-05 2024-09-16T00:10:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731149+0000", 2024-09-16T00:10:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7800000000000002e-05 2024-09-16T00:10:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4187 2.18 2:1cc859d9:::200.00000012:head [write 934390~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.509 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:02.510 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7298931670000002, 2024-09-16T00:10:02.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022820340000000001, 2024-09-16T00:10:02.510 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.510 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.510 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.510 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:02.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:02.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708457+0000", 2024-09-16T00:10:02.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6899999999999996e-07 2024-09-16T00:10:02.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708462+0000", 2024-09-16T00:10:02.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3090000000000003e-06 2024-09-16T00:10:02.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708472+0000", 2024-09-16T00:10:02.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7499999999999998e-06 2024-09-16T00:10:02.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708479+0000", 2024-09-16T00:10:02.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1189999999999999e-06 2024-09-16T00:10:02.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721895+0000", 2024-09-16T00:10:02.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013415664000000001 2024-09-16T00:10:02.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721941+0000", 2024-09-16T00:10:02.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6031e-05 2024-09-16T00:10:02.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722007+0000", 2024-09-16T00:10:02.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6503999999999999e-05 2024-09-16T00:10:02.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725277+0000", 2024-09-16T00:10:02.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003269554 2024-09-16T00:10:02.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731158+0000", 2024-09-16T00:10:02.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058813750000000003 2024-09-16T00:10:02.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731240+0000", 2024-09-16T00:10:02.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1815000000000002e-05 2024-09-16T00:10:02.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731276+0000", 2024-09-16T00:10:02.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6449999999999998e-05 2024-09-16T00:10:02.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.516 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.517 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.517 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4188 2.18 2:1cc859d9:::200.00000012:head [write 936299~2419 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.517 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:02.517 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7294722240000002, 2024-09-16T00:10:02.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022518565000000001, 2024-09-16T00:10:02.517 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.517 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.518 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.518 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:02.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:02.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708878+0000", 2024-09-16T00:10:02.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8800000000000001e-07 2024-09-16T00:10:02.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708884+0000", 2024-09-16T00:10:02.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9800000000000003e-06 2024-09-16T00:10:02.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708895+0000", 2024-09-16T00:10:02.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1416e-05 2024-09-16T00:10:02.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708903+0000", 2024-09-16T00:10:02.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4449999999999999e-06 2024-09-16T00:10:02.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722327+0000", 2024-09-16T00:10:02.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013424681000000001 2024-09-16T00:10:02.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722408+0000", 2024-09-16T00:10:02.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0092000000000005e-05 2024-09-16T00:10:02.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722538+0000", 2024-09-16T00:10:02.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013046600000000001 2024-09-16T00:10:02.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726665+0000", 2024-09-16T00:10:02.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041267420000000001 2024-09-16T00:10:02.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731284+0000", 2024-09-16T00:10:02.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046195469999999999 2024-09-16T00:10:02.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731358+0000", 2024-09-16T00:10:02.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4002000000000001e-05 2024-09-16T00:10:02.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731396+0000", 2024-09-16T00:10:02.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7305999999999998e-05 2024-09-16T00:10:02.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.523 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.523 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4189 2.18 2:1cc859d9:::200.00000012:head [write 938718~1910 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.524 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:02.524 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7285651079999997, 2024-09-16T00:10:02.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021703626, 2024-09-16T00:10:02.524 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.524 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.524 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.524 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:02.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:02.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709785+0000", 2024-09-16T00:10:02.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.043e-06 2024-09-16T00:10:02.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709791+0000", 2024-09-16T00:10:02.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9499999999999998e-06 2024-09-16T00:10:02.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709804+0000", 2024-09-16T00:10:02.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2958e-05 2024-09-16T00:10:02.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709812+0000", 2024-09-16T00:10:02.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9869999999999999e-06 2024-09-16T00:10:02.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723058+0000", 2024-09-16T00:10:02.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013245497 2024-09-16T00:10:02.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723131+0000", 2024-09-16T00:10:02.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3879000000000002e-05 2024-09-16T00:10:02.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723241+0000", 2024-09-16T00:10:02.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109435 2024-09-16T00:10:02.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726708+0000", 2024-09-16T00:10:02.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034671340000000002 2024-09-16T00:10:02.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731403+0000", 2024-09-16T00:10:02.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046953979999999999 2024-09-16T00:10:02.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731454+0000", 2024-09-16T00:10:02.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0747e-05 2024-09-16T00:10:02.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731488+0000", 2024-09-16T00:10:02.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3597999999999999e-05 2024-09-16T00:10:02.530 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.530 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.530 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.530 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4190 2.18 2:1cc859d9:::200.00000012:head [write 940628~2420 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.530 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:02.530 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.728092943, 2024-09-16T00:10:02.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021346812999999999, 2024-09-16T00:10:02.530 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.530 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.531 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.531 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:02.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:02.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710257+0000", 2024-09-16T00:10:02.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8100000000000001e-07 2024-09-16T00:10:02.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710263+0000", 2024-09-16T00:10:02.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8050000000000003e-06 2024-09-16T00:10:02.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710273+0000", 2024-09-16T00:10:02.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0005e-05 2024-09-16T00:10:02.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710280+0000", 2024-09-16T00:10:02.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3819999999999998e-06 2024-09-16T00:10:02.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723653+0000", 2024-09-16T00:10:02.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013372866000000001 2024-09-16T00:10:02.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723733+0000", 2024-09-16T00:10:02.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9396999999999999e-05 2024-09-16T00:10:02.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723852+0000", 2024-09-16T00:10:02.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011919899999999999 2024-09-16T00:10:02.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726752+0000", 2024-09-16T00:10:02.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028999659999999999 2024-09-16T00:10:02.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731495+0000", 2024-09-16T00:10:02.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047432209999999997 2024-09-16T00:10:02.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731568+0000", 2024-09-16T00:10:02.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2452000000000004e-05 2024-09-16T00:10:02.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.536 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.536 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731603+0000", 2024-09-16T00:10:02.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5639000000000002e-05 2024-09-16T00:10:02.536 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.536 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.536 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.536 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4192 2.18 2:1cc859d9:::200.00000012:head [write 943048~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.536 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:02.536 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7273601030000001, 2024-09-16T00:10:02.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021688433999999999, 2024-09-16T00:10:02.537 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.537 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.537 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.537 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:02.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:02.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710990+0000", 2024-09-16T00:10:02.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1999999999999998e-07 2024-09-16T00:10:02.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710996+0000", 2024-09-16T00:10:02.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:10:02.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711007+0000", 2024-09-16T00:10:02.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1471000000000001e-05 2024-09-16T00:10:02.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711014+0000", 2024-09-16T00:10:02.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7780000000000002e-06 2024-09-16T00:10:02.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724471+0000", 2024-09-16T00:10:02.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013457435 2024-09-16T00:10:02.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724547+0000", 2024-09-16T00:10:02.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5383999999999995e-05 2024-09-16T00:10:02.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724663+0000", 2024-09-16T00:10:02.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116348 2024-09-16T00:10:02.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731612+0000", 2024-09-16T00:10:02.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069491409999999998 2024-09-16T00:10:02.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732573+0000", 2024-09-16T00:10:02.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0009611 2024-09-16T00:10:02.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732638+0000", 2024-09-16T00:10:02.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919000000000006e-05 2024-09-16T00:10:02.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732678+0000", 2024-09-16T00:10:02.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9230999999999998e-05 2024-09-16T00:10:02.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.542 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.542 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4193 2.18 2:1cc859d9:::200.00000012:head [write 944447~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.542 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:02.542 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7268143949999999, 2024-09-16T00:10:02.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021286851999999998, 2024-09-16T00:10:02.543 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.543 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.543 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.543 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:02.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:02.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711536+0000", 2024-09-16T00:10:02.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0189999999999999e-06 2024-09-16T00:10:02.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711542+0000", 2024-09-16T00:10:02.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7169999999999996e-06 2024-09-16T00:10:02.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711554+0000", 2024-09-16T00:10:02.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2089000000000001e-05 2024-09-16T00:10:02.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711559+0000", 2024-09-16T00:10:02.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3739999999999997e-06 2024-09-16T00:10:02.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725311+0000", 2024-09-16T00:10:02.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013751559 2024-09-16T00:10:02.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725390+0000", 2024-09-16T00:10:02.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9779999999999998e-05 2024-09-16T00:10:02.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725509+0000", 2024-09-16T00:10:02.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118973 2024-09-16T00:10:02.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731617+0000", 2024-09-16T00:10:02.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061079000000000003 2024-09-16T00:10:02.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732762+0000", 2024-09-16T00:10:02.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011449769999999999 2024-09-16T00:10:02.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732796+0000", 2024-09-16T00:10:02.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3587999999999998e-05 2024-09-16T00:10:02.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732822+0000", 2024-09-16T00:10:02.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5876000000000001e-05 2024-09-16T00:10:02.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.548 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.548 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4194 2.18 2:1cc859d9:::200.00000012:head [write 946356~2929 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.548 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:02.549 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.725839364, 2024-09-16T00:10:02.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020453244999999998, 2024-09-16T00:10:02.549 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.549 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.549 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.549 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:02.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:02.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712511+0000", 2024-09-16T00:10:02.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0999999999999997e-07 2024-09-16T00:10:02.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712517+0000", 2024-09-16T00:10:02.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8370000000000002e-06 2024-09-16T00:10:02.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712528+0000", 2024-09-16T00:10:02.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1528e-05 2024-09-16T00:10:02.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712536+0000", 2024-09-16T00:10:02.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3939999999999998e-06 2024-09-16T00:10:02.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726004+0000", 2024-09-16T00:10:02.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013468177 2024-09-16T00:10:02.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726085+0000", 2024-09-16T00:10:02.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1626999999999996e-05 2024-09-16T00:10:02.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726205+0000", 2024-09-16T00:10:02.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119751 2024-09-16T00:10:02.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731622+0000", 2024-09-16T00:10:02.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054165419999999999 2024-09-16T00:10:02.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732867+0000", 2024-09-16T00:10:02.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001244855 2024-09-16T00:10:02.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732924+0000", 2024-09-16T00:10:02.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7395999999999997e-05 2024-09-16T00:10:02.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732963+0000", 2024-09-16T00:10:02.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9227999999999998e-05 2024-09-16T00:10:02.554 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.554 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.554 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.554 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4195 2.18 2:1cc859d9:::200.00000012:head [write 949285~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.554 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:02.554 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7247877040000001, 2024-09-16T00:10:02.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019498099000000001, 2024-09-16T00:10:02.554 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.555 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.555 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.555 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:02.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:02.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713563+0000", 2024-09-16T00:10:02.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T00:10:02.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713569+0000", 2024-09-16T00:10:02.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8710000000000002e-06 2024-09-16T00:10:02.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713582+0000", 2024-09-16T00:10:02.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3134e-05 2024-09-16T00:10:02.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713591+0000", 2024-09-16T00:10:02.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9889999999999998e-06 2024-09-16T00:10:02.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726786+0000", 2024-09-16T00:10:02.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013194991999999999 2024-09-16T00:10:02.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726856+0000", 2024-09-16T00:10:02.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0017000000000004e-05 2024-09-16T00:10:02.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726967+0000", 2024-09-16T00:10:02.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111815 2024-09-16T00:10:02.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732530+0000", 2024-09-16T00:10:02.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055624940000000003 2024-09-16T00:10:02.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733001+0000", 2024-09-16T00:10:02.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000470817 2024-09-16T00:10:02.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733030+0000", 2024-09-16T00:10:02.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9051e-05 2024-09-16T00:10:02.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733060+0000", 2024-09-16T00:10:02.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9819e-05 2024-09-16T00:10:02.561 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.561 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.561 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.562 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4215 2.18 2:1cc859d9:::200.00000012:head [write 991028~1910 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.562 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:02.562 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.7050919929999999, 2024-09-16T00:10:02.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019416676000000001, 2024-09-16T00:10:02.562 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.562 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.562 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.562 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:02.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:02.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733258+0000", 2024-09-16T00:10:02.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:10:02.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733264+0000", 2024-09-16T00:10:02.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4090000000000002e-06 2024-09-16T00:10:02.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733273+0000", 2024-09-16T00:10:02.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4769999999999992e-06 2024-09-16T00:10:02.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733280+0000", 2024-09-16T00:10:02.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2989999999999999e-06 2024-09-16T00:10:02.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.742966+0000", 2024-09-16T00:10:02.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009685585 2024-09-16T00:10:02.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743046+0000", 2024-09-16T00:10:02.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0469999999999994e-05 2024-09-16T00:10:02.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743165+0000", 2024-09-16T00:10:02.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118847 2024-09-16T00:10:02.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.746968+0000", 2024-09-16T00:10:02.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038025099999999998 2024-09-16T00:10:02.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752559+0000", 2024-09-16T00:10:02.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055911290000000002 2024-09-16T00:10:02.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752634+0000", 2024-09-16T00:10:02.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5102e-05 2024-09-16T00:10:02.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752674+0000", 2024-09-16T00:10:02.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9878e-05 2024-09-16T00:10:02.568 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.568 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.568 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.568 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4216 2.18 2:1cc859d9:::200.00000012:head [write 992938~2755 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:02.568 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:02.568 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.704546085, 2024-09-16T00:10:02.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019035976, 2024-09-16T00:10:02.568 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:02.568 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:02.569 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:02.569 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:02.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:02.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:02.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:02.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:02.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:02.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:02.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733804+0000", 2024-09-16T00:10:02.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2299999999999999e-07 2024-09-16T00:10:02.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:02.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733810+0000", 2024-09-16T00:10:02.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7259999999999999e-06 2024-09-16T00:10:02.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:02.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733822+0000", 2024-09-16T00:10:02.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2204e-05 2024-09-16T00:10:02.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:02.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733829+0000", 2024-09-16T00:10:02.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3889999999999999e-06 2024-09-16T00:10:02.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:02.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743675+0000", 2024-09-16T00:10:02.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098457990000000006 2024-09-16T00:10:02.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:02.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743752+0000", 2024-09-16T00:10:02.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6279000000000006e-05 2024-09-16T00:10:02.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:02.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743867+0000", 2024-09-16T00:10:02.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115319 2024-09-16T00:10:02.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:02.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.746971+0000", 2024-09-16T00:10:02.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031042399999999999 2024-09-16T00:10:02.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:02.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752723+0000", 2024-09-16T00:10:02.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057518329999999996 2024-09-16T00:10:02.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:02.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752804+0000", 2024-09-16T00:10:02.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0822999999999995e-05 2024-09-16T00:10:02.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:02.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:02.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:02.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752839+0000", 2024-09-16T00:10:02.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5441000000000002e-05 2024-09-16T00:10:02.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.574 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.574 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:02.574 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:02.574 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:02.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.439+0000 7f5e95add640 1 --2- 172.21.15.139:0/2514231304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e9015b280 0x7f5e90166190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5e88009d90 tx=0x7f5e8802a6f0 comp rx=0 tx=0).stop 2024-09-16T00:10:02.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.439+0000 7f5e95add640 1 -- 172.21.15.139:0/2514231304 shutdown_connections 2024-09-16T00:10:02.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.439+0000 7f5e95add640 1 --2- 172.21.15.139:0/2514231304 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e9015b280 0x7f5e90166190 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:02.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.439+0000 7f5e95add640 1 -- 172.21.15.139:0/2514231304 >> 172.21.15.139:0/2514231304 conn(0x7f5e9006ca40 msgr2=0x7f5e90067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:02.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.439+0000 7f5e95add640 1 -- 172.21.15.139:0/2514231304 shutdown_connections 2024-09-16T00:10:02.575 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.439+0000 7f5e95add640 1 -- 172.21.15.139:0/2514231304 wait complete. 2024-09-16T00:10:02.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.440+0000 7fc40f7fe640 1 --2- 172.21.15.139:0/3947918887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4100c64a0 0x7fc4100730c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc4100ca610 tx=0x7fc40802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8be5e097166b3a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:02.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.440+0000 7fc40e7fc640 1 -- 172.21.15.139:0/3947918887 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408007740 con 0x7fc4100c64a0 2024-09-16T00:10:02.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.440+0000 7fc40e7fc640 1 -- 172.21.15.139:0/3947918887 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4080078e0 con 0x7fc4100c64a0 2024-09-16T00:10:02.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.440+0000 7fc40e7fc640 1 -- 172.21.15.139:0/3947918887 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408007cb0 con 0x7fc4100c64a0 2024-09-16T00:10:02.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.440+0000 7f5e95add640 1 Processor -- start 2024-09-16T00:10:02.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.440+0000 7fc4153cf640 1 -- 172.21.15.139:0/3947918887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4100c64a0 msgr2=0x7fc4100730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:02.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.440+0000 7fc4153cf640 1 --2- 172.21.15.139:0/3947918887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4100c64a0 0x7fc4100730c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc4100ca610 tx=0x7fc40802a680 comp rx=0 tx=0).stop 2024-09-16T00:10:02.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.440+0000 7fc4153cf640 1 -- 172.21.15.139:0/3947918887 shutdown_connections 2024-09-16T00:10:02.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.440+0000 7fc4153cf640 1 --2- 172.21.15.139:0/3947918887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4100c64a0 0x7fc4100730c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:02.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.440+0000 7fc4153cf640 1 --2- 172.21.15.139:0/3947918887 >> v2:172.21.15.164:3300/0 conn(0x7fc41006d080 0x7fc4100c7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:02.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.441+0000 7fc4153cf640 1 -- 172.21.15.139:0/3947918887 >> 172.21.15.139:0/3947918887 conn(0x7fc4100cc740 msgr2=0x7fc4100ccb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:02.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.441+0000 7f5e95add640 1 -- start start 2024-09-16T00:10:02.578 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:10:02.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.441+0000 7fc4153cf640 1 -- 172.21.15.139:0/3947918887 shutdown_connections 2024-09-16T00:10:02.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.441+0000 7fc4153cf640 1 -- 172.21.15.139:0/3947918887 wait complete. 2024-09-16T00:10:02.580 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:10:02.581 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:02.578+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:02.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.441+0000 7f5e95add640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e90068060 0x7f5e900cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:02.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.441+0000 7f5e95add640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e90154c20 0x7f5e90157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:02.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.441+0000 7f5e95add640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e9015b280 0x7f5e901f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:02.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.441+0000 7f5e95add640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9015e3b0 con 0x7f5e90068060 2024-09-16T00:10:02.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.441+0000 7f5e95add640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5e900cc260 con 0x7f5e90154c20 2024-09-16T00:10:02.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.441+0000 7f5e95add640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9015e690 con 0x7f5e9015b280 2024-09-16T00:10:02.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.441+0000 7f5e8f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e90068060 0x7f5e900cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:02.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7f5e8f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e90068060 0x7f5e900cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55862/0 (socket says 172.21.15.139:55862) 2024-09-16T00:10:02.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7f5e8f7fe640 1 -- 172.21.15.139:0/2267242893 learned_addr learned my addr 172.21.15.139:0/2267242893 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:02.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7f5e8ffff640 1 --2- 172.21.15.139:0/2267242893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e9015b280 0x7f5e901f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:02.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7f5e8effd640 1 --2- 172.21.15.139:0/2267242893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e90154c20 0x7f5e90157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:02.582 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7f5e8f7fe640 1 -- 172.21.15.139:0/2267242893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e9015b280 msgr2=0x7f5e901f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:02.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7f5e8f7fe640 1 --2- 172.21.15.139:0/2267242893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e9015b280 0x7f5e901f45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:02.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7f5e8f7fe640 1 -- 172.21.15.139:0/2267242893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e90154c20 msgr2=0x7f5e90157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:02.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7f5e8f7fe640 1 --2- 172.21.15.139:0/2267242893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e90154c20 0x7f5e90157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:02.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7f5e8f7fe640 1 -- 172.21.15.139:0/2267242893 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e901f3e30 con 0x7f5e90068060 2024-09-16T00:10:02.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7fc4153cf640 1 Processor -- start 2024-09-16T00:10:02.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7fc4153cf640 1 -- start start 2024-09-16T00:10:02.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7f5e8f7fe640 1 --2- 172.21.15.139:0/2267242893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e90068060 0x7f5e900cfbf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000c500 tx=0x7f5e8000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:02.585 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7f5e77fff640 1 -- 172.21.15.139:0/2267242893 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80017070 con 0x7f5e90068060 2024-09-16T00:10:02.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7f5e77fff640 1 -- 172.21.15.139:0/2267242893 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e80005ce0 con 0x7f5e90068060 2024-09-16T00:10:02.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7f5e77fff640 1 -- 172.21.15.139:0/2267242893 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80007330 con 0x7f5e90068060 2024-09-16T00:10:02.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7f5e95add640 1 -- 172.21.15.139:0/2267242893 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e90155280 con 0x7f5e90068060 2024-09-16T00:10:02.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.442+0000 7f5e95add640 1 -- 172.21.15.139:0/2267242893 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e900c5340 con 0x7f5e90068060 2024-09-16T00:10:02.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc4153cf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41006d080 0x7fc41006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:02.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc4153cf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4100c60c0 0x7fc4101efc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:02.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc4153cf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc4100c64a0 0x7fc410157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:02.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc4153cf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc410152ca0 con 0x7fc41006d080 2024-09-16T00:10:02.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc4153cf640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fc410067d20 con 0x7fc4100c60c0 2024-09-16T00:10:02.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc4153cf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fc410152e20 con 0x7fc4100c64a0 2024-09-16T00:10:02.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc407fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4100c60c0 0x7fc4101efc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:02.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc407fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4100c60c0 0x7fc4101efc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:49840/0 (socket says 172.21.15.139:49840) 2024-09-16T00:10:02.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc407fff640 1 -- 172.21.15.139:0/923504031 learned_addr learned my addr 172.21.15.139:0/923504031 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:02.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc40f7fe640 1 --2- 172.21.15.139:0/923504031 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc4100c64a0 0x7fc410157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:02.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc40effd640 1 --2- 172.21.15.139:0/923504031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41006d080 0x7fc41006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:02.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc407fff640 1 -- 172.21.15.139:0/923504031 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc4100c64a0 msgr2=0x7fc410157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:02.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc407fff640 1 --2- 172.21.15.139:0/923504031 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc4100c64a0 0x7fc410157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:02.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc407fff640 1 -- 172.21.15.139:0/923504031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41006d080 msgr2=0x7fc41006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:02.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc407fff640 1 --2- 172.21.15.139:0/923504031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41006d080 0x7fc41006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:02.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc407fff640 1 -- 172.21.15.139:0/923504031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4101f3d70 con 0x7fc4100c60c0 2024-09-16T00:10:02.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.443+0000 7fc40effd640 1 --2- 172.21.15.139:0/923504031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41006d080 0x7fc41006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:10:02.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.444+0000 7f5e95add640 1 -- 172.21.15.139:0/2267242893 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e900686b0 con 0x7f5e90068060 2024-09-16T00:10:02.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.444+0000 7fc407fff640 1 --2- 172.21.15.139:0/923504031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4100c60c0 0x7fc4101efc50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800e9e0 tx=0x7fc3f800eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:02.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.444+0000 7fc4077fe640 1 -- 172.21.15.139:0/923504031 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8017070 con 0x7fc4100c60c0 2024-09-16T00:10:02.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.444+0000 7fc4077fe640 1 -- 172.21.15.139:0/923504031 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f800c9d0 con 0x7fc4100c60c0 2024-09-16T00:10:02.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.444+0000 7fc4077fe640 1 -- 172.21.15.139:0/923504031 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f800cdc0 con 0x7fc4100c60c0 2024-09-16T00:10:02.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.444+0000 7fc4153cf640 1 -- 172.21.15.139:0/923504031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41014d190 con 0x7fc4100c60c0 2024-09-16T00:10:02.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.444+0000 7fc4153cf640 1 -- 172.21.15.139:0/923504031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4101e4010 con 0x7fc4100c60c0 2024-09-16T00:10:02.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.444+0000 7f5e77fff640 1 -- 172.21.15.139:0/2267242893 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5e8001a070 con 0x7f5e90068060 2024-09-16T00:10:02.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.444+0000 7f5e77fff640 1 --2- 172.21.15.139:0/2267242893 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e7c06c2b0 0x7f5e7c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:02.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.445+0000 7f5e77fff640 1 -- 172.21.15.139:0/2267242893 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5973+0+0 (secure 0 0 0) 0x7f5e8008aa10 con 0x7f5e90068060 2024-09-16T00:10:02.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.445+0000 7f5e8effd640 1 --2- 172.21.15.139:0/2267242893 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e7c06c2b0 0x7f5e7c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:02.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.446+0000 7f5e8effd640 1 --2- 172.21.15.139:0/2267242893 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e7c06c2b0 0x7f5e7c06e770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5e901eb160 tx=0x7f5e84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:02.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.446+0000 7fc4153cf640 1 -- 172.21.15.139:0/923504031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4100c8bf0 con 0x7fc4100c60c0 2024-09-16T00:10:02.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.446+0000 7fc4077fe640 1 -- 172.21.15.139:0/923504031 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc3f801a070 con 0x7fc4100c60c0 2024-09-16T00:10:02.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.446+0000 7fc4077fe640 1 --2- 172.21.15.139:0/923504031 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc3f006c2e0 0x7fc3f006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:02.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.446+0000 7fc40effd640 1 --2- 172.21.15.139:0/923504031 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc3f006c2e0 0x7fc3f006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:02.595 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.446+0000 7fc4077fe640 1 -- 172.21.15.139:0/923504031 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5973+0+0 (secure 0 0 0) 0x7fc3f808b480 con 0x7fc4100c60c0 2024-09-16T00:10:02.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.447+0000 7fc40effd640 1 --2- 172.21.15.139:0/923504031 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc3f006c2e0 0x7fc3f006e7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc400007fd0 tx=0x7fc400009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:02.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.450+0000 7f5e77fff640 1 -- 172.21.15.139:0/2267242893 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5e80058a50 con 0x7f5e90068060 2024-09-16T00:10:02.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.452+0000 7fc4077fe640 1 -- 172.21.15.139:0/923504031 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc3f8059490 con 0x7fc4100c60c0 2024-09-16T00:10:02.598 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:10:02.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.653+0000 7fc4153cf640 1 -- 172.21.15.139:0/923504031 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fc4100c9e90 con 0x7fc4100c60c0 2024-09-16T00:10:02.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:02.659+0000 7f5e95add640 1 -- 172.21.15.139:0/2267242893 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) -- 0x7f5e900cd720 con 0x7f5e90068060 2024-09-16T00:10:02.679 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:02.679+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:02.730 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:02.730 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:02.730 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:02.730 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:02.739 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:10:02.781 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:02.780+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:02.882 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:02.882 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:02.882 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:02.882 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:02.882 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:02.882 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:02.881+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:02.893 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:10:02.983 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:02.982+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:03.079 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:03.079 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:03.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:03.079 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:03.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.079 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:10:03.080 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:10:03.080 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 159.99639834300001, 2024-09-16T00:10:03.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:10:03.080 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.080 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.080 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.080 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:03.080 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:10:03.080 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:10:03.080 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:10:03.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.080 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:10:03.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:10:03.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.081 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.081 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.081 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.081 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:10:03.081 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:10:03.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:10:03.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:10:03.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:10:03.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:10:03.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:10:03.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:10:03.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:10:03.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:10:03.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:10:03.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:10:03.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:10:03.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:10:03.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:10:03.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:10:03.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:10:03.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:10:03.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:10:03.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:10:03.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:10:03.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:10:03.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:10:03.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:10:03.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:10:03.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:10:03.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:10:03.086 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.086 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.086 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.087 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2402 2.8 2:17051619:::200.0000000f:head [write 2106308~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.087 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.664943+0000", 2024-09-16T00:10:03.087 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.412285312, 2024-09-16T00:10:03.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024647203999999999, 2024-09-16T00:10:03.087 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.087 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.087 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.087 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.664943+0000", 2024-09-16T00:10:03.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.664943+0000", 2024-09-16T00:10:03.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.664943+0000", 2024-09-16T00:10:03.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8999999999999995e-07 2024-09-16T00:10:03.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.664947+0000", 2024-09-16T00:10:03.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.146e-06 2024-09-16T00:10:03.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.664953+0000", 2024-09-16T00:10:03.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7179999999999999e-06 2024-09-16T00:10:03.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.664957+0000", 2024-09-16T00:10:03.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-16T00:10:03.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.682748+0000", 2024-09-16T00:10:03.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017790166999999999 2024-09-16T00:10:03.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.682792+0000", 2024-09-16T00:10:03.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4400999999999997e-05 2024-09-16T00:10:03.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.682859+0000", 2024-09-16T00:10:03.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6867999999999997e-05 2024-09-16T00:10:03.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.684686+0000", 2024-09-16T00:10:03.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018267120000000001 2024-09-16T00:10:03.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689507+0000", 2024-09-16T00:10:03.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048218230000000003 2024-09-16T00:10:03.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689558+0000", 2024-09-16T00:10:03.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0118999999999999e-05 2024-09-16T00:10:03.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689590+0000", 2024-09-16T00:10:03.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2224000000000002e-05 2024-09-16T00:10:03.092 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.092 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.092 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.093 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2408 2.8 2:17051619:::200.0000000f:head [write 2116866~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.093 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.668321+0000", 2024-09-16T00:10:03.093 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.408906812, 2024-09-16T00:10:03.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.029956139999999999, 2024-09-16T00:10:03.093 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.093 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.093 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.093 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.668321+0000", 2024-09-16T00:10:03.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.668321+0000", 2024-09-16T00:10:03.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.668322+0000", 2024-09-16T00:10:03.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0900000000000001e-07 2024-09-16T00:10:03.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.668326+0000", 2024-09-16T00:10:03.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5549999999999996e-06 2024-09-16T00:10:03.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.668336+0000", 2024-09-16T00:10:03.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:10:03.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.668342+0000", 2024-09-16T00:10:03.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1800000000000001e-06 2024-09-16T00:10:03.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.686415+0000", 2024-09-16T00:10:03.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018073371000000001 2024-09-16T00:10:03.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.686493+0000", 2024-09-16T00:10:03.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7428000000000005e-05 2024-09-16T00:10:03.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.686605+0000", 2024-09-16T00:10:03.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011246 2024-09-16T00:10:03.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689921+0000", 2024-09-16T00:10:03.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033161190000000002 2024-09-16T00:10:03.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698219+0000", 2024-09-16T00:10:03.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082978380000000001 2024-09-16T00:10:03.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.098 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698252+0000", 2024-09-16T00:10:03.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2407000000000003e-05 2024-09-16T00:10:03.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698277+0000", 2024-09-16T00:10:03.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5554999999999998e-05 2024-09-16T00:10:03.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.099 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.099 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.100 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2409 2.8 2:17051619:::200.0000000f:head [write 2119111~5159 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.100 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.669949+0000", 2024-09-16T00:10:03.100 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.407279063000001, 2024-09-16T00:10:03.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028439168000000001, 2024-09-16T00:10:03.100 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.100 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.100 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.100 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.669949+0000", 2024-09-16T00:10:03.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.669949+0000", 2024-09-16T00:10:03.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.669950+0000", 2024-09-16T00:10:03.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1299999999999999e-07 2024-09-16T00:10:03.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.669956+0000", 2024-09-16T00:10:03.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3690000000000003e-06 2024-09-16T00:10:03.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.669965+0000", 2024-09-16T00:10:03.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1849999999999993e-06 2024-09-16T00:10:03.102 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.669969+0000", 2024-09-16T00:10:03.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2849999999999996e-06 2024-09-16T00:10:03.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687050+0000", 2024-09-16T00:10:03.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017080733000000001 2024-09-16T00:10:03.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.103 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687126+0000", 2024-09-16T00:10:03.103 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5867999999999999e-05 2024-09-16T00:10:03.103 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.103 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.103 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687236+0000", 2024-09-16T00:10:03.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110442 2024-09-16T00:10:03.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689926+0000", 2024-09-16T00:10:03.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026899469999999998 2024-09-16T00:10:03.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.104 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.104 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.104 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698326+0000", 2024-09-16T00:10:03.104 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0083998240000000002 2024-09-16T00:10:03.104 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698358+0000", 2024-09-16T00:10:03.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1464000000000002e-05 2024-09-16T00:10:03.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.105 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.105 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.105 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698388+0000", 2024-09-16T00:10:03.105 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0238000000000001e-05 2024-09-16T00:10:03.105 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.105 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.105 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.105 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.106 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2410 2.8 2:17051619:::200.0000000f:head [write 2124270~2419 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.106 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.670334+0000", 2024-09-16T00:10:03.106 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.406893867000001, 2024-09-16T00:10:03.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028210857999999998, 2024-09-16T00:10:03.106 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.106 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.106 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.106 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.106 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.106 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.106 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.670334+0000", 2024-09-16T00:10:03.106 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.106 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.670334+0000", 2024-09-16T00:10:03.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.670335+0000", 2024-09-16T00:10:03.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6000000000000003e-07 2024-09-16T00:10:03.107 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.107 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.107 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.107 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.670339+0000", 2024-09-16T00:10:03.107 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0940000000000001e-06 2024-09-16T00:10:03.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.670346+0000", 2024-09-16T00:10:03.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.605e-06 2024-09-16T00:10:03.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.108 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.670351+0000", 2024-09-16T00:10:03.108 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2710000000000003e-06 2024-09-16T00:10:03.108 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.108 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.108 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688139+0000", 2024-09-16T00:10:03.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017788780000000001 2024-09-16T00:10:03.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688218+0000", 2024-09-16T00:10:03.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8320999999999998e-05 2024-09-16T00:10:03.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.109 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.109 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.109 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688316+0000", 2024-09-16T00:10:03.109 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8071000000000003e-05 2024-09-16T00:10:03.109 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.690555+0000", 2024-09-16T00:10:03.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022395039999999998 2024-09-16T00:10:03.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698454+0000", 2024-09-16T00:10:03.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0078986219999999992 2024-09-16T00:10:03.110 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.110 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.110 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.110 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698507+0000", 2024-09-16T00:10:03.110 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2952999999999999e-05 2024-09-16T00:10:03.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.111 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.111 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.111 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698545+0000", 2024-09-16T00:10:03.111 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7976999999999999e-05 2024-09-16T00:10:03.111 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.111 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.111 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.111 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.112 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2411 2.8 2:17051619:::200.0000000f:head [write 2126689~6053 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.112 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.671637+0000", 2024-09-16T00:10:03.112 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.405590569999999, 2024-09-16T00:10:03.112 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0269952, 2024-09-16T00:10:03.112 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.112 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.112 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.112 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.112 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.112 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.112 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.671637+0000", 2024-09-16T00:10:03.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.671637+0000", 2024-09-16T00:10:03.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.113 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.113 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.671638+0000", 2024-09-16T00:10:03.113 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.054e-06 2024-09-16T00:10:03.113 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.113 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.671648+0000", 2024-09-16T00:10:03.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3190000000000001e-06 2024-09-16T00:10:03.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.671658+0000", 2024-09-16T00:10:03.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0227000000000001e-05 2024-09-16T00:10:03.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.114 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.114 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.114 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.671665+0000", 2024-09-16T00:10:03.114 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.922e-06 2024-09-16T00:10:03.114 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688668+0000", 2024-09-16T00:10:03.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017003429 2024-09-16T00:10:03.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688712+0000", 2024-09-16T00:10:03.115 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3664000000000001e-05 2024-09-16T00:10:03.115 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.115 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.115 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.115 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688773+0000", 2024-09-16T00:10:03.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0968000000000003e-05 2024-09-16T00:10:03.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.690561+0000", 2024-09-16T00:10:03.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001788408 2024-09-16T00:10:03.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.116 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.116 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698585+0000", 2024-09-16T00:10:03.116 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0080236279999999997 2024-09-16T00:10:03.116 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.116 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698613+0000", 2024-09-16T00:10:03.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8004000000000002e-05 2024-09-16T00:10:03.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.117 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.117 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698632+0000", 2024-09-16T00:10:03.117 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9576999999999999e-05 2024-09-16T00:10:03.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.117 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.117 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.117 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.117 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.117 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2412 2.8 2:17051619:::200.0000000f:head [write 2132742~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.118 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.672301+0000", 2024-09-16T00:10:03.118 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.404927106000001, 2024-09-16T00:10:03.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026473445000000002, 2024-09-16T00:10:03.118 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.118 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.118 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.118 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.118 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.118 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.672301+0000", 2024-09-16T00:10:03.118 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.118 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.118 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.672301+0000", 2024-09-16T00:10:03.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.672301+0000", 2024-09-16T00:10:03.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.37e-07 2024-09-16T00:10:03.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.119 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.119 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.119 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.672307+0000", 2024-09-16T00:10:03.119 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4410000000000001e-06 2024-09-16T00:10:03.119 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.672318+0000", 2024-09-16T00:10:03.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1246e-05 2024-09-16T00:10:03.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.672326+0000", 2024-09-16T00:10:03.120 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6120000000000001e-06 2024-09-16T00:10:03.120 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.120 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.120 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.120 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689084+0000", 2024-09-16T00:10:03.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016758021000000001 2024-09-16T00:10:03.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689128+0000", 2024-09-16T00:10:03.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3863000000000003e-05 2024-09-16T00:10:03.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.121 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689190+0000", 2024-09-16T00:10:03.121 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2589000000000007e-05 2024-09-16T00:10:03.121 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.121 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.121 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.691512+0000", 2024-09-16T00:10:03.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023218800000000001 2024-09-16T00:10:03.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698686+0000", 2024-09-16T00:10:03.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071734069999999997 2024-09-16T00:10:03.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.122 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.122 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698737+0000", 2024-09-16T00:10:03.122 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1371e-05 2024-09-16T00:10:03.122 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.122 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.123 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.123 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698774+0000", 2024-09-16T00:10:03.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7277999999999998e-05 2024-09-16T00:10:03.123 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.123 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.123 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.123 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.123 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.123 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2413 2.8 2:17051619:::200.0000000f:head [write 2134651~2765 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.123 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.673125+0000", 2024-09-16T00:10:03.123 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.404103002999999, 2024-09-16T00:10:03.123 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025733072999999999, 2024-09-16T00:10:03.123 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.124 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.124 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.124 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.673125+0000", 2024-09-16T00:10:03.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.124 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.124 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.124 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.673125+0000", 2024-09-16T00:10:03.124 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.124 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.673126+0000", 2024-09-16T00:10:03.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9599999999999998e-07 2024-09-16T00:10:03.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.673139+0000", 2024-09-16T00:10:03.125 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3063e-05 2024-09-16T00:10:03.125 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.125 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.125 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.125 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.673150+0000", 2024-09-16T00:10:03.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1491999999999999e-05 2024-09-16T00:10:03.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.673157+0000", 2024-09-16T00:10:03.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8739999999999997e-06 2024-09-16T00:10:03.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.126 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.690110+0000", 2024-09-16T00:10:03.126 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016952826000000001 2024-09-16T00:10:03.126 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.126 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.126 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.690156+0000", 2024-09-16T00:10:03.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5930000000000002e-05 2024-09-16T00:10:03.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.690224+0000", 2024-09-16T00:10:03.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7595999999999995e-05 2024-09-16T00:10:03.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.127 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.127 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.692680+0000", 2024-09-16T00:10:03.127 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024567970000000001 2024-09-16T00:10:03.127 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.127 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698813+0000", 2024-09-16T00:10:03.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061326870000000004 2024-09-16T00:10:03.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698839+0000", 2024-09-16T00:10:03.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6299999999999999e-05 2024-09-16T00:10:03.128 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.128 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.128 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.128 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698858+0000", 2024-09-16T00:10:03.128 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8612000000000001e-05 2024-09-16T00:10:03.129 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.129 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.129 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.129 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.129 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2414 2.8 2:17051619:::200.0000000f:head [write 2137416~3263 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.129 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.674025+0000", 2024-09-16T00:10:03.129 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.403202865000001, 2024-09-16T00:10:03.129 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024972957, 2024-09-16T00:10:03.129 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.129 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.129 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.129 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.129 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.674025+0000", 2024-09-16T00:10:03.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.674025+0000", 2024-09-16T00:10:03.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.130 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.130 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.130 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.130 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.674026+0000", 2024-09-16T00:10:03.130 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4399999999999999e-07 2024-09-16T00:10:03.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.674031+0000", 2024-09-16T00:10:03.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1680000000000003e-06 2024-09-16T00:10:03.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.131 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.674039+0000", 2024-09-16T00:10:03.131 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2360000000000004e-06 2024-09-16T00:10:03.131 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.131 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.131 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.674044+0000", 2024-09-16T00:10:03.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9930000000000003e-06 2024-09-16T00:10:03.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.690590+0000", 2024-09-16T00:10:03.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016545905999999999 2024-09-16T00:10:03.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.132 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.132 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.690635+0000", 2024-09-16T00:10:03.132 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4455e-05 2024-09-16T00:10:03.132 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.132 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.690702+0000", 2024-09-16T00:10:03.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7587000000000002e-05 2024-09-16T00:10:03.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.692686+0000", 2024-09-16T00:10:03.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019839810000000001 2024-09-16T00:10:03.133 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.133 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.133 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.133 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698906+0000", 2024-09-16T00:10:03.133 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062200520000000002 2024-09-16T00:10:03.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698962+0000", 2024-09-16T00:10:03.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5909000000000003e-05 2024-09-16T00:10:03.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.134 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.134 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.134 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698998+0000", 2024-09-16T00:10:03.134 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5825999999999999e-05 2024-09-16T00:10:03.134 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.134 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.134 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.134 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.135 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2439 2.8 2:17051619:::200.0000000f:head [write 2194864~2754 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.135 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.687880+0000", 2024-09-16T00:10:03.135 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.389348329000001, 2024-09-16T00:10:03.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026317367000000001, 2024-09-16T00:10:03.135 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.135 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.135 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.135 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.135 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.135 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.135 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687880+0000", 2024-09-16T00:10:03.135 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.135 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687880+0000", 2024-09-16T00:10:03.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687880+0000", 2024-09-16T00:10:03.136 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1800000000000005e-07 2024-09-16T00:10:03.136 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.136 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.136 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.136 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687885+0000", 2024-09-16T00:10:03.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9300000000000002e-06 2024-09-16T00:10:03.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687896+0000", 2024-09-16T00:10:03.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0499999999999999e-05 2024-09-16T00:10:03.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.137 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.137 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687902+0000", 2024-09-16T00:10:03.137 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.144e-06 2024-09-16T00:10:03.137 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.137 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.707045+0000", 2024-09-16T00:10:03.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019143104000000001 2024-09-16T00:10:03.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.707090+0000", 2024-09-16T00:10:03.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5367000000000001e-05 2024-09-16T00:10:03.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.138 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.138 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.138 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.707156+0000", 2024-09-16T00:10:03.138 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5977999999999997e-05 2024-09-16T00:10:03.138 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.711185+0000", 2024-09-16T00:10:03.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040285440000000002 2024-09-16T00:10:03.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714139+0000", 2024-09-16T00:10:03.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002954071 2024-09-16T00:10:03.139 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.139 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.139 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.139 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714173+0000", 2024-09-16T00:10:03.139 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4084000000000001e-05 2024-09-16T00:10:03.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.140 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.140 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714197+0000", 2024-09-16T00:10:03.140 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3927e-05 2024-09-16T00:10:03.140 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.140 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.140 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.140 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.140 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.140 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2440 2.8 2:17051619:::200.0000000f:head [write 2197618~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.140 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.688247+0000", 2024-09-16T00:10:03.140 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.388980966, 2024-09-16T00:10:03.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026045682000000001, 2024-09-16T00:10:03.141 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.141 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.141 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.141 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688247+0000", 2024-09-16T00:10:03.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.141 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.141 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.141 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.141 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688247+0000", 2024-09-16T00:10:03.141 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688248+0000", 2024-09-16T00:10:03.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4e-07 2024-09-16T00:10:03.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.142 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688253+0000", 2024-09-16T00:10:03.142 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6500000000000001e-06 2024-09-16T00:10:03.142 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.142 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.142 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688266+0000", 2024-09-16T00:10:03.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2797e-05 2024-09-16T00:10:03.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688274+0000", 2024-09-16T00:10:03.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6939999999999995e-06 2024-09-16T00:10:03.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.143 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.143 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.707476+0000", 2024-09-16T00:10:03.143 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019202520000000001 2024-09-16T00:10:03.143 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.143 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.707557+0000", 2024-09-16T00:10:03.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1083999999999998e-05 2024-09-16T00:10:03.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.707673+0000", 2024-09-16T00:10:03.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116029 2024-09-16T00:10:03.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.144 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.144 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.144 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.711526+0000", 2024-09-16T00:10:03.144 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038529060000000001 2024-09-16T00:10:03.144 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714230+0000", 2024-09-16T00:10:03.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00270396 2024-09-16T00:10:03.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714260+0000", 2024-09-16T00:10:03.145 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9238999999999999e-05 2024-09-16T00:10:03.145 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.145 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.145 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.145 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714293+0000", 2024-09-16T00:10:03.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2963000000000003e-05 2024-09-16T00:10:03.146 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.146 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.146 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.146 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.146 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.146 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2441 2.8 2:17051619:::200.0000000f:head [write 2199527~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.146 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.688682+0000", 2024-09-16T00:10:03.146 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.388545561000001, 2024-09-16T00:10:03.146 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025692883999999999, 2024-09-16T00:10:03.146 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.146 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.146 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.146 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688682+0000", 2024-09-16T00:10:03.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688682+0000", 2024-09-16T00:10:03.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.147 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.147 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.147 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.147 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688683+0000", 2024-09-16T00:10:03.147 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.09e-07 2024-09-16T00:10:03.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688689+0000", 2024-09-16T00:10:03.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.536e-06 2024-09-16T00:10:03.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.148 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688699+0000", 2024-09-16T00:10:03.148 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9219999999999992e-06 2024-09-16T00:10:03.148 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.148 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.148 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688705+0000", 2024-09-16T00:10:03.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6549999999999998e-06 2024-09-16T00:10:03.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.707989+0000", 2024-09-16T00:10:03.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019283733000000001 2024-09-16T00:10:03.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.149 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.149 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.708029+0000", 2024-09-16T00:10:03.149 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9756999999999999e-05 2024-09-16T00:10:03.149 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.149 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.708092+0000", 2024-09-16T00:10:03.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3181000000000004e-05 2024-09-16T00:10:03.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.711530+0000", 2024-09-16T00:10:03.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003437842 2024-09-16T00:10:03.150 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.150 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.150 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.150 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714329+0000", 2024-09-16T00:10:03.150 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027988179999999998 2024-09-16T00:10:03.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714356+0000", 2024-09-16T00:10:03.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7107e-05 2024-09-16T00:10:03.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.151 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.151 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.151 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714375+0000", 2024-09-16T00:10:03.151 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9423999999999999e-05 2024-09-16T00:10:03.151 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.151 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.151 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.151 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.152 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2442 2.8 2:17051619:::200.0000000f:head [write 2200926~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.152 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.688930+0000", 2024-09-16T00:10:03.152 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.388297584, 2024-09-16T00:10:03.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025533200999999998, 2024-09-16T00:10:03.152 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.152 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.152 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.152 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.152 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.152 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.152 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688930+0000", 2024-09-16T00:10:03.152 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688930+0000", 2024-09-16T00:10:03.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.153 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688931+0000", 2024-09-16T00:10:03.153 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3e-07 2024-09-16T00:10:03.153 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.153 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.153 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688935+0000", 2024-09-16T00:10:03.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8139999999999998e-06 2024-09-16T00:10:03.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688942+0000", 2024-09-16T00:10:03.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9009999999999997e-06 2024-09-16T00:10:03.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.154 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.154 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688946+0000", 2024-09-16T00:10:03.154 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3810000000000001e-06 2024-09-16T00:10:03.154 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.154 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.708503+0000", 2024-09-16T00:10:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019556819999999999 2024-09-16T00:10:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.708583+0000", 2024-09-16T00:10:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0476000000000007e-05 2024-09-16T00:10:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.708709+0000", 2024-09-16T00:10:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012553799999999999 2024-09-16T00:10:03.155 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.711533+0000", 2024-09-16T00:10:03.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028240090000000002 2024-09-16T00:10:03.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714408+0000", 2024-09-16T00:10:03.156 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002875542 2024-09-16T00:10:03.156 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.156 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.156 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.156 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714444+0000", 2024-09-16T00:10:03.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5967999999999999e-05 2024-09-16T00:10:03.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.157 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.157 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714463+0000", 2024-09-16T00:10:03.157 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9122000000000001e-05 2024-09-16T00:10:03.157 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.157 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.157 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.157 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.157 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.157 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2443 2.8 2:17051619:::200.0000000f:head [write 2202835~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.157 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.689498+0000", 2024-09-16T00:10:03.157 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.387729362, 2024-09-16T00:10:03.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025058134999999999, 2024-09-16T00:10:03.158 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.158 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.158 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.158 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689498+0000", 2024-09-16T00:10:03.158 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.158 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.158 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.158 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.158 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689498+0000", 2024-09-16T00:10:03.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689499+0000", 2024-09-16T00:10:03.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4799999999999997e-07 2024-09-16T00:10:03.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.159 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689505+0000", 2024-09-16T00:10:03.159 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2789999999999998e-06 2024-09-16T00:10:03.159 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.159 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.159 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689514+0000", 2024-09-16T00:10:03.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1470000000000007e-06 2024-09-16T00:10:03.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689519+0000", 2024-09-16T00:10:03.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.775e-06 2024-09-16T00:10:03.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.160 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.160 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.160 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.708987+0000", 2024-09-16T00:10:03.160 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019468467999999999 2024-09-16T00:10:03.160 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.709032+0000", 2024-09-16T00:10:03.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4811999999999997e-05 2024-09-16T00:10:03.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.709098+0000", 2024-09-16T00:10:03.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5968000000000003e-05 2024-09-16T00:10:03.161 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.161 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.161 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.161 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.711536+0000", 2024-09-16T00:10:03.161 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024380650000000001 2024-09-16T00:10:03.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714497+0000", 2024-09-16T00:10:03.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029612330000000002 2024-09-16T00:10:03.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714536+0000", 2024-09-16T00:10:03.162 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:10:03.162 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.162 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.162 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.162 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714557+0000", 2024-09-16T00:10:03.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0380000000000001e-05 2024-09-16T00:10:03.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.163 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.163 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.163 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.163 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.163 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2444 2.8 2:17051619:::200.0000000f:head [write 2204744~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.163 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.689810+0000", 2024-09-16T00:10:03.163 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.387417712, 2024-09-16T00:10:03.163 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024842684, 2024-09-16T00:10:03.163 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.163 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.163 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.164 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689810+0000", 2024-09-16T00:10:03.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689810+0000", 2024-09-16T00:10:03.164 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.164 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.164 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.164 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.164 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689811+0000", 2024-09-16T00:10:03.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.032e-06 2024-09-16T00:10:03.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689815+0000", 2024-09-16T00:10:03.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3089999999999997e-06 2024-09-16T00:10:03.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.165 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689824+0000", 2024-09-16T00:10:03.165 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2849999999999995e-06 2024-09-16T00:10:03.165 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.165 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.165 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689828+0000", 2024-09-16T00:10:03.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-16T00:10:03.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.709404+0000", 2024-09-16T00:10:03.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019575134000000001 2024-09-16T00:10:03.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.166 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.166 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.166 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.709457+0000", 2024-09-16T00:10:03.166 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3140000000000003e-05 2024-09-16T00:10:03.166 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.709523+0000", 2024-09-16T00:10:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6688999999999998e-05 2024-09-16T00:10:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.711539+0000", 2024-09-16T00:10:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020153110000000001 2024-09-16T00:10:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714593+0000", 2024-09-16T00:10:03.167 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030542759999999999 2024-09-16T00:10:03.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714633+0000", 2024-09-16T00:10:03.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9817000000000002e-05 2024-09-16T00:10:03.168 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.168 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.168 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.168 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714653+0000", 2024-09-16T00:10:03.168 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0052e-05 2024-09-16T00:10:03.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.168 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.168 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.169 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.169 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2456 2.8 2:17051619:::200.0000000f:head [write 2229343~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.169 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.696827+0000", 2024-09-16T00:10:03.169 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.380401332, 2024-09-16T00:10:03.169 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025114133, 2024-09-16T00:10:03.169 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.169 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.169 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.169 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.169 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.169 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.169 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.696827+0000", 2024-09-16T00:10:03.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.696827+0000", 2024-09-16T00:10:03.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.170 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.696827+0000", 2024-09-16T00:10:03.170 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T00:10:03.170 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.170 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.170 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.696832+0000", 2024-09-16T00:10:03.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T00:10:03.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.696841+0000", 2024-09-16T00:10:03.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1239999999999994e-06 2024-09-16T00:10:03.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.171 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.171 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.171 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.696847+0000", 2024-09-16T00:10:03.171 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2559999999999999e-06 2024-09-16T00:10:03.171 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.715538+0000", 2024-09-16T00:10:03.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018690612999999998 2024-09-16T00:10:03.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.715584+0000", 2024-09-16T00:10:03.172 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6100999999999998e-05 2024-09-16T00:10:03.172 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.172 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.172 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.172 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.715648+0000", 2024-09-16T00:10:03.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4451999999999999e-05 2024-09-16T00:10:03.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.717300+0000", 2024-09-16T00:10:03.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016512510000000001 2024-09-16T00:10:03.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.173 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.721874+0000", 2024-09-16T00:10:03.173 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045744549999999998 2024-09-16T00:10:03.173 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.173 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.173 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.721905+0000", 2024-09-16T00:10:03.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0898999999999997e-05 2024-09-16T00:10:03.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.174 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.174 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.721941+0000", 2024-09-16T00:10:03.174 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5756000000000001e-05 2024-09-16T00:10:03.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.174 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.174 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.174 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.174 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.174 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2457 2.8 2:17051619:::200.0000000f:head [write 2231252~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.175 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.697232+0000", 2024-09-16T00:10:03.175 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.379995449999999, 2024-09-16T00:10:03.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024791982000000001, 2024-09-16T00:10:03.175 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.175 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.175 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.175 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.175 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.697232+0000", 2024-09-16T00:10:03.175 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.175 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.175 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.175 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.697232+0000", 2024-09-16T00:10:03.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.697233+0000", 2024-09-16T00:10:03.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8400000000000005e-07 2024-09-16T00:10:03.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.176 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.176 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.176 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.697237+0000", 2024-09-16T00:10:03.176 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7359999999999999e-06 2024-09-16T00:10:03.176 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.697245+0000", 2024-09-16T00:10:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3329999999999997e-06 2024-09-16T00:10:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.697251+0000", 2024-09-16T00:10:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9549999999999997e-06 2024-09-16T00:10:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.715904+0000", 2024-09-16T00:10:03.177 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018653075000000002 2024-09-16T00:10:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.715981+0000", 2024-09-16T00:10:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7337999999999997e-05 2024-09-16T00:10:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.716110+0000", 2024-09-16T00:10:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012847100000000001 2024-09-16T00:10:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.178 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.717302+0000", 2024-09-16T00:10:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011925029999999999 2024-09-16T00:10:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.721976+0000", 2024-09-16T00:10:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004673478 2024-09-16T00:10:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.722002+0000", 2024-09-16T00:10:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6438999999999999e-05 2024-09-16T00:10:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.179 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.722024+0000", 2024-09-16T00:10:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2169999999999999e-05 2024-09-16T00:10:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2683 2.8 2:17051619:::200.0000000f:head [write 3733504~9752 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:50.010774+0000", 2024-09-16T00:10:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.066453815999999, 2024-09-16T00:10:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.057674191999999999, 2024-09-16T00:10:03.180 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.010774+0000", 2024-09-16T00:10:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.010774+0000", 2024-09-16T00:10:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.181 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.010776+0000", 2024-09-16T00:10:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9709999999999998e-06 2024-09-16T00:10:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.010795+0000", 2024-09-16T00:10:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8732e-05 2024-09-16T00:10:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.010813+0000", 2024-09-16T00:10:03.182 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8199999999999999e-05 2024-09-16T00:10:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.010822+0000", 2024-09-16T00:10:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4870000000000008e-06 2024-09-16T00:10:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.010876+0000", 2024-09-16T00:10:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3594999999999998e-05 2024-09-16T00:10:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.183 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.010931+0000", 2024-09-16T00:10:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5047000000000003e-05 2024-09-16T00:10:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.011025+0000", 2024-09-16T00:10:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4384999999999998e-05 2024-09-16T00:10:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.011988+0000", 2024-09-16T00:10:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00096290000000000004 2024-09-16T00:10:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.184 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.068368+0000", 2024-09-16T00:10:03.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.056379187999999997 2024-09-16T00:10:03.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.068407+0000", 2024-09-16T00:10:03.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.964e-05 2024-09-16T00:10:03.185 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.185 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.185 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.185 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.068448+0000", 2024-09-16T00:10:03.185 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1047000000000002e-05 2024-09-16T00:10:03.185 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.186 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.186 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.186 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.186 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2684 2.8 2:17051619:::200.0000000f:head [write 3743256~15003 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.186 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:50.023831+0000", 2024-09-16T00:10:03.186 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.053396414, 2024-09-16T00:10:03.186 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045369923, 2024-09-16T00:10:03.186 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.186 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.186 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.186 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.186 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.023831+0000", 2024-09-16T00:10:03.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.023831+0000", 2024-09-16T00:10:03.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.187 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.187 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.187 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.023833+0000", 2024-09-16T00:10:03.187 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9630000000000001e-06 2024-09-16T00:10:03.187 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.023858+0000", 2024-09-16T00:10:03.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4610999999999999e-05 2024-09-16T00:10:03.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.023874+0000", 2024-09-16T00:10:03.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6472000000000001e-05 2024-09-16T00:10:03.188 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.188 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.188 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.188 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.023882+0000", 2024-09-16T00:10:03.188 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7829999999999999e-06 2024-09-16T00:10:03.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.023928+0000", 2024-09-16T00:10:03.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6008999999999999e-05 2024-09-16T00:10:03.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.189 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.023991+0000", 2024-09-16T00:10:03.189 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2298000000000006e-05 2024-09-16T00:10:03.189 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.189 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.189 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024075+0000", 2024-09-16T00:10:03.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4178999999999995e-05 2024-09-16T00:10:03.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.025111+0000", 2024-09-16T00:10:03.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010360499999999999 2024-09-16T00:10:03.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.190 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.190 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.190 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.069142+0000", 2024-09-16T00:10:03.190 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044031305999999999 2024-09-16T00:10:03.190 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.069175+0000", 2024-09-16T00:10:03.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3182000000000002e-05 2024-09-16T00:10:03.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.191 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.191 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.069201+0000", 2024-09-16T00:10:03.191 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6069999999999999e-05 2024-09-16T00:10:03.191 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.191 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.191 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.191 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.191 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.192 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2685 2.8 2:17051619:::200.0000000f:head [write 3758259~1959 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:03.192 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:50.024646+0000", 2024-09-16T00:10:03.192 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 13.052582136, 2024-09-16T00:10:03.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045535516999999998, 2024-09-16T00:10:03.192 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.192 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.192 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.192 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.192 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.192 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.192 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024646+0000", 2024-09-16T00:10:03.192 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.192 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024646+0000", 2024-09-16T00:10:03.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024647+0000", 2024-09-16T00:10:03.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9400000000000004e-07 2024-09-16T00:10:03.193 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.193 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.193 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.193 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024652+0000", 2024-09-16T00:10:03.193 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4949999999999999e-06 2024-09-16T00:10:03.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024671+0000", 2024-09-16T00:10:03.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9139000000000001e-05 2024-09-16T00:10:03.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.194 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024680+0000", 2024-09-16T00:10:03.194 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9560000000000003e-06 2024-09-16T00:10:03.194 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.194 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.194 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024716+0000", 2024-09-16T00:10:03.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6310000000000003e-05 2024-09-16T00:10:03.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024761+0000", 2024-09-16T00:10:03.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4258000000000002e-05 2024-09-16T00:10:03.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.195 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.195 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:03.195 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024827+0000", 2024-09-16T00:10:03.195 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6528999999999999e-05 2024-09-16T00:10:03.195 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.025686+0000", 2024-09-16T00:10:03.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085850800000000001 2024-09-16T00:10:03.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.070124+0000", 2024-09-16T00:10:03.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044437890000000001 2024-09-16T00:10:03.196 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.196 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.196 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.196 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.070156+0000", 2024-09-16T00:10:03.196 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2481999999999998e-05 2024-09-16T00:10:03.197 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.197 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.197 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.197 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.070181+0000", 2024-09-16T00:10:03.197 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5055999999999999e-05 2024-09-16T00:10:03.197 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.197 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.197 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.197 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.197 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.197 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:03.201 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:03.198+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:03.202 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:10:03.278 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.277+0000 7f5e77fff640 1 -- 172.21.15.139:0/2267242893 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v97) ==== 113+0+0 (secure 0 0 0) 0x7f5e8005cd50 con 0x7f5e90068060 2024-09-16T00:10:03.278 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T00:10:03.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.280+0000 7f5e95add640 1 -- 172.21.15.139:0/2267242893 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e7c06c2b0 msgr2=0x7f5e7c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:03.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.280+0000 7f5e95add640 1 --2- 172.21.15.139:0/2267242893 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e7c06c2b0 0x7f5e7c06e770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5e901eb160 tx=0x7f5e84009040 comp rx=0 tx=0).stop 2024-09-16T00:10:03.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.280+0000 7f5e95add640 1 -- 172.21.15.139:0/2267242893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e90068060 msgr2=0x7f5e900cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.280+0000 7f5e95add640 1 --2- 172.21.15.139:0/2267242893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e90068060 0x7f5e900cfbf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000c500 tx=0x7f5e8000c9c0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.280+0000 7f5e95add640 1 -- 172.21.15.139:0/2267242893 shutdown_connections 2024-09-16T00:10:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.280+0000 7f5e95add640 1 --2- 172.21.15.139:0/2267242893 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5e7c06c2b0 0x7f5e7c06e770 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.280+0000 7f5e95add640 1 --2- 172.21.15.139:0/2267242893 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5e9015b280 0x7f5e901f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.280+0000 7f5e95add640 1 --2- 172.21.15.139:0/2267242893 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5e90154c20 0x7f5e90157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.280+0000 7f5e95add640 1 --2- 172.21.15.139:0/2267242893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e90068060 0x7f5e900cfbf0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.280+0000 7f5e95add640 1 -- 172.21.15.139:0/2267242893 >> 172.21.15.139:0/2267242893 conn(0x7f5e9006ca40 msgr2=0x7f5e9014ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.280+0000 7f5e95add640 1 -- 172.21.15.139:0/2267242893 shutdown_connections 2024-09-16T00:10:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.280+0000 7f5e95add640 1 -- 172.21.15.139:0/2267242893 wait complete. 2024-09-16T00:10:03.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.280+0000 7fc4077fe640 1 -- 172.21.15.139:0/923504031 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v97) ==== 101+0+0 (secure 0 0 0) 0x7fc3f805d790 con 0x7fc4100c60c0 2024-09-16T00:10:03.283 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is unset 2024-09-16T00:10:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.283+0000 7fc4153cf640 1 -- 172.21.15.139:0/923504031 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc3f006c2e0 msgr2=0x7fc3f006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.283+0000 7fc4153cf640 1 --2- 172.21.15.139:0/923504031 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc3f006c2e0 0x7fc3f006e7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc400007fd0 tx=0x7fc400009040 comp rx=0 tx=0).stop 2024-09-16T00:10:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.283+0000 7fc4153cf640 1 -- 172.21.15.139:0/923504031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4100c60c0 msgr2=0x7fc4101efc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:03.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.283+0000 7fc4153cf640 1 --2- 172.21.15.139:0/923504031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4100c60c0 0x7fc4101efc50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800e9e0 tx=0x7fc3f800eea0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.283+0000 7fc4153cf640 1 -- 172.21.15.139:0/923504031 shutdown_connections 2024-09-16T00:10:03.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.283+0000 7fc4153cf640 1 --2- 172.21.15.139:0/923504031 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fc3f006c2e0 0x7fc3f006e7a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.283+0000 7fc4153cf640 1 --2- 172.21.15.139:0/923504031 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fc4100c64a0 0x7fc410157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.283+0000 7fc4153cf640 1 --2- 172.21.15.139:0/923504031 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fc4100c60c0 0x7fc4101efc50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.283+0000 7fc4153cf640 1 --2- 172.21.15.139:0/923504031 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc41006d080 0x7fc41006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.283+0000 7fc4153cf640 1 -- 172.21.15.139:0/923504031 >> 172.21.15.139:0/923504031 conn(0x7fc4100cc740 msgr2=0x7fc410152360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:03.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.283+0000 7fc4153cf640 1 -- 172.21.15.139:0/923504031 shutdown_connections 2024-09-16T00:10:03.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.283+0000 7fc4153cf640 1 -- 172.21.15.139:0/923504031 wait complete. 2024-09-16T00:10:03.296 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T00:10:03.299 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:03.299+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:03.312 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:03.312 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:03.312 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:03.313 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:03.321 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:10:03.401 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:03.400+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:03.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.444+0000 7fdf5ff18640 1 Processor -- start 2024-09-16T00:10:03.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.444+0000 7fdf5ff18640 1 -- start start 2024-09-16T00:10:03.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.445+0000 7fdf5ff18640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf58068060 0x7fdf5815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:03.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.445+0000 7fdf5ff18640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdf580ccc90 0x7fdf580cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:03.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.445+0000 7fdf5ff18640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf580cc4b0 con 0x7fdf58154c20 2024-09-16T00:10:03.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.445+0000 7fdf5ff18640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdf580c5830 con 0x7fdf580ccc90 2024-09-16T00:10:03.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.445+0000 7fdf5ff18640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5815b7c0 con 0x7fdf58068060 2024-09-16T00:10:03.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.445+0000 7fdf5e4b5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdf580ccc90 0x7fdf580cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:03.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.445+0000 7fdf5e4b5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdf580ccc90 0x7fdf580cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55878/0 (socket says 172.21.15.139:55878) 2024-09-16T00:10:03.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.445+0000 7fdf5e4b5640 1 -- 172.21.15.139:0/1138491852 learned_addr learned my addr 172.21.15.139:0/1138491852 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:03.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.445+0000 7fdf5d4b3640 1 --2- 172.21.15.139:0/1138491852 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf58068060 0x7fdf5815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:03.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.446+0000 7fdf5e4b5640 1 -- 172.21.15.139:0/1138491852 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf58068060 msgr2=0x7fdf5815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:03.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.446+0000 7fdf5e4b5640 1 --2- 172.21.15.139:0/1138491852 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf58068060 0x7fdf5815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.446+0000 7fdf5e4b5640 1 -- 172.21.15.139:0/1138491852 >> v1:172.21.15.164:6789/0 conn(0x7fdf58154c20 legacy=0x7fdf580c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:03.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.446+0000 7fdf5e4b5640 1 -- 172.21.15.139:0/1138491852 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf580cc790 con 0x7fdf580ccc90 2024-09-16T00:10:03.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.446+0000 7fdf5d4b3640 1 --2- 172.21.15.139:0/1138491852 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf58068060 0x7fdf5815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:10:03.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.446+0000 7fdf5e4b5640 1 --2- 172.21.15.139:0/1138491852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf580ccc90 0x7fdf580cd0c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009e50 tx=0x7fdf5002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e992bf3d9f2e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:03.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.446+0000 7fdf5ccb2640 1 -- 172.21.15.139:0/1138491852 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50036070 con 0x7fdf580ccc90 2024-09-16T00:10:03.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.446+0000 7fdf5ccb2640 1 -- 172.21.15.139:0/1138491852 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf50031600 con 0x7fdf580ccc90 2024-09-16T00:10:03.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.446+0000 7fdf5ccb2640 1 -- 172.21.15.139:0/1138491852 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf500319d0 con 0x7fdf580ccc90 2024-09-16T00:10:03.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.447+0000 7fdf5ff18640 1 -- 172.21.15.139:0/1138491852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf580ccc90 msgr2=0x7fdf580cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:03.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.447+0000 7fdf5ff18640 1 --2- 172.21.15.139:0/1138491852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf580ccc90 0x7fdf580cd0c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009e50 tx=0x7fdf5002a7a0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.447+0000 7fdf5ff18640 1 -- 172.21.15.139:0/1138491852 shutdown_connections 2024-09-16T00:10:03.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.447+0000 7fdf5ff18640 1 --2- 172.21.15.139:0/1138491852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf580ccc90 0x7fdf580cd0c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.447+0000 7fdf5ff18640 1 --2- 172.21.15.139:0/1138491852 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf58068060 0x7fdf5815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.447+0000 7fdf5ff18640 1 -- 172.21.15.139:0/1138491852 >> 172.21.15.139:0/1138491852 conn(0x7fdf5806ca40 msgr2=0x7fdf58067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:03.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.447+0000 7fdf5ff18640 1 -- 172.21.15.139:0/1138491852 shutdown_connections 2024-09-16T00:10:03.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.447+0000 7fdf5ff18640 1 -- 172.21.15.139:0/1138491852 wait complete. 2024-09-16T00:10:03.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.448+0000 7fdf5ff18640 1 Processor -- start 2024-09-16T00:10:03.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.449+0000 7fdf5ff18640 1 -- start start 2024-09-16T00:10:03.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.449+0000 7fdf5ff18640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf58068060 0x7fdf580c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:03.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.449+0000 7fdf5ff18640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf580ccc90 0x7fdf5815d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:03.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.449+0000 7fdf5ff18640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf58154c20 0x7fdf580c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:03.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.449+0000 7fdf5ff18640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5815e0d0 con 0x7fdf58154c20 2024-09-16T00:10:03.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.449+0000 7fdf5ff18640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fdf580cc260 con 0x7fdf580ccc90 2024-09-16T00:10:03.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.449+0000 7fdf5ff18640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5815e3b0 con 0x7fdf58068060 2024-09-16T00:10:03.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.449+0000 7fdf5e4b5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf58154c20 0x7fdf580c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:03.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.449+0000 7fdf5d4b3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf580ccc90 0x7fdf5815d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:03.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.449+0000 7fdf5dcb4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf58068060 0x7fdf580c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:03.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.450+0000 7fdf5e4b5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf58154c20 0x7fdf580c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55886/0 (socket says 172.21.15.139:55886) 2024-09-16T00:10:03.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.450+0000 7fdf5e4b5640 1 -- 172.21.15.139:0/2867366428 learned_addr learned my addr 172.21.15.139:0/2867366428 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:03.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.450+0000 7fdf5d4b3640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf580ccc90 0x7fdf5815d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:49846/0 (socket says 172.21.15.139:49846) 2024-09-16T00:10:03.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.450+0000 7fdf5e4b5640 1 -- 172.21.15.139:0/2867366428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf58068060 msgr2=0x7fdf580c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:03.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.450+0000 7fdf5e4b5640 1 --2- 172.21.15.139:0/2867366428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf58068060 0x7fdf580c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.450+0000 7fdf5e4b5640 1 -- 172.21.15.139:0/2867366428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf580ccc90 msgr2=0x7fdf5815d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:03.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.450+0000 7fdf5e4b5640 1 --2- 172.21.15.139:0/2867366428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf580ccc90 0x7fdf5815d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.450+0000 7fdf5e4b5640 1 -- 172.21.15.139:0/2867366428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5815a220 con 0x7fdf58154c20 2024-09-16T00:10:03.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.450+0000 7fdf5e4b5640 1 --2- 172.21.15.139:0/2867366428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf58154c20 0x7fdf580c8400 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdf50034040 tx=0x7fdf50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:03.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.450+0000 7fdf4e7fc640 1 -- 172.21.15.139:0/2867366428 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50036050 con 0x7fdf58154c20 2024-09-16T00:10:03.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.450+0000 7fdf4e7fc640 1 -- 172.21.15.139:0/2867366428 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf50042070 con 0x7fdf58154c20 2024-09-16T00:10:03.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.450+0000 7fdf5ff18640 1 -- 172.21.15.139:0/2867366428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf581f1020 con 0x7fdf58154c20 2024-09-16T00:10:03.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.450+0000 7fdf4e7fc640 1 -- 172.21.15.139:0/2867366428 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50009510 con 0x7fdf58154c20 2024-09-16T00:10:03.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.450+0000 7fdf5ff18640 1 -- 172.21.15.139:0/2867366428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf581f15e0 con 0x7fdf58154c20 2024-09-16T00:10:03.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.451+0000 7fdf5ff18640 1 -- 172.21.15.139:0/2867366428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf580b3bb0 con 0x7fdf58154c20 2024-09-16T00:10:03.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.452+0000 7fdf4e7fc640 1 -- 172.21.15.139:0/2867366428 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdf50007bf0 con 0x7fdf58154c20 2024-09-16T00:10:03.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.453+0000 7fdf4e7fc640 1 --2- 172.21.15.139:0/2867366428 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdf3c06c2e0 0x7fdf3c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:03.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.453+0000 7fdf5dcb4640 1 --2- 172.21.15.139:0/2867366428 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdf3c06c2e0 0x7fdf3c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:03.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.453+0000 7fdf4e7fc640 1 -- 172.21.15.139:0/2867366428 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5973+0+0 (secure 0 0 0) 0x7fdf500abc20 con 0x7fdf58154c20 2024-09-16T00:10:03.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.453+0000 7fdf5dcb4640 1 --2- 172.21.15.139:0/2867366428 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdf3c06c2e0 0x7fdf3c06e7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdf540016e0 tx=0x7fdf54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:03.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.457+0000 7fdf4e7fc640 1 -- 172.21.15.139:0/2867366428 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdf580b3bb0 con 0x7fdf58154c20 2024-09-16T00:10:03.470 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:03.470 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:03.470 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:03.470 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:03.470 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:03.479 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:10:03.522 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:03.522+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:03.590 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:03.590 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:03.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:03.590 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:03.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.591 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5178 3.2c 3:375e1b9b:::100000030c8.00000002:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:03.591 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:03.591 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.77987378900000004, 2024-09-16T00:10:03.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012118438, 2024-09-16T00:10:03.591 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.591 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.591 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.591 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:03.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:03.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808909+0000", 2024-09-16T00:10:03.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8160000000000001e-06 2024-09-16T00:10:03.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808921+0000", 2024-09-16T00:10:03.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1117999999999999e-05 2024-09-16T00:10:03.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808941+0000", 2024-09-16T00:10:03.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0712000000000001e-05 2024-09-16T00:10:03.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808952+0000", 2024-09-16T00:10:03.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0193e-05 2024-09-16T00:10:03.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809019+0000", 2024-09-16T00:10:03.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7404999999999996e-05 2024-09-16T00:10:03.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809100+0000", 2024-09-16T00:10:03.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1029e-05 2024-09-16T00:10:03.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:03.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809206+0000", 2024-09-16T00:10:03.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105603 2024-09-16T00:10:03.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.815162+0000", 2024-09-16T00:10:03.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059567250000000004 2024-09-16T00:10:03.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.820925+0000", 2024-09-16T00:10:03.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057623479999999996 2024-09-16T00:10:03.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.820987+0000", 2024-09-16T00:10:03.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2680000000000003e-05 2024-09-16T00:10:03.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.821026+0000", 2024-09-16T00:10:03.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8809000000000001e-05 2024-09-16T00:10:03.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.596 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.596 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5196 3.26 3:66ea4dfb:::100000030f0.00000001:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.77868676100000001, 2024-09-16T00:10:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095222459999999998, 2024-09-16T00:10:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3499999999999995e-07 2024-09-16T00:10:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810102+0000", 2024-09-16T00:10:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7390000000000002e-06 2024-09-16T00:10:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810112+0000", 2024-09-16T00:10:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7550000000000006e-06 2024-09-16T00:10:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810127+0000", 2024-09-16T00:10:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4749999999999999e-05 2024-09-16T00:10:03.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810184+0000", 2024-09-16T00:10:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7701999999999997e-05 2024-09-16T00:10:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810256+0000", 2024-09-16T00:10:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1595999999999997e-05 2024-09-16T00:10:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:03.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810346+0000", 2024-09-16T00:10:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0043999999999994e-05 2024-09-16T00:10:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.815239+0000", 2024-09-16T00:10:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048934470000000004 2024-09-16T00:10:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819526+0000", 2024-09-16T00:10:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042868630000000001 2024-09-16T00:10:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819577+0000", 2024-09-16T00:10:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0445999999999998e-05 2024-09-16T00:10:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819617+0000", 2024-09-16T00:10:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0169000000000001e-05 2024-09-16T00:10:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.602 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5208 3.2c 3:35053d24:::100000031dc.00000001:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.816469+0000", 2024-09-16T00:10:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.77231196499999999, 2024-09-16T00:10:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070242580000000002, 2024-09-16T00:10:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.603 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.816469+0000", 2024-09-16T00:10:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.816469+0000", 2024-09-16T00:10:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.816471+0000", 2024-09-16T00:10:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.564e-06 2024-09-16T00:10:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.816476+0000", 2024-09-16T00:10:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3159999999999996e-06 2024-09-16T00:10:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.816489+0000", 2024-09-16T00:10:03.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2278e-05 2024-09-16T00:10:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.816496+0000", 2024-09-16T00:10:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2810000000000003e-06 2024-09-16T00:10:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.816614+0000", 2024-09-16T00:10:03.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011796899999999999 2024-09-16T00:10:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.816691+0000", 2024-09-16T00:10:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7577999999999995e-05 2024-09-16T00:10:03.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.816786+0000", 2024-09-16T00:10:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4239000000000004e-05 2024-09-16T00:10:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819846+0000", 2024-09-16T00:10:03.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030603679999999999 2024-09-16T00:10:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.823388+0000", 2024-09-16T00:10:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003541715 2024-09-16T00:10:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.823455+0000", 2024-09-16T00:10:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7445000000000002e-05 2024-09-16T00:10:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.823494+0000", 2024-09-16T00:10:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8504999999999998e-05 2024-09-16T00:10:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5214 2.11 2:8c4c9674:::100000031e5.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.818494+0000", 2024-09-16T00:10:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.77028735400000004, 2024-09-16T00:10:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071416159999999999, 2024-09-16T00:10:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.611 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.818494+0000", 2024-09-16T00:10:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.818494+0000", 2024-09-16T00:10:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.818495+0000", 2024-09-16T00:10:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.328e-06 2024-09-16T00:10:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.818501+0000", 2024-09-16T00:10:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2009999999999998e-06 2024-09-16T00:10:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.818516+0000", 2024-09-16T00:10:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5077e-05 2024-09-16T00:10:03.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.818523+0000", 2024-09-16T00:10:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3150000000000003e-06 2024-09-16T00:10:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.818583+0000", 2024-09-16T00:10:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0429e-05 2024-09-16T00:10:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.818710+0000", 2024-09-16T00:10:03.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012688700000000001 2024-09-16T00:10:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.825636+0000", 2024-09-16T00:10:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069253789999999997 2024-09-16T00:10:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5215 3.2c 3:366cbeb6:::100000031e6.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.819340+0000", 2024-09-16T00:10:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.76944119099999997, 2024-09-16T00:10:03.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0076582689999999997, 2024-09-16T00:10:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819340+0000", 2024-09-16T00:10:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819340+0000", 2024-09-16T00:10:03.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819341+0000", 2024-09-16T00:10:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5000000000000001e-07 2024-09-16T00:10:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819345+0000", 2024-09-16T00:10:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0239999999999999e-06 2024-09-16T00:10:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819360+0000", 2024-09-16T00:10:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4513e-05 2024-09-16T00:10:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819367+0000", 2024-09-16T00:10:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8460000000000008e-06 2024-09-16T00:10:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819437+0000", 2024-09-16T00:10:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9851999999999995e-05 2024-09-16T00:10:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819560+0000", 2024-09-16T00:10:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000123058 2024-09-16T00:10:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.826999+0000", 2024-09-16T00:10:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074381259999999998 2024-09-16T00:10:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.619 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5244 3.20 3:068a06d7:::100000032f2.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.76159125400000005, 2024-09-16T00:10:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094083010000000009, 2024-09-16T00:10:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827191+0000", 2024-09-16T00:10:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2330000000000001e-06 2024-09-16T00:10:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827195+0000", 2024-09-16T00:10:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8750000000000002e-06 2024-09-16T00:10:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827206+0000", 2024-09-16T00:10:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0473e-05 2024-09-16T00:10:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827213+0000", 2024-09-16T00:10:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:10:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827275+0000", 2024-09-16T00:10:03.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2532000000000004e-05 2024-09-16T00:10:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827392+0000", 2024-09-16T00:10:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116266 2024-09-16T00:10:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.836599+0000", 2024-09-16T00:10:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092068069999999991 2024-09-16T00:10:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5245 3.26 3:64562175:::100000032f3.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.827707+0000", 2024-09-16T00:10:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.761074484, 2024-09-16T00:10:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090401860000000004, 2024-09-16T00:10:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827707+0000", 2024-09-16T00:10:03.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827707+0000", 2024-09-16T00:10:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827708+0000", 2024-09-16T00:10:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5199999999999996e-07 2024-09-16T00:10:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827712+0000", 2024-09-16T00:10:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9530000000000001e-06 2024-09-16T00:10:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827722+0000", 2024-09-16T00:10:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0524e-05 2024-09-16T00:10:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827736+0000", 2024-09-16T00:10:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3713999999999999e-05 2024-09-16T00:10:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827801+0000", 2024-09-16T00:10:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5580000000000006e-05 2024-09-16T00:10:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827924+0000", 2024-09-16T00:10:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012308499999999999 2024-09-16T00:10:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.836747+0000", 2024-09-16T00:10:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088226780000000005 2024-09-16T00:10:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5251 2.17 2:e817bfdf:::100000037ad.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.827758+0000", 2024-09-16T00:10:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.76102386, 2024-09-16T00:10:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090250260000000002, 2024-09-16T00:10:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.628 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827758+0000", 2024-09-16T00:10:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827758+0000", 2024-09-16T00:10:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827758+0000", 2024-09-16T00:10:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000003e-07 2024-09-16T00:10:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827761+0000", 2024-09-16T00:10:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.095e-06 2024-09-16T00:10:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827772+0000", 2024-09-16T00:10:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0767e-05 2024-09-16T00:10:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827779+0000", 2024-09-16T00:10:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-16T00:10:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827837+0000", 2024-09-16T00:10:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7615000000000003e-05 2024-09-16T00:10:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827942+0000", 2024-09-16T00:10:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105393 2024-09-16T00:10:03.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.836783+0000", 2024-09-16T00:10:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088403279999999997 2024-09-16T00:10:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5252 2.1a 2:5913b086:::1000000336d.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.76065543099999999, 2024-09-16T00:10:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010086441999999999, 2024-09-16T00:10:03.632 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828127+0000", 2024-09-16T00:10:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:10:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828140+0000", 2024-09-16T00:10:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3614e-05 2024-09-16T00:10:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828149+0000", 2024-09-16T00:10:03.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7250000000000003e-06 2024-09-16T00:10:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828156+0000", 2024-09-16T00:10:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9430000000000003e-06 2024-09-16T00:10:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828274+0000", 2024-09-16T00:10:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118121 2024-09-16T00:10:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828381+0000", 2024-09-16T00:10:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107379 2024-09-16T00:10:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.838212+0000", 2024-09-16T00:10:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098310089999999999 2024-09-16T00:10:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(3.2e epoch 96/94 rep_tid 1436 entries 96'208 (0'0) error 3:74bd5d8c:::10000003205.00000000:head by mds.0.4:5255 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 95'206)", 2024-09-16T00:10:03.636 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.829732+0000", 2024-09-16T00:10:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.75904940799999998, 2024-09-16T00:10:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0070354290000000002, 2024-09-16T00:10:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.637 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:10:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.829732+0000", 2024-09-16T00:10:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.829732+0000", 2024-09-16T00:10:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.829733+0000", 2024-09-16T00:10:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2699999999999999e-07 2024-09-16T00:10:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.829741+0000", 2024-09-16T00:10:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1009999999999992e-06 2024-09-16T00:10:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.829763+0000", 2024-09-16T00:10:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2064000000000001e-05 2024-09-16T00:10:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.829777+0000", 2024-09-16T00:10:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4037e-05 2024-09-16T00:10:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.829849+0000", 2024-09-16T00:10:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2212999999999995e-05 2024-09-16T00:10:03.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.836767+0000", 2024-09-16T00:10:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069182870000000004 2024-09-16T00:10:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5263 3.10 3:08fcb374:::10000003285.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.831176+0000", 2024-09-16T00:10:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.75760541299999995, 2024-09-16T00:10:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005972003, 2024-09-16T00:10:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.831176+0000", 2024-09-16T00:10:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.831176+0000", 2024-09-16T00:10:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.831178+0000", 2024-09-16T00:10:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.53e-06 2024-09-16T00:10:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.831182+0000", 2024-09-16T00:10:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8980000000000004e-06 2024-09-16T00:10:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.831196+0000", 2024-09-16T00:10:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3553000000000001e-05 2024-09-16T00:10:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.831204+0000", 2024-09-16T00:10:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5259999999999996e-06 2024-09-16T00:10:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.831272+0000", 2024-09-16T00:10:03.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8830000000000003e-05 2024-09-16T00:10:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.831398+0000", 2024-09-16T00:10:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000125337 2024-09-16T00:10:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.837148+0000", 2024-09-16T00:10:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057503290000000002 2024-09-16T00:10:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.645 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5282 3.36 3:6d09700c:::1000000346e.00000002:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.834462+0000", 2024-09-16T00:10:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.75431901199999996, 2024-09-16T00:10:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061022819999999997, 2024-09-16T00:10:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.834462+0000", 2024-09-16T00:10:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.834462+0000", 2024-09-16T00:10:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.834464+0000", 2024-09-16T00:10:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.601e-06 2024-09-16T00:10:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.834470+0000", 2024-09-16T00:10:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.037e-06 2024-09-16T00:10:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.834486+0000", 2024-09-16T00:10:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5693999999999999e-05 2024-09-16T00:10:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.834494+0000", 2024-09-16T00:10:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2069999999999995e-06 2024-09-16T00:10:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.834554+0000", 2024-09-16T00:10:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9941000000000001e-05 2024-09-16T00:10:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.834706+0000", 2024-09-16T00:10:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015179399999999999 2024-09-16T00:10:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.840565+0000", 2024-09-16T00:10:03.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058590079999999998 2024-09-16T00:10:03.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.650 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.650 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.650 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5286 3.26 3:661d81ee:::100000034b1.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:03.650 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.835257+0000", 2024-09-16T00:10:03.651 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.75352475399999996, 2024-09-16T00:10:03.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084025320000000008, 2024-09-16T00:10:03.651 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.651 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.651 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:03.651 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835257+0000", 2024-09-16T00:10:03.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835257+0000", 2024-09-16T00:10:03.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835257+0000", 2024-09-16T00:10:03.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4700000000000002e-07 2024-09-16T00:10:03.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835260+0000", 2024-09-16T00:10:03.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.593e-06 2024-09-16T00:10:03.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835267+0000", 2024-09-16T00:10:03.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4880000000000001e-06 2024-09-16T00:10:03.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835274+0000", 2024-09-16T00:10:03.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2920000000000001e-06 2024-09-16T00:10:03.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835358+0000", 2024-09-16T00:10:03.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4907999999999994e-05 2024-09-16T00:10:03.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835475+0000", 2024-09-16T00:10:03.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116267 2024-09-16T00:10:03.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.843659+0000", 2024-09-16T00:10:03.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081845370000000004 2024-09-16T00:10:03.655 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.655 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.655 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.655 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5290 3.35 3:af85dbe5:::100000034d0.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:03.655 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.835287+0000", 2024-09-16T00:10:03.655 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.75349470799999996, 2024-09-16T00:10:03.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063315089999999999, 2024-09-16T00:10:03.655 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.655 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.655 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.655 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835287+0000", 2024-09-16T00:10:03.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835287+0000", 2024-09-16T00:10:03.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835287+0000", 2024-09-16T00:10:03.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5299999999999999e-07 2024-09-16T00:10:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835290+0000", 2024-09-16T00:10:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.791e-06 2024-09-16T00:10:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835297+0000", 2024-09-16T00:10:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2400000000000001e-06 2024-09-16T00:10:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835302+0000", 2024-09-16T00:10:03.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2390000000000002e-06 2024-09-16T00:10:03.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835362+0000", 2024-09-16T00:10:03.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9728000000000001e-05 2024-09-16T00:10:03.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835445+0000", 2024-09-16T00:10:03.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2550000000000001e-05 2024-09-16T00:10:03.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:03.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835540+0000", 2024-09-16T00:10:03.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5068999999999994e-05 2024-09-16T00:10:03.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:03.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.837062+0000", 2024-09-16T00:10:03.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0015226269999999999 2024-09-16T00:10:03.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.659 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:03.659 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841516+0000", 2024-09-16T00:10:03.659 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044535759999999999 2024-09-16T00:10:03.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841566+0000", 2024-09-16T00:10:03.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9558999999999998e-05 2024-09-16T00:10:03.660 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.660 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.660 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.660 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841618+0000", 2024-09-16T00:10:03.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2676999999999997e-05 2024-09-16T00:10:03.660 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.660 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.660 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.661 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5296 3.10 3:08253ab3:::100000037b2.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:03.661 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.839046+0000", 2024-09-16T00:10:03.661 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.74973561, 2024-09-16T00:10:03.661 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086364030000000008, 2024-09-16T00:10:03.661 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.661 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.661 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:03.661 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.661 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.661 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.661 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839046+0000", 2024-09-16T00:10:03.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839046+0000", 2024-09-16T00:10:03.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839047+0000", 2024-09-16T00:10:03.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0299999999999998e-07 2024-09-16T00:10:03.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839050+0000", 2024-09-16T00:10:03.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4989999999999999e-06 2024-09-16T00:10:03.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839062+0000", 2024-09-16T00:10:03.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2128e-05 2024-09-16T00:10:03.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839646+0000", 2024-09-16T00:10:03.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058339999999999998 2024-09-16T00:10:03.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839707+0000", 2024-09-16T00:10:03.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1780999999999997e-05 2024-09-16T00:10:03.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839842+0000", 2024-09-16T00:10:03.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013437799999999999 2024-09-16T00:10:03.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.847682+0000", 2024-09-16T00:10:03.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0078404140000000004 2024-09-16T00:10:03.666 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.666 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.666 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.666 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.19 epoch 96/94 rep_tid 2313 entries 96'15 (0'0) error 2:9f9cad8a:::100000033d1.00000000:head by mds.0.4:5304 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 95'13)", 2024-09-16T00:10:03.666 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.841700+0000", 2024-09-16T00:10:03.666 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.74708151099999998, 2024-09-16T00:10:03.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086308140000000005, 2024-09-16T00:10:03.666 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.666 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.666 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:10:03.666 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841700+0000", 2024-09-16T00:10:03.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841700+0000", 2024-09-16T00:10:03.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841701+0000", 2024-09-16T00:10:03.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4199999999999995e-07 2024-09-16T00:10:03.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841709+0000", 2024-09-16T00:10:03.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4060000000000005e-06 2024-09-16T00:10:03.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841728+0000", 2024-09-16T00:10:03.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8868999999999998e-05 2024-09-16T00:10:03.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841741+0000", 2024-09-16T00:10:03.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3283e-05 2024-09-16T00:10:03.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841834+0000", 2024-09-16T00:10:03.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2312999999999995e-05 2024-09-16T00:10:03.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.850331+0000", 2024-09-16T00:10:03.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084972009999999994 2024-09-16T00:10:03.669 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.669 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.670 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.670 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:5307 3.1e e96/94 3:7b9ea4ae:::10000003595.00000000:head v 96'234, mlcod=95'232)", 2024-09-16T00:10:03.670 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.841762+0000", 2024-09-16T00:10:03.670 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.74701899299999996, 2024-09-16T00:10:03.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086486400000000008, 2024-09-16T00:10:03.670 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.670 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.670 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:03.670 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841762+0000", 2024-09-16T00:10:03.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841762+0000", 2024-09-16T00:10:03.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841763+0000", 2024-09-16T00:10:03.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7e-07 2024-09-16T00:10:03.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841767+0000", 2024-09-16T00:10:03.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2799999999999997e-06 2024-09-16T00:10:03.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841775+0000", 2024-09-16T00:10:03.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5349999999999999e-06 2024-09-16T00:10:03.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841780+0000", 2024-09-16T00:10:03.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0429999999999998e-06 2024-09-16T00:10:03.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841856+0000", 2024-09-16T00:10:03.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6532999999999994e-05 2024-09-16T00:10:03.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841874+0000", 2024-09-16T00:10:03.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8079000000000001e-05 2024-09-16T00:10:03.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:03.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.850370+0000", 2024-09-16T00:10:03.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084952169999999994 2024-09-16T00:10:03.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.850411+0000", 2024-09-16T00:10:03.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1582999999999998e-05 2024-09-16T00:10:03.674 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.674 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.674 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.675 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.19 epoch 96/94 rep_tid 2315 entries 96'16 (0'0) error 2:9c58fd0b:::100000036fd.00000000:head by mds.0.4:5310 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 95'13)", 2024-09-16T00:10:03.675 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.842654+0000", 2024-09-16T00:10:03.675 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.74612770100000003, 2024-09-16T00:10:03.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0076927239999999997, 2024-09-16T00:10:03.675 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.675 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.675 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:10:03.675 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.842654+0000", 2024-09-16T00:10:03.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.842654+0000", 2024-09-16T00:10:03.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.842655+0000", 2024-09-16T00:10:03.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0500000000000002e-07 2024-09-16T00:10:03.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.842663+0000", 2024-09-16T00:10:03.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1710000000000002e-06 2024-09-16T00:10:03.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.842681+0000", 2024-09-16T00:10:03.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.819e-05 2024-09-16T00:10:03.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.842694+0000", 2024-09-16T00:10:03.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3049e-05 2024-09-16T00:10:03.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.842762+0000", 2024-09-16T00:10:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8019e-05 2024-09-16T00:10:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.850346+0000", 2024-09-16T00:10:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0075844900000000002 2024-09-16T00:10:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.678 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5317 3.35 3:af59e6c6:::10000003766.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:03.679 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.844437+0000", 2024-09-16T00:10:03.679 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.74434484300000003, 2024-09-16T00:10:03.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064870470000000001, 2024-09-16T00:10:03.679 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.679 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.679 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:03.679 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.844437+0000", 2024-09-16T00:10:03.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.844437+0000", 2024-09-16T00:10:03.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.844438+0000", 2024-09-16T00:10:03.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.319e-06 2024-09-16T00:10:03.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.844445+0000", 2024-09-16T00:10:03.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9410000000000001e-06 2024-09-16T00:10:03.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.844459+0000", 2024-09-16T00:10:03.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4313e-05 2024-09-16T00:10:03.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.844468+0000", 2024-09-16T00:10:03.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3299999999999999e-06 2024-09-16T00:10:03.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.844551+0000", 2024-09-16T00:10:03.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3938999999999997e-05 2024-09-16T00:10:03.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.844665+0000", 2024-09-16T00:10:03.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113219 2024-09-16T00:10:03.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.850924+0000", 2024-09-16T00:10:03.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062589860000000002 2024-09-16T00:10:03.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.682 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.682 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.683 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5405 2.7 2:e5777499:::1000000397b.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:03.683 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.861530+0000", 2024-09-16T00:10:03.683 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.72725181400000005, 2024-09-16T00:10:03.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059901909999999997, 2024-09-16T00:10:03.683 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:03.683 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:03.683 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:03.683 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:03.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:03.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.861530+0000", 2024-09-16T00:10:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.861530+0000", 2024-09-16T00:10:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.861531+0000", 2024-09-16T00:10:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.02e-06 2024-09-16T00:10:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.861537+0000", 2024-09-16T00:10:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4400000000000002e-06 2024-09-16T00:10:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.861549+0000", 2024-09-16T00:10:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2368000000000001e-05 2024-09-16T00:10:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.861557+0000", 2024-09-16T00:10:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6210000000000004e-06 2024-09-16T00:10:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.861607+0000", 2024-09-16T00:10:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9855999999999998e-05 2024-09-16T00:10:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.861715+0000", 2024-09-16T00:10:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107704 2024-09-16T00:10:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.867520+0000", 2024-09-16T00:10:03.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058051819999999999 2024-09-16T00:10:03.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.687 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.687 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:03.687 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:03.687 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:03.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.660+0000 7fdf5ff18640 1 -- 172.21.15.139:0/2867366428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) -- 0x7fdf5815e690 con 0x7fdf58154c20 2024-09-16T00:10:03.689 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:03.688+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:03.689 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:10:03.789 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:03.789+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:03.805 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:03.806 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:03.806 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:03.806 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:03.818 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:10:03.847 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:10:03.903 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:03.903+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:03.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.966+0000 7f7f9914b640 1 Processor -- start 2024-09-16T00:10:03.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.966+0000 7f7f9914b640 1 -- start start 2024-09-16T00:10:03.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.967+0000 7f7f9914b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7f94154b10 0x7f7f94070800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:03.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.967+0000 7f7f9914b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7f9415dd20 0x7f7f9415e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:03.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.967+0000 7f7f9914b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f94154f50 con 0x7f7f94154730 2024-09-16T00:10:03.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.967+0000 7f7f9914b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7f9406ff70 con 0x7f7f9415dd20 2024-09-16T00:10:03.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.967+0000 7f7f9914b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7f94070210 con 0x7f7f94154b10 2024-09-16T00:10:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.967+0000 7f7f92d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f7f94154730 0x7f7f940c9920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:58186/0 (socket says 172.21.15.139:58186) 2024-09-16T00:10:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.967+0000 7f7f92d9d640 1 -- 172.21.15.139:0/2515625071 learned_addr learned my addr 172.21.15.139:0/2515625071 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.967+0000 7f7f9259c640 1 --2- 172.21.15.139:0/2515625071 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7f94154b10 0x7f7f94070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.967+0000 7f7f9359e640 1 --2- 172.21.15.139:0/2515625071 >> v2:172.21.15.164:3300/0 conn(0x7f7f9415dd20 0x7f7f9415e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.967+0000 7f7f91d9b640 1 -- 172.21.15.139:0/2515625071 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3164016986 0 0) 0x7f7f94154f50 con 0x7f7f94154730 2024-09-16T00:10:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.967+0000 7f7f91d9b640 1 -- 172.21.15.139:0/2515625071 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f7c003520 con 0x7f7f94154730 2024-09-16T00:10:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.967+0000 7f7f9259c640 1 -- 172.21.15.139:0/2515625071 >> v2:172.21.15.164:3300/0 conn(0x7f7f9415dd20 msgr2=0x7f7f9415e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.967+0000 7f7f9259c640 1 --2- 172.21.15.139:0/2515625071 >> v2:172.21.15.164:3300/0 conn(0x7f7f9415dd20 0x7f7f9415e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.967+0000 7f7f9259c640 1 -- 172.21.15.139:0/2515625071 >> v1:172.21.15.139:6789/0 conn(0x7f7f94154730 legacy=0x7f7f940c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:03.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.967+0000 7f7f9259c640 1 -- 172.21.15.139:0/2515625071 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f9406ddf0 con 0x7f7f94154b10 2024-09-16T00:10:03.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.968+0000 7f7f9259c640 1 --2- 172.21.15.139:0/2515625071 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7f94154b10 0x7f7f94070800 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7f880097a0 tx=0x7f7f8802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15ad60337c588146 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.968+0000 7f7f91d9b640 1 -- 172.21.15.139:0/2515625071 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88037070 con 0x7f7f94154b10 2024-09-16T00:10:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.968+0000 7f7f91d9b640 1 -- 172.21.15.139:0/2515625071 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f88032440 con 0x7f7f94154b10 2024-09-16T00:10:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.968+0000 7f7f91d9b640 1 -- 172.21.15.139:0/2515625071 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f880327f0 con 0x7f7f94154b10 2024-09-16T00:10:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.969+0000 7f7f9914b640 1 -- 172.21.15.139:0/2515625071 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7f94154b10 msgr2=0x7f7f94070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.969+0000 7f7f9914b640 1 --2- 172.21.15.139:0/2515625071 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7f94154b10 0x7f7f94070800 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7f880097a0 tx=0x7f7f8802a210 comp rx=0 tx=0).stop 2024-09-16T00:10:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.969+0000 7f7f9914b640 1 -- 172.21.15.139:0/2515625071 shutdown_connections 2024-09-16T00:10:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.969+0000 7f7f9914b640 1 --2- 172.21.15.139:0/2515625071 >> v2:172.21.15.164:3300/0 conn(0x7f7f9415dd20 0x7f7f9415e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.969+0000 7f7f9914b640 1 --2- 172.21.15.139:0/2515625071 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7f94154b10 0x7f7f94070800 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.969+0000 7f7f9914b640 1 -- 172.21.15.139:0/2515625071 >> 172.21.15.139:0/2515625071 conn(0x7f7f94071320 msgr2=0x7f7f94071700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.969+0000 7f7f9914b640 1 -- 172.21.15.139:0/2515625071 shutdown_connections 2024-09-16T00:10:03.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.969+0000 7f7f9914b640 1 -- 172.21.15.139:0/2515625071 wait complete. 2024-09-16T00:10:03.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.970+0000 7f7f9914b640 1 Processor -- start 2024-09-16T00:10:03.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.971+0000 7f7f9914b640 1 -- start start 2024-09-16T00:10:03.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.971+0000 7f7f9914b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f94154730 0x7f7f940af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:03.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.971+0000 7f7f9914b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f94154b10 0x7f7f9414c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:03.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.971+0000 7f7f9914b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7f9415dd20 0x7f7f9414c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:03.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.971+0000 7f7f9914b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9406d030 con 0x7f7f94154730 2024-09-16T00:10:03.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.971+0000 7f7f9914b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7f9406cd50 con 0x7f7f94154b10 2024-09-16T00:10:03.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.971+0000 7f7f9914b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7f9406d310 con 0x7f7f9415dd20 2024-09-16T00:10:03.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.971+0000 7f7f92d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f94154730 0x7f7f940af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:03.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.971+0000 7f7f9259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f94154b10 0x7f7f9414c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:03.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.971+0000 7f7f9359e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7f9415dd20 0x7f7f9414c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:03.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.971+0000 7f7f92d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f94154730 0x7f7f940af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55896/0 (socket says 172.21.15.139:55896) 2024-09-16T00:10:03.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.971+0000 7f7f92d9d640 1 -- 172.21.15.139:0/3830354265 learned_addr learned my addr 172.21.15.139:0/3830354265 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:03.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.972+0000 7f7f9259c640 1 -- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7f9415dd20 msgr2=0x7f7f9414c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:03.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.972+0000 7f7f9259c640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7f9415dd20 0x7f7f9414c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.972+0000 7f7f9259c640 1 -- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f94154730 msgr2=0x7f7f940af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:03.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.972+0000 7f7f9259c640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7f94154730 0x7f7f940af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:03.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.972+0000 7f7f9259c640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f9415d640 con 0x7f7f94154b10 2024-09-16T00:10:03.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.972+0000 7f7f9259c640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f94154b10 0x7f7f9414c0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7f88031000 tx=0x7f7f880046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:03.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.972+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88041030 con 0x7f7f94154b10 2024-09-16T00:10:03.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.972+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f8803d420 con 0x7f7f94154b10 2024-09-16T00:10:03.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.972+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88037040 con 0x7f7f94154b10 2024-09-16T00:10:03.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.972+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f941f0b90 con 0x7f7f94154b10 2024-09-16T00:10:03.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.972+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f940cd130 con 0x7f7f94154b10 2024-09-16T00:10:03.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.973+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f7f9406d5f0 con 0x7f7f94154b10 2024-09-16T00:10:03.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.973+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7f88044070 con 0x7f7f94154b10 2024-09-16T00:10:03.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.974+0000 7f7f7b7fe640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7f8006c2b0 0x7f7f8006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:03.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.974+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5973+0+0 (secure 0 0 0) 0x7f7f880ab9b0 con 0x7f7f94154b10 2024-09-16T00:10:03.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.974+0000 7f7f92d9d640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7f8006c2b0 0x7f7f8006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:03.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.974+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v97) ==== 54+0+16 (secure 0 0 0) 0x7f7f880b0220 con 0x7f7f94154b10 2024-09-16T00:10:03.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.974+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f7f940cd370 0x7f7f940cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:03.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.974+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7f940c5640 con 0x7f7f940cd370 2024-09-16T00:10:03.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.974+0000 7f7f9359e640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f7f940cd370 0x7f7f940cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:03.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.975+0000 7f7f92d9d640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7f8006c2b0 0x7f7f8006e770 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7f84005880 tx=0x7f7f84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:03.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.975+0000 7f7f9359e640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f7f940cd370 0x7f7f940cd930 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:03.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.976+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f940c5640 con 0x7f7f940cd370 2024-09-16T00:10:03.978 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:03.979 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:03.979 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:03.987 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:10:03.990 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:03.990+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:03.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.991+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7f94153900 con 0x7f7f940cd370 2024-09-16T00:10:03.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.993+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7f94153900 con 0x7f7f940cd370 2024-09-16T00:10:03.995 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.993+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f7f940afed0 0x7f7f941f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.993+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f7f9414fce0 con 0x7f7f940afed0 2024-09-16T00:10:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.994+0000 7f7f92d9d640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f7f940afed0 0x7f7f941f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.994+0000 7f7f92d9d640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f7f940afed0 0x7f7f941f35e0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:03.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:03.995+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f9414fce0 con 0x7f7f940afed0 2024-09-16T00:10:04.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.012+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7f940c81f0 con 0x7f7f940afed0 2024-09-16T00:10:04.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.014+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7f940c81f0 con 0x7f7f940afed0 2024-09-16T00:10:04.015 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:04.016 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:04.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.014+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f7f941f4030 0x7f7f94062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:04.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.014+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f7f941f4440 con 0x7f7f941f4030 2024-09-16T00:10:04.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.014+0000 7f7f9259c640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f7f941f4030 0x7f7f94062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:04.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.015+0000 7f7f9259c640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f7f941f4030 0x7f7f94062630 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:04.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.016+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f941f4440 con 0x7f7f941f4030 2024-09-16T00:10:04.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.031+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7f94062b70 con 0x7f7f941f4030 2024-09-16T00:10:04.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.032+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7f94062b70 con 0x7f7f941f4030 2024-09-16T00:10:04.033 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:04.034 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:04.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.033+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f7f94150cb0 0x7f7f941510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:04.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.033+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f7f94151610 con 0x7f7f94150cb0 2024-09-16T00:10:04.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.033+0000 7f7f9359e640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f7f94150cb0 0x7f7f941510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:04.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.033+0000 7f7f9359e640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f7f94150cb0 0x7f7f941510d0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:04.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.034+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f94151610 con 0x7f7f94150cb0 2024-09-16T00:10:04.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.049+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7f94151610 con 0x7f7f94150cb0 2024-09-16T00:10:04.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.050+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7f94151610 con 0x7f7f94150cb0 2024-09-16T00:10:04.051 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:04.052 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:04.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.051+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f7f9402cb50 0x7f7f9402cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:04.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.051+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f7f94151a70 con 0x7f7f9402cb50 2024-09-16T00:10:04.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.051+0000 7f7f92d9d640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f7f9402cb50 0x7f7f9402cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:04.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.051+0000 7f7f92d9d640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f7f9402cb50 0x7f7f9402cf90 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:04.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.052+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f94151a70 con 0x7f7f9402cb50 2024-09-16T00:10:04.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.068+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7f9402d4d0 con 0x7f7f9402cb50 2024-09-16T00:10:04.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.070+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7f9402d4d0 con 0x7f7f9402cb50 2024-09-16T00:10:04.071 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:04.072 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:04.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.071+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f7f94204960 0x7f7f94206d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:04.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.071+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f7f9402d9e0 con 0x7f7f94204960 2024-09-16T00:10:04.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.071+0000 7f7f9259c640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f7f94204960 0x7f7f94206d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:04.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.071+0000 7f7f9259c640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f7f94204960 0x7f7f94206d50 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:04.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.072+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f9402d9e0 con 0x7f7f94204960 2024-09-16T00:10:04.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.086+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7f9402d9e0 con 0x7f7f94204960 2024-09-16T00:10:04.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.088+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7f9402d9e0 con 0x7f7f94204960 2024-09-16T00:10:04.089 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:04.090 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:04.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.088+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f7f942077a0 0x7f7f94209b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:04.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.088+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f7f9420c960 con 0x7f7f942077a0 2024-09-16T00:10:04.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.088+0000 7f7f9359e640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f7f942077a0 0x7f7f94209b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:04.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.088+0000 7f7f9359e640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f7f942077a0 0x7f7f94209b90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:04.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.089+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f9420c960 con 0x7f7f942077a0 2024-09-16T00:10:04.092 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:04.091+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:04.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.103+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7f9420a0d0 con 0x7f7f942077a0 2024-09-16T00:10:04.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.105+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7f9420a0d0 con 0x7f7f942077a0 2024-09-16T00:10:04.106 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:04.107 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:04.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.105+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f7f9420a7f0 0x7f7f9420cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:04.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.105+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f7f9420fb40 con 0x7f7f9420a7f0 2024-09-16T00:10:04.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.106+0000 7f7f92d9d640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f7f9420a7f0 0x7f7f9420cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:04.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.106+0000 7f7f92d9d640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f7f9420a7f0 0x7f7f9420cbe0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:04.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.107+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f9420fb40 con 0x7f7f9420a7f0 2024-09-16T00:10:04.109 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:04.110 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:04.119 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:10:04.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.122+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7f9420fb40 con 0x7f7f9420a7f0 2024-09-16T00:10:04.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f7b7fe640 1 -- 172.21.15.139:0/3830354265 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7f9420fb40 con 0x7f7f9420a7f0 2024-09-16T00:10:04.125 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:10:04.126 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:10:04.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f7f940cd370 msgr2=0x7f7f940cd930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:04.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f7f940cd370 0x7f7f940cd930 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f7f940afed0 msgr2=0x7f7f941f35e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:04.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f7f940afed0 0x7f7f941f35e0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f7f941f4030 msgr2=0x7f7f94062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:04.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f7f941f4030 0x7f7f94062630 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f7f94150cb0 msgr2=0x7f7f941510d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:04.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f7f94150cb0 0x7f7f941510d0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f7f9402cb50 msgr2=0x7f7f9402cf90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:04.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f7f9402cb50 0x7f7f9402cf90 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f7f94204960 msgr2=0x7f7f94206d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:04.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f7f94204960 0x7f7f94206d50 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f7f942077a0 msgr2=0x7f7f94209b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:04.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f7f942077a0 0x7f7f94209b90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f7f9420a7f0 msgr2=0x7f7f9420cbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:04.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f7f9420a7f0 0x7f7f9420cbe0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9359e640 1 -- 172.21.15.139:0/3830354265 reap_dead start 2024-09-16T00:10:04.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7f8006c2b0 msgr2=0x7f7f8006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:04.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7f8006c2b0 0x7f7f8006e770 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7f84005880 tx=0x7f7f84009040 comp rx=0 tx=0).stop 2024-09-16T00:10:04.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9359e640 1 -- 172.21.15.139:0/3830354265 reap_dead start 2024-09-16T00:10:04.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f94154b10 msgr2=0x7f7f9414c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:04.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9359e640 1 -- 172.21.15.139:0/3830354265 reap_dead start 2024-09-16T00:10:04.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 --2- 172.21.15.139:0/3830354265 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7f94154b10 0x7f7f9414c0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7f88031000 tx=0x7f7f880046c0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9359e640 1 -- 172.21.15.139:0/3830354265 reap_dead start 2024-09-16T00:10:04.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9359e640 1 -- 172.21.15.139:0/3830354265 reap_dead start 2024-09-16T00:10:04.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 shutdown_connections 2024-09-16T00:10:04.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 >> 172.21.15.139:0/3830354265 conn(0x7f7f94071320 msgr2=0x7f7f94152dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:04.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.124+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 shutdown_connections 2024-09-16T00:10:04.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.125+0000 7f7f9914b640 1 -- 172.21.15.139:0/3830354265 wait complete. 2024-09-16T00:10:04.192 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:04.192+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:04.234 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:04.234 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:04.244 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:10:04.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.287+0000 7fdf4e7fc640 1 -- 172.21.15.139:0/2867366428 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v98) ==== 113+0+0 (secure 0 0 0) 0x7fdf5004b020 con 0x7fdf58154c20 2024-09-16T00:10:04.288 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T00:10:04.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.289+0000 7fdf5ff18640 1 -- 172.21.15.139:0/2867366428 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdf3c06c2e0 msgr2=0x7fdf3c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:04.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.289+0000 7fdf5ff18640 1 --2- 172.21.15.139:0/2867366428 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdf3c06c2e0 0x7fdf3c06e7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdf540016e0 tx=0x7fdf54009040 comp rx=0 tx=0).stop 2024-09-16T00:10:04.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.289+0000 7fdf5ff18640 1 -- 172.21.15.139:0/2867366428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf58154c20 msgr2=0x7fdf580c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:04.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.289+0000 7fdf5ff18640 1 --2- 172.21.15.139:0/2867366428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf58154c20 0x7fdf580c8400 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdf50034040 tx=0x7fdf50004a40 comp rx=0 tx=0).stop 2024-09-16T00:10:04.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.289+0000 7fdf5ff18640 1 -- 172.21.15.139:0/2867366428 shutdown_connections 2024-09-16T00:10:04.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.289+0000 7fdf5ff18640 1 --2- 172.21.15.139:0/2867366428 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fdf3c06c2e0 0x7fdf3c06e7a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.289+0000 7fdf5ff18640 1 --2- 172.21.15.139:0/2867366428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf58154c20 0x7fdf580c8400 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.289+0000 7fdf5ff18640 1 --2- 172.21.15.139:0/2867366428 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdf580ccc90 0x7fdf5815d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.289+0000 7fdf5ff18640 1 --2- 172.21.15.139:0/2867366428 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fdf58068060 0x7fdf580c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.289+0000 7fdf5ff18640 1 -- 172.21.15.139:0/2867366428 >> 172.21.15.139:0/2867366428 conn(0x7fdf5806ca40 msgr2=0x7fdf5814f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:04.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.290+0000 7fdf5ff18640 1 -- 172.21.15.139:0/2867366428 shutdown_connections 2024-09-16T00:10:04.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.290+0000 7fdf5ff18640 1 -- 172.21.15.139:0/2867366428 wait complete. 2024-09-16T00:10:04.293 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:04.293+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:04.304 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T00:10:04.358 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:04.359 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:04.369 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:10:04.394 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:04.394+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:04.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.454+0000 7f5f69131640 1 Processor -- start 2024-09-16T00:10:04.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.454+0000 7f5f69131640 1 -- start start 2024-09-16T00:10:04.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.456+0000 7f5f69131640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5f640c6940 0x7f5f6415c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:04.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.456+0000 7f5f69131640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5f640c6d20 0x7f5f641524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:04.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.456+0000 7f5f69131640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f6415cb40 con 0x7f5f6406d9b0 2024-09-16T00:10:04.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.456+0000 7f5f69131640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5f6406c3c0 con 0x7f5f640c6940 2024-09-16T00:10:04.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.456+0000 7f5f69131640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5f640c7ee0 con 0x7f5f640c6d20 2024-09-16T00:10:04.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.456+0000 7f5f62d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5f640c6940 0x7f5f6415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:04.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.456+0000 7f5f6359e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5f6406d9b0 0x7f5f6406f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:58200/0 (socket says 172.21.15.139:58200) 2024-09-16T00:10:04.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.456+0000 7f5f6359e640 1 -- 172.21.15.139:0/2068902999 learned_addr learned my addr 172.21.15.139:0/2068902999 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:04.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.456+0000 7f5f6259c640 1 --2- 172.21.15.139:0/2068902999 >> v2:172.21.15.164:3300/0 conn(0x7f5f640c6d20 0x7f5f641524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:04.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.456+0000 7f5f62d9d640 1 -- 172.21.15.139:0/2068902999 >> v2:172.21.15.164:3300/0 conn(0x7f5f640c6d20 msgr2=0x7f5f641524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:04.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.456+0000 7f5f62d9d640 1 --2- 172.21.15.139:0/2068902999 >> v2:172.21.15.164:3300/0 conn(0x7f5f640c6d20 0x7f5f641524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.456+0000 7f5f62d9d640 1 -- 172.21.15.139:0/2068902999 >> v1:172.21.15.139:6789/0 conn(0x7f5f6406d9b0 legacy=0x7f5f6406f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:04.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.456+0000 7f5f61d9b640 1 -- 172.21.15.139:0/2068902999 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2210292713 0 0) 0x7f5f5c001490 con 0x7f5f6406d9b0 2024-09-16T00:10:04.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.456+0000 7f5f62d9d640 1 -- 172.21.15.139:0/2068902999 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f6406bb20 con 0x7f5f640c6940 2024-09-16T00:10:04.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.457+0000 7f5f62d9d640 1 --2- 172.21.15.139:0/2068902999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f640c6940 0x7f5f6415c600 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5f6406cff0 tx=0x7f5f5402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591515640a0bbce1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:04.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.457+0000 7f5f61d9b640 1 -- 172.21.15.139:0/2068902999 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54037070 con 0x7f5f640c6940 2024-09-16T00:10:04.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.457+0000 7f5f61d9b640 1 -- 172.21.15.139:0/2068902999 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f54032440 con 0x7f5f640c6940 2024-09-16T00:10:04.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.457+0000 7f5f61d9b640 1 -- 172.21.15.139:0/2068902999 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54032810 con 0x7f5f640c6940 2024-09-16T00:10:04.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.458+0000 7f5f69131640 1 -- 172.21.15.139:0/2068902999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f640c6940 msgr2=0x7f5f6415c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:04.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.458+0000 7f5f69131640 1 --2- 172.21.15.139:0/2068902999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f640c6940 0x7f5f6415c600 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5f6406cff0 tx=0x7f5f5402a250 comp rx=0 tx=0).stop 2024-09-16T00:10:04.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.458+0000 7f5f69131640 1 -- 172.21.15.139:0/2068902999 shutdown_connections 2024-09-16T00:10:04.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.458+0000 7f5f69131640 1 --2- 172.21.15.139:0/2068902999 >> v2:172.21.15.164:3300/0 conn(0x7f5f640c6d20 0x7f5f641524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.458+0000 7f5f69131640 1 --2- 172.21.15.139:0/2068902999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f640c6940 0x7f5f6415c600 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.458+0000 7f5f69131640 1 -- 172.21.15.139:0/2068902999 >> 172.21.15.139:0/2068902999 conn(0x7f5f64070460 msgr2=0x7f5f64070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:04.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.458+0000 7f5f69131640 1 -- 172.21.15.139:0/2068902999 shutdown_connections 2024-09-16T00:10:04.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.458+0000 7f5f69131640 1 -- 172.21.15.139:0/2068902999 wait complete. 2024-09-16T00:10:04.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.460+0000 7f5f69131640 1 Processor -- start 2024-09-16T00:10:04.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.460+0000 7f5f69131640 1 -- start start 2024-09-16T00:10:04.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.460+0000 7f5f69131640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f6406d9b0 0x7f5f641480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:04.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.460+0000 7f5f69131640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f640c6940 0x7f5f64142850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:04.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.460+0000 7f5f69131640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f640c6d20 0x7f5f64142d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:04.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.460+0000 7f5f69131640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6406efc0 con 0x7f5f6406d9b0 2024-09-16T00:10:04.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.460+0000 7f5f69131640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5f6406c0e0 con 0x7f5f640c6d20 2024-09-16T00:10:04.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.460+0000 7f5f69131640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5f6406fbf0 con 0x7f5f640c6940 2024-09-16T00:10:04.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f62d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f6406d9b0 0x7f5f641480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:04.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f6359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f640c6d20 0x7f5f64142d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:04.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f62d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f6406d9b0 0x7f5f641480d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55908/0 (socket says 172.21.15.139:55908) 2024-09-16T00:10:04.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f6359e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f640c6d20 0x7f5f64142d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:49874/0 (socket says 172.21.15.139:49874) 2024-09-16T00:10:04.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f62d9d640 1 -- 172.21.15.139:0/1371071383 learned_addr learned my addr 172.21.15.139:0/1371071383 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:04.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f6259c640 1 --2- 172.21.15.139:0/1371071383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f640c6940 0x7f5f64142850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:04.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f6359e640 1 -- 172.21.15.139:0/1371071383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f640c6940 msgr2=0x7f5f64142850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:04.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f6359e640 1 --2- 172.21.15.139:0/1371071383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f640c6940 0x7f5f64142850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f6359e640 1 -- 172.21.15.139:0/1371071383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f6406d9b0 msgr2=0x7f5f641480d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:04.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f6359e640 1 --2- 172.21.15.139:0/1371071383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f6406d9b0 0x7f5f641480d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:04.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f6359e640 1 -- 172.21.15.139:0/1371071383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f640c61f0 con 0x7f5f640c6d20 2024-09-16T00:10:04.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f6359e640 1 --2- 172.21.15.139:0/1371071383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f640c6d20 0x7f5f64142d90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00c6e0 tx=0x7f5f5c00cba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:04.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f4b7fe640 1 -- 172.21.15.139:0/1371071383 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c018070 con 0x7f5f640c6d20 2024-09-16T00:10:04.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f4b7fe640 1 -- 172.21.15.139:0/1371071383 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f5c004d60 con 0x7f5f640c6d20 2024-09-16T00:10:04.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f69131640 1 -- 172.21.15.139:0/1371071383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f64143890 con 0x7f5f640c6d20 2024-09-16T00:10:04.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f4b7fe640 1 -- 172.21.15.139:0/1371071383 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c00a880 con 0x7f5f640c6d20 2024-09-16T00:10:04.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.461+0000 7f5f69131640 1 -- 172.21.15.139:0/1371071383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f640ca690 con 0x7f5f640c6d20 2024-09-16T00:10:04.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.463+0000 7f5f69131640 1 -- 172.21.15.139:0/1371071383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f640c79c0 con 0x7f5f640c6d20 2024-09-16T00:10:04.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.463+0000 7f5f4b7fe640 1 -- 172.21.15.139:0/1371071383 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5f5c01b070 con 0x7f5f640c6d20 2024-09-16T00:10:04.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.463+0000 7f5f4b7fe640 1 --2- 172.21.15.139:0/1371071383 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5f5006c2b0 0x7f5f5006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:04.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.464+0000 7f5f4b7fe640 1 -- 172.21.15.139:0/1371071383 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5973+0+0 (secure 0 0 0) 0x7f5f5c08a970 con 0x7f5f640c6d20 2024-09-16T00:10:04.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.464+0000 7f5f62d9d640 1 --2- 172.21.15.139:0/1371071383 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5f5006c2b0 0x7f5f5006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:04.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.464+0000 7f5f62d9d640 1 --2- 172.21.15.139:0/1371071383 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5f5006c2b0 0x7f5f5006e770 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009560 tx=0x7f5f5402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:04.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.470+0000 7f5f4b7fe640 1 -- 172.21.15.139:0/1371071383 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5f6415cb40 con 0x7f5f640c6d20 2024-09-16T00:10:04.495 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:04.495+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:04.526 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:04.526 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:04.536 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:10:04.596 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:04.596+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:04.649 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:04.649 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:04.661 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:10:04.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:04.676+0000 7f5f69131640 1 -- 172.21.15.139:0/1371071383 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) -- 0x7f5f640cbd50 con 0x7f5f640c6d20 2024-09-16T00:10:04.696 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:04.696+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:04.771 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:04.772 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:04.781 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:10:04.822 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:04.821+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:04.900 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:04.898+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:04.954 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:04.954 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:04.964 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:10:05.008 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:05.007+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:05.072 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:05.072 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:05.081 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:10:05.122 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:05.122+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:05.202 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:05.201+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:05.228 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:05.228 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:05.237 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:10:05.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.292+0000 7f5f4b7fe640 1 -- 172.21.15.139:0/1371071383 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v99) ==== 113+0+0 (secure 0 0 0) 0x7f5f5c0589f0 con 0x7f5f640c6d20 2024-09-16T00:10:05.306 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T00:10:05.306 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T00:10:05.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.295+0000 7f5f69131640 1 -- 172.21.15.139:0/1371071383 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5f5006c2b0 msgr2=0x7f5f5006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:05.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.295+0000 7f5f69131640 1 --2- 172.21.15.139:0/1371071383 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5f5006c2b0 0x7f5f5006e770 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009560 tx=0x7f5f5402e000 comp rx=0 tx=0).stop 2024-09-16T00:10:05.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.295+0000 7f5f69131640 1 -- 172.21.15.139:0/1371071383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f640c6d20 msgr2=0x7f5f64142d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:05.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.295+0000 7f5f69131640 1 --2- 172.21.15.139:0/1371071383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f640c6d20 0x7f5f64142d90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00c6e0 tx=0x7f5f5c00cba0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.295+0000 7f5f69131640 1 -- 172.21.15.139:0/1371071383 shutdown_connections 2024-09-16T00:10:05.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.295+0000 7f5f69131640 1 --2- 172.21.15.139:0/1371071383 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5f5006c2b0 0x7f5f5006e770 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.295+0000 7f5f69131640 1 --2- 172.21.15.139:0/1371071383 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5f640c6d20 0x7f5f64142d90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.295+0000 7f5f69131640 1 --2- 172.21.15.139:0/1371071383 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5f640c6940 0x7f5f64142850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.295+0000 7f5f69131640 1 --2- 172.21.15.139:0/1371071383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f6406d9b0 0x7f5f641480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.295+0000 7f5f69131640 1 -- 172.21.15.139:0/1371071383 >> 172.21.15.139:0/1371071383 conn(0x7f5f64070460 msgr2=0x7f5f640cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:05.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.295+0000 7f5f69131640 1 -- 172.21.15.139:0/1371071383 shutdown_connections 2024-09-16T00:10:05.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.295+0000 7f5f69131640 1 -- 172.21.15.139:0/1371071383 wait complete. 2024-09-16T00:10:05.310 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T00:10:05.310 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:05.307+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:05.406 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:05.406 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:05.408 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:05.408+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:05.419 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:10:05.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.456+0000 7f8c7e152640 1 Processor -- start 2024-09-16T00:10:05.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.456+0000 7f8c7e152640 1 -- start start 2024-09-16T00:10:05.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.457+0000 7f8c7e152640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8c78154c20 0x7f8c7815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:05.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.457+0000 7f8c7e152640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8c78068060 0x7f8c780c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:05.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.457+0000 7f8c7e152640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c780cc4b0 con 0x7f8c780ccc90 2024-09-16T00:10:05.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.457+0000 7f8c7e152640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8c7815b7c0 con 0x7f8c78068060 2024-09-16T00:10:05.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.457+0000 7f8c7e152640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f8c780cc260 con 0x7f8c78154c20 2024-09-16T00:10:05.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.457+0000 7f8c76ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8c78068060 0x7f8c780c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:05.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.457+0000 7f8c76ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8c78068060 0x7f8c780c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55918/0 (socket says 172.21.15.139:55918) 2024-09-16T00:10:05.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.457+0000 7f8c76ffd640 1 -- 172.21.15.139:0/3177944167 learned_addr learned my addr 172.21.15.139:0/3177944167 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:05.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.457+0000 7f8c777fe640 1 --2- 172.21.15.139:0/3177944167 >> v2:172.21.15.164:3300/0 conn(0x7f8c78154c20 0x7f8c7815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:05.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.458+0000 7f8c76ffd640 1 -- 172.21.15.139:0/3177944167 >> v2:172.21.15.164:3300/0 conn(0x7f8c78154c20 msgr2=0x7f8c7815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:05.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.458+0000 7f8c76ffd640 1 --2- 172.21.15.139:0/3177944167 >> v2:172.21.15.164:3300/0 conn(0x7f8c78154c20 0x7f8c7815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.458+0000 7f8c76ffd640 1 -- 172.21.15.139:0/3177944167 >> v1:172.21.15.164:6789/0 conn(0x7f8c780ccc90 legacy=0x7f8c780cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:05.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.458+0000 7f8c76ffd640 1 -- 172.21.15.139:0/3177944167 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c780cc790 con 0x7f8c78068060 2024-09-16T00:10:05.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.458+0000 7f8c76ffd640 1 --2- 172.21.15.139:0/3177944167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c78068060 0x7f8c780c5120 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8c780d1290 tx=0x7f8c6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c1145019361a32 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:05.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.459+0000 7f8c767fc640 1 -- 172.21.15.139:0/3177944167 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c004030 con 0x7f8c78068060 2024-09-16T00:10:05.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.459+0000 7f8c767fc640 1 -- 172.21.15.139:0/3177944167 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6c0041d0 con 0x7f8c78068060 2024-09-16T00:10:05.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.459+0000 7f8c767fc640 1 -- 172.21.15.139:0/3177944167 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c003610 con 0x7f8c78068060 2024-09-16T00:10:05.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.459+0000 7f8c7e152640 1 -- 172.21.15.139:0/3177944167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c78068060 msgr2=0x7f8c780c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:05.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.459+0000 7f8c7e152640 1 --2- 172.21.15.139:0/3177944167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c78068060 0x7f8c780c5120 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8c780d1290 tx=0x7f8c6c02a270 comp rx=0 tx=0).stop 2024-09-16T00:10:05.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.459+0000 7f8c7e152640 1 -- 172.21.15.139:0/3177944167 shutdown_connections 2024-09-16T00:10:05.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.459+0000 7f8c7e152640 1 --2- 172.21.15.139:0/3177944167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c78068060 0x7f8c780c5120 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.459+0000 7f8c7e152640 1 --2- 172.21.15.139:0/3177944167 >> v2:172.21.15.164:3300/0 conn(0x7f8c78154c20 0x7f8c7815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.459+0000 7f8c7e152640 1 -- 172.21.15.139:0/3177944167 >> 172.21.15.139:0/3177944167 conn(0x7f8c7806ca40 msgr2=0x7f8c78067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:05.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.459+0000 7f8c7e152640 1 -- 172.21.15.139:0/3177944167 shutdown_connections 2024-09-16T00:10:05.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.459+0000 7f8c7e152640 1 -- 172.21.15.139:0/3177944167 wait complete. 2024-09-16T00:10:05.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.461+0000 7f8c7e152640 1 Processor -- start 2024-09-16T00:10:05.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.461+0000 7f8c7e152640 1 -- start start 2024-09-16T00:10:05.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c7e152640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c78068060 0x7f8c7815bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:05.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c7e152640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c780ccc90 0x7f8c781f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:05.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c7e152640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8c78154c20 0x7f8c781f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:05.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c7e152640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7815e3b0 con 0x7f8c780ccc90 2024-09-16T00:10:05.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c7e152640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7815e0d0 con 0x7f8c78068060 2024-09-16T00:10:05.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c7e152640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8c7815e690 con 0x7f8c78154c20 2024-09-16T00:10:05.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c76ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c780ccc90 0x7f8c781f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:05.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c76ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c780ccc90 0x7f8c781f2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55922/0 (socket says 172.21.15.139:55922) 2024-09-16T00:10:05.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c76ffd640 1 -- 172.21.15.139:0/4058275199 learned_addr learned my addr 172.21.15.139:0/4058275199 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:05.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c777fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c78068060 0x7f8c7815bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:05.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c77fff640 1 --2- 172.21.15.139:0/4058275199 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8c78154c20 0x7f8c781f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:05.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c76ffd640 1 -- 172.21.15.139:0/4058275199 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8c78154c20 msgr2=0x7f8c781f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:05.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c76ffd640 1 --2- 172.21.15.139:0/4058275199 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8c78154c20 0x7f8c781f31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c76ffd640 1 -- 172.21.15.139:0/4058275199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c78068060 msgr2=0x7f8c7815bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:05.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c76ffd640 1 --2- 172.21.15.139:0/4058275199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c78068060 0x7f8c7815bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c76ffd640 1 -- 172.21.15.139:0/4058275199 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7815d130 con 0x7f8c780ccc90 2024-09-16T00:10:05.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.462+0000 7f8c777fe640 1 --2- 172.21.15.139:0/4058275199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c78068060 0x7f8c7815bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:05.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.463+0000 7f8c76ffd640 1 --2- 172.21.15.139:0/4058275199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c780ccc90 0x7f8c781f2c80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c000c00 tx=0x7f8c6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:05.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.463+0000 7f8c5ffff640 1 -- 172.21.15.139:0/4058275199 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c041020 con 0x7f8c780ccc90 2024-09-16T00:10:05.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.463+0000 7f8c5ffff640 1 -- 172.21.15.139:0/4058275199 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6c003c50 con 0x7f8c780ccc90 2024-09-16T00:10:05.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.463+0000 7f8c7e152640 1 -- 172.21.15.139:0/4058275199 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c781f1be0 con 0x7f8c780ccc90 2024-09-16T00:10:05.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.463+0000 7f8c5ffff640 1 -- 172.21.15.139:0/4058275199 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c004030 con 0x7f8c780ccc90 2024-09-16T00:10:05.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.463+0000 7f8c7e152640 1 -- 172.21.15.139:0/4058275199 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c781f21a0 con 0x7f8c780ccc90 2024-09-16T00:10:05.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.464+0000 7f8c7e152640 1 -- 172.21.15.139:0/4058275199 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c780686b0 con 0x7f8c780ccc90 2024-09-16T00:10:05.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.464+0000 7f8c5ffff640 1 -- 172.21.15.139:0/4058275199 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8c6c044020 con 0x7f8c780ccc90 2024-09-16T00:10:05.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.465+0000 7f8c5ffff640 1 --2- 172.21.15.139:0/4058275199 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8c6406c2e0 0x7f8c6406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:05.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.465+0000 7f8c5ffff640 1 -- 172.21.15.139:0/4058275199 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5973+0+0 (secure 0 0 0) 0x7f8c6c0abf20 con 0x7f8c780ccc90 2024-09-16T00:10:05.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.465+0000 7f8c777fe640 1 --2- 172.21.15.139:0/4058275199 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8c6406c2e0 0x7f8c6406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:05.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.465+0000 7f8c777fe640 1 --2- 172.21.15.139:0/4058275199 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8c6406c2e0 0x7f8c6406e7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009f30 tx=0x7f8c68009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:05.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.469+0000 7f8c5ffff640 1 -- 172.21.15.139:0/4058275199 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8c6c04b030 con 0x7f8c780ccc90 2024-09-16T00:10:05.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.475+0000 7f9c82f9f640 1 Processor -- start 2024-09-16T00:10:05.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.475+0000 7f9c82f9f640 1 -- start start 2024-09-16T00:10:05.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.476+0000 7f9c82f9f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9c7c154c20 0x7f9c7c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.476+0000 7f9c82f9f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c7c068060 0x7f9c7c0c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.476+0000 7f9c82f9f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c7c0cc4b0 con 0x7f9c7c0ccc90 2024-09-16T00:10:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.476+0000 7f9c82f9f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9c7c15b7c0 con 0x7f9c7c154c20 2024-09-16T00:10:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.476+0000 7f9c82f9f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9c7c0cc260 con 0x7f9c7c068060 2024-09-16T00:10:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.476+0000 7f9c80d3b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9c7c154c20 0x7f9c7c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.476+0000 7f9c80d3b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9c7c154c20 0x7f9c7c15b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55930/0 (socket says 172.21.15.139:55930) 2024-09-16T00:10:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.476+0000 7f9c80d3b640 1 -- 172.21.15.139:0/445053324 learned_addr learned my addr 172.21.15.139:0/445053324 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.476+0000 7f9c7bfff640 1 --2- 172.21.15.139:0/445053324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c7c068060 0x7f9c7c0c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.477+0000 7f9c80d3b640 1 -- 172.21.15.139:0/445053324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c7c068060 msgr2=0x7f9c7c0c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.477+0000 7f9c80d3b640 1 --2- 172.21.15.139:0/445053324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c7c068060 0x7f9c7c0c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.477+0000 7f9c80d3b640 1 -- 172.21.15.139:0/445053324 >> v1:172.21.15.164:6789/0 conn(0x7f9c7c0ccc90 legacy=0x7f9c7c0cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:05.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.477+0000 7f9c80d3b640 1 -- 172.21.15.139:0/445053324 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7c0cc790 con 0x7f9c7c154c20 2024-09-16T00:10:05.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.477+0000 7f9c80d3b640 1 --2- 172.21.15.139:0/445053324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7c154c20 0x7f9c7c15b280 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009870 tx=0x7f9c7002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443ad7a0d55ac27a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:05.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.478+0000 7f9c7b7fe640 1 -- 172.21.15.139:0/445053324 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70037070 con 0x7f9c7c154c20 2024-09-16T00:10:05.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.478+0000 7f9c7b7fe640 1 -- 172.21.15.139:0/445053324 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7002ae50 con 0x7f9c7c154c20 2024-09-16T00:10:05.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.478+0000 7f9c7b7fe640 1 -- 172.21.15.139:0/445053324 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70032860 con 0x7f9c7c154c20 2024-09-16T00:10:05.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.478+0000 7f9c82f9f640 1 -- 172.21.15.139:0/445053324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7c154c20 msgr2=0x7f9c7c15b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:05.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.478+0000 7f9c82f9f640 1 --2- 172.21.15.139:0/445053324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7c154c20 0x7f9c7c15b280 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009870 tx=0x7f9c7002a300 comp rx=0 tx=0).stop 2024-09-16T00:10:05.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.478+0000 7f9c82f9f640 1 -- 172.21.15.139:0/445053324 shutdown_connections 2024-09-16T00:10:05.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.478+0000 7f9c82f9f640 1 --2- 172.21.15.139:0/445053324 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c7c068060 0x7f9c7c0c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.478+0000 7f9c82f9f640 1 --2- 172.21.15.139:0/445053324 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7c154c20 0x7f9c7c15b280 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.478+0000 7f9c82f9f640 1 -- 172.21.15.139:0/445053324 >> 172.21.15.139:0/445053324 conn(0x7f9c7c06ca40 msgr2=0x7f9c7c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:05.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.478+0000 7f9c82f9f640 1 -- 172.21.15.139:0/445053324 shutdown_connections 2024-09-16T00:10:05.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.478+0000 7f9c82f9f640 1 -- 172.21.15.139:0/445053324 wait complete. 2024-09-16T00:10:05.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.480+0000 7f9c82f9f640 1 Processor -- start 2024-09-16T00:10:05.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.480+0000 7f9c82f9f640 1 -- start start 2024-09-16T00:10:05.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c82f9f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7c068060 0x7f9c7c14e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:05.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c82f9f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c7c0ccc90 0x7f9c7c0b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:05.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c82f9f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c7c154c20 0x7f9c7c145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:05.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c82f9f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c15e3b0 con 0x7f9c7c068060 2024-09-16T00:10:05.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c82f9f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7c15e0d0 con 0x7f9c7c0ccc90 2024-09-16T00:10:05.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c82f9f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9c7c15e690 con 0x7f9c7c154c20 2024-09-16T00:10:05.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c80d3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7c068060 0x7f9c7c14e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:05.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c80d3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7c068060 0x7f9c7c14e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55942/0 (socket says 172.21.15.139:55942) 2024-09-16T00:10:05.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c80d3b640 1 -- 172.21.15.139:0/3691389732 learned_addr learned my addr 172.21.15.139:0/3691389732 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:05.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c7bfff640 1 --2- 172.21.15.139:0/3691389732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c7c0ccc90 0x7f9c7c0b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:05.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c8153c640 1 --2- 172.21.15.139:0/3691389732 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c7c154c20 0x7f9c7c145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:05.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c7bfff640 1 -- 172.21.15.139:0/3691389732 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c7c154c20 msgr2=0x7f9c7c145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:05.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c7bfff640 1 --2- 172.21.15.139:0/3691389732 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c7c154c20 0x7f9c7c145ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c7bfff640 1 -- 172.21.15.139:0/3691389732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7c068060 msgr2=0x7f9c7c14e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:05.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c7bfff640 1 --2- 172.21.15.139:0/3691389732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7c068060 0x7f9c7c14e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:05.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c7bfff640 1 -- 172.21.15.139:0/3691389732 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7c15ba60 con 0x7f9c7c0ccc90 2024-09-16T00:10:05.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c80d3b640 1 --2- 172.21.15.139:0/3691389732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7c068060 0x7f9c7c14e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:05.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c7bfff640 1 --2- 172.21.15.139:0/3691389732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c7c0ccc90 0x7f9c7c0b5ac0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800e840 tx=0x7f9c6800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:05.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.481+0000 7f9c797fa640 1 -- 172.21.15.139:0/3691389732 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c68017070 con 0x7f9c7c0ccc90 2024-09-16T00:10:05.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.482+0000 7f9c797fa640 1 -- 172.21.15.139:0/3691389732 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c6800c940 con 0x7f9c7c0ccc90 2024-09-16T00:10:05.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.482+0000 7f9c82f9f640 1 -- 172.21.15.139:0/3691389732 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7c15d220 con 0x7f9c7c0ccc90 2024-09-16T00:10:05.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.482+0000 7f9c797fa640 1 -- 172.21.15.139:0/3691389732 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6800ccf0 con 0x7f9c7c0ccc90 2024-09-16T00:10:05.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.482+0000 7f9c82f9f640 1 -- 172.21.15.139:0/3691389732 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c7c1f1ea0 con 0x7f9c7c0ccc90 2024-09-16T00:10:05.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.483+0000 7f9c82f9f640 1 -- 172.21.15.139:0/3691389732 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c7c0686b0 con 0x7f9c7c0ccc90 2024-09-16T00:10:05.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.483+0000 7f9c797fa640 1 -- 172.21.15.139:0/3691389732 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9c6801a070 con 0x7f9c7c0ccc90 2024-09-16T00:10:05.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.483+0000 7f9c797fa640 1 --2- 172.21.15.139:0/3691389732 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9c6006c2b0 0x7f9c6006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:05.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.484+0000 7f9c797fa640 1 -- 172.21.15.139:0/3691389732 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5973+0+0 (secure 0 0 0) 0x7f9c6808b410 con 0x7f9c7c0ccc90 2024-09-16T00:10:05.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.484+0000 7f9c80d3b640 1 --2- 172.21.15.139:0/3691389732 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9c6006c2b0 0x7f9c6006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:05.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.484+0000 7f9c80d3b640 1 --2- 172.21.15.139:0/3691389732 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9c6006c2b0 0x7f9c6006e770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9c700090b0 tx=0x7f9c7002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:05.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.488+0000 7f9c797fa640 1 -- 172.21.15.139:0/3691389732 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9c680594b0 con 0x7f9c7c0ccc90 2024-09-16T00:10:05.509 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:05.508+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:05.542 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:05.542 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:05.552 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:10:05.636 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:05.636+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:05.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.671+0000 7f8c7e152640 1 -- 172.21.15.139:0/4058275199 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f8c780cca70 con 0x7f8c780ccc90 2024-09-16T00:10:05.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:05.684+0000 7f9c82f9f640 1 -- 172.21.15.139:0/3691389732 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) -- 0x7f9c7c15ced0 con 0x7f9c7c0ccc90 2024-09-16T00:10:05.706 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:05.706 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:05.710 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:05.710+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:05.715 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:10:05.811 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:05.811+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:05.899 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:05.899 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:05.912 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:10:05.912 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:05.911+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:05.925 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T00:10:05.925 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T00:10:06.012 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:06.012+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:06.060 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:06.060 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:06.069 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:10:06.113 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:06.113+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:06.138 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:10:06.178 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:06.178 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:06.188 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:10:06.232 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:06.232+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:06.238 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.238+0000 7f0e21847640 1 Processor -- start 2024-09-16T00:10:06.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.238+0000 7f0e21847640 1 -- start start 2024-09-16T00:10:06.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.238+0000 7f0e21847640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0e1c0d0870 0x7f0e1c0b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.238+0000 7f0e21847640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0e1c0b3430 0x7f0e1c0b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.238+0000 7f0e21847640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1c15adc0 con 0x7f0e1c0d0490 2024-09-16T00:10:06.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.238+0000 7f0e21847640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0e1c15b940 con 0x7f0e1c0b3430 2024-09-16T00:10:06.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.238+0000 7f0e21847640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f0e1c0b3bb0 con 0x7f0e1c0d0870 2024-09-16T00:10:06.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.238+0000 7f0e1b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0e1c0b3430 0x7f0e1c0b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.239+0000 7f0e1affd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f0e1c0d0490 0x7f0e1c0b2c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:58216/0 (socket says 172.21.15.139:58216) 2024-09-16T00:10:06.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.239+0000 7f0e1b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0e1c0b3430 0x7f0e1c0b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55950/0 (socket says 172.21.15.139:55950) 2024-09-16T00:10:06.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.239+0000 7f0e1affd640 1 -- 172.21.15.139:0/3144397803 learned_addr learned my addr 172.21.15.139:0/3144397803 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:06.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.239+0000 7f0e1a7fc640 1 --2- 172.21.15.139:0/3144397803 >> v2:172.21.15.164:3300/0 conn(0x7f0e1c0d0870 0x7f0e1c0b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.239+0000 7f0e1b7fe640 1 -- 172.21.15.139:0/3144397803 >> v2:172.21.15.164:3300/0 conn(0x7f0e1c0d0870 msgr2=0x7f0e1c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:06.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.239+0000 7f0e1b7fe640 1 --2- 172.21.15.139:0/3144397803 >> v2:172.21.15.164:3300/0 conn(0x7f0e1c0d0870 0x7f0e1c0b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.239+0000 7f0e1b7fe640 1 -- 172.21.15.139:0/3144397803 >> v1:172.21.15.139:6789/0 conn(0x7f0e1c0d0490 legacy=0x7f0e1c0b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.239+0000 7f0e19ffb640 1 -- 172.21.15.139:0/3144397803 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3969877717 0 0) 0x7f0e1c15adc0 con 0x7f0e1c0d0490 2024-09-16T00:10:06.241 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.239+0000 7f0e1b7fe640 1 -- 172.21.15.139:0/3144397803 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c15b0a0 con 0x7f0e1c0b3430 2024-09-16T00:10:06.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.240+0000 7f0e1b7fe640 1 --2- 172.21.15.139:0/3144397803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e1c0b3430 0x7f0e1c0b4580 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0b5cc0 tx=0x7f0e1402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d3610dd9c5449f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:06.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.240+0000 7f0e19ffb640 1 -- 172.21.15.139:0/3144397803 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14007740 con 0x7f0e1c0b3430 2024-09-16T00:10:06.242 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.240+0000 7f0e19ffb640 1 -- 172.21.15.139:0/3144397803 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e140078e0 con 0x7f0e1c0b3430 2024-09-16T00:10:06.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.240+0000 7f0e19ffb640 1 -- 172.21.15.139:0/3144397803 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14007cb0 con 0x7f0e1c0b3430 2024-09-16T00:10:06.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.240+0000 7f0e21847640 1 -- 172.21.15.139:0/3144397803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e1c0b3430 msgr2=0x7f0e1c0b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.240+0000 7f0e21847640 1 --2- 172.21.15.139:0/3144397803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e1c0b3430 0x7f0e1c0b4580 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0b5cc0 tx=0x7f0e1402a710 comp rx=0 tx=0).stop 2024-09-16T00:10:06.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.240+0000 7f0e21847640 1 -- 172.21.15.139:0/3144397803 shutdown_connections 2024-09-16T00:10:06.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.240+0000 7f0e21847640 1 --2- 172.21.15.139:0/3144397803 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e1c0b3430 0x7f0e1c0b4580 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.240+0000 7f0e21847640 1 --2- 172.21.15.139:0/3144397803 >> v2:172.21.15.164:3300/0 conn(0x7f0e1c0d0870 0x7f0e1c0b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.240+0000 7f0e21847640 1 -- 172.21.15.139:0/3144397803 >> 172.21.15.139:0/3144397803 conn(0x7f0e1c06c520 msgr2=0x7f0e1c06c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:06.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.240+0000 7f0e21847640 1 -- 172.21.15.139:0/3144397803 shutdown_connections 2024-09-16T00:10:06.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.241+0000 7f0e21847640 1 -- 172.21.15.139:0/3144397803 wait complete. 2024-09-16T00:10:06.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.242+0000 7f0e21847640 1 Processor -- start 2024-09-16T00:10:06.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.242+0000 7f0e21847640 1 -- start start 2024-09-16T00:10:06.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e21847640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e1c0b3430 0x7f0e1c14fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e21847640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e1c0d0490 0x7f0e1c1f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e21847640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0e1c0d0870 0x7f0e1c1f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e21847640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c15a980 con 0x7f0e1c0b3430 2024-09-16T00:10:06.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e21847640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c15b660 con 0x7f0e1c0d0490 2024-09-16T00:10:06.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e21847640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c15ab00 con 0x7f0e1c0d0870 2024-09-16T00:10:06.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e1affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e1c0b3430 0x7f0e1c14fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e1a7fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e1c0d0490 0x7f0e1c1f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e1affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e1c0b3430 0x7f0e1c14fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55964/0 (socket says 172.21.15.139:55964) 2024-09-16T00:10:06.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e1affd640 1 -- 172.21.15.139:0/3770836035 learned_addr learned my addr 172.21.15.139:0/3770836035 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:06.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e1b7fe640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0e1c0d0870 0x7f0e1c1f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e1affd640 1 -- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0e1c0d0870 msgr2=0x7f0e1c1f43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e1affd640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0e1c0d0870 0x7f0e1c1f43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e1affd640 1 -- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e1c0d0490 msgr2=0x7f0e1c1f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e1affd640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e1c0d0490 0x7f0e1c1f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e1affd640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c14c010 con 0x7f0e1c0b3430 2024-09-16T00:10:06.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e1a7fc640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0e1c0d0490 0x7f0e1c1f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:06.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e1affd640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e1c0b3430 0x7f0e1c14fcf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00c500 tx=0x7f0e0c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:06.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c018070 con 0x7f0e1c0b3430 2024-09-16T00:10:06.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0c007580 con 0x7f0e1c0b3430 2024-09-16T00:10:06.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1c1552f0 con 0x7f0e1c0b3430 2024-09-16T00:10:06.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c004a20 con 0x7f0e1c0b3430 2024-09-16T00:10:06.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.243+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1c14cd50 con 0x7f0e1c0b3430 2024-09-16T00:10:06.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.244+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f0e1c0cc130 con 0x7f0e1c0b3430 2024-09-16T00:10:06.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.245+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0e0c01b070 con 0x7f0e1c0b3430 2024-09-16T00:10:06.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.245+0000 7f0e03fff640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0e0806c2e0 0x7f0e0806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.246+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5973+0+0 (secure 0 0 0) 0x7f0e0c08a700 con 0x7f0e1c0b3430 2024-09-16T00:10:06.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.246+0000 7f0e1a7fc640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0e0806c2e0 0x7f0e0806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.246+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v99) ==== 54+0+16 (secure 0 0 0) 0x7f0e1c0cc130 con 0x7f0e1c0b3430 2024-09-16T00:10:06.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.246+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f0e1c0cb620 0x7f0e1c14cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.246+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e1c151100 con 0x7f0e1c0cb620 2024-09-16T00:10:06.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.246+0000 7f0e1a7fc640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0e0806c2e0 0x7f0e0806e7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0cc6a0 tx=0x7f0e10009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:06.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.246+0000 7f0e1b7fe640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f0e1c0cb620 0x7f0e1c14cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.246+0000 7f0e1b7fe640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f0e1c0cb620 0x7f0e1c14cf50 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:06.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.247+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e1c151100 con 0x7f0e1c0cb620 2024-09-16T00:10:06.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.261+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0e1c0b2c40 con 0x7f0e1c0cb620 2024-09-16T00:10:06.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.263+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0e1c0b2c40 con 0x7f0e1c0cb620 2024-09-16T00:10:06.264 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.263+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f0e1c0cba00 0x7f0e1c14ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.263+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f0e1c14b070 con 0x7f0e1c0cba00 2024-09-16T00:10:06.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.263+0000 7f0e1affd640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f0e1c0cba00 0x7f0e1c14ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.263+0000 7f0e1affd640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f0e1c0cba00 0x7f0e1c14ab30 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:06.265 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.264+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e1c14b070 con 0x7f0e1c0cba00 2024-09-16T00:10:06.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.278+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0e1c159d80 con 0x7f0e1c0cba00 2024-09-16T00:10:06.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.280+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0e1c159d80 con 0x7f0e1c0cba00 2024-09-16T00:10:06.283 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:06.283 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:06.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.280+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f0e1c14dcd0 0x7f0e1c14e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.280+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f0e1c159aa0 con 0x7f0e1c14dcd0 2024-09-16T00:10:06.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.280+0000 7f0e1a7fc640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f0e1c14dcd0 0x7f0e1c14e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.281+0000 7f0e1a7fc640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f0e1c14dcd0 0x7f0e1c14e0f0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:06.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.281+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e1c159aa0 con 0x7f0e1c14dcd0 2024-09-16T00:10:06.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.295+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0e1c1597c0 con 0x7f0e1c14dcd0 2024-09-16T00:10:06.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.296+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0e1c1597c0 con 0x7f0e1c14dcd0 2024-09-16T00:10:06.297 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:06.298 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:06.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.297+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0e1c062f00 0x7f0e1c0ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.297+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f0e1c1594e0 con 0x7f0e1c062f00 2024-09-16T00:10:06.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.297+0000 7f0e1b7fe640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0e1c062f00 0x7f0e1c0ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.297+0000 7f0e1b7fe640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0e1c062f00 0x7f0e1c0ce0b0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:06.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.298+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e1c1594e0 con 0x7f0e1c062f00 2024-09-16T00:10:06.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.299+0000 7f8c5ffff640 1 -- 172.21.15.139:0/4058275199 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v100) ==== 87+0+0 (secure 0 0 0) 0x7f8c6c079f30 con 0x7f8c780ccc90 2024-09-16T00:10:06.300 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is set 2024-09-16T00:10:06.301 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:06.301 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:06.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.302+0000 7f8c7e152640 1 -- 172.21.15.139:0/4058275199 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8c6406c2e0 msgr2=0x7f8c6406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.302+0000 7f8c7e152640 1 --2- 172.21.15.139:0/4058275199 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8c6406c2e0 0x7f8c6406e7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009f30 tx=0x7f8c68009190 comp rx=0 tx=0).stop 2024-09-16T00:10:06.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.302+0000 7f8c7e152640 1 -- 172.21.15.139:0/4058275199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c780ccc90 msgr2=0x7f8c781f2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.302+0000 7f8c7e152640 1 --2- 172.21.15.139:0/4058275199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c780ccc90 0x7f8c781f2c80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c000c00 tx=0x7f8c6c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.302+0000 7f8c7e152640 1 -- 172.21.15.139:0/4058275199 shutdown_connections 2024-09-16T00:10:06.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.302+0000 7f8c7e152640 1 --2- 172.21.15.139:0/4058275199 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8c6406c2e0 0x7f8c6406e7a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.302+0000 7f8c7e152640 1 --2- 172.21.15.139:0/4058275199 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8c78154c20 0x7f8c781f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.302+0000 7f8c7e152640 1 --2- 172.21.15.139:0/4058275199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8c780ccc90 0x7f8c781f2c80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.302+0000 7f8c7e152640 1 --2- 172.21.15.139:0/4058275199 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8c78068060 0x7f8c7815bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.302+0000 7f8c7e152640 1 -- 172.21.15.139:0/4058275199 >> 172.21.15.139:0/4058275199 conn(0x7f8c7806ca40 msgr2=0x7f8c78158670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:06.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.302+0000 7f8c7e152640 1 -- 172.21.15.139:0/4058275199 shutdown_connections 2024-09-16T00:10:06.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.302+0000 7f8c7e152640 1 -- 172.21.15.139:0/4058275199 wait complete. 2024-09-16T00:10:06.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.309+0000 7f9c797fa640 1 -- 172.21.15.139:0/3691389732 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v100) ==== 113+0+0 (secure 0 0 0) 0x7f9c6805d7b0 con 0x7f9c7c0ccc90 2024-09-16T00:10:06.310 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T00:10:06.312 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:10:06.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.311+0000 7f9c82f9f640 1 -- 172.21.15.139:0/3691389732 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9c6006c2b0 msgr2=0x7f9c6006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.311+0000 7f9c82f9f640 1 --2- 172.21.15.139:0/3691389732 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9c6006c2b0 0x7f9c6006e770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9c700090b0 tx=0x7f9c7002e000 comp rx=0 tx=0).stop 2024-09-16T00:10:06.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.311+0000 7f9c82f9f640 1 -- 172.21.15.139:0/3691389732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c7c0ccc90 msgr2=0x7f9c7c0b5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.311+0000 7f9c82f9f640 1 --2- 172.21.15.139:0/3691389732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c7c0ccc90 0x7f9c7c0b5ac0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800e840 tx=0x7f9c6800ed00 comp rx=0 tx=0).stop 2024-09-16T00:10:06.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.311+0000 7f9c82f9f640 1 -- 172.21.15.139:0/3691389732 shutdown_connections 2024-09-16T00:10:06.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.311+0000 7f9c82f9f640 1 --2- 172.21.15.139:0/3691389732 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9c6006c2b0 0x7f9c6006e770 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.311+0000 7f9c82f9f640 1 --2- 172.21.15.139:0/3691389732 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c7c154c20 0x7f9c7c145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.311+0000 7f9c82f9f640 1 --2- 172.21.15.139:0/3691389732 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c7c0ccc90 0x7f9c7c0b5ac0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.311+0000 7f9c82f9f640 1 --2- 172.21.15.139:0/3691389732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c7c068060 0x7f9c7c14e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.311+0000 7f9c82f9f640 1 -- 172.21.15.139:0/3691389732 >> 172.21.15.139:0/3691389732 conn(0x7f9c7c06ca40 msgr2=0x7f9c7c150060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:06.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.311+0000 7f9c82f9f640 1 -- 172.21.15.139:0/3691389732 shutdown_connections 2024-09-16T00:10:06.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.311+0000 7f9c82f9f640 1 -- 172.21.15.139:0/3691389732 wait complete. 2024-09-16T00:10:06.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.313+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0e1c0ce5f0 con 0x7f0e1c062f00 2024-09-16T00:10:06.315 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:06.315+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:06.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.315+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0e1c0ce5f0 con 0x7f0e1c062f00 2024-09-16T00:10:06.316 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:06.317 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:06.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.315+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f0e1c0ceb00 0x7f0e1c02c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.315+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f0e1c0635b0 con 0x7f0e1c0ceb00 2024-09-16T00:10:06.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.316+0000 7f0e1affd640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f0e1c0ceb00 0x7f0e1c02c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.316+0000 7f0e1affd640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f0e1c0ceb00 0x7f0e1c02c640 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:06.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.317+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e1c0635b0 con 0x7f0e1c0ceb00 2024-09-16T00:10:06.324 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T00:10:06.332 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.332+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0e1c063360 con 0x7f0e1c0ceb00 2024-09-16T00:10:06.334 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:06.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.333+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0e1c063360 con 0x7f0e1c0ceb00 2024-09-16T00:10:06.335 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:06.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.333+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f0e1c02d090 0x7f0e1c02d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.333+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f0e1c063570 con 0x7f0e1c02d090 2024-09-16T00:10:06.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.334+0000 7f0e1a7fc640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f0e1c02d090 0x7f0e1c02d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.334+0000 7f0e1a7fc640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f0e1c02d090 0x7f0e1c02d4f0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:06.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.335+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e1c063570 con 0x7f0e1c02d090 2024-09-16T00:10:06.350 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.349+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0e1c063570 con 0x7f0e1c02d090 2024-09-16T00:10:06.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.351+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0e1c063570 con 0x7f0e1c02d090 2024-09-16T00:10:06.352 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:06.353 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:06.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.351+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f0e1c207700 0x7f0e1c209af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.351+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f0e1c20c8c0 con 0x7f0e1c207700 2024-09-16T00:10:06.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.351+0000 7f0e1b7fe640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f0e1c207700 0x7f0e1c209af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.352+0000 7f0e1b7fe640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f0e1c207700 0x7f0e1c209af0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:06.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.353+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e1c20c8c0 con 0x7f0e1c207700 2024-09-16T00:10:06.369 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.368+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0e1c20a030 con 0x7f0e1c207700 2024-09-16T00:10:06.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.371+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0e1c20a030 con 0x7f0e1c207700 2024-09-16T00:10:06.372 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:06.373 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:06.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.371+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f0e1c20a6b0 0x7f0e1c20ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.371+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f0e1c20fc50 con 0x7f0e1c20a6b0 2024-09-16T00:10:06.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.371+0000 7f0e1affd640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f0e1c20a6b0 0x7f0e1c20ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.372+0000 7f0e1affd640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f0e1c20a6b0 0x7f0e1c20ab10 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:06.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.372+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e1c20fc50 con 0x7f0e1c20a6b0 2024-09-16T00:10:06.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.387+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f0e1c20b160 con 0x7f0e1c20a6b0 2024-09-16T00:10:06.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e03fff640 1 -- 172.21.15.139:0/3770836035 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0e1c20b160 con 0x7f0e1c20a6b0 2024-09-16T00:10:06.391 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:10:06.392 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:10:06.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f0e1c0cb620 msgr2=0x7f0e1c14cf50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f0e1c0cb620 0x7f0e1c14cf50 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f0e1c0cba00 msgr2=0x7f0e1c14ab30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f0e1c0cba00 0x7f0e1c14ab30 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f0e1c14dcd0 msgr2=0x7f0e1c14e0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f0e1c14dcd0 0x7f0e1c14e0f0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0e1c062f00 msgr2=0x7f0e1c0ce0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0e1c062f00 0x7f0e1c0ce0b0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f0e1c0ceb00 msgr2=0x7f0e1c02c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f0e1c0ceb00 0x7f0e1c02c640 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e1b7fe640 1 -- 172.21.15.139:0/3770836035 reap_dead start 2024-09-16T00:10:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f0e1c02d090 msgr2=0x7f0e1c02d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f0e1c02d090 0x7f0e1c02d4f0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e1b7fe640 1 -- 172.21.15.139:0/3770836035 reap_dead start 2024-09-16T00:10:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f0e1c207700 msgr2=0x7f0e1c209af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f0e1c207700 0x7f0e1c209af0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f0e1c20a6b0 msgr2=0x7f0e1c20ab10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f0e1c20a6b0 0x7f0e1c20ab10 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e1b7fe640 1 -- 172.21.15.139:0/3770836035 reap_dead start 2024-09-16T00:10:06.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0e0806c2e0 msgr2=0x7f0e0806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0e0806c2e0 0x7f0e0806e7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0cc6a0 tx=0x7f0e10009190 comp rx=0 tx=0).stop 2024-09-16T00:10:06.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e1c0b3430 msgr2=0x7f0e1c14fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e1c0b3430 0x7f0e1c14fcf0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00c500 tx=0x7f0e0c00c9c0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 shutdown_connections 2024-09-16T00:10:06.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0e0806c2e0 0x7f0e0806e7a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 --2- 172.21.15.139:0/3770836035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e1c0b3430 0x7f0e1c14fcf0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 >> 172.21.15.139:0/3770836035 conn(0x7f0e1c06c520 msgr2=0x7f0e1c1585e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:06.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 shutdown_connections 2024-09-16T00:10:06.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.390+0000 7f0e21847640 1 -- 172.21.15.139:0/3770836035 wait complete. 2024-09-16T00:10:06.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.440+0000 7f8fb9401640 1 Processor -- start 2024-09-16T00:10:06.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.440+0000 7f8fb9401640 1 -- start start 2024-09-16T00:10:06.441 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:06.440+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:06.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.441+0000 7f8fb9401640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8fb41518e0 0x7f8fb40c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.441+0000 7f8fb9401640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fb4070c20 0x7f8fb40c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.441+0000 7f8fb9401640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb40c8c20 con 0x7f8fb40c87d0 2024-09-16T00:10:06.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.441+0000 7f8fb9401640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f8fb4158400 con 0x7f8fb41518e0 2024-09-16T00:10:06.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.441+0000 7f8fb9401640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb40c7730 con 0x7f8fb4070c20 2024-09-16T00:10:06.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.441+0000 7f8faa7fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fb4070c20 0x7f8fb40c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.441+0000 7f8fb2ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8fb41518e0 0x7f8fb40c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.441+0000 7f8faa7fc640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fb4070c20 0x7f8fb40c4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:40414/0 (socket says 172.21.15.139:40414) 2024-09-16T00:10:06.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.441+0000 7f8faa7fc640 1 -- 172.21.15.139:0/4248212386 learned_addr learned my addr 172.21.15.139:0/4248212386 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:06.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.441+0000 7f8fb37fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f8fb40c87d0 0x7f8fb40735e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:52346/0 (socket says 172.21.15.139:52346) 2024-09-16T00:10:06.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.441+0000 7f8fb2ffd640 1 -- 172.21.15.139:0/4248212386 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fb4070c20 msgr2=0x7f8fb40c4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:06.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.441+0000 7f8fb2ffd640 1 --2- 172.21.15.139:0/4248212386 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fb4070c20 0x7f8fb40c4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.441+0000 7f8fb2ffd640 1 -- 172.21.15.139:0/4248212386 >> v1:172.21.15.164:6789/0 conn(0x7f8fb40c87d0 legacy=0x7f8fb40735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.441+0000 7f8fb2ffd640 1 -- 172.21.15.139:0/4248212386 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb40c7a10 con 0x7f8fb41518e0 2024-09-16T00:10:06.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.441+0000 7f8faa7fc640 1 --2- 172.21.15.139:0/4248212386 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fb4070c20 0x7f8fb40c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:06.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.442+0000 7f8fb2ffd640 1 --2- 172.21.15.139:0/4248212386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fb41518e0 0x7f8fb40c6fb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40097a0 tx=0x7f8fa402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5da298924514787e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:06.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.442+0000 7f8fb27fc640 1 -- 172.21.15.139:0/4248212386 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4037070 con 0x7f8fb41518e0 2024-09-16T00:10:06.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.442+0000 7f8fb27fc640 1 -- 172.21.15.139:0/4248212386 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa4032440 con 0x7f8fb41518e0 2024-09-16T00:10:06.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.442+0000 7f8fb27fc640 1 -- 172.21.15.139:0/4248212386 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4032810 con 0x7f8fb41518e0 2024-09-16T00:10:06.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.443+0000 7f8fb9401640 1 -- 172.21.15.139:0/4248212386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fb41518e0 msgr2=0x7f8fb40c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.443+0000 7f8fb9401640 1 --2- 172.21.15.139:0/4248212386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fb41518e0 0x7f8fb40c6fb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8fa40097a0 tx=0x7f8fa402a180 comp rx=0 tx=0).stop 2024-09-16T00:10:06.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.443+0000 7f8fb9401640 1 -- 172.21.15.139:0/4248212386 shutdown_connections 2024-09-16T00:10:06.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.443+0000 7f8fb9401640 1 --2- 172.21.15.139:0/4248212386 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fb4070c20 0x7f8fb40c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.443+0000 7f8fb9401640 1 --2- 172.21.15.139:0/4248212386 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fb41518e0 0x7f8fb40c6fb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.443+0000 7f8fb9401640 1 -- 172.21.15.139:0/4248212386 >> 172.21.15.139:0/4248212386 conn(0x7f8fb406ff70 msgr2=0x7f8fb4070380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:06.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.443+0000 7f8fb9401640 1 -- 172.21.15.139:0/4248212386 shutdown_connections 2024-09-16T00:10:06.446 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.443+0000 7f8fb9401640 1 -- 172.21.15.139:0/4248212386 wait complete. 2024-09-16T00:10:06.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.444+0000 7f8fb9401640 1 Processor -- start 2024-09-16T00:10:06.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.444+0000 7f8fb9401640 1 -- start start 2024-09-16T00:10:06.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8fb9401640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb4070c20 0x7f8fb40cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8fb9401640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fb40c87d0 0x7f8fb41531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8fb9401640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fb41518e0 0x7f8fb41536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8fb9401640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb40c82b0 con 0x7f8fb4070c20 2024-09-16T00:10:06.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8fb9401640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb40c7fd0 con 0x7f8fb40c87d0 2024-09-16T00:10:06.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8fb9401640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb40c9200 con 0x7f8fb41518e0 2024-09-16T00:10:06.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8fb2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb4070c20 0x7f8fb40cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8fb2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb4070c20 0x7f8fb40cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:55966/0 (socket says 172.21.15.139:55966) 2024-09-16T00:10:06.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8fb2ffd640 1 -- 172.21.15.139:0/1260091249 learned_addr learned my addr 172.21.15.139:0/1260091249 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:06.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8fb37fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fb41518e0 0x7f8fb41536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8faa7fc640 1 --2- 172.21.15.139:0/1260091249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fb40c87d0 0x7f8fb41531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8fb2ffd640 1 -- 172.21.15.139:0/1260091249 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fb41518e0 msgr2=0x7f8fb41536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.450 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8fb2ffd640 1 --2- 172.21.15.139:0/1260091249 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fb41518e0 0x7f8fb41536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8fb2ffd640 1 -- 172.21.15.139:0/1260091249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fb40c87d0 msgr2=0x7f8fb41531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8fb2ffd640 1 --2- 172.21.15.139:0/1260091249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fb40c87d0 0x7f8fb41531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8fb2ffd640 1 -- 172.21.15.139:0/1260091249 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb41ef2b0 con 0x7f8fb4070c20 2024-09-16T00:10:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.445+0000 7f8fb2ffd640 1 --2- 172.21.15.139:0/1260091249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb4070c20 0x7f8fb40cc020 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8fa402a1c0 tx=0x7f8fa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.446+0000 7f8fabfff640 1 -- 172.21.15.139:0/1260091249 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4041020 con 0x7f8fb4070c20 2024-09-16T00:10:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.446+0000 7f8fabfff640 1 -- 172.21.15.139:0/1260091249 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa4030700 con 0x7f8fb4070c20 2024-09-16T00:10:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.446+0000 7f8fb9401640 1 -- 172.21.15.139:0/1260091249 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb41eefd0 con 0x7f8fb4070c20 2024-09-16T00:10:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.446+0000 7f8fabfff640 1 -- 172.21.15.139:0/1260091249 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4037040 con 0x7f8fb4070c20 2024-09-16T00:10:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.446+0000 7f8fb9401640 1 -- 172.21.15.139:0/1260091249 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb41571f0 con 0x7f8fb4070c20 2024-09-16T00:10:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.447+0000 7f8fb9401640 1 -- 172.21.15.139:0/1260091249 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb4152aa0 con 0x7f8fb4070c20 2024-09-16T00:10:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.447+0000 7f8fabfff640 1 -- 172.21.15.139:0/1260091249 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8fa4044020 con 0x7f8fb4070c20 2024-09-16T00:10:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.448+0000 7f8fabfff640 1 --2- 172.21.15.139:0/1260091249 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f9406c2b0 0x7f8f9406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:06.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.448+0000 7f8fabfff640 1 -- 172.21.15.139:0/1260091249 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5973+0+0 (secure 0 0 0) 0x7f8fa40abc30 con 0x7f8fb4070c20 2024-09-16T00:10:06.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.448+0000 7f8faa7fc640 1 --2- 172.21.15.139:0/1260091249 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f9406c2b0 0x7f8f9406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:06.452 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.448+0000 7f8faa7fc640 1 --2- 172.21.15.139:0/1260091249 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f9406c2b0 0x7f8f9406e770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8fb41f29a0 tx=0x7f8f9c009240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:06.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.452+0000 7f8fabfff640 1 -- 172.21.15.139:0/1260091249 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8fb4152aa0 con 0x7f8fb4070c20 2024-09-16T00:10:06.476 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:06.476 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:06.476 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:06.476 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:06.486 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:10:06.529 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:06.529+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:06.606 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:06.607 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:06.607 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:06.616 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:10:06.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:06.653+0000 7f8fb9401640 1 -- 172.21.15.139:0/1260091249 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) -- 0x7f8fb41573c0 con 0x7f8fb4070c20 2024-09-16T00:10:06.657 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:06.657+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:06.719 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:06.718+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:06.773 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:06.773 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:06.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:06.773 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:06.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.773 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4203.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-16T00:10:06.773 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:10:06.774 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 163.69023656499999, 2024-09-16T00:10:06.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88755623400000005, 2024-09-16T00:10:06.774 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.774 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.774 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.774 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:06.774 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4203", 2024-09-16T00:10:06.774 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "172.21.15.139:0/2049479133", 2024-09-16T00:10:06.774 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 1 2024-09-16T00:10:06.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.774 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:10:06.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080829+0000", 2024-09-16T00:10:06.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080831+0000", 2024-09-16T00:10:06.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.252e-06 2024-09-16T00:10:06.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080838+0000", 2024-09-16T00:10:06.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.849e-06 2024-09-16T00:10:06.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080847+0000", 2024-09-16T00:10:06.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9040000000000004e-06 2024-09-16T00:10:06.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080858+0000", 2024-09-16T00:10:06.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0013000000000001e-05 2024-09-16T00:10:06.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080911+0000", 2024-09-16T00:10:06.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3730000000000002e-05 2024-09-16T00:10:06.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for peered", 2024-09-16T00:10:06.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.080919+0000", 2024-09-16T00:10:06.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8960000000000003e-06 2024-09-16T00:10:06.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965069+0000", 2024-09-16T00:10:06.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.88414986100000004 2024-09-16T00:10:06.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965225+0000", 2024-09-16T00:10:06.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015566899999999999 2024-09-16T00:10:06.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.965414+0000", 2024-09-16T00:10:06.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00018943000000000001 2024-09-16T00:10:06.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.967957+0000", 2024-09-16T00:10:06.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025432549999999999 2024-09-16T00:10:06.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968317+0000", 2024-09-16T00:10:06.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00035963299999999999 2024-09-16T00:10:06.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968356+0000", 2024-09-16T00:10:06.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:10:06.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:07:23.968386+0000", 2024-09-16T00:10:06.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9581999999999999e-05 2024-09-16T00:10:06.780 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.780 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.780 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.780 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2402 2.8 2:17051619:::200.0000000f:head [write 2106308~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.780 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.664943+0000", 2024-09-16T00:10:06.781 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.106123534000002, 2024-09-16T00:10:06.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024647203999999999, 2024-09-16T00:10:06.781 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.781 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.781 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.781 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.664943+0000", 2024-09-16T00:10:06.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.664943+0000", 2024-09-16T00:10:06.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.664943+0000", 2024-09-16T00:10:06.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8999999999999995e-07 2024-09-16T00:10:06.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.664947+0000", 2024-09-16T00:10:06.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.146e-06 2024-09-16T00:10:06.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.664953+0000", 2024-09-16T00:10:06.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7179999999999999e-06 2024-09-16T00:10:06.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.664957+0000", 2024-09-16T00:10:06.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1359999999999999e-06 2024-09-16T00:10:06.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.682748+0000", 2024-09-16T00:10:06.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017790166999999999 2024-09-16T00:10:06.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.682792+0000", 2024-09-16T00:10:06.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4400999999999997e-05 2024-09-16T00:10:06.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.682859+0000", 2024-09-16T00:10:06.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6867999999999997e-05 2024-09-16T00:10:06.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.684686+0000", 2024-09-16T00:10:06.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0018267120000000001 2024-09-16T00:10:06.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689507+0000", 2024-09-16T00:10:06.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048218230000000003 2024-09-16T00:10:06.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689558+0000", 2024-09-16T00:10:06.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0118999999999999e-05 2024-09-16T00:10:06.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689590+0000", 2024-09-16T00:10:06.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2224000000000002e-05 2024-09-16T00:10:06.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.786 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.786 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.786 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2408 2.8 2:17051619:::200.0000000f:head [write 2116866~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.787 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.668321+0000", 2024-09-16T00:10:06.787 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.102745034000002, 2024-09-16T00:10:06.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.029956139999999999, 2024-09-16T00:10:06.787 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.787 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.787 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.787 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.668321+0000", 2024-09-16T00:10:06.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.668321+0000", 2024-09-16T00:10:06.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.668322+0000", 2024-09-16T00:10:06.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0900000000000001e-07 2024-09-16T00:10:06.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.668326+0000", 2024-09-16T00:10:06.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5549999999999996e-06 2024-09-16T00:10:06.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.668336+0000", 2024-09-16T00:10:06.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:10:06.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.668342+0000", 2024-09-16T00:10:06.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1800000000000001e-06 2024-09-16T00:10:06.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.686415+0000", 2024-09-16T00:10:06.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018073371000000001 2024-09-16T00:10:06.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.686493+0000", 2024-09-16T00:10:06.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7428000000000005e-05 2024-09-16T00:10:06.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.686605+0000", 2024-09-16T00:10:06.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011246 2024-09-16T00:10:06.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689921+0000", 2024-09-16T00:10:06.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0033161190000000002 2024-09-16T00:10:06.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698219+0000", 2024-09-16T00:10:06.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082978380000000001 2024-09-16T00:10:06.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698252+0000", 2024-09-16T00:10:06.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2407000000000003e-05 2024-09-16T00:10:06.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698277+0000", 2024-09-16T00:10:06.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5554999999999998e-05 2024-09-16T00:10:06.792 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.792 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.792 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.792 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2409 2.8 2:17051619:::200.0000000f:head [write 2119111~5159 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.792 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.669949+0000", 2024-09-16T00:10:06.792 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.101117285000001, 2024-09-16T00:10:06.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028439168000000001, 2024-09-16T00:10:06.792 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.793 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.793 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.793 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.669949+0000", 2024-09-16T00:10:06.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.669949+0000", 2024-09-16T00:10:06.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.669950+0000", 2024-09-16T00:10:06.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1299999999999999e-07 2024-09-16T00:10:06.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.669956+0000", 2024-09-16T00:10:06.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3690000000000003e-06 2024-09-16T00:10:06.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.669965+0000", 2024-09-16T00:10:06.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1849999999999993e-06 2024-09-16T00:10:06.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.669969+0000", 2024-09-16T00:10:06.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2849999999999996e-06 2024-09-16T00:10:06.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687050+0000", 2024-09-16T00:10:06.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017080733000000001 2024-09-16T00:10:06.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687126+0000", 2024-09-16T00:10:06.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5867999999999999e-05 2024-09-16T00:10:06.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687236+0000", 2024-09-16T00:10:06.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000110442 2024-09-16T00:10:06.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689926+0000", 2024-09-16T00:10:06.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026899469999999998 2024-09-16T00:10:06.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698326+0000", 2024-09-16T00:10:06.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0083998240000000002 2024-09-16T00:10:06.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698358+0000", 2024-09-16T00:10:06.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1464000000000002e-05 2024-09-16T00:10:06.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698388+0000", 2024-09-16T00:10:06.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0238000000000001e-05 2024-09-16T00:10:06.797 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.798 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.798 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.798 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2410 2.8 2:17051619:::200.0000000f:head [write 2124270~2419 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.798 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.670334+0000", 2024-09-16T00:10:06.798 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.100732089000001, 2024-09-16T00:10:06.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.028210857999999998, 2024-09-16T00:10:06.798 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.798 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.798 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.798 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.670334+0000", 2024-09-16T00:10:06.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.670334+0000", 2024-09-16T00:10:06.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.670335+0000", 2024-09-16T00:10:06.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6000000000000003e-07 2024-09-16T00:10:06.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.670339+0000", 2024-09-16T00:10:06.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0940000000000001e-06 2024-09-16T00:10:06.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.670346+0000", 2024-09-16T00:10:06.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.605e-06 2024-09-16T00:10:06.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.670351+0000", 2024-09-16T00:10:06.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2710000000000003e-06 2024-09-16T00:10:06.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688139+0000", 2024-09-16T00:10:06.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017788780000000001 2024-09-16T00:10:06.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688218+0000", 2024-09-16T00:10:06.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8320999999999998e-05 2024-09-16T00:10:06.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688316+0000", 2024-09-16T00:10:06.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8071000000000003e-05 2024-09-16T00:10:06.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.690555+0000", 2024-09-16T00:10:06.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022395039999999998 2024-09-16T00:10:06.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698454+0000", 2024-09-16T00:10:06.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0078986219999999992 2024-09-16T00:10:06.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698507+0000", 2024-09-16T00:10:06.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2952999999999999e-05 2024-09-16T00:10:06.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698545+0000", 2024-09-16T00:10:06.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7976999999999999e-05 2024-09-16T00:10:06.803 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.803 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.803 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.804 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2411 2.8 2:17051619:::200.0000000f:head [write 2126689~6053 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.804 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.671637+0000", 2024-09-16T00:10:06.804 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.099428792000001, 2024-09-16T00:10:06.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0269952, 2024-09-16T00:10:06.804 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.804 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.804 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.804 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.671637+0000", 2024-09-16T00:10:06.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.671637+0000", 2024-09-16T00:10:06.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.671638+0000", 2024-09-16T00:10:06.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.054e-06 2024-09-16T00:10:06.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.671648+0000", 2024-09-16T00:10:06.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3190000000000001e-06 2024-09-16T00:10:06.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.671658+0000", 2024-09-16T00:10:06.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0227000000000001e-05 2024-09-16T00:10:06.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.671665+0000", 2024-09-16T00:10:06.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.922e-06 2024-09-16T00:10:06.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688668+0000", 2024-09-16T00:10:06.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017003429 2024-09-16T00:10:06.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688712+0000", 2024-09-16T00:10:06.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3664000000000001e-05 2024-09-16T00:10:06.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688773+0000", 2024-09-16T00:10:06.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0968000000000003e-05 2024-09-16T00:10:06.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.690561+0000", 2024-09-16T00:10:06.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001788408 2024-09-16T00:10:06.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698585+0000", 2024-09-16T00:10:06.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0080236279999999997 2024-09-16T00:10:06.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698613+0000", 2024-09-16T00:10:06.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8004000000000002e-05 2024-09-16T00:10:06.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698632+0000", 2024-09-16T00:10:06.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9576999999999999e-05 2024-09-16T00:10:06.809 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.810 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.810 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2412 2.8 2:17051619:::200.0000000f:head [write 2132742~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.810 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.672301+0000", 2024-09-16T00:10:06.810 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.098765327999999, 2024-09-16T00:10:06.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026473445000000002, 2024-09-16T00:10:06.810 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.810 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.810 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.810 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.672301+0000", 2024-09-16T00:10:06.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.672301+0000", 2024-09-16T00:10:06.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.672301+0000", 2024-09-16T00:10:06.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.37e-07 2024-09-16T00:10:06.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.672307+0000", 2024-09-16T00:10:06.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4410000000000001e-06 2024-09-16T00:10:06.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.672318+0000", 2024-09-16T00:10:06.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1246e-05 2024-09-16T00:10:06.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.672326+0000", 2024-09-16T00:10:06.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6120000000000001e-06 2024-09-16T00:10:06.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689084+0000", 2024-09-16T00:10:06.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016758021000000001 2024-09-16T00:10:06.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689128+0000", 2024-09-16T00:10:06.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3863000000000003e-05 2024-09-16T00:10:06.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689190+0000", 2024-09-16T00:10:06.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2589000000000007e-05 2024-09-16T00:10:06.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.691512+0000", 2024-09-16T00:10:06.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023218800000000001 2024-09-16T00:10:06.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698686+0000", 2024-09-16T00:10:06.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0071734069999999997 2024-09-16T00:10:06.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698737+0000", 2024-09-16T00:10:06.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1371e-05 2024-09-16T00:10:06.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698774+0000", 2024-09-16T00:10:06.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7277999999999998e-05 2024-09-16T00:10:06.815 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.815 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.815 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.816 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2413 2.8 2:17051619:::200.0000000f:head [write 2134651~2765 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.816 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.673125+0000", 2024-09-16T00:10:06.816 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.097941225, 2024-09-16T00:10:06.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025733072999999999, 2024-09-16T00:10:06.816 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.816 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.816 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.816 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.673125+0000", 2024-09-16T00:10:06.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.673125+0000", 2024-09-16T00:10:06.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.673126+0000", 2024-09-16T00:10:06.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9599999999999998e-07 2024-09-16T00:10:06.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.673139+0000", 2024-09-16T00:10:06.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3063e-05 2024-09-16T00:10:06.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.673150+0000", 2024-09-16T00:10:06.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1491999999999999e-05 2024-09-16T00:10:06.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.673157+0000", 2024-09-16T00:10:06.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8739999999999997e-06 2024-09-16T00:10:06.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.690110+0000", 2024-09-16T00:10:06.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016952826000000001 2024-09-16T00:10:06.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.690156+0000", 2024-09-16T00:10:06.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5930000000000002e-05 2024-09-16T00:10:06.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.690224+0000", 2024-09-16T00:10:06.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7595999999999995e-05 2024-09-16T00:10:06.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.692680+0000", 2024-09-16T00:10:06.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024567970000000001 2024-09-16T00:10:06.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698813+0000", 2024-09-16T00:10:06.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061326870000000004 2024-09-16T00:10:06.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698839+0000", 2024-09-16T00:10:06.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6299999999999999e-05 2024-09-16T00:10:06.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698858+0000", 2024-09-16T00:10:06.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8612000000000001e-05 2024-09-16T00:10:06.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.821 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.821 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2414 2.8 2:17051619:::200.0000000f:head [write 2137416~3263 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.822 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.674025+0000", 2024-09-16T00:10:06.822 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.097041087000001, 2024-09-16T00:10:06.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024972957, 2024-09-16T00:10:06.822 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.822 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.822 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.822 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.674025+0000", 2024-09-16T00:10:06.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.674025+0000", 2024-09-16T00:10:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.674026+0000", 2024-09-16T00:10:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4399999999999999e-07 2024-09-16T00:10:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.674031+0000", 2024-09-16T00:10:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1680000000000003e-06 2024-09-16T00:10:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.674039+0000", 2024-09-16T00:10:06.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2360000000000004e-06 2024-09-16T00:10:06.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.674044+0000", 2024-09-16T00:10:06.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9930000000000003e-06 2024-09-16T00:10:06.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.690590+0000", 2024-09-16T00:10:06.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016545905999999999 2024-09-16T00:10:06.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.690635+0000", 2024-09-16T00:10:06.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4455e-05 2024-09-16T00:10:06.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.690702+0000", 2024-09-16T00:10:06.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7587000000000002e-05 2024-09-16T00:10:06.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.692686+0000", 2024-09-16T00:10:06.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019839810000000001 2024-09-16T00:10:06.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698906+0000", 2024-09-16T00:10:06.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0062200520000000002 2024-09-16T00:10:06.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698962+0000", 2024-09-16T00:10:06.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5909000000000003e-05 2024-09-16T00:10:06.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.698998+0000", 2024-09-16T00:10:06.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5825999999999999e-05 2024-09-16T00:10:06.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.827 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.827 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2439 2.8 2:17051619:::200.0000000f:head [write 2194864~2754 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.827 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.687880+0000", 2024-09-16T00:10:06.827 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.083186551000001, 2024-09-16T00:10:06.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026317367000000001, 2024-09-16T00:10:06.827 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.828 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.828 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.828 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687880+0000", 2024-09-16T00:10:06.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.828 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.828 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687880+0000", 2024-09-16T00:10:06.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687880+0000", 2024-09-16T00:10:06.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1800000000000005e-07 2024-09-16T00:10:06.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687885+0000", 2024-09-16T00:10:06.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9300000000000002e-06 2024-09-16T00:10:06.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687896+0000", 2024-09-16T00:10:06.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0499999999999999e-05 2024-09-16T00:10:06.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.687902+0000", 2024-09-16T00:10:06.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.144e-06 2024-09-16T00:10:06.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.707045+0000", 2024-09-16T00:10:06.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019143104000000001 2024-09-16T00:10:06.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.707090+0000", 2024-09-16T00:10:06.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5367000000000001e-05 2024-09-16T00:10:06.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.707156+0000", 2024-09-16T00:10:06.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5977999999999997e-05 2024-09-16T00:10:06.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.711185+0000", 2024-09-16T00:10:06.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040285440000000002 2024-09-16T00:10:06.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714139+0000", 2024-09-16T00:10:06.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002954071 2024-09-16T00:10:06.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714173+0000", 2024-09-16T00:10:06.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4084000000000001e-05 2024-09-16T00:10:06.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714197+0000", 2024-09-16T00:10:06.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.3927e-05 2024-09-16T00:10:06.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.833 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.833 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2440 2.8 2:17051619:::200.0000000f:head [write 2197618~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.833 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.688247+0000", 2024-09-16T00:10:06.833 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.082819187999998, 2024-09-16T00:10:06.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.026045682000000001, 2024-09-16T00:10:06.833 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.833 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.833 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.833 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688247+0000", 2024-09-16T00:10:06.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688247+0000", 2024-09-16T00:10:06.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688248+0000", 2024-09-16T00:10:06.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4e-07 2024-09-16T00:10:06.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688253+0000", 2024-09-16T00:10:06.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6500000000000001e-06 2024-09-16T00:10:06.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688266+0000", 2024-09-16T00:10:06.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2797e-05 2024-09-16T00:10:06.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688274+0000", 2024-09-16T00:10:06.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6939999999999995e-06 2024-09-16T00:10:06.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.707476+0000", 2024-09-16T00:10:06.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019202520000000001 2024-09-16T00:10:06.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.707557+0000", 2024-09-16T00:10:06.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1083999999999998e-05 2024-09-16T00:10:06.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.707673+0000", 2024-09-16T00:10:06.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116029 2024-09-16T00:10:06.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.711526+0000", 2024-09-16T00:10:06.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038529060000000001 2024-09-16T00:10:06.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714230+0000", 2024-09-16T00:10:06.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00270396 2024-09-16T00:10:06.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714260+0000", 2024-09-16T00:10:06.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9238999999999999e-05 2024-09-16T00:10:06.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714293+0000", 2024-09-16T00:10:06.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2963000000000003e-05 2024-09-16T00:10:06.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.838 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.839 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.839 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2441 2.8 2:17051619:::200.0000000f:head [write 2199527~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.839 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.688682+0000", 2024-09-16T00:10:06.839 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.082383783000001, 2024-09-16T00:10:06.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025692883999999999, 2024-09-16T00:10:06.839 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.839 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.839 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.839 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688682+0000", 2024-09-16T00:10:06.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688682+0000", 2024-09-16T00:10:06.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688683+0000", 2024-09-16T00:10:06.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.09e-07 2024-09-16T00:10:06.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688689+0000", 2024-09-16T00:10:06.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.536e-06 2024-09-16T00:10:06.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688699+0000", 2024-09-16T00:10:06.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9219999999999992e-06 2024-09-16T00:10:06.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688705+0000", 2024-09-16T00:10:06.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6549999999999998e-06 2024-09-16T00:10:06.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.707989+0000", 2024-09-16T00:10:06.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019283733000000001 2024-09-16T00:10:06.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.708029+0000", 2024-09-16T00:10:06.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9756999999999999e-05 2024-09-16T00:10:06.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.708092+0000", 2024-09-16T00:10:06.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3181000000000004e-05 2024-09-16T00:10:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.711530+0000", 2024-09-16T00:10:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003437842 2024-09-16T00:10:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714329+0000", 2024-09-16T00:10:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0027988179999999998 2024-09-16T00:10:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714356+0000", 2024-09-16T00:10:06.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7107e-05 2024-09-16T00:10:06.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714375+0000", 2024-09-16T00:10:06.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9423999999999999e-05 2024-09-16T00:10:06.844 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.844 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.844 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.844 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2442 2.8 2:17051619:::200.0000000f:head [write 2200926~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.844 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.688930+0000", 2024-09-16T00:10:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.082135806, 2024-09-16T00:10:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025533200999999998, 2024-09-16T00:10:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688930+0000", 2024-09-16T00:10:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688930+0000", 2024-09-16T00:10:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688931+0000", 2024-09-16T00:10:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3e-07 2024-09-16T00:10:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688935+0000", 2024-09-16T00:10:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8139999999999998e-06 2024-09-16T00:10:06.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688942+0000", 2024-09-16T00:10:06.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9009999999999997e-06 2024-09-16T00:10:06.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.688946+0000", 2024-09-16T00:10:06.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3810000000000001e-06 2024-09-16T00:10:06.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.708503+0000", 2024-09-16T00:10:06.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019556819999999999 2024-09-16T00:10:06.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.708583+0000", 2024-09-16T00:10:06.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0476000000000007e-05 2024-09-16T00:10:06.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.708709+0000", 2024-09-16T00:10:06.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012553799999999999 2024-09-16T00:10:06.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.711533+0000", 2024-09-16T00:10:06.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028240090000000002 2024-09-16T00:10:06.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714408+0000", 2024-09-16T00:10:06.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.002875542 2024-09-16T00:10:06.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714444+0000", 2024-09-16T00:10:06.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5967999999999999e-05 2024-09-16T00:10:06.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714463+0000", 2024-09-16T00:10:06.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9122000000000001e-05 2024-09-16T00:10:06.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.850 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.850 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.850 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2443 2.8 2:17051619:::200.0000000f:head [write 2202835~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.850 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.689498+0000", 2024-09-16T00:10:06.850 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.081567583999998, 2024-09-16T00:10:06.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025058134999999999, 2024-09-16T00:10:06.850 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.850 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.850 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.851 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689498+0000", 2024-09-16T00:10:06.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689498+0000", 2024-09-16T00:10:06.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689499+0000", 2024-09-16T00:10:06.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4799999999999997e-07 2024-09-16T00:10:06.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689505+0000", 2024-09-16T00:10:06.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2789999999999998e-06 2024-09-16T00:10:06.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689514+0000", 2024-09-16T00:10:06.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1470000000000007e-06 2024-09-16T00:10:06.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689519+0000", 2024-09-16T00:10:06.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.775e-06 2024-09-16T00:10:06.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.708987+0000", 2024-09-16T00:10:06.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019468467999999999 2024-09-16T00:10:06.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.709032+0000", 2024-09-16T00:10:06.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4811999999999997e-05 2024-09-16T00:10:06.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.709098+0000", 2024-09-16T00:10:06.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5968000000000003e-05 2024-09-16T00:10:06.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.711536+0000", 2024-09-16T00:10:06.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024380650000000001 2024-09-16T00:10:06.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714497+0000", 2024-09-16T00:10:06.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0029612330000000002 2024-09-16T00:10:06.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714536+0000", 2024-09-16T00:10:06.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9159999999999998e-05 2024-09-16T00:10:06.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714557+0000", 2024-09-16T00:10:06.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0380000000000001e-05 2024-09-16T00:10:06.855 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.856 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.856 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.856 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2444 2.8 2:17051619:::200.0000000f:head [write 2204744~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.856 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.689810+0000", 2024-09-16T00:10:06.856 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.081255934000001, 2024-09-16T00:10:06.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024842684, 2024-09-16T00:10:06.856 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.856 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.856 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.856 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689810+0000", 2024-09-16T00:10:06.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689810+0000", 2024-09-16T00:10:06.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689811+0000", 2024-09-16T00:10:06.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.032e-06 2024-09-16T00:10:06.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689815+0000", 2024-09-16T00:10:06.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3089999999999997e-06 2024-09-16T00:10:06.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689824+0000", 2024-09-16T00:10:06.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2849999999999995e-06 2024-09-16T00:10:06.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.689828+0000", 2024-09-16T00:10:06.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.639e-06 2024-09-16T00:10:06.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.709404+0000", 2024-09-16T00:10:06.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019575134000000001 2024-09-16T00:10:06.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.709457+0000", 2024-09-16T00:10:06.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3140000000000003e-05 2024-09-16T00:10:06.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.709523+0000", 2024-09-16T00:10:06.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6688999999999998e-05 2024-09-16T00:10:06.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.711539+0000", 2024-09-16T00:10:06.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0020153110000000001 2024-09-16T00:10:06.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714593+0000", 2024-09-16T00:10:06.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030542759999999999 2024-09-16T00:10:06.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714633+0000", 2024-09-16T00:10:06.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9817000000000002e-05 2024-09-16T00:10:06.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.714653+0000", 2024-09-16T00:10:06.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0052e-05 2024-09-16T00:10:06.861 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.861 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.861 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.862 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2456 2.8 2:17051619:::200.0000000f:head [write 2229343~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.862 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.696827+0000", 2024-09-16T00:10:06.862 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.074239553999998, 2024-09-16T00:10:06.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.025114133, 2024-09-16T00:10:06.862 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.862 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.862 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.862 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.696827+0000", 2024-09-16T00:10:06.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.696827+0000", 2024-09-16T00:10:06.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.696827+0000", 2024-09-16T00:10:06.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4799999999999998e-07 2024-09-16T00:10:06.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.696832+0000", 2024-09-16T00:10:06.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.578e-06 2024-09-16T00:10:06.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.696841+0000", 2024-09-16T00:10:06.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1239999999999994e-06 2024-09-16T00:10:06.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.696847+0000", 2024-09-16T00:10:06.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2559999999999999e-06 2024-09-16T00:10:06.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.715538+0000", 2024-09-16T00:10:06.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018690612999999998 2024-09-16T00:10:06.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.715584+0000", 2024-09-16T00:10:06.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6100999999999998e-05 2024-09-16T00:10:06.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.715648+0000", 2024-09-16T00:10:06.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4451999999999999e-05 2024-09-16T00:10:06.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.717300+0000", 2024-09-16T00:10:06.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0016512510000000001 2024-09-16T00:10:06.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.721874+0000", 2024-09-16T00:10:06.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045744549999999998 2024-09-16T00:10:06.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.721905+0000", 2024-09-16T00:10:06.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0898999999999997e-05 2024-09-16T00:10:06.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.721941+0000", 2024-09-16T00:10:06.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5756000000000001e-05 2024-09-16T00:10:06.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.867 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.867 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.867 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2457 2.8 2:17051619:::200.0000000f:head [write 2231252~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.867 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.697232+0000", 2024-09-16T00:10:06.867 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.073833671999999, 2024-09-16T00:10:06.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.024791982000000001, 2024-09-16T00:10:06.867 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.868 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.868 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.868 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.697232+0000", 2024-09-16T00:10:06.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.697232+0000", 2024-09-16T00:10:06.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.697233+0000", 2024-09-16T00:10:06.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8400000000000005e-07 2024-09-16T00:10:06.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.697237+0000", 2024-09-16T00:10:06.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7359999999999999e-06 2024-09-16T00:10:06.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.697245+0000", 2024-09-16T00:10:06.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3329999999999997e-06 2024-09-16T00:10:06.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.697251+0000", 2024-09-16T00:10:06.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9549999999999997e-06 2024-09-16T00:10:06.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.715904+0000", 2024-09-16T00:10:06.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018653075000000002 2024-09-16T00:10:06.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.715981+0000", 2024-09-16T00:10:06.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7337999999999997e-05 2024-09-16T00:10:06.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.716110+0000", 2024-09-16T00:10:06.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012847100000000001 2024-09-16T00:10:06.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.717302+0000", 2024-09-16T00:10:06.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011925029999999999 2024-09-16T00:10:06.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.721976+0000", 2024-09-16T00:10:06.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004673478 2024-09-16T00:10:06.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.722002+0000", 2024-09-16T00:10:06.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6438999999999999e-05 2024-09-16T00:10:06.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.722024+0000", 2024-09-16T00:10:06.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2169999999999999e-05 2024-09-16T00:10:06.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.873 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.873 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.873 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2683 2.8 2:17051619:::200.0000000f:head [write 3733504~9752 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.873 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:50.010774+0000", 2024-09-16T00:10:06.873 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 16.760292037999999, 2024-09-16T00:10:06.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.057674191999999999, 2024-09-16T00:10:06.873 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.873 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.873 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.873 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.010774+0000", 2024-09-16T00:10:06.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.010774+0000", 2024-09-16T00:10:06.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.010776+0000", 2024-09-16T00:10:06.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9709999999999998e-06 2024-09-16T00:10:06.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.010795+0000", 2024-09-16T00:10:06.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8732e-05 2024-09-16T00:10:06.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.010813+0000", 2024-09-16T00:10:06.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8199999999999999e-05 2024-09-16T00:10:06.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.010822+0000", 2024-09-16T00:10:06.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4870000000000008e-06 2024-09-16T00:10:06.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.010876+0000", 2024-09-16T00:10:06.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3594999999999998e-05 2024-09-16T00:10:06.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.010931+0000", 2024-09-16T00:10:06.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5047000000000003e-05 2024-09-16T00:10:06.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.011025+0000", 2024-09-16T00:10:06.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4384999999999998e-05 2024-09-16T00:10:06.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.011988+0000", 2024-09-16T00:10:06.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00096290000000000004 2024-09-16T00:10:06.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.068368+0000", 2024-09-16T00:10:06.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.056379187999999997 2024-09-16T00:10:06.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.068407+0000", 2024-09-16T00:10:06.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.964e-05 2024-09-16T00:10:06.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.068448+0000", 2024-09-16T00:10:06.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1047000000000002e-05 2024-09-16T00:10:06.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.878 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.879 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2684 2.8 2:17051619:::200.0000000f:head [write 3743256~15003 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.879 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:50.023831+0000", 2024-09-16T00:10:06.879 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 16.747234636000002, 2024-09-16T00:10:06.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045369923, 2024-09-16T00:10:06.879 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.879 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.879 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.879 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.023831+0000", 2024-09-16T00:10:06.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.023831+0000", 2024-09-16T00:10:06.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.023833+0000", 2024-09-16T00:10:06.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9630000000000001e-06 2024-09-16T00:10:06.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.023858+0000", 2024-09-16T00:10:06.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.4610999999999999e-05 2024-09-16T00:10:06.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.023874+0000", 2024-09-16T00:10:06.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6472000000000001e-05 2024-09-16T00:10:06.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.023882+0000", 2024-09-16T00:10:06.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7829999999999999e-06 2024-09-16T00:10:06.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.023928+0000", 2024-09-16T00:10:06.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6008999999999999e-05 2024-09-16T00:10:06.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.023991+0000", 2024-09-16T00:10:06.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2298000000000006e-05 2024-09-16T00:10:06.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024075+0000", 2024-09-16T00:10:06.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4178999999999995e-05 2024-09-16T00:10:06.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.025111+0000", 2024-09-16T00:10:06.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0010360499999999999 2024-09-16T00:10:06.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.069142+0000", 2024-09-16T00:10:06.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044031305999999999 2024-09-16T00:10:06.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.069175+0000", 2024-09-16T00:10:06.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3182000000000002e-05 2024-09-16T00:10:06.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.069201+0000", 2024-09-16T00:10:06.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6069999999999999e-05 2024-09-16T00:10:06.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.884 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.885 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:2685 2.8 2:17051619:::200.0000000f:head [write 3758259~1959 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e89)", 2024-09-16T00:10:06.885 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:50.024646+0000", 2024-09-16T00:10:06.885 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 16.746420358000002, 2024-09-16T00:10:06.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.045535516999999998, 2024-09-16T00:10:06.885 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:06.885 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:06.885 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:06.885 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:06.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:06.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024646+0000", 2024-09-16T00:10:06.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:06.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024646+0000", 2024-09-16T00:10:06.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:06.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:06.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024647+0000", 2024-09-16T00:10:06.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9400000000000004e-07 2024-09-16T00:10:06.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:06.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024652+0000", 2024-09-16T00:10:06.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4949999999999999e-06 2024-09-16T00:10:06.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:06.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024671+0000", 2024-09-16T00:10:06.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9139000000000001e-05 2024-09-16T00:10:06.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:06.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024680+0000", 2024-09-16T00:10:06.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9560000000000003e-06 2024-09-16T00:10:06.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:06.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024716+0000", 2024-09-16T00:10:06.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6310000000000003e-05 2024-09-16T00:10:06.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:06.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024761+0000", 2024-09-16T00:10:06.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4258000000000002e-05 2024-09-16T00:10:06.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:06.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.024827+0000", 2024-09-16T00:10:06.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6528999999999999e-05 2024-09-16T00:10:06.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:06.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.025686+0000", 2024-09-16T00:10:06.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00085850800000000001 2024-09-16T00:10:06.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:06.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.070124+0000", 2024-09-16T00:10:06.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.044437890000000001 2024-09-16T00:10:06.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:06.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.070156+0000", 2024-09-16T00:10:06.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2481999999999998e-05 2024-09-16T00:10:06.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:06.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:06.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:06.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:50.070181+0000", 2024-09-16T00:10:06.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5055999999999999e-05 2024-09-16T00:10:06.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.890 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:06.890 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:06.891 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:06.892 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:10:06.893 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:06.892+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:06.993 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:06.992+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:07.049 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:07.049 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:07.049 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:07.049 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:07.058 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:10:07.094 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:07.093+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:07.178 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:07.178 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:07.178 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:07.178 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:07.178 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:07.189 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:10:07.232 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:07.232+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:07.295 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:07.295+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:07.300 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:07.300 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:07.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:07.300 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:07.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.300 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5178 3.2c 3:375e1b9b:::100000030c8.00000002:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:07.300 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:07.300 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4896572109999999, 2024-09-16T00:10:07.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012118438, 2024-09-16T00:10:07.300 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.300 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.301 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:07.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:07.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808909+0000", 2024-09-16T00:10:07.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8160000000000001e-06 2024-09-16T00:10:07.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808921+0000", 2024-09-16T00:10:07.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1117999999999999e-05 2024-09-16T00:10:07.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808941+0000", 2024-09-16T00:10:07.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0712000000000001e-05 2024-09-16T00:10:07.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808952+0000", 2024-09-16T00:10:07.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0193e-05 2024-09-16T00:10:07.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809019+0000", 2024-09-16T00:10:07.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7404999999999996e-05 2024-09-16T00:10:07.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809100+0000", 2024-09-16T00:10:07.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1029e-05 2024-09-16T00:10:07.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:07.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809206+0000", 2024-09-16T00:10:07.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105603 2024-09-16T00:10:07.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:07.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.815162+0000", 2024-09-16T00:10:07.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059567250000000004 2024-09-16T00:10:07.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.820925+0000", 2024-09-16T00:10:07.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057623479999999996 2024-09-16T00:10:07.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:07.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.820987+0000", 2024-09-16T00:10:07.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2680000000000003e-05 2024-09-16T00:10:07.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.821026+0000", 2024-09-16T00:10:07.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8809000000000001e-05 2024-09-16T00:10:07.308 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.308 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.308 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.309 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5196 3.26 3:66ea4dfb:::100000030f0.00000001:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:07.309 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:07.309 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4884701830000004, 2024-09-16T00:10:07.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095222459999999998, 2024-09-16T00:10:07.309 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.309 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.309 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.309 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:07.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:07.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:07.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3499999999999995e-07 2024-09-16T00:10:07.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810102+0000", 2024-09-16T00:10:07.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7390000000000002e-06 2024-09-16T00:10:07.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810112+0000", 2024-09-16T00:10:07.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7550000000000006e-06 2024-09-16T00:10:07.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810127+0000", 2024-09-16T00:10:07.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4749999999999999e-05 2024-09-16T00:10:07.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810184+0000", 2024-09-16T00:10:07.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7701999999999997e-05 2024-09-16T00:10:07.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810256+0000", 2024-09-16T00:10:07.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1595999999999997e-05 2024-09-16T00:10:07.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:07.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810346+0000", 2024-09-16T00:10:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0043999999999994e-05 2024-09-16T00:10:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.815239+0000", 2024-09-16T00:10:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048934470000000004 2024-09-16T00:10:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819526+0000", 2024-09-16T00:10:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042868630000000001 2024-09-16T00:10:07.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819577+0000", 2024-09-16T00:10:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0445999999999998e-05 2024-09-16T00:10:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819617+0000", 2024-09-16T00:10:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0169000000000001e-05 2024-09-16T00:10:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5244 3.20 3:068a06d7:::100000032f2.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4713746759999999, 2024-09-16T00:10:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094083010000000009, 2024-09-16T00:10:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827191+0000", 2024-09-16T00:10:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2330000000000001e-06 2024-09-16T00:10:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827195+0000", 2024-09-16T00:10:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8750000000000002e-06 2024-09-16T00:10:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827206+0000", 2024-09-16T00:10:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0473e-05 2024-09-16T00:10:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827213+0000", 2024-09-16T00:10:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:10:07.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827275+0000", 2024-09-16T00:10:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2532000000000004e-05 2024-09-16T00:10:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827392+0000", 2024-09-16T00:10:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116266 2024-09-16T00:10:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.836599+0000", 2024-09-16T00:10:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092068069999999991 2024-09-16T00:10:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5245 3.26 3:64562175:::100000032f3.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.827707+0000", 2024-09-16T00:10:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.470857906, 2024-09-16T00:10:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090401860000000004, 2024-09-16T00:10:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827707+0000", 2024-09-16T00:10:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827707+0000", 2024-09-16T00:10:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827708+0000", 2024-09-16T00:10:07.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5199999999999996e-07 2024-09-16T00:10:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827712+0000", 2024-09-16T00:10:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9530000000000001e-06 2024-09-16T00:10:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827722+0000", 2024-09-16T00:10:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0524e-05 2024-09-16T00:10:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827736+0000", 2024-09-16T00:10:07.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3713999999999999e-05 2024-09-16T00:10:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827801+0000", 2024-09-16T00:10:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5580000000000006e-05 2024-09-16T00:10:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827924+0000", 2024-09-16T00:10:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012308499999999999 2024-09-16T00:10:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.836747+0000", 2024-09-16T00:10:07.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088226780000000005 2024-09-16T00:10:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5251 2.17 2:e817bfdf:::100000037ad.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.827758+0000", 2024-09-16T00:10:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.470807282, 2024-09-16T00:10:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090250260000000002, 2024-09-16T00:10:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827758+0000", 2024-09-16T00:10:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827758+0000", 2024-09-16T00:10:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827758+0000", 2024-09-16T00:10:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000003e-07 2024-09-16T00:10:07.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827761+0000", 2024-09-16T00:10:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.095e-06 2024-09-16T00:10:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827772+0000", 2024-09-16T00:10:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0767e-05 2024-09-16T00:10:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827779+0000", 2024-09-16T00:10:07.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-16T00:10:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827837+0000", 2024-09-16T00:10:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7615000000000003e-05 2024-09-16T00:10:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827942+0000", 2024-09-16T00:10:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105393 2024-09-16T00:10:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.836783+0000", 2024-09-16T00:10:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088403279999999997 2024-09-16T00:10:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5252 2.1a 2:5913b086:::1000000336d.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4704388530000001, 2024-09-16T00:10:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010086441999999999, 2024-09-16T00:10:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:07.327 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828127+0000", 2024-09-16T00:10:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:10:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828140+0000", 2024-09-16T00:10:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3614e-05 2024-09-16T00:10:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828149+0000", 2024-09-16T00:10:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7250000000000003e-06 2024-09-16T00:10:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828156+0000", 2024-09-16T00:10:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9430000000000003e-06 2024-09-16T00:10:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828274+0000", 2024-09-16T00:10:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118121 2024-09-16T00:10:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828381+0000", 2024-09-16T00:10:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107379 2024-09-16T00:10:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.838212+0000", 2024-09-16T00:10:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098310089999999999 2024-09-16T00:10:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5286 3.26 3:661d81ee:::100000034b1.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.835257+0000", 2024-09-16T00:10:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.463308176, 2024-09-16T00:10:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084025320000000008, 2024-09-16T00:10:07.331 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835257+0000", 2024-09-16T00:10:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835257+0000", 2024-09-16T00:10:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835257+0000", 2024-09-16T00:10:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4700000000000002e-07 2024-09-16T00:10:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835260+0000", 2024-09-16T00:10:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.593e-06 2024-09-16T00:10:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835267+0000", 2024-09-16T00:10:07.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4880000000000001e-06 2024-09-16T00:10:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835274+0000", 2024-09-16T00:10:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2920000000000001e-06 2024-09-16T00:10:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835358+0000", 2024-09-16T00:10:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4907999999999994e-05 2024-09-16T00:10:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835475+0000", 2024-09-16T00:10:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116267 2024-09-16T00:10:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.843659+0000", 2024-09-16T00:10:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081845370000000004 2024-09-16T00:10:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5296 3.10 3:08253ab3:::100000037b2.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.839046+0000", 2024-09-16T00:10:07.335 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4595190320000002, 2024-09-16T00:10:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086364030000000008, 2024-09-16T00:10:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839046+0000", 2024-09-16T00:10:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839046+0000", 2024-09-16T00:10:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839047+0000", 2024-09-16T00:10:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0299999999999998e-07 2024-09-16T00:10:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839050+0000", 2024-09-16T00:10:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4989999999999999e-06 2024-09-16T00:10:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839062+0000", 2024-09-16T00:10:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2128e-05 2024-09-16T00:10:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839646+0000", 2024-09-16T00:10:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058339999999999998 2024-09-16T00:10:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839707+0000", 2024-09-16T00:10:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1780999999999997e-05 2024-09-16T00:10:07.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839842+0000", 2024-09-16T00:10:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013437799999999999 2024-09-16T00:10:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.847682+0000", 2024-09-16T00:10:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0078404140000000004 2024-09-16T00:10:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.19 epoch 96/94 rep_tid 2313 entries 96'15 (0'0) error 2:9f9cad8a:::100000033d1.00000000:head by mds.0.4:5304 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 95'13)", 2024-09-16T00:10:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.841700+0000", 2024-09-16T00:10:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4568649330000003, 2024-09-16T00:10:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086308140000000005, 2024-09-16T00:10:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:10:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841700+0000", 2024-09-16T00:10:07.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841700+0000", 2024-09-16T00:10:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841701+0000", 2024-09-16T00:10:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4199999999999995e-07 2024-09-16T00:10:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841709+0000", 2024-09-16T00:10:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4060000000000005e-06 2024-09-16T00:10:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841728+0000", 2024-09-16T00:10:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8868999999999998e-05 2024-09-16T00:10:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841741+0000", 2024-09-16T00:10:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3283e-05 2024-09-16T00:10:07.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841834+0000", 2024-09-16T00:10:07.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2312999999999995e-05 2024-09-16T00:10:07.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.850331+0000", 2024-09-16T00:10:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084972009999999994 2024-09-16T00:10:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:5307 3.1e e96/94 3:7b9ea4ae:::10000003595.00000000:head v 96'234, mlcod=95'232)", 2024-09-16T00:10:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.841762+0000", 2024-09-16T00:10:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4568024150000003, 2024-09-16T00:10:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086486400000000008, 2024-09-16T00:10:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.344 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841762+0000", 2024-09-16T00:10:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841762+0000", 2024-09-16T00:10:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841763+0000", 2024-09-16T00:10:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7e-07 2024-09-16T00:10:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841767+0000", 2024-09-16T00:10:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2799999999999997e-06 2024-09-16T00:10:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841775+0000", 2024-09-16T00:10:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5349999999999999e-06 2024-09-16T00:10:07.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841780+0000", 2024-09-16T00:10:07.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0429999999999998e-06 2024-09-16T00:10:07.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841856+0000", 2024-09-16T00:10:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6532999999999994e-05 2024-09-16T00:10:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841874+0000", 2024-09-16T00:10:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8079000000000001e-05 2024-09-16T00:10:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:07.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.850370+0000", 2024-09-16T00:10:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084952169999999994 2024-09-16T00:10:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.850411+0000", 2024-09-16T00:10:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1582999999999998e-05 2024-09-16T00:10:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.350 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5528 3.a 3:50a2513a:::10000003108.00000001:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97)", 2024-09-16T00:10:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:03.812487+0000", 2024-09-16T00:10:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4860776200000001, 2024-09-16T00:10:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0085445009999999995, 2024-09-16T00:10:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812487+0000", 2024-09-16T00:10:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812487+0000", 2024-09-16T00:10:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812489+0000", 2024-09-16T00:10:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6589999999999999e-06 2024-09-16T00:10:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812495+0000", 2024-09-16T00:10:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3579999999999999e-06 2024-09-16T00:10:07.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812511+0000", 2024-09-16T00:10:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6011999999999998e-05 2024-09-16T00:10:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812521+0000", 2024-09-16T00:10:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3500000000000003e-06 2024-09-16T00:10:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812594+0000", 2024-09-16T00:10:07.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3594e-05 2024-09-16T00:10:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812677+0000", 2024-09-16T00:10:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2590999999999996e-05 2024-09-16T00:10:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812781+0000", 2024-09-16T00:10:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000104206 2024-09-16T00:10:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.818292+0000", 2024-09-16T00:10:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055108140000000002 2024-09-16T00:10:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.820928+0000", 2024-09-16T00:10:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026360200000000002 2024-09-16T00:10:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.820983+0000", 2024-09-16T00:10:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000003e-05 2024-09-16T00:10:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.821032+0000", 2024-09-16T00:10:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9147000000000002e-05 2024-09-16T00:10:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:5527 3.33 e97/94 3:cf859f74:::10000003108.00000000:head v 97'245, mlcod=96'243)", 2024-09-16T00:10:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:03.813310+0000", 2024-09-16T00:10:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.485254597, 2024-09-16T00:10:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081025200000000002, 2024-09-16T00:10:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.356 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.813310+0000", 2024-09-16T00:10:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.813310+0000", 2024-09-16T00:10:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.813311+0000", 2024-09-16T00:10:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1299999999999998e-07 2024-09-16T00:10:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.813317+0000", 2024-09-16T00:10:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8590000000000001e-06 2024-09-16T00:10:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.813327+0000", 2024-09-16T00:10:07.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9799999999999993e-06 2024-09-16T00:10:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.813333+0000", 2024-09-16T00:10:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0780000000000001e-06 2024-09-16T00:10:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.813385+0000", 2024-09-16T00:10:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2181000000000001e-05 2024-09-16T00:10:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.813404+0000", 2024-09-16T00:10:07.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8559999999999998e-05 2024-09-16T00:10:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.821362+0000", 2024-09-16T00:10:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007957904 2024-09-16T00:10:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.821413+0000", 2024-09-16T00:10:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1044999999999997e-05 2024-09-16T00:10:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.360 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5582 3.20 3:05175347:::100000032f5.00000001:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97)", 2024-09-16T00:10:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:03.824971+0000", 2024-09-16T00:10:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4735941449999999, 2024-09-16T00:10:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088623420000000005, 2024-09-16T00:10:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.824971+0000", 2024-09-16T00:10:07.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.824971+0000", 2024-09-16T00:10:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.824971+0000", 2024-09-16T00:10:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5899999999999996e-07 2024-09-16T00:10:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.824975+0000", 2024-09-16T00:10:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7780000000000001e-06 2024-09-16T00:10:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.824984+0000", 2024-09-16T00:10:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1209999999999996e-06 2024-09-16T00:10:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.824992+0000", 2024-09-16T00:10:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.469e-06 2024-09-16T00:10:07.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.825044+0000", 2024-09-16T00:10:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2544999999999999e-05 2024-09-16T00:10:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.825162+0000", 2024-09-16T00:10:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011764000000000001 2024-09-16T00:10:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.833833+0000", 2024-09-16T00:10:07.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086711300000000009 2024-09-16T00:10:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5736 3.35 3:ae174d34:::1000000354d.00000001:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97)", 2024-09-16T00:10:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:03.853877+0000", 2024-09-16T00:10:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.4446878490000001, 2024-09-16T00:10:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0077740020000000003, 2024-09-16T00:10:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.853877+0000", 2024-09-16T00:10:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.853877+0000", 2024-09-16T00:10:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.853878+0000", 2024-09-16T00:10:07.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3200000000000003e-07 2024-09-16T00:10:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.853882+0000", 2024-09-16T00:10:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2799999999999997e-06 2024-09-16T00:10:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.853897+0000", 2024-09-16T00:10:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4491000000000001e-05 2024-09-16T00:10:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.853904+0000", 2024-09-16T00:10:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5889999999999997e-06 2024-09-16T00:10:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.853957+0000", 2024-09-16T00:10:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2493e-05 2024-09-16T00:10:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854064+0000", 2024-09-16T00:10:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010674400000000001 2024-09-16T00:10:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.861651+0000", 2024-09-16T00:10:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007587473 2024-09-16T00:10:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5743 3.36 3:6c273be3:::1000000385e.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97)", 2024-09-16T00:10:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:03.854364+0000", 2024-09-16T00:10:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.44420101, 2024-09-16T00:10:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099844070000000007, 2024-09-16T00:10:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.369 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854364+0000", 2024-09-16T00:10:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854364+0000", 2024-09-16T00:10:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854365+0000", 2024-09-16T00:10:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3399999999999999e-07 2024-09-16T00:10:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854368+0000", 2024-09-16T00:10:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4300000000000002e-06 2024-09-16T00:10:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854376+0000", 2024-09-16T00:10:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2770000000000005e-06 2024-09-16T00:10:07.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854384+0000", 2024-09-16T00:10:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7200000000000006e-06 2024-09-16T00:10:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854455+0000", 2024-09-16T00:10:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0631999999999997e-05 2024-09-16T00:10:07.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854560+0000", 2024-09-16T00:10:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105338 2024-09-16T00:10:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.864348+0000", 2024-09-16T00:10:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097883759999999997 2024-09-16T00:10:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5745 2.17 2:eeb9ee5e:::1000000387c.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97)", 2024-09-16T00:10:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:03.855859+0000", 2024-09-16T00:10:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.442705798, 2024-09-16T00:10:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084735950000000004, 2024-09-16T00:10:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.855859+0000", 2024-09-16T00:10:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.855859+0000", 2024-09-16T00:10:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.855860+0000", 2024-09-16T00:10:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6799999999999999e-07 2024-09-16T00:10:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.855864+0000", 2024-09-16T00:10:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3080000000000001e-06 2024-09-16T00:10:07.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.855875+0000", 2024-09-16T00:10:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1062e-05 2024-09-16T00:10:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.855883+0000", 2024-09-16T00:10:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2790000000000001e-06 2024-09-16T00:10:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.855934+0000", 2024-09-16T00:10:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1208000000000002e-05 2024-09-16T00:10:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.856043+0000", 2024-09-16T00:10:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109529 2024-09-16T00:10:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.864333+0000", 2024-09-16T00:10:07.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082893410000000004 2024-09-16T00:10:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:6532 3.26 3:67aa8e4f:::100000031c5.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100)", 2024-09-16T00:10:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:06.849896+0000", 2024-09-16T00:10:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.448668607, 2024-09-16T00:10:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090082240000000004, 2024-09-16T00:10:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.849896+0000", 2024-09-16T00:10:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.849896+0000", 2024-09-16T00:10:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.849897+0000", 2024-09-16T00:10:07.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2900000000000002e-07 2024-09-16T00:10:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.849901+0000", 2024-09-16T00:10:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T00:10:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.849912+0000", 2024-09-16T00:10:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0546e-05 2024-09-16T00:10:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.849919+0000", 2024-09-16T00:10:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5179999999999998e-06 2024-09-16T00:10:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.854785+0000", 2024-09-16T00:10:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048656960000000001 2024-09-16T00:10:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.854914+0000", 2024-09-16T00:10:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012934599999999999 2024-09-16T00:10:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.858904+0000", 2024-09-16T00:10:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039901119999999997 2024-09-16T00:10:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:6601 3.35 3:af7d55b1:::10000003232.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100)", 2024-09-16T00:10:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:06.866600+0000", 2024-09-16T00:10:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.43196456599999999, 2024-09-16T00:10:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092432980000000005, 2024-09-16T00:10:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.382 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866600+0000", 2024-09-16T00:10:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866600+0000", 2024-09-16T00:10:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866602+0000", 2024-09-16T00:10:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.649e-06 2024-09-16T00:10:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866607+0000", 2024-09-16T00:10:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7330000000000003e-06 2024-09-16T00:10:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866618+0000", 2024-09-16T00:10:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1506e-05 2024-09-16T00:10:07.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866626+0000", 2024-09-16T00:10:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.498e-06 2024-09-16T00:10:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866685+0000", 2024-09-16T00:10:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8950000000000003e-05 2024-09-16T00:10:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866809+0000", 2024-09-16T00:10:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012389899999999999 2024-09-16T00:10:07.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875844+0000", 2024-09-16T00:10:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090350629999999994 2024-09-16T00:10:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:6632 3.20 3:04677f23:::100000034fe.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100)", 2024-09-16T00:10:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:06.875758+0000", 2024-09-16T00:10:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.42280640400000002, 2024-09-16T00:10:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0085444470000000002, 2024-09-16T00:10:07.386 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875758+0000", 2024-09-16T00:10:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875758+0000", 2024-09-16T00:10:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875759+0000", 2024-09-16T00:10:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8400000000000003e-07 2024-09-16T00:10:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875763+0000", 2024-09-16T00:10:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.427e-06 2024-09-16T00:10:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875779+0000", 2024-09-16T00:10:07.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5973000000000001e-05 2024-09-16T00:10:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875785+0000", 2024-09-16T00:10:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5660000000000003e-06 2024-09-16T00:10:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875840+0000", 2024-09-16T00:10:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4487000000000002e-05 2024-09-16T00:10:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875919+0000", 2024-09-16T00:10:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8693e-05 2024-09-16T00:10:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.876012+0000", 2024-09-16T00:10:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3966000000000001e-05 2024-09-16T00:10:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.877285+0000", 2024-09-16T00:10:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012726809999999999 2024-09-16T00:10:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.884204+0000", 2024-09-16T00:10:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069192910000000002 2024-09-16T00:10:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.884260+0000", 2024-09-16T00:10:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5995999999999997e-05 2024-09-16T00:10:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.884303+0000", 2024-09-16T00:10:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2483e-05 2024-09-16T00:10:07.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.392 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.392 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.392 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:6635 3.20 3:0697f031:::10000003500.00000002:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100)", 2024-09-16T00:10:07.392 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:06.875805+0000", 2024-09-16T00:10:07.392 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.42276021400000002, 2024-09-16T00:10:07.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086054329999999991, 2024-09-16T00:10:07.392 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:07.392 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:07.392 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.392 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:07.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:07.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875805+0000", 2024-09-16T00:10:07.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:07.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875805+0000", 2024-09-16T00:10:07.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:07.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:07.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875805+0000", 2024-09-16T00:10:07.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0900000000000002e-07 2024-09-16T00:10:07.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:07.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875808+0000", 2024-09-16T00:10:07.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6000000000000001e-06 2024-09-16T00:10:07.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:07.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875815+0000", 2024-09-16T00:10:07.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8689999999999998e-06 2024-09-16T00:10:07.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875820+0000", 2024-09-16T00:10:07.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2129999999999999e-06 2024-09-16T00:10:07.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:07.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.876345+0000", 2024-09-16T00:10:07.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052494100000000001 2024-09-16T00:10:07.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:07.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.876415+0000", 2024-09-16T00:10:07.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0358999999999995e-05 2024-09-16T00:10:07.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:07.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.876521+0000", 2024-09-16T00:10:07.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105889 2024-09-16T00:10:07.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:07.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.877890+0000", 2024-09-16T00:10:07.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013687009999999999 2024-09-16T00:10:07.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.884339+0000", 2024-09-16T00:10:07.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064493190000000002 2024-09-16T00:10:07.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:07.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.884380+0000", 2024-09-16T00:10:07.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0533e-05 2024-09-16T00:10:07.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:07.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:07.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:07.397 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.884410+0000", 2024-09-16T00:10:07.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0499999999999999e-05 2024-09-16T00:10:07.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.397 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:07.397 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:07.398 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:07.400 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:07.398+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:07.400 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:10:07.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.304+0000 7f8fabfff640 1 -- 172.21.15.139:0/1260091249 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v101) ==== 113+0+0 (secure 0 0 0) 0x7f8fa404b030 con 0x7f8fb4070c20 2024-09-16T00:10:07.401 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T00:10:07.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.306+0000 7f8fb9401640 1 -- 172.21.15.139:0/1260091249 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f9406c2b0 msgr2=0x7f8f9406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:07.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.306+0000 7f8fb9401640 1 --2- 172.21.15.139:0/1260091249 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f9406c2b0 0x7f8f9406e770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8fb41f29a0 tx=0x7f8f9c009240 comp rx=0 tx=0).stop 2024-09-16T00:10:07.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.306+0000 7f8fb9401640 1 -- 172.21.15.139:0/1260091249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb4070c20 msgr2=0x7f8fb40cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:07.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.307+0000 7f8fb9401640 1 --2- 172.21.15.139:0/1260091249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb4070c20 0x7f8fb40cc020 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8fa402a1c0 tx=0x7f8fa40046c0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.307+0000 7f8fb9401640 1 -- 172.21.15.139:0/1260091249 shutdown_connections 2024-09-16T00:10:07.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.307+0000 7f8fb9401640 1 --2- 172.21.15.139:0/1260091249 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8f9406c2b0 0x7f8f9406e770 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.307+0000 7f8fb9401640 1 --2- 172.21.15.139:0/1260091249 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8fb41518e0 0x7f8fb41536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.307+0000 7f8fb9401640 1 --2- 172.21.15.139:0/1260091249 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8fb40c87d0 0x7f8fb41531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.307+0000 7f8fb9401640 1 --2- 172.21.15.139:0/1260091249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb4070c20 0x7f8fb40cc020 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.307+0000 7f8fb9401640 1 -- 172.21.15.139:0/1260091249 >> 172.21.15.139:0/1260091249 conn(0x7f8fb406ff70 msgr2=0x7f8fb414f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:07.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.307+0000 7f8fb9401640 1 -- 172.21.15.139:0/1260091249 shutdown_connections 2024-09-16T00:10:07.402 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.307+0000 7f8fb9401640 1 -- 172.21.15.139:0/1260091249 wait complete. 2024-09-16T00:10:07.402 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T00:10:07.500 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:07.500+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:07.550 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:07.550 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:07.550 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:07.550 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:07.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.550+0000 7f42de76d640 1 Processor -- start 2024-09-16T00:10:07.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.550+0000 7f42de76d640 1 -- start start 2024-09-16T00:10:07.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.551+0000 7f42de76d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f42d8154c20 0x7f42d815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:07.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.551+0000 7f42de76d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d80cc4b0 con 0x7f42d8068060 2024-09-16T00:10:07.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.551+0000 7f42de76d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d80cc790 con 0x7f42d80ccc90 2024-09-16T00:10:07.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.551+0000 7f42de76d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f42d80c5830 con 0x7f42d8154c20 2024-09-16T00:10:07.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.551+0000 7f42cf7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f42d8068060 0x7f42d80c5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45704/0 (socket says 172.21.15.139:45704) 2024-09-16T00:10:07.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.551+0000 7f42cf7fe640 1 -- 172.21.15.139:0/2980142860 learned_addr learned my addr 172.21.15.139:0/2980142860 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:07.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.551+0000 7f42d7fff640 1 --2- 172.21.15.139:0/2980142860 >> v2:172.21.15.164:3300/0 conn(0x7f42d8154c20 0x7f42d815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:07.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.551+0000 7f42d77fe640 1 -- 172.21.15.139:0/2980142860 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2127238958 0 0) 0x7f42d80cc4b0 con 0x7f42d8068060 2024-09-16T00:10:07.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.551+0000 7f42d77fe640 1 -- 172.21.15.139:0/2980142860 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42c4003520 con 0x7f42d8068060 2024-09-16T00:10:07.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.551+0000 7f42d77fe640 1 -- 172.21.15.139:0/2980142860 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2892343246 0 0) 0x7f42d80cc790 con 0x7f42d80ccc90 2024-09-16T00:10:07.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.551+0000 7f42d77fe640 1 -- 172.21.15.139:0/2980142860 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42d80cc4b0 con 0x7f42d80ccc90 2024-09-16T00:10:07.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.552+0000 7f42d7fff640 1 -- 172.21.15.139:0/2980142860 >> v1:172.21.15.164:6789/0 conn(0x7f42d80ccc90 legacy=0x7f42d8165180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:07.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.552+0000 7f42d7fff640 1 -- 172.21.15.139:0/2980142860 >> v1:172.21.15.139:6789/0 conn(0x7f42d8068060 legacy=0x7f42d80c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:07.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.552+0000 7f42d7fff640 1 -- 172.21.15.139:0/2980142860 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d815ddf0 con 0x7f42d8154c20 2024-09-16T00:10:07.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.552+0000 7f42d7fff640 1 --2- 172.21.15.139:0/2980142860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d8154c20 0x7f42d815b280 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f42c80097a0 tx=0x7f42c802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cce6d53959f0c7d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:07.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.553+0000 7f42d77fe640 1 -- 172.21.15.139:0/2980142860 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8037070 con 0x7f42d8154c20 2024-09-16T00:10:07.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.553+0000 7f42d77fe640 1 -- 172.21.15.139:0/2980142860 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c8032440 con 0x7f42d8154c20 2024-09-16T00:10:07.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.553+0000 7f42d77fe640 1 -- 172.21.15.139:0/2980142860 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c80327f0 con 0x7f42d8154c20 2024-09-16T00:10:07.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.554+0000 7f42de76d640 1 -- 172.21.15.139:0/2980142860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d8154c20 msgr2=0x7f42d815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:07.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.554+0000 7f42de76d640 1 --2- 172.21.15.139:0/2980142860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d8154c20 0x7f42d815b280 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f42c80097a0 tx=0x7f42c802a230 comp rx=0 tx=0).stop 2024-09-16T00:10:07.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.554+0000 7f42de76d640 1 -- 172.21.15.139:0/2980142860 shutdown_connections 2024-09-16T00:10:07.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.554+0000 7f42de76d640 1 --2- 172.21.15.139:0/2980142860 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d8154c20 0x7f42d815b280 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.554+0000 7f42de76d640 1 -- 172.21.15.139:0/2980142860 >> 172.21.15.139:0/2980142860 conn(0x7f42d806ca40 msgr2=0x7f42d8067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:07.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.554+0000 7f42de76d640 1 -- 172.21.15.139:0/2980142860 shutdown_connections 2024-09-16T00:10:07.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.554+0000 7f42de76d640 1 -- 172.21.15.139:0/2980142860 wait complete. 2024-09-16T00:10:07.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.556+0000 7f42de76d640 1 Processor -- start 2024-09-16T00:10:07.556 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.556+0000 7f42de76d640 1 -- start start 2024-09-16T00:10:07.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.557+0000 7f42de76d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42d8068060 0x7f42d81f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:07.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.557+0000 7f42de76d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42d80ccc90 0x7f42d80cf450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:07.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.557+0000 7f42de76d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d8154c20 0x7f42d815b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:07.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.557+0000 7f42de76d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42d80cc260 con 0x7f42d8068060 2024-09-16T00:10:07.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.557+0000 7f42de76d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f42d815e3b0 con 0x7f42d8154c20 2024-09-16T00:10:07.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.557+0000 7f42de76d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f42d815e690 con 0x7f42d80ccc90 2024-09-16T00:10:07.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.557+0000 7f42d7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42d8068060 0x7f42d81f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:07.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.557+0000 7f42d7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42d8068060 0x7f42d81f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49136/0 (socket says 172.21.15.139:49136) 2024-09-16T00:10:07.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.557+0000 7f42d7fff640 1 -- 172.21.15.139:0/1622126197 learned_addr learned my addr 172.21.15.139:0/1622126197 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:07.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.557+0000 7f42dcd0a640 1 --2- 172.21.15.139:0/1622126197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d8154c20 0x7f42d815b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:07.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.557+0000 7f42cf7fe640 1 --2- 172.21.15.139:0/1622126197 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42d80ccc90 0x7f42d80cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:07.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.557+0000 7f42d7fff640 1 -- 172.21.15.139:0/1622126197 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42d80ccc90 msgr2=0x7f42d80cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:07.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.557+0000 7f42d7fff640 1 --2- 172.21.15.139:0/1622126197 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42d80ccc90 0x7f42d80cf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.557+0000 7f42d7fff640 1 -- 172.21.15.139:0/1622126197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d8154c20 msgr2=0x7f42d815b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:07.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.557+0000 7f42d7fff640 1 --2- 172.21.15.139:0/1622126197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d8154c20 0x7f42d815b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:07.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.557+0000 7f42d7fff640 1 -- 172.21.15.139:0/1622126197 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d80b4ac0 con 0x7f42d8068060 2024-09-16T00:10:07.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.558+0000 7f42d7fff640 1 --2- 172.21.15.139:0/1622126197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42d8068060 0x7f42d81f4b00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f42c802a270 tx=0x7f42c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:07.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.558+0000 7f42d57fa640 1 -- 172.21.15.139:0/1622126197 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8041030 con 0x7f42d8068060 2024-09-16T00:10:07.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.558+0000 7f42d57fa640 1 -- 172.21.15.139:0/1622126197 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42c803d4e0 con 0x7f42d8068060 2024-09-16T00:10:07.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.558+0000 7f42d57fa640 1 -- 172.21.15.139:0/1622126197 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8037040 con 0x7f42d8068060 2024-09-16T00:10:07.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.558+0000 7f42de76d640 1 -- 172.21.15.139:0/1622126197 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d81f0d90 con 0x7f42d8068060 2024-09-16T00:10:07.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.558+0000 7f42de76d640 1 -- 172.21.15.139:0/1622126197 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d81f1350 con 0x7f42d8068060 2024-09-16T00:10:07.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.559+0000 7f42de76d640 1 -- 172.21.15.139:0/1622126197 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d815b780 con 0x7f42d8068060 2024-09-16T00:10:07.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.559+0000 7f42d57fa640 1 -- 172.21.15.139:0/1622126197 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f42c8044070 con 0x7f42d8068060 2024-09-16T00:10:07.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.560+0000 7f42d57fa640 1 --2- 172.21.15.139:0/1622126197 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f42b806c2b0 0x7f42b806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:07.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.560+0000 7f42d57fa640 1 -- 172.21.15.139:0/1622126197 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5973+0+0 (secure 0 0 0) 0x7f42c80abb60 con 0x7f42d8068060 2024-09-16T00:10:07.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.560+0000 7f42cf7fe640 1 --2- 172.21.15.139:0/1622126197 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f42b806c2b0 0x7f42b806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:07.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.561+0000 7f42cf7fe640 1 --2- 172.21.15.139:0/1622126197 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f42b806c2b0 0x7f42b806e770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f42c0007fd0 tx=0x7f42c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:07.563 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:10:07.565 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.565+0000 7f42d57fa640 1 -- 172.21.15.139:0/1622126197 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f42c8079af0 con 0x7f42d8068060 2024-09-16T00:10:07.606 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:07.606+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:07.673 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:07.673 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:07.673 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:07.673 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:07.673 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:07.682 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:10:07.724 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:07.723+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:07.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:07.772+0000 7f42de76d640 1 -- 172.21.15.139:0/1622126197 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) -- 0x7f42d80686b0 con 0x7f42d8068060 2024-09-16T00:10:07.803 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:07.802+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:07.837 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:07.837 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:07.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:07.837 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:07.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.838 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:219 2.1f 2:f83d65d8:::200.00000004:head [write 863658~2915703 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e65)", 2024-09-16T00:10:07.838 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:10:07.838 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 56.963877496999999, 2024-09-16T00:10:07.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013772627000000001, 2024-09-16T00:10:07.838 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.838 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.838 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.838 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:10:07.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:10:07.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870941+0000", 2024-09-16T00:10:07.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4e-07 2024-09-16T00:10:07.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877134+0000", 2024-09-16T00:10:07.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061936769999999999 2024-09-16T00:10:07.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877153+0000", 2024-09-16T00:10:07.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8408000000000001e-05 2024-09-16T00:10:07.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877161+0000", 2024-09-16T00:10:07.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2709999999999992e-06 2024-09-16T00:10:07.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877225+0000", 2024-09-16T00:10:07.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4190999999999996e-05 2024-09-16T00:10:07.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877289+0000", 2024-09-16T00:10:07.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4058000000000002e-05 2024-09-16T00:10:07.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:07.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877376+0000", 2024-09-16T00:10:07.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7227999999999999e-05 2024-09-16T00:10:07.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.882925+0000", 2024-09-16T00:10:07.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055481050000000002 2024-09-16T00:10:07.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884635+0000", 2024-09-16T00:10:07.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017102759999999999 2024-09-16T00:10:07.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884674+0000", 2024-09-16T00:10:07.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8914e-05 2024-09-16T00:10:07.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884712+0000", 2024-09-16T00:10:07.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658999999999997e-05 2024-09-16T00:10:07.843 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.843 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.843 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.843 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10767 3.1b 3:dab1607f:::100000030fd.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:07.843 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.379273+0000", 2024-09-16T00:10:07.843 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 29.455544764999999, 2024-09-16T00:10:07.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012051947, 2024-09-16T00:10:07.843 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.843 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.843 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.844 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.844 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.844 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.844 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10767 2024-09-16T00:10:07.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.844 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.379273+0000", 2024-09-16T00:10:07.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.379273+0000", 2024-09-16T00:10:07.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.379275+0000", 2024-09-16T00:10:07.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6230000000000001e-06 2024-09-16T00:10:07.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381583+0000", 2024-09-16T00:10:07.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023079939999999998 2024-09-16T00:10:07.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381617+0000", 2024-09-16T00:10:07.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4227999999999998e-05 2024-09-16T00:10:07.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381633+0000", 2024-09-16T00:10:07.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5642e-05 2024-09-16T00:10:07.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381739+0000", 2024-09-16T00:10:07.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010638500000000001 2024-09-16T00:10:07.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381850+0000", 2024-09-16T00:10:07.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110164 2024-09-16T00:10:07.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:07.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.382033+0000", 2024-09-16T00:10:07.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018355699999999999 2024-09-16T00:10:07.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.388313+0000", 2024-09-16T00:10:07.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006279588 2024-09-16T00:10:07.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.391197+0000", 2024-09-16T00:10:07.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028841130000000002 2024-09-16T00:10:07.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.391263+0000", 2024-09-16T00:10:07.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6105000000000005e-05 2024-09-16T00:10:07.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.391324+0000", 2024-09-16T00:10:07.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1548000000000001e-05 2024-09-16T00:10:07.849 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.849 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.849 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.850 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10768 3.2b 3:d738455c:::100000030db.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:07.851 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:10:07.853 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 29.440692324, 2024-09-16T00:10:07.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012411195999999999, 2024-09-16T00:10:07.854 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.854 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.854 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.854 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.854 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.854 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.854 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10768 2024-09-16T00:10:07.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.854 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:10:07.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:10:07.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394129+0000", 2024-09-16T00:10:07.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4100000000000001e-06 2024-09-16T00:10:07.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395723+0000", 2024-09-16T00:10:07.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015938409999999999 2024-09-16T00:10:07.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395745+0000", 2024-09-16T00:10:07.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1611e-05 2024-09-16T00:10:07.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395782+0000", 2024-09-16T00:10:07.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7292000000000001e-05 2024-09-16T00:10:07.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395856+0000", 2024-09-16T00:10:07.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3675000000000002e-05 2024-09-16T00:10:07.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395924+0000", 2024-09-16T00:10:07.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8646999999999995e-05 2024-09-16T00:10:07.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:07.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.396069+0000", 2024-09-16T00:10:07.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014455199999999999 2024-09-16T00:10:07.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.403755+0000", 2024-09-16T00:10:07.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0076863160000000003 2024-09-16T00:10:07.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406407+0000", 2024-09-16T00:10:07.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026516220000000002 2024-09-16T00:10:07.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406477+0000", 2024-09-16T00:10:07.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0204999999999996e-05 2024-09-16T00:10:07.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406536+0000", 2024-09-16T00:10:07.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9024999999999998e-05 2024-09-16T00:10:07.859 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.859 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.859 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.860 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10941 3.1 3:8246fad2:::10000003173.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:07.860 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:10:07.860 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 28.378938987000002, 2024-09-16T00:10:07.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01286782, 2024-09-16T00:10:07.860 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.860 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.860 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.860 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.861 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.861 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.861 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10941 2024-09-16T00:10:07.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.861 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:10:07.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:10:07.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.864 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.864 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455881+0000", 2024-09-16T00:10:07.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7269999999999999e-06 2024-09-16T00:10:07.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457462+0000", 2024-09-16T00:10:07.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015811690000000001 2024-09-16T00:10:07.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457487+0000", 2024-09-16T00:10:07.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4607e-05 2024-09-16T00:10:07.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457497+0000", 2024-09-16T00:10:07.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9939999999999995e-06 2024-09-16T00:10:07.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457571+0000", 2024-09-16T00:10:07.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3867000000000003e-05 2024-09-16T00:10:07.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457689+0000", 2024-09-16T00:10:07.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000118814 2024-09-16T00:10:07.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:07.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457804+0000", 2024-09-16T00:10:07.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114702 2024-09-16T00:10:07.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.463422+0000", 2024-09-16T00:10:07.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0056179239999999998 2024-09-16T00:10:07.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468641+0000", 2024-09-16T00:10:07.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052192109999999996 2024-09-16T00:10:07.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468703+0000", 2024-09-16T00:10:07.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1268999999999999e-05 2024-09-16T00:10:07.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468746+0000", 2024-09-16T00:10:07.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:10:07.871 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.871 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.872 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.872 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11742 3.1b 3:d847de26:::100000032cd.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:07.872 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:10:07.872 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 24.778447402000001, 2024-09-16T00:10:07.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012791903, 2024-09-16T00:10:07.872 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.872 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.872 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.872 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.872 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.872 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.873 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11742 2024-09-16T00:10:07.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.873 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:10:07.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:10:07.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056373+0000", 2024-09-16T00:10:07.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9799999999999998e-06 2024-09-16T00:10:07.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058604+0000", 2024-09-16T00:10:07.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022316020000000001 2024-09-16T00:10:07.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058630+0000", 2024-09-16T00:10:07.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5425000000000001e-05 2024-09-16T00:10:07.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058643+0000", 2024-09-16T00:10:07.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3361e-05 2024-09-16T00:10:07.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058731+0000", 2024-09-16T00:10:07.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7270999999999998e-05 2024-09-16T00:10:07.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.876 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.876 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058894+0000", 2024-09-16T00:10:07.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016376800000000001 2024-09-16T00:10:07.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.876 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:07.876 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.059050+0000", 2024-09-16T00:10:07.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015603700000000001 2024-09-16T00:10:07.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.876 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.877 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.066364+0000", 2024-09-16T00:10:07.877 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073138639999999998 2024-09-16T00:10:07.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.877 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.877 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069025+0000", 2024-09-16T00:10:07.877 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026607660000000002 2024-09-16T00:10:07.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.877 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.877 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069083+0000", 2024-09-16T00:10:07.877 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7680999999999999e-05 2024-09-16T00:10:07.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069162+0000", 2024-09-16T00:10:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9147999999999995e-05 2024-09-16T00:10:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11934 3.1b 3:d94fa8da:::10000003190.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:10:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 24.146404127, 2024-09-16T00:10:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.017110325999999999, 2024-09-16T00:10:07.878 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11934 2024-09-16T00:10:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:10:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:10:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688416+0000", 2024-09-16T00:10:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.875e-06 2024-09-16T00:10:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689633+0000", 2024-09-16T00:10:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001216997 2024-09-16T00:10:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689655+0000", 2024-09-16T00:10:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2285000000000001e-05 2024-09-16T00:10:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689665+0000", 2024-09-16T00:10:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7089999999999997e-06 2024-09-16T00:10:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689739+0000", 2024-09-16T00:10:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3973999999999993e-05 2024-09-16T00:10:07.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689898+0000", 2024-09-16T00:10:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015941100000000001 2024-09-16T00:10:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.690062+0000", 2024-09-16T00:10:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000163633 2024-09-16T00:10:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.695224+0000", 2024-09-16T00:10:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051620210000000001 2024-09-16T00:10:07.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705434+0000", 2024-09-16T00:10:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010209701 2024-09-16T00:10:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705488+0000", 2024-09-16T00:10:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3974000000000002e-05 2024-09-16T00:10:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705523+0000", 2024-09-16T00:10:07.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5746e-05 2024-09-16T00:10:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11941 3.12 3:4a0e6682:::1000000334b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:10:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 24.111386614000001, 2024-09-16T00:10:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01458448, 2024-09-16T00:10:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.884 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11941 2024-09-16T00:10:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:10:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:10:07.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723433+0000", 2024-09-16T00:10:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3410000000000001e-06 2024-09-16T00:10:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724966+0000", 2024-09-16T00:10:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001533242 2024-09-16T00:10:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724983+0000", 2024-09-16T00:10:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7156999999999999e-05 2024-09-16T00:10:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724995+0000", 2024-09-16T00:10:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2089999999999999e-05 2024-09-16T00:10:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725062+0000", 2024-09-16T00:10:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6026999999999997e-05 2024-09-16T00:10:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725159+0000", 2024-09-16T00:10:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7374000000000007e-05 2024-09-16T00:10:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725289+0000", 2024-09-16T00:10:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012988 2024-09-16T00:10:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.731944+0000", 2024-09-16T00:10:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066547439999999998 2024-09-16T00:10:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.737956+0000", 2024-09-16T00:10:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0060121630000000001 2024-09-16T00:10:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.737985+0000", 2024-09-16T00:10:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9352999999999998e-05 2024-09-16T00:10:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.738015+0000", 2024-09-16T00:10:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0108999999999999e-05 2024-09-16T00:10:07.889 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11954 3.6 3:625e0e90:::100000032ef.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:10:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 24.066935047000001, 2024-09-16T00:10:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012484808999999999, 2024-09-16T00:10:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.890 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11954 2024-09-16T00:10:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:10:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:10:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767886+0000", 2024-09-16T00:10:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6650000000000001e-06 2024-09-16T00:10:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770082+0000", 2024-09-16T00:10:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021964979999999999 2024-09-16T00:10:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770104+0000", 2024-09-16T00:10:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1671999999999999e-05 2024-09-16T00:10:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770114+0000", 2024-09-16T00:10:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0336000000000001e-05 2024-09-16T00:10:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770197+0000", 2024-09-16T00:10:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2997000000000005e-05 2024-09-16T00:10:07.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770324+0000", 2024-09-16T00:10:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012680100000000001 2024-09-16T00:10:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770497+0000", 2024-09-16T00:10:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000172776 2024-09-16T00:10:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.776330+0000", 2024-09-16T00:10:07.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058330939999999996 2024-09-16T00:10:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780252+0000", 2024-09-16T00:10:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039215689999999997 2024-09-16T00:10:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780317+0000", 2024-09-16T00:10:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4948000000000002e-05 2024-09-16T00:10:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780367+0000", 2024-09-16T00:10:07.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0453e-05 2024-09-16T00:10:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12198 3.12 3:4a0e6682:::1000000334b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.683089+0000", 2024-09-16T00:10:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.151727996999998, 2024-09-16T00:10:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011990244000000001, 2024-09-16T00:10:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.896 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12198 2024-09-16T00:10:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.683089+0000", 2024-09-16T00:10:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.683089+0000", 2024-09-16T00:10:07.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.683092+0000", 2024-09-16T00:10:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.931e-06 2024-09-16T00:10:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684715+0000", 2024-09-16T00:10:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00162269 2024-09-16T00:10:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684734+0000", 2024-09-16T00:10:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9545999999999999e-05 2024-09-16T00:10:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684745+0000", 2024-09-16T00:10:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0131e-05 2024-09-16T00:10:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684814+0000", 2024-09-16T00:10:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9740000000000007e-05 2024-09-16T00:10:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684871+0000", 2024-09-16T00:10:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6215999999999998e-05 2024-09-16T00:10:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684980+0000", 2024-09-16T00:10:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109713 2024-09-16T00:10:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.690215+0000", 2024-09-16T00:10:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052349049999999998 2024-09-16T00:10:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.694970+0000", 2024-09-16T00:10:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047552649999999998 2024-09-16T00:10:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.695030+0000", 2024-09-16T00:10:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9086999999999999e-05 2024-09-16T00:10:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.695080+0000", 2024-09-16T00:10:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0019999999999999e-05 2024-09-16T00:10:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13263 3.1b 3:d9746e52:::1000000361c.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.785956+0000", 2024-09-16T00:10:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 20.048861057, 2024-09-16T00:10:07.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011930029, 2024-09-16T00:10:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13263 2024-09-16T00:10:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.785956+0000", 2024-09-16T00:10:07.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.785956+0000", 2024-09-16T00:10:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.785959+0000", 2024-09-16T00:10:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6089999999999999e-06 2024-09-16T00:10:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787198+0000", 2024-09-16T00:10:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012395889999999999 2024-09-16T00:10:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787225+0000", 2024-09-16T00:10:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6101999999999998e-05 2024-09-16T00:10:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.905 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787239+0000", 2024-09-16T00:10:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4384e-05 2024-09-16T00:10:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787329+0000", 2024-09-16T00:10:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0555000000000003e-05 2024-09-16T00:10:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787503+0000", 2024-09-16T00:10:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017311599999999999 2024-09-16T00:10:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787676+0000", 2024-09-16T00:10:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017352900000000001 2024-09-16T00:10:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.793377+0000", 2024-09-16T00:10:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057008409999999999 2024-09-16T00:10:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.797765+0000", 2024-09-16T00:10:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004388042 2024-09-16T00:10:07.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.797831+0000", 2024-09-16T00:10:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.614e-05 2024-09-16T00:10:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.797886+0000", 2024-09-16T00:10:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5121999999999998e-05 2024-09-16T00:10:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13618 3.1 3:80dd39f3:::1000000336c.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.108808+0000", 2024-09-16T00:10:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.726009217000001, 2024-09-16T00:10:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013244522, 2024-09-16T00:10:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13618 2024-09-16T00:10:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.909 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.108808+0000", 2024-09-16T00:10:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.108808+0000", 2024-09-16T00:10:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.108812+0000", 2024-09-16T00:10:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4640000000000002e-06 2024-09-16T00:10:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.110957+0000", 2024-09-16T00:10:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002145185 2024-09-16T00:10:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.110974+0000", 2024-09-16T00:10:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7108999999999998e-05 2024-09-16T00:10:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.110982+0000", 2024-09-16T00:10:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6330000000000002e-06 2024-09-16T00:10:07.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.111130+0000", 2024-09-16T00:10:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014738899999999999 2024-09-16T00:10:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.111299+0000", 2024-09-16T00:10:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016908400000000001 2024-09-16T00:10:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.111470+0000", 2024-09-16T00:10:07.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170667 2024-09-16T00:10:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.117141+0000", 2024-09-16T00:10:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005670993 2024-09-16T00:10:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.121945+0000", 2024-09-16T00:10:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0048042370000000003 2024-09-16T00:10:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.121998+0000", 2024-09-16T00:10:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2802e-05 2024-09-16T00:10:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.122053+0000", 2024-09-16T00:10:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4959e-05 2024-09-16T00:10:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.915 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13628 3.12 3:491d0769:::100000031da.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.134341+0000", 2024-09-16T00:10:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.700475916999999, 2024-09-16T00:10:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011817984, 2024-09-16T00:10:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13628 2024-09-16T00:10:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.134341+0000", 2024-09-16T00:10:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.134341+0000", 2024-09-16T00:10:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.134344+0000", 2024-09-16T00:10:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.785e-06 2024-09-16T00:10:07.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.135856+0000", 2024-09-16T00:10:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015116610000000001 2024-09-16T00:10:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.135878+0000", 2024-09-16T00:10:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2603999999999999e-05 2024-09-16T00:10:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.135890+0000", 2024-09-16T00:10:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1372000000000001e-05 2024-09-16T00:10:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.135958+0000", 2024-09-16T00:10:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7792000000000004e-05 2024-09-16T00:10:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.136023+0000", 2024-09-16T00:10:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5239000000000003e-05 2024-09-16T00:10:07.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.136147+0000", 2024-09-16T00:10:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012371600000000001 2024-09-16T00:10:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.141510+0000", 2024-09-16T00:10:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053634800000000003 2024-09-16T00:10:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.146037+0000", 2024-09-16T00:10:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045272130000000004 2024-09-16T00:10:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.146105+0000", 2024-09-16T00:10:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8038000000000001e-05 2024-09-16T00:10:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.146159+0000", 2024-09-16T00:10:07.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4083999999999999e-05 2024-09-16T00:10:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13687 3.32 3:4e600466:::1000000377a.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.340141+0000", 2024-09-16T00:10:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.494676294000001, 2024-09-16T00:10:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012781707999999999, 2024-09-16T00:10:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.922 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13687 2024-09-16T00:10:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.340141+0000", 2024-09-16T00:10:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.340141+0000", 2024-09-16T00:10:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.340143+0000", 2024-09-16T00:10:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9520000000000001e-06 2024-09-16T00:10:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343159+0000", 2024-09-16T00:10:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030156290000000001 2024-09-16T00:10:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343178+0000", 2024-09-16T00:10:07.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9219999999999999e-05 2024-09-16T00:10:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343212+0000", 2024-09-16T00:10:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3745999999999998e-05 2024-09-16T00:10:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343293+0000", 2024-09-16T00:10:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1074000000000003e-05 2024-09-16T00:10:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343430+0000", 2024-09-16T00:10:07.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013718700000000001 2024-09-16T00:10:07.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:07.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343589+0000", 2024-09-16T00:10:07.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000158806 2024-09-16T00:10:07.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.352468+0000", 2024-09-16T00:10:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0088798209999999995 2024-09-16T00:10:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.352835+0000", 2024-09-16T00:10:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00036653400000000001 2024-09-16T00:10:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.352884+0000", 2024-09-16T00:10:07.927 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8881000000000002e-05 2024-09-16T00:10:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.352923+0000", 2024-09-16T00:10:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8858e-05 2024-09-16T00:10:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.928 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13722 3.6 3:639728a8:::100000030cc.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.442783+0000", 2024-09-16T00:10:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.392034451000001, 2024-09-16T00:10:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013776286, 2024-09-16T00:10:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.929 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13722 2024-09-16T00:10:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.442783+0000", 2024-09-16T00:10:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.442783+0000", 2024-09-16T00:10:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.442785+0000", 2024-09-16T00:10:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9369999999999998e-06 2024-09-16T00:10:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444337+0000", 2024-09-16T00:10:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015527169999999999 2024-09-16T00:10:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444362+0000", 2024-09-16T00:10:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4513999999999999e-05 2024-09-16T00:10:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444375+0000", 2024-09-16T00:10:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3052e-05 2024-09-16T00:10:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444451+0000", 2024-09-16T00:10:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5814000000000003e-05 2024-09-16T00:10:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444566+0000", 2024-09-16T00:10:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011508199999999999 2024-09-16T00:10:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444727+0000", 2024-09-16T00:10:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000160743 2024-09-16T00:10:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.451504+0000", 2024-09-16T00:10:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006777773 2024-09-16T00:10:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.456446+0000", 2024-09-16T00:10:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004941834 2024-09-16T00:10:07.934 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.456510+0000", 2024-09-16T00:10:07.935 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3880999999999994e-05 2024-09-16T00:10:07.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.456559+0000", 2024-09-16T00:10:07.935 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8939e-05 2024-09-16T00:10:07.935 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.935 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13773 3.10 3:0a21c9f4:::100000034e7.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.578694+0000", 2024-09-16T00:10:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.256123516999999, 2024-09-16T00:10:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012227249000000001, 2024-09-16T00:10:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.936 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13773 2024-09-16T00:10:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.578694+0000", 2024-09-16T00:10:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.578694+0000", 2024-09-16T00:10:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.578698+0000", 2024-09-16T00:10:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8700000000000002e-06 2024-09-16T00:10:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.581979+0000", 2024-09-16T00:10:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003281455 2024-09-16T00:10:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.581994+0000", 2024-09-16T00:10:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4861000000000001e-05 2024-09-16T00:10:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.582003+0000", 2024-09-16T00:10:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0399999999999998e-06 2024-09-16T00:10:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.582070+0000", 2024-09-16T00:10:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7189999999999999e-05 2024-09-16T00:10:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.582158+0000", 2024-09-16T00:10:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7843000000000006e-05 2024-09-16T00:10:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.590796+0000", 2024-09-16T00:10:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0086376820000000007 2024-09-16T00:10:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.590842+0000", 2024-09-16T00:10:07.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6779000000000001e-05 2024-09-16T00:10:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.590921+0000", 2024-09-16T00:10:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8529000000000007e-05 2024-09-16T00:10:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13876 3.17 3:e847251e:::10000003522.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.796002+0000", 2024-09-16T00:10:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.038814943999999, 2024-09-16T00:10:07.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013015385000000001, 2024-09-16T00:10:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13876 2024-09-16T00:10:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.796002+0000", 2024-09-16T00:10:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.796002+0000", 2024-09-16T00:10:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.796004+0000", 2024-09-16T00:10:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1090000000000001e-06 2024-09-16T00:10:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797545+0000", 2024-09-16T00:10:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015402219999999999 2024-09-16T00:10:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797570+0000", 2024-09-16T00:10:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4905999999999998e-05 2024-09-16T00:10:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797580+0000", 2024-09-16T00:10:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0886e-05 2024-09-16T00:10:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797652+0000", 2024-09-16T00:10:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1050000000000006e-05 2024-09-16T00:10:07.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797814+0000", 2024-09-16T00:10:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000162578 2024-09-16T00:10:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797965+0000", 2024-09-16T00:10:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015092099999999999 2024-09-16T00:10:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.804256+0000", 2024-09-16T00:10:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0062911349999999998 2024-09-16T00:10:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.808943+0000", 2024-09-16T00:10:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0046868200000000004 2024-09-16T00:10:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.808985+0000", 2024-09-16T00:10:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1659000000000002e-05 2024-09-16T00:10:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.809018+0000", 2024-09-16T00:10:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3099000000000003e-05 2024-09-16T00:10:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13963 3.2b 3:d4ebc4d4:::100000034a5.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:50.005950+0000", 2024-09-16T00:10:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 17.828867710000001, 2024-09-16T00:10:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.062103750999999999, 2024-09-16T00:10:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.947 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13963 2024-09-16T00:10:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.005950+0000", 2024-09-16T00:10:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.005950+0000", 2024-09-16T00:10:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.005952+0000", 2024-09-16T00:10:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.255e-06 2024-09-16T00:10:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008027+0000", 2024-09-16T00:10:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002075019 2024-09-16T00:10:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008051+0000", 2024-09-16T00:10:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3906000000000001e-05 2024-09-16T00:10:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008063+0000", 2024-09-16T00:10:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2564e-05 2024-09-16T00:10:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008137+0000", 2024-09-16T00:10:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3946e-05 2024-09-16T00:10:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008234+0000", 2024-09-16T00:10:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6497999999999997e-05 2024-09-16T00:10:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008377+0000", 2024-09-16T00:10:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014313300000000001 2024-09-16T00:10:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.951 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.067062+0000", 2024-09-16T00:10:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058685043999999999 2024-09-16T00:10:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.067963+0000", 2024-09-16T00:10:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000901413 2024-09-16T00:10:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.068013+0000", 2024-09-16T00:10:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9997000000000003e-05 2024-09-16T00:10:07.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.068053+0000", 2024-09-16T00:10:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9975999999999998e-05 2024-09-16T00:10:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2683 2.8 e89/19 2:17051619:::200.0000000f:head v 89'470, mlcod=89'468)", 2024-09-16T00:10:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:50.011143+0000", 2024-09-16T00:10:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 17.823674060999998, 2024-09-16T00:10:07.953 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.056951953, 2024-09-16T00:10:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.011143+0000", 2024-09-16T00:10:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.011143+0000", 2024-09-16T00:10:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.011145+0000", 2024-09-16T00:10:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2869999999999999e-06 2024-09-16T00:10:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.011161+0000", 2024-09-16T00:10:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6920999999999999e-05 2024-09-16T00:10:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063382+0000", 2024-09-16T00:10:07.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.052221072 2024-09-16T00:10:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063394+0000", 2024-09-16T00:10:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1212999999999999e-05 2024-09-16T00:10:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063454+0000", 2024-09-16T00:10:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9852999999999998e-05 2024-09-16T00:10:07.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063477+0000", 2024-09-16T00:10:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3638999999999998e-05 2024-09-16T00:10:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.068064+0000", 2024-09-16T00:10:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045870599999999996 2024-09-16T00:10:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.068095+0000", 2024-09-16T00:10:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0908000000000003e-05 2024-09-16T00:10:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13967 3.32 3:4d814898:::10000003587.00000001:head [write 0~4194304 [2@0]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:50.020209+0000", 2024-09-16T00:10:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 17.814608382999999, 2024-09-16T00:10:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.050981571000000003, 2024-09-16T00:10:07.958 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13967 2024-09-16T00:10:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.020209+0000", 2024-09-16T00:10:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.020209+0000", 2024-09-16T00:10:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.020212+0000", 2024-09-16T00:10:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.802e-06 2024-09-16T00:10:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.021676+0000", 2024-09-16T00:10:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014645909999999999 2024-09-16T00:10:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063378+0000", 2024-09-16T00:10:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.041701657000000003 2024-09-16T00:10:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063395+0000", 2024-09-16T00:10:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7469000000000002e-05 2024-09-16T00:10:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063457+0000", 2024-09-16T00:10:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1450000000000003e-05 2024-09-16T00:10:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063621+0000", 2024-09-16T00:10:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000164026 2024-09-16T00:10:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063777+0000", 2024-09-16T00:10:07.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015628899999999999 2024-09-16T00:10:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.070303+0000", 2024-09-16T00:10:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0065262860000000001 2024-09-16T00:10:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.071105+0000", 2024-09-16T00:10:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000801423 2024-09-16T00:10:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.071154+0000", 2024-09-16T00:10:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8949000000000002e-05 2024-09-16T00:10:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.071190+0000", 2024-09-16T00:10:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6628999999999998e-05 2024-09-16T00:10:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14815 3.17 3:e9f3a865:::10000003294.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.505112+0000", 2024-09-16T00:10:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.329705435999999, 2024-09-16T00:10:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011798104, 2024-09-16T00:10:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14815 2024-09-16T00:10:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:07.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.505112+0000", 2024-09-16T00:10:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.505112+0000", 2024-09-16T00:10:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.505114+0000", 2024-09-16T00:10:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.99e-06 2024-09-16T00:10:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.507662+0000", 2024-09-16T00:10:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025480870000000001 2024-09-16T00:10:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.507692+0000", 2024-09-16T00:10:07.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9842999999999999e-05 2024-09-16T00:10:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.507705+0000", 2024-09-16T00:10:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3079e-05 2024-09-16T00:10:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.507817+0000", 2024-09-16T00:10:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011216499999999999 2024-09-16T00:10:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:07.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.507917+0000", 2024-09-16T00:10:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9686999999999996e-05 2024-09-16T00:10:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.508080+0000", 2024-09-16T00:10:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016345800000000001 2024-09-16T00:10:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.515396+0000", 2024-09-16T00:10:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073156189999999998 2024-09-16T00:10:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.516834+0000", 2024-09-16T00:10:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014385470000000001 2024-09-16T00:10:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.516875+0000", 2024-09-16T00:10:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0558e-05 2024-09-16T00:10:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.516910+0000", 2024-09-16T00:10:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5070999999999997e-05 2024-09-16T00:10:07.970 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.971 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.971 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.971 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:07.971 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:07.971 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:07.973 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:10:07.973 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:07.971+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:08.073 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:08.073+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:08.122 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:08.123 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:08.123 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:08.123 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:08.129 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:10:08.174 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:08.174+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:08.240 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:08.240 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:08.240 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:08.240 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:08.240 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:08.250 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:10:08.274 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:08.274+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:08.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.310+0000 7f42d57fa640 1 -- 172.21.15.139:0/1622126197 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v102) ==== 113+0+0 (secure 0 0 0) 0x7f42c807ddf0 con 0x7f42d8068060 2024-09-16T00:10:08.311 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T00:10:08.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.313+0000 7f42de76d640 1 -- 172.21.15.139:0/1622126197 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f42b806c2b0 msgr2=0x7f42b806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.313 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.313+0000 7f42de76d640 1 --2- 172.21.15.139:0/1622126197 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f42b806c2b0 0x7f42b806e770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f42c0007fd0 tx=0x7f42c0009040 comp rx=0 tx=0).stop 2024-09-16T00:10:08.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.313+0000 7f42de76d640 1 -- 172.21.15.139:0/1622126197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42d8068060 msgr2=0x7f42d81f4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.313+0000 7f42de76d640 1 --2- 172.21.15.139:0/1622126197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42d8068060 0x7f42d81f4b00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f42c802a270 tx=0x7f42c80046c0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.313+0000 7f42de76d640 1 -- 172.21.15.139:0/1622126197 shutdown_connections 2024-09-16T00:10:08.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.313+0000 7f42de76d640 1 --2- 172.21.15.139:0/1622126197 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f42b806c2b0 0x7f42b806e770 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.313+0000 7f42de76d640 1 --2- 172.21.15.139:0/1622126197 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d8154c20 0x7f42d815b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.313+0000 7f42de76d640 1 --2- 172.21.15.139:0/1622126197 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42d80ccc90 0x7f42d80cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.313+0000 7f42de76d640 1 --2- 172.21.15.139:0/1622126197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42d8068060 0x7f42d81f4b00 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.313+0000 7f42de76d640 1 -- 172.21.15.139:0/1622126197 >> 172.21.15.139:0/1622126197 conn(0x7f42d806ca40 msgr2=0x7f42d814b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:08.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.313+0000 7f42de76d640 1 -- 172.21.15.139:0/1622126197 shutdown_connections 2024-09-16T00:10:08.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.313+0000 7f42de76d640 1 -- 172.21.15.139:0/1622126197 wait complete. 2024-09-16T00:10:08.316 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T00:10:08.327 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T00:10:08.364 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:08.364 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:08.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:08.365 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:08.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.365 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3161 2.d 2:b02ffd62:::200.00000010:head [write 3427859~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.365 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:08.365 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.698042236999999, 2024-09-16T00:10:08.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026748516, 2024-09-16T00:10:08.365 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.365 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.365 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.365 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:08.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:08.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:08.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.15e-07 2024-09-16T00:10:08.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662637+0000", 2024-09-16T00:10:08.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8009999999999999e-06 2024-09-16T00:10:08.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662645+0000", 2024-09-16T00:10:08.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8299999999999996e-06 2024-09-16T00:10:08.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662650+0000", 2024-09-16T00:10:08.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.605e-06 2024-09-16T00:10:08.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.682553+0000", 2024-09-16T00:10:08.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01990331 2024-09-16T00:10:08.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.682636+0000", 2024-09-16T00:10:08.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2921000000000001e-05 2024-09-16T00:10:08.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.682780+0000", 2024-09-16T00:10:08.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000144361 2024-09-16T00:10:08.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.684809+0000", 2024-09-16T00:10:08.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002029005 2024-09-16T00:10:08.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689293+0000", 2024-09-16T00:10:08.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044841079999999997 2024-09-16T00:10:08.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689339+0000", 2024-09-16T00:10:08.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5668999999999999e-05 2024-09-16T00:10:08.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689382+0000", 2024-09-16T00:10:08.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3491000000000001e-05 2024-09-16T00:10:08.371 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.371 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.371 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.372 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3162 2.d 2:b02ffd62:::200.00000010:head [write 3430104~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.372 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.662858+0000", 2024-09-16T00:10:08.372 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.697818120999999, 2024-09-16T00:10:08.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026630083999999998, 2024-09-16T00:10:08.372 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.372 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.372 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.372 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662858+0000", 2024-09-16T00:10:08.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662858+0000", 2024-09-16T00:10:08.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662859+0000", 2024-09-16T00:10:08.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:10:08.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662862+0000", 2024-09-16T00:10:08.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6119999999999999e-06 2024-09-16T00:10:08.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662870+0000", 2024-09-16T00:10:08.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2360000000000004e-06 2024-09-16T00:10:08.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662874+0000", 2024-09-16T00:10:08.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2610000000000002e-06 2024-09-16T00:10:08.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.683241+0000", 2024-09-16T00:10:08.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.020367495999999999 2024-09-16T00:10:08.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.683295+0000", 2024-09-16T00:10:08.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4230000000000001e-05 2024-09-16T00:10:08.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.683380+0000", 2024-09-16T00:10:08.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4367e-05 2024-09-16T00:10:08.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.684845+0000", 2024-09-16T00:10:08.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014653190000000001 2024-09-16T00:10:08.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689392+0000", 2024-09-16T00:10:08.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045474280000000001 2024-09-16T00:10:08.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689445+0000", 2024-09-16T00:10:08.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2444000000000001e-05 2024-09-16T00:10:08.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689488+0000", 2024-09-16T00:10:08.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3124999999999998e-05 2024-09-16T00:10:08.378 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.378 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.378 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.378 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3166 2.d 2:b02ffd62:::200.00000010:head [write 3437392~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.378 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.664403+0000", 2024-09-16T00:10:08.378 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.696273243, 2024-09-16T00:10:08.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.027480674, 2024-09-16T00:10:08.378 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.378 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.378 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.379 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664403+0000", 2024-09-16T00:10:08.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664403+0000", 2024-09-16T00:10:08.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664404+0000", 2024-09-16T00:10:08.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4099999999999998e-07 2024-09-16T00:10:08.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664408+0000", 2024-09-16T00:10:08.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8169999999999998e-06 2024-09-16T00:10:08.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664418+0000", 2024-09-16T00:10:08.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6909999999999992e-06 2024-09-16T00:10:08.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664425+0000", 2024-09-16T00:10:08.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6309999999999997e-06 2024-09-16T00:10:08.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.685372+0000", 2024-09-16T00:10:08.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.020947548999999999 2024-09-16T00:10:08.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.685454+0000", 2024-09-16T00:10:08.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1355999999999998e-05 2024-09-16T00:10:08.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.685585+0000", 2024-09-16T00:10:08.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000130922 2024-09-16T00:10:08.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687329+0000", 2024-09-16T00:10:08.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017445710000000001 2024-09-16T00:10:08.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691757+0000", 2024-09-16T00:10:08.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004428268 2024-09-16T00:10:08.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691830+0000", 2024-09-16T00:10:08.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2237999999999995e-05 2024-09-16T00:10:08.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691884+0000", 2024-09-16T00:10:08.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3890000000000001e-05 2024-09-16T00:10:08.384 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.384 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.384 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.384 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3167 2.d 2:b02ffd62:::200.00000010:head [write 3439637~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.384 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.664627+0000", 2024-09-16T00:10:08.384 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.696048671, 2024-09-16T00:10:08.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.027347698, 2024-09-16T00:10:08.384 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.384 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.384 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.385 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664627+0000", 2024-09-16T00:10:08.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664627+0000", 2024-09-16T00:10:08.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664628+0000", 2024-09-16T00:10:08.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0500000000000003e-07 2024-09-16T00:10:08.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664632+0000", 2024-09-16T00:10:08.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8770000000000003e-06 2024-09-16T00:10:08.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664639+0000", 2024-09-16T00:10:08.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6590000000000001e-06 2024-09-16T00:10:08.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664642+0000", 2024-09-16T00:10:08.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8550000000000004e-06 2024-09-16T00:10:08.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.686530+0000", 2024-09-16T00:10:08.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.021887127999999999 2024-09-16T00:10:08.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.686577+0000", 2024-09-16T00:10:08.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7181000000000001e-05 2024-09-16T00:10:08.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.686667+0000", 2024-09-16T00:10:08.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0074000000000005e-05 2024-09-16T00:10:08.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688714+0000", 2024-09-16T00:10:08.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020467749999999998 2024-09-16T00:10:08.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691894+0000", 2024-09-16T00:10:08.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031806529999999999 2024-09-16T00:10:08.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691939+0000", 2024-09-16T00:10:08.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4579999999999997e-05 2024-09-16T00:10:08.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691975+0000", 2024-09-16T00:10:08.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6310999999999998e-05 2024-09-16T00:10:08.391 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.391 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.391 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.391 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3168 2.d 2:b02ffd62:::200.00000010:head [write 3441036~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.391 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:08.391 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.695582590000001, 2024-09-16T00:10:08.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026965759999999998, 2024-09-16T00:10:08.391 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.392 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.392 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.392 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:08.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:08.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:08.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7999999999999996e-07 2024-09-16T00:10:08.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665098+0000", 2024-09-16T00:10:08.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3200000000000001e-06 2024-09-16T00:10:08.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665104+0000", 2024-09-16T00:10:08.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4809999999999997e-06 2024-09-16T00:10:08.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665112+0000", 2024-09-16T00:10:08.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.67e-06 2024-09-16T00:10:08.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687392+0000", 2024-09-16T00:10:08.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.022279699 2024-09-16T00:10:08.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687461+0000", 2024-09-16T00:10:08.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8476e-05 2024-09-16T00:10:08.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687577+0000", 2024-09-16T00:10:08.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011661 2024-09-16T00:10:08.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688774+0000", 2024-09-16T00:10:08.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011962660000000001 2024-09-16T00:10:08.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691984+0000", 2024-09-16T00:10:08.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032106320000000002 2024-09-16T00:10:08.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692028+0000", 2024-09-16T00:10:08.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3866999999999999e-05 2024-09-16T00:10:08.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692059+0000", 2024-09-16T00:10:08.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1259e-05 2024-09-16T00:10:08.396 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.397 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.397 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.397 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3169 2.d 2:b02ffd62:::200.00000010:head [write 3443281~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.397 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:08.397 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.695126053999999, 2024-09-16T00:10:08.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.030090967999999999, 2024-09-16T00:10:08.397 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.397 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.397 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.397 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:08.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:08.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:08.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3799999999999998e-07 2024-09-16T00:10:08.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665554+0000", 2024-09-16T00:10:08.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.112e-06 2024-09-16T00:10:08.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665559+0000", 2024-09-16T00:10:08.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9559999999999996e-06 2024-09-16T00:10:08.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665562+0000", 2024-09-16T00:10:08.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2909999999999999e-06 2024-09-16T00:10:08.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687954+0000", 2024-09-16T00:10:08.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.022391714 2024-09-16T00:10:08.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688016+0000", 2024-09-16T00:10:08.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2384000000000004e-05 2024-09-16T00:10:08.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688147+0000", 2024-09-16T00:10:08.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013099300000000001 2024-09-16T00:10:08.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690321+0000", 2024-09-16T00:10:08.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021744339999999998 2024-09-16T00:10:08.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695548+0000", 2024-09-16T00:10:08.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052266079999999998 2024-09-16T00:10:08.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695595+0000", 2024-09-16T00:10:08.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7200000000000002e-05 2024-09-16T00:10:08.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695641+0000", 2024-09-16T00:10:08.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5837999999999997e-05 2024-09-16T00:10:08.403 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.403 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.403 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.403 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3170 2.d 2:b02ffd62:::200.00000010:head [write 3445526~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.403 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:08.403 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.694932993, 2024-09-16T00:10:08.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029987413000000001, 2024-09-16T00:10:08.404 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.404 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.404 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.404 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:08.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:08.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:08.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7500000000000001e-07 2024-09-16T00:10:08.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665753+0000", 2024-09-16T00:10:08.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0005e-05 2024-09-16T00:10:08.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665759+0000", 2024-09-16T00:10:08.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.169e-06 2024-09-16T00:10:08.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665762+0000", 2024-09-16T00:10:08.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8460000000000001e-06 2024-09-16T00:10:08.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688811+0000", 2024-09-16T00:10:08.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023048169 2024-09-16T00:10:08.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688885+0000", 2024-09-16T00:10:08.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4788000000000003e-05 2024-09-16T00:10:08.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688989+0000", 2024-09-16T00:10:08.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000103689 2024-09-16T00:10:08.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690350+0000", 2024-09-16T00:10:08.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013609329999999999 2024-09-16T00:10:08.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695652+0000", 2024-09-16T00:10:08.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053019080000000001 2024-09-16T00:10:08.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695698+0000", 2024-09-16T00:10:08.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5775000000000001e-05 2024-09-16T00:10:08.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695731+0000", 2024-09-16T00:10:08.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2756000000000003e-05 2024-09-16T00:10:08.410 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.410 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.410 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.410 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3171 2.d 2:b02ffd62:::200.00000010:head [write 3446925~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.410 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665997+0000", 2024-09-16T00:10:08.410 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.694678902, 2024-09-16T00:10:08.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029824032, 2024-09-16T00:10:08.411 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.411 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.411 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.411 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665997+0000", 2024-09-16T00:10:08.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665997+0000", 2024-09-16T00:10:08.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665998+0000", 2024-09-16T00:10:08.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:10:08.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666000+0000", 2024-09-16T00:10:08.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6290000000000002e-06 2024-09-16T00:10:08.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666005+0000", 2024-09-16T00:10:08.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.437e-06 2024-09-16T00:10:08.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666008+0000", 2024-09-16T00:10:08.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7910000000000002e-06 2024-09-16T00:10:08.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689799+0000", 2024-09-16T00:10:08.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023790504000000001 2024-09-16T00:10:08.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689907+0000", 2024-09-16T00:10:08.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108464 2024-09-16T00:10:08.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689994+0000", 2024-09-16T00:10:08.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7043e-05 2024-09-16T00:10:08.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691737+0000", 2024-09-16T00:10:08.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001742527 2024-09-16T00:10:08.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695739+0000", 2024-09-16T00:10:08.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040020189999999999 2024-09-16T00:10:08.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695786+0000", 2024-09-16T00:10:08.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7219999999999999e-05 2024-09-16T00:10:08.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695821+0000", 2024-09-16T00:10:08.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5058000000000002e-05 2024-09-16T00:10:08.417 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.417 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.418 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.418 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3172 2.d 2:b02ffd62:::200.00000010:head [write 3448324~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.418 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.666428+0000", 2024-09-16T00:10:08.418 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.694247952, 2024-09-16T00:10:08.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029477316, 2024-09-16T00:10:08.418 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.418 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.418 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.419 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666428+0000", 2024-09-16T00:10:08.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666428+0000", 2024-09-16T00:10:08.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666429+0000", 2024-09-16T00:10:08.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3699999999999999e-07 2024-09-16T00:10:08.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666433+0000", 2024-09-16T00:10:08.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4939999999999997e-06 2024-09-16T00:10:08.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666443+0000", 2024-09-16T00:10:08.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0899999999999994e-06 2024-09-16T00:10:08.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666447+0000", 2024-09-16T00:10:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8930000000000004e-06 2024-09-16T00:10:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690390+0000", 2024-09-16T00:10:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023942215999999999 2024-09-16T00:10:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690464+0000", 2024-09-16T00:10:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4678999999999994e-05 2024-09-16T00:10:08.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690576+0000", 2024-09-16T00:10:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011162000000000001 2024-09-16T00:10:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692126+0000", 2024-09-16T00:10:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015496520000000001 2024-09-16T00:10:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695828+0000", 2024-09-16T00:10:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037028510000000001 2024-09-16T00:10:08.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695870+0000", 2024-09-16T00:10:08.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1180999999999998e-05 2024-09-16T00:10:08.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695905+0000", 2024-09-16T00:10:08.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5803000000000003e-05 2024-09-16T00:10:08.423 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.423 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.423 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.424 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3173 2.d 2:b02ffd62:::200.00000010:head [write 3449723~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.424 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.666884+0000", 2024-09-16T00:10:08.424 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.693791996, 2024-09-16T00:10:08.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.02908875, 2024-09-16T00:10:08.424 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.424 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.424 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.424 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666884+0000", 2024-09-16T00:10:08.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666884+0000", 2024-09-16T00:10:08.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666885+0000", 2024-09-16T00:10:08.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0999999999999998e-07 2024-09-16T00:10:08.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666889+0000", 2024-09-16T00:10:08.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4299999999999999e-06 2024-09-16T00:10:08.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666895+0000", 2024-09-16T00:10:08.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0510000000000002e-06 2024-09-16T00:10:08.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666899+0000", 2024-09-16T00:10:08.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0489999999999997e-06 2024-09-16T00:10:08.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691161+0000", 2024-09-16T00:10:08.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.024262156999999999 2024-09-16T00:10:08.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691253+0000", 2024-09-16T00:10:08.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1278000000000003e-05 2024-09-16T00:10:08.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691348+0000", 2024-09-16T00:10:08.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4777000000000004e-05 2024-09-16T00:10:08.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692211+0000", 2024-09-16T00:10:08.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086351699999999995 2024-09-16T00:10:08.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695913+0000", 2024-09-16T00:10:08.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037019840000000002 2024-09-16T00:10:08.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695952+0000", 2024-09-16T00:10:08.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8535999999999998e-05 2024-09-16T00:10:08.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695973+0000", 2024-09-16T00:10:08.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1260999999999999e-05 2024-09-16T00:10:08.429 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.429 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.429 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.429 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3174 2.d 2:b02ffd62:::200.00000010:head [write 3451968~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.429 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.667323+0000", 2024-09-16T00:10:08.430 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.693353126, 2024-09-16T00:10:08.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028788859999999999, 2024-09-16T00:10:08.430 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.430 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.430 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.430 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667323+0000", 2024-09-16T00:10:08.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667323+0000", 2024-09-16T00:10:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667324+0000", 2024-09-16T00:10:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0400000000000005e-07 2024-09-16T00:10:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667330+0000", 2024-09-16T00:10:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3300000000000004e-06 2024-09-16T00:10:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667338+0000", 2024-09-16T00:10:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0059999999999992e-06 2024-09-16T00:10:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667344+0000", 2024-09-16T00:10:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6409999999999999e-06 2024-09-16T00:10:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692253+0000", 2024-09-16T00:10:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.024909280999999998 2024-09-16T00:10:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692311+0000", 2024-09-16T00:10:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7873e-05 2024-09-16T00:10:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692395+0000", 2024-09-16T00:10:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4071000000000002e-05 2024-09-16T00:10:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695977+0000", 2024-09-16T00:10:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035823460000000001 2024-09-16T00:10:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696034+0000", 2024-09-16T00:10:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6697999999999998e-05 2024-09-16T00:10:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696078+0000", 2024-09-16T00:10:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3696000000000003e-05 2024-09-16T00:10:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696112+0000", 2024-09-16T00:10:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4113999999999999e-05 2024-09-16T00:10:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3175 2.d 2:b02ffd62:::200.00000010:head [write 3454213~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.693012288, 2024-09-16T00:10:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.030153698999999999, 2024-09-16T00:10:08.435 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2e-07 2024-09-16T00:10:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667668+0000", 2024-09-16T00:10:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2109999999999998e-06 2024-09-16T00:10:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667673+0000", 2024-09-16T00:10:08.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8409999999999996e-06 2024-09-16T00:10:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667686+0000", 2024-09-16T00:10:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2928000000000001e-05 2024-09-16T00:10:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692964+0000", 2024-09-16T00:10:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.025277642999999999 2024-09-16T00:10:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.694418+0000", 2024-09-16T00:10:08.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001453693 2024-09-16T00:10:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.694505+0000", 2024-09-16T00:10:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7700000000000004e-05 2024-09-16T00:10:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697114+0000", 2024-09-16T00:10:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026083460000000001 2024-09-16T00:10:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697697+0000", 2024-09-16T00:10:08.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00058297399999999999 2024-09-16T00:10:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697772+0000", 2024-09-16T00:10:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5433999999999996e-05 2024-09-16T00:10:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697818+0000", 2024-09-16T00:10:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5408999999999998e-05 2024-09-16T00:10:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3176 2.d 2:b02ffd62:::200.00000010:head [write 3456458~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.668006+0000", 2024-09-16T00:10:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.692670382999999, 2024-09-16T00:10:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029935413000000001, 2024-09-16T00:10:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668006+0000", 2024-09-16T00:10:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668006+0000", 2024-09-16T00:10:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668007+0000", 2024-09-16T00:10:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5799999999999998e-07 2024-09-16T00:10:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668012+0000", 2024-09-16T00:10:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9320000000000004e-06 2024-09-16T00:10:08.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668023+0000", 2024-09-16T00:10:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1273e-05 2024-09-16T00:10:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668030+0000", 2024-09-16T00:10:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0509999999999999e-06 2024-09-16T00:10:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.694928+0000", 2024-09-16T00:10:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026898114000000001 2024-09-16T00:10:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695003+0000", 2024-09-16T00:10:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4880000000000001e-05 2024-09-16T00:10:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695116+0000", 2024-09-16T00:10:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011307900000000001 2024-09-16T00:10:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697119+0000", 2024-09-16T00:10:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020032159999999999 2024-09-16T00:10:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697867+0000", 2024-09-16T00:10:08.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074744999999999998 2024-09-16T00:10:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697908+0000", 2024-09-16T00:10:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1287e-05 2024-09-16T00:10:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697941+0000", 2024-09-16T00:10:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3272999999999998e-05 2024-09-16T00:10:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.446 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3177 2.d 2:b02ffd62:::200.00000010:head [write 3457857~1398 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.691335523999999, 2024-09-16T00:10:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029906856999999998, 2024-09-16T00:10:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.447 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:08.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1299999999999998e-07 2024-09-16T00:10:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669347+0000", 2024-09-16T00:10:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.327e-06 2024-09-16T00:10:08.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669359+0000", 2024-09-16T00:10:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1698e-05 2024-09-16T00:10:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669366+0000", 2024-09-16T00:10:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7959999999999996e-06 2024-09-16T00:10:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696168+0000", 2024-09-16T00:10:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026802093999999999 2024-09-16T00:10:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696258+0000", 2024-09-16T00:10:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0074000000000005e-05 2024-09-16T00:10:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696383+0000", 2024-09-16T00:10:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012455699999999999 2024-09-16T00:10:08.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698602+0000", 2024-09-16T00:10:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002218921 2024-09-16T00:10:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699152+0000", 2024-09-16T00:10:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055024399999999997 2024-09-16T00:10:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699206+0000", 2024-09-16T00:10:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3882999999999999e-05 2024-09-16T00:10:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699247+0000", 2024-09-16T00:10:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1350000000000002e-05 2024-09-16T00:10:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3178 2.d 2:b02ffd62:::200.00000010:head [write 3459255~2511 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.671779+0000", 2024-09-16T00:10:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.688896644, 2024-09-16T00:10:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028813986999999999, 2024-09-16T00:10:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671779+0000", 2024-09-16T00:10:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671779+0000", 2024-09-16T00:10:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671780+0000", 2024-09-16T00:10:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3600000000000002e-07 2024-09-16T00:10:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671785+0000", 2024-09-16T00:10:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.074e-06 2024-09-16T00:10:08.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671795+0000", 2024-09-16T00:10:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2040000000000003e-06 2024-09-16T00:10:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671800+0000", 2024-09-16T00:10:08.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9420000000000003e-06 2024-09-16T00:10:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697159+0000", 2024-09-16T00:10:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.025359619999999999 2024-09-16T00:10:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697214+0000", 2024-09-16T00:10:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5318000000000001e-05 2024-09-16T00:10:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697297+0000", 2024-09-16T00:10:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2087000000000002e-05 2024-09-16T00:10:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699258+0000", 2024-09-16T00:10:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019619529999999998 2024-09-16T00:10:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700499+0000", 2024-09-16T00:10:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001240072 2024-09-16T00:10:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700555+0000", 2024-09-16T00:10:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6459999999999998e-05 2024-09-16T00:10:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700593+0000", 2024-09-16T00:10:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8420999999999997e-05 2024-09-16T00:10:08.460 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3179 2.d 2:b02ffd62:::200.00000010:head [write 3461766~2418 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.672548+0000", 2024-09-16T00:10:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.688128547, 2024-09-16T00:10:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028160178000000001, 2024-09-16T00:10:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.461 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672548+0000", 2024-09-16T00:10:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672548+0000", 2024-09-16T00:10:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672549+0000", 2024-09-16T00:10:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5300000000000002e-07 2024-09-16T00:10:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672562+0000", 2024-09-16T00:10:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3808000000000001e-05 2024-09-16T00:10:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672575+0000", 2024-09-16T00:10:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2870999999999999e-05 2024-09-16T00:10:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672587+0000", 2024-09-16T00:10:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.203e-05 2024-09-16T00:10:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697994+0000", 2024-09-16T00:10:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.025406373999999999 2024-09-16T00:10:08.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698074+0000", 2024-09-16T00:10:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0802000000000003e-05 2024-09-16T00:10:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698196+0000", 2024-09-16T00:10:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012154699999999999 2024-09-16T00:10:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700456+0000", 2024-09-16T00:10:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022602249999999998 2024-09-16T00:10:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700632+0000", 2024-09-16T00:10:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017570699999999999 2024-09-16T00:10:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700676+0000", 2024-09-16T00:10:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3764000000000003e-05 2024-09-16T00:10:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700708+0000", 2024-09-16T00:10:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2097000000000001e-05 2024-09-16T00:10:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.467 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3180 2.d 2:b02ffd62:::200.00000010:head [write 3464184~1398 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.686620596000001, 2024-09-16T00:10:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028112109, 2024-09-16T00:10:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:10:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674061+0000", 2024-09-16T00:10:08.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4440000000000001e-06 2024-09-16T00:10:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674069+0000", 2024-09-16T00:10:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7639999999999994e-06 2024-09-16T00:10:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674076+0000", 2024-09-16T00:10:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5590000000000002e-06 2024-09-16T00:10:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698639+0000", 2024-09-16T00:10:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.024562471999999998 2024-09-16T00:10:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698693+0000", 2024-09-16T00:10:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4432999999999999e-05 2024-09-16T00:10:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698807+0000", 2024-09-16T00:10:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113607 2024-09-16T00:10:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700727+0000", 2024-09-16T00:10:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019201750000000001 2024-09-16T00:10:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702076+0000", 2024-09-16T00:10:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001349423 2024-09-16T00:10:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702128+0000", 2024-09-16T00:10:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2157000000000002e-05 2024-09-16T00:10:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702168+0000", 2024-09-16T00:10:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9292000000000003e-05 2024-09-16T00:10:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3181 2.d 2:b02ffd62:::200.00000010:head [write 3465582~2754 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.675904+0000", 2024-09-16T00:10:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.684772284999999, 2024-09-16T00:10:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026402452, 2024-09-16T00:10:08.473 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675904+0000", 2024-09-16T00:10:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675904+0000", 2024-09-16T00:10:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675905+0000", 2024-09-16T00:10:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.043e-06 2024-09-16T00:10:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675911+0000", 2024-09-16T00:10:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1970000000000001e-06 2024-09-16T00:10:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675924+0000", 2024-09-16T00:10:08.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.252e-05 2024-09-16T00:10:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675931+0000", 2024-09-16T00:10:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5800000000000003e-06 2024-09-16T00:10:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699280+0000", 2024-09-16T00:10:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023348634 2024-09-16T00:10:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699329+0000", 2024-09-16T00:10:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9175999999999998e-05 2024-09-16T00:10:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699400+0000", 2024-09-16T00:10:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0649999999999996e-05 2024-09-16T00:10:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700734+0000", 2024-09-16T00:10:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001334816 2024-09-16T00:10:08.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702219+0000", 2024-09-16T00:10:08.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014847930000000001 2024-09-16T00:10:08.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702267+0000", 2024-09-16T00:10:08.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8133000000000001e-05 2024-09-16T00:10:08.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702306+0000", 2024-09-16T00:10:08.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8909999999999998e-05 2024-09-16T00:10:08.479 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.479 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.479 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.479 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3184 2.d 2:b02ffd62:::200.00000010:head [write 3478077~11949 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.479 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.681430+0000", 2024-09-16T00:10:08.479 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.679246042999999, 2024-09-16T00:10:08.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.02662339, 2024-09-16T00:10:08.479 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.479 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.479 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.479 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681430+0000", 2024-09-16T00:10:08.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681430+0000", 2024-09-16T00:10:08.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681431+0000", 2024-09-16T00:10:08.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9999999999999995e-07 2024-09-16T00:10:08.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681443+0000", 2024-09-16T00:10:08.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1568e-05 2024-09-16T00:10:08.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681453+0000", 2024-09-16T00:10:08.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8819999999999996e-06 2024-09-16T00:10:08.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681458+0000", 2024-09-16T00:10:08.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3709999999999999e-06 2024-09-16T00:10:08.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.701384+0000", 2024-09-16T00:10:08.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.019926342999999999 2024-09-16T00:10:08.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.701458+0000", 2024-09-16T00:10:08.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3854000000000001e-05 2024-09-16T00:10:08.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.701570+0000", 2024-09-16T00:10:08.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000111437 2024-09-16T00:10:08.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702913+0000", 2024-09-16T00:10:08.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013437259999999999 2024-09-16T00:10:08.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.707984+0000", 2024-09-16T00:10:08.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050705669999999998 2024-09-16T00:10:08.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708016+0000", 2024-09-16T00:10:08.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2527999999999998e-05 2024-09-16T00:10:08.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708053+0000", 2024-09-16T00:10:08.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7113999999999997e-05 2024-09-16T00:10:08.484 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.484 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.484 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.484 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3185 2.d 2:b02ffd62:::200.00000010:head [write 3490026~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:08.484 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.681685+0000", 2024-09-16T00:10:08.484 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 14.678990772000001, 2024-09-16T00:10:08.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026552664, 2024-09-16T00:10:08.485 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:08.485 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:08.485 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:08.485 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:08.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:08.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681685+0000", 2024-09-16T00:10:08.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.486 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:08.486 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681685+0000", 2024-09-16T00:10:08.486 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:08.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.486 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:08.486 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681686+0000", 2024-09-16T00:10:08.486 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.06e-07 2024-09-16T00:10:08.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.486 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:08.486 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681692+0000", 2024-09-16T00:10:08.486 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6840000000000001e-06 2024-09-16T00:10:08.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:08.487 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681697+0000", 2024-09-16T00:10:08.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0340000000000004e-06 2024-09-16T00:10:08.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:08.487 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681700+0000", 2024-09-16T00:10:08.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3589999999999999e-06 2024-09-16T00:10:08.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:08.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702500+0000", 2024-09-16T00:10:08.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.020799746000000001 2024-09-16T00:10:08.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:08.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702550+0000", 2024-09-16T00:10:08.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9895999999999998e-05 2024-09-16T00:10:08.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:08.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702649+0000", 2024-09-16T00:10:08.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9500999999999995e-05 2024-09-16T00:10:08.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:08.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.705343+0000", 2024-09-16T00:10:08.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026938130000000002 2024-09-16T00:10:08.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:08.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708128+0000", 2024-09-16T00:10:08.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027852979999999999 2024-09-16T00:10:08.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:08.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708202+0000", 2024-09-16T00:10:08.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4046000000000002e-05 2024-09-16T00:10:08.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:08.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:08.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:08.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708238+0000", 2024-09-16T00:10:08.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5781000000000002e-05 2024-09-16T00:10:08.490 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.490 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.490 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.490 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:08.490 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:08.490 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:08.491 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:10:08.492 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:08.491+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:08.493 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:10:08.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.422+0000 7f529aeb7640 1 Processor -- start 2024-09-16T00:10:08.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.422+0000 7f529aeb7640 1 -- start start 2024-09-16T00:10:08.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.423+0000 7f529aeb7640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f52940c8b50 0x7f529415de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.423+0000 7f529aeb7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5294070d90 0x7f52940ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.423+0000 7f529aeb7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f529406ccd0 con 0x7f529415da90 2024-09-16T00:10:08.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.423+0000 7f529aeb7640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f529415e3b0 con 0x7f52940c8b50 2024-09-16T00:10:08.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.423+0000 7f529aeb7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f52940ca720 con 0x7f5294070d90 2024-09-16T00:10:08.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.423+0000 7f5298c53640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f529415da90 0x7f5294070410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45716/0 (socket says 172.21.15.139:45716) 2024-09-16T00:10:08.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.423+0000 7f5298c53640 1 -- 172.21.15.139:0/1120923392 learned_addr learned my addr 172.21.15.139:0/1120923392 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:08.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.424+0000 7f5293fff640 1 --2- 172.21.15.139:0/1120923392 >> v2:172.21.15.164:3300/0 conn(0x7f52940c8b50 0x7f529415de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.424+0000 7f5299454640 1 --2- 172.21.15.139:0/1120923392 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5294070d90 0x7f52940ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.424+0000 7f52937fe640 1 -- 172.21.15.139:0/1120923392 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 819975765 0 0) 0x7f529406ccd0 con 0x7f529415da90 2024-09-16T00:10:08.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.424+0000 7f52937fe640 1 -- 172.21.15.139:0/1120923392 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5284003520 con 0x7f529415da90 2024-09-16T00:10:08.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.424+0000 7f5293fff640 1 -- 172.21.15.139:0/1120923392 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5294070d90 msgr2=0x7f52940ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:08.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.424+0000 7f5293fff640 1 --2- 172.21.15.139:0/1120923392 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5294070d90 0x7f52940ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.424+0000 7f5293fff640 1 -- 172.21.15.139:0/1120923392 >> v1:172.21.15.139:6789/0 conn(0x7f529415da90 legacy=0x7f5294070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.424+0000 7f5293fff640 1 -- 172.21.15.139:0/1120923392 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529406cf40 con 0x7f52940c8b50 2024-09-16T00:10:08.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.425+0000 7f5293fff640 1 --2- 172.21.15.139:0/1120923392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52940c8b50 0x7f529415de70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f52800097a0 tx=0x7f528002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8258c24f1918e915 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.425+0000 7f52937fe640 1 -- 172.21.15.139:0/1120923392 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280037070 con 0x7f52940c8b50 2024-09-16T00:10:08.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.425+0000 7f52937fe640 1 -- 172.21.15.139:0/1120923392 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5280032440 con 0x7f52940c8b50 2024-09-16T00:10:08.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.425+0000 7f52937fe640 1 -- 172.21.15.139:0/1120923392 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52800327f0 con 0x7f52940c8b50 2024-09-16T00:10:08.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.425+0000 7f529aeb7640 1 -- 172.21.15.139:0/1120923392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52940c8b50 msgr2=0x7f529415de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.426+0000 7f529aeb7640 1 --2- 172.21.15.139:0/1120923392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52940c8b50 0x7f529415de70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f52800097a0 tx=0x7f528002a180 comp rx=0 tx=0).stop 2024-09-16T00:10:08.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.426+0000 7f529aeb7640 1 -- 172.21.15.139:0/1120923392 shutdown_connections 2024-09-16T00:10:08.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.426+0000 7f529aeb7640 1 --2- 172.21.15.139:0/1120923392 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5294070d90 0x7f52940ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.426+0000 7f529aeb7640 1 --2- 172.21.15.139:0/1120923392 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f52940c8b50 0x7f529415de70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.426+0000 7f529aeb7640 1 -- 172.21.15.139:0/1120923392 >> 172.21.15.139:0/1120923392 conn(0x7f529406c150 msgr2=0x7f5294067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:08.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.426+0000 7f529aeb7640 1 -- 172.21.15.139:0/1120923392 shutdown_connections 2024-09-16T00:10:08.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.426+0000 7f529aeb7640 1 -- 172.21.15.139:0/1120923392 wait complete. 2024-09-16T00:10:08.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.427+0000 7f529aeb7640 1 Processor -- start 2024-09-16T00:10:08.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.427+0000 7f529aeb7640 1 -- start start 2024-09-16T00:10:08.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f529aeb7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5294070d90 0x7f52941f0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f529aeb7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52940c8b50 0x7f5294150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f529aeb7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f529415da90 0x7f5294151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f529aeb7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f52940ca2e0 con 0x7f52940c8b50 2024-09-16T00:10:08.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f529aeb7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f52940caa00 con 0x7f5294070d90 2024-09-16T00:10:08.501 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f529aeb7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f52940ca460 con 0x7f529415da90 2024-09-16T00:10:08.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f5293fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52940c8b50 0x7f5294150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f5298c53640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5294070d90 0x7f52941f0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f5293fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52940c8b50 0x7f5294150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49138/0 (socket says 172.21.15.139:49138) 2024-09-16T00:10:08.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f5298c53640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5294070d90 0x7f52941f0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:42250/0 (socket says 172.21.15.139:42250) 2024-09-16T00:10:08.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f5293fff640 1 -- 172.21.15.139:0/407238117 learned_addr learned my addr 172.21.15.139:0/407238117 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:08.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f5299454640 1 --2- 172.21.15.139:0/407238117 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f529415da90 0x7f5294151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f5293fff640 1 -- 172.21.15.139:0/407238117 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f529415da90 msgr2=0x7f5294151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f5293fff640 1 --2- 172.21.15.139:0/407238117 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f529415da90 0x7f5294151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.502 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f5293fff640 1 -- 172.21.15.139:0/407238117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5294070d90 msgr2=0x7f52941f0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f5293fff640 1 --2- 172.21.15.139:0/407238117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5294070d90 0x7f52941f0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f5293fff640 1 -- 172.21.15.139:0/407238117 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52941f3e00 con 0x7f52940c8b50 2024-09-16T00:10:08.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f5298c53640 1 --2- 172.21.15.139:0/407238117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5294070d90 0x7f52941f0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:08.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.428+0000 7f5293fff640 1 --2- 172.21.15.139:0/407238117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52940c8b50 0x7f5294150fe0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5280035040 tx=0x7f52800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.429+0000 7f52917fa640 1 -- 172.21.15.139:0/407238117 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280037040 con 0x7f52940c8b50 2024-09-16T00:10:08.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.429+0000 7f52917fa640 1 -- 172.21.15.139:0/407238117 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52800044c0 con 0x7f52940c8b50 2024-09-16T00:10:08.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.429+0000 7f529aeb7640 1 -- 172.21.15.139:0/407238117 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52941efd70 con 0x7f52940c8b50 2024-09-16T00:10:08.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.429+0000 7f52917fa640 1 -- 172.21.15.139:0/407238117 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52800325f0 con 0x7f52940c8b50 2024-09-16T00:10:08.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.429+0000 7f529aeb7640 1 -- 172.21.15.139:0/407238117 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52941f0330 con 0x7f52940c8b50 2024-09-16T00:10:08.503 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.430+0000 7f529aeb7640 1 -- 172.21.15.139:0/407238117 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f529406e9f0 con 0x7f52940c8b50 2024-09-16T00:10:08.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.430+0000 7f52917fa640 1 -- 172.21.15.139:0/407238117 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5280044020 con 0x7f52940c8b50 2024-09-16T00:10:08.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.431+0000 7f52917fa640 1 --2- 172.21.15.139:0/407238117 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f527806c300 0x7f527806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.431+0000 7f5298c53640 1 --2- 172.21.15.139:0/407238117 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f527806c300 0x7f527806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.431+0000 7f52917fa640 1 -- 172.21.15.139:0/407238117 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5973+0+0 (secure 0 0 0) 0x7f52800ab490 con 0x7f52940c8b50 2024-09-16T00:10:08.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.431+0000 7f5298c53640 1 --2- 172.21.15.139:0/407238117 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f527806c300 0x7f527806e7c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f52880011e0 tx=0x7f5288007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.436+0000 7f52917fa640 1 -- 172.21.15.139:0/407238117 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5280079610 con 0x7f52940c8b50 2024-09-16T00:10:08.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.472+0000 7fbb2c18f640 1 Processor -- start 2024-09-16T00:10:08.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.472+0000 7fbb2c18f640 1 -- start start 2024-09-16T00:10:08.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.473+0000 7fbb2c18f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbb24068060 0x7fbb2415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.473+0000 7fbb2c18f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbb240ccc90 0x7fbb240cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.473+0000 7fbb2c18f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb240cc4b0 con 0x7fbb24154c20 2024-09-16T00:10:08.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.473+0000 7fbb2c18f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbb240c5830 con 0x7fbb24068060 2024-09-16T00:10:08.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.473+0000 7fbb2c18f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fbb2415b7c0 con 0x7fbb240ccc90 2024-09-16T00:10:08.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.473+0000 7fbb2972a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbb24068060 0x7fbb2415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.473+0000 7fbb2972a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbb24068060 0x7fbb2415b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49146/0 (socket says 172.21.15.139:49146) 2024-09-16T00:10:08.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.473+0000 7fbb2972a640 1 -- 172.21.15.139:0/1282531808 learned_addr learned my addr 172.21.15.139:0/1282531808 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:08.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.473+0000 7fbb2a72c640 1 --2- 172.21.15.139:0/1282531808 >> v2:172.21.15.164:3300/0 conn(0x7fbb240ccc90 0x7fbb240cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.473+0000 7fbb2972a640 1 -- 172.21.15.139:0/1282531808 >> v2:172.21.15.164:3300/0 conn(0x7fbb240ccc90 msgr2=0x7fbb240cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:08.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.473+0000 7fbb2972a640 1 --2- 172.21.15.139:0/1282531808 >> v2:172.21.15.164:3300/0 conn(0x7fbb240ccc90 0x7fbb240cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.473+0000 7fbb2972a640 1 -- 172.21.15.139:0/1282531808 >> v1:172.21.15.164:6789/0 conn(0x7fbb24154c20 legacy=0x7fbb240c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.473+0000 7fbb2972a640 1 -- 172.21.15.139:0/1282531808 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb240cc790 con 0x7fbb24068060 2024-09-16T00:10:08.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.473+0000 7fbb28f29640 1 -- 172.21.15.139:0/1282531808 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3353319670 0 0) 0x7fbb240cc4b0 con 0x7fbb24154c20 2024-09-16T00:10:08.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.474+0000 7fbb2972a640 1 --2- 172.21.15.139:0/1282531808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb24068060 0x7fbb2415b280 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbb240c7300 tx=0x7fbb1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e17c17c9ae2aeb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.474+0000 7fbb28f29640 1 -- 172.21.15.139:0/1282531808 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14004030 con 0x7fbb24068060 2024-09-16T00:10:08.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.474+0000 7fbb28f29640 1 -- 172.21.15.139:0/1282531808 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb140041d0 con 0x7fbb24068060 2024-09-16T00:10:08.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.474+0000 7fbb28f29640 1 -- 172.21.15.139:0/1282531808 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14003610 con 0x7fbb24068060 2024-09-16T00:10:08.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.475+0000 7fbb2c18f640 1 -- 172.21.15.139:0/1282531808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb24068060 msgr2=0x7fbb2415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.475+0000 7fbb2c18f640 1 --2- 172.21.15.139:0/1282531808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb24068060 0x7fbb2415b280 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbb240c7300 tx=0x7fbb1402a270 comp rx=0 tx=0).stop 2024-09-16T00:10:08.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.475+0000 7fbb2c18f640 1 -- 172.21.15.139:0/1282531808 shutdown_connections 2024-09-16T00:10:08.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.475+0000 7fbb2c18f640 1 --2- 172.21.15.139:0/1282531808 >> v2:172.21.15.164:3300/0 conn(0x7fbb240ccc90 0x7fbb240cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.475+0000 7fbb2c18f640 1 --2- 172.21.15.139:0/1282531808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb24068060 0x7fbb2415b280 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.475+0000 7fbb2c18f640 1 -- 172.21.15.139:0/1282531808 >> 172.21.15.139:0/1282531808 conn(0x7fbb2406ca40 msgr2=0x7fbb24067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:08.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.475+0000 7fbb2c18f640 1 -- 172.21.15.139:0/1282531808 shutdown_connections 2024-09-16T00:10:08.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.475+0000 7fbb2c18f640 1 -- 172.21.15.139:0/1282531808 wait complete. 2024-09-16T00:10:08.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.476+0000 7fbb2c18f640 1 Processor -- start 2024-09-16T00:10:08.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.476+0000 7fbb2c18f640 1 -- start start 2024-09-16T00:10:08.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb2c18f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb24068060 0x7fbb2414a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb2c18f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb240ccc90 0x7fbb240c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb2c18f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb24154c20 0x7fbb241f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb2c18f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2415e0d0 con 0x7fbb240ccc90 2024-09-16T00:10:08.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb2c18f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb240cc260 con 0x7fbb24068060 2024-09-16T00:10:08.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb2c18f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2415e3b0 con 0x7fbb24154c20 2024-09-16T00:10:08.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb2972a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb240ccc90 0x7fbb240c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb29f2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb24068060 0x7fbb2414a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb29f2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb24068060 0x7fbb2414a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:42264/0 (socket says 172.21.15.139:42264) 2024-09-16T00:10:08.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb2972a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb240ccc90 0x7fbb240c9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49162/0 (socket says 172.21.15.139:49162) 2024-09-16T00:10:08.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb29f2b640 1 -- 172.21.15.139:0/1021314376 learned_addr learned my addr 172.21.15.139:0/1021314376 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:08.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb2a72c640 1 --2- 172.21.15.139:0/1021314376 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb24154c20 0x7fbb241f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb2972a640 1 -- 172.21.15.139:0/1021314376 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb24154c20 msgr2=0x7fbb241f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb2972a640 1 --2- 172.21.15.139:0/1021314376 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb24154c20 0x7fbb241f3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb2972a640 1 -- 172.21.15.139:0/1021314376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb24068060 msgr2=0x7fbb2414a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb2972a640 1 --2- 172.21.15.139:0/1021314376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb24068060 0x7fbb2414a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.477+0000 7fbb2972a640 1 -- 172.21.15.139:0/1021314376 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb24158860 con 0x7fbb240ccc90 2024-09-16T00:10:08.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.478+0000 7fbb2972a640 1 --2- 172.21.15.139:0/1021314376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb240ccc90 0x7fbb240c9510 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbb24074130 tx=0x7fbb140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.478+0000 7fbb1a7fc640 1 -- 172.21.15.139:0/1021314376 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14041030 con 0x7fbb240ccc90 2024-09-16T00:10:08.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.478+0000 7fbb1a7fc640 1 -- 172.21.15.139:0/1021314376 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb14003c50 con 0x7fbb240ccc90 2024-09-16T00:10:08.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.478+0000 7fbb2c18f640 1 -- 172.21.15.139:0/1021314376 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb24067e40 con 0x7fbb240ccc90 2024-09-16T00:10:08.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.478+0000 7fbb1a7fc640 1 -- 172.21.15.139:0/1021314376 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14004030 con 0x7fbb240ccc90 2024-09-16T00:10:08.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.478+0000 7fbb2c18f640 1 -- 172.21.15.139:0/1021314376 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb240c8d10 con 0x7fbb240ccc90 2024-09-16T00:10:08.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.479+0000 7fbb2c18f640 1 -- 172.21.15.139:0/1021314376 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb241f46d0 con 0x7fbb240ccc90 2024-09-16T00:10:08.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.480+0000 7fbb1a7fc640 1 -- 172.21.15.139:0/1021314376 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbb14044070 con 0x7fbb240ccc90 2024-09-16T00:10:08.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.480+0000 7fbb1a7fc640 1 --2- 172.21.15.139:0/1021314376 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb0806c2b0 0x7fbb0806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.480+0000 7fbb1a7fc640 1 -- 172.21.15.139:0/1021314376 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5973+0+0 (secure 0 0 0) 0x7fbb140abd60 con 0x7fbb240ccc90 2024-09-16T00:10:08.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.480+0000 7fbb29f2b640 1 --2- 172.21.15.139:0/1021314376 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb0806c2b0 0x7fbb0806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.481+0000 7fbb29f2b640 1 --2- 172.21.15.139:0/1021314376 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb0806c2b0 0x7fbb0806e770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbb20000f60 tx=0x7fbb20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.485+0000 7fbb1a7fc640 1 -- 172.21.15.139:0/1021314376 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbb241f46d0 con 0x7fbb240ccc90 2024-09-16T00:10:08.592 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:08.592+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:08.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.640+0000 7f529aeb7640 1 -- 172.21.15.139:0/407238117 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f5294070b70 con 0x7f52940c8b50 2024-09-16T00:10:08.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.640+0000 7fd73b7c6640 1 Processor -- start 2024-09-16T00:10:08.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.641+0000 7fd73b7c6640 1 -- start start 2024-09-16T00:10:08.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.641+0000 7fd73b7c6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd734071660 0x7fd7340c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.641+0000 7fd73b7c6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd73406e9d0 0x7fd73415cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.641+0000 7fd73b7c6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7340c8e40 con 0x7fd734071a40 2024-09-16T00:10:08.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.641+0000 7fd73b7c6640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fd73406d2e0 con 0x7fd73406e9d0 2024-09-16T00:10:08.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.641+0000 7fd73b7c6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd73406ccd0 con 0x7fd734071660 2024-09-16T00:10:08.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.642+0000 7fd739562640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd734071660 0x7fd7340c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.642+0000 7fd739562640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd734071660 0x7fd7340c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:56126/0 (socket says 172.21.15.139:56126) 2024-09-16T00:10:08.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.642+0000 7fd738d61640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd73406e9d0 0x7fd73415cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.642+0000 7fd739562640 1 -- 172.21.15.139:0/1730585003 learned_addr learned my addr 172.21.15.139:0/1730585003 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:08.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.642+0000 7fd739d63640 1 --1- 172.21.15.139:0/1730585003 >> v1:172.21.15.164:6789/0 conn(0x7fd734071a40 0x7fd73415b290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:44100/0 (socket says 172.21.15.139:44100) 2024-09-16T00:10:08.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.642+0000 7fd739562640 1 -- 172.21.15.139:0/1730585003 >> v2:172.21.15.164:3300/0 conn(0x7fd73406e9d0 msgr2=0x7fd73415cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:08.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.642+0000 7fd739562640 1 --2- 172.21.15.139:0/1730585003 >> v2:172.21.15.164:3300/0 conn(0x7fd73406e9d0 0x7fd73415cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.642+0000 7fd739562640 1 -- 172.21.15.139:0/1730585003 >> v1:172.21.15.164:6789/0 conn(0x7fd734071a40 legacy=0x7fd73415b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.642+0000 7fd739562640 1 -- 172.21.15.139:0/1730585003 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7340ce910 con 0x7fd734071660 2024-09-16T00:10:08.644 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.642+0000 7fd738d61640 1 --2- 172.21.15.139:0/1730585003 >> v2:172.21.15.164:3300/0 conn(0x7fd73406e9d0 0x7fd73415cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:08.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.643+0000 7fd739562640 1 --2- 172.21.15.139:0/1730585003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd734071660 0x7fd7340c8900 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd730009870 tx=0x7fd73002a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b44486ba2cc2e5dd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.643+0000 7fd72bfff640 1 -- 172.21.15.139:0/1730585003 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730037070 con 0x7fd734071660 2024-09-16T00:10:08.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.643+0000 7fd72bfff640 1 -- 172.21.15.139:0/1730585003 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73002ae50 con 0x7fd734071660 2024-09-16T00:10:08.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.643+0000 7fd72bfff640 1 -- 172.21.15.139:0/1730585003 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd730032860 con 0x7fd734071660 2024-09-16T00:10:08.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.644+0000 7fd73b7c6640 1 -- 172.21.15.139:0/1730585003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd734071660 msgr2=0x7fd7340c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.644+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/1730585003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd734071660 0x7fd7340c8900 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd730009870 tx=0x7fd73002a300 comp rx=0 tx=0).stop 2024-09-16T00:10:08.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.644+0000 7fd73b7c6640 1 -- 172.21.15.139:0/1730585003 shutdown_connections 2024-09-16T00:10:08.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.644+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/1730585003 >> v2:172.21.15.164:3300/0 conn(0x7fd73406e9d0 0x7fd73415cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.644+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/1730585003 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd734071660 0x7fd7340c8900 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.644+0000 7fd73b7c6640 1 -- 172.21.15.139:0/1730585003 >> 172.21.15.139:0/1730585003 conn(0x7fd73406fb00 msgr2=0x7fd7340738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:08.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.644+0000 7fd73b7c6640 1 -- 172.21.15.139:0/1730585003 shutdown_connections 2024-09-16T00:10:08.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.644+0000 7fd73b7c6640 1 -- 172.21.15.139:0/1730585003 wait complete. 2024-09-16T00:10:08.646 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:08.646 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:08.647 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:08.647 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:08.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.646+0000 7fd73b7c6640 1 Processor -- start 2024-09-16T00:10:08.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.646+0000 7fd73b7c6640 1 -- start start 2024-09-16T00:10:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.647+0000 7fd73b7c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd73406e9d0 0x7fd7341e5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.647+0000 7fd73b7c6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd734071660 0x7fd734159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.647+0000 7fd73b7c6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd734071a40 0x7fd734151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.647+0000 7fd73b7c6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd7340ceed0 con 0x7fd73406e9d0 2024-09-16T00:10:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.647+0000 7fd73b7c6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd73406cf70 con 0x7fd734071660 2024-09-16T00:10:08.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.647+0000 7fd73b7c6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd73415b9a0 con 0x7fd734071a40 2024-09-16T00:10:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.647+0000 7fd739562640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd73406e9d0 0x7fd7341e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.647+0000 7fd739d63640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd734071a40 0x7fd734151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.647+0000 7fd738d61640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd734071660 0x7fd734159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.647+0000 7fd739562640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd73406e9d0 0x7fd7341e5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49168/0 (socket says 172.21.15.139:49168) 2024-09-16T00:10:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.647+0000 7fd739562640 1 -- 172.21.15.139:0/2671632042 learned_addr learned my addr 172.21.15.139:0/2671632042 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.648+0000 7fd739d63640 1 -- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd734071660 msgr2=0x7fd734159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.648+0000 7fd739d63640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd734071660 0x7fd734159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.648+0000 7fd739d63640 1 -- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd73406e9d0 msgr2=0x7fd7341e5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.648+0000 7fd739d63640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd73406e9d0 0x7fd7341e5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.648+0000 7fd739d63640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7340c72b0 con 0x7fd734071a40 2024-09-16T00:10:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.648+0000 7fd739562640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd73406e9d0 0x7fd7341e5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.648+0000 7fd739d63640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd734071a40 0x7fd734151490 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00c730 tx=0x7fd72c00cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.648+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c018070 con 0x7fd734071a40 2024-09-16T00:10:08.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.648+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd72c004d60 con 0x7fd734071a40 2024-09-16T00:10:08.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.648+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7341f3950 con 0x7fd734071a40 2024-09-16T00:10:08.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.648+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72c00a860 con 0x7fd734071a40 2024-09-16T00:10:08.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.648+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7341f3f10 con 0x7fd734071a40 2024-09-16T00:10:08.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.649+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fd7340ce140 con 0x7fd734071a40 2024-09-16T00:10:08.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.650+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd72c01b070 con 0x7fd734071a40 2024-09-16T00:10:08.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.650+0000 7fd729ffb640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd71806c2e0 0x7fd71806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.650+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(102..102 src has 1..102) ==== 5973+0+0 (secure 0 0 0) 0x7fd72c08ab70 con 0x7fd734071a40 2024-09-16T00:10:08.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.650+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v102) ==== 54+0+16 (secure 0 0 0) 0x7fd72c08f1e0 con 0x7fd734071a40 2024-09-16T00:10:08.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.650+0000 7fd739562640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd71806c2e0 0x7fd71806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.650+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fd7341e39c0 0x7fd7341e6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.651+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd7341f4110 con 0x7fd7341e39c0 2024-09-16T00:10:08.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.651+0000 7fd738d61640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fd7341e39c0 0x7fd7341e6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.651+0000 7fd739562640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd71806c2e0 0x7fd71806e7a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd7300090b0 tx=0x7fd73002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.651+0000 7fd738d61640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fd7341e39c0 0x7fd7341e6d50 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.652+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd7341f4110 con 0x7fd7341e39c0 2024-09-16T00:10:08.656 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:10:08.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.684+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd734151ee0 con 0x7fd7341e39c0 2024-09-16T00:10:08.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.686+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd734151ee0 con 0x7fd7341e39c0 2024-09-16T00:10:08.687 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:08.688 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:08.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.686+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fd7340c7aa0 0x7fd7340c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.686+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fd7340cd860 con 0x7fd7340c7aa0 2024-09-16T00:10:08.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.686+0000 7fd739562640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fd7340c7aa0 0x7fd7340c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.687+0000 7fd739562640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fd7340c7aa0 0x7fd7340c7e80 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.687+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd7340cd860 con 0x7fd7340c7aa0 2024-09-16T00:10:08.693 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:08.693+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:08.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.716+0000 7fbb2c18f640 1 -- 172.21.15.139:0/1021314376 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) -- 0x7fbb2415e690 con 0x7fbb240ccc90 2024-09-16T00:10:08.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.718+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd7340008d0 con 0x7fd7340c7aa0 2024-09-16T00:10:08.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.719+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd7340008d0 con 0x7fd7340c7aa0 2024-09-16T00:10:08.720 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:08.721 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:08.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.720+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fd734062b40 0x7fd734062f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.720+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fd73415a4e0 con 0x7fd734062b40 2024-09-16T00:10:08.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.720+0000 7fd738d61640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fd734062b40 0x7fd734062f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.720+0000 7fd738d61640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fd734062b40 0x7fd734062f80 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.721+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd73415a4e0 con 0x7fd734062b40 2024-09-16T00:10:08.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.736+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd7340ca250 con 0x7fd734062b40 2024-09-16T00:10:08.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.737+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd7340ca250 con 0x7fd734062b40 2024-09-16T00:10:08.741 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:08.742 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:08.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.737+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fd7340cab10 0x7fd7340caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.737+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fd7340cb4d0 con 0x7fd7340cab10 2024-09-16T00:10:08.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.738+0000 7fd739d63640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fd7340cab10 0x7fd7340caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.738+0000 7fd739d63640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fd7340cab10 0x7fd7340caf90 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.739+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd72c077e50 con 0x7fd7340cab10 2024-09-16T00:10:08.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.754+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd73402c640 con 0x7fd7340cab10 2024-09-16T00:10:08.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.756+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd73402c640 con 0x7fd7340cab10 2024-09-16T00:10:08.757 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:08.758 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:08.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.757+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fd73402cf10 0x7fd73402d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.757+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fd73402d850 con 0x7fd73402cf10 2024-09-16T00:10:08.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.757+0000 7fd739562640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fd73402cf10 0x7fd73402d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.757+0000 7fd739562640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fd73402cf10 0x7fd73402d310 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.758+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd73402d850 con 0x7fd73402cf10 2024-09-16T00:10:08.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.786+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd73402d850 con 0x7fd73402cf10 2024-09-16T00:10:08.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.788+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd73402d850 con 0x7fd73402cf10 2024-09-16T00:10:08.789 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:08.789 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:08.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.788+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fd73402d850 0x7fd734206fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.788+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fd734209d70 con 0x7fd73402d850 2024-09-16T00:10:08.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.788+0000 7fd738d61640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fd73402d850 0x7fd734206fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.789+0000 7fd738d61640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fd73402d850 0x7fd734206fa0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.789+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd734209d70 con 0x7fd73402d850 2024-09-16T00:10:08.794 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:08.794+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:08.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.804+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd7342074e0 con 0x7fd73402d850 2024-09-16T00:10:08.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.806+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd7342074e0 con 0x7fd73402d850 2024-09-16T00:10:08.807 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:08.808 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:08.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.806+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7fd734207b60 0x7fd734209f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.806+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fd73420cda0 con 0x7fd734207b60 2024-09-16T00:10:08.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.806+0000 7fd739d63640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7fd734207b60 0x7fd734209f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.806+0000 7fd739d63640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7fd734207b60 0x7fd734209f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.808 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:08.808 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:08.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.807+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd73420cda0 con 0x7fd734207b60 2024-09-16T00:10:08.818 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:10:08.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.822+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd73420a760 con 0x7fd734207b60 2024-09-16T00:10:08.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.825+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd73420a760 con 0x7fd734207b60 2024-09-16T00:10:08.826 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:08.826 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:08.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.825+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd73420ae80 0x7fd73420b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:08.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.825+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fd734210140 con 0x7fd73420ae80 2024-09-16T00:10:08.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.825+0000 7fd739562640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd73420ae80 0x7fd73420b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:08.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.825+0000 7fd739562640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd73420ae80 0x7fd73420b2e0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:08.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.826+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd734210140 con 0x7fd73420ae80 2024-09-16T00:10:08.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.842+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd73420b820 con 0x7fd73420ae80 2024-09-16T00:10:08.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd729ffb640 1 -- 172.21.15.139:0/2671632042 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd73420b820 con 0x7fd73420ae80 2024-09-16T00:10:08.845 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:10:08.846 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:10:08.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fd7341e39c0 msgr2=0x7fd7341e6d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fd7341e39c0 0x7fd7341e6d50 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fd7340c7aa0 msgr2=0x7fd7340c7e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fd7340c7aa0 0x7fd7340c7e80 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fd734062b40 msgr2=0x7fd734062f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fd734062b40 0x7fd734062f80 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fd7340cab10 msgr2=0x7fd7340caf90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fd7340cab10 0x7fd7340caf90 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd739d63640 1 -- 172.21.15.139:0/2671632042 reap_dead start 2024-09-16T00:10:08.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fd73402cf10 msgr2=0x7fd73402d310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fd73402cf10 0x7fd73402d310 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd739d63640 1 -- 172.21.15.139:0/2671632042 reap_dead start 2024-09-16T00:10:08.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fd73402d850 msgr2=0x7fd734206fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fd73402d850 0x7fd734206fa0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7fd734207b60 msgr2=0x7fd734209f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7fd734207b60 0x7fd734209f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd73420ae80 msgr2=0x7fd73420b2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fd73420ae80 0x7fd73420b2e0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd71806c2e0 msgr2=0x7fd71806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd71806c2e0 0x7fd71806e7a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd7300090b0 tx=0x7fd73002e000 comp rx=0 tx=0).stop 2024-09-16T00:10:08.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd734071a40 msgr2=0x7fd734151490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:08.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 --2- 172.21.15.139:0/2671632042 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd734071a40 0x7fd734151490 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00c730 tx=0x7fd72c00cbf0 comp rx=0 tx=0).stop 2024-09-16T00:10:08.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd739d63640 1 -- 172.21.15.139:0/2671632042 reap_dead start 2024-09-16T00:10:08.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 shutdown_connections 2024-09-16T00:10:08.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 >> 172.21.15.139:0/2671632042 conn(0x7fd73406fb00 msgr2=0x7fd7340cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:08.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 shutdown_connections 2024-09-16T00:10:08.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:08.844+0000 7fd73b7c6640 1 -- 172.21.15.139:0/2671632042 wait complete. 2024-09-16T00:10:08.896 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:08.895+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:08.954 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:08.955 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:08.963 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:10:08.996 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:08.996+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:09.072 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:09.072 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:09.081 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:10:09.097 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:09.097+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:09.187 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:09.187 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:09.196 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:10:09.198 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:09.198+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:09.299 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:09.298+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:09.303 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:09.303 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:09.312 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:10:09.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.317+0000 7f52917fa640 1 -- 172.21.15.139:0/407238117 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v103) ==== 97+0+0 (secure 0 0 0) 0x7f528007d910 con 0x7f52940c8b50 2024-09-16T00:10:09.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.317+0000 7fbb1a7fc640 1 -- 172.21.15.139:0/1021314376 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v103) ==== 113+0+0 (secure 0 0 0) 0x7fbb14079d70 con 0x7fbb240ccc90 2024-09-16T00:10:09.318 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T00:10:09.319 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is set 2024-09-16T00:10:09.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7f529aeb7640 1 -- 172.21.15.139:0/407238117 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f527806c300 msgr2=0x7f527806e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:09.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7f529aeb7640 1 --2- 172.21.15.139:0/407238117 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f527806c300 0x7f527806e7c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f52880011e0 tx=0x7f5288007d30 comp rx=0 tx=0).stop 2024-09-16T00:10:09.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7f529aeb7640 1 -- 172.21.15.139:0/407238117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52940c8b50 msgr2=0x7f5294150fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:09.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7f529aeb7640 1 --2- 172.21.15.139:0/407238117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52940c8b50 0x7f5294150fe0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5280035040 tx=0x7f52800046c0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7fbb2c18f640 1 -- 172.21.15.139:0/1021314376 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb0806c2b0 msgr2=0x7fbb0806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:09.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7fbb2c18f640 1 --2- 172.21.15.139:0/1021314376 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb0806c2b0 0x7fbb0806e770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbb20000f60 tx=0x7fbb20009040 comp rx=0 tx=0).stop 2024-09-16T00:10:09.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7fbb2c18f640 1 -- 172.21.15.139:0/1021314376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb240ccc90 msgr2=0x7fbb240c9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:09.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7fbb2c18f640 1 --2- 172.21.15.139:0/1021314376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb240ccc90 0x7fbb240c9510 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbb24074130 tx=0x7fbb140046c0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.322 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7f529aeb7640 1 -- 172.21.15.139:0/407238117 shutdown_connections 2024-09-16T00:10:09.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7f529aeb7640 1 --2- 172.21.15.139:0/407238117 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f527806c300 0x7f527806e7c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7f529aeb7640 1 --2- 172.21.15.139:0/407238117 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f529415da90 0x7f5294151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7f529aeb7640 1 --2- 172.21.15.139:0/407238117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52940c8b50 0x7f5294150fe0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7f529aeb7640 1 --2- 172.21.15.139:0/407238117 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5294070d90 0x7f52941f0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7f529aeb7640 1 -- 172.21.15.139:0/407238117 >> 172.21.15.139:0/407238117 conn(0x7f529406c150 msgr2=0x7f5294070410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:09.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7fbb2c18f640 1 -- 172.21.15.139:0/1021314376 shutdown_connections 2024-09-16T00:10:09.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7fbb2c18f640 1 --2- 172.21.15.139:0/1021314376 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb0806c2b0 0x7fbb0806e770 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7fbb2c18f640 1 --2- 172.21.15.139:0/1021314376 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb24154c20 0x7fbb241f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7fbb2c18f640 1 --2- 172.21.15.139:0/1021314376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb240ccc90 0x7fbb240c9510 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7fbb2c18f640 1 --2- 172.21.15.139:0/1021314376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb24068060 0x7fbb2414a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7fbb2c18f640 1 -- 172.21.15.139:0/1021314376 >> 172.21.15.139:0/1021314376 conn(0x7fbb2406ca40 msgr2=0x7fbb240c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:09.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7fbb2c18f640 1 -- 172.21.15.139:0/1021314376 shutdown_connections 2024-09-16T00:10:09.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7fbb2c18f640 1 -- 172.21.15.139:0/1021314376 wait complete. 2024-09-16T00:10:09.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7f529aeb7640 1 -- 172.21.15.139:0/407238117 shutdown_connections 2024-09-16T00:10:09.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.320+0000 7f529aeb7640 1 -- 172.21.15.139:0/407238117 wait complete. 2024-09-16T00:10:09.334 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T00:10:09.334 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:10:09.419 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:09.419+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:09.423 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:09.423 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:09.433 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:10:09.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.474+0000 7ff306718640 1 Processor -- start 2024-09-16T00:10:09.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.474+0000 7ff306718640 1 -- start start 2024-09-16T00:10:09.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.475+0000 7ff306718640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3000cbe50 0x7ff3000cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:09.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.475+0000 7ff306718640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff30006ca30 con 0x7ff3000c7b90 2024-09-16T00:10:09.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.475+0000 7ff306718640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff30015df20 con 0x7ff3000c67e0 2024-09-16T00:10:09.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.475+0000 7ff306718640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff30006cd10 con 0x7ff3000cbe50 2024-09-16T00:10:09.477 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.475+0000 7ff2ff7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff3000c7b90 0x7ff30015a810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45718/0 (socket says 172.21.15.139:45718) 2024-09-16T00:10:09.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.475+0000 7ff2ff7fe640 1 -- 172.21.15.139:0/1979487465 learned_addr learned my addr 172.21.15.139:0/1979487465 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:09.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.475+0000 7ff304cb5640 1 --2- 172.21.15.139:0/1979487465 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3000cbe50 0x7ff3000cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:09.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.475+0000 7ff2feffd640 1 -- 172.21.15.139:0/1979487465 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3106312649 0 0) 0x7ff30006ca30 con 0x7ff3000c7b90 2024-09-16T00:10:09.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.475+0000 7ff2feffd640 1 -- 172.21.15.139:0/1979487465 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2e8003520 con 0x7ff3000c7b90 2024-09-16T00:10:09.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.476+0000 7ff2feffd640 1 -- 172.21.15.139:0/1979487465 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3150992425 0 0) 0x7ff30015df20 con 0x7ff3000c67e0 2024-09-16T00:10:09.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.476+0000 7ff2feffd640 1 -- 172.21.15.139:0/1979487465 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff30006ca30 con 0x7ff3000c67e0 2024-09-16T00:10:09.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.476+0000 7ff304cb5640 1 -- 172.21.15.139:0/1979487465 >> v1:172.21.15.164:6789/0 conn(0x7ff3000c67e0 legacy=0x7ff30006c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:09.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.476+0000 7ff304cb5640 1 -- 172.21.15.139:0/1979487465 >> v1:172.21.15.139:6789/0 conn(0x7ff3000c7b90 legacy=0x7ff30015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:09.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.476+0000 7ff2feffd640 1 -- 172.21.15.139:0/1979487465 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1918437778 0 0) 0x7ff2e8003520 con 0x7ff3000c7b90 2024-09-16T00:10:09.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.476+0000 7ff304cb5640 1 -- 172.21.15.139:0/1979487465 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30015e200 con 0x7ff3000cbe50 2024-09-16T00:10:09.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.476+0000 7ff304cb5640 1 --2- 172.21.15.139:0/1979487465 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3000cbe50 0x7ff3000cc2e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009dc0 tx=0x7ff2f802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b01adcf6a6f32e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:09.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.477+0000 7ff2feffd640 1 -- 172.21.15.139:0/1979487465 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4001670 con 0x7ff3000cbe50 2024-09-16T00:10:09.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.477+0000 7ff2feffd640 1 -- 172.21.15.139:0/1979487465 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f802ae50 con 0x7ff3000cbe50 2024-09-16T00:10:09.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.477+0000 7ff2feffd640 1 -- 172.21.15.139:0/1979487465 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8007740 con 0x7ff3000cbe50 2024-09-16T00:10:09.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.477+0000 7ff306718640 1 -- 172.21.15.139:0/1979487465 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3000cbe50 msgr2=0x7ff3000cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:09.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.477+0000 7ff306718640 1 --2- 172.21.15.139:0/1979487465 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3000cbe50 0x7ff3000cc2e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009dc0 tx=0x7ff2f802a6f0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.477+0000 7ff306718640 1 -- 172.21.15.139:0/1979487465 shutdown_connections 2024-09-16T00:10:09.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.477+0000 7ff306718640 1 --2- 172.21.15.139:0/1979487465 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3000cbe50 0x7ff3000cc2e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.477+0000 7ff306718640 1 -- 172.21.15.139:0/1979487465 >> 172.21.15.139:0/1979487465 conn(0x7ff3000d05f0 msgr2=0x7ff3000d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:09.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.477+0000 7ff306718640 1 -- 172.21.15.139:0/1979487465 shutdown_connections 2024-09-16T00:10:09.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.477+0000 7ff306718640 1 -- 172.21.15.139:0/1979487465 wait complete. 2024-09-16T00:10:09.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.478+0000 7ff306718640 1 Processor -- start 2024-09-16T00:10:09.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.479+0000 7ff306718640 1 -- start start 2024-09-16T00:10:09.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.479+0000 7ff306718640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3000c67e0 0x7ff3001f0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:09.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.479+0000 7ff306718640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3000c7b90 0x7ff3000cf370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:09.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.479+0000 7ff306718640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3000cbe50 0x7ff30014a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:09.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.479+0000 7ff306718640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff3000c7fe0 con 0x7ff3000c7b90 2024-09-16T00:10:09.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.479+0000 7ff306718640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff30015e850 con 0x7ff3000cbe50 2024-09-16T00:10:09.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.479+0000 7ff306718640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff30015af20 con 0x7ff3000c67e0 2024-09-16T00:10:09.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.479+0000 7ff2ff7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3000c7b90 0x7ff3000cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:09.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.479+0000 7ff2ff7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3000c7b90 0x7ff3000cf370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49174/0 (socket says 172.21.15.139:49174) 2024-09-16T00:10:09.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.479+0000 7ff2ff7fe640 1 -- 172.21.15.139:0/2563837891 learned_addr learned my addr 172.21.15.139:0/2563837891 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:09.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.479+0000 7ff2fffff640 1 --2- 172.21.15.139:0/2563837891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3000c67e0 0x7ff3001f0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:09.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.479+0000 7ff304cb5640 1 --2- 172.21.15.139:0/2563837891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3000cbe50 0x7ff30014a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:09.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.480+0000 7ff2ff7fe640 1 -- 172.21.15.139:0/2563837891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3000c67e0 msgr2=0x7ff3001f0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:09.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.480+0000 7ff2ff7fe640 1 --2- 172.21.15.139:0/2563837891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3000c67e0 0x7ff3001f0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.480+0000 7ff2ff7fe640 1 -- 172.21.15.139:0/2563837891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3000cbe50 msgr2=0x7ff30014a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:09.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.480+0000 7ff2ff7fe640 1 --2- 172.21.15.139:0/2563837891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3000cbe50 0x7ff30014a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.480+0000 7ff2ff7fe640 1 -- 172.21.15.139:0/2563837891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3001f11e0 con 0x7ff3000c7b90 2024-09-16T00:10:09.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.480+0000 7ff2ff7fe640 1 --2- 172.21.15.139:0/2563837891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3000c7b90 0x7ff3000cf370 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400c750 tx=0x7ff2f400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:09.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.480+0000 7ff2fcff9640 1 -- 172.21.15.139:0/2563837891 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4017070 con 0x7ff3000c7b90 2024-09-16T00:10:09.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.480+0000 7ff2fcff9640 1 -- 172.21.15.139:0/2563837891 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2f4005ad0 con 0x7ff3000c7b90 2024-09-16T00:10:09.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.480+0000 7ff306718640 1 -- 172.21.15.139:0/2563837891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff300153280 con 0x7ff3000c7b90 2024-09-16T00:10:09.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.480+0000 7ff2fcff9640 1 -- 172.21.15.139:0/2563837891 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4007070 con 0x7ff3000c7b90 2024-09-16T00:10:09.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.480+0000 7ff306718640 1 -- 172.21.15.139:0/2563837891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3001f04a0 con 0x7ff3000c7b90 2024-09-16T00:10:09.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.481+0000 7ff306718640 1 -- 172.21.15.139:0/2563837891 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3001537a0 con 0x7ff3000c7b90 2024-09-16T00:10:09.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.482+0000 7ff2fcff9640 1 -- 172.21.15.139:0/2563837891 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff2f401a070 con 0x7ff3000c7b90 2024-09-16T00:10:09.484 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.482+0000 7ff2fcff9640 1 --2- 172.21.15.139:0/2563837891 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff2ec06c2b0 0x7ff2ec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:09.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.482+0000 7ff2fffff640 1 --2- 172.21.15.139:0/2563837891 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff2ec06c2b0 0x7ff2ec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:09.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.482+0000 7ff2fcff9640 1 -- 172.21.15.139:0/2563837891 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5973+0+0 (secure 0 0 0) 0x7ff2f408a910 con 0x7ff3000c7b90 2024-09-16T00:10:09.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.483+0000 7ff2fffff640 1 --2- 172.21.15.139:0/2563837891 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff2ec06c2b0 0x7ff2ec06e770 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00011e0 tx=0x7ff2f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:09.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.487+0000 7ff2fcff9640 1 -- 172.21.15.139:0/2563837891 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff3001537a0 con 0x7ff3000c7b90 2024-09-16T00:10:09.501 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:09.501+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:09.549 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:09.549 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:09.563 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:10:09.602 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:09.601+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:09.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.662+0000 7ff306718640 1 -- 172.21.15.139:0/2563837891 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3001f0290 con 0x7ff2ec06c2b0 2024-09-16T00:10:09.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.668+0000 7ff2fcff9640 1 -- 172.21.15.139:0/2563837891 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+255469 (secure 0 0 0) 0x7ff3001f0290 con 0x7ff2ec06c2b0 2024-09-16T00:10:09.672 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:10:09.674 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:10:09.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.671+0000 7ff306718640 1 -- 172.21.15.139:0/2563837891 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff2ec06c2b0 msgr2=0x7ff2ec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:09.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.671+0000 7ff306718640 1 --2- 172.21.15.139:0/2563837891 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff2ec06c2b0 0x7ff2ec06e770 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00011e0 tx=0x7ff2f0009040 comp rx=0 tx=0).stop 2024-09-16T00:10:09.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.671+0000 7ff306718640 1 -- 172.21.15.139:0/2563837891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3000c7b90 msgr2=0x7ff3000cf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:09.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.671+0000 7ff306718640 1 --2- 172.21.15.139:0/2563837891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3000c7b90 0x7ff3000cf370 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400c750 tx=0x7ff2f400cc10 comp rx=0 tx=0).stop 2024-09-16T00:10:09.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.671+0000 7ff306718640 1 -- 172.21.15.139:0/2563837891 shutdown_connections 2024-09-16T00:10:09.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.671+0000 7ff306718640 1 --2- 172.21.15.139:0/2563837891 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff2ec06c2b0 0x7ff2ec06e770 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.671+0000 7ff306718640 1 --2- 172.21.15.139:0/2563837891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff3000cbe50 0x7ff30014a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.671+0000 7ff306718640 1 --2- 172.21.15.139:0/2563837891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3000c7b90 0x7ff3000cf370 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.671+0000 7ff306718640 1 --2- 172.21.15.139:0/2563837891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff3000c67e0 0x7ff3001f0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.671+0000 7ff306718640 1 -- 172.21.15.139:0/2563837891 >> 172.21.15.139:0/2563837891 conn(0x7ff3000d05f0 msgr2=0x7ff3000c4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:09.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.672+0000 7ff306718640 1 -- 172.21.15.139:0/2563837891 shutdown_connections 2024-09-16T00:10:09.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.672+0000 7ff306718640 1 -- 172.21.15.139:0/2563837891 wait complete. 2024-09-16T00:10:09.684 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-16T00:10:09.319018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":349240153,"num_objects":10802,"num_object_clones":0,"num_object_copies":21604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10802,"num_whiteouts":0,"num_read":2733,"num_read_kb":1362337,"num_write":17810,"num_write_kb":4175218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":452,"num_bytes_recovered":292544036,"num_keys_recovered":1983,"num_objects_omap":78,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19399,"ondisk_log_size":19399,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":534836,"kb_used_data":256728,"kb_used_omap":232,"kb_used_meta":277719,"kb_avail":754439884,"statfs":{"total":773094113280,"available":772546441216,"internally_reserved":0,"allocated":262889472,"data_stored":999459042,"data_compressed":75623551,"data_compressed_allocated":97230848,"data_compressed_original":890647543,"omap_allocated":238291,"internal_metadata":284384557},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-898304194,"num_objects":-345,"num_object_clones":0,"num_object_copies":-690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":473,"num_write_kb":2138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.028875"},"pg_stats":[{"pgid":"3.3f","version":"100'267","reported_seq":554,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.884488+0000","last_change":"2024-09-16T00:10:00.622682+0000","last_active":"2024-09-16T00:10:06.884488+0000","last_peered":"2024-09-16T00:10:06.884488+0000","last_clean":"2024-09-16T00:10:06.884488+0000","last_became_active":"2024-09-16T00:10:00.286893+0000","last_became_peered":"2024-09-16T00:10:00.286893+0000","last_unstale":"2024-09-16T00:10:06.884488+0000","last_undegraded":"2024-09-16T00:10:06.884488+0000","last_fullsized":"2024-09-16T00:10:06.884488+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:21.452686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":562528,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":64,"num_read_kb":42086,"num_write":242,"num_write_kb":112716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":31107821,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"99'238","reported_seq":429,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.897532+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:10:05.897532+0000","last_peered":"2024-09-16T00:10:05.897532+0000","last_clean":"2024-09-16T00:10:05.897532+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:10:05.897532+0000","last_undegraded":"2024-09-16T00:10:05.897532+0000","last_fullsized":"2024-09-16T00:10:05.897532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4995574,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":25,"num_read_kb":23064,"num_write":220,"num_write_kb":47220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"99'208","reported_seq":404,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.861931+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:10:05.861931+0000","last_peered":"2024-09-16T00:10:05.861931+0000","last_clean":"2024-09-16T00:10:05.861931+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:10:05.861931+0000","last_undegraded":"2024-09-16T00:10:05.861931+0000","last_fullsized":"2024-09-16T00:10:05.861931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9209061,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":30,"num_read_kb":15782,"num_write":193,"num_write_kb":54932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'225","reported_seq":424,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.883945+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:10:06.883945+0000","last_peered":"2024-09-16T00:10:06.883945+0000","last_clean":"2024-09-16T00:10:06.883945+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:10:06.883945+0000","last_undegraded":"2024-09-16T00:10:06.883945+0000","last_fullsized":"2024-09-16T00:10:06.883945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":651768,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":30,"num_read_kb":14080,"num_write":204,"num_write_kb":63191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"99'267","reported_seq":476,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.899986+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:10:05.899986+0000","last_peered":"2024-09-16T00:10:05.899986+0000","last_clean":"2024-09-16T00:10:05.899986+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:10:05.899986+0000","last_undegraded":"2024-09-16T00:10:05.899986+0000","last_fullsized":"2024-09-16T00:10:05.899986+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8067643,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":69,"num_read_kb":36096,"num_write":252,"num_write_kb":89616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"98'213","reported_seq":733,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.864474+0000","last_change":"2024-09-16T00:10:00.875898+0000","last_active":"2024-09-16T00:10:04.864474+0000","last_peered":"2024-09-16T00:10:04.864474+0000","last_clean":"2024-09-16T00:10:04.864474+0000","last_became_active":"2024-09-16T00:10:00.278705+0000","last_became_peered":"2024-09-16T00:10:00.278705+0000","last_unstale":"2024-09-16T00:10:04.864474+0000","last_undegraded":"2024-09-16T00:10:04.864474+0000","last_fullsized":"2024-09-16T00:10:04.864474+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:45:25.113058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7455179,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":16,"num_read_kb":6174,"num_write":196,"num_write_kb":50827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":23596926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"100'246","reported_seq":447,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.869376+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:10:06.869376+0000","last_peered":"2024-09-16T00:10:06.869376+0000","last_clean":"2024-09-16T00:10:06.869376+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:10:06.869376+0000","last_undegraded":"2024-09-16T00:10:06.869376+0000","last_fullsized":"2024-09-16T00:10:06.869376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1669868,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":34,"num_read_kb":16577,"num_write":228,"num_write_kb":64470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"98'220","reported_seq":391,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.857892+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:10:04.857892+0000","last_peered":"2024-09-16T00:10:04.857892+0000","last_clean":"2024-09-16T00:10:04.857892+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:10:04.857892+0000","last_undegraded":"2024-09-16T00:10:04.857892+0000","last_fullsized":"2024-09-16T00:10:04.857892+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15472397,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":33,"num_read_kb":16776,"num_write":197,"num_write_kb":80164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"100'221","reported_seq":395,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.900265+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:10:06.900265+0000","last_peered":"2024-09-16T00:10:06.900265+0000","last_clean":"2024-09-16T00:10:06.900265+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:10:06.900265+0000","last_undegraded":"2024-09-16T00:10:06.900265+0000","last_fullsized":"2024-09-16T00:10:06.900265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1173835,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":7,"num_read_kb":1432,"num_write":195,"num_write_kb":30311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"101'214","reported_seq":295,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.878457+0000","last_change":"2024-09-16T00:10:00.734336+0000","last_active":"2024-09-16T00:10:07.878457+0000","last_peered":"2024-09-16T00:10:07.878457+0000","last_clean":"2024-09-16T00:10:07.878457+0000","last_became_active":"2024-09-16T00:10:00.284452+0000","last_became_peered":"2024-09-16T00:10:00.284452+0000","last_unstale":"2024-09-16T00:10:07.878457+0000","last_undegraded":"2024-09-16T00:10:07.878457+0000","last_fullsized":"2024-09-16T00:10:07.878457+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:56:50.530439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":656293,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":6,"num_read_kb":4096,"num_write":182,"num_write_kb":29952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":8736255,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"101'198","reported_seq":290,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.870666+0000","last_change":"2024-09-16T00:10:00.587110+0000","last_active":"2024-09-16T00:10:07.870666+0000","last_peered":"2024-09-16T00:10:07.870666+0000","last_clean":"2024-09-16T00:10:07.870666+0000","last_became_active":"2024-09-16T00:10:00.280273+0000","last_became_peered":"2024-09-16T00:10:00.280273+0000","last_unstale":"2024-09-16T00:10:07.870666+0000","last_undegraded":"2024-09-16T00:10:07.870666+0000","last_fullsized":"2024-09-16T00:10:07.870666+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:01:57.902939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":581234,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":6,"num_read_kb":975,"num_write":182,"num_write_kb":25518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":10753732,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"100'219","reported_seq":466,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.858146+0000","last_change":"2024-09-16T00:10:00.797372+0000","last_active":"2024-09-16T00:10:06.858146+0000","last_peered":"2024-09-16T00:10:06.858146+0000","last_clean":"2024-09-16T00:10:06.858146+0000","last_became_active":"2024-09-16T00:10:00.278765+0000","last_became_peered":"2024-09-16T00:10:00.278765+0000","last_unstale":"2024-09-16T00:10:06.858146+0000","last_undegraded":"2024-09-16T00:10:06.858146+0000","last_fullsized":"2024-09-16T00:10:06.858146+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:03.318919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":580504,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":30,"num_read_kb":11563,"num_write":201,"num_write_kb":58933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":29565877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'256","reported_seq":527,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.900879+0000","last_change":"2024-09-16T00:10:00.715647+0000","last_active":"2024-09-16T00:10:05.900879+0000","last_peered":"2024-09-16T00:10:05.900879+0000","last_clean":"2024-09-16T00:10:05.900879+0000","last_became_active":"2024-09-16T00:10:00.286050+0000","last_became_peered":"2024-09-16T00:10:00.286050+0000","last_unstale":"2024-09-16T00:10:05.900879+0000","last_undegraded":"2024-09-16T00:10:05.900879+0000","last_fullsized":"2024-09-16T00:10:05.900879+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:07.462899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5614093,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":56,"num_read_kb":35908,"num_write":231,"num_write_kb":104863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":30230864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"99'234","reported_seq":449,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.897499+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:10:05.897499+0000","last_peered":"2024-09-16T00:10:05.897499+0000","last_clean":"2024-09-16T00:10:05.897499+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:10:05.897499+0000","last_undegraded":"2024-09-16T00:10:05.897499+0000","last_fullsized":"2024-09-16T00:10:05.897499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20189743,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":49,"num_read_kb":23364,"num_write":212,"num_write_kb":43321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'219","reported_seq":418,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.872775+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:10:05.872775+0000","last_peered":"2024-09-16T00:10:05.872775+0000","last_clean":"2024-09-16T00:10:05.872775+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:10:05.872775+0000","last_undegraded":"2024-09-16T00:10:05.872775+0000","last_fullsized":"2024-09-16T00:10:05.872775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4753934,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":33,"num_read_kb":14230,"num_write":194,"num_write_kb":33768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"100'257","reported_seq":483,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.871621+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:10:06.871621+0000","last_peered":"2024-09-16T00:10:06.871621+0000","last_clean":"2024-09-16T00:10:06.871621+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:10:06.871621+0000","last_undegraded":"2024-09-16T00:10:06.871621+0000","last_fullsized":"2024-09-16T00:10:06.871621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":900420,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":57,"num_read_kb":40234,"num_write":234,"num_write_kb":67785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"100'227","reported_seq":449,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.899718+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:10:06.899718+0000","last_peered":"2024-09-16T00:10:06.899718+0000","last_clean":"2024-09-16T00:10:06.899718+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:10:06.899718+0000","last_undegraded":"2024-09-16T00:10:06.899718+0000","last_fullsized":"2024-09-16T00:10:06.899718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":593433,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":53,"num_read_kb":22524,"num_write":208,"num_write_kb":46842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"98'214","reported_seq":448,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.856695+0000","last_change":"2024-09-16T00:10:00.461437+0000","last_active":"2024-09-16T00:10:04.856695+0000","last_peered":"2024-09-16T00:10:04.856695+0000","last_clean":"2024-09-16T00:10:04.856695+0000","last_became_active":"2024-09-16T00:10:00.282592+0000","last_became_peered":"2024-09-16T00:10:00.282592+0000","last_unstale":"2024-09-16T00:10:04.856695+0000","last_undegraded":"2024-09-16T00:10:04.856695+0000","last_fullsized":"2024-09-16T00:10:04.856695+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:09.733353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15678980,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":11,"num_read_kb":8711,"num_write":198,"num_write_kb":46855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":21285626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"100'252","reported_seq":509,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.896695+0000","last_change":"2024-09-16T00:09:04.210763+0000","last_active":"2024-09-16T00:10:06.896695+0000","last_peered":"2024-09-16T00:10:06.896695+0000","last_clean":"2024-09-16T00:10:06.896695+0000","last_became_active":"2024-09-16T00:09:04.158723+0000","last_became_peered":"2024-09-16T00:09:04.158723+0000","last_unstale":"2024-09-16T00:10:06.896695+0000","last_undegraded":"2024-09-16T00:10:06.896695+0000","last_fullsized":"2024-09-16T00:10:06.896695+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:25:48.973773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9214973,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":25,"num_read_kb":14437,"num_write":232,"num_write_kb":83979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":103497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"99'236","reported_seq":327,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.763404+0000","last_change":"2024-09-16T00:10:00.566870+0000","last_active":"2024-09-16T00:10:07.763404+0000","last_peered":"2024-09-16T00:10:07.763404+0000","last_clean":"2024-09-16T00:10:07.763404+0000","last_became_active":"2024-09-16T00:10:00.281258+0000","last_became_peered":"2024-09-16T00:10:00.281258+0000","last_unstale":"2024-09-16T00:10:07.763404+0000","last_undegraded":"2024-09-16T00:10:07.763404+0000","last_fullsized":"2024-09-16T00:10:07.763404+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:29.197743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":670826,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":13,"num_read_kb":15502,"num_write":219,"num_write_kb":30222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":17199573,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"99'240","reported_seq":461,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.899321+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:10:05.899321+0000","last_peered":"2024-09-16T00:10:05.899321+0000","last_clean":"2024-09-16T00:10:05.899321+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:10:05.899321+0000","last_undegraded":"2024-09-16T00:10:05.899321+0000","last_fullsized":"2024-09-16T00:10:05.899321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16302996,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":55,"num_read_kb":34318,"num_write":224,"num_write_kb":91769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"100'270","reported_seq":501,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.898232+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:10:06.898232+0000","last_peered":"2024-09-16T00:10:06.898232+0000","last_clean":"2024-09-16T00:10:06.898232+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:10:06.898232+0000","last_undegraded":"2024-09-16T00:10:06.898232+0000","last_fullsized":"2024-09-16T00:10:06.898232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5069924,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":62,"num_read_kb":35528,"num_write":249,"num_write_kb":92852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'226","reported_seq":420,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.897517+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:10:05.897517+0000","last_peered":"2024-09-16T00:10:05.897517+0000","last_clean":"2024-09-16T00:10:05.897517+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:10:05.897517+0000","last_undegraded":"2024-09-16T00:10:05.897517+0000","last_fullsized":"2024-09-16T00:10:05.897517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4967085,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":27,"num_read_kb":13410,"num_write":202,"num_write_kb":58805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'237","reported_seq":428,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.903078+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:10:06.903078+0000","last_peered":"2024-09-16T00:10:06.903078+0000","last_clean":"2024-09-16T00:10:06.903078+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:10:06.903078+0000","last_undegraded":"2024-09-16T00:10:06.903078+0000","last_fullsized":"2024-09-16T00:10:06.903078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":781271,"num_objects":184,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":0,"num_read":22,"num_read_kb":12682,"num_write":215,"num_write_kb":54869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"99'237","reported_seq":434,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.867474+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:10:05.867474+0000","last_peered":"2024-09-16T00:10:05.867474+0000","last_clean":"2024-09-16T00:10:05.867474+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:10:05.867474+0000","last_undegraded":"2024-09-16T00:10:05.867474+0000","last_fullsized":"2024-09-16T00:10:05.867474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":801473,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":30,"num_read_kb":14188,"num_write":227,"num_write_kb":59178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"101'257","reported_seq":398,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.885032+0000","last_change":"2024-09-16T00:10:00.794575+0000","last_active":"2024-09-16T00:10:07.885032+0000","last_peered":"2024-09-16T00:10:07.885032+0000","last_clean":"2024-09-16T00:10:07.885032+0000","last_became_active":"2024-09-16T00:10:00.285311+0000","last_became_peered":"2024-09-16T00:10:00.285311+0000","last_unstale":"2024-09-16T00:10:07.885032+0000","last_undegraded":"2024-09-16T00:10:07.885032+0000","last_fullsized":"2024-09-16T00:10:07.885032+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:13:49.591876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1705691,"num_objects":191,"num_object_clones":0,"num_object_copies":382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":191,"num_whiteouts":0,"num_read":27,"num_read_kb":12330,"num_write":230,"num_write_kb":35146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":12903786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'238","reported_seq":450,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.882477+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:10:05.882477+0000","last_peered":"2024-09-16T00:10:05.882477+0000","last_clean":"2024-09-16T00:10:05.882477+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:10:05.882477+0000","last_undegraded":"2024-09-16T00:10:05.882477+0000","last_fullsized":"2024-09-16T00:10:05.882477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4917795,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":46,"num_read_kb":21104,"num_write":221,"num_write_kb":75742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"99'224","reported_seq":432,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.872807+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:10:05.872807+0000","last_peered":"2024-09-16T00:10:05.872807+0000","last_clean":"2024-09-16T00:10:05.872807+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:10:05.872807+0000","last_undegraded":"2024-09-16T00:10:05.872807+0000","last_fullsized":"2024-09-16T00:10:05.872807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2134022,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":41,"num_read_kb":16408,"num_write":208,"num_write_kb":51674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"99'224","reported_seq":449,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.882793+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:10:05.882793+0000","last_peered":"2024-09-16T00:10:05.882793+0000","last_clean":"2024-09-16T00:10:05.882793+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:10:05.882793+0000","last_undegraded":"2024-09-16T00:10:05.882793+0000","last_fullsized":"2024-09-16T00:10:05.882793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9063115,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":59,"num_read_kb":33174,"num_write":205,"num_write_kb":66849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"98'283","reported_seq":481,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.874916+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:10:04.874916+0000","last_peered":"2024-09-16T00:10:04.874916+0000","last_clean":"2024-09-16T00:10:04.874916+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:10:04.874916+0000","last_undegraded":"2024-09-16T00:10:04.874916+0000","last_fullsized":"2024-09-16T00:10:04.874916+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5422585,"num_objects":194,"num_object_clones":0,"num_object_copies":388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":194,"num_whiteouts":0,"num_read":60,"num_read_kb":27231,"num_write":259,"num_write_kb":104771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"98'271","reported_seq":514,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.298030+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:10:04.298030+0000","last_peered":"2024-09-16T00:10:04.298030+0000","last_clean":"2024-09-16T00:10:04.298030+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:10:04.298030+0000","last_undegraded":"2024-09-16T00:10:04.298030+0000","last_fullsized":"2024-09-16T00:10:04.298030+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4869645,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":73,"num_read_kb":35655,"num_write":256,"num_write_kb":97245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"101'241","reported_seq":334,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.763190+0000","last_change":"2024-09-16T00:10:00.535553+0000","last_active":"2024-09-16T00:10:07.763190+0000","last_peered":"2024-09-16T00:10:07.763190+0000","last_clean":"2024-09-16T00:10:07.763190+0000","last_became_active":"2024-09-16T00:10:00.281378+0000","last_became_peered":"2024-09-16T00:10:00.281378+0000","last_unstale":"2024-09-16T00:10:07.763190+0000","last_undegraded":"2024-09-16T00:10:07.763190+0000","last_fullsized":"2024-09-16T00:10:07.763190+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:49.548709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2880131,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":32,"num_read_kb":20395,"num_write":221,"num_write_kb":32731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":22876917,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"100'234","reported_seq":482,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.905306+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:10:06.905306+0000","last_peered":"2024-09-16T00:10:06.905306+0000","last_clean":"2024-09-16T00:10:06.905306+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:10:06.905306+0000","last_undegraded":"2024-09-16T00:10:06.905306+0000","last_fullsized":"2024-09-16T00:10:06.905306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538720,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":80,"num_read_kb":48141,"num_write":217,"num_write_kb":63611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"99'71","reported_seq":240,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.869516+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:10:05.869516+0000","last_peered":"2024-09-16T00:10:05.869516+0000","last_clean":"2024-09-16T00:10:05.869516+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:10:05.869516+0000","last_undegraded":"2024-09-16T00:10:05.869516+0000","last_fullsized":"2024-09-16T00:10:05.869516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"99'198","reported_seq":391,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.891264+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:10:05.891264+0000","last_peered":"2024-09-16T00:10:05.891264+0000","last_clean":"2024-09-16T00:10:05.891264+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:10:05.891264+0000","last_undegraded":"2024-09-16T00:10:05.891264+0000","last_fullsized":"2024-09-16T00:10:05.891264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718052,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":27,"num_read_kb":16420,"num_write":181,"num_write_kb":62710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"101'55","reported_seq":132,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.763450+0000","last_change":"2024-09-16T00:10:00.282958+0000","last_active":"2024-09-16T00:10:07.763450+0000","last_peered":"2024-09-16T00:10:07.763450+0000","last_clean":"2024-09-16T00:10:07.763450+0000","last_became_active":"2024-09-16T00:10:00.282541+0000","last_became_peered":"2024-09-16T00:10:00.282541+0000","last_unstale":"2024-09-16T00:10:07.763450+0000","last_undegraded":"2024-09-16T00:10:07.763450+0000","last_fullsized":"2024-09-16T00:10:07.763450+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:51.198787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":4157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"98'20","reported_seq":210,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.291497+0000","last_change":"2024-09-16T00:10:00.313589+0000","last_active":"2024-09-16T00:10:05.291497+0000","last_peered":"2024-09-16T00:10:05.291497+0000","last_clean":"2024-09-16T00:10:05.291497+0000","last_became_active":"2024-09-16T00:10:00.284602+0000","last_became_peered":"2024-09-16T00:10:00.284602+0000","last_unstale":"2024-09-16T00:10:05.291497+0000","last_undegraded":"2024-09-16T00:10:05.291497+0000","last_fullsized":"2024-09-16T00:10:05.291497+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:30.032698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":14,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"99'250","reported_seq":455,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.890409+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:10:05.890409+0000","last_peered":"2024-09-16T00:10:05.890409+0000","last_clean":"2024-09-16T00:10:05.890409+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:10:05.890409+0000","last_undegraded":"2024-09-16T00:10:05.890409+0000","last_fullsized":"2024-09-16T00:10:05.890409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1102178,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":38,"num_read_kb":18480,"num_write":230,"num_write_kb":71748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"100'781","reported_seq":952,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.308153+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:10:06.308153+0000","last_peered":"2024-09-16T00:10:06.308153+0000","last_clean":"2024-09-16T00:10:06.308153+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:10:06.308153+0000","last_undegraded":"2024-09-16T00:10:06.308153+0000","last_fullsized":"2024-09-16T00:10:06.308153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":766,"num_write_kb":8591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'249","reported_seq":505,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.858711+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:10:05.858711+0000","last_peered":"2024-09-16T00:10:05.858711+0000","last_clean":"2024-09-16T00:10:05.858711+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:10:05.858711+0000","last_undegraded":"2024-09-16T00:10:05.858711+0000","last_fullsized":"2024-09-16T00:10:05.858711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5073652,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":90,"num_read_kb":46871,"num_write":229,"num_write_kb":84187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"101'23","reported_seq":166,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.763556+0000","last_change":"2024-09-16T00:10:00.283876+0000","last_active":"2024-09-16T00:10:07.763556+0000","last_peered":"2024-09-16T00:10:07.763556+0000","last_clean":"2024-09-16T00:10:07.763556+0000","last_became_active":"2024-09-16T00:10:00.283555+0000","last_became_peered":"2024-09-16T00:10:00.283555+0000","last_unstale":"2024-09-16T00:10:07.763556+0000","last_undegraded":"2024-09-16T00:10:07.763556+0000","last_fullsized":"2024-09-16T00:10:07.763556+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:48.543737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"100'263","reported_seq":467,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.876189+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:10:06.876189+0000","last_peered":"2024-09-16T00:10:06.876189+0000","last_clean":"2024-09-16T00:10:06.876189+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:10:06.876189+0000","last_undegraded":"2024-09-16T00:10:06.876189+0000","last_fullsized":"2024-09-16T00:10:06.876189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":952497,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":35,"num_read_kb":16686,"num_write":238,"num_write_kb":80112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"100'11","reported_seq":189,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.308858+0000","last_change":"2024-09-16T00:10:00.285386+0000","last_active":"2024-09-16T00:10:06.308858+0000","last_peered":"2024-09-16T00:10:06.308858+0000","last_clean":"2024-09-16T00:10:06.308858+0000","last_became_active":"2024-09-16T00:10:00.284717+0000","last_became_peered":"2024-09-16T00:10:00.284717+0000","last_unstale":"2024-09-16T00:10:06.308858+0000","last_undegraded":"2024-09-16T00:10:06.308858+0000","last_fullsized":"2024-09-16T00:10:06.308858+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:07.794163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"99'218","reported_seq":427,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.870657+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:10:05.870657+0000","last_peered":"2024-09-16T00:10:05.870657+0000","last_clean":"2024-09-16T00:10:05.870657+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:10:05.870657+0000","last_undegraded":"2024-09-16T00:10:05.870657+0000","last_fullsized":"2024-09-16T00:10:05.870657+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9495587,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":43,"num_read_kb":18030,"num_write":200,"num_write_kb":55428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"99'51","reported_seq":219,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.901383+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:10:05.901383+0000","last_peered":"2024-09-16T00:10:05.901383+0000","last_clean":"2024-09-16T00:10:05.901383+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:10:05.901383+0000","last_undegraded":"2024-09-16T00:10:05.901383+0000","last_fullsized":"2024-09-16T00:10:05.901383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"100'219","reported_seq":434,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.867932+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:10:06.867932+0000","last_peered":"2024-09-16T00:10:06.867932+0000","last_clean":"2024-09-16T00:10:06.867932+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:10:06.867932+0000","last_undegraded":"2024-09-16T00:10:06.867932+0000","last_fullsized":"2024-09-16T00:10:06.867932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1018812,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":47,"num_read_kb":22960,"num_write":195,"num_write_kb":63161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"101'42","reported_seq":130,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.763354+0000","last_change":"2024-09-16T00:10:00.300895+0000","last_active":"2024-09-16T00:10:07.763354+0000","last_peered":"2024-09-16T00:10:07.763354+0000","last_clean":"2024-09-16T00:10:07.763354+0000","last_became_active":"2024-09-16T00:10:00.283474+0000","last_became_peered":"2024-09-16T00:10:00.283474+0000","last_unstale":"2024-09-16T00:10:07.763354+0000","last_undegraded":"2024-09-16T00:10:07.763354+0000","last_fullsized":"2024-09-16T00:10:07.763354+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:28:02.563817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":13200,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"100'242","reported_seq":455,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.902727+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:10:06.902727+0000","last_peered":"2024-09-16T00:10:06.902727+0000","last_clean":"2024-09-16T00:10:06.902727+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:10:06.902727+0000","last_undegraded":"2024-09-16T00:10:06.902727+0000","last_fullsized":"2024-09-16T00:10:06.902727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":744709,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":46,"num_read_kb":23285,"num_write":212,"num_write_kb":67093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"98'428","reported_seq":596,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.542303+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:10:05.542303+0000","last_peered":"2024-09-16T00:10:05.542303+0000","last_clean":"2024-09-16T00:10:05.542303+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:10:05.542303+0000","last_undegraded":"2024-09-16T00:10:05.542303+0000","last_fullsized":"2024-09-16T00:10:05.542303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":415,"num_write_kb":8413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"99'229","reported_seq":423,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.881557+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:10:05.881557+0000","last_peered":"2024-09-16T00:10:05.881557+0000","last_clean":"2024-09-16T00:10:05.881557+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:10:05.881557+0000","last_undegraded":"2024-09-16T00:10:05.881557+0000","last_fullsized":"2024-09-16T00:10:05.881557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6060822,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":28,"num_read_kb":13263,"num_write":207,"num_write_kb":51174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"100'19","reported_seq":190,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.881888+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:10:06.881888+0000","last_peered":"2024-09-16T00:10:06.881888+0000","last_clean":"2024-09-16T00:10:06.881888+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:10:06.881888+0000","last_undegraded":"2024-09-16T00:10:06.881888+0000","last_fullsized":"2024-09-16T00:10:06.881888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'231","reported_seq":428,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.860806+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:10:06.860806+0000","last_peered":"2024-09-16T00:10:06.860806+0000","last_clean":"2024-09-16T00:10:06.860806+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:10:06.860806+0000","last_undegraded":"2024-09-16T00:10:06.860806+0000","last_fullsized":"2024-09-16T00:10:06.860806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5759002,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":30,"num_read_kb":13990,"num_write":207,"num_write_kb":63962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"100'30","reported_seq":90,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.763159+0000","last_change":"2024-09-16T00:10:00.294519+0000","last_active":"2024-09-16T00:10:07.763159+0000","last_peered":"2024-09-16T00:10:07.763159+0000","last_clean":"2024-09-16T00:10:07.763159+0000","last_became_active":"2024-09-16T00:10:00.283066+0000","last_became_peered":"2024-09-16T00:10:00.283066+0000","last_unstale":"2024-09-16T00:10:07.763159+0000","last_undegraded":"2024-09-16T00:10:07.763159+0000","last_fullsized":"2024-09-16T00:10:07.763159+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:32.621176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"101'249","reported_seq":344,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.870945+0000","last_change":"2024-09-16T00:10:00.460910+0000","last_active":"2024-09-16T00:10:07.870945+0000","last_peered":"2024-09-16T00:10:07.870945+0000","last_clean":"2024-09-16T00:10:07.870945+0000","last_became_active":"2024-09-16T00:10:00.280750+0000","last_became_peered":"2024-09-16T00:10:00.280750+0000","last_unstale":"2024-09-16T00:10:07.870945+0000","last_undegraded":"2024-09-16T00:10:07.870945+0000","last_fullsized":"2024-09-16T00:10:07.870945+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:47.634650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":957990,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":35,"num_read_kb":19573,"num_write":227,"num_write_kb":50826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":33491520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'23","reported_seq":190,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.298071+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:10:04.298071+0000","last_peered":"2024-09-16T00:10:04.298071+0000","last_clean":"2024-09-16T00:10:04.298071+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:10:04.298071+0000","last_undegraded":"2024-09-16T00:10:04.298071+0000","last_fullsized":"2024-09-16T00:10:04.298071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"100'264","reported_seq":463,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.327916+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:10:06.327916+0000","last_peered":"2024-09-16T00:10:06.327916+0000","last_clean":"2024-09-16T00:10:06.327916+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:10:06.327916+0000","last_undegraded":"2024-09-16T00:10:06.327916+0000","last_fullsized":"2024-09-16T00:10:06.327916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":889252,"num_objects":192,"num_object_clones":0,"num_object_copies":384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":32,"num_read_kb":20419,"num_write":235,"num_write_kb":43035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"100'22","reported_seq":192,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.861832+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:10:06.861832+0000","last_peered":"2024-09-16T00:10:06.861832+0000","last_clean":"2024-09-16T00:10:06.861832+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:10:06.861832+0000","last_undegraded":"2024-09-16T00:10:06.861832+0000","last_fullsized":"2024-09-16T00:10:06.861832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"100'208","reported_seq":403,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.893550+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:10:06.893550+0000","last_peered":"2024-09-16T00:10:06.893550+0000","last_clean":"2024-09-16T00:10:06.893550+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:10:06.893550+0000","last_undegraded":"2024-09-16T00:10:06.893550+0000","last_fullsized":"2024-09-16T00:10:06.893550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":573831,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":28,"num_read_kb":10790,"num_write":180,"num_write_kb":21585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"98'243","reported_seq":410,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.903119+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:10:04.903119+0000","last_peered":"2024-09-16T00:10:04.903119+0000","last_clean":"2024-09-16T00:10:04.903119+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:10:04.903119+0000","last_undegraded":"2024-09-16T00:10:04.903119+0000","last_fullsized":"2024-09-16T00:10:04.903119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":4238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"99'210","reported_seq":382,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.328322+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:10:06.328322+0000","last_peered":"2024-09-16T00:10:06.328322+0000","last_clean":"2024-09-16T00:10:06.328322+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:10:06.328322+0000","last_undegraded":"2024-09-16T00:10:06.328322+0000","last_fullsized":"2024-09-16T00:10:06.328322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582034,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":5,"num_read_kb":350,"num_write":191,"num_write_kb":38221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"99'490","reported_seq":658,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.875825+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:10:05.875825+0000","last_peered":"2024-09-16T00:10:05.875825+0000","last_clean":"2024-09-16T00:10:05.875825+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:10:05.875825+0000","last_undegraded":"2024-09-16T00:10:05.875825+0000","last_fullsized":"2024-09-16T00:10:05.875825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":472,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"100'237","reported_seq":410,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.890683+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:10:06.890683+0000","last_peered":"2024-09-16T00:10:06.890683+0000","last_clean":"2024-09-16T00:10:06.890683+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:10:06.890683+0000","last_undegraded":"2024-09-16T00:10:06.890683+0000","last_fullsized":"2024-09-16T00:10:06.890683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5326874,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":4,"num_read_kb":675,"num_write":209,"num_write_kb":34696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"98'18","reported_seq":185,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.298225+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:10:04.298225+0000","last_peered":"2024-09-16T00:10:04.298225+0000","last_clean":"2024-09-16T00:10:04.298225+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:10:04.298225+0000","last_undegraded":"2024-09-16T00:10:04.298225+0000","last_fullsized":"2024-09-16T00:10:04.298225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'226","reported_seq":449,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.891625+0000","last_change":"2024-09-16T00:08:47.953122+0000","last_active":"2024-09-16T00:10:06.891625+0000","last_peered":"2024-09-16T00:10:06.891625+0000","last_clean":"2024-09-16T00:10:06.891625+0000","last_became_active":"2024-09-16T00:08:47.952799+0000","last_became_peered":"2024-09-16T00:08:47.952799+0000","last_unstale":"2024-09-16T00:10:06.891625+0000","last_undegraded":"2024-09-16T00:10:06.891625+0000","last_fullsized":"2024-09-16T00:10:06.891625+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:03.447562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":913003,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":46,"num_read_kb":22590,"num_write":210,"num_write_kb":59088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":325,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.327921+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:10:06.327921+0000","last_peered":"2024-09-16T00:10:06.327921+0000","last_clean":"2024-09-16T00:10:06.327921+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:10:06.327921+0000","last_undegraded":"2024-09-16T00:10:06.327921+0000","last_fullsized":"2024-09-16T00:10:06.327921+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"100'13","reported_seq":184,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.853612+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:10:06.853612+0000","last_peered":"2024-09-16T00:10:06.853612+0000","last_clean":"2024-09-16T00:10:06.853612+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:10:06.853612+0000","last_undegraded":"2024-09-16T00:10:06.853612+0000","last_fullsized":"2024-09-16T00:10:06.853612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"99'215","reported_seq":416,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.907475+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:10:05.907475+0000","last_peered":"2024-09-16T00:10:05.907475+0000","last_clean":"2024-09-16T00:10:05.907475+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:10:05.907475+0000","last_undegraded":"2024-09-16T00:10:05.907475+0000","last_fullsized":"2024-09-16T00:10:05.907475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":668263,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":34,"num_read_kb":16103,"num_write":188,"num_write_kb":46241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"100'354","reported_seq":523,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.881764+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:10:06.881764+0000","last_peered":"2024-09-16T00:10:06.881764+0000","last_clean":"2024-09-16T00:10:06.881764+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:10:06.881764+0000","last_undegraded":"2024-09-16T00:10:06.881764+0000","last_fullsized":"2024-09-16T00:10:06.881764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":8359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"99'235","reported_seq":448,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.875264+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:10:05.875264+0000","last_peered":"2024-09-16T00:10:05.875264+0000","last_clean":"2024-09-16T00:10:05.875264+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:10:05.875264+0000","last_undegraded":"2024-09-16T00:10:05.875264+0000","last_fullsized":"2024-09-16T00:10:05.875264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4986232,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":47,"num_read_kb":29385,"num_write":209,"num_write_kb":83610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"98'68","reported_seq":291,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.804261+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:10:04.804261+0000","last_peered":"2024-09-16T00:10:04.804261+0000","last_clean":"2024-09-16T00:10:04.804261+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:10:04.804261+0000","last_undegraded":"2024-09-16T00:10:04.804261+0000","last_fullsized":"2024-09-16T00:10:04.804261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":195738,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":225,"num_write":58,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"99'227","reported_seq":446,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.891967+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:10:05.891967+0000","last_peered":"2024-09-16T00:10:05.891967+0000","last_clean":"2024-09-16T00:10:05.891967+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:10:05.891967+0000","last_undegraded":"2024-09-16T00:10:05.891967+0000","last_fullsized":"2024-09-16T00:10:05.891967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4928326,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":53,"num_read_kb":28288,"num_write":208,"num_write_kb":66949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"100'360","reported_seq":531,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.876121+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:10:06.876121+0000","last_peered":"2024-09-16T00:10:06.876121+0000","last_clean":"2024-09-16T00:10:06.876121+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:10:06.876121+0000","last_undegraded":"2024-09-16T00:10:06.876121+0000","last_fullsized":"2024-09-16T00:10:06.876121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":350,"num_write_kb":8381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"100'229","reported_seq":433,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.870503+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:10:06.870503+0000","last_peered":"2024-09-16T00:10:06.870503+0000","last_clean":"2024-09-16T00:10:06.870503+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:10:06.870503+0000","last_undegraded":"2024-09-16T00:10:06.870503+0000","last_fullsized":"2024-09-16T00:10:06.870503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":834000,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":36,"num_read_kb":22188,"num_write":203,"num_write_kb":71552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"99'18","reported_seq":186,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.542423+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:10:05.542423+0000","last_peered":"2024-09-16T00:10:05.542423+0000","last_clean":"2024-09-16T00:10:05.542423+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:10:05.542423+0000","last_undegraded":"2024-09-16T00:10:05.542423+0000","last_fullsized":"2024-09-16T00:10:05.542423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"100'259","reported_seq":467,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.892269+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:10:06.892269+0000","last_peered":"2024-09-16T00:10:06.892269+0000","last_clean":"2024-09-16T00:10:06.892269+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:10:06.892269+0000","last_undegraded":"2024-09-16T00:10:06.892269+0000","last_fullsized":"2024-09-16T00:10:06.892269+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":501227,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":39,"num_read_kb":24752,"num_write":223,"num_write_kb":75609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"97'15","reported_seq":184,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.328294+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:10:06.328294+0000","last_peered":"2024-09-16T00:10:06.328294+0000","last_clean":"2024-09-16T00:10:06.328294+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:10:06.328294+0000","last_undegraded":"2024-09-16T00:10:06.328294+0000","last_fullsized":"2024-09-16T00:10:06.328294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"99'239","reported_seq":463,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.900433+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:10:05.900433+0000","last_peered":"2024-09-16T00:10:05.900433+0000","last_clean":"2024-09-16T00:10:05.900433+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:10:05.900433+0000","last_undegraded":"2024-09-16T00:10:05.900433+0000","last_fullsized":"2024-09-16T00:10:05.900433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14187708,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":58,"num_read_kb":31221,"num_write":213,"num_write_kb":76143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"99'99","reported_seq":296,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.864821+0000","last_change":"2024-09-16T00:10:00.330950+0000","last_active":"2024-09-16T00:10:05.864821+0000","last_peered":"2024-09-16T00:10:05.864821+0000","last_clean":"2024-09-16T00:10:05.864821+0000","last_became_active":"2024-09-16T00:10:00.283631+0000","last_became_peered":"2024-09-16T00:10:00.283631+0000","last_unstale":"2024-09-16T00:10:05.864821+0000","last_undegraded":"2024-09-16T00:10:05.864821+0000","last_fullsized":"2024-09-16T00:10:05.864821+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:04.314165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":217,"num_write_kb":1451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":1909,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"99'193","reported_seq":399,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.878296+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:10:05.878296+0000","last_peered":"2024-09-16T00:10:05.878296+0000","last_clean":"2024-09-16T00:10:05.878296+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:10:05.878296+0000","last_undegraded":"2024-09-16T00:10:05.878296+0000","last_fullsized":"2024-09-16T00:10:05.878296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":879403,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":39,"num_read_kb":19274,"num_write":174,"num_write_kb":22071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"101'16","reported_seq":162,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.866573+0000","last_change":"2024-09-16T00:10:00.319924+0000","last_active":"2024-09-16T00:10:07.866573+0000","last_peered":"2024-09-16T00:10:07.866573+0000","last_clean":"2024-09-16T00:10:07.866573+0000","last_became_active":"2024-09-16T00:10:00.287979+0000","last_became_peered":"2024-09-16T00:10:00.287979+0000","last_unstale":"2024-09-16T00:10:07.866573+0000","last_undegraded":"2024-09-16T00:10:07.866573+0000","last_fullsized":"2024-09-16T00:10:07.866573+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:15.202658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":18,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"99'268","reported_seq":477,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.886387+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:10:05.886387+0000","last_peered":"2024-09-16T00:10:05.886387+0000","last_clean":"2024-09-16T00:10:05.886387+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:10:05.886387+0000","last_undegraded":"2024-09-16T00:10:05.886387+0000","last_fullsized":"2024-09-16T00:10:05.886387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1683893,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":43,"num_read_kb":17894,"num_write":249,"num_write_kb":88843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"100'515","reported_seq":684,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.327872+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:10:06.327872+0000","last_peered":"2024-09-16T00:10:06.327872+0000","last_clean":"2024-09-16T00:10:06.327872+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:10:06.327872+0000","last_undegraded":"2024-09-16T00:10:06.327872+0000","last_fullsized":"2024-09-16T00:10:06.327872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":500,"num_write_kb":4358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"100'263","reported_seq":449,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.899857+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:10:06.899857+0000","last_peered":"2024-09-16T00:10:06.899857+0000","last_clean":"2024-09-16T00:10:06.899857+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:10:06.899857+0000","last_undegraded":"2024-09-16T00:10:06.899857+0000","last_fullsized":"2024-09-16T00:10:06.899857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":714314,"num_objects":189,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":189,"num_whiteouts":0,"num_read":18,"num_read_kb":9206,"num_write":245,"num_write_kb":79714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"99'17","reported_seq":185,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.542401+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:10:05.542401+0000","last_peered":"2024-09-16T00:10:05.542401+0000","last_clean":"2024-09-16T00:10:05.542401+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:10:05.542401+0000","last_undegraded":"2024-09-16T00:10:05.542401+0000","last_fullsized":"2024-09-16T00:10:05.542401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"99'211","reported_seq":393,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.890385+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:10:05.890385+0000","last_peered":"2024-09-16T00:10:05.890385+0000","last_clean":"2024-09-16T00:10:05.890385+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:10:05.890385+0000","last_undegraded":"2024-09-16T00:10:05.890385+0000","last_fullsized":"2024-09-16T00:10:05.890385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2323343,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":15,"num_read_kb":8384,"num_write":193,"num_write_kb":50710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"99'20","reported_seq":188,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.858710+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:10:05.858710+0000","last_peered":"2024-09-16T00:10:05.858710+0000","last_clean":"2024-09-16T00:10:05.858710+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:10:05.858710+0000","last_undegraded":"2024-09-16T00:10:05.858710+0000","last_fullsized":"2024-09-16T00:10:05.858710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"99'248","reported_seq":484,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.881767+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:10:05.881767+0000","last_peered":"2024-09-16T00:10:05.881767+0000","last_clean":"2024-09-16T00:10:05.881767+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:10:05.881767+0000","last_undegraded":"2024-09-16T00:10:05.881767+0000","last_fullsized":"2024-09-16T00:10:05.881767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4864327,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":69,"num_read_kb":37740,"num_write":228,"num_write_kb":67449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"100'22","reported_seq":191,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.328059+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:10:06.328059+0000","last_peered":"2024-09-16T00:10:06.328059+0000","last_clean":"2024-09-16T00:10:06.328059+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:10:06.328059+0000","last_undegraded":"2024-09-16T00:10:06.328059+0000","last_fullsized":"2024-09-16T00:10:06.328059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"101'284","reported_seq":352,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.897576+0000","last_change":"2024-09-16T00:10:00.699942+0000","last_active":"2024-09-16T00:10:07.897576+0000","last_peered":"2024-09-16T00:10:07.897576+0000","last_clean":"2024-09-16T00:10:07.897576+0000","last_became_active":"2024-09-16T00:10:00.281972+0000","last_became_peered":"2024-09-16T00:10:00.281972+0000","last_unstale":"2024-09-16T00:10:07.897576+0000","last_undegraded":"2024-09-16T00:10:07.897576+0000","last_fullsized":"2024-09-16T00:10:07.897576+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:21.943709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":801168,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":82,"num_read_kb":43345,"num_write":257,"num_write_kb":121191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":29546741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"100'249","reported_seq":487,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.905684+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:10:06.905684+0000","last_peered":"2024-09-16T00:10:06.905684+0000","last_clean":"2024-09-16T00:10:06.905684+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:10:06.905684+0000","last_undegraded":"2024-09-16T00:10:06.905684+0000","last_fullsized":"2024-09-16T00:10:06.905684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":529274,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":69,"num_read_kb":32882,"num_write":221,"num_write_kb":108251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"100'124","reported_seq":293,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.031563+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:10:07.031563+0000","last_peered":"2024-09-16T00:10:07.031563+0000","last_clean":"2024-09-16T00:10:07.031563+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:10:07.031563+0000","last_undegraded":"2024-09-16T00:10:07.031563+0000","last_fullsized":"2024-09-16T00:10:07.031563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2108106,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":2126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"100'236","reported_seq":474,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.900042+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:10:06.900042+0000","last_peered":"2024-09-16T00:10:06.900042+0000","last_clean":"2024-09-16T00:10:06.900042+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:10:06.900042+0000","last_undegraded":"2024-09-16T00:10:06.900042+0000","last_fullsized":"2024-09-16T00:10:06.900042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856930,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":69,"num_read_kb":36278,"num_write":219,"num_write_kb":113361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"100'98","reported_seq":297,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.861266+0000","last_change":"2024-09-16T00:10:00.301968+0000","last_active":"2024-09-16T00:10:06.861266+0000","last_peered":"2024-09-16T00:10:06.861266+0000","last_clean":"2024-09-16T00:10:06.861266+0000","last_became_active":"2024-09-16T00:10:00.280330+0000","last_became_peered":"2024-09-16T00:10:00.280330+0000","last_unstale":"2024-09-16T00:10:06.861266+0000","last_undegraded":"2024-09-16T00:10:06.861266+0000","last_fullsized":"2024-09-16T00:10:06.861266+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:10.395459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83,"num_write_kb":8277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":30,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"98'193","reported_seq":350,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.839867+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:10:04.839867+0000","last_peered":"2024-09-16T00:10:04.839867+0000","last_clean":"2024-09-16T00:10:04.839867+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:10:04.839867+0000","last_undegraded":"2024-09-16T00:10:04.839867+0000","last_fullsized":"2024-09-16T00:10:04.839867+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5212462,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":19,"num_read_kb":12352,"num_write":171,"num_write_kb":38097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"99'49","reported_seq":218,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.844193+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:10:05.844193+0000","last_peered":"2024-09-16T00:10:05.844193+0000","last_clean":"2024-09-16T00:10:05.844193+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:10:05.844193+0000","last_undegraded":"2024-09-16T00:10:05.844193+0000","last_fullsized":"2024-09-16T00:10:05.844193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":4141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"99'245","reported_seq":532,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.859182+0000","last_change":"2024-09-16T00:10:00.556848+0000","last_active":"2024-09-16T00:10:05.859182+0000","last_peered":"2024-09-16T00:10:05.859182+0000","last_clean":"2024-09-16T00:10:05.859182+0000","last_became_active":"2024-09-16T00:10:00.283734+0000","last_became_peered":"2024-09-16T00:10:00.283734+0000","last_unstale":"2024-09-16T00:10:05.859182+0000","last_undegraded":"2024-09-16T00:10:05.859182+0000","last_fullsized":"2024-09-16T00:10:05.859182+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:55:01.843183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10366791,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":76,"num_read_kb":40163,"num_write":225,"num_write_kb":81063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":21131701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"99'67","reported_seq":235,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.891538+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:10:05.891538+0000","last_peered":"2024-09-16T00:10:05.891538+0000","last_clean":"2024-09-16T00:10:05.891538+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:10:05.891538+0000","last_undegraded":"2024-09-16T00:10:05.891538+0000","last_fullsized":"2024-09-16T00:10:05.891538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":4138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":266652695,"num_objects":10686,"num_object_clones":0,"num_object_copies":21372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10686,"num_whiteouts":0,"num_read":2561,"num_read_kb":1362012,"num_write":13731,"num_write_kb":4088607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":433,"num_bytes_recovered":292530836,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":170156032,"data_stored":484283483,"data_compressed":36073906,"data_compressed_allocated":45531136,"data_compressed_original":411141071,"omap_allocated":0,"internal_metadata":0},"log_size":15100,"ondisk_log_size":15100,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":81997090,"num_objects":114,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":0,"num_read":56,"num_read_kb":225,"num_write":3920,"num_write_kb":84195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":13200,"num_keys_recovered":1983,"num_objects_omap":78,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57200640,"data_stored":164021652,"data_compressed":23525572,"data_compressed_allocated":30339072,"data_compressed_original":137220096,"omap_allocated":225539,"internal_metadata":0},"log_size":4182,"ondisk_log_size":4182,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509477,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69600,"kb_used_data":38416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302240,"statfs":{"total":96636764160,"available":96565493760,"internally_reserved":0,"allocated":39337984,"data_stored":128722246,"data_compressed":12163488,"data_compressed_allocated":15265792,"data_compressed_original":112582856,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":94,"seq":403726925827,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51644,"kb_used_data":22376,"kb_used_omap":221,"kb_used_meta":29026,"kb_avail":94320196,"statfs":{"total":96636764160,"available":96583880704,"internally_reserved":0,"allocated":22913024,"data_stored":47693263,"data_compressed":5421008,"data_compressed_allocated":6537216,"data_compressed_original":37240832,"omap_allocated":227133,"internal_metadata":29722819},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509476,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67304,"kb_used_data":31960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94304536,"statfs":{"total":96636764160,"available":96567844864,"internally_reserved":0,"allocated":32727040,"data_stored":113512292,"data_compressed":8764433,"data_compressed_allocated":11464704,"data_compressed_original":100405014,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509477,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92532,"kb_used_data":39132,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94279308,"statfs":{"total":96636764160,"available":96542011392,"internally_reserved":0,"allocated":40071168,"data_stored":118809031,"data_compressed":10131031,"data_compressed_allocated":13107200,"data_compressed_original":99938182,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509475,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62144,"kb_used_data":30956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94309696,"statfs":{"total":96636764160,"available":96573128704,"internally_reserved":0,"allocated":31698944,"data_stored":142894816,"data_compressed":10441672,"data_compressed_allocated":13262848,"data_compressed_original":132760869,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542180,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55600,"kb_used_data":28572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94316240,"statfs":{"total":96636764160,"available":96579829760,"internally_reserved":0,"allocated":29257728,"data_stored":95007838,"data_compressed":7236032,"data_compressed_allocated":9465856,"data_compressed_original":80777479,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542180,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74604,"kb_used_data":43412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94297236,"statfs":{"total":96636764160,"available":96560369664,"internally_reserved":0,"allocated":44453888,"data_stored":201010966,"data_compressed":12781574,"data_compressed_allocated":16740352,"data_compressed_original":180988079,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61408,"kb_used_data":21904,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94310432,"statfs":{"total":96636764160,"available":96573882368,"internally_reserved":0,"allocated":22429696,"data_stored":151808590,"data_compressed":8684313,"data_compressed_allocated":11386880,"data_compressed_original":145954232,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7749632,"data_stored":20972158,"data_compressed":2936172,"data_compressed_allocated":3887104,"data_compressed_original":17121280,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8351744,"data_stored":20971680,"data_compressed":2925068,"data_compressed_allocated":3854336,"data_compressed_original":16482304,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1835008,"data_stored":4390042,"data_compressed":600007,"data_compressed_allocated":819200,"data_compressed_original":3375104,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4550656,"data_stored":17001098,"data_compressed":2587033,"data_compressed_allocated":3162112,"data_compressed_original":15622144,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12107776,"data_stored":29360254,"data_compressed":4035409,"data_compressed_allocated":5427200,"data_compressed_original":22691840,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8396800,"data_stored":18885412,"data_compressed":2588884,"data_compressed_allocated":3518464,"data_compressed_original":14012416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3444736,"data_stored":16777888,"data_compressed":2643028,"data_compressed_allocated":3112960,"data_compressed_original":16453632,"omap_allocated":225539,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10764288,"data_stored":35663120,"data_compressed":5209971,"data_compressed_allocated":6557696,"data_compressed_original":31461376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":31567872,"data_stored":120026655,"data_compressed":7028518,"data_compressed_allocated":9076736,"data_compressed_original":104519619,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19738624,"data_stored":73483805,"data_compressed":4310964,"data_compressed_allocated":5611520,"data_compressed_original":64295175,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17551360,"data_stored":115204320,"data_compressed":6654056,"data_compressed_allocated":8654848,"data_compressed_original":110864767,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22536192,"data_stored":72329904,"data_compressed":5346322,"data_compressed_allocated":6766592,"data_compressed_original":64194856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":22589440,"data_stored":23109601,"data_compressed":2990360,"data_compressed_allocated":3551232,"data_compressed_original":11505706,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":19238912,"data_stored":32573040,"data_compressed":3315963,"data_compressed_allocated":4128768,"data_compressed_original":24914970,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":13205504,"data_stored":13520627,"data_compressed":2181098,"data_compressed_allocated":2633728,"data_compressed_original":8196096,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":23728128,"data_stored":34035531,"data_compressed":4246625,"data_compressed_allocated":5107712,"data_compressed_original":22649882,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:10:09.750 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:10:09.753 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:09.753 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:09.754 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:10:09.754 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:09.752+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:09.853 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:09.852+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:09.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.855+0000 7f9851ed7640 1 Processor -- start 2024-09-16T00:10:09.856 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.855+0000 7f9851ed7640 1 -- start start 2024-09-16T00:10:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.856+0000 7f9851ed7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f984c0c8cc0 0x7f984c06fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.856+0000 7f9851ed7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f984c070120 0x7f984c0705d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.856+0000 7f9851ed7640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f984c0cb640 con 0x7f984c1528b0 2024-09-16T00:10:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.856+0000 7f9851ed7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f984c152c90 con 0x7f984c070120 2024-09-16T00:10:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.856+0000 7f9851ed7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f984c0c9990 con 0x7f984c0c8cc0 2024-09-16T00:10:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.856+0000 7f984bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f984c070120 0x7f984c0705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.856+0000 7f984bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f984c070120 0x7f984c0705d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49178/0 (socket says 172.21.15.139:49178) 2024-09-16T00:10:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.856+0000 7f984bfff640 1 -- 172.21.15.139:0/287507328 learned_addr learned my addr 172.21.15.139:0/287507328 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.857+0000 7f984affd640 1 --2- 172.21.15.139:0/287507328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f984c0c8cc0 0x7f984c06fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.857+0000 7f984bfff640 1 -- 172.21.15.139:0/287507328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f984c0c8cc0 msgr2=0x7f984c06fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:09.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.857+0000 7f984bfff640 1 --2- 172.21.15.139:0/287507328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f984c0c8cc0 0x7f984c06fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.857+0000 7f984bfff640 1 -- 172.21.15.139:0/287507328 >> v1:172.21.15.164:6789/0 conn(0x7f984c1528b0 legacy=0x7f984c0caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:09.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.857+0000 7f984bfff640 1 -- 172.21.15.139:0/287507328 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984c0c32e0 con 0x7f984c070120 2024-09-16T00:10:09.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.857+0000 7f984bfff640 1 --2- 172.21.15.139:0/287507328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f984c070120 0x7f984c0705d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9844009e50 tx=0x7f984402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f66f844c4f6b11 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:09.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.858+0000 7f984a7fc640 1 -- 172.21.15.139:0/287507328 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9844036070 con 0x7f984c070120 2024-09-16T00:10:09.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.858+0000 7f984a7fc640 1 -- 172.21.15.139:0/287507328 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9844031600 con 0x7f984c070120 2024-09-16T00:10:09.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.858+0000 7f984a7fc640 1 -- 172.21.15.139:0/287507328 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98440319d0 con 0x7f984c070120 2024-09-16T00:10:09.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.858+0000 7f9851ed7640 1 -- 172.21.15.139:0/287507328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f984c070120 msgr2=0x7f984c0705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:09.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.858+0000 7f9851ed7640 1 --2- 172.21.15.139:0/287507328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f984c070120 0x7f984c0705d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9844009e50 tx=0x7f984402a710 comp rx=0 tx=0).stop 2024-09-16T00:10:09.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.858+0000 7f9851ed7640 1 -- 172.21.15.139:0/287507328 shutdown_connections 2024-09-16T00:10:09.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.858+0000 7f9851ed7640 1 --2- 172.21.15.139:0/287507328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f984c070120 0x7f984c0705d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.858+0000 7f9851ed7640 1 --2- 172.21.15.139:0/287507328 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f984c0c8cc0 0x7f984c06fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.858+0000 7f9851ed7640 1 -- 172.21.15.139:0/287507328 >> 172.21.15.139:0/287507328 conn(0x7f984c067d70 msgr2=0x7f984c068180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:09.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.858+0000 7f9851ed7640 1 -- 172.21.15.139:0/287507328 shutdown_connections 2024-09-16T00:10:09.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.858+0000 7f9851ed7640 1 -- 172.21.15.139:0/287507328 wait complete. 2024-09-16T00:10:09.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.859+0000 7f9851ed7640 1 Processor -- start 2024-09-16T00:10:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.859+0000 7f9851ed7640 1 -- start start 2024-09-16T00:10:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.860+0000 7f9851ed7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f984c070120 0x7f984c146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.860+0000 7f9851ed7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f984c0c8cc0 0x7f984c1eed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.860+0000 7f9851ed7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f984c1528b0 0x7f984c1ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.860+0000 7f9851ed7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f984c15c860 con 0x7f984c070120 2024-09-16T00:10:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.860+0000 7f9851ed7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f984c0caaa0 con 0x7f984c1528b0 2024-09-16T00:10:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.860+0000 7f9851ed7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f984c0c96b0 con 0x7f984c0c8cc0 2024-09-16T00:10:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.860+0000 7f984b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f984c070120 0x7f984c146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.860+0000 7f984b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f984c070120 0x7f984c146eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49184/0 (socket says 172.21.15.139:49184) 2024-09-16T00:10:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.860+0000 7f984b7fe640 1 -- 172.21.15.139:0/1397625052 learned_addr learned my addr 172.21.15.139:0/1397625052 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.860+0000 7f984bfff640 1 --2- 172.21.15.139:0/1397625052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f984c1528b0 0x7f984c1ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:09.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.860+0000 7f984affd640 1 --2- 172.21.15.139:0/1397625052 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f984c0c8cc0 0x7f984c1eed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.861+0000 7f984b7fe640 1 -- 172.21.15.139:0/1397625052 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f984c0c8cc0 msgr2=0x7f984c1eed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.861+0000 7f984b7fe640 1 --2- 172.21.15.139:0/1397625052 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f984c0c8cc0 0x7f984c1eed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.861+0000 7f984b7fe640 1 -- 172.21.15.139:0/1397625052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f984c1528b0 msgr2=0x7f984c1ef2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.861+0000 7f984b7fe640 1 --2- 172.21.15.139:0/1397625052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f984c1528b0 0x7f984c1ef2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.861+0000 7f984b7fe640 1 -- 172.21.15.139:0/1397625052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984c0c5e90 con 0x7f984c070120 2024-09-16T00:10:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.861+0000 7f984b7fe640 1 --2- 172.21.15.139:0/1397625052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f984c070120 0x7f984c146eb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f983c00c430 tx=0x7f983c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.861+0000 7f9833fff640 1 -- 172.21.15.139:0/1397625052 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c0052d0 con 0x7f984c070120 2024-09-16T00:10:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.861+0000 7f9833fff640 1 -- 172.21.15.139:0/1397625052 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f983c005490 con 0x7f984c070120 2024-09-16T00:10:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.861+0000 7f9851ed7640 1 -- 172.21.15.139:0/1397625052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984c0c8810 con 0x7f984c070120 2024-09-16T00:10:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.861+0000 7f9833fff640 1 -- 172.21.15.139:0/1397625052 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c007260 con 0x7f984c070120 2024-09-16T00:10:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.861+0000 7f9851ed7640 1 -- 172.21.15.139:0/1397625052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984c0c5bb0 con 0x7f984c070120 2024-09-16T00:10:09.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.862+0000 7f9851ed7640 1 -- 172.21.15.139:0/1397625052 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984c0ca730 con 0x7f984c070120 2024-09-16T00:10:09.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.862+0000 7f9833fff640 1 -- 172.21.15.139:0/1397625052 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f983c007400 con 0x7f984c070120 2024-09-16T00:10:09.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.863+0000 7f9833fff640 1 --2- 172.21.15.139:0/1397625052 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f983806c2b0 0x7f983806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:09.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.863+0000 7f9833fff640 1 -- 172.21.15.139:0/1397625052 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5973+0+0 (secure 0 0 0) 0x7f983c08aa20 con 0x7f984c070120 2024-09-16T00:10:09.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.863+0000 7f984affd640 1 --2- 172.21.15.139:0/1397625052 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f983806c2b0 0x7f983806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:09.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.863+0000 7f984affd640 1 --2- 172.21.15.139:0/1397625052 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f983806c2b0 0x7f983806e770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f984c152230 tx=0x7f9840009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:09.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:09.867+0000 7f9833fff640 1 -- 172.21.15.139:0/1397625052 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f983c0589a0 con 0x7f984c070120 2024-09-16T00:10:09.940 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:09.941 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:09.953 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:09.953+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:09.955 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:10:10.054 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:10.054+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:10.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.072+0000 7f9851ed7640 1 -- 172.21.15.139:0/1397625052 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f984c0b43a0 con 0x7f983806c2b0 2024-09-16T00:10:10.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.079+0000 7f9833fff640 1 -- 172.21.15.139:0/1397625052 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+255469 (secure 0 0 0) 0x7f984c0b43a0 con 0x7f983806c2b0 2024-09-16T00:10:10.083 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:10:10.084 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:10:10.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.082+0000 7f9851ed7640 1 -- 172.21.15.139:0/1397625052 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f983806c2b0 msgr2=0x7f983806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:10.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.082+0000 7f9851ed7640 1 --2- 172.21.15.139:0/1397625052 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f983806c2b0 0x7f983806e770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f984c152230 tx=0x7f9840009190 comp rx=0 tx=0).stop 2024-09-16T00:10:10.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.082+0000 7f9851ed7640 1 -- 172.21.15.139:0/1397625052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f984c070120 msgr2=0x7f984c146eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:10.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.082+0000 7f9851ed7640 1 --2- 172.21.15.139:0/1397625052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f984c070120 0x7f984c146eb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f983c00c430 tx=0x7f983c00c8f0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.082+0000 7f9851ed7640 1 -- 172.21.15.139:0/1397625052 shutdown_connections 2024-09-16T00:10:10.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.082+0000 7f9851ed7640 1 --2- 172.21.15.139:0/1397625052 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f983806c2b0 0x7f983806e770 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.082+0000 7f9851ed7640 1 --2- 172.21.15.139:0/1397625052 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f984c1528b0 0x7f984c1ef2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.082+0000 7f9851ed7640 1 --2- 172.21.15.139:0/1397625052 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f984c0c8cc0 0x7f984c1eed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.082+0000 7f9851ed7640 1 --2- 172.21.15.139:0/1397625052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f984c070120 0x7f984c146eb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.082+0000 7f9851ed7640 1 -- 172.21.15.139:0/1397625052 >> 172.21.15.139:0/1397625052 conn(0x7f984c067d70 msgr2=0x7f984c14d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:10.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.083+0000 7f9851ed7640 1 -- 172.21.15.139:0/1397625052 shutdown_connections 2024-09-16T00:10:10.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.083+0000 7f9851ed7640 1 -- 172.21.15.139:0/1397625052 wait complete. 2024-09-16T00:10:10.094 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-16T00:10:10.053369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":349240153,"num_objects":10802,"num_object_clones":0,"num_object_copies":21604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10802,"num_whiteouts":0,"num_read":2733,"num_read_kb":1362337,"num_write":17810,"num_write_kb":4175218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":452,"num_bytes_recovered":292544036,"num_keys_recovered":1983,"num_objects_omap":78,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19399,"ondisk_log_size":19399,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":534836,"kb_used_data":256728,"kb_used_omap":232,"kb_used_meta":277719,"kb_avail":754439884,"statfs":{"total":773094113280,"available":772546441216,"internally_reserved":0,"allocated":262889472,"data_stored":999459042,"data_compressed":75623551,"data_compressed_allocated":97230848,"data_compressed_original":890647543,"omap_allocated":238291,"internal_metadata":284384557},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-898304194,"num_objects":-345,"num_object_clones":0,"num_object_copies":-690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":473,"num_write_kb":2138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000575"},"pg_stats":[{"pgid":"3.3f","version":"100'267","reported_seq":554,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.884488+0000","last_change":"2024-09-16T00:10:00.622682+0000","last_active":"2024-09-16T00:10:06.884488+0000","last_peered":"2024-09-16T00:10:06.884488+0000","last_clean":"2024-09-16T00:10:06.884488+0000","last_became_active":"2024-09-16T00:10:00.286893+0000","last_became_peered":"2024-09-16T00:10:00.286893+0000","last_unstale":"2024-09-16T00:10:06.884488+0000","last_undegraded":"2024-09-16T00:10:06.884488+0000","last_fullsized":"2024-09-16T00:10:06.884488+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:21.452686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":562528,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":64,"num_read_kb":42086,"num_write":242,"num_write_kb":112716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":31107821,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"99'238","reported_seq":429,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.897532+0000","last_change":"2024-09-16T00:07:33.625816+0000","last_active":"2024-09-16T00:10:05.897532+0000","last_peered":"2024-09-16T00:10:05.897532+0000","last_clean":"2024-09-16T00:10:05.897532+0000","last_became_active":"2024-09-16T00:07:33.625561+0000","last_became_peered":"2024-09-16T00:07:33.625561+0000","last_unstale":"2024-09-16T00:10:05.897532+0000","last_undegraded":"2024-09-16T00:10:05.897532+0000","last_fullsized":"2024-09-16T00:10:05.897532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4995574,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":25,"num_read_kb":23064,"num_write":220,"num_write_kb":47220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"99'208","reported_seq":404,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.861931+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:10:05.861931+0000","last_peered":"2024-09-16T00:10:05.861931+0000","last_clean":"2024-09-16T00:10:05.861931+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:10:05.861931+0000","last_undegraded":"2024-09-16T00:10:05.861931+0000","last_fullsized":"2024-09-16T00:10:05.861931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9209061,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":30,"num_read_kb":15782,"num_write":193,"num_write_kb":54932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'225","reported_seq":424,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.883945+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:10:06.883945+0000","last_peered":"2024-09-16T00:10:06.883945+0000","last_clean":"2024-09-16T00:10:06.883945+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:10:06.883945+0000","last_undegraded":"2024-09-16T00:10:06.883945+0000","last_fullsized":"2024-09-16T00:10:06.883945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":651768,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":30,"num_read_kb":14080,"num_write":204,"num_write_kb":63191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"99'267","reported_seq":476,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.899986+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:10:05.899986+0000","last_peered":"2024-09-16T00:10:05.899986+0000","last_clean":"2024-09-16T00:10:05.899986+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:10:05.899986+0000","last_undegraded":"2024-09-16T00:10:05.899986+0000","last_fullsized":"2024-09-16T00:10:05.899986+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8067643,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":69,"num_read_kb":36096,"num_write":252,"num_write_kb":89616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"98'213","reported_seq":733,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.864474+0000","last_change":"2024-09-16T00:10:00.875898+0000","last_active":"2024-09-16T00:10:04.864474+0000","last_peered":"2024-09-16T00:10:04.864474+0000","last_clean":"2024-09-16T00:10:04.864474+0000","last_became_active":"2024-09-16T00:10:00.278705+0000","last_became_peered":"2024-09-16T00:10:00.278705+0000","last_unstale":"2024-09-16T00:10:04.864474+0000","last_undegraded":"2024-09-16T00:10:04.864474+0000","last_fullsized":"2024-09-16T00:10:04.864474+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:45:25.113058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7455179,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":16,"num_read_kb":6174,"num_write":196,"num_write_kb":50827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":23596926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"100'246","reported_seq":447,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.869376+0000","last_change":"2024-09-16T00:07:33.633314+0000","last_active":"2024-09-16T00:10:06.869376+0000","last_peered":"2024-09-16T00:10:06.869376+0000","last_clean":"2024-09-16T00:10:06.869376+0000","last_became_active":"2024-09-16T00:07:33.633053+0000","last_became_peered":"2024-09-16T00:07:33.633053+0000","last_unstale":"2024-09-16T00:10:06.869376+0000","last_undegraded":"2024-09-16T00:10:06.869376+0000","last_fullsized":"2024-09-16T00:10:06.869376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1669868,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":34,"num_read_kb":16577,"num_write":228,"num_write_kb":64470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"98'220","reported_seq":391,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.857892+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:10:04.857892+0000","last_peered":"2024-09-16T00:10:04.857892+0000","last_clean":"2024-09-16T00:10:04.857892+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:10:04.857892+0000","last_undegraded":"2024-09-16T00:10:04.857892+0000","last_fullsized":"2024-09-16T00:10:04.857892+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15472397,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":33,"num_read_kb":16776,"num_write":197,"num_write_kb":80164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"100'221","reported_seq":395,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.900265+0000","last_change":"2024-09-16T00:07:33.638895+0000","last_active":"2024-09-16T00:10:06.900265+0000","last_peered":"2024-09-16T00:10:06.900265+0000","last_clean":"2024-09-16T00:10:06.900265+0000","last_became_active":"2024-09-16T00:07:33.638628+0000","last_became_peered":"2024-09-16T00:07:33.638628+0000","last_unstale":"2024-09-16T00:10:06.900265+0000","last_undegraded":"2024-09-16T00:10:06.900265+0000","last_fullsized":"2024-09-16T00:10:06.900265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1173835,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":7,"num_read_kb":1432,"num_write":195,"num_write_kb":30311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"101'214","reported_seq":295,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.878457+0000","last_change":"2024-09-16T00:10:00.734336+0000","last_active":"2024-09-16T00:10:07.878457+0000","last_peered":"2024-09-16T00:10:07.878457+0000","last_clean":"2024-09-16T00:10:07.878457+0000","last_became_active":"2024-09-16T00:10:00.284452+0000","last_became_peered":"2024-09-16T00:10:00.284452+0000","last_unstale":"2024-09-16T00:10:07.878457+0000","last_undegraded":"2024-09-16T00:10:07.878457+0000","last_fullsized":"2024-09-16T00:10:07.878457+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:56:50.530439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":656293,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":6,"num_read_kb":4096,"num_write":182,"num_write_kb":29952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":8736255,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"101'198","reported_seq":290,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.870666+0000","last_change":"2024-09-16T00:10:00.587110+0000","last_active":"2024-09-16T00:10:07.870666+0000","last_peered":"2024-09-16T00:10:07.870666+0000","last_clean":"2024-09-16T00:10:07.870666+0000","last_became_active":"2024-09-16T00:10:00.280273+0000","last_became_peered":"2024-09-16T00:10:00.280273+0000","last_unstale":"2024-09-16T00:10:07.870666+0000","last_undegraded":"2024-09-16T00:10:07.870666+0000","last_fullsized":"2024-09-16T00:10:07.870666+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:01:57.902939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":581234,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":6,"num_read_kb":975,"num_write":182,"num_write_kb":25518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":10753732,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"100'219","reported_seq":466,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.858146+0000","last_change":"2024-09-16T00:10:00.797372+0000","last_active":"2024-09-16T00:10:06.858146+0000","last_peered":"2024-09-16T00:10:06.858146+0000","last_clean":"2024-09-16T00:10:06.858146+0000","last_became_active":"2024-09-16T00:10:00.278765+0000","last_became_peered":"2024-09-16T00:10:00.278765+0000","last_unstale":"2024-09-16T00:10:06.858146+0000","last_undegraded":"2024-09-16T00:10:06.858146+0000","last_fullsized":"2024-09-16T00:10:06.858146+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:03.318919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":580504,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":30,"num_read_kb":11563,"num_write":201,"num_write_kb":58933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":29565877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"99'256","reported_seq":527,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.900879+0000","last_change":"2024-09-16T00:10:00.715647+0000","last_active":"2024-09-16T00:10:05.900879+0000","last_peered":"2024-09-16T00:10:05.900879+0000","last_clean":"2024-09-16T00:10:05.900879+0000","last_became_active":"2024-09-16T00:10:00.286050+0000","last_became_peered":"2024-09-16T00:10:00.286050+0000","last_unstale":"2024-09-16T00:10:05.900879+0000","last_undegraded":"2024-09-16T00:10:05.900879+0000","last_fullsized":"2024-09-16T00:10:05.900879+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:07.462899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5614093,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":56,"num_read_kb":35908,"num_write":231,"num_write_kb":104863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":30230864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"99'234","reported_seq":449,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.897499+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:10:05.897499+0000","last_peered":"2024-09-16T00:10:05.897499+0000","last_clean":"2024-09-16T00:10:05.897499+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:10:05.897499+0000","last_undegraded":"2024-09-16T00:10:05.897499+0000","last_fullsized":"2024-09-16T00:10:05.897499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20189743,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":49,"num_read_kb":23364,"num_write":212,"num_write_kb":43321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"99'219","reported_seq":418,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.872775+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:10:05.872775+0000","last_peered":"2024-09-16T00:10:05.872775+0000","last_clean":"2024-09-16T00:10:05.872775+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:10:05.872775+0000","last_undegraded":"2024-09-16T00:10:05.872775+0000","last_fullsized":"2024-09-16T00:10:05.872775+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4753934,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":33,"num_read_kb":14230,"num_write":194,"num_write_kb":33768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"100'257","reported_seq":483,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.871621+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:10:06.871621+0000","last_peered":"2024-09-16T00:10:06.871621+0000","last_clean":"2024-09-16T00:10:06.871621+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:10:06.871621+0000","last_undegraded":"2024-09-16T00:10:06.871621+0000","last_fullsized":"2024-09-16T00:10:06.871621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":900420,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":57,"num_read_kb":40234,"num_write":234,"num_write_kb":67785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"100'227","reported_seq":449,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.899718+0000","last_change":"2024-09-16T00:07:33.632931+0000","last_active":"2024-09-16T00:10:06.899718+0000","last_peered":"2024-09-16T00:10:06.899718+0000","last_clean":"2024-09-16T00:10:06.899718+0000","last_became_active":"2024-09-16T00:07:33.632606+0000","last_became_peered":"2024-09-16T00:07:33.632606+0000","last_unstale":"2024-09-16T00:10:06.899718+0000","last_undegraded":"2024-09-16T00:10:06.899718+0000","last_fullsized":"2024-09-16T00:10:06.899718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:44.239817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":593433,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":53,"num_read_kb":22524,"num_write":208,"num_write_kb":46842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"98'214","reported_seq":448,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.856695+0000","last_change":"2024-09-16T00:10:00.461437+0000","last_active":"2024-09-16T00:10:04.856695+0000","last_peered":"2024-09-16T00:10:04.856695+0000","last_clean":"2024-09-16T00:10:04.856695+0000","last_became_active":"2024-09-16T00:10:00.282592+0000","last_became_peered":"2024-09-16T00:10:00.282592+0000","last_unstale":"2024-09-16T00:10:04.856695+0000","last_undegraded":"2024-09-16T00:10:04.856695+0000","last_fullsized":"2024-09-16T00:10:04.856695+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:09.733353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15678980,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":11,"num_read_kb":8711,"num_write":198,"num_write_kb":46855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":21285626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"100'252","reported_seq":509,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.896695+0000","last_change":"2024-09-16T00:09:04.210763+0000","last_active":"2024-09-16T00:10:06.896695+0000","last_peered":"2024-09-16T00:10:06.896695+0000","last_clean":"2024-09-16T00:10:06.896695+0000","last_became_active":"2024-09-16T00:09:04.158723+0000","last_became_peered":"2024-09-16T00:09:04.158723+0000","last_unstale":"2024-09-16T00:10:06.896695+0000","last_undegraded":"2024-09-16T00:10:06.896695+0000","last_fullsized":"2024-09-16T00:10:06.896695+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:25:48.973773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9214973,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":25,"num_read_kb":14437,"num_write":232,"num_write_kb":83979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":103497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"99'236","reported_seq":327,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.763404+0000","last_change":"2024-09-16T00:10:00.566870+0000","last_active":"2024-09-16T00:10:07.763404+0000","last_peered":"2024-09-16T00:10:07.763404+0000","last_clean":"2024-09-16T00:10:07.763404+0000","last_became_active":"2024-09-16T00:10:00.281258+0000","last_became_peered":"2024-09-16T00:10:00.281258+0000","last_unstale":"2024-09-16T00:10:07.763404+0000","last_undegraded":"2024-09-16T00:10:07.763404+0000","last_fullsized":"2024-09-16T00:10:07.763404+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:29.197743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":670826,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":13,"num_read_kb":15502,"num_write":219,"num_write_kb":30222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":17199573,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"99'240","reported_seq":461,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.899321+0000","last_change":"2024-09-16T00:07:33.630993+0000","last_active":"2024-09-16T00:10:05.899321+0000","last_peered":"2024-09-16T00:10:05.899321+0000","last_clean":"2024-09-16T00:10:05.899321+0000","last_became_active":"2024-09-16T00:07:33.630707+0000","last_became_peered":"2024-09-16T00:07:33.630707+0000","last_unstale":"2024-09-16T00:10:05.899321+0000","last_undegraded":"2024-09-16T00:10:05.899321+0000","last_fullsized":"2024-09-16T00:10:05.899321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:37:09.034147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16302996,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":55,"num_read_kb":34318,"num_write":224,"num_write_kb":91769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"100'270","reported_seq":501,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.898232+0000","last_change":"2024-09-16T00:07:33.631726+0000","last_active":"2024-09-16T00:10:06.898232+0000","last_peered":"2024-09-16T00:10:06.898232+0000","last_clean":"2024-09-16T00:10:06.898232+0000","last_became_active":"2024-09-16T00:07:33.631395+0000","last_became_peered":"2024-09-16T00:07:33.631395+0000","last_unstale":"2024-09-16T00:10:06.898232+0000","last_undegraded":"2024-09-16T00:10:06.898232+0000","last_fullsized":"2024-09-16T00:10:06.898232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":270,"log_dups_size":0,"ondisk_log_size":270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5069924,"num_objects":173,"num_object_clones":0,"num_object_copies":346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":173,"num_whiteouts":0,"num_read":62,"num_read_kb":35528,"num_write":249,"num_write_kb":92852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"99'226","reported_seq":420,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.897517+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:10:05.897517+0000","last_peered":"2024-09-16T00:10:05.897517+0000","last_clean":"2024-09-16T00:10:05.897517+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:10:05.897517+0000","last_undegraded":"2024-09-16T00:10:05.897517+0000","last_fullsized":"2024-09-16T00:10:05.897517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4967085,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":27,"num_read_kb":13410,"num_write":202,"num_write_kb":58805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"100'237","reported_seq":428,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.903078+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:10:06.903078+0000","last_peered":"2024-09-16T00:10:06.903078+0000","last_clean":"2024-09-16T00:10:06.903078+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:10:06.903078+0000","last_undegraded":"2024-09-16T00:10:06.903078+0000","last_fullsized":"2024-09-16T00:10:06.903078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":781271,"num_objects":184,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":0,"num_read":22,"num_read_kb":12682,"num_write":215,"num_write_kb":54869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"99'237","reported_seq":434,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.867474+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:10:05.867474+0000","last_peered":"2024-09-16T00:10:05.867474+0000","last_clean":"2024-09-16T00:10:05.867474+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:10:05.867474+0000","last_undegraded":"2024-09-16T00:10:05.867474+0000","last_fullsized":"2024-09-16T00:10:05.867474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":801473,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":30,"num_read_kb":14188,"num_write":227,"num_write_kb":59178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"101'257","reported_seq":398,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.885032+0000","last_change":"2024-09-16T00:10:00.794575+0000","last_active":"2024-09-16T00:10:07.885032+0000","last_peered":"2024-09-16T00:10:07.885032+0000","last_clean":"2024-09-16T00:10:07.885032+0000","last_became_active":"2024-09-16T00:10:00.285311+0000","last_became_peered":"2024-09-16T00:10:00.285311+0000","last_unstale":"2024-09-16T00:10:07.885032+0000","last_undegraded":"2024-09-16T00:10:07.885032+0000","last_fullsized":"2024-09-16T00:10:07.885032+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:13:49.591876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1705691,"num_objects":191,"num_object_clones":0,"num_object_copies":382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":191,"num_whiteouts":0,"num_read":27,"num_read_kb":12330,"num_write":230,"num_write_kb":35146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":12903786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"99'238","reported_seq":450,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.882477+0000","last_change":"2024-09-16T00:07:33.624990+0000","last_active":"2024-09-16T00:10:05.882477+0000","last_peered":"2024-09-16T00:10:05.882477+0000","last_clean":"2024-09-16T00:10:05.882477+0000","last_became_active":"2024-09-16T00:07:33.624705+0000","last_became_peered":"2024-09-16T00:07:33.624705+0000","last_unstale":"2024-09-16T00:10:05.882477+0000","last_undegraded":"2024-09-16T00:10:05.882477+0000","last_fullsized":"2024-09-16T00:10:05.882477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4917795,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":46,"num_read_kb":21104,"num_write":221,"num_write_kb":75742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"99'224","reported_seq":432,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.872807+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:10:05.872807+0000","last_peered":"2024-09-16T00:10:05.872807+0000","last_clean":"2024-09-16T00:10:05.872807+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:10:05.872807+0000","last_undegraded":"2024-09-16T00:10:05.872807+0000","last_fullsized":"2024-09-16T00:10:05.872807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2134022,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":41,"num_read_kb":16408,"num_write":208,"num_write_kb":51674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"99'224","reported_seq":449,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.882793+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:10:05.882793+0000","last_peered":"2024-09-16T00:10:05.882793+0000","last_clean":"2024-09-16T00:10:05.882793+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:10:05.882793+0000","last_undegraded":"2024-09-16T00:10:05.882793+0000","last_fullsized":"2024-09-16T00:10:05.882793+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9063115,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":59,"num_read_kb":33174,"num_write":205,"num_write_kb":66849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"98'283","reported_seq":481,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.874916+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:10:04.874916+0000","last_peered":"2024-09-16T00:10:04.874916+0000","last_clean":"2024-09-16T00:10:04.874916+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:10:04.874916+0000","last_undegraded":"2024-09-16T00:10:04.874916+0000","last_fullsized":"2024-09-16T00:10:04.874916+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5422585,"num_objects":194,"num_object_clones":0,"num_object_copies":388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":194,"num_whiteouts":0,"num_read":60,"num_read_kb":27231,"num_write":259,"num_write_kb":104771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"98'271","reported_seq":514,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.298030+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:10:04.298030+0000","last_peered":"2024-09-16T00:10:04.298030+0000","last_clean":"2024-09-16T00:10:04.298030+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:10:04.298030+0000","last_undegraded":"2024-09-16T00:10:04.298030+0000","last_fullsized":"2024-09-16T00:10:04.298030+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4869645,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":73,"num_read_kb":35655,"num_write":256,"num_write_kb":97245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"101'241","reported_seq":334,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.763190+0000","last_change":"2024-09-16T00:10:00.535553+0000","last_active":"2024-09-16T00:10:07.763190+0000","last_peered":"2024-09-16T00:10:07.763190+0000","last_clean":"2024-09-16T00:10:07.763190+0000","last_became_active":"2024-09-16T00:10:00.281378+0000","last_became_peered":"2024-09-16T00:10:00.281378+0000","last_unstale":"2024-09-16T00:10:07.763190+0000","last_undegraded":"2024-09-16T00:10:07.763190+0000","last_fullsized":"2024-09-16T00:10:07.763190+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:49.548709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2880131,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":32,"num_read_kb":20395,"num_write":221,"num_write_kb":32731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":22876917,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"100'234","reported_seq":482,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.905306+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:10:06.905306+0000","last_peered":"2024-09-16T00:10:06.905306+0000","last_clean":"2024-09-16T00:10:06.905306+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:10:06.905306+0000","last_undegraded":"2024-09-16T00:10:06.905306+0000","last_fullsized":"2024-09-16T00:10:06.905306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":538720,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":80,"num_read_kb":48141,"num_write":217,"num_write_kb":63611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"99'71","reported_seq":240,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.869516+0000","last_change":"2024-09-16T00:07:32.627934+0000","last_active":"2024-09-16T00:10:05.869516+0000","last_peered":"2024-09-16T00:10:05.869516+0000","last_clean":"2024-09-16T00:10:05.869516+0000","last_became_active":"2024-09-16T00:07:32.627660+0000","last_became_peered":"2024-09-16T00:07:32.627660+0000","last_unstale":"2024-09-16T00:10:05.869516+0000","last_undegraded":"2024-09-16T00:10:05.869516+0000","last_fullsized":"2024-09-16T00:10:05.869516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:37:43.527307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"99'198","reported_seq":391,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.891264+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:10:05.891264+0000","last_peered":"2024-09-16T00:10:05.891264+0000","last_clean":"2024-09-16T00:10:05.891264+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:10:05.891264+0000","last_undegraded":"2024-09-16T00:10:05.891264+0000","last_fullsized":"2024-09-16T00:10:05.891264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4718052,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":27,"num_read_kb":16420,"num_write":181,"num_write_kb":62710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"101'55","reported_seq":132,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.763450+0000","last_change":"2024-09-16T00:10:00.282958+0000","last_active":"2024-09-16T00:10:07.763450+0000","last_peered":"2024-09-16T00:10:07.763450+0000","last_clean":"2024-09-16T00:10:07.763450+0000","last_became_active":"2024-09-16T00:10:00.282541+0000","last_became_peered":"2024-09-16T00:10:00.282541+0000","last_unstale":"2024-09-16T00:10:07.763450+0000","last_undegraded":"2024-09-16T00:10:07.763450+0000","last_fullsized":"2024-09-16T00:10:07.763450+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:51.198787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":4157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"98'20","reported_seq":210,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.291497+0000","last_change":"2024-09-16T00:10:00.313589+0000","last_active":"2024-09-16T00:10:05.291497+0000","last_peered":"2024-09-16T00:10:05.291497+0000","last_clean":"2024-09-16T00:10:05.291497+0000","last_became_active":"2024-09-16T00:10:00.284602+0000","last_became_peered":"2024-09-16T00:10:00.284602+0000","last_unstale":"2024-09-16T00:10:05.291497+0000","last_undegraded":"2024-09-16T00:10:05.291497+0000","last_fullsized":"2024-09-16T00:10:05.291497+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:30.032698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":14,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"99'250","reported_seq":455,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.890409+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:10:05.890409+0000","last_peered":"2024-09-16T00:10:05.890409+0000","last_clean":"2024-09-16T00:10:05.890409+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:10:05.890409+0000","last_undegraded":"2024-09-16T00:10:05.890409+0000","last_fullsized":"2024-09-16T00:10:05.890409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1102178,"num_objects":181,"num_object_clones":0,"num_object_copies":362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":181,"num_whiteouts":0,"num_read":38,"num_read_kb":18480,"num_write":230,"num_write_kb":71748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"100'781","reported_seq":952,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.308153+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:10:06.308153+0000","last_peered":"2024-09-16T00:10:06.308153+0000","last_clean":"2024-09-16T00:10:06.308153+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:10:06.308153+0000","last_undegraded":"2024-09-16T00:10:06.308153+0000","last_fullsized":"2024-09-16T00:10:06.308153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":766,"num_write_kb":8591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"99'249","reported_seq":505,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.858711+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:10:05.858711+0000","last_peered":"2024-09-16T00:10:05.858711+0000","last_clean":"2024-09-16T00:10:05.858711+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:10:05.858711+0000","last_undegraded":"2024-09-16T00:10:05.858711+0000","last_fullsized":"2024-09-16T00:10:05.858711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5073652,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":90,"num_read_kb":46871,"num_write":229,"num_write_kb":84187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"101'23","reported_seq":166,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.763556+0000","last_change":"2024-09-16T00:10:00.283876+0000","last_active":"2024-09-16T00:10:07.763556+0000","last_peered":"2024-09-16T00:10:07.763556+0000","last_clean":"2024-09-16T00:10:07.763556+0000","last_became_active":"2024-09-16T00:10:00.283555+0000","last_became_peered":"2024-09-16T00:10:00.283555+0000","last_unstale":"2024-09-16T00:10:07.763556+0000","last_undegraded":"2024-09-16T00:10:07.763556+0000","last_fullsized":"2024-09-16T00:10:07.763556+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:48.543737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"100'263","reported_seq":467,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.876189+0000","last_change":"2024-09-16T00:07:33.631873+0000","last_active":"2024-09-16T00:10:06.876189+0000","last_peered":"2024-09-16T00:10:06.876189+0000","last_clean":"2024-09-16T00:10:06.876189+0000","last_became_active":"2024-09-16T00:07:33.631335+0000","last_became_peered":"2024-09-16T00:07:33.631335+0000","last_unstale":"2024-09-16T00:10:06.876189+0000","last_undegraded":"2024-09-16T00:10:06.876189+0000","last_fullsized":"2024-09-16T00:10:06.876189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:05.119302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":952497,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":35,"num_read_kb":16686,"num_write":238,"num_write_kb":80112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"100'11","reported_seq":189,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.308858+0000","last_change":"2024-09-16T00:10:00.285386+0000","last_active":"2024-09-16T00:10:06.308858+0000","last_peered":"2024-09-16T00:10:06.308858+0000","last_clean":"2024-09-16T00:10:06.308858+0000","last_became_active":"2024-09-16T00:10:00.284717+0000","last_became_peered":"2024-09-16T00:10:00.284717+0000","last_unstale":"2024-09-16T00:10:06.308858+0000","last_undegraded":"2024-09-16T00:10:06.308858+0000","last_fullsized":"2024-09-16T00:10:06.308858+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:07.794163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"99'218","reported_seq":427,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.870657+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:10:05.870657+0000","last_peered":"2024-09-16T00:10:05.870657+0000","last_clean":"2024-09-16T00:10:05.870657+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:10:05.870657+0000","last_undegraded":"2024-09-16T00:10:05.870657+0000","last_fullsized":"2024-09-16T00:10:05.870657+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9495587,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":43,"num_read_kb":18030,"num_write":200,"num_write_kb":55428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"99'51","reported_seq":219,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.901383+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:10:05.901383+0000","last_peered":"2024-09-16T00:10:05.901383+0000","last_clean":"2024-09-16T00:10:05.901383+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:10:05.901383+0000","last_undegraded":"2024-09-16T00:10:05.901383+0000","last_fullsized":"2024-09-16T00:10:05.901383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"100'219","reported_seq":434,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.867932+0000","last_change":"2024-09-16T00:07:33.635650+0000","last_active":"2024-09-16T00:10:06.867932+0000","last_peered":"2024-09-16T00:10:06.867932+0000","last_clean":"2024-09-16T00:10:06.867932+0000","last_became_active":"2024-09-16T00:07:33.635345+0000","last_became_peered":"2024-09-16T00:07:33.635345+0000","last_unstale":"2024-09-16T00:10:06.867932+0000","last_undegraded":"2024-09-16T00:10:06.867932+0000","last_fullsized":"2024-09-16T00:10:06.867932+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:46.764419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1018812,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":47,"num_read_kb":22960,"num_write":195,"num_write_kb":63161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"101'42","reported_seq":130,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.763354+0000","last_change":"2024-09-16T00:10:00.300895+0000","last_active":"2024-09-16T00:10:07.763354+0000","last_peered":"2024-09-16T00:10:07.763354+0000","last_clean":"2024-09-16T00:10:07.763354+0000","last_became_active":"2024-09-16T00:10:00.283474+0000","last_became_peered":"2024-09-16T00:10:00.283474+0000","last_unstale":"2024-09-16T00:10:07.763354+0000","last_undegraded":"2024-09-16T00:10:07.763354+0000","last_fullsized":"2024-09-16T00:10:07.763354+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:28:02.563817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":13200,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"100'242","reported_seq":455,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.902727+0000","last_change":"2024-09-16T00:07:33.636226+0000","last_active":"2024-09-16T00:10:06.902727+0000","last_peered":"2024-09-16T00:10:06.902727+0000","last_clean":"2024-09-16T00:10:06.902727+0000","last_became_active":"2024-09-16T00:07:33.635935+0000","last_became_peered":"2024-09-16T00:07:33.635935+0000","last_unstale":"2024-09-16T00:10:06.902727+0000","last_undegraded":"2024-09-16T00:10:06.902727+0000","last_fullsized":"2024-09-16T00:10:06.902727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:29.667718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":744709,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":46,"num_read_kb":23285,"num_write":212,"num_write_kb":67093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"98'428","reported_seq":596,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.542303+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:10:05.542303+0000","last_peered":"2024-09-16T00:10:05.542303+0000","last_clean":"2024-09-16T00:10:05.542303+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:10:05.542303+0000","last_undegraded":"2024-09-16T00:10:05.542303+0000","last_fullsized":"2024-09-16T00:10:05.542303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":415,"num_write_kb":8413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"99'229","reported_seq":423,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.881557+0000","last_change":"2024-09-16T00:07:33.629539+0000","last_active":"2024-09-16T00:10:05.881557+0000","last_peered":"2024-09-16T00:10:05.881557+0000","last_clean":"2024-09-16T00:10:05.881557+0000","last_became_active":"2024-09-16T00:07:33.629242+0000","last_became_peered":"2024-09-16T00:07:33.629242+0000","last_unstale":"2024-09-16T00:10:05.881557+0000","last_undegraded":"2024-09-16T00:10:05.881557+0000","last_fullsized":"2024-09-16T00:10:05.881557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:41:22.194409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6060822,"num_objects":174,"num_object_clones":0,"num_object_copies":348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":174,"num_whiteouts":0,"num_read":28,"num_read_kb":13263,"num_write":207,"num_write_kb":51174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"100'19","reported_seq":190,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.881888+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:10:06.881888+0000","last_peered":"2024-09-16T00:10:06.881888+0000","last_clean":"2024-09-16T00:10:06.881888+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:10:06.881888+0000","last_undegraded":"2024-09-16T00:10:06.881888+0000","last_fullsized":"2024-09-16T00:10:06.881888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'231","reported_seq":428,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.860806+0000","last_change":"2024-09-16T00:07:33.637578+0000","last_active":"2024-09-16T00:10:06.860806+0000","last_peered":"2024-09-16T00:10:06.860806+0000","last_clean":"2024-09-16T00:10:06.860806+0000","last_became_active":"2024-09-16T00:07:33.637289+0000","last_became_peered":"2024-09-16T00:07:33.637289+0000","last_unstale":"2024-09-16T00:10:06.860806+0000","last_undegraded":"2024-09-16T00:10:06.860806+0000","last_fullsized":"2024-09-16T00:10:06.860806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5759002,"num_objects":161,"num_object_clones":0,"num_object_copies":322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":161,"num_whiteouts":0,"num_read":30,"num_read_kb":13990,"num_write":207,"num_write_kb":63962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"100'30","reported_seq":90,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.763159+0000","last_change":"2024-09-16T00:10:00.294519+0000","last_active":"2024-09-16T00:10:07.763159+0000","last_peered":"2024-09-16T00:10:07.763159+0000","last_clean":"2024-09-16T00:10:07.763159+0000","last_became_active":"2024-09-16T00:10:00.283066+0000","last_became_peered":"2024-09-16T00:10:00.283066+0000","last_unstale":"2024-09-16T00:10:07.763159+0000","last_undegraded":"2024-09-16T00:10:07.763159+0000","last_fullsized":"2024-09-16T00:10:07.763159+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:32.621176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"101'249","reported_seq":344,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.870945+0000","last_change":"2024-09-16T00:10:00.460910+0000","last_active":"2024-09-16T00:10:07.870945+0000","last_peered":"2024-09-16T00:10:07.870945+0000","last_clean":"2024-09-16T00:10:07.870945+0000","last_became_active":"2024-09-16T00:10:00.280750+0000","last_became_peered":"2024-09-16T00:10:00.280750+0000","last_unstale":"2024-09-16T00:10:07.870945+0000","last_undegraded":"2024-09-16T00:10:07.870945+0000","last_fullsized":"2024-09-16T00:10:07.870945+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:47.634650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":957990,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":35,"num_read_kb":19573,"num_write":227,"num_write_kb":50826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":33491520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'23","reported_seq":190,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.298071+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:10:04.298071+0000","last_peered":"2024-09-16T00:10:04.298071+0000","last_clean":"2024-09-16T00:10:04.298071+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:10:04.298071+0000","last_undegraded":"2024-09-16T00:10:04.298071+0000","last_fullsized":"2024-09-16T00:10:04.298071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"100'264","reported_seq":463,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.327916+0000","last_change":"2024-09-16T00:07:33.639738+0000","last_active":"2024-09-16T00:10:06.327916+0000","last_peered":"2024-09-16T00:10:06.327916+0000","last_clean":"2024-09-16T00:10:06.327916+0000","last_became_active":"2024-09-16T00:07:33.639450+0000","last_became_peered":"2024-09-16T00:07:33.639450+0000","last_unstale":"2024-09-16T00:10:06.327916+0000","last_undegraded":"2024-09-16T00:10:06.327916+0000","last_fullsized":"2024-09-16T00:10:06.327916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":889252,"num_objects":192,"num_object_clones":0,"num_object_copies":384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":192,"num_whiteouts":0,"num_read":32,"num_read_kb":20419,"num_write":235,"num_write_kb":43035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"100'22","reported_seq":192,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.861832+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:10:06.861832+0000","last_peered":"2024-09-16T00:10:06.861832+0000","last_clean":"2024-09-16T00:10:06.861832+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:10:06.861832+0000","last_undegraded":"2024-09-16T00:10:06.861832+0000","last_fullsized":"2024-09-16T00:10:06.861832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"100'208","reported_seq":403,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.893550+0000","last_change":"2024-09-16T00:07:33.636927+0000","last_active":"2024-09-16T00:10:06.893550+0000","last_peered":"2024-09-16T00:10:06.893550+0000","last_clean":"2024-09-16T00:10:06.893550+0000","last_became_active":"2024-09-16T00:07:33.636347+0000","last_became_peered":"2024-09-16T00:07:33.636347+0000","last_unstale":"2024-09-16T00:10:06.893550+0000","last_undegraded":"2024-09-16T00:10:06.893550+0000","last_fullsized":"2024-09-16T00:10:06.893550+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:03.010823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":573831,"num_objects":150,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":150,"num_whiteouts":0,"num_read":28,"num_read_kb":10790,"num_write":180,"num_write_kb":21585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"98'243","reported_seq":410,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.903119+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:10:04.903119+0000","last_peered":"2024-09-16T00:10:04.903119+0000","last_clean":"2024-09-16T00:10:04.903119+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:10:04.903119+0000","last_undegraded":"2024-09-16T00:10:04.903119+0000","last_fullsized":"2024-09-16T00:10:04.903119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":236,"num_write_kb":4238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"99'210","reported_seq":382,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.328322+0000","last_change":"2024-09-16T00:07:33.639640+0000","last_active":"2024-09-16T00:10:06.328322+0000","last_peered":"2024-09-16T00:10:06.328322+0000","last_clean":"2024-09-16T00:10:06.328322+0000","last_became_active":"2024-09-16T00:07:33.639342+0000","last_became_peered":"2024-09-16T00:07:33.639342+0000","last_unstale":"2024-09-16T00:10:06.328322+0000","last_undegraded":"2024-09-16T00:10:06.328322+0000","last_fullsized":"2024-09-16T00:10:06.328322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:17.957110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582034,"num_objects":160,"num_object_clones":0,"num_object_copies":320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":160,"num_whiteouts":0,"num_read":5,"num_read_kb":350,"num_write":191,"num_write_kb":38221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"99'490","reported_seq":658,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.875825+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:10:05.875825+0000","last_peered":"2024-09-16T00:10:05.875825+0000","last_clean":"2024-09-16T00:10:05.875825+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:10:05.875825+0000","last_undegraded":"2024-09-16T00:10:05.875825+0000","last_fullsized":"2024-09-16T00:10:05.875825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":472,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"100'237","reported_seq":410,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.890683+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:10:06.890683+0000","last_peered":"2024-09-16T00:10:06.890683+0000","last_clean":"2024-09-16T00:10:06.890683+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:10:06.890683+0000","last_undegraded":"2024-09-16T00:10:06.890683+0000","last_fullsized":"2024-09-16T00:10:06.890683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5326874,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":4,"num_read_kb":675,"num_write":209,"num_write_kb":34696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"98'18","reported_seq":185,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.298225+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:10:04.298225+0000","last_peered":"2024-09-16T00:10:04.298225+0000","last_clean":"2024-09-16T00:10:04.298225+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:10:04.298225+0000","last_undegraded":"2024-09-16T00:10:04.298225+0000","last_fullsized":"2024-09-16T00:10:04.298225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'226","reported_seq":449,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.891625+0000","last_change":"2024-09-16T00:08:47.953122+0000","last_active":"2024-09-16T00:10:06.891625+0000","last_peered":"2024-09-16T00:10:06.891625+0000","last_clean":"2024-09-16T00:10:06.891625+0000","last_became_active":"2024-09-16T00:08:47.952799+0000","last_became_peered":"2024-09-16T00:08:47.952799+0000","last_unstale":"2024-09-16T00:10:06.891625+0000","last_undegraded":"2024-09-16T00:10:06.891625+0000","last_fullsized":"2024-09-16T00:10:06.891625+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:03.447562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":913003,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":46,"num_read_kb":22590,"num_write":210,"num_write_kb":59088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":325,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.327921+0000","last_change":"2024-09-16T00:07:23.967734+0000","last_active":"2024-09-16T00:10:06.327921+0000","last_peered":"2024-09-16T00:10:06.327921+0000","last_clean":"2024-09-16T00:10:06.327921+0000","last_became_active":"2024-09-16T00:07:23.964797+0000","last_became_peered":"2024-09-16T00:07:23.964797+0000","last_unstale":"2024-09-16T00:10:06.327921+0000","last_undegraded":"2024-09-16T00:10:06.327921+0000","last_fullsized":"2024-09-16T00:10:06.327921+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:03.935047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"100'13","reported_seq":184,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.853612+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:10:06.853612+0000","last_peered":"2024-09-16T00:10:06.853612+0000","last_clean":"2024-09-16T00:10:06.853612+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:10:06.853612+0000","last_undegraded":"2024-09-16T00:10:06.853612+0000","last_fullsized":"2024-09-16T00:10:06.853612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"99'215","reported_seq":416,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.907475+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:10:05.907475+0000","last_peered":"2024-09-16T00:10:05.907475+0000","last_clean":"2024-09-16T00:10:05.907475+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:10:05.907475+0000","last_undegraded":"2024-09-16T00:10:05.907475+0000","last_fullsized":"2024-09-16T00:10:05.907475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":668263,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":34,"num_read_kb":16103,"num_write":188,"num_write_kb":46241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"100'354","reported_seq":523,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.881764+0000","last_change":"2024-09-16T00:07:32.629361+0000","last_active":"2024-09-16T00:10:06.881764+0000","last_peered":"2024-09-16T00:10:06.881764+0000","last_clean":"2024-09-16T00:10:06.881764+0000","last_became_active":"2024-09-16T00:07:32.629119+0000","last_became_peered":"2024-09-16T00:07:32.629119+0000","last_unstale":"2024-09-16T00:10:06.881764+0000","last_undegraded":"2024-09-16T00:10:06.881764+0000","last_fullsized":"2024-09-16T00:10:06.881764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":344,"num_write_kb":8359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"99'235","reported_seq":448,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.875264+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:10:05.875264+0000","last_peered":"2024-09-16T00:10:05.875264+0000","last_clean":"2024-09-16T00:10:05.875264+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:10:05.875264+0000","last_undegraded":"2024-09-16T00:10:05.875264+0000","last_fullsized":"2024-09-16T00:10:05.875264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4986232,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":47,"num_read_kb":29385,"num_write":209,"num_write_kb":83610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"98'68","reported_seq":291,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.804261+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:10:04.804261+0000","last_peered":"2024-09-16T00:10:04.804261+0000","last_clean":"2024-09-16T00:10:04.804261+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:10:04.804261+0000","last_undegraded":"2024-09-16T00:10:04.804261+0000","last_fullsized":"2024-09-16T00:10:04.804261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":195738,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":225,"num_write":58,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"99'227","reported_seq":446,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.891967+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:10:05.891967+0000","last_peered":"2024-09-16T00:10:05.891967+0000","last_clean":"2024-09-16T00:10:05.891967+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:10:05.891967+0000","last_undegraded":"2024-09-16T00:10:05.891967+0000","last_fullsized":"2024-09-16T00:10:05.891967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4928326,"num_objects":170,"num_object_clones":0,"num_object_copies":340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":170,"num_whiteouts":0,"num_read":53,"num_read_kb":28288,"num_write":208,"num_write_kb":66949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"100'360","reported_seq":531,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.876121+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:10:06.876121+0000","last_peered":"2024-09-16T00:10:06.876121+0000","last_clean":"2024-09-16T00:10:06.876121+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:10:06.876121+0000","last_undegraded":"2024-09-16T00:10:06.876121+0000","last_fullsized":"2024-09-16T00:10:06.876121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":350,"num_write_kb":8381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"100'229","reported_seq":433,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.870503+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:10:06.870503+0000","last_peered":"2024-09-16T00:10:06.870503+0000","last_clean":"2024-09-16T00:10:06.870503+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:10:06.870503+0000","last_undegraded":"2024-09-16T00:10:06.870503+0000","last_fullsized":"2024-09-16T00:10:06.870503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":834000,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":36,"num_read_kb":22188,"num_write":203,"num_write_kb":71552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"99'18","reported_seq":186,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.542423+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:10:05.542423+0000","last_peered":"2024-09-16T00:10:05.542423+0000","last_clean":"2024-09-16T00:10:05.542423+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:10:05.542423+0000","last_undegraded":"2024-09-16T00:10:05.542423+0000","last_fullsized":"2024-09-16T00:10:05.542423+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"100'259","reported_seq":467,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.892269+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:10:06.892269+0000","last_peered":"2024-09-16T00:10:06.892269+0000","last_clean":"2024-09-16T00:10:06.892269+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:10:06.892269+0000","last_undegraded":"2024-09-16T00:10:06.892269+0000","last_fullsized":"2024-09-16T00:10:06.892269+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":501227,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":39,"num_read_kb":24752,"num_write":223,"num_write_kb":75609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"97'15","reported_seq":184,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.328294+0000","last_change":"2024-09-16T00:07:32.628783+0000","last_active":"2024-09-16T00:10:06.328294+0000","last_peered":"2024-09-16T00:10:06.328294+0000","last_clean":"2024-09-16T00:10:06.328294+0000","last_became_active":"2024-09-16T00:07:32.628483+0000","last_became_peered":"2024-09-16T00:07:32.628483+0000","last_unstale":"2024-09-16T00:10:06.328294+0000","last_undegraded":"2024-09-16T00:10:06.328294+0000","last_fullsized":"2024-09-16T00:10:06.328294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:34.718482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"99'239","reported_seq":463,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.900433+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:10:05.900433+0000","last_peered":"2024-09-16T00:10:05.900433+0000","last_clean":"2024-09-16T00:10:05.900433+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:10:05.900433+0000","last_undegraded":"2024-09-16T00:10:05.900433+0000","last_fullsized":"2024-09-16T00:10:05.900433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14187708,"num_objects":157,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":157,"num_whiteouts":0,"num_read":58,"num_read_kb":31221,"num_write":213,"num_write_kb":76143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"99'99","reported_seq":296,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.864821+0000","last_change":"2024-09-16T00:10:00.330950+0000","last_active":"2024-09-16T00:10:05.864821+0000","last_peered":"2024-09-16T00:10:05.864821+0000","last_clean":"2024-09-16T00:10:05.864821+0000","last_became_active":"2024-09-16T00:10:00.283631+0000","last_became_peered":"2024-09-16T00:10:00.283631+0000","last_unstale":"2024-09-16T00:10:05.864821+0000","last_undegraded":"2024-09-16T00:10:05.864821+0000","last_fullsized":"2024-09-16T00:10:05.864821+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:04.314165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":217,"num_write_kb":1451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":1909,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"99'193","reported_seq":399,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.878296+0000","last_change":"2024-09-16T00:07:33.631975+0000","last_active":"2024-09-16T00:10:05.878296+0000","last_peered":"2024-09-16T00:10:05.878296+0000","last_clean":"2024-09-16T00:10:05.878296+0000","last_became_active":"2024-09-16T00:07:33.631680+0000","last_became_peered":"2024-09-16T00:07:33.631680+0000","last_unstale":"2024-09-16T00:10:05.878296+0000","last_undegraded":"2024-09-16T00:10:05.878296+0000","last_fullsized":"2024-09-16T00:10:05.878296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:39:49.315035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":879403,"num_objects":149,"num_object_clones":0,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":39,"num_read_kb":19274,"num_write":174,"num_write_kb":22071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"101'16","reported_seq":162,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.866573+0000","last_change":"2024-09-16T00:10:00.319924+0000","last_active":"2024-09-16T00:10:07.866573+0000","last_peered":"2024-09-16T00:10:07.866573+0000","last_clean":"2024-09-16T00:10:07.866573+0000","last_became_active":"2024-09-16T00:10:00.287979+0000","last_became_peered":"2024-09-16T00:10:00.287979+0000","last_unstale":"2024-09-16T00:10:07.866573+0000","last_undegraded":"2024-09-16T00:10:07.866573+0000","last_fullsized":"2024-09-16T00:10:07.866573+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:15.202658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":18,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"99'268","reported_seq":477,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.886387+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:10:05.886387+0000","last_peered":"2024-09-16T00:10:05.886387+0000","last_clean":"2024-09-16T00:10:05.886387+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:10:05.886387+0000","last_undegraded":"2024-09-16T00:10:05.886387+0000","last_fullsized":"2024-09-16T00:10:05.886387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1683893,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":43,"num_read_kb":17894,"num_write":249,"num_write_kb":88843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"100'515","reported_seq":684,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.327872+0000","last_change":"2024-09-16T00:07:32.629863+0000","last_active":"2024-09-16T00:10:06.327872+0000","last_peered":"2024-09-16T00:10:06.327872+0000","last_clean":"2024-09-16T00:10:06.327872+0000","last_became_active":"2024-09-16T00:07:32.629584+0000","last_became_peered":"2024-09-16T00:07:32.629584+0000","last_unstale":"2024-09-16T00:10:06.327872+0000","last_undegraded":"2024-09-16T00:10:06.327872+0000","last_fullsized":"2024-09-16T00:10:06.327872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":500,"num_write_kb":4358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"100'263","reported_seq":449,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.899857+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:10:06.899857+0000","last_peered":"2024-09-16T00:10:06.899857+0000","last_clean":"2024-09-16T00:10:06.899857+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:10:06.899857+0000","last_undegraded":"2024-09-16T00:10:06.899857+0000","last_fullsized":"2024-09-16T00:10:06.899857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":714314,"num_objects":189,"num_object_clones":0,"num_object_copies":378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":189,"num_whiteouts":0,"num_read":18,"num_read_kb":9206,"num_write":245,"num_write_kb":79714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"99'17","reported_seq":185,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.542401+0000","last_change":"2024-09-16T00:07:32.622348+0000","last_active":"2024-09-16T00:10:05.542401+0000","last_peered":"2024-09-16T00:10:05.542401+0000","last_clean":"2024-09-16T00:10:05.542401+0000","last_became_active":"2024-09-16T00:07:32.621833+0000","last_became_peered":"2024-09-16T00:07:32.621833+0000","last_unstale":"2024-09-16T00:10:05.542401+0000","last_undegraded":"2024-09-16T00:10:05.542401+0000","last_fullsized":"2024-09-16T00:10:05.542401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:34.044889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"99'211","reported_seq":393,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.890385+0000","last_change":"2024-09-16T00:07:33.622385+0000","last_active":"2024-09-16T00:10:05.890385+0000","last_peered":"2024-09-16T00:10:05.890385+0000","last_clean":"2024-09-16T00:10:05.890385+0000","last_became_active":"2024-09-16T00:07:33.622106+0000","last_became_peered":"2024-09-16T00:07:33.622106+0000","last_unstale":"2024-09-16T00:10:05.890385+0000","last_undegraded":"2024-09-16T00:10:05.890385+0000","last_fullsized":"2024-09-16T00:10:05.890385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:04:45.298372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2323343,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":15,"num_read_kb":8384,"num_write":193,"num_write_kb":50710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"99'20","reported_seq":188,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.858710+0000","last_change":"2024-09-16T00:07:32.627447+0000","last_active":"2024-09-16T00:10:05.858710+0000","last_peered":"2024-09-16T00:10:05.858710+0000","last_clean":"2024-09-16T00:10:05.858710+0000","last_became_active":"2024-09-16T00:07:32.627000+0000","last_became_peered":"2024-09-16T00:07:32.627000+0000","last_unstale":"2024-09-16T00:10:05.858710+0000","last_undegraded":"2024-09-16T00:10:05.858710+0000","last_fullsized":"2024-09-16T00:10:05.858710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"99'248","reported_seq":484,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.881767+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:10:05.881767+0000","last_peered":"2024-09-16T00:10:05.881767+0000","last_clean":"2024-09-16T00:10:05.881767+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:10:05.881767+0000","last_undegraded":"2024-09-16T00:10:05.881767+0000","last_fullsized":"2024-09-16T00:10:05.881767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4864327,"num_objects":172,"num_object_clones":0,"num_object_copies":344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":172,"num_whiteouts":0,"num_read":69,"num_read_kb":37740,"num_write":228,"num_write_kb":67449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"100'22","reported_seq":191,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.328059+0000","last_change":"2024-09-16T00:07:32.630838+0000","last_active":"2024-09-16T00:10:06.328059+0000","last_peered":"2024-09-16T00:10:06.328059+0000","last_clean":"2024-09-16T00:10:06.328059+0000","last_became_active":"2024-09-16T00:07:32.630483+0000","last_became_peered":"2024-09-16T00:07:32.630483+0000","last_unstale":"2024-09-16T00:10:06.328059+0000","last_undegraded":"2024-09-16T00:10:06.328059+0000","last_fullsized":"2024-09-16T00:10:06.328059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"101'284","reported_seq":352,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.897576+0000","last_change":"2024-09-16T00:10:00.699942+0000","last_active":"2024-09-16T00:10:07.897576+0000","last_peered":"2024-09-16T00:10:07.897576+0000","last_clean":"2024-09-16T00:10:07.897576+0000","last_became_active":"2024-09-16T00:10:00.281972+0000","last_became_peered":"2024-09-16T00:10:00.281972+0000","last_unstale":"2024-09-16T00:10:07.897576+0000","last_undegraded":"2024-09-16T00:10:07.897576+0000","last_fullsized":"2024-09-16T00:10:07.897576+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:21.943709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":801168,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":82,"num_read_kb":43345,"num_write":257,"num_write_kb":121191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":29546741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"100'249","reported_seq":487,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.905684+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:10:06.905684+0000","last_peered":"2024-09-16T00:10:06.905684+0000","last_clean":"2024-09-16T00:10:06.905684+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:10:06.905684+0000","last_undegraded":"2024-09-16T00:10:06.905684+0000","last_fullsized":"2024-09-16T00:10:06.905684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":529274,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":69,"num_read_kb":32882,"num_write":221,"num_write_kb":108251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"100'124","reported_seq":293,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:07.031563+0000","last_change":"2024-09-16T00:07:32.630562+0000","last_active":"2024-09-16T00:10:07.031563+0000","last_peered":"2024-09-16T00:10:07.031563+0000","last_clean":"2024-09-16T00:10:07.031563+0000","last_became_active":"2024-09-16T00:07:32.630275+0000","last_became_peered":"2024-09-16T00:07:32.630275+0000","last_unstale":"2024-09-16T00:10:07.031563+0000","last_undegraded":"2024-09-16T00:10:07.031563+0000","last_fullsized":"2024-09-16T00:10:07.031563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2108106,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113,"num_write_kb":2126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"100'236","reported_seq":474,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.900042+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:10:06.900042+0000","last_peered":"2024-09-16T00:10:06.900042+0000","last_clean":"2024-09-16T00:10:06.900042+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:10:06.900042+0000","last_undegraded":"2024-09-16T00:10:06.900042+0000","last_fullsized":"2024-09-16T00:10:06.900042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":856930,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":69,"num_read_kb":36278,"num_write":219,"num_write_kb":113361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"100'98","reported_seq":297,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-16T00:10:06.861266+0000","last_change":"2024-09-16T00:10:00.301968+0000","last_active":"2024-09-16T00:10:06.861266+0000","last_peered":"2024-09-16T00:10:06.861266+0000","last_clean":"2024-09-16T00:10:06.861266+0000","last_became_active":"2024-09-16T00:10:00.280330+0000","last_became_peered":"2024-09-16T00:10:00.280330+0000","last_unstale":"2024-09-16T00:10:06.861266+0000","last_undegraded":"2024-09-16T00:10:06.861266+0000","last_fullsized":"2024-09-16T00:10:06.861266+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:10.395459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83,"num_write_kb":8277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":30,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"98'193","reported_seq":350,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-09-16T00:10:04.839867+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:10:04.839867+0000","last_peered":"2024-09-16T00:10:04.839867+0000","last_clean":"2024-09-16T00:10:04.839867+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:10:04.839867+0000","last_undegraded":"2024-09-16T00:10:04.839867+0000","last_fullsized":"2024-09-16T00:10:04.839867+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5212462,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":19,"num_read_kb":12352,"num_write":171,"num_write_kb":38097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"99'49","reported_seq":218,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.844193+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:10:05.844193+0000","last_peered":"2024-09-16T00:10:05.844193+0000","last_clean":"2024-09-16T00:10:05.844193+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:10:05.844193+0000","last_undegraded":"2024-09-16T00:10:05.844193+0000","last_fullsized":"2024-09-16T00:10:05.844193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":4141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"99'245","reported_seq":532,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.859182+0000","last_change":"2024-09-16T00:10:00.556848+0000","last_active":"2024-09-16T00:10:05.859182+0000","last_peered":"2024-09-16T00:10:05.859182+0000","last_clean":"2024-09-16T00:10:05.859182+0000","last_became_active":"2024-09-16T00:10:00.283734+0000","last_became_peered":"2024-09-16T00:10:00.283734+0000","last_unstale":"2024-09-16T00:10:05.859182+0000","last_undegraded":"2024-09-16T00:10:05.859182+0000","last_fullsized":"2024-09-16T00:10:05.859182+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:55:01.843183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10366791,"num_objects":167,"num_object_clones":0,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":0,"num_read":76,"num_read_kb":40163,"num_write":225,"num_write_kb":81063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":21131701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"99'67","reported_seq":235,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-16T00:10:05.891538+0000","last_change":"2024-09-16T00:07:32.629873+0000","last_active":"2024-09-16T00:10:05.891538+0000","last_peered":"2024-09-16T00:10:05.891538+0000","last_clean":"2024-09-16T00:10:05.891538+0000","last_became_active":"2024-09-16T00:07:32.629620+0000","last_became_peered":"2024-09-16T00:07:32.629620+0000","last_unstale":"2024-09-16T00:10:05.891538+0000","last_undegraded":"2024-09-16T00:10:05.891538+0000","last_fullsized":"2024-09-16T00:10:05.891538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:56:25.178999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":4138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":266652695,"num_objects":10686,"num_object_clones":0,"num_object_copies":21372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10686,"num_whiteouts":0,"num_read":2561,"num_read_kb":1362012,"num_write":13731,"num_write_kb":4088607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":433,"num_bytes_recovered":292530836,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":170156032,"data_stored":484283483,"data_compressed":36073906,"data_compressed_allocated":45531136,"data_compressed_original":411141071,"omap_allocated":0,"internal_metadata":0},"log_size":15100,"ondisk_log_size":15100,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":81997090,"num_objects":114,"num_object_clones":0,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":0,"num_read":56,"num_read_kb":225,"num_write":3920,"num_write_kb":84195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":13200,"num_keys_recovered":1983,"num_objects_omap":78,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57200640,"data_stored":164021652,"data_compressed":23525572,"data_compressed_allocated":30339072,"data_compressed_original":137220096,"omap_allocated":225539,"internal_metadata":0},"log_size":4182,"ondisk_log_size":4182,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509477,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69600,"kb_used_data":38416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302240,"statfs":{"total":96636764160,"available":96565493760,"internally_reserved":0,"allocated":39337984,"data_stored":128722246,"data_compressed":12163488,"data_compressed_allocated":15265792,"data_compressed_original":112582856,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":94,"seq":403726925827,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51644,"kb_used_data":22376,"kb_used_omap":221,"kb_used_meta":29026,"kb_avail":94320196,"statfs":{"total":96636764160,"available":96583880704,"internally_reserved":0,"allocated":22913024,"data_stored":47693263,"data_compressed":5421008,"data_compressed_allocated":6537216,"data_compressed_original":37240832,"omap_allocated":227133,"internal_metadata":29722819},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509476,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67304,"kb_used_data":31960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94304536,"statfs":{"total":96636764160,"available":96567844864,"internally_reserved":0,"allocated":32727040,"data_stored":113512292,"data_compressed":8764433,"data_compressed_allocated":11464704,"data_compressed_original":100405014,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509477,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92532,"kb_used_data":39132,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94279308,"statfs":{"total":96636764160,"available":96542011392,"internally_reserved":0,"allocated":40071168,"data_stored":118809031,"data_compressed":10131031,"data_compressed_allocated":13107200,"data_compressed_original":99938182,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509475,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62144,"kb_used_data":30956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94309696,"statfs":{"total":96636764160,"available":96573128704,"internally_reserved":0,"allocated":31698944,"data_stored":142894816,"data_compressed":10441672,"data_compressed_allocated":13262848,"data_compressed_original":132760869,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542180,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55600,"kb_used_data":28572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94316240,"statfs":{"total":96636764160,"available":96579829760,"internally_reserved":0,"allocated":29257728,"data_stored":95007838,"data_compressed":7236032,"data_compressed_allocated":9465856,"data_compressed_original":80777479,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542180,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74604,"kb_used_data":43412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94297236,"statfs":{"total":96636764160,"available":96560369664,"internally_reserved":0,"allocated":44453888,"data_stored":201010966,"data_compressed":12781574,"data_compressed_allocated":16740352,"data_compressed_original":180988079,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61408,"kb_used_data":21904,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94310432,"statfs":{"total":96636764160,"available":96573882368,"internally_reserved":0,"allocated":22429696,"data_stored":151808590,"data_compressed":8684313,"data_compressed_allocated":11386880,"data_compressed_original":145954232,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7749632,"data_stored":20972158,"data_compressed":2936172,"data_compressed_allocated":3887104,"data_compressed_original":17121280,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8351744,"data_stored":20971680,"data_compressed":2925068,"data_compressed_allocated":3854336,"data_compressed_original":16482304,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1835008,"data_stored":4390042,"data_compressed":600007,"data_compressed_allocated":819200,"data_compressed_original":3375104,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4550656,"data_stored":17001098,"data_compressed":2587033,"data_compressed_allocated":3162112,"data_compressed_original":15622144,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12107776,"data_stored":29360254,"data_compressed":4035409,"data_compressed_allocated":5427200,"data_compressed_original":22691840,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8396800,"data_stored":18885412,"data_compressed":2588884,"data_compressed_allocated":3518464,"data_compressed_original":14012416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3444736,"data_stored":16777888,"data_compressed":2643028,"data_compressed_allocated":3112960,"data_compressed_original":16453632,"omap_allocated":225539,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10764288,"data_stored":35663120,"data_compressed":5209971,"data_compressed_allocated":6557696,"data_compressed_original":31461376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":31567872,"data_stored":120026655,"data_compressed":7028518,"data_compressed_allocated":9076736,"data_compressed_original":104519619,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19738624,"data_stored":73483805,"data_compressed":4310964,"data_compressed_allocated":5611520,"data_compressed_original":64295175,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":17551360,"data_stored":115204320,"data_compressed":6654056,"data_compressed_allocated":8654848,"data_compressed_original":110864767,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22536192,"data_stored":72329904,"data_compressed":5346322,"data_compressed_allocated":6766592,"data_compressed_original":64194856,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":22589440,"data_stored":23109601,"data_compressed":2990360,"data_compressed_allocated":3551232,"data_compressed_original":11505706,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":19238912,"data_stored":32573040,"data_compressed":3315963,"data_compressed_allocated":4128768,"data_compressed_original":24914970,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":13205504,"data_stored":13520627,"data_compressed":2181098,"data_compressed_allocated":2633728,"data_compressed_original":8196096,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":23728128,"data_stored":34035531,"data_compressed":4246625,"data_compressed_allocated":5107712,"data_compressed_original":22649882,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:10:10.098 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T00:10:10.098 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:10:10.098 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-16T00:10:10.099 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T00:10:10.099 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T00:10:10.099 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-16T00:10:10.117 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:10.117 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:10.129 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:10:10.170 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:10.170+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:10.242 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:10.242 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "" 2024-09-16T00:10:10.242 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:10.255 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:10.255+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:10.277 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:10.277 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:10.286 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:10:10.371 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:10.371+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:10.457 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:10.456+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:10.462 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:10.463 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:10.472 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:10:10.558 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:10.557+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:10.633 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:10.633 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:10.645 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:10:10.659 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:10.659+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:10.757 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:10.757 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:10.760 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:10.760+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:10.767 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:10:10.859 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:10:10.861 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:10.860+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:10.940 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:10.940 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:10.957 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:10:10.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.980+0000 7fabc6837640 1 Processor -- start 2024-09-16T00:10:10.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.980+0000 7fabc6837640 1 -- start start 2024-09-16T00:10:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.981+0000 7fabc6837640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fabc00d0490 0x7fabc00b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.981+0000 7fabc6837640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fabc00b4750 0x7fabc01546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.981+0000 7fabc6837640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc00b3180 con 0x7fabc00d0870 2024-09-16T00:10:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.981+0000 7fabc6837640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fabc015b940 con 0x7fabc00d0490 2024-09-16T00:10:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.981+0000 7fabc6837640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fabc00b3bb0 con 0x7fabc00b4750 2024-09-16T00:10:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.981+0000 7fabbffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fabc00d0490 0x7fabc00b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.981+0000 7fabbffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fabc00d0490 0x7fabc00b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49192/0 (socket says 172.21.15.139:49192) 2024-09-16T00:10:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.981+0000 7fabbffff640 1 -- 172.21.15.139:0/3631918704 learned_addr learned my addr 172.21.15.139:0/3631918704 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:10.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.981+0000 7fabc4dd4640 1 --2- 172.21.15.139:0/3631918704 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fabc00b4750 0x7fabc01546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.981+0000 7fabbffff640 1 -- 172.21.15.139:0/3631918704 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fabc00b4750 msgr2=0x7fabc01546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.981+0000 7fabbffff640 1 --2- 172.21.15.139:0/3631918704 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fabc00b4750 0x7fabc01546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.981+0000 7fabbffff640 1 -- 172.21.15.139:0/3631918704 >> v1:172.21.15.164:6789/0 conn(0x7fabc00d0870 legacy=0x7fabc00b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.981+0000 7fabbffff640 1 -- 172.21.15.139:0/3631918704 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc015adc0 con 0x7fabc00d0490 2024-09-16T00:10:10.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.982+0000 7fabbffff640 1 --2- 172.21.15.139:0/3631918704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc00d0490 0x7fabc00b2c40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009870 tx=0x7fabb002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5b23ca93ab56a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.982+0000 7fabbeffd640 1 -- 172.21.15.139:0/3631918704 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0037070 con 0x7fabc00d0490 2024-09-16T00:10:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.982+0000 7fabbeffd640 1 -- 172.21.15.139:0/3631918704 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb002ae50 con 0x7fabc00d0490 2024-09-16T00:10:10.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.982+0000 7fabbeffd640 1 -- 172.21.15.139:0/3631918704 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0032860 con 0x7fabc00d0490 2024-09-16T00:10:10.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.983+0000 7fabc6837640 1 -- 172.21.15.139:0/3631918704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc00d0490 msgr2=0x7fabc00b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:10.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.983+0000 7fabc6837640 1 --2- 172.21.15.139:0/3631918704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc00d0490 0x7fabc00b2c40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009870 tx=0x7fabb002a300 comp rx=0 tx=0).stop 2024-09-16T00:10:10.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.983+0000 7fabc6837640 1 -- 172.21.15.139:0/3631918704 shutdown_connections 2024-09-16T00:10:10.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.983+0000 7fabc6837640 1 --2- 172.21.15.139:0/3631918704 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fabc00b4750 0x7fabc01546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.983+0000 7fabc6837640 1 --2- 172.21.15.139:0/3631918704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc00d0490 0x7fabc00b2c40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.983+0000 7fabc6837640 1 -- 172.21.15.139:0/3631918704 >> 172.21.15.139:0/3631918704 conn(0x7fabc006c520 msgr2=0x7fabc006c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:10.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.983+0000 7fabc6837640 1 -- 172.21.15.139:0/3631918704 shutdown_connections 2024-09-16T00:10:10.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.983+0000 7fabc6837640 1 -- 172.21.15.139:0/3631918704 wait complete. 2024-09-16T00:10:10.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.984+0000 7fabc6837640 1 Processor -- start 2024-09-16T00:10:10.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.984+0000 7fabc6837640 1 -- start start 2024-09-16T00:10:10.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabc6837640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabc00b4750 0x7fabc0155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:10.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabc6837640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc00d0490 0x7fabc00ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:10.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabc6837640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fabc00d0870 0x7fabc0152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:10.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabc6837640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fabc015b660 con 0x7fabc00d0490 2024-09-16T00:10:10.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabc6837640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fabc015b380 con 0x7fabc00b4750 2024-09-16T00:10:10.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabc6837640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fabc015a980 con 0x7fabc00d0870 2024-09-16T00:10:10.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabbf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc00d0490 0x7fabc00ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:10.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabbf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc00d0490 0x7fabc00ccf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49194/0 (socket says 172.21.15.139:49194) 2024-09-16T00:10:10.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabbf7fe640 1 -- 172.21.15.139:0/3124209998 learned_addr learned my addr 172.21.15.139:0/3124209998 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:10.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabc4dd4640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fabc00d0870 0x7fabc0152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:10.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabbffff640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabc00b4750 0x7fabc0155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:10.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabbf7fe640 1 -- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fabc00d0870 msgr2=0x7fabc0152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:10.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabbf7fe640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fabc00d0870 0x7fabc0152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabbf7fe640 1 -- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabc00b4750 msgr2=0x7fabc0155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:10.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabbf7fe640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fabc00b4750 0x7fabc0155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:10.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabbf7fe640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc014f8d0 con 0x7fabc00d0490 2024-09-16T00:10:10.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.985+0000 7fabbf7fe640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc00d0490 0x7fabc00ccf10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fabb400c3f0 tx=0x7fabb400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:10.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.986+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb40052d0 con 0x7fabc00d0490 2024-09-16T00:10:10.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.986+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb4005490 con 0x7fabc00d0490 2024-09-16T00:10:10.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.986+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4007260 con 0x7fabc00d0490 2024-09-16T00:10:10.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.986+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc014fb40 con 0x7fabc00d0490 2024-09-16T00:10:10.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.986+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc01f3e10 con 0x7fabc00d0490 2024-09-16T00:10:10.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.987+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fabc00b3430 con 0x7fabc00d0490 2024-09-16T00:10:10.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.988+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fabb4007400 con 0x7fabc00d0490 2024-09-16T00:10:10.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.988+0000 7fabbcff9640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fabac06c2b0 0x7fabac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:10.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.988+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5973+0+0 (secure 0 0 0) 0x7fabb408a8f0 con 0x7fabc00d0490 2024-09-16T00:10:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.988+0000 7fabbffff640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fabac06c2b0 0x7fabac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.988+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v103) ==== 54+0+16 (secure 0 0 0) 0x7fabb408f240 con 0x7fabc00d0490 2024-09-16T00:10:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.988+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fabc0152fd0 0x7fabc01f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.988+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fabc014a130 con 0x7fabc0152fd0 2024-09-16T00:10:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.989+0000 7fabbffff640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fabac06c2b0 0x7fabac06e770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fabb0034000 tx=0x7fabb002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.989+0000 7fabc4dd4640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fabc0152fd0 0x7fabc01f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.989+0000 7fabc4dd4640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fabc0152fd0 0x7fabc01f4010 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:10.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:10.989+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fabc014a130 con 0x7fabc0152fd0 2024-09-16T00:10:11.001 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:11.001+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:11.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.005+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fabc00cb2d0 con 0x7fabc0152fd0 2024-09-16T00:10:11.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.007+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fabc00cb2d0 con 0x7fabc0152fd0 2024-09-16T00:10:11.008 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:11.008 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:11.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.007+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fabc0147610 0x7fabc01479f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:11.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.007+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fabc0147f30 con 0x7fabc0147610 2024-09-16T00:10:11.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.007+0000 7fabbffff640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fabc0147610 0x7fabc01479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:11.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.007+0000 7fabbffff640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fabc0147610 0x7fabc01479f0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:11.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.008+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fabc0147f30 con 0x7fabc0147610 2024-09-16T00:10:11.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.023+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fabc00caff0 con 0x7fabc0147610 2024-09-16T00:10:11.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.024+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fabc00caff0 con 0x7fabc0147610 2024-09-16T00:10:11.025 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:11.026 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:11.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.024+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fabc00cf610 0x7fabc00cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:11.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.024+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fabc00cff30 con 0x7fabc00cf610 2024-09-16T00:10:11.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.024+0000 7fabbf7fe640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fabc00cf610 0x7fabc00cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:11.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.025+0000 7fabbf7fe640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fabc00cf610 0x7fabc00cf9f0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:11.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.025+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fabc00cff30 con 0x7fabc00cf610 2024-09-16T00:10:11.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.040+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fabc0062630 con 0x7fabc00cf610 2024-09-16T00:10:11.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.041+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fabc0062630 con 0x7fabc00cf610 2024-09-16T00:10:11.042 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:11.044 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:11.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.041+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fabc0062fa0 0x7fabc00cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:11.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.041+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fabc00633c0 con 0x7fabc0062fa0 2024-09-16T00:10:11.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.041+0000 7fabc4dd4640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fabc0062fa0 0x7fabc00cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:11.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.042+0000 7fabc4dd4640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fabc0062fa0 0x7fabc00cb630 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:11.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.042+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fabc00633c0 con 0x7fabc0062fa0 2024-09-16T00:10:11.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.057+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fabc00633c0 con 0x7fabc0062fa0 2024-09-16T00:10:11.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.059+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fabc00633c0 con 0x7fabc0062fa0 2024-09-16T00:10:11.060 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:11.061 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.059+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fabc002c640 0x7fabc002ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.059+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fabc00635d0 con 0x7fabc002c640 2024-09-16T00:10:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.059+0000 7fabbffff640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fabc002c640 0x7fabc002ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.060+0000 7fabbffff640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fabc002c640 0x7fabc002ca60 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:11.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.061+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fabc00635d0 con 0x7fabc002c640 2024-09-16T00:10:11.063 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:11.063+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:11.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.075+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fabc00635d0 con 0x7fabc002c640 2024-09-16T00:10:11.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.077+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fabc00635d0 con 0x7fabc002c640 2024-09-16T00:10:11.079 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.078+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fabc002d4b0 0x7fabc0206e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.078+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fabc002daf0 con 0x7fabc002d4b0 2024-09-16T00:10:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.078+0000 7fabbf7fe640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fabc002d4b0 0x7fabc0206e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:11.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.078+0000 7fabbf7fe640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fabc002d4b0 0x7fabc0206e20 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:11.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.079+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fabc002daf0 con 0x7fabc002d4b0 2024-09-16T00:10:11.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.094+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fabc002dac0 con 0x7fabc002d4b0 2024-09-16T00:10:11.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.096+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fabc002dac0 con 0x7fabc002d4b0 2024-09-16T00:10:11.096 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:11.097 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:11.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.096+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7fabc0207870 0x7fabc0209c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:11.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.096+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fabc020cc40 con 0x7fabc0207870 2024-09-16T00:10:11.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.096+0000 7fabc4dd4640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7fabc0207870 0x7fabc0209c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:11.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.096+0000 7fabc4dd4640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7fabc0207870 0x7fabc0209c60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:11.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.097+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fabc020cc40 con 0x7fabc0207870 2024-09-16T00:10:11.110 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:11.110 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:11.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.111+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fabc020cc10 con 0x7fabc0207870 2024-09-16T00:10:11.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.113+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fabc020cc10 con 0x7fabc0207870 2024-09-16T00:10:11.114 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:11.114 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:11.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.113+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fabc020be70 0x7fabc020c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:11.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.113+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fabc0210010 con 0x7fabc020be70 2024-09-16T00:10:11.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.113+0000 7fabbffff640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fabc020be70 0x7fabc020c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:11.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.114+0000 7fabbffff640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fabc020be70 0x7fabc020c2d0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:11.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.115+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fabc0210010 con 0x7fabc020be70 2024-09-16T00:10:11.120 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:10:11.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.129+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fabc020a220 con 0x7fabc020be70 2024-09-16T00:10:11.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabbcff9640 1 -- 172.21.15.139:0/3124209998 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fabc020a220 con 0x7fabc020be70 2024-09-16T00:10:11.132 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:10:11.133 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:10:11.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fabc0152fd0 msgr2=0x7fabc01f4010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fabc0152fd0 0x7fabc01f4010 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fabc0147610 msgr2=0x7fabc01479f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fabc0147610 0x7fabc01479f0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fabc00cf610 msgr2=0x7fabc00cf9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7fabc00cf610 0x7fabc00cf9f0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc4dd4640 1 -- 172.21.15.139:0/3124209998 reap_dead start 2024-09-16T00:10:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fabc0062fa0 msgr2=0x7fabc00cb630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fabc0062fa0 0x7fabc00cb630 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fabc002c640 msgr2=0x7fabc002ca60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7fabc002c640 0x7fabc002ca60 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fabc002d4b0 msgr2=0x7fabc0206e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7fabc002d4b0 0x7fabc0206e20 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7fabc0207870 msgr2=0x7fabc0209c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7fabc0207870 0x7fabc0209c60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fabc020be70 msgr2=0x7fabc020c2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7fabc020be70 0x7fabc020c2d0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc4dd4640 1 -- 172.21.15.139:0/3124209998 reap_dead start 2024-09-16T00:10:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fabac06c2b0 msgr2=0x7fabac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fabac06c2b0 0x7fabac06e770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fabb0034000 tx=0x7fabb002e000 comp rx=0 tx=0).stop 2024-09-16T00:10:11.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc00d0490 msgr2=0x7fabc00ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc00d0490 0x7fabc00ccf10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fabb400c3f0 tx=0x7fabb400c8b0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 shutdown_connections 2024-09-16T00:10:11.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fabac06c2b0 0x7fabac06e770 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 --2- 172.21.15.139:0/3124209998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabc00d0490 0x7fabc00ccf10 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 >> 172.21.15.139:0/3124209998 conn(0x7fabc006c520 msgr2=0x7fabc014a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:11.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 shutdown_connections 2024-09-16T00:10:11.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.131+0000 7fabc6837640 1 -- 172.21.15.139:0/3124209998 wait complete. 2024-09-16T00:10:11.205 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:11.205+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:11.264 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:11.263+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:11.281 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:11.281 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:11.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:11.281 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:11.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.281 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5178 3.2c 3:375e1b9b:::100000030c8.00000002:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:11.282 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:11.282 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.4691167370000002, 2024-09-16T00:10:11.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012118438, 2024-09-16T00:10:11.282 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.282 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.282 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.282 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:11.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:11.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808909+0000", 2024-09-16T00:10:11.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8160000000000001e-06 2024-09-16T00:10:11.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808921+0000", 2024-09-16T00:10:11.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1117999999999999e-05 2024-09-16T00:10:11.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808941+0000", 2024-09-16T00:10:11.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0712000000000001e-05 2024-09-16T00:10:11.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808952+0000", 2024-09-16T00:10:11.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0193e-05 2024-09-16T00:10:11.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809019+0000", 2024-09-16T00:10:11.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7404999999999996e-05 2024-09-16T00:10:11.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809100+0000", 2024-09-16T00:10:11.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1029e-05 2024-09-16T00:10:11.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:11.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809206+0000", 2024-09-16T00:10:11.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105603 2024-09-16T00:10:11.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:11.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.815162+0000", 2024-09-16T00:10:11.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059567250000000004 2024-09-16T00:10:11.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.820925+0000", 2024-09-16T00:10:11.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057623479999999996 2024-09-16T00:10:11.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:11.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.820987+0000", 2024-09-16T00:10:11.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2680000000000003e-05 2024-09-16T00:10:11.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.821026+0000", 2024-09-16T00:10:11.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8809000000000001e-05 2024-09-16T00:10:11.287 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.287 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.287 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.287 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5196 3.26 3:66ea4dfb:::100000030f0.00000001:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:11.287 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:11.287 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.4679297089999999, 2024-09-16T00:10:11.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095222459999999998, 2024-09-16T00:10:11.288 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.288 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.288 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.288 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:11.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:11.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:11.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3499999999999995e-07 2024-09-16T00:10:11.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810102+0000", 2024-09-16T00:10:11.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7390000000000002e-06 2024-09-16T00:10:11.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810112+0000", 2024-09-16T00:10:11.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7550000000000006e-06 2024-09-16T00:10:11.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810127+0000", 2024-09-16T00:10:11.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4749999999999999e-05 2024-09-16T00:10:11.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810184+0000", 2024-09-16T00:10:11.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7701999999999997e-05 2024-09-16T00:10:11.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810256+0000", 2024-09-16T00:10:11.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1595999999999997e-05 2024-09-16T00:10:11.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:11.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810346+0000", 2024-09-16T00:10:11.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0043999999999994e-05 2024-09-16T00:10:11.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:11.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.815239+0000", 2024-09-16T00:10:11.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048934470000000004 2024-09-16T00:10:11.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819526+0000", 2024-09-16T00:10:11.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042868630000000001 2024-09-16T00:10:11.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:11.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819577+0000", 2024-09-16T00:10:11.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0445999999999998e-05 2024-09-16T00:10:11.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819617+0000", 2024-09-16T00:10:11.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0169000000000001e-05 2024-09-16T00:10:11.292 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.293 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.293 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.293 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5244 3.20 3:068a06d7:::100000032f2.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:11.293 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:11.293 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.4508342019999994, 2024-09-16T00:10:11.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094083010000000009, 2024-09-16T00:10:11.293 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.293 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.293 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:11.293 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:11.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:11.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827191+0000", 2024-09-16T00:10:11.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2330000000000001e-06 2024-09-16T00:10:11.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827195+0000", 2024-09-16T00:10:11.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8750000000000002e-06 2024-09-16T00:10:11.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827206+0000", 2024-09-16T00:10:11.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0473e-05 2024-09-16T00:10:11.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827213+0000", 2024-09-16T00:10:11.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:10:11.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827275+0000", 2024-09-16T00:10:11.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2532000000000004e-05 2024-09-16T00:10:11.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827392+0000", 2024-09-16T00:10:11.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116266 2024-09-16T00:10:11.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.836599+0000", 2024-09-16T00:10:11.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092068069999999991 2024-09-16T00:10:11.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.297 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.297 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.297 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5245 3.26 3:64562175:::100000032f3.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:11.297 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.827707+0000", 2024-09-16T00:10:11.297 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.4503174320000003, 2024-09-16T00:10:11.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090401860000000004, 2024-09-16T00:10:11.297 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.298 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.298 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:11.298 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827707+0000", 2024-09-16T00:10:11.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827707+0000", 2024-09-16T00:10:11.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827708+0000", 2024-09-16T00:10:11.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5199999999999996e-07 2024-09-16T00:10:11.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827712+0000", 2024-09-16T00:10:11.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9530000000000001e-06 2024-09-16T00:10:11.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827722+0000", 2024-09-16T00:10:11.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0524e-05 2024-09-16T00:10:11.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827736+0000", 2024-09-16T00:10:11.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3713999999999999e-05 2024-09-16T00:10:11.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827801+0000", 2024-09-16T00:10:11.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5580000000000006e-05 2024-09-16T00:10:11.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827924+0000", 2024-09-16T00:10:11.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012308499999999999 2024-09-16T00:10:11.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.836747+0000", 2024-09-16T00:10:11.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088226780000000005 2024-09-16T00:10:11.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5251 2.17 2:e817bfdf:::100000037ad.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:11.301 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.827758+0000", 2024-09-16T00:10:11.301 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.4502668080000003, 2024-09-16T00:10:11.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090250260000000002, 2024-09-16T00:10:11.302 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.302 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.302 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:11.302 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827758+0000", 2024-09-16T00:10:11.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827758+0000", 2024-09-16T00:10:11.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827758+0000", 2024-09-16T00:10:11.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7400000000000003e-07 2024-09-16T00:10:11.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827761+0000", 2024-09-16T00:10:11.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.095e-06 2024-09-16T00:10:11.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827772+0000", 2024-09-16T00:10:11.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0767e-05 2024-09-16T00:10:11.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827779+0000", 2024-09-16T00:10:11.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2540000000000003e-06 2024-09-16T00:10:11.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827837+0000", 2024-09-16T00:10:11.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7615000000000003e-05 2024-09-16T00:10:11.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827942+0000", 2024-09-16T00:10:11.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105393 2024-09-16T00:10:11.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.836783+0000", 2024-09-16T00:10:11.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088403279999999997 2024-09-16T00:10:11.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.306 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.306 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5252 2.1a 2:5913b086:::1000000336d.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:11.306 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:11.306 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.4498983790000004, 2024-09-16T00:10:11.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010086441999999999, 2024-09-16T00:10:11.306 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.306 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.306 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:11.306 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:11.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:11.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828127+0000", 2024-09-16T00:10:11.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:10:11.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828140+0000", 2024-09-16T00:10:11.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3614e-05 2024-09-16T00:10:11.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828149+0000", 2024-09-16T00:10:11.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7250000000000003e-06 2024-09-16T00:10:11.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828156+0000", 2024-09-16T00:10:11.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9430000000000003e-06 2024-09-16T00:10:11.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828274+0000", 2024-09-16T00:10:11.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118121 2024-09-16T00:10:11.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828381+0000", 2024-09-16T00:10:11.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107379 2024-09-16T00:10:11.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.838212+0000", 2024-09-16T00:10:11.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098310089999999999 2024-09-16T00:10:11.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.310 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.310 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.310 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5286 3.26 3:661d81ee:::100000034b1.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:11.310 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.835257+0000", 2024-09-16T00:10:11.310 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.4427677019999994, 2024-09-16T00:10:11.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084025320000000008, 2024-09-16T00:10:11.310 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.311 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.311 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:11.311 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835257+0000", 2024-09-16T00:10:11.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835257+0000", 2024-09-16T00:10:11.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835257+0000", 2024-09-16T00:10:11.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4700000000000002e-07 2024-09-16T00:10:11.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835260+0000", 2024-09-16T00:10:11.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.593e-06 2024-09-16T00:10:11.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835267+0000", 2024-09-16T00:10:11.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4880000000000001e-06 2024-09-16T00:10:11.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835274+0000", 2024-09-16T00:10:11.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2920000000000001e-06 2024-09-16T00:10:11.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835358+0000", 2024-09-16T00:10:11.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4907999999999994e-05 2024-09-16T00:10:11.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.835475+0000", 2024-09-16T00:10:11.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116267 2024-09-16T00:10:11.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.843659+0000", 2024-09-16T00:10:11.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081845370000000004 2024-09-16T00:10:11.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.314 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.314 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.314 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5296 3.10 3:08253ab3:::100000037b2.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:11.315 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.839046+0000", 2024-09-16T00:10:11.315 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.4389785580000005, 2024-09-16T00:10:11.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086364030000000008, 2024-09-16T00:10:11.315 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.315 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.315 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:11.315 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839046+0000", 2024-09-16T00:10:11.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839046+0000", 2024-09-16T00:10:11.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839047+0000", 2024-09-16T00:10:11.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0299999999999998e-07 2024-09-16T00:10:11.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839050+0000", 2024-09-16T00:10:11.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4989999999999999e-06 2024-09-16T00:10:11.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839062+0000", 2024-09-16T00:10:11.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2128e-05 2024-09-16T00:10:11.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839646+0000", 2024-09-16T00:10:11.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00058339999999999998 2024-09-16T00:10:11.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839707+0000", 2024-09-16T00:10:11.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1780999999999997e-05 2024-09-16T00:10:11.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.839842+0000", 2024-09-16T00:10:11.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013437799999999999 2024-09-16T00:10:11.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.847682+0000", 2024-09-16T00:10:11.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0078404140000000004 2024-09-16T00:10:11.319 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.319 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.319 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.319 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "pg_update_log_missing(2.19 epoch 96/94 rep_tid 2313 entries 96'15 (0'0) error 2:9f9cad8a:::100000033d1.00000000:head by mds.0.4:5304 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 95'13)", 2024-09-16T00:10:11.319 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.841700+0000", 2024-09-16T00:10:11.319 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.4363244589999997, 2024-09-16T00:10:11.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086308140000000005, 2024-09-16T00:10:11.319 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.319 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.319 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "reached pg", 2024-09-16T00:10:11.319 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841700+0000", 2024-09-16T00:10:11.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841700+0000", 2024-09-16T00:10:11.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841701+0000", 2024-09-16T00:10:11.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4199999999999995e-07 2024-09-16T00:10:11.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841709+0000", 2024-09-16T00:10:11.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4060000000000005e-06 2024-09-16T00:10:11.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841728+0000", 2024-09-16T00:10:11.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8868999999999998e-05 2024-09-16T00:10:11.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841741+0000", 2024-09-16T00:10:11.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3283e-05 2024-09-16T00:10:11.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841834+0000", 2024-09-16T00:10:11.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2312999999999995e-05 2024-09-16T00:10:11.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.850331+0000", 2024-09-16T00:10:11.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084972009999999994 2024-09-16T00:10:11.322 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.322 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.323 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:5307 3.1e e96/94 3:7b9ea4ae:::10000003595.00000000:head v 96'234, mlcod=95'232)", 2024-09-16T00:10:11.323 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.841762+0000", 2024-09-16T00:10:11.323 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.4362619409999997, 2024-09-16T00:10:11.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086486400000000008, 2024-09-16T00:10:11.323 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.323 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.324 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.324 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841762+0000", 2024-09-16T00:10:11.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841762+0000", 2024-09-16T00:10:11.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841763+0000", 2024-09-16T00:10:11.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7e-07 2024-09-16T00:10:11.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841767+0000", 2024-09-16T00:10:11.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2799999999999997e-06 2024-09-16T00:10:11.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841775+0000", 2024-09-16T00:10:11.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5349999999999999e-06 2024-09-16T00:10:11.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841780+0000", 2024-09-16T00:10:11.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0429999999999998e-06 2024-09-16T00:10:11.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841856+0000", 2024-09-16T00:10:11.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6532999999999994e-05 2024-09-16T00:10:11.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.841874+0000", 2024-09-16T00:10:11.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8079000000000001e-05 2024-09-16T00:10:11.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:11.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.850370+0000", 2024-09-16T00:10:11.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084952169999999994 2024-09-16T00:10:11.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.850411+0000", 2024-09-16T00:10:11.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1582999999999998e-05 2024-09-16T00:10:11.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.327 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.328 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.328 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5528 3.a 3:50a2513a:::10000003108.00000001:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97)", 2024-09-16T00:10:11.328 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:03.812487+0000", 2024-09-16T00:10:11.328 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.465537146, 2024-09-16T00:10:11.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0085445009999999995, 2024-09-16T00:10:11.328 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.328 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.328 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.328 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812487+0000", 2024-09-16T00:10:11.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812487+0000", 2024-09-16T00:10:11.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812489+0000", 2024-09-16T00:10:11.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6589999999999999e-06 2024-09-16T00:10:11.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812495+0000", 2024-09-16T00:10:11.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3579999999999999e-06 2024-09-16T00:10:11.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812511+0000", 2024-09-16T00:10:11.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6011999999999998e-05 2024-09-16T00:10:11.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812521+0000", 2024-09-16T00:10:11.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3500000000000003e-06 2024-09-16T00:10:11.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812594+0000", 2024-09-16T00:10:11.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3594e-05 2024-09-16T00:10:11.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812677+0000", 2024-09-16T00:10:11.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2590999999999996e-05 2024-09-16T00:10:11.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:11.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.812781+0000", 2024-09-16T00:10:11.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000104206 2024-09-16T00:10:11.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.818292+0000", 2024-09-16T00:10:11.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055108140000000002 2024-09-16T00:10:11.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:11.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.820928+0000", 2024-09-16T00:10:11.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026360200000000002 2024-09-16T00:10:11.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:11.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.820983+0000", 2024-09-16T00:10:11.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4750000000000003e-05 2024-09-16T00:10:11.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.821032+0000", 2024-09-16T00:10:11.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9147000000000002e-05 2024-09-16T00:10:11.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.333 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.333 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.333 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:5527 3.33 e97/94 3:cf859f74:::10000003108.00000000:head v 97'245, mlcod=96'243)", 2024-09-16T00:10:11.333 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:03.813310+0000", 2024-09-16T00:10:11.333 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.4647141230000003, 2024-09-16T00:10:11.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0081025200000000002, 2024-09-16T00:10:11.334 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.334 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.334 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.334 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.813310+0000", 2024-09-16T00:10:11.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.813310+0000", 2024-09-16T00:10:11.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.813311+0000", 2024-09-16T00:10:11.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1299999999999998e-07 2024-09-16T00:10:11.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.813317+0000", 2024-09-16T00:10:11.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8590000000000001e-06 2024-09-16T00:10:11.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.813327+0000", 2024-09-16T00:10:11.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9799999999999993e-06 2024-09-16T00:10:11.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.813333+0000", 2024-09-16T00:10:11.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0780000000000001e-06 2024-09-16T00:10:11.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.813385+0000", 2024-09-16T00:10:11.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2181000000000001e-05 2024-09-16T00:10:11.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.813404+0000", 2024-09-16T00:10:11.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8559999999999998e-05 2024-09-16T00:10:11.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:11.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.821362+0000", 2024-09-16T00:10:11.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007957904 2024-09-16T00:10:11.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.821413+0000", 2024-09-16T00:10:11.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1044999999999997e-05 2024-09-16T00:10:11.337 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.338 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.338 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.338 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5582 3.20 3:05175347:::100000032f5.00000001:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97)", 2024-09-16T00:10:11.338 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:03.824971+0000", 2024-09-16T00:10:11.338 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.4530536710000002, 2024-09-16T00:10:11.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0088623420000000005, 2024-09-16T00:10:11.338 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.338 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.338 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:11.338 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.824971+0000", 2024-09-16T00:10:11.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.824971+0000", 2024-09-16T00:10:11.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.824971+0000", 2024-09-16T00:10:11.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5899999999999996e-07 2024-09-16T00:10:11.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.824975+0000", 2024-09-16T00:10:11.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7780000000000001e-06 2024-09-16T00:10:11.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.824984+0000", 2024-09-16T00:10:11.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1209999999999996e-06 2024-09-16T00:10:11.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.824992+0000", 2024-09-16T00:10:11.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.469e-06 2024-09-16T00:10:11.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.825044+0000", 2024-09-16T00:10:11.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2544999999999999e-05 2024-09-16T00:10:11.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.825162+0000", 2024-09-16T00:10:11.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011764000000000001 2024-09-16T00:10:11.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.833833+0000", 2024-09-16T00:10:11.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086711300000000009 2024-09-16T00:10:11.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.342 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.342 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5743 3.36 3:6c273be3:::1000000385e.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97)", 2024-09-16T00:10:11.342 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:03.854364+0000", 2024-09-16T00:10:11.342 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.4236605359999999, 2024-09-16T00:10:11.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099844070000000007, 2024-09-16T00:10:11.342 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.342 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.342 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:11.343 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854364+0000", 2024-09-16T00:10:11.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854364+0000", 2024-09-16T00:10:11.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854365+0000", 2024-09-16T00:10:11.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3399999999999999e-07 2024-09-16T00:10:11.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854368+0000", 2024-09-16T00:10:11.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4300000000000002e-06 2024-09-16T00:10:11.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854376+0000", 2024-09-16T00:10:11.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2770000000000005e-06 2024-09-16T00:10:11.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854384+0000", 2024-09-16T00:10:11.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7200000000000006e-06 2024-09-16T00:10:11.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854455+0000", 2024-09-16T00:10:11.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0631999999999997e-05 2024-09-16T00:10:11.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854560+0000", 2024-09-16T00:10:11.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105338 2024-09-16T00:10:11.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.864348+0000", 2024-09-16T00:10:11.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097883759999999997 2024-09-16T00:10:11.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.346 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.346 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5745 2.17 2:eeb9ee5e:::1000000387c.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97)", 2024-09-16T00:10:11.346 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:03.855859+0000", 2024-09-16T00:10:11.346 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 7.4221653239999998, 2024-09-16T00:10:11.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0084735950000000004, 2024-09-16T00:10:11.346 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.347 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.347 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:11.347 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.855859+0000", 2024-09-16T00:10:11.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.855859+0000", 2024-09-16T00:10:11.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.855860+0000", 2024-09-16T00:10:11.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6799999999999999e-07 2024-09-16T00:10:11.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.855864+0000", 2024-09-16T00:10:11.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3080000000000001e-06 2024-09-16T00:10:11.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.855875+0000", 2024-09-16T00:10:11.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1062e-05 2024-09-16T00:10:11.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.855883+0000", 2024-09-16T00:10:11.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2790000000000001e-06 2024-09-16T00:10:11.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.855934+0000", 2024-09-16T00:10:11.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1208000000000002e-05 2024-09-16T00:10:11.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.856043+0000", 2024-09-16T00:10:11.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109529 2024-09-16T00:10:11.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.864333+0000", 2024-09-16T00:10:11.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0082893410000000004 2024-09-16T00:10:11.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.350 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.350 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.350 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:6532 3.26 3:67aa8e4f:::100000031c5.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100)", 2024-09-16T00:10:11.350 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:06.849896+0000", 2024-09-16T00:10:11.350 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4281281330000004, 2024-09-16T00:10:11.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090082240000000004, 2024-09-16T00:10:11.351 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.351 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.351 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:11.351 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.849896+0000", 2024-09-16T00:10:11.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.849896+0000", 2024-09-16T00:10:11.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.849897+0000", 2024-09-16T00:10:11.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2900000000000002e-07 2024-09-16T00:10:11.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.849901+0000", 2024-09-16T00:10:11.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T00:10:11.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.849912+0000", 2024-09-16T00:10:11.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0546e-05 2024-09-16T00:10:11.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.849919+0000", 2024-09-16T00:10:11.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5179999999999998e-06 2024-09-16T00:10:11.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.854785+0000", 2024-09-16T00:10:11.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048656960000000001 2024-09-16T00:10:11.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.854914+0000", 2024-09-16T00:10:11.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012934599999999999 2024-09-16T00:10:11.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.858904+0000", 2024-09-16T00:10:11.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039901119999999997 2024-09-16T00:10:11.354 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.354 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.354 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.355 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:6601 3.35 3:af7d55b1:::10000003232.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100)", 2024-09-16T00:10:11.355 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:06.866600+0000", 2024-09-16T00:10:11.355 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4114240919999999, 2024-09-16T00:10:11.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092432980000000005, 2024-09-16T00:10:11.355 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.355 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.355 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:11.355 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866600+0000", 2024-09-16T00:10:11.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866600+0000", 2024-09-16T00:10:11.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866602+0000", 2024-09-16T00:10:11.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.649e-06 2024-09-16T00:10:11.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866607+0000", 2024-09-16T00:10:11.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7330000000000003e-06 2024-09-16T00:10:11.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866618+0000", 2024-09-16T00:10:11.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1506e-05 2024-09-16T00:10:11.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866626+0000", 2024-09-16T00:10:11.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.498e-06 2024-09-16T00:10:11.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866685+0000", 2024-09-16T00:10:11.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8950000000000003e-05 2024-09-16T00:10:11.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866809+0000", 2024-09-16T00:10:11.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012389899999999999 2024-09-16T00:10:11.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875844+0000", 2024-09-16T00:10:11.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090350629999999994 2024-09-16T00:10:11.361 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.362 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.362 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.362 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:6632 3.20 3:04677f23:::100000034fe.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100)", 2024-09-16T00:10:11.362 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:06.875758+0000", 2024-09-16T00:10:11.362 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4022659300000004, 2024-09-16T00:10:11.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0085444470000000002, 2024-09-16T00:10:11.362 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.362 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.362 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.362 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875758+0000", 2024-09-16T00:10:11.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875758+0000", 2024-09-16T00:10:11.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875759+0000", 2024-09-16T00:10:11.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8400000000000003e-07 2024-09-16T00:10:11.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875763+0000", 2024-09-16T00:10:11.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.427e-06 2024-09-16T00:10:11.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875779+0000", 2024-09-16T00:10:11.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5973000000000001e-05 2024-09-16T00:10:11.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875785+0000", 2024-09-16T00:10:11.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5660000000000003e-06 2024-09-16T00:10:11.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875840+0000", 2024-09-16T00:10:11.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4487000000000002e-05 2024-09-16T00:10:11.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875919+0000", 2024-09-16T00:10:11.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8693e-05 2024-09-16T00:10:11.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:11.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.876012+0000", 2024-09-16T00:10:11.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3966000000000001e-05 2024-09-16T00:10:11.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:11.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.877285+0000", 2024-09-16T00:10:11.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012726809999999999 2024-09-16T00:10:11.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.884204+0000", 2024-09-16T00:10:11.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069192910000000002 2024-09-16T00:10:11.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:11.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.884260+0000", 2024-09-16T00:10:11.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5995999999999997e-05 2024-09-16T00:10:11.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.884303+0000", 2024-09-16T00:10:11.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2483e-05 2024-09-16T00:10:11.367 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.367 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.367 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.367 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:6635 3.20 3:0697f031:::10000003500.00000002:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100)", 2024-09-16T00:10:11.367 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:06.875805+0000", 2024-09-16T00:10:11.368 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.4022197399999996, 2024-09-16T00:10:11.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086054329999999991, 2024-09-16T00:10:11.368 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.368 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.368 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.368 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875805+0000", 2024-09-16T00:10:11.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875805+0000", 2024-09-16T00:10:11.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875805+0000", 2024-09-16T00:10:11.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0900000000000002e-07 2024-09-16T00:10:11.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875808+0000", 2024-09-16T00:10:11.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6000000000000001e-06 2024-09-16T00:10:11.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875815+0000", 2024-09-16T00:10:11.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8689999999999998e-06 2024-09-16T00:10:11.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875820+0000", 2024-09-16T00:10:11.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2129999999999999e-06 2024-09-16T00:10:11.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.876345+0000", 2024-09-16T00:10:11.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00052494100000000001 2024-09-16T00:10:11.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.876415+0000", 2024-09-16T00:10:11.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0358999999999995e-05 2024-09-16T00:10:11.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:11.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.876521+0000", 2024-09-16T00:10:11.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105889 2024-09-16T00:10:11.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:11.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.877890+0000", 2024-09-16T00:10:11.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0013687009999999999 2024-09-16T00:10:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.884339+0000", 2024-09-16T00:10:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0064493190000000002 2024-09-16T00:10:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.884380+0000", 2024-09-16T00:10:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0533e-05 2024-09-16T00:10:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.884410+0000", 2024-09-16T00:10:11.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0499999999999999e-05 2024-09-16T00:10:11.373 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.373 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.373 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.373 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:6991 3.26 3:6659f12b:::10000002eb1.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e101)", 2024-09-16T00:10:11.373 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:07.882447+0000", 2024-09-16T00:10:11.373 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 3.3955778240000001, 2024-09-16T00:10:11.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0079011910000000001, 2024-09-16T00:10:11.373 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:11.373 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:11.373 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.374 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:11.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:11.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:07.882447+0000", 2024-09-16T00:10:11.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:11.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:07.882447+0000", 2024-09-16T00:10:11.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:11.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:11.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:07.882448+0000", 2024-09-16T00:10:11.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.096e-06 2024-09-16T00:10:11.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:11.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:07.882453+0000", 2024-09-16T00:10:11.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0860000000000001e-06 2024-09-16T00:10:11.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:11.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:07.882466+0000", 2024-09-16T00:10:11.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3669000000000001e-05 2024-09-16T00:10:11.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:07.882474+0000", 2024-09-16T00:10:11.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.729e-06 2024-09-16T00:10:11.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:11.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:07.882537+0000", 2024-09-16T00:10:11.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2617e-05 2024-09-16T00:10:11.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:11.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:07.882704+0000", 2024-09-16T00:10:11.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016757499999999999 2024-09-16T00:10:11.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:11.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:07.882797+0000", 2024-09-16T00:10:11.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2202999999999998e-05 2024-09-16T00:10:11.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:11.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:07.886981+0000", 2024-09-16T00:10:11.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041846690000000002 2024-09-16T00:10:11.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:07.890258+0000", 2024-09-16T00:10:11.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00327663 2024-09-16T00:10:11.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:11.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:07.890307+0000", 2024-09-16T00:10:11.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9336000000000003e-05 2024-09-16T00:10:11.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:11.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:11.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:11.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:07.890348+0000", 2024-09-16T00:10:11.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0581000000000003e-05 2024-09-16T00:10:11.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.378 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:11.379 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:11.379 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:11.379 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T00:10:11.380 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:11.380+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:11.381 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:10:11.481 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:11.481+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:11.485 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.484+0000 7fcb17445640 1 Processor -- start 2024-09-16T00:10:11.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.485+0000 7fcb17445640 1 -- start start 2024-09-16T00:10:11.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.486+0000 7fcb17445640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb1015b280 0x7fcb10166190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:11.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.486+0000 7fcb17445640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb100cc4b0 con 0x7fcb10068060 2024-09-16T00:10:11.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.486+0000 7fcb17445640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb100cc790 con 0x7fcb10154c20 2024-09-16T00:10:11.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.486+0000 7fcb17445640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcb100c5830 con 0x7fcb1015b280 2024-09-16T00:10:11.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.486+0000 7fcb149e0640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fcb10068060 0x7fcb100ccc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45730/0 (socket says 172.21.15.139:45730) 2024-09-16T00:10:11.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.486+0000 7fcb149e0640 1 -- 172.21.15.139:0/1342613731 learned_addr learned my addr 172.21.15.139:0/1342613731 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:11.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.486+0000 7fcb159e2640 1 --2- 172.21.15.139:0/1342613731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb1015b280 0x7fcb10166190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:11.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.486+0000 7fcb07fff640 1 -- 172.21.15.139:0/1342613731 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1959607580 0 0) 0x7fcb100cc4b0 con 0x7fcb10068060 2024-09-16T00:10:11.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.486+0000 7fcb07fff640 1 -- 172.21.15.139:0/1342613731 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcafc003520 con 0x7fcb10068060 2024-09-16T00:10:11.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.486+0000 7fcb07fff640 1 -- 172.21.15.139:0/1342613731 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3248364889 0 0) 0x7fcb100cc790 con 0x7fcb10154c20 2024-09-16T00:10:11.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.486+0000 7fcb07fff640 1 -- 172.21.15.139:0/1342613731 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb100cc4b0 con 0x7fcb10154c20 2024-09-16T00:10:11.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.486+0000 7fcb159e2640 1 -- 172.21.15.139:0/1342613731 >> v1:172.21.15.164:6789/0 conn(0x7fcb10154c20 legacy=0x7fcb100c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.486+0000 7fcb159e2640 1 -- 172.21.15.139:0/1342613731 >> v1:172.21.15.139:6789/0 conn(0x7fcb10068060 legacy=0x7fcb100ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.486+0000 7fcb159e2640 1 -- 172.21.15.139:0/1342613731 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1015ddf0 con 0x7fcb1015b280 2024-09-16T00:10:11.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.487+0000 7fcb159e2640 1 --2- 172.21.15.139:0/1342613731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb1015b280 0x7fcb10166190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009dc0 tx=0x7fcb0802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38277b08ef5fe880 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:11.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.487+0000 7fcb07fff640 1 -- 172.21.15.139:0/1342613731 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08007740 con 0x7fcb1015b280 2024-09-16T00:10:11.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.487+0000 7fcb07fff640 1 -- 172.21.15.139:0/1342613731 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb080078e0 con 0x7fcb1015b280 2024-09-16T00:10:11.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.487+0000 7fcb07fff640 1 -- 172.21.15.139:0/1342613731 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08007cb0 con 0x7fcb1015b280 2024-09-16T00:10:11.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.488+0000 7fcb17445640 1 -- 172.21.15.139:0/1342613731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb1015b280 msgr2=0x7fcb10166190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.488+0000 7fcb17445640 1 --2- 172.21.15.139:0/1342613731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb1015b280 0x7fcb10166190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009dc0 tx=0x7fcb0802a6d0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.488+0000 7fcb17445640 1 -- 172.21.15.139:0/1342613731 shutdown_connections 2024-09-16T00:10:11.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.488+0000 7fcb17445640 1 --2- 172.21.15.139:0/1342613731 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb1015b280 0x7fcb10166190 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.488+0000 7fcb17445640 1 -- 172.21.15.139:0/1342613731 >> 172.21.15.139:0/1342613731 conn(0x7fcb1006ca40 msgr2=0x7fcb10067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:11.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.488+0000 7fcb17445640 1 -- 172.21.15.139:0/1342613731 shutdown_connections 2024-09-16T00:10:11.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.488+0000 7fcb17445640 1 -- 172.21.15.139:0/1342613731 wait complete. 2024-09-16T00:10:11.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.489+0000 7fcb17445640 1 Processor -- start 2024-09-16T00:10:11.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.489+0000 7fcb17445640 1 -- start start 2024-09-16T00:10:11.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.490+0000 7fcb17445640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10068060 0x7fcb1015d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:11.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.490+0000 7fcb17445640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb10154c20 0x7fcb101502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:11.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.490+0000 7fcb17445640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb1015b280 0x7fcb10150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:11.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.490+0000 7fcb17445640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1015e3b0 con 0x7fcb10068060 2024-09-16T00:10:11.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.490+0000 7fcb17445640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcb100cc260 con 0x7fcb10154c20 2024-09-16T00:10:11.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.490+0000 7fcb17445640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcb1015e690 con 0x7fcb1015b280 2024-09-16T00:10:11.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.490+0000 7fcb151e1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10068060 0x7fcb1015d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:11.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.490+0000 7fcb149e0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb10154c20 0x7fcb101502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:11.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.490+0000 7fcb159e2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb1015b280 0x7fcb10150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:11.493 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:11.493 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:11.493 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:11.493 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:11.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.490+0000 7fcb149e0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb10154c20 0x7fcb101502e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:42312/0 (socket says 172.21.15.139:42312) 2024-09-16T00:10:11.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.490+0000 7fcb151e1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10068060 0x7fcb1015d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49208/0 (socket says 172.21.15.139:49208) 2024-09-16T00:10:11.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.490+0000 7fcb149e0640 1 -- 172.21.15.139:0/2758485754 learned_addr learned my addr 172.21.15.139:0/2758485754 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:11.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.490+0000 7fcb159e2640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb1015b280 0x7fcb10150820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:56208/0 (socket says 172.21.15.139:56208) 2024-09-16T00:10:11.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.491+0000 7fcb159e2640 1 -- 172.21.15.139:0/2758485754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb10154c20 msgr2=0x7fcb101502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.491+0000 7fcb159e2640 1 --2- 172.21.15.139:0/2758485754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb10154c20 0x7fcb101502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.491+0000 7fcb159e2640 1 -- 172.21.15.139:0/2758485754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10068060 msgr2=0x7fcb1015d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:11.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.491+0000 7fcb159e2640 1 --2- 172.21.15.139:0/2758485754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10068060 0x7fcb1015d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:11.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.491+0000 7fcb159e2640 1 -- 172.21.15.139:0/2758485754 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb101f3e50 con 0x7fcb1015b280 2024-09-16T00:10:11.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.491+0000 7fcb151e1640 1 --2- 172.21.15.139:0/2758485754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10068060 0x7fcb1015d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:11.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.491+0000 7fcb159e2640 1 --2- 172.21.15.139:0/2758485754 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb1015b280 0x7fcb10150820 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcb08031000 tx=0x7fcb08004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:11.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.491+0000 7fcb05ffb640 1 -- 172.21.15.139:0/2758485754 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0803f070 con 0x7fcb1015b280 2024-09-16T00:10:11.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.491+0000 7fcb05ffb640 1 -- 172.21.15.139:0/2758485754 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb08031bc0 con 0x7fcb1015b280 2024-09-16T00:10:11.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.491+0000 7fcb17445640 1 -- 172.21.15.139:0/2758485754 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb101f09c0 con 0x7fcb1015b280 2024-09-16T00:10:11.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.491+0000 7fcb05ffb640 1 -- 172.21.15.139:0/2758485754 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08009450 con 0x7fcb1015b280 2024-09-16T00:10:11.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.491+0000 7fcb17445640 1 -- 172.21.15.139:0/2758485754 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb101f0f80 con 0x7fcb1015b280 2024-09-16T00:10:11.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.492+0000 7fcb17445640 1 -- 172.21.15.139:0/2758485754 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb100686b0 con 0x7fcb1015b280 2024-09-16T00:10:11.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.493+0000 7fcb05ffb640 1 -- 172.21.15.139:0/2758485754 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcb08007790 con 0x7fcb1015b280 2024-09-16T00:10:11.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.493+0000 7fcb05ffb640 1 --2- 172.21.15.139:0/2758485754 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcaf406c2e0 0x7fcaf406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:11.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.494+0000 7fcb05ffb640 1 -- 172.21.15.139:0/2758485754 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(103..103 src has 1..103) ==== 5973+0+0 (secure 0 0 0) 0x7fcb080aca60 con 0x7fcb1015b280 2024-09-16T00:10:11.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.494+0000 7fcb151e1640 1 --2- 172.21.15.139:0/2758485754 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcaf406c2e0 0x7fcaf406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:11.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.494+0000 7fcb151e1640 1 --2- 172.21.15.139:0/2758485754 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcaf406c2e0 0x7fcaf406e7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c007fd0 tx=0x7fcb0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:11.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.498+0000 7fcb05ffb640 1 -- 172.21.15.139:0/2758485754 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcb08049060 con 0x7fcb1015b280 2024-09-16T00:10:11.502 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:10:11.582 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:11.582+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:11.613 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:11.614 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:11.614 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:11.614 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:11.614 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:11.623 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:10:11.683 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:11.683+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:11.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:11.701+0000 7fcb17445640 1 -- 172.21.15.139:0/2758485754 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fcb100cd720 con 0x7fcb1015b280 2024-09-16T00:10:11.773 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:11.773 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:11.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:11.774 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:11.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.774 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:219 2.1f 2:f83d65d8:::200.00000004:head [write 863658~2915703 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e65)", 2024-09-16T00:10:11.774 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:10:11.774 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.900318255999998, 2024-09-16T00:10:11.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013772627000000001, 2024-09-16T00:10:11.774 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.774 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.774 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.774 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:10:11.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.775 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.775 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:10:11.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.775 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.775 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870941+0000", 2024-09-16T00:10:11.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4e-07 2024-09-16T00:10:11.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.775 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.776 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877134+0000", 2024-09-16T00:10:11.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061936769999999999 2024-09-16T00:10:11.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.776 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.776 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877153+0000", 2024-09-16T00:10:11.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8408000000000001e-05 2024-09-16T00:10:11.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.776 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.776 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877161+0000", 2024-09-16T00:10:11.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2709999999999992e-06 2024-09-16T00:10:11.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877225+0000", 2024-09-16T00:10:11.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4190999999999996e-05 2024-09-16T00:10:11.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877289+0000", 2024-09-16T00:10:11.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4058000000000002e-05 2024-09-16T00:10:11.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:11.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877376+0000", 2024-09-16T00:10:11.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7227999999999999e-05 2024-09-16T00:10:11.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.882925+0000", 2024-09-16T00:10:11.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055481050000000002 2024-09-16T00:10:11.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884635+0000", 2024-09-16T00:10:11.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017102759999999999 2024-09-16T00:10:11.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884674+0000", 2024-09-16T00:10:11.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8914e-05 2024-09-16T00:10:11.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884712+0000", 2024-09-16T00:10:11.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658999999999997e-05 2024-09-16T00:10:11.779 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.779 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.779 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.779 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10767 3.1b 3:dab1607f:::100000030fd.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:11.779 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.379273+0000", 2024-09-16T00:10:11.780 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.391985523999999, 2024-09-16T00:10:11.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012051947, 2024-09-16T00:10:11.780 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.780 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.780 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.780 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.780 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.780 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.780 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10767 2024-09-16T00:10:11.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.780 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.379273+0000", 2024-09-16T00:10:11.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.379273+0000", 2024-09-16T00:10:11.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.379275+0000", 2024-09-16T00:10:11.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6230000000000001e-06 2024-09-16T00:10:11.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.782 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.782 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.782 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381583+0000", 2024-09-16T00:10:11.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023079939999999998 2024-09-16T00:10:11.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.782 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.782 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.782 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381617+0000", 2024-09-16T00:10:11.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4227999999999998e-05 2024-09-16T00:10:11.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.782 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381633+0000", 2024-09-16T00:10:11.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5642e-05 2024-09-16T00:10:11.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381739+0000", 2024-09-16T00:10:11.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010638500000000001 2024-09-16T00:10:11.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.381850+0000", 2024-09-16T00:10:11.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110164 2024-09-16T00:10:11.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:11.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.382033+0000", 2024-09-16T00:10:11.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018355699999999999 2024-09-16T00:10:11.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.388313+0000", 2024-09-16T00:10:11.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006279588 2024-09-16T00:10:11.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.391197+0000", 2024-09-16T00:10:11.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0028841130000000002 2024-09-16T00:10:11.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.786 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.391263+0000", 2024-09-16T00:10:11.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6105000000000005e-05 2024-09-16T00:10:11.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.786 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.786 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.391324+0000", 2024-09-16T00:10:11.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1548000000000001e-05 2024-09-16T00:10:11.786 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.786 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.786 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.786 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10768 3.2b 3:d738455c:::100000030db.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:11.786 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:10:11.787 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.377133082999997, 2024-09-16T00:10:11.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012411195999999999, 2024-09-16T00:10:11.787 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.787 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.787 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.787 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.787 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.787 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.787 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10768 2024-09-16T00:10:11.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.788 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.788 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.788 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.788 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:10:11.788 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.788 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.788 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.788 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:10:11.788 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.789 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.789 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394129+0000", 2024-09-16T00:10:11.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4100000000000001e-06 2024-09-16T00:10:11.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.789 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.789 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395723+0000", 2024-09-16T00:10:11.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015938409999999999 2024-09-16T00:10:11.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395745+0000", 2024-09-16T00:10:11.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1611e-05 2024-09-16T00:10:11.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395782+0000", 2024-09-16T00:10:11.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7292000000000001e-05 2024-09-16T00:10:11.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395856+0000", 2024-09-16T00:10:11.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3675000000000002e-05 2024-09-16T00:10:11.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395924+0000", 2024-09-16T00:10:11.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8646999999999995e-05 2024-09-16T00:10:11.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:11.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.396069+0000", 2024-09-16T00:10:11.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014455199999999999 2024-09-16T00:10:11.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.403755+0000", 2024-09-16T00:10:11.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0076863160000000003 2024-09-16T00:10:11.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406407+0000", 2024-09-16T00:10:11.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026516220000000002 2024-09-16T00:10:11.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406477+0000", 2024-09-16T00:10:11.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0204999999999996e-05 2024-09-16T00:10:11.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406536+0000", 2024-09-16T00:10:11.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9024999999999998e-05 2024-09-16T00:10:11.793 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.793 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.794 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.794 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10941 3.1 3:8246fad2:::10000003173.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:11.794 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:10:11.794 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.315379745999998, 2024-09-16T00:10:11.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01286782, 2024-09-16T00:10:11.794 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.794 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.794 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.795 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.795 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.795 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.795 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10941 2024-09-16T00:10:11.795 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.795 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.795 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.795 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.795 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:10:11.795 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.795 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.795 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:10:11.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455881+0000", 2024-09-16T00:10:11.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7269999999999999e-06 2024-09-16T00:10:11.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457462+0000", 2024-09-16T00:10:11.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015811690000000001 2024-09-16T00:10:11.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457487+0000", 2024-09-16T00:10:11.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4607e-05 2024-09-16T00:10:11.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457497+0000", 2024-09-16T00:10:11.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9939999999999995e-06 2024-09-16T00:10:11.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457571+0000", 2024-09-16T00:10:11.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3867000000000003e-05 2024-09-16T00:10:11.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457689+0000", 2024-09-16T00:10:11.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000118814 2024-09-16T00:10:11.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:11.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457804+0000", 2024-09-16T00:10:11.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114702 2024-09-16T00:10:11.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.463422+0000", 2024-09-16T00:10:11.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0056179239999999998 2024-09-16T00:10:11.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468641+0000", 2024-09-16T00:10:11.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052192109999999996 2024-09-16T00:10:11.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468703+0000", 2024-09-16T00:10:11.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1268999999999999e-05 2024-09-16T00:10:11.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.800 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468746+0000", 2024-09-16T00:10:11.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:10:11.800 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.800 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.800 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.800 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11742 3.1b 3:d847de26:::100000032cd.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:11.800 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:10:11.800 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 28.714888161000001, 2024-09-16T00:10:11.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012791903, 2024-09-16T00:10:11.801 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.801 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.801 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.801 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.801 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.801 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.801 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11742 2024-09-16T00:10:11.801 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.801 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.801 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.801 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.801 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:10:11.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:10:11.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056373+0000", 2024-09-16T00:10:11.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9799999999999998e-06 2024-09-16T00:10:11.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058604+0000", 2024-09-16T00:10:11.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022316020000000001 2024-09-16T00:10:11.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058630+0000", 2024-09-16T00:10:11.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5425000000000001e-05 2024-09-16T00:10:11.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058643+0000", 2024-09-16T00:10:11.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3361e-05 2024-09-16T00:10:11.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058731+0000", 2024-09-16T00:10:11.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7270999999999998e-05 2024-09-16T00:10:11.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058894+0000", 2024-09-16T00:10:11.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016376800000000001 2024-09-16T00:10:11.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:11.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.059050+0000", 2024-09-16T00:10:11.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015603700000000001 2024-09-16T00:10:11.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.066364+0000", 2024-09-16T00:10:11.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073138639999999998 2024-09-16T00:10:11.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069025+0000", 2024-09-16T00:10:11.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026607660000000002 2024-09-16T00:10:11.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069083+0000", 2024-09-16T00:10:11.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7680999999999999e-05 2024-09-16T00:10:11.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069162+0000", 2024-09-16T00:10:11.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9147999999999995e-05 2024-09-16T00:10:11.806 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.806 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.806 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.806 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11934 3.1b 3:d94fa8da:::10000003190.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:11.806 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:10:11.806 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 28.082844886, 2024-09-16T00:10:11.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.017110325999999999, 2024-09-16T00:10:11.807 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.807 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.807 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.807 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.807 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.807 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.807 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11934 2024-09-16T00:10:11.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.807 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.807 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:10:11.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:10:11.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688416+0000", 2024-09-16T00:10:11.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.875e-06 2024-09-16T00:10:11.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689633+0000", 2024-09-16T00:10:11.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001216997 2024-09-16T00:10:11.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689655+0000", 2024-09-16T00:10:11.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2285000000000001e-05 2024-09-16T00:10:11.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689665+0000", 2024-09-16T00:10:11.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7089999999999997e-06 2024-09-16T00:10:11.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689739+0000", 2024-09-16T00:10:11.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3973999999999993e-05 2024-09-16T00:10:11.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689898+0000", 2024-09-16T00:10:11.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015941100000000001 2024-09-16T00:10:11.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:11.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.690062+0000", 2024-09-16T00:10:11.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000163633 2024-09-16T00:10:11.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.695224+0000", 2024-09-16T00:10:11.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051620210000000001 2024-09-16T00:10:11.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705434+0000", 2024-09-16T00:10:11.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010209701 2024-09-16T00:10:11.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705488+0000", 2024-09-16T00:10:11.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3974000000000002e-05 2024-09-16T00:10:11.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705523+0000", 2024-09-16T00:10:11.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5746e-05 2024-09-16T00:10:11.812 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.812 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.812 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.812 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11941 3.12 3:4a0e6682:::1000000334b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:11.812 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:10:11.812 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 28.047827373000001, 2024-09-16T00:10:11.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01458448, 2024-09-16T00:10:11.812 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.813 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.813 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.813 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.813 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.813 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.813 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11941 2024-09-16T00:10:11.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.813 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.813 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.813 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:10:11.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:10:11.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723433+0000", 2024-09-16T00:10:11.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3410000000000001e-06 2024-09-16T00:10:11.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724966+0000", 2024-09-16T00:10:11.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001533242 2024-09-16T00:10:11.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724983+0000", 2024-09-16T00:10:11.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7156999999999999e-05 2024-09-16T00:10:11.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724995+0000", 2024-09-16T00:10:11.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2089999999999999e-05 2024-09-16T00:10:11.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725062+0000", 2024-09-16T00:10:11.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6026999999999997e-05 2024-09-16T00:10:11.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725159+0000", 2024-09-16T00:10:11.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7374000000000007e-05 2024-09-16T00:10:11.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:11.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725289+0000", 2024-09-16T00:10:11.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012988 2024-09-16T00:10:11.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.731944+0000", 2024-09-16T00:10:11.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066547439999999998 2024-09-16T00:10:11.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.737956+0000", 2024-09-16T00:10:11.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0060121630000000001 2024-09-16T00:10:11.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.737985+0000", 2024-09-16T00:10:11.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9352999999999998e-05 2024-09-16T00:10:11.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.738015+0000", 2024-09-16T00:10:11.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0108999999999999e-05 2024-09-16T00:10:11.823 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.823 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.823 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.823 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11954 3.6 3:625e0e90:::100000032ef.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:11.823 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:10:11.823 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 28.003375806000001, 2024-09-16T00:10:11.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012484808999999999, 2024-09-16T00:10:11.824 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.824 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.824 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.824 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.824 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.824 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.824 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11954 2024-09-16T00:10:11.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.824 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.824 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:10:11.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:10:11.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767886+0000", 2024-09-16T00:10:11.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6650000000000001e-06 2024-09-16T00:10:11.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770082+0000", 2024-09-16T00:10:11.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021964979999999999 2024-09-16T00:10:11.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770104+0000", 2024-09-16T00:10:11.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1671999999999999e-05 2024-09-16T00:10:11.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770114+0000", 2024-09-16T00:10:11.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0336000000000001e-05 2024-09-16T00:10:11.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770197+0000", 2024-09-16T00:10:11.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2997000000000005e-05 2024-09-16T00:10:11.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770324+0000", 2024-09-16T00:10:11.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012680100000000001 2024-09-16T00:10:11.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:11.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770497+0000", 2024-09-16T00:10:11.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000172776 2024-09-16T00:10:11.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.776330+0000", 2024-09-16T00:10:11.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058330939999999996 2024-09-16T00:10:11.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780252+0000", 2024-09-16T00:10:11.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039215689999999997 2024-09-16T00:10:11.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780317+0000", 2024-09-16T00:10:11.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4948000000000002e-05 2024-09-16T00:10:11.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780367+0000", 2024-09-16T00:10:11.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0453e-05 2024-09-16T00:10:11.831 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.831 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.831 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.832 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12198 3.12 3:4a0e6682:::1000000334b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:11.832 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.683089+0000", 2024-09-16T00:10:11.832 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.088168756000002, 2024-09-16T00:10:11.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011990244000000001, 2024-09-16T00:10:11.832 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.832 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.832 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.832 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.832 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.832 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.832 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12198 2024-09-16T00:10:11.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.832 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.683089+0000", 2024-09-16T00:10:11.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.683089+0000", 2024-09-16T00:10:11.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.683092+0000", 2024-09-16T00:10:11.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.931e-06 2024-09-16T00:10:11.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684715+0000", 2024-09-16T00:10:11.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00162269 2024-09-16T00:10:11.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684734+0000", 2024-09-16T00:10:11.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9545999999999999e-05 2024-09-16T00:10:11.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684745+0000", 2024-09-16T00:10:11.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0131e-05 2024-09-16T00:10:11.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684814+0000", 2024-09-16T00:10:11.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9740000000000007e-05 2024-09-16T00:10:11.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684871+0000", 2024-09-16T00:10:11.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6215999999999998e-05 2024-09-16T00:10:11.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:11.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.684980+0000", 2024-09-16T00:10:11.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109713 2024-09-16T00:10:11.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.690215+0000", 2024-09-16T00:10:11.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052349049999999998 2024-09-16T00:10:11.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.694970+0000", 2024-09-16T00:10:11.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047552649999999998 2024-09-16T00:10:11.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.695030+0000", 2024-09-16T00:10:11.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9086999999999999e-05 2024-09-16T00:10:11.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.695080+0000", 2024-09-16T00:10:11.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0019999999999999e-05 2024-09-16T00:10:11.837 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.837 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.837 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.838 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13263 3.1b 3:d9746e52:::1000000361c.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:11.838 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.785956+0000", 2024-09-16T00:10:11.838 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.985301816, 2024-09-16T00:10:11.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011930029, 2024-09-16T00:10:11.838 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.838 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.838 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.838 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.838 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.838 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.838 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13263 2024-09-16T00:10:11.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.838 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.785956+0000", 2024-09-16T00:10:11.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.785956+0000", 2024-09-16T00:10:11.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.785959+0000", 2024-09-16T00:10:11.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6089999999999999e-06 2024-09-16T00:10:11.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787198+0000", 2024-09-16T00:10:11.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012395889999999999 2024-09-16T00:10:11.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787225+0000", 2024-09-16T00:10:11.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6101999999999998e-05 2024-09-16T00:10:11.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787239+0000", 2024-09-16T00:10:11.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4384e-05 2024-09-16T00:10:11.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787329+0000", 2024-09-16T00:10:11.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0555000000000003e-05 2024-09-16T00:10:11.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787503+0000", 2024-09-16T00:10:11.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017311599999999999 2024-09-16T00:10:11.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:11.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.787676+0000", 2024-09-16T00:10:11.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017352900000000001 2024-09-16T00:10:11.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.793377+0000", 2024-09-16T00:10:11.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0057008409999999999 2024-09-16T00:10:11.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.797765+0000", 2024-09-16T00:10:11.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004388042 2024-09-16T00:10:11.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.797831+0000", 2024-09-16T00:10:11.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.614e-05 2024-09-16T00:10:11.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.797886+0000", 2024-09-16T00:10:11.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5121999999999998e-05 2024-09-16T00:10:11.843 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.843 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.843 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.843 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13618 3.1 3:80dd39f3:::1000000336c.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:11.843 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.108808+0000", 2024-09-16T00:10:11.844 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.662449976000001, 2024-09-16T00:10:11.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013244522, 2024-09-16T00:10:11.844 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.844 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.844 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.844 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.844 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.844 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.844 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13618 2024-09-16T00:10:11.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.844 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.108808+0000", 2024-09-16T00:10:11.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.108808+0000", 2024-09-16T00:10:11.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.108812+0000", 2024-09-16T00:10:11.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4640000000000002e-06 2024-09-16T00:10:11.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.110957+0000", 2024-09-16T00:10:11.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002145185 2024-09-16T00:10:11.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.110974+0000", 2024-09-16T00:10:11.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7108999999999998e-05 2024-09-16T00:10:11.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.846 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.110982+0000", 2024-09-16T00:10:11.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6330000000000002e-06 2024-09-16T00:10:11.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.111130+0000", 2024-09-16T00:10:11.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014738899999999999 2024-09-16T00:10:11.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.111299+0000", 2024-09-16T00:10:11.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016908400000000001 2024-09-16T00:10:11.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:11.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.111470+0000", 2024-09-16T00:10:11.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170667 2024-09-16T00:10:11.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.117141+0000", 2024-09-16T00:10:11.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005670993 2024-09-16T00:10:11.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.121945+0000", 2024-09-16T00:10:11.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0048042370000000003 2024-09-16T00:10:11.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.121998+0000", 2024-09-16T00:10:11.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2802e-05 2024-09-16T00:10:11.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.122053+0000", 2024-09-16T00:10:11.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4959e-05 2024-09-16T00:10:11.849 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.849 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.849 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.849 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13628 3.12 3:491d0769:::100000031da.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:11.849 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.134341+0000", 2024-09-16T00:10:11.850 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.636916675999998, 2024-09-16T00:10:11.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011817984, 2024-09-16T00:10:11.850 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.850 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.850 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.850 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.850 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.850 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.850 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13628 2024-09-16T00:10:11.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.850 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.134341+0000", 2024-09-16T00:10:11.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.134341+0000", 2024-09-16T00:10:11.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.134344+0000", 2024-09-16T00:10:11.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.785e-06 2024-09-16T00:10:11.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.135856+0000", 2024-09-16T00:10:11.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015116610000000001 2024-09-16T00:10:11.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.135878+0000", 2024-09-16T00:10:11.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2603999999999999e-05 2024-09-16T00:10:11.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.135890+0000", 2024-09-16T00:10:11.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1372000000000001e-05 2024-09-16T00:10:11.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.135958+0000", 2024-09-16T00:10:11.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7792000000000004e-05 2024-09-16T00:10:11.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.136023+0000", 2024-09-16T00:10:11.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5239000000000003e-05 2024-09-16T00:10:11.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:11.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.136147+0000", 2024-09-16T00:10:11.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012371600000000001 2024-09-16T00:10:11.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.141510+0000", 2024-09-16T00:10:11.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053634800000000003 2024-09-16T00:10:11.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.146037+0000", 2024-09-16T00:10:11.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045272130000000004 2024-09-16T00:10:11.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.146105+0000", 2024-09-16T00:10:11.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8038000000000001e-05 2024-09-16T00:10:11.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.146159+0000", 2024-09-16T00:10:11.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4083999999999999e-05 2024-09-16T00:10:11.856 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.856 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.856 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.856 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13687 3.32 3:4e600466:::1000000377a.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:11.856 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.340141+0000", 2024-09-16T00:10:11.856 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.431117053000001, 2024-09-16T00:10:11.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012781707999999999, 2024-09-16T00:10:11.856 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.856 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.857 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.857 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.857 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.857 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.857 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13687 2024-09-16T00:10:11.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.857 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.340141+0000", 2024-09-16T00:10:11.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.340141+0000", 2024-09-16T00:10:11.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.340143+0000", 2024-09-16T00:10:11.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9520000000000001e-06 2024-09-16T00:10:11.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343159+0000", 2024-09-16T00:10:11.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030156290000000001 2024-09-16T00:10:11.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343178+0000", 2024-09-16T00:10:11.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9219999999999999e-05 2024-09-16T00:10:11.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343212+0000", 2024-09-16T00:10:11.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3745999999999998e-05 2024-09-16T00:10:11.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343293+0000", 2024-09-16T00:10:11.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1074000000000003e-05 2024-09-16T00:10:11.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343430+0000", 2024-09-16T00:10:11.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013718700000000001 2024-09-16T00:10:11.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:11.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343589+0000", 2024-09-16T00:10:11.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000158806 2024-09-16T00:10:11.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.352468+0000", 2024-09-16T00:10:11.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0088798209999999995 2024-09-16T00:10:11.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.352835+0000", 2024-09-16T00:10:11.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00036653400000000001 2024-09-16T00:10:11.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.352884+0000", 2024-09-16T00:10:11.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8881000000000002e-05 2024-09-16T00:10:11.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.352923+0000", 2024-09-16T00:10:11.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8858e-05 2024-09-16T00:10:11.863 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.863 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.863 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.863 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13722 3.6 3:639728a8:::100000030cc.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:11.863 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.442783+0000", 2024-09-16T00:10:11.863 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.328475210000001, 2024-09-16T00:10:11.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013776286, 2024-09-16T00:10:11.863 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.864 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.864 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.864 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.864 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.864 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.864 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13722 2024-09-16T00:10:11.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.864 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.864 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.864 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.442783+0000", 2024-09-16T00:10:11.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.442783+0000", 2024-09-16T00:10:11.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.442785+0000", 2024-09-16T00:10:11.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9369999999999998e-06 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444337+0000", 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015527169999999999 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444362+0000", 2024-09-16T00:10:11.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4513999999999999e-05 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444375+0000", 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3052e-05 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444451+0000", 2024-09-16T00:10:11.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5814000000000003e-05 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444566+0000", 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011508199999999999 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444727+0000", 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000160743 2024-09-16T00:10:11.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.451504+0000", 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006777773 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.456446+0000", 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004941834 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.456510+0000", 2024-09-16T00:10:11.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3880999999999994e-05 2024-09-16T00:10:11.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.456559+0000", 2024-09-16T00:10:11.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8939e-05 2024-09-16T00:10:11.870 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.870 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.870 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.870 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13773 3.10 3:0a21c9f4:::100000034e7.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:11.870 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.578694+0000", 2024-09-16T00:10:11.870 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.192564275999999, 2024-09-16T00:10:11.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012227249000000001, 2024-09-16T00:10:11.871 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.871 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.871 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.871 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.871 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.871 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.871 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13773 2024-09-16T00:10:11.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.871 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.578694+0000", 2024-09-16T00:10:11.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.872 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.578694+0000", 2024-09-16T00:10:11.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.872 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.578698+0000", 2024-09-16T00:10:11.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8700000000000002e-06 2024-09-16T00:10:11.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.872 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.581979+0000", 2024-09-16T00:10:11.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003281455 2024-09-16T00:10:11.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.581994+0000", 2024-09-16T00:10:11.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4861000000000001e-05 2024-09-16T00:10:11.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.582003+0000", 2024-09-16T00:10:11.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0399999999999998e-06 2024-09-16T00:10:11.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.582070+0000", 2024-09-16T00:10:11.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7189999999999999e-05 2024-09-16T00:10:11.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.582158+0000", 2024-09-16T00:10:11.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7843000000000006e-05 2024-09-16T00:10:11.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.590796+0000", 2024-09-16T00:10:11.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0086376820000000007 2024-09-16T00:10:11.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.590842+0000", 2024-09-16T00:10:11.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6779000000000001e-05 2024-09-16T00:10:11.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.590921+0000", 2024-09-16T00:10:11.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8529000000000007e-05 2024-09-16T00:10:11.875 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.875 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.875 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.875 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13876 3.17 3:e847251e:::10000003522.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:11.875 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.796002+0000", 2024-09-16T00:10:11.875 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 21.975255702999998, 2024-09-16T00:10:11.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013015385000000001, 2024-09-16T00:10:11.876 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.876 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.876 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.876 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.876 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.876 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.876 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13876 2024-09-16T00:10:11.876 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.876 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.876 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.876 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.796002+0000", 2024-09-16T00:10:11.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.877 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.877 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.796002+0000", 2024-09-16T00:10:11.877 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.877 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.877 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.796004+0000", 2024-09-16T00:10:11.877 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1090000000000001e-06 2024-09-16T00:10:11.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.877 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.877 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797545+0000", 2024-09-16T00:10:11.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015402219999999999 2024-09-16T00:10:11.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797570+0000", 2024-09-16T00:10:11.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4905999999999998e-05 2024-09-16T00:10:11.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797580+0000", 2024-09-16T00:10:11.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0886e-05 2024-09-16T00:10:11.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797652+0000", 2024-09-16T00:10:11.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1050000000000006e-05 2024-09-16T00:10:11.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797814+0000", 2024-09-16T00:10:11.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000162578 2024-09-16T00:10:11.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:11.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797965+0000", 2024-09-16T00:10:11.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015092099999999999 2024-09-16T00:10:11.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.804256+0000", 2024-09-16T00:10:11.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0062911349999999998 2024-09-16T00:10:11.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.808943+0000", 2024-09-16T00:10:11.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0046868200000000004 2024-09-16T00:10:11.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.808985+0000", 2024-09-16T00:10:11.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1659000000000002e-05 2024-09-16T00:10:11.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.809018+0000", 2024-09-16T00:10:11.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3099000000000003e-05 2024-09-16T00:10:11.881 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.881 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.881 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.881 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13963 3.2b 3:d4ebc4d4:::100000034a5.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:11.881 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:50.005950+0000", 2024-09-16T00:10:11.881 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 21.765308469000001, 2024-09-16T00:10:11.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.062103750999999999, 2024-09-16T00:10:11.881 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.881 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.881 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.882 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.882 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.882 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.882 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13963 2024-09-16T00:10:11.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.882 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.005950+0000", 2024-09-16T00:10:11.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.005950+0000", 2024-09-16T00:10:11.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.005952+0000", 2024-09-16T00:10:11.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.255e-06 2024-09-16T00:10:11.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008027+0000", 2024-09-16T00:10:11.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002075019 2024-09-16T00:10:11.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.884 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008051+0000", 2024-09-16T00:10:11.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3906000000000001e-05 2024-09-16T00:10:11.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.884 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008063+0000", 2024-09-16T00:10:11.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2564e-05 2024-09-16T00:10:11.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.884 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008137+0000", 2024-09-16T00:10:11.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3946e-05 2024-09-16T00:10:11.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008234+0000", 2024-09-16T00:10:11.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6497999999999997e-05 2024-09-16T00:10:11.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:11.885 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008377+0000", 2024-09-16T00:10:11.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014313300000000001 2024-09-16T00:10:11.885 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.885 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.067062+0000", 2024-09-16T00:10:11.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058685043999999999 2024-09-16T00:10:11.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.067963+0000", 2024-09-16T00:10:11.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000901413 2024-09-16T00:10:11.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.068013+0000", 2024-09-16T00:10:11.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9997000000000003e-05 2024-09-16T00:10:11.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.068053+0000", 2024-09-16T00:10:11.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9975999999999998e-05 2024-09-16T00:10:11.887 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.887 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.887 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.887 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2683 2.8 e89/19 2:17051619:::200.0000000f:head v 89'470, mlcod=89'468)", 2024-09-16T00:10:11.887 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:50.011143+0000", 2024-09-16T00:10:11.887 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 21.760114819999998, 2024-09-16T00:10:11.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.056951953, 2024-09-16T00:10:11.888 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.888 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.888 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.888 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.011143+0000", 2024-09-16T00:10:11.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.011143+0000", 2024-09-16T00:10:11.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.011145+0000", 2024-09-16T00:10:11.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2869999999999999e-06 2024-09-16T00:10:11.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.011161+0000", 2024-09-16T00:10:11.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6920999999999999e-05 2024-09-16T00:10:11.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063382+0000", 2024-09-16T00:10:11.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.052221072 2024-09-16T00:10:11.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063394+0000", 2024-09-16T00:10:11.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1212999999999999e-05 2024-09-16T00:10:11.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063454+0000", 2024-09-16T00:10:11.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9852999999999998e-05 2024-09-16T00:10:11.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063477+0000", 2024-09-16T00:10:11.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3638999999999998e-05 2024-09-16T00:10:11.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.068064+0000", 2024-09-16T00:10:11.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045870599999999996 2024-09-16T00:10:11.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.068095+0000", 2024-09-16T00:10:11.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0908000000000003e-05 2024-09-16T00:10:11.891 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.891 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.892 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.892 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13967 3.32 3:4d814898:::10000003587.00000001:head [write 0~4194304 [2@0]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:11.892 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:50.020209+0000", 2024-09-16T00:10:11.892 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 21.751049141999999, 2024-09-16T00:10:11.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.050981571000000003, 2024-09-16T00:10:11.892 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.892 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.892 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.892 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.892 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.892 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.892 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13967 2024-09-16T00:10:11.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.893 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.020209+0000", 2024-09-16T00:10:11.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.893 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.020209+0000", 2024-09-16T00:10:11.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.893 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.020212+0000", 2024-09-16T00:10:11.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.802e-06 2024-09-16T00:10:11.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.021676+0000", 2024-09-16T00:10:11.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014645909999999999 2024-09-16T00:10:11.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063378+0000", 2024-09-16T00:10:11.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.041701657000000003 2024-09-16T00:10:11.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063395+0000", 2024-09-16T00:10:11.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7469000000000002e-05 2024-09-16T00:10:11.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063457+0000", 2024-09-16T00:10:11.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1450000000000003e-05 2024-09-16T00:10:11.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063621+0000", 2024-09-16T00:10:11.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000164026 2024-09-16T00:10:11.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.896 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:11.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063777+0000", 2024-09-16T00:10:11.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015628899999999999 2024-09-16T00:10:11.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.896 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.070303+0000", 2024-09-16T00:10:11.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0065262860000000001 2024-09-16T00:10:11.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.896 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.071105+0000", 2024-09-16T00:10:11.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000801423 2024-09-16T00:10:11.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.897 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.071154+0000", 2024-09-16T00:10:11.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8949000000000002e-05 2024-09-16T00:10:11.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.897 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.071190+0000", 2024-09-16T00:10:11.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6628999999999998e-05 2024-09-16T00:10:11.897 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.897 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.897 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.898 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14815 3.17 3:e9f3a865:::10000003294.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:11.898 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.505112+0000", 2024-09-16T00:10:11.898 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.266146195000001, 2024-09-16T00:10:11.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011798104, 2024-09-16T00:10:11.898 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:11.898 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:11.898 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:11.898 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:11.898 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:11.898 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:11.898 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14815 2024-09-16T00:10:11.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.898 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:11.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:11.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.505112+0000", 2024-09-16T00:10:11.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.899 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:11.899 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.505112+0000", 2024-09-16T00:10:11.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:11.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:11.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.505114+0000", 2024-09-16T00:10:11.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.99e-06 2024-09-16T00:10:11.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:11.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.507662+0000", 2024-09-16T00:10:11.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0025480870000000001 2024-09-16T00:10:11.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:11.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.507692+0000", 2024-09-16T00:10:11.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9842999999999999e-05 2024-09-16T00:10:11.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:11.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.507705+0000", 2024-09-16T00:10:11.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3079e-05 2024-09-16T00:10:11.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:11.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.507817+0000", 2024-09-16T00:10:11.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011216499999999999 2024-09-16T00:10:11.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:11.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.507917+0000", 2024-09-16T00:10:11.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9686999999999996e-05 2024-09-16T00:10:11.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:11.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.508080+0000", 2024-09-16T00:10:11.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016345800000000001 2024-09-16T00:10:11.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:11.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.515396+0000", 2024-09-16T00:10:11.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073156189999999998 2024-09-16T00:10:11.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:11.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.516834+0000", 2024-09-16T00:10:11.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014385470000000001 2024-09-16T00:10:11.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:11.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.516875+0000", 2024-09-16T00:10:11.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0558e-05 2024-09-16T00:10:11.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:11.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:11.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:11.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.516910+0000", 2024-09-16T00:10:11.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5070999999999997e-05 2024-09-16T00:10:11.903 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.903 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.903 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.903 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:11.904 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:11.904 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:11.905 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:10:11.906 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:11.905+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:12.006 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:12.006+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:12.054 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:12.054 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:12.054 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:12.054 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:12.063 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:10:12.147 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:12.146+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:12.208 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:12.208+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:12.212 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:12.213 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:12.213 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:12.213 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:12.213 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:12.222 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:10:12.309 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:12.309+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:12.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:12.332+0000 7fcb05ffb640 1 -- 172.21.15.139:0/2758485754 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v104) ==== 91+0+0 (secure 0 0 0) 0x7fcb0807ab00 con 0x7fcb1015b280 2024-09-16T00:10:12.333 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is unset 2024-09-16T00:10:12.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:12.334+0000 7fcb17445640 1 -- 172.21.15.139:0/2758485754 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcaf406c2e0 msgr2=0x7fcaf406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:12.334+0000 7fcb17445640 1 --2- 172.21.15.139:0/2758485754 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcaf406c2e0 0x7fcaf406e7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c007fd0 tx=0x7fcb0c009040 comp rx=0 tx=0).stop 2024-09-16T00:10:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:12.334+0000 7fcb17445640 1 -- 172.21.15.139:0/2758485754 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb1015b280 msgr2=0x7fcb10150820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:12.334+0000 7fcb17445640 1 --2- 172.21.15.139:0/2758485754 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb1015b280 0x7fcb10150820 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcb08031000 tx=0x7fcb08004a40 comp rx=0 tx=0).stop 2024-09-16T00:10:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:12.334+0000 7fcb17445640 1 -- 172.21.15.139:0/2758485754 shutdown_connections 2024-09-16T00:10:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:12.334+0000 7fcb17445640 1 --2- 172.21.15.139:0/2758485754 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcaf406c2e0 0x7fcaf406e7a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:12.334+0000 7fcb17445640 1 --2- 172.21.15.139:0/2758485754 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcb1015b280 0x7fcb10150820 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:12.334+0000 7fcb17445640 1 --2- 172.21.15.139:0/2758485754 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcb10154c20 0x7fcb101502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:12.334+0000 7fcb17445640 1 --2- 172.21.15.139:0/2758485754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcb10068060 0x7fcb1015d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:12.334+0000 7fcb17445640 1 -- 172.21.15.139:0/2758485754 >> 172.21.15.139:0/2758485754 conn(0x7fcb1006ca40 msgr2=0x7fcb100c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:12.334+0000 7fcb17445640 1 -- 172.21.15.139:0/2758485754 shutdown_connections 2024-09-16T00:10:12.336 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:12.334+0000 7fcb17445640 1 -- 172.21.15.139:0/2758485754 wait complete. 2024-09-16T00:10:12.376 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:12.377 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:12.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:12.377 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:12.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.377 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3161 2.d 2:b02ffd62:::200.00000010:head [write 3427859~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.377 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:12.377 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.712065689999999, 2024-09-16T00:10:12.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026748516, 2024-09-16T00:10:12.377 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.377 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.377 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.377 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:12.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:12.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:12.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.15e-07 2024-09-16T00:10:12.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662637+0000", 2024-09-16T00:10:12.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8009999999999999e-06 2024-09-16T00:10:12.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662645+0000", 2024-09-16T00:10:12.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8299999999999996e-06 2024-09-16T00:10:12.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662650+0000", 2024-09-16T00:10:12.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.605e-06 2024-09-16T00:10:12.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.682553+0000", 2024-09-16T00:10:12.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01990331 2024-09-16T00:10:12.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.682636+0000", 2024-09-16T00:10:12.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2921000000000001e-05 2024-09-16T00:10:12.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.682780+0000", 2024-09-16T00:10:12.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000144361 2024-09-16T00:10:12.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.684809+0000", 2024-09-16T00:10:12.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002029005 2024-09-16T00:10:12.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689293+0000", 2024-09-16T00:10:12.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044841079999999997 2024-09-16T00:10:12.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689339+0000", 2024-09-16T00:10:12.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5668999999999999e-05 2024-09-16T00:10:12.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689382+0000", 2024-09-16T00:10:12.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3491000000000001e-05 2024-09-16T00:10:12.382 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.383 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.383 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.383 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3162 2.d 2:b02ffd62:::200.00000010:head [write 3430104~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.383 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.662858+0000", 2024-09-16T00:10:12.383 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.711841574000001, 2024-09-16T00:10:12.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026630083999999998, 2024-09-16T00:10:12.383 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.383 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.383 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.383 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662858+0000", 2024-09-16T00:10:12.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662858+0000", 2024-09-16T00:10:12.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662859+0000", 2024-09-16T00:10:12.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:10:12.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662862+0000", 2024-09-16T00:10:12.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6119999999999999e-06 2024-09-16T00:10:12.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662870+0000", 2024-09-16T00:10:12.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2360000000000004e-06 2024-09-16T00:10:12.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662874+0000", 2024-09-16T00:10:12.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2610000000000002e-06 2024-09-16T00:10:12.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.683241+0000", 2024-09-16T00:10:12.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.020367495999999999 2024-09-16T00:10:12.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.683295+0000", 2024-09-16T00:10:12.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4230000000000001e-05 2024-09-16T00:10:12.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.683380+0000", 2024-09-16T00:10:12.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4367e-05 2024-09-16T00:10:12.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.684845+0000", 2024-09-16T00:10:12.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014653190000000001 2024-09-16T00:10:12.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689392+0000", 2024-09-16T00:10:12.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045474280000000001 2024-09-16T00:10:12.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689445+0000", 2024-09-16T00:10:12.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2444000000000001e-05 2024-09-16T00:10:12.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689488+0000", 2024-09-16T00:10:12.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3124999999999998e-05 2024-09-16T00:10:12.388 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.388 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.388 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.388 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3166 2.d 2:b02ffd62:::200.00000010:head [write 3437392~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.389 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.664403+0000", 2024-09-16T00:10:12.389 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.710296696, 2024-09-16T00:10:12.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.027480674, 2024-09-16T00:10:12.389 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.389 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.389 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.389 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664403+0000", 2024-09-16T00:10:12.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664403+0000", 2024-09-16T00:10:12.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664404+0000", 2024-09-16T00:10:12.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4099999999999998e-07 2024-09-16T00:10:12.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664408+0000", 2024-09-16T00:10:12.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8169999999999998e-06 2024-09-16T00:10:12.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664418+0000", 2024-09-16T00:10:12.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6909999999999992e-06 2024-09-16T00:10:12.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664425+0000", 2024-09-16T00:10:12.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6309999999999997e-06 2024-09-16T00:10:12.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.685372+0000", 2024-09-16T00:10:12.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.020947548999999999 2024-09-16T00:10:12.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.685454+0000", 2024-09-16T00:10:12.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1355999999999998e-05 2024-09-16T00:10:12.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.685585+0000", 2024-09-16T00:10:12.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000130922 2024-09-16T00:10:12.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687329+0000", 2024-09-16T00:10:12.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017445710000000001 2024-09-16T00:10:12.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691757+0000", 2024-09-16T00:10:12.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004428268 2024-09-16T00:10:12.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691830+0000", 2024-09-16T00:10:12.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2237999999999995e-05 2024-09-16T00:10:12.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691884+0000", 2024-09-16T00:10:12.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3890000000000001e-05 2024-09-16T00:10:12.395 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.395 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.395 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.395 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3167 2.d 2:b02ffd62:::200.00000010:head [write 3439637~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.395 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.664627+0000", 2024-09-16T00:10:12.395 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.710072124, 2024-09-16T00:10:12.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.027347698, 2024-09-16T00:10:12.395 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.395 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.395 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.395 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664627+0000", 2024-09-16T00:10:12.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664627+0000", 2024-09-16T00:10:12.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664628+0000", 2024-09-16T00:10:12.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0500000000000003e-07 2024-09-16T00:10:12.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664632+0000", 2024-09-16T00:10:12.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8770000000000003e-06 2024-09-16T00:10:12.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664639+0000", 2024-09-16T00:10:12.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6590000000000001e-06 2024-09-16T00:10:12.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664642+0000", 2024-09-16T00:10:12.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8550000000000004e-06 2024-09-16T00:10:12.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.686530+0000", 2024-09-16T00:10:12.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.021887127999999999 2024-09-16T00:10:12.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.686577+0000", 2024-09-16T00:10:12.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7181000000000001e-05 2024-09-16T00:10:12.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.686667+0000", 2024-09-16T00:10:12.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0074000000000005e-05 2024-09-16T00:10:12.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688714+0000", 2024-09-16T00:10:12.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020467749999999998 2024-09-16T00:10:12.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691894+0000", 2024-09-16T00:10:12.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031806529999999999 2024-09-16T00:10:12.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691939+0000", 2024-09-16T00:10:12.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4579999999999997e-05 2024-09-16T00:10:12.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691975+0000", 2024-09-16T00:10:12.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6310999999999998e-05 2024-09-16T00:10:12.400 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.400 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.400 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.401 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3168 2.d 2:b02ffd62:::200.00000010:head [write 3441036~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.401 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:12.401 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.709606043000001, 2024-09-16T00:10:12.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026965759999999998, 2024-09-16T00:10:12.401 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.401 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.401 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.401 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:12.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:12.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:12.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7999999999999996e-07 2024-09-16T00:10:12.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665098+0000", 2024-09-16T00:10:12.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3200000000000001e-06 2024-09-16T00:10:12.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665104+0000", 2024-09-16T00:10:12.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4809999999999997e-06 2024-09-16T00:10:12.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665112+0000", 2024-09-16T00:10:12.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.67e-06 2024-09-16T00:10:12.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687392+0000", 2024-09-16T00:10:12.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.022279699 2024-09-16T00:10:12.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687461+0000", 2024-09-16T00:10:12.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8476e-05 2024-09-16T00:10:12.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687577+0000", 2024-09-16T00:10:12.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011661 2024-09-16T00:10:12.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688774+0000", 2024-09-16T00:10:12.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011962660000000001 2024-09-16T00:10:12.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691984+0000", 2024-09-16T00:10:12.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032106320000000002 2024-09-16T00:10:12.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692028+0000", 2024-09-16T00:10:12.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3866999999999999e-05 2024-09-16T00:10:12.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692059+0000", 2024-09-16T00:10:12.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1259e-05 2024-09-16T00:10:12.407 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.407 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.407 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.407 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3169 2.d 2:b02ffd62:::200.00000010:head [write 3443281~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.408 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:12.408 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.709149506999999, 2024-09-16T00:10:12.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.030090967999999999, 2024-09-16T00:10:12.408 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.408 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.408 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.408 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:12.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:12.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:12.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3799999999999998e-07 2024-09-16T00:10:12.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665554+0000", 2024-09-16T00:10:12.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.112e-06 2024-09-16T00:10:12.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665559+0000", 2024-09-16T00:10:12.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9559999999999996e-06 2024-09-16T00:10:12.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665562+0000", 2024-09-16T00:10:12.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2909999999999999e-06 2024-09-16T00:10:12.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687954+0000", 2024-09-16T00:10:12.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.022391714 2024-09-16T00:10:12.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688016+0000", 2024-09-16T00:10:12.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2384000000000004e-05 2024-09-16T00:10:12.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688147+0000", 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013099300000000001 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690321+0000", 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021744339999999998 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695548+0000", 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052266079999999998 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695595+0000", 2024-09-16T00:10:12.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7200000000000002e-05 2024-09-16T00:10:12.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695641+0000", 2024-09-16T00:10:12.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5837999999999997e-05 2024-09-16T00:10:12.415 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.415 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.415 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.415 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3170 2.d 2:b02ffd62:::200.00000010:head [write 3445526~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.415 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:12.416 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.708956445999998, 2024-09-16T00:10:12.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029987413000000001, 2024-09-16T00:10:12.416 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.416 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.416 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.416 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:12.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:12.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:12.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7500000000000001e-07 2024-09-16T00:10:12.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665753+0000", 2024-09-16T00:10:12.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0005e-05 2024-09-16T00:10:12.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665759+0000", 2024-09-16T00:10:12.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.169e-06 2024-09-16T00:10:12.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665762+0000", 2024-09-16T00:10:12.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8460000000000001e-06 2024-09-16T00:10:12.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688811+0000", 2024-09-16T00:10:12.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023048169 2024-09-16T00:10:12.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688885+0000", 2024-09-16T00:10:12.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4788000000000003e-05 2024-09-16T00:10:12.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688989+0000", 2024-09-16T00:10:12.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000103689 2024-09-16T00:10:12.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690350+0000", 2024-09-16T00:10:12.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013609329999999999 2024-09-16T00:10:12.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695652+0000", 2024-09-16T00:10:12.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053019080000000001 2024-09-16T00:10:12.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695698+0000", 2024-09-16T00:10:12.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5775000000000001e-05 2024-09-16T00:10:12.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695731+0000", 2024-09-16T00:10:12.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2756000000000003e-05 2024-09-16T00:10:12.420 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.421 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.421 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.421 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3171 2.d 2:b02ffd62:::200.00000010:head [write 3446925~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.421 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665997+0000", 2024-09-16T00:10:12.421 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.708702355, 2024-09-16T00:10:12.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029824032, 2024-09-16T00:10:12.421 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.421 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.421 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.421 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665997+0000", 2024-09-16T00:10:12.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665997+0000", 2024-09-16T00:10:12.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665998+0000", 2024-09-16T00:10:12.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:10:12.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666000+0000", 2024-09-16T00:10:12.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6290000000000002e-06 2024-09-16T00:10:12.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666005+0000", 2024-09-16T00:10:12.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.437e-06 2024-09-16T00:10:12.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666008+0000", 2024-09-16T00:10:12.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7910000000000002e-06 2024-09-16T00:10:12.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689799+0000", 2024-09-16T00:10:12.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023790504000000001 2024-09-16T00:10:12.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689907+0000", 2024-09-16T00:10:12.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108464 2024-09-16T00:10:12.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689994+0000", 2024-09-16T00:10:12.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7043e-05 2024-09-16T00:10:12.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691737+0000", 2024-09-16T00:10:12.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001742527 2024-09-16T00:10:12.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695739+0000", 2024-09-16T00:10:12.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040020189999999999 2024-09-16T00:10:12.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695786+0000", 2024-09-16T00:10:12.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7219999999999999e-05 2024-09-16T00:10:12.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695821+0000", 2024-09-16T00:10:12.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5058000000000002e-05 2024-09-16T00:10:12.426 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.426 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.426 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.426 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3172 2.d 2:b02ffd62:::200.00000010:head [write 3448324~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.426 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.666428+0000", 2024-09-16T00:10:12.427 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.708271405000001, 2024-09-16T00:10:12.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029477316, 2024-09-16T00:10:12.427 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.427 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.427 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.427 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666428+0000", 2024-09-16T00:10:12.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666428+0000", 2024-09-16T00:10:12.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666429+0000", 2024-09-16T00:10:12.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3699999999999999e-07 2024-09-16T00:10:12.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666433+0000", 2024-09-16T00:10:12.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4939999999999997e-06 2024-09-16T00:10:12.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666443+0000", 2024-09-16T00:10:12.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0899999999999994e-06 2024-09-16T00:10:12.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666447+0000", 2024-09-16T00:10:12.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8930000000000004e-06 2024-09-16T00:10:12.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690390+0000", 2024-09-16T00:10:12.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023942215999999999 2024-09-16T00:10:12.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690464+0000", 2024-09-16T00:10:12.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4678999999999994e-05 2024-09-16T00:10:12.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690576+0000", 2024-09-16T00:10:12.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011162000000000001 2024-09-16T00:10:12.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692126+0000", 2024-09-16T00:10:12.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015496520000000001 2024-09-16T00:10:12.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695828+0000", 2024-09-16T00:10:12.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037028510000000001 2024-09-16T00:10:12.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695870+0000", 2024-09-16T00:10:12.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1180999999999998e-05 2024-09-16T00:10:12.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695905+0000", 2024-09-16T00:10:12.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5803000000000003e-05 2024-09-16T00:10:12.432 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.432 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.432 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.432 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3173 2.d 2:b02ffd62:::200.00000010:head [write 3449723~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.432 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.666884+0000", 2024-09-16T00:10:12.432 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.707815449000002, 2024-09-16T00:10:12.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.02908875, 2024-09-16T00:10:12.432 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.433 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.433 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.433 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666884+0000", 2024-09-16T00:10:12.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666884+0000", 2024-09-16T00:10:12.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666885+0000", 2024-09-16T00:10:12.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0999999999999998e-07 2024-09-16T00:10:12.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666889+0000", 2024-09-16T00:10:12.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4299999999999999e-06 2024-09-16T00:10:12.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666895+0000", 2024-09-16T00:10:12.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0510000000000002e-06 2024-09-16T00:10:12.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666899+0000", 2024-09-16T00:10:12.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0489999999999997e-06 2024-09-16T00:10:12.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691161+0000", 2024-09-16T00:10:12.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.024262156999999999 2024-09-16T00:10:12.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691253+0000", 2024-09-16T00:10:12.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1278000000000003e-05 2024-09-16T00:10:12.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691348+0000", 2024-09-16T00:10:12.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4777000000000004e-05 2024-09-16T00:10:12.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692211+0000", 2024-09-16T00:10:12.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086351699999999995 2024-09-16T00:10:12.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695913+0000", 2024-09-16T00:10:12.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037019840000000002 2024-09-16T00:10:12.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695952+0000", 2024-09-16T00:10:12.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8535999999999998e-05 2024-09-16T00:10:12.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695973+0000", 2024-09-16T00:10:12.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1260999999999999e-05 2024-09-16T00:10:12.441 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.441 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.441 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.442 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3174 2.d 2:b02ffd62:::200.00000010:head [write 3451968~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.442 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.667323+0000", 2024-09-16T00:10:12.442 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.707376578999998, 2024-09-16T00:10:12.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028788859999999999, 2024-09-16T00:10:12.442 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.442 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.442 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.442 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667323+0000", 2024-09-16T00:10:12.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667323+0000", 2024-09-16T00:10:12.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667324+0000", 2024-09-16T00:10:12.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0400000000000005e-07 2024-09-16T00:10:12.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667330+0000", 2024-09-16T00:10:12.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3300000000000004e-06 2024-09-16T00:10:12.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667338+0000", 2024-09-16T00:10:12.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0059999999999992e-06 2024-09-16T00:10:12.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667344+0000", 2024-09-16T00:10:12.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6409999999999999e-06 2024-09-16T00:10:12.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692253+0000", 2024-09-16T00:10:12.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.024909280999999998 2024-09-16T00:10:12.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692311+0000", 2024-09-16T00:10:12.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7873e-05 2024-09-16T00:10:12.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692395+0000", 2024-09-16T00:10:12.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4071000000000002e-05 2024-09-16T00:10:12.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695977+0000", 2024-09-16T00:10:12.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035823460000000001 2024-09-16T00:10:12.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696034+0000", 2024-09-16T00:10:12.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6697999999999998e-05 2024-09-16T00:10:12.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696078+0000", 2024-09-16T00:10:12.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3696000000000003e-05 2024-09-16T00:10:12.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696112+0000", 2024-09-16T00:10:12.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4113999999999999e-05 2024-09-16T00:10:12.447 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.447 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.447 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.447 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3175 2.d 2:b02ffd62:::200.00000010:head [write 3454213~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.447 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:12.447 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.707035740999999, 2024-09-16T00:10:12.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.030153698999999999, 2024-09-16T00:10:12.447 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.447 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.447 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.447 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:12.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:12.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:12.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2e-07 2024-09-16T00:10:12.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667668+0000", 2024-09-16T00:10:12.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2109999999999998e-06 2024-09-16T00:10:12.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667673+0000", 2024-09-16T00:10:12.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8409999999999996e-06 2024-09-16T00:10:12.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667686+0000", 2024-09-16T00:10:12.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2928000000000001e-05 2024-09-16T00:10:12.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692964+0000", 2024-09-16T00:10:12.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.025277642999999999 2024-09-16T00:10:12.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.694418+0000", 2024-09-16T00:10:12.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001453693 2024-09-16T00:10:12.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.694505+0000", 2024-09-16T00:10:12.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7700000000000004e-05 2024-09-16T00:10:12.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697114+0000", 2024-09-16T00:10:12.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026083460000000001 2024-09-16T00:10:12.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697697+0000", 2024-09-16T00:10:12.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00058297399999999999 2024-09-16T00:10:12.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697772+0000", 2024-09-16T00:10:12.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5433999999999996e-05 2024-09-16T00:10:12.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697818+0000", 2024-09-16T00:10:12.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5408999999999998e-05 2024-09-16T00:10:12.452 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.452 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.452 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.452 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3176 2.d 2:b02ffd62:::200.00000010:head [write 3456458~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.453 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.668006+0000", 2024-09-16T00:10:12.453 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.706693835999999, 2024-09-16T00:10:12.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029935413000000001, 2024-09-16T00:10:12.453 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.453 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.453 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.453 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668006+0000", 2024-09-16T00:10:12.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668006+0000", 2024-09-16T00:10:12.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668007+0000", 2024-09-16T00:10:12.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5799999999999998e-07 2024-09-16T00:10:12.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668012+0000", 2024-09-16T00:10:12.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9320000000000004e-06 2024-09-16T00:10:12.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668023+0000", 2024-09-16T00:10:12.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1273e-05 2024-09-16T00:10:12.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668030+0000", 2024-09-16T00:10:12.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0509999999999999e-06 2024-09-16T00:10:12.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.694928+0000", 2024-09-16T00:10:12.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026898114000000001 2024-09-16T00:10:12.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695003+0000", 2024-09-16T00:10:12.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4880000000000001e-05 2024-09-16T00:10:12.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695116+0000", 2024-09-16T00:10:12.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011307900000000001 2024-09-16T00:10:12.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697119+0000", 2024-09-16T00:10:12.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020032159999999999 2024-09-16T00:10:12.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697867+0000", 2024-09-16T00:10:12.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074744999999999998 2024-09-16T00:10:12.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697908+0000", 2024-09-16T00:10:12.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1287e-05 2024-09-16T00:10:12.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697941+0000", 2024-09-16T00:10:12.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3272999999999998e-05 2024-09-16T00:10:12.458 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.458 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.458 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.458 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3177 2.d 2:b02ffd62:::200.00000010:head [write 3457857~1398 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.458 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:12.458 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.705358976999999, 2024-09-16T00:10:12.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029906856999999998, 2024-09-16T00:10:12.458 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.458 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.458 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.458 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:12.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:12.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:12.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1299999999999998e-07 2024-09-16T00:10:12.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669347+0000", 2024-09-16T00:10:12.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.327e-06 2024-09-16T00:10:12.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669359+0000", 2024-09-16T00:10:12.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1698e-05 2024-09-16T00:10:12.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669366+0000", 2024-09-16T00:10:12.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7959999999999996e-06 2024-09-16T00:10:12.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696168+0000", 2024-09-16T00:10:12.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026802093999999999 2024-09-16T00:10:12.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696258+0000", 2024-09-16T00:10:12.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0074000000000005e-05 2024-09-16T00:10:12.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696383+0000", 2024-09-16T00:10:12.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012455699999999999 2024-09-16T00:10:12.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698602+0000", 2024-09-16T00:10:12.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002218921 2024-09-16T00:10:12.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699152+0000", 2024-09-16T00:10:12.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055024399999999997 2024-09-16T00:10:12.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.463 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699206+0000", 2024-09-16T00:10:12.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3882999999999999e-05 2024-09-16T00:10:12.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.463 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699247+0000", 2024-09-16T00:10:12.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1350000000000002e-05 2024-09-16T00:10:12.464 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.464 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.464 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.464 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3178 2.d 2:b02ffd62:::200.00000010:head [write 3459255~2511 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.464 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.671779+0000", 2024-09-16T00:10:12.464 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.702920097, 2024-09-16T00:10:12.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028813986999999999, 2024-09-16T00:10:12.464 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.464 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.464 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.464 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671779+0000", 2024-09-16T00:10:12.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671779+0000", 2024-09-16T00:10:12.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671780+0000", 2024-09-16T00:10:12.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3600000000000002e-07 2024-09-16T00:10:12.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671785+0000", 2024-09-16T00:10:12.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.074e-06 2024-09-16T00:10:12.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671795+0000", 2024-09-16T00:10:12.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2040000000000003e-06 2024-09-16T00:10:12.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671800+0000", 2024-09-16T00:10:12.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9420000000000003e-06 2024-09-16T00:10:12.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697159+0000", 2024-09-16T00:10:12.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.025359619999999999 2024-09-16T00:10:12.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697214+0000", 2024-09-16T00:10:12.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5318000000000001e-05 2024-09-16T00:10:12.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697297+0000", 2024-09-16T00:10:12.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2087000000000002e-05 2024-09-16T00:10:12.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699258+0000", 2024-09-16T00:10:12.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019619529999999998 2024-09-16T00:10:12.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700499+0000", 2024-09-16T00:10:12.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001240072 2024-09-16T00:10:12.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700555+0000", 2024-09-16T00:10:12.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6459999999999998e-05 2024-09-16T00:10:12.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700593+0000", 2024-09-16T00:10:12.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8420999999999997e-05 2024-09-16T00:10:12.469 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.469 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.469 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.470 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3179 2.d 2:b02ffd62:::200.00000010:head [write 3461766~2418 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.470 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.672548+0000", 2024-09-16T00:10:12.470 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.702152000000002, 2024-09-16T00:10:12.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028160178000000001, 2024-09-16T00:10:12.470 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.470 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.470 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.470 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672548+0000", 2024-09-16T00:10:12.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672548+0000", 2024-09-16T00:10:12.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672549+0000", 2024-09-16T00:10:12.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5300000000000002e-07 2024-09-16T00:10:12.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672562+0000", 2024-09-16T00:10:12.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3808000000000001e-05 2024-09-16T00:10:12.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672575+0000", 2024-09-16T00:10:12.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2870999999999999e-05 2024-09-16T00:10:12.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672587+0000", 2024-09-16T00:10:12.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.203e-05 2024-09-16T00:10:12.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697994+0000", 2024-09-16T00:10:12.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.025406373999999999 2024-09-16T00:10:12.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698074+0000", 2024-09-16T00:10:12.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0802000000000003e-05 2024-09-16T00:10:12.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698196+0000", 2024-09-16T00:10:12.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012154699999999999 2024-09-16T00:10:12.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700456+0000", 2024-09-16T00:10:12.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022602249999999998 2024-09-16T00:10:12.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700632+0000", 2024-09-16T00:10:12.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017570699999999999 2024-09-16T00:10:12.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700676+0000", 2024-09-16T00:10:12.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3764000000000003e-05 2024-09-16T00:10:12.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700708+0000", 2024-09-16T00:10:12.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2097000000000001e-05 2024-09-16T00:10:12.475 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.475 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.475 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.475 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3180 2.d 2:b02ffd62:::200.00000010:head [write 3464184~1398 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.475 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:12.475 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.700644049000001, 2024-09-16T00:10:12.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028112109, 2024-09-16T00:10:12.475 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.475 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.475 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.475 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:12.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:12.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:12.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:10:12.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674061+0000", 2024-09-16T00:10:12.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4440000000000001e-06 2024-09-16T00:10:12.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674069+0000", 2024-09-16T00:10:12.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7639999999999994e-06 2024-09-16T00:10:12.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674076+0000", 2024-09-16T00:10:12.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5590000000000002e-06 2024-09-16T00:10:12.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698639+0000", 2024-09-16T00:10:12.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.024562471999999998 2024-09-16T00:10:12.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698693+0000", 2024-09-16T00:10:12.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4432999999999999e-05 2024-09-16T00:10:12.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698807+0000", 2024-09-16T00:10:12.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113607 2024-09-16T00:10:12.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700727+0000", 2024-09-16T00:10:12.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019201750000000001 2024-09-16T00:10:12.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702076+0000", 2024-09-16T00:10:12.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001349423 2024-09-16T00:10:12.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702128+0000", 2024-09-16T00:10:12.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2157000000000002e-05 2024-09-16T00:10:12.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702168+0000", 2024-09-16T00:10:12.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9292000000000003e-05 2024-09-16T00:10:12.480 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.480 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.480 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.480 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3181 2.d 2:b02ffd62:::200.00000010:head [write 3465582~2754 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.480 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.675904+0000", 2024-09-16T00:10:12.481 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.698795738000001, 2024-09-16T00:10:12.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026402452, 2024-09-16T00:10:12.481 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.481 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.481 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.481 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675904+0000", 2024-09-16T00:10:12.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675904+0000", 2024-09-16T00:10:12.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675905+0000", 2024-09-16T00:10:12.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.043e-06 2024-09-16T00:10:12.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675911+0000", 2024-09-16T00:10:12.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1970000000000001e-06 2024-09-16T00:10:12.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675924+0000", 2024-09-16T00:10:12.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.252e-05 2024-09-16T00:10:12.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675931+0000", 2024-09-16T00:10:12.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5800000000000003e-06 2024-09-16T00:10:12.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699280+0000", 2024-09-16T00:10:12.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023348634 2024-09-16T00:10:12.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699329+0000", 2024-09-16T00:10:12.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9175999999999998e-05 2024-09-16T00:10:12.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699400+0000", 2024-09-16T00:10:12.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0649999999999996e-05 2024-09-16T00:10:12.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700734+0000", 2024-09-16T00:10:12.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001334816 2024-09-16T00:10:12.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702219+0000", 2024-09-16T00:10:12.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014847930000000001 2024-09-16T00:10:12.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702267+0000", 2024-09-16T00:10:12.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8133000000000001e-05 2024-09-16T00:10:12.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702306+0000", 2024-09-16T00:10:12.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8909999999999998e-05 2024-09-16T00:10:12.486 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.486 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.486 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.486 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3184 2.d 2:b02ffd62:::200.00000010:head [write 3478077~11949 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.486 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.681430+0000", 2024-09-16T00:10:12.486 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.693269495999999, 2024-09-16T00:10:12.486 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.02662339, 2024-09-16T00:10:12.486 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.486 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.486 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.486 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.487 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681430+0000", 2024-09-16T00:10:12.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.487 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681430+0000", 2024-09-16T00:10:12.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.487 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681431+0000", 2024-09-16T00:10:12.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9999999999999995e-07 2024-09-16T00:10:12.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681443+0000", 2024-09-16T00:10:12.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1568e-05 2024-09-16T00:10:12.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681453+0000", 2024-09-16T00:10:12.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8819999999999996e-06 2024-09-16T00:10:12.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681458+0000", 2024-09-16T00:10:12.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3709999999999999e-06 2024-09-16T00:10:12.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.701384+0000", 2024-09-16T00:10:12.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.019926342999999999 2024-09-16T00:10:12.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.701458+0000", 2024-09-16T00:10:12.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3854000000000001e-05 2024-09-16T00:10:12.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.701570+0000", 2024-09-16T00:10:12.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000111437 2024-09-16T00:10:12.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702913+0000", 2024-09-16T00:10:12.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013437259999999999 2024-09-16T00:10:12.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.707984+0000", 2024-09-16T00:10:12.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050705669999999998 2024-09-16T00:10:12.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.491 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.491 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708016+0000", 2024-09-16T00:10:12.491 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2527999999999998e-05 2024-09-16T00:10:12.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.491 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.491 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708053+0000", 2024-09-16T00:10:12.491 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7113999999999997e-05 2024-09-16T00:10:12.491 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.491 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.491 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.492 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3185 2.d 2:b02ffd62:::200.00000010:head [write 3490026~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:12.492 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.681685+0000", 2024-09-16T00:10:12.492 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.693014224999999, 2024-09-16T00:10:12.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026552664, 2024-09-16T00:10:12.492 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.492 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.492 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.492 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.492 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.492 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681685+0000", 2024-09-16T00:10:12.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.492 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.493 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.493 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681685+0000", 2024-09-16T00:10:12.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.493 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.493 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681686+0000", 2024-09-16T00:10:12.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.06e-07 2024-09-16T00:10:12.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.493 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.493 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681692+0000", 2024-09-16T00:10:12.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6840000000000001e-06 2024-09-16T00:10:12.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681697+0000", 2024-09-16T00:10:12.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0340000000000004e-06 2024-09-16T00:10:12.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681700+0000", 2024-09-16T00:10:12.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3589999999999999e-06 2024-09-16T00:10:12.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702500+0000", 2024-09-16T00:10:12.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.020799746000000001 2024-09-16T00:10:12.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702550+0000", 2024-09-16T00:10:12.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9895999999999998e-05 2024-09-16T00:10:12.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:12.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702649+0000", 2024-09-16T00:10:12.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9500999999999995e-05 2024-09-16T00:10:12.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.496 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.705343+0000", 2024-09-16T00:10:12.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026938130000000002 2024-09-16T00:10:12.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.496 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.496 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708128+0000", 2024-09-16T00:10:12.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027852979999999999 2024-09-16T00:10:12.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.496 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.496 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708202+0000", 2024-09-16T00:10:12.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4046000000000002e-05 2024-09-16T00:10:12.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708238+0000", 2024-09-16T00:10:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5781000000000002e-05 2024-09-16T00:10:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.497 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.497 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:12.499 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:10:12.500 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:12.498+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:12.599 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:12.599+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:12.650 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:12.650 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:12.650 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:12.650 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:12.659 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:10:12.703 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:12.702+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:12.766 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:12.766 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:12.766 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:12.766 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:12.766 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:12.776 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:10:12.817 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:12.816+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:12.903 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:12.902+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:12.952 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:12.952 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:12.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:12.952 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:12.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.952 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5850 3.3a 3:5c3cfc2b:::10000001adf.00000000:head [write 0~116] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:12.952 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:12.953 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 57.532797461000001, 2024-09-16T00:10:12.953 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66931295800000001, 2024-09-16T00:10:12.953 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.953 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.953 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.953 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:12.953 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:12.953 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:12.953 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5850 2024-09-16T00:10:12.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.953 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.953 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.953 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.953 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:12.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.954 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.954 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:12.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.954 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.954 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415938+0000", 2024-09-16T00:10:12.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7099999999999999e-06 2024-09-16T00:10:12.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.954 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415945+0000", 2024-09-16T00:10:12.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:10:12.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415959+0000", 2024-09-16T00:10:12.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3545e-05 2024-09-16T00:10:12.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415966+0000", 2024-09-16T00:10:12.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6379999999999998e-06 2024-09-16T00:10:12.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.956 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416013+0000", 2024-09-16T00:10:12.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7314999999999997e-05 2024-09-16T00:10:12.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.956 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:12.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416020+0000", 2024-09-16T00:10:12.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.52e-06 2024-09-16T00:10:12.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082393+0000", 2024-09-16T00:10:12.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66637251799999997 2024-09-16T00:10:12.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082537+0000", 2024-09-16T00:10:12.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014418800000000001 2024-09-16T00:10:12.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:12.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082689+0000", 2024-09-16T00:10:12.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015198499999999999 2024-09-16T00:10:12.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084930+0000", 2024-09-16T00:10:12.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002241041 2024-09-16T00:10:12.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084994+0000", 2024-09-16T00:10:12.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3274000000000004e-05 2024-09-16T00:10:12.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.959 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085049+0000", 2024-09-16T00:10:12.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.554e-05 2024-09-16T00:10:12.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.959 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085250+0000", 2024-09-16T00:10:12.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020060799999999999 2024-09-16T00:10:12.959 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.959 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.959 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.960 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5910 3.3a 3:5ec98a62:::10000001b22.00000000:head [write 0~1274] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:12.960 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:12.960 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 57.385853822999998, 2024-09-16T00:10:12.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.52286026600000002, 2024-09-16T00:10:12.960 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.960 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.960 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.960 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:12.961 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:12.961 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:12.961 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5910 2024-09-16T00:10:12.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.961 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:12.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.961 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.961 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.961 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:12.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562881+0000", 2024-09-16T00:10:12.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:10:12.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562887+0000", 2024-09-16T00:10:12.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2870000000000004e-06 2024-09-16T00:10:12.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562897+0000", 2024-09-16T00:10:12.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.064e-05 2024-09-16T00:10:12.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562902+0000", 2024-09-16T00:10:12.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2340000000000002e-06 2024-09-16T00:10:12.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562958+0000", 2024-09-16T00:10:12.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5523999999999999e-05 2024-09-16T00:10:12.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:12.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562968+0000", 2024-09-16T00:10:12.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0394000000000001e-05 2024-09-16T00:10:12.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083461+0000", 2024-09-16T00:10:12.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.520492968 2024-09-16T00:10:12.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083645+0000", 2024-09-16T00:10:12.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018346399999999999 2024-09-16T00:10:12.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:12.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083844+0000", 2024-09-16T00:10:12.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000199268 2024-09-16T00:10:12.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085410+0000", 2024-09-16T00:10:12.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015662359999999999 2024-09-16T00:10:12.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085493+0000", 2024-09-16T00:10:12.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2478999999999994e-05 2024-09-16T00:10:12.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085563+0000", 2024-09-16T00:10:12.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0159000000000004e-05 2024-09-16T00:10:12.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085741+0000", 2024-09-16T00:10:12.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017764300000000001 2024-09-16T00:10:12.966 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.967 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.967 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.967 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5962 3.3a 3:5c7b4d29:::10000001b5d.00000000:head [write 0~3074] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:12.967 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:12.967 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 57.256317006000003, 2024-09-16T00:10:12.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39368737500000001, 2024-09-16T00:10:12.967 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.967 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.967 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.967 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:12.967 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:12.967 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:12.968 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5962 2024-09-16T00:10:12.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.968 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:12.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:12.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692418+0000", 2024-09-16T00:10:12.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3510000000000001e-06 2024-09-16T00:10:12.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692425+0000", 2024-09-16T00:10:12.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3770000000000001e-06 2024-09-16T00:10:12.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.969 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.969 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692440+0000", 2024-09-16T00:10:12.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4975e-05 2024-09-16T00:10:12.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692448+0000", 2024-09-16T00:10:12.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6279999999999994e-06 2024-09-16T00:10:12.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692499+0000", 2024-09-16T00:10:12.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0872999999999999e-05 2024-09-16T00:10:12.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:12.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692507+0000", 2024-09-16T00:10:12.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9889999999999992e-06 2024-09-16T00:10:12.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.971 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084221+0000", 2024-09-16T00:10:12.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39171379699999997 2024-09-16T00:10:12.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.971 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084446+0000", 2024-09-16T00:10:12.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022519900000000001 2024-09-16T00:10:12.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.971 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:12.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084672+0000", 2024-09-16T00:10:12.972 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000225405 2024-09-16T00:10:12.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.972 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.972 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085832+0000", 2024-09-16T00:10:12.972 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011600040000000001 2024-09-16T00:10:12.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.972 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.972 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085864+0000", 2024-09-16T00:10:12.972 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2592999999999998e-05 2024-09-16T00:10:12.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.972 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.973 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085943+0000", 2024-09-16T00:10:12.973 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8534000000000004e-05 2024-09-16T00:10:12.973 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.973 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.973 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.973 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086105+0000", 2024-09-16T00:10:12.973 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016165 2024-09-16T00:10:12.973 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.973 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.973 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.973 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.973 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.973 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5963 3.3a 3:5e4192f5:::10000001b5e.00000000:head [write 0~152] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:12.973 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:12.974 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 57.254252575000002, 2024-09-16T00:10:12.974 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39413463900000001, 2024-09-16T00:10:12.974 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.974 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.974 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.974 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:12.974 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:12.974 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:12.974 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5963 2024-09-16T00:10:12.974 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.974 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.974 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.974 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.974 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:12.975 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.975 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.975 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:12.975 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.975 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.975 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:12.975 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:10:12.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.975 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.976 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694486+0000", 2024-09-16T00:10:12.976 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0960000000000003e-06 2024-09-16T00:10:12.976 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.976 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.976 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.976 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694496+0000", 2024-09-16T00:10:12.976 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2189999999999993e-06 2024-09-16T00:10:12.976 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.976 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.976 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.977 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694500+0000", 2024-09-16T00:10:12.977 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9529999999999998e-06 2024-09-16T00:10:12.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.977 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.977 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694552+0000", 2024-09-16T00:10:12.977 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1360999999999998e-05 2024-09-16T00:10:12.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.977 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:12.977 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694561+0000", 2024-09-16T00:10:12.977 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1700000000000003e-06 2024-09-16T00:10:12.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.978 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.978 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086346+0000", 2024-09-16T00:10:12.978 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39178468 2024-09-16T00:10:12.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.978 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.978 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.978 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086480+0000", 2024-09-16T00:10:12.978 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013469299999999999 2024-09-16T00:10:12.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.978 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.978 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:12.978 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086790+0000", 2024-09-16T00:10:12.978 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030923300000000001 2024-09-16T00:10:12.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.979 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.979 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.979 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088361+0000", 2024-09-16T00:10:12.979 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015717050000000001 2024-09-16T00:10:12.979 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.979 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.979 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.979 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088432+0000", 2024-09-16T00:10:12.979 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0922999999999998e-05 2024-09-16T00:10:12.979 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.979 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.979 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.979 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088489+0000", 2024-09-16T00:10:12.979 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6653999999999997e-05 2024-09-16T00:10:12.980 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.980 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.980 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.980 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088616+0000", 2024-09-16T00:10:12.980 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012730099999999999 2024-09-16T00:10:12.980 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.980 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.980 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.980 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.980 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.980 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:6082 3.3a 3:5c50f65b:::10000001be7.00000000:head [write 0~2439] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:12.980 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:12.980 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 56.960479718999999, 2024-09-16T00:10:12.980 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.10065850799999999, 2024-09-16T00:10:12.981 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.981 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.981 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.981 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:12.981 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:12.981 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:12.981 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 6082 2024-09-16T00:10:12.981 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.981 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.981 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.981 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.981 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:12.981 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.981 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.982 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.982 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:12.982 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.982 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.982 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.982 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988256+0000", 2024-09-16T00:10:12.982 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.065e-06 2024-09-16T00:10:12.982 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.982 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.982 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988262+0000", 2024-09-16T00:10:12.982 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4640000000000003e-06 2024-09-16T00:10:12.983 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.983 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.983 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.983 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988272+0000", 2024-09-16T00:10:12.983 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0335e-05 2024-09-16T00:10:12.983 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.983 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.983 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.983 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988279+0000", 2024-09-16T00:10:12.983 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:10:12.983 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.983 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.983 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.983 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988324+0000", 2024-09-16T00:10:12.983 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4953000000000001e-05 2024-09-16T00:10:12.984 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.984 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.984 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:12.984 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988331+0000", 2024-09-16T00:10:12.984 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5989999999999998e-06 2024-09-16T00:10:12.984 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.984 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.984 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.984 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087272+0000", 2024-09-16T00:10:12.984 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.098941045000000005 2024-09-16T00:10:12.984 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.984 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.984 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.984 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087486+0000", 2024-09-16T00:10:12.985 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00021360199999999999 2024-09-16T00:10:12.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.985 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.985 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:12.985 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087740+0000", 2024-09-16T00:10:12.985 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00025456799999999999 2024-09-16T00:10:12.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.985 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.985 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.985 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088638+0000", 2024-09-16T00:10:12.985 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00089793900000000005 2024-09-16T00:10:12.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.985 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.985 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.986 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088673+0000", 2024-09-16T00:10:12.986 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5111999999999999e-05 2024-09-16T00:10:12.986 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.986 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.987 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.987 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088722+0000", 2024-09-16T00:10:12.987 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8758000000000003e-05 2024-09-16T00:10:12.987 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.987 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.987 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.987 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088913+0000", 2024-09-16T00:10:12.987 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000190953 2024-09-16T00:10:12.988 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.988 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.988 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.988 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.988 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10643 3.21 3:8453163e:::100000030b3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:12.988 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:12.988 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.349003525999997, 2024-09-16T00:10:12.988 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013028809000000001, 2024-09-16T00:10:12.988 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.988 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.988 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.989 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:12.989 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:12.989 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:12.989 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10643 2024-09-16T00:10:12.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.989 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.989 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.989 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.989 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:12.989 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.990 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.990 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:12.990 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.990 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.990 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.990 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599734+0000", 2024-09-16T00:10:12.990 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-06 2024-09-16T00:10:12.990 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.991 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.991 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603345+0000", 2024-09-16T00:10:12.991 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003611628 2024-09-16T00:10:12.991 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.991 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.991 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.991 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603368+0000", 2024-09-16T00:10:12.991 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3017e-05 2024-09-16T00:10:12.991 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.991 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.991 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:12.992 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603379+0000", 2024-09-16T00:10:12.992 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0528e-05 2024-09-16T00:10:12.992 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.992 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.992 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:12.992 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603448+0000", 2024-09-16T00:10:12.992 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9080999999999998e-05 2024-09-16T00:10:12.992 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.992 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.992 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:12.993 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603623+0000", 2024-09-16T00:10:12.993 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017525599999999999 2024-09-16T00:10:12.993 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.993 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.993 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:12.993 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603766+0000", 2024-09-16T00:10:12.993 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000142977 2024-09-16T00:10:12.993 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.993 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.993 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:12.994 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.611250+0000", 2024-09-16T00:10:12.994 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074835240000000001 2024-09-16T00:10:12.994 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.994 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.994 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:12.994 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612617+0000", 2024-09-16T00:10:12.994 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001367793 2024-09-16T00:10:12.994 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.994 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.994 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:12.995 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612700+0000", 2024-09-16T00:10:12.995 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3123999999999999e-05 2024-09-16T00:10:12.995 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.995 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.995 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:12.995 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612759+0000", 2024-09-16T00:10:12.995 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8950999999999999e-05 2024-09-16T00:10:12.995 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.995 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:12.995 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:12.995 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.996 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.996 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10652 3.22 3:44e46336:::100000030c1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:12.996 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:10:12.996 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.320951893999997, 2024-09-16T00:10:12.996 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011405732, 2024-09-16T00:10:12.996 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:12.996 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:12.996 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:12.997 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:12.997 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:12.997 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:12.997 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10652 2024-09-16T00:10:12.997 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.997 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:12.997 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.997 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:12.997 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:10:12.997 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.998 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.998 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.998 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:12.998 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:10:12.998 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:12.998 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.998 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.998 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:12.998 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627785+0000", 2024-09-16T00:10:12.998 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6970000000000002e-06 2024-09-16T00:10:12.998 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.998 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:12.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629610+0000", 2024-09-16T00:10:12.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018250359999999999 2024-09-16T00:10:12.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:12.999 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629639+0000", 2024-09-16T00:10:12.999 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8676000000000001e-05 2024-09-16T00:10:12.999 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:12.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:12.999 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:13.000 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629652+0000", 2024-09-16T00:10:13.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3572e-05 2024-09-16T00:10:13.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:13.000 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629735+0000", 2024-09-16T00:10:13.000 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.25e-05 2024-09-16T00:10:13.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.000 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:13.001 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629925+0000", 2024-09-16T00:10:13.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019011000000000001 2024-09-16T00:10:13.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.001 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:10:13.001 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.630075+0000", 2024-09-16T00:10:13.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000149841 2024-09-16T00:10:13.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.001 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:13.001 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.636470+0000", 2024-09-16T00:10:13.001 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063952289999999997 2024-09-16T00:10:13.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.002 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:13.002 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639098+0000", 2024-09-16T00:10:13.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026277980000000002 2024-09-16T00:10:13.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.002 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:13.002 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639154+0000", 2024-09-16T00:10:13.002 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6413999999999999e-05 2024-09-16T00:10:13.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.003 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:13.003 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639188+0000", 2024-09-16T00:10:13.003 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3859000000000002e-05 2024-09-16T00:10:13.003 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.003 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:13.003 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.003 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11217 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:13.003 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:13.004 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.374454872999998, 2024-09-16T00:10:13.004 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01231253, 2024-09-16T00:10:13.004 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:13.004 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:13.004 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:13.004 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:13.004 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:13.004 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:13.004 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11217 2024-09-16T00:10:13.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.004 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:13.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.005 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:13.005 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:13.005 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.005 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:13.005 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:13.005 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.005 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.005 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:13.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574282+0000", 2024-09-16T00:10:13.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8499999999999998e-06 2024-09-16T00:10:13.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:13.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575804+0000", 2024-09-16T00:10:13.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001521859 2024-09-16T00:10:13.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.006 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:13.006 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575826+0000", 2024-09-16T00:10:13.006 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2059999999999999e-05 2024-09-16T00:10:13.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:13.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575837+0000", 2024-09-16T00:10:13.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1313e-05 2024-09-16T00:10:13.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.007 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:13.007 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575920+0000", 2024-09-16T00:10:13.007 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2918999999999997e-05 2024-09-16T00:10:13.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:13.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576088+0000", 2024-09-16T00:10:13.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016803799999999999 2024-09-16T00:10:13.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:13.008 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576233+0000", 2024-09-16T00:10:13.008 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014508199999999999 2024-09-16T00:10:13.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.008 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:13.009 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.580500+0000", 2024-09-16T00:10:13.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004266644 2024-09-16T00:10:13.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.009 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:13.009 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586479+0000", 2024-09-16T00:10:13.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059791820000000004 2024-09-16T00:10:13.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.009 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:13.009 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586541+0000", 2024-09-16T00:10:13.009 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1463000000000004e-05 2024-09-16T00:10:13.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.010 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:13.010 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586592+0000", 2024-09-16T00:10:13.010 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1119999999999998e-05 2024-09-16T00:10:13.010 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.010 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:13.010 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.010 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11767 3.38 3:1fa50e65:::10000003117.00000000:head [write 0~4194304 [3@1169732]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:13.011 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:13.011 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 29.785050338000001, 2024-09-16T00:10:13.011 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011555678999999999, 2024-09-16T00:10:13.011 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:13.011 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:13.011 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:13.011 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:13.011 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:13.012 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:13.012 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11767 2024-09-16T00:10:13.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.012 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:13.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.012 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:13.012 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:13.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:13.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:13.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.013 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:13.013 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163686+0000", 2024-09-16T00:10:13.013 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3259999999999998e-06 2024-09-16T00:10:13.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:13.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165053+0000", 2024-09-16T00:10:13.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013665859999999999 2024-09-16T00:10:13.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:13.014 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165075+0000", 2024-09-16T00:10:13.014 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2206999999999999e-05 2024-09-16T00:10:13.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.014 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:13.015 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165084+0000", 2024-09-16T00:10:13.015 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0710000000000001e-06 2024-09-16T00:10:13.015 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.015 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.015 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:13.015 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165157+0000", 2024-09-16T00:10:13.015 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2619000000000004e-05 2024-09-16T00:10:13.015 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.015 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.015 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:13.015 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165249+0000", 2024-09-16T00:10:13.016 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2517999999999997e-05 2024-09-16T00:10:13.016 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.016 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.016 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:13.016 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165360+0000", 2024-09-16T00:10:13.016 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011134899999999999 2024-09-16T00:10:13.016 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.016 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.016 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:13.016 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.169320+0000", 2024-09-16T00:10:13.016 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039593040000000003 2024-09-16T00:10:13.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.017 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.017 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:13.017 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175135+0000", 2024-09-16T00:10:13.017 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005814891 2024-09-16T00:10:13.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.017 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.017 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:13.017 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175199+0000", 2024-09-16T00:10:13.017 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4474999999999995e-05 2024-09-16T00:10:13.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.018 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:13.018 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175239+0000", 2024-09-16T00:10:13.018 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0333000000000002e-05 2024-09-16T00:10:13.018 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.018 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:13.018 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.018 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.018 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11789 3.21 3:8453163e:::100000030b3.00000000:head [write 0~3801088 [4@1194504]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:13.018 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:10:13.018 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 29.697362242000001, 2024-09-16T00:10:13.019 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011097602, 2024-09-16T00:10:13.019 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:13.019 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:13.019 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:13.019 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:13.019 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:13.019 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:13.019 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11789 2024-09-16T00:10:13.020 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.020 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:13.020 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.020 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:13.020 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:10:13.020 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.020 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.020 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.020 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:13.020 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:10:13.020 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.021 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.021 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.021 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:13.021 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251373+0000", 2024-09-16T00:10:13.021 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.544e-06 2024-09-16T00:10:13.021 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.021 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.021 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:13.021 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252710+0000", 2024-09-16T00:10:13.021 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013362929999999999 2024-09-16T00:10:13.021 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.021 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.022 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:13.022 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252730+0000", 2024-09-16T00:10:13.022 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0579999999999999e-05 2024-09-16T00:10:13.022 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.022 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.022 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:13.022 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252741+0000", 2024-09-16T00:10:13.022 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0505e-05 2024-09-16T00:10:13.022 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.022 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.022 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:13.022 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252820+0000", 2024-09-16T00:10:13.023 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9098999999999996e-05 2024-09-16T00:10:13.023 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.023 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.023 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:13.023 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252931+0000", 2024-09-16T00:10:13.023 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011073600000000001 2024-09-16T00:10:13.023 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.023 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.023 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:13.023 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.253101+0000", 2024-09-16T00:10:13.023 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017038000000000001 2024-09-16T00:10:13.023 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:13.024 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.261813+0000", 2024-09-16T00:10:13.024 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087119469999999994 2024-09-16T00:10:13.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:13.024 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262338+0000", 2024-09-16T00:10:13.024 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00052493799999999997 2024-09-16T00:10:13.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:13.025 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262430+0000", 2024-09-16T00:10:13.025 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1626999999999995e-05 2024-09-16T00:10:13.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.025 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.025 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:13.025 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262470+0000", 2024-09-16T00:10:13.025 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9953000000000002e-05 2024-09-16T00:10:13.025 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.025 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:13.025 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.026 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.026 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11890 3.38 3:1e014616:::100000033e9.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:13.026 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:13.026 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 29.377397160000001, 2024-09-16T00:10:13.026 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011833791999999999, 2024-09-16T00:10:13.026 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:13.026 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:13.026 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:13.026 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:13.026 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:13.026 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:13.026 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11890 2024-09-16T00:10:13.027 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.027 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:13.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.027 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:13.027 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:13.027 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.027 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.027 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:13.027 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:13.027 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.027 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.028 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:13.028 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571340+0000", 2024-09-16T00:10:13.028 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.36e-06 2024-09-16T00:10:13.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.028 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:13.028 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572866+0000", 2024-09-16T00:10:13.028 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001525549 2024-09-16T00:10:13.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.028 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:13.029 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572884+0000", 2024-09-16T00:10:13.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7796999999999999e-05 2024-09-16T00:10:13.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.029 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:13.029 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572897+0000", 2024-09-16T00:10:13.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3567000000000001e-05 2024-09-16T00:10:13.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.029 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:13.029 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572970+0000", 2024-09-16T00:10:13.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2751999999999997e-05 2024-09-16T00:10:13.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:13.030 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573053+0000", 2024-09-16T00:10:13.030 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3230000000000001e-05 2024-09-16T00:10:13.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:13.030 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573168+0000", 2024-09-16T00:10:13.030 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114536 2024-09-16T00:10:13.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.031 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:13.031 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.577923+0000", 2024-09-16T00:10:13.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047548720000000003 2024-09-16T00:10:13.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.031 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:13.031 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583081+0000", 2024-09-16T00:10:13.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051587630000000002 2024-09-16T00:10:13.032 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.032 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.032 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:13.032 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583137+0000", 2024-09-16T00:10:13.033 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5259000000000001e-05 2024-09-16T00:10:13.033 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.033 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.033 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:13.033 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583171+0000", 2024-09-16T00:10:13.033 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4106999999999997e-05 2024-09-16T00:10:13.033 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.033 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:13.034 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.034 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:12037 3.9 e87/20 3:928db521:::1000000326d.00000000:head v 87'199, mlcod=87'197)", 2024-09-16T00:10:13.034 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:13.034 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 28.946207209000001, 2024-09-16T00:10:13.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012236153, 2024-09-16T00:10:13.034 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:13.035 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:13.035 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:13.035 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:13.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:13.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:13.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:13.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:13.036 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.036 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.036 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:13.036 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002529+0000", 2024-09-16T00:10:13.036 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4889999999999998e-06 2024-09-16T00:10:13.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.036 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.036 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:13.036 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007064+0000", 2024-09-16T00:10:13.036 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045346220000000003 2024-09-16T00:10:13.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.037 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:13.037 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007093+0000", 2024-09-16T00:10:13.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9009e-05 2024-09-16T00:10:13.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.037 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:13.037 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007109+0000", 2024-09-16T00:10:13.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5846e-05 2024-09-16T00:10:13.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.038 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:13.038 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007196+0000", 2024-09-16T00:10:13.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7507000000000001e-05 2024-09-16T00:10:13.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.038 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:13.038 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007227+0000", 2024-09-16T00:10:13.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0916e-05 2024-09-16T00:10:13.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.038 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:13.039 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014658+0000", 2024-09-16T00:10:13.039 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074305090000000001 2024-09-16T00:10:13.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.039 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.039 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:13.039 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014763+0000", 2024-09-16T00:10:13.039 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010525499999999999 2024-09-16T00:10:13.039 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.039 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:13.039 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12398 3.2e 3:77a11d08:::10000003432.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.707528426, 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012092999, 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:13.040 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12398 2024-09-16T00:10:13.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.041 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:13.041 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.041 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:13.041 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:13.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.042 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:13.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:13.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:13.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241209+0000", 2024-09-16T00:10:13.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2710000000000001e-06 2024-09-16T00:10:13.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:13.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243840+0000", 2024-09-16T00:10:13.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026307919999999999 2024-09-16T00:10:13.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.044 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:13.044 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243872+0000", 2024-09-16T00:10:13.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2699e-05 2024-09-16T00:10:13.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.044 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:13.044 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243887+0000", 2024-09-16T00:10:13.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4348000000000001e-05 2024-09-16T00:10:13.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:13.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243972+0000", 2024-09-16T00:10:13.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5358999999999994e-05 2024-09-16T00:10:13.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:13.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244132+0000", 2024-09-16T00:10:13.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015972499999999999 2024-09-16T00:10:13.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.046 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:13.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253153+0000", 2024-09-16T00:10:13.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0090210280000000004 2024-09-16T00:10:13.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.046 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:13.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253213+0000", 2024-09-16T00:10:13.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0300999999999997e-05 2024-09-16T00:10:13.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.046 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:13.047 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253299+0000", 2024-09-16T00:10:13.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5476000000000006e-05 2024-09-16T00:10:13.047 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.047 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:13.047 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.047 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.047 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12400 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304 [2@815045]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:13.047 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:10:13.047 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.704490692, 2024-09-16T00:10:13.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011140066000000001, 2024-09-16T00:10:13.047 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:13.048 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:13.048 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:13.048 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:13.048 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:13.048 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:13.049 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12400 2024-09-16T00:10:13.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.049 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:13.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.049 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:13.049 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:10:13.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.050 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:13.050 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:10:13.050 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.050 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.050 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.050 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:13.050 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244245+0000", 2024-09-16T00:10:13.050 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.248e-06 2024-09-16T00:10:13.050 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.051 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:13.051 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246467+0000", 2024-09-16T00:10:13.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022217949999999999 2024-09-16T00:10:13.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.051 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:13.051 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246496+0000", 2024-09-16T00:10:13.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9484000000000001e-05 2024-09-16T00:10:13.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:13.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246509+0000", 2024-09-16T00:10:13.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3390000000000001e-05 2024-09-16T00:10:13.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:13.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246586+0000", 2024-09-16T00:10:13.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6692000000000004e-05 2024-09-16T00:10:13.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:13.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246680+0000", 2024-09-16T00:10:13.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3468999999999996e-05 2024-09-16T00:10:13.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:13.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246850+0000", 2024-09-16T00:10:13.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170206 2024-09-16T00:10:13.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.054 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:13.054 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253478+0000", 2024-09-16T00:10:13.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066287530000000002 2024-09-16T00:10:13.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.054 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:13.054 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255318+0000", 2024-09-16T00:10:13.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001839292 2024-09-16T00:10:13.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.054 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:13.054 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255354+0000", 2024-09-16T00:10:13.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5964999999999999e-05 2024-09-16T00:10:13.055 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.055 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:13.055 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255384+0000", 2024-09-16T00:10:13.055 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9771999999999999e-05 2024-09-16T00:10:13.055 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.055 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:13.055 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.055 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.055 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13317 3.38 3:1cda8244:::100000035dc.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:13.055 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:10:13.055 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 25.030136165999998, 2024-09-16T00:10:13.056 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011377063, 2024-09-16T00:10:13.056 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:13.056 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:13.056 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:13.056 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:13.056 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:13.056 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:13.056 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13317 2024-09-16T00:10:13.056 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.056 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:13.056 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.056 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:13.056 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:10:13.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:13.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:10:13.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:13.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918602+0000", 2024-09-16T00:10:13.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8160000000000004e-06 2024-09-16T00:10:13.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.058 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:13.058 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920364+0000", 2024-09-16T00:10:13.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017623630000000001 2024-09-16T00:10:13.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.058 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:13.058 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920391+0000", 2024-09-16T00:10:13.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7248000000000001e-05 2024-09-16T00:10:13.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.058 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:13.058 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920405+0000", 2024-09-16T00:10:13.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3969e-05 2024-09-16T00:10:13.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.059 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:13.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920486+0000", 2024-09-16T00:10:13.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0737999999999998e-05 2024-09-16T00:10:13.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.059 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:13.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920594+0000", 2024-09-16T00:10:13.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108269 2024-09-16T00:10:13.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.059 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:13.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920730+0000", 2024-09-16T00:10:13.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000135229 2024-09-16T00:10:13.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:13.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.928847+0000", 2024-09-16T00:10:13.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081178299999999995 2024-09-16T00:10:13.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:13.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929862+0000", 2024-09-16T00:10:13.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010147540000000001 2024-09-16T00:10:13.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:13.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929923+0000", 2024-09-16T00:10:13.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0452999999999999e-05 2024-09-16T00:10:13.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:13.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929975+0000", 2024-09-16T00:10:13.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2394e-05 2024-09-16T00:10:13.061 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.061 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:13.061 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.061 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:13326 3.9 e88/20 3:92f796cf:::1000000350f.00000000:head v 88'214, mlcod=88'212)", 2024-09-16T00:10:13.061 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:10:13.061 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 24.987615375000001, 2024-09-16T00:10:13.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01100597, 2024-09-16T00:10:13.062 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:13.062 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:13.062 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:13.062 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:13.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.062 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:13.062 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:10:13.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.062 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:13.062 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:10:13.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:13.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961121+0000", 2024-09-16T00:10:13.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.815e-06 2024-09-16T00:10:13.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:13.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965747+0000", 2024-09-16T00:10:13.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0046260570000000003 2024-09-16T00:10:13.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:13.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965780+0000", 2024-09-16T00:10:13.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2851000000000001e-05 2024-09-16T00:10:13.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:13.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965788+0000", 2024-09-16T00:10:13.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0180000000000001e-06 2024-09-16T00:10:13.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:13.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965859+0000", 2024-09-16T00:10:13.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1318999999999999e-05 2024-09-16T00:10:13.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:13.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965886+0000", 2024-09-16T00:10:13.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7509999999999999e-05 2024-09-16T00:10:13.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:13.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.972066+0000", 2024-09-16T00:10:13.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061800759999999996 2024-09-16T00:10:13.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:13.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.972125+0000", 2024-09-16T00:10:13.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8323999999999999e-05 2024-09-16T00:10:13.066 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.066 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:13.066 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.066 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13942 3.22 3:44ed4521:::100000036d5.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:13.066 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:10:13.066 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.026250062999999, 2024-09-16T00:10:13.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011341061, 2024-09-16T00:10:13.066 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:13.066 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:13.066 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:13.066 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:13.066 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:13.067 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:13.067 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13942 2024-09-16T00:10:13.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.067 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:13.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:13.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:10:13.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:13.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:10:13.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.068 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.068 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:13.068 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922487+0000", 2024-09-16T00:10:13.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0180000000000002e-06 2024-09-16T00:10:13.068 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.068 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.068 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:13.068 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923720+0000", 2024-09-16T00:10:13.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012333470000000001 2024-09-16T00:10:13.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:13.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923749+0000", 2024-09-16T00:10:13.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8748999999999999e-05 2024-09-16T00:10:13.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:13.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923770+0000", 2024-09-16T00:10:13.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1189999999999999e-05 2024-09-16T00:10:13.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:13.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923868+0000", 2024-09-16T00:10:13.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7675000000000002e-05 2024-09-16T00:10:13.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:13.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.924047+0000", 2024-09-16T00:10:13.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017935100000000001 2024-09-16T00:10:13.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:10:13.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.924223+0000", 2024-09-16T00:10:13.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000175661 2024-09-16T00:10:13.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:13.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.931544+0000", 2024-09-16T00:10:13.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073204589999999996 2024-09-16T00:10:13.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:13.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933692+0000", 2024-09-16T00:10:13.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021488760000000001 2024-09-16T00:10:13.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:13.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933774+0000", 2024-09-16T00:10:13.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1855999999999997e-05 2024-09-16T00:10:13.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:13.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933825+0000", 2024-09-16T00:10:13.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0878999999999999e-05 2024-09-16T00:10:13.072 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.072 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:13.072 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.073 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14867 3.1f 3:faedf583:::1000000393b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:13.073 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:10:13.073 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.201760326999999, 2024-09-16T00:10:13.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011128677999999999, 2024-09-16T00:10:13.073 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:13.073 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:13.073 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:13.073 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:13.073 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:13.073 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:13.073 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14867 2024-09-16T00:10:13.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.074 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:13.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:13.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:10:13.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:13.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:10:13.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:13.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746976+0000", 2024-09-16T00:10:13.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1169999999999998e-06 2024-09-16T00:10:13.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:13.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748348+0000", 2024-09-16T00:10:13.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013721860000000001 2024-09-16T00:10:13.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:13.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748371+0000", 2024-09-16T00:10:13.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2938999999999998e-05 2024-09-16T00:10:13.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:13.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748381+0000", 2024-09-16T00:10:13.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5300000000000002e-06 2024-09-16T00:10:13.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:13.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748458+0000", 2024-09-16T00:10:13.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7389000000000001e-05 2024-09-16T00:10:13.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.077 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.077 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:13.077 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748626+0000", 2024-09-16T00:10:13.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016757499999999999 2024-09-16T00:10:13.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.077 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.077 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:13.077 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748813+0000", 2024-09-16T00:10:13.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018743399999999999 2024-09-16T00:10:13.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:13.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.753998+0000", 2024-09-16T00:10:13.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051851140000000002 2024-09-16T00:10:13.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:13.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.757990+0000", 2024-09-16T00:10:13.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039914169999999997 2024-09-16T00:10:13.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:13.079 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.758054+0000", 2024-09-16T00:10:13.079 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4513999999999999e-05 2024-09-16T00:10:13.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:13.080 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.758103+0000", 2024-09-16T00:10:13.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8463e-05 2024-09-16T00:10:13.080 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.080 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:13.080 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.080 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15003 3.21 3:8673bb5f:::100000030ed.00000000:head [read 2097152~2097152 [2@1395527]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:13.080 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:13.080 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.780338331999999, 2024-09-16T00:10:13.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058433321000000003, 2024-09-16T00:10:13.081 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:13.081 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:13.081 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:10:13.081 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:13.081 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:13.081 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:13.081 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15003 2024-09-16T00:10:13.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.081 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:13.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:13.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:13.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:13.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:13.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:13.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168397+0000", 2024-09-16T00:10:13.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3900000000000004e-07 2024-09-16T00:10:13.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:13.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168401+0000", 2024-09-16T00:10:13.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4050000000000002e-06 2024-09-16T00:10:13.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:13.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168411+0000", 2024-09-16T00:10:13.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6539999999999994e-06 2024-09-16T00:10:13.084 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.084 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.084 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:13.084 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168417+0000", 2024-09-16T00:10:13.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6719999999999998e-06 2024-09-16T00:10:13.084 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.084 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.084 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:13.084 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168467+0000", 2024-09-16T00:10:13.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9815999999999999e-05 2024-09-16T00:10:13.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:13.085 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168535+0000", 2024-09-16T00:10:13.085 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7669999999999994e-05 2024-09-16T00:10:13.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:13.085 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.226829+0000", 2024-09-16T00:10:13.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058294264999999998 2024-09-16T00:10:13.086 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.086 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:13.086 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.086 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15005 3.1f 3:f8460f10:::10000003370.00000000:head [read 1048576~1048576 [2@4194304]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:13.086 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:13.086 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.778657620000001, 2024-09-16T00:10:13.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.055844088, 2024-09-16T00:10:13.087 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:13.087 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:13.087 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:10:13.087 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:13.087 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:13.087 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:13.087 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15005 2024-09-16T00:10:13.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.087 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:13.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:13.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:13.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:13.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:13.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:13.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:13.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170078+0000", 2024-09-16T00:10:13.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.235e-06 2024-09-16T00:10:13.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:13.089 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170084+0000", 2024-09-16T00:10:13.089 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9739999999999999e-06 2024-09-16T00:10:13.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:13.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225526+0000", 2024-09-16T00:10:13.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.055441777999999997 2024-09-16T00:10:13.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.090 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.090 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:13.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225542+0000", 2024-09-16T00:10:13.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6606999999999999e-05 2024-09-16T00:10:13.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.090 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.090 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:13.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225668+0000", 2024-09-16T00:10:13.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012631 2024-09-16T00:10:13.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.091 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:13.091 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225744+0000", 2024-09-16T00:10:13.091 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5794e-05 2024-09-16T00:10:13.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:13.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:13.091 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:13.091 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225921+0000", 2024-09-16T00:10:13.091 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017639000000000001 2024-09-16T00:10:13.091 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.091 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:13.091 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.092 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:13.092 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:13.092 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:13.093 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:10:13.094 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:13.092+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:13.145 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:10:13.193 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:13.193+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:13.247 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:13.247 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:13.247 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:13.247 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:13.258 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:10:13.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.283+0000 7f4c1164c640 1 Processor -- start 2024-09-16T00:10:13.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.283+0000 7f4c1164c640 1 -- start start 2024-09-16T00:10:13.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.284+0000 7f4c1164c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4c0c06dfe0 0x7f4c0c1515e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.284+0000 7f4c1164c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c0c158520 con 0x7f4c0c06cea0 2024-09-16T00:10:13.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.284+0000 7f4c1164c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c0c073dc0 con 0x7f4c0c151b20 2024-09-16T00:10:13.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.284+0000 7f4c1164c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4c0c151f50 con 0x7f4c0c06dfe0 2024-09-16T00:10:13.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.284+0000 7f4c0a7fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4c0c06dfe0 0x7f4c0c1515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.284+0000 7f4c0affd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4c0c06cea0 0x7f4c0c06d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45746/0 (socket says 172.21.15.139:45746) 2024-09-16T00:10:13.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.284+0000 7f4c0affd640 1 -- 172.21.15.139:0/1442893935 learned_addr learned my addr 172.21.15.139:0/1442893935 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:13.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.284+0000 7f4c09ffb640 1 -- 172.21.15.139:0/1442893935 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3728079435 0 0) 0x7f4c0c073dc0 con 0x7f4c0c151b20 2024-09-16T00:10:13.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.284+0000 7f4c09ffb640 1 -- 172.21.15.139:0/1442893935 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bf4003520 con 0x7f4c0c151b20 2024-09-16T00:10:13.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.284+0000 7f4c09ffb640 1 -- 172.21.15.139:0/1442893935 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2317878641 0 0) 0x7f4c0c158520 con 0x7f4c0c06cea0 2024-09-16T00:10:13.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.284+0000 7f4c09ffb640 1 -- 172.21.15.139:0/1442893935 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c0c073dc0 con 0x7f4c0c06cea0 2024-09-16T00:10:13.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.284+0000 7f4c0a7fc640 1 -- 172.21.15.139:0/1442893935 >> v1:172.21.15.164:6789/0 conn(0x7f4c0c151b20 legacy=0x7f4c0c072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.284+0000 7f4c0a7fc640 1 -- 172.21.15.139:0/1442893935 >> v1:172.21.15.139:6789/0 conn(0x7f4c0c06cea0 legacy=0x7f4c0c06d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.284+0000 7f4c0a7fc640 1 -- 172.21.15.139:0/1442893935 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c0740a0 con 0x7f4c0c06dfe0 2024-09-16T00:10:13.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.285+0000 7f4c0a7fc640 1 --2- 172.21.15.139:0/1442893935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c0c06dfe0 0x7f4c0c1515e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4c000097a0 tx=0x7f4c0002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ce064c7ac9bf4a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.285+0000 7f4c09ffb640 1 -- 172.21.15.139:0/1442893935 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00037070 con 0x7f4c0c06dfe0 2024-09-16T00:10:13.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.285+0000 7f4c09ffb640 1 -- 172.21.15.139:0/1442893935 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c00032640 con 0x7f4c0c06dfe0 2024-09-16T00:10:13.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.285+0000 7f4c09ffb640 1 -- 172.21.15.139:0/1442893935 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c00032a10 con 0x7f4c0c06dfe0 2024-09-16T00:10:13.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.286+0000 7f4c1164c640 1 -- 172.21.15.139:0/1442893935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c0c06dfe0 msgr2=0x7f4c0c1515e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.286+0000 7f4c1164c640 1 --2- 172.21.15.139:0/1442893935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c0c06dfe0 0x7f4c0c1515e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4c000097a0 tx=0x7f4c0002a2c0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.286+0000 7f4c1164c640 1 -- 172.21.15.139:0/1442893935 shutdown_connections 2024-09-16T00:10:13.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.286+0000 7f4c1164c640 1 --2- 172.21.15.139:0/1442893935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c0c06dfe0 0x7f4c0c1515e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.286+0000 7f4c1164c640 1 -- 172.21.15.139:0/1442893935 >> 172.21.15.139:0/1442893935 conn(0x7f4c0c0c9af0 msgr2=0x7f4c0c0c9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:13.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.286+0000 7f4c1164c640 1 -- 172.21.15.139:0/1442893935 shutdown_connections 2024-09-16T00:10:13.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.286+0000 7f4c1164c640 1 -- 172.21.15.139:0/1442893935 wait complete. 2024-09-16T00:10:13.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.287+0000 7f4c1164c640 1 Processor -- start 2024-09-16T00:10:13.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.287+0000 7f4c1164c640 1 -- start start 2024-09-16T00:10:13.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.288+0000 7f4c1164c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c0c06cea0 0x7f4c0c0c2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.288+0000 7f4c1164c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c0c06dfe0 0x7f4c0c1f0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.288+0000 7f4c1164c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c0c151b20 0x7f4c0c166730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.288+0000 7f4c1164c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c072a70 con 0x7f4c0c151b20 2024-09-16T00:10:13.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.288+0000 7f4c1164c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c0caa30 con 0x7f4c0c06cea0 2024-09-16T00:10:13.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.288+0000 7f4c1164c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4c0c06e410 con 0x7f4c0c06dfe0 2024-09-16T00:10:13.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.288+0000 7f4c0affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c0c06cea0 0x7f4c0c0c2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.288+0000 7f4c0affd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c0c06cea0 0x7f4c0c0c2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:42320/0 (socket says 172.21.15.139:42320) 2024-09-16T00:10:13.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.288+0000 7f4c0affd640 1 -- 172.21.15.139:0/820592953 learned_addr learned my addr 172.21.15.139:0/820592953 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:13.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.288+0000 7f4c0b7fe640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c0c151b20 0x7f4c0c166730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.288+0000 7f4c0a7fc640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c0c06dfe0 0x7f4c0c1f0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.289+0000 7f4c0affd640 1 -- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c0c06dfe0 msgr2=0x7f4c0c1f0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.289+0000 7f4c0affd640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4c0c06dfe0 0x7f4c0c1f0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.289+0000 7f4c0affd640 1 -- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c0c151b20 msgr2=0x7f4c0c166730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.289+0000 7f4c0affd640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c0c151b20 0x7f4c0c166730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.289+0000 7f4c0affd640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c1f0310 con 0x7f4c0c06cea0 2024-09-16T00:10:13.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.289+0000 7f4c0affd640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c0c06cea0 0x7f4c0c0c2940 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00c500 tx=0x7f4bfc00c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.289+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc017070 con 0x7f4c0c06cea0 2024-09-16T00:10:13.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.289+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bfc005ce0 con 0x7f4c0c06cea0 2024-09-16T00:10:13.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.289+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0c0c7370 con 0x7f4c0c06cea0 2024-09-16T00:10:13.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.289+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc007330 con 0x7f4c0c06cea0 2024-09-16T00:10:13.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.289+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c0c1f4610 con 0x7f4c0c06cea0 2024-09-16T00:10:13.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.290+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f4c0c075fc0 con 0x7f4c0c06cea0 2024-09-16T00:10:13.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.291+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4bfc01a070 con 0x7f4c0c06cea0 2024-09-16T00:10:13.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.291+0000 7f4bf37fe640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4bf806c2e0 0x7f4bf806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.291+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5973+0+0 (secure 0 0 0) 0x7f4bfc08a8e0 con 0x7f4c0c06cea0 2024-09-16T00:10:13.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.291+0000 7f4c0a7fc640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4bf806c2e0 0x7f4bf806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.291+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v104) ==== 54+0+16 (secure 0 0 0) 0x7f4c0c075fc0 con 0x7f4c0c06cea0 2024-09-16T00:10:13.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.291+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f4c0c1f04e0 0x7f4c0c0c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.291+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c0c0c41d0 con 0x7f4c0c1f04e0 2024-09-16T00:10:13.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.292+0000 7f4c0b7fe640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f4c0c1f04e0 0x7f4c0c0c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.292+0000 7f4c0a7fc640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4bf806c2e0 0x7f4bf806e7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4c00034000 tx=0x7f4c0002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.292+0000 7f4c0b7fe640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f4c0c1f04e0 0x7f4c0c0c6260 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.293+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c0c0c41d0 con 0x7f4c0c1f04e0 2024-09-16T00:10:13.295 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:13.295+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:13.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.309+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4c0c1437f0 con 0x7f4c0c1f04e0 2024-09-16T00:10:13.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.310+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c0c1437f0 con 0x7f4c0c1f04e0 2024-09-16T00:10:13.311 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:13.312 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:13.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.311+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f4c0c0c34d0 0x7f4c0c164ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.311+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f4c0c166c70 con 0x7f4c0c0c34d0 2024-09-16T00:10:13.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.311+0000 7f4c0affd640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f4c0c0c34d0 0x7f4c0c164ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.311+0000 7f4c0affd640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f4c0c0c34d0 0x7f4c0c164ff0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.312+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c0c166c70 con 0x7f4c0c0c34d0 2024-09-16T00:10:13.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.327+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4c0c1f0940 con 0x7f4c0c0c34d0 2024-09-16T00:10:13.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.328+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c0c1f0940 con 0x7f4c0c0c34d0 2024-09-16T00:10:13.329 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:13.330 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:13.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.329+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f4c0c165a40 0x7f4c0c062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.329+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f4c0c0c3a20 con 0x7f4c0c165a40 2024-09-16T00:10:13.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.329+0000 7f4c0a7fc640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f4c0c165a40 0x7f4c0c062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.329+0000 7f4c0a7fc640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f4c0c165a40 0x7f4c0c062630 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.330+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c0c0c3a20 con 0x7f4c0c165a40 2024-09-16T00:10:13.346 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.345+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4c0c062b70 con 0x7f4c0c165a40 2024-09-16T00:10:13.348 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.347+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c0c062b70 con 0x7f4c0c165a40 2024-09-16T00:10:13.348 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:13.349 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:13.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.347+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f4c0c06e890 0x7f4c0c06ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.347+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f4c0c06f230 con 0x7f4c0c06e890 2024-09-16T00:10:13.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.348+0000 7f4c0b7fe640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f4c0c06e890 0x7f4c0c06ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.348+0000 7f4c0b7fe640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f4c0c06e890 0x7f4c0c06ecf0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.349 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.349+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c0c06f230 con 0x7f4c0c06e890 2024-09-16T00:10:13.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.363+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4c0c06f230 con 0x7f4c0c06e890 2024-09-16T00:10:13.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.365+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c0c06f230 con 0x7f4c0c06e890 2024-09-16T00:10:13.366 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:13.367 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:13.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.365+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f4c0c02cb50 0x7f4c0c02cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.365+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f4c0c06f740 con 0x7f4c0c02cb50 2024-09-16T00:10:13.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.365+0000 7f4c0affd640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f4c0c02cb50 0x7f4c0c02cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.366+0000 7f4c0affd640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f4c0c02cb50 0x7f4c0c02cf90 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.366+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c0c06f740 con 0x7f4c0c02cb50 2024-09-16T00:10:13.371 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:13.371 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:13.382 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:10:13.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.382+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4c0c06f740 con 0x7f4c0c02cb50 2024-09-16T00:10:13.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.384+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c0c06f740 con 0x7f4c0c02cb50 2024-09-16T00:10:13.385 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:13.386 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:13.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.384+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f4c0c204c00 0x7f4c0c206ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.384+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f4c0c02d9e0 con 0x7f4c0c204c00 2024-09-16T00:10:13.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.384+0000 7f4c0a7fc640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f4c0c204c00 0x7f4c0c206ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.385+0000 7f4c0a7fc640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f4c0c204c00 0x7f4c0c206ff0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.385+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c0c02d9e0 con 0x7f4c0c204c00 2024-09-16T00:10:13.396 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:13.396+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:13.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.399+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4c0c02d9e0 con 0x7f4c0c204c00 2024-09-16T00:10:13.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.401+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c0c02d9e0 con 0x7f4c0c204c00 2024-09-16T00:10:13.402 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:13.403 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:13.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.401+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f4c0c207a40 0x7f4c0c209e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.401+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f4c0c02d9e0 con 0x7f4c0c207a40 2024-09-16T00:10:13.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.401+0000 7f4c0b7fe640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f4c0c207a40 0x7f4c0c209e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.403 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.402+0000 7f4c0b7fe640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f4c0c207a40 0x7f4c0c209e30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.404 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.403+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c0c02d9e0 con 0x7f4c0c207a40 2024-09-16T00:10:13.418 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.417+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4c0c02d9e0 con 0x7f4c0c207a40 2024-09-16T00:10:13.420 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.420+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c0c02d9e0 con 0x7f4c0c207a40 2024-09-16T00:10:13.421 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:13.421 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:13.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.420+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f4c0c20a880 0x7f4c0c20cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:13.421 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.420+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f4c0c20faf0 con 0x7f4c0c20a880 2024-09-16T00:10:13.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.420+0000 7f4c0affd640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f4c0c20a880 0x7f4c0c20cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:13.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.420+0000 7f4c0affd640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f4c0c20a880 0x7f4c0c20cc70 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:13.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.421+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c0c20faf0 con 0x7f4c0c20a880 2024-09-16T00:10:13.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.435+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4c0c20d480 con 0x7f4c0c20a880 2024-09-16T00:10:13.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4bf37fe640 1 -- 172.21.15.139:0/820592953 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c0c20d480 con 0x7f4c0c20a880 2024-09-16T00:10:13.439 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:10:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:10:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f4c0c1f04e0 msgr2=0x7f4c0c0c6260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f4c0c1f04e0 0x7f4c0c0c6260 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f4c0c0c34d0 msgr2=0x7f4c0c164ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f4c0c0c34d0 0x7f4c0c164ff0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f4c0c165a40 msgr2=0x7f4c0c062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f4c0c165a40 0x7f4c0c062630 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f4c0c06e890 msgr2=0x7f4c0c06ecf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f4c0c06e890 0x7f4c0c06ecf0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f4c0c02cb50 msgr2=0x7f4c0c02cf90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c0b7fe640 1 -- 172.21.15.139:0/820592953 reap_dead start 2024-09-16T00:10:13.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f4c0c02cb50 0x7f4c0c02cf90 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c0b7fe640 1 -- 172.21.15.139:0/820592953 reap_dead start 2024-09-16T00:10:13.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f4c0c204c00 msgr2=0x7f4c0c206ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.442 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f4c0c204c00 0x7f4c0c206ff0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f4c0c207a40 msgr2=0x7f4c0c209e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f4c0c207a40 0x7f4c0c209e30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f4c0c20a880 msgr2=0x7f4c0c20cc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f4c0c20a880 0x7f4c0c20cc70 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4bf806c2e0 msgr2=0x7f4bf806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4bf806c2e0 0x7f4bf806e7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4c00034000 tx=0x7f4c0002e000 comp rx=0 tx=0).stop 2024-09-16T00:10:13.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c0c06cea0 msgr2=0x7f4c0c0c2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:13.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c0b7fe640 1 -- 172.21.15.139:0/820592953 reap_dead start 2024-09-16T00:10:13.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c0c06cea0 0x7f4c0c0c2940 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00c500 tx=0x7f4bfc00c9c0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 shutdown_connections 2024-09-16T00:10:13.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 --2- 172.21.15.139:0/820592953 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4c0c06cea0 0x7f4c0c0c2940 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:13.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 >> 172.21.15.139:0/820592953 conn(0x7f4c0c0c9af0 msgr2=0x7f4c0c06d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:13.443 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 shutdown_connections 2024-09-16T00:10:13.444 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:13.438+0000 7f4c1164c640 1 -- 172.21.15.139:0/820592953 wait complete. 2024-09-16T00:10:13.493 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:13.493 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:13.497 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:13.496+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:13.502 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:10:13.598 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:13.597+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:13.610 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:13.610 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:13.619 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:10:13.700 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:13.699+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:13.736 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:13.736 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:13.745 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:10:13.800 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:13.799+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:13.854 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:13.854 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:13.863 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:10:13.904 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:13.904+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:14.002 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:14.001+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:14.040 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:14.049 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:10:14.103 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:14.102+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:14.157 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:14.166 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:10:14.204 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:14.203+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:14.276 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:14.277 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:14.286 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:10:14.304 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:14.304+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:14.348 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T00:10:14.394 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:14.394 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:14.403 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:10:14.404 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:14.404+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:14.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.452+0000 7f58b268d640 1 Processor -- start 2024-09-16T00:10:14.453 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.452+0000 7f58b268d640 1 -- start start 2024-09-16T00:10:14.454 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.452+0000 7f58b268d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f58ac0c6940 0x7f58ac15c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.452+0000 7f58b268d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ac152bb0 0x7f58ac06f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.452+0000 7f58b268d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58ac15cb40 con 0x7f58ac0c6d20 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.452+0000 7f58b268d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f58ac06c3c0 con 0x7f58ac0c6940 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.452+0000 7f58b268d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f58ac0c7ee0 con 0x7f58ac152bb0 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.453+0000 7f58abfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f58ac0c6940 0x7f58ac15c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.453+0000 7f58abfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f58ac0c6940 0x7f58ac15c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49224/0 (socket says 172.21.15.139:49224) 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.453+0000 7f58abfff640 1 -- 172.21.15.139:0/2338967789 learned_addr learned my addr 172.21.15.139:0/2338967789 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.453+0000 7f58b0c2a640 1 --2- 172.21.15.139:0/2338967789 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ac152bb0 0x7f58ac06f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.453+0000 7f58abfff640 1 -- 172.21.15.139:0/2338967789 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ac152bb0 msgr2=0x7f58ac06f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.453+0000 7f58abfff640 1 --2- 172.21.15.139:0/2338967789 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ac152bb0 0x7f58ac06f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.453+0000 7f58abfff640 1 -- 172.21.15.139:0/2338967789 >> v1:172.21.15.164:6789/0 conn(0x7f58ac0c6d20 legacy=0x7f58ac1524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.453+0000 7f58abfff640 1 -- 172.21.15.139:0/2338967789 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ac06bb20 con 0x7f58ac0c6940 2024-09-16T00:10:14.455 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.454+0000 7f58abfff640 1 --2- 172.21.15.139:0/2338967789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ac0c6940 0x7f58ac15c600 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f589c009870 tx=0x7f589c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3275260f08576464 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.454+0000 7f58aaffd640 1 -- 172.21.15.139:0/2338967789 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c037070 con 0x7f58ac0c6940 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.454+0000 7f58aaffd640 1 -- 172.21.15.139:0/2338967789 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f589c02ae50 con 0x7f58ac0c6940 2024-09-16T00:10:14.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.454+0000 7f58aaffd640 1 -- 172.21.15.139:0/2338967789 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c032860 con 0x7f58ac0c6940 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.454+0000 7f58b268d640 1 -- 172.21.15.139:0/2338967789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ac0c6940 msgr2=0x7f58ac15c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.454+0000 7f58b268d640 1 --2- 172.21.15.139:0/2338967789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ac0c6940 0x7f58ac15c600 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f589c009870 tx=0x7f589c02a300 comp rx=0 tx=0).stop 2024-09-16T00:10:14.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.454+0000 7f58b268d640 1 -- 172.21.15.139:0/2338967789 shutdown_connections 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.454+0000 7f58b268d640 1 --2- 172.21.15.139:0/2338967789 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ac152bb0 0x7f58ac06f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.454+0000 7f58b268d640 1 --2- 172.21.15.139:0/2338967789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ac0c6940 0x7f58ac15c600 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.454+0000 7f58b268d640 1 -- 172.21.15.139:0/2338967789 >> 172.21.15.139:0/2338967789 conn(0x7f58ac070460 msgr2=0x7f58ac070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.454+0000 7f58b268d640 1 -- 172.21.15.139:0/2338967789 shutdown_connections 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.454+0000 7f58b268d640 1 -- 172.21.15.139:0/2338967789 wait complete. 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.455+0000 7f58b268d640 1 Processor -- start 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.456+0000 7f58b268d640 1 -- start start 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.456+0000 7f58b268d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ac0c6940 0x7f58ac1f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.456+0000 7f58b268d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58ac0c6d20 0x7f58ac147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.456+0000 7f58b268d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ac152bb0 0x7f58ac1eea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.456+0000 7f58b268d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac06efc0 con 0x7f58ac0c6940 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.456+0000 7f58b268d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f58ac06c0e0 con 0x7f58ac0c6d20 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.456+0000 7f58b268d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f58ac06d9b0 con 0x7f58ac152bb0 2024-09-16T00:10:14.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.456+0000 7f58abfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ac0c6940 0x7f58ac1f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.457+0000 7f58abfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ac0c6940 0x7f58ac1f2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49234/0 (socket says 172.21.15.139:49234) 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.457+0000 7f58abfff640 1 -- 172.21.15.139:0/1072648649 learned_addr learned my addr 172.21.15.139:0/1072648649 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.457+0000 7f58b0c2a640 1 --2- 172.21.15.139:0/1072648649 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ac152bb0 0x7f58ac1eea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.457+0000 7f58ab7fe640 1 --2- 172.21.15.139:0/1072648649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58ac0c6d20 0x7f58ac147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.457+0000 7f58abfff640 1 -- 172.21.15.139:0/1072648649 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ac152bb0 msgr2=0x7f58ac1eea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.457+0000 7f58abfff640 1 --2- 172.21.15.139:0/1072648649 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ac152bb0 0x7f58ac1eea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.457+0000 7f58abfff640 1 -- 172.21.15.139:0/1072648649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58ac0c6d20 msgr2=0x7f58ac147fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.457+0000 7f58abfff640 1 --2- 172.21.15.139:0/1072648649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58ac0c6d20 0x7f58ac147fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.457+0000 7f58abfff640 1 -- 172.21.15.139:0/1072648649 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ac0ca470 con 0x7f58ac0c6940 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.457+0000 7f58abfff640 1 --2- 172.21.15.139:0/1072648649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ac0c6940 0x7f58ac1f2310 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f589c000c00 tx=0x7f589c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.457+0000 7f58a8ff9640 1 -- 172.21.15.139:0/1072648649 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c041070 con 0x7f58ac0c6940 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.457+0000 7f58a8ff9640 1 -- 172.21.15.139:0/1072648649 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f589c03c660 con 0x7f58ac0c6940 2024-09-16T00:10:14.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.457+0000 7f58b268d640 1 -- 172.21.15.139:0/1072648649 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ac1f2850 con 0x7f58ac0c6940 2024-09-16T00:10:14.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.457+0000 7f58a8ff9640 1 -- 172.21.15.139:0/1072648649 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c037040 con 0x7f58ac0c6940 2024-09-16T00:10:14.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.457+0000 7f58b268d640 1 -- 172.21.15.139:0/1072648649 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ac0c9bd0 con 0x7f58ac0c6940 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.458+0000 7f58b268d640 1 -- 172.21.15.139:0/1072648649 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58ac14c9b0 con 0x7f58ac0c6940 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.459+0000 7f58a8ff9640 1 -- 172.21.15.139:0/1072648649 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f589c044070 con 0x7f58ac0c6940 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.459+0000 7f58a8ff9640 1 --2- 172.21.15.139:0/1072648649 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f589806c2b0 0x7f589806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.459+0000 7f58a8ff9640 1 -- 172.21.15.139:0/1072648649 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5973+0+0 (secure 0 0 0) 0x7f589c0abf90 con 0x7f58ac0c6940 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.459+0000 7f58ab7fe640 1 --2- 172.21.15.139:0/1072648649 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f589806c2b0 0x7f589806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:14.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.460+0000 7f58ab7fe640 1 --2- 172.21.15.139:0/1072648649 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f589806c2b0 0x7f589806e770 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f58ac1e5e10 tx=0x7f58a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:14.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.464+0000 7f58a8ff9640 1 -- 172.21.15.139:0/1072648649 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f58ac14c9b0 con 0x7f58ac0c6940 2024-09-16T00:10:14.506 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:14.506+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:14.509 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:14.509 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:14.518 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:10:14.607 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:14.606+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:14.626 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:14.626 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:14.636 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:10:14.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:14.667+0000 7f58b268d640 1 -- 172.21.15.139:0/1072648649 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f58ac0c79c0 con 0x7f58ac0c6940 2024-09-16T00:10:14.710 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:14.709+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:14.744 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:14.744 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:14.754 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:10:14.838 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:14.838+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:14.902 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:14.902 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:14.912 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:14.912+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:14.915 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:10:15.013 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:15.013+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:15.102 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:15.102 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:15.110 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:10:15.155 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:15.154+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:15.215 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:15.215+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:15.223 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:15.224 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:15.233 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:10:15.251 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-09-16T00:10:15.252 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:10:15.252 INFO:tasks.thrashosds.thrasher:Reweighting osd 1 to 0.5229187413364552 2024-09-16T00:10:15.252 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 0.5229187413364552 2024-09-16T00:10:15.316 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:15.315+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:15.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.332+0000 7f58a8ff9640 1 -- 172.21.15.139:0/1072648649 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v105) ==== 101+0+0 (secure 0 0 0) 0x7f589c07a0c0 con 0x7f58ac0c6940 2024-09-16T00:10:15.333 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is unset 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.336+0000 7f58b268d640 1 -- 172.21.15.139:0/1072648649 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f589806c2b0 msgr2=0x7f589806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.336+0000 7f58b268d640 1 --2- 172.21.15.139:0/1072648649 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f589806c2b0 0x7f589806e770 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f58ac1e5e10 tx=0x7f58a0009040 comp rx=0 tx=0).stop 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.336+0000 7f58b268d640 1 -- 172.21.15.139:0/1072648649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ac0c6940 msgr2=0x7f58ac1f2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.336+0000 7f58b268d640 1 --2- 172.21.15.139:0/1072648649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ac0c6940 0x7f58ac1f2310 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f589c000c00 tx=0x7f589c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.336+0000 7f58b268d640 1 -- 172.21.15.139:0/1072648649 shutdown_connections 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.336+0000 7f58b268d640 1 --2- 172.21.15.139:0/1072648649 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f589806c2b0 0x7f589806e770 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.336+0000 7f58b268d640 1 --2- 172.21.15.139:0/1072648649 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f58ac152bb0 0x7f58ac1eea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.336+0000 7f58b268d640 1 --2- 172.21.15.139:0/1072648649 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f58ac0c6d20 0x7f58ac147fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.336+0000 7f58b268d640 1 --2- 172.21.15.139:0/1072648649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58ac0c6940 0x7f58ac1f2310 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.336+0000 7f58b268d640 1 -- 172.21.15.139:0/1072648649 >> 172.21.15.139:0/1072648649 conn(0x7f58ac070460 msgr2=0x7f58ac150a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:15.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.336+0000 7f58b268d640 1 -- 172.21.15.139:0/1072648649 shutdown_connections 2024-09-16T00:10:15.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.337+0000 7f58b268d640 1 -- 172.21.15.139:0/1072648649 wait complete. 2024-09-16T00:10:15.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.388+0000 7fd5a62d7640 1 Processor -- start 2024-09-16T00:10:15.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.388+0000 7fd5a62d7640 1 -- start start 2024-09-16T00:10:15.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.388+0000 7fd5a62d7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5a0151be0 0x7fd5a00c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.388+0000 7fd5a62d7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd5a00c63c0 0x7fd5a015b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.388+0000 7fd5a62d7640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5a015ad00 con 0x7fd5a0151800 2024-09-16T00:10:15.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.388+0000 7fd5a62d7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd5a00c5010 con 0x7fd5a00c63c0 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.388+0000 7fd5a62d7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a00c9c40 con 0x7fd5a0151be0 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.389+0000 7fd5a4874640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd5a00c63c0 0x7fd5a015b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.389+0000 7fd5a4874640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd5a00c63c0 0x7fd5a015b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49242/0 (socket says 172.21.15.139:49242) 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.389+0000 7fd5a4874640 1 -- 172.21.15.139:0/61938685 learned_addr learned my addr 172.21.15.139:0/61938685 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.389+0000 7fd59ffff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd5a0151800 0x7fd5a00c3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:44136/0 (socket says 172.21.15.139:44136) 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.389+0000 7fd59f7fe640 1 --2- 172.21.15.139:0/61938685 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5a0151be0 0x7fd5a00c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.389+0000 7fd5a4874640 1 -- 172.21.15.139:0/61938685 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5a0151be0 msgr2=0x7fd5a00c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.389+0000 7fd5a4874640 1 --2- 172.21.15.139:0/61938685 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5a0151be0 0x7fd5a00c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.389+0000 7fd5a4874640 1 -- 172.21.15.139:0/61938685 >> v1:172.21.15.164:6789/0 conn(0x7fd5a0151800 legacy=0x7fd5a00c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.389+0000 7fd59effd640 1 -- 172.21.15.139:0/61938685 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3119254844 0 0) 0x7fd5a015ad00 con 0x7fd5a0151800 2024-09-16T00:10:15.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.389+0000 7fd5a4874640 1 -- 172.21.15.139:0/61938685 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a00c5650 con 0x7fd5a00c63c0 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.390+0000 7fd5a4874640 1 --2- 172.21.15.139:0/61938685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a00c63c0 0x7fd5a015b7a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd598009e50 tx=0x7fd59802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb37288137c0aa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.390+0000 7fd59effd640 1 -- 172.21.15.139:0/61938685 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598036070 con 0x7fd5a00c63c0 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.390+0000 7fd59effd640 1 -- 172.21.15.139:0/61938685 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd598031600 con 0x7fd5a00c63c0 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.390+0000 7fd59effd640 1 -- 172.21.15.139:0/61938685 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5980319d0 con 0x7fd5a00c63c0 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.391+0000 7fd5a62d7640 1 -- 172.21.15.139:0/61938685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a00c63c0 msgr2=0x7fd5a015b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.391+0000 7fd5a62d7640 1 --2- 172.21.15.139:0/61938685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a00c63c0 0x7fd5a015b7a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd598009e50 tx=0x7fd59802a710 comp rx=0 tx=0).stop 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.391+0000 7fd5a62d7640 1 -- 172.21.15.139:0/61938685 shutdown_connections 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.391+0000 7fd5a62d7640 1 --2- 172.21.15.139:0/61938685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a00c63c0 0x7fd5a015b7a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.391+0000 7fd5a62d7640 1 --2- 172.21.15.139:0/61938685 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5a0151be0 0x7fd5a00c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.391+0000 7fd5a62d7640 1 -- 172.21.15.139:0/61938685 >> 172.21.15.139:0/61938685 conn(0x7fd5a006da40 msgr2=0x7fd5a006f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.391+0000 7fd5a62d7640 1 -- 172.21.15.139:0/61938685 shutdown_connections 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.391+0000 7fd5a62d7640 1 -- 172.21.15.139:0/61938685 wait complete. 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.392+0000 7fd5a62d7640 1 Processor -- start 2024-09-16T00:10:15.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.392+0000 7fd5a62d7640 1 -- start start 2024-09-16T00:10:15.395 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:15.395 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.393+0000 7fd5a62d7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a00c63c0 0x7fd5a014e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.393+0000 7fd5a62d7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5a0151800 0x7fd5a014db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.393+0000 7fd5a62d7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5a0151be0 0x7fd5a0147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.393+0000 7fd5a62d7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a00c5c10 con 0x7fd5a00c63c0 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.393+0000 7fd5a62d7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a015afa0 con 0x7fd5a0151be0 2024-09-16T00:10:15.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.393+0000 7fd5a62d7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a00c52d0 con 0x7fd5a0151800 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.393+0000 7fd59ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a00c63c0 0x7fd5a014e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.393+0000 7fd59f7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5a0151800 0x7fd5a014db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.393+0000 7fd5a4874640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5a0151be0 0x7fd5a0147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.393+0000 7fd59ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a00c63c0 0x7fd5a014e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49254/0 (socket says 172.21.15.139:49254) 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.393+0000 7fd59ffff640 1 -- 172.21.15.139:0/457183132 learned_addr learned my addr 172.21.15.139:0/457183132 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.394+0000 7fd59ffff640 1 -- 172.21.15.139:0/457183132 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5a0151800 msgr2=0x7fd5a014db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.394+0000 7fd59ffff640 1 --2- 172.21.15.139:0/457183132 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5a0151800 0x7fd5a014db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.394+0000 7fd59ffff640 1 -- 172.21.15.139:0/457183132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5a0151be0 msgr2=0x7fd5a0147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.394+0000 7fd59ffff640 1 --2- 172.21.15.139:0/457183132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5a0151be0 0x7fd5a0147900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.394+0000 7fd59ffff640 1 -- 172.21.15.139:0/457183132 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a0154a00 con 0x7fd5a00c63c0 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.394+0000 7fd59f7fe640 1 --2- 172.21.15.139:0/457183132 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5a0151800 0x7fd5a014db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.394+0000 7fd59ffff640 1 --2- 172.21.15.139:0/457183132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a00c63c0 0x7fd5a014e400 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd59000c4d0 tx=0x7fd59000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:15.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.394+0000 7fd59cff9640 1 -- 172.21.15.139:0/457183132 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590018070 con 0x7fd5a00c63c0 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.394+0000 7fd59cff9640 1 -- 172.21.15.139:0/457183132 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5900074e0 con 0x7fd5a00c63c0 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.394+0000 7fd5a62d7640 1 -- 172.21.15.139:0/457183132 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a0154ce0 con 0x7fd5a00c63c0 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.394+0000 7fd59cff9640 1 -- 172.21.15.139:0/457183132 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590004a20 con 0x7fd5a00c63c0 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.394+0000 7fd5a62d7640 1 -- 172.21.15.139:0/457183132 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a014cbb0 con 0x7fd5a00c63c0 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.395+0000 7fd5a62d7640 1 -- 172.21.15.139:0/457183132 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a014c6c0 con 0x7fd5a00c63c0 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.396+0000 7fd59cff9640 1 -- 172.21.15.139:0/457183132 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd59001b070 con 0x7fd5a00c63c0 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.396+0000 7fd59cff9640 1 --2- 172.21.15.139:0/457183132 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd58c06c2e0 0x7fd58c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.396+0000 7fd59cff9640 1 -- 172.21.15.139:0/457183132 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5973+0+0 (secure 0 0 0) 0x7fd59008a840 con 0x7fd5a00c63c0 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.396+0000 7fd59f7fe640 1 --2- 172.21.15.139:0/457183132 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd58c06c2e0 0x7fd58c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.399 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.397+0000 7fd59f7fe640 1 --2- 172.21.15.139:0/457183132 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd58c06c2e0 0x7fd58c06e7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd5a01460e0 tx=0x7fd594009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:15.401 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.401+0000 7fd59cff9640 1 -- 172.21.15.139:0/457183132 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd5900587c0 con 0x7fd5a00c63c0 2024-09-16T00:10:15.404 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:10:15.445 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:15.444+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:15.451 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:10:15.532 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:15.531+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:15.556 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:15.556 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:15.566 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:10:15.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.595+0000 7f2489796640 1 Processor -- start 2024-09-16T00:10:15.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.595+0000 7f2489796640 1 -- start start 2024-09-16T00:10:15.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.596+0000 7f2489796640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f24840d0490 0x7f24840b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.596+0000 7f2489796640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24840b3180 con 0x7f24840d0870 2024-09-16T00:10:15.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.596+0000 7f2489796640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f248415adc0 con 0x7f24840b4750 2024-09-16T00:10:15.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.596+0000 7f2489796640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f248415b940 con 0x7f24840d0490 2024-09-16T00:10:15.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.596+0000 7f24827fc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f24840d0870 0x7f24840b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45756/0 (socket says 172.21.15.139:45756) 2024-09-16T00:10:15.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.596+0000 7f24827fc640 1 -- 172.21.15.139:0/1827861545 learned_addr learned my addr 172.21.15.139:0/1827861545 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:15.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.596+0000 7f2482ffd640 1 --2- 172.21.15.139:0/1827861545 >> v2:172.21.15.164:3300/0 conn(0x7f24840d0490 0x7f24840b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.597+0000 7f2481ffb640 1 -- 172.21.15.139:0/1827861545 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2389227869 0 0) 0x7f24840b3180 con 0x7f24840d0870 2024-09-16T00:10:15.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.597+0000 7f2481ffb640 1 -- 172.21.15.139:0/1827861545 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f246c003520 con 0x7f24840d0870 2024-09-16T00:10:15.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.597+0000 7f2482ffd640 1 -- 172.21.15.139:0/1827861545 >> v1:172.21.15.164:6789/0 conn(0x7f24840b4750 legacy=0x7f24841546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.597+0000 7f2482ffd640 1 -- 172.21.15.139:0/1827861545 >> v1:172.21.15.139:6789/0 conn(0x7f24840d0870 legacy=0x7f24840b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.597+0000 7f2482ffd640 1 -- 172.21.15.139:0/1827861545 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248415b0a0 con 0x7f24840d0490 2024-09-16T00:10:15.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.598+0000 7f2482ffd640 1 --2- 172.21.15.139:0/1827861545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24840d0490 0x7f24840b2c40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f24740097d0 tx=0x7f247402a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=417274ccfcdbf3fd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:15.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.598+0000 7f2481ffb640 1 -- 172.21.15.139:0/1827861545 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474037070 con 0x7f24840d0490 2024-09-16T00:10:15.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.598+0000 7f2481ffb640 1 -- 172.21.15.139:0/1827861545 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2474032440 con 0x7f24840d0490 2024-09-16T00:10:15.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.598+0000 7f2481ffb640 1 -- 172.21.15.139:0/1827861545 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24740327f0 con 0x7f24840d0490 2024-09-16T00:10:15.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.598+0000 7f2489796640 1 -- 172.21.15.139:0/1827861545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24840d0490 msgr2=0x7f24840b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.598+0000 7f2489796640 1 --2- 172.21.15.139:0/1827861545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24840d0490 0x7f24840b2c40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f24740097d0 tx=0x7f247402a2e0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.599+0000 7f2489796640 1 -- 172.21.15.139:0/1827861545 shutdown_connections 2024-09-16T00:10:15.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.599+0000 7f2489796640 1 --2- 172.21.15.139:0/1827861545 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24840d0490 0x7f24840b2c40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.599+0000 7f2489796640 1 -- 172.21.15.139:0/1827861545 >> 172.21.15.139:0/1827861545 conn(0x7f248406c520 msgr2=0x7f248406c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:15.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.599+0000 7f2489796640 1 -- 172.21.15.139:0/1827861545 shutdown_connections 2024-09-16T00:10:15.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.599+0000 7f2489796640 1 -- 172.21.15.139:0/1827861545 wait complete. 2024-09-16T00:10:15.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.600+0000 7f2489796640 1 Processor -- start 2024-09-16T00:10:15.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.600+0000 7f2489796640 1 -- start start 2024-09-16T00:10:15.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.601+0000 7f2489796640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24840b4750 0x7f24841500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.601+0000 7f2489796640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24840d0490 0x7f24841f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.601+0000 7f2489796640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24840d0870 0x7f24841f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.601+0000 7f2489796640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f248415b660 con 0x7f24840d0870 2024-09-16T00:10:15.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.601+0000 7f2489796640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f24840b3bb0 con 0x7f24840d0490 2024-09-16T00:10:15.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.601+0000 7f2489796640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f248415a980 con 0x7f24840b4750 2024-09-16T00:10:15.604 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.601+0000 7fd5a62d7640 1 -- 172.21.15.139:0/457183132 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 0.5229187413364552} v 0) -- 0x7fd5a00c96f0 con 0x7fd5a00c63c0 2024-09-16T00:10:15.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.601+0000 7f24837fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24840d0870 0x7f24841f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.601+0000 7f2482ffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24840b4750 0x7f24841500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.601+0000 7f2482ffd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24840b4750 0x7f24841500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:56258/0 (socket says 172.21.15.139:56258) 2024-09-16T00:10:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.601+0000 7f2482ffd640 1 -- 172.21.15.139:0/2943814105 learned_addr learned my addr 172.21.15.139:0/2943814105 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.601+0000 7f24827fc640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24840d0490 0x7f24841f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.602+0000 7f2482ffd640 1 -- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24840d0490 msgr2=0x7f24841f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.602+0000 7f2482ffd640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24840d0490 0x7f24841f1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.602+0000 7f2482ffd640 1 -- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24840d0870 msgr2=0x7f24841f21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.602+0000 7f2482ffd640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24840d0870 0x7f24841f21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.602+0000 7f2482ffd640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24841f3d50 con 0x7f24840b4750 2024-09-16T00:10:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.602+0000 7f24827fc640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f24840d0490 0x7f24841f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.602+0000 7f24837fe640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24840d0870 0x7f24841f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.602+0000 7f2482ffd640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24840b4750 0x7f24841500b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2474035040 tx=0x7f24740046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:15.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.602+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24740044e0 con 0x7f24840b4750 2024-09-16T00:10:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.602+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2474003810 con 0x7f24840b4750 2024-09-16T00:10:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.602+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24841f4030 con 0x7f24840b4750 2024-09-16T00:10:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.602+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474003c00 con 0x7f24840b4750 2024-09-16T00:10:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.602+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f24841f45f0 con 0x7f24840b4750 2024-09-16T00:10:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.603+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f24840b3430 con 0x7f24840b4750 2024-09-16T00:10:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.604+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2474032440 con 0x7f24840b4750 2024-09-16T00:10:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.604+0000 7f246b7fe640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f247006c2b0 0x7f247006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.604+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(105..105 src has 1..105) ==== 5973+0+0 (secure 0 0 0) 0x7f24740b3c90 con 0x7f24840b4750 2024-09-16T00:10:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.604+0000 7f24827fc640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f247006c2b0 0x7f247006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.604+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v105) ==== 54+0+16 (secure 0 0 0) 0x7f2474044070 con 0x7f24840b4750 2024-09-16T00:10:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.605+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f24841f47c0 0x7f2484158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.605+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24840cc420 con 0x7f24841f47c0 2024-09-16T00:10:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.605+0000 7f24837fe640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f24841f47c0 0x7f2484158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.605+0000 7f24827fc640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f247006c2b0 0x7f247006e770 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2484156a90 tx=0x7f2478007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.605+0000 7f24837fe640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f24841f47c0 0x7f2484158560 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:15.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.606+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f24840cc420 con 0x7f24841f47c0 2024-09-16T00:10:15.619 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:15.618+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:15.622 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.622+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f24840cc140 con 0x7f24841f47c0 2024-09-16T00:10:15.624 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.623+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f24840cc140 con 0x7f24841f47c0 2024-09-16T00:10:15.625 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:15.627 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:15.627 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.623+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f24841572c0 0x7f248414a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.623+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f24840cbe60 con 0x7f24841572c0 2024-09-16T00:10:15.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.624+0000 7f2482ffd640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f24841572c0 0x7f248414a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.624+0000 7f2482ffd640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f24841572c0 0x7f248414a510 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:15.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.625+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f24840cbe60 con 0x7f24841572c0 2024-09-16T00:10:15.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.640+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f24840cbb80 con 0x7f24841572c0 2024-09-16T00:10:15.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.642+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f24840cbb80 con 0x7f24841572c0 2024-09-16T00:10:15.644 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:15.645 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:15.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.642+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f24841552f0 0x7f2484155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.642+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f2484155c70 con 0x7f24841552f0 2024-09-16T00:10:15.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.642+0000 7f24827fc640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f24841552f0 0x7f2484155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.643+0000 7f24827fc640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f24841552f0 0x7f2484155730 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:15.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.643+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2484155c70 con 0x7f24841552f0 2024-09-16T00:10:15.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.659+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2484156100 con 0x7f24841552f0 2024-09-16T00:10:15.662 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.661+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2484156100 con 0x7f24841552f0 2024-09-16T00:10:15.662 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:15.663 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:15.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.661+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f2484062f00 0x7f24840ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.661+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f2484063380 con 0x7f2484062f00 2024-09-16T00:10:15.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.661+0000 7f24837fe640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f2484062f00 0x7f24840ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.662+0000 7f24837fe640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f2484062f00 0x7f24840ce390 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:15.663 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.663+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2484063380 con 0x7f2484062f00 2024-09-16T00:10:15.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.678+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2484063380 con 0x7f2484062f00 2024-09-16T00:10:15.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.680+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2484063380 con 0x7f2484062f00 2024-09-16T00:10:15.681 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:15.682 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:15.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.681+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f24840cf090 0x7f248402c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.681+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f2484063380 con 0x7f24840cf090 2024-09-16T00:10:15.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.681+0000 7f2482ffd640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f24840cf090 0x7f248402c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.681+0000 7f2482ffd640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f24840cf090 0x7f248402c640 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:15.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.682+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2484063380 con 0x7f24840cf090 2024-09-16T00:10:15.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.698+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2484063380 con 0x7f24840cf090 2024-09-16T00:10:15.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.700+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2484063380 con 0x7f24840cf090 2024-09-16T00:10:15.700 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:15.701 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:15.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.700+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f248402d090 0x7f248402d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.700+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f2484063590 con 0x7f248402d090 2024-09-16T00:10:15.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.700+0000 7f24827fc640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f248402d090 0x7f248402d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.701+0000 7f24827fc640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f248402d090 0x7f248402d4f0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:15.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.701+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2484063590 con 0x7f248402d090 2024-09-16T00:10:15.713 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:15.714 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:15.714 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:15.714 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:15.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.717+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f248402da30 con 0x7f248402d090 2024-09-16T00:10:15.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.718+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f248402da30 con 0x7f248402d090 2024-09-16T00:10:15.719 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:15.720 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:15.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.718+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f2484203430 0x7f2484205820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.718+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f24842085f0 con 0x7f2484203430 2024-09-16T00:10:15.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.719+0000 7f24837fe640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f2484203430 0x7f2484205820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.720 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:15.719+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:15.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.719+0000 7f24837fe640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f2484203430 0x7f2484205820 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:15.722 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:10:15.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.720+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f24842085f0 con 0x7f2484203430 2024-09-16T00:10:15.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.735+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2484205d60 con 0x7f2484203430 2024-09-16T00:10:15.738 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:15.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.736+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2484205d60 con 0x7f2484203430 2024-09-16T00:10:15.739 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:15.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.737+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f24842063e0 0x7f2484206820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:15.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.737+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f248420b980 con 0x7f24842063e0 2024-09-16T00:10:15.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.737+0000 7f2482ffd640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f24842063e0 0x7f2484206820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:15.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.737+0000 7f2482ffd640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f24842063e0 0x7f2484206820 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:15.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.738+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f248420b980 con 0x7f24842063e0 2024-09-16T00:10:15.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.753+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f2484206e70 con 0x7f24842063e0 2024-09-16T00:10:15.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.756+0000 7f246b7fe640 1 -- 172.21.15.139:0/2943814105 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2484206e70 con 0x7f24842063e0 2024-09-16T00:10:15.758 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:10:15.759 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:10:15.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.756+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f24841f47c0 msgr2=0x7f2484158560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f24841f47c0 0x7f2484158560 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f24841572c0 msgr2=0x7f248414a510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f24841572c0 0x7f248414a510 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f24841552f0 msgr2=0x7f2484155730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f24841552f0 0x7f2484155730 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f2484062f00 msgr2=0x7f24840ce390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f2484062f00 0x7f24840ce390 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f24837fe640 1 -- 172.21.15.139:0/2943814105 reap_dead start 2024-09-16T00:10:15.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f24840cf090 msgr2=0x7f248402c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f24840cf090 0x7f248402c640 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f24837fe640 1 -- 172.21.15.139:0/2943814105 reap_dead start 2024-09-16T00:10:15.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f248402d090 msgr2=0x7f248402d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f248402d090 0x7f248402d4f0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f2484203430 msgr2=0x7f2484205820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f2484203430 0x7f2484205820 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f24842063e0 msgr2=0x7f2484206820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f24842063e0 0x7f2484206820 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f247006c2b0 msgr2=0x7f247006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f247006c2b0 0x7f247006e770 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2484156a90 tx=0x7f2478007d30 comp rx=0 tx=0).stop 2024-09-16T00:10:15.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24840b4750 msgr2=0x7f24841500b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:15.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f24837fe640 1 -- 172.21.15.139:0/2943814105 reap_dead start 2024-09-16T00:10:15.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24840b4750 0x7f24841500b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2474035040 tx=0x7f24740046c0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 shutdown_connections 2024-09-16T00:10:15.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 --2- 172.21.15.139:0/2943814105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f24840b4750 0x7f24841500b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:15.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 >> 172.21.15.139:0/2943814105 conn(0x7f248406c520 msgr2=0x7f2484074d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:15.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 shutdown_connections 2024-09-16T00:10:15.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:15.757+0000 7f2489796640 1 -- 172.21.15.139:0/2943814105 wait complete. 2024-09-16T00:10:15.821 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:15.820+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:15.848 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:15.849 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:15.849 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:15.849 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:15.849 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:15.858 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:10:15.942 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:15.941+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:16.022 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:16.023 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:16.023 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:16.023 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:16.023 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.023 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3161 2.d 2:b02ffd62:::200.00000010:head [write 3427859~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.023 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:16.023 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.357933086999999, 2024-09-16T00:10:16.023 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026748516, 2024-09-16T00:10:16.023 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.023 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.023 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.023 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.024 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:16.024 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.024 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:16.024 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.024 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.025 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:16.025 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.15e-07 2024-09-16T00:10:16.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.025 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.025 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.025 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662637+0000", 2024-09-16T00:10:16.025 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8009999999999999e-06 2024-09-16T00:10:16.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.025 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.025 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.025 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662645+0000", 2024-09-16T00:10:16.025 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8299999999999996e-06 2024-09-16T00:10:16.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.026 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.026 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.026 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662650+0000", 2024-09-16T00:10:16.026 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.605e-06 2024-09-16T00:10:16.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.026 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.026 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.026 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.682553+0000", 2024-09-16T00:10:16.026 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01990331 2024-09-16T00:10:16.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.026 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.026 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.026 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.682636+0000", 2024-09-16T00:10:16.027 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2921000000000001e-05 2024-09-16T00:10:16.027 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.027 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.027 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.682780+0000", 2024-09-16T00:10:16.027 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000144361 2024-09-16T00:10:16.027 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.027 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.027 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.684809+0000", 2024-09-16T00:10:16.027 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002029005 2024-09-16T00:10:16.027 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.028 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.028 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689293+0000", 2024-09-16T00:10:16.028 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044841079999999997 2024-09-16T00:10:16.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.028 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.028 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689339+0000", 2024-09-16T00:10:16.028 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5668999999999999e-05 2024-09-16T00:10:16.028 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.029 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.029 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689382+0000", 2024-09-16T00:10:16.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3491000000000001e-05 2024-09-16T00:10:16.029 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.029 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.029 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.029 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3162 2.d 2:b02ffd62:::200.00000010:head [write 3430104~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.029 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.662858+0000", 2024-09-16T00:10:16.029 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.357708971000001, 2024-09-16T00:10:16.029 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026630083999999998, 2024-09-16T00:10:16.029 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.030 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.030 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.030 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.030 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662858+0000", 2024-09-16T00:10:16.030 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.030 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.030 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662858+0000", 2024-09-16T00:10:16.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.031 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.031 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662859+0000", 2024-09-16T00:10:16.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:10:16.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.031 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.031 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662862+0000", 2024-09-16T00:10:16.031 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6119999999999999e-06 2024-09-16T00:10:16.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.031 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.032 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662870+0000", 2024-09-16T00:10:16.032 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2360000000000004e-06 2024-09-16T00:10:16.032 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.032 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.032 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.032 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662874+0000", 2024-09-16T00:10:16.032 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2610000000000002e-06 2024-09-16T00:10:16.032 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.032 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.032 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.033 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.683241+0000", 2024-09-16T00:10:16.033 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.020367495999999999 2024-09-16T00:10:16.033 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.033 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.033 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.033 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.683295+0000", 2024-09-16T00:10:16.033 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4230000000000001e-05 2024-09-16T00:10:16.033 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.033 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.033 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.033 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.683380+0000", 2024-09-16T00:10:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4367e-05 2024-09-16T00:10:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.684845+0000", 2024-09-16T00:10:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014653190000000001 2024-09-16T00:10:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689392+0000", 2024-09-16T00:10:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045474280000000001 2024-09-16T00:10:16.034 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689445+0000", 2024-09-16T00:10:16.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2444000000000001e-05 2024-09-16T00:10:16.035 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.035 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.035 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.035 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689488+0000", 2024-09-16T00:10:16.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3124999999999998e-05 2024-09-16T00:10:16.036 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.036 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.036 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.036 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.036 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.036 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3166 2.d 2:b02ffd62:::200.00000010:head [write 3437392~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.664403+0000", 2024-09-16T00:10:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.356164093, 2024-09-16T00:10:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.027480674, 2024-09-16T00:10:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664403+0000", 2024-09-16T00:10:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664403+0000", 2024-09-16T00:10:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664404+0000", 2024-09-16T00:10:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4099999999999998e-07 2024-09-16T00:10:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.038 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664408+0000", 2024-09-16T00:10:16.039 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8169999999999998e-06 2024-09-16T00:10:16.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.039 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.039 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.039 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664418+0000", 2024-09-16T00:10:16.039 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6909999999999992e-06 2024-09-16T00:10:16.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.039 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.039 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.039 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664425+0000", 2024-09-16T00:10:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6309999999999997e-06 2024-09-16T00:10:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.685372+0000", 2024-09-16T00:10:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.020947548999999999 2024-09-16T00:10:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.685454+0000", 2024-09-16T00:10:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1355999999999998e-05 2024-09-16T00:10:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.041 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.041 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.685585+0000", 2024-09-16T00:10:16.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000130922 2024-09-16T00:10:16.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.041 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.041 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.041 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687329+0000", 2024-09-16T00:10:16.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017445710000000001 2024-09-16T00:10:16.042 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.042 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.042 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691757+0000", 2024-09-16T00:10:16.042 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004428268 2024-09-16T00:10:16.042 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.042 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.042 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691830+0000", 2024-09-16T00:10:16.042 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2237999999999995e-05 2024-09-16T00:10:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691884+0000", 2024-09-16T00:10:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3890000000000001e-05 2024-09-16T00:10:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3167 2.d 2:b02ffd62:::200.00000010:head [write 3439637~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.043 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.664627+0000", 2024-09-16T00:10:16.044 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.355939521, 2024-09-16T00:10:16.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.027347698, 2024-09-16T00:10:16.044 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.044 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.044 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.044 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664627+0000", 2024-09-16T00:10:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664627+0000", 2024-09-16T00:10:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664628+0000", 2024-09-16T00:10:16.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0500000000000003e-07 2024-09-16T00:10:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664632+0000", 2024-09-16T00:10:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8770000000000003e-06 2024-09-16T00:10:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664639+0000", 2024-09-16T00:10:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6590000000000001e-06 2024-09-16T00:10:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664642+0000", 2024-09-16T00:10:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8550000000000004e-06 2024-09-16T00:10:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.686530+0000", 2024-09-16T00:10:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.021887127999999999 2024-09-16T00:10:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.686577+0000", 2024-09-16T00:10:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7181000000000001e-05 2024-09-16T00:10:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.047 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.686667+0000", 2024-09-16T00:10:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0074000000000005e-05 2024-09-16T00:10:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688714+0000", 2024-09-16T00:10:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020467749999999998 2024-09-16T00:10:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691894+0000", 2024-09-16T00:10:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031806529999999999 2024-09-16T00:10:16.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.049 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.049 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691939+0000", 2024-09-16T00:10:16.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4579999999999997e-05 2024-09-16T00:10:16.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.049 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691975+0000", 2024-09-16T00:10:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6310999999999998e-05 2024-09-16T00:10:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3168 2.d 2:b02ffd62:::200.00000010:head [write 3441036~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.355473440000001, 2024-09-16T00:10:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026965759999999998, 2024-09-16T00:10:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.050 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7999999999999996e-07 2024-09-16T00:10:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665098+0000", 2024-09-16T00:10:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3200000000000001e-06 2024-09-16T00:10:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665104+0000", 2024-09-16T00:10:16.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4809999999999997e-06 2024-09-16T00:10:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665112+0000", 2024-09-16T00:10:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.67e-06 2024-09-16T00:10:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687392+0000", 2024-09-16T00:10:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.022279699 2024-09-16T00:10:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.054 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687461+0000", 2024-09-16T00:10:16.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8476e-05 2024-09-16T00:10:16.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.054 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.054 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687577+0000", 2024-09-16T00:10:16.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011661 2024-09-16T00:10:16.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.054 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.054 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688774+0000", 2024-09-16T00:10:16.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011962660000000001 2024-09-16T00:10:16.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691984+0000", 2024-09-16T00:10:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032106320000000002 2024-09-16T00:10:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692028+0000", 2024-09-16T00:10:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3866999999999999e-05 2024-09-16T00:10:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692059+0000", 2024-09-16T00:10:16.055 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1259e-05 2024-09-16T00:10:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3169 2.d 2:b02ffd62:::200.00000010:head [write 3443281~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.355016903999999, 2024-09-16T00:10:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.030090967999999999, 2024-09-16T00:10:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.056 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3799999999999998e-07 2024-09-16T00:10:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665554+0000", 2024-09-16T00:10:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.112e-06 2024-09-16T00:10:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665559+0000", 2024-09-16T00:10:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9559999999999996e-06 2024-09-16T00:10:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665562+0000", 2024-09-16T00:10:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2909999999999999e-06 2024-09-16T00:10:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687954+0000", 2024-09-16T00:10:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.022391714 2024-09-16T00:10:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688016+0000", 2024-09-16T00:10:16.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2384000000000004e-05 2024-09-16T00:10:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688147+0000", 2024-09-16T00:10:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013099300000000001 2024-09-16T00:10:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690321+0000", 2024-09-16T00:10:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021744339999999998 2024-09-16T00:10:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695548+0000", 2024-09-16T00:10:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052266079999999998 2024-09-16T00:10:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695595+0000", 2024-09-16T00:10:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7200000000000002e-05 2024-09-16T00:10:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695641+0000", 2024-09-16T00:10:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5837999999999997e-05 2024-09-16T00:10:16.061 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3170 2.d 2:b02ffd62:::200.00000010:head [write 3445526~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.354823842999998, 2024-09-16T00:10:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029987413000000001, 2024-09-16T00:10:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.062 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7500000000000001e-07 2024-09-16T00:10:16.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665753+0000", 2024-09-16T00:10:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0005e-05 2024-09-16T00:10:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665759+0000", 2024-09-16T00:10:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.169e-06 2024-09-16T00:10:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665762+0000", 2024-09-16T00:10:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8460000000000001e-06 2024-09-16T00:10:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688811+0000", 2024-09-16T00:10:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023048169 2024-09-16T00:10:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688885+0000", 2024-09-16T00:10:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4788000000000003e-05 2024-09-16T00:10:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688989+0000", 2024-09-16T00:10:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000103689 2024-09-16T00:10:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690350+0000", 2024-09-16T00:10:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013609329999999999 2024-09-16T00:10:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695652+0000", 2024-09-16T00:10:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053019080000000001 2024-09-16T00:10:16.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695698+0000", 2024-09-16T00:10:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5775000000000001e-05 2024-09-16T00:10:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695731+0000", 2024-09-16T00:10:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2756000000000003e-05 2024-09-16T00:10:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3171 2.d 2:b02ffd62:::200.00000010:head [write 3446925~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665997+0000", 2024-09-16T00:10:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.354569752, 2024-09-16T00:10:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029824032, 2024-09-16T00:10:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665997+0000", 2024-09-16T00:10:16.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665997+0000", 2024-09-16T00:10:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665998+0000", 2024-09-16T00:10:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:10:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666000+0000", 2024-09-16T00:10:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6290000000000002e-06 2024-09-16T00:10:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666005+0000", 2024-09-16T00:10:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.437e-06 2024-09-16T00:10:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666008+0000", 2024-09-16T00:10:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7910000000000002e-06 2024-09-16T00:10:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689799+0000", 2024-09-16T00:10:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023790504000000001 2024-09-16T00:10:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689907+0000", 2024-09-16T00:10:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108464 2024-09-16T00:10:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689994+0000", 2024-09-16T00:10:16.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7043e-05 2024-09-16T00:10:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691737+0000", 2024-09-16T00:10:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001742527 2024-09-16T00:10:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695739+0000", 2024-09-16T00:10:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040020189999999999 2024-09-16T00:10:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695786+0000", 2024-09-16T00:10:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7219999999999999e-05 2024-09-16T00:10:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695821+0000", 2024-09-16T00:10:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5058000000000002e-05 2024-09-16T00:10:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3172 2.d 2:b02ffd62:::200.00000010:head [write 3448324~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.666428+0000", 2024-09-16T00:10:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.354138802000001, 2024-09-16T00:10:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029477316, 2024-09-16T00:10:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666428+0000", 2024-09-16T00:10:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666428+0000", 2024-09-16T00:10:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666429+0000", 2024-09-16T00:10:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3699999999999999e-07 2024-09-16T00:10:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666433+0000", 2024-09-16T00:10:16.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4939999999999997e-06 2024-09-16T00:10:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666443+0000", 2024-09-16T00:10:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0899999999999994e-06 2024-09-16T00:10:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666447+0000", 2024-09-16T00:10:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8930000000000004e-06 2024-09-16T00:10:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690390+0000", 2024-09-16T00:10:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023942215999999999 2024-09-16T00:10:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690464+0000", 2024-09-16T00:10:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4678999999999994e-05 2024-09-16T00:10:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690576+0000", 2024-09-16T00:10:16.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011162000000000001 2024-09-16T00:10:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692126+0000", 2024-09-16T00:10:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015496520000000001 2024-09-16T00:10:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695828+0000", 2024-09-16T00:10:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037028510000000001 2024-09-16T00:10:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695870+0000", 2024-09-16T00:10:16.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1180999999999998e-05 2024-09-16T00:10:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695905+0000", 2024-09-16T00:10:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5803000000000003e-05 2024-09-16T00:10:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3173 2.d 2:b02ffd62:::200.00000010:head [write 3449723~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.666884+0000", 2024-09-16T00:10:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.353682846000002, 2024-09-16T00:10:16.079 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.02908875, 2024-09-16T00:10:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666884+0000", 2024-09-16T00:10:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666884+0000", 2024-09-16T00:10:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666885+0000", 2024-09-16T00:10:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0999999999999998e-07 2024-09-16T00:10:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666889+0000", 2024-09-16T00:10:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4299999999999999e-06 2024-09-16T00:10:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666895+0000", 2024-09-16T00:10:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0510000000000002e-06 2024-09-16T00:10:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666899+0000", 2024-09-16T00:10:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0489999999999997e-06 2024-09-16T00:10:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691161+0000", 2024-09-16T00:10:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.024262156999999999 2024-09-16T00:10:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691253+0000", 2024-09-16T00:10:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1278000000000003e-05 2024-09-16T00:10:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691348+0000", 2024-09-16T00:10:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4777000000000004e-05 2024-09-16T00:10:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692211+0000", 2024-09-16T00:10:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086351699999999995 2024-09-16T00:10:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695913+0000", 2024-09-16T00:10:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037019840000000002 2024-09-16T00:10:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695952+0000", 2024-09-16T00:10:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8535999999999998e-05 2024-09-16T00:10:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695973+0000", 2024-09-16T00:10:16.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1260999999999999e-05 2024-09-16T00:10:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3174 2.d 2:b02ffd62:::200.00000010:head [write 3451968~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.667323+0000", 2024-09-16T00:10:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.353243976000002, 2024-09-16T00:10:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028788859999999999, 2024-09-16T00:10:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.085 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667323+0000", 2024-09-16T00:10:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667323+0000", 2024-09-16T00:10:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667324+0000", 2024-09-16T00:10:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0400000000000005e-07 2024-09-16T00:10:16.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667330+0000", 2024-09-16T00:10:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3300000000000004e-06 2024-09-16T00:10:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667338+0000", 2024-09-16T00:10:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0059999999999992e-06 2024-09-16T00:10:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.087 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667344+0000", 2024-09-16T00:10:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6409999999999999e-06 2024-09-16T00:10:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692253+0000", 2024-09-16T00:10:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.024909280999999998 2024-09-16T00:10:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692311+0000", 2024-09-16T00:10:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7873e-05 2024-09-16T00:10:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692395+0000", 2024-09-16T00:10:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4071000000000002e-05 2024-09-16T00:10:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695977+0000", 2024-09-16T00:10:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035823460000000001 2024-09-16T00:10:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696034+0000", 2024-09-16T00:10:16.089 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6697999999999998e-05 2024-09-16T00:10:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696078+0000", 2024-09-16T00:10:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3696000000000003e-05 2024-09-16T00:10:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696112+0000", 2024-09-16T00:10:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4113999999999999e-05 2024-09-16T00:10:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3175 2.d 2:b02ffd62:::200.00000010:head [write 3454213~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.352903137999999, 2024-09-16T00:10:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.030153698999999999, 2024-09-16T00:10:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2e-07 2024-09-16T00:10:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667668+0000", 2024-09-16T00:10:16.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2109999999999998e-06 2024-09-16T00:10:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667673+0000", 2024-09-16T00:10:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8409999999999996e-06 2024-09-16T00:10:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667686+0000", 2024-09-16T00:10:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2928000000000001e-05 2024-09-16T00:10:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692964+0000", 2024-09-16T00:10:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.025277642999999999 2024-09-16T00:10:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.694418+0000", 2024-09-16T00:10:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001453693 2024-09-16T00:10:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.694505+0000", 2024-09-16T00:10:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7700000000000004e-05 2024-09-16T00:10:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.094 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697114+0000", 2024-09-16T00:10:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026083460000000001 2024-09-16T00:10:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697697+0000", 2024-09-16T00:10:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00058297399999999999 2024-09-16T00:10:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697772+0000", 2024-09-16T00:10:16.095 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5433999999999996e-05 2024-09-16T00:10:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697818+0000", 2024-09-16T00:10:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5408999999999998e-05 2024-09-16T00:10:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3176 2.d 2:b02ffd62:::200.00000010:head [write 3456458~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.668006+0000", 2024-09-16T00:10:16.096 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.352561232999999, 2024-09-16T00:10:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029935413000000001, 2024-09-16T00:10:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668006+0000", 2024-09-16T00:10:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668006+0000", 2024-09-16T00:10:16.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668007+0000", 2024-09-16T00:10:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5799999999999998e-07 2024-09-16T00:10:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668012+0000", 2024-09-16T00:10:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9320000000000004e-06 2024-09-16T00:10:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668023+0000", 2024-09-16T00:10:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1273e-05 2024-09-16T00:10:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668030+0000", 2024-09-16T00:10:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0509999999999999e-06 2024-09-16T00:10:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.694928+0000", 2024-09-16T00:10:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026898114000000001 2024-09-16T00:10:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695003+0000", 2024-09-16T00:10:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4880000000000001e-05 2024-09-16T00:10:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695116+0000", 2024-09-16T00:10:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011307900000000001 2024-09-16T00:10:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697119+0000", 2024-09-16T00:10:16.100 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020032159999999999 2024-09-16T00:10:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697867+0000", 2024-09-16T00:10:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074744999999999998 2024-09-16T00:10:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697908+0000", 2024-09-16T00:10:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1287e-05 2024-09-16T00:10:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697941+0000", 2024-09-16T00:10:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3272999999999998e-05 2024-09-16T00:10:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3177 2.d 2:b02ffd62:::200.00000010:head [write 3457857~1398 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.351226373999999, 2024-09-16T00:10:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029906856999999998, 2024-09-16T00:10:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.102 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.103 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1299999999999998e-07 2024-09-16T00:10:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669347+0000", 2024-09-16T00:10:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.327e-06 2024-09-16T00:10:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669359+0000", 2024-09-16T00:10:16.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1698e-05 2024-09-16T00:10:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669366+0000", 2024-09-16T00:10:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7959999999999996e-06 2024-09-16T00:10:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696168+0000", 2024-09-16T00:10:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026802093999999999 2024-09-16T00:10:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696258+0000", 2024-09-16T00:10:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0074000000000005e-05 2024-09-16T00:10:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696383+0000", 2024-09-16T00:10:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012455699999999999 2024-09-16T00:10:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698602+0000", 2024-09-16T00:10:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002218921 2024-09-16T00:10:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699152+0000", 2024-09-16T00:10:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055024399999999997 2024-09-16T00:10:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699206+0000", 2024-09-16T00:10:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3882999999999999e-05 2024-09-16T00:10:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699247+0000", 2024-09-16T00:10:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1350000000000002e-05 2024-09-16T00:10:16.107 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3178 2.d 2:b02ffd62:::200.00000010:head [write 3459255~2511 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.671779+0000", 2024-09-16T00:10:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.348787494, 2024-09-16T00:10:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028813986999999999, 2024-09-16T00:10:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.108 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671779+0000", 2024-09-16T00:10:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671779+0000", 2024-09-16T00:10:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671780+0000", 2024-09-16T00:10:16.109 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3600000000000002e-07 2024-09-16T00:10:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671785+0000", 2024-09-16T00:10:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.074e-06 2024-09-16T00:10:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671795+0000", 2024-09-16T00:10:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2040000000000003e-06 2024-09-16T00:10:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.110 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671800+0000", 2024-09-16T00:10:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9420000000000003e-06 2024-09-16T00:10:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697159+0000", 2024-09-16T00:10:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.025359619999999999 2024-09-16T00:10:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697214+0000", 2024-09-16T00:10:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5318000000000001e-05 2024-09-16T00:10:16.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697297+0000", 2024-09-16T00:10:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2087000000000002e-05 2024-09-16T00:10:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699258+0000", 2024-09-16T00:10:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019619529999999998 2024-09-16T00:10:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700499+0000", 2024-09-16T00:10:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001240072 2024-09-16T00:10:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700555+0000", 2024-09-16T00:10:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6459999999999998e-05 2024-09-16T00:10:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700593+0000", 2024-09-16T00:10:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8420999999999997e-05 2024-09-16T00:10:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.113 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3179 2.d 2:b02ffd62:::200.00000010:head [write 3461766~2418 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.672548+0000", 2024-09-16T00:10:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.348019397000002, 2024-09-16T00:10:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028160178000000001, 2024-09-16T00:10:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672548+0000", 2024-09-16T00:10:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672548+0000", 2024-09-16T00:10:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672549+0000", 2024-09-16T00:10:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5300000000000002e-07 2024-09-16T00:10:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672562+0000", 2024-09-16T00:10:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3808000000000001e-05 2024-09-16T00:10:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672575+0000", 2024-09-16T00:10:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2870999999999999e-05 2024-09-16T00:10:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672587+0000", 2024-09-16T00:10:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.203e-05 2024-09-16T00:10:16.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697994+0000", 2024-09-16T00:10:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.025406373999999999 2024-09-16T00:10:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698074+0000", 2024-09-16T00:10:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0802000000000003e-05 2024-09-16T00:10:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698196+0000", 2024-09-16T00:10:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012154699999999999 2024-09-16T00:10:16.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700456+0000", 2024-09-16T00:10:16.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022602249999999998 2024-09-16T00:10:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700632+0000", 2024-09-16T00:10:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017570699999999999 2024-09-16T00:10:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700676+0000", 2024-09-16T00:10:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3764000000000003e-05 2024-09-16T00:10:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700708+0000", 2024-09-16T00:10:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2097000000000001e-05 2024-09-16T00:10:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3180 2.d 2:b02ffd62:::200.00000010:head [write 3464184~1398 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.346511446000001, 2024-09-16T00:10:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028112109, 2024-09-16T00:10:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.121 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:10:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674061+0000", 2024-09-16T00:10:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4440000000000001e-06 2024-09-16T00:10:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674069+0000", 2024-09-16T00:10:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7639999999999994e-06 2024-09-16T00:10:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674076+0000", 2024-09-16T00:10:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5590000000000002e-06 2024-09-16T00:10:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698639+0000", 2024-09-16T00:10:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.024562471999999998 2024-09-16T00:10:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698693+0000", 2024-09-16T00:10:16.124 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4432999999999999e-05 2024-09-16T00:10:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698807+0000", 2024-09-16T00:10:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113607 2024-09-16T00:10:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700727+0000", 2024-09-16T00:10:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019201750000000001 2024-09-16T00:10:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702076+0000", 2024-09-16T00:10:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001349423 2024-09-16T00:10:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702128+0000", 2024-09-16T00:10:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2157000000000002e-05 2024-09-16T00:10:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702168+0000", 2024-09-16T00:10:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9292000000000003e-05 2024-09-16T00:10:16.126 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3181 2.d 2:b02ffd62:::200.00000010:head [write 3465582~2754 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.675904+0000", 2024-09-16T00:10:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.344663135000001, 2024-09-16T00:10:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026402452, 2024-09-16T00:10:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.127 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675904+0000", 2024-09-16T00:10:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675904+0000", 2024-09-16T00:10:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675905+0000", 2024-09-16T00:10:16.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.043e-06 2024-09-16T00:10:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675911+0000", 2024-09-16T00:10:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1970000000000001e-06 2024-09-16T00:10:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675924+0000", 2024-09-16T00:10:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.252e-05 2024-09-16T00:10:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.129 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675931+0000", 2024-09-16T00:10:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5800000000000003e-06 2024-09-16T00:10:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699280+0000", 2024-09-16T00:10:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023348634 2024-09-16T00:10:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699329+0000", 2024-09-16T00:10:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9175999999999998e-05 2024-09-16T00:10:16.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699400+0000", 2024-09-16T00:10:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0649999999999996e-05 2024-09-16T00:10:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700734+0000", 2024-09-16T00:10:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001334816 2024-09-16T00:10:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702219+0000", 2024-09-16T00:10:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014847930000000001 2024-09-16T00:10:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702267+0000", 2024-09-16T00:10:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8133000000000001e-05 2024-09-16T00:10:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702306+0000", 2024-09-16T00:10:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8909999999999998e-05 2024-09-16T00:10:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.132 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3184 2.d 2:b02ffd62:::200.00000010:head [write 3478077~11949 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.681430+0000", 2024-09-16T00:10:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.339136892999999, 2024-09-16T00:10:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.02662339, 2024-09-16T00:10:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.133 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681430+0000", 2024-09-16T00:10:16.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681430+0000", 2024-09-16T00:10:16.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681431+0000", 2024-09-16T00:10:16.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9999999999999995e-07 2024-09-16T00:10:16.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.135 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.135 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681443+0000", 2024-09-16T00:10:16.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1568e-05 2024-09-16T00:10:16.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.135 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.135 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681453+0000", 2024-09-16T00:10:16.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8819999999999996e-06 2024-09-16T00:10:16.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.135 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.135 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681458+0000", 2024-09-16T00:10:16.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3709999999999999e-06 2024-09-16T00:10:16.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.136 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.136 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.701384+0000", 2024-09-16T00:10:16.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.019926342999999999 2024-09-16T00:10:16.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.136 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.136 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.701458+0000", 2024-09-16T00:10:16.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3854000000000001e-05 2024-09-16T00:10:16.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.701570+0000", 2024-09-16T00:10:16.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000111437 2024-09-16T00:10:16.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702913+0000", 2024-09-16T00:10:16.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013437259999999999 2024-09-16T00:10:16.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.707984+0000", 2024-09-16T00:10:16.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050705669999999998 2024-09-16T00:10:16.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708016+0000", 2024-09-16T00:10:16.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2527999999999998e-05 2024-09-16T00:10:16.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708053+0000", 2024-09-16T00:10:16.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7113999999999997e-05 2024-09-16T00:10:16.138 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.138 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.138 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.139 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3185 2.d 2:b02ffd62:::200.00000010:head [write 3490026~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:16.139 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.681685+0000", 2024-09-16T00:10:16.139 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.338881621999999, 2024-09-16T00:10:16.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026552664, 2024-09-16T00:10:16.139 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.139 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.139 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.139 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681685+0000", 2024-09-16T00:10:16.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.140 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.140 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.140 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681685+0000", 2024-09-16T00:10:16.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.140 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.140 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.140 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681686+0000", 2024-09-16T00:10:16.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.06e-07 2024-09-16T00:10:16.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.140 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.140 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.140 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681692+0000", 2024-09-16T00:10:16.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6840000000000001e-06 2024-09-16T00:10:16.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681697+0000", 2024-09-16T00:10:16.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0340000000000004e-06 2024-09-16T00:10:16.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681700+0000", 2024-09-16T00:10:16.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3589999999999999e-06 2024-09-16T00:10:16.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702500+0000", 2024-09-16T00:10:16.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.020799746000000001 2024-09-16T00:10:16.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702550+0000", 2024-09-16T00:10:16.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9895999999999998e-05 2024-09-16T00:10:16.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:16.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702649+0000", 2024-09-16T00:10:16.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9500999999999995e-05 2024-09-16T00:10:16.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.705343+0000", 2024-09-16T00:10:16.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026938130000000002 2024-09-16T00:10:16.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708128+0000", 2024-09-16T00:10:16.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027852979999999999 2024-09-16T00:10:16.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708202+0000", 2024-09-16T00:10:16.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4046000000000002e-05 2024-09-16T00:10:16.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708238+0000", 2024-09-16T00:10:16.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5781000000000002e-05 2024-09-16T00:10:16.144 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.144 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.144 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.144 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.144 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.144 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:16.145 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:16.022+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:16.146 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:10:16.146 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:16.145+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:16.246 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:16.245+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:16.297 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:16.297 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:16.297 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:16.297 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:16.306 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:10:16.334 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:16.333+0000 7fd59cff9640 1 -- 172.21.15.139:0/457183132 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 0.5229187413364552}]=0 reweighted osd.1 to 0.522919 (85de) v106) ==== 134+0+0 (secure 0 0 0) 0x7fd59005cac0 con 0x7fd5a00c63c0 2024-09-16T00:10:16.334 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.1 to 0.522919 (85de) 2024-09-16T00:10:16.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:16.337+0000 7fd5a62d7640 1 -- 172.21.15.139:0/457183132 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd58c06c2e0 msgr2=0x7fd58c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:16.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:16.337+0000 7fd5a62d7640 1 --2- 172.21.15.139:0/457183132 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd58c06c2e0 0x7fd58c06e7a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd5a01460e0 tx=0x7fd594009190 comp rx=0 tx=0).stop 2024-09-16T00:10:16.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:16.337+0000 7fd5a62d7640 1 -- 172.21.15.139:0/457183132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a00c63c0 msgr2=0x7fd5a014e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:16.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:16.337+0000 7fd5a62d7640 1 --2- 172.21.15.139:0/457183132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a00c63c0 0x7fd5a014e400 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd59000c4d0 tx=0x7fd59000c990 comp rx=0 tx=0).stop 2024-09-16T00:10:16.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:16.337+0000 7fd5a62d7640 1 -- 172.21.15.139:0/457183132 shutdown_connections 2024-09-16T00:10:16.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:16.337+0000 7fd5a62d7640 1 --2- 172.21.15.139:0/457183132 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd58c06c2e0 0x7fd58c06e7a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:16.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:16.337+0000 7fd5a62d7640 1 --2- 172.21.15.139:0/457183132 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd5a0151be0 0x7fd5a0147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:16.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:16.337+0000 7fd5a62d7640 1 --2- 172.21.15.139:0/457183132 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd5a0151800 0x7fd5a014db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:16.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:16.337+0000 7fd5a62d7640 1 --2- 172.21.15.139:0/457183132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd5a00c63c0 0x7fd5a014e400 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:16.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:16.337+0000 7fd5a62d7640 1 -- 172.21.15.139:0/457183132 >> 172.21.15.139:0/457183132 conn(0x7fd5a006da40 msgr2=0x7fd5a00c3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:16.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:16.337+0000 7fd5a62d7640 1 -- 172.21.15.139:0/457183132 shutdown_connections 2024-09-16T00:10:16.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:16.337+0000 7fd5a62d7640 1 -- 172.21.15.139:0/457183132 wait complete. 2024-09-16T00:10:16.347 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:16.346+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:16.416 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:16.416 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:16.416 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:16.425 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:10:16.448 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:16.447+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:16.530 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:16.531 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:16.531 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:16.531 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:16.531 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.531 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5850 3.3a 3:5c3cfc2b:::10000001adf.00000000:head [write 0~116] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:16.531 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:16.531 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 61.112832783999998, 2024-09-16T00:10:16.531 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66931295800000001, 2024-09-16T00:10:16.531 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.531 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.531 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.532 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.532 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.532 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.532 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5850 2024-09-16T00:10:16.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.532 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.532 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.532 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.532 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:16.532 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.532 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.532 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:16.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.533 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.533 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.533 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415938+0000", 2024-09-16T00:10:16.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7099999999999999e-06 2024-09-16T00:10:16.533 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.533 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.533 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415945+0000", 2024-09-16T00:10:16.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:10:16.533 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.533 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.534 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.534 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415959+0000", 2024-09-16T00:10:16.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3545e-05 2024-09-16T00:10:16.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.534 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.534 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.534 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415966+0000", 2024-09-16T00:10:16.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6379999999999998e-06 2024-09-16T00:10:16.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.534 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.534 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.534 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416013+0000", 2024-09-16T00:10:16.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7314999999999997e-05 2024-09-16T00:10:16.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.535 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.535 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:16.535 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416020+0000", 2024-09-16T00:10:16.535 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.52e-06 2024-09-16T00:10:16.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.535 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.535 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.535 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082393+0000", 2024-09-16T00:10:16.535 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66637251799999997 2024-09-16T00:10:16.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.535 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.535 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.535 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082537+0000", 2024-09-16T00:10:16.535 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014418800000000001 2024-09-16T00:10:16.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.536 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.536 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:16.536 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082689+0000", 2024-09-16T00:10:16.536 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015198499999999999 2024-09-16T00:10:16.536 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.536 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.536 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.536 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084930+0000", 2024-09-16T00:10:16.536 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002241041 2024-09-16T00:10:16.536 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.536 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.536 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.536 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084994+0000", 2024-09-16T00:10:16.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3274000000000004e-05 2024-09-16T00:10:16.537 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.537 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.537 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.537 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085049+0000", 2024-09-16T00:10:16.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.554e-05 2024-09-16T00:10:16.537 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.537 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.537 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.537 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085250+0000", 2024-09-16T00:10:16.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020060799999999999 2024-09-16T00:10:16.537 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.537 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.538 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.538 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.538 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.538 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5910 3.3a 3:5ec98a62:::10000001b22.00000000:head [write 0~1274] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:16.538 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:16.538 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.965889146000002, 2024-09-16T00:10:16.538 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.52286026600000002, 2024-09-16T00:10:16.538 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.538 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.538 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.538 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.538 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.539 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.539 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5910 2024-09-16T00:10:16.539 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.539 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.539 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.539 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.539 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:16.539 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.539 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.539 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.539 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.539 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:16.539 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.539 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.540 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.540 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.540 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562881+0000", 2024-09-16T00:10:16.540 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:10:16.540 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.540 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.540 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.540 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562887+0000", 2024-09-16T00:10:16.540 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2870000000000004e-06 2024-09-16T00:10:16.540 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.540 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.540 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.540 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562897+0000", 2024-09-16T00:10:16.541 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.064e-05 2024-09-16T00:10:16.541 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.541 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.541 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.541 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562902+0000", 2024-09-16T00:10:16.541 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2340000000000002e-06 2024-09-16T00:10:16.541 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.541 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.541 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.541 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562958+0000", 2024-09-16T00:10:16.541 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5523999999999999e-05 2024-09-16T00:10:16.541 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.541 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.541 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:16.542 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562968+0000", 2024-09-16T00:10:16.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0394000000000001e-05 2024-09-16T00:10:16.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.542 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.542 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083461+0000", 2024-09-16T00:10:16.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.520492968 2024-09-16T00:10:16.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.542 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.542 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083645+0000", 2024-09-16T00:10:16.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018346399999999999 2024-09-16T00:10:16.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.543 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:16.543 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083844+0000", 2024-09-16T00:10:16.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000199268 2024-09-16T00:10:16.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.543 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.543 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085410+0000", 2024-09-16T00:10:16.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015662359999999999 2024-09-16T00:10:16.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.543 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.543 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085493+0000", 2024-09-16T00:10:16.544 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2478999999999994e-05 2024-09-16T00:10:16.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085563+0000", 2024-09-16T00:10:16.544 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0159000000000004e-05 2024-09-16T00:10:16.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085741+0000", 2024-09-16T00:10:16.544 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017764300000000001 2024-09-16T00:10:16.544 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.544 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.545 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.545 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.545 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5962 3.3a 3:5c7b4d29:::10000001b5d.00000000:head [write 0~3074] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:16.545 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:16.545 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.836352329, 2024-09-16T00:10:16.545 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39368737500000001, 2024-09-16T00:10:16.545 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.545 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.545 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.545 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.545 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.546 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.546 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5962 2024-09-16T00:10:16.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.546 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.546 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.546 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:16.546 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.546 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.546 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:16.546 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.547 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.547 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.547 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.547 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692418+0000", 2024-09-16T00:10:16.547 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3510000000000001e-06 2024-09-16T00:10:16.547 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.547 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.547 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.547 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692425+0000", 2024-09-16T00:10:16.547 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3770000000000001e-06 2024-09-16T00:10:16.547 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.547 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.547 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.547 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692440+0000", 2024-09-16T00:10:16.548 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4975e-05 2024-09-16T00:10:16.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.548 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.548 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692448+0000", 2024-09-16T00:10:16.548 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6279999999999994e-06 2024-09-16T00:10:16.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.548 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.548 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692499+0000", 2024-09-16T00:10:16.548 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0872999999999999e-05 2024-09-16T00:10:16.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.548 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:16.549 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692507+0000", 2024-09-16T00:10:16.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9889999999999992e-06 2024-09-16T00:10:16.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.549 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084221+0000", 2024-09-16T00:10:16.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39171379699999997 2024-09-16T00:10:16.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.549 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084446+0000", 2024-09-16T00:10:16.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022519900000000001 2024-09-16T00:10:16.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:16.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084672+0000", 2024-09-16T00:10:16.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000225405 2024-09-16T00:10:16.550 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.550 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085832+0000", 2024-09-16T00:10:16.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011600040000000001 2024-09-16T00:10:16.550 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.550 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085864+0000", 2024-09-16T00:10:16.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2592999999999998e-05 2024-09-16T00:10:16.550 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.551 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.551 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085943+0000", 2024-09-16T00:10:16.551 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8534000000000004e-05 2024-09-16T00:10:16.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.551 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.551 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.551 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086105+0000", 2024-09-16T00:10:16.551 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016165 2024-09-16T00:10:16.551 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.551 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.551 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.551 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.551 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5963 3.3a 3:5e4192f5:::10000001b5e.00000000:head [write 0~152] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:16.552 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:16.552 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.834287897999999, 2024-09-16T00:10:16.552 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39413463900000001, 2024-09-16T00:10:16.552 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.552 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.552 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.552 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.552 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.552 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.552 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5963 2024-09-16T00:10:16.552 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.552 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.552 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.552 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:16.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.553 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:16.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.553 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:16.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:10:16.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.553 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.554 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694486+0000", 2024-09-16T00:10:16.554 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0960000000000003e-06 2024-09-16T00:10:16.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.554 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.554 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694496+0000", 2024-09-16T00:10:16.554 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2189999999999993e-06 2024-09-16T00:10:16.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.554 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.554 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694500+0000", 2024-09-16T00:10:16.554 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9529999999999998e-06 2024-09-16T00:10:16.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.554 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.555 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694552+0000", 2024-09-16T00:10:16.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1360999999999998e-05 2024-09-16T00:10:16.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.555 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.555 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:16.555 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694561+0000", 2024-09-16T00:10:16.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1700000000000003e-06 2024-09-16T00:10:16.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.555 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.555 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.555 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086346+0000", 2024-09-16T00:10:16.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39178468 2024-09-16T00:10:16.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.555 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.556 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.556 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086480+0000", 2024-09-16T00:10:16.556 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013469299999999999 2024-09-16T00:10:16.556 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.556 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.556 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:16.556 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086790+0000", 2024-09-16T00:10:16.556 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030923300000000001 2024-09-16T00:10:16.556 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.556 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.556 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.556 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088361+0000", 2024-09-16T00:10:16.556 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015717050000000001 2024-09-16T00:10:16.556 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.556 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088432+0000", 2024-09-16T00:10:16.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0922999999999998e-05 2024-09-16T00:10:16.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088489+0000", 2024-09-16T00:10:16.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6653999999999997e-05 2024-09-16T00:10:16.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088616+0000", 2024-09-16T00:10:16.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012730099999999999 2024-09-16T00:10:16.557 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.558 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.558 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.558 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:6082 3.3a 3:5c50f65b:::10000001be7.00000000:head [write 0~2439] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:16.558 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:16.558 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.540515042000003, 2024-09-16T00:10:16.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.10065850799999999, 2024-09-16T00:10:16.558 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.558 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.558 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.558 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.558 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.559 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.559 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 6082 2024-09-16T00:10:16.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.559 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.559 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:16.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.559 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:16.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988256+0000", 2024-09-16T00:10:16.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.065e-06 2024-09-16T00:10:16.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988262+0000", 2024-09-16T00:10:16.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4640000000000003e-06 2024-09-16T00:10:16.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988272+0000", 2024-09-16T00:10:16.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0335e-05 2024-09-16T00:10:16.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.561 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988279+0000", 2024-09-16T00:10:16.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:10:16.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.561 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988324+0000", 2024-09-16T00:10:16.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4953000000000001e-05 2024-09-16T00:10:16.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.562 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.562 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:16.562 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988331+0000", 2024-09-16T00:10:16.562 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5989999999999998e-06 2024-09-16T00:10:16.562 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.562 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.562 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.562 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087272+0000", 2024-09-16T00:10:16.562 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.098941045000000005 2024-09-16T00:10:16.562 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.562 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.562 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.562 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087486+0000", 2024-09-16T00:10:16.562 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00021360199999999999 2024-09-16T00:10:16.562 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.563 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:16.563 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087740+0000", 2024-09-16T00:10:16.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00025456799999999999 2024-09-16T00:10:16.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.563 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.563 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088638+0000", 2024-09-16T00:10:16.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00089793900000000005 2024-09-16T00:10:16.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.563 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.563 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088673+0000", 2024-09-16T00:10:16.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5111999999999999e-05 2024-09-16T00:10:16.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.564 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088722+0000", 2024-09-16T00:10:16.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8758000000000003e-05 2024-09-16T00:10:16.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.564 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088913+0000", 2024-09-16T00:10:16.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000190953 2024-09-16T00:10:16.564 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.564 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.564 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.565 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.565 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10643 3.21 3:8453163e:::100000030b3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:16.565 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:16.565 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.929038849000001, 2024-09-16T00:10:16.565 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013028809000000001, 2024-09-16T00:10:16.565 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.565 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.565 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.565 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.565 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.565 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.565 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10643 2024-09-16T00:10:16.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.565 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.566 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.566 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:16.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.566 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.566 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:16.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.566 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.566 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599734+0000", 2024-09-16T00:10:16.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-06 2024-09-16T00:10:16.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603345+0000", 2024-09-16T00:10:16.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003611628 2024-09-16T00:10:16.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603368+0000", 2024-09-16T00:10:16.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3017e-05 2024-09-16T00:10:16.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.568 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603379+0000", 2024-09-16T00:10:16.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0528e-05 2024-09-16T00:10:16.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.568 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.568 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603448+0000", 2024-09-16T00:10:16.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9080999999999998e-05 2024-09-16T00:10:16.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.568 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.568 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603623+0000", 2024-09-16T00:10:16.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017525599999999999 2024-09-16T00:10:16.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:16.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603766+0000", 2024-09-16T00:10:16.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000142977 2024-09-16T00:10:16.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.611250+0000", 2024-09-16T00:10:16.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074835240000000001 2024-09-16T00:10:16.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612617+0000", 2024-09-16T00:10:16.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001367793 2024-09-16T00:10:16.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612700+0000", 2024-09-16T00:10:16.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3123999999999999e-05 2024-09-16T00:10:16.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612759+0000", 2024-09-16T00:10:16.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8950999999999999e-05 2024-09-16T00:10:16.570 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.570 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.570 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.571 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10652 3.22 3:44e46336:::100000030c1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:16.571 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:10:16.571 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.900987217000001, 2024-09-16T00:10:16.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011405732, 2024-09-16T00:10:16.571 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.571 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.571 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.571 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.571 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.571 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.571 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10652 2024-09-16T00:10:16.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.571 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:10:16.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:10:16.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627785+0000", 2024-09-16T00:10:16.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6970000000000002e-06 2024-09-16T00:10:16.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.573 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.573 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629610+0000", 2024-09-16T00:10:16.573 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018250359999999999 2024-09-16T00:10:16.573 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.573 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.573 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.573 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629639+0000", 2024-09-16T00:10:16.573 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8676000000000001e-05 2024-09-16T00:10:16.573 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.573 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.573 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.573 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629652+0000", 2024-09-16T00:10:16.573 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3572e-05 2024-09-16T00:10:16.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.574 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629735+0000", 2024-09-16T00:10:16.574 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.25e-05 2024-09-16T00:10:16.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.574 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629925+0000", 2024-09-16T00:10:16.574 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019011000000000001 2024-09-16T00:10:16.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:10:16.574 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.630075+0000", 2024-09-16T00:10:16.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000149841 2024-09-16T00:10:16.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.636470+0000", 2024-09-16T00:10:16.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063952289999999997 2024-09-16T00:10:16.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639098+0000", 2024-09-16T00:10:16.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026277980000000002 2024-09-16T00:10:16.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.576 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639154+0000", 2024-09-16T00:10:16.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6413999999999999e-05 2024-09-16T00:10:16.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.576 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.576 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.576 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639188+0000", 2024-09-16T00:10:16.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3859000000000002e-05 2024-09-16T00:10:16.576 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.576 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.576 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.576 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.576 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11217 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:16.576 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:16.577 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 35.954490196000002, 2024-09-16T00:10:16.577 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01231253, 2024-09-16T00:10:16.577 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.577 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.577 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.577 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.577 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.577 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.577 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11217 2024-09-16T00:10:16.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.577 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:16.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:16.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574282+0000", 2024-09-16T00:10:16.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8499999999999998e-06 2024-09-16T00:10:16.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575804+0000", 2024-09-16T00:10:16.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001521859 2024-09-16T00:10:16.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575826+0000", 2024-09-16T00:10:16.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2059999999999999e-05 2024-09-16T00:10:16.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575837+0000", 2024-09-16T00:10:16.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1313e-05 2024-09-16T00:10:16.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575920+0000", 2024-09-16T00:10:16.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2918999999999997e-05 2024-09-16T00:10:16.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576088+0000", 2024-09-16T00:10:16.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016803799999999999 2024-09-16T00:10:16.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:16.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576233+0000", 2024-09-16T00:10:16.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014508199999999999 2024-09-16T00:10:16.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.581 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.581 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.580500+0000", 2024-09-16T00:10:16.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004266644 2024-09-16T00:10:16.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.581 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.581 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586479+0000", 2024-09-16T00:10:16.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059791820000000004 2024-09-16T00:10:16.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.582 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.582 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586541+0000", 2024-09-16T00:10:16.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1463000000000004e-05 2024-09-16T00:10:16.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.582 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.582 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586592+0000", 2024-09-16T00:10:16.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1119999999999998e-05 2024-09-16T00:10:16.582 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.582 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.583 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.583 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11767 3.38 3:1fa50e65:::10000003117.00000000:head [write 0~4194304 [3@1169732]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:16.583 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:16.583 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.365085661000002, 2024-09-16T00:10:16.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011555678999999999, 2024-09-16T00:10:16.583 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.583 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.583 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.583 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.583 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.584 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.584 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11767 2024-09-16T00:10:16.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.584 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.584 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.584 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:16.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.584 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.584 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:16.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163686+0000", 2024-09-16T00:10:16.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3259999999999998e-06 2024-09-16T00:10:16.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165053+0000", 2024-09-16T00:10:16.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013665859999999999 2024-09-16T00:10:16.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165075+0000", 2024-09-16T00:10:16.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2206999999999999e-05 2024-09-16T00:10:16.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165084+0000", 2024-09-16T00:10:16.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0710000000000001e-06 2024-09-16T00:10:16.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.587 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.587 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165157+0000", 2024-09-16T00:10:16.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2619000000000004e-05 2024-09-16T00:10:16.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.587 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.587 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165249+0000", 2024-09-16T00:10:16.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2517999999999997e-05 2024-09-16T00:10:16.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:16.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165360+0000", 2024-09-16T00:10:16.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011134899999999999 2024-09-16T00:10:16.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.169320+0000", 2024-09-16T00:10:16.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039593040000000003 2024-09-16T00:10:16.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175135+0000", 2024-09-16T00:10:16.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005814891 2024-09-16T00:10:16.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175199+0000", 2024-09-16T00:10:16.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4474999999999995e-05 2024-09-16T00:10:16.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175239+0000", 2024-09-16T00:10:16.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0333000000000002e-05 2024-09-16T00:10:16.589 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.589 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.590 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.590 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11789 3.21 3:8453163e:::100000030b3.00000000:head [write 0~3801088 [4@1194504]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:16.590 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:10:16.590 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.277397565000001, 2024-09-16T00:10:16.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011097602, 2024-09-16T00:10:16.590 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.590 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.590 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.590 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.590 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.590 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.590 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11789 2024-09-16T00:10:16.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.591 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:10:16.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:10:16.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.592 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251373+0000", 2024-09-16T00:10:16.592 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.544e-06 2024-09-16T00:10:16.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.592 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.592 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252710+0000", 2024-09-16T00:10:16.592 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013362929999999999 2024-09-16T00:10:16.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.592 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.592 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252730+0000", 2024-09-16T00:10:16.592 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0579999999999999e-05 2024-09-16T00:10:16.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.593 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.593 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.593 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252741+0000", 2024-09-16T00:10:16.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0505e-05 2024-09-16T00:10:16.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.593 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.593 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.593 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252820+0000", 2024-09-16T00:10:16.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9098999999999996e-05 2024-09-16T00:10:16.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.593 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.593 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.593 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252931+0000", 2024-09-16T00:10:16.594 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011073600000000001 2024-09-16T00:10:16.594 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.594 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.594 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:16.594 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.253101+0000", 2024-09-16T00:10:16.594 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017038000000000001 2024-09-16T00:10:16.594 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.594 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.594 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.594 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.261813+0000", 2024-09-16T00:10:16.594 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087119469999999994 2024-09-16T00:10:16.594 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.594 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.594 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.595 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262338+0000", 2024-09-16T00:10:16.595 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00052493799999999997 2024-09-16T00:10:16.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.595 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.595 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.595 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262430+0000", 2024-09-16T00:10:16.595 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1626999999999995e-05 2024-09-16T00:10:16.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.595 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.595 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.595 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262470+0000", 2024-09-16T00:10:16.595 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9953000000000002e-05 2024-09-16T00:10:16.595 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.595 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.596 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.596 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.596 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.596 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11890 3.38 3:1e014616:::100000033e9.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:16.596 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:16.596 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.957432482999998, 2024-09-16T00:10:16.596 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011833791999999999, 2024-09-16T00:10:16.596 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.596 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.596 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.596 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.596 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.596 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.596 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11890 2024-09-16T00:10:16.597 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.597 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.597 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.597 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:16.597 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.597 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.597 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.597 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:16.597 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.597 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.597 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.598 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571340+0000", 2024-09-16T00:10:16.598 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.36e-06 2024-09-16T00:10:16.598 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.598 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.598 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.598 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572866+0000", 2024-09-16T00:10:16.598 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001525549 2024-09-16T00:10:16.598 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.598 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.598 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.598 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572884+0000", 2024-09-16T00:10:16.598 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7796999999999999e-05 2024-09-16T00:10:16.598 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.598 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.599 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.599 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572897+0000", 2024-09-16T00:10:16.599 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3567000000000001e-05 2024-09-16T00:10:16.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.599 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.599 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572970+0000", 2024-09-16T00:10:16.599 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2751999999999997e-05 2024-09-16T00:10:16.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.599 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.599 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573053+0000", 2024-09-16T00:10:16.599 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3230000000000001e-05 2024-09-16T00:10:16.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:16.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573168+0000", 2024-09-16T00:10:16.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114536 2024-09-16T00:10:16.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.577923+0000", 2024-09-16T00:10:16.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047548720000000003 2024-09-16T00:10:16.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583081+0000", 2024-09-16T00:10:16.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051587630000000002 2024-09-16T00:10:16.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.601 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.601 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583137+0000", 2024-09-16T00:10:16.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5259000000000001e-05 2024-09-16T00:10:16.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.601 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.601 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583171+0000", 2024-09-16T00:10:16.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4106999999999997e-05 2024-09-16T00:10:16.601 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.601 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.601 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.602 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:12037 3.9 e87/20 3:928db521:::1000000326d.00000000:head v 87'199, mlcod=87'197)", 2024-09-16T00:10:16.602 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:16.602 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.526242531999998, 2024-09-16T00:10:16.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012236153, 2024-09-16T00:10:16.602 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.602 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.602 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.602 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:16.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.603 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.603 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:16.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.603 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.603 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002529+0000", 2024-09-16T00:10:16.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4889999999999998e-06 2024-09-16T00:10:16.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.603 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.603 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007064+0000", 2024-09-16T00:10:16.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045346220000000003 2024-09-16T00:10:16.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.604 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.604 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007093+0000", 2024-09-16T00:10:16.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9009e-05 2024-09-16T00:10:16.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.604 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.604 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007109+0000", 2024-09-16T00:10:16.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5846e-05 2024-09-16T00:10:16.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.604 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007196+0000", 2024-09-16T00:10:16.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7507000000000001e-05 2024-09-16T00:10:16.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007227+0000", 2024-09-16T00:10:16.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0916e-05 2024-09-16T00:10:16.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014658+0000", 2024-09-16T00:10:16.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074305090000000001 2024-09-16T00:10:16.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014763+0000", 2024-09-16T00:10:16.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010525499999999999 2024-09-16T00:10:16.606 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.606 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.606 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.606 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12398 3.2e 3:77a11d08:::10000003432.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:16.606 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:16.606 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.287563749, 2024-09-16T00:10:16.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012092999, 2024-09-16T00:10:16.606 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.607 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.607 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.607 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.607 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.607 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.607 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12398 2024-09-16T00:10:16.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.607 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.607 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.607 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:16.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.608 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:16.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.608 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241209+0000", 2024-09-16T00:10:16.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2710000000000001e-06 2024-09-16T00:10:16.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243840+0000", 2024-09-16T00:10:16.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026307919999999999 2024-09-16T00:10:16.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243872+0000", 2024-09-16T00:10:16.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2699e-05 2024-09-16T00:10:16.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243887+0000", 2024-09-16T00:10:16.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4348000000000001e-05 2024-09-16T00:10:16.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243972+0000", 2024-09-16T00:10:16.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5358999999999994e-05 2024-09-16T00:10:16.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244132+0000", 2024-09-16T00:10:16.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015972499999999999 2024-09-16T00:10:16.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253153+0000", 2024-09-16T00:10:16.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0090210280000000004 2024-09-16T00:10:16.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253213+0000", 2024-09-16T00:10:16.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0300999999999997e-05 2024-09-16T00:10:16.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253299+0000", 2024-09-16T00:10:16.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5476000000000006e-05 2024-09-16T00:10:16.612 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.612 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.612 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.612 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12400 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304 [2@815045]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:16.612 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:10:16.612 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.284526015000001, 2024-09-16T00:10:16.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011140066000000001, 2024-09-16T00:10:16.612 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.612 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.612 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.612 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.612 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.613 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.613 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12400 2024-09-16T00:10:16.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.613 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.613 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.613 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:10:16.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.613 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.613 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:10:16.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244245+0000", 2024-09-16T00:10:16.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.248e-06 2024-09-16T00:10:16.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246467+0000", 2024-09-16T00:10:16.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022217949999999999 2024-09-16T00:10:16.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246496+0000", 2024-09-16T00:10:16.615 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9484000000000001e-05 2024-09-16T00:10:16.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.615 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.615 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246509+0000", 2024-09-16T00:10:16.615 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3390000000000001e-05 2024-09-16T00:10:16.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.615 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.615 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246586+0000", 2024-09-16T00:10:16.615 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6692000000000004e-05 2024-09-16T00:10:16.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.615 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246680+0000", 2024-09-16T00:10:16.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3468999999999996e-05 2024-09-16T00:10:16.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:16.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246850+0000", 2024-09-16T00:10:16.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170206 2024-09-16T00:10:16.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253478+0000", 2024-09-16T00:10:16.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066287530000000002 2024-09-16T00:10:16.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255318+0000", 2024-09-16T00:10:16.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001839292 2024-09-16T00:10:16.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.617 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255354+0000", 2024-09-16T00:10:16.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5964999999999999e-05 2024-09-16T00:10:16.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.618 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.618 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255384+0000", 2024-09-16T00:10:16.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9771999999999999e-05 2024-09-16T00:10:16.618 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.618 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.618 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.618 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.618 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.618 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13317 3.38 3:1cda8244:::100000035dc.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:16.618 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:10:16.618 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 28.610171488999999, 2024-09-16T00:10:16.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011377063, 2024-09-16T00:10:16.619 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.619 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.619 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.619 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.619 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.619 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.619 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13317 2024-09-16T00:10:16.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.619 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.619 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.619 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:10:16.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:10:16.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918602+0000", 2024-09-16T00:10:16.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8160000000000004e-06 2024-09-16T00:10:16.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920364+0000", 2024-09-16T00:10:16.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017623630000000001 2024-09-16T00:10:16.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920391+0000", 2024-09-16T00:10:16.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7248000000000001e-05 2024-09-16T00:10:16.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920405+0000", 2024-09-16T00:10:16.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3969e-05 2024-09-16T00:10:16.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920486+0000", 2024-09-16T00:10:16.622 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0737999999999998e-05 2024-09-16T00:10:16.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.622 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.622 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920594+0000", 2024-09-16T00:10:16.622 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108269 2024-09-16T00:10:16.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.622 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:16.622 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920730+0000", 2024-09-16T00:10:16.622 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000135229 2024-09-16T00:10:16.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.928847+0000", 2024-09-16T00:10:16.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081178299999999995 2024-09-16T00:10:16.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929862+0000", 2024-09-16T00:10:16.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010147540000000001 2024-09-16T00:10:16.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929923+0000", 2024-09-16T00:10:16.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0452999999999999e-05 2024-09-16T00:10:16.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.624 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929975+0000", 2024-09-16T00:10:16.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2394e-05 2024-09-16T00:10:16.624 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.624 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.624 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.624 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:13326 3.9 e88/20 3:92f796cf:::1000000350f.00000000:head v 88'214, mlcod=88'212)", 2024-09-16T00:10:16.624 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:10:16.625 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 28.567650698000001, 2024-09-16T00:10:16.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01100597, 2024-09-16T00:10:16.625 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.625 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.625 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.625 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.625 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:10:16.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.625 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:10:16.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.626 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961121+0000", 2024-09-16T00:10:16.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.815e-06 2024-09-16T00:10:16.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.626 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965747+0000", 2024-09-16T00:10:16.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0046260570000000003 2024-09-16T00:10:16.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.626 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.627 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965780+0000", 2024-09-16T00:10:16.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2851000000000001e-05 2024-09-16T00:10:16.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.627 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965788+0000", 2024-09-16T00:10:16.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0180000000000001e-06 2024-09-16T00:10:16.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.627 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965859+0000", 2024-09-16T00:10:16.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1318999999999999e-05 2024-09-16T00:10:16.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965886+0000", 2024-09-16T00:10:16.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7509999999999999e-05 2024-09-16T00:10:16.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.972066+0000", 2024-09-16T00:10:16.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061800759999999996 2024-09-16T00:10:16.628 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.628 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.972125+0000", 2024-09-16T00:10:16.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8323999999999999e-05 2024-09-16T00:10:16.629 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.629 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.629 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.629 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13942 3.22 3:44ed4521:::100000036d5.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:16.629 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:10:16.629 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.606285386, 2024-09-16T00:10:16.629 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011341061, 2024-09-16T00:10:16.629 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.629 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.629 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.630 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.630 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.630 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.630 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13942 2024-09-16T00:10:16.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.630 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.630 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:10:16.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.630 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.630 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.630 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.630 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:10:16.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.631 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.631 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922487+0000", 2024-09-16T00:10:16.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0180000000000002e-06 2024-09-16T00:10:16.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.631 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.631 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923720+0000", 2024-09-16T00:10:16.631 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012333470000000001 2024-09-16T00:10:16.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.631 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.631 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.632 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923749+0000", 2024-09-16T00:10:16.632 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8748999999999999e-05 2024-09-16T00:10:16.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.632 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.632 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923770+0000", 2024-09-16T00:10:16.632 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1189999999999999e-05 2024-09-16T00:10:16.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.632 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.632 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923868+0000", 2024-09-16T00:10:16.632 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7675000000000002e-05 2024-09-16T00:10:16.632 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.924047+0000", 2024-09-16T00:10:16.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017935100000000001 2024-09-16T00:10:16.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:10:16.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.924223+0000", 2024-09-16T00:10:16.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000175661 2024-09-16T00:10:16.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.931544+0000", 2024-09-16T00:10:16.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073204589999999996 2024-09-16T00:10:16.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.634 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933692+0000", 2024-09-16T00:10:16.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021488760000000001 2024-09-16T00:10:16.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.634 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933774+0000", 2024-09-16T00:10:16.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1855999999999997e-05 2024-09-16T00:10:16.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933825+0000", 2024-09-16T00:10:16.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0878999999999999e-05 2024-09-16T00:10:16.635 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.635 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.635 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.635 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14867 3.1f 3:faedf583:::1000000393b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:16.635 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:10:16.635 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.781795649999999, 2024-09-16T00:10:16.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011128677999999999, 2024-09-16T00:10:16.635 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.635 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.635 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:16.636 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.636 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.636 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.636 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14867 2024-09-16T00:10:16.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.636 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:10:16.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:10:16.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746976+0000", 2024-09-16T00:10:16.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1169999999999998e-06 2024-09-16T00:10:16.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748348+0000", 2024-09-16T00:10:16.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013721860000000001 2024-09-16T00:10:16.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748371+0000", 2024-09-16T00:10:16.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2938999999999998e-05 2024-09-16T00:10:16.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748381+0000", 2024-09-16T00:10:16.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5300000000000002e-06 2024-09-16T00:10:16.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748458+0000", 2024-09-16T00:10:16.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7389000000000001e-05 2024-09-16T00:10:16.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748626+0000", 2024-09-16T00:10:16.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016757499999999999 2024-09-16T00:10:16.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:16.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748813+0000", 2024-09-16T00:10:16.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018743399999999999 2024-09-16T00:10:16.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:16.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.753998+0000", 2024-09-16T00:10:16.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051851140000000002 2024-09-16T00:10:16.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:16.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.757990+0000", 2024-09-16T00:10:16.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039914169999999997 2024-09-16T00:10:16.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:16.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.758054+0000", 2024-09-16T00:10:16.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4513999999999999e-05 2024-09-16T00:10:16.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.758103+0000", 2024-09-16T00:10:16.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8463e-05 2024-09-16T00:10:16.641 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.641 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.641 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.641 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15003 3.21 3:8673bb5f:::100000030ed.00000000:head [read 2097152~2097152 [2@1395527]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:16.641 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:16.641 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.360373655, 2024-09-16T00:10:16.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058433321000000003, 2024-09-16T00:10:16.641 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.641 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.641 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:10:16.641 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.642 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.642 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.642 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15003 2024-09-16T00:10:16.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.642 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:16.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.642 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.642 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.642 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:16.642 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.642 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168397+0000", 2024-09-16T00:10:16.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3900000000000004e-07 2024-09-16T00:10:16.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168401+0000", 2024-09-16T00:10:16.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4050000000000002e-06 2024-09-16T00:10:16.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168411+0000", 2024-09-16T00:10:16.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6539999999999994e-06 2024-09-16T00:10:16.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168417+0000", 2024-09-16T00:10:16.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6719999999999998e-06 2024-09-16T00:10:16.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168467+0000", 2024-09-16T00:10:16.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9815999999999999e-05 2024-09-16T00:10:16.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168535+0000", 2024-09-16T00:10:16.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7669999999999994e-05 2024-09-16T00:10:16.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.645 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.226829+0000", 2024-09-16T00:10:16.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058294264999999998 2024-09-16T00:10:16.645 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.645 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.645 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.645 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15005 3.1f 3:f8460f10:::10000003370.00000000:head [read 1048576~1048576 [2@4194304]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:16.645 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:16.645 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.358692943000001, 2024-09-16T00:10:16.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.055844088, 2024-09-16T00:10:16.646 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.646 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.646 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:10:16.646 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:16.646 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:16.646 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:16.646 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15005 2024-09-16T00:10:16.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.646 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.646 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.646 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:16.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:16.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170078+0000", 2024-09-16T00:10:16.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.235e-06 2024-09-16T00:10:16.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170084+0000", 2024-09-16T00:10:16.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9739999999999999e-06 2024-09-16T00:10:16.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225526+0000", 2024-09-16T00:10:16.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.055441777999999997 2024-09-16T00:10:16.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225542+0000", 2024-09-16T00:10:16.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6606999999999999e-05 2024-09-16T00:10:16.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225668+0000", 2024-09-16T00:10:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012631 2024-09-16T00:10:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225744+0000", 2024-09-16T00:10:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5794e-05 2024-09-16T00:10:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225921+0000", 2024-09-16T00:10:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017639000000000001 2024-09-16T00:10:16.649 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.650 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.650 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.650 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.650 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.650 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:16.651 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:10:16.652 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:16.651+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:16.752 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:16.751+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:16.794 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:16.794 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:16.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.795 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9780 2.a 2.e3a83a2a (undecoded) ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e106)", 2024-09-16T00:10:16.795 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:16.795 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.456980002, 2024-09-16T00:10:16.795 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.45699924200000003, 2024-09-16T00:10:16.795 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:16.795 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:16.795 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "waiting for peered", 2024-09-16T00:10:16.795 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:16.795 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.795 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:16.795 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:16.795 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.795 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:16.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:16.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:16.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:16.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337345+0000", 2024-09-16T00:10:16.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.007e-06 2024-09-16T00:10:16.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:16.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337416+0000", 2024-09-16T00:10:16.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1260999999999994e-05 2024-09-16T00:10:16.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:16.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337427+0000", 2024-09-16T00:10:16.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1225e-05 2024-09-16T00:10:16.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:16.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337434+0000", 2024-09-16T00:10:16.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8929999999999999e-06 2024-09-16T00:10:16.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:16.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337494+0000", 2024-09-16T00:10:16.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9670999999999998e-05 2024-09-16T00:10:16.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:16.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:16.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:16.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337505+0000", 2024-09-16T00:10:16.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0934999999999999e-05 2024-09-16T00:10:16.798 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.798 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:16.798 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.798 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:16.798 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:10:16.798 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 1 2024-09-16T00:10:16.798 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:16.803 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:10:16.853 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:16.852+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:16.906 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:16.906 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:16.907 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:16.907 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:16.907 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:16.915 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:10:16.960 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:16.959+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:17.055 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:17.054+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:17.057 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:17.057 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:17.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:17.057 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:17.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.057 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:89 2.1b 2:ddb4e4d7:::200.00000002:head [write 2591630~1401 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:10:17.057 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:10:17.058 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 73.406127966, 2024-09-16T00:10:17.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011802609, 2024-09-16T00:10:17.058 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.058 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.058 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.058 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.058 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.058 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:10:17.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.058 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.058 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649080+0000", 2024-09-16T00:10:17.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.059 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649081+0000", 2024-09-16T00:10:17.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7100000000000002e-07 2024-09-16T00:10:17.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.059 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.059 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649089+0000", 2024-09-16T00:10:17.059 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7640000000000005e-06 2024-09-16T00:10:17.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.059 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.059 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649094+0000", 2024-09-16T00:10:17.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6219999999999997e-06 2024-09-16T00:10:17.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649099+0000", 2024-09-16T00:10:17.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6970000000000001e-06 2024-09-16T00:10:17.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655288+0000", 2024-09-16T00:10:17.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061885569999999999 2024-09-16T00:10:17.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655365+0000", 2024-09-16T00:10:17.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7723000000000001e-05 2024-09-16T00:10:17.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:17.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655483+0000", 2024-09-16T00:10:17.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117895 2024-09-16T00:10:17.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.062 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.062 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657758+0000", 2024-09-16T00:10:17.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022752369999999998 2024-09-16T00:10:17.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.062 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.062 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660803+0000", 2024-09-16T00:10:17.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030441499999999998 2024-09-16T00:10:17.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.062 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.062 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660850+0000", 2024-09-16T00:10:17.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7017000000000001e-05 2024-09-16T00:10:17.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660883+0000", 2024-09-16T00:10:17.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3476000000000003e-05 2024-09-16T00:10:17.064 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.064 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.064 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.064 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:90 2.1b 2:ddb4e4d7:::200.00000002:head [write 2593031~1404 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:10:17.064 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:17.064 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 73.406090061, 2024-09-16T00:10:17.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011882886, 2024-09-16T00:10:17.064 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.065 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.065 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.065 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:17.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:17.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649119+0000", 2024-09-16T00:10:17.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:10:17.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649125+0000", 2024-09-16T00:10:17.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1650000000000003e-06 2024-09-16T00:10:17.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649134+0000", 2024-09-16T00:10:17.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5669999999999995e-06 2024-09-16T00:10:17.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649141+0000", 2024-09-16T00:10:17.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9040000000000003e-06 2024-09-16T00:10:17.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655796+0000", 2024-09-16T00:10:17.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066553649999999999 2024-09-16T00:10:17.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.068 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.068 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.068 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655868+0000", 2024-09-16T00:10:17.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2219999999999996e-05 2024-09-16T00:10:17.068 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.068 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.068 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:17.068 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655986+0000", 2024-09-16T00:10:17.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117457 2024-09-16T00:10:17.068 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657794+0000", 2024-09-16T00:10:17.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001807894 2024-09-16T00:10:17.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660890+0000", 2024-09-16T00:10:17.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030958449999999998 2024-09-16T00:10:17.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.069 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.069 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660966+0000", 2024-09-16T00:10:17.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6382000000000002e-05 2024-09-16T00:10:17.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.661001+0000", 2024-09-16T00:10:17.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5349000000000003e-05 2024-09-16T00:10:17.070 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.070 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.070 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.070 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10851 3.29 3:9786b8f5:::10000003100.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:17.070 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:17.070 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 38.162426017000001, 2024-09-16T00:10:17.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012109089999999999, 2024-09-16T00:10:17.071 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.071 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.071 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.071 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:17.071 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:17.071 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.071 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10851 2024-09-16T00:10:17.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.071 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:17.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:17.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892786+0000", 2024-09-16T00:10:17.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4819999999999999e-06 2024-09-16T00:10:17.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894302+0000", 2024-09-16T00:10:17.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015156099999999999 2024-09-16T00:10:17.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.073 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.073 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894319+0000", 2024-09-16T00:10:17.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7098000000000001e-05 2024-09-16T00:10:17.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.073 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.073 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894329+0000", 2024-09-16T00:10:17.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9180000000000006e-06 2024-09-16T00:10:17.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894394+0000", 2024-09-16T00:10:17.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5797999999999996e-05 2024-09-16T00:10:17.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894532+0000", 2024-09-16T00:10:17.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000138128 2024-09-16T00:10:17.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:17.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894633+0000", 2024-09-16T00:10:17.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010037100000000001 2024-09-16T00:10:17.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.899212+0000", 2024-09-16T00:10:17.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045789369999999999 2024-09-16T00:10:17.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904771+0000", 2024-09-16T00:10:17.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055591169999999997 2024-09-16T00:10:17.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904838+0000", 2024-09-16T00:10:17.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7124000000000003e-05 2024-09-16T00:10:17.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.076 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.076 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904892+0000", 2024-09-16T00:10:17.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3507000000000001e-05 2024-09-16T00:10:17.077 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.077 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.077 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.077 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.077 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10957 3.24 3:2748b9d7:::1000000313b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:17.077 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:17.077 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.540521664000003, 2024-09-16T00:10:17.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012163525, 2024-09-16T00:10:17.077 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.077 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.077 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.077 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:17.077 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:17.078 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.078 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10957 2024-09-16T00:10:17.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.078 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:17.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:17.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.079 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514690+0000", 2024-09-16T00:10:17.079 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.382e-06 2024-09-16T00:10:17.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.079 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516204+0000", 2024-09-16T00:10:17.079 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001513831 2024-09-16T00:10:17.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.079 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516232+0000", 2024-09-16T00:10:17.079 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7657999999999998e-05 2024-09-16T00:10:17.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.080 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.080 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516246+0000", 2024-09-16T00:10:17.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3953e-05 2024-09-16T00:10:17.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.080 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.080 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516374+0000", 2024-09-16T00:10:17.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012882 2024-09-16T00:10:17.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.080 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516588+0000", 2024-09-16T00:10:17.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000213439 2024-09-16T00:10:17.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:17.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516748+0000", 2024-09-16T00:10:17.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015971900000000001 2024-09-16T00:10:17.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.524102+0000", 2024-09-16T00:10:17.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073542499999999997 2024-09-16T00:10:17.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526714+0000", 2024-09-16T00:10:17.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026122910000000001 2024-09-16T00:10:17.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526791+0000", 2024-09-16T00:10:17.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6997999999999997e-05 2024-09-16T00:10:17.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526850+0000", 2024-09-16T00:10:17.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9184000000000001e-05 2024-09-16T00:10:17.083 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.083 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.083 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.084 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11098 3.15 e86/20 3:aa8f498f:::100000030ba.00000000:head v 86'170, mlcod=86'168)", 2024-09-16T00:10:17.084 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:17.084 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.033738497999998, 2024-09-16T00:10:17.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013832641999999999, 2024-09-16T00:10:17.084 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.084 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.084 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.084 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.084 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.084 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.084 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:17.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.084 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.085 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:17.085 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.085 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021472+0000", 2024-09-16T00:10:17.085 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9489999999999999e-06 2024-09-16T00:10:17.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.085 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027628+0000", 2024-09-16T00:10:17.085 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061563859999999998 2024-09-16T00:10:17.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.086 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027645+0000", 2024-09-16T00:10:17.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7108e-05 2024-09-16T00:10:17.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.086 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.086 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027655+0000", 2024-09-16T00:10:17.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7999999999999993e-06 2024-09-16T00:10:17.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.086 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.087 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027712+0000", 2024-09-16T00:10:17.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6826000000000001e-05 2024-09-16T00:10:17.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.087 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.087 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027738+0000", 2024-09-16T00:10:17.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5601000000000001e-05 2024-09-16T00:10:17.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.087 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.087 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.035237+0000", 2024-09-16T00:10:17.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074998699999999996 2024-09-16T00:10:17.087 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.087 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.035303+0000", 2024-09-16T00:10:17.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5102000000000001e-05 2024-09-16T00:10:17.088 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.088 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.088 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.088 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11099 3.b 3:d2f5b389:::100000030e7.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:17.088 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:17.088 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.029693682999998, 2024-09-16T00:10:17.088 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012707456000000001, 2024-09-16T00:10:17.088 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.089 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.089 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.089 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:17.089 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:17.089 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.089 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11099 2024-09-16T00:10:17.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.089 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.089 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:17.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.090 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.090 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:17.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.090 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.090 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025517+0000", 2024-09-16T00:10:17.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:10:17.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.090 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.090 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.091 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026955+0000", 2024-09-16T00:10:17.091 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001437219 2024-09-16T00:10:17.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.091 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.091 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026982+0000", 2024-09-16T00:10:17.091 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7523999999999999e-05 2024-09-16T00:10:17.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.091 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.091 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026996+0000", 2024-09-16T00:10:17.091 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4379000000000001e-05 2024-09-16T00:10:17.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.092 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.092 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027076+0000", 2024-09-16T00:10:17.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9933999999999997e-05 2024-09-16T00:10:17.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.092 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.092 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027185+0000", 2024-09-16T00:10:17.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010875899999999999 2024-09-16T00:10:17.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:17.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027380+0000", 2024-09-16T00:10:17.093 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019477700000000001 2024-09-16T00:10:17.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.093 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038072+0000", 2024-09-16T00:10:17.093 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010692234 2024-09-16T00:10:17.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.093 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038115+0000", 2024-09-16T00:10:17.093 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3291999999999998e-05 2024-09-16T00:10:17.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.094 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.094 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.094 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038172+0000", 2024-09-16T00:10:17.094 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6981000000000002e-05 2024-09-16T00:10:17.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.094 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.094 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.094 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038222+0000", 2024-09-16T00:10:17.094 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9737000000000002e-05 2024-09-16T00:10:17.095 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.095 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.095 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.095 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.095 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.095 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11109 3.27 3:e5179c63:::10000003176.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:17.095 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:17.095 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.984229399999997, 2024-09-16T00:10:17.095 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012250990999999999, 2024-09-16T00:10:17.096 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.096 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.096 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.096 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:17.096 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:17.096 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.096 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11109 2024-09-16T00:10:17.096 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.096 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.096 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.096 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.097 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:17.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.097 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.097 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.097 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.097 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:17.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.097 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.097 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.097 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.097 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070981+0000", 2024-09-16T00:10:17.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.289e-06 2024-09-16T00:10:17.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072616+0000", 2024-09-16T00:10:17.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016347569999999999 2024-09-16T00:10:17.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072638+0000", 2024-09-16T00:10:17.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1512999999999999e-05 2024-09-16T00:10:17.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072649+0000", 2024-09-16T00:10:17.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1626e-05 2024-09-16T00:10:17.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.099 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072775+0000", 2024-09-16T00:10:17.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012553900000000001 2024-09-16T00:10:17.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.099 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072856+0000", 2024-09-16T00:10:17.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0882000000000003e-05 2024-09-16T00:10:17.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:17.100 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072990+0000", 2024-09-16T00:10:17.100 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013483599999999999 2024-09-16T00:10:17.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.100 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.078354+0000", 2024-09-16T00:10:17.100 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053639500000000001 2024-09-16T00:10:17.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083150+0000", 2024-09-16T00:10:17.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004795517 2024-09-16T00:10:17.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083186+0000", 2024-09-16T00:10:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6019000000000002e-05 2024-09-16T00:10:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083230+0000", 2024-09-16T00:10:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4063000000000001e-05 2024-09-16T00:10:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.102 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11234 3.b 3:d2f5b389:::100000030e7.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.431613632000001, 2024-09-16T00:10:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.015217661, 2024-09-16T00:10:17.103 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11234 2024-09-16T00:10:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.104 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:17.105 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623598+0000", 2024-09-16T00:10:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1580000000000001e-06 2024-09-16T00:10:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.106 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626844+0000", 2024-09-16T00:10:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032459670000000002 2024-09-16T00:10:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626868+0000", 2024-09-16T00:10:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3584e-05 2024-09-16T00:10:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.107 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626879+0000", 2024-09-16T00:10:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1545999999999999e-05 2024-09-16T00:10:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626950+0000", 2024-09-16T00:10:17.108 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0784999999999994e-05 2024-09-16T00:10:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.627051+0000", 2024-09-16T00:10:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010155899999999999 2024-09-16T00:10:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.109 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.627178+0000", 2024-09-16T00:10:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012614899999999999 2024-09-16T00:10:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.635220+0000", 2024-09-16T00:10:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0080421490000000002 2024-09-16T00:10:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638699+0000", 2024-09-16T00:10:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034797500000000002 2024-09-16T00:10:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638760+0000", 2024-09-16T00:10:17.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0319000000000003e-05 2024-09-16T00:10:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638812+0000", 2024-09-16T00:10:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:10:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11777 3.b 3:d342b4ff:::10000003113.00000000:head [write 0~4194304 [2@1171070]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.863974224000003, 2024-09-16T00:10:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013079176, 2024-09-16T00:10:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.113 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.114 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:17.114 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:17.114 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.114 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11777 2024-09-16T00:10:17.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.114 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.114 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:17.114 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:17.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191237+0000", 2024-09-16T00:10:17.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3369999999999998e-06 2024-09-16T00:10:17.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192825+0000", 2024-09-16T00:10:17.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015887189999999999 2024-09-16T00:10:17.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192846+0000", 2024-09-16T00:10:17.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0670999999999999e-05 2024-09-16T00:10:17.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192858+0000", 2024-09-16T00:10:17.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.223e-05 2024-09-16T00:10:17.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192957+0000", 2024-09-16T00:10:17.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8416000000000001e-05 2024-09-16T00:10:17.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.193031+0000", 2024-09-16T00:10:17.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4381999999999994e-05 2024-09-16T00:10:17.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.117 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:17.117 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.193164+0000", 2024-09-16T00:10:17.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013349100000000001 2024-09-16T00:10:17.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204144+0000", 2024-09-16T00:10:17.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010979917 2024-09-16T00:10:17.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204204+0000", 2024-09-16T00:10:17.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0025999999999998e-05 2024-09-16T00:10:17.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.119 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.119 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204265+0000", 2024-09-16T00:10:17.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0645e-05 2024-09-16T00:10:17.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.119 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.119 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204313+0000", 2024-09-16T00:10:17.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8341999999999999e-05 2024-09-16T00:10:17.119 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.119 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.120 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.120 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11981 3.c e87/20 3:33faa5a0:::10000003215.00000000:head v 87'186, mlcod=87'184)", 2024-09-16T00:10:17.120 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:17.120 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.223739401000003, 2024-09-16T00:10:17.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011828697000000001, 2024-09-16T00:10:17.120 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.120 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.121 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.121 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:17.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:17.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831471+0000", 2024-09-16T00:10:17.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6220000000000001e-06 2024-09-16T00:10:17.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836584+0000", 2024-09-16T00:10:17.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051133510000000004 2024-09-16T00:10:17.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836606+0000", 2024-09-16T00:10:17.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2079e-05 2024-09-16T00:10:17.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.123 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836624+0000", 2024-09-16T00:10:17.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8162e-05 2024-09-16T00:10:17.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.123 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.123 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836704+0000", 2024-09-16T00:10:17.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9867e-05 2024-09-16T00:10:17.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.124 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.124 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836730+0000", 2024-09-16T00:10:17.124 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6285999999999999e-05 2024-09-16T00:10:17.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.124 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.124 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.843247+0000", 2024-09-16T00:10:17.124 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0065165650000000002 2024-09-16T00:10:17.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.124 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.124 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.843298+0000", 2024-09-16T00:10:17.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0764999999999999e-05 2024-09-16T00:10:17.125 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.125 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.125 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.125 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13365 3.2 3:4298088d:::100000035be.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:17.125 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:17.125 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 29.036037422, 2024-09-16T00:10:17.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014036442999999999, 2024-09-16T00:10:17.125 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.125 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.125 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.126 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:17.126 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:17.126 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.126 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13365 2024-09-16T00:10:17.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.126 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:17.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:17.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.127 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.127 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019174+0000", 2024-09-16T00:10:17.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1250000000000001e-06 2024-09-16T00:10:17.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020613+0000", 2024-09-16T00:10:17.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014391930000000001 2024-09-16T00:10:17.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020638+0000", 2024-09-16T00:10:17.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4657000000000001e-05 2024-09-16T00:10:17.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020651+0000", 2024-09-16T00:10:17.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3451e-05 2024-09-16T00:10:17.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.129 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.129 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020744+0000", 2024-09-16T00:10:17.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3010000000000006e-05 2024-09-16T00:10:17.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.129 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.129 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020912+0000", 2024-09-16T00:10:17.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016763800000000001 2024-09-16T00:10:17.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:17.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.021043+0000", 2024-09-16T00:10:17.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013099800000000001 2024-09-16T00:10:17.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026194+0000", 2024-09-16T00:10:17.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051510879999999998 2024-09-16T00:10:17.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033132+0000", 2024-09-16T00:10:17.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069377279999999998 2024-09-16T00:10:17.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033169+0000", 2024-09-16T00:10:17.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6897000000000003e-05 2024-09-16T00:10:17.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033207+0000", 2024-09-16T00:10:17.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658000000000002e-05 2024-09-16T00:10:17.131 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.131 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.131 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.132 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13368 3.27 3:e75b72a1:::100000032e8.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:17.132 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:17.132 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 29.028671539000001, 2024-09-16T00:10:17.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013615175, 2024-09-16T00:10:17.132 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.132 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.132 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.132 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:17.132 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:17.132 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.132 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13368 2024-09-16T00:10:17.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.132 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.133 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:17.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.133 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:17.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026539+0000", 2024-09-16T00:10:17.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.798e-06 2024-09-16T00:10:17.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027825+0000", 2024-09-16T00:10:17.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012861839999999999 2024-09-16T00:10:17.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027850+0000", 2024-09-16T00:10:17.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5048000000000001e-05 2024-09-16T00:10:17.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.135 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.135 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027865+0000", 2024-09-16T00:10:17.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4939e-05 2024-09-16T00:10:17.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.135 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.135 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027944+0000", 2024-09-16T00:10:17.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8881999999999995e-05 2024-09-16T00:10:17.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.135 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.135 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.028043+0000", 2024-09-16T00:10:17.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9135000000000005e-05 2024-09-16T00:10:17.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.136 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:17.136 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.028158+0000", 2024-09-16T00:10:17.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011527999999999999 2024-09-16T00:10:17.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.136 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.136 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.039292+0000", 2024-09-16T00:10:17.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011133568999999999 2024-09-16T00:10:17.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.039948+0000", 2024-09-16T00:10:17.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00065625700000000004 2024-09-16T00:10:17.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.040061+0000", 2024-09-16T00:10:17.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011284400000000001 2024-09-16T00:10:17.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.040152+0000", 2024-09-16T00:10:17.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1238999999999998e-05 2024-09-16T00:10:17.138 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.138 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.138 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.138 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:13366 3.15 e88/20 3:aa8f498f:::100000030ba.00000000:head v 88'196, mlcod=88'194)", 2024-09-16T00:10:17.138 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:17.138 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 29.025246463999999, 2024-09-16T00:10:17.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012822817, 2024-09-16T00:10:17.138 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.138 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.138 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.138 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:17.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:17.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029963+0000", 2024-09-16T00:10:17.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4270000000000001e-06 2024-09-16T00:10:17.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.140 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.140 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.140 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034078+0000", 2024-09-16T00:10:17.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041148360000000002 2024-09-16T00:10:17.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.140 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.140 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.140 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034094+0000", 2024-09-16T00:10:17.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.535e-05 2024-09-16T00:10:17.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.140 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.140 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034103+0000", 2024-09-16T00:10:17.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:10:17.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.036791+0000", 2024-09-16T00:10:17.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026878340000000001 2024-09-16T00:10:17.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.036818+0000", 2024-09-16T00:10:17.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6885000000000001e-05 2024-09-16T00:10:17.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.042713+0000", 2024-09-16T00:10:17.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058950399999999998 2024-09-16T00:10:17.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.042785+0000", 2024-09-16T00:10:17.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1927000000000004e-05 2024-09-16T00:10:17.143 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.143 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.143 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.143 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13405 3.27 3:e708eda4:::100000034c4.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:17.143 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:17.143 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 28.899044751000002, 2024-09-16T00:10:17.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012003333, 2024-09-16T00:10:17.143 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.144 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.144 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.144 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:17.144 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:17.144 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.144 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13405 2024-09-16T00:10:17.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.144 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:17.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.145 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.145 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:17.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.145 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.145 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156166+0000", 2024-09-16T00:10:17.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6230000000000001e-06 2024-09-16T00:10:17.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.145 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.145 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157868+0000", 2024-09-16T00:10:17.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017017 2024-09-16T00:10:17.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157891+0000", 2024-09-16T00:10:17.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2931000000000001e-05 2024-09-16T00:10:17.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157904+0000", 2024-09-16T00:10:17.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2763e-05 2024-09-16T00:10:17.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.147 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157988+0000", 2024-09-16T00:10:17.147 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3849000000000003e-05 2024-09-16T00:10:17.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.147 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.147 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.158083+0000", 2024-09-16T00:10:17.147 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5939000000000004e-05 2024-09-16T00:10:17.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.147 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:17.147 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.158194+0000", 2024-09-16T00:10:17.147 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110963 2024-09-16T00:10:17.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.164635+0000", 2024-09-16T00:10:17.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0064405570000000004 2024-09-16T00:10:17.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168088+0000", 2024-09-16T00:10:17.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034534639999999998 2024-09-16T00:10:17.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.149 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168140+0000", 2024-09-16T00:10:17.149 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1041000000000001e-05 2024-09-16T00:10:17.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.150 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.150 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.150 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168167+0000", 2024-09-16T00:10:17.150 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7503000000000001e-05 2024-09-16T00:10:17.150 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.150 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.150 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.150 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.150 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13685 3.b 3:d26c1580:::100000032e4.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:17.150 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:17.151 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.720357786000001, 2024-09-16T00:10:17.151 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014868124999999999, 2024-09-16T00:10:17.151 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.151 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.151 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.151 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:17.151 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:17.151 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.151 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13685 2024-09-16T00:10:17.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.152 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.152 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.152 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.152 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:17.152 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.152 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.152 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.152 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.153 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:17.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.153 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.153 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.153 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334854+0000", 2024-09-16T00:10:17.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0000000000000001e-06 2024-09-16T00:10:17.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.153 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.153 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.154 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337155+0000", 2024-09-16T00:10:17.154 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023016870000000002 2024-09-16T00:10:17.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.154 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.154 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.154 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337178+0000", 2024-09-16T00:10:17.154 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2898e-05 2024-09-16T00:10:17.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.154 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337190+0000", 2024-09-16T00:10:17.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1817999999999999e-05 2024-09-16T00:10:17.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337301+0000", 2024-09-16T00:10:17.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110951 2024-09-16T00:10:17.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337403+0000", 2024-09-16T00:10:17.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000102118 2024-09-16T00:10:17.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.156 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.156 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:17.156 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337545+0000", 2024-09-16T00:10:17.156 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014167199999999999 2024-09-16T00:10:17.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.156 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.156 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.156 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.346336+0000", 2024-09-16T00:10:17.156 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087911450000000002 2024-09-16T00:10:17.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.156 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.156 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349641+0000", 2024-09-16T00:10:17.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033047570000000002 2024-09-16T00:10:17.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349680+0000", 2024-09-16T00:10:17.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9054999999999998e-05 2024-09-16T00:10:17.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.157 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.157 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349719+0000", 2024-09-16T00:10:17.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9023999999999998e-05 2024-09-16T00:10:17.157 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.158 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.158 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.158 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14624 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:17.158 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:17.158 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 24.440598585, 2024-09-16T00:10:17.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012123376, 2024-09-16T00:10:17.158 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.158 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.158 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.158 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:17.158 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:17.159 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.159 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14624 2024-09-16T00:10:17.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.159 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.159 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.159 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.159 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:17.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.159 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.159 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.159 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:17.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.160 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.160 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614614+0000", 2024-09-16T00:10:17.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9500000000000003e-06 2024-09-16T00:10:17.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.160 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.160 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616480+0000", 2024-09-16T00:10:17.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018663169999999999 2024-09-16T00:10:17.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.160 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.160 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616506+0000", 2024-09-16T00:10:17.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6205999999999999e-05 2024-09-16T00:10:17.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.161 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.161 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.161 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616517+0000", 2024-09-16T00:10:17.161 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0399e-05 2024-09-16T00:10:17.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.161 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.161 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.161 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616595+0000", 2024-09-16T00:10:17.161 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8170999999999994e-05 2024-09-16T00:10:17.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.161 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.161 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.161 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616683+0000", 2024-09-16T00:10:17.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8578999999999993e-05 2024-09-16T00:10:17.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:17.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616825+0000", 2024-09-16T00:10:17.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014186299999999999 2024-09-16T00:10:17.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.624948+0000", 2024-09-16T00:10:17.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081230420000000005 2024-09-16T00:10:17.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626617+0000", 2024-09-16T00:10:17.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016681980000000001 2024-09-16T00:10:17.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626678+0000", 2024-09-16T00:10:17.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1362e-05 2024-09-16T00:10:17.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626733+0000", 2024-09-16T00:10:17.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5288999999999999e-05 2024-09-16T00:10:17.163 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.163 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.164 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.164 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.164 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14672 3.29 3:9641a85f:::1000000360e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:17.164 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:17.164 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 24.236141408000002, 2024-09-16T00:10:17.164 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012258025000000001, 2024-09-16T00:10:17.164 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.164 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.164 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.165 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:17.165 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:17.165 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.165 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14672 2024-09-16T00:10:17.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.165 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.165 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.165 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:17.165 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:17.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819070+0000", 2024-09-16T00:10:17.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0589999999999998e-06 2024-09-16T00:10:17.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821701+0000", 2024-09-16T00:10:17.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026309100000000002 2024-09-16T00:10:17.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821727+0000", 2024-09-16T00:10:17.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6372000000000001e-05 2024-09-16T00:10:17.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.168 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821738+0000", 2024-09-16T00:10:17.168 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0759e-05 2024-09-16T00:10:17.168 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.168 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.168 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.168 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821821+0000", 2024-09-16T00:10:17.168 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2854000000000003e-05 2024-09-16T00:10:17.168 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.168 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.168 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.168 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821920+0000", 2024-09-16T00:10:17.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8900999999999994e-05 2024-09-16T00:10:17.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.169 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:17.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.822084+0000", 2024-09-16T00:10:17.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016432799999999999 2024-09-16T00:10:17.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.170 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.170 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.170 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.829148+0000", 2024-09-16T00:10:17.170 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0070641460000000003 2024-09-16T00:10:17.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.170 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.170 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.170 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831242+0000", 2024-09-16T00:10:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002093293 2024-09-16T00:10:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831280+0000", 2024-09-16T00:10:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8707000000000001e-05 2024-09-16T00:10:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831325+0000", 2024-09-16T00:10:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4696e-05 2024-09-16T00:10:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.171 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.172 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14829 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304 [2@2556691]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:17.172 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:17.172 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.48362199, 2024-09-16T00:10:17.172 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014399683, 2024-09-16T00:10:17.172 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.172 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.172 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.172 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:17.172 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:17.172 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.172 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14829 2024-09-16T00:10:17.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.173 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.173 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.173 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.173 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:17.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.173 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.173 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.173 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:17.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.174 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.174 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.174 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571590+0000", 2024-09-16T00:10:17.174 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:10:17.174 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.174 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.174 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573133+0000", 2024-09-16T00:10:17.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015422109999999999 2024-09-16T00:10:17.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573161+0000", 2024-09-16T00:10:17.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8045e-05 2024-09-16T00:10:17.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573174+0000", 2024-09-16T00:10:17.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3193999999999999e-05 2024-09-16T00:10:17.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573365+0000", 2024-09-16T00:10:17.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019096400000000001 2024-09-16T00:10:17.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573467+0000", 2024-09-16T00:10:17.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101789 2024-09-16T00:10:17.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:17.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573610+0000", 2024-09-16T00:10:17.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143404 2024-09-16T00:10:17.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.583614+0000", 2024-09-16T00:10:17.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010004422000000001 2024-09-16T00:10:17.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.179 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585556+0000", 2024-09-16T00:10:17.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019414009999999999 2024-09-16T00:10:17.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.180 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585939+0000", 2024-09-16T00:10:17.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038340999999999999 2024-09-16T00:10:17.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585986+0000", 2024-09-16T00:10:17.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6924e-05 2024-09-16T00:10:17.181 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.182 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.182 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.182 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14914 3.b 3:d23f6103:::10000003277.00000000:head [write 0~4194304 [3@58131]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:17.182 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:17.183 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 23.172418219000001, 2024-09-16T00:10:17.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013007654, 2024-09-16T00:10:17.183 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.183 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.183 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.183 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:17.184 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:17.184 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.184 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14914 2024-09-16T00:10:17.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.184 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:17.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.186 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.186 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:17.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.186 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.186 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882793+0000", 2024-09-16T00:10:17.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.65e-06 2024-09-16T00:10:17.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885193+0000", 2024-09-16T00:10:17.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024003449999999999 2024-09-16T00:10:17.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885214+0000", 2024-09-16T00:10:17.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0398e-05 2024-09-16T00:10:17.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885224+0000", 2024-09-16T00:10:17.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0253e-05 2024-09-16T00:10:17.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885299+0000", 2024-09-16T00:10:17.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5434999999999999e-05 2024-09-16T00:10:17.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885389+0000", 2024-09-16T00:10:17.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9209999999999995e-05 2024-09-16T00:10:17.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:17.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885485+0000", 2024-09-16T00:10:17.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6218999999999995e-05 2024-09-16T00:10:17.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.192 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.192 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.893457+0000", 2024-09-16T00:10:17.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0079722009999999999 2024-09-16T00:10:17.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895681+0000", 2024-09-16T00:10:17.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022241449999999999 2024-09-16T00:10:17.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895732+0000", 2024-09-16T00:10:17.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1269e-05 2024-09-16T00:10:17.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895798+0000", 2024-09-16T00:10:17.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5529000000000002e-05 2024-09-16T00:10:17.195 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.195 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.195 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.195 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15058 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304 [2@2556691]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-16T00:10:17.195 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:17.195 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.720049625000001, 2024-09-16T00:10:17.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012275842, 2024-09-16T00:10:17.196 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:17.196 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:17.196 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.196 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:17.196 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:17.196 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.196 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15058 2024-09-16T00:10:17.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.197 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:17.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.197 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:17.197 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:17.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:17.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:17.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:17.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:17.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335161+0000", 2024-09-16T00:10:17.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.367e-06 2024-09-16T00:10:17.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:17.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337545+0000", 2024-09-16T00:10:17.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023839040000000001 2024-09-16T00:10:17.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:17.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337572+0000", 2024-09-16T00:10:17.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.673e-05 2024-09-16T00:10:17.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337583+0000", 2024-09-16T00:10:17.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1048e-05 2024-09-16T00:10:17.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:17.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337660+0000", 2024-09-16T00:10:17.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7459999999999994e-05 2024-09-16T00:10:17.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:17.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337770+0000", 2024-09-16T00:10:17.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109634 2024-09-16T00:10:17.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:17.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337926+0000", 2024-09-16T00:10:17.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015601999999999999 2024-09-16T00:10:17.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.345689+0000", 2024-09-16T00:10:17.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0077625630000000001 2024-09-16T00:10:17.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:17.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347324+0000", 2024-09-16T00:10:17.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016351479999999999 2024-09-16T00:10:17.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:17.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347387+0000", 2024-09-16T00:10:17.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3199000000000003e-05 2024-09-16T00:10:17.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:17.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:17.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:17.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347435+0000", 2024-09-16T00:10:17.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7769000000000003e-05 2024-09-16T00:10:17.202 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.202 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.202 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.202 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:17.202 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:17.202 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:17.204 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:10:17.204 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:17.203+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:17.305 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:17.304+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:17.308 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:17.308 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:17.309 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:17.309 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:17.318 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:10:17.353 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T00:10:17.406 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:17.405+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:17.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.457+0000 7f9f66b9f640 1 Processor -- start 2024-09-16T00:10:17.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.457+0000 7f9f66b9f640 1 -- start start 2024-09-16T00:10:17.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.458+0000 7f9f66b9f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9f60154b00 0x7f9f6015b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.458+0000 7f9f66b9f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f60154ee0 con 0x7f9f600caee0 2024-09-16T00:10:17.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.458+0000 7f9f66b9f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f600b2bb0 con 0x7f9f600b44f0 2024-09-16T00:10:17.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.458+0000 7f9f66b9f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6015b760 con 0x7f9f60154b00 2024-09-16T00:10:17.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.458+0000 7f9f6539c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f9f600caee0 0x7f9f600d04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56226/0 (socket says 172.21.15.139:56226) 2024-09-16T00:10:17.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.458+0000 7f9f6539c640 1 -- 172.21.15.139:0/3838803 learned_addr learned my addr 172.21.15.139:0/3838803 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:17.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.458+0000 7f9f65b9d640 1 --2- 172.21.15.139:0/3838803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9f60154b00 0x7f9f6015b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.458+0000 7f9f64b9b640 1 -- 172.21.15.139:0/3838803 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 947984250 0 0) 0x7f9f60154ee0 con 0x7f9f600caee0 2024-09-16T00:10:17.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.458+0000 7f9f64b9b640 1 -- 172.21.15.139:0/3838803 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f4c003520 con 0x7f9f600caee0 2024-09-16T00:10:17.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.459+0000 7f9f64b9b640 1 -- 172.21.15.139:0/3838803 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4038495561 0 0) 0x7f9f600b2bb0 con 0x7f9f600b44f0 2024-09-16T00:10:17.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.459+0000 7f9f64b9b640 1 -- 172.21.15.139:0/3838803 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f60154ee0 con 0x7f9f600b44f0 2024-09-16T00:10:17.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.459+0000 7f9f65b9d640 1 -- 172.21.15.139:0/3838803 >> v1:172.21.15.164:6789/0 conn(0x7f9f600b44f0 legacy=0x7f9f60165040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:17.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.459+0000 7f9f65b9d640 1 -- 172.21.15.139:0/3838803 >> v1:172.21.15.139:6789/0 conn(0x7f9f600caee0 legacy=0x7f9f600d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:17.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.459+0000 7f9f65b9d640 1 -- 172.21.15.139:0/3838803 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f600b2e90 con 0x7f9f60154b00 2024-09-16T00:10:17.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.459+0000 7f9f65b9d640 1 --2- 172.21.15.139:0/3838803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9f60154b00 0x7f9f6015b220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c0097d0 tx=0x7f9f5c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aef067e996b3afe server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:17.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.459+0000 7f9f64b9b640 1 -- 172.21.15.139:0/3838803 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c037070 con 0x7f9f60154b00 2024-09-16T00:10:17.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.460+0000 7f9f64b9b640 1 -- 172.21.15.139:0/3838803 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f5c032440 con 0x7f9f60154b00 2024-09-16T00:10:17.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.460+0000 7f9f64b9b640 1 -- 172.21.15.139:0/3838803 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0327f0 con 0x7f9f60154b00 2024-09-16T00:10:17.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.460+0000 7f9f66b9f640 1 -- 172.21.15.139:0/3838803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9f60154b00 msgr2=0x7f9f6015b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:17.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.460+0000 7f9f66b9f640 1 --2- 172.21.15.139:0/3838803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9f60154b00 0x7f9f6015b220 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c0097d0 tx=0x7f9f5c02a210 comp rx=0 tx=0).stop 2024-09-16T00:10:17.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.460+0000 7f9f66b9f640 1 -- 172.21.15.139:0/3838803 shutdown_connections 2024-09-16T00:10:17.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.460+0000 7f9f66b9f640 1 --2- 172.21.15.139:0/3838803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9f60154b00 0x7f9f6015b220 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:17.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.460+0000 7f9f66b9f640 1 -- 172.21.15.139:0/3838803 >> 172.21.15.139:0/3838803 conn(0x7f9f6006c500 msgr2=0x7f9f6006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:17.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.460+0000 7f9f66b9f640 1 -- 172.21.15.139:0/3838803 shutdown_connections 2024-09-16T00:10:17.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.460+0000 7f9f66b9f640 1 -- 172.21.15.139:0/3838803 wait complete. 2024-09-16T00:10:17.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.461+0000 7f9f66b9f640 1 Processor -- start 2024-09-16T00:10:17.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.461+0000 7f9f66b9f640 1 -- start start 2024-09-16T00:10:17.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.462+0000 7f9f66b9f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9f600b44f0 0x7f9f6014c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.462+0000 7f9f66b9f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f600caee0 0x7f9f60156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.462+0000 7f9f66b9f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f60154b00 0x7f9f600cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.462+0000 7f9f66b9f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9f600b4060 con 0x7f9f600caee0 2024-09-16T00:10:17.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.462+0000 7f9f66b9f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9f600b37c0 con 0x7f9f60154b00 2024-09-16T00:10:17.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.462+0000 7f9f66b9f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9f600b3cf0 con 0x7f9f600b44f0 2024-09-16T00:10:17.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.462+0000 7f9f6539c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f600caee0 0x7f9f60156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.462+0000 7f9f65b9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9f600b44f0 0x7f9f6014c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.462+0000 7f9f6539c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f600caee0 0x7f9f60156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52078/0 (socket says 172.21.15.139:52078) 2024-09-16T00:10:17.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.462+0000 7f9f6539c640 1 -- 172.21.15.139:0/3877160095 learned_addr learned my addr 172.21.15.139:0/3877160095 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:17.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.462+0000 7f9f6639e640 1 --2- 172.21.15.139:0/3877160095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f60154b00 0x7f9f600cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.462+0000 7f9f6539c640 1 -- 172.21.15.139:0/3877160095 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9f600b44f0 msgr2=0x7f9f6014c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.462+0000 7f9f6539c640 1 --2- 172.21.15.139:0/3877160095 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9f600b44f0 0x7f9f6014c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.462+0000 7f9f6539c640 1 -- 172.21.15.139:0/3877160095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f60154b00 msgr2=0x7f9f600cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.462+0000 7f9f6539c640 1 --2- 172.21.15.139:0/3877160095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f60154b00 0x7f9f600cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.462+0000 7f9f6539c640 1 -- 172.21.15.139:0/3877160095 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f601536a0 con 0x7f9f600caee0 2024-09-16T00:10:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.463+0000 7f9f6539c640 1 --2- 172.21.15.139:0/3877160095 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f600caee0 0x7f9f60156780 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000c500 tx=0x7f9f5000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.463+0000 7f9f567fc640 1 -- 172.21.15.139:0/3877160095 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50017070 con 0x7f9f600caee0 2024-09-16T00:10:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.463+0000 7f9f567fc640 1 -- 172.21.15.139:0/3877160095 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f50005d00 con 0x7f9f600caee0 2024-09-16T00:10:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.463+0000 7f9f66b9f640 1 -- 172.21.15.139:0/3877160095 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f601551b0 con 0x7f9f600caee0 2024-09-16T00:10:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.463+0000 7f9f567fc640 1 -- 172.21.15.139:0/3877160095 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50007330 con 0x7f9f600caee0 2024-09-16T00:10:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.463+0000 7f9f66b9f640 1 -- 172.21.15.139:0/3877160095 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f60155770 con 0x7f9f600caee0 2024-09-16T00:10:17.469 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.464+0000 7f9f66b9f640 1 -- 172.21.15.139:0/3877160095 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f600cb2e0 con 0x7f9f600caee0 2024-09-16T00:10:17.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.464+0000 7f9f567fc640 1 -- 172.21.15.139:0/3877160095 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9f5001a070 con 0x7f9f600caee0 2024-09-16T00:10:17.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.465+0000 7f9f567fc640 1 --2- 172.21.15.139:0/3877160095 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9f4406c2b0 0x7f9f4406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.465+0000 7f9f567fc640 1 -- 172.21.15.139:0/3877160095 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6098+0+0 (secure 0 0 0) 0x7f9f5008a910 con 0x7f9f600caee0 2024-09-16T00:10:17.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.465+0000 7f9f65b9d640 1 --2- 172.21.15.139:0/3877160095 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9f4406c2b0 0x7f9f4406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.465+0000 7f9f65b9d640 1 --2- 172.21.15.139:0/3877160095 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9f4406c2b0 0x7f9f4406e770 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c034000 tx=0x7f9f5c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:17.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.469+0000 7f9f567fc640 1 -- 172.21.15.139:0/3877160095 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9f500589f0 con 0x7f9f600caee0 2024-09-16T00:10:17.473 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:17.473 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:17.473 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:17.473 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:17.474 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:17.483 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:10:17.506 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:17.506+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:17.592 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:17.592 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:17.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:17.593 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:17.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.593 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10780 3.a 3:500c83b2:::100000030b4.00000000:head [write 0~4194304 [3@2137521]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:17.593 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:17.593 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 39.136237866999998, 2024-09-16T00:10:17.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015526478, 2024-09-16T00:10:17.593 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.593 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.594 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.594 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.594 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.594 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.594 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10780 2024-09-16T00:10:17.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.594 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:17.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:17.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452917+0000", 2024-09-16T00:10:17.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2180000000000002e-06 2024-09-16T00:10:17.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462177+0000", 2024-09-16T00:10:17.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092593769999999992 2024-09-16T00:10:17.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462196+0000", 2024-09-16T00:10:17.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9745000000000002e-05 2024-09-16T00:10:17.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462207+0000", 2024-09-16T00:10:17.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0524e-05 2024-09-16T00:10:17.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462284+0000", 2024-09-16T00:10:17.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7608000000000006e-05 2024-09-16T00:10:17.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462376+0000", 2024-09-16T00:10:17.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1811999999999994e-05 2024-09-16T00:10:17.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468357+0000", 2024-09-16T00:10:17.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059806809999999998 2024-09-16T00:10:17.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468386+0000", 2024-09-16T00:10:17.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9068999999999999e-05 2024-09-16T00:10:17.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468441+0000", 2024-09-16T00:10:17.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5444e-05 2024-09-16T00:10:17.603 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.603 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.604 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.604 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10831 3.1a 3:59b8f98c:::100000030df.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:17.604 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:17.604 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.813430730999997, 2024-09-16T00:10:17.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014692843000000001, 2024-09-16T00:10:17.604 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.604 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.604 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.604 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.604 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.605 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.605 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10831 2024-09-16T00:10:17.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.605 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:17.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:17.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775723+0000", 2024-09-16T00:10:17.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.367e-06 2024-09-16T00:10:17.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780402+0000", 2024-09-16T00:10:17.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046785769999999997 2024-09-16T00:10:17.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780421+0000", 2024-09-16T00:10:17.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9179000000000001e-05 2024-09-16T00:10:17.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780440+0000", 2024-09-16T00:10:17.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9015e-05 2024-09-16T00:10:17.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780533+0000", 2024-09-16T00:10:17.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3197999999999998e-05 2024-09-16T00:10:17.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780618+0000", 2024-09-16T00:10:17.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4787e-05 2024-09-16T00:10:17.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:17.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780748+0000", 2024-09-16T00:10:17.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012927199999999999 2024-09-16T00:10:17.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.785049+0000", 2024-09-16T00:10:17.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043015750000000002 2024-09-16T00:10:17.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790312+0000", 2024-09-16T00:10:17.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052633319999999999 2024-09-16T00:10:17.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790369+0000", 2024-09-16T00:10:17.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6848999999999997e-05 2024-09-16T00:10:17.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790415+0000", 2024-09-16T00:10:17.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5692000000000002e-05 2024-09-16T00:10:17.610 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.610 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.610 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.611 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10915 3.1a 3:5a6e15a2:::1000000318e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:17.611 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:17.611 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.373924809999998, 2024-09-16T00:10:17.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014599472, 2024-09-16T00:10:17.611 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.611 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.611 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.611 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.611 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.611 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.611 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10915 2024-09-16T00:10:17.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.611 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:17.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:17.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215230+0000", 2024-09-16T00:10:17.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0090000000000002e-06 2024-09-16T00:10:17.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219556+0000", 2024-09-16T00:10:17.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043259539999999999 2024-09-16T00:10:17.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219577+0000", 2024-09-16T00:10:17.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1271999999999999e-05 2024-09-16T00:10:17.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219589+0000", 2024-09-16T00:10:17.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1766000000000001e-05 2024-09-16T00:10:17.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219663+0000", 2024-09-16T00:10:17.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4196999999999995e-05 2024-09-16T00:10:17.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219733+0000", 2024-09-16T00:10:17.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9553000000000003e-05 2024-09-16T00:10:17.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:17.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219827+0000", 2024-09-16T00:10:17.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4395999999999995e-05 2024-09-16T00:10:17.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.618 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.618 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.225724+0000", 2024-09-16T00:10:17.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058968359999999999 2024-09-16T00:10:17.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229705+0000", 2024-09-16T00:10:17.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003981409 2024-09-16T00:10:17.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229772+0000", 2024-09-16T00:10:17.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6137999999999996e-05 2024-09-16T00:10:17.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229828+0000", 2024-09-16T00:10:17.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5942000000000001e-05 2024-09-16T00:10:17.622 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.622 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.622 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.623 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11074 3.14 3:2834b7ee:::10000003131.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:17.623 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:17.623 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.676384687000002, 2024-09-16T00:10:17.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015588350000000001, 2024-09-16T00:10:17.624 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.624 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.624 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.624 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.625 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.625 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11074 2024-09-16T00:10:17.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.625 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:17.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:17.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912769+0000", 2024-09-16T00:10:17.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4600000000000004e-07 2024-09-16T00:10:17.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918264+0000", 2024-09-16T00:10:17.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054954890000000001 2024-09-16T00:10:17.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.630 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918281+0000", 2024-09-16T00:10:17.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6160000000000001e-05 2024-09-16T00:10:17.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.630 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918289+0000", 2024-09-16T00:10:17.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1650000000000006e-06 2024-09-16T00:10:17.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918386+0000", 2024-09-16T00:10:17.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7288999999999996e-05 2024-09-16T00:10:17.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918535+0000", 2024-09-16T00:10:17.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014876200000000001 2024-09-16T00:10:17.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:17.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918639+0000", 2024-09-16T00:10:17.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010470400000000001 2024-09-16T00:10:17.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.924703+0000", 2024-09-16T00:10:17.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060639759999999996 2024-09-16T00:10:17.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928246+0000", 2024-09-16T00:10:17.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035421720000000001 2024-09-16T00:10:17.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.636 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.636 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928305+0000", 2024-09-16T00:10:17.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9014000000000001e-05 2024-09-16T00:10:17.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928357+0000", 2024-09-16T00:10:17.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1873000000000003e-05 2024-09-16T00:10:17.637 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.638 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.638 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.638 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11111 3.9 3:93301cc4:::10000003210.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:17.639 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:17.639 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.499646485, 2024-09-16T00:10:17.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014336099999999999, 2024-09-16T00:10:17.639 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.639 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.640 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.640 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.640 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11111 2024-09-16T00:10:17.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.640 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:17.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:17.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089520+0000", 2024-09-16T00:10:17.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3903000000000001e-05 2024-09-16T00:10:17.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094463+0000", 2024-09-16T00:10:17.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049423000000000002 2024-09-16T00:10:17.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094494+0000", 2024-09-16T00:10:17.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1029999999999999e-05 2024-09-16T00:10:17.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094503+0000", 2024-09-16T00:10:17.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0180000000000007e-06 2024-09-16T00:10:17.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094573+0000", 2024-09-16T00:10:17.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0457000000000007e-05 2024-09-16T00:10:17.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094737+0000", 2024-09-16T00:10:17.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016408699999999999 2024-09-16T00:10:17.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:17.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094897+0000", 2024-09-16T00:10:17.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015938899999999999 2024-09-16T00:10:17.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.102326+0000", 2024-09-16T00:10:17.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074291230000000002 2024-09-16T00:10:17.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103723+0000", 2024-09-16T00:10:17.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001397255 2024-09-16T00:10:17.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103786+0000", 2024-09-16T00:10:17.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3489000000000001e-05 2024-09-16T00:10:17.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103842+0000", 2024-09-16T00:10:17.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6048999999999998e-05 2024-09-16T00:10:17.648 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.648 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.648 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.648 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:11934 3.1b e87/20 3:d94fa8da:::10000003190.00000000:head v 87'152, mlcod=87'150)", 2024-09-16T00:10:17.649 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:17.649 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.898901789, 2024-09-16T00:10:17.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014984934, 2024-09-16T00:10:17.649 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.649 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.649 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.650 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:17.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:17.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690252+0000", 2024-09-16T00:10:17.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8899999999999998e-07 2024-09-16T00:10:17.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.653 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.653 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.653 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696026+0000", 2024-09-16T00:10:17.653 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057745219999999998 2024-09-16T00:10:17.653 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.654 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.654 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696039+0000", 2024-09-16T00:10:17.654 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2611e-05 2024-09-16T00:10:17.654 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.654 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.655 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696047+0000", 2024-09-16T00:10:17.655 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5399999999999998e-06 2024-09-16T00:10:17.655 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.655 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.655 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696123+0000", 2024-09-16T00:10:17.656 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6349999999999999e-05 2024-09-16T00:10:17.656 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.656 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.656 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.656 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696144+0000", 2024-09-16T00:10:17.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1512999999999999e-05 2024-09-16T00:10:17.657 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.657 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.657 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.657 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705188+0000", 2024-09-16T00:10:17.657 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090438459999999995 2024-09-16T00:10:17.658 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.658 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.658 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.658 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705236+0000", 2024-09-16T00:10:17.658 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7663000000000001e-05 2024-09-16T00:10:17.658 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.658 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.659 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.659 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.659 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.659 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11936 3.3 3:c0886fb9:::100000031a1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:17.659 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:17.659 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.888756737000001, 2024-09-16T00:10:17.660 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015336379000000001, 2024-09-16T00:10:17.660 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.660 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.660 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.660 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.660 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.661 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.661 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11936 2024-09-16T00:10:17.661 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.661 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.662 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.662 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:17.662 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.662 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.662 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.663 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:17.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.663 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.663 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700399+0000", 2024-09-16T00:10:17.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6029999999999999e-06 2024-09-16T00:10:17.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704706+0000", 2024-09-16T00:10:17.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043070529999999999 2024-09-16T00:10:17.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704720+0000", 2024-09-16T00:10:17.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4629e-05 2024-09-16T00:10:17.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704729+0000", 2024-09-16T00:10:17.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6009999999999995e-06 2024-09-16T00:10:17.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704805+0000", 2024-09-16T00:10:17.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6015999999999999e-05 2024-09-16T00:10:17.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704866+0000", 2024-09-16T00:10:17.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0878999999999998e-05 2024-09-16T00:10:17.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.669 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:17.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704990+0000", 2024-09-16T00:10:17.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000124058 2024-09-16T00:10:17.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.714314+0000", 2024-09-16T00:10:17.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009324321 2024-09-16T00:10:17.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715632+0000", 2024-09-16T00:10:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001318121 2024-09-16T00:10:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715687+0000", 2024-09-16T00:10:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4135999999999998e-05 2024-09-16T00:10:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715732+0000", 2024-09-16T00:10:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961999999999998e-05 2024-09-16T00:10:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.672 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11953 3.1a 3:580e2503:::1000000311b.00000000:head [write 0~4194304 [2@2058066]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.822417584, 2024-09-16T00:10:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016299973999999998, 2024-09-16T00:10:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.673 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11953 2024-09-16T00:10:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766741+0000", 2024-09-16T00:10:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2939999999999996e-06 2024-09-16T00:10:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771165+0000", 2024-09-16T00:10:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044248229999999996 2024-09-16T00:10:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771181+0000", 2024-09-16T00:10:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5243e-05 2024-09-16T00:10:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771188+0000", 2024-09-16T00:10:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6429999999999995e-06 2024-09-16T00:10:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771238+0000", 2024-09-16T00:10:17.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9567999999999997e-05 2024-09-16T00:10:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771293+0000", 2024-09-16T00:10:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5207000000000002e-05 2024-09-16T00:10:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771380+0000", 2024-09-16T00:10:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6716000000000001e-05 2024-09-16T00:10:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.776193+0000", 2024-09-16T00:10:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048133350000000002 2024-09-16T00:10:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.782931+0000", 2024-09-16T00:10:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067376550000000004 2024-09-16T00:10:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.782987+0000", 2024-09-16T00:10:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6443000000000002e-05 2024-09-16T00:10:17.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.783035+0000", 2024-09-16T00:10:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8047000000000003e-05 2024-09-16T00:10:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12037 3.9 3:928db521:::1000000326d.00000000:head [write 0~4194304 [2@1655200]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.591209976000002, 2024-09-16T00:10:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017082494, 2024-09-16T00:10:17.679 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12037 2024-09-16T00:10:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997944+0000", 2024-09-16T00:10:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.517e-06 2024-09-16T00:10:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002177+0000", 2024-09-16T00:10:17.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042329300000000002 2024-09-16T00:10:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002197+0000", 2024-09-16T00:10:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9769e-05 2024-09-16T00:10:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002207+0000", 2024-09-16T00:10:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0178e-05 2024-09-16T00:10:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002266+0000", 2024-09-16T00:10:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9215999999999997e-05 2024-09-16T00:10:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002340+0000", 2024-09-16T00:10:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3798000000000001e-05 2024-09-16T00:10:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002454+0000", 2024-09-16T00:10:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113551 2024-09-16T00:10:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.006270+0000", 2024-09-16T00:10:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038163839999999999 2024-09-16T00:10:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.014957+0000", 2024-09-16T00:10:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086869119999999998 2024-09-16T00:10:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.014996+0000", 2024-09-16T00:10:17.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8878000000000003e-05 2024-09-16T00:10:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.015025+0000", 2024-09-16T00:10:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9360999999999999e-05 2024-09-16T00:10:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12245 3.1a 3:599a1cd1:::10000003252.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.806512558999998, 2024-09-16T00:10:17.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017131203000000001, 2024-09-16T00:10:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12245 2024-09-16T00:10:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782642+0000", 2024-09-16T00:10:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.311e-06 2024-09-16T00:10:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786551+0000", 2024-09-16T00:10:17.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039095229999999998 2024-09-16T00:10:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786566+0000", 2024-09-16T00:10:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4724e-05 2024-09-16T00:10:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786575+0000", 2024-09-16T00:10:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8449999999999992e-06 2024-09-16T00:10:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786636+0000", 2024-09-16T00:10:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.143e-05 2024-09-16T00:10:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786699+0000", 2024-09-16T00:10:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3218000000000004e-05 2024-09-16T00:10:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786796+0000", 2024-09-16T00:10:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6347000000000004e-05 2024-09-16T00:10:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.792369+0000", 2024-09-16T00:10:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055729780000000001 2024-09-16T00:10:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799706+0000", 2024-09-16T00:10:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073374349999999998 2024-09-16T00:10:17.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799739+0000", 2024-09-16T00:10:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2833999999999998e-05 2024-09-16T00:10:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799771+0000", 2024-09-16T00:10:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2558000000000002e-05 2024-09-16T00:10:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12292 3.3 3:c00779fd:::1000000320e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.692122462, 2024-09-16T00:10:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013828353999999999, 2024-09-16T00:10:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12292 2024-09-16T00:10:17.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897039+0000", 2024-09-16T00:10:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3299999999999999e-06 2024-09-16T00:10:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900830+0000", 2024-09-16T00:10:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037915340000000001 2024-09-16T00:10:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900850+0000", 2024-09-16T00:10:17.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9400000000000001e-05 2024-09-16T00:10:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900860+0000", 2024-09-16T00:10:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0609e-05 2024-09-16T00:10:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900928+0000", 2024-09-16T00:10:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7570000000000005e-05 2024-09-16T00:10:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.901027+0000", 2024-09-16T00:10:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9456000000000005e-05 2024-09-16T00:10:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.901168+0000", 2024-09-16T00:10:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014082799999999999 2024-09-16T00:10:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.906325+0000", 2024-09-16T00:10:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005156702 2024-09-16T00:10:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910796+0000", 2024-09-16T00:10:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044714350000000002 2024-09-16T00:10:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910829+0000", 2024-09-16T00:10:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3229000000000003e-05 2024-09-16T00:10:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910859+0000", 2024-09-16T00:10:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9261e-05 2024-09-16T00:10:17.697 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13326 3.9 3:92f796cf:::1000000350f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.632432179999999, 2024-09-16T00:10:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015673229, 2024-09-16T00:10:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.698 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13326 2024-09-16T00:10:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956722+0000", 2024-09-16T00:10:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3200000000000001e-06 2024-09-16T00:10:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960803+0000", 2024-09-16T00:10:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040807029999999998 2024-09-16T00:10:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960820+0000", 2024-09-16T00:10:17.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7478000000000001e-05 2024-09-16T00:10:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960830+0000", 2024-09-16T00:10:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:10:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960903+0000", 2024-09-16T00:10:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3057000000000002e-05 2024-09-16T00:10:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960970+0000", 2024-09-16T00:10:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7063000000000005e-05 2024-09-16T00:10:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.961067+0000", 2024-09-16T00:10:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6906999999999999e-05 2024-09-16T00:10:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.967945+0000", 2024-09-16T00:10:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068787359999999999 2024-09-16T00:10:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972328+0000", 2024-09-16T00:10:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043826170000000001 2024-09-16T00:10:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972363+0000", 2024-09-16T00:10:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4486000000000002e-05 2024-09-16T00:10:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972394+0000", 2024-09-16T00:10:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1343999999999997e-05 2024-09-16T00:10:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.703 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13429 3.1a 3:58dcd24a:::100000036d5.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.277862163999998, 2024-09-16T00:10:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014237671, 2024-09-16T00:10:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.704 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13429 2024-09-16T00:10:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311292+0000", 2024-09-16T00:10:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4890000000000001e-06 2024-09-16T00:10:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316378+0000", 2024-09-16T00:10:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050859290000000003 2024-09-16T00:10:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316399+0000", 2024-09-16T00:10:17.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1345e-05 2024-09-16T00:10:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316410+0000", 2024-09-16T00:10:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0295999999999999e-05 2024-09-16T00:10:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316493+0000", 2024-09-16T00:10:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3533000000000001e-05 2024-09-16T00:10:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316622+0000", 2024-09-16T00:10:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012887900000000001 2024-09-16T00:10:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316725+0000", 2024-09-16T00:10:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010295599999999999 2024-09-16T00:10:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.320948+0000", 2024-09-16T00:10:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042229820000000001 2024-09-16T00:10:17.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325415+0000", 2024-09-16T00:10:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044673849999999999 2024-09-16T00:10:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325487+0000", 2024-09-16T00:10:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1087999999999994e-05 2024-09-16T00:10:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325528+0000", 2024-09-16T00:10:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1789000000000003e-05 2024-09-16T00:10:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13527 3.3 3:c0886fb9:::100000031a1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.803702075, 2024-09-16T00:10:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015187236, 2024-09-16T00:10:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.710 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13527 2024-09-16T00:10:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785453+0000", 2024-09-16T00:10:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1689999999999999e-06 2024-09-16T00:10:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791191+0000", 2024-09-16T00:10:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057378339999999998 2024-09-16T00:10:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791217+0000", 2024-09-16T00:10:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6480999999999999e-05 2024-09-16T00:10:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791233+0000", 2024-09-16T00:10:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5841e-05 2024-09-16T00:10:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791319+0000", 2024-09-16T00:10:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5527999999999998e-05 2024-09-16T00:10:17.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791413+0000", 2024-09-16T00:10:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4685000000000006e-05 2024-09-16T00:10:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791563+0000", 2024-09-16T00:10:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000149624 2024-09-16T00:10:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.795749+0000", 2024-09-16T00:10:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004186286 2024-09-16T00:10:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800577+0000", 2024-09-16T00:10:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048273040000000001 2024-09-16T00:10:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800611+0000", 2024-09-16T00:10:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4270000000000002e-05 2024-09-16T00:10:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.715 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800638+0000", 2024-09-16T00:10:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7214000000000001e-05 2024-09-16T00:10:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13724 3.9 3:90613e6c:::100000034c4.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.133630329999999, 2024-09-16T00:10:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014039800999999999, 2024-09-16T00:10:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.716 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13724 2024-09-16T00:10:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455524+0000", 2024-09-16T00:10:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.437e-06 2024-09-16T00:10:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459592+0000", 2024-09-16T00:10:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040678729999999996 2024-09-16T00:10:17.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459610+0000", 2024-09-16T00:10:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7688000000000001e-05 2024-09-16T00:10:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459617+0000", 2024-09-16T00:10:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.374e-06 2024-09-16T00:10:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459678+0000", 2024-09-16T00:10:17.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1346999999999994e-05 2024-09-16T00:10:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459824+0000", 2024-09-16T00:10:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014586699999999999 2024-09-16T00:10:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459908+0000", 2024-09-16T00:10:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3521000000000002e-05 2024-09-16T00:10:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.464227+0000", 2024-09-16T00:10:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043197060000000004 2024-09-16T00:10:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469499+0000", 2024-09-16T00:10:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005271838 2024-09-16T00:10:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469534+0000", 2024-09-16T00:10:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4712000000000002e-05 2024-09-16T00:10:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469562+0000", 2024-09-16T00:10:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8438000000000001e-05 2024-09-16T00:10:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.722 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13741 3.3 3:c1377c76:::10000003664.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.086460384999999, 2024-09-16T00:10:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016142843, 2024-09-16T00:10:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.723 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13741 2024-09-16T00:10:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502694+0000", 2024-09-16T00:10:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0449999999999999e-06 2024-09-16T00:10:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507229+0000", 2024-09-16T00:10:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045357410000000003 2024-09-16T00:10:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507248+0000", 2024-09-16T00:10:17.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9127000000000002e-05 2024-09-16T00:10:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507259+0000", 2024-09-16T00:10:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0448999999999999e-05 2024-09-16T00:10:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507357+0000", 2024-09-16T00:10:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8400000000000007e-05 2024-09-16T00:10:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507514+0000", 2024-09-16T00:10:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000157195 2024-09-16T00:10:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507617+0000", 2024-09-16T00:10:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010262199999999999 2024-09-16T00:10:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.514047+0000", 2024-09-16T00:10:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006430018 2024-09-16T00:10:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518743+0000", 2024-09-16T00:10:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046963040000000001 2024-09-16T00:10:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518796+0000", 2024-09-16T00:10:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2305000000000002e-05 2024-09-16T00:10:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518835+0000", 2024-09-16T00:10:17.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9637e-05 2024-09-16T00:10:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14633 3.9 3:9170e82c:::100000035d6.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.967080128999999, 2024-09-16T00:10:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014466105, 2024-09-16T00:10:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.729 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14633 2024-09-16T00:10:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:17.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622074+0000", 2024-09-16T00:10:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.761e-06 2024-09-16T00:10:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626712+0000", 2024-09-16T00:10:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004637263 2024-09-16T00:10:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626724+0000", 2024-09-16T00:10:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2479e-05 2024-09-16T00:10:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626730+0000", 2024-09-16T00:10:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7309999999999998e-06 2024-09-16T00:10:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626791+0000", 2024-09-16T00:10:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0854999999999999e-05 2024-09-16T00:10:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626857+0000", 2024-09-16T00:10:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6549000000000003e-05 2024-09-16T00:10:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626943+0000", 2024-09-16T00:10:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6141000000000001e-05 2024-09-16T00:10:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.630854+0000", 2024-09-16T00:10:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039100790000000003 2024-09-16T00:10:17.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636468+0000", 2024-09-16T00:10:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056140570000000004 2024-09-16T00:10:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636500+0000", 2024-09-16T00:10:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2441999999999999e-05 2024-09-16T00:10:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636539+0000", 2024-09-16T00:10:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8748000000000002e-05 2024-09-16T00:10:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14667 3.9 3:9379a4af:::10000003352.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 24.792068123, 2024-09-16T00:10:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014580585, 2024-09-16T00:10:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.735 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14667 2024-09-16T00:10:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797086+0000", 2024-09-16T00:10:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.362e-06 2024-09-16T00:10:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801415+0000", 2024-09-16T00:10:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043288449999999996 2024-09-16T00:10:17.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801435+0000", 2024-09-16T00:10:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0305999999999998e-05 2024-09-16T00:10:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801442+0000", 2024-09-16T00:10:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6290000000000004e-06 2024-09-16T00:10:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801497+0000", 2024-09-16T00:10:17.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5111999999999997e-05 2024-09-16T00:10:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801639+0000", 2024-09-16T00:10:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014168000000000001 2024-09-16T00:10:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801788+0000", 2024-09-16T00:10:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014963300000000001 2024-09-16T00:10:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.805407+0000", 2024-09-16T00:10:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036186119999999998 2024-09-16T00:10:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811584+0000", 2024-09-16T00:10:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061772889999999999 2024-09-16T00:10:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811635+0000", 2024-09-16T00:10:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0453e-05 2024-09-16T00:10:17.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811665+0000", 2024-09-16T00:10:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0664000000000003e-05 2024-09-16T00:10:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14951 3.1a 3:59aee80a:::1000000394f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.622030228, 2024-09-16T00:10:17.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015993941000000001, 2024-09-16T00:10:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14951 2024-09-16T00:10:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:17.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967124+0000", 2024-09-16T00:10:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.86e-06 2024-09-16T00:10:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971258+0000", 2024-09-16T00:10:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041334780000000003 2024-09-16T00:10:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971274+0000", 2024-09-16T00:10:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6201e-05 2024-09-16T00:10:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971284+0000", 2024-09-16T00:10:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4779999999999997e-06 2024-09-16T00:10:17.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971384+0000", 2024-09-16T00:10:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9878999999999997e-05 2024-09-16T00:10:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971518+0000", 2024-09-16T00:10:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013459400000000001 2024-09-16T00:10:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:17.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971604+0000", 2024-09-16T00:10:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6215e-05 2024-09-16T00:10:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.976673+0000", 2024-09-16T00:10:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050685920000000002 2024-09-16T00:10:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983043+0000", 2024-09-16T00:10:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063698410000000002 2024-09-16T00:10:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983082+0000", 2024-09-16T00:10:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9261999999999998e-05 2024-09-16T00:10:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983117+0000", 2024-09-16T00:10:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4541e-05 2024-09-16T00:10:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.747 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:15152 3.3 3:c388c7e2:::100000032ec.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-16T00:10:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.861484118, 2024-09-16T00:10:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014603532000000001, 2024-09-16T00:10:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15152 2024-09-16T00:10:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:17.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727674+0000", 2024-09-16T00:10:17.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7699999999999998e-06 2024-09-16T00:10:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732274+0000", 2024-09-16T00:10:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045994299999999998 2024-09-16T00:10:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732286+0000", 2024-09-16T00:10:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2016e-05 2024-09-16T00:10:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732294+0000", 2024-09-16T00:10:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7049999999999999e-06 2024-09-16T00:10:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732359+0000", 2024-09-16T00:10:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5296000000000006e-05 2024-09-16T00:10:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732501+0000", 2024-09-16T00:10:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000141731 2024-09-16T00:10:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732629+0000", 2024-09-16T00:10:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012841299999999999 2024-09-16T00:10:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.737511+0000", 2024-09-16T00:10:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048822520000000001 2024-09-16T00:10:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:17.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742172+0000", 2024-09-16T00:10:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046610610000000002 2024-09-16T00:10:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742227+0000", 2024-09-16T00:10:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4497999999999999e-05 2024-09-16T00:10:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742272+0000", 2024-09-16T00:10:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5359999999999999e-05 2024-09-16T00:10:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.753 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:17.754 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:17.754 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:17.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.666+0000 7f9f66b9f640 1 -- 172.21.15.139:0/3877160095 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f9f600cba70 con 0x7f9f600caee0 2024-09-16T00:10:17.755 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:10:17.756 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:17.754+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:17.770 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:10:17.856 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:17.855+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:17.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.873+0000 7f6786027640 1 Processor -- start 2024-09-16T00:10:17.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.873+0000 7f6786027640 1 -- start start 2024-09-16T00:10:17.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f6786027640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6780071660 0x7f67800cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f6786027640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6780071a40 0x7f678015c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f6786027640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67800cac90 0x7f678015c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.875 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f6786027640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f678015c000 con 0x7f6780071660 2024-09-16T00:10:17.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f6786027640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f67800c74f0 con 0x7f6780071a40 2024-09-16T00:10:17.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f6786027640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f67800c77d0 con 0x7f67800cac90 2024-09-16T00:10:17.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f677f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6780071660 0x7f67800cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f677f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6780071660 0x7f67800cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52092/0 (socket says 172.21.15.139:52092) 2024-09-16T00:10:17.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f677f7fe640 1 -- 172.21.15.139:0/2850604682 learned_addr learned my addr 172.21.15.139:0/2850604682 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:17.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f677ffff640 1 --2- 172.21.15.139:0/2850604682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67800cac90 0x7f678015c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f677effd640 1 --2- 172.21.15.139:0/2850604682 >> v2:172.21.15.164:3300/0 conn(0x7f6780071a40 0x7f678015c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f677f7fe640 1 -- 172.21.15.139:0/2850604682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67800cac90 msgr2=0x7f678015c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:17.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f677f7fe640 1 --2- 172.21.15.139:0/2850604682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67800cac90 0x7f678015c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:17.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f677f7fe640 1 -- 172.21.15.139:0/2850604682 >> v2:172.21.15.164:3300/0 conn(0x7f6780071a40 msgr2=0x7f678015c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:17.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f677f7fe640 1 --2- 172.21.15.139:0/2850604682 >> v2:172.21.15.164:3300/0 conn(0x7f6780071a40 0x7f678015c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:17.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f677f7fe640 1 -- 172.21.15.139:0/2850604682 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67800c7ab0 con 0x7f6780071660 2024-09-16T00:10:17.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.874+0000 7f677effd640 1 --2- 172.21.15.139:0/2850604682 >> v2:172.21.15.164:3300/0 conn(0x7f6780071a40 0x7f678015c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:10:17.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.875+0000 7f677f7fe640 1 --2- 172.21.15.139:0/2850604682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6780071660 0x7f67800cb4c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f67800cab40 tx=0x7f676802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262da94766d2edfb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:17.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.875+0000 7f677e7fc640 1 -- 172.21.15.139:0/2850604682 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768037070 con 0x7f6780071660 2024-09-16T00:10:17.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.875+0000 7f677e7fc640 1 -- 172.21.15.139:0/2850604682 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6768032440 con 0x7f6780071660 2024-09-16T00:10:17.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.875+0000 7f677e7fc640 1 -- 172.21.15.139:0/2850604682 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768032810 con 0x7f6780071660 2024-09-16T00:10:17.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.876+0000 7f6786027640 1 -- 172.21.15.139:0/2850604682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6780071660 msgr2=0x7f67800cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:17.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.876+0000 7f6786027640 1 --2- 172.21.15.139:0/2850604682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6780071660 0x7f67800cb4c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f67800cab40 tx=0x7f676802a1e0 comp rx=0 tx=0).stop 2024-09-16T00:10:17.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.876+0000 7f6786027640 1 -- 172.21.15.139:0/2850604682 shutdown_connections 2024-09-16T00:10:17.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.876+0000 7f6786027640 1 --2- 172.21.15.139:0/2850604682 >> v2:172.21.15.164:3300/0 conn(0x7f6780071a40 0x7f678015c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:17.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.876+0000 7f6786027640 1 --2- 172.21.15.139:0/2850604682 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67800cac90 0x7f678015c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:17.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.876+0000 7f6786027640 1 --2- 172.21.15.139:0/2850604682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6780071660 0x7f67800cb4c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:17.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.876+0000 7f6786027640 1 -- 172.21.15.139:0/2850604682 >> 172.21.15.139:0/2850604682 conn(0x7f6780073950 msgr2=0x7f678006bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:17.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.876+0000 7f6786027640 1 -- 172.21.15.139:0/2850604682 shutdown_connections 2024-09-16T00:10:17.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.876+0000 7f6786027640 1 -- 172.21.15.139:0/2850604682 wait complete. 2024-09-16T00:10:17.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.877+0000 7f6786027640 1 Processor -- start 2024-09-16T00:10:17.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.877+0000 7f6786027640 1 -- start start 2024-09-16T00:10:17.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f6786027640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6780071660 0x7f67801ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f6786027640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6780071a40 0x7f67801f1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f6786027640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67800cac90 0x7f678014d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f6786027640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f67800c8ba0 con 0x7f67800cac90 2024-09-16T00:10:17.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f6786027640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f67800c8070 con 0x7f6780071a40 2024-09-16T00:10:17.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f6786027640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f67800cba00 con 0x7f6780071660 2024-09-16T00:10:17.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f677f7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6780071660 0x7f67801ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f677effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6780071a40 0x7f67801f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f677ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67800cac90 0x7f678014d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f677f7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6780071660 0x7f67801ef2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:53644/0 (socket says 172.21.15.139:53644) 2024-09-16T00:10:17.881 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f677f7fe640 1 -- 172.21.15.139:0/2065105255 learned_addr learned my addr 172.21.15.139:0/2065105255 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:17.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f677effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6780071a40 0x7f67801f1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:60460/0 (socket says 172.21.15.139:60460) 2024-09-16T00:10:17.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f677ffff640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67800cac90 0x7f678014d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52100/0 (socket says 172.21.15.139:52100) 2024-09-16T00:10:17.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f677f7fe640 1 -- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6780071a40 msgr2=0x7f67801f1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:17.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f677f7fe640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6780071a40 0x7f67801f1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:17.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f677f7fe640 1 -- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67800cac90 msgr2=0x7f678014d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:17.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f677f7fe640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67800cac90 0x7f678014d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:17.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f677f7fe640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67801ef7e0 con 0x7f6780071660 2024-09-16T00:10:17.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.878+0000 7f677effd640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6780071a40 0x7f67801f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:17.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.879+0000 7f677f7fe640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6780071660 0x7f67801ef2a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6768000c00 tx=0x7f67680046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:17.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.879+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676803ab10 con 0x7f6780071660 2024-09-16T00:10:17.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.879+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f676803acb0 con 0x7f6780071660 2024-09-16T00:10:17.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.879+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67801f1ba0 con 0x7f6780071660 2024-09-16T00:10:17.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.879+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676803ae60 con 0x7f6780071660 2024-09-16T00:10:17.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.879+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f678014b600 con 0x7f6780071660 2024-09-16T00:10:17.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.880+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f67800c63b0 con 0x7f6780071660 2024-09-16T00:10:17.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.881+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6768044630 con 0x7f6780071660 2024-09-16T00:10:17.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.881+0000 7f6777fff640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f676006c2b0 0x7f676006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.881+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(107..107 src has 1..107) ==== 6098+0+0 (secure 0 0 0) 0x7f67680afb70 con 0x7f6780071660 2024-09-16T00:10:17.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.881+0000 7f677effd640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f676006c2b0 0x7f676006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.881+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v107) ==== 54+0+16 (secure 0 0 0) 0x7f67800c63b0 con 0x7f6780071660 2024-09-16T00:10:17.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.882+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f67800c4bd0 0x7f6780149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.882+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f678006ec40 con 0x7f67800c4bd0 2024-09-16T00:10:17.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.882+0000 7f677effd640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f676006c2b0 0x7f676006e770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f67800c5260 tx=0x7f6770008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:17.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.882+0000 7f677ffff640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f67800c4bd0 0x7f6780149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.882+0000 7f677ffff640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f67800c4bd0 0x7f6780149220 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:17.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.883+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f678006ec40 con 0x7f67800c4bd0 2024-09-16T00:10:17.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.898+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f678014f6e0 con 0x7f67800c4bd0 2024-09-16T00:10:17.900 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.900+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f678014f6e0 con 0x7f67800c4bd0 2024-09-16T00:10:17.901 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:17.901 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:17.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.900+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f6780149c70 0x7f678014a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.900+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f678014f120 con 0x7f6780149c70 2024-09-16T00:10:17.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.900+0000 7f677f7fe640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f6780149c70 0x7f678014a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.900+0000 7f677f7fe640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f6780149c70 0x7f678014a0b0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:17.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.901+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f678014f120 con 0x7f6780149c70 2024-09-16T00:10:17.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.917+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f678014ee40 con 0x7f6780149c70 2024-09-16T00:10:17.919 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:17.919 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:17.919 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:17.919 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:17.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.919+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f678014ee40 con 0x7f6780149c70 2024-09-16T00:10:17.921 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:17.922 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:17.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.919+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f67801f2300 0x7f67801f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.919+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f67800c1eb0 con 0x7f67801f2300 2024-09-16T00:10:17.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.919+0000 7f677effd640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f67801f2300 0x7f67801f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.920+0000 7f677effd640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f67801f2300 0x7f67801f2740 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:17.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.920+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67800c1eb0 con 0x7f67801f2300 2024-09-16T00:10:17.929 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:10:17.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.934+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f67800c1eb0 con 0x7f67801f2300 2024-09-16T00:10:17.937 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.936+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f67800c1eb0 con 0x7f67801f2300 2024-09-16T00:10:17.937 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:17.939 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:17.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.936+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f6780062f00 0x7f67800c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.936+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f67800c1eb0 con 0x7f6780062f00 2024-09-16T00:10:17.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.936+0000 7f677ffff640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f6780062f00 0x7f67800c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.939 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.937+0000 7f677ffff640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f6780062f00 0x7f67800c90e0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:17.940 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.937+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67800c1eb0 con 0x7f6780062f00 2024-09-16T00:10:17.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.952+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f67800c1eb0 con 0x7f6780062f00 2024-09-16T00:10:17.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.953+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f67800c1eb0 con 0x7f6780062f00 2024-09-16T00:10:17.954 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:17.955 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:17.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.953+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f67800c9b30 0x7f678002c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.953+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f67800c1eb0 con 0x7f67800c9b30 2024-09-16T00:10:17.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.954+0000 7f677f7fe640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f67800c9b30 0x7f678002c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.954+0000 7f677f7fe640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f67800c9b30 0x7f678002c640 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:17.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.955+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67800c1eb0 con 0x7f67800c9b30 2024-09-16T00:10:17.969 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:17.968+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:17.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.968+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f67800c1eb0 con 0x7f67800c9b30 2024-09-16T00:10:17.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.970+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f67800c1eb0 con 0x7f67800c9b30 2024-09-16T00:10:17.971 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:17.972 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:17.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.971+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f678002d090 0x7f678002d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.971+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f67800c1eb0 con 0x7f678002d090 2024-09-16T00:10:17.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.971+0000 7f677effd640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f678002d090 0x7f678002d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.971+0000 7f677effd640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f678002d090 0x7f678002d4f0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:17.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.972+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67800c1eb0 con 0x7f678002d090 2024-09-16T00:10:17.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.986+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f67800c1eb0 con 0x7f678002d090 2024-09-16T00:10:17.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.988+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f67800c1eb0 con 0x7f678002d090 2024-09-16T00:10:17.989 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:17.990 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:17.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.988+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f67802036e0 0x7f6780205ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:17.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.988+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f678002da30 con 0x7f67802036e0 2024-09-16T00:10:17.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.989+0000 7f677ffff640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f67802036e0 0x7f6780205ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:17.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.989+0000 7f677ffff640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f67802036e0 0x7f6780205ad0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:17.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:17.990+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f678002da30 con 0x7f67802036e0 2024-09-16T00:10:18.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.003+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f678002da30 con 0x7f67802036e0 2024-09-16T00:10:18.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.006+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f678002da30 con 0x7f67802036e0 2024-09-16T00:10:18.007 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:18.008 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:18.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.006+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f6780206520 0x7f6780208910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:18.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.006+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f678020b6e0 con 0x7f6780206520 2024-09-16T00:10:18.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.006+0000 7f677f7fe640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f6780206520 0x7f6780208910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:18.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.007+0000 7f677f7fe640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f6780206520 0x7f6780208910 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:18.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.008+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f678020b6e0 con 0x7f6780206520 2024-09-16T00:10:18.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.021+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6780208e50 con 0x7f6780206520 2024-09-16T00:10:18.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.023+0000 7f6777fff640 1 -- 172.21.15.139:0/2065105255 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6780208e50 con 0x7f6780206520 2024-09-16T00:10:18.025 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:10:18.027 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:10:18.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.023+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f67800c4bd0 msgr2=0x7f6780149220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.023+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f67800c4bd0 0x7f6780149220 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.023+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f6780149c70 msgr2=0x7f678014a0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.023+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f6780149c70 0x7f678014a0b0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.023+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f67801f2300 msgr2=0x7f67801f2740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.023+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f67801f2300 0x7f67801f2740 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.023+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f6780062f00 msgr2=0x7f67800c90e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.023+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f6780062f00 0x7f67800c90e0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.023+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f67800c9b30 msgr2=0x7f678002c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.023+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f67800c9b30 0x7f678002c640 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.023+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f678002d090 msgr2=0x7f678002d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.023+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f678002d090 0x7f678002d4f0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.023+0000 7f677ffff640 1 -- 172.21.15.139:0/2065105255 reap_dead start 2024-09-16T00:10:18.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.023+0000 7f677ffff640 1 -- 172.21.15.139:0/2065105255 reap_dead start 2024-09-16T00:10:18.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f67802036e0 msgr2=0x7f6780205ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f677ffff640 1 -- 172.21.15.139:0/2065105255 reap_dead start 2024-09-16T00:10:18.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f67802036e0 0x7f6780205ad0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f6780206520 msgr2=0x7f6780208910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f6780206520 0x7f6780208910 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f676006c2b0 msgr2=0x7f676006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f676006c2b0 0x7f676006e770 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f67800c5260 tx=0x7f6770008040 comp rx=0 tx=0).stop 2024-09-16T00:10:18.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6780071660 msgr2=0x7f67801ef2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6780071660 0x7f67801ef2a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6768000c00 tx=0x7f67680046c0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 shutdown_connections 2024-09-16T00:10:18.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f67802036e0 0x7f6780205ad0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f6780206520 0x7f6780208910 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f676006c2b0 0x7f676006e770 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f6786027640 1 --2- 172.21.15.139:0/2065105255 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6780071660 0x7f67801ef2a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 >> 172.21.15.139:0/2065105255 conn(0x7f6780073950 msgr2=0x7f67800cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:18.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 shutdown_connections 2024-09-16T00:10:18.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.024+0000 7f6786027640 1 -- 172.21.15.139:0/2065105255 wait complete. 2024-09-16T00:10:18.057 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:18.056+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:18.088 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:18.088 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:18.100 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:10:18.158 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:18.156+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:18.250 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:18.251 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:18.259 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:10:18.260 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:18.258+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:18.337 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.335+0000 7f9f567fc640 1 -- 172.21.15.139:0/3877160095 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v108) ==== 87+0+0 (secure 0 0 0) 0x7f9f5005ccf0 con 0x7f9f600caee0 2024-09-16T00:10:18.337 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is set 2024-09-16T00:10:18.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.337+0000 7f9f66b9f640 1 -- 172.21.15.139:0/3877160095 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9f4406c2b0 msgr2=0x7f9f4406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.337+0000 7f9f66b9f640 1 --2- 172.21.15.139:0/3877160095 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9f4406c2b0 0x7f9f4406e770 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c034000 tx=0x7f9f5c02e000 comp rx=0 tx=0).stop 2024-09-16T00:10:18.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.337+0000 7f9f66b9f640 1 -- 172.21.15.139:0/3877160095 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f600caee0 msgr2=0x7f9f60156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:18.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.337+0000 7f9f66b9f640 1 --2- 172.21.15.139:0/3877160095 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f600caee0 0x7f9f60156780 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000c500 tx=0x7f9f5000c9c0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.337+0000 7f9f66b9f640 1 -- 172.21.15.139:0/3877160095 shutdown_connections 2024-09-16T00:10:18.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.337+0000 7f9f66b9f640 1 --2- 172.21.15.139:0/3877160095 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9f4406c2b0 0x7f9f4406e770 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.337+0000 7f9f66b9f640 1 --2- 172.21.15.139:0/3877160095 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9f60154b00 0x7f9f600cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.337+0000 7f9f66b9f640 1 --2- 172.21.15.139:0/3877160095 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f600caee0 0x7f9f60156780 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.337+0000 7f9f66b9f640 1 --2- 172.21.15.139:0/3877160095 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9f600b44f0 0x7f9f6014c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:18.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.337+0000 7f9f66b9f640 1 -- 172.21.15.139:0/3877160095 >> 172.21.15.139:0/3877160095 conn(0x7f9f6006c500 msgr2=0x7f9f60151a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:18.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.337+0000 7f9f66b9f640 1 -- 172.21.15.139:0/3877160095 shutdown_connections 2024-09-16T00:10:18.340 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:18.337+0000 7f9f66b9f640 1 -- 172.21.15.139:0/3877160095 wait complete. 2024-09-16T00:10:18.359 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:18.358+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:18.448 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:18.449 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:18.459 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:10:18.500 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:18.499+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:18.561 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:18.560+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:18.638 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:18.639 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:18.649 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:10:18.693 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:18.692+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:18.758 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:18.758 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:18.763 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:18.762+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:18.768 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:10:18.864 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:18.863+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:18.915 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:18.915 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:18.925 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:10:18.966 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:18.965+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:19.066 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:19.064+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:19.079 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:19.079 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:19.091 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:10:19.167 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:19.165+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:19.208 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:19.209 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:19.218 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:10:19.267 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:19.265+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:19.326 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:19.326 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:19.337 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:10:19.368 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:19.367+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:19.469 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:19.468+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:19.485 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:19.485 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:19.494 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:10:19.571 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:19.570+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:19.644 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:19.644 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:19.653 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:10:19.672 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:19.670+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:19.768 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:19.768 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:19.772 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:19.771+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:19.777 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:10:19.873 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:19.872+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:19.933 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:19.933 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:19.942 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:10:19.984 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:19.982+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:20.037 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:10:20.080 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:20.079+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:20.096 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:20.096 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:20.105 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:10:20.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.137+0000 7f015cf8f640 1 Processor -- start 2024-09-16T00:10:20.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.137+0000 7f015cf8f640 1 -- start start 2024-09-16T00:10:20.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.138+0000 7f015cf8f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01580d0490 0x7f01580b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.138+0000 7f015cf8f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01580b3180 con 0x7f01580d0870 2024-09-16T00:10:20.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.138+0000 7f015cf8f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f015815adc0 con 0x7f01580b4750 2024-09-16T00:10:20.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.138+0000 7f015cf8f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f015815b940 con 0x7f01580d0490 2024-09-16T00:10:20.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.138+0000 7f0155d9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f01580d0870 0x7f01580b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56238/0 (socket says 172.21.15.139:56238) 2024-09-16T00:10:20.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.138+0000 7f0155d9b640 1 -- 172.21.15.139:0/41436738 learned_addr learned my addr 172.21.15.139:0/41436738 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:20.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.138+0000 7f015659c640 1 --2- 172.21.15.139:0/41436738 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01580d0490 0x7f01580b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.139+0000 7f015559a640 1 -- 172.21.15.139:0/41436738 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4052683370 0 0) 0x7f01580b3180 con 0x7f01580d0870 2024-09-16T00:10:20.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.139+0000 7f015559a640 1 -- 172.21.15.139:0/41436738 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0140003520 con 0x7f01580d0870 2024-09-16T00:10:20.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.139+0000 7f015559a640 1 -- 172.21.15.139:0/41436738 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1316192748 0 0) 0x7f015815adc0 con 0x7f01580b4750 2024-09-16T00:10:20.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.139+0000 7f015559a640 1 -- 172.21.15.139:0/41436738 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01580b3180 con 0x7f01580b4750 2024-09-16T00:10:20.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.139+0000 7f015659c640 1 -- 172.21.15.139:0/41436738 >> v1:172.21.15.164:6789/0 conn(0x7f01580b4750 legacy=0x7f01581546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.139+0000 7f015659c640 1 -- 172.21.15.139:0/41436738 >> v1:172.21.15.139:6789/0 conn(0x7f01580d0870 legacy=0x7f01580b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.139+0000 7f015659c640 1 -- 172.21.15.139:0/41436738 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f015815b0a0 con 0x7f01580d0490 2024-09-16T00:10:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.140+0000 7f015659c640 1 --2- 172.21.15.139:0/41436738 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01580d0490 0x7f01580b2c40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0148009800 tx=0x7f014802a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c86922b5bb44b0d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.140+0000 7f015559a640 1 -- 172.21.15.139:0/41436738 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148037070 con 0x7f01580d0490 2024-09-16T00:10:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.140+0000 7f015559a640 1 -- 172.21.15.139:0/41436738 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0148032440 con 0x7f01580d0490 2024-09-16T00:10:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.140+0000 7f015559a640 1 -- 172.21.15.139:0/41436738 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0148032810 con 0x7f01580d0490 2024-09-16T00:10:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.140+0000 7f015cf8f640 1 -- 172.21.15.139:0/41436738 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01580d0490 msgr2=0x7f01580b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.140+0000 7f015cf8f640 1 --2- 172.21.15.139:0/41436738 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01580d0490 0x7f01580b2c40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0148009800 tx=0x7f014802a2c0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.141+0000 7f015cf8f640 1 -- 172.21.15.139:0/41436738 shutdown_connections 2024-09-16T00:10:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.141+0000 7f015cf8f640 1 --2- 172.21.15.139:0/41436738 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01580d0490 0x7f01580b2c40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.141+0000 7f015cf8f640 1 -- 172.21.15.139:0/41436738 >> 172.21.15.139:0/41436738 conn(0x7f015806c520 msgr2=0x7f015806c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:20.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.141+0000 7f015cf8f640 1 -- 172.21.15.139:0/41436738 shutdown_connections 2024-09-16T00:10:20.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.141+0000 7f015cf8f640 1 -- 172.21.15.139:0/41436738 wait complete. 2024-09-16T00:10:20.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.142+0000 7f015cf8f640 1 Processor -- start 2024-09-16T00:10:20.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.142+0000 7f015cf8f640 1 -- start start 2024-09-16T00:10:20.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f015cf8f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01580b4750 0x7f01581f3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f015cf8f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01580d0490 0x7f01580c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f015cf8f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01580d0870 0x7f01580ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f015cf8f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f015815b660 con 0x7f01580d0870 2024-09-16T00:10:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f015cf8f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f01580b3bb0 con 0x7f01580d0490 2024-09-16T00:10:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f015cf8f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f015815a980 con 0x7f01580b4750 2024-09-16T00:10:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f0156d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01580d0870 0x7f01580ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f0156d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01580d0870 0x7f01580ca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52108/0 (socket says 172.21.15.139:52108) 2024-09-16T00:10:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f0156d9d640 1 -- 172.21.15.139:0/330975597 learned_addr learned my addr 172.21.15.139:0/330975597 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f0155d9b640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01580d0490 0x7f01580c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f015659c640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01580b4750 0x7f01581f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f0156d9d640 1 -- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01580b4750 msgr2=0x7f01581f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f0156d9d640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f01580b4750 0x7f01581f3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f0156d9d640 1 -- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01580d0490 msgr2=0x7f01580c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f0156d9d640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01580d0490 0x7f01580c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f0156d9d640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01580c3d30 con 0x7f01580d0870 2024-09-16T00:10:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f0155d9b640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01580d0490 0x7f01580c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f0156d9d640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01580d0870 0x7f01580ca440 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f015000c7a0 tx=0x7f015000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150003b70 con 0x7f01580d0870 2024-09-16T00:10:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.143+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0150003d10 con 0x7f01580d0870 2024-09-16T00:10:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.144+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01580c86b0 con 0x7f01580d0870 2024-09-16T00:10:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.144+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015000aa30 con 0x7f01580d0870 2024-09-16T00:10:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.144+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01580c8c50 con 0x7f01580d0870 2024-09-16T00:10:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.144+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f01580b3430 con 0x7f01580d0870 2024-09-16T00:10:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.145+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f015000abd0 con 0x7f01580d0870 2024-09-16T00:10:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.145+0000 7f0146ffd640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f013806c2b0 0x7f013806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.146+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5973+0+0 (secure 0 0 0) 0x7f015008b330 con 0x7f01580d0870 2024-09-16T00:10:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.146+0000 7f015659c640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f013806c2b0 0x7f013806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.146+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v109) ==== 54+0+16 (secure 0 0 0) 0x7f01580c3d30 con 0x7f01580d0870 2024-09-16T00:10:20.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.146+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f01580c98e0 0x7f01580cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.146+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01580cb3d0 con 0x7f01580c98e0 2024-09-16T00:10:20.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.146+0000 7f015659c640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f013806c2b0 0x7f013806e770 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0148034000 tx=0x7f014802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:20.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.146+0000 7f0155d9b640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f01580c98e0 0x7f01580cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.146+0000 7f0155d9b640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f01580c98e0 0x7f01580cf080 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:20.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.147+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f01580cb3d0 con 0x7f01580c98e0 2024-09-16T00:10:20.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.163+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f01580ceb60 con 0x7f01580c98e0 2024-09-16T00:10:20.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.164+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f01580ceb60 con 0x7f01580c98e0 2024-09-16T00:10:20.166 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:20.167 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:20.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.165+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f01581f28f0 0x7f01581f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.165+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f01581f2d50 con 0x7f01581f28f0 2024-09-16T00:10:20.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.165+0000 7f015659c640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f01581f28f0 0x7f01581f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.165+0000 7f015659c640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f01581f28f0 0x7f01581f3ed0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:20.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.166+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f01581f2d50 con 0x7f01581f28f0 2024-09-16T00:10:20.177 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:20.175+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:20.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.180+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f01580ce880 con 0x7f01581f28f0 2024-09-16T00:10:20.184 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.182+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f01580ce880 con 0x7f01581f28f0 2024-09-16T00:10:20.184 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:20.185 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:20.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.182+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f01581f4920 0x7f0158062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.182+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f01580ce5a0 con 0x7f01581f4920 2024-09-16T00:10:20.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.182+0000 7f0155d9b640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f01581f4920 0x7f0158062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.183+0000 7f0155d9b640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f01581f4920 0x7f0158062630 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:20.185 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.183+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f01580ce5a0 con 0x7f01581f4920 2024-09-16T00:10:20.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.197+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f01580c8290 con 0x7f01581f4920 2024-09-16T00:10:20.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.199+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f01580c8290 con 0x7f01581f4920 2024-09-16T00:10:20.201 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:20.201 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:20.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.199+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0158063080 0x7f015814ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.199+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f0158063500 con 0x7f0158063080 2024-09-16T00:10:20.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.199+0000 7f0156d9d640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0158063080 0x7f015814ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.199+0000 7f0156d9d640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0158063080 0x7f015814ebb0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:20.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.200+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0158063500 con 0x7f0158063080 2024-09-16T00:10:20.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.214+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f015814f340 con 0x7f0158063080 2024-09-16T00:10:20.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.216+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f015814f340 con 0x7f0158063080 2024-09-16T00:10:20.218 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:20.220 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:20.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.216+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f015802c640 0x7f015802caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.216+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f015814fa60 con 0x7f015802c640 2024-09-16T00:10:20.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.216+0000 7f015659c640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f015802c640 0x7f015802caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.217+0000 7f015659c640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f015802c640 0x7f015802caa0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:20.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.218+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f015814fa60 con 0x7f015802c640 2024-09-16T00:10:20.233 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.231+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f015814fa60 con 0x7f015802c640 2024-09-16T00:10:20.235 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.233+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f015814fa60 con 0x7f015802c640 2024-09-16T00:10:20.235 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:20.236 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:20.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.233+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f015802d4f0 0x7f0158207260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.233+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f015802d970 con 0x7f015802d4f0 2024-09-16T00:10:20.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.233+0000 7f0155d9b640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f015802d4f0 0x7f0158207260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.234+0000 7f0155d9b640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f015802d4f0 0x7f0158207260 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:20.236 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.234+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f015802d970 con 0x7f015802d4f0 2024-09-16T00:10:20.248 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:20.248 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:20.248 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:20.248 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:20.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.248+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f015802d970 con 0x7f015802d4f0 2024-09-16T00:10:20.252 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.250+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f015802d970 con 0x7f015802d4f0 2024-09-16T00:10:20.252 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:20.253 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:20.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.250+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f0158207cb0 0x7f015820a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.250+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f015802db50 con 0x7f0158207cb0 2024-09-16T00:10:20.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.250+0000 7f0156d9d640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f0158207cb0 0x7f015820a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.251+0000 7f0156d9d640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f0158207cb0 0x7f015820a0a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:20.253 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.252+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f015802db50 con 0x7f0158207cb0 2024-09-16T00:10:20.257 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:10:20.268 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.266+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f015802db50 con 0x7f0158207cb0 2024-09-16T00:10:20.270 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.269+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f015802db50 con 0x7f0158207cb0 2024-09-16T00:10:20.270 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:20.271 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:20.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.269+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f015820aaf0 0x7f015820cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.269+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f015820fcb0 con 0x7f015820aaf0 2024-09-16T00:10:20.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.269+0000 7f015659c640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f015820aaf0 0x7f015820cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.271 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.269+0000 7f015659c640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f015820aaf0 0x7f015820cee0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:20.272 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.270+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f015820fcb0 con 0x7f015820aaf0 2024-09-16T00:10:20.278 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:20.276+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:20.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.285+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f015820d420 con 0x7f015820aaf0 2024-09-16T00:10:20.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f0146ffd640 1 -- 172.21.15.139:0/330975597 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f015820d420 con 0x7f015820aaf0 2024-09-16T00:10:20.289 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:10:20.290 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:10:20.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f01580c98e0 msgr2=0x7f01580cf080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f01580c98e0 0x7f01580cf080 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f01581f28f0 msgr2=0x7f01581f3ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f01581f28f0 0x7f01581f3ed0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f01581f4920 msgr2=0x7f0158062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f01581f4920 0x7f0158062630 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0158063080 msgr2=0x7f015814ebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0158063080 0x7f015814ebb0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f0156d9d640 1 -- 172.21.15.139:0/330975597 reap_dead start 2024-09-16T00:10:20.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f015802c640 msgr2=0x7f015802caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f015802c640 0x7f015802caa0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f0156d9d640 1 -- 172.21.15.139:0/330975597 reap_dead start 2024-09-16T00:10:20.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f015802d4f0 msgr2=0x7f0158207260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f015802d4f0 0x7f0158207260 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f0158207cb0 msgr2=0x7f015820a0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f0158207cb0 0x7f015820a0a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f015820aaf0 msgr2=0x7f015820cee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f015820aaf0 0x7f015820cee0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f013806c2b0 msgr2=0x7f013806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f013806c2b0 0x7f013806e770 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0148034000 tx=0x7f014802e000 comp rx=0 tx=0).stop 2024-09-16T00:10:20.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01580d0870 msgr2=0x7f01580ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f015cf8f640 1 --2- 172.21.15.139:0/330975597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01580d0870 0x7f01580ca440 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f015000c7a0 tx=0x7f015000cc60 comp rx=0 tx=0).stop 2024-09-16T00:10:20.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f0156d9d640 1 -- 172.21.15.139:0/330975597 reap_dead start 2024-09-16T00:10:20.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.287+0000 7f0156d9d640 1 -- 172.21.15.139:0/330975597 reap_dead start 2024-09-16T00:10:20.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.288+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 shutdown_connections 2024-09-16T00:10:20.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.288+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 >> 172.21.15.139:0/330975597 conn(0x7f015806c520 msgr2=0x7f0158159cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:20.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.288+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 shutdown_connections 2024-09-16T00:10:20.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.288+0000 7f015cf8f640 1 -- 172.21.15.139:0/330975597 wait complete. 2024-09-16T00:10:20.359 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T00:10:20.389 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:20.389 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:20.389 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:20.390 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:20.390 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:20.390 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:20.388+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:20.399 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:10:20.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.468+0000 7f6168fc6640 1 Processor -- start 2024-09-16T00:10:20.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.468+0000 7f6168fc6640 1 -- start start 2024-09-16T00:10:20.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.469+0000 7f6168fc6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f61640ce430 0x7f61640b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.469+0000 7f6168fc6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61640ce810 0x7f61640c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.469+0000 7f6168fc6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61640703e0 con 0x7f61640c9430 2024-09-16T00:10:20.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.469+0000 7f6168fc6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f61640c5750 con 0x7f61640ce430 2024-09-16T00:10:20.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.469+0000 7f6168fc6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6164070680 con 0x7f61640ce810 2024-09-16T00:10:20.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.470+0000 7f616259c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f61640ce430 0x7f61640b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.470+0000 7f6162d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f61640c9430 0x7f61640ca2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56244/0 (socket says 172.21.15.139:56244) 2024-09-16T00:10:20.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.470+0000 7f616259c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f61640ce430 0x7f61640b31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52114/0 (socket says 172.21.15.139:52114) 2024-09-16T00:10:20.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.470+0000 7f6162d9d640 1 -- 172.21.15.139:0/2311407125 learned_addr learned my addr 172.21.15.139:0/2311407125 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:20.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.470+0000 7f6161d9b640 1 --2- 172.21.15.139:0/2311407125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61640ce810 0x7f61640c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.470+0000 7f616259c640 1 -- 172.21.15.139:0/2311407125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61640ce810 msgr2=0x7f61640c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:20.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.470+0000 7f616259c640 1 --2- 172.21.15.139:0/2311407125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61640ce810 0x7f61640c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.470+0000 7f616259c640 1 -- 172.21.15.139:0/2311407125 >> v1:172.21.15.139:6789/0 conn(0x7f61640c9430 legacy=0x7f61640ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.470+0000 7f616259c640 1 -- 172.21.15.139:0/2311407125 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6164070920 con 0x7f61640ce430 2024-09-16T00:10:20.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.471+0000 7f616259c640 1 --2- 172.21.15.139:0/2311407125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61640ce430 0x7f61640b31c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f61640c58d0 tx=0x7f615402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac9edf1f323f7dc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:20.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.471+0000 7f616159a640 1 -- 172.21.15.139:0/2311407125 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6154037070 con 0x7f61640ce430 2024-09-16T00:10:20.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.471+0000 7f616159a640 1 -- 172.21.15.139:0/2311407125 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6154032440 con 0x7f61640ce430 2024-09-16T00:10:20.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.471+0000 7f616159a640 1 -- 172.21.15.139:0/2311407125 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6154032810 con 0x7f61640ce430 2024-09-16T00:10:20.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.471+0000 7f6168fc6640 1 -- 172.21.15.139:0/2311407125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61640ce430 msgr2=0x7f61640b31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.471+0000 7f6168fc6640 1 --2- 172.21.15.139:0/2311407125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61640ce430 0x7f61640b31c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f61640c58d0 tx=0x7f615402a180 comp rx=0 tx=0).stop 2024-09-16T00:10:20.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.472+0000 7f6168fc6640 1 -- 172.21.15.139:0/2311407125 shutdown_connections 2024-09-16T00:10:20.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.472+0000 7f6168fc6640 1 --2- 172.21.15.139:0/2311407125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61640ce810 0x7f61640c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.472+0000 7f6168fc6640 1 --2- 172.21.15.139:0/2311407125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61640ce430 0x7f61640b31c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.472+0000 7f6168fc6640 1 -- 172.21.15.139:0/2311407125 >> 172.21.15.139:0/2311407125 conn(0x7f616406c150 msgr2=0x7f6164067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:20.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.472+0000 7f6168fc6640 1 -- 172.21.15.139:0/2311407125 shutdown_connections 2024-09-16T00:10:20.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.472+0000 7f6168fc6640 1 -- 172.21.15.139:0/2311407125 wait complete. 2024-09-16T00:10:20.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.473+0000 7f6168fc6640 1 Processor -- start 2024-09-16T00:10:20.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.473+0000 7f6168fc6640 1 -- start start 2024-09-16T00:10:20.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.474+0000 7f6168fc6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61640c9430 0x7f6164152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.474+0000 7f6168fc6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61640ce430 0x7f61640c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.474+0000 7f6168fc6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61640ce810 0x7f61640c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.474+0000 7f6168fc6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f61640b3700 con 0x7f61640ce810 2024-09-16T00:10:20.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.474+0000 7f6168fc6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6164070ee0 con 0x7f61640c9430 2024-09-16T00:10:20.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.474+0000 7f6168fc6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f61640b3880 con 0x7f61640ce430 2024-09-16T00:10:20.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.474+0000 7f6162d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61640ce810 0x7f61640c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.474+0000 7f616259c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61640c9430 0x7f6164152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.474+0000 7f6162d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61640ce810 0x7f61640c70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52126/0 (socket says 172.21.15.139:52126) 2024-09-16T00:10:20.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.474+0000 7f6162d9d640 1 -- 172.21.15.139:0/2882474046 learned_addr learned my addr 172.21.15.139:0/2882474046 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:20.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.474+0000 7f6161d9b640 1 --2- 172.21.15.139:0/2882474046 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61640ce430 0x7f61640c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.474+0000 7f6162d9d640 1 -- 172.21.15.139:0/2882474046 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61640ce430 msgr2=0x7f61640c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.474+0000 7f6162d9d640 1 --2- 172.21.15.139:0/2882474046 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61640ce430 0x7f61640c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.475+0000 7f6162d9d640 1 -- 172.21.15.139:0/2882474046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61640c9430 msgr2=0x7f6164152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:20.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.475+0000 7f6162d9d640 1 --2- 172.21.15.139:0/2882474046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61640c9430 0x7f6164152260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:20.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.475+0000 7f6162d9d640 1 -- 172.21.15.139:0/2882474046 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61641ee8d0 con 0x7f61640ce810 2024-09-16T00:10:20.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.475+0000 7f6162d9d640 1 --2- 172.21.15.139:0/2882474046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61640ce810 0x7f61640c70f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f615c00cbf0 tx=0x7f615c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:20.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.475+0000 7f6152ffd640 1 -- 172.21.15.139:0/2882474046 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c017070 con 0x7f61640ce810 2024-09-16T00:10:20.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.475+0000 7f6152ffd640 1 -- 172.21.15.139:0/2882474046 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f615c00ad40 con 0x7f61640ce810 2024-09-16T00:10:20.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.475+0000 7f6168fc6640 1 -- 172.21.15.139:0/2882474046 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61641eeb90 con 0x7f61640ce810 2024-09-16T00:10:20.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.475+0000 7f6152ffd640 1 -- 172.21.15.139:0/2882474046 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c012600 con 0x7f61640ce810 2024-09-16T00:10:20.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.475+0000 7f6168fc6640 1 -- 172.21.15.139:0/2882474046 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61641ef150 con 0x7f61640ce810 2024-09-16T00:10:20.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.476+0000 7f6168fc6640 1 -- 172.21.15.139:0/2882474046 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61640cedf0 con 0x7f61640ce810 2024-09-16T00:10:20.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.477+0000 7f6152ffd640 1 -- 172.21.15.139:0/2882474046 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f615c01a070 con 0x7f61640ce810 2024-09-16T00:10:20.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.477+0000 7f6152ffd640 1 --2- 172.21.15.139:0/2882474046 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f614406c2b0 0x7f614406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:20.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.477+0000 7f6152ffd640 1 -- 172.21.15.139:0/2882474046 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5973+0+0 (secure 0 0 0) 0x7f615c08b290 con 0x7f61640ce810 2024-09-16T00:10:20.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.477+0000 7f616259c640 1 --2- 172.21.15.139:0/2882474046 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f614406c2b0 0x7f614406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:20.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.478+0000 7f616259c640 1 --2- 172.21.15.139:0/2882474046 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f614406c2b0 0x7f614406e770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6154034000 tx=0x7f615402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:20.481 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:20.479+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:20.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.482+0000 7f6152ffd640 1 -- 172.21.15.139:0/2882474046 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f61640cedf0 con 0x7f61640ce810 2024-09-16T00:10:20.555 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:20.555 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:20.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:20.555 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:20.556 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.556 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5850 3.3a 3:5c3cfc2b:::10000001adf.00000000:head [write 0~116] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:20.556 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:20.556 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.137250530000003, 2024-09-16T00:10:20.556 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66931295800000001, 2024-09-16T00:10:20.556 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.556 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.556 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.556 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.556 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.556 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.557 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5850 2024-09-16T00:10:20.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.557 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:20.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.558 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:20.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415938+0000", 2024-09-16T00:10:20.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7099999999999999e-06 2024-09-16T00:10:20.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.559 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415945+0000", 2024-09-16T00:10:20.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:10:20.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.559 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415959+0000", 2024-09-16T00:10:20.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3545e-05 2024-09-16T00:10:20.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415966+0000", 2024-09-16T00:10:20.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6379999999999998e-06 2024-09-16T00:10:20.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416013+0000", 2024-09-16T00:10:20.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7314999999999997e-05 2024-09-16T00:10:20.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:20.561 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416020+0000", 2024-09-16T00:10:20.561 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.52e-06 2024-09-16T00:10:20.561 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.561 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.561 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.562 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082393+0000", 2024-09-16T00:10:20.562 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66637251799999997 2024-09-16T00:10:20.562 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.562 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.562 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.563 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082537+0000", 2024-09-16T00:10:20.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014418800000000001 2024-09-16T00:10:20.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.563 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:20.563 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082689+0000", 2024-09-16T00:10:20.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015198499999999999 2024-09-16T00:10:20.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.563 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:20.563 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084930+0000", 2024-09-16T00:10:20.563 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002241041 2024-09-16T00:10:20.563 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.563 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:20.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084994+0000", 2024-09-16T00:10:20.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3274000000000004e-05 2024-09-16T00:10:20.564 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.564 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.564 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.564 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085049+0000", 2024-09-16T00:10:20.564 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.554e-05 2024-09-16T00:10:20.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.565 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.565 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.565 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085250+0000", 2024-09-16T00:10:20.565 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020060799999999999 2024-09-16T00:10:20.565 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.565 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.565 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.565 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.565 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.565 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5910 3.3a 3:5ec98a62:::10000001b22.00000000:head [write 0~1274] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:20.566 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:20.566 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.990306892000007, 2024-09-16T00:10:20.566 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.52286026600000002, 2024-09-16T00:10:20.566 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.566 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.566 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.566 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.566 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.566 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.566 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5910 2024-09-16T00:10:20.566 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.566 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.566 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:20.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.567 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.567 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:20.567 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.567 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.567 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.568 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.568 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562881+0000", 2024-09-16T00:10:20.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:10:20.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.568 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.568 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562887+0000", 2024-09-16T00:10:20.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2870000000000004e-06 2024-09-16T00:10:20.568 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.568 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.568 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.568 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562897+0000", 2024-09-16T00:10:20.568 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.064e-05 2024-09-16T00:10:20.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562902+0000", 2024-09-16T00:10:20.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2340000000000002e-06 2024-09-16T00:10:20.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.569 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562958+0000", 2024-09-16T00:10:20.569 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5523999999999999e-05 2024-09-16T00:10:20.569 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.569 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.569 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:20.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562968+0000", 2024-09-16T00:10:20.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0394000000000001e-05 2024-09-16T00:10:20.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083461+0000", 2024-09-16T00:10:20.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.520492968 2024-09-16T00:10:20.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.570 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.570 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083645+0000", 2024-09-16T00:10:20.570 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018346399999999999 2024-09-16T00:10:20.570 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.570 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.571 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:20.571 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083844+0000", 2024-09-16T00:10:20.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000199268 2024-09-16T00:10:20.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.571 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:20.571 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085410+0000", 2024-09-16T00:10:20.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015662359999999999 2024-09-16T00:10:20.571 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.571 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.571 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:20.571 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085493+0000", 2024-09-16T00:10:20.571 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2478999999999994e-05 2024-09-16T00:10:20.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085563+0000", 2024-09-16T00:10:20.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0159000000000004e-05 2024-09-16T00:10:20.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.572 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.572 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.572 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085741+0000", 2024-09-16T00:10:20.572 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017764300000000001 2024-09-16T00:10:20.572 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.572 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.572 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.572 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.573 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.573 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5962 3.3a 3:5c7b4d29:::10000001b5d.00000000:head [write 0~3074] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:20.573 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:20.573 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.860770075000005, 2024-09-16T00:10:20.573 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39368737500000001, 2024-09-16T00:10:20.573 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.573 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.573 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.573 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.573 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.573 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.573 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5962 2024-09-16T00:10:20.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.574 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.574 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:20.574 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.574 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:20.574 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.574 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.574 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.574 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692418+0000", 2024-09-16T00:10:20.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3510000000000001e-06 2024-09-16T00:10:20.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.575 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.575 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692425+0000", 2024-09-16T00:10:20.575 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3770000000000001e-06 2024-09-16T00:10:20.575 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.575 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.576 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.576 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692440+0000", 2024-09-16T00:10:20.576 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4975e-05 2024-09-16T00:10:20.576 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.576 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692448+0000", 2024-09-16T00:10:20.577 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6279999999999994e-06 2024-09-16T00:10:20.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.577 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.577 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692499+0000", 2024-09-16T00:10:20.577 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0872999999999999e-05 2024-09-16T00:10:20.577 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:20.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692507+0000", 2024-09-16T00:10:20.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9889999999999992e-06 2024-09-16T00:10:20.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084221+0000", 2024-09-16T00:10:20.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39171379699999997 2024-09-16T00:10:20.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084446+0000", 2024-09-16T00:10:20.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022519900000000001 2024-09-16T00:10:20.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:20.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084672+0000", 2024-09-16T00:10:20.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000225405 2024-09-16T00:10:20.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:20.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085832+0000", 2024-09-16T00:10:20.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011600040000000001 2024-09-16T00:10:20.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:20.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085864+0000", 2024-09-16T00:10:20.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2592999999999998e-05 2024-09-16T00:10:20.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085943+0000", 2024-09-16T00:10:20.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8534000000000004e-05 2024-09-16T00:10:20.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.581 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.581 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086105+0000", 2024-09-16T00:10:20.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016165 2024-09-16T00:10:20.581 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.581 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.581 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.581 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5963 3.3a 3:5e4192f5:::10000001b5e.00000000:head [write 0~152] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:20.581 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:20.582 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.858705643999997, 2024-09-16T00:10:20.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39413463900000001, 2024-09-16T00:10:20.582 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.582 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.582 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.582 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.583 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.583 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.583 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5963 2024-09-16T00:10:20.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.583 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.583 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:20.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.583 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.583 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:20.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.584 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.584 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:20.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:10:20.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.584 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.584 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694486+0000", 2024-09-16T00:10:20.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0960000000000003e-06 2024-09-16T00:10:20.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.584 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694496+0000", 2024-09-16T00:10:20.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2189999999999993e-06 2024-09-16T00:10:20.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694500+0000", 2024-09-16T00:10:20.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9529999999999998e-06 2024-09-16T00:10:20.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694552+0000", 2024-09-16T00:10:20.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1360999999999998e-05 2024-09-16T00:10:20.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:20.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694561+0000", 2024-09-16T00:10:20.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1700000000000003e-06 2024-09-16T00:10:20.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086346+0000", 2024-09-16T00:10:20.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39178468 2024-09-16T00:10:20.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.587 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.587 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086480+0000", 2024-09-16T00:10:20.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013469299999999999 2024-09-16T00:10:20.587 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.587 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:20.587 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086790+0000", 2024-09-16T00:10:20.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030923300000000001 2024-09-16T00:10:20.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:20.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088361+0000", 2024-09-16T00:10:20.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015717050000000001 2024-09-16T00:10:20.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:20.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088432+0000", 2024-09-16T00:10:20.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0922999999999998e-05 2024-09-16T00:10:20.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.590 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088489+0000", 2024-09-16T00:10:20.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6653999999999997e-05 2024-09-16T00:10:20.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088616+0000", 2024-09-16T00:10:20.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012730099999999999 2024-09-16T00:10:20.591 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.591 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.592 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.592 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:6082 3.3a 3:5c50f65b:::10000001be7.00000000:head [write 0~2439] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:20.592 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:20.592 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 64.564932787999993, 2024-09-16T00:10:20.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.10065850799999999, 2024-09-16T00:10:20.593 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.593 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.593 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.593 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.594 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.594 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.594 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 6082 2024-09-16T00:10:20.594 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.594 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.595 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.595 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.595 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:20.595 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.595 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.596 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.596 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:20.596 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.596 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.596 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.597 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.597 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988256+0000", 2024-09-16T00:10:20.597 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.065e-06 2024-09-16T00:10:20.597 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.597 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.598 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988262+0000", 2024-09-16T00:10:20.598 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4640000000000003e-06 2024-09-16T00:10:20.598 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.599 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.599 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988272+0000", 2024-09-16T00:10:20.599 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0335e-05 2024-09-16T00:10:20.599 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.599 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.600 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.600 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988279+0000", 2024-09-16T00:10:20.600 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:10:20.600 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.600 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.601 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.601 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988324+0000", 2024-09-16T00:10:20.601 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4953000000000001e-05 2024-09-16T00:10:20.601 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.601 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.601 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:20.602 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988331+0000", 2024-09-16T00:10:20.602 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5989999999999998e-06 2024-09-16T00:10:20.602 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.602 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.602 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.603 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087272+0000", 2024-09-16T00:10:20.603 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.098941045000000005 2024-09-16T00:10:20.603 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.603 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.603 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.603 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087486+0000", 2024-09-16T00:10:20.604 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00021360199999999999 2024-09-16T00:10:20.604 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.604 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.604 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:20.604 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087740+0000", 2024-09-16T00:10:20.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00025456799999999999 2024-09-16T00:10:20.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.605 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.605 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:20.605 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088638+0000", 2024-09-16T00:10:20.605 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00089793900000000005 2024-09-16T00:10:20.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:20.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088673+0000", 2024-09-16T00:10:20.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5111999999999999e-05 2024-09-16T00:10:20.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.606 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088722+0000", 2024-09-16T00:10:20.606 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8758000000000003e-05 2024-09-16T00:10:20.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.606 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.606 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.607 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088913+0000", 2024-09-16T00:10:20.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000190953 2024-09-16T00:10:20.607 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.607 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.607 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.607 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.607 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.607 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10643 3.21 3:8453163e:::100000030b3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:20.607 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:20.607 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 44.953456594999999, 2024-09-16T00:10:20.607 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013028809000000001, 2024-09-16T00:10:20.607 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.607 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.607 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.607 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.608 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.608 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.608 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10643 2024-09-16T00:10:20.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.608 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.608 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:20.608 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.608 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.608 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.608 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.608 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:20.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.609 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599734+0000", 2024-09-16T00:10:20.609 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-06 2024-09-16T00:10:20.609 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.609 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.609 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603345+0000", 2024-09-16T00:10:20.610 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003611628 2024-09-16T00:10:20.610 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.610 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.610 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.610 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603368+0000", 2024-09-16T00:10:20.611 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3017e-05 2024-09-16T00:10:20.611 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.611 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.611 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.611 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603379+0000", 2024-09-16T00:10:20.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0528e-05 2024-09-16T00:10:20.612 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.612 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.612 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.612 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603448+0000", 2024-09-16T00:10:20.612 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9080999999999998e-05 2024-09-16T00:10:20.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.613 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.613 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.613 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603623+0000", 2024-09-16T00:10:20.613 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017525599999999999 2024-09-16T00:10:20.613 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.614 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.614 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:20.614 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603766+0000", 2024-09-16T00:10:20.614 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000142977 2024-09-16T00:10:20.614 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.615 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:20.615 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.611250+0000", 2024-09-16T00:10:20.615 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074835240000000001 2024-09-16T00:10:20.615 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.615 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:20.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612617+0000", 2024-09-16T00:10:20.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001367793 2024-09-16T00:10:20.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.616 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612700+0000", 2024-09-16T00:10:20.616 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3123999999999999e-05 2024-09-16T00:10:20.616 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.616 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.616 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.617 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612759+0000", 2024-09-16T00:10:20.617 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8950999999999999e-05 2024-09-16T00:10:20.617 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.617 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.617 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.617 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.617 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.617 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10652 3.22 3:44e46336:::100000030c1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:20.617 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:10:20.617 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 44.925404962999998, 2024-09-16T00:10:20.618 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011405732, 2024-09-16T00:10:20.618 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.618 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.618 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.618 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.618 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.618 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.619 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10652 2024-09-16T00:10:20.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.619 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.619 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.619 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:10:20.619 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.619 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.619 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.619 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.619 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:10:20.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627785+0000", 2024-09-16T00:10:20.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6970000000000002e-06 2024-09-16T00:10:20.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.620 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.620 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629610+0000", 2024-09-16T00:10:20.620 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018250359999999999 2024-09-16T00:10:20.620 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.620 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629639+0000", 2024-09-16T00:10:20.621 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8676000000000001e-05 2024-09-16T00:10:20.621 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.621 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.621 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.621 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629652+0000", 2024-09-16T00:10:20.622 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3572e-05 2024-09-16T00:10:20.622 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.622 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.622 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.622 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629735+0000", 2024-09-16T00:10:20.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.25e-05 2024-09-16T00:10:20.623 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.623 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.623 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.623 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629925+0000", 2024-09-16T00:10:20.623 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019011000000000001 2024-09-16T00:10:20.624 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.624 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.624 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:10:20.624 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.630075+0000", 2024-09-16T00:10:20.624 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000149841 2024-09-16T00:10:20.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.625 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.625 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:20.625 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.636470+0000", 2024-09-16T00:10:20.625 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063952289999999997 2024-09-16T00:10:20.625 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.626 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.626 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:20.626 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639098+0000", 2024-09-16T00:10:20.626 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026277980000000002 2024-09-16T00:10:20.626 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.627 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.627 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639154+0000", 2024-09-16T00:10:20.627 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6413999999999999e-05 2024-09-16T00:10:20.627 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.627 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.628 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.628 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639188+0000", 2024-09-16T00:10:20.628 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3859000000000002e-05 2024-09-16T00:10:20.628 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.628 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.628 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.629 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.629 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.629 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11217 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:20.629 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:20.629 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 39.978907941999999, 2024-09-16T00:10:20.630 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01231253, 2024-09-16T00:10:20.630 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.630 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.630 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.631 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.631 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.631 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.631 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11217 2024-09-16T00:10:20.631 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.632 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.632 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.632 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.632 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:20.633 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.633 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.633 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.633 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.633 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:20.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.634 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.634 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.634 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.634 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574282+0000", 2024-09-16T00:10:20.634 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8499999999999998e-06 2024-09-16T00:10:20.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.635 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.635 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575804+0000", 2024-09-16T00:10:20.635 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001521859 2024-09-16T00:10:20.635 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.635 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575826+0000", 2024-09-16T00:10:20.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2059999999999999e-05 2024-09-16T00:10:20.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575837+0000", 2024-09-16T00:10:20.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1313e-05 2024-09-16T00:10:20.636 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.636 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.636 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.636 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575920+0000", 2024-09-16T00:10:20.636 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2918999999999997e-05 2024-09-16T00:10:20.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576088+0000", 2024-09-16T00:10:20.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016803799999999999 2024-09-16T00:10:20.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.637 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.637 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:20.637 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576233+0000", 2024-09-16T00:10:20.637 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014508199999999999 2024-09-16T00:10:20.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:20.638 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.580500+0000", 2024-09-16T00:10:20.638 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004266644 2024-09-16T00:10:20.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.638 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.638 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:20.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586479+0000", 2024-09-16T00:10:20.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059791820000000004 2024-09-16T00:10:20.639 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.639 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.639 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.639 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586541+0000", 2024-09-16T00:10:20.639 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1463000000000004e-05 2024-09-16T00:10:20.640 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.640 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.640 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.640 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586592+0000", 2024-09-16T00:10:20.640 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1119999999999998e-05 2024-09-16T00:10:20.640 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.640 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.640 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.641 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.641 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.641 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11767 3.38 3:1fa50e65:::10000003117.00000000:head [write 0~4194304 [3@1169732]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:20.641 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:20.641 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.389503406999999, 2024-09-16T00:10:20.641 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011555678999999999, 2024-09-16T00:10:20.642 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.642 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.642 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.642 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.642 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.642 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.642 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11767 2024-09-16T00:10:20.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.643 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.643 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.643 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.643 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:20.643 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.643 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:20.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.644 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.644 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.644 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.644 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163686+0000", 2024-09-16T00:10:20.644 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3259999999999998e-06 2024-09-16T00:10:20.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.645 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165053+0000", 2024-09-16T00:10:20.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013665859999999999 2024-09-16T00:10:20.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.645 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165075+0000", 2024-09-16T00:10:20.645 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2206999999999999e-05 2024-09-16T00:10:20.645 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.645 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.645 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.645 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165084+0000", 2024-09-16T00:10:20.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0710000000000001e-06 2024-09-16T00:10:20.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.646 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.646 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165157+0000", 2024-09-16T00:10:20.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2619000000000004e-05 2024-09-16T00:10:20.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.646 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.646 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165249+0000", 2024-09-16T00:10:20.646 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2517999999999997e-05 2024-09-16T00:10:20.646 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.646 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.646 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:20.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165360+0000", 2024-09-16T00:10:20.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011134899999999999 2024-09-16T00:10:20.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:20.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.169320+0000", 2024-09-16T00:10:20.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039593040000000003 2024-09-16T00:10:20.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.647 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.647 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:20.647 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175135+0000", 2024-09-16T00:10:20.647 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005814891 2024-09-16T00:10:20.647 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175199+0000", 2024-09-16T00:10:20.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4474999999999995e-05 2024-09-16T00:10:20.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.648 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.648 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175239+0000", 2024-09-16T00:10:20.648 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0333000000000002e-05 2024-09-16T00:10:20.648 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.648 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.648 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.648 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.648 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.649 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11789 3.21 3:8453163e:::100000030b3.00000000:head [write 0~3801088 [4@1194504]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:20.649 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:10:20.649 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.301815310999999, 2024-09-16T00:10:20.649 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011097602, 2024-09-16T00:10:20.649 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.649 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.649 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.649 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.649 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.649 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.649 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11789 2024-09-16T00:10:20.649 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.649 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.650 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.650 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:10:20.650 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.650 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.650 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.650 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.650 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:10:20.651 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.651 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.651 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.651 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.651 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251373+0000", 2024-09-16T00:10:20.651 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.544e-06 2024-09-16T00:10:20.651 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.652 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.652 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252710+0000", 2024-09-16T00:10:20.652 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013362929999999999 2024-09-16T00:10:20.652 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.652 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.653 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252730+0000", 2024-09-16T00:10:20.653 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0579999999999999e-05 2024-09-16T00:10:20.653 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.653 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.653 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.653 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252741+0000", 2024-09-16T00:10:20.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0505e-05 2024-09-16T00:10:20.654 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.654 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.654 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.654 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252820+0000", 2024-09-16T00:10:20.654 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9098999999999996e-05 2024-09-16T00:10:20.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.655 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.655 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252931+0000", 2024-09-16T00:10:20.655 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011073600000000001 2024-09-16T00:10:20.655 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.655 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.656 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:20.656 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.253101+0000", 2024-09-16T00:10:20.656 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017038000000000001 2024-09-16T00:10:20.656 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.656 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.657 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:20.657 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.261813+0000", 2024-09-16T00:10:20.657 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087119469999999994 2024-09-16T00:10:20.657 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.657 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.657 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:20.657 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262338+0000", 2024-09-16T00:10:20.658 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00052493799999999997 2024-09-16T00:10:20.658 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.658 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.659 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262430+0000", 2024-09-16T00:10:20.659 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1626999999999995e-05 2024-09-16T00:10:20.659 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.659 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.659 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.659 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262470+0000", 2024-09-16T00:10:20.660 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9953000000000002e-05 2024-09-16T00:10:20.660 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.660 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.660 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.660 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.661 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.661 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11890 3.38 3:1e014616:::100000033e9.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:20.661 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:20.661 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.981850229000003, 2024-09-16T00:10:20.661 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011833791999999999, 2024-09-16T00:10:20.661 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.662 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.662 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.662 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.662 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.663 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.663 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11890 2024-09-16T00:10:20.663 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.663 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.663 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.664 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.664 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:20.664 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.664 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.664 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.665 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:20.665 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.665 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.665 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.665 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.665 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571340+0000", 2024-09-16T00:10:20.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.36e-06 2024-09-16T00:10:20.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572866+0000", 2024-09-16T00:10:20.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001525549 2024-09-16T00:10:20.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572884+0000", 2024-09-16T00:10:20.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7796999999999999e-05 2024-09-16T00:10:20.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572897+0000", 2024-09-16T00:10:20.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3567000000000001e-05 2024-09-16T00:10:20.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572970+0000", 2024-09-16T00:10:20.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2751999999999997e-05 2024-09-16T00:10:20.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.668 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573053+0000", 2024-09-16T00:10:20.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3230000000000001e-05 2024-09-16T00:10:20.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.668 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:20.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573168+0000", 2024-09-16T00:10:20.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114536 2024-09-16T00:10:20.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.668 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:20.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.577923+0000", 2024-09-16T00:10:20.669 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047548720000000003 2024-09-16T00:10:20.669 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.669 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:20.669 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583081+0000", 2024-09-16T00:10:20.669 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051587630000000002 2024-09-16T00:10:20.669 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.669 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.669 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583137+0000", 2024-09-16T00:10:20.669 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5259000000000001e-05 2024-09-16T00:10:20.669 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.669 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.669 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583171+0000", 2024-09-16T00:10:20.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4106999999999997e-05 2024-09-16T00:10:20.670 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.670 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.670 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.670 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:12037 3.9 e87/20 3:928db521:::1000000326d.00000000:head v 87'199, mlcod=87'197)", 2024-09-16T00:10:20.670 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:20.670 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.550660278000002, 2024-09-16T00:10:20.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012236153, 2024-09-16T00:10:20.670 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.670 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.670 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.671 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:20.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:20.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002529+0000", 2024-09-16T00:10:20.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4889999999999998e-06 2024-09-16T00:10:20.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007064+0000", 2024-09-16T00:10:20.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045346220000000003 2024-09-16T00:10:20.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007093+0000", 2024-09-16T00:10:20.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9009e-05 2024-09-16T00:10:20.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007109+0000", 2024-09-16T00:10:20.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5846e-05 2024-09-16T00:10:20.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007196+0000", 2024-09-16T00:10:20.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7507000000000001e-05 2024-09-16T00:10:20.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.674 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.674 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007227+0000", 2024-09-16T00:10:20.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0916e-05 2024-09-16T00:10:20.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.674 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.674 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014658+0000", 2024-09-16T00:10:20.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074305090000000001 2024-09-16T00:10:20.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.674 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.674 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014763+0000", 2024-09-16T00:10:20.675 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010525499999999999 2024-09-16T00:10:20.675 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.675 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.675 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.675 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.675 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.675 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12398 3.2e 3:77a11d08:::10000003432.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:20.675 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:20.675 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 35.311981494999998, 2024-09-16T00:10:20.675 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012092999, 2024-09-16T00:10:20.675 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.675 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.675 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.675 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.676 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.676 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.676 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12398 2024-09-16T00:10:20.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.676 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.676 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.676 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:20.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.676 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.676 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:20.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241209+0000", 2024-09-16T00:10:20.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2710000000000001e-06 2024-09-16T00:10:20.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243840+0000", 2024-09-16T00:10:20.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026307919999999999 2024-09-16T00:10:20.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243872+0000", 2024-09-16T00:10:20.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2699e-05 2024-09-16T00:10:20.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243887+0000", 2024-09-16T00:10:20.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4348000000000001e-05 2024-09-16T00:10:20.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243972+0000", 2024-09-16T00:10:20.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5358999999999994e-05 2024-09-16T00:10:20.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244132+0000", 2024-09-16T00:10:20.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015972499999999999 2024-09-16T00:10:20.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:20.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253153+0000", 2024-09-16T00:10:20.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0090210280000000004 2024-09-16T00:10:20.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253213+0000", 2024-09-16T00:10:20.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0300999999999997e-05 2024-09-16T00:10:20.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253299+0000", 2024-09-16T00:10:20.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5476000000000006e-05 2024-09-16T00:10:20.680 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.680 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.680 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.680 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12400 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304 [2@815045]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:20.680 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:10:20.680 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 35.308943761000002, 2024-09-16T00:10:20.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011140066000000001, 2024-09-16T00:10:20.680 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.680 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.680 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.681 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.681 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.681 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.681 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12400 2024-09-16T00:10:20.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.681 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.681 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.681 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:10:20.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.681 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.681 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:10:20.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244245+0000", 2024-09-16T00:10:20.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.248e-06 2024-09-16T00:10:20.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246467+0000", 2024-09-16T00:10:20.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022217949999999999 2024-09-16T00:10:20.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246496+0000", 2024-09-16T00:10:20.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9484000000000001e-05 2024-09-16T00:10:20.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246509+0000", 2024-09-16T00:10:20.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3390000000000001e-05 2024-09-16T00:10:20.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246586+0000", 2024-09-16T00:10:20.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6692000000000004e-05 2024-09-16T00:10:20.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246680+0000", 2024-09-16T00:10:20.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3468999999999996e-05 2024-09-16T00:10:20.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:20.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246850+0000", 2024-09-16T00:10:20.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170206 2024-09-16T00:10:20.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:20.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253478+0000", 2024-09-16T00:10:20.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066287530000000002 2024-09-16T00:10:20.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:20.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255318+0000", 2024-09-16T00:10:20.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001839292 2024-09-16T00:10:20.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255354+0000", 2024-09-16T00:10:20.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5964999999999999e-05 2024-09-16T00:10:20.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.686 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255384+0000", 2024-09-16T00:10:20.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9771999999999999e-05 2024-09-16T00:10:20.686 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.686 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.686 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.686 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13317 3.38 3:1cda8244:::100000035dc.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:20.686 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:10:20.686 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.634589235, 2024-09-16T00:10:20.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011377063, 2024-09-16T00:10:20.686 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.686 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.686 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.686 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.687 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.687 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.687 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13317 2024-09-16T00:10:20.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.687 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:10:20.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:10:20.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918602+0000", 2024-09-16T00:10:20.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8160000000000004e-06 2024-09-16T00:10:20.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920364+0000", 2024-09-16T00:10:20.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017623630000000001 2024-09-16T00:10:20.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920391+0000", 2024-09-16T00:10:20.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7248000000000001e-05 2024-09-16T00:10:20.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920405+0000", 2024-09-16T00:10:20.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3969e-05 2024-09-16T00:10:20.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920486+0000", 2024-09-16T00:10:20.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0737999999999998e-05 2024-09-16T00:10:20.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920594+0000", 2024-09-16T00:10:20.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108269 2024-09-16T00:10:20.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:20.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920730+0000", 2024-09-16T00:10:20.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000135229 2024-09-16T00:10:20.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:20.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.928847+0000", 2024-09-16T00:10:20.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081178299999999995 2024-09-16T00:10:20.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:20.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929862+0000", 2024-09-16T00:10:20.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010147540000000001 2024-09-16T00:10:20.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929923+0000", 2024-09-16T00:10:20.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0452999999999999e-05 2024-09-16T00:10:20.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929975+0000", 2024-09-16T00:10:20.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2394e-05 2024-09-16T00:10:20.691 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.691 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.692 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.692 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:13326 3.9 e88/20 3:92f796cf:::1000000350f.00000000:head v 88'214, mlcod=88'212)", 2024-09-16T00:10:20.692 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:10:20.692 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.592068443999999, 2024-09-16T00:10:20.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01100597, 2024-09-16T00:10:20.692 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.692 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.692 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.692 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.692 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.692 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:10:20.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:10:20.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961121+0000", 2024-09-16T00:10:20.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.815e-06 2024-09-16T00:10:20.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965747+0000", 2024-09-16T00:10:20.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0046260570000000003 2024-09-16T00:10:20.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965780+0000", 2024-09-16T00:10:20.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2851000000000001e-05 2024-09-16T00:10:20.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965788+0000", 2024-09-16T00:10:20.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0180000000000001e-06 2024-09-16T00:10:20.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965859+0000", 2024-09-16T00:10:20.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1318999999999999e-05 2024-09-16T00:10:20.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965886+0000", 2024-09-16T00:10:20.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7509999999999999e-05 2024-09-16T00:10:20.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.972066+0000", 2024-09-16T00:10:20.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061800759999999996 2024-09-16T00:10:20.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.972125+0000", 2024-09-16T00:10:20.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8323999999999999e-05 2024-09-16T00:10:20.696 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.696 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.696 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.696 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13942 3.22 3:44ed4521:::100000036d5.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:20.696 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:10:20.696 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.630703132000001, 2024-09-16T00:10:20.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011341061, 2024-09-16T00:10:20.696 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.697 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.697 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.697 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.697 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.697 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.697 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13942 2024-09-16T00:10:20.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.697 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:10:20.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:10:20.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922487+0000", 2024-09-16T00:10:20.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0180000000000002e-06 2024-09-16T00:10:20.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.698 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.698 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923720+0000", 2024-09-16T00:10:20.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012333470000000001 2024-09-16T00:10:20.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923749+0000", 2024-09-16T00:10:20.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8748999999999999e-05 2024-09-16T00:10:20.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923770+0000", 2024-09-16T00:10:20.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1189999999999999e-05 2024-09-16T00:10:20.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923868+0000", 2024-09-16T00:10:20.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7675000000000002e-05 2024-09-16T00:10:20.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.924047+0000", 2024-09-16T00:10:20.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017935100000000001 2024-09-16T00:10:20.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:10:20.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.924223+0000", 2024-09-16T00:10:20.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000175661 2024-09-16T00:10:20.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:20.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.931544+0000", 2024-09-16T00:10:20.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073204589999999996 2024-09-16T00:10:20.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:20.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933692+0000", 2024-09-16T00:10:20.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021488760000000001 2024-09-16T00:10:20.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933774+0000", 2024-09-16T00:10:20.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1855999999999997e-05 2024-09-16T00:10:20.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933825+0000", 2024-09-16T00:10:20.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0878999999999999e-05 2024-09-16T00:10:20.702 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.702 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.702 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.702 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14867 3.1f 3:faedf583:::1000000393b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:20.702 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:10:20.702 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.806213396, 2024-09-16T00:10:20.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011128677999999999, 2024-09-16T00:10:20.703 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.703 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.703 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:20.703 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.703 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.703 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.703 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14867 2024-09-16T00:10:20.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.703 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:10:20.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:10:20.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.704 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746976+0000", 2024-09-16T00:10:20.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1169999999999998e-06 2024-09-16T00:10:20.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.704 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748348+0000", 2024-09-16T00:10:20.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013721860000000001 2024-09-16T00:10:20.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748371+0000", 2024-09-16T00:10:20.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2938999999999998e-05 2024-09-16T00:10:20.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748381+0000", 2024-09-16T00:10:20.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5300000000000002e-06 2024-09-16T00:10:20.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748458+0000", 2024-09-16T00:10:20.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7389000000000001e-05 2024-09-16T00:10:20.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748626+0000", 2024-09-16T00:10:20.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016757499999999999 2024-09-16T00:10:20.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:20.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748813+0000", 2024-09-16T00:10:20.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018743399999999999 2024-09-16T00:10:20.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:20.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.753998+0000", 2024-09-16T00:10:20.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051851140000000002 2024-09-16T00:10:20.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:20.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.757990+0000", 2024-09-16T00:10:20.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039914169999999997 2024-09-16T00:10:20.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:20.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.758054+0000", 2024-09-16T00:10:20.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4513999999999999e-05 2024-09-16T00:10:20.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.758103+0000", 2024-09-16T00:10:20.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8463e-05 2024-09-16T00:10:20.708 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.708 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.708 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.708 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15003 3.21 3:8673bb5f:::100000030ed.00000000:head [read 2097152~2097152 [2@1395527]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:20.708 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:20.708 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.384791401000001, 2024-09-16T00:10:20.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058433321000000003, 2024-09-16T00:10:20.709 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.709 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.709 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:10:20.709 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.709 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.709 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.709 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15003 2024-09-16T00:10:20.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.709 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:20.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:20.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168397+0000", 2024-09-16T00:10:20.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3900000000000004e-07 2024-09-16T00:10:20.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.710 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168401+0000", 2024-09-16T00:10:20.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4050000000000002e-06 2024-09-16T00:10:20.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168411+0000", 2024-09-16T00:10:20.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6539999999999994e-06 2024-09-16T00:10:20.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168417+0000", 2024-09-16T00:10:20.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6719999999999998e-06 2024-09-16T00:10:20.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168467+0000", 2024-09-16T00:10:20.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9815999999999999e-05 2024-09-16T00:10:20.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168535+0000", 2024-09-16T00:10:20.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7669999999999994e-05 2024-09-16T00:10:20.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.226829+0000", 2024-09-16T00:10:20.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058294264999999998 2024-09-16T00:10:20.713 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.713 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.713 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.713 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15005 3.1f 3:f8460f10:::10000003370.00000000:head [read 1048576~1048576 [2@4194304]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:20.713 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:20.713 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.383110688999999, 2024-09-16T00:10:20.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.055844088, 2024-09-16T00:10:20.713 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:20.713 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:20.713 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:10:20.713 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:20.713 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:20.714 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:20.714 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15005 2024-09-16T00:10:20.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.714 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:20.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:20.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:20.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:20.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:20.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:20.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:20.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170078+0000", 2024-09-16T00:10:20.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.235e-06 2024-09-16T00:10:20.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.715 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:20.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170084+0000", 2024-09-16T00:10:20.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9739999999999999e-06 2024-09-16T00:10:20.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:20.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225526+0000", 2024-09-16T00:10:20.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.055441777999999997 2024-09-16T00:10:20.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:20.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225542+0000", 2024-09-16T00:10:20.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6606999999999999e-05 2024-09-16T00:10:20.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:20.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225668+0000", 2024-09-16T00:10:20.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012631 2024-09-16T00:10:20.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:20.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225744+0000", 2024-09-16T00:10:20.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5794e-05 2024-09-16T00:10:20.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:20.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:20.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:20.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225921+0000", 2024-09-16T00:10:20.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017639000000000001 2024-09-16T00:10:20.717 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.717 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.718 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.718 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:20.718 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:20.718 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:20.719 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:10:20.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:20.676+0000 7f6168fc6640 1 -- 172.21.15.139:0/2882474046 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f616406ffc0 con 0x7f61640ce810 2024-09-16T00:10:20.720 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:20.717+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:20.819 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:20.818+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:20.869 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:20.870 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:20.870 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:20.870 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:20.879 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:10:20.920 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:20.919+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:21.022 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:21.020+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:21.055 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:21.055 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:21.055 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:21.055 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:21.055 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:21.064 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:10:21.122 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:21.121+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:21.174 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:21.174 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:21.174 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:21.175 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:21.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.175 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:90 2.1b 2:ddb4e4d7:::200.00000002:head [write 2593031~1404 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:10:21.175 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:21.175 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 77.522897689000004, 2024-09-16T00:10:21.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011882886, 2024-09-16T00:10:21.175 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.175 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.175 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.175 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:21.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:21.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649119+0000", 2024-09-16T00:10:21.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:10:21.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649125+0000", 2024-09-16T00:10:21.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1650000000000003e-06 2024-09-16T00:10:21.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649134+0000", 2024-09-16T00:10:21.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5669999999999995e-06 2024-09-16T00:10:21.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649141+0000", 2024-09-16T00:10:21.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9040000000000003e-06 2024-09-16T00:10:21.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655796+0000", 2024-09-16T00:10:21.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066553649999999999 2024-09-16T00:10:21.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655868+0000", 2024-09-16T00:10:21.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2219999999999996e-05 2024-09-16T00:10:21.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:21.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655986+0000", 2024-09-16T00:10:21.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117457 2024-09-16T00:10:21.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.179 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657794+0000", 2024-09-16T00:10:21.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001807894 2024-09-16T00:10:21.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.179 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:21.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660890+0000", 2024-09-16T00:10:21.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030958449999999998 2024-09-16T00:10:21.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.179 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660966+0000", 2024-09-16T00:10:21.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6382000000000002e-05 2024-09-16T00:10:21.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.180 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.661001+0000", 2024-09-16T00:10:21.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5349000000000003e-05 2024-09-16T00:10:21.180 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.180 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.180 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.180 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10851 3.29 3:9786b8f5:::10000003100.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:21.180 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:21.180 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 42.279233644999998, 2024-09-16T00:10:21.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012109089999999999, 2024-09-16T00:10:21.181 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.181 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.181 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.181 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:21.181 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:21.181 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.181 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10851 2024-09-16T00:10:21.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.181 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:21.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:21.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892786+0000", 2024-09-16T00:10:21.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4819999999999999e-06 2024-09-16T00:10:21.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894302+0000", 2024-09-16T00:10:21.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015156099999999999 2024-09-16T00:10:21.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894319+0000", 2024-09-16T00:10:21.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7098000000000001e-05 2024-09-16T00:10:21.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894329+0000", 2024-09-16T00:10:21.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9180000000000006e-06 2024-09-16T00:10:21.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894394+0000", 2024-09-16T00:10:21.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5797999999999996e-05 2024-09-16T00:10:21.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894532+0000", 2024-09-16T00:10:21.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000138128 2024-09-16T00:10:21.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:21.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894633+0000", 2024-09-16T00:10:21.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010037100000000001 2024-09-16T00:10:21.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:21.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.899212+0000", 2024-09-16T00:10:21.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045789369999999999 2024-09-16T00:10:21.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904771+0000", 2024-09-16T00:10:21.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055591169999999997 2024-09-16T00:10:21.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904838+0000", 2024-09-16T00:10:21.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7124000000000003e-05 2024-09-16T00:10:21.185 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.185 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.186 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.186 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904892+0000", 2024-09-16T00:10:21.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3507000000000001e-05 2024-09-16T00:10:21.186 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.186 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.186 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.186 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.186 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.186 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10957 3.24 3:2748b9d7:::1000000313b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:21.186 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:21.186 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.657329292, 2024-09-16T00:10:21.186 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012163525, 2024-09-16T00:10:21.186 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.186 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.187 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.187 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:21.187 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:21.187 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.187 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10957 2024-09-16T00:10:21.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.187 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:21.187 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.187 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.187 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.187 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.187 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:21.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514690+0000", 2024-09-16T00:10:21.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.382e-06 2024-09-16T00:10:21.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.188 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.188 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516204+0000", 2024-09-16T00:10:21.188 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001513831 2024-09-16T00:10:21.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.188 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516232+0000", 2024-09-16T00:10:21.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7657999999999998e-05 2024-09-16T00:10:21.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516246+0000", 2024-09-16T00:10:21.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3953e-05 2024-09-16T00:10:21.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.189 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.189 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.189 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516374+0000", 2024-09-16T00:10:21.189 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012882 2024-09-16T00:10:21.189 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516588+0000", 2024-09-16T00:10:21.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000213439 2024-09-16T00:10:21.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:21.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516748+0000", 2024-09-16T00:10:21.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015971900000000001 2024-09-16T00:10:21.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.190 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.190 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:21.190 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.524102+0000", 2024-09-16T00:10:21.190 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073542499999999997 2024-09-16T00:10:21.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526714+0000", 2024-09-16T00:10:21.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026122910000000001 2024-09-16T00:10:21.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526791+0000", 2024-09-16T00:10:21.191 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6997999999999997e-05 2024-09-16T00:10:21.191 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.191 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.191 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.191 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526850+0000", 2024-09-16T00:10:21.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9184000000000001e-05 2024-09-16T00:10:21.192 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.192 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.192 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.192 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.192 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11098 3.15 e86/20 3:aa8f498f:::100000030ba.00000000:head v 86'170, mlcod=86'168)", 2024-09-16T00:10:21.192 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:21.192 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.150546126000002, 2024-09-16T00:10:21.192 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013832641999999999, 2024-09-16T00:10:21.192 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.192 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.192 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.193 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:21.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.193 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.193 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.193 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:21.193 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021472+0000", 2024-09-16T00:10:21.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9489999999999999e-06 2024-09-16T00:10:21.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027628+0000", 2024-09-16T00:10:21.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061563859999999998 2024-09-16T00:10:21.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.194 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.194 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.194 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027645+0000", 2024-09-16T00:10:21.194 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7108e-05 2024-09-16T00:10:21.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027655+0000", 2024-09-16T00:10:21.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7999999999999993e-06 2024-09-16T00:10:21.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027712+0000", 2024-09-16T00:10:21.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6826000000000001e-05 2024-09-16T00:10:21.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.195 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.195 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.195 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027738+0000", 2024-09-16T00:10:21.195 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5601000000000001e-05 2024-09-16T00:10:21.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.035237+0000", 2024-09-16T00:10:21.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074998699999999996 2024-09-16T00:10:21.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.196 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.196 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.196 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.035303+0000", 2024-09-16T00:10:21.196 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5102000000000001e-05 2024-09-16T00:10:21.196 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.196 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.196 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.197 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.197 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11099 3.b 3:d2f5b389:::100000030e7.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:21.197 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:21.197 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.146501311000002, 2024-09-16T00:10:21.197 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012707456000000001, 2024-09-16T00:10:21.197 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.197 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.197 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.197 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:21.197 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:21.197 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.197 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11099 2024-09-16T00:10:21.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.198 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:21.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.198 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:21.198 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.198 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.198 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025517+0000", 2024-09-16T00:10:21.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:10:21.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026955+0000", 2024-09-16T00:10:21.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001437219 2024-09-16T00:10:21.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.199 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.199 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026982+0000", 2024-09-16T00:10:21.199 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7523999999999999e-05 2024-09-16T00:10:21.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.199 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026996+0000", 2024-09-16T00:10:21.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4379000000000001e-05 2024-09-16T00:10:21.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027076+0000", 2024-09-16T00:10:21.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9933999999999997e-05 2024-09-16T00:10:21.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.200 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.200 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.200 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027185+0000", 2024-09-16T00:10:21.200 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010875899999999999 2024-09-16T00:10:21.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:21.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027380+0000", 2024-09-16T00:10:21.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019477700000000001 2024-09-16T00:10:21.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.201 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038072+0000", 2024-09-16T00:10:21.201 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010692234 2024-09-16T00:10:21.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.201 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.201 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:21.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038115+0000", 2024-09-16T00:10:21.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3291999999999998e-05 2024-09-16T00:10:21.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038172+0000", 2024-09-16T00:10:21.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6981000000000002e-05 2024-09-16T00:10:21.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.202 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.202 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.202 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038222+0000", 2024-09-16T00:10:21.202 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9737000000000002e-05 2024-09-16T00:10:21.202 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.202 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.203 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.203 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.203 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11109 3.27 3:e5179c63:::10000003176.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:21.203 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:21.203 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.101037028, 2024-09-16T00:10:21.203 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012250990999999999, 2024-09-16T00:10:21.203 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.203 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.203 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.203 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:21.203 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:21.203 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.203 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11109 2024-09-16T00:10:21.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.204 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:21.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.204 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:21.204 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.204 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.204 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070981+0000", 2024-09-16T00:10:21.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.289e-06 2024-09-16T00:10:21.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.205 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072616+0000", 2024-09-16T00:10:21.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016347569999999999 2024-09-16T00:10:21.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.205 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.205 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072638+0000", 2024-09-16T00:10:21.205 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1512999999999999e-05 2024-09-16T00:10:21.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.205 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072649+0000", 2024-09-16T00:10:21.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1626e-05 2024-09-16T00:10:21.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072775+0000", 2024-09-16T00:10:21.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012553900000000001 2024-09-16T00:10:21.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.206 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.206 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.206 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072856+0000", 2024-09-16T00:10:21.206 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0882000000000003e-05 2024-09-16T00:10:21.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:21.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072990+0000", 2024-09-16T00:10:21.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013483599999999999 2024-09-16T00:10:21.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:21.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.078354+0000", 2024-09-16T00:10:21.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053639500000000001 2024-09-16T00:10:21.207 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.207 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.207 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.207 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083150+0000", 2024-09-16T00:10:21.207 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004795517 2024-09-16T00:10:21.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083186+0000", 2024-09-16T00:10:21.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6019000000000002e-05 2024-09-16T00:10:21.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.208 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.208 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.208 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083230+0000", 2024-09-16T00:10:21.208 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4063000000000001e-05 2024-09-16T00:10:21.208 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.208 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.208 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.209 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.209 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11234 3.b 3:d2f5b389:::100000030e7.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:21.209 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:21.209 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.548421259999998, 2024-09-16T00:10:21.209 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.015217661, 2024-09-16T00:10:21.209 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.209 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.209 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.209 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:21.209 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:21.210 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.210 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11234 2024-09-16T00:10:21.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.210 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:21.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.210 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.210 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.210 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.210 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:21.210 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.211 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.211 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.211 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623598+0000", 2024-09-16T00:10:21.211 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1580000000000001e-06 2024-09-16T00:10:21.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.212 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.212 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626844+0000", 2024-09-16T00:10:21.212 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032459670000000002 2024-09-16T00:10:21.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.212 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.212 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.212 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626868+0000", 2024-09-16T00:10:21.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3584e-05 2024-09-16T00:10:21.213 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.213 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.213 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.213 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626879+0000", 2024-09-16T00:10:21.213 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1545999999999999e-05 2024-09-16T00:10:21.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.214 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626950+0000", 2024-09-16T00:10:21.214 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0784999999999994e-05 2024-09-16T00:10:21.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.214 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.214 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.215 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.627051+0000", 2024-09-16T00:10:21.215 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010155899999999999 2024-09-16T00:10:21.215 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.215 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.215 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:21.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.627178+0000", 2024-09-16T00:10:21.216 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012614899999999999 2024-09-16T00:10:21.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.216 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.216 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:21.216 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.635220+0000", 2024-09-16T00:10:21.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0080421490000000002 2024-09-16T00:10:21.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.217 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.217 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.217 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638699+0000", 2024-09-16T00:10:21.217 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034797500000000002 2024-09-16T00:10:21.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.218 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638760+0000", 2024-09-16T00:10:21.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0319000000000003e-05 2024-09-16T00:10:21.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.218 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.218 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.218 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638812+0000", 2024-09-16T00:10:21.218 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:10:21.218 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.218 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.218 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.219 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.219 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11777 3.b 3:d342b4ff:::10000003113.00000000:head [write 0~4194304 [2@1171070]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:21.219 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:21.219 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.980781852, 2024-09-16T00:10:21.219 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013079176, 2024-09-16T00:10:21.219 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.219 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.219 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.219 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:21.219 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:21.219 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.219 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11777 2024-09-16T00:10:21.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.220 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:21.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.220 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:21.220 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.220 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.220 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191237+0000", 2024-09-16T00:10:21.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3369999999999998e-06 2024-09-16T00:10:21.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192825+0000", 2024-09-16T00:10:21.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015887189999999999 2024-09-16T00:10:21.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.221 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.221 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.221 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192846+0000", 2024-09-16T00:10:21.221 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0670999999999999e-05 2024-09-16T00:10:21.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192858+0000", 2024-09-16T00:10:21.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.223e-05 2024-09-16T00:10:21.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192957+0000", 2024-09-16T00:10:21.222 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8416000000000001e-05 2024-09-16T00:10:21.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.222 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.222 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.222 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.193031+0000", 2024-09-16T00:10:21.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4381999999999994e-05 2024-09-16T00:10:21.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:21.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.193164+0000", 2024-09-16T00:10:21.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013349100000000001 2024-09-16T00:10:21.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:21.223 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204144+0000", 2024-09-16T00:10:21.223 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010979917 2024-09-16T00:10:21.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.223 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.223 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204204+0000", 2024-09-16T00:10:21.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0025999999999998e-05 2024-09-16T00:10:21.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204265+0000", 2024-09-16T00:10:21.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0645e-05 2024-09-16T00:10:21.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.224 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.224 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.224 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204313+0000", 2024-09-16T00:10:21.224 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8341999999999999e-05 2024-09-16T00:10:21.224 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.224 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.225 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.225 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.225 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11981 3.c e87/20 3:33faa5a0:::10000003215.00000000:head v 87'186, mlcod=87'184)", 2024-09-16T00:10:21.225 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:21.225 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.340547029, 2024-09-16T00:10:21.225 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011828697000000001, 2024-09-16T00:10:21.226 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.226 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.226 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.226 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.226 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.226 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.226 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:21.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.227 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.227 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.227 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:21.227 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.228 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.228 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.228 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831471+0000", 2024-09-16T00:10:21.228 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6220000000000001e-06 2024-09-16T00:10:21.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.229 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.229 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836584+0000", 2024-09-16T00:10:21.229 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051133510000000004 2024-09-16T00:10:21.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.229 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.230 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836606+0000", 2024-09-16T00:10:21.230 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2079e-05 2024-09-16T00:10:21.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.230 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.230 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.231 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836624+0000", 2024-09-16T00:10:21.231 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8162e-05 2024-09-16T00:10:21.231 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.231 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.231 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836704+0000", 2024-09-16T00:10:21.232 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9867e-05 2024-09-16T00:10:21.232 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.232 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.232 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.232 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836730+0000", 2024-09-16T00:10:21.233 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6285999999999999e-05 2024-09-16T00:10:21.233 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.233 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.233 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.233 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.843247+0000", 2024-09-16T00:10:21.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0065165650000000002 2024-09-16T00:10:21.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.234 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.234 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.234 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.843298+0000", 2024-09-16T00:10:21.234 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0764999999999999e-05 2024-09-16T00:10:21.234 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.235 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.235 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.235 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.235 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13365 3.2 3:4298088d:::100000035be.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:21.235 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:21.236 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.152845050000003, 2024-09-16T00:10:21.236 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014036442999999999, 2024-09-16T00:10:21.236 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.236 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.236 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.236 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:21.236 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:21.237 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.237 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13365 2024-09-16T00:10:21.237 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.237 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.237 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.237 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.237 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:21.237 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.238 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:21.238 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.238 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.238 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.238 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019174+0000", 2024-09-16T00:10:21.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1250000000000001e-06 2024-09-16T00:10:21.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.239 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.239 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.239 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020613+0000", 2024-09-16T00:10:21.239 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014391930000000001 2024-09-16T00:10:21.239 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.240 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020638+0000", 2024-09-16T00:10:21.240 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4657000000000001e-05 2024-09-16T00:10:21.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.240 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.240 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020651+0000", 2024-09-16T00:10:21.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3451e-05 2024-09-16T00:10:21.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.241 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.241 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.241 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020744+0000", 2024-09-16T00:10:21.241 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3010000000000006e-05 2024-09-16T00:10:21.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.242 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.242 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020912+0000", 2024-09-16T00:10:21.242 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016763800000000001 2024-09-16T00:10:21.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.242 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.242 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:21.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.021043+0000", 2024-09-16T00:10:21.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013099800000000001 2024-09-16T00:10:21.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:21.243 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026194+0000", 2024-09-16T00:10:21.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051510879999999998 2024-09-16T00:10:21.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.243 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033132+0000", 2024-09-16T00:10:21.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069377279999999998 2024-09-16T00:10:21.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033169+0000", 2024-09-16T00:10:21.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6897000000000003e-05 2024-09-16T00:10:21.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033207+0000", 2024-09-16T00:10:21.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658000000000002e-05 2024-09-16T00:10:21.244 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.245 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.245 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.245 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13368 3.27 3:e75b72a1:::100000032e8.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:21.245 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:21.245 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.145479166999998, 2024-09-16T00:10:21.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013615175, 2024-09-16T00:10:21.245 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.245 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.245 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.245 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:21.245 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:21.246 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.246 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13368 2024-09-16T00:10:21.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.246 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:21.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:21.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026539+0000", 2024-09-16T00:10:21.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.798e-06 2024-09-16T00:10:21.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027825+0000", 2024-09-16T00:10:21.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012861839999999999 2024-09-16T00:10:21.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027850+0000", 2024-09-16T00:10:21.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5048000000000001e-05 2024-09-16T00:10:21.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027865+0000", 2024-09-16T00:10:21.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4939e-05 2024-09-16T00:10:21.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027944+0000", 2024-09-16T00:10:21.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8881999999999995e-05 2024-09-16T00:10:21.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.028043+0000", 2024-09-16T00:10:21.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9135000000000005e-05 2024-09-16T00:10:21.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:21.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.028158+0000", 2024-09-16T00:10:21.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011527999999999999 2024-09-16T00:10:21.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.039292+0000", 2024-09-16T00:10:21.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011133568999999999 2024-09-16T00:10:21.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:21.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.039948+0000", 2024-09-16T00:10:21.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00065625700000000004 2024-09-16T00:10:21.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.040061+0000", 2024-09-16T00:10:21.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011284400000000001 2024-09-16T00:10:21.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.040152+0000", 2024-09-16T00:10:21.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1238999999999998e-05 2024-09-16T00:10:21.255 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.255 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.255 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.256 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:13366 3.15 e88/20 3:aa8f498f:::100000030ba.00000000:head v 88'196, mlcod=88'194)", 2024-09-16T00:10:21.256 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:21.256 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.142054092000002, 2024-09-16T00:10:21.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012822817, 2024-09-16T00:10:21.256 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.256 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.256 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.256 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:21.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:21.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029963+0000", 2024-09-16T00:10:21.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4270000000000001e-06 2024-09-16T00:10:21.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034078+0000", 2024-09-16T00:10:21.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041148360000000002 2024-09-16T00:10:21.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034094+0000", 2024-09-16T00:10:21.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.535e-05 2024-09-16T00:10:21.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034103+0000", 2024-09-16T00:10:21.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:10:21.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.036791+0000", 2024-09-16T00:10:21.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026878340000000001 2024-09-16T00:10:21.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.036818+0000", 2024-09-16T00:10:21.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6885000000000001e-05 2024-09-16T00:10:21.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.042713+0000", 2024-09-16T00:10:21.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058950399999999998 2024-09-16T00:10:21.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.042785+0000", 2024-09-16T00:10:21.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1927000000000004e-05 2024-09-16T00:10:21.261 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.261 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.261 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.262 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13405 3.27 3:e708eda4:::100000034c4.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:21.262 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:21.262 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 33.015852379000002, 2024-09-16T00:10:21.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012003333, 2024-09-16T00:10:21.262 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.262 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.263 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.263 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:21.263 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:21.263 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.263 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13405 2024-09-16T00:10:21.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.263 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:21.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:21.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156166+0000", 2024-09-16T00:10:21.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6230000000000001e-06 2024-09-16T00:10:21.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157868+0000", 2024-09-16T00:10:21.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017017 2024-09-16T00:10:21.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157891+0000", 2024-09-16T00:10:21.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2931000000000001e-05 2024-09-16T00:10:21.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157904+0000", 2024-09-16T00:10:21.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2763e-05 2024-09-16T00:10:21.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157988+0000", 2024-09-16T00:10:21.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3849000000000003e-05 2024-09-16T00:10:21.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.158083+0000", 2024-09-16T00:10:21.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5939000000000004e-05 2024-09-16T00:10:21.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.267 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:21.267 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.158194+0000", 2024-09-16T00:10:21.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110963 2024-09-16T00:10:21.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:21.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.164635+0000", 2024-09-16T00:10:21.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0064405570000000004 2024-09-16T00:10:21.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168088+0000", 2024-09-16T00:10:21.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034534639999999998 2024-09-16T00:10:21.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168140+0000", 2024-09-16T00:10:21.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1041000000000001e-05 2024-09-16T00:10:21.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168167+0000", 2024-09-16T00:10:21.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7503000000000001e-05 2024-09-16T00:10:21.269 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.269 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.269 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.269 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13685 3.b 3:d26c1580:::100000032e4.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:21.269 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:21.270 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.837165414000001, 2024-09-16T00:10:21.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014868124999999999, 2024-09-16T00:10:21.270 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.270 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.270 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.270 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:21.270 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:21.270 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.270 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13685 2024-09-16T00:10:21.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.270 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:21.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:21.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334854+0000", 2024-09-16T00:10:21.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0000000000000001e-06 2024-09-16T00:10:21.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337155+0000", 2024-09-16T00:10:21.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023016870000000002 2024-09-16T00:10:21.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337178+0000", 2024-09-16T00:10:21.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2898e-05 2024-09-16T00:10:21.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337190+0000", 2024-09-16T00:10:21.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1817999999999999e-05 2024-09-16T00:10:21.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337301+0000", 2024-09-16T00:10:21.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110951 2024-09-16T00:10:21.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337403+0000", 2024-09-16T00:10:21.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000102118 2024-09-16T00:10:21.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:21.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337545+0000", 2024-09-16T00:10:21.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014167199999999999 2024-09-16T00:10:21.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:21.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.346336+0000", 2024-09-16T00:10:21.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087911450000000002 2024-09-16T00:10:21.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349641+0000", 2024-09-16T00:10:21.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033047570000000002 2024-09-16T00:10:21.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349680+0000", 2024-09-16T00:10:21.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9054999999999998e-05 2024-09-16T00:10:21.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349719+0000", 2024-09-16T00:10:21.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9023999999999998e-05 2024-09-16T00:10:21.276 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.276 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.276 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.276 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14624 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:21.276 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:21.276 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 28.557406213, 2024-09-16T00:10:21.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012123376, 2024-09-16T00:10:21.276 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.276 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.276 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.276 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:21.277 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:21.277 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.277 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14624 2024-09-16T00:10:21.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.277 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:21.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:21.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614614+0000", 2024-09-16T00:10:21.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9500000000000003e-06 2024-09-16T00:10:21.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616480+0000", 2024-09-16T00:10:21.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018663169999999999 2024-09-16T00:10:21.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616506+0000", 2024-09-16T00:10:21.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6205999999999999e-05 2024-09-16T00:10:21.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.279 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.279 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.279 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616517+0000", 2024-09-16T00:10:21.279 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0399e-05 2024-09-16T00:10:21.279 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616595+0000", 2024-09-16T00:10:21.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8170999999999994e-05 2024-09-16T00:10:21.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616683+0000", 2024-09-16T00:10:21.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8578999999999993e-05 2024-09-16T00:10:21.280 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.280 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:21.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616825+0000", 2024-09-16T00:10:21.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014186299999999999 2024-09-16T00:10:21.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:21.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.624948+0000", 2024-09-16T00:10:21.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081230420000000005 2024-09-16T00:10:21.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.281 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.281 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626617+0000", 2024-09-16T00:10:21.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016681980000000001 2024-09-16T00:10:21.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.282 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.282 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626678+0000", 2024-09-16T00:10:21.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1362e-05 2024-09-16T00:10:21.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.282 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.282 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.282 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626733+0000", 2024-09-16T00:10:21.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5288999999999999e-05 2024-09-16T00:10:21.282 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.282 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.282 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.282 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.283 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14672 3.29 3:9641a85f:::1000000360e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:21.283 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:21.283 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 28.352949035999998, 2024-09-16T00:10:21.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012258025000000001, 2024-09-16T00:10:21.283 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.283 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.283 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.283 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:21.283 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:21.283 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.283 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14672 2024-09-16T00:10:21.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.283 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:21.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:21.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819070+0000", 2024-09-16T00:10:21.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0589999999999998e-06 2024-09-16T00:10:21.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821701+0000", 2024-09-16T00:10:21.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026309100000000002 2024-09-16T00:10:21.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821727+0000", 2024-09-16T00:10:21.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6372000000000001e-05 2024-09-16T00:10:21.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821738+0000", 2024-09-16T00:10:21.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0759e-05 2024-09-16T00:10:21.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821821+0000", 2024-09-16T00:10:21.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2854000000000003e-05 2024-09-16T00:10:21.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821920+0000", 2024-09-16T00:10:21.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8900999999999994e-05 2024-09-16T00:10:21.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:21.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.822084+0000", 2024-09-16T00:10:21.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016432799999999999 2024-09-16T00:10:21.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.287 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:21.287 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.829148+0000", 2024-09-16T00:10:21.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0070641460000000003 2024-09-16T00:10:21.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.288 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831242+0000", 2024-09-16T00:10:21.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002093293 2024-09-16T00:10:21.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.288 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831280+0000", 2024-09-16T00:10:21.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8707000000000001e-05 2024-09-16T00:10:21.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831325+0000", 2024-09-16T00:10:21.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4696e-05 2024-09-16T00:10:21.289 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.289 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.289 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.289 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14829 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304 [2@2556691]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:21.289 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:21.289 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.600429618, 2024-09-16T00:10:21.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014399683, 2024-09-16T00:10:21.289 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.289 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.290 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.290 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:21.290 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:21.290 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.290 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14829 2024-09-16T00:10:21.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.290 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:21.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:21.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571590+0000", 2024-09-16T00:10:21.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:10:21.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573133+0000", 2024-09-16T00:10:21.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015422109999999999 2024-09-16T00:10:21.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573161+0000", 2024-09-16T00:10:21.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8045e-05 2024-09-16T00:10:21.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573174+0000", 2024-09-16T00:10:21.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3193999999999999e-05 2024-09-16T00:10:21.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573365+0000", 2024-09-16T00:10:21.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019096400000000001 2024-09-16T00:10:21.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573467+0000", 2024-09-16T00:10:21.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101789 2024-09-16T00:10:21.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.293 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:21.293 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573610+0000", 2024-09-16T00:10:21.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143404 2024-09-16T00:10:21.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.583614+0000", 2024-09-16T00:10:21.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010004422000000001 2024-09-16T00:10:21.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:21.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585556+0000", 2024-09-16T00:10:21.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019414009999999999 2024-09-16T00:10:21.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585939+0000", 2024-09-16T00:10:21.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038340999999999999 2024-09-16T00:10:21.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585986+0000", 2024-09-16T00:10:21.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6924e-05 2024-09-16T00:10:21.295 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.295 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.295 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.296 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14914 3.b 3:d23f6103:::10000003277.00000000:head [write 0~4194304 [3@58131]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:21.296 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:21.296 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 27.289225847000001, 2024-09-16T00:10:21.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013007654, 2024-09-16T00:10:21.296 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.296 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.296 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.296 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:21.296 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:21.296 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.297 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14914 2024-09-16T00:10:21.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.297 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:21.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:21.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882793+0000", 2024-09-16T00:10:21.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.65e-06 2024-09-16T00:10:21.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.300 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885193+0000", 2024-09-16T00:10:21.300 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024003449999999999 2024-09-16T00:10:21.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.300 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.300 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885214+0000", 2024-09-16T00:10:21.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0398e-05 2024-09-16T00:10:21.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885224+0000", 2024-09-16T00:10:21.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0253e-05 2024-09-16T00:10:21.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885299+0000", 2024-09-16T00:10:21.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5434999999999999e-05 2024-09-16T00:10:21.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885389+0000", 2024-09-16T00:10:21.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9209999999999995e-05 2024-09-16T00:10:21.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:21.303 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885485+0000", 2024-09-16T00:10:21.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6218999999999995e-05 2024-09-16T00:10:21.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.304 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.304 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.893457+0000", 2024-09-16T00:10:21.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0079722009999999999 2024-09-16T00:10:21.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.304 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:21.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895681+0000", 2024-09-16T00:10:21.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022241449999999999 2024-09-16T00:10:21.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.306 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895732+0000", 2024-09-16T00:10:21.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1269e-05 2024-09-16T00:10:21.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.306 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.306 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895798+0000", 2024-09-16T00:10:21.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5529000000000002e-05 2024-09-16T00:10:21.307 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.307 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.307 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.307 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15058 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304 [2@2556691]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-16T00:10:21.307 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:21.307 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 26.836857253000002, 2024-09-16T00:10:21.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012275842, 2024-09-16T00:10:21.307 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.307 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.308 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.308 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:21.308 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:21.308 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.308 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15058 2024-09-16T00:10:21.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.308 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:21.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:21.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335161+0000", 2024-09-16T00:10:21.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.367e-06 2024-09-16T00:10:21.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337545+0000", 2024-09-16T00:10:21.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023839040000000001 2024-09-16T00:10:21.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337572+0000", 2024-09-16T00:10:21.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.673e-05 2024-09-16T00:10:21.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337583+0000", 2024-09-16T00:10:21.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1048e-05 2024-09-16T00:10:21.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337660+0000", 2024-09-16T00:10:21.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7459999999999994e-05 2024-09-16T00:10:21.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:21.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337770+0000", 2024-09-16T00:10:21.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109634 2024-09-16T00:10:21.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:21.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337926+0000", 2024-09-16T00:10:21.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015601999999999999 2024-09-16T00:10:21.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.345689+0000", 2024-09-16T00:10:21.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0077625630000000001 2024-09-16T00:10:21.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:21.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347324+0000", 2024-09-16T00:10:21.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016351479999999999 2024-09-16T00:10:21.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:21.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347387+0000", 2024-09-16T00:10:21.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3199000000000003e-05 2024-09-16T00:10:21.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347435+0000", 2024-09-16T00:10:21.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7769000000000003e-05 2024-09-16T00:10:21.315 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.315 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.315 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.315 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9780 2.a 2:545c15c7:::200.00000017:head [write 1367106~16252 [fadvise_dontneed]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e106)", 2024-09-16T00:10:21.315 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:21.315 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 4.8346731290000005, 2024-09-16T00:10:21.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0208196810000001, 2024-09-16T00:10:21.315 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:21.315 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:21.315 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:10:21.316 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:21.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:21.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:21.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:21.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:21.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:21.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:21.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337345+0000", 2024-09-16T00:10:21.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.007e-06 2024-09-16T00:10:21.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:21.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337416+0000", 2024-09-16T00:10:21.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1260999999999994e-05 2024-09-16T00:10:21.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:21.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337427+0000", 2024-09-16T00:10:21.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1225e-05 2024-09-16T00:10:21.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337434+0000", 2024-09-16T00:10:21.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8929999999999999e-06 2024-09-16T00:10:21.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337494+0000", 2024-09-16T00:10:21.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9670999999999998e-05 2024-09-16T00:10:21.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:21.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337505+0000", 2024-09-16T00:10:21.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0934999999999999e-05 2024-09-16T00:10:21.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.345708+0000", 2024-09-16T00:10:21.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0082033290000001 2024-09-16T00:10:21.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for missing object", 2024-09-16T00:10:21.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.345819+0000", 2024-09-16T00:10:21.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110299 2024-09-16T00:10:21.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.346937+0000", 2024-09-16T00:10:21.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011184120000000001 2024-09-16T00:10:21.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for degraded object", 2024-09-16T00:10:21.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.346968+0000", 2024-09-16T00:10:21.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0664999999999999e-05 2024-09-16T00:10:21.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:21.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.358107+0000", 2024-09-16T00:10:21.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011139046 2024-09-16T00:10:21.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:21.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:21.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:21.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.358163+0000", 2024-09-16T00:10:21.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5937999999999999e-05 2024-09-16T00:10:21.320 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.321 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.321 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.321 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:21.321 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:21.321 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:21.322 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:10:21.323 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:21.320+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:21.352 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-09-16T00:10:21.352 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:10:21.352 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-16T00:10:21.352 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T00:10:21.353 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T00:10:21.353 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-16T00:10:21.374 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:21.372+0000 7f6152ffd640 1 -- 172.21.15.139:0/2882474046 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v111) ==== 97+0+0 (secure 0 0 0) 0x7f615c059210 con 0x7f61640ce810 2024-09-16T00:10:21.374 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is set 2024-09-16T00:10:21.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:21.375+0000 7f6168fc6640 1 -- 172.21.15.139:0/2882474046 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f614406c2b0 msgr2=0x7f614406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:21.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:21.375+0000 7f6168fc6640 1 --2- 172.21.15.139:0/2882474046 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f614406c2b0 0x7f614406e770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6154034000 tx=0x7f615402e000 comp rx=0 tx=0).stop 2024-09-16T00:10:21.377 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:21.375+0000 7f6168fc6640 1 -- 172.21.15.139:0/2882474046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61640ce810 msgr2=0x7f61640c70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:21.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:21.375+0000 7f6168fc6640 1 --2- 172.21.15.139:0/2882474046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61640ce810 0x7f61640c70f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f615c00cbf0 tx=0x7f615c00a430 comp rx=0 tx=0).stop 2024-09-16T00:10:21.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:21.375+0000 7f6168fc6640 1 -- 172.21.15.139:0/2882474046 shutdown_connections 2024-09-16T00:10:21.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:21.375+0000 7f6168fc6640 1 --2- 172.21.15.139:0/2882474046 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f614406c2b0 0x7f614406e770 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:21.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:21.375+0000 7f6168fc6640 1 --2- 172.21.15.139:0/2882474046 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61640ce810 0x7f61640c70f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:21.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:21.375+0000 7f6168fc6640 1 --2- 172.21.15.139:0/2882474046 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61640ce430 0x7f61640c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:21.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:21.375+0000 7f6168fc6640 1 --2- 172.21.15.139:0/2882474046 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61640c9430 0x7f6164152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:21.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:21.375+0000 7f6168fc6640 1 -- 172.21.15.139:0/2882474046 >> 172.21.15.139:0/2882474046 conn(0x7f616406c150 msgr2=0x7f616415aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:21.378 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:21.375+0000 7f6168fc6640 1 -- 172.21.15.139:0/2882474046 shutdown_connections 2024-09-16T00:10:21.379 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:21.375+0000 7f6168fc6640 1 -- 172.21.15.139:0/2882474046 wait complete. 2024-09-16T00:10:21.422 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:21.421+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:21.455 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:21.455 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:21.455 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:21.465 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:10:21.475 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:21.475 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T00:10:21.475 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:21.524 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:21.522+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:21.610 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:21.611 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:21.611 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:21.611 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:21.611 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:21.620 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:10:21.625 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:21.623+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:21.725 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:21.724+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:21.732 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:21.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10780 3.a 3:500c83b2:::100000030b4.00000000:head [write 0~4194304 [3@2137521]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.275902658, 2024-09-16T00:10:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015526478, 2024-09-16T00:10:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.733 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10780 2024-09-16T00:10:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452917+0000", 2024-09-16T00:10:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2180000000000002e-06 2024-09-16T00:10:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462177+0000", 2024-09-16T00:10:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092593769999999992 2024-09-16T00:10:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462196+0000", 2024-09-16T00:10:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9745000000000002e-05 2024-09-16T00:10:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462207+0000", 2024-09-16T00:10:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0524e-05 2024-09-16T00:10:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462284+0000", 2024-09-16T00:10:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7608000000000006e-05 2024-09-16T00:10:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462376+0000", 2024-09-16T00:10:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1811999999999994e-05 2024-09-16T00:10:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468357+0000", 2024-09-16T00:10:21.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059806809999999998 2024-09-16T00:10:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468386+0000", 2024-09-16T00:10:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9068999999999999e-05 2024-09-16T00:10:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468441+0000", 2024-09-16T00:10:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5444e-05 2024-09-16T00:10:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10831 3.1a 3:59b8f98c:::100000030df.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.953095521999998, 2024-09-16T00:10:21.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014692843000000001, 2024-09-16T00:10:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10831 2024-09-16T00:10:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775723+0000", 2024-09-16T00:10:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.367e-06 2024-09-16T00:10:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780402+0000", 2024-09-16T00:10:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046785769999999997 2024-09-16T00:10:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780421+0000", 2024-09-16T00:10:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9179000000000001e-05 2024-09-16T00:10:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780440+0000", 2024-09-16T00:10:21.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9015e-05 2024-09-16T00:10:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780533+0000", 2024-09-16T00:10:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3197999999999998e-05 2024-09-16T00:10:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780618+0000", 2024-09-16T00:10:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4787e-05 2024-09-16T00:10:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780748+0000", 2024-09-16T00:10:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012927199999999999 2024-09-16T00:10:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.785049+0000", 2024-09-16T00:10:21.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043015750000000002 2024-09-16T00:10:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790312+0000", 2024-09-16T00:10:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052633319999999999 2024-09-16T00:10:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790369+0000", 2024-09-16T00:10:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6848999999999997e-05 2024-09-16T00:10:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790415+0000", 2024-09-16T00:10:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5692000000000002e-05 2024-09-16T00:10:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.748 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10915 3.1a 3:5a6e15a2:::1000000318e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.513589601, 2024-09-16T00:10:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014599472, 2024-09-16T00:10:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.749 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10915 2024-09-16T00:10:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215230+0000", 2024-09-16T00:10:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0090000000000002e-06 2024-09-16T00:10:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219556+0000", 2024-09-16T00:10:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043259539999999999 2024-09-16T00:10:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219577+0000", 2024-09-16T00:10:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1271999999999999e-05 2024-09-16T00:10:21.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219589+0000", 2024-09-16T00:10:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1766000000000001e-05 2024-09-16T00:10:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219663+0000", 2024-09-16T00:10:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4196999999999995e-05 2024-09-16T00:10:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219733+0000", 2024-09-16T00:10:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9553000000000003e-05 2024-09-16T00:10:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219827+0000", 2024-09-16T00:10:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4395999999999995e-05 2024-09-16T00:10:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.225724+0000", 2024-09-16T00:10:21.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058968359999999999 2024-09-16T00:10:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229705+0000", 2024-09-16T00:10:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003981409 2024-09-16T00:10:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229772+0000", 2024-09-16T00:10:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6137999999999996e-05 2024-09-16T00:10:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229828+0000", 2024-09-16T00:10:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5942000000000001e-05 2024-09-16T00:10:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11074 3.14 3:2834b7ee:::10000003131.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.816049477999996, 2024-09-16T00:10:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015588350000000001, 2024-09-16T00:10:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.755 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11074 2024-09-16T00:10:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912769+0000", 2024-09-16T00:10:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4600000000000004e-07 2024-09-16T00:10:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918264+0000", 2024-09-16T00:10:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054954890000000001 2024-09-16T00:10:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918281+0000", 2024-09-16T00:10:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6160000000000001e-05 2024-09-16T00:10:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918289+0000", 2024-09-16T00:10:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1650000000000006e-06 2024-09-16T00:10:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918386+0000", 2024-09-16T00:10:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7288999999999996e-05 2024-09-16T00:10:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918535+0000", 2024-09-16T00:10:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014876200000000001 2024-09-16T00:10:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918639+0000", 2024-09-16T00:10:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010470400000000001 2024-09-16T00:10:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.924703+0000", 2024-09-16T00:10:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060639759999999996 2024-09-16T00:10:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928246+0000", 2024-09-16T00:10:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035421720000000001 2024-09-16T00:10:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928305+0000", 2024-09-16T00:10:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9014000000000001e-05 2024-09-16T00:10:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928357+0000", 2024-09-16T00:10:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1873000000000003e-05 2024-09-16T00:10:21.760 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11111 3.9 3:93301cc4:::10000003210.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.639311276000001, 2024-09-16T00:10:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014336099999999999, 2024-09-16T00:10:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.761 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11111 2024-09-16T00:10:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089520+0000", 2024-09-16T00:10:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3903000000000001e-05 2024-09-16T00:10:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094463+0000", 2024-09-16T00:10:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049423000000000002 2024-09-16T00:10:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094494+0000", 2024-09-16T00:10:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1029999999999999e-05 2024-09-16T00:10:21.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094503+0000", 2024-09-16T00:10:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0180000000000007e-06 2024-09-16T00:10:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094573+0000", 2024-09-16T00:10:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0457000000000007e-05 2024-09-16T00:10:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094737+0000", 2024-09-16T00:10:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016408699999999999 2024-09-16T00:10:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094897+0000", 2024-09-16T00:10:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015938899999999999 2024-09-16T00:10:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.102326+0000", 2024-09-16T00:10:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074291230000000002 2024-09-16T00:10:21.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103723+0000", 2024-09-16T00:10:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001397255 2024-09-16T00:10:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103786+0000", 2024-09-16T00:10:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3489000000000001e-05 2024-09-16T00:10:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103842+0000", 2024-09-16T00:10:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6048999999999998e-05 2024-09-16T00:10:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:11934 3.1b e87/20 3:d94fa8da:::10000003190.00000000:head v 87'152, mlcod=87'150)", 2024-09-16T00:10:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.038566580000001, 2024-09-16T00:10:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014984934, 2024-09-16T00:10:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.767 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:21.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690252+0000", 2024-09-16T00:10:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8899999999999998e-07 2024-09-16T00:10:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696026+0000", 2024-09-16T00:10:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057745219999999998 2024-09-16T00:10:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696039+0000", 2024-09-16T00:10:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2611e-05 2024-09-16T00:10:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696047+0000", 2024-09-16T00:10:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5399999999999998e-06 2024-09-16T00:10:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696123+0000", 2024-09-16T00:10:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6349999999999999e-05 2024-09-16T00:10:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696144+0000", 2024-09-16T00:10:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1512999999999999e-05 2024-09-16T00:10:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705188+0000", 2024-09-16T00:10:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090438459999999995 2024-09-16T00:10:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705236+0000", 2024-09-16T00:10:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7663000000000001e-05 2024-09-16T00:10:21.771 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11936 3.3 3:c0886fb9:::100000031a1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.028421528000003, 2024-09-16T00:10:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015336379000000001, 2024-09-16T00:10:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.772 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11936 2024-09-16T00:10:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:21.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700399+0000", 2024-09-16T00:10:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6029999999999999e-06 2024-09-16T00:10:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704706+0000", 2024-09-16T00:10:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043070529999999999 2024-09-16T00:10:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704720+0000", 2024-09-16T00:10:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4629e-05 2024-09-16T00:10:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704729+0000", 2024-09-16T00:10:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6009999999999995e-06 2024-09-16T00:10:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704805+0000", 2024-09-16T00:10:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6015999999999999e-05 2024-09-16T00:10:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704866+0000", 2024-09-16T00:10:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0878999999999998e-05 2024-09-16T00:10:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704990+0000", 2024-09-16T00:10:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000124058 2024-09-16T00:10:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.714314+0000", 2024-09-16T00:10:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009324321 2024-09-16T00:10:21.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715632+0000", 2024-09-16T00:10:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001318121 2024-09-16T00:10:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715687+0000", 2024-09-16T00:10:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4135999999999998e-05 2024-09-16T00:10:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715732+0000", 2024-09-16T00:10:21.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961999999999998e-05 2024-09-16T00:10:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11953 3.1a 3:580e2503:::1000000311b.00000000:head [write 0~4194304 [2@2058066]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.962082375000001, 2024-09-16T00:10:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016299973999999998, 2024-09-16T00:10:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.778 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11953 2024-09-16T00:10:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766741+0000", 2024-09-16T00:10:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2939999999999996e-06 2024-09-16T00:10:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771165+0000", 2024-09-16T00:10:21.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044248229999999996 2024-09-16T00:10:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771181+0000", 2024-09-16T00:10:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5243e-05 2024-09-16T00:10:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771188+0000", 2024-09-16T00:10:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6429999999999995e-06 2024-09-16T00:10:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771238+0000", 2024-09-16T00:10:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9567999999999997e-05 2024-09-16T00:10:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771293+0000", 2024-09-16T00:10:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5207000000000002e-05 2024-09-16T00:10:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771380+0000", 2024-09-16T00:10:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6716000000000001e-05 2024-09-16T00:10:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.776193+0000", 2024-09-16T00:10:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048133350000000002 2024-09-16T00:10:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.782931+0000", 2024-09-16T00:10:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067376550000000004 2024-09-16T00:10:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.782987+0000", 2024-09-16T00:10:21.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6443000000000002e-05 2024-09-16T00:10:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.783035+0000", 2024-09-16T00:10:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8047000000000003e-05 2024-09-16T00:10:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12037 3.9 3:928db521:::1000000326d.00000000:head [write 0~4194304 [2@1655200]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:21.784 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.730874767000003, 2024-09-16T00:10:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017082494, 2024-09-16T00:10:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12037 2024-09-16T00:10:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:21.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997944+0000", 2024-09-16T00:10:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.517e-06 2024-09-16T00:10:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002177+0000", 2024-09-16T00:10:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042329300000000002 2024-09-16T00:10:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002197+0000", 2024-09-16T00:10:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9769e-05 2024-09-16T00:10:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002207+0000", 2024-09-16T00:10:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0178e-05 2024-09-16T00:10:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002266+0000", 2024-09-16T00:10:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9215999999999997e-05 2024-09-16T00:10:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002340+0000", 2024-09-16T00:10:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3798000000000001e-05 2024-09-16T00:10:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002454+0000", 2024-09-16T00:10:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113551 2024-09-16T00:10:21.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.006270+0000", 2024-09-16T00:10:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038163839999999999 2024-09-16T00:10:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.014957+0000", 2024-09-16T00:10:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086869119999999998 2024-09-16T00:10:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.014996+0000", 2024-09-16T00:10:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8878000000000003e-05 2024-09-16T00:10:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.015025+0000", 2024-09-16T00:10:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9360999999999999e-05 2024-09-16T00:10:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12245 3.1a 3:599a1cd1:::10000003252.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:21.790 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 36.946177349999999, 2024-09-16T00:10:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017131203000000001, 2024-09-16T00:10:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12245 2024-09-16T00:10:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782642+0000", 2024-09-16T00:10:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.311e-06 2024-09-16T00:10:21.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786551+0000", 2024-09-16T00:10:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039095229999999998 2024-09-16T00:10:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786566+0000", 2024-09-16T00:10:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4724e-05 2024-09-16T00:10:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786575+0000", 2024-09-16T00:10:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8449999999999992e-06 2024-09-16T00:10:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786636+0000", 2024-09-16T00:10:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.143e-05 2024-09-16T00:10:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786699+0000", 2024-09-16T00:10:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3218000000000004e-05 2024-09-16T00:10:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:21.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786796+0000", 2024-09-16T00:10:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6347000000000004e-05 2024-09-16T00:10:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.792369+0000", 2024-09-16T00:10:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055729780000000001 2024-09-16T00:10:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799706+0000", 2024-09-16T00:10:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073374349999999998 2024-09-16T00:10:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799739+0000", 2024-09-16T00:10:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2833999999999998e-05 2024-09-16T00:10:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799771+0000", 2024-09-16T00:10:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2558000000000002e-05 2024-09-16T00:10:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12292 3.3 3:c00779fd:::1000000320e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 36.831787253000002, 2024-09-16T00:10:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013828353999999999, 2024-09-16T00:10:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12292 2024-09-16T00:10:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.797 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897039+0000", 2024-09-16T00:10:21.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3299999999999999e-06 2024-09-16T00:10:21.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900830+0000", 2024-09-16T00:10:21.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037915340000000001 2024-09-16T00:10:21.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900850+0000", 2024-09-16T00:10:21.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9400000000000001e-05 2024-09-16T00:10:21.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900860+0000", 2024-09-16T00:10:21.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0609e-05 2024-09-16T00:10:21.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900928+0000", 2024-09-16T00:10:21.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7570000000000005e-05 2024-09-16T00:10:21.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.901027+0000", 2024-09-16T00:10:21.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9456000000000005e-05 2024-09-16T00:10:21.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:21.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.901168+0000", 2024-09-16T00:10:21.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014082799999999999 2024-09-16T00:10:21.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.906325+0000", 2024-09-16T00:10:21.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005156702 2024-09-16T00:10:21.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910796+0000", 2024-09-16T00:10:21.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044714350000000002 2024-09-16T00:10:21.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910829+0000", 2024-09-16T00:10:21.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3229000000000003e-05 2024-09-16T00:10:21.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910859+0000", 2024-09-16T00:10:21.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9261e-05 2024-09-16T00:10:21.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.802 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.803 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13326 3.9 3:92f796cf:::1000000350f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:21.803 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:21.803 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.772096971000003, 2024-09-16T00:10:21.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015673229, 2024-09-16T00:10:21.803 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.803 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.804 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.804 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.804 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.804 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.804 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13326 2024-09-16T00:10:21.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.805 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:21.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:21.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956722+0000", 2024-09-16T00:10:21.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3200000000000001e-06 2024-09-16T00:10:21.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960803+0000", 2024-09-16T00:10:21.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040807029999999998 2024-09-16T00:10:21.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960820+0000", 2024-09-16T00:10:21.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7478000000000001e-05 2024-09-16T00:10:21.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960830+0000", 2024-09-16T00:10:21.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:10:21.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960903+0000", 2024-09-16T00:10:21.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3057000000000002e-05 2024-09-16T00:10:21.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960970+0000", 2024-09-16T00:10:21.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7063000000000005e-05 2024-09-16T00:10:21.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:21.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.961067+0000", 2024-09-16T00:10:21.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6906999999999999e-05 2024-09-16T00:10:21.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.967945+0000", 2024-09-16T00:10:21.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068787359999999999 2024-09-16T00:10:21.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972328+0000", 2024-09-16T00:10:21.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043826170000000001 2024-09-16T00:10:21.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972363+0000", 2024-09-16T00:10:21.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4486000000000002e-05 2024-09-16T00:10:21.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972394+0000", 2024-09-16T00:10:21.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1343999999999997e-05 2024-09-16T00:10:21.813 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.813 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.813 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.813 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13429 3.1a 3:58dcd24a:::100000036d5.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:21.813 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:21.813 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.417526955, 2024-09-16T00:10:21.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014237671, 2024-09-16T00:10:21.814 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.814 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.814 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.814 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.814 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13429 2024-09-16T00:10:21.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.814 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:21.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:21.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311292+0000", 2024-09-16T00:10:21.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4890000000000001e-06 2024-09-16T00:10:21.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316378+0000", 2024-09-16T00:10:21.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050859290000000003 2024-09-16T00:10:21.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316399+0000", 2024-09-16T00:10:21.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1345e-05 2024-09-16T00:10:21.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316410+0000", 2024-09-16T00:10:21.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0295999999999999e-05 2024-09-16T00:10:21.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316493+0000", 2024-09-16T00:10:21.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3533000000000001e-05 2024-09-16T00:10:21.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316622+0000", 2024-09-16T00:10:21.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012887900000000001 2024-09-16T00:10:21.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:21.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316725+0000", 2024-09-16T00:10:21.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010295599999999999 2024-09-16T00:10:21.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.320948+0000", 2024-09-16T00:10:21.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042229820000000001 2024-09-16T00:10:21.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325415+0000", 2024-09-16T00:10:21.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044673849999999999 2024-09-16T00:10:21.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325487+0000", 2024-09-16T00:10:21.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1087999999999994e-05 2024-09-16T00:10:21.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325528+0000", 2024-09-16T00:10:21.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1789000000000003e-05 2024-09-16T00:10:21.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.821 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.822 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13527 3.3 3:c0886fb9:::100000031a1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:21.822 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:21.822 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.943366865999998, 2024-09-16T00:10:21.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015187236, 2024-09-16T00:10:21.822 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.822 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.822 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.822 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13527 2024-09-16T00:10:21.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.823 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:21.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:21.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785453+0000", 2024-09-16T00:10:21.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1689999999999999e-06 2024-09-16T00:10:21.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791191+0000", 2024-09-16T00:10:21.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057378339999999998 2024-09-16T00:10:21.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791217+0000", 2024-09-16T00:10:21.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6480999999999999e-05 2024-09-16T00:10:21.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791233+0000", 2024-09-16T00:10:21.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5841e-05 2024-09-16T00:10:21.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791319+0000", 2024-09-16T00:10:21.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5527999999999998e-05 2024-09-16T00:10:21.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791413+0000", 2024-09-16T00:10:21.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4685000000000006e-05 2024-09-16T00:10:21.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:21.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791563+0000", 2024-09-16T00:10:21.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000149624 2024-09-16T00:10:21.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.795749+0000", 2024-09-16T00:10:21.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004186286 2024-09-16T00:10:21.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800577+0000", 2024-09-16T00:10:21.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048273040000000001 2024-09-16T00:10:21.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800611+0000", 2024-09-16T00:10:21.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4270000000000002e-05 2024-09-16T00:10:21.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800638+0000", 2024-09-16T00:10:21.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7214000000000001e-05 2024-09-16T00:10:21.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.827 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.828 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13724 3.9 3:90613e6c:::100000034c4.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:21.828 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:21.828 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.273295120999997, 2024-09-16T00:10:21.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014039800999999999, 2024-09-16T00:10:21.828 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.828 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.828 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.829 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.829 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13724 2024-09-16T00:10:21.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.829 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:21.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:21.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455524+0000", 2024-09-16T00:10:21.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.437e-06 2024-09-16T00:10:21.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459592+0000", 2024-09-16T00:10:21.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040678729999999996 2024-09-16T00:10:21.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459610+0000", 2024-09-16T00:10:21.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7688000000000001e-05 2024-09-16T00:10:21.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459617+0000", 2024-09-16T00:10:21.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.374e-06 2024-09-16T00:10:21.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459678+0000", 2024-09-16T00:10:21.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1346999999999994e-05 2024-09-16T00:10:21.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459824+0000", 2024-09-16T00:10:21.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014586699999999999 2024-09-16T00:10:21.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:21.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459908+0000", 2024-09-16T00:10:21.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3521000000000002e-05 2024-09-16T00:10:21.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.464227+0000", 2024-09-16T00:10:21.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043197060000000004 2024-09-16T00:10:21.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469499+0000", 2024-09-16T00:10:21.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005271838 2024-09-16T00:10:21.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469534+0000", 2024-09-16T00:10:21.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4712000000000002e-05 2024-09-16T00:10:21.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469562+0000", 2024-09-16T00:10:21.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8438000000000001e-05 2024-09-16T00:10:21.834 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.834 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.834 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.835 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13741 3.3 3:c1377c76:::10000003664.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:21.835 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:21.835 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 32.226125175999996, 2024-09-16T00:10:21.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016142843, 2024-09-16T00:10:21.835 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.835 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.835 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.835 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.835 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13741 2024-09-16T00:10:21.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.835 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:21.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:21.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502694+0000", 2024-09-16T00:10:21.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0449999999999999e-06 2024-09-16T00:10:21.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507229+0000", 2024-09-16T00:10:21.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045357410000000003 2024-09-16T00:10:21.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507248+0000", 2024-09-16T00:10:21.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9127000000000002e-05 2024-09-16T00:10:21.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507259+0000", 2024-09-16T00:10:21.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0448999999999999e-05 2024-09-16T00:10:21.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507357+0000", 2024-09-16T00:10:21.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8400000000000007e-05 2024-09-16T00:10:21.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507514+0000", 2024-09-16T00:10:21.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000157195 2024-09-16T00:10:21.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:21.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507617+0000", 2024-09-16T00:10:21.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010262199999999999 2024-09-16T00:10:21.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.514047+0000", 2024-09-16T00:10:21.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006430018 2024-09-16T00:10:21.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518743+0000", 2024-09-16T00:10:21.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046963040000000001 2024-09-16T00:10:21.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518796+0000", 2024-09-16T00:10:21.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2305000000000002e-05 2024-09-16T00:10:21.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518835+0000", 2024-09-16T00:10:21.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9637e-05 2024-09-16T00:10:21.846 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.846 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.846 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.847 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14633 3.9 3:9170e82c:::100000035d6.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:21.847 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:21.847 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 29.106744920000001, 2024-09-16T00:10:21.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014466105, 2024-09-16T00:10:21.847 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.848 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.848 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.848 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.848 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14633 2024-09-16T00:10:21.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.849 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:21.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:21.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622074+0000", 2024-09-16T00:10:21.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.761e-06 2024-09-16T00:10:21.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626712+0000", 2024-09-16T00:10:21.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004637263 2024-09-16T00:10:21.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626724+0000", 2024-09-16T00:10:21.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2479e-05 2024-09-16T00:10:21.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626730+0000", 2024-09-16T00:10:21.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7309999999999998e-06 2024-09-16T00:10:21.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626791+0000", 2024-09-16T00:10:21.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0854999999999999e-05 2024-09-16T00:10:21.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626857+0000", 2024-09-16T00:10:21.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6549000000000003e-05 2024-09-16T00:10:21.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:21.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626943+0000", 2024-09-16T00:10:21.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6141000000000001e-05 2024-09-16T00:10:21.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.630854+0000", 2024-09-16T00:10:21.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039100790000000003 2024-09-16T00:10:21.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636468+0000", 2024-09-16T00:10:21.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056140570000000004 2024-09-16T00:10:21.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636500+0000", 2024-09-16T00:10:21.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2441999999999999e-05 2024-09-16T00:10:21.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636539+0000", 2024-09-16T00:10:21.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8748000000000002e-05 2024-09-16T00:10:21.856 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.856 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.856 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.856 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14667 3.9 3:9379a4af:::10000003352.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:21.856 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:21.856 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.931732914000001, 2024-09-16T00:10:21.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014580585, 2024-09-16T00:10:21.856 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.857 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.857 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.857 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.857 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14667 2024-09-16T00:10:21.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.857 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:21.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:21.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797086+0000", 2024-09-16T00:10:21.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.362e-06 2024-09-16T00:10:21.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801415+0000", 2024-09-16T00:10:21.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043288449999999996 2024-09-16T00:10:21.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801435+0000", 2024-09-16T00:10:21.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0305999999999998e-05 2024-09-16T00:10:21.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801442+0000", 2024-09-16T00:10:21.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6290000000000004e-06 2024-09-16T00:10:21.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801497+0000", 2024-09-16T00:10:21.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5111999999999997e-05 2024-09-16T00:10:21.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801639+0000", 2024-09-16T00:10:21.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014168000000000001 2024-09-16T00:10:21.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:21.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801788+0000", 2024-09-16T00:10:21.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014963300000000001 2024-09-16T00:10:21.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.805407+0000", 2024-09-16T00:10:21.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036186119999999998 2024-09-16T00:10:21.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811584+0000", 2024-09-16T00:10:21.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061772889999999999 2024-09-16T00:10:21.862 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811635+0000", 2024-09-16T00:10:21.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0453e-05 2024-09-16T00:10:21.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811665+0000", 2024-09-16T00:10:21.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0664000000000003e-05 2024-09-16T00:10:21.864 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.864 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.864 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.864 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14951 3.1a 3:59aee80a:::1000000394f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:21.864 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:21.864 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.761695019000001, 2024-09-16T00:10:21.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015993941000000001, 2024-09-16T00:10:21.865 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.865 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.865 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.865 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.865 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14951 2024-09-16T00:10:21.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.865 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:21.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:21.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967124+0000", 2024-09-16T00:10:21.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.86e-06 2024-09-16T00:10:21.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971258+0000", 2024-09-16T00:10:21.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041334780000000003 2024-09-16T00:10:21.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971274+0000", 2024-09-16T00:10:21.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6201e-05 2024-09-16T00:10:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971284+0000", 2024-09-16T00:10:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4779999999999997e-06 2024-09-16T00:10:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971384+0000", 2024-09-16T00:10:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9878999999999997e-05 2024-09-16T00:10:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971518+0000", 2024-09-16T00:10:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013459400000000001 2024-09-16T00:10:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971604+0000", 2024-09-16T00:10:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6215e-05 2024-09-16T00:10:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.976673+0000", 2024-09-16T00:10:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050685920000000002 2024-09-16T00:10:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983043+0000", 2024-09-16T00:10:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063698410000000002 2024-09-16T00:10:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983082+0000", 2024-09-16T00:10:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9261999999999998e-05 2024-09-16T00:10:21.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983117+0000", 2024-09-16T00:10:21.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4541e-05 2024-09-16T00:10:21.871 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.871 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.871 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.871 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:15152 3.3 3:c388c7e2:::100000032ec.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-16T00:10:21.871 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:21.871 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.001148909000001, 2024-09-16T00:10:21.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014603532000000001, 2024-09-16T00:10:21.872 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:21.872 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:21.872 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:21.872 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:21.872 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:21.872 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:21.872 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15152 2024-09-16T00:10:21.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.872 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:21.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:21.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:21.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:21.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:21.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:21.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:21.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727674+0000", 2024-09-16T00:10:21.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7699999999999998e-06 2024-09-16T00:10:21.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:21.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732274+0000", 2024-09-16T00:10:21.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045994299999999998 2024-09-16T00:10:21.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:21.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732286+0000", 2024-09-16T00:10:21.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2016e-05 2024-09-16T00:10:21.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:21.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732294+0000", 2024-09-16T00:10:21.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7049999999999999e-06 2024-09-16T00:10:21.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:21.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732359+0000", 2024-09-16T00:10:21.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5296000000000006e-05 2024-09-16T00:10:21.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:21.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732501+0000", 2024-09-16T00:10:21.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000141731 2024-09-16T00:10:21.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:21.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732629+0000", 2024-09-16T00:10:21.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012841299999999999 2024-09-16T00:10:21.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:21.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.737511+0000", 2024-09-16T00:10:21.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048822520000000001 2024-09-16T00:10:21.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:21.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742172+0000", 2024-09-16T00:10:21.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046610610000000002 2024-09-16T00:10:21.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:21.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742227+0000", 2024-09-16T00:10:21.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4497999999999999e-05 2024-09-16T00:10:21.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:21.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:21.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:21.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742272+0000", 2024-09-16T00:10:21.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5359999999999999e-05 2024-09-16T00:10:21.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.878 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:21.878 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:21.878 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:21.879 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:21.878+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:21.880 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:10:21.980 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:21.979+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:22.062 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:11691 3.35 e111/94 3:ad66404a:::10000002d4d.00000000:head v 111'249, mlcod=110'247)", 2024-09-16T00:10:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:22.049118+0000", 2024-09-16T00:10:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0085280890000000008, 2024-09-16T00:10:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.008557116, 2024-09-16T00:10:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.062 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.049118+0000", 2024-09-16T00:10:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.049118+0000", 2024-09-16T00:10:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.049118+0000", 2024-09-16T00:10:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2099999999999997e-07 2024-09-16T00:10:22.063 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.049122+0000", 2024-09-16T00:10:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0629999999999999e-06 2024-09-16T00:10:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.049131+0000", 2024-09-16T00:10:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8459999999999997e-06 2024-09-16T00:10:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.064 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.049137+0000", 2024-09-16T00:10:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6509999999999998e-06 2024-09-16T00:10:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052659+0000", 2024-09-16T00:10:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035221969999999999 2024-09-16T00:10:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052680+0000", 2024-09-16T00:10:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1240999999999999e-05 2024-09-16T00:10:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.065 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:11716 3.d e111/51 3:b2bbf284:::10000002d66.00000000:head v 111'297, mlcod=111'295)", 2024-09-16T00:10:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:22.051050+0000", 2024-09-16T00:10:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0065958470000000002, 2024-09-16T00:10:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066880710000000003, 2024-09-16T00:10:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.051050+0000", 2024-09-16T00:10:22.066 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.051050+0000", 2024-09-16T00:10:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.051051+0000", 2024-09-16T00:10:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4200000000000005e-07 2024-09-16T00:10:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.067 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.051057+0000", 2024-09-16T00:10:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6860000000000003e-06 2024-09-16T00:10:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.051066+0000", 2024-09-16T00:10:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.696e-06 2024-09-16T00:10:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.051073+0000", 2024-09-16T00:10:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9249999999999998e-06 2024-09-16T00:10:22.068 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.053422+0000", 2024-09-16T00:10:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0023483900000000001 2024-09-16T00:10:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.053444+0000", 2024-09-16T00:10:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2736000000000001e-05 2024-09-16T00:10:22.069 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:11690 3.2 e111/20 3:437732b4:::10000002d4c.00000000:head v 111'268, mlcod=110'266)", 2024-09-16T00:10:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:22.052249+0000", 2024-09-16T00:10:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0053972309999999997, 2024-09-16T00:10:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055507619999999999, 2024-09-16T00:10:22.070 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052249+0000", 2024-09-16T00:10:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.072 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052249+0000", 2024-09-16T00:10:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052249+0000", 2024-09-16T00:10:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0800000000000004e-07 2024-09-16T00:10:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052257+0000", 2024-09-16T00:10:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4560000000000003e-06 2024-09-16T00:10:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.073 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052270+0000", 2024-09-16T00:10:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.311e-05 2024-09-16T00:10:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052276+0000", 2024-09-16T00:10:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6039999999999998e-06 2024-09-16T00:10:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052606+0000", 2024-09-16T00:10:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000329976 2024-09-16T00:10:22.074 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052626+0000", 2024-09-16T00:10:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9439000000000001e-05 2024-09-16T00:10:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:11727 3.2a 3:573f7dba:::10000002d71.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111)", 2024-09-16T00:10:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:22.052625+0000", 2024-09-16T00:10:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0050210860000000001, 2024-09-16T00:10:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052393860000000004, 2024-09-16T00:10:22.075 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "waiting for sub ops", 2024-09-16T00:10:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052625+0000", 2024-09-16T00:10:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052625+0000", 2024-09-16T00:10:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.076 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052626+0000", 2024-09-16T00:10:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2160000000000001e-06 2024-09-16T00:10:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052631+0000", 2024-09-16T00:10:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6280000000000004e-06 2024-09-16T00:10:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052642+0000", 2024-09-16T00:10:22.077 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1610999999999999e-05 2024-09-16T00:10:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052650+0000", 2024-09-16T00:10:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2599999999999999e-06 2024-09-16T00:10:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052869+0000", 2024-09-16T00:10:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00021964 2024-09-16T00:10:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.078 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052972+0000", 2024-09-16T00:10:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102487 2024-09-16T00:10:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.053054+0000", 2024-09-16T00:10:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2287999999999995e-05 2024-09-16T00:10:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:11739 3.28 3:162ccf33:::10000002d7d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111)", 2024-09-16T00:10:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:22.052873+0000", 2024-09-16T00:10:22.079 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0047733360000000004, 2024-09-16T00:10:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050721000000000004, 2024-09-16T00:10:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "waiting for sub ops", 2024-09-16T00:10:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052873+0000", 2024-09-16T00:10:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.080 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052873+0000", 2024-09-16T00:10:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052873+0000", 2024-09-16T00:10:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1299999999999999e-07 2024-09-16T00:10:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.082 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052877+0000", 2024-09-16T00:10:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7670000000000001e-06 2024-09-16T00:10:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052888+0000", 2024-09-16T00:10:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0370000000000001e-05 2024-09-16T00:10:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.052895+0000", 2024-09-16T00:10:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0450000000000003e-06 2024-09-16T00:10:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.083 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.053257+0000", 2024-09-16T00:10:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00036292399999999998 2024-09-16T00:10:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.053367+0000", 2024-09-16T00:10:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109277 2024-09-16T00:10:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:10:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.053509+0000", 2024-09-16T00:10:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000142519 2024-09-16T00:10:22.084 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.055931+0000", 2024-09-16T00:10:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024215539999999998 2024-09-16T00:10:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.057921+0000", 2024-09-16T00:10:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0019897869999999998 2024-09-16T00:10:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.085 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:11720 3.37 e111/20 3:ed6ec24b:::10000002d6a.00000000:head v 111'286, mlcod=111'284)", 2024-09-16T00:10:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:22.053448+0000", 2024-09-16T00:10:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0041977899999999999, 2024-09-16T00:10:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045760719999999996, 2024-09-16T00:10:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.053448+0000", 2024-09-16T00:10:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.086 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.053448+0000", 2024-09-16T00:10:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.053449+0000", 2024-09-16T00:10:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1900000000000003e-07 2024-09-16T00:10:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.053454+0000", 2024-09-16T00:10:22.087 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4280000000000004e-06 2024-09-16T00:10:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.053465+0000", 2024-09-16T00:10:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0888e-05 2024-09-16T00:10:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.053472+0000", 2024-09-16T00:10:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1670000000000002e-06 2024-09-16T00:10:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.088 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.053547+0000", 2024-09-16T00:10:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4893000000000003e-05 2024-09-16T00:10:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.053568+0000", 2024-09-16T00:10:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1185000000000001e-05 2024-09-16T00:10:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:11732 3.2 e111/20 3:40c7dddf:::10000002d76.00000000:head v 111'269, mlcod=110'266)", 2024-09-16T00:10:22.089 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:22.054849+0000", 2024-09-16T00:10:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.0027970809999999999, 2024-09-16T00:10:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0032335839999999999, 2024-09-16T00:10:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.054849+0000", 2024-09-16T00:10:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.090 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.054849+0000", 2024-09-16T00:10:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.054849+0000", 2024-09-16T00:10:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6600000000000003e-07 2024-09-16T00:10:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.054852+0000", 2024-09-16T00:10:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.96e-06 2024-09-16T00:10:22.091 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.054862+0000", 2024-09-16T00:10:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.465e-06 2024-09-16T00:10:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.054866+0000", 2024-09-16T00:10:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9309999999999999e-06 2024-09-16T00:10:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.092 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.054933+0000", 2024-09-16T00:10:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7045999999999995e-05 2024-09-16T00:10:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.054952+0000", 2024-09-16T00:10:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8711000000000001e-05 2024-09-16T00:10:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.093 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:11745 3.28 3:1715ad80:::10000002d83.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111)", 2024-09-16T00:10:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:22.055456+0000", 2024-09-16T00:10:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.002189421, 2024-09-16T00:10:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026911999999999999, 2024-09-16T00:10:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "waiting for sub ops", 2024-09-16T00:10:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.055456+0000", 2024-09-16T00:10:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.094 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.055456+0000", 2024-09-16T00:10:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.055457+0000", 2024-09-16T00:10:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0100000000000005e-07 2024-09-16T00:10:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.055459+0000", 2024-09-16T00:10:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.916e-06 2024-09-16T00:10:22.095 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.055466+0000", 2024-09-16T00:10:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0319999999999998e-06 2024-09-16T00:10:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.055471+0000", 2024-09-16T00:10:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4790000000000004e-06 2024-09-16T00:10:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.096 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.056771+0000", 2024-09-16T00:10:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012994 2024-09-16T00:10:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.056873+0000", 2024-09-16T00:10:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010173199999999999 2024-09-16T00:10:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:10:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.056975+0000", 2024-09-16T00:10:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000102859 2024-09-16T00:10:22.097 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:11750 3.28 3:1786c2b7:::10000002d88.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111)", 2024-09-16T00:10:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:22.056988+0000", 2024-09-16T00:10:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 0.00065831400000000001, 2024-09-16T00:10:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0012378739999999999, 2024-09-16T00:10:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "waiting for sub ops", 2024-09-16T00:10:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.098 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.056988+0000", 2024-09-16T00:10:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.056988+0000", 2024-09-16T00:10:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.056988+0000", 2024-09-16T00:10:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.23e-07 2024-09-16T00:10:22.099 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.056992+0000", 2024-09-16T00:10:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7390000000000001e-06 2024-09-16T00:10:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.057003+0000", 2024-09-16T00:10:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0815000000000001e-05 2024-09-16T00:10:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.100 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.057010+0000", 2024-09-16T00:10:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.8700000000000003e-06 2024-09-16T00:10:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.057309+0000", 2024-09-16T00:10:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00029880999999999999 2024-09-16T00:10:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.057423+0000", 2024-09-16T00:10:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113887 2024-09-16T00:10:22.101 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 3", 2024-09-16T00:10:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.057519+0000", 2024-09-16T00:10:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6810000000000003e-05 2024-09-16T00:10:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: ], 2024-09-16T00:10:22.102 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 9 2024-09-16T00:10:22.102 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:22.104 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:10:22.104 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:22.102+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:22.205 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:22.203+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:22.254 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:22.254 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:22.254 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:22.255 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:22.255 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:22.264 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:10:22.301 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:10:22.345 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:22.344+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:22.378 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:22.378 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:22.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:22.378 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:22.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.378 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10658 3.1c 3:38c4c24e:::100000030bb.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:22.378 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:22.379 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.583435021, 2024-09-16T00:10:22.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021232445999999999, 2024-09-16T00:10:22.379 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.379 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.379 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:22.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:22.379 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:22.379 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10658 2024-09-16T00:10:22.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.379 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:22.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:22.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791398+0000", 2024-09-16T00:10:22.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3880000000000001e-06 2024-09-16T00:10:22.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800951+0000", 2024-09-16T00:10:22.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095532110000000007 2024-09-16T00:10:22.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800966+0000", 2024-09-16T00:10:22.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5058999999999999e-05 2024-09-16T00:10:22.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800974+0000", 2024-09-16T00:10:22.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1240000000000005e-06 2024-09-16T00:10:22.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801127+0000", 2024-09-16T00:10:22.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000153103 2024-09-16T00:10:22.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801207+0000", 2024-09-16T00:10:22.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9569999999999999e-05 2024-09-16T00:10:22.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801322+0000", 2024-09-16T00:10:22.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115358 2024-09-16T00:10:22.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.805760+0000", 2024-09-16T00:10:22.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044379409999999999 2024-09-16T00:10:22.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812528+0000", 2024-09-16T00:10:22.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067678249999999999 2024-09-16T00:10:22.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812586+0000", 2024-09-16T00:10:22.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8164000000000001e-05 2024-09-16T00:10:22.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812629+0000", 2024-09-16T00:10:22.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2703000000000001e-05 2024-09-16T00:10:22.387 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.387 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.388 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.388 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4122 2.18 2:1cc859d9:::200.00000012:head [write 798310~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.388 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:22.389 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.715895341, 2024-09-16T00:10:22.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019267580999999999, 2024-09-16T00:10:22.389 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.389 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.389 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.390 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:22.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:22.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658937+0000", 2024-09-16T00:10:22.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7700000000000004e-07 2024-09-16T00:10:22.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658942+0000", 2024-09-16T00:10:22.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.48e-06 2024-09-16T00:10:22.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658952+0000", 2024-09-16T00:10:22.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9469999999999998e-06 2024-09-16T00:10:22.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658959+0000", 2024-09-16T00:10:22.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4400000000000002e-06 2024-09-16T00:10:22.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670507+0000", 2024-09-16T00:10:22.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011548159000000001 2024-09-16T00:10:22.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670579+0000", 2024-09-16T00:10:22.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2082999999999994e-05 2024-09-16T00:10:22.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670686+0000", 2024-09-16T00:10:22.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107069 2024-09-16T00:10:22.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678001+0000", 2024-09-16T00:10:22.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073148780000000004 2024-09-16T00:10:22.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678104+0000", 2024-09-16T00:10:22.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010290599999999999 2024-09-16T00:10:22.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678163+0000", 2024-09-16T00:10:22.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8859e-05 2024-09-16T00:10:22.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678204+0000", 2024-09-16T00:10:22.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0982999999999997e-05 2024-09-16T00:10:22.396 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.396 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.397 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4123 2.18 2:1cc859d9:::200.00000012:head [write 799709~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.397 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:22.397 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.71532616, 2024-09-16T00:10:22.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018857569000000001, 2024-09-16T00:10:22.397 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.397 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.397 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.397 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:22.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:22.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659506+0000", 2024-09-16T00:10:22.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.78e-07 2024-09-16T00:10:22.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659512+0000", 2024-09-16T00:10:22.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0100000000000001e-06 2024-09-16T00:10:22.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659523+0000", 2024-09-16T00:10:22.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-16T00:10:22.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659531+0000", 2024-09-16T00:10:22.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2010000000000002e-06 2024-09-16T00:10:22.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671124+0000", 2024-09-16T00:10:22.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011593696000000001 2024-09-16T00:10:22.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671171+0000", 2024-09-16T00:10:22.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6974999999999997e-05 2024-09-16T00:10:22.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671243+0000", 2024-09-16T00:10:22.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2131000000000004e-05 2024-09-16T00:10:22.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678008+0000", 2024-09-16T00:10:22.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067642329999999997 2024-09-16T00:10:22.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678261+0000", 2024-09-16T00:10:22.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025378999999999999 2024-09-16T00:10:22.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678320+0000", 2024-09-16T00:10:22.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8452000000000002e-05 2024-09-16T00:10:22.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678363+0000", 2024-09-16T00:10:22.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2831999999999999e-05 2024-09-16T00:10:22.403 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.403 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.403 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.403 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4126 2.18 2:1cc859d9:::200.00000012:head [write 806443~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.403 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:22.403 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.713928373999998, 2024-09-16T00:10:22.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019331159000000001, 2024-09-16T00:10:22.403 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.403 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.403 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.403 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:22.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:22.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660904+0000", 2024-09-16T00:10:22.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6800000000000009e-07 2024-09-16T00:10:22.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660909+0000", 2024-09-16T00:10:22.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2730000000000002e-06 2024-09-16T00:10:22.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660919+0000", 2024-09-16T00:10:22.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4960000000000002e-06 2024-09-16T00:10:22.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660923+0000", 2024-09-16T00:10:22.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6260000000000003e-06 2024-09-16T00:10:22.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672698+0000", 2024-09-16T00:10:22.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011774573999999999 2024-09-16T00:10:22.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672784+0000", 2024-09-16T00:10:22.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6074000000000003e-05 2024-09-16T00:10:22.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672897+0000", 2024-09-16T00:10:22.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011267199999999999 2024-09-16T00:10:22.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678026+0000", 2024-09-16T00:10:22.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051297089999999997 2024-09-16T00:10:22.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680147+0000", 2024-09-16T00:10:22.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021209409999999999 2024-09-16T00:10:22.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680197+0000", 2024-09-16T00:10:22.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0068000000000002e-05 2024-09-16T00:10:22.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680234+0000", 2024-09-16T00:10:22.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6758000000000003e-05 2024-09-16T00:10:22.409 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.409 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.409 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.409 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4127 2.18 2:1cc859d9:::200.00000012:head [write 808688~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.409 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:22.409 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.713646525000001, 2024-09-16T00:10:22.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019226493000000001, 2024-09-16T00:10:22.409 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.409 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.410 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.410 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:22.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:22.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661186+0000", 2024-09-16T00:10:22.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2399999999999996e-07 2024-09-16T00:10:22.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661192+0000", 2024-09-16T00:10:22.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9259999999999997e-06 2024-09-16T00:10:22.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661203+0000", 2024-09-16T00:10:22.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1174e-05 2024-09-16T00:10:22.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661213+0000", 2024-09-16T00:10:22.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0579e-05 2024-09-16T00:10:22.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673408+0000", 2024-09-16T00:10:22.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012194442 2024-09-16T00:10:22.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673482+0000", 2024-09-16T00:10:22.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4258000000000006e-05 2024-09-16T00:10:22.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673592+0000", 2024-09-16T00:10:22.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109733 2024-09-16T00:10:22.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678033+0000", 2024-09-16T00:10:22.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044411629999999997 2024-09-16T00:10:22.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680287+0000", 2024-09-16T00:10:22.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022538469999999998 2024-09-16T00:10:22.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680367+0000", 2024-09-16T00:10:22.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9808999999999994e-05 2024-09-16T00:10:22.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.414 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.414 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680411+0000", 2024-09-16T00:10:22.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4638000000000002e-05 2024-09-16T00:10:22.415 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.415 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.415 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.415 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4128 2.18 2:1cc859d9:::200.00000012:head [write 810933~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.415 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:22.415 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.713227394, 2024-09-16T00:10:22.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018982046999999998, 2024-09-16T00:10:22.415 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.415 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.415 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.415 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:22.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:22.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661605+0000", 2024-09-16T00:10:22.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.031e-06 2024-09-16T00:10:22.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661610+0000", 2024-09-16T00:10:22.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5010000000000004e-06 2024-09-16T00:10:22.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661623+0000", 2024-09-16T00:10:22.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2286000000000001e-05 2024-09-16T00:10:22.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661630+0000", 2024-09-16T00:10:22.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000002e-06 2024-09-16T00:10:22.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674011+0000", 2024-09-16T00:10:22.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012380923 2024-09-16T00:10:22.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674092+0000", 2024-09-16T00:10:22.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1267999999999995e-05 2024-09-16T00:10:22.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.420 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674216+0000", 2024-09-16T00:10:22.420 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012331700000000001 2024-09-16T00:10:22.420 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.420 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.420 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678039+0000", 2024-09-16T00:10:22.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038232489999999999 2024-09-16T00:10:22.421 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.421 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.421 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.421 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680493+0000", 2024-09-16T00:10:22.421 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024543429999999999 2024-09-16T00:10:22.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680546+0000", 2024-09-16T00:10:22.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3239000000000003e-05 2024-09-16T00:10:22.422 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.422 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.422 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.422 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680586+0000", 2024-09-16T00:10:22.422 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9496999999999998e-05 2024-09-16T00:10:22.422 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.422 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.422 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.423 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.423 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4129 2.18 2:1cc859d9:::200.00000012:head [write 813178~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.423 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:22.423 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.712870243000001, 2024-09-16T00:10:22.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018785932000000002, 2024-09-16T00:10:22.423 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.423 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.423 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.423 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.423 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.423 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.423 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:22.423 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:22.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.424 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661962+0000", 2024-09-16T00:10:22.424 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-07 2024-09-16T00:10:22.424 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.424 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.424 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661966+0000", 2024-09-16T00:10:22.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-06 2024-09-16T00:10:22.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.662015+0000", 2024-09-16T00:10:22.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8364e-05 2024-09-16T00:10:22.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.425 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.425 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.425 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.662023+0000", 2024-09-16T00:10:22.425 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2090000000000005e-06 2024-09-16T00:10:22.425 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674744+0000", 2024-09-16T00:10:22.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012721423000000001 2024-09-16T00:10:22.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674825+0000", 2024-09-16T00:10:22.426 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1036000000000001e-05 2024-09-16T00:10:22.426 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.426 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.426 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.426 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674946+0000", 2024-09-16T00:10:22.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120727 2024-09-16T00:10:22.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678045+0000", 2024-09-16T00:10:22.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030993280000000002 2024-09-16T00:10:22.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.427 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680646+0000", 2024-09-16T00:10:22.427 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026005300000000002 2024-09-16T00:10:22.427 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.427 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.427 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680705+0000", 2024-09-16T00:10:22.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9141000000000002e-05 2024-09-16T00:10:22.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.428 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.428 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680747+0000", 2024-09-16T00:10:22.428 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2064000000000002e-05 2024-09-16T00:10:22.428 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.428 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.428 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.428 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.428 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.428 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4184 2.18 2:1cc859d9:::200.00000012:head [write 927308~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.429 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:22.429 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.668821648000002, 2024-09-16T00:10:22.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023503514999999999, 2024-09-16T00:10:22.429 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.429 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.429 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.429 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.429 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.429 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.429 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:22.429 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.429 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:22.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706011+0000", 2024-09-16T00:10:22.430 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8400000000000003e-07 2024-09-16T00:10:22.430 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.430 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.430 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.430 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706016+0000", 2024-09-16T00:10:22.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7479999999999999e-06 2024-09-16T00:10:22.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706028+0000", 2024-09-16T00:10:22.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1477e-05 2024-09-16T00:10:22.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.431 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706035+0000", 2024-09-16T00:10:22.431 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1509999999999998e-06 2024-09-16T00:10:22.431 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.431 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.431 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.719820+0000", 2024-09-16T00:10:22.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013785168 2024-09-16T00:10:22.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.719888+0000", 2024-09-16T00:10:22.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7631000000000004e-05 2024-09-16T00:10:22.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.432 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.432 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.432 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720003+0000", 2024-09-16T00:10:22.432 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114849 2024-09-16T00:10:22.432 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725150+0000", 2024-09-16T00:10:22.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051469970000000004 2024-09-16T00:10:22.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729386+0000", 2024-09-16T00:10:22.433 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042368529999999996 2024-09-16T00:10:22.433 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.433 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.433 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.433 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729461+0000", 2024-09-16T00:10:22.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4935e-05 2024-09-16T00:10:22.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.434 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.434 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729513+0000", 2024-09-16T00:10:22.434 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1822e-05 2024-09-16T00:10:22.434 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.434 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.434 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.434 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.434 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.434 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4185 2.18 2:1cc859d9:::200.00000012:head [write 929217~3264 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.434 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:22.435 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.667815468000001, 2024-09-16T00:10:22.435 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023998430000000001, 2024-09-16T00:10:22.435 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.435 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.435 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.435 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.436 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.436 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.436 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:22.436 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.436 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.437 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.437 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.437 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:22.437 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.437 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.438 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.438 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707017+0000", 2024-09-16T00:10:22.438 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9999999999999996e-07 2024-09-16T00:10:22.438 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.438 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.439 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.439 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707023+0000", 2024-09-16T00:10:22.439 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1789999999999996e-06 2024-09-16T00:10:22.439 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.439 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.440 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707035+0000", 2024-09-16T00:10:22.440 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2106000000000001e-05 2024-09-16T00:10:22.440 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.440 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.440 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707042+0000", 2024-09-16T00:10:22.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2960000000000001e-06 2024-09-16T00:10:22.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720503+0000", 2024-09-16T00:10:22.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013460952999999999 2024-09-16T00:10:22.441 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.441 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.441 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.441 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720583+0000", 2024-09-16T00:10:22.441 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0171000000000002e-05 2024-09-16T00:10:22.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720668+0000", 2024-09-16T00:10:22.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4128999999999994e-05 2024-09-16T00:10:22.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.442 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725193+0000", 2024-09-16T00:10:22.442 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045257700000000001 2024-09-16T00:10:22.442 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.442 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.442 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.730924+0000", 2024-09-16T00:10:22.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057307690000000001 2024-09-16T00:10:22.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.730972+0000", 2024-09-16T00:10:22.443 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7756000000000002e-05 2024-09-16T00:10:22.443 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.443 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.443 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.443 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731014+0000", 2024-09-16T00:10:22.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2401000000000003e-05 2024-09-16T00:10:22.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.444 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.444 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.444 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.444 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.444 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4186 2.18 2:1cc859d9:::200.00000012:head [write 932481~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.444 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:22.444 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.667227311000001, 2024-09-16T00:10:22.444 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023544934999999999, 2024-09-16T00:10:22.444 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.444 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.444 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.445 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:22.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.445 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:22.445 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.445 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.445 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.445 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707605+0000", 2024-09-16T00:10:22.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1500000000000003e-07 2024-09-16T00:10:22.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707610+0000", 2024-09-16T00:10:22.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7049999999999998e-06 2024-09-16T00:10:22.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.446 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.446 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.446 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707618+0000", 2024-09-16T00:10:22.446 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6149999999999997e-06 2024-09-16T00:10:22.446 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707623+0000", 2024-09-16T00:10:22.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0660000000000003e-06 2024-09-16T00:10:22.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.447 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.447 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721143+0000", 2024-09-16T00:10:22.447 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01351976 2024-09-16T00:10:22.447 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.447 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721221+0000", 2024-09-16T00:10:22.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7918999999999997e-05 2024-09-16T00:10:22.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.448 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721335+0000", 2024-09-16T00:10:22.448 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113781 2024-09-16T00:10:22.448 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.448 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.448 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725236+0000", 2024-09-16T00:10:22.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039010379999999999 2024-09-16T00:10:22.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731025+0000", 2024-09-16T00:10:22.449 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057896659999999997 2024-09-16T00:10:22.449 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.449 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.449 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.449 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731091+0000", 2024-09-16T00:10:22.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.567e-05 2024-09-16T00:10:22.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.450 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.450 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731149+0000", 2024-09-16T00:10:22.450 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7800000000000002e-05 2024-09-16T00:10:22.450 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.450 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.450 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.450 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.450 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.451 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4187 2.18 2:1cc859d9:::200.00000012:head [write 934390~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.451 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:22.451 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.666375181999999, 2024-09-16T00:10:22.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022820340000000001, 2024-09-16T00:10:22.451 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.451 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.451 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.451 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.451 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.451 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.451 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:22.451 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.451 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:22.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.452 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.452 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708457+0000", 2024-09-16T00:10:22.452 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6899999999999996e-07 2024-09-16T00:10:22.452 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.452 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708462+0000", 2024-09-16T00:10:22.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3090000000000003e-06 2024-09-16T00:10:22.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708472+0000", 2024-09-16T00:10:22.453 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7499999999999998e-06 2024-09-16T00:10:22.453 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.453 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.453 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.453 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708479+0000", 2024-09-16T00:10:22.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1189999999999999e-06 2024-09-16T00:10:22.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721895+0000", 2024-09-16T00:10:22.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013415664000000001 2024-09-16T00:10:22.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.454 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.454 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.454 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721941+0000", 2024-09-16T00:10:22.454 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6031e-05 2024-09-16T00:10:22.454 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722007+0000", 2024-09-16T00:10:22.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6503999999999999e-05 2024-09-16T00:10:22.455 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.455 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.455 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.455 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725277+0000", 2024-09-16T00:10:22.455 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003269554 2024-09-16T00:10:22.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.456 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.456 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.456 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731158+0000", 2024-09-16T00:10:22.456 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058813750000000003 2024-09-16T00:10:22.456 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.457 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.457 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.457 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731240+0000", 2024-09-16T00:10:22.457 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1815000000000002e-05 2024-09-16T00:10:22.457 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.458 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.458 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731276+0000", 2024-09-16T00:10:22.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6449999999999998e-05 2024-09-16T00:10:22.458 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.458 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.458 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.458 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.458 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.458 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4188 2.18 2:1cc859d9:::200.00000012:head [write 936299~2419 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.458 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:22.458 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.665954239000001, 2024-09-16T00:10:22.458 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022518565000000001, 2024-09-16T00:10:22.458 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.459 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.459 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.459 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:22.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.459 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.459 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.459 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:22.459 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.459 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708878+0000", 2024-09-16T00:10:22.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8800000000000001e-07 2024-09-16T00:10:22.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.460 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708884+0000", 2024-09-16T00:10:22.460 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9800000000000003e-06 2024-09-16T00:10:22.460 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.460 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.460 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708895+0000", 2024-09-16T00:10:22.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1416e-05 2024-09-16T00:10:22.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708903+0000", 2024-09-16T00:10:22.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4449999999999999e-06 2024-09-16T00:10:22.461 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.461 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.461 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.461 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722327+0000", 2024-09-16T00:10:22.461 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013424681000000001 2024-09-16T00:10:22.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722408+0000", 2024-09-16T00:10:22.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0092000000000005e-05 2024-09-16T00:10:22.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.462 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722538+0000", 2024-09-16T00:10:22.462 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013046600000000001 2024-09-16T00:10:22.462 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.462 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.462 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726665+0000", 2024-09-16T00:10:22.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041267420000000001 2024-09-16T00:10:22.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731284+0000", 2024-09-16T00:10:22.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046195469999999999 2024-09-16T00:10:22.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.463 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.463 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731358+0000", 2024-09-16T00:10:22.463 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4002000000000001e-05 2024-09-16T00:10:22.463 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.463 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.464 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.464 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731396+0000", 2024-09-16T00:10:22.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7305999999999998e-05 2024-09-16T00:10:22.464 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.464 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.464 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.464 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.464 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.464 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4189 2.18 2:1cc859d9:::200.00000012:head [write 938718~1910 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.464 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:22.464 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.665047123000001, 2024-09-16T00:10:22.464 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021703626, 2024-09-16T00:10:22.464 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.465 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.465 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.465 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:22.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.465 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.465 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.465 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.465 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:22.465 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709785+0000", 2024-09-16T00:10:22.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.043e-06 2024-09-16T00:10:22.466 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.466 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.466 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.466 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709791+0000", 2024-09-16T00:10:22.466 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9499999999999998e-06 2024-09-16T00:10:22.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.467 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709804+0000", 2024-09-16T00:10:22.467 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2958e-05 2024-09-16T00:10:22.467 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.467 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.467 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.468 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709812+0000", 2024-09-16T00:10:22.468 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9869999999999999e-06 2024-09-16T00:10:22.468 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.468 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.468 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.469 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723058+0000", 2024-09-16T00:10:22.469 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013245497 2024-09-16T00:10:22.469 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.469 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.470 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723131+0000", 2024-09-16T00:10:22.470 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3879000000000002e-05 2024-09-16T00:10:22.470 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.470 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.470 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723241+0000", 2024-09-16T00:10:22.471 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109435 2024-09-16T00:10:22.471 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.471 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.471 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.471 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726708+0000", 2024-09-16T00:10:22.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034671340000000002 2024-09-16T00:10:22.472 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.472 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.472 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.472 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731403+0000", 2024-09-16T00:10:22.472 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046953979999999999 2024-09-16T00:10:22.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.473 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.473 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.473 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731454+0000", 2024-09-16T00:10:22.473 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0747e-05 2024-09-16T00:10:22.473 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.474 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.474 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.474 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731488+0000", 2024-09-16T00:10:22.474 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3597999999999999e-05 2024-09-16T00:10:22.474 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.474 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.475 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.475 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.475 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.475 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4190 2.18 2:1cc859d9:::200.00000012:head [write 940628~2420 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.475 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:22.475 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.664574957999999, 2024-09-16T00:10:22.476 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021346812999999999, 2024-09-16T00:10:22.476 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.476 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.476 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.476 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.476 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.477 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.477 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:22.477 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.477 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.477 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.478 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:22.478 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.478 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.478 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.478 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710257+0000", 2024-09-16T00:10:22.479 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8100000000000001e-07 2024-09-16T00:10:22.479 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.479 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.479 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.479 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710263+0000", 2024-09-16T00:10:22.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8050000000000003e-06 2024-09-16T00:10:22.480 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.480 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.480 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.480 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710273+0000", 2024-09-16T00:10:22.480 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0005e-05 2024-09-16T00:10:22.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.481 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.481 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.481 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710280+0000", 2024-09-16T00:10:22.481 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3819999999999998e-06 2024-09-16T00:10:22.481 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.482 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.482 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723653+0000", 2024-09-16T00:10:22.482 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013372866000000001 2024-09-16T00:10:22.482 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.482 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.483 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723733+0000", 2024-09-16T00:10:22.483 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9396999999999999e-05 2024-09-16T00:10:22.483 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.483 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.483 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723852+0000", 2024-09-16T00:10:22.484 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011919899999999999 2024-09-16T00:10:22.484 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.484 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.484 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.484 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726752+0000", 2024-09-16T00:10:22.485 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028999659999999999 2024-09-16T00:10:22.485 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.485 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.485 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.485 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731495+0000", 2024-09-16T00:10:22.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047432209999999997 2024-09-16T00:10:22.486 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.486 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.486 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.486 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731568+0000", 2024-09-16T00:10:22.486 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2452000000000004e-05 2024-09-16T00:10:22.487 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.487 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.487 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.487 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731603+0000", 2024-09-16T00:10:22.487 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5639000000000002e-05 2024-09-16T00:10:22.487 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.488 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.488 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.488 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.488 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.488 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4192 2.18 2:1cc859d9:::200.00000012:head [write 943048~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.488 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:22.489 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.663842118000002, 2024-09-16T00:10:22.489 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021688433999999999, 2024-09-16T00:10:22.489 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.489 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.489 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.489 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:22.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.490 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.490 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:22.490 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.490 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.490 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710990+0000", 2024-09-16T00:10:22.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1999999999999998e-07 2024-09-16T00:10:22.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710996+0000", 2024-09-16T00:10:22.491 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:10:22.491 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.491 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.491 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.491 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711007+0000", 2024-09-16T00:10:22.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1471000000000001e-05 2024-09-16T00:10:22.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711014+0000", 2024-09-16T00:10:22.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7780000000000002e-06 2024-09-16T00:10:22.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.492 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.492 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724471+0000", 2024-09-16T00:10:22.492 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013457435 2024-09-16T00:10:22.492 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.492 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724547+0000", 2024-09-16T00:10:22.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5383999999999995e-05 2024-09-16T00:10:22.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724663+0000", 2024-09-16T00:10:22.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116348 2024-09-16T00:10:22.493 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.493 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.493 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.493 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731612+0000", 2024-09-16T00:10:22.493 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069491409999999998 2024-09-16T00:10:22.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732573+0000", 2024-09-16T00:10:22.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0009611 2024-09-16T00:10:22.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.494 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732638+0000", 2024-09-16T00:10:22.494 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919000000000006e-05 2024-09-16T00:10:22.494 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.494 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.494 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.495 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732678+0000", 2024-09-16T00:10:22.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9230999999999998e-05 2024-09-16T00:10:22.495 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.495 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.495 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.495 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.495 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.495 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4193 2.18 2:1cc859d9:::200.00000012:head [write 944447~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.495 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:22.495 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.663296410000001, 2024-09-16T00:10:22.495 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021286851999999998, 2024-09-16T00:10:22.495 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.495 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.495 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.496 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711536+0000", 2024-09-16T00:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0189999999999999e-06 2024-09-16T00:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711542+0000", 2024-09-16T00:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7169999999999996e-06 2024-09-16T00:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711554+0000", 2024-09-16T00:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2089000000000001e-05 2024-09-16T00:10:22.497 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711559+0000", 2024-09-16T00:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3739999999999997e-06 2024-09-16T00:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725311+0000", 2024-09-16T00:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013751559 2024-09-16T00:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725390+0000", 2024-09-16T00:10:22.498 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9779999999999998e-05 2024-09-16T00:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725509+0000", 2024-09-16T00:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118973 2024-09-16T00:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731617+0000", 2024-09-16T00:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061079000000000003 2024-09-16T00:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.499 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732762+0000", 2024-09-16T00:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011449769999999999 2024-09-16T00:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732796+0000", 2024-09-16T00:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3587999999999998e-05 2024-09-16T00:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732822+0000", 2024-09-16T00:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5876000000000001e-05 2024-09-16T00:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.500 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4194 2.18 2:1cc859d9:::200.00000012:head [write 946356~2929 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.662321379000002, 2024-09-16T00:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020453244999999998, 2024-09-16T00:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.501 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712511+0000", 2024-09-16T00:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0999999999999997e-07 2024-09-16T00:10:22.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712517+0000", 2024-09-16T00:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8370000000000002e-06 2024-09-16T00:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712528+0000", 2024-09-16T00:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1528e-05 2024-09-16T00:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712536+0000", 2024-09-16T00:10:22.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3939999999999998e-06 2024-09-16T00:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726004+0000", 2024-09-16T00:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013468177 2024-09-16T00:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726085+0000", 2024-09-16T00:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1626999999999996e-05 2024-09-16T00:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726205+0000", 2024-09-16T00:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119751 2024-09-16T00:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731622+0000", 2024-09-16T00:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054165419999999999 2024-09-16T00:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732867+0000", 2024-09-16T00:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001244855 2024-09-16T00:10:22.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732924+0000", 2024-09-16T00:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7395999999999997e-05 2024-09-16T00:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732963+0000", 2024-09-16T00:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9227999999999998e-05 2024-09-16T00:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4195 2.18 2:1cc859d9:::200.00000012:head [write 949285~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.661269719, 2024-09-16T00:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019498099000000001, 2024-09-16T00:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713563+0000", 2024-09-16T00:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T00:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713569+0000", 2024-09-16T00:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8710000000000002e-06 2024-09-16T00:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713582+0000", 2024-09-16T00:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3134e-05 2024-09-16T00:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713591+0000", 2024-09-16T00:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9889999999999998e-06 2024-09-16T00:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726786+0000", 2024-09-16T00:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013194991999999999 2024-09-16T00:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726856+0000", 2024-09-16T00:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0017000000000004e-05 2024-09-16T00:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726967+0000", 2024-09-16T00:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111815 2024-09-16T00:10:22.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732530+0000", 2024-09-16T00:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055624940000000003 2024-09-16T00:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733001+0000", 2024-09-16T00:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000470817 2024-09-16T00:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733030+0000", 2024-09-16T00:10:22.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9051e-05 2024-09-16T00:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733060+0000", 2024-09-16T00:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9819e-05 2024-09-16T00:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4215 2.18 2:1cc859d9:::200.00000012:head [write 991028~1910 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:22.512 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.641574007999999, 2024-09-16T00:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019416676000000001, 2024-09-16T00:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:22.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733258+0000", 2024-09-16T00:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733264+0000", 2024-09-16T00:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4090000000000002e-06 2024-09-16T00:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733273+0000", 2024-09-16T00:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4769999999999992e-06 2024-09-16T00:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733280+0000", 2024-09-16T00:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2989999999999999e-06 2024-09-16T00:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.742966+0000", 2024-09-16T00:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009685585 2024-09-16T00:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743046+0000", 2024-09-16T00:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0469999999999994e-05 2024-09-16T00:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743165+0000", 2024-09-16T00:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118847 2024-09-16T00:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.746968+0000", 2024-09-16T00:10:22.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038025099999999998 2024-09-16T00:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752559+0000", 2024-09-16T00:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055911290000000002 2024-09-16T00:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752634+0000", 2024-09-16T00:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5102e-05 2024-09-16T00:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752674+0000", 2024-09-16T00:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9878e-05 2024-09-16T00:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4216 2.18 2:1cc859d9:::200.00000012:head [write 992938~2755 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 23.6410281, 2024-09-16T00:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019035976, 2024-09-16T00:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:22.518 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:22.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733804+0000", 2024-09-16T00:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2299999999999999e-07 2024-09-16T00:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733810+0000", 2024-09-16T00:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7259999999999999e-06 2024-09-16T00:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733822+0000", 2024-09-16T00:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2204e-05 2024-09-16T00:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733829+0000", 2024-09-16T00:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3889999999999999e-06 2024-09-16T00:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743675+0000", 2024-09-16T00:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098457990000000006 2024-09-16T00:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743752+0000", 2024-09-16T00:10:22.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6279000000000006e-05 2024-09-16T00:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743867+0000", 2024-09-16T00:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115319 2024-09-16T00:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.746971+0000", 2024-09-16T00:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031042399999999999 2024-09-16T00:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:22.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752723+0000", 2024-09-16T00:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057518329999999996 2024-09-16T00:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752804+0000", 2024-09-16T00:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0822999999999995e-05 2024-09-16T00:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752839+0000", 2024-09-16T00:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5441000000000002e-05 2024-09-16T00:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.524 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:22.524 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:22.524 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:22.526 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:10:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.401+0000 7f0a2c02e640 1 Processor -- start 2024-09-16T00:10:22.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.401+0000 7f0a2c02e640 1 -- start start 2024-09-16T00:10:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.402+0000 7f0a2c02e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0a241517a0 0x7f0a240c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.402+0000 7f0a2c02e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a24151b80 0x7f0a240c2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.402+0000 7f0a2c02e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0a240c9280 0x7f0a240c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.402+0000 7f0a2c02e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0a240737f0 con 0x7f0a240c9280 2024-09-16T00:10:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.402+0000 7f0a2c02e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f0a240c13c0 con 0x7f0a241517a0 2024-09-16T00:10:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.402+0000 7f0a2c02e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0a240c2440 con 0x7f0a24151b80 2024-09-16T00:10:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.402+0000 7f0a2a5cb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0a240c9280 0x7f0a240c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.402+0000 7f0a2a5cb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0a240c9280 0x7f0a240c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52128/0 (socket says 172.21.15.139:52128) 2024-09-16T00:10:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.402+0000 7f0a2a5cb640 1 -- 172.21.15.139:0/2835598001 learned_addr learned my addr 172.21.15.139:0/2835598001 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.402+0000 7f0a295c9640 1 --2- 172.21.15.139:0/2835598001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a24151b80 0x7f0a240c2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.402+0000 7f0a29dca640 1 --2- 172.21.15.139:0/2835598001 >> v2:172.21.15.164:3300/0 conn(0x7f0a241517a0 0x7f0a240c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.402+0000 7f0a2a5cb640 1 -- 172.21.15.139:0/2835598001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a24151b80 msgr2=0x7f0a240c2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:22.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.402+0000 7f0a2a5cb640 1 --2- 172.21.15.139:0/2835598001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a24151b80 0x7f0a240c2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.402+0000 7f0a2a5cb640 1 -- 172.21.15.139:0/2835598001 >> v2:172.21.15.164:3300/0 conn(0x7f0a241517a0 msgr2=0x7f0a240c1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.402+0000 7f0a2a5cb640 1 --2- 172.21.15.139:0/2835598001 >> v2:172.21.15.164:3300/0 conn(0x7f0a241517a0 0x7f0a240c1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.402+0000 7f0a2a5cb640 1 -- 172.21.15.139:0/2835598001 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a240c9730 con 0x7f0a240c9280 2024-09-16T00:10:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.403+0000 7f0a2a5cb640 1 --2- 172.21.15.139:0/2835598001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240c9280 0x7f0a240c7c90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0a240c8770 tx=0x7f0a1c02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb19cb0b889b487 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.403+0000 7f0a28dc8640 1 -- 172.21.15.139:0/2835598001 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c036070 con 0x7f0a240c9280 2024-09-16T00:10:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.403+0000 7f0a28dc8640 1 -- 172.21.15.139:0/2835598001 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a1c031600 con 0x7f0a240c9280 2024-09-16T00:10:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.403+0000 7f0a28dc8640 1 -- 172.21.15.139:0/2835598001 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c0319d0 con 0x7f0a240c9280 2024-09-16T00:10:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.404+0000 7f0a2c02e640 1 -- 172.21.15.139:0/2835598001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240c9280 msgr2=0x7f0a240c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.404+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/2835598001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240c9280 0x7f0a240c7c90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0a240c8770 tx=0x7f0a1c02a750 comp rx=0 tx=0).stop 2024-09-16T00:10:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.404+0000 7f0a2c02e640 1 -- 172.21.15.139:0/2835598001 shutdown_connections 2024-09-16T00:10:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.404+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/2835598001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240c9280 0x7f0a240c7c90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.404+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/2835598001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a24151b80 0x7f0a240c2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.404+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/2835598001 >> v2:172.21.15.164:3300/0 conn(0x7f0a241517a0 0x7f0a240c1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.404+0000 7f0a2c02e640 1 -- 172.21.15.139:0/2835598001 >> 172.21.15.139:0/2835598001 conn(0x7f0a2406e960 msgr2=0x7f0a2406f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:22.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.404+0000 7f0a2c02e640 1 -- 172.21.15.139:0/2835598001 shutdown_connections 2024-09-16T00:10:22.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.404+0000 7f0a2c02e640 1 -- 172.21.15.139:0/2835598001 wait complete. 2024-09-16T00:10:22.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.405+0000 7f0a2c02e640 1 Processor -- start 2024-09-16T00:10:22.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.405+0000 7f0a2c02e640 1 -- start start 2024-09-16T00:10:22.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.405+0000 7f0a2c02e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240c9280 0x7f0a241e3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:22.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.405+0000 7f0a2c02e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a241517a0 0x7f0a241503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:22.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.405+0000 7f0a2c02e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a24151b80 0x7f0a240ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:22.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.405+0000 7f0a2c02e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0a24073a20 con 0x7f0a240c9280 2024-09-16T00:10:22.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.405+0000 7f0a2c02e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0a240c18f0 con 0x7f0a241517a0 2024-09-16T00:10:22.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.405+0000 7f0a2c02e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0a240c3310 con 0x7f0a24151b80 2024-09-16T00:10:22.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.406+0000 7f0a29dca640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240c9280 0x7f0a241e3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:22.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.406+0000 7f0a29dca640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240c9280 0x7f0a241e3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52144/0 (socket says 172.21.15.139:52144) 2024-09-16T00:10:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.406+0000 7f0a29dca640 1 -- 172.21.15.139:0/1824370269 learned_addr learned my addr 172.21.15.139:0/1824370269 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.406+0000 7f0a295c9640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a241517a0 0x7f0a241503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.406+0000 7f0a2a5cb640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a24151b80 0x7f0a240ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.406+0000 7f0a29dca640 1 -- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a24151b80 msgr2=0x7f0a240ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.406+0000 7f0a29dca640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0a24151b80 0x7f0a240ca880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.406+0000 7f0a29dca640 1 -- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a241517a0 msgr2=0x7f0a241503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.406+0000 7f0a29dca640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0a241517a0 0x7f0a241503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.406+0000 7f0a29dca640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a241e37b0 con 0x7f0a240c9280 2024-09-16T00:10:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.406+0000 7f0a29dca640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240c9280 0x7f0a241e3270 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000e8c0 tx=0x7f0a2000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.406+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a20017070 con 0x7f0a240c9280 2024-09-16T00:10:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.406+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a2000c940 con 0x7f0a240c9280 2024-09-16T00:10:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.406+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a241dfe30 con 0x7f0a240c9280 2024-09-16T00:10:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.406+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a2000cd10 con 0x7f0a240c9280 2024-09-16T00:10:22.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.406+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a241e03b0 con 0x7f0a240c9280 2024-09-16T00:10:22.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.407+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f0a240aec50 con 0x7f0a240c9280 2024-09-16T00:10:22.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.408+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0a2001a070 con 0x7f0a240c9280 2024-09-16T00:10:22.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.408+0000 7f0a1a7fc640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a0806c2e0 0x7f0a0806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:22.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.408+0000 7f0a295c9640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a0806c2e0 0x7f0a0806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:22.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.408+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5973+0+0 (secure 0 0 0) 0x7f0a2008b760 con 0x7f0a240c9280 2024-09-16T00:10:22.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.408+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v111) ==== 54+0+16 (secure 0 0 0) 0x7f0a20091170 con 0x7f0a240c9280 2024-09-16T00:10:22.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.408+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f0a241e1570 0x7f0a24145330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:22.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.409+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a241e2c00 con 0x7f0a241e1570 2024-09-16T00:10:22.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.409+0000 7f0a295c9640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a0806c2e0 0x7f0a0806e7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0a240cd160 tx=0x7f0a14009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:22.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.409+0000 7f0a2a5cb640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f0a241e1570 0x7f0a24145330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:22.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.409+0000 7f0a2a5cb640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f0a241e1570 0x7f0a24145330 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:22.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.410+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a1c037040 con 0x7f0a241e1570 2024-09-16T00:10:22.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.425+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0a240c83e0 con 0x7f0a241e1570 2024-09-16T00:10:22.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.426+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0a240c83e0 con 0x7f0a241e1570 2024-09-16T00:10:22.536 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:22.537 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:22.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.427+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f0a240cb460 0x7f0a241dec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:22.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.427+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f0a240ae6b0 con 0x7f0a240cb460 2024-09-16T00:10:22.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.427+0000 7f0a29dca640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f0a240cb460 0x7f0a241dec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:22.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.427+0000 7f0a29dca640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f0a240cb460 0x7f0a241dec50 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:22.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.428+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a240ae6b0 con 0x7f0a240cb460 2024-09-16T00:10:22.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.442+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0a240c11a0 con 0x7f0a240cb460 2024-09-16T00:10:22.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.444+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0a240c11a0 con 0x7f0a240cb460 2024-09-16T00:10:22.538 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:22.539 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:22.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.444+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f0a240cadc0 0x7f0a2413e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:22.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.444+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f0a2413ee00 con 0x7f0a240cadc0 2024-09-16T00:10:22.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.444+0000 7f0a295c9640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f0a240cadc0 0x7f0a2413e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:22.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.445+0000 7f0a295c9640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f0a240cadc0 0x7f0a2413e8c0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:22.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.445+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a2413ee00 con 0x7f0a240cadc0 2024-09-16T00:10:22.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.459+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0a2413f280 con 0x7f0a240cadc0 2024-09-16T00:10:22.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.461+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0a2413f280 con 0x7f0a240cadc0 2024-09-16T00:10:22.540 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:22.541 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:22.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.461+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0a24062b40 0x7f0a24062f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:22.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.461+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f0a240634c0 con 0x7f0a24062b40 2024-09-16T00:10:22.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.461+0000 7f0a2a5cb640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0a24062b40 0x7f0a24062f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:22.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.461+0000 7f0a2a5cb640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0a24062b40 0x7f0a24062f80 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:22.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.462+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a240634c0 con 0x7f0a24062b40 2024-09-16T00:10:22.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.476+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0a2414f210 con 0x7f0a24062b40 2024-09-16T00:10:22.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.478+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0a2414f210 con 0x7f0a24062b40 2024-09-16T00:10:22.542 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:22.542 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:22.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.478+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f0a2414f930 0x7f0a2402c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:22.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.478+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f0a240634c0 con 0x7f0a2414f930 2024-09-16T00:10:22.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.478+0000 7f0a29dca640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f0a2414f930 0x7f0a2402c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:22.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.478+0000 7f0a29dca640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f0a2414f930 0x7f0a2402c640 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:22.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.479+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a240634c0 con 0x7f0a2414f930 2024-09-16T00:10:22.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.493+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0a240634c0 con 0x7f0a2414f930 2024-09-16T00:10:22.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.494+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0a240634c0 con 0x7f0a2414f930 2024-09-16T00:10:22.543 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:22.544 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:22.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.495+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f0a2414fd50 0x7f0a2402d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:22.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.495+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f0a2402d7b0 con 0x7f0a2414fd50 2024-09-16T00:10:22.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.495+0000 7f0a295c9640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f0a2414fd50 0x7f0a2402d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:22.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.495+0000 7f0a295c9640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f0a2414fd50 0x7f0a2402d090 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:22.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.496+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a2402d7b0 con 0x7f0a2414fd50 2024-09-16T00:10:22.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.510+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0a2402d9f0 con 0x7f0a2414fd50 2024-09-16T00:10:22.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.511+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0a2402d9f0 con 0x7f0a2414fd50 2024-09-16T00:10:22.545 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:22.546 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:22.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.512+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f0a24204550 0x7f0a24206940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:22.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.512+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f0a24209710 con 0x7f0a24204550 2024-09-16T00:10:22.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.512+0000 7f0a2a5cb640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f0a24204550 0x7f0a24206940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:22.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.512+0000 7f0a2a5cb640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f0a24204550 0x7f0a24206940 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:22.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.513+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a24209710 con 0x7f0a24204550 2024-09-16T00:10:22.546 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:22.523+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:22.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.527+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0a24206e80 con 0x7f0a24204550 2024-09-16T00:10:22.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.529+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0a24206e80 con 0x7f0a24204550 2024-09-16T00:10:22.547 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:22.548 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:22.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.530+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f0a242075a0 0x7f0a24207a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:22.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.530+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f0a2420ca80 con 0x7f0a242075a0 2024-09-16T00:10:22.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.530+0000 7f0a29dca640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f0a242075a0 0x7f0a24207a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:22.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.530+0000 7f0a29dca640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f0a242075a0 0x7f0a24207a00 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:22.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.531+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a2420ca80 con 0x7f0a242075a0 2024-09-16T00:10:22.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.545+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f0a24207f90 con 0x7f0a242075a0 2024-09-16T00:10:22.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a1a7fc640 1 -- 172.21.15.139:0/1824370269 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f0a24207f90 con 0x7f0a242075a0 2024-09-16T00:10:22.549 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:10:22.551 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:10:22.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f0a241e1570 msgr2=0x7f0a24145330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:22.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f0a241e1570 0x7f0a24145330 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f0a240cb460 msgr2=0x7f0a241dec50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:22.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f0a240cb460 0x7f0a241dec50 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f0a240cadc0 msgr2=0x7f0a2413e8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:22.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f0a240cadc0 0x7f0a2413e8c0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0a24062b40 msgr2=0x7f0a24062f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:22.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f0a24062b40 0x7f0a24062f80 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.551 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f0a2414f930 msgr2=0x7f0a2402c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:22.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f0a2414f930 0x7f0a2402c640 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f0a2414fd50 msgr2=0x7f0a2402d090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:22.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f0a2414fd50 0x7f0a2402d090 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2a5cb640 1 -- 172.21.15.139:0/1824370269 reap_dead start 2024-09-16T00:10:22.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f0a24204550 msgr2=0x7f0a24206940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:22.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f0a24204550 0x7f0a24206940 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2a5cb640 1 -- 172.21.15.139:0/1824370269 reap_dead start 2024-09-16T00:10:22.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f0a242075a0 msgr2=0x7f0a24207a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:22.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f0a242075a0 0x7f0a24207a00 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2a5cb640 1 -- 172.21.15.139:0/1824370269 reap_dead start 2024-09-16T00:10:22.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2a5cb640 1 -- 172.21.15.139:0/1824370269 reap_dead start 2024-09-16T00:10:22.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a0806c2e0 msgr2=0x7f0a0806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:22.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a0806c2e0 0x7f0a0806e7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0a240cd160 tx=0x7f0a14009190 comp rx=0 tx=0).stop 2024-09-16T00:10:22.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240c9280 msgr2=0x7f0a241e3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:22.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240c9280 0x7f0a241e3270 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000e8c0 tx=0x7f0a2000ed80 comp rx=0 tx=0).stop 2024-09-16T00:10:22.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 shutdown_connections 2024-09-16T00:10:22.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0a0806c2e0 0x7f0a0806e7a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 --2- 172.21.15.139:0/1824370269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a240c9280 0x7f0a241e3270 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:22.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.547+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 >> 172.21.15.139:0/1824370269 conn(0x7f0a2406e960 msgr2=0x7f0a24155250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:22.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.548+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 shutdown_connections 2024-09-16T00:10:22.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:22.548+0000 7f0a2c02e640 1 -- 172.21.15.139:0/1824370269 wait complete. 2024-09-16T00:10:22.625 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:22.624+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:22.658 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:22.658 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:22.670 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-16T00:10:22.726 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:22.724+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:22.778 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:22.778 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:22.788 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-16T00:10:22.832 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:22.831+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:22.899 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:22.899 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:22.908 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:10:22.929 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:22.927+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:23.030 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:23.028+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:23.098 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:23.099 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:23.108 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:10:23.151 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:23.150+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:23.216 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:23.216 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:23.224 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:10:23.232 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:23.229+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:23.332 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:23.330+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:23.373 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:23.373 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:23.383 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:10:23.392 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T00:10:23.470 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:23.468+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:23.492 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:23.492 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:23.502 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:10:23.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.528+0000 7fb3eaf6d640 1 Processor -- start 2024-09-16T00:10:23.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.528+0000 7fb3eaf6d640 1 -- start start 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.529+0000 7fb3eaf6d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3e40c60c0 0x7fb3e40c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.529+0000 7fb3eaf6d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb3e40c64a0 0x7fb3e40730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.529+0000 7fb3eaf6d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3e4159660 con 0x7fb3e406d080 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.529+0000 7fb3eaf6d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb3e4159140 con 0x7fb3e40c64a0 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.529+0000 7fb3eaf6d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e4072360 con 0x7fb3e40c60c0 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.529+0000 7fb3e8d09640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3e40c60c0 0x7fb3e40c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.529+0000 7fb3e950a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb3e40c64a0 0x7fb3e40730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.529+0000 7fb3e8d09640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3e40c60c0 0x7fb3e40c7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:53694/0 (socket says 172.21.15.139:53694) 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.529+0000 7fb3e8d09640 1 -- 172.21.15.139:0/3468069445 learned_addr learned my addr 172.21.15.139:0/3468069445 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.529+0000 7fb3e950a640 1 --2- 172.21.15.139:0/3468069445 >> v2:172.21.15.164:3300/0 conn(0x7fb3e40c64a0 0x7fb3e40730c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:60508/0 (socket says 172.21.15.139:60508) 2024-09-16T00:10:23.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.529+0000 7fb3e3fff640 1 --1- 172.21.15.139:0/3468069445 >> v1:172.21.15.164:6789/0 conn(0x7fb3e406d080 0x7fb3e4152590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:38236/0 (socket says 172.21.15.139:38236) 2024-09-16T00:10:23.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.530+0000 7fb3e8d09640 1 -- 172.21.15.139:0/3468069445 >> v2:172.21.15.164:3300/0 conn(0x7fb3e40c64a0 msgr2=0x7fb3e40730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.530+0000 7fb3e8d09640 1 --2- 172.21.15.139:0/3468069445 >> v2:172.21.15.164:3300/0 conn(0x7fb3e40c64a0 0x7fb3e40730c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.530+0000 7fb3e8d09640 1 -- 172.21.15.139:0/3468069445 >> v1:172.21.15.164:6789/0 conn(0x7fb3e406d080 legacy=0x7fb3e4152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.530+0000 7fb3e37fe640 1 -- 172.21.15.139:0/3468069445 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2465352544 0 0) 0x7fb3e4159660 con 0x7fb3e406d080 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.530+0000 7fb3e8d09640 1 -- 172.21.15.139:0/3468069445 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e4067a80 con 0x7fb3e40c60c0 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.530+0000 7fb3e950a640 1 --2- 172.21.15.139:0/3468069445 >> v2:172.21.15.164:3300/0 conn(0x7fb3e40c64a0 0x7fb3e40730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.531+0000 7fb3e8d09640 1 --2- 172.21.15.139:0/3468069445 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3e40c60c0 0x7fb3e40c7140 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009870 tx=0x7fb3dc02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0aa76c6ac109adf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.531+0000 7fb3e37fe640 1 -- 172.21.15.139:0/3468069445 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc037070 con 0x7fb3e40c60c0 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.531+0000 7fb3e37fe640 1 -- 172.21.15.139:0/3468069445 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3dc032440 con 0x7fb3e40c60c0 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.531+0000 7fb3e37fe640 1 -- 172.21.15.139:0/3468069445 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc032810 con 0x7fb3e40c60c0 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.531+0000 7fb3eaf6d640 1 -- 172.21.15.139:0/3468069445 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3e40c60c0 msgr2=0x7fb3e40c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.531+0000 7fb3eaf6d640 1 --2- 172.21.15.139:0/3468069445 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3e40c60c0 0x7fb3e40c7140 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009870 tx=0x7fb3dc02a270 comp rx=0 tx=0).stop 2024-09-16T00:10:23.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.531+0000 7fb3eaf6d640 1 -- 172.21.15.139:0/3468069445 shutdown_connections 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.531+0000 7fb3eaf6d640 1 --2- 172.21.15.139:0/3468069445 >> v2:172.21.15.164:3300/0 conn(0x7fb3e40c64a0 0x7fb3e40730c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.531+0000 7fb3eaf6d640 1 --2- 172.21.15.139:0/3468069445 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3e40c60c0 0x7fb3e40c7140 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.531+0000 7fb3eaf6d640 1 -- 172.21.15.139:0/3468069445 >> 172.21.15.139:0/3468069445 conn(0x7fb3e40cc740 msgr2=0x7fb3e40ccb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.531+0000 7fb3eaf6d640 1 -- 172.21.15.139:0/3468069445 shutdown_connections 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.531+0000 7fb3eaf6d640 1 -- 172.21.15.139:0/3468069445 wait complete. 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.533+0000 7fb3eaf6d640 1 Processor -- start 2024-09-16T00:10:23.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.533+0000 7fb3eaf6d640 1 -- start start 2024-09-16T00:10:23.537 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:23.532+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.533+0000 7fb3eaf6d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3e406d080 0x7fb3e4158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.533+0000 7fb3eaf6d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3e40c60c0 0x7fb3e4148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3eaf6d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e40c64a0 0x7fb3e4157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3eaf6d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e4152ca0 con 0x7fb3e40c64a0 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3eaf6d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e4067d20 con 0x7fb3e40c60c0 2024-09-16T00:10:23.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3eaf6d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb3e4152e20 con 0x7fb3e406d080 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3e3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3e40c60c0 0x7fb3e4148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3e8d09640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3e406d080 0x7fb3e4158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3e950a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e40c64a0 0x7fb3e4157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3e8d09640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3e406d080 0x7fb3e4158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:53710/0 (socket says 172.21.15.139:53710) 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3e3fff640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3e40c60c0 0x7fb3e4148410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:60512/0 (socket says 172.21.15.139:60512) 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3e8d09640 1 -- 172.21.15.139:0/2254632846 learned_addr learned my addr 172.21.15.139:0/2254632846 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3e950a640 1 --2- 172.21.15.139:0/2254632846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e40c64a0 0x7fb3e4157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52152/0 (socket says 172.21.15.139:52152) 2024-09-16T00:10:23.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3e8d09640 1 -- 172.21.15.139:0/2254632846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3e40c60c0 msgr2=0x7fb3e4148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3e8d09640 1 --2- 172.21.15.139:0/2254632846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3e40c60c0 0x7fb3e4148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3e8d09640 1 -- 172.21.15.139:0/2254632846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e40c64a0 msgr2=0x7fb3e4157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3e8d09640 1 --2- 172.21.15.139:0/2254632846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e40c64a0 0x7fb3e4157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3e8d09640 1 -- 172.21.15.139:0/2254632846 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e4158850 con 0x7fb3e406d080 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3e8d09640 1 --2- 172.21.15.139:0/2254632846 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3e406d080 0x7fb3e4158210 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc000c00 tx=0x7fb3dc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3e17fa640 1 -- 172.21.15.139:0/2254632846 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc041070 con 0x7fb3e406d080 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3e17fa640 1 -- 172.21.15.139:0/2254632846 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3dc03c890 con 0x7fb3e406d080 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3eaf6d640 1 -- 172.21.15.139:0/2254632846 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e40c5380 con 0x7fb3e406d080 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.534+0000 7fb3e17fa640 1 -- 172.21.15.139:0/2254632846 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc037040 con 0x7fb3e406d080 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.535+0000 7fb3eaf6d640 1 -- 172.21.15.139:0/2254632846 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e40c5940 con 0x7fb3e406d080 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.535+0000 7fb3eaf6d640 1 -- 172.21.15.139:0/2254632846 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e40cbae0 con 0x7fb3e406d080 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.536+0000 7fb3e17fa640 1 -- 172.21.15.139:0/2254632846 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb3dc044070 con 0x7fb3e406d080 2024-09-16T00:10:23.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.537+0000 7fb3e17fa640 1 --2- 172.21.15.139:0/2254632846 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb3c406c2b0 0x7fb3c406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:23.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.537+0000 7fb3e17fa640 1 -- 172.21.15.139:0/2254632846 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(111..111 src has 1..111) ==== 5973+0+0 (secure 0 0 0) 0x7fb3dc0abdf0 con 0x7fb3e406d080 2024-09-16T00:10:23.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.537+0000 7fb3e3fff640 1 --2- 172.21.15.139:0/2254632846 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb3c406c2b0 0x7fb3c406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:23.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.537+0000 7fb3e3fff640 1 --2- 172.21.15.139:0/2254632846 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb3c406c2b0 0x7fb3c406e770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb3e41525e0 tx=0x7fb3d4001700 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:23.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.541+0000 7fb3e17fa640 1 -- 172.21.15.139:0/2254632846 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb3e40cbae0 con 0x7fb3e406d080 2024-09-16T00:10:23.634 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:23.633+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:23.640 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:23.640 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:23.650 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:10:23.735 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:23.734+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:23.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:23.739+0000 7fb3eaf6d640 1 -- 172.21.15.139:0/2254632846 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fb3e40c8bf0 con 0x7fb3e406d080 2024-09-16T00:10:23.801 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:23.802 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:23.811 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:10:23.837 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:23.835+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:23.937 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:23.935+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:23.957 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:23.957 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:23.967 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:10:24.037 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:24.036+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:24.116 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:24.116 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:24.126 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:10:24.170 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:24.169+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:24.235 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:24.235 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:24.241 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:24.239+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:24.246 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:10:24.341 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:24.339+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:24.387 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.385+0000 7fb3e17fa640 1 -- 172.21.15.139:0/2254632846 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v112) ==== 91+0+0 (secure 0 0 0) 0x7fb3dc079d60 con 0x7fb3e406d080 2024-09-16T00:10:24.388 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is unset 2024-09-16T00:10:24.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.388+0000 7fb3eaf6d640 1 -- 172.21.15.139:0/2254632846 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb3c406c2b0 msgr2=0x7fb3c406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:24.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.388+0000 7fb3eaf6d640 1 --2- 172.21.15.139:0/2254632846 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb3c406c2b0 0x7fb3c406e770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb3e41525e0 tx=0x7fb3d4001700 comp rx=0 tx=0).stop 2024-09-16T00:10:24.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.388+0000 7fb3eaf6d640 1 -- 172.21.15.139:0/2254632846 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3e406d080 msgr2=0x7fb3e4158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:24.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.388+0000 7fb3eaf6d640 1 --2- 172.21.15.139:0/2254632846 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3e406d080 0x7fb3e4158210 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc000c00 tx=0x7fb3dc0046c0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.388+0000 7fb3eaf6d640 1 -- 172.21.15.139:0/2254632846 shutdown_connections 2024-09-16T00:10:24.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.388+0000 7fb3eaf6d640 1 --2- 172.21.15.139:0/2254632846 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb3c406c2b0 0x7fb3c406e770 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.388+0000 7fb3eaf6d640 1 --2- 172.21.15.139:0/2254632846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3e40c64a0 0x7fb3e4157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.388+0000 7fb3eaf6d640 1 --2- 172.21.15.139:0/2254632846 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb3e40c60c0 0x7fb3e4148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.388+0000 7fb3eaf6d640 1 --2- 172.21.15.139:0/2254632846 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb3e406d080 0x7fb3e4158210 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.388+0000 7fb3eaf6d640 1 -- 172.21.15.139:0/2254632846 >> 172.21.15.139:0/2254632846 conn(0x7fb3e40cc740 msgr2=0x7fb3e40cbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:24.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.388+0000 7fb3eaf6d640 1 -- 172.21.15.139:0/2254632846 shutdown_connections 2024-09-16T00:10:24.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.388+0000 7fb3eaf6d640 1 -- 172.21.15.139:0/2254632846 wait complete. 2024-09-16T00:10:24.406 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:24.406 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:24.416 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:10:24.457 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:24.455+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:24.543 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:24.541+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:24.561 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:10:24.565 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:24.565 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:24.575 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:10:24.659 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:24.658+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:24.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.662+0000 7f72b5aba640 1 Processor -- start 2024-09-16T00:10:24.664 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.662+0000 7f72b5aba640 1 -- start start 2024-09-16T00:10:24.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.663+0000 7f72b5aba640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f72b00d0870 0x7f72b00b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:24.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.663+0000 7f72b5aba640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f72b00b3430 0x7f72b00b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:24.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.663+0000 7f72b5aba640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72b015adc0 con 0x7f72b00d0490 2024-09-16T00:10:24.665 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.663+0000 7f72b5aba640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f72b015b940 con 0x7f72b00b3430 2024-09-16T00:10:24.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.663+0000 7f72b5aba640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f72b00b3bb0 con 0x7f72b00d0870 2024-09-16T00:10:24.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.663+0000 7f72af7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f72b00d0490 0x7f72b00b2c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56246/0 (socket says 172.21.15.139:56246) 2024-09-16T00:10:24.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.663+0000 7f72af7fe640 1 -- 172.21.15.139:0/3883180803 learned_addr learned my addr 172.21.15.139:0/3883180803 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:24.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.663+0000 7f72aeffd640 1 --2- 172.21.15.139:0/3883180803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f72b00d0870 0x7f72b00b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:24.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.663+0000 7f72affff640 1 --2- 172.21.15.139:0/3883180803 >> v2:172.21.15.164:3300/0 conn(0x7f72b00b3430 0x7f72b00b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:24.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.663+0000 7f72ae7fc640 1 -- 172.21.15.139:0/3883180803 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1512000107 0 0) 0x7f72b015adc0 con 0x7f72b00d0490 2024-09-16T00:10:24.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.663+0000 7f72ae7fc640 1 -- 172.21.15.139:0/3883180803 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7298003520 con 0x7f72b00d0490 2024-09-16T00:10:24.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.663+0000 7f72affff640 1 -- 172.21.15.139:0/3883180803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f72b00d0870 msgr2=0x7f72b00b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:24.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.663+0000 7f72affff640 1 --2- 172.21.15.139:0/3883180803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f72b00d0870 0x7f72b00b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.663+0000 7f72affff640 1 -- 172.21.15.139:0/3883180803 >> v1:172.21.15.139:6789/0 conn(0x7f72b00d0490 legacy=0x7f72b00b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:24.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.663+0000 7f72affff640 1 -- 172.21.15.139:0/3883180803 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b015b0a0 con 0x7f72b00b3430 2024-09-16T00:10:24.666 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.663+0000 7f72aeffd640 1 --2- 172.21.15.139:0/3883180803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f72b00d0870 0x7f72b00b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:24.667 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.664+0000 7f72affff640 1 --2- 172.21.15.139:0/3883180803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72b00b3430 0x7f72b00b4580 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f72a8009dc0 tx=0x7f72a802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3662149be9194aa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:24.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.664+0000 7f72ae7fc640 1 -- 172.21.15.139:0/3883180803 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8007740 con 0x7f72b00b3430 2024-09-16T00:10:24.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.664+0000 7f72ae7fc640 1 -- 172.21.15.139:0/3883180803 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a80078e0 con 0x7f72b00b3430 2024-09-16T00:10:24.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.664+0000 7f72ae7fc640 1 -- 172.21.15.139:0/3883180803 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8007c90 con 0x7f72b00b3430 2024-09-16T00:10:24.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.665+0000 7f72b5aba640 1 -- 172.21.15.139:0/3883180803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72b00b3430 msgr2=0x7f72b00b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:24.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.665+0000 7f72b5aba640 1 --2- 172.21.15.139:0/3883180803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72b00b3430 0x7f72b00b4580 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f72a8009dc0 tx=0x7f72a802a710 comp rx=0 tx=0).stop 2024-09-16T00:10:24.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.665+0000 7f72b5aba640 1 -- 172.21.15.139:0/3883180803 shutdown_connections 2024-09-16T00:10:24.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.665+0000 7f72b5aba640 1 --2- 172.21.15.139:0/3883180803 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72b00b3430 0x7f72b00b4580 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.665+0000 7f72b5aba640 1 --2- 172.21.15.139:0/3883180803 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f72b00d0870 0x7f72b00b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.665+0000 7f72b5aba640 1 -- 172.21.15.139:0/3883180803 >> 172.21.15.139:0/3883180803 conn(0x7f72b006c520 msgr2=0x7f72b006c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:24.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.665+0000 7f72b5aba640 1 -- 172.21.15.139:0/3883180803 shutdown_connections 2024-09-16T00:10:24.668 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.665+0000 7f72b5aba640 1 -- 172.21.15.139:0/3883180803 wait complete. 2024-09-16T00:10:24.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.666+0000 7f72b5aba640 1 Processor -- start 2024-09-16T00:10:24.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.666+0000 7f72b5aba640 1 -- start start 2024-09-16T00:10:24.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.667+0000 7f72b5aba640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f72b00b3430 0x7f72b014a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:24.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.667+0000 7f72b5aba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72b00d0490 0x7f72b01f3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:24.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.667+0000 7f72b5aba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b00d0870 0x7f72b01467b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:24.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.667+0000 7f72b5aba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f72b015a980 con 0x7f72b00d0870 2024-09-16T00:10:24.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.667+0000 7f72b5aba640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f72b015b660 con 0x7f72b00d0490 2024-09-16T00:10:24.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.667+0000 7f72b5aba640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f72b015ab00 con 0x7f72b00b3430 2024-09-16T00:10:24.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.667+0000 7f72affff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b00d0870 0x7f72b01467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:24.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.667+0000 7f72affff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b00d0870 0x7f72b01467b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52154/0 (socket says 172.21.15.139:52154) 2024-09-16T00:10:24.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.667+0000 7f72affff640 1 -- 172.21.15.139:0/1605330516 learned_addr learned my addr 172.21.15.139:0/1605330516 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:24.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.667+0000 7f72af7fe640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f72b00b3430 0x7f72b014a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:24.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.667+0000 7f72aeffd640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72b00d0490 0x7f72b01f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:24.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.667+0000 7f72affff640 1 -- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f72b00b3430 msgr2=0x7f72b014a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:24.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.667+0000 7f72affff640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f72b00b3430 0x7f72b014a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.668+0000 7f72affff640 1 -- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72b00d0490 msgr2=0x7f72b01f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:24.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.668+0000 7f72affff640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72b00d0490 0x7f72b01f3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.668+0000 7f72affff640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b0157490 con 0x7f72b00d0870 2024-09-16T00:10:24.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.668+0000 7f72aeffd640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f72b00d0490 0x7f72b01f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:24.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.668+0000 7f72affff640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b00d0870 0x7f72b01467b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f72a8009eb0 tx=0x7f72a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:24.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.668+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8009450 con 0x7f72b00d0870 2024-09-16T00:10:24.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.668+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a8004260 con 0x7f72b00d0870 2024-09-16T00:10:24.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.668+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b015a380 con 0x7f72b00d0870 2024-09-16T00:10:24.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.668+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8004650 con 0x7f72b00d0870 2024-09-16T00:10:24.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.668+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b0157750 con 0x7f72b00d0870 2024-09-16T00:10:24.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.669+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f72b00b2c40 con 0x7f72b00d0870 2024-09-16T00:10:24.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.670+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f72a803b410 con 0x7f72b00d0870 2024-09-16T00:10:24.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.670+0000 7f7297fff640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f729c06c2b0 0x7f729c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:24.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.670+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5973+0+0 (secure 0 0 0) 0x7f72a80ac920 con 0x7f72b00d0870 2024-09-16T00:10:24.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.670+0000 7f72af7fe640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f729c06c2b0 0x7f729c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:24.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.670+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v112) ==== 54+0+16 (secure 0 0 0) 0x7f72a803f030 con 0x7f72b00d0870 2024-09-16T00:10:24.675 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.670+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f72b0157950 0x7f72b0147200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:24.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.670+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f72b00b4370 con 0x7f72b0157950 2024-09-16T00:10:24.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.671+0000 7f72aeffd640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f72b0157950 0x7f72b0147200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:24.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.671+0000 7f72af7fe640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f729c06c2b0 0x7f729c06e770 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f72a0005880 tx=0x7f72a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:24.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.671+0000 7f72aeffd640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f72b0157950 0x7f72b0147200 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:24.676 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.672+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f72b00b4370 con 0x7f72b0157950 2024-09-16T00:10:24.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.688+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f72b00b4e20 con 0x7f72b0157950 2024-09-16T00:10:24.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.690+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f72b00b4e20 con 0x7f72b0157950 2024-09-16T00:10:24.692 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:24.693 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:24.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.691+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f72b0159cb0 0x7f72b00c2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:24.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.691+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f72b015a0d0 con 0x7f72b0159cb0 2024-09-16T00:10:24.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.691+0000 7f72af7fe640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f72b0159cb0 0x7f72b00c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:24.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.691+0000 7f72af7fe640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f72b0159cb0 0x7f72b00c2590 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:24.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.692+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f72b015a0d0 con 0x7f72b0159cb0 2024-09-16T00:10:24.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.707+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f72b00b4b40 con 0x7f72b0159cb0 2024-09-16T00:10:24.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.709+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f72b00b4b40 con 0x7f72b0159cb0 2024-09-16T00:10:24.711 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:24.711 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:24.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.709+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f72b00c2fe0 0x7f72b0062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:24.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.709+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f72b0150bc0 con 0x7f72b00c2fe0 2024-09-16T00:10:24.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.709+0000 7f72aeffd640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f72b00c2fe0 0x7f72b0062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:24.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.709+0000 7f72aeffd640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f72b00c2fe0 0x7f72b0062630 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:24.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.710+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f72b0150bc0 con 0x7f72b00c2fe0 2024-09-16T00:10:24.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.725+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f72b01f3e60 con 0x7f72b00c2fe0 2024-09-16T00:10:24.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.726+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f72b01f3e60 con 0x7f72b00c2fe0 2024-09-16T00:10:24.728 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:24.729 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:24.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.726+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f72b0063080 0x7f72b0148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:24.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.726+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f72b01f4070 con 0x7f72b0063080 2024-09-16T00:10:24.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.727+0000 7f72affff640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f72b0063080 0x7f72b0148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:24.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.727+0000 7f72affff640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f72b0063080 0x7f72b0148ce0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:24.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.727+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f72b01f4070 con 0x7f72b0063080 2024-09-16T00:10:24.731 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:24.731 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:24.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:24.731 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:24.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.731 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5850 3.3a 3:5c3cfc2b:::10000001adf.00000000:head [write 0~116] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:24.731 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:24.731 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 69.312843905999998, 2024-09-16T00:10:24.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66931295800000001, 2024-09-16T00:10:24.731 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.731 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.731 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.732 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.732 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.732 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.732 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5850 2024-09-16T00:10:24.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.732 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.732 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:24.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.732 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.732 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:24.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415938+0000", 2024-09-16T00:10:24.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7099999999999999e-06 2024-09-16T00:10:24.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415945+0000", 2024-09-16T00:10:24.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:10:24.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415959+0000", 2024-09-16T00:10:24.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3545e-05 2024-09-16T00:10:24.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415966+0000", 2024-09-16T00:10:24.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6379999999999998e-06 2024-09-16T00:10:24.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416013+0000", 2024-09-16T00:10:24.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7314999999999997e-05 2024-09-16T00:10:24.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:24.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416020+0000", 2024-09-16T00:10:24.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.52e-06 2024-09-16T00:10:24.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082393+0000", 2024-09-16T00:10:24.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66637251799999997 2024-09-16T00:10:24.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082537+0000", 2024-09-16T00:10:24.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014418800000000001 2024-09-16T00:10:24.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:24.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082689+0000", 2024-09-16T00:10:24.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015198499999999999 2024-09-16T00:10:24.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:24.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084930+0000", 2024-09-16T00:10:24.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002241041 2024-09-16T00:10:24.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:24.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084994+0000", 2024-09-16T00:10:24.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3274000000000004e-05 2024-09-16T00:10:24.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085049+0000", 2024-09-16T00:10:24.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.554e-05 2024-09-16T00:10:24.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085250+0000", 2024-09-16T00:10:24.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020060799999999999 2024-09-16T00:10:24.740 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.740 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.740 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.740 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5910 3.3a 3:5ec98a62:::10000001b22.00000000:head [write 0~1274] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:24.740 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:24.740 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 69.165900268000001, 2024-09-16T00:10:24.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.52286026600000002, 2024-09-16T00:10:24.740 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.741 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.741 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.741 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.741 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.741 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.741 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5910 2024-09-16T00:10:24.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.741 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:24.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:24.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562881+0000", 2024-09-16T00:10:24.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:10:24.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562887+0000", 2024-09-16T00:10:24.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2870000000000004e-06 2024-09-16T00:10:24.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562897+0000", 2024-09-16T00:10:24.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.064e-05 2024-09-16T00:10:24.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562902+0000", 2024-09-16T00:10:24.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2340000000000002e-06 2024-09-16T00:10:24.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562958+0000", 2024-09-16T00:10:24.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5523999999999999e-05 2024-09-16T00:10:24.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:24.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562968+0000", 2024-09-16T00:10:24.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0394000000000001e-05 2024-09-16T00:10:24.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083461+0000", 2024-09-16T00:10:24.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.520492968 2024-09-16T00:10:24.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083645+0000", 2024-09-16T00:10:24.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018346399999999999 2024-09-16T00:10:24.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:24.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083844+0000", 2024-09-16T00:10:24.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000199268 2024-09-16T00:10:24.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.745 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.745 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:24.745 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085410+0000", 2024-09-16T00:10:24.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015662359999999999 2024-09-16T00:10:24.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:24.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085493+0000", 2024-09-16T00:10:24.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2478999999999994e-05 2024-09-16T00:10:24.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085563+0000", 2024-09-16T00:10:24.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0159000000000004e-05 2024-09-16T00:10:24.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085741+0000", 2024-09-16T00:10:24.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017764300000000001 2024-09-16T00:10:24.747 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.747 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.747 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.747 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5962 3.3a 3:5c7b4d29:::10000001b5d.00000000:head [write 0~3074] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:24.747 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:24.747 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 69.036363451, 2024-09-16T00:10:24.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39368737500000001, 2024-09-16T00:10:24.747 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.747 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.747 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.748 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.748 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.748 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.748 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5962 2024-09-16T00:10:24.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.748 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:24.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:24.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692418+0000", 2024-09-16T00:10:24.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3510000000000001e-06 2024-09-16T00:10:24.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692425+0000", 2024-09-16T00:10:24.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3770000000000001e-06 2024-09-16T00:10:24.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692440+0000", 2024-09-16T00:10:24.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4975e-05 2024-09-16T00:10:24.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692448+0000", 2024-09-16T00:10:24.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6279999999999994e-06 2024-09-16T00:10:24.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692499+0000", 2024-09-16T00:10:24.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0872999999999999e-05 2024-09-16T00:10:24.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:24.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692507+0000", 2024-09-16T00:10:24.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9889999999999992e-06 2024-09-16T00:10:24.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084221+0000", 2024-09-16T00:10:24.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39171379699999997 2024-09-16T00:10:24.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084446+0000", 2024-09-16T00:10:24.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022519900000000001 2024-09-16T00:10:24.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:24.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084672+0000", 2024-09-16T00:10:24.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000225405 2024-09-16T00:10:24.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:24.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085832+0000", 2024-09-16T00:10:24.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011600040000000001 2024-09-16T00:10:24.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:24.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085864+0000", 2024-09-16T00:10:24.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2592999999999998e-05 2024-09-16T00:10:24.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085943+0000", 2024-09-16T00:10:24.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8534000000000004e-05 2024-09-16T00:10:24.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086105+0000", 2024-09-16T00:10:24.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016165 2024-09-16T00:10:24.753 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.753 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.753 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.753 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5963 3.3a 3:5e4192f5:::10000001b5e.00000000:head [write 0~152] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:24.754 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:24.754 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 69.034299020000006, 2024-09-16T00:10:24.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39413463900000001, 2024-09-16T00:10:24.754 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.754 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.754 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.754 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.754 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.754 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.754 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5963 2024-09-16T00:10:24.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.754 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:24.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:24.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:24.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:10:24.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694486+0000", 2024-09-16T00:10:24.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0960000000000003e-06 2024-09-16T00:10:24.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694496+0000", 2024-09-16T00:10:24.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2189999999999993e-06 2024-09-16T00:10:24.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694500+0000", 2024-09-16T00:10:24.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9529999999999998e-06 2024-09-16T00:10:24.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694552+0000", 2024-09-16T00:10:24.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1360999999999998e-05 2024-09-16T00:10:24.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:24.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694561+0000", 2024-09-16T00:10:24.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1700000000000003e-06 2024-09-16T00:10:24.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086346+0000", 2024-09-16T00:10:24.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39178468 2024-09-16T00:10:24.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086480+0000", 2024-09-16T00:10:24.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013469299999999999 2024-09-16T00:10:24.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:24.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086790+0000", 2024-09-16T00:10:24.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030923300000000001 2024-09-16T00:10:24.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:24.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088361+0000", 2024-09-16T00:10:24.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015717050000000001 2024-09-16T00:10:24.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:24.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088432+0000", 2024-09-16T00:10:24.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0922999999999998e-05 2024-09-16T00:10:24.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088489+0000", 2024-09-16T00:10:24.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6653999999999997e-05 2024-09-16T00:10:24.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088616+0000", 2024-09-16T00:10:24.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012730099999999999 2024-09-16T00:10:24.760 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.760 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.760 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.760 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:6082 3.3a 3:5c50f65b:::10000001be7.00000000:head [write 0~2439] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:24.760 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:24.760 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 68.740526164000002, 2024-09-16T00:10:24.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.10065850799999999, 2024-09-16T00:10:24.761 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.761 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.761 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.761 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.761 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.761 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.761 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 6082 2024-09-16T00:10:24.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.761 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:24.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:24.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988256+0000", 2024-09-16T00:10:24.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.065e-06 2024-09-16T00:10:24.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988262+0000", 2024-09-16T00:10:24.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4640000000000003e-06 2024-09-16T00:10:24.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.763 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.763 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988272+0000", 2024-09-16T00:10:24.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0335e-05 2024-09-16T00:10:24.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.763 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.763 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988279+0000", 2024-09-16T00:10:24.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:10:24.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.763 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.764 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988324+0000", 2024-09-16T00:10:24.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4953000000000001e-05 2024-09-16T00:10:24.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.764 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.764 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:24.764 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988331+0000", 2024-09-16T00:10:24.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5989999999999998e-06 2024-09-16T00:10:24.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.764 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.764 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.764 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087272+0000", 2024-09-16T00:10:24.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.098941045000000005 2024-09-16T00:10:24.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087486+0000", 2024-09-16T00:10:24.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00021360199999999999 2024-09-16T00:10:24.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:24.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087740+0000", 2024-09-16T00:10:24.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00025456799999999999 2024-09-16T00:10:24.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:24.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088638+0000", 2024-09-16T00:10:24.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00089793900000000005 2024-09-16T00:10:24.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:24.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088673+0000", 2024-09-16T00:10:24.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5111999999999999e-05 2024-09-16T00:10:24.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088722+0000", 2024-09-16T00:10:24.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8758000000000003e-05 2024-09-16T00:10:24.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088913+0000", 2024-09-16T00:10:24.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000190953 2024-09-16T00:10:24.767 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.767 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.768 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.768 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10643 3.21 3:8453163e:::100000030b3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:24.768 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:24.768 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.129049971000001, 2024-09-16T00:10:24.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013028809000000001, 2024-09-16T00:10:24.768 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.768 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.768 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.769 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.769 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.769 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.769 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10643 2024-09-16T00:10:24.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.769 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:24.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.770 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.770 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:24.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.770 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.770 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599734+0000", 2024-09-16T00:10:24.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-06 2024-09-16T00:10:24.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603345+0000", 2024-09-16T00:10:24.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003611628 2024-09-16T00:10:24.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603368+0000", 2024-09-16T00:10:24.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3017e-05 2024-09-16T00:10:24.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603379+0000", 2024-09-16T00:10:24.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0528e-05 2024-09-16T00:10:24.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603448+0000", 2024-09-16T00:10:24.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9080999999999998e-05 2024-09-16T00:10:24.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603623+0000", 2024-09-16T00:10:24.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017525599999999999 2024-09-16T00:10:24.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:24.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603766+0000", 2024-09-16T00:10:24.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000142977 2024-09-16T00:10:24.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:24.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.611250+0000", 2024-09-16T00:10:24.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074835240000000001 2024-09-16T00:10:24.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:24.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612617+0000", 2024-09-16T00:10:24.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001367793 2024-09-16T00:10:24.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612700+0000", 2024-09-16T00:10:24.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3123999999999999e-05 2024-09-16T00:10:24.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612759+0000", 2024-09-16T00:10:24.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8950999999999999e-05 2024-09-16T00:10:24.774 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.775 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.775 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.775 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10652 3.22 3:44e46336:::100000030c1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:24.775 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:10:24.775 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.100998339, 2024-09-16T00:10:24.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011405732, 2024-09-16T00:10:24.775 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.775 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.775 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.775 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.776 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.776 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.776 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10652 2024-09-16T00:10:24.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.776 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.776 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.776 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:10:24.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.776 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.776 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:10:24.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627785+0000", 2024-09-16T00:10:24.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6970000000000002e-06 2024-09-16T00:10:24.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629610+0000", 2024-09-16T00:10:24.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018250359999999999 2024-09-16T00:10:24.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629639+0000", 2024-09-16T00:10:24.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8676000000000001e-05 2024-09-16T00:10:24.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629652+0000", 2024-09-16T00:10:24.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3572e-05 2024-09-16T00:10:24.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629735+0000", 2024-09-16T00:10:24.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.25e-05 2024-09-16T00:10:24.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629925+0000", 2024-09-16T00:10:24.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019011000000000001 2024-09-16T00:10:24.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:10:24.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.630075+0000", 2024-09-16T00:10:24.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000149841 2024-09-16T00:10:24.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:24.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.636470+0000", 2024-09-16T00:10:24.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063952289999999997 2024-09-16T00:10:24.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:24.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639098+0000", 2024-09-16T00:10:24.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026277980000000002 2024-09-16T00:10:24.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639154+0000", 2024-09-16T00:10:24.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6413999999999999e-05 2024-09-16T00:10:24.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639188+0000", 2024-09-16T00:10:24.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3859000000000002e-05 2024-09-16T00:10:24.781 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.781 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.781 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.781 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11217 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:24.782 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:24.782 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 44.154501318000001, 2024-09-16T00:10:24.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01231253, 2024-09-16T00:10:24.782 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.782 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.782 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.782 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.782 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.782 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.782 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11217 2024-09-16T00:10:24.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.782 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.782 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.782 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:24.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:24.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574282+0000", 2024-09-16T00:10:24.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8499999999999998e-06 2024-09-16T00:10:24.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575804+0000", 2024-09-16T00:10:24.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001521859 2024-09-16T00:10:24.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575826+0000", 2024-09-16T00:10:24.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2059999999999999e-05 2024-09-16T00:10:24.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575837+0000", 2024-09-16T00:10:24.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1313e-05 2024-09-16T00:10:24.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575920+0000", 2024-09-16T00:10:24.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2918999999999997e-05 2024-09-16T00:10:24.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576088+0000", 2024-09-16T00:10:24.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016803799999999999 2024-09-16T00:10:24.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:24.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576233+0000", 2024-09-16T00:10:24.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014508199999999999 2024-09-16T00:10:24.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.786 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:24.786 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.580500+0000", 2024-09-16T00:10:24.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004266644 2024-09-16T00:10:24.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.786 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:24.786 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586479+0000", 2024-09-16T00:10:24.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059791820000000004 2024-09-16T00:10:24.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586541+0000", 2024-09-16T00:10:24.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1463000000000004e-05 2024-09-16T00:10:24.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586592+0000", 2024-09-16T00:10:24.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1119999999999998e-05 2024-09-16T00:10:24.787 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.787 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.787 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.788 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11767 3.38 3:1fa50e65:::10000003117.00000000:head [write 0~4194304 [3@1169732]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:24.788 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:24.788 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.565096783000001, 2024-09-16T00:10:24.788 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011555678999999999, 2024-09-16T00:10:24.788 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.788 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.788 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.788 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.788 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.788 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.788 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11767 2024-09-16T00:10:24.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.788 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.789 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.789 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:24.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.789 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.789 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:24.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.789 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163686+0000", 2024-09-16T00:10:24.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3259999999999998e-06 2024-09-16T00:10:24.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165053+0000", 2024-09-16T00:10:24.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013665859999999999 2024-09-16T00:10:24.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165075+0000", 2024-09-16T00:10:24.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2206999999999999e-05 2024-09-16T00:10:24.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165084+0000", 2024-09-16T00:10:24.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0710000000000001e-06 2024-09-16T00:10:24.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165157+0000", 2024-09-16T00:10:24.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2619000000000004e-05 2024-09-16T00:10:24.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165249+0000", 2024-09-16T00:10:24.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2517999999999997e-05 2024-09-16T00:10:24.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:24.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165360+0000", 2024-09-16T00:10:24.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011134899999999999 2024-09-16T00:10:24.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:24.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.169320+0000", 2024-09-16T00:10:24.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039593040000000003 2024-09-16T00:10:24.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:24.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175135+0000", 2024-09-16T00:10:24.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005814891 2024-09-16T00:10:24.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175199+0000", 2024-09-16T00:10:24.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4474999999999995e-05 2024-09-16T00:10:24.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175239+0000", 2024-09-16T00:10:24.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0333000000000002e-05 2024-09-16T00:10:24.794 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.794 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.794 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.794 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11789 3.21 3:8453163e:::100000030b3.00000000:head [write 0~3801088 [4@1194504]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:24.794 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:10:24.794 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.477408687, 2024-09-16T00:10:24.795 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011097602, 2024-09-16T00:10:24.795 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.795 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.795 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.795 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.795 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.795 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.795 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11789 2024-09-16T00:10:24.795 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.795 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.795 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.795 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:10:24.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:10:24.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251373+0000", 2024-09-16T00:10:24.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.544e-06 2024-09-16T00:10:24.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252710+0000", 2024-09-16T00:10:24.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013362929999999999 2024-09-16T00:10:24.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252730+0000", 2024-09-16T00:10:24.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0579999999999999e-05 2024-09-16T00:10:24.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252741+0000", 2024-09-16T00:10:24.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0505e-05 2024-09-16T00:10:24.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252820+0000", 2024-09-16T00:10:24.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9098999999999996e-05 2024-09-16T00:10:24.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252931+0000", 2024-09-16T00:10:24.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011073600000000001 2024-09-16T00:10:24.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.253101+0000", 2024-09-16T00:10:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017038000000000001 2024-09-16T00:10:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.261813+0000", 2024-09-16T00:10:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087119469999999994 2024-09-16T00:10:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262338+0000", 2024-09-16T00:10:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00052493799999999997 2024-09-16T00:10:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262430+0000", 2024-09-16T00:10:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1626999999999995e-05 2024-09-16T00:10:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262470+0000", 2024-09-16T00:10:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9953000000000002e-05 2024-09-16T00:10:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.800 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11890 3.38 3:1e014616:::100000033e9.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:24.801 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:24.801 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.157443604999997, 2024-09-16T00:10:24.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011833791999999999, 2024-09-16T00:10:24.801 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.801 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.801 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.801 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.801 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.801 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.801 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11890 2024-09-16T00:10:24.801 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.801 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.801 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:24.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:24.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571340+0000", 2024-09-16T00:10:24.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.36e-06 2024-09-16T00:10:24.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572866+0000", 2024-09-16T00:10:24.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001525549 2024-09-16T00:10:24.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.803 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572884+0000", 2024-09-16T00:10:24.803 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7796999999999999e-05 2024-09-16T00:10:24.803 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.803 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.803 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572897+0000", 2024-09-16T00:10:24.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3567000000000001e-05 2024-09-16T00:10:24.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572970+0000", 2024-09-16T00:10:24.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2751999999999997e-05 2024-09-16T00:10:24.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573053+0000", 2024-09-16T00:10:24.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3230000000000001e-05 2024-09-16T00:10:24.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573168+0000", 2024-09-16T00:10:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114536 2024-09-16T00:10:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.577923+0000", 2024-09-16T00:10:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047548720000000003 2024-09-16T00:10:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:24.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583081+0000", 2024-09-16T00:10:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051587630000000002 2024-09-16T00:10:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583137+0000", 2024-09-16T00:10:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5259000000000001e-05 2024-09-16T00:10:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583171+0000", 2024-09-16T00:10:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4106999999999997e-05 2024-09-16T00:10:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.806 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:12037 3.9 e87/20 3:928db521:::1000000326d.00000000:head v 87'199, mlcod=87'197)", 2024-09-16T00:10:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.726253653999997, 2024-09-16T00:10:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012236153, 2024-09-16T00:10:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002529+0000", 2024-09-16T00:10:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4889999999999998e-06 2024-09-16T00:10:24.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007064+0000", 2024-09-16T00:10:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045346220000000003 2024-09-16T00:10:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007093+0000", 2024-09-16T00:10:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9009e-05 2024-09-16T00:10:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007109+0000", 2024-09-16T00:10:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5846e-05 2024-09-16T00:10:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007196+0000", 2024-09-16T00:10:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7507000000000001e-05 2024-09-16T00:10:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007227+0000", 2024-09-16T00:10:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0916e-05 2024-09-16T00:10:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014658+0000", 2024-09-16T00:10:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074305090000000001 2024-09-16T00:10:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014763+0000", 2024-09-16T00:10:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010525499999999999 2024-09-16T00:10:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12398 3.2e 3:77a11d08:::10000003432.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:24.811 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 39.487574871, 2024-09-16T00:10:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012092999, 2024-09-16T00:10:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12398 2024-09-16T00:10:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241209+0000", 2024-09-16T00:10:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2710000000000001e-06 2024-09-16T00:10:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243840+0000", 2024-09-16T00:10:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026307919999999999 2024-09-16T00:10:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243872+0000", 2024-09-16T00:10:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2699e-05 2024-09-16T00:10:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243887+0000", 2024-09-16T00:10:24.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4348000000000001e-05 2024-09-16T00:10:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243972+0000", 2024-09-16T00:10:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5358999999999994e-05 2024-09-16T00:10:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244132+0000", 2024-09-16T00:10:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015972499999999999 2024-09-16T00:10:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:24.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253153+0000", 2024-09-16T00:10:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0090210280000000004 2024-09-16T00:10:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253213+0000", 2024-09-16T00:10:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0300999999999997e-05 2024-09-16T00:10:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253299+0000", 2024-09-16T00:10:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5476000000000006e-05 2024-09-16T00:10:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.816 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12400 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304 [2@815045]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:10:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 39.484537136999997, 2024-09-16T00:10:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011140066000000001, 2024-09-16T00:10:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.817 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12400 2024-09-16T00:10:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:10:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:10:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.818 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244245+0000", 2024-09-16T00:10:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.248e-06 2024-09-16T00:10:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246467+0000", 2024-09-16T00:10:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022217949999999999 2024-09-16T00:10:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246496+0000", 2024-09-16T00:10:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9484000000000001e-05 2024-09-16T00:10:24.819 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246509+0000", 2024-09-16T00:10:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3390000000000001e-05 2024-09-16T00:10:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246586+0000", 2024-09-16T00:10:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6692000000000004e-05 2024-09-16T00:10:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246680+0000", 2024-09-16T00:10:24.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3468999999999996e-05 2024-09-16T00:10:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246850+0000", 2024-09-16T00:10:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170206 2024-09-16T00:10:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253478+0000", 2024-09-16T00:10:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066287530000000002 2024-09-16T00:10:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:24.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255318+0000", 2024-09-16T00:10:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001839292 2024-09-16T00:10:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255354+0000", 2024-09-16T00:10:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5964999999999999e-05 2024-09-16T00:10:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255384+0000", 2024-09-16T00:10:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9771999999999999e-05 2024-09-16T00:10:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.822 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13317 3.38 3:1cda8244:::100000035dc.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:10:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.810182611000002, 2024-09-16T00:10:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011377063, 2024-09-16T00:10:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.823 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13317 2024-09-16T00:10:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:10:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:10:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918602+0000", 2024-09-16T00:10:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8160000000000004e-06 2024-09-16T00:10:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920364+0000", 2024-09-16T00:10:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017623630000000001 2024-09-16T00:10:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920391+0000", 2024-09-16T00:10:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7248000000000001e-05 2024-09-16T00:10:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920405+0000", 2024-09-16T00:10:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3969e-05 2024-09-16T00:10:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920486+0000", 2024-09-16T00:10:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0737999999999998e-05 2024-09-16T00:10:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920594+0000", 2024-09-16T00:10:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108269 2024-09-16T00:10:24.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920730+0000", 2024-09-16T00:10:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000135229 2024-09-16T00:10:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.928847+0000", 2024-09-16T00:10:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081178299999999995 2024-09-16T00:10:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:24.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929862+0000", 2024-09-16T00:10:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010147540000000001 2024-09-16T00:10:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929923+0000", 2024-09-16T00:10:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0452999999999999e-05 2024-09-16T00:10:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929975+0000", 2024-09-16T00:10:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2394e-05 2024-09-16T00:10:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.828 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:13326 3.9 e88/20 3:92f796cf:::1000000350f.00000000:head v 88'214, mlcod=88'212)", 2024-09-16T00:10:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:10:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 36.767661820000001, 2024-09-16T00:10:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01100597, 2024-09-16T00:10:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:10:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961119+0000", 2024-09-16T00:10:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.961121+0000", 2024-09-16T00:10:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.815e-06 2024-09-16T00:10:24.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965747+0000", 2024-09-16T00:10:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0046260570000000003 2024-09-16T00:10:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965780+0000", 2024-09-16T00:10:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2851000000000001e-05 2024-09-16T00:10:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965788+0000", 2024-09-16T00:10:24.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0180000000000001e-06 2024-09-16T00:10:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965859+0000", 2024-09-16T00:10:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1318999999999999e-05 2024-09-16T00:10:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.965886+0000", 2024-09-16T00:10:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7509999999999999e-05 2024-09-16T00:10:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.972066+0000", 2024-09-16T00:10:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061800759999999996 2024-09-16T00:10:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.972125+0000", 2024-09-16T00:10:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8323999999999999e-05 2024-09-16T00:10:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.833 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13942 3.22 3:44ed4521:::100000036d5.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:10:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 34.806296508000003, 2024-09-16T00:10:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011341061, 2024-09-16T00:10:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13942 2024-09-16T00:10:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:10:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:10:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922487+0000", 2024-09-16T00:10:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0180000000000002e-06 2024-09-16T00:10:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923720+0000", 2024-09-16T00:10:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012333470000000001 2024-09-16T00:10:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923749+0000", 2024-09-16T00:10:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8748999999999999e-05 2024-09-16T00:10:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923770+0000", 2024-09-16T00:10:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1189999999999999e-05 2024-09-16T00:10:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923868+0000", 2024-09-16T00:10:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7675000000000002e-05 2024-09-16T00:10:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.924047+0000", 2024-09-16T00:10:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017935100000000001 2024-09-16T00:10:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:10:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.924223+0000", 2024-09-16T00:10:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000175661 2024-09-16T00:10:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.931544+0000", 2024-09-16T00:10:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073204589999999996 2024-09-16T00:10:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:24.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933692+0000", 2024-09-16T00:10:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021488760000000001 2024-09-16T00:10:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933774+0000", 2024-09-16T00:10:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1855999999999997e-05 2024-09-16T00:10:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933825+0000", 2024-09-16T00:10:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0878999999999999e-05 2024-09-16T00:10:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.839 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14867 3.1f 3:faedf583:::1000000393b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:10:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.981806771999999, 2024-09-16T00:10:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011128677999999999, 2024-09-16T00:10:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.840 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14867 2024-09-16T00:10:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:10:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:10:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746976+0000", 2024-09-16T00:10:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1169999999999998e-06 2024-09-16T00:10:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748348+0000", 2024-09-16T00:10:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013721860000000001 2024-09-16T00:10:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748371+0000", 2024-09-16T00:10:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2938999999999998e-05 2024-09-16T00:10:24.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748381+0000", 2024-09-16T00:10:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5300000000000002e-06 2024-09-16T00:10:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748458+0000", 2024-09-16T00:10:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7389000000000001e-05 2024-09-16T00:10:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748626+0000", 2024-09-16T00:10:24.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016757499999999999 2024-09-16T00:10:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748813+0000", 2024-09-16T00:10:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018743399999999999 2024-09-16T00:10:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.753998+0000", 2024-09-16T00:10:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051851140000000002 2024-09-16T00:10:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.757990+0000", 2024-09-16T00:10:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039914169999999997 2024-09-16T00:10:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.758054+0000", 2024-09-16T00:10:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4513999999999999e-05 2024-09-16T00:10:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.758103+0000", 2024-09-16T00:10:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8463e-05 2024-09-16T00:10:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.845 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15003 3.21 3:8673bb5f:::100000030ed.00000000:head [read 2097152~2097152 [2@1395527]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.560384776999999, 2024-09-16T00:10:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058433321000000003, 2024-09-16T00:10:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:10:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.846 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15003 2024-09-16T00:10:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168397+0000", 2024-09-16T00:10:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3900000000000004e-07 2024-09-16T00:10:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168401+0000", 2024-09-16T00:10:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4050000000000002e-06 2024-09-16T00:10:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168411+0000", 2024-09-16T00:10:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6539999999999994e-06 2024-09-16T00:10:24.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168417+0000", 2024-09-16T00:10:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6719999999999998e-06 2024-09-16T00:10:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168467+0000", 2024-09-16T00:10:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9815999999999999e-05 2024-09-16T00:10:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168535+0000", 2024-09-16T00:10:24.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7669999999999994e-05 2024-09-16T00:10:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.226829+0000", 2024-09-16T00:10:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058294264999999998 2024-09-16T00:10:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15005 3.1f 3:f8460f10:::10000003370.00000000:head [read 1048576~1048576 [2@4194304]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.558704065000001, 2024-09-16T00:10:24.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.055844088, 2024-09-16T00:10:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:10:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15005 2024-09-16T00:10:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:24.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170078+0000", 2024-09-16T00:10:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.235e-06 2024-09-16T00:10:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:24.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170084+0000", 2024-09-16T00:10:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9739999999999999e-06 2024-09-16T00:10:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225526+0000", 2024-09-16T00:10:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.055441777999999997 2024-09-16T00:10:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225542+0000", 2024-09-16T00:10:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6606999999999999e-05 2024-09-16T00:10:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225668+0000", 2024-09-16T00:10:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012631 2024-09-16T00:10:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225744+0000", 2024-09-16T00:10:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5794e-05 2024-09-16T00:10:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225921+0000", 2024-09-16T00:10:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017639000000000001 2024-09-16T00:10:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.854 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.855 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.855 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:24.855 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:24.855 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:24.857 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:10:24.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.743+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f72b01f4070 con 0x7f72b0063080 2024-09-16T00:10:24.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.745+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f72b01f4070 con 0x7f72b0063080 2024-09-16T00:10:24.857 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:24.858 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:24.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.745+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f72b0149730 0x7f72b002c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:24.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.745+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f72b0149b50 con 0x7f72b0149730 2024-09-16T00:10:24.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.745+0000 7f72af7fe640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f72b0149730 0x7f72b002c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:24.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.746+0000 7f72af7fe640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f72b0149730 0x7f72b002c640 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:24.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.746+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f72b0149b50 con 0x7f72b0149730 2024-09-16T00:10:24.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.766+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f72b0149b50 con 0x7f72b0149730 2024-09-16T00:10:24.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.768+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f72b0149b50 con 0x7f72b0149730 2024-09-16T00:10:24.859 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:24.860 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:24.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.768+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f72b002d090 0x7f72b002d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:24.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.768+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f72b002db60 con 0x7f72b002d090 2024-09-16T00:10:24.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.769+0000 7f72aeffd640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f72b002d090 0x7f72b002d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:24.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.769+0000 7f72aeffd640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f72b002d090 0x7f72b002d4f0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:24.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.770+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f72a4014930 con 0x7f72b002d090 2024-09-16T00:10:24.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.784+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f72b0206f40 con 0x7f72b002d090 2024-09-16T00:10:24.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.787+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f72b0206f40 con 0x7f72b002d090 2024-09-16T00:10:24.861 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:24.862 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:24.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.787+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f72b0208dc0 0x7f72b02091e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:24.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.787+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f72b0208460 con 0x7f72b0208dc0 2024-09-16T00:10:24.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.787+0000 7f72affff640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f72b0208dc0 0x7f72b02091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:24.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.787+0000 7f72affff640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f72b0208dc0 0x7f72b02091e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:24.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.788+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f72b0208460 con 0x7f72b0208dc0 2024-09-16T00:10:24.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.802+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f72b0207260 con 0x7f72b0208dc0 2024-09-16T00:10:24.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.804+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f72b0207260 con 0x7f72b0208dc0 2024-09-16T00:10:24.863 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:24.864 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:24.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.804+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f72b02078e0 0x7f72b0207d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:24.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.804+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f72b02082c0 con 0x7f72b02078e0 2024-09-16T00:10:24.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.804+0000 7f72af7fe640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f72b02078e0 0x7f72b0207d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:24.864 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.805+0000 7f72af7fe640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f72b02078e0 0x7f72b0207d00 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:24.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.805+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== osd.7 v2:172.21.15.164:6808/1158838808 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f72b02082c0 con 0x7f72b02078e0 2024-09-16T00:10:24.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.820+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f72b00634c0 con 0x7f72b02078e0 2024-09-16T00:10:24.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.821+0000 7f7297fff640 1 -- 172.21.15.139:0/1605330516 <== osd.7 v2:172.21.15.164:6808/1158838808 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f72b00634c0 con 0x7f72b02078e0 2024-09-16T00:10:24.865 INFO:teuthology.orchestra.run.smithi139.stdout:osd.7: {} 2024-09-16T00:10:24.865 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:24.855+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:24.867 INFO:teuthology.orchestra.run.smithi139.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T00:10:24.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.821+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f72b0157950 msgr2=0x7f72b0147200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:24.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.821+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f72b0157950 0x7f72b0147200 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.821+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f72b0159cb0 msgr2=0x7f72b00c2590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:24.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.821+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f72b0159cb0 0x7f72b00c2590 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.821+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f72b00c2fe0 msgr2=0x7f72b0062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:24.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.821+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f72b00c2fe0 0x7f72b0062630 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.821+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f72b0063080 msgr2=0x7f72b0148ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:24.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.821+0000 7f72affff640 1 -- 172.21.15.139:0/1605330516 reap_dead start 2024-09-16T00:10:24.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.821+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f72b0063080 0x7f72b0148ce0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f72b0149730 msgr2=0x7f72b002c640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:24.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f72b0149730 0x7f72b002c640 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f72b002d090 msgr2=0x7f72b002d4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:24.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f72b002d090 0x7f72b002d4f0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f72b0208dc0 msgr2=0x7f72b02091e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:24.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f72b0208dc0 0x7f72b02091e0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f72b02078e0 msgr2=0x7f72b0207d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:24.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f72b02078e0 0x7f72b0207d00 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72affff640 1 -- 172.21.15.139:0/1605330516 reap_dead start 2024-09-16T00:10:24.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f729c06c2b0 msgr2=0x7f729c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:24.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f729c06c2b0 0x7f729c06e770 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f72a0005880 tx=0x7f72a0009040 comp rx=0 tx=0).stop 2024-09-16T00:10:24.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b00d0870 msgr2=0x7f72b01467b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:24.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b00d0870 0x7f72b01467b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f72a8009eb0 tx=0x7f72a8004a40 comp rx=0 tx=0).stop 2024-09-16T00:10:24.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 shutdown_connections 2024-09-16T00:10:24.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f729c06c2b0 0x7f729c06e770 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 --2- 172.21.15.139:0/1605330516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f72b00d0870 0x7f72b01467b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:24.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 >> 172.21.15.139:0/1605330516 conn(0x7f72b006c520 msgr2=0x7f72b00ccca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:24.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 shutdown_connections 2024-09-16T00:10:24.870 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:24.822+0000 7f72b5aba640 1 -- 172.21.15.139:0/1605330516 wait complete. 2024-09-16T00:10:24.957 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:24.955+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:25.015 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:25.015 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:25.015 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:25.015 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:25.026 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:10:25.067 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:25.065+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:25.159 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:25.158+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:25.172 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:25.172 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:25.173 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:25.173 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:25.173 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:25.182 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:10:25.260 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:25.259+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:25.294 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:25.294 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:90 2.1b 2:ddb4e4d7:::200.00000002:head [write 2593031~1404 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 81.643420668000005, 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011882886, 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649119+0000", 2024-09-16T00:10:25.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:10:25.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649125+0000", 2024-09-16T00:10:25.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1650000000000003e-06 2024-09-16T00:10:25.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649134+0000", 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5669999999999995e-06 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649141+0000", 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9040000000000003e-06 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655796+0000", 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066553649999999999 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655868+0000", 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2219999999999996e-05 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655986+0000", 2024-09-16T00:10:25.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117457 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657794+0000", 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001807894 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660890+0000", 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030958449999999998 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.300 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660966+0000", 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6382000000000002e-05 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.661001+0000", 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5349000000000003e-05 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10851 3.29 3:9786b8f5:::10000003100.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:25.301 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 46.399756623999998, 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012109089999999999, 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10851 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892786+0000", 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4819999999999999e-06 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894302+0000", 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015156099999999999 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894319+0000", 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7098000000000001e-05 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894329+0000", 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9180000000000006e-06 2024-09-16T00:10:25.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894394+0000", 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5797999999999996e-05 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894532+0000", 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000138128 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894633+0000", 2024-09-16T00:10:25.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010037100000000001 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.899212+0000", 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045789369999999999 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904771+0000", 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055591169999999997 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.306 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904838+0000", 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7124000000000003e-05 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904892+0000", 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3507000000000001e-05 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10957 3.24 3:2748b9d7:::1000000313b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.777852271, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012163525, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.308 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10957 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514690+0000", 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.382e-06 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.310 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516204+0000", 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001513831 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516232+0000", 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7657999999999998e-05 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516246+0000", 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3953e-05 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516374+0000", 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012882 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516588+0000", 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000213439 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516748+0000", 2024-09-16T00:10:25.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015971900000000001 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.524102+0000", 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073542499999999997 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526714+0000", 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026122910000000001 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526791+0000", 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6997999999999997e-05 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526850+0000", 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9184000000000001e-05 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11098 3.15 e86/20 3:aa8f498f:::100000030ba.00000000:head v 86'170, mlcod=86'168)", 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:25.314 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.271069105000002, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013832641999999999, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:25.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021472+0000", 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9489999999999999e-06 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027628+0000", 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061563859999999998 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027645+0000", 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7108e-05 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027655+0000", 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7999999999999993e-06 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027712+0000", 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6826000000000001e-05 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027738+0000", 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5601000000000001e-05 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.035237+0000", 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074998699999999996 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.035303+0000", 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5102000000000001e-05 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.318 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11099 3.b 3:d2f5b389:::100000030e7.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.267024290000002, 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012707456000000001, 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.319 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11099 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025517+0000", 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026955+0000", 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001437219 2024-09-16T00:10:25.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026982+0000", 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7523999999999999e-05 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026996+0000", 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4379000000000001e-05 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027076+0000", 2024-09-16T00:10:25.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9933999999999997e-05 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027185+0000", 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010875899999999999 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027380+0000", 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019477700000000001 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038072+0000", 2024-09-16T00:10:25.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010692234 2024-09-16T00:10:25.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:25.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038115+0000", 2024-09-16T00:10:25.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3291999999999998e-05 2024-09-16T00:10:25.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038172+0000", 2024-09-16T00:10:25.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6981000000000002e-05 2024-09-16T00:10:25.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038222+0000", 2024-09-16T00:10:25.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9737000000000002e-05 2024-09-16T00:10:25.325 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.325 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.325 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.325 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11109 3.27 3:e5179c63:::10000003176.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:25.325 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:25.325 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.221560007000001, 2024-09-16T00:10:25.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012250990999999999, 2024-09-16T00:10:25.325 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.326 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.326 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.326 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:25.326 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:25.326 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.326 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11109 2024-09-16T00:10:25.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.326 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:25.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:25.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070981+0000", 2024-09-16T00:10:25.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.289e-06 2024-09-16T00:10:25.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072616+0000", 2024-09-16T00:10:25.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016347569999999999 2024-09-16T00:10:25.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072638+0000", 2024-09-16T00:10:25.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1512999999999999e-05 2024-09-16T00:10:25.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072649+0000", 2024-09-16T00:10:25.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1626e-05 2024-09-16T00:10:25.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072775+0000", 2024-09-16T00:10:25.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012553900000000001 2024-09-16T00:10:25.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072856+0000", 2024-09-16T00:10:25.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0882000000000003e-05 2024-09-16T00:10:25.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:25.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072990+0000", 2024-09-16T00:10:25.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013483599999999999 2024-09-16T00:10:25.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:25.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.078354+0000", 2024-09-16T00:10:25.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053639500000000001 2024-09-16T00:10:25.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083150+0000", 2024-09-16T00:10:25.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004795517 2024-09-16T00:10:25.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083186+0000", 2024-09-16T00:10:25.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6019000000000002e-05 2024-09-16T00:10:25.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083230+0000", 2024-09-16T00:10:25.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4063000000000001e-05 2024-09-16T00:10:25.331 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.331 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.331 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.332 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11234 3.b 3:d2f5b389:::100000030e7.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:25.332 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:25.332 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 44.668944238999998, 2024-09-16T00:10:25.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.015217661, 2024-09-16T00:10:25.332 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.332 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.332 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.332 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:25.332 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:25.332 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.332 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11234 2024-09-16T00:10:25.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.332 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.333 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:25.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.333 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:25.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.333 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623598+0000", 2024-09-16T00:10:25.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1580000000000001e-06 2024-09-16T00:10:25.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626844+0000", 2024-09-16T00:10:25.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032459670000000002 2024-09-16T00:10:25.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626868+0000", 2024-09-16T00:10:25.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3584e-05 2024-09-16T00:10:25.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626879+0000", 2024-09-16T00:10:25.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1545999999999999e-05 2024-09-16T00:10:25.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626950+0000", 2024-09-16T00:10:25.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0784999999999994e-05 2024-09-16T00:10:25.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.627051+0000", 2024-09-16T00:10:25.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010155899999999999 2024-09-16T00:10:25.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:25.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.627178+0000", 2024-09-16T00:10:25.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012614899999999999 2024-09-16T00:10:25.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:25.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.635220+0000", 2024-09-16T00:10:25.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0080421490000000002 2024-09-16T00:10:25.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638699+0000", 2024-09-16T00:10:25.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034797500000000002 2024-09-16T00:10:25.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638760+0000", 2024-09-16T00:10:25.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0319000000000003e-05 2024-09-16T00:10:25.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638812+0000", 2024-09-16T00:10:25.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:10:25.337 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.337 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.337 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.338 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11777 3.b 3:d342b4ff:::10000003113.00000000:head [write 0~4194304 [2@1171070]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:25.338 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:25.338 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 42.101304831, 2024-09-16T00:10:25.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013079176, 2024-09-16T00:10:25.338 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.338 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.338 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.338 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:25.338 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:25.338 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.338 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11777 2024-09-16T00:10:25.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.339 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:25.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:25.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191237+0000", 2024-09-16T00:10:25.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3369999999999998e-06 2024-09-16T00:10:25.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.340 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192825+0000", 2024-09-16T00:10:25.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015887189999999999 2024-09-16T00:10:25.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192846+0000", 2024-09-16T00:10:25.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0670999999999999e-05 2024-09-16T00:10:25.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192858+0000", 2024-09-16T00:10:25.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.223e-05 2024-09-16T00:10:25.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192957+0000", 2024-09-16T00:10:25.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8416000000000001e-05 2024-09-16T00:10:25.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.193031+0000", 2024-09-16T00:10:25.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4381999999999994e-05 2024-09-16T00:10:25.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:25.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.193164+0000", 2024-09-16T00:10:25.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013349100000000001 2024-09-16T00:10:25.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:25.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204144+0000", 2024-09-16T00:10:25.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010979917 2024-09-16T00:10:25.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204204+0000", 2024-09-16T00:10:25.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0025999999999998e-05 2024-09-16T00:10:25.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204265+0000", 2024-09-16T00:10:25.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0645e-05 2024-09-16T00:10:25.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204313+0000", 2024-09-16T00:10:25.344 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8341999999999999e-05 2024-09-16T00:10:25.344 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.344 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.344 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.344 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.344 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11981 3.c e87/20 3:33faa5a0:::10000003215.00000000:head v 87'186, mlcod=87'184)", 2024-09-16T00:10:25.344 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:25.344 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.461070008, 2024-09-16T00:10:25.344 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011828697000000001, 2024-09-16T00:10:25.344 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.344 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.344 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.344 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:25.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:25.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831471+0000", 2024-09-16T00:10:25.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6220000000000001e-06 2024-09-16T00:10:25.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.346 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836584+0000", 2024-09-16T00:10:25.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051133510000000004 2024-09-16T00:10:25.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.346 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836606+0000", 2024-09-16T00:10:25.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2079e-05 2024-09-16T00:10:25.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836624+0000", 2024-09-16T00:10:25.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8162e-05 2024-09-16T00:10:25.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836704+0000", 2024-09-16T00:10:25.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9867e-05 2024-09-16T00:10:25.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836730+0000", 2024-09-16T00:10:25.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6285999999999999e-05 2024-09-16T00:10:25.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.843247+0000", 2024-09-16T00:10:25.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0065165650000000002 2024-09-16T00:10:25.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.843298+0000", 2024-09-16T00:10:25.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0764999999999999e-05 2024-09-16T00:10:25.348 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.348 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.348 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.349 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13365 3.2 3:4298088d:::100000035be.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:25.349 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:25.349 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.273368028999997, 2024-09-16T00:10:25.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014036442999999999, 2024-09-16T00:10:25.349 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.349 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.350 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.350 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:25.350 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:25.350 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.352 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13365 2024-09-16T00:10:25.352 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.352 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.352 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.352 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:25.352 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.352 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.352 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.352 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:25.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019174+0000", 2024-09-16T00:10:25.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1250000000000001e-06 2024-09-16T00:10:25.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020613+0000", 2024-09-16T00:10:25.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014391930000000001 2024-09-16T00:10:25.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020638+0000", 2024-09-16T00:10:25.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4657000000000001e-05 2024-09-16T00:10:25.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020651+0000", 2024-09-16T00:10:25.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3451e-05 2024-09-16T00:10:25.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020744+0000", 2024-09-16T00:10:25.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3010000000000006e-05 2024-09-16T00:10:25.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020912+0000", 2024-09-16T00:10:25.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016763800000000001 2024-09-16T00:10:25.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:25.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.021043+0000", 2024-09-16T00:10:25.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013099800000000001 2024-09-16T00:10:25.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:25.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026194+0000", 2024-09-16T00:10:25.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051510879999999998 2024-09-16T00:10:25.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033132+0000", 2024-09-16T00:10:25.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069377279999999998 2024-09-16T00:10:25.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033169+0000", 2024-09-16T00:10:25.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6897000000000003e-05 2024-09-16T00:10:25.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033207+0000", 2024-09-16T00:10:25.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658000000000002e-05 2024-09-16T00:10:25.357 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.357 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.357 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.358 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13368 3.27 3:e75b72a1:::100000032e8.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:25.358 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:25.358 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.266002145999998, 2024-09-16T00:10:25.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013615175, 2024-09-16T00:10:25.358 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.358 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.358 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.358 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:25.358 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:25.358 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.358 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13368 2024-09-16T00:10:25.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.359 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:25.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:25.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026539+0000", 2024-09-16T00:10:25.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.798e-06 2024-09-16T00:10:25.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027825+0000", 2024-09-16T00:10:25.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012861839999999999 2024-09-16T00:10:25.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027850+0000", 2024-09-16T00:10:25.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5048000000000001e-05 2024-09-16T00:10:25.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027865+0000", 2024-09-16T00:10:25.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4939e-05 2024-09-16T00:10:25.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027944+0000", 2024-09-16T00:10:25.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8881999999999995e-05 2024-09-16T00:10:25.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.028043+0000", 2024-09-16T00:10:25.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9135000000000005e-05 2024-09-16T00:10:25.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:25.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.028158+0000", 2024-09-16T00:10:25.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011527999999999999 2024-09-16T00:10:25.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.039292+0000", 2024-09-16T00:10:25.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011133568999999999 2024-09-16T00:10:25.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:25.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.039948+0000", 2024-09-16T00:10:25.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00065625700000000004 2024-09-16T00:10:25.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.040061+0000", 2024-09-16T00:10:25.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011284400000000001 2024-09-16T00:10:25.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.040152+0000", 2024-09-16T00:10:25.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1238999999999998e-05 2024-09-16T00:10:25.363 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.363 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.363 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.364 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:13366 3.15 e88/20 3:aa8f498f:::100000030ba.00000000:head v 88'196, mlcod=88'194)", 2024-09-16T00:10:25.364 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:25.364 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.262577071000003, 2024-09-16T00:10:25.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012822817, 2024-09-16T00:10:25.364 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.364 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.364 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.364 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.364 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.364 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:25.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:25.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029963+0000", 2024-09-16T00:10:25.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4270000000000001e-06 2024-09-16T00:10:25.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034078+0000", 2024-09-16T00:10:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041148360000000002 2024-09-16T00:10:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034094+0000", 2024-09-16T00:10:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.535e-05 2024-09-16T00:10:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034103+0000", 2024-09-16T00:10:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:10:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.036791+0000", 2024-09-16T00:10:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026878340000000001 2024-09-16T00:10:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.036818+0000", 2024-09-16T00:10:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6885000000000001e-05 2024-09-16T00:10:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.042713+0000", 2024-09-16T00:10:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058950399999999998 2024-09-16T00:10:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.042785+0000", 2024-09-16T00:10:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1927000000000004e-05 2024-09-16T00:10:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13405 3.27 3:e708eda4:::100000034c4.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 37.136375358000002, 2024-09-16T00:10:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012003333, 2024-09-16T00:10:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.368 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13405 2024-09-16T00:10:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156166+0000", 2024-09-16T00:10:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6230000000000001e-06 2024-09-16T00:10:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157868+0000", 2024-09-16T00:10:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017017 2024-09-16T00:10:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157891+0000", 2024-09-16T00:10:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2931000000000001e-05 2024-09-16T00:10:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157904+0000", 2024-09-16T00:10:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2763e-05 2024-09-16T00:10:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157988+0000", 2024-09-16T00:10:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3849000000000003e-05 2024-09-16T00:10:25.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.158083+0000", 2024-09-16T00:10:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5939000000000004e-05 2024-09-16T00:10:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.158194+0000", 2024-09-16T00:10:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110963 2024-09-16T00:10:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.164635+0000", 2024-09-16T00:10:25.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0064405570000000004 2024-09-16T00:10:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168088+0000", 2024-09-16T00:10:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034534639999999998 2024-09-16T00:10:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168140+0000", 2024-09-16T00:10:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1041000000000001e-05 2024-09-16T00:10:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168167+0000", 2024-09-16T00:10:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7503000000000001e-05 2024-09-16T00:10:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13685 3.b 3:d26c1580:::100000032e4.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 35.957688392999998, 2024-09-16T00:10:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014868124999999999, 2024-09-16T00:10:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:25.374 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13685 2024-09-16T00:10:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:25.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334854+0000", 2024-09-16T00:10:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0000000000000001e-06 2024-09-16T00:10:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337155+0000", 2024-09-16T00:10:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023016870000000002 2024-09-16T00:10:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337178+0000", 2024-09-16T00:10:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2898e-05 2024-09-16T00:10:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337190+0000", 2024-09-16T00:10:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1817999999999999e-05 2024-09-16T00:10:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337301+0000", 2024-09-16T00:10:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110951 2024-09-16T00:10:25.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337403+0000", 2024-09-16T00:10:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000102118 2024-09-16T00:10:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337545+0000", 2024-09-16T00:10:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014167199999999999 2024-09-16T00:10:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.346336+0000", 2024-09-16T00:10:25.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087911450000000002 2024-09-16T00:10:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349641+0000", 2024-09-16T00:10:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033047570000000002 2024-09-16T00:10:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349680+0000", 2024-09-16T00:10:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9054999999999998e-05 2024-09-16T00:10:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349719+0000", 2024-09-16T00:10:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9023999999999998e-05 2024-09-16T00:10:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14624 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.677929192000001, 2024-09-16T00:10:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012123376, 2024-09-16T00:10:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.380 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14624 2024-09-16T00:10:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614614+0000", 2024-09-16T00:10:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9500000000000003e-06 2024-09-16T00:10:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616480+0000", 2024-09-16T00:10:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018663169999999999 2024-09-16T00:10:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616506+0000", 2024-09-16T00:10:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6205999999999999e-05 2024-09-16T00:10:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616517+0000", 2024-09-16T00:10:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0399e-05 2024-09-16T00:10:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616595+0000", 2024-09-16T00:10:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8170999999999994e-05 2024-09-16T00:10:25.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616683+0000", 2024-09-16T00:10:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8578999999999993e-05 2024-09-16T00:10:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616825+0000", 2024-09-16T00:10:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014186299999999999 2024-09-16T00:10:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:25.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.624948+0000", 2024-09-16T00:10:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081230420000000005 2024-09-16T00:10:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626617+0000", 2024-09-16T00:10:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016681980000000001 2024-09-16T00:10:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626678+0000", 2024-09-16T00:10:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1362e-05 2024-09-16T00:10:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626733+0000", 2024-09-16T00:10:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5288999999999999e-05 2024-09-16T00:10:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14672 3.29 3:9641a85f:::1000000360e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 32.473472014999999, 2024-09-16T00:10:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012258025000000001, 2024-09-16T00:10:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.386 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14672 2024-09-16T00:10:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819070+0000", 2024-09-16T00:10:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0589999999999998e-06 2024-09-16T00:10:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821701+0000", 2024-09-16T00:10:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026309100000000002 2024-09-16T00:10:25.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821727+0000", 2024-09-16T00:10:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6372000000000001e-05 2024-09-16T00:10:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821738+0000", 2024-09-16T00:10:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0759e-05 2024-09-16T00:10:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821821+0000", 2024-09-16T00:10:25.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2854000000000003e-05 2024-09-16T00:10:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821920+0000", 2024-09-16T00:10:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8900999999999994e-05 2024-09-16T00:10:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.822084+0000", 2024-09-16T00:10:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016432799999999999 2024-09-16T00:10:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.829148+0000", 2024-09-16T00:10:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0070641460000000003 2024-09-16T00:10:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831242+0000", 2024-09-16T00:10:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002093293 2024-09-16T00:10:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831280+0000", 2024-09-16T00:10:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8707000000000001e-05 2024-09-16T00:10:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831325+0000", 2024-09-16T00:10:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4696e-05 2024-09-16T00:10:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14829 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304 [2@2556691]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.720952597, 2024-09-16T00:10:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014399683, 2024-09-16T00:10:25.392 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14829 2024-09-16T00:10:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571590+0000", 2024-09-16T00:10:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:10:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573133+0000", 2024-09-16T00:10:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015422109999999999 2024-09-16T00:10:25.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573161+0000", 2024-09-16T00:10:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8045e-05 2024-09-16T00:10:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573174+0000", 2024-09-16T00:10:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3193999999999999e-05 2024-09-16T00:10:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573365+0000", 2024-09-16T00:10:25.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019096400000000001 2024-09-16T00:10:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573467+0000", 2024-09-16T00:10:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101789 2024-09-16T00:10:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573610+0000", 2024-09-16T00:10:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143404 2024-09-16T00:10:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.583614+0000", 2024-09-16T00:10:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010004422000000001 2024-09-16T00:10:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585556+0000", 2024-09-16T00:10:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019414009999999999 2024-09-16T00:10:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585939+0000", 2024-09-16T00:10:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038340999999999999 2024-09-16T00:10:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585986+0000", 2024-09-16T00:10:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6924e-05 2024-09-16T00:10:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14914 3.b 3:d23f6103:::10000003277.00000000:head [write 0~4194304 [3@58131]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.409748826000001, 2024-09-16T00:10:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013007654, 2024-09-16T00:10:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.398 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14914 2024-09-16T00:10:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882793+0000", 2024-09-16T00:10:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.65e-06 2024-09-16T00:10:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885193+0000", 2024-09-16T00:10:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024003449999999999 2024-09-16T00:10:25.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885214+0000", 2024-09-16T00:10:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0398e-05 2024-09-16T00:10:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885224+0000", 2024-09-16T00:10:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0253e-05 2024-09-16T00:10:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885299+0000", 2024-09-16T00:10:25.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5434999999999999e-05 2024-09-16T00:10:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885389+0000", 2024-09-16T00:10:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9209999999999995e-05 2024-09-16T00:10:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885485+0000", 2024-09-16T00:10:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6218999999999995e-05 2024-09-16T00:10:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.893457+0000", 2024-09-16T00:10:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0079722009999999999 2024-09-16T00:10:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895681+0000", 2024-09-16T00:10:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022241449999999999 2024-09-16T00:10:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895732+0000", 2024-09-16T00:10:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1269e-05 2024-09-16T00:10:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895798+0000", 2024-09-16T00:10:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5529000000000002e-05 2024-09-16T00:10:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15058 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304 [2@2556691]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-16T00:10:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.957380231999998, 2024-09-16T00:10:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012275842, 2024-09-16T00:10:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.404 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15058 2024-09-16T00:10:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:25.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335161+0000", 2024-09-16T00:10:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.367e-06 2024-09-16T00:10:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337545+0000", 2024-09-16T00:10:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023839040000000001 2024-09-16T00:10:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337572+0000", 2024-09-16T00:10:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.673e-05 2024-09-16T00:10:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337583+0000", 2024-09-16T00:10:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1048e-05 2024-09-16T00:10:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337660+0000", 2024-09-16T00:10:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7459999999999994e-05 2024-09-16T00:10:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337770+0000", 2024-09-16T00:10:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109634 2024-09-16T00:10:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337926+0000", 2024-09-16T00:10:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015601999999999999 2024-09-16T00:10:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.345689+0000", 2024-09-16T00:10:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0077625630000000001 2024-09-16T00:10:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347324+0000", 2024-09-16T00:10:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016351479999999999 2024-09-16T00:10:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347387+0000", 2024-09-16T00:10:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3199000000000003e-05 2024-09-16T00:10:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347435+0000", 2024-09-16T00:10:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7769000000000003e-05 2024-09-16T00:10:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9780 2.a 2:545c15c7:::200.00000017:head [write 1367106~16252 [fadvise_dontneed]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e106)", 2024-09-16T00:10:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.9551961079999991, 2024-09-16T00:10:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0208196810000001, 2024-09-16T00:10:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:25.410 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:10:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337345+0000", 2024-09-16T00:10:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.007e-06 2024-09-16T00:10:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337416+0000", 2024-09-16T00:10:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1260999999999994e-05 2024-09-16T00:10:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337427+0000", 2024-09-16T00:10:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1225e-05 2024-09-16T00:10:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337434+0000", 2024-09-16T00:10:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8929999999999999e-06 2024-09-16T00:10:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337494+0000", 2024-09-16T00:10:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9670999999999998e-05 2024-09-16T00:10:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337505+0000", 2024-09-16T00:10:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0934999999999999e-05 2024-09-16T00:10:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.345708+0000", 2024-09-16T00:10:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0082033290000001 2024-09-16T00:10:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for missing object", 2024-09-16T00:10:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.345819+0000", 2024-09-16T00:10:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110299 2024-09-16T00:10:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.346937+0000", 2024-09-16T00:10:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011184120000000001 2024-09-16T00:10:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for degraded object", 2024-09-16T00:10:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.346968+0000", 2024-09-16T00:10:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0664999999999999e-05 2024-09-16T00:10:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.358107+0000", 2024-09-16T00:10:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011139046 2024-09-16T00:10:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:25.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:25.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.358163+0000", 2024-09-16T00:10:25.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5937999999999999e-05 2024-09-16T00:10:25.416 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.416 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.416 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.416 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:25.416 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:25.416 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:25.417 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:10:25.418 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:25.416+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:25.518 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:25.517+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:25.526 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:25.527 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:25.527 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:25.527 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:25.536 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:10:25.620 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:25.619+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:25.714 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:25.714 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:25.714 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:25.714 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:25.714 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:25.719 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:25.719+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:25.723 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:10:25.820 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:25.819+0000 7fd693b63640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 36666) UID: 0 2024-09-16T00:10:25.872 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10780 3.a 3:500c83b2:::100000030b4.00000000:head [write 0~4194304 [3@2137521]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.417278262000004, 2024-09-16T00:10:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015526478, 2024-09-16T00:10:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10780 2024-09-16T00:10:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:25.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452917+0000", 2024-09-16T00:10:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2180000000000002e-06 2024-09-16T00:10:25.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462177+0000", 2024-09-16T00:10:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092593769999999992 2024-09-16T00:10:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462196+0000", 2024-09-16T00:10:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9745000000000002e-05 2024-09-16T00:10:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462207+0000", 2024-09-16T00:10:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0524e-05 2024-09-16T00:10:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462284+0000", 2024-09-16T00:10:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7608000000000006e-05 2024-09-16T00:10:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462376+0000", 2024-09-16T00:10:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1811999999999994e-05 2024-09-16T00:10:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468357+0000", 2024-09-16T00:10:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059806809999999998 2024-09-16T00:10:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468386+0000", 2024-09-16T00:10:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9068999999999999e-05 2024-09-16T00:10:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468441+0000", 2024-09-16T00:10:25.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5444e-05 2024-09-16T00:10:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10831 3.1a 3:59b8f98c:::100000030df.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.094471126000002, 2024-09-16T00:10:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014692843000000001, 2024-09-16T00:10:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.879 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10831 2024-09-16T00:10:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775723+0000", 2024-09-16T00:10:25.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.367e-06 2024-09-16T00:10:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780402+0000", 2024-09-16T00:10:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046785769999999997 2024-09-16T00:10:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780421+0000", 2024-09-16T00:10:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9179000000000001e-05 2024-09-16T00:10:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780440+0000", 2024-09-16T00:10:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9015e-05 2024-09-16T00:10:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780533+0000", 2024-09-16T00:10:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3197999999999998e-05 2024-09-16T00:10:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780618+0000", 2024-09-16T00:10:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4787e-05 2024-09-16T00:10:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780748+0000", 2024-09-16T00:10:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012927199999999999 2024-09-16T00:10:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.785049+0000", 2024-09-16T00:10:25.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043015750000000002 2024-09-16T00:10:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790312+0000", 2024-09-16T00:10:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052633319999999999 2024-09-16T00:10:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790369+0000", 2024-09-16T00:10:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6848999999999997e-05 2024-09-16T00:10:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:25.885 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790415+0000", 2024-09-16T00:10:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5692000000000002e-05 2024-09-16T00:10:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10915 3.1a 3:5a6e15a2:::1000000318e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.654965205000003, 2024-09-16T00:10:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014599472, 2024-09-16T00:10:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.886 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10915 2024-09-16T00:10:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:25.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215230+0000", 2024-09-16T00:10:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0090000000000002e-06 2024-09-16T00:10:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219556+0000", 2024-09-16T00:10:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043259539999999999 2024-09-16T00:10:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219577+0000", 2024-09-16T00:10:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1271999999999999e-05 2024-09-16T00:10:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219589+0000", 2024-09-16T00:10:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1766000000000001e-05 2024-09-16T00:10:25.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219663+0000", 2024-09-16T00:10:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4196999999999995e-05 2024-09-16T00:10:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219733+0000", 2024-09-16T00:10:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9553000000000003e-05 2024-09-16T00:10:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219827+0000", 2024-09-16T00:10:25.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4395999999999995e-05 2024-09-16T00:10:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.225724+0000", 2024-09-16T00:10:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058968359999999999 2024-09-16T00:10:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229705+0000", 2024-09-16T00:10:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003981409 2024-09-16T00:10:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.891 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229772+0000", 2024-09-16T00:10:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6137999999999996e-05 2024-09-16T00:10:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229828+0000", 2024-09-16T00:10:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5942000000000001e-05 2024-09-16T00:10:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11074 3.14 3:2834b7ee:::10000003131.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:25.892 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.957425082, 2024-09-16T00:10:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015588350000000001, 2024-09-16T00:10:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11074 2024-09-16T00:10:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912769+0000", 2024-09-16T00:10:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4600000000000004e-07 2024-09-16T00:10:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918264+0000", 2024-09-16T00:10:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054954890000000001 2024-09-16T00:10:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918281+0000", 2024-09-16T00:10:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6160000000000001e-05 2024-09-16T00:10:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918289+0000", 2024-09-16T00:10:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1650000000000006e-06 2024-09-16T00:10:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918386+0000", 2024-09-16T00:10:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7288999999999996e-05 2024-09-16T00:10:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918535+0000", 2024-09-16T00:10:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014876200000000001 2024-09-16T00:10:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918639+0000", 2024-09-16T00:10:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010470400000000001 2024-09-16T00:10:25.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.924703+0000", 2024-09-16T00:10:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060639759999999996 2024-09-16T00:10:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928246+0000", 2024-09-16T00:10:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035421720000000001 2024-09-16T00:10:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.897 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928305+0000", 2024-09-16T00:10:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9014000000000001e-05 2024-09-16T00:10:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928357+0000", 2024-09-16T00:10:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1873000000000003e-05 2024-09-16T00:10:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.898 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11111 3.9 3:93301cc4:::10000003210.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.780686879999998, 2024-09-16T00:10:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014336099999999999, 2024-09-16T00:10:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.899 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11111 2024-09-16T00:10:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089520+0000", 2024-09-16T00:10:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3903000000000001e-05 2024-09-16T00:10:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094463+0000", 2024-09-16T00:10:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049423000000000002 2024-09-16T00:10:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094494+0000", 2024-09-16T00:10:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1029999999999999e-05 2024-09-16T00:10:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094503+0000", 2024-09-16T00:10:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0180000000000007e-06 2024-09-16T00:10:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094573+0000", 2024-09-16T00:10:25.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0457000000000007e-05 2024-09-16T00:10:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094737+0000", 2024-09-16T00:10:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016408699999999999 2024-09-16T00:10:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094897+0000", 2024-09-16T00:10:25.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015938899999999999 2024-09-16T00:10:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.102326+0000", 2024-09-16T00:10:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074291230000000002 2024-09-16T00:10:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103723+0000", 2024-09-16T00:10:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001397255 2024-09-16T00:10:25.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103786+0000", 2024-09-16T00:10:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3489000000000001e-05 2024-09-16T00:10:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103842+0000", 2024-09-16T00:10:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6048999999999998e-05 2024-09-16T00:10:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:11934 3.1b e87/20 3:d94fa8da:::10000003190.00000000:head v 87'152, mlcod=87'150)", 2024-09-16T00:10:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.179942183999998, 2024-09-16T00:10:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014984934, 2024-09-16T00:10:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:25.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:25.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:25.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:25.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:25.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690252+0000", 2024-09-16T00:10:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8899999999999998e-07 2024-09-16T00:10:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696026+0000", 2024-09-16T00:10:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057745219999999998 2024-09-16T00:10:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:25.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696039+0000", 2024-09-16T00:10:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2611e-05 2024-09-16T00:10:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696047+0000", 2024-09-16T00:10:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5399999999999998e-06 2024-09-16T00:10:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696123+0000", 2024-09-16T00:10:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6349999999999999e-05 2024-09-16T00:10:25.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696144+0000", 2024-09-16T00:10:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1512999999999999e-05 2024-09-16T00:10:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705188+0000", 2024-09-16T00:10:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090438459999999995 2024-09-16T00:10:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705236+0000", 2024-09-16T00:10:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7663000000000001e-05 2024-09-16T00:10:25.910 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11936 3.3 3:c0886fb9:::100000031a1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.169797131999999, 2024-09-16T00:10:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015336379000000001, 2024-09-16T00:10:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:25.911 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11936 2024-09-16T00:10:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700399+0000", 2024-09-16T00:10:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6029999999999999e-06 2024-09-16T00:10:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704706+0000", 2024-09-16T00:10:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043070529999999999 2024-09-16T00:10:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704720+0000", 2024-09-16T00:10:25.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4629e-05 2024-09-16T00:10:25.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704729+0000", 2024-09-16T00:10:25.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6009999999999995e-06 2024-09-16T00:10:25.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:25.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704805+0000", 2024-09-16T00:10:25.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6015999999999999e-05 2024-09-16T00:10:25.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:25.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704866+0000", 2024-09-16T00:10:25.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0878999999999998e-05 2024-09-16T00:10:25.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:25.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704990+0000", 2024-09-16T00:10:25.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000124058 2024-09-16T00:10:25.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.714314+0000", 2024-09-16T00:10:25.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009324321 2024-09-16T00:10:25.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:25.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715632+0000", 2024-09-16T00:10:25.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001318121 2024-09-16T00:10:25.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:25.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715687+0000", 2024-09-16T00:10:25.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4135999999999998e-05 2024-09-16T00:10:25.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:25.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715732+0000", 2024-09-16T00:10:25.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961999999999998e-05 2024-09-16T00:10:25.916 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.917 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:25.917 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.917 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11953 3.1a 3:580e2503:::1000000311b.00000000:head [write 0~4194304 [2@2058066]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:25.917 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:25.917 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.103457978999998, 2024-09-16T00:10:25.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016299973999999998, 2024-09-16T00:10:25.917 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.917 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.917 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.917 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:25.917 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:25.918 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.918 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11953 2024-09-16T00:10:25.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.918 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:25.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:25.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:25.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:25.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:25.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:25.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766741+0000", 2024-09-16T00:10:25.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2939999999999996e-06 2024-09-16T00:10:25.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:25.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771165+0000", 2024-09-16T00:10:25.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044248229999999996 2024-09-16T00:10:25.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:25.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771181+0000", 2024-09-16T00:10:25.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5243e-05 2024-09-16T00:10:25.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771188+0000", 2024-09-16T00:10:25.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6429999999999995e-06 2024-09-16T00:10:25.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:25.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771238+0000", 2024-09-16T00:10:25.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9567999999999997e-05 2024-09-16T00:10:25.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:25.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771293+0000", 2024-09-16T00:10:25.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5207000000000002e-05 2024-09-16T00:10:25.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:25.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771380+0000", 2024-09-16T00:10:25.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6716000000000001e-05 2024-09-16T00:10:25.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:25.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.776193+0000", 2024-09-16T00:10:25.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048133350000000002 2024-09-16T00:10:25.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.782931+0000", 2024-09-16T00:10:25.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067376550000000004 2024-09-16T00:10:25.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:25.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.782987+0000", 2024-09-16T00:10:25.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6443000000000002e-05 2024-09-16T00:10:25.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:25.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.783035+0000", 2024-09-16T00:10:25.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8047000000000003e-05 2024-09-16T00:10:25.923 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.923 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:25.923 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.923 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12037 3.9 3:928db521:::1000000326d.00000000:head [write 0~4194304 [2@1655200]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:25.923 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:25.923 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.872250371, 2024-09-16T00:10:25.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017082494, 2024-09-16T00:10:25.923 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.924 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.924 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.924 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:25.924 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:25.924 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.924 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12037 2024-09-16T00:10:25.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.924 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:25.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:25.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:25.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:25.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:25.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:25.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997944+0000", 2024-09-16T00:10:25.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.517e-06 2024-09-16T00:10:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002177+0000", 2024-09-16T00:10:25.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042329300000000002 2024-09-16T00:10:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002197+0000", 2024-09-16T00:10:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9769e-05 2024-09-16T00:10:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002207+0000", 2024-09-16T00:10:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0178e-05 2024-09-16T00:10:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002266+0000", 2024-09-16T00:10:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9215999999999997e-05 2024-09-16T00:10:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002340+0000", 2024-09-16T00:10:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3798000000000001e-05 2024-09-16T00:10:25.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002454+0000", 2024-09-16T00:10:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113551 2024-09-16T00:10:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.006270+0000", 2024-09-16T00:10:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038163839999999999 2024-09-16T00:10:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.014957+0000", 2024-09-16T00:10:25.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086869119999999998 2024-09-16T00:10:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.014996+0000", 2024-09-16T00:10:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8878000000000003e-05 2024-09-16T00:10:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.015025+0000", 2024-09-16T00:10:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9360999999999999e-05 2024-09-16T00:10:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12245 3.1a 3:599a1cd1:::10000003252.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.087552954000003, 2024-09-16T00:10:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017131203000000001, 2024-09-16T00:10:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12245 2024-09-16T00:10:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.931 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:25.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782642+0000", 2024-09-16T00:10:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.311e-06 2024-09-16T00:10:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786551+0000", 2024-09-16T00:10:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039095229999999998 2024-09-16T00:10:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786566+0000", 2024-09-16T00:10:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4724e-05 2024-09-16T00:10:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786575+0000", 2024-09-16T00:10:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8449999999999992e-06 2024-09-16T00:10:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786636+0000", 2024-09-16T00:10:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.143e-05 2024-09-16T00:10:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:25.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786699+0000", 2024-09-16T00:10:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3218000000000004e-05 2024-09-16T00:10:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786796+0000", 2024-09-16T00:10:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6347000000000004e-05 2024-09-16T00:10:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.792369+0000", 2024-09-16T00:10:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055729780000000001 2024-09-16T00:10:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799706+0000", 2024-09-16T00:10:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073374349999999998 2024-09-16T00:10:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799739+0000", 2024-09-16T00:10:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2833999999999998e-05 2024-09-16T00:10:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:25.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799771+0000", 2024-09-16T00:10:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2558000000000002e-05 2024-09-16T00:10:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12292 3.3 3:c00779fd:::1000000320e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:25.937 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 40.973162856999998, 2024-09-16T00:10:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013828353999999999, 2024-09-16T00:10:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:25.938 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12292 2024-09-16T00:10:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:25.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897039+0000", 2024-09-16T00:10:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3299999999999999e-06 2024-09-16T00:10:25.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900830+0000", 2024-09-16T00:10:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037915340000000001 2024-09-16T00:10:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900850+0000", 2024-09-16T00:10:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9400000000000001e-05 2024-09-16T00:10:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900860+0000", 2024-09-16T00:10:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0609e-05 2024-09-16T00:10:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900928+0000", 2024-09-16T00:10:25.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7570000000000005e-05 2024-09-16T00:10:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:25.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.901027+0000", 2024-09-16T00:10:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9456000000000005e-05 2024-09-16T00:10:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.901168+0000", 2024-09-16T00:10:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014082799999999999 2024-09-16T00:10:25.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.906325+0000", 2024-09-16T00:10:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005156702 2024-09-16T00:10:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910796+0000", 2024-09-16T00:10:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044714350000000002 2024-09-16T00:10:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910829+0000", 2024-09-16T00:10:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3229000000000003e-05 2024-09-16T00:10:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910859+0000", 2024-09-16T00:10:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9261e-05 2024-09-16T00:10:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:25.950 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.952 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13326 3.9 3:92f796cf:::1000000350f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.913472575, 2024-09-16T00:10:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015673229, 2024-09-16T00:10:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:25.953 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13326 2024-09-16T00:10:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:25.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956722+0000", 2024-09-16T00:10:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3200000000000001e-06 2024-09-16T00:10:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960803+0000", 2024-09-16T00:10:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040807029999999998 2024-09-16T00:10:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:25.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960820+0000", 2024-09-16T00:10:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7478000000000001e-05 2024-09-16T00:10:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960830+0000", 2024-09-16T00:10:25.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:10:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960903+0000", 2024-09-16T00:10:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3057000000000002e-05 2024-09-16T00:10:25.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960970+0000", 2024-09-16T00:10:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7063000000000005e-05 2024-09-16T00:10:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.961067+0000", 2024-09-16T00:10:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6906999999999999e-05 2024-09-16T00:10:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.967945+0000", 2024-09-16T00:10:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068787359999999999 2024-09-16T00:10:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972328+0000", 2024-09-16T00:10:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043826170000000001 2024-09-16T00:10:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972363+0000", 2024-09-16T00:10:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4486000000000002e-05 2024-09-16T00:10:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972394+0000", 2024-09-16T00:10:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1343999999999997e-05 2024-09-16T00:10:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.965 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13429 3.1a 3:58dcd24a:::100000036d5.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.558902559000003, 2024-09-16T00:10:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014237671, 2024-09-16T00:10:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:25.966 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13429 2024-09-16T00:10:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311292+0000", 2024-09-16T00:10:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4890000000000001e-06 2024-09-16T00:10:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316378+0000", 2024-09-16T00:10:25.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050859290000000003 2024-09-16T00:10:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316399+0000", 2024-09-16T00:10:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1345e-05 2024-09-16T00:10:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316410+0000", 2024-09-16T00:10:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0295999999999999e-05 2024-09-16T00:10:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316493+0000", 2024-09-16T00:10:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3533000000000001e-05 2024-09-16T00:10:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316622+0000", 2024-09-16T00:10:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012887900000000001 2024-09-16T00:10:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316725+0000", 2024-09-16T00:10:25.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010295599999999999 2024-09-16T00:10:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.320948+0000", 2024-09-16T00:10:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042229820000000001 2024-09-16T00:10:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325415+0000", 2024-09-16T00:10:25.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044673849999999999 2024-09-16T00:10:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325487+0000", 2024-09-16T00:10:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1087999999999994e-05 2024-09-16T00:10:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325528+0000", 2024-09-16T00:10:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1789000000000003e-05 2024-09-16T00:10:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:25.972 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13527 3.3 3:c0886fb9:::100000031a1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.084742470000002, 2024-09-16T00:10:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015187236, 2024-09-16T00:10:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:25.973 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13527 2024-09-16T00:10:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:25.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785453+0000", 2024-09-16T00:10:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1689999999999999e-06 2024-09-16T00:10:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791191+0000", 2024-09-16T00:10:25.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057378339999999998 2024-09-16T00:10:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791217+0000", 2024-09-16T00:10:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6480999999999999e-05 2024-09-16T00:10:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791233+0000", 2024-09-16T00:10:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5841e-05 2024-09-16T00:10:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791319+0000", 2024-09-16T00:10:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5527999999999998e-05 2024-09-16T00:10:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791413+0000", 2024-09-16T00:10:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4685000000000006e-05 2024-09-16T00:10:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791563+0000", 2024-09-16T00:10:25.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000149624 2024-09-16T00:10:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.795749+0000", 2024-09-16T00:10:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004186286 2024-09-16T00:10:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800577+0000", 2024-09-16T00:10:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048273040000000001 2024-09-16T00:10:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800611+0000", 2024-09-16T00:10:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4270000000000002e-05 2024-09-16T00:10:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800638+0000", 2024-09-16T00:10:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7214000000000001e-05 2024-09-16T00:10:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.979 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13724 3.9 3:90613e6c:::100000034c4.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 36.414670725000001, 2024-09-16T00:10:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014039800999999999, 2024-09-16T00:10:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.980 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13724 2024-09-16T00:10:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455524+0000", 2024-09-16T00:10:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.437e-06 2024-09-16T00:10:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459592+0000", 2024-09-16T00:10:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040678729999999996 2024-09-16T00:10:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459610+0000", 2024-09-16T00:10:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7688000000000001e-05 2024-09-16T00:10:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459617+0000", 2024-09-16T00:10:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.374e-06 2024-09-16T00:10:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459678+0000", 2024-09-16T00:10:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1346999999999994e-05 2024-09-16T00:10:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459824+0000", 2024-09-16T00:10:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014586699999999999 2024-09-16T00:10:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459908+0000", 2024-09-16T00:10:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3521000000000002e-05 2024-09-16T00:10:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.464227+0000", 2024-09-16T00:10:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043197060000000004 2024-09-16T00:10:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469499+0000", 2024-09-16T00:10:25.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005271838 2024-09-16T00:10:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469534+0000", 2024-09-16T00:10:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4712000000000002e-05 2024-09-16T00:10:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:25.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469562+0000", 2024-09-16T00:10:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8438000000000001e-05 2024-09-16T00:10:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13741 3.3 3:c1377c76:::10000003664.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:25.988 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 36.36750078, 2024-09-16T00:10:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016142843, 2024-09-16T00:10:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:25.989 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13741 2024-09-16T00:10:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502694+0000", 2024-09-16T00:10:25.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0449999999999999e-06 2024-09-16T00:10:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507229+0000", 2024-09-16T00:10:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045357410000000003 2024-09-16T00:10:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:25.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507248+0000", 2024-09-16T00:10:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9127000000000002e-05 2024-09-16T00:10:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507259+0000", 2024-09-16T00:10:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0448999999999999e-05 2024-09-16T00:10:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507357+0000", 2024-09-16T00:10:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8400000000000007e-05 2024-09-16T00:10:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507514+0000", 2024-09-16T00:10:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000157195 2024-09-16T00:10:25.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507617+0000", 2024-09-16T00:10:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010262199999999999 2024-09-16T00:10:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:25.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.514047+0000", 2024-09-16T00:10:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006430018 2024-09-16T00:10:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518743+0000", 2024-09-16T00:10:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046963040000000001 2024-09-16T00:10:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518796+0000", 2024-09-16T00:10:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2305000000000002e-05 2024-09-16T00:10:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518835+0000", 2024-09-16T00:10:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9637e-05 2024-09-16T00:10:25.997 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14633 3.9 3:9170e82c:::100000035d6.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.248120524000001, 2024-09-16T00:10:25.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014466105, 2024-09-16T00:10:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14633 2024-09-16T00:10:25.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:26.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622074+0000", 2024-09-16T00:10:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.761e-06 2024-09-16T00:10:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626712+0000", 2024-09-16T00:10:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004637263 2024-09-16T00:10:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626724+0000", 2024-09-16T00:10:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2479e-05 2024-09-16T00:10:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626730+0000", 2024-09-16T00:10:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7309999999999998e-06 2024-09-16T00:10:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626791+0000", 2024-09-16T00:10:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0854999999999999e-05 2024-09-16T00:10:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626857+0000", 2024-09-16T00:10:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6549000000000003e-05 2024-09-16T00:10:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626943+0000", 2024-09-16T00:10:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6141000000000001e-05 2024-09-16T00:10:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.630854+0000", 2024-09-16T00:10:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039100790000000003 2024-09-16T00:10:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636468+0000", 2024-09-16T00:10:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056140570000000004 2024-09-16T00:10:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636500+0000", 2024-09-16T00:10:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2441999999999999e-05 2024-09-16T00:10:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636539+0000", 2024-09-16T00:10:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8748000000000002e-05 2024-09-16T00:10:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14667 3.9 3:9379a4af:::10000003352.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:26.005 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 33.073108517999998, 2024-09-16T00:10:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014580585, 2024-09-16T00:10:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14667 2024-09-16T00:10:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797086+0000", 2024-09-16T00:10:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.362e-06 2024-09-16T00:10:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801415+0000", 2024-09-16T00:10:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043288449999999996 2024-09-16T00:10:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801435+0000", 2024-09-16T00:10:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0305999999999998e-05 2024-09-16T00:10:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801442+0000", 2024-09-16T00:10:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6290000000000004e-06 2024-09-16T00:10:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801497+0000", 2024-09-16T00:10:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5111999999999997e-05 2024-09-16T00:10:26.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801639+0000", 2024-09-16T00:10:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014168000000000001 2024-09-16T00:10:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801788+0000", 2024-09-16T00:10:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014963300000000001 2024-09-16T00:10:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.805407+0000", 2024-09-16T00:10:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036186119999999998 2024-09-16T00:10:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811584+0000", 2024-09-16T00:10:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061772889999999999 2024-09-16T00:10:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811635+0000", 2024-09-16T00:10:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0453e-05 2024-09-16T00:10:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811665+0000", 2024-09-16T00:10:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0664000000000003e-05 2024-09-16T00:10:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14951 3.1a 3:59aee80a:::1000000394f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.903070623000001, 2024-09-16T00:10:26.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015993941000000001, 2024-09-16T00:10:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14951 2024-09-16T00:10:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967124+0000", 2024-09-16T00:10:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.86e-06 2024-09-16T00:10:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971258+0000", 2024-09-16T00:10:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041334780000000003 2024-09-16T00:10:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971274+0000", 2024-09-16T00:10:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6201e-05 2024-09-16T00:10:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971284+0000", 2024-09-16T00:10:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4779999999999997e-06 2024-09-16T00:10:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971384+0000", 2024-09-16T00:10:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9878999999999997e-05 2024-09-16T00:10:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971518+0000", 2024-09-16T00:10:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013459400000000001 2024-09-16T00:10:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971604+0000", 2024-09-16T00:10:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6215e-05 2024-09-16T00:10:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.976673+0000", 2024-09-16T00:10:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050685920000000002 2024-09-16T00:10:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983043+0000", 2024-09-16T00:10:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063698410000000002 2024-09-16T00:10:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983082+0000", 2024-09-16T00:10:26.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9261999999999998e-05 2024-09-16T00:10:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983117+0000", 2024-09-16T00:10:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4541e-05 2024-09-16T00:10:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:15152 3.3 3:c388c7e2:::100000032ec.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-16T00:10:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.142524513000001, 2024-09-16T00:10:26.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014603532000000001, 2024-09-16T00:10:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15152 2024-09-16T00:10:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727674+0000", 2024-09-16T00:10:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7699999999999998e-06 2024-09-16T00:10:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732274+0000", 2024-09-16T00:10:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045994299999999998 2024-09-16T00:10:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732286+0000", 2024-09-16T00:10:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2016e-05 2024-09-16T00:10:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732294+0000", 2024-09-16T00:10:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7049999999999999e-06 2024-09-16T00:10:26.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732359+0000", 2024-09-16T00:10:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5296000000000006e-05 2024-09-16T00:10:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732501+0000", 2024-09-16T00:10:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000141731 2024-09-16T00:10:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732629+0000", 2024-09-16T00:10:26.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012841299999999999 2024-09-16T00:10:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.737511+0000", 2024-09-16T00:10:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048822520000000001 2024-09-16T00:10:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742172+0000", 2024-09-16T00:10:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046610610000000002 2024-09-16T00:10:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742227+0000", 2024-09-16T00:10:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4497999999999999e-05 2024-09-16T00:10:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742272+0000", 2024-09-16T00:10:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5359999999999999e-05 2024-09-16T00:10:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.024 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.024 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:26.026 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:10:26.026 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:26.024+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:26.128 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:26.126+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:26.227 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:26.225+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:26.246 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:26.246 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:26.246 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:26.246 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:26.258 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:10:26.327 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:26.326+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:26.402 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T00:10:26.412 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:26.412 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:26.412 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:26.412 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:26.412 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:26.424 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:10:26.447 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:26.445+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:26.485 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-09-16T00:10:26.485 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:10:26.485 INFO:tasks.thrashosds.thrasher:Killing osd 7, live_osds are [0, 1, 2, 3, 4, 5, 7, 6] 2024-09-16T00:10:26.486 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-16T00:10:26.486 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:10:26.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.503+0000 7f983e83c640 1 Processor -- start 2024-09-16T00:10:26.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.503+0000 7f983e83c640 1 -- start start 2024-09-16T00:10:26.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.504+0000 7f983e83c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9838150f10 0x7f98380c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:26.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.504+0000 7f983e83c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f98380c3520 0x7f98380c8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:26.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.504+0000 7f983e83c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98380cbcf0 con 0x7f983815a2b0 2024-09-16T00:10:26.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.504+0000 7f983e83c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f98380c74f0 con 0x7f9838150f10 2024-09-16T00:10:26.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.504+0000 7f983e83c640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f98380cb730 con 0x7f98380c3520 2024-09-16T00:10:26.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.504+0000 7f9837fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9838150f10 0x7f98380c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:26.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.504+0000 7f983cdd9640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f983815a2b0 0x7f983815a6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56258/0 (socket says 172.21.15.139:56258) 2024-09-16T00:10:26.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.504+0000 7f983cdd9640 1 -- 172.21.15.139:0/703191925 learned_addr learned my addr 172.21.15.139:0/703191925 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:26.507 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.504+0000 7f98377fe640 1 --2- 172.21.15.139:0/703191925 >> v2:172.21.15.164:3300/0 conn(0x7f98380c3520 0x7f98380c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:26.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.505+0000 7f9837fff640 1 -- 172.21.15.139:0/703191925 >> v2:172.21.15.164:3300/0 conn(0x7f98380c3520 msgr2=0x7f98380c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:26.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.505+0000 7f9837fff640 1 --2- 172.21.15.139:0/703191925 >> v2:172.21.15.164:3300/0 conn(0x7f98380c3520 0x7f98380c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:26.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.505+0000 7f9837fff640 1 -- 172.21.15.139:0/703191925 >> v1:172.21.15.139:6789/0 conn(0x7f983815a2b0 legacy=0x7f983815a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:26.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.505+0000 7f9837fff640 1 -- 172.21.15.139:0/703191925 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98380cba10 con 0x7f9838150f10 2024-09-16T00:10:26.508 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.505+0000 7f9836ffd640 1 -- 172.21.15.139:0/703191925 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1990783513 0 0) 0x7f9830001490 con 0x7f983815a2b0 2024-09-16T00:10:26.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.505+0000 7f9837fff640 1 --2- 172.21.15.139:0/703191925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9838150f10 0x7f98380c7e00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f983806f3b0 tx=0x7f982802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14934995516d5fa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:26.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.505+0000 7f9836ffd640 1 -- 172.21.15.139:0/703191925 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828037070 con 0x7f9838150f10 2024-09-16T00:10:26.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.506+0000 7f9836ffd640 1 -- 172.21.15.139:0/703191925 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9828032440 con 0x7f9838150f10 2024-09-16T00:10:26.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.506+0000 7f9836ffd640 1 -- 172.21.15.139:0/703191925 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828032810 con 0x7f9838150f10 2024-09-16T00:10:26.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.506+0000 7f983e83c640 1 -- 172.21.15.139:0/703191925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9838150f10 msgr2=0x7f98380c7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:26.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.506+0000 7f983e83c640 1 --2- 172.21.15.139:0/703191925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9838150f10 0x7f98380c7e00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f983806f3b0 tx=0x7f982802a1a0 comp rx=0 tx=0).stop 2024-09-16T00:10:26.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.506+0000 7f983e83c640 1 -- 172.21.15.139:0/703191925 shutdown_connections 2024-09-16T00:10:26.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.506+0000 7f983e83c640 1 --2- 172.21.15.139:0/703191925 >> v2:172.21.15.164:3300/0 conn(0x7f98380c3520 0x7f98380c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:26.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.506+0000 7f983e83c640 1 --2- 172.21.15.139:0/703191925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9838150f10 0x7f98380c7e00 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:26.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.506+0000 7f983e83c640 1 -- 172.21.15.139:0/703191925 >> 172.21.15.139:0/703191925 conn(0x7f98380c91d0 msgr2=0x7f98380c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:26.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.506+0000 7f983e83c640 1 -- 172.21.15.139:0/703191925 shutdown_connections 2024-09-16T00:10:26.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.506+0000 7f983e83c640 1 -- 172.21.15.139:0/703191925 wait complete. 2024-09-16T00:10:26.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.507+0000 7f983e83c640 1 Processor -- start 2024-09-16T00:10:26.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.508+0000 7f983e83c640 1 -- start start 2024-09-16T00:10:26.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.508+0000 7f983e83c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98380c3520 0x7f98381f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:26.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.508+0000 7f983e83c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9838150f10 0x7f98381f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:26.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.508+0000 7f983e83c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f983815a2b0 0x7f9838152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:26.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.508+0000 7f983e83c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f98380cb420 con 0x7f983815a2b0 2024-09-16T00:10:26.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.508+0000 7f983e83c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f983806ea70 con 0x7f9838150f10 2024-09-16T00:10:26.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.508+0000 7f983e83c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f98380c70d0 con 0x7f98380c3520 2024-09-16T00:10:26.512 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.508+0000 7f983cdd9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f983815a2b0 0x7f9838152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.509+0000 7f98377fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9838150f10 0x7f98381f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.509+0000 7f983cdd9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f983815a2b0 0x7f9838152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52158/0 (socket says 172.21.15.139:52158) 2024-09-16T00:10:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.509+0000 7f983cdd9640 1 -- 172.21.15.139:0/3218041659 learned_addr learned my addr 172.21.15.139:0/3218041659 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.509+0000 7f9837fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98380c3520 0x7f98381f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.509+0000 7f983cdd9640 1 -- 172.21.15.139:0/3218041659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98380c3520 msgr2=0x7f98381f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.509+0000 7f983cdd9640 1 --2- 172.21.15.139:0/3218041659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98380c3520 0x7f98381f35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.509+0000 7f983cdd9640 1 -- 172.21.15.139:0/3218041659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9838150f10 msgr2=0x7f98381f3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.509+0000 7f983cdd9640 1 --2- 172.21.15.139:0/3218041659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9838150f10 0x7f98381f3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.509+0000 7f983cdd9640 1 -- 172.21.15.139:0/3218041659 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98381e3740 con 0x7f983815a2b0 2024-09-16T00:10:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.509+0000 7f983cdd9640 1 --2- 172.21.15.139:0/3218041659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f983815a2b0 0x7f9838152270 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f983000c730 tx=0x7f983000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.509+0000 7f9834ff9640 1 -- 172.21.15.139:0/3218041659 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830018070 con 0x7f983815a2b0 2024-09-16T00:10:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.509+0000 7f9834ff9640 1 -- 172.21.15.139:0/3218041659 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f983000ce40 con 0x7f983815a2b0 2024-09-16T00:10:26.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.509+0000 7f983e83c640 1 -- 172.21.15.139:0/3218041659 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98381f4140 con 0x7f983815a2b0 2024-09-16T00:10:26.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.509+0000 7f9834ff9640 1 -- 172.21.15.139:0/3218041659 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983000a860 con 0x7f983815a2b0 2024-09-16T00:10:26.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.509+0000 7f983e83c640 1 -- 172.21.15.139:0/3218041659 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9838158ef0 con 0x7f983815a2b0 2024-09-16T00:10:26.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.510+0000 7f983e83c640 1 -- 172.21.15.139:0/3218041659 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98381512f0 con 0x7f983815a2b0 2024-09-16T00:10:26.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.511+0000 7f9834ff9640 1 -- 172.21.15.139:0/3218041659 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f983001b070 con 0x7f983815a2b0 2024-09-16T00:10:26.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.512+0000 7f9834ff9640 1 --2- 172.21.15.139:0/3218041659 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f981806c2b0 0x7f981806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:26.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.512+0000 7f9837fff640 1 --2- 172.21.15.139:0/3218041659 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f981806c2b0 0x7f981806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:26.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.512+0000 7f9834ff9640 1 -- 172.21.15.139:0/3218041659 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5973+0+0 (secure 0 0 0) 0x7f983008aaa0 con 0x7f983815a2b0 2024-09-16T00:10:26.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.513+0000 7f9837fff640 1 --2- 172.21.15.139:0/3218041659 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f981806c2b0 0x7f981806e770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9828034000 tx=0x7f982802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:26.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.516+0000 7f9834ff9640 1 -- 172.21.15.139:0/3218041659 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f98380cbcf0 con 0x7f983815a2b0 2024-09-16T00:10:26.520 INFO:tasks.ceph.osd.7:Stopped 2024-09-16T00:10:26.520 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.7 2024-09-16T00:10:26.521 INFO:tasks.thrashosds.thrasher:remote for osd 7 is ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T00:10:26.521 DEBUG:teuthology.orchestra.run.smithi164:> type ceph-objectstore-tool 2024-09-16T00:10:26.529 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:26.527+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:26.537 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10658 3.1c 3:38c4c24e:::100000030bb.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.743527008999997, 2024-09-16T00:10:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021232445999999999, 2024-09-16T00:10:26.537 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10658 2024-09-16T00:10:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791398+0000", 2024-09-16T00:10:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3880000000000001e-06 2024-09-16T00:10:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800951+0000", 2024-09-16T00:10:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095532110000000007 2024-09-16T00:10:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800966+0000", 2024-09-16T00:10:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5058999999999999e-05 2024-09-16T00:10:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800974+0000", 2024-09-16T00:10:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1240000000000005e-06 2024-09-16T00:10:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801127+0000", 2024-09-16T00:10:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000153103 2024-09-16T00:10:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801207+0000", 2024-09-16T00:10:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9569999999999999e-05 2024-09-16T00:10:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801322+0000", 2024-09-16T00:10:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115358 2024-09-16T00:10:26.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.805760+0000", 2024-09-16T00:10:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044379409999999999 2024-09-16T00:10:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812528+0000", 2024-09-16T00:10:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067678249999999999 2024-09-16T00:10:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812586+0000", 2024-09-16T00:10:26.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8164000000000001e-05 2024-09-16T00:10:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812629+0000", 2024-09-16T00:10:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2703000000000001e-05 2024-09-16T00:10:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4122 2.18 2:1cc859d9:::200.00000012:head [write 798310~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:26.543 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.875987329000001, 2024-09-16T00:10:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019267580999999999, 2024-09-16T00:10:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:26.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658937+0000", 2024-09-16T00:10:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7700000000000004e-07 2024-09-16T00:10:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658942+0000", 2024-09-16T00:10:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.48e-06 2024-09-16T00:10:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658952+0000", 2024-09-16T00:10:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9469999999999998e-06 2024-09-16T00:10:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658959+0000", 2024-09-16T00:10:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4400000000000002e-06 2024-09-16T00:10:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670507+0000", 2024-09-16T00:10:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011548159000000001 2024-09-16T00:10:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670579+0000", 2024-09-16T00:10:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2082999999999994e-05 2024-09-16T00:10:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670686+0000", 2024-09-16T00:10:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107069 2024-09-16T00:10:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678001+0000", 2024-09-16T00:10:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073148780000000004 2024-09-16T00:10:26.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678104+0000", 2024-09-16T00:10:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010290599999999999 2024-09-16T00:10:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678163+0000", 2024-09-16T00:10:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8859e-05 2024-09-16T00:10:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678204+0000", 2024-09-16T00:10:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0982999999999997e-05 2024-09-16T00:10:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4123 2.18 2:1cc859d9:::200.00000012:head [write 799709~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.875418148000001, 2024-09-16T00:10:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018857569000000001, 2024-09-16T00:10:26.549 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659506+0000", 2024-09-16T00:10:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.78e-07 2024-09-16T00:10:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659512+0000", 2024-09-16T00:10:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0100000000000001e-06 2024-09-16T00:10:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659523+0000", 2024-09-16T00:10:26.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-16T00:10:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659531+0000", 2024-09-16T00:10:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2010000000000002e-06 2024-09-16T00:10:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671124+0000", 2024-09-16T00:10:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011593696000000001 2024-09-16T00:10:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671171+0000", 2024-09-16T00:10:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6974999999999997e-05 2024-09-16T00:10:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671243+0000", 2024-09-16T00:10:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2131000000000004e-05 2024-09-16T00:10:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678008+0000", 2024-09-16T00:10:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067642329999999997 2024-09-16T00:10:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678261+0000", 2024-09-16T00:10:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025378999999999999 2024-09-16T00:10:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678320+0000", 2024-09-16T00:10:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8452000000000002e-05 2024-09-16T00:10:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678363+0000", 2024-09-16T00:10:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2831999999999999e-05 2024-09-16T00:10:26.554 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4126 2.18 2:1cc859d9:::200.00000012:head [write 806443~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.874020362, 2024-09-16T00:10:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019331159000000001, 2024-09-16T00:10:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660904+0000", 2024-09-16T00:10:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6800000000000009e-07 2024-09-16T00:10:26.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660909+0000", 2024-09-16T00:10:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2730000000000002e-06 2024-09-16T00:10:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660919+0000", 2024-09-16T00:10:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4960000000000002e-06 2024-09-16T00:10:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660923+0000", 2024-09-16T00:10:26.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6260000000000003e-06 2024-09-16T00:10:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672698+0000", 2024-09-16T00:10:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011774573999999999 2024-09-16T00:10:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672784+0000", 2024-09-16T00:10:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6074000000000003e-05 2024-09-16T00:10:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672897+0000", 2024-09-16T00:10:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011267199999999999 2024-09-16T00:10:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678026+0000", 2024-09-16T00:10:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051297089999999997 2024-09-16T00:10:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680147+0000", 2024-09-16T00:10:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021209409999999999 2024-09-16T00:10:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680197+0000", 2024-09-16T00:10:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0068000000000002e-05 2024-09-16T00:10:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680234+0000", 2024-09-16T00:10:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6758000000000003e-05 2024-09-16T00:10:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.560 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4127 2.18 2:1cc859d9:::200.00000012:head [write 808688~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.873738512999999, 2024-09-16T00:10:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019226493000000001, 2024-09-16T00:10:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661186+0000", 2024-09-16T00:10:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2399999999999996e-07 2024-09-16T00:10:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661192+0000", 2024-09-16T00:10:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9259999999999997e-06 2024-09-16T00:10:26.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661203+0000", 2024-09-16T00:10:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1174e-05 2024-09-16T00:10:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661213+0000", 2024-09-16T00:10:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0579e-05 2024-09-16T00:10:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673408+0000", 2024-09-16T00:10:26.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012194442 2024-09-16T00:10:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673482+0000", 2024-09-16T00:10:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4258000000000006e-05 2024-09-16T00:10:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673592+0000", 2024-09-16T00:10:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109733 2024-09-16T00:10:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678033+0000", 2024-09-16T00:10:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044411629999999997 2024-09-16T00:10:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680287+0000", 2024-09-16T00:10:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022538469999999998 2024-09-16T00:10:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680367+0000", 2024-09-16T00:10:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9808999999999994e-05 2024-09-16T00:10:26.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680411+0000", 2024-09-16T00:10:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4638000000000002e-05 2024-09-16T00:10:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4128 2.18 2:1cc859d9:::200.00000012:head [write 810933~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.873319381999998, 2024-09-16T00:10:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018982046999999998, 2024-09-16T00:10:26.566 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661605+0000", 2024-09-16T00:10:26.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.031e-06 2024-09-16T00:10:26.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661610+0000", 2024-09-16T00:10:26.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5010000000000004e-06 2024-09-16T00:10:26.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661623+0000", 2024-09-16T00:10:26.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2286000000000001e-05 2024-09-16T00:10:26.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661630+0000", 2024-09-16T00:10:26.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000002e-06 2024-09-16T00:10:26.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674011+0000", 2024-09-16T00:10:26.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012380923 2024-09-16T00:10:26.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674092+0000", 2024-09-16T00:10:26.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1267999999999995e-05 2024-09-16T00:10:26.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674216+0000", 2024-09-16T00:10:26.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012331700000000001 2024-09-16T00:10:26.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678039+0000", 2024-09-16T00:10:26.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038232489999999999 2024-09-16T00:10:26.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680493+0000", 2024-09-16T00:10:26.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024543429999999999 2024-09-16T00:10:26.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680546+0000", 2024-09-16T00:10:26.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3239000000000003e-05 2024-09-16T00:10:26.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680586+0000", 2024-09-16T00:10:26.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9496999999999998e-05 2024-09-16T00:10:26.572 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.572 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.572 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.572 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4129 2.18 2:1cc859d9:::200.00000012:head [write 813178~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.572 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:26.572 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.872962230999999, 2024-09-16T00:10:26.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018785932000000002, 2024-09-16T00:10:26.573 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.573 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.573 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.573 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:26.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.573 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.573 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.573 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.573 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:26.573 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661962+0000", 2024-09-16T00:10:26.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-07 2024-09-16T00:10:26.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661966+0000", 2024-09-16T00:10:26.574 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-06 2024-09-16T00:10:26.574 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.574 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.574 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.574 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.662015+0000", 2024-09-16T00:10:26.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8364e-05 2024-09-16T00:10:26.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.662023+0000", 2024-09-16T00:10:26.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2090000000000005e-06 2024-09-16T00:10:26.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.575 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.575 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674744+0000", 2024-09-16T00:10:26.575 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012721423000000001 2024-09-16T00:10:26.575 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.575 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674825+0000", 2024-09-16T00:10:26.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1036000000000001e-05 2024-09-16T00:10:26.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674946+0000", 2024-09-16T00:10:26.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120727 2024-09-16T00:10:26.576 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.576 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.576 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.576 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678045+0000", 2024-09-16T00:10:26.576 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030993280000000002 2024-09-16T00:10:26.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680646+0000", 2024-09-16T00:10:26.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026005300000000002 2024-09-16T00:10:26.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680705+0000", 2024-09-16T00:10:26.577 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9141000000000002e-05 2024-09-16T00:10:26.577 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.577 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.577 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.577 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680747+0000", 2024-09-16T00:10:26.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2064000000000002e-05 2024-09-16T00:10:26.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.578 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.578 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.578 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.578 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.578 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4184 2.18 2:1cc859d9:::200.00000012:head [write 927308~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.578 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:26.578 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.828913635999999, 2024-09-16T00:10:26.578 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023503514999999999, 2024-09-16T00:10:26.578 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.578 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.578 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.579 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:26.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:26.579 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.579 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.579 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.579 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.579 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706011+0000", 2024-09-16T00:10:26.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8400000000000003e-07 2024-09-16T00:10:26.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706016+0000", 2024-09-16T00:10:26.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7479999999999999e-06 2024-09-16T00:10:26.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.580 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.580 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706028+0000", 2024-09-16T00:10:26.580 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1477e-05 2024-09-16T00:10:26.580 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.580 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706035+0000", 2024-09-16T00:10:26.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1509999999999998e-06 2024-09-16T00:10:26.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.719820+0000", 2024-09-16T00:10:26.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013785168 2024-09-16T00:10:26.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.581 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.581 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.581 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.719888+0000", 2024-09-16T00:10:26.581 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7631000000000004e-05 2024-09-16T00:10:26.581 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720003+0000", 2024-09-16T00:10:26.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114849 2024-09-16T00:10:26.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725150+0000", 2024-09-16T00:10:26.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051469970000000004 2024-09-16T00:10:26.582 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.582 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.582 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.582 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729386+0000", 2024-09-16T00:10:26.582 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042368529999999996 2024-09-16T00:10:26.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729461+0000", 2024-09-16T00:10:26.583 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4935e-05 2024-09-16T00:10:26.583 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.583 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.583 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.583 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729513+0000", 2024-09-16T00:10:26.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1822e-05 2024-09-16T00:10:26.584 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.584 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.584 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.584 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.584 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.584 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4185 2.18 2:1cc859d9:::200.00000012:head [write 929217~3264 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.584 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:26.584 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.827907455999998, 2024-09-16T00:10:26.584 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023998430000000001, 2024-09-16T00:10:26.584 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.584 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.584 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.584 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:26.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:26.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.585 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.585 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.585 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.585 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707017+0000", 2024-09-16T00:10:26.585 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9999999999999996e-07 2024-09-16T00:10:26.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707023+0000", 2024-09-16T00:10:26.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1789999999999996e-06 2024-09-16T00:10:26.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707035+0000", 2024-09-16T00:10:26.586 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2106000000000001e-05 2024-09-16T00:10:26.586 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.586 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.586 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.586 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707042+0000", 2024-09-16T00:10:26.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2960000000000001e-06 2024-09-16T00:10:26.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720503+0000", 2024-09-16T00:10:26.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013460952999999999 2024-09-16T00:10:26.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.587 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720583+0000", 2024-09-16T00:10:26.587 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0171000000000002e-05 2024-09-16T00:10:26.587 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.587 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.587 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720668+0000", 2024-09-16T00:10:26.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4128999999999994e-05 2024-09-16T00:10:26.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725193+0000", 2024-09-16T00:10:26.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045257700000000001 2024-09-16T00:10:26.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.588 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.588 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.588 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.730924+0000", 2024-09-16T00:10:26.588 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057307690000000001 2024-09-16T00:10:26.588 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.730972+0000", 2024-09-16T00:10:26.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7756000000000002e-05 2024-09-16T00:10:26.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.589 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.589 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731014+0000", 2024-09-16T00:10:26.589 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2401000000000003e-05 2024-09-16T00:10:26.589 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.589 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.589 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.589 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.589 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.590 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4186 2.18 2:1cc859d9:::200.00000012:head [write 932481~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.590 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:26.590 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.827319298999999, 2024-09-16T00:10:26.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023544934999999999, 2024-09-16T00:10:26.590 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.590 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.590 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.590 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.590 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.590 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:26.590 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.590 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.590 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:26.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707605+0000", 2024-09-16T00:10:26.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1500000000000003e-07 2024-09-16T00:10:26.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.591 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.591 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.591 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707610+0000", 2024-09-16T00:10:26.591 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7049999999999998e-06 2024-09-16T00:10:26.591 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707618+0000", 2024-09-16T00:10:26.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6149999999999997e-06 2024-09-16T00:10:26.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707623+0000", 2024-09-16T00:10:26.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0660000000000003e-06 2024-09-16T00:10:26.592 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.592 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.592 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.592 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721143+0000", 2024-09-16T00:10:26.592 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01351976 2024-09-16T00:10:26.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721221+0000", 2024-09-16T00:10:26.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7918999999999997e-05 2024-09-16T00:10:26.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721335+0000", 2024-09-16T00:10:26.593 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113781 2024-09-16T00:10:26.593 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.593 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.593 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.593 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725236+0000", 2024-09-16T00:10:26.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039010379999999999 2024-09-16T00:10:26.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731025+0000", 2024-09-16T00:10:26.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057896659999999997 2024-09-16T00:10:26.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.594 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.594 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731091+0000", 2024-09-16T00:10:26.594 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.567e-05 2024-09-16T00:10:26.594 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.594 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.595 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.595 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731149+0000", 2024-09-16T00:10:26.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7800000000000002e-05 2024-09-16T00:10:26.595 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.595 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.595 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.595 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.595 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.595 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4187 2.18 2:1cc859d9:::200.00000012:head [write 934390~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.595 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:26.595 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.826467170000001, 2024-09-16T00:10:26.595 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022820340000000001, 2024-09-16T00:10:26.595 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.595 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.596 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.596 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:26.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.596 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.596 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:26.596 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.596 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.596 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708457+0000", 2024-09-16T00:10:26.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6899999999999996e-07 2024-09-16T00:10:26.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708462+0000", 2024-09-16T00:10:26.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3090000000000003e-06 2024-09-16T00:10:26.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.597 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.597 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.597 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708472+0000", 2024-09-16T00:10:26.597 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7499999999999998e-06 2024-09-16T00:10:26.597 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708479+0000", 2024-09-16T00:10:26.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1189999999999999e-06 2024-09-16T00:10:26.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721895+0000", 2024-09-16T00:10:26.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013415664000000001 2024-09-16T00:10:26.598 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.598 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.598 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.598 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721941+0000", 2024-09-16T00:10:26.598 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6031e-05 2024-09-16T00:10:26.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722007+0000", 2024-09-16T00:10:26.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6503999999999999e-05 2024-09-16T00:10:26.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725277+0000", 2024-09-16T00:10:26.599 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003269554 2024-09-16T00:10:26.599 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.599 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.599 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.599 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731158+0000", 2024-09-16T00:10:26.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058813750000000003 2024-09-16T00:10:26.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731240+0000", 2024-09-16T00:10:26.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1815000000000002e-05 2024-09-16T00:10:26.600 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.600 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.600 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.600 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731276+0000", 2024-09-16T00:10:26.600 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6449999999999998e-05 2024-09-16T00:10:26.600 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.600 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.601 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.601 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.601 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4188 2.18 2:1cc859d9:::200.00000012:head [write 936299~2419 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.601 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:26.601 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.826046226999999, 2024-09-16T00:10:26.601 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022518565000000001, 2024-09-16T00:10:26.601 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.601 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.601 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.601 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.601 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.601 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.601 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:26.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:26.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.602 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.602 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708878+0000", 2024-09-16T00:10:26.602 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8800000000000001e-07 2024-09-16T00:10:26.602 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.602 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708884+0000", 2024-09-16T00:10:26.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9800000000000003e-06 2024-09-16T00:10:26.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708895+0000", 2024-09-16T00:10:26.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1416e-05 2024-09-16T00:10:26.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.603 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.603 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.603 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708903+0000", 2024-09-16T00:10:26.603 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4449999999999999e-06 2024-09-16T00:10:26.603 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722327+0000", 2024-09-16T00:10:26.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013424681000000001 2024-09-16T00:10:26.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.604 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722408+0000", 2024-09-16T00:10:26.604 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0092000000000005e-05 2024-09-16T00:10:26.604 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.604 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.604 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722538+0000", 2024-09-16T00:10:26.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013046600000000001 2024-09-16T00:10:26.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726665+0000", 2024-09-16T00:10:26.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041267420000000001 2024-09-16T00:10:26.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.605 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.605 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731284+0000", 2024-09-16T00:10:26.605 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046195469999999999 2024-09-16T00:10:26.605 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.605 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731358+0000", 2024-09-16T00:10:26.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4002000000000001e-05 2024-09-16T00:10:26.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.606 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.606 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731396+0000", 2024-09-16T00:10:26.606 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7305999999999998e-05 2024-09-16T00:10:26.606 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.606 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.606 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.606 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.606 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4189 2.18 2:1cc859d9:::200.00000012:head [write 938718~1910 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.825139110999999, 2024-09-16T00:10:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021703626, 2024-09-16T00:10:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.607 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709785+0000", 2024-09-16T00:10:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.043e-06 2024-09-16T00:10:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.608 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709791+0000", 2024-09-16T00:10:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9499999999999998e-06 2024-09-16T00:10:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709804+0000", 2024-09-16T00:10:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2958e-05 2024-09-16T00:10:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709812+0000", 2024-09-16T00:10:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9869999999999999e-06 2024-09-16T00:10:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.609 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723058+0000", 2024-09-16T00:10:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013245497 2024-09-16T00:10:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723131+0000", 2024-09-16T00:10:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3879000000000002e-05 2024-09-16T00:10:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723241+0000", 2024-09-16T00:10:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109435 2024-09-16T00:10:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.610 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726708+0000", 2024-09-16T00:10:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034671340000000002 2024-09-16T00:10:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731403+0000", 2024-09-16T00:10:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046953979999999999 2024-09-16T00:10:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731454+0000", 2024-09-16T00:10:26.611 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0747e-05 2024-09-16T00:10:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731488+0000", 2024-09-16T00:10:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3597999999999999e-05 2024-09-16T00:10:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4190 2.18 2:1cc859d9:::200.00000012:head [write 940628~2420 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.824666946000001, 2024-09-16T00:10:26.612 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021346812999999999, 2024-09-16T00:10:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:26.613 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710257+0000", 2024-09-16T00:10:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8100000000000001e-07 2024-09-16T00:10:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710263+0000", 2024-09-16T00:10:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8050000000000003e-06 2024-09-16T00:10:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.614 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710273+0000", 2024-09-16T00:10:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0005e-05 2024-09-16T00:10:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710280+0000", 2024-09-16T00:10:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3819999999999998e-06 2024-09-16T00:10:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723653+0000", 2024-09-16T00:10:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013372866000000001 2024-09-16T00:10:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.615 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723733+0000", 2024-09-16T00:10:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9396999999999999e-05 2024-09-16T00:10:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723852+0000", 2024-09-16T00:10:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011919899999999999 2024-09-16T00:10:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726752+0000", 2024-09-16T00:10:26.616 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028999659999999999 2024-09-16T00:10:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731495+0000", 2024-09-16T00:10:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047432209999999997 2024-09-16T00:10:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731568+0000", 2024-09-16T00:10:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2452000000000004e-05 2024-09-16T00:10:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.617 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731603+0000", 2024-09-16T00:10:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5639000000000002e-05 2024-09-16T00:10:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4192 2.18 2:1cc859d9:::200.00000012:head [write 943048~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.823934105999999, 2024-09-16T00:10:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021688433999999999, 2024-09-16T00:10:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.618 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.619 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710990+0000", 2024-09-16T00:10:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1999999999999998e-07 2024-09-16T00:10:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710996+0000", 2024-09-16T00:10:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:10:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711007+0000", 2024-09-16T00:10:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1471000000000001e-05 2024-09-16T00:10:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.620 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711014+0000", 2024-09-16T00:10:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7780000000000002e-06 2024-09-16T00:10:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724471+0000", 2024-09-16T00:10:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013457435 2024-09-16T00:10:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724547+0000", 2024-09-16T00:10:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5383999999999995e-05 2024-09-16T00:10:26.621 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724663+0000", 2024-09-16T00:10:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116348 2024-09-16T00:10:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731612+0000", 2024-09-16T00:10:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069491409999999998 2024-09-16T00:10:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.622 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732573+0000", 2024-09-16T00:10:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0009611 2024-09-16T00:10:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732638+0000", 2024-09-16T00:10:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919000000000006e-05 2024-09-16T00:10:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.623 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732678+0000", 2024-09-16T00:10:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9230999999999998e-05 2024-09-16T00:10:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4193 2.18 2:1cc859d9:::200.00000012:head [write 944447~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.823388397999999, 2024-09-16T00:10:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021286851999999998, 2024-09-16T00:10:26.624 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.625 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711536+0000", 2024-09-16T00:10:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0189999999999999e-06 2024-09-16T00:10:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711542+0000", 2024-09-16T00:10:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7169999999999996e-06 2024-09-16T00:10:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.626 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711554+0000", 2024-09-16T00:10:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2089000000000001e-05 2024-09-16T00:10:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711559+0000", 2024-09-16T00:10:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3739999999999997e-06 2024-09-16T00:10:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725311+0000", 2024-09-16T00:10:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013751559 2024-09-16T00:10:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.627 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725390+0000", 2024-09-16T00:10:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9779999999999998e-05 2024-09-16T00:10:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725509+0000", 2024-09-16T00:10:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118973 2024-09-16T00:10:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731617+0000", 2024-09-16T00:10:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061079000000000003 2024-09-16T00:10:26.628 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732762+0000", 2024-09-16T00:10:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011449769999999999 2024-09-16T00:10:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732796+0000", 2024-09-16T00:10:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3587999999999998e-05 2024-09-16T00:10:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.629 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732822+0000", 2024-09-16T00:10:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5876000000000001e-05 2024-09-16T00:10:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4194 2.18 2:1cc859d9:::200.00000012:head [write 946356~2929 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.822413366999999, 2024-09-16T00:10:26.630 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020453244999999998, 2024-09-16T00:10:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.631 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712511+0000", 2024-09-16T00:10:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0999999999999997e-07 2024-09-16T00:10:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712517+0000", 2024-09-16T00:10:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8370000000000002e-06 2024-09-16T00:10:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.632 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712528+0000", 2024-09-16T00:10:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1528e-05 2024-09-16T00:10:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712536+0000", 2024-09-16T00:10:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3939999999999998e-06 2024-09-16T00:10:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726004+0000", 2024-09-16T00:10:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013468177 2024-09-16T00:10:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.633 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726085+0000", 2024-09-16T00:10:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1626999999999996e-05 2024-09-16T00:10:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726205+0000", 2024-09-16T00:10:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119751 2024-09-16T00:10:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731622+0000", 2024-09-16T00:10:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054165419999999999 2024-09-16T00:10:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.634 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732867+0000", 2024-09-16T00:10:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001244855 2024-09-16T00:10:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732924+0000", 2024-09-16T00:10:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7395999999999997e-05 2024-09-16T00:10:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732963+0000", 2024-09-16T00:10:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9227999999999998e-05 2024-09-16T00:10:26.635 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4195 2.18 2:1cc859d9:::200.00000012:head [write 949285~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.821361707000001, 2024-09-16T00:10:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019498099000000001, 2024-09-16T00:10:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.636 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713563+0000", 2024-09-16T00:10:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T00:10:26.637 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713569+0000", 2024-09-16T00:10:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8710000000000002e-06 2024-09-16T00:10:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713582+0000", 2024-09-16T00:10:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3134e-05 2024-09-16T00:10:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.638 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713591+0000", 2024-09-16T00:10:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9889999999999998e-06 2024-09-16T00:10:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726786+0000", 2024-09-16T00:10:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013194991999999999 2024-09-16T00:10:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726856+0000", 2024-09-16T00:10:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0017000000000004e-05 2024-09-16T00:10:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.639 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726967+0000", 2024-09-16T00:10:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111815 2024-09-16T00:10:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732530+0000", 2024-09-16T00:10:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055624940000000003 2024-09-16T00:10:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733001+0000", 2024-09-16T00:10:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000470817 2024-09-16T00:10:26.640 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733030+0000", 2024-09-16T00:10:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9051e-05 2024-09-16T00:10:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733060+0000", 2024-09-16T00:10:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9819e-05 2024-09-16T00:10:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.641 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4215 2.18 2:1cc859d9:::200.00000012:head [write 991028~1910 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.801665996000001, 2024-09-16T00:10:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019416676000000001, 2024-09-16T00:10:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.642 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733258+0000", 2024-09-16T00:10:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:10:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.643 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733264+0000", 2024-09-16T00:10:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4090000000000002e-06 2024-09-16T00:10:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733273+0000", 2024-09-16T00:10:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4769999999999992e-06 2024-09-16T00:10:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733280+0000", 2024-09-16T00:10:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2989999999999999e-06 2024-09-16T00:10:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.644 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.742966+0000", 2024-09-16T00:10:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009685585 2024-09-16T00:10:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743046+0000", 2024-09-16T00:10:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0469999999999994e-05 2024-09-16T00:10:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743165+0000", 2024-09-16T00:10:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118847 2024-09-16T00:10:26.645 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.746968+0000", 2024-09-16T00:10:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038025099999999998 2024-09-16T00:10:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752559+0000", 2024-09-16T00:10:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055911290000000002 2024-09-16T00:10:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.646 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752634+0000", 2024-09-16T00:10:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5102e-05 2024-09-16T00:10:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752674+0000", 2024-09-16T00:10:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9878e-05 2024-09-16T00:10:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4216 2.18 2:1cc859d9:::200.00000012:head [write 992938~2755 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:26.647 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 27.801120088000001, 2024-09-16T00:10:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019035976, 2024-09-16T00:10:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:26.648 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733804+0000", 2024-09-16T00:10:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2299999999999999e-07 2024-09-16T00:10:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733810+0000", 2024-09-16T00:10:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7259999999999999e-06 2024-09-16T00:10:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:26.649 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733822+0000", 2024-09-16T00:10:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2204e-05 2024-09-16T00:10:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733829+0000", 2024-09-16T00:10:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3889999999999999e-06 2024-09-16T00:10:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743675+0000", 2024-09-16T00:10:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098457990000000006 2024-09-16T00:10:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.650 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743752+0000", 2024-09-16T00:10:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6279000000000006e-05 2024-09-16T00:10:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743867+0000", 2024-09-16T00:10:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115319 2024-09-16T00:10:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.746971+0000", 2024-09-16T00:10:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031042399999999999 2024-09-16T00:10:26.651 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752723+0000", 2024-09-16T00:10:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057518329999999996 2024-09-16T00:10:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752804+0000", 2024-09-16T00:10:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0822999999999995e-05 2024-09-16T00:10:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752839+0000", 2024-09-16T00:10:26.652 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5441000000000002e-05 2024-09-16T00:10:26.653 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.653 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.653 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.653 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:26.653 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:26.653 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:26.655 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:10:26.656 INFO:teuthology.orchestra.run.smithi164.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-16T00:10:26.656 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --op list-pgs 2024-09-16T00:10:26.656 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:26.653+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:26.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.716+0000 7f983e83c640 1 -- 172.21.15.139:0/3218041659 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f98380c9c50 con 0x7f983815a2b0 2024-09-16T00:10:26.755 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:26.753+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:26.812 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:26.812 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:26.812 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:26.812 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:26.821 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:10:26.862 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:26.861+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:26.871 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:10:26.957 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:26.956+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:26.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.970+0000 7f8adb39f640 1 Processor -- start 2024-09-16T00:10:26.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.970+0000 7f8adb39f640 1 -- start start 2024-09-16T00:10:26.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.971+0000 7f8adb39f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8ad4072560 0x7f8ad40c8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.971+0000 7f8adb39f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8ad4073f40 0x7f8ad40c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.971+0000 7f8adb39f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ad4072f70 con 0x7f8ad415b760 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.971+0000 7f8adb39f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad415bb40 con 0x7f8ad4073f40 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.971+0000 7f8adb39f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad4072cd0 con 0x7f8ad4072560 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.972+0000 7f8adab9e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8ad4073f40 0x7f8ad40c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.972+0000 7f8ada39d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f8ad415b760 0x7f8ad406ed50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56262/0 (socket says 172.21.15.139:56262) 2024-09-16T00:10:26.975 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.972+0000 7f8ada39d640 1 -- 172.21.15.139:0/875799354 learned_addr learned my addr 172.21.15.139:0/875799354 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:26.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.972+0000 7f8adab9e640 1 --2- 172.21.15.139:0/875799354 >> v2:172.21.15.139:3300/0 conn(0x7f8ad4073f40 0x7f8ad40c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52162/0 (socket says 172.21.15.139:52162) 2024-09-16T00:10:26.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.972+0000 7f8ad9b9c640 1 --2- 172.21.15.139:0/875799354 >> v2:172.21.15.164:3300/0 conn(0x7f8ad4072560 0x7f8ad40c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:26.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.972+0000 7f8adab9e640 1 -- 172.21.15.139:0/875799354 >> v2:172.21.15.164:3300/0 conn(0x7f8ad4072560 msgr2=0x7f8ad40c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:26.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.972+0000 7f8adab9e640 1 --2- 172.21.15.139:0/875799354 >> v2:172.21.15.164:3300/0 conn(0x7f8ad4072560 0x7f8ad40c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:26.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.972+0000 7f8adab9e640 1 -- 172.21.15.139:0/875799354 >> v1:172.21.15.139:6789/0 conn(0x7f8ad415b760 legacy=0x7f8ad406ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:26.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.972+0000 7f8adab9e640 1 -- 172.21.15.139:0/875799354 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad40c78b0 con 0x7f8ad4073f40 2024-09-16T00:10:26.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.972+0000 7f8ad939b640 1 -- 172.21.15.139:0/875799354 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2052045666 0 0) 0x7f8ad4072f70 con 0x7f8ad415b760 2024-09-16T00:10:26.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.973+0000 7f8adab9e640 1 --2- 172.21.15.139:0/875799354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ad4073f40 0x7f8ad40c4a00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40c0d70 tx=0x7f8ac402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e1ce51fdda5759 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:26.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.973+0000 7f8ad939b640 1 -- 172.21.15.139:0/875799354 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4007740 con 0x7f8ad4073f40 2024-09-16T00:10:26.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.973+0000 7f8ad939b640 1 -- 172.21.15.139:0/875799354 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac40078e0 con 0x7f8ad4073f40 2024-09-16T00:10:26.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.973+0000 7f8ad939b640 1 -- 172.21.15.139:0/875799354 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4007cb0 con 0x7f8ad4073f40 2024-09-16T00:10:26.979 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:26.979 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:26.979 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:26.979 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:26.979 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:26.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.973+0000 7f8adb39f640 1 -- 172.21.15.139:0/875799354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ad4073f40 msgr2=0x7f8ad40c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:26.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.973+0000 7f8adb39f640 1 --2- 172.21.15.139:0/875799354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ad4073f40 0x7f8ad40c4a00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40c0d70 tx=0x7f8ac402a680 comp rx=0 tx=0).stop 2024-09-16T00:10:26.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.973+0000 7f8adb39f640 1 -- 172.21.15.139:0/875799354 shutdown_connections 2024-09-16T00:10:26.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.973+0000 7f8adb39f640 1 --2- 172.21.15.139:0/875799354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ad4073f40 0x7f8ad40c4a00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:26.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.973+0000 7f8adb39f640 1 --2- 172.21.15.139:0/875799354 >> v2:172.21.15.164:3300/0 conn(0x7f8ad4072560 0x7f8ad40c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:26.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.973+0000 7f8adb39f640 1 -- 172.21.15.139:0/875799354 >> 172.21.15.139:0/875799354 conn(0x7f8ad406ff70 msgr2=0x7f8ad4070380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:26.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.973+0000 7f8adb39f640 1 -- 172.21.15.139:0/875799354 shutdown_connections 2024-09-16T00:10:26.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.973+0000 7f8adb39f640 1 -- 172.21.15.139:0/875799354 wait complete. 2024-09-16T00:10:26.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.975+0000 7f8adb39f640 1 Processor -- start 2024-09-16T00:10:26.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.975+0000 7f8adb39f640 1 -- start start 2024-09-16T00:10:26.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.975+0000 7f8adb39f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8ad4072560 0x7f8ad41500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:26.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.975+0000 7f8adb39f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ad4073f40 0x7f8ad41505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:26.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.975+0000 7f8adb39f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ad415b760 0x7f8ad41f3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:26.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.975+0000 7f8adb39f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad40c8150 con 0x7f8ad415b760 2024-09-16T00:10:26.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.975+0000 7f8adb39f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad40c7e70 con 0x7f8ad4073f40 2024-09-16T00:10:26.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.975+0000 7f8adb39f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad4070b20 con 0x7f8ad4072560 2024-09-16T00:10:26.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.975+0000 7f8adab9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ad415b760 0x7f8ad41f3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:26.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.975+0000 7f8adab9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ad415b760 0x7f8ad41f3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52164/0 (socket says 172.21.15.139:52164) 2024-09-16T00:10:26.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.975+0000 7f8adab9e640 1 -- 172.21.15.139:0/648038927 learned_addr learned my addr 172.21.15.139:0/648038927 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:26.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.975+0000 7f8ad9b9c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ad4073f40 0x7f8ad41505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:26.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.975+0000 7f8ada39d640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8ad4072560 0x7f8ad41500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:26.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.976+0000 7f8adab9e640 1 -- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8ad4072560 msgr2=0x7f8ad41500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:26.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.976+0000 7f8adab9e640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8ad4072560 0x7f8ad41500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:26.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.976+0000 7f8adab9e640 1 -- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ad4073f40 msgr2=0x7f8ad41505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:26.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.976+0000 7f8adab9e640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8ad4073f40 0x7f8ad41505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:26.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.976+0000 7f8adab9e640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad40cb8c0 con 0x7f8ad415b760 2024-09-16T00:10:26.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.976+0000 7f8adab9e640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ad415b760 0x7f8ad41f3810 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009e50 tx=0x7f8ac4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:26.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.976+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4040070 con 0x7f8ad415b760 2024-09-16T00:10:26.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.976+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ac4031bf0 con 0x7f8ad415b760 2024-09-16T00:10:26.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.976+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad40cbb80 con 0x7f8ad415b760 2024-09-16T00:10:26.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.976+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4009450 con 0x7f8ad415b760 2024-09-16T00:10:26.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.976+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad40cc140 con 0x7f8ad415b760 2024-09-16T00:10:26.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.977+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f8ad406eab0 con 0x7f8ad415b760 2024-09-16T00:10:26.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.978+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8ac4039040 con 0x7f8ad415b760 2024-09-16T00:10:26.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.978+0000 7f8ad2ffd640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8ac006c2b0 0x7f8ac006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:26.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.978+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5973+0+0 (secure 0 0 0) 0x7f8ac40accf0 con 0x7f8ad415b760 2024-09-16T00:10:26.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.978+0000 7f8ada39d640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8ac006c2b0 0x7f8ac006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:26.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.978+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v112) ==== 54+0+16 (secure 0 0 0) 0x7f8ac40af370 con 0x7f8ad415b760 2024-09-16T00:10:26.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.979+0000 7f8adb39f640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f8ad40cc8d0 0x7f8ad41f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:26.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.979+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ad41f4290 con 0x7f8ad40cc8d0 2024-09-16T00:10:26.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.979+0000 7f8ada39d640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8ac006c2b0 0x7f8ac006e770 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8acc000f60 tx=0x7f8acc007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:26.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.979+0000 7f8ad9b9c640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f8ad40cc8d0 0x7f8ad41f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:26.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.979+0000 7f8ad9b9c640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f8ad40cc8d0 0x7f8ad41f3d50 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:26.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.980+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ad41f4290 con 0x7f8ad40cc8d0 2024-09-16T00:10:26.997 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:10:26.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.994+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ad414c1f0 con 0x7f8ad40cc8d0 2024-09-16T00:10:26.998 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.996+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ad414c1f0 con 0x7f8ad40cc8d0 2024-09-16T00:10:26.999 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:27.000 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:27.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.996+0000 7f8adb39f640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f8ad414ae70 0x7f8ad414b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:27.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.996+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f8ad40c5bd0 con 0x7f8ad414ae70 2024-09-16T00:10:27.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.996+0000 7f8ada39d640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f8ad414ae70 0x7f8ad414b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:27.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.997+0000 7f8ada39d640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f8ad414ae70 0x7f8ad414b270 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:27.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:26.997+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ad40c5bd0 con 0x7f8ad414ae70 2024-09-16T00:10:27.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.013+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ad414ba00 con 0x7f8ad414ae70 2024-09-16T00:10:27.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.014+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ad414ba00 con 0x7f8ad414ae70 2024-09-16T00:10:27.016 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:27.017 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:27.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.015+0000 7f8adb39f640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8ad41f0540 0x7f8ad41f0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:27.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.015+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f8ad41f0e90 con 0x7f8ad41f0540 2024-09-16T00:10:27.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.015+0000 7f8ad9b9c640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8ad41f0540 0x7f8ad41f0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:27.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.015+0000 7f8ad9b9c640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f8ad41f0540 0x7f8ad41f0950 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:27.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.016+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ad41f0e90 con 0x7f8ad41f0540 2024-09-16T00:10:27.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.029+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ad4062880 con 0x7f8ad41f0540 2024-09-16T00:10:27.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.030+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ad4062880 con 0x7f8ad41f0540 2024-09-16T00:10:27.032 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:27.034 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:27.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.031+0000 7f8adb39f640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8ad4063360 0x7f8ad406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:27.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.031+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f8ad4062880 con 0x7f8ad4063360 2024-09-16T00:10:27.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.031+0000 7f8adab9e640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8ad4063360 0x7f8ad406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:27.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.031+0000 7f8adab9e640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f8ad4063360 0x7f8ad406d930 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:27.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.032+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ad4062880 con 0x7f8ad4063360 2024-09-16T00:10:27.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.045+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ad4062880 con 0x7f8ad4063360 2024-09-16T00:10:27.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.046+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ad4062880 con 0x7f8ad4063360 2024-09-16T00:10:27.048 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:27.049 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:27.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.047+0000 7f8adb39f640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f8ad402c640 0x7f8ad402caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:27.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.047+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f8ad4062880 con 0x7f8ad402c640 2024-09-16T00:10:27.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.047+0000 7f8ada39d640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f8ad402c640 0x7f8ad402caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:27.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.047+0000 7f8ada39d640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f8ad402c640 0x7f8ad402caa0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:27.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.048+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ad4062880 con 0x7f8ad402c640 2024-09-16T00:10:27.059 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:27.057+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:27.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.069+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ad4062880 con 0x7f8ad402c640 2024-09-16T00:10:27.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.073+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ad4062880 con 0x7f8ad402c640 2024-09-16T00:10:27.075 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:27.075 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:27.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.073+0000 7f8adb39f640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f8ad402d4f0 0x7f8ad4206b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:27.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.073+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f8ad4062880 con 0x7f8ad402d4f0 2024-09-16T00:10:27.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.073+0000 7f8ad9b9c640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f8ad402d4f0 0x7f8ad4206b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:27.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.074+0000 7f8ad9b9c640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f8ad402d4f0 0x7f8ad4206b40 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:27.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.075+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ad4062880 con 0x7f8ad402d4f0 2024-09-16T00:10:27.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.095+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ad402d950 con 0x7f8ad402d4f0 2024-09-16T00:10:27.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.097+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ad402d950 con 0x7f8ad402d4f0 2024-09-16T00:10:27.099 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:27.100 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:27.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.097+0000 7f8adb39f640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f8ad4207590 0x7f8ad4209980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:27.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.097+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f8ad420c750 con 0x7f8ad4207590 2024-09-16T00:10:27.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.098+0000 7f8adab9e640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f8ad4207590 0x7f8ad4209980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:27.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.098+0000 7f8adab9e640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f8ad4207590 0x7f8ad4209980 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:27.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.099+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ad420c750 con 0x7f8ad4207590 2024-09-16T00:10:27.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.112+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8ad4209ec0 con 0x7f8ad4207590 2024-09-16T00:10:27.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.116+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8ad4209ec0 con 0x7f8ad4207590 2024-09-16T00:10:27.118 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:27.118 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:27.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.116+0000 7f8adb39f640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8ad420a5e0 0x7f8ad420c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:27.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.116+0000 7f8adb39f640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f8ad420f920 con 0x7f8ad420a5e0 2024-09-16T00:10:27.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.116+0000 7f8ada39d640 1 -- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8ad420a5e0 msgr2=0x7f8ad420c9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6808/1158838808 2024-09-16T00:10:27.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.116+0000 7f8ada39d640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8ad420a5e0 0x7f8ad420c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:10:27.144 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:27.144 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:27.153 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:10:27.158 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:27.157+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:27.260 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:27.258+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:27.262 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:27.262 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:27.272 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:10:27.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.317+0000 7f8ada39d640 1 -- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8ad420a5e0 msgr2=0x7f8ad420c9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6808/1158838808 2024-09-16T00:10:27.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.317+0000 7f8ada39d640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8ad420a5e0 0x7f8ad420c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:10:27.361 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:27.359+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:27.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.390+0000 7f9834ff9640 1 -- 172.21.15.139:0/3218041659 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v113) ==== 101+0+0 (secure 0 0 0) 0x7f9830058a20 con 0x7f983815a2b0 2024-09-16T00:10:27.392 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is unset 2024-09-16T00:10:27.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.393+0000 7f983e83c640 1 -- 172.21.15.139:0/3218041659 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f981806c2b0 msgr2=0x7f981806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:27.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.393+0000 7f983e83c640 1 --2- 172.21.15.139:0/3218041659 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f981806c2b0 0x7f981806e770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9828034000 tx=0x7f982802e000 comp rx=0 tx=0).stop 2024-09-16T00:10:27.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.393+0000 7f983e83c640 1 -- 172.21.15.139:0/3218041659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f983815a2b0 msgr2=0x7f9838152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:27.394 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.393+0000 7f983e83c640 1 --2- 172.21.15.139:0/3218041659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f983815a2b0 0x7f9838152270 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f983000c730 tx=0x7f983000cbf0 comp rx=0 tx=0).stop 2024-09-16T00:10:27.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.393+0000 7f983e83c640 1 -- 172.21.15.139:0/3218041659 shutdown_connections 2024-09-16T00:10:27.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.393+0000 7f983e83c640 1 --2- 172.21.15.139:0/3218041659 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f981806c2b0 0x7f981806e770 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:27.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.393+0000 7f983e83c640 1 --2- 172.21.15.139:0/3218041659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f983815a2b0 0x7f9838152270 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:27.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.393+0000 7f983e83c640 1 --2- 172.21.15.139:0/3218041659 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9838150f10 0x7f98381f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:27.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.393+0000 7f983e83c640 1 --2- 172.21.15.139:0/3218041659 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f98380c3520 0x7f98381f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:27.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.393+0000 7f983e83c640 1 -- 172.21.15.139:0/3218041659 >> 172.21.15.139:0/3218041659 conn(0x7f98380c91d0 msgr2=0x7f983806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:27.395 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.393+0000 7f983e83c640 1 -- 172.21.15.139:0/3218041659 shutdown_connections 2024-09-16T00:10:27.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.393+0000 7f983e83c640 1 -- 172.21.15.139:0/3218041659 wait complete. 2024-09-16T00:10:27.424 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:27.425 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:27.434 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:10:27.475 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:27.473+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:27.562 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:27.560+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:27.579 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:27.579 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:27.588 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:10:27.674 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:27.673+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:27.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.718+0000 7f8ada39d640 1 -- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8ad420a5e0 msgr2=0x7f8ad420c9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6808/1158838808 2024-09-16T00:10:27.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:27.718+0000 7f8ada39d640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8ad420a5e0 0x7f8ad420c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T00:10:27.742 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:27.742 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:27.751 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:10:27.765 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:27.763+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:27.866 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:27.864+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:27.902 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:27.902 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:27.911 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:10:27.966 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:27.965+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:28.060 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:28.060 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:28.067 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:28.066+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:28.069 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:10:28.169 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:28.167+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:28.184 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:28.185 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:28.194 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:10:28.269 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:28.268+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:28.303 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:28.303 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:28.313 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:10:28.370 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:28.369+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:28.472 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:28.470+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:28.495 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:28.495 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:28.511 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:10:28.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:28.519+0000 7f8ada39d640 1 -- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8ad420a5e0 msgr2=0x7f8ad420c9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6808/1158838808 2024-09-16T00:10:28.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:28.519+0000 7f8ada39d640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8ad420a5e0 0x7f8ad420c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T00:10:28.572 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:28.570+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:28.621 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:28.621 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:28.633 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:10:28.697 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:28.673+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:28.763 INFO:teuthology.orchestra.run.smithi164.stdout:3.f 2024-09-16T00:10:28.763 INFO:teuthology.orchestra.run.smithi164.stdout:3.e 2024-09-16T00:10:28.763 INFO:teuthology.orchestra.run.smithi164.stdout:3.7 2024-09-16T00:10:28.763 INFO:teuthology.orchestra.run.smithi164.stdout:3.3f 2024-09-16T00:10:28.763 INFO:teuthology.orchestra.run.smithi164.stdout:3.39 2024-09-16T00:10:28.763 INFO:teuthology.orchestra.run.smithi164.stdout:3.37 2024-09-16T00:10:28.763 INFO:teuthology.orchestra.run.smithi164.stdout:3.2f 2024-09-16T00:10:28.764 INFO:teuthology.orchestra.run.smithi164.stdout:3.2d 2024-09-16T00:10:28.764 INFO:teuthology.orchestra.run.smithi164.stdout:3.2a 2024-09-16T00:10:28.764 INFO:teuthology.orchestra.run.smithi164.stdout:3.26 2024-09-16T00:10:28.764 INFO:teuthology.orchestra.run.smithi164.stdout:3.13 2024-09-16T00:10:28.764 INFO:teuthology.orchestra.run.smithi164.stdout:3.12 2024-09-16T00:10:28.764 INFO:teuthology.orchestra.run.smithi164.stdout:3.11 2024-09-16T00:10:28.764 INFO:teuthology.orchestra.run.smithi164.stdout:3.8 2024-09-16T00:10:28.764 INFO:teuthology.orchestra.run.smithi164.stdout:3.16 2024-09-16T00:10:28.764 INFO:teuthology.orchestra.run.smithi164.stdout:2.b 2024-09-16T00:10:28.764 INFO:teuthology.orchestra.run.smithi164.stdout:3.d 2024-09-16T00:10:28.764 INFO:teuthology.orchestra.run.smithi164.stdout:2.a 2024-09-16T00:10:28.764 INFO:teuthology.orchestra.run.smithi164.stdout:2.8 2024-09-16T00:10:28.764 INFO:teuthology.orchestra.run.smithi164.stdout:2.9 2024-09-16T00:10:28.765 INFO:teuthology.orchestra.run.smithi164.stdout:2.7 2024-09-16T00:10:28.765 INFO:teuthology.orchestra.run.smithi164.stdout:2.5 2024-09-16T00:10:28.765 INFO:teuthology.orchestra.run.smithi164.stdout:3.15 2024-09-16T00:10:28.765 INFO:teuthology.orchestra.run.smithi164.stdout:2.1f 2024-09-16T00:10:28.765 INFO:teuthology.orchestra.run.smithi164.stdout:2.1d 2024-09-16T00:10:28.765 INFO:teuthology.orchestra.run.smithi164.stdout:3.14 2024-09-16T00:10:28.765 INFO:teuthology.orchestra.run.smithi164.stdout:2.1c 2024-09-16T00:10:28.765 INFO:teuthology.orchestra.run.smithi164.stdout:2.1b 2024-09-16T00:10:28.765 INFO:teuthology.orchestra.run.smithi164.stdout:2.0 2024-09-16T00:10:28.765 INFO:teuthology.orchestra.run.smithi164.stdout:3.2b 2024-09-16T00:10:28.765 INFO:teuthology.orchestra.run.smithi164.stdout:1.0 2024-09-16T00:10:28.773 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:28.772+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:28.780 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:28.780 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:28.789 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:10:28.874 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:28.873+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:28.945 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:28.945 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:28.954 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:10:28.975 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:28.973+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:29.076 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:29.074+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:29.100 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:29.100 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:29.110 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:10:29.176 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:29.175+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:29.220 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:29.220 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:29.229 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:10:29.277 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:29.276+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:29.286 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --op export --pgid 3.14 --file /var/log/ceph/exp.3.14.7 2024-09-16T00:10:29.345 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:29.345 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:29.355 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:10:29.398 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:29.397+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:29.410 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T00:10:29.479 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:29.477+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:29.484 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:29.484 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:29.496 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:10:29.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.512+0000 7fb8faad9640 1 Processor -- start 2024-09-16T00:10:29.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.512+0000 7fb8faad9640 1 -- start start 2024-09-16T00:10:29.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.513+0000 7fb8faad9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb8f4158020 0x7fb8f4162e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:29.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.513+0000 7fb8faad9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8f40c8170 con 0x7fb8f40c71e0 2024-09-16T00:10:29.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.513+0000 7fb8faad9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8f40c8450 con 0x7fb8f4151800 2024-09-16T00:10:29.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.513+0000 7fb8faad9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb8f40c7f20 con 0x7fb8f4158020 2024-09-16T00:10:29.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.513+0000 7fb8f3fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb8f40c71e0 0x7fb8f4157910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:52080/0 (socket says 172.21.15.139:52080) 2024-09-16T00:10:29.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.513+0000 7fb8f3fff640 1 -- 172.21.15.139:0/2131298587 learned_addr learned my addr 172.21.15.139:0/2131298587 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:29.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.513+0000 7fb8f9076640 1 --2- 172.21.15.139:0/2131298587 >> v2:172.21.15.164:3300/0 conn(0x7fb8f4158020 0x7fb8f4162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:29.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.513+0000 7fb8f37fe640 1 -- 172.21.15.139:0/2131298587 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4161359265 0 0) 0x7fb8f40c8170 con 0x7fb8f40c71e0 2024-09-16T00:10:29.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.513+0000 7fb8f37fe640 1 -- 172.21.15.139:0/2131298587 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8e4003520 con 0x7fb8f40c71e0 2024-09-16T00:10:29.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.513+0000 7fb8f37fe640 1 -- 172.21.15.139:0/2131298587 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3737818691 0 0) 0x7fb8f40c8450 con 0x7fb8f4151800 2024-09-16T00:10:29.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.513+0000 7fb8f37fe640 1 -- 172.21.15.139:0/2131298587 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8f40c8170 con 0x7fb8f4151800 2024-09-16T00:10:29.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.513+0000 7fb8f9076640 1 -- 172.21.15.139:0/2131298587 >> v1:172.21.15.164:6789/0 conn(0x7fb8f4151800 legacy=0x7fb8f406eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:29.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.513+0000 7fb8f9076640 1 -- 172.21.15.139:0/2131298587 >> v1:172.21.15.139:6789/0 conn(0x7fb8f40c71e0 legacy=0x7fb8f4157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:29.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.513+0000 7fb8f9076640 1 -- 172.21.15.139:0/2131298587 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f40c8730 con 0x7fb8f4158020 2024-09-16T00:10:29.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.514+0000 7fb8f9076640 1 --2- 172.21.15.139:0/2131298587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8f4158020 0x7fb8f4162e50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec009d90 tx=0x7fb8ec02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=955142fa3ec5f09e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:29.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.514+0000 7fb8f37fe640 1 -- 172.21.15.139:0/2131298587 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec007740 con 0x7fb8f4158020 2024-09-16T00:10:29.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.514+0000 7fb8f37fe640 1 -- 172.21.15.139:0/2131298587 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8ec0078e0 con 0x7fb8f4158020 2024-09-16T00:10:29.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.514+0000 7fb8f37fe640 1 -- 172.21.15.139:0/2131298587 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec007cb0 con 0x7fb8f4158020 2024-09-16T00:10:29.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.515+0000 7fb8faad9640 1 -- 172.21.15.139:0/2131298587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8f4158020 msgr2=0x7fb8f4162e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:29.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.515+0000 7fb8faad9640 1 --2- 172.21.15.139:0/2131298587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8f4158020 0x7fb8f4162e50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec009d90 tx=0x7fb8ec02a660 comp rx=0 tx=0).stop 2024-09-16T00:10:29.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.515+0000 7fb8faad9640 1 -- 172.21.15.139:0/2131298587 shutdown_connections 2024-09-16T00:10:29.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.515+0000 7fb8faad9640 1 --2- 172.21.15.139:0/2131298587 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8f4158020 0x7fb8f4162e50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:29.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.515+0000 7fb8faad9640 1 -- 172.21.15.139:0/2131298587 >> 172.21.15.139:0/2131298587 conn(0x7fb8f4070460 msgr2=0x7fb8f4070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:29.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.515+0000 7fb8faad9640 1 -- 172.21.15.139:0/2131298587 shutdown_connections 2024-09-16T00:10:29.518 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.515+0000 7fb8faad9640 1 -- 172.21.15.139:0/2131298587 wait complete. 2024-09-16T00:10:29.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.516+0000 7fb8faad9640 1 Processor -- start 2024-09-16T00:10:29.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.516+0000 7fb8faad9640 1 -- start start 2024-09-16T00:10:29.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.517+0000 7fb8faad9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8f40c71e0 0x7fb8f41e57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:29.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.517+0000 7fb8faad9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb8f4151800 0x7fb8f41e3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:29.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.517+0000 7fb8faad9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8f4158020 0x7fb8f41f1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:29.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.517+0000 7fb8faad9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f40c44c0 con 0x7fb8f40c71e0 2024-09-16T00:10:29.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.517+0000 7fb8faad9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f4151be0 con 0x7fb8f4158020 2024-09-16T00:10:29.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.517+0000 7fb8faad9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb8f40c4640 con 0x7fb8f4151800 2024-09-16T00:10:29.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.517+0000 7fb8f8875640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8f40c71e0 0x7fb8f41e57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:29.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.517+0000 7fb8f8875640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8f40c71e0 0x7fb8f41e57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49474/0 (socket says 172.21.15.139:49474) 2024-09-16T00:10:29.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.517+0000 7fb8f8875640 1 -- 172.21.15.139:0/2555235450 learned_addr learned my addr 172.21.15.139:0/2555235450 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:29.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.517+0000 7fb8f3fff640 1 --2- 172.21.15.139:0/2555235450 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb8f4151800 0x7fb8f41e3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:29.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.517+0000 7fb8f9076640 1 --2- 172.21.15.139:0/2555235450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8f4158020 0x7fb8f41f1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:29.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.517+0000 7fb8f8875640 1 -- 172.21.15.139:0/2555235450 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb8f4151800 msgr2=0x7fb8f41e3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:29.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.517+0000 7fb8f8875640 1 --2- 172.21.15.139:0/2555235450 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb8f4151800 0x7fb8f41e3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:29.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.517+0000 7fb8f8875640 1 -- 172.21.15.139:0/2555235450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8f4158020 msgr2=0x7fb8f41f1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:29.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.517+0000 7fb8f8875640 1 --2- 172.21.15.139:0/2555235450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8f4158020 0x7fb8f41f1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:29.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.517+0000 7fb8f8875640 1 -- 172.21.15.139:0/2555235450 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f41e5d10 con 0x7fb8f40c71e0 2024-09-16T00:10:29.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.518+0000 7fb8f8875640 1 --2- 172.21.15.139:0/2555235450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8f40c71e0 0x7fb8f41e57d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800c460 tx=0x7fb8e800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:29.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.518+0000 7fb8f17fa640 1 -- 172.21.15.139:0/2555235450 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8017070 con 0x7fb8f40c71e0 2024-09-16T00:10:29.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.518+0000 7fb8f17fa640 1 -- 172.21.15.139:0/2555235450 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e80052f0 con 0x7fb8f40c71e0 2024-09-16T00:10:29.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.518+0000 7fb8f17fa640 1 -- 172.21.15.139:0/2555235450 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8003830 con 0x7fb8f40c71e0 2024-09-16T00:10:29.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.518+0000 7fb8faad9640 1 -- 172.21.15.139:0/2555235450 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f41f2670 con 0x7fb8f40c71e0 2024-09-16T00:10:29.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.518+0000 7fb8faad9640 1 -- 172.21.15.139:0/2555235450 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f41e4fd0 con 0x7fb8f40c71e0 2024-09-16T00:10:29.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.519+0000 7fb8faad9640 1 -- 172.21.15.139:0/2555235450 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f406f750 con 0x7fb8f40c71e0 2024-09-16T00:10:29.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.519+0000 7fb8f17fa640 1 -- 172.21.15.139:0/2555235450 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb8e801a070 con 0x7fb8f40c71e0 2024-09-16T00:10:29.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.520+0000 7fb8f17fa640 1 --2- 172.21.15.139:0/2555235450 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb8d806c2b0 0x7fb8d806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:29.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.520+0000 7fb8f17fa640 1 -- 172.21.15.139:0/2555235450 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5973+0+0 (secure 0 0 0) 0x7fb8e808a890 con 0x7fb8f40c71e0 2024-09-16T00:10:29.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.520+0000 7fb8f3fff640 1 --2- 172.21.15.139:0/2555235450 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb8d806c2b0 0x7fb8d806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:29.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.520+0000 7fb8f3fff640 1 --2- 172.21.15.139:0/2555235450 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb8d806c2b0 0x7fb8d806e770 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb8f41f2b00 tx=0x7fb8e0007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:29.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.524+0000 7fb8f17fa640 1 -- 172.21.15.139:0/2555235450 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb8e80588a0 con 0x7fb8f40c71e0 2024-09-16T00:10:29.580 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:29.579+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:29.674 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:29.674 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:29.681 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:29.680+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:29.685 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:10:29.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:29.721+0000 7fb8faad9640 1 -- 172.21.15.139:0/2555235450 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7fb8f40c5ac0 con 0x7fb8f40c71e0 2024-09-16T00:10:29.783 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:29.781+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:29.838 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:29.838 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:29.847 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:10:29.884 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:29.882+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:29.985 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:29.984+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:30.008 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:30.008 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:30.019 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:10:30.086 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:30.085+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:30.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:30.120+0000 7f8ada39d640 1 -- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8ad420a5e0 msgr2=0x7f8ad420c9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.164:6808/1158838808 2024-09-16T00:10:30.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:30.120+0000 7f8ada39d640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8ad420a5e0 0x7f8ad420c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-16T00:10:30.150 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:30.151 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:30.163 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:10:30.204 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:30.203+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:30.287 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:30.287+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:30.325 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:30.326 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:30.336 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:10:30.388 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:30.388+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:30.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:30.407+0000 7fb8f17fa640 1 -- 172.21.15.139:0/2555235450 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v115) ==== 87+0+0 (secure 0 0 0) 0x7fb8e805cba0 con 0x7fb8f40c71e0 2024-09-16T00:10:30.408 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is set 2024-09-16T00:10:30.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:30.410+0000 7fb8faad9640 1 -- 172.21.15.139:0/2555235450 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb8d806c2b0 msgr2=0x7fb8d806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:30.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:30.410+0000 7fb8faad9640 1 --2- 172.21.15.139:0/2555235450 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb8d806c2b0 0x7fb8d806e770 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb8f41f2b00 tx=0x7fb8e0007d50 comp rx=0 tx=0).stop 2024-09-16T00:10:30.411 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:30.410+0000 7fb8faad9640 1 -- 172.21.15.139:0/2555235450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8f40c71e0 msgr2=0x7fb8f41e57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:30.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:30.410+0000 7fb8faad9640 1 --2- 172.21.15.139:0/2555235450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8f40c71e0 0x7fb8f41e57d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800c460 tx=0x7fb8e800c920 comp rx=0 tx=0).stop 2024-09-16T00:10:30.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:30.410+0000 7fb8faad9640 1 -- 172.21.15.139:0/2555235450 shutdown_connections 2024-09-16T00:10:30.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:30.410+0000 7fb8faad9640 1 --2- 172.21.15.139:0/2555235450 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb8d806c2b0 0x7fb8d806e770 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:30.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:30.410+0000 7fb8faad9640 1 --2- 172.21.15.139:0/2555235450 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb8f4158020 0x7fb8f41f1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:30.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:30.410+0000 7fb8faad9640 1 --2- 172.21.15.139:0/2555235450 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb8f4151800 0x7fb8f41e3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:30.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:30.410+0000 7fb8faad9640 1 --2- 172.21.15.139:0/2555235450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8f40c71e0 0x7fb8f41e57d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:30.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:30.410+0000 7fb8faad9640 1 -- 172.21.15.139:0/2555235450 >> 172.21.15.139:0/2555235450 conn(0x7fb8f4070460 msgr2=0x7fb8f41dff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:30.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:30.410+0000 7fb8faad9640 1 -- 172.21.15.139:0/2555235450 shutdown_connections 2024-09-16T00:10:30.412 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:30.410+0000 7fb8faad9640 1 -- 172.21.15.139:0/2555235450 wait complete. 2024-09-16T00:10:30.450 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:30.450 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:30.459 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:10:30.501 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:30.500+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:30.590 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:30.590+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:30.610 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:30.610 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:30.619 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:10:30.704 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:30.703+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:30.770 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:30.770 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:30.780 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:10:30.821 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:30.820+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:30.893 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:30.891+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:30.931 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:30.931 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:30.941 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:10:30.950 INFO:teuthology.orchestra.run.smithi164.stderr:Exporting 3.14 info 3.14( v 112'310 (0'0,112'310] local-lis/les=20/21 n=63 ec=20/20 lis/c=20/20 les/c/f=21/21/0 sis=20) 2024-09-16T00:10:30.957 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:283cfb5b:::10000002254.00000000:head# 2024-09-16T00:10:30.957 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2849e444:::10000000583.00000000:head# 2024-09-16T00:10:30.958 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:28628fd5:::10000000a63.00000000:head# 2024-09-16T00:10:30.958 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:287d9ebf:::10000000a94.00000000:head# 2024-09-16T00:10:30.958 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:288e48f2:::10000002c14.00000000:head# 2024-09-16T00:10:30.959 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2890e941:::100000004d5.00000000:head# 2024-09-16T00:10:30.959 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:289c9a94:::10000002566.00000000:head# 2024-09-16T00:10:30.959 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:28a1b3d4:::100000026e2.00000000:head# 2024-09-16T00:10:30.966 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:28a2fcee:::10000000768.00000000:head# 2024-09-16T00:10:30.966 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:28a48acc:::10000000992.00000000:head# 2024-09-16T00:10:30.966 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:28b60791:::100000024e9.00000000:head# 2024-09-16T00:10:30.966 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:28cc090d:::1000000276c.00000000:head# 2024-09-16T00:10:30.966 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:28cdf2b1:::10000002bb1.00000000:head# 2024-09-16T00:10:30.966 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:290bfd0e:::10000001f93.00000000:head# 2024-09-16T00:10:30.966 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:292337b6:::100000023f6.00000000:head# 2024-09-16T00:10:30.966 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:29255704:::10000002c84.00000000:head# 2024-09-16T00:10:30.967 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2926fa2e:::10000000526.00000000:head# 2024-09-16T00:10:30.967 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2940346c:::100000028f1.00000000:head# 2024-09-16T00:10:30.967 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:294d4142:::10000002bfb.00000000:head# 2024-09-16T00:10:30.967 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:295cf825:::1000000215c.00000000:head# 2024-09-16T00:10:30.967 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:29783fe8:::1000000239d.00000000:head# 2024-09-16T00:10:30.967 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:29812fbf:::1000000307b.00000000:head# 2024-09-16T00:10:30.967 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2987f2bd:::100000023d6.00000000:head# 2024-09-16T00:10:30.967 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:298ca926:::100000024ed.00000000:head# 2024-09-16T00:10:30.967 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:298e2c1d:::1000000208b.00000000:head# 2024-09-16T00:10:30.967 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:29aa64e6:::10000000a8e.00000000:head# 2024-09-16T00:10:30.967 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:29b82759:::10000003083.00000000:head# 2024-09-16T00:10:30.967 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:29bcedda:::1000000309c.00000000:head# 2024-09-16T00:10:30.967 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:29bebe8d:::10000002598.00000000:head# 2024-09-16T00:10:30.968 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:29efae1c:::10000002b29.00000000:head# 2024-09-16T00:10:30.968 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:29f55c0b:::10000001fb6.00000000:head# 2024-09-16T00:10:30.968 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2a1ac447:::10000000568.00000000:head# 2024-09-16T00:10:30.968 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2a3d254e:::1000000282d.00000000:head# 2024-09-16T00:10:30.968 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2a52a257:::1000000247e.00000000:head# 2024-09-16T00:10:30.968 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2a587adf:::100000021d6.00000000:head# 2024-09-16T00:10:30.968 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2a63e1fc:::10000002472.00000000:head# 2024-09-16T00:10:30.968 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2a699bb3:::10000002962.00000000:head# 2024-09-16T00:10:30.968 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2a713e73:::10000000a2a.00000000:head# 2024-09-16T00:10:30.968 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2a75b760:::100000022b1.00000000:head# 2024-09-16T00:10:30.968 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2a8a3e7d:::10000002874.00000000:head# 2024-09-16T00:10:30.968 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2aa40b4a:::10000002cbb.00000000:head# 2024-09-16T00:10:30.971 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2aa7f009:::1000000088f.00000000:head# 2024-09-16T00:10:30.971 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2ac0c492:::10000002920.00000000:head# 2024-09-16T00:10:30.971 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2adb749a:::10000002ebc.00000000:head# 2024-09-16T00:10:30.971 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2ae0acad:::10000002b23.00000000:head# 2024-09-16T00:10:30.971 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2aea0116:::100000006cb.00000000:head# 2024-09-16T00:10:30.971 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2aee4717:::100000009a6.00000000:head# 2024-09-16T00:10:30.972 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2b0e64fc:::10000002456.00000000:head# 2024-09-16T00:10:30.972 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2b163ac1:::100000029ee.00000000:head# 2024-09-16T00:10:30.972 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2b2f79de:::10000000997.00000000:head# 2024-09-16T00:10:30.972 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2b50196b:::10000000558.00000000:head# 2024-09-16T00:10:30.972 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2b68aeaf:::100000020d0.00000000:head# 2024-09-16T00:10:30.972 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2b74aa0a:::1000000248e.00000000:head# 2024-09-16T00:10:30.972 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2b7c73ac:::10000000728.00000000:head# 2024-09-16T00:10:30.972 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2b80a786:::10000003030.00000000:head# 2024-09-16T00:10:30.972 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2b80f830:::10000002257.00000000:head# 2024-09-16T00:10:30.972 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2b9165f6:::10000002208.00000000:head# 2024-09-16T00:10:30.973 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2b94c4dc:::10000002463.00000000:head# 2024-09-16T00:10:30.973 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2bbbaa06:::10000003078.00000000:head# 2024-09-16T00:10:30.973 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2bbf5d77:::10000002946.00000000:head# 2024-09-16T00:10:30.973 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2be42a66:::100000028ef.00000000:head# 2024-09-16T00:10:30.973 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2bf94b1e:::10000002924.00000000:head# 2024-09-16T00:10:30.974 INFO:teuthology.orchestra.run.smithi164.stderr:Read #3:2bfc4227:::100000022eb.00000000:head# 2024-09-16T00:10:30.974 INFO:teuthology.orchestra.run.smithi164.stderr:Export successful 2024-09-16T00:10:31.002 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:31.000+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:31.050 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:31.050 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:31.100 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:31.098+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:31.198 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:10:31.198 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:31.195+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:31.299 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:31.296+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:31.368 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:31.369 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:31.379 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:10:31.430 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:31.430+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:31.488 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --force --op remove --pgid 3.14 2024-09-16T00:10:31.496 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:31.496 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:31.506 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:10:31.533 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:31.531+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:31.601 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:31.601+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:31.654 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:31.654 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:31.663 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:10:31.705 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:31.703+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:31.805 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:31.805+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:31.813 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:31.814 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:31.823 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:10:31.909 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:31.908+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:31.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:31.977+0000 7f8ad37fe640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=113}) -- 0x7f8aac0034d0 con 0x7f8ad415b760 2024-09-16T00:10:31.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:31.977+0000 7f8ad37fe640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] -- ping magic: 0 -- 0x7f8aac0038a0 con 0x7f8ad420a5e0 2024-09-16T00:10:31.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:31.978+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== mon.0 v2:172.21.15.139:3300/0 7 ==== osd_map(113..115 src has 1..115) ==== 1549+0+0 (secure 0 0 0) 0x7f8ac40ab220 con 0x7f8ad415b760 2024-09-16T00:10:31.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:31.978+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8ad41f0e90 con 0x7f8ad415b760 2024-09-16T00:10:31.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:31.978+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8ad420a5e0 msgr2=0x7f8ad420c9d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:10:31.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:31.978+0000 7f8ad2ffd640 1 --2- 172.21.15.139:0/648038927 >> [v2:172.21.15.164:6808/1158838808,v1:172.21.15.164:6809/1158838808] conn(0x7f8ad420a5e0 0x7f8ad420c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:31.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:31.978+0000 7f8ad2ffd640 1 -- 172.21.15.139:0/648038927 <== mon.0 v2:172.21.15.139:3300/0 8 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f8ac4088050 con 0x7f8ad415b760 2024-09-16T00:10:31.981 INFO:teuthology.orchestra.run.smithi139.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T00:10:31.982 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:31.982 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:31.983 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T00:10:31.983 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T00:10:31.989 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:10:32.031 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:32.030+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:32.109 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:32.108+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:32.154 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:32.154 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:32.163 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:10:32.209 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:32.208+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:32.310 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:32.310+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:32.315 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:32.315 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:32.324 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:10:32.411 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:32.410+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:32.423 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T00:10:32.443 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:32.443 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:32.455 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:10:32.512 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:32.511+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:32.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.526+0000 7f9c0fa65640 1 Processor -- start 2024-09-16T00:10:32.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.526+0000 7f9c0fa65640 1 -- start start 2024-09-16T00:10:32.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.527+0000 7f9c0fa65640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c08068060 0x7f9c0815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:32.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.527+0000 7f9c0fa65640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c080cc4b0 con 0x7f9c08154c20 2024-09-16T00:10:32.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.527+0000 7f9c0fa65640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c080cc790 con 0x7f9c080ccc90 2024-09-16T00:10:32.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.527+0000 7f9c0fa65640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9c080c5830 con 0x7f9c08068060 2024-09-16T00:10:32.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.527+0000 7f9c0d000640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c08068060 0x7f9c0815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:32.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.527+0000 7f9c0d801640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f9c08154c20 0x7f9c080c5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:52088/0 (socket says 172.21.15.139:52088) 2024-09-16T00:10:32.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.527+0000 7f9c0d801640 1 -- 172.21.15.139:0/2174744661 learned_addr learned my addr 172.21.15.139:0/2174744661 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:32.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.527+0000 7f9c0e002640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f9c080ccc90 0x7f9c08165180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:53694/0 (socket says 172.21.15.139:53694) 2024-09-16T00:10:32.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.527+0000 7f9bfffff640 1 -- 172.21.15.139:0/2174744661 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4031797176 0 0) 0x7f9c080cc4b0 con 0x7f9c08154c20 2024-09-16T00:10:32.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.527+0000 7f9bfffff640 1 -- 172.21.15.139:0/2174744661 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bf4003520 con 0x7f9c08154c20 2024-09-16T00:10:32.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.527+0000 7f9bfffff640 1 -- 172.21.15.139:0/2174744661 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 759635617 0 0) 0x7f9c080cc790 con 0x7f9c080ccc90 2024-09-16T00:10:32.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.527+0000 7f9bfffff640 1 -- 172.21.15.139:0/2174744661 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c080cc4b0 con 0x7f9c080ccc90 2024-09-16T00:10:32.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.527+0000 7f9c0d000640 1 -- 172.21.15.139:0/2174744661 >> v1:172.21.15.164:6789/0 conn(0x7f9c080ccc90 legacy=0x7f9c08165180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:32.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.527+0000 7f9c0d000640 1 -- 172.21.15.139:0/2174744661 >> v1:172.21.15.139:6789/0 conn(0x7f9c08154c20 legacy=0x7f9c080c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:32.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.527+0000 7f9c0d000640 1 -- 172.21.15.139:0/2174744661 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0815ddf0 con 0x7f9c08068060 2024-09-16T00:10:32.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.528+0000 7f9c0d000640 1 --2- 172.21.15.139:0/2174744661 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c08068060 0x7f9c0815b280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80097c0 tx=0x7f9bf802a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7dd74e958d9bfba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:32.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.528+0000 7f9bfffff640 1 -- 172.21.15.139:0/2174744661 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf8037070 con 0x7f9c08068060 2024-09-16T00:10:32.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.528+0000 7f9bfffff640 1 -- 172.21.15.139:0/2174744661 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf8032440 con 0x7f9c08068060 2024-09-16T00:10:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.528+0000 7f9bfffff640 1 -- 172.21.15.139:0/2174744661 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf80327f0 con 0x7f9c08068060 2024-09-16T00:10:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.529+0000 7f9c0fa65640 1 -- 172.21.15.139:0/2174744661 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c08068060 msgr2=0x7f9c0815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.529+0000 7f9c0fa65640 1 --2- 172.21.15.139:0/2174744661 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c08068060 0x7f9c0815b280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80097c0 tx=0x7f9bf802a290 comp rx=0 tx=0).stop 2024-09-16T00:10:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.529+0000 7f9c0fa65640 1 -- 172.21.15.139:0/2174744661 shutdown_connections 2024-09-16T00:10:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.529+0000 7f9c0fa65640 1 --2- 172.21.15.139:0/2174744661 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c08068060 0x7f9c0815b280 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.529+0000 7f9c0fa65640 1 -- 172.21.15.139:0/2174744661 >> 172.21.15.139:0/2174744661 conn(0x7f9c0806ca40 msgr2=0x7f9c08067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.529+0000 7f9c0fa65640 1 -- 172.21.15.139:0/2174744661 shutdown_connections 2024-09-16T00:10:32.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.529+0000 7f9c0fa65640 1 -- 172.21.15.139:0/2174744661 wait complete. 2024-09-16T00:10:32.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.530+0000 7f9c0fa65640 1 Processor -- start 2024-09-16T00:10:32.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.531+0000 7f9c0fa65640 1 -- start start 2024-09-16T00:10:32.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.531+0000 7f9c0fa65640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c08068060 0x7f9c081f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:32.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.531+0000 7f9c0fa65640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c080ccc90 0x7f9c080c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:32.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.531+0000 7f9c0fa65640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c08154c20 0x7f9c081f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:32.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.531+0000 7f9c0fa65640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9c080cc260 con 0x7f9c08154c20 2024-09-16T00:10:32.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.531+0000 7f9c0fa65640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0815e3b0 con 0x7f9c080ccc90 2024-09-16T00:10:32.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.531+0000 7f9c0fa65640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9c0815e690 con 0x7f9c08068060 2024-09-16T00:10:32.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.531+0000 7f9c0e002640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c08154c20 0x7f9c081f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:32.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.531+0000 7f9c0e002640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c08154c20 0x7f9c081f4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49488/0 (socket says 172.21.15.139:49488) 2024-09-16T00:10:32.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.531+0000 7f9c0e002640 1 -- 172.21.15.139:0/37254 learned_addr learned my addr 172.21.15.139:0/37254 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:32.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.531+0000 7f9c0d000640 1 --2- 172.21.15.139:0/37254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c080ccc90 0x7f9c080c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:32.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.531+0000 7f9c0d801640 1 --2- 172.21.15.139:0/37254 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c08068060 0x7f9c081f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:32.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.532+0000 7f9c0e002640 1 -- 172.21.15.139:0/37254 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c08068060 msgr2=0x7f9c081f13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:32.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.532+0000 7f9c0e002640 1 --2- 172.21.15.139:0/37254 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c08068060 0x7f9c081f13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:32.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.532+0000 7f9c0e002640 1 -- 172.21.15.139:0/37254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c080ccc90 msgr2=0x7f9c080c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:32.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.532+0000 7f9c0e002640 1 --2- 172.21.15.139:0/37254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c080ccc90 0x7f9c080c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:32.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.532+0000 7f9c0e002640 1 -- 172.21.15.139:0/37254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c080d0b00 con 0x7f9c08154c20 2024-09-16T00:10:32.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.532+0000 7f9c0d801640 1 --2- 172.21.15.139:0/37254 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c08068060 0x7f9c081f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:10:32.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.532+0000 7f9c0e002640 1 --2- 172.21.15.139:0/37254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c08154c20 0x7f9c081f4140 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000c700 tx=0x7f9c0000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:32.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.532+0000 7f9bfdffb640 1 -- 172.21.15.139:0/37254 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00017070 con 0x7f9c08154c20 2024-09-16T00:10:32.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.532+0000 7f9bfdffb640 1 -- 172.21.15.139:0/37254 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c0000ce10 con 0x7f9c08154c20 2024-09-16T00:10:32.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.532+0000 7f9c0fa65640 1 -- 172.21.15.139:0/37254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0815d6b0 con 0x7f9c08154c20 2024-09-16T00:10:32.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.532+0000 7f9bfdffb640 1 -- 172.21.15.139:0/37254 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000a860 con 0x7f9c08154c20 2024-09-16T00:10:32.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.532+0000 7f9c0fa65640 1 -- 172.21.15.139:0/37254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0815cad0 con 0x7f9c08154c20 2024-09-16T00:10:32.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.533+0000 7f9c0fa65640 1 -- 172.21.15.139:0/37254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0815d0c0 con 0x7f9c08154c20 2024-09-16T00:10:32.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.533+0000 7f9bfdffb640 1 -- 172.21.15.139:0/37254 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9c0001a070 con 0x7f9c08154c20 2024-09-16T00:10:32.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.534+0000 7f9bfdffb640 1 --2- 172.21.15.139:0/37254 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9bec06c2b0 0x7f9bec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:32.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.534+0000 7f9bfdffb640 1 -- 172.21.15.139:0/37254 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5973+0+0 (secure 0 0 0) 0x7f9c0008aa30 con 0x7f9c08154c20 2024-09-16T00:10:32.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.534+0000 7f9c0d801640 1 --2- 172.21.15.139:0/37254 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9bec06c2b0 0x7f9bec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:32.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.534+0000 7f9c0d801640 1 --2- 172.21.15.139:0/37254 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9bec06c2b0 0x7f9bec06e770 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9c04007fd0 tx=0x7f9c04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:32.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.538+0000 7f9bfdffb640 1 -- 172.21.15.139:0/37254 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9c00058ad0 con 0x7f9c08154c20 2024-09-16T00:10:32.577 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:32.577 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:32.586 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:10:32.627 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:32.626+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:32.715 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:32.714+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:32.734 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:32.734 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:32.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:32.734+0000 7f9c0fa65640 1 -- 172.21.15.139:0/37254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f9c080686b0 con 0x7f9c08154c20 2024-09-16T00:10:32.743 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:10:32.816 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:32.815+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:32.852 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:32.852 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:32.861 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:10:32.917 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:32.915+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:32.930 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:10:32.933 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:10:32.974 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:32.974 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:32.983 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:10:33.017 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:33.017+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:33.118 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:33.117+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:33.158 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:33.159 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:33.171 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:10:33.219 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:33.218+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:33.320 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:33.319+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:33.324 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:33.324 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:33.333 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:10:33.421 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:33.420+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:33.444 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:33.444 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:33.451 INFO:teuthology.orchestra.run.smithi164.stdout: marking collection for removal 2024-09-16T00:10:33.451 INFO:teuthology.orchestra.run.smithi164.stdout:setting '_remove' omap key 2024-09-16T00:10:33.451 INFO:teuthology.orchestra.run.smithi164.stdout:finish_remove_pgs 3.14_head removing 3.14 2024-09-16T00:10:33.453 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:10:33.456 INFO:teuthology.orchestra.run.smithi164.stdout:Remove successful 2024-09-16T00:10:33.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:33.458+0000 7f9bfdffb640 1 -- 172.21.15.139:0/37254 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v116) ==== 97+0+0 (secure 0 0 0) 0x7f9c0005cdd0 con 0x7f9c08154c20 2024-09-16T00:10:33.459 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is set 2024-09-16T00:10:33.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:33.461+0000 7f9c0fa65640 1 -- 172.21.15.139:0/37254 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9bec06c2b0 msgr2=0x7f9bec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:33.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:33.461+0000 7f9c0fa65640 1 --2- 172.21.15.139:0/37254 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9bec06c2b0 0x7f9bec06e770 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9c04007fd0 tx=0x7f9c04009040 comp rx=0 tx=0).stop 2024-09-16T00:10:33.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:33.461+0000 7f9c0fa65640 1 -- 172.21.15.139:0/37254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c08154c20 msgr2=0x7f9c081f4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:33.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:33.461+0000 7f9c0fa65640 1 --2- 172.21.15.139:0/37254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c08154c20 0x7f9c081f4140 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000c700 tx=0x7f9c0000cbc0 comp rx=0 tx=0).stop 2024-09-16T00:10:33.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:33.461+0000 7f9c0fa65640 1 -- 172.21.15.139:0/37254 shutdown_connections 2024-09-16T00:10:33.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:33.461+0000 7f9c0fa65640 1 --2- 172.21.15.139:0/37254 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9bec06c2b0 0x7f9bec06e770 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:33.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:33.461+0000 7f9c0fa65640 1 --2- 172.21.15.139:0/37254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c08154c20 0x7f9c081f4140 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:33.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:33.461+0000 7f9c0fa65640 1 --2- 172.21.15.139:0/37254 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9c080ccc90 0x7f9c080c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:33.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:33.461+0000 7f9c0fa65640 1 --2- 172.21.15.139:0/37254 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9c08068060 0x7f9c081f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:33.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:33.461+0000 7f9c0fa65640 1 -- 172.21.15.139:0/37254 >> 172.21.15.139:0/37254 conn(0x7f9c0806ca40 msgr2=0x7f9c08153c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:33.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:33.461+0000 7f9c0fa65640 1 -- 172.21.15.139:0/37254 shutdown_connections 2024-09-16T00:10:33.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:33.461+0000 7f9c0fa65640 1 -- 172.21.15.139:0/37254 wait complete. 2024-09-16T00:10:33.522 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:33.521+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:33.574 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:33.574 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:33.583 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:10:33.624 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:33.623+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:33.693 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:33.693 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:33.702 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:10:33.746 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:33.746+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:33.808 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:33.809 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:33.817 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:10:33.824 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:33.823+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:33.924 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:33.924+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:33.964 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.14.7 2024-09-16T00:10:33.967 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:33.967 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:33.976 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:10:33.984 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:10:34.045 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:34.044+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:34.131 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:34.127+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:34.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.139+0000 7f44af4b6640 1 Processor -- start 2024-09-16T00:10:34.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.140+0000 7f44af4b6640 1 -- start start 2024-09-16T00:10:34.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.142+0000 7f44af4b6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f44a8161b10 0x7f44a80cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:34.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.142+0000 7f44af4b6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f44a80cbd00 0x7f44a80cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:34.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.142+0000 7f44af4b6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44a80cb880 con 0x7f44a8160560 2024-09-16T00:10:34.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.142+0000 7f44af4b6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f44a8161ef0 con 0x7f44a80cbd00 2024-09-16T00:10:34.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.142+0000 7f44af4b6640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f44a80d0150 con 0x7f44a8161b10 2024-09-16T00:10:34.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.142+0000 7f44adcb3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f44a80cbd00 0x7f44a80cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:34.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.142+0000 7f44ae4b4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f44a8161b10 0x7f44a80cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:34.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.142+0000 7f44adcb3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f44a80cbd00 0x7f44a80cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49504/0 (socket says 172.21.15.139:49504) 2024-09-16T00:10:34.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.142+0000 7f44adcb3640 1 -- 172.21.15.139:0/4039412936 learned_addr learned my addr 172.21.15.139:0/4039412936 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.142+0000 7f44adcb3640 1 -- 172.21.15.139:0/4039412936 >> v2:172.21.15.164:3300/0 conn(0x7f44a8161b10 msgr2=0x7f44a80cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.142+0000 7f44adcb3640 1 --2- 172.21.15.139:0/4039412936 >> v2:172.21.15.164:3300/0 conn(0x7f44a8161b10 0x7f44a80cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.142+0000 7f44adcb3640 1 -- 172.21.15.139:0/4039412936 >> v1:172.21.15.164:6789/0 conn(0x7f44a8160560 legacy=0x7f44a81609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:34.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.142+0000 7f44adcb3640 1 -- 172.21.15.139:0/4039412936 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a80d0430 con 0x7f44a80cbd00 2024-09-16T00:10:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.144+0000 7f44adcb3640 1 --2- 172.21.15.139:0/4039412936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a80cbd00 0x7f44a80cc150 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f44a808d330 tx=0x7f44a402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81a66549a029224 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.144+0000 7f44ad4b2640 1 -- 172.21.15.139:0/4039412936 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4037070 con 0x7f44a80cbd00 2024-09-16T00:10:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.144+0000 7f44ad4b2640 1 -- 172.21.15.139:0/4039412936 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a4032640 con 0x7f44a80cbd00 2024-09-16T00:10:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.144+0000 7f44ad4b2640 1 -- 172.21.15.139:0/4039412936 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4032a50 con 0x7f44a80cbd00 2024-09-16T00:10:34.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.145+0000 7f44af4b6640 1 -- 172.21.15.139:0/4039412936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a80cbd00 msgr2=0x7f44a80cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:34.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.145+0000 7f44af4b6640 1 --2- 172.21.15.139:0/4039412936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a80cbd00 0x7f44a80cc150 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f44a808d330 tx=0x7f44a402a2c0 comp rx=0 tx=0).stop 2024-09-16T00:10:34.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.145+0000 7f44af4b6640 1 -- 172.21.15.139:0/4039412936 shutdown_connections 2024-09-16T00:10:34.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.145+0000 7f44af4b6640 1 --2- 172.21.15.139:0/4039412936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a80cbd00 0x7f44a80cc150 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:34.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.145+0000 7f44af4b6640 1 --2- 172.21.15.139:0/4039412936 >> v2:172.21.15.164:3300/0 conn(0x7f44a8161b10 0x7f44a80cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:34.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.145+0000 7f44af4b6640 1 -- 172.21.15.139:0/4039412936 >> 172.21.15.139:0/4039412936 conn(0x7f44a806dd30 msgr2=0x7f44a806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:34.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.145+0000 7f44af4b6640 1 -- 172.21.15.139:0/4039412936 shutdown_connections 2024-09-16T00:10:34.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.145+0000 7f44af4b6640 1 -- 172.21.15.139:0/4039412936 wait complete. 2024-09-16T00:10:34.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.147+0000 7f44af4b6640 1 Processor -- start 2024-09-16T00:10:34.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.147+0000 7f44af4b6640 1 -- start start 2024-09-16T00:10:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.148+0000 7f44af4b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a80cbd00 0x7f44a8162880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.148+0000 7f44af4b6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f44a8160560 0x7f44a8159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44ae4b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a80cbd00 0x7f44a8162880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44af4b6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a8161b10 0x7f44a8159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44ae4b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a80cbd00 0x7f44a8162880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49512/0 (socket says 172.21.15.139:49512) 2024-09-16T00:10:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44adcb3640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f44a8160560 0x7f44a8159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44adcb3640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f44a8160560 0x7f44a8159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:45512/0 (socket says 172.21.15.139:45512) 2024-09-16T00:10:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44af4b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f44a80cd700 con 0x7f44a80cbd00 2024-09-16T00:10:34.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44af4b6640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f44a80d09f0 con 0x7f44a8161b10 2024-09-16T00:10:34.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44ae4b4640 1 -- 172.21.15.139:0/1037440961 learned_addr learned my addr 172.21.15.139:0/1037440961 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:34.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44af4b6640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f44a80cb420 con 0x7f44a8160560 2024-09-16T00:10:34.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44aecb5640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a8161b10 0x7f44a8159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:34.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44aecb5640 1 -- 172.21.15.139:0/1037440961 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f44a8160560 msgr2=0x7f44a8159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:34.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44aecb5640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f44a8160560 0x7f44a8159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:34.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44aecb5640 1 -- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a80cbd00 msgr2=0x7f44a8162880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:34.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44aecb5640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a80cbd00 0x7f44a8162880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:34.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44aecb5640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a81f4ad0 con 0x7f44a8161b10 2024-09-16T00:10:34.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44adcb3640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f44a8160560 0x7f44a8159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:34.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44ae4b4640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44a80cbd00 0x7f44a8162880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:34.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.149+0000 7f44aecb5640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f44a8161b10 0x7f44a8159f40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f44a000c760 tx=0x7f44a000cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:34.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.150+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0016020 con 0x7f44a8161b10 2024-09-16T00:10:34.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.150+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a0006d50 con 0x7f44a8161b10 2024-09-16T00:10:34.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.150+0000 7f44af4b6640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a81f4db0 con 0x7f44a8161b10 2024-09-16T00:10:34.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.150+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0003bb0 con 0x7f44a8161b10 2024-09-16T00:10:34.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.150+0000 7f44af4b6640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a81f5370 con 0x7f44a8161b10 2024-09-16T00:10:34.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.151+0000 7f44af4b6640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f44a80cb0b0 con 0x7f44a8161b10 2024-09-16T00:10:34.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.152+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f44a000a850 con 0x7f44a8161b10 2024-09-16T00:10:34.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.152+0000 7f449affd640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f448c06c2b0 0x7f448c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:34.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.152+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5973+0+0 (secure 0 0 0) 0x7f44a008b080 con 0x7f44a8161b10 2024-09-16T00:10:34.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.152+0000 7f44ae4b4640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f448c06c2b0 0x7f448c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:34.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.152+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v116) ==== 54+0+16 (secure 0 0 0) 0x7f44a0020020 con 0x7f44a8161b10 2024-09-16T00:10:34.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.153+0000 7f4498ff9640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f4484001650 0x7f4484003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:34.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.153+0000 7f4498ff9640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4484006bb0 con 0x7f4484001650 2024-09-16T00:10:34.154 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.153+0000 7f44ae4b4640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f448c06c2b0 0x7f448c06e770 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f449c009f50 tx=0x7f449c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:34.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.153+0000 7f44adcb3640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f4484001650 0x7f4484003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:34.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.153+0000 7f44adcb3640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f4484001650 0x7f4484003b10 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:34.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.154+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4484006bb0 con 0x7f4484001650 2024-09-16T00:10:34.157 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:34.157 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:34.169 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:10:34.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.169+0000 7f4498ff9640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4484005c00 con 0x7f4484001650 2024-09-16T00:10:34.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.170+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4484005c00 con 0x7f4484001650 2024-09-16T00:10:34.171 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:34.172 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:34.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.171+0000 7f4498ff9640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f4484004190 0x7f4484004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:34.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.171+0000 7f4498ff9640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f4484004b80 con 0x7f4484004190 2024-09-16T00:10:34.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.171+0000 7f44ae4b4640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f4484004190 0x7f4484004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:34.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.171+0000 7f44ae4b4640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f4484004190 0x7f4484004640 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:34.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.172+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4484004b80 con 0x7f4484004190 2024-09-16T00:10:34.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.187+0000 7f4498ff9640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f44840050f0 con 0x7f4484004190 2024-09-16T00:10:34.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.189+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f44840050f0 con 0x7f4484004190 2024-09-16T00:10:34.190 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:34.190 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:34.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.189+0000 7f4498ff9640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f4484009340 0x7f448400b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:34.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.189+0000 7f4498ff9640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f448400e660 con 0x7f4484009340 2024-09-16T00:10:34.190 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.189+0000 7f44adcb3640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f4484009340 0x7f448400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:34.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.189+0000 7f44adcb3640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f4484009340 0x7f448400b730 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:34.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.190+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f448400e660 con 0x7f4484009340 2024-09-16T00:10:34.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.204+0000 7f4498ff9640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f448400bf40 con 0x7f4484009340 2024-09-16T00:10:34.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.207+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f448400bf40 con 0x7f4484009340 2024-09-16T00:10:34.207 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:34.208 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:34.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.207+0000 7f4498ff9640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f448400c9c0 0x7f448400ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:34.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.207+0000 7f4498ff9640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f448400d5b0 con 0x7f448400c9c0 2024-09-16T00:10:34.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.207+0000 7f44aecb5640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f448400c9c0 0x7f448400ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:34.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.207+0000 7f44aecb5640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f448400c9c0 0x7f448400ce70 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:34.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.208+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f448400d5b0 con 0x7f448400c9c0 2024-09-16T00:10:34.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.224+0000 7f4498ff9640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f448400dac0 con 0x7f448400c9c0 2024-09-16T00:10:34.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.225+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f448400dac0 con 0x7f448400c9c0 2024-09-16T00:10:34.226 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:34.227 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:34.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.225+0000 7f4498ff9640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f448400e090 0x7f4484012830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:34.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.225+0000 7f4498ff9640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f4484015790 con 0x7f448400e090 2024-09-16T00:10:34.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.226+0000 7f44ae4b4640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f448400e090 0x7f4484012830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:34.227 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.226+0000 7f44ae4b4640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f448400e090 0x7f4484012830 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:34.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.227+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4484015790 con 0x7f448400e090 2024-09-16T00:10:34.228 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:34.227+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:34.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.242+0000 7f4498ff9640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f44840144b0 con 0x7f448400e090 2024-09-16T00:10:34.244 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.244+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f44840144b0 con 0x7f448400e090 2024-09-16T00:10:34.245 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:34.245 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:34.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.244+0000 7f4498ff9640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f4484014f90 0x7f4484012d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:34.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.244+0000 7f4498ff9640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f44840153f0 con 0x7f4484014f90 2024-09-16T00:10:34.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.244+0000 7f44adcb3640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f4484014f90 0x7f4484012d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:34.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.245+0000 7f44adcb3640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f4484014f90 0x7f4484012d70 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:34.246 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.245+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f44840153f0 con 0x7f4484014f90 2024-09-16T00:10:34.261 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.260+0000 7f4498ff9640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f44840153f0 con 0x7f4484014f90 2024-09-16T00:10:34.262 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.262+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f44840153f0 con 0x7f4484014f90 2024-09-16T00:10:34.263 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:34.263 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:34.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.262+0000 7f4498ff9640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f44840137c0 0x7f4484013c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:34.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.262+0000 7f4498ff9640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f44840142a0 con 0x7f44840137c0 2024-09-16T00:10:34.263 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.262+0000 7f44aecb5640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f44840137c0 0x7f4484013c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:34.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.263+0000 7f44aecb5640 1 --2- 172.21.15.139:0/1037440961 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f44840137c0 0x7f4484013c00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:34.264 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.263+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f44840142a0 con 0x7f44840137c0 2024-09-16T00:10:34.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.279+0000 7f4498ff9640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f44840146e0 con 0x7f44840137c0 2024-09-16T00:10:34.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.281+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f44840146e0 con 0x7f44840137c0 2024-09-16T00:10:34.282 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:34.282 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:34.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.281+0000 7f4498ff9640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=117}) -- 0x7f448401b350 con 0x7f44a8161b10 2024-09-16T00:10:34.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.281+0000 7f4498ff9640 1 -- 172.21.15.139:0/1037440961 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f44840146e0 con 0x7f44a8161b10 2024-09-16T00:10:34.283 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:34.281+0000 7f449affd640 1 -- 172.21.15.139:0/1037440961 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7f44a0092070 con 0x7f44a8161b10 2024-09-16T00:10:34.283 INFO:teuthology.orchestra.run.smithi139.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T00:10:34.285 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:34.285 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:34.285 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:34.285 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:34.285 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:34.286 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T00:10:34.286 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T00:10:34.294 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:10:34.337 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:34.335+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:34.430 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:34.429+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:34.445 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:34.446 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:34.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:34.446 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:34.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.446 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3161 2.d 2:b02ffd62:::200.00000010:head [write 3427859~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.446 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:34.446 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.780848382999999, 2024-09-16T00:10:34.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026748516, 2024-09-16T00:10:34.446 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.446 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.446 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.446 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:34.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:34.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:34.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.15e-07 2024-09-16T00:10:34.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662637+0000", 2024-09-16T00:10:34.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8009999999999999e-06 2024-09-16T00:10:34.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662645+0000", 2024-09-16T00:10:34.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8299999999999996e-06 2024-09-16T00:10:34.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662650+0000", 2024-09-16T00:10:34.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.605e-06 2024-09-16T00:10:34.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.449 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.449 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.682553+0000", 2024-09-16T00:10:34.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01990331 2024-09-16T00:10:34.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.682636+0000", 2024-09-16T00:10:34.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2921000000000001e-05 2024-09-16T00:10:34.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.682780+0000", 2024-09-16T00:10:34.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000144361 2024-09-16T00:10:34.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.684809+0000", 2024-09-16T00:10:34.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002029005 2024-09-16T00:10:34.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689293+0000", 2024-09-16T00:10:34.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044841079999999997 2024-09-16T00:10:34.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689339+0000", 2024-09-16T00:10:34.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5668999999999999e-05 2024-09-16T00:10:34.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689382+0000", 2024-09-16T00:10:34.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3491000000000001e-05 2024-09-16T00:10:34.452 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.452 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.452 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.452 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3162 2.d 2:b02ffd62:::200.00000010:head [write 3430104~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.452 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.662858+0000", 2024-09-16T00:10:34.452 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.780624267, 2024-09-16T00:10:34.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026630083999999998, 2024-09-16T00:10:34.452 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.453 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.453 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.453 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662858+0000", 2024-09-16T00:10:34.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662858+0000", 2024-09-16T00:10:34.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662859+0000", 2024-09-16T00:10:34.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:10:34.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662862+0000", 2024-09-16T00:10:34.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6119999999999999e-06 2024-09-16T00:10:34.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662870+0000", 2024-09-16T00:10:34.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2360000000000004e-06 2024-09-16T00:10:34.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662874+0000", 2024-09-16T00:10:34.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2610000000000002e-06 2024-09-16T00:10:34.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.683241+0000", 2024-09-16T00:10:34.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.020367495999999999 2024-09-16T00:10:34.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.683295+0000", 2024-09-16T00:10:34.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4230000000000001e-05 2024-09-16T00:10:34.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.683380+0000", 2024-09-16T00:10:34.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4367e-05 2024-09-16T00:10:34.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.684845+0000", 2024-09-16T00:10:34.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014653190000000001 2024-09-16T00:10:34.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689392+0000", 2024-09-16T00:10:34.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045474280000000001 2024-09-16T00:10:34.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689445+0000", 2024-09-16T00:10:34.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2444000000000001e-05 2024-09-16T00:10:34.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689488+0000", 2024-09-16T00:10:34.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3124999999999998e-05 2024-09-16T00:10:34.458 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.458 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.458 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.458 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3166 2.d 2:b02ffd62:::200.00000010:head [write 3437392~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.458 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.664403+0000", 2024-09-16T00:10:34.458 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.779079389000003, 2024-09-16T00:10:34.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.027480674, 2024-09-16T00:10:34.458 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.458 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.458 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.458 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664403+0000", 2024-09-16T00:10:34.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664403+0000", 2024-09-16T00:10:34.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664404+0000", 2024-09-16T00:10:34.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4099999999999998e-07 2024-09-16T00:10:34.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664408+0000", 2024-09-16T00:10:34.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8169999999999998e-06 2024-09-16T00:10:34.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664418+0000", 2024-09-16T00:10:34.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6909999999999992e-06 2024-09-16T00:10:34.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664425+0000", 2024-09-16T00:10:34.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6309999999999997e-06 2024-09-16T00:10:34.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.685372+0000", 2024-09-16T00:10:34.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.020947548999999999 2024-09-16T00:10:34.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.461 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.685454+0000", 2024-09-16T00:10:34.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1355999999999998e-05 2024-09-16T00:10:34.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.685585+0000", 2024-09-16T00:10:34.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000130922 2024-09-16T00:10:34.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687329+0000", 2024-09-16T00:10:34.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017445710000000001 2024-09-16T00:10:34.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691757+0000", 2024-09-16T00:10:34.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004428268 2024-09-16T00:10:34.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.463 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691830+0000", 2024-09-16T00:10:34.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2237999999999995e-05 2024-09-16T00:10:34.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.463 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691884+0000", 2024-09-16T00:10:34.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3890000000000001e-05 2024-09-16T00:10:34.463 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.464 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.464 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.464 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3167 2.d 2:b02ffd62:::200.00000010:head [write 3439637~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.464 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.664627+0000", 2024-09-16T00:10:34.464 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.778854817000003, 2024-09-16T00:10:34.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.027347698, 2024-09-16T00:10:34.464 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.464 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.464 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.464 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664627+0000", 2024-09-16T00:10:34.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664627+0000", 2024-09-16T00:10:34.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664628+0000", 2024-09-16T00:10:34.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0500000000000003e-07 2024-09-16T00:10:34.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664632+0000", 2024-09-16T00:10:34.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8770000000000003e-06 2024-09-16T00:10:34.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664639+0000", 2024-09-16T00:10:34.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6590000000000001e-06 2024-09-16T00:10:34.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664642+0000", 2024-09-16T00:10:34.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8550000000000004e-06 2024-09-16T00:10:34.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.686530+0000", 2024-09-16T00:10:34.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.021887127999999999 2024-09-16T00:10:34.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.686577+0000", 2024-09-16T00:10:34.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7181000000000001e-05 2024-09-16T00:10:34.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.686667+0000", 2024-09-16T00:10:34.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0074000000000005e-05 2024-09-16T00:10:34.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688714+0000", 2024-09-16T00:10:34.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020467749999999998 2024-09-16T00:10:34.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691894+0000", 2024-09-16T00:10:34.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031806529999999999 2024-09-16T00:10:34.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691939+0000", 2024-09-16T00:10:34.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4579999999999997e-05 2024-09-16T00:10:34.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691975+0000", 2024-09-16T00:10:34.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6310999999999998e-05 2024-09-16T00:10:34.469 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.469 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.469 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.469 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3168 2.d 2:b02ffd62:::200.00000010:head [write 3441036~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.470 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:34.470 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.778388735999997, 2024-09-16T00:10:34.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026965759999999998, 2024-09-16T00:10:34.470 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.470 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.470 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.470 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:34.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:34.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:34.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7999999999999996e-07 2024-09-16T00:10:34.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665098+0000", 2024-09-16T00:10:34.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3200000000000001e-06 2024-09-16T00:10:34.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665104+0000", 2024-09-16T00:10:34.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4809999999999997e-06 2024-09-16T00:10:34.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665112+0000", 2024-09-16T00:10:34.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.67e-06 2024-09-16T00:10:34.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687392+0000", 2024-09-16T00:10:34.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.022279699 2024-09-16T00:10:34.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687461+0000", 2024-09-16T00:10:34.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8476e-05 2024-09-16T00:10:34.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.473 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687577+0000", 2024-09-16T00:10:34.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011661 2024-09-16T00:10:34.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.473 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.473 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688774+0000", 2024-09-16T00:10:34.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011962660000000001 2024-09-16T00:10:34.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691984+0000", 2024-09-16T00:10:34.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032106320000000002 2024-09-16T00:10:34.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692028+0000", 2024-09-16T00:10:34.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3866999999999999e-05 2024-09-16T00:10:34.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692059+0000", 2024-09-16T00:10:34.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1259e-05 2024-09-16T00:10:34.475 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.475 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.475 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.475 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3169 2.d 2:b02ffd62:::200.00000010:head [write 3443281~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.475 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:34.475 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.777932200000002, 2024-09-16T00:10:34.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.030090967999999999, 2024-09-16T00:10:34.475 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.476 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.476 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.476 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:34.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:34.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:34.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3799999999999998e-07 2024-09-16T00:10:34.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665554+0000", 2024-09-16T00:10:34.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.112e-06 2024-09-16T00:10:34.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665559+0000", 2024-09-16T00:10:34.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9559999999999996e-06 2024-09-16T00:10:34.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665562+0000", 2024-09-16T00:10:34.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2909999999999999e-06 2024-09-16T00:10:34.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687954+0000", 2024-09-16T00:10:34.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.022391714 2024-09-16T00:10:34.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688016+0000", 2024-09-16T00:10:34.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2384000000000004e-05 2024-09-16T00:10:34.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688147+0000", 2024-09-16T00:10:34.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013099300000000001 2024-09-16T00:10:34.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690321+0000", 2024-09-16T00:10:34.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021744339999999998 2024-09-16T00:10:34.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695548+0000", 2024-09-16T00:10:34.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052266079999999998 2024-09-16T00:10:34.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695595+0000", 2024-09-16T00:10:34.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7200000000000002e-05 2024-09-16T00:10:34.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695641+0000", 2024-09-16T00:10:34.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5837999999999997e-05 2024-09-16T00:10:34.481 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.481 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.481 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.481 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3170 2.d 2:b02ffd62:::200.00000010:head [write 3445526~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.481 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:34.481 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.777739138999998, 2024-09-16T00:10:34.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029987413000000001, 2024-09-16T00:10:34.481 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.481 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.481 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.481 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:34.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:34.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:34.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7500000000000001e-07 2024-09-16T00:10:34.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665753+0000", 2024-09-16T00:10:34.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0005e-05 2024-09-16T00:10:34.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665759+0000", 2024-09-16T00:10:34.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.169e-06 2024-09-16T00:10:34.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665762+0000", 2024-09-16T00:10:34.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8460000000000001e-06 2024-09-16T00:10:34.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688811+0000", 2024-09-16T00:10:34.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023048169 2024-09-16T00:10:34.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688885+0000", 2024-09-16T00:10:34.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4788000000000003e-05 2024-09-16T00:10:34.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688989+0000", 2024-09-16T00:10:34.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000103689 2024-09-16T00:10:34.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690350+0000", 2024-09-16T00:10:34.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013609329999999999 2024-09-16T00:10:34.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.485 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.485 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695652+0000", 2024-09-16T00:10:34.485 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053019080000000001 2024-09-16T00:10:34.485 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.485 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.486 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.486 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695698+0000", 2024-09-16T00:10:34.486 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5775000000000001e-05 2024-09-16T00:10:34.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.486 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.486 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695731+0000", 2024-09-16T00:10:34.486 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2756000000000003e-05 2024-09-16T00:10:34.486 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.486 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.486 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.486 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.486 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.487 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3171 2.d 2:b02ffd62:::200.00000010:head [write 3446925~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.487 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665997+0000", 2024-09-16T00:10:34.487 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.777485048000003, 2024-09-16T00:10:34.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029824032, 2024-09-16T00:10:34.487 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.487 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.487 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.487 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.487 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.487 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665997+0000", 2024-09-16T00:10:34.487 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.487 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.487 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665997+0000", 2024-09-16T00:10:34.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665998+0000", 2024-09-16T00:10:34.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:10:34.488 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.488 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.488 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.488 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666000+0000", 2024-09-16T00:10:34.488 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6290000000000002e-06 2024-09-16T00:10:34.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666005+0000", 2024-09-16T00:10:34.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.437e-06 2024-09-16T00:10:34.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.489 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666008+0000", 2024-09-16T00:10:34.489 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7910000000000002e-06 2024-09-16T00:10:34.489 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.489 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.489 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689799+0000", 2024-09-16T00:10:34.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023790504000000001 2024-09-16T00:10:34.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689907+0000", 2024-09-16T00:10:34.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108464 2024-09-16T00:10:34.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.490 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.490 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689994+0000", 2024-09-16T00:10:34.490 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7043e-05 2024-09-16T00:10:34.490 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.490 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.491 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.491 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691737+0000", 2024-09-16T00:10:34.491 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001742527 2024-09-16T00:10:34.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.491 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.491 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695739+0000", 2024-09-16T00:10:34.491 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040020189999999999 2024-09-16T00:10:34.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.491 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.491 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.491 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695786+0000", 2024-09-16T00:10:34.491 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7219999999999999e-05 2024-09-16T00:10:34.491 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.492 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.492 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695821+0000", 2024-09-16T00:10:34.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5058000000000002e-05 2024-09-16T00:10:34.492 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.492 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.492 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.492 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.492 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.492 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3172 2.d 2:b02ffd62:::200.00000010:head [write 3448324~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.492 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.666428+0000", 2024-09-16T00:10:34.492 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.777054098000001, 2024-09-16T00:10:34.492 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029477316, 2024-09-16T00:10:34.493 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.493 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.493 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.493 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.493 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.493 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666428+0000", 2024-09-16T00:10:34.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.493 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.493 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.493 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.493 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666428+0000", 2024-09-16T00:10:34.493 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666429+0000", 2024-09-16T00:10:34.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3699999999999999e-07 2024-09-16T00:10:34.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666433+0000", 2024-09-16T00:10:34.494 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4939999999999997e-06 2024-09-16T00:10:34.494 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.494 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.494 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.494 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666443+0000", 2024-09-16T00:10:34.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0899999999999994e-06 2024-09-16T00:10:34.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666447+0000", 2024-09-16T00:10:34.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8930000000000004e-06 2024-09-16T00:10:34.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.495 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690390+0000", 2024-09-16T00:10:34.495 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023942215999999999 2024-09-16T00:10:34.495 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.495 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.495 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.496 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690464+0000", 2024-09-16T00:10:34.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4678999999999994e-05 2024-09-16T00:10:34.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.496 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.496 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690576+0000", 2024-09-16T00:10:34.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011162000000000001 2024-09-16T00:10:34.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.496 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.496 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.496 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692126+0000", 2024-09-16T00:10:34.496 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015496520000000001 2024-09-16T00:10:34.496 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.497 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695828+0000", 2024-09-16T00:10:34.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037028510000000001 2024-09-16T00:10:34.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.497 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695870+0000", 2024-09-16T00:10:34.497 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1180999999999998e-05 2024-09-16T00:10:34.497 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.497 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.497 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.497 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695905+0000", 2024-09-16T00:10:34.498 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5803000000000003e-05 2024-09-16T00:10:34.498 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.498 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.498 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.498 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.498 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.498 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3173 2.d 2:b02ffd62:::200.00000010:head [write 3449723~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.498 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.666884+0000", 2024-09-16T00:10:34.498 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.776598141999997, 2024-09-16T00:10:34.498 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.02908875, 2024-09-16T00:10:34.498 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.498 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.498 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.498 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.499 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666884+0000", 2024-09-16T00:10:34.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.499 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666884+0000", 2024-09-16T00:10:34.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.499 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.499 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.499 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.499 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666885+0000", 2024-09-16T00:10:34.499 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0999999999999998e-07 2024-09-16T00:10:34.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.500 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.500 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666889+0000", 2024-09-16T00:10:34.500 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4299999999999999e-06 2024-09-16T00:10:34.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.500 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.500 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666895+0000", 2024-09-16T00:10:34.500 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0510000000000002e-06 2024-09-16T00:10:34.500 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.500 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.500 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666899+0000", 2024-09-16T00:10:34.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0489999999999997e-06 2024-09-16T00:10:34.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.501 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691161+0000", 2024-09-16T00:10:34.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.024262156999999999 2024-09-16T00:10:34.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.501 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.501 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.501 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691253+0000", 2024-09-16T00:10:34.501 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1278000000000003e-05 2024-09-16T00:10:34.501 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.502 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.502 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691348+0000", 2024-09-16T00:10:34.502 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4777000000000004e-05 2024-09-16T00:10:34.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.502 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.502 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692211+0000", 2024-09-16T00:10:34.502 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086351699999999995 2024-09-16T00:10:34.502 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.502 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.502 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.502 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695913+0000", 2024-09-16T00:10:34.503 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037019840000000002 2024-09-16T00:10:34.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.503 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.503 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695952+0000", 2024-09-16T00:10:34.503 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8535999999999998e-05 2024-09-16T00:10:34.503 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.503 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.503 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.503 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695973+0000", 2024-09-16T00:10:34.503 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1260999999999999e-05 2024-09-16T00:10:34.503 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.503 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.503 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3174 2.d 2:b02ffd62:::200.00000010:head [write 3451968~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.667323+0000", 2024-09-16T00:10:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.776159272000001, 2024-09-16T00:10:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028788859999999999, 2024-09-16T00:10:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.504 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667323+0000", 2024-09-16T00:10:34.505 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.505 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.505 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667323+0000", 2024-09-16T00:10:34.505 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.505 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.505 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667324+0000", 2024-09-16T00:10:34.505 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0400000000000005e-07 2024-09-16T00:10:34.505 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.505 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.505 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.506 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667330+0000", 2024-09-16T00:10:34.506 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3300000000000004e-06 2024-09-16T00:10:34.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.506 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.506 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.506 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667338+0000", 2024-09-16T00:10:34.506 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0059999999999992e-06 2024-09-16T00:10:34.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.506 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.506 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.506 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667344+0000", 2024-09-16T00:10:34.506 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6409999999999999e-06 2024-09-16T00:10:34.506 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.507 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.507 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692253+0000", 2024-09-16T00:10:34.507 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.024909280999999998 2024-09-16T00:10:34.507 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.507 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.507 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692311+0000", 2024-09-16T00:10:34.507 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7873e-05 2024-09-16T00:10:34.507 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.507 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.507 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.507 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692395+0000", 2024-09-16T00:10:34.507 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4071000000000002e-05 2024-09-16T00:10:34.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.508 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.508 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695977+0000", 2024-09-16T00:10:34.508 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035823460000000001 2024-09-16T00:10:34.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.508 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.508 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696034+0000", 2024-09-16T00:10:34.508 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6697999999999998e-05 2024-09-16T00:10:34.508 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.508 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.508 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.508 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696078+0000", 2024-09-16T00:10:34.509 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3696000000000003e-05 2024-09-16T00:10:34.509 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.509 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.509 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.509 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696112+0000", 2024-09-16T00:10:34.509 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4113999999999999e-05 2024-09-16T00:10:34.509 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.509 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.509 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.509 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.509 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.509 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3175 2.d 2:b02ffd62:::200.00000010:head [write 3454213~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.509 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:34.510 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.775818434000001, 2024-09-16T00:10:34.510 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.030153698999999999, 2024-09-16T00:10:34.510 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.510 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.510 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.510 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.510 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.510 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.510 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:34.510 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.510 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.510 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.510 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.510 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:34.511 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.511 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.511 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.511 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:34.511 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2e-07 2024-09-16T00:10:34.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.511 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.511 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.511 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667668+0000", 2024-09-16T00:10:34.511 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2109999999999998e-06 2024-09-16T00:10:34.511 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.511 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.511 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.512 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667673+0000", 2024-09-16T00:10:34.512 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8409999999999996e-06 2024-09-16T00:10:34.512 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.512 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.512 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.512 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667686+0000", 2024-09-16T00:10:34.512 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2928000000000001e-05 2024-09-16T00:10:34.512 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.512 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.512 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.512 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692964+0000", 2024-09-16T00:10:34.512 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.025277642999999999 2024-09-16T00:10:34.512 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.512 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.513 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.513 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.694418+0000", 2024-09-16T00:10:34.513 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001453693 2024-09-16T00:10:34.513 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.513 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.513 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.513 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.694505+0000", 2024-09-16T00:10:34.513 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7700000000000004e-05 2024-09-16T00:10:34.513 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.513 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.513 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.513 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697114+0000", 2024-09-16T00:10:34.513 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026083460000000001 2024-09-16T00:10:34.514 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.514 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.514 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.514 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697697+0000", 2024-09-16T00:10:34.514 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00058297399999999999 2024-09-16T00:10:34.514 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.514 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.514 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.514 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697772+0000", 2024-09-16T00:10:34.514 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5433999999999996e-05 2024-09-16T00:10:34.514 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.514 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.514 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.514 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697818+0000", 2024-09-16T00:10:34.515 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5408999999999998e-05 2024-09-16T00:10:34.515 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.515 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.515 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.515 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.515 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.515 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3176 2.d 2:b02ffd62:::200.00000010:head [write 3456458~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.515 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.668006+0000", 2024-09-16T00:10:34.515 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.775476529000002, 2024-09-16T00:10:34.515 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029935413000000001, 2024-09-16T00:10:34.515 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.515 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.515 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.515 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.516 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.516 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.516 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668006+0000", 2024-09-16T00:10:34.516 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.516 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.516 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.516 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.516 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668006+0000", 2024-09-16T00:10:34.516 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.516 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.516 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.516 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.516 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668007+0000", 2024-09-16T00:10:34.517 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5799999999999998e-07 2024-09-16T00:10:34.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.517 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.517 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.517 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668012+0000", 2024-09-16T00:10:34.517 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9320000000000004e-06 2024-09-16T00:10:34.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.517 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.517 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.517 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668023+0000", 2024-09-16T00:10:34.517 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1273e-05 2024-09-16T00:10:34.517 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.517 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.518 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.518 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668030+0000", 2024-09-16T00:10:34.518 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0509999999999999e-06 2024-09-16T00:10:34.518 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.518 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.518 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.518 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.694928+0000", 2024-09-16T00:10:34.518 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026898114000000001 2024-09-16T00:10:34.518 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.518 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.518 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.518 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695003+0000", 2024-09-16T00:10:34.518 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4880000000000001e-05 2024-09-16T00:10:34.519 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.519 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.519 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.519 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695116+0000", 2024-09-16T00:10:34.519 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011307900000000001 2024-09-16T00:10:34.519 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.519 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.519 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.519 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697119+0000", 2024-09-16T00:10:34.519 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020032159999999999 2024-09-16T00:10:34.519 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.519 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.519 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.519 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697867+0000", 2024-09-16T00:10:34.520 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074744999999999998 2024-09-16T00:10:34.520 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.520 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.520 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.520 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697908+0000", 2024-09-16T00:10:34.520 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1287e-05 2024-09-16T00:10:34.520 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.520 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.520 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.520 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697941+0000", 2024-09-16T00:10:34.520 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3272999999999998e-05 2024-09-16T00:10:34.520 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.520 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.520 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.521 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.521 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.521 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3177 2.d 2:b02ffd62:::200.00000010:head [write 3457857~1398 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.521 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:34.521 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.774141669999999, 2024-09-16T00:10:34.521 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029906856999999998, 2024-09-16T00:10:34.521 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.521 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.521 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.521 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.521 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.521 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.521 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:34.522 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.522 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.522 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.522 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.522 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:34.522 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.522 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.522 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.522 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.522 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:34.522 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1299999999999998e-07 2024-09-16T00:10:34.522 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.522 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.522 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.523 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669347+0000", 2024-09-16T00:10:34.523 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.327e-06 2024-09-16T00:10:34.523 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.523 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.523 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.523 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669359+0000", 2024-09-16T00:10:34.523 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1698e-05 2024-09-16T00:10:34.523 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.523 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.523 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.523 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669366+0000", 2024-09-16T00:10:34.523 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7959999999999996e-06 2024-09-16T00:10:34.523 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.523 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.524 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.524 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696168+0000", 2024-09-16T00:10:34.524 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026802093999999999 2024-09-16T00:10:34.524 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.524 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.524 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.524 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696258+0000", 2024-09-16T00:10:34.524 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0074000000000005e-05 2024-09-16T00:10:34.524 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.524 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.524 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.524 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696383+0000", 2024-09-16T00:10:34.524 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012455699999999999 2024-09-16T00:10:34.525 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.525 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.525 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.525 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698602+0000", 2024-09-16T00:10:34.525 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002218921 2024-09-16T00:10:34.525 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.525 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.525 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.525 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699152+0000", 2024-09-16T00:10:34.525 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055024399999999997 2024-09-16T00:10:34.525 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.525 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.525 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.525 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699206+0000", 2024-09-16T00:10:34.526 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3882999999999999e-05 2024-09-16T00:10:34.526 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.526 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.526 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.526 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699247+0000", 2024-09-16T00:10:34.526 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1350000000000002e-05 2024-09-16T00:10:34.526 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.526 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.526 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.526 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.526 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.526 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3178 2.d 2:b02ffd62:::200.00000010:head [write 3459255~2511 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.526 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.671779+0000", 2024-09-16T00:10:34.526 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.771702789999999, 2024-09-16T00:10:34.527 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028813986999999999, 2024-09-16T00:10:34.527 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.527 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.527 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.527 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.527 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.527 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.527 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671779+0000", 2024-09-16T00:10:34.527 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.527 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.527 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.527 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.527 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671779+0000", 2024-09-16T00:10:34.527 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.528 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.528 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.528 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.528 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671780+0000", 2024-09-16T00:10:34.528 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3600000000000002e-07 2024-09-16T00:10:34.528 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.528 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.528 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.528 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671785+0000", 2024-09-16T00:10:34.528 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.074e-06 2024-09-16T00:10:34.528 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.528 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.528 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.529 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671795+0000", 2024-09-16T00:10:34.529 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2040000000000003e-06 2024-09-16T00:10:34.529 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.529 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.529 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.529 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671800+0000", 2024-09-16T00:10:34.529 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9420000000000003e-06 2024-09-16T00:10:34.529 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.529 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.529 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.529 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697159+0000", 2024-09-16T00:10:34.529 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.025359619999999999 2024-09-16T00:10:34.529 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.529 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.530 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.530 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697214+0000", 2024-09-16T00:10:34.530 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5318000000000001e-05 2024-09-16T00:10:34.530 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.530 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.530 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.530 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697297+0000", 2024-09-16T00:10:34.530 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2087000000000002e-05 2024-09-16T00:10:34.530 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.530 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.530 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.530 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699258+0000", 2024-09-16T00:10:34.530 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019619529999999998 2024-09-16T00:10:34.531 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.531 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.531 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.531 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700499+0000", 2024-09-16T00:10:34.531 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001240072 2024-09-16T00:10:34.531 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.531 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.531 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.531 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700555+0000", 2024-09-16T00:10:34.531 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6459999999999998e-05 2024-09-16T00:10:34.531 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.531 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.531 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.531 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700593+0000", 2024-09-16T00:10:34.532 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8420999999999997e-05 2024-09-16T00:10:34.532 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.532 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.532 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.532 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.532 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.532 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3179 2.d 2:b02ffd62:::200.00000010:head [write 3461766~2418 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.532 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.672548+0000", 2024-09-16T00:10:34.532 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.770934693000001, 2024-09-16T00:10:34.532 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028160178000000001, 2024-09-16T00:10:34.532 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.532 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.532 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.533 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.533 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.533 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672548+0000", 2024-09-16T00:10:34.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.533 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.533 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.533 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672548+0000", 2024-09-16T00:10:34.533 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.533 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.533 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.533 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.533 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672549+0000", 2024-09-16T00:10:34.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5300000000000002e-07 2024-09-16T00:10:34.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.534 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.534 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.534 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672562+0000", 2024-09-16T00:10:34.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3808000000000001e-05 2024-09-16T00:10:34.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.534 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.534 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.534 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672575+0000", 2024-09-16T00:10:34.534 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2870999999999999e-05 2024-09-16T00:10:34.534 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.534 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.535 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.535 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672587+0000", 2024-09-16T00:10:34.535 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.203e-05 2024-09-16T00:10:34.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.535 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.535 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.535 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697994+0000", 2024-09-16T00:10:34.535 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.025406373999999999 2024-09-16T00:10:34.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.535 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.535 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.535 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698074+0000", 2024-09-16T00:10:34.535 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0802000000000003e-05 2024-09-16T00:10:34.535 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.536 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.536 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.536 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698196+0000", 2024-09-16T00:10:34.536 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012154699999999999 2024-09-16T00:10:34.536 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.536 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.536 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.536 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700456+0000", 2024-09-16T00:10:34.536 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022602249999999998 2024-09-16T00:10:34.536 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.536 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.536 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.536 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700632+0000", 2024-09-16T00:10:34.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017570699999999999 2024-09-16T00:10:34.537 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.537 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.537 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.537 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700676+0000", 2024-09-16T00:10:34.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3764000000000003e-05 2024-09-16T00:10:34.537 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.537 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.537 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.537 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700708+0000", 2024-09-16T00:10:34.537 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2097000000000001e-05 2024-09-16T00:10:34.537 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.537 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.537 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.538 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.538 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.538 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3180 2.d 2:b02ffd62:::200.00000010:head [write 3464184~1398 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.538 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:34.538 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.769426742, 2024-09-16T00:10:34.538 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028112109, 2024-09-16T00:10:34.538 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.538 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.538 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.538 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.538 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.538 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.538 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:34.538 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.539 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.539 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.539 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.539 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:34.539 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.539 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.539 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.539 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.539 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:34.539 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:10:34.539 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.539 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.539 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.540 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674061+0000", 2024-09-16T00:10:34.540 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4440000000000001e-06 2024-09-16T00:10:34.540 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.540 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.540 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.540 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674069+0000", 2024-09-16T00:10:34.540 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7639999999999994e-06 2024-09-16T00:10:34.540 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.540 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.540 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.541 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674076+0000", 2024-09-16T00:10:34.541 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5590000000000002e-06 2024-09-16T00:10:34.541 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.541 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.541 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.541 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698639+0000", 2024-09-16T00:10:34.541 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.024562471999999998 2024-09-16T00:10:34.541 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.541 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.541 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.541 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698693+0000", 2024-09-16T00:10:34.541 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4432999999999999e-05 2024-09-16T00:10:34.541 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698807+0000", 2024-09-16T00:10:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113607 2024-09-16T00:10:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700727+0000", 2024-09-16T00:10:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019201750000000001 2024-09-16T00:10:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702076+0000", 2024-09-16T00:10:34.542 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001349423 2024-09-16T00:10:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702128+0000", 2024-09-16T00:10:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2157000000000002e-05 2024-09-16T00:10:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702168+0000", 2024-09-16T00:10:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9292000000000003e-05 2024-09-16T00:10:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.543 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3181 2.d 2:b02ffd62:::200.00000010:head [write 3465582~2754 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.675904+0000", 2024-09-16T00:10:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.767578430999997, 2024-09-16T00:10:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026402452, 2024-09-16T00:10:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675904+0000", 2024-09-16T00:10:34.544 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675904+0000", 2024-09-16T00:10:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675905+0000", 2024-09-16T00:10:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.043e-06 2024-09-16T00:10:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.545 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675911+0000", 2024-09-16T00:10:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1970000000000001e-06 2024-09-16T00:10:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675924+0000", 2024-09-16T00:10:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.252e-05 2024-09-16T00:10:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675931+0000", 2024-09-16T00:10:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5800000000000003e-06 2024-09-16T00:10:34.546 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699280+0000", 2024-09-16T00:10:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023348634 2024-09-16T00:10:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699329+0000", 2024-09-16T00:10:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9175999999999998e-05 2024-09-16T00:10:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699400+0000", 2024-09-16T00:10:34.547 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0649999999999996e-05 2024-09-16T00:10:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700734+0000", 2024-09-16T00:10:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001334816 2024-09-16T00:10:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702219+0000", 2024-09-16T00:10:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014847930000000001 2024-09-16T00:10:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.548 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702267+0000", 2024-09-16T00:10:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8133000000000001e-05 2024-09-16T00:10:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702306+0000", 2024-09-16T00:10:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8909999999999998e-05 2024-09-16T00:10:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3184 2.d 2:b02ffd62:::200.00000010:head [write 3478077~11949 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.549 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.681430+0000", 2024-09-16T00:10:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.762052189000002, 2024-09-16T00:10:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.02662339, 2024-09-16T00:10:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681430+0000", 2024-09-16T00:10:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.550 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681430+0000", 2024-09-16T00:10:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681431+0000", 2024-09-16T00:10:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9999999999999995e-07 2024-09-16T00:10:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681443+0000", 2024-09-16T00:10:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1568e-05 2024-09-16T00:10:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.551 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681453+0000", 2024-09-16T00:10:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8819999999999996e-06 2024-09-16T00:10:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681458+0000", 2024-09-16T00:10:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3709999999999999e-06 2024-09-16T00:10:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.701384+0000", 2024-09-16T00:10:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.019926342999999999 2024-09-16T00:10:34.552 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.701458+0000", 2024-09-16T00:10:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3854000000000001e-05 2024-09-16T00:10:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.701570+0000", 2024-09-16T00:10:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000111437 2024-09-16T00:10:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.553 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702913+0000", 2024-09-16T00:10:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013437259999999999 2024-09-16T00:10:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.707984+0000", 2024-09-16T00:10:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050705669999999998 2024-09-16T00:10:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708016+0000", 2024-09-16T00:10:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2527999999999998e-05 2024-09-16T00:10:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.554 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708053+0000", 2024-09-16T00:10:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7113999999999997e-05 2024-09-16T00:10:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3185 2.d 2:b02ffd62:::200.00000010:head [write 3490026~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.681685+0000", 2024-09-16T00:10:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.761796918000002, 2024-09-16T00:10:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026552664, 2024-09-16T00:10:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:34.555 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681685+0000", 2024-09-16T00:10:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681685+0000", 2024-09-16T00:10:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.556 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681686+0000", 2024-09-16T00:10:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.06e-07 2024-09-16T00:10:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681692+0000", 2024-09-16T00:10:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6840000000000001e-06 2024-09-16T00:10:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681697+0000", 2024-09-16T00:10:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0340000000000004e-06 2024-09-16T00:10:34.557 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681700+0000", 2024-09-16T00:10:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3589999999999999e-06 2024-09-16T00:10:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702500+0000", 2024-09-16T00:10:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.020799746000000001 2024-09-16T00:10:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702550+0000", 2024-09-16T00:10:34.558 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9895999999999998e-05 2024-09-16T00:10:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702649+0000", 2024-09-16T00:10:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9500999999999995e-05 2024-09-16T00:10:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.705343+0000", 2024-09-16T00:10:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026938130000000002 2024-09-16T00:10:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.559 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708128+0000", 2024-09-16T00:10:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027852979999999999 2024-09-16T00:10:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708202+0000", 2024-09-16T00:10:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4046000000000002e-05 2024-09-16T00:10:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708238+0000", 2024-09-16T00:10:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5781000000000002e-05 2024-09-16T00:10:34.560 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:34.561 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:34.561 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:34.562 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:10:34.563 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:34.561+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:34.663 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:34.662+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:34.708 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:34.709 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:34.709 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:34.709 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:34.718 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:10:34.803 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:34.802+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:34.865 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:34.864+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:34.871 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:34.872 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:34.872 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:34.872 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:34.872 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:34.881 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:10:34.966 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:34.965+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:35.034 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:35.034 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5850 3.3a 3:5c3cfc2b:::10000001adf.00000000:head [write 0~116] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.614975208000004, 2024-09-16T00:10:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66931295800000001, 2024-09-16T00:10:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.035 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.036 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5850 2024-09-16T00:10:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.037 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:35.038 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415938+0000", 2024-09-16T00:10:35.039 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7099999999999999e-06 2024-09-16T00:10:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415945+0000", 2024-09-16T00:10:35.040 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:10:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415959+0000", 2024-09-16T00:10:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3545e-05 2024-09-16T00:10:35.041 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415966+0000", 2024-09-16T00:10:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6379999999999998e-06 2024-09-16T00:10:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.042 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416013+0000", 2024-09-16T00:10:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7314999999999997e-05 2024-09-16T00:10:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.043 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416020+0000", 2024-09-16T00:10:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.52e-06 2024-09-16T00:10:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.044 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082393+0000", 2024-09-16T00:10:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66637251799999997 2024-09-16T00:10:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.045 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082537+0000", 2024-09-16T00:10:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014418800000000001 2024-09-16T00:10:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:35.046 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082689+0000", 2024-09-16T00:10:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015198499999999999 2024-09-16T00:10:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084930+0000", 2024-09-16T00:10:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002241041 2024-09-16T00:10:35.047 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084994+0000", 2024-09-16T00:10:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3274000000000004e-05 2024-09-16T00:10:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085049+0000", 2024-09-16T00:10:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.554e-05 2024-09-16T00:10:35.048 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085250+0000", 2024-09-16T00:10:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020060799999999999 2024-09-16T00:10:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5910 3.3a 3:5ec98a62:::10000001b22.00000000:head [write 0~1274] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:35.049 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.468031569999994, 2024-09-16T00:10:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.52286026600000002, 2024-09-16T00:10:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5910 2024-09-16T00:10:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.050 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562881+0000", 2024-09-16T00:10:35.051 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:10:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562887+0000", 2024-09-16T00:10:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2870000000000004e-06 2024-09-16T00:10:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562897+0000", 2024-09-16T00:10:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.064e-05 2024-09-16T00:10:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.052 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562902+0000", 2024-09-16T00:10:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2340000000000002e-06 2024-09-16T00:10:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562958+0000", 2024-09-16T00:10:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5523999999999999e-05 2024-09-16T00:10:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:35.053 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562968+0000", 2024-09-16T00:10:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0394000000000001e-05 2024-09-16T00:10:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083461+0000", 2024-09-16T00:10:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.520492968 2024-09-16T00:10:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083645+0000", 2024-09-16T00:10:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018346399999999999 2024-09-16T00:10:35.054 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083844+0000", 2024-09-16T00:10:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000199268 2024-09-16T00:10:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085410+0000", 2024-09-16T00:10:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015662359999999999 2024-09-16T00:10:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.055 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085493+0000", 2024-09-16T00:10:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2478999999999994e-05 2024-09-16T00:10:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.056 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085563+0000", 2024-09-16T00:10:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0159000000000004e-05 2024-09-16T00:10:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085741+0000", 2024-09-16T00:10:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017764300000000001 2024-09-16T00:10:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.057 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5962 3.3a 3:5c7b4d29:::10000001b5d.00000000:head [write 0~3074] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.338494753000006, 2024-09-16T00:10:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39368737500000001, 2024-09-16T00:10:35.058 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5962 2024-09-16T00:10:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.059 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.060 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692418+0000", 2024-09-16T00:10:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3510000000000001e-06 2024-09-16T00:10:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.061 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692425+0000", 2024-09-16T00:10:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3770000000000001e-06 2024-09-16T00:10:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692440+0000", 2024-09-16T00:10:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4975e-05 2024-09-16T00:10:35.062 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692448+0000", 2024-09-16T00:10:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6279999999999994e-06 2024-09-16T00:10:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.063 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692499+0000", 2024-09-16T00:10:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0872999999999999e-05 2024-09-16T00:10:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692507+0000", 2024-09-16T00:10:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9889999999999992e-06 2024-09-16T00:10:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.064 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084221+0000", 2024-09-16T00:10:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39171379699999997 2024-09-16T00:10:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084446+0000", 2024-09-16T00:10:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022519900000000001 2024-09-16T00:10:35.065 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084672+0000", 2024-09-16T00:10:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000225405 2024-09-16T00:10:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085832+0000", 2024-09-16T00:10:35.066 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011600040000000001 2024-09-16T00:10:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085864+0000", 2024-09-16T00:10:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2592999999999998e-05 2024-09-16T00:10:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.067 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085943+0000", 2024-09-16T00:10:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8534000000000004e-05 2024-09-16T00:10:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086105+0000", 2024-09-16T00:10:35.068 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016165 2024-09-16T00:10:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5963 3.3a 3:5e4192f5:::10000001b5e.00000000:head [write 0~152] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.336430321999998, 2024-09-16T00:10:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39413463900000001, 2024-09-16T00:10:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.069 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5963 2024-09-16T00:10:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:35.070 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:10:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694486+0000", 2024-09-16T00:10:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0960000000000003e-06 2024-09-16T00:10:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.071 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694496+0000", 2024-09-16T00:10:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2189999999999993e-06 2024-09-16T00:10:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694500+0000", 2024-09-16T00:10:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9529999999999998e-06 2024-09-16T00:10:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694552+0000", 2024-09-16T00:10:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1360999999999998e-05 2024-09-16T00:10:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.072 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694561+0000", 2024-09-16T00:10:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1700000000000003e-06 2024-09-16T00:10:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086346+0000", 2024-09-16T00:10:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39178468 2024-09-16T00:10:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.073 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086480+0000", 2024-09-16T00:10:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013469299999999999 2024-09-16T00:10:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086790+0000", 2024-09-16T00:10:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030923300000000001 2024-09-16T00:10:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088361+0000", 2024-09-16T00:10:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015717050000000001 2024-09-16T00:10:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.074 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088432+0000", 2024-09-16T00:10:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0922999999999998e-05 2024-09-16T00:10:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088489+0000", 2024-09-16T00:10:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6653999999999997e-05 2024-09-16T00:10:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088616+0000", 2024-09-16T00:10:35.075 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012730099999999999 2024-09-16T00:10:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:6082 3.3a 3:5c50f65b:::10000001be7.00000000:head [write 0~2439] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 79.042657465999994, 2024-09-16T00:10:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.10065850799999999, 2024-09-16T00:10:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.076 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 6082 2024-09-16T00:10:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.077 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988256+0000", 2024-09-16T00:10:35.078 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.065e-06 2024-09-16T00:10:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988262+0000", 2024-09-16T00:10:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4640000000000003e-06 2024-09-16T00:10:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.079 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988272+0000", 2024-09-16T00:10:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0335e-05 2024-09-16T00:10:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988279+0000", 2024-09-16T00:10:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:10:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.080 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988324+0000", 2024-09-16T00:10:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4953000000000001e-05 2024-09-16T00:10:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988331+0000", 2024-09-16T00:10:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5989999999999998e-06 2024-09-16T00:10:35.081 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087272+0000", 2024-09-16T00:10:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.098941045000000005 2024-09-16T00:10:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.082 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087486+0000", 2024-09-16T00:10:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00021360199999999999 2024-09-16T00:10:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087740+0000", 2024-09-16T00:10:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00025456799999999999 2024-09-16T00:10:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.083 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088638+0000", 2024-09-16T00:10:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00089793900000000005 2024-09-16T00:10:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088673+0000", 2024-09-16T00:10:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5111999999999999e-05 2024-09-16T00:10:35.084 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088722+0000", 2024-09-16T00:10:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8758000000000003e-05 2024-09-16T00:10:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.085 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088913+0000", 2024-09-16T00:10:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000190953 2024-09-16T00:10:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.086 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10643 3.21 3:8453163e:::100000030b3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 59.431181273, 2024-09-16T00:10:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013028809000000001, 2024-09-16T00:10:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.087 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10643 2024-09-16T00:10:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.088 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:35.089 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.089 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.089 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.089 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599734+0000", 2024-09-16T00:10:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-06 2024-09-16T00:10:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.090 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603345+0000", 2024-09-16T00:10:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003611628 2024-09-16T00:10:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.091 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603368+0000", 2024-09-16T00:10:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3017e-05 2024-09-16T00:10:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603379+0000", 2024-09-16T00:10:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0528e-05 2024-09-16T00:10:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.092 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603448+0000", 2024-09-16T00:10:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9080999999999998e-05 2024-09-16T00:10:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.093 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603623+0000", 2024-09-16T00:10:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017525599999999999 2024-09-16T00:10:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603766+0000", 2024-09-16T00:10:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000142977 2024-09-16T00:10:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.611250+0000", 2024-09-16T00:10:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074835240000000001 2024-09-16T00:10:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.094 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612617+0000", 2024-09-16T00:10:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001367793 2024-09-16T00:10:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612700+0000", 2024-09-16T00:10:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3123999999999999e-05 2024-09-16T00:10:35.095 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612759+0000", 2024-09-16T00:10:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8950999999999999e-05 2024-09-16T00:10:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10652 3.22 3:44e46336:::100000030c1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:10:35.096 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 59.403129641, 2024-09-16T00:10:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011405732, 2024-09-16T00:10:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10652 2024-09-16T00:10:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.097 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:10:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627782+0000", 2024-09-16T00:10:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.627785+0000", 2024-09-16T00:10:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6970000000000002e-06 2024-09-16T00:10:35.098 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629610+0000", 2024-09-16T00:10:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018250359999999999 2024-09-16T00:10:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629639+0000", 2024-09-16T00:10:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8676000000000001e-05 2024-09-16T00:10:35.099 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629652+0000", 2024-09-16T00:10:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3572e-05 2024-09-16T00:10:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629735+0000", 2024-09-16T00:10:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.25e-05 2024-09-16T00:10:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.100 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.629925+0000", 2024-09-16T00:10:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019011000000000001 2024-09-16T00:10:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:10:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.630075+0000", 2024-09-16T00:10:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000149841 2024-09-16T00:10:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.636470+0000", 2024-09-16T00:10:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0063952289999999997 2024-09-16T00:10:35.101 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639098+0000", 2024-09-16T00:10:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026277980000000002 2024-09-16T00:10:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639154+0000", 2024-09-16T00:10:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6413999999999999e-05 2024-09-16T00:10:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.102 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.639188+0000", 2024-09-16T00:10:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3859000000000002e-05 2024-09-16T00:10:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11217 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 54.456632620000001, 2024-09-16T00:10:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01231253, 2024-09-16T00:10:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.103 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11217 2024-09-16T00:10:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.104 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574282+0000", 2024-09-16T00:10:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8499999999999998e-06 2024-09-16T00:10:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575804+0000", 2024-09-16T00:10:35.105 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001521859 2024-09-16T00:10:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575826+0000", 2024-09-16T00:10:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2059999999999999e-05 2024-09-16T00:10:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575837+0000", 2024-09-16T00:10:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1313e-05 2024-09-16T00:10:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.106 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575920+0000", 2024-09-16T00:10:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2918999999999997e-05 2024-09-16T00:10:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576088+0000", 2024-09-16T00:10:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016803799999999999 2024-09-16T00:10:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576233+0000", 2024-09-16T00:10:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014508199999999999 2024-09-16T00:10:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.107 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.580500+0000", 2024-09-16T00:10:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004266644 2024-09-16T00:10:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586479+0000", 2024-09-16T00:10:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059791820000000004 2024-09-16T00:10:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.108 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586541+0000", 2024-09-16T00:10:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1463000000000004e-05 2024-09-16T00:10:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586592+0000", 2024-09-16T00:10:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1119999999999998e-05 2024-09-16T00:10:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.109 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11767 3.38 3:1fa50e65:::10000003117.00000000:head [write 0~4194304 [3@1169732]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.867228085000001, 2024-09-16T00:10:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011555678999999999, 2024-09-16T00:10:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11767 2024-09-16T00:10:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.110 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.111 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163686+0000", 2024-09-16T00:10:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3259999999999998e-06 2024-09-16T00:10:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165053+0000", 2024-09-16T00:10:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013665859999999999 2024-09-16T00:10:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165075+0000", 2024-09-16T00:10:35.112 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2206999999999999e-05 2024-09-16T00:10:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165084+0000", 2024-09-16T00:10:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0710000000000001e-06 2024-09-16T00:10:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.113 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165157+0000", 2024-09-16T00:10:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2619000000000004e-05 2024-09-16T00:10:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165249+0000", 2024-09-16T00:10:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2517999999999997e-05 2024-09-16T00:10:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.114 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165360+0000", 2024-09-16T00:10:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011134899999999999 2024-09-16T00:10:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.169320+0000", 2024-09-16T00:10:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039593040000000003 2024-09-16T00:10:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175135+0000", 2024-09-16T00:10:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005814891 2024-09-16T00:10:35.115 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175199+0000", 2024-09-16T00:10:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4474999999999995e-05 2024-09-16T00:10:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175239+0000", 2024-09-16T00:10:35.116 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0333000000000002e-05 2024-09-16T00:10:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11789 3.21 3:8453163e:::100000030b3.00000000:head [write 0~3801088 [4@1194504]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:10:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.779539989, 2024-09-16T00:10:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011097602, 2024-09-16T00:10:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.117 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11789 2024-09-16T00:10:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:10:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.118 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251372+0000", 2024-09-16T00:10:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.251373+0000", 2024-09-16T00:10:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.544e-06 2024-09-16T00:10:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252710+0000", 2024-09-16T00:10:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013362929999999999 2024-09-16T00:10:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.119 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252730+0000", 2024-09-16T00:10:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0579999999999999e-05 2024-09-16T00:10:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252741+0000", 2024-09-16T00:10:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0505e-05 2024-09-16T00:10:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252820+0000", 2024-09-16T00:10:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9098999999999996e-05 2024-09-16T00:10:35.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.252931+0000", 2024-09-16T00:10:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011073600000000001 2024-09-16T00:10:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.253101+0000", 2024-09-16T00:10:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017038000000000001 2024-09-16T00:10:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.261813+0000", 2024-09-16T00:10:35.121 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087119469999999994 2024-09-16T00:10:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262338+0000", 2024-09-16T00:10:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00052493799999999997 2024-09-16T00:10:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262430+0000", 2024-09-16T00:10:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1626999999999995e-05 2024-09-16T00:10:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.122 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.262470+0000", 2024-09-16T00:10:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9953000000000002e-05 2024-09-16T00:10:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11890 3.38 3:1e014616:::100000033e9.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.459574906999997, 2024-09-16T00:10:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011833791999999999, 2024-09-16T00:10:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.123 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11890 2024-09-16T00:10:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.124 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571340+0000", 2024-09-16T00:10:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.36e-06 2024-09-16T00:10:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572866+0000", 2024-09-16T00:10:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001525549 2024-09-16T00:10:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.125 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572884+0000", 2024-09-16T00:10:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7796999999999999e-05 2024-09-16T00:10:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572897+0000", 2024-09-16T00:10:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3567000000000001e-05 2024-09-16T00:10:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572970+0000", 2024-09-16T00:10:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2751999999999997e-05 2024-09-16T00:10:35.126 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573053+0000", 2024-09-16T00:10:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3230000000000001e-05 2024-09-16T00:10:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573168+0000", 2024-09-16T00:10:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114536 2024-09-16T00:10:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.127 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.577923+0000", 2024-09-16T00:10:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047548720000000003 2024-09-16T00:10:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583081+0000", 2024-09-16T00:10:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051587630000000002 2024-09-16T00:10:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.128 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583137+0000", 2024-09-16T00:10:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5259000000000001e-05 2024-09-16T00:10:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583171+0000", 2024-09-16T00:10:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4106999999999997e-05 2024-09-16T00:10:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:12037 3.9 e87/20 3:928db521:::1000000326d.00000000:head v 87'199, mlcod=87'197)", 2024-09-16T00:10:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:35.129 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.028384955999996, 2024-09-16T00:10:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012236153, 2024-09-16T00:10:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.130 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002529+0000", 2024-09-16T00:10:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4889999999999998e-06 2024-09-16T00:10:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007064+0000", 2024-09-16T00:10:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045346220000000003 2024-09-16T00:10:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.131 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007093+0000", 2024-09-16T00:10:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9009e-05 2024-09-16T00:10:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007109+0000", 2024-09-16T00:10:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5846e-05 2024-09-16T00:10:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.132 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007196+0000", 2024-09-16T00:10:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7507000000000001e-05 2024-09-16T00:10:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007227+0000", 2024-09-16T00:10:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0916e-05 2024-09-16T00:10:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014658+0000", 2024-09-16T00:10:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074305090000000001 2024-09-16T00:10:35.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014763+0000", 2024-09-16T00:10:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010525499999999999 2024-09-16T00:10:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12398 3.2e 3:77a11d08:::10000003432.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:35.134 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.789706172999999, 2024-09-16T00:10:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012092999, 2024-09-16T00:10:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12398 2024-09-16T00:10:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.135 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241209+0000", 2024-09-16T00:10:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2710000000000001e-06 2024-09-16T00:10:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.136 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243840+0000", 2024-09-16T00:10:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026307919999999999 2024-09-16T00:10:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243872+0000", 2024-09-16T00:10:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2699e-05 2024-09-16T00:10:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243887+0000", 2024-09-16T00:10:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4348000000000001e-05 2024-09-16T00:10:35.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243972+0000", 2024-09-16T00:10:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5358999999999994e-05 2024-09-16T00:10:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244132+0000", 2024-09-16T00:10:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015972499999999999 2024-09-16T00:10:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253153+0000", 2024-09-16T00:10:35.138 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0090210280000000004 2024-09-16T00:10:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253213+0000", 2024-09-16T00:10:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0300999999999997e-05 2024-09-16T00:10:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253299+0000", 2024-09-16T00:10:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5476000000000006e-05 2024-09-16T00:10:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.139 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12400 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304 [2@815045]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:10:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.786668439000003, 2024-09-16T00:10:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011140066000000001, 2024-09-16T00:10:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.140 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12400 2024-09-16T00:10:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:10:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.141 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244243+0000", 2024-09-16T00:10:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244245+0000", 2024-09-16T00:10:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.248e-06 2024-09-16T00:10:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246467+0000", 2024-09-16T00:10:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022217949999999999 2024-09-16T00:10:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.142 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246496+0000", 2024-09-16T00:10:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9484000000000001e-05 2024-09-16T00:10:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246509+0000", 2024-09-16T00:10:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3390000000000001e-05 2024-09-16T00:10:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246586+0000", 2024-09-16T00:10:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6692000000000004e-05 2024-09-16T00:10:35.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246680+0000", 2024-09-16T00:10:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3468999999999996e-05 2024-09-16T00:10:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.246850+0000", 2024-09-16T00:10:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170206 2024-09-16T00:10:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.144 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253478+0000", 2024-09-16T00:10:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066287530000000002 2024-09-16T00:10:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255318+0000", 2024-09-16T00:10:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001839292 2024-09-16T00:10:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.145 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255354+0000", 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5964999999999999e-05 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.255384+0000", 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9771999999999999e-05 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13317 3.38 3:1cda8244:::100000035dc.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:35.146 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 47.112313913000001, 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011377063, 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13317 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.147 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918598+0000", 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.918602+0000", 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8160000000000004e-06 2024-09-16T00:10:35.148 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920364+0000", 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017623630000000001 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.149 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920391+0000", 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7248000000000001e-05 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.150 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920405+0000", 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3969e-05 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.151 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920486+0000", 2024-09-16T00:10:35.152 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0737999999999998e-05 2024-09-16T00:10:35.152 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.152 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.152 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.152 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920594+0000", 2024-09-16T00:10:35.152 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108269 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.920730+0000", 2024-09-16T00:10:35.153 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000135229 2024-09-16T00:10:35.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.154 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.154 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.154 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.928847+0000", 2024-09-16T00:10:35.154 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081178299999999995 2024-09-16T00:10:35.154 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929862+0000", 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0010147540000000001 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929923+0000", 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0452999999999999e-05 2024-09-16T00:10:35.155 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.156 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.156 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.156 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:47.929975+0000", 2024-09-16T00:10:35.156 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2394e-05 2024-09-16T00:10:35.156 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.156 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.156 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13942 3.22 3:44ed4521:::100000036d5.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.108427810000002, 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011341061, 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.157 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13942 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.158 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922484+0000", 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.922487+0000", 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0180000000000002e-06 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923720+0000", 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012333470000000001 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.159 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923749+0000", 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8748999999999999e-05 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923770+0000", 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1189999999999999e-05 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.160 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.923868+0000", 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7675000000000002e-05 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.924047+0000", 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017935100000000001 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 3", 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.924223+0000", 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000175661 2024-09-16T00:10:35.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.931544+0000", 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073204589999999996 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933692+0000", 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021488760000000001 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.162 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933774+0000", 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1855999999999997e-05 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.933825+0000", 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0878999999999999e-05 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14867 3.1f 3:faedf583:::1000000393b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:10:35.163 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.283938073999998, 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011128677999999999, 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14867 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.164 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746974+0000", 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.746976+0000", 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1169999999999998e-06 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.165 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748348+0000", 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013721860000000001 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748371+0000", 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2938999999999998e-05 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748381+0000", 2024-09-16T00:10:35.166 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5300000000000002e-06 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748458+0000", 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7389000000000001e-05 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748626+0000", 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016757499999999999 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:35.167 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.748813+0000", 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018743399999999999 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.753998+0000", 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051851140000000002 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.757990+0000", 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039914169999999997 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.168 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.758054+0000", 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4513999999999999e-05 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.758103+0000", 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8463e-05 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.169 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15003 3.21 3:8673bb5f:::100000030ed.00000000:head [read 2097152~2097152 [2@1395527]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.862516079000002, 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058433321000000003, 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.170 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15003 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.171 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168397+0000", 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3900000000000004e-07 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168401+0000", 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4050000000000002e-06 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168411+0000", 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6539999999999994e-06 2024-09-16T00:10:35.172 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168417+0000", 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6719999999999998e-06 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168467+0000", 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9815999999999999e-05 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.173 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168535+0000", 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7669999999999994e-05 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.226829+0000", 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058294264999999998 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15005 3.1f 3:f8460f10:::10000003370.00000000:head [read 1048576~1048576 [2@4194304]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:35.174 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 40.860835367, 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.055844088, 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15005 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.175 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170078+0000", 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.235e-06 2024-09-16T00:10:35.176 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170084+0000", 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9739999999999999e-06 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225526+0000", 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.055441777999999997 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.177 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225542+0000", 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6606999999999999e-05 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225668+0000", 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012631 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225744+0000", 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5794e-05 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.178 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225921+0000", 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017639000000000001 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13174 3.38 3:1d27f0cd:::10000000db7.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:26.136612+0000", 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 8.89430035, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013450962, 2024-09-16T00:10:35.179 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136612+0000", 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.180 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136612+0000", 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136612+0000", 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9400000000000004e-07 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136616+0000", 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0559999999999998e-06 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.181 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136628+0000", 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1494999999999999e-05 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136637+0000", 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3300000000000005e-06 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136700+0000", 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2889e-05 2024-09-16T00:10:35.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136830+0000", 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012989300000000001 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136927+0000", 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7115999999999996e-05 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.183 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.138274+0000", 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013470800000000001 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.149951+0000", 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011676485 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.150002+0000", 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1573999999999998e-05 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.184 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.150063+0000", 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0250000000000001e-05 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.185 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:35.187 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:10:35.187 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:35.185+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:35.287 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:35.287+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:35.335 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:35.335 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:35.335 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:35.335 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:35.344 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:10:35.388 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:35.387+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:35.474 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T00:10:35.489 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:35.488+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:35.496 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:35.497 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:35.497 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:35.497 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:35.497 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:35.507 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:10:35.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.577+0000 7fd445227640 1 Processor -- start 2024-09-16T00:10:35.579 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.578+0000 7fd445227640 1 -- start start 2024-09-16T00:10:35.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.578+0000 7fd445227640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd440154bd0 0x7fd44015b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:35.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.578+0000 7fd445227640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd44006ea20 0x7fd4400c3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:35.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.578+0000 7fd445227640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd44006cd40 con 0x7fd4400c8b50 2024-09-16T00:10:35.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.578+0000 7fd445227640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fd44015a9d0 con 0x7fd44006ea20 2024-09-16T00:10:35.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.578+0000 7fd445227640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd44006cfc0 con 0x7fd440154bd0 2024-09-16T00:10:35.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.579+0000 7fd43ed9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd440154bd0 0x7fd44015b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:35.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.579+0000 7fd43e59c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd44006ea20 0x7fd4400c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:35.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.579+0000 7fd43f59e640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd4400c8b50 0x7fd4400ce0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:53704/0 (socket says 172.21.15.139:53704) 2024-09-16T00:10:35.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.579+0000 7fd43ed9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd440154bd0 0x7fd44015b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:45524/0 (socket says 172.21.15.139:45524) 2024-09-16T00:10:35.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.579+0000 7fd43e59c640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd44006ea20 0x7fd4400c3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:45274/0 (socket says 172.21.15.139:45274) 2024-09-16T00:10:35.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.579+0000 7fd43f59e640 1 -- 172.21.15.139:0/2755554469 learned_addr learned my addr 172.21.15.139:0/2755554469 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:35.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.579+0000 7fd43ed9d640 1 -- 172.21.15.139:0/2755554469 >> v2:172.21.15.164:3300/0 conn(0x7fd44006ea20 msgr2=0x7fd4400c3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:35.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.579+0000 7fd43ed9d640 1 --2- 172.21.15.139:0/2755554469 >> v2:172.21.15.164:3300/0 conn(0x7fd44006ea20 0x7fd4400c3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:35.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.579+0000 7fd43ed9d640 1 -- 172.21.15.139:0/2755554469 >> v1:172.21.15.164:6789/0 conn(0x7fd4400c8b50 legacy=0x7fd4400ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:35.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.579+0000 7fd43dd9b640 1 -- 172.21.15.139:0/2755554469 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1158537402 0 0) 0x7fd438001490 con 0x7fd4400c8b50 2024-09-16T00:10:35.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.579+0000 7fd43ed9d640 1 -- 172.21.15.139:0/2755554469 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44015ac70 con 0x7fd440154bd0 2024-09-16T00:10:35.581 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.579+0000 7fd43e59c640 1 --2- 172.21.15.139:0/2755554469 >> v2:172.21.15.164:3300/0 conn(0x7fd44006ea20 0x7fd4400c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:35.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.580+0000 7fd43ed9d640 1 --2- 172.21.15.139:0/2755554469 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd440154bd0 0x7fd44015b470 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd4300097a0 tx=0x7fd43002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36323aca5e9b58ed server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:35.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.580+0000 7fd43dd9b640 1 -- 172.21.15.139:0/2755554469 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430037070 con 0x7fd440154bd0 2024-09-16T00:10:35.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.580+0000 7fd43dd9b640 1 -- 172.21.15.139:0/2755554469 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd430032440 con 0x7fd440154bd0 2024-09-16T00:10:35.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.580+0000 7fd43dd9b640 1 -- 172.21.15.139:0/2755554469 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430032810 con 0x7fd440154bd0 2024-09-16T00:10:35.583 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.580+0000 7fd445227640 1 -- 172.21.15.139:0/2755554469 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd440154bd0 msgr2=0x7fd44015b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:35.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.580+0000 7fd445227640 1 --2- 172.21.15.139:0/2755554469 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd440154bd0 0x7fd44015b470 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd4300097a0 tx=0x7fd43002a180 comp rx=0 tx=0).stop 2024-09-16T00:10:35.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.580+0000 7fd445227640 1 -- 172.21.15.139:0/2755554469 shutdown_connections 2024-09-16T00:10:35.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.580+0000 7fd445227640 1 --2- 172.21.15.139:0/2755554469 >> v2:172.21.15.164:3300/0 conn(0x7fd44006ea20 0x7fd4400c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:35.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.580+0000 7fd445227640 1 --2- 172.21.15.139:0/2755554469 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd440154bd0 0x7fd44015b470 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:35.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.580+0000 7fd445227640 1 -- 172.21.15.139:0/2755554469 >> 172.21.15.139:0/2755554469 conn(0x7fd44006c150 msgr2=0x7fd440067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:35.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.581+0000 7fd445227640 1 -- 172.21.15.139:0/2755554469 shutdown_connections 2024-09-16T00:10:35.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.581+0000 7fd445227640 1 -- 172.21.15.139:0/2755554469 wait complete. 2024-09-16T00:10:35.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.582+0000 7fd445227640 1 Processor -- start 2024-09-16T00:10:35.584 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.582+0000 7fd445227640 1 -- start start 2024-09-16T00:10:35.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd445227640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd44006ea20 0x7fd44014a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:35.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd445227640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4400c8b50 0x7fd4401f0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:35.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd445227640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd440154bd0 0x7fd4401f0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:35.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd445227640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd44006d2a0 con 0x7fd440154bd0 2024-09-16T00:10:35.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd445227640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fd440154fb0 con 0x7fd4400c8b50 2024-09-16T00:10:35.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd445227640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fd44006d420 con 0x7fd44006ea20 2024-09-16T00:10:35.587 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd43ed9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd44006ea20 0x7fd44014a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:35.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd43e59c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4400c8b50 0x7fd4401f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:35.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd43f59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd440154bd0 0x7fd4401f0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:35.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd43ed9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd44006ea20 0x7fd44014a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:45540/0 (socket says 172.21.15.139:45540) 2024-09-16T00:10:35.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd43ed9d640 1 -- 172.21.15.139:0/135399181 learned_addr learned my addr 172.21.15.139:0/135399181 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:35.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd43e59c640 1 -- 172.21.15.139:0/135399181 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd44006ea20 msgr2=0x7fd44014a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:35.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd43e59c640 1 --2- 172.21.15.139:0/135399181 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd44006ea20 0x7fd44014a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:35.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd43e59c640 1 -- 172.21.15.139:0/135399181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd440154bd0 msgr2=0x7fd4401f0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:35.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd43e59c640 1 --2- 172.21.15.139:0/135399181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd440154bd0 0x7fd4401f0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:35.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd43e59c640 1 -- 172.21.15.139:0/135399181 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd440150d40 con 0x7fd4400c8b50 2024-09-16T00:10:35.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd43f59e640 1 --2- 172.21.15.139:0/135399181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd440154bd0 0x7fd4401f0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd43ed9d640 1 --2- 172.21.15.139:0/135399181 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd44006ea20 0x7fd44014a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd43e59c640 1 --2- 172.21.15.139:0/135399181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4400c8b50 0x7fd4401f0110 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd43400e8d0 tx=0x7fd43400ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.583+0000 7fd4277fe640 1 -- 172.21.15.139:0/135399181 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434017070 con 0x7fd4400c8b50 2024-09-16T00:10:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.584+0000 7fd4277fe640 1 -- 172.21.15.139:0/135399181 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd43400c940 con 0x7fd4400c8b50 2024-09-16T00:10:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.584+0000 7fd445227640 1 -- 172.21.15.139:0/135399181 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd440150fe0 con 0x7fd4400c8b50 2024-09-16T00:10:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.584+0000 7fd4277fe640 1 -- 172.21.15.139:0/135399181 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43400cd30 con 0x7fd4400c8b50 2024-09-16T00:10:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.584+0000 7fd445227640 1 -- 172.21.15.139:0/135399181 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4401f3b00 con 0x7fd4400c8b50 2024-09-16T00:10:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.585+0000 7fd445227640 1 -- 172.21.15.139:0/135399181 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4400c3170 con 0x7fd4400c8b50 2024-09-16T00:10:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.585+0000 7fd4277fe640 1 -- 172.21.15.139:0/135399181 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd43401a070 con 0x7fd4400c8b50 2024-09-16T00:10:35.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.585+0000 7fd4277fe640 1 --2- 172.21.15.139:0/135399181 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd42c06c2b0 0x7fd42c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:35.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.586+0000 7fd4277fe640 1 -- 172.21.15.139:0/135399181 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5973+0+0 (secure 0 0 0) 0x7fd43408b260 con 0x7fd4400c8b50 2024-09-16T00:10:35.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.586+0000 7fd43ed9d640 1 --2- 172.21.15.139:0/135399181 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd42c06c2b0 0x7fd42c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:35.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.586+0000 7fd43ed9d640 1 --2- 172.21.15.139:0/135399181 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd42c06c2b0 0x7fd42c06e770 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd43002a1c0 tx=0x7fd43002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:35.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.590+0000 7fd4277fe640 1 -- 172.21.15.139:0/135399181 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd434059300 con 0x7fd4400c8b50 2024-09-16T00:10:35.591 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:35.590+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:35.662 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:35.662 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:35.662 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:35.662 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:90 2.1b 2:ddb4e4d7:::200.00000002:head [write 2593031~1404 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:10:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 92.010131076999997, 2024-09-16T00:10:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011882886, 2024-09-16T00:10:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:35.663 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649119+0000", 2024-09-16T00:10:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:10:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.664 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649125+0000", 2024-09-16T00:10:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1650000000000003e-06 2024-09-16T00:10:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649134+0000", 2024-09-16T00:10:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5669999999999995e-06 2024-09-16T00:10:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649141+0000", 2024-09-16T00:10:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9040000000000003e-06 2024-09-16T00:10:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.665 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655796+0000", 2024-09-16T00:10:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066553649999999999 2024-09-16T00:10:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655868+0000", 2024-09-16T00:10:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2219999999999996e-05 2024-09-16T00:10:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655986+0000", 2024-09-16T00:10:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117457 2024-09-16T00:10:35.666 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657794+0000", 2024-09-16T00:10:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001807894 2024-09-16T00:10:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660890+0000", 2024-09-16T00:10:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030958449999999998 2024-09-16T00:10:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660966+0000", 2024-09-16T00:10:35.667 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6382000000000002e-05 2024-09-16T00:10:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.661001+0000", 2024-09-16T00:10:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5349000000000003e-05 2024-09-16T00:10:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10851 3.29 3:9786b8f5:::10000003100.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 56.766467032999998, 2024-09-16T00:10:35.668 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012109089999999999, 2024-09-16T00:10:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10851 2024-09-16T00:10:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.669 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892786+0000", 2024-09-16T00:10:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4819999999999999e-06 2024-09-16T00:10:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894302+0000", 2024-09-16T00:10:35.670 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015156099999999999 2024-09-16T00:10:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894319+0000", 2024-09-16T00:10:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7098000000000001e-05 2024-09-16T00:10:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894329+0000", 2024-09-16T00:10:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9180000000000006e-06 2024-09-16T00:10:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.671 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894394+0000", 2024-09-16T00:10:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5797999999999996e-05 2024-09-16T00:10:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894532+0000", 2024-09-16T00:10:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000138128 2024-09-16T00:10:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894633+0000", 2024-09-16T00:10:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010037100000000001 2024-09-16T00:10:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.672 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.899212+0000", 2024-09-16T00:10:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045789369999999999 2024-09-16T00:10:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904771+0000", 2024-09-16T00:10:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055591169999999997 2024-09-16T00:10:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.673 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904838+0000", 2024-09-16T00:10:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7124000000000003e-05 2024-09-16T00:10:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904892+0000", 2024-09-16T00:10:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3507000000000001e-05 2024-09-16T00:10:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.674 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10957 3.24 3:2748b9d7:::1000000313b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 56.14456268, 2024-09-16T00:10:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012163525, 2024-09-16T00:10:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.675 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10957 2024-09-16T00:10:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:35.676 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514690+0000", 2024-09-16T00:10:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.382e-06 2024-09-16T00:10:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516204+0000", 2024-09-16T00:10:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001513831 2024-09-16T00:10:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.677 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516232+0000", 2024-09-16T00:10:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7657999999999998e-05 2024-09-16T00:10:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516246+0000", 2024-09-16T00:10:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3953e-05 2024-09-16T00:10:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516374+0000", 2024-09-16T00:10:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012882 2024-09-16T00:10:35.678 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516588+0000", 2024-09-16T00:10:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000213439 2024-09-16T00:10:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516748+0000", 2024-09-16T00:10:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015971900000000001 2024-09-16T00:10:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.524102+0000", 2024-09-16T00:10:35.679 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073542499999999997 2024-09-16T00:10:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526714+0000", 2024-09-16T00:10:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026122910000000001 2024-09-16T00:10:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526791+0000", 2024-09-16T00:10:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6997999999999997e-05 2024-09-16T00:10:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.680 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526850+0000", 2024-09-16T00:10:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9184000000000001e-05 2024-09-16T00:10:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11098 3.15 e86/20 3:aa8f498f:::100000030ba.00000000:head v 86'170, mlcod=86'168)", 2024-09-16T00:10:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.637779514000002, 2024-09-16T00:10:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013832641999999999, 2024-09-16T00:10:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.681 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.682 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021472+0000", 2024-09-16T00:10:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9489999999999999e-06 2024-09-16T00:10:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027628+0000", 2024-09-16T00:10:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061563859999999998 2024-09-16T00:10:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027645+0000", 2024-09-16T00:10:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7108e-05 2024-09-16T00:10:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.683 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027655+0000", 2024-09-16T00:10:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7999999999999993e-06 2024-09-16T00:10:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027712+0000", 2024-09-16T00:10:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6826000000000001e-05 2024-09-16T00:10:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027738+0000", 2024-09-16T00:10:35.684 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5601000000000001e-05 2024-09-16T00:10:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.035237+0000", 2024-09-16T00:10:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074998699999999996 2024-09-16T00:10:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.035303+0000", 2024-09-16T00:10:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5102000000000001e-05 2024-09-16T00:10:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.685 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11099 3.b 3:d2f5b389:::100000030e7.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.633734699000001, 2024-09-16T00:10:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012707456000000001, 2024-09-16T00:10:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.686 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11099 2024-09-16T00:10:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.687 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025517+0000", 2024-09-16T00:10:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:10:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026955+0000", 2024-09-16T00:10:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001437219 2024-09-16T00:10:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.688 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026982+0000", 2024-09-16T00:10:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7523999999999999e-05 2024-09-16T00:10:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026996+0000", 2024-09-16T00:10:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4379000000000001e-05 2024-09-16T00:10:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027076+0000", 2024-09-16T00:10:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9933999999999997e-05 2024-09-16T00:10:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.689 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027185+0000", 2024-09-16T00:10:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010875899999999999 2024-09-16T00:10:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027380+0000", 2024-09-16T00:10:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019477700000000001 2024-09-16T00:10:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038072+0000", 2024-09-16T00:10:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010692234 2024-09-16T00:10:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.690 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038115+0000", 2024-09-16T00:10:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3291999999999998e-05 2024-09-16T00:10:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038172+0000", 2024-09-16T00:10:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6981000000000002e-05 2024-09-16T00:10:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038222+0000", 2024-09-16T00:10:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9737000000000002e-05 2024-09-16T00:10:35.691 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11109 3.27 3:e5179c63:::10000003176.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.588270416, 2024-09-16T00:10:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012250990999999999, 2024-09-16T00:10:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.692 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11109 2024-09-16T00:10:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.693 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070981+0000", 2024-09-16T00:10:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.289e-06 2024-09-16T00:10:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072616+0000", 2024-09-16T00:10:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016347569999999999 2024-09-16T00:10:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072638+0000", 2024-09-16T00:10:35.694 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1512999999999999e-05 2024-09-16T00:10:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072649+0000", 2024-09-16T00:10:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1626e-05 2024-09-16T00:10:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072775+0000", 2024-09-16T00:10:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012553900000000001 2024-09-16T00:10:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.695 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072856+0000", 2024-09-16T00:10:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0882000000000003e-05 2024-09-16T00:10:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072990+0000", 2024-09-16T00:10:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013483599999999999 2024-09-16T00:10:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.078354+0000", 2024-09-16T00:10:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053639500000000001 2024-09-16T00:10:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.696 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083150+0000", 2024-09-16T00:10:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004795517 2024-09-16T00:10:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083186+0000", 2024-09-16T00:10:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6019000000000002e-05 2024-09-16T00:10:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083230+0000", 2024-09-16T00:10:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4063000000000001e-05 2024-09-16T00:10:35.697 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11234 3.b 3:d2f5b389:::100000030e7.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.035654647999998, 2024-09-16T00:10:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.015217661, 2024-09-16T00:10:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.698 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11234 2024-09-16T00:10:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.699 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623598+0000", 2024-09-16T00:10:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1580000000000001e-06 2024-09-16T00:10:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626844+0000", 2024-09-16T00:10:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032459670000000002 2024-09-16T00:10:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626868+0000", 2024-09-16T00:10:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3584e-05 2024-09-16T00:10:35.700 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626879+0000", 2024-09-16T00:10:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1545999999999999e-05 2024-09-16T00:10:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626950+0000", 2024-09-16T00:10:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0784999999999994e-05 2024-09-16T00:10:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.701 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.627051+0000", 2024-09-16T00:10:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010155899999999999 2024-09-16T00:10:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.627178+0000", 2024-09-16T00:10:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012614899999999999 2024-09-16T00:10:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.635220+0000", 2024-09-16T00:10:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0080421490000000002 2024-09-16T00:10:35.702 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638699+0000", 2024-09-16T00:10:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034797500000000002 2024-09-16T00:10:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638760+0000", 2024-09-16T00:10:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0319000000000003e-05 2024-09-16T00:10:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638812+0000", 2024-09-16T00:10:35.703 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:10:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11777 3.b 3:d342b4ff:::10000003113.00000000:head [write 0~4194304 [2@1171070]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 52.46801524, 2024-09-16T00:10:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013079176, 2024-09-16T00:10:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.704 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11777 2024-09-16T00:10:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:35.705 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191237+0000", 2024-09-16T00:10:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3369999999999998e-06 2024-09-16T00:10:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192825+0000", 2024-09-16T00:10:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015887189999999999 2024-09-16T00:10:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.706 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192846+0000", 2024-09-16T00:10:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0670999999999999e-05 2024-09-16T00:10:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192858+0000", 2024-09-16T00:10:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.223e-05 2024-09-16T00:10:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192957+0000", 2024-09-16T00:10:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8416000000000001e-05 2024-09-16T00:10:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.707 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.193031+0000", 2024-09-16T00:10:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4381999999999994e-05 2024-09-16T00:10:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.193164+0000", 2024-09-16T00:10:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013349100000000001 2024-09-16T00:10:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204144+0000", 2024-09-16T00:10:35.708 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010979917 2024-09-16T00:10:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204204+0000", 2024-09-16T00:10:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0025999999999998e-05 2024-09-16T00:10:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204265+0000", 2024-09-16T00:10:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0645e-05 2024-09-16T00:10:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.709 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204313+0000", 2024-09-16T00:10:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8341999999999999e-05 2024-09-16T00:10:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11981 3.c e87/20 3:33faa5a0:::10000003215.00000000:head v 87'186, mlcod=87'184)", 2024-09-16T00:10:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.827780417, 2024-09-16T00:10:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011828697000000001, 2024-09-16T00:10:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.710 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.711 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831471+0000", 2024-09-16T00:10:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6220000000000001e-06 2024-09-16T00:10:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836584+0000", 2024-09-16T00:10:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051133510000000004 2024-09-16T00:10:35.712 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836606+0000", 2024-09-16T00:10:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2079e-05 2024-09-16T00:10:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836624+0000", 2024-09-16T00:10:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8162e-05 2024-09-16T00:10:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.713 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836704+0000", 2024-09-16T00:10:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9867e-05 2024-09-16T00:10:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836730+0000", 2024-09-16T00:10:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6285999999999999e-05 2024-09-16T00:10:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.843247+0000", 2024-09-16T00:10:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0065165650000000002 2024-09-16T00:10:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.714 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.715 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.843298+0000", 2024-09-16T00:10:35.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0764999999999999e-05 2024-09-16T00:10:35.715 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.715 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.715 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.715 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.715 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.715 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13365 3.2 3:4298088d:::100000035be.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:35.715 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:35.715 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 47.640078438000003, 2024-09-16T00:10:35.715 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014036442999999999, 2024-09-16T00:10:35.715 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.715 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.716 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.716 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.716 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.716 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.716 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13365 2024-09-16T00:10:35.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.716 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.716 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:35.716 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.716 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.716 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.716 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:35.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019174+0000", 2024-09-16T00:10:35.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1250000000000001e-06 2024-09-16T00:10:35.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.717 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.717 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.717 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020613+0000", 2024-09-16T00:10:35.717 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014391930000000001 2024-09-16T00:10:35.717 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020638+0000", 2024-09-16T00:10:35.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4657000000000001e-05 2024-09-16T00:10:35.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020651+0000", 2024-09-16T00:10:35.718 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3451e-05 2024-09-16T00:10:35.718 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.718 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.718 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.718 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020744+0000", 2024-09-16T00:10:35.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3010000000000006e-05 2024-09-16T00:10:35.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020912+0000", 2024-09-16T00:10:35.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016763800000000001 2024-09-16T00:10:35.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.719 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:35.719 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.021043+0000", 2024-09-16T00:10:35.719 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013099800000000001 2024-09-16T00:10:35.719 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.719 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026194+0000", 2024-09-16T00:10:35.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051510879999999998 2024-09-16T00:10:35.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033132+0000", 2024-09-16T00:10:35.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069377279999999998 2024-09-16T00:10:35.720 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.720 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.720 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.720 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033169+0000", 2024-09-16T00:10:35.720 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6897000000000003e-05 2024-09-16T00:10:35.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.721 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.721 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033207+0000", 2024-09-16T00:10:35.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658000000000002e-05 2024-09-16T00:10:35.721 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.721 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.721 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.721 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.721 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.721 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13368 3.27 3:e75b72a1:::100000032e8.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:35.721 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:35.721 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 47.632712554999998, 2024-09-16T00:10:35.721 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013615175, 2024-09-16T00:10:35.722 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.722 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.722 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.722 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.722 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.722 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.722 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13368 2024-09-16T00:10:35.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.722 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.722 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.722 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.722 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:35.722 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.722 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:35.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026539+0000", 2024-09-16T00:10:35.723 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.798e-06 2024-09-16T00:10:35.723 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.723 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.723 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.723 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027825+0000", 2024-09-16T00:10:35.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012861839999999999 2024-09-16T00:10:35.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027850+0000", 2024-09-16T00:10:35.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5048000000000001e-05 2024-09-16T00:10:35.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.724 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027865+0000", 2024-09-16T00:10:35.724 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4939e-05 2024-09-16T00:10:35.724 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.724 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.724 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027944+0000", 2024-09-16T00:10:35.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8881999999999995e-05 2024-09-16T00:10:35.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.028043+0000", 2024-09-16T00:10:35.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9135000000000005e-05 2024-09-16T00:10:35.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.725 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.725 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:35.725 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.028158+0000", 2024-09-16T00:10:35.725 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011527999999999999 2024-09-16T00:10:35.725 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.039292+0000", 2024-09-16T00:10:35.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011133568999999999 2024-09-16T00:10:35.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.039948+0000", 2024-09-16T00:10:35.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00065625700000000004 2024-09-16T00:10:35.726 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.726 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.726 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.726 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.040061+0000", 2024-09-16T00:10:35.726 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011284400000000001 2024-09-16T00:10:35.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.727 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.727 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.040152+0000", 2024-09-16T00:10:35.727 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1238999999999998e-05 2024-09-16T00:10:35.727 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.727 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.727 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.727 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.727 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.727 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:13366 3.15 e88/20 3:aa8f498f:::100000030ba.00000000:head v 88'196, mlcod=88'194)", 2024-09-16T00:10:35.727 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:35.727 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 47.629287480000002, 2024-09-16T00:10:35.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012822817, 2024-09-16T00:10:35.728 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.728 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.728 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.728 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:35.728 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.728 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.728 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.728 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.728 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:35.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029963+0000", 2024-09-16T00:10:35.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4270000000000001e-06 2024-09-16T00:10:35.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.729 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034078+0000", 2024-09-16T00:10:35.729 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041148360000000002 2024-09-16T00:10:35.729 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.729 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.729 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034094+0000", 2024-09-16T00:10:35.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.535e-05 2024-09-16T00:10:35.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034103+0000", 2024-09-16T00:10:35.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:10:35.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.730 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.730 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.730 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.036791+0000", 2024-09-16T00:10:35.730 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026878340000000001 2024-09-16T00:10:35.730 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.036818+0000", 2024-09-16T00:10:35.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6885000000000001e-05 2024-09-16T00:10:35.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.042713+0000", 2024-09-16T00:10:35.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058950399999999998 2024-09-16T00:10:35.731 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.731 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.731 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.731 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.042785+0000", 2024-09-16T00:10:35.731 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1927000000000004e-05 2024-09-16T00:10:35.732 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.732 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.732 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.732 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.732 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.732 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13405 3.27 3:e708eda4:::100000034c4.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:35.732 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:35.732 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 47.503085767000002, 2024-09-16T00:10:35.732 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012003333, 2024-09-16T00:10:35.732 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.732 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.732 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.733 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.733 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.733 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.733 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13405 2024-09-16T00:10:35.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.733 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:35.733 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.733 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.733 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.733 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:35.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156166+0000", 2024-09-16T00:10:35.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6230000000000001e-06 2024-09-16T00:10:35.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.734 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157868+0000", 2024-09-16T00:10:35.734 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017017 2024-09-16T00:10:35.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.734 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.734 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157891+0000", 2024-09-16T00:10:35.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2931000000000001e-05 2024-09-16T00:10:35.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157904+0000", 2024-09-16T00:10:35.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2763e-05 2024-09-16T00:10:35.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.735 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.735 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157988+0000", 2024-09-16T00:10:35.735 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3849000000000003e-05 2024-09-16T00:10:35.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.735 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.158083+0000", 2024-09-16T00:10:35.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5939000000000004e-05 2024-09-16T00:10:35.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:35.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.158194+0000", 2024-09-16T00:10:35.736 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110963 2024-09-16T00:10:35.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.736 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.736 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.736 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.164635+0000", 2024-09-16T00:10:35.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0064405570000000004 2024-09-16T00:10:35.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168088+0000", 2024-09-16T00:10:35.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034534639999999998 2024-09-16T00:10:35.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.737 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.737 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168140+0000", 2024-09-16T00:10:35.737 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1041000000000001e-05 2024-09-16T00:10:35.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.737 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.738 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.738 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168167+0000", 2024-09-16T00:10:35.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7503000000000001e-05 2024-09-16T00:10:35.738 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.738 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.738 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.738 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.738 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13685 3.b 3:d26c1580:::100000032e4.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:35.738 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:35.738 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 46.324398801999997, 2024-09-16T00:10:35.738 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014868124999999999, 2024-09-16T00:10:35.738 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.739 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.739 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.739 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.739 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.739 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.739 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13685 2024-09-16T00:10:35.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.739 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.739 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.739 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.739 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:35.739 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.739 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:35.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.740 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334854+0000", 2024-09-16T00:10:35.740 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0000000000000001e-06 2024-09-16T00:10:35.740 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.740 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.740 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337155+0000", 2024-09-16T00:10:35.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023016870000000002 2024-09-16T00:10:35.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337178+0000", 2024-09-16T00:10:35.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2898e-05 2024-09-16T00:10:35.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.741 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.741 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337190+0000", 2024-09-16T00:10:35.741 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1817999999999999e-05 2024-09-16T00:10:35.741 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.741 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337301+0000", 2024-09-16T00:10:35.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110951 2024-09-16T00:10:35.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337403+0000", 2024-09-16T00:10:35.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000102118 2024-09-16T00:10:35.742 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.742 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.742 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:35.742 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337545+0000", 2024-09-16T00:10:35.742 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014167199999999999 2024-09-16T00:10:35.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.346336+0000", 2024-09-16T00:10:35.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087911450000000002 2024-09-16T00:10:35.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.743 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349641+0000", 2024-09-16T00:10:35.743 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033047570000000002 2024-09-16T00:10:35.743 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.743 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.743 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349680+0000", 2024-09-16T00:10:35.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9054999999999998e-05 2024-09-16T00:10:35.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.744 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.744 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349719+0000", 2024-09-16T00:10:35.744 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9023999999999998e-05 2024-09-16T00:10:35.744 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.744 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.744 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.744 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.744 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.745 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14624 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:35.745 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:35.745 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 43.044639601, 2024-09-16T00:10:35.745 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012123376, 2024-09-16T00:10:35.745 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.745 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.745 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.745 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.745 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.745 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.745 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14624 2024-09-16T00:10:35.745 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.745 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:35.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:35.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.746 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614614+0000", 2024-09-16T00:10:35.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9500000000000003e-06 2024-09-16T00:10:35.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616480+0000", 2024-09-16T00:10:35.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018663169999999999 2024-09-16T00:10:35.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616506+0000", 2024-09-16T00:10:35.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6205999999999999e-05 2024-09-16T00:10:35.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616517+0000", 2024-09-16T00:10:35.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0399e-05 2024-09-16T00:10:35.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616595+0000", 2024-09-16T00:10:35.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8170999999999994e-05 2024-09-16T00:10:35.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616683+0000", 2024-09-16T00:10:35.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8578999999999993e-05 2024-09-16T00:10:35.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:35.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616825+0000", 2024-09-16T00:10:35.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014186299999999999 2024-09-16T00:10:35.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.624948+0000", 2024-09-16T00:10:35.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081230420000000005 2024-09-16T00:10:35.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626617+0000", 2024-09-16T00:10:35.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016681980000000001 2024-09-16T00:10:35.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626678+0000", 2024-09-16T00:10:35.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1362e-05 2024-09-16T00:10:35.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626733+0000", 2024-09-16T00:10:35.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5288999999999999e-05 2024-09-16T00:10:35.750 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.750 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.750 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.751 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14672 3.29 3:9641a85f:::1000000360e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:35.751 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:35.751 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 42.840182423999998, 2024-09-16T00:10:35.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012258025000000001, 2024-09-16T00:10:35.751 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.751 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.751 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.751 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.751 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.751 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.751 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14672 2024-09-16T00:10:35.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.752 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:35.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.752 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.752 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:35.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.752 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819070+0000", 2024-09-16T00:10:35.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0589999999999998e-06 2024-09-16T00:10:35.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821701+0000", 2024-09-16T00:10:35.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026309100000000002 2024-09-16T00:10:35.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821727+0000", 2024-09-16T00:10:35.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6372000000000001e-05 2024-09-16T00:10:35.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821738+0000", 2024-09-16T00:10:35.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0759e-05 2024-09-16T00:10:35.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821821+0000", 2024-09-16T00:10:35.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2854000000000003e-05 2024-09-16T00:10:35.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821920+0000", 2024-09-16T00:10:35.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8900999999999994e-05 2024-09-16T00:10:35.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:35.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.822084+0000", 2024-09-16T00:10:35.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016432799999999999 2024-09-16T00:10:35.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.829148+0000", 2024-09-16T00:10:35.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0070641460000000003 2024-09-16T00:10:35.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831242+0000", 2024-09-16T00:10:35.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002093293 2024-09-16T00:10:35.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831280+0000", 2024-09-16T00:10:35.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8707000000000001e-05 2024-09-16T00:10:35.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831325+0000", 2024-09-16T00:10:35.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4696e-05 2024-09-16T00:10:35.757 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.757 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.757 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.757 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14829 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304 [2@2556691]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:35.757 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:35.757 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 42.087663006, 2024-09-16T00:10:35.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014399683, 2024-09-16T00:10:35.757 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.757 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.757 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.757 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.758 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.758 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.758 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14829 2024-09-16T00:10:35.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.758 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:35.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.758 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.758 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:35.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571590+0000", 2024-09-16T00:10:35.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:10:35.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573133+0000", 2024-09-16T00:10:35.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015422109999999999 2024-09-16T00:10:35.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573161+0000", 2024-09-16T00:10:35.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8045e-05 2024-09-16T00:10:35.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573174+0000", 2024-09-16T00:10:35.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3193999999999999e-05 2024-09-16T00:10:35.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573365+0000", 2024-09-16T00:10:35.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019096400000000001 2024-09-16T00:10:35.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573467+0000", 2024-09-16T00:10:35.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101789 2024-09-16T00:10:35.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:35.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573610+0000", 2024-09-16T00:10:35.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143404 2024-09-16T00:10:35.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.583614+0000", 2024-09-16T00:10:35.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010004422000000001 2024-09-16T00:10:35.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585556+0000", 2024-09-16T00:10:35.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019414009999999999 2024-09-16T00:10:35.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585939+0000", 2024-09-16T00:10:35.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038340999999999999 2024-09-16T00:10:35.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.763 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.763 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585986+0000", 2024-09-16T00:10:35.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6924e-05 2024-09-16T00:10:35.763 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.763 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.763 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.763 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14914 3.b 3:d23f6103:::10000003277.00000000:head [write 0~4194304 [3@58131]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:35.763 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:35.763 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.776459234999997, 2024-09-16T00:10:35.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013007654, 2024-09-16T00:10:35.763 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.764 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.764 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.764 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.764 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.764 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.764 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14914 2024-09-16T00:10:35.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.764 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.764 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.764 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.764 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:35.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882793+0000", 2024-09-16T00:10:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.65e-06 2024-09-16T00:10:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885193+0000", 2024-09-16T00:10:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024003449999999999 2024-09-16T00:10:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885214+0000", 2024-09-16T00:10:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0398e-05 2024-09-16T00:10:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885224+0000", 2024-09-16T00:10:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0253e-05 2024-09-16T00:10:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885299+0000", 2024-09-16T00:10:35.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5434999999999999e-05 2024-09-16T00:10:35.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885389+0000", 2024-09-16T00:10:35.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9209999999999995e-05 2024-09-16T00:10:35.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:35.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885485+0000", 2024-09-16T00:10:35.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6218999999999995e-05 2024-09-16T00:10:35.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.893457+0000", 2024-09-16T00:10:35.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0079722009999999999 2024-09-16T00:10:35.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895681+0000", 2024-09-16T00:10:35.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022241449999999999 2024-09-16T00:10:35.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895732+0000", 2024-09-16T00:10:35.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1269e-05 2024-09-16T00:10:35.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895798+0000", 2024-09-16T00:10:35.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5529000000000002e-05 2024-09-16T00:10:35.769 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.769 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.769 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.769 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15058 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304 [2@2556691]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-16T00:10:35.769 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:35.769 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 41.324090640999998, 2024-09-16T00:10:35.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012275842, 2024-09-16T00:10:35.769 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.770 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.770 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:35.770 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:35.770 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:35.770 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:35.770 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15058 2024-09-16T00:10:35.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.770 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.770 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.770 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:35.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:35.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335161+0000", 2024-09-16T00:10:35.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.367e-06 2024-09-16T00:10:35.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337545+0000", 2024-09-16T00:10:35.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023839040000000001 2024-09-16T00:10:35.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337572+0000", 2024-09-16T00:10:35.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.673e-05 2024-09-16T00:10:35.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337583+0000", 2024-09-16T00:10:35.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1048e-05 2024-09-16T00:10:35.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337660+0000", 2024-09-16T00:10:35.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7459999999999994e-05 2024-09-16T00:10:35.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:35.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337770+0000", 2024-09-16T00:10:35.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109634 2024-09-16T00:10:35.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:35.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337926+0000", 2024-09-16T00:10:35.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015601999999999999 2024-09-16T00:10:35.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:35.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.345689+0000", 2024-09-16T00:10:35.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0077625630000000001 2024-09-16T00:10:35.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:35.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347324+0000", 2024-09-16T00:10:35.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016351479999999999 2024-09-16T00:10:35.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:35.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347387+0000", 2024-09-16T00:10:35.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3199000000000003e-05 2024-09-16T00:10:35.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.775 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.775 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347435+0000", 2024-09-16T00:10:35.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7769000000000003e-05 2024-09-16T00:10:35.775 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.775 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.775 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.775 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9780 2.a 2:545c15c7:::200.00000017:head [write 1367106~16252 [fadvise_dontneed]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e106)", 2024-09-16T00:10:35.775 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:35.775 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.321906516999999, 2024-09-16T00:10:35.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0208196810000001, 2024-09-16T00:10:35.776 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:35.776 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:35.776 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:10:35.776 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:35.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.776 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:35.776 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:35.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.776 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:35.776 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:35.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:35.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:35.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337345+0000", 2024-09-16T00:10:35.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.007e-06 2024-09-16T00:10:35.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:35.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337416+0000", 2024-09-16T00:10:35.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1260999999999994e-05 2024-09-16T00:10:35.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:35.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337427+0000", 2024-09-16T00:10:35.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1225e-05 2024-09-16T00:10:35.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:35.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337434+0000", 2024-09-16T00:10:35.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8929999999999999e-06 2024-09-16T00:10:35.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337494+0000", 2024-09-16T00:10:35.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9670999999999998e-05 2024-09-16T00:10:35.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:35.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337505+0000", 2024-09-16T00:10:35.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0934999999999999e-05 2024-09-16T00:10:35.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.345708+0000", 2024-09-16T00:10:35.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0082033290000001 2024-09-16T00:10:35.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for missing object", 2024-09-16T00:10:35.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.345819+0000", 2024-09-16T00:10:35.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110299 2024-09-16T00:10:35.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.346937+0000", 2024-09-16T00:10:35.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011184120000000001 2024-09-16T00:10:35.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for degraded object", 2024-09-16T00:10:35.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.346968+0000", 2024-09-16T00:10:35.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0664999999999999e-05 2024-09-16T00:10:35.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:35.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.358107+0000", 2024-09-16T00:10:35.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011139046 2024-09-16T00:10:35.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:35.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:35.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:35.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.358163+0000", 2024-09-16T00:10:35.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5937999999999999e-05 2024-09-16T00:10:35.781 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.781 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.781 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.781 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:35.782 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:35.782 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:35.783 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:10:35.784 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:35.782+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:35.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:35.783+0000 7fd445227640 1 -- 172.21.15.139:0/135399181 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7fd44006e420 con 0x7fd4400c8b50 2024-09-16T00:10:35.884 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:35.882+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:35.916 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:35.916 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:35.916 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:35.916 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:35.917 INFO:teuthology.orchestra.run.smithi164.stdout:get_pg_num_history pg_num_history pg_num_history(e112 pg_nums {1={16=1},2={19=32},3={20=64}} deleted_pools ) 2024-09-16T00:10:35.917 INFO:teuthology.orchestra.run.smithi164.stdout:Importing pgid 3.14 2024-09-16T00:10:35.918 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:283cfb5b:::10000002254.00000000:head# 2024-09-16T00:10:35.918 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.918 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2849e444:::10000000583.00000000:head# 2024-09-16T00:10:35.919 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.919 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:28628fd5:::10000000a63.00000000:head# 2024-09-16T00:10:35.919 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.919 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:287d9ebf:::10000000a94.00000000:head# 2024-09-16T00:10:35.920 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.920 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:288e48f2:::10000002c14.00000000:head# 2024-09-16T00:10:35.920 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.920 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2890e941:::100000004d5.00000000:head# 2024-09-16T00:10:35.920 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.921 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:289c9a94:::10000002566.00000000:head# 2024-09-16T00:10:35.921 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.921 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:28a1b3d4:::100000026e2.00000000:head# 2024-09-16T00:10:35.921 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.922 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:28a2fcee:::10000000768.00000000:head# 2024-09-16T00:10:35.922 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.922 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:28a48acc:::10000000992.00000000:head# 2024-09-16T00:10:35.922 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.923 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:28b60791:::100000024e9.00000000:head# 2024-09-16T00:10:35.923 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.923 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:28cc090d:::1000000276c.00000000:head# 2024-09-16T00:10:35.924 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.924 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:28cdf2b1:::10000002bb1.00000000:head# 2024-09-16T00:10:35.924 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.924 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:290bfd0e:::10000001f93.00000000:head# 2024-09-16T00:10:35.924 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.925 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:292337b6:::100000023f6.00000000:head# 2024-09-16T00:10:35.925 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.925 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:29255704:::10000002c84.00000000:head# 2024-09-16T00:10:35.925 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.925 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2926fa2e:::10000000526.00000000:head# 2024-09-16T00:10:35.925 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.925 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2940346c:::100000028f1.00000000:head# 2024-09-16T00:10:35.926 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.926 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:294d4142:::10000002bfb.00000000:head# 2024-09-16T00:10:35.926 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.926 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:295cf825:::1000000215c.00000000:head# 2024-09-16T00:10:35.926 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.927 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:10:35.930 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:29783fe8:::1000000239d.00000000:head# 2024-09-16T00:10:35.930 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.930 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:29812fbf:::1000000307b.00000000:head# 2024-09-16T00:10:35.930 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.930 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2987f2bd:::100000023d6.00000000:head# 2024-09-16T00:10:35.930 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.930 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:298ca926:::100000024ed.00000000:head# 2024-09-16T00:10:35.930 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.930 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:298e2c1d:::1000000208b.00000000:head# 2024-09-16T00:10:35.931 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.931 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:29aa64e6:::10000000a8e.00000000:head# 2024-09-16T00:10:35.931 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.931 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:29b82759:::10000003083.00000000:head# 2024-09-16T00:10:35.931 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.931 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:29bcedda:::1000000309c.00000000:head# 2024-09-16T00:10:35.931 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.931 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:29bebe8d:::10000002598.00000000:head# 2024-09-16T00:10:35.931 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.931 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:29efae1c:::10000002b29.00000000:head# 2024-09-16T00:10:35.931 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.931 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:29f55c0b:::10000001fb6.00000000:head# 2024-09-16T00:10:35.931 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2a1ac447:::10000000568.00000000:head# 2024-09-16T00:10:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2a3d254e:::1000000282d.00000000:head# 2024-09-16T00:10:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2a52a257:::1000000247e.00000000:head# 2024-09-16T00:10:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2a587adf:::100000021d6.00000000:head# 2024-09-16T00:10:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2a63e1fc:::10000002472.00000000:head# 2024-09-16T00:10:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2a699bb3:::10000002962.00000000:head# 2024-09-16T00:10:35.934 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.935 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2a713e73:::10000000a2a.00000000:head# 2024-09-16T00:10:35.935 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.935 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2a75b760:::100000022b1.00000000:head# 2024-09-16T00:10:35.935 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.935 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2a8a3e7d:::10000002874.00000000:head# 2024-09-16T00:10:35.935 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.935 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2aa40b4a:::10000002cbb.00000000:head# 2024-09-16T00:10:35.935 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.936 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2aa7f009:::1000000088f.00000000:head# 2024-09-16T00:10:35.936 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.936 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2ac0c492:::10000002920.00000000:head# 2024-09-16T00:10:35.936 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.937 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2adb749a:::10000002ebc.00000000:head# 2024-09-16T00:10:35.937 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.937 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2ae0acad:::10000002b23.00000000:head# 2024-09-16T00:10:35.937 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.937 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2aea0116:::100000006cb.00000000:head# 2024-09-16T00:10:35.937 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.938 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2aee4717:::100000009a6.00000000:head# 2024-09-16T00:10:35.938 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.938 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2b0e64fc:::10000002456.00000000:head# 2024-09-16T00:10:35.938 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.938 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2b163ac1:::100000029ee.00000000:head# 2024-09-16T00:10:35.938 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.938 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2b2f79de:::10000000997.00000000:head# 2024-09-16T00:10:35.938 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.939 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2b50196b:::10000000558.00000000:head# 2024-09-16T00:10:35.939 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.939 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2b68aeaf:::100000020d0.00000000:head# 2024-09-16T00:10:35.939 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.939 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2b74aa0a:::1000000248e.00000000:head# 2024-09-16T00:10:35.940 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.940 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2b7c73ac:::10000000728.00000000:head# 2024-09-16T00:10:35.940 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.941 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2b80a786:::10000003030.00000000:head# 2024-09-16T00:10:35.941 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.941 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2b80f830:::10000002257.00000000:head# 2024-09-16T00:10:35.941 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.942 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2b9165f6:::10000002208.00000000:head# 2024-09-16T00:10:35.942 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.942 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2b94c4dc:::10000002463.00000000:head# 2024-09-16T00:10:35.942 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.942 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2bbbaa06:::10000003078.00000000:head# 2024-09-16T00:10:35.942 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.942 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2bbf5d77:::10000002946.00000000:head# 2024-09-16T00:10:35.942 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.943 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2be42a66:::100000028ef.00000000:head# 2024-09-16T00:10:35.943 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.943 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2bf94b1e:::10000002924.00000000:head# 2024-09-16T00:10:35.943 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.943 INFO:teuthology.orchestra.run.smithi164.stdout:Write #3:2bfc4227:::100000022eb.00000000:head# 2024-09-16T00:10:35.944 INFO:teuthology.orchestra.run.smithi164.stdout:snapset 1=[]:{} 2024-09-16T00:10:35.944 INFO:teuthology.orchestra.run.smithi164.stdout:write_pg epoch 112 info 3.14( v 112'310 (0'0,112'310] local-lis/les=20/21 n=63 ec=20/20 lis/c=20/20 les/c/f=21/21/0 sis=20) 2024-09-16T00:10:35.946 INFO:teuthology.orchestra.run.smithi164.stdout:Import successful 2024-09-16T00:10:35.985 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:35.984+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:36.051 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:36.051 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:36.051 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:36.051 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:36.051 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:36.062 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:10:36.103 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:36.102+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:36.187 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:36.187+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:36.244 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:36.244 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10780 3.a 3:500c83b2:::100000030b4.00000000:head [write 0~4194304 [3@2137521]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 57.788841613999999, 2024-09-16T00:10:36.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015526478, 2024-09-16T00:10:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.246 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10780 2024-09-16T00:10:36.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452917+0000", 2024-09-16T00:10:36.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2180000000000002e-06 2024-09-16T00:10:36.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462177+0000", 2024-09-16T00:10:36.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092593769999999992 2024-09-16T00:10:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462196+0000", 2024-09-16T00:10:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9745000000000002e-05 2024-09-16T00:10:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462207+0000", 2024-09-16T00:10:36.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0524e-05 2024-09-16T00:10:36.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462284+0000", 2024-09-16T00:10:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7608000000000006e-05 2024-09-16T00:10:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462376+0000", 2024-09-16T00:10:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1811999999999994e-05 2024-09-16T00:10:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468357+0000", 2024-09-16T00:10:36.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059806809999999998 2024-09-16T00:10:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468386+0000", 2024-09-16T00:10:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9068999999999999e-05 2024-09-16T00:10:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468441+0000", 2024-09-16T00:10:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5444e-05 2024-09-16T00:10:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.257 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10831 3.1a 3:59b8f98c:::100000030df.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 57.466034477999997, 2024-09-16T00:10:36.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014692843000000001, 2024-09-16T00:10:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.259 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10831 2024-09-16T00:10:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775723+0000", 2024-09-16T00:10:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.367e-06 2024-09-16T00:10:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780402+0000", 2024-09-16T00:10:36.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046785769999999997 2024-09-16T00:10:36.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780421+0000", 2024-09-16T00:10:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9179000000000001e-05 2024-09-16T00:10:36.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780440+0000", 2024-09-16T00:10:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9015e-05 2024-09-16T00:10:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780533+0000", 2024-09-16T00:10:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3197999999999998e-05 2024-09-16T00:10:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780618+0000", 2024-09-16T00:10:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4787e-05 2024-09-16T00:10:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780748+0000", 2024-09-16T00:10:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012927199999999999 2024-09-16T00:10:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.785049+0000", 2024-09-16T00:10:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043015750000000002 2024-09-16T00:10:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790312+0000", 2024-09-16T00:10:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052633319999999999 2024-09-16T00:10:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790369+0000", 2024-09-16T00:10:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6848999999999997e-05 2024-09-16T00:10:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790415+0000", 2024-09-16T00:10:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5692000000000002e-05 2024-09-16T00:10:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.273 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.274 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10915 3.1a 3:5a6e15a2:::1000000318e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 57.026528556999999, 2024-09-16T00:10:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014599472, 2024-09-16T00:10:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.275 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10915 2024-09-16T00:10:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.276 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.278 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:36.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215230+0000", 2024-09-16T00:10:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0090000000000002e-06 2024-09-16T00:10:36.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219556+0000", 2024-09-16T00:10:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043259539999999999 2024-09-16T00:10:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219577+0000", 2024-09-16T00:10:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1271999999999999e-05 2024-09-16T00:10:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219589+0000", 2024-09-16T00:10:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1766000000000001e-05 2024-09-16T00:10:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219663+0000", 2024-09-16T00:10:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4196999999999995e-05 2024-09-16T00:10:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219733+0000", 2024-09-16T00:10:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9553000000000003e-05 2024-09-16T00:10:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219827+0000", 2024-09-16T00:10:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4395999999999995e-05 2024-09-16T00:10:36.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.225724+0000", 2024-09-16T00:10:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058968359999999999 2024-09-16T00:10:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229705+0000", 2024-09-16T00:10:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003981409 2024-09-16T00:10:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229772+0000", 2024-09-16T00:10:36.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6137999999999996e-05 2024-09-16T00:10:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229828+0000", 2024-09-16T00:10:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5942000000000001e-05 2024-09-16T00:10:36.287 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11074 3.14 3:2834b7ee:::10000003131.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.328988434000003, 2024-09-16T00:10:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015588350000000001, 2024-09-16T00:10:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.288 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11074 2024-09-16T00:10:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.289 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912769+0000", 2024-09-16T00:10:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4600000000000004e-07 2024-09-16T00:10:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918264+0000", 2024-09-16T00:10:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054954890000000001 2024-09-16T00:10:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918281+0000", 2024-09-16T00:10:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6160000000000001e-05 2024-09-16T00:10:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918289+0000", 2024-09-16T00:10:36.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1650000000000006e-06 2024-09-16T00:10:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918386+0000", 2024-09-16T00:10:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7288999999999996e-05 2024-09-16T00:10:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918535+0000", 2024-09-16T00:10:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014876200000000001 2024-09-16T00:10:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918639+0000", 2024-09-16T00:10:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010470400000000001 2024-09-16T00:10:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.924703+0000", 2024-09-16T00:10:36.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060639759999999996 2024-09-16T00:10:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928246+0000", 2024-09-16T00:10:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035421720000000001 2024-09-16T00:10:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928305+0000", 2024-09-16T00:10:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9014000000000001e-05 2024-09-16T00:10:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928357+0000", 2024-09-16T00:10:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1873000000000003e-05 2024-09-16T00:10:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11111 3.9 3:93301cc4:::10000003210.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.152250232, 2024-09-16T00:10:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014336099999999999, 2024-09-16T00:10:36.300 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11111 2024-09-16T00:10:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089520+0000", 2024-09-16T00:10:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3903000000000001e-05 2024-09-16T00:10:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094463+0000", 2024-09-16T00:10:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049423000000000002 2024-09-16T00:10:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094494+0000", 2024-09-16T00:10:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1029999999999999e-05 2024-09-16T00:10:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094503+0000", 2024-09-16T00:10:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0180000000000007e-06 2024-09-16T00:10:36.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094573+0000", 2024-09-16T00:10:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0457000000000007e-05 2024-09-16T00:10:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094737+0000", 2024-09-16T00:10:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016408699999999999 2024-09-16T00:10:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:36.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094897+0000", 2024-09-16T00:10:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015938899999999999 2024-09-16T00:10:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.102326+0000", 2024-09-16T00:10:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074291230000000002 2024-09-16T00:10:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103723+0000", 2024-09-16T00:10:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001397255 2024-09-16T00:10:36.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103786+0000", 2024-09-16T00:10:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3489000000000001e-05 2024-09-16T00:10:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103842+0000", 2024-09-16T00:10:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6048999999999998e-05 2024-09-16T00:10:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:11934 3.1b e87/20 3:d94fa8da:::10000003190.00000000:head v 87'152, mlcod=87'150)", 2024-09-16T00:10:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.551505536000001, 2024-09-16T00:10:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014984934, 2024-09-16T00:10:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:36.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690252+0000", 2024-09-16T00:10:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8899999999999998e-07 2024-09-16T00:10:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696026+0000", 2024-09-16T00:10:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057745219999999998 2024-09-16T00:10:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696039+0000", 2024-09-16T00:10:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2611e-05 2024-09-16T00:10:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696047+0000", 2024-09-16T00:10:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5399999999999998e-06 2024-09-16T00:10:36.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696123+0000", 2024-09-16T00:10:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6349999999999999e-05 2024-09-16T00:10:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696144+0000", 2024-09-16T00:10:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1512999999999999e-05 2024-09-16T00:10:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705188+0000", 2024-09-16T00:10:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090438459999999995 2024-09-16T00:10:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705236+0000", 2024-09-16T00:10:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7663000000000001e-05 2024-09-16T00:10:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.311 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11936 3.3 3:c0886fb9:::100000031a1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.541360484000002, 2024-09-16T00:10:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015336379000000001, 2024-09-16T00:10:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11936 2024-09-16T00:10:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700399+0000", 2024-09-16T00:10:36.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6029999999999999e-06 2024-09-16T00:10:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704706+0000", 2024-09-16T00:10:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043070529999999999 2024-09-16T00:10:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704720+0000", 2024-09-16T00:10:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4629e-05 2024-09-16T00:10:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704729+0000", 2024-09-16T00:10:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6009999999999995e-06 2024-09-16T00:10:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704805+0000", 2024-09-16T00:10:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6015999999999999e-05 2024-09-16T00:10:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704866+0000", 2024-09-16T00:10:36.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0878999999999998e-05 2024-09-16T00:10:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704990+0000", 2024-09-16T00:10:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000124058 2024-09-16T00:10:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.714314+0000", 2024-09-16T00:10:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009324321 2024-09-16T00:10:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715632+0000", 2024-09-16T00:10:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001318121 2024-09-16T00:10:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715687+0000", 2024-09-16T00:10:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4135999999999998e-05 2024-09-16T00:10:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715732+0000", 2024-09-16T00:10:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961999999999998e-05 2024-09-16T00:10:36.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11953 3.1a 3:580e2503:::1000000311b.00000000:head [write 0~4194304 [2@2058066]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:36.318 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.475021331000001, 2024-09-16T00:10:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016299973999999998, 2024-09-16T00:10:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11953 2024-09-16T00:10:36.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766741+0000", 2024-09-16T00:10:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2939999999999996e-06 2024-09-16T00:10:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771165+0000", 2024-09-16T00:10:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044248229999999996 2024-09-16T00:10:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771181+0000", 2024-09-16T00:10:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5243e-05 2024-09-16T00:10:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771188+0000", 2024-09-16T00:10:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6429999999999995e-06 2024-09-16T00:10:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771238+0000", 2024-09-16T00:10:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9567999999999997e-05 2024-09-16T00:10:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771293+0000", 2024-09-16T00:10:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5207000000000002e-05 2024-09-16T00:10:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771380+0000", 2024-09-16T00:10:36.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6716000000000001e-05 2024-09-16T00:10:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.776193+0000", 2024-09-16T00:10:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048133350000000002 2024-09-16T00:10:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.782931+0000", 2024-09-16T00:10:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067376550000000004 2024-09-16T00:10:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.782987+0000", 2024-09-16T00:10:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6443000000000002e-05 2024-09-16T00:10:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.783035+0000", 2024-09-16T00:10:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8047000000000003e-05 2024-09-16T00:10:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12037 3.9 3:928db521:::1000000326d.00000000:head [write 0~4194304 [2@1655200]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:36.325 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.243813723000002, 2024-09-16T00:10:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017082494, 2024-09-16T00:10:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.326 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12037 2024-09-16T00:10:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997944+0000", 2024-09-16T00:10:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.517e-06 2024-09-16T00:10:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002177+0000", 2024-09-16T00:10:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042329300000000002 2024-09-16T00:10:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002197+0000", 2024-09-16T00:10:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9769e-05 2024-09-16T00:10:36.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002207+0000", 2024-09-16T00:10:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0178e-05 2024-09-16T00:10:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002266+0000", 2024-09-16T00:10:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9215999999999997e-05 2024-09-16T00:10:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002340+0000", 2024-09-16T00:10:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3798000000000001e-05 2024-09-16T00:10:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:36.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002454+0000", 2024-09-16T00:10:36.501 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113551 2024-09-16T00:10:36.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.006270+0000", 2024-09-16T00:10:36.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038163839999999999 2024-09-16T00:10:36.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.502 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.014957+0000", 2024-09-16T00:10:36.502 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086869119999999998 2024-09-16T00:10:36.502 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.502 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.502 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.014996+0000", 2024-09-16T00:10:36.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8878000000000003e-05 2024-09-16T00:10:36.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.503 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.503 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.015025+0000", 2024-09-16T00:10:36.503 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9360999999999999e-05 2024-09-16T00:10:36.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.503 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.503 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.503 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.503 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.503 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12245 3.1a 3:599a1cd1:::10000003252.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:36.503 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:36.504 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.459116305999999, 2024-09-16T00:10:36.504 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017131203000000001, 2024-09-16T00:10:36.504 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.504 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.504 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.504 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.504 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12245 2024-09-16T00:10:36.504 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.504 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.504 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.504 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:36.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:36.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.505 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.505 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782642+0000", 2024-09-16T00:10:36.505 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.311e-06 2024-09-16T00:10:36.505 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.505 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786551+0000", 2024-09-16T00:10:36.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039095229999999998 2024-09-16T00:10:36.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786566+0000", 2024-09-16T00:10:36.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4724e-05 2024-09-16T00:10:36.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.506 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.506 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.506 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786575+0000", 2024-09-16T00:10:36.506 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8449999999999992e-06 2024-09-16T00:10:36.506 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786636+0000", 2024-09-16T00:10:36.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.143e-05 2024-09-16T00:10:36.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786699+0000", 2024-09-16T00:10:36.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3218000000000004e-05 2024-09-16T00:10:36.507 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.507 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.507 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:36.507 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786796+0000", 2024-09-16T00:10:36.507 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6347000000000004e-05 2024-09-16T00:10:36.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.792369+0000", 2024-09-16T00:10:36.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055729780000000001 2024-09-16T00:10:36.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.508 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799706+0000", 2024-09-16T00:10:36.508 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073374349999999998 2024-09-16T00:10:36.508 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.508 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.508 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799739+0000", 2024-09-16T00:10:36.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2833999999999998e-05 2024-09-16T00:10:36.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.509 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.509 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799771+0000", 2024-09-16T00:10:36.509 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2558000000000002e-05 2024-09-16T00:10:36.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.509 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.509 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.509 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.509 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.509 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12292 3.3 3:c00779fd:::1000000320e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:36.509 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:36.510 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.344726209000001, 2024-09-16T00:10:36.510 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013828353999999999, 2024-09-16T00:10:36.510 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.510 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.510 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.510 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.510 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12292 2024-09-16T00:10:36.510 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.511 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:36.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:36.511 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.511 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.511 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.511 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.511 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897039+0000", 2024-09-16T00:10:36.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3299999999999999e-06 2024-09-16T00:10:36.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900830+0000", 2024-09-16T00:10:36.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037915340000000001 2024-09-16T00:10:36.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.512 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900850+0000", 2024-09-16T00:10:36.512 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9400000000000001e-05 2024-09-16T00:10:36.512 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.512 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.512 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900860+0000", 2024-09-16T00:10:36.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0609e-05 2024-09-16T00:10:36.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900928+0000", 2024-09-16T00:10:36.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7570000000000005e-05 2024-09-16T00:10:36.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.513 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.513 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.513 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.901027+0000", 2024-09-16T00:10:36.513 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9456000000000005e-05 2024-09-16T00:10:36.513 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:36.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.901168+0000", 2024-09-16T00:10:36.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014082799999999999 2024-09-16T00:10:36.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.906325+0000", 2024-09-16T00:10:36.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005156702 2024-09-16T00:10:36.514 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.514 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.514 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.514 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910796+0000", 2024-09-16T00:10:36.514 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044714350000000002 2024-09-16T00:10:36.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910829+0000", 2024-09-16T00:10:36.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3229000000000003e-05 2024-09-16T00:10:36.515 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.515 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.515 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.515 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910859+0000", 2024-09-16T00:10:36.515 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9261e-05 2024-09-16T00:10:36.515 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.516 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.516 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.516 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.516 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.516 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13326 3.9 3:92f796cf:::1000000350f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:36.516 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:36.516 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.285035927000003, 2024-09-16T00:10:36.516 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015673229, 2024-09-16T00:10:36.516 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.516 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.516 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.516 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.517 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.517 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13326 2024-09-16T00:10:36.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.517 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:36.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.517 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.517 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.517 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:36.517 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.517 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956722+0000", 2024-09-16T00:10:36.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3200000000000001e-06 2024-09-16T00:10:36.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.518 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.518 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.518 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960803+0000", 2024-09-16T00:10:36.518 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040807029999999998 2024-09-16T00:10:36.518 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.519 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.519 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960820+0000", 2024-09-16T00:10:36.519 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7478000000000001e-05 2024-09-16T00:10:36.519 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.519 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960830+0000", 2024-09-16T00:10:36.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:10:36.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960903+0000", 2024-09-16T00:10:36.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3057000000000002e-05 2024-09-16T00:10:36.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.520 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.520 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.520 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960970+0000", 2024-09-16T00:10:36.520 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7063000000000005e-05 2024-09-16T00:10:36.520 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:36.521 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.961067+0000", 2024-09-16T00:10:36.521 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6906999999999999e-05 2024-09-16T00:10:36.521 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.521 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.521 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.967945+0000", 2024-09-16T00:10:36.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068787359999999999 2024-09-16T00:10:36.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972328+0000", 2024-09-16T00:10:36.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043826170000000001 2024-09-16T00:10:36.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.522 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.522 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.522 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972363+0000", 2024-09-16T00:10:36.522 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4486000000000002e-05 2024-09-16T00:10:36.522 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.523 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.523 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972394+0000", 2024-09-16T00:10:36.523 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1343999999999997e-05 2024-09-16T00:10:36.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.523 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.523 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.523 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.523 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.523 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13429 3.1a 3:58dcd24a:::100000036d5.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:36.523 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:36.523 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.930465910999999, 2024-09-16T00:10:36.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014237671, 2024-09-16T00:10:36.524 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.524 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.524 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.524 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.524 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13429 2024-09-16T00:10:36.524 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.524 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.524 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.524 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.524 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:36.524 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:36.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311292+0000", 2024-09-16T00:10:36.525 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4890000000000001e-06 2024-09-16T00:10:36.525 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.525 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.525 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.525 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316378+0000", 2024-09-16T00:10:36.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050859290000000003 2024-09-16T00:10:36.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316399+0000", 2024-09-16T00:10:36.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1345e-05 2024-09-16T00:10:36.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.526 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316410+0000", 2024-09-16T00:10:36.526 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0295999999999999e-05 2024-09-16T00:10:36.526 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.526 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.526 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316493+0000", 2024-09-16T00:10:36.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3533000000000001e-05 2024-09-16T00:10:36.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316622+0000", 2024-09-16T00:10:36.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012887900000000001 2024-09-16T00:10:36.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.527 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:36.527 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316725+0000", 2024-09-16T00:10:36.527 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010295599999999999 2024-09-16T00:10:36.527 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.527 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.320948+0000", 2024-09-16T00:10:36.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042229820000000001 2024-09-16T00:10:36.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325415+0000", 2024-09-16T00:10:36.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044673849999999999 2024-09-16T00:10:36.528 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.528 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.528 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.528 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325487+0000", 2024-09-16T00:10:36.528 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1087999999999994e-05 2024-09-16T00:10:36.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.529 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.529 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325528+0000", 2024-09-16T00:10:36.529 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1789000000000003e-05 2024-09-16T00:10:36.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.529 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.529 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.529 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.529 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.529 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13527 3.3 3:c0886fb9:::100000031a1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:36.529 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:36.529 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.456305821999997, 2024-09-16T00:10:36.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015187236, 2024-09-16T00:10:36.530 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.530 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.530 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.530 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.530 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13527 2024-09-16T00:10:36.530 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.530 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.530 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.530 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.530 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:36.530 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:36.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785453+0000", 2024-09-16T00:10:36.531 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1689999999999999e-06 2024-09-16T00:10:36.531 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.531 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.531 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.531 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791191+0000", 2024-09-16T00:10:36.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057378339999999998 2024-09-16T00:10:36.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.532 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791217+0000", 2024-09-16T00:10:36.532 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6480999999999999e-05 2024-09-16T00:10:36.532 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.532 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.532 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791233+0000", 2024-09-16T00:10:36.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5841e-05 2024-09-16T00:10:36.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791319+0000", 2024-09-16T00:10:36.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5527999999999998e-05 2024-09-16T00:10:36.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.533 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.533 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791413+0000", 2024-09-16T00:10:36.533 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4685000000000006e-05 2024-09-16T00:10:36.533 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.533 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:36.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791563+0000", 2024-09-16T00:10:36.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000149624 2024-09-16T00:10:36.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.795749+0000", 2024-09-16T00:10:36.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004186286 2024-09-16T00:10:36.534 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.534 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.534 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.534 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800577+0000", 2024-09-16T00:10:36.534 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048273040000000001 2024-09-16T00:10:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800611+0000", 2024-09-16T00:10:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4270000000000002e-05 2024-09-16T00:10:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800638+0000", 2024-09-16T00:10:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7214000000000001e-05 2024-09-16T00:10:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.535 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13724 3.9 3:90613e6c:::100000034c4.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.786234077000003, 2024-09-16T00:10:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014039800999999999, 2024-09-16T00:10:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13724 2024-09-16T00:10:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.536 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:36.537 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455524+0000", 2024-09-16T00:10:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.437e-06 2024-09-16T00:10:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459592+0000", 2024-09-16T00:10:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040678729999999996 2024-09-16T00:10:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.538 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459610+0000", 2024-09-16T00:10:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7688000000000001e-05 2024-09-16T00:10:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459617+0000", 2024-09-16T00:10:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.374e-06 2024-09-16T00:10:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459678+0000", 2024-09-16T00:10:36.539 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1346999999999994e-05 2024-09-16T00:10:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459824+0000", 2024-09-16T00:10:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014586699999999999 2024-09-16T00:10:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459908+0000", 2024-09-16T00:10:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3521000000000002e-05 2024-09-16T00:10:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.540 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.464227+0000", 2024-09-16T00:10:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043197060000000004 2024-09-16T00:10:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469499+0000", 2024-09-16T00:10:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005271838 2024-09-16T00:10:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469534+0000", 2024-09-16T00:10:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4712000000000002e-05 2024-09-16T00:10:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.541 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469562+0000", 2024-09-16T00:10:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8438000000000001e-05 2024-09-16T00:10:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13741 3.3 3:c1377c76:::10000003664.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.739064132000003, 2024-09-16T00:10:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016142843, 2024-09-16T00:10:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.542 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13741 2024-09-16T00:10:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.543 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502694+0000", 2024-09-16T00:10:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0449999999999999e-06 2024-09-16T00:10:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507229+0000", 2024-09-16T00:10:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045357410000000003 2024-09-16T00:10:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507248+0000", 2024-09-16T00:10:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9127000000000002e-05 2024-09-16T00:10:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507259+0000", 2024-09-16T00:10:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0448999999999999e-05 2024-09-16T00:10:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.545 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507357+0000", 2024-09-16T00:10:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8400000000000007e-05 2024-09-16T00:10:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507514+0000", 2024-09-16T00:10:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000157195 2024-09-16T00:10:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507617+0000", 2024-09-16T00:10:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010262199999999999 2024-09-16T00:10:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.546 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.514047+0000", 2024-09-16T00:10:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006430018 2024-09-16T00:10:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518743+0000", 2024-09-16T00:10:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046963040000000001 2024-09-16T00:10:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518796+0000", 2024-09-16T00:10:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2305000000000002e-05 2024-09-16T00:10:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.547 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518835+0000", 2024-09-16T00:10:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9637e-05 2024-09-16T00:10:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14633 3.9 3:9170e82c:::100000035d6.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.619683876000003, 2024-09-16T00:10:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014466105, 2024-09-16T00:10:36.548 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.549 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.549 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.549 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.549 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.549 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.549 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14633 2024-09-16T00:10:36.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.549 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.549 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.549 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.549 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:36.549 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.549 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:36.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622074+0000", 2024-09-16T00:10:36.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.761e-06 2024-09-16T00:10:36.550 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.550 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.550 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.550 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626712+0000", 2024-09-16T00:10:36.550 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004637263 2024-09-16T00:10:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626724+0000", 2024-09-16T00:10:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2479e-05 2024-09-16T00:10:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626730+0000", 2024-09-16T00:10:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7309999999999998e-06 2024-09-16T00:10:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.551 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626791+0000", 2024-09-16T00:10:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0854999999999999e-05 2024-09-16T00:10:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626857+0000", 2024-09-16T00:10:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6549000000000003e-05 2024-09-16T00:10:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626943+0000", 2024-09-16T00:10:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6141000000000001e-05 2024-09-16T00:10:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.552 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.630854+0000", 2024-09-16T00:10:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039100790000000003 2024-09-16T00:10:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636468+0000", 2024-09-16T00:10:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056140570000000004 2024-09-16T00:10:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636500+0000", 2024-09-16T00:10:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2441999999999999e-05 2024-09-16T00:10:36.553 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636539+0000", 2024-09-16T00:10:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8748000000000002e-05 2024-09-16T00:10:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14667 3.9 3:9379a4af:::10000003352.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.444671870000001, 2024-09-16T00:10:36.554 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014580585, 2024-09-16T00:10:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14667 2024-09-16T00:10:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.555 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797086+0000", 2024-09-16T00:10:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.362e-06 2024-09-16T00:10:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801415+0000", 2024-09-16T00:10:36.556 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043288449999999996 2024-09-16T00:10:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801435+0000", 2024-09-16T00:10:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0305999999999998e-05 2024-09-16T00:10:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801442+0000", 2024-09-16T00:10:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6290000000000004e-06 2024-09-16T00:10:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.557 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801497+0000", 2024-09-16T00:10:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5111999999999997e-05 2024-09-16T00:10:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801639+0000", 2024-09-16T00:10:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014168000000000001 2024-09-16T00:10:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801788+0000", 2024-09-16T00:10:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014963300000000001 2024-09-16T00:10:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.558 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.805407+0000", 2024-09-16T00:10:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036186119999999998 2024-09-16T00:10:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811584+0000", 2024-09-16T00:10:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061772889999999999 2024-09-16T00:10:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.559 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811635+0000", 2024-09-16T00:10:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0453e-05 2024-09-16T00:10:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811665+0000", 2024-09-16T00:10:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0664000000000003e-05 2024-09-16T00:10:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14951 3.1a 3:59aee80a:::1000000394f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:36.560 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 42.274633975, 2024-09-16T00:10:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015993941000000001, 2024-09-16T00:10:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14951 2024-09-16T00:10:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.561 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967124+0000", 2024-09-16T00:10:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.86e-06 2024-09-16T00:10:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.562 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971258+0000", 2024-09-16T00:10:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041334780000000003 2024-09-16T00:10:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971274+0000", 2024-09-16T00:10:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6201e-05 2024-09-16T00:10:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971284+0000", 2024-09-16T00:10:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4779999999999997e-06 2024-09-16T00:10:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.563 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971384+0000", 2024-09-16T00:10:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9878999999999997e-05 2024-09-16T00:10:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971518+0000", 2024-09-16T00:10:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013459400000000001 2024-09-16T00:10:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971604+0000", 2024-09-16T00:10:36.564 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6215e-05 2024-09-16T00:10:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.976673+0000", 2024-09-16T00:10:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050685920000000002 2024-09-16T00:10:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983043+0000", 2024-09-16T00:10:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063698410000000002 2024-09-16T00:10:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.565 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983082+0000", 2024-09-16T00:10:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9261999999999998e-05 2024-09-16T00:10:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983117+0000", 2024-09-16T00:10:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4541e-05 2024-09-16T00:10:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:15152 3.3 3:c388c7e2:::100000032ec.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-16T00:10:36.566 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.514087865, 2024-09-16T00:10:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014603532000000001, 2024-09-16T00:10:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15152 2024-09-16T00:10:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.567 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727674+0000", 2024-09-16T00:10:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7699999999999998e-06 2024-09-16T00:10:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.568 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732274+0000", 2024-09-16T00:10:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045994299999999998 2024-09-16T00:10:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732286+0000", 2024-09-16T00:10:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2016e-05 2024-09-16T00:10:36.569 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732294+0000", 2024-09-16T00:10:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7049999999999999e-06 2024-09-16T00:10:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732359+0000", 2024-09-16T00:10:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5296000000000006e-05 2024-09-16T00:10:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732501+0000", 2024-09-16T00:10:36.570 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000141731 2024-09-16T00:10:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732629+0000", 2024-09-16T00:10:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012841299999999999 2024-09-16T00:10:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.737511+0000", 2024-09-16T00:10:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048822520000000001 2024-09-16T00:10:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.571 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742172+0000", 2024-09-16T00:10:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046610610000000002 2024-09-16T00:10:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742227+0000", 2024-09-16T00:10:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4497999999999999e-05 2024-09-16T00:10:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742272+0000", 2024-09-16T00:10:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5359999999999999e-05 2024-09-16T00:10:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.572 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:36.573 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:36.573 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:36.573 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:10:36.575 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:10:36.577 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f /var/log/ceph/exp.3.14.7 2024-09-16T00:10:36.577 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:36.573+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.476+0000 7fd4277fe640 1 -- 172.21.15.139:0/135399181 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v117) ==== 91+0+0 (secure 0 0 0) 0x7fd43405d600 con 0x7fd4400c8b50 2024-09-16T00:10:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is unset 2024-09-16T00:10:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.478+0000 7fd445227640 1 -- 172.21.15.139:0/135399181 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd42c06c2b0 msgr2=0x7fd42c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.478+0000 7fd445227640 1 --2- 172.21.15.139:0/135399181 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd42c06c2b0 0x7fd42c06e770 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd43002a1c0 tx=0x7fd43002e000 comp rx=0 tx=0).stop 2024-09-16T00:10:36.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.478+0000 7fd445227640 1 -- 172.21.15.139:0/135399181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4400c8b50 msgr2=0x7fd4401f0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:36.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.478+0000 7fd445227640 1 --2- 172.21.15.139:0/135399181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4400c8b50 0x7fd4401f0110 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd43400e8d0 tx=0x7fd43400ed90 comp rx=0 tx=0).stop 2024-09-16T00:10:36.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.478+0000 7fd445227640 1 -- 172.21.15.139:0/135399181 shutdown_connections 2024-09-16T00:10:36.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.478+0000 7fd445227640 1 --2- 172.21.15.139:0/135399181 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fd42c06c2b0 0x7fd42c06e770 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:36.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.478+0000 7fd445227640 1 --2- 172.21.15.139:0/135399181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd440154bd0 0x7fd4401f0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:36.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.478+0000 7fd445227640 1 --2- 172.21.15.139:0/135399181 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd4400c8b50 0x7fd4401f0110 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:36.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.478+0000 7fd445227640 1 --2- 172.21.15.139:0/135399181 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fd44006ea20 0x7fd44014a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:36.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.478+0000 7fd445227640 1 -- 172.21.15.139:0/135399181 >> 172.21.15.139:0/135399181 conn(0x7fd44006c150 msgr2=0x7fd440067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:36.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.479+0000 7fd445227640 1 -- 172.21.15.139:0/135399181 shutdown_connections 2024-09-16T00:10:36.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.479+0000 7fd445227640 1 -- 172.21.15.139:0/135399181 wait complete. 2024-09-16T00:10:36.675 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:36.675+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:36.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.718+0000 7f525a1b7640 1 Processor -- start 2024-09-16T00:10:36.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.718+0000 7f525a1b7640 1 -- start start 2024-09-16T00:10:36.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.719+0000 7f525a1b7640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f52540ca760 0x7f525415b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.719+0000 7f525a1b7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525406cb30 0x7f525415b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.719+0000 7f525a1b7640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52540b2b00 con 0x7f525415e360 2024-09-16T00:10:36.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.719+0000 7f525a1b7640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5254073c10 con 0x7f52540ca760 2024-09-16T00:10:36.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.719+0000 7f525a1b7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f525415bcd0 con 0x7f525406cb30 2024-09-16T00:10:36.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.719+0000 7f524bfff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f52540ca760 0x7f525415b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.719+0000 7f52537fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f525415e360 0x7f525415c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:53712/0 (socket says 172.21.15.139:53712) 2024-09-16T00:10:36.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.719+0000 7f52537fe640 1 -- 172.21.15.139:0/1076955718 learned_addr learned my addr 172.21.15.139:0/1076955718 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:36.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.719+0000 7f524bfff640 1 --2- 172.21.15.139:0/1076955718 >> v2:172.21.15.164:3300/0 conn(0x7f52540ca760 0x7f525415b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:45294/0 (socket says 172.21.15.139:45294) 2024-09-16T00:10:36.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.719+0000 7f5253fff640 1 --2- 172.21.15.139:0/1076955718 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525406cb30 0x7f525415b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.719+0000 7f5253fff640 1 -- 172.21.15.139:0/1076955718 >> v2:172.21.15.164:3300/0 conn(0x7f52540ca760 msgr2=0x7f525415b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:36.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.719+0000 7f5253fff640 1 --2- 172.21.15.139:0/1076955718 >> v2:172.21.15.164:3300/0 conn(0x7f52540ca760 0x7f525415b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:36.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.719+0000 7f5253fff640 1 -- 172.21.15.139:0/1076955718 >> v1:172.21.15.164:6789/0 conn(0x7f525415e360 legacy=0x7f525415c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:36.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.719+0000 7f5253fff640 1 -- 172.21.15.139:0/1076955718 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525415ecc0 con 0x7f525406cb30 2024-09-16T00:10:36.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.719+0000 7f5252ffd640 1 -- 172.21.15.139:0/1076955718 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1736956041 0 0) 0x7f52540b2b00 con 0x7f525415e360 2024-09-16T00:10:36.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.719+0000 7f524bfff640 1 --2- 172.21.15.139:0/1076955718 >> v2:172.21.15.164:3300/0 conn(0x7f52540ca760 0x7f525415b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:36.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.720+0000 7f5253fff640 1 --2- 172.21.15.139:0/1076955718 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525406cb30 0x7f525415b790 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5254073d90 tx=0x7f524c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d39e80fe47d6eea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:36.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.720+0000 7f5252ffd640 1 -- 172.21.15.139:0/1076955718 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c007740 con 0x7f525406cb30 2024-09-16T00:10:36.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.720+0000 7f5252ffd640 1 -- 172.21.15.139:0/1076955718 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f524c0078e0 con 0x7f525406cb30 2024-09-16T00:10:36.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.720+0000 7f5252ffd640 1 -- 172.21.15.139:0/1076955718 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524c007cb0 con 0x7f525406cb30 2024-09-16T00:10:36.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.721+0000 7f525a1b7640 1 -- 172.21.15.139:0/1076955718 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525406cb30 msgr2=0x7f525415b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:36.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.721+0000 7f525a1b7640 1 --2- 172.21.15.139:0/1076955718 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525406cb30 0x7f525415b790 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5254073d90 tx=0x7f524c02a6d0 comp rx=0 tx=0).stop 2024-09-16T00:10:36.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.721+0000 7f525a1b7640 1 -- 172.21.15.139:0/1076955718 shutdown_connections 2024-09-16T00:10:36.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.721+0000 7f525a1b7640 1 --2- 172.21.15.139:0/1076955718 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525406cb30 0x7f525415b790 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:36.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.721+0000 7f525a1b7640 1 --2- 172.21.15.139:0/1076955718 >> v2:172.21.15.164:3300/0 conn(0x7f52540ca760 0x7f525415b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:36.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.721+0000 7f525a1b7640 1 -- 172.21.15.139:0/1076955718 >> 172.21.15.139:0/1076955718 conn(0x7f52540c9af0 msgr2=0x7f52540c9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:36.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.721+0000 7f525a1b7640 1 -- 172.21.15.139:0/1076955718 shutdown_connections 2024-09-16T00:10:36.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.721+0000 7f525a1b7640 1 -- 172.21.15.139:0/1076955718 wait complete. 2024-09-16T00:10:36.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.722+0000 7f525a1b7640 1 Processor -- start 2024-09-16T00:10:36.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.722+0000 7f525a1b7640 1 -- start start 2024-09-16T00:10:36.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f525a1b7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f525406cb30 0x7f52541f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f525a1b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52540ca760 0x7f52540aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f525a1b7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525415e360 0x7f52540af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f525a1b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f52540738a0 con 0x7f52540ca760 2024-09-16T00:10:36.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f525a1b7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5254073320 con 0x7f525406cb30 2024-09-16T00:10:36.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f525a1b7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f525406fc30 con 0x7f525415e360 2024-09-16T00:10:36.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f524bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52540ca760 0x7f52540aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f5253fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525415e360 0x7f52540af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f52537fe640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f525406cb30 0x7f52541f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f524bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52540ca760 0x7f52540aef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:49518/0 (socket says 172.21.15.139:49518) 2024-09-16T00:10:36.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f524bfff640 1 -- 172.21.15.139:0/3769996489 learned_addr learned my addr 172.21.15.139:0/3769996489 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:36.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f524bfff640 1 -- 172.21.15.139:0/3769996489 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525415e360 msgr2=0x7f52540af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:36.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f524bfff640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525415e360 0x7f52540af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:36.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f524bfff640 1 -- 172.21.15.139:0/3769996489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f525406cb30 msgr2=0x7f52541f39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:36.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f524bfff640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f525406cb30 0x7f52541f39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:36.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f524bfff640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52541f43f0 con 0x7f52540ca760 2024-09-16T00:10:36.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.723+0000 7f5253fff640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f525415e360 0x7f52540af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:36.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.724+0000 7f524bfff640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52540ca760 0x7f52540aef40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f523c00ebe0 tx=0x7f523c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:36.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.724+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c017070 con 0x7f52540ca760 2024-09-16T00:10:36.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.724+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f523c00cbd0 con 0x7f52540ca760 2024-09-16T00:10:36.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.724+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c013470 con 0x7f52540ca760 2024-09-16T00:10:36.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.724+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52540cadf0 con 0x7f52540ca760 2024-09-16T00:10:36.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.724+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52540cb390 con 0x7f52540ca760 2024-09-16T00:10:36.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.725+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f525406e2a0 con 0x7f52540ca760 2024-09-16T00:10:36.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.725+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f523c01a070 con 0x7f52540ca760 2024-09-16T00:10:36.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.725+0000 7f5250ff9640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f523406c2b0 0x7f523406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.726+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5973+0+0 (secure 0 0 0) 0x7f523c08b6e0 con 0x7f52540ca760 2024-09-16T00:10:36.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.726+0000 7f52537fe640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f523406c2b0 0x7f523406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.726+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v117) ==== 54+0+16 (secure 0 0 0) 0x7f523c091170 con 0x7f52540ca760 2024-09-16T00:10:36.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.726+0000 7f525a1b7640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f52541f3ee0 0x7f52540afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.726+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52540c71e0 con 0x7f52541f3ee0 2024-09-16T00:10:36.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.726+0000 7f52537fe640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f523406c2b0 0x7f523406e770 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5244007fd0 tx=0x7f5244007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:36.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.726+0000 7f5253fff640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f52541f3ee0 0x7f52540afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.726+0000 7f5253fff640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f52541f3ee0 0x7f52540afe50 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:36.731 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:36.732 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:36.732 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:36.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.727+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52540c71e0 con 0x7f52541f3ee0 2024-09-16T00:10:36.739 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:10:36.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.742+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f525415bfb0 con 0x7f52541f3ee0 2024-09-16T00:10:36.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.744+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f525415bfb0 con 0x7f52541f3ee0 2024-09-16T00:10:36.745 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:36.746 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:36.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.744+0000 7f525a1b7640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5254150640 0x7f5254150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.744+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f52540b08a0 con 0x7f5254150640 2024-09-16T00:10:36.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.744+0000 7f52537fe640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5254150640 0x7f5254150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.745+0000 7f52537fe640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f5254150640 0x7f5254150a20 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:36.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.745+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52540b08a0 con 0x7f5254150640 2024-09-16T00:10:36.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.760+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5254150f60 con 0x7f5254150640 2024-09-16T00:10:36.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.762+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5254150f60 con 0x7f5254150640 2024-09-16T00:10:36.763 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.762+0000 7f525a1b7640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f52541f06d0 0x7f52541f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.762+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f5254150f60 con 0x7f52541f06d0 2024-09-16T00:10:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.762+0000 7f524bfff640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f52541f06d0 0x7f52541f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.762+0000 7f524bfff640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f52541f06d0 0x7f52541f0ab0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:36.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.763+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5254150f60 con 0x7f52541f06d0 2024-09-16T00:10:36.776 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:36.776+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:36.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.777+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5254150f60 con 0x7f52541f06d0 2024-09-16T00:10:36.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.779+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5254150f60 con 0x7f52541f06d0 2024-09-16T00:10:36.780 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:36.780 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:36.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.779+0000 7f525a1b7640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f5254062f00 0x7f5254158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.779+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f5254150f60 con 0x7f5254062f00 2024-09-16T00:10:36.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.779+0000 7f5253fff640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f5254062f00 0x7f5254158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.779+0000 7f5253fff640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f5254062f00 0x7f5254158ba0 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:36.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.780+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5254150f60 con 0x7f5254062f00 2024-09-16T00:10:36.794 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.793+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f52540635f0 con 0x7f5254062f00 2024-09-16T00:10:36.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.795+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f524c0075c0 con 0x7f5254062f00 2024-09-16T00:10:36.796 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:36.797 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:36.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.795+0000 7f525a1b7640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f52541595f0 0x7f525402c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.795+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f5254150f60 con 0x7f52541595f0 2024-09-16T00:10:36.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.796+0000 7f52537fe640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f52541595f0 0x7f525402c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.796+0000 7f52537fe640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f52541595f0 0x7f525402c640 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:36.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.797+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5254150f60 con 0x7f52541595f0 2024-09-16T00:10:36.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.812+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5254150f60 con 0x7f52541595f0 2024-09-16T00:10:36.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.815+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5254150f60 con 0x7f52541595f0 2024-09-16T00:10:36.816 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:36.817 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:36.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.815+0000 7f525a1b7640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f525402d090 0x7f525402d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.815+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f525402da10 con 0x7f525402d090 2024-09-16T00:10:36.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.815+0000 7f524bfff640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f525402d090 0x7f525402d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.816+0000 7f524bfff640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f525402d090 0x7f525402d4d0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:36.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.816+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f525402da10 con 0x7f525402d090 2024-09-16T00:10:36.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.831+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f525402da10 con 0x7f525402d090 2024-09-16T00:10:36.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.833+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f525402da10 con 0x7f525402d090 2024-09-16T00:10:36.834 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:36.835 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:36.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.833+0000 7f525a1b7640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f52542071e0 0x7f52542095d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:36.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.833+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f525420c3a0 con 0x7f52542071e0 2024-09-16T00:10:36.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.833+0000 7f5253fff640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f52542071e0 0x7f52542095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:36.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.833+0000 7f5253fff640 1 --2- 172.21.15.139:0/3769996489 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f52542071e0 0x7f52542095d0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:36.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.834+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f525420c3a0 con 0x7f52542071e0 2024-09-16T00:10:36.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.848+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f5254209b90 con 0x7f52542071e0 2024-09-16T00:10:36.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.849+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5254209b90 con 0x7f52542071e0 2024-09-16T00:10:36.850 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:36.851 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:36.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.849+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=118}) -- 0x7f525420c3a0 con 0x7f52540ca760 2024-09-16T00:10:36.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.849+0000 7f525a1b7640 1 -- 172.21.15.139:0/3769996489 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f525420c730 con 0x7f52540ca760 2024-09-16T00:10:36.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:36.850+0000 7f5250ff9640 1 -- 172.21.15.139:0/3769996489 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_get_version_reply(handle=1 version=117) ==== 24+0+0 (secure 0 0 0) 0x7f523c021040 con 0x7f52540ca760 2024-09-16T00:10:36.851 INFO:teuthology.orchestra.run.smithi139.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T00:10:36.851 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:36.852 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:36.854 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T00:10:36.854 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T00:10:36.863 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:10:36.907 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:36.906+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:36.970 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:36.970 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:36.978 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:36.978+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:36.979 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:10:37.080 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:37.079+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:37.146 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:37.146 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:37.162 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:10:37.206 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:37.205+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:37.271 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:37.271 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:37.280 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:10:37.321 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:37.320+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:37.382 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:37.381+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:37.427 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:37.427 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:37.436 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:10:37.483 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:37.483+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:37.551 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:37.551 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:37.560 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:10:37.583 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:37.582+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:37.685 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:37.684+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:37.711 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:37.712 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:37.720 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:10:37.804 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:37.804+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:37.873 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:37.873 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:37.882 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:10:37.886 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:37.885+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:37.987 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:37.986+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:38.036 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:38.036 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:38.045 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:10:38.088 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:38.087+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:38.188 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:38.187+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:38.217 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:38.217 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:38.227 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:10:38.289 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:38.287+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:38.379 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:38.379 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:38.388 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:10:38.389 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:38.388+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:38.490 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:38.489+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:38.499 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:38.500 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:38.509 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:10:38.579 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T00:10:38.621 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:38.621 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:38.621 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:38.620+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:38.630 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:10:38.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.686+0000 7f7dfb2e4640 1 Processor -- start 2024-09-16T00:10:38.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.687+0000 7f7dfb2e4640 1 -- start start 2024-09-16T00:10:38.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.687+0000 7f7dfb2e4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7df4152b10 0x7f7df4070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.687+0000 7f7dfb2e4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df41590c0 0x7f7df40c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.687+0000 7f7dfb2e4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f7df40c36f0 0x7f7df4159b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.687+0000 7f7dfb2e4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7df40c54a0 con 0x7f7df4152b10 2024-09-16T00:10:38.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.687+0000 7f7dfb2e4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f7df40c51c0 con 0x7f7df40c36f0 2024-09-16T00:10:38.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.687+0000 7f7dfb2e4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7df4070090 con 0x7f7df41590c0 2024-09-16T00:10:38.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.688+0000 7f7df9080640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7df4152b10 0x7f7df4070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.688+0000 7f7df9080640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7df4152b10 0x7f7df4070de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38562/0 (socket says 172.21.15.139:38562) 2024-09-16T00:10:38.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.688+0000 7f7df9080640 1 -- 172.21.15.139:0/669170658 learned_addr learned my addr 172.21.15.139:0/669170658 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:38.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.688+0000 7f7df887f640 1 --2- 172.21.15.139:0/669170658 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df41590c0 0x7f7df40c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.688+0000 7f7df9881640 1 --2- 172.21.15.139:0/669170658 >> v2:172.21.15.164:3300/0 conn(0x7f7df40c36f0 0x7f7df4159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.688+0000 7f7df9080640 1 -- 172.21.15.139:0/669170658 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df41590c0 msgr2=0x7f7df40c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:38.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.688+0000 7f7df9080640 1 --2- 172.21.15.139:0/669170658 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df41590c0 0x7f7df40c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.688+0000 7f7df9080640 1 -- 172.21.15.139:0/669170658 >> v2:172.21.15.164:3300/0 conn(0x7f7df40c36f0 msgr2=0x7f7df4159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:38.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.688+0000 7f7df9080640 1 --2- 172.21.15.139:0/669170658 >> v2:172.21.15.164:3300/0 conn(0x7f7df40c36f0 0x7f7df4159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.688+0000 7f7df9080640 1 -- 172.21.15.139:0/669170658 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df4070370 con 0x7f7df4152b10 2024-09-16T00:10:38.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.689+0000 7f7df9080640 1 --2- 172.21.15.139:0/669170658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7df4152b10 0x7f7df4070de0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7df40c89b0 tx=0x7f7df002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a08f23c7a8c4d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:38.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.689+0000 7f7debfff640 1 -- 172.21.15.139:0/669170658 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0037070 con 0x7f7df4152b10 2024-09-16T00:10:38.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.689+0000 7f7debfff640 1 -- 172.21.15.139:0/669170658 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7df0032440 con 0x7f7df4152b10 2024-09-16T00:10:38.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.689+0000 7f7debfff640 1 -- 172.21.15.139:0/669170658 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0032810 con 0x7f7df4152b10 2024-09-16T00:10:38.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.690+0000 7f7dfb2e4640 1 -- 172.21.15.139:0/669170658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7df4152b10 msgr2=0x7f7df4070de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:38.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.690+0000 7f7dfb2e4640 1 --2- 172.21.15.139:0/669170658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7df4152b10 0x7f7df4070de0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7df40c89b0 tx=0x7f7df002a1e0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.690+0000 7f7dfb2e4640 1 -- 172.21.15.139:0/669170658 shutdown_connections 2024-09-16T00:10:38.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.690+0000 7f7dfb2e4640 1 --2- 172.21.15.139:0/669170658 >> v2:172.21.15.164:3300/0 conn(0x7f7df40c36f0 0x7f7df4159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.690+0000 7f7dfb2e4640 1 --2- 172.21.15.139:0/669170658 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df41590c0 0x7f7df40c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.690+0000 7f7dfb2e4640 1 --2- 172.21.15.139:0/669170658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7df4152b10 0x7f7df4070de0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.690+0000 7f7dfb2e4640 1 -- 172.21.15.139:0/669170658 >> 172.21.15.139:0/669170658 conn(0x7f7df406c150 msgr2=0x7f7df4067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:38.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.690+0000 7f7dfb2e4640 1 -- 172.21.15.139:0/669170658 shutdown_connections 2024-09-16T00:10:38.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.690+0000 7f7dfb2e4640 1 -- 172.21.15.139:0/669170658 wait complete. 2024-09-16T00:10:38.693 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:38.692+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:38.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.691+0000 7f7dfb2e4640 1 Processor -- start 2024-09-16T00:10:38.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.691+0000 7f7dfb2e4640 1 -- start start 2024-09-16T00:10:38.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7dfb2e4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df40c36f0 0x7f7df4147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7dfb2e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7df4152b10 0x7f7df4147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7dfb2e4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df41590c0 0x7f7df4157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7dfb2e4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7df40ce650 con 0x7f7df4152b10 2024-09-16T00:10:38.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7dfb2e4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f7df40ce4d0 con 0x7f7df40c36f0 2024-09-16T00:10:38.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7dfb2e4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f7df40ce7d0 con 0x7f7df41590c0 2024-09-16T00:10:38.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7df887f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7df4152b10 0x7f7df4147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7df9080640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df40c36f0 0x7f7df4147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7df887f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7df4152b10 0x7f7df4147bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38564/0 (socket says 172.21.15.139:38564) 2024-09-16T00:10:38.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7df887f640 1 -- 172.21.15.139:0/3953741791 learned_addr learned my addr 172.21.15.139:0/3953741791 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:38.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7df9881640 1 --2- 172.21.15.139:0/3953741791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df41590c0 0x7f7df4157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7df887f640 1 -- 172.21.15.139:0/3953741791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df41590c0 msgr2=0x7f7df4157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:38.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7df887f640 1 --2- 172.21.15.139:0/3953741791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df41590c0 0x7f7df4157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7df887f640 1 -- 172.21.15.139:0/3953741791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df40c36f0 msgr2=0x7f7df4147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:38.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7df887f640 1 --2- 172.21.15.139:0/3953741791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df40c36f0 0x7f7df4147670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7df887f640 1 -- 172.21.15.139:0/3953741791 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df4158610 con 0x7f7df4152b10 2024-09-16T00:10:38.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7df9080640 1 --2- 172.21.15.139:0/3953741791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df40c36f0 0x7f7df4147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:38.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.692+0000 7f7df887f640 1 --2- 172.21.15.139:0/3953741791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7df4152b10 0x7f7df4147bb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7de400e840 tx=0x7f7de400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:38.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.693+0000 7f7de9ffb640 1 -- 172.21.15.139:0/3953741791 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4017070 con 0x7f7df4152b10 2024-09-16T00:10:38.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.693+0000 7f7de9ffb640 1 -- 172.21.15.139:0/3953741791 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de400c940 con 0x7f7df4152b10 2024-09-16T00:10:38.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.693+0000 7f7dfb2e4640 1 -- 172.21.15.139:0/3953741791 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df40ca540 con 0x7f7df4152b10 2024-09-16T00:10:38.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.693+0000 7f7de9ffb640 1 -- 172.21.15.139:0/3953741791 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de400cd10 con 0x7f7df4152b10 2024-09-16T00:10:38.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.693+0000 7f7dfb2e4640 1 -- 172.21.15.139:0/3953741791 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df40cab00 con 0x7f7df4152b10 2024-09-16T00:10:38.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.694+0000 7f7dfb2e4640 1 -- 172.21.15.139:0/3953741791 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df40cad00 con 0x7f7df4152b10 2024-09-16T00:10:38.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.694+0000 7f7de9ffb640 1 -- 172.21.15.139:0/3953741791 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7de401a070 con 0x7f7df4152b10 2024-09-16T00:10:38.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.695+0000 7f7de9ffb640 1 --2- 172.21.15.139:0/3953741791 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7dd806c2b0 0x7f7dd806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.695+0000 7f7de9ffb640 1 -- 172.21.15.139:0/3953741791 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5973+0+0 (secure 0 0 0) 0x7f7de408b290 con 0x7f7df4152b10 2024-09-16T00:10:38.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.695+0000 7f7df9080640 1 --2- 172.21.15.139:0/3953741791 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7dd806c2b0 0x7f7dd806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.696+0000 7f7df9080640 1 --2- 172.21.15.139:0/3953741791 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7dd806c2b0 0x7f7dd806e770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7df0034000 tx=0x7f7df002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:38.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.700+0000 7f7de9ffb640 1 -- 172.21.15.139:0/3953741791 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7de4059330 con 0x7f7df4152b10 2024-09-16T00:10:38.786 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:38.786 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:38.794 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:38.793+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:38.796 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:10:38.855 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:10:38.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.894+0000 7f7dfb2e4640 1 -- 172.21.15.139:0/3953741791 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f7df40ceac0 con 0x7f7df4152b10 2024-09-16T00:10:38.895 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:38.894+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:38.948 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:38.948 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:38.958 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:10:38.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.980+0000 7ff92aa1e640 1 Processor -- start 2024-09-16T00:10:38.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.980+0000 7ff92aa1e640 1 -- start start 2024-09-16T00:10:38.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.981+0000 7ff92aa1e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9240cad70 0x7ff9240c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.981+0000 7ff92aa1e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff9240c9a10 0x7ff92406f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.981+0000 7ff92aa1e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9240705e0 con 0x7ff9240cb150 2024-09-16T00:10:38.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.981+0000 7ff92aa1e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff9240c8a00 con 0x7ff9240c9a10 2024-09-16T00:10:38.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.981+0000 7ff92aa1e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff9240c8c50 con 0x7ff9240cad70 2024-09-16T00:10:38.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.981+0000 7ff9237fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff9240c9a10 0x7ff92406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.981+0000 7ff9237fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff9240c9a10 0x7ff92406f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38574/0 (socket says 172.21.15.139:38574) 2024-09-16T00:10:38.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.981+0000 7ff9237fe640 1 -- 172.21.15.139:0/3715158770 learned_addr learned my addr 172.21.15.139:0/3715158770 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:38.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.982+0000 7ff923fff640 1 --2- 172.21.15.139:0/3715158770 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9240cad70 0x7ff9240c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.982+0000 7ff9237fe640 1 -- 172.21.15.139:0/3715158770 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9240cad70 msgr2=0x7ff9240c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:38.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.982+0000 7ff9237fe640 1 --2- 172.21.15.139:0/3715158770 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9240cad70 0x7ff9240c5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.982+0000 7ff9237fe640 1 -- 172.21.15.139:0/3715158770 >> v1:172.21.15.139:6789/0 conn(0x7ff9240cb150 legacy=0x7ff92406fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:38.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.982+0000 7ff9237fe640 1 -- 172.21.15.139:0/3715158770 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9240c8f30 con 0x7ff9240c9a10 2024-09-16T00:10:38.988 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.982+0000 7ff922ffd640 1 -- 172.21.15.139:0/3715158770 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2975663175 0 0) 0x7ff9240705e0 con 0x7ff9240cb150 2024-09-16T00:10:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.982+0000 7ff9237fe640 1 --2- 172.21.15.139:0/3715158770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9240c9a10 0x7ff92406f850 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff9180098a0 tx=0x7ff91802a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6163b5542e1959 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.983+0000 7ff922ffd640 1 -- 172.21.15.139:0/3715158770 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918037070 con 0x7ff9240c9a10 2024-09-16T00:10:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.983+0000 7ff922ffd640 1 -- 172.21.15.139:0/3715158770 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff918032640 con 0x7ff9240c9a10 2024-09-16T00:10:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.983+0000 7ff922ffd640 1 -- 172.21.15.139:0/3715158770 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918032a10 con 0x7ff9240c9a10 2024-09-16T00:10:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.983+0000 7ff92aa1e640 1 -- 172.21.15.139:0/3715158770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9240c9a10 msgr2=0x7ff92406f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.983+0000 7ff92aa1e640 1 --2- 172.21.15.139:0/3715158770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9240c9a10 0x7ff92406f850 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff9180098a0 tx=0x7ff91802a330 comp rx=0 tx=0).stop 2024-09-16T00:10:38.989 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.983+0000 7ff92aa1e640 1 -- 172.21.15.139:0/3715158770 shutdown_connections 2024-09-16T00:10:38.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.983+0000 7ff92aa1e640 1 --2- 172.21.15.139:0/3715158770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9240c9a10 0x7ff92406f850 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.983+0000 7ff92aa1e640 1 --2- 172.21.15.139:0/3715158770 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9240cad70 0x7ff9240c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.983+0000 7ff92aa1e640 1 -- 172.21.15.139:0/3715158770 >> 172.21.15.139:0/3715158770 conn(0x7ff9240713f0 msgr2=0x7ff9240717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:38.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.983+0000 7ff92aa1e640 1 -- 172.21.15.139:0/3715158770 shutdown_connections 2024-09-16T00:10:38.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.983+0000 7ff92aa1e640 1 -- 172.21.15.139:0/3715158770 wait complete. 2024-09-16T00:10:38.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.985+0000 7ff92aa1e640 1 Processor -- start 2024-09-16T00:10:38.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.985+0000 7ff92aa1e640 1 -- start start 2024-09-16T00:10:38.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.985+0000 7ff92aa1e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9240c9a10 0x7ff9241e3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.985+0000 7ff92aa1e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9240cad70 0x7ff924154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff92aa1e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9240cb150 0x7ff9240c4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff92aa1e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff9240c5b40 con 0x7ff9240cb150 2024-09-16T00:10:38.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff92aa1e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff9240c94f0 con 0x7ff9240cad70 2024-09-16T00:10:38.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff92aa1e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff924070bb0 con 0x7ff9240c9a10 2024-09-16T00:10:38.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff923fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9240c9a10 0x7ff9241e3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff923fff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9240c9a10 0x7ff9241e3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:60212/0 (socket says 172.21.15.139:60212) 2024-09-16T00:10:38.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff923fff640 1 -- 172.21.15.139:0/166373105 learned_addr learned my addr 172.21.15.139:0/166373105 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:38.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff9237fe640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9240cad70 0x7ff924154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff928fbb640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9240cb150 0x7ff9240c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff923fff640 1 -- 172.21.15.139:0/166373105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9240cad70 msgr2=0x7ff924154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:38.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff923fff640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9240cad70 0x7ff924154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff923fff640 1 -- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9240cb150 msgr2=0x7ff9240c4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff923fff640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9240cb150 0x7ff9240c4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff923fff640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9240c28c0 con 0x7ff9240c9a10 2024-09-16T00:10:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff928fbb640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9240cb150 0x7ff9240c4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:10:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff923fff640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9240c9a10 0x7ff9241e3800 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff91400e9e0 tx=0x7ff91400eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914017070 con 0x7ff9240c9a10 2024-09-16T00:10:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff91400c9d0 con 0x7ff9240c9a10 2024-09-16T00:10:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91400cdc0 con 0x7ff9240c9a10 2024-09-16T00:10:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.986+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9240c2ba0 con 0x7ff9240c9a10 2024-09-16T00:10:38.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.987+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9240c3160 con 0x7ff9240c9a10 2024-09-16T00:10:38.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.987+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7ff9240c33a0 con 0x7ff9240c9a10 2024-09-16T00:10:38.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.988+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff91401a070 con 0x7ff9240c9a10 2024-09-16T00:10:38.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.989+0000 7ff920ff9640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff91006c2b0 0x7ff91006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.989+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(117..117 src has 1..117) ==== 5973+0+0 (secure 0 0 0) 0x7ff91408b8c0 con 0x7ff9240c9a10 2024-09-16T00:10:38.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.989+0000 7ff9237fe640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff91006c2b0 0x7ff91006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.989+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v117) ==== 54+0+16 (secure 0 0 0) 0x7ff9240c33a0 con 0x7ff9240c9a10 2024-09-16T00:10:38.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.989+0000 7ff92aa1e640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7ff924155410 0x7ff9240c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:38.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.989+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9240ca940 con 0x7ff924155410 2024-09-16T00:10:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.989+0000 7ff928fbb640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7ff924155410 0x7ff9240c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.989+0000 7ff9237fe640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff91006c2b0 0x7ff91006e770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff918034000 tx=0x7ff91802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.990+0000 7ff928fbb640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7ff924155410 0x7ff9240c4790 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:38.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:38.990+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9240ca940 con 0x7ff924155410 2024-09-16T00:10:38.996 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:38.995+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:39.006 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.005+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff9240c7730 con 0x7ff924155410 2024-09-16T00:10:39.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.007+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff9240c7730 con 0x7ff924155410 2024-09-16T00:10:39.008 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:39.009 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:39.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.007+0000 7ff92aa1e640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7ff9241e20f0 0x7ff9241e24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:39.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.007+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7ff9240c7450 con 0x7ff9241e20f0 2024-09-16T00:10:39.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.007+0000 7ff923fff640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7ff9241e20f0 0x7ff9241e24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:39.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.008+0000 7ff923fff640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7ff9241e20f0 0x7ff9241e24d0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:39.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.008+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9240c7450 con 0x7ff9241e20f0 2024-09-16T00:10:39.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.039+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff9240c7170 con 0x7ff9241e20f0 2024-09-16T00:10:39.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.040+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff9240c7170 con 0x7ff9241e20f0 2024-09-16T00:10:39.042 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:39.043 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:39.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.040+0000 7ff92aa1e640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7ff9241f20c0 0x7ff9241f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:39.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.040+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7ff9241e2b80 con 0x7ff9241f20c0 2024-09-16T00:10:39.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.041+0000 7ff9237fe640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7ff9241f20c0 0x7ff9241f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:39.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.041+0000 7ff9237fe640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7ff9241f20c0 0x7ff9241f24e0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:39.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.042+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9241e2b80 con 0x7ff9241f20c0 2024-09-16T00:10:39.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.057+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff924062900 con 0x7ff9241f20c0 2024-09-16T00:10:39.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.058+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff924062900 con 0x7ff9241f20c0 2024-09-16T00:10:39.060 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:39.061 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:39.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.059+0000 7ff92aa1e640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7ff9240b1300 0x7ff9240b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:39.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.059+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7ff9240633e0 con 0x7ff9240b1300 2024-09-16T00:10:39.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.059+0000 7ff928fbb640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7ff9240b1300 0x7ff9240b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:39.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.059+0000 7ff928fbb640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7ff9240b1300 0x7ff9240b1740 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:39.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.060+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9240633e0 con 0x7ff9240b1300 2024-09-16T00:10:39.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.075+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff9240b1f50 con 0x7ff9240b1300 2024-09-16T00:10:39.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.078+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff9240b1f50 con 0x7ff9240b1300 2024-09-16T00:10:39.079 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:39.080 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:39.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.078+0000 7ff92aa1e640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7ff92402cb50 0x7ff92402cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:39.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.078+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7ff9241e2d30 con 0x7ff92402cb50 2024-09-16T00:10:39.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.078+0000 7ff923fff640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7ff92402cb50 0x7ff92402cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:39.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.079+0000 7ff923fff640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7ff92402cb50 0x7ff92402cfb0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:39.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.079+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff914091020 con 0x7ff92402cb50 2024-09-16T00:10:39.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.094+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff9240635c0 con 0x7ff92402cb50 2024-09-16T00:10:39.096 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:39.095+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:39.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.096+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff9240635c0 con 0x7ff92402cb50 2024-09-16T00:10:39.097 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:39.098 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:39.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.096+0000 7ff92aa1e640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7ff9240b2040 0x7ff924204f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:39.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.096+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7ff9240635c0 con 0x7ff9240b2040 2024-09-16T00:10:39.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.097+0000 7ff9237fe640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7ff9240b2040 0x7ff924204f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:39.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.097+0000 7ff9237fe640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7ff9240b2040 0x7ff924204f40 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:39.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.098+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9240635c0 con 0x7ff9240b2040 2024-09-16T00:10:39.103 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:39.103 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:39.103 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:39.103 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:39.103 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:39.113 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:10:39.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.113+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff9240635c0 con 0x7ff9240b2040 2024-09-16T00:10:39.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.115+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff9240635c0 con 0x7ff9240b2040 2024-09-16T00:10:39.116 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:39.117 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:39.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.115+0000 7ff92aa1e640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7ff924205990 0x7ff924207d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:39.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.115+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7ff92402db10 con 0x7ff924205990 2024-09-16T00:10:39.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.115+0000 7ff928fbb640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7ff924205990 0x7ff924207d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:39.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.116+0000 7ff928fbb640 1 --2- 172.21.15.139:0/166373105 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7ff924205990 0x7ff924207d80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:39.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.117+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff92402db10 con 0x7ff924205990 2024-09-16T00:10:39.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.131+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff9242082c0 con 0x7ff924205990 2024-09-16T00:10:39.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.134+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff9242082c0 con 0x7ff924205990 2024-09-16T00:10:39.135 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:39.136 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:39.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.135+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=118}) -- 0x7ff92402dae0 con 0x7ff9240c9a10 2024-09-16T00:10:39.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.135+0000 7ff92aa1e640 1 -- 172.21.15.139:0/166373105 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff92420ad00 con 0x7ff9240c9a10 2024-09-16T00:10:39.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.137+0000 7ff920ff9640 1 -- 172.21.15.139:0/166373105 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_get_version_reply(handle=1 version=117) ==== 24+0+0 (secure 0 0 0) 0x7ff914021040 con 0x7ff9240c9a10 2024-09-16T00:10:39.138 INFO:teuthology.orchestra.run.smithi139.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T00:10:39.144 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T00:10:39.145 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T00:10:39.197 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:39.196+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:39.242 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:39.243 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:39.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:39.243 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:39.243 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.243 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:90 2.1b 2:ddb4e4d7:::200.00000002:head [write 2593031~1404 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:10:39.243 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:39.243 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 95.591574460999993, 2024-09-16T00:10:39.243 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011882886, 2024-09-16T00:10:39.244 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.244 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.244 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.244 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.244 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.244 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.244 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:39.244 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.244 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:39.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649119+0000", 2024-09-16T00:10:39.245 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:10:39.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.245 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.245 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.245 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649125+0000", 2024-09-16T00:10:39.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1650000000000003e-06 2024-09-16T00:10:39.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649134+0000", 2024-09-16T00:10:39.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5669999999999995e-06 2024-09-16T00:10:39.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.246 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649141+0000", 2024-09-16T00:10:39.246 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9040000000000003e-06 2024-09-16T00:10:39.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.246 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.246 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655796+0000", 2024-09-16T00:10:39.247 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066553649999999999 2024-09-16T00:10:39.247 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.247 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.247 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.247 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655868+0000", 2024-09-16T00:10:39.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2219999999999996e-05 2024-09-16T00:10:39.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:39.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655986+0000", 2024-09-16T00:10:39.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117457 2024-09-16T00:10:39.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.248 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.248 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.248 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657794+0000", 2024-09-16T00:10:39.248 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001807894 2024-09-16T00:10:39.248 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:39.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660890+0000", 2024-09-16T00:10:39.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030958449999999998 2024-09-16T00:10:39.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660966+0000", 2024-09-16T00:10:39.249 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6382000000000002e-05 2024-09-16T00:10:39.249 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.249 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.249 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.249 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.661001+0000", 2024-09-16T00:10:39.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5349000000000003e-05 2024-09-16T00:10:39.250 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.250 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.250 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.250 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.250 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.250 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10851 3.29 3:9786b8f5:::10000003100.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:39.250 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:39.250 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.347910417000001, 2024-09-16T00:10:39.250 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012109089999999999, 2024-09-16T00:10:39.250 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.250 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.250 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.250 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:39.251 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:39.251 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.251 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10851 2024-09-16T00:10:39.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.251 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:39.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.251 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.251 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.251 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.251 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:39.251 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892786+0000", 2024-09-16T00:10:39.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4819999999999999e-06 2024-09-16T00:10:39.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.252 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894302+0000", 2024-09-16T00:10:39.252 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015156099999999999 2024-09-16T00:10:39.252 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.252 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.252 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894319+0000", 2024-09-16T00:10:39.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7098000000000001e-05 2024-09-16T00:10:39.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894329+0000", 2024-09-16T00:10:39.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9180000000000006e-06 2024-09-16T00:10:39.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.253 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.253 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.253 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894394+0000", 2024-09-16T00:10:39.253 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5797999999999996e-05 2024-09-16T00:10:39.253 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894532+0000", 2024-09-16T00:10:39.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000138128 2024-09-16T00:10:39.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:39.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894633+0000", 2024-09-16T00:10:39.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010037100000000001 2024-09-16T00:10:39.254 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.254 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.254 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:39.254 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.899212+0000", 2024-09-16T00:10:39.254 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045789369999999999 2024-09-16T00:10:39.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904771+0000", 2024-09-16T00:10:39.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055591169999999997 2024-09-16T00:10:39.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904838+0000", 2024-09-16T00:10:39.255 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7124000000000003e-05 2024-09-16T00:10:39.255 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.255 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.255 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.255 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904892+0000", 2024-09-16T00:10:39.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3507000000000001e-05 2024-09-16T00:10:39.256 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.256 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.256 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.256 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.256 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.256 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10957 3.24 3:2748b9d7:::1000000313b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:39.256 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:39.256 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 59.726006064000003, 2024-09-16T00:10:39.256 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012163525, 2024-09-16T00:10:39.256 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.256 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.256 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.257 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:39.257 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:39.257 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.257 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10957 2024-09-16T00:10:39.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.257 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:39.257 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.257 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.257 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.257 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.257 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:39.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514690+0000", 2024-09-16T00:10:39.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.382e-06 2024-09-16T00:10:39.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.258 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.258 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.258 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516204+0000", 2024-09-16T00:10:39.258 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001513831 2024-09-16T00:10:39.258 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516232+0000", 2024-09-16T00:10:39.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7657999999999998e-05 2024-09-16T00:10:39.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516246+0000", 2024-09-16T00:10:39.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3953e-05 2024-09-16T00:10:39.259 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.259 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.259 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.259 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516374+0000", 2024-09-16T00:10:39.259 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012882 2024-09-16T00:10:39.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516588+0000", 2024-09-16T00:10:39.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000213439 2024-09-16T00:10:39.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:39.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516748+0000", 2024-09-16T00:10:39.260 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015971900000000001 2024-09-16T00:10:39.260 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.260 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.260 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:39.260 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.524102+0000", 2024-09-16T00:10:39.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073542499999999997 2024-09-16T00:10:39.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526714+0000", 2024-09-16T00:10:39.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026122910000000001 2024-09-16T00:10:39.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.261 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.261 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526791+0000", 2024-09-16T00:10:39.261 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6997999999999997e-05 2024-09-16T00:10:39.261 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.261 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.262 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.262 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526850+0000", 2024-09-16T00:10:39.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9184000000000001e-05 2024-09-16T00:10:39.262 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.262 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.262 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.262 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.262 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.262 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11098 3.15 e86/20 3:aa8f498f:::100000030ba.00000000:head v 86'170, mlcod=86'168)", 2024-09-16T00:10:39.262 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:39.262 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 59.219222897999998, 2024-09-16T00:10:39.262 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013832641999999999, 2024-09-16T00:10:39.262 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.263 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.263 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.263 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:39.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.263 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.263 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.263 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:39.263 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.263 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021472+0000", 2024-09-16T00:10:39.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9489999999999999e-06 2024-09-16T00:10:39.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027628+0000", 2024-09-16T00:10:39.264 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061563859999999998 2024-09-16T00:10:39.264 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.264 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.264 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.264 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027645+0000", 2024-09-16T00:10:39.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7108e-05 2024-09-16T00:10:39.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027655+0000", 2024-09-16T00:10:39.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7999999999999993e-06 2024-09-16T00:10:39.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.265 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027712+0000", 2024-09-16T00:10:39.265 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6826000000000001e-05 2024-09-16T00:10:39.265 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.265 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.265 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027738+0000", 2024-09-16T00:10:39.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5601000000000001e-05 2024-09-16T00:10:39.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.035237+0000", 2024-09-16T00:10:39.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074998699999999996 2024-09-16T00:10:39.266 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.266 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.266 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.266 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.035303+0000", 2024-09-16T00:10:39.266 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5102000000000001e-05 2024-09-16T00:10:39.266 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.267 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.267 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.267 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.267 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.267 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11099 3.b 3:d2f5b389:::100000030e7.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:39.267 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:39.267 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 59.215178082999998, 2024-09-16T00:10:39.267 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012707456000000001, 2024-09-16T00:10:39.267 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.267 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.267 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.267 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:39.267 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:39.268 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.268 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11099 2024-09-16T00:10:39.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.268 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:39.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.268 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.268 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.268 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.268 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:39.268 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025517+0000", 2024-09-16T00:10:39.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:10:39.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.269 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026955+0000", 2024-09-16T00:10:39.269 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001437219 2024-09-16T00:10:39.269 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.269 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.269 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026982+0000", 2024-09-16T00:10:39.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7523999999999999e-05 2024-09-16T00:10:39.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026996+0000", 2024-09-16T00:10:39.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4379000000000001e-05 2024-09-16T00:10:39.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.270 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.270 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027076+0000", 2024-09-16T00:10:39.270 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9933999999999997e-05 2024-09-16T00:10:39.270 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.270 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027185+0000", 2024-09-16T00:10:39.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010875899999999999 2024-09-16T00:10:39.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:39.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027380+0000", 2024-09-16T00:10:39.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019477700000000001 2024-09-16T00:10:39.271 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.271 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.271 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.271 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038072+0000", 2024-09-16T00:10:39.271 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010692234 2024-09-16T00:10:39.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:39.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038115+0000", 2024-09-16T00:10:39.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3291999999999998e-05 2024-09-16T00:10:39.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.272 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.272 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038172+0000", 2024-09-16T00:10:39.272 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6981000000000002e-05 2024-09-16T00:10:39.272 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.272 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.273 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.273 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038222+0000", 2024-09-16T00:10:39.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9737000000000002e-05 2024-09-16T00:10:39.273 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.273 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.273 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.273 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.273 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.273 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11109 3.27 3:e5179c63:::10000003176.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:39.273 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:39.273 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 59.169713799999997, 2024-09-16T00:10:39.273 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012250990999999999, 2024-09-16T00:10:39.273 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.274 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.274 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.274 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:39.274 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:39.274 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.274 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11109 2024-09-16T00:10:39.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.274 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.274 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.274 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:39.274 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.274 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.274 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:39.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070981+0000", 2024-09-16T00:10:39.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.289e-06 2024-09-16T00:10:39.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.275 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.275 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.275 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072616+0000", 2024-09-16T00:10:39.275 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016347569999999999 2024-09-16T00:10:39.275 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.276 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.276 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072638+0000", 2024-09-16T00:10:39.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1512999999999999e-05 2024-09-16T00:10:39.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.276 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.276 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072649+0000", 2024-09-16T00:10:39.276 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1626e-05 2024-09-16T00:10:39.276 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.276 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.276 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.276 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072775+0000", 2024-09-16T00:10:39.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012553900000000001 2024-09-16T00:10:39.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072856+0000", 2024-09-16T00:10:39.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0882000000000003e-05 2024-09-16T00:10:39.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:39.277 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072990+0000", 2024-09-16T00:10:39.277 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013483599999999999 2024-09-16T00:10:39.277 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.277 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.277 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:39.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.078354+0000", 2024-09-16T00:10:39.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053639500000000001 2024-09-16T00:10:39.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083150+0000", 2024-09-16T00:10:39.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004795517 2024-09-16T00:10:39.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.278 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.278 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083186+0000", 2024-09-16T00:10:39.278 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6019000000000002e-05 2024-09-16T00:10:39.278 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.278 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.280 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.280 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083230+0000", 2024-09-16T00:10:39.280 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4063000000000001e-05 2024-09-16T00:10:39.281 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.281 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.281 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.281 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.281 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.281 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11234 3.b 3:d2f5b389:::100000030e7.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:39.281 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:39.281 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.617098032000001, 2024-09-16T00:10:39.281 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.015217661, 2024-09-16T00:10:39.281 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.281 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.281 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.281 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:39.281 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:39.282 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.282 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11234 2024-09-16T00:10:39.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.282 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.282 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.282 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.282 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:39.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.282 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.282 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.282 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:39.282 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.282 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623598+0000", 2024-09-16T00:10:39.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1580000000000001e-06 2024-09-16T00:10:39.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626844+0000", 2024-09-16T00:10:39.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032459670000000002 2024-09-16T00:10:39.283 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.283 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.283 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.283 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626868+0000", 2024-09-16T00:10:39.283 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3584e-05 2024-09-16T00:10:39.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626879+0000", 2024-09-16T00:10:39.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1545999999999999e-05 2024-09-16T00:10:39.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.284 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626950+0000", 2024-09-16T00:10:39.284 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0784999999999994e-05 2024-09-16T00:10:39.284 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.284 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.284 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.627051+0000", 2024-09-16T00:10:39.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010155899999999999 2024-09-16T00:10:39.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:39.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.627178+0000", 2024-09-16T00:10:39.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012614899999999999 2024-09-16T00:10:39.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.285 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:39.285 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.635220+0000", 2024-09-16T00:10:39.285 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0080421490000000002 2024-09-16T00:10:39.285 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.285 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638699+0000", 2024-09-16T00:10:39.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034797500000000002 2024-09-16T00:10:39.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638760+0000", 2024-09-16T00:10:39.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0319000000000003e-05 2024-09-16T00:10:39.286 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.286 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.286 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.286 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638812+0000", 2024-09-16T00:10:39.286 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:10:39.286 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.287 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.287 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.287 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.287 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.287 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11777 3.b 3:d342b4ff:::10000003113.00000000:head [write 0~4194304 [2@1171070]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:39.287 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:39.287 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 56.049458624000003, 2024-09-16T00:10:39.287 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013079176, 2024-09-16T00:10:39.287 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.287 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.287 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.287 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:39.287 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:39.287 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.288 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11777 2024-09-16T00:10:39.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.288 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.288 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:39.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.288 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.288 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.288 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:39.288 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.288 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191237+0000", 2024-09-16T00:10:39.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3369999999999998e-06 2024-09-16T00:10:39.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192825+0000", 2024-09-16T00:10:39.289 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015887189999999999 2024-09-16T00:10:39.289 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.289 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.289 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.289 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192846+0000", 2024-09-16T00:10:39.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0670999999999999e-05 2024-09-16T00:10:39.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192858+0000", 2024-09-16T00:10:39.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.223e-05 2024-09-16T00:10:39.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.290 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192957+0000", 2024-09-16T00:10:39.290 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8416000000000001e-05 2024-09-16T00:10:39.290 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.290 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.290 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.193031+0000", 2024-09-16T00:10:39.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4381999999999994e-05 2024-09-16T00:10:39.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:39.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.193164+0000", 2024-09-16T00:10:39.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013349100000000001 2024-09-16T00:10:39.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.291 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:39.291 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204144+0000", 2024-09-16T00:10:39.291 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010979917 2024-09-16T00:10:39.291 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.291 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204204+0000", 2024-09-16T00:10:39.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0025999999999998e-05 2024-09-16T00:10:39.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204265+0000", 2024-09-16T00:10:39.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0645e-05 2024-09-16T00:10:39.292 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.292 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.292 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.292 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204313+0000", 2024-09-16T00:10:39.292 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8341999999999999e-05 2024-09-16T00:10:39.293 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.293 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.293 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.293 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.293 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11981 3.c e87/20 3:33faa5a0:::10000003215.00000000:head v 87'186, mlcod=87'184)", 2024-09-16T00:10:39.293 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:39.293 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.409223801000003, 2024-09-16T00:10:39.293 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011828697000000001, 2024-09-16T00:10:39.293 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.293 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.293 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.293 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.293 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:39.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:39.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.294 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.294 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.294 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831471+0000", 2024-09-16T00:10:39.294 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6220000000000001e-06 2024-09-16T00:10:39.294 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836584+0000", 2024-09-16T00:10:39.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051133510000000004 2024-09-16T00:10:39.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836606+0000", 2024-09-16T00:10:39.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2079e-05 2024-09-16T00:10:39.295 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.295 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.295 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.295 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836624+0000", 2024-09-16T00:10:39.295 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8162e-05 2024-09-16T00:10:39.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836704+0000", 2024-09-16T00:10:39.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9867e-05 2024-09-16T00:10:39.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836730+0000", 2024-09-16T00:10:39.296 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6285999999999999e-05 2024-09-16T00:10:39.296 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.296 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.296 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.296 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.843247+0000", 2024-09-16T00:10:39.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0065165650000000002 2024-09-16T00:10:39.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.297 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.297 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.843298+0000", 2024-09-16T00:10:39.297 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0764999999999999e-05 2024-09-16T00:10:39.297 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.297 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.297 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.297 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.297 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.297 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13365 3.2 3:4298088d:::100000035be.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:39.297 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:39.298 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.221521822, 2024-09-16T00:10:39.298 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014036442999999999, 2024-09-16T00:10:39.298 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.298 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.298 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.298 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:39.298 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:39.298 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.298 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13365 2024-09-16T00:10:39.298 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.298 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.298 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.298 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.298 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:39.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:39.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.299 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019174+0000", 2024-09-16T00:10:39.299 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1250000000000001e-06 2024-09-16T00:10:39.299 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.299 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.299 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.300 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020613+0000", 2024-09-16T00:10:39.300 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014391930000000001 2024-09-16T00:10:39.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.300 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.300 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020638+0000", 2024-09-16T00:10:39.300 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4657000000000001e-05 2024-09-16T00:10:39.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.300 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.300 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.300 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020651+0000", 2024-09-16T00:10:39.300 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3451e-05 2024-09-16T00:10:39.300 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020744+0000", 2024-09-16T00:10:39.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3010000000000006e-05 2024-09-16T00:10:39.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020912+0000", 2024-09-16T00:10:39.301 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016763800000000001 2024-09-16T00:10:39.301 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.301 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.301 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:39.301 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.021043+0000", 2024-09-16T00:10:39.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013099800000000001 2024-09-16T00:10:39.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:39.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026194+0000", 2024-09-16T00:10:39.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051510879999999998 2024-09-16T00:10:39.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.302 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.302 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033132+0000", 2024-09-16T00:10:39.302 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069377279999999998 2024-09-16T00:10:39.302 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.302 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.303 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033169+0000", 2024-09-16T00:10:39.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6897000000000003e-05 2024-09-16T00:10:39.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.303 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.303 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033207+0000", 2024-09-16T00:10:39.303 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658000000000002e-05 2024-09-16T00:10:39.303 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.303 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.303 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.303 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.303 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.304 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13368 3.27 3:e75b72a1:::100000032e8.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:39.304 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:39.304 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.214155939000001, 2024-09-16T00:10:39.304 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013615175, 2024-09-16T00:10:39.304 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.304 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.304 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.304 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:39.304 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:39.304 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.304 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13368 2024-09-16T00:10:39.304 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.304 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.304 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:39.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:39.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.305 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.305 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.305 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026539+0000", 2024-09-16T00:10:39.305 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.798e-06 2024-09-16T00:10:39.305 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.306 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.306 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027825+0000", 2024-09-16T00:10:39.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012861839999999999 2024-09-16T00:10:39.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.306 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.306 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027850+0000", 2024-09-16T00:10:39.306 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5048000000000001e-05 2024-09-16T00:10:39.306 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.306 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.306 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.306 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027865+0000", 2024-09-16T00:10:39.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4939e-05 2024-09-16T00:10:39.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027944+0000", 2024-09-16T00:10:39.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8881999999999995e-05 2024-09-16T00:10:39.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.307 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.307 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.307 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.028043+0000", 2024-09-16T00:10:39.307 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9135000000000005e-05 2024-09-16T00:10:39.307 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:39.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.028158+0000", 2024-09-16T00:10:39.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011527999999999999 2024-09-16T00:10:39.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.308 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.039292+0000", 2024-09-16T00:10:39.308 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011133568999999999 2024-09-16T00:10:39.308 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.308 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.308 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:39.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.039948+0000", 2024-09-16T00:10:39.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00065625700000000004 2024-09-16T00:10:39.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.309 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.040061+0000", 2024-09-16T00:10:39.309 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011284400000000001 2024-09-16T00:10:39.309 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.309 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.309 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.310 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.040152+0000", 2024-09-16T00:10:39.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1238999999999998e-05 2024-09-16T00:10:39.310 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.310 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.310 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.310 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.310 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.310 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:13366 3.15 e88/20 3:aa8f498f:::100000030ba.00000000:head v 88'196, mlcod=88'194)", 2024-09-16T00:10:39.310 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:39.310 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.210730863999999, 2024-09-16T00:10:39.310 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012822817, 2024-09-16T00:10:39.310 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.310 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.311 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.311 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:39.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.311 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:39.311 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.311 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.311 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.311 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029963+0000", 2024-09-16T00:10:39.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4270000000000001e-06 2024-09-16T00:10:39.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034078+0000", 2024-09-16T00:10:39.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041148360000000002 2024-09-16T00:10:39.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.312 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.312 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.312 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034094+0000", 2024-09-16T00:10:39.312 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.535e-05 2024-09-16T00:10:39.312 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034103+0000", 2024-09-16T00:10:39.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:10:39.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.036791+0000", 2024-09-16T00:10:39.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026878340000000001 2024-09-16T00:10:39.313 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.313 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.313 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.313 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.036818+0000", 2024-09-16T00:10:39.313 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6885000000000001e-05 2024-09-16T00:10:39.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.042713+0000", 2024-09-16T00:10:39.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058950399999999998 2024-09-16T00:10:39.314 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.314 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.314 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.314 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.042785+0000", 2024-09-16T00:10:39.314 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1927000000000004e-05 2024-09-16T00:10:39.314 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.314 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.314 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.315 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.315 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13405 3.27 3:e708eda4:::100000034c4.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:39.315 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:39.315 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.084529150999998, 2024-09-16T00:10:39.315 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012003333, 2024-09-16T00:10:39.315 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.315 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.315 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.315 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:39.315 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:39.315 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.315 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13405 2024-09-16T00:10:39.315 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.316 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:39.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:39.316 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.316 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.316 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.316 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.316 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156166+0000", 2024-09-16T00:10:39.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6230000000000001e-06 2024-09-16T00:10:39.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157868+0000", 2024-09-16T00:10:39.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017017 2024-09-16T00:10:39.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.317 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.317 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157891+0000", 2024-09-16T00:10:39.317 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2931000000000001e-05 2024-09-16T00:10:39.317 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.317 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157904+0000", 2024-09-16T00:10:39.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2763e-05 2024-09-16T00:10:39.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157988+0000", 2024-09-16T00:10:39.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3849000000000003e-05 2024-09-16T00:10:39.318 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.318 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.318 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.158083+0000", 2024-09-16T00:10:39.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5939000000000004e-05 2024-09-16T00:10:39.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:39.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.158194+0000", 2024-09-16T00:10:39.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110963 2024-09-16T00:10:39.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:39.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.164635+0000", 2024-09-16T00:10:39.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0064405570000000004 2024-09-16T00:10:39.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168088+0000", 2024-09-16T00:10:39.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034534639999999998 2024-09-16T00:10:39.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168140+0000", 2024-09-16T00:10:39.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1041000000000001e-05 2024-09-16T00:10:39.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168167+0000", 2024-09-16T00:10:39.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7503000000000001e-05 2024-09-16T00:10:39.321 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.321 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.321 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.321 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13685 3.b 3:d26c1580:::100000032e4.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:39.321 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:39.321 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 49.905842186000001, 2024-09-16T00:10:39.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014868124999999999, 2024-09-16T00:10:39.321 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.321 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.322 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.322 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:39.322 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:39.322 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.322 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13685 2024-09-16T00:10:39.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.322 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:39.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:39.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334854+0000", 2024-09-16T00:10:39.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0000000000000001e-06 2024-09-16T00:10:39.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337155+0000", 2024-09-16T00:10:39.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023016870000000002 2024-09-16T00:10:39.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337178+0000", 2024-09-16T00:10:39.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2898e-05 2024-09-16T00:10:39.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337190+0000", 2024-09-16T00:10:39.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1817999999999999e-05 2024-09-16T00:10:39.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337301+0000", 2024-09-16T00:10:39.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110951 2024-09-16T00:10:39.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337403+0000", 2024-09-16T00:10:39.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000102118 2024-09-16T00:10:39.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:39.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337545+0000", 2024-09-16T00:10:39.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014167199999999999 2024-09-16T00:10:39.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:39.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.346336+0000", 2024-09-16T00:10:39.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087911450000000002 2024-09-16T00:10:39.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349641+0000", 2024-09-16T00:10:39.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033047570000000002 2024-09-16T00:10:39.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349680+0000", 2024-09-16T00:10:39.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9054999999999998e-05 2024-09-16T00:10:39.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349719+0000", 2024-09-16T00:10:39.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9023999999999998e-05 2024-09-16T00:10:39.327 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.327 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.327 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.327 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14624 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:39.327 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:39.327 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 46.626082984999996, 2024-09-16T00:10:39.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012123376, 2024-09-16T00:10:39.327 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.327 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.328 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.328 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:39.328 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:39.328 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.328 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14624 2024-09-16T00:10:39.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.328 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:39.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:39.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614614+0000", 2024-09-16T00:10:39.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9500000000000003e-06 2024-09-16T00:10:39.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616480+0000", 2024-09-16T00:10:39.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018663169999999999 2024-09-16T00:10:39.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616506+0000", 2024-09-16T00:10:39.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6205999999999999e-05 2024-09-16T00:10:39.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616517+0000", 2024-09-16T00:10:39.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0399e-05 2024-09-16T00:10:39.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616595+0000", 2024-09-16T00:10:39.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8170999999999994e-05 2024-09-16T00:10:39.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616683+0000", 2024-09-16T00:10:39.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8578999999999993e-05 2024-09-16T00:10:39.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:39.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616825+0000", 2024-09-16T00:10:39.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014186299999999999 2024-09-16T00:10:39.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:39.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.624948+0000", 2024-09-16T00:10:39.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081230420000000005 2024-09-16T00:10:39.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626617+0000", 2024-09-16T00:10:39.332 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016681980000000001 2024-09-16T00:10:39.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.332 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.332 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626678+0000", 2024-09-16T00:10:39.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1362e-05 2024-09-16T00:10:39.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.333 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.333 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626733+0000", 2024-09-16T00:10:39.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5288999999999999e-05 2024-09-16T00:10:39.333 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.333 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.333 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.333 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.333 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.333 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14672 3.29 3:9641a85f:::1000000360e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:39.333 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:39.333 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 46.421625808000002, 2024-09-16T00:10:39.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012258025000000001, 2024-09-16T00:10:39.334 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.334 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.334 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.334 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:39.334 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:39.334 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.334 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14672 2024-09-16T00:10:39.334 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.334 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:39.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:39.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819070+0000", 2024-09-16T00:10:39.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0589999999999998e-06 2024-09-16T00:10:39.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821701+0000", 2024-09-16T00:10:39.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026309100000000002 2024-09-16T00:10:39.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821727+0000", 2024-09-16T00:10:39.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6372000000000001e-05 2024-09-16T00:10:39.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821738+0000", 2024-09-16T00:10:39.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0759e-05 2024-09-16T00:10:39.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821821+0000", 2024-09-16T00:10:39.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2854000000000003e-05 2024-09-16T00:10:39.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821920+0000", 2024-09-16T00:10:39.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8900999999999994e-05 2024-09-16T00:10:39.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:39.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.822084+0000", 2024-09-16T00:10:39.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016432799999999999 2024-09-16T00:10:39.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:39.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.829148+0000", 2024-09-16T00:10:39.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0070641460000000003 2024-09-16T00:10:39.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831242+0000", 2024-09-16T00:10:39.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002093293 2024-09-16T00:10:39.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831280+0000", 2024-09-16T00:10:39.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8707000000000001e-05 2024-09-16T00:10:39.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831325+0000", 2024-09-16T00:10:39.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4696e-05 2024-09-16T00:10:39.339 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.339 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.339 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.339 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14829 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304 [2@2556691]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:39.339 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:39.339 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.669106390000003, 2024-09-16T00:10:39.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014399683, 2024-09-16T00:10:39.340 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.340 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.340 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.340 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:39.340 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:39.340 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.340 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14829 2024-09-16T00:10:39.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.340 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.340 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:39.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:39.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571590+0000", 2024-09-16T00:10:39.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:10:39.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573133+0000", 2024-09-16T00:10:39.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015422109999999999 2024-09-16T00:10:39.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573161+0000", 2024-09-16T00:10:39.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8045e-05 2024-09-16T00:10:39.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573174+0000", 2024-09-16T00:10:39.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3193999999999999e-05 2024-09-16T00:10:39.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573365+0000", 2024-09-16T00:10:39.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019096400000000001 2024-09-16T00:10:39.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573467+0000", 2024-09-16T00:10:39.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101789 2024-09-16T00:10:39.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:39.344 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573610+0000", 2024-09-16T00:10:39.344 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143404 2024-09-16T00:10:39.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.344 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.344 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.344 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.583614+0000", 2024-09-16T00:10:39.344 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010004422000000001 2024-09-16T00:10:39.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.344 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:39.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585556+0000", 2024-09-16T00:10:39.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019414009999999999 2024-09-16T00:10:39.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585939+0000", 2024-09-16T00:10:39.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038340999999999999 2024-09-16T00:10:39.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585986+0000", 2024-09-16T00:10:39.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6924e-05 2024-09-16T00:10:39.345 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.346 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.346 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.346 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14914 3.b 3:d23f6103:::10000003277.00000000:head [write 0~4194304 [3@58131]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:39.346 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:39.346 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 45.357902619000001, 2024-09-16T00:10:39.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013007654, 2024-09-16T00:10:39.346 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.346 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.346 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.346 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:39.346 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:39.346 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.347 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14914 2024-09-16T00:10:39.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.347 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:39.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:39.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882793+0000", 2024-09-16T00:10:39.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.65e-06 2024-09-16T00:10:39.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885193+0000", 2024-09-16T00:10:39.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024003449999999999 2024-09-16T00:10:39.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885214+0000", 2024-09-16T00:10:39.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0398e-05 2024-09-16T00:10:39.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885224+0000", 2024-09-16T00:10:39.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0253e-05 2024-09-16T00:10:39.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885299+0000", 2024-09-16T00:10:39.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5434999999999999e-05 2024-09-16T00:10:39.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.350 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885389+0000", 2024-09-16T00:10:39.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9209999999999995e-05 2024-09-16T00:10:39.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.350 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:39.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885485+0000", 2024-09-16T00:10:39.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6218999999999995e-05 2024-09-16T00:10:39.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.351 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.351 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.893457+0000", 2024-09-16T00:10:39.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0079722009999999999 2024-09-16T00:10:39.351 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.351 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.351 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:39.351 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895681+0000", 2024-09-16T00:10:39.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022241449999999999 2024-09-16T00:10:39.351 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.351 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.351 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.351 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895732+0000", 2024-09-16T00:10:39.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1269e-05 2024-09-16T00:10:39.351 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.352 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.352 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895798+0000", 2024-09-16T00:10:39.352 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5529000000000002e-05 2024-09-16T00:10:39.352 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.352 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.352 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.352 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.352 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15058 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304 [2@2556691]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-16T00:10:39.352 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:39.352 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 44.905534025000001, 2024-09-16T00:10:39.352 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012275842, 2024-09-16T00:10:39.352 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.353 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.353 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.353 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:39.353 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:39.353 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.353 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15058 2024-09-16T00:10:39.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.353 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:39.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:39.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335161+0000", 2024-09-16T00:10:39.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.367e-06 2024-09-16T00:10:39.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337545+0000", 2024-09-16T00:10:39.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023839040000000001 2024-09-16T00:10:39.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337572+0000", 2024-09-16T00:10:39.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.673e-05 2024-09-16T00:10:39.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337583+0000", 2024-09-16T00:10:39.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1048e-05 2024-09-16T00:10:39.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337660+0000", 2024-09-16T00:10:39.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7459999999999994e-05 2024-09-16T00:10:39.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:39.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337770+0000", 2024-09-16T00:10:39.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109634 2024-09-16T00:10:39.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:39.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337926+0000", 2024-09-16T00:10:39.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015601999999999999 2024-09-16T00:10:39.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.345689+0000", 2024-09-16T00:10:39.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0077625630000000001 2024-09-16T00:10:39.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:39.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347324+0000", 2024-09-16T00:10:39.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016351479999999999 2024-09-16T00:10:39.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:39.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347387+0000", 2024-09-16T00:10:39.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3199000000000003e-05 2024-09-16T00:10:39.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.358 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.358 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347435+0000", 2024-09-16T00:10:39.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7769000000000003e-05 2024-09-16T00:10:39.358 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.358 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.358 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.358 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9780 2.a 2:545c15c7:::200.00000017:head [write 1367106~16252 [fadvise_dontneed]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e106)", 2024-09-16T00:10:39.358 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:39.358 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 22.903349900999999, 2024-09-16T00:10:39.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0208196810000001, 2024-09-16T00:10:39.359 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:39.359 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:39.359 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:10:39.359 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:39.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:39.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:39.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:39.359 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:39.359 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:39.359 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:39.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337345+0000", 2024-09-16T00:10:39.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.007e-06 2024-09-16T00:10:39.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:39.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337416+0000", 2024-09-16T00:10:39.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1260999999999994e-05 2024-09-16T00:10:39.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:39.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337427+0000", 2024-09-16T00:10:39.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1225e-05 2024-09-16T00:10:39.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337434+0000", 2024-09-16T00:10:39.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8929999999999999e-06 2024-09-16T00:10:39.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337494+0000", 2024-09-16T00:10:39.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9670999999999998e-05 2024-09-16T00:10:39.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:39.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337505+0000", 2024-09-16T00:10:39.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0934999999999999e-05 2024-09-16T00:10:39.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.345708+0000", 2024-09-16T00:10:39.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0082033290000001 2024-09-16T00:10:39.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for missing object", 2024-09-16T00:10:39.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.345819+0000", 2024-09-16T00:10:39.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110299 2024-09-16T00:10:39.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.346937+0000", 2024-09-16T00:10:39.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011184120000000001 2024-09-16T00:10:39.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for degraded object", 2024-09-16T00:10:39.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.346968+0000", 2024-09-16T00:10:39.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0664999999999999e-05 2024-09-16T00:10:39.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:39.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.358107+0000", 2024-09-16T00:10:39.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011139046 2024-09-16T00:10:39.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:39.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:39.364 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:39.364 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.358163+0000", 2024-09-16T00:10:39.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5937999999999999e-05 2024-09-16T00:10:39.364 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.364 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.364 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.364 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:39.364 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:39.364 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:39.365 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:39.364+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:39.366 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:10:39.467 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:39.465+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:39.472 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:39.472 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:39.473 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:39.473 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:39.482 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:10:39.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.523+0000 7f7de9ffb640 1 -- 172.21.15.139:0/3953741791 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v118) ==== 101+0+0 (secure 0 0 0) 0x7f7de405d630 con 0x7f7df4152b10 2024-09-16T00:10:39.524 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is unset 2024-09-16T00:10:39.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.525+0000 7f7dfb2e4640 1 -- 172.21.15.139:0/3953741791 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7dd806c2b0 msgr2=0x7f7dd806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:39.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.525+0000 7f7dfb2e4640 1 --2- 172.21.15.139:0/3953741791 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7dd806c2b0 0x7f7dd806e770 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7df0034000 tx=0x7f7df002e000 comp rx=0 tx=0).stop 2024-09-16T00:10:39.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.525+0000 7f7dfb2e4640 1 -- 172.21.15.139:0/3953741791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7df4152b10 msgr2=0x7f7df4147bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:39.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.525+0000 7f7dfb2e4640 1 --2- 172.21.15.139:0/3953741791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7df4152b10 0x7f7df4147bb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7de400e840 tx=0x7f7de400ed00 comp rx=0 tx=0).stop 2024-09-16T00:10:39.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.525+0000 7f7dfb2e4640 1 -- 172.21.15.139:0/3953741791 shutdown_connections 2024-09-16T00:10:39.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.525+0000 7f7dfb2e4640 1 --2- 172.21.15.139:0/3953741791 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f7dd806c2b0 0x7f7dd806e770 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:39.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.525+0000 7f7dfb2e4640 1 --2- 172.21.15.139:0/3953741791 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f7df41590c0 0x7f7df4157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:39.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.525+0000 7f7dfb2e4640 1 --2- 172.21.15.139:0/3953741791 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7df4152b10 0x7f7df4147bb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:39.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.525+0000 7f7dfb2e4640 1 --2- 172.21.15.139:0/3953741791 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f7df40c36f0 0x7f7df4147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:39.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.525+0000 7f7dfb2e4640 1 -- 172.21.15.139:0/3953741791 >> 172.21.15.139:0/3953741791 conn(0x7f7df406c150 msgr2=0x7f7df4148c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:39.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.526+0000 7f7dfb2e4640 1 -- 172.21.15.139:0/3953741791 shutdown_connections 2024-09-16T00:10:39.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:39.526+0000 7f7dfb2e4640 1 -- 172.21.15.139:0/3953741791 wait complete. 2024-09-16T00:10:39.568 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:39.567+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:39.633 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:39.633 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:39.633 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:39.633 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:39.633 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:39.642 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:10:39.668 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:39.667+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:39.753 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:39.753 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:39.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:39.753 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:39.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.754 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10780 3.a 3:500c83b2:::100000030b4.00000000:head [write 0~4194304 [3@2137521]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:39.754 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:39.754 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.298058095000002, 2024-09-16T00:10:39.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015526478, 2024-09-16T00:10:39.754 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.754 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.754 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.754 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.754 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10780 2024-09-16T00:10:39.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.754 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:39.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:39.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452917+0000", 2024-09-16T00:10:39.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2180000000000002e-06 2024-09-16T00:10:39.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462177+0000", 2024-09-16T00:10:39.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092593769999999992 2024-09-16T00:10:39.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462196+0000", 2024-09-16T00:10:39.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9745000000000002e-05 2024-09-16T00:10:39.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462207+0000", 2024-09-16T00:10:39.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0524e-05 2024-09-16T00:10:39.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462284+0000", 2024-09-16T00:10:39.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7608000000000006e-05 2024-09-16T00:10:39.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462376+0000", 2024-09-16T00:10:39.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1811999999999994e-05 2024-09-16T00:10:39.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468357+0000", 2024-09-16T00:10:39.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059806809999999998 2024-09-16T00:10:39.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468386+0000", 2024-09-16T00:10:39.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9068999999999999e-05 2024-09-16T00:10:39.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468441+0000", 2024-09-16T00:10:39.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5444e-05 2024-09-16T00:10:39.759 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.759 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.760 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.760 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10831 3.1a 3:59b8f98c:::100000030df.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:39.760 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:39.760 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.975250959, 2024-09-16T00:10:39.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014692843000000001, 2024-09-16T00:10:39.760 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.760 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.760 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.760 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.761 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.761 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10831 2024-09-16T00:10:39.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.761 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:39.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:39.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775723+0000", 2024-09-16T00:10:39.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.367e-06 2024-09-16T00:10:39.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780402+0000", 2024-09-16T00:10:39.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046785769999999997 2024-09-16T00:10:39.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780421+0000", 2024-09-16T00:10:39.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9179000000000001e-05 2024-09-16T00:10:39.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780440+0000", 2024-09-16T00:10:39.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9015e-05 2024-09-16T00:10:39.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780533+0000", 2024-09-16T00:10:39.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3197999999999998e-05 2024-09-16T00:10:39.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780618+0000", 2024-09-16T00:10:39.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4787e-05 2024-09-16T00:10:39.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:39.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780748+0000", 2024-09-16T00:10:39.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012927199999999999 2024-09-16T00:10:39.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.785049+0000", 2024-09-16T00:10:39.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043015750000000002 2024-09-16T00:10:39.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790312+0000", 2024-09-16T00:10:39.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052633319999999999 2024-09-16T00:10:39.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790369+0000", 2024-09-16T00:10:39.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6848999999999997e-05 2024-09-16T00:10:39.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790415+0000", 2024-09-16T00:10:39.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5692000000000002e-05 2024-09-16T00:10:39.766 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.766 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.766 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.766 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10915 3.1a 3:5a6e15a2:::1000000318e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:39.766 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:39.766 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 60.535745038000002, 2024-09-16T00:10:39.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014599472, 2024-09-16T00:10:39.766 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.766 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.767 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.767 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.767 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10915 2024-09-16T00:10:39.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.767 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:39.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:39.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215230+0000", 2024-09-16T00:10:39.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0090000000000002e-06 2024-09-16T00:10:39.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219556+0000", 2024-09-16T00:10:39.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043259539999999999 2024-09-16T00:10:39.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219577+0000", 2024-09-16T00:10:39.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1271999999999999e-05 2024-09-16T00:10:39.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219589+0000", 2024-09-16T00:10:39.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1766000000000001e-05 2024-09-16T00:10:39.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219663+0000", 2024-09-16T00:10:39.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4196999999999995e-05 2024-09-16T00:10:39.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219733+0000", 2024-09-16T00:10:39.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9553000000000003e-05 2024-09-16T00:10:39.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:39.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219827+0000", 2024-09-16T00:10:39.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4395999999999995e-05 2024-09-16T00:10:39.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.225724+0000", 2024-09-16T00:10:39.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058968359999999999 2024-09-16T00:10:39.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229705+0000", 2024-09-16T00:10:39.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003981409 2024-09-16T00:10:39.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229772+0000", 2024-09-16T00:10:39.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6137999999999996e-05 2024-09-16T00:10:39.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229828+0000", 2024-09-16T00:10:39.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5942000000000001e-05 2024-09-16T00:10:39.772 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.772 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.772 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.772 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11074 3.14 3:2834b7ee:::10000003131.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:39.772 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:39.773 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.838204914999999, 2024-09-16T00:10:39.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015588350000000001, 2024-09-16T00:10:39.773 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.773 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.773 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.773 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.773 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11074 2024-09-16T00:10:39.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.773 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:39.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:39.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912769+0000", 2024-09-16T00:10:39.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4600000000000004e-07 2024-09-16T00:10:39.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918264+0000", 2024-09-16T00:10:39.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054954890000000001 2024-09-16T00:10:39.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918281+0000", 2024-09-16T00:10:39.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6160000000000001e-05 2024-09-16T00:10:39.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918289+0000", 2024-09-16T00:10:39.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1650000000000006e-06 2024-09-16T00:10:39.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918386+0000", 2024-09-16T00:10:39.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7288999999999996e-05 2024-09-16T00:10:39.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918535+0000", 2024-09-16T00:10:39.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014876200000000001 2024-09-16T00:10:39.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:39.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918639+0000", 2024-09-16T00:10:39.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010470400000000001 2024-09-16T00:10:39.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.924703+0000", 2024-09-16T00:10:39.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060639759999999996 2024-09-16T00:10:39.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928246+0000", 2024-09-16T00:10:39.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035421720000000001 2024-09-16T00:10:39.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928305+0000", 2024-09-16T00:10:39.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9014000000000001e-05 2024-09-16T00:10:39.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928357+0000", 2024-09-16T00:10:39.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1873000000000003e-05 2024-09-16T00:10:39.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.779 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.779 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.779 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11111 3.9 3:93301cc4:::10000003210.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:39.779 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:39.779 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 59.661466713000003, 2024-09-16T00:10:39.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014336099999999999, 2024-09-16T00:10:39.779 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.779 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.780 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.780 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.780 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11111 2024-09-16T00:10:39.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.780 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:39.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:39.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089520+0000", 2024-09-16T00:10:39.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3903000000000001e-05 2024-09-16T00:10:39.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094463+0000", 2024-09-16T00:10:39.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049423000000000002 2024-09-16T00:10:39.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094494+0000", 2024-09-16T00:10:39.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1029999999999999e-05 2024-09-16T00:10:39.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094503+0000", 2024-09-16T00:10:39.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0180000000000007e-06 2024-09-16T00:10:39.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094573+0000", 2024-09-16T00:10:39.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0457000000000007e-05 2024-09-16T00:10:39.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094737+0000", 2024-09-16T00:10:39.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016408699999999999 2024-09-16T00:10:39.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:39.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094897+0000", 2024-09-16T00:10:39.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015938899999999999 2024-09-16T00:10:39.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.102326+0000", 2024-09-16T00:10:39.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074291230000000002 2024-09-16T00:10:39.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103723+0000", 2024-09-16T00:10:39.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001397255 2024-09-16T00:10:39.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103786+0000", 2024-09-16T00:10:39.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3489000000000001e-05 2024-09-16T00:10:39.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103842+0000", 2024-09-16T00:10:39.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6048999999999998e-05 2024-09-16T00:10:39.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.785 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.785 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.785 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:11934 3.1b e87/20 3:d94fa8da:::10000003190.00000000:head v 87'152, mlcod=87'150)", 2024-09-16T00:10:39.785 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:39.785 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.060722017000003, 2024-09-16T00:10:39.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014984934, 2024-09-16T00:10:39.785 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.786 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.786 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.786 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:39.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:39.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690252+0000", 2024-09-16T00:10:39.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8899999999999998e-07 2024-09-16T00:10:39.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696026+0000", 2024-09-16T00:10:39.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057745219999999998 2024-09-16T00:10:39.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696039+0000", 2024-09-16T00:10:39.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2611e-05 2024-09-16T00:10:39.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696047+0000", 2024-09-16T00:10:39.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5399999999999998e-06 2024-09-16T00:10:39.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696123+0000", 2024-09-16T00:10:39.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6349999999999999e-05 2024-09-16T00:10:39.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696144+0000", 2024-09-16T00:10:39.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1512999999999999e-05 2024-09-16T00:10:39.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705188+0000", 2024-09-16T00:10:39.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090438459999999995 2024-09-16T00:10:39.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705236+0000", 2024-09-16T00:10:39.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7663000000000001e-05 2024-09-16T00:10:39.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.790 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.790 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.790 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11936 3.3 3:c0886fb9:::100000031a1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:39.790 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:39.790 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 56.050576964999998, 2024-09-16T00:10:39.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015336379000000001, 2024-09-16T00:10:39.791 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.791 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.791 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.791 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.791 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11936 2024-09-16T00:10:39.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.791 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:39.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:39.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700399+0000", 2024-09-16T00:10:39.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6029999999999999e-06 2024-09-16T00:10:39.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704706+0000", 2024-09-16T00:10:39.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043070529999999999 2024-09-16T00:10:39.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704720+0000", 2024-09-16T00:10:39.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4629e-05 2024-09-16T00:10:39.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704729+0000", 2024-09-16T00:10:39.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6009999999999995e-06 2024-09-16T00:10:39.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.793 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704805+0000", 2024-09-16T00:10:39.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6015999999999999e-05 2024-09-16T00:10:39.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704866+0000", 2024-09-16T00:10:39.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0878999999999998e-05 2024-09-16T00:10:39.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:39.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704990+0000", 2024-09-16T00:10:39.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000124058 2024-09-16T00:10:39.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.714314+0000", 2024-09-16T00:10:39.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009324321 2024-09-16T00:10:39.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715632+0000", 2024-09-16T00:10:39.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001318121 2024-09-16T00:10:39.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715687+0000", 2024-09-16T00:10:39.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4135999999999998e-05 2024-09-16T00:10:39.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715732+0000", 2024-09-16T00:10:39.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961999999999998e-05 2024-09-16T00:10:39.796 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.796 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.796 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.796 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11953 3.1a 3:580e2503:::1000000311b.00000000:head [write 0~4194304 [2@2058066]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:39.796 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:39.796 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.984237812000003, 2024-09-16T00:10:39.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016299973999999998, 2024-09-16T00:10:39.796 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.797 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.797 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.797 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.797 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11953 2024-09-16T00:10:39.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.797 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:39.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:39.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766741+0000", 2024-09-16T00:10:39.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2939999999999996e-06 2024-09-16T00:10:39.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771165+0000", 2024-09-16T00:10:39.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044248229999999996 2024-09-16T00:10:39.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771181+0000", 2024-09-16T00:10:39.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5243e-05 2024-09-16T00:10:39.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.799 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.799 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771188+0000", 2024-09-16T00:10:39.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6429999999999995e-06 2024-09-16T00:10:39.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771238+0000", 2024-09-16T00:10:39.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9567999999999997e-05 2024-09-16T00:10:39.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771293+0000", 2024-09-16T00:10:39.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5207000000000002e-05 2024-09-16T00:10:39.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:39.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771380+0000", 2024-09-16T00:10:39.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6716000000000001e-05 2024-09-16T00:10:39.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.776193+0000", 2024-09-16T00:10:39.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048133350000000002 2024-09-16T00:10:39.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.782931+0000", 2024-09-16T00:10:39.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067376550000000004 2024-09-16T00:10:39.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.782987+0000", 2024-09-16T00:10:39.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6443000000000002e-05 2024-09-16T00:10:39.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.783035+0000", 2024-09-16T00:10:39.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8047000000000003e-05 2024-09-16T00:10:39.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.802 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.802 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.803 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12037 3.9 3:928db521:::1000000326d.00000000:head [write 0~4194304 [2@1655200]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:39.803 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:39.803 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 55.753030203999998, 2024-09-16T00:10:39.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017082494, 2024-09-16T00:10:39.803 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.803 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.803 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.803 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.804 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12037 2024-09-16T00:10:39.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.804 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:39.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:39.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997944+0000", 2024-09-16T00:10:39.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.517e-06 2024-09-16T00:10:39.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002177+0000", 2024-09-16T00:10:39.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042329300000000002 2024-09-16T00:10:39.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002197+0000", 2024-09-16T00:10:39.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9769e-05 2024-09-16T00:10:39.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002207+0000", 2024-09-16T00:10:39.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0178e-05 2024-09-16T00:10:39.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002266+0000", 2024-09-16T00:10:39.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9215999999999997e-05 2024-09-16T00:10:39.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002340+0000", 2024-09-16T00:10:39.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3798000000000001e-05 2024-09-16T00:10:39.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:39.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002454+0000", 2024-09-16T00:10:39.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113551 2024-09-16T00:10:39.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.006270+0000", 2024-09-16T00:10:39.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038163839999999999 2024-09-16T00:10:39.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.014957+0000", 2024-09-16T00:10:39.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086869119999999998 2024-09-16T00:10:39.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.014996+0000", 2024-09-16T00:10:39.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8878000000000003e-05 2024-09-16T00:10:39.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.015025+0000", 2024-09-16T00:10:39.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9360999999999999e-05 2024-09-16T00:10:39.809 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.809 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.809 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.809 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12245 3.1a 3:599a1cd1:::10000003252.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:39.809 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:39.809 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.968332787000001, 2024-09-16T00:10:39.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017131203000000001, 2024-09-16T00:10:39.809 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.809 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.810 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.810 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.810 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12245 2024-09-16T00:10:39.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.810 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:39.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:39.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782642+0000", 2024-09-16T00:10:39.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.311e-06 2024-09-16T00:10:39.811 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786551+0000", 2024-09-16T00:10:39.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039095229999999998 2024-09-16T00:10:39.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786566+0000", 2024-09-16T00:10:39.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4724e-05 2024-09-16T00:10:39.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786575+0000", 2024-09-16T00:10:39.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8449999999999992e-06 2024-09-16T00:10:39.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786636+0000", 2024-09-16T00:10:39.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.143e-05 2024-09-16T00:10:39.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786699+0000", 2024-09-16T00:10:39.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3218000000000004e-05 2024-09-16T00:10:39.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:39.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786796+0000", 2024-09-16T00:10:39.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6347000000000004e-05 2024-09-16T00:10:39.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.792369+0000", 2024-09-16T00:10:39.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055729780000000001 2024-09-16T00:10:39.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799706+0000", 2024-09-16T00:10:39.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073374349999999998 2024-09-16T00:10:39.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799739+0000", 2024-09-16T00:10:39.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2833999999999998e-05 2024-09-16T00:10:39.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799771+0000", 2024-09-16T00:10:39.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2558000000000002e-05 2024-09-16T00:10:39.815 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.815 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.815 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.816 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12292 3.3 3:c00779fd:::1000000320e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:39.816 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:39.816 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.853942689999997, 2024-09-16T00:10:39.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013828353999999999, 2024-09-16T00:10:39.816 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.816 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.816 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.816 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.816 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.816 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.816 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12292 2024-09-16T00:10:39.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.817 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:39.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:39.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897039+0000", 2024-09-16T00:10:39.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3299999999999999e-06 2024-09-16T00:10:39.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900830+0000", 2024-09-16T00:10:39.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037915340000000001 2024-09-16T00:10:39.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900850+0000", 2024-09-16T00:10:39.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9400000000000001e-05 2024-09-16T00:10:39.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900860+0000", 2024-09-16T00:10:39.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0609e-05 2024-09-16T00:10:39.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900928+0000", 2024-09-16T00:10:39.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7570000000000005e-05 2024-09-16T00:10:39.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.901027+0000", 2024-09-16T00:10:39.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9456000000000005e-05 2024-09-16T00:10:39.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:39.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.901168+0000", 2024-09-16T00:10:39.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014082799999999999 2024-09-16T00:10:39.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.906325+0000", 2024-09-16T00:10:39.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005156702 2024-09-16T00:10:39.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910796+0000", 2024-09-16T00:10:39.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044714350000000002 2024-09-16T00:10:39.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910829+0000", 2024-09-16T00:10:39.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3229000000000003e-05 2024-09-16T00:10:39.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910859+0000", 2024-09-16T00:10:39.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9261e-05 2024-09-16T00:10:39.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.821 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.821 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.822 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13326 3.9 3:92f796cf:::1000000350f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:39.822 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:39.822 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.794252407999998, 2024-09-16T00:10:39.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015673229, 2024-09-16T00:10:39.822 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.822 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.822 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.822 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.822 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13326 2024-09-16T00:10:39.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.822 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:39.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:39.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956722+0000", 2024-09-16T00:10:39.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3200000000000001e-06 2024-09-16T00:10:39.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960803+0000", 2024-09-16T00:10:39.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040807029999999998 2024-09-16T00:10:39.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960820+0000", 2024-09-16T00:10:39.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7478000000000001e-05 2024-09-16T00:10:39.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960830+0000", 2024-09-16T00:10:39.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:10:39.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960903+0000", 2024-09-16T00:10:39.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3057000000000002e-05 2024-09-16T00:10:39.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960970+0000", 2024-09-16T00:10:39.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7063000000000005e-05 2024-09-16T00:10:39.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:39.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.961067+0000", 2024-09-16T00:10:39.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6906999999999999e-05 2024-09-16T00:10:39.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.967945+0000", 2024-09-16T00:10:39.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068787359999999999 2024-09-16T00:10:39.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972328+0000", 2024-09-16T00:10:39.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043826170000000001 2024-09-16T00:10:39.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972363+0000", 2024-09-16T00:10:39.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4486000000000002e-05 2024-09-16T00:10:39.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972394+0000", 2024-09-16T00:10:39.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1343999999999997e-05 2024-09-16T00:10:39.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.827 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.827 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.828 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13429 3.1a 3:58dcd24a:::100000036d5.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:39.828 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:39.828 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 51.439682392000002, 2024-09-16T00:10:39.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014237671, 2024-09-16T00:10:39.828 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.828 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.828 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.828 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.828 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13429 2024-09-16T00:10:39.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.829 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:39.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:39.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311292+0000", 2024-09-16T00:10:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4890000000000001e-06 2024-09-16T00:10:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316378+0000", 2024-09-16T00:10:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050859290000000003 2024-09-16T00:10:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316399+0000", 2024-09-16T00:10:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1345e-05 2024-09-16T00:10:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316410+0000", 2024-09-16T00:10:39.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0295999999999999e-05 2024-09-16T00:10:39.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316493+0000", 2024-09-16T00:10:39.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3533000000000001e-05 2024-09-16T00:10:39.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316622+0000", 2024-09-16T00:10:39.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012887900000000001 2024-09-16T00:10:39.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:39.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316725+0000", 2024-09-16T00:10:39.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010295599999999999 2024-09-16T00:10:39.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.320948+0000", 2024-09-16T00:10:39.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042229820000000001 2024-09-16T00:10:39.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325415+0000", 2024-09-16T00:10:39.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044673849999999999 2024-09-16T00:10:39.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325487+0000", 2024-09-16T00:10:39.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1087999999999994e-05 2024-09-16T00:10:39.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325528+0000", 2024-09-16T00:10:39.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1789000000000003e-05 2024-09-16T00:10:39.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.833 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.833 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.834 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13527 3.3 3:c0886fb9:::100000031a1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:39.834 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:39.834 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.965522303, 2024-09-16T00:10:39.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015187236, 2024-09-16T00:10:39.834 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.834 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.834 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.834 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.834 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13527 2024-09-16T00:10:39.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.835 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:39.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:39.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785453+0000", 2024-09-16T00:10:39.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1689999999999999e-06 2024-09-16T00:10:39.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791191+0000", 2024-09-16T00:10:39.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057378339999999998 2024-09-16T00:10:39.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791217+0000", 2024-09-16T00:10:39.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6480999999999999e-05 2024-09-16T00:10:39.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791233+0000", 2024-09-16T00:10:39.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5841e-05 2024-09-16T00:10:39.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791319+0000", 2024-09-16T00:10:39.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5527999999999998e-05 2024-09-16T00:10:39.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791413+0000", 2024-09-16T00:10:39.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4685000000000006e-05 2024-09-16T00:10:39.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:39.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791563+0000", 2024-09-16T00:10:39.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000149624 2024-09-16T00:10:39.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.795749+0000", 2024-09-16T00:10:39.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004186286 2024-09-16T00:10:39.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800577+0000", 2024-09-16T00:10:39.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048273040000000001 2024-09-16T00:10:39.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800611+0000", 2024-09-16T00:10:39.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4270000000000002e-05 2024-09-16T00:10:39.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800638+0000", 2024-09-16T00:10:39.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7214000000000001e-05 2024-09-16T00:10:39.839 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.839 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.840 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.840 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13724 3.9 3:90613e6c:::100000034c4.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:39.840 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:39.840 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.295450557999999, 2024-09-16T00:10:39.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014039800999999999, 2024-09-16T00:10:39.840 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.840 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.840 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.840 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.841 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13724 2024-09-16T00:10:39.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.841 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:39.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:39.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455524+0000", 2024-09-16T00:10:39.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.437e-06 2024-09-16T00:10:39.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459592+0000", 2024-09-16T00:10:39.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040678729999999996 2024-09-16T00:10:39.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459610+0000", 2024-09-16T00:10:39.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7688000000000001e-05 2024-09-16T00:10:39.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459617+0000", 2024-09-16T00:10:39.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.374e-06 2024-09-16T00:10:39.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459678+0000", 2024-09-16T00:10:39.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1346999999999994e-05 2024-09-16T00:10:39.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459824+0000", 2024-09-16T00:10:39.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014586699999999999 2024-09-16T00:10:39.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:39.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459908+0000", 2024-09-16T00:10:39.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3521000000000002e-05 2024-09-16T00:10:39.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.464227+0000", 2024-09-16T00:10:39.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043197060000000004 2024-09-16T00:10:39.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469499+0000", 2024-09-16T00:10:39.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005271838 2024-09-16T00:10:39.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469534+0000", 2024-09-16T00:10:39.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4712000000000002e-05 2024-09-16T00:10:39.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469562+0000", 2024-09-16T00:10:39.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8438000000000001e-05 2024-09-16T00:10:39.845 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.845 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.846 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.846 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13741 3.3 3:c1377c76:::10000003664.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:39.846 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:39.846 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 50.248280612999999, 2024-09-16T00:10:39.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016142843, 2024-09-16T00:10:39.846 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.846 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.847 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.847 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.847 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.847 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.847 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13741 2024-09-16T00:10:39.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.847 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:39.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:39.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502694+0000", 2024-09-16T00:10:39.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0449999999999999e-06 2024-09-16T00:10:39.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507229+0000", 2024-09-16T00:10:39.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045357410000000003 2024-09-16T00:10:39.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507248+0000", 2024-09-16T00:10:39.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9127000000000002e-05 2024-09-16T00:10:39.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507259+0000", 2024-09-16T00:10:39.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0448999999999999e-05 2024-09-16T00:10:39.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507357+0000", 2024-09-16T00:10:39.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8400000000000007e-05 2024-09-16T00:10:39.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507514+0000", 2024-09-16T00:10:39.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000157195 2024-09-16T00:10:39.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:39.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507617+0000", 2024-09-16T00:10:39.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010262199999999999 2024-09-16T00:10:39.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.514047+0000", 2024-09-16T00:10:39.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006430018 2024-09-16T00:10:39.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518743+0000", 2024-09-16T00:10:39.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046963040000000001 2024-09-16T00:10:39.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518796+0000", 2024-09-16T00:10:39.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2305000000000002e-05 2024-09-16T00:10:39.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518835+0000", 2024-09-16T00:10:39.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9637e-05 2024-09-16T00:10:39.853 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.853 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.853 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.853 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14633 3.9 3:9170e82c:::100000035d6.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:39.853 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:39.853 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 47.128900356999999, 2024-09-16T00:10:39.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014466105, 2024-09-16T00:10:39.854 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.854 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.854 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.854 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.854 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.854 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.854 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14633 2024-09-16T00:10:39.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.854 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:39.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:39.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622074+0000", 2024-09-16T00:10:39.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.761e-06 2024-09-16T00:10:39.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626712+0000", 2024-09-16T00:10:39.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004637263 2024-09-16T00:10:39.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626724+0000", 2024-09-16T00:10:39.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2479e-05 2024-09-16T00:10:39.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.856 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.856 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626730+0000", 2024-09-16T00:10:39.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7309999999999998e-06 2024-09-16T00:10:39.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.856 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626791+0000", 2024-09-16T00:10:39.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0854999999999999e-05 2024-09-16T00:10:39.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626857+0000", 2024-09-16T00:10:39.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6549000000000003e-05 2024-09-16T00:10:39.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:39.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626943+0000", 2024-09-16T00:10:39.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6141000000000001e-05 2024-09-16T00:10:39.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.630854+0000", 2024-09-16T00:10:39.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039100790000000003 2024-09-16T00:10:39.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636468+0000", 2024-09-16T00:10:39.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056140570000000004 2024-09-16T00:10:39.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636500+0000", 2024-09-16T00:10:39.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2441999999999999e-05 2024-09-16T00:10:39.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636539+0000", 2024-09-16T00:10:39.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8748000000000002e-05 2024-09-16T00:10:39.859 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.859 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.859 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.859 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14667 3.9 3:9379a4af:::10000003352.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:39.860 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:39.860 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 46.953888351000003, 2024-09-16T00:10:39.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014580585, 2024-09-16T00:10:39.860 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.860 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.860 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.860 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.860 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14667 2024-09-16T00:10:39.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.860 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:39.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:39.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797086+0000", 2024-09-16T00:10:39.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.362e-06 2024-09-16T00:10:39.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801415+0000", 2024-09-16T00:10:39.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043288449999999996 2024-09-16T00:10:39.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801435+0000", 2024-09-16T00:10:39.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0305999999999998e-05 2024-09-16T00:10:39.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801442+0000", 2024-09-16T00:10:39.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6290000000000004e-06 2024-09-16T00:10:39.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801497+0000", 2024-09-16T00:10:39.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5111999999999997e-05 2024-09-16T00:10:39.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801639+0000", 2024-09-16T00:10:39.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014168000000000001 2024-09-16T00:10:39.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:39.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801788+0000", 2024-09-16T00:10:39.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014963300000000001 2024-09-16T00:10:39.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.805407+0000", 2024-09-16T00:10:39.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036186119999999998 2024-09-16T00:10:39.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811584+0000", 2024-09-16T00:10:39.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061772889999999999 2024-09-16T00:10:39.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811635+0000", 2024-09-16T00:10:39.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0453e-05 2024-09-16T00:10:39.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811665+0000", 2024-09-16T00:10:39.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0664000000000003e-05 2024-09-16T00:10:39.868 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.868 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.868 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.868 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14951 3.1a 3:59aee80a:::1000000394f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:39.868 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:39.868 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.783850456000003, 2024-09-16T00:10:39.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015993941000000001, 2024-09-16T00:10:39.868 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.868 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.868 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.869 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.869 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14951 2024-09-16T00:10:39.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.869 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:39.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:39.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967124+0000", 2024-09-16T00:10:39.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.86e-06 2024-09-16T00:10:39.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971258+0000", 2024-09-16T00:10:39.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041334780000000003 2024-09-16T00:10:39.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971274+0000", 2024-09-16T00:10:39.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6201e-05 2024-09-16T00:10:39.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971284+0000", 2024-09-16T00:10:39.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4779999999999997e-06 2024-09-16T00:10:39.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971384+0000", 2024-09-16T00:10:39.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9878999999999997e-05 2024-09-16T00:10:39.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971518+0000", 2024-09-16T00:10:39.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013459400000000001 2024-09-16T00:10:39.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:39.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971604+0000", 2024-09-16T00:10:39.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6215e-05 2024-09-16T00:10:39.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.976673+0000", 2024-09-16T00:10:39.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050685920000000002 2024-09-16T00:10:39.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983043+0000", 2024-09-16T00:10:39.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063698410000000002 2024-09-16T00:10:39.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983082+0000", 2024-09-16T00:10:39.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9261999999999998e-05 2024-09-16T00:10:39.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983117+0000", 2024-09-16T00:10:39.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4541e-05 2024-09-16T00:10:39.874 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.874 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.874 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.874 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:15152 3.3 3:c388c7e2:::100000032ec.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-16T00:10:39.874 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:39.874 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.023304346000003, 2024-09-16T00:10:39.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014603532000000001, 2024-09-16T00:10:39.874 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:39.874 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:39.875 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:39.875 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:39.875 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:39.875 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:39.875 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15152 2024-09-16T00:10:39.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.875 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:39.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:39.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:39.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:39.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:39.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:39.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:39.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727674+0000", 2024-09-16T00:10:39.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7699999999999998e-06 2024-09-16T00:10:39.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:39.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732274+0000", 2024-09-16T00:10:39.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045994299999999998 2024-09-16T00:10:39.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:39.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732286+0000", 2024-09-16T00:10:39.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2016e-05 2024-09-16T00:10:39.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:39.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732294+0000", 2024-09-16T00:10:39.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7049999999999999e-06 2024-09-16T00:10:39.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:39.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732359+0000", 2024-09-16T00:10:39.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5296000000000006e-05 2024-09-16T00:10:39.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:39.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732501+0000", 2024-09-16T00:10:39.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000141731 2024-09-16T00:10:39.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:39.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732629+0000", 2024-09-16T00:10:39.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012841299999999999 2024-09-16T00:10:39.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:39.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.737511+0000", 2024-09-16T00:10:39.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048822520000000001 2024-09-16T00:10:39.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:39.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742172+0000", 2024-09-16T00:10:39.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046610610000000002 2024-09-16T00:10:39.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:39.879 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742227+0000", 2024-09-16T00:10:39.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4497999999999999e-05 2024-09-16T00:10:39.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:39.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:39.879 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:39.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742272+0000", 2024-09-16T00:10:39.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5359999999999999e-05 2024-09-16T00:10:39.880 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.880 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.880 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.880 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:39.880 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:39.880 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:39.882 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:10:39.882 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:39.880+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:39.982 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:39.981+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:40.037 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:40.037 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:40.037 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:40.037 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:40.048 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:10:40.084 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:40.082+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:40.163 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:40.163 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:40.163 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:40.177 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:10:40.221 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:40.220+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:40.285 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:40.284+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:40.296 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:40.296 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10658 3.1c 3:38c4c24e:::100000030bb.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 64.502807809999993, 2024-09-16T00:10:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021232445999999999, 2024-09-16T00:10:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10658 2024-09-16T00:10:40.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791398+0000", 2024-09-16T00:10:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3880000000000001e-06 2024-09-16T00:10:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800951+0000", 2024-09-16T00:10:40.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095532110000000007 2024-09-16T00:10:40.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800966+0000", 2024-09-16T00:10:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5058999999999999e-05 2024-09-16T00:10:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800974+0000", 2024-09-16T00:10:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1240000000000005e-06 2024-09-16T00:10:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801127+0000", 2024-09-16T00:10:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000153103 2024-09-16T00:10:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801207+0000", 2024-09-16T00:10:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9569999999999999e-05 2024-09-16T00:10:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801322+0000", 2024-09-16T00:10:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115358 2024-09-16T00:10:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.805760+0000", 2024-09-16T00:10:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044379409999999999 2024-09-16T00:10:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812528+0000", 2024-09-16T00:10:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067678249999999999 2024-09-16T00:10:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812586+0000", 2024-09-16T00:10:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8164000000000001e-05 2024-09-16T00:10:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812629+0000", 2024-09-16T00:10:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2703000000000001e-05 2024-09-16T00:10:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4122 2.18 2:1cc859d9:::200.00000012:head [write 798310~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.63526813, 2024-09-16T00:10:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019267580999999999, 2024-09-16T00:10:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658937+0000", 2024-09-16T00:10:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7700000000000004e-07 2024-09-16T00:10:40.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658942+0000", 2024-09-16T00:10:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.48e-06 2024-09-16T00:10:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658952+0000", 2024-09-16T00:10:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9469999999999998e-06 2024-09-16T00:10:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658959+0000", 2024-09-16T00:10:40.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4400000000000002e-06 2024-09-16T00:10:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670507+0000", 2024-09-16T00:10:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011548159000000001 2024-09-16T00:10:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670579+0000", 2024-09-16T00:10:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2082999999999994e-05 2024-09-16T00:10:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670686+0000", 2024-09-16T00:10:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107069 2024-09-16T00:10:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678001+0000", 2024-09-16T00:10:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073148780000000004 2024-09-16T00:10:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678104+0000", 2024-09-16T00:10:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010290599999999999 2024-09-16T00:10:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678163+0000", 2024-09-16T00:10:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8859e-05 2024-09-16T00:10:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678204+0000", 2024-09-16T00:10:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0982999999999997e-05 2024-09-16T00:10:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.313 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4123 2.18 2:1cc859d9:::200.00000012:head [write 799709~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.634698948999997, 2024-09-16T00:10:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018857569000000001, 2024-09-16T00:10:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659506+0000", 2024-09-16T00:10:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.78e-07 2024-09-16T00:10:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659512+0000", 2024-09-16T00:10:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0100000000000001e-06 2024-09-16T00:10:40.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659523+0000", 2024-09-16T00:10:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-16T00:10:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659531+0000", 2024-09-16T00:10:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2010000000000002e-06 2024-09-16T00:10:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671124+0000", 2024-09-16T00:10:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011593696000000001 2024-09-16T00:10:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671171+0000", 2024-09-16T00:10:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6974999999999997e-05 2024-09-16T00:10:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671243+0000", 2024-09-16T00:10:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2131000000000004e-05 2024-09-16T00:10:40.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678008+0000", 2024-09-16T00:10:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067642329999999997 2024-09-16T00:10:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678261+0000", 2024-09-16T00:10:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025378999999999999 2024-09-16T00:10:40.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678320+0000", 2024-09-16T00:10:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8452000000000002e-05 2024-09-16T00:10:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678363+0000", 2024-09-16T00:10:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2831999999999999e-05 2024-09-16T00:10:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4126 2.18 2:1cc859d9:::200.00000012:head [write 806443~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.633301162999999, 2024-09-16T00:10:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019331159000000001, 2024-09-16T00:10:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660904+0000", 2024-09-16T00:10:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6800000000000009e-07 2024-09-16T00:10:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660909+0000", 2024-09-16T00:10:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2730000000000002e-06 2024-09-16T00:10:40.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660919+0000", 2024-09-16T00:10:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4960000000000002e-06 2024-09-16T00:10:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660923+0000", 2024-09-16T00:10:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6260000000000003e-06 2024-09-16T00:10:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672698+0000", 2024-09-16T00:10:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011774573999999999 2024-09-16T00:10:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672784+0000", 2024-09-16T00:10:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6074000000000003e-05 2024-09-16T00:10:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672897+0000", 2024-09-16T00:10:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011267199999999999 2024-09-16T00:10:40.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678026+0000", 2024-09-16T00:10:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051297089999999997 2024-09-16T00:10:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680147+0000", 2024-09-16T00:10:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021209409999999999 2024-09-16T00:10:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680197+0000", 2024-09-16T00:10:40.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0068000000000002e-05 2024-09-16T00:10:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680234+0000", 2024-09-16T00:10:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6758000000000003e-05 2024-09-16T00:10:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4127 2.18 2:1cc859d9:::200.00000012:head [write 808688~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.633019314000002, 2024-09-16T00:10:40.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019226493000000001, 2024-09-16T00:10:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:40.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661186+0000", 2024-09-16T00:10:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2399999999999996e-07 2024-09-16T00:10:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661192+0000", 2024-09-16T00:10:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9259999999999997e-06 2024-09-16T00:10:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661203+0000", 2024-09-16T00:10:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1174e-05 2024-09-16T00:10:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661213+0000", 2024-09-16T00:10:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0579e-05 2024-09-16T00:10:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673408+0000", 2024-09-16T00:10:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012194442 2024-09-16T00:10:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673482+0000", 2024-09-16T00:10:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4258000000000006e-05 2024-09-16T00:10:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673592+0000", 2024-09-16T00:10:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109733 2024-09-16T00:10:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678033+0000", 2024-09-16T00:10:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044411629999999997 2024-09-16T00:10:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680287+0000", 2024-09-16T00:10:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022538469999999998 2024-09-16T00:10:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680367+0000", 2024-09-16T00:10:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9808999999999994e-05 2024-09-16T00:10:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680411+0000", 2024-09-16T00:10:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4638000000000002e-05 2024-09-16T00:10:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4128 2.18 2:1cc859d9:::200.00000012:head [write 810933~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.632600183000001, 2024-09-16T00:10:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018982046999999998, 2024-09-16T00:10:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.331 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661605+0000", 2024-09-16T00:10:40.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.031e-06 2024-09-16T00:10:40.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661610+0000", 2024-09-16T00:10:40.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5010000000000004e-06 2024-09-16T00:10:40.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661623+0000", 2024-09-16T00:10:40.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2286000000000001e-05 2024-09-16T00:10:40.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661630+0000", 2024-09-16T00:10:40.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000002e-06 2024-09-16T00:10:40.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674011+0000", 2024-09-16T00:10:40.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012380923 2024-09-16T00:10:40.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674092+0000", 2024-09-16T00:10:40.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1267999999999995e-05 2024-09-16T00:10:40.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674216+0000", 2024-09-16T00:10:40.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012331700000000001 2024-09-16T00:10:40.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678039+0000", 2024-09-16T00:10:40.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038232489999999999 2024-09-16T00:10:40.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680493+0000", 2024-09-16T00:10:40.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024543429999999999 2024-09-16T00:10:40.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680546+0000", 2024-09-16T00:10:40.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3239000000000003e-05 2024-09-16T00:10:40.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680586+0000", 2024-09-16T00:10:40.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9496999999999998e-05 2024-09-16T00:10:40.337 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.337 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.337 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.337 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4129 2.18 2:1cc859d9:::200.00000012:head [write 813178~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.337 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:40.337 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.632243031999998, 2024-09-16T00:10:40.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018785932000000002, 2024-09-16T00:10:40.337 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.337 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.337 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.337 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:40.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:40.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661962+0000", 2024-09-16T00:10:40.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-07 2024-09-16T00:10:40.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661966+0000", 2024-09-16T00:10:40.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-06 2024-09-16T00:10:40.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.662015+0000", 2024-09-16T00:10:40.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8364e-05 2024-09-16T00:10:40.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.662023+0000", 2024-09-16T00:10:40.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2090000000000005e-06 2024-09-16T00:10:40.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674744+0000", 2024-09-16T00:10:40.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012721423000000001 2024-09-16T00:10:40.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674825+0000", 2024-09-16T00:10:40.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1036000000000001e-05 2024-09-16T00:10:40.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674946+0000", 2024-09-16T00:10:40.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120727 2024-09-16T00:10:40.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678045+0000", 2024-09-16T00:10:40.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030993280000000002 2024-09-16T00:10:40.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680646+0000", 2024-09-16T00:10:40.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026005300000000002 2024-09-16T00:10:40.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680705+0000", 2024-09-16T00:10:40.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9141000000000002e-05 2024-09-16T00:10:40.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680747+0000", 2024-09-16T00:10:40.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2064000000000002e-05 2024-09-16T00:10:40.342 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.343 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.343 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.343 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4184 2.18 2:1cc859d9:::200.00000012:head [write 927308~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.343 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:40.343 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.588194436999999, 2024-09-16T00:10:40.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023503514999999999, 2024-09-16T00:10:40.343 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.343 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.343 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.343 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:40.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:40.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706011+0000", 2024-09-16T00:10:40.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8400000000000003e-07 2024-09-16T00:10:40.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706016+0000", 2024-09-16T00:10:40.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7479999999999999e-06 2024-09-16T00:10:40.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.346 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706028+0000", 2024-09-16T00:10:40.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1477e-05 2024-09-16T00:10:40.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.346 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706035+0000", 2024-09-16T00:10:40.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1509999999999998e-06 2024-09-16T00:10:40.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.719820+0000", 2024-09-16T00:10:40.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013785168 2024-09-16T00:10:40.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.719888+0000", 2024-09-16T00:10:40.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7631000000000004e-05 2024-09-16T00:10:40.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720003+0000", 2024-09-16T00:10:40.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114849 2024-09-16T00:10:40.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725150+0000", 2024-09-16T00:10:40.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051469970000000004 2024-09-16T00:10:40.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729386+0000", 2024-09-16T00:10:40.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042368529999999996 2024-09-16T00:10:40.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729461+0000", 2024-09-16T00:10:40.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4935e-05 2024-09-16T00:10:40.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729513+0000", 2024-09-16T00:10:40.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1822e-05 2024-09-16T00:10:40.349 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.349 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.349 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.350 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4185 2.18 2:1cc859d9:::200.00000012:head [write 929217~3264 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.350 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:40.350 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.587188257000001, 2024-09-16T00:10:40.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023998430000000001, 2024-09-16T00:10:40.350 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.350 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.350 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.350 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:40.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:40.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707017+0000", 2024-09-16T00:10:40.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9999999999999996e-07 2024-09-16T00:10:40.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707023+0000", 2024-09-16T00:10:40.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1789999999999996e-06 2024-09-16T00:10:40.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707035+0000", 2024-09-16T00:10:40.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2106000000000001e-05 2024-09-16T00:10:40.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707042+0000", 2024-09-16T00:10:40.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2960000000000001e-06 2024-09-16T00:10:40.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720503+0000", 2024-09-16T00:10:40.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013460952999999999 2024-09-16T00:10:40.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720583+0000", 2024-09-16T00:10:40.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0171000000000002e-05 2024-09-16T00:10:40.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720668+0000", 2024-09-16T00:10:40.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4128999999999994e-05 2024-09-16T00:10:40.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725193+0000", 2024-09-16T00:10:40.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045257700000000001 2024-09-16T00:10:40.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.730924+0000", 2024-09-16T00:10:40.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057307690000000001 2024-09-16T00:10:40.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.730972+0000", 2024-09-16T00:10:40.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7756000000000002e-05 2024-09-16T00:10:40.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731014+0000", 2024-09-16T00:10:40.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2401000000000003e-05 2024-09-16T00:10:40.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.355 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.355 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.356 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4186 2.18 2:1cc859d9:::200.00000012:head [write 932481~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.356 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:40.356 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.586600099999998, 2024-09-16T00:10:40.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023544934999999999, 2024-09-16T00:10:40.356 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.356 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.356 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.356 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:40.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:40.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707605+0000", 2024-09-16T00:10:40.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1500000000000003e-07 2024-09-16T00:10:40.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.357 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.357 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707610+0000", 2024-09-16T00:10:40.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7049999999999998e-06 2024-09-16T00:10:40.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707618+0000", 2024-09-16T00:10:40.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6149999999999997e-06 2024-09-16T00:10:40.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707623+0000", 2024-09-16T00:10:40.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0660000000000003e-06 2024-09-16T00:10:40.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721143+0000", 2024-09-16T00:10:40.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01351976 2024-09-16T00:10:40.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721221+0000", 2024-09-16T00:10:40.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7918999999999997e-05 2024-09-16T00:10:40.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721335+0000", 2024-09-16T00:10:40.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113781 2024-09-16T00:10:40.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725236+0000", 2024-09-16T00:10:40.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039010379999999999 2024-09-16T00:10:40.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731025+0000", 2024-09-16T00:10:40.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057896659999999997 2024-09-16T00:10:40.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731091+0000", 2024-09-16T00:10:40.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.567e-05 2024-09-16T00:10:40.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731149+0000", 2024-09-16T00:10:40.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7800000000000002e-05 2024-09-16T00:10:40.361 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.361 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.361 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.362 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4187 2.18 2:1cc859d9:::200.00000012:head [write 934390~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.362 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:40.362 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.585747971000004, 2024-09-16T00:10:40.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022820340000000001, 2024-09-16T00:10:40.362 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.362 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.362 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.362 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:40.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:40.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708457+0000", 2024-09-16T00:10:40.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6899999999999996e-07 2024-09-16T00:10:40.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708462+0000", 2024-09-16T00:10:40.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3090000000000003e-06 2024-09-16T00:10:40.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708472+0000", 2024-09-16T00:10:40.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7499999999999998e-06 2024-09-16T00:10:40.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708479+0000", 2024-09-16T00:10:40.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1189999999999999e-06 2024-09-16T00:10:40.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721895+0000", 2024-09-16T00:10:40.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013415664000000001 2024-09-16T00:10:40.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721941+0000", 2024-09-16T00:10:40.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6031e-05 2024-09-16T00:10:40.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722007+0000", 2024-09-16T00:10:40.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6503999999999999e-05 2024-09-16T00:10:40.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725277+0000", 2024-09-16T00:10:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003269554 2024-09-16T00:10:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731158+0000", 2024-09-16T00:10:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058813750000000003 2024-09-16T00:10:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731240+0000", 2024-09-16T00:10:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1815000000000002e-05 2024-09-16T00:10:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.368 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731276+0000", 2024-09-16T00:10:40.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6449999999999998e-05 2024-09-16T00:10:40.368 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.368 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.368 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.368 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4188 2.18 2:1cc859d9:::200.00000012:head [write 936299~2419 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.368 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:40.368 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.585327028000002, 2024-09-16T00:10:40.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022518565000000001, 2024-09-16T00:10:40.368 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.369 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.369 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.369 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:40.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:40.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708878+0000", 2024-09-16T00:10:40.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8800000000000001e-07 2024-09-16T00:10:40.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708884+0000", 2024-09-16T00:10:40.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9800000000000003e-06 2024-09-16T00:10:40.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708895+0000", 2024-09-16T00:10:40.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1416e-05 2024-09-16T00:10:40.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708903+0000", 2024-09-16T00:10:40.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4449999999999999e-06 2024-09-16T00:10:40.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722327+0000", 2024-09-16T00:10:40.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013424681000000001 2024-09-16T00:10:40.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722408+0000", 2024-09-16T00:10:40.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0092000000000005e-05 2024-09-16T00:10:40.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722538+0000", 2024-09-16T00:10:40.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013046600000000001 2024-09-16T00:10:40.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726665+0000", 2024-09-16T00:10:40.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041267420000000001 2024-09-16T00:10:40.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731284+0000", 2024-09-16T00:10:40.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046195469999999999 2024-09-16T00:10:40.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731358+0000", 2024-09-16T00:10:40.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4002000000000001e-05 2024-09-16T00:10:40.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731396+0000", 2024-09-16T00:10:40.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7305999999999998e-05 2024-09-16T00:10:40.374 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.374 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.374 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.374 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4189 2.18 2:1cc859d9:::200.00000012:head [write 938718~1910 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.374 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:40.374 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.584419912000001, 2024-09-16T00:10:40.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021703626, 2024-09-16T00:10:40.374 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.374 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.374 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.374 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:40.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:40.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709785+0000", 2024-09-16T00:10:40.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.043e-06 2024-09-16T00:10:40.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709791+0000", 2024-09-16T00:10:40.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9499999999999998e-06 2024-09-16T00:10:40.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709804+0000", 2024-09-16T00:10:40.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2958e-05 2024-09-16T00:10:40.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709812+0000", 2024-09-16T00:10:40.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9869999999999999e-06 2024-09-16T00:10:40.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723058+0000", 2024-09-16T00:10:40.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013245497 2024-09-16T00:10:40.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723131+0000", 2024-09-16T00:10:40.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3879000000000002e-05 2024-09-16T00:10:40.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723241+0000", 2024-09-16T00:10:40.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109435 2024-09-16T00:10:40.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726708+0000", 2024-09-16T00:10:40.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034671340000000002 2024-09-16T00:10:40.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731403+0000", 2024-09-16T00:10:40.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046953979999999999 2024-09-16T00:10:40.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731454+0000", 2024-09-16T00:10:40.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0747e-05 2024-09-16T00:10:40.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731488+0000", 2024-09-16T00:10:40.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3597999999999999e-05 2024-09-16T00:10:40.379 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.379 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.379 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.380 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4190 2.18 2:1cc859d9:::200.00000012:head [write 940628~2420 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.380 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:40.380 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.583947747000003, 2024-09-16T00:10:40.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021346812999999999, 2024-09-16T00:10:40.380 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.380 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.380 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.380 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:40.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:40.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710257+0000", 2024-09-16T00:10:40.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8100000000000001e-07 2024-09-16T00:10:40.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710263+0000", 2024-09-16T00:10:40.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8050000000000003e-06 2024-09-16T00:10:40.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710273+0000", 2024-09-16T00:10:40.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0005e-05 2024-09-16T00:10:40.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710280+0000", 2024-09-16T00:10:40.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3819999999999998e-06 2024-09-16T00:10:40.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723653+0000", 2024-09-16T00:10:40.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013372866000000001 2024-09-16T00:10:40.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723733+0000", 2024-09-16T00:10:40.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9396999999999999e-05 2024-09-16T00:10:40.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723852+0000", 2024-09-16T00:10:40.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011919899999999999 2024-09-16T00:10:40.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726752+0000", 2024-09-16T00:10:40.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028999659999999999 2024-09-16T00:10:40.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731495+0000", 2024-09-16T00:10:40.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047432209999999997 2024-09-16T00:10:40.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731568+0000", 2024-09-16T00:10:40.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2452000000000004e-05 2024-09-16T00:10:40.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731603+0000", 2024-09-16T00:10:40.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5639000000000002e-05 2024-09-16T00:10:40.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.385 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.385 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.385 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4192 2.18 2:1cc859d9:::200.00000012:head [write 943048~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.385 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:40.385 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.583214906999999, 2024-09-16T00:10:40.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021688433999999999, 2024-09-16T00:10:40.386 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.386 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.386 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.386 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:40.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:40.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710990+0000", 2024-09-16T00:10:40.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1999999999999998e-07 2024-09-16T00:10:40.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710996+0000", 2024-09-16T00:10:40.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:10:40.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711007+0000", 2024-09-16T00:10:40.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1471000000000001e-05 2024-09-16T00:10:40.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711014+0000", 2024-09-16T00:10:40.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7780000000000002e-06 2024-09-16T00:10:40.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724471+0000", 2024-09-16T00:10:40.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013457435 2024-09-16T00:10:40.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724547+0000", 2024-09-16T00:10:40.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5383999999999995e-05 2024-09-16T00:10:40.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724663+0000", 2024-09-16T00:10:40.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116348 2024-09-16T00:10:40.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731612+0000", 2024-09-16T00:10:40.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069491409999999998 2024-09-16T00:10:40.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732573+0000", 2024-09-16T00:10:40.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0009611 2024-09-16T00:10:40.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732638+0000", 2024-09-16T00:10:40.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919000000000006e-05 2024-09-16T00:10:40.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732678+0000", 2024-09-16T00:10:40.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9230999999999998e-05 2024-09-16T00:10:40.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.391 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.391 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.391 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4193 2.18 2:1cc859d9:::200.00000012:head [write 944447~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.391 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:40.391 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.582669199000001, 2024-09-16T00:10:40.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021286851999999998, 2024-09-16T00:10:40.391 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.391 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.391 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.392 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:40.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:40.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711536+0000", 2024-09-16T00:10:40.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0189999999999999e-06 2024-09-16T00:10:40.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711542+0000", 2024-09-16T00:10:40.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7169999999999996e-06 2024-09-16T00:10:40.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711554+0000", 2024-09-16T00:10:40.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2089000000000001e-05 2024-09-16T00:10:40.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711559+0000", 2024-09-16T00:10:40.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3739999999999997e-06 2024-09-16T00:10:40.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725311+0000", 2024-09-16T00:10:40.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013751559 2024-09-16T00:10:40.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725390+0000", 2024-09-16T00:10:40.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9779999999999998e-05 2024-09-16T00:10:40.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725509+0000", 2024-09-16T00:10:40.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118973 2024-09-16T00:10:40.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731617+0000", 2024-09-16T00:10:40.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061079000000000003 2024-09-16T00:10:40.395 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.395 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.395 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.395 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732762+0000", 2024-09-16T00:10:40.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011449769999999999 2024-09-16T00:10:40.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732796+0000", 2024-09-16T00:10:40.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3587999999999998e-05 2024-09-16T00:10:40.396 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.396 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.396 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.396 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732822+0000", 2024-09-16T00:10:40.396 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5876000000000001e-05 2024-09-16T00:10:40.396 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.396 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.397 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.397 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.397 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4194 2.18 2:1cc859d9:::200.00000012:head [write 946356~2929 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.397 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:40.397 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.581694167999999, 2024-09-16T00:10:40.397 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020453244999999998, 2024-09-16T00:10:40.397 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.397 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.397 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.397 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.397 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.397 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:40.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:40.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.398 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.398 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712511+0000", 2024-09-16T00:10:40.398 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0999999999999997e-07 2024-09-16T00:10:40.398 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.398 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712517+0000", 2024-09-16T00:10:40.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8370000000000002e-06 2024-09-16T00:10:40.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712528+0000", 2024-09-16T00:10:40.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1528e-05 2024-09-16T00:10:40.399 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.399 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.399 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.399 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712536+0000", 2024-09-16T00:10:40.399 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3939999999999998e-06 2024-09-16T00:10:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726004+0000", 2024-09-16T00:10:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013468177 2024-09-16T00:10:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726085+0000", 2024-09-16T00:10:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1626999999999996e-05 2024-09-16T00:10:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.400 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726205+0000", 2024-09-16T00:10:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119751 2024-09-16T00:10:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731622+0000", 2024-09-16T00:10:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054165419999999999 2024-09-16T00:10:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732867+0000", 2024-09-16T00:10:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001244855 2024-09-16T00:10:40.401 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732924+0000", 2024-09-16T00:10:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7395999999999997e-05 2024-09-16T00:10:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732963+0000", 2024-09-16T00:10:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9227999999999998e-05 2024-09-16T00:10:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.402 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4195 2.18 2:1cc859d9:::200.00000012:head [write 949285~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.580642507999997, 2024-09-16T00:10:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019498099000000001, 2024-09-16T00:10:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.403 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713563+0000", 2024-09-16T00:10:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T00:10:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.404 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713569+0000", 2024-09-16T00:10:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8710000000000002e-06 2024-09-16T00:10:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713582+0000", 2024-09-16T00:10:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3134e-05 2024-09-16T00:10:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713591+0000", 2024-09-16T00:10:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9889999999999998e-06 2024-09-16T00:10:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.405 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726786+0000", 2024-09-16T00:10:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013194991999999999 2024-09-16T00:10:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726856+0000", 2024-09-16T00:10:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0017000000000004e-05 2024-09-16T00:10:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726967+0000", 2024-09-16T00:10:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111815 2024-09-16T00:10:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.406 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732530+0000", 2024-09-16T00:10:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055624940000000003 2024-09-16T00:10:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733001+0000", 2024-09-16T00:10:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000470817 2024-09-16T00:10:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733030+0000", 2024-09-16T00:10:40.407 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9051e-05 2024-09-16T00:10:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733060+0000", 2024-09-16T00:10:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9819e-05 2024-09-16T00:10:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4215 2.18 2:1cc859d9:::200.00000012:head [write 991028~1910 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.560946797, 2024-09-16T00:10:40.408 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019416676000000001, 2024-09-16T00:10:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.409 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733258+0000", 2024-09-16T00:10:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:10:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733264+0000", 2024-09-16T00:10:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4090000000000002e-06 2024-09-16T00:10:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.410 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733273+0000", 2024-09-16T00:10:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4769999999999992e-06 2024-09-16T00:10:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733280+0000", 2024-09-16T00:10:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2989999999999999e-06 2024-09-16T00:10:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.742966+0000", 2024-09-16T00:10:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009685585 2024-09-16T00:10:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.411 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743046+0000", 2024-09-16T00:10:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0469999999999994e-05 2024-09-16T00:10:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743165+0000", 2024-09-16T00:10:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118847 2024-09-16T00:10:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.746968+0000", 2024-09-16T00:10:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038025099999999998 2024-09-16T00:10:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.412 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752559+0000", 2024-09-16T00:10:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055911290000000002 2024-09-16T00:10:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752634+0000", 2024-09-16T00:10:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5102e-05 2024-09-16T00:10:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752674+0000", 2024-09-16T00:10:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9878e-05 2024-09-16T00:10:40.413 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4216 2.18 2:1cc859d9:::200.00000012:head [write 992938~2755 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.560400889, 2024-09-16T00:10:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019035976, 2024-09-16T00:10:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.414 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733804+0000", 2024-09-16T00:10:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2299999999999999e-07 2024-09-16T00:10:40.415 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733810+0000", 2024-09-16T00:10:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7259999999999999e-06 2024-09-16T00:10:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733822+0000", 2024-09-16T00:10:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2204e-05 2024-09-16T00:10:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733829+0000", 2024-09-16T00:10:40.416 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3889999999999999e-06 2024-09-16T00:10:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743675+0000", 2024-09-16T00:10:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098457990000000006 2024-09-16T00:10:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743752+0000", 2024-09-16T00:10:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6279000000000006e-05 2024-09-16T00:10:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.417 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743867+0000", 2024-09-16T00:10:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115319 2024-09-16T00:10:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.746971+0000", 2024-09-16T00:10:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031042399999999999 2024-09-16T00:10:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752723+0000", 2024-09-16T00:10:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057518329999999996 2024-09-16T00:10:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.418 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752804+0000", 2024-09-16T00:10:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0822999999999995e-05 2024-09-16T00:10:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752839+0000", 2024-09-16T00:10:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5441000000000002e-05 2024-09-16T00:10:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.419 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.419 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:40.421 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:10:40.421 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:40.420+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:40.522 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:40.521+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:40.597 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:40.597 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:40.597 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:40.597 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:40.607 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:10:40.623 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:40.622+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:40.724 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:40.723+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:40.778 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:40.779 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:40.779 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:40.779 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:40.779 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:40.790 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:10:40.834 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:40.833+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:40.916 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5178 3.2c 3:375e1b9b:::100000030c8.00000002:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.10517566, 2024-09-16T00:10:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012118438, 2024-09-16T00:10:40.916 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808909+0000", 2024-09-16T00:10:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8160000000000001e-06 2024-09-16T00:10:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808921+0000", 2024-09-16T00:10:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1117999999999999e-05 2024-09-16T00:10:40.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808941+0000", 2024-09-16T00:10:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0712000000000001e-05 2024-09-16T00:10:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808952+0000", 2024-09-16T00:10:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0193e-05 2024-09-16T00:10:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809019+0000", 2024-09-16T00:10:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7404999999999996e-05 2024-09-16T00:10:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809100+0000", 2024-09-16T00:10:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1029e-05 2024-09-16T00:10:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809206+0000", 2024-09-16T00:10:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105603 2024-09-16T00:10:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.815162+0000", 2024-09-16T00:10:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059567250000000004 2024-09-16T00:10:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.820925+0000", 2024-09-16T00:10:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057623479999999996 2024-09-16T00:10:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.820987+0000", 2024-09-16T00:10:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2680000000000003e-05 2024-09-16T00:10:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.821026+0000", 2024-09-16T00:10:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8809000000000001e-05 2024-09-16T00:10:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5196 3.26 3:66ea4dfb:::100000030f0.00000001:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:40.924 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.103988631999997, 2024-09-16T00:10:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095222459999999998, 2024-09-16T00:10:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:40.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3499999999999995e-07 2024-09-16T00:10:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810102+0000", 2024-09-16T00:10:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7390000000000002e-06 2024-09-16T00:10:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810112+0000", 2024-09-16T00:10:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7550000000000006e-06 2024-09-16T00:10:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810127+0000", 2024-09-16T00:10:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4749999999999999e-05 2024-09-16T00:10:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810184+0000", 2024-09-16T00:10:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7701999999999997e-05 2024-09-16T00:10:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810256+0000", 2024-09-16T00:10:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1595999999999997e-05 2024-09-16T00:10:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810346+0000", 2024-09-16T00:10:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0043999999999994e-05 2024-09-16T00:10:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.815239+0000", 2024-09-16T00:10:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048934470000000004 2024-09-16T00:10:40.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819526+0000", 2024-09-16T00:10:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042868630000000001 2024-09-16T00:10:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819577+0000", 2024-09-16T00:10:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0445999999999998e-05 2024-09-16T00:10:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819617+0000", 2024-09-16T00:10:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0169000000000001e-05 2024-09-16T00:10:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5244 3.20 3:068a06d7:::100000032f2.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.086893125000003, 2024-09-16T00:10:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094083010000000009, 2024-09-16T00:10:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:40.930 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827191+0000", 2024-09-16T00:10:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2330000000000001e-06 2024-09-16T00:10:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827195+0000", 2024-09-16T00:10:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8750000000000002e-06 2024-09-16T00:10:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827206+0000", 2024-09-16T00:10:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0473e-05 2024-09-16T00:10:40.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827213+0000", 2024-09-16T00:10:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:10:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827275+0000", 2024-09-16T00:10:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2532000000000004e-05 2024-09-16T00:10:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827392+0000", 2024-09-16T00:10:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116266 2024-09-16T00:10:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.836599+0000", 2024-09-16T00:10:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092068069999999991 2024-09-16T00:10:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5252 2.1a 2:5913b086:::1000000336d.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:40.934 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 38.085957301999997, 2024-09-16T00:10:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010086441999999999, 2024-09-16T00:10:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828127+0000", 2024-09-16T00:10:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:10:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828140+0000", 2024-09-16T00:10:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3614e-05 2024-09-16T00:10:40.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828149+0000", 2024-09-16T00:10:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7250000000000003e-06 2024-09-16T00:10:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828156+0000", 2024-09-16T00:10:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9430000000000003e-06 2024-09-16T00:10:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828274+0000", 2024-09-16T00:10:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118121 2024-09-16T00:10:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828381+0000", 2024-09-16T00:10:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107379 2024-09-16T00:10:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.838212+0000", 2024-09-16T00:10:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098310089999999999 2024-09-16T00:10:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5743 3.36 3:6c273be3:::1000000385e.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97)", 2024-09-16T00:10:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:03.854364+0000", 2024-09-16T00:10:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.059719459, 2024-09-16T00:10:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099844070000000007, 2024-09-16T00:10:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:40.939 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854364+0000", 2024-09-16T00:10:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854364+0000", 2024-09-16T00:10:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854365+0000", 2024-09-16T00:10:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3399999999999999e-07 2024-09-16T00:10:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854368+0000", 2024-09-16T00:10:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4300000000000002e-06 2024-09-16T00:10:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854376+0000", 2024-09-16T00:10:40.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2770000000000005e-06 2024-09-16T00:10:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854384+0000", 2024-09-16T00:10:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7200000000000006e-06 2024-09-16T00:10:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854455+0000", 2024-09-16T00:10:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0631999999999997e-05 2024-09-16T00:10:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854560+0000", 2024-09-16T00:10:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105338 2024-09-16T00:10:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.864348+0000", 2024-09-16T00:10:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097883759999999997 2024-09-16T00:10:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:6601 3.35 3:af7d55b1:::10000003232.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100)", 2024-09-16T00:10:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:06.866600+0000", 2024-09-16T00:10:40.943 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.047483014999997, 2024-09-16T00:10:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092432980000000005, 2024-09-16T00:10:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866600+0000", 2024-09-16T00:10:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866600+0000", 2024-09-16T00:10:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866602+0000", 2024-09-16T00:10:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.649e-06 2024-09-16T00:10:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866607+0000", 2024-09-16T00:10:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7330000000000003e-06 2024-09-16T00:10:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866618+0000", 2024-09-16T00:10:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1506e-05 2024-09-16T00:10:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866626+0000", 2024-09-16T00:10:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.498e-06 2024-09-16T00:10:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866685+0000", 2024-09-16T00:10:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8950000000000003e-05 2024-09-16T00:10:40.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866809+0000", 2024-09-16T00:10:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012389899999999999 2024-09-16T00:10:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875844+0000", 2024-09-16T00:10:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090350629999999994 2024-09-16T00:10:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:8541 3.3f e104/94 3:fd643af1:::10000001465.00000000:head v 104'282, mlcod=103'280)", 2024-09-16T00:10:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:12.911145+0000", 2024-09-16T00:10:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.002938435000001, 2024-09-16T00:10:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094161980000000006, 2024-09-16T00:10:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911145+0000", 2024-09-16T00:10:40.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911145+0000", 2024-09-16T00:10:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911146+0000", 2024-09-16T00:10:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0699999999999996e-07 2024-09-16T00:10:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911151+0000", 2024-09-16T00:10:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8710000000000002e-06 2024-09-16T00:10:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911161+0000", 2024-09-16T00:10:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9350000000000006e-06 2024-09-16T00:10:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911168+0000", 2024-09-16T00:10:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6340000000000003e-06 2024-09-16T00:10:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.950 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911238+0000", 2024-09-16T00:10:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0118999999999997e-05 2024-09-16T00:10:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911258+0000", 2024-09-16T00:10:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9361999999999999e-05 2024-09-16T00:10:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.920518+0000", 2024-09-16T00:10:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092603249999999998 2024-09-16T00:10:40.951 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.920561+0000", 2024-09-16T00:10:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3244999999999997e-05 2024-09-16T00:10:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:8587 3.35 3:ad562626:::10000001599.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104)", 2024-09-16T00:10:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:12.911217+0000", 2024-09-16T00:10:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 28.002866507, 2024-09-16T00:10:40.952 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095415759999999995, 2024-09-16T00:10:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911217+0000", 2024-09-16T00:10:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911217+0000", 2024-09-16T00:10:40.953 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911218+0000", 2024-09-16T00:10:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:10:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911222+0000", 2024-09-16T00:10:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.301e-06 2024-09-16T00:10:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.954 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911232+0000", 2024-09-16T00:10:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0244000000000001e-05 2024-09-16T00:10:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911239+0000", 2024-09-16T00:10:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9e-06 2024-09-16T00:10:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911324+0000", 2024-09-16T00:10:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5222999999999994e-05 2024-09-16T00:10:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.955 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911496+0000", 2024-09-16T00:10:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000172065 2024-09-16T00:10:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911600+0000", 2024-09-16T00:10:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000104002 2024-09-16T00:10:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.916599+0000", 2024-09-16T00:10:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004998911 2024-09-16T00:10:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.956 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.920663+0000", 2024-09-16T00:10:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040635799999999998 2024-09-16T00:10:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.920715+0000", 2024-09-16T00:10:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2549000000000001e-05 2024-09-16T00:10:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.920758+0000", 2024-09-16T00:10:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3018e-05 2024-09-16T00:10:40.957 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:11691 3.35 3:ad66404a:::10000002d4d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111)", 2024-09-16T00:10:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:22.048217+0000", 2024-09-16T00:10:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.865866819000001, 2024-09-16T00:10:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010683378, 2024-09-16T00:10:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.958 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048217+0000", 2024-09-16T00:10:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048217+0000", 2024-09-16T00:10:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048218+0000", 2024-09-16T00:10:40.959 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.722e-06 2024-09-16T00:10:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048223+0000", 2024-09-16T00:10:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0590000000000002e-06 2024-09-16T00:10:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048239+0000", 2024-09-16T00:10:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5271e-05 2024-09-16T00:10:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.960 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048246+0000", 2024-09-16T00:10:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8490000000000006e-06 2024-09-16T00:10:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048300+0000", 2024-09-16T00:10:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3109999999999998e-05 2024-09-16T00:10:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048495+0000", 2024-09-16T00:10:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000195198 2024-09-16T00:10:40.961 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048583+0000", 2024-09-16T00:10:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8472000000000003e-05 2024-09-16T00:10:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.049175+0000", 2024-09-16T00:10:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00059207800000000001 2024-09-16T00:10:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.962 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.058797+0000", 2024-09-16T00:10:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0096219329999999992 2024-09-16T00:10:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.058856+0000", 2024-09-16T00:10:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8782000000000001e-05 2024-09-16T00:10:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.058900+0000", 2024-09-16T00:10:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3903999999999998e-05 2024-09-16T00:10:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.963 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:12961 2.1a 2:595229b5:::10000000cd4.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.088496+0000", 2024-09-16T00:10:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.825587719, 2024-09-16T00:10:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013027634, 2024-09-16T00:10:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.964 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088496+0000", 2024-09-16T00:10:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088496+0000", 2024-09-16T00:10:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088496+0000", 2024-09-16T00:10:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:10:40.965 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088498+0000", 2024-09-16T00:10:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.317e-06 2024-09-16T00:10:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088506+0000", 2024-09-16T00:10:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1459999999999999e-06 2024-09-16T00:10:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088511+0000", 2024-09-16T00:10:40.966 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.541e-06 2024-09-16T00:10:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088815+0000", 2024-09-16T00:10:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030421000000000002 2024-09-16T00:10:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088988+0000", 2024-09-16T00:10:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000172356 2024-09-16T00:10:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.967 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.089076+0000", 2024-09-16T00:10:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8270999999999996e-05 2024-09-16T00:10:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.090357+0000", 2024-09-16T00:10:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001280661 2024-09-16T00:10:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.101242+0000", 2024-09-16T00:10:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010885238 2024-09-16T00:10:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.968 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.101489+0000", 2024-09-16T00:10:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000247538 2024-09-16T00:10:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.101523+0000", 2024-09-16T00:10:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.379e-05 2024-09-16T00:10:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.969 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13112 3.2c 3:36940359:::10000000d76.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.122269+0000", 2024-09-16T00:10:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.791814112000001, 2024-09-16T00:10:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010413552, 2024-09-16T00:10:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122269+0000", 2024-09-16T00:10:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.970 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122269+0000", 2024-09-16T00:10:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122270+0000", 2024-09-16T00:10:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6499999999999999e-07 2024-09-16T00:10:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122272+0000", 2024-09-16T00:10:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5189999999999999e-06 2024-09-16T00:10:40.971 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122279+0000", 2024-09-16T00:10:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7039999999999997e-06 2024-09-16T00:10:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122284+0000", 2024-09-16T00:10:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5670000000000002e-06 2024-09-16T00:10:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.972 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122835+0000", 2024-09-16T00:10:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00055031399999999999 2024-09-16T00:10:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122993+0000", 2024-09-16T00:10:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015790499999999999 2024-09-16T00:10:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.123078+0000", 2024-09-16T00:10:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5803000000000005e-05 2024-09-16T00:10:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.973 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.123884+0000", 2024-09-16T00:10:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00080534599999999997 2024-09-16T00:10:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.132579+0000", 2024-09-16T00:10:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086952409999999994 2024-09-16T00:10:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.132643+0000", 2024-09-16T00:10:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4289999999999996e-05 2024-09-16T00:10:40.974 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.132683+0000", 2024-09-16T00:10:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9397999999999998e-05 2024-09-16T00:10:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13227 3.26 3:64bfa412:::10000000dee.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.146915+0000", 2024-09-16T00:10:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.76716826, 2024-09-16T00:10:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012561259, 2024-09-16T00:10:40.975 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146915+0000", 2024-09-16T00:10:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146915+0000", 2024-09-16T00:10:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.976 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146916+0000", 2024-09-16T00:10:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.155e-06 2024-09-16T00:10:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146922+0000", 2024-09-16T00:10:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3249999999999998e-06 2024-09-16T00:10:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146938+0000", 2024-09-16T00:10:40.977 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5974999999999999e-05 2024-09-16T00:10:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146946+0000", 2024-09-16T00:10:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3189999999999995e-06 2024-09-16T00:10:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.147095+0000", 2024-09-16T00:10:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014879800000000001 2024-09-16T00:10:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.147262+0000", 2024-09-16T00:10:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000167353 2024-09-16T00:10:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.147357+0000", 2024-09-16T00:10:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5199999999999997e-05 2024-09-16T00:10:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.148582+0000", 2024-09-16T00:10:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001224679 2024-09-16T00:10:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.979 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.159363+0000", 2024-09-16T00:10:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010780839 2024-09-16T00:10:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.159422+0000", 2024-09-16T00:10:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9731999999999997e-05 2024-09-16T00:10:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.159476+0000", 2024-09-16T00:10:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3884000000000001e-05 2024-09-16T00:10:40.980 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13251 3.36 3:6f745cbe:::10000000e09.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.152775+0000", 2024-09-16T00:10:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.761308372, 2024-09-16T00:10:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011312687, 2024-09-16T00:10:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.981 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152775+0000", 2024-09-16T00:10:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152775+0000", 2024-09-16T00:10:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152776+0000", 2024-09-16T00:10:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.393e-06 2024-09-16T00:10:40.982 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152782+0000", 2024-09-16T00:10:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2329999999999997e-06 2024-09-16T00:10:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152797+0000", 2024-09-16T00:10:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5515e-05 2024-09-16T00:10:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152808+0000", 2024-09-16T00:10:40.983 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1178e-05 2024-09-16T00:10:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152860+0000", 2024-09-16T00:10:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1644999999999998e-05 2024-09-16T00:10:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.153026+0000", 2024-09-16T00:10:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016563400000000001 2024-09-16T00:10:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.984 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.153127+0000", 2024-09-16T00:10:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010173 2024-09-16T00:10:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.154610+0000", 2024-09-16T00:10:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001482917 2024-09-16T00:10:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.164000+0000", 2024-09-16T00:10:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093899110000000008 2024-09-16T00:10:40.985 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.164048+0000", 2024-09-16T00:10:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8378000000000003e-05 2024-09-16T00:10:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.164088+0000", 2024-09-16T00:10:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9153000000000003e-05 2024-09-16T00:10:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.986 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13260 3.35 3:ad57d493:::10000000e12.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.152827+0000", 2024-09-16T00:10:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.761256766000001, 2024-09-16T00:10:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012977191000000001, 2024-09-16T00:10:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152827+0000", 2024-09-16T00:10:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.987 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152827+0000", 2024-09-16T00:10:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152827+0000", 2024-09-16T00:10:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2600000000000002e-07 2024-09-16T00:10:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.988 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152830+0000", 2024-09-16T00:10:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9229999999999998e-06 2024-09-16T00:10:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152838+0000", 2024-09-16T00:10:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8790000000000003e-06 2024-09-16T00:10:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152844+0000", 2024-09-16T00:10:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7769999999999999e-06 2024-09-16T00:10:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.989 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152892+0000", 2024-09-16T00:10:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7879e-05 2024-09-16T00:10:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.153052+0000", 2024-09-16T00:10:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016057199999999999 2024-09-16T00:10:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.153139+0000", 2024-09-16T00:10:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6679000000000001e-05 2024-09-16T00:10:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.990 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.155267+0000", 2024-09-16T00:10:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021280869999999999 2024-09-16T00:10:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.165714+0000", 2024-09-16T00:10:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010447303 2024-09-16T00:10:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.165764+0000", 2024-09-16T00:10:40.991 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.024e-05 2024-09-16T00:10:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.165804+0000", 2024-09-16T00:10:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9326000000000003e-05 2024-09-16T00:10:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13359 3.35 3:afdc6484:::10000000e78.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.173189+0000", 2024-09-16T00:10:40.992 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.740894781, 2024-09-16T00:10:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011094559, 2024-09-16T00:10:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173189+0000", 2024-09-16T00:10:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.993 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173189+0000", 2024-09-16T00:10:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173190+0000", 2024-09-16T00:10:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.249e-06 2024-09-16T00:10:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173194+0000", 2024-09-16T00:10:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T00:10:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.994 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173203+0000", 2024-09-16T00:10:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3500000000000003e-06 2024-09-16T00:10:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173210+0000", 2024-09-16T00:10:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.331e-06 2024-09-16T00:10:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173371+0000", 2024-09-16T00:10:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016161999999999999 2024-09-16T00:10:40.995 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173548+0000", 2024-09-16T00:10:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000176622 2024-09-16T00:10:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173645+0000", 2024-09-16T00:10:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7345999999999999e-05 2024-09-16T00:10:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:40.996 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.175459+0000", 2024-09-16T00:10:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001814062 2024-09-16T00:10:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.184205+0000", 2024-09-16T00:10:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0087454479999999994 2024-09-16T00:10:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.184249+0000", 2024-09-16T00:10:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4026999999999997e-05 2024-09-16T00:10:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.997 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.184283+0000", 2024-09-16T00:10:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4326999999999998e-05 2024-09-16T00:10:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13435 3.2c 3:34a22dc0:::10000000ec9.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.187542+0000", 2024-09-16T00:10:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.726541768000001, 2024-09-16T00:10:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011912524000000001, 2024-09-16T00:10:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:40.998 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187542+0000", 2024-09-16T00:10:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187542+0000", 2024-09-16T00:10:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:40.999 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187542+0000", 2024-09-16T00:10:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0999999999999998e-07 2024-09-16T00:10:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187545+0000", 2024-09-16T00:10:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6620000000000001e-06 2024-09-16T00:10:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187552+0000", 2024-09-16T00:10:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4030000000000001e-06 2024-09-16T00:10:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.000 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187558+0000", 2024-09-16T00:10:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5289999999999999e-06 2024-09-16T00:10:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.188067+0000", 2024-09-16T00:10:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050960000000000003 2024-09-16T00:10:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.188230+0000", 2024-09-16T00:10:41.001 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000162275 2024-09-16T00:10:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.188323+0000", 2024-09-16T00:10:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3778999999999998e-05 2024-09-16T00:10:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.189485+0000", 2024-09-16T00:10:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011611659999999999 2024-09-16T00:10:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.002 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199354+0000", 2024-09-16T00:10:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098691790000000005 2024-09-16T00:10:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199404+0000", 2024-09-16T00:10:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.041e-05 2024-09-16T00:10:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199454+0000", 2024-09-16T00:10:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9910999999999997e-05 2024-09-16T00:10:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:41.003 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13491 3.26 3:66db0c90:::10000000f05.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.199350+0000", 2024-09-16T00:10:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 14.714733254, 2024-09-16T00:10:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009654167, 2024-09-16T00:10:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:41.004 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199350+0000", 2024-09-16T00:10:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199350+0000", 2024-09-16T00:10:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199352+0000", 2024-09-16T00:10:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.621e-06 2024-09-16T00:10:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.005 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199361+0000", 2024-09-16T00:10:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0669999999999998e-06 2024-09-16T00:10:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199375+0000", 2024-09-16T00:10:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4474999999999999e-05 2024-09-16T00:10:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199383+0000", 2024-09-16T00:10:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8250000000000005e-06 2024-09-16T00:10:41.006 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199500+0000", 2024-09-16T00:10:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011675099999999999 2024-09-16T00:10:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199662+0000", 2024-09-16T00:10:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000161955 2024-09-16T00:10:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199749+0000", 2024-09-16T00:10:41.007 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7498999999999997e-05 2024-09-16T00:10:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.202268+0000", 2024-09-16T00:10:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025186829999999999 2024-09-16T00:10:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.208868+0000", 2024-09-16T00:10:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066001870000000004 2024-09-16T00:10:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.008 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.208962+0000", 2024-09-16T00:10:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3434e-05 2024-09-16T00:10:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.209004+0000", 2024-09-16T00:10:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2670000000000003e-05 2024-09-16T00:10:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:16859 3.35 3:ad0b5202:::1000000245b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116)", 2024-09-16T00:10:41.009 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:36.173717+0000", 2024-09-16T00:10:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.7403659959999995, 2024-09-16T00:10:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097368839999999995, 2024-09-16T00:10:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173717+0000", 2024-09-16T00:10:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.010 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173717+0000", 2024-09-16T00:10:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173718+0000", 2024-09-16T00:10:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.01e-07 2024-09-16T00:10:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173720+0000", 2024-09-16T00:10:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2029999999999999e-06 2024-09-16T00:10:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.011 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173727+0000", 2024-09-16T00:10:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6209999999999998e-06 2024-09-16T00:10:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173733+0000", 2024-09-16T00:10:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4960000000000001e-06 2024-09-16T00:10:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173787+0000", 2024-09-16T00:10:41.012 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4354999999999997e-05 2024-09-16T00:10:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173953+0000", 2024-09-16T00:10:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016555000000000001 2024-09-16T00:10:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.174041+0000", 2024-09-16T00:10:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7738999999999995e-05 2024-09-16T00:10:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:41.013 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.181641+0000", 2024-09-16T00:10:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007599939 2024-09-16T00:10:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.183355+0000", 2024-09-16T00:10:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017146869999999999 2024-09-16T00:10:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.183404+0000", 2024-09-16T00:10:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8999000000000003e-05 2024-09-16T00:10:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.014 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.183454+0000", 2024-09-16T00:10:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9894e-05 2024-09-16T00:10:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:16912 3.1e e116/94 3:79d63390:::1000000249f.00000000:head v 116'374, mlcod=116'372)", 2024-09-16T00:10:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:36.184345+0000", 2024-09-16T00:10:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.729738352, 2024-09-16T00:10:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093746300000000001, 2024-09-16T00:10:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:41.015 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184345+0000", 2024-09-16T00:10:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184345+0000", 2024-09-16T00:10:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.016 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184346+0000", 2024-09-16T00:10:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6000000000000003e-07 2024-09-16T00:10:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184351+0000", 2024-09-16T00:10:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3530000000000002e-06 2024-09-16T00:10:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184360+0000", 2024-09-16T00:10:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9579999999999996e-06 2024-09-16T00:10:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.017 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184366+0000", 2024-09-16T00:10:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1279999999999996e-06 2024-09-16T00:10:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184704+0000", 2024-09-16T00:10:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00033787599999999997 2024-09-16T00:10:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184722+0000", 2024-09-16T00:10:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7533e-05 2024-09-16T00:10:41.018 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.193676+0000", 2024-09-16T00:10:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089544919999999997 2024-09-16T00:10:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.193720+0000", 2024-09-16T00:10:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3529999999999998e-05 2024-09-16T00:10:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:41.019 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:17004 2.17 2:eb27e31b:::1000000255e.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116)", 2024-09-16T00:10:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:36.199476+0000", 2024-09-16T00:10:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 4.7146076880000001, 2024-09-16T00:10:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094096179999999998, 2024-09-16T00:10:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:41.020 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199476+0000", 2024-09-16T00:10:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199476+0000", 2024-09-16T00:10:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.021 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199477+0000", 2024-09-16T00:10:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4619999999999999e-06 2024-09-16T00:10:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199484+0000", 2024-09-16T00:10:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.438e-06 2024-09-16T00:10:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199499+0000", 2024-09-16T00:10:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5512e-05 2024-09-16T00:10:41.022 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199508+0000", 2024-09-16T00:10:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7360000000000007e-06 2024-09-16T00:10:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199570+0000", 2024-09-16T00:10:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2130000000000003e-05 2024-09-16T00:10:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199691+0000", 2024-09-16T00:10:41.023 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121271 2024-09-16T00:10:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.208885+0000", 2024-09-16T00:10:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0091940689999999992 2024-09-16T00:10:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:41.024 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:41.024 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:41.026 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:41.024+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:41.026 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:10:41.129 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:41.129+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:41.136 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:41.136 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:41.136 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:41.136 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:41.145 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T00:10:41.147 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:10:41.231 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:41.231+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:41.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.288+0000 7f61374f1640 1 Processor -- start 2024-09-16T00:10:41.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.288+0000 7f61374f1640 1 -- start start 2024-09-16T00:10:41.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.289+0000 7f61374f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61301519d0 0x7f61300c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.289+0000 7f61374f1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f61300c8a80 0x7f61300c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.289+0000 7f61374f1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6130067ad0 con 0x7f61300c58f0 2024-09-16T00:10:41.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.289+0000 7f61374f1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f613006d320 con 0x7f61300c8a80 2024-09-16T00:10:41.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.289+0000 7f61374f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6130067db0 con 0x7f61301519d0 2024-09-16T00:10:41.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.289+0000 7f6135a8e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f61300c8a80 0x7f61300c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.289+0000 7f613528d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61301519d0 0x7f61300c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.289+0000 7f6134a8c640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f61300c58f0 0x7f61300c8370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:43894/0 (socket says 172.21.15.139:43894) 2024-09-16T00:10:41.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.289+0000 7f6135a8e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f61300c8a80 0x7f61300c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38590/0 (socket says 172.21.15.139:38590) 2024-09-16T00:10:41.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.289+0000 7f6134a8c640 1 -- 172.21.15.139:0/1569638532 learned_addr learned my addr 172.21.15.139:0/1569638532 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:41.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.290+0000 7f6127fff640 1 -- 172.21.15.139:0/1569638532 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1817176793 0 0) 0x7f6130067ad0 con 0x7f61300c58f0 2024-09-16T00:10:41.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.290+0000 7f6135a8e640 1 -- 172.21.15.139:0/1569638532 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61301519d0 msgr2=0x7f61300c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:41.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.290+0000 7f6135a8e640 1 --2- 172.21.15.139:0/1569638532 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61301519d0 0x7f61300c3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.290+0000 7f6135a8e640 1 -- 172.21.15.139:0/1569638532 >> v1:172.21.15.164:6789/0 conn(0x7f61300c58f0 legacy=0x7f61300c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:41.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.290+0000 7f6135a8e640 1 -- 172.21.15.139:0/1569638532 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613006ccd0 con 0x7f61300c8a80 2024-09-16T00:10:41.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.291+0000 7f6135a8e640 1 --2- 172.21.15.139:0/1569638532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61300c8a80 0x7f61300c5d30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6128009d90 tx=0x7f612802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85da54c96abae19 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:41.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.291+0000 7f6127fff640 1 -- 172.21.15.139:0/1569638532 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128007740 con 0x7f61300c8a80 2024-09-16T00:10:41.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.291+0000 7f6127fff640 1 -- 172.21.15.139:0/1569638532 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61280078e0 con 0x7f61300c8a80 2024-09-16T00:10:41.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.291+0000 7f6127fff640 1 -- 172.21.15.139:0/1569638532 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128007cb0 con 0x7f61300c8a80 2024-09-16T00:10:41.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.291+0000 7f61374f1640 1 -- 172.21.15.139:0/1569638532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61300c8a80 msgr2=0x7f61300c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:41.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.291+0000 7f61374f1640 1 --2- 172.21.15.139:0/1569638532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61300c8a80 0x7f61300c5d30 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6128009d90 tx=0x7f612802a6f0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.291+0000 7f61374f1640 1 -- 172.21.15.139:0/1569638532 shutdown_connections 2024-09-16T00:10:41.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.291+0000 7f61374f1640 1 --2- 172.21.15.139:0/1569638532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61300c8a80 0x7f61300c5d30 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.291+0000 7f61374f1640 1 --2- 172.21.15.139:0/1569638532 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61301519d0 0x7f61300c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.291+0000 7f61374f1640 1 -- 172.21.15.139:0/1569638532 >> 172.21.15.139:0/1569638532 conn(0x7f61300c92d0 msgr2=0x7f61300c96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:41.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.292+0000 7f61374f1640 1 -- 172.21.15.139:0/1569638532 shutdown_connections 2024-09-16T00:10:41.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.292+0000 7f61374f1640 1 -- 172.21.15.139:0/1569638532 wait complete. 2024-09-16T00:10:41.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.293+0000 7f61374f1640 1 Processor -- start 2024-09-16T00:10:41.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.293+0000 7f61374f1640 1 -- start start 2024-09-16T00:10:41.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.293+0000 7f61374f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61300c58f0 0x7f61301f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.293+0000 7f61374f1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61300c8a80 0x7f6130157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.293+0000 7f61374f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61301519d0 0x7f6130152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.293+0000 7f61374f1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f61300c4210 con 0x7f61300c8a80 2024-09-16T00:10:41.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.293+0000 7f61374f1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f61300c6ba0 con 0x7f61301519d0 2024-09-16T00:10:41.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.293+0000 7f61374f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6130151db0 con 0x7f61300c58f0 2024-09-16T00:10:41.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.294+0000 7f6134a8c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61300c8a80 0x7f6130157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.294+0000 7f613528d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61300c58f0 0x7f61301f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.294+0000 7f613528d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61300c58f0 0x7f61301f1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:60228/0 (socket says 172.21.15.139:60228) 2024-09-16T00:10:41.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.294+0000 7f613528d640 1 -- 172.21.15.139:0/3033023705 learned_addr learned my addr 172.21.15.139:0/3033023705 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:41.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.294+0000 7f6135a8e640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61301519d0 0x7f6130152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.294+0000 7f613528d640 1 -- 172.21.15.139:0/3033023705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61301519d0 msgr2=0x7f6130152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:41.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.294+0000 7f613528d640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f61301519d0 0x7f6130152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.294+0000 7f613528d640 1 -- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61300c8a80 msgr2=0x7f6130157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:41.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.294+0000 7f613528d640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61300c8a80 0x7f6130157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.294+0000 7f613528d640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61300ca6b0 con 0x7f61300c58f0 2024-09-16T00:10:41.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.294+0000 7f613528d640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f61300c58f0 0x7f61301f1d10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f612c009b20 tx=0x7f612c00d160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:41.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.294+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c0103a0 con 0x7f61300c58f0 2024-09-16T00:10:41.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.294+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f612c00dae0 con 0x7f61300c58f0 2024-09-16T00:10:41.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.294+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61300ca970 con 0x7f61300c58f0 2024-09-16T00:10:41.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.294+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c00de90 con 0x7f61300c58f0 2024-09-16T00:10:41.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.295+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f61300caf30 con 0x7f61300c58f0 2024-09-16T00:10:41.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.295+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f61300ca020 con 0x7f61300c58f0 2024-09-16T00:10:41.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.296+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f612c019850 con 0x7f61300c58f0 2024-09-16T00:10:41.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.297+0000 7f6125ffb640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f611406c450 0x7f611406e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.297+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(118..118 src has 1..118) ==== 5973+0+0 (secure 0 0 0) 0x7f612c093a50 con 0x7f61300c58f0 2024-09-16T00:10:41.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.297+0000 7f6134a8c640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f611406c450 0x7f611406e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.297+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v118) ==== 54+0+16 (secure 0 0 0) 0x7f612c01e030 con 0x7f61300c58f0 2024-09-16T00:10:41.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.297+0000 7f61374f1640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f613014de00 0x7f61301520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.297+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f61301df800 con 0x7f613014de00 2024-09-16T00:10:41.301 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.297+0000 7f6134a8c640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f611406c450 0x7f611406e910 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6130150010 tx=0x7f6120007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:41.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.297+0000 7f6135a8e640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f613014de00 0x7f61301520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.302 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.297+0000 7f6135a8e640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7f613014de00 0x7f61301520e0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:41.302 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:41.302 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:41.302 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:41.302 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:41.302 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:41.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.298+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f61301df800 con 0x7f613014de00 2024-09-16T00:10:41.309 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:10:41.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.314+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f61300008d0 con 0x7f613014de00 2024-09-16T00:10:41.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.316+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f61300008d0 con 0x7f613014de00 2024-09-16T00:10:41.317 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:41.317 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:41.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.316+0000 7f61374f1640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f613014e1e0 0x7f613014d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.316+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f6130158500 con 0x7f613014e1e0 2024-09-16T00:10:41.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.316+0000 7f613528d640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f613014e1e0 0x7f613014d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.316+0000 7f613528d640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f613014e1e0 0x7f613014d040 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:41.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.317+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6130158500 con 0x7f613014e1e0 2024-09-16T00:10:41.331 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:41.330+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:41.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.333+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6130152620 con 0x7f613014e1e0 2024-09-16T00:10:41.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.334+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6130152620 con 0x7f613014e1e0 2024-09-16T00:10:41.338 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:41.340 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:41.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.335+0000 7f61374f1640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f61301ee4f0 0x7f6130062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.335+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f61301ee950 con 0x7f61301ee4f0 2024-09-16T00:10:41.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.335+0000 7f6134a8c640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f61301ee4f0 0x7f6130062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.335+0000 7f6134a8c640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f61301ee4f0 0x7f6130062630 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:41.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.336+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f61301ee950 con 0x7f61301ee4f0 2024-09-16T00:10:41.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.351+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f61301ee950 con 0x7f61301ee4f0 2024-09-16T00:10:41.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.352+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f61301ee950 con 0x7f61301ee4f0 2024-09-16T00:10:41.353 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:41.354 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:41.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.353+0000 7f61374f1640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f61301e1a10 0x7f61301e1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.354 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.353+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f61301ee950 con 0x7f61301e1a10 2024-09-16T00:10:41.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.353+0000 7f6135a8e640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f61301e1a10 0x7f61301e1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.353+0000 7f6135a8e640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f61301e1a10 0x7f61301e1e70 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:41.355 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.354+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f61301ee950 con 0x7f61301e1a10 2024-09-16T00:10:41.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.369+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f61301ee950 con 0x7f61301e1a10 2024-09-16T00:10:41.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.371+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f61301ee950 con 0x7f61301e1a10 2024-09-16T00:10:41.372 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:41.373 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:41.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.371+0000 7f61374f1640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f613002c640 0x7f613002ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.371+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f61301ee950 con 0x7f613002c640 2024-09-16T00:10:41.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.372+0000 7f613528d640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f613002c640 0x7f613002ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.372+0000 7f613528d640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f613002c640 0x7f613002ca80 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:41.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.373+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f61301ee950 con 0x7f613002c640 2024-09-16T00:10:41.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.388+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f61301ee950 con 0x7f613002c640 2024-09-16T00:10:41.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.390+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f61301ee950 con 0x7f613002c640 2024-09-16T00:10:41.391 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:41.391 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:41.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.390+0000 7f61374f1640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f613002d4d0 0x7f61302026c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.390+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f61301e2990 con 0x7f613002d4d0 2024-09-16T00:10:41.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.390+0000 7f6134a8c640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f613002d4d0 0x7f61302026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.391+0000 7f6134a8c640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f613002d4d0 0x7f61302026c0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:41.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.391+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f61301e2990 con 0x7f613002d4d0 2024-09-16T00:10:41.407 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.406+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f61301e2990 con 0x7f613002d4d0 2024-09-16T00:10:41.408 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.408+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f61301e2990 con 0x7f613002d4d0 2024-09-16T00:10:41.408 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:41.409 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:41.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.408+0000 7f61374f1640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f6130203110 0x7f6130205500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.408+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f613002db90 con 0x7f6130203110 2024-09-16T00:10:41.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.408+0000 7f6135a8e640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f6130203110 0x7f6130205500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.409 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.408+0000 7f6135a8e640 1 --2- 172.21.15.139:0/3033023705 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f6130203110 0x7f6130205500 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:41.410 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.409+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6128007e10 con 0x7f6130203110 2024-09-16T00:10:41.422 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:41.423 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:41.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.424+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f6130205d10 con 0x7f6130203110 2024-09-16T00:10:41.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.426+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f6130205d10 con 0x7f6130203110 2024-09-16T00:10:41.426 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:41.427 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:41.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.426+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=119}) -- 0x7f6130207630 con 0x7f61300c58f0 2024-09-16T00:10:41.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.426+0000 7f61374f1640 1 -- 172.21.15.139:0/3033023705 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6130205d10 con 0x7f61300c58f0 2024-09-16T00:10:41.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.426+0000 7f6125ffb640 1 -- 172.21.15.139:0/3033023705 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f612c027040 con 0x7f61300c58f0 2024-09-16T00:10:41.427 INFO:teuthology.orchestra.run.smithi139.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T00:10:41.430 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T00:10:41.431 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T00:10:41.432 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:10:41.472 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:41.471+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:41.533 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:41.532+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:41.538 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-16T00:10:41.584 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:41.585 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:41.594 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:10:41.638 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:41.637+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:41.644 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [7] live_osds: [0, 1, 2, 3, 4, 5, 6] 2024-09-16T00:10:41.645 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T00:10:41.645 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T00:10:41.645 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-16T00:10:41.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.646+0000 7f6fb40a6640 1 Processor -- start 2024-09-16T00:10:41.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.646+0000 7f6fb40a6640 1 -- start start 2024-09-16T00:10:41.650 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.647+0000 7f6fb40a6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fac068060 0x7f6fac15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.647+0000 7f6fb40a6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fac0cc4b0 con 0x7f6fac0ccc90 2024-09-16T00:10:41.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.647+0000 7f6fb40a6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fac0cc790 con 0x7f6fac154c20 2024-09-16T00:10:41.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.647+0000 7f6fb40a6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6fac0c5830 con 0x7f6fac068060 2024-09-16T00:10:41.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.647+0000 7f6fb2643640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f6fac0ccc90 0x7f6fac165180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:40450/0 (socket says 172.21.15.139:40450) 2024-09-16T00:10:41.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.647+0000 7f6fb2643640 1 -- 172.21.15.139:0/2514320053 learned_addr learned my addr 172.21.15.139:0/2514320053 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:41.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.647+0000 7f6fb1641640 1 --2- 172.21.15.139:0/2514320053 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fac068060 0x7f6fac15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.647+0000 7f6fb0e40640 1 -- 172.21.15.139:0/2514320053 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3634192440 0 0) 0x7f6fac0cc4b0 con 0x7f6fac0ccc90 2024-09-16T00:10:41.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.647+0000 7f6fb0e40640 1 -- 172.21.15.139:0/2514320053 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f98003520 con 0x7f6fac0ccc90 2024-09-16T00:10:41.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.647+0000 7f6fb0e40640 1 -- 172.21.15.139:0/2514320053 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 744602502 0 0) 0x7f6fac0cc790 con 0x7f6fac154c20 2024-09-16T00:10:41.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.647+0000 7f6fb0e40640 1 -- 172.21.15.139:0/2514320053 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fac0cc4b0 con 0x7f6fac154c20 2024-09-16T00:10:41.651 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.648+0000 7f6fb1641640 1 -- 172.21.15.139:0/2514320053 >> v1:172.21.15.164:6789/0 conn(0x7f6fac154c20 legacy=0x7f6fac0c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:41.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.648+0000 7f6fb1641640 1 -- 172.21.15.139:0/2514320053 >> v1:172.21.15.139:6789/0 conn(0x7f6fac0ccc90 legacy=0x7f6fac165180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:41.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.648+0000 7f6fb1641640 1 -- 172.21.15.139:0/2514320053 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fac15ddf0 con 0x7f6fac068060 2024-09-16T00:10:41.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.648+0000 7f6fb1641640 1 --2- 172.21.15.139:0/2514320053 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fac068060 0x7f6fac15b280 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0097c0 tx=0x7f6f9c02a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=636215f7eb9cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:41.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.649+0000 7f6fb0e40640 1 -- 172.21.15.139:0/2514320053 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c037070 con 0x7f6fac068060 2024-09-16T00:10:41.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.649+0000 7f6fb0e40640 1 -- 172.21.15.139:0/2514320053 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f9c032440 con 0x7f6fac068060 2024-09-16T00:10:41.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.649+0000 7f6fb0e40640 1 -- 172.21.15.139:0/2514320053 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0327f0 con 0x7f6fac068060 2024-09-16T00:10:41.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.649+0000 7f6fb40a6640 1 -- 172.21.15.139:0/2514320053 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fac068060 msgr2=0x7f6fac15b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:41.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.649+0000 7f6fb40a6640 1 --2- 172.21.15.139:0/2514320053 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fac068060 0x7f6fac15b280 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0097c0 tx=0x7f6f9c02a290 comp rx=0 tx=0).stop 2024-09-16T00:10:41.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.649+0000 7f6fb40a6640 1 -- 172.21.15.139:0/2514320053 shutdown_connections 2024-09-16T00:10:41.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.649+0000 7f6fb40a6640 1 --2- 172.21.15.139:0/2514320053 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fac068060 0x7f6fac15b280 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.649+0000 7f6fb40a6640 1 -- 172.21.15.139:0/2514320053 >> 172.21.15.139:0/2514320053 conn(0x7f6fac06ca40 msgr2=0x7f6fac067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:41.652 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.649+0000 7f6fb40a6640 1 -- 172.21.15.139:0/2514320053 shutdown_connections 2024-09-16T00:10:41.653 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.650+0000 7f6fb40a6640 1 -- 172.21.15.139:0/2514320053 wait complete. 2024-09-16T00:10:41.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.651+0000 7f6fb40a6640 1 Processor -- start 2024-09-16T00:10:41.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.651+0000 7f6fb40a6640 1 -- start start 2024-09-16T00:10:41.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb40a6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fac068060 0x7f6fac0d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb40a6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fac0ccc90 0x7f6fac15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb40a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fac154c20 0x7f6fac1591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb40a6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac0cc260 con 0x7f6fac154c20 2024-09-16T00:10:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb40a6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac15e3b0 con 0x7f6fac068060 2024-09-16T00:10:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb40a6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6fac15e690 con 0x7f6fac0ccc90 2024-09-16T00:10:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb2643640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fac154c20 0x7f6fac1591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb1e42640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fac068060 0x7f6fac0d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb1641640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fac0ccc90 0x7f6fac15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb2643640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fac154c20 0x7f6fac1591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38604/0 (socket says 172.21.15.139:38604) 2024-09-16T00:10:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb1e42640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fac068060 0x7f6fac0d0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:53386/0 (socket says 172.21.15.139:53386) 2024-09-16T00:10:41.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb1641640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fac0ccc90 0x7f6fac15cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:60260/0 (socket says 172.21.15.139:60260) 2024-09-16T00:10:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb2643640 1 -- 172.21.15.139:0/509570773 learned_addr learned my addr 172.21.15.139:0/509570773 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb2643640 1 -- 172.21.15.139:0/509570773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fac0ccc90 msgr2=0x7f6fac15cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb2643640 1 --2- 172.21.15.139:0/509570773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fac0ccc90 0x7f6fac15cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb2643640 1 -- 172.21.15.139:0/509570773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fac068060 msgr2=0x7f6fac0d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb2643640 1 --2- 172.21.15.139:0/509570773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fac068060 0x7f6fac0d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb2643640 1 -- 172.21.15.139:0/509570773 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fac1f1b50 con 0x7f6fac154c20 2024-09-16T00:10:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.652+0000 7f6fb2643640 1 --2- 172.21.15.139:0/509570773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fac154c20 0x7f6fac1591f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400caa0 tx=0x7f6fa400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:41.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.653+0000 7f6fa27fc640 1 -- 172.21.15.139:0/509570773 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4017070 con 0x7f6fac154c20 2024-09-16T00:10:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.653+0000 7f6fa27fc640 1 -- 172.21.15.139:0/509570773 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa400ac20 con 0x7f6fac154c20 2024-09-16T00:10:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.653+0000 7f6fb40a6640 1 -- 172.21.15.139:0/509570773 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fac158710 con 0x7f6fac154c20 2024-09-16T00:10:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.653+0000 7f6fa27fc640 1 -- 172.21.15.139:0/509570773 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4012640 con 0x7f6fac154c20 2024-09-16T00:10:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.653+0000 7f6fb40a6640 1 -- 172.21.15.139:0/509570773 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fac14fd30 con 0x7f6fac154c20 2024-09-16T00:10:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.654+0000 7f6fb40a6640 1 -- 172.21.15.139:0/509570773 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fac075fc0 con 0x7f6fac154c20 2024-09-16T00:10:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.655+0000 7f6fa27fc640 1 -- 172.21.15.139:0/509570773 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6fa401a070 con 0x7f6fac154c20 2024-09-16T00:10:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.655+0000 7f6fa27fc640 1 --2- 172.21.15.139:0/509570773 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6f9006c2e0 0x7f6f9006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.655+0000 7f6fa27fc640 1 -- 172.21.15.139:0/509570773 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5973+0+0 (secure 0 0 0) 0x7f6fa408b1a0 con 0x7f6fac154c20 2024-09-16T00:10:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.655+0000 7f6fb1e42640 1 --2- 172.21.15.139:0/509570773 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6f9006c2e0 0x7f6f9006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.656+0000 7f6fb1e42640 1 --2- 172.21.15.139:0/509570773 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6f9006c2e0 0x7f6f9006e7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80011e0 tx=0x7f6fa8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:41.661 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.661+0000 7f6fa27fc640 1 -- 172.21.15.139:0/509570773 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6fac075fc0 con 0x7f6fac154c20 2024-09-16T00:10:41.709 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:41.709 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:41.719 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:10:41.735 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:41.734+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:41.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.752+0000 7ff99bda6640 1 Processor -- start 2024-09-16T00:10:41.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.752+0000 7ff99bda6640 1 -- start start 2024-09-16T00:10:41.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.753+0000 7ff99bda6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff994071540 0x7ff9940c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.753+0000 7ff99bda6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9940c8bc0 0x7ff994071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.753+0000 7ff99bda6640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff99415e280 0x7ff99415e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.753+0000 7ff99bda6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff99415ddf0 con 0x7ff994071540 2024-09-16T00:10:41.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.753+0000 7ff99bda6640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7ff9940cb930 con 0x7ff99415e280 2024-09-16T00:10:41.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.753+0000 7ff99bda6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff99406cdd0 con 0x7ff9940c8bc0 2024-09-16T00:10:41.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.753+0000 7ff999b42640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff994071540 0x7ff9940c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.753+0000 7ff999b42640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff994071540 0x7ff9940c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38612/0 (socket says 172.21.15.139:38612) 2024-09-16T00:10:41.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.753+0000 7ff999b42640 1 -- 172.21.15.139:0/3753567672 learned_addr learned my addr 172.21.15.139:0/3753567672 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:41.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.753+0000 7ff999341640 1 --2- 172.21.15.139:0/3753567672 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9940c8bc0 0x7ff994071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.753+0000 7ff99a343640 1 --2- 172.21.15.139:0/3753567672 >> v2:172.21.15.164:3300/0 conn(0x7ff99415e280 0x7ff99415e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.754+0000 7ff999b42640 1 -- 172.21.15.139:0/3753567672 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9940c8bc0 msgr2=0x7ff994071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:41.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.754+0000 7ff999b42640 1 --2- 172.21.15.139:0/3753567672 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9940c8bc0 0x7ff994071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.754+0000 7ff999b42640 1 -- 172.21.15.139:0/3753567672 >> v2:172.21.15.164:3300/0 conn(0x7ff99415e280 msgr2=0x7ff99415e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:41.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.754+0000 7ff999b42640 1 --2- 172.21.15.139:0/3753567672 >> v2:172.21.15.164:3300/0 conn(0x7ff99415e280 0x7ff99415e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.754+0000 7ff999b42640 1 -- 172.21.15.139:0/3753567672 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99406d0b0 con 0x7ff994071540 2024-09-16T00:10:41.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.754+0000 7ff999b42640 1 --2- 172.21.15.139:0/3753567672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff994071540 0x7ff9940c9d20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff994074320 tx=0x7ff99002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649d505d159a8155 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:41.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.754+0000 7ff998b40640 1 -- 172.21.15.139:0/3753567672 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990037070 con 0x7ff994071540 2024-09-16T00:10:41.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.755+0000 7ff998b40640 1 -- 172.21.15.139:0/3753567672 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff990032440 con 0x7ff994071540 2024-09-16T00:10:41.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.755+0000 7ff998b40640 1 -- 172.21.15.139:0/3753567672 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990032810 con 0x7ff994071540 2024-09-16T00:10:41.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.755+0000 7ff99bda6640 1 -- 172.21.15.139:0/3753567672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff994071540 msgr2=0x7ff9940c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:41.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.755+0000 7ff99bda6640 1 --2- 172.21.15.139:0/3753567672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff994071540 0x7ff9940c9d20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff994074320 tx=0x7ff99002a1e0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.755+0000 7ff99bda6640 1 -- 172.21.15.139:0/3753567672 shutdown_connections 2024-09-16T00:10:41.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.755+0000 7ff99bda6640 1 --2- 172.21.15.139:0/3753567672 >> v2:172.21.15.164:3300/0 conn(0x7ff99415e280 0x7ff99415e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.755+0000 7ff99bda6640 1 --2- 172.21.15.139:0/3753567672 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff9940c8bc0 0x7ff994071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.755+0000 7ff99bda6640 1 --2- 172.21.15.139:0/3753567672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff994071540 0x7ff9940c9d20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.755+0000 7ff99bda6640 1 -- 172.21.15.139:0/3753567672 >> 172.21.15.139:0/3753567672 conn(0x7ff99406bb20 msgr2=0x7ff99406f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:41.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.756+0000 7ff99bda6640 1 -- 172.21.15.139:0/3753567672 shutdown_connections 2024-09-16T00:10:41.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.756+0000 7ff99bda6640 1 -- 172.21.15.139:0/3753567672 wait complete. 2024-09-16T00:10:41.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.757+0000 7ff99bda6640 1 Processor -- start 2024-09-16T00:10:41.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.757+0000 7ff99bda6640 1 -- start start 2024-09-16T00:10:41.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.758+0000 7ff99bda6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff994071540 0x7ff9940c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.758+0000 7ff99bda6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9940c8bc0 0x7ff9940c55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.758+0000 7ff99bda6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff99415e280 0x7ff9940c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.758+0000 7ff99bda6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff99406ecc0 con 0x7ff994071540 2024-09-16T00:10:41.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.758+0000 7ff99bda6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff99406eb40 con 0x7ff9940c8bc0 2024-09-16T00:10:41.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.758+0000 7ff99bda6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff994154db0 con 0x7ff99415e280 2024-09-16T00:10:41.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.758+0000 7ff999341640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9940c8bc0 0x7ff9940c55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.758+0000 7ff999b42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff994071540 0x7ff9940c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.758+0000 7ff999341640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9940c8bc0 0x7ff9940c55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:53404/0 (socket says 172.21.15.139:53404) 2024-09-16T00:10:41.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.758+0000 7ff999341640 1 -- 172.21.15.139:0/972306756 learned_addr learned my addr 172.21.15.139:0/972306756 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:41.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.758+0000 7ff999b42640 1 --2- 172.21.15.139:0/972306756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff994071540 0x7ff9940c9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38614/0 (socket says 172.21.15.139:38614) 2024-09-16T00:10:41.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.758+0000 7ff99a343640 1 --2- 172.21.15.139:0/972306756 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff99415e280 0x7ff9940c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.759+0000 7ff999341640 1 -- 172.21.15.139:0/972306756 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff99415e280 msgr2=0x7ff9940c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:41.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.759+0000 7ff999341640 1 --2- 172.21.15.139:0/972306756 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff99415e280 0x7ff9940c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.759+0000 7ff999341640 1 -- 172.21.15.139:0/972306756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff994071540 msgr2=0x7ff9940c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:41.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.759+0000 7ff999341640 1 --2- 172.21.15.139:0/972306756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff994071540 0x7ff9940c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:41.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.759+0000 7ff999341640 1 -- 172.21.15.139:0/972306756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9940c8370 con 0x7ff9940c8bc0 2024-09-16T00:10:41.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.759+0000 7ff999341640 1 --2- 172.21.15.139:0/972306756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9940c8bc0 0x7ff9940c55c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff98400b140 tx=0x7ff98400b600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:41.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.759+0000 7ff98a7fc640 1 -- 172.21.15.139:0/972306756 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98400be30 con 0x7ff9940c8bc0 2024-09-16T00:10:41.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.759+0000 7ff98a7fc640 1 -- 172.21.15.139:0/972306756 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff984004020 con 0x7ff9940c8bc0 2024-09-16T00:10:41.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.759+0000 7ff99bda6640 1 -- 172.21.15.139:0/972306756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9941e4650 con 0x7ff9940c8bc0 2024-09-16T00:10:41.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.759+0000 7ff98a7fc640 1 -- 172.21.15.139:0/972306756 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9840043f0 con 0x7ff9940c8bc0 2024-09-16T00:10:41.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.759+0000 7ff99bda6640 1 -- 172.21.15.139:0/972306756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9941e4c10 con 0x7ff9940c8bc0 2024-09-16T00:10:41.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.760+0000 7ff99bda6640 1 -- 172.21.15.139:0/972306756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff99414ee30 con 0x7ff9940c8bc0 2024-09-16T00:10:41.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.761+0000 7ff98a7fc640 1 -- 172.21.15.139:0/972306756 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff984003610 con 0x7ff9940c8bc0 2024-09-16T00:10:41.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.761+0000 7ff98a7fc640 1 --2- 172.21.15.139:0/972306756 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff97806c2b0 0x7ff97806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:41.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.762+0000 7ff98a7fc640 1 -- 172.21.15.139:0/972306756 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5973+0+0 (secure 0 0 0) 0x7ff9840933e0 con 0x7ff9940c8bc0 2024-09-16T00:10:41.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.762+0000 7ff999b42640 1 --2- 172.21.15.139:0/972306756 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff97806c2b0 0x7ff97806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:41.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.762+0000 7ff999b42640 1 --2- 172.21.15.139:0/972306756 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff97806c2b0 0x7ff97806e770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff990034000 tx=0x7ff99002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:41.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.766+0000 7ff98a7fc640 1 -- 172.21.15.139:0/972306756 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff99414ee30 con 0x7ff9940c8bc0 2024-09-16T00:10:41.837 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:41.835+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:41.868 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:41.868 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:41.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.868+0000 7f6fb40a6640 1 -- 172.21.15.139:0/509570773 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f6fac15b770 con 0x7f6fac154c20 2024-09-16T00:10:41.877 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:10:41.938 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:41.937+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:41.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.948+0000 7ff99bda6640 1 -- 172.21.15.139:0/972306756 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7ff99406def0 con 0x7ff9940c8bc0 2024-09-16T00:10:41.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.950+0000 7ff98a7fc640 1 -- 172.21.15.139:0/972306756 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v118) ==== 81+0+35816 (secure 0 0 0) 0x7ff99406def0 con 0x7ff9940c8bc0 2024-09-16T00:10:41.953 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:10:41.954 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:41.954 INFO:teuthology.orchestra.run.smithi139.stdout: "epoch": 118, 2024-09-16T00:10:41.954 INFO:teuthology.orchestra.run.smithi139.stdout: "fsid": "2d3366e5-6a59-4b59-93b2-615e72ac7015", 2024-09-16T00:10:41.954 INFO:teuthology.orchestra.run.smithi139.stdout: "created": "2024-09-16T00:07:09.967833+0000", 2024-09-16T00:10:41.954 INFO:teuthology.orchestra.run.smithi139.stdout: "modified": "2024-09-16T00:10:39.519538+0000", 2024-09-16T00:10:41.954 INFO:teuthology.orchestra.run.smithi139.stdout: "last_up_change": "2024-09-16T00:10:27.386387+0000", 2024-09-16T00:10:41.954 INFO:teuthology.orchestra.run.smithi139.stdout: "last_in_change": "2024-09-16T00:07:12.025620+0000", 2024-09-16T00:10:41.954 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T00:10:41.954 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_num": 5799936, 2024-09-16T00:10:41.954 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_set": [ 2024-09-16T00:10:41.954 INFO:teuthology.orchestra.run.smithi139.stdout: "pglog_hardlimit", 2024-09-16T00:10:41.954 INFO:teuthology.orchestra.run.smithi139.stdout: "purged_snapdirs", 2024-09-16T00:10:41.954 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_deletes", 2024-09-16T00:10:41.954 INFO:teuthology.orchestra.run.smithi139.stdout: "sortbitwise" 2024-09-16T00:10:41.955 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:10:41.955 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_version": 7, 2024-09-16T00:10:41.955 INFO:teuthology.orchestra.run.smithi139.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T00:10:41.955 INFO:teuthology.orchestra.run.smithi139.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T00:10:41.955 INFO:teuthology.orchestra.run.smithi139.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T00:10:41.955 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_snapshot": "", 2024-09-16T00:10:41.955 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_max": 3, 2024-09-16T00:10:41.955 INFO:teuthology.orchestra.run.smithi139.stdout: "max_osd": 8, 2024-09-16T00:10:41.955 INFO:teuthology.orchestra.run.smithi139.stdout: "require_min_compat_client": "luminous", 2024-09-16T00:10:41.955 INFO:teuthology.orchestra.run.smithi139.stdout: "min_compat_client": "luminous", 2024-09-16T00:10:41.955 INFO:teuthology.orchestra.run.smithi139.stdout: "require_osd_release": "squid", 2024-09-16T00:10:41.955 INFO:teuthology.orchestra.run.smithi139.stdout: "allow_crimson": false, 2024-09-16T00:10:41.955 INFO:teuthology.orchestra.run.smithi139.stdout: "pools": [ 2024-09-16T00:10:41.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.956 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 1, 2024-09-16T00:10:41.956 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": ".mgr", 2024-09-16T00:10:41.956 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:22.056303+0000", 2024-09-16T00:10:41.956 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:10:41.956 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:10:41.956 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:10:41.956 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:10:41.956 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:10:41.956 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:10:41.956 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:10:41.956 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:10:41.956 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:10:41.957 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:10:41.957 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:10:41.957 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:10:41.957 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:10:41.957 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 1, 2024-09-16T00:10:41.957 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 1, 2024-09-16T00:10:41.957 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 1, 2024-09-16T00:10:41.957 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 1, 2024-09-16T00:10:41.957 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 1, 2024-09-16T00:10:41.958 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:10:41.958 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:10:41.958 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:10:41.958 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:10:41.958 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:10:41.958 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:10:41.958 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:10:41.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.958 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "18", 2024-09-16T00:10:41.958 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:10:41.958 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:10:41.958 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:10:41.958 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:10:41.958 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:10:41.959 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:10:41.959 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:10:41.959 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:10:41.959 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:10:41.959 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:10:41.959 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:10:41.959 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:10:41.959 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:10:41.959 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:10:41.959 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:10:41.959 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:10:41.959 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:10:41.959 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:10:41.960 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:10:41.960 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:10:41.960 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:10:41.960 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:10:41.960 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:10:41.960 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:10:41.960 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:10:41.960 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:10:41.961 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.961 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:10:41.961 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:10:41.961 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:10:41.961 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:10:41.961 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:10:41.961 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:10:41.961 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:10:41.961 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:10:41.961 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:10:41.961 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:10:41.961 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:10:41.961 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:10:41.962 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_max": 32, 2024-09-16T00:10:41.962 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 1 2024-09-16T00:10:41.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.962 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:10:41.962 INFO:teuthology.orchestra.run.smithi139.stdout: "mgr": {} 2024-09-16T00:10:41.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.962 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:10:41.962 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:10:41.962 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 7.690000057220459, 2024-09-16T00:10:41.962 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 7.690000057220459, 2024-09-16T00:10:41.962 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 0.12999999523162842, 2024-09-16T00:10:41.962 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 1, 2024-09-16T00:10:41.962 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 1, 2024-09-16T00:10:41.962 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:10:41.963 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:10:41.963 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:10:41.963 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.963 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 2, 2024-09-16T00:10:41.963 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": "cephfs_metadata", 2024-09-16T00:10:41.963 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:30.768649+0000", 2024-09-16T00:10:41.963 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:10:41.963 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:10:41.963 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:10:41.963 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:10:41.963 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:10:41.964 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:10:41.964 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:10:41.964 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:10:41.964 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:10:41.964 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:10:41.964 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:10:41.964 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:10:41.964 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:10:41.964 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 32, 2024-09-16T00:10:41.964 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 32, 2024-09-16T00:10:41.964 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 32, 2024-09-16T00:10:41.964 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 32, 2024-09-16T00:10:41.964 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 32, 2024-09-16T00:10:41.965 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:10:41.965 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:10:41.965 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:10:41.965 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:10:41.965 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:10:41.965 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:10:41.965 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:10:41.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.965 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "21", 2024-09-16T00:10:41.965 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:10:41.965 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:10:41.965 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:10:41.965 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:10:41.965 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:10:41.966 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:10:41.966 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:10:41.966 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:10:41.966 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:10:41.966 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:10:41.966 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:10:41.966 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:10:41.966 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:10:41.966 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:10:41.966 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:10:41.966 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:10:41.966 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:10:41.966 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:10:41.966 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:10:41.967 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:10:41.967 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:10:41.967 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:10:41.967 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:10:41.967 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:10:41.967 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:10:41.967 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:10:41.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.967 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:10:41.967 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:10:41.967 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:10:41.967 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:10:41.967 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:10:41.967 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:10:41.968 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:10:41.968 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:10:41.968 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:10:41.968 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:10:41.968 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:10:41.968 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:10:41.968 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_bias": 4, 2024-09-16T00:10:41.968 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 16, 2024-09-16T00:10:41.968 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_priority": 5 2024-09-16T00:10:41.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.968 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:10:41.968 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs": { 2024-09-16T00:10:41.968 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": "cephfs" 2024-09-16T00:10:41.968 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.969 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:10:41.969 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:10:41.969 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 1.7400000095367432, 2024-09-16T00:10:41.969 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 1.7400000095367432, 2024-09-16T00:10:41.969 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 0.87999999523162842, 2024-09-16T00:10:41.969 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-16T00:10:41.969 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-16T00:10:41.969 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:10:41.969 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:10:41.969 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:10:41.969 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.969 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.970 INFO:teuthology.orchestra.run.smithi139.stdout: "pool": 3, 2024-09-16T00:10:41.970 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_name": "cephfs_data", 2024-09-16T00:10:41.970 INFO:teuthology.orchestra.run.smithi139.stdout: "create_time": "2024-09-16T00:07:32.008209+0000", 2024-09-16T00:10:41.970 INFO:teuthology.orchestra.run.smithi139.stdout: "flags": 1, 2024-09-16T00:10:41.970 INFO:teuthology.orchestra.run.smithi139.stdout: "flags_names": "hashpspool", 2024-09-16T00:10:41.970 INFO:teuthology.orchestra.run.smithi139.stdout: "type": 1, 2024-09-16T00:10:41.970 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 2, 2024-09-16T00:10:41.970 INFO:teuthology.orchestra.run.smithi139.stdout: "min_size": 1, 2024-09-16T00:10:41.970 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_rule": 0, 2024-09-16T00:10:41.970 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_count": 0, 2024-09-16T00:10:41.970 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_target": 0, 2024-09-16T00:10:41.970 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T00:10:41.970 INFO:teuthology.orchestra.run.smithi139.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T00:10:41.970 INFO:teuthology.orchestra.run.smithi139.stdout: "is_stretch_pool": false, 2024-09-16T00:10:41.971 INFO:teuthology.orchestra.run.smithi139.stdout: "object_hash": 2, 2024-09-16T00:10:41.971 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_autoscale_mode": "off", 2024-09-16T00:10:41.971 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num": 64, 2024-09-16T00:10:41.971 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num": 64, 2024-09-16T00:10:41.971 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_placement_num_target": 64, 2024-09-16T00:10:41.971 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_target": 64, 2024-09-16T00:10:41.971 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_pending": 64, 2024-09-16T00:10:41.971 INFO:teuthology.orchestra.run.smithi139.stdout: "last_pg_merge_meta": { 2024-09-16T00:10:41.971 INFO:teuthology.orchestra.run.smithi139.stdout: "source_pgid": "0.0", 2024-09-16T00:10:41.971 INFO:teuthology.orchestra.run.smithi139.stdout: "ready_epoch": 0, 2024-09-16T00:10:41.971 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_started": 0, 2024-09-16T00:10:41.971 INFO:teuthology.orchestra.run.smithi139.stdout: "last_epoch_clean": 0, 2024-09-16T00:10:41.971 INFO:teuthology.orchestra.run.smithi139.stdout: "source_version": "0'0", 2024-09-16T00:10:41.972 INFO:teuthology.orchestra.run.smithi139.stdout: "target_version": "0'0" 2024-09-16T00:10:41.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.972 INFO:teuthology.orchestra.run.smithi139.stdout: "last_change": "22", 2024-09-16T00:10:41.972 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend": "0", 2024-09-16T00:10:41.972 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T00:10:41.972 INFO:teuthology.orchestra.run.smithi139.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T00:10:41.972 INFO:teuthology.orchestra.run.smithi139.stdout: "auid": 0, 2024-09-16T00:10:41.972 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_mode": "selfmanaged", 2024-09-16T00:10:41.972 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_seq": 0, 2024-09-16T00:10:41.972 INFO:teuthology.orchestra.run.smithi139.stdout: "snap_epoch": 0, 2024-09-16T00:10:41.972 INFO:teuthology.orchestra.run.smithi139.stdout: "pool_snaps": [], 2024-09-16T00:10:41.972 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps": "[]", 2024-09-16T00:10:41.972 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_bytes": 0, 2024-09-16T00:10:41.972 INFO:teuthology.orchestra.run.smithi139.stdout: "quota_max_objects": 0, 2024-09-16T00:10:41.973 INFO:teuthology.orchestra.run.smithi139.stdout: "tiers": [], 2024-09-16T00:10:41.973 INFO:teuthology.orchestra.run.smithi139.stdout: "tier_of": -1, 2024-09-16T00:10:41.973 INFO:teuthology.orchestra.run.smithi139.stdout: "read_tier": -1, 2024-09-16T00:10:41.973 INFO:teuthology.orchestra.run.smithi139.stdout: "write_tier": -1, 2024-09-16T00:10:41.973 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_mode": "none", 2024-09-16T00:10:41.973 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_bytes": 0, 2024-09-16T00:10:41.973 INFO:teuthology.orchestra.run.smithi139.stdout: "target_max_objects": 0, 2024-09-16T00:10:41.973 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T00:10:41.973 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T00:10:41.973 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T00:10:41.973 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_flush_age": 0, 2024-09-16T00:10:41.973 INFO:teuthology.orchestra.run.smithi139.stdout: "cache_min_evict_age": 0, 2024-09-16T00:10:41.973 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profile": "", 2024-09-16T00:10:41.973 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_params": { 2024-09-16T00:10:41.974 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "none" 2024-09-16T00:10:41.974 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.974 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_period": 0, 2024-09-16T00:10:41.974 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_count": 0, 2024-09-16T00:10:41.974 INFO:teuthology.orchestra.run.smithi139.stdout: "use_gmt_hitset": true, 2024-09-16T00:10:41.974 INFO:teuthology.orchestra.run.smithi139.stdout: "min_read_recency_for_promote": 0, 2024-09-16T00:10:41.974 INFO:teuthology.orchestra.run.smithi139.stdout: "min_write_recency_for_promote": 0, 2024-09-16T00:10:41.974 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T00:10:41.974 INFO:teuthology.orchestra.run.smithi139.stdout: "hit_set_search_last_n": 0, 2024-09-16T00:10:41.974 INFO:teuthology.orchestra.run.smithi139.stdout: "grade_table": [], 2024-09-16T00:10:41.974 INFO:teuthology.orchestra.run.smithi139.stdout: "stripe_width": 0, 2024-09-16T00:10:41.974 INFO:teuthology.orchestra.run.smithi139.stdout: "expected_num_objects": 0, 2024-09-16T00:10:41.974 INFO:teuthology.orchestra.run.smithi139.stdout: "fast_read": false, 2024-09-16T00:10:41.974 INFO:teuthology.orchestra.run.smithi139.stdout: "options": { 2024-09-16T00:10:41.975 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_num_min": 64, 2024-09-16T00:10:41.975 INFO:teuthology.orchestra.run.smithi139.stdout: "target_size_ratio": 0.89999997615814209 2024-09-16T00:10:41.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.975 INFO:teuthology.orchestra.run.smithi139.stdout: "application_metadata": { 2024-09-16T00:10:41.975 INFO:teuthology.orchestra.run.smithi139.stdout: "cephfs": { 2024-09-16T00:10:41.975 INFO:teuthology.orchestra.run.smithi139.stdout: "data": "cephfs" 2024-09-16T00:10:41.975 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.975 INFO:teuthology.orchestra.run.smithi139.stdout: "read_balance": { 2024-09-16T00:10:41.975 INFO:teuthology.orchestra.run.smithi139.stdout: "score_type": "Fair distribution", 2024-09-16T00:10:41.975 INFO:teuthology.orchestra.run.smithi139.stdout: "score_acting": 1.6100000143051147, 2024-09-16T00:10:41.975 INFO:teuthology.orchestra.run.smithi139.stdout: "score_stable": 1.6100000143051147, 2024-09-16T00:10:41.975 INFO:teuthology.orchestra.run.smithi139.stdout: "optimal_score": 0.87999999523162842, 2024-09-16T00:10:41.975 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_acting": 1.4199999570846558, 2024-09-16T00:10:41.976 INFO:teuthology.orchestra.run.smithi139.stdout: "raw_score_stable": 1.4199999570846558, 2024-09-16T00:10:41.976 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity_weighted": 1, 2024-09-16T00:10:41.976 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity": 1, 2024-09-16T00:10:41.976 INFO:teuthology.orchestra.run.smithi139.stdout: "average_primary_affinity_weighted": 1 2024-09-16T00:10:41.976 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.976 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.976 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:10:41.976 INFO:teuthology.orchestra.run.smithi139.stdout: "osds": [ 2024-09-16T00:10:41.976 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.976 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 0, 2024-09-16T00:10:41.976 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "83699a05-4937-43c3-aefc-b669bed9bb6f", 2024-09-16T00:10:41.976 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:10:41.976 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:10:41.976 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:10:41.977 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:10:41.977 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:10:41.977 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:10:41.977 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 14, 2024-09-16T00:10:41.977 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 113, 2024-09-16T00:10:41.977 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:10:41.977 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:10:41.977 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:10:41.977 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:41.977 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.977 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:41.977 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6816", 2024-09-16T00:10:41.977 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:10:41.977 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.978 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.978 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:41.978 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6817", 2024-09-16T00:10:41.978 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:10:41.978 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.978 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.978 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:10:41.978 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:41.978 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.978 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:41.978 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6818", 2024-09-16T00:10:41.978 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:10:41.978 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.979 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.979 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:41.979 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6819", 2024-09-16T00:10:41.979 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:10:41.979 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.979 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.979 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.979 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:10:41.979 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:41.979 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.979 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:41.979 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6822", 2024-09-16T00:10:41.979 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:10:41.979 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.980 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.980 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:41.980 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6823", 2024-09-16T00:10:41.980 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:10:41.980 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.980 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.980 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.980 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:10:41.980 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:41.980 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.980 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:41.980 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6820", 2024-09-16T00:10:41.980 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:10:41.981 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.981 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.981 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:41.981 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6821", 2024-09-16T00:10:41.981 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1974901438 2024-09-16T00:10:41.981 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.981 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.981 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.981 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6817/1974901438", 2024-09-16T00:10:41.981 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6819/1974901438", 2024-09-16T00:10:41.981 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6823/1974901438", 2024-09-16T00:10:41.981 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6821/1974901438", 2024-09-16T00:10:41.981 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:10:41.981 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:10:41.982 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:10:41.982 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.982 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.982 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.982 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 1, 2024-09-16T00:10:41.982 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "9c04991e-9695-4bbb-8011-bf613bf404b3", 2024-09-16T00:10:41.982 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:10:41.982 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:10:41.982 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 0.522918701171875, 2024-09-16T00:10:41.982 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:10:41.982 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:10:41.982 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:10:41.982 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 14, 2024-09-16T00:10:41.982 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 113, 2024-09-16T00:10:41.983 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:10:41.983 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:10:41.983 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:10:41.983 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:41.983 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.983 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:41.983 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6808", 2024-09-16T00:10:41.983 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:10:41.983 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.983 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.983 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:41.983 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6809", 2024-09-16T00:10:41.983 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:10:41.983 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:10:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6810", 2024-09-16T00:10:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:10:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6811", 2024-09-16T00:10:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:10:41.984 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.985 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.985 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:10:41.985 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:41.985 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.985 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:41.985 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6814", 2024-09-16T00:10:41.985 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:10:41.985 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.985 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.985 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:41.985 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6815", 2024-09-16T00:10:41.985 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:10:41.985 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.986 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.986 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.986 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:10:41.986 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:41.986 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.986 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:41.986 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6812", 2024-09-16T00:10:41.986 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:10:41.986 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.986 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.986 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:41.986 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6813", 2024-09-16T00:10:41.986 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2232380998 2024-09-16T00:10:41.986 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.987 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.987 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.987 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6809/2232380998", 2024-09-16T00:10:41.987 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6811/2232380998", 2024-09-16T00:10:41.987 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6815/2232380998", 2024-09-16T00:10:41.987 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6813/2232380998", 2024-09-16T00:10:41.987 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:10:41.987 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:10:41.987 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:10:41.987 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.987 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.987 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.987 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 2, 2024-09-16T00:10:41.987 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "8048fc64-72b3-4866-ac62-6a8b2bf6f5c8", 2024-09-16T00:10:41.988 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:10:41.988 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:10:41.988 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:10:41.988 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:10:41.988 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:10:41.988 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:10:41.988 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 13, 2024-09-16T00:10:41.988 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 94, 2024-09-16T00:10:41.988 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:10:41.988 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:10:41.988 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:10:41.988 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:41.988 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.988 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:41.989 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6800", 2024-09-16T00:10:41.989 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:10:41.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.989 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.989 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:41.989 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6801", 2024-09-16T00:10:41.989 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:10:41.989 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.989 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.989 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.989 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:10:41.989 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:41.989 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.989 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:41.990 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6802", 2024-09-16T00:10:41.990 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:10:41.990 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.990 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:41.990 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6803", 2024-09-16T00:10:41.990 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:10:41.990 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.990 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.990 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.990 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:10:41.990 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:41.990 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.990 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:41.991 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6806", 2024-09-16T00:10:41.991 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:10:41.991 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.991 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.991 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:41.991 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6807", 2024-09-16T00:10:41.991 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:10:41.991 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.991 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.991 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.991 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:10:41.991 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:41.991 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.991 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:41.992 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6804", 2024-09-16T00:10:41.992 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:10:41.992 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.992 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.992 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:41.992 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6805", 2024-09-16T00:10:41.992 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4180588905 2024-09-16T00:10:41.992 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.992 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.992 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.992 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6801/4180588905", 2024-09-16T00:10:41.992 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6803/4180588905", 2024-09-16T00:10:41.992 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6807/4180588905", 2024-09-16T00:10:41.993 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6805/4180588905", 2024-09-16T00:10:41.993 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:10:41.993 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:10:41.993 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:10:41.993 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.993 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.993 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.993 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 3, 2024-09-16T00:10:41.993 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "c34ac77b-354c-4884-902d-3e2a84f5eb27", 2024-09-16T00:10:41.993 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:10:41.993 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:10:41.993 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:10:41.993 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:10:41.993 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:10:41.994 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:10:41.994 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:10:41.994 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 113, 2024-09-16T00:10:41.994 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:10:41.994 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:10:41.994 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:10:41.994 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:41.994 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.994 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:41.994 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6826", 2024-09-16T00:10:41.994 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:10:41.994 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.994 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.994 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:41.995 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6827", 2024-09-16T00:10:41.995 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:10:41.995 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.995 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.995 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.995 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:10:41.995 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:41.995 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.995 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:41.995 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6828", 2024-09-16T00:10:41.995 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:10:41.995 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.995 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.995 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:41.996 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6829", 2024-09-16T00:10:41.996 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:10:41.996 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.996 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.996 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.996 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:10:41.996 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:41.996 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.996 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:41.996 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6832", 2024-09-16T00:10:41.996 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:10:41.996 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.996 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.996 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:41.997 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6833", 2024-09-16T00:10:41.997 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:10:41.997 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.997 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.997 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.997 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:10:41.997 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:41.997 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.997 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:41.997 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6830", 2024-09-16T00:10:41.997 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:10:41.997 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.997 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.997 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:41.998 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.139:6831", 2024-09-16T00:10:41.998 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3947768492 2024-09-16T00:10:41.998 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:41.998 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.998 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.998 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.139:6827/3947768492", 2024-09-16T00:10:41.998 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.139:6829/3947768492", 2024-09-16T00:10:41.998 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.139:6833/3947768492", 2024-09-16T00:10:41.998 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.139:6831/3947768492", 2024-09-16T00:10:41.998 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:10:41.998 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:10:41.998 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:10:41.998 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:41.998 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:41.999 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:41.999 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 4, 2024-09-16T00:10:41.999 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "ba9a7c93-9672-43e1-8c98-2878cdf7f548", 2024-09-16T00:10:41.999 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:10:41.999 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:10:41.999 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:10:41.999 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:10:41.999 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:10:41.999 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:10:41.999 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:10:41.999 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 113, 2024-09-16T00:10:41.999 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:10:41.999 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:10:41.999 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:10:42.000 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:42.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.000 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:42.000 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6816", 2024-09-16T00:10:42.000 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:10:42.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.000 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.000 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:42.000 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6817", 2024-09-16T00:10:42.000 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:10:42.000 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.000 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.000 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.001 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:10:42.001 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:42.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.001 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:42.001 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6818", 2024-09-16T00:10:42.001 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:10:42.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.001 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.001 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:42.001 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6819", 2024-09-16T00:10:42.001 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:10:42.001 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.001 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.001 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.002 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:10:42.002 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:42.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.002 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:42.002 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6822", 2024-09-16T00:10:42.002 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:10:42.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.002 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.002 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:42.002 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6823", 2024-09-16T00:10:42.002 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:10:42.002 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.002 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.002 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.002 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:10:42.003 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:42.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.003 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:42.003 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6820", 2024-09-16T00:10:42.003 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:10:42.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.003 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.003 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:42.003 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6821", 2024-09-16T00:10:42.003 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 642632916 2024-09-16T00:10:42.003 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.003 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.003 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.004 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6817/642632916", 2024-09-16T00:10:42.004 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6819/642632916", 2024-09-16T00:10:42.004 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6823/642632916", 2024-09-16T00:10:42.004 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6821/642632916", 2024-09-16T00:10:42.004 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:10:42.004 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:10:42.004 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:10:42.004 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.004 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.004 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.004 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 5, 2024-09-16T00:10:42.004 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "46a0916b-474c-411b-84ad-86f27d4cbf53", 2024-09-16T00:10:42.004 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:10:42.004 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:10:42.005 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:10:42.005 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:10:42.005 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:10:42.005 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:10:42.005 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:10:42.005 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 113, 2024-09-16T00:10:42.005 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 0, 2024-09-16T00:10:42.005 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:10:42.005 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:10:42.005 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:42.005 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.005 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:42.005 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6800", 2024-09-16T00:10:42.005 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:10:42.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.006 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:42.006 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6801", 2024-09-16T00:10:42.006 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:10:42.006 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.006 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.006 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.006 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:10:42.006 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:42.006 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.006 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:42.006 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6802", 2024-09-16T00:10:42.006 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:10:42.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.007 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:42.007 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6803", 2024-09-16T00:10:42.007 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:10:42.007 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.007 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.007 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.007 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:10:42.007 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:42.007 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.007 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:42.007 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6806", 2024-09-16T00:10:42.007 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:10:42.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.008 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:42.008 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6807", 2024-09-16T00:10:42.008 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:10:42.008 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.008 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.008 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.008 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:10:42.008 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:42.008 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.008 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:42.008 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6804", 2024-09-16T00:10:42.008 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:10:42.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.009 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.009 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:42.009 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6805", 2024-09-16T00:10:42.009 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1051778383 2024-09-16T00:10:42.009 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.009 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.009 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.009 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6801/1051778383", 2024-09-16T00:10:42.009 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6803/1051778383", 2024-09-16T00:10:42.009 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6807/1051778383", 2024-09-16T00:10:42.009 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6805/1051778383", 2024-09-16T00:10:42.009 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:10:42.009 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:10:42.010 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:10:42.010 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.010 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.010 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.010 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 6, 2024-09-16T00:10:42.010 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "9b6b33cc-d6d9-4a0a-8143-526073a58b93", 2024-09-16T00:10:42.010 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 1, 2024-09-16T00:10:42.010 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:10:42.010 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:10:42.010 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:10:42.010 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 15, 2024-09-16T00:10:42.010 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 82, 2024-09-16T00:10:42.010 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 94, 2024-09-16T00:10:42.011 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 113, 2024-09-16T00:10:42.011 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 84, 2024-09-16T00:10:42.011 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:10:42.011 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:10:42.011 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:42.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.011 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:42.011 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6824", 2024-09-16T00:10:42.011 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 164138206 2024-09-16T00:10:42.011 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.011 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.011 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:42.011 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6825", 2024-09-16T00:10:42.011 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 164138206 2024-09-16T00:10:42.011 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.012 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.012 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:10:42.012 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:42.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.012 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:42.012 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6826", 2024-09-16T00:10:42.012 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 164138206 2024-09-16T00:10:42.012 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.012 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.012 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:42.012 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6827", 2024-09-16T00:10:42.012 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 164138206 2024-09-16T00:10:42.013 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.013 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.013 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:10:42.013 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:42.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.013 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:42.013 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6830", 2024-09-16T00:10:42.013 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 164138206 2024-09-16T00:10:42.013 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.013 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.013 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:42.013 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6831", 2024-09-16T00:10:42.013 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 164138206 2024-09-16T00:10:42.014 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.014 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.014 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:10:42.014 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:42.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.014 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:42.014 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6828", 2024-09-16T00:10:42.014 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 164138206 2024-09-16T00:10:42.014 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.014 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.014 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:42.014 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6829", 2024-09-16T00:10:42.014 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 164138206 2024-09-16T00:10:42.015 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.015 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.015 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.015 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6825/164138206", 2024-09-16T00:10:42.015 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6827/164138206", 2024-09-16T00:10:42.015 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6831/164138206", 2024-09-16T00:10:42.015 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6829/164138206", 2024-09-16T00:10:42.015 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:10:42.015 INFO:teuthology.orchestra.run.smithi139.stdout: "exists", 2024-09-16T00:10:42.015 INFO:teuthology.orchestra.run.smithi139.stdout: "up" 2024-09-16T00:10:42.015 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.015 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.015 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.015 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 7, 2024-09-16T00:10:42.016 INFO:teuthology.orchestra.run.smithi139.stdout: "uuid": "50bcd808-3d3e-4a72-88e4-30decb53cfea", 2024-09-16T00:10:42.016 INFO:teuthology.orchestra.run.smithi139.stdout: "up": 0, 2024-09-16T00:10:42.016 INFO:teuthology.orchestra.run.smithi139.stdout: "in": 1, 2024-09-16T00:10:42.016 INFO:teuthology.orchestra.run.smithi139.stdout: "weight": 1, 2024-09-16T00:10:42.016 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_affinity": 1, 2024-09-16T00:10:42.016 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_begin": 0, 2024-09-16T00:10:42.016 INFO:teuthology.orchestra.run.smithi139.stdout: "last_clean_end": 0, 2024-09-16T00:10:42.016 INFO:teuthology.orchestra.run.smithi139.stdout: "up_from": 15, 2024-09-16T00:10:42.016 INFO:teuthology.orchestra.run.smithi139.stdout: "up_thru": 109, 2024-09-16T00:10:42.016 INFO:teuthology.orchestra.run.smithi139.stdout: "down_at": 113, 2024-09-16T00:10:42.016 INFO:teuthology.orchestra.run.smithi139.stdout: "lost_at": 0, 2024-09-16T00:10:42.016 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addrs": { 2024-09-16T00:10:42.016 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:42.016 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.017 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:42.017 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6808", 2024-09-16T00:10:42.017 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:10:42.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.017 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.017 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:42.017 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6809", 2024-09-16T00:10:42.017 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:10:42.017 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.017 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.017 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.017 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addrs": { 2024-09-16T00:10:42.017 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:42.017 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.018 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:42.018 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6810", 2024-09-16T00:10:42.018 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:10:42.018 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.018 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:42.018 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6811", 2024-09-16T00:10:42.018 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:10:42.018 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.018 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.018 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.018 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addrs": { 2024-09-16T00:10:42.018 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:42.018 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.019 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:42.019 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6814", 2024-09-16T00:10:42.019 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:10:42.019 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.019 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.019 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:42.019 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6815", 2024-09-16T00:10:42.019 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:10:42.019 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.019 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.019 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.019 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addrs": { 2024-09-16T00:10:42.019 INFO:teuthology.orchestra.run.smithi139.stdout: "addrvec": [ 2024-09-16T00:10:42.019 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.020 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v2", 2024-09-16T00:10:42.020 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6812", 2024-09-16T00:10:42.020 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:10:42.020 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.020 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.020 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-16T00:10:42.020 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "172.21.15.164:6813", 2024-09-16T00:10:42.020 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1158838808 2024-09-16T00:10:42.020 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.020 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.020 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.020 INFO:teuthology.orchestra.run.smithi139.stdout: "public_addr": "172.21.15.164:6809/1158838808", 2024-09-16T00:10:42.020 INFO:teuthology.orchestra.run.smithi139.stdout: "cluster_addr": "172.21.15.164:6811/1158838808", 2024-09-16T00:10:42.020 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_back_addr": "172.21.15.164:6815/1158838808", 2024-09-16T00:10:42.021 INFO:teuthology.orchestra.run.smithi139.stdout: "heartbeat_front_addr": "172.21.15.164:6813/1158838808", 2024-09-16T00:10:42.021 INFO:teuthology.orchestra.run.smithi139.stdout: "state": [ 2024-09-16T00:10:42.021 INFO:teuthology.orchestra.run.smithi139.stdout: "exists" 2024-09-16T00:10:42.021 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.021 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.021 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:10:42.021 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_xinfo": [ 2024-09-16T00:10:42.021 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.021 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 0, 2024-09-16T00:10:42.021 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:10:42.021 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:10:42.021 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:10:42.021 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:10:42.021 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:10:42.022 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.843117+0000", 2024-09-16T00:10:42.022 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:10:42.022 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.022 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.022 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 1, 2024-09-16T00:10:42.022 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:10:42.022 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:10:42.022 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:10:42.022 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:10:42.022 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:10:42.022 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.622789+0000", 2024-09-16T00:10:42.022 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:10:42.022 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.022 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.023 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 2, 2024-09-16T00:10:42.023 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:10:42.023 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:10:42.023 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:10:42.023 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:10:42.023 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:10:42.023 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:17.919457+0000", 2024-09-16T00:10:42.023 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:10:42.023 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.023 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.023 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 3, 2024-09-16T00:10:42.023 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:10:42.023 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:10:42.023 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:10:42.024 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:10:42.024 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:10:42.024 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:19.330341+0000", 2024-09-16T00:10:42.024 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:10:42.024 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.024 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.024 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 4, 2024-09-16T00:10:42.024 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:10:42.024 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:10:42.024 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:10:42.024 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:10:42.024 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:10:42.024 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.815640+0000", 2024-09-16T00:10:42.025 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:10:42.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.025 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.025 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 5, 2024-09-16T00:10:42.025 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:10:42.025 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:10:42.025 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:10:42.025 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:10:42.025 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:10:42.025 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.083046+0000", 2024-09-16T00:10:42.025 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 0 2024-09-16T00:10:42.025 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.025 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.025 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 6, 2024-09-16T00:10:42.026 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:10:42.026 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:10:42.026 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:10:42.026 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:10:42.026 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:10:42.026 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.551499+0000", 2024-09-16T00:10:42.026 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 84 2024-09-16T00:10:42.026 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.026 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.026 INFO:teuthology.orchestra.run.smithi139.stdout: "osd": 7, 2024-09-16T00:10:42.026 INFO:teuthology.orchestra.run.smithi139.stdout: "down_stamp": "0.000000", 2024-09-16T00:10:42.026 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_probability": 0, 2024-09-16T00:10:42.026 INFO:teuthology.orchestra.run.smithi139.stdout: "laggy_interval": 0, 2024-09-16T00:10:42.026 INFO:teuthology.orchestra.run.smithi139.stdout: "features": 4540701547738038271, 2024-09-16T00:10:42.027 INFO:teuthology.orchestra.run.smithi139.stdout: "old_weight": 0, 2024-09-16T00:10:42.027 INFO:teuthology.orchestra.run.smithi139.stdout: "last_purged_snaps_scrub": "2024-09-16T00:07:18.377179+0000", 2024-09-16T00:10:42.027 INFO:teuthology.orchestra.run.smithi139.stdout: "dead_epoch": 113 2024-09-16T00:10:42.027 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.027 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:10:42.027 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap": [ 2024-09-16T00:10:42.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.027 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.2d", 2024-09-16T00:10:42.027 INFO:teuthology.orchestra.run.smithi139.stdout: "osds": [ 2024-09-16T00:10:42.027 INFO:teuthology.orchestra.run.smithi139.stdout: 7, 2024-09-16T00:10:42.027 INFO:teuthology.orchestra.run.smithi139.stdout: 5 2024-09-16T00:10:42.027 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.027 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.027 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.027 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.3a", 2024-09-16T00:10:42.028 INFO:teuthology.orchestra.run.smithi139.stdout: "osds": [ 2024-09-16T00:10:42.028 INFO:teuthology.orchestra.run.smithi139.stdout: 5, 2024-09-16T00:10:42.028 INFO:teuthology.orchestra.run.smithi139.stdout: 6 2024-09-16T00:10:42.028 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.028 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.028 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:10:42.028 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap_items": [ 2024-09-16T00:10:42.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.028 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.1f", 2024-09-16T00:10:42.028 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:10:42.028 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.028 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 0, 2024-09-16T00:10:42.028 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:10:42.028 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.029 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.029 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.21", 2024-09-16T00:10:42.029 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:10:42.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.029 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 5, 2024-09-16T00:10:42.029 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 1 2024-09-16T00:10:42.029 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.029 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.029 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.029 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.029 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.22", 2024-09-16T00:10:42.029 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:10:42.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.030 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 5, 2024-09-16T00:10:42.030 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:10:42.030 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.030 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.030 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.030 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.38", 2024-09-16T00:10:42.030 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:10:42.030 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.030 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 3, 2024-09-16T00:10:42.030 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:10:42.030 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.030 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.031 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.3a", 2024-09-16T00:10:42.031 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:10:42.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.031 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 5, 2024-09-16T00:10:42.031 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 2 2024-09-16T00:10:42.031 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.031 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.031 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.031 INFO:teuthology.orchestra.run.smithi139.stdout: "pgid": "3.3b", 2024-09-16T00:10:42.031 INFO:teuthology.orchestra.run.smithi139.stdout: "mappings": [ 2024-09-16T00:10:42.031 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:42.032 INFO:teuthology.orchestra.run.smithi139.stdout: "from": 7, 2024-09-16T00:10:42.032 INFO:teuthology.orchestra.run.smithi139.stdout: "to": 1 2024-09-16T00:10:42.032 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.032 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:42.032 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.032 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:10:42.032 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_upmap_primaries": [], 2024-09-16T00:10:42.032 INFO:teuthology.orchestra.run.smithi139.stdout: "pg_temp": [], 2024-09-16T00:10:42.032 INFO:teuthology.orchestra.run.smithi139.stdout: "primary_temp": [], 2024-09-16T00:10:42.032 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklist": {}, 2024-09-16T00:10:42.032 INFO:teuthology.orchestra.run.smithi139.stdout: "range_blocklist": {}, 2024-09-16T00:10:42.032 INFO:teuthology.orchestra.run.smithi139.stdout: "erasure_code_profiles": { 2024-09-16T00:10:42.032 INFO:teuthology.orchestra.run.smithi139.stdout: "default": { 2024-09-16T00:10:42.032 INFO:teuthology.orchestra.run.smithi139.stdout: "crush-failure-domain": "osd", 2024-09-16T00:10:42.033 INFO:teuthology.orchestra.run.smithi139.stdout: "k": "2", 2024-09-16T00:10:42.033 INFO:teuthology.orchestra.run.smithi139.stdout: "m": "1", 2024-09-16T00:10:42.033 INFO:teuthology.orchestra.run.smithi139.stdout: "plugin": "jerasure", 2024-09-16T00:10:42.033 INFO:teuthology.orchestra.run.smithi139.stdout: "technique": "reed_sol_van" 2024-09-16T00:10:42.033 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.033 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:42.033 INFO:teuthology.orchestra.run.smithi139.stdout: "removed_snaps_queue": [], 2024-09-16T00:10:42.033 INFO:teuthology.orchestra.run.smithi139.stdout: "new_removed_snaps": [], 2024-09-16T00:10:42.033 INFO:teuthology.orchestra.run.smithi139.stdout: "new_purged_snaps": [], 2024-09-16T00:10:42.033 INFO:teuthology.orchestra.run.smithi139.stdout: "crush_node_flags": {}, 2024-09-16T00:10:42.033 INFO:teuthology.orchestra.run.smithi139.stdout: "device_class_flags": {}, 2024-09-16T00:10:42.033 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode": { 2024-09-16T00:10:42.033 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode_enabled": false, 2024-09-16T00:10:42.033 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_bucket_count": 0, 2024-09-16T00:10:42.034 INFO:teuthology.orchestra.run.smithi139.stdout: "degraded_stretch_mode": 0, 2024-09-16T00:10:42.034 INFO:teuthology.orchestra.run.smithi139.stdout: "recovering_stretch_mode": 0, 2024-09-16T00:10:42.034 INFO:teuthology.orchestra.run.smithi139.stdout: "stretch_mode_bucket": 0 2024-09-16T00:10:42.034 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:42.034 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:42.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.952+0000 7ff99bda6640 1 -- 172.21.15.139:0/972306756 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff97806c2b0 msgr2=0x7ff97806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:42.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.952+0000 7ff99bda6640 1 --2- 172.21.15.139:0/972306756 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff97806c2b0 0x7ff97806e770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff990034000 tx=0x7ff99002e000 comp rx=0 tx=0).stop 2024-09-16T00:10:42.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.952+0000 7ff99bda6640 1 -- 172.21.15.139:0/972306756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9940c8bc0 msgr2=0x7ff9940c55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:42.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.952+0000 7ff99bda6640 1 --2- 172.21.15.139:0/972306756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9940c8bc0 0x7ff9940c55c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff98400b140 tx=0x7ff98400b600 comp rx=0 tx=0).stop 2024-09-16T00:10:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.952+0000 7ff99bda6640 1 -- 172.21.15.139:0/972306756 shutdown_connections 2024-09-16T00:10:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.952+0000 7ff99bda6640 1 --2- 172.21.15.139:0/972306756 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff97806c2b0 0x7ff97806e770 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.952+0000 7ff99bda6640 1 --2- 172.21.15.139:0/972306756 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff99415e280 0x7ff9940c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.952+0000 7ff99bda6640 1 --2- 172.21.15.139:0/972306756 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff9940c8bc0 0x7ff9940c55c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.952+0000 7ff99bda6640 1 --2- 172.21.15.139:0/972306756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff994071540 0x7ff9940c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.952+0000 7ff99bda6640 1 -- 172.21.15.139:0/972306756 >> 172.21.15.139:0/972306756 conn(0x7ff99406bb20 msgr2=0x7ff994151630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.952+0000 7ff99bda6640 1 -- 172.21.15.139:0/972306756 shutdown_connections 2024-09-16T00:10:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:41.952+0000 7ff99bda6640 1 -- 172.21.15.139:0/972306756 wait complete. 2024-09-16T00:10:42.037 INFO:tasks.thrashosds.thrasher:j is {'epoch': 118, 'fsid': '2d3366e5-6a59-4b59-93b2-615e72ac7015', 'created': '2024-09-16T00:07:09.967833+0000', 'modified': '2024-09-16T00:10:39.519538+0000', 'last_up_change': '2024-09-16T00:10:27.386387+0000', 'last_in_change': '2024-09-16T00:07:12.025620+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 7, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:07:22.056303+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.690000057220459, 'score_stable': 7.690000057220459, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T00:07:30.768649+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T00:07:32.008209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6100000143051147, 'score_stable': 1.6100000143051147, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.4199999570846558, 'raw_score_stable': 1.4199999570846558, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '83699a05-4937-43c3-aefc-b669bed9bb6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6816', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6817', 'nonce': 1974901438}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6818', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6819', 'nonce': 1974901438}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6822', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6823', 'nonce': 1974901438}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6820', 'nonce': 1974901438}, {'type': 'v1', 'addr': '172.21.15.139:6821', 'nonce': 1974901438}]}, 'public_addr': '172.21.15.139:6817/1974901438', 'cluster_addr': '172.21.15.139:6819/1974901438', 'heartbeat_back_addr': '172.21.15.139:6823/1974901438', 'heartbeat_front_addr': '172.21.15.139:6821/1974901438', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '9c04991e-9695-4bbb-8011-bf613bf404b3', 'up': 1, 'in': 1, 'weight': 0.522918701171875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6808', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6809', 'nonce': 2232380998}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6810', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6811', 'nonce': 2232380998}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6814', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6815', 'nonce': 2232380998}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6812', 'nonce': 2232380998}, {'type': 'v1', 'addr': '172.21.15.139:6813', 'nonce': 2232380998}]}, 'public_addr': '172.21.15.139:6809/2232380998', 'cluster_addr': '172.21.15.139:6811/2232380998', 'heartbeat_back_addr': '172.21.15.139:6815/2232380998', 'heartbeat_front_addr': '172.21.15.139:6813/2232380998', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8048fc64-72b3-4866-ac62-6a8b2bf6f5c8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6800', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6801', 'nonce': 4180588905}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6802', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6803', 'nonce': 4180588905}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6806', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6807', 'nonce': 4180588905}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6804', 'nonce': 4180588905}, {'type': 'v1', 'addr': '172.21.15.139:6805', 'nonce': 4180588905}]}, 'public_addr': '172.21.15.139:6801/4180588905', 'cluster_addr': '172.21.15.139:6803/4180588905', 'heartbeat_back_addr': '172.21.15.139:6807/4180588905', 'heartbeat_front_addr': '172.21.15.139:6805/4180588905', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c34ac77b-354c-4884-902d-3e2a84f5eb27', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6826', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6827', 'nonce': 3947768492}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6828', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6829', 'nonce': 3947768492}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6832', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6833', 'nonce': 3947768492}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6830', 'nonce': 3947768492}, {'type': 'v1', 'addr': '172.21.15.139:6831', 'nonce': 3947768492}]}, 'public_addr': '172.21.15.139:6827/3947768492', 'cluster_addr': '172.21.15.139:6829/3947768492', 'heartbeat_back_addr': '172.21.15.139:6833/3947768492', 'heartbeat_front_addr': '172.21.15.139:6831/3947768492', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'ba9a7c93-9672-43e1-8c98-2878cdf7f548', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6816', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6817', 'nonce': 642632916}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6818', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6819', 'nonce': 642632916}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6822', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6823', 'nonce': 642632916}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6820', 'nonce': 642632916}, {'type': 'v1', 'addr': '172.21.15.164:6821', 'nonce': 642632916}]}, 'public_addr': '172.21.15.164:6817/642632916', 'cluster_addr': '172.21.15.164:6819/642632916', 'heartbeat_back_addr': '172.21.15.164:6823/642632916', 'heartbeat_front_addr': '172.21.15.164:6821/642632916', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '46a0916b-474c-411b-84ad-86f27d4cbf53', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 113, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6800', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6801', 'nonce': 1051778383}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6802', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6803', 'nonce': 1051778383}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6806', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6807', 'nonce': 1051778383}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6804', 'nonce': 1051778383}, {'type': 'v1', 'addr': '172.21.15.164:6805', 'nonce': 1051778383}]}, 'public_addr': '172.21.15.164:6801/1051778383', 'cluster_addr': '172.21.15.164:6803/1051778383', 'heartbeat_back_addr': '172.21.15.164:6807/1051778383', 'heartbeat_front_addr': '172.21.15.164:6805/1051778383', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '9b6b33cc-d6d9-4a0a-8143-526073a58b93', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 15, 'last_clean_end': 82, 'up_from': 94, 'up_thru': 113, 'down_at': 84, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6824', 'nonce': 164138206}, {'type': 'v1', 'addr': '172.21.15.164:6825', 'nonce': 164138206}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6826', 'nonce': 164138206}, {'type': 'v1', 'addr': '172.21.15.164:6827', 'nonce': 164138206}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6830', 'nonce': 164138206}, {'type': 'v1', 'addr': '172.21.15.164:6831', 'nonce': 164138206}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6828', 'nonce': 164138206}, {'type': 'v1', 'addr': '172.21.15.164:6829', 'nonce': 164138206}]}, 'public_addr': '172.21.15.164:6825/164138206', 'cluster_addr': '172.21.15.164:6827/164138206', 'heartbeat_back_addr': '172.21.15.164:6831/164138206', 'heartbeat_front_addr': '172.21.15.164:6829/164138206', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '50bcd808-3d3e-4a72-88e4-30decb53cfea', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 109, 'down_at': 113, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6808', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6809', 'nonce': 1158838808}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6810', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6811', 'nonce': 1158838808}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6814', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6815', 'nonce': 1158838808}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.164:6812', 'nonce': 1158838808}, {'type': 'v1', 'addr': '172.21.15.164:6813', 'nonce': 1158838808}]}, 'public_addr': '172.21.15.164:6809/1158838808', 'cluster_addr': '172.21.15.164:6811/1158838808', 'heartbeat_back_addr': '172.21.15.164:6815/1158838808', 'heartbeat_front_addr': '172.21.15.164:6813/1158838808', 'state': ['exists']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.843117+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.622789+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:17.919457+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:19.330341+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.815640+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.083046+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.551499+0000', 'dead_epoch': 84}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T00:07:18.377179+0000', 'dead_epoch': 113}], 'pg_upmap': [{'pgid': '3.2d', 'osds': [7, 5]}, {'pgid': '3.3a', 'osds': [5, 6]}], 'pg_upmap_items': [{'pgid': '3.1f', 'mappings': [{'from': 0, 'to': 2}]}, {'pgid': '3.21', 'mappings': [{'from': 5, 'to': 1}]}, {'pgid': '3.22', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '3.38', 'mappings': [{'from': 3, 'to': 2}]}, {'pgid': '3.3a', 'mappings': [{'from': 5, 'to': 2}]}, {'pgid': '3.3b', 'mappings': [{'from': 7, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T00:10:42.037 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.3a 2024-09-16T00:10:42.037 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.3a 2024-09-16T00:10:42.038 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:42.038 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:42.039 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:10:42.040 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:42.039+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:42.141 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:42.140+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:42.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.208+0000 7efe6be3c640 1 Processor -- start 2024-09-16T00:10:42.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.208+0000 7efe6be3c640 1 -- start start 2024-09-16T00:10:42.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.209+0000 7efe6be3c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe640c9430 0x7efe64163560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:42.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.209+0000 7efe6be3c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe640703e0 con 0x7efe640ce810 2024-09-16T00:10:42.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.209+0000 7efe6be3c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe64070680 con 0x7efe640ce430 2024-09-16T00:10:42.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.209+0000 7efe6be3c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7efe640c5750 con 0x7efe640c9430 2024-09-16T00:10:42.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.209+0000 7efe693d7640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7efe640ce810 0x7efe640c5be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:40458/0 (socket says 172.21.15.139:40458) 2024-09-16T00:10:42.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.209+0000 7efe693d7640 1 -- 172.21.15.139:0/848642065 learned_addr learned my addr 172.21.15.139:0/848642065 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:42.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.209+0000 7efe6a3d9640 1 --2- 172.21.15.139:0/848642065 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe640c9430 0x7efe64163560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:42.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.209+0000 7efe68bd6640 1 -- 172.21.15.139:0/848642065 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2744833542 0 0) 0x7efe640703e0 con 0x7efe640ce810 2024-09-16T00:10:42.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.209+0000 7efe68bd6640 1 -- 172.21.15.139:0/848642065 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe50003520 con 0x7efe640ce810 2024-09-16T00:10:42.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.209+0000 7efe68bd6640 1 -- 172.21.15.139:0/848642065 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2839737930 0 0) 0x7efe64070680 con 0x7efe640ce430 2024-09-16T00:10:42.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.209+0000 7efe68bd6640 1 -- 172.21.15.139:0/848642065 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe640703e0 con 0x7efe640ce430 2024-09-16T00:10:42.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.209+0000 7efe6a3d9640 1 -- 172.21.15.139:0/848642065 >> v1:172.21.15.164:6789/0 conn(0x7efe640ce430 legacy=0x7efe640b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:42.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.209+0000 7efe6a3d9640 1 -- 172.21.15.139:0/848642065 >> v1:172.21.15.139:6789/0 conn(0x7efe640ce810 legacy=0x7efe640c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:42.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.209+0000 7efe6a3d9640 1 -- 172.21.15.139:0/848642065 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe64070920 con 0x7efe640c9430 2024-09-16T00:10:42.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.210+0000 7efe6a3d9640 1 --2- 172.21.15.139:0/848642065 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe640c9430 0x7efe64163560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009dc0 tx=0x7efe5c02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dec1248c398beb18 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:42.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.211+0000 7efe68bd6640 1 -- 172.21.15.139:0/848642065 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c007740 con 0x7efe640c9430 2024-09-16T00:10:42.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.211+0000 7efe68bd6640 1 -- 172.21.15.139:0/848642065 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe5c0078e0 con 0x7efe640c9430 2024-09-16T00:10:42.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.211+0000 7efe68bd6640 1 -- 172.21.15.139:0/848642065 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c007cb0 con 0x7efe640c9430 2024-09-16T00:10:42.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.211+0000 7efe6be3c640 1 -- 172.21.15.139:0/848642065 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe640c9430 msgr2=0x7efe64163560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:42.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.211+0000 7efe6be3c640 1 --2- 172.21.15.139:0/848642065 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe640c9430 0x7efe64163560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009dc0 tx=0x7efe5c02a6d0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.211+0000 7efe6be3c640 1 -- 172.21.15.139:0/848642065 shutdown_connections 2024-09-16T00:10:42.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.211+0000 7efe6be3c640 1 --2- 172.21.15.139:0/848642065 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe640c9430 0x7efe64163560 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.211+0000 7efe6be3c640 1 -- 172.21.15.139:0/848642065 >> 172.21.15.139:0/848642065 conn(0x7efe6406c150 msgr2=0x7efe64067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:42.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.212+0000 7efe6be3c640 1 -- 172.21.15.139:0/848642065 shutdown_connections 2024-09-16T00:10:42.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.212+0000 7efe6be3c640 1 -- 172.21.15.139:0/848642065 wait complete. 2024-09-16T00:10:42.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.213+0000 7efe6be3c640 1 Processor -- start 2024-09-16T00:10:42.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.213+0000 7efe6be3c640 1 -- start start 2024-09-16T00:10:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe6be3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe640c9430 0x7efe64155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe6be3c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe640ce430 0x7efe64157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe6be3c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe640ce810 0x7efe641501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe6be3c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efe6406ffc0 con 0x7efe640c9430 2024-09-16T00:10:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe6be3c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efe64070ee0 con 0x7efe640ce430 2024-09-16T00:10:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe6be3c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7efe64070140 con 0x7efe640ce810 2024-09-16T00:10:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe69bd8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe640c9430 0x7efe64155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe69bd8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe640c9430 0x7efe64155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38624/0 (socket says 172.21.15.139:38624) 2024-09-16T00:10:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe69bd8640 1 -- 172.21.15.139:0/2999025583 learned_addr learned my addr 172.21.15.139:0/2999025583 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe6a3d9640 1 --2- 172.21.15.139:0/2999025583 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe640ce810 0x7efe641501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe693d7640 1 --2- 172.21.15.139:0/2999025583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe640ce430 0x7efe64157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe69bd8640 1 -- 172.21.15.139:0/2999025583 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe640ce810 msgr2=0x7efe641501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:42.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe69bd8640 1 --2- 172.21.15.139:0/2999025583 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe640ce810 0x7efe641501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe69bd8640 1 -- 172.21.15.139:0/2999025583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe640ce430 msgr2=0x7efe64157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:42.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe69bd8640 1 --2- 172.21.15.139:0/2999025583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe640ce430 0x7efe64157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.217 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:42.217 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:42.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe69bd8640 1 -- 172.21.15.139:0/2999025583 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe640c7a50 con 0x7efe640c9430 2024-09-16T00:10:42.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.214+0000 7efe69bd8640 1 --2- 172.21.15.139:0/2999025583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe640c9430 0x7efe64155700 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efe6000c490 tx=0x7efe6000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:42.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.215+0000 7efe5a7fc640 1 -- 172.21.15.139:0/2999025583 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe600052d0 con 0x7efe640c9430 2024-09-16T00:10:42.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.215+0000 7efe5a7fc640 1 -- 172.21.15.139:0/2999025583 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe60005470 con 0x7efe640c9430 2024-09-16T00:10:42.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.215+0000 7efe6be3c640 1 -- 172.21.15.139:0/2999025583 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe640b17f0 con 0x7efe640c9430 2024-09-16T00:10:42.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.215+0000 7efe5a7fc640 1 -- 172.21.15.139:0/2999025583 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60012870 con 0x7efe640c9430 2024-09-16T00:10:42.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.215+0000 7efe6be3c640 1 -- 172.21.15.139:0/2999025583 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe641ee9b0 con 0x7efe640c9430 2024-09-16T00:10:42.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.216+0000 7efe6be3c640 1 -- 172.21.15.139:0/2999025583 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe640b31c0 con 0x7efe640c9430 2024-09-16T00:10:42.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.217+0000 7efe5a7fc640 1 -- 172.21.15.139:0/2999025583 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efe60012a10 con 0x7efe640c9430 2024-09-16T00:10:42.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.217+0000 7efe5a7fc640 1 --2- 172.21.15.139:0/2999025583 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efe4806c2b0 0x7efe4806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:42.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.217+0000 7efe5a7fc640 1 -- 172.21.15.139:0/2999025583 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5973+0+0 (secure 0 0 0) 0x7efe6008bc30 con 0x7efe640c9430 2024-09-16T00:10:42.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.217+0000 7efe693d7640 1 --2- 172.21.15.139:0/2999025583 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efe4806c2b0 0x7efe4806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:42.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.218+0000 7efe693d7640 1 --2- 172.21.15.139:0/2999025583 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efe4806c2b0 0x7efe4806e770 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efe640c52e0 tx=0x7efe54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:42.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.221+0000 7efe5a7fc640 1 -- 172.21.15.139:0/2999025583 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efe60059b90 con 0x7efe640c9430 2024-09-16T00:10:42.225 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:10:42.267 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:42.266+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:42.343 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:42.342+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:42.375 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:42.376 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:42.385 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:10:42.417 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.416+0000 7efe6be3c640 1 -- 172.21.15.139:0/2999025583 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd rm-pg-upmap", "pgid": "3.3a"} v 0) -- 0x7efe64151910 con 0x7efe640c9430 2024-09-16T00:10:42.460 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:42.460+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:42.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.532+0000 7f6fa27fc640 1 -- 172.21.15.139:0/509570773 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v119) ==== 87+0+0 (secure 0 0 0) 0x7f6fac15b770 con 0x7f6fac154c20 2024-09-16T00:10:42.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.532+0000 7efe5a7fc640 1 -- 172.21.15.139:0/2999025583 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd rm-pg-upmap", "pgid": "3.3a"}]=0 clear 3.3a pg_upmap mapping v119) ==== 106+0+0 (secure 0 0 0) 0x7efe6005de90 con 0x7efe640c9430 2024-09-16T00:10:42.533 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is set 2024-09-16T00:10:42.533 INFO:teuthology.orchestra.run.smithi139.stderr:clear 3.3a pg_upmap mapping 2024-09-16T00:10:42.534 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:42.534 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:42.535 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7f6fb40a6640 1 -- 172.21.15.139:0/509570773 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6f9006c2e0 msgr2=0x7f6f9006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:42.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7f6fb40a6640 1 --2- 172.21.15.139:0/509570773 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6f9006c2e0 0x7f6f9006e7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80011e0 tx=0x7f6fa8009040 comp rx=0 tx=0).stop 2024-09-16T00:10:42.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7f6fb40a6640 1 -- 172.21.15.139:0/509570773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fac154c20 msgr2=0x7f6fac1591f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:42.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7f6fb40a6640 1 --2- 172.21.15.139:0/509570773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fac154c20 0x7f6fac1591f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400caa0 tx=0x7f6fa400cf60 comp rx=0 tx=0).stop 2024-09-16T00:10:42.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7f6fb40a6640 1 -- 172.21.15.139:0/509570773 shutdown_connections 2024-09-16T00:10:42.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7f6fb40a6640 1 --2- 172.21.15.139:0/509570773 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6f9006c2e0 0x7f6f9006e7a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7f6fb40a6640 1 --2- 172.21.15.139:0/509570773 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6fac154c20 0x7f6fac1591f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7f6fb40a6640 1 --2- 172.21.15.139:0/509570773 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6fac0ccc90 0x7f6fac15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7f6fb40a6640 1 --2- 172.21.15.139:0/509570773 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6fac068060 0x7f6fac0d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7f6fb40a6640 1 -- 172.21.15.139:0/509570773 >> 172.21.15.139:0/509570773 conn(0x7f6fac06ca40 msgr2=0x7f6fac0ca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:42.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7f6fb40a6640 1 -- 172.21.15.139:0/509570773 shutdown_connections 2024-09-16T00:10:42.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7efe6be3c640 1 -- 172.21.15.139:0/2999025583 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efe4806c2b0 msgr2=0x7efe4806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:42.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7efe6be3c640 1 --2- 172.21.15.139:0/2999025583 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efe4806c2b0 0x7efe4806e770 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7efe640c52e0 tx=0x7efe54009040 comp rx=0 tx=0).stop 2024-09-16T00:10:42.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7efe6be3c640 1 -- 172.21.15.139:0/2999025583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe640c9430 msgr2=0x7efe64155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:42.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7efe6be3c640 1 --2- 172.21.15.139:0/2999025583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe640c9430 0x7efe64155700 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efe6000c490 tx=0x7efe6000c950 comp rx=0 tx=0).stop 2024-09-16T00:10:42.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7f6fb40a6640 1 -- 172.21.15.139:0/509570773 wait complete. 2024-09-16T00:10:42.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7efe6be3c640 1 -- 172.21.15.139:0/2999025583 shutdown_connections 2024-09-16T00:10:42.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7efe6be3c640 1 --2- 172.21.15.139:0/2999025583 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efe4806c2b0 0x7efe4806e770 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7efe6be3c640 1 --2- 172.21.15.139:0/2999025583 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efe640ce810 0x7efe641501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7efe6be3c640 1 --2- 172.21.15.139:0/2999025583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efe640ce430 0x7efe64157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.535+0000 7efe6be3c640 1 --2- 172.21.15.139:0/2999025583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe640c9430 0x7efe64155700 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:42.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.536+0000 7efe6be3c640 1 -- 172.21.15.139:0/2999025583 >> 172.21.15.139:0/2999025583 conn(0x7efe6406c150 msgr2=0x7efe6415aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:42.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.536+0000 7efe6be3c640 1 -- 172.21.15.139:0/2999025583 shutdown_connections 2024-09-16T00:10:42.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:42.536+0000 7efe6be3c640 1 -- 172.21.15.139:0/2999025583 wait complete. 2024-09-16T00:10:42.545 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:42.544+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:42.547 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:10:42.646 INFO:tasks.ceph.osd.5.smithi164.stderr:2024-09-16T00:10:42.644+0000 7fe16af2d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 36665) UID: 0 2024-09-16T00:10:42.702 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:42.702 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:42.712 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:10:42.755 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:42.755+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:42.818 INFO:teuthology.orchestra.run.smithi139.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:42.818 INFO:teuthology.orchestra.run.smithi139.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:42.827 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:10:42.869 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:42.868+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:42.948 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:42.948+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:42.962 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:42.962 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:42.973 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:10:43.050 INFO:tasks.ceph.osd.4.smithi164.stderr:2024-09-16T00:10:43.048+0000 7f3432440640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 36658) UID: 0 2024-09-16T00:10:43.094 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:43.095 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:43.106 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:10:43.150 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:43.149+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:43.250 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:43.250+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:43.279 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:43.279 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:43.289 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:10:43.351 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:43.350+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:43.398 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:43.398 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:43.408 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:10:43.432 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T00:10:43.477 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:43.476+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:43.516 INFO:teuthology.orchestra.run.smithi164.stdout:ERROR: (22) Invalid argument 2024-09-16T00:10:43.517 INFO:teuthology.orchestra.run.smithi164.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T00:10:43.525 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:10:43.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.531+0000 7ff4f25d6640 1 Processor -- start 2024-09-16T00:10:43.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.531+0000 7ff4f25d6640 1 -- start start 2024-09-16T00:10:43.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.532+0000 7ff4f25d6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff4ec157b00 0x7ff4ec073b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:43.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.532+0000 7ff4f25d6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4ec151b10 con 0x7ff4ec072830 2024-09-16T00:10:43.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.532+0000 7ff4f25d6640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4ec0c3d20 con 0x7ff4ec151730 2024-09-16T00:10:43.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.532+0000 7ff4f25d6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff4ec0c3ad0 con 0x7ff4ec157b00 2024-09-16T00:10:43.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.532+0000 7ff4f0b73640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff4ec157b00 0x7ff4ec073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:43.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.532+0000 7ff4eb7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff4ec072830 0x7ff4ec1573f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:40468/0 (socket says 172.21.15.139:40468) 2024-09-16T00:10:43.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.532+0000 7ff4eb7fe640 1 -- 172.21.15.139:0/3053490619 learned_addr learned my addr 172.21.15.139:0/3053490619 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:43.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.532+0000 7ff4ebfff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ff4ec151730 0x7ff4ec073030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:43920/0 (socket says 172.21.15.139:43920) 2024-09-16T00:10:43.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.532+0000 7ff4eaffd640 1 -- 172.21.15.139:0/3053490619 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4064618329 0 0) 0x7ff4ec0c3d20 con 0x7ff4ec151730 2024-09-16T00:10:43.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.532+0000 7ff4f0b73640 1 -- 172.21.15.139:0/3053490619 >> v1:172.21.15.164:6789/0 conn(0x7ff4ec151730 legacy=0x7ff4ec073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:43.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.532+0000 7ff4f0b73640 1 -- 172.21.15.139:0/3053490619 >> v1:172.21.15.139:6789/0 conn(0x7ff4ec072830 legacy=0x7ff4ec1573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:43.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.532+0000 7ff4f0b73640 1 -- 172.21.15.139:0/3053490619 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ec0c4000 con 0x7ff4ec157b00 2024-09-16T00:10:43.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.533+0000 7ff4f0b73640 1 --2- 172.21.15.139:0/3053490619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4ec157b00 0x7ff4ec073b70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0c3c50 tx=0x7ff4e402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f39dd984320d1eb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:43.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.535+0000 7ff4eaffd640 1 -- 172.21.15.139:0/3053490619 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4007740 con 0x7ff4ec157b00 2024-09-16T00:10:43.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.535+0000 7ff4eaffd640 1 -- 172.21.15.139:0/3053490619 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e40078e0 con 0x7ff4ec157b00 2024-09-16T00:10:43.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.535+0000 7ff4eaffd640 1 -- 172.21.15.139:0/3053490619 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4007cb0 con 0x7ff4ec157b00 2024-09-16T00:10:43.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.536+0000 7ff4f25d6640 1 -- 172.21.15.139:0/3053490619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4ec157b00 msgr2=0x7ff4ec073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:43.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.536+0000 7ff4f25d6640 1 --2- 172.21.15.139:0/3053490619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4ec157b00 0x7ff4ec073b70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0c3c50 tx=0x7ff4e402a680 comp rx=0 tx=0).stop 2024-09-16T00:10:43.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.536+0000 7ff4f25d6640 1 -- 172.21.15.139:0/3053490619 shutdown_connections 2024-09-16T00:10:43.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.536+0000 7ff4f25d6640 1 --2- 172.21.15.139:0/3053490619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4ec157b00 0x7ff4ec073b70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:43.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.536+0000 7ff4f25d6640 1 -- 172.21.15.139:0/3053490619 >> 172.21.15.139:0/3053490619 conn(0x7ff4ec067950 msgr2=0x7ff4ec067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:43.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.536+0000 7ff4f25d6640 1 -- 172.21.15.139:0/3053490619 shutdown_connections 2024-09-16T00:10:43.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.536+0000 7ff4f25d6640 1 -- 172.21.15.139:0/3053490619 wait complete. 2024-09-16T00:10:43.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.538+0000 7ff4f25d6640 1 Processor -- start 2024-09-16T00:10:43.539 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.538+0000 7ff4f25d6640 1 -- start start 2024-09-16T00:10:43.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.539+0000 7ff4f25d6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4ec072830 0x7ff4ec1f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:43.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.539+0000 7ff4f25d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4ec151730 0x7ff4ec1e2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:43.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.539+0000 7ff4f25d6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff4ec157b00 0x7ff4ec14cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:43.541 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.539+0000 7ff4f25d6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec072c60 con 0x7ff4ec151730 2024-09-16T00:10:43.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.539+0000 7ff4f25d6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec0cd8c0 con 0x7ff4ec072830 2024-09-16T00:10:43.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.539+0000 7ff4f25d6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ec0cd550 con 0x7ff4ec157b00 2024-09-16T00:10:43.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.539+0000 7ff4eb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4ec151730 0x7ff4ec1e2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:43.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.539+0000 7ff4eb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4ec151730 0x7ff4ec1e2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38636/0 (socket says 172.21.15.139:38636) 2024-09-16T00:10:43.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.539+0000 7ff4eb7fe640 1 -- 172.21.15.139:0/1931277779 learned_addr learned my addr 172.21.15.139:0/1931277779 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:43.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.539+0000 7ff4ebfff640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4ec072830 0x7ff4ec1f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:43.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.539+0000 7ff4eb7fe640 1 -- 172.21.15.139:0/1931277779 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff4ec157b00 msgr2=0x7ff4ec14cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:43.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.539+0000 7ff4eb7fe640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff4ec157b00 0x7ff4ec14cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:43.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.539+0000 7ff4eb7fe640 1 -- 172.21.15.139:0/1931277779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4ec072830 msgr2=0x7ff4ec1f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:43.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.539+0000 7ff4eb7fe640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4ec072830 0x7ff4ec1f2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:43.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.539+0000 7ff4eb7fe640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ec0ccd00 con 0x7ff4ec151730 2024-09-16T00:10:43.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.539+0000 7ff4ebfff640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff4ec072830 0x7ff4ec1f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:10:43.542 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.540+0000 7ff4eb7fe640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4ec151730 0x7ff4ec1e2ed0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000c7b0 tx=0x7ff4e000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:43.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.540+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0018070 con 0x7ff4ec151730 2024-09-16T00:10:43.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.540+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e00072d0 con 0x7ff4ec151730 2024-09-16T00:10:43.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.540+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ec1f2860 con 0x7ff4ec151730 2024-09-16T00:10:43.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.540+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00047e0 con 0x7ff4ec151730 2024-09-16T00:10:43.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.540+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ec15b580 con 0x7ff4ec151730 2024-09-16T00:10:43.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.541+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7ff4ec06d9a0 con 0x7ff4ec151730 2024-09-16T00:10:43.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.541+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff4e001b070 con 0x7ff4ec151730 2024-09-16T00:10:43.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.542+0000 7ff4e8ff9640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff4d806c2e0 0x7ff4d806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:43.544 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.542+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5969+0+0 (secure 0 0 0) 0x7ff4e008a9c0 con 0x7ff4ec151730 2024-09-16T00:10:43.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.542+0000 7ff4ebfff640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff4d806c2e0 0x7ff4d806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:43.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.542+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v120) ==== 54+0+16 (secure 0 0 0) 0x7ff4e008ce20 con 0x7ff4ec151730 2024-09-16T00:10:43.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.542+0000 7ff4f25d6640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7ff4ec1f1e90 0x7ff4ec14d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:43.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.542+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff4ec0c2800 con 0x7ff4ec1f1e90 2024-09-16T00:10:43.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.542+0000 7ff4ebfff640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff4d806c2e0 0x7ff4d806e7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc000f60 tx=0x7ff4dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:43.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.542+0000 7ff4f0b73640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7ff4ec1f1e90 0x7ff4ec14d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:43.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.543+0000 7ff4f0b73640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7ff4ec1f1e90 0x7ff4ec14d150 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:43.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.543+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff4ec0c2800 con 0x7ff4ec1f1e90 2024-09-16T00:10:43.551 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:43.551+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:43.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.558+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff4ec072e10 con 0x7ff4ec1f1e90 2024-09-16T00:10:43.560 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.560+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff4ec072e10 con 0x7ff4ec1f1e90 2024-09-16T00:10:43.561 INFO:teuthology.orchestra.run.smithi139.stdout:osd.0: {} 2024-09-16T00:10:43.562 INFO:teuthology.orchestra.run.smithi139.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T00:10:43.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.560+0000 7ff4f25d6640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7ff4ec14e270 0x7ff4ec1e3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:43.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.560+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7ff4ec14aef0 con 0x7ff4ec14e270 2024-09-16T00:10:43.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.560+0000 7ff4ebfff640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7ff4ec14e270 0x7ff4ec1e3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:43.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.561+0000 7ff4ebfff640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7ff4ec14e270 0x7ff4ec1e3b90 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:43.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.561+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff4ec14aef0 con 0x7ff4ec14e270 2024-09-16T00:10:43.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.576+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff4ec14ac10 con 0x7ff4ec14e270 2024-09-16T00:10:43.578 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.578+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff4ec14ac10 con 0x7ff4ec14e270 2024-09-16T00:10:43.579 INFO:teuthology.orchestra.run.smithi139.stdout:osd.1: {} 2024-09-16T00:10:43.579 INFO:teuthology.orchestra.run.smithi139.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T00:10:43.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.578+0000 7ff4f25d6640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7ff4ec1f19d0 0x7ff4ec06c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:43.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.578+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7ff4ec1ee410 con 0x7ff4ec1f19d0 2024-09-16T00:10:43.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.578+0000 7ff4eb7fe640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7ff4ec1f19d0 0x7ff4ec06c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:43.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.578+0000 7ff4eb7fe640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7ff4ec1f19d0 0x7ff4ec06c030 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:43.580 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.579+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff4ec1ee410 con 0x7ff4ec1f19d0 2024-09-16T00:10:43.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.593+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff4ec06c570 con 0x7ff4ec1f19d0 2024-09-16T00:10:43.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.595+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff4ec06c570 con 0x7ff4ec1f19d0 2024-09-16T00:10:43.596 INFO:teuthology.orchestra.run.smithi139.stdout:osd.2: {} 2024-09-16T00:10:43.597 INFO:teuthology.orchestra.run.smithi139.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T00:10:43.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.595+0000 7ff4f25d6640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7ff4ec062b40 0x7ff4ec062f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:43.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.595+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7ff4ec0634e0 con 0x7ff4ec062b40 2024-09-16T00:10:43.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.595+0000 7ff4f0b73640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7ff4ec062b40 0x7ff4ec062f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:43.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.596+0000 7ff4f0b73640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7ff4ec062b40 0x7ff4ec062f20 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:43.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.596+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff4ec0634e0 con 0x7ff4ec062b40 2024-09-16T00:10:43.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.610+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff4ec0cb130 con 0x7ff4ec062b40 2024-09-16T00:10:43.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.612+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff4ec0cb130 con 0x7ff4ec062b40 2024-09-16T00:10:43.613 INFO:teuthology.orchestra.run.smithi139.stdout:osd.3: {} 2024-09-16T00:10:43.613 INFO:teuthology.orchestra.run.smithi139.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T00:10:43.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.612+0000 7ff4f25d6640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7ff4ec0cb5c0 0x7ff4ec0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:43.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.612+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7ff4ec203f00 con 0x7ff4ec0cb5c0 2024-09-16T00:10:43.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.612+0000 7ff4ebfff640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7ff4ec0cb5c0 0x7ff4ec0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:43.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.613+0000 7ff4ebfff640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7ff4ec0cb5c0 0x7ff4ec0cb9e0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:43.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.613+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff4ec203f00 con 0x7ff4ec0cb5c0 2024-09-16T00:10:43.628 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.627+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff4ec202d80 con 0x7ff4ec0cb5c0 2024-09-16T00:10:43.630 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.629+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff4ec202d80 con 0x7ff4ec0cb5c0 2024-09-16T00:10:43.630 INFO:teuthology.orchestra.run.smithi139.stdout:osd.4: {} 2024-09-16T00:10:43.631 INFO:teuthology.orchestra.run.smithi139.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T00:10:43.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.629+0000 7ff4f25d6640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7ff4ec2034f0 0x7ff4ec203930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:43.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.629+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7ff4ec2072d0 con 0x7ff4ec2034f0 2024-09-16T00:10:43.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.629+0000 7ff4eb7fe640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7ff4ec2034f0 0x7ff4ec203930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:43.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.630+0000 7ff4eb7fe640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7ff4ec2034f0 0x7ff4ec203930 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:43.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.631+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff4ec2072d0 con 0x7ff4ec2034f0 2024-09-16T00:10:43.646 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.645+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff4ec206150 con 0x7ff4ec2034f0 2024-09-16T00:10:43.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.647+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff4ec206150 con 0x7ff4ec2034f0 2024-09-16T00:10:43.648 INFO:teuthology.orchestra.run.smithi139.stdout:osd.5: {} 2024-09-16T00:10:43.648 INFO:teuthology.orchestra.run.smithi139.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T00:10:43.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.647+0000 7ff4f25d6640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7ff4ec2067d0 0x7ff4ec206bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:43.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.647+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7ff4ec20a690 con 0x7ff4ec2067d0 2024-09-16T00:10:43.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.647+0000 7ff4f0b73640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7ff4ec2067d0 0x7ff4ec206bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:43.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.647+0000 7ff4f0b73640 1 --2- 172.21.15.139:0/1931277779 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7ff4ec2067d0 0x7ff4ec206bf0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:43.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.649+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff4ec20a690 con 0x7ff4ec2067d0 2024-09-16T00:10:43.652 INFO:tasks.ceph.osd.1.smithi139.stderr:2024-09-16T00:10:43.652+0000 7fe8f0a93640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36450) UID: 0 2024-09-16T00:10:43.662 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10658 3.1c 3:38c4c24e:::100000030bb.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 67.868939389000005, 2024-09-16T00:10:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021232445999999999, 2024-09-16T00:10:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:43.663 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10658 2024-09-16T00:10:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:43.664 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791398+0000", 2024-09-16T00:10:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3880000000000001e-06 2024-09-16T00:10:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800951+0000", 2024-09-16T00:10:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095532110000000007 2024-09-16T00:10:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.665 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800966+0000", 2024-09-16T00:10:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5058999999999999e-05 2024-09-16T00:10:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800974+0000", 2024-09-16T00:10:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1240000000000005e-06 2024-09-16T00:10:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801127+0000", 2024-09-16T00:10:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000153103 2024-09-16T00:10:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.666 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801207+0000", 2024-09-16T00:10:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9569999999999999e-05 2024-09-16T00:10:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801322+0000", 2024-09-16T00:10:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115358 2024-09-16T00:10:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.805760+0000", 2024-09-16T00:10:43.667 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044379409999999999 2024-09-16T00:10:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812528+0000", 2024-09-16T00:10:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067678249999999999 2024-09-16T00:10:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812586+0000", 2024-09-16T00:10:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8164000000000001e-05 2024-09-16T00:10:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.668 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812629+0000", 2024-09-16T00:10:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2703000000000001e-05 2024-09-16T00:10:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4122 2.18 2:1cc859d9:::200.00000012:head [write 798310~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.001399708999998, 2024-09-16T00:10:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019267580999999999, 2024-09-16T00:10:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.670 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658937+0000", 2024-09-16T00:10:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7700000000000004e-07 2024-09-16T00:10:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658942+0000", 2024-09-16T00:10:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.48e-06 2024-09-16T00:10:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658952+0000", 2024-09-16T00:10:43.671 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9469999999999998e-06 2024-09-16T00:10:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658959+0000", 2024-09-16T00:10:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4400000000000002e-06 2024-09-16T00:10:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670507+0000", 2024-09-16T00:10:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011548159000000001 2024-09-16T00:10:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.672 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670579+0000", 2024-09-16T00:10:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2082999999999994e-05 2024-09-16T00:10:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670686+0000", 2024-09-16T00:10:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107069 2024-09-16T00:10:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678001+0000", 2024-09-16T00:10:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073148780000000004 2024-09-16T00:10:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.673 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678104+0000", 2024-09-16T00:10:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010290599999999999 2024-09-16T00:10:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678163+0000", 2024-09-16T00:10:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8859e-05 2024-09-16T00:10:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678204+0000", 2024-09-16T00:10:43.674 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0982999999999997e-05 2024-09-16T00:10:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4123 2.18 2:1cc859d9:::200.00000012:head [write 799709~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 45.000830528000002, 2024-09-16T00:10:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018857569000000001, 2024-09-16T00:10:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.675 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659506+0000", 2024-09-16T00:10:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.78e-07 2024-09-16T00:10:43.676 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659512+0000", 2024-09-16T00:10:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0100000000000001e-06 2024-09-16T00:10:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659523+0000", 2024-09-16T00:10:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-16T00:10:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.677 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659531+0000", 2024-09-16T00:10:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2010000000000002e-06 2024-09-16T00:10:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671124+0000", 2024-09-16T00:10:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011593696000000001 2024-09-16T00:10:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671171+0000", 2024-09-16T00:10:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6974999999999997e-05 2024-09-16T00:10:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.678 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671243+0000", 2024-09-16T00:10:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2131000000000004e-05 2024-09-16T00:10:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678008+0000", 2024-09-16T00:10:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067642329999999997 2024-09-16T00:10:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678261+0000", 2024-09-16T00:10:43.679 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025378999999999999 2024-09-16T00:10:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678320+0000", 2024-09-16T00:10:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8452000000000002e-05 2024-09-16T00:10:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678363+0000", 2024-09-16T00:10:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2831999999999999e-05 2024-09-16T00:10:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.680 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4126 2.18 2:1cc859d9:::200.00000012:head [write 806443~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.999432742000003, 2024-09-16T00:10:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019331159000000001, 2024-09-16T00:10:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:43.681 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660904+0000", 2024-09-16T00:10:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6800000000000009e-07 2024-09-16T00:10:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.682 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660909+0000", 2024-09-16T00:10:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2730000000000002e-06 2024-09-16T00:10:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660919+0000", 2024-09-16T00:10:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4960000000000002e-06 2024-09-16T00:10:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660923+0000", 2024-09-16T00:10:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6260000000000003e-06 2024-09-16T00:10:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.683 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672698+0000", 2024-09-16T00:10:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011774573999999999 2024-09-16T00:10:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672784+0000", 2024-09-16T00:10:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6074000000000003e-05 2024-09-16T00:10:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672897+0000", 2024-09-16T00:10:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011267199999999999 2024-09-16T00:10:43.684 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678026+0000", 2024-09-16T00:10:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051297089999999997 2024-09-16T00:10:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680147+0000", 2024-09-16T00:10:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021209409999999999 2024-09-16T00:10:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.685 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680197+0000", 2024-09-16T00:10:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0068000000000002e-05 2024-09-16T00:10:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680234+0000", 2024-09-16T00:10:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6758000000000003e-05 2024-09-16T00:10:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4127 2.18 2:1cc859d9:::200.00000012:head [write 808688~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:43.686 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.999150892999999, 2024-09-16T00:10:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019226493000000001, 2024-09-16T00:10:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.687 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661186+0000", 2024-09-16T00:10:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2399999999999996e-07 2024-09-16T00:10:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661192+0000", 2024-09-16T00:10:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9259999999999997e-06 2024-09-16T00:10:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.688 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661203+0000", 2024-09-16T00:10:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1174e-05 2024-09-16T00:10:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661213+0000", 2024-09-16T00:10:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0579e-05 2024-09-16T00:10:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673408+0000", 2024-09-16T00:10:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012194442 2024-09-16T00:10:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.689 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673482+0000", 2024-09-16T00:10:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4258000000000006e-05 2024-09-16T00:10:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673592+0000", 2024-09-16T00:10:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109733 2024-09-16T00:10:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678033+0000", 2024-09-16T00:10:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044411629999999997 2024-09-16T00:10:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.690 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680287+0000", 2024-09-16T00:10:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022538469999999998 2024-09-16T00:10:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680367+0000", 2024-09-16T00:10:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9808999999999994e-05 2024-09-16T00:10:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680411+0000", 2024-09-16T00:10:43.691 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4638000000000002e-05 2024-09-16T00:10:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4128 2.18 2:1cc859d9:::200.00000012:head [write 810933~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.998731761999998, 2024-09-16T00:10:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018982046999999998, 2024-09-16T00:10:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.692 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661605+0000", 2024-09-16T00:10:43.693 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.031e-06 2024-09-16T00:10:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661610+0000", 2024-09-16T00:10:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5010000000000004e-06 2024-09-16T00:10:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661623+0000", 2024-09-16T00:10:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2286000000000001e-05 2024-09-16T00:10:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.694 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661630+0000", 2024-09-16T00:10:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000002e-06 2024-09-16T00:10:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674011+0000", 2024-09-16T00:10:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012380923 2024-09-16T00:10:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674092+0000", 2024-09-16T00:10:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1267999999999995e-05 2024-09-16T00:10:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.695 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674216+0000", 2024-09-16T00:10:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012331700000000001 2024-09-16T00:10:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678039+0000", 2024-09-16T00:10:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038232489999999999 2024-09-16T00:10:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680493+0000", 2024-09-16T00:10:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024543429999999999 2024-09-16T00:10:43.696 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680546+0000", 2024-09-16T00:10:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3239000000000003e-05 2024-09-16T00:10:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680586+0000", 2024-09-16T00:10:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9496999999999998e-05 2024-09-16T00:10:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.697 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4129 2.18 2:1cc859d9:::200.00000012:head [write 813178~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.998374611000003, 2024-09-16T00:10:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018785932000000002, 2024-09-16T00:10:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.698 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661962+0000", 2024-09-16T00:10:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-07 2024-09-16T00:10:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.699 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661966+0000", 2024-09-16T00:10:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-06 2024-09-16T00:10:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.662015+0000", 2024-09-16T00:10:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8364e-05 2024-09-16T00:10:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.662023+0000", 2024-09-16T00:10:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2090000000000005e-06 2024-09-16T00:10:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.700 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674744+0000", 2024-09-16T00:10:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012721423000000001 2024-09-16T00:10:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674825+0000", 2024-09-16T00:10:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1036000000000001e-05 2024-09-16T00:10:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674946+0000", 2024-09-16T00:10:43.701 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120727 2024-09-16T00:10:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678045+0000", 2024-09-16T00:10:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030993280000000002 2024-09-16T00:10:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680646+0000", 2024-09-16T00:10:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026005300000000002 2024-09-16T00:10:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.702 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680705+0000", 2024-09-16T00:10:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9141000000000002e-05 2024-09-16T00:10:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680747+0000", 2024-09-16T00:10:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2064000000000002e-05 2024-09-16T00:10:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.703 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4184 2.18 2:1cc859d9:::200.00000012:head [write 927308~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.954326016000003, 2024-09-16T00:10:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023503514999999999, 2024-09-16T00:10:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.704 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706011+0000", 2024-09-16T00:10:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8400000000000003e-07 2024-09-16T00:10:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706016+0000", 2024-09-16T00:10:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7479999999999999e-06 2024-09-16T00:10:43.705 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706028+0000", 2024-09-16T00:10:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1477e-05 2024-09-16T00:10:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706035+0000", 2024-09-16T00:10:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1509999999999998e-06 2024-09-16T00:10:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.706 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.719820+0000", 2024-09-16T00:10:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013785168 2024-09-16T00:10:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.719888+0000", 2024-09-16T00:10:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7631000000000004e-05 2024-09-16T00:10:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720003+0000", 2024-09-16T00:10:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114849 2024-09-16T00:10:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.707 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725150+0000", 2024-09-16T00:10:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051469970000000004 2024-09-16T00:10:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729386+0000", 2024-09-16T00:10:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042368529999999996 2024-09-16T00:10:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729461+0000", 2024-09-16T00:10:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4935e-05 2024-09-16T00:10:43.708 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729513+0000", 2024-09-16T00:10:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1822e-05 2024-09-16T00:10:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4185 2.18 2:1cc859d9:::200.00000012:head [write 929217~3264 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.953319835999999, 2024-09-16T00:10:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023998430000000001, 2024-09-16T00:10:43.709 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.710 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707017+0000", 2024-09-16T00:10:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9999999999999996e-07 2024-09-16T00:10:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707023+0000", 2024-09-16T00:10:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1789999999999996e-06 2024-09-16T00:10:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707035+0000", 2024-09-16T00:10:43.711 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2106000000000001e-05 2024-09-16T00:10:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707042+0000", 2024-09-16T00:10:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2960000000000001e-06 2024-09-16T00:10:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720503+0000", 2024-09-16T00:10:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013460952999999999 2024-09-16T00:10:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.712 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720583+0000", 2024-09-16T00:10:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0171000000000002e-05 2024-09-16T00:10:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720668+0000", 2024-09-16T00:10:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4128999999999994e-05 2024-09-16T00:10:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725193+0000", 2024-09-16T00:10:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045257700000000001 2024-09-16T00:10:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.713 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.730924+0000", 2024-09-16T00:10:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057307690000000001 2024-09-16T00:10:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.730972+0000", 2024-09-16T00:10:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7756000000000002e-05 2024-09-16T00:10:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731014+0000", 2024-09-16T00:10:43.714 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2401000000000003e-05 2024-09-16T00:10:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4186 2.18 2:1cc859d9:::200.00000012:head [write 932481~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.952731679000003, 2024-09-16T00:10:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023544934999999999, 2024-09-16T00:10:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.715 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707605+0000", 2024-09-16T00:10:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1500000000000003e-07 2024-09-16T00:10:43.716 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707610+0000", 2024-09-16T00:10:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7049999999999998e-06 2024-09-16T00:10:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707618+0000", 2024-09-16T00:10:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6149999999999997e-06 2024-09-16T00:10:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707623+0000", 2024-09-16T00:10:43.717 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0660000000000003e-06 2024-09-16T00:10:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721143+0000", 2024-09-16T00:10:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01351976 2024-09-16T00:10:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721221+0000", 2024-09-16T00:10:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7918999999999997e-05 2024-09-16T00:10:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.718 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721335+0000", 2024-09-16T00:10:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113781 2024-09-16T00:10:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725236+0000", 2024-09-16T00:10:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039010379999999999 2024-09-16T00:10:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731025+0000", 2024-09-16T00:10:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057896659999999997 2024-09-16T00:10:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.719 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731091+0000", 2024-09-16T00:10:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.567e-05 2024-09-16T00:10:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731149+0000", 2024-09-16T00:10:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7800000000000002e-05 2024-09-16T00:10:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.720 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4187 2.18 2:1cc859d9:::200.00000012:head [write 934390~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.951879550000001, 2024-09-16T00:10:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022820340000000001, 2024-09-16T00:10:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.721 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708457+0000", 2024-09-16T00:10:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6899999999999996e-07 2024-09-16T00:10:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.722 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708462+0000", 2024-09-16T00:10:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3090000000000003e-06 2024-09-16T00:10:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708472+0000", 2024-09-16T00:10:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7499999999999998e-06 2024-09-16T00:10:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708479+0000", 2024-09-16T00:10:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1189999999999999e-06 2024-09-16T00:10:43.723 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721895+0000", 2024-09-16T00:10:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013415664000000001 2024-09-16T00:10:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721941+0000", 2024-09-16T00:10:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6031e-05 2024-09-16T00:10:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722007+0000", 2024-09-16T00:10:43.724 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6503999999999999e-05 2024-09-16T00:10:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725277+0000", 2024-09-16T00:10:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003269554 2024-09-16T00:10:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731158+0000", 2024-09-16T00:10:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058813750000000003 2024-09-16T00:10:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.725 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731240+0000", 2024-09-16T00:10:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1815000000000002e-05 2024-09-16T00:10:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731276+0000", 2024-09-16T00:10:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6449999999999998e-05 2024-09-16T00:10:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.726 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4188 2.18 2:1cc859d9:::200.00000012:head [write 936299~2419 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.951458606999999, 2024-09-16T00:10:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022518565000000001, 2024-09-16T00:10:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.727 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708878+0000", 2024-09-16T00:10:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8800000000000001e-07 2024-09-16T00:10:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708884+0000", 2024-09-16T00:10:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9800000000000003e-06 2024-09-16T00:10:43.728 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708895+0000", 2024-09-16T00:10:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1416e-05 2024-09-16T00:10:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708903+0000", 2024-09-16T00:10:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4449999999999999e-06 2024-09-16T00:10:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722327+0000", 2024-09-16T00:10:43.729 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013424681000000001 2024-09-16T00:10:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722408+0000", 2024-09-16T00:10:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0092000000000005e-05 2024-09-16T00:10:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722538+0000", 2024-09-16T00:10:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013046600000000001 2024-09-16T00:10:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.730 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726665+0000", 2024-09-16T00:10:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041267420000000001 2024-09-16T00:10:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731284+0000", 2024-09-16T00:10:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046195469999999999 2024-09-16T00:10:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731358+0000", 2024-09-16T00:10:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4002000000000001e-05 2024-09-16T00:10:43.731 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731396+0000", 2024-09-16T00:10:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7305999999999998e-05 2024-09-16T00:10:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4189 2.18 2:1cc859d9:::200.00000012:head [write 938718~1910 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.950551490999999, 2024-09-16T00:10:43.732 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021703626, 2024-09-16T00:10:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.733 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709785+0000", 2024-09-16T00:10:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.043e-06 2024-09-16T00:10:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709791+0000", 2024-09-16T00:10:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9499999999999998e-06 2024-09-16T00:10:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709804+0000", 2024-09-16T00:10:43.734 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2958e-05 2024-09-16T00:10:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709812+0000", 2024-09-16T00:10:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9869999999999999e-06 2024-09-16T00:10:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723058+0000", 2024-09-16T00:10:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013245497 2024-09-16T00:10:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.735 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723131+0000", 2024-09-16T00:10:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3879000000000002e-05 2024-09-16T00:10:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723241+0000", 2024-09-16T00:10:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109435 2024-09-16T00:10:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726708+0000", 2024-09-16T00:10:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034671340000000002 2024-09-16T00:10:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731403+0000", 2024-09-16T00:10:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046953979999999999 2024-09-16T00:10:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731454+0000", 2024-09-16T00:10:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0747e-05 2024-09-16T00:10:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731488+0000", 2024-09-16T00:10:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3597999999999999e-05 2024-09-16T00:10:43.737 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4190 2.18 2:1cc859d9:::200.00000012:head [write 940628~2420 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.950079326000001, 2024-09-16T00:10:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021346812999999999, 2024-09-16T00:10:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710257+0000", 2024-09-16T00:10:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8100000000000001e-07 2024-09-16T00:10:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710263+0000", 2024-09-16T00:10:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8050000000000003e-06 2024-09-16T00:10:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710273+0000", 2024-09-16T00:10:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0005e-05 2024-09-16T00:10:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710280+0000", 2024-09-16T00:10:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3819999999999998e-06 2024-09-16T00:10:43.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723653+0000", 2024-09-16T00:10:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013372866000000001 2024-09-16T00:10:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723733+0000", 2024-09-16T00:10:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9396999999999999e-05 2024-09-16T00:10:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723852+0000", 2024-09-16T00:10:43.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011919899999999999 2024-09-16T00:10:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726752+0000", 2024-09-16T00:10:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028999659999999999 2024-09-16T00:10:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731495+0000", 2024-09-16T00:10:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047432209999999997 2024-09-16T00:10:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731568+0000", 2024-09-16T00:10:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2452000000000004e-05 2024-09-16T00:10:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731603+0000", 2024-09-16T00:10:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5639000000000002e-05 2024-09-16T00:10:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4192 2.18 2:1cc859d9:::200.00000012:head [write 943048~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:43.743 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.949346486000003, 2024-09-16T00:10:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021688433999999999, 2024-09-16T00:10:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710990+0000", 2024-09-16T00:10:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1999999999999998e-07 2024-09-16T00:10:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710996+0000", 2024-09-16T00:10:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:10:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711007+0000", 2024-09-16T00:10:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1471000000000001e-05 2024-09-16T00:10:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711014+0000", 2024-09-16T00:10:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7780000000000002e-06 2024-09-16T00:10:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724471+0000", 2024-09-16T00:10:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013457435 2024-09-16T00:10:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724547+0000", 2024-09-16T00:10:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5383999999999995e-05 2024-09-16T00:10:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724663+0000", 2024-09-16T00:10:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116348 2024-09-16T00:10:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731612+0000", 2024-09-16T00:10:43.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069491409999999998 2024-09-16T00:10:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732573+0000", 2024-09-16T00:10:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0009611 2024-09-16T00:10:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732638+0000", 2024-09-16T00:10:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919000000000006e-05 2024-09-16T00:10:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732678+0000", 2024-09-16T00:10:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9230999999999998e-05 2024-09-16T00:10:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4193 2.18 2:1cc859d9:::200.00000012:head [write 944447~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.948800777999999, 2024-09-16T00:10:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021286851999999998, 2024-09-16T00:10:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.749 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711536+0000", 2024-09-16T00:10:43.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0189999999999999e-06 2024-09-16T00:10:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711542+0000", 2024-09-16T00:10:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7169999999999996e-06 2024-09-16T00:10:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711554+0000", 2024-09-16T00:10:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2089000000000001e-05 2024-09-16T00:10:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711559+0000", 2024-09-16T00:10:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3739999999999997e-06 2024-09-16T00:10:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725311+0000", 2024-09-16T00:10:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013751559 2024-09-16T00:10:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725390+0000", 2024-09-16T00:10:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9779999999999998e-05 2024-09-16T00:10:43.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725509+0000", 2024-09-16T00:10:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118973 2024-09-16T00:10:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731617+0000", 2024-09-16T00:10:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061079000000000003 2024-09-16T00:10:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732762+0000", 2024-09-16T00:10:43.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011449769999999999 2024-09-16T00:10:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732796+0000", 2024-09-16T00:10:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3587999999999998e-05 2024-09-16T00:10:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732822+0000", 2024-09-16T00:10:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5876000000000001e-05 2024-09-16T00:10:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.754 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4194 2.18 2:1cc859d9:::200.00000012:head [write 946356~2929 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.947825747000003, 2024-09-16T00:10:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020453244999999998, 2024-09-16T00:10:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:43.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712511+0000", 2024-09-16T00:10:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0999999999999997e-07 2024-09-16T00:10:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.756 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712517+0000", 2024-09-16T00:10:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8370000000000002e-06 2024-09-16T00:10:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712528+0000", 2024-09-16T00:10:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1528e-05 2024-09-16T00:10:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712536+0000", 2024-09-16T00:10:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3939999999999998e-06 2024-09-16T00:10:43.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726004+0000", 2024-09-16T00:10:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013468177 2024-09-16T00:10:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726085+0000", 2024-09-16T00:10:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1626999999999996e-05 2024-09-16T00:10:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726205+0000", 2024-09-16T00:10:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119751 2024-09-16T00:10:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731622+0000", 2024-09-16T00:10:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054165419999999999 2024-09-16T00:10:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732867+0000", 2024-09-16T00:10:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001244855 2024-09-16T00:10:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732924+0000", 2024-09-16T00:10:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7395999999999997e-05 2024-09-16T00:10:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732963+0000", 2024-09-16T00:10:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9227999999999998e-05 2024-09-16T00:10:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.760 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4195 2.18 2:1cc859d9:::200.00000012:head [write 949285~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.946774087000001, 2024-09-16T00:10:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019498099000000001, 2024-09-16T00:10:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713563+0000", 2024-09-16T00:10:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T00:10:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713569+0000", 2024-09-16T00:10:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8710000000000002e-06 2024-09-16T00:10:43.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713582+0000", 2024-09-16T00:10:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3134e-05 2024-09-16T00:10:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713591+0000", 2024-09-16T00:10:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9889999999999998e-06 2024-09-16T00:10:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726786+0000", 2024-09-16T00:10:43.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013194991999999999 2024-09-16T00:10:43.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726856+0000", 2024-09-16T00:10:43.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0017000000000004e-05 2024-09-16T00:10:43.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726967+0000", 2024-09-16T00:10:43.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111815 2024-09-16T00:10:43.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732530+0000", 2024-09-16T00:10:43.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055624940000000003 2024-09-16T00:10:43.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733001+0000", 2024-09-16T00:10:43.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000470817 2024-09-16T00:10:43.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.765 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.765 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733030+0000", 2024-09-16T00:10:43.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9051e-05 2024-09-16T00:10:43.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733060+0000", 2024-09-16T00:10:43.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9819e-05 2024-09-16T00:10:43.766 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.766 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.766 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.766 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4215 2.18 2:1cc859d9:::200.00000012:head [write 991028~1910 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.766 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:43.766 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.927078375999997, 2024-09-16T00:10:43.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019416676000000001, 2024-09-16T00:10:43.767 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.767 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.767 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.767 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:43.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:43.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733258+0000", 2024-09-16T00:10:43.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:10:43.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733264+0000", 2024-09-16T00:10:43.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4090000000000002e-06 2024-09-16T00:10:43.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733273+0000", 2024-09-16T00:10:43.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4769999999999992e-06 2024-09-16T00:10:43.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733280+0000", 2024-09-16T00:10:43.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2989999999999999e-06 2024-09-16T00:10:43.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.742966+0000", 2024-09-16T00:10:43.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009685585 2024-09-16T00:10:43.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743046+0000", 2024-09-16T00:10:43.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0469999999999994e-05 2024-09-16T00:10:43.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743165+0000", 2024-09-16T00:10:43.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118847 2024-09-16T00:10:43.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.746968+0000", 2024-09-16T00:10:43.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038025099999999998 2024-09-16T00:10:43.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752559+0000", 2024-09-16T00:10:43.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055911290000000002 2024-09-16T00:10:43.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752634+0000", 2024-09-16T00:10:43.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5102e-05 2024-09-16T00:10:43.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752674+0000", 2024-09-16T00:10:43.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9878e-05 2024-09-16T00:10:43.772 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.772 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.772 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.772 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4216 2.18 2:1cc859d9:::200.00000012:head [write 992938~2755 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:43.772 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:43.772 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.926532467999998, 2024-09-16T00:10:43.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019035976, 2024-09-16T00:10:43.772 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:43.772 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:43.772 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:43.773 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:43.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:43.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:43.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:43.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:43.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:43.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:43.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733804+0000", 2024-09-16T00:10:43.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2299999999999999e-07 2024-09-16T00:10:43.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:43.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733810+0000", 2024-09-16T00:10:43.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7259999999999999e-06 2024-09-16T00:10:43.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:43.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733822+0000", 2024-09-16T00:10:43.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2204e-05 2024-09-16T00:10:43.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:43.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733829+0000", 2024-09-16T00:10:43.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3889999999999999e-06 2024-09-16T00:10:43.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:43.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743675+0000", 2024-09-16T00:10:43.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098457990000000006 2024-09-16T00:10:43.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:43.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743752+0000", 2024-09-16T00:10:43.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6279000000000006e-05 2024-09-16T00:10:43.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:43.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743867+0000", 2024-09-16T00:10:43.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115319 2024-09-16T00:10:43.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:43.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.746971+0000", 2024-09-16T00:10:43.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031042399999999999 2024-09-16T00:10:43.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:43.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752723+0000", 2024-09-16T00:10:43.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057518329999999996 2024-09-16T00:10:43.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:43.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752804+0000", 2024-09-16T00:10:43.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0822999999999995e-05 2024-09-16T00:10:43.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:43.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:43.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:43.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752839+0000", 2024-09-16T00:10:43.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5441000000000002e-05 2024-09-16T00:10:43.777 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.777 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.777 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.777 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:43.778 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:43.778 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:43.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.663+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7ff4ec02c640 con 0x7ff4ec2067d0 2024-09-16T00:10:43.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.666+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7ff4ec02c640 con 0x7ff4ec2067d0 2024-09-16T00:10:43.779 INFO:teuthology.orchestra.run.smithi139.stdout:osd.6: {} 2024-09-16T00:10:43.780 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:10:43.781 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T00:10:43.781 INFO:teuthology.orchestra.run.smithi139.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T00:10:43.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.666+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=121}) -- 0x7ff4ec20a690 con 0x7ff4ec151730 2024-09-16T00:10:43.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.666+0000 7ff4f25d6640 1 -- 172.21.15.139:0/1931277779 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff4ec20a900 con 0x7ff4ec151730 2024-09-16T00:10:43.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:43.666+0000 7ff4e8ff9640 1 -- 172.21.15.139:0/1931277779 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_get_version_reply(handle=1 version=120) ==== 24+0+0 (secure 0 0 0) 0x7ff4e0087060 con 0x7ff4ec151730 2024-09-16T00:10:43.781 INFO:teuthology.orchestra.run.smithi139.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-09-16T00:10:43.782 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T00:10:43.782 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:43.778+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:43.880 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:43.879+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:43.893 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:43.893 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:43.894 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:43.894 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:43.905 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:10:43.981 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:43.980+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:44.071 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:44.071 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:44.071 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:44.071 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:44.071 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:44.080 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:10:44.082 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:44.081+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:44.182 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:44.182+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:44.220 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5178 3.2c 3:375e1b9b:::100000030c8.00000002:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.409690427999998, 2024-09-16T00:10:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012118438, 2024-09-16T00:10:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.221 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.222 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808909+0000", 2024-09-16T00:10:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8160000000000001e-06 2024-09-16T00:10:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808921+0000", 2024-09-16T00:10:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1117999999999999e-05 2024-09-16T00:10:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808941+0000", 2024-09-16T00:10:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0712000000000001e-05 2024-09-16T00:10:44.223 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808952+0000", 2024-09-16T00:10:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0193e-05 2024-09-16T00:10:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809019+0000", 2024-09-16T00:10:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7404999999999996e-05 2024-09-16T00:10:44.224 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809100+0000", 2024-09-16T00:10:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1029e-05 2024-09-16T00:10:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809206+0000", 2024-09-16T00:10:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105603 2024-09-16T00:10:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:44.225 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.815162+0000", 2024-09-16T00:10:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059567250000000004 2024-09-16T00:10:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.820925+0000", 2024-09-16T00:10:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057623479999999996 2024-09-16T00:10:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.820987+0000", 2024-09-16T00:10:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2680000000000003e-05 2024-09-16T00:10:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.226 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.821026+0000", 2024-09-16T00:10:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8809000000000001e-05 2024-09-16T00:10:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5196 3.26 3:66ea4dfb:::100000030f0.00000001:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:44.227 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.408503400000001, 2024-09-16T00:10:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095222459999999998, 2024-09-16T00:10:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.228 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3499999999999995e-07 2024-09-16T00:10:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810102+0000", 2024-09-16T00:10:44.229 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7390000000000002e-06 2024-09-16T00:10:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810112+0000", 2024-09-16T00:10:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7550000000000006e-06 2024-09-16T00:10:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810127+0000", 2024-09-16T00:10:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4749999999999999e-05 2024-09-16T00:10:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.230 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810184+0000", 2024-09-16T00:10:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7701999999999997e-05 2024-09-16T00:10:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810256+0000", 2024-09-16T00:10:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1595999999999997e-05 2024-09-16T00:10:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810346+0000", 2024-09-16T00:10:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0043999999999994e-05 2024-09-16T00:10:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.231 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.815239+0000", 2024-09-16T00:10:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048934470000000004 2024-09-16T00:10:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819526+0000", 2024-09-16T00:10:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042868630000000001 2024-09-16T00:10:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819577+0000", 2024-09-16T00:10:44.232 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0445999999999998e-05 2024-09-16T00:10:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819617+0000", 2024-09-16T00:10:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0169000000000001e-05 2024-09-16T00:10:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5244 3.20 3:068a06d7:::100000032f2.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:44.233 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.391407893, 2024-09-16T00:10:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094083010000000009, 2024-09-16T00:10:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.234 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827191+0000", 2024-09-16T00:10:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2330000000000001e-06 2024-09-16T00:10:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827195+0000", 2024-09-16T00:10:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8750000000000002e-06 2024-09-16T00:10:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.235 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827206+0000", 2024-09-16T00:10:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0473e-05 2024-09-16T00:10:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827213+0000", 2024-09-16T00:10:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:10:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827275+0000", 2024-09-16T00:10:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2532000000000004e-05 2024-09-16T00:10:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.236 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827392+0000", 2024-09-16T00:10:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116266 2024-09-16T00:10:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.836599+0000", 2024-09-16T00:10:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092068069999999991 2024-09-16T00:10:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.237 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5252 2.1a 2:5913b086:::1000000336d.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 41.390472070000001, 2024-09-16T00:10:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010086441999999999, 2024-09-16T00:10:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.238 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828127+0000", 2024-09-16T00:10:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:10:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828140+0000", 2024-09-16T00:10:44.239 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3614e-05 2024-09-16T00:10:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828149+0000", 2024-09-16T00:10:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7250000000000003e-06 2024-09-16T00:10:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828156+0000", 2024-09-16T00:10:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9430000000000003e-06 2024-09-16T00:10:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.240 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828274+0000", 2024-09-16T00:10:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118121 2024-09-16T00:10:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828381+0000", 2024-09-16T00:10:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107379 2024-09-16T00:10:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.838212+0000", 2024-09-16T00:10:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098310089999999999 2024-09-16T00:10:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.241 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5743 3.36 3:6c273be3:::1000000385e.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97)", 2024-09-16T00:10:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:03.854364+0000", 2024-09-16T00:10:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 40.364234226999997, 2024-09-16T00:10:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099844070000000007, 2024-09-16T00:10:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.242 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854364+0000", 2024-09-16T00:10:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854364+0000", 2024-09-16T00:10:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854365+0000", 2024-09-16T00:10:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3399999999999999e-07 2024-09-16T00:10:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.243 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854368+0000", 2024-09-16T00:10:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4300000000000002e-06 2024-09-16T00:10:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854376+0000", 2024-09-16T00:10:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2770000000000005e-06 2024-09-16T00:10:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854384+0000", 2024-09-16T00:10:44.244 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7200000000000006e-06 2024-09-16T00:10:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854455+0000", 2024-09-16T00:10:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0631999999999997e-05 2024-09-16T00:10:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854560+0000", 2024-09-16T00:10:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105338 2024-09-16T00:10:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.245 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.864348+0000", 2024-09-16T00:10:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097883759999999997 2024-09-16T00:10:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:6601 3.35 3:af7d55b1:::10000003232.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100)", 2024-09-16T00:10:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:06.866600+0000", 2024-09-16T00:10:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 37.351997783000002, 2024-09-16T00:10:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092432980000000005, 2024-09-16T00:10:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:44.246 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866600+0000", 2024-09-16T00:10:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866600+0000", 2024-09-16T00:10:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.247 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866602+0000", 2024-09-16T00:10:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.649e-06 2024-09-16T00:10:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866607+0000", 2024-09-16T00:10:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7330000000000003e-06 2024-09-16T00:10:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.248 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866618+0000", 2024-09-16T00:10:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1506e-05 2024-09-16T00:10:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866626+0000", 2024-09-16T00:10:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.498e-06 2024-09-16T00:10:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866685+0000", 2024-09-16T00:10:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8950000000000003e-05 2024-09-16T00:10:44.249 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866809+0000", 2024-09-16T00:10:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012389899999999999 2024-09-16T00:10:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875844+0000", 2024-09-16T00:10:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090350629999999994 2024-09-16T00:10:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.250 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:8541 3.3f e104/94 3:fd643af1:::10000001465.00000000:head v 104'282, mlcod=103'280)", 2024-09-16T00:10:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:12.911145+0000", 2024-09-16T00:10:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.307453203000001, 2024-09-16T00:10:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094161980000000006, 2024-09-16T00:10:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911145+0000", 2024-09-16T00:10:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.251 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911145+0000", 2024-09-16T00:10:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911146+0000", 2024-09-16T00:10:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0699999999999996e-07 2024-09-16T00:10:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911151+0000", 2024-09-16T00:10:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8710000000000002e-06 2024-09-16T00:10:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.252 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911161+0000", 2024-09-16T00:10:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9350000000000006e-06 2024-09-16T00:10:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911168+0000", 2024-09-16T00:10:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6340000000000003e-06 2024-09-16T00:10:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911238+0000", 2024-09-16T00:10:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0118999999999997e-05 2024-09-16T00:10:44.253 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911258+0000", 2024-09-16T00:10:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9361999999999999e-05 2024-09-16T00:10:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.920518+0000", 2024-09-16T00:10:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092603249999999998 2024-09-16T00:10:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.254 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.920561+0000", 2024-09-16T00:10:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3244999999999997e-05 2024-09-16T00:10:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:8587 3.35 3:ad562626:::10000001599.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104)", 2024-09-16T00:10:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:12.911217+0000", 2024-09-16T00:10:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 31.307381275000001, 2024-09-16T00:10:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095415759999999995, 2024-09-16T00:10:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.255 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911217+0000", 2024-09-16T00:10:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911217+0000", 2024-09-16T00:10:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911218+0000", 2024-09-16T00:10:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:10:44.256 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911222+0000", 2024-09-16T00:10:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.301e-06 2024-09-16T00:10:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911232+0000", 2024-09-16T00:10:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0244000000000001e-05 2024-09-16T00:10:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.257 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911239+0000", 2024-09-16T00:10:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9e-06 2024-09-16T00:10:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911324+0000", 2024-09-16T00:10:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5222999999999994e-05 2024-09-16T00:10:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911496+0000", 2024-09-16T00:10:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000172065 2024-09-16T00:10:44.258 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911600+0000", 2024-09-16T00:10:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000104002 2024-09-16T00:10:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.916599+0000", 2024-09-16T00:10:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004998911 2024-09-16T00:10:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.920663+0000", 2024-09-16T00:10:44.259 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040635799999999998 2024-09-16T00:10:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.920715+0000", 2024-09-16T00:10:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2549000000000001e-05 2024-09-16T00:10:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.920758+0000", 2024-09-16T00:10:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3018e-05 2024-09-16T00:10:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.260 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:11691 3.35 3:ad66404a:::10000002d4d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111)", 2024-09-16T00:10:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:22.048217+0000", 2024-09-16T00:10:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 22.170381587000001, 2024-09-16T00:10:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010683378, 2024-09-16T00:10:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048217+0000", 2024-09-16T00:10:44.261 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048217+0000", 2024-09-16T00:10:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048218+0000", 2024-09-16T00:10:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.722e-06 2024-09-16T00:10:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.262 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048223+0000", 2024-09-16T00:10:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0590000000000002e-06 2024-09-16T00:10:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048239+0000", 2024-09-16T00:10:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5271e-05 2024-09-16T00:10:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048246+0000", 2024-09-16T00:10:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8490000000000006e-06 2024-09-16T00:10:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.263 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048300+0000", 2024-09-16T00:10:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3109999999999998e-05 2024-09-16T00:10:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048495+0000", 2024-09-16T00:10:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000195198 2024-09-16T00:10:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048583+0000", 2024-09-16T00:10:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8472000000000003e-05 2024-09-16T00:10:44.264 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.049175+0000", 2024-09-16T00:10:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00059207800000000001 2024-09-16T00:10:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.058797+0000", 2024-09-16T00:10:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0096219329999999992 2024-09-16T00:10:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:44.265 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.058856+0000", 2024-09-16T00:10:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8782000000000001e-05 2024-09-16T00:10:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.058900+0000", 2024-09-16T00:10:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3903999999999998e-05 2024-09-16T00:10:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:12961 2.1a 2:595229b5:::10000000cd4.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.088496+0000", 2024-09-16T00:10:44.266 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.130102486999998, 2024-09-16T00:10:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013027634, 2024-09-16T00:10:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088496+0000", 2024-09-16T00:10:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.267 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088496+0000", 2024-09-16T00:10:44.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088496+0000", 2024-09-16T00:10:44.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:10:44.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.268 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.268 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.268 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088498+0000", 2024-09-16T00:10:44.268 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.317e-06 2024-09-16T00:10:44.268 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088506+0000", 2024-09-16T00:10:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1459999999999999e-06 2024-09-16T00:10:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088511+0000", 2024-09-16T00:10:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.541e-06 2024-09-16T00:10:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088815+0000", 2024-09-16T00:10:44.269 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030421000000000002 2024-09-16T00:10:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088988+0000", 2024-09-16T00:10:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000172356 2024-09-16T00:10:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.089076+0000", 2024-09-16T00:10:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8270999999999996e-05 2024-09-16T00:10:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:44.270 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.090357+0000", 2024-09-16T00:10:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001280661 2024-09-16T00:10:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.101242+0000", 2024-09-16T00:10:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010885238 2024-09-16T00:10:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.101489+0000", 2024-09-16T00:10:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000247538 2024-09-16T00:10:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.271 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.101523+0000", 2024-09-16T00:10:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.379e-05 2024-09-16T00:10:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13112 3.2c 3:36940359:::10000000d76.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.122269+0000", 2024-09-16T00:10:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.096328880000002, 2024-09-16T00:10:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010413552, 2024-09-16T00:10:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.272 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122269+0000", 2024-09-16T00:10:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122269+0000", 2024-09-16T00:10:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.273 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122270+0000", 2024-09-16T00:10:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6499999999999999e-07 2024-09-16T00:10:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122272+0000", 2024-09-16T00:10:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5189999999999999e-06 2024-09-16T00:10:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122279+0000", 2024-09-16T00:10:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7039999999999997e-06 2024-09-16T00:10:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.274 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122284+0000", 2024-09-16T00:10:44.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5670000000000002e-06 2024-09-16T00:10:44.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122835+0000", 2024-09-16T00:10:44.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00055031399999999999 2024-09-16T00:10:44.275 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.275 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.275 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.275 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122993+0000", 2024-09-16T00:10:44.275 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015790499999999999 2024-09-16T00:10:44.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:44.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.123078+0000", 2024-09-16T00:10:44.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5803000000000005e-05 2024-09-16T00:10:44.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.276 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:44.276 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.123884+0000", 2024-09-16T00:10:44.276 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00080534599999999997 2024-09-16T00:10:44.276 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.276 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.132579+0000", 2024-09-16T00:10:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086952409999999994 2024-09-16T00:10:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.132643+0000", 2024-09-16T00:10:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4289999999999996e-05 2024-09-16T00:10:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.132683+0000", 2024-09-16T00:10:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9397999999999998e-05 2024-09-16T00:10:44.277 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13227 3.26 3:64bfa412:::10000000dee.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.146915+0000", 2024-09-16T00:10:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.071683027999999, 2024-09-16T00:10:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012561259, 2024-09-16T00:10:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.278 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146915+0000", 2024-09-16T00:10:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146915+0000", 2024-09-16T00:10:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.279 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146916+0000", 2024-09-16T00:10:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.155e-06 2024-09-16T00:10:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146922+0000", 2024-09-16T00:10:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3249999999999998e-06 2024-09-16T00:10:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146938+0000", 2024-09-16T00:10:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5974999999999999e-05 2024-09-16T00:10:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.280 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146946+0000", 2024-09-16T00:10:44.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3189999999999995e-06 2024-09-16T00:10:44.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.147095+0000", 2024-09-16T00:10:44.281 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014879800000000001 2024-09-16T00:10:44.281 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.281 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.281 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.281 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.147262+0000", 2024-09-16T00:10:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000167353 2024-09-16T00:10:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.147357+0000", 2024-09-16T00:10:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5199999999999997e-05 2024-09-16T00:10:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.148582+0000", 2024-09-16T00:10:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001224679 2024-09-16T00:10:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.159363+0000", 2024-09-16T00:10:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010780839 2024-09-16T00:10:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.159422+0000", 2024-09-16T00:10:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9731999999999997e-05 2024-09-16T00:10:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.159476+0000", 2024-09-16T00:10:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3884000000000001e-05 2024-09-16T00:10:44.283 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13251 3.36 3:6f745cbe:::10000000e09.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.152775+0000", 2024-09-16T00:10:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.065823139999999, 2024-09-16T00:10:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011312687, 2024-09-16T00:10:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.284 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152775+0000", 2024-09-16T00:10:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152775+0000", 2024-09-16T00:10:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152776+0000", 2024-09-16T00:10:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.393e-06 2024-09-16T00:10:44.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152782+0000", 2024-09-16T00:10:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2329999999999997e-06 2024-09-16T00:10:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152797+0000", 2024-09-16T00:10:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5515e-05 2024-09-16T00:10:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152808+0000", 2024-09-16T00:10:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1178e-05 2024-09-16T00:10:44.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152860+0000", 2024-09-16T00:10:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1644999999999998e-05 2024-09-16T00:10:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.153026+0000", 2024-09-16T00:10:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016563400000000001 2024-09-16T00:10:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:44.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.153127+0000", 2024-09-16T00:10:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010173 2024-09-16T00:10:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.154610+0000", 2024-09-16T00:10:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001482917 2024-09-16T00:10:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.164000+0000", 2024-09-16T00:10:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093899110000000008 2024-09-16T00:10:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.164048+0000", 2024-09-16T00:10:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8378000000000003e-05 2024-09-16T00:10:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.164088+0000", 2024-09-16T00:10:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9153000000000003e-05 2024-09-16T00:10:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13260 3.35 3:ad57d493:::10000000e12.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:44.289 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.152827+0000", 2024-09-16T00:10:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.065771534, 2024-09-16T00:10:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012977191000000001, 2024-09-16T00:10:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152827+0000", 2024-09-16T00:10:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152827+0000", 2024-09-16T00:10:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152827+0000", 2024-09-16T00:10:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2600000000000002e-07 2024-09-16T00:10:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152830+0000", 2024-09-16T00:10:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9229999999999998e-06 2024-09-16T00:10:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152838+0000", 2024-09-16T00:10:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8790000000000003e-06 2024-09-16T00:10:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152844+0000", 2024-09-16T00:10:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7769999999999999e-06 2024-09-16T00:10:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152892+0000", 2024-09-16T00:10:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7879e-05 2024-09-16T00:10:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.153052+0000", 2024-09-16T00:10:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016057199999999999 2024-09-16T00:10:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.153139+0000", 2024-09-16T00:10:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6679000000000001e-05 2024-09-16T00:10:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.155267+0000", 2024-09-16T00:10:44.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021280869999999999 2024-09-16T00:10:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.165714+0000", 2024-09-16T00:10:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010447303 2024-09-16T00:10:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.165764+0000", 2024-09-16T00:10:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.024e-05 2024-09-16T00:10:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.165804+0000", 2024-09-16T00:10:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9326000000000003e-05 2024-09-16T00:10:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13359 3.35 3:afdc6484:::10000000e78.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.173189+0000", 2024-09-16T00:10:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.045409548999999, 2024-09-16T00:10:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011094559, 2024-09-16T00:10:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.295 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173189+0000", 2024-09-16T00:10:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173189+0000", 2024-09-16T00:10:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173190+0000", 2024-09-16T00:10:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.249e-06 2024-09-16T00:10:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173194+0000", 2024-09-16T00:10:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T00:10:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173203+0000", 2024-09-16T00:10:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3500000000000003e-06 2024-09-16T00:10:44.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173210+0000", 2024-09-16T00:10:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.331e-06 2024-09-16T00:10:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173371+0000", 2024-09-16T00:10:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016161999999999999 2024-09-16T00:10:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173548+0000", 2024-09-16T00:10:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000176622 2024-09-16T00:10:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173645+0000", 2024-09-16T00:10:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7345999999999999e-05 2024-09-16T00:10:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.175459+0000", 2024-09-16T00:10:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001814062 2024-09-16T00:10:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.184205+0000", 2024-09-16T00:10:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0087454479999999994 2024-09-16T00:10:44.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.184249+0000", 2024-09-16T00:10:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4026999999999997e-05 2024-09-16T00:10:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.184283+0000", 2024-09-16T00:10:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4326999999999998e-05 2024-09-16T00:10:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13435 3.2c 3:34a22dc0:::10000000ec9.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.187542+0000", 2024-09-16T00:10:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.031056536000001, 2024-09-16T00:10:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011912524000000001, 2024-09-16T00:10:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187542+0000", 2024-09-16T00:10:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187542+0000", 2024-09-16T00:10:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187542+0000", 2024-09-16T00:10:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0999999999999998e-07 2024-09-16T00:10:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187545+0000", 2024-09-16T00:10:44.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6620000000000001e-06 2024-09-16T00:10:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187552+0000", 2024-09-16T00:10:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4030000000000001e-06 2024-09-16T00:10:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187558+0000", 2024-09-16T00:10:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5289999999999999e-06 2024-09-16T00:10:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.188067+0000", 2024-09-16T00:10:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050960000000000003 2024-09-16T00:10:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.188230+0000", 2024-09-16T00:10:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000162275 2024-09-16T00:10:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.188323+0000", 2024-09-16T00:10:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3778999999999998e-05 2024-09-16T00:10:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.189485+0000", 2024-09-16T00:10:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011611659999999999 2024-09-16T00:10:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199354+0000", 2024-09-16T00:10:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098691790000000005 2024-09-16T00:10:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:44.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199404+0000", 2024-09-16T00:10:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.041e-05 2024-09-16T00:10:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199454+0000", 2024-09-16T00:10:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9910999999999997e-05 2024-09-16T00:10:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13491 3.26 3:66db0c90:::10000000f05.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:44.306 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.199350+0000", 2024-09-16T00:10:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 18.019248021999999, 2024-09-16T00:10:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009654167, 2024-09-16T00:10:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199350+0000", 2024-09-16T00:10:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199350+0000", 2024-09-16T00:10:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199352+0000", 2024-09-16T00:10:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.621e-06 2024-09-16T00:10:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199361+0000", 2024-09-16T00:10:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0669999999999998e-06 2024-09-16T00:10:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199375+0000", 2024-09-16T00:10:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4474999999999999e-05 2024-09-16T00:10:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199383+0000", 2024-09-16T00:10:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8250000000000005e-06 2024-09-16T00:10:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199500+0000", 2024-09-16T00:10:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011675099999999999 2024-09-16T00:10:44.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199662+0000", 2024-09-16T00:10:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000161955 2024-09-16T00:10:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199749+0000", 2024-09-16T00:10:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7498999999999997e-05 2024-09-16T00:10:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.202268+0000", 2024-09-16T00:10:44.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025186829999999999 2024-09-16T00:10:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.208868+0000", 2024-09-16T00:10:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066001870000000004 2024-09-16T00:10:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.208962+0000", 2024-09-16T00:10:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3434e-05 2024-09-16T00:10:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.209004+0000", 2024-09-16T00:10:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2670000000000003e-05 2024-09-16T00:10:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:16859 3.35 3:ad0b5202:::1000000245b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116)", 2024-09-16T00:10:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:36.173717+0000", 2024-09-16T00:10:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.0448807640000002, 2024-09-16T00:10:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097368839999999995, 2024-09-16T00:10:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.312 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173717+0000", 2024-09-16T00:10:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173717+0000", 2024-09-16T00:10:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173718+0000", 2024-09-16T00:10:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.01e-07 2024-09-16T00:10:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173720+0000", 2024-09-16T00:10:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2029999999999999e-06 2024-09-16T00:10:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173727+0000", 2024-09-16T00:10:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6209999999999998e-06 2024-09-16T00:10:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173733+0000", 2024-09-16T00:10:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4960000000000001e-06 2024-09-16T00:10:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173787+0000", 2024-09-16T00:10:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4354999999999997e-05 2024-09-16T00:10:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173953+0000", 2024-09-16T00:10:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016555000000000001 2024-09-16T00:10:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.174041+0000", 2024-09-16T00:10:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7738999999999995e-05 2024-09-16T00:10:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.181641+0000", 2024-09-16T00:10:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007599939 2024-09-16T00:10:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.183355+0000", 2024-09-16T00:10:44.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017146869999999999 2024-09-16T00:10:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.183404+0000", 2024-09-16T00:10:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8999000000000003e-05 2024-09-16T00:10:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.183454+0000", 2024-09-16T00:10:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9894e-05 2024-09-16T00:10:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:16912 3.1e e116/94 3:79d63390:::1000000249f.00000000:head v 116'374, mlcod=116'372)", 2024-09-16T00:10:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:36.184345+0000", 2024-09-16T00:10:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.0342531200000007, 2024-09-16T00:10:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093746300000000001, 2024-09-16T00:10:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.318 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184345+0000", 2024-09-16T00:10:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184345+0000", 2024-09-16T00:10:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184346+0000", 2024-09-16T00:10:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6000000000000003e-07 2024-09-16T00:10:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184351+0000", 2024-09-16T00:10:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3530000000000002e-06 2024-09-16T00:10:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184360+0000", 2024-09-16T00:10:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9579999999999996e-06 2024-09-16T00:10:44.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184366+0000", 2024-09-16T00:10:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1279999999999996e-06 2024-09-16T00:10:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184704+0000", 2024-09-16T00:10:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00033787599999999997 2024-09-16T00:10:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184722+0000", 2024-09-16T00:10:44.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7533e-05 2024-09-16T00:10:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.193676+0000", 2024-09-16T00:10:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089544919999999997 2024-09-16T00:10:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.193720+0000", 2024-09-16T00:10:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3529999999999998e-05 2024-09-16T00:10:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.322 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:17004 2.17 2:eb27e31b:::1000000255e.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116)", 2024-09-16T00:10:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:36.199476+0000", 2024-09-16T00:10:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 8.0191224559999998, 2024-09-16T00:10:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094096179999999998, 2024-09-16T00:10:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199476+0000", 2024-09-16T00:10:44.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199476+0000", 2024-09-16T00:10:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199477+0000", 2024-09-16T00:10:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4619999999999999e-06 2024-09-16T00:10:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:44.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199484+0000", 2024-09-16T00:10:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.438e-06 2024-09-16T00:10:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199499+0000", 2024-09-16T00:10:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5512e-05 2024-09-16T00:10:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199508+0000", 2024-09-16T00:10:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7360000000000007e-06 2024-09-16T00:10:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199570+0000", 2024-09-16T00:10:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2130000000000003e-05 2024-09-16T00:10:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199691+0000", 2024-09-16T00:10:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121271 2024-09-16T00:10:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.208885+0000", 2024-09-16T00:10:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0091940689999999992 2024-09-16T00:10:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.326 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.327 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:44.327 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:44.327 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:44.328 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-16T00:10:44.329 INFO:tasks.ceph.osd.0.smithi139.stderr:2024-09-16T00:10:44.327+0000 7f265dae9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36443) UID: 0 2024-09-16T00:10:44.429 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:44.428+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:44.437 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:44.437 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:44.437 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:44.437 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:44.446 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-16T00:10:44.530 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:44.530+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:44.552 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-16T00:10:44.561 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:44.561 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:44.561 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:44.561 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:44.562 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:44.574 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-16T00:10:44.657 INFO:tasks.ceph.osd.2.smithi139.stderr:2024-09-16T00:10:44.657+0000 7fa4c7387640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36459) UID: 0 2024-09-16T00:10:44.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.677+0000 7fcad0386640 1 Processor -- start 2024-09-16T00:10:44.678 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.677+0000 7fcad0386640 1 -- start start 2024-09-16T00:10:44.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.678+0000 7fcad0386640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcac8151680 0x7fcac8157500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:44.679 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.678+0000 7fcad0386640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcac80c56d0 0x7fcac815b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:44.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.678+0000 7fcad0386640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcac8151a60 con 0x7fcac80c70d0 2024-09-16T00:10:44.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.679+0000 7fcad0386640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcac815aa70 con 0x7fcac8151680 2024-09-16T00:10:44.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.679+0000 7fcad0386640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcac80c5be0 con 0x7fcac80c56d0 2024-09-16T00:10:44.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.679+0000 7fcace923640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fcac80c70d0 0x7fcac80c3bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:40478/0 (socket says 172.21.15.139:40478) 2024-09-16T00:10:44.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.679+0000 7fcace122640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcac8151680 0x7fcac8157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:44.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.679+0000 7fcace923640 1 -- 172.21.15.139:0/4141626104 learned_addr learned my addr 172.21.15.139:0/4141626104 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:44.680 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.679+0000 7fcacd921640 1 --2- 172.21.15.139:0/4141626104 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcac80c56d0 0x7fcac815b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:44.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.679+0000 7fcacd921640 1 -- 172.21.15.139:0/4141626104 >> v2:172.21.15.139:3300/0 conn(0x7fcac8151680 msgr2=0x7fcac8157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:44.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.679+0000 7fcacd921640 1 --2- 172.21.15.139:0/4141626104 >> v2:172.21.15.139:3300/0 conn(0x7fcac8151680 0x7fcac8157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:44.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.679+0000 7fcacd921640 1 -- 172.21.15.139:0/4141626104 >> v1:172.21.15.139:6789/0 conn(0x7fcac80c70d0 legacy=0x7fcac80c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:44.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.679+0000 7fcacd921640 1 -- 172.21.15.139:0/4141626104 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac80c5ec0 con 0x7fcac80c56d0 2024-09-16T00:10:44.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.680+0000 7fcacd921640 1 --2- 172.21.15.139:0/4141626104 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcac80c56d0 0x7fcac815b510 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcab80097a0 tx=0x7fcab802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bd94e1fdfccac1f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:44.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.680+0000 7fcacd120640 1 -- 172.21.15.139:0/4141626104 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8037070 con 0x7fcac80c56d0 2024-09-16T00:10:44.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.680+0000 7fcacd120640 1 -- 172.21.15.139:0/4141626104 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcab8032440 con 0x7fcac80c56d0 2024-09-16T00:10:44.681 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.680+0000 7fcacd120640 1 -- 172.21.15.139:0/4141626104 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab8032810 con 0x7fcac80c56d0 2024-09-16T00:10:44.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.681+0000 7fcad0386640 1 -- 172.21.15.139:0/4141626104 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcac80c56d0 msgr2=0x7fcac815b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:44.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.681+0000 7fcad0386640 1 --2- 172.21.15.139:0/4141626104 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcac80c56d0 0x7fcac815b510 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcab80097a0 tx=0x7fcab802a180 comp rx=0 tx=0).stop 2024-09-16T00:10:44.682 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.681+0000 7fcad0386640 1 -- 172.21.15.139:0/4141626104 shutdown_connections 2024-09-16T00:10:44.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.681+0000 7fcad0386640 1 --2- 172.21.15.139:0/4141626104 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcac80c56d0 0x7fcac815b510 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:44.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.681+0000 7fcad0386640 1 --2- 172.21.15.139:0/4141626104 >> v2:172.21.15.139:3300/0 conn(0x7fcac8151680 0x7fcac8157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:44.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.681+0000 7fcad0386640 1 -- 172.21.15.139:0/4141626104 >> 172.21.15.139:0/4141626104 conn(0x7fcac80c91d0 msgr2=0x7fcac80c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:44.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.681+0000 7fcad0386640 1 -- 172.21.15.139:0/4141626104 shutdown_connections 2024-09-16T00:10:44.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.681+0000 7fcad0386640 1 -- 172.21.15.139:0/4141626104 wait complete. 2024-09-16T00:10:44.684 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-09-16T00:10:44.684 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-16T00:10:44.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.683+0000 7fcad0386640 1 Processor -- start 2024-09-16T00:10:44.685 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.683+0000 7fcad0386640 1 -- start start 2024-09-16T00:10:44.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcad0386640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac80c56d0 0x7fcac814c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:44.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcad0386640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcac80c70d0 0x7fcac814c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:44.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcad0386640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcac8151680 0x7fcac8159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:44.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcace923640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcac8151680 0x7fcac8159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:44.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcace122640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac80c56d0 0x7fcac814c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:44.686 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcacd921640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcac80c70d0 0x7fcac814c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:44.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcace923640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcac8151680 0x7fcac8159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38662/0 (socket says 172.21.15.139:38662) 2024-09-16T00:10:44.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcacd921640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcac80c70d0 0x7fcac814c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:60328/0 (socket says 172.21.15.139:60328) 2024-09-16T00:10:44.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcace923640 1 -- 172.21.15.139:0/1507230524 learned_addr learned my addr 172.21.15.139:0/1507230524 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:44.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcad0386640 1 -- 172.21.15.139:0/1507230524 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcac80c7640 con 0x7fcac8151680 2024-09-16T00:10:44.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcad0386640 1 -- 172.21.15.139:0/1507230524 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcac815ad10 con 0x7fcac80c56d0 2024-09-16T00:10:44.687 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcad0386640 1 -- 172.21.15.139:0/1507230524 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcac80c77c0 con 0x7fcac80c70d0 2024-09-16T00:10:44.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcace923640 1 -- 172.21.15.139:0/1507230524 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcac80c70d0 msgr2=0x7fcac814c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:44.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcace923640 1 --2- 172.21.15.139:0/1507230524 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcac80c70d0 0x7fcac814c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:44.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcace923640 1 -- 172.21.15.139:0/1507230524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac80c56d0 msgr2=0x7fcac814c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:44.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcace923640 1 --2- 172.21.15.139:0/1507230524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac80c56d0 0x7fcac814c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:44.688 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.685+0000 7fcace923640 1 -- 172.21.15.139:0/1507230524 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac80c2b40 con 0x7fcac8151680 2024-09-16T00:10:44.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.686+0000 7fcacd921640 1 --2- 172.21.15.139:0/1507230524 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcac80c70d0 0x7fcac814c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:10:44.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.686+0000 7fcace923640 1 --2- 172.21.15.139:0/1507230524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcac8151680 0x7fcac8159620 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcac000ce20 tx=0x7fcac000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:44.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.686+0000 7fcabeffd640 1 -- 172.21.15.139:0/1507230524 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0017070 con 0x7fcac8151680 2024-09-16T00:10:44.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.686+0000 7fcabeffd640 1 -- 172.21.15.139:0/1507230524 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac000ad40 con 0x7fcac8151680 2024-09-16T00:10:44.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.686+0000 7fcabeffd640 1 -- 172.21.15.139:0/1507230524 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0012600 con 0x7fcac8151680 2024-09-16T00:10:44.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.686+0000 7fcad0386640 1 -- 172.21.15.139:0/1507230524 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac80c2de0 con 0x7fcac8151680 2024-09-16T00:10:44.689 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.687+0000 7fcad0386640 1 -- 172.21.15.139:0/1507230524 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac8158070 con 0x7fcac8151680 2024-09-16T00:10:44.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.688+0000 7fcad0386640 1 -- 172.21.15.139:0/1507230524 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac80c8350 con 0x7fcac8151680 2024-09-16T00:10:44.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.689+0000 7fcabeffd640 1 -- 172.21.15.139:0/1507230524 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcac001a070 con 0x7fcac8151680 2024-09-16T00:10:44.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.689+0000 7fcabeffd640 1 --2- 172.21.15.139:0/1507230524 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcaac06c330 0x7fcaac06e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:44.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.689+0000 7fcabeffd640 1 -- 172.21.15.139:0/1507230524 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5969+0+0 (secure 0 0 0) 0x7fcac008b200 con 0x7fcac8151680 2024-09-16T00:10:44.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.689+0000 7fcace122640 1 --2- 172.21.15.139:0/1507230524 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcaac06c330 0x7fcaac06e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:44.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.690+0000 7fcace122640 1 --2- 172.21.15.139:0/1507230524 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcaac06c330 0x7fcaac06e7f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009aa0 tx=0x7fcac4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:44.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.695+0000 7fcabeffd640 1 -- 172.21.15.139:0/1507230524 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcac00591f0 con 0x7fcac8151680 2024-09-16T00:10:44.732 INFO:tasks.ceph.osd.3.smithi139.stderr:2024-09-16T00:10:44.731+0000 7fe1ce8e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36457) UID: 0 2024-09-16T00:10:44.745 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:44.745 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:219 2.1f 2:f83d65d8:::200.00000004:head [write 863658~2915703 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e65)", 2024-09-16T00:10:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:10:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 93.872516985000004, 2024-09-16T00:10:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013772627000000001, 2024-09-16T00:10:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:10:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870940+0000", 2024-09-16T00:10:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.870941+0000", 2024-09-16T00:10:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4e-07 2024-09-16T00:10:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877134+0000", 2024-09-16T00:10:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061936769999999999 2024-09-16T00:10:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877153+0000", 2024-09-16T00:10:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8408000000000001e-05 2024-09-16T00:10:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877161+0000", 2024-09-16T00:10:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2709999999999992e-06 2024-09-16T00:10:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877225+0000", 2024-09-16T00:10:44.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4190999999999996e-05 2024-09-16T00:10:44.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877289+0000", 2024-09-16T00:10:44.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4058000000000002e-05 2024-09-16T00:10:44.749 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.749 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.749 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:44.749 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.877376+0000", 2024-09-16T00:10:44.749 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7227999999999999e-05 2024-09-16T00:10:44.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.882925+0000", 2024-09-16T00:10:44.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055481050000000002 2024-09-16T00:10:44.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884635+0000", 2024-09-16T00:10:44.750 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017102759999999999 2024-09-16T00:10:44.750 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.750 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.750 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.750 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884674+0000", 2024-09-16T00:10:44.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8914e-05 2024-09-16T00:10:44.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.751 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.751 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:10.884712+0000", 2024-09-16T00:10:44.751 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658999999999997e-05 2024-09-16T00:10:44.751 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.751 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.751 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.751 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.751 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.751 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10768 3.2b 3:d738455c:::100000030db.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:44.751 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:10:44.751 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 66.349331812000003, 2024-09-16T00:10:44.752 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012411195999999999, 2024-09-16T00:10:44.752 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.752 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.752 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.752 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:44.752 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:44.752 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:44.752 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10768 2024-09-16T00:10:44.752 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.753 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:10:44.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.753 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394125+0000", 2024-09-16T00:10:44.753 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.753 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.753 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.753 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.394129+0000", 2024-09-16T00:10:44.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4100000000000001e-06 2024-09-16T00:10:44.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395723+0000", 2024-09-16T00:10:44.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015938409999999999 2024-09-16T00:10:44.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.754 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.754 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.754 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395745+0000", 2024-09-16T00:10:44.754 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1611e-05 2024-09-16T00:10:44.754 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395782+0000", 2024-09-16T00:10:44.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7292000000000001e-05 2024-09-16T00:10:44.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395856+0000", 2024-09-16T00:10:44.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3675000000000002e-05 2024-09-16T00:10:44.755 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.755 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.755 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.755 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.395924+0000", 2024-09-16T00:10:44.755 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8646999999999995e-05 2024-09-16T00:10:44.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:44.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.396069+0000", 2024-09-16T00:10:44.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014455199999999999 2024-09-16T00:10:44.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.403755+0000", 2024-09-16T00:10:44.756 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0076863160000000003 2024-09-16T00:10:44.756 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.756 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.756 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.756 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406407+0000", 2024-09-16T00:10:44.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026516220000000002 2024-09-16T00:10:44.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406477+0000", 2024-09-16T00:10:44.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0204999999999996e-05 2024-09-16T00:10:44.757 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.757 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.757 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.757 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.406536+0000", 2024-09-16T00:10:44.757 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9024999999999998e-05 2024-09-16T00:10:44.757 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.757 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.758 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.758 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.758 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10941 3.1 3:8246fad2:::10000003173.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:44.758 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:10:44.758 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.287578475000004, 2024-09-16T00:10:44.758 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01286782, 2024-09-16T00:10:44.758 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.758 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.758 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.758 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:44.758 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:44.758 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:44.758 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10941 2024-09-16T00:10:44.758 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.759 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:10:44.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.759 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455878+0000", 2024-09-16T00:10:44.759 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.759 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.759 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.759 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.455881+0000", 2024-09-16T00:10:44.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7269999999999999e-06 2024-09-16T00:10:44.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457462+0000", 2024-09-16T00:10:44.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015811690000000001 2024-09-16T00:10:44.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.760 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.760 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.760 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457487+0000", 2024-09-16T00:10:44.760 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4607e-05 2024-09-16T00:10:44.760 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457497+0000", 2024-09-16T00:10:44.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9939999999999995e-06 2024-09-16T00:10:44.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457571+0000", 2024-09-16T00:10:44.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3867000000000003e-05 2024-09-16T00:10:44.761 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.761 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.761 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.761 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457689+0000", 2024-09-16T00:10:44.761 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000118814 2024-09-16T00:10:44.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:44.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.457804+0000", 2024-09-16T00:10:44.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114702 2024-09-16T00:10:44.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.762 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.463422+0000", 2024-09-16T00:10:44.762 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0056179239999999998 2024-09-16T00:10:44.762 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.762 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.762 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.763 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468641+0000", 2024-09-16T00:10:44.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052192109999999996 2024-09-16T00:10:44.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.763 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.763 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468703+0000", 2024-09-16T00:10:44.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1268999999999999e-05 2024-09-16T00:10:44.763 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.763 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.763 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.763 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.468746+0000", 2024-09-16T00:10:44.763 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3535999999999998e-05 2024-09-16T00:10:44.763 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.763 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.764 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.764 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.764 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.764 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11742 3.1b 3:d847de26:::100000032cd.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:44.764 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:10:44.764 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 61.687086890000003, 2024-09-16T00:10:44.764 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012791903, 2024-09-16T00:10:44.764 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.764 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.764 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.764 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:44.764 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:44.764 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:44.765 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11742 2024-09-16T00:10:44.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.765 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:10:44.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.765 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.765 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056370+0000", 2024-09-16T00:10:44.765 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.765 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.765 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.056373+0000", 2024-09-16T00:10:44.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9799999999999998e-06 2024-09-16T00:10:44.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058604+0000", 2024-09-16T00:10:44.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022316020000000001 2024-09-16T00:10:44.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.766 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.766 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.766 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058630+0000", 2024-09-16T00:10:44.766 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5425000000000001e-05 2024-09-16T00:10:44.766 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058643+0000", 2024-09-16T00:10:44.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3361e-05 2024-09-16T00:10:44.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058731+0000", 2024-09-16T00:10:44.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7270999999999998e-05 2024-09-16T00:10:44.767 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.767 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.767 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.767 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.058894+0000", 2024-09-16T00:10:44.767 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016376800000000001 2024-09-16T00:10:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.059050+0000", 2024-09-16T00:10:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015603700000000001 2024-09-16T00:10:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.066364+0000", 2024-09-16T00:10:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073138639999999998 2024-09-16T00:10:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069025+0000", 2024-09-16T00:10:44.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026607660000000002 2024-09-16T00:10:44.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069083+0000", 2024-09-16T00:10:44.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7680999999999999e-05 2024-09-16T00:10:44.769 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.769 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.769 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.769 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.069162+0000", 2024-09-16T00:10:44.769 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9147999999999995e-05 2024-09-16T00:10:44.769 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.769 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.770 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.770 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.770 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.770 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11934 3.1b 3:d94fa8da:::10000003190.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:44.770 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:10:44.770 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 61.055043615000002, 2024-09-16T00:10:44.770 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.017110325999999999, 2024-09-16T00:10:44.770 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.770 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.770 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.770 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:44.770 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:44.770 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:44.771 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11934 2024-09-16T00:10:44.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.771 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:10:44.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.771 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.771 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688413+0000", 2024-09-16T00:10:44.771 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.771 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.771 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.688416+0000", 2024-09-16T00:10:44.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.875e-06 2024-09-16T00:10:44.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689633+0000", 2024-09-16T00:10:44.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001216997 2024-09-16T00:10:44.772 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.772 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.772 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.772 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689655+0000", 2024-09-16T00:10:44.772 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2285000000000001e-05 2024-09-16T00:10:44.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689665+0000", 2024-09-16T00:10:44.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7089999999999997e-06 2024-09-16T00:10:44.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689739+0000", 2024-09-16T00:10:44.773 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3973999999999993e-05 2024-09-16T00:10:44.773 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.773 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.773 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.773 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.689898+0000", 2024-09-16T00:10:44.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015941100000000001 2024-09-16T00:10:44.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:44.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.690062+0000", 2024-09-16T00:10:44.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000163633 2024-09-16T00:10:44.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.774 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.695224+0000", 2024-09-16T00:10:44.774 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051620210000000001 2024-09-16T00:10:44.774 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.774 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.774 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.775 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705434+0000", 2024-09-16T00:10:44.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010209701 2024-09-16T00:10:44.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.775 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.775 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705488+0000", 2024-09-16T00:10:44.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3974000000000002e-05 2024-09-16T00:10:44.775 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.775 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.775 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.775 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.705523+0000", 2024-09-16T00:10:44.775 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5746e-05 2024-09-16T00:10:44.775 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.776 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.776 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.776 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.776 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.776 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11941 3.12 3:4a0e6682:::1000000334b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:44.776 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:10:44.776 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 61.020026102000003, 2024-09-16T00:10:44.776 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01458448, 2024-09-16T00:10:44.776 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.776 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.776 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.777 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:44.777 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:44.777 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:44.777 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11941 2024-09-16T00:10:44.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.777 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:10:44.777 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.777 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.777 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.777 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.777 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723431+0000", 2024-09-16T00:10:44.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.723433+0000", 2024-09-16T00:10:44.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3410000000000001e-06 2024-09-16T00:10:44.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.778 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.778 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724966+0000", 2024-09-16T00:10:44.778 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001533242 2024-09-16T00:10:44.778 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.778 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724983+0000", 2024-09-16T00:10:44.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7156999999999999e-05 2024-09-16T00:10:44.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.724995+0000", 2024-09-16T00:10:44.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2089999999999999e-05 2024-09-16T00:10:44.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.779 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.779 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.779 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725062+0000", 2024-09-16T00:10:44.779 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6026999999999997e-05 2024-09-16T00:10:44.779 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725159+0000", 2024-09-16T00:10:44.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7374000000000007e-05 2024-09-16T00:10:44.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:44.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.725289+0000", 2024-09-16T00:10:44.780 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012988 2024-09-16T00:10:44.780 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.780 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.780 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.780 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.731944+0000", 2024-09-16T00:10:44.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066547439999999998 2024-09-16T00:10:44.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.737956+0000", 2024-09-16T00:10:44.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0060121630000000001 2024-09-16T00:10:44.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.781 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.781 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.737985+0000", 2024-09-16T00:10:44.781 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9352999999999998e-05 2024-09-16T00:10:44.781 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.781 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.782 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.782 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.738015+0000", 2024-09-16T00:10:44.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0108999999999999e-05 2024-09-16T00:10:44.782 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.782 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.782 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.782 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.782 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.782 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11954 3.6 3:625e0e90:::100000032ef.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:44.782 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:10:44.782 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.975574535, 2024-09-16T00:10:44.782 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012484808999999999, 2024-09-16T00:10:44.783 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.783 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.783 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.783 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:44.783 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:44.783 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:44.783 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11954 2024-09-16T00:10:44.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.783 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.783 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.783 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.783 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:10:44.783 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.783 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767882+0000", 2024-09-16T00:10:44.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.767886+0000", 2024-09-16T00:10:44.784 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6650000000000001e-06 2024-09-16T00:10:44.784 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.784 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.784 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.784 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770082+0000", 2024-09-16T00:10:44.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021964979999999999 2024-09-16T00:10:44.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770104+0000", 2024-09-16T00:10:44.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1671999999999999e-05 2024-09-16T00:10:44.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.785 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.785 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770114+0000", 2024-09-16T00:10:44.785 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0336000000000001e-05 2024-09-16T00:10:44.785 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.785 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.786 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.786 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770197+0000", 2024-09-16T00:10:44.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2997000000000005e-05 2024-09-16T00:10:44.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.786 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.786 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770324+0000", 2024-09-16T00:10:44.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012680100000000001 2024-09-16T00:10:44.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.786 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.786 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:44.786 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.770497+0000", 2024-09-16T00:10:44.786 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000172776 2024-09-16T00:10:44.786 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.776330+0000", 2024-09-16T00:10:44.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058330939999999996 2024-09-16T00:10:44.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780252+0000", 2024-09-16T00:10:44.787 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039215689999999997 2024-09-16T00:10:44.787 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.787 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.787 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.787 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780317+0000", 2024-09-16T00:10:44.788 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4948000000000002e-05 2024-09-16T00:10:44.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.788 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.788 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.788 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.780367+0000", 2024-09-16T00:10:44.788 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0453e-05 2024-09-16T00:10:44.788 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.788 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.788 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.788 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.788 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.788 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13618 3.1 3:80dd39f3:::1000000336c.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:44.788 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.108808+0000", 2024-09-16T00:10:44.789 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.634648705000004, 2024-09-16T00:10:44.789 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013244522, 2024-09-16T00:10:44.789 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.789 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.789 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.789 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:44.789 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:44.789 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:44.789 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13618 2024-09-16T00:10:44.789 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.789 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.789 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.789 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.108808+0000", 2024-09-16T00:10:44.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.108808+0000", 2024-09-16T00:10:44.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.790 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.790 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.108812+0000", 2024-09-16T00:10:44.790 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4640000000000002e-06 2024-09-16T00:10:44.790 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.790 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.110957+0000", 2024-09-16T00:10:44.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002145185 2024-09-16T00:10:44.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.110974+0000", 2024-09-16T00:10:44.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7108999999999998e-05 2024-09-16T00:10:44.791 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.791 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.791 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.791 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.110982+0000", 2024-09-16T00:10:44.791 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6330000000000002e-06 2024-09-16T00:10:44.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.111130+0000", 2024-09-16T00:10:44.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014738899999999999 2024-09-16T00:10:44.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.792 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.111299+0000", 2024-09-16T00:10:44.792 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016908400000000001 2024-09-16T00:10:44.792 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.792 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.792 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:44.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.111470+0000", 2024-09-16T00:10:44.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000170667 2024-09-16T00:10:44.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.117141+0000", 2024-09-16T00:10:44.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005670993 2024-09-16T00:10:44.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.793 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.793 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.121945+0000", 2024-09-16T00:10:44.793 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0048042370000000003 2024-09-16T00:10:44.793 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.793 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.121998+0000", 2024-09-16T00:10:44.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2802e-05 2024-09-16T00:10:44.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.794 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.794 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.122053+0000", 2024-09-16T00:10:44.794 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4959e-05 2024-09-16T00:10:44.794 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.794 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.794 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.794 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.794 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.794 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13687 3.32 3:4e600466:::1000000377a.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:44.795 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.340141+0000", 2024-09-16T00:10:44.795 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.403315782, 2024-09-16T00:10:44.795 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012781707999999999, 2024-09-16T00:10:44.795 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.795 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.795 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.795 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:44.795 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:44.795 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:44.795 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13687 2024-09-16T00:10:44.795 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.795 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.795 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.795 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.340141+0000", 2024-09-16T00:10:44.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.340141+0000", 2024-09-16T00:10:44.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.796 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.796 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.796 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.340143+0000", 2024-09-16T00:10:44.796 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9520000000000001e-06 2024-09-16T00:10:44.796 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343159+0000", 2024-09-16T00:10:44.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030156290000000001 2024-09-16T00:10:44.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343178+0000", 2024-09-16T00:10:44.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9219999999999999e-05 2024-09-16T00:10:44.797 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.797 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.797 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.797 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343212+0000", 2024-09-16T00:10:44.797 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3745999999999998e-05 2024-09-16T00:10:44.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343293+0000", 2024-09-16T00:10:44.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1074000000000003e-05 2024-09-16T00:10:44.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343430+0000", 2024-09-16T00:10:44.798 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013718700000000001 2024-09-16T00:10:44.798 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.798 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.798 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:44.798 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.343589+0000", 2024-09-16T00:10:44.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000158806 2024-09-16T00:10:44.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.352468+0000", 2024-09-16T00:10:44.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0088798209999999995 2024-09-16T00:10:44.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.799 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.352835+0000", 2024-09-16T00:10:44.799 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00036653400000000001 2024-09-16T00:10:44.799 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.799 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.799 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.352884+0000", 2024-09-16T00:10:44.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8881000000000002e-05 2024-09-16T00:10:44.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.800 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.800 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.352923+0000", 2024-09-16T00:10:44.800 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8858e-05 2024-09-16T00:10:44.800 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.800 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.800 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.800 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.800 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.800 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13722 3.6 3:639728a8:::100000030cc.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:44.800 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.442783+0000", 2024-09-16T00:10:44.801 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.300673938999999, 2024-09-16T00:10:44.801 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013776286, 2024-09-16T00:10:44.801 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.801 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.801 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.801 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:44.801 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:44.801 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:44.801 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13722 2024-09-16T00:10:44.801 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.801 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.801 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.801 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.801 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.442783+0000", 2024-09-16T00:10:44.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.442783+0000", 2024-09-16T00:10:44.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.802 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.442785+0000", 2024-09-16T00:10:44.802 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9369999999999998e-06 2024-09-16T00:10:44.802 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.802 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.802 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444337+0000", 2024-09-16T00:10:44.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015527169999999999 2024-09-16T00:10:44.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.804 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444362+0000", 2024-09-16T00:10:44.804 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4513999999999999e-05 2024-09-16T00:10:44.804 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.804 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.804 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444375+0000", 2024-09-16T00:10:44.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3052e-05 2024-09-16T00:10:44.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444451+0000", 2024-09-16T00:10:44.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5814000000000003e-05 2024-09-16T00:10:44.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.805 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.805 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444566+0000", 2024-09-16T00:10:44.805 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011508199999999999 2024-09-16T00:10:44.805 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.805 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:44.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.444727+0000", 2024-09-16T00:10:44.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000160743 2024-09-16T00:10:44.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.451504+0000", 2024-09-16T00:10:44.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.006777773 2024-09-16T00:10:44.806 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.806 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.806 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.806 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.456446+0000", 2024-09-16T00:10:44.806 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004941834 2024-09-16T00:10:44.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.807 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.456510+0000", 2024-09-16T00:10:44.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3880999999999994e-05 2024-09-16T00:10:44.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.807 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.807 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.807 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.456559+0000", 2024-09-16T00:10:44.807 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8939e-05 2024-09-16T00:10:44.807 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.807 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.807 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.807 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.808 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.808 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13773 3.10 3:0a21c9f4:::100000034e7.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:44.808 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.578694+0000", 2024-09-16T00:10:44.808 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 55.164763004999998, 2024-09-16T00:10:44.808 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012227249000000001, 2024-09-16T00:10:44.808 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.808 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.808 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.808 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:44.808 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:44.808 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:44.808 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13773 2024-09-16T00:10:44.808 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.808 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.578694+0000", 2024-09-16T00:10:44.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.578694+0000", 2024-09-16T00:10:44.809 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.809 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.809 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.809 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.809 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.578698+0000", 2024-09-16T00:10:44.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8700000000000002e-06 2024-09-16T00:10:44.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.581979+0000", 2024-09-16T00:10:44.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003281455 2024-09-16T00:10:44.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.810 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.581994+0000", 2024-09-16T00:10:44.810 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4861000000000001e-05 2024-09-16T00:10:44.810 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.810 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.810 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.582003+0000", 2024-09-16T00:10:44.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0399999999999998e-06 2024-09-16T00:10:44.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.582070+0000", 2024-09-16T00:10:44.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7189999999999999e-05 2024-09-16T00:10:44.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.811 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.811 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.582158+0000", 2024-09-16T00:10:44.811 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7843000000000006e-05 2024-09-16T00:10:44.811 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.811 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.590796+0000", 2024-09-16T00:10:44.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0086376820000000007 2024-09-16T00:10:44.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.590842+0000", 2024-09-16T00:10:44.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6779000000000001e-05 2024-09-16T00:10:44.812 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.812 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.812 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.812 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.590921+0000", 2024-09-16T00:10:44.812 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8529000000000007e-05 2024-09-16T00:10:44.812 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.813 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.813 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.813 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.813 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.813 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13876 3.17 3:e847251e:::10000003522.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:44.813 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.796002+0000", 2024-09-16T00:10:44.813 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 54.947454432000001, 2024-09-16T00:10:44.813 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013015385000000001, 2024-09-16T00:10:44.813 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.813 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.813 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.813 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:44.813 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:44.813 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:44.814 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13876 2024-09-16T00:10:44.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.814 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.796002+0000", 2024-09-16T00:10:44.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.814 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.814 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.814 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.796002+0000", 2024-09-16T00:10:44.814 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.814 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.796004+0000", 2024-09-16T00:10:44.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1090000000000001e-06 2024-09-16T00:10:44.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797545+0000", 2024-09-16T00:10:44.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015402219999999999 2024-09-16T00:10:44.815 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.815 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.815 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.815 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797570+0000", 2024-09-16T00:10:44.815 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4905999999999998e-05 2024-09-16T00:10:44.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797580+0000", 2024-09-16T00:10:44.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0886e-05 2024-09-16T00:10:44.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797652+0000", 2024-09-16T00:10:44.816 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1050000000000006e-05 2024-09-16T00:10:44.816 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.816 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.816 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.816 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797814+0000", 2024-09-16T00:10:44.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000162578 2024-09-16T00:10:44.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:44.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.797965+0000", 2024-09-16T00:10:44.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015092099999999999 2024-09-16T00:10:44.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.817 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.804256+0000", 2024-09-16T00:10:44.817 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0062911349999999998 2024-09-16T00:10:44.817 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.817 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.817 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.818 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.808943+0000", 2024-09-16T00:10:44.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0046868200000000004 2024-09-16T00:10:44.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.818 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.818 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.818 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.808985+0000", 2024-09-16T00:10:44.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1659000000000002e-05 2024-09-16T00:10:44.818 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.818 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.818 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.818 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.809018+0000", 2024-09-16T00:10:44.818 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3099000000000003e-05 2024-09-16T00:10:44.818 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.819 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.819 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.819 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.819 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.819 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13963 3.2b 3:d4ebc4d4:::100000034a5.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:44.819 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:50.005950+0000", 2024-09-16T00:10:44.819 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 54.737507198000003, 2024-09-16T00:10:44.819 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.062103750999999999, 2024-09-16T00:10:44.819 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.819 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.819 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.819 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:44.819 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:44.820 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:44.820 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13963 2024-09-16T00:10:44.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.820 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.005950+0000", 2024-09-16T00:10:44.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.820 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.820 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.820 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.005950+0000", 2024-09-16T00:10:44.820 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.820 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.005952+0000", 2024-09-16T00:10:44.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.255e-06 2024-09-16T00:10:44.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008027+0000", 2024-09-16T00:10:44.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002075019 2024-09-16T00:10:44.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.821 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.821 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.821 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008051+0000", 2024-09-16T00:10:44.821 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3906000000000001e-05 2024-09-16T00:10:44.821 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008063+0000", 2024-09-16T00:10:44.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2564e-05 2024-09-16T00:10:44.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008137+0000", 2024-09-16T00:10:44.822 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3946e-05 2024-09-16T00:10:44.822 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.822 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.822 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.822 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008234+0000", 2024-09-16T00:10:44.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6497999999999997e-05 2024-09-16T00:10:44.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:44.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.008377+0000", 2024-09-16T00:10:44.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014313300000000001 2024-09-16T00:10:44.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.067062+0000", 2024-09-16T00:10:44.823 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058685043999999999 2024-09-16T00:10:44.823 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.823 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.823 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.823 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.067963+0000", 2024-09-16T00:10:44.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000901413 2024-09-16T00:10:44.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.824 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.068013+0000", 2024-09-16T00:10:44.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9997000000000003e-05 2024-09-16T00:10:44.824 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.824 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.824 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.824 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.068053+0000", 2024-09-16T00:10:44.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9975999999999998e-05 2024-09-16T00:10:44.824 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.824 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.824 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.825 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.825 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(mds.0.4:2683 2.8 e89/19 2:17051619:::200.0000000f:head v 89'470, mlcod=89'468)", 2024-09-16T00:10:44.825 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:50.011143+0000", 2024-09-16T00:10:44.825 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 54.732313548999997, 2024-09-16T00:10:44.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.056951953, 2024-09-16T00:10:44.825 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.825 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.825 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.825 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.825 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.825 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.011143+0000", 2024-09-16T00:10:44.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.011143+0000", 2024-09-16T00:10:44.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.011145+0000", 2024-09-16T00:10:44.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2869999999999999e-06 2024-09-16T00:10:44.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.011161+0000", 2024-09-16T00:10:44.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6920999999999999e-05 2024-09-16T00:10:44.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063382+0000", 2024-09-16T00:10:44.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.052221072 2024-09-16T00:10:44.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063394+0000", 2024-09-16T00:10:44.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1212999999999999e-05 2024-09-16T00:10:44.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063454+0000", 2024-09-16T00:10:44.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9852999999999998e-05 2024-09-16T00:10:44.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063477+0000", 2024-09-16T00:10:44.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3638999999999998e-05 2024-09-16T00:10:44.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.068064+0000", 2024-09-16T00:10:44.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045870599999999996 2024-09-16T00:10:44.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.068095+0000", 2024-09-16T00:10:44.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0908000000000003e-05 2024-09-16T00:10:44.829 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.829 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.829 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.829 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13967 3.32 3:4d814898:::10000003587.00000001:head [write 0~4194304 [2@0]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:44.829 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:50.020209+0000", 2024-09-16T00:10:44.829 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 54.723247870999998, 2024-09-16T00:10:44.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.050981571000000003, 2024-09-16T00:10:44.829 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.830 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.830 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.830 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:44.830 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:44.830 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:44.830 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13967 2024-09-16T00:10:44.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.830 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.020209+0000", 2024-09-16T00:10:44.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.020209+0000", 2024-09-16T00:10:44.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.020212+0000", 2024-09-16T00:10:44.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.802e-06 2024-09-16T00:10:44.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.021676+0000", 2024-09-16T00:10:44.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014645909999999999 2024-09-16T00:10:44.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063378+0000", 2024-09-16T00:10:44.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.041701657000000003 2024-09-16T00:10:44.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063395+0000", 2024-09-16T00:10:44.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7469000000000002e-05 2024-09-16T00:10:44.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.832 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.832 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063457+0000", 2024-09-16T00:10:44.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1450000000000003e-05 2024-09-16T00:10:44.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063621+0000", 2024-09-16T00:10:44.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000164026 2024-09-16T00:10:44.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:44.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.063777+0000", 2024-09-16T00:10:44.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015628899999999999 2024-09-16T00:10:44.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.070303+0000", 2024-09-16T00:10:44.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0065262860000000001 2024-09-16T00:10:44.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.071105+0000", 2024-09-16T00:10:44.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000801423 2024-09-16T00:10:44.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.071154+0000", 2024-09-16T00:10:44.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8949000000000002e-05 2024-09-16T00:10:44.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:50.071190+0000", 2024-09-16T00:10:44.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6628999999999998e-05 2024-09-16T00:10:44.835 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.835 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.835 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.835 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:8762 3.6 3:60322b16:::10000001c24.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104)", 2024-09-16T00:10:44.835 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:12.945986+0000", 2024-09-16T00:10:44.835 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 31.797470526000001, 2024-09-16T00:10:44.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012381489000000001, 2024-09-16T00:10:44.835 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.835 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.836 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.836 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:12.945986+0000", 2024-09-16T00:10:44.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:12.945986+0000", 2024-09-16T00:10:44.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:12.945987+0000", 2024-09-16T00:10:44.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4099999999999998e-07 2024-09-16T00:10:44.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:12.945991+0000", 2024-09-16T00:10:44.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9160000000000003e-06 2024-09-16T00:10:44.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:12.945999+0000", 2024-09-16T00:10:44.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.622e-06 2024-09-16T00:10:44.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:12.946003+0000", 2024-09-16T00:10:44.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6920000000000001e-06 2024-09-16T00:10:44.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:12.946054+0000", 2024-09-16T00:10:44.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0927999999999998e-05 2024-09-16T00:10:44.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:12.946190+0000", 2024-09-16T00:10:44.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000135637 2024-09-16T00:10:44.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:44.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:12.946280+0000", 2024-09-16T00:10:44.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9833999999999994e-05 2024-09-16T00:10:44.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:12.947225+0000", 2024-09-16T00:10:44.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000945256 2024-09-16T00:10:44.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.839 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.839 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:12.958275+0000", 2024-09-16T00:10:44.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011050357 2024-09-16T00:10:44.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:12.958329+0000", 2024-09-16T00:10:44.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3820999999999999e-05 2024-09-16T00:10:44.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:12.958368+0000", 2024-09-16T00:10:44.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8685e-05 2024-09-16T00:10:44.840 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.840 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.840 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.841 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13095 3.2b 3:d704c0cc:::10000000d64.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:44.841 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:26.118623+0000", 2024-09-16T00:10:44.841 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.624833981999998, 2024-09-16T00:10:44.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.015368266, 2024-09-16T00:10:44.841 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.841 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.841 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.841 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.118623+0000", 2024-09-16T00:10:44.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.118623+0000", 2024-09-16T00:10:44.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.118624+0000", 2024-09-16T00:10:44.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6400000000000001e-07 2024-09-16T00:10:44.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.118627+0000", 2024-09-16T00:10:44.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8179999999999997e-06 2024-09-16T00:10:44.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.118638+0000", 2024-09-16T00:10:44.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.102e-05 2024-09-16T00:10:44.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.118645+0000", 2024-09-16T00:10:44.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9430000000000003e-06 2024-09-16T00:10:44.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.118688+0000", 2024-09-16T00:10:44.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.2864000000000002e-05 2024-09-16T00:10:44.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.118770+0000", 2024-09-16T00:10:44.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1749999999999995e-05 2024-09-16T00:10:44.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:44.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.118827+0000", 2024-09-16T00:10:44.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7278999999999998e-05 2024-09-16T00:10:44.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.119297+0000", 2024-09-16T00:10:44.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00046942400000000002 2024-09-16T00:10:44.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.133912+0000", 2024-09-16T00:10:44.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014615326999999999 2024-09-16T00:10:44.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.133957+0000", 2024-09-16T00:10:44.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4796999999999998e-05 2024-09-16T00:10:44.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.846 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.133991+0000", 2024-09-16T00:10:44.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4279999999999997e-05 2024-09-16T00:10:44.846 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.846 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.846 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.846 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13101 3.2b 3:d7b609b9:::10000000d6b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:44.846 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:26.120177+0000", 2024-09-16T00:10:44.846 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.623280073, 2024-09-16T00:10:44.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014162437999999999, 2024-09-16T00:10:44.846 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.847 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.847 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.847 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120177+0000", 2024-09-16T00:10:44.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120177+0000", 2024-09-16T00:10:44.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120177+0000", 2024-09-16T00:10:44.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4199999999999996e-07 2024-09-16T00:10:44.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120180+0000", 2024-09-16T00:10:44.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5469999999999999e-06 2024-09-16T00:10:44.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120187+0000", 2024-09-16T00:10:44.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5440000000000001e-06 2024-09-16T00:10:44.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120191+0000", 2024-09-16T00:10:44.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0969999999999999e-06 2024-09-16T00:10:44.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120230+0000", 2024-09-16T00:10:44.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8611e-05 2024-09-16T00:10:44.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120306+0000", 2024-09-16T00:10:44.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5656999999999997e-05 2024-09-16T00:10:44.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:44.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120362+0000", 2024-09-16T00:10:44.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6159000000000002e-05 2024-09-16T00:10:44.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.121240+0000", 2024-09-16T00:10:44.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00087779600000000002 2024-09-16T00:10:44.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.134259+0000", 2024-09-16T00:10:44.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013018905000000001 2024-09-16T00:10:44.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.134299+0000", 2024-09-16T00:10:44.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0880000000000002e-05 2024-09-16T00:10:44.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.134339+0000", 2024-09-16T00:10:44.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9700000000000003e-05 2024-09-16T00:10:44.851 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.851 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.851 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.852 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13105 3.12 3:49d39c28:::10000000d6f.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:44.852 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:26.120405+0000", 2024-09-16T00:10:44.852 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.623051762999999, 2024-09-16T00:10:44.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014056056000000001, 2024-09-16T00:10:44.852 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.852 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.852 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.852 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120405+0000", 2024-09-16T00:10:44.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120405+0000", 2024-09-16T00:10:44.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120406+0000", 2024-09-16T00:10:44.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1200000000000003e-07 2024-09-16T00:10:44.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.853 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.853 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120408+0000", 2024-09-16T00:10:44.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.447e-06 2024-09-16T00:10:44.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120413+0000", 2024-09-16T00:10:44.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0719999999999999e-06 2024-09-16T00:10:44.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120416+0000", 2024-09-16T00:10:44.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4000000000000001e-06 2024-09-16T00:10:44.854 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120453+0000", 2024-09-16T00:10:44.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6353999999999998e-05 2024-09-16T00:10:44.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120526+0000", 2024-09-16T00:10:44.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2948999999999996e-05 2024-09-16T00:10:44.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:44.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120614+0000", 2024-09-16T00:10:44.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8240999999999998e-05 2024-09-16T00:10:44.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.121256+0000", 2024-09-16T00:10:44.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00064181399999999999 2024-09-16T00:10:44.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.134376+0000", 2024-09-16T00:10:44.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013120386 2024-09-16T00:10:44.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.134424+0000", 2024-09-16T00:10:44.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8265e-05 2024-09-16T00:10:44.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.134461+0000", 2024-09-16T00:10:44.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6616000000000003e-05 2024-09-16T00:10:44.857 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.857 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.857 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.857 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13108 3.2b 3:d6938053:::10000000d72.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:44.857 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:26.120675+0000", 2024-09-16T00:10:44.857 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 18.622781548999999, 2024-09-16T00:10:44.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013817121, 2024-09-16T00:10:44.857 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:44.858 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:44.858 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:44.858 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:44.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:44.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120675+0000", 2024-09-16T00:10:44.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:44.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120675+0000", 2024-09-16T00:10:44.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:44.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:44.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120676+0000", 2024-09-16T00:10:44.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8800000000000001e-07 2024-09-16T00:10:44.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:44.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120681+0000", 2024-09-16T00:10:44.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7720000000000002e-06 2024-09-16T00:10:44.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:44.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120691+0000", 2024-09-16T00:10:44.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0118999999999999e-05 2024-09-16T00:10:44.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:44.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120698+0000", 2024-09-16T00:10:44.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7270000000000001e-06 2024-09-16T00:10:44.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:44.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120776+0000", 2024-09-16T00:10:44.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8317000000000002e-05 2024-09-16T00:10:44.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:44.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120879+0000", 2024-09-16T00:10:44.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000103254 2024-09-16T00:10:44.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:44.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.120972+0000", 2024-09-16T00:10:44.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2454000000000006e-05 2024-09-16T00:10:44.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:44.861 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.121394+0000", 2024-09-16T00:10:44.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00042253700000000001 2024-09-16T00:10:44.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.861 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:44.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.134427+0000", 2024-09-16T00:10:44.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013032225 2024-09-16T00:10:44.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:44.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.134470+0000", 2024-09-16T00:10:44.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3220999999999998e-05 2024-09-16T00:10:44.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:44.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:44.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:44.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.134492+0000", 2024-09-16T00:10:44.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2606999999999999e-05 2024-09-16T00:10:44.862 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.862 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.862 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.863 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:44.863 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:44.863 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:44.863 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-16T00:10:44.864 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:44.864+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:44.901 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:44.901+0000 7fcad0386640 1 -- 172.21.15.139:0/1507230524 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fcac80c8fb0 con 0x7fcac8151680 2024-09-16T00:10:44.966 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:44.965+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:44.970 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:44.970 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:44.970 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:44.970 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:44.979 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-16T00:10:45.012 DEBUG:teuthology.parallel:result is None 2024-09-16T00:10:45.012 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T00:10:45.066 INFO:tasks.ceph.osd.6.smithi164.stderr:2024-09-16T00:10:45.066+0000 7f5a0444f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 50779) UID: 0 2024-09-16T00:10:45.084 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T00:10:45.084 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-16T00:10:45.096 INFO:tasks.thrashosds:joining thrashosds 2024-09-16T00:10:45.156 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:45.156 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:45.156 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:45.156 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:45.156 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:45.166 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-16T00:10:45.318 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:45.318 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:45.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:45.318 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:45.318 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.318 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3161 2.d 2:b02ffd62:::200.00000010:head [write 3427859~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.318 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:45.318 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.653279855000001, 2024-09-16T00:10:45.318 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026748516, 2024-09-16T00:10:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.319 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662634+0000", 2024-09-16T00:10:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.15e-07 2024-09-16T00:10:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662637+0000", 2024-09-16T00:10:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8009999999999999e-06 2024-09-16T00:10:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.320 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662645+0000", 2024-09-16T00:10:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8299999999999996e-06 2024-09-16T00:10:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662650+0000", 2024-09-16T00:10:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.605e-06 2024-09-16T00:10:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.682553+0000", 2024-09-16T00:10:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01990331 2024-09-16T00:10:45.321 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.682636+0000", 2024-09-16T00:10:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2921000000000001e-05 2024-09-16T00:10:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.682780+0000", 2024-09-16T00:10:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000144361 2024-09-16T00:10:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.322 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.684809+0000", 2024-09-16T00:10:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002029005 2024-09-16T00:10:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689293+0000", 2024-09-16T00:10:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0044841079999999997 2024-09-16T00:10:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689339+0000", 2024-09-16T00:10:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5668999999999999e-05 2024-09-16T00:10:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.323 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689382+0000", 2024-09-16T00:10:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3491000000000001e-05 2024-09-16T00:10:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3162 2.d 2:b02ffd62:::200.00000010:head [write 3430104~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.662858+0000", 2024-09-16T00:10:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.653055739000003, 2024-09-16T00:10:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026630083999999998, 2024-09-16T00:10:45.324 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662858+0000", 2024-09-16T00:10:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662858+0000", 2024-09-16T00:10:45.325 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662859+0000", 2024-09-16T00:10:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:10:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662862+0000", 2024-09-16T00:10:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6119999999999999e-06 2024-09-16T00:10:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.326 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662870+0000", 2024-09-16T00:10:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2360000000000004e-06 2024-09-16T00:10:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.662874+0000", 2024-09-16T00:10:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2610000000000002e-06 2024-09-16T00:10:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.683241+0000", 2024-09-16T00:10:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.020367495999999999 2024-09-16T00:10:45.327 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.683295+0000", 2024-09-16T00:10:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4230000000000001e-05 2024-09-16T00:10:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.683380+0000", 2024-09-16T00:10:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4367e-05 2024-09-16T00:10:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.328 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.684845+0000", 2024-09-16T00:10:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014653190000000001 2024-09-16T00:10:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.329 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689392+0000", 2024-09-16T00:10:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045474280000000001 2024-09-16T00:10:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689445+0000", 2024-09-16T00:10:45.330 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2444000000000001e-05 2024-09-16T00:10:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689488+0000", 2024-09-16T00:10:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3124999999999998e-05 2024-09-16T00:10:45.331 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.332 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3166 2.d 2:b02ffd62:::200.00000010:head [write 3437392~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.333 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.664403+0000", 2024-09-16T00:10:45.333 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.651510860999998, 2024-09-16T00:10:45.333 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.027480674, 2024-09-16T00:10:45.333 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.333 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.333 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.334 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.334 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.334 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.334 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664403+0000", 2024-09-16T00:10:45.334 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664403+0000", 2024-09-16T00:10:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.335 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664404+0000", 2024-09-16T00:10:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4099999999999998e-07 2024-09-16T00:10:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664408+0000", 2024-09-16T00:10:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8169999999999998e-06 2024-09-16T00:10:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664418+0000", 2024-09-16T00:10:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6909999999999992e-06 2024-09-16T00:10:45.336 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664425+0000", 2024-09-16T00:10:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6309999999999997e-06 2024-09-16T00:10:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.685372+0000", 2024-09-16T00:10:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.020947548999999999 2024-09-16T00:10:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.337 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.685454+0000", 2024-09-16T00:10:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1355999999999998e-05 2024-09-16T00:10:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.685585+0000", 2024-09-16T00:10:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000130922 2024-09-16T00:10:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.338 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687329+0000", 2024-09-16T00:10:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017445710000000001 2024-09-16T00:10:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691757+0000", 2024-09-16T00:10:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004428268 2024-09-16T00:10:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691830+0000", 2024-09-16T00:10:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2237999999999995e-05 2024-09-16T00:10:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.339 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691884+0000", 2024-09-16T00:10:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3890000000000001e-05 2024-09-16T00:10:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3167 2.d 2:b02ffd62:::200.00000010:head [write 3439637~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.340 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.664627+0000", 2024-09-16T00:10:45.341 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.651286288999998, 2024-09-16T00:10:45.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.027347698, 2024-09-16T00:10:45.341 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.341 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.341 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.341 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.341 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.341 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664627+0000", 2024-09-16T00:10:45.341 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.341 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.341 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.342 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664627+0000", 2024-09-16T00:10:45.342 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.342 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.342 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.342 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.343 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664628+0000", 2024-09-16T00:10:45.343 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0500000000000003e-07 2024-09-16T00:10:45.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.343 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.343 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.344 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664632+0000", 2024-09-16T00:10:45.344 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8770000000000003e-06 2024-09-16T00:10:45.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.344 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.344 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.344 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664639+0000", 2024-09-16T00:10:45.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6590000000000001e-06 2024-09-16T00:10:45.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.345 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.345 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.345 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.664642+0000", 2024-09-16T00:10:45.345 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8550000000000004e-06 2024-09-16T00:10:45.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.686530+0000", 2024-09-16T00:10:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.021887127999999999 2024-09-16T00:10:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.346 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.686577+0000", 2024-09-16T00:10:45.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7181000000000001e-05 2024-09-16T00:10:45.347 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.347 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.347 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.347 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.686667+0000", 2024-09-16T00:10:45.347 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0074000000000005e-05 2024-09-16T00:10:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688714+0000", 2024-09-16T00:10:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020467749999999998 2024-09-16T00:10:45.348 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691894+0000", 2024-09-16T00:10:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0031806529999999999 2024-09-16T00:10:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.349 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691939+0000", 2024-09-16T00:10:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4579999999999997e-05 2024-09-16T00:10:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.350 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691975+0000", 2024-09-16T00:10:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6310999999999998e-05 2024-09-16T00:10:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3168 2.d 2:b02ffd62:::200.00000010:head [write 3441036~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.650820207999999, 2024-09-16T00:10:45.351 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026965759999999998, 2024-09-16T00:10:45.352 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.352 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.352 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.352 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.352 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.352 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.352 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:45.352 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:45.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.353 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665094+0000", 2024-09-16T00:10:45.353 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7999999999999996e-07 2024-09-16T00:10:45.353 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.353 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.353 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665098+0000", 2024-09-16T00:10:45.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3200000000000001e-06 2024-09-16T00:10:45.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.354 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665104+0000", 2024-09-16T00:10:45.354 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4809999999999997e-06 2024-09-16T00:10:45.354 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.354 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.354 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665112+0000", 2024-09-16T00:10:45.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.67e-06 2024-09-16T00:10:45.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687392+0000", 2024-09-16T00:10:45.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.022279699 2024-09-16T00:10:45.355 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.355 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.355 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.355 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687461+0000", 2024-09-16T00:10:45.355 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8476e-05 2024-09-16T00:10:45.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687577+0000", 2024-09-16T00:10:45.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011661 2024-09-16T00:10:45.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.356 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688774+0000", 2024-09-16T00:10:45.356 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011962660000000001 2024-09-16T00:10:45.356 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.356 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.356 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691984+0000", 2024-09-16T00:10:45.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032106320000000002 2024-09-16T00:10:45.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692028+0000", 2024-09-16T00:10:45.357 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3866999999999999e-05 2024-09-16T00:10:45.357 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.357 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.357 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.357 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692059+0000", 2024-09-16T00:10:45.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1259e-05 2024-09-16T00:10:45.358 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.358 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.358 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.358 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.358 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.358 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3169 2.d 2:b02ffd62:::200.00000010:head [write 3443281~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.358 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:45.358 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.650363671999997, 2024-09-16T00:10:45.358 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.030090967999999999, 2024-09-16T00:10:45.359 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.359 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.359 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.359 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.359 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.359 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:45.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:45.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.360 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.360 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.360 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.360 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665550+0000", 2024-09-16T00:10:45.360 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3799999999999998e-07 2024-09-16T00:10:45.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.361 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.361 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.361 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665554+0000", 2024-09-16T00:10:45.361 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.112e-06 2024-09-16T00:10:45.361 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665559+0000", 2024-09-16T00:10:45.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9559999999999996e-06 2024-09-16T00:10:45.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665562+0000", 2024-09-16T00:10:45.362 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2909999999999999e-06 2024-09-16T00:10:45.362 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.362 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.362 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.362 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.687954+0000", 2024-09-16T00:10:45.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.022391714 2024-09-16T00:10:45.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688016+0000", 2024-09-16T00:10:45.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2384000000000004e-05 2024-09-16T00:10:45.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.363 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688147+0000", 2024-09-16T00:10:45.363 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013099300000000001 2024-09-16T00:10:45.363 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.363 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.363 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.364 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690321+0000", 2024-09-16T00:10:45.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0021744339999999998 2024-09-16T00:10:45.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.364 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.364 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695548+0000", 2024-09-16T00:10:45.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0052266079999999998 2024-09-16T00:10:45.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.364 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.364 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695595+0000", 2024-09-16T00:10:45.364 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7200000000000002e-05 2024-09-16T00:10:45.364 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.364 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.365 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.365 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695641+0000", 2024-09-16T00:10:45.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5837999999999997e-05 2024-09-16T00:10:45.365 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.365 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.365 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.365 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.365 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.365 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3170 2.d 2:b02ffd62:::200.00000010:head [write 3445526~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.365 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:45.365 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.650170611, 2024-09-16T00:10:45.365 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029987413000000001, 2024-09-16T00:10:45.365 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.366 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.366 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.366 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:45.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.366 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.366 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:45.366 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.366 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.366 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665743+0000", 2024-09-16T00:10:45.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7500000000000001e-07 2024-09-16T00:10:45.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665753+0000", 2024-09-16T00:10:45.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0005e-05 2024-09-16T00:10:45.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.367 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.367 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.367 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665759+0000", 2024-09-16T00:10:45.367 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.169e-06 2024-09-16T00:10:45.367 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665762+0000", 2024-09-16T00:10:45.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8460000000000001e-06 2024-09-16T00:10:45.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688811+0000", 2024-09-16T00:10:45.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023048169 2024-09-16T00:10:45.368 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.368 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.368 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.368 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688885+0000", 2024-09-16T00:10:45.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4788000000000003e-05 2024-09-16T00:10:45.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.688989+0000", 2024-09-16T00:10:45.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000103689 2024-09-16T00:10:45.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690350+0000", 2024-09-16T00:10:45.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013609329999999999 2024-09-16T00:10:45.369 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695652+0000", 2024-09-16T00:10:45.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053019080000000001 2024-09-16T00:10:45.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695698+0000", 2024-09-16T00:10:45.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5775000000000001e-05 2024-09-16T00:10:45.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695731+0000", 2024-09-16T00:10:45.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2756000000000003e-05 2024-09-16T00:10:45.371 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.371 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.371 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.371 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3171 2.d 2:b02ffd62:::200.00000010:head [write 3446925~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.371 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.665997+0000", 2024-09-16T00:10:45.371 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.649916519999998, 2024-09-16T00:10:45.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029824032, 2024-09-16T00:10:45.371 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.371 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.371 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.371 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665997+0000", 2024-09-16T00:10:45.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665997+0000", 2024-09-16T00:10:45.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.665998+0000", 2024-09-16T00:10:45.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3999999999999997e-07 2024-09-16T00:10:45.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666000+0000", 2024-09-16T00:10:45.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6290000000000002e-06 2024-09-16T00:10:45.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666005+0000", 2024-09-16T00:10:45.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.437e-06 2024-09-16T00:10:45.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666008+0000", 2024-09-16T00:10:45.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7910000000000002e-06 2024-09-16T00:10:45.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689799+0000", 2024-09-16T00:10:45.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023790504000000001 2024-09-16T00:10:45.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689907+0000", 2024-09-16T00:10:45.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000108464 2024-09-16T00:10:45.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.689994+0000", 2024-09-16T00:10:45.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7043e-05 2024-09-16T00:10:45.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691737+0000", 2024-09-16T00:10:45.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001742527 2024-09-16T00:10:45.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.375 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.375 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695739+0000", 2024-09-16T00:10:45.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0040020189999999999 2024-09-16T00:10:45.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695786+0000", 2024-09-16T00:10:45.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7219999999999999e-05 2024-09-16T00:10:45.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695821+0000", 2024-09-16T00:10:45.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5058000000000002e-05 2024-09-16T00:10:45.376 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.376 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.377 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.377 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3172 2.d 2:b02ffd62:::200.00000010:head [write 3448324~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.377 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.666428+0000", 2024-09-16T00:10:45.377 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.649485570000003, 2024-09-16T00:10:45.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029477316, 2024-09-16T00:10:45.377 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.377 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.377 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.377 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666428+0000", 2024-09-16T00:10:45.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666428+0000", 2024-09-16T00:10:45.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666429+0000", 2024-09-16T00:10:45.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3699999999999999e-07 2024-09-16T00:10:45.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666433+0000", 2024-09-16T00:10:45.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4939999999999997e-06 2024-09-16T00:10:45.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666443+0000", 2024-09-16T00:10:45.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0899999999999994e-06 2024-09-16T00:10:45.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666447+0000", 2024-09-16T00:10:45.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8930000000000004e-06 2024-09-16T00:10:45.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690390+0000", 2024-09-16T00:10:45.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023942215999999999 2024-09-16T00:10:45.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690464+0000", 2024-09-16T00:10:45.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4678999999999994e-05 2024-09-16T00:10:45.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.690576+0000", 2024-09-16T00:10:45.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011162000000000001 2024-09-16T00:10:45.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.381 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.381 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692126+0000", 2024-09-16T00:10:45.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015496520000000001 2024-09-16T00:10:45.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695828+0000", 2024-09-16T00:10:45.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037028510000000001 2024-09-16T00:10:45.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695870+0000", 2024-09-16T00:10:45.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1180999999999998e-05 2024-09-16T00:10:45.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695905+0000", 2024-09-16T00:10:45.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5803000000000003e-05 2024-09-16T00:10:45.383 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.383 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.383 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.383 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3173 2.d 2:b02ffd62:::200.00000010:head [write 3449723~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.383 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.666884+0000", 2024-09-16T00:10:45.383 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.649029614, 2024-09-16T00:10:45.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.02908875, 2024-09-16T00:10:45.383 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.383 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.384 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.384 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666884+0000", 2024-09-16T00:10:45.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666884+0000", 2024-09-16T00:10:45.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666885+0000", 2024-09-16T00:10:45.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0999999999999998e-07 2024-09-16T00:10:45.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666889+0000", 2024-09-16T00:10:45.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4299999999999999e-06 2024-09-16T00:10:45.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666895+0000", 2024-09-16T00:10:45.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0510000000000002e-06 2024-09-16T00:10:45.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.666899+0000", 2024-09-16T00:10:45.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0489999999999997e-06 2024-09-16T00:10:45.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691161+0000", 2024-09-16T00:10:45.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.024262156999999999 2024-09-16T00:10:45.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691253+0000", 2024-09-16T00:10:45.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1278000000000003e-05 2024-09-16T00:10:45.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.691348+0000", 2024-09-16T00:10:45.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.4777000000000004e-05 2024-09-16T00:10:45.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.387 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692211+0000", 2024-09-16T00:10:45.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00086351699999999995 2024-09-16T00:10:45.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695913+0000", 2024-09-16T00:10:45.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0037019840000000002 2024-09-16T00:10:45.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695952+0000", 2024-09-16T00:10:45.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8535999999999998e-05 2024-09-16T00:10:45.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695973+0000", 2024-09-16T00:10:45.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1260999999999999e-05 2024-09-16T00:10:45.389 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.389 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.389 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.389 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3174 2.d 2:b02ffd62:::200.00000010:head [write 3451968~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.389 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.667323+0000", 2024-09-16T00:10:45.389 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.648590744000003, 2024-09-16T00:10:45.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028788859999999999, 2024-09-16T00:10:45.390 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.390 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.390 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.390 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667323+0000", 2024-09-16T00:10:45.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667323+0000", 2024-09-16T00:10:45.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667324+0000", 2024-09-16T00:10:45.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0400000000000005e-07 2024-09-16T00:10:45.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667330+0000", 2024-09-16T00:10:45.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3300000000000004e-06 2024-09-16T00:10:45.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667338+0000", 2024-09-16T00:10:45.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0059999999999992e-06 2024-09-16T00:10:45.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.392 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667344+0000", 2024-09-16T00:10:45.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6409999999999999e-06 2024-09-16T00:10:45.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692253+0000", 2024-09-16T00:10:45.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.024909280999999998 2024-09-16T00:10:45.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692311+0000", 2024-09-16T00:10:45.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7873e-05 2024-09-16T00:10:45.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692395+0000", 2024-09-16T00:10:45.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4071000000000002e-05 2024-09-16T00:10:45.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695977+0000", 2024-09-16T00:10:45.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0035823460000000001 2024-09-16T00:10:45.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696034+0000", 2024-09-16T00:10:45.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6697999999999998e-05 2024-09-16T00:10:45.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696078+0000", 2024-09-16T00:10:45.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3696000000000003e-05 2024-09-16T00:10:45.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696112+0000", 2024-09-16T00:10:45.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4113999999999999e-05 2024-09-16T00:10:45.395 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.395 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.395 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.396 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3175 2.d 2:b02ffd62:::200.00000010:head [write 3454213~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.396 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:45.396 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.648249905999997, 2024-09-16T00:10:45.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.030153698999999999, 2024-09-16T00:10:45.396 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.396 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.396 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.396 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:45.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:45.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667664+0000", 2024-09-16T00:10:45.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2e-07 2024-09-16T00:10:45.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667668+0000", 2024-09-16T00:10:45.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2109999999999998e-06 2024-09-16T00:10:45.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667673+0000", 2024-09-16T00:10:45.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8409999999999996e-06 2024-09-16T00:10:45.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.667686+0000", 2024-09-16T00:10:45.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2928000000000001e-05 2024-09-16T00:10:45.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.398 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.398 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.398 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.692964+0000", 2024-09-16T00:10:45.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.025277642999999999 2024-09-16T00:10:45.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.694418+0000", 2024-09-16T00:10:45.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001453693 2024-09-16T00:10:45.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.694505+0000", 2024-09-16T00:10:45.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7700000000000004e-05 2024-09-16T00:10:45.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697114+0000", 2024-09-16T00:10:45.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026083460000000001 2024-09-16T00:10:45.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697697+0000", 2024-09-16T00:10:45.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00058297399999999999 2024-09-16T00:10:45.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697772+0000", 2024-09-16T00:10:45.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5433999999999996e-05 2024-09-16T00:10:45.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697818+0000", 2024-09-16T00:10:45.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.5408999999999998e-05 2024-09-16T00:10:45.402 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.402 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.402 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.403 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3176 2.d 2:b02ffd62:::200.00000010:head [write 3456458~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.403 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.668006+0000", 2024-09-16T00:10:45.403 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.647908000999998, 2024-09-16T00:10:45.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029935413000000001, 2024-09-16T00:10:45.403 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.403 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.404 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.404 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.404 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.404 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668006+0000", 2024-09-16T00:10:45.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668006+0000", 2024-09-16T00:10:45.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668007+0000", 2024-09-16T00:10:45.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5799999999999998e-07 2024-09-16T00:10:45.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668012+0000", 2024-09-16T00:10:45.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9320000000000004e-06 2024-09-16T00:10:45.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668023+0000", 2024-09-16T00:10:45.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1273e-05 2024-09-16T00:10:45.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.668030+0000", 2024-09-16T00:10:45.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0509999999999999e-06 2024-09-16T00:10:45.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.694928+0000", 2024-09-16T00:10:45.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026898114000000001 2024-09-16T00:10:45.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695003+0000", 2024-09-16T00:10:45.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4880000000000001e-05 2024-09-16T00:10:45.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.695116+0000", 2024-09-16T00:10:45.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011307900000000001 2024-09-16T00:10:45.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697119+0000", 2024-09-16T00:10:45.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0020032159999999999 2024-09-16T00:10:45.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.410 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.410 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697867+0000", 2024-09-16T00:10:45.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00074744999999999998 2024-09-16T00:10:45.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697908+0000", 2024-09-16T00:10:45.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1287e-05 2024-09-16T00:10:45.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697941+0000", 2024-09-16T00:10:45.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3272999999999998e-05 2024-09-16T00:10:45.411 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.411 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.411 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.412 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3177 2.d 2:b02ffd62:::200.00000010:head [write 3457857~1398 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.412 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:45.412 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.646573142000001, 2024-09-16T00:10:45.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.029906856999999998, 2024-09-16T00:10:45.412 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.412 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.412 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.412 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:45.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:45.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669341+0000", 2024-09-16T00:10:45.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1299999999999998e-07 2024-09-16T00:10:45.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669347+0000", 2024-09-16T00:10:45.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.327e-06 2024-09-16T00:10:45.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669359+0000", 2024-09-16T00:10:45.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1698e-05 2024-09-16T00:10:45.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.669366+0000", 2024-09-16T00:10:45.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7959999999999996e-06 2024-09-16T00:10:45.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696168+0000", 2024-09-16T00:10:45.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026802093999999999 2024-09-16T00:10:45.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696258+0000", 2024-09-16T00:10:45.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0074000000000005e-05 2024-09-16T00:10:45.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.696383+0000", 2024-09-16T00:10:45.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012455699999999999 2024-09-16T00:10:45.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698602+0000", 2024-09-16T00:10:45.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002218921 2024-09-16T00:10:45.416 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699152+0000", 2024-09-16T00:10:45.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00055024399999999997 2024-09-16T00:10:45.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699206+0000", 2024-09-16T00:10:45.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3882999999999999e-05 2024-09-16T00:10:45.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699247+0000", 2024-09-16T00:10:45.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.1350000000000002e-05 2024-09-16T00:10:45.418 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.419 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.419 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.419 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3178 2.d 2:b02ffd62:::200.00000010:head [write 3459255~2511 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.420 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.671779+0000", 2024-09-16T00:10:45.420 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.644134262000001, 2024-09-16T00:10:45.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028813986999999999, 2024-09-16T00:10:45.420 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.420 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.420 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.421 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671779+0000", 2024-09-16T00:10:45.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671779+0000", 2024-09-16T00:10:45.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671780+0000", 2024-09-16T00:10:45.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3600000000000002e-07 2024-09-16T00:10:45.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671785+0000", 2024-09-16T00:10:45.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.074e-06 2024-09-16T00:10:45.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671795+0000", 2024-09-16T00:10:45.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2040000000000003e-06 2024-09-16T00:10:45.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.671800+0000", 2024-09-16T00:10:45.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9420000000000003e-06 2024-09-16T00:10:45.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697159+0000", 2024-09-16T00:10:45.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.025359619999999999 2024-09-16T00:10:45.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697214+0000", 2024-09-16T00:10:45.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5318000000000001e-05 2024-09-16T00:10:45.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697297+0000", 2024-09-16T00:10:45.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2087000000000002e-05 2024-09-16T00:10:45.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699258+0000", 2024-09-16T00:10:45.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019619529999999998 2024-09-16T00:10:45.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700499+0000", 2024-09-16T00:10:45.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001240072 2024-09-16T00:10:45.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700555+0000", 2024-09-16T00:10:45.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6459999999999998e-05 2024-09-16T00:10:45.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700593+0000", 2024-09-16T00:10:45.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8420999999999997e-05 2024-09-16T00:10:45.426 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.426 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.426 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.426 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3179 2.d 2:b02ffd62:::200.00000010:head [write 3461766~2418 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.426 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.672548+0000", 2024-09-16T00:10:45.427 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.643366165000003, 2024-09-16T00:10:45.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028160178000000001, 2024-09-16T00:10:45.427 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.427 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.427 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.427 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672548+0000", 2024-09-16T00:10:45.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672548+0000", 2024-09-16T00:10:45.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672549+0000", 2024-09-16T00:10:45.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5300000000000002e-07 2024-09-16T00:10:45.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672562+0000", 2024-09-16T00:10:45.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3808000000000001e-05 2024-09-16T00:10:45.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672575+0000", 2024-09-16T00:10:45.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2870999999999999e-05 2024-09-16T00:10:45.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.672587+0000", 2024-09-16T00:10:45.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.203e-05 2024-09-16T00:10:45.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.697994+0000", 2024-09-16T00:10:45.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.025406373999999999 2024-09-16T00:10:45.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698074+0000", 2024-09-16T00:10:45.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0802000000000003e-05 2024-09-16T00:10:45.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698196+0000", 2024-09-16T00:10:45.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012154699999999999 2024-09-16T00:10:45.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700456+0000", 2024-09-16T00:10:45.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022602249999999998 2024-09-16T00:10:45.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700632+0000", 2024-09-16T00:10:45.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017570699999999999 2024-09-16T00:10:45.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700676+0000", 2024-09-16T00:10:45.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3764000000000003e-05 2024-09-16T00:10:45.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700708+0000", 2024-09-16T00:10:45.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2097000000000001e-05 2024-09-16T00:10:45.434 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.434 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.434 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.434 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3180 2.d 2:b02ffd62:::200.00000010:head [write 3464184~1398 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.434 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:45.434 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.641858214000003, 2024-09-16T00:10:45.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.028112109, 2024-09-16T00:10:45.435 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.435 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.435 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.435 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:45.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:45.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674056+0000", 2024-09-16T00:10:45.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:10:45.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.437 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.437 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674061+0000", 2024-09-16T00:10:45.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4440000000000001e-06 2024-09-16T00:10:45.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674069+0000", 2024-09-16T00:10:45.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7639999999999994e-06 2024-09-16T00:10:45.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.674076+0000", 2024-09-16T00:10:45.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5590000000000002e-06 2024-09-16T00:10:45.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698639+0000", 2024-09-16T00:10:45.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.024562471999999998 2024-09-16T00:10:45.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698693+0000", 2024-09-16T00:10:45.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4432999999999999e-05 2024-09-16T00:10:45.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.698807+0000", 2024-09-16T00:10:45.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000113607 2024-09-16T00:10:45.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700727+0000", 2024-09-16T00:10:45.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019201750000000001 2024-09-16T00:10:45.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702076+0000", 2024-09-16T00:10:45.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001349423 2024-09-16T00:10:45.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702128+0000", 2024-09-16T00:10:45.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2157000000000002e-05 2024-09-16T00:10:45.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702168+0000", 2024-09-16T00:10:45.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9292000000000003e-05 2024-09-16T00:10:45.441 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.441 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.441 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.441 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3181 2.d 2:b02ffd62:::200.00000010:head [write 3465582~2754 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.442 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.675904+0000", 2024-09-16T00:10:45.442 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.640009902999999, 2024-09-16T00:10:45.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026402452, 2024-09-16T00:10:45.442 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.442 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.442 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.442 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675904+0000", 2024-09-16T00:10:45.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675904+0000", 2024-09-16T00:10:45.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.443 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.443 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675905+0000", 2024-09-16T00:10:45.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.043e-06 2024-09-16T00:10:45.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675911+0000", 2024-09-16T00:10:45.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1970000000000001e-06 2024-09-16T00:10:45.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675924+0000", 2024-09-16T00:10:45.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.252e-05 2024-09-16T00:10:45.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.675931+0000", 2024-09-16T00:10:45.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5800000000000003e-06 2024-09-16T00:10:45.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699280+0000", 2024-09-16T00:10:45.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.023348634 2024-09-16T00:10:45.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699329+0000", 2024-09-16T00:10:45.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9175999999999998e-05 2024-09-16T00:10:45.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.699400+0000", 2024-09-16T00:10:45.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0649999999999996e-05 2024-09-16T00:10:45.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.700734+0000", 2024-09-16T00:10:45.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001334816 2024-09-16T00:10:45.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702219+0000", 2024-09-16T00:10:45.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014847930000000001 2024-09-16T00:10:45.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702267+0000", 2024-09-16T00:10:45.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8133000000000001e-05 2024-09-16T00:10:45.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702306+0000", 2024-09-16T00:10:45.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8909999999999998e-05 2024-09-16T00:10:45.448 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.448 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.448 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.449 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3184 2.d 2:b02ffd62:::200.00000010:head [write 3478077~11949 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.449 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.681430+0000", 2024-09-16T00:10:45.449 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.634483660999997, 2024-09-16T00:10:45.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.02662339, 2024-09-16T00:10:45.449 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.449 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.450 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.450 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681430+0000", 2024-09-16T00:10:45.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681430+0000", 2024-09-16T00:10:45.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681431+0000", 2024-09-16T00:10:45.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9999999999999995e-07 2024-09-16T00:10:45.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681443+0000", 2024-09-16T00:10:45.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1568e-05 2024-09-16T00:10:45.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681453+0000", 2024-09-16T00:10:45.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8819999999999996e-06 2024-09-16T00:10:45.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681458+0000", 2024-09-16T00:10:45.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3709999999999999e-06 2024-09-16T00:10:45.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.455 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.455 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.701384+0000", 2024-09-16T00:10:45.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.019926342999999999 2024-09-16T00:10:45.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.701458+0000", 2024-09-16T00:10:45.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3854000000000001e-05 2024-09-16T00:10:45.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.701570+0000", 2024-09-16T00:10:45.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000111437 2024-09-16T00:10:45.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702913+0000", 2024-09-16T00:10:45.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013437259999999999 2024-09-16T00:10:45.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.707984+0000", 2024-09-16T00:10:45.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0050705669999999998 2024-09-16T00:10:45.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708016+0000", 2024-09-16T00:10:45.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2527999999999998e-05 2024-09-16T00:10:45.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.461 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.461 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708053+0000", 2024-09-16T00:10:45.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.7113999999999997e-05 2024-09-16T00:10:45.461 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.461 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.461 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.462 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:3185 2.d 2:b02ffd62:::200.00000010:head [write 3490026~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e90)", 2024-09-16T00:10:45.462 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.681685+0000", 2024-09-16T00:10:45.462 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.634228389999997, 2024-09-16T00:10:45.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.026552664, 2024-09-16T00:10:45.463 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.463 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.463 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.463 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681685+0000", 2024-09-16T00:10:45.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681685+0000", 2024-09-16T00:10:45.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681686+0000", 2024-09-16T00:10:45.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.06e-07 2024-09-16T00:10:45.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681692+0000", 2024-09-16T00:10:45.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6840000000000001e-06 2024-09-16T00:10:45.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681697+0000", 2024-09-16T00:10:45.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0340000000000004e-06 2024-09-16T00:10:45.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.467 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.681700+0000", 2024-09-16T00:10:45.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3589999999999999e-06 2024-09-16T00:10:45.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702500+0000", 2024-09-16T00:10:45.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.020799746000000001 2024-09-16T00:10:45.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702550+0000", 2024-09-16T00:10:45.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9895999999999998e-05 2024-09-16T00:10:45.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:45.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.702649+0000", 2024-09-16T00:10:45.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9500999999999995e-05 2024-09-16T00:10:45.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.705343+0000", 2024-09-16T00:10:45.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026938130000000002 2024-09-16T00:10:45.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708128+0000", 2024-09-16T00:10:45.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0027852979999999999 2024-09-16T00:10:45.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708202+0000", 2024-09-16T00:10:45.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4046000000000002e-05 2024-09-16T00:10:45.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.708238+0000", 2024-09-16T00:10:45.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5781000000000002e-05 2024-09-16T00:10:45.470 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.470 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.470 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.471 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.471 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.471 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:45.472 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-16T00:10:45.577 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:45.577 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:45.577 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.577 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:20154 3.3a 3.c3374c3a (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e122)", 2024-09-16T00:10:45.577 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:45.216686+0000", 2024-09-16T00:10:45.578 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.36039093100000003, 2024-09-16T00:10:45.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.360412118, 2024-09-16T00:10:45.578 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.578 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.578 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "waiting for peered", 2024-09-16T00:10:45.578 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.578 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.578 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.578 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.216686+0000", 2024-09-16T00:10:45.578 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.578 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.216686+0000", 2024-09-16T00:10:45.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.579 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.216687+0000", 2024-09-16T00:10:45.579 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.4300000000000002e-07 2024-09-16T00:10:45.579 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.579 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.579 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.216691+0000", 2024-09-16T00:10:45.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.3289999999999998e-06 2024-09-16T00:10:45.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.580 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.580 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.580 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.216699+0000", 2024-09-16T00:10:45.580 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7809999999999994e-06 2024-09-16T00:10:45.580 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.581 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.581 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.216719+0000", 2024-09-16T00:10:45.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9352e-05 2024-09-16T00:10:45.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.581 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.581 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.581 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.216787+0000", 2024-09-16T00:10:45.581 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8214999999999997e-05 2024-09-16T00:10:45.581 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.582 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:45.582 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.216797+0000", 2024-09-16T00:10:45.582 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0311e-05 2024-09-16T00:10:45.582 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.582 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.582 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.582 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.582 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.582 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:20054 3.3a 3.e5e888ba (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e122)", 2024-09-16T00:10:45.582 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:45.197975+0000", 2024-09-16T00:10:45.583 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.379102041, 2024-09-16T00:10:45.583 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.37915950999999998, 2024-09-16T00:10:45.583 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.583 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.583 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "waiting for peered", 2024-09-16T00:10:45.583 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.583 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.583 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.583 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.197975+0000", 2024-09-16T00:10:45.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.584 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.584 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.197975+0000", 2024-09-16T00:10:45.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.584 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.584 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.197976+0000", 2024-09-16T00:10:45.584 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5600000000000004e-07 2024-09-16T00:10:45.584 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.584 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.584 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.197981+0000", 2024-09-16T00:10:45.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4630000000000003e-06 2024-09-16T00:10:45.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.197990+0000", 2024-09-16T00:10:45.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2280000000000004e-06 2024-09-16T00:10:45.585 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.585 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.585 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.585 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.197998+0000", 2024-09-16T00:10:45.585 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8699999999999992e-06 2024-09-16T00:10:45.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.198056+0000", 2024-09-16T00:10:45.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8081000000000002e-05 2024-09-16T00:10:45.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.586 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.586 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:45.586 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.198066+0000", 2024-09-16T00:10:45.586 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0159999999999999e-05 2024-09-16T00:10:45.586 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.586 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.586 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.586 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.587 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:19921 3.3a 3.4a6658fa (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e122)", 2024-09-16T00:10:45.587 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:45.176181+0000", 2024-09-16T00:10:45.587 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.40089676499999999, 2024-09-16T00:10:45.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.400985588, 2024-09-16T00:10:45.587 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.587 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.587 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "waiting for peered", 2024-09-16T00:10:45.587 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.587 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.587 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.587 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.176181+0000", 2024-09-16T00:10:45.587 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.176181+0000", 2024-09-16T00:10:45.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.588 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.176181+0000", 2024-09-16T00:10:45.588 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.1100000000000005e-07 2024-09-16T00:10:45.588 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.588 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.588 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.176186+0000", 2024-09-16T00:10:45.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4859999999999999e-06 2024-09-16T00:10:45.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.176196+0000", 2024-09-16T00:10:45.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0393e-05 2024-09-16T00:10:45.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.589 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.589 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.176205+0000", 2024-09-16T00:10:45.589 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3809999999999999e-06 2024-09-16T00:10:45.589 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.589 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.590 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.176265+0000", 2024-09-16T00:10:45.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0114000000000001e-05 2024-09-16T00:10:45.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.590 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:45.590 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.176275+0000", 2024-09-16T00:10:45.590 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0111e-05 2024-09-16T00:10:45.590 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.590 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.590 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.590 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.590 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.590 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:20105 3.3a 3.10507fa (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e122)", 2024-09-16T00:10:45.590 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:45.206850+0000", 2024-09-16T00:10:45.591 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.370227478, 2024-09-16T00:10:45.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.37034698700000002, 2024-09-16T00:10:45.591 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.591 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.591 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "waiting for peered", 2024-09-16T00:10:45.591 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.206850+0000", 2024-09-16T00:10:45.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.591 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.591 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.591 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.206850+0000", 2024-09-16T00:10:45.591 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.592 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.592 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.206850+0000", 2024-09-16T00:10:45.592 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4700000000000001e-07 2024-09-16T00:10:45.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.592 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.592 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.206853+0000", 2024-09-16T00:10:45.592 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.881e-06 2024-09-16T00:10:45.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.592 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.592 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.593 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.206861+0000", 2024-09-16T00:10:45.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7360000000000001e-06 2024-09-16T00:10:45.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.593 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.593 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.593 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.206866+0000", 2024-09-16T00:10:45.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.4750000000000001e-06 2024-09-16T00:10:45.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.593 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.593 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.593 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.206912+0000", 2024-09-16T00:10:45.593 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.545e-05 2024-09-16T00:10:45.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.593 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.594 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:45.594 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.206920+0000", 2024-09-16T00:10:45.594 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.303e-06 2024-09-16T00:10:45.594 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.594 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.594 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.594 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.594 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.594 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:19994 3.3a 3.f040b17a (undecoded) ondisk+write+known_if_redirected+full_force+supports_pool_eio e122)", 2024-09-16T00:10:45.594 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:45.188718+0000", 2024-09-16T00:10:45.594 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 0.38835922099999998, 2024-09-16T00:10:45.594 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.38851344900000001, 2024-09-16T00:10:45.594 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.595 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.595 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "waiting for peered", 2024-09-16T00:10:45.595 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.595 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.595 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.595 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.188718+0000", 2024-09-16T00:10:45.595 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.595 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.595 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.595 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.188718+0000", 2024-09-16T00:10:45.595 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.595 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.595 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.596 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.596 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.188719+0000", 2024-09-16T00:10:45.596 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0219999999999999e-06 2024-09-16T00:10:45.596 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.596 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.596 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.596 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.188724+0000", 2024-09-16T00:10:45.596 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0799999999999996e-06 2024-09-16T00:10:45.596 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.596 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.596 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.596 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.188738+0000", 2024-09-16T00:10:45.596 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3529999999999999e-05 2024-09-16T00:10:45.597 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.597 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.597 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.188745+0000", 2024-09-16T00:10:45.597 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7530000000000001e-06 2024-09-16T00:10:45.597 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.597 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.597 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.188829+0000", 2024-09-16T00:10:45.597 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3907999999999997e-05 2024-09-16T00:10:45.597 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.597 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.597 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:45.597 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:45.188845+0000", 2024-09-16T00:10:45.597 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6090999999999999e-05 2024-09-16T00:10:45.598 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.598 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.598 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.598 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.598 INFO:teuthology.orchestra.run.smithi139.stdout: ], 2024-09-16T00:10:45.598 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 5 2024-09-16T00:10:45.598 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:45.599 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-16T00:10:45.703 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:45.704 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:45.704 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:45.704 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:45.704 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:45.712 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-16T00:10:45.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:45.785+0000 7fcabeffd640 1 -- 172.21.15.139:0/1507230524 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v123) ==== 97+0+0 (secure 0 0 0) 0x7fcac005d4f0 con 0x7fcac8151680 2024-09-16T00:10:45.786 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is set 2024-09-16T00:10:45.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:45.789+0000 7fcad0386640 1 -- 172.21.15.139:0/1507230524 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcaac06c330 msgr2=0x7fcaac06e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:45.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:45.789+0000 7fcad0386640 1 --2- 172.21.15.139:0/1507230524 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcaac06c330 0x7fcaac06e7f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009aa0 tx=0x7fcac4009190 comp rx=0 tx=0).stop 2024-09-16T00:10:45.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:45.789+0000 7fcad0386640 1 -- 172.21.15.139:0/1507230524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcac8151680 msgr2=0x7fcac8159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:45.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:45.789+0000 7fcad0386640 1 --2- 172.21.15.139:0/1507230524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcac8151680 0x7fcac8159620 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcac000ce20 tx=0x7fcac000a430 comp rx=0 tx=0).stop 2024-09-16T00:10:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:45.789+0000 7fcad0386640 1 -- 172.21.15.139:0/1507230524 shutdown_connections 2024-09-16T00:10:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:45.789+0000 7fcad0386640 1 --2- 172.21.15.139:0/1507230524 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcaac06c330 0x7fcaac06e7f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:45.789+0000 7fcad0386640 1 --2- 172.21.15.139:0/1507230524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcac8151680 0x7fcac8159620 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:45.789+0000 7fcad0386640 1 --2- 172.21.15.139:0/1507230524 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcac80c70d0 0x7fcac814c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:45.789+0000 7fcad0386640 1 --2- 172.21.15.139:0/1507230524 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcac80c56d0 0x7fcac814c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:45.789+0000 7fcad0386640 1 -- 172.21.15.139:0/1507230524 >> 172.21.15.139:0/1507230524 conn(0x7fcac80c91d0 msgr2=0x7fcac814e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:45.790+0000 7fcad0386640 1 -- 172.21.15.139:0/1507230524 shutdown_connections 2024-09-16T00:10:45.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:45.790+0000 7fcad0386640 1 -- 172.21.15.139:0/1507230524 wait complete. 2024-09-16T00:10:45.824 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:45.824 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:45.824 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:45.825 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:45.825 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.825 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5850 3.3a 3:5c3cfc2b:::10000001adf.00000000:head [write 0~116] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:45.825 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:45.825 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 90.406308542999994, 2024-09-16T00:10:45.825 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66931295800000001, 2024-09-16T00:10:45.825 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.825 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.825 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.825 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:45.825 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:45.825 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:45.825 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5850 2024-09-16T00:10:45.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.826 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:45.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.826 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.826 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415937+0000", 2024-09-16T00:10:45.826 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.826 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.826 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415938+0000", 2024-09-16T00:10:45.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7099999999999999e-06 2024-09-16T00:10:45.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415945+0000", 2024-09-16T00:10:45.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0759999999999997e-06 2024-09-16T00:10:45.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.827 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.827 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.827 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415959+0000", 2024-09-16T00:10:45.827 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3545e-05 2024-09-16T00:10:45.827 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.415966+0000", 2024-09-16T00:10:45.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6379999999999998e-06 2024-09-16T00:10:45.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416013+0000", 2024-09-16T00:10:45.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7314999999999997e-05 2024-09-16T00:10:45.828 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.828 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.828 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:45.828 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.416020+0000", 2024-09-16T00:10:45.828 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.52e-06 2024-09-16T00:10:45.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082393+0000", 2024-09-16T00:10:45.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.66637251799999997 2024-09-16T00:10:45.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082537+0000", 2024-09-16T00:10:45.829 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014418800000000001 2024-09-16T00:10:45.829 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.829 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.829 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:45.829 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.082689+0000", 2024-09-16T00:10:45.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015198499999999999 2024-09-16T00:10:45.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084930+0000", 2024-09-16T00:10:45.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002241041 2024-09-16T00:10:45.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.830 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.830 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.830 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084994+0000", 2024-09-16T00:10:45.830 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3274000000000004e-05 2024-09-16T00:10:45.830 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085049+0000", 2024-09-16T00:10:45.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.554e-05 2024-09-16T00:10:45.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.831 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.831 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.831 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085250+0000", 2024-09-16T00:10:45.831 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00020060799999999999 2024-09-16T00:10:45.831 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.831 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.831 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.831 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.832 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.832 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5910 3.3a 3:5ec98a62:::10000001b22.00000000:head [write 0~1274] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:45.832 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:45.832 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 90.259364904999998, 2024-09-16T00:10:45.832 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.52286026600000002, 2024-09-16T00:10:45.832 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.832 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.832 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.832 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:45.832 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:45.832 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:45.832 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5910 2024-09-16T00:10:45.832 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.833 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:45.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.833 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.833 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562880+0000", 2024-09-16T00:10:45.833 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.833 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.833 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562881+0000", 2024-09-16T00:10:45.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:10:45.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562887+0000", 2024-09-16T00:10:45.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2870000000000004e-06 2024-09-16T00:10:45.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.834 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.834 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.834 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562897+0000", 2024-09-16T00:10:45.834 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.064e-05 2024-09-16T00:10:45.834 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562902+0000", 2024-09-16T00:10:45.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2340000000000002e-06 2024-09-16T00:10:45.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562958+0000", 2024-09-16T00:10:45.835 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5523999999999999e-05 2024-09-16T00:10:45.835 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.835 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.835 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:45.835 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.562968+0000", 2024-09-16T00:10:45.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0394000000000001e-05 2024-09-16T00:10:45.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083461+0000", 2024-09-16T00:10:45.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.520492968 2024-09-16T00:10:45.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.836 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.836 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.836 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083645+0000", 2024-09-16T00:10:45.836 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00018346399999999999 2024-09-16T00:10:45.836 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:45.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.083844+0000", 2024-09-16T00:10:45.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000199268 2024-09-16T00:10:45.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085410+0000", 2024-09-16T00:10:45.837 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015662359999999999 2024-09-16T00:10:45.837 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.837 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.837 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.837 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085493+0000", 2024-09-16T00:10:45.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2478999999999994e-05 2024-09-16T00:10:45.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085563+0000", 2024-09-16T00:10:45.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0159000000000004e-05 2024-09-16T00:10:45.838 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.838 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.838 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.838 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085741+0000", 2024-09-16T00:10:45.838 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017764300000000001 2024-09-16T00:10:45.838 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.838 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.838 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.839 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.839 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.839 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5962 3.3a 3:5c7b4d29:::10000001b5d.00000000:head [write 0~3074] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:45.839 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:45.839 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 90.129828087999996, 2024-09-16T00:10:45.839 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39368737500000001, 2024-09-16T00:10:45.839 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.839 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.839 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.839 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:45.839 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:45.839 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:45.839 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5962 2024-09-16T00:10:45.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.840 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:45.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.840 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692417+0000", 2024-09-16T00:10:45.840 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.840 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.840 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.840 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692418+0000", 2024-09-16T00:10:45.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3510000000000001e-06 2024-09-16T00:10:45.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692425+0000", 2024-09-16T00:10:45.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3770000000000001e-06 2024-09-16T00:10:45.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.841 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.841 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.841 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692440+0000", 2024-09-16T00:10:45.841 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4975e-05 2024-09-16T00:10:45.841 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692448+0000", 2024-09-16T00:10:45.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.6279999999999994e-06 2024-09-16T00:10:45.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692499+0000", 2024-09-16T00:10:45.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0872999999999999e-05 2024-09-16T00:10:45.842 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.842 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.842 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:45.842 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.692507+0000", 2024-09-16T00:10:45.842 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9889999999999992e-06 2024-09-16T00:10:45.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084221+0000", 2024-09-16T00:10:45.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39171379699999997 2024-09-16T00:10:45.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.843 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084446+0000", 2024-09-16T00:10:45.843 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00022519900000000001 2024-09-16T00:10:45.843 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.843 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.843 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:45.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.084672+0000", 2024-09-16T00:10:45.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000225405 2024-09-16T00:10:45.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085832+0000", 2024-09-16T00:10:45.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011600040000000001 2024-09-16T00:10:45.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.844 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.844 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085864+0000", 2024-09-16T00:10:45.844 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2592999999999998e-05 2024-09-16T00:10:45.844 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.844 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.085943+0000", 2024-09-16T00:10:45.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8534000000000004e-05 2024-09-16T00:10:45.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.845 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.845 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086105+0000", 2024-09-16T00:10:45.845 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016165 2024-09-16T00:10:45.845 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.845 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.845 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.845 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.845 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.846 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:5963 3.3a 3:5e4192f5:::10000001b5e.00000000:head [write 0~152] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:45.846 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:45.846 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 90.127763657000003, 2024-09-16T00:10:45.846 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39413463900000001, 2024-09-16T00:10:45.846 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.846 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.846 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.846 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:45.846 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:45.846 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:45.846 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 5963 2024-09-16T00:10:45.846 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.846 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.846 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:45.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:45.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.847 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.847 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.847 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.847 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694482+0000", 2024-09-16T00:10:45.847 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:10:45.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694486+0000", 2024-09-16T00:10:45.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0960000000000003e-06 2024-09-16T00:10:45.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694496+0000", 2024-09-16T00:10:45.848 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2189999999999993e-06 2024-09-16T00:10:45.848 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.848 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.848 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.848 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694500+0000", 2024-09-16T00:10:45.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9529999999999998e-06 2024-09-16T00:10:45.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694552+0000", 2024-09-16T00:10:45.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1360999999999998e-05 2024-09-16T00:10:45.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.849 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:45.849 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.694561+0000", 2024-09-16T00:10:45.849 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1700000000000003e-06 2024-09-16T00:10:45.849 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.849 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086346+0000", 2024-09-16T00:10:45.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.39178468 2024-09-16T00:10:45.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086480+0000", 2024-09-16T00:10:45.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013469299999999999 2024-09-16T00:10:45.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.850 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.850 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:45.850 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.086790+0000", 2024-09-16T00:10:45.850 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00030923300000000001 2024-09-16T00:10:45.850 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088361+0000", 2024-09-16T00:10:45.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015717050000000001 2024-09-16T00:10:45.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088432+0000", 2024-09-16T00:10:45.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0922999999999998e-05 2024-09-16T00:10:45.851 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.851 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.851 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.851 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088489+0000", 2024-09-16T00:10:45.851 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6653999999999997e-05 2024-09-16T00:10:45.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.852 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.852 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.852 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088616+0000", 2024-09-16T00:10:45.852 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012730099999999999 2024-09-16T00:10:45.852 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.852 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.852 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.852 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.853 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.853 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:6082 3.3a 3:5c50f65b:::10000001be7.00000000:head [write 0~2439] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-16T00:10:45.853 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:45.853 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 89.833990800999999, 2024-09-16T00:10:45.853 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.10065850799999999, 2024-09-16T00:10:45.853 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.853 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.853 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.853 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:45.853 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:45.853 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:45.853 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 6082 2024-09-16T00:10:45.853 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.854 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.854 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.854 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.854 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:45.854 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988254+0000", 2024-09-16T00:10:45.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.855 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.855 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988256+0000", 2024-09-16T00:10:45.855 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.065e-06 2024-09-16T00:10:45.855 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.855 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988262+0000", 2024-09-16T00:10:45.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4640000000000003e-06 2024-09-16T00:10:45.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988272+0000", 2024-09-16T00:10:45.856 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0335e-05 2024-09-16T00:10:45.856 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.856 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.856 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.856 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988279+0000", 2024-09-16T00:10:45.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:10:45.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988324+0000", 2024-09-16T00:10:45.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4953000000000001e-05 2024-09-16T00:10:45.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.857 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.857 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:45.857 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:15.988331+0000", 2024-09-16T00:10:45.857 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5989999999999998e-06 2024-09-16T00:10:45.857 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087272+0000", 2024-09-16T00:10:45.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.098941045000000005 2024-09-16T00:10:45.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087486+0000", 2024-09-16T00:10:45.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00021360199999999999 2024-09-16T00:10:45.858 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.858 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.858 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:45.858 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.087740+0000", 2024-09-16T00:10:45.858 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00025456799999999999 2024-09-16T00:10:45.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088638+0000", 2024-09-16T00:10:45.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00089793900000000005 2024-09-16T00:10:45.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.859 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.859 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.859 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088673+0000", 2024-09-16T00:10:45.859 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5111999999999999e-05 2024-09-16T00:10:45.859 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088722+0000", 2024-09-16T00:10:45.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8758000000000003e-05 2024-09-16T00:10:45.860 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.860 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.860 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.860 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:16.088913+0000", 2024-09-16T00:10:45.860 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000190953 2024-09-16T00:10:45.860 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.860 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.861 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.861 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.861 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.861 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10643 3.21 3:8453163e:::100000030b3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:45.861 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:45.861 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 70.222514607999997, 2024-09-16T00:10:45.861 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013028809000000001, 2024-09-16T00:10:45.861 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.861 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.861 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.861 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:45.862 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:45.862 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:45.862 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10643 2024-09-16T00:10:45.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.862 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:45.862 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.862 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.862 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.862 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.862 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599731+0000", 2024-09-16T00:10:45.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.599734+0000", 2024-09-16T00:10:45.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9299999999999999e-06 2024-09-16T00:10:45.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.863 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.863 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603345+0000", 2024-09-16T00:10:45.863 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.003611628 2024-09-16T00:10:45.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.863 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.864 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.864 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603368+0000", 2024-09-16T00:10:45.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3017e-05 2024-09-16T00:10:45.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.864 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.864 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603379+0000", 2024-09-16T00:10:45.864 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0528e-05 2024-09-16T00:10:45.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.864 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.864 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.864 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603448+0000", 2024-09-16T00:10:45.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9080999999999998e-05 2024-09-16T00:10:45.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603623+0000", 2024-09-16T00:10:45.865 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017525599999999999 2024-09-16T00:10:45.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.865 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.865 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:45.865 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.603766+0000", 2024-09-16T00:10:45.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000142977 2024-09-16T00:10:45.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.611250+0000", 2024-09-16T00:10:45.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074835240000000001 2024-09-16T00:10:45.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.866 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.866 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612617+0000", 2024-09-16T00:10:45.866 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001367793 2024-09-16T00:10:45.866 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.866 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612700+0000", 2024-09-16T00:10:45.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3123999999999999e-05 2024-09-16T00:10:45.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.867 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.867 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:35.612759+0000", 2024-09-16T00:10:45.867 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.8950999999999999e-05 2024-09-16T00:10:45.867 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.867 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.867 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.867 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.867 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11217 3.1f 3:fbcdb02a:::100000031c3.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.247965954999998, 2024-09-16T00:10:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.01231253, 2024-09-16T00:10:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11217 2024-09-16T00:10:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.868 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574279+0000", 2024-09-16T00:10:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.869 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.574282+0000", 2024-09-16T00:10:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8499999999999998e-06 2024-09-16T00:10:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575804+0000", 2024-09-16T00:10:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001521859 2024-09-16T00:10:45.870 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575826+0000", 2024-09-16T00:10:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2059999999999999e-05 2024-09-16T00:10:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575837+0000", 2024-09-16T00:10:45.871 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1313e-05 2024-09-16T00:10:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.575920+0000", 2024-09-16T00:10:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2918999999999997e-05 2024-09-16T00:10:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576088+0000", 2024-09-16T00:10:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016803799999999999 2024-09-16T00:10:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.872 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.576233+0000", 2024-09-16T00:10:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014508199999999999 2024-09-16T00:10:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.873 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.580500+0000", 2024-09-16T00:10:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004266644 2024-09-16T00:10:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586479+0000", 2024-09-16T00:10:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0059791820000000004 2024-09-16T00:10:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586541+0000", 2024-09-16T00:10:45.874 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1463000000000004e-05 2024-09-16T00:10:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.586592+0000", 2024-09-16T00:10:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1119999999999998e-05 2024-09-16T00:10:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.875 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11767 3.38 3:1fa50e65:::10000003117.00000000:head [write 0~4194304 [3@1169732]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 62.658561419999998, 2024-09-16T00:10:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011555678999999999, 2024-09-16T00:10:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:45.876 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11767 2024-09-16T00:10:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.877 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163684+0000", 2024-09-16T00:10:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.163686+0000", 2024-09-16T00:10:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3259999999999998e-06 2024-09-16T00:10:45.878 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165053+0000", 2024-09-16T00:10:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013665859999999999 2024-09-16T00:10:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.879 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165075+0000", 2024-09-16T00:10:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2206999999999999e-05 2024-09-16T00:10:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165084+0000", 2024-09-16T00:10:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.0710000000000001e-06 2024-09-16T00:10:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.880 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165157+0000", 2024-09-16T00:10:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2619000000000004e-05 2024-09-16T00:10:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165249+0000", 2024-09-16T00:10:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.2517999999999997e-05 2024-09-16T00:10:45.881 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.165360+0000", 2024-09-16T00:10:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011134899999999999 2024-09-16T00:10:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.882 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.169320+0000", 2024-09-16T00:10:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0039593040000000003 2024-09-16T00:10:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175135+0000", 2024-09-16T00:10:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.005814891 2024-09-16T00:10:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175199+0000", 2024-09-16T00:10:45.883 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.4474999999999995e-05 2024-09-16T00:10:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.175239+0000", 2024-09-16T00:10:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0333000000000002e-05 2024-09-16T00:10:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.884 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11890 3.38 3:1e014616:::100000033e9.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 62.250908242000001, 2024-09-16T00:10:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011833791999999999, 2024-09-16T00:10:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.885 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11890 2024-09-16T00:10:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.886 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571337+0000", 2024-09-16T00:10:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.571340+0000", 2024-09-16T00:10:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.36e-06 2024-09-16T00:10:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.887 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572866+0000", 2024-09-16T00:10:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001525549 2024-09-16T00:10:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572884+0000", 2024-09-16T00:10:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7796999999999999e-05 2024-09-16T00:10:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.888 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572897+0000", 2024-09-16T00:10:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3567000000000001e-05 2024-09-16T00:10:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.572970+0000", 2024-09-16T00:10:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2751999999999997e-05 2024-09-16T00:10:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.889 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573053+0000", 2024-09-16T00:10:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3230000000000001e-05 2024-09-16T00:10:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.573168+0000", 2024-09-16T00:10:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000114536 2024-09-16T00:10:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.890 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.577923+0000", 2024-09-16T00:10:45.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0047548720000000003 2024-09-16T00:10:45.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.891 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583081+0000", 2024-09-16T00:10:45.891 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051587630000000002 2024-09-16T00:10:45.891 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.891 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.891 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583137+0000", 2024-09-16T00:10:45.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5259000000000001e-05 2024-09-16T00:10:45.892 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.892 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.892 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.892 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.583171+0000", 2024-09-16T00:10:45.892 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4106999999999997e-05 2024-09-16T00:10:45.892 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:12037 3.9 e87/20 3:928db521:::1000000326d.00000000:head v 87'199, mlcod=87'197)", 2024-09-16T00:10:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 61.819718291000001, 2024-09-16T00:10:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012236153, 2024-09-16T00:10:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.893 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002527+0000", 2024-09-16T00:10:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.894 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.002529+0000", 2024-09-16T00:10:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4889999999999998e-06 2024-09-16T00:10:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007064+0000", 2024-09-16T00:10:45.895 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045346220000000003 2024-09-16T00:10:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007093+0000", 2024-09-16T00:10:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.9009e-05 2024-09-16T00:10:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007109+0000", 2024-09-16T00:10:45.896 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5846e-05 2024-09-16T00:10:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007196+0000", 2024-09-16T00:10:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.7507000000000001e-05 2024-09-16T00:10:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.007227+0000", 2024-09-16T00:10:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0916e-05 2024-09-16T00:10:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.897 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014658+0000", 2024-09-16T00:10:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074305090000000001 2024-09-16T00:10:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:44.014763+0000", 2024-09-16T00:10:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010525499999999999 2024-09-16T00:10:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.898 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:12398 3.2e 3:77a11d08:::10000003432.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 60.581039508000003, 2024-09-16T00:10:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012092999, 2024-09-16T00:10:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 12398 2024-09-16T00:10:45.899 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241206+0000", 2024-09-16T00:10:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.900 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.241209+0000", 2024-09-16T00:10:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2710000000000001e-06 2024-09-16T00:10:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243840+0000", 2024-09-16T00:10:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026307919999999999 2024-09-16T00:10:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243872+0000", 2024-09-16T00:10:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.2699e-05 2024-09-16T00:10:45.901 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243887+0000", 2024-09-16T00:10:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4348000000000001e-05 2024-09-16T00:10:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.243972+0000", 2024-09-16T00:10:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5358999999999994e-05 2024-09-16T00:10:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.902 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.244132+0000", 2024-09-16T00:10:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015972499999999999 2024-09-16T00:10:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253153+0000", 2024-09-16T00:10:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0090210280000000004 2024-09-16T00:10:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253213+0000", 2024-09-16T00:10:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0300999999999997e-05 2024-09-16T00:10:45.903 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:45.253299+0000", 2024-09-16T00:10:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5476000000000006e-05 2024-09-16T00:10:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15003 3.21 3:8673bb5f:::100000030ed.00000000:head [read 2097152~2097152 [2@1395527]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:45.904 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.653849414, 2024-09-16T00:10:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058433321000000003, 2024-09-16T00:10:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:10:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15003 2024-09-16T00:10:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.905 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168396+0000", 2024-09-16T00:10:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168397+0000", 2024-09-16T00:10:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3900000000000004e-07 2024-09-16T00:10:45.906 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168401+0000", 2024-09-16T00:10:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4050000000000002e-06 2024-09-16T00:10:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168411+0000", 2024-09-16T00:10:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6539999999999994e-06 2024-09-16T00:10:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.907 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168417+0000", 2024-09-16T00:10:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6719999999999998e-06 2024-09-16T00:10:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168467+0000", 2024-09-16T00:10:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9815999999999999e-05 2024-09-16T00:10:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.168535+0000", 2024-09-16T00:10:45.908 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7669999999999994e-05 2024-09-16T00:10:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.226829+0000", 2024-09-16T00:10:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.058294264999999998 2024-09-16T00:10:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15005 3.1f 3:f8460f10:::10000003370.00000000:head [read 1048576~1048576 [2@4194304]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:45.909 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 51.652168701999997, 2024-09-16T00:10:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.055844088, 2024-09-16T00:10:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "started", 2024-09-16T00:10:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15005 2024-09-16T00:10:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.910 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170077+0000", 2024-09-16T00:10:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170078+0000", 2024-09-16T00:10:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.235e-06 2024-09-16T00:10:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.911 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.170084+0000", 2024-09-16T00:10:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9739999999999999e-06 2024-09-16T00:10:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.912 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225526+0000", 2024-09-16T00:10:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.055441777999999997 2024-09-16T00:10:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225542+0000", 2024-09-16T00:10:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6606999999999999e-05 2024-09-16T00:10:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.913 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225668+0000", 2024-09-16T00:10:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012631 2024-09-16T00:10:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225744+0000", 2024-09-16T00:10:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5794e-05 2024-09-16T00:10:45.914 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.225921+0000", 2024-09-16T00:10:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00017639000000000001 2024-09-16T00:10:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:13174 3.38 3:1d27f0cd:::10000000db7.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:45.915 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:26.136612+0000", 2024-09-16T00:10:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 19.685633684999999, 2024-09-16T00:10:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013450962, 2024-09-16T00:10:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136612+0000", 2024-09-16T00:10:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.916 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136612+0000", 2024-09-16T00:10:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136612+0000", 2024-09-16T00:10:45.917 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9400000000000004e-07 2024-09-16T00:10:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136616+0000", 2024-09-16T00:10:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.0559999999999998e-06 2024-09-16T00:10:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136628+0000", 2024-09-16T00:10:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1494999999999999e-05 2024-09-16T00:10:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.918 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136637+0000", 2024-09-16T00:10:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3300000000000005e-06 2024-09-16T00:10:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136700+0000", 2024-09-16T00:10:45.919 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2889e-05 2024-09-16T00:10:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136830+0000", 2024-09-16T00:10:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012989300000000001 2024-09-16T00:10:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:45.920 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.136927+0000", 2024-09-16T00:10:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7115999999999996e-05 2024-09-16T00:10:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.138274+0000", 2024-09-16T00:10:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013470800000000001 2024-09-16T00:10:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.921 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.149951+0000", 2024-09-16T00:10:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011676485 2024-09-16T00:10:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.150002+0000", 2024-09-16T00:10:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1573999999999998e-05 2024-09-16T00:10:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.922 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:26.150063+0000", 2024-09-16T00:10:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0250000000000001e-05 2024-09-16T00:10:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.923 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:19626 3.3a 3:5ddfbb71:::100000026f8.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120)", 2024-09-16T00:10:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:44.193406+0000", 2024-09-16T00:10:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.628839226, 2024-09-16T00:10:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.36385258100000001, 2024-09-16T00:10:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.924 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.193406+0000", 2024-09-16T00:10:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.193406+0000", 2024-09-16T00:10:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.925 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.193407+0000", 2024-09-16T00:10:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.22e-06 2024-09-16T00:10:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.193412+0000", 2024-09-16T00:10:45.926 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6269999999999999e-06 2024-09-16T00:10:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.193426+0000", 2024-09-16T00:10:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4379000000000001e-05 2024-09-16T00:10:45.927 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.193434+0000", 2024-09-16T00:10:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0090000000000007e-06 2024-09-16T00:10:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.193495+0000", 2024-09-16T00:10:45.928 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0766000000000001e-05 2024-09-16T00:10:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.193510+0000", 2024-09-16T00:10:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5287000000000001e-05 2024-09-16T00:10:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.929 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.555380+0000", 2024-09-16T00:10:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.36186981400000001 2024-09-16T00:10:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.555534+0000", 2024-09-16T00:10:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015346899999999999 2024-09-16T00:10:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:45.930 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.555646+0000", 2024-09-16T00:10:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011261599999999999 2024-09-16T00:10:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.557060+0000", 2024-09-16T00:10:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014141469999999999 2024-09-16T00:10:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.931 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.557079+0000", 2024-09-16T00:10:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8601e-05 2024-09-16T00:10:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.557133+0000", 2024-09-16T00:10:45.932 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3742999999999997e-05 2024-09-16T00:10:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.557259+0000", 2024-09-16T00:10:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012590299999999999 2024-09-16T00:10:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.933 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:19632 3.3a 3:5ce9cbf5:::100000026fe.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120)", 2024-09-16T00:10:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:44.193554+0000", 2024-09-16T00:10:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.6286908979999999, 2024-09-16T00:10:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.36521039700000002, 2024-09-16T00:10:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.193554+0000", 2024-09-16T00:10:45.934 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.193554+0000", 2024-09-16T00:10:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.935 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.193555+0000", 2024-09-16T00:10:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4400000000000001e-07 2024-09-16T00:10:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.193557+0000", 2024-09-16T00:10:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6649999999999999e-06 2024-09-16T00:10:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.193564+0000", 2024-09-16T00:10:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.6150000000000002e-06 2024-09-16T00:10:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.936 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.193569+0000", 2024-09-16T00:10:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9359999999999998e-06 2024-09-16T00:10:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.193617+0000", 2024-09-16T00:10:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7547999999999999e-05 2024-09-16T00:10:45.937 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.193625+0000", 2024-09-16T00:10:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8009999999999993e-06 2024-09-16T00:10:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.556484+0000", 2024-09-16T00:10:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.36285883800000002 2024-09-16T00:10:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.938 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.556642+0000", 2024-09-16T00:10:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000157598 2024-09-16T00:10:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.556760+0000", 2024-09-16T00:10:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000118029 2024-09-16T00:10:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.939 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.558549+0000", 2024-09-16T00:10:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001789662 2024-09-16T00:10:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.558567+0000", 2024-09-16T00:10:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7091e-05 2024-09-16T00:10:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.558620+0000", 2024-09-16T00:10:45.940 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3801000000000002e-05 2024-09-16T00:10:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.558765+0000", 2024-09-16T00:10:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014436900000000001 2024-09-16T00:10:45.941 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:19666 3.3a 3:5f4ebc27:::1000000276d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120)", 2024-09-16T00:10:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:44.199824+0000", 2024-09-16T00:10:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.6224207530000001, 2024-09-16T00:10:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.359746593, 2024-09-16T00:10:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.942 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.199824+0000", 2024-09-16T00:10:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.199824+0000", 2024-09-16T00:10:45.943 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.199825+0000", 2024-09-16T00:10:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3e-07 2024-09-16T00:10:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.199829+0000", 2024-09-16T00:10:45.944 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4989999999999999e-06 2024-09-16T00:10:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.199837+0000", 2024-09-16T00:10:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3639999999999999e-06 2024-09-16T00:10:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.199843+0000", 2024-09-16T00:10:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.2450000000000003e-06 2024-09-16T00:10:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.945 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.199893+0000", 2024-09-16T00:10:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9951999999999999e-05 2024-09-16T00:10:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:45.946 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.199900+0000", 2024-09-16T00:10:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9070000000000001e-06 2024-09-16T00:10:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.557304+0000", 2024-09-16T00:10:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.357403796 2024-09-16T00:10:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.947 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.557442+0000", 2024-09-16T00:10:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013839500000000001 2024-09-16T00:10:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.557575+0000", 2024-09-16T00:10:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000132485 2024-09-16T00:10:45.948 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.558821+0000", 2024-09-16T00:10:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012458949999999999 2024-09-16T00:10:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.949 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.559417+0000", 2024-09-16T00:10:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00059672900000000005 2024-09-16T00:10:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.559465+0000", 2024-09-16T00:10:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7531000000000003e-05 2024-09-16T00:10:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.559571+0000", 2024-09-16T00:10:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000106165 2024-09-16T00:10:45.950 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.951 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.951 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.951 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.951 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:19699 3.3a 3:5e102132:::1000000287a.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120)", 2024-09-16T00:10:45.951 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:44.208598+0000", 2024-09-16T00:10:45.951 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.613647316, 2024-09-16T00:10:45.951 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.35189092599999999, 2024-09-16T00:10:45.951 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.951 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.951 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.951 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.952 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.952 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.208598+0000", 2024-09-16T00:10:45.952 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.952 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.953 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.953 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.208598+0000", 2024-09-16T00:10:45.953 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.953 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.953 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.953 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.208599+0000", 2024-09-16T00:10:45.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.136e-06 2024-09-16T00:10:45.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.954 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.954 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.954 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.208604+0000", 2024-09-16T00:10:45.954 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.096e-06 2024-09-16T00:10:45.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.208620+0000", 2024-09-16T00:10:45.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.5637999999999998e-05 2024-09-16T00:10:45.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.955 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.955 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.955 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.208628+0000", 2024-09-16T00:10:45.955 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9920000000000007e-06 2024-09-16T00:10:45.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.956 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.208691+0000", 2024-09-16T00:10:45.956 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3643e-05 2024-09-16T00:10:45.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.956 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.956 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:45.956 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.208702+0000", 2024-09-16T00:10:45.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0844e-05 2024-09-16T00:10:45.957 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.957 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.957 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.957 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.557931+0000", 2024-09-16T00:10:45.957 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.34922856099999999 2024-09-16T00:10:45.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.558079+0000", 2024-09-16T00:10:45.958 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014848699999999999 2024-09-16T00:10:45.958 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.958 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.958 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:45.958 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.558202+0000", 2024-09-16T00:10:45.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012229499999999999 2024-09-16T00:10:45.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.959 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.559584+0000", 2024-09-16T00:10:45.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0013819349999999999 2024-09-16T00:10:45.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.959 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.959 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.560300+0000", 2024-09-16T00:10:45.959 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00071649699999999997 2024-09-16T00:10:45.959 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.959 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.960 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.560360+0000", 2024-09-16T00:10:45.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9746e-05 2024-09-16T00:10:45.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.960 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.960 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.560489+0000", 2024-09-16T00:10:45.960 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012905600000000001 2024-09-16T00:10:45.960 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.960 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.960 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.960 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.961 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:19727 3.3a 3:5fa3d824:::10000002912.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120)", 2024-09-16T00:10:45.961 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:44.214417+0000", 2024-09-16T00:10:45.961 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.6078278830000001, 2024-09-16T00:10:45.961 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.34691664799999999, 2024-09-16T00:10:45.961 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.961 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.962 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.962 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.214417+0000", 2024-09-16T00:10:45.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.962 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.962 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.962 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.962 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.214417+0000", 2024-09-16T00:10:45.962 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.214418+0000", 2024-09-16T00:10:45.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6300000000000004e-07 2024-09-16T00:10:45.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.963 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.963 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.214422+0000", 2024-09-16T00:10:45.963 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6969999999999999e-06 2024-09-16T00:10:45.963 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.963 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.214430+0000", 2024-09-16T00:10:45.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3170000000000002e-06 2024-09-16T00:10:45.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.964 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.964 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.964 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.214437+0000", 2024-09-16T00:10:45.964 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9820000000000002e-06 2024-09-16T00:10:45.964 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.214507+0000", 2024-09-16T00:10:45.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9406999999999995e-05 2024-09-16T00:10:45.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.965 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.965 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:45.965 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.214517+0000", 2024-09-16T00:10:45.965 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0309000000000001e-05 2024-09-16T00:10:45.965 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.558870+0000", 2024-09-16T00:10:45.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.34435344400000001 2024-09-16T00:10:45.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.966 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.559019+0000", 2024-09-16T00:10:45.966 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014905300000000001 2024-09-16T00:10:45.966 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.966 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.966 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:45.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.559147+0000", 2024-09-16T00:10:45.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012787900000000001 2024-09-16T00:10:45.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.967 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.560560+0000", 2024-09-16T00:10:45.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001412665 2024-09-16T00:10:45.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.967 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.967 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.561142+0000", 2024-09-16T00:10:45.967 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00058223599999999997 2024-09-16T00:10:45.967 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.967 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.561192+0000", 2024-09-16T00:10:45.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9428000000000002e-05 2024-09-16T00:10:45.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.968 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.968 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.968 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.561334+0000", 2024-09-16T00:10:45.968 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014246800000000001 2024-09-16T00:10:45.968 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.968 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.968 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.968 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.969 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.969 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:19813 3.3a 3:5dbb0084:::10000002b64.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e120)", 2024-09-16T00:10:45.969 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:44.230287+0000", 2024-09-16T00:10:45.969 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 1.5919578570000001, 2024-09-16T00:10:45.969 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.33134834800000001, 2024-09-16T00:10:45.969 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:45.969 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:45.969 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:45.969 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:45.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:45.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.230287+0000", 2024-09-16T00:10:45.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:45.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.230287+0000", 2024-09-16T00:10:45.970 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:45.970 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.970 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.970 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:45.970 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.230288+0000", 2024-09-16T00:10:45.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6700000000000002e-07 2024-09-16T00:10:45.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.971 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:45.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.230293+0000", 2024-09-16T00:10:45.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6720000000000003e-06 2024-09-16T00:10:45.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.971 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.971 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:45.971 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.230308+0000", 2024-09-16T00:10:45.971 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4939999999999999e-05 2024-09-16T00:10:45.971 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.972 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:45.972 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.230316+0000", 2024-09-16T00:10:45.972 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0749999999999998e-06 2024-09-16T00:10:45.972 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.972 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.972 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.972 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.230409+0000", 2024-09-16T00:10:45.972 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3404000000000002e-05 2024-09-16T00:10:45.973 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.973 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.973 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:45.973 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.230422+0000", 2024-09-16T00:10:45.973 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2853e-05 2024-09-16T00:10:45.973 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.973 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.973 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:45.973 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.559630+0000", 2024-09-16T00:10:45.973 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.32920763199999997 2024-09-16T00:10:45.973 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.973 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.973 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:45.974 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.559797+0000", 2024-09-16T00:10:45.974 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000167014 2024-09-16T00:10:45.974 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.974 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.974 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:45.974 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.559929+0000", 2024-09-16T00:10:45.974 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013156499999999999 2024-09-16T00:10:45.974 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.974 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.974 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:45.974 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.561349+0000", 2024-09-16T00:10:45.974 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001420022 2024-09-16T00:10:45.974 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.974 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.975 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:45.975 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.561442+0000", 2024-09-16T00:10:45.975 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3670000000000003e-05 2024-09-16T00:10:45.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.975 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:45.975 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.561499+0000", 2024-09-16T00:10:45.975 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.7018999999999997e-05 2024-09-16T00:10:45.975 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:45.975 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:45.975 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:45.975 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:44.561636+0000", 2024-09-16T00:10:45.975 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000136515 2024-09-16T00:10:45.975 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.976 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.976 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.976 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:45.976 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:45.976 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:45.977 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-16T00:10:46.122 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:46.122 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:46.122 INFO:teuthology.orchestra.run.smithi139.stdout: "num_ops": 0 2024-09-16T00:10:46.122 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:46.131 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-16T00:10:46.254 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:46.254 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [], 2024-09-16T00:10:46.254 INFO:teuthology.orchestra.run.smithi139.stdout: "complaint_time": 180, 2024-09-16T00:10:46.255 INFO:teuthology.orchestra.run.smithi139.stdout: "num_blocked_ops": 0 2024-09-16T00:10:46.255 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:46.263 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-16T00:10:46.368 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:10:46.368 INFO:teuthology.orchestra.run.smithi139.stdout: "size": 20, 2024-09-16T00:10:46.368 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 600, 2024-09-16T00:10:46.369 INFO:teuthology.orchestra.run.smithi139.stdout: "ops": [ 2024-09-16T00:10:46.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.369 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:90 2.1b 2:ddb4e4d7:::200.00000002:head [write 2593031~1404 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e62)", 2024-09-16T00:10:46.369 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:46.369 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 102.717302828, 2024-09-16T00:10:46.369 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011882886, 2024-09-16T00:10:46.369 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.369 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.369 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.369 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.369 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.369 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.369 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:46.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649118+0000", 2024-09-16T00:10:46.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.370 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649119+0000", 2024-09-16T00:10:46.370 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.3799999999999996e-07 2024-09-16T00:10:46.370 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.370 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.370 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649125+0000", 2024-09-16T00:10:46.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1650000000000003e-06 2024-09-16T00:10:46.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649134+0000", 2024-09-16T00:10:46.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.5669999999999995e-06 2024-09-16T00:10:46.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.371 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.371 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.371 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.649141+0000", 2024-09-16T00:10:46.371 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.9040000000000003e-06 2024-09-16T00:10:46.371 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655796+0000", 2024-09-16T00:10:46.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0066553649999999999 2024-09-16T00:10:46.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.372 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.372 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655868+0000", 2024-09-16T00:10:46.372 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.2219999999999996e-05 2024-09-16T00:10:46.372 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.372 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:46.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.655986+0000", 2024-09-16T00:10:46.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000117457 2024-09-16T00:10:46.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.657794+0000", 2024-09-16T00:10:46.373 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001807894 2024-09-16T00:10:46.373 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.373 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.373 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:46.373 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660890+0000", 2024-09-16T00:10:46.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0030958449999999998 2024-09-16T00:10:46.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.660966+0000", 2024-09-16T00:10:46.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6382000000000002e-05 2024-09-16T00:10:46.374 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.374 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.374 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.374 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:03.661001+0000", 2024-09-16T00:10:46.374 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.5349000000000003e-05 2024-09-16T00:10:46.374 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.374 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.375 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.375 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.375 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10851 3.29 3:9786b8f5:::10000003100.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:46.375 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:46.375 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 67.473638784000002, 2024-09-16T00:10:46.375 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012109089999999999, 2024-09-16T00:10:46.375 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.375 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.375 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.375 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:46.375 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:46.375 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.375 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10851 2024-09-16T00:10:46.375 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.376 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:46.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.376 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892782+0000", 2024-09-16T00:10:46.376 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.376 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.376 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.376 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.892786+0000", 2024-09-16T00:10:46.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.4819999999999999e-06 2024-09-16T00:10:46.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894302+0000", 2024-09-16T00:10:46.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015156099999999999 2024-09-16T00:10:46.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.377 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.377 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894319+0000", 2024-09-16T00:10:46.377 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7098000000000001e-05 2024-09-16T00:10:46.377 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.377 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894329+0000", 2024-09-16T00:10:46.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9180000000000006e-06 2024-09-16T00:10:46.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894394+0000", 2024-09-16T00:10:46.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5797999999999996e-05 2024-09-16T00:10:46.378 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.378 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.378 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.378 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894532+0000", 2024-09-16T00:10:46.378 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000138128 2024-09-16T00:10:46.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:46.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.894633+0000", 2024-09-16T00:10:46.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010037100000000001 2024-09-16T00:10:46.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:46.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.899212+0000", 2024-09-16T00:10:46.379 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0045789369999999999 2024-09-16T00:10:46.379 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.379 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.379 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.379 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904771+0000", 2024-09-16T00:10:46.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0055591169999999997 2024-09-16T00:10:46.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904838+0000", 2024-09-16T00:10:46.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.7124000000000003e-05 2024-09-16T00:10:46.380 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.380 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.380 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.380 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:38.904892+0000", 2024-09-16T00:10:46.380 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.3507000000000001e-05 2024-09-16T00:10:46.380 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.380 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.380 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.381 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.381 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.381 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:10957 3.24 3:2748b9d7:::1000000313b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:46.381 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:46.381 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 66.851734430999997, 2024-09-16T00:10:46.381 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012163525, 2024-09-16T00:10:46.381 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.381 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.381 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.381 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:46.381 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:46.381 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.381 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 10957 2024-09-16T00:10:46.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.382 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:46.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.382 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514687+0000", 2024-09-16T00:10:46.382 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.382 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.382 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.382 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.514690+0000", 2024-09-16T00:10:46.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.382e-06 2024-09-16T00:10:46.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516204+0000", 2024-09-16T00:10:46.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001513831 2024-09-16T00:10:46.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.383 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.383 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516232+0000", 2024-09-16T00:10:46.383 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7657999999999998e-05 2024-09-16T00:10:46.383 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.383 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516246+0000", 2024-09-16T00:10:46.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3953e-05 2024-09-16T00:10:46.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516374+0000", 2024-09-16T00:10:46.384 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012882 2024-09-16T00:10:46.384 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.384 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.384 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.384 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516588+0000", 2024-09-16T00:10:46.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000213439 2024-09-16T00:10:46.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:46.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.516748+0000", 2024-09-16T00:10:46.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015971900000000001 2024-09-16T00:10:46.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:46.385 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.524102+0000", 2024-09-16T00:10:46.385 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0073542499999999997 2024-09-16T00:10:46.385 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.385 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.385 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526714+0000", 2024-09-16T00:10:46.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026122910000000001 2024-09-16T00:10:46.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526791+0000", 2024-09-16T00:10:46.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.6997999999999997e-05 2024-09-16T00:10:46.386 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.386 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.386 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.386 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:39.526850+0000", 2024-09-16T00:10:46.386 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9184000000000001e-05 2024-09-16T00:10:46.386 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.387 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.387 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.387 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.387 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11098 3.15 e86/20 3:aa8f498f:::100000030ba.00000000:head v 86'170, mlcod=86'168)", 2024-09-16T00:10:46.387 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:46.387 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 66.344951265000006, 2024-09-16T00:10:46.387 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013832641999999999, 2024-09-16T00:10:46.387 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.387 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.387 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.387 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.387 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.387 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:46.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021470+0000", 2024-09-16T00:10:46.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.388 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.388 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.388 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.021472+0000", 2024-09-16T00:10:46.388 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.9489999999999999e-06 2024-09-16T00:10:46.388 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027628+0000", 2024-09-16T00:10:46.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0061563859999999998 2024-09-16T00:10:46.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027645+0000", 2024-09-16T00:10:46.389 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.7108e-05 2024-09-16T00:10:46.389 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.389 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.389 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.389 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027655+0000", 2024-09-16T00:10:46.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.7999999999999993e-06 2024-09-16T00:10:46.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027712+0000", 2024-09-16T00:10:46.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6826000000000001e-05 2024-09-16T00:10:46.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.390 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027738+0000", 2024-09-16T00:10:46.390 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5601000000000001e-05 2024-09-16T00:10:46.390 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.390 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.390 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.035237+0000", 2024-09-16T00:10:46.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0074998699999999996 2024-09-16T00:10:46.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.391 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.391 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.035303+0000", 2024-09-16T00:10:46.391 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5102000000000001e-05 2024-09-16T00:10:46.391 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.391 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.391 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.391 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.391 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.391 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11099 3.b 3:d2f5b389:::100000030e7.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:46.391 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:46.392 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 66.340906450000006, 2024-09-16T00:10:46.392 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012707456000000001, 2024-09-16T00:10:46.392 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.392 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.392 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.392 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:46.392 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:46.392 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.392 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11099 2024-09-16T00:10:46.392 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.392 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.392 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.392 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:46.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025515+0000", 2024-09-16T00:10:46.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.393 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.393 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.393 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.025517+0000", 2024-09-16T00:10:46.393 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6199999999999999e-06 2024-09-16T00:10:46.393 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026955+0000", 2024-09-16T00:10:46.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.001437219 2024-09-16T00:10:46.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026982+0000", 2024-09-16T00:10:46.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7523999999999999e-05 2024-09-16T00:10:46.394 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.394 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.394 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.394 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.026996+0000", 2024-09-16T00:10:46.394 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4379000000000001e-05 2024-09-16T00:10:46.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027076+0000", 2024-09-16T00:10:46.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9933999999999997e-05 2024-09-16T00:10:46.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.395 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027185+0000", 2024-09-16T00:10:46.395 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010875899999999999 2024-09-16T00:10:46.395 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.395 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.395 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:46.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.027380+0000", 2024-09-16T00:10:46.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019477700000000001 2024-09-16T00:10:46.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038072+0000", 2024-09-16T00:10:46.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010692234 2024-09-16T00:10:46.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.396 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:46.396 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038115+0000", 2024-09-16T00:10:46.396 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.3291999999999998e-05 2024-09-16T00:10:46.396 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.396 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038172+0000", 2024-09-16T00:10:46.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.6981000000000002e-05 2024-09-16T00:10:46.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.397 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.397 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.038222+0000", 2024-09-16T00:10:46.397 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.9737000000000002e-05 2024-09-16T00:10:46.397 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.397 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.397 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.397 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.397 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.398 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11109 3.27 3:e5179c63:::10000003176.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:46.398 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:46.398 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 66.295442167000004, 2024-09-16T00:10:46.398 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012250990999999999, 2024-09-16T00:10:46.398 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.398 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.398 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.398 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:46.398 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:46.398 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.398 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11109 2024-09-16T00:10:46.398 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.398 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:46.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070979+0000", 2024-09-16T00:10:46.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.399 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.399 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.399 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.399 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.070981+0000", 2024-09-16T00:10:46.399 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.289e-06 2024-09-16T00:10:46.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072616+0000", 2024-09-16T00:10:46.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016347569999999999 2024-09-16T00:10:46.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.400 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072638+0000", 2024-09-16T00:10:46.400 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.1512999999999999e-05 2024-09-16T00:10:46.400 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.400 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.400 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072649+0000", 2024-09-16T00:10:46.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1626e-05 2024-09-16T00:10:46.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072775+0000", 2024-09-16T00:10:46.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012553900000000001 2024-09-16T00:10:46.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.401 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.401 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072856+0000", 2024-09-16T00:10:46.401 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.0882000000000003e-05 2024-09-16T00:10:46.401 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.401 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:46.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.072990+0000", 2024-09-16T00:10:46.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013483599999999999 2024-09-16T00:10:46.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:46.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.078354+0000", 2024-09-16T00:10:46.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0053639500000000001 2024-09-16T00:10:46.402 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.402 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.402 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.402 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083150+0000", 2024-09-16T00:10:46.402 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.004795517 2024-09-16T00:10:46.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083186+0000", 2024-09-16T00:10:46.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6019000000000002e-05 2024-09-16T00:10:46.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.403 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.403 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.403 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.083230+0000", 2024-09-16T00:10:46.403 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4063000000000001e-05 2024-09-16T00:10:46.403 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.403 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.403 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.403 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.404 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.404 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11234 3.b 3:d2f5b389:::100000030e7.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:46.404 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:46.404 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 65.742826398999995, 2024-09-16T00:10:46.404 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.015217661, 2024-09-16T00:10:46.404 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.404 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.404 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.404 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:46.404 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:46.404 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.404 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11234 2024-09-16T00:10:46.404 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.404 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:46.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623595+0000", 2024-09-16T00:10:46.405 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.405 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.405 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.405 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.405 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.623598+0000", 2024-09-16T00:10:46.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1580000000000001e-06 2024-09-16T00:10:46.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626844+0000", 2024-09-16T00:10:46.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0032459670000000002 2024-09-16T00:10:46.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.406 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.406 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626868+0000", 2024-09-16T00:10:46.406 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3584e-05 2024-09-16T00:10:46.406 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.406 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626879+0000", 2024-09-16T00:10:46.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1545999999999999e-05 2024-09-16T00:10:46.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.626950+0000", 2024-09-16T00:10:46.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.0784999999999994e-05 2024-09-16T00:10:46.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.407 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.407 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.407 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.627051+0000", 2024-09-16T00:10:46.407 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00010155899999999999 2024-09-16T00:10:46.407 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:46.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.627178+0000", 2024-09-16T00:10:46.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00012614899999999999 2024-09-16T00:10:46.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:46.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.635220+0000", 2024-09-16T00:10:46.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0080421490000000002 2024-09-16T00:10:46.408 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.408 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.408 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.408 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638699+0000", 2024-09-16T00:10:46.408 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034797500000000002 2024-09-16T00:10:46.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638760+0000", 2024-09-16T00:10:46.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0319000000000003e-05 2024-09-16T00:10:46.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.409 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.409 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.409 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:40.638812+0000", 2024-09-16T00:10:46.409 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.2695000000000003e-05 2024-09-16T00:10:46.409 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.409 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.409 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.409 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.410 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.410 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:11777 3.b 3:d342b4ff:::10000003113.00000000:head [write 0~4194304 [2@1171070]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:46.410 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:46.410 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 63.175186990999997, 2024-09-16T00:10:46.410 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013079176, 2024-09-16T00:10:46.410 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.410 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.410 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.410 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:46.410 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:46.410 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.410 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 11777 2024-09-16T00:10:46.410 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.410 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:46.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191234+0000", 2024-09-16T00:10:46.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.411 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.411 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.411 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.411 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.191237+0000", 2024-09-16T00:10:46.411 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.3369999999999998e-06 2024-09-16T00:10:46.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192825+0000", 2024-09-16T00:10:46.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015887189999999999 2024-09-16T00:10:46.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.412 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192846+0000", 2024-09-16T00:10:46.412 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0670999999999999e-05 2024-09-16T00:10:46.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.412 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.412 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192858+0000", 2024-09-16T00:10:46.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.223e-05 2024-09-16T00:10:46.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.192957+0000", 2024-09-16T00:10:46.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8416000000000001e-05 2024-09-16T00:10:46.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.413 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.413 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.193031+0000", 2024-09-16T00:10:46.413 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.4381999999999994e-05 2024-09-16T00:10:46.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.413 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:46.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.193164+0000", 2024-09-16T00:10:46.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013349100000000001 2024-09-16T00:10:46.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:46.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204144+0000", 2024-09-16T00:10:46.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010979917 2024-09-16T00:10:46.414 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.414 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.414 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.414 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204204+0000", 2024-09-16T00:10:46.414 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0025999999999998e-05 2024-09-16T00:10:46.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204265+0000", 2024-09-16T00:10:46.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.0645e-05 2024-09-16T00:10:46.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.415 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.415 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.415 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.204313+0000", 2024-09-16T00:10:46.415 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.8341999999999999e-05 2024-09-16T00:10:46.415 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.415 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.415 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.415 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.416 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:11981 3.c e87/20 3:33faa5a0:::10000003215.00000000:head v 87'186, mlcod=87'184)", 2024-09-16T00:10:46.416 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:46.416 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 62.534952167999997, 2024-09-16T00:10:46.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011828697000000001, 2024-09-16T00:10:46.416 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.416 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.416 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.416 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.416 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.416 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.416 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:46.416 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831469+0000", 2024-09-16T00:10:46.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.831471+0000", 2024-09-16T00:10:46.417 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.6220000000000001e-06 2024-09-16T00:10:46.417 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.417 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.417 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.417 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836584+0000", 2024-09-16T00:10:46.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051133510000000004 2024-09-16T00:10:46.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836606+0000", 2024-09-16T00:10:46.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2079e-05 2024-09-16T00:10:46.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.418 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836624+0000", 2024-09-16T00:10:46.418 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.8162e-05 2024-09-16T00:10:46.418 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.418 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.418 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836704+0000", 2024-09-16T00:10:46.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.9867e-05 2024-09-16T00:10:46.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.836730+0000", 2024-09-16T00:10:46.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6285999999999999e-05 2024-09-16T00:10:46.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.419 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.419 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.843247+0000", 2024-09-16T00:10:46.419 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0065165650000000002 2024-09-16T00:10:46.419 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.419 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.420 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.420 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:43.843298+0000", 2024-09-16T00:10:46.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.0764999999999999e-05 2024-09-16T00:10:46.420 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.420 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.420 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.420 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.420 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.420 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13365 3.2 3:4298088d:::100000035be.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:46.420 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:46.420 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.347250189, 2024-09-16T00:10:46.420 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014036442999999999, 2024-09-16T00:10:46.420 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.420 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.421 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.421 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:46.421 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:46.421 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.421 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13365 2024-09-16T00:10:46.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.421 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.421 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.421 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:46.421 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.421 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.421 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019171+0000", 2024-09-16T00:10:46.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.019174+0000", 2024-09-16T00:10:46.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.1250000000000001e-06 2024-09-16T00:10:46.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.422 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.422 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020613+0000", 2024-09-16T00:10:46.422 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0014391930000000001 2024-09-16T00:10:46.422 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.422 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020638+0000", 2024-09-16T00:10:46.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.4657000000000001e-05 2024-09-16T00:10:46.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020651+0000", 2024-09-16T00:10:46.423 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3451e-05 2024-09-16T00:10:46.423 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.423 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.423 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.423 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020744+0000", 2024-09-16T00:10:46.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.3010000000000006e-05 2024-09-16T00:10:46.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.020912+0000", 2024-09-16T00:10:46.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016763800000000001 2024-09-16T00:10:46.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:46.424 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.021043+0000", 2024-09-16T00:10:46.424 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00013099800000000001 2024-09-16T00:10:46.424 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.424 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.424 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:46.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026194+0000", 2024-09-16T00:10:46.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0051510879999999998 2024-09-16T00:10:46.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033132+0000", 2024-09-16T00:10:46.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0069377279999999998 2024-09-16T00:10:46.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.425 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.425 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.425 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033169+0000", 2024-09-16T00:10:46.425 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.6897000000000003e-05 2024-09-16T00:10:46.425 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.426 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.426 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.033207+0000", 2024-09-16T00:10:46.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8658000000000002e-05 2024-09-16T00:10:46.426 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.426 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.426 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.426 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.426 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.426 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13368 3.27 3:e75b72a1:::100000032e8.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:46.426 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:46.426 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.339884306000002, 2024-09-16T00:10:46.426 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013615175, 2024-09-16T00:10:46.426 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.427 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.427 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.427 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:46.427 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:46.427 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.427 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13368 2024-09-16T00:10:46.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.427 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.427 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.427 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:46.427 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.427 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.427 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026537+0000", 2024-09-16T00:10:46.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.026539+0000", 2024-09-16T00:10:46.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.798e-06 2024-09-16T00:10:46.428 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.428 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.428 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.428 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027825+0000", 2024-09-16T00:10:46.428 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0012861839999999999 2024-09-16T00:10:46.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027850+0000", 2024-09-16T00:10:46.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.5048000000000001e-05 2024-09-16T00:10:46.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.429 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.429 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027865+0000", 2024-09-16T00:10:46.429 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4939e-05 2024-09-16T00:10:46.429 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.429 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.027944+0000", 2024-09-16T00:10:46.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8881999999999995e-05 2024-09-16T00:10:46.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.028043+0000", 2024-09-16T00:10:46.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.9135000000000005e-05 2024-09-16T00:10:46.430 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.430 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.430 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:46.430 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.028158+0000", 2024-09-16T00:10:46.430 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011527999999999999 2024-09-16T00:10:46.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.039292+0000", 2024-09-16T00:10:46.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011133568999999999 2024-09-16T00:10:46.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:46.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.039948+0000", 2024-09-16T00:10:46.431 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00065625700000000004 2024-09-16T00:10:46.431 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.431 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.431 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.431 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.040061+0000", 2024-09-16T00:10:46.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00011284400000000001 2024-09-16T00:10:46.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.432 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.432 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.040152+0000", 2024-09-16T00:10:46.432 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.1238999999999998e-05 2024-09-16T00:10:46.432 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.432 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.432 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.432 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.432 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.432 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_repop(client.4638.0:13366 3.15 e88/20 3:aa8f498f:::100000030ba.00000000:head v 88'196, mlcod=88'194)", 2024-09-16T00:10:46.432 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:46.433 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.336459230999999, 2024-09-16T00:10:46.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012822817, 2024-09-16T00:10:46.433 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.433 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.433 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.433 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.433 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:46.433 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.433 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.433 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.433 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029962+0000", 2024-09-16T00:10:46.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.029963+0000", 2024-09-16T00:10:46.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.4270000000000001e-06 2024-09-16T00:10:46.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.434 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.434 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034078+0000", 2024-09-16T00:10:46.434 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0041148360000000002 2024-09-16T00:10:46.434 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.434 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034094+0000", 2024-09-16T00:10:46.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.535e-05 2024-09-16T00:10:46.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.034103+0000", 2024-09-16T00:10:46.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:10:46.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.435 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.435 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.435 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.036791+0000", 2024-09-16T00:10:46.435 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026878340000000001 2024-09-16T00:10:46.435 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.036818+0000", 2024-09-16T00:10:46.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6885000000000001e-05 2024-09-16T00:10:46.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.042713+0000", 2024-09-16T00:10:46.436 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0058950399999999998 2024-09-16T00:10:46.436 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.436 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.436 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.436 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.042785+0000", 2024-09-16T00:10:46.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1927000000000004e-05 2024-09-16T00:10:46.437 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.437 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.437 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.437 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.437 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.437 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13405 3.27 3:e708eda4:::100000034c4.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:46.437 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:46.437 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 58.210257517999999, 2024-09-16T00:10:46.437 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012003333, 2024-09-16T00:10:46.437 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.437 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.437 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.437 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:46.438 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:46.438 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.438 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13405 2024-09-16T00:10:46.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.438 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:46.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.438 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.438 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.438 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.438 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156164+0000", 2024-09-16T00:10:46.438 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.156166+0000", 2024-09-16T00:10:46.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6230000000000001e-06 2024-09-16T00:10:46.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157868+0000", 2024-09-16T00:10:46.439 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0017017 2024-09-16T00:10:46.439 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.439 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.439 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.439 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157891+0000", 2024-09-16T00:10:46.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2931000000000001e-05 2024-09-16T00:10:46.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157904+0000", 2024-09-16T00:10:46.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.2763e-05 2024-09-16T00:10:46.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.157988+0000", 2024-09-16T00:10:46.440 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.3849000000000003e-05 2024-09-16T00:10:46.440 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.440 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.440 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.440 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.158083+0000", 2024-09-16T00:10:46.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.5939000000000004e-05 2024-09-16T00:10:46.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:46.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.158194+0000", 2024-09-16T00:10:46.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110963 2024-09-16T00:10:46.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:46.441 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.164635+0000", 2024-09-16T00:10:46.441 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0064405570000000004 2024-09-16T00:10:46.441 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.441 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.441 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168088+0000", 2024-09-16T00:10:46.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0034534639999999998 2024-09-16T00:10:46.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168140+0000", 2024-09-16T00:10:46.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1041000000000001e-05 2024-09-16T00:10:46.442 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.442 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.442 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.442 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:48.168167+0000", 2024-09-16T00:10:46.442 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.7503000000000001e-05 2024-09-16T00:10:46.442 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.442 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.443 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.443 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.443 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.443 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:13685 3.b 3:d26c1580:::100000032e4.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:46.443 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:46.443 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 57.031570553000002, 2024-09-16T00:10:46.443 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014868124999999999, 2024-09-16T00:10:46.443 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.443 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.443 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.443 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:46.443 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:46.443 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.443 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 13685 2024-09-16T00:10:46.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.444 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:46.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.444 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.444 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334851+0000", 2024-09-16T00:10:46.444 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.444 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.444 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.334854+0000", 2024-09-16T00:10:46.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0000000000000001e-06 2024-09-16T00:10:46.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337155+0000", 2024-09-16T00:10:46.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023016870000000002 2024-09-16T00:10:46.445 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.445 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.445 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.445 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337178+0000", 2024-09-16T00:10:46.445 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.2898e-05 2024-09-16T00:10:46.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337190+0000", 2024-09-16T00:10:46.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1817999999999999e-05 2024-09-16T00:10:46.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337301+0000", 2024-09-16T00:10:46.446 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110951 2024-09-16T00:10:46.446 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.446 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.446 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.446 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337403+0000", 2024-09-16T00:10:46.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000102118 2024-09-16T00:10:46.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:46.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.337545+0000", 2024-09-16T00:10:46.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014167199999999999 2024-09-16T00:10:46.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:46.447 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.346336+0000", 2024-09-16T00:10:46.447 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0087911450000000002 2024-09-16T00:10:46.447 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.447 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.447 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349641+0000", 2024-09-16T00:10:46.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0033047570000000002 2024-09-16T00:10:46.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349680+0000", 2024-09-16T00:10:46.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9054999999999998e-05 2024-09-16T00:10:46.448 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.448 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.448 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.448 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:49.349719+0000", 2024-09-16T00:10:46.448 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9023999999999998e-05 2024-09-16T00:10:46.448 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.449 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.449 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.449 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.449 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.449 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14624 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:46.449 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:46.449 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 53.751811351999997, 2024-09-16T00:10:46.449 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012123376, 2024-09-16T00:10:46.449 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.449 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.449 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.449 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:46.449 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:46.449 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.449 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14624 2024-09-16T00:10:46.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.450 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:46.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.450 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614610+0000", 2024-09-16T00:10:46.450 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.450 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.450 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.450 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.614614+0000", 2024-09-16T00:10:46.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9500000000000003e-06 2024-09-16T00:10:46.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616480+0000", 2024-09-16T00:10:46.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0018663169999999999 2024-09-16T00:10:46.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.451 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.451 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616506+0000", 2024-09-16T00:10:46.451 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6205999999999999e-05 2024-09-16T00:10:46.451 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.451 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616517+0000", 2024-09-16T00:10:46.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0399e-05 2024-09-16T00:10:46.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616595+0000", 2024-09-16T00:10:46.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.8170999999999994e-05 2024-09-16T00:10:46.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.452 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.452 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.452 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616683+0000", 2024-09-16T00:10:46.452 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.8578999999999993e-05 2024-09-16T00:10:46.452 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:46.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.616825+0000", 2024-09-16T00:10:46.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00014186299999999999 2024-09-16T00:10:46.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:46.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.624948+0000", 2024-09-16T00:10:46.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0081230420000000005 2024-09-16T00:10:46.453 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.453 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.453 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.453 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626617+0000", 2024-09-16T00:10:46.453 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016681980000000001 2024-09-16T00:10:46.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626678+0000", 2024-09-16T00:10:46.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.1362e-05 2024-09-16T00:10:46.454 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.454 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.454 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.454 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.626733+0000", 2024-09-16T00:10:46.454 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5288999999999999e-05 2024-09-16T00:10:46.454 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.454 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.454 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.455 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.455 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14672 3.29 3:9641a85f:::1000000360e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:46.455 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:46.455 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 53.547354175000002, 2024-09-16T00:10:46.455 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012258025000000001, 2024-09-16T00:10:46.455 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.455 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.455 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.455 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:46.455 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:46.455 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.455 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14672 2024-09-16T00:10:46.455 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819067+0000", 2024-09-16T00:10:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.456 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.819070+0000", 2024-09-16T00:10:46.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0589999999999998e-06 2024-09-16T00:10:46.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821701+0000", 2024-09-16T00:10:46.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0026309100000000002 2024-09-16T00:10:46.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.457 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821727+0000", 2024-09-16T00:10:46.457 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.6372000000000001e-05 2024-09-16T00:10:46.457 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.457 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.457 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821738+0000", 2024-09-16T00:10:46.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0759e-05 2024-09-16T00:10:46.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821821+0000", 2024-09-16T00:10:46.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.2854000000000003e-05 2024-09-16T00:10:46.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.458 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.458 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.821920+0000", 2024-09-16T00:10:46.458 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.8900999999999994e-05 2024-09-16T00:10:46.458 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.458 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:46.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.822084+0000", 2024-09-16T00:10:46.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00016432799999999999 2024-09-16T00:10:46.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:46.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.829148+0000", 2024-09-16T00:10:46.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0070641460000000003 2024-09-16T00:10:46.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.459 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.459 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.459 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831242+0000", 2024-09-16T00:10:46.459 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.002093293 2024-09-16T00:10:46.459 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831280+0000", 2024-09-16T00:10:46.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.8707000000000001e-05 2024-09-16T00:10:46.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.460 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.460 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:52.831325+0000", 2024-09-16T00:10:46.460 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.4696e-05 2024-09-16T00:10:46.460 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.460 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.460 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.460 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.460 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.461 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14829 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304 [2@2556691]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:46.461 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:46.461 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 52.794834756999997, 2024-09-16T00:10:46.461 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.014399683, 2024-09-16T00:10:46.461 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.461 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.461 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.461 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:46.461 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:46.461 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.461 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14829 2024-09-16T00:10:46.461 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.461 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:46.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571586+0000", 2024-09-16T00:10:46.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.462 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.462 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.462 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.462 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.571590+0000", 2024-09-16T00:10:46.462 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.9190000000000001e-06 2024-09-16T00:10:46.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.463 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573133+0000", 2024-09-16T00:10:46.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0015422109999999999 2024-09-16T00:10:46.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.463 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573161+0000", 2024-09-16T00:10:46.463 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.8045e-05 2024-09-16T00:10:46.463 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.463 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.463 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.463 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573174+0000", 2024-09-16T00:10:46.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.3193999999999999e-05 2024-09-16T00:10:46.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573365+0000", 2024-09-16T00:10:46.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00019096400000000001 2024-09-16T00:10:46.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.464 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573467+0000", 2024-09-16T00:10:46.464 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000101789 2024-09-16T00:10:46.464 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.464 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.464 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:46.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.573610+0000", 2024-09-16T00:10:46.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000143404 2024-09-16T00:10:46.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.583614+0000", 2024-09-16T00:10:46.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.010004422000000001 2024-09-16T00:10:46.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.465 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:46.465 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585556+0000", 2024-09-16T00:10:46.465 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0019414009999999999 2024-09-16T00:10:46.465 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.465 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585939+0000", 2024-09-16T00:10:46.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00038340999999999999 2024-09-16T00:10:46.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.466 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.466 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.585986+0000", 2024-09-16T00:10:46.466 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.6924e-05 2024-09-16T00:10:46.466 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.466 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.466 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.466 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.466 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.466 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:14914 3.b 3:d23f6103:::10000003277.00000000:head [write 0~4194304 [3@58131]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:46.467 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:46.467 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 52.483630986000001, 2024-09-16T00:10:46.467 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.013007654, 2024-09-16T00:10:46.467 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.467 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.467 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.467 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:46.467 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:46.467 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.467 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 14914 2024-09-16T00:10:46.467 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.467 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.467 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.467 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:46.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882790+0000", 2024-09-16T00:10:46.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.468 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.468 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.882793+0000", 2024-09-16T00:10:46.468 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.65e-06 2024-09-16T00:10:46.468 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.468 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885193+0000", 2024-09-16T00:10:46.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0024003449999999999 2024-09-16T00:10:46.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885214+0000", 2024-09-16T00:10:46.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.0398e-05 2024-09-16T00:10:46.469 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.469 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.469 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.469 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885224+0000", 2024-09-16T00:10:46.469 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0253e-05 2024-09-16T00:10:46.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885299+0000", 2024-09-16T00:10:46.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.5434999999999999e-05 2024-09-16T00:10:46.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885389+0000", 2024-09-16T00:10:46.470 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 8.9209999999999995e-05 2024-09-16T00:10:46.470 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.470 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.470 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:46.470 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.885485+0000", 2024-09-16T00:10:46.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 9.6218999999999995e-05 2024-09-16T00:10:46.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.893457+0000", 2024-09-16T00:10:46.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0079722009999999999 2024-09-16T00:10:46.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:46.471 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895681+0000", 2024-09-16T00:10:46.471 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0022241449999999999 2024-09-16T00:10:46.471 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.471 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.471 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895732+0000", 2024-09-16T00:10:46.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.1269e-05 2024-09-16T00:10:46.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.472 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.472 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:53.895798+0000", 2024-09-16T00:10:46.472 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.5529000000000002e-05 2024-09-16T00:10:46.472 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.472 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.472 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.472 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.472 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.472 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(client.4638.0:15058 3.18 3:1b95746d:::1000000335b.00000000:head [write 0~4194304 [2@2556691]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-16T00:10:46.472 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:46.473 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 52.031262392000002, 2024-09-16T00:10:46.473 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.012275842, 2024-09-16T00:10:46.473 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.473 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.473 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.473 INFO:teuthology.orchestra.run.smithi139.stdout: "client_info": { 2024-09-16T00:10:46.473 INFO:teuthology.orchestra.run.smithi139.stdout: "client": "client.4638", 2024-09-16T00:10:46.473 INFO:teuthology.orchestra.run.smithi139.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.473 INFO:teuthology.orchestra.run.smithi139.stdout: "tid": 15058 2024-09-16T00:10:46.473 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.474 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:46.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335159+0000", 2024-09-16T00:10:46.474 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.474 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.474 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.474 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.474 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.335161+0000", 2024-09-16T00:10:46.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.367e-06 2024-09-16T00:10:46.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337545+0000", 2024-09-16T00:10:46.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0023839040000000001 2024-09-16T00:10:46.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.475 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337572+0000", 2024-09-16T00:10:46.475 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.673e-05 2024-09-16T00:10:46.475 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.475 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.475 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337583+0000", 2024-09-16T00:10:46.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1048e-05 2024-09-16T00:10:46.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337660+0000", 2024-09-16T00:10:46.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.7459999999999994e-05 2024-09-16T00:10:46.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.476 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "started", 2024-09-16T00:10:46.476 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337770+0000", 2024-09-16T00:10:46.476 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000109634 2024-09-16T00:10:46.476 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.476 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:46.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.337926+0000", 2024-09-16T00:10:46.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.00015601999999999999 2024-09-16T00:10:46.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.345689+0000", 2024-09-16T00:10:46.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0077625630000000001 2024-09-16T00:10:46.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.477 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.477 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "op_commit", 2024-09-16T00:10:46.477 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347324+0000", 2024-09-16T00:10:46.477 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0016351479999999999 2024-09-16T00:10:46.477 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "commit_sent", 2024-09-16T00:10:46.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347387+0000", 2024-09-16T00:10:46.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.3199000000000003e-05 2024-09-16T00:10:46.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.478 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.478 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:09:54.347435+0000", 2024-09-16T00:10:46.478 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 4.7769000000000003e-05 2024-09-16T00:10:46.478 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.478 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.478 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.478 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.478 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.479 INFO:teuthology.orchestra.run.smithi139.stdout: "description": "osd_op(mds.0.4:9780 2.a 2:545c15c7:::200.00000017:head [write 1367106~16252 [fadvise_dontneed]] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e106)", 2024-09-16T00:10:46.479 INFO:teuthology.orchestra.run.smithi139.stdout: "initiated_at": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:46.479 INFO:teuthology.orchestra.run.smithi139.stdout: "age": 30.029078267999999, 2024-09-16T00:10:46.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0208196810000001, 2024-09-16T00:10:46.479 INFO:teuthology.orchestra.run.smithi139.stdout: "continuous": false, 2024-09-16T00:10:46.479 INFO:teuthology.orchestra.run.smithi139.stdout: "type_data": { 2024-09-16T00:10:46.479 INFO:teuthology.orchestra.run.smithi139.stdout: "flag_point": "reached pg", 2024-09-16T00:10:46.479 INFO:teuthology.orchestra.run.smithi139.stdout: "events": [ 2024-09-16T00:10:46.479 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.479 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "initiated", 2024-09-16T00:10:46.479 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:46.479 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.479 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "header_read", 2024-09-16T00:10:46.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337343+0000", 2024-09-16T00:10:46.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0 2024-09-16T00:10:46.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "throttled", 2024-09-16T00:10:46.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337345+0000", 2024-09-16T00:10:46.480 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 2.007e-06 2024-09-16T00:10:46.480 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.480 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.480 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "all_read", 2024-09-16T00:10:46.480 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337416+0000", 2024-09-16T00:10:46.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 7.1260999999999994e-05 2024-09-16T00:10:46.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "dispatched", 2024-09-16T00:10:46.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337427+0000", 2024-09-16T00:10:46.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.1225e-05 2024-09-16T00:10:46.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.481 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337434+0000", 2024-09-16T00:10:46.481 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 6.8929999999999999e-06 2024-09-16T00:10:46.481 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.481 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.481 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337494+0000", 2024-09-16T00:10:46.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.9670999999999998e-05 2024-09-16T00:10:46.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for peered", 2024-09-16T00:10:46.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:16.337505+0000", 2024-09-16T00:10:46.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0934999999999999e-05 2024-09-16T00:10:46.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.482 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.482 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.482 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.345708+0000", 2024-09-16T00:10:46.482 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 1.0082033290000001 2024-09-16T00:10:46.482 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for missing object", 2024-09-16T00:10:46.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.345819+0000", 2024-09-16T00:10:46.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.000110299 2024-09-16T00:10:46.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.346937+0000", 2024-09-16T00:10:46.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.0011184120000000001 2024-09-16T00:10:46.483 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.483 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.483 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "waiting for degraded object", 2024-09-16T00:10:46.483 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.346968+0000", 2024-09-16T00:10:46.483 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 3.0664999999999999e-05 2024-09-16T00:10:46.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "reached_pg", 2024-09-16T00:10:46.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.358107+0000", 2024-09-16T00:10:46.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 0.011139046 2024-09-16T00:10:46.484 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-16T00:10:46.484 INFO:teuthology.orchestra.run.smithi139.stdout: { 2024-09-16T00:10:46.484 INFO:teuthology.orchestra.run.smithi139.stdout: "event": "done", 2024-09-16T00:10:46.484 INFO:teuthology.orchestra.run.smithi139.stdout: "time": "2024-09-16T00:10:17.358163+0000", 2024-09-16T00:10:46.484 INFO:teuthology.orchestra.run.smithi139.stdout: "duration": 5.5937999999999999e-05 2024-09-16T00:10:46.484 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.484 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.484 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.484 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-16T00:10:46.485 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-09-16T00:10:46.485 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:10:46.486 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-16T00:10:46.602 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:46.602 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:46.602 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:46.602 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:46.611 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-16T00:10:46.715 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:46.715 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:46.715 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:46.715 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:46.715 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:46.724 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-16T00:10:46.831 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:46.831 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:46.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:46.831 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:46.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.831 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10780 3.a 3:500c83b2:::100000030b4.00000000:head [write 0~4194304 [3@2137521]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:46.831 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:46.832 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.375662641000005, 2024-09-16T00:10:46.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015526478, 2024-09-16T00:10:46.832 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.832 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.832 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.832 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.832 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10780 2024-09-16T00:10:46.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.832 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:46.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452915+0000", 2024-09-16T00:10:46.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.452917+0000", 2024-09-16T00:10:46.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2180000000000002e-06 2024-09-16T00:10:46.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462177+0000", 2024-09-16T00:10:46.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092593769999999992 2024-09-16T00:10:46.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462196+0000", 2024-09-16T00:10:46.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9745000000000002e-05 2024-09-16T00:10:46.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.834 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.834 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462207+0000", 2024-09-16T00:10:46.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0524e-05 2024-09-16T00:10:46.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462284+0000", 2024-09-16T00:10:46.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7608000000000006e-05 2024-09-16T00:10:46.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.462376+0000", 2024-09-16T00:10:46.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1811999999999994e-05 2024-09-16T00:10:46.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468357+0000", 2024-09-16T00:10:46.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059806809999999998 2024-09-16T00:10:46.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468386+0000", 2024-09-16T00:10:46.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9068999999999999e-05 2024-09-16T00:10:46.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.468441+0000", 2024-09-16T00:10:46.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5444e-05 2024-09-16T00:10:46.836 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.836 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.837 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.837 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10831 3.1a 3:59b8f98c:::100000030df.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-16T00:10:46.837 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:46.837 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 68.052855504999997, 2024-09-16T00:10:46.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014692843000000001, 2024-09-16T00:10:46.837 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.837 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.837 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.837 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.837 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.837 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.837 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10831 2024-09-16T00:10:46.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.838 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:46.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775722+0000", 2024-09-16T00:10:46.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.775723+0000", 2024-09-16T00:10:46.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.367e-06 2024-09-16T00:10:46.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780402+0000", 2024-09-16T00:10:46.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046785769999999997 2024-09-16T00:10:46.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780421+0000", 2024-09-16T00:10:46.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9179000000000001e-05 2024-09-16T00:10:46.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780440+0000", 2024-09-16T00:10:46.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9015e-05 2024-09-16T00:10:46.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780533+0000", 2024-09-16T00:10:46.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3197999999999998e-05 2024-09-16T00:10:46.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780618+0000", 2024-09-16T00:10:46.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4787e-05 2024-09-16T00:10:46.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:46.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.780748+0000", 2024-09-16T00:10:46.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012927199999999999 2024-09-16T00:10:46.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.785049+0000", 2024-09-16T00:10:46.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043015750000000002 2024-09-16T00:10:46.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790312+0000", 2024-09-16T00:10:46.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0052633319999999999 2024-09-16T00:10:46.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790369+0000", 2024-09-16T00:10:46.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6848999999999997e-05 2024-09-16T00:10:46.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.843 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.843 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:38.790415+0000", 2024-09-16T00:10:46.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5692000000000002e-05 2024-09-16T00:10:46.843 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.843 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.843 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.843 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.843 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.843 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10915 3.1a 3:5a6e15a2:::1000000318e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:46.843 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:46.843 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 67.613349584000005, 2024-09-16T00:10:46.843 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014599472, 2024-09-16T00:10:46.843 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.844 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.844 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.844 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.844 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.844 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.844 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10915 2024-09-16T00:10:46.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.844 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.844 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.844 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.844 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:46.844 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.844 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215228+0000", 2024-09-16T00:10:46.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.215230+0000", 2024-09-16T00:10:46.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0090000000000002e-06 2024-09-16T00:10:46.845 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.845 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.845 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.845 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219556+0000", 2024-09-16T00:10:46.845 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043259539999999999 2024-09-16T00:10:46.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219577+0000", 2024-09-16T00:10:46.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1271999999999999e-05 2024-09-16T00:10:46.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219589+0000", 2024-09-16T00:10:46.846 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1766000000000001e-05 2024-09-16T00:10:46.846 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.846 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.846 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.846 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219663+0000", 2024-09-16T00:10:46.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4196999999999995e-05 2024-09-16T00:10:46.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219733+0000", 2024-09-16T00:10:46.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9553000000000003e-05 2024-09-16T00:10:46.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.847 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:46.847 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.219827+0000", 2024-09-16T00:10:46.847 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4395999999999995e-05 2024-09-16T00:10:46.847 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.847 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.225724+0000", 2024-09-16T00:10:46.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058968359999999999 2024-09-16T00:10:46.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229705+0000", 2024-09-16T00:10:46.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003981409 2024-09-16T00:10:46.848 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.848 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.848 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.848 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229772+0000", 2024-09-16T00:10:46.848 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6137999999999996e-05 2024-09-16T00:10:46.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.849 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.849 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.229828+0000", 2024-09-16T00:10:46.849 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5942000000000001e-05 2024-09-16T00:10:46.849 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.849 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.849 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.849 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.849 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.849 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11074 3.14 3:2834b7ee:::10000003131.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:46.849 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:46.849 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.915809460999995, 2024-09-16T00:10:46.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015588350000000001, 2024-09-16T00:10:46.850 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.850 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.850 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.850 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.850 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11074 2024-09-16T00:10:46.850 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.850 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.850 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.850 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.850 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:46.850 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912768+0000", 2024-09-16T00:10:46.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.851 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.912769+0000", 2024-09-16T00:10:46.851 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4600000000000004e-07 2024-09-16T00:10:46.851 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.851 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.851 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918264+0000", 2024-09-16T00:10:46.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054954890000000001 2024-09-16T00:10:46.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918281+0000", 2024-09-16T00:10:46.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6160000000000001e-05 2024-09-16T00:10:46.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.852 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.852 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.852 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918289+0000", 2024-09-16T00:10:46.852 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1650000000000006e-06 2024-09-16T00:10:46.852 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918386+0000", 2024-09-16T00:10:46.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7288999999999996e-05 2024-09-16T00:10:46.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918535+0000", 2024-09-16T00:10:46.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014876200000000001 2024-09-16T00:10:46.853 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.853 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.853 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:46.853 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.918639+0000", 2024-09-16T00:10:46.853 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010470400000000001 2024-09-16T00:10:46.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.924703+0000", 2024-09-16T00:10:46.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0060639759999999996 2024-09-16T00:10:46.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928246+0000", 2024-09-16T00:10:46.854 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0035421720000000001 2024-09-16T00:10:46.854 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.854 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.854 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.854 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928305+0000", 2024-09-16T00:10:46.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9014000000000001e-05 2024-09-16T00:10:46.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.855 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.855 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:39.928357+0000", 2024-09-16T00:10:46.855 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1873000000000003e-05 2024-09-16T00:10:46.855 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.855 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.855 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.855 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.855 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.855 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11111 3.9 3:93301cc4:::10000003210.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-16T00:10:46.855 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:46.855 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 66.739071258999999, 2024-09-16T00:10:46.856 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014336099999999999, 2024-09-16T00:10:46.856 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.856 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.856 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.856 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.856 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11111 2024-09-16T00:10:46.856 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.857 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:46.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089506+0000", 2024-09-16T00:10:46.857 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.857 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.857 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.857 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.857 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.089520+0000", 2024-09-16T00:10:46.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3903000000000001e-05 2024-09-16T00:10:46.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094463+0000", 2024-09-16T00:10:46.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0049423000000000002 2024-09-16T00:10:46.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.858 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094494+0000", 2024-09-16T00:10:46.858 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1029999999999999e-05 2024-09-16T00:10:46.858 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.858 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.858 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094503+0000", 2024-09-16T00:10:46.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0180000000000007e-06 2024-09-16T00:10:46.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094573+0000", 2024-09-16T00:10:46.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0457000000000007e-05 2024-09-16T00:10:46.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.859 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.859 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094737+0000", 2024-09-16T00:10:46.859 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016408699999999999 2024-09-16T00:10:46.859 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.859 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:46.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.094897+0000", 2024-09-16T00:10:46.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015938899999999999 2024-09-16T00:10:46.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.102326+0000", 2024-09-16T00:10:46.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0074291230000000002 2024-09-16T00:10:46.860 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.860 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.860 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.860 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103723+0000", 2024-09-16T00:10:46.860 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001397255 2024-09-16T00:10:46.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103786+0000", 2024-09-16T00:10:46.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3489000000000001e-05 2024-09-16T00:10:46.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.861 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.861 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.861 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:40.103842+0000", 2024-09-16T00:10:46.861 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6048999999999998e-05 2024-09-16T00:10:46.861 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.861 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.861 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.861 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.862 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(client.4638.0:11934 3.1b e87/20 3:d94fa8da:::10000003190.00000000:head v 87'152, mlcod=87'150)", 2024-09-16T00:10:46.862 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:46.862 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 63.138326563, 2024-09-16T00:10:46.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014984934, 2024-09-16T00:10:46.862 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.862 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.862 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.862 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.862 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.862 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.862 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:46.862 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690251+0000", 2024-09-16T00:10:46.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.690252+0000", 2024-09-16T00:10:46.863 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8899999999999998e-07 2024-09-16T00:10:46.863 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.863 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.863 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.863 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696026+0000", 2024-09-16T00:10:46.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057745219999999998 2024-09-16T00:10:46.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696039+0000", 2024-09-16T00:10:46.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2611e-05 2024-09-16T00:10:46.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.864 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.864 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696047+0000", 2024-09-16T00:10:46.864 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5399999999999998e-06 2024-09-16T00:10:46.864 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.864 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696123+0000", 2024-09-16T00:10:46.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6349999999999999e-05 2024-09-16T00:10:46.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.696144+0000", 2024-09-16T00:10:46.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1512999999999999e-05 2024-09-16T00:10:46.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.865 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.865 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.865 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705188+0000", 2024-09-16T00:10:46.865 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090438459999999995 2024-09-16T00:10:46.865 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.866 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.866 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.705236+0000", 2024-09-16T00:10:46.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7663000000000001e-05 2024-09-16T00:10:46.866 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.866 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.866 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.866 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.866 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.866 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11936 3.3 3:c0886fb9:::100000031a1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:46.866 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:46.866 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 63.128181511000001, 2024-09-16T00:10:46.866 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015336379000000001, 2024-09-16T00:10:46.866 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.867 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.867 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.867 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.867 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.867 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.867 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11936 2024-09-16T00:10:46.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.867 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.867 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.867 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:46.867 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.867 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.867 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700396+0000", 2024-09-16T00:10:46.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.700399+0000", 2024-09-16T00:10:46.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6029999999999999e-06 2024-09-16T00:10:46.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.868 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.868 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.868 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704706+0000", 2024-09-16T00:10:46.868 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043070529999999999 2024-09-16T00:10:46.868 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704720+0000", 2024-09-16T00:10:46.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4629e-05 2024-09-16T00:10:46.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704729+0000", 2024-09-16T00:10:46.869 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6009999999999995e-06 2024-09-16T00:10:46.869 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.869 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.869 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.869 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704805+0000", 2024-09-16T00:10:46.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6015999999999999e-05 2024-09-16T00:10:46.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704866+0000", 2024-09-16T00:10:46.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0878999999999998e-05 2024-09-16T00:10:46.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:46.870 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.704990+0000", 2024-09-16T00:10:46.870 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000124058 2024-09-16T00:10:46.870 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.870 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.870 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.714314+0000", 2024-09-16T00:10:46.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009324321 2024-09-16T00:10:46.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715632+0000", 2024-09-16T00:10:46.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001318121 2024-09-16T00:10:46.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.871 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.871 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715687+0000", 2024-09-16T00:10:46.871 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4135999999999998e-05 2024-09-16T00:10:46.871 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.871 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.872 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.872 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.715732+0000", 2024-09-16T00:10:46.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5961999999999998e-05 2024-09-16T00:10:46.872 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.872 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.872 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.872 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.872 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.872 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:11953 3.1a 3:580e2503:::1000000311b.00000000:head [write 0~4194304 [2@2058066]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:46.872 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:46.872 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 63.061842358, 2024-09-16T00:10:46.872 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016299973999999998, 2024-09-16T00:10:46.872 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.873 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.873 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.873 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.873 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 11953 2024-09-16T00:10:46.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.873 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.873 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.873 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:46.873 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.873 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.873 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766735+0000", 2024-09-16T00:10:46.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.874 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.874 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.874 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.766741+0000", 2024-09-16T00:10:46.874 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2939999999999996e-06 2024-09-16T00:10:46.874 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771165+0000", 2024-09-16T00:10:46.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044248229999999996 2024-09-16T00:10:46.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771181+0000", 2024-09-16T00:10:46.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5243e-05 2024-09-16T00:10:46.875 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.875 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.875 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.875 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771188+0000", 2024-09-16T00:10:46.875 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6429999999999995e-06 2024-09-16T00:10:46.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771238+0000", 2024-09-16T00:10:46.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9567999999999997e-05 2024-09-16T00:10:46.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.876 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771293+0000", 2024-09-16T00:10:46.876 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5207000000000002e-05 2024-09-16T00:10:46.876 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.876 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.876 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:46.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.771380+0000", 2024-09-16T00:10:46.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6716000000000001e-05 2024-09-16T00:10:46.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.776193+0000", 2024-09-16T00:10:46.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048133350000000002 2024-09-16T00:10:46.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.877 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.877 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.782931+0000", 2024-09-16T00:10:46.877 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067376550000000004 2024-09-16T00:10:46.877 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.877 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.782987+0000", 2024-09-16T00:10:46.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6443000000000002e-05 2024-09-16T00:10:46.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.878 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.878 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.783035+0000", 2024-09-16T00:10:46.878 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8047000000000003e-05 2024-09-16T00:10:46.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.878 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.878 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.878 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.878 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.878 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12037 3.9 3:928db521:::1000000326d.00000000:head [write 0~4194304 [2@1655200]] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:46.879 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:46.879 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 62.830634750000002, 2024-09-16T00:10:46.879 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017082494, 2024-09-16T00:10:46.879 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.879 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.879 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.879 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.879 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.879 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.879 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12037 2024-09-16T00:10:46.879 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.879 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.879 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:46.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997943+0000", 2024-09-16T00:10:46.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.880 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.880 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.880 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:43.997944+0000", 2024-09-16T00:10:46.880 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.517e-06 2024-09-16T00:10:46.880 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002177+0000", 2024-09-16T00:10:46.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042329300000000002 2024-09-16T00:10:46.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002197+0000", 2024-09-16T00:10:46.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9769e-05 2024-09-16T00:10:46.881 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.881 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.881 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.881 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002207+0000", 2024-09-16T00:10:46.881 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0178e-05 2024-09-16T00:10:46.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002266+0000", 2024-09-16T00:10:46.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9215999999999997e-05 2024-09-16T00:10:46.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002340+0000", 2024-09-16T00:10:46.882 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3798000000000001e-05 2024-09-16T00:10:46.882 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.882 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.882 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:46.882 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.002454+0000", 2024-09-16T00:10:46.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113551 2024-09-16T00:10:46.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.006270+0000", 2024-09-16T00:10:46.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038163839999999999 2024-09-16T00:10:46.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.883 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.014957+0000", 2024-09-16T00:10:46.883 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086869119999999998 2024-09-16T00:10:46.883 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.883 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.883 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.014996+0000", 2024-09-16T00:10:46.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8878000000000003e-05 2024-09-16T00:10:46.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.884 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.884 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.015025+0000", 2024-09-16T00:10:46.884 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9360999999999999e-05 2024-09-16T00:10:46.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.884 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.884 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.884 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.884 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.884 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12245 3.1a 3:599a1cd1:::10000003252.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:46.885 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:46.885 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 62.045937332999998, 2024-09-16T00:10:46.885 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.017131203000000001, 2024-09-16T00:10:46.885 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.885 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.885 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.885 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.885 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.885 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.885 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12245 2024-09-16T00:10:46.885 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.885 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.885 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.885 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:46.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782640+0000", 2024-09-16T00:10:46.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.886 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.886 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.886 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.782642+0000", 2024-09-16T00:10:46.886 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.311e-06 2024-09-16T00:10:46.886 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786551+0000", 2024-09-16T00:10:46.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039095229999999998 2024-09-16T00:10:46.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786566+0000", 2024-09-16T00:10:46.887 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4724e-05 2024-09-16T00:10:46.887 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.887 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.887 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.887 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786575+0000", 2024-09-16T00:10:46.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8449999999999992e-06 2024-09-16T00:10:46.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786636+0000", 2024-09-16T00:10:46.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.143e-05 2024-09-16T00:10:46.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.888 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786699+0000", 2024-09-16T00:10:46.888 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3218000000000004e-05 2024-09-16T00:10:46.888 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.888 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.888 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:46.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.786796+0000", 2024-09-16T00:10:46.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6347000000000004e-05 2024-09-16T00:10:46.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.792369+0000", 2024-09-16T00:10:46.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055729780000000001 2024-09-16T00:10:46.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.889 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.889 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799706+0000", 2024-09-16T00:10:46.889 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073374349999999998 2024-09-16T00:10:46.889 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.889 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799739+0000", 2024-09-16T00:10:46.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2833999999999998e-05 2024-09-16T00:10:46.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.890 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.890 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.799771+0000", 2024-09-16T00:10:46.890 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2558000000000002e-05 2024-09-16T00:10:46.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.890 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.890 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.890 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.890 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.891 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:12292 3.3 3:c00779fd:::1000000320e.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-16T00:10:46.891 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:46.891 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 61.931547236, 2024-09-16T00:10:46.891 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013828353999999999, 2024-09-16T00:10:46.891 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.891 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.891 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.891 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.891 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 12292 2024-09-16T00:10:46.891 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.891 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:46.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897030+0000", 2024-09-16T00:10:46.892 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.892 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.892 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.892 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.892 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.897039+0000", 2024-09-16T00:10:46.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3299999999999999e-06 2024-09-16T00:10:46.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900830+0000", 2024-09-16T00:10:46.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0037915340000000001 2024-09-16T00:10:46.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.893 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900850+0000", 2024-09-16T00:10:46.893 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9400000000000001e-05 2024-09-16T00:10:46.893 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.893 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.893 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900860+0000", 2024-09-16T00:10:46.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0609e-05 2024-09-16T00:10:46.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.900928+0000", 2024-09-16T00:10:46.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7570000000000005e-05 2024-09-16T00:10:46.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.894 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.894 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.901027+0000", 2024-09-16T00:10:46.894 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9456000000000005e-05 2024-09-16T00:10:46.894 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.894 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:46.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.901168+0000", 2024-09-16T00:10:46.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014082799999999999 2024-09-16T00:10:46.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.906325+0000", 2024-09-16T00:10:46.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005156702 2024-09-16T00:10:46.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.895 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.895 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.895 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910796+0000", 2024-09-16T00:10:46.895 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044714350000000002 2024-09-16T00:10:46.895 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910829+0000", 2024-09-16T00:10:46.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3229000000000003e-05 2024-09-16T00:10:46.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.896 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.896 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.896 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:44.910859+0000", 2024-09-16T00:10:46.896 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9261e-05 2024-09-16T00:10:46.896 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.896 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.896 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.896 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.897 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.897 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13326 3.9 3:92f796cf:::1000000350f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-16T00:10:46.897 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:46.897 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 58.871856954000002, 2024-09-16T00:10:46.897 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015673229, 2024-09-16T00:10:46.897 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.897 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.897 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.897 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.897 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13326 2024-09-16T00:10:46.897 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.897 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:46.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956721+0000", 2024-09-16T00:10:46.898 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.898 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.898 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.898 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.898 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.956722+0000", 2024-09-16T00:10:46.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3200000000000001e-06 2024-09-16T00:10:46.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960803+0000", 2024-09-16T00:10:46.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040807029999999998 2024-09-16T00:10:46.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.899 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960820+0000", 2024-09-16T00:10:46.899 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7478000000000001e-05 2024-09-16T00:10:46.899 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.899 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.899 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960830+0000", 2024-09-16T00:10:46.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5179999999999993e-06 2024-09-16T00:10:46.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960903+0000", 2024-09-16T00:10:46.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3057000000000002e-05 2024-09-16T00:10:46.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.900 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.900 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.960970+0000", 2024-09-16T00:10:46.900 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7063000000000005e-05 2024-09-16T00:10:46.900 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.900 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:46.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.961067+0000", 2024-09-16T00:10:46.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6906999999999999e-05 2024-09-16T00:10:46.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.967945+0000", 2024-09-16T00:10:46.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0068787359999999999 2024-09-16T00:10:46.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.901 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.901 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.901 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972328+0000", 2024-09-16T00:10:46.901 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043826170000000001 2024-09-16T00:10:46.901 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972363+0000", 2024-09-16T00:10:46.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4486000000000002e-05 2024-09-16T00:10:46.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.902 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.902 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:47.972394+0000", 2024-09-16T00:10:46.902 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.1343999999999997e-05 2024-09-16T00:10:46.902 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.902 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.902 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.902 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.902 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.902 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13429 3.1a 3:58dcd24a:::100000036d5.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:46.903 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:46.903 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 58.517286937999998, 2024-09-16T00:10:46.903 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014237671, 2024-09-16T00:10:46.903 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.903 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.903 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.903 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.903 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.903 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.903 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13429 2024-09-16T00:10:46.903 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.903 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.903 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:46.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311291+0000", 2024-09-16T00:10:46.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.904 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.904 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.904 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.311292+0000", 2024-09-16T00:10:46.904 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4890000000000001e-06 2024-09-16T00:10:46.904 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316378+0000", 2024-09-16T00:10:46.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050859290000000003 2024-09-16T00:10:46.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316399+0000", 2024-09-16T00:10:46.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1345e-05 2024-09-16T00:10:46.905 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.905 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.905 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.905 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316410+0000", 2024-09-16T00:10:46.905 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0295999999999999e-05 2024-09-16T00:10:46.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316493+0000", 2024-09-16T00:10:46.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3533000000000001e-05 2024-09-16T00:10:46.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316622+0000", 2024-09-16T00:10:46.906 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012887900000000001 2024-09-16T00:10:46.906 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.906 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.906 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:46.906 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.316725+0000", 2024-09-16T00:10:46.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010295599999999999 2024-09-16T00:10:46.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.320948+0000", 2024-09-16T00:10:46.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042229820000000001 2024-09-16T00:10:46.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.907 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325415+0000", 2024-09-16T00:10:46.907 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044673849999999999 2024-09-16T00:10:46.907 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.907 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.907 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325487+0000", 2024-09-16T00:10:46.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1087999999999994e-05 2024-09-16T00:10:46.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.908 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.908 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.325528+0000", 2024-09-16T00:10:46.908 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.1789000000000003e-05 2024-09-16T00:10:46.908 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.908 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.908 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.908 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.908 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.908 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13527 3.3 3:c0886fb9:::100000031a1.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:46.908 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:46.909 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 58.043126848999997, 2024-09-16T00:10:46.909 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015187236, 2024-09-16T00:10:46.909 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.909 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.909 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.909 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.909 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.909 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.909 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13527 2024-09-16T00:10:46.909 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.909 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.909 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.909 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.909 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:46.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785451+0000", 2024-09-16T00:10:46.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.910 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.785453+0000", 2024-09-16T00:10:46.910 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.1689999999999999e-06 2024-09-16T00:10:46.910 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.910 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.910 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791191+0000", 2024-09-16T00:10:46.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057378339999999998 2024-09-16T00:10:46.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791217+0000", 2024-09-16T00:10:46.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6480999999999999e-05 2024-09-16T00:10:46.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.911 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.911 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791233+0000", 2024-09-16T00:10:46.911 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5841e-05 2024-09-16T00:10:46.911 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.911 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791319+0000", 2024-09-16T00:10:46.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5527999999999998e-05 2024-09-16T00:10:46.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791413+0000", 2024-09-16T00:10:46.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4685000000000006e-05 2024-09-16T00:10:46.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.912 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.912 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:46.912 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.791563+0000", 2024-09-16T00:10:46.912 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000149624 2024-09-16T00:10:46.912 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.795749+0000", 2024-09-16T00:10:46.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004186286 2024-09-16T00:10:46.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800577+0000", 2024-09-16T00:10:46.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048273040000000001 2024-09-16T00:10:46.913 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.913 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.913 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.913 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800611+0000", 2024-09-16T00:10:46.913 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4270000000000002e-05 2024-09-16T00:10:46.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.914 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.914 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:48.800638+0000", 2024-09-16T00:10:46.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.7214000000000001e-05 2024-09-16T00:10:46.914 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.914 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.914 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.914 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.914 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.914 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13724 3.9 3:90613e6c:::100000034c4.00000001:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:46.914 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:46.914 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 57.373055104000002, 2024-09-16T00:10:46.914 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014039800999999999, 2024-09-16T00:10:46.915 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.915 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.915 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.915 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.915 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13724 2024-09-16T00:10:46.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.915 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.915 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.915 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.915 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:46.915 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.915 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455523+0000", 2024-09-16T00:10:46.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.455524+0000", 2024-09-16T00:10:46.916 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.437e-06 2024-09-16T00:10:46.916 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.916 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.916 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.916 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459592+0000", 2024-09-16T00:10:46.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040678729999999996 2024-09-16T00:10:46.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459610+0000", 2024-09-16T00:10:46.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7688000000000001e-05 2024-09-16T00:10:46.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.917 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459617+0000", 2024-09-16T00:10:46.917 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.374e-06 2024-09-16T00:10:46.917 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.917 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.917 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459678+0000", 2024-09-16T00:10:46.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1346999999999994e-05 2024-09-16T00:10:46.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459824+0000", 2024-09-16T00:10:46.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014586699999999999 2024-09-16T00:10:46.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.918 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:46.918 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.459908+0000", 2024-09-16T00:10:46.918 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3521000000000002e-05 2024-09-16T00:10:46.918 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.918 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.464227+0000", 2024-09-16T00:10:46.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043197060000000004 2024-09-16T00:10:46.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469499+0000", 2024-09-16T00:10:46.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.005271838 2024-09-16T00:10:46.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.919 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.919 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.919 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469534+0000", 2024-09-16T00:10:46.919 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4712000000000002e-05 2024-09-16T00:10:46.919 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.920 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.920 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.469562+0000", 2024-09-16T00:10:46.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.8438000000000001e-05 2024-09-16T00:10:46.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.920 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.920 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.920 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.920 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.920 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:13741 3.3 3:c1377c76:::10000003664.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T00:10:46.920 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:46.920 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 57.325885159000002, 2024-09-16T00:10:46.920 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.016142843, 2024-09-16T00:10:46.921 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.921 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.921 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.921 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.921 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 13741 2024-09-16T00:10:46.921 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.921 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.921 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.921 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.921 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:46.921 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502692+0000", 2024-09-16T00:10:46.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.502694+0000", 2024-09-16T00:10:46.922 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0449999999999999e-06 2024-09-16T00:10:46.922 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.922 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.922 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.922 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507229+0000", 2024-09-16T00:10:46.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045357410000000003 2024-09-16T00:10:46.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507248+0000", 2024-09-16T00:10:46.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9127000000000002e-05 2024-09-16T00:10:46.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.923 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507259+0000", 2024-09-16T00:10:46.923 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0448999999999999e-05 2024-09-16T00:10:46.923 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.923 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.923 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507357+0000", 2024-09-16T00:10:46.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.8400000000000007e-05 2024-09-16T00:10:46.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507514+0000", 2024-09-16T00:10:46.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000157195 2024-09-16T00:10:46.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.924 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:46.924 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.507617+0000", 2024-09-16T00:10:46.924 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010262199999999999 2024-09-16T00:10:46.924 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.924 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.514047+0000", 2024-09-16T00:10:46.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.006430018 2024-09-16T00:10:46.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518743+0000", 2024-09-16T00:10:46.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046963040000000001 2024-09-16T00:10:46.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.925 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.925 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.925 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518796+0000", 2024-09-16T00:10:46.925 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2305000000000002e-05 2024-09-16T00:10:46.925 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.926 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.926 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:49.518835+0000", 2024-09-16T00:10:46.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9637e-05 2024-09-16T00:10:46.926 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.926 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.926 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.926 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.926 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.926 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14633 3.9 3:9170e82c:::100000035d6.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:46.926 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:46.926 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.206504903000003, 2024-09-16T00:10:46.926 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014466105, 2024-09-16T00:10:46.927 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.927 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.927 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.927 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.927 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.927 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.927 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14633 2024-09-16T00:10:46.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.927 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.927 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.927 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.927 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:46.927 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.927 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622073+0000", 2024-09-16T00:10:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.622074+0000", 2024-09-16T00:10:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.761e-06 2024-09-16T00:10:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626712+0000", 2024-09-16T00:10:46.928 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004637263 2024-09-16T00:10:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626724+0000", 2024-09-16T00:10:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2479e-05 2024-09-16T00:10:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626730+0000", 2024-09-16T00:10:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7309999999999998e-06 2024-09-16T00:10:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.929 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626791+0000", 2024-09-16T00:10:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0854999999999999e-05 2024-09-16T00:10:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626857+0000", 2024-09-16T00:10:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6549000000000003e-05 2024-09-16T00:10:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.626943+0000", 2024-09-16T00:10:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6141000000000001e-05 2024-09-16T00:10:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.930 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.630854+0000", 2024-09-16T00:10:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039100790000000003 2024-09-16T00:10:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636468+0000", 2024-09-16T00:10:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0056140570000000004 2024-09-16T00:10:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636500+0000", 2024-09-16T00:10:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.2441999999999999e-05 2024-09-16T00:10:46.931 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.636539+0000", 2024-09-16T00:10:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8748000000000002e-05 2024-09-16T00:10:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14667 3.9 3:9379a4af:::10000003352.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 54.031492897, 2024-09-16T00:10:46.932 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014580585, 2024-09-16T00:10:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14667 2024-09-16T00:10:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.933 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797085+0000", 2024-09-16T00:10:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.797086+0000", 2024-09-16T00:10:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.362e-06 2024-09-16T00:10:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801415+0000", 2024-09-16T00:10:46.934 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0043288449999999996 2024-09-16T00:10:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801435+0000", 2024-09-16T00:10:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0305999999999998e-05 2024-09-16T00:10:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801442+0000", 2024-09-16T00:10:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6290000000000004e-06 2024-09-16T00:10:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.935 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801497+0000", 2024-09-16T00:10:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5111999999999997e-05 2024-09-16T00:10:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801639+0000", 2024-09-16T00:10:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014168000000000001 2024-09-16T00:10:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 2", 2024-09-16T00:10:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.801788+0000", 2024-09-16T00:10:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014963300000000001 2024-09-16T00:10:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.936 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.805407+0000", 2024-09-16T00:10:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0036186119999999998 2024-09-16T00:10:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811584+0000", 2024-09-16T00:10:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061772889999999999 2024-09-16T00:10:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811635+0000", 2024-09-16T00:10:46.937 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0453e-05 2024-09-16T00:10:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:52.811665+0000", 2024-09-16T00:10:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.0664000000000003e-05 2024-09-16T00:10:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:14951 3.1a 3:59aee80a:::1000000394f.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T00:10:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:46.938 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.861455002, 2024-09-16T00:10:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.015993941000000001, 2024-09-16T00:10:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 14951 2024-09-16T00:10:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:46.939 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967123+0000", 2024-09-16T00:10:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.967124+0000", 2024-09-16T00:10:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.86e-06 2024-09-16T00:10:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.940 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971258+0000", 2024-09-16T00:10:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041334780000000003 2024-09-16T00:10:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971274+0000", 2024-09-16T00:10:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.6201e-05 2024-09-16T00:10:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971284+0000", 2024-09-16T00:10:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4779999999999997e-06 2024-09-16T00:10:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971384+0000", 2024-09-16T00:10:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9878999999999997e-05 2024-09-16T00:10:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971518+0000", 2024-09-16T00:10:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013459400000000001 2024-09-16T00:10:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 1", 2024-09-16T00:10:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.971604+0000", 2024-09-16T00:10:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6215e-05 2024-09-16T00:10:46.942 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.976673+0000", 2024-09-16T00:10:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0050685920000000002 2024-09-16T00:10:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983043+0000", 2024-09-16T00:10:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0063698410000000002 2024-09-16T00:10:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983082+0000", 2024-09-16T00:10:46.943 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9261999999999998e-05 2024-09-16T00:10:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:53.983117+0000", 2024-09-16T00:10:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4541e-05 2024-09-16T00:10:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:15152 3.3 3:c388c7e2:::100000032ec.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e91)", 2024-09-16T00:10:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 52.100908892, 2024-09-16T00:10:46.944 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.014603532000000001, 2024-09-16T00:10:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 15152 2024-09-16T00:10:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.945 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727669+0000", 2024-09-16T00:10:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.727674+0000", 2024-09-16T00:10:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7699999999999998e-06 2024-09-16T00:10:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732274+0000", 2024-09-16T00:10:46.946 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045994299999999998 2024-09-16T00:10:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732286+0000", 2024-09-16T00:10:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2016e-05 2024-09-16T00:10:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732294+0000", 2024-09-16T00:10:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7049999999999999e-06 2024-09-16T00:10:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:46.947 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732359+0000", 2024-09-16T00:10:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5296000000000006e-05 2024-09-16T00:10:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732501+0000", 2024-09-16T00:10:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000141731 2024-09-16T00:10:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 0", 2024-09-16T00:10:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.732629+0000", 2024-09-16T00:10:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012841299999999999 2024-09-16T00:10:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.948 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.737511+0000", 2024-09-16T00:10:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048822520000000001 2024-09-16T00:10:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742172+0000", 2024-09-16T00:10:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046610610000000002 2024-09-16T00:10:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742227+0000", 2024-09-16T00:10:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4497999999999999e-05 2024-09-16T00:10:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:46.949 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:46.950 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:46.950 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:54.742272+0000", 2024-09-16T00:10:46.950 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.5359999999999999e-05 2024-09-16T00:10:46.950 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.950 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.950 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.950 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:46.950 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:46.950 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:46.951 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-16T00:10:47.057 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:47.057 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:47.058 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:47.058 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:47.066 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-16T00:10:47.169 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:47.169 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:47.169 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:47.169 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:47.169 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:47.178 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-16T00:10:47.284 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:47.284 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:47.284 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:47.284 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:47.285 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.285 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(client.4638.0:10658 3.1c 3:38c4c24e:::100000030bb.00000000:head [write 0~4194304] snapc 1=[] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-09-16T00:10:47.285 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:47.285 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 71.491104351999994, 2024-09-16T00:10:47.285 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021232445999999999, 2024-09-16T00:10:47.285 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.285 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.285 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client_info": { 2024-09-16T00:10:47.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client": "client.4638", 2024-09-16T00:10:47.285 INFO:teuthology.orchestra.run.smithi164.stdout: "client_addr": "192.168.0.1:0/3876595971", 2024-09-16T00:10:47.285 INFO:teuthology.orchestra.run.smithi164.stdout: "tid": 10658 2024-09-16T00:10:47.285 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.285 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:47.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791396+0000", 2024-09-16T00:10:47.286 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.286 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.286 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.286 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.286 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.791398+0000", 2024-09-16T00:10:47.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3880000000000001e-06 2024-09-16T00:10:47.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800951+0000", 2024-09-16T00:10:47.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095532110000000007 2024-09-16T00:10:47.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800966+0000", 2024-09-16T00:10:47.287 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5058999999999999e-05 2024-09-16T00:10:47.287 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.287 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.287 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.287 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.800974+0000", 2024-09-16T00:10:47.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1240000000000005e-06 2024-09-16T00:10:47.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801127+0000", 2024-09-16T00:10:47.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000153103 2024-09-16T00:10:47.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.288 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.288 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801207+0000", 2024-09-16T00:10:47.288 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9569999999999999e-05 2024-09-16T00:10:47.288 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.288 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.801322+0000", 2024-09-16T00:10:47.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115358 2024-09-16T00:10:47.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.805760+0000", 2024-09-16T00:10:47.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044379409999999999 2024-09-16T00:10:47.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.289 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.289 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.289 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812528+0000", 2024-09-16T00:10:47.289 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067678249999999999 2024-09-16T00:10:47.289 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812586+0000", 2024-09-16T00:10:47.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8164000000000001e-05 2024-09-16T00:10:47.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.290 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.290 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:35.812629+0000", 2024-09-16T00:10:47.290 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2703000000000001e-05 2024-09-16T00:10:47.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.290 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.290 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.290 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.290 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.290 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4122 2.18 2:1cc859d9:::200.00000012:head [write 798310~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.291 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:47.291 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.623564672000001, 2024-09-16T00:10:47.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019267580999999999, 2024-09-16T00:10:47.291 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.291 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.291 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.291 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.291 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:47.291 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.291 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.291 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.291 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658936+0000", 2024-09-16T00:10:47.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658937+0000", 2024-09-16T00:10:47.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7700000000000004e-07 2024-09-16T00:10:47.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.292 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.292 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658942+0000", 2024-09-16T00:10:47.292 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.48e-06 2024-09-16T00:10:47.292 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.292 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658952+0000", 2024-09-16T00:10:47.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9469999999999998e-06 2024-09-16T00:10:47.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.658959+0000", 2024-09-16T00:10:47.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4400000000000002e-06 2024-09-16T00:10:47.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.293 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.293 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.293 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670507+0000", 2024-09-16T00:10:47.293 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011548159000000001 2024-09-16T00:10:47.293 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670579+0000", 2024-09-16T00:10:47.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2082999999999994e-05 2024-09-16T00:10:47.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.670686+0000", 2024-09-16T00:10:47.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107069 2024-09-16T00:10:47.294 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.294 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.294 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.294 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678001+0000", 2024-09-16T00:10:47.294 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0073148780000000004 2024-09-16T00:10:47.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678104+0000", 2024-09-16T00:10:47.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010290599999999999 2024-09-16T00:10:47.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678163+0000", 2024-09-16T00:10:47.295 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8859e-05 2024-09-16T00:10:47.295 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.295 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.295 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.295 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678204+0000", 2024-09-16T00:10:47.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0982999999999997e-05 2024-09-16T00:10:47.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.296 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.296 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.296 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.296 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.296 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4123 2.18 2:1cc859d9:::200.00000012:head [write 799709~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.296 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:47.296 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.622995490999998, 2024-09-16T00:10:47.296 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018857569000000001, 2024-09-16T00:10:47.296 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.296 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.296 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.297 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:47.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659505+0000", 2024-09-16T00:10:47.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.297 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.297 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.297 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.297 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659506+0000", 2024-09-16T00:10:47.297 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.78e-07 2024-09-16T00:10:47.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659512+0000", 2024-09-16T00:10:47.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0100000000000001e-06 2024-09-16T00:10:47.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.298 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659523+0000", 2024-09-16T00:10:47.298 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1270999999999999e-05 2024-09-16T00:10:47.298 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.298 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.298 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.659531+0000", 2024-09-16T00:10:47.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2010000000000002e-06 2024-09-16T00:10:47.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671124+0000", 2024-09-16T00:10:47.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011593696000000001 2024-09-16T00:10:47.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.299 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.299 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671171+0000", 2024-09-16T00:10:47.299 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6974999999999997e-05 2024-09-16T00:10:47.299 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.299 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.671243+0000", 2024-09-16T00:10:47.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2131000000000004e-05 2024-09-16T00:10:47.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678008+0000", 2024-09-16T00:10:47.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0067642329999999997 2024-09-16T00:10:47.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.300 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.300 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.300 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678261+0000", 2024-09-16T00:10:47.300 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00025378999999999999 2024-09-16T00:10:47.300 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678320+0000", 2024-09-16T00:10:47.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8452000000000002e-05 2024-09-16T00:10:47.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.301 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.301 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678363+0000", 2024-09-16T00:10:47.301 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2831999999999999e-05 2024-09-16T00:10:47.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.301 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.301 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.301 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.301 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.302 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4126 2.18 2:1cc859d9:::200.00000012:head [write 806443~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.302 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:47.302 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.621597704999999, 2024-09-16T00:10:47.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019331159000000001, 2024-09-16T00:10:47.302 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.302 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.302 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.302 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.302 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.302 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:47.302 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.302 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.302 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660903+0000", 2024-09-16T00:10:47.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660904+0000", 2024-09-16T00:10:47.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.6800000000000009e-07 2024-09-16T00:10:47.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.303 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.303 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.303 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660909+0000", 2024-09-16T00:10:47.303 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2730000000000002e-06 2024-09-16T00:10:47.303 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660919+0000", 2024-09-16T00:10:47.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4960000000000002e-06 2024-09-16T00:10:47.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.660923+0000", 2024-09-16T00:10:47.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6260000000000003e-06 2024-09-16T00:10:47.304 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.304 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.304 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.304 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672698+0000", 2024-09-16T00:10:47.304 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011774573999999999 2024-09-16T00:10:47.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672784+0000", 2024-09-16T00:10:47.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6074000000000003e-05 2024-09-16T00:10:47.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.672897+0000", 2024-09-16T00:10:47.305 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011267199999999999 2024-09-16T00:10:47.305 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.305 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.305 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.305 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678026+0000", 2024-09-16T00:10:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051297089999999997 2024-09-16T00:10:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680147+0000", 2024-09-16T00:10:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021209409999999999 2024-09-16T00:10:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680197+0000", 2024-09-16T00:10:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0068000000000002e-05 2024-09-16T00:10:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.306 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.307 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680234+0000", 2024-09-16T00:10:47.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6758000000000003e-05 2024-09-16T00:10:47.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.307 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.307 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.307 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.307 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.307 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4127 2.18 2:1cc859d9:::200.00000012:head [write 808688~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.307 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:47.307 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.621315856000002, 2024-09-16T00:10:47.307 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019226493000000001, 2024-09-16T00:10:47.307 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.307 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.307 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.308 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:47.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661185+0000", 2024-09-16T00:10:47.308 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.308 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.308 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.308 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.308 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661186+0000", 2024-09-16T00:10:47.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2399999999999996e-07 2024-09-16T00:10:47.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661192+0000", 2024-09-16T00:10:47.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9259999999999997e-06 2024-09-16T00:10:47.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.309 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661203+0000", 2024-09-16T00:10:47.309 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1174e-05 2024-09-16T00:10:47.309 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.309 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.309 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661213+0000", 2024-09-16T00:10:47.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0579e-05 2024-09-16T00:10:47.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673408+0000", 2024-09-16T00:10:47.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012194442 2024-09-16T00:10:47.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.310 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673482+0000", 2024-09-16T00:10:47.310 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4258000000000006e-05 2024-09-16T00:10:47.310 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.310 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.310 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.673592+0000", 2024-09-16T00:10:47.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109733 2024-09-16T00:10:47.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678033+0000", 2024-09-16T00:10:47.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0044411629999999997 2024-09-16T00:10:47.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.311 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.311 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.311 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680287+0000", 2024-09-16T00:10:47.311 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0022538469999999998 2024-09-16T00:10:47.311 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680367+0000", 2024-09-16T00:10:47.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9808999999999994e-05 2024-09-16T00:10:47.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.312 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.312 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680411+0000", 2024-09-16T00:10:47.312 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4638000000000002e-05 2024-09-16T00:10:47.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.312 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.312 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.312 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.312 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.313 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4128 2.18 2:1cc859d9:::200.00000012:head [write 810933~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.313 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:47.313 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.620896725000001, 2024-09-16T00:10:47.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018982046999999998, 2024-09-16T00:10:47.313 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.313 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.313 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.313 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.313 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.313 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:47.313 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.313 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.313 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661604+0000", 2024-09-16T00:10:47.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661605+0000", 2024-09-16T00:10:47.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.031e-06 2024-09-16T00:10:47.314 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.314 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.314 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.314 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661610+0000", 2024-09-16T00:10:47.314 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5010000000000004e-06 2024-09-16T00:10:47.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661623+0000", 2024-09-16T00:10:47.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2286000000000001e-05 2024-09-16T00:10:47.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661630+0000", 2024-09-16T00:10:47.315 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3930000000000002e-06 2024-09-16T00:10:47.315 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.315 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.315 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.315 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674011+0000", 2024-09-16T00:10:47.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012380923 2024-09-16T00:10:47.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674092+0000", 2024-09-16T00:10:47.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1267999999999995e-05 2024-09-16T00:10:47.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.316 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674216+0000", 2024-09-16T00:10:47.316 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012331700000000001 2024-09-16T00:10:47.316 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.316 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.316 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678039+0000", 2024-09-16T00:10:47.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038232489999999999 2024-09-16T00:10:47.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680493+0000", 2024-09-16T00:10:47.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0024543429999999999 2024-09-16T00:10:47.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.317 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.317 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680546+0000", 2024-09-16T00:10:47.317 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3239000000000003e-05 2024-09-16T00:10:47.317 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.317 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.318 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.318 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680586+0000", 2024-09-16T00:10:47.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9496999999999998e-05 2024-09-16T00:10:47.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.318 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.318 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.318 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.318 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.318 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4129 2.18 2:1cc859d9:::200.00000012:head [write 813178~2245 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.318 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:47.318 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.620539573999999, 2024-09-16T00:10:47.318 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.018785932000000002, 2024-09-16T00:10:47.318 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.318 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.319 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.319 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:47.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.319 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661961+0000", 2024-09-16T00:10:47.319 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.319 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.319 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.319 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661962+0000", 2024-09-16T00:10:47.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8000000000000004e-07 2024-09-16T00:10:47.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.661966+0000", 2024-09-16T00:10:47.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2300000000000002e-06 2024-09-16T00:10:47.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.320 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.320 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.320 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.662015+0000", 2024-09-16T00:10:47.320 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8364e-05 2024-09-16T00:10:47.320 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.662023+0000", 2024-09-16T00:10:47.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2090000000000005e-06 2024-09-16T00:10:47.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674744+0000", 2024-09-16T00:10:47.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012721423000000001 2024-09-16T00:10:47.321 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.321 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.321 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.321 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674825+0000", 2024-09-16T00:10:47.321 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1036000000000001e-05 2024-09-16T00:10:47.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.674946+0000", 2024-09-16T00:10:47.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000120727 2024-09-16T00:10:47.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.678045+0000", 2024-09-16T00:10:47.322 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0030993280000000002 2024-09-16T00:10:47.322 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.322 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.322 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.322 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680646+0000", 2024-09-16T00:10:47.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0026005300000000002 2024-09-16T00:10:47.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680705+0000", 2024-09-16T00:10:47.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9141000000000002e-05 2024-09-16T00:10:47.323 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.323 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.323 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.323 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.680747+0000", 2024-09-16T00:10:47.323 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2064000000000002e-05 2024-09-16T00:10:47.323 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.323 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.324 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.324 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.324 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4184 2.18 2:1cc859d9:::200.00000012:head [write 927308~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.324 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:47.324 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.576490978999999, 2024-09-16T00:10:47.324 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023503514999999999, 2024-09-16T00:10:47.324 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.324 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.324 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.324 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.324 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.324 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.324 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706010+0000", 2024-09-16T00:10:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706011+0000", 2024-09-16T00:10:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8400000000000003e-07 2024-09-16T00:10:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.325 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706016+0000", 2024-09-16T00:10:47.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7479999999999999e-06 2024-09-16T00:10:47.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706028+0000", 2024-09-16T00:10:47.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1477e-05 2024-09-16T00:10:47.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.326 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.326 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.706035+0000", 2024-09-16T00:10:47.326 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1509999999999998e-06 2024-09-16T00:10:47.326 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.326 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.719820+0000", 2024-09-16T00:10:47.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013785168 2024-09-16T00:10:47.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.719888+0000", 2024-09-16T00:10:47.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7631000000000004e-05 2024-09-16T00:10:47.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.327 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.327 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.327 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720003+0000", 2024-09-16T00:10:47.327 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000114849 2024-09-16T00:10:47.327 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725150+0000", 2024-09-16T00:10:47.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0051469970000000004 2024-09-16T00:10:47.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729386+0000", 2024-09-16T00:10:47.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042368529999999996 2024-09-16T00:10:47.328 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.328 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.328 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.328 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729461+0000", 2024-09-16T00:10:47.328 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4935e-05 2024-09-16T00:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.729513+0000", 2024-09-16T00:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1822e-05 2024-09-16T00:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4185 2.18 2:1cc859d9:::200.00000012:head [write 929217~3264 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.575484799000002, 2024-09-16T00:10:47.329 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023998430000000001, 2024-09-16T00:10:47.330 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.330 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.330 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.330 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:47.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.330 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.330 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.330 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707016+0000", 2024-09-16T00:10:47.330 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.330 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707017+0000", 2024-09-16T00:10:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9999999999999996e-07 2024-09-16T00:10:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707023+0000", 2024-09-16T00:10:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1789999999999996e-06 2024-09-16T00:10:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707035+0000", 2024-09-16T00:10:47.331 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2106000000000001e-05 2024-09-16T00:10:47.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707042+0000", 2024-09-16T00:10:47.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2960000000000001e-06 2024-09-16T00:10:47.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720503+0000", 2024-09-16T00:10:47.332 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013460952999999999 2024-09-16T00:10:47.332 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.332 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.332 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.332 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720583+0000", 2024-09-16T00:10:47.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0171000000000002e-05 2024-09-16T00:10:47.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.720668+0000", 2024-09-16T00:10:47.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.4128999999999994e-05 2024-09-16T00:10:47.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.333 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725193+0000", 2024-09-16T00:10:47.333 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0045257700000000001 2024-09-16T00:10:47.333 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.333 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.333 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.730924+0000", 2024-09-16T00:10:47.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057307690000000001 2024-09-16T00:10:47.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.730972+0000", 2024-09-16T00:10:47.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7756000000000002e-05 2024-09-16T00:10:47.334 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.334 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.334 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.334 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731014+0000", 2024-09-16T00:10:47.334 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2401000000000003e-05 2024-09-16T00:10:47.334 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.334 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.335 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.335 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.335 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4186 2.18 2:1cc859d9:::200.00000012:head [write 932481~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.335 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:47.335 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.574896641999999, 2024-09-16T00:10:47.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.023544934999999999, 2024-09-16T00:10:47.335 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.335 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.335 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.335 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.335 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.335 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.335 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:47.335 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707604+0000", 2024-09-16T00:10:47.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707605+0000", 2024-09-16T00:10:47.336 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.1500000000000003e-07 2024-09-16T00:10:47.336 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.336 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.336 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.336 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707610+0000", 2024-09-16T00:10:47.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7049999999999998e-06 2024-09-16T00:10:47.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707618+0000", 2024-09-16T00:10:47.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6149999999999997e-06 2024-09-16T00:10:47.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.337 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.707623+0000", 2024-09-16T00:10:47.337 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0660000000000003e-06 2024-09-16T00:10:47.337 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.337 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.337 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721143+0000", 2024-09-16T00:10:47.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.01351976 2024-09-16T00:10:47.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721221+0000", 2024-09-16T00:10:47.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7918999999999997e-05 2024-09-16T00:10:47.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.338 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.338 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721335+0000", 2024-09-16T00:10:47.338 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000113781 2024-09-16T00:10:47.338 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.338 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725236+0000", 2024-09-16T00:10:47.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0039010379999999999 2024-09-16T00:10:47.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731025+0000", 2024-09-16T00:10:47.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057896659999999997 2024-09-16T00:10:47.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.339 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.339 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.339 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731091+0000", 2024-09-16T00:10:47.339 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.567e-05 2024-09-16T00:10:47.339 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.340 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.340 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731149+0000", 2024-09-16T00:10:47.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7800000000000002e-05 2024-09-16T00:10:47.340 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.340 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.340 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.340 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.340 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.340 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4187 2.18 2:1cc859d9:::200.00000012:head [write 934390~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.340 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:47.340 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.574044512999997, 2024-09-16T00:10:47.340 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022820340000000001, 2024-09-16T00:10:47.341 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.341 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.341 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.341 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:47.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.341 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.341 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.341 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708456+0000", 2024-09-16T00:10:47.341 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.341 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708457+0000", 2024-09-16T00:10:47.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6899999999999996e-07 2024-09-16T00:10:47.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708462+0000", 2024-09-16T00:10:47.342 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3090000000000003e-06 2024-09-16T00:10:47.342 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.342 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.342 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.342 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708472+0000", 2024-09-16T00:10:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7499999999999998e-06 2024-09-16T00:10:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708479+0000", 2024-09-16T00:10:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1189999999999999e-06 2024-09-16T00:10:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721895+0000", 2024-09-16T00:10:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013415664000000001 2024-09-16T00:10:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.343 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.721941+0000", 2024-09-16T00:10:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6031e-05 2024-09-16T00:10:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722007+0000", 2024-09-16T00:10:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6503999999999999e-05 2024-09-16T00:10:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725277+0000", 2024-09-16T00:10:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.003269554 2024-09-16T00:10:47.344 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731158+0000", 2024-09-16T00:10:47.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0058813750000000003 2024-09-16T00:10:47.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731240+0000", 2024-09-16T00:10:47.345 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1815000000000002e-05 2024-09-16T00:10:47.345 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.345 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.345 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.345 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731276+0000", 2024-09-16T00:10:47.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.6449999999999998e-05 2024-09-16T00:10:47.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.346 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.346 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.346 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.346 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.346 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4188 2.18 2:1cc859d9:::200.00000012:head [write 936299~2419 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.346 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:47.346 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.573623570000002, 2024-09-16T00:10:47.346 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.022518565000000001, 2024-09-16T00:10:47.346 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.346 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.346 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.346 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:47.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708877+0000", 2024-09-16T00:10:47.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.347 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.347 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.347 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.347 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708878+0000", 2024-09-16T00:10:47.347 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8800000000000001e-07 2024-09-16T00:10:47.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708884+0000", 2024-09-16T00:10:47.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9800000000000003e-06 2024-09-16T00:10:47.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708895+0000", 2024-09-16T00:10:47.348 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1416e-05 2024-09-16T00:10:47.348 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.348 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.348 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.348 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.708903+0000", 2024-09-16T00:10:47.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4449999999999999e-06 2024-09-16T00:10:47.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722327+0000", 2024-09-16T00:10:47.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013424681000000001 2024-09-16T00:10:47.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722408+0000", 2024-09-16T00:10:47.349 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0092000000000005e-05 2024-09-16T00:10:47.349 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.349 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.349 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.349 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.722538+0000", 2024-09-16T00:10:47.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00013046600000000001 2024-09-16T00:10:47.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726665+0000", 2024-09-16T00:10:47.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0041267420000000001 2024-09-16T00:10:47.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.350 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731284+0000", 2024-09-16T00:10:47.350 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046195469999999999 2024-09-16T00:10:47.350 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.350 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.350 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731358+0000", 2024-09-16T00:10:47.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4002000000000001e-05 2024-09-16T00:10:47.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.351 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.351 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731396+0000", 2024-09-16T00:10:47.351 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.7305999999999998e-05 2024-09-16T00:10:47.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.351 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.351 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.351 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.351 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.351 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4189 2.18 2:1cc859d9:::200.00000012:head [write 938718~1910 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.351 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:47.352 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.572716454000002, 2024-09-16T00:10:47.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021703626, 2024-09-16T00:10:47.352 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.352 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.352 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.352 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:47.352 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.352 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.352 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.352 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.352 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709784+0000", 2024-09-16T00:10:47.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709785+0000", 2024-09-16T00:10:47.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.043e-06 2024-09-16T00:10:47.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.353 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709791+0000", 2024-09-16T00:10:47.353 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9499999999999998e-06 2024-09-16T00:10:47.353 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.353 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.353 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709804+0000", 2024-09-16T00:10:47.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2958e-05 2024-09-16T00:10:47.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.709812+0000", 2024-09-16T00:10:47.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9869999999999999e-06 2024-09-16T00:10:47.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.354 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.354 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723058+0000", 2024-09-16T00:10:47.354 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013245497 2024-09-16T00:10:47.354 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.354 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723131+0000", 2024-09-16T00:10:47.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3879000000000002e-05 2024-09-16T00:10:47.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723241+0000", 2024-09-16T00:10:47.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000109435 2024-09-16T00:10:47.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.355 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.355 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.355 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726708+0000", 2024-09-16T00:10:47.355 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0034671340000000002 2024-09-16T00:10:47.355 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731403+0000", 2024-09-16T00:10:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0046953979999999999 2024-09-16T00:10:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731454+0000", 2024-09-16T00:10:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0747e-05 2024-09-16T00:10:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731488+0000", 2024-09-16T00:10:47.356 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3597999999999999e-05 2024-09-16T00:10:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4190 2.18 2:1cc859d9:::200.00000012:head [write 940628~2420 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.572244288999997, 2024-09-16T00:10:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021346812999999999, 2024-09-16T00:10:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.357 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710256+0000", 2024-09-16T00:10:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710257+0000", 2024-09-16T00:10:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8100000000000001e-07 2024-09-16T00:10:47.358 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710263+0000", 2024-09-16T00:10:47.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8050000000000003e-06 2024-09-16T00:10:47.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710273+0000", 2024-09-16T00:10:47.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0005e-05 2024-09-16T00:10:47.359 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.359 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.359 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.359 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710280+0000", 2024-09-16T00:10:47.359 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3819999999999998e-06 2024-09-16T00:10:47.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723653+0000", 2024-09-16T00:10:47.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013372866000000001 2024-09-16T00:10:47.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723733+0000", 2024-09-16T00:10:47.360 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9396999999999999e-05 2024-09-16T00:10:47.360 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.360 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.360 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.360 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.723852+0000", 2024-09-16T00:10:47.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011919899999999999 2024-09-16T00:10:47.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726752+0000", 2024-09-16T00:10:47.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0028999659999999999 2024-09-16T00:10:47.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.361 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731495+0000", 2024-09-16T00:10:47.361 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0047432209999999997 2024-09-16T00:10:47.361 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.361 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.361 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731568+0000", 2024-09-16T00:10:47.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2452000000000004e-05 2024-09-16T00:10:47.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.362 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.362 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731603+0000", 2024-09-16T00:10:47.362 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5639000000000002e-05 2024-09-16T00:10:47.362 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.362 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.362 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.362 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.362 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.362 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4192 2.18 2:1cc859d9:::200.00000012:head [write 943048~1399 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.362 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:47.362 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.571511448999999, 2024-09-16T00:10:47.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021688433999999999, 2024-09-16T00:10:47.363 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.363 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.363 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.363 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:47.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.363 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.363 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.363 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.363 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710989+0000", 2024-09-16T00:10:47.363 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710990+0000", 2024-09-16T00:10:47.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1999999999999998e-07 2024-09-16T00:10:47.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.710996+0000", 2024-09-16T00:10:47.364 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8069999999999996e-06 2024-09-16T00:10:47.364 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.364 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.364 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.364 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711007+0000", 2024-09-16T00:10:47.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1471000000000001e-05 2024-09-16T00:10:47.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711014+0000", 2024-09-16T00:10:47.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7780000000000002e-06 2024-09-16T00:10:47.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.365 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724471+0000", 2024-09-16T00:10:47.365 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013457435 2024-09-16T00:10:47.365 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.365 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.365 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724547+0000", 2024-09-16T00:10:47.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5383999999999995e-05 2024-09-16T00:10:47.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.724663+0000", 2024-09-16T00:10:47.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116348 2024-09-16T00:10:47.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.366 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.366 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731612+0000", 2024-09-16T00:10:47.366 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0069491409999999998 2024-09-16T00:10:47.366 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.366 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732573+0000", 2024-09-16T00:10:47.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0009611 2024-09-16T00:10:47.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732638+0000", 2024-09-16T00:10:47.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4919000000000006e-05 2024-09-16T00:10:47.367 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.367 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.367 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.367 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732678+0000", 2024-09-16T00:10:47.367 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9230999999999998e-05 2024-09-16T00:10:47.367 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.368 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.368 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.368 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.368 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4193 2.18 2:1cc859d9:::200.00000012:head [write 944447~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.368 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:47.368 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.570965741000002, 2024-09-16T00:10:47.368 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.021286851999999998, 2024-09-16T00:10:47.368 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.368 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.368 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.368 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.368 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.368 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:47.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711535+0000", 2024-09-16T00:10:47.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.369 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.369 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711536+0000", 2024-09-16T00:10:47.369 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0189999999999999e-06 2024-09-16T00:10:47.369 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.369 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711542+0000", 2024-09-16T00:10:47.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7169999999999996e-06 2024-09-16T00:10:47.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711554+0000", 2024-09-16T00:10:47.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2089000000000001e-05 2024-09-16T00:10:47.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.370 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.370 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.711559+0000", 2024-09-16T00:10:47.370 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3739999999999997e-06 2024-09-16T00:10:47.370 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.370 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725311+0000", 2024-09-16T00:10:47.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013751559 2024-09-16T00:10:47.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725390+0000", 2024-09-16T00:10:47.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.9779999999999998e-05 2024-09-16T00:10:47.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.371 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.371 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.371 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.725509+0000", 2024-09-16T00:10:47.371 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118973 2024-09-16T00:10:47.371 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731617+0000", 2024-09-16T00:10:47.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0061079000000000003 2024-09-16T00:10:47.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732762+0000", 2024-09-16T00:10:47.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011449769999999999 2024-09-16T00:10:47.372 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.372 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.372 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.372 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732796+0000", 2024-09-16T00:10:47.372 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.3587999999999998e-05 2024-09-16T00:10:47.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.373 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.373 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732822+0000", 2024-09-16T00:10:47.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5876000000000001e-05 2024-09-16T00:10:47.373 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.373 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.373 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.373 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.373 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.373 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4194 2.18 2:1cc859d9:::200.00000012:head [write 946356~2929 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.373 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:47.373 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.569990709999999, 2024-09-16T00:10:47.373 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.020453244999999998, 2024-09-16T00:10:47.374 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.374 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.374 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.374 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:47.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.374 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.374 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.374 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712510+0000", 2024-09-16T00:10:47.374 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.374 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712511+0000", 2024-09-16T00:10:47.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0999999999999997e-07 2024-09-16T00:10:47.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712517+0000", 2024-09-16T00:10:47.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8370000000000002e-06 2024-09-16T00:10:47.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.375 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.375 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.375 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712528+0000", 2024-09-16T00:10:47.375 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1528e-05 2024-09-16T00:10:47.375 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.712536+0000", 2024-09-16T00:10:47.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3939999999999998e-06 2024-09-16T00:10:47.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726004+0000", 2024-09-16T00:10:47.376 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013468177 2024-09-16T00:10:47.376 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.376 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.376 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.376 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726085+0000", 2024-09-16T00:10:47.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1626999999999996e-05 2024-09-16T00:10:47.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726205+0000", 2024-09-16T00:10:47.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000119751 2024-09-16T00:10:47.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.731622+0000", 2024-09-16T00:10:47.377 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0054165419999999999 2024-09-16T00:10:47.377 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.377 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.377 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.377 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732867+0000", 2024-09-16T00:10:47.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001244855 2024-09-16T00:10:47.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732924+0000", 2024-09-16T00:10:47.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7395999999999997e-05 2024-09-16T00:10:47.378 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.378 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.378 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.378 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732963+0000", 2024-09-16T00:10:47.378 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9227999999999998e-05 2024-09-16T00:10:47.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.378 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.378 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.379 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.379 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4195 2.18 2:1cc859d9:::200.00000012:head [write 949285~1909 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.379 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:47.379 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.568939049999997, 2024-09-16T00:10:47.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019498099000000001, 2024-09-16T00:10:47.379 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.379 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.379 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.379 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.379 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.379 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.379 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:47.379 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713562+0000", 2024-09-16T00:10:47.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713563+0000", 2024-09-16T00:10:47.380 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1000000000000001e-06 2024-09-16T00:10:47.380 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.380 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.380 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.380 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713569+0000", 2024-09-16T00:10:47.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8710000000000002e-06 2024-09-16T00:10:47.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713582+0000", 2024-09-16T00:10:47.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3134e-05 2024-09-16T00:10:47.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.381 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.713591+0000", 2024-09-16T00:10:47.381 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9889999999999998e-06 2024-09-16T00:10:47.381 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.381 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.381 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726786+0000", 2024-09-16T00:10:47.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013194991999999999 2024-09-16T00:10:47.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726856+0000", 2024-09-16T00:10:47.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0017000000000004e-05 2024-09-16T00:10:47.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.382 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.382 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.726967+0000", 2024-09-16T00:10:47.382 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000111815 2024-09-16T00:10:47.382 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.382 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.732530+0000", 2024-09-16T00:10:47.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055624940000000003 2024-09-16T00:10:47.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733001+0000", 2024-09-16T00:10:47.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000470817 2024-09-16T00:10:47.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.383 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.383 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733030+0000", 2024-09-16T00:10:47.383 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9051e-05 2024-09-16T00:10:47.383 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.383 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.384 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.384 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733060+0000", 2024-09-16T00:10:47.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9819e-05 2024-09-16T00:10:47.384 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.384 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.384 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.384 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.384 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.384 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4215 2.18 2:1cc859d9:::200.00000012:head [write 991028~1910 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.384 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:47.384 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.549243339, 2024-09-16T00:10:47.384 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019416676000000001, 2024-09-16T00:10:47.384 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.385 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.385 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.385 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:47.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.385 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733257+0000", 2024-09-16T00:10:47.385 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.385 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.385 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.385 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733258+0000", 2024-09-16T00:10:47.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7000000000000003e-07 2024-09-16T00:10:47.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733264+0000", 2024-09-16T00:10:47.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4090000000000002e-06 2024-09-16T00:10:47.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.386 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.386 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733273+0000", 2024-09-16T00:10:47.386 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.4769999999999992e-06 2024-09-16T00:10:47.386 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.386 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733280+0000", 2024-09-16T00:10:47.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.2989999999999999e-06 2024-09-16T00:10:47.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.742966+0000", 2024-09-16T00:10:47.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009685585 2024-09-16T00:10:47.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.387 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.387 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.387 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743046+0000", 2024-09-16T00:10:47.387 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0469999999999994e-05 2024-09-16T00:10:47.387 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743165+0000", 2024-09-16T00:10:47.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118847 2024-09-16T00:10:47.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.746968+0000", 2024-09-16T00:10:47.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0038025099999999998 2024-09-16T00:10:47.388 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.388 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.388 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.388 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752559+0000", 2024-09-16T00:10:47.388 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0055911290000000002 2024-09-16T00:10:47.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752634+0000", 2024-09-16T00:10:47.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.5102e-05 2024-09-16T00:10:47.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.389 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.389 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.389 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752674+0000", 2024-09-16T00:10:47.389 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9878e-05 2024-09-16T00:10:47.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.389 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.389 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.389 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.390 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:4216 2.18 2:1cc859d9:::200.00000012:head [write 992938~2755 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93)", 2024-09-16T00:10:47.390 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:47.390 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 48.548697431000001, 2024-09-16T00:10:47.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.019035976, 2024-09-16T00:10:47.390 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.390 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.390 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.390 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.390 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.390 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:47.390 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.390 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.390 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733803+0000", 2024-09-16T00:10:47.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733804+0000", 2024-09-16T00:10:47.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.2299999999999999e-07 2024-09-16T00:10:47.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.391 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.391 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.391 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733810+0000", 2024-09-16T00:10:47.391 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7259999999999999e-06 2024-09-16T00:10:47.391 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733822+0000", 2024-09-16T00:10:47.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2204e-05 2024-09-16T00:10:47.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.733829+0000", 2024-09-16T00:10:47.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3889999999999999e-06 2024-09-16T00:10:47.392 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.392 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.392 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.392 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743675+0000", 2024-09-16T00:10:47.392 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098457990000000006 2024-09-16T00:10:47.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743752+0000", 2024-09-16T00:10:47.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6279000000000006e-05 2024-09-16T00:10:47.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.743867+0000", 2024-09-16T00:10:47.393 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000115319 2024-09-16T00:10:47.393 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.393 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.393 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.393 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.746971+0000", 2024-09-16T00:10:47.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0031042399999999999 2024-09-16T00:10:47.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752723+0000", 2024-09-16T00:10:47.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057518329999999996 2024-09-16T00:10:47.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752804+0000", 2024-09-16T00:10:47.394 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.0822999999999995e-05 2024-09-16T00:10:47.394 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.394 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.394 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.394 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:09:58.752839+0000", 2024-09-16T00:10:47.395 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.5441000000000002e-05 2024-09-16T00:10:47.395 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.395 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.395 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.395 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.395 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.395 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:47.396 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-16T00:10:47.500 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:47.500 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:47.500 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:47.501 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:47.509 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-16T00:10:47.552 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-16T00:10:47.552 INFO:tasks.thrashosds.thrasher:Reviving osd 7 2024-09-16T00:10:47.552 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-16T00:10:47.552 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-16T00:10:47.555 INFO:tasks.ceph.osd.7:Started 2024-09-16T00:10:47.555 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:10:47.618 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:47.618 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:47.618 INFO:teuthology.orchestra.run.smithi164.stdout: "complaint_time": 180, 2024-09-16T00:10:47.618 INFO:teuthology.orchestra.run.smithi164.stdout: "num_blocked_ops": 0 2024-09-16T00:10:47.619 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:47.629 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-16T00:10:47.657 INFO:teuthology.orchestra.run.smithi164.stderr:no valid command found; 10 closest matches: 2024-09-16T00:10:47.658 INFO:teuthology.orchestra.run.smithi164.stderr:0 2024-09-16T00:10:47.658 INFO:teuthology.orchestra.run.smithi164.stderr:1 2024-09-16T00:10:47.658 INFO:teuthology.orchestra.run.smithi164.stderr:2 2024-09-16T00:10:47.658 INFO:teuthology.orchestra.run.smithi164.stderr:abort 2024-09-16T00:10:47.658 INFO:teuthology.orchestra.run.smithi164.stderr:assert 2024-09-16T00:10:47.658 INFO:teuthology.orchestra.run.smithi164.stderr:config diff 2024-09-16T00:10:47.658 INFO:teuthology.orchestra.run.smithi164.stderr:config diff get 2024-09-16T00:10:47.658 INFO:teuthology.orchestra.run.smithi164.stderr:config get 2024-09-16T00:10:47.658 INFO:teuthology.orchestra.run.smithi164.stderr:config help [] 2024-09-16T00:10:47.658 INFO:teuthology.orchestra.run.smithi164.stderr:config set ... 2024-09-16T00:10:47.658 INFO:teuthology.orchestra.run.smithi164.stderr:admin_socket: invalid command 2024-09-16T00:10:47.660 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T00:10:47.660 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2024-09-16T00:10:47.735 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:47.735 INFO:teuthology.orchestra.run.smithi164.stdout: "size": 20, 2024-09-16T00:10:47.735 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 600, 2024-09-16T00:10:47.735 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [ 2024-09-16T00:10:47.735 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.735 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5178 3.2c 3:375e1b9b:::100000030c8.00000002:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:47.735 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:47.735 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.924597442, 2024-09-16T00:10:47.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012118438, 2024-09-16T00:10:47.736 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.736 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.736 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.736 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:47.736 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.736 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.736 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.736 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.736 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808908+0000", 2024-09-16T00:10:47.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808909+0000", 2024-09-16T00:10:47.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.8160000000000001e-06 2024-09-16T00:10:47.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.737 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.737 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808921+0000", 2024-09-16T00:10:47.737 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1117999999999999e-05 2024-09-16T00:10:47.737 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.737 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808941+0000", 2024-09-16T00:10:47.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.0712000000000001e-05 2024-09-16T00:10:47.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.808952+0000", 2024-09-16T00:10:47.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0193e-05 2024-09-16T00:10:47.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.738 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.738 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.738 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809019+0000", 2024-09-16T00:10:47.738 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7404999999999996e-05 2024-09-16T00:10:47.738 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809100+0000", 2024-09-16T00:10:47.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.1029e-05 2024-09-16T00:10:47.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.809206+0000", 2024-09-16T00:10:47.739 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105603 2024-09-16T00:10:47.739 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.739 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.739 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.739 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.815162+0000", 2024-09-16T00:10:47.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0059567250000000004 2024-09-16T00:10:47.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.820925+0000", 2024-09-16T00:10:47.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0057623479999999996 2024-09-16T00:10:47.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.740 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.740 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.820987+0000", 2024-09-16T00:10:47.740 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2680000000000003e-05 2024-09-16T00:10:47.740 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.740 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.741 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.741 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.821026+0000", 2024-09-16T00:10:47.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8809000000000001e-05 2024-09-16T00:10:47.741 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.741 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.741 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.741 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.741 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.741 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5196 3.26 3:66ea4dfb:::100000030f0.00000001:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:47.741 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:47.741 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.923410414000003, 2024-09-16T00:10:47.741 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095222459999999998, 2024-09-16T00:10:47.741 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.741 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.742 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.742 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:47.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.742 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.742 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:47.742 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.742 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.742 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810095+0000", 2024-09-16T00:10:47.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.3499999999999995e-07 2024-09-16T00:10:47.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810102+0000", 2024-09-16T00:10:47.743 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7390000000000002e-06 2024-09-16T00:10:47.743 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.743 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.743 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.743 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810112+0000", 2024-09-16T00:10:47.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7550000000000006e-06 2024-09-16T00:10:47.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.744 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.744 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810127+0000", 2024-09-16T00:10:47.744 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4749999999999999e-05 2024-09-16T00:10:47.744 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.744 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810184+0000", 2024-09-16T00:10:47.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7701999999999997e-05 2024-09-16T00:10:47.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810256+0000", 2024-09-16T00:10:47.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1595999999999997e-05 2024-09-16T00:10:47.745 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.745 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.745 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:47.745 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.810346+0000", 2024-09-16T00:10:47.745 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0043999999999994e-05 2024-09-16T00:10:47.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.815239+0000", 2024-09-16T00:10:47.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0048934470000000004 2024-09-16T00:10:47.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.746 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819526+0000", 2024-09-16T00:10:47.746 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0042868630000000001 2024-09-16T00:10:47.746 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.746 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.746 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819577+0000", 2024-09-16T00:10:47.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0445999999999998e-05 2024-09-16T00:10:47.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.747 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.747 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.819617+0000", 2024-09-16T00:10:47.747 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.0169000000000001e-05 2024-09-16T00:10:47.747 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.747 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.747 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.747 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.747 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.747 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5244 3.20 3:068a06d7:::100000032f2.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:47.747 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:47.748 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.906314907000002, 2024-09-16T00:10:47.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094083010000000009, 2024-09-16T00:10:47.748 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.748 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.748 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:47.748 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.748 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:47.748 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.748 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.748 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.748 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827190+0000", 2024-09-16T00:10:47.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827191+0000", 2024-09-16T00:10:47.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.2330000000000001e-06 2024-09-16T00:10:47.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.749 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.749 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827195+0000", 2024-09-16T00:10:47.749 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.8750000000000002e-06 2024-09-16T00:10:47.749 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.749 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827206+0000", 2024-09-16T00:10:47.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0473e-05 2024-09-16T00:10:47.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827213+0000", 2024-09-16T00:10:47.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1149999999999996e-06 2024-09-16T00:10:47.750 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.750 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.750 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.750 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827275+0000", 2024-09-16T00:10:47.750 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2532000000000004e-05 2024-09-16T00:10:47.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.827392+0000", 2024-09-16T00:10:47.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000116266 2024-09-16T00:10:47.751 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.751 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.751 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.751 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.836599+0000", 2024-09-16T00:10:47.751 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092068069999999991 2024-09-16T00:10:47.751 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.751 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.751 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.752 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.752 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5252 2.1a 2:5913b086:::1000000336d.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e96)", 2024-09-16T00:10:47.752 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:47.752 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 44.905379084000003, 2024-09-16T00:10:47.752 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010086441999999999, 2024-09-16T00:10:47.752 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.752 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.752 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:47.752 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.752 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.752 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.752 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:47.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828126+0000", 2024-09-16T00:10:47.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.753 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.753 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828127+0000", 2024-09-16T00:10:47.753 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.5099999999999999e-07 2024-09-16T00:10:47.753 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.753 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828140+0000", 2024-09-16T00:10:47.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.3614e-05 2024-09-16T00:10:47.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828149+0000", 2024-09-16T00:10:47.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7250000000000003e-06 2024-09-16T00:10:47.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.754 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.754 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.754 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828156+0000", 2024-09-16T00:10:47.754 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9430000000000003e-06 2024-09-16T00:10:47.754 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828274+0000", 2024-09-16T00:10:47.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000118121 2024-09-16T00:10:47.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.755 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.828381+0000", 2024-09-16T00:10:47.755 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000107379 2024-09-16T00:10:47.755 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.755 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.755 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.756 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:02.838212+0000", 2024-09-16T00:10:47.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098310089999999999 2024-09-16T00:10:47.756 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.756 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.756 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.756 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.756 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.756 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:5743 3.36 3:6c273be3:::1000000385e.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e97)", 2024-09-16T00:10:47.756 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:03.854364+0000", 2024-09-16T00:10:47.756 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 43.879141240999999, 2024-09-16T00:10:47.756 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0099844070000000007, 2024-09-16T00:10:47.756 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.756 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.757 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:47.757 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854364+0000", 2024-09-16T00:10:47.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.757 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.757 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854364+0000", 2024-09-16T00:10:47.757 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.757 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.757 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854365+0000", 2024-09-16T00:10:47.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.3399999999999999e-07 2024-09-16T00:10:47.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854368+0000", 2024-09-16T00:10:47.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4300000000000002e-06 2024-09-16T00:10:47.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.758 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.758 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.758 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854376+0000", 2024-09-16T00:10:47.758 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.2770000000000005e-06 2024-09-16T00:10:47.758 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854384+0000", 2024-09-16T00:10:47.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.7200000000000006e-06 2024-09-16T00:10:47.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854455+0000", 2024-09-16T00:10:47.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0631999999999997e-05 2024-09-16T00:10:47.759 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.759 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.759 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.759 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.854560+0000", 2024-09-16T00:10:47.759 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000105338 2024-09-16T00:10:47.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.760 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.760 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:03.864348+0000", 2024-09-16T00:10:47.760 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097883759999999997 2024-09-16T00:10:47.760 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.760 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.760 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.760 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.760 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.760 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:6601 3.35 3:af7d55b1:::10000003232.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e100)", 2024-09-16T00:10:47.760 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:06.866600+0000", 2024-09-16T00:10:47.761 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 40.866904796999997, 2024-09-16T00:10:47.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092432980000000005, 2024-09-16T00:10:47.761 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.761 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.761 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:47.761 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.761 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866600+0000", 2024-09-16T00:10:47.761 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.761 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.761 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.761 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866600+0000", 2024-09-16T00:10:47.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866602+0000", 2024-09-16T00:10:47.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.649e-06 2024-09-16T00:10:47.762 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.762 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.762 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.762 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866607+0000", 2024-09-16T00:10:47.762 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7330000000000003e-06 2024-09-16T00:10:47.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866618+0000", 2024-09-16T00:10:47.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1506e-05 2024-09-16T00:10:47.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866626+0000", 2024-09-16T00:10:47.763 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.498e-06 2024-09-16T00:10:47.763 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.763 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.763 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.763 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866685+0000", 2024-09-16T00:10:47.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8950000000000003e-05 2024-09-16T00:10:47.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.866809+0000", 2024-09-16T00:10:47.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00012389899999999999 2024-09-16T00:10:47.764 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.764 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.764 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.764 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:06.875844+0000", 2024-09-16T00:10:47.764 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0090350629999999994 2024-09-16T00:10:47.764 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.765 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.765 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.765 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.765 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:8541 3.3f e104/94 3:fd643af1:::10000001465.00000000:head v 104'282, mlcod=103'280)", 2024-09-16T00:10:47.765 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:12.911145+0000", 2024-09-16T00:10:47.765 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.822360216999996, 2024-09-16T00:10:47.765 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094161980000000006, 2024-09-16T00:10:47.765 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.765 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.765 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.765 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.765 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911145+0000", 2024-09-16T00:10:47.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911145+0000", 2024-09-16T00:10:47.766 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.766 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.766 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.766 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.766 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911146+0000", 2024-09-16T00:10:47.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0699999999999996e-07 2024-09-16T00:10:47.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911151+0000", 2024-09-16T00:10:47.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8710000000000002e-06 2024-09-16T00:10:47.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.767 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.767 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911161+0000", 2024-09-16T00:10:47.767 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.9350000000000006e-06 2024-09-16T00:10:47.767 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.767 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911168+0000", 2024-09-16T00:10:47.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6340000000000003e-06 2024-09-16T00:10:47.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911238+0000", 2024-09-16T00:10:47.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.0118999999999997e-05 2024-09-16T00:10:47.768 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.768 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.768 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.768 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911258+0000", 2024-09-16T00:10:47.768 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.9361999999999999e-05 2024-09-16T00:10:47.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.920518+0000", 2024-09-16T00:10:47.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0092603249999999998 2024-09-16T00:10:47.769 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.769 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.769 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.769 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.920561+0000", 2024-09-16T00:10:47.769 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3244999999999997e-05 2024-09-16T00:10:47.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.769 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.769 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.770 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.770 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:8587 3.35 3:ad562626:::10000001599.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e104)", 2024-09-16T00:10:47.770 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:12.911217+0000", 2024-09-16T00:10:47.770 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 34.822288288999999, 2024-09-16T00:10:47.770 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0095415759999999995, 2024-09-16T00:10:47.770 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.770 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.770 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.770 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.770 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.770 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.770 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911217+0000", 2024-09-16T00:10:47.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911217+0000", 2024-09-16T00:10:47.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.771 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.771 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911218+0000", 2024-09-16T00:10:47.771 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8299999999999996e-07 2024-09-16T00:10:47.771 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.771 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911222+0000", 2024-09-16T00:10:47.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.301e-06 2024-09-16T00:10:47.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911232+0000", 2024-09-16T00:10:47.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.0244000000000001e-05 2024-09-16T00:10:47.772 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.772 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.772 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.772 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911239+0000", 2024-09-16T00:10:47.772 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.9e-06 2024-09-16T00:10:47.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911324+0000", 2024-09-16T00:10:47.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5222999999999994e-05 2024-09-16T00:10:47.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911496+0000", 2024-09-16T00:10:47.773 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000172065 2024-09-16T00:10:47.773 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.773 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.773 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:47.773 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.911600+0000", 2024-09-16T00:10:47.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000104002 2024-09-16T00:10:47.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.916599+0000", 2024-09-16T00:10:47.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.004998911 2024-09-16T00:10:47.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.774 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.774 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.920663+0000", 2024-09-16T00:10:47.774 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0040635799999999998 2024-09-16T00:10:47.774 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.774 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.920715+0000", 2024-09-16T00:10:47.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2549000000000001e-05 2024-09-16T00:10:47.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.775 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.775 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:12.920758+0000", 2024-09-16T00:10:47.775 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3018e-05 2024-09-16T00:10:47.775 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.775 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.775 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.775 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.775 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.775 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:11691 3.35 3:ad66404a:::10000002d4d.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e111)", 2024-09-16T00:10:47.776 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:22.048217+0000", 2024-09-16T00:10:47.776 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 25.685288601, 2024-09-16T00:10:47.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010683378, 2024-09-16T00:10:47.776 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.776 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.776 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.776 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.776 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.776 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048217+0000", 2024-09-16T00:10:47.776 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.776 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.776 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048217+0000", 2024-09-16T00:10:47.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048218+0000", 2024-09-16T00:10:47.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.722e-06 2024-09-16T00:10:47.777 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.777 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.777 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.777 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048223+0000", 2024-09-16T00:10:47.777 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.0590000000000002e-06 2024-09-16T00:10:47.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048239+0000", 2024-09-16T00:10:47.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5271e-05 2024-09-16T00:10:47.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048246+0000", 2024-09-16T00:10:47.778 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8490000000000006e-06 2024-09-16T00:10:47.778 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.778 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.778 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.778 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048300+0000", 2024-09-16T00:10:47.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3109999999999998e-05 2024-09-16T00:10:47.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048495+0000", 2024-09-16T00:10:47.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000195198 2024-09-16T00:10:47.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:47.779 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.048583+0000", 2024-09-16T00:10:47.779 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8472000000000003e-05 2024-09-16T00:10:47.779 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.779 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.779 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.049175+0000", 2024-09-16T00:10:47.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00059207800000000001 2024-09-16T00:10:47.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.058797+0000", 2024-09-16T00:10:47.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0096219329999999992 2024-09-16T00:10:47.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.780 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.780 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.780 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.058856+0000", 2024-09-16T00:10:47.780 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.8782000000000001e-05 2024-09-16T00:10:47.780 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.781 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.781 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:22.058900+0000", 2024-09-16T00:10:47.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3903999999999998e-05 2024-09-16T00:10:47.781 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.781 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.781 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.781 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.781 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.781 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:12961 2.1a 2:595229b5:::10000000cd4.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:47.781 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.088496+0000", 2024-09-16T00:10:47.781 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.645009501000001, 2024-09-16T00:10:47.781 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.013027634, 2024-09-16T00:10:47.782 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.782 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.782 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.782 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088496+0000", 2024-09-16T00:10:47.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.782 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.782 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.782 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.782 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088496+0000", 2024-09-16T00:10:47.782 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088496+0000", 2024-09-16T00:10:47.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.6599999999999996e-07 2024-09-16T00:10:47.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.783 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088498+0000", 2024-09-16T00:10:47.783 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.317e-06 2024-09-16T00:10:47.783 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.783 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.783 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088506+0000", 2024-09-16T00:10:47.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.1459999999999999e-06 2024-09-16T00:10:47.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088511+0000", 2024-09-16T00:10:47.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.541e-06 2024-09-16T00:10:47.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.784 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.784 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.784 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088815+0000", 2024-09-16T00:10:47.784 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00030421000000000002 2024-09-16T00:10:47.784 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.088988+0000", 2024-09-16T00:10:47.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000172356 2024-09-16T00:10:47.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.785 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.089076+0000", 2024-09-16T00:10:47.785 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.8270999999999996e-05 2024-09-16T00:10:47.785 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.785 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.785 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.090357+0000", 2024-09-16T00:10:47.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001280661 2024-09-16T00:10:47.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.101242+0000", 2024-09-16T00:10:47.786 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010885238 2024-09-16T00:10:47.786 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.786 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.786 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.786 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.101489+0000", 2024-09-16T00:10:47.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000247538 2024-09-16T00:10:47.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.787 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.787 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.101523+0000", 2024-09-16T00:10:47.787 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.379e-05 2024-09-16T00:10:47.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.787 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.787 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.787 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.787 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.787 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13112 3.2c 3:36940359:::10000000d76.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:47.787 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.122269+0000", 2024-09-16T00:10:47.787 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.611235894, 2024-09-16T00:10:47.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010413552, 2024-09-16T00:10:47.788 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.788 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.788 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.788 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122269+0000", 2024-09-16T00:10:47.788 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.788 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.788 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.788 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.788 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122269+0000", 2024-09-16T00:10:47.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122270+0000", 2024-09-16T00:10:47.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.6499999999999999e-07 2024-09-16T00:10:47.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.789 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.789 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122272+0000", 2024-09-16T00:10:47.789 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.5189999999999999e-06 2024-09-16T00:10:47.789 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.789 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122279+0000", 2024-09-16T00:10:47.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.7039999999999997e-06 2024-09-16T00:10:47.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122284+0000", 2024-09-16T00:10:47.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5670000000000002e-06 2024-09-16T00:10:47.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.790 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.790 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.790 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122835+0000", 2024-09-16T00:10:47.790 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00055031399999999999 2024-09-16T00:10:47.790 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.122993+0000", 2024-09-16T00:10:47.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00015790499999999999 2024-09-16T00:10:47.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.123078+0000", 2024-09-16T00:10:47.791 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.5803000000000005e-05 2024-09-16T00:10:47.791 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.791 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.791 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.791 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.123884+0000", 2024-09-16T00:10:47.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00080534599999999997 2024-09-16T00:10:47.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.132579+0000", 2024-09-16T00:10:47.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0086952409999999994 2024-09-16T00:10:47.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.792 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.132643+0000", 2024-09-16T00:10:47.792 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4289999999999996e-05 2024-09-16T00:10:47.792 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.792 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.792 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.793 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.132683+0000", 2024-09-16T00:10:47.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9397999999999998e-05 2024-09-16T00:10:47.793 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.793 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.793 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.793 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.793 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.793 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13227 3.26 3:64bfa412:::10000000dee.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:47.793 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.146915+0000", 2024-09-16T00:10:47.793 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.586590042000001, 2024-09-16T00:10:47.793 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012561259, 2024-09-16T00:10:47.793 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.793 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.794 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.794 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146915+0000", 2024-09-16T00:10:47.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.794 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.794 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146915+0000", 2024-09-16T00:10:47.794 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.794 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.794 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146916+0000", 2024-09-16T00:10:47.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.155e-06 2024-09-16T00:10:47.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146922+0000", 2024-09-16T00:10:47.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3249999999999998e-06 2024-09-16T00:10:47.795 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.795 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.795 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.795 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146938+0000", 2024-09-16T00:10:47.795 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5974999999999999e-05 2024-09-16T00:10:47.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.146946+0000", 2024-09-16T00:10:47.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.3189999999999995e-06 2024-09-16T00:10:47.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.147095+0000", 2024-09-16T00:10:47.796 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00014879800000000001 2024-09-16T00:10:47.796 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.796 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.796 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.796 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.147262+0000", 2024-09-16T00:10:47.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000167353 2024-09-16T00:10:47.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:47.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.147357+0000", 2024-09-16T00:10:47.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.5199999999999997e-05 2024-09-16T00:10:47.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.797 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.797 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.148582+0000", 2024-09-16T00:10:47.797 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001224679 2024-09-16T00:10:47.797 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.797 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.159363+0000", 2024-09-16T00:10:47.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010780839 2024-09-16T00:10:47.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.159422+0000", 2024-09-16T00:10:47.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.9731999999999997e-05 2024-09-16T00:10:47.798 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.798 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.798 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.798 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.159476+0000", 2024-09-16T00:10:47.798 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3884000000000001e-05 2024-09-16T00:10:47.799 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.799 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.799 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.799 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.799 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.799 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13251 3.36 3:6f745cbe:::10000000e09.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:47.799 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.152775+0000", 2024-09-16T00:10:47.799 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.580730154000001, 2024-09-16T00:10:47.799 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011312687, 2024-09-16T00:10:47.799 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.799 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.799 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.799 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152775+0000", 2024-09-16T00:10:47.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152775+0000", 2024-09-16T00:10:47.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.800 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.800 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.800 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.800 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152776+0000", 2024-09-16T00:10:47.800 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.393e-06 2024-09-16T00:10:47.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152782+0000", 2024-09-16T00:10:47.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2329999999999997e-06 2024-09-16T00:10:47.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.801 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152797+0000", 2024-09-16T00:10:47.801 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5515e-05 2024-09-16T00:10:47.801 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.801 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.801 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152808+0000", 2024-09-16T00:10:47.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.1178e-05 2024-09-16T00:10:47.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152860+0000", 2024-09-16T00:10:47.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.1644999999999998e-05 2024-09-16T00:10:47.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.802 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.802 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.802 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.153026+0000", 2024-09-16T00:10:47.802 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016563400000000001 2024-09-16T00:10:47.802 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:47.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.153127+0000", 2024-09-16T00:10:47.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00010173 2024-09-16T00:10:47.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.154610+0000", 2024-09-16T00:10:47.803 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001482917 2024-09-16T00:10:47.803 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.803 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.803 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.803 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.164000+0000", 2024-09-16T00:10:47.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093899110000000008 2024-09-16T00:10:47.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.164048+0000", 2024-09-16T00:10:47.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8378000000000003e-05 2024-09-16T00:10:47.804 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.804 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.804 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.804 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.164088+0000", 2024-09-16T00:10:47.804 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9153000000000003e-05 2024-09-16T00:10:47.804 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.804 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.804 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.805 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.805 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13260 3.35 3:ad57d493:::10000000e12.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:47.805 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.152827+0000", 2024-09-16T00:10:47.805 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.580678548000002, 2024-09-16T00:10:47.805 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.012977191000000001, 2024-09-16T00:10:47.805 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.805 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.805 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.805 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.805 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.805 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.805 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152827+0000", 2024-09-16T00:10:47.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152827+0000", 2024-09-16T00:10:47.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.806 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.806 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.806 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.806 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152827+0000", 2024-09-16T00:10:47.806 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.2600000000000002e-07 2024-09-16T00:10:47.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152830+0000", 2024-09-16T00:10:47.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.9229999999999998e-06 2024-09-16T00:10:47.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152838+0000", 2024-09-16T00:10:47.807 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8790000000000003e-06 2024-09-16T00:10:47.807 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.807 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.807 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.807 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152844+0000", 2024-09-16T00:10:47.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.7769999999999999e-06 2024-09-16T00:10:47.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.152892+0000", 2024-09-16T00:10:47.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.7879e-05 2024-09-16T00:10:47.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.808 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.808 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.153052+0000", 2024-09-16T00:10:47.808 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016057199999999999 2024-09-16T00:10:47.808 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.808 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:47.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.153139+0000", 2024-09-16T00:10:47.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.6679000000000001e-05 2024-09-16T00:10:47.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.155267+0000", 2024-09-16T00:10:47.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0021280869999999999 2024-09-16T00:10:47.809 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.809 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.809 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.809 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.165714+0000", 2024-09-16T00:10:47.809 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.010447303 2024-09-16T00:10:47.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.165764+0000", 2024-09-16T00:10:47.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.024e-05 2024-09-16T00:10:47.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.810 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.810 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.810 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.165804+0000", 2024-09-16T00:10:47.810 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.9326000000000003e-05 2024-09-16T00:10:47.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.810 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.810 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.810 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.811 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13359 3.35 3:afdc6484:::10000000e78.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:47.811 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.173189+0000", 2024-09-16T00:10:47.811 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.560316563000001, 2024-09-16T00:10:47.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011094559, 2024-09-16T00:10:47.811 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.811 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.811 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.811 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.811 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.811 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.811 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173189+0000", 2024-09-16T00:10:47.811 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173189+0000", 2024-09-16T00:10:47.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173190+0000", 2024-09-16T00:10:47.812 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.249e-06 2024-09-16T00:10:47.812 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.812 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.812 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.812 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173194+0000", 2024-09-16T00:10:47.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.177e-06 2024-09-16T00:10:47.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173203+0000", 2024-09-16T00:10:47.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3500000000000003e-06 2024-09-16T00:10:47.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.813 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.813 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173210+0000", 2024-09-16T00:10:47.813 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.331e-06 2024-09-16T00:10:47.813 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.813 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173371+0000", 2024-09-16T00:10:47.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016161999999999999 2024-09-16T00:10:47.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173548+0000", 2024-09-16T00:10:47.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000176622 2024-09-16T00:10:47.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.814 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.814 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:47.814 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.173645+0000", 2024-09-16T00:10:47.814 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.7345999999999999e-05 2024-09-16T00:10:47.814 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.175459+0000", 2024-09-16T00:10:47.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.001814062 2024-09-16T00:10:47.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.184205+0000", 2024-09-16T00:10:47.815 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0087454479999999994 2024-09-16T00:10:47.815 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.815 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.815 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.815 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.184249+0000", 2024-09-16T00:10:47.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.4026999999999997e-05 2024-09-16T00:10:47.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.816 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.816 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.184283+0000", 2024-09-16T00:10:47.816 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 3.4326999999999998e-05 2024-09-16T00:10:47.816 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.816 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.816 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.816 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.816 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.816 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13435 3.2c 3:34a22dc0:::10000000ec9.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:47.816 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.187542+0000", 2024-09-16T00:10:47.817 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.54596355, 2024-09-16T00:10:47.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.011912524000000001, 2024-09-16T00:10:47.817 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.817 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.817 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.817 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187542+0000", 2024-09-16T00:10:47.817 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.817 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.817 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.817 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.817 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187542+0000", 2024-09-16T00:10:47.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187542+0000", 2024-09-16T00:10:47.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.0999999999999998e-07 2024-09-16T00:10:47.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.818 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.818 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.818 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187545+0000", 2024-09-16T00:10:47.818 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.6620000000000001e-06 2024-09-16T00:10:47.818 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187552+0000", 2024-09-16T00:10:47.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.4030000000000001e-06 2024-09-16T00:10:47.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.819 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.187558+0000", 2024-09-16T00:10:47.819 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.5289999999999999e-06 2024-09-16T00:10:47.819 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.819 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.819 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.188067+0000", 2024-09-16T00:10:47.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00050960000000000003 2024-09-16T00:10:47.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.188230+0000", 2024-09-16T00:10:47.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000162275 2024-09-16T00:10:47.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.820 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.820 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 4", 2024-09-16T00:10:47.820 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.188323+0000", 2024-09-16T00:10:47.820 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3778999999999998e-05 2024-09-16T00:10:47.820 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.189485+0000", 2024-09-16T00:10:47.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0011611659999999999 2024-09-16T00:10:47.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199354+0000", 2024-09-16T00:10:47.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0098691790000000005 2024-09-16T00:10:47.821 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.821 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.821 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.821 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199404+0000", 2024-09-16T00:10:47.821 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.041e-05 2024-09-16T00:10:47.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.822 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.822 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199454+0000", 2024-09-16T00:10:47.822 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9910999999999997e-05 2024-09-16T00:10:47.822 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.822 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.822 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.822 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.822 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.822 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:13491 3.26 3:66db0c90:::10000000f05.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e112)", 2024-09-16T00:10:47.822 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:26.199350+0000", 2024-09-16T00:10:47.822 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 21.534155036000001, 2024-09-16T00:10:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.009654167, 2024-09-16T00:10:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199350+0000", 2024-09-16T00:10:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199350+0000", 2024-09-16T00:10:47.823 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199352+0000", 2024-09-16T00:10:47.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.621e-06 2024-09-16T00:10:47.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.824 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199361+0000", 2024-09-16T00:10:47.824 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.0669999999999998e-06 2024-09-16T00:10:47.824 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.824 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.824 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199375+0000", 2024-09-16T00:10:47.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4474999999999999e-05 2024-09-16T00:10:47.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199383+0000", 2024-09-16T00:10:47.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.8250000000000005e-06 2024-09-16T00:10:47.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.825 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.825 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199500+0000", 2024-09-16T00:10:47.825 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00011675099999999999 2024-09-16T00:10:47.825 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.825 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199662+0000", 2024-09-16T00:10:47.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000161955 2024-09-16T00:10:47.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 7", 2024-09-16T00:10:47.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.199749+0000", 2024-09-16T00:10:47.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7498999999999997e-05 2024-09-16T00:10:47.826 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.826 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.826 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.826 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.202268+0000", 2024-09-16T00:10:47.826 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0025186829999999999 2024-09-16T00:10:47.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.208868+0000", 2024-09-16T00:10:47.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0066001870000000004 2024-09-16T00:10:47.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.208962+0000", 2024-09-16T00:10:47.827 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 9.3434e-05 2024-09-16T00:10:47.827 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.827 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.827 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.827 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:26.209004+0000", 2024-09-16T00:10:47.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.2670000000000003e-05 2024-09-16T00:10:47.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.828 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.828 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.828 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.828 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.828 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:16859 3.35 3:ad0b5202:::1000000245b.00000000:head [delete] snapc 1=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116)", 2024-09-16T00:10:47.828 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:36.173717+0000", 2024-09-16T00:10:47.828 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.559787778, 2024-09-16T00:10:47.828 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0097368839999999995, 2024-09-16T00:10:47.828 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.828 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.828 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.829 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173717+0000", 2024-09-16T00:10:47.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.829 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.829 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.829 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173717+0000", 2024-09-16T00:10:47.829 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.829 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173718+0000", 2024-09-16T00:10:47.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.01e-07 2024-09-16T00:10:47.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173720+0000", 2024-09-16T00:10:47.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 2.2029999999999999e-06 2024-09-16T00:10:47.830 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.830 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.830 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.830 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173727+0000", 2024-09-16T00:10:47.830 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.6209999999999998e-06 2024-09-16T00:10:47.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173733+0000", 2024-09-16T00:10:47.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.4960000000000001e-06 2024-09-16T00:10:47.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.831 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173787+0000", 2024-09-16T00:10:47.831 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.4354999999999997e-05 2024-09-16T00:10:47.831 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.831 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.831 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.173953+0000", 2024-09-16T00:10:47.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00016555000000000001 2024-09-16T00:10:47.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "waiting for subops from 5", 2024-09-16T00:10:47.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.174041+0000", 2024-09-16T00:10:47.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7738999999999995e-05 2024-09-16T00:10:47.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.832 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.832 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "op_commit", 2024-09-16T00:10:47.832 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.181641+0000", 2024-09-16T00:10:47.832 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.007599939 2024-09-16T00:10:47.832 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "sub_op_commit_rec", 2024-09-16T00:10:47.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.183355+0000", 2024-09-16T00:10:47.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0017146869999999999 2024-09-16T00:10:47.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.183404+0000", 2024-09-16T00:10:47.833 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.8999000000000003e-05 2024-09-16T00:10:47.833 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.833 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.833 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.833 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.183454+0000", 2024-09-16T00:10:47.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.9894e-05 2024-09-16T00:10:47.834 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.834 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.834 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.834 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.834 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.834 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_repop(mds.0.4:16912 3.1e e116/94 3:79d63390:::1000000249f.00000000:head v 116'374, mlcod=116'372)", 2024-09-16T00:10:47.834 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:36.184345+0000", 2024-09-16T00:10:47.834 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.549160133999999, 2024-09-16T00:10:47.834 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0093746300000000001, 2024-09-16T00:10:47.834 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.834 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.834 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T00:10:47.834 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184345+0000", 2024-09-16T00:10:47.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184345+0000", 2024-09-16T00:10:47.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.835 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.835 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.835 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.835 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184346+0000", 2024-09-16T00:10:47.835 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 7.6000000000000003e-07 2024-09-16T00:10:47.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184351+0000", 2024-09-16T00:10:47.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 5.3530000000000002e-06 2024-09-16T00:10:47.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.836 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184360+0000", 2024-09-16T00:10:47.836 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.9579999999999996e-06 2024-09-16T00:10:47.836 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.836 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.836 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184366+0000", 2024-09-16T00:10:47.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.1279999999999996e-06 2024-09-16T00:10:47.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184704+0000", 2024-09-16T00:10:47.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.00033787599999999997 2024-09-16T00:10:47.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.837 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.837 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.184722+0000", 2024-09-16T00:10:47.837 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.7533e-05 2024-09-16T00:10:47.837 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.837 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "commit_sent", 2024-09-16T00:10:47.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.193676+0000", 2024-09-16T00:10:47.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0089544919999999997 2024-09-16T00:10:47.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.838 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.838 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.193720+0000", 2024-09-16T00:10:47.838 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 4.3529999999999998e-05 2024-09-16T00:10:47.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.838 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.838 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.838 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.838 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.839 INFO:teuthology.orchestra.run.smithi164.stdout: "description": "osd_op(mds.0.4:17004 2.17 2:eb27e31b:::1000000255e.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e116)", 2024-09-16T00:10:47.839 INFO:teuthology.orchestra.run.smithi164.stdout: "initiated_at": "2024-09-16T00:10:36.199476+0000", 2024-09-16T00:10:47.839 INFO:teuthology.orchestra.run.smithi164.stdout: "age": 11.53402947, 2024-09-16T00:10:47.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0094096179999999998, 2024-09-16T00:10:47.839 INFO:teuthology.orchestra.run.smithi164.stdout: "continuous": false, 2024-09-16T00:10:47.839 INFO:teuthology.orchestra.run.smithi164.stdout: "type_data": { 2024-09-16T00:10:47.839 INFO:teuthology.orchestra.run.smithi164.stdout: "flag_point": "started", 2024-09-16T00:10:47.839 INFO:teuthology.orchestra.run.smithi164.stdout: "events": [ 2024-09-16T00:10:47.839 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.839 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "initiated", 2024-09-16T00:10:47.839 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199476+0000", 2024-09-16T00:10:47.839 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.839 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "header_read", 2024-09-16T00:10:47.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199476+0000", 2024-09-16T00:10:47.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0 2024-09-16T00:10:47.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "throttled", 2024-09-16T00:10:47.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199477+0000", 2024-09-16T00:10:47.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.4619999999999999e-06 2024-09-16T00:10:47.840 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.840 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.840 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "all_read", 2024-09-16T00:10:47.840 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199484+0000", 2024-09-16T00:10:47.840 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.438e-06 2024-09-16T00:10:47.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "dispatched", 2024-09-16T00:10:47.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199499+0000", 2024-09-16T00:10:47.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 1.5512e-05 2024-09-16T00:10:47.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "queued_for_pg", 2024-09-16T00:10:47.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199508+0000", 2024-09-16T00:10:47.841 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 8.7360000000000007e-06 2024-09-16T00:10:47.841 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.841 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.841 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "reached_pg", 2024-09-16T00:10:47.841 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199570+0000", 2024-09-16T00:10:47.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 6.2130000000000003e-05 2024-09-16T00:10:47.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "started", 2024-09-16T00:10:47.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.199691+0000", 2024-09-16T00:10:47.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.000121271 2024-09-16T00:10:47.842 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2024-09-16T00:10:47.842 INFO:teuthology.orchestra.run.smithi164.stdout: { 2024-09-16T00:10:47.842 INFO:teuthology.orchestra.run.smithi164.stdout: "event": "done", 2024-09-16T00:10:47.842 INFO:teuthology.orchestra.run.smithi164.stdout: "time": "2024-09-16T00:10:36.208885+0000", 2024-09-16T00:10:47.842 INFO:teuthology.orchestra.run.smithi164.stdout: "duration": 0.0091940689999999992 2024-09-16T00:10:47.842 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.842 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.842 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.843 INFO:teuthology.orchestra.run.smithi164.stdout: } 2024-09-16T00:10:47.843 INFO:teuthology.orchestra.run.smithi164.stdout: ] 2024-09-16T00:10:47.843 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:47.843 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-16T00:10:47.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.955+0000 7f74d4568640 1 Processor -- start 2024-09-16T00:10:47.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.955+0000 7f74d4568640 1 -- start start 2024-09-16T00:10:47.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.956+0000 7f74d4568640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f74cc151b50 0x7f74cc0729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:47.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.956+0000 7f74d4568640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74cc0c37a0 con 0x7f74cc0c98c0 2024-09-16T00:10:47.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.956+0000 7f74d4568640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74cc0c3a80 con 0x7f74cc157e10 2024-09-16T00:10:47.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.956+0000 7f74d4568640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f74cc0c7380 con 0x7f74cc151b50 2024-09-16T00:10:47.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.956+0000 7f74d2304640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f74cc151b50 0x7f74cc0729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:47.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.956+0000 7f74d1b03640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f74cc0c98c0 0x7f74cc157700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:58106/0 (socket says 172.21.15.139:58106) 2024-09-16T00:10:47.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.956+0000 7f74d2304640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f74cc151b50 0x7f74cc0729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38982/0 (socket says 172.21.15.139:38982) 2024-09-16T00:10:47.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.956+0000 7f74d1b03640 1 -- 172.21.15.139:0/4081775857 learned_addr learned my addr 172.21.15.139:0/4081775857 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:47.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.956+0000 7f74d2304640 1 -- 172.21.15.139:0/4081775857 >> v1:172.21.15.164:6789/0 conn(0x7f74cc157e10 legacy=0x7f74cc161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:47.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.956+0000 7f74d2304640 1 -- 172.21.15.139:0/4081775857 >> v1:172.21.15.139:6789/0 conn(0x7f74cc0c98c0 legacy=0x7f74cc157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:47.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.956+0000 7f74d2304640 1 -- 172.21.15.139:0/4081775857 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74cc0c7660 con 0x7f74cc151b50 2024-09-16T00:10:47.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.956+0000 7f74d1302640 1 -- 172.21.15.139:0/4081775857 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2415001706 0 0) 0x7f74cc0c37a0 con 0x7f74cc0c98c0 2024-09-16T00:10:47.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.957+0000 7f74d2304640 1 --2- 172.21.15.139:0/4081775857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74cc151b50 0x7f74cc0729c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f74c80097a0 tx=0x7f74c802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad40cdf53e22a6b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:47.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.957+0000 7f74d1302640 1 -- 172.21.15.139:0/4081775857 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8037070 con 0x7f74cc151b50 2024-09-16T00:10:47.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.957+0000 7f74d1302640 1 -- 172.21.15.139:0/4081775857 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c8032640 con 0x7f74cc151b50 2024-09-16T00:10:47.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.957+0000 7f74d1302640 1 -- 172.21.15.139:0/4081775857 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8032a10 con 0x7f74cc151b50 2024-09-16T00:10:47.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.958+0000 7f74d4568640 1 -- 172.21.15.139:0/4081775857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74cc151b50 msgr2=0x7f74cc0729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:47.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.958+0000 7f74d4568640 1 --2- 172.21.15.139:0/4081775857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74cc151b50 0x7f74cc0729c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f74c80097a0 tx=0x7f74c802a2c0 comp rx=0 tx=0).stop 2024-09-16T00:10:47.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.958+0000 7f74d4568640 1 -- 172.21.15.139:0/4081775857 shutdown_connections 2024-09-16T00:10:47.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.958+0000 7f74d4568640 1 --2- 172.21.15.139:0/4081775857 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74cc151b50 0x7f74cc0729c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:47.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.958+0000 7f74d4568640 1 -- 172.21.15.139:0/4081775857 >> 172.21.15.139:0/4081775857 conn(0x7f74cc06da40 msgr2=0x7f74cc06f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:47.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.958+0000 7f74d4568640 1 -- 172.21.15.139:0/4081775857 shutdown_connections 2024-09-16T00:10:47.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.958+0000 7f74d4568640 1 -- 172.21.15.139:0/4081775857 wait complete. 2024-09-16T00:10:47.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.959+0000 7f74d4568640 1 Processor -- start 2024-09-16T00:10:47.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.959+0000 7f74d4568640 1 -- start start 2024-09-16T00:10:47.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d4568640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc0c98c0 0x7f74cc1f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:47.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d4568640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc151b50 0x7f74cc148a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:47.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d4568640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74cc157e10 0x7f74cc1ede00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:47.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d4568640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc0c6490 con 0x7f74cc157e10 2024-09-16T00:10:47.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d4568640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74cc0c6310 con 0x7f74cc0c98c0 2024-09-16T00:10:47.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d4568640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f74cc0c6610 con 0x7f74cc151b50 2024-09-16T00:10:47.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d2b05640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74cc157e10 0x7f74cc1ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:47.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d2b05640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74cc157e10 0x7f74cc1ede00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:38998/0 (socket says 172.21.15.139:38998) 2024-09-16T00:10:47.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d2b05640 1 -- 172.21.15.139:0/621861679 learned_addr learned my addr 172.21.15.139:0/621861679 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:47.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d2304640 1 --2- 172.21.15.139:0/621861679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc0c98c0 0x7f74cc1f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d1b03640 1 --2- 172.21.15.139:0/621861679 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc151b50 0x7f74cc148a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d2b05640 1 -- 172.21.15.139:0/621861679 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc151b50 msgr2=0x7f74cc148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d2b05640 1 --2- 172.21.15.139:0/621861679 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc151b50 0x7f74cc148a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d2b05640 1 -- 172.21.15.139:0/621861679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc0c98c0 msgr2=0x7f74cc1f1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d2b05640 1 --2- 172.21.15.139:0/621861679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc0c98c0 0x7f74cc1f1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d2b05640 1 -- 172.21.15.139:0/621861679 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74cc1e3100 con 0x7f74cc157e10 2024-09-16T00:10:47.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d2b05640 1 --2- 172.21.15.139:0/621861679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74cc157e10 0x7f74cc1ede00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f74c400c7d0 tx=0x7f74c400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74c2ffd640 1 -- 172.21.15.139:0/621861679 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4003b70 con 0x7f74cc157e10 2024-09-16T00:10:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74c2ffd640 1 -- 172.21.15.139:0/621861679 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c4003d30 con 0x7f74cc157e10 2024-09-16T00:10:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d4568640 1 -- 172.21.15.139:0/621861679 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74cc0cad40 con 0x7f74cc157e10 2024-09-16T00:10:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74c2ffd640 1 -- 172.21.15.139:0/621861679 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c400aa90 con 0x7f74cc157e10 2024-09-16T00:10:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.960+0000 7f74d4568640 1 -- 172.21.15.139:0/621861679 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74cc1f1650 con 0x7f74cc157e10 2024-09-16T00:10:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.961+0000 7f74d4568640 1 -- 172.21.15.139:0/621861679 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74cc0c3c50 con 0x7f74cc157e10 2024-09-16T00:10:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.962+0000 7f74c2ffd640 1 -- 172.21.15.139:0/621861679 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f74c400ac30 con 0x7f74cc157e10 2024-09-16T00:10:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.962+0000 7f74c2ffd640 1 --2- 172.21.15.139:0/621861679 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f74b006c2e0 0x7f74b006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.962+0000 7f74c2ffd640 1 -- 172.21.15.139:0/621861679 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5944+0+0 (secure 0 0 0) 0x7f74c408b310 con 0x7f74cc157e10 2024-09-16T00:10:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.962+0000 7f74d2304640 1 --2- 172.21.15.139:0/621861679 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f74b006c2e0 0x7f74b006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:47.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.963+0000 7f74d2304640 1 --2- 172.21.15.139:0/621861679 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f74b006c2e0 0x7f74b006e7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f74c8034000 tx=0x7f74c802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:47.970 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:47.967+0000 7f74c2ffd640 1 -- 172.21.15.139:0/621861679 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f74cc0c3c50 con 0x7f74cc157e10 2024-09-16T00:10:48.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.141+0000 7f74d4568640 1 -- 172.21.15.139:0/621861679 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f74cc0c6c00 con 0x7f74cc157e10 2024-09-16T00:10:48.721 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:48.719+0000 7fd8f0330640 -1 Falling back to public interface 2024-09-16T00:10:48.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.796+0000 7f74c2ffd640 1 -- 172.21.15.139:0/621861679 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v124) ==== 91+0+0 (secure 0 0 0) 0x7f74c4059340 con 0x7f74cc157e10 2024-09-16T00:10:48.797 INFO:teuthology.orchestra.run.smithi139.stderr:noscrub is unset 2024-09-16T00:10:48.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.799+0000 7f74d4568640 1 -- 172.21.15.139:0/621861679 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f74b006c2e0 msgr2=0x7f74b006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:48.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.799+0000 7f74d4568640 1 --2- 172.21.15.139:0/621861679 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f74b006c2e0 0x7f74b006e7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f74c8034000 tx=0x7f74c802e000 comp rx=0 tx=0).stop 2024-09-16T00:10:48.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.799+0000 7f74d4568640 1 -- 172.21.15.139:0/621861679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74cc157e10 msgr2=0x7f74cc1ede00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:48.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.799+0000 7f74d4568640 1 --2- 172.21.15.139:0/621861679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74cc157e10 0x7f74cc1ede00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f74c400c7d0 tx=0x7f74c400cc90 comp rx=0 tx=0).stop 2024-09-16T00:10:48.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.799+0000 7f74d4568640 1 -- 172.21.15.139:0/621861679 shutdown_connections 2024-09-16T00:10:48.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.799+0000 7f74d4568640 1 --2- 172.21.15.139:0/621861679 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f74b006c2e0 0x7f74b006e7a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.799+0000 7f74d4568640 1 --2- 172.21.15.139:0/621861679 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74cc157e10 0x7f74cc1ede00 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.799+0000 7f74d4568640 1 --2- 172.21.15.139:0/621861679 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74cc151b50 0x7f74cc148a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.799+0000 7f74d4568640 1 --2- 172.21.15.139:0/621861679 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74cc0c98c0 0x7f74cc1f1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.799+0000 7f74d4568640 1 -- 172.21.15.139:0/621861679 >> 172.21.15.139:0/621861679 conn(0x7f74cc06da40 msgr2=0x7f74cc0c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:48.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.799+0000 7f74d4568640 1 -- 172.21.15.139:0/621861679 shutdown_connections 2024-09-16T00:10:48.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.799+0000 7f74d4568640 1 -- 172.21.15.139:0/621861679 wait complete. 2024-09-16T00:10:48.813 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-16T00:10:48.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.915+0000 7f4a4d6ba640 1 Processor -- start 2024-09-16T00:10:48.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.915+0000 7f4a4d6ba640 1 -- start start 2024-09-16T00:10:48.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.916+0000 7f4a4d6ba640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4a481527f0 0x7f4a480c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:48.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.916+0000 7f4a4d6ba640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4a480c8250 0x7f4a48158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:48.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.916+0000 7f4a4d6ba640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a480707f0 con 0x7f4a4806f7f0 2024-09-16T00:10:48.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.916+0000 7f4a4d6ba640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4a4806ff20 con 0x7f4a481527f0 2024-09-16T00:10:48.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.916+0000 7f4a4d6ba640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f4a48070170 con 0x7f4a480c8250 2024-09-16T00:10:48.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.916+0000 7f4a46ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4a481527f0 0x7f4a480c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:48.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.916+0000 7f4a46ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4a481527f0 0x7f4a480c4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39014/0 (socket says 172.21.15.139:39014) 2024-09-16T00:10:48.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.916+0000 7f4a46ffd640 1 -- 172.21.15.139:0/2978670797 learned_addr learned my addr 172.21.15.139:0/2978670797 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:48.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.916+0000 7f4a477fe640 1 --2- 172.21.15.139:0/2978670797 >> v2:172.21.15.164:3300/0 conn(0x7f4a480c8250 0x7f4a48158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:48.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.916+0000 7f4a46ffd640 1 -- 172.21.15.139:0/2978670797 >> v2:172.21.15.164:3300/0 conn(0x7f4a480c8250 msgr2=0x7f4a48158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:48.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.916+0000 7f4a46ffd640 1 --2- 172.21.15.139:0/2978670797 >> v2:172.21.15.164:3300/0 conn(0x7f4a480c8250 0x7f4a48158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.916+0000 7f4a46ffd640 1 -- 172.21.15.139:0/2978670797 >> v1:172.21.15.164:6789/0 conn(0x7f4a4806f7f0 legacy=0x7f4a480ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:48.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.916+0000 7f4a46ffd640 1 -- 172.21.15.139:0/2978670797 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a48070a90 con 0x7f4a481527f0 2024-09-16T00:10:48.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.917+0000 7f4a46ffd640 1 --2- 172.21.15.139:0/2978670797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a481527f0 0x7f4a480c4e80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4a4815cbf0 tx=0x7f4a3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d727ca778e20e5d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:48.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.917+0000 7f4a45ffb640 1 -- 172.21.15.139:0/2978670797 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38037070 con 0x7f4a481527f0 2024-09-16T00:10:48.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.917+0000 7f4a45ffb640 1 -- 172.21.15.139:0/2978670797 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a38032440 con 0x7f4a481527f0 2024-09-16T00:10:48.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.917+0000 7f4a45ffb640 1 -- 172.21.15.139:0/2978670797 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38032810 con 0x7f4a481527f0 2024-09-16T00:10:48.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.918+0000 7f4a4d6ba640 1 -- 172.21.15.139:0/2978670797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a481527f0 msgr2=0x7f4a480c4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:48.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.918+0000 7f4a4d6ba640 1 --2- 172.21.15.139:0/2978670797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a481527f0 0x7f4a480c4e80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4a4815cbf0 tx=0x7f4a3802a1e0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.918+0000 7f4a4d6ba640 1 -- 172.21.15.139:0/2978670797 shutdown_connections 2024-09-16T00:10:48.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.918+0000 7f4a4d6ba640 1 --2- 172.21.15.139:0/2978670797 >> v2:172.21.15.164:3300/0 conn(0x7f4a480c8250 0x7f4a48158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.918+0000 7f4a4d6ba640 1 --2- 172.21.15.139:0/2978670797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a481527f0 0x7f4a480c4e80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.918+0000 7f4a4d6ba640 1 -- 172.21.15.139:0/2978670797 >> 172.21.15.139:0/2978670797 conn(0x7f4a4806c150 msgr2=0x7f4a48067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:48.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.918+0000 7f4a4d6ba640 1 -- 172.21.15.139:0/2978670797 shutdown_connections 2024-09-16T00:10:48.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.918+0000 7f4a4d6ba640 1 -- 172.21.15.139:0/2978670797 wait complete. 2024-09-16T00:10:48.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.919+0000 7f4a4d6ba640 1 Processor -- start 2024-09-16T00:10:48.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.919+0000 7f4a4d6ba640 1 -- start start 2024-09-16T00:10:48.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a4d6ba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a4806f7f0 0x7f4a4815a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:48.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a4d6ba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a480c8250 0x7f4a4815acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:48.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a4d6ba640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a481527f0 0x7f4a480c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:48.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a4d6ba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4a480ce7d0 con 0x7f4a4806f7f0 2024-09-16T00:10:48.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a4d6ba640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f4a48152bd0 con 0x7f4a480c8250 2024-09-16T00:10:48.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a4d6ba640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4a480ce950 con 0x7f4a481527f0 2024-09-16T00:10:48.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a46ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a4806f7f0 0x7f4a4815a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:48.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a46ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a4806f7f0 0x7f4a4815a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39024/0 (socket says 172.21.15.139:39024) 2024-09-16T00:10:48.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a46ffd640 1 -- 172.21.15.139:0/2037153778 learned_addr learned my addr 172.21.15.139:0/2037153778 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:48.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a477fe640 1 --2- 172.21.15.139:0/2037153778 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a481527f0 0x7f4a480c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:48.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a467fc640 1 --2- 172.21.15.139:0/2037153778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a480c8250 0x7f4a4815acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:48.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a46ffd640 1 -- 172.21.15.139:0/2037153778 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a481527f0 msgr2=0x7f4a480c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:48.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a46ffd640 1 --2- 172.21.15.139:0/2037153778 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a481527f0 0x7f4a480c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a46ffd640 1 -- 172.21.15.139:0/2037153778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a480c8250 msgr2=0x7f4a4815acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:48.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a46ffd640 1 --2- 172.21.15.139:0/2037153778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a480c8250 0x7f4a4815acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:48.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a46ffd640 1 -- 172.21.15.139:0/2037153778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a480c5040 con 0x7f4a4806f7f0 2024-09-16T00:10:48.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.920+0000 7f4a467fc640 1 --2- 172.21.15.139:0/2037153778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a480c8250 0x7f4a4815acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:10:48.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.921+0000 7f4a46ffd640 1 --2- 172.21.15.139:0/2037153778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a4806f7f0 0x7f4a4815a7b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4a38000c00 tx=0x7f4a380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:48.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.921+0000 7f4a2f7fe640 1 -- 172.21.15.139:0/2037153778 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38041020 con 0x7f4a4806f7f0 2024-09-16T00:10:48.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.921+0000 7f4a2f7fe640 1 -- 172.21.15.139:0/2037153778 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a380306c0 con 0x7f4a4806f7f0 2024-09-16T00:10:48.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.921+0000 7f4a2f7fe640 1 -- 172.21.15.139:0/2037153778 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38037040 con 0x7f4a4806f7f0 2024-09-16T00:10:48.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.921+0000 7f4a4d6ba640 1 -- 172.21.15.139:0/2037153778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a48157020 con 0x7f4a4806f7f0 2024-09-16T00:10:48.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.921+0000 7f4a4d6ba640 1 -- 172.21.15.139:0/2037153778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4815b7f0 con 0x7f4a4806f7f0 2024-09-16T00:10:48.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.922+0000 7f4a4d6ba640 1 -- 172.21.15.139:0/2037153778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a48158580 con 0x7f4a4806f7f0 2024-09-16T00:10:48.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.922+0000 7f4a2f7fe640 1 -- 172.21.15.139:0/2037153778 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4a38044020 con 0x7f4a4806f7f0 2024-09-16T00:10:48.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.923+0000 7f4a2f7fe640 1 --2- 172.21.15.139:0/2037153778 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4a3406c2e0 0x7f4a3406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:48.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.923+0000 7f4a2f7fe640 1 -- 172.21.15.139:0/2037153778 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5944+0+0 (secure 0 0 0) 0x7f4a380abc50 con 0x7f4a4806f7f0 2024-09-16T00:10:48.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.923+0000 7f4a467fc640 1 --2- 172.21.15.139:0/2037153778 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4a3406c2e0 0x7f4a3406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:48.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.923+0000 7f4a467fc640 1 --2- 172.21.15.139:0/2037153778 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4a3406c2e0 0x7f4a3406e7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4a480cd4d0 tx=0x7f4a3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:48.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:48.927+0000 7f4a2f7fe640 1 -- 172.21.15.139:0/2037153778 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4a380b1020 con 0x7f4a4806f7f0 2024-09-16T00:10:49.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:49.102+0000 7f4a4d6ba640 1 -- 172.21.15.139:0/2037153778 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f4a480755e0 con 0x7f4a4806f7f0 2024-09-16T00:10:49.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:49.797+0000 7f4a2f7fe640 1 -- 172.21.15.139:0/2037153778 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v125) ==== 101+0+0 (secure 0 0 0) 0x7f4a3804b030 con 0x7f4a4806f7f0 2024-09-16T00:10:49.798 INFO:teuthology.orchestra.run.smithi139.stderr:nodeep-scrub is unset 2024-09-16T00:10:49.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:49.800+0000 7f4a4d6ba640 1 -- 172.21.15.139:0/2037153778 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4a3406c2e0 msgr2=0x7f4a3406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:49.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:49.800+0000 7f4a4d6ba640 1 --2- 172.21.15.139:0/2037153778 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4a3406c2e0 0x7f4a3406e7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4a480cd4d0 tx=0x7f4a3c009040 comp rx=0 tx=0).stop 2024-09-16T00:10:49.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:49.800+0000 7f4a4d6ba640 1 -- 172.21.15.139:0/2037153778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a4806f7f0 msgr2=0x7f4a4815a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:49.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:49.800+0000 7f4a4d6ba640 1 --2- 172.21.15.139:0/2037153778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a4806f7f0 0x7f4a4815a7b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4a38000c00 tx=0x7f4a380046c0 comp rx=0 tx=0).stop 2024-09-16T00:10:49.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:49.800+0000 7f4a4d6ba640 1 -- 172.21.15.139:0/2037153778 shutdown_connections 2024-09-16T00:10:49.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:49.800+0000 7f4a4d6ba640 1 --2- 172.21.15.139:0/2037153778 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f4a3406c2e0 0x7f4a3406e7a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:49.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:49.800+0000 7f4a4d6ba640 1 --2- 172.21.15.139:0/2037153778 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4a481527f0 0x7f4a480c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:49.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:49.800+0000 7f4a4d6ba640 1 --2- 172.21.15.139:0/2037153778 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4a480c8250 0x7f4a4815acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:49.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:49.800+0000 7f4a4d6ba640 1 --2- 172.21.15.139:0/2037153778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a4806f7f0 0x7f4a4815a7b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:49.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:49.800+0000 7f4a4d6ba640 1 -- 172.21.15.139:0/2037153778 >> 172.21.15.139:0/2037153778 conn(0x7f4a4806c150 msgr2=0x7f4a480cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:49.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:49.800+0000 7f4a4d6ba640 1 -- 172.21.15.139:0/2037153778 shutdown_connections 2024-09-16T00:10:49.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:49.801+0000 7f4a4d6ba640 1 -- 172.21.15.139:0/2037153778 wait complete. 2024-09-16T00:10:52.661 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:10:52.776 INFO:teuthology.orchestra.run.smithi164.stderr:no valid command found; 10 closest matches: 2024-09-16T00:10:52.777 INFO:teuthology.orchestra.run.smithi164.stderr:0 2024-09-16T00:10:52.777 INFO:teuthology.orchestra.run.smithi164.stderr:1 2024-09-16T00:10:52.777 INFO:teuthology.orchestra.run.smithi164.stderr:2 2024-09-16T00:10:52.777 INFO:teuthology.orchestra.run.smithi164.stderr:abort 2024-09-16T00:10:52.777 INFO:teuthology.orchestra.run.smithi164.stderr:assert 2024-09-16T00:10:52.777 INFO:teuthology.orchestra.run.smithi164.stderr:bluefs debug_inject_read_zeros 2024-09-16T00:10:52.777 INFO:teuthology.orchestra.run.smithi164.stderr:bluefs files list 2024-09-16T00:10:52.777 INFO:teuthology.orchestra.run.smithi164.stderr:bluefs stats 2024-09-16T00:10:52.777 INFO:teuthology.orchestra.run.smithi164.stderr:bluestore allocator dump block 2024-09-16T00:10:52.777 INFO:teuthology.orchestra.run.smithi164.stderr:bluestore allocator fragmentation block 2024-09-16T00:10:52.777 INFO:teuthology.orchestra.run.smithi164.stderr:admin_socket: invalid command 2024-09-16T00:10:52.779 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T00:10:52.779 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2024-09-16T00:10:52.830 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:52.830+0000 7fd8f0330640 -1 osd.7 112 log_to_monitors true 2024-09-16T00:10:53.820 INFO:tasks.ceph.osd.7.smithi164.stderr:2024-09-16T00:10:53.820+0000 7fd8e40df640 -1 osd.7 112 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:10:57.781 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-16T00:10:57.902 INFO:teuthology.orchestra.run.smithi164.stdout:{ 2024-09-16T00:10:57.902 INFO:teuthology.orchestra.run.smithi164.stdout: "ops": [], 2024-09-16T00:10:57.902 INFO:teuthology.orchestra.run.smithi164.stdout: "num_ops": 0 2024-09-16T00:10:57.902 INFO:teuthology.orchestra.run.smithi164.stdout:} 2024-09-16T00:10:57.911 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-16T00:10:58.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.019+0000 7f3d224c0640 1 Processor -- start 2024-09-16T00:10:58.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.019+0000 7f3d224c0640 1 -- start start 2024-09-16T00:10:58.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.019+0000 7f3d224c0640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3d1c0d0490 0x7f3d1c0b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.019+0000 7f3d224c0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0b4750 0x7f3d1c1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.019+0000 7f3d224c0640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d1c0b3180 con 0x7f3d1c0d0870 2024-09-16T00:10:58.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.019+0000 7f3d224c0640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f3d1c15b940 con 0x7f3d1c0d0490 2024-09-16T00:10:58.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.019+0000 7f3d224c0640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3d1c0b3bb0 con 0x7f3d1c0b4750 2024-09-16T00:10:58.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.020+0000 7f3d20a5d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0b4750 0x7f3d1c1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.020+0000 7f3d20a5d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0b4750 0x7f3d1c1546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:52172/0 (socket says 172.21.15.139:52172) 2024-09-16T00:10:58.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.020+0000 7f3d20a5d640 1 -- 172.21.15.139:0/1585285051 learned_addr learned my addr 172.21.15.139:0/1585285051 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:58.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.020+0000 7f3d1bfff640 1 --2- 172.21.15.139:0/1585285051 >> v2:172.21.15.164:3300/0 conn(0x7f3d1c0d0490 0x7f3d1c0b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.020+0000 7f3d20a5d640 1 -- 172.21.15.139:0/1585285051 >> v2:172.21.15.164:3300/0 conn(0x7f3d1c0d0490 msgr2=0x7f3d1c0b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:58.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.020+0000 7f3d20a5d640 1 --2- 172.21.15.139:0/1585285051 >> v2:172.21.15.164:3300/0 conn(0x7f3d1c0d0490 0x7f3d1c0b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.020+0000 7f3d20a5d640 1 -- 172.21.15.139:0/1585285051 >> v1:172.21.15.164:6789/0 conn(0x7f3d1c0d0870 legacy=0x7f3d1c0b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.020+0000 7f3d20a5d640 1 -- 172.21.15.139:0/1585285051 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1c15adc0 con 0x7f3d1c0b4750 2024-09-16T00:10:58.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.020+0000 7f3d1b7fe640 1 -- 172.21.15.139:0/1585285051 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3404384451 0 0) 0x7f3d1c0b3180 con 0x7f3d1c0d0870 2024-09-16T00:10:58.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.021+0000 7f3d20a5d640 1 --2- 172.21.15.139:0/1585285051 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0b4750 0x7f3d1c1546a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3d14009e50 tx=0x7f3d1402a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a981798c40a28187 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:58.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.021+0000 7f3d1b7fe640 1 -- 172.21.15.139:0/1585285051 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d14036070 con 0x7f3d1c0b4750 2024-09-16T00:10:58.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.021+0000 7f3d1b7fe640 1 -- 172.21.15.139:0/1585285051 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d14031600 con 0x7f3d1c0b4750 2024-09-16T00:10:58.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.021+0000 7f3d1b7fe640 1 -- 172.21.15.139:0/1585285051 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d140319d0 con 0x7f3d1c0b4750 2024-09-16T00:10:58.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.022+0000 7f3d224c0640 1 -- 172.21.15.139:0/1585285051 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0b4750 msgr2=0x7f3d1c1546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.022+0000 7f3d224c0640 1 --2- 172.21.15.139:0/1585285051 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0b4750 0x7f3d1c1546a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3d14009e50 tx=0x7f3d1402a7a0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.022+0000 7f3d224c0640 1 -- 172.21.15.139:0/1585285051 shutdown_connections 2024-09-16T00:10:58.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.022+0000 7f3d224c0640 1 --2- 172.21.15.139:0/1585285051 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0b4750 0x7f3d1c1546a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.022+0000 7f3d224c0640 1 --2- 172.21.15.139:0/1585285051 >> v2:172.21.15.164:3300/0 conn(0x7f3d1c0d0490 0x7f3d1c0b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.022+0000 7f3d224c0640 1 -- 172.21.15.139:0/1585285051 >> 172.21.15.139:0/1585285051 conn(0x7f3d1c06c520 msgr2=0x7f3d1c06c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:58.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.022+0000 7f3d224c0640 1 -- 172.21.15.139:0/1585285051 shutdown_connections 2024-09-16T00:10:58.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.022+0000 7f3d224c0640 1 -- 172.21.15.139:0/1585285051 wait complete. 2024-09-16T00:10:58.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.023+0000 7f3d224c0640 1 Processor -- start 2024-09-16T00:10:58.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.023+0000 7f3d224c0640 1 -- start start 2024-09-16T00:10:58.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d224c0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c0b4750 0x7f3d1c149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d224c0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c0d0490 0x7f3d1c1f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d224c0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0d0870 0x7f3d1c1f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d224c0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c15b660 con 0x7f3d1c0b4750 2024-09-16T00:10:58.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d224c0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c15b380 con 0x7f3d1c0d0490 2024-09-16T00:10:58.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d224c0640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3d1c15a980 con 0x7f3d1c0d0870 2024-09-16T00:10:58.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d1bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c0b4750 0x7f3d1c149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d1bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c0b4750 0x7f3d1c149de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60268/0 (socket says 172.21.15.139:60268) 2024-09-16T00:10:58.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d1bfff640 1 -- 172.21.15.139:0/3146082272 learned_addr learned my addr 172.21.15.139:0/3146082272 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:58.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d20a5d640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0d0870 0x7f3d1c1f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d13fff640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c0d0490 0x7f3d1c1f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d1bfff640 1 -- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0d0870 msgr2=0x7f3d1c1f4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d1bfff640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0d0870 0x7f3d1c1f4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d1bfff640 1 -- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c0d0490 msgr2=0x7f3d1c1f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d1bfff640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c0d0490 0x7f3d1c1f4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d1bfff640 1 -- 172.21.15.139:0/3146082272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1c0c3de0 con 0x7f3d1c0b4750 2024-09-16T00:10:58.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d1bfff640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c0b4750 0x7f3d1c149de0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00e9e0 tx=0x7f3d0c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:58.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.024+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c017070 con 0x7f3d1c0b4750 2024-09-16T00:10:58.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.025+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d0c00c9d0 con 0x7f3d1c0b4750 2024-09-16T00:10:58.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.025+0000 7f3d224c0640 1 -- 172.21.15.139:0/3146082272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d1c1f3d60 con 0x7f3d1c0b4750 2024-09-16T00:10:58.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.025+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c00cdc0 con 0x7f3d1c0b4750 2024-09-16T00:10:58.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.025+0000 7f3d224c0640 1 -- 172.21.15.139:0/3146082272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d1c0c4620 con 0x7f3d1c0b4750 2024-09-16T00:10:58.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.025+0000 7f3d224c0640 1 -- 172.21.15.139:0/3146082272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3d1c0b3430 con 0x7f3d1c0b4750 2024-09-16T00:10:58.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.026+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3d0c01a070 con 0x7f3d1c0b4750 2024-09-16T00:10:58.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.027+0000 7f3d197fa640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3cfc06c2e0 0x7f3cfc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.027+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5944+0+0 (secure 0 0 0) 0x7f3d0c08b7f0 con 0x7f3d1c0b4750 2024-09-16T00:10:58.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.027+0000 7f3d13fff640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3cfc06c2e0 0x7f3cfc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.027+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1289 (secure 0 0 0) 0x7f3d0c08c5e0 con 0x7f3d1c0b4750 2024-09-16T00:10:58.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.027+0000 7f3d224c0640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c148210 0x7f3d1c0c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.027+0000 7f3d224c0640 1 -- 172.21.15.139:0/3146082272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c0d1200 con 0x7f3d1c148210 2024-09-16T00:10:58.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.027+0000 7f3d13fff640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3cfc06c2e0 0x7f3cfc06e7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0cc1b0 tx=0x7f3d04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:58.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.027+0000 7f3d20a5d640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c148210 0x7f3d1c0c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.027+0000 7f3d20a5d640 1 -- 172.21.15.139:0/3146082272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- command(tid 2: {"prefix": "get_command_descriptions"}) -- 0x7f3d1c151ce0 con 0x7f3d1c148210 2024-09-16T00:10:58.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.028+0000 7f3d20a5d640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c148210 0x7f3d1c0c4860 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0b5cc0 tx=0x7f3d14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:58.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.028+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d14040030 con 0x7f3d1c148210 2024-09-16T00:10:58.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.028+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.0 v2:172.21.15.139:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3d14037040 con 0x7f3d1c148210 2024-09-16T00:10:58.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.028+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c148210 msgr2=0x7f3d1c0c4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.028+0000 7f3d197fa640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c148210 0x7f3d1c0c4860 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0b5cc0 tx=0x7f3d14004a40 comp rx=0 tx=0).stop 2024-09-16T00:10:58.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.028+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c148210 msgr2=0x7f3d1c0c4860 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T00:10:58.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.028+0000 7f3d197fa640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c148210 0x7f3d1c0c4860 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.034+0000 7f3d224c0640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c0c2c10 0x7f3d1c0c3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.034+0000 7f3d224c0640 1 -- 172.21.15.139:0/3146082272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c0c3550 con 0x7f3d1c0c2c10 2024-09-16T00:10:58.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.034+0000 7f3d20a5d640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c0c2c10 0x7f3d1c0c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.034+0000 7f3d20a5d640 1 -- 172.21.15.139:0/3146082272 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- command(tid 3: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7f3d1c159bb0 con 0x7f3d1c0c2c10 2024-09-16T00:10:58.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.035+0000 7f3d20a5d640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c0c2c10 0x7f3d1c0c3010 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3d1402a7e0 tx=0x7f3d1402f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:58.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.035+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d14040070 con 0x7f3d1c0c2c10 2024-09-16T00:10:58.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.036+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.0 v2:172.21.15.139:3300/0 2 ==== command_reply(tid 3: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f3d14009af0 con 0x7f3d1c0c2c10 2024-09-16T00:10:58.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.036+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c0c2c10 msgr2=0x7f3d1c0c3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.036+0000 7f3d197fa640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c0c2c10 0x7f3d1c0c3010 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3d1402a7e0 tx=0x7f3d1402f000 comp rx=0 tx=0).stop 2024-09-16T00:10:58.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.036+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c0c2c10 msgr2=0x7f3d1c0c3010 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T00:10:58.038 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.036+0000 7f3d197fa640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c0c2c10 0x7f3d1c0c3010 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.039 INFO:teuthology.orchestra.run.smithi139.stdout:mon.a: {} 2024-09-16T00:10:58.040 INFO:teuthology.orchestra.run.smithi139.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T00:10:58.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.036+0000 7f3d224c0640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c1455a0 0x7f3d1c1459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.036+0000 7f3d224c0640 1 -- 172.21.15.139:0/3146082272 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c1598d0 con 0x7f3d1c1455a0 2024-09-16T00:10:58.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.037+0000 7f3d20a5d640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c1455a0 0x7f3d1c1459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.037+0000 7f3d20a5d640 1 -- 172.21.15.139:0/3146082272 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- command(tid 4: {"prefix": "get_command_descriptions"}) -- 0x7f3d1c1595f0 con 0x7f3d1c1455a0 2024-09-16T00:10:58.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.037+0000 7f3d20a5d640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c1455a0 0x7f3d1c1459b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c06ee10 tx=0x7f3d14004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:58.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.037+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1405c070 con 0x7f3d1c1455a0 2024-09-16T00:10:58.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.038+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.1 v2:172.21.15.164:3300/0 2 ==== command_reply(tid 4: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3d14061450 con 0x7f3d1c1455a0 2024-09-16T00:10:58.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.038+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c1455a0 msgr2=0x7f3d1c1459b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.038+0000 7f3d197fa640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c1455a0 0x7f3d1c1459b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c06ee10 tx=0x7f3d14004a40 comp rx=0 tx=0).stop 2024-09-16T00:10:58.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.038+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c1455a0 msgr2=0x7f3d1c1459b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T00:10:58.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.038+0000 7f3d20a5d640 1 -- 172.21.15.139:0/3146082272 reap_dead start 2024-09-16T00:10:58.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.038+0000 7f3d197fa640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c1455a0 0x7f3d1c1459b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.044+0000 7f3d224c0640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c145ef0 0x7f3d1c1f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.044+0000 7f3d224c0640 1 -- 172.21.15.139:0/3146082272 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1c14ed40 con 0x7f3d1c145ef0 2024-09-16T00:10:58.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.044+0000 7f3d20a5d640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c145ef0 0x7f3d1c1f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.044+0000 7f3d20a5d640 1 -- 172.21.15.139:0/3146082272 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- command(tid 5: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7f3d1c0008d0 con 0x7f3d1c145ef0 2024-09-16T00:10:58.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.044+0000 7f3d20a5d640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c145ef0 0x7f3d1c1f4240 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3d140093e0 tx=0x7f3d1404f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:58.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.044+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d14036070 con 0x7f3d1c145ef0 2024-09-16T00:10:58.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.046+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.1 v2:172.21.15.164:3300/0 2 ==== command_reply(tid 5: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f3d1406b420 con 0x7f3d1c145ef0 2024-09-16T00:10:58.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.046+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c145ef0 msgr2=0x7f3d1c1f4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.046+0000 7f3d197fa640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c145ef0 0x7f3d1c1f4240 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3d140093e0 tx=0x7f3d1404f000 comp rx=0 tx=0).stop 2024-09-16T00:10:58.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.046+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c145ef0 msgr2=0x7f3d1c1f4240 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T00:10:58.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.046+0000 7f3d197fa640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3d1c145ef0 0x7f3d1c1f4240 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.047 INFO:teuthology.orchestra.run.smithi139.stdout:mon.b: {} 2024-09-16T00:10:58.049 INFO:teuthology.orchestra.run.smithi139.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T00:10:58.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.046+0000 7f3d224c0640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c1f4780 0x7f3d1c0cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.046+0000 7f3d224c0640 1 -- 172.21.15.139:0/3146082272 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3d1c1f4b60 con 0x7f3d1c1f4780 2024-09-16T00:10:58.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.046+0000 7f3d20a5d640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c1f4780 0x7f3d1c0cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.047+0000 7f3d20a5d640 1 -- 172.21.15.139:0/3146082272 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f3d1c14a990 con 0x7f3d1c1f4780 2024-09-16T00:10:58.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.047+0000 7f3d20a5d640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c1f4780 0x7f3d1c0cedd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d14004af0 tx=0x7f3d1405a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:58.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.047+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d14036040 con 0x7f3d1c1f4780 2024-09-16T00:10:58.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.047+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.2 v2:172.21.15.164:3301/0 2 ==== command_reply(tid 6: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3d1406b8a0 con 0x7f3d1c1f4780 2024-09-16T00:10:58.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.047+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c1f4780 msgr2=0x7f3d1c0cedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.047+0000 7f3d197fa640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c1f4780 0x7f3d1c0cedd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d14004af0 tx=0x7f3d1405a000 comp rx=0 tx=0).stop 2024-09-16T00:10:58.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.047+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c1f4780 msgr2=0x7f3d1c0cedd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T00:10:58.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.047+0000 7f3d197fa640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c1f4780 0x7f3d1c0cedd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.053+0000 7f3d224c0640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0cf310 0x7f3d1c0cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.053+0000 7f3d224c0640 1 -- 172.21.15.139:0/3146082272 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3d1c14a320 con 0x7f3d1c0cf310 2024-09-16T00:10:58.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.053+0000 7f3d20a5d640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0cf310 0x7f3d1c0cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.054+0000 7f3d20a5d640 1 -- 172.21.15.139:0/3146082272 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7f3d1c1462d0 con 0x7f3d1c0cf310 2024-09-16T00:10:58.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.054+0000 7f3d20a5d640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0cf310 0x7f3d1c0cf6f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3d14004590 tx=0x7f3d14004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:58.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.054+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1407e070 con 0x7f3d1c0cf310 2024-09-16T00:10:58.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.055+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 <== mon.2 v2:172.21.15.164:3301/0 2 ==== command_reply(tid 7: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f3d14079240 con 0x7f3d1c0cf310 2024-09-16T00:10:58.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.055+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0cf310 msgr2=0x7f3d1c0cf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.055+0000 7f3d197fa640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0cf310 0x7f3d1c0cf6f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3d14004590 tx=0x7f3d14004a40 comp rx=0 tx=0).stop 2024-09-16T00:10:58.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.055+0000 7f3d197fa640 1 -- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0cf310 msgr2=0x7f3d1c0cf6f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-16T00:10:58.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.055+0000 7f3d197fa640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3d1c0cf310 0x7f3d1c0cf6f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.057 INFO:teuthology.orchestra.run.smithi139.stdout:mon.c: {} 2024-09-16T00:10:58.058 INFO:teuthology.orchestra.run.smithi139.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T00:10:58.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.055+0000 7f3d224c0640 1 -- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3cfc06c2e0 msgr2=0x7f3cfc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.055+0000 7f3d224c0640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3cfc06c2e0 0x7f3cfc06e7a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0cc1b0 tx=0x7f3d04009040 comp rx=0 tx=0).stop 2024-09-16T00:10:58.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.055+0000 7f3d224c0640 1 -- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c0b4750 msgr2=0x7f3d1c149de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.055+0000 7f3d224c0640 1 --2- 172.21.15.139:0/3146082272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d1c0b4750 0x7f3d1c149de0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c00e9e0 tx=0x7f3d0c00eea0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.056+0000 7f3d20a5d640 1 -- 172.21.15.139:0/3146082272 reap_dead start 2024-09-16T00:10:58.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.056+0000 7f3d224c0640 1 -- 172.21.15.139:0/3146082272 shutdown_connections 2024-09-16T00:10:58.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.056+0000 7f3d224c0640 1 -- 172.21.15.139:0/3146082272 >> 172.21.15.139:0/3146082272 conn(0x7f3d1c06c520 msgr2=0x7f3d1c0cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:58.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.056+0000 7f3d224c0640 1 -- 172.21.15.139:0/3146082272 shutdown_connections 2024-09-16T00:10:58.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.056+0000 7f3d224c0640 1 -- 172.21.15.139:0/3146082272 wait complete. 2024-09-16T00:10:58.069 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-16T00:10:58.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.171+0000 7fa12158d640 1 Processor -- start 2024-09-16T00:10:58.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.171+0000 7fa12158d640 1 -- start start 2024-09-16T00:10:58.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.171+0000 7fa12158d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa11c154c20 0x7fa11c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.171+0000 7fa12158d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa11c0ccc90 0x7fa11c0cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.171+0000 7fa12158d640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa11c0cc4b0 con 0x7fa11c068060 2024-09-16T00:10:58.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.171+0000 7fa12158d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa11c0c5830 con 0x7fa11c0ccc90 2024-09-16T00:10:58.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.171+0000 7fa12158d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa11c15b7c0 con 0x7fa11c154c20 2024-09-16T00:10:58.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.172+0000 7fa11b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa11c0ccc90 0x7fa11c0cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.172+0000 7fa11b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa11c0ccc90 0x7fa11c0cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60282/0 (socket says 172.21.15.139:60282) 2024-09-16T00:10:58.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.172+0000 7fa11b7fe640 1 -- 172.21.15.139:0/459993173 learned_addr learned my addr 172.21.15.139:0/459993173 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:58.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.172+0000 7fa11affd640 1 --2- 172.21.15.139:0/459993173 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa11c154c20 0x7fa11c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.172+0000 7fa11b7fe640 1 -- 172.21.15.139:0/459993173 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa11c154c20 msgr2=0x7fa11c15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:58.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.172+0000 7fa11b7fe640 1 --2- 172.21.15.139:0/459993173 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa11c154c20 0x7fa11c15b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.172+0000 7fa11b7fe640 1 -- 172.21.15.139:0/459993173 >> v1:172.21.15.164:6789/0 conn(0x7fa11c068060 legacy=0x7fa11c0c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:58.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.172+0000 7fa11b7fe640 1 -- 172.21.15.139:0/459993173 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11c0cc790 con 0x7fa11c0ccc90 2024-09-16T00:10:58.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.173+0000 7fa11b7fe640 1 --2- 172.21.15.139:0/459993173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa11c0ccc90 0x7fa11c0cd0e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa114009e50 tx=0x7fa11402a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f44cb687e048d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:58.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.173+0000 7fa119ffb640 1 -- 172.21.15.139:0/459993173 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114036070 con 0x7fa11c0ccc90 2024-09-16T00:10:58.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.173+0000 7fa119ffb640 1 -- 172.21.15.139:0/459993173 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa114031600 con 0x7fa11c0ccc90 2024-09-16T00:10:58.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.173+0000 7fa119ffb640 1 -- 172.21.15.139:0/459993173 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1140319d0 con 0x7fa11c0ccc90 2024-09-16T00:10:58.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.173+0000 7fa12158d640 1 -- 172.21.15.139:0/459993173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa11c0ccc90 msgr2=0x7fa11c0cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.173+0000 7fa12158d640 1 --2- 172.21.15.139:0/459993173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa11c0ccc90 0x7fa11c0cd0e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa114009e50 tx=0x7fa11402a750 comp rx=0 tx=0).stop 2024-09-16T00:10:58.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.173+0000 7fa12158d640 1 -- 172.21.15.139:0/459993173 shutdown_connections 2024-09-16T00:10:58.177 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.173+0000 7fa12158d640 1 --2- 172.21.15.139:0/459993173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa11c0ccc90 0x7fa11c0cd0e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.173+0000 7fa12158d640 1 --2- 172.21.15.139:0/459993173 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa11c154c20 0x7fa11c15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.173+0000 7fa12158d640 1 -- 172.21.15.139:0/459993173 >> 172.21.15.139:0/459993173 conn(0x7fa11c06ca40 msgr2=0x7fa11c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:58.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.173+0000 7fa12158d640 1 -- 172.21.15.139:0/459993173 shutdown_connections 2024-09-16T00:10:58.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.173+0000 7fa12158d640 1 -- 172.21.15.139:0/459993173 wait complete. 2024-09-16T00:10:58.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.175+0000 7fa12158d640 1 Processor -- start 2024-09-16T00:10:58.178 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.175+0000 7fa12158d640 1 -- start start 2024-09-16T00:10:58.180 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.175+0000 7fa12158d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa11c068060 0x7fa11c15a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.175+0000 7fa12158d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa11c0ccc90 0x7fa11c1f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.175+0000 7fa12158d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa11c154c20 0x7fa11c145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.175+0000 7fa12158d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c15e0d0 con 0x7fa11c068060 2024-09-16T00:10:58.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.175+0000 7fa12158d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c0cc260 con 0x7fa11c154c20 2024-09-16T00:10:58.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.175+0000 7fa12158d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa11c15e3b0 con 0x7fa11c0ccc90 2024-09-16T00:10:58.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.175+0000 7fa11affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa11c068060 0x7fa11c15a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.175+0000 7fa11affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa11c068060 0x7fa11c15a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60298/0 (socket says 172.21.15.139:60298) 2024-09-16T00:10:58.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.175+0000 7fa11affd640 1 -- 172.21.15.139:0/388630148 learned_addr learned my addr 172.21.15.139:0/388630148 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:58.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.176+0000 7fa11a7fc640 1 --2- 172.21.15.139:0/388630148 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa11c0ccc90 0x7fa11c1f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.176+0000 7fa11affd640 1 -- 172.21.15.139:0/388630148 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa11c0ccc90 msgr2=0x7fa11c1f4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.176+0000 7fa11b7fe640 1 --2- 172.21.15.139:0/388630148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa11c154c20 0x7fa11c145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.181 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.176+0000 7fa11affd640 1 --2- 172.21.15.139:0/388630148 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa11c0ccc90 0x7fa11c1f4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.176+0000 7fa11affd640 1 -- 172.21.15.139:0/388630148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa11c154c20 msgr2=0x7fa11c145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.176+0000 7fa11affd640 1 --2- 172.21.15.139:0/388630148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa11c154c20 0x7fa11c145b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.176+0000 7fa11affd640 1 -- 172.21.15.139:0/388630148 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11c1584e0 con 0x7fa11c068060 2024-09-16T00:10:58.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.176+0000 7fa11a7fc640 1 --2- 172.21.15.139:0/388630148 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa11c0ccc90 0x7fa11c1f4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:10:58.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.176+0000 7fa11affd640 1 --2- 172.21.15.139:0/388630148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa11c068060 0x7fa11c15a780 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00d5f0 tx=0x7fa10c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:58.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.176+0000 7fa1037fe640 1 -- 172.21.15.139:0/388630148 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0125f0 con 0x7fa11c068060 2024-09-16T00:10:58.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.176+0000 7fa1037fe640 1 -- 172.21.15.139:0/388630148 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa10c019070 con 0x7fa11c068060 2024-09-16T00:10:58.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.176+0000 7fa12158d640 1 -- 172.21.15.139:0/388630148 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11c1f4d20 con 0x7fa11c068060 2024-09-16T00:10:58.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.176+0000 7fa1037fe640 1 -- 172.21.15.139:0/388630148 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0129d0 con 0x7fa11c068060 2024-09-16T00:10:58.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.176+0000 7fa12158d640 1 -- 172.21.15.139:0/388630148 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa11c158a60 con 0x7fa11c068060 2024-09-16T00:10:58.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.177+0000 7fa12158d640 1 -- 172.21.15.139:0/388630148 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa11c153f90 con 0x7fa11c068060 2024-09-16T00:10:58.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.178+0000 7fa1037fe640 1 -- 172.21.15.139:0/388630148 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa10c012b70 con 0x7fa11c068060 2024-09-16T00:10:58.182 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.178+0000 7fa1037fe640 1 --2- 172.21.15.139:0/388630148 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa10806c2b0 0x7fa10806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.178+0000 7fa1037fe640 1 -- 172.21.15.139:0/388630148 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5944+0+0 (secure 0 0 0) 0x7fa10c08b8a0 con 0x7fa11c068060 2024-09-16T00:10:58.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.178+0000 7fa11a7fc640 1 --2- 172.21.15.139:0/388630148 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa10806c2b0 0x7fa10806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.179+0000 7fa11a7fc640 1 --2- 172.21.15.139:0/388630148 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa10806c2b0 0x7fa10806e770 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0cb7f0 tx=0x7fa110009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:58.183 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.183+0000 7fa1037fe640 1 -- 172.21.15.139:0/388630148 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa11c153f90 con 0x7fa11c068060 2024-09-16T00:10:58.357 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.356+0000 7fa12158d640 1 -- 172.21.15.139:0/388630148 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) -- 0x7fa11c15e690 con 0x7fa11c068060 2024-09-16T00:10:58.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.828+0000 7fa1037fe640 1 -- 172.21.15.139:0/388630148 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v131) ==== 113+0+0 (secure 0 0 0) 0x7fa10c059fb0 con 0x7fa11c068060 2024-09-16T00:10:58.829 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T00:10:58.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.831+0000 7fa12158d640 1 -- 172.21.15.139:0/388630148 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa10806c2b0 msgr2=0x7fa10806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.831+0000 7fa12158d640 1 --2- 172.21.15.139:0/388630148 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa10806c2b0 0x7fa10806e770 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0cb7f0 tx=0x7fa110009040 comp rx=0 tx=0).stop 2024-09-16T00:10:58.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.831+0000 7fa12158d640 1 -- 172.21.15.139:0/388630148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa11c068060 msgr2=0x7fa11c15a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.831+0000 7fa12158d640 1 --2- 172.21.15.139:0/388630148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa11c068060 0x7fa11c15a780 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00d5f0 tx=0x7fa10c00dab0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.831+0000 7fa12158d640 1 -- 172.21.15.139:0/388630148 shutdown_connections 2024-09-16T00:10:58.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.831+0000 7fa12158d640 1 --2- 172.21.15.139:0/388630148 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa10806c2b0 0x7fa10806e770 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.831+0000 7fa12158d640 1 --2- 172.21.15.139:0/388630148 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa11c154c20 0x7fa11c145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.831+0000 7fa12158d640 1 --2- 172.21.15.139:0/388630148 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa11c0ccc90 0x7fa11c1f4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.831+0000 7fa12158d640 1 --2- 172.21.15.139:0/388630148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa11c068060 0x7fa11c15a780 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.831+0000 7fa12158d640 1 -- 172.21.15.139:0/388630148 >> 172.21.15.139:0/388630148 conn(0x7fa11c06ca40 msgr2=0x7fa11c150990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:58.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.831+0000 7fa12158d640 1 -- 172.21.15.139:0/388630148 shutdown_connections 2024-09-16T00:10:58.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.831+0000 7fa12158d640 1 -- 172.21.15.139:0/388630148 wait complete. 2024-09-16T00:10:58.845 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-16T00:10:58.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.946+0000 7f75e2665640 1 Processor -- start 2024-09-16T00:10:58.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.946+0000 7f75e2665640 1 -- start start 2024-09-16T00:10:58.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.946+0000 7f75e2665640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75dc151be0 0x7f75dc0c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.946+0000 7f75e2665640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75dc15ad00 con 0x7f75dc0c63c0 2024-09-16T00:10:58.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.946+0000 7f75e2665640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75dc0c5650 con 0x7f75dc151800 2024-09-16T00:10:58.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.946+0000 7f75e2665640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f75dc0c5010 con 0x7f75dc151be0 2024-09-16T00:10:58.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.947+0000 7f75e0c02640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f75dc0c63c0 0x7f75dc0729c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54440/0 (socket says 172.21.15.139:54440) 2024-09-16T00:10:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.947+0000 7f75e0c02640 1 -- 172.21.15.139:0/552385816 learned_addr learned my addr 172.21.15.139:0/552385816 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:10:58.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.947+0000 7f75db7fe640 1 --2- 172.21.15.139:0/552385816 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75dc151be0 0x7f75dc0c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.947+0000 7f75daffd640 1 -- 172.21.15.139:0/552385816 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 556507950 0 0) 0x7f75dc15ad00 con 0x7f75dc0c63c0 2024-09-16T00:10:58.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.947+0000 7f75daffd640 1 -- 172.21.15.139:0/552385816 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75c8003520 con 0x7f75dc0c63c0 2024-09-16T00:10:58.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.947+0000 7f75daffd640 1 -- 172.21.15.139:0/552385816 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 892221401 0 0) 0x7f75dc0c5650 con 0x7f75dc151800 2024-09-16T00:10:58.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.947+0000 7f75daffd640 1 -- 172.21.15.139:0/552385816 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75dc15ad00 con 0x7f75dc151800 2024-09-16T00:10:58.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.947+0000 7f75db7fe640 1 -- 172.21.15.139:0/552385816 >> v1:172.21.15.164:6789/0 conn(0x7f75dc151800 legacy=0x7f75dc0c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.947+0000 7f75db7fe640 1 -- 172.21.15.139:0/552385816 >> v1:172.21.15.139:6789/0 conn(0x7f75dc0c63c0 legacy=0x7f75dc0729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.947+0000 7f75db7fe640 1 -- 172.21.15.139:0/552385816 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc0c5930 con 0x7f75dc151be0 2024-09-16T00:10:58.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.948+0000 7f75db7fe640 1 --2- 172.21.15.139:0/552385816 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75dc151be0 0x7f75dc0c6c00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f75c40097d0 tx=0x7f75c402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae366a04b3fdb2b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:58.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.948+0000 7f75daffd640 1 -- 172.21.15.139:0/552385816 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4037070 con 0x7f75dc151be0 2024-09-16T00:10:58.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.948+0000 7f75daffd640 1 -- 172.21.15.139:0/552385816 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c4032440 con 0x7f75dc151be0 2024-09-16T00:10:58.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.948+0000 7f75daffd640 1 -- 172.21.15.139:0/552385816 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c40327f0 con 0x7f75dc151be0 2024-09-16T00:10:58.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.949+0000 7f75e2665640 1 -- 172.21.15.139:0/552385816 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75dc151be0 msgr2=0x7f75dc0c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.956 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.949+0000 7f75e2665640 1 --2- 172.21.15.139:0/552385816 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75dc151be0 0x7f75dc0c6c00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f75c40097d0 tx=0x7f75c402a210 comp rx=0 tx=0).stop 2024-09-16T00:10:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.949+0000 7f75e2665640 1 -- 172.21.15.139:0/552385816 shutdown_connections 2024-09-16T00:10:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.949+0000 7f75e2665640 1 --2- 172.21.15.139:0/552385816 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75dc151be0 0x7f75dc0c6c00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.949+0000 7f75e2665640 1 -- 172.21.15.139:0/552385816 >> 172.21.15.139:0/552385816 conn(0x7f75dc06da40 msgr2=0x7f75dc06f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.949+0000 7f75e2665640 1 -- 172.21.15.139:0/552385816 shutdown_connections 2024-09-16T00:10:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.949+0000 7f75e2665640 1 -- 172.21.15.139:0/552385816 wait complete. 2024-09-16T00:10:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.950+0000 7f75e2665640 1 Processor -- start 2024-09-16T00:10:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.950+0000 7f75e2665640 1 -- start start 2024-09-16T00:10:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75e2665640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75dc0c63c0 0x7f75dc141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75e2665640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75dc151800 0x7f75dc0cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75e2665640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75dc151be0 0x7f75dc0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75e2665640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc0c5c10 con 0x7f75dc0c63c0 2024-09-16T00:10:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75e2665640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f75dc0c9c40 con 0x7f75dc151800 2024-09-16T00:10:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75e2665640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f75dc0c52d0 con 0x7f75dc151be0 2024-09-16T00:10:58.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75dbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75dc0c63c0 0x7f75dc141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75dbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75dc0c63c0 0x7f75dc141da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60310/0 (socket says 172.21.15.139:60310) 2024-09-16T00:10:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75dbfff640 1 -- 172.21.15.139:0/2020813054 learned_addr learned my addr 172.21.15.139:0/2020813054 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75db7fe640 1 --2- 172.21.15.139:0/2020813054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75dc151800 0x7f75dc0cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75e0c02640 1 --2- 172.21.15.139:0/2020813054 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75dc151be0 0x7f75dc0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75dbfff640 1 -- 172.21.15.139:0/2020813054 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75dc151be0 msgr2=0x7f75dc0ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75dbfff640 1 --2- 172.21.15.139:0/2020813054 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75dc151be0 0x7f75dc0ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75dbfff640 1 -- 172.21.15.139:0/2020813054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75dc151800 msgr2=0x7f75dc0cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75dbfff640 1 --2- 172.21.15.139:0/2020813054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75dc151800 0x7f75dc0cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75dbfff640 1 -- 172.21.15.139:0/2020813054 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75dc141010 con 0x7f75dc0c63c0 2024-09-16T00:10:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75e0c02640 1 --2- 172.21.15.139:0/2020813054 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75dc151be0 0x7f75dc0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:10:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75dbfff640 1 --2- 172.21.15.139:0/2020813054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75dc0c63c0 0x7f75dc141da0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00c530 tx=0x7f75cc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75d8ff9640 1 -- 172.21.15.139:0/2020813054 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0052d0 con 0x7f75dc0c63c0 2024-09-16T00:10:58.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75d8ff9640 1 -- 172.21.15.139:0/2020813054 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75cc005490 con 0x7f75dc0c63c0 2024-09-16T00:10:58.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.951+0000 7f75e2665640 1 -- 172.21.15.139:0/2020813054 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75dc140d30 con 0x7f75dc0c63c0 2024-09-16T00:10:58.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.952+0000 7f75d8ff9640 1 -- 172.21.15.139:0/2020813054 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc012870 con 0x7f75dc0c63c0 2024-09-16T00:10:58.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.952+0000 7f75e2665640 1 -- 172.21.15.139:0/2020813054 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75dc0cb410 con 0x7f75dc0c63c0 2024-09-16T00:10:58.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.953+0000 7f75e2665640 1 -- 172.21.15.139:0/2020813054 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75dc0c7d80 con 0x7f75dc0c63c0 2024-09-16T00:10:58.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.953+0000 7f75d8ff9640 1 -- 172.21.15.139:0/2020813054 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f75cc012a10 con 0x7f75dc0c63c0 2024-09-16T00:10:58.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.953+0000 7f75d8ff9640 1 --2- 172.21.15.139:0/2020813054 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f75bc06c2e0 0x7f75bc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:58.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.954+0000 7f75d8ff9640 1 -- 172.21.15.139:0/2020813054 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5944+0+0 (secure 0 0 0) 0x7f75cc08b110 con 0x7f75dc0c63c0 2024-09-16T00:10:58.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.954+0000 7f75db7fe640 1 --2- 172.21.15.139:0/2020813054 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f75bc06c2e0 0x7f75bc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:58.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.954+0000 7f75db7fe640 1 --2- 172.21.15.139:0/2020813054 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f75bc06c2e0 0x7f75bc06e7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f75c4034000 tx=0x7f75c402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:58.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:58.958+0000 7f75d8ff9640 1 -- 172.21.15.139:0/2020813054 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f75cc059710 con 0x7f75dc0c63c0 2024-09-16T00:10:59.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.133+0000 7f75e2665640 1 -- 172.21.15.139:0/2020813054 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} v 0) -- 0x7f75dc0727a0 con 0x7f75dc0c63c0 2024-09-16T00:10:59.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.830+0000 7f75d8ff9640 1 -- 172.21.15.139:0/2020813054 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]=0 set osd.0 primary-affinity to 1 (8655362) v132) ==== 133+0+0 (secure 0 0 0) 0x7f75cc059010 con 0x7f75dc0c63c0 2024-09-16T00:10:59.831 INFO:teuthology.orchestra.run.smithi139.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-16T00:10:59.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.833+0000 7f75e2665640 1 -- 172.21.15.139:0/2020813054 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f75bc06c2e0 msgr2=0x7f75bc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:59.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.833+0000 7f75e2665640 1 --2- 172.21.15.139:0/2020813054 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f75bc06c2e0 0x7f75bc06e7a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f75c4034000 tx=0x7f75c402e000 comp rx=0 tx=0).stop 2024-09-16T00:10:59.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.833+0000 7f75e2665640 1 -- 172.21.15.139:0/2020813054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75dc0c63c0 msgr2=0x7f75dc141da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:59.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.833+0000 7f75e2665640 1 --2- 172.21.15.139:0/2020813054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75dc0c63c0 0x7f75dc141da0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00c530 tx=0x7f75cc00c9f0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.833+0000 7f75e2665640 1 -- 172.21.15.139:0/2020813054 shutdown_connections 2024-09-16T00:10:59.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.833+0000 7f75e2665640 1 --2- 172.21.15.139:0/2020813054 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f75bc06c2e0 0x7f75bc06e7a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.833+0000 7f75e2665640 1 --2- 172.21.15.139:0/2020813054 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f75dc151be0 0x7f75dc0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.833+0000 7f75e2665640 1 --2- 172.21.15.139:0/2020813054 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f75dc151800 0x7f75dc0cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.833+0000 7f75e2665640 1 --2- 172.21.15.139:0/2020813054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75dc0c63c0 0x7f75dc141da0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.833+0000 7f75e2665640 1 -- 172.21.15.139:0/2020813054 >> 172.21.15.139:0/2020813054 conn(0x7f75dc06da40 msgr2=0x7f75dc140140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:59.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.833+0000 7f75e2665640 1 -- 172.21.15.139:0/2020813054 shutdown_connections 2024-09-16T00:10:59.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.833+0000 7f75e2665640 1 -- 172.21.15.139:0/2020813054 wait complete. 2024-09-16T00:10:59.848 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-16T00:10:59.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.954+0000 7fe1f1911640 1 Processor -- start 2024-09-16T00:10:59.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.954+0000 7fe1f1911640 1 -- start start 2024-09-16T00:10:59.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.955+0000 7fe1f1911640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe1ec068060 0x7fe1ec15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:59.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.955+0000 7fe1f1911640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe1ec0ccc90 0x7fe1ec0cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:59.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.955+0000 7fe1f1911640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1ec0cc4b0 con 0x7fe1ec154c20 2024-09-16T00:10:59.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.955+0000 7fe1f1911640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe1ec0c5830 con 0x7fe1ec0ccc90 2024-09-16T00:10:59.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.955+0000 7fe1f1911640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fe1ec15b7c0 con 0x7fe1ec068060 2024-09-16T00:10:59.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.955+0000 7fe1eb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe1ec0ccc90 0x7fe1ec0cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:59.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.955+0000 7fe1eb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe1ec0ccc90 0x7fe1ec0cd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60314/0 (socket says 172.21.15.139:60314) 2024-09-16T00:10:59.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.955+0000 7fe1eb7fe640 1 -- 172.21.15.139:0/2824057525 learned_addr learned my addr 172.21.15.139:0/2824057525 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:59.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.955+0000 7fe1ea7fc640 1 --2- 172.21.15.139:0/2824057525 >> v2:172.21.15.164:3300/0 conn(0x7fe1ec068060 0x7fe1ec15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:59.957 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.955+0000 7fe1eb7fe640 1 -- 172.21.15.139:0/2824057525 >> v2:172.21.15.164:3300/0 conn(0x7fe1ec068060 msgr2=0x7fe1ec15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:59.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.955+0000 7fe1eb7fe640 1 --2- 172.21.15.139:0/2824057525 >> v2:172.21.15.164:3300/0 conn(0x7fe1ec068060 0x7fe1ec15b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.955+0000 7fe1eb7fe640 1 -- 172.21.15.139:0/2824057525 >> v1:172.21.15.164:6789/0 conn(0x7fe1ec154c20 legacy=0x7fe1ec0c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:10:59.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.955+0000 7fe1eb7fe640 1 -- 172.21.15.139:0/2824057525 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ec0cc790 con 0x7fe1ec0ccc90 2024-09-16T00:10:59.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.955+0000 7fe1ea7fc640 1 --2- 172.21.15.139:0/2824057525 >> v2:172.21.15.164:3300/0 conn(0x7fe1ec068060 0x7fe1ec15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:10:59.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.956+0000 7fe1eb7fe640 1 --2- 172.21.15.139:0/2824057525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1ec0ccc90 0x7fe1ec0cd0a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0685f0 tx=0x7fe1e402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c225cadc75be8f3a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:59.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.956+0000 7fe1e9ffb640 1 -- 172.21.15.139:0/2824057525 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4036070 con 0x7fe1ec0ccc90 2024-09-16T00:10:59.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.956+0000 7fe1e9ffb640 1 -- 172.21.15.139:0/2824057525 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1e4031600 con 0x7fe1ec0ccc90 2024-09-16T00:10:59.958 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.956+0000 7fe1e9ffb640 1 -- 172.21.15.139:0/2824057525 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40319d0 con 0x7fe1ec0ccc90 2024-09-16T00:10:59.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.957+0000 7fe1f1911640 1 -- 172.21.15.139:0/2824057525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1ec0ccc90 msgr2=0x7fe1ec0cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:59.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.957+0000 7fe1f1911640 1 --2- 172.21.15.139:0/2824057525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1ec0ccc90 0x7fe1ec0cd0a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec0685f0 tx=0x7fe1e402a6f0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.957+0000 7fe1f1911640 1 -- 172.21.15.139:0/2824057525 shutdown_connections 2024-09-16T00:10:59.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.957+0000 7fe1f1911640 1 --2- 172.21.15.139:0/2824057525 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1ec0ccc90 0x7fe1ec0cd0a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.957+0000 7fe1f1911640 1 --2- 172.21.15.139:0/2824057525 >> v2:172.21.15.164:3300/0 conn(0x7fe1ec068060 0x7fe1ec15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.957+0000 7fe1f1911640 1 -- 172.21.15.139:0/2824057525 >> 172.21.15.139:0/2824057525 conn(0x7fe1ec06ca40 msgr2=0x7fe1ec067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:10:59.962 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.957+0000 7fe1f1911640 1 -- 172.21.15.139:0/2824057525 shutdown_connections 2024-09-16T00:10:59.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.957+0000 7fe1f1911640 1 -- 172.21.15.139:0/2824057525 wait complete. 2024-09-16T00:10:59.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.958+0000 7fe1f1911640 1 Processor -- start 2024-09-16T00:10:59.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.958+0000 7fe1f1911640 1 -- start start 2024-09-16T00:10:59.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1f1911640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1ec068060 0x7fe1ec15d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:59.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1f1911640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1ec0ccc90 0x7fe1ec153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:59.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1f1911640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe1ec154c20 0x7fe1ec147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:59.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1f1911640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec15e0d0 con 0x7fe1ec0ccc90 2024-09-16T00:10:59.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1f1911640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec0cc260 con 0x7fe1ec068060 2024-09-16T00:10:59.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1f1911640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe1ec15e3b0 con 0x7fe1ec154c20 2024-09-16T00:10:59.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1ea7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1ec0ccc90 0x7fe1ec153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:59.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1ea7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1ec0ccc90 0x7fe1ec153d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60326/0 (socket says 172.21.15.139:60326) 2024-09-16T00:10:59.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1ea7fc640 1 -- 172.21.15.139:0/3738453940 learned_addr learned my addr 172.21.15.139:0/3738453940 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:10:59.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1eaffd640 1 --2- 172.21.15.139:0/3738453940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1ec068060 0x7fe1ec15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:59.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1eb7fe640 1 --2- 172.21.15.139:0/3738453940 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe1ec154c20 0x7fe1ec147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:59.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1ea7fc640 1 -- 172.21.15.139:0/3738453940 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe1ec154c20 msgr2=0x7fe1ec147d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:59.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1ea7fc640 1 --2- 172.21.15.139:0/3738453940 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe1ec154c20 0x7fe1ec147d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1ea7fc640 1 -- 172.21.15.139:0/3738453940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1ec068060 msgr2=0x7fe1ec15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:10:59.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1ea7fc640 1 --2- 172.21.15.139:0/3738453940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1ec068060 0x7fe1ec15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:10:59.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1ea7fc640 1 -- 172.21.15.139:0/3738453940 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ec15dbb0 con 0x7fe1ec0ccc90 2024-09-16T00:10:59.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1ea7fc640 1 --2- 172.21.15.139:0/3738453940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1ec0ccc90 0x7fe1ec153d40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c820 tx=0x7fe1e000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:59.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1d3fff640 1 -- 172.21.15.139:0/3738453940 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0005b90 con 0x7fe1ec0ccc90 2024-09-16T00:10:59.965 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1d3fff640 1 -- 172.21.15.139:0/3738453940 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1e0005d30 con 0x7fe1ec0ccc90 2024-09-16T00:10:59.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1d3fff640 1 -- 172.21.15.139:0/3738453940 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0007a30 con 0x7fe1ec0ccc90 2024-09-16T00:10:59.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.959+0000 7fe1f1911640 1 -- 172.21.15.139:0/3738453940 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ec15ce80 con 0x7fe1ec0ccc90 2024-09-16T00:10:59.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.960+0000 7fe1f1911640 1 -- 172.21.15.139:0/3738453940 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ec14ef60 con 0x7fe1ec0ccc90 2024-09-16T00:10:59.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.960+0000 7fe1f1911640 1 -- 172.21.15.139:0/3738453940 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ec15e690 con 0x7fe1ec0ccc90 2024-09-16T00:10:59.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.961+0000 7fe1d3fff640 1 -- 172.21.15.139:0/3738453940 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe1e0007bd0 con 0x7fe1ec0ccc90 2024-09-16T00:10:59.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.961+0000 7fe1d3fff640 1 --2- 172.21.15.139:0/3738453940 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe1d806c2e0 0x7fe1d806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:10:59.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.962+0000 7fe1d3fff640 1 -- 172.21.15.139:0/3738453940 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5944+0+0 (secure 0 0 0) 0x7fe1e008b440 con 0x7fe1ec0ccc90 2024-09-16T00:10:59.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.962+0000 7fe1eaffd640 1 --2- 172.21.15.139:0/3738453940 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe1d806c2e0 0x7fe1d806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:10:59.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.962+0000 7fe1eaffd640 1 --2- 172.21.15.139:0/3738453940 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe1d806c2e0 0x7fe1d806e7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc006fd0 tx=0x7fe1dc009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:10:59.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:10:59.966+0000 7fe1d3fff640 1 -- 172.21.15.139:0/3738453940 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe1e0015020 con 0x7fe1ec0ccc90 2024-09-16T00:11:00.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.142+0000 7fe1f1911640 1 -- 172.21.15.139:0/3738453940 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) -- 0x7fe1ec0cca70 con 0x7fe1ec0ccc90 2024-09-16T00:11:00.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.839+0000 7fe1d3fff640 1 -- 172.21.15.139:0/3738453940 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v133) ==== 113+0+0 (secure 0 0 0) 0x7fe1e0059500 con 0x7fe1ec0ccc90 2024-09-16T00:11:00.840 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T00:11:00.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.841+0000 7fe1f1911640 1 -- 172.21.15.139:0/3738453940 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe1d806c2e0 msgr2=0x7fe1d806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:00.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.841+0000 7fe1f1911640 1 --2- 172.21.15.139:0/3738453940 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe1d806c2e0 0x7fe1d806e7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc006fd0 tx=0x7fe1dc009060 comp rx=0 tx=0).stop 2024-09-16T00:11:00.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.841+0000 7fe1f1911640 1 -- 172.21.15.139:0/3738453940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1ec0ccc90 msgr2=0x7fe1ec153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:00.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.841+0000 7fe1f1911640 1 --2- 172.21.15.139:0/3738453940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1ec0ccc90 0x7fe1ec153d40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c820 tx=0x7fe1e000cc20 comp rx=0 tx=0).stop 2024-09-16T00:11:00.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.841+0000 7fe1f1911640 1 -- 172.21.15.139:0/3738453940 shutdown_connections 2024-09-16T00:11:00.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.841+0000 7fe1f1911640 1 --2- 172.21.15.139:0/3738453940 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe1d806c2e0 0x7fe1d806e7a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:00.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.841+0000 7fe1f1911640 1 --2- 172.21.15.139:0/3738453940 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe1ec154c20 0x7fe1ec147d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:00.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.841+0000 7fe1f1911640 1 --2- 172.21.15.139:0/3738453940 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1ec0ccc90 0x7fe1ec153d40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:00.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.841+0000 7fe1f1911640 1 --2- 172.21.15.139:0/3738453940 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe1ec068060 0x7fe1ec15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:00.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.841+0000 7fe1f1911640 1 -- 172.21.15.139:0/3738453940 >> 172.21.15.139:0/3738453940 conn(0x7fe1ec06ca40 msgr2=0x7fe1ec1589a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:00.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.842+0000 7fe1f1911640 1 -- 172.21.15.139:0/3738453940 shutdown_connections 2024-09-16T00:11:00.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.842+0000 7fe1f1911640 1 -- 172.21.15.139:0/3738453940 wait complete. 2024-09-16T00:11:00.858 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-16T00:11:00.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.958+0000 7f22a7531640 1 Processor -- start 2024-09-16T00:11:00.959 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.958+0000 7f22a7531640 1 -- start start 2024-09-16T00:11:00.960 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.959+0000 7f22a7531640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22a0151800 0x7f22a00c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.959+0000 7f22a7531640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f22a00c63c0 0x7f22a015b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.959+0000 7f22a7531640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22a015ad00 con 0x7f22a0151be0 2024-09-16T00:11:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.959+0000 7f22a7531640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f22a00c5010 con 0x7f22a00c63c0 2024-09-16T00:11:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.959+0000 7f22a7531640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f22a00c9c40 con 0x7f22a0151800 2024-09-16T00:11:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.959+0000 7f22a5ace640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f22a00c63c0 0x7f22a015b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.959+0000 7f22a5ace640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f22a00c63c0 0x7f22a015b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60330/0 (socket says 172.21.15.139:60330) 2024-09-16T00:11:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.959+0000 7f22a5ace640 1 -- 172.21.15.139:0/1542750325 learned_addr learned my addr 172.21.15.139:0/1542750325 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.959+0000 7f22a52cd640 1 --2- 172.21.15.139:0/1542750325 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22a0151800 0x7f22a00c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.960+0000 7f22a5ace640 1 -- 172.21.15.139:0/1542750325 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22a0151800 msgr2=0x7f22a00c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.960+0000 7f22a5ace640 1 --2- 172.21.15.139:0/1542750325 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22a0151800 0x7f22a00c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.960+0000 7f22a5ace640 1 -- 172.21.15.139:0/1542750325 >> v1:172.21.15.164:6789/0 conn(0x7f22a0151be0 legacy=0x7f22a00c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:00.961 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.960+0000 7f22a5ace640 1 -- 172.21.15.139:0/1542750325 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a00c5650 con 0x7f22a00c63c0 2024-09-16T00:11:00.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.960+0000 7f22a5ace640 1 --2- 172.21.15.139:0/1542750325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22a00c63c0 0x7f22a015b7a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2298009870 tx=0x7f229802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243a907a70dd1b5f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:00.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.960+0000 7f229ffff640 1 -- 172.21.15.139:0/1542750325 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298037070 con 0x7f22a00c63c0 2024-09-16T00:11:00.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.960+0000 7f229ffff640 1 -- 172.21.15.139:0/1542750325 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2298032440 con 0x7f22a00c63c0 2024-09-16T00:11:00.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.960+0000 7f229ffff640 1 -- 172.21.15.139:0/1542750325 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298032810 con 0x7f22a00c63c0 2024-09-16T00:11:00.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.961+0000 7f22a7531640 1 -- 172.21.15.139:0/1542750325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22a00c63c0 msgr2=0x7f22a015b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:00.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.961+0000 7f22a7531640 1 --2- 172.21.15.139:0/1542750325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22a00c63c0 0x7f22a015b7a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2298009870 tx=0x7f229802a270 comp rx=0 tx=0).stop 2024-09-16T00:11:00.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.961+0000 7f22a7531640 1 -- 172.21.15.139:0/1542750325 shutdown_connections 2024-09-16T00:11:00.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.961+0000 7f22a7531640 1 --2- 172.21.15.139:0/1542750325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22a00c63c0 0x7f22a015b7a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.961+0000 7f22a7531640 1 --2- 172.21.15.139:0/1542750325 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22a0151800 0x7f22a00c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.961+0000 7f22a7531640 1 -- 172.21.15.139:0/1542750325 >> 172.21.15.139:0/1542750325 conn(0x7f22a006da40 msgr2=0x7f22a006f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.961+0000 7f22a7531640 1 -- 172.21.15.139:0/1542750325 shutdown_connections 2024-09-16T00:11:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.961+0000 7f22a7531640 1 -- 172.21.15.139:0/1542750325 wait complete. 2024-09-16T00:11:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.962+0000 7f22a7531640 1 Processor -- start 2024-09-16T00:11:00.964 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.962+0000 7f22a7531640 1 -- start start 2024-09-16T00:11:00.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a7531640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a00c63c0 0x7f22a01f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:00.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a7531640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22a0151800 0x7f22a00cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:00.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a7531640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22a0151be0 0x7f22a0153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:00.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a7531640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22a00c5c10 con 0x7f22a0151be0 2024-09-16T00:11:00.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a7531640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f22a015afa0 con 0x7f22a00c63c0 2024-09-16T00:11:00.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a7531640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f22a00c52d0 con 0x7f22a0151800 2024-09-16T00:11:00.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a5ace640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22a0151be0 0x7f22a0153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:00.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a5ace640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22a0151be0 0x7f22a0153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60340/0 (socket says 172.21.15.139:60340) 2024-09-16T00:11:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a5ace640 1 -- 172.21.15.139:0/3439069971 learned_addr learned my addr 172.21.15.139:0/3439069971 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a52cd640 1 --2- 172.21.15.139:0/3439069971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a00c63c0 0x7f22a01f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a4acc640 1 --2- 172.21.15.139:0/3439069971 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22a0151800 0x7f22a00cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a5ace640 1 -- 172.21.15.139:0/3439069971 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22a0151800 msgr2=0x7f22a00cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a5ace640 1 --2- 172.21.15.139:0/3439069971 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22a0151800 0x7f22a00cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a5ace640 1 -- 172.21.15.139:0/3439069971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a00c63c0 msgr2=0x7f22a01f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a5ace640 1 --2- 172.21.15.139:0/3439069971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a00c63c0 0x7f22a01f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a5ace640 1 -- 172.21.15.139:0/3439069971 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a0154d20 con 0x7f22a0151be0 2024-09-16T00:11:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a4acc640 1 --2- 172.21.15.139:0/3439069971 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22a0151800 0x7f22a00cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.963+0000 7f22a52cd640 1 --2- 172.21.15.139:0/3439069971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a00c63c0 0x7f22a01f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.964+0000 7f22a5ace640 1 --2- 172.21.15.139:0/3439069971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22a0151be0 0x7f22a0153700 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2298009870 tx=0x7f22980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.964+0000 7f229dffb640 1 -- 172.21.15.139:0/3439069971 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298037040 con 0x7f22a0151be0 2024-09-16T00:11:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.964+0000 7f229dffb640 1 -- 172.21.15.139:0/3439069971 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2298042070 con 0x7f22a0151be0 2024-09-16T00:11:00.967 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.964+0000 7f22a7531640 1 -- 172.21.15.139:0/3439069971 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a0152340 con 0x7f22a0151be0 2024-09-16T00:11:00.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.964+0000 7f229dffb640 1 -- 172.21.15.139:0/3439069971 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298030920 con 0x7f22a0151be0 2024-09-16T00:11:00.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.964+0000 7f22a7531640 1 -- 172.21.15.139:0/3439069971 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a0152900 con 0x7f22a0151be0 2024-09-16T00:11:00.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.965+0000 7f22a7531640 1 -- 172.21.15.139:0/3439069971 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a015b7a0 con 0x7f22a0151be0 2024-09-16T00:11:00.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.965+0000 7f229dffb640 1 -- 172.21.15.139:0/3439069971 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2298030ac0 con 0x7f22a0151be0 2024-09-16T00:11:00.968 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.966+0000 7f229dffb640 1 --2- 172.21.15.139:0/3439069971 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f228c06c2e0 0x7f228c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:00.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.966+0000 7f229dffb640 1 -- 172.21.15.139:0/3439069971 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5944+0+0 (secure 0 0 0) 0x7f22980abe40 con 0x7f22a0151be0 2024-09-16T00:11:00.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.966+0000 7f22a52cd640 1 --2- 172.21.15.139:0/3439069971 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f228c06c2e0 0x7f228c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:00.969 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.966+0000 7f22a52cd640 1 --2- 172.21.15.139:0/3439069971 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f228c06c2e0 0x7f228c06e7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2290009b20 tx=0x7f22900039d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:00.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:00.970+0000 7f229dffb640 1 -- 172.21.15.139:0/3439069971 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2298079e70 con 0x7f22a0151be0 2024-09-16T00:11:01.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.148+0000 7f22a7531640 1 -- 172.21.15.139:0/3439069971 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} v 0) -- 0x7f22a00c96f0 con 0x7f22a0151be0 2024-09-16T00:11:01.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.847+0000 7f229dffb640 1 -- 172.21.15.139:0/3439069971 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]=0 set osd.1 primary-affinity to 1 (8655362) v134) ==== 133+0+0 (secure 0 0 0) 0x7f229807e170 con 0x7f22a0151be0 2024-09-16T00:11:01.848 INFO:teuthology.orchestra.run.smithi139.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-16T00:11:01.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.851+0000 7f22a7531640 1 -- 172.21.15.139:0/3439069971 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f228c06c2e0 msgr2=0x7f228c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:01.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.851+0000 7f22a7531640 1 --2- 172.21.15.139:0/3439069971 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f228c06c2e0 0x7f228c06e7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2290009b20 tx=0x7f22900039d0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.851+0000 7f22a7531640 1 -- 172.21.15.139:0/3439069971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22a0151be0 msgr2=0x7f22a0153700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:01.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.851+0000 7f22a7531640 1 --2- 172.21.15.139:0/3439069971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22a0151be0 0x7f22a0153700 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2298009870 tx=0x7f22980046c0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.851+0000 7f22a7531640 1 -- 172.21.15.139:0/3439069971 shutdown_connections 2024-09-16T00:11:01.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.851+0000 7f22a7531640 1 --2- 172.21.15.139:0/3439069971 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f228c06c2e0 0x7f228c06e7a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.851+0000 7f22a7531640 1 --2- 172.21.15.139:0/3439069971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22a0151be0 0x7f22a0153700 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.851+0000 7f22a7531640 1 --2- 172.21.15.139:0/3439069971 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f22a0151800 0x7f22a00cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.851+0000 7f22a7531640 1 --2- 172.21.15.139:0/3439069971 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f22a00c63c0 0x7f22a01f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.851+0000 7f22a7531640 1 -- 172.21.15.139:0/3439069971 >> 172.21.15.139:0/3439069971 conn(0x7f22a006da40 msgr2=0x7f22a006f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:01.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.853+0000 7f22a7531640 1 -- 172.21.15.139:0/3439069971 shutdown_connections 2024-09-16T00:11:01.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.853+0000 7f22a7531640 1 -- 172.21.15.139:0/3439069971 wait complete. 2024-09-16T00:11:01.869 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-16T00:11:01.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.971+0000 7f677a023640 1 Processor -- start 2024-09-16T00:11:01.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.971+0000 7f677a023640 1 -- start start 2024-09-16T00:11:01.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.971+0000 7f677a023640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f67741527f0 0x7f67740c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:01.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.971+0000 7f677a023640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f677406f7f0 0x7f6774158580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:01.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.971+0000 7f677a023640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67740707f0 con 0x7f67740c8250 2024-09-16T00:11:01.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.972+0000 7f677a023640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f677406ff20 con 0x7f677406f7f0 2024-09-16T00:11:01.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.972+0000 7f677a023640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f6774070170 con 0x7f67741527f0 2024-09-16T00:11:01.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.972+0000 7f6772ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f677406f7f0 0x7f6774158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:01.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.972+0000 7f6772ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f677406f7f0 0x7f6774158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60354/0 (socket says 172.21.15.139:60354) 2024-09-16T00:11:01.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.972+0000 7f6772ffd640 1 -- 172.21.15.139:0/1654774855 learned_addr learned my addr 172.21.15.139:0/1654774855 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:01.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.972+0000 7f67737fe640 1 --2- 172.21.15.139:0/1654774855 >> v2:172.21.15.164:3300/0 conn(0x7f67741527f0 0x7f67740c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:01.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.972+0000 7f6772ffd640 1 -- 172.21.15.139:0/1654774855 >> v2:172.21.15.164:3300/0 conn(0x7f67741527f0 msgr2=0x7f67740c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:01.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.972+0000 7f6772ffd640 1 --2- 172.21.15.139:0/1654774855 >> v2:172.21.15.164:3300/0 conn(0x7f67741527f0 0x7f67740c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.972+0000 7f6772ffd640 1 -- 172.21.15.139:0/1654774855 >> v1:172.21.15.164:6789/0 conn(0x7f67740c8250 legacy=0x7f67740ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:01.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.972+0000 7f6772ffd640 1 -- 172.21.15.139:0/1654774855 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6774070a90 con 0x7f677406f7f0 2024-09-16T00:11:01.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.972+0000 7f67737fe640 1 --2- 172.21.15.139:0/1654774855 >> v2:172.21.15.164:3300/0 conn(0x7f67741527f0 0x7f67740c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:01.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.973+0000 7f6772ffd640 1 --2- 172.21.15.139:0/1654774855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f677406f7f0 0x7f6774158580 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f67740cbc90 tx=0x7f676802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d3f9c6f01971db server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:01.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.973+0000 7f67727fc640 1 -- 172.21.15.139:0/1654774855 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768037070 con 0x7f677406f7f0 2024-09-16T00:11:01.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.973+0000 7f67727fc640 1 -- 172.21.15.139:0/1654774855 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6768032440 con 0x7f677406f7f0 2024-09-16T00:11:01.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.973+0000 7f67727fc640 1 -- 172.21.15.139:0/1654774855 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768032810 con 0x7f677406f7f0 2024-09-16T00:11:01.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.973+0000 7f677a023640 1 -- 172.21.15.139:0/1654774855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f677406f7f0 msgr2=0x7f6774158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:01.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.973+0000 7f677a023640 1 --2- 172.21.15.139:0/1654774855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f677406f7f0 0x7f6774158580 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f67740cbc90 tx=0x7f676802a1a0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.973+0000 7f677a023640 1 -- 172.21.15.139:0/1654774855 shutdown_connections 2024-09-16T00:11:01.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.973+0000 7f677a023640 1 --2- 172.21.15.139:0/1654774855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f677406f7f0 0x7f6774158580 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.973+0000 7f677a023640 1 --2- 172.21.15.139:0/1654774855 >> v2:172.21.15.164:3300/0 conn(0x7f67741527f0 0x7f67740c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.973+0000 7f677a023640 1 -- 172.21.15.139:0/1654774855 >> 172.21.15.139:0/1654774855 conn(0x7f677406c150 msgr2=0x7f6774067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:01.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.973+0000 7f677a023640 1 -- 172.21.15.139:0/1654774855 shutdown_connections 2024-09-16T00:11:01.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.973+0000 7f677a023640 1 -- 172.21.15.139:0/1654774855 wait complete. 2024-09-16T00:11:01.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.975+0000 7f677a023640 1 Processor -- start 2024-09-16T00:11:01.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.975+0000 7f677a023640 1 -- start start 2024-09-16T00:11:01.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.975+0000 7f677a023640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f677406f7f0 0x7f6774159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:01.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.975+0000 7f677a023640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67740c8250 0x7f677415afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:01.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.975+0000 7f677a023640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67741527f0 0x7f677415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:01.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.975+0000 7f677a023640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f67740ce7d0 con 0x7f677406f7f0 2024-09-16T00:11:01.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.975+0000 7f677a023640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6774152bd0 con 0x7f67740c8250 2024-09-16T00:11:01.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.975+0000 7f677a023640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f67740ce950 con 0x7f67741527f0 2024-09-16T00:11:01.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.975+0000 7f67737fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f677406f7f0 0x7f6774159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:01.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.976+0000 7f67737fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f677406f7f0 0x7f6774159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60358/0 (socket says 172.21.15.139:60358) 2024-09-16T00:11:01.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.976+0000 7f67737fe640 1 -- 172.21.15.139:0/1968085374 learned_addr learned my addr 172.21.15.139:0/1968085374 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:01.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.976+0000 7f67737fe640 1 -- 172.21.15.139:0/1968085374 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67741527f0 msgr2=0x7f677415b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:01.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.976+0000 7f6773fff640 1 --2- 172.21.15.139:0/1968085374 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67741527f0 0x7f677415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:01.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.976+0000 7f6772ffd640 1 --2- 172.21.15.139:0/1968085374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67740c8250 0x7f677415afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:01.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.976+0000 7f67737fe640 1 --2- 172.21.15.139:0/1968085374 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67741527f0 0x7f677415b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.976+0000 7f67737fe640 1 -- 172.21.15.139:0/1968085374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67740c8250 msgr2=0x7f677415afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:01.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.976+0000 7f67737fe640 1 --2- 172.21.15.139:0/1968085374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67740c8250 0x7f677415afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:01.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.976+0000 7f67737fe640 1 -- 172.21.15.139:0/1968085374 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67741e3c90 con 0x7f677406f7f0 2024-09-16T00:11:01.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.976+0000 7f67737fe640 1 --2- 172.21.15.139:0/1968085374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f677406f7f0 0x7f6774159720 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f676400c820 tx=0x7f676400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:01.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.976+0000 7f675bfff640 1 -- 172.21.15.139:0/1968085374 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764015070 con 0x7f677406f7f0 2024-09-16T00:11:01.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.976+0000 7f675bfff640 1 -- 172.21.15.139:0/1968085374 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6764007a30 con 0x7f677406f7f0 2024-09-16T00:11:01.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.976+0000 7f675bfff640 1 -- 172.21.15.139:0/1968085374 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764007de0 con 0x7f677406f7f0 2024-09-16T00:11:01.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.976+0000 7f677a023640 1 -- 172.21.15.139:0/1968085374 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67741ee5c0 con 0x7f677406f7f0 2024-09-16T00:11:01.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.976+0000 7f677a023640 1 -- 172.21.15.139:0/1968085374 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67741eeb80 con 0x7f677406f7f0 2024-09-16T00:11:01.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.977+0000 7f677a023640 1 -- 172.21.15.139:0/1968085374 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6774158580 con 0x7f677406f7f0 2024-09-16T00:11:01.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.978+0000 7f675bfff640 1 -- 172.21.15.139:0/1968085374 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6764018070 con 0x7f677406f7f0 2024-09-16T00:11:01.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.978+0000 7f675bfff640 1 --2- 172.21.15.139:0/1968085374 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f676006c2b0 0x7f676006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:01.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.978+0000 7f6772ffd640 1 --2- 172.21.15.139:0/1968085374 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f676006c2b0 0x7f676006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:01.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.978+0000 7f675bfff640 1 -- 172.21.15.139:0/1968085374 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 6102+0+0 (secure 0 0 0) 0x7f676408b450 con 0x7f677406f7f0 2024-09-16T00:11:01.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.979+0000 7f6772ffd640 1 --2- 172.21.15.139:0/1968085374 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f676006c2b0 0x7f676006e770 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6768034000 tx=0x7f676802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:01.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:01.983+0000 7f675bfff640 1 -- 172.21.15.139:0/1968085374 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6764059ac0 con 0x7f677406f7f0 2024-09-16T00:11:02.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.161+0000 7f677a023640 1 -- 172.21.15.139:0/1968085374 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) -- 0x7f67740ce2e0 con 0x7f677406f7f0 2024-09-16T00:11:02.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.849+0000 7f675bfff640 1 -- 172.21.15.139:0/1968085374 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v135) ==== 113+0+0 (secure 0 0 0) 0x7f67640593c0 con 0x7f677406f7f0 2024-09-16T00:11:02.850 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T00:11:02.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.857+0000 7f677a023640 1 -- 172.21.15.139:0/1968085374 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f676006c2b0 msgr2=0x7f676006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:02.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.857+0000 7f677a023640 1 --2- 172.21.15.139:0/1968085374 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f676006c2b0 0x7f676006e770 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6768034000 tx=0x7f676802e000 comp rx=0 tx=0).stop 2024-09-16T00:11:02.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.857+0000 7f677a023640 1 -- 172.21.15.139:0/1968085374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f677406f7f0 msgr2=0x7f6774159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:02.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.857+0000 7f677a023640 1 --2- 172.21.15.139:0/1968085374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f677406f7f0 0x7f6774159720 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f676400c820 tx=0x7f676400cce0 comp rx=0 tx=0).stop 2024-09-16T00:11:02.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.859+0000 7f677a023640 1 -- 172.21.15.139:0/1968085374 shutdown_connections 2024-09-16T00:11:02.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.859+0000 7f677a023640 1 --2- 172.21.15.139:0/1968085374 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f676006c2b0 0x7f676006e770 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:02.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.859+0000 7f677a023640 1 --2- 172.21.15.139:0/1968085374 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f67741527f0 0x7f677415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:02.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.859+0000 7f677a023640 1 --2- 172.21.15.139:0/1968085374 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f67740c8250 0x7f677415afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:02.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.859+0000 7f677a023640 1 --2- 172.21.15.139:0/1968085374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f677406f7f0 0x7f6774159720 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:02.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.859+0000 7f677a023640 1 -- 172.21.15.139:0/1968085374 >> 172.21.15.139:0/1968085374 conn(0x7f677406c150 msgr2=0x7f67740c91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:02.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.861+0000 7f677a023640 1 -- 172.21.15.139:0/1968085374 shutdown_connections 2024-09-16T00:11:02.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.861+0000 7f677a023640 1 -- 172.21.15.139:0/1968085374 wait complete. 2024-09-16T00:11:02.883 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-16T00:11:02.989 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:11:02.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.989+0000 7effcf467640 1 Processor -- start 2024-09-16T00:11:02.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.989+0000 7effcf467640 1 -- start start 2024-09-16T00:11:02.992 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:11:02.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.990+0000 7effcf467640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effc806cda0 0x7effc8067b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:02.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.990+0000 7effcf467640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7effc806d180 0x7effc815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:02.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.990+0000 7effcf467640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7effc80c8990 0x7effc8154990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.991+0000 7effcf467640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7effc815e160 con 0x7effc80c8990 2024-09-16T00:11:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.991+0000 7effcf467640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7effc815f0f0 con 0x7effc806d180 2024-09-16T00:11:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.991+0000 7effcf467640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7effc8070680 con 0x7effc806cda0 2024-09-16T00:11:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.991+0000 7effcda04640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7effc80c8990 0x7effc8154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.991+0000 7effcca02640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7effc806d180 0x7effc815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.991+0000 7effcda04640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7effc80c8990 0x7effc8154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60374/0 (socket says 172.21.15.139:60374) 2024-09-16T00:11:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.991+0000 7effcda04640 1 -- 172.21.15.139:0/4013524130 learned_addr learned my addr 172.21.15.139:0/4013524130 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.991+0000 7effcd203640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effc806cda0 0x7effc8067b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.991+0000 7effcda04640 1 -- 172.21.15.139:0/4013524130 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effc806cda0 msgr2=0x7effc8067b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.991+0000 7effcda04640 1 --2- 172.21.15.139:0/4013524130 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effc806cda0 0x7effc8067b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.991+0000 7effcda04640 1 -- 172.21.15.139:0/4013524130 >> v2:172.21.15.164:3300/0 conn(0x7effc806d180 msgr2=0x7effc815ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.991+0000 7effcda04640 1 --2- 172.21.15.139:0/4013524130 >> v2:172.21.15.164:3300/0 conn(0x7effc806d180 0x7effc815ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.991+0000 7effcda04640 1 -- 172.21.15.139:0/4013524130 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc8070960 con 0x7effc80c8990 2024-09-16T00:11:02.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.992+0000 7effcda04640 1 --2- 172.21.15.139:0/4013524130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc80c8990 0x7effc8154990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7effc8071640 tx=0x7effc002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0734f5de32a558 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:02.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.992+0000 7effbffff640 1 -- 172.21.15.139:0/4013524130 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0036070 con 0x7effc80c8990 2024-09-16T00:11:02.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.992+0000 7effbffff640 1 -- 172.21.15.139:0/4013524130 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc0031600 con 0x7effc80c8990 2024-09-16T00:11:02.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.992+0000 7effbffff640 1 -- 172.21.15.139:0/4013524130 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc00319d0 con 0x7effc80c8990 2024-09-16T00:11:02.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.993+0000 7effcf467640 1 -- 172.21.15.139:0/4013524130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc80c8990 msgr2=0x7effc8154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:02.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.993+0000 7effcf467640 1 --2- 172.21.15.139:0/4013524130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc80c8990 0x7effc8154990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7effc8071640 tx=0x7effc002a730 comp rx=0 tx=0).stop 2024-09-16T00:11:02.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.993+0000 7effcf467640 1 -- 172.21.15.139:0/4013524130 shutdown_connections 2024-09-16T00:11:02.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.993+0000 7effcf467640 1 --2- 172.21.15.139:0/4013524130 >> v2:172.21.15.164:3300/0 conn(0x7effc806d180 0x7effc815ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:02.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.993+0000 7effcf467640 1 --2- 172.21.15.139:0/4013524130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc80c8990 0x7effc8154990 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:02.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.993+0000 7effcf467640 1 --2- 172.21.15.139:0/4013524130 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effc806cda0 0x7effc8067b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:02.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.993+0000 7effcf467640 1 -- 172.21.15.139:0/4013524130 >> 172.21.15.139:0/4013524130 conn(0x7effc806f800 msgr2=0x7effc806fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:02.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.993+0000 7effcf467640 1 -- 172.21.15.139:0/4013524130 shutdown_connections 2024-09-16T00:11:02.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.994+0000 7effcf467640 1 -- 172.21.15.139:0/4013524130 wait complete. 2024-09-16T00:11:02.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.995+0000 7effcf467640 1 Processor -- start 2024-09-16T00:11:02.997 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.995+0000 7effcf467640 1 -- start start 2024-09-16T00:11:02.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcf467640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc806cda0 0x7effc815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:02.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcf467640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effc806d180 0x7effc81f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:02.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcf467640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc80c8990 0x7effc814d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:02.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcf467640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7effc80720d0 con 0x7effc80c8990 2024-09-16T00:11:02.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcf467640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7effc815e690 con 0x7effc806cda0 2024-09-16T00:11:02.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcf467640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7effc8072250 con 0x7effc806d180 2024-09-16T00:11:02.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcd203640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc806cda0 0x7effc815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:02.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcca02640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effc806d180 0x7effc81f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:02.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcda04640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc80c8990 0x7effc814d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:02.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcd203640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc806cda0 0x7effc815a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56846/0 (socket says 172.21.15.139:56846) 2024-09-16T00:11:03.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcd203640 1 -- 172.21.15.139:0/160056001 learned_addr learned my addr 172.21.15.139:0/160056001 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:03.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcda04640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc80c8990 0x7effc814d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60382/0 (socket says 172.21.15.139:60382) 2024-09-16T00:11:03.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcca02640 1 -- 172.21.15.139:0/160056001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc806cda0 msgr2=0x7effc815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:03.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcca02640 1 --2- 172.21.15.139:0/160056001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc806cda0 0x7effc815a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:03.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcca02640 1 -- 172.21.15.139:0/160056001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc80c8990 msgr2=0x7effc814d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:03.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcca02640 1 --2- 172.21.15.139:0/160056001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc80c8990 0x7effc814d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:03.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcca02640 1 -- 172.21.15.139:0/160056001 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc80cc150 con 0x7effc806d180 2024-09-16T00:11:03.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.996+0000 7effcca02640 1 --2- 172.21.15.139:0/160056001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effc806d180 0x7effc81f3b00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7effc8158780 tx=0x7effb800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:03.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.997+0000 7effbdffb640 1 -- 172.21.15.139:0/160056001 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8017070 con 0x7effc806d180 2024-09-16T00:11:03.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.997+0000 7effbdffb640 1 -- 172.21.15.139:0/160056001 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb800c940 con 0x7effc806d180 2024-09-16T00:11:03.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.997+0000 7effcf467640 1 -- 172.21.15.139:0/160056001 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc815cf90 con 0x7effc806d180 2024-09-16T00:11:03.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.997+0000 7effbdffb640 1 -- 172.21.15.139:0/160056001 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb800cd10 con 0x7effc806d180 2024-09-16T00:11:03.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.997+0000 7effcf467640 1 -- 172.21.15.139:0/160056001 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7effc81f4570 con 0x7effc806d180 2024-09-16T00:11:03.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.998+0000 7effcf467640 1 -- 172.21.15.139:0/160056001 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc815b5f0 con 0x7effc806d180 2024-09-16T00:11:03.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:02.999+0000 7effbdffb640 1 -- 172.21.15.139:0/160056001 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7effb801a070 con 0x7effc806d180 2024-09-16T00:11:03.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.000+0000 7effbdffb640 1 --2- 172.21.15.139:0/160056001 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7effac06c2e0 0x7effac06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:03.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.000+0000 7effbdffb640 1 -- 172.21.15.139:0/160056001 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(135..135 src has 1..135) ==== 6102+0+0 (secure 0 0 0) 0x7effb808b350 con 0x7effc806d180 2024-09-16T00:11:03.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.000+0000 7effcd203640 1 --2- 172.21.15.139:0/160056001 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7effac06c2e0 0x7effac06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:03.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.000+0000 7effcd203640 1 --2- 172.21.15.139:0/160056001 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7effac06c2e0 0x7effac06e7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7effc814e190 tx=0x7effc4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:03.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.005+0000 7effbdffb640 1 -- 172.21.15.139:0/160056001 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7effb8059370 con 0x7effc806d180 2024-09-16T00:11:03.239 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.238+0000 7effcf467640 1 -- 172.21.15.139:0/160056001 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} v 0) -- 0x7effc80c8430 con 0x7effc806d180 2024-09-16T00:11:03.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.884+0000 7effbdffb640 1 -- 172.21.15.139:0/160056001 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]=0 set osd.2 primary-affinity to 1 (8655362) v136) ==== 133+0+0 (secure 0 0 0) 0x7effb805d670 con 0x7effc806d180 2024-09-16T00:11:03.885 INFO:teuthology.orchestra.run.smithi139.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-16T00:11:03.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.887+0000 7effcf467640 1 -- 172.21.15.139:0/160056001 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7effac06c2e0 msgr2=0x7effac06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:03.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.887+0000 7effcf467640 1 --2- 172.21.15.139:0/160056001 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7effac06c2e0 0x7effac06e7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7effc814e190 tx=0x7effc4009190 comp rx=0 tx=0).stop 2024-09-16T00:11:03.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.887+0000 7effcf467640 1 -- 172.21.15.139:0/160056001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effc806d180 msgr2=0x7effc81f3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:03.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.887+0000 7effcf467640 1 --2- 172.21.15.139:0/160056001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effc806d180 0x7effc81f3b00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7effc8158780 tx=0x7effb800ecb0 comp rx=0 tx=0).stop 2024-09-16T00:11:03.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.887+0000 7effcf467640 1 -- 172.21.15.139:0/160056001 shutdown_connections 2024-09-16T00:11:03.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.887+0000 7effcf467640 1 --2- 172.21.15.139:0/160056001 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7effac06c2e0 0x7effac06e7a0 secure :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7effc814e190 tx=0x7effc4009190 comp rx=0 tx=0).stop 2024-09-16T00:11:03.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.887+0000 7effcf467640 1 --2- 172.21.15.139:0/160056001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effc80c8990 0x7effc814d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:03.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.887+0000 7effcf467640 1 --2- 172.21.15.139:0/160056001 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7effc806d180 0x7effc81f3b00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:03.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.887+0000 7effcf467640 1 --2- 172.21.15.139:0/160056001 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7effc806cda0 0x7effc815a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:03.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.887+0000 7effcf467640 1 -- 172.21.15.139:0/160056001 >> 172.21.15.139:0/160056001 conn(0x7effc806f800 msgr2=0x7effc8152420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:03.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.887+0000 7effcf467640 1 -- 172.21.15.139:0/160056001 shutdown_connections 2024-09-16T00:11:03.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.887+0000 7effcf467640 1 -- 172.21.15.139:0/160056001 wait complete. 2024-09-16T00:11:03.899 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-16T00:11:03.999 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.999+0000 7f5b712ba640 1 Processor -- start 2024-09-16T00:11:04.000 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.999+0000 7f5b712ba640 1 -- start start 2024-09-16T00:11:04.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.999+0000 7f5b712ba640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5b6c154bd0 0x7f5b6c15b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:04.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.999+0000 7f5b712ba640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5b6c06ea20 0x7f5b6c0c3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:04.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.999+0000 7f5b712ba640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b6c06cd40 con 0x7f5b6c0c8b50 2024-09-16T00:11:04.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.999+0000 7f5b712ba640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5b6c15a9d0 con 0x7f5b6c06ea20 2024-09-16T00:11:04.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:03.999+0000 7f5b712ba640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5b6c06cfc0 con 0x7f5b6c154bd0 2024-09-16T00:11:04.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.000+0000 7f5b6affd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5b6c154bd0 0x7f5b6c15b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:04.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.000+0000 7f5b6a7fc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5b6c06ea20 0x7f5b6c0c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:04.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.000+0000 7f5b6a7fc640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5b6c06ea20 0x7f5b6c0c3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56860/0 (socket says 172.21.15.139:56860) 2024-09-16T00:11:04.001 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.000+0000 7f5b6affd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5b6c154bd0 0x7f5b6c15b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:52310/0 (socket says 172.21.15.139:52310) 2024-09-16T00:11:04.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.000+0000 7f5b6a7fc640 1 -- 172.21.15.139:0/1855752030 learned_addr learned my addr 172.21.15.139:0/1855752030 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:04.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.000+0000 7f5b6b7fe640 1 --1- 172.21.15.139:0/1855752030 >> v1:172.21.15.164:6789/0 conn(0x7f5b6c0c8b50 0x7f5b6c0ce0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:44596/0 (socket says 172.21.15.139:44596) 2024-09-16T00:11:04.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.000+0000 7f5b6affd640 1 -- 172.21.15.139:0/1855752030 >> v2:172.21.15.164:3300/0 conn(0x7f5b6c06ea20 msgr2=0x7f5b6c0c3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:04.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.000+0000 7f5b6affd640 1 --2- 172.21.15.139:0/1855752030 >> v2:172.21.15.164:3300/0 conn(0x7f5b6c06ea20 0x7f5b6c0c3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:04.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.000+0000 7f5b6affd640 1 -- 172.21.15.139:0/1855752030 >> v1:172.21.15.164:6789/0 conn(0x7f5b6c0c8b50 legacy=0x7f5b6c0ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:04.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.000+0000 7f5b6affd640 1 -- 172.21.15.139:0/1855752030 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c15ac70 con 0x7f5b6c154bd0 2024-09-16T00:11:04.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.000+0000 7f5b6a7fc640 1 --2- 172.21.15.139:0/1855752030 >> v2:172.21.15.164:3300/0 conn(0x7f5b6c06ea20 0x7f5b6c0c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:11:04.002 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.000+0000 7f5b69ffb640 1 -- 172.21.15.139:0/1855752030 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2451680181 0 0) 0x7f5b64001490 con 0x7f5b6c0c8b50 2024-09-16T00:11:04.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.001+0000 7f5b6affd640 1 --2- 172.21.15.139:0/1855752030 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5b6c154bd0 0x7f5b6c15b470 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0097a0 tx=0x7f5b5c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98dd52aaf149cec6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:04.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.001+0000 7f5b69ffb640 1 -- 172.21.15.139:0/1855752030 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c037070 con 0x7f5b6c154bd0 2024-09-16T00:11:04.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.001+0000 7f5b69ffb640 1 -- 172.21.15.139:0/1855752030 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b5c032440 con 0x7f5b6c154bd0 2024-09-16T00:11:04.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.001+0000 7f5b69ffb640 1 -- 172.21.15.139:0/1855752030 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c032810 con 0x7f5b6c154bd0 2024-09-16T00:11:04.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.001+0000 7f5b712ba640 1 -- 172.21.15.139:0/1855752030 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5b6c154bd0 msgr2=0x7f5b6c15b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:04.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.001+0000 7f5b712ba640 1 --2- 172.21.15.139:0/1855752030 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5b6c154bd0 0x7f5b6c15b470 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0097a0 tx=0x7f5b5c02a180 comp rx=0 tx=0).stop 2024-09-16T00:11:04.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.001+0000 7f5b712ba640 1 -- 172.21.15.139:0/1855752030 shutdown_connections 2024-09-16T00:11:04.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.001+0000 7f5b712ba640 1 --2- 172.21.15.139:0/1855752030 >> v2:172.21.15.164:3300/0 conn(0x7f5b6c06ea20 0x7f5b6c0c3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:04.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.001+0000 7f5b712ba640 1 --2- 172.21.15.139:0/1855752030 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5b6c154bd0 0x7f5b6c15b470 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:04.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.002+0000 7f5b712ba640 1 -- 172.21.15.139:0/1855752030 >> 172.21.15.139:0/1855752030 conn(0x7f5b6c06c150 msgr2=0x7f5b6c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:04.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.002+0000 7f5b712ba640 1 -- 172.21.15.139:0/1855752030 shutdown_connections 2024-09-16T00:11:04.004 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.002+0000 7f5b712ba640 1 -- 172.21.15.139:0/1855752030 wait complete. 2024-09-16T00:11:04.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.003+0000 7f5b712ba640 1 Processor -- start 2024-09-16T00:11:04.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.003+0000 7f5b712ba640 1 -- start start 2024-09-16T00:11:04.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b712ba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b6c06ea20 0x7f5b6c14a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:04.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b712ba640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6c0c8b50 0x7f5b6c1f0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:04.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b712ba640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5b6c154bd0 0x7f5b6c1f0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:04.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b712ba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c06d2a0 con 0x7f5b6c06ea20 2024-09-16T00:11:04.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b712ba640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c154fb0 con 0x7f5b6c0c8b50 2024-09-16T00:11:04.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b712ba640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5b6c06d420 con 0x7f5b6c154bd0 2024-09-16T00:11:04.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b6affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b6c06ea20 0x7f5b6c14a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:04.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b6affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b6c06ea20 0x7f5b6c14a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60388/0 (socket says 172.21.15.139:60388) 2024-09-16T00:11:04.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b6affd640 1 -- 172.21.15.139:0/18479175 learned_addr learned my addr 172.21.15.139:0/18479175 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:04.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b6b7fe640 1 --2- 172.21.15.139:0/18479175 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5b6c154bd0 0x7f5b6c1f0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:04.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b6a7fc640 1 --2- 172.21.15.139:0/18479175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6c0c8b50 0x7f5b6c1f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:04.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b6affd640 1 -- 172.21.15.139:0/18479175 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5b6c154bd0 msgr2=0x7f5b6c1f0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:04.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b6affd640 1 --2- 172.21.15.139:0/18479175 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5b6c154bd0 0x7f5b6c1f0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:04.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b6affd640 1 -- 172.21.15.139:0/18479175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6c0c8b50 msgr2=0x7f5b6c1f0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:04.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b6affd640 1 --2- 172.21.15.139:0/18479175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6c0c8b50 0x7f5b6c1f0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:04.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b6affd640 1 -- 172.21.15.139:0/18479175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c150d40 con 0x7f5b6c06ea20 2024-09-16T00:11:04.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b6a7fc640 1 --2- 172.21.15.139:0/18479175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6c0c8b50 0x7f5b6c1f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:04.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b6affd640 1 --2- 172.21.15.139:0/18479175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b6c06ea20 0x7f5b6c14a500 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0097a0 tx=0x7f5b5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:04.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b637fe640 1 -- 172.21.15.139:0/18479175 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c037050 con 0x7f5b6c06ea20 2024-09-16T00:11:04.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b712ba640 1 -- 172.21.15.139:0/18479175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c150fe0 con 0x7f5b6c06ea20 2024-09-16T00:11:04.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b637fe640 1 -- 172.21.15.139:0/18479175 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b5c038040 con 0x7f5b6c06ea20 2024-09-16T00:11:04.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b637fe640 1 -- 172.21.15.139:0/18479175 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c004490 con 0x7f5b6c06ea20 2024-09-16T00:11:04.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.004+0000 7f5b712ba640 1 -- 172.21.15.139:0/18479175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c1f3b00 con 0x7f5b6c06ea20 2024-09-16T00:11:04.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.005+0000 7f5b712ba640 1 -- 172.21.15.139:0/18479175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b6c0c3170 con 0x7f5b6c06ea20 2024-09-16T00:11:04.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.006+0000 7f5b637fe640 1 -- 172.21.15.139:0/18479175 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5b5c03c6d0 con 0x7f5b6c06ea20 2024-09-16T00:11:04.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.007+0000 7f5b637fe640 1 --2- 172.21.15.139:0/18479175 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5b4c06c2b0 0x7f5b4c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:04.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.007+0000 7f5b6a7fc640 1 --2- 172.21.15.139:0/18479175 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5b4c06c2b0 0x7f5b4c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:04.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.007+0000 7f5b637fe640 1 -- 172.21.15.139:0/18479175 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5944+0+0 (secure 0 0 0) 0x7f5b5c0abf10 con 0x7f5b6c06ea20 2024-09-16T00:11:04.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.007+0000 7f5b6a7fc640 1 --2- 172.21.15.139:0/18479175 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5b4c06c2b0 0x7f5b4c06e770 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009630 tx=0x7f5b54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:04.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.011+0000 7f5b637fe640 1 -- 172.21.15.139:0/18479175 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5b5c07a620 con 0x7f5b6c06ea20 2024-09-16T00:11:04.191 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.190+0000 7f5b712ba640 1 -- 172.21.15.139:0/18479175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) -- 0x7f5b6c06e420 con 0x7f5b6c06ea20 2024-09-16T00:11:04.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.882+0000 7f5b637fe640 1 -- 172.21.15.139:0/18479175 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v137) ==== 113+0+0 (secure 0 0 0) 0x7f5b5c079f20 con 0x7f5b6c06ea20 2024-09-16T00:11:04.884 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T00:11:04.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.885+0000 7f5b712ba640 1 -- 172.21.15.139:0/18479175 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5b4c06c2b0 msgr2=0x7f5b4c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:04.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.885+0000 7f5b712ba640 1 --2- 172.21.15.139:0/18479175 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5b4c06c2b0 0x7f5b4c06e770 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009630 tx=0x7f5b54008040 comp rx=0 tx=0).stop 2024-09-16T00:11:04.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.885+0000 7f5b712ba640 1 -- 172.21.15.139:0/18479175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b6c06ea20 msgr2=0x7f5b6c14a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:04.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.885+0000 7f5b712ba640 1 --2- 172.21.15.139:0/18479175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b6c06ea20 0x7f5b6c14a500 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0097a0 tx=0x7f5b5c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:11:04.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.885+0000 7f5b712ba640 1 -- 172.21.15.139:0/18479175 shutdown_connections 2024-09-16T00:11:04.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.885+0000 7f5b712ba640 1 --2- 172.21.15.139:0/18479175 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f5b4c06c2b0 0x7f5b4c06e770 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:04.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.885+0000 7f5b712ba640 1 --2- 172.21.15.139:0/18479175 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5b6c154bd0 0x7f5b6c1f0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:04.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.885+0000 7f5b712ba640 1 --2- 172.21.15.139:0/18479175 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5b6c0c8b50 0x7f5b6c1f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:04.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.885+0000 7f5b712ba640 1 --2- 172.21.15.139:0/18479175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5b6c06ea20 0x7f5b6c14a500 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:04.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.885+0000 7f5b712ba640 1 -- 172.21.15.139:0/18479175 >> 172.21.15.139:0/18479175 conn(0x7f5b6c06c150 msgr2=0x7f5b6c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:04.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.885+0000 7f5b712ba640 1 -- 172.21.15.139:0/18479175 shutdown_connections 2024-09-16T00:11:04.887 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:04.885+0000 7f5b712ba640 1 -- 172.21.15.139:0/18479175 wait complete. 2024-09-16T00:11:04.903 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-16T00:11:05.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.004+0000 7f54de2f1640 1 Processor -- start 2024-09-16T00:11:05.005 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.004+0000 7f54de2f1640 1 -- start start 2024-09-16T00:11:05.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.005+0000 7f54de2f1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f54d81524f0 0x7f54d8158630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:05.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.005+0000 7f54de2f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54d8072f00 0x7f54d80c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:05.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.005+0000 7f54de2f1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54d80702c0 con 0x7f54d81528d0 2024-09-16T00:11:05.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.005+0000 7f54de2f1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f54d80c9ae0 con 0x7f54d81524f0 2024-09-16T00:11:05.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.005+0000 7f54de2f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f54d80c6c00 con 0x7f54d8072f00 2024-09-16T00:11:05.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.005+0000 7f54d7fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f54d81524f0 0x7f54d8158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:05.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.005+0000 7f54d77fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f54d81528d0 0x7f54d80c5910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54456/0 (socket says 172.21.15.139:54456) 2024-09-16T00:11:05.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.005+0000 7f54d77fe640 1 -- 172.21.15.139:0/1831452196 learned_addr learned my addr 172.21.15.139:0/1831452196 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:05.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.005+0000 7f54dc88e640 1 --2- 172.21.15.139:0/1831452196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54d8072f00 0x7f54d80c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:05.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.005+0000 7f54d7fff640 1 -- 172.21.15.139:0/1831452196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54d8072f00 msgr2=0x7f54d80c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:05.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.005+0000 7f54d7fff640 1 --2- 172.21.15.139:0/1831452196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54d8072f00 0x7f54d80c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.007 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.005+0000 7f54d7fff640 1 -- 172.21.15.139:0/1831452196 >> v1:172.21.15.139:6789/0 conn(0x7f54d81528d0 legacy=0x7f54d80c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:05.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.005+0000 7f54d6ffd640 1 -- 172.21.15.139:0/1831452196 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3560523160 0 0) 0x7f54d80702c0 con 0x7f54d81528d0 2024-09-16T00:11:05.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.005+0000 7f54d7fff640 1 -- 172.21.15.139:0/1831452196 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d80c8320 con 0x7f54d81524f0 2024-09-16T00:11:05.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.006+0000 7f54d7fff640 1 --2- 172.21.15.139:0/1831452196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54d81524f0 0x7f54d8158630 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f54c80097a0 tx=0x7f54c802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18342fa33376ca3e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:05.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.006+0000 7f54d6ffd640 1 -- 172.21.15.139:0/1831452196 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8037070 con 0x7f54d81524f0 2024-09-16T00:11:05.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.006+0000 7f54d6ffd640 1 -- 172.21.15.139:0/1831452196 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c8032440 con 0x7f54d81524f0 2024-09-16T00:11:05.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.006+0000 7f54d6ffd640 1 -- 172.21.15.139:0/1831452196 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8032810 con 0x7f54d81524f0 2024-09-16T00:11:05.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.006+0000 7f54de2f1640 1 -- 172.21.15.139:0/1831452196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54d81524f0 msgr2=0x7f54d8158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:05.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.006+0000 7f54de2f1640 1 --2- 172.21.15.139:0/1831452196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54d81524f0 0x7f54d8158630 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f54c80097a0 tx=0x7f54c802a180 comp rx=0 tx=0).stop 2024-09-16T00:11:05.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.006+0000 7f54de2f1640 1 -- 172.21.15.139:0/1831452196 shutdown_connections 2024-09-16T00:11:05.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.006+0000 7f54de2f1640 1 --2- 172.21.15.139:0/1831452196 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54d8072f00 0x7f54d80c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.006+0000 7f54de2f1640 1 --2- 172.21.15.139:0/1831452196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54d81524f0 0x7f54d8158630 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.006+0000 7f54de2f1640 1 -- 172.21.15.139:0/1831452196 >> 172.21.15.139:0/1831452196 conn(0x7f54d806da40 msgr2=0x7f54d806f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:05.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.006+0000 7f54de2f1640 1 -- 172.21.15.139:0/1831452196 shutdown_connections 2024-09-16T00:11:05.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.006+0000 7f54de2f1640 1 -- 172.21.15.139:0/1831452196 wait complete. 2024-09-16T00:11:05.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.008+0000 7f54de2f1640 1 Processor -- start 2024-09-16T00:11:05.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.008+0000 7f54de2f1640 1 -- start start 2024-09-16T00:11:05.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.008+0000 7f54de2f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54d8072f00 0x7f54d81eee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:05.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.008+0000 7f54de2f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54d81524f0 0x7f54d814a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:05.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.008+0000 7f54de2f1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54d81528d0 0x7f54d80ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:05.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.008+0000 7f54de2f1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f54d806ffe0 con 0x7f54d81528d0 2024-09-16T00:11:05.012 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.008+0000 7f54de2f1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f54d806fd00 con 0x7f54d81524f0 2024-09-16T00:11:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.008+0000 7f54de2f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f54d80c8da0 con 0x7f54d8072f00 2024-09-16T00:11:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.009+0000 7f54dc88e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54d81528d0 0x7f54d80ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.009+0000 7f54dc88e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54d81528d0 0x7f54d80ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60400/0 (socket says 172.21.15.139:60400) 2024-09-16T00:11:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.009+0000 7f54dc88e640 1 -- 172.21.15.139:0/1816293952 learned_addr learned my addr 172.21.15.139:0/1816293952 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.009+0000 7f54d77fe640 1 --2- 172.21.15.139:0/1816293952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54d81524f0 0x7f54d814a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.009+0000 7f54dc88e640 1 -- 172.21.15.139:0/1816293952 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54d8072f00 msgr2=0x7f54d81eee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.009+0000 7f54dc88e640 1 --2- 172.21.15.139:0/1816293952 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54d8072f00 0x7f54d81eee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.009+0000 7f54dc88e640 1 -- 172.21.15.139:0/1816293952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54d81524f0 msgr2=0x7f54d814a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.009+0000 7f54dc88e640 1 --2- 172.21.15.139:0/1816293952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54d81524f0 0x7f54d814a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.009+0000 7f54dc88e640 1 -- 172.21.15.139:0/1816293952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d81ef3a0 con 0x7f54d81528d0 2024-09-16T00:11:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.009+0000 7f54dc88e640 1 --2- 172.21.15.139:0/1816293952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54d81528d0 0x7f54d80ca8a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f54d000cd70 tx=0x7f54d000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.009+0000 7f54d4ff9640 1 -- 172.21.15.139:0/1816293952 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0003b90 con 0x7f54d81528d0 2024-09-16T00:11:05.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.009+0000 7f54d4ff9640 1 -- 172.21.15.139:0/1816293952 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d0003d30 con 0x7f54d81528d0 2024-09-16T00:11:05.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.009+0000 7f54de2f1640 1 -- 172.21.15.139:0/1816293952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d8146320 con 0x7f54d81528d0 2024-09-16T00:11:05.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.009+0000 7f54d4ff9640 1 -- 172.21.15.139:0/1816293952 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0004240 con 0x7f54d81528d0 2024-09-16T00:11:05.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.009+0000 7f54de2f1640 1 -- 172.21.15.139:0/1816293952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d8145d60 con 0x7f54d81528d0 2024-09-16T00:11:05.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.010+0000 7f54de2f1640 1 -- 172.21.15.139:0/1816293952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54d80718b0 con 0x7f54d81528d0 2024-09-16T00:11:05.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.011+0000 7f54d4ff9640 1 -- 172.21.15.139:0/1816293952 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f54d00043e0 con 0x7f54d81528d0 2024-09-16T00:11:05.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.011+0000 7f54d4ff9640 1 --2- 172.21.15.139:0/1816293952 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f54c406c2b0 0x7f54c406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:05.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.011+0000 7f54d4ff9640 1 -- 172.21.15.139:0/1816293952 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5944+0+0 (secure 0 0 0) 0x7f54d008bf30 con 0x7f54d81528d0 2024-09-16T00:11:05.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.011+0000 7f54d7fff640 1 --2- 172.21.15.139:0/1816293952 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f54c406c2b0 0x7f54c406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:05.015 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.012+0000 7f54d7fff640 1 --2- 172.21.15.139:0/1816293952 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f54c406c2b0 0x7f54c406e770 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f54c8034000 tx=0x7f54c802c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:05.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.015+0000 7f54d4ff9640 1 -- 172.21.15.139:0/1816293952 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f54d005a640 con 0x7f54d81528d0 2024-09-16T00:11:05.192 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.191+0000 7f54de2f1640 1 -- 172.21.15.139:0/1816293952 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} v 0) -- 0x7f54d814b800 con 0x7f54d81528d0 2024-09-16T00:11:05.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.884+0000 7f54d4ff9640 1 -- 172.21.15.139:0/1816293952 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]=0 set osd.3 primary-affinity to 1 (8655362) v138) ==== 133+0+0 (secure 0 0 0) 0x7f54d0059f40 con 0x7f54d81528d0 2024-09-16T00:11:05.885 INFO:teuthology.orchestra.run.smithi139.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-16T00:11:05.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.887+0000 7f54de2f1640 1 -- 172.21.15.139:0/1816293952 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f54c406c2b0 msgr2=0x7f54c406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:05.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.887+0000 7f54de2f1640 1 --2- 172.21.15.139:0/1816293952 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f54c406c2b0 0x7f54c406e770 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f54c8034000 tx=0x7f54c802c000 comp rx=0 tx=0).stop 2024-09-16T00:11:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.887+0000 7f54de2f1640 1 -- 172.21.15.139:0/1816293952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54d81528d0 msgr2=0x7f54d80ca8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.887+0000 7f54de2f1640 1 --2- 172.21.15.139:0/1816293952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54d81528d0 0x7f54d80ca8a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f54d000cd70 tx=0x7f54d000a660 comp rx=0 tx=0).stop 2024-09-16T00:11:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.887+0000 7f54de2f1640 1 -- 172.21.15.139:0/1816293952 shutdown_connections 2024-09-16T00:11:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.887+0000 7f54de2f1640 1 --2- 172.21.15.139:0/1816293952 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f54c406c2b0 0x7f54c406e770 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.887+0000 7f54de2f1640 1 --2- 172.21.15.139:0/1816293952 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f54d81528d0 0x7f54d80ca8a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.887+0000 7f54de2f1640 1 --2- 172.21.15.139:0/1816293952 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f54d81524f0 0x7f54d814a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.887+0000 7f54de2f1640 1 --2- 172.21.15.139:0/1816293952 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f54d8072f00 0x7f54d81eee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.887+0000 7f54de2f1640 1 -- 172.21.15.139:0/1816293952 >> 172.21.15.139:0/1816293952 conn(0x7f54d806da40 msgr2=0x7f54d81576b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.887+0000 7f54de2f1640 1 -- 172.21.15.139:0/1816293952 shutdown_connections 2024-09-16T00:11:05.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:05.887+0000 7f54de2f1640 1 -- 172.21.15.139:0/1816293952 wait complete. 2024-09-16T00:11:05.904 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-16T00:11:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.012+0000 7f07fa6f1640 1 Processor -- start 2024-09-16T00:11:06.013 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.012+0000 7f07fa6f1640 1 -- start start 2024-09-16T00:11:06.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.012+0000 7f07fa6f1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f07f415b000 0x7f07f415b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:06.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.012+0000 7f07fa6f1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f40c7250 con 0x7f07f4154bf0 2024-09-16T00:11:06.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.012+0000 7f07fa6f1640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f406c530 con 0x7f07f4073740 2024-09-16T00:11:06.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.012+0000 7f07fa6f1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f07f415ab70 con 0x7f07f415b000 2024-09-16T00:11:06.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.013+0000 7f07f37fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f07f415b000 0x7f07f415b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:06.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.013+0000 7f07f3fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f07f4154bf0 0x7f07f4072e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:54468/0 (socket says 172.21.15.139:54468) 2024-09-16T00:11:06.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.013+0000 7f07f3fff640 1 -- 172.21.15.139:0/1827991675 learned_addr learned my addr 172.21.15.139:0/1827991675 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:06.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.013+0000 7f07f37fe640 1 -- 172.21.15.139:0/1827991675 >> v1:172.21.15.164:6789/0 conn(0x7f07f4073740 legacy=0x7f07f41651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:06.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.013+0000 7f07f37fe640 1 -- 172.21.15.139:0/1827991675 >> v1:172.21.15.139:6789/0 conn(0x7f07f4154bf0 legacy=0x7f07f4072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:06.014 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.013+0000 7f07f37fe640 1 -- 172.21.15.139:0/1827991675 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f415a820 con 0x7f07f415b000 2024-09-16T00:11:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.014+0000 7f07f37fe640 1 --2- 172.21.15.139:0/1827991675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f415b000 0x7f07f415b420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009790 tx=0x7f07e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fd6c7431c1537d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.014+0000 7f07f2ffd640 1 -- 172.21.15.139:0/1827991675 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8037070 con 0x7f07f415b000 2024-09-16T00:11:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.014+0000 7f07f2ffd640 1 -- 172.21.15.139:0/1827991675 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e8032440 con 0x7f07f415b000 2024-09-16T00:11:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.014+0000 7f07f2ffd640 1 -- 172.21.15.139:0/1827991675 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e8032810 con 0x7f07f415b000 2024-09-16T00:11:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.014+0000 7f07fa6f1640 1 -- 172.21.15.139:0/1827991675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f415b000 msgr2=0x7f07f415b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.014+0000 7f07fa6f1640 1 --2- 172.21.15.139:0/1827991675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f415b000 0x7f07f415b420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009790 tx=0x7f07e802a270 comp rx=0 tx=0).stop 2024-09-16T00:11:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.014+0000 7f07fa6f1640 1 -- 172.21.15.139:0/1827991675 shutdown_connections 2024-09-16T00:11:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.014+0000 7f07fa6f1640 1 --2- 172.21.15.139:0/1827991675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f415b000 0x7f07f415b420 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.014+0000 7f07fa6f1640 1 -- 172.21.15.139:0/1827991675 >> 172.21.15.139:0/1827991675 conn(0x7f07f40c9350 msgr2=0x7f07f40c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.014+0000 7f07fa6f1640 1 -- 172.21.15.139:0/1827991675 shutdown_connections 2024-09-16T00:11:06.016 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.014+0000 7f07fa6f1640 1 -- 172.21.15.139:0/1827991675 wait complete. 2024-09-16T00:11:06.018 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.015+0000 7f07fa6f1640 1 Processor -- start 2024-09-16T00:11:06.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.016+0000 7f07fa6f1640 1 -- start start 2024-09-16T00:11:06.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.016+0000 7f07fa6f1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f4073740 0x7f07f41e55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:06.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.016+0000 7f07fa6f1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f4154bf0 0x7f07f41f0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:06.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.016+0000 7f07fa6f1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f07f415b000 0x7f07f41f1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:06.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.016+0000 7f07fa6f1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f07f406db30 con 0x7f07f4073740 2024-09-16T00:11:06.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.016+0000 7f07fa6f1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f07f4154fd0 con 0x7f07f4154bf0 2024-09-16T00:11:06.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.016+0000 7f07fa6f1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f07f4073560 con 0x7f07f415b000 2024-09-16T00:11:06.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.016+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f4073740 0x7f07f41e55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:06.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.017+0000 7f07f3fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f4073740 0x7f07f41e55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60424/0 (socket says 172.21.15.139:60424) 2024-09-16T00:11:06.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.017+0000 7f07f3fff640 1 -- 172.21.15.139:0/2725105674 learned_addr learned my addr 172.21.15.139:0/2725105674 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:06.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.017+0000 7f07f37fe640 1 --2- 172.21.15.139:0/2725105674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f4154bf0 0x7f07f41f0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:06.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.017+0000 7f07f8c8e640 1 --2- 172.21.15.139:0/2725105674 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f07f415b000 0x7f07f41f1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:06.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.017+0000 7f07f3fff640 1 -- 172.21.15.139:0/2725105674 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f07f415b000 msgr2=0x7f07f41f1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:06.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.017+0000 7f07f3fff640 1 --2- 172.21.15.139:0/2725105674 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f07f415b000 0x7f07f41f1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:06.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.017+0000 7f07f3fff640 1 -- 172.21.15.139:0/2725105674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f4154bf0 msgr2=0x7f07f41f0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:06.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.017+0000 7f07f3fff640 1 --2- 172.21.15.139:0/2725105674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f4154bf0 0x7f07f41f0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:06.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.017+0000 7f07f3fff640 1 -- 172.21.15.139:0/2725105674 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f4155830 con 0x7f07f4073740 2024-09-16T00:11:06.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.017+0000 7f07f37fe640 1 --2- 172.21.15.139:0/2725105674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f4154bf0 0x7f07f41f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:06.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.017+0000 7f07f3fff640 1 --2- 172.21.15.139:0/2725105674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f4073740 0x7f07f41e55f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f07e400c430 tx=0x7f07e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:06.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.017+0000 7f07f0ff9640 1 -- 172.21.15.139:0/2725105674 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e4017070 con 0x7f07f4073740 2024-09-16T00:11:06.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.017+0000 7f07f0ff9640 1 -- 172.21.15.139:0/2725105674 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e4005ce0 con 0x7f07f4073740 2024-09-16T00:11:06.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.017+0000 7f07fa6f1640 1 -- 172.21.15.139:0/2725105674 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f4155570 con 0x7f07f4073740 2024-09-16T00:11:06.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.017+0000 7f07f0ff9640 1 -- 172.21.15.139:0/2725105674 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e40072f0 con 0x7f07f4073740 2024-09-16T00:11:06.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.017+0000 7f07fa6f1640 1 -- 172.21.15.139:0/2725105674 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f4156f30 con 0x7f07f4073740 2024-09-16T00:11:06.020 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.018+0000 7f07fa6f1640 1 -- 172.21.15.139:0/2725105674 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f406d090 con 0x7f07f4073740 2024-09-16T00:11:06.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.019+0000 7f07f0ff9640 1 -- 172.21.15.139:0/2725105674 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f07e401a070 con 0x7f07f4073740 2024-09-16T00:11:06.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.019+0000 7f07f0ff9640 1 --2- 172.21.15.139:0/2725105674 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07e006c2b0 0x7f07e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:06.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.019+0000 7f07f0ff9640 1 -- 172.21.15.139:0/2725105674 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5944+0+0 (secure 0 0 0) 0x7f07e408a8c0 con 0x7f07f4073740 2024-09-16T00:11:06.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.019+0000 7f07f37fe640 1 --2- 172.21.15.139:0/2725105674 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07e006c2b0 0x7f07e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:06.021 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.020+0000 7f07f37fe640 1 --2- 172.21.15.139:0/2725105674 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07e006c2b0 0x7f07e006e770 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f07e8034000 tx=0x7f07e802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:06.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.024+0000 7f07f0ff9640 1 -- 172.21.15.139:0/2725105674 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f07f406d090 con 0x7f07f4073740 2024-09-16T00:11:06.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.201+0000 7f07fa6f1640 1 -- 172.21.15.139:0/2725105674 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) -- 0x7f07f406cdb0 con 0x7f07f4073740 2024-09-16T00:11:06.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.890+0000 7f07f0ff9640 1 -- 172.21.15.139:0/2725105674 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v139) ==== 113+0+0 (secure 0 0 0) 0x7f07f40c7250 con 0x7f07f4073740 2024-09-16T00:11:06.892 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T00:11:06.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.893+0000 7f07fa6f1640 1 -- 172.21.15.139:0/2725105674 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07e006c2b0 msgr2=0x7f07e006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:06.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.893+0000 7f07fa6f1640 1 --2- 172.21.15.139:0/2725105674 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07e006c2b0 0x7f07e006e770 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f07e8034000 tx=0x7f07e802e000 comp rx=0 tx=0).stop 2024-09-16T00:11:06.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.893+0000 7f07fa6f1640 1 -- 172.21.15.139:0/2725105674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f4073740 msgr2=0x7f07f41e55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:06.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.893+0000 7f07fa6f1640 1 --2- 172.21.15.139:0/2725105674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f4073740 0x7f07f41e55f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f07e400c430 tx=0x7f07e400c8f0 comp rx=0 tx=0).stop 2024-09-16T00:11:06.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.893+0000 7f07fa6f1640 1 -- 172.21.15.139:0/2725105674 shutdown_connections 2024-09-16T00:11:06.895 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.893+0000 7f07fa6f1640 1 --2- 172.21.15.139:0/2725105674 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07e006c2b0 0x7f07e006e770 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:06.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.893+0000 7f07fa6f1640 1 --2- 172.21.15.139:0/2725105674 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f07f415b000 0x7f07f41f1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:06.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.893+0000 7f07fa6f1640 1 --2- 172.21.15.139:0/2725105674 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07f4154bf0 0x7f07f41f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:06.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.893+0000 7f07fa6f1640 1 --2- 172.21.15.139:0/2725105674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07f4073740 0x7f07f41e55f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:06.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.893+0000 7f07fa6f1640 1 -- 172.21.15.139:0/2725105674 >> 172.21.15.139:0/2725105674 conn(0x7f07f40c9350 msgr2=0x7f07f4072e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:06.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.893+0000 7f07fa6f1640 1 -- 172.21.15.139:0/2725105674 shutdown_connections 2024-09-16T00:11:06.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:06.893+0000 7f07fa6f1640 1 -- 172.21.15.139:0/2725105674 wait complete. 2024-09-16T00:11:06.909 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-16T00:11:07.017 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.015+0000 7f13a675f640 1 Processor -- start 2024-09-16T00:11:07.216 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.015+0000 7f13a675f640 1 -- start start 2024-09-16T00:11:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.016+0000 7f13a675f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f13a0151b50 0x7f13a00729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.016+0000 7f13a675f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f13a00c98c0 0x7f13a0072f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.016+0000 7f13a675f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13a00c37a0 con 0x7f13a00c6310 2024-09-16T00:11:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.016+0000 7f13a675f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f13a00c3a80 con 0x7f13a0151b50 2024-09-16T00:11:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.016+0000 7f13a675f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f13a00c7380 con 0x7f13a00c98c0 2024-09-16T00:11:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.016+0000 7f139ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f13a0151b50 0x7f13a00729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.016+0000 7f139ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f13a0151b50 0x7f13a00729c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60436/0 (socket says 172.21.15.139:60436) 2024-09-16T00:11:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.016+0000 7f139ffff640 1 -- 172.21.15.139:0/100482770 learned_addr learned my addr 172.21.15.139:0/100482770 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.016+0000 7f139f7fe640 1 --2- 172.21.15.139:0/100482770 >> v2:172.21.15.164:3300/0 conn(0x7f13a00c98c0 0x7f13a0072f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.016+0000 7f139ffff640 1 -- 172.21.15.139:0/100482770 >> v2:172.21.15.164:3300/0 conn(0x7f13a00c98c0 msgr2=0x7f13a0072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:07.221 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.016+0000 7f139ffff640 1 --2- 172.21.15.139:0/100482770 >> v2:172.21.15.164:3300/0 conn(0x7f13a00c98c0 0x7f13a0072f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.016+0000 7f139ffff640 1 -- 172.21.15.139:0/100482770 >> v1:172.21.15.164:6789/0 conn(0x7f13a00c6310 legacy=0x7f13a0157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:07.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.016+0000 7f139ffff640 1 -- 172.21.15.139:0/100482770 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a00c7660 con 0x7f13a0151b50 2024-09-16T00:11:07.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.017+0000 7f139ffff640 1 --2- 172.21.15.139:0/100482770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13a0151b50 0x7f13a00729c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f13a00c7dd0 tx=0x7f139002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738e9ad0e0737231 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:07.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.017+0000 7f139effd640 1 -- 172.21.15.139:0/100482770 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390037070 con 0x7f13a0151b50 2024-09-16T00:11:07.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.017+0000 7f139effd640 1 -- 172.21.15.139:0/100482770 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139002ae50 con 0x7f13a0151b50 2024-09-16T00:11:07.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.017+0000 7f139effd640 1 -- 172.21.15.139:0/100482770 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390032860 con 0x7f13a0151b50 2024-09-16T00:11:07.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.018+0000 7f13a675f640 1 -- 172.21.15.139:0/100482770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13a0151b50 msgr2=0x7f13a00729c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:07.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.018+0000 7f13a675f640 1 --2- 172.21.15.139:0/100482770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13a0151b50 0x7f13a00729c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f13a00c7dd0 tx=0x7f139002a300 comp rx=0 tx=0).stop 2024-09-16T00:11:07.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.018+0000 7f13a675f640 1 -- 172.21.15.139:0/100482770 shutdown_connections 2024-09-16T00:11:07.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.018+0000 7f13a675f640 1 --2- 172.21.15.139:0/100482770 >> v2:172.21.15.164:3300/0 conn(0x7f13a00c98c0 0x7f13a0072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.018+0000 7f13a675f640 1 --2- 172.21.15.139:0/100482770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13a0151b50 0x7f13a00729c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.018+0000 7f13a675f640 1 -- 172.21.15.139:0/100482770 >> 172.21.15.139:0/100482770 conn(0x7f13a006da40 msgr2=0x7f13a006f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:07.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.018+0000 7f13a675f640 1 -- 172.21.15.139:0/100482770 shutdown_connections 2024-09-16T00:11:07.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.018+0000 7f13a675f640 1 -- 172.21.15.139:0/100482770 wait complete. 2024-09-16T00:11:07.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.019+0000 7f13a675f640 1 Processor -- start 2024-09-16T00:11:07.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.019+0000 7f13a675f640 1 -- start start 2024-09-16T00:11:07.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f13a675f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13a00c6310 0x7f13a0154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:07.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f13a675f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a00c98c0 0x7f13a0141be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:07.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f13a675f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a0151b50 0x7f13a0155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:07.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f13a675f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f13a0157f90 con 0x7f13a00c6310 2024-09-16T00:11:07.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f13a675f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f13a0157e10 con 0x7f13a00c98c0 2024-09-16T00:11:07.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f13a675f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f13a01570e0 con 0x7f13a0151b50 2024-09-16T00:11:07.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f139ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13a00c6310 0x7f13a0154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:07.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f139ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13a00c6310 0x7f13a0154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60440/0 (socket says 172.21.15.139:60440) 2024-09-16T00:11:07.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f139ffff640 1 -- 172.21.15.139:0/972273267 learned_addr learned my addr 172.21.15.139:0/972273267 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:07.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f139f7fe640 1 --2- 172.21.15.139:0/972273267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a00c98c0 0x7f13a0141be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:07.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f13a4cfc640 1 --2- 172.21.15.139:0/972273267 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a0151b50 0x7f13a0155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:07.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f139ffff640 1 -- 172.21.15.139:0/972273267 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a0151b50 msgr2=0x7f13a0155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:07.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f139ffff640 1 --2- 172.21.15.139:0/972273267 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a0151b50 0x7f13a0155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f139ffff640 1 -- 172.21.15.139:0/972273267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a00c98c0 msgr2=0x7f13a0141be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:07.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f139ffff640 1 --2- 172.21.15.139:0/972273267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a00c98c0 0x7f13a0141be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f139ffff640 1 -- 172.21.15.139:0/972273267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a01ee1d0 con 0x7f13a00c6310 2024-09-16T00:11:07.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f13a4cfc640 1 --2- 172.21.15.139:0/972273267 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a0151b50 0x7f13a0155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:07.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.020+0000 7f139ffff640 1 --2- 172.21.15.139:0/972273267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13a00c6310 0x7f13a0154c30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1390000c00 tx=0x7f13900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:07.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.021+0000 7f139cff9640 1 -- 172.21.15.139:0/972273267 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390041020 con 0x7f13a00c6310 2024-09-16T00:11:07.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.021+0000 7f139cff9640 1 -- 172.21.15.139:0/972273267 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139003c670 con 0x7f13a00c6310 2024-09-16T00:11:07.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.021+0000 7f13a675f640 1 -- 172.21.15.139:0/972273267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a01ee4b0 con 0x7f13a00c6310 2024-09-16T00:11:07.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.021+0000 7f139cff9640 1 -- 172.21.15.139:0/972273267 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390037040 con 0x7f13a00c6310 2024-09-16T00:11:07.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.021+0000 7f13a675f640 1 -- 172.21.15.139:0/972273267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a01eea70 con 0x7f13a00c6310 2024-09-16T00:11:07.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.022+0000 7f13a675f640 1 -- 172.21.15.139:0/972273267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a00c6c00 con 0x7f13a00c6310 2024-09-16T00:11:07.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.022+0000 7f139cff9640 1 -- 172.21.15.139:0/972273267 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1390044020 con 0x7f13a00c6310 2024-09-16T00:11:07.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.023+0000 7f139cff9640 1 --2- 172.21.15.139:0/972273267 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f138c06c2e0 0x7f138c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:07.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.023+0000 7f139cff9640 1 -- 172.21.15.139:0/972273267 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5944+0+0 (secure 0 0 0) 0x7f13900ab7b0 con 0x7f13a00c6310 2024-09-16T00:11:07.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.023+0000 7f139f7fe640 1 --2- 172.21.15.139:0/972273267 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f138c06c2e0 0x7f138c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:07.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.023+0000 7f139f7fe640 1 --2- 172.21.15.139:0/972273267 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f138c06c2e0 0x7f138c06e7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f13a0142540 tx=0x7f1394008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:07.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.027+0000 7f139cff9640 1 -- 172.21.15.139:0/972273267 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f139004b070 con 0x7f13a00c6310 2024-09-16T00:11:07.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.204+0000 7f13a675f640 1 -- 172.21.15.139:0/972273267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} v 0) -- 0x7f13a00c96a0 con 0x7f13a00c6310 2024-09-16T00:11:07.899 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.897+0000 7f139cff9640 1 -- 172.21.15.139:0/972273267 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]=0 set osd.4 primary-affinity to 1 (8655362) v140) ==== 133+0+0 (secure 0 0 0) 0x7f1390079e30 con 0x7f13a00c6310 2024-09-16T00:11:07.899 INFO:teuthology.orchestra.run.smithi139.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-16T00:11:07.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.900+0000 7f13a675f640 1 -- 172.21.15.139:0/972273267 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f138c06c2e0 msgr2=0x7f138c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:07.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.900+0000 7f13a675f640 1 --2- 172.21.15.139:0/972273267 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f138c06c2e0 0x7f138c06e7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f13a0142540 tx=0x7f1394008040 comp rx=0 tx=0).stop 2024-09-16T00:11:07.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.900+0000 7f13a675f640 1 -- 172.21.15.139:0/972273267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13a00c6310 msgr2=0x7f13a0154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:07.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.900+0000 7f13a675f640 1 --2- 172.21.15.139:0/972273267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13a00c6310 0x7f13a0154c30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1390000c00 tx=0x7f13900046c0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.900+0000 7f13a675f640 1 -- 172.21.15.139:0/972273267 shutdown_connections 2024-09-16T00:11:07.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.900+0000 7f13a675f640 1 --2- 172.21.15.139:0/972273267 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f138c06c2e0 0x7f138c06e7a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.900+0000 7f13a675f640 1 --2- 172.21.15.139:0/972273267 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f13a0151b50 0x7f13a0155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.902 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.900+0000 7f13a675f640 1 --2- 172.21.15.139:0/972273267 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f13a00c98c0 0x7f13a0141be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.900+0000 7f13a675f640 1 --2- 172.21.15.139:0/972273267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13a00c6310 0x7f13a0154c30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:07.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.900+0000 7f13a675f640 1 -- 172.21.15.139:0/972273267 >> 172.21.15.139:0/972273267 conn(0x7f13a006da40 msgr2=0x7f13a0142e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:07.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.900+0000 7f13a675f640 1 -- 172.21.15.139:0/972273267 shutdown_connections 2024-09-16T00:11:07.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:07.900+0000 7f13a675f640 1 -- 172.21.15.139:0/972273267 wait complete. 2024-09-16T00:11:07.917 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-16T00:11:08.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.021+0000 7fcd59caf640 1 Processor -- start 2024-09-16T00:11:08.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.021+0000 7fcd59caf640 1 -- start start 2024-09-16T00:11:08.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.022+0000 7fcd59caf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd54068060 0x7fcd5415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:08.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.022+0000 7fcd59caf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd540cc4b0 con 0x7fcd540ccc90 2024-09-16T00:11:08.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.022+0000 7fcd59caf640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd540cc790 con 0x7fcd54154c20 2024-09-16T00:11:08.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.022+0000 7fcd59caf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcd540c5830 con 0x7fcd54068060 2024-09-16T00:11:08.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.022+0000 7fcd53fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fcd540ccc90 0x7fcd54165180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46098/0 (socket says 172.21.15.139:46098) 2024-09-16T00:11:08.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.022+0000 7fcd53fff640 1 -- 172.21.15.139:0/1288013444 learned_addr learned my addr 172.21.15.139:0/1288013444 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:08.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.022+0000 7fcd52ffd640 1 --2- 172.21.15.139:0/1288013444 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd54068060 0x7fcd5415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:08.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.022+0000 7fcd527fc640 1 -- 172.21.15.139:0/1288013444 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 287439437 0 0) 0x7fcd540cc4b0 con 0x7fcd540ccc90 2024-09-16T00:11:08.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.022+0000 7fcd527fc640 1 -- 172.21.15.139:0/1288013444 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd3c003520 con 0x7fcd540ccc90 2024-09-16T00:11:08.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd527fc640 1 -- 172.21.15.139:0/1288013444 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3829528442 0 0) 0x7fcd540cc790 con 0x7fcd54154c20 2024-09-16T00:11:08.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd527fc640 1 -- 172.21.15.139:0/1288013444 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd540cc4b0 con 0x7fcd54154c20 2024-09-16T00:11:08.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd527fc640 1 -- 172.21.15.139:0/1288013444 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1216874961 0 0) 0x7fcd3c003520 con 0x7fcd540ccc90 2024-09-16T00:11:08.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd527fc640 1 -- 172.21.15.139:0/1288013444 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd54068060 msgr2=0x7fcd5415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:08.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd527fc640 1 --2- 172.21.15.139:0/1288013444 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd54068060 0x7fcd5415b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:08.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd527fc640 1 -- 172.21.15.139:0/1288013444 >> v1:172.21.15.164:6789/0 conn(0x7fcd54154c20 legacy=0x7fcd540c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:08.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd527fc640 1 -- 172.21.15.139:0/1288013444 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5415ddf0 con 0x7fcd540ccc90 2024-09-16T00:11:08.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd527fc640 1 -- 172.21.15.139:0/1288013444 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7fcd4c0019f0 con 0x7fcd540ccc90 2024-09-16T00:11:08.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd527fc640 1 -- 172.21.15.139:0/1288013444 >> v1:172.21.15.139:6789/0 conn(0x7fcd540ccc90 legacy=0x7fcd54165180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:08.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd527fc640 1 --2- 172.21.15.139:0/1288013444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd3c003b70 0x7fcd3c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:08.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd527fc640 1 --2- 172.21.15.139:0/1288013444 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd3c004800 0x7fcd3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:08.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd527fc640 1 --2- 172.21.15.139:0/1288013444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd3c0051c0 0x7fcd3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:08.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd527fc640 1 -- 172.21.15.139:0/1288013444 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcd540c5830 con 0x7fcd3c003b70 2024-09-16T00:11:08.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd527fc640 1 -- 172.21.15.139:0/1288013444 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c00bf60 con 0x7fcd3c0051c0 2024-09-16T00:11:08.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd527fc640 1 -- 172.21.15.139:0/1288013444 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcd3c00c160 con 0x7fcd3c004800 2024-09-16T00:11:08.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd52ffd640 1 --2- 172.21.15.139:0/1288013444 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd54068060 0x7fcd5415b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:11:08.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd537fe640 1 --2- 172.21.15.139:0/1288013444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd3c003b70 0x7fcd3c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:08.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd53fff640 1 --2- 172.21.15.139:0/1288013444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd3c0051c0 0x7fcd3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:08.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd52ffd640 1 --2- 172.21.15.139:0/1288013444 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd3c004800 0x7fcd3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:08.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd537fe640 1 -- 172.21.15.139:0/1288013444 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd3c004800 msgr2=0x7fcd3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:08.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd537fe640 1 --2- 172.21.15.139:0/1288013444 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd3c004800 0x7fcd3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:08.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd537fe640 1 -- 172.21.15.139:0/1288013444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd3c0051c0 msgr2=0x7fcd3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:08.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd537fe640 1 --2- 172.21.15.139:0/1288013444 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd3c0051c0 0x7fcd3c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:08.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.023+0000 7fcd537fe640 1 -- 172.21.15.139:0/1288013444 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3c003520 con 0x7fcd3c003b70 2024-09-16T00:11:08.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.024+0000 7fcd537fe640 1 --2- 172.21.15.139:0/1288013444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd3c003b70 0x7fcd3c003f70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcd440018c0 tx=0x7fcd3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e5c20e9cbaee8f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:08.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.024+0000 7fcd527fc640 1 -- 172.21.15.139:0/1288013444 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd440014e0 con 0x7fcd3c003b70 2024-09-16T00:11:08.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.024+0000 7fcd527fc640 1 -- 172.21.15.139:0/1288013444 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd440052f0 con 0x7fcd3c003b70 2024-09-16T00:11:08.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.024+0000 7fcd527fc640 1 -- 172.21.15.139:0/1288013444 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4402ac90 con 0x7fcd3c003b70 2024-09-16T00:11:08.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.024+0000 7fcd59caf640 1 -- 172.21.15.139:0/1288013444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd3c003b70 msgr2=0x7fcd3c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:08.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.025+0000 7fcd59caf640 1 --2- 172.21.15.139:0/1288013444 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd3c003b70 0x7fcd3c003f70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcd440018c0 tx=0x7fcd3c005830 comp rx=0 tx=0).stop 2024-09-16T00:11:08.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.025+0000 7fcd53fff640 1 -- 172.21.15.139:0/1288013444 reap_dead start 2024-09-16T00:11:08.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.025+0000 7fcd59caf640 1 -- 172.21.15.139:0/1288013444 shutdown_connections 2024-09-16T00:11:08.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.025+0000 7fcd59caf640 1 -- 172.21.15.139:0/1288013444 >> 172.21.15.139:0/1288013444 conn(0x7fcd5406ca40 msgr2=0x7fcd54067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:08.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.025+0000 7fcd59caf640 1 -- 172.21.15.139:0/1288013444 shutdown_connections 2024-09-16T00:11:08.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.025+0000 7fcd59caf640 1 -- 172.21.15.139:0/1288013444 wait complete. 2024-09-16T00:11:08.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.026+0000 7fcd59caf640 1 Processor -- start 2024-09-16T00:11:08.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.026+0000 7fcd59caf640 1 -- start start 2024-09-16T00:11:08.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd59caf640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd3c003b70 0x7fcd5415c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:08.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd59caf640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd54153860 0x7fcd54157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:08.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd59caf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd54157c50 0x7fcd541f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:08.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd59caf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5415e3b0 con 0x7fcd54157c50 2024-09-16T00:11:08.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd59caf640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fcd540cc260 con 0x7fcd54153860 2024-09-16T00:11:08.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd59caf640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5415e690 con 0x7fcd3c003b70 2024-09-16T00:11:08.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd53fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd54157c50 0x7fcd541f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:08.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd53fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd54157c50 0x7fcd541f4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60104/0 (socket says 172.21.15.139:60104) 2024-09-16T00:11:08.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd53fff640 1 -- 172.21.15.139:0/3842065505 learned_addr learned my addr 172.21.15.139:0/3842065505 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:08.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd537fe640 1 --2- 172.21.15.139:0/3842065505 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd3c003b70 0x7fcd5415c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:08.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd52ffd640 1 --2- 172.21.15.139:0/3842065505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd54153860 0x7fcd54157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:08.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd53fff640 1 -- 172.21.15.139:0/3842065505 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd3c003b70 msgr2=0x7fcd5415c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:08.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd53fff640 1 --2- 172.21.15.139:0/3842065505 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd3c003b70 0x7fcd5415c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:08.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd53fff640 1 -- 172.21.15.139:0/3842065505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd54153860 msgr2=0x7fcd54157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:08.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd53fff640 1 --2- 172.21.15.139:0/3842065505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd54153860 0x7fcd54157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:08.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd53fff640 1 -- 172.21.15.139:0/3842065505 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd54153560 con 0x7fcd54157c50 2024-09-16T00:11:08.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd52ffd640 1 --2- 172.21.15.139:0/3842065505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd54153860 0x7fcd54157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:08.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd53fff640 1 --2- 172.21.15.139:0/3842065505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd54157c50 0x7fcd541f4530 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcd540cde00 tx=0x7fcd4c007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:08.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd3bfff640 1 -- 172.21.15.139:0/3842065505 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c00aa50 con 0x7fcd54157c50 2024-09-16T00:11:08.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd3bfff640 1 -- 172.21.15.139:0/3842065505 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd4c019070 con 0x7fcd54157c50 2024-09-16T00:11:08.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd59caf640 1 -- 172.21.15.139:0/3842065505 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5415d950 con 0x7fcd54157c50 2024-09-16T00:11:08.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd3bfff640 1 -- 172.21.15.139:0/3842065505 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c00ae50 con 0x7fcd54157c50 2024-09-16T00:11:08.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.027+0000 7fcd59caf640 1 -- 172.21.15.139:0/3842065505 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd540b5f50 con 0x7fcd54157c50 2024-09-16T00:11:08.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.028+0000 7fcd59caf640 1 -- 172.21.15.139:0/3842065505 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd540b30d0 con 0x7fcd54157c50 2024-09-16T00:11:08.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.029+0000 7fcd3bfff640 1 -- 172.21.15.139:0/3842065505 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcd4c01e460 con 0x7fcd54157c50 2024-09-16T00:11:08.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.029+0000 7fcd3bfff640 1 --2- 172.21.15.139:0/3842065505 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcd4006c450 0x7fcd4006e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:08.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.029+0000 7fcd3bfff640 1 -- 172.21.15.139:0/3842065505 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5944+0+0 (secure 0 0 0) 0x7fcd4c08b9b0 con 0x7fcd54157c50 2024-09-16T00:11:08.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.029+0000 7fcd537fe640 1 --2- 172.21.15.139:0/3842065505 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcd4006c450 0x7fcd4006e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:08.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.030+0000 7fcd537fe640 1 --2- 172.21.15.139:0/3842065505 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcd4006c450 0x7fcd4006e910 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcd44032000 tx=0x7fcd4402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:08.037 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.033+0000 7fcd3bfff640 1 -- 172.21.15.139:0/3842065505 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcd540b30d0 con 0x7fcd54157c50 2024-09-16T00:11:08.215 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.213+0000 7fcd59caf640 1 -- 172.21.15.139:0/3842065505 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) -- 0x7fcd540686b0 con 0x7fcd54157c50 2024-09-16T00:11:08.905 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.903+0000 7fcd3bfff640 1 -- 172.21.15.139:0/3842065505 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v141) ==== 113+0+0 (secure 0 0 0) 0x7fcd5415d950 con 0x7fcd54157c50 2024-09-16T00:11:08.905 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T00:11:08.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.906+0000 7fcd59caf640 1 -- 172.21.15.139:0/3842065505 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcd4006c450 msgr2=0x7fcd4006e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:08.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.906+0000 7fcd59caf640 1 --2- 172.21.15.139:0/3842065505 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcd4006c450 0x7fcd4006e910 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcd44032000 tx=0x7fcd4402e000 comp rx=0 tx=0).stop 2024-09-16T00:11:08.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.906+0000 7fcd59caf640 1 -- 172.21.15.139:0/3842065505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd54157c50 msgr2=0x7fcd541f4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.906+0000 7fcd59caf640 1 --2- 172.21.15.139:0/3842065505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd54157c50 0x7fcd541f4530 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcd540cde00 tx=0x7fcd4c007bd0 comp rx=0 tx=0).stop 2024-09-16T00:11:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.906+0000 7fcd59caf640 1 -- 172.21.15.139:0/3842065505 shutdown_connections 2024-09-16T00:11:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.906+0000 7fcd59caf640 1 --2- 172.21.15.139:0/3842065505 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fcd4006c450 0x7fcd4006e910 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.906+0000 7fcd59caf640 1 --2- 172.21.15.139:0/3842065505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd54157c50 0x7fcd541f4530 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.906+0000 7fcd59caf640 1 --2- 172.21.15.139:0/3842065505 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fcd54153860 0x7fcd54157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.906+0000 7fcd59caf640 1 --2- 172.21.15.139:0/3842065505 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fcd3c003b70 0x7fcd5415c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.906+0000 7fcd59caf640 1 -- 172.21.15.139:0/3842065505 >> 172.21.15.139:0/3842065505 conn(0x7fcd5406ca40 msgr2=0x7fcd540c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.906+0000 7fcd59caf640 1 -- 172.21.15.139:0/3842065505 shutdown_connections 2024-09-16T00:11:08.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:08.906+0000 7fcd59caf640 1 -- 172.21.15.139:0/3842065505 wait complete. 2024-09-16T00:11:08.920 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-16T00:11:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.020+0000 7fb7f8a4c640 1 Processor -- start 2024-09-16T00:11:09.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.020+0000 7fb7f8a4c640 1 -- start start 2024-09-16T00:11:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.021+0000 7fb7f8a4c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7f40c61d0 0x7fb7f4158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.021+0000 7fb7f8a4c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb7f40c65b0 0x7fb7f4152380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.021+0000 7fb7f8a4c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f406bd70 con 0x7fb7f406d780 2024-09-16T00:11:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.021+0000 7fb7f8a4c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb7f406bb20 con 0x7fb7f40c65b0 2024-09-16T00:11:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.021+0000 7fb7f8a4c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f40c89e0 con 0x7fb7f40c61d0 2024-09-16T00:11:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.021+0000 7fb7e9d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb7f40c65b0 0x7fb7f4152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:09.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.021+0000 7fb7f2d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb7f406d780 0x7fb7f4073210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46110/0 (socket says 172.21.15.139:46110) 2024-09-16T00:11:09.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.021+0000 7fb7f2d9d640 1 -- 172.21.15.139:0/218269573 learned_addr learned my addr 172.21.15.139:0/218269573 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:09.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.021+0000 7fb7f259c640 1 --2- 172.21.15.139:0/218269573 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7f40c61d0 0x7fb7f4158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:09.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.021+0000 7fb7e9d9b640 1 -- 172.21.15.139:0/218269573 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7f40c61d0 msgr2=0x7fb7f4158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:09.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.021+0000 7fb7e9d9b640 1 --2- 172.21.15.139:0/218269573 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7f40c61d0 0x7fb7f4158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:09.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.021+0000 7fb7e9d9b640 1 -- 172.21.15.139:0/218269573 >> v1:172.21.15.139:6789/0 conn(0x7fb7f406d780 legacy=0x7fb7f4073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:09.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.021+0000 7fb7e9d9b640 1 -- 172.21.15.139:0/218269573 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f406c050 con 0x7fb7f40c65b0 2024-09-16T00:11:09.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.021+0000 7fb7f1d9b640 1 -- 172.21.15.139:0/218269573 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2552430727 0 0) 0x7fb7f406bd70 con 0x7fb7f406d780 2024-09-16T00:11:09.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.022+0000 7fb7e9d9b640 1 --2- 172.21.15.139:0/218269573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7f40c65b0 0x7fb7f4152380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc009790 tx=0x7fb7dc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72eb7eefba3d4e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:09.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.022+0000 7fb7f1d9b640 1 -- 172.21.15.139:0/218269573 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc037070 con 0x7fb7f40c65b0 2024-09-16T00:11:09.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.022+0000 7fb7f1d9b640 1 -- 172.21.15.139:0/218269573 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7dc032440 con 0x7fb7f40c65b0 2024-09-16T00:11:09.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.022+0000 7fb7f1d9b640 1 -- 172.21.15.139:0/218269573 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc032810 con 0x7fb7f40c65b0 2024-09-16T00:11:09.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.022+0000 7fb7f8a4c640 1 -- 172.21.15.139:0/218269573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7f40c65b0 msgr2=0x7fb7f4152380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:09.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.022+0000 7fb7f8a4c640 1 --2- 172.21.15.139:0/218269573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7f40c65b0 0x7fb7f4152380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc009790 tx=0x7fb7dc02a210 comp rx=0 tx=0).stop 2024-09-16T00:11:09.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.022+0000 7fb7f8a4c640 1 -- 172.21.15.139:0/218269573 shutdown_connections 2024-09-16T00:11:09.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.022+0000 7fb7f8a4c640 1 --2- 172.21.15.139:0/218269573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7f40c65b0 0x7fb7f4152380 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:09.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.022+0000 7fb7f8a4c640 1 --2- 172.21.15.139:0/218269573 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7f40c61d0 0x7fb7f4158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:09.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.022+0000 7fb7f8a4c640 1 -- 172.21.15.139:0/218269573 >> 172.21.15.139:0/218269573 conn(0x7fb7f40c9350 msgr2=0x7fb7f40c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:09.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.022+0000 7fb7f8a4c640 1 -- 172.21.15.139:0/218269573 shutdown_connections 2024-09-16T00:11:09.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.023+0000 7fb7f8a4c640 1 -- 172.21.15.139:0/218269573 wait complete. 2024-09-16T00:11:09.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.024+0000 7fb7f8a4c640 1 Processor -- start 2024-09-16T00:11:09.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.024+0000 7fb7f8a4c640 1 -- start start 2024-09-16T00:11:09.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7f8a4c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7f406d780 0x7fb7f40c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:09.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7f8a4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7f40c61d0 0x7fb7f40b9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:09.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7f8a4c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7f40c65b0 0x7fb7f40b9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:09.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7f8a4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f4072930 con 0x7fb7f40c61d0 2024-09-16T00:11:09.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7f8a4c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f40c8f10 con 0x7fb7f406d780 2024-09-16T00:11:09.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7f8a4c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f4073920 con 0x7fb7f40c65b0 2024-09-16T00:11:09.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7e9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7f40c61d0 0x7fb7f40b9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:09.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7e9d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7f40c61d0 0x7fb7f40b9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60108/0 (socket says 172.21.15.139:60108) 2024-09-16T00:11:09.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7e9d9b640 1 -- 172.21.15.139:0/2644818157 learned_addr learned my addr 172.21.15.139:0/2644818157 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:09.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7f2d9d640 1 --2- 172.21.15.139:0/2644818157 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7f40c65b0 0x7fb7f40b9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:09.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7f259c640 1 --2- 172.21.15.139:0/2644818157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7f406d780 0x7fb7f40c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:09.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7e9d9b640 1 -- 172.21.15.139:0/2644818157 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7f40c65b0 msgr2=0x7fb7f40b9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:09.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7e9d9b640 1 --2- 172.21.15.139:0/2644818157 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7f40c65b0 0x7fb7f40b9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:09.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7e9d9b640 1 -- 172.21.15.139:0/2644818157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7f406d780 msgr2=0x7fb7f40c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:09.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7e9d9b640 1 --2- 172.21.15.139:0/2644818157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7f406d780 0x7fb7f40c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:09.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7e9d9b640 1 -- 172.21.15.139:0/2644818157 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f41533d0 con 0x7fb7f40c61d0 2024-09-16T00:11:09.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7f259c640 1 --2- 172.21.15.139:0/2644818157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7f406d780 0x7fb7f40c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:09.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7e9d9b640 1 --2- 172.21.15.139:0/2644818157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7f40c61d0 0x7fb7f40b9250 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0098b0 tx=0x7fb7dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:09.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7eb7fe640 1 -- 172.21.15.139:0/2644818157 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc040020 con 0x7fb7f40c61d0 2024-09-16T00:11:09.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7eb7fe640 1 -- 172.21.15.139:0/2644818157 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7dc038040 con 0x7fb7f40c61d0 2024-09-16T00:11:09.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7f8a4c640 1 -- 172.21.15.139:0/2644818157 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f4154070 con 0x7fb7f40c61d0 2024-09-16T00:11:09.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7eb7fe640 1 -- 172.21.15.139:0/2644818157 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc037070 con 0x7fb7f40c61d0 2024-09-16T00:11:09.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.025+0000 7fb7f8a4c640 1 -- 172.21.15.139:0/2644818157 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f4154600 con 0x7fb7f40c61d0 2024-09-16T00:11:09.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.027+0000 7fb7f8a4c640 1 -- 172.21.15.139:0/2644818157 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f406bd70 con 0x7fb7f40c61d0 2024-09-16T00:11:09.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.027+0000 7fb7eb7fe640 1 -- 172.21.15.139:0/2644818157 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb7dc043020 con 0x7fb7f40c61d0 2024-09-16T00:11:09.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.027+0000 7fb7eb7fe640 1 --2- 172.21.15.139:0/2644818157 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb7d406c2b0 0x7fb7d406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:09.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.027+0000 7fb7eb7fe640 1 -- 172.21.15.139:0/2644818157 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5944+0+0 (secure 0 0 0) 0x7fb7dc0abaa0 con 0x7fb7f40c61d0 2024-09-16T00:11:09.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.027+0000 7fb7f259c640 1 --2- 172.21.15.139:0/2644818157 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb7d406c2b0 0x7fb7d406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:09.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.028+0000 7fb7f259c640 1 --2- 172.21.15.139:0/2644818157 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb7d406c2b0 0x7fb7d406e770 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4006fd0 tx=0x7fb7e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:09.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.032+0000 7fb7eb7fe640 1 -- 172.21.15.139:0/2644818157 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb7f406bd70 con 0x7fb7f40c61d0 2024-09-16T00:11:09.211 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.209+0000 7fb7f8a4c640 1 -- 172.21.15.139:0/2644818157 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} v 0) -- 0x7fb7f406f450 con 0x7fb7f40c61d0 2024-09-16T00:11:09.912 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.910+0000 7fb7eb7fe640 1 -- 172.21.15.139:0/2644818157 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]=0 set osd.5 primary-affinity to 1 (8655362) v142) ==== 133+0+0 (secure 0 0 0) 0x7fb7dc04a030 con 0x7fb7f40c61d0 2024-09-16T00:11:09.912 INFO:teuthology.orchestra.run.smithi139.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-16T00:11:09.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.913+0000 7fb7f8a4c640 1 -- 172.21.15.139:0/2644818157 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb7d406c2b0 msgr2=0x7fb7d406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:09.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.913+0000 7fb7f8a4c640 1 --2- 172.21.15.139:0/2644818157 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb7d406c2b0 0x7fb7d406e770 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4006fd0 tx=0x7fb7e4008040 comp rx=0 tx=0).stop 2024-09-16T00:11:09.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.913+0000 7fb7f8a4c640 1 -- 172.21.15.139:0/2644818157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7f40c61d0 msgr2=0x7fb7f40b9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:09.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.913+0000 7fb7f8a4c640 1 --2- 172.21.15.139:0/2644818157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7f40c61d0 0x7fb7f40b9250 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0098b0 tx=0x7fb7dc0046c0 comp rx=0 tx=0).stop 2024-09-16T00:11:09.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.913+0000 7fb7f8a4c640 1 -- 172.21.15.139:0/2644818157 shutdown_connections 2024-09-16T00:11:09.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.913+0000 7fb7f8a4c640 1 --2- 172.21.15.139:0/2644818157 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb7d406c2b0 0x7fb7d406e770 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:09.915 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.913+0000 7fb7f8a4c640 1 --2- 172.21.15.139:0/2644818157 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb7f40c65b0 0x7fb7f40b9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:09.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.913+0000 7fb7f8a4c640 1 --2- 172.21.15.139:0/2644818157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7f40c61d0 0x7fb7f40b9250 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:09.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.913+0000 7fb7f8a4c640 1 --2- 172.21.15.139:0/2644818157 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb7f406d780 0x7fb7f40c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:09.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.913+0000 7fb7f8a4c640 1 -- 172.21.15.139:0/2644818157 >> 172.21.15.139:0/2644818157 conn(0x7fb7f40c9350 msgr2=0x7fb7f415b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:09.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.913+0000 7fb7f8a4c640 1 -- 172.21.15.139:0/2644818157 shutdown_connections 2024-09-16T00:11:09.916 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:09.913+0000 7fb7f8a4c640 1 -- 172.21.15.139:0/2644818157 wait complete. 2024-09-16T00:11:09.928 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-16T00:11:10.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.031+0000 7f112560e640 1 Processor -- start 2024-09-16T00:11:10.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.031+0000 7f112560e640 1 -- start start 2024-09-16T00:11:10.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.032+0000 7f112560e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1120152740 0x7f112015bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:10.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.032+0000 7f112560e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f112015c150 0x7f1120162fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:10.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.032+0000 7f112560e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11200c39e0 con 0x7f11200c8230 2024-09-16T00:11:10.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.032+0000 7f112560e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f11200c5440 con 0x7f1120152740 2024-09-16T00:11:10.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.032+0000 7f112560e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1120152b20 con 0x7f112015c150 2024-09-16T00:11:10.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.032+0000 7f111effd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1120152740 0x7f112015bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:10.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.032+0000 7f111e7fc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f11200c8230 0x7f1120071520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46124/0 (socket says 172.21.15.139:46124) 2024-09-16T00:11:10.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.032+0000 7f111e7fc640 1 -- 172.21.15.139:0/986730915 learned_addr learned my addr 172.21.15.139:0/986730915 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:10.035 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.032+0000 7f111f7fe640 1 --2- 172.21.15.139:0/986730915 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f112015c150 0x7f1120162fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:10.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.032+0000 7f111effd640 1 -- 172.21.15.139:0/986730915 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f112015c150 msgr2=0x7f1120162fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:10.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.032+0000 7f111effd640 1 --2- 172.21.15.139:0/986730915 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f112015c150 0x7f1120162fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:10.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.032+0000 7f111effd640 1 -- 172.21.15.139:0/986730915 >> v1:172.21.15.139:6789/0 conn(0x7f11200c8230 legacy=0x7f1120071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:10.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.032+0000 7f111dffb640 1 -- 172.21.15.139:0/986730915 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2283677190 0 0) 0x7f11200c39e0 con 0x7f11200c8230 2024-09-16T00:11:10.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.032+0000 7f111effd640 1 -- 172.21.15.139:0/986730915 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11200c4ef0 con 0x7f1120152740 2024-09-16T00:11:10.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.033+0000 7f111effd640 1 --2- 172.21.15.139:0/986730915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1120152740 0x7f112015bc10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1118009d90 tx=0x7f111802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e813f4ef4c77d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:10.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.033+0000 7f111dffb640 1 -- 172.21.15.139:0/986730915 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118007740 con 0x7f1120152740 2024-09-16T00:11:10.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.033+0000 7f111dffb640 1 -- 172.21.15.139:0/986730915 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11180078e0 con 0x7f1120152740 2024-09-16T00:11:10.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.033+0000 7f111dffb640 1 -- 172.21.15.139:0/986730915 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118007cb0 con 0x7f1120152740 2024-09-16T00:11:10.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.034+0000 7f112560e640 1 -- 172.21.15.139:0/986730915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1120152740 msgr2=0x7f112015bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:10.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.034+0000 7f112560e640 1 --2- 172.21.15.139:0/986730915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1120152740 0x7f112015bc10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1118009d90 tx=0x7f111802a6f0 comp rx=0 tx=0).stop 2024-09-16T00:11:10.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.034+0000 7f112560e640 1 -- 172.21.15.139:0/986730915 shutdown_connections 2024-09-16T00:11:10.039 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.034+0000 7f112560e640 1 --2- 172.21.15.139:0/986730915 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f112015c150 0x7f1120162fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:10.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.034+0000 7f112560e640 1 --2- 172.21.15.139:0/986730915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1120152740 0x7f112015bc10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:10.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.034+0000 7f112560e640 1 -- 172.21.15.139:0/986730915 >> 172.21.15.139:0/986730915 conn(0x7f112006bb20 msgr2=0x7f112006f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:10.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.034+0000 7f112560e640 1 -- 172.21.15.139:0/986730915 shutdown_connections 2024-09-16T00:11:10.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.034+0000 7f112560e640 1 -- 172.21.15.139:0/986730915 wait complete. 2024-09-16T00:11:10.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.035+0000 7f112560e640 1 Processor -- start 2024-09-16T00:11:10.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.035+0000 7f112560e640 1 -- start start 2024-09-16T00:11:10.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f112560e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f11200c8230 0x7f11201f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:10.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f112560e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1120152740 0x7f1120157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:10.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f112560e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f112015c150 0x7f112006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:10.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f112560e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f11201594e0 con 0x7f112015c150 2024-09-16T00:11:10.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f112560e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1120159240 con 0x7f1120152740 2024-09-16T00:11:10.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f112560e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f11200c31e0 con 0x7f11200c8230 2024-09-16T00:11:10.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f111f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f112015c150 0x7f112006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:10.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f111f7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f112015c150 0x7f112006d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60138/0 (socket says 172.21.15.139:60138) 2024-09-16T00:11:10.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f111f7fe640 1 -- 172.21.15.139:0/3086286840 learned_addr learned my addr 172.21.15.139:0/3086286840 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:10.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f111e7fc640 1 --2- 172.21.15.139:0/3086286840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1120152740 0x7f1120157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:10.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f111effd640 1 --2- 172.21.15.139:0/3086286840 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f11200c8230 0x7f11201f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:10.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f111f7fe640 1 -- 172.21.15.139:0/3086286840 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f11200c8230 msgr2=0x7f11201f09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:10.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f111f7fe640 1 --2- 172.21.15.139:0/3086286840 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f11200c8230 0x7f11201f09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:10.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f111f7fe640 1 -- 172.21.15.139:0/3086286840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1120152740 msgr2=0x7f1120157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:10.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f111f7fe640 1 --2- 172.21.15.139:0/3086286840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1120152740 0x7f1120157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:10.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f111f7fe640 1 -- 172.21.15.139:0/3086286840 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112006e040 con 0x7f112015c150 2024-09-16T00:11:10.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f111effd640 1 --2- 172.21.15.139:0/3086286840 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f11200c8230 0x7f11201f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:10.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f111f7fe640 1 --2- 172.21.15.139:0/3086286840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f112015c150 0x7f112006d1f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f111000d5d0 tx=0x7f111000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:10.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.036+0000 7f11177fe640 1 -- 172.21.15.139:0/3086286840 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11100125f0 con 0x7f112015c150 2024-09-16T00:11:10.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.037+0000 7f11177fe640 1 -- 172.21.15.139:0/3086286840 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1110019070 con 0x7f112015c150 2024-09-16T00:11:10.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.037+0000 7f11177fe640 1 -- 172.21.15.139:0/3086286840 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11100129f0 con 0x7f112015c150 2024-09-16T00:11:10.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.037+0000 7f112560e640 1 -- 172.21.15.139:0/3086286840 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112006e300 con 0x7f112015c150 2024-09-16T00:11:10.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.037+0000 7f112560e640 1 -- 172.21.15.139:0/3086286840 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f112006e8c0 con 0x7f112015c150 2024-09-16T00:11:10.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.038+0000 7f112560e640 1 -- 172.21.15.139:0/3086286840 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11200cb510 con 0x7f112015c150 2024-09-16T00:11:10.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.038+0000 7f11177fe640 1 -- 172.21.15.139:0/3086286840 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1110012b90 con 0x7f112015c150 2024-09-16T00:11:10.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.038+0000 7f11177fe640 1 --2- 172.21.15.139:0/3086286840 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f110006c2b0 0x7f110006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:10.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.039+0000 7f11177fe640 1 -- 172.21.15.139:0/3086286840 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5944+0+0 (secure 0 0 0) 0x7f111008b840 con 0x7f112015c150 2024-09-16T00:11:10.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.039+0000 7f111effd640 1 --2- 172.21.15.139:0/3086286840 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f110006c2b0 0x7f110006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:10.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.039+0000 7f111effd640 1 --2- 172.21.15.139:0/3086286840 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f110006c2b0 0x7f110006e770 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1118034000 tx=0x7f111802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:10.045 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.044+0000 7f11177fe640 1 -- 172.21.15.139:0/3086286840 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1110059ed0 con 0x7f112015c150 2024-09-16T00:11:10.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.220+0000 7f112560e640 1 -- 172.21.15.139:0/3086286840 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) -- 0x7f1120151580 con 0x7f112015c150 2024-09-16T00:11:10.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.918+0000 7f11177fe640 1 -- 172.21.15.139:0/3086286840 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v143) ==== 113+0+0 (secure 0 0 0) 0x7f11100597d0 con 0x7f112015c150 2024-09-16T00:11:10.920 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T00:11:10.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.921+0000 7f112560e640 1 -- 172.21.15.139:0/3086286840 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f110006c2b0 msgr2=0x7f110006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:10.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.921+0000 7f112560e640 1 --2- 172.21.15.139:0/3086286840 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f110006c2b0 0x7f110006e770 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1118034000 tx=0x7f111802f000 comp rx=0 tx=0).stop 2024-09-16T00:11:10.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.921+0000 7f112560e640 1 -- 172.21.15.139:0/3086286840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f112015c150 msgr2=0x7f112006d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:10.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.921+0000 7f112560e640 1 --2- 172.21.15.139:0/3086286840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f112015c150 0x7f112006d1f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f111000d5d0 tx=0x7f111000da90 comp rx=0 tx=0).stop 2024-09-16T00:11:10.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.921+0000 7f112560e640 1 -- 172.21.15.139:0/3086286840 shutdown_connections 2024-09-16T00:11:10.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.921+0000 7f112560e640 1 --2- 172.21.15.139:0/3086286840 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f110006c2b0 0x7f110006e770 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:10.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.921+0000 7f112560e640 1 --2- 172.21.15.139:0/3086286840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f112015c150 0x7f112006d1f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:10.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.921+0000 7f112560e640 1 --2- 172.21.15.139:0/3086286840 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1120152740 0x7f1120157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:10.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.921+0000 7f112560e640 1 --2- 172.21.15.139:0/3086286840 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f11200c8230 0x7f11201f09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:10.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.921+0000 7f112560e640 1 -- 172.21.15.139:0/3086286840 >> 172.21.15.139:0/3086286840 conn(0x7f112006bb20 msgr2=0x7f11200c9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:10.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.921+0000 7f112560e640 1 -- 172.21.15.139:0/3086286840 shutdown_connections 2024-09-16T00:11:10.924 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:10.921+0000 7f112560e640 1 -- 172.21.15.139:0/3086286840 wait complete. 2024-09-16T00:11:10.936 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-16T00:11:11.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.038+0000 7f62e0df1640 1 Processor -- start 2024-09-16T00:11:11.040 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.038+0000 7f62e0df1640 1 -- start start 2024-09-16T00:11:11.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.039+0000 7f62e0df1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f62dc1524f0 0x7f62dc158630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:11.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.039+0000 7f62e0df1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62dc1528d0 0x7f62dc0c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:11.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.039+0000 7f62e0df1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62dc0702c0 con 0x7f62dc072f00 2024-09-16T00:11:11.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.039+0000 7f62e0df1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f62dc0c9ae0 con 0x7f62dc1524f0 2024-09-16T00:11:11.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.039+0000 7f62e0df1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f62dc0c6c00 con 0x7f62dc1528d0 2024-09-16T00:11:11.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.039+0000 7f62da59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f62dc1524f0 0x7f62dc158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:11.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.039+0000 7f62dad9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f62dc072f00 0x7f62dc0c5910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46130/0 (socket says 172.21.15.139:46130) 2024-09-16T00:11:11.041 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.039+0000 7f62dad9d640 1 -- 172.21.15.139:0/4030738697 learned_addr learned my addr 172.21.15.139:0/4030738697 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:11.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.039+0000 7f62da59c640 1 -- 172.21.15.139:0/4030738697 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62dc1528d0 msgr2=0x7f62dc0c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:11.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.039+0000 7f62da59c640 1 --2- 172.21.15.139:0/4030738697 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62dc1528d0 0x7f62dc0c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:11.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.039+0000 7f62da59c640 1 -- 172.21.15.139:0/4030738697 >> v1:172.21.15.139:6789/0 conn(0x7f62dc072f00 legacy=0x7f62dc0c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:11.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.039+0000 7f62da59c640 1 -- 172.21.15.139:0/4030738697 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62dc0c8320 con 0x7f62dc1524f0 2024-09-16T00:11:11.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.040+0000 7f62da59c640 1 --2- 172.21.15.139:0/4030738697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc1524f0 0x7f62dc158630 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0097a0 tx=0x7f62cc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3342985c6538181e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:11.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.040+0000 7f62d959a640 1 -- 172.21.15.139:0/4030738697 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc037070 con 0x7f62dc1524f0 2024-09-16T00:11:11.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.040+0000 7f62d959a640 1 -- 172.21.15.139:0/4030738697 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62cc032440 con 0x7f62dc1524f0 2024-09-16T00:11:11.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.040+0000 7f62d959a640 1 -- 172.21.15.139:0/4030738697 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc032810 con 0x7f62dc1524f0 2024-09-16T00:11:11.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.041+0000 7f62e0df1640 1 -- 172.21.15.139:0/4030738697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc1524f0 msgr2=0x7f62dc158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:11.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.041+0000 7f62e0df1640 1 --2- 172.21.15.139:0/4030738697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc1524f0 0x7f62dc158630 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0097a0 tx=0x7f62cc02a180 comp rx=0 tx=0).stop 2024-09-16T00:11:11.043 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.041+0000 7f62e0df1640 1 -- 172.21.15.139:0/4030738697 shutdown_connections 2024-09-16T00:11:11.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.041+0000 7f62e0df1640 1 --2- 172.21.15.139:0/4030738697 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62dc1528d0 0x7f62dc0c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:11.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.041+0000 7f62e0df1640 1 --2- 172.21.15.139:0/4030738697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc1524f0 0x7f62dc158630 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:11.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.041+0000 7f62e0df1640 1 -- 172.21.15.139:0/4030738697 >> 172.21.15.139:0/4030738697 conn(0x7f62dc06da40 msgr2=0x7f62dc06f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:11.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.041+0000 7f62e0df1640 1 -- 172.21.15.139:0/4030738697 shutdown_connections 2024-09-16T00:11:11.044 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.041+0000 7f62e0df1640 1 -- 172.21.15.139:0/4030738697 wait complete. 2024-09-16T00:11:11.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.042+0000 7f62e0df1640 1 Processor -- start 2024-09-16T00:11:11.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.042+0000 7f62e0df1640 1 -- start start 2024-09-16T00:11:11.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62e0df1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc072f00 0x7f62dc145e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62e0df1640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62dc1524f0 0x7f62dc0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62e0df1640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62dc1528d0 0x7f62dc153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62e0df1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f62dc06ffe0 con 0x7f62dc072f00 2024-09-16T00:11:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62e0df1640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f62dc06fd00 con 0x7f62dc1524f0 2024-09-16T00:11:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62e0df1640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f62dc0c8da0 con 0x7f62dc1528d0 2024-09-16T00:11:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62da59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc072f00 0x7f62dc145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62da59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc072f00 0x7f62dc145e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60142/0 (socket says 172.21.15.139:60142) 2024-09-16T00:11:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62da59c640 1 -- 172.21.15.139:0/3786893166 learned_addr learned my addr 172.21.15.139:0/3786893166 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62dad9d640 1 --2- 172.21.15.139:0/3786893166 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62dc1528d0 0x7f62dc153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62d9d9b640 1 --2- 172.21.15.139:0/3786893166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62dc1524f0 0x7f62dc0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62da59c640 1 -- 172.21.15.139:0/3786893166 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62dc1528d0 msgr2=0x7f62dc153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:11.047 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62da59c640 1 --2- 172.21.15.139:0/3786893166 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62dc1528d0 0x7f62dc153950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62da59c640 1 -- 172.21.15.139:0/3786893166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62dc1524f0 msgr2=0x7f62dc0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62da59c640 1 --2- 172.21.15.139:0/3786893166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62dc1524f0 0x7f62dc0cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62da59c640 1 -- 172.21.15.139:0/3786893166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62dc155b50 con 0x7f62dc072f00 2024-09-16T00:11:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62d9d9b640 1 --2- 172.21.15.139:0/3786893166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62dc1524f0 0x7f62dc0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62dad9d640 1 --2- 172.21.15.139:0/3786893166 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62dc1528d0 0x7f62dc153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62da59c640 1 --2- 172.21.15.139:0/3786893166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc072f00 0x7f62dc145e90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0098c0 tx=0x7f62cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.043+0000 7f62caffd640 1 -- 172.21.15.139:0/3786893166 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc037040 con 0x7f62dc072f00 2024-09-16T00:11:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.044+0000 7f62caffd640 1 -- 172.21.15.139:0/3786893166 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62cc042070 con 0x7f62dc072f00 2024-09-16T00:11:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.044+0000 7f62e0df1640 1 -- 172.21.15.139:0/3786893166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62dc155e10 con 0x7f62dc072f00 2024-09-16T00:11:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.044+0000 7f62caffd640 1 -- 172.21.15.139:0/3786893166 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc030a70 con 0x7f62dc072f00 2024-09-16T00:11:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.044+0000 7f62e0df1640 1 -- 172.21.15.139:0/3786893166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62dc146920 con 0x7f62dc072f00 2024-09-16T00:11:11.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.044+0000 7f62e0df1640 1 -- 172.21.15.139:0/3786893166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62dc0718b0 con 0x7f62dc072f00 2024-09-16T00:11:11.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.045+0000 7f62caffd640 1 -- 172.21.15.139:0/3786893166 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f62cc030c10 con 0x7f62dc072f00 2024-09-16T00:11:11.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.045+0000 7f62caffd640 1 --2- 172.21.15.139:0/3786893166 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f62bc06c2b0 0x7f62bc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:11.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.045+0000 7f62caffd640 1 -- 172.21.15.139:0/3786893166 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5944+0+0 (secure 0 0 0) 0x7f62cc0abfa0 con 0x7f62dc072f00 2024-09-16T00:11:11.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.045+0000 7f62d9d9b640 1 --2- 172.21.15.139:0/3786893166 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f62bc06c2b0 0x7f62bc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:11.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.046+0000 7f62d9d9b640 1 --2- 172.21.15.139:0/3786893166 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f62bc06c2b0 0x7f62bc06e770 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f62dc156a30 tx=0x7f62d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:11.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.050+0000 7f62caffd640 1 -- 172.21.15.139:0/3786893166 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f62cc045360 con 0x7f62dc072f00 2024-09-16T00:11:11.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.224+0000 7f62e0df1640 1 -- 172.21.15.139:0/3786893166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} v 0) -- 0x7f62dc156780 con 0x7f62dc072f00 2024-09-16T00:11:11.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.930+0000 7f62caffd640 1 -- 172.21.15.139:0/3786893166 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]=0 set osd.6 primary-affinity to 1 (8655362) v144) ==== 133+0+0 (secure 0 0 0) 0x7f62cc07a620 con 0x7f62dc072f00 2024-09-16T00:11:11.932 INFO:teuthology.orchestra.run.smithi139.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-16T00:11:11.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.933+0000 7f62e0df1640 1 -- 172.21.15.139:0/3786893166 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f62bc06c2b0 msgr2=0x7f62bc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:11.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.933+0000 7f62e0df1640 1 --2- 172.21.15.139:0/3786893166 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f62bc06c2b0 0x7f62bc06e770 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f62dc156a30 tx=0x7f62d0008040 comp rx=0 tx=0).stop 2024-09-16T00:11:11.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.933+0000 7f62e0df1640 1 -- 172.21.15.139:0/3786893166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc072f00 msgr2=0x7f62dc145e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:11.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.933+0000 7f62e0df1640 1 --2- 172.21.15.139:0/3786893166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc072f00 0x7f62dc145e90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0098c0 tx=0x7f62cc0046c0 comp rx=0 tx=0).stop 2024-09-16T00:11:11.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.933+0000 7f62e0df1640 1 -- 172.21.15.139:0/3786893166 shutdown_connections 2024-09-16T00:11:11.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.933+0000 7f62e0df1640 1 --2- 172.21.15.139:0/3786893166 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f62bc06c2b0 0x7f62bc06e770 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:11.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.933+0000 7f62e0df1640 1 --2- 172.21.15.139:0/3786893166 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f62dc1528d0 0x7f62dc153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:11.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.933+0000 7f62e0df1640 1 --2- 172.21.15.139:0/3786893166 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f62dc1524f0 0x7f62dc0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:11.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.933+0000 7f62e0df1640 1 --2- 172.21.15.139:0/3786893166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62dc072f00 0x7f62dc145e90 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:11.935 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.933+0000 7f62e0df1640 1 -- 172.21.15.139:0/3786893166 >> 172.21.15.139:0/3786893166 conn(0x7f62dc06da40 msgr2=0x7f62dc157690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:11.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.933+0000 7f62e0df1640 1 -- 172.21.15.139:0/3786893166 shutdown_connections 2024-09-16T00:11:11.936 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:11.933+0000 7f62e0df1640 1 -- 172.21.15.139:0/3786893166 wait complete. 2024-09-16T00:11:11.949 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-16T00:11:12.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.051+0000 7ff116149640 1 Processor -- start 2024-09-16T00:11:12.053 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.051+0000 7ff116149640 1 -- start start 2024-09-16T00:11:12.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.051+0000 7ff116149640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff110154c20 0x7ff11015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:12.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.051+0000 7ff116149640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff1100ccc90 0x7ff1100cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:12.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.051+0000 7ff116149640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1100cc4b0 con 0x7ff110068060 2024-09-16T00:11:12.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.051+0000 7ff116149640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff1100c5830 con 0x7ff110154c20 2024-09-16T00:11:12.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.051+0000 7ff116149640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7ff11015b7c0 con 0x7ff1100ccc90 2024-09-16T00:11:12.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.052+0000 7ff10f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff110154c20 0x7ff11015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:12.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.052+0000 7ff10f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff110154c20 0x7ff11015b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60152/0 (socket says 172.21.15.139:60152) 2024-09-16T00:11:12.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.052+0000 7ff10f7fe640 1 -- 172.21.15.139:0/3329767199 learned_addr learned my addr 172.21.15.139:0/3329767199 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:12.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.052+0000 7ff10f7fe640 1 -- 172.21.15.139:0/3329767199 >> v2:172.21.15.164:3300/0 conn(0x7ff1100ccc90 msgr2=0x7ff1100cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:12.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.052+0000 7ff10f7fe640 1 --2- 172.21.15.139:0/3329767199 >> v2:172.21.15.164:3300/0 conn(0x7ff1100ccc90 0x7ff1100cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.054 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.052+0000 7ff10f7fe640 1 -- 172.21.15.139:0/3329767199 >> v1:172.21.15.164:6789/0 conn(0x7ff110068060 legacy=0x7ff1100c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:12.055 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.052+0000 7ff10f7fe640 1 -- 172.21.15.139:0/3329767199 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1100cc790 con 0x7ff110154c20 2024-09-16T00:11:12.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.053+0000 7ff10f7fe640 1 --2- 172.21.15.139:0/3329767199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff110154c20 0x7ff11015b280 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff1100c7300 tx=0x7ff10002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70bd2f51e5e85f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:12.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.053+0000 7ff10e7fc640 1 -- 172.21.15.139:0/3329767199 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100037070 con 0x7ff110154c20 2024-09-16T00:11:12.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.053+0000 7ff10e7fc640 1 -- 172.21.15.139:0/3329767199 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff100032440 con 0x7ff110154c20 2024-09-16T00:11:12.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.053+0000 7ff10e7fc640 1 -- 172.21.15.139:0/3329767199 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff100032810 con 0x7ff110154c20 2024-09-16T00:11:12.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.053+0000 7ff116149640 1 -- 172.21.15.139:0/3329767199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff110154c20 msgr2=0x7ff11015b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:12.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.053+0000 7ff116149640 1 --2- 172.21.15.139:0/3329767199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff110154c20 0x7ff11015b280 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff1100c7300 tx=0x7ff10002a230 comp rx=0 tx=0).stop 2024-09-16T00:11:12.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.053+0000 7ff116149640 1 -- 172.21.15.139:0/3329767199 shutdown_connections 2024-09-16T00:11:12.056 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.053+0000 7ff116149640 1 --2- 172.21.15.139:0/3329767199 >> v2:172.21.15.164:3300/0 conn(0x7ff1100ccc90 0x7ff1100cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.053+0000 7ff116149640 1 --2- 172.21.15.139:0/3329767199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff110154c20 0x7ff11015b280 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.053+0000 7ff116149640 1 -- 172.21.15.139:0/3329767199 >> 172.21.15.139:0/3329767199 conn(0x7ff11006ca40 msgr2=0x7ff110067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:12.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.053+0000 7ff116149640 1 -- 172.21.15.139:0/3329767199 shutdown_connections 2024-09-16T00:11:12.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.053+0000 7ff116149640 1 -- 172.21.15.139:0/3329767199 wait complete. 2024-09-16T00:11:12.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.054+0000 7ff116149640 1 Processor -- start 2024-09-16T00:11:12.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.055+0000 7ff116149640 1 -- start start 2024-09-16T00:11:12.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.055+0000 7ff116149640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff110068060 0x7ff110149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:12.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.055+0000 7ff116149640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1100ccc90 0x7ff110153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:12.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.055+0000 7ff116149640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff110154c20 0x7ff1101f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:12.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.055+0000 7ff116149640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff11015e0d0 con 0x7ff1100ccc90 2024-09-16T00:11:12.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.055+0000 7ff116149640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff1100cc260 con 0x7ff110154c20 2024-09-16T00:11:12.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.055+0000 7ff116149640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff11015e3b0 con 0x7ff110068060 2024-09-16T00:11:12.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.055+0000 7ff10effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1100ccc90 0x7ff110153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:12.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.055+0000 7ff10effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1100ccc90 0x7ff110153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60168/0 (socket says 172.21.15.139:60168) 2024-09-16T00:11:12.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.055+0000 7ff10effd640 1 -- 172.21.15.139:0/2603667231 learned_addr learned my addr 172.21.15.139:0/2603667231 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:12.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.056+0000 7ff10ffff640 1 --2- 172.21.15.139:0/2603667231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff110154c20 0x7ff1101f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:12.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.056+0000 7ff10f7fe640 1 --2- 172.21.15.139:0/2603667231 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff110068060 0x7ff110149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:12.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.056+0000 7ff10effd640 1 -- 172.21.15.139:0/2603667231 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff110068060 msgr2=0x7ff110149d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:12.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.056+0000 7ff10effd640 1 --2- 172.21.15.139:0/2603667231 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff110068060 0x7ff110149d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.056+0000 7ff10effd640 1 -- 172.21.15.139:0/2603667231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff110154c20 msgr2=0x7ff1101f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:12.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.056+0000 7ff10effd640 1 --2- 172.21.15.139:0/2603667231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff110154c20 0x7ff1101f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.056+0000 7ff10effd640 1 -- 172.21.15.139:0/2603667231 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff110153ea0 con 0x7ff1100ccc90 2024-09-16T00:11:12.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.056+0000 7ff10ffff640 1 --2- 172.21.15.139:0/2603667231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff110154c20 0x7ff1101f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:12.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.056+0000 7ff10f7fe640 1 --2- 172.21.15.139:0/2603667231 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff110068060 0x7ff110149d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:12.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.056+0000 7ff10effd640 1 --2- 172.21.15.139:0/2603667231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1100ccc90 0x7ff110153180 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800c400 tx=0x7ff0f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:12.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.056+0000 7ff107fff640 1 -- 172.21.15.139:0/2603667231 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8017070 con 0x7ff1100ccc90 2024-09-16T00:11:12.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.056+0000 7ff107fff640 1 -- 172.21.15.139:0/2603667231 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f8003640 con 0x7ff1100ccc90 2024-09-16T00:11:12.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.056+0000 7ff116149640 1 -- 172.21.15.139:0/2603667231 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1101525c0 con 0x7ff1100ccc90 2024-09-16T00:11:12.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.056+0000 7ff107fff640 1 -- 172.21.15.139:0/2603667231 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8003a30 con 0x7ff1100ccc90 2024-09-16T00:11:12.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.056+0000 7ff116149640 1 -- 172.21.15.139:0/2603667231 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1101f1d90 con 0x7ff1100ccc90 2024-09-16T00:11:12.061 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.057+0000 7ff116149640 1 -- 172.21.15.139:0/2603667231 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff11015e690 con 0x7ff1100ccc90 2024-09-16T00:11:12.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.058+0000 7ff107fff640 1 -- 172.21.15.139:0/2603667231 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff0f801a070 con 0x7ff1100ccc90 2024-09-16T00:11:12.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.058+0000 7ff107fff640 1 --2- 172.21.15.139:0/2603667231 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff0f006c2e0 0x7ff0f006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:12.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.058+0000 7ff107fff640 1 -- 172.21.15.139:0/2603667231 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5944+0+0 (secure 0 0 0) 0x7ff0f808a910 con 0x7ff1100ccc90 2024-09-16T00:11:12.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.058+0000 7ff10f7fe640 1 --2- 172.21.15.139:0/2603667231 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff0f006c2e0 0x7ff0f006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:12.062 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.058+0000 7ff10f7fe640 1 --2- 172.21.15.139:0/2603667231 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff0f006c2e0 0x7ff0f006e7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff100034000 tx=0x7ff10002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:12.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.062+0000 7ff107fff640 1 -- 172.21.15.139:0/2603667231 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff0f8058f90 con 0x7ff1100ccc90 2024-09-16T00:11:12.243 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.241+0000 7ff116149640 1 -- 172.21.15.139:0/2603667231 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) -- 0x7ff11014ea70 con 0x7ff1100ccc90 2024-09-16T00:11:12.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.936+0000 7ff107fff640 1 -- 172.21.15.139:0/2603667231 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v145) ==== 113+0+0 (secure 0 0 0) 0x7ff0f8058890 con 0x7ff1100ccc90 2024-09-16T00:11:12.939 INFO:teuthology.orchestra.run.smithi139.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T00:11:12.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.939+0000 7ff116149640 1 -- 172.21.15.139:0/2603667231 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff0f006c2e0 msgr2=0x7ff0f006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:12.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.939+0000 7ff116149640 1 --2- 172.21.15.139:0/2603667231 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff0f006c2e0 0x7ff0f006e7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff100034000 tx=0x7ff10002e000 comp rx=0 tx=0).stop 2024-09-16T00:11:12.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.939+0000 7ff116149640 1 -- 172.21.15.139:0/2603667231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1100ccc90 msgr2=0x7ff110153180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:12.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.939+0000 7ff116149640 1 --2- 172.21.15.139:0/2603667231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1100ccc90 0x7ff110153180 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff0f800c400 tx=0x7ff0f800c8c0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.939+0000 7ff116149640 1 -- 172.21.15.139:0/2603667231 shutdown_connections 2024-09-16T00:11:12.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.939+0000 7ff116149640 1 --2- 172.21.15.139:0/2603667231 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff0f006c2e0 0x7ff0f006e7a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.941 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.939+0000 7ff116149640 1 --2- 172.21.15.139:0/2603667231 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff110154c20 0x7ff1101f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.939+0000 7ff116149640 1 --2- 172.21.15.139:0/2603667231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1100ccc90 0x7ff110153180 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.939+0000 7ff116149640 1 --2- 172.21.15.139:0/2603667231 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff110068060 0x7ff110149d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:12.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.939+0000 7ff116149640 1 -- 172.21.15.139:0/2603667231 >> 172.21.15.139:0/2603667231 conn(0x7ff11006ca40 msgr2=0x7ff1100c69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:12.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.939+0000 7ff116149640 1 -- 172.21.15.139:0/2603667231 shutdown_connections 2024-09-16T00:11:12.942 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:12.939+0000 7ff116149640 1 -- 172.21.15.139:0/2603667231 wait complete. 2024-09-16T00:11:12.954 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-16T00:11:13.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.055+0000 7faceff4b640 1 Processor -- start 2024-09-16T00:11:13.057 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.055+0000 7faceff4b640 1 -- start start 2024-09-16T00:11:13.058 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.056+0000 7faceff4b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7face80c6a20 0x7face806f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:13.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.056+0000 7faceff4b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7face80c6e00 0x7face815bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:13.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.056+0000 7faceff4b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7face806f210 con 0x7face806ea70 2024-09-16T00:11:13.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.056+0000 7faceff4b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7face806f520 con 0x7face80c6e00 2024-09-16T00:11:13.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.056+0000 7faceff4b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7face806ef20 con 0x7face80c6a20 2024-09-16T00:11:13.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.056+0000 7faced4e6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7face80c6e00 0x7face815bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:13.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.056+0000 7facee4e8640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7face806ea70 0x7face80c9530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46142/0 (socket says 172.21.15.139:46142) 2024-09-16T00:11:13.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.056+0000 7facee4e8640 1 -- 172.21.15.139:0/3078252432 learned_addr learned my addr 172.21.15.139:0/3078252432 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:13.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.057+0000 7facedce7640 1 --2- 172.21.15.139:0/3078252432 >> v2:172.21.15.164:3300/0 conn(0x7face80c6a20 0x7face806f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:13.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.057+0000 7faced4e6640 1 -- 172.21.15.139:0/3078252432 >> v2:172.21.15.164:3300/0 conn(0x7face80c6a20 msgr2=0x7face806f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:13.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.057+0000 7faced4e6640 1 --2- 172.21.15.139:0/3078252432 >> v2:172.21.15.164:3300/0 conn(0x7face80c6a20 0x7face806f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:13.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.057+0000 7faced4e6640 1 -- 172.21.15.139:0/3078252432 >> v1:172.21.15.139:6789/0 conn(0x7face806ea70 legacy=0x7face80c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:13.059 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.057+0000 7faced4e6640 1 -- 172.21.15.139:0/3078252432 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face80ca080 con 0x7face80c6e00 2024-09-16T00:11:13.060 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.057+0000 7facedce7640 1 --2- 172.21.15.139:0/3078252432 >> v2:172.21.15.164:3300/0 conn(0x7face80c6a20 0x7face806f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:13.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.057+0000 7faced4e6640 1 --2- 172.21.15.139:0/3078252432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7face80c6e00 0x7face815bba0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7face8071490 tx=0x7facd802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0b659c1a4d95e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:13.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.057+0000 7facecce5640 1 -- 172.21.15.139:0/3078252432 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8037070 con 0x7face80c6e00 2024-09-16T00:11:13.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.057+0000 7facecce5640 1 -- 172.21.15.139:0/3078252432 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd8032440 con 0x7face80c6e00 2024-09-16T00:11:13.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.057+0000 7facecce5640 1 -- 172.21.15.139:0/3078252432 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8032810 con 0x7face80c6e00 2024-09-16T00:11:13.063 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.058+0000 7faceff4b640 1 -- 172.21.15.139:0/3078252432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7face80c6e00 msgr2=0x7face815bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:13.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.058+0000 7faceff4b640 1 --2- 172.21.15.139:0/3078252432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7face80c6e00 0x7face815bba0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7face8071490 tx=0x7facd802a230 comp rx=0 tx=0).stop 2024-09-16T00:11:13.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.058+0000 7faceff4b640 1 -- 172.21.15.139:0/3078252432 shutdown_connections 2024-09-16T00:11:13.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.058+0000 7faceff4b640 1 --2- 172.21.15.139:0/3078252432 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7face80c6e00 0x7face815bba0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:13.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.058+0000 7faceff4b640 1 --2- 172.21.15.139:0/3078252432 >> v2:172.21.15.164:3300/0 conn(0x7face80c6a20 0x7face806f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:13.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.058+0000 7faceff4b640 1 -- 172.21.15.139:0/3078252432 >> 172.21.15.139:0/3078252432 conn(0x7face806cc30 msgr2=0x7face806d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:13.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.058+0000 7faceff4b640 1 -- 172.21.15.139:0/3078252432 shutdown_connections 2024-09-16T00:11:13.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.058+0000 7faceff4b640 1 -- 172.21.15.139:0/3078252432 wait complete. 2024-09-16T00:11:13.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.059+0000 7faceff4b640 1 Processor -- start 2024-09-16T00:11:13.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.059+0000 7faceff4b640 1 -- start start 2024-09-16T00:11:13.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7faceff4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7face806ea70 0x7face8154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:13.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7faceff4b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7face80c6a20 0x7face80cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:13.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7faceff4b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7face80c6e00 0x7face814b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:13.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7faceff4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7face80ca600 con 0x7face806ea70 2024-09-16T00:11:13.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7faceff4b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7face80ca320 con 0x7face80c6e00 2024-09-16T00:11:13.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7faceff4b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7face80ca8e0 con 0x7face80c6a20 2024-09-16T00:11:13.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7facedce7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7face806ea70 0x7face8154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:13.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7facedce7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7face806ea70 0x7face8154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60198/0 (socket says 172.21.15.139:60198) 2024-09-16T00:11:13.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7facedce7640 1 -- 172.21.15.139:0/2459669099 learned_addr learned my addr 172.21.15.139:0/2459669099 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:13.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7faced4e6640 1 --2- 172.21.15.139:0/2459669099 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7face80c6a20 0x7face80cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:13.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7facee4e8640 1 --2- 172.21.15.139:0/2459669099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7face80c6e00 0x7face814b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:13.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7facedce7640 1 -- 172.21.15.139:0/2459669099 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7face80c6a20 msgr2=0x7face80cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:13.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7facedce7640 1 --2- 172.21.15.139:0/2459669099 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7face80c6a20 0x7face80cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:13.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7facedce7640 1 -- 172.21.15.139:0/2459669099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7face80c6e00 msgr2=0x7face814b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:13.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7facedce7640 1 --2- 172.21.15.139:0/2459669099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7face80c6e00 0x7face814b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:13.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7facedce7640 1 -- 172.21.15.139:0/2459669099 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face80cc530 con 0x7face806ea70 2024-09-16T00:11:13.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7facee4e8640 1 --2- 172.21.15.139:0/2459669099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7face80c6e00 0x7face814b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:13.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7faced4e6640 1 --2- 172.21.15.139:0/2459669099 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7face80c6a20 0x7face80cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:13.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.060+0000 7facedce7640 1 --2- 172.21.15.139:0/2459669099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7face806ea70 0x7face8154b10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7face400c7f0 tx=0x7face400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:13.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.061+0000 7facde7fc640 1 -- 172.21.15.139:0/2459669099 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4015070 con 0x7face806ea70 2024-09-16T00:11:13.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.061+0000 7facde7fc640 1 -- 172.21.15.139:0/2459669099 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face40079d0 con 0x7face806ea70 2024-09-16T00:11:13.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.061+0000 7facde7fc640 1 -- 172.21.15.139:0/2459669099 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4007da0 con 0x7face806ea70 2024-09-16T00:11:13.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.061+0000 7faceff4b640 1 -- 172.21.15.139:0/2459669099 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face81e3e40 con 0x7face806ea70 2024-09-16T00:11:13.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.061+0000 7faceff4b640 1 -- 172.21.15.139:0/2459669099 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face81530e0 con 0x7face806ea70 2024-09-16T00:11:13.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.062+0000 7faceff4b640 1 -- 172.21.15.139:0/2459669099 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face80c8d80 con 0x7face806ea70 2024-09-16T00:11:13.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.062+0000 7facde7fc640 1 -- 172.21.15.139:0/2459669099 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7face4018070 con 0x7face806ea70 2024-09-16T00:11:13.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.062+0000 7facde7fc640 1 --2- 172.21.15.139:0/2459669099 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faccc06c2e0 0x7faccc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:13.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.063+0000 7facde7fc640 1 -- 172.21.15.139:0/2459669099 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5944+0+0 (secure 0 0 0) 0x7face408b3e0 con 0x7face806ea70 2024-09-16T00:11:13.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.063+0000 7faced4e6640 1 --2- 172.21.15.139:0/2459669099 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faccc06c2e0 0x7faccc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:13.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.063+0000 7faced4e6640 1 --2- 172.21.15.139:0/2459669099 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faccc06c2e0 0x7faccc06e7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7facd8034000 tx=0x7facd802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:13.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.067+0000 7facde7fc640 1 -- 172.21.15.139:0/2459669099 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7face80c8d80 con 0x7face806ea70 2024-09-16T00:11:13.245 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.243+0000 7faceff4b640 1 -- 172.21.15.139:0/2459669099 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} v 0) -- 0x7face80cb2e0 con 0x7face806ea70 2024-09-16T00:11:13.948 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.946+0000 7facde7fc640 1 -- 172.21.15.139:0/2459669099 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]=0 set osd.7 primary-affinity to 1 (8655362) v146) ==== 133+0+0 (secure 0 0 0) 0x7face4059af0 con 0x7face806ea70 2024-09-16T00:11:13.949 INFO:teuthology.orchestra.run.smithi139.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-16T00:11:13.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.949+0000 7faceff4b640 1 -- 172.21.15.139:0/2459669099 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faccc06c2e0 msgr2=0x7faccc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:13.951 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.949+0000 7faceff4b640 1 --2- 172.21.15.139:0/2459669099 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faccc06c2e0 0x7faccc06e7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7facd8034000 tx=0x7facd802e000 comp rx=0 tx=0).stop 2024-09-16T00:11:13.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.949+0000 7faceff4b640 1 -- 172.21.15.139:0/2459669099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7face806ea70 msgr2=0x7face8154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:13.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.949+0000 7faceff4b640 1 --2- 172.21.15.139:0/2459669099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7face806ea70 0x7face8154b10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7face400c7f0 tx=0x7face400cbf0 comp rx=0 tx=0).stop 2024-09-16T00:11:13.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.949+0000 7faceff4b640 1 -- 172.21.15.139:0/2459669099 shutdown_connections 2024-09-16T00:11:13.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.949+0000 7faceff4b640 1 --2- 172.21.15.139:0/2459669099 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faccc06c2e0 0x7faccc06e7a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:13.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.949+0000 7faceff4b640 1 --2- 172.21.15.139:0/2459669099 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7face80c6e00 0x7face814b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:13.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.949+0000 7faceff4b640 1 --2- 172.21.15.139:0/2459669099 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7face80c6a20 0x7face80cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:13.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.949+0000 7faceff4b640 1 --2- 172.21.15.139:0/2459669099 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7face806ea70 0x7face8154b10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:13.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.949+0000 7faceff4b640 1 -- 172.21.15.139:0/2459669099 >> 172.21.15.139:0/2459669099 conn(0x7face806cc30 msgr2=0x7face814d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:13.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.949+0000 7faceff4b640 1 -- 172.21.15.139:0/2459669099 shutdown_connections 2024-09-16T00:11:13.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:13.950+0000 7faceff4b640 1 -- 172.21.15.139:0/2459669099 wait complete. 2024-09-16T00:11:13.965 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-16T00:11:13.966 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-16T00:11:13.966 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-16T00:11:13.966 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-16T00:11:13.966 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T00:11:13.966 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:11:14.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.071+0000 7ffabdae6640 1 Processor -- start 2024-09-16T00:11:14.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.071+0000 7ffabdae6640 1 -- start start 2024-09-16T00:11:14.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.072+0000 7ffabdae6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffab81591d0 0x7ffab80c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.072+0000 7ffabdae6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffab80c8180 0x7ffab80c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.072+0000 7ffabdae6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffab806ffb0 con 0x7ffab8152700 2024-09-16T00:11:14.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.072+0000 7ffabdae6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ffab806fd60 con 0x7ffab81591d0 2024-09-16T00:11:14.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.072+0000 7ffabdae6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ffab81586f0 con 0x7ffab80c8180 2024-09-16T00:11:14.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.072+0000 7ffab6ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffab81591d0 0x7ffab80c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.072+0000 7ffab77fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ffab8152700 0x7ffab806d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46148/0 (socket says 172.21.15.139:46148) 2024-09-16T00:11:14.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.072+0000 7ffab77fe640 1 -- 172.21.15.139:0/1114104640 learned_addr learned my addr 172.21.15.139:0/1114104640 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:14.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.072+0000 7ffab6ffd640 1 -- 172.21.15.139:0/1114104640 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffab80c8180 msgr2=0x7ffab80c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:14.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.072+0000 7ffab7fff640 1 --2- 172.21.15.139:0/1114104640 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffab80c8180 0x7ffab80c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.072+0000 7ffab67fc640 1 -- 172.21.15.139:0/1114104640 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1823380443 0 0) 0x7ffab806ffb0 con 0x7ffab8152700 2024-09-16T00:11:14.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.072+0000 7ffab6ffd640 1 --2- 172.21.15.139:0/1114104640 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffab80c8180 0x7ffab80c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.072+0000 7ffab6ffd640 1 -- 172.21.15.139:0/1114104640 >> v1:172.21.15.139:6789/0 conn(0x7ffab8152700 legacy=0x7ffab806d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.072+0000 7ffab6ffd640 1 -- 172.21.15.139:0/1114104640 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab81589d0 con 0x7ffab81591d0 2024-09-16T00:11:14.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.073+0000 7ffab6ffd640 1 --2- 172.21.15.139:0/1114104640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffab81591d0 0x7ffab80c5b20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0097a0 tx=0x7ffaac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51861d2331d733d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.073+0000 7ffab67fc640 1 -- 172.21.15.139:0/1114104640 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac037070 con 0x7ffab81591d0 2024-09-16T00:11:14.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.073+0000 7ffab67fc640 1 -- 172.21.15.139:0/1114104640 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaac032440 con 0x7ffab81591d0 2024-09-16T00:11:14.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.073+0000 7ffab67fc640 1 -- 172.21.15.139:0/1114104640 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac032810 con 0x7ffab81591d0 2024-09-16T00:11:14.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.074+0000 7ffabdae6640 1 -- 172.21.15.139:0/1114104640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffab81591d0 msgr2=0x7ffab80c5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.074+0000 7ffabdae6640 1 --2- 172.21.15.139:0/1114104640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffab81591d0 0x7ffab80c5b20 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0097a0 tx=0x7ffaac02a180 comp rx=0 tx=0).stop 2024-09-16T00:11:14.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.074+0000 7ffabdae6640 1 -- 172.21.15.139:0/1114104640 shutdown_connections 2024-09-16T00:11:14.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.074+0000 7ffabdae6640 1 --2- 172.21.15.139:0/1114104640 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffab80c8180 0x7ffab80c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.074+0000 7ffabdae6640 1 --2- 172.21.15.139:0/1114104640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffab81591d0 0x7ffab80c5b20 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.074+0000 7ffabdae6640 1 -- 172.21.15.139:0/1114104640 >> 172.21.15.139:0/1114104640 conn(0x7ffab80cc410 msgr2=0x7ffab80cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.074+0000 7ffabdae6640 1 -- 172.21.15.139:0/1114104640 shutdown_connections 2024-09-16T00:11:14.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.074+0000 7ffabdae6640 1 -- 172.21.15.139:0/1114104640 wait complete. 2024-09-16T00:11:14.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.075+0000 7ffabdae6640 1 Processor -- start 2024-09-16T00:11:14.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.075+0000 7ffabdae6640 1 -- start start 2024-09-16T00:11:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.076+0000 7ffabdae6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffab80c8180 0x7ffab80c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.076+0000 7ffabdae6640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffab8152700 0x7ffab80cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.076+0000 7ffabdae6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffab81591d0 0x7ffab81e3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.076+0000 7ffabdae6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffab80c8880 con 0x7ffab81591d0 2024-09-16T00:11:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.076+0000 7ffabdae6640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ffab806f8c0 con 0x7ffab8152700 2024-09-16T00:11:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.076+0000 7ffabdae6640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ffab80c8a00 con 0x7ffab80c8180 2024-09-16T00:11:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.076+0000 7ffab7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffab81591d0 0x7ffab81e3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.076+0000 7ffab7fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffab81591d0 0x7ffab81e3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60224/0 (socket says 172.21.15.139:60224) 2024-09-16T00:11:14.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.076+0000 7ffab7fff640 1 -- 172.21.15.139:0/1477801380 learned_addr learned my addr 172.21.15.139:0/1477801380 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.076+0000 7ffab77fe640 1 --2- 172.21.15.139:0/1477801380 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffab80c8180 0x7ffab80c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.076+0000 7ffab6ffd640 1 --2- 172.21.15.139:0/1477801380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffab8152700 0x7ffab80cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.076+0000 7ffab7fff640 1 -- 172.21.15.139:0/1477801380 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffab80c8180 msgr2=0x7ffab80c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.076+0000 7ffab7fff640 1 --2- 172.21.15.139:0/1477801380 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffab80c8180 0x7ffab80c52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.076+0000 7ffab7fff640 1 -- 172.21.15.139:0/1477801380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffab8152700 msgr2=0x7ffab80cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.076+0000 7ffab7fff640 1 --2- 172.21.15.139:0/1477801380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffab8152700 0x7ffab80cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.076+0000 7ffab7fff640 1 -- 172.21.15.139:0/1477801380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab8149200 con 0x7ffab81591d0 2024-09-16T00:11:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.077+0000 7ffab7fff640 1 --2- 172.21.15.139:0/1477801380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffab81591d0 0x7ffab81e3610 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ffab000cda0 tx=0x7ffab000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.077+0000 7ffa9ffff640 1 -- 172.21.15.139:0/1477801380 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0003b90 con 0x7ffab81591d0 2024-09-16T00:11:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.077+0000 7ffa9ffff640 1 -- 172.21.15.139:0/1477801380 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab0003d30 con 0x7ffab81591d0 2024-09-16T00:11:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.077+0000 7ffabdae6640 1 -- 172.21.15.139:0/1477801380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab80caf60 con 0x7ffab81591d0 2024-09-16T00:11:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.077+0000 7ffa9ffff640 1 -- 172.21.15.139:0/1477801380 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0004240 con 0x7ffab81591d0 2024-09-16T00:11:14.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.077+0000 7ffabdae6640 1 -- 172.21.15.139:0/1477801380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab81f14e0 con 0x7ffab81591d0 2024-09-16T00:11:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.078+0000 7ffabdae6640 1 -- 172.21.15.139:0/1477801380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab806dcd0 con 0x7ffab81591d0 2024-09-16T00:11:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.078+0000 7ffa9ffff640 1 -- 172.21.15.139:0/1477801380 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ffab00043e0 con 0x7ffab81591d0 2024-09-16T00:11:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.079+0000 7ffa9ffff640 1 --2- 172.21.15.139:0/1477801380 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffaa406c2e0 0x7ffaa406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.079+0000 7ffab77fe640 1 --2- 172.21.15.139:0/1477801380 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffaa406c2e0 0x7ffaa406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.079+0000 7ffa9ffff640 1 -- 172.21.15.139:0/1477801380 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7ffab008bf30 con 0x7ffab81591d0 2024-09-16T00:11:14.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.079+0000 7ffab77fe640 1 --2- 172.21.15.139:0/1477801380 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffaa406c2e0 0x7ffaa406e7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8000f60 tx=0x7ffaa8007b50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.083+0000 7ffa9ffff640 1 -- 172.21.15.139:0/1477801380 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ffab806dcd0 con 0x7ffab81591d0 2024-09-16T00:11:14.250 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.249+0000 7ffabdae6640 1 -- 172.21.15.139:0/1477801380 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffab80c6e10 con 0x7ffab81591d0 2024-09-16T00:11:14.251 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.250+0000 7ffa9ffff640 1 -- 172.21.15.139:0/1477801380 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+17050 (secure 0 0 0) 0x7ffab005a640 con 0x7ffab81591d0 2024-09-16T00:11:14.252 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:11:14.252 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":146,"fsid":"2d3366e5-6a59-4b59-93b2-615e72ac7015","created":"2024-09-16T00:07:09.967833+0000","modified":"2024-09-16T00:11:13.946870+0000","last_up_change":"2024-09-16T00:10:54.809507+0000","last_in_change":"2024-09-16T00:07:12.025620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:07:22.056303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:07:30.768649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:07:32.008209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83699a05-4937-43c3-aefc-b669bed9bb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6817","nonce":1974901438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6819","nonce":1974901438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6823","nonce":1974901438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6821","nonce":1974901438}]},"public_addr":"172.21.15.139:6817/1974901438","cluster_addr":"172.21.15.139:6819/1974901438","heartbeat_back_addr":"172.21.15.139:6823/1974901438","heartbeat_front_addr":"172.21.15.139:6821/1974901438","state":["exists","up"]},{"osd":1,"uuid":"9c04991e-9695-4bbb-8011-bf613bf404b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6809","nonce":2232380998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6811","nonce":2232380998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6815","nonce":2232380998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6813","nonce":2232380998}]},"public_addr":"172.21.15.139:6809/2232380998","cluster_addr":"172.21.15.139:6811/2232380998","heartbeat_back_addr":"172.21.15.139:6815/2232380998","heartbeat_front_addr":"172.21.15.139:6813/2232380998","state":["exists","up"]},{"osd":2,"uuid":"8048fc64-72b3-4866-ac62-6a8b2bf6f5c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6801","nonce":4180588905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6803","nonce":4180588905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6807","nonce":4180588905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6805","nonce":4180588905}]},"public_addr":"172.21.15.139:6801/4180588905","cluster_addr":"172.21.15.139:6803/4180588905","heartbeat_back_addr":"172.21.15.139:6807/4180588905","heartbeat_front_addr":"172.21.15.139:6805/4180588905","state":["exists","up"]},{"osd":3,"uuid":"c34ac77b-354c-4884-902d-3e2a84f5eb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6827","nonce":3947768492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6829","nonce":3947768492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6833","nonce":3947768492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6831","nonce":3947768492}]},"public_addr":"172.21.15.139:6827/3947768492","cluster_addr":"172.21.15.139:6829/3947768492","heartbeat_back_addr":"172.21.15.139:6833/3947768492","heartbeat_front_addr":"172.21.15.139:6831/3947768492","state":["exists","up"]},{"osd":4,"uuid":"ba9a7c93-9672-43e1-8c98-2878cdf7f548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6817","nonce":642632916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6819","nonce":642632916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6823","nonce":642632916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6821","nonce":642632916}]},"public_addr":"172.21.15.164:6817/642632916","cluster_addr":"172.21.15.164:6819/642632916","heartbeat_back_addr":"172.21.15.164:6823/642632916","heartbeat_front_addr":"172.21.15.164:6821/642632916","state":["exists","up"]},{"osd":5,"uuid":"46a0916b-474c-411b-84ad-86f27d4cbf53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6801","nonce":1051778383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6803","nonce":1051778383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6807","nonce":1051778383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6805","nonce":1051778383}]},"public_addr":"172.21.15.164:6801/1051778383","cluster_addr":"172.21.15.164:6803/1051778383","heartbeat_back_addr":"172.21.15.164:6807/1051778383","heartbeat_front_addr":"172.21.15.164:6805/1051778383","state":["exists","up"]},{"osd":6,"uuid":"9b6b33cc-d6d9-4a0a-8143-526073a58b93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":82,"up_from":94,"up_thru":127,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":164138206},{"type":"v1","addr":"172.21.15.164:6825","nonce":164138206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":164138206},{"type":"v1","addr":"172.21.15.164:6827","nonce":164138206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":164138206},{"type":"v1","addr":"172.21.15.164:6831","nonce":164138206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":164138206},{"type":"v1","addr":"172.21.15.164:6829","nonce":164138206}]},"public_addr":"172.21.15.164:6825/164138206","cluster_addr":"172.21.15.164:6827/164138206","heartbeat_back_addr":"172.21.15.164:6831/164138206","heartbeat_front_addr":"172.21.15.164:6829/164138206","state":["exists","up"]},{"osd":7,"uuid":"50bcd808-3d3e-4a72-88e4-30decb53cfea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":112,"up_from":127,"up_thru":136,"down_at":113,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2782017910},{"type":"v1","addr":"172.21.15.164:6809","nonce":2782017910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2782017910},{"type":"v1","addr":"172.21.15.164:6811","nonce":2782017910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2782017910},{"type":"v1","addr":"172.21.15.164:6815","nonce":2782017910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2782017910},{"type":"v1","addr":"172.21.15.164:6813","nonce":2782017910}]},"public_addr":"172.21.15.164:6809/2782017910","cluster_addr":"172.21.15.164:6811/2782017910","heartbeat_back_addr":"172.21.15.164:6815/2782017910","heartbeat_front_addr":"172.21.15.164:6813/2782017910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.843117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.622789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:17.919457+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:19.330341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.815640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.083046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.551499+0000","dead_epoch":84},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.377179+0000","dead_epoch":113}],"pg_upmap":[{"pgid":"3.2d","osds":[7,5]}],"pg_upmap_items":[{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:11:14.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.252+0000 7ffabdae6640 1 -- 172.21.15.139:0/1477801380 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffaa406c2e0 msgr2=0x7ffaa406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.252+0000 7ffabdae6640 1 --2- 172.21.15.139:0/1477801380 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffaa406c2e0 0x7ffaa406e7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8000f60 tx=0x7ffaa8007b50 comp rx=0 tx=0).stop 2024-09-16T00:11:14.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.252+0000 7ffabdae6640 1 -- 172.21.15.139:0/1477801380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffab81591d0 msgr2=0x7ffab81e3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.252+0000 7ffabdae6640 1 --2- 172.21.15.139:0/1477801380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffab81591d0 0x7ffab81e3610 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ffab000cda0 tx=0x7ffab000a660 comp rx=0 tx=0).stop 2024-09-16T00:11:14.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.252+0000 7ffabdae6640 1 -- 172.21.15.139:0/1477801380 shutdown_connections 2024-09-16T00:11:14.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.252+0000 7ffabdae6640 1 --2- 172.21.15.139:0/1477801380 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ffaa406c2e0 0x7ffaa406e7a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.252+0000 7ffabdae6640 1 --2- 172.21.15.139:0/1477801380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffab81591d0 0x7ffab81e3610 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.254 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.252+0000 7ffabdae6640 1 --2- 172.21.15.139:0/1477801380 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ffab8152700 0x7ffab80cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.252+0000 7ffabdae6640 1 --2- 172.21.15.139:0/1477801380 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ffab80c8180 0x7ffab80c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.252+0000 7ffabdae6640 1 -- 172.21.15.139:0/1477801380 >> 172.21.15.139:0/1477801380 conn(0x7ffab80cc410 msgr2=0x7ffab814d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.252+0000 7ffabdae6640 1 -- 172.21.15.139:0/1477801380 shutdown_connections 2024-09-16T00:11:14.255 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.252+0000 7ffabdae6640 1 -- 172.21.15.139:0/1477801380 wait complete. 2024-09-16T00:11:14.265 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T00:11:14.266 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:11:14.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.362+0000 7f385579f640 1 Processor -- start 2024-09-16T00:11:14.364 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.362+0000 7f385579f640 1 -- start start 2024-09-16T00:11:14.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.362+0000 7f385579f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f38500cb230 0x7f385006e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.362+0000 7f385579f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38500c8c90 con 0x7f38500ce610 2024-09-16T00:11:14.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.362+0000 7f385579f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f385006ce80 con 0x7f38500ce230 2024-09-16T00:11:14.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.362+0000 7f385579f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f385006cc30 con 0x7f38500cb230 2024-09-16T00:11:14.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.363+0000 7f3854f9e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f38500cb230 0x7f385006e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.365 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.363+0000 7f3854f9e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f38500cb230 0x7f385006e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60240/0 (socket says 172.21.15.139:60240) 2024-09-16T00:11:14.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.363+0000 7f3854f9e640 1 -- 172.21.15.139:0/640969754 learned_addr learned my addr 172.21.15.139:0/640969754 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:14.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.363+0000 7f384f7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f38500ce610 0x7f38500cab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46152/0 (socket says 172.21.15.139:46152) 2024-09-16T00:11:14.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.363+0000 7f3854f9e640 1 -- 172.21.15.139:0/640969754 >> v1:172.21.15.164:6789/0 conn(0x7f38500ce230 legacy=0x7f385006fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.363+0000 7f3854f9e640 1 -- 172.21.15.139:0/640969754 >> v1:172.21.15.139:6789/0 conn(0x7f38500ce610 legacy=0x7f38500cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.366 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.363+0000 7f3854f9e640 1 -- 172.21.15.139:0/640969754 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385006d160 con 0x7f38500cb230 2024-09-16T00:11:14.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.364+0000 7f3854f9e640 1 --2- 172.21.15.139:0/640969754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38500cb230 0x7f385006e8d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f385006cdb0 tx=0x7f384802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7893c65a7961ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.364+0000 7f384effd640 1 -- 172.21.15.139:0/640969754 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848036070 con 0x7f38500cb230 2024-09-16T00:11:14.367 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.364+0000 7f384effd640 1 -- 172.21.15.139:0/640969754 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3848031600 con 0x7f38500cb230 2024-09-16T00:11:14.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.364+0000 7f384effd640 1 -- 172.21.15.139:0/640969754 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38480319d0 con 0x7f38500cb230 2024-09-16T00:11:14.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.364+0000 7f385579f640 1 -- 172.21.15.139:0/640969754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38500cb230 msgr2=0x7f385006e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.364+0000 7f385579f640 1 --2- 172.21.15.139:0/640969754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38500cb230 0x7f385006e8d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f385006cdb0 tx=0x7f384802a730 comp rx=0 tx=0).stop 2024-09-16T00:11:14.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.364+0000 7f385579f640 1 -- 172.21.15.139:0/640969754 shutdown_connections 2024-09-16T00:11:14.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.364+0000 7f385579f640 1 --2- 172.21.15.139:0/640969754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38500cb230 0x7f385006e8d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.364+0000 7f385579f640 1 -- 172.21.15.139:0/640969754 >> 172.21.15.139:0/640969754 conn(0x7f3850073d60 msgr2=0x7f3850067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.364+0000 7f385579f640 1 -- 172.21.15.139:0/640969754 shutdown_connections 2024-09-16T00:11:14.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.364+0000 7f385579f640 1 -- 172.21.15.139:0/640969754 wait complete. 2024-09-16T00:11:14.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.365+0000 7f385579f640 1 Processor -- start 2024-09-16T00:11:14.368 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.366+0000 7f385579f640 1 -- start start 2024-09-16T00:11:14.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.366+0000 7f385579f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38500cb230 0x7f38500c4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.366+0000 7f385579f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f38500ce230 0x7f38500c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.366+0000 7f385579f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38500ce610 0x7f3850153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.366+0000 7f385579f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f385015e3e0 con 0x7f38500ce610 2024-09-16T00:11:14.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.366+0000 7f385579f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f38500ca600 con 0x7f38500cb230 2024-09-16T00:11:14.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.366+0000 7f385579f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f385015e560 con 0x7f38500ce230 2024-09-16T00:11:14.371 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.366+0000 7f3854f9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38500ce610 0x7f3850153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.367+0000 7f3854f9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38500ce610 0x7f3850153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60252/0 (socket says 172.21.15.139:60252) 2024-09-16T00:11:14.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.367+0000 7f3854f9e640 1 -- 172.21.15.139:0/2478972146 learned_addr learned my addr 172.21.15.139:0/2478972146 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:14.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.367+0000 7f384f7fe640 1 --2- 172.21.15.139:0/2478972146 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f38500ce230 0x7f38500c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.367+0000 7f384ffff640 1 --2- 172.21.15.139:0/2478972146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38500cb230 0x7f38500c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.367+0000 7f3854f9e640 1 -- 172.21.15.139:0/2478972146 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f38500ce230 msgr2=0x7f38500c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.367+0000 7f3854f9e640 1 --2- 172.21.15.139:0/2478972146 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f38500ce230 0x7f38500c1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.367+0000 7f3854f9e640 1 -- 172.21.15.139:0/2478972146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38500cb230 msgr2=0x7f38500c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.367+0000 7f3854f9e640 1 --2- 172.21.15.139:0/2478972146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38500cb230 0x7f38500c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.367+0000 7f3854f9e640 1 -- 172.21.15.139:0/2478972146 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38500c25e0 con 0x7f38500ce610 2024-09-16T00:11:14.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.367+0000 7f3854f9e640 1 --2- 172.21.15.139:0/2478972146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38500ce610 0x7f3850153980 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3848009f40 tx=0x7f3848004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.367+0000 7f384cff9640 1 -- 172.21.15.139:0/2478972146 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848036050 con 0x7f38500ce610 2024-09-16T00:11:14.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.367+0000 7f384cff9640 1 -- 172.21.15.139:0/2478972146 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3848042070 con 0x7f38500ce610 2024-09-16T00:11:14.372 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.367+0000 7f384cff9640 1 -- 172.21.15.139:0/2478972146 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848009510 con 0x7f38500ce610 2024-09-16T00:11:14.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.367+0000 7f385579f640 1 -- 172.21.15.139:0/2478972146 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38500c78e0 con 0x7f38500ce610 2024-09-16T00:11:14.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.367+0000 7f385579f640 1 -- 172.21.15.139:0/2478972146 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38500c7ea0 con 0x7f38500ce610 2024-09-16T00:11:14.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.368+0000 7f385579f640 1 -- 172.21.15.139:0/2478972146 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f385015dd30 con 0x7f38500ce610 2024-09-16T00:11:14.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.369+0000 7f384cff9640 1 -- 172.21.15.139:0/2478972146 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3848007bb0 con 0x7f38500ce610 2024-09-16T00:11:14.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.369+0000 7f384cff9640 1 --2- 172.21.15.139:0/2478972146 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f383c06c2b0 0x7f383c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.369+0000 7f384cff9640 1 -- 172.21.15.139:0/2478972146 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f38480ac2f0 con 0x7f38500ce610 2024-09-16T00:11:14.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.369+0000 7f384ffff640 1 --2- 172.21.15.139:0/2478972146 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f383c06c2b0 0x7f383c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.370+0000 7f384ffff640 1 --2- 172.21.15.139:0/2478972146 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f383c06c2b0 0x7f383c06e770 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3840001780 tx=0x7f3840009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.375 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.374+0000 7f384cff9640 1 -- 172.21.15.139:0/2478972146 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f385015dd30 con 0x7f38500ce610 2024-09-16T00:11:14.545 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.543+0000 7f385579f640 1 -- 172.21.15.139:0/2478972146 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f385015d490 con 0x7f38500ce610 2024-09-16T00:11:14.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.544+0000 7f384cff9640 1 -- 172.21.15.139:0/2478972146 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+17050 (secure 0 0 0) 0x7f385015d490 con 0x7f38500ce610 2024-09-16T00:11:14.546 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:11:14.546 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":146,"fsid":"2d3366e5-6a59-4b59-93b2-615e72ac7015","created":"2024-09-16T00:07:09.967833+0000","modified":"2024-09-16T00:11:13.946870+0000","last_up_change":"2024-09-16T00:10:54.809507+0000","last_in_change":"2024-09-16T00:07:12.025620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:07:22.056303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:07:30.768649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:07:32.008209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83699a05-4937-43c3-aefc-b669bed9bb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6817","nonce":1974901438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6819","nonce":1974901438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6823","nonce":1974901438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6821","nonce":1974901438}]},"public_addr":"172.21.15.139:6817/1974901438","cluster_addr":"172.21.15.139:6819/1974901438","heartbeat_back_addr":"172.21.15.139:6823/1974901438","heartbeat_front_addr":"172.21.15.139:6821/1974901438","state":["exists","up"]},{"osd":1,"uuid":"9c04991e-9695-4bbb-8011-bf613bf404b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6809","nonce":2232380998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6811","nonce":2232380998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6815","nonce":2232380998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6813","nonce":2232380998}]},"public_addr":"172.21.15.139:6809/2232380998","cluster_addr":"172.21.15.139:6811/2232380998","heartbeat_back_addr":"172.21.15.139:6815/2232380998","heartbeat_front_addr":"172.21.15.139:6813/2232380998","state":["exists","up"]},{"osd":2,"uuid":"8048fc64-72b3-4866-ac62-6a8b2bf6f5c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6801","nonce":4180588905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6803","nonce":4180588905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6807","nonce":4180588905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6805","nonce":4180588905}]},"public_addr":"172.21.15.139:6801/4180588905","cluster_addr":"172.21.15.139:6803/4180588905","heartbeat_back_addr":"172.21.15.139:6807/4180588905","heartbeat_front_addr":"172.21.15.139:6805/4180588905","state":["exists","up"]},{"osd":3,"uuid":"c34ac77b-354c-4884-902d-3e2a84f5eb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6827","nonce":3947768492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6829","nonce":3947768492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6833","nonce":3947768492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6831","nonce":3947768492}]},"public_addr":"172.21.15.139:6827/3947768492","cluster_addr":"172.21.15.139:6829/3947768492","heartbeat_back_addr":"172.21.15.139:6833/3947768492","heartbeat_front_addr":"172.21.15.139:6831/3947768492","state":["exists","up"]},{"osd":4,"uuid":"ba9a7c93-9672-43e1-8c98-2878cdf7f548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6817","nonce":642632916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6819","nonce":642632916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6823","nonce":642632916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6821","nonce":642632916}]},"public_addr":"172.21.15.164:6817/642632916","cluster_addr":"172.21.15.164:6819/642632916","heartbeat_back_addr":"172.21.15.164:6823/642632916","heartbeat_front_addr":"172.21.15.164:6821/642632916","state":["exists","up"]},{"osd":5,"uuid":"46a0916b-474c-411b-84ad-86f27d4cbf53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6801","nonce":1051778383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6803","nonce":1051778383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6807","nonce":1051778383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6805","nonce":1051778383}]},"public_addr":"172.21.15.164:6801/1051778383","cluster_addr":"172.21.15.164:6803/1051778383","heartbeat_back_addr":"172.21.15.164:6807/1051778383","heartbeat_front_addr":"172.21.15.164:6805/1051778383","state":["exists","up"]},{"osd":6,"uuid":"9b6b33cc-d6d9-4a0a-8143-526073a58b93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":82,"up_from":94,"up_thru":127,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":164138206},{"type":"v1","addr":"172.21.15.164:6825","nonce":164138206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":164138206},{"type":"v1","addr":"172.21.15.164:6827","nonce":164138206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":164138206},{"type":"v1","addr":"172.21.15.164:6831","nonce":164138206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":164138206},{"type":"v1","addr":"172.21.15.164:6829","nonce":164138206}]},"public_addr":"172.21.15.164:6825/164138206","cluster_addr":"172.21.15.164:6827/164138206","heartbeat_back_addr":"172.21.15.164:6831/164138206","heartbeat_front_addr":"172.21.15.164:6829/164138206","state":["exists","up"]},{"osd":7,"uuid":"50bcd808-3d3e-4a72-88e4-30decb53cfea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":112,"up_from":127,"up_thru":136,"down_at":113,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2782017910},{"type":"v1","addr":"172.21.15.164:6809","nonce":2782017910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2782017910},{"type":"v1","addr":"172.21.15.164:6811","nonce":2782017910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2782017910},{"type":"v1","addr":"172.21.15.164:6815","nonce":2782017910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2782017910},{"type":"v1","addr":"172.21.15.164:6813","nonce":2782017910}]},"public_addr":"172.21.15.164:6809/2782017910","cluster_addr":"172.21.15.164:6811/2782017910","heartbeat_back_addr":"172.21.15.164:6815/2782017910","heartbeat_front_addr":"172.21.15.164:6813/2782017910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.843117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.622789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:17.919457+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:19.330341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.815640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.083046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.551499+0000","dead_epoch":84},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.377179+0000","dead_epoch":113}],"pg_upmap":[{"pgid":"3.2d","osds":[7,5]}],"pg_upmap_items":[{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:11:14.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.546+0000 7f385579f640 1 -- 172.21.15.139:0/2478972146 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f383c06c2b0 msgr2=0x7f383c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.546+0000 7f385579f640 1 --2- 172.21.15.139:0/2478972146 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f383c06c2b0 0x7f383c06e770 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3840001780 tx=0x7f3840009040 comp rx=0 tx=0).stop 2024-09-16T00:11:14.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.546+0000 7f385579f640 1 -- 172.21.15.139:0/2478972146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38500ce610 msgr2=0x7f3850153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.546+0000 7f385579f640 1 --2- 172.21.15.139:0/2478972146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38500ce610 0x7f3850153980 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3848009f40 tx=0x7f3848004a40 comp rx=0 tx=0).stop 2024-09-16T00:11:14.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.546+0000 7f385579f640 1 -- 172.21.15.139:0/2478972146 shutdown_connections 2024-09-16T00:11:14.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.546+0000 7f385579f640 1 --2- 172.21.15.139:0/2478972146 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f383c06c2b0 0x7f383c06e770 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.546+0000 7f385579f640 1 --2- 172.21.15.139:0/2478972146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38500ce610 0x7f3850153980 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.546+0000 7f385579f640 1 --2- 172.21.15.139:0/2478972146 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f38500ce230 0x7f38500c1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.546+0000 7f385579f640 1 --2- 172.21.15.139:0/2478972146 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f38500cb230 0x7f38500c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.546+0000 7f385579f640 1 -- 172.21.15.139:0/2478972146 >> 172.21.15.139:0/2478972146 conn(0x7f3850073d60 msgr2=0x7f3850158e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.546+0000 7f385579f640 1 -- 172.21.15.139:0/2478972146 shutdown_connections 2024-09-16T00:11:14.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.546+0000 7f385579f640 1 -- 172.21.15.139:0/2478972146 wait complete. 2024-09-16T00:11:14.560 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T00:11:14.560 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T00:11:14.561 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T00:11:14.561 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-16T00:11:14.561 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-16T00:11:14.562 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-16T00:11:14.562 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-16T00:11:14.563 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-16T00:11:14.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.727+0000 7f3e572aa640 1 Processor -- start 2024-09-16T00:11:14.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.727+0000 7f3e572aa640 1 -- start start 2024-09-16T00:11:14.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.728+0000 7f3e572aa640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3e500ca730 0x7f3e50158e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.728+0000 7f3e572aa640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e500c81b0 0x7f3e50162d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.728+0000 7f3e572aa640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e500caf10 con 0x7f3e50071290 2024-09-16T00:11:14.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.728+0000 7f3e572aa640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3e500c5230 con 0x7f3e500ca730 2024-09-16T00:11:14.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.728+0000 7f3e572aa640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3e501593b0 con 0x7f3e500c81b0 2024-09-16T00:11:14.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.729+0000 7f506e6d0640 1 Processor -- start 2024-09-16T00:11:14.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.729+0000 7f506e6d0640 1 -- start start 2024-09-16T00:11:14.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.729+0000 7f3e55aa7640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f3e50071290 0x7f3e500737f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:44786/0 (socket says 172.21.15.139:44786) 2024-09-16T00:11:14.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.729+0000 7f3e55aa7640 1 -- 172.21.15.139:0/1739453537 learned_addr learned my addr 172.21.15.139:0/1739453537 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:14.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.729+0000 7f3e56aa9640 1 --2- 172.21.15.139:0/1739453537 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e500c81b0 0x7f3e50162d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.729+0000 7f3e562a8640 1 --2- 172.21.15.139:0/1739453537 >> v2:172.21.15.139:3300/0 conn(0x7f3e500ca730 0x7f3e50158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.729+0000 7f9b4f042640 1 Processor -- start 2024-09-16T00:11:14.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.729+0000 7f9b4f042640 1 -- start start 2024-09-16T00:11:14.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.729+0000 7feaa5f86640 1 Processor -- start 2024-09-16T00:11:14.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.729+0000 7feaa5f86640 1 -- start start 2024-09-16T00:11:14.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.730+0000 7f3e56aa9640 1 -- 172.21.15.139:0/1739453537 >> v2:172.21.15.139:3300/0 conn(0x7f3e500ca730 msgr2=0x7f3e50158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:14.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.730+0000 7f3e56aa9640 1 --2- 172.21.15.139:0/1739453537 >> v2:172.21.15.139:3300/0 conn(0x7f3e500ca730 0x7f3e50158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.730+0000 7f3e56aa9640 1 -- 172.21.15.139:0/1739453537 >> v1:172.21.15.164:6789/0 conn(0x7f3e50071290 legacy=0x7f3e500737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.730+0000 7f3e56aa9640 1 -- 172.21.15.139:0/1739453537 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e500cab10 con 0x7f3e500c81b0 2024-09-16T00:11:14.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.730+0000 7feaa5f86640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7feaa0070040 0x7feaa00c9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.730+0000 7feaa5f86640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaa00727d0 con 0x7feaa0158f00 2024-09-16T00:11:14.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.730+0000 7feaa5f86640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feaa0072ab0 con 0x7feaa006fc20 2024-09-16T00:11:14.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.730+0000 7feaa5f86640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7feaa00c8c90 con 0x7feaa0070040 2024-09-16T00:11:14.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.730+0000 7f506e6d0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f50680c36b0 0x7f5068151190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.730+0000 7f506e6d0640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5068067950 0x7f50680c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f506e6d0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5068067d60 0x7f50680c8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f506e6d0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f50680cabd0 con 0x7f50680c36b0 2024-09-16T00:11:14.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f506e6d0640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f50680cbca0 con 0x7f5068067950 2024-09-16T00:11:14.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f506e6d0640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f50680cb950 con 0x7f5068067d60 2024-09-16T00:11:14.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7fea9f7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7feaa0158f00 0x7feaa00c6040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46154/0 (socket says 172.21.15.139:46154) 2024-09-16T00:11:14.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7fea9f7fe640 1 -- 172.21.15.139:0/1417019134 learned_addr learned my addr 172.21.15.139:0/1417019134 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:14.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f9b4f042640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b480cafe0 0x7f9b480c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f9b4f042640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9b480cb420 0x7f9b480c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f9b4f042640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b480cbe00 con 0x7f9b480cac00 2024-09-16T00:11:14.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f9b4f042640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f9b480c44c0 con 0x7f9b480cb420 2024-09-16T00:11:14.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f9b4f042640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9b480cccb0 con 0x7f9b480cafe0 2024-09-16T00:11:14.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7fe2a98c9640 1 Processor -- start 2024-09-16T00:11:14.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f5067fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f50680c36b0 0x7f5068151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f5067fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f50680c36b0 0x7f5068151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60276/0 (socket says 172.21.15.139:60276) 2024-09-16T00:11:14.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f5067fff640 1 -- 172.21.15.139:0/3658843786 learned_addr learned my addr 172.21.15.139:0/3658843786 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:14.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7fea9ffff640 1 --2- 172.21.15.139:0/1417019134 >> v2:172.21.15.139:3300/0 conn(0x7feaa0070040 0x7feaa00c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f9b4e040640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f9b480cac00 0x7f9b480c4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46168/0 (socket says 172.21.15.139:46168) 2024-09-16T00:11:14.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f9b4e040640 1 -- 172.21.15.139:0/4213453310 learned_addr learned my addr 172.21.15.139:0/4213453310 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:14.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f9b4d83f640 1 --2- 172.21.15.139:0/4213453310 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b480cafe0 0x7f9b480c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f506cc6d640 1 --2- 172.21.15.139:0/3658843786 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5068067d60 0x7f50680c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f50677fe640 1 --2- 172.21.15.139:0/3658843786 >> v2:172.21.15.164:3300/0 conn(0x7f5068067950 0x7f50680c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7f5067fff640 1 -- 172.21.15.139:0/3658843786 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5068067d60 msgr2=0x7f50680c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:14.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7f5067fff640 1 --2- 172.21.15.139:0/3658843786 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5068067d60 0x7f50680c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7f5067fff640 1 -- 172.21.15.139:0/3658843786 >> v2:172.21.15.164:3300/0 conn(0x7f5068067950 msgr2=0x7f50680c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:14.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7f5067fff640 1 --2- 172.21.15.139:0/3658843786 >> v2:172.21.15.164:3300/0 conn(0x7f5068067950 0x7f50680c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7f5067fff640 1 -- 172.21.15.139:0/3658843786 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50680c7580 con 0x7f50680c36b0 2024-09-16T00:11:14.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f9b4e841640 1 --2- 172.21.15.139:0/4213453310 >> v2:172.21.15.164:3300/0 conn(0x7f9b480cb420 0x7f9b480c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7f9b4d03e640 1 -- 172.21.15.139:0/4213453310 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3024832605 0 0) 0x7f9b480cbe00 con 0x7f9b480cac00 2024-09-16T00:11:14.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7f9b4d03e640 1 -- 172.21.15.139:0/4213453310 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b34003520 con 0x7f9b480cac00 2024-09-16T00:11:14.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f6085d9d640 1 Processor -- start 2024-09-16T00:11:14.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f6085d9d640 1 -- start start 2024-09-16T00:11:14.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f1d6a00a640 1 Processor -- start 2024-09-16T00:11:14.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7f1d6a00a640 1 -- start start 2024-09-16T00:11:14.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.731+0000 7fe2a98c9640 1 -- start start 2024-09-16T00:11:14.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7f9b4e841640 1 -- 172.21.15.139:0/4213453310 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b480cafe0 msgr2=0x7f9b480c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:14.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7f9b4e841640 1 --2- 172.21.15.139:0/4213453310 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b480cafe0 0x7f9b480c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7f9b4e841640 1 -- 172.21.15.139:0/4213453310 >> v1:172.21.15.139:6789/0 conn(0x7f9b480cac00 legacy=0x7f9b480c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7f9b4e841640 1 -- 172.21.15.139:0/4213453310 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b480cc0a0 con 0x7f9b480cb420 2024-09-16T00:11:14.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7fe2a98c9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe2a4151850 0x7fe2a40c6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7fe2a98c9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe2a4151c30 0x7fe2a415ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7fe2a98c9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a40c8d30 con 0x7fe2a40c7390 2024-09-16T00:11:14.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7fe2a98c9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe2a40c88a0 con 0x7fe2a4151c30 2024-09-16T00:11:14.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7fe2a98c9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fe2a40c6a90 con 0x7fe2a4151850 2024-09-16T00:11:14.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7f1d6a00a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d5c045120 0x7f1d5c0d48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7f1d6a00a640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1d5c00de10 0x7f1d5c005a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7fe2a2ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe2a4151850 0x7fe2a40c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7fe2a2ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe2a4151850 0x7fe2a40c6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39888/0 (socket says 172.21.15.139:39888) 2024-09-16T00:11:14.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7fe2a2ffd640 1 -- 172.21.15.139:0/40346832 learned_addr learned my addr 172.21.15.139:0/40346832 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:14.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7f1d6a00a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d5c04aa70 con 0x7f1d5c04c0c0 2024-09-16T00:11:14.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7f1d6a00a640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f1d5c044c90 con 0x7f1d5c00de10 2024-09-16T00:11:14.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7f1d6a00a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1d5c0d4e20 con 0x7f1d5c045120 2024-09-16T00:11:14.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f6085d9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6080071600 0x7f60800c6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f6085d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f608015a6b0 0x7f60800c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f6085d9d640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f608006fbc0 0x7f608006ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f6085d9d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f608015aba0 con 0x7f608015a6b0 2024-09-16T00:11:14.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f6085d9d640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f608006cb90 con 0x7f608006fbc0 2024-09-16T00:11:14.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7fea9ffff640 1 -- 172.21.15.139:0/1417019134 >> v1:172.21.15.164:6789/0 conn(0x7feaa006fc20 legacy=0x7feaa00cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7fea9ffff640 1 -- 172.21.15.139:0/1417019134 >> v1:172.21.15.139:6789/0 conn(0x7feaa0158f00 legacy=0x7feaa00c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.732+0000 7fea9ffff640 1 -- 172.21.15.139:0/1417019134 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa0072d90 con 0x7feaa0070040 2024-09-16T00:11:14.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f6085d9d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f608006ce70 con 0x7f6080071600 2024-09-16T00:11:14.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f1d62ffd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f1d5c00de10 0x7f1d5c005a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f607ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f608015a6b0 0x7f60800c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f6084d9b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6080071600 0x7f60800c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f607ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f608015a6b0 0x7f60800c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60300/0 (socket says 172.21.15.139:60300) 2024-09-16T00:11:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f5067fff640 1 --2- 172.21.15.139:0/3658843786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50680c36b0 0x7f5068151190 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f50680c3450 tx=0x7f505c02a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6204c39f457c95bf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f1d637fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d5c045120 0x7f1d5c0d48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f1d637fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d5c045120 0x7f1d5c0d48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:58278/0 (socket says 172.21.15.139:58278) 2024-09-16T00:11:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f1d637fe640 1 -- 172.21.15.139:0/1056413808 learned_addr learned my addr 172.21.15.139:0/1056413808 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:14.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7fea9ffff640 1 --2- 172.21.15.139:0/1417019134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa0070040 0x7feaa00c9120 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fea940097a0 tx=0x7fea9402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc852a49e3258a14 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f6084d9b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6080071600 0x7f60800c6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:58290/0 (socket says 172.21.15.139:58290) 2024-09-16T00:11:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f6084d9b640 1 -- 172.21.15.139:0/1410492800 learned_addr learned my addr 172.21.15.139:0/1410492800 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7fea9e7fc640 1 -- 172.21.15.139:0/1417019134 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea94037070 con 0x7feaa0070040 2024-09-16T00:11:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7fea9e7fc640 1 -- 172.21.15.139:0/1417019134 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea94032640 con 0x7feaa0070040 2024-09-16T00:11:14.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7fea9e7fc640 1 -- 172.21.15.139:0/1417019134 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea94032a10 con 0x7feaa0070040 2024-09-16T00:11:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f1d627fc640 1 -- 172.21.15.139:0/1056413808 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2739826821 0 0) 0x7f1d5c04aa70 con 0x7f1d5c04c0c0 2024-09-16T00:11:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f1d627fc640 1 -- 172.21.15.139:0/1056413808 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d54003520 con 0x7f1d5c04c0c0 2024-09-16T00:11:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7fe2a27fc640 1 --2- 172.21.15.139:0/40346832 >> v2:172.21.15.139:3300/0 conn(0x7fe2a4151c30 0x7fe2a415ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7fe2a27fc640 1 -- 172.21.15.139:0/40346832 >> v2:172.21.15.164:3300/0 conn(0x7fe2a4151850 msgr2=0x7fe2a40c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:14.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7fe2a27fc640 1 --2- 172.21.15.139:0/40346832 >> v2:172.21.15.164:3300/0 conn(0x7fe2a4151850 0x7fe2a40c6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7fe2a27fc640 1 -- 172.21.15.139:0/40346832 >> v1:172.21.15.164:6789/0 conn(0x7fe2a40c7390 legacy=0x7fe2a40c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:14.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7fe2a27fc640 1 -- 172.21.15.139:0/40346832 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a40c67b0 con 0x7fe2a4151c30 2024-09-16T00:11:14.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f5066ffd640 1 -- 172.21.15.139:0/3658843786 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505c037070 con 0x7f50680c36b0 2024-09-16T00:11:14.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f5066ffd640 1 -- 172.21.15.139:0/3658843786 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f505c032440 con 0x7f50680c36b0 2024-09-16T00:11:14.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f608559c640 1 --2- 172.21.15.139:0/1410492800 >> v2:172.21.15.164:3300/0 conn(0x7f608006fbc0 0x7f608006ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f6084d9b640 1 -- 172.21.15.139:0/1410492800 >> v2:172.21.15.164:3300/0 conn(0x7f608006fbc0 msgr2=0x7f608006ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:14.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f6084d9b640 1 --2- 172.21.15.139:0/1410492800 >> v2:172.21.15.164:3300/0 conn(0x7f608006fbc0 0x7f608006ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f6084d9b640 1 -- 172.21.15.139:0/1410492800 >> v2:172.21.15.139:3300/0 conn(0x7f608015a6b0 msgr2=0x7f60800c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:14.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f6084d9b640 1 --2- 172.21.15.139:0/1410492800 >> v2:172.21.15.139:3300/0 conn(0x7f608015a6b0 0x7f60800c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f6084d9b640 1 -- 172.21.15.139:0/1410492800 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608006d150 con 0x7f6080071600 2024-09-16T00:11:14.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f607ffff640 1 --2- 172.21.15.139:0/1410492800 >> v2:172.21.15.139:3300/0 conn(0x7f608015a6b0 0x7f60800c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:11:14.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f608559c640 1 --2- 172.21.15.139:0/1410492800 >> v2:172.21.15.164:3300/0 conn(0x7f608006fbc0 0x7f608006ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:11:14.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f1d637fe640 1 -- 172.21.15.139:0/1056413808 >> v2:172.21.15.164:3300/0 conn(0x7f1d5c00de10 msgr2=0x7f1d5c005a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:14.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f1d637fe640 1 --2- 172.21.15.139:0/1056413808 >> v2:172.21.15.164:3300/0 conn(0x7f1d5c00de10 0x7f1d5c005a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f1d637fe640 1 -- 172.21.15.139:0/1056413808 >> v1:172.21.15.164:6789/0 conn(0x7f1d5c04c0c0 legacy=0x7f1d5c0ddc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f1d637fe640 1 -- 172.21.15.139:0/1056413808 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5c04ad10 con 0x7f1d5c045120 2024-09-16T00:11:14.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f9b4e841640 1 --2- 172.21.15.139:0/4213453310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b480cb420 0x7f9b480c8fc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9b480cce30 tx=0x7f9b3802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8f447c8f75c6c9a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f506e6d0640 1 -- 172.21.15.139:0/3658843786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50680c36b0 msgr2=0x7f5068151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f506e6d0640 1 --2- 172.21.15.139:0/3658843786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50680c36b0 0x7f5068151190 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f50680c3450 tx=0x7f505c02a160 comp rx=0 tx=0).stop 2024-09-16T00:11:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f506e6d0640 1 -- 172.21.15.139:0/3658843786 shutdown_connections 2024-09-16T00:11:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.733+0000 7f3e56aa9640 1 --2- 172.21.15.139:0/1739453537 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e500c81b0 0x7f3e50162d20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009d90 tx=0x7f3e4802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12cabf5dae887e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f3e552a6640 1 -- 172.21.15.139:0/1739453537 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48007740 con 0x7f3e500c81b0 2024-09-16T00:11:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f3e552a6640 1 -- 172.21.15.139:0/1739453537 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e480078e0 con 0x7f3e500c81b0 2024-09-16T00:11:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f3e552a6640 1 -- 172.21.15.139:0/1739453537 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48007cb0 con 0x7f3e500c81b0 2024-09-16T00:11:14.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f506e6d0640 1 --2- 172.21.15.139:0/3658843786 >> v2:172.21.15.164:3300/0 conn(0x7f5068067950 0x7f50680c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f506e6d0640 1 --2- 172.21.15.139:0/3658843786 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f5068067d60 0x7f50680c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f506e6d0640 1 --2- 172.21.15.139:0/3658843786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50680c36b0 0x7f5068151190 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7feaa5f86640 1 -- 172.21.15.139:0/1417019134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa0070040 msgr2=0x7feaa00c9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7feaa5f86640 1 --2- 172.21.15.139:0/1417019134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa0070040 0x7feaa00c9120 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fea940097a0 tx=0x7fea9402a2c0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7f506e6d0640 1 -- 172.21.15.139:0/3658843786 >> 172.21.15.139:0/3658843786 conn(0x7f50680cc740 msgr2=0x7f50680ccb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.734+0000 7fe2a27fc640 1 --2- 172.21.15.139:0/40346832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2a4151c30 0x7fe2a415ac10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4075c60 tx=0x7fe29402a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423bd812c4df3762 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f9b4d03e640 1 -- 172.21.15.139:0/4213453310 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38037070 con 0x7f9b480cb420 2024-09-16T00:11:14.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f9b4d03e640 1 -- 172.21.15.139:0/4213453310 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b3802ac30 con 0x7f9b480cb420 2024-09-16T00:11:14.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f9b4d03e640 1 -- 172.21.15.139:0/4213453310 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44001670 con 0x7f9b480cb420 2024-09-16T00:11:14.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7fe2a1ffb640 1 -- 172.21.15.139:0/40346832 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294037070 con 0x7fe2a4151c30 2024-09-16T00:11:14.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f506e6d0640 1 -- 172.21.15.139:0/3658843786 shutdown_connections 2024-09-16T00:11:14.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f1d637fe640 1 --2- 172.21.15.139:0/1056413808 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d5c045120 0x7f1d5c0d48e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009870 tx=0x7f1d5002a2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3ff8dcccd71aa15 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7feaa5f86640 1 -- 172.21.15.139:0/1417019134 shutdown_connections 2024-09-16T00:11:14.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7feaa5f86640 1 --2- 172.21.15.139:0/1417019134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa0070040 0x7feaa00c9120 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7feaa5f86640 1 -- 172.21.15.139:0/1417019134 >> 172.21.15.139:0/1417019134 conn(0x7feaa006da40 msgr2=0x7feaa006f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7feaa5f86640 1 -- 172.21.15.139:0/1417019134 shutdown_connections 2024-09-16T00:11:14.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7feaa5f86640 1 -- 172.21.15.139:0/1417019134 wait complete. 2024-09-16T00:11:14.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f3e572aa640 1 -- 172.21.15.139:0/1739453537 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e500c81b0 msgr2=0x7f3e50162d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f3e572aa640 1 --2- 172.21.15.139:0/1739453537 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e500c81b0 0x7f3e50162d20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3e48009d90 tx=0x7f3e4802a5f0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f3e572aa640 1 -- 172.21.15.139:0/1739453537 shutdown_connections 2024-09-16T00:11:14.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f6084d9b640 1 --2- 172.21.15.139:0/1410492800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6080071600 0x7f60800c6790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f608006ea70 tx=0x7f6078032a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e6bfba5ccfd7350 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7fe2b1124640 1 Processor -- start 2024-09-16T00:11:14.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f1d627fc640 1 -- 172.21.15.139:0/1056413808 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50037070 con 0x7f1d5c045120 2024-09-16T00:11:14.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f3e572aa640 1 --2- 172.21.15.139:0/1739453537 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e500c81b0 0x7f3e50162d20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f3e572aa640 1 --2- 172.21.15.139:0/1739453537 >> v2:172.21.15.139:3300/0 conn(0x7f3e500ca730 0x7f3e50158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f3e572aa640 1 -- 172.21.15.139:0/1739453537 >> 172.21.15.139:0/1739453537 conn(0x7f3e5006bb20 msgr2=0x7f3e5006bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f1d627fc640 1 -- 172.21.15.139:0/1056413808 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d50032440 con 0x7f1d5c045120 2024-09-16T00:11:14.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f3e572aa640 1 -- 172.21.15.139:0/1739453537 shutdown_connections 2024-09-16T00:11:14.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7fe2b1124640 1 -- start start 2024-09-16T00:11:14.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f3e572aa640 1 -- 172.21.15.139:0/1739453537 wait complete. 2024-09-16T00:11:14.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7f9b4f042640 1 -- 172.21.15.139:0/4213453310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b480cb420 msgr2=0x7f9b480c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7f9b4f042640 1 --2- 172.21.15.139:0/4213453310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b480cb420 0x7f9b480c8fc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9b480cce30 tx=0x7f9b3802a1a0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7f607f7fe640 1 -- 172.21.15.139:0/1410492800 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078036830 con 0x7f6080071600 2024-09-16T00:11:14.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7f607f7fe640 1 -- 172.21.15.139:0/1410492800 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60780369d0 con 0x7f6080071600 2024-09-16T00:11:14.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7f607f7fe640 1 -- 172.21.15.139:0/1410492800 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078036da0 con 0x7f6080071600 2024-09-16T00:11:14.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7f1d627fc640 1 -- 172.21.15.139:0/1056413808 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50032850 con 0x7f1d5c045120 2024-09-16T00:11:14.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.735+0000 7f506e6d0640 1 -- 172.21.15.139:0/3658843786 wait complete. 2024-09-16T00:11:14.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7fe2a1ffb640 1 -- 172.21.15.139:0/40346832 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe294032440 con 0x7fe2a4151c30 2024-09-16T00:11:14.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7fe2a1ffb640 1 -- 172.21.15.139:0/40346832 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe294032810 con 0x7fe2a4151c30 2024-09-16T00:11:14.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7f9b4f042640 1 -- 172.21.15.139:0/4213453310 shutdown_connections 2024-09-16T00:11:14.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7f9b4f042640 1 --2- 172.21.15.139:0/4213453310 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b480cb420 0x7f9b480c8fc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7f9b4f042640 1 --2- 172.21.15.139:0/4213453310 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b480cafe0 0x7f9b480c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7f9b4f042640 1 -- 172.21.15.139:0/4213453310 >> 172.21.15.139:0/4213453310 conn(0x7f9b4806e270 msgr2=0x7f9b4806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7f1d6a00a640 1 -- 172.21.15.139:0/1056413808 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d5c045120 msgr2=0x7f1d5c0d48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7f1d6a00a640 1 --2- 172.21.15.139:0/1056413808 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d5c045120 0x7f1d5c0d48e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009870 tx=0x7f1d5002a2c0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7feaa5f86640 1 Processor -- start 2024-09-16T00:11:14.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7f506e6d0640 1 Processor -- start 2024-09-16T00:11:14.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7feaa5f86640 1 -- start start 2024-09-16T00:11:14.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7fe2b1124640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2ac0c6fe0 0x7fe2ac06e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7fe2b1124640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe2ac157e20 0x7fe2ac0b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7f3e572aa640 1 Processor -- start 2024-09-16T00:11:14.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7f6085d9d640 1 -- 172.21.15.139:0/1410492800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6080071600 msgr2=0x7f60800c6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7fe2abfff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2ac0c6fe0 0x7fe2ac06e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7f6085d9d640 1 --2- 172.21.15.139:0/1410492800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6080071600 0x7f60800c6790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f608006ea70 tx=0x7f6078032a70 comp rx=0 tx=0).stop 2024-09-16T00:11:14.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7fe2b1124640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2ac06e590 con 0x7fe2ac073350 2024-09-16T00:11:14.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7fe2ab7fe640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fe2ac073350 0x7fe2ac0c73c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:44842/0 (socket says 172.21.15.139:44842) 2024-09-16T00:11:14.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7f3e572aa640 1 -- start start 2024-09-16T00:11:14.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7fe2ab7fe640 1 -- 172.21.15.139:0/656535330 learned_addr learned my addr 172.21.15.139:0/656535330 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:14.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7f6085d9d640 1 -- 172.21.15.139:0/1410492800 shutdown_connections 2024-09-16T00:11:14.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7f6085d9d640 1 --2- 172.21.15.139:0/1410492800 >> v2:172.21.15.164:3300/0 conn(0x7f608006fbc0 0x7f608006ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7f6085d9d640 1 --2- 172.21.15.139:0/1410492800 >> v2:172.21.15.139:3300/0 conn(0x7f608015a6b0 0x7f60800c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7f6085d9d640 1 --2- 172.21.15.139:0/1410492800 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6080071600 0x7f60800c6790 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.766 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7f6085d9d640 1 -- 172.21.15.139:0/1410492800 >> 172.21.15.139:0/1410492800 conn(0x7f60800cbe90 msgr2=0x7f60800cc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7fe2a98c9640 1 -- 172.21.15.139:0/40346832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2a4151c30 msgr2=0x7fe2a415ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7fe2a98c9640 1 --2- 172.21.15.139:0/40346832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2a4151c30 0x7fe2a415ac10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4075c60 tx=0x7fe29402a1f0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7fe2b0923640 1 --2- 172.21.15.139:0/656535330 >> v2:172.21.15.164:3300/0 conn(0x7fe2ac157e20 0x7fe2ac0b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7fe2b1124640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fe2ac06ccd0 con 0x7fe2ac157e20 2024-09-16T00:11:14.767 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7fe2b1124640 1 -- 172.21.15.139:0/656535330 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac06cfb0 con 0x7fe2ac0c6fe0 2024-09-16T00:11:14.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2a98c9640 1 -- 172.21.15.139:0/40346832 shutdown_connections 2024-09-16T00:11:14.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2aaffd640 1 -- 172.21.15.139:0/656535330 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2922695680 0 0) 0x7fe2ac06e590 con 0x7fe2ac073350 2024-09-16T00:11:14.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2a98c9640 1 --2- 172.21.15.139:0/40346832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2a4151c30 0x7fe2a415ac10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2a98c9640 1 --2- 172.21.15.139:0/40346832 >> v2:172.21.15.164:3300/0 conn(0x7fe2a4151850 0x7fe2a40c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2a98c9640 1 -- 172.21.15.139:0/40346832 >> 172.21.15.139:0/40346832 conn(0x7fe2a406cc30 msgr2=0x7fe2a406d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2aaffd640 1 -- 172.21.15.139:0/656535330 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe294003520 con 0x7fe2ac073350 2024-09-16T00:11:14.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7f9b4f042640 1 -- 172.21.15.139:0/4213453310 shutdown_connections 2024-09-16T00:11:14.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.736+0000 7f9b4f042640 1 -- 172.21.15.139:0/4213453310 wait complete. 2024-09-16T00:11:14.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7feaa5f86640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feaa006fc20 0x7feaa01f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7feaa5f86640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa0070040 0x7feaa00c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7feaa5f86640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaa0158f00 0x7feaa00cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7feaa5f86640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feaa00cca20 con 0x7feaa0070040 2024-09-16T00:11:14.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7feaa5f86640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7feaa00cc8a0 con 0x7feaa0158f00 2024-09-16T00:11:14.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7feaa5f86640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7feaa01592e0 con 0x7feaa006fc20 2024-09-16T00:11:14.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f1d6a00a640 1 -- 172.21.15.139:0/1056413808 shutdown_connections 2024-09-16T00:11:14.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f1d6a00a640 1 --2- 172.21.15.139:0/1056413808 >> v2:172.21.15.164:3300/0 conn(0x7f1d5c00de10 0x7f1d5c005a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f1d6a00a640 1 --2- 172.21.15.139:0/1056413808 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d5c045120 0x7f1d5c0d48e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f1d6a00a640 1 -- 172.21.15.139:0/1056413808 >> 172.21.15.139:0/1056413808 conn(0x7f1d5c04fc80 msgr2=0x7f1d5c050090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2aaffd640 1 -- 172.21.15.139:0/656535330 <== mon.1 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3263899777 0 0) 0x7fe294003520 con 0x7fe2ac073350 2024-09-16T00:11:14.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2aaffd640 1 -- 172.21.15.139:0/656535330 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2ac0c6fe0 msgr2=0x7fe2ac06e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:14.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2aaffd640 1 --2- 172.21.15.139:0/656535330 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2ac0c6fe0 0x7fe2ac06e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2aaffd640 1 -- 172.21.15.139:0/656535330 >> v2:172.21.15.164:3300/0 conn(0x7fe2ac157e20 msgr2=0x7fe2ac0b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:14.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2aaffd640 1 --2- 172.21.15.139:0/656535330 >> v2:172.21.15.164:3300/0 conn(0x7fe2ac157e20 0x7fe2ac0b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2aaffd640 1 -- 172.21.15.139:0/656535330 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac06d290 con 0x7fe2ac073350 2024-09-16T00:11:14.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2aaffd640 1 -- 172.21.15.139:0/656535330 <== mon.1 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7fe2a0001670 con 0x7fe2ac073350 2024-09-16T00:11:14.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f1d6a00a640 1 -- 172.21.15.139:0/1056413808 shutdown_connections 2024-09-16T00:11:14.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2aaffd640 1 -- 172.21.15.139:0/656535330 >> v1:172.21.15.164:6789/0 conn(0x7fe2ac073350 legacy=0x7fe2ac0c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2aaffd640 1 --2- 172.21.15.139:0/656535330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2940039d0 0x7fe294003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fea9effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa0070040 0x7feaa00c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fea9effd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa0070040 0x7feaa00c44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60312/0 (socket says 172.21.15.139:60312) 2024-09-16T00:11:14.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fea9effd640 1 -- 172.21.15.139:0/1603190478 learned_addr learned my addr 172.21.15.139:0/1603190478 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:14.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2aaffd640 1 --2- 172.21.15.139:0/656535330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe294004350 0x7fe294004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f9b4f042640 1 Processor -- start 2024-09-16T00:11:14.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.737+0000 7f506e6d0640 1 -- start start 2024-09-16T00:11:14.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f506e6d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5068067950 0x7f50680ba110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f506e6d0640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5068067d60 0x7f5068154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f506e6d0640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50680c36b0 0x7f5068155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f506e6d0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f50680cb420 con 0x7f5068067950 2024-09-16T00:11:14.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f506e6d0640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f50680cb100 con 0x7f5068067d60 2024-09-16T00:11:14.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f506e6d0640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f506813cb00 con 0x7f50680c36b0 2024-09-16T00:11:14.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2a98c9640 1 -- 172.21.15.139:0/40346832 shutdown_connections 2024-09-16T00:11:14.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f6085d9d640 1 -- 172.21.15.139:0/1410492800 shutdown_connections 2024-09-16T00:11:14.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fea9ffff640 1 --2- 172.21.15.139:0/1603190478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaa0158f00 0x7feaa00cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.775 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2aaffd640 1 --2- 172.21.15.139:0/656535330 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe294004d40 0x7fe2940051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fea9f7fe640 1 --2- 172.21.15.139:0/1603190478 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feaa006fc20 0x7feaa01f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2aaffd640 1 -- 172.21.15.139:0/656535330 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac06ccd0 con 0x7fe294004350 2024-09-16T00:11:14.776 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2aaffd640 1 -- 172.21.15.139:0/656535330 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac06cfb0 con 0x7fe2940039d0 2024-09-16T00:11:14.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f3e572aa640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e50071290 0x7f3e501f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7fe2aaffd640 1 -- 172.21.15.139:0/656535330 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe294010290 con 0x7fe294004d40 2024-09-16T00:11:14.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7fe2b0923640 1 --2- 172.21.15.139:0/656535330 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe294004d40 0x7fe2940051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.777 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f506cc6d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50680c36b0 0x7f5068155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f506cc6d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50680c36b0 0x7f5068155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:58316/0 (socket says 172.21.15.139:58316) 2024-09-16T00:11:14.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f506cc6d640 1 -- 172.21.15.139:0/3730141799 learned_addr learned my addr 172.21.15.139:0/3730141799 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:14.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f5067fff640 1 --2- 172.21.15.139:0/3730141799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5068067950 0x7f50680ba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f6085d9d640 1 -- 172.21.15.139:0/1410492800 wait complete. 2024-09-16T00:11:14.778 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f3e572aa640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e500c81b0 0x7f3e501ec490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f1d6a00a640 1 -- 172.21.15.139:0/1056413808 wait complete. 2024-09-16T00:11:14.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7fe2a98c9640 1 -- 172.21.15.139:0/40346832 wait complete. 2024-09-16T00:11:14.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7fea9effd640 1 -- 172.21.15.139:0/1603190478 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feaa006fc20 msgr2=0x7feaa01f7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7fea9effd640 1 --2- 172.21.15.139:0/1603190478 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7feaa006fc20 0x7feaa01f7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7fea9effd640 1 -- 172.21.15.139:0/1603190478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaa0158f00 msgr2=0x7feaa00cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7fea9effd640 1 --2- 172.21.15.139:0/1603190478 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7feaa0158f00 0x7feaa00cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.779 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7fea9effd640 1 -- 172.21.15.139:0/1603190478 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa0155ea0 con 0x7feaa0070040 2024-09-16T00:11:14.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7fe2abfff640 1 --2- 172.21.15.139:0/656535330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2940039d0 0x7fe294003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f3e572aa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e500ca730 0x7f3e5014d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.780 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f3e572aa640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3e500c4f50 con 0x7f3e500ca730 2024-09-16T00:11:14.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f3e572aa640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3e500c4c70 con 0x7f3e50071290 2024-09-16T00:11:14.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f3e572aa640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3e50158670 con 0x7f3e500c81b0 2024-09-16T00:11:14.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f3e562a8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e50071290 0x7f3e501f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f3e562a8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e50071290 0x7f3e501f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39944/0 (socket says 172.21.15.139:39944) 2024-09-16T00:11:14.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f3e562a8640 1 -- 172.21.15.139:0/1404189891 learned_addr learned my addr 172.21.15.139:0/1404189891 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:14.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f3e56aa9640 1 --2- 172.21.15.139:0/1404189891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e500ca730 0x7f3e5014d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f3e55aa7640 1 --2- 172.21.15.139:0/1404189891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e500c81b0 0x7f3e501ec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7fe2abfff640 1 -- 172.21.15.139:0/656535330 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe294004d40 msgr2=0x7fe2940051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:14.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7fe2abfff640 1 --2- 172.21.15.139:0/656535330 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe294004d40 0x7fe2940051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7fe2abfff640 1 -- 172.21.15.139:0/656535330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe294004350 msgr2=0x7fe294004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:14.781 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7fe2abfff640 1 --2- 172.21.15.139:0/656535330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe294004350 0x7fe294004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7fe2abfff640 1 -- 172.21.15.139:0/656535330 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac06d290 con 0x7fe2940039d0 2024-09-16T00:11:14.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f3e562a8640 1 -- 172.21.15.139:0/1404189891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e500c81b0 msgr2=0x7f3e501ec490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f3e562a8640 1 --2- 172.21.15.139:0/1404189891 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3e500c81b0 0x7f3e501ec490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f3e562a8640 1 -- 172.21.15.139:0/1404189891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e500ca730 msgr2=0x7f3e5014d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f3e562a8640 1 --2- 172.21.15.139:0/1404189891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3e500ca730 0x7f3e5014d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f3e562a8640 1 -- 172.21.15.139:0/1404189891 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e50155a70 con 0x7f3e50071290 2024-09-16T00:11:14.782 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7fea9effd640 1 --2- 172.21.15.139:0/1603190478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa0070040 0x7feaa00c44f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fea9800c790 tx=0x7fea9800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7f3e562a8640 1 --2- 172.21.15.139:0/1404189891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e50071290 0x7f3e501f0580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00c820 tx=0x7f3e4c00cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7f50677fe640 1 --2- 172.21.15.139:0/3730141799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5068067d60 0x7f5068154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.783 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.738+0000 7f9b4f042640 1 -- start start 2024-09-16T00:11:14.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f9b4f042640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b480cac00 0x7f9b480c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f9b4f042640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b480cafe0 0x7f9b480c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f9b4f042640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b480cb420 0x7f9b4815ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f9b4f042640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9b480cc940 con 0x7f9b480cac00 2024-09-16T00:11:14.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f9b4f042640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9b480cc660 con 0x7f9b480cafe0 2024-09-16T00:11:14.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.739+0000 7f9b4f042640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9b480c7410 con 0x7f9b480cb420 2024-09-16T00:11:14.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7f9b4e040640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b480cac00 0x7f9b480c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.784 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7f3e46ffd640 1 -- 172.21.15.139:0/1404189891 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c015070 con 0x7f3e50071290 2024-09-16T00:11:14.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7f9b4e040640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b480cac00 0x7f9b480c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60362/0 (socket says 172.21.15.139:60362) 2024-09-16T00:11:14.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7f9b4e040640 1 -- 172.21.15.139:0/1694397569 learned_addr learned my addr 172.21.15.139:0/1694397569 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:14.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7f6085d9d640 1 Processor -- start 2024-09-16T00:11:14.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7f6085d9d640 1 -- start start 2024-09-16T00:11:14.785 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7fea87fff640 1 -- 172.21.15.139:0/1603190478 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea98003b70 con 0x7feaa0070040 2024-09-16T00:11:14.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7feaa5f86640 1 -- 172.21.15.139:0/1603190478 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa0156140 con 0x7feaa0070040 2024-09-16T00:11:14.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7f3e572aa640 1 -- 172.21.15.139:0/1404189891 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e500c5a00 con 0x7f3e50071290 2024-09-16T00:11:14.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7feaa5f86640 1 -- 172.21.15.139:0/1603190478 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa015b5f0 con 0x7feaa0070040 2024-09-16T00:11:14.786 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7f3e572aa640 1 -- 172.21.15.139:0/1404189891 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e501efd80 con 0x7f3e50071290 2024-09-16T00:11:14.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7fea87fff640 1 -- 172.21.15.139:0/1603190478 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea98003d30 con 0x7feaa0070040 2024-09-16T00:11:14.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7f9b4d83f640 1 --2- 172.21.15.139:0/1694397569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b480cafe0 0x7f9b480c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7f1d6a00a640 1 Processor -- start 2024-09-16T00:11:14.787 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7f1d6a00a640 1 -- start start 2024-09-16T00:11:14.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7fe2a98c9640 1 Processor -- start 2024-09-16T00:11:14.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7f506cc6d640 1 -- 172.21.15.139:0/3730141799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5068067d60 msgr2=0x7f5068154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7f506cc6d640 1 --2- 172.21.15.139:0/3730141799 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f5068067d60 0x7f5068154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7f506cc6d640 1 -- 172.21.15.139:0/3730141799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5068067950 msgr2=0x7f50680ba110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7f506cc6d640 1 --2- 172.21.15.139:0/3730141799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5068067950 0x7f50680ba110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.788 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7f506cc6d640 1 -- 172.21.15.139:0/3730141799 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5068156bb0 con 0x7f50680c36b0 2024-09-16T00:11:14.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7f9b4e040640 1 -- 172.21.15.139:0/1694397569 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b480cb420 msgr2=0x7f9b4815ab40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:11:14.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7f9b4e040640 1 --2- 172.21.15.139:0/1694397569 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9b480cb420 0x7f9b4815ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7f9b4e040640 1 -- 172.21.15.139:0/1694397569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b480cafe0 msgr2=0x7f9b480c8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7f9b4e040640 1 --2- 172.21.15.139:0/1694397569 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9b480cafe0 0x7f9b480c8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7f9b4e040640 1 -- 172.21.15.139:0/1694397569 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4815dbd0 con 0x7f9b480cac00 2024-09-16T00:11:14.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7f506cc6d640 1 --2- 172.21.15.139:0/3730141799 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50680c36b0 0x7f5068155110 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f506000eed0 tx=0x7f506000b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.789 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7f1d6a00a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d5c00de10 0x7f1d5c0ec020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7f6085d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f608006fbc0 0x7f6080151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7f5064ff9640 1 -- 172.21.15.139:0/3730141799 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060018020 con 0x7f50680c36b0 2024-09-16T00:11:14.790 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7fea87fff640 1 -- 172.21.15.139:0/1603190478 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9800aa90 con 0x7feaa0070040 2024-09-16T00:11:14.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f6085d9d640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6080071600 0x7f6080156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f5064ff9640 1 -- 172.21.15.139:0/3730141799 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5060014400 con 0x7f50680c36b0 2024-09-16T00:11:14.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f6085d9d640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f608015a6b0 0x7f60801ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f9b4e040640 1 --2- 172.21.15.139:0/1694397569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b480cac00 0x7f9b480c7bd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400c7d0 tx=0x7f9b4400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f6085d9d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f60800c5dc0 con 0x7f608006fbc0 2024-09-16T00:11:14.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f6085d9d640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f608006d710 con 0x7f6080071600 2024-09-16T00:11:14.792 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f6085d9d640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f60800c5f40 con 0x7f608015a6b0 2024-09-16T00:11:14.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f5064ff9640 1 -- 172.21.15.139:0/3730141799 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50600147f0 con 0x7f50680c36b0 2024-09-16T00:11:14.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f506e6d0640 1 -- 172.21.15.139:0/3730141799 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5068157220 con 0x7f50680c36b0 2024-09-16T00:11:14.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f506e6d0640 1 -- 172.21.15.139:0/3730141799 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f50680b9050 con 0x7f50680c36b0 2024-09-16T00:11:14.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f608559c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f608015a6b0 0x7f60801ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f608559c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f608015a6b0 0x7f60801ec3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:58362/0 (socket says 172.21.15.139:58362) 2024-09-16T00:11:14.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f608559c640 1 -- 172.21.15.139:0/2333378253 learned_addr learned my addr 172.21.15.139:0/2333378253 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:14.793 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f607ffff640 1 --2- 172.21.15.139:0/2333378253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6080071600 0x7f6080156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f9b3effd640 1 -- 172.21.15.139:0/1694397569 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44018070 con 0x7f9b480cac00 2024-09-16T00:11:14.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f9b3effd640 1 -- 172.21.15.139:0/1694397569 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b440074e0 con 0x7f9b480cac00 2024-09-16T00:11:14.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f9b3effd640 1 -- 172.21.15.139:0/1694397569 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44004a20 con 0x7f9b480cac00 2024-09-16T00:11:14.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f9b4f042640 1 -- 172.21.15.139:0/1694397569 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b481f4f70 con 0x7f9b480cac00 2024-09-16T00:11:14.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f9b4f042640 1 -- 172.21.15.139:0/1694397569 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b481f5250 con 0x7f9b480cac00 2024-09-16T00:11:14.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f3e46ffd640 1 -- 172.21.15.139:0/1404189891 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e4c004020 con 0x7f3e50071290 2024-09-16T00:11:14.795 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7fe2a98c9640 1 -- start start 2024-09-16T00:11:14.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f608559c640 1 -- 172.21.15.139:0/2333378253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6080071600 msgr2=0x7f6080156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f608559c640 1 --2- 172.21.15.139:0/2333378253 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6080071600 0x7f6080156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f608559c640 1 -- 172.21.15.139:0/2333378253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f608006fbc0 msgr2=0x7f6080151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f608559c640 1 --2- 172.21.15.139:0/2333378253 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f608006fbc0 0x7f6080151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.742+0000 7f608559c640 1 -- 172.21.15.139:0/2333378253 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6080153920 con 0x7f608015a6b0 2024-09-16T00:11:14.796 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7f3e46ffd640 1 -- 172.21.15.139:0/1404189891 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c0043f0 con 0x7f3e50071290 2024-09-16T00:11:14.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.743+0000 7f506e6d0640 1 -- 172.21.15.139:0/3730141799 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5068157740 con 0x7f50680c36b0 2024-09-16T00:11:14.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7f9b4f042640 1 -- 172.21.15.139:0/1694397569 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b480cba50 con 0x7f9b480cac00 2024-09-16T00:11:14.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a98c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2a40c7390 0x7fe2a4153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a98c9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2a4151850 0x7fe2a4156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.797 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a98c9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2a4151c30 0x7fe2a414bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.743+0000 7f608559c640 1 --2- 172.21.15.139:0/2333378253 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f608015a6b0 0x7f60801ec3a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f607000efc0 tx=0x7f6070009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.743+0000 7f607d7fa640 1 -- 172.21.15.139:0/2333378253 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070018030 con 0x7f608015a6b0 2024-09-16T00:11:14.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a98c9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a40c8530 con 0x7fe2a40c7390 2024-09-16T00:11:14.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a98c9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a40c8250 con 0x7fe2a4151850 2024-09-16T00:11:14.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a98c9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a40c96c0 con 0x7fe2a4151c30 2024-09-16T00:11:14.798 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.743+0000 7f607d7fa640 1 -- 172.21.15.139:0/2333378253 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f607000cde0 con 0x7f608015a6b0 2024-09-16T00:11:14.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.743+0000 7f607d7fa640 1 -- 172.21.15.139:0/2333378253 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070003bb0 con 0x7f608015a6b0 2024-09-16T00:11:14.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.743+0000 7f6085d9d640 1 -- 172.21.15.139:0/2333378253 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6080068e20 con 0x7f608015a6b0 2024-09-16T00:11:14.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.743+0000 7f6085d9d640 1 -- 172.21.15.139:0/2333378253 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f608014de90 con 0x7f608015a6b0 2024-09-16T00:11:14.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7fe2abfff640 1 --2- 172.21.15.139:0/656535330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2940039d0 0x7fe294003e10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac071050 tx=0x7fe294005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec1d7430d2fc6d7f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7fe2aaffd640 1 -- 172.21.15.139:0/656535330 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c036070 con 0x7fe2940039d0 2024-09-16T00:11:14.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7fe2aaffd640 1 -- 172.21.15.139:0/656535330 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29c031400 con 0x7fe2940039d0 2024-09-16T00:11:14.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.740+0000 7fe2aaffd640 1 -- 172.21.15.139:0/656535330 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c009650 con 0x7fe2940039d0 2024-09-16T00:11:14.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7fe2b1124640 1 -- 172.21.15.139:0/656535330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2940039d0 msgr2=0x7fe294003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.799 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.741+0000 7fe2b1124640 1 --2- 172.21.15.139:0/656535330 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2940039d0 0x7fe294003e10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac071050 tx=0x7fe294005830 comp rx=0 tx=0).stop 2024-09-16T00:11:14.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2a4151850 0x7fe2a4156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2a40c7390 0x7fe2a4153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a27fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2a4151850 0x7fe2a4156030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:39976/0 (socket says 172.21.15.139:39976) 2024-09-16T00:11:14.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2a40c7390 0x7fe2a4153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60392/0 (socket says 172.21.15.139:60392) 2024-09-16T00:11:14.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a2ffd640 1 -- 172.21.15.139:0/2608939809 learned_addr learned my addr 172.21.15.139:0/2608939809 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:14.800 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a37fe640 1 --2- 172.21.15.139:0/2608939809 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2a4151c30 0x7fe2a414bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a2ffd640 1 -- 172.21.15.139:0/2608939809 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2a4151c30 msgr2=0x7fe2a414bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a2ffd640 1 --2- 172.21.15.139:0/2608939809 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2a4151c30 0x7fe2a414bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a2ffd640 1 -- 172.21.15.139:0/2608939809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2a4151850 msgr2=0x7fe2a4156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a2ffd640 1 --2- 172.21.15.139:0/2608939809 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2a4151850 0x7fe2a4156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.744+0000 7fe2a2ffd640 1 -- 172.21.15.139:0/2608939809 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a4156d40 con 0x7fe2a40c7390 2024-09-16T00:11:14.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f6085d9d640 1 -- 172.21.15.139:0/2333378253 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f608014d960 con 0x7f608015a6b0 2024-09-16T00:11:14.801 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7fe2a2ffd640 1 --2- 172.21.15.139:0/2608939809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2a40c7390 0x7fe2a4153060 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe29800be50 tx=0x7fe298010df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7fe2b0923640 1 -- 172.21.15.139:0/656535330 reap_dead start 2024-09-16T00:11:14.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7fe2b1124640 1 -- 172.21.15.139:0/656535330 shutdown_connections 2024-09-16T00:11:14.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7fe2b1124640 1 -- 172.21.15.139:0/656535330 >> 172.21.15.139:0/656535330 conn(0x7fe2ac0c8a00 msgr2=0x7fe2ac0c8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7fea87fff640 1 -- 172.21.15.139:0/1603190478 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fea9800ac30 con 0x7feaa0070040 2024-09-16T00:11:14.802 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f5064ff9640 1 -- 172.21.15.139:0/3730141799 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5060005d10 con 0x7f50680c36b0 2024-09-16T00:11:14.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7fe28bfff640 1 -- 172.21.15.139:0/2608939809 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29800ecf0 con 0x7fe2a40c7390 2024-09-16T00:11:14.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7fe28bfff640 1 -- 172.21.15.139:0/2608939809 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe298022400 con 0x7fe2a40c7390 2024-09-16T00:11:14.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7fe28bfff640 1 -- 172.21.15.139:0/2608939809 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2980227b0 con 0x7fe2a40c7390 2024-09-16T00:11:14.803 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7fe2b1124640 1 -- 172.21.15.139:0/656535330 shutdown_connections 2024-09-16T00:11:14.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7fe2b1124640 1 -- 172.21.15.139:0/656535330 wait complete. 2024-09-16T00:11:14.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7fe2a98c9640 1 -- 172.21.15.139:0/2608939809 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a41ed150 con 0x7fe2a40c7390 2024-09-16T00:11:14.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7fe2a98c9640 1 -- 172.21.15.139:0/2608939809 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a4157290 con 0x7fe2a40c7390 2024-09-16T00:11:14.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f9b3effd640 1 -- 172.21.15.139:0/1694397569 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9b44004bc0 con 0x7f9b480cac00 2024-09-16T00:11:14.804 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f607d7fa640 1 -- 172.21.15.139:0/2333378253 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6070005d10 con 0x7f608015a6b0 2024-09-16T00:11:14.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f5064ff9640 1 --2- 172.21.15.139:0/3730141799 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f505406c2b0 0x7f505406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f9b3effd640 1 --2- 172.21.15.139:0/1694397569 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9b2c06c2e0 0x7f9b2c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7feaa5f86640 1 -- 172.21.15.139:0/1603190478 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea80001030 con 0x7feaa0070040 2024-09-16T00:11:14.805 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f1d6a00a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d5c045120 0x7f1d5c0e9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f9b3effd640 1 -- 172.21.15.139:0/1694397569 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f9b4408afa0 con 0x7f9b480cac00 2024-09-16T00:11:14.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f9b3effd640 1 --2- 172.21.15.139:0/1694397569 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f9b2c0748c0 0x7f9b2c076d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f1d6a00a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d5c04c0c0 0x7f1d5c04e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f9b3effd640 1 -- 172.21.15.139:0/1694397569 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b2c077430 con 0x7f9b2c0748c0 2024-09-16T00:11:14.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f9b3effd640 1 -- 172.21.15.139:0/1694397569 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7f9b4408b390 con 0x7f9b480cac00 2024-09-16T00:11:14.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f9b4d83f640 1 --2- 172.21.15.139:0/1694397569 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9b2c06c2e0 0x7f9b2c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f1d6a00a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c0464c0 con 0x7f1d5c00de10 2024-09-16T00:11:14.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f1d6a00a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c0461e0 con 0x7f1d5c04c0c0 2024-09-16T00:11:14.807 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f607d7fa640 1 --2- 172.21.15.139:0/2333378253 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f606006c2b0 0x7f606006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f1d6a00a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f1d5c0467a0 con 0x7f1d5c045120 2024-09-16T00:11:14.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f1d637fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d5c00de10 0x7f1d5c0ec020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f1d637fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d5c00de10 0x7f1d5c0ec020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60370/0 (socket says 172.21.15.139:60370) 2024-09-16T00:11:14.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f1d637fe640 1 -- 172.21.15.139:0/3703458782 learned_addr learned my addr 172.21.15.139:0/3703458782 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:14.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f607d7fa640 1 -- 172.21.15.139:0/2333378253 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f607008bf90 con 0x7f608015a6b0 2024-09-16T00:11:14.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7fea87fff640 1 --2- 172.21.15.139:0/1603190478 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fea8c06c2b0 0x7fea8c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f1d62ffd640 1 --2- 172.21.15.139:0/3703458782 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d5c045120 0x7f1d5c0e9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f1d637fe640 1 -- 172.21.15.139:0/3703458782 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d5c045120 msgr2=0x7f1d5c0e9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f1d637fe640 1 --2- 172.21.15.139:0/3703458782 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1d5c045120 0x7f1d5c0e9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f1d637fe640 1 -- 172.21.15.139:0/3703458782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d5c04c0c0 msgr2=0x7f1d5c04e4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:11:14.809 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f6084d9b640 1 --2- 172.21.15.139:0/2333378253 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f606006c2b0 0x7f606006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f1d637fe640 1 --2- 172.21.15.139:0/3703458782 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1d5c04c0c0 0x7f1d5c04e4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f1d637fe640 1 -- 172.21.15.139:0/3703458782 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5c0ea220 con 0x7f1d5c00de10 2024-09-16T00:11:14.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f1d637fe640 1 --2- 172.21.15.139:0/3703458782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d5c00de10 0x7f1d5c0ec020 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009870 tx=0x7f1d500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f1d4ffff640 1 -- 172.21.15.139:0/3703458782 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50041020 con 0x7f1d5c00de10 2024-09-16T00:11:14.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f607d7fa640 1 --2- 172.21.15.139:0/2333378253 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f6060074890 0x7f6060076d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f1d4ffff640 1 -- 172.21.15.139:0/3703458782 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d5003d6e0 con 0x7f1d5c00de10 2024-09-16T00:11:14.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f1d4ffff640 1 -- 172.21.15.139:0/3703458782 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50037040 con 0x7f1d5c00de10 2024-09-16T00:11:14.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.746+0000 7f1d6a00a640 1 -- 172.21.15.139:0/3703458782 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d5c17cbd0 con 0x7f1d5c00de10 2024-09-16T00:11:14.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.747+0000 7f607d7fa640 1 -- 172.21.15.139:0/2333378253 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f607000be50 con 0x7f6060074890 2024-09-16T00:11:14.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.747+0000 7f607d7fa640 1 -- 172.21.15.139:0/2333378253 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7f6070091560 con 0x7f608015a6b0 2024-09-16T00:11:14.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.747+0000 7f1d6a00a640 1 -- 172.21.15.139:0/3703458782 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d5c0e9a00 con 0x7f1d5c00de10 2024-09-16T00:11:14.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.747+0000 7fea87fff640 1 -- 172.21.15.139:0/1603190478 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7fea9808b340 con 0x7feaa0070040 2024-09-16T00:11:14.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.747+0000 7fea9f7fe640 1 --2- 172.21.15.139:0/1603190478 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fea8c06c2b0 0x7fea8c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.747+0000 7fea87fff640 1 --2- 172.21.15.139:0/1603190478 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7fea8c074810 0x7fea8c076cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.747+0000 7fea87fff640 1 -- 172.21.15.139:0/1603190478 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea980079d0 con 0x7fea8c074810 2024-09-16T00:11:14.812 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.747+0000 7fea87fff640 1 -- 172.21.15.139:0/1603190478 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7fea98092050 con 0x7feaa0070040 2024-09-16T00:11:14.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.747+0000 7f607ffff640 1 --2- 172.21.15.139:0/2333378253 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f6060074890 0x7f6060076d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.747+0000 7fe2b1124640 1 Processor -- start 2024-09-16T00:11:14.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.747+0000 7fe2b1124640 1 -- start start 2024-09-16T00:11:14.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.747+0000 7f6084d9b640 1 --2- 172.21.15.139:0/2333378253 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f606006c2b0 0x7f606006e770 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f607803d000 tx=0x7f6078037000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.813 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.747+0000 7f607ffff640 1 --2- 172.21.15.139:0/2333378253 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f6060074890 0x7f6060076d30 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7fea9f7fe640 1 --2- 172.21.15.139:0/1603190478 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fea8c06c2b0 0x7fea8c06e770 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fea90007fd0 tx=0x7fea90007e80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.747+0000 7f9b4e841640 1 --2- 172.21.15.139:0/1694397569 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f9b2c0748c0 0x7f9b2c076d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7f9b4d83f640 1 --2- 172.21.15.139:0/1694397569 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9b2c06c2e0 0x7f9b2c06e7a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9b480c6c90 tx=0x7f9b400098d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7f9b4e841640 1 --2- 172.21.15.139:0/1694397569 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f9b2c0748c0 0x7f9b2c076d60 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7fe2b1124640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2ac1465f0 0x7fe2ac1f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7fe2b1124640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2ac14a650 0x7fe2ac14ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7fe2b1124640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2ac1f5540 0x7fe2ac14f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7fe2b1124640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac0c7ad0 con 0x7fe2ac1f5540 2024-09-16T00:11:14.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7fe2b1124640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac06d850 con 0x7fe2ac1465f0 2024-09-16T00:11:14.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7fe2b1124640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac0c6800 con 0x7fe2ac14a650 2024-09-16T00:11:14.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7fea9ffff640 1 --2- 172.21.15.139:0/1603190478 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7fea8c074810 0x7fea8c076cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.745+0000 7f3e572aa640 1 -- 172.21.15.139:0/1404189891 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3e38001030 con 0x7f3e50071290 2024-09-16T00:11:14.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.747+0000 7f3e46ffd640 1 -- 172.21.15.139:0/1404189891 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3e4c007830 con 0x7f3e50071290 2024-09-16T00:11:14.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7fe2b0923640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2ac1f5540 0x7fe2ac14f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7fe2b0923640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2ac1f5540 0x7fe2ac14f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60402/0 (socket says 172.21.15.139:60402) 2024-09-16T00:11:14.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7fe2b0923640 1 -- 172.21.15.139:0/2188819 learned_addr learned my addr 172.21.15.139:0/2188819 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:14.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7fe2ab7fe640 1 --2- 172.21.15.139:0/2188819 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2ac14a650 0x7fe2ac14ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7f3e46ffd640 1 --2- 172.21.15.139:0/1404189891 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3e3406c400 0x7f3e3406e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7fe2abfff640 1 --2- 172.21.15.139:0/2188819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2ac1465f0 0x7fe2ac1f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7f3e55aa7640 1 --2- 172.21.15.139:0/1404189891 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3e3406c400 0x7f3e3406e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7f3e46ffd640 1 -- 172.21.15.139:0/1404189891 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f3e4c08b3a0 con 0x7f3e50071290 2024-09-16T00:11:14.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7f3e46ffd640 1 --2- 172.21.15.139:0/1404189891 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f3e340749e0 0x7f3e34076e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.818 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7f3e46ffd640 1 -- 172.21.15.139:0/1404189891 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3e4c01ae50 con 0x7f3e340749e0 2024-09-16T00:11:14.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7fe2b0923640 1 -- 172.21.15.139:0/2188819 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2ac14a650 msgr2=0x7fe2ac14ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7fe2b0923640 1 --2- 172.21.15.139:0/2188819 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe2ac14a650 0x7fe2ac14ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7fe2b0923640 1 -- 172.21.15.139:0/2188819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2ac1465f0 msgr2=0x7fe2ac1f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7fe2b0923640 1 --2- 172.21.15.139:0/2188819 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe2ac1465f0 0x7fe2ac1f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7fe2b0923640 1 -- 172.21.15.139:0/2188819 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac152e10 con 0x7fe2ac1f5540 2024-09-16T00:11:14.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7f9b3effd640 1 -- 172.21.15.139:0/1694397569 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b2c077430 con 0x7f9b2c0748c0 2024-09-16T00:11:14.819 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7f607d7fa640 1 -- 172.21.15.139:0/2333378253 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f607000be50 con 0x7f6060074890 2024-09-16T00:11:14.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7f1d4ffff640 1 -- 172.21.15.139:0/3703458782 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1d50044020 con 0x7f1d5c00de10 2024-09-16T00:11:14.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7f5067fff640 1 --2- 172.21.15.139:0/3730141799 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f505406c2b0 0x7f505406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.820 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7f3e46ffd640 1 -- 172.21.15.139:0/1404189891 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7f3e4c092050 con 0x7f3e50071290 2024-09-16T00:11:14.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7f3e56aa9640 1 --2- 172.21.15.139:0/1404189891 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f3e340749e0 0x7f3e34076e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7f3e55aa7640 1 --2- 172.21.15.139:0/1404189891 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3e3406c400 0x7f3e3406e8c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3e501583b0 tx=0x7f3e40007c60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.750+0000 7fea9ffff640 1 --2- 172.21.15.139:0/1603190478 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7fea8c074810 0x7fea8c076cb0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7f5064ff9640 1 -- 172.21.15.139:0/3730141799 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f506008bfb0 con 0x7f50680c36b0 2024-09-16T00:11:14.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.750+0000 7fe2b0923640 1 --2- 172.21.15.139:0/2188819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2ac1f5540 0x7fe2ac14f390 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4004270 tx=0x7fe2a4004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7f5064ff9640 1 --2- 172.21.15.139:0/3730141799 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5054074830 0x7f5054076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7f5064ff9640 1 -- 172.21.15.139:0/3730141799 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f506001bec0 con 0x7f5054074830 2024-09-16T00:11:14.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.748+0000 7f5064ff9640 1 -- 172.21.15.139:0/3730141799 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7f506008c3a0 con 0x7f50680c36b0 2024-09-16T00:11:14.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.749+0000 7f5067fff640 1 --2- 172.21.15.139:0/3730141799 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f505406c2b0 0x7f505406e770 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f505c034000 tx=0x7f505c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.750+0000 7f50677fe640 1 --2- 172.21.15.139:0/3730141799 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5054074830 0x7f5054076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.750+0000 7fe2a98c9640 1 -- 172.21.15.139:0/2608939809 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe284001030 con 0x7fe2a40c7390 2024-09-16T00:11:14.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.750+0000 7f1d6a00a640 1 -- 172.21.15.139:0/3703458782 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1d5c04b310 con 0x7f1d5c00de10 2024-09-16T00:11:14.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.750+0000 7fe2a8ff9640 1 -- 172.21.15.139:0/2188819 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4004860 con 0x7fe2ac1f5540 2024-09-16T00:11:14.823 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.750+0000 7f50677fe640 1 --2- 172.21.15.139:0/3730141799 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5054074830 0x7f5054076cd0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.750+0000 7f1d4ffff640 1 --2- 172.21.15.139:0/3703458782 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1d4806c380 0x7f1d4806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.750+0000 7f1d4ffff640 1 -- 172.21.15.139:0/3703458782 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f1d500abb30 con 0x7f1d5c00de10 2024-09-16T00:11:14.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.751+0000 7f1d4ffff640 1 --2- 172.21.15.139:0/3703458782 >> [v2:172.21.15.164:6808/2782017910,v1:172.21.15.164:6809/2782017910] conn(0x7f1d48074880 0x7f1d48076d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.751+0000 7f1d4ffff640 1 -- 172.21.15.139:0/3703458782 --> [v2:172.21.15.164:6808/2782017910,v1:172.21.15.164:6809/2782017910] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1d50033e50 con 0x7f1d48074880 2024-09-16T00:11:14.824 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.751+0000 7f1d62ffd640 1 --2- 172.21.15.139:0/3703458782 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1d4806c380 0x7f1d4806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.751+0000 7fe28bfff640 1 -- 172.21.15.139:0/2608939809 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe29802b3a0 con 0x7fe2a40c7390 2024-09-16T00:11:14.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.751+0000 7f1d4ffff640 1 -- 172.21.15.139:0/3703458782 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7f1d500b2050 con 0x7f1d5c00de10 2024-09-16T00:11:14.825 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.750+0000 7fe2a8ff9640 1 -- 172.21.15.139:0/2188819 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a4011ce0 con 0x7fe2ac1f5540 2024-09-16T00:11:14.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.750+0000 7fe2b1124640 1 -- 172.21.15.139:0/2188819 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ac14b940 con 0x7fe2ac1f5540 2024-09-16T00:11:14.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.750+0000 7fe2b1124640 1 -- 172.21.15.139:0/2188819 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ac0c2e20 con 0x7fe2ac1f5540 2024-09-16T00:11:14.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.751+0000 7fe2a8ff9640 1 -- 172.21.15.139:0/2188819 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4004860 con 0x7fe2ac1f5540 2024-09-16T00:11:14.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.751+0000 7fe28bfff640 1 --2- 172.21.15.139:0/2608939809 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe29006c2b0 0x7fe29006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.826 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.751+0000 7f5064ff9640 1 -- 172.21.15.139:0/3730141799 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f506001bec0 con 0x7f5054074830 2024-09-16T00:11:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.751+0000 7f3e56aa9640 1 --2- 172.21.15.139:0/1404189891 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f3e340749e0 0x7f3e34076e80 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.751+0000 7fea87fff640 1 -- 172.21.15.139:0/1603190478 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea980079d0 con 0x7fea8c074810 2024-09-16T00:11:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.751+0000 7fe28bfff640 1 -- 172.21.15.139:0/2608939809 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7fe298094470 con 0x7fe2a40c7390 2024-09-16T00:11:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.751+0000 7fe28bfff640 1 --2- 172.21.15.139:0/2608939809 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe290074890 0x7fe290076d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.751+0000 7fe28bfff640 1 -- 172.21.15.139:0/2608939809 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe290077400 con 0x7fe290074890 2024-09-16T00:11:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.751+0000 7fe28bfff640 1 -- 172.21.15.139:0/2608939809 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7fe29809b050 con 0x7fe2a40c7390 2024-09-16T00:11:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.752+0000 7fe2a27fc640 1 --2- 172.21.15.139:0/2608939809 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe29006c2b0 0x7fe29006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.752+0000 7fe2a37fe640 1 --2- 172.21.15.139:0/2608939809 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe290074890 0x7fe290076d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.751+0000 7f1d63fff640 1 --2- 172.21.15.139:0/3703458782 >> [v2:172.21.15.164:6808/2782017910,v1:172.21.15.164:6809/2782017910] conn(0x7f1d48074880 0x7f1d48076d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.752+0000 7f1d62ffd640 1 --2- 172.21.15.139:0/3703458782 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1d4806c380 0x7f1d4806e840 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c04dff0 tx=0x7f1d58008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.752+0000 7fe2b1124640 1 -- 172.21.15.139:0/2188819 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe2ac14be60 con 0x7fe2ac1f5540 2024-09-16T00:11:14.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.752+0000 7f3e46ffd640 1 -- 172.21.15.139:0/1404189891 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3e4c01ae50 con 0x7f3e340749e0 2024-09-16T00:11:14.829 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.753+0000 7f1d63fff640 1 --2- 172.21.15.139:0/3703458782 >> [v2:172.21.15.164:6808/2782017910,v1:172.21.15.164:6809/2782017910] conn(0x7f1d48074880 0x7f1d48076d20 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.753+0000 7fe2a27fc640 1 --2- 172.21.15.139:0/2608939809 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe29006c2b0 0x7fe29006e770 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe294034000 tx=0x7fe29402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.753+0000 7fe2a37fe640 1 --2- 172.21.15.139:0/2608939809 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe290074890 0x7fe290076d30 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.753+0000 7fe2a8ff9640 1 -- 172.21.15.139:0/2188819 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe2a40193e0 con 0x7fe2ac1f5540 2024-09-16T00:11:14.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.754+0000 7fe2a8ff9640 1 --2- 172.21.15.139:0/2188819 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe29806c4e0 0x7fe29806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.830 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.754+0000 7fe2a8ff9640 1 -- 172.21.15.139:0/2188819 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7fe2a4055c40 con 0x7fe2ac1f5540 2024-09-16T00:11:14.831 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.754+0000 7fe28bfff640 1 -- 172.21.15.139:0/2608939809 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe290077400 con 0x7fe290074890 2024-09-16T00:11:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.754+0000 7fe2abfff640 1 --2- 172.21.15.139:0/2188819 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe29806c4e0 0x7fe29806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.754+0000 7fe2a8ff9640 1 --2- 172.21.15.139:0/2188819 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe298074a40 0x7fe298076ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.754+0000 7fe2a8ff9640 1 -- 172.21.15.139:0/2188819 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe2a401de50 con 0x7fe298074a40 2024-09-16T00:11:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.755+0000 7fe2abfff640 1 --2- 172.21.15.139:0/2188819 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe29806c4e0 0x7fe29806e9a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe29c033000 tx=0x7fe29c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.755+0000 7fe2ab7fe640 1 --2- 172.21.15.139:0/2188819 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe298074a40 0x7fe298076ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.755+0000 7fe2a8ff9640 1 -- 172.21.15.139:0/2188819 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7fe2a4055870 con 0x7fe2ac1f5540 2024-09-16T00:11:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.755+0000 7fe2ab7fe640 1 --2- 172.21.15.139:0/2188819 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe298074a40 0x7fe298076ee0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:14.832 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.755+0000 7f1d4ffff640 1 -- 172.21.15.139:0/3703458782 <== osd.7 v2:172.21.15.164:6808/2782017910 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1d50033e50 con 0x7f1d48074880 2024-09-16T00:11:14.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.757+0000 7fe2a8ff9640 1 -- 172.21.15.139:0/2188819 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2a401de50 con 0x7fe298074a40 2024-09-16T00:11:14.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.779+0000 7f6085d9d640 1 -- 172.21.15.139:0/2333378253 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f60801ec080 con 0x7f6060074890 2024-09-16T00:11:14.833 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.779+0000 7f607d7fa640 1 -- 172.21.15.139:0/2333378253 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f60801ec080 con 0x7f6060074890 2024-09-16T00:11:14.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.780+0000 7f6076ffd640 1 -- 172.21.15.139:0/2333378253 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f6060074890 msgr2=0x7f6060076d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.780+0000 7f6076ffd640 1 --2- 172.21.15.139:0/2333378253 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7f6060074890 0x7f6060076d30 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.780+0000 7f6076ffd640 1 -- 172.21.15.139:0/2333378253 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f606006c2b0 msgr2=0x7f606006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.780+0000 7f6076ffd640 1 --2- 172.21.15.139:0/2333378253 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f606006c2b0 0x7f606006e770 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f607803d000 tx=0x7f6078037000 comp rx=0 tx=0).stop 2024-09-16T00:11:14.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.780+0000 7f6076ffd640 1 -- 172.21.15.139:0/2333378253 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f608015a6b0 msgr2=0x7f60801ec3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.780+0000 7f6076ffd640 1 --2- 172.21.15.139:0/2333378253 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f608015a6b0 0x7f60801ec3a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f607000efc0 tx=0x7f6070009f90 comp rx=0 tx=0).stop 2024-09-16T00:11:14.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.780+0000 7f608559c640 1 -- 172.21.15.139:0/2333378253 reap_dead start 2024-09-16T00:11:14.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.780+0000 7f6076ffd640 1 -- 172.21.15.139:0/2333378253 shutdown_connections 2024-09-16T00:11:14.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.780+0000 7f6076ffd640 1 -- 172.21.15.139:0/2333378253 >> 172.21.15.139:0/2333378253 conn(0x7f60800cbe90 msgr2=0x7f60800c7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.780+0000 7f6076ffd640 1 -- 172.21.15.139:0/2333378253 shutdown_connections 2024-09-16T00:11:14.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.781+0000 7f3e572aa640 1 -- 172.21.15.139:0/1404189891 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3e38002d50 con 0x7f3e340749e0 2024-09-16T00:11:14.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.781+0000 7f6076ffd640 1 -- 172.21.15.139:0/2333378253 wait complete. 2024-09-16T00:11:14.836 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.781+0000 7feaa5f86640 1 -- 172.21.15.139:0/1603190478 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea80002d10 con 0x7fea8c074810 2024-09-16T00:11:14.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.782+0000 7f3e46ffd640 1 -- 172.21.15.139:0/1404189891 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3e38002d50 con 0x7f3e340749e0 2024-09-16T00:11:14.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.782+0000 7fea87fff640 1 -- 172.21.15.139:0/1603190478 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fea80002d10 con 0x7fea8c074810 2024-09-16T00:11:14.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.782+0000 7f3e572aa640 1 -- 172.21.15.139:0/1404189891 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f3e340749e0 msgr2=0x7f3e34076e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.782+0000 7f3e572aa640 1 --2- 172.21.15.139:0/1404189891 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f3e340749e0 0x7f3e34076e80 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.837 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.782+0000 7f3e572aa640 1 -- 172.21.15.139:0/1404189891 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3e3406c400 msgr2=0x7f3e3406e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.782+0000 7f3e572aa640 1 --2- 172.21.15.139:0/1404189891 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f3e3406c400 0x7f3e3406e8c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3e501583b0 tx=0x7f3e40007c60 comp rx=0 tx=0).stop 2024-09-16T00:11:14.838 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.782+0000 7f9b4f042640 1 -- 172.21.15.139:0/1694397569 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b480cba50 con 0x7f9b2c0748c0 2024-09-16T00:11:14.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.783+0000 7f3e572aa640 1 -- 172.21.15.139:0/1404189891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e50071290 msgr2=0x7f3e501f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.783+0000 7f3e572aa640 1 --2- 172.21.15.139:0/1404189891 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3e50071290 0x7f3e501f0580 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00c820 tx=0x7f3e4c00cce0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.783+0000 7f3e56aa9640 1 -- 172.21.15.139:0/1404189891 reap_dead start 2024-09-16T00:11:14.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.783+0000 7f3e572aa640 1 -- 172.21.15.139:0/1404189891 shutdown_connections 2024-09-16T00:11:14.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.783+0000 7f3e572aa640 1 -- 172.21.15.139:0/1404189891 >> 172.21.15.139:0/1404189891 conn(0x7f3e5006bb20 msgr2=0x7f3e501de4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.783+0000 7feaa5f86640 1 -- 172.21.15.139:0/1603190478 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7fea8c074810 msgr2=0x7fea8c076cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.839 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.783+0000 7feaa5f86640 1 --2- 172.21.15.139:0/1603190478 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7fea8c074810 0x7fea8c076cb0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.783+0000 7f3e572aa640 1 -- 172.21.15.139:0/1404189891 shutdown_connections 2024-09-16T00:11:14.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.783+0000 7f3e572aa640 1 -- 172.21.15.139:0/1404189891 wait complete. 2024-09-16T00:11:14.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.783+0000 7feaa5f86640 1 -- 172.21.15.139:0/1603190478 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fea8c06c2b0 msgr2=0x7fea8c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.783+0000 7feaa5f86640 1 --2- 172.21.15.139:0/1603190478 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fea8c06c2b0 0x7fea8c06e770 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fea90007fd0 tx=0x7fea90007e80 comp rx=0 tx=0).stop 2024-09-16T00:11:14.840 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.783+0000 7f9b3effd640 1 -- 172.21.15.139:0/1694397569 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9b480cba50 con 0x7f9b2c0748c0 2024-09-16T00:11:14.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.783+0000 7f506e6d0640 1 -- 172.21.15.139:0/3730141799 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f50680c71d0 con 0x7f5054074830 2024-09-16T00:11:14.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.783+0000 7feaa5f86640 1 -- 172.21.15.139:0/1603190478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa0070040 msgr2=0x7feaa00c44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.783+0000 7feaa5f86640 1 --2- 172.21.15.139:0/1603190478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feaa0070040 0x7feaa00c44f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fea9800c790 tx=0x7fea9800cc50 comp rx=0 tx=0).stop 2024-09-16T00:11:14.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.783+0000 7fea9ffff640 1 -- 172.21.15.139:0/1603190478 reap_dead start 2024-09-16T00:11:14.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.784+0000 7feaa5f86640 1 -- 172.21.15.139:0/1603190478 shutdown_connections 2024-09-16T00:11:14.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.784+0000 7feaa5f86640 1 -- 172.21.15.139:0/1603190478 >> 172.21.15.139:0/1603190478 conn(0x7feaa006da40 msgr2=0x7feaa00c68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.784+0000 7feaa5f86640 1 -- 172.21.15.139:0/1603190478 shutdown_connections 2024-09-16T00:11:14.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.784+0000 7f5064ff9640 1 -- 172.21.15.139:0/3730141799 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f50680c71d0 con 0x7f5054074830 2024-09-16T00:11:14.842 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.784+0000 7f9b4f042640 1 -- 172.21.15.139:0/1694397569 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f9b2c0748c0 msgr2=0x7f9b2c076d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.784+0000 7f9b4f042640 1 --2- 172.21.15.139:0/1694397569 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f9b2c0748c0 0x7f9b2c076d60 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.784+0000 7feaa5f86640 1 -- 172.21.15.139:0/1603190478 wait complete. 2024-09-16T00:11:14.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.784+0000 7f9b4f042640 1 -- 172.21.15.139:0/1694397569 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9b2c06c2e0 msgr2=0x7f9b2c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.784+0000 7fe2a98c9640 1 -- 172.21.15.139:0/2608939809 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe284002dd0 con 0x7fe290074890 2024-09-16T00:11:14.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.784+0000 7f9b4f042640 1 --2- 172.21.15.139:0/1694397569 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9b2c06c2e0 0x7f9b2c06e7a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9b480c6c90 tx=0x7f9b400098d0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.784+0000 7f9b4f042640 1 -- 172.21.15.139:0/1694397569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b480cac00 msgr2=0x7f9b480c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.784+0000 7f9b4f042640 1 --2- 172.21.15.139:0/1694397569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b480cac00 0x7f9b480c7bd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400c7d0 tx=0x7f9b4400cc90 comp rx=0 tx=0).stop 2024-09-16T00:11:14.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.784+0000 7f9b4e841640 1 -- 172.21.15.139:0/1694397569 reap_dead start 2024-09-16T00:11:14.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.784+0000 7f9b4f042640 1 -- 172.21.15.139:0/1694397569 shutdown_connections 2024-09-16T00:11:14.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.784+0000 7f9b4f042640 1 -- 172.21.15.139:0/1694397569 >> 172.21.15.139:0/1694397569 conn(0x7f9b4806e270 msgr2=0x7f9b480c50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.785+0000 7fe28bfff640 1 -- 172.21.15.139:0/2608939809 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe284002dd0 con 0x7fe290074890 2024-09-16T00:11:14.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.785+0000 7f9b4f042640 1 -- 172.21.15.139:0/1694397569 shutdown_connections 2024-09-16T00:11:14.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.785+0000 7f9b4f042640 1 -- 172.21.15.139:0/1694397569 wait complete. 2024-09-16T00:11:14.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.785+0000 7fe2a98c9640 1 -- 172.21.15.139:0/2608939809 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe290074890 msgr2=0x7fe290076d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.785+0000 7fe2a98c9640 1 --2- 172.21.15.139:0/2608939809 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fe290074890 0x7fe290076d30 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.785+0000 7f506e6d0640 1 -- 172.21.15.139:0/3730141799 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5054074830 msgr2=0x7f5054076cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.785+0000 7f506e6d0640 1 --2- 172.21.15.139:0/3730141799 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f5054074830 0x7f5054076cd0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7f506e6d0640 1 -- 172.21.15.139:0/3730141799 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f505406c2b0 msgr2=0x7f505406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7fe2b1124640 1 -- 172.21.15.139:0/2188819 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe2ac073830 con 0x7fe298074a40 2024-09-16T00:11:14.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7f506e6d0640 1 --2- 172.21.15.139:0/3730141799 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f505406c2b0 0x7f505406e770 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f505c034000 tx=0x7f505c02e000 comp rx=0 tx=0).stop 2024-09-16T00:11:14.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7fe2a98c9640 1 -- 172.21.15.139:0/2608939809 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe29006c2b0 msgr2=0x7fe29006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7fe2a98c9640 1 --2- 172.21.15.139:0/2608939809 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe29006c2b0 0x7fe29006e770 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe294034000 tx=0x7fe29402e000 comp rx=0 tx=0).stop 2024-09-16T00:11:14.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7f506e6d0640 1 -- 172.21.15.139:0/3730141799 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50680c36b0 msgr2=0x7f5068155110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7fe2a98c9640 1 -- 172.21.15.139:0/2608939809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2a40c7390 msgr2=0x7fe2a4153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7fe2a98c9640 1 --2- 172.21.15.139:0/2608939809 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2a40c7390 0x7fe2a4153060 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe29800be50 tx=0x7fe298010df0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7fe2a37fe640 1 -- 172.21.15.139:0/2608939809 reap_dead start 2024-09-16T00:11:14.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7f506e6d0640 1 --2- 172.21.15.139:0/3730141799 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f50680c36b0 0x7f5068155110 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f506000eed0 tx=0x7f506000b480 comp rx=0 tx=0).stop 2024-09-16T00:11:14.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7f506cc6d640 1 -- 172.21.15.139:0/3730141799 reap_dead start 2024-09-16T00:11:14.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7fe2a8ff9640 1 -- 172.21.15.139:0/2188819 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe2ac073830 con 0x7fe298074a40 2024-09-16T00:11:14.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7fe2a98c9640 1 -- 172.21.15.139:0/2608939809 shutdown_connections 2024-09-16T00:11:14.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7fe2a98c9640 1 -- 172.21.15.139:0/2608939809 >> 172.21.15.139:0/2608939809 conn(0x7fe2a406cc30 msgr2=0x7fe2a406f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.787+0000 7fe2a98c9640 1 -- 172.21.15.139:0/2608939809 shutdown_connections 2024-09-16T00:11:14.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.787+0000 7fe2a98c9640 1 -- 172.21.15.139:0/2608939809 wait complete. 2024-09-16T00:11:14.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.787+0000 7fe2b1124640 1 -- 172.21.15.139:0/2188819 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe298074a40 msgr2=0x7fe298076ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.787+0000 7fe2b1124640 1 --2- 172.21.15.139:0/2188819 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7fe298074a40 0x7fe298076ee0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7f506e6d0640 1 -- 172.21.15.139:0/3730141799 shutdown_connections 2024-09-16T00:11:14.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7f506e6d0640 1 -- 172.21.15.139:0/3730141799 >> 172.21.15.139:0/3730141799 conn(0x7f50680cc740 msgr2=0x7f5068157fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.786+0000 7f506e6d0640 1 -- 172.21.15.139:0/3730141799 shutdown_connections 2024-09-16T00:11:14.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.787+0000 7f506e6d0640 1 -- 172.21.15.139:0/3730141799 wait complete. 2024-09-16T00:11:14.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.787+0000 7f1d6a00a640 1 -- 172.21.15.139:0/3703458782 --> [v2:172.21.15.164:6808/2782017910,v1:172.21.15.164:6809/2782017910] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1d5c005b00 con 0x7f1d48074880 2024-09-16T00:11:14.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.787+0000 7fe2b1124640 1 -- 172.21.15.139:0/2188819 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe29806c4e0 msgr2=0x7fe29806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.787+0000 7fe2b1124640 1 --2- 172.21.15.139:0/2188819 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe29806c4e0 0x7fe29806e9a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe29c033000 tx=0x7fe29c008030 comp rx=0 tx=0).stop 2024-09-16T00:11:14.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.787+0000 7fe2b1124640 1 -- 172.21.15.139:0/2188819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2ac1f5540 msgr2=0x7fe2ac14f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.787+0000 7fe2b1124640 1 --2- 172.21.15.139:0/2188819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe2ac1f5540 0x7fe2ac14f390 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4004270 tx=0x7fe2a4004670 comp rx=0 tx=0).stop 2024-09-16T00:11:14.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.787+0000 7fe2b0923640 1 -- 172.21.15.139:0/2188819 reap_dead start 2024-09-16T00:11:14.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.787+0000 7f1d4ffff640 1 -- 172.21.15.139:0/3703458782 <== osd.7 v2:172.21.15.164:6808/2782017910 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1d5c005b00 con 0x7f1d48074880 2024-09-16T00:11:14.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.788+0000 7fe2b1124640 1 -- 172.21.15.139:0/2188819 shutdown_connections 2024-09-16T00:11:14.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.788+0000 7fe2b1124640 1 -- 172.21.15.139:0/2188819 >> 172.21.15.139:0/2188819 conn(0x7fe2ac0c8a00 msgr2=0x7fe2ac0c9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.788+0000 7fe2b1124640 1 -- 172.21.15.139:0/2188819 shutdown_connections 2024-09-16T00:11:14.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.788+0000 7fe2b1124640 1 -- 172.21.15.139:0/2188819 wait complete. 2024-09-16T00:11:14.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.788+0000 7f1d4dffb640 1 -- 172.21.15.139:0/3703458782 >> [v2:172.21.15.164:6808/2782017910,v1:172.21.15.164:6809/2782017910] conn(0x7f1d48074880 msgr2=0x7f1d48076d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.789+0000 7f1d4dffb640 1 --2- 172.21.15.139:0/3703458782 >> [v2:172.21.15.164:6808/2782017910,v1:172.21.15.164:6809/2782017910] conn(0x7f1d48074880 0x7f1d48076d20 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.789+0000 7f1d4dffb640 1 -- 172.21.15.139:0/3703458782 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1d4806c380 msgr2=0x7f1d4806e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.789+0000 7f1d4dffb640 1 --2- 172.21.15.139:0/3703458782 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f1d4806c380 0x7f1d4806e840 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c04dff0 tx=0x7f1d58008040 comp rx=0 tx=0).stop 2024-09-16T00:11:14.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.789+0000 7f1d4dffb640 1 -- 172.21.15.139:0/3703458782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d5c00de10 msgr2=0x7f1d5c0ec020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:14.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.789+0000 7f1d4dffb640 1 --2- 172.21.15.139:0/3703458782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d5c00de10 0x7f1d5c0ec020 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009870 tx=0x7f1d500046c0 comp rx=0 tx=0).stop 2024-09-16T00:11:14.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.789+0000 7f1d63fff640 1 -- 172.21.15.139:0/3703458782 reap_dead start 2024-09-16T00:11:14.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.790+0000 7f1d4dffb640 1 -- 172.21.15.139:0/3703458782 shutdown_connections 2024-09-16T00:11:14.854 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.790+0000 7f1d4dffb640 1 -- 172.21.15.139:0/3703458782 >> 172.21.15.139:0/3703458782 conn(0x7f1d5c04fc80 msgr2=0x7f1d5c0d2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:14.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.792+0000 7f1d4dffb640 1 -- 172.21.15.139:0/3703458782 shutdown_connections 2024-09-16T00:11:14.855 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:14.792+0000 7f1d4dffb640 1 -- 172.21.15.139:0/3703458782 wait complete. 2024-09-16T00:11:14.855 INFO:teuthology.orchestra.run.smithi139.stdout:64424509489 2024-09-16T00:11:14.856 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T00:11:14.856 INFO:teuthology.orchestra.run.smithi139.stdout:64424509490 2024-09-16T00:11:14.857 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T00:11:14.857 INFO:teuthology.orchestra.run.smithi139.stdout:403726925841 2024-09-16T00:11:14.858 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T00:11:14.858 INFO:teuthology.orchestra.run.smithi139.stdout:60129542194 2024-09-16T00:11:14.859 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T00:11:14.859 INFO:teuthology.orchestra.run.smithi139.stdout:55834574898 2024-09-16T00:11:14.860 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T00:11:14.860 INFO:teuthology.orchestra.run.smithi139.stdout:64424509491 2024-09-16T00:11:14.861 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T00:11:14.861 INFO:teuthology.orchestra.run.smithi139.stdout:60129542194 2024-09-16T00:11:14.862 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T00:11:14.863 INFO:teuthology.orchestra.run.smithi139.stdout:545460846598 2024-09-16T00:11:14.863 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T00:11:15.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.061+0000 7fa1ef2c9640 1 Processor -- start 2024-09-16T00:11:15.064 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.061+0000 7fa1ef2c9640 1 -- start start 2024-09-16T00:11:15.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.062+0000 7fa1ef2c9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa1e806fc20 0x7fa1e816b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.062+0000 7fa1ef2c9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa1e8167690 0x7fa1e80c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.062+0000 7fa1ef2c9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1e80c6040 con 0x7fa1e80cc150 2024-09-16T00:11:15.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.062+0000 7fa1ef2c9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa1e80c6320 con 0x7fa1e806fc20 2024-09-16T00:11:15.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.062+0000 7fa1ef2c9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e80c6600 con 0x7fa1e8167690 2024-09-16T00:11:15.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.063+0000 7fa1ed065640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa1e806fc20 0x7fa1e816b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.063+0000 7fa1ed866640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa1e80cc150 0x7fa1e80c5290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46174/0 (socket says 172.21.15.139:46174) 2024-09-16T00:11:15.065 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.063+0000 7f86d8645640 1 Processor -- start 2024-09-16T00:11:15.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.063+0000 7fa1ed866640 1 -- 172.21.15.139:0/3151825377 learned_addr learned my addr 172.21.15.139:0/3151825377 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:15.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.063+0000 7f86d8645640 1 -- start start 2024-09-16T00:11:15.066 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.063+0000 7fa1ec864640 1 --2- 172.21.15.139:0/3151825377 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa1e8167690 0x7fa1e80c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.063+0000 7f78e26a4640 1 Processor -- start 2024-09-16T00:11:15.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.063+0000 7f78e26a4640 1 -- start start 2024-09-16T00:11:15.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.063+0000 7fa1ec864640 1 -- 172.21.15.139:0/3151825377 >> v2:172.21.15.139:3300/0 conn(0x7fa1e806fc20 msgr2=0x7fa1e816b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:15.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.063+0000 7fa1ec864640 1 --2- 172.21.15.139:0/3151825377 >> v2:172.21.15.139:3300/0 conn(0x7fa1e806fc20 0x7fa1e816b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.063+0000 7fa1ec864640 1 -- 172.21.15.139:0/3151825377 >> v1:172.21.15.139:6789/0 conn(0x7fa1e80cc150 legacy=0x7fa1e80c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.067 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.063+0000 7fa1ec864640 1 -- 172.21.15.139:0/3151825377 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e816ac00 con 0x7fa1e8167690 2024-09-16T00:11:15.068 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.064+0000 7f3438b78640 1 Processor -- start 2024-09-16T00:11:15.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.064+0000 7f86d8645640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86d0154a80 0x7f86d0072af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.064+0000 7f86d8645640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f86d0073030 0x7f86d0165260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f86d8645640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86d00c6f80 con 0x7f86d00c8020 2024-09-16T00:11:15.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f86d8645640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f86d00702d0 con 0x7f86d0073030 2024-09-16T00:11:15.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f86d8645640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f86d006fd50 con 0x7f86d0154a80 2024-09-16T00:11:15.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f86d6be2640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f86d0073030 0x7f86d0165260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.064+0000 7f78e26a4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78dc06ca00 0x7f78dc157530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f78dbfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f78dc06c620 0x7f78dc150990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46200/0 (socket says 172.21.15.139:46200) 2024-09-16T00:11:15.069 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f78dbfff640 1 -- 172.21.15.139:0/1692528265 learned_addr learned my addr 172.21.15.139:0/1692528265 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:15.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f78db7fe640 1 --2- 172.21.15.139:0/1692528265 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78dc06ca00 0x7f78dc157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f86d5be0640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f86d00c8020 0x7f86d00cc150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46188/0 (socket says 172.21.15.139:46188) 2024-09-16T00:11:15.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f86d5be0640 1 -- 172.21.15.139:0/3403883421 learned_addr learned my addr 172.21.15.139:0/3403883421 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:15.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f78e26a4640 1 -- 172.21.15.139:0/1692528265 --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78dc0c6a80 con 0x7f78dc06c620 2024-09-16T00:11:15.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f78e26a4640 1 -- 172.21.15.139:0/1692528265 --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78dc072ef0 con 0x7f78dc0b20e0 2024-09-16T00:11:15.070 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f78e26a4640 1 -- 172.21.15.139:0/1692528265 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f78dc157a70 con 0x7f78dc06ca00 2024-09-16T00:11:15.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7fa1ec864640 1 --2- 172.21.15.139:0/3151825377 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa1e8167690 0x7fa1e80c96a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009d90 tx=0x7fa1e002a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5372d05a5fcfe5db server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f86d63e1640 1 --2- 172.21.15.139:0/3403883421 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86d0154a80 0x7f86d0072af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.071 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f3438b78640 1 -- start start 2024-09-16T00:11:15.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f78daffd640 1 -- 172.21.15.139:0/1692528265 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 372943553 0 0) 0x7f78dc072ef0 con 0x7f78dc0b20e0 2024-09-16T00:11:15.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f78daffd640 1 -- 172.21.15.139:0/1692528265 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78c4003520 con 0x7f78dc0b20e0 2024-09-16T00:11:15.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f78db7fe640 1 -- 172.21.15.139:0/1692528265 >> v1:172.21.15.164:6789/0 conn(0x7f78dc0b20e0 legacy=0x7f78dc0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.072 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f78db7fe640 1 -- 172.21.15.139:0/1692528265 >> v1:172.21.15.139:6789/0 conn(0x7f78dc06c620 legacy=0x7f78dc150990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f78db7fe640 1 -- 172.21.15.139:0/1692528265 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78dc0731d0 con 0x7f78dc06ca00 2024-09-16T00:11:15.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.066+0000 7fa1dffff640 1 -- 172.21.15.139:0/3151825377 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0007740 con 0x7fa1e8167690 2024-09-16T00:11:15.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.066+0000 7f644948b640 1 Processor -- start 2024-09-16T00:11:15.073 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.066+0000 7fa1dffff640 1 -- 172.21.15.139:0/3151825377 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e00078e0 con 0x7fa1e8167690 2024-09-16T00:11:15.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.066+0000 7fa1dffff640 1 -- 172.21.15.139:0/3151825377 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0007cb0 con 0x7fa1e8167690 2024-09-16T00:11:15.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.066+0000 7f644948b640 1 -- start start 2024-09-16T00:11:15.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f86d63e1640 1 -- 172.21.15.139:0/3403883421 >> v2:172.21.15.164:3300/0 conn(0x7f86d0073030 msgr2=0x7f86d0165260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:15.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f86d63e1640 1 --2- 172.21.15.139:0/3403883421 >> v2:172.21.15.164:3300/0 conn(0x7f86d0073030 0x7f86d0165260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f86d63e1640 1 -- 172.21.15.139:0/3403883421 >> v1:172.21.15.139:6789/0 conn(0x7f86d00c8020 legacy=0x7f86d00cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.065+0000 7f86d63e1640 1 -- 172.21.15.139:0/3403883421 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d00c6ca0 con 0x7f86d0154a80 2024-09-16T00:11:15.074 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.066+0000 7f86d6be2640 1 --2- 172.21.15.139:0/3403883421 >> v2:172.21.15.164:3300/0 conn(0x7f86d0073030 0x7f86d0165260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:15.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.066+0000 7f3438b78640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3434158f00 0x7f34340c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7f78db7fe640 1 --2- 172.21.15.139:0/1692528265 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78dc06ca00 0x7f78dc157530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f78d00097d0 tx=0x7f78d002c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=539bc58c611f5c79 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.075 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7f3438b78640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f343406fc20 0x7f34340cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7f78daffd640 1 -- 172.21.15.139:0/1692528265 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0038070 con 0x7f78dc06ca00 2024-09-16T00:11:15.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7f3438b78640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34340c6580 con 0x7f3434070040 2024-09-16T00:11:15.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7f343259c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3434158f00 0x7f34340c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7fa1ef2c9640 1 -- 172.21.15.139:0/3151825377 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa1e8167690 msgr2=0x7fa1e80c96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7fa1ef2c9640 1 --2- 172.21.15.139:0/3151825377 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa1e8167690 0x7fa1e80c96a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009d90 tx=0x7fa1e002a6f0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7fa1ef2c9640 1 -- 172.21.15.139:0/3151825377 shutdown_connections 2024-09-16T00:11:15.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7fa1ef2c9640 1 --2- 172.21.15.139:0/3151825377 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa1e8167690 0x7fa1e80c96a0 secure :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009d90 tx=0x7fa1e002a6f0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7fa1ef2c9640 1 --2- 172.21.15.139:0/3151825377 >> v2:172.21.15.139:3300/0 conn(0x7fa1e806fc20 0x7fa1e816b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.077 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7fa1ef2c9640 1 -- 172.21.15.139:0/3151825377 >> 172.21.15.139:0/3151825377 conn(0x7fa1e806da40 msgr2=0x7fa1e806f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:15.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7f3438b78640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f34340c8c90 con 0x7f343406fc20 2024-09-16T00:11:15.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7f3438b78640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f34340727d0 con 0x7f3434158f00 2024-09-16T00:11:15.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7f3431d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f343406fc20 0x7f34340cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7f86d63e1640 1 --2- 172.21.15.139:0/3403883421 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86d0154a80 0x7f86d0072af0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f86c40097a0 tx=0x7f86c402a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d107f53b7edde48 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7f3431d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f343406fc20 0x7f34340cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60420/0 (socket says 172.21.15.139:60420) 2024-09-16T00:11:15.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7f343259c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3434158f00 0x7f34340c6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:58432/0 (socket says 172.21.15.139:58432) 2024-09-16T00:11:15.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7f343259c640 1 -- 172.21.15.139:0/2813053523 learned_addr learned my addr 172.21.15.139:0/2813053523 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:15.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7f78daffd640 1 -- 172.21.15.139:0/1692528265 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78d0031710 con 0x7f78dc06ca00 2024-09-16T00:11:15.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7f78daffd640 1 -- 172.21.15.139:0/1692528265 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0031ae0 con 0x7f78dc06ca00 2024-09-16T00:11:15.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7fa1ef2c9640 1 -- 172.21.15.139:0/3151825377 shutdown_connections 2024-09-16T00:11:15.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7fa1ef2c9640 1 -- 172.21.15.139:0/3151825377 wait complete. 2024-09-16T00:11:15.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.067+0000 7f86d53df640 1 -- 172.21.15.139:0/3403883421 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4037070 con 0x7f86d0154a80 2024-09-16T00:11:15.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f86d53df640 1 -- 172.21.15.139:0/3403883421 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c402ae40 con 0x7f86d0154a80 2024-09-16T00:11:15.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f86d53df640 1 -- 172.21.15.139:0/3403883421 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c4032660 con 0x7f86d0154a80 2024-09-16T00:11:15.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f644948b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f644415ab80 0x7f644415afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f6443fff640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f6444154c50 0x7f64440cbcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:44854/0 (socket says 172.21.15.139:44854) 2024-09-16T00:11:15.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f343259c640 1 -- 172.21.15.139:0/2813053523 >> v2:172.21.15.139:3300/0 conn(0x7f343406fc20 msgr2=0x7f34340cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:15.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f6443fff640 1 -- 172.21.15.139:0/3620585876 learned_addr learned my addr 172.21.15.139:0/3620585876 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:15.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f343259c640 1 --2- 172.21.15.139:0/2813053523 >> v2:172.21.15.139:3300/0 conn(0x7f343406fc20 0x7f34340cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f343259c640 1 -- 172.21.15.139:0/2813053523 >> v1:172.21.15.139:6789/0 conn(0x7f3434070040 legacy=0x7f34340c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:15.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f343259c640 1 -- 172.21.15.139:0/2813053523 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3434072ab0 con 0x7f3434158f00 2024-09-16T00:11:15.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f78e26a4640 1 -- 172.21.15.139:0/1692528265 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78dc06ca00 msgr2=0x7f78dc157530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f78e26a4640 1 --2- 172.21.15.139:0/1692528265 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78dc06ca00 0x7f78dc157530 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f78d00097d0 tx=0x7f78d002c4b0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f3431d9b640 1 --2- 172.21.15.139:0/2813053523 >> v2:172.21.15.139:3300/0 conn(0x7f343406fc20 0x7f34340cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:15.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f644948b640 1 -- 172.21.15.139:0/3620585876 --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64440c7a40 con 0x7f64440c75d0 2024-09-16T00:11:15.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f644948b640 1 -- 172.21.15.139:0/3620585876 --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644415b520 con 0x7f6444154c50 2024-09-16T00:11:15.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f6448c8a640 1 --2- 172.21.15.139:0/3620585876 >> v2:172.21.15.164:3300/0 conn(0x7f644415ab80 0x7f644415afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.083 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f8de645a640 1 Processor -- start 2024-09-16T00:11:15.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.068+0000 7f8de645a640 1 -- start start 2024-09-16T00:11:15.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f644948b640 1 -- 172.21.15.139:0/3620585876 --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f64440cc400 con 0x7f644415ab80 2024-09-16T00:11:15.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f86d8645640 1 -- 172.21.15.139:0/3403883421 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86d0154a80 msgr2=0x7f86d0072af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f60bb29b640 1 Processor -- start 2024-09-16T00:11:15.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f6442ffd640 1 -- 172.21.15.139:0/3620585876 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2240410886 0 0) 0x7f644415b520 con 0x7f6444154c50 2024-09-16T00:11:15.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f343259c640 1 --2- 172.21.15.139:0/2813053523 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3434158f00 0x7f34340c6040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3424009870 tx=0x7f342402a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=207362e46cf7707 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f6442ffd640 1 -- 172.21.15.139:0/3620585876 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f642c003520 con 0x7f6444154c50 2024-09-16T00:11:15.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f86d8645640 1 --2- 172.21.15.139:0/3403883421 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86d0154a80 0x7f86d0072af0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f86c40097a0 tx=0x7f86c402a0f0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f78e26a4640 1 -- 172.21.15.139:0/1692528265 shutdown_connections 2024-09-16T00:11:15.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f78e26a4640 1 --2- 172.21.15.139:0/1692528265 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78dc06ca00 0x7f78dc157530 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f78e26a4640 1 -- 172.21.15.139:0/1692528265 >> 172.21.15.139:0/1692528265 conn(0x7f78dc0c8ae0 msgr2=0x7f78dc0c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:15.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f6442ffd640 1 -- 172.21.15.139:0/3620585876 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1692154172 0 0) 0x7f64440c7a40 con 0x7f64440c75d0 2024-09-16T00:11:15.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f6442ffd640 1 -- 172.21.15.139:0/3620585876 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f644415b520 con 0x7f64440c75d0 2024-09-16T00:11:15.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f8de645a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8de0150ad0 0x7f8de0157360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f8de645a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8de006fc20 0x7f8de00cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f8de645a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8de015a120 con 0x7f8de00cc690 2024-09-16T00:11:15.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7fa1ef2c9640 1 Processor -- start 2024-09-16T00:11:15.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f8de645a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8de00c6330 con 0x7f8de006fc20 2024-09-16T00:11:15.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f343159a640 1 -- 172.21.15.139:0/2813053523 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424037070 con 0x7f3434158f00 2024-09-16T00:11:15.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f343159a640 1 -- 172.21.15.139:0/2813053523 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3424032440 con 0x7f3434158f00 2024-09-16T00:11:15.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f343159a640 1 -- 172.21.15.139:0/2813053523 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424032810 con 0x7f3434158f00 2024-09-16T00:11:15.088 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f8de645a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8de0159ed0 con 0x7f8de0150ad0 2024-09-16T00:11:15.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7fa1ef2c9640 1 -- start start 2024-09-16T00:11:15.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f8ddffff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8de0150ad0 0x7f8de0157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f60bb29b640 1 -- start start 2024-09-16T00:11:15.089 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f6442ffd640 1 -- 172.21.15.139:0/3620585876 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 6093789 0 0) 0x7f644415b520 con 0x7f64440c75d0 2024-09-16T00:11:15.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f8ddffff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8de0150ad0 0x7f8de0157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:58448/0 (socket says 172.21.15.139:58448) 2024-09-16T00:11:15.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f8ddffff640 1 -- 172.21.15.139:0/2090548289 learned_addr learned my addr 172.21.15.139:0/2090548289 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:15.090 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.069+0000 7f8ddf7fe640 1 --2- 172.21.15.139:0/2090548289 >> v2:172.21.15.139:3300/0 conn(0x7f8de006fc20 0x7f8de00cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f8ddeffd640 1 -- 172.21.15.139:0/2090548289 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3429258397 0 0) 0x7f8de015a120 con 0x7f8de00cc690 2024-09-16T00:11:15.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f8ddffff640 1 -- 172.21.15.139:0/2090548289 >> v2:172.21.15.139:3300/0 conn(0x7f8de006fc20 msgr2=0x7f8de00cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:15.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f8ddffff640 1 --2- 172.21.15.139:0/2090548289 >> v2:172.21.15.139:3300/0 conn(0x7f8de006fc20 0x7f8de00cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f8ddffff640 1 -- 172.21.15.139:0/2090548289 >> v1:172.21.15.164:6789/0 conn(0x7f8de00cc690 legacy=0x7f8de0072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f8ddffff640 1 -- 172.21.15.139:0/2090548289 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de015a400 con 0x7f8de0150ad0 2024-09-16T00:11:15.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f86d8645640 1 -- 172.21.15.139:0/3403883421 shutdown_connections 2024-09-16T00:11:15.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f86d8645640 1 --2- 172.21.15.139:0/3403883421 >> v2:172.21.15.164:3300/0 conn(0x7f86d0073030 0x7f86d0165260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f86d8645640 1 --2- 172.21.15.139:0/3403883421 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86d0154a80 0x7f86d0072af0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.091 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f86d8645640 1 -- 172.21.15.139:0/3403883421 >> 172.21.15.139:0/3403883421 conn(0x7f86d006da40 msgr2=0x7f86d006f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:15.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f78e26a4640 1 -- 172.21.15.139:0/1692528265 shutdown_connections 2024-09-16T00:11:15.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f78e26a4640 1 -- 172.21.15.139:0/1692528265 wait complete. 2024-09-16T00:11:15.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f86d8645640 1 -- 172.21.15.139:0/3403883421 shutdown_connections 2024-09-16T00:11:15.092 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f86d8645640 1 -- 172.21.15.139:0/3403883421 wait complete. 2024-09-16T00:11:15.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f3438b78640 1 -- 172.21.15.139:0/2813053523 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3434158f00 msgr2=0x7f34340c6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f3438b78640 1 --2- 172.21.15.139:0/2813053523 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3434158f00 0x7f34340c6040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3424009870 tx=0x7f342402a2a0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f3438b78640 1 -- 172.21.15.139:0/2813053523 shutdown_connections 2024-09-16T00:11:15.093 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f3438b78640 1 --2- 172.21.15.139:0/2813053523 >> v2:172.21.15.139:3300/0 conn(0x7f343406fc20 0x7f34340cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f3438b78640 1 --2- 172.21.15.139:0/2813053523 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3434158f00 0x7f34340c6040 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f3438b78640 1 -- 172.21.15.139:0/2813053523 >> 172.21.15.139:0/2813053523 conn(0x7f343406da40 msgr2=0x7f343406f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:15.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7fa1ef2c9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e806fc20 0x7fa1e815d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7fa1ef2c9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa1e80cc150 0x7fa1e816a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7fa1ef2c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1e815d810 0x7fa1e816a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7fa1ef2c9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e8167a90 con 0x7fa1e815d810 2024-09-16T00:11:15.094 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7fa1ef2c9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e816b1c0 con 0x7fa1e806fc20 2024-09-16T00:11:15.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7fa1ef2c9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e80c3af0 con 0x7fa1e80cc150 2024-09-16T00:11:15.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f60bb29b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f60b4158f00 0x7f60b40c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f60bb29b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60b406fc20 0x7f60b40cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f60b8836640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60b406fc20 0x7f60b40cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f60b8836640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60b406fc20 0x7f60b40cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:58450/0 (socket says 172.21.15.139:58450) 2024-09-16T00:11:15.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f60b8836640 1 -- 172.21.15.139:0/4165613890 learned_addr learned my addr 172.21.15.139:0/4165613890 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:15.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f60b9037640 1 --2- 172.21.15.139:0/4165613890 >> v2:172.21.15.164:3300/0 conn(0x7f60b4158f00 0x7f60b40c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.095 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f3438b78640 1 -- 172.21.15.139:0/2813053523 shutdown_connections 2024-09-16T00:11:15.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7fa1ed065640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e806fc20 0x7fa1e815d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f60bb29b640 1 -- 172.21.15.139:0/4165613890 --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60b40c6580 con 0x7f60b4070040 2024-09-16T00:11:15.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f60bb29b640 1 -- 172.21.15.139:0/4165613890 --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f60b40c8c90 con 0x7f60b4158f00 2024-09-16T00:11:15.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f60bb29b640 1 -- 172.21.15.139:0/4165613890 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f60b40727d0 con 0x7f60b406fc20 2024-09-16T00:11:15.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f3438b78640 1 -- 172.21.15.139:0/2813053523 wait complete. 2024-09-16T00:11:15.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7fa1ed065640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e806fc20 0x7fa1e815d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:40024/0 (socket says 172.21.15.139:40024) 2024-09-16T00:11:15.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7fa1ed065640 1 -- 172.21.15.139:0/3152819064 learned_addr learned my addr 172.21.15.139:0/3152819064 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:15.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7fa1ed866640 1 --2- 172.21.15.139:0/3152819064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1e815d810 0x7fa1e816a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.097 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f6442ffd640 1 -- 172.21.15.139:0/3620585876 >> v2:172.21.15.164:3300/0 conn(0x7f644415ab80 msgr2=0x7f644415afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:15.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f8ddffff640 1 --2- 172.21.15.139:0/2090548289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8de0150ad0 0x7f8de0157360 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400a910 tx=0x7f8dd400a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=191a21a1a0c3fa3b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f6442ffd640 1 --2- 172.21.15.139:0/3620585876 >> v2:172.21.15.164:3300/0 conn(0x7f644415ab80 0x7f644415afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7fa1ed065640 1 -- 172.21.15.139:0/3152819064 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa1e80cc150 msgr2=0x7fa1e816a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7fa1ed065640 1 --2- 172.21.15.139:0/3152819064 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa1e80cc150 0x7fa1e816a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.072+0000 7f8ddeffd640 1 -- 172.21.15.139:0/2090548289 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4038070 con 0x7f8de0150ad0 2024-09-16T00:11:15.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.072+0000 7f8ddeffd640 1 -- 172.21.15.139:0/2090548289 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd4030480 con 0x7f8de0150ad0 2024-09-16T00:11:15.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7fa1ed065640 1 -- 172.21.15.139:0/3152819064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1e815d810 msgr2=0x7fa1e816a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.072+0000 7fa1ed065640 1 --2- 172.21.15.139:0/3152819064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1e815d810 0x7fa1e816a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.072+0000 7fa1ed065640 1 -- 172.21.15.139:0/3152819064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e80cb740 con 0x7fa1e806fc20 2024-09-16T00:11:15.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f60b9037640 1 -- 172.21.15.139:0/4165613890 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60b406fc20 msgr2=0x7f60b40cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:15.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.072+0000 7fa1ed866640 1 --2- 172.21.15.139:0/3152819064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1e815d810 0x7fa1e816a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:15.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.072+0000 7f78e26a4640 1 Processor -- start 2024-09-16T00:11:15.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f60b9037640 1 --2- 172.21.15.139:0/4165613890 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60b406fc20 0x7f60b40cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f60b9037640 1 -- 172.21.15.139:0/4165613890 >> v1:172.21.15.164:6789/0 conn(0x7f60b4070040 legacy=0x7f60b40c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:15.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f60b9037640 1 -- 172.21.15.139:0/4165613890 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b4072ab0 con 0x7f60b4158f00 2024-09-16T00:11:15.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.072+0000 7fa1ed065640 1 --2- 172.21.15.139:0/3152819064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e806fc20 0x7fa1e815d2d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400eba0 tx=0x7fa1e400d720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.070+0000 7f6442ffd640 1 -- 172.21.15.139:0/3620585876 >> v1:172.21.15.164:6789/0 conn(0x7f6444154c50 legacy=0x7f64440cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f6442ffd640 1 -- 172.21.15.139:0/3620585876 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644415de20 con 0x7f64440c75d0 2024-09-16T00:11:15.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f6442ffd640 1 -- 172.21.15.139:0/3620585876 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7f6438001670 con 0x7f64440c75d0 2024-09-16T00:11:15.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f6442ffd640 1 -- 172.21.15.139:0/3620585876 >> v1:172.21.15.139:6789/0 conn(0x7f64440c75d0 legacy=0x7f64440cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f6442ffd640 1 --2- 172.21.15.139:0/3620585876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f642c003b70 0x7f642c003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f6442ffd640 1 --2- 172.21.15.139:0/3620585876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f642c004800 0x7f642c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f6442ffd640 1 --2- 172.21.15.139:0/3620585876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f642c0051c0 0x7f642c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f6442ffd640 1 -- 172.21.15.139:0/3620585876 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f64440cc400 con 0x7f642c004800 2024-09-16T00:11:15.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f6442ffd640 1 -- 172.21.15.139:0/3620585876 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f642c00bf60 con 0x7f642c003b70 2024-09-16T00:11:15.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f6442ffd640 1 -- 172.21.15.139:0/3620585876 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f642c00c160 con 0x7f642c0051c0 2024-09-16T00:11:15.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f64437fe640 1 --2- 172.21.15.139:0/3620585876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f642c004800 0x7f642c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f6443fff640 1 --2- 172.21.15.139:0/3620585876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f642c003b70 0x7f642c003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f64437fe640 1 -- 172.21.15.139:0/3620585876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f642c0051c0 msgr2=0x7f642c00b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:11:15.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f64437fe640 1 --2- 172.21.15.139:0/3620585876 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f642c0051c0 0x7f642c00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f64437fe640 1 -- 172.21.15.139:0/3620585876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f642c003b70 msgr2=0x7f642c003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:15.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f64437fe640 1 --2- 172.21.15.139:0/3620585876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f642c003b70 0x7f642c003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.071+0000 7f64437fe640 1 -- 172.21.15.139:0/3620585876 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644415de20 con 0x7f642c004800 2024-09-16T00:11:15.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.072+0000 7f6448c8a640 1 -- 172.21.15.139:0/3620585876 reap_dead start 2024-09-16T00:11:15.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.072+0000 7f86d8645640 1 Processor -- start 2024-09-16T00:11:15.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.072+0000 7f60b9037640 1 --2- 172.21.15.139:0/4165613890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60b4158f00 0x7f60b40c6040 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f60a40097a0 tx=0x7f60a402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed2b815551c5e8a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.072+0000 7f86d8645640 1 -- start start 2024-09-16T00:11:15.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f8de645a640 1 -- 172.21.15.139:0/2090548289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8de0150ad0 msgr2=0x7f8de0157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f8de645a640 1 --2- 172.21.15.139:0/2090548289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8de0150ad0 0x7f8de0157360 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400a910 tx=0x7f8dd400a250 comp rx=0 tx=0).stop 2024-09-16T00:11:15.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f3438b78640 1 Processor -- start 2024-09-16T00:11:15.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f6443fff640 1 --2- 172.21.15.139:0/3620585876 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f642c003b70 0x7f642c003f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:15.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.072+0000 7fa1ddffb640 1 -- 172.21.15.139:0/3152819064 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e400eea0 con 0x7fa1e806fc20 2024-09-16T00:11:15.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7fa1ddffb640 1 -- 172.21.15.139:0/3152819064 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e4010640 con 0x7fa1e806fc20 2024-09-16T00:11:15.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7fa1ddffb640 1 -- 172.21.15.139:0/3152819064 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4010a30 con 0x7fa1e806fc20 2024-09-16T00:11:15.107 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f3438b78640 1 -- start start 2024-09-16T00:11:15.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f78e26a4640 1 -- start start 2024-09-16T00:11:15.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f8de645a640 1 -- 172.21.15.139:0/2090548289 shutdown_connections 2024-09-16T00:11:15.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f8de645a640 1 --2- 172.21.15.139:0/2090548289 >> v2:172.21.15.139:3300/0 conn(0x7f8de006fc20 0x7f8de00cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f8de645a640 1 --2- 172.21.15.139:0/2090548289 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8de0150ad0 0x7f8de0157360 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f8de645a640 1 -- 172.21.15.139:0/2090548289 >> 172.21.15.139:0/2090548289 conn(0x7f8de006da40 msgr2=0x7f8de006f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:15.108 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f64437fe640 1 --2- 172.21.15.139:0/3620585876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f642c004800 0x7f642c004c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f643800c190 tx=0x7f642c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06b1e9ecfa5ffb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f8de645a640 1 -- 172.21.15.139:0/2090548289 shutdown_connections 2024-09-16T00:11:15.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f60abfff640 1 -- 172.21.15.139:0/4165613890 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4037070 con 0x7f60b4158f00 2024-09-16T00:11:15.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f60abfff640 1 -- 172.21.15.139:0/4165613890 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a4032440 con 0x7f60b4158f00 2024-09-16T00:11:15.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f60abfff640 1 -- 172.21.15.139:0/4165613890 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a40327f0 con 0x7f60b4158f00 2024-09-16T00:11:15.109 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7fa1ef2c9640 1 -- 172.21.15.139:0/3152819064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e80cb0b0 con 0x7fa1e806fc20 2024-09-16T00:11:15.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f6442ffd640 1 -- 172.21.15.139:0/3620585876 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438007c40 con 0x7f642c004800 2024-09-16T00:11:15.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f6442ffd640 1 -- 172.21.15.139:0/3620585876 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6438031620 con 0x7f642c004800 2024-09-16T00:11:15.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f6442ffd640 1 -- 172.21.15.139:0/3620585876 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438031a10 con 0x7f642c004800 2024-09-16T00:11:15.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f86d8645640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86d0073030 0x7f86d01f0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f78e26a4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78dc06c620 0x7f78dc153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f3438b78640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f343406fc20 0x7f34340c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f78e26a4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78dc06ca00 0x7f78dc1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f78e26a4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78dc0b20e0 0x7f78dc150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f78e26a4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc06ceb0 con 0x7f78dc06ca00 2024-09-16T00:11:15.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f343259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f343406fc20 0x7f34340c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f343259c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f343406fc20 0x7f34340c50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60464/0 (socket says 172.21.15.139:60464) 2024-09-16T00:11:15.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f3438b78640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3434070040 0x7f343414c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f343259c640 1 -- 172.21.15.139:0/2334481530 learned_addr learned my addr 172.21.15.139:0/2334481530 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:15.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f3438b78640 1 --2- 172.21.15.139:0/2334481530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3434158f00 0x7f34341f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f78e26a4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f78dc0c6d80 con 0x7f78dc0b20e0 2024-09-16T00:11:15.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f78e26a4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f78dc072be0 con 0x7f78dc06c620 2024-09-16T00:11:15.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f3438b78640 1 -- 172.21.15.139:0/2334481530 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f34341592e0 con 0x7f343406fc20 2024-09-16T00:11:15.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f3438b78640 1 -- 172.21.15.139:0/2334481530 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f3434073070 con 0x7f3434070040 2024-09-16T00:11:15.113 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f60bb29b640 1 -- 172.21.15.139:0/4165613890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60b4158f00 msgr2=0x7f60b40c6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f60bb29b640 1 --2- 172.21.15.139:0/4165613890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60b4158f00 0x7f60b40c6040 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f60a40097a0 tx=0x7f60a402a1e0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f3438b78640 1 -- 172.21.15.139:0/2334481530 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f3434159460 con 0x7f3434158f00 2024-09-16T00:11:15.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f78db7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78dc06ca00 0x7f78dc1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.114 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f644948b640 1 -- 172.21.15.139:0/3620585876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f642c004800 msgr2=0x7f642c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f644948b640 1 --2- 172.21.15.139:0/3620585876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f642c004800 0x7f642c004c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f643800c190 tx=0x7f642c005830 comp rx=0 tx=0).stop 2024-09-16T00:11:15.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7fa1ef2c9640 1 -- 172.21.15.139:0/3152819064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e81fdb30 con 0x7fa1e806fc20 2024-09-16T00:11:15.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f78db7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78dc06ca00 0x7f78dc1536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60462/0 (socket says 172.21.15.139:60462) 2024-09-16T00:11:15.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.074+0000 7f78db7fe640 1 -- 172.21.15.139:0/1976817518 learned_addr learned my addr 172.21.15.139:0/1976817518 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:15.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f78dbfff640 1 --2- 172.21.15.139:0/1976817518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78dc06c620 0x7f78dc153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f3432d9d640 1 --2- 172.21.15.139:0/2334481530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3434158f00 0x7f34341f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f86d8645640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86d0154a80 0x7f86d0159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f86d8645640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d015d420 0x7f86d015d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f86d8645640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f86d00718b0 con 0x7f86d0073030 2024-09-16T00:11:15.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f86d8645640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f86d00725d0 con 0x7f86d015d420 2024-09-16T00:11:15.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f86d8645640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f86d015a920 con 0x7f86d0154a80 2024-09-16T00:11:15.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f86d63e1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86d0073030 0x7f86d01f0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.073+0000 7f8de645a640 1 -- 172.21.15.139:0/2090548289 wait complete. 2024-09-16T00:11:15.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f8de645a640 1 Processor -- start 2024-09-16T00:11:15.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f644948b640 1 -- 172.21.15.139:0/3620585876 shutdown_connections 2024-09-16T00:11:15.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f644948b640 1 --2- 172.21.15.139:0/3620585876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f642c004800 0x7f642c004c80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f644948b640 1 -- 172.21.15.139:0/3620585876 >> 172.21.15.139:0/3620585876 conn(0x7f644406c7c0 msgr2=0x7f644406de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:15.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f3431d9b640 1 --2- 172.21.15.139:0/2334481530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3434070040 0x7f343414c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f644948b640 1 -- 172.21.15.139:0/3620585876 shutdown_connections 2024-09-16T00:11:15.118 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f644948b640 1 -- 172.21.15.139:0/3620585876 wait complete. 2024-09-16T00:11:15.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f86d6be2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d015d420 0x7f86d015d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f86d6be2640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d015d420 0x7f86d015d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:40054/0 (socket says 172.21.15.139:40054) 2024-09-16T00:11:15.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f343259c640 1 -- 172.21.15.139:0/2334481530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3434158f00 msgr2=0x7f34341f80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f343259c640 1 --2- 172.21.15.139:0/2334481530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3434158f00 0x7f34341f80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f343259c640 1 -- 172.21.15.139:0/2334481530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3434070040 msgr2=0x7f343414c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f343259c640 1 --2- 172.21.15.139:0/2334481530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3434070040 0x7f343414c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.119 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f343259c640 1 -- 172.21.15.139:0/2334481530 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3434153370 con 0x7f343406fc20 2024-09-16T00:11:15.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f86d6be2640 1 -- 172.21.15.139:0/3805815554 learned_addr learned my addr 172.21.15.139:0/3805815554 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:15.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7fa1ef2c9640 1 -- 172.21.15.139:0/3152819064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e80c4560 con 0x7fa1e806fc20 2024-09-16T00:11:15.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f60bb29b640 1 -- 172.21.15.139:0/4165613890 shutdown_connections 2024-09-16T00:11:15.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f60bb29b640 1 --2- 172.21.15.139:0/4165613890 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60b406fc20 0x7f60b40cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f60bb29b640 1 --2- 172.21.15.139:0/4165613890 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60b4158f00 0x7f60b40c6040 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.120 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f60bb29b640 1 -- 172.21.15.139:0/4165613890 >> 172.21.15.139:0/4165613890 conn(0x7f60b406da40 msgr2=0x7f60b406f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:15.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f8de645a640 1 -- start start 2024-09-16T00:11:15.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f78dbfff640 1 -- 172.21.15.139:0/1976817518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78dc0b20e0 msgr2=0x7f78dc150e00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:11:15.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f78dbfff640 1 --2- 172.21.15.139:0/1976817518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78dc0b20e0 0x7f78dc150e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f78dbfff640 1 -- 172.21.15.139:0/1976817518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78dc06ca00 msgr2=0x7f78dc1536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f78dbfff640 1 --2- 172.21.15.139:0/1976817518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78dc06ca00 0x7f78dc1536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.075+0000 7f78dbfff640 1 -- 172.21.15.139:0/1976817518 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78dc1ec450 con 0x7f78dc06c620 2024-09-16T00:11:15.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f78dbfff640 1 --2- 172.21.15.139:0/1976817518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78dc06c620 0x7f78dc153190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f78cc000f60 tx=0x7f78cc00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f60bb29b640 1 -- 172.21.15.139:0/4165613890 shutdown_connections 2024-09-16T00:11:15.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f343259c640 1 --2- 172.21.15.139:0/2334481530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f343406fc20 0x7f34340c50b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3424009990 tx=0x7f34240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f60bb29b640 1 -- 172.21.15.139:0/4165613890 wait complete. 2024-09-16T00:11:15.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f86d6be2640 1 -- 172.21.15.139:0/3805815554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86d0154a80 msgr2=0x7f86d0159690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:11:15.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f86d6be2640 1 --2- 172.21.15.139:0/3805815554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86d0154a80 0x7f86d0159690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f86d6be2640 1 -- 172.21.15.139:0/3805815554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86d0073030 msgr2=0x7f86d01f0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f86d6be2640 1 --2- 172.21.15.139:0/3805815554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86d0073030 0x7f86d01f0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f86d6be2640 1 -- 172.21.15.139:0/3805815554 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d01f0f50 con 0x7f86d015d420 2024-09-16T00:11:15.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7fa1ddffb640 1 -- 172.21.15.139:0/3152819064 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa1e4010bd0 con 0x7fa1e806fc20 2024-09-16T00:11:15.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f3422ffd640 1 -- 172.21.15.139:0/2334481530 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424041020 con 0x7f343406fc20 2024-09-16T00:11:15.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f78d8ff9640 1 -- 172.21.15.139:0/1976817518 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc017070 con 0x7f78dc06c620 2024-09-16T00:11:15.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f78d8ff9640 1 -- 172.21.15.139:0/1976817518 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78cc005ce0 con 0x7f78dc06c620 2024-09-16T00:11:15.124 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f78d8ff9640 1 -- 172.21.15.139:0/1976817518 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78cc0072d0 con 0x7f78dc06c620 2024-09-16T00:11:15.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7fa1ddffb640 1 --2- 172.21.15.139:0/3152819064 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa1cc06c3b0 0x7fa1cc06e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7f644948b640 1 Processor -- start 2024-09-16T00:11:15.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7f78e26a4640 1 -- 172.21.15.139:0/1976817518 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78dc155cc0 con 0x7f78dc06c620 2024-09-16T00:11:15.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7f78e26a4640 1 -- 172.21.15.139:0/1976817518 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f78dc155480 con 0x7f78dc06c620 2024-09-16T00:11:15.125 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.078+0000 7f644948b640 1 -- start start 2024-09-16T00:11:15.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f8de645a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8de006fc20 0x7f8de00ba730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.078+0000 7f86d6be2640 1 --2- 172.21.15.139:0/3805815554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d015d420 0x7f86d015d8d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f86c800cef0 tx=0x7f86c800a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.126 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f3438b78640 1 -- 172.21.15.139:0/2334481530 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3434153650 con 0x7f343406fc20 2024-09-16T00:11:15.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7f3438b78640 1 -- 172.21.15.139:0/2334481530 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3434153c10 con 0x7f343406fc20 2024-09-16T00:11:15.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.078+0000 7f3422ffd640 1 -- 172.21.15.139:0/2334481530 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3424030800 con 0x7f343406fc20 2024-09-16T00:11:15.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.078+0000 7f60bb29b640 1 Processor -- start 2024-09-16T00:11:15.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.078+0000 7fa1ddffb640 1 -- 172.21.15.139:0/3152819064 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7fa1e408bea0 con 0x7fa1e806fc20 2024-09-16T00:11:15.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.078+0000 7f3422ffd640 1 -- 172.21.15.139:0/2334481530 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424037040 con 0x7f343406fc20 2024-09-16T00:11:15.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.076+0000 7f8de645a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8de00cc690 0x7f8de01fcb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7f8de645a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8de0150ad0 0x7f8de01573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7f8de645a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8de015a9c0 con 0x7f8de0150ad0 2024-09-16T00:11:15.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7f8de645a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f8de0071fe0 con 0x7f8de006fc20 2024-09-16T00:11:15.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.078+0000 7f78e26a4640 1 -- 172.21.15.139:0/1976817518 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78dc069850 con 0x7f78dc06c620 2024-09-16T00:11:15.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7f8de645a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f8de015aca0 con 0x7f8de00cc690 2024-09-16T00:11:15.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7f8de49f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8de0150ad0 0x7f8de01573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7f8de49f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8de0150ad0 0x7f8de01573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60474/0 (socket says 172.21.15.139:60474) 2024-09-16T00:11:15.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7f8de49f7640 1 -- 172.21.15.139:0/2544517264 learned_addr learned my addr 172.21.15.139:0/2544517264 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:15.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7f8de49f7640 1 -- 172.21.15.139:0/2544517264 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8de00cc690 msgr2=0x7f8de01fcb60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:11:15.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7f8de49f7640 1 --2- 172.21.15.139:0/2544517264 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8de00cc690 0x7f8de01fcb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7f8de49f7640 1 -- 172.21.15.139:0/2544517264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8de006fc20 msgr2=0x7f8de00ba730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:11:15.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7f8de49f7640 1 --2- 172.21.15.139:0/2544517264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8de006fc20 0x7f8de00ba730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.077+0000 7f8de49f7640 1 -- 172.21.15.139:0/2544517264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de0157a80 con 0x7f8de0150ad0 2024-09-16T00:11:15.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.078+0000 7f8de49f7640 1 --2- 172.21.15.139:0/2544517264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8de0150ad0 0x7f8de01573e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800c7c0 tx=0x7f8dd800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.078+0000 7f8ddcff9640 1 -- 172.21.15.139:0/2544517264 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8003da0 con 0x7f8de0150ad0 2024-09-16T00:11:15.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.078+0000 7f8de645a640 1 -- 172.21.15.139:0/2544517264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de0157d20 con 0x7f8de0150ad0 2024-09-16T00:11:15.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.079+0000 7f644948b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f642c004800 0x7f644415cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.130 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.079+0000 7f3422ffd640 1 -- 172.21.15.139:0/2334481530 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3424044020 con 0x7f343406fc20 2024-09-16T00:11:15.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.079+0000 7f78d8ff9640 1 -- 172.21.15.139:0/1976817518 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f78cc01a070 con 0x7f78dc06c620 2024-09-16T00:11:15.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.079+0000 7f9ff78c9640 1 Processor -- start 2024-09-16T00:11:15.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.078+0000 7f60bb29b640 1 -- start start 2024-09-16T00:11:15.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.079+0000 7f60bb29b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60b406fc20 0x7f60b415e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.079+0000 7f8ddcff9640 1 -- 172.21.15.139:0/2544517264 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd8005070 con 0x7f8de0150ad0 2024-09-16T00:11:15.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.079+0000 7f6443fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f642c004800 0x7f644415cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.079+0000 7f9ff78c9640 1 -- start start 2024-09-16T00:11:15.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.079+0000 7f60b9037640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60b406fc20 0x7f60b415e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.132 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.079+0000 7f6443fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f642c004800 0x7f644415cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60480/0 (socket says 172.21.15.139:60480) 2024-09-16T00:11:15.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.079+0000 7f6443fff640 1 -- 172.21.15.139:0/798299048 learned_addr learned my addr 172.21.15.139:0/798299048 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:15.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.079+0000 7fa1ec864640 1 --2- 172.21.15.139:0/3152819064 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa1cc06c3b0 0x7fa1cc06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f644948b640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f644415a390 0x7f64440c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.133 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.079+0000 7f60b9037640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60b406fc20 0x7f60b415e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60486/0 (socket says 172.21.15.139:60486) 2024-09-16T00:11:15.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f644948b640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64440c42c0 0x7f64441589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f60b9037640 1 -- 172.21.15.139:0/2586254182 learned_addr learned my addr 172.21.15.139:0/2586254182 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:15.134 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f644948b640 1 -- 172.21.15.139:0/798299048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f644415e6c0 con 0x7f642c004800 2024-09-16T00:11:15.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f644948b640 1 -- 172.21.15.139:0/798299048 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f644415e3e0 con 0x7f64440c42c0 2024-09-16T00:11:15.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f644948b640 1 -- 172.21.15.139:0/798299048 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f644415e9a0 con 0x7f644415a390 2024-09-16T00:11:15.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f64437fe640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f644415a390 0x7f64440c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f60bb29b640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60b4070040 0x7f60b415c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f3438b78640 1 -- 172.21.15.139:0/2334481530 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3418005290 con 0x7f343406fc20 2024-09-16T00:11:15.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f6448c8a640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64440c42c0 0x7f64441589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f60bb29b640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60b4158f00 0x7f60b4152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f78d8ff9640 1 --2- 172.21.15.139:0/1976817518 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f78c806c3b0 0x7f78c806e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f78d8ff9640 1 -- 172.21.15.139:0/1976817518 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f78cc08a6a0 con 0x7f78dc06c620 2024-09-16T00:11:15.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f3422ffd640 1 --2- 172.21.15.139:0/2334481530 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f341406c380 0x7f341406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f60b8836640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60b4070040 0x7f60b415c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7fa1ec864640 1 --2- 172.21.15.139:0/3152819064 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa1cc06c3b0 0x7fa1cc06e870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0034000 tx=0x7fa1e002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f78db7fe640 1 --2- 172.21.15.139:0/1976817518 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f78c806c3b0 0x7f78c806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f3422ffd640 1 -- 172.21.15.139:0/2334481530 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f34240abd50 con 0x7f343406fc20 2024-09-16T00:11:15.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f3431d9b640 1 --2- 172.21.15.139:0/2334481530 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f341406c380 0x7f341406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f78db7fe640 1 --2- 172.21.15.139:0/1976817518 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f78c806c3b0 0x7f78c806e870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f78dc14cdd0 tx=0x7f78d0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f9ff78c9640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9ff0068060 0x7f9ff015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.138 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f9ff78c9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9ff00ccc90 0x7f9ff00cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.080+0000 7f8ddcff9640 1 -- 172.21.15.139:0/2544517264 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8005440 con 0x7f8de0150ad0 2024-09-16T00:11:15.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f6443fff640 1 -- 172.21.15.139:0/798299048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f644415a390 msgr2=0x7f64440c3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f6443fff640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f644415a390 0x7f64440c3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f6443fff640 1 -- 172.21.15.139:0/798299048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64440c42c0 msgr2=0x7f64441589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f6443fff640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64440c42c0 0x7f64441589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.139 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f6443fff640 1 -- 172.21.15.139:0/798299048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6444159c20 con 0x7f642c004800 2024-09-16T00:11:15.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f9ff78c9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff00cc4b0 con 0x7f9ff0154c20 2024-09-16T00:11:15.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f9ff78c9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9ff00c5830 con 0x7f9ff00ccc90 2024-09-16T00:11:15.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f9ff78c9640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f9ff015b7c0 con 0x7f9ff0068060 2024-09-16T00:11:15.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f9ff4e64640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9ff0068060 0x7f9ff015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f8de645a640 1 -- 172.21.15.139:0/2544517264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de00c31f0 con 0x7f8de0150ad0 2024-09-16T00:11:15.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f9ff5e66640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9ff00ccc90 0x7f9ff00cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f9ff5e66640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9ff00ccc90 0x7f9ff00cd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60502/0 (socket says 172.21.15.139:60502) 2024-09-16T00:11:15.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f9ff5e66640 1 -- 172.21.15.139:0/145140267 learned_addr learned my addr 172.21.15.139:0/145140267 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:15.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f6443fff640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f642c004800 0x7f644415cd30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f643400d8c0 tx=0x7f643400dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f64437fe640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f644415a390 0x7f64440c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:11:15.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f3431d9b640 1 --2- 172.21.15.139:0/2334481530 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f341406c380 0x7f341406e840 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f343414f020 tx=0x7f3428008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f6448c8a640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64440c42c0 0x7f64441589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:11:15.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f60bb29b640 1 -- 172.21.15.139:0/2586254182 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f60b41592e0 con 0x7f60b406fc20 2024-09-16T00:11:15.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f60b9838640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60b4158f00 0x7f60b4152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f60bb29b640 1 -- 172.21.15.139:0/2586254182 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f60b4073070 con 0x7f60b4070040 2024-09-16T00:11:15.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f60bb29b640 1 -- 172.21.15.139:0/2586254182 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f60b4159460 con 0x7f60b4158f00 2024-09-16T00:11:15.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f9ff5e66640 1 -- 172.21.15.139:0/145140267 >> v2:172.21.15.164:3300/0 conn(0x7f9ff0068060 msgr2=0x7f9ff015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:15.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f9ff5e66640 1 --2- 172.21.15.139:0/145140267 >> v2:172.21.15.164:3300/0 conn(0x7f9ff0068060 0x7f9ff015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f9ff5e66640 1 -- 172.21.15.139:0/145140267 >> v1:172.21.15.164:6789/0 conn(0x7f9ff0154c20 legacy=0x7f9ff00c5120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:11:15.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.081+0000 7f9ff5e66640 1 -- 172.21.15.139:0/145140267 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff00cc790 con 0x7f9ff00ccc90 2024-09-16T00:11:15.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.082+0000 7f86c2ffd640 1 -- 172.21.15.139:0/3805815554 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c801a070 con 0x7f86d015d420 2024-09-16T00:11:15.143 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.082+0000 7fa1ddffb640 1 -- 172.21.15.139:0/3152819064 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa1e80c4560 con 0x7fa1e806fc20 2024-09-16T00:11:15.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.082+0000 7f60b9037640 1 -- 172.21.15.139:0/2586254182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60b4158f00 msgr2=0x7f60b4152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.082+0000 7f86c2ffd640 1 -- 172.21.15.139:0/3805815554 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86c800d860 con 0x7f86d015d420 2024-09-16T00:11:15.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.082+0000 7f86d8645640 1 -- 172.21.15.139:0/3805815554 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d015e550 con 0x7f86d015d420 2024-09-16T00:11:15.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.082+0000 7f60b9037640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60b4158f00 0x7f60b4152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.082+0000 7f60b9037640 1 -- 172.21.15.139:0/2586254182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60b4070040 msgr2=0x7f60b415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.082+0000 7f60b9037640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60b4070040 0x7f60b415c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.082+0000 7f60b9037640 1 -- 172.21.15.139:0/2586254182 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b415e7f0 con 0x7f60b406fc20 2024-09-16T00:11:15.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.082+0000 7f60b8836640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60b4070040 0x7f60b415c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:15.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.082+0000 7f86d8645640 1 -- 172.21.15.139:0/3805815554 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d015e810 con 0x7f86d015d420 2024-09-16T00:11:15.145 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.082+0000 7f60b9037640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60b406fc20 0x7f60b415e2b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f60a40098c0 tx=0x7f60a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.082+0000 7f8de645a640 1 -- 172.21.15.139:0/2544517264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de0070060 con 0x7f8de0150ad0 2024-09-16T00:11:15.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.082+0000 7f8ddcff9640 1 -- 172.21.15.139:0/2544517264 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8dd80055e0 con 0x7f8de0150ad0 2024-09-16T00:11:15.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.083+0000 7f8ddcff9640 1 --2- 172.21.15.139:0/2544517264 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8dcc06c330 0x7f8dcc06e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.083+0000 7f8ddffff640 1 --2- 172.21.15.139:0/2544517264 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8dcc06c330 0x7f8dcc06e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.146 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.083+0000 7f86c2ffd640 1 -- 172.21.15.139:0/3805815554 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86c800dc50 con 0x7f86d015d420 2024-09-16T00:11:15.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.084+0000 7f8ddffff640 1 --2- 172.21.15.139:0/2544517264 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8dcc06c330 0x7f8dcc06e7f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4035000 tx=0x7f8dd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.084+0000 7f9ff5e66640 1 --2- 172.21.15.139:0/145140267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff00ccc90 0x7f9ff00cd0a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00685f0 tx=0x7f9fe802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a877817ad9cfd6e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.084+0000 7f86d8645640 1 -- 172.21.15.139:0/3805815554 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d0154e60 con 0x7f86d015d420 2024-09-16T00:11:15.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.084+0000 7f86c2ffd640 1 -- 172.21.15.139:0/3805815554 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f86c801d070 con 0x7f86d015d420 2024-09-16T00:11:15.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.084+0000 7f86c2ffd640 1 --2- 172.21.15.139:0/3805815554 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f86b406c330 0x7f86b406e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.084+0000 7f78d8ff9640 1 -- 172.21.15.139:0/1976817518 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f78cc058740 con 0x7f78dc06c620 2024-09-16T00:11:15.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.084+0000 7f86d63e1640 1 --2- 172.21.15.139:0/3805815554 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f86b406c330 0x7f86b406e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.085+0000 7f86c2ffd640 1 -- 172.21.15.139:0/3805815554 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f86c808bc80 con 0x7f86d015d420 2024-09-16T00:11:15.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.085+0000 7f86d63e1640 1 --2- 172.21.15.139:0/3805815554 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f86b406c330 0x7f86b406e7f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f86c4034000 tx=0x7f86c402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.089+0000 7f86c2ffd640 1 -- 172.21.15.139:0/3805815554 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f86c805a3e0 con 0x7f86d015d420 2024-09-16T00:11:15.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.089+0000 7f6440ff9640 1 -- 172.21.15.139:0/798299048 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434019070 con 0x7f642c004800 2024-09-16T00:11:15.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.089+0000 7f60a9ffb640 1 -- 172.21.15.139:0/2586254182 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4037070 con 0x7f60b406fc20 2024-09-16T00:11:15.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.089+0000 7f644948b640 1 -- 172.21.15.139:0/798299048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64441f0ef0 con 0x7f642c004800 2024-09-16T00:11:15.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.089+0000 7f3422ffd640 1 -- 172.21.15.139:0/2334481530 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f342404b030 con 0x7f343406fc20 2024-09-16T00:11:15.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.090+0000 7f644948b640 1 -- 172.21.15.139:0/798299048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f644415d270 con 0x7f642c004800 2024-09-16T00:11:15.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.090+0000 7f9fe7fff640 1 -- 172.21.15.139:0/145140267 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8036070 con 0x7f9ff00ccc90 2024-09-16T00:11:15.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.090+0000 7f8ddcff9640 1 -- 172.21.15.139:0/2544517264 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f8dd8056c80 con 0x7f8de0150ad0 2024-09-16T00:11:15.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.090+0000 7f6440ff9640 1 -- 172.21.15.139:0/798299048 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64340072d0 con 0x7f642c004800 2024-09-16T00:11:15.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.091+0000 7f6440ff9640 1 -- 172.21.15.139:0/798299048 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64340047e0 con 0x7f642c004800 2024-09-16T00:11:15.151 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.091+0000 7f644948b640 1 -- 172.21.15.139:0/798299048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6424005290 con 0x7f642c004800 2024-09-16T00:11:15.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.089+0000 7f60bb29b640 1 -- 172.21.15.139:0/2586254182 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b41e9070 con 0x7f60b406fc20 2024-09-16T00:11:15.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.090+0000 7f60a9ffb640 1 -- 172.21.15.139:0/2586254182 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a4038040 con 0x7f60b406fc20 2024-09-16T00:11:15.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.090+0000 7f60bb29b640 1 -- 172.21.15.139:0/2586254182 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b41f54f0 con 0x7f60b406fc20 2024-09-16T00:11:15.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.091+0000 7f60a9ffb640 1 -- 172.21.15.139:0/2586254182 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4003d90 con 0x7f60b406fc20 2024-09-16T00:11:15.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.092+0000 7f9fe7fff640 1 -- 172.21.15.139:0/145140267 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe8031600 con 0x7f9ff00ccc90 2024-09-16T00:11:15.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.092+0000 7f9fe7fff640 1 -- 172.21.15.139:0/145140267 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe80319d0 con 0x7f9ff00ccc90 2024-09-16T00:11:15.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.093+0000 7f60bb29b640 1 -- 172.21.15.139:0/2586254182 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b40c6040 con 0x7f60b406fc20 2024-09-16T00:11:15.155 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.093+0000 7f9ff78c9640 1 -- 172.21.15.139:0/145140267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff00ccc90 msgr2=0x7f9ff00cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.093+0000 7f9ff78c9640 1 --2- 172.21.15.139:0/145140267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff00ccc90 0x7f9ff00cd0a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00685f0 tx=0x7f9fe802a6f0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.093+0000 7f9ff78c9640 1 -- 172.21.15.139:0/145140267 shutdown_connections 2024-09-16T00:11:15.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.093+0000 7f9ff78c9640 1 --2- 172.21.15.139:0/145140267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff00ccc90 0x7f9ff00cd0a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.093+0000 7f9ff78c9640 1 --2- 172.21.15.139:0/145140267 >> v2:172.21.15.164:3300/0 conn(0x7f9ff0068060 0x7f9ff015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.093+0000 7f9ff78c9640 1 -- 172.21.15.139:0/145140267 >> 172.21.15.139:0/145140267 conn(0x7f9ff006ca40 msgr2=0x7f9ff0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:15.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.093+0000 7f9ff78c9640 1 -- 172.21.15.139:0/145140267 shutdown_connections 2024-09-16T00:11:15.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.093+0000 7f9ff78c9640 1 -- 172.21.15.139:0/145140267 wait complete. 2024-09-16T00:11:15.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.095+0000 7f9ff78c9640 1 Processor -- start 2024-09-16T00:11:15.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.095+0000 7f9ff78c9640 1 -- start start 2024-09-16T00:11:15.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.096+0000 7f9ff78c9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ff0068060 0x7f9ff0153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.096+0000 7f9ff78c9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff00ccc90 0x7f9ff01540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.096+0000 7f9ff78c9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff0154c20 0x7f9ff015bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.096+0000 7f9ff78c9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff015e0d0 con 0x7f9ff0154c20 2024-09-16T00:11:15.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.096+0000 7f9ff78c9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff00cc260 con 0x7f9ff00ccc90 2024-09-16T00:11:15.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.096+0000 7f9ff78c9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff015e3b0 con 0x7f9ff0068060 2024-09-16T00:11:15.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.096+0000 7f9ff5665640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ff0068060 0x7f9ff0153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.096+0000 7f9ff5e66640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff0154c20 0x7f9ff015bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.096+0000 7f9ff5665640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ff0068060 0x7f9ff0153b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:58542/0 (socket says 172.21.15.139:58542) 2024-09-16T00:11:15.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.096+0000 7f9ff5665640 1 -- 172.21.15.139:0/2978858235 learned_addr learned my addr 172.21.15.139:0/2978858235 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:15.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.096+0000 7f9ff4e64640 1 --2- 172.21.15.139:0/2978858235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff00ccc90 0x7f9ff01540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.096+0000 7f9ff5665640 1 -- 172.21.15.139:0/2978858235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff00ccc90 msgr2=0x7f9ff01540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.096+0000 7f9ff5665640 1 --2- 172.21.15.139:0/2978858235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff00ccc90 0x7f9ff01540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.096+0000 7f9ff5665640 1 -- 172.21.15.139:0/2978858235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff0154c20 msgr2=0x7f9ff015bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.096+0000 7f9ff5665640 1 --2- 172.21.15.139:0/2978858235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff0154c20 0x7f9ff015bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.096+0000 7f9ff5665640 1 -- 172.21.15.139:0/2978858235 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff015da90 con 0x7f9ff0068060 2024-09-16T00:11:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.097+0000 7f9ff5e66640 1 --2- 172.21.15.139:0/2978858235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff0154c20 0x7f9ff015bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:11:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.097+0000 7f9ff5665640 1 --2- 172.21.15.139:0/2978858235 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ff0068060 0x7f9ff0153b80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00b3e0 tx=0x7f9fec00b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.097+0000 7f9fe5ffb640 1 -- 172.21.15.139:0/2978858235 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec015020 con 0x7f9ff0068060 2024-09-16T00:11:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.097+0000 7f9fe5ffb640 1 -- 172.21.15.139:0/2978858235 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fec011850 con 0x7f9ff0068060 2024-09-16T00:11:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.097+0000 7f9fe5ffb640 1 -- 172.21.15.139:0/2978858235 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec011c40 con 0x7f9ff0068060 2024-09-16T00:11:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.097+0000 7f9ff78c9640 1 -- 172.21.15.139:0/2978858235 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff01f11d0 con 0x7f9ff0068060 2024-09-16T00:11:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.097+0000 7f9ff78c9640 1 -- 172.21.15.139:0/2978858235 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff0148ab0 con 0x7f9ff0068060 2024-09-16T00:11:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.099+0000 7f9fe5ffb640 1 -- 172.21.15.139:0/2978858235 <== mon.2 v2:172.21.15.164:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9fec018020 con 0x7f9ff0068060 2024-09-16T00:11:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.099+0000 7f9ff78c9640 1 -- 172.21.15.139:0/2978858235 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff015e690 con 0x7f9ff0068060 2024-09-16T00:11:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.100+0000 7f9fe5ffb640 1 --2- 172.21.15.139:0/2978858235 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9fd406c2b0 0x7f9fd406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.100+0000 7f9fe5ffb640 1 -- 172.21.15.139:0/2978858235 <== mon.2 v2:172.21.15.164:3301/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f9fec08b520 con 0x7f9ff0068060 2024-09-16T00:11:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.100+0000 7f9ff4e64640 1 --2- 172.21.15.139:0/2978858235 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9fd406c2b0 0x7f9fd406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.101+0000 7f9ff4e64640 1 --2- 172.21.15.139:0/2978858235 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9fd406c2b0 0x7f9fd406e770 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00cddd0 tx=0x7f9fe0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.101+0000 7f8ddcff9640 1 -- 172.21.15.139:0/2544517264 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8dd8091030 con 0x7f8de0150ad0 2024-09-16T00:11:15.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.104+0000 7f60a9ffb640 1 -- 172.21.15.139:0/2586254182 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f60a4043020 con 0x7f60b406fc20 2024-09-16T00:11:15.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.104+0000 7f6440ff9640 1 -- 172.21.15.139:0/798299048 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f643401c070 con 0x7f642c004800 2024-09-16T00:11:15.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.104+0000 7f6440ff9640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f643006c2b0 0x7f643006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.104+0000 7f6440ff9640 1 -- 172.21.15.139:0/798299048 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f643408af90 con 0x7f642c004800 2024-09-16T00:11:15.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.104+0000 7f60a9ffb640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f609806c2b0 0x7f609806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:15.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.104+0000 7f60a9ffb640 1 -- 172.21.15.139:0/2586254182 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f60a40ab9e0 con 0x7f60b406fc20 2024-09-16T00:11:15.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.105+0000 7f64437fe640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f643006c2b0 0x7f643006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.105+0000 7f60b8836640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f609806c2b0 0x7f609806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:15.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.106+0000 7f60b8836640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f609806c2b0 0x7f609806e770 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f60b4153050 tx=0x7f60b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.106+0000 7f64437fe640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f643006c2b0 0x7f643006e770 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6438034000 tx=0x7f6438044040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:15.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.106+0000 7f9fe5ffb640 1 -- 172.21.15.139:0/2978858235 <== mon.2 v2:172.21.15.164:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9fec01f020 con 0x7f9ff0068060 2024-09-16T00:11:15.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.115+0000 7f6440ff9640 1 -- 172.21.15.139:0/798299048 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6434059030 con 0x7f642c004800 2024-09-16T00:11:15.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.119+0000 7f60a9ffb640 1 -- 172.21.15.139:0/2586254182 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f60a407a0d0 con 0x7f60b406fc20 2024-09-16T00:11:15.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.458+0000 7f78e26a4640 1 -- 172.21.15.139:0/1976817518 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f78dc06bdc0 con 0x7f78dc06c620 2024-09-16T00:11:15.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.459+0000 7f78d8ff9640 1 -- 172.21.15.139:0/1976817518 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f78cc05ca40 con 0x7f78dc06c620 2024-09-16T00:11:15.461 INFO:teuthology.orchestra.run.smithi139.stdout:403726925840 2024-09-16T00:11:15.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.462+0000 7f78c27fc640 1 -- 172.21.15.139:0/1976817518 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f78c806c3b0 msgr2=0x7f78c806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.462+0000 7f78c27fc640 1 --2- 172.21.15.139:0/1976817518 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f78c806c3b0 0x7f78c806e870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f78dc14cdd0 tx=0x7f78d0008030 comp rx=0 tx=0).stop 2024-09-16T00:11:15.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.463+0000 7f78c27fc640 1 -- 172.21.15.139:0/1976817518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78dc06c620 msgr2=0x7f78dc153190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.463+0000 7f78c27fc640 1 --2- 172.21.15.139:0/1976817518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78dc06c620 0x7f78dc153190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f78cc000f60 tx=0x7f78cc00c800 comp rx=0 tx=0).stop 2024-09-16T00:11:15.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.463+0000 7f78c27fc640 1 -- 172.21.15.139:0/1976817518 shutdown_connections 2024-09-16T00:11:15.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.463+0000 7f78c27fc640 1 --2- 172.21.15.139:0/1976817518 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f78c806c3b0 0x7f78c806e870 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.463+0000 7f78c27fc640 1 --2- 172.21.15.139:0/1976817518 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f78dc0b20e0 0x7f78dc150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.463+0000 7f78c27fc640 1 --2- 172.21.15.139:0/1976817518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78dc06ca00 0x7f78dc1536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.463+0000 7f78c27fc640 1 --2- 172.21.15.139:0/1976817518 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f78dc06c620 0x7f78dc153190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.463+0000 7f78c27fc640 1 -- 172.21.15.139:0/1976817518 >> 172.21.15.139:0/1976817518 conn(0x7f78dc0c8ae0 msgr2=0x7f78dc0ca010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:15.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.464+0000 7f78c27fc640 1 -- 172.21.15.139:0/1976817518 shutdown_connections 2024-09-16T00:11:15.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.464+0000 7f78c27fc640 1 -- 172.21.15.139:0/1976817518 wait complete. 2024-09-16T00:11:15.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.469+0000 7fa1ef2c9640 1 -- 172.21.15.139:0/3152819064 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fa1e816b6e0 con 0x7fa1e806fc20 2024-09-16T00:11:15.472 INFO:teuthology.orchestra.run.smithi139.stdout:64424509490 2024-09-16T00:11:15.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.470+0000 7fa1ddffb640 1 -- 172.21.15.139:0/3152819064 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa1e4059eb0 con 0x7fa1e806fc20 2024-09-16T00:11:15.475 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.473+0000 7fa1cb7fe640 1 -- 172.21.15.139:0/3152819064 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa1cc06c3b0 msgr2=0x7fa1cc06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.473+0000 7fa1cb7fe640 1 --2- 172.21.15.139:0/3152819064 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa1cc06c3b0 0x7fa1cc06e870 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0034000 tx=0x7fa1e002e000 comp rx=0 tx=0).stop 2024-09-16T00:11:15.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.473+0000 7fa1cb7fe640 1 -- 172.21.15.139:0/3152819064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e806fc20 msgr2=0x7fa1e815d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.473+0000 7fa1cb7fe640 1 --2- 172.21.15.139:0/3152819064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e806fc20 0x7fa1e815d2d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400eba0 tx=0x7fa1e400d720 comp rx=0 tx=0).stop 2024-09-16T00:11:15.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.474+0000 7fa1cb7fe640 1 -- 172.21.15.139:0/3152819064 shutdown_connections 2024-09-16T00:11:15.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.474+0000 7fa1cb7fe640 1 --2- 172.21.15.139:0/3152819064 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa1cc06c3b0 0x7fa1cc06e870 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.474+0000 7fa1cb7fe640 1 --2- 172.21.15.139:0/3152819064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1e815d810 0x7fa1e816a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.474+0000 7fa1cb7fe640 1 --2- 172.21.15.139:0/3152819064 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa1e80cc150 0x7fa1e816a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.474+0000 7fa1cb7fe640 1 --2- 172.21.15.139:0/3152819064 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa1e806fc20 0x7fa1e815d2d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.474+0000 7fa1cb7fe640 1 -- 172.21.15.139:0/3152819064 >> 172.21.15.139:0/3152819064 conn(0x7fa1e806da40 msgr2=0x7fa1e80c4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:15.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.474+0000 7fa1cb7fe640 1 -- 172.21.15.139:0/3152819064 shutdown_connections 2024-09-16T00:11:15.476 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.474+0000 7fa1cb7fe640 1 -- 172.21.15.139:0/3152819064 wait complete. 2024-09-16T00:11:15.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 403726925841 got 403726925840 for osd.6 2024-09-16T00:11:15.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.485+0000 7f3438b78640 1 -- 172.21.15.139:0/2334481530 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3418005580 con 0x7f343406fc20 2024-09-16T00:11:15.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.486+0000 7f3422ffd640 1 -- 172.21.15.139:0/2334481530 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f342407a3b0 con 0x7f343406fc20 2024-09-16T00:11:15.487 INFO:teuthology.orchestra.run.smithi139.stdout:64424509489 2024-09-16T00:11:15.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.488+0000 7f86d8645640 1 -- 172.21.15.139:0/3805815554 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f86d01530b0 con 0x7f86d015d420 2024-09-16T00:11:15.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.489+0000 7f3438b78640 1 -- 172.21.15.139:0/2334481530 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f341406c380 msgr2=0x7f341406e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.489+0000 7f3438b78640 1 --2- 172.21.15.139:0/2334481530 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f341406c380 0x7f341406e840 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f343414f020 tx=0x7f3428008040 comp rx=0 tx=0).stop 2024-09-16T00:11:15.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.489+0000 7f3438b78640 1 -- 172.21.15.139:0/2334481530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f343406fc20 msgr2=0x7f34340c50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.489+0000 7f3438b78640 1 --2- 172.21.15.139:0/2334481530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f343406fc20 0x7f34340c50b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3424009990 tx=0x7f34240046c0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.489+0000 7f3438b78640 1 -- 172.21.15.139:0/2334481530 shutdown_connections 2024-09-16T00:11:15.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.489+0000 7f3438b78640 1 --2- 172.21.15.139:0/2334481530 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f341406c380 0x7f341406e840 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.489+0000 7f3438b78640 1 --2- 172.21.15.139:0/2334481530 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f3434158f00 0x7f34341f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.491 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.489+0000 7f3438b78640 1 --2- 172.21.15.139:0/2334481530 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3434070040 0x7f343414c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.489+0000 7f3438b78640 1 --2- 172.21.15.139:0/2334481530 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f343406fc20 0x7f34340c50b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.489+0000 7f3438b78640 1 -- 172.21.15.139:0/2334481530 >> 172.21.15.139:0/2334481530 conn(0x7f343406da40 msgr2=0x7f3434156d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:15.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.489+0000 7f3438b78640 1 -- 172.21.15.139:0/2334481530 shutdown_connections 2024-09-16T00:11:15.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.489+0000 7f3438b78640 1 -- 172.21.15.139:0/2334481530 wait complete. 2024-09-16T00:11:15.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.489+0000 7f86c2ffd640 1 -- 172.21.15.139:0/3805815554 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f86c8059ce0 con 0x7f86d015d420 2024-09-16T00:11:15.492 INFO:teuthology.orchestra.run.smithi139.stdout:60129542193 2024-09-16T00:11:15.493 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509491 got 64424509490 for osd.4 2024-09-16T00:11:15.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.492+0000 7f86d8645640 1 -- 172.21.15.139:0/3805815554 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f86b406c330 msgr2=0x7f86b406e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.492+0000 7f86d8645640 1 --2- 172.21.15.139:0/3805815554 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f86b406c330 0x7f86b406e7f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f86c4034000 tx=0x7f86c402e000 comp rx=0 tx=0).stop 2024-09-16T00:11:15.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.492+0000 7f86d8645640 1 -- 172.21.15.139:0/3805815554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d015d420 msgr2=0x7f86d015d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.492+0000 7f86d8645640 1 --2- 172.21.15.139:0/3805815554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d015d420 0x7f86d015d8d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f86c800cef0 tx=0x7f86c800a710 comp rx=0 tx=0).stop 2024-09-16T00:11:15.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.492+0000 7f86d8645640 1 -- 172.21.15.139:0/3805815554 shutdown_connections 2024-09-16T00:11:15.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.492+0000 7f86d8645640 1 --2- 172.21.15.139:0/3805815554 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f86b406c330 0x7f86b406e7f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.492+0000 7f86d8645640 1 --2- 172.21.15.139:0/3805815554 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86d015d420 0x7f86d015d8d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.492+0000 7f86d8645640 1 --2- 172.21.15.139:0/3805815554 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86d0154a80 0x7f86d0159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.494 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.492+0000 7f86d8645640 1 --2- 172.21.15.139:0/3805815554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86d0073030 0x7f86d01f0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.492+0000 7f86d8645640 1 -- 172.21.15.139:0/3805815554 >> 172.21.15.139:0/3805815554 conn(0x7f86d006da40 msgr2=0x7f86d01538b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:15.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.492+0000 7f86d8645640 1 -- 172.21.15.139:0/3805815554 shutdown_connections 2024-09-16T00:11:15.495 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.492+0000 7f86d8645640 1 -- 172.21.15.139:0/3805815554 wait complete. 2024-09-16T00:11:15.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.494+0000 7f644948b640 1 -- 172.21.15.139:0/798299048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f6424005850 con 0x7f642c004800 2024-09-16T00:11:15.496 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.494+0000 7f6440ff9640 1 -- 172.21.15.139:0/798299048 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f643405d330 con 0x7f642c004800 2024-09-16T00:11:15.496 INFO:teuthology.orchestra.run.smithi139.stdout:55834574897 2024-09-16T00:11:15.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.497+0000 7f644948b640 1 -- 172.21.15.139:0/798299048 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f643006c2b0 msgr2=0x7f643006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.497+0000 7f644948b640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f643006c2b0 0x7f643006e770 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6438034000 tx=0x7f6438044040 comp rx=0 tx=0).stop 2024-09-16T00:11:15.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.497+0000 7f644948b640 1 -- 172.21.15.139:0/798299048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f642c004800 msgr2=0x7f644415cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.497+0000 7f644948b640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f642c004800 0x7f644415cd30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f643400d8c0 tx=0x7f643400dd80 comp rx=0 tx=0).stop 2024-09-16T00:11:15.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.497+0000 7f644948b640 1 -- 172.21.15.139:0/798299048 shutdown_connections 2024-09-16T00:11:15.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.497+0000 7f644948b640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f643006c2b0 0x7f643006e770 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.497+0000 7f644948b640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f64440c42c0 0x7f64441589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.499 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.497+0000 7f644948b640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f644415a390 0x7f64440c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.497+0000 7f644948b640 1 --2- 172.21.15.139:0/798299048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f642c004800 0x7f644415cd30 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.497+0000 7f644948b640 1 -- 172.21.15.139:0/798299048 >> 172.21.15.139:0/798299048 conn(0x7f644406c7c0 msgr2=0x7f64441537b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:15.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.497+0000 7f644948b640 1 -- 172.21.15.139:0/798299048 shutdown_connections 2024-09-16T00:11:15.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.497+0000 7f644948b640 1 -- 172.21.15.139:0/798299048 wait complete. 2024-09-16T00:11:15.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.498+0000 7f9ff78c9640 1 -- 172.21.15.139:0/2978858235 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9ff00c51c0 con 0x7f9ff0068060 2024-09-16T00:11:15.500 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.498+0000 7f9fe5ffb640 1 -- 172.21.15.139:0/2978858235 <== mon.2 v2:172.21.15.164:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9fec059a70 con 0x7f9ff0068060 2024-09-16T00:11:15.501 INFO:teuthology.orchestra.run.smithi139.stdout:60129542193 2024-09-16T00:11:15.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.502+0000 7f9ff78c9640 1 -- 172.21.15.139:0/2978858235 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9fd406c2b0 msgr2=0x7f9fd406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.502+0000 7f8de645a640 1 -- 172.21.15.139:0/2544517264 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8de0150600 con 0x7f8de0150ad0 2024-09-16T00:11:15.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.502+0000 7f9ff78c9640 1 --2- 172.21.15.139:0/2978858235 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9fd406c2b0 0x7f9fd406e770 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00cddd0 tx=0x7f9fe0008040 comp rx=0 tx=0).stop 2024-09-16T00:11:15.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.502+0000 7f9ff78c9640 1 -- 172.21.15.139:0/2978858235 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ff0068060 msgr2=0x7f9ff0153b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.502+0000 7f9ff78c9640 1 --2- 172.21.15.139:0/2978858235 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ff0068060 0x7f9ff0153b80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00b3e0 tx=0x7f9fec00b8a0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.503+0000 7f9ff78c9640 1 -- 172.21.15.139:0/2978858235 shutdown_connections 2024-09-16T00:11:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.503+0000 7f9ff78c9640 1 --2- 172.21.15.139:0/2978858235 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f9fd406c2b0 0x7f9fd406e770 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.503+0000 7f9ff78c9640 1 --2- 172.21.15.139:0/2978858235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff0154c20 0x7f9ff015bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.503+0000 7f9ff78c9640 1 --2- 172.21.15.139:0/2978858235 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9ff00ccc90 0x7f9ff01540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.503+0000 7f9ff78c9640 1 --2- 172.21.15.139:0/2978858235 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f9ff0068060 0x7f9ff0153b80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.503+0000 7f9ff78c9640 1 -- 172.21.15.139:0/2978858235 >> 172.21.15.139:0/2978858235 conn(0x7f9ff006ca40 msgr2=0x7f9ff0158850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.503+0000 7f9ff78c9640 1 -- 172.21.15.139:0/2978858235 shutdown_connections 2024-09-16T00:11:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.503+0000 7f9ff78c9640 1 -- 172.21.15.139:0/2978858235 wait complete. 2024-09-16T00:11:15.506 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.503+0000 7f8ddcff9640 1 -- 172.21.15.139:0/2544517264 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8dd80592e0 con 0x7f8de0150ad0 2024-09-16T00:11:15.507 INFO:teuthology.orchestra.run.smithi139.stdout:64424509488 2024-09-16T00:11:15.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542194 got 60129542193 for osd.1 2024-09-16T00:11:15.509 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.507+0000 7f8de645a640 1 -- 172.21.15.139:0/2544517264 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8dcc06c330 msgr2=0x7f8dcc06e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.507+0000 7f8de645a640 1 --2- 172.21.15.139:0/2544517264 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8dcc06c330 0x7f8dcc06e7f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4035000 tx=0x7f8dd4008040 comp rx=0 tx=0).stop 2024-09-16T00:11:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.507+0000 7f8de645a640 1 -- 172.21.15.139:0/2544517264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8de0150ad0 msgr2=0x7f8de01573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.507+0000 7f8de645a640 1 --2- 172.21.15.139:0/2544517264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8de0150ad0 0x7f8de01573e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8dd800c7c0 tx=0x7f8dd800cc80 comp rx=0 tx=0).stop 2024-09-16T00:11:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.507+0000 7f8de645a640 1 -- 172.21.15.139:0/2544517264 shutdown_connections 2024-09-16T00:11:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.507+0000 7f8de645a640 1 --2- 172.21.15.139:0/2544517264 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f8dcc06c330 0x7f8dcc06e7f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.507+0000 7f8de645a640 1 --2- 172.21.15.139:0/2544517264 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8de0150ad0 0x7f8de01573e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.507+0000 7f8de645a640 1 --2- 172.21.15.139:0/2544517264 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8de00cc690 0x7f8de01fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.507+0000 7f8de645a640 1 --2- 172.21.15.139:0/2544517264 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8de006fc20 0x7f8de00ba730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.507+0000 7f8de645a640 1 -- 172.21.15.139:0/2544517264 >> 172.21.15.139:0/2544517264 conn(0x7f8de006da40 msgr2=0x7f8de00c7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.507+0000 7f8de645a640 1 -- 172.21.15.139:0/2544517264 shutdown_connections 2024-09-16T00:11:15.510 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.507+0000 7f8de645a640 1 -- 172.21.15.139:0/2544517264 wait complete. 2024-09-16T00:11:15.511 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509490 got 64424509489 for osd.5 2024-09-16T00:11:15.515 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574898 got 55834574897 for osd.2 2024-09-16T00:11:15.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542194 got 60129542193 for osd.0 2024-09-16T00:11:15.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.518+0000 7f60bb29b640 1 -- 172.21.15.139:0/2586254182 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f60b40c7d40 con 0x7f60b406fc20 2024-09-16T00:11:15.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.519+0000 7f60a9ffb640 1 -- 172.21.15.139:0/2586254182 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f60a40799d0 con 0x7f60b406fc20 2024-09-16T00:11:15.521 INFO:teuthology.orchestra.run.smithi139.stdout:545460846597 2024-09-16T00:11:15.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.521+0000 7f60bb29b640 1 -- 172.21.15.139:0/2586254182 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f609806c2b0 msgr2=0x7f609806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.521+0000 7f60bb29b640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f609806c2b0 0x7f609806e770 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f60b4153050 tx=0x7f60b0008040 comp rx=0 tx=0).stop 2024-09-16T00:11:15.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.521+0000 7f60bb29b640 1 -- 172.21.15.139:0/2586254182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60b406fc20 msgr2=0x7f60b415e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:15.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.521+0000 7f60bb29b640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60b406fc20 0x7f60b415e2b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f60a40098c0 tx=0x7f60a40046c0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.523 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.521+0000 7f60bb29b640 1 -- 172.21.15.139:0/2586254182 shutdown_connections 2024-09-16T00:11:15.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.521+0000 7f60bb29b640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f609806c2b0 0x7f609806e770 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.521+0000 7f60bb29b640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f60b4158f00 0x7f60b4152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.521+0000 7f60bb29b640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f60b4070040 0x7f60b415c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.521+0000 7f60bb29b640 1 --2- 172.21.15.139:0/2586254182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60b406fc20 0x7f60b415e2b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:15.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.521+0000 7f60bb29b640 1 -- 172.21.15.139:0/2586254182 >> 172.21.15.139:0/2586254182 conn(0x7f60b406da40 msgr2=0x7f60b40c5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:15.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.521+0000 7f60bb29b640 1 -- 172.21.15.139:0/2586254182 shutdown_connections 2024-09-16T00:11:15.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:15.521+0000 7f60bb29b640 1 -- 172.21.15.139:0/2586254182 wait complete. 2024-09-16T00:11:15.525 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509489 got 64424509488 for osd.3 2024-09-16T00:11:15.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 545460846598 got 545460846597 for osd.7 2024-09-16T00:11:16.484 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-16T00:11:16.493 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-16T00:11:16.510 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T00:11:16.512 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-16T00:11:16.515 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T00:11:16.521 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T00:11:16.526 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-16T00:11:16.536 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-16T00:11:16.631 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.627+0000 7f6d32123640 1 Processor -- start 2024-09-16T00:11:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.627+0000 7f6d32123640 1 -- start start 2024-09-16T00:11:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.628+0000 7f6d32123640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6d2c152620 0x7f6d2c070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.628+0000 7f6d32123640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6d2c0c9bb0 0x7f6d2c152a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.628+0000 7f6d32123640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c0c82d0 0x7f6d2c1585e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.628+0000 7f6d32123640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6d2c0c53f0 con 0x7f6d2c0c9bb0 2024-09-16T00:11:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.628+0000 7f6d32123640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f6d2c0c9720 con 0x7f6d2c152620 2024-09-16T00:11:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.628+0000 7f6d32123640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6d2c0c9420 con 0x7f6d2c0c82d0 2024-09-16T00:11:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.628+0000 7f6d2affd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6d2c0c9bb0 0x7f6d2c152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.628+0000 7f6d2affd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6d2c0c9bb0 0x7f6d2c152a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60526/0 (socket says 172.21.15.139:60526) 2024-09-16T00:11:16.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.628+0000 7f6d2affd640 1 -- 172.21.15.139:0/3313322725 learned_addr learned my addr 172.21.15.139:0/3313322725 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:16.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.629+0000 7f6d2affd640 1 -- 172.21.15.139:0/3313322725 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c0c82d0 msgr2=0x7f6d2c1585e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:11:16.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.629+0000 7f6d2affd640 1 --2- 172.21.15.139:0/3313322725 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c0c82d0 0x7f6d2c1585e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.629+0000 7f6d2affd640 1 -- 172.21.15.139:0/3313322725 >> v2:172.21.15.164:3300/0 conn(0x7f6d2c152620 msgr2=0x7f6d2c070180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:11:16.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.629+0000 7f6d2affd640 1 --2- 172.21.15.139:0/3313322725 >> v2:172.21.15.164:3300/0 conn(0x7f6d2c152620 0x7f6d2c070180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.629+0000 7f6d2affd640 1 -- 172.21.15.139:0/3313322725 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2c0c8e10 con 0x7f6d2c0c9bb0 2024-09-16T00:11:16.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.630+0000 7f6d2affd640 1 --2- 172.21.15.139:0/3313322725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d2c0c9bb0 0x7f6d2c152a00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0cbd60 tx=0x7f6d1c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23ea2bc5364e823 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.631+0000 7f6d2a7fc640 1 -- 172.21.15.139:0/3313322725 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c037070 con 0x7f6d2c0c9bb0 2024-09-16T00:11:16.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.631+0000 7f6d2a7fc640 1 -- 172.21.15.139:0/3313322725 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d1c032440 con 0x7f6d2c0c9bb0 2024-09-16T00:11:16.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.631+0000 7f6d2a7fc640 1 -- 172.21.15.139:0/3313322725 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c032810 con 0x7f6d2c0c9bb0 2024-09-16T00:11:16.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.632+0000 7f6d32123640 1 -- 172.21.15.139:0/3313322725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d2c0c9bb0 msgr2=0x7f6d2c152a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.632+0000 7f6d32123640 1 --2- 172.21.15.139:0/3313322725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d2c0c9bb0 0x7f6d2c152a00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0cbd60 tx=0x7f6d1c02a210 comp rx=0 tx=0).stop 2024-09-16T00:11:16.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.632+0000 7f6d32123640 1 -- 172.21.15.139:0/3313322725 shutdown_connections 2024-09-16T00:11:16.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.632+0000 7f6d32123640 1 --2- 172.21.15.139:0/3313322725 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c0c82d0 0x7f6d2c1585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.632+0000 7f6d32123640 1 --2- 172.21.15.139:0/3313322725 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d2c0c9bb0 0x7f6d2c152a00 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.632+0000 7f6d32123640 1 --2- 172.21.15.139:0/3313322725 >> v2:172.21.15.164:3300/0 conn(0x7f6d2c152620 0x7f6d2c070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.632+0000 7f6d32123640 1 -- 172.21.15.139:0/3313322725 >> 172.21.15.139:0/3313322725 conn(0x7f6d2c06c150 msgr2=0x7f6d2c067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:16.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.632+0000 7f6d32123640 1 -- 172.21.15.139:0/3313322725 shutdown_connections 2024-09-16T00:11:16.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.632+0000 7f6d32123640 1 -- 172.21.15.139:0/3313322725 wait complete. 2024-09-16T00:11:16.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.634+0000 7f6d32123640 1 Processor -- start 2024-09-16T00:11:16.637 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.634+0000 7f6d32123640 1 -- start start 2024-09-16T00:11:16.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.636+0000 7f6d32123640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d2c0c82d0 0x7f6d2c0c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.636+0000 7f6d32123640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d2c0c9bb0 0x7f6d2c1e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.636+0000 7f6d32123640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c152620 0x7f6d2c1e2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.636+0000 7f6d2affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d2c0c9bb0 0x7f6d2c1e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.636+0000 7f6d2affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d2c0c9bb0 0x7f6d2c1e22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60530/0 (socket says 172.21.15.139:60530) 2024-09-16T00:11:16.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.636+0000 7f6d32123640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c0c3620 con 0x7f6d2c0c9bb0 2024-09-16T00:11:16.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.636+0000 7f6d32123640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c0c3340 con 0x7f6d2c0c82d0 2024-09-16T00:11:16.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.636+0000 7f6d32123640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f6d2c158b20 con 0x7f6d2c152620 2024-09-16T00:11:16.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.636+0000 7f6d2affd640 1 -- 172.21.15.139:0/294912180 learned_addr learned my addr 172.21.15.139:0/294912180 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:16.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.636+0000 7f6d2bfff640 1 --2- 172.21.15.139:0/294912180 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c152620 0x7f6d2c1e2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.637+0000 7f6d2affd640 1 -- 172.21.15.139:0/294912180 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c152620 msgr2=0x7f6d2c1e2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.637+0000 7f6d2affd640 1 --2- 172.21.15.139:0/294912180 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c152620 0x7f6d2c1e2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.637+0000 7f6d2affd640 1 -- 172.21.15.139:0/294912180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d2c0c82d0 msgr2=0x7f6d2c0c7340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:11:16.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.637+0000 7f6d2affd640 1 --2- 172.21.15.139:0/294912180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d2c0c82d0 0x7f6d2c0c7340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.637+0000 7f6d2affd640 1 -- 172.21.15.139:0/294912180 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2c06ec40 con 0x7f6d2c0c9bb0 2024-09-16T00:11:16.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.637+0000 7f6d2affd640 1 --2- 172.21.15.139:0/294912180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d2c0c9bb0 0x7f6d2c1e22c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009960 tx=0x7f6d1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.638+0000 7f6d13fff640 1 -- 172.21.15.139:0/294912180 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c041070 con 0x7f6d2c0c9bb0 2024-09-16T00:11:16.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.638+0000 7f6d13fff640 1 -- 172.21.15.139:0/294912180 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d1c030970 con 0x7f6d2c0c9bb0 2024-09-16T00:11:16.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.639+0000 7f6d13fff640 1 -- 172.21.15.139:0/294912180 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c037040 con 0x7f6d2c0c9bb0 2024-09-16T00:11:16.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.639+0000 7f6d32123640 1 -- 172.21.15.139:0/294912180 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2c1e35b0 con 0x7f6d2c0c9bb0 2024-09-16T00:11:16.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.639+0000 7f6d32123640 1 -- 172.21.15.139:0/294912180 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d2c1e3e50 con 0x7f6d2c0c9bb0 2024-09-16T00:11:16.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.643+0000 7f6d13fff640 1 -- 172.21.15.139:0/294912180 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6d1c044070 con 0x7f6d2c0c9bb0 2024-09-16T00:11:16.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.644+0000 7f6d13fff640 1 --2- 172.21.15.139:0/294912180 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6d1806c2b0 0x7f6d1806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.644+0000 7f6d13fff640 1 -- 172.21.15.139:0/294912180 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f6d1c0abfa0 con 0x7f6d2c0c9bb0 2024-09-16T00:11:16.647 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.644+0000 7f6d2b7fe640 1 --2- 172.21.15.139:0/294912180 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6d1806c2b0 0x7f6d1806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.645+0000 7f6d2b7fe640 1 --2- 172.21.15.139:0/294912180 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6d1806c2b0 0x7f6d1806e770 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6d200096c0 tx=0x7f6d20006bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.646+0000 7f6d11ffb640 1 -- 172.21.15.139:0/294912180 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d04005290 con 0x7f6d2c0c9bb0 2024-09-16T00:11:16.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.655+0000 7f6d13fff640 1 -- 172.21.15.139:0/294912180 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6d2c0c8e10 con 0x7f6d2c0c9bb0 2024-09-16T00:11:16.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.691+0000 7faf0db2b640 1 Processor -- start 2024-09-16T00:11:16.694 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.691+0000 7faf0db2b640 1 -- start start 2024-09-16T00:11:16.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.693+0000 7faf0db2b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faf000d5850 0x7faf00007730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.693+0000 7faf0db2b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf0004d310 0x7faf00045d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.693+0000 7faf0db2b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf000dc3a0 con 0x7faf00047a80 2024-09-16T00:11:16.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.693+0000 7faf0db2b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7faf000dbe90 con 0x7faf000d5850 2024-09-16T00:11:16.695 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.693+0000 7faf0db2b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7faf000dc0c0 con 0x7faf0004d310 2024-09-16T00:11:16.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.693+0000 7faf0d32a640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7faf00047a80 0x7faf000dea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46236/0 (socket says 172.21.15.139:46236) 2024-09-16T00:11:16.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.693+0000 7faf0d32a640 1 -- 172.21.15.139:0/3685876912 learned_addr learned my addr 172.21.15.139:0/3685876912 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:16.696 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.694+0000 7f0086b59640 1 Processor -- start 2024-09-16T00:11:16.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.693+0000 7faf0cb29640 1 --2- 172.21.15.139:0/3685876912 >> v2:172.21.15.139:3300/0 conn(0x7faf000d5850 0x7faf00007730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.693+0000 7f0d2c576640 1 Processor -- start 2024-09-16T00:11:16.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.693+0000 7f0d2c576640 1 -- start start 2024-09-16T00:11:16.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7f0d2c576640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0d241529c0 0x7f0d2415c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.694+0000 7f0086b59640 1 -- start start 2024-09-16T00:11:16.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7f0086b59640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f00800ca740 0x7f00800c7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7f0086b59640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00800c7e40 0x7f0080162220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7f0086b59640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00800c83b0 con 0x7f00800cab20 2024-09-16T00:11:16.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7f0086b59640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f008006eb50 con 0x7f00800ca740 2024-09-16T00:11:16.698 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7f0086b59640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f00800cb700 con 0x7f00800c7e40 2024-09-16T00:11:16.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7f0d2c576640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0d240c42e0 0x7f0d240ca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7f0d2c576640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d240c4730 con 0x7f0d24068380 2024-09-16T00:11:16.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7f0d2c576640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0d240c9560 con 0x7f0d241529c0 2024-09-16T00:11:16.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7f0d2c576640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0d240c35b0 con 0x7f0d240c42e0 2024-09-16T00:11:16.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.694+0000 7faf07fff640 1 --2- 172.21.15.139:0/3685876912 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf0004d310 0x7faf00045d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.699 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.694+0000 7faf077fe640 1 -- 172.21.15.139:0/3685876912 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1332314190 0 0) 0x7faf000dc3a0 con 0x7faf00047a80 2024-09-16T00:11:16.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.694+0000 7faf077fe640 1 -- 172.21.15.139:0/3685876912 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faef8003520 con 0x7faf00047a80 2024-09-16T00:11:16.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7faf0cb29640 1 -- 172.21.15.139:0/3685876912 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf0004d310 msgr2=0x7faf00045d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:16.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7faf0cb29640 1 --2- 172.21.15.139:0/3685876912 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf0004d310 0x7faf00045d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7faf0cb29640 1 -- 172.21.15.139:0/3685876912 >> v1:172.21.15.139:6789/0 conn(0x7faf00047a80 legacy=0x7faf000dea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7faf0cb29640 1 -- 172.21.15.139:0/3685876912 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf00046cc0 con 0x7faf000d5850 2024-09-16T00:11:16.700 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.696+0000 7f00850f6640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00800c7e40 0x7f0080162220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7f0d2a312640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0d241529c0 0x7f0d2415c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7f0d2a312640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0d241529c0 0x7f0d2415c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60546/0 (socket says 172.21.15.139:60546) 2024-09-16T00:11:16.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7f0d2a312640 1 -- 172.21.15.139:0/1481491521 learned_addr learned my addr 172.21.15.139:0/1481491521 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:16.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.695+0000 7f0d2ab13640 1 --2- 172.21.15.139:0/1481491521 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0d240c42e0 0x7f0d240ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.696+0000 7f0d2a312640 1 -- 172.21.15.139:0/1481491521 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0d240c42e0 msgr2=0x7f0d240ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:16.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.696+0000 7f0d2a312640 1 --2- 172.21.15.139:0/1481491521 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0d240c42e0 0x7f0d240ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.696+0000 7f0d2a312640 1 -- 172.21.15.139:0/1481491521 >> v1:172.21.15.164:6789/0 conn(0x7f0d24068380 legacy=0x7f0d240ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:16.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.696+0000 7f0d2a312640 1 -- 172.21.15.139:0/1481491521 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d240b1690 con 0x7f0d241529c0 2024-09-16T00:11:16.701 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.696+0000 7faf0cb29640 1 --2- 172.21.15.139:0/3685876912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf000d5850 0x7faf00007730 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7faefc0097a0 tx=0x7faefc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42ba59f6a624b33 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.696+0000 7f007ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f00800cab20 0x7f00800c58b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:46242/0 (socket says 172.21.15.139:46242) 2024-09-16T00:11:16.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.696+0000 7f007ffff640 1 -- 172.21.15.139:0/2801101127 learned_addr learned my addr 172.21.15.139:0/2801101127 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:16.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.696+0000 7f00848f5640 1 --2- 172.21.15.139:0/2801101127 >> v2:172.21.15.139:3300/0 conn(0x7f00800ca740 0x7f00800c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.702 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.696+0000 7f00848f5640 1 -- 172.21.15.139:0/2801101127 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00800c7e40 msgr2=0x7f0080162220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:16.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.696+0000 7f00848f5640 1 --2- 172.21.15.139:0/2801101127 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00800c7e40 0x7f0080162220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.696+0000 7f00848f5640 1 -- 172.21.15.139:0/2801101127 >> v1:172.21.15.139:6789/0 conn(0x7f00800cab20 legacy=0x7f00800c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.696+0000 7f00848f5640 1 -- 172.21.15.139:0/2801101127 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00800cb950 con 0x7f00800ca740 2024-09-16T00:11:16.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.697+0000 7faf077fe640 1 -- 172.21.15.139:0/3685876912 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc037070 con 0x7faf000d5850 2024-09-16T00:11:16.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.697+0000 7faf077fe640 1 -- 172.21.15.139:0/3685876912 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faefc032440 con 0x7faf000d5850 2024-09-16T00:11:16.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.697+0000 7fce27778640 1 Processor -- start 2024-09-16T00:11:16.704 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.697+0000 7f00848f5640 1 --2- 172.21.15.139:0/2801101127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00800ca740 0x7f00800c7900 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0070009d90 tx=0x7f007002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e68b75060149635 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.698+0000 7faf0db2b640 1 -- 172.21.15.139:0/3685876912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf000d5850 msgr2=0x7faf00007730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.698+0000 7faf0db2b640 1 --2- 172.21.15.139:0/3685876912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf000d5850 0x7faf00007730 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7faefc0097a0 tx=0x7faefc02a180 comp rx=0 tx=0).stop 2024-09-16T00:11:16.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.698+0000 7faf0db2b640 1 -- 172.21.15.139:0/3685876912 shutdown_connections 2024-09-16T00:11:16.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.698+0000 7faf0db2b640 1 --2- 172.21.15.139:0/3685876912 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf0004d310 0x7faf00045d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.698+0000 7faf0db2b640 1 --2- 172.21.15.139:0/3685876912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf000d5850 0x7faf00007730 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.698+0000 7faf0db2b640 1 -- 172.21.15.139:0/3685876912 >> 172.21.15.139:0/3685876912 conn(0x7faf0004d830 msgr2=0x7faf0004dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:16.705 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.698+0000 7f0d2a312640 1 --2- 172.21.15.139:0/1481491521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d241529c0 0x7f0d2415c7a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009ed0 tx=0x7f0d1c032fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a892b603cd8f219d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.697+0000 7fce27778640 1 -- start start 2024-09-16T00:11:16.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.698+0000 7f0d29310640 1 -- 172.21.15.139:0/1481491521 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c033aa0 con 0x7f0d241529c0 2024-09-16T00:11:16.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.698+0000 7f0d29310640 1 -- 172.21.15.139:0/1481491521 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d1c033c40 con 0x7f0d241529c0 2024-09-16T00:11:16.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.699+0000 7f0d29310640 1 -- 172.21.15.139:0/1481491521 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c03b3f0 con 0x7f0d241529c0 2024-09-16T00:11:16.706 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.699+0000 7f007f7fe640 1 -- 172.21.15.139:0/2801101127 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070007740 con 0x7f00800ca740 2024-09-16T00:11:16.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.699+0000 7fce27778640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fce200cc330 0x7fce201572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.699+0000 7fce27778640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fce200c8ad0 0x7fce20150630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.699+0000 7fce27778640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce200c9b40 con 0x7fce2006de20 2024-09-16T00:11:16.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.699+0000 7fce27778640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fce200720c0 con 0x7fce200c8ad0 2024-09-16T00:11:16.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.699+0000 7fce27778640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fce200c8fc0 con 0x7fce200cc330 2024-09-16T00:11:16.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.699+0000 7f007f7fe640 1 -- 172.21.15.139:0/2801101127 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00700078e0 con 0x7f00800ca740 2024-09-16T00:11:16.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.699+0000 7fce24d13640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fce2006de20 0x7fce20072550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:44914/0 (socket says 172.21.15.139:44914) 2024-09-16T00:11:16.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.699+0000 7fce24d13640 1 -- 172.21.15.139:0/2285098011 learned_addr learned my addr 172.21.15.139:0/2285098011 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:16.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.699+0000 7fce25514640 1 --2- 172.21.15.139:0/2285098011 >> v2:172.21.15.164:3300/0 conn(0x7fce200cc330 0x7fce201572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.699+0000 7fce25d15640 1 --2- 172.21.15.139:0/2285098011 >> v2:172.21.15.139:3300/0 conn(0x7fce200c8ad0 0x7fce20150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.700+0000 7f6657084640 1 Processor -- start 2024-09-16T00:11:16.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.699+0000 7f0d2c576640 1 -- 172.21.15.139:0/1481491521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d241529c0 msgr2=0x7f0d2415c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.699+0000 7f0d2c576640 1 --2- 172.21.15.139:0/1481491521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d241529c0 0x7f0d2415c7a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009ed0 tx=0x7f0d1c032fa0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.700+0000 7f6657084640 1 -- start start 2024-09-16T00:11:16.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.700+0000 7fce17fff640 1 -- 172.21.15.139:0/2285098011 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1484759349 0 0) 0x7fce200c9b40 con 0x7fce2006de20 2024-09-16T00:11:16.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.700+0000 7fce17fff640 1 -- 172.21.15.139:0/2285098011 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce0c003520 con 0x7fce2006de20 2024-09-16T00:11:16.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.700+0000 7f0d2c576640 1 -- 172.21.15.139:0/1481491521 shutdown_connections 2024-09-16T00:11:16.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.700+0000 7f0d2c576640 1 --2- 172.21.15.139:0/1481491521 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0d240c42e0 0x7f0d240ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.700+0000 7f0d2c576640 1 --2- 172.21.15.139:0/1481491521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d241529c0 0x7f0d2415c7a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.700+0000 7f0d2c576640 1 -- 172.21.15.139:0/1481491521 >> 172.21.15.139:0/1481491521 conn(0x7f0d2406bb20 msgr2=0x7f0d24067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:16.710 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.698+0000 7faf0db2b640 1 -- 172.21.15.139:0/3685876912 shutdown_connections 2024-09-16T00:11:16.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.698+0000 7faf0db2b640 1 -- 172.21.15.139:0/3685876912 wait complete. 2024-09-16T00:11:16.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.700+0000 7fce25514640 1 -- 172.21.15.139:0/2285098011 >> v2:172.21.15.139:3300/0 conn(0x7fce200c8ad0 msgr2=0x7fce20150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:16.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.700+0000 7fce17fff640 1 -- 172.21.15.139:0/2285098011 <== mon.1 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3472440565 0 0) 0x7fce0c003520 con 0x7fce2006de20 2024-09-16T00:11:16.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.700+0000 7fce25514640 1 --2- 172.21.15.139:0/2285098011 >> v2:172.21.15.139:3300/0 conn(0x7fce200c8ad0 0x7fce20150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.700+0000 7fce25514640 1 -- 172.21.15.139:0/2285098011 >> v1:172.21.15.164:6789/0 conn(0x7fce2006de20 legacy=0x7fce20072550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.700+0000 7faf0db2b640 1 Processor -- start 2024-09-16T00:11:16.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.700+0000 7faf0db2b640 1 -- start start 2024-09-16T00:11:16.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.700+0000 7fce25514640 1 -- 172.21.15.139:0/2285098011 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce200c92a0 con 0x7fce200cc330 2024-09-16T00:11:16.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7f0086b59640 1 -- 172.21.15.139:0/2801101127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00800ca740 msgr2=0x7f00800c7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7f0086b59640 1 --2- 172.21.15.139:0/2801101127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00800ca740 0x7f00800c7900 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0070009d90 tx=0x7f007002a660 comp rx=0 tx=0).stop 2024-09-16T00:11:16.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7f0086b59640 1 -- 172.21.15.139:0/2801101127 shutdown_connections 2024-09-16T00:11:16.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7fce25d15640 1 --2- 172.21.15.139:0/2285098011 >> v2:172.21.15.139:3300/0 conn(0x7fce200c8ad0 0x7fce20150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:11:16.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7faf0db2b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf00047a80 0x7faf000512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7faf0db2b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf0004d310 0x7faf00051830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7faf0db2b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf000d5850 0x7faf00048130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.713 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7faf0db2b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faf00047560 con 0x7faf00047a80 2024-09-16T00:11:16.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7faf0db2b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf00047280 con 0x7faf000d5850 2024-09-16T00:11:16.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7faf0db2b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7faf000468c0 con 0x7faf0004d310 2024-09-16T00:11:16.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7f6657084640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f665015e0d0 0x7f6650072900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7f6657084640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f665015e4b0 0x7f6650072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7f6657084640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f665015e990 con 0x7f665006f910 2024-09-16T00:11:16.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7f6657084640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f665015ec70 con 0x7f665015e0d0 2024-09-16T00:11:16.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7f6657084640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f66500c8500 con 0x7f665015e4b0 2024-09-16T00:11:16.714 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7f0086b59640 1 --2- 172.21.15.139:0/2801101127 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00800c7e40 0x7f0080162220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7f0086b59640 1 --2- 172.21.15.139:0/2801101127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00800ca740 0x7f00800c7900 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7f0086b59640 1 -- 172.21.15.139:0/2801101127 >> 172.21.15.139:0/2801101127 conn(0x7f00800c91d0 msgr2=0x7f00800c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:16.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7f0d2c576640 1 -- 172.21.15.139:0/1481491521 shutdown_connections 2024-09-16T00:11:16.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.701+0000 7f0d2c576640 1 -- 172.21.15.139:0/1481491521 wait complete. 2024-09-16T00:11:16.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7faf0cb29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf00047a80 0x7faf000512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7fb556952640 1 Processor -- start 2024-09-16T00:11:16.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7f6655881640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f665006f910 0x7f66500b1ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:44920/0 (socket says 172.21.15.139:44920) 2024-09-16T00:11:16.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7f6655881640 1 -- 172.21.15.139:0/3035069048 learned_addr learned my addr 172.21.15.139:0/3035069048 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:16.716 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7f0086b59640 1 -- 172.21.15.139:0/2801101127 shutdown_connections 2024-09-16T00:11:16.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7fce25514640 1 --2- 172.21.15.139:0/2285098011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce200cc330 0x7fce201572f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fce200c9d70 tx=0x7fce1002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe806182c306100f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7faf0cb29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf00047a80 0x7faf000512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60568/0 (socket says 172.21.15.139:60568) 2024-09-16T00:11:16.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7faf0cb29640 1 -- 172.21.15.139:0/3426804660 learned_addr learned my addr 172.21.15.139:0/3426804660 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:16.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7faf0d32a640 1 --2- 172.21.15.139:0/3426804660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf000d5850 0x7faf00048130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.717 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7fb556952640 1 -- start start 2024-09-16T00:11:16.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7f6655080640 1 -- 172.21.15.139:0/3035069048 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3041475423 0 0) 0x7f665015e990 con 0x7f665006f910 2024-09-16T00:11:16.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7f6655080640 1 -- 172.21.15.139:0/3035069048 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f663c003520 con 0x7f665006f910 2024-09-16T00:11:16.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7fce17fff640 1 -- 172.21.15.139:0/2285098011 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce10037070 con 0x7fce200cc330 2024-09-16T00:11:16.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7fce17fff640 1 -- 172.21.15.139:0/2285098011 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce1002ac50 con 0x7fce200cc330 2024-09-16T00:11:16.718 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7fce17fff640 1 -- 172.21.15.139:0/2285098011 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce180019f0 con 0x7fce200cc330 2024-09-16T00:11:16.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7faf0cb29640 1 -- 172.21.15.139:0/3426804660 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf0004d310 msgr2=0x7faf00051830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:11:16.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7faf0cb29640 1 --2- 172.21.15.139:0/3426804660 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf0004d310 0x7faf00051830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7faf0cb29640 1 -- 172.21.15.139:0/3426804660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf000d5850 msgr2=0x7faf00048130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7faf0cb29640 1 --2- 172.21.15.139:0/3426804660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf000d5850 0x7faf00048130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.719 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7faf0cb29640 1 -- 172.21.15.139:0/3426804660 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf000df770 con 0x7faf00047a80 2024-09-16T00:11:16.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7f6656082640 1 --2- 172.21.15.139:0/3035069048 >> v2:172.21.15.164:3300/0 conn(0x7f665015e0d0 0x7f6650072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7f6655080640 1 -- 172.21.15.139:0/3035069048 <== mon.1 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2024499359 0 0) 0x7f663c003520 con 0x7f665006f910 2024-09-16T00:11:16.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7f6655080640 1 -- 172.21.15.139:0/3035069048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f665015e4b0 msgr2=0x7f6650072e40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:11:16.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7f6655080640 1 --2- 172.21.15.139:0/3035069048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f665015e4b0 0x7f6650072e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7f6655080640 1 -- 172.21.15.139:0/3035069048 >> v2:172.21.15.164:3300/0 conn(0x7f665015e0d0 msgr2=0x7f6650072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:16.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.702+0000 7f6655080640 1 --2- 172.21.15.139:0/3035069048 >> v2:172.21.15.164:3300/0 conn(0x7f665015e0d0 0x7f6650072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7f6655080640 1 -- 172.21.15.139:0/3035069048 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66500c87e0 con 0x7f665006f910 2024-09-16T00:11:16.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7f6655080640 1 -- 172.21.15.139:0/3035069048 <== mon.1 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7f664c001690 con 0x7f665006f910 2024-09-16T00:11:16.720 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7f6656082640 1 --2- 172.21.15.139:0/3035069048 >> v2:172.21.15.164:3300/0 conn(0x7f665015e0d0 0x7f6650072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:16.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7f6655080640 1 -- 172.21.15.139:0/3035069048 >> v1:172.21.15.164:6789/0 conn(0x7f665006f910 legacy=0x7f66500b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7faf0cb29640 1 --2- 172.21.15.139:0/3426804660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf00047a80 0x7faf000512f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7faefc000c00 tx=0x7faefc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7f6655080640 1 --2- 172.21.15.139:0/3035069048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f663c0039b0 0x7f663c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.721 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7fb556952640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb5500ca740 0x7fb5500c7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7f6655080640 1 --2- 172.21.15.139:0/3035069048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f663c004800 0x7f663c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7f6655080640 1 --2- 172.21.15.139:0/3035069048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f665015e4b0 0x7f663c005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7f6655080640 1 -- 172.21.15.139:0/3035069048 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f665015ec70 con 0x7f663c004800 2024-09-16T00:11:16.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7f6655080640 1 -- 172.21.15.139:0/3035069048 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f66500c8500 con 0x7f663c0039b0 2024-09-16T00:11:16.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7f6655080640 1 -- 172.21.15.139:0/3035069048 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f663c00bcc0 con 0x7f665015e4b0 2024-09-16T00:11:16.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7fce27778640 1 -- 172.21.15.139:0/2285098011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce200cc330 msgr2=0x7fce201572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.722 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7fce27778640 1 --2- 172.21.15.139:0/2285098011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce200cc330 0x7fce201572f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fce200c9d70 tx=0x7fce1002a1c0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7f0086b59640 1 -- 172.21.15.139:0/2801101127 wait complete. 2024-09-16T00:11:16.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7fb556952640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5500cab20 0x7fb5500c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7fb556952640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5500c83b0 con 0x7fb5500c7e40 2024-09-16T00:11:16.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7fb556952640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb55006eb50 con 0x7fb5500ca740 2024-09-16T00:11:16.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7fb554eef640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fb5500c7e40 0x7fb550162220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:44928/0 (socket says 172.21.15.139:44928) 2024-09-16T00:11:16.723 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7fb554eef640 1 -- 172.21.15.139:0/37108450 learned_addr learned my addr 172.21.15.139:0/37108450 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:16.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7fb556952640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb5500cb700 con 0x7fb5500cab20 2024-09-16T00:11:16.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7faf057fa640 1 -- 172.21.15.139:0/3426804660 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc037050 con 0x7faf00047a80 2024-09-16T00:11:16.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7faf0db2b640 1 -- 172.21.15.139:0/3426804660 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf00008970 con 0x7faf00047a80 2024-09-16T00:11:16.724 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7f6656082640 1 --2- 172.21.15.139:0/3035069048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f663c0039b0 0x7f663c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7faf057fa640 1 -- 172.21.15.139:0/3426804660 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faefc038040 con 0x7faf00047a80 2024-09-16T00:11:16.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7faf0db2b640 1 -- 172.21.15.139:0/3426804660 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf0004ad70 con 0x7faf00047a80 2024-09-16T00:11:16.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7f6656082640 1 -- 172.21.15.139:0/3035069048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f665015e4b0 msgr2=0x7f663c005290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:11:16.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7f6656082640 1 --2- 172.21.15.139:0/3035069048 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f665015e4b0 0x7f663c005290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7f6656082640 1 -- 172.21.15.139:0/3035069048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f663c004800 msgr2=0x7f663c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:16.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7f6656082640 1 --2- 172.21.15.139:0/3035069048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f663c004800 0x7f663c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.725 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7f6656082640 1 -- 172.21.15.139:0/3035069048 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663c003520 con 0x7f663c0039b0 2024-09-16T00:11:16.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7fb54ffff640 1 --2- 172.21.15.139:0/37108450 >> v2:172.21.15.139:3300/0 conn(0x7fb5500ca740 0x7fb5500c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7fb54effd640 1 -- 172.21.15.139:0/37108450 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 840114852 0 0) 0x7fb5500c83b0 con 0x7fb5500c7e40 2024-09-16T00:11:16.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7fb54effd640 1 -- 172.21.15.139:0/37108450 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb538003520 con 0x7fb5500c7e40 2024-09-16T00:11:16.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7faf057fa640 1 -- 172.21.15.139:0/3426804660 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faefc037050 con 0x7faf00047a80 2024-09-16T00:11:16.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7f0086b59640 1 Processor -- start 2024-09-16T00:11:16.726 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7f0086b59640 1 -- start start 2024-09-16T00:11:16.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7f0d2c576640 1 Processor -- start 2024-09-16T00:11:16.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7fb54ffff640 1 -- 172.21.15.139:0/37108450 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5500cab20 msgr2=0x7fb5500c58b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:11:16.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7fb54ffff640 1 --2- 172.21.15.139:0/37108450 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5500cab20 0x7fb5500c58b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7fb54ffff640 1 -- 172.21.15.139:0/37108450 >> v1:172.21.15.164:6789/0 conn(0x7fb5500c7e40 legacy=0x7fb550162220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.727 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7fb54ffff640 1 -- 172.21.15.139:0/37108450 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5500cb950 con 0x7fb5500ca740 2024-09-16T00:11:16.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.703+0000 7f0d2c576640 1 -- start start 2024-09-16T00:11:16.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7f0d2c576640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0d24068380 0x7f0d241f03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7f0d2c576640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d240c42e0 0x7f0d240ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7f0d2c576640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d241529c0 0x7f0d241ec9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7f0d2c576640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d240c3800 con 0x7f0d240c42e0 2024-09-16T00:11:16.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.704+0000 7f0d2c576640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0d240c9a90 con 0x7f0d241529c0 2024-09-16T00:11:16.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7fb54ffff640 1 --2- 172.21.15.139:0/37108450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5500ca740 0x7fb5500c7900 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb5400097a0 tx=0x7fb54002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd31b46e5fb626d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7fb54effd640 1 -- 172.21.15.139:0/37108450 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540037070 con 0x7fb5500ca740 2024-09-16T00:11:16.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7fb54effd640 1 -- 172.21.15.139:0/37108450 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb540032440 con 0x7fb5500ca740 2024-09-16T00:11:16.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7f0086b59640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00800c7e40 0x7f0080139750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7f6656082640 1 --2- 172.21.15.139:0/3035069048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f663c0039b0 0x7f663c003d90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f664000aa80 tx=0x7f663c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff7466a86831be60 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7f0d2c576640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0d240c3ae0 con 0x7f0d24068380 2024-09-16T00:11:16.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7fb54effd640 1 -- 172.21.15.139:0/37108450 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540032810 con 0x7fb5500ca740 2024-09-16T00:11:16.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7fce27778640 1 -- 172.21.15.139:0/2285098011 shutdown_connections 2024-09-16T00:11:16.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7fce27778640 1 --2- 172.21.15.139:0/2285098011 >> v2:172.21.15.139:3300/0 conn(0x7fce200c8ad0 0x7fce20150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7f0086b59640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00800cab20 0x7f0080164cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7fce27778640 1 --2- 172.21.15.139:0/2285098011 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce200cc330 0x7fce201572f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7fce27778640 1 -- 172.21.15.139:0/2285098011 >> 172.21.15.139:0/2285098011 conn(0x7fce2006da40 msgr2=0x7fce2006f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:16.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7fce27778640 1 -- 172.21.15.139:0/2285098011 shutdown_connections 2024-09-16T00:11:16.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7fce27778640 1 -- 172.21.15.139:0/2285098011 wait complete. 2024-09-16T00:11:16.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7faf057fa640 1 -- 172.21.15.139:0/3426804660 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7faefc045020 con 0x7faf00047a80 2024-09-16T00:11:16.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7f0086b59640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0080163fa0 0x7f0080164450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7f0d2ab13640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d241529c0 0x7f0d241ec9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7f0d2ab13640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d241529c0 0x7f0d241ec9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:40138/0 (socket says 172.21.15.139:40138) 2024-09-16T00:11:16.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7f0d2ab13640 1 -- 172.21.15.139:0/1643206352 learned_addr learned my addr 172.21.15.139:0/1643206352 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:16.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7f0086b59640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f00800cc1f0 con 0x7f0080163fa0 2024-09-16T00:11:16.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7f0086b59640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f00800cbf10 con 0x7f00800cab20 2024-09-16T00:11:16.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7f00850f6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0080163fa0 0x7f0080164450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7f00850f6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0080163fa0 0x7f0080164450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60604/0 (socket says 172.21.15.139:60604) 2024-09-16T00:11:16.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7f00850f6640 1 -- 172.21.15.139:0/646155762 learned_addr learned my addr 172.21.15.139:0/646155762 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:16.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7f0086b59640 1 -- 172.21.15.139:0/646155762 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f00800c5fc0 con 0x7f00800c7e40 2024-09-16T00:11:16.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.706+0000 7faf057fa640 1 --2- 172.21.15.139:0/3426804660 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faeec06c4b0 0x7faeec06e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7faf057fa640 1 -- 172.21.15.139:0/3426804660 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7faefc0ab2a0 con 0x7faf00047a80 2024-09-16T00:11:16.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7fb556952640 1 -- 172.21.15.139:0/37108450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5500ca740 msgr2=0x7fb5500c7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7fb556952640 1 --2- 172.21.15.139:0/37108450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5500ca740 0x7fb5500c7900 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb5400097a0 tx=0x7fb54002a250 comp rx=0 tx=0).stop 2024-09-16T00:11:16.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7f00848f5640 1 --2- 172.21.15.139:0/646155762 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00800c7e40 0x7f0080139750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7f007ffff640 1 --2- 172.21.15.139:0/646155762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00800cab20 0x7f0080164cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7faf07fff640 1 --2- 172.21.15.139:0/3426804660 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faeec06c4b0 0x7faeec06e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7f6655080640 1 -- 172.21.15.139:0/3035069048 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640037070 con 0x7f663c0039b0 2024-09-16T00:11:16.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7f00850f6640 1 -- 172.21.15.139:0/646155762 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00800c7e40 msgr2=0x7f0080139750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7f00850f6640 1 --2- 172.21.15.139:0/646155762 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00800c7e40 0x7f0080139750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7f00850f6640 1 -- 172.21.15.139:0/646155762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00800cab20 msgr2=0x7f0080164cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7f00850f6640 1 --2- 172.21.15.139:0/646155762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00800cab20 0x7f0080164cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7f00850f6640 1 -- 172.21.15.139:0/646155762 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0080149eb0 con 0x7f0080163fa0 2024-09-16T00:11:16.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7f6655080640 1 -- 172.21.15.139:0/3035069048 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664002acd0 con 0x7f663c0039b0 2024-09-16T00:11:16.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7fb556952640 1 -- 172.21.15.139:0/37108450 shutdown_connections 2024-09-16T00:11:16.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7f6655080640 1 -- 172.21.15.139:0/3035069048 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664000a650 con 0x7f663c0039b0 2024-09-16T00:11:16.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7fb556952640 1 --2- 172.21.15.139:0/37108450 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5500cab20 0x7fb5500c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7fce27778640 1 Processor -- start 2024-09-16T00:11:16.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7fb556952640 1 --2- 172.21.15.139:0/37108450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5500ca740 0x7fb5500c7900 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7fb556952640 1 -- 172.21.15.139:0/37108450 >> 172.21.15.139:0/37108450 conn(0x7fb5500c91d0 msgr2=0x7fb5500c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:16.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.708+0000 7faf07fff640 1 --2- 172.21.15.139:0/3426804660 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faeec06c4b0 0x7faeec06e970 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faf000076f0 tx=0x7faef4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.708+0000 7f6657084640 1 -- 172.21.15.139:0/3035069048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f663c0039b0 msgr2=0x7f663c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.708+0000 7f6657084640 1 --2- 172.21.15.139:0/3035069048 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f663c0039b0 0x7f663c003d90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f664000aa80 tx=0x7f663c005830 comp rx=0 tx=0).stop 2024-09-16T00:11:16.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7f0d2a312640 1 --2- 172.21.15.139:0/1643206352 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0d24068380 0x7f0d241f03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7f0d29b11640 1 --2- 172.21.15.139:0/1643206352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d240c42e0 0x7f0d240ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7f0d2ab13640 1 -- 172.21.15.139:0/1643206352 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0d24068380 msgr2=0x7f0d241f03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7f0d2ab13640 1 --2- 172.21.15.139:0/1643206352 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0d24068380 0x7f0d241f03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7f0d2ab13640 1 -- 172.21.15.139:0/1643206352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d240c42e0 msgr2=0x7f0d240ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7f0d2ab13640 1 --2- 172.21.15.139:0/1643206352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d240c42e0 0x7f0d240ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7f0d2ab13640 1 -- 172.21.15.139:0/1643206352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d240cb850 con 0x7f0d241529c0 2024-09-16T00:11:16.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.705+0000 7f0d2ab13640 1 --2- 172.21.15.139:0/1643206352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d241529c0 0x7f0d241ec9c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400d800 tx=0x7f0d1400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7f0d1affd640 1 -- 172.21.15.139:0/1643206352 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1400b6c0 con 0x7f0d241529c0 2024-09-16T00:11:16.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.707+0000 7f00850f6640 1 --2- 172.21.15.139:0/646155762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0080163fa0 0x7f0080164450 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f006800d830 tx=0x7f006800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.708+0000 7f0d1affd640 1 -- 172.21.15.139:0/1643206352 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d14019070 con 0x7f0d241529c0 2024-09-16T00:11:16.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.708+0000 7f0d1affd640 1 -- 172.21.15.139:0/1643206352 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1400baa0 con 0x7f0d241529c0 2024-09-16T00:11:16.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.708+0000 7fce27778640 1 -- start start 2024-09-16T00:11:16.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.708+0000 7f007d7fa640 1 -- 172.21.15.139:0/646155762 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006800b6c0 con 0x7f0080163fa0 2024-09-16T00:11:16.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.708+0000 7f007d7fa640 1 -- 172.21.15.139:0/646155762 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0068019070 con 0x7f0080163fa0 2024-09-16T00:11:16.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.708+0000 7f0d2c576640 1 -- 172.21.15.139:0/1643206352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d240cb570 con 0x7f0d241529c0 2024-09-16T00:11:16.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.708+0000 7f0d2c576640 1 -- 172.21.15.139:0/1643206352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d240b27b0 con 0x7f0d241529c0 2024-09-16T00:11:16.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.708+0000 7f6656883640 1 -- 172.21.15.139:0/3035069048 reap_dead start 2024-09-16T00:11:16.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.708+0000 7f007d7fa640 1 -- 172.21.15.139:0/646155762 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006800ba80 con 0x7f0080163fa0 2024-09-16T00:11:16.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.708+0000 7fb556952640 1 -- 172.21.15.139:0/37108450 shutdown_connections 2024-09-16T00:11:16.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.708+0000 7f0086b59640 1 -- 172.21.15.139:0/646155762 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00801634a0 con 0x7f0080163fa0 2024-09-16T00:11:16.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.708+0000 7fce27778640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce2006de20 0x7fce20145ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.709+0000 7fce27778640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce200c8ad0 0x7fce20141c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.709+0000 7fce27778640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce200cc330 0x7fce201467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.709+0000 7fce27778640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fce20072c60 con 0x7fce200c8ad0 2024-09-16T00:11:16.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.709+0000 7fce27778640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fce200c9860 con 0x7fce200cc330 2024-09-16T00:11:16.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.709+0000 7fce27778640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fce20072de0 con 0x7fce2006de20 2024-09-16T00:11:16.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.709+0000 7fce25514640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce2006de20 0x7fce20145ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.709+0000 7fce24d13640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce200c8ad0 0x7fce20141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.709+0000 7fce25d15640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce200cc330 0x7fce201467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.709+0000 7faf0db2b640 1 -- 172.21.15.139:0/3426804660 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee8005290 con 0x7faf00047a80 2024-09-16T00:11:16.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.709+0000 7f0086b59640 1 -- 172.21.15.139:0/646155762 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0080149450 con 0x7f0080163fa0 2024-09-16T00:11:16.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.709+0000 7fce25d15640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce200cc330 0x7fce201467c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:40154/0 (socket says 172.21.15.139:40154) 2024-09-16T00:11:16.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.710+0000 7fce25d15640 1 -- 172.21.15.139:0/2728184862 learned_addr learned my addr 172.21.15.139:0/2728184862 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:16.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.708+0000 7fb556952640 1 -- 172.21.15.139:0/37108450 wait complete. 2024-09-16T00:11:16.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.710+0000 7fce25d15640 1 -- 172.21.15.139:0/2728184862 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce2006de20 msgr2=0x7fce20145ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.710+0000 7fce25d15640 1 --2- 172.21.15.139:0/2728184862 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce2006de20 0x7fce20145ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.710+0000 7fce25d15640 1 -- 172.21.15.139:0/2728184862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce200c8ad0 msgr2=0x7fce20141c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.710+0000 7fce25d15640 1 --2- 172.21.15.139:0/2728184862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce200c8ad0 0x7fce20141c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.710+0000 7fce25d15640 1 -- 172.21.15.139:0/2728184862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce20142170 con 0x7fce200cc330 2024-09-16T00:11:16.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.710+0000 7fce25514640 1 --2- 172.21.15.139:0/2728184862 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce2006de20 0x7fce20145ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:16.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.710+0000 7fce24d13640 1 --2- 172.21.15.139:0/2728184862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce200c8ad0 0x7fce20141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:11:16.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.710+0000 7f0d1affd640 1 -- 172.21.15.139:0/1643206352 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0d1400bc40 con 0x7f0d241529c0 2024-09-16T00:11:16.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.710+0000 7fb556952640 1 Processor -- start 2024-09-16T00:11:16.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.711+0000 7fce25d15640 1 --2- 172.21.15.139:0/2728184862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce200cc330 0x7fce201467c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00e930 tx=0x7fce1c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.711+0000 7fce15ffb640 1 -- 172.21.15.139:0/2728184862 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c017070 con 0x7fce200cc330 2024-09-16T00:11:16.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.711+0000 7fce15ffb640 1 -- 172.21.15.139:0/2728184862 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce1c00ca00 con 0x7fce200cc330 2024-09-16T00:11:16.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.711+0000 7fce15ffb640 1 -- 172.21.15.139:0/2728184862 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00cdd0 con 0x7fce200cc330 2024-09-16T00:11:16.747 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.711+0000 7fb556952640 1 -- start start 2024-09-16T00:11:16.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.710+0000 7f0d2c576640 1 -- 172.21.15.139:0/1643206352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d240c8d60 con 0x7f0d241529c0 2024-09-16T00:11:16.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.711+0000 7f0d1affd640 1 --2- 172.21.15.139:0/1643206352 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0d0806c380 0x7f0d0806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.711+0000 7f0d1affd640 1 -- 172.21.15.139:0/1643206352 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f0d1408b970 con 0x7f0d241529c0 2024-09-16T00:11:16.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7fb556952640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5500c7e40 0x7fb550161020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.710+0000 7f6657084640 1 -- 172.21.15.139:0/3035069048 shutdown_connections 2024-09-16T00:11:16.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.710+0000 7f6657084640 1 -- 172.21.15.139:0/3035069048 >> 172.21.15.139:0/3035069048 conn(0x7f6650070160 msgr2=0x7f6650070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:16.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7f6657084640 1 -- 172.21.15.139:0/3035069048 shutdown_connections 2024-09-16T00:11:16.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7f6657084640 1 -- 172.21.15.139:0/3035069048 wait complete. 2024-09-16T00:11:16.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7fce27778640 1 -- 172.21.15.139:0/2728184862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce20154f40 con 0x7fce200cc330 2024-09-16T00:11:16.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7fce27778640 1 -- 172.21.15.139:0/2728184862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce201441f0 con 0x7fce200cc330 2024-09-16T00:11:16.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7f0086b59640 1 -- 172.21.15.139:0/646155762 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0080149650 con 0x7f0080163fa0 2024-09-16T00:11:16.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7f0d2a312640 1 --2- 172.21.15.139:0/1643206352 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0d0806c380 0x7f0d0806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7fb556952640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5500ca740 0x7fb5501547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7fb556952640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5501531a0 0x7fb550153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7fb556952640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb5500cc1f0 con 0x7fb5501531a0 2024-09-16T00:11:16.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7fb556952640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb5500cbf10 con 0x7fb5500c7e40 2024-09-16T00:11:16.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7fb556952640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb550162930 con 0x7fb5500ca740 2024-09-16T00:11:16.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7fb54f7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5500ca740 0x7fb5501547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7fb554eef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5501531a0 0x7fb550153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7fb554eef640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5501531a0 0x7fb550153650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60616/0 (socket says 172.21.15.139:60616) 2024-09-16T00:11:16.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7fb54f7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5500ca740 0x7fb5501547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:58676/0 (socket says 172.21.15.139:58676) 2024-09-16T00:11:16.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7fb554eef640 1 -- 172.21.15.139:0/1637145024 learned_addr learned my addr 172.21.15.139:0/1637145024 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:16.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.712+0000 7fb54ffff640 1 --2- 172.21.15.139:0/1637145024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5500c7e40 0x7fb550161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.713+0000 7fb554eef640 1 -- 172.21.15.139:0/1637145024 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5500ca740 msgr2=0x7fb5501547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.713+0000 7fb554eef640 1 --2- 172.21.15.139:0/1637145024 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5500ca740 0x7fb5501547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.713+0000 7fb554eef640 1 -- 172.21.15.139:0/1637145024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5500c7e40 msgr2=0x7fb550161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.713+0000 7fb554eef640 1 --2- 172.21.15.139:0/1637145024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5500c7e40 0x7fb550161020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.713+0000 7fb554eef640 1 -- 172.21.15.139:0/1637145024 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5501eac00 con 0x7fb5501531a0 2024-09-16T00:11:16.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.713+0000 7f0d2a312640 1 --2- 172.21.15.139:0/1643206352 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0d0806c380 0x7f0d0806e840 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c03c000 tx=0x7f0d1c037000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.713+0000 7fb554eef640 1 --2- 172.21.15.139:0/1637145024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5501531a0 0x7fb550153650 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb54800cb40 tx=0x7fb54800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.713+0000 7f007d7fa640 1 -- 172.21.15.139:0/646155762 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f006801e460 con 0x7f0080163fa0 2024-09-16T00:11:16.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.713+0000 7fce27778640 1 -- 172.21.15.139:0/2728184862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce201553b0 con 0x7fce200cc330 2024-09-16T00:11:16.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.714+0000 7f007d7fa640 1 --2- 172.21.15.139:0/646155762 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f006006c330 0x7f006006e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.714+0000 7f00848f5640 1 --2- 172.21.15.139:0/646155762 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f006006c330 0x7f006006e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.714+0000 7f00848f5640 1 --2- 172.21.15.139:0/646155762 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f006006c330 0x7f006006e7f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0070034000 tx=0x7f0070033040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.714+0000 7f6657084640 1 Processor -- start 2024-09-16T00:11:16.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.714+0000 7f6657084640 1 -- start start 2024-09-16T00:11:16.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6657084640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6650157690 0x7f6650157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6657084640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66500c6a80 0x7f6650158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6657084640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f66501e64e0 0x7f66501f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6657084640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f66500c81d0 con 0x7f6650157690 2024-09-16T00:11:16.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6657084640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f66500b2990 con 0x7f66500c6a80 2024-09-16T00:11:16.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6657084640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f665006e110 con 0x7f66501e64e0 2024-09-16T00:11:16.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6656082640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6650157690 0x7f6650157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f0d1affd640 1 -- 172.21.15.139:0/1643206352 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0d14059f40 con 0x7f0d241529c0 2024-09-16T00:11:16.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6656082640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6650157690 0x7f6650157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60632/0 (socket says 172.21.15.139:60632) 2024-09-16T00:11:16.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6656082640 1 -- 172.21.15.139:0/1259615603 learned_addr learned my addr 172.21.15.139:0/1259615603 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:16.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6655881640 1 --2- 172.21.15.139:0/1259615603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66500c6a80 0x7f6650158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.755 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6656883640 1 --2- 172.21.15.139:0/1259615603 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f66501e64e0 0x7f66501f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6656082640 1 -- 172.21.15.139:0/1259615603 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f66501e64e0 msgr2=0x7f66501f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6656082640 1 --2- 172.21.15.139:0/1259615603 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f66501e64e0 0x7f66501f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6656082640 1 -- 172.21.15.139:0/1259615603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66500c6a80 msgr2=0x7f6650158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6656082640 1 --2- 172.21.15.139:0/1259615603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66500c6a80 0x7f6650158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6656082640 1 -- 172.21.15.139:0/1259615603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6650157fb0 con 0x7f6650157690 2024-09-16T00:11:16.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.716+0000 7f6655881640 1 --2- 172.21.15.139:0/1259615603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66500c6a80 0x7f6650158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:11:16.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.717+0000 7f6656082640 1 --2- 172.21.15.139:0/1259615603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6650157690 0x7f6650157a70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f664000aa80 tx=0x7f6640004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.756 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.718+0000 7fce15ffb640 1 -- 172.21.15.139:0/2728184862 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fce1c004020 con 0x7fce200cc330 2024-09-16T00:11:16.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.718+0000 7fce15ffb640 1 --2- 172.21.15.139:0/2728184862 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fce0406c330 0x7fce0406e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.718+0000 7fce15ffb640 1 -- 172.21.15.139:0/2728184862 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7fce1c08b280 con 0x7fce200cc330 2024-09-16T00:11:16.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.718+0000 7fce25514640 1 --2- 172.21.15.139:0/2728184862 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fce0406c330 0x7fce0406e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.719+0000 7faf057fa640 1 -- 172.21.15.139:0/3426804660 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faefc079970 con 0x7faf00047a80 2024-09-16T00:11:16.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.719+0000 7f007d7fa640 1 -- 172.21.15.139:0/646155762 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f006800be30 con 0x7f0080163fa0 2024-09-16T00:11:16.757 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.719+0000 7fce25514640 1 --2- 172.21.15.139:0/2728184862 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fce0406c330 0x7fce0406e7f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fce10034000 tx=0x7fce1002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.719+0000 7fb54cff9640 1 -- 172.21.15.139:0/1637145024 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548017070 con 0x7fb5501531a0 2024-09-16T00:11:16.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.719+0000 7fb556952640 1 -- 172.21.15.139:0/1637145024 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5501f56f0 con 0x7fb5501531a0 2024-09-16T00:11:16.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.719+0000 7fb556952640 1 -- 172.21.15.139:0/1637145024 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb55006e0c0 con 0x7fb5501531a0 2024-09-16T00:11:16.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.720+0000 7f6646ffd640 1 -- 172.21.15.139:0/1259615603 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640040020 con 0x7f6650157690 2024-09-16T00:11:16.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.720+0000 7fb5367fc640 1 -- 172.21.15.139:0/1637145024 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb530005290 con 0x7fb5501531a0 2024-09-16T00:11:16.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.721+0000 7f6657084640 1 -- 172.21.15.139:0/1259615603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6650155a30 con 0x7f6650157690 2024-09-16T00:11:16.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.721+0000 7f6657084640 1 -- 172.21.15.139:0/1259615603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66501f0a60 con 0x7f6650157690 2024-09-16T00:11:16.759 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.721+0000 7fce15ffb640 1 -- 172.21.15.139:0/2728184862 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fce201553b0 con 0x7fce200cc330 2024-09-16T00:11:16.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.722+0000 7f6657084640 1 -- 172.21.15.139:0/1259615603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6650070c80 con 0x7f6650157690 2024-09-16T00:11:16.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.728+0000 7f007d7fa640 1 -- 172.21.15.139:0/646155762 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0080149650 con 0x7f0080163fa0 2024-09-16T00:11:16.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.729+0000 7f6646ffd640 1 -- 172.21.15.139:0/1259615603 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6640038040 con 0x7f6650157690 2024-09-16T00:11:16.760 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.729+0000 7f6646ffd640 1 -- 172.21.15.139:0/1259615603 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6640037070 con 0x7f6650157690 2024-09-16T00:11:16.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.728+0000 7fb54cff9640 1 -- 172.21.15.139:0/1637145024 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb54800ac40 con 0x7fb5501531a0 2024-09-16T00:11:16.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.728+0000 7fb54cff9640 1 -- 172.21.15.139:0/1637145024 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548021060 con 0x7fb5501531a0 2024-09-16T00:11:16.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.733+0000 7fb54cff9640 1 -- 172.21.15.139:0/1637145024 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb54801a070 con 0x7fb5501531a0 2024-09-16T00:11:16.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.733+0000 7fb54cff9640 1 --2- 172.21.15.139:0/1637145024 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb53c06c2b0 0x7fb53c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.734+0000 7f6646ffd640 1 -- 172.21.15.139:0/1259615603 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6640043020 con 0x7f6650157690 2024-09-16T00:11:16.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.735+0000 7f6646ffd640 1 --2- 172.21.15.139:0/1259615603 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f663406c2b0 0x7f663406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.734+0000 7fb54ffff640 1 --2- 172.21.15.139:0/1637145024 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb53c06c2b0 0x7fb53c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.735+0000 7f6646ffd640 1 -- 172.21.15.139:0/1259615603 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f66400abaf0 con 0x7f6650157690 2024-09-16T00:11:16.762 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.735+0000 7fb54ffff640 1 --2- 172.21.15.139:0/1637145024 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb53c06c2b0 0x7fb53c06e770 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb540034000 tx=0x7fb54002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.735+0000 7f6655881640 1 --2- 172.21.15.139:0/1259615603 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f663406c2b0 0x7f663406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.735+0000 7fb54cff9640 1 -- 172.21.15.139:0/1637145024 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7fb54808b590 con 0x7fb5501531a0 2024-09-16T00:11:16.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.736+0000 7f6655881640 1 --2- 172.21.15.139:0/1259615603 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f663406c2b0 0x7f663406e770 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f664c001800 tx=0x7f664c001d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.744+0000 7fb54cff9640 1 -- 172.21.15.139:0/1637145024 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb548059630 con 0x7fb5501531a0 2024-09-16T00:11:16.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.746+0000 7f4835cb7640 1 Processor -- start 2024-09-16T00:11:16.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.746+0000 7f4835cb7640 1 -- start start 2024-09-16T00:11:16.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.748+0000 7f4835cb7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f48300ccc90 0x7f48300b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.748+0000 7f4835cb7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48300c5380 con 0x7f48300cc060 2024-09-16T00:11:16.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.748+0000 7f4835cb7640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4830159380 con 0x7f4830162b50 2024-09-16T00:11:16.764 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.748+0000 7f4835cb7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f48300686b0 con 0x7f48300ccc90 2024-09-16T00:11:16.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.748+0000 7f482f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f48300ccc90 0x7f48300b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.748+0000 7f482f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f48300ccc90 0x7f48300b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60648/0 (socket says 172.21.15.139:60648) 2024-09-16T00:11:16.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.748+0000 7f482f7fe640 1 -- 172.21.15.139:0/3216917608 learned_addr learned my addr 172.21.15.139:0/3216917608 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:16.765 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.748+0000 7f6646ffd640 1 -- 172.21.15.139:0/1259615603 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f664004a030 con 0x7f6650157690 2024-09-16T00:11:16.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.750+0000 7f482e7fc640 1 -- 172.21.15.139:0/3216917608 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1605042442 0 0) 0x7f48300c5380 con 0x7f48300cc060 2024-09-16T00:11:16.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.750+0000 7f482e7fc640 1 -- 172.21.15.139:0/3216917608 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4818003520 con 0x7f48300cc060 2024-09-16T00:11:16.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.750+0000 7f482f7fe640 1 -- 172.21.15.139:0/3216917608 >> v1:172.21.15.164:6789/0 conn(0x7f4830162b50 legacy=0x7f4830169590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.750+0000 7f482f7fe640 1 -- 172.21.15.139:0/3216917608 >> v1:172.21.15.139:6789/0 conn(0x7f48300cc060 legacy=0x7f4830162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.750+0000 7f482f7fe640 1 -- 172.21.15.139:0/3216917608 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483015f800 con 0x7f48300ccc90 2024-09-16T00:11:16.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.752+0000 7f482f7fe640 1 --2- 172.21.15.139:0/3216917608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48300ccc90 0x7f48300b2dd0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4824009870 tx=0x7f482402a2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fe465ff7863249 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.752+0000 7f482e7fc640 1 -- 172.21.15.139:0/3216917608 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824037070 con 0x7f48300ccc90 2024-09-16T00:11:16.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.752+0000 7f482e7fc640 1 -- 172.21.15.139:0/3216917608 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4824032640 con 0x7f48300ccc90 2024-09-16T00:11:16.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.752+0000 7f482e7fc640 1 -- 172.21.15.139:0/3216917608 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824032a10 con 0x7f48300ccc90 2024-09-16T00:11:16.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.753+0000 7f4835cb7640 1 -- 172.21.15.139:0/3216917608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48300ccc90 msgr2=0x7f48300b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.753+0000 7f4835cb7640 1 --2- 172.21.15.139:0/3216917608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48300ccc90 0x7f48300b2dd0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4824009870 tx=0x7f482402a2f0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.754+0000 7f4835cb7640 1 -- 172.21.15.139:0/3216917608 shutdown_connections 2024-09-16T00:11:16.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.754+0000 7f4835cb7640 1 --2- 172.21.15.139:0/3216917608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48300ccc90 0x7f48300b2dd0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.754+0000 7f4835cb7640 1 -- 172.21.15.139:0/3216917608 >> 172.21.15.139:0/3216917608 conn(0x7f483006ca40 msgr2=0x7f4830067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:16.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.755+0000 7f4835cb7640 1 -- 172.21.15.139:0/3216917608 shutdown_connections 2024-09-16T00:11:16.769 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.755+0000 7f4835cb7640 1 -- 172.21.15.139:0/3216917608 wait complete. 2024-09-16T00:11:16.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.757+0000 7f4835cb7640 1 Processor -- start 2024-09-16T00:11:16.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.757+0000 7f4835cb7640 1 -- start start 2024-09-16T00:11:16.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.758+0000 7f4835cb7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48300cc060 0x7f4830158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.758+0000 7f4835cb7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48300ccc90 0x7f48301e8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.758+0000 7f4835cb7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4830162b50 0x7f48301e86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.758+0000 7f4835cb7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4830068060 con 0x7f4830162b50 2024-09-16T00:11:16.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.758+0000 7f4835cb7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f48300b6180 con 0x7f48300ccc90 2024-09-16T00:11:16.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.758+0000 7f4835cb7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4830068340 con 0x7f48300cc060 2024-09-16T00:11:16.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.759+0000 7f482effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48300ccc90 0x7f48301e8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.759+0000 7f482f7fe640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48300cc060 0x7f4830158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.759+0000 7f482effd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48300ccc90 0x7f48301e8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:40166/0 (socket says 172.21.15.139:40166) 2024-09-16T00:11:16.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.759+0000 7f482effd640 1 -- 172.21.15.139:0/855604318 learned_addr learned my addr 172.21.15.139:0/855604318 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:16.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.759+0000 7f482ffff640 1 --2- 172.21.15.139:0/855604318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4830162b50 0x7f48301e86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.760+0000 7f482effd640 1 -- 172.21.15.139:0/855604318 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48300cc060 msgr2=0x7f4830158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.760+0000 7f482effd640 1 --2- 172.21.15.139:0/855604318 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48300cc060 0x7f4830158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.760+0000 7f482effd640 1 -- 172.21.15.139:0/855604318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4830162b50 msgr2=0x7f48301e86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:16.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.760+0000 7f482effd640 1 --2- 172.21.15.139:0/855604318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4830162b50 0x7f48301e86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:16.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.760+0000 7f482effd640 1 -- 172.21.15.139:0/855604318 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48301f4720 con 0x7f48300ccc90 2024-09-16T00:11:16.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.760+0000 7f482effd640 1 --2- 172.21.15.139:0/855604318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48300ccc90 0x7f48301e8160 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f482000c400 tx=0x7f482000c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.761+0000 7f4817fff640 1 -- 172.21.15.139:0/855604318 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820017070 con 0x7f48300ccc90 2024-09-16T00:11:16.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.761+0000 7f4835cb7640 1 -- 172.21.15.139:0/855604318 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48301f4a00 con 0x7f48300ccc90 2024-09-16T00:11:16.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.761+0000 7f4835cb7640 1 -- 172.21.15.139:0/855604318 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48301f4fc0 con 0x7f48300ccc90 2024-09-16T00:11:16.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.762+0000 7f4817fff640 1 -- 172.21.15.139:0/855604318 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48200052d0 con 0x7f48300ccc90 2024-09-16T00:11:16.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.763+0000 7f4817fff640 1 -- 172.21.15.139:0/855604318 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820003810 con 0x7f48300ccc90 2024-09-16T00:11:16.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.765+0000 7f4817fff640 1 -- 172.21.15.139:0/855604318 <== mon.1 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f482001a070 con 0x7f48300ccc90 2024-09-16T00:11:16.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.766+0000 7f4835cb7640 1 -- 172.21.15.139:0/855604318 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4810005290 con 0x7f48300ccc90 2024-09-16T00:11:16.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.767+0000 7f4817fff640 1 --2- 172.21.15.139:0/855604318 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f481c06c380 0x7f481c06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:16.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.767+0000 7f482f7fe640 1 --2- 172.21.15.139:0/855604318 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f481c06c380 0x7f481c06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:16.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.767+0000 7f4817fff640 1 -- 172.21.15.139:0/855604318 <== mon.1 v2:172.21.15.164:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f482008aa00 con 0x7f48300ccc90 2024-09-16T00:11:16.772 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.768+0000 7f482f7fe640 1 --2- 172.21.15.139:0/855604318 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f481c06c380 0x7f481c06e840 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4824034000 tx=0x7f482402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:16.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:16.772+0000 7f4817fff640 1 -- 172.21.15.139:0/855604318 <== mon.1 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4820058980 con 0x7f48300ccc90 2024-09-16T00:11:17.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.020+0000 7f6d11ffb640 1 -- 172.21.15.139:0/294912180 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6d04005850 con 0x7f6d2c0c9bb0 2024-09-16T00:11:17.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.021+0000 7f6d13fff640 1 -- 172.21.15.139:0/294912180 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6d1c07a500 con 0x7f6d2c0c9bb0 2024-09-16T00:11:17.024 INFO:teuthology.orchestra.run.smithi139.stdout:403726925841 2024-09-16T00:11:17.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.024+0000 7f6d32123640 1 -- 172.21.15.139:0/294912180 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6d1806c2b0 msgr2=0x7f6d1806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.024+0000 7f6d32123640 1 --2- 172.21.15.139:0/294912180 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6d1806c2b0 0x7f6d1806e770 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6d200096c0 tx=0x7f6d20006bd0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.025+0000 7f6d32123640 1 -- 172.21.15.139:0/294912180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d2c0c9bb0 msgr2=0x7f6d2c1e22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.025+0000 7f6d32123640 1 --2- 172.21.15.139:0/294912180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d2c0c9bb0 0x7f6d2c1e22c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009960 tx=0x7f6d1c0046c0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.025+0000 7f6d32123640 1 -- 172.21.15.139:0/294912180 shutdown_connections 2024-09-16T00:11:17.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.025+0000 7f6d32123640 1 --2- 172.21.15.139:0/294912180 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f6d1806c2b0 0x7f6d1806e770 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.025+0000 7f6d32123640 1 --2- 172.21.15.139:0/294912180 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f6d2c152620 0x7f6d2c1e2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.025+0000 7f6d32123640 1 --2- 172.21.15.139:0/294912180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d2c0c9bb0 0x7f6d2c1e22c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.025+0000 7f6d32123640 1 --2- 172.21.15.139:0/294912180 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6d2c0c82d0 0x7f6d2c0c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.025+0000 7f6d32123640 1 -- 172.21.15.139:0/294912180 >> 172.21.15.139:0/294912180 conn(0x7f6d2c06c150 msgr2=0x7f6d2c0ccd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:17.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.026+0000 7f6d32123640 1 -- 172.21.15.139:0/294912180 shutdown_connections 2024-09-16T00:11:17.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.026+0000 7f6d32123640 1 -- 172.21.15.139:0/294912180 wait complete. 2024-09-16T00:11:17.048 INFO:tasks.ceph.ceph_manager.ceph:need seq 403726925841 got 403726925841 for osd.6 2024-09-16T00:11:17.048 DEBUG:teuthology.parallel:result is None 2024-09-16T00:11:17.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.094+0000 7f4835cb7640 1 -- 172.21.15.139:0/855604318 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f4810005580 con 0x7f48300ccc90 2024-09-16T00:11:17.096 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.094+0000 7f4817fff640 1 -- 172.21.15.139:0/855604318 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f482005cc80 con 0x7f48300ccc90 2024-09-16T00:11:17.096 INFO:teuthology.orchestra.run.smithi139.stdout:545460846598 2024-09-16T00:11:17.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.097+0000 7f4835cb7640 1 -- 172.21.15.139:0/855604318 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f481c06c380 msgr2=0x7f481c06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.097+0000 7f4835cb7640 1 --2- 172.21.15.139:0/855604318 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f481c06c380 0x7f481c06e840 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4824034000 tx=0x7f482402e000 comp rx=0 tx=0).stop 2024-09-16T00:11:17.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.097+0000 7f4835cb7640 1 -- 172.21.15.139:0/855604318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48300ccc90 msgr2=0x7f48301e8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.097+0000 7f4835cb7640 1 --2- 172.21.15.139:0/855604318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48300ccc90 0x7f48301e8160 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f482000c400 tx=0x7f482000c8c0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.097+0000 7f4835cb7640 1 -- 172.21.15.139:0/855604318 shutdown_connections 2024-09-16T00:11:17.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.097+0000 7f4835cb7640 1 --2- 172.21.15.139:0/855604318 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f481c06c380 0x7f481c06e840 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.097+0000 7f4835cb7640 1 --2- 172.21.15.139:0/855604318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4830162b50 0x7f48301e86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.097+0000 7f4835cb7640 1 --2- 172.21.15.139:0/855604318 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48300ccc90 0x7f48301e8160 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.097+0000 7f4835cb7640 1 --2- 172.21.15.139:0/855604318 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f48300cc060 0x7f4830158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.097+0000 7f4835cb7640 1 -- 172.21.15.139:0/855604318 >> 172.21.15.139:0/855604318 conn(0x7f483006ca40 msgr2=0x7f4830067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:17.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.098+0000 7f4835cb7640 1 -- 172.21.15.139:0/855604318 shutdown_connections 2024-09-16T00:11:17.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.098+0000 7f4835cb7640 1 -- 172.21.15.139:0/855604318 wait complete. 2024-09-16T00:11:17.105 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.103+0000 7f0d2c576640 1 -- 172.21.15.139:0/1643206352 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0d240ca440 con 0x7f0d241529c0 2024-09-16T00:11:17.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.104+0000 7f0d1affd640 1 -- 172.21.15.139:0/1643206352 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0d14059840 con 0x7f0d241529c0 2024-09-16T00:11:17.106 INFO:teuthology.orchestra.run.smithi139.stdout:64424509489 2024-09-16T00:11:17.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.107+0000 7f0d2c576640 1 -- 172.21.15.139:0/1643206352 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0d0806c380 msgr2=0x7f0d0806e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.107+0000 7f0d2c576640 1 --2- 172.21.15.139:0/1643206352 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0d0806c380 0x7f0d0806e840 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c03c000 tx=0x7f0d1c037000 comp rx=0 tx=0).stop 2024-09-16T00:11:17.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.108+0000 7f0d2c576640 1 -- 172.21.15.139:0/1643206352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d241529c0 msgr2=0x7f0d241ec9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.108+0000 7f0d2c576640 1 --2- 172.21.15.139:0/1643206352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d241529c0 0x7f0d241ec9c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0d1400d800 tx=0x7f0d1400dcc0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.110 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.108+0000 7f0d2c576640 1 -- 172.21.15.139:0/1643206352 shutdown_connections 2024-09-16T00:11:17.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.108+0000 7f0d2c576640 1 --2- 172.21.15.139:0/1643206352 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0d0806c380 0x7f0d0806e840 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.108+0000 7f0d2c576640 1 --2- 172.21.15.139:0/1643206352 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d241529c0 0x7f0d241ec9c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.108+0000 7f0d2c576640 1 --2- 172.21.15.139:0/1643206352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d240c42e0 0x7f0d240ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.108+0000 7f0d2c576640 1 --2- 172.21.15.139:0/1643206352 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0d24068380 0x7f0d241f03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.108+0000 7f0d2c576640 1 -- 172.21.15.139:0/1643206352 >> 172.21.15.139:0/1643206352 conn(0x7f0d2406bb20 msgr2=0x7f0d24150980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:17.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.108+0000 7f0d2c576640 1 -- 172.21.15.139:0/1643206352 shutdown_connections 2024-09-16T00:11:17.111 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.108+0000 7f0d2c576640 1 -- 172.21.15.139:0/1643206352 wait complete. 2024-09-16T00:11:17.112 INFO:tasks.ceph.ceph_manager.ceph:need seq 545460846598 got 545460846598 for osd.7 2024-09-16T00:11:17.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.109+0000 7fce27778640 1 -- 172.21.15.139:0/2728184862 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fce200cc710 con 0x7fce200cc330 2024-09-16T00:11:17.112 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.110+0000 7fce15ffb640 1 -- 172.21.15.139:0/2728184862 <== mon.1 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fce1c059300 con 0x7fce200cc330 2024-09-16T00:11:17.113 DEBUG:teuthology.parallel:result is None 2024-09-16T00:11:17.113 INFO:teuthology.orchestra.run.smithi139.stdout:64424509490 2024-09-16T00:11:17.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.113+0000 7fce27778640 1 -- 172.21.15.139:0/2728184862 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fce0406c330 msgr2=0x7fce0406e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.113+0000 7fce27778640 1 --2- 172.21.15.139:0/2728184862 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fce0406c330 0x7fce0406e7f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fce10034000 tx=0x7fce1002e000 comp rx=0 tx=0).stop 2024-09-16T00:11:17.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.113+0000 7fce27778640 1 -- 172.21.15.139:0/2728184862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce200cc330 msgr2=0x7fce201467c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.113+0000 7fce27778640 1 --2- 172.21.15.139:0/2728184862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce200cc330 0x7fce201467c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00e930 tx=0x7fce1c00edf0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.115 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.113+0000 7fce27778640 1 -- 172.21.15.139:0/2728184862 shutdown_connections 2024-09-16T00:11:17.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.113+0000 7fce27778640 1 --2- 172.21.15.139:0/2728184862 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fce0406c330 0x7fce0406e7f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.113+0000 7fce27778640 1 --2- 172.21.15.139:0/2728184862 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fce200cc330 0x7fce201467c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.113+0000 7fce27778640 1 --2- 172.21.15.139:0/2728184862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce200c8ad0 0x7fce20141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.113+0000 7fce27778640 1 --2- 172.21.15.139:0/2728184862 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fce2006de20 0x7fce20145ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.113+0000 7fce27778640 1 -- 172.21.15.139:0/2728184862 >> 172.21.15.139:0/2728184862 conn(0x7fce2006da40 msgr2=0x7fce200c50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:17.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.113+0000 7fce27778640 1 -- 172.21.15.139:0/2728184862 shutdown_connections 2024-09-16T00:11:17.116 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.114+0000 7fce27778640 1 -- 172.21.15.139:0/2728184862 wait complete. 2024-09-16T00:11:17.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.114+0000 7faf0db2b640 1 -- 172.21.15.139:0/3426804660 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7faee8005850 con 0x7faf00047a80 2024-09-16T00:11:17.117 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.114+0000 7faf057fa640 1 -- 172.21.15.139:0/3426804660 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faefc079270 con 0x7faf00047a80 2024-09-16T00:11:17.117 INFO:teuthology.orchestra.run.smithi139.stdout:64424509491 2024-09-16T00:11:17.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.118+0000 7faf0db2b640 1 -- 172.21.15.139:0/3426804660 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faeec06c4b0 msgr2=0x7faeec06e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.118+0000 7faf0db2b640 1 --2- 172.21.15.139:0/3426804660 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faeec06c4b0 0x7faeec06e970 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faf000076f0 tx=0x7faef4008040 comp rx=0 tx=0).stop 2024-09-16T00:11:17.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.118+0000 7faf0db2b640 1 -- 172.21.15.139:0/3426804660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf00047a80 msgr2=0x7faf000512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.118+0000 7faf0db2b640 1 --2- 172.21.15.139:0/3426804660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf00047a80 0x7faf000512f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7faefc000c00 tx=0x7faefc0046c0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.119+0000 7faf0db2b640 1 -- 172.21.15.139:0/3426804660 shutdown_connections 2024-09-16T00:11:17.121 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.119+0000 7faf0db2b640 1 --2- 172.21.15.139:0/3426804660 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faeec06c4b0 0x7faeec06e970 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.119+0000 7faf0db2b640 1 --2- 172.21.15.139:0/3426804660 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf000d5850 0x7faf00048130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.119+0000 7faf0db2b640 1 --2- 172.21.15.139:0/3426804660 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf0004d310 0x7faf00051830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.119+0000 7faf0db2b640 1 --2- 172.21.15.139:0/3426804660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf00047a80 0x7faf000512f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.119+0000 7faf0db2b640 1 -- 172.21.15.139:0/3426804660 >> 172.21.15.139:0/3426804660 conn(0x7faf0004d830 msgr2=0x7faf000359b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:17.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.119+0000 7faf0db2b640 1 -- 172.21.15.139:0/3426804660 shutdown_connections 2024-09-16T00:11:17.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.119+0000 7faf0db2b640 1 -- 172.21.15.139:0/3426804660 wait complete. 2024-09-16T00:11:17.122 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.120+0000 7f0086b59640 1 -- 172.21.15.139:0/646155762 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f00800ca740 con 0x7f0080163fa0 2024-09-16T00:11:17.123 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.121+0000 7f007d7fa640 1 -- 172.21.15.139:0/646155762 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f006805a1c0 con 0x7f0080163fa0 2024-09-16T00:11:17.123 INFO:teuthology.orchestra.run.smithi139.stdout:55834574899 2024-09-16T00:11:17.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.125+0000 7f0086b59640 1 -- 172.21.15.139:0/646155762 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f006006c330 msgr2=0x7f006006e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.125+0000 7f0086b59640 1 --2- 172.21.15.139:0/646155762 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f006006c330 0x7f006006e7f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0070034000 tx=0x7f0070033040 comp rx=0 tx=0).stop 2024-09-16T00:11:17.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.125+0000 7f0086b59640 1 -- 172.21.15.139:0/646155762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0080163fa0 msgr2=0x7f0080164450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.125+0000 7f0086b59640 1 --2- 172.21.15.139:0/646155762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0080163fa0 0x7f0080164450 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f006800d830 tx=0x7f006800dcf0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.125+0000 7f0086b59640 1 -- 172.21.15.139:0/646155762 shutdown_connections 2024-09-16T00:11:17.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.125+0000 7f0086b59640 1 --2- 172.21.15.139:0/646155762 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f006006c330 0x7f006006e7f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.127 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.125+0000 7f0086b59640 1 --2- 172.21.15.139:0/646155762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0080163fa0 0x7f0080164450 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.125+0000 7f0086b59640 1 --2- 172.21.15.139:0/646155762 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f00800cab20 0x7f0080164cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.125+0000 7f0086b59640 1 --2- 172.21.15.139:0/646155762 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f00800c7e40 0x7f0080139750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.128 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.125+0000 7f0086b59640 1 -- 172.21.15.139:0/646155762 >> 172.21.15.139:0/646155762 conn(0x7f00800c91d0 msgr2=0x7f0080072bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:17.129 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509489 got 64424509489 for osd.3 2024-09-16T00:11:17.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.125+0000 7f0086b59640 1 -- 172.21.15.139:0/646155762 shutdown_connections 2024-09-16T00:11:17.129 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.125+0000 7f0086b59640 1 -- 172.21.15.139:0/646155762 wait complete. 2024-09-16T00:11:17.129 DEBUG:teuthology.parallel:result is None 2024-09-16T00:11:17.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509490 got 64424509490 for osd.5 2024-09-16T00:11:17.130 DEBUG:teuthology.parallel:result is None 2024-09-16T00:11:17.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.129+0000 7fb5367fc640 1 -- 172.21.15.139:0/1637145024 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb530005850 con 0x7fb5501531a0 2024-09-16T00:11:17.131 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.130+0000 7fb54cff9640 1 -- 172.21.15.139:0/1637145024 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb54805d930 con 0x7fb5501531a0 2024-09-16T00:11:17.132 INFO:teuthology.orchestra.run.smithi139.stdout:60129542194 2024-09-16T00:11:17.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.132+0000 7fb5367fc640 1 -- 172.21.15.139:0/1637145024 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb53c06c2b0 msgr2=0x7fb53c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.132+0000 7fb5367fc640 1 --2- 172.21.15.139:0/1637145024 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb53c06c2b0 0x7fb53c06e770 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb540034000 tx=0x7fb54002e000 comp rx=0 tx=0).stop 2024-09-16T00:11:17.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.132+0000 7fb5367fc640 1 -- 172.21.15.139:0/1637145024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5501531a0 msgr2=0x7fb550153650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.135 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.132+0000 7fb5367fc640 1 --2- 172.21.15.139:0/1637145024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5501531a0 0x7fb550153650 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb54800cb40 tx=0x7fb54800a060 comp rx=0 tx=0).stop 2024-09-16T00:11:17.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.132+0000 7fb5367fc640 1 -- 172.21.15.139:0/1637145024 shutdown_connections 2024-09-16T00:11:17.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.132+0000 7fb5367fc640 1 --2- 172.21.15.139:0/1637145024 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb53c06c2b0 0x7fb53c06e770 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.132+0000 7fb5367fc640 1 --2- 172.21.15.139:0/1637145024 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5501531a0 0x7fb550153650 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.132+0000 7fb5367fc640 1 --2- 172.21.15.139:0/1637145024 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb5500ca740 0x7fb5501547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.132+0000 7fb5367fc640 1 --2- 172.21.15.139:0/1637145024 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb5500c7e40 0x7fb550161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.132+0000 7fb5367fc640 1 -- 172.21.15.139:0/1637145024 >> 172.21.15.139:0/1637145024 conn(0x7fb5500c91d0 msgr2=0x7fb550072a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:17.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.132+0000 7fb5367fc640 1 -- 172.21.15.139:0/1637145024 shutdown_connections 2024-09-16T00:11:17.136 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.132+0000 7fb5367fc640 1 -- 172.21.15.139:0/1637145024 wait complete. 2024-09-16T00:11:17.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509491 got 64424509491 for osd.4 2024-09-16T00:11:17.137 DEBUG:teuthology.parallel:result is None 2024-09-16T00:11:17.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.142+0000 7f6657084640 1 -- 172.21.15.139:0/1259615603 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f66500c7000 con 0x7f6650157690 2024-09-16T00:11:17.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.143+0000 7f6646ffd640 1 -- 172.21.15.139:0/1259615603 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6640079b00 con 0x7f6650157690 2024-09-16T00:11:17.145 INFO:teuthology.orchestra.run.smithi139.stdout:60129542194 2024-09-16T00:11:17.145 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574898 got 55834574899 for osd.2 2024-09-16T00:11:17.146 DEBUG:teuthology.parallel:result is None 2024-09-16T00:11:17.146 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542194 got 60129542194 for osd.0 2024-09-16T00:11:17.146 DEBUG:teuthology.parallel:result is None 2024-09-16T00:11:17.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.145+0000 7f6657084640 1 -- 172.21.15.139:0/1259615603 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f663406c2b0 msgr2=0x7f663406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.145+0000 7f6657084640 1 --2- 172.21.15.139:0/1259615603 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f663406c2b0 0x7f663406e770 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f664c001800 tx=0x7f664c001d30 comp rx=0 tx=0).stop 2024-09-16T00:11:17.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.145+0000 7f6657084640 1 -- 172.21.15.139:0/1259615603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6650157690 msgr2=0x7f6650157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.145+0000 7f6657084640 1 --2- 172.21.15.139:0/1259615603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6650157690 0x7f6650157a70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f664000aa80 tx=0x7f6640004730 comp rx=0 tx=0).stop 2024-09-16T00:11:17.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.145+0000 7f6657084640 1 -- 172.21.15.139:0/1259615603 shutdown_connections 2024-09-16T00:11:17.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.145+0000 7f6657084640 1 --2- 172.21.15.139:0/1259615603 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f663406c2b0 0x7f663406e770 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.148 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.145+0000 7f6657084640 1 --2- 172.21.15.139:0/1259615603 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f66501e64e0 0x7f66501f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.145+0000 7f6657084640 1 --2- 172.21.15.139:0/1259615603 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f66500c6a80 0x7f6650158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.145+0000 7f6657084640 1 --2- 172.21.15.139:0/1259615603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6650157690 0x7f6650157a70 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.145+0000 7f6657084640 1 -- 172.21.15.139:0/1259615603 >> 172.21.15.139:0/1259615603 conn(0x7f6650070160 msgr2=0x7f665006e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:17.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.145+0000 7f6657084640 1 -- 172.21.15.139:0/1259615603 shutdown_connections 2024-09-16T00:11:17.149 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.145+0000 7f6657084640 1 -- 172.21.15.139:0/1259615603 wait complete. 2024-09-16T00:11:17.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542194 got 60129542194 for osd.1 2024-09-16T00:11:17.158 DEBUG:teuthology.parallel:result is None 2024-09-16T00:11:17.158 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T00:11:17.159 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:11:17.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.295+0000 7f46c22e4640 1 Processor -- start 2024-09-16T00:11:17.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.295+0000 7f46c22e4640 1 -- start start 2024-09-16T00:11:17.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.296+0000 7f46c22e4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f46bc15b5c0 0x7f46bc0c6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.296+0000 7f46c22e4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46bc158550 con 0x7f46bc072c30 2024-09-16T00:11:17.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.296+0000 7f46c22e4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46bc1576f0 con 0x7f46bc073010 2024-09-16T00:11:17.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.296+0000 7f46c22e4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f46bc1579d0 con 0x7f46bc15b5c0 2024-09-16T00:11:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.296+0000 7f46c0881640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f46bc15b5c0 0x7f46bc0c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.296+0000 7f46bbfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f46bc072c30 0x7f46bc0c3590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45678/0 (socket says 172.21.15.139:45678) 2024-09-16T00:11:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.296+0000 7f46bbfff640 1 -- 172.21.15.139:0/3679227979 learned_addr learned my addr 172.21.15.139:0/3679227979 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.296+0000 7f46c0881640 1 -- 172.21.15.139:0/3679227979 >> v1:172.21.15.164:6789/0 conn(0x7f46bc073010 legacy=0x7f46bc15aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.296+0000 7f46c0881640 1 -- 172.21.15.139:0/3679227979 >> v1:172.21.15.139:6789/0 conn(0x7f46bc072c30 legacy=0x7f46bc0c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.296+0000 7f46c0881640 1 -- 172.21.15.139:0/3679227979 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46bc157cb0 con 0x7f46bc15b5c0 2024-09-16T00:11:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.297+0000 7f46c0881640 1 --2- 172.21.15.139:0/3679227979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46bc15b5c0 0x7f46bc0c6210 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009d90 tx=0x7f46b402c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871e3b5a871ecdcd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.297+0000 7f46baffd640 1 -- 172.21.15.139:0/3679227979 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4036020 con 0x7f46bc15b5c0 2024-09-16T00:11:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.297+0000 7f46baffd640 1 -- 172.21.15.139:0/3679227979 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b4032800 con 0x7f46bc15b5c0 2024-09-16T00:11:17.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.297+0000 7f46baffd640 1 -- 172.21.15.139:0/3679227979 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4032bd0 con 0x7f46bc15b5c0 2024-09-16T00:11:17.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.298+0000 7f46c22e4640 1 -- 172.21.15.139:0/3679227979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46bc15b5c0 msgr2=0x7f46bc0c6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.298+0000 7f46c22e4640 1 --2- 172.21.15.139:0/3679227979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46bc15b5c0 0x7f46bc0c6210 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009d90 tx=0x7f46b402c940 comp rx=0 tx=0).stop 2024-09-16T00:11:17.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.298+0000 7f46c22e4640 1 -- 172.21.15.139:0/3679227979 shutdown_connections 2024-09-16T00:11:17.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.298+0000 7f46c22e4640 1 --2- 172.21.15.139:0/3679227979 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46bc15b5c0 0x7f46bc0c6210 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.298+0000 7f46c22e4640 1 -- 172.21.15.139:0/3679227979 >> 172.21.15.139:0/3679227979 conn(0x7f46bc067950 msgr2=0x7f46bc067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:17.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.298+0000 7f46c22e4640 1 -- 172.21.15.139:0/3679227979 shutdown_connections 2024-09-16T00:11:17.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.298+0000 7f46c22e4640 1 -- 172.21.15.139:0/3679227979 wait complete. 2024-09-16T00:11:17.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.299+0000 7f46c22e4640 1 Processor -- start 2024-09-16T00:11:17.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.299+0000 7f46c22e4640 1 -- start start 2024-09-16T00:11:17.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.300+0000 7f46c22e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46bc072c30 0x7f46bc1ebe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.300+0000 7f46c22e4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46bc073010 0x7f46bc1efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.300+0000 7f46c22e4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46bc15b5c0 0x7f46bc153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.300+0000 7f46c22e4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc0c6b20 con 0x7f46bc072c30 2024-09-16T00:11:17.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.300+0000 7f46c22e4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc158270 con 0x7f46bc073010 2024-09-16T00:11:17.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.300+0000 7f46c22e4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f46bc151020 con 0x7f46bc15b5c0 2024-09-16T00:11:17.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.300+0000 7f46bbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46bc072c30 0x7f46bc1ebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.300+0000 7f46bbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46bc072c30 0x7f46bc1ebe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43310/0 (socket says 172.21.15.139:43310) 2024-09-16T00:11:17.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.300+0000 7f46bbfff640 1 -- 172.21.15.139:0/3031368139 learned_addr learned my addr 172.21.15.139:0/3031368139 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:17.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.300+0000 7f46bbfff640 1 -- 172.21.15.139:0/3031368139 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46bc15b5c0 msgr2=0x7f46bc153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.300+0000 7f46bbfff640 1 --2- 172.21.15.139:0/3031368139 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46bc15b5c0 0x7f46bc153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.300+0000 7f46bbfff640 1 -- 172.21.15.139:0/3031368139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46bc073010 msgr2=0x7f46bc1efdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.300+0000 7f46bbfff640 1 --2- 172.21.15.139:0/3031368139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46bc073010 0x7f46bc1efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.300+0000 7f46bbfff640 1 -- 172.21.15.139:0/3031368139 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46bc0c8c00 con 0x7f46bc072c30 2024-09-16T00:11:17.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.300+0000 7f46bbfff640 1 --2- 172.21.15.139:0/3031368139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46bc072c30 0x7f46bc1ebe30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00c430 tx=0x7f46ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:17.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.300+0000 7f46b8ff9640 1 -- 172.21.15.139:0/3031368139 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac017070 con 0x7f46bc072c30 2024-09-16T00:11:17.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.301+0000 7f46b8ff9640 1 -- 172.21.15.139:0/3031368139 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ac005ce0 con 0x7f46bc072c30 2024-09-16T00:11:17.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.301+0000 7f46c22e4640 1 -- 172.21.15.139:0/3031368139 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46bc0c8ee0 con 0x7f46bc072c30 2024-09-16T00:11:17.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.301+0000 7f46b8ff9640 1 -- 172.21.15.139:0/3031368139 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0072f0 con 0x7f46bc072c30 2024-09-16T00:11:17.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.301+0000 7f46c22e4640 1 -- 172.21.15.139:0/3031368139 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46bc1ef5f0 con 0x7f46bc072c30 2024-09-16T00:11:17.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.302+0000 7f46c22e4640 1 -- 172.21.15.139:0/3031368139 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46bc14fdc0 con 0x7f46bc072c30 2024-09-16T00:11:17.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.302+0000 7f46b8ff9640 1 -- 172.21.15.139:0/3031368139 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f46ac01a070 con 0x7f46bc072c30 2024-09-16T00:11:17.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.302+0000 7f46b8ff9640 1 --2- 172.21.15.139:0/3031368139 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f46a806c2b0 0x7f46a806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.303+0000 7f46b8ff9640 1 -- 172.21.15.139:0/3031368139 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f46ac08a890 con 0x7f46bc072c30 2024-09-16T00:11:17.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.303+0000 7f46bb7fe640 1 --2- 172.21.15.139:0/3031368139 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f46a806c2b0 0x7f46a806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.303+0000 7f46bb7fe640 1 --2- 172.21.15.139:0/3031368139 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f46a806c2b0 0x7f46a806e770 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f46bc151de0 tx=0x7f46b00079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:17.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.307+0000 7f46b8ff9640 1 -- 172.21.15.139:0/3031368139 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f46bc14fdc0 con 0x7f46bc072c30 2024-09-16T00:11:17.474 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.471+0000 7f46c22e4640 1 -- 172.21.15.139:0/3031368139 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46bc0c6210 con 0x7f46a806c2b0 2024-09-16T00:11:17.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.478+0000 7f46b8ff9640 1 -- 172.21.15.139:0/3031368139 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254583 (secure 0 0 0) 0x7f46bc0c6210 con 0x7f46a806c2b0 2024-09-16T00:11:17.481 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:11:17.486 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:11:17.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.481+0000 7f46c22e4640 1 -- 172.21.15.139:0/3031368139 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f46a806c2b0 msgr2=0x7f46a806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.481+0000 7f46c22e4640 1 --2- 172.21.15.139:0/3031368139 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f46a806c2b0 0x7f46a806e770 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f46bc151de0 tx=0x7f46b00079b0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.481+0000 7f46c22e4640 1 -- 172.21.15.139:0/3031368139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46bc072c30 msgr2=0x7f46bc1ebe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.481+0000 7f46c22e4640 1 --2- 172.21.15.139:0/3031368139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46bc072c30 0x7f46bc1ebe30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00c430 tx=0x7f46ac00c8f0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.481+0000 7f46c22e4640 1 -- 172.21.15.139:0/3031368139 shutdown_connections 2024-09-16T00:11:17.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.481+0000 7f46c22e4640 1 --2- 172.21.15.139:0/3031368139 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f46a806c2b0 0x7f46a806e770 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.488 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.481+0000 7f46c22e4640 1 --2- 172.21.15.139:0/3031368139 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46bc15b5c0 0x7f46bc153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.481+0000 7f46c22e4640 1 --2- 172.21.15.139:0/3031368139 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46bc073010 0x7f46bc1efdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.481+0000 7f46c22e4640 1 --2- 172.21.15.139:0/3031368139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46bc072c30 0x7f46bc1ebe30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.481+0000 7f46c22e4640 1 -- 172.21.15.139:0/3031368139 >> 172.21.15.139:0/3031368139 conn(0x7f46bc067950 msgr2=0x7f46bc14f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:17.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.481+0000 7f46c22e4640 1 -- 172.21.15.139:0/3031368139 shutdown_connections 2024-09-16T00:11:17.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.481+0000 7f46c22e4640 1 -- 172.21.15.139:0/3031368139 wait complete. 2024-09-16T00:11:17.494 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-16T00:11:16.066417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":129638821,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":2853,"num_read_kb":1363646,"num_write":30194,"num_write_kb":4222082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3232,"num_bytes_recovered":321049829,"num_keys_recovered":3117,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33178,"ondisk_log_size":33178,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":462188,"kb_used_data":98336,"kb_used_omap":273,"kb_used_meta":363374,"kb_avail":754512532,"statfs":{"total":773094113280,"available":772620832768,"internally_reserved":0,"allocated":100696064,"data_stored":266031034,"data_compressed":38804642,"data_compressed_allocated":49901568,"data_compressed_original":222470144,"omap_allocated":280397,"internal_metadata":372095155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.145130"},"pg_stats":[{"pgid":"3.3f","version":"123'434","reported_seq":867,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137191+0000","last_change":"2024-09-16T00:10:56.416283+0000","last_active":"2024-09-16T00:11:13.137191+0000","last_peered":"2024-09-16T00:11:13.137191+0000","last_clean":"2024-09-16T00:11:13.137191+0000","last_became_active":"2024-09-16T00:10:55.845182+0000","last_became_peered":"2024-09-16T00:10:55.845182+0000","last_unstale":"2024-09-16T00:11:13.137191+0000","last_undegraded":"2024-09-16T00:11:13.137191+0000","last_fullsized":"2024-09-16T00:11:13.137191+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:52.137388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":42086,"num_write":409,"num_write_kb":112716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":31107821,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"123'417","reported_seq":1091,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.943324+0000","last_change":"2024-09-16T00:11:04.897161+0000","last_active":"2024-09-16T00:11:12.943324+0000","last_peered":"2024-09-16T00:11:12.943324+0000","last_clean":"2024-09-16T00:11:12.943324+0000","last_became_active":"2024-09-16T00:11:04.894999+0000","last_became_peered":"2024-09-16T00:11:04.894999+0000","last_unstale":"2024-09-16T00:11:12.943324+0000","last_undegraded":"2024-09-16T00:11:12.943324+0000","last_fullsized":"2024-09-16T00:11:12.943324+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:13.015652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":23064,"num_write":397,"num_write_kb":47220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":689591,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"123'367","reported_seq":657,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381045+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:11:14.381045+0000","last_peered":"2024-09-16T00:11:14.381045+0000","last_clean":"2024-09-16T00:11:14.381045+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:11:14.381045+0000","last_undegraded":"2024-09-16T00:11:14.381045+0000","last_fullsized":"2024-09-16T00:11:14.381045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15782,"num_write":349,"num_write_kb":54932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"123'389","reported_seq":679,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951475+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:11:13.951475+0000","last_peered":"2024-09-16T00:11:13.951475+0000","last_clean":"2024-09-16T00:11:13.951475+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:11:13.951475+0000","last_undegraded":"2024-09-16T00:11:13.951475+0000","last_fullsized":"2024-09-16T00:11:13.951475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":14080,"num_write":364,"num_write_kb":63191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"123'445","reported_seq":746,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.942767+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:11:12.942767+0000","last_peered":"2024-09-16T00:11:12.942767+0000","last_clean":"2024-09-16T00:11:12.942767+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:11:12.942767+0000","last_undegraded":"2024-09-16T00:11:12.942767+0000","last_fullsized":"2024-09-16T00:11:12.942767+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":36096,"num_write":427,"num_write_kb":89616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"123'384","reported_seq":1352,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952787+0000","last_change":"2024-09-16T00:10:45.795247+0000","last_active":"2024-09-16T00:11:13.952787+0000","last_peered":"2024-09-16T00:11:13.952787+0000","last_clean":"2024-09-16T00:11:13.952787+0000","last_became_active":"2024-09-16T00:10:45.793434+0000","last_became_peered":"2024-09-16T00:10:45.793434+0000","last_unstale":"2024-09-16T00:11:13.952787+0000","last_undegraded":"2024-09-16T00:11:13.952787+0000","last_fullsized":"2024-09-16T00:11:13.952787+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:55.019829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":6174,"num_write":362,"num_write_kb":50827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":425,"num_bytes_recovered":23619955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"123'420","reported_seq":779,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136743+0000","last_change":"2024-09-16T00:10:56.030536+0000","last_active":"2024-09-16T00:11:13.136743+0000","last_peered":"2024-09-16T00:11:13.136743+0000","last_clean":"2024-09-16T00:11:13.136743+0000","last_became_active":"2024-09-16T00:10:55.835516+0000","last_became_peered":"2024-09-16T00:10:55.835516+0000","last_unstale":"2024-09-16T00:11:13.136743+0000","last_undegraded":"2024-09-16T00:11:13.136743+0000","last_fullsized":"2024-09-16T00:11:13.136743+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:43.152015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":16577,"num_write":401,"num_write_kb":64470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"123'374","reported_seq":641,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952597+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:11:13.952597+0000","last_peered":"2024-09-16T00:11:13.952597+0000","last_clean":"2024-09-16T00:11:13.952597+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:11:13.952597+0000","last_undegraded":"2024-09-16T00:11:13.952597+0000","last_fullsized":"2024-09-16T00:11:13.952597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":16776,"num_write":347,"num_write_kb":80164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"123'394","reported_seq":724,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137150+0000","last_change":"2024-09-16T00:10:56.261956+0000","last_active":"2024-09-16T00:11:13.137150+0000","last_peered":"2024-09-16T00:11:13.137150+0000","last_clean":"2024-09-16T00:11:13.137150+0000","last_became_active":"2024-09-16T00:10:55.840604+0000","last_became_peered":"2024-09-16T00:10:55.840604+0000","last_unstale":"2024-09-16T00:11:13.137150+0000","last_undegraded":"2024-09-16T00:11:13.137150+0000","last_fullsized":"2024-09-16T00:11:13.137150+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:39.279660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1432,"num_write":368,"num_write_kb":30311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"123'367","reported_seq":536,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011843+0000","last_change":"2024-09-16T00:10:00.734336+0000","last_active":"2024-09-16T00:11:14.011843+0000","last_peered":"2024-09-16T00:11:14.011843+0000","last_clean":"2024-09-16T00:11:14.011843+0000","last_became_active":"2024-09-16T00:10:00.284452+0000","last_became_peered":"2024-09-16T00:10:00.284452+0000","last_unstale":"2024-09-16T00:11:14.011843+0000","last_undegraded":"2024-09-16T00:11:14.011843+0000","last_fullsized":"2024-09-16T00:11:14.011843+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:56:50.530439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4096,"num_write":335,"num_write_kb":29952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":8736255,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"123'355","reported_seq":535,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011551+0000","last_change":"2024-09-16T00:10:00.587110+0000","last_active":"2024-09-16T00:11:14.011551+0000","last_peered":"2024-09-16T00:11:14.011551+0000","last_clean":"2024-09-16T00:11:14.011551+0000","last_became_active":"2024-09-16T00:10:00.280273+0000","last_became_peered":"2024-09-16T00:10:00.280273+0000","last_unstale":"2024-09-16T00:11:14.011551+0000","last_undegraded":"2024-09-16T00:11:14.011551+0000","last_fullsized":"2024-09-16T00:11:14.011551+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:01:57.902939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":975,"num_write":339,"num_write_kb":25518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":10753732,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"123'366","reported_seq":705,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952011+0000","last_change":"2024-09-16T00:10:00.797372+0000","last_active":"2024-09-16T00:11:13.952011+0000","last_peered":"2024-09-16T00:11:13.952011+0000","last_clean":"2024-09-16T00:11:13.952011+0000","last_became_active":"2024-09-16T00:10:00.278765+0000","last_became_peered":"2024-09-16T00:10:00.278765+0000","last_unstale":"2024-09-16T00:11:13.952011+0000","last_undegraded":"2024-09-16T00:11:13.952011+0000","last_fullsized":"2024-09-16T00:11:13.952011+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:03.318919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":11563,"num_write":348,"num_write_kb":58933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":29565877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"123'419","reported_seq":782,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544719+0000","last_change":"2024-09-16T00:10:00.715647+0000","last_active":"2024-09-16T00:11:13.544719+0000","last_peered":"2024-09-16T00:11:13.544719+0000","last_clean":"2024-09-16T00:11:13.544719+0000","last_became_active":"2024-09-16T00:10:00.286050+0000","last_became_peered":"2024-09-16T00:10:00.286050+0000","last_unstale":"2024-09-16T00:11:13.544719+0000","last_undegraded":"2024-09-16T00:11:13.544719+0000","last_fullsized":"2024-09-16T00:11:13.544719+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:07.462899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":35908,"num_write":393,"num_write_kb":104863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":30230864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"123'415","reported_seq":724,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.380880+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:11:14.380880+0000","last_peered":"2024-09-16T00:11:14.380880+0000","last_clean":"2024-09-16T00:11:14.380880+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:11:14.380880+0000","last_undegraded":"2024-09-16T00:11:14.380880+0000","last_fullsized":"2024-09-16T00:11:14.380880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":23364,"num_write":390,"num_write_kb":43321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"123'392","reported_seq":682,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.942692+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:11:12.942692+0000","last_peered":"2024-09-16T00:11:12.942692+0000","last_clean":"2024-09-16T00:11:12.942692+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:11:12.942692+0000","last_undegraded":"2024-09-16T00:11:12.942692+0000","last_fullsized":"2024-09-16T00:11:12.942692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":14230,"num_write":362,"num_write_kb":33768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"123'439","reported_seq":756,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951004+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:11:13.951004+0000","last_peered":"2024-09-16T00:11:13.951004+0000","last_clean":"2024-09-16T00:11:13.951004+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:11:13.951004+0000","last_undegraded":"2024-09-16T00:11:13.951004+0000","last_fullsized":"2024-09-16T00:11:13.951004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":40234,"num_write":414,"num_write_kb":67785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"123'394","reported_seq":890,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952446+0000","last_change":"2024-09-16T00:10:56.577752+0000","last_active":"2024-09-16T00:11:13.952446+0000","last_peered":"2024-09-16T00:11:13.952446+0000","last_clean":"2024-09-16T00:11:13.952446+0000","last_became_active":"2024-09-16T00:10:55.845880+0000","last_became_peered":"2024-09-16T00:10:55.845880+0000","last_unstale":"2024-09-16T00:11:13.952446+0000","last_undegraded":"2024-09-16T00:11:13.952446+0000","last_fullsized":"2024-09-16T00:11:13.952446+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:32.037945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":22524,"num_write":374,"num_write_kb":46842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"123'391","reported_seq":720,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952778+0000","last_change":"2024-09-16T00:10:00.461437+0000","last_active":"2024-09-16T00:11:13.952778+0000","last_peered":"2024-09-16T00:11:13.952778+0000","last_clean":"2024-09-16T00:11:13.952778+0000","last_became_active":"2024-09-16T00:10:00.282592+0000","last_became_peered":"2024-09-16T00:10:00.282592+0000","last_unstale":"2024-09-16T00:11:13.952778+0000","last_undegraded":"2024-09-16T00:11:13.952778+0000","last_fullsized":"2024-09-16T00:11:13.952778+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:09.733353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8711,"num_write":366,"num_write_kb":46855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":21285626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"123'417","reported_seq":825,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137126+0000","last_change":"2024-09-16T00:10:56.142127+0000","last_active":"2024-09-16T00:11:13.137126+0000","last_peered":"2024-09-16T00:11:13.137126+0000","last_clean":"2024-09-16T00:11:13.137126+0000","last_became_active":"2024-09-16T00:10:55.832595+0000","last_became_peered":"2024-09-16T00:10:55.832595+0000","last_unstale":"2024-09-16T00:11:13.137126+0000","last_undegraded":"2024-09-16T00:11:13.137126+0000","last_fullsized":"2024-09-16T00:11:13.137126+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:44:56.072997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":14437,"num_write":396,"num_write_kb":83979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":103497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"123'423","reported_seq":602,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011546+0000","last_change":"2024-09-16T00:10:00.566870+0000","last_active":"2024-09-16T00:11:14.011546+0000","last_peered":"2024-09-16T00:11:14.011546+0000","last_clean":"2024-09-16T00:11:14.011546+0000","last_became_active":"2024-09-16T00:10:00.281258+0000","last_became_peered":"2024-09-16T00:10:00.281258+0000","last_unstale":"2024-09-16T00:11:14.011546+0000","last_undegraded":"2024-09-16T00:11:14.011546+0000","last_fullsized":"2024-09-16T00:11:14.011546+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:29.197743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":15502,"num_write":406,"num_write_kb":30222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":17199573,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"123'404","reported_seq":883,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381025+0000","last_change":"2024-09-16T00:10:56.104046+0000","last_active":"2024-09-16T00:11:14.381025+0000","last_peered":"2024-09-16T00:11:14.381025+0000","last_clean":"2024-09-16T00:11:14.381025+0000","last_became_active":"2024-09-16T00:10:55.835150+0000","last_became_peered":"2024-09-16T00:10:55.835150+0000","last_unstale":"2024-09-16T00:11:14.381025+0000","last_undegraded":"2024-09-16T00:11:14.381025+0000","last_fullsized":"2024-09-16T00:11:14.381025+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:08.552926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":34318,"num_write":385,"num_write_kb":91769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"123'444","reported_seq":918,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951970+0000","last_change":"2024-09-16T00:10:55.939360+0000","last_active":"2024-09-16T00:11:13.951970+0000","last_peered":"2024-09-16T00:11:13.951970+0000","last_clean":"2024-09-16T00:11:13.951970+0000","last_became_active":"2024-09-16T00:10:55.833335+0000","last_became_peered":"2024-09-16T00:10:55.833335+0000","last_unstale":"2024-09-16T00:11:13.951970+0000","last_undegraded":"2024-09-16T00:11:13.951970+0000","last_fullsized":"2024-09-16T00:11:13.951970+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:08:29.270004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":35528,"num_write":422,"num_write_kb":92852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"123'395","reported_seq":681,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544399+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:11:13.544399+0000","last_peered":"2024-09-16T00:11:13.544399+0000","last_clean":"2024-09-16T00:11:13.544399+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:11:13.544399+0000","last_undegraded":"2024-09-16T00:11:13.544399+0000","last_fullsized":"2024-09-16T00:11:13.544399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":13410,"num_write":369,"num_write_kb":58805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"123'421","reported_seq":703,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951092+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:11:13.951092+0000","last_peered":"2024-09-16T00:11:13.951092+0000","last_clean":"2024-09-16T00:11:13.951092+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:11:13.951092+0000","last_undegraded":"2024-09-16T00:11:13.951092+0000","last_fullsized":"2024-09-16T00:11:13.951092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":12682,"num_write":399,"num_write_kb":54869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"123'421","reported_seq":710,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544202+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:11:13.544202+0000","last_peered":"2024-09-16T00:11:13.544202+0000","last_clean":"2024-09-16T00:11:13.544202+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:11:13.544202+0000","last_undegraded":"2024-09-16T00:11:13.544202+0000","last_fullsized":"2024-09-16T00:11:13.544202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":14188,"num_write":408,"num_write_kb":59178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"123'448","reported_seq":862,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011709+0000","last_change":"2024-09-16T00:10:56.267499+0000","last_active":"2024-09-16T00:11:14.011709+0000","last_peered":"2024-09-16T00:11:14.011709+0000","last_clean":"2024-09-16T00:11:14.011709+0000","last_became_active":"2024-09-16T00:10:55.841967+0000","last_became_peered":"2024-09-16T00:10:55.841967+0000","last_unstale":"2024-09-16T00:11:14.011709+0000","last_undegraded":"2024-09-16T00:11:14.011709+0000","last_fullsized":"2024-09-16T00:11:14.011709+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:15.324718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":12330,"num_write":421,"num_write_kb":35146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":12903786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"123'402","reported_seq":1050,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.943063+0000","last_change":"2024-09-16T00:11:04.895240+0000","last_active":"2024-09-16T00:11:12.943063+0000","last_peered":"2024-09-16T00:11:12.943063+0000","last_clean":"2024-09-16T00:11:12.943063+0000","last_became_active":"2024-09-16T00:11:04.892794+0000","last_became_peered":"2024-09-16T00:11:04.892794+0000","last_unstale":"2024-09-16T00:11:12.943063+0000","last_undegraded":"2024-09-16T00:11:12.943063+0000","last_fullsized":"2024-09-16T00:11:12.943063+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:44.365114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":21104,"num_write":383,"num_write_kb":75742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":628439,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"123'386","reported_seq":686,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.543974+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:11:13.543974+0000","last_peered":"2024-09-16T00:11:13.543974+0000","last_clean":"2024-09-16T00:11:13.543974+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:11:13.543974+0000","last_undegraded":"2024-09-16T00:11:13.543974+0000","last_fullsized":"2024-09-16T00:11:13.543974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":16408,"num_write":366,"num_write_kb":51674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"123'398","reported_seq":717,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381121+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:11:14.381121+0000","last_peered":"2024-09-16T00:11:14.381121+0000","last_clean":"2024-09-16T00:11:14.381121+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:11:14.381121+0000","last_undegraded":"2024-09-16T00:11:14.381121+0000","last_fullsized":"2024-09-16T00:11:14.381121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":33174,"num_write":373,"num_write_kb":66849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"123'486","reported_seq":780,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952852+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:11:13.952852+0000","last_peered":"2024-09-16T00:11:13.952852+0000","last_clean":"2024-09-16T00:11:13.952852+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:11:13.952852+0000","last_undegraded":"2024-09-16T00:11:13.952852+0000","last_fullsized":"2024-09-16T00:11:13.952852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":27231,"num_write":453,"num_write_kb":104771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"123'451","reported_seq":789,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952668+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:11:13.952668+0000","last_peered":"2024-09-16T00:11:13.952668+0000","last_clean":"2024-09-16T00:11:13.952668+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:11:13.952668+0000","last_undegraded":"2024-09-16T00:11:13.952668+0000","last_fullsized":"2024-09-16T00:11:13.952668+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":35655,"num_write":431,"num_write_kb":97245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"123'421","reported_seq":602,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011470+0000","last_change":"2024-09-16T00:10:00.535553+0000","last_active":"2024-09-16T00:11:14.011470+0000","last_peered":"2024-09-16T00:11:14.011470+0000","last_clean":"2024-09-16T00:11:14.011470+0000","last_became_active":"2024-09-16T00:10:00.281378+0000","last_became_peered":"2024-09-16T00:10:00.281378+0000","last_unstale":"2024-09-16T00:11:14.011470+0000","last_undegraded":"2024-09-16T00:11:14.011470+0000","last_fullsized":"2024-09-16T00:11:14.011470+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:49.548709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20395,"num_write":401,"num_write_kb":32731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":22876917,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"123'401","reported_seq":741,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951565+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:11:13.951565+0000","last_peered":"2024-09-16T00:11:13.951565+0000","last_clean":"2024-09-16T00:11:13.951565+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:11:13.951565+0000","last_undegraded":"2024-09-16T00:11:13.951565+0000","last_fullsized":"2024-09-16T00:11:13.951565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":48141,"num_write":381,"num_write_kb":63611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"120'126","reported_seq":400,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544826+0000","last_change":"2024-09-16T00:10:55.854872+0000","last_active":"2024-09-16T00:11:13.544826+0000","last_peered":"2024-09-16T00:11:13.544826+0000","last_clean":"2024-09-16T00:11:13.544826+0000","last_became_active":"2024-09-16T00:10:55.843367+0000","last_became_peered":"2024-09-16T00:10:55.843367+0000","last_unstale":"2024-09-16T00:11:13.544826+0000","last_undegraded":"2024-09-16T00:11:13.544826+0000","last_fullsized":"2024-09-16T00:11:13.544826+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:07.797903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"123'346","reported_seq":633,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381167+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:11:14.381167+0000","last_peered":"2024-09-16T00:11:14.381167+0000","last_clean":"2024-09-16T00:11:14.381167+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:11:14.381167+0000","last_undegraded":"2024-09-16T00:11:14.381167+0000","last_fullsized":"2024-09-16T00:11:14.381167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":16420,"num_write":326,"num_write_kb":62710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"122'340","reported_seq":505,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011382+0000","last_change":"2024-09-16T00:10:00.282958+0000","last_active":"2024-09-16T00:11:14.011382+0000","last_peered":"2024-09-16T00:11:14.011382+0000","last_clean":"2024-09-16T00:11:14.011382+0000","last_became_active":"2024-09-16T00:10:00.282541+0000","last_became_peered":"2024-09-16T00:10:00.282541+0000","last_unstale":"2024-09-16T00:11:14.011382+0000","last_undegraded":"2024-09-16T00:11:14.011382+0000","last_fullsized":"2024-09-16T00:11:14.011382+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:51.198787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":285,"num_write_kb":12471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"123'226","reported_seq":508,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544507+0000","last_change":"2024-09-16T00:10:00.313589+0000","last_active":"2024-09-16T00:11:13.544507+0000","last_peered":"2024-09-16T00:11:13.544507+0000","last_clean":"2024-09-16T00:11:13.544507+0000","last_became_active":"2024-09-16T00:10:00.284602+0000","last_became_peered":"2024-09-16T00:10:00.284602+0000","last_unstale":"2024-09-16T00:11:13.544507+0000","last_undegraded":"2024-09-16T00:11:13.544507+0000","last_fullsized":"2024-09-16T00:11:13.544507+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:30.032698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":14,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"123'433","reported_seq":730,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544169+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:11:13.544169+0000","last_peered":"2024-09-16T00:11:13.544169+0000","last_clean":"2024-09-16T00:11:13.544169+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:11:13.544169+0000","last_undegraded":"2024-09-16T00:11:13.544169+0000","last_fullsized":"2024-09-16T00:11:13.544169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":18480,"num_write":411,"num_write_kb":71748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"123'833","reported_seq":1095,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951164+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:11:13.951164+0000","last_peered":"2024-09-16T00:11:13.951164+0000","last_clean":"2024-09-16T00:11:13.951164+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:11:13.951164+0000","last_undegraded":"2024-09-16T00:11:13.951164+0000","last_fullsized":"2024-09-16T00:11:13.951164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":8591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"123'425","reported_seq":772,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.942671+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:11:12.942671+0000","last_peered":"2024-09-16T00:11:12.942671+0000","last_clean":"2024-09-16T00:11:12.942671+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:11:12.942671+0000","last_undegraded":"2024-09-16T00:11:12.942671+0000","last_fullsized":"2024-09-16T00:11:12.942671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":46871,"num_write":401,"num_write_kb":84187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"123'65","reported_seq":296,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011701+0000","last_change":"2024-09-16T00:10:00.283876+0000","last_active":"2024-09-16T00:11:14.011701+0000","last_peered":"2024-09-16T00:11:14.011701+0000","last_clean":"2024-09-16T00:11:14.011701+0000","last_became_active":"2024-09-16T00:10:00.283555+0000","last_became_peered":"2024-09-16T00:10:00.283555+0000","last_unstale":"2024-09-16T00:11:14.011701+0000","last_undegraded":"2024-09-16T00:11:14.011701+0000","last_fullsized":"2024-09-16T00:11:14.011701+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:48.543737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"123'440","reported_seq":904,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951434+0000","last_change":"2024-09-16T00:10:56.502658+0000","last_active":"2024-09-16T00:11:13.951434+0000","last_peered":"2024-09-16T00:11:13.951434+0000","last_clean":"2024-09-16T00:11:13.951434+0000","last_became_active":"2024-09-16T00:10:55.840027+0000","last_became_peered":"2024-09-16T00:10:55.840027+0000","last_unstale":"2024-09-16T00:11:13.951434+0000","last_undegraded":"2024-09-16T00:11:13.951434+0000","last_fullsized":"2024-09-16T00:11:13.951434+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:52.105868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":16686,"num_write":415,"num_write_kb":80112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"123'225","reported_seq":494,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951555+0000","last_change":"2024-09-16T00:10:00.285386+0000","last_active":"2024-09-16T00:11:13.951555+0000","last_peered":"2024-09-16T00:11:13.951555+0000","last_clean":"2024-09-16T00:11:13.951555+0000","last_became_active":"2024-09-16T00:10:00.284717+0000","last_became_peered":"2024-09-16T00:10:00.284717+0000","last_unstale":"2024-09-16T00:11:13.951555+0000","last_undegraded":"2024-09-16T00:11:13.951555+0000","last_fullsized":"2024-09-16T00:11:13.951555+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:07.794163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":4181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"123'378","reported_seq":681,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381204+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:11:14.381204+0000","last_peered":"2024-09-16T00:11:14.381204+0000","last_clean":"2024-09-16T00:11:14.381204+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:11:14.381204+0000","last_undegraded":"2024-09-16T00:11:14.381204+0000","last_fullsized":"2024-09-16T00:11:14.381204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":18030,"num_write":359,"num_write_kb":55428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"125'131","reported_seq":390,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.943147+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:11:12.943147+0000","last_peered":"2024-09-16T00:11:12.943147+0000","last_clean":"2024-09-16T00:11:12.943147+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:11:12.943147+0000","last_undegraded":"2024-09-16T00:11:12.943147+0000","last_fullsized":"2024-09-16T00:11:12.943147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"123'373","reported_seq":818,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952022+0000","last_change":"2024-09-16T00:10:56.173134+0000","last_active":"2024-09-16T00:11:13.952022+0000","last_peered":"2024-09-16T00:11:13.952022+0000","last_clean":"2024-09-16T00:11:13.952022+0000","last_became_active":"2024-09-16T00:10:55.833273+0000","last_became_peered":"2024-09-16T00:10:55.833273+0000","last_unstale":"2024-09-16T00:11:13.952022+0000","last_undegraded":"2024-09-16T00:11:13.952022+0000","last_fullsized":"2024-09-16T00:11:13.952022+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:11.402988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":22960,"num_write":348,"num_write_kb":63161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"125'238","reported_seq":414,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011582+0000","last_change":"2024-09-16T00:10:00.300895+0000","last_active":"2024-09-16T00:11:14.011582+0000","last_peered":"2024-09-16T00:11:14.011582+0000","last_clean":"2024-09-16T00:11:14.011582+0000","last_became_active":"2024-09-16T00:10:00.283474+0000","last_became_peered":"2024-09-16T00:10:00.283474+0000","last_unstale":"2024-09-16T00:11:14.011582+0000","last_undegraded":"2024-09-16T00:11:14.011582+0000","last_fullsized":"2024-09-16T00:11:14.011582+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:28:02.563817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10145896,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":186,"num_write_kb":10014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":13200,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"123'403","reported_seq":774,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136869+0000","last_change":"2024-09-16T00:10:56.320793+0000","last_active":"2024-09-16T00:11:13.136869+0000","last_peered":"2024-09-16T00:11:13.136869+0000","last_clean":"2024-09-16T00:11:13.136869+0000","last_became_active":"2024-09-16T00:10:55.836462+0000","last_became_peered":"2024-09-16T00:10:55.836462+0000","last_unstale":"2024-09-16T00:11:13.136869+0000","last_undegraded":"2024-09-16T00:11:13.136869+0000","last_fullsized":"2024-09-16T00:11:13.136869+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:07:47.717822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":23285,"num_write":372,"num_write_kb":67093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"122'470","reported_seq":732,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381212+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:11:14.381212+0000","last_peered":"2024-09-16T00:11:14.381212+0000","last_clean":"2024-09-16T00:11:14.381212+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:11:14.381212+0000","last_undegraded":"2024-09-16T00:11:14.381212+0000","last_fullsized":"2024-09-16T00:11:14.381212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":419,"num_write_kb":8413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"123'404","reported_seq":872,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.380881+0000","last_change":"2024-09-16T00:10:56.427976+0000","last_active":"2024-09-16T00:11:14.380881+0000","last_peered":"2024-09-16T00:11:14.380881+0000","last_clean":"2024-09-16T00:11:14.380881+0000","last_became_active":"2024-09-16T00:10:55.839196+0000","last_became_peered":"2024-09-16T00:10:55.839196+0000","last_unstale":"2024-09-16T00:11:14.380881+0000","last_undegraded":"2024-09-16T00:11:14.380881+0000","last_fullsized":"2024-09-16T00:11:14.380881+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:37.852194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":13263,"num_write":381,"num_write_kb":51174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"123'59","reported_seq":321,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952149+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:11:13.952149+0000","last_peered":"2024-09-16T00:11:13.952149+0000","last_clean":"2024-09-16T00:11:13.952149+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:11:13.952149+0000","last_undegraded":"2024-09-16T00:11:13.952149+0000","last_fullsized":"2024-09-16T00:11:13.952149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"123'394","reported_seq":733,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136975+0000","last_change":"2024-09-16T00:10:56.085067+0000","last_active":"2024-09-16T00:11:13.136975+0000","last_peered":"2024-09-16T00:11:13.136975+0000","last_clean":"2024-09-16T00:11:13.136975+0000","last_became_active":"2024-09-16T00:10:55.834112+0000","last_became_peered":"2024-09-16T00:10:55.834112+0000","last_unstale":"2024-09-16T00:11:13.136975+0000","last_undegraded":"2024-09-16T00:11:13.136975+0000","last_fullsized":"2024-09-16T00:11:13.136975+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:35.278069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":13990,"num_write":368,"num_write_kb":63962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"120'70","reported_seq":218,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011398+0000","last_change":"2024-09-16T00:10:00.294519+0000","last_active":"2024-09-16T00:11:14.011398+0000","last_peered":"2024-09-16T00:11:14.011398+0000","last_clean":"2024-09-16T00:11:14.011398+0000","last_became_active":"2024-09-16T00:10:00.283066+0000","last_became_peered":"2024-09-16T00:10:00.283066+0000","last_unstale":"2024-09-16T00:11:14.011398+0000","last_undegraded":"2024-09-16T00:11:14.011398+0000","last_fullsized":"2024-09-16T00:11:14.011398+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:32.621176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"123'424","reported_seq":607,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011378+0000","last_change":"2024-09-16T00:10:00.460910+0000","last_active":"2024-09-16T00:11:14.011378+0000","last_peered":"2024-09-16T00:11:14.011378+0000","last_clean":"2024-09-16T00:11:14.011378+0000","last_became_active":"2024-09-16T00:10:00.280750+0000","last_became_peered":"2024-09-16T00:10:00.280750+0000","last_unstale":"2024-09-16T00:11:14.011378+0000","last_undegraded":"2024-09-16T00:11:14.011378+0000","last_fullsized":"2024-09-16T00:11:14.011378+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:47.634650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":19573,"num_write":402,"num_write_kb":50826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":33491520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"125'72","reported_seq":334,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952685+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:11:13.952685+0000","last_peered":"2024-09-16T00:11:13.952685+0000","last_clean":"2024-09-16T00:11:13.952685+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:11:13.952685+0000","last_undegraded":"2024-09-16T00:11:13.952685+0000","last_fullsized":"2024-09-16T00:11:13.952685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"123'457","reported_seq":814,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136679+0000","last_change":"2024-09-16T00:10:55.922694+0000","last_active":"2024-09-16T00:11:13.136679+0000","last_peered":"2024-09-16T00:11:13.136679+0000","last_clean":"2024-09-16T00:11:13.136679+0000","last_became_active":"2024-09-16T00:10:55.827989+0000","last_became_peered":"2024-09-16T00:10:55.827989+0000","last_unstale":"2024-09-16T00:11:13.136679+0000","last_undegraded":"2024-09-16T00:11:13.136679+0000","last_fullsized":"2024-09-16T00:11:13.136679+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:23.666346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20419,"num_write":427,"num_write_kb":43035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"123'145","reported_seq":407,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951453+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:11:13.951453+0000","last_peered":"2024-09-16T00:11:13.951453+0000","last_clean":"2024-09-16T00:11:13.951453+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:11:13.951453+0000","last_undegraded":"2024-09-16T00:11:13.951453+0000","last_fullsized":"2024-09-16T00:11:13.951453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91,"num_write_kb":4217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"123'358","reported_seq":694,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137061+0000","last_change":"2024-09-16T00:10:55.971163+0000","last_active":"2024-09-16T00:11:13.137061+0000","last_peered":"2024-09-16T00:11:13.137061+0000","last_clean":"2024-09-16T00:11:13.137061+0000","last_became_active":"2024-09-16T00:10:55.834401+0000","last_became_peered":"2024-09-16T00:10:55.834401+0000","last_unstale":"2024-09-16T00:11:13.137061+0000","last_undegraded":"2024-09-16T00:11:13.137061+0000","last_fullsized":"2024-09-16T00:11:13.137061+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:05.466545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":10790,"num_write":330,"num_write_kb":21585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"120'282","reported_seq":544,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952893+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:11:13.952893+0000","last_peered":"2024-09-16T00:11:13.952893+0000","last_clean":"2024-09-16T00:11:13.952893+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:11:13.952893+0000","last_undegraded":"2024-09-16T00:11:13.952893+0000","last_fullsized":"2024-09-16T00:11:13.952893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":238,"num_write_kb":4238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"123'372","reported_seq":696,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137259+0000","last_change":"2024-09-16T00:10:56.200838+0000","last_active":"2024-09-16T00:11:13.137259+0000","last_peered":"2024-09-16T00:11:13.137259+0000","last_clean":"2024-09-16T00:11:13.137259+0000","last_became_active":"2024-09-16T00:10:55.838254+0000","last_became_peered":"2024-09-16T00:10:55.838254+0000","last_unstale":"2024-09-16T00:11:13.137259+0000","last_undegraded":"2024-09-16T00:11:13.137259+0000","last_fullsized":"2024-09-16T00:11:13.137259+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:30.944542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":350,"num_write":351,"num_write_kb":38221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"122'534","reported_seq":793,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.943228+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:11:12.943228+0000","last_peered":"2024-09-16T00:11:12.943228+0000","last_clean":"2024-09-16T00:11:12.943228+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:11:12.943228+0000","last_undegraded":"2024-09-16T00:11:12.943228+0000","last_fullsized":"2024-09-16T00:11:12.943228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":473,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"123'418","reported_seq":682,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951696+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:11:13.951696+0000","last_peered":"2024-09-16T00:11:13.951696+0000","last_clean":"2024-09-16T00:11:13.951696+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:11:13.951696+0000","last_undegraded":"2024-09-16T00:11:13.951696+0000","last_fullsized":"2024-09-16T00:11:13.951696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":675,"num_write":389,"num_write_kb":34696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"123'72","reported_seq":334,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952967+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:11:13.952967+0000","last_peered":"2024-09-16T00:11:13.952967+0000","last_clean":"2024-09-16T00:11:13.952967+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:11:13.952967+0000","last_undegraded":"2024-09-16T00:11:13.952967+0000","last_fullsized":"2024-09-16T00:11:13.952967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"122'397","reported_seq":761,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136957+0000","last_change":"2024-09-16T00:10:56.368023+0000","last_active":"2024-09-16T00:11:13.136957+0000","last_peered":"2024-09-16T00:11:13.136957+0000","last_clean":"2024-09-16T00:11:13.136957+0000","last_became_active":"2024-09-16T00:10:55.839714+0000","last_became_peered":"2024-09-16T00:10:55.839714+0000","last_unstale":"2024-09-16T00:11:13.136957+0000","last_undegraded":"2024-09-16T00:11:13.136957+0000","last_fullsized":"2024-09-16T00:11:13.136957+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:04.192344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":22590,"num_write":379,"num_write_kb":59088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":372,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136608+0000","last_change":"2024-09-16T00:10:55.832444+0000","last_active":"2024-09-16T00:11:13.136608+0000","last_peered":"2024-09-16T00:11:13.136608+0000","last_clean":"2024-09-16T00:11:13.136608+0000","last_became_active":"2024-09-16T00:10:55.831881+0000","last_became_peered":"2024-09-16T00:10:55.831881+0000","last_unstale":"2024-09-16T00:11:13.136608+0000","last_undegraded":"2024-09-16T00:11:13.136608+0000","last_fullsized":"2024-09-16T00:11:13.136608+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:48.243166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"123'63","reported_seq":325,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951198+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:11:13.951198+0000","last_peered":"2024-09-16T00:11:13.951198+0000","last_clean":"2024-09-16T00:11:13.951198+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:11:13.951198+0000","last_undegraded":"2024-09-16T00:11:13.951198+0000","last_fullsized":"2024-09-16T00:11:13.951198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"123'369","reported_seq":662,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544613+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:11:13.544613+0000","last_peered":"2024-09-16T00:11:13.544613+0000","last_clean":"2024-09-16T00:11:13.544613+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:11:13.544613+0000","last_undegraded":"2024-09-16T00:11:13.544613+0000","last_fullsized":"2024-09-16T00:11:13.544613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":16103,"num_write":342,"num_write_kb":46241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"123'399","reported_seq":643,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136617+0000","last_change":"2024-09-16T00:11:04.896119+0000","last_active":"2024-09-16T00:11:13.136617+0000","last_peered":"2024-09-16T00:11:13.136617+0000","last_clean":"2024-09-16T00:11:13.136617+0000","last_became_active":"2024-09-16T00:11:04.895672+0000","last_became_peered":"2024-09-16T00:11:04.895672+0000","last_unstale":"2024-09-16T00:11:13.136617+0000","last_undegraded":"2024-09-16T00:11:13.136617+0000","last_fullsized":"2024-09-16T00:11:13.136617+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:45.173943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":346,"num_write_kb":8359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":16777216,"num_keys_recovered":28,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"123'393","reported_seq":700,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.380785+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:11:14.380785+0000","last_peered":"2024-09-16T00:11:14.380785+0000","last_clean":"2024-09-16T00:11:14.380785+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:11:14.380785+0000","last_undegraded":"2024-09-16T00:11:14.380785+0000","last_fullsized":"2024-09-16T00:11:14.380785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":29385,"num_write":362,"num_write_kb":83610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"125'242","reported_seq":680,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952710+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:11:13.952710+0000","last_peered":"2024-09-16T00:11:13.952710+0000","last_clean":"2024-09-16T00:11:13.952710+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:11:13.952710+0000","last_undegraded":"2024-09-16T00:11:13.952710+0000","last_fullsized":"2024-09-16T00:11:13.952710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461247,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":176,"num_read_kb":1534,"num_write":179,"num_write_kb":1542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"123'400","reported_seq":710,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.942629+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:11:12.942629+0000","last_peered":"2024-09-16T00:11:12.942629+0000","last_clean":"2024-09-16T00:11:12.942629+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:11:12.942629+0000","last_undegraded":"2024-09-16T00:11:12.942629+0000","last_fullsized":"2024-09-16T00:11:12.942629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":28288,"num_write":378,"num_write_kb":66949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"119'400","reported_seq":662,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952094+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:11:13.952094+0000","last_peered":"2024-09-16T00:11:13.952094+0000","last_clean":"2024-09-16T00:11:13.952094+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:11:13.952094+0000","last_undegraded":"2024-09-16T00:11:13.952094+0000","last_fullsized":"2024-09-16T00:11:13.952094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":353,"num_write_kb":8381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"123'387","reported_seq":683,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951764+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:11:13.951764+0000","last_peered":"2024-09-16T00:11:13.951764+0000","last_clean":"2024-09-16T00:11:13.951764+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:11:13.951764+0000","last_undegraded":"2024-09-16T00:11:13.951764+0000","last_fullsized":"2024-09-16T00:11:13.951764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":22188,"num_write":360,"num_write_kb":71552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"122'57","reported_seq":316,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.942953+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:11:12.942953+0000","last_peered":"2024-09-16T00:11:12.942953+0000","last_clean":"2024-09-16T00:11:12.942953+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:11:12.942953+0000","last_undegraded":"2024-09-16T00:11:12.942953+0000","last_fullsized":"2024-09-16T00:11:12.942953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"123'426","reported_seq":725,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952294+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:11:13.952294+0000","last_peered":"2024-09-16T00:11:13.952294+0000","last_clean":"2024-09-16T00:11:13.952294+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:11:13.952294+0000","last_undegraded":"2024-09-16T00:11:13.952294+0000","last_fullsized":"2024-09-16T00:11:13.952294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":24752,"num_write":390,"num_write_kb":75609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"123'258","reported_seq":301,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137011+0000","last_change":"2024-09-16T00:10:58.839395+0000","last_active":"2024-09-16T00:11:13.137011+0000","last_peered":"2024-09-16T00:11:13.137011+0000","last_clean":"2024-09-16T00:11:13.137011+0000","last_became_active":"2024-09-16T00:10:58.839124+0000","last_became_peered":"2024-09-16T00:10:58.839124+0000","last_unstale":"2024-09-16T00:11:13.137011+0000","last_undegraded":"2024-09-16T00:11:13.137011+0000","last_fullsized":"2024-09-16T00:11:13.137011+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:29.136640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":202,"num_write_kb":4192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"123'398","reported_seq":713,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.942847+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:11:12.942847+0000","last_peered":"2024-09-16T00:11:12.942847+0000","last_clean":"2024-09-16T00:11:12.942847+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:11:12.942847+0000","last_undegraded":"2024-09-16T00:11:12.942847+0000","last_fullsized":"2024-09-16T00:11:12.942847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":31221,"num_write":370,"num_write_kb":76143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"125'186","reported_seq":480,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.943411+0000","last_change":"2024-09-16T00:11:04.894640+0000","last_active":"2024-09-16T00:11:12.943411+0000","last_peered":"2024-09-16T00:11:12.943411+0000","last_clean":"2024-09-16T00:11:12.943411+0000","last_became_active":"2024-09-16T00:11:04.894163+0000","last_became_peered":"2024-09-16T00:11:04.894163+0000","last_unstale":"2024-09-16T00:11:12.943411+0000","last_undegraded":"2024-09-16T00:11:12.943411+0000","last_fullsized":"2024-09-16T00:11:12.943411+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:41.395352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":345,"num_write_kb":1559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":2964,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"123'344","reported_seq":784,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544293+0000","last_change":"2024-09-16T00:10:56.334688+0000","last_active":"2024-09-16T00:11:13.544293+0000","last_peered":"2024-09-16T00:11:13.544293+0000","last_clean":"2024-09-16T00:11:13.544293+0000","last_became_active":"2024-09-16T00:10:55.845451+0000","last_became_peered":"2024-09-16T00:10:55.845451+0000","last_unstale":"2024-09-16T00:11:13.544293+0000","last_undegraded":"2024-09-16T00:11:13.544293+0000","last_fullsized":"2024-09-16T00:11:13.544293+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:04.965027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":19274,"num_write":323,"num_write_kb":22071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"123'63","reported_seq":309,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011696+0000","last_change":"2024-09-16T00:10:55.858845+0000","last_active":"2024-09-16T00:11:14.011696+0000","last_peered":"2024-09-16T00:11:14.011696+0000","last_clean":"2024-09-16T00:11:14.011696+0000","last_became_active":"2024-09-16T00:10:55.845687+0000","last_became_peered":"2024-09-16T00:10:55.845687+0000","last_unstale":"2024-09-16T00:11:14.011696+0000","last_undegraded":"2024-09-16T00:11:14.011696+0000","last_fullsized":"2024-09-16T00:11:14.011696+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:53.322675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"123'447","reported_seq":750,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381154+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:11:14.381154+0000","last_peered":"2024-09-16T00:11:14.381154+0000","last_clean":"2024-09-16T00:11:14.381154+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:11:14.381154+0000","last_undegraded":"2024-09-16T00:11:14.381154+0000","last_fullsized":"2024-09-16T00:11:14.381154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":17894,"num_write":426,"num_write_kb":88843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"123'564","reported_seq":784,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136761+0000","last_change":"2024-09-16T00:10:55.852895+0000","last_active":"2024-09-16T00:11:13.136761+0000","last_peered":"2024-09-16T00:11:13.136761+0000","last_clean":"2024-09-16T00:11:13.136761+0000","last_became_active":"2024-09-16T00:10:55.828121+0000","last_became_peered":"2024-09-16T00:10:55.828121+0000","last_unstale":"2024-09-16T00:11:13.136761+0000","last_undegraded":"2024-09-16T00:11:13.136761+0000","last_fullsized":"2024-09-16T00:11:13.136761+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:57.000428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"123'454","reported_seq":732,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951802+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:11:13.951802+0000","last_peered":"2024-09-16T00:11:13.951802+0000","last_clean":"2024-09-16T00:11:13.951802+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:11:13.951802+0000","last_undegraded":"2024-09-16T00:11:13.951802+0000","last_fullsized":"2024-09-16T00:11:13.951802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":9206,"num_write":434,"num_write_kb":79714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"122'57","reported_seq":327,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.942786+0000","last_change":"2024-09-16T00:10:55.850694+0000","last_active":"2024-09-16T00:11:12.942786+0000","last_peered":"2024-09-16T00:11:12.942786+0000","last_clean":"2024-09-16T00:11:12.942786+0000","last_became_active":"2024-09-16T00:10:55.837749+0000","last_became_peered":"2024-09-16T00:10:55.837749+0000","last_unstale":"2024-09-16T00:11:12.942786+0000","last_undegraded":"2024-09-16T00:11:12.942786+0000","last_fullsized":"2024-09-16T00:11:12.942786+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:53.941848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"123'382","reported_seq":1525,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544264+0000","last_change":"2024-09-16T00:11:04.892185+0000","last_active":"2024-09-16T00:11:13.544264+0000","last_peered":"2024-09-16T00:11:13.544264+0000","last_clean":"2024-09-16T00:11:13.544264+0000","last_became_active":"2024-09-16T00:11:04.890897+0000","last_became_peered":"2024-09-16T00:11:04.890897+0000","last_unstale":"2024-09-16T00:11:13.544264+0000","last_undegraded":"2024-09-16T00:11:13.544264+0000","last_fullsized":"2024-09-16T00:11:13.544264+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:18.504239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":8384,"num_write":359,"num_write_kb":50710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":421,"num_bytes_recovered":615552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"122'180","reported_seq":467,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.942719+0000","last_change":"2024-09-16T00:11:04.891195+0000","last_active":"2024-09-16T00:11:12.942719+0000","last_peered":"2024-09-16T00:11:12.942719+0000","last_clean":"2024-09-16T00:11:12.942719+0000","last_became_active":"2024-09-16T00:11:04.890722+0000","last_became_peered":"2024-09-16T00:11:04.890722+0000","last_unstale":"2024-09-16T00:11:12.942719+0000","last_undegraded":"2024-09-16T00:11:12.942719+0000","last_fullsized":"2024-09-16T00:11:12.942719+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:36.844666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":4191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":5577662,"num_keys_recovered":51,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"123'420","reported_seq":748,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544746+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:11:13.544746+0000","last_peered":"2024-09-16T00:11:13.544746+0000","last_clean":"2024-09-16T00:11:13.544746+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:11:13.544746+0000","last_undegraded":"2024-09-16T00:11:13.544746+0000","last_fullsized":"2024-09-16T00:11:13.544746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":37740,"num_write":400,"num_write_kb":67449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"122'114","reported_seq":323,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136851+0000","last_change":"2024-09-16T00:10:55.854939+0000","last_active":"2024-09-16T00:11:13.136851+0000","last_peered":"2024-09-16T00:11:13.136851+0000","last_clean":"2024-09-16T00:11:13.136851+0000","last_became_active":"2024-09-16T00:10:55.832841+0000","last_became_peered":"2024-09-16T00:10:55.832841+0000","last_unstale":"2024-09-16T00:11:13.136851+0000","last_undegraded":"2024-09-16T00:11:13.136851+0000","last_fullsized":"2024-09-16T00:11:13.136851+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:06:48.016972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":8303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"123'452","reported_seq":608,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011438+0000","last_change":"2024-09-16T00:10:00.699942+0000","last_active":"2024-09-16T00:11:14.011438+0000","last_peered":"2024-09-16T00:11:14.011438+0000","last_clean":"2024-09-16T00:11:14.011438+0000","last_became_active":"2024-09-16T00:10:00.281972+0000","last_became_peered":"2024-09-16T00:10:00.281972+0000","last_unstale":"2024-09-16T00:11:14.011438+0000","last_undegraded":"2024-09-16T00:11:14.011438+0000","last_fullsized":"2024-09-16T00:11:14.011438+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:21.943709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":43345,"num_write":425,"num_write_kb":121191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":29546741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"123'396","reported_seq":725,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952254+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:11:13.952254+0000","last_peered":"2024-09-16T00:11:13.952254+0000","last_clean":"2024-09-16T00:11:13.952254+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:11:13.952254+0000","last_undegraded":"2024-09-16T00:11:13.952254+0000","last_fullsized":"2024-09-16T00:11:13.952254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":32882,"num_write":368,"num_write_kb":108251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"123'219","reported_seq":436,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136812+0000","last_change":"2024-09-16T00:10:55.835894+0000","last_active":"2024-09-16T00:11:13.136812+0000","last_peered":"2024-09-16T00:11:13.136812+0000","last_clean":"2024-09-16T00:11:13.136812+0000","last_became_active":"2024-09-16T00:10:55.835301+0000","last_became_peered":"2024-09-16T00:10:55.835301+0000","last_unstale":"2024-09-16T00:11:13.136812+0000","last_undegraded":"2024-09-16T00:11:13.136812+0000","last_fullsized":"2024-09-16T00:11:13.136812+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:32.623825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":4194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"123'385","reported_seq":714,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951587+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:11:13.951587+0000","last_peered":"2024-09-16T00:11:13.951587+0000","last_clean":"2024-09-16T00:11:13.951587+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:11:13.951587+0000","last_undegraded":"2024-09-16T00:11:13.951587+0000","last_fullsized":"2024-09-16T00:11:13.951587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":36278,"num_write":367,"num_write_kb":113361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"123'263","reported_seq":492,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136841+0000","last_change":"2024-09-16T00:10:55.856431+0000","last_active":"2024-09-16T00:11:13.136841+0000","last_peered":"2024-09-16T00:11:13.136841+0000","last_clean":"2024-09-16T00:11:13.136841+0000","last_became_active":"2024-09-16T00:10:55.843843+0000","last_became_peered":"2024-09-16T00:10:55.843843+0000","last_unstale":"2024-09-16T00:11:13.136841+0000","last_undegraded":"2024-09-16T00:11:13.136841+0000","last_fullsized":"2024-09-16T00:11:13.136841+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:11.989047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":197,"num_write_kb":12428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":30,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"123'348","reported_seq":601,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952826+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:11:13.952826+0000","last_peered":"2024-09-16T00:11:13.952826+0000","last_clean":"2024-09-16T00:11:13.952826+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:11:13.952826+0000","last_undegraded":"2024-09-16T00:11:13.952826+0000","last_fullsized":"2024-09-16T00:11:13.952826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":12352,"num_write":323,"num_write_kb":38097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"122'93","reported_seq":354,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544743+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:11:13.544743+0000","last_peered":"2024-09-16T00:11:13.544743+0000","last_clean":"2024-09-16T00:11:13.544743+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:11:13.544743+0000","last_undegraded":"2024-09-16T00:11:13.544743+0000","last_fullsized":"2024-09-16T00:11:13.544743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":4141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"123'414","reported_seq":793,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544646+0000","last_change":"2024-09-16T00:10:00.556848+0000","last_active":"2024-09-16T00:11:13.544646+0000","last_peered":"2024-09-16T00:11:13.544646+0000","last_clean":"2024-09-16T00:11:13.544646+0000","last_became_active":"2024-09-16T00:10:00.283734+0000","last_became_peered":"2024-09-16T00:10:00.283734+0000","last_unstale":"2024-09-16T00:11:13.544646+0000","last_undegraded":"2024-09-16T00:11:13.544646+0000","last_fullsized":"2024-09-16T00:11:13.544646+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:55:01.843183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":40163,"num_write":392,"num_write_kb":81063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":21131701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"123'98","reported_seq":372,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381221+0000","last_change":"2024-09-16T00:10:55.856802+0000","last_active":"2024-09-16T00:11:14.381221+0000","last_peered":"2024-09-16T00:11:14.381221+0000","last_clean":"2024-09-16T00:11:14.381221+0000","last_became_active":"2024-09-16T00:10:55.844922+0000","last_became_peered":"2024-09-16T00:10:55.844922+0000","last_unstale":"2024-09-16T00:11:14.381221+0000","last_undegraded":"2024-09-16T00:11:14.381221+0000","last_fullsized":"2024-09-16T00:11:14.381221+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:19.134467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":4138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2561,"num_read_kb":1362012,"num_write":24417,"num_write_kb":4088607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3177,"num_bytes_recovered":294487447,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5234,"internal_metadata":0},"log_size":25917,"ondisk_log_size":25917,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":129048453,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":176,"num_read_kb":1534,"num_write":5618,"num_write_kb":131059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":26562382,"num_keys_recovered":3117,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86642688,"data_stored":258096906,"data_compressed":38739888,"data_compressed_allocated":49811456,"data_compressed_original":221323264,"omap_allocated":262411,"internal_metadata":0},"log_size":7144,"ondisk_log_size":7144,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":127,"seq":545460846598,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57992,"kb_used_data":16044,"kb_used_omap":42,"kb_used_meta":41877,"kb_avail":94313848,"statfs":{"total":96636764160,"available":96577380352,"internally_reserved":0,"allocated":16429056,"data_stored":51762927,"data_compressed":7683094,"data_compressed_allocated":9629696,"data_compressed_original":45862912,"omap_allocated":43696,"internal_metadata":42882384},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":94,"seq":403726925841,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56032,"kb_used_data":14280,"kb_used_omap":221,"kb_used_meta":41506,"kb_avail":94315808,"statfs":{"total":96636764160,"available":96579387392,"internally_reserved":0,"allocated":14622720,"data_stored":44546809,"data_compressed":6872875,"data_compressed_allocated":8704000,"data_compressed_original":39534592,"omap_allocated":227133,"internal_metadata":42502339},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509490,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73848,"kb_used_data":12128,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94297992,"statfs":{"total":96636764160,"available":96561143808,"internally_reserved":0,"allocated":12419072,"data_stored":26012395,"data_compressed":3584779,"data_compressed_allocated":4849664,"data_compressed_original":19345408,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509491,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63496,"kb_used_data":18412,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94308344,"statfs":{"total":96636764160,"available":96571744256,"internally_reserved":0,"allocated":18853888,"data_stored":46983951,"data_compressed":6735741,"data_compressed_allocated":8769536,"data_compressed_original":37810176,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51040,"kb_used_data":11528,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320800,"statfs":{"total":96636764160,"available":96584499200,"internally_reserved":0,"allocated":11804672,"data_stored":33418992,"data_compressed":5001493,"data_compressed_allocated":6361088,"data_compressed_original":28876800,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542194,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67128,"kb_used_data":9564,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94304712,"statfs":{"total":96636764160,"available":96568025088,"internally_reserved":0,"allocated":9793536,"data_stored":26006313,"data_compressed":3864818,"data_compressed_allocated":4833280,"data_compressed_original":21946368,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542194,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57372,"kb_used_data":12292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94314468,"statfs":{"total":96636764160,"available":96578015232,"internally_reserved":0,"allocated":12587008,"data_stored":30797615,"data_compressed":4284061,"data_compressed_allocated":5672960,"data_compressed_original":24797184,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35280,"kb_used_data":4088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336560,"statfs":{"total":96636764160,"available":96600637440,"internally_reserved":0,"allocated":4186112,"data_stored":6502032,"data_compressed":777781,"data_compressed_allocated":1081344,"data_compressed_original":4296704,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10776576,"data_stored":29360766,"data_compressed":4251684,"data_compressed_allocated":5627904,"data_compressed_original":24223744,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8060928,"data_stored":25165984,"data_compressed":3864818,"data_compressed_allocated":4833280,"data_compressed_original":21946368,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2441216,"data_stored":5655551,"data_compressed":777781,"data_compressed_allocated":1081344,"data_compressed_original":4296704,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10072064,"data_stored":32578663,"data_compressed":5001493,"data_compressed_allocated":6361088,"data_compressed_original":28876800,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17108992,"data_stored":46137470,"data_compressed":6735741,"data_compressed_allocated":8769536,"data_compressed_original":37810176,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10674176,"data_stored":25165914,"data_compressed":3584779,"data_compressed_allocated":4849664,"data_compressed_original":19345408,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12877824,"data_stored":43700328,"data_compressed":6872875,"data_compressed_allocated":8704000,"data_compressed_original":39534592,"omap_allocated":225539,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14630912,"data_stored":50332230,"data_compressed":7650717,"data_compressed_allocated":9584640,"data_compressed_original":45289472,"omap_allocated":36872,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5234,"internal_metadata":0}]}} 2024-09-16T00:11:17.499 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:11:17.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.631+0000 7fa36c518640 1 Processor -- start 2024-09-16T00:11:17.633 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.631+0000 7fa36c518640 1 -- start start 2024-09-16T00:11:17.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.632+0000 7fa36c518640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa364154db0 0x7fa36406cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.632+0000 7fa36c518640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa364074030 0x7fa3640ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.632+0000 7fa36c518640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa36415dfd0 con 0x7fa364072850 2024-09-16T00:11:17.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.632+0000 7fa36c518640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fa36406d4a0 con 0x7fa364074030 2024-09-16T00:11:17.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.632+0000 7fa36c518640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa36415e240 con 0x7fa364154db0 2024-09-16T00:11:17.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.633+0000 7fa36a2b4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa364154db0 0x7fa36406cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.635 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.633+0000 7fa369ab3640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa364074030 0x7fa3640ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.633+0000 7fa369ab3640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fa364074030 0x7fa3640ae850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.139:56790/0 (socket says 172.21.15.139:56790) 2024-09-16T00:11:17.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.633+0000 7fa36aab5640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fa364072850 0x7fa364073460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:37574/0 (socket says 172.21.15.139:37574) 2024-09-16T00:11:17.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.633+0000 7fa369ab3640 1 -- 172.21.15.139:0/590140025 learned_addr learned my addr 172.21.15.139:0/590140025 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:17.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.633+0000 7fa36a2b4640 1 --2- 172.21.15.139:0/590140025 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa364154db0 0x7fa36406cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:44520/0 (socket says 172.21.15.139:44520) 2024-09-16T00:11:17.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.633+0000 7fa36a2b4640 1 -- 172.21.15.139:0/590140025 >> v2:172.21.15.164:3300/0 conn(0x7fa364074030 msgr2=0x7fa3640ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:17.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.633+0000 7fa36a2b4640 1 --2- 172.21.15.139:0/590140025 >> v2:172.21.15.164:3300/0 conn(0x7fa364074030 0x7fa3640ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.633+0000 7fa36a2b4640 1 -- 172.21.15.139:0/590140025 >> v1:172.21.15.164:6789/0 conn(0x7fa364072850 legacy=0x7fa364073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.633+0000 7fa36a2b4640 1 -- 172.21.15.139:0/590140025 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36415e520 con 0x7fa364154db0 2024-09-16T00:11:17.636 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.633+0000 7fa369ab3640 1 --2- 172.21.15.139:0/590140025 >> v2:172.21.15.164:3300/0 conn(0x7fa364074030 0x7fa3640ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:11:17.639 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.634+0000 7fa36a2b4640 1 --2- 172.21.15.139:0/590140025 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa364154db0 0x7fa36406cc40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa3600097a0 tx=0x7fa36002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ddc3784d2128252 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:17.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.634+0000 7fa3692b2640 1 -- 172.21.15.139:0/590140025 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360037070 con 0x7fa364154db0 2024-09-16T00:11:17.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.634+0000 7fa3692b2640 1 -- 172.21.15.139:0/590140025 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa36002ae50 con 0x7fa364154db0 2024-09-16T00:11:17.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.634+0000 7fa3692b2640 1 -- 172.21.15.139:0/590140025 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360032660 con 0x7fa364154db0 2024-09-16T00:11:17.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.634+0000 7fa36c518640 1 -- 172.21.15.139:0/590140025 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa364154db0 msgr2=0x7fa36406cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.634+0000 7fa36c518640 1 --2- 172.21.15.139:0/590140025 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa364154db0 0x7fa36406cc40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa3600097a0 tx=0x7fa36002a100 comp rx=0 tx=0).stop 2024-09-16T00:11:17.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.634+0000 7fa36c518640 1 -- 172.21.15.139:0/590140025 shutdown_connections 2024-09-16T00:11:17.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.634+0000 7fa36c518640 1 --2- 172.21.15.139:0/590140025 >> v2:172.21.15.164:3300/0 conn(0x7fa364074030 0x7fa3640ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.634+0000 7fa36c518640 1 --2- 172.21.15.139:0/590140025 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa364154db0 0x7fa36406cc40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.634+0000 7fa36c518640 1 -- 172.21.15.139:0/590140025 >> 172.21.15.139:0/590140025 conn(0x7fa364070440 msgr2=0x7fa364070820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:17.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.635+0000 7fa36c518640 1 -- 172.21.15.139:0/590140025 shutdown_connections 2024-09-16T00:11:17.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.635+0000 7fa36c518640 1 -- 172.21.15.139:0/590140025 wait complete. 2024-09-16T00:11:17.640 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.636+0000 7fa36c518640 1 Processor -- start 2024-09-16T00:11:17.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.636+0000 7fa36c518640 1 -- start start 2024-09-16T00:11:17.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.636+0000 7fa36c518640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa364072850 0x7fa3640cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.636+0000 7fa36c518640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa364154db0 0x7fa3641e6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.636+0000 7fa36c518640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa364150640 0x7fa3641f07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa36c518640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa36415ed20 con 0x7fa364154db0 2024-09-16T00:11:17.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa36c518640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fa36406d780 con 0x7fa364072850 2024-09-16T00:11:17.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa36c518640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fa3640b01a0 con 0x7fa364150640 2024-09-16T00:11:17.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa369ab3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa364154db0 0x7fa3641e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa369ab3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa364154db0 0x7fa3641e6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43312/0 (socket says 172.21.15.139:43312) 2024-09-16T00:11:17.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa369ab3640 1 -- 172.21.15.139:0/744870075 learned_addr learned my addr 172.21.15.139:0/744870075 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:17.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa36a2b4640 1 --2- 172.21.15.139:0/744870075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa364072850 0x7fa3640cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa36aab5640 1 --2- 172.21.15.139:0/744870075 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa364150640 0x7fa3641f07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa369ab3640 1 -- 172.21.15.139:0/744870075 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa364150640 msgr2=0x7fa3641f07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.641 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa369ab3640 1 --2- 172.21.15.139:0/744870075 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa364150640 0x7fa3641f07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa369ab3640 1 -- 172.21.15.139:0/744870075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa364072850 msgr2=0x7fa3640cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa369ab3640 1 --2- 172.21.15.139:0/744870075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa364072850 0x7fa3640cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa369ab3640 1 -- 172.21.15.139:0/744870075 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3641f43b0 con 0x7fa364154db0 2024-09-16T00:11:17.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa369ab3640 1 --2- 172.21.15.139:0/744870075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa364154db0 0x7fa3641e6060 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa35400e870 tx=0x7fa35400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:17.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa35affd640 1 -- 172.21.15.139:0/744870075 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354017070 con 0x7fa364154db0 2024-09-16T00:11:17.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa35affd640 1 -- 172.21.15.139:0/744870075 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa35400c940 con 0x7fa364154db0 2024-09-16T00:11:17.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa35affd640 1 -- 172.21.15.139:0/744870075 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35400cd10 con 0x7fa364154db0 2024-09-16T00:11:17.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa36c518640 1 -- 172.21.15.139:0/744870075 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3641f4690 con 0x7fa364154db0 2024-09-16T00:11:17.642 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.637+0000 7fa36c518640 1 -- 172.21.15.139:0/744870075 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3640b06d0 con 0x7fa364154db0 2024-09-16T00:11:17.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.638+0000 7fa36c518640 1 -- 172.21.15.139:0/744870075 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa36406cc40 con 0x7fa364154db0 2024-09-16T00:11:17.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.639+0000 7fa35affd640 1 -- 172.21.15.139:0/744870075 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa35401a070 con 0x7fa364154db0 2024-09-16T00:11:17.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.639+0000 7fa35affd640 1 --2- 172.21.15.139:0/744870075 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa34806c2b0 0x7fa34806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.639+0000 7fa35affd640 1 -- 172.21.15.139:0/744870075 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7fa35408b630 con 0x7fa364154db0 2024-09-16T00:11:17.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.639+0000 7fa36a2b4640 1 --2- 172.21.15.139:0/744870075 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa34806c2b0 0x7fa34806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.643 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.640+0000 7fa36a2b4640 1 --2- 172.21.15.139:0/744870075 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa34806c2b0 0x7fa34806e770 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa360034000 tx=0x7fa36002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:17.645 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.644+0000 7fa35affd640 1 -- 172.21.15.139:0/744870075 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa354059660 con 0x7fa364154db0 2024-09-16T00:11:17.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.806+0000 7fa36c518640 1 -- 172.21.15.139:0/744870075 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa36415b870 con 0x7fa34806c2b0 2024-09-16T00:11:17.814 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.812+0000 7fa35affd640 1 -- 172.21.15.139:0/744870075 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254583 (secure 0 0 0) 0x7fa36415b870 con 0x7fa34806c2b0 2024-09-16T00:11:17.818 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:11:17.819 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:11:17.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.815+0000 7fa36c518640 1 -- 172.21.15.139:0/744870075 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa34806c2b0 msgr2=0x7fa34806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.815+0000 7fa36c518640 1 --2- 172.21.15.139:0/744870075 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa34806c2b0 0x7fa34806e770 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa360034000 tx=0x7fa36002e000 comp rx=0 tx=0).stop 2024-09-16T00:11:17.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.815+0000 7fa36c518640 1 -- 172.21.15.139:0/744870075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa364154db0 msgr2=0x7fa3641e6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.815+0000 7fa36c518640 1 --2- 172.21.15.139:0/744870075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa364154db0 0x7fa3641e6060 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa35400e870 tx=0x7fa35400ed30 comp rx=0 tx=0).stop 2024-09-16T00:11:17.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.815+0000 7fa36c518640 1 -- 172.21.15.139:0/744870075 shutdown_connections 2024-09-16T00:11:17.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.815+0000 7fa36c518640 1 --2- 172.21.15.139:0/744870075 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fa34806c2b0 0x7fa34806e770 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.815+0000 7fa36c518640 1 --2- 172.21.15.139:0/744870075 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fa364150640 0x7fa3641f07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.815+0000 7fa36c518640 1 --2- 172.21.15.139:0/744870075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa364154db0 0x7fa3641e6060 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.815+0000 7fa36c518640 1 --2- 172.21.15.139:0/744870075 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fa364072850 0x7fa3640cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.821 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.815+0000 7fa36c518640 1 -- 172.21.15.139:0/744870075 >> 172.21.15.139:0/744870075 conn(0x7fa364070440 msgr2=0x7fa364153e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:17.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.815+0000 7fa36c518640 1 -- 172.21.15.139:0/744870075 shutdown_connections 2024-09-16T00:11:17.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.815+0000 7fa36c518640 1 -- 172.21.15.139:0/744870075 wait complete. 2024-09-16T00:11:17.828 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-16T00:11:16.066417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":129638821,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":2853,"num_read_kb":1363646,"num_write":30194,"num_write_kb":4222082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3232,"num_bytes_recovered":321049829,"num_keys_recovered":3117,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33178,"ondisk_log_size":33178,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":462188,"kb_used_data":98336,"kb_used_omap":273,"kb_used_meta":363374,"kb_avail":754512532,"statfs":{"total":773094113280,"available":772620832768,"internally_reserved":0,"allocated":100696064,"data_stored":266031034,"data_compressed":38804642,"data_compressed_allocated":49901568,"data_compressed_original":222470144,"omap_allocated":280397,"internal_metadata":372095155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.145130"},"pg_stats":[{"pgid":"3.3f","version":"123'434","reported_seq":867,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137191+0000","last_change":"2024-09-16T00:10:56.416283+0000","last_active":"2024-09-16T00:11:13.137191+0000","last_peered":"2024-09-16T00:11:13.137191+0000","last_clean":"2024-09-16T00:11:13.137191+0000","last_became_active":"2024-09-16T00:10:55.845182+0000","last_became_peered":"2024-09-16T00:10:55.845182+0000","last_unstale":"2024-09-16T00:11:13.137191+0000","last_undegraded":"2024-09-16T00:11:13.137191+0000","last_fullsized":"2024-09-16T00:11:13.137191+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:52.137388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":42086,"num_write":409,"num_write_kb":112716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":31107821,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"123'417","reported_seq":1091,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.943324+0000","last_change":"2024-09-16T00:11:04.897161+0000","last_active":"2024-09-16T00:11:12.943324+0000","last_peered":"2024-09-16T00:11:12.943324+0000","last_clean":"2024-09-16T00:11:12.943324+0000","last_became_active":"2024-09-16T00:11:04.894999+0000","last_became_peered":"2024-09-16T00:11:04.894999+0000","last_unstale":"2024-09-16T00:11:12.943324+0000","last_undegraded":"2024-09-16T00:11:12.943324+0000","last_fullsized":"2024-09-16T00:11:12.943324+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:13.015652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":23064,"num_write":397,"num_write_kb":47220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":689591,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"123'367","reported_seq":657,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381045+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:11:14.381045+0000","last_peered":"2024-09-16T00:11:14.381045+0000","last_clean":"2024-09-16T00:11:14.381045+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:11:14.381045+0000","last_undegraded":"2024-09-16T00:11:14.381045+0000","last_fullsized":"2024-09-16T00:11:14.381045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15782,"num_write":349,"num_write_kb":54932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"123'389","reported_seq":679,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951475+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:11:13.951475+0000","last_peered":"2024-09-16T00:11:13.951475+0000","last_clean":"2024-09-16T00:11:13.951475+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:11:13.951475+0000","last_undegraded":"2024-09-16T00:11:13.951475+0000","last_fullsized":"2024-09-16T00:11:13.951475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":14080,"num_write":364,"num_write_kb":63191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"123'445","reported_seq":746,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.942767+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:11:12.942767+0000","last_peered":"2024-09-16T00:11:12.942767+0000","last_clean":"2024-09-16T00:11:12.942767+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:11:12.942767+0000","last_undegraded":"2024-09-16T00:11:12.942767+0000","last_fullsized":"2024-09-16T00:11:12.942767+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":36096,"num_write":427,"num_write_kb":89616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"123'384","reported_seq":1352,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952787+0000","last_change":"2024-09-16T00:10:45.795247+0000","last_active":"2024-09-16T00:11:13.952787+0000","last_peered":"2024-09-16T00:11:13.952787+0000","last_clean":"2024-09-16T00:11:13.952787+0000","last_became_active":"2024-09-16T00:10:45.793434+0000","last_became_peered":"2024-09-16T00:10:45.793434+0000","last_unstale":"2024-09-16T00:11:13.952787+0000","last_undegraded":"2024-09-16T00:11:13.952787+0000","last_fullsized":"2024-09-16T00:11:13.952787+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:55.019829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":6174,"num_write":362,"num_write_kb":50827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":425,"num_bytes_recovered":23619955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"123'420","reported_seq":779,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136743+0000","last_change":"2024-09-16T00:10:56.030536+0000","last_active":"2024-09-16T00:11:13.136743+0000","last_peered":"2024-09-16T00:11:13.136743+0000","last_clean":"2024-09-16T00:11:13.136743+0000","last_became_active":"2024-09-16T00:10:55.835516+0000","last_became_peered":"2024-09-16T00:10:55.835516+0000","last_unstale":"2024-09-16T00:11:13.136743+0000","last_undegraded":"2024-09-16T00:11:13.136743+0000","last_fullsized":"2024-09-16T00:11:13.136743+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:43.152015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":16577,"num_write":401,"num_write_kb":64470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"123'374","reported_seq":641,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952597+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:11:13.952597+0000","last_peered":"2024-09-16T00:11:13.952597+0000","last_clean":"2024-09-16T00:11:13.952597+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:11:13.952597+0000","last_undegraded":"2024-09-16T00:11:13.952597+0000","last_fullsized":"2024-09-16T00:11:13.952597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":16776,"num_write":347,"num_write_kb":80164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"123'394","reported_seq":724,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137150+0000","last_change":"2024-09-16T00:10:56.261956+0000","last_active":"2024-09-16T00:11:13.137150+0000","last_peered":"2024-09-16T00:11:13.137150+0000","last_clean":"2024-09-16T00:11:13.137150+0000","last_became_active":"2024-09-16T00:10:55.840604+0000","last_became_peered":"2024-09-16T00:10:55.840604+0000","last_unstale":"2024-09-16T00:11:13.137150+0000","last_undegraded":"2024-09-16T00:11:13.137150+0000","last_fullsized":"2024-09-16T00:11:13.137150+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:39.279660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1432,"num_write":368,"num_write_kb":30311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"123'367","reported_seq":536,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011843+0000","last_change":"2024-09-16T00:10:00.734336+0000","last_active":"2024-09-16T00:11:14.011843+0000","last_peered":"2024-09-16T00:11:14.011843+0000","last_clean":"2024-09-16T00:11:14.011843+0000","last_became_active":"2024-09-16T00:10:00.284452+0000","last_became_peered":"2024-09-16T00:10:00.284452+0000","last_unstale":"2024-09-16T00:11:14.011843+0000","last_undegraded":"2024-09-16T00:11:14.011843+0000","last_fullsized":"2024-09-16T00:11:14.011843+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:56:50.530439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4096,"num_write":335,"num_write_kb":29952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":8736255,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"123'355","reported_seq":535,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011551+0000","last_change":"2024-09-16T00:10:00.587110+0000","last_active":"2024-09-16T00:11:14.011551+0000","last_peered":"2024-09-16T00:11:14.011551+0000","last_clean":"2024-09-16T00:11:14.011551+0000","last_became_active":"2024-09-16T00:10:00.280273+0000","last_became_peered":"2024-09-16T00:10:00.280273+0000","last_unstale":"2024-09-16T00:11:14.011551+0000","last_undegraded":"2024-09-16T00:11:14.011551+0000","last_fullsized":"2024-09-16T00:11:14.011551+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:01:57.902939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":975,"num_write":339,"num_write_kb":25518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":10753732,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"123'366","reported_seq":705,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952011+0000","last_change":"2024-09-16T00:10:00.797372+0000","last_active":"2024-09-16T00:11:13.952011+0000","last_peered":"2024-09-16T00:11:13.952011+0000","last_clean":"2024-09-16T00:11:13.952011+0000","last_became_active":"2024-09-16T00:10:00.278765+0000","last_became_peered":"2024-09-16T00:10:00.278765+0000","last_unstale":"2024-09-16T00:11:13.952011+0000","last_undegraded":"2024-09-16T00:11:13.952011+0000","last_fullsized":"2024-09-16T00:11:13.952011+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:03.318919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":11563,"num_write":348,"num_write_kb":58933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":29565877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"123'419","reported_seq":782,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544719+0000","last_change":"2024-09-16T00:10:00.715647+0000","last_active":"2024-09-16T00:11:13.544719+0000","last_peered":"2024-09-16T00:11:13.544719+0000","last_clean":"2024-09-16T00:11:13.544719+0000","last_became_active":"2024-09-16T00:10:00.286050+0000","last_became_peered":"2024-09-16T00:10:00.286050+0000","last_unstale":"2024-09-16T00:11:13.544719+0000","last_undegraded":"2024-09-16T00:11:13.544719+0000","last_fullsized":"2024-09-16T00:11:13.544719+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:07.462899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":35908,"num_write":393,"num_write_kb":104863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":30230864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"123'415","reported_seq":724,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.380880+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:11:14.380880+0000","last_peered":"2024-09-16T00:11:14.380880+0000","last_clean":"2024-09-16T00:11:14.380880+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:11:14.380880+0000","last_undegraded":"2024-09-16T00:11:14.380880+0000","last_fullsized":"2024-09-16T00:11:14.380880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":23364,"num_write":390,"num_write_kb":43321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"123'392","reported_seq":682,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.942692+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:11:12.942692+0000","last_peered":"2024-09-16T00:11:12.942692+0000","last_clean":"2024-09-16T00:11:12.942692+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:11:12.942692+0000","last_undegraded":"2024-09-16T00:11:12.942692+0000","last_fullsized":"2024-09-16T00:11:12.942692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":14230,"num_write":362,"num_write_kb":33768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"123'439","reported_seq":756,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951004+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:11:13.951004+0000","last_peered":"2024-09-16T00:11:13.951004+0000","last_clean":"2024-09-16T00:11:13.951004+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:11:13.951004+0000","last_undegraded":"2024-09-16T00:11:13.951004+0000","last_fullsized":"2024-09-16T00:11:13.951004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":40234,"num_write":414,"num_write_kb":67785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"123'394","reported_seq":890,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952446+0000","last_change":"2024-09-16T00:10:56.577752+0000","last_active":"2024-09-16T00:11:13.952446+0000","last_peered":"2024-09-16T00:11:13.952446+0000","last_clean":"2024-09-16T00:11:13.952446+0000","last_became_active":"2024-09-16T00:10:55.845880+0000","last_became_peered":"2024-09-16T00:10:55.845880+0000","last_unstale":"2024-09-16T00:11:13.952446+0000","last_undegraded":"2024-09-16T00:11:13.952446+0000","last_fullsized":"2024-09-16T00:11:13.952446+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:32.037945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":22524,"num_write":374,"num_write_kb":46842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"123'391","reported_seq":720,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952778+0000","last_change":"2024-09-16T00:10:00.461437+0000","last_active":"2024-09-16T00:11:13.952778+0000","last_peered":"2024-09-16T00:11:13.952778+0000","last_clean":"2024-09-16T00:11:13.952778+0000","last_became_active":"2024-09-16T00:10:00.282592+0000","last_became_peered":"2024-09-16T00:10:00.282592+0000","last_unstale":"2024-09-16T00:11:13.952778+0000","last_undegraded":"2024-09-16T00:11:13.952778+0000","last_fullsized":"2024-09-16T00:11:13.952778+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:09.733353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8711,"num_write":366,"num_write_kb":46855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":21285626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"123'417","reported_seq":825,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137126+0000","last_change":"2024-09-16T00:10:56.142127+0000","last_active":"2024-09-16T00:11:13.137126+0000","last_peered":"2024-09-16T00:11:13.137126+0000","last_clean":"2024-09-16T00:11:13.137126+0000","last_became_active":"2024-09-16T00:10:55.832595+0000","last_became_peered":"2024-09-16T00:10:55.832595+0000","last_unstale":"2024-09-16T00:11:13.137126+0000","last_undegraded":"2024-09-16T00:11:13.137126+0000","last_fullsized":"2024-09-16T00:11:13.137126+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:44:56.072997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":14437,"num_write":396,"num_write_kb":83979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":103497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"123'423","reported_seq":602,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011546+0000","last_change":"2024-09-16T00:10:00.566870+0000","last_active":"2024-09-16T00:11:14.011546+0000","last_peered":"2024-09-16T00:11:14.011546+0000","last_clean":"2024-09-16T00:11:14.011546+0000","last_became_active":"2024-09-16T00:10:00.281258+0000","last_became_peered":"2024-09-16T00:10:00.281258+0000","last_unstale":"2024-09-16T00:11:14.011546+0000","last_undegraded":"2024-09-16T00:11:14.011546+0000","last_fullsized":"2024-09-16T00:11:14.011546+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:29.197743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":15502,"num_write":406,"num_write_kb":30222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":17199573,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"123'404","reported_seq":883,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381025+0000","last_change":"2024-09-16T00:10:56.104046+0000","last_active":"2024-09-16T00:11:14.381025+0000","last_peered":"2024-09-16T00:11:14.381025+0000","last_clean":"2024-09-16T00:11:14.381025+0000","last_became_active":"2024-09-16T00:10:55.835150+0000","last_became_peered":"2024-09-16T00:10:55.835150+0000","last_unstale":"2024-09-16T00:11:14.381025+0000","last_undegraded":"2024-09-16T00:11:14.381025+0000","last_fullsized":"2024-09-16T00:11:14.381025+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:08.552926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":34318,"num_write":385,"num_write_kb":91769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"123'444","reported_seq":918,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951970+0000","last_change":"2024-09-16T00:10:55.939360+0000","last_active":"2024-09-16T00:11:13.951970+0000","last_peered":"2024-09-16T00:11:13.951970+0000","last_clean":"2024-09-16T00:11:13.951970+0000","last_became_active":"2024-09-16T00:10:55.833335+0000","last_became_peered":"2024-09-16T00:10:55.833335+0000","last_unstale":"2024-09-16T00:11:13.951970+0000","last_undegraded":"2024-09-16T00:11:13.951970+0000","last_fullsized":"2024-09-16T00:11:13.951970+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:08:29.270004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":35528,"num_write":422,"num_write_kb":92852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"123'395","reported_seq":681,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544399+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:11:13.544399+0000","last_peered":"2024-09-16T00:11:13.544399+0000","last_clean":"2024-09-16T00:11:13.544399+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:11:13.544399+0000","last_undegraded":"2024-09-16T00:11:13.544399+0000","last_fullsized":"2024-09-16T00:11:13.544399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":13410,"num_write":369,"num_write_kb":58805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"123'421","reported_seq":703,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951092+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:11:13.951092+0000","last_peered":"2024-09-16T00:11:13.951092+0000","last_clean":"2024-09-16T00:11:13.951092+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:11:13.951092+0000","last_undegraded":"2024-09-16T00:11:13.951092+0000","last_fullsized":"2024-09-16T00:11:13.951092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":12682,"num_write":399,"num_write_kb":54869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"123'421","reported_seq":710,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544202+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:11:13.544202+0000","last_peered":"2024-09-16T00:11:13.544202+0000","last_clean":"2024-09-16T00:11:13.544202+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:11:13.544202+0000","last_undegraded":"2024-09-16T00:11:13.544202+0000","last_fullsized":"2024-09-16T00:11:13.544202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":14188,"num_write":408,"num_write_kb":59178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"123'448","reported_seq":862,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011709+0000","last_change":"2024-09-16T00:10:56.267499+0000","last_active":"2024-09-16T00:11:14.011709+0000","last_peered":"2024-09-16T00:11:14.011709+0000","last_clean":"2024-09-16T00:11:14.011709+0000","last_became_active":"2024-09-16T00:10:55.841967+0000","last_became_peered":"2024-09-16T00:10:55.841967+0000","last_unstale":"2024-09-16T00:11:14.011709+0000","last_undegraded":"2024-09-16T00:11:14.011709+0000","last_fullsized":"2024-09-16T00:11:14.011709+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:15.324718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":12330,"num_write":421,"num_write_kb":35146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":12903786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"123'402","reported_seq":1050,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.943063+0000","last_change":"2024-09-16T00:11:04.895240+0000","last_active":"2024-09-16T00:11:12.943063+0000","last_peered":"2024-09-16T00:11:12.943063+0000","last_clean":"2024-09-16T00:11:12.943063+0000","last_became_active":"2024-09-16T00:11:04.892794+0000","last_became_peered":"2024-09-16T00:11:04.892794+0000","last_unstale":"2024-09-16T00:11:12.943063+0000","last_undegraded":"2024-09-16T00:11:12.943063+0000","last_fullsized":"2024-09-16T00:11:12.943063+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:44.365114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":21104,"num_write":383,"num_write_kb":75742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":628439,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"123'386","reported_seq":686,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.543974+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:11:13.543974+0000","last_peered":"2024-09-16T00:11:13.543974+0000","last_clean":"2024-09-16T00:11:13.543974+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:11:13.543974+0000","last_undegraded":"2024-09-16T00:11:13.543974+0000","last_fullsized":"2024-09-16T00:11:13.543974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":16408,"num_write":366,"num_write_kb":51674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"123'398","reported_seq":717,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381121+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:11:14.381121+0000","last_peered":"2024-09-16T00:11:14.381121+0000","last_clean":"2024-09-16T00:11:14.381121+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:11:14.381121+0000","last_undegraded":"2024-09-16T00:11:14.381121+0000","last_fullsized":"2024-09-16T00:11:14.381121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":33174,"num_write":373,"num_write_kb":66849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"123'486","reported_seq":780,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952852+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:11:13.952852+0000","last_peered":"2024-09-16T00:11:13.952852+0000","last_clean":"2024-09-16T00:11:13.952852+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:11:13.952852+0000","last_undegraded":"2024-09-16T00:11:13.952852+0000","last_fullsized":"2024-09-16T00:11:13.952852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":27231,"num_write":453,"num_write_kb":104771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"123'451","reported_seq":789,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952668+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:11:13.952668+0000","last_peered":"2024-09-16T00:11:13.952668+0000","last_clean":"2024-09-16T00:11:13.952668+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:11:13.952668+0000","last_undegraded":"2024-09-16T00:11:13.952668+0000","last_fullsized":"2024-09-16T00:11:13.952668+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":35655,"num_write":431,"num_write_kb":97245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"123'421","reported_seq":602,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011470+0000","last_change":"2024-09-16T00:10:00.535553+0000","last_active":"2024-09-16T00:11:14.011470+0000","last_peered":"2024-09-16T00:11:14.011470+0000","last_clean":"2024-09-16T00:11:14.011470+0000","last_became_active":"2024-09-16T00:10:00.281378+0000","last_became_peered":"2024-09-16T00:10:00.281378+0000","last_unstale":"2024-09-16T00:11:14.011470+0000","last_undegraded":"2024-09-16T00:11:14.011470+0000","last_fullsized":"2024-09-16T00:11:14.011470+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:49.548709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20395,"num_write":401,"num_write_kb":32731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":22876917,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"123'401","reported_seq":741,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951565+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:11:13.951565+0000","last_peered":"2024-09-16T00:11:13.951565+0000","last_clean":"2024-09-16T00:11:13.951565+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:11:13.951565+0000","last_undegraded":"2024-09-16T00:11:13.951565+0000","last_fullsized":"2024-09-16T00:11:13.951565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":48141,"num_write":381,"num_write_kb":63611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"120'126","reported_seq":400,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544826+0000","last_change":"2024-09-16T00:10:55.854872+0000","last_active":"2024-09-16T00:11:13.544826+0000","last_peered":"2024-09-16T00:11:13.544826+0000","last_clean":"2024-09-16T00:11:13.544826+0000","last_became_active":"2024-09-16T00:10:55.843367+0000","last_became_peered":"2024-09-16T00:10:55.843367+0000","last_unstale":"2024-09-16T00:11:13.544826+0000","last_undegraded":"2024-09-16T00:11:13.544826+0000","last_fullsized":"2024-09-16T00:11:13.544826+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:07.797903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"123'346","reported_seq":633,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381167+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:11:14.381167+0000","last_peered":"2024-09-16T00:11:14.381167+0000","last_clean":"2024-09-16T00:11:14.381167+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:11:14.381167+0000","last_undegraded":"2024-09-16T00:11:14.381167+0000","last_fullsized":"2024-09-16T00:11:14.381167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":16420,"num_write":326,"num_write_kb":62710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"122'340","reported_seq":505,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011382+0000","last_change":"2024-09-16T00:10:00.282958+0000","last_active":"2024-09-16T00:11:14.011382+0000","last_peered":"2024-09-16T00:11:14.011382+0000","last_clean":"2024-09-16T00:11:14.011382+0000","last_became_active":"2024-09-16T00:10:00.282541+0000","last_became_peered":"2024-09-16T00:10:00.282541+0000","last_unstale":"2024-09-16T00:11:14.011382+0000","last_undegraded":"2024-09-16T00:11:14.011382+0000","last_fullsized":"2024-09-16T00:11:14.011382+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:51.198787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":285,"num_write_kb":12471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"123'226","reported_seq":508,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544507+0000","last_change":"2024-09-16T00:10:00.313589+0000","last_active":"2024-09-16T00:11:13.544507+0000","last_peered":"2024-09-16T00:11:13.544507+0000","last_clean":"2024-09-16T00:11:13.544507+0000","last_became_active":"2024-09-16T00:10:00.284602+0000","last_became_peered":"2024-09-16T00:10:00.284602+0000","last_unstale":"2024-09-16T00:11:13.544507+0000","last_undegraded":"2024-09-16T00:11:13.544507+0000","last_fullsized":"2024-09-16T00:11:13.544507+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:30.032698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":14,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"123'433","reported_seq":730,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544169+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:11:13.544169+0000","last_peered":"2024-09-16T00:11:13.544169+0000","last_clean":"2024-09-16T00:11:13.544169+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:11:13.544169+0000","last_undegraded":"2024-09-16T00:11:13.544169+0000","last_fullsized":"2024-09-16T00:11:13.544169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":18480,"num_write":411,"num_write_kb":71748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"123'833","reported_seq":1095,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951164+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:11:13.951164+0000","last_peered":"2024-09-16T00:11:13.951164+0000","last_clean":"2024-09-16T00:11:13.951164+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:11:13.951164+0000","last_undegraded":"2024-09-16T00:11:13.951164+0000","last_fullsized":"2024-09-16T00:11:13.951164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":8591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"123'425","reported_seq":772,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.942671+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:11:12.942671+0000","last_peered":"2024-09-16T00:11:12.942671+0000","last_clean":"2024-09-16T00:11:12.942671+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:11:12.942671+0000","last_undegraded":"2024-09-16T00:11:12.942671+0000","last_fullsized":"2024-09-16T00:11:12.942671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":46871,"num_write":401,"num_write_kb":84187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"123'65","reported_seq":296,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011701+0000","last_change":"2024-09-16T00:10:00.283876+0000","last_active":"2024-09-16T00:11:14.011701+0000","last_peered":"2024-09-16T00:11:14.011701+0000","last_clean":"2024-09-16T00:11:14.011701+0000","last_became_active":"2024-09-16T00:10:00.283555+0000","last_became_peered":"2024-09-16T00:10:00.283555+0000","last_unstale":"2024-09-16T00:11:14.011701+0000","last_undegraded":"2024-09-16T00:11:14.011701+0000","last_fullsized":"2024-09-16T00:11:14.011701+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:48.543737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"123'440","reported_seq":904,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951434+0000","last_change":"2024-09-16T00:10:56.502658+0000","last_active":"2024-09-16T00:11:13.951434+0000","last_peered":"2024-09-16T00:11:13.951434+0000","last_clean":"2024-09-16T00:11:13.951434+0000","last_became_active":"2024-09-16T00:10:55.840027+0000","last_became_peered":"2024-09-16T00:10:55.840027+0000","last_unstale":"2024-09-16T00:11:13.951434+0000","last_undegraded":"2024-09-16T00:11:13.951434+0000","last_fullsized":"2024-09-16T00:11:13.951434+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:52.105868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":16686,"num_write":415,"num_write_kb":80112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"123'225","reported_seq":494,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951555+0000","last_change":"2024-09-16T00:10:00.285386+0000","last_active":"2024-09-16T00:11:13.951555+0000","last_peered":"2024-09-16T00:11:13.951555+0000","last_clean":"2024-09-16T00:11:13.951555+0000","last_became_active":"2024-09-16T00:10:00.284717+0000","last_became_peered":"2024-09-16T00:10:00.284717+0000","last_unstale":"2024-09-16T00:11:13.951555+0000","last_undegraded":"2024-09-16T00:11:13.951555+0000","last_fullsized":"2024-09-16T00:11:13.951555+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:07.794163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":4181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"123'378","reported_seq":681,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381204+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:11:14.381204+0000","last_peered":"2024-09-16T00:11:14.381204+0000","last_clean":"2024-09-16T00:11:14.381204+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:11:14.381204+0000","last_undegraded":"2024-09-16T00:11:14.381204+0000","last_fullsized":"2024-09-16T00:11:14.381204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":18030,"num_write":359,"num_write_kb":55428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"125'131","reported_seq":390,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.943147+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:11:12.943147+0000","last_peered":"2024-09-16T00:11:12.943147+0000","last_clean":"2024-09-16T00:11:12.943147+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:11:12.943147+0000","last_undegraded":"2024-09-16T00:11:12.943147+0000","last_fullsized":"2024-09-16T00:11:12.943147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"123'373","reported_seq":818,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952022+0000","last_change":"2024-09-16T00:10:56.173134+0000","last_active":"2024-09-16T00:11:13.952022+0000","last_peered":"2024-09-16T00:11:13.952022+0000","last_clean":"2024-09-16T00:11:13.952022+0000","last_became_active":"2024-09-16T00:10:55.833273+0000","last_became_peered":"2024-09-16T00:10:55.833273+0000","last_unstale":"2024-09-16T00:11:13.952022+0000","last_undegraded":"2024-09-16T00:11:13.952022+0000","last_fullsized":"2024-09-16T00:11:13.952022+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:11.402988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":22960,"num_write":348,"num_write_kb":63161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"125'238","reported_seq":414,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011582+0000","last_change":"2024-09-16T00:10:00.300895+0000","last_active":"2024-09-16T00:11:14.011582+0000","last_peered":"2024-09-16T00:11:14.011582+0000","last_clean":"2024-09-16T00:11:14.011582+0000","last_became_active":"2024-09-16T00:10:00.283474+0000","last_became_peered":"2024-09-16T00:10:00.283474+0000","last_unstale":"2024-09-16T00:11:14.011582+0000","last_undegraded":"2024-09-16T00:11:14.011582+0000","last_fullsized":"2024-09-16T00:11:14.011582+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:28:02.563817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10145896,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":186,"num_write_kb":10014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":13200,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"123'403","reported_seq":774,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136869+0000","last_change":"2024-09-16T00:10:56.320793+0000","last_active":"2024-09-16T00:11:13.136869+0000","last_peered":"2024-09-16T00:11:13.136869+0000","last_clean":"2024-09-16T00:11:13.136869+0000","last_became_active":"2024-09-16T00:10:55.836462+0000","last_became_peered":"2024-09-16T00:10:55.836462+0000","last_unstale":"2024-09-16T00:11:13.136869+0000","last_undegraded":"2024-09-16T00:11:13.136869+0000","last_fullsized":"2024-09-16T00:11:13.136869+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:07:47.717822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":23285,"num_write":372,"num_write_kb":67093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"122'470","reported_seq":732,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381212+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:11:14.381212+0000","last_peered":"2024-09-16T00:11:14.381212+0000","last_clean":"2024-09-16T00:11:14.381212+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:11:14.381212+0000","last_undegraded":"2024-09-16T00:11:14.381212+0000","last_fullsized":"2024-09-16T00:11:14.381212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":419,"num_write_kb":8413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"123'404","reported_seq":872,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.380881+0000","last_change":"2024-09-16T00:10:56.427976+0000","last_active":"2024-09-16T00:11:14.380881+0000","last_peered":"2024-09-16T00:11:14.380881+0000","last_clean":"2024-09-16T00:11:14.380881+0000","last_became_active":"2024-09-16T00:10:55.839196+0000","last_became_peered":"2024-09-16T00:10:55.839196+0000","last_unstale":"2024-09-16T00:11:14.380881+0000","last_undegraded":"2024-09-16T00:11:14.380881+0000","last_fullsized":"2024-09-16T00:11:14.380881+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:37.852194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":13263,"num_write":381,"num_write_kb":51174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"123'59","reported_seq":321,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952149+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:11:13.952149+0000","last_peered":"2024-09-16T00:11:13.952149+0000","last_clean":"2024-09-16T00:11:13.952149+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:11:13.952149+0000","last_undegraded":"2024-09-16T00:11:13.952149+0000","last_fullsized":"2024-09-16T00:11:13.952149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"123'394","reported_seq":733,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136975+0000","last_change":"2024-09-16T00:10:56.085067+0000","last_active":"2024-09-16T00:11:13.136975+0000","last_peered":"2024-09-16T00:11:13.136975+0000","last_clean":"2024-09-16T00:11:13.136975+0000","last_became_active":"2024-09-16T00:10:55.834112+0000","last_became_peered":"2024-09-16T00:10:55.834112+0000","last_unstale":"2024-09-16T00:11:13.136975+0000","last_undegraded":"2024-09-16T00:11:13.136975+0000","last_fullsized":"2024-09-16T00:11:13.136975+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:35.278069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":13990,"num_write":368,"num_write_kb":63962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"120'70","reported_seq":218,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011398+0000","last_change":"2024-09-16T00:10:00.294519+0000","last_active":"2024-09-16T00:11:14.011398+0000","last_peered":"2024-09-16T00:11:14.011398+0000","last_clean":"2024-09-16T00:11:14.011398+0000","last_became_active":"2024-09-16T00:10:00.283066+0000","last_became_peered":"2024-09-16T00:10:00.283066+0000","last_unstale":"2024-09-16T00:11:14.011398+0000","last_undegraded":"2024-09-16T00:11:14.011398+0000","last_fullsized":"2024-09-16T00:11:14.011398+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:32.621176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"123'424","reported_seq":607,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011378+0000","last_change":"2024-09-16T00:10:00.460910+0000","last_active":"2024-09-16T00:11:14.011378+0000","last_peered":"2024-09-16T00:11:14.011378+0000","last_clean":"2024-09-16T00:11:14.011378+0000","last_became_active":"2024-09-16T00:10:00.280750+0000","last_became_peered":"2024-09-16T00:10:00.280750+0000","last_unstale":"2024-09-16T00:11:14.011378+0000","last_undegraded":"2024-09-16T00:11:14.011378+0000","last_fullsized":"2024-09-16T00:11:14.011378+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:47.634650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":19573,"num_write":402,"num_write_kb":50826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":33491520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"125'72","reported_seq":334,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952685+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:11:13.952685+0000","last_peered":"2024-09-16T00:11:13.952685+0000","last_clean":"2024-09-16T00:11:13.952685+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:11:13.952685+0000","last_undegraded":"2024-09-16T00:11:13.952685+0000","last_fullsized":"2024-09-16T00:11:13.952685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"123'457","reported_seq":814,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136679+0000","last_change":"2024-09-16T00:10:55.922694+0000","last_active":"2024-09-16T00:11:13.136679+0000","last_peered":"2024-09-16T00:11:13.136679+0000","last_clean":"2024-09-16T00:11:13.136679+0000","last_became_active":"2024-09-16T00:10:55.827989+0000","last_became_peered":"2024-09-16T00:10:55.827989+0000","last_unstale":"2024-09-16T00:11:13.136679+0000","last_undegraded":"2024-09-16T00:11:13.136679+0000","last_fullsized":"2024-09-16T00:11:13.136679+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:23.666346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20419,"num_write":427,"num_write_kb":43035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"123'145","reported_seq":407,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951453+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:11:13.951453+0000","last_peered":"2024-09-16T00:11:13.951453+0000","last_clean":"2024-09-16T00:11:13.951453+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:11:13.951453+0000","last_undegraded":"2024-09-16T00:11:13.951453+0000","last_fullsized":"2024-09-16T00:11:13.951453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91,"num_write_kb":4217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"123'358","reported_seq":694,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137061+0000","last_change":"2024-09-16T00:10:55.971163+0000","last_active":"2024-09-16T00:11:13.137061+0000","last_peered":"2024-09-16T00:11:13.137061+0000","last_clean":"2024-09-16T00:11:13.137061+0000","last_became_active":"2024-09-16T00:10:55.834401+0000","last_became_peered":"2024-09-16T00:10:55.834401+0000","last_unstale":"2024-09-16T00:11:13.137061+0000","last_undegraded":"2024-09-16T00:11:13.137061+0000","last_fullsized":"2024-09-16T00:11:13.137061+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:05.466545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":10790,"num_write":330,"num_write_kb":21585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"120'282","reported_seq":544,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952893+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:11:13.952893+0000","last_peered":"2024-09-16T00:11:13.952893+0000","last_clean":"2024-09-16T00:11:13.952893+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:11:13.952893+0000","last_undegraded":"2024-09-16T00:11:13.952893+0000","last_fullsized":"2024-09-16T00:11:13.952893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":238,"num_write_kb":4238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"123'372","reported_seq":696,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137259+0000","last_change":"2024-09-16T00:10:56.200838+0000","last_active":"2024-09-16T00:11:13.137259+0000","last_peered":"2024-09-16T00:11:13.137259+0000","last_clean":"2024-09-16T00:11:13.137259+0000","last_became_active":"2024-09-16T00:10:55.838254+0000","last_became_peered":"2024-09-16T00:10:55.838254+0000","last_unstale":"2024-09-16T00:11:13.137259+0000","last_undegraded":"2024-09-16T00:11:13.137259+0000","last_fullsized":"2024-09-16T00:11:13.137259+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:30.944542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":350,"num_write":351,"num_write_kb":38221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"122'534","reported_seq":793,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.943228+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:11:12.943228+0000","last_peered":"2024-09-16T00:11:12.943228+0000","last_clean":"2024-09-16T00:11:12.943228+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:11:12.943228+0000","last_undegraded":"2024-09-16T00:11:12.943228+0000","last_fullsized":"2024-09-16T00:11:12.943228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":473,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"123'418","reported_seq":682,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951696+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:11:13.951696+0000","last_peered":"2024-09-16T00:11:13.951696+0000","last_clean":"2024-09-16T00:11:13.951696+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:11:13.951696+0000","last_undegraded":"2024-09-16T00:11:13.951696+0000","last_fullsized":"2024-09-16T00:11:13.951696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":675,"num_write":389,"num_write_kb":34696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"123'72","reported_seq":334,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952967+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:11:13.952967+0000","last_peered":"2024-09-16T00:11:13.952967+0000","last_clean":"2024-09-16T00:11:13.952967+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:11:13.952967+0000","last_undegraded":"2024-09-16T00:11:13.952967+0000","last_fullsized":"2024-09-16T00:11:13.952967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"122'397","reported_seq":761,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136957+0000","last_change":"2024-09-16T00:10:56.368023+0000","last_active":"2024-09-16T00:11:13.136957+0000","last_peered":"2024-09-16T00:11:13.136957+0000","last_clean":"2024-09-16T00:11:13.136957+0000","last_became_active":"2024-09-16T00:10:55.839714+0000","last_became_peered":"2024-09-16T00:10:55.839714+0000","last_unstale":"2024-09-16T00:11:13.136957+0000","last_undegraded":"2024-09-16T00:11:13.136957+0000","last_fullsized":"2024-09-16T00:11:13.136957+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:04.192344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":22590,"num_write":379,"num_write_kb":59088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":372,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136608+0000","last_change":"2024-09-16T00:10:55.832444+0000","last_active":"2024-09-16T00:11:13.136608+0000","last_peered":"2024-09-16T00:11:13.136608+0000","last_clean":"2024-09-16T00:11:13.136608+0000","last_became_active":"2024-09-16T00:10:55.831881+0000","last_became_peered":"2024-09-16T00:10:55.831881+0000","last_unstale":"2024-09-16T00:11:13.136608+0000","last_undegraded":"2024-09-16T00:11:13.136608+0000","last_fullsized":"2024-09-16T00:11:13.136608+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:48.243166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"123'63","reported_seq":325,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951198+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:11:13.951198+0000","last_peered":"2024-09-16T00:11:13.951198+0000","last_clean":"2024-09-16T00:11:13.951198+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:11:13.951198+0000","last_undegraded":"2024-09-16T00:11:13.951198+0000","last_fullsized":"2024-09-16T00:11:13.951198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"123'369","reported_seq":662,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544613+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:11:13.544613+0000","last_peered":"2024-09-16T00:11:13.544613+0000","last_clean":"2024-09-16T00:11:13.544613+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:11:13.544613+0000","last_undegraded":"2024-09-16T00:11:13.544613+0000","last_fullsized":"2024-09-16T00:11:13.544613+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":16103,"num_write":342,"num_write_kb":46241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"123'399","reported_seq":643,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136617+0000","last_change":"2024-09-16T00:11:04.896119+0000","last_active":"2024-09-16T00:11:13.136617+0000","last_peered":"2024-09-16T00:11:13.136617+0000","last_clean":"2024-09-16T00:11:13.136617+0000","last_became_active":"2024-09-16T00:11:04.895672+0000","last_became_peered":"2024-09-16T00:11:04.895672+0000","last_unstale":"2024-09-16T00:11:13.136617+0000","last_undegraded":"2024-09-16T00:11:13.136617+0000","last_fullsized":"2024-09-16T00:11:13.136617+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:45.173943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":346,"num_write_kb":8359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":16777216,"num_keys_recovered":28,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"123'393","reported_seq":700,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.380785+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:11:14.380785+0000","last_peered":"2024-09-16T00:11:14.380785+0000","last_clean":"2024-09-16T00:11:14.380785+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:11:14.380785+0000","last_undegraded":"2024-09-16T00:11:14.380785+0000","last_fullsized":"2024-09-16T00:11:14.380785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":29385,"num_write":362,"num_write_kb":83610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"125'242","reported_seq":680,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952710+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:11:13.952710+0000","last_peered":"2024-09-16T00:11:13.952710+0000","last_clean":"2024-09-16T00:11:13.952710+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:11:13.952710+0000","last_undegraded":"2024-09-16T00:11:13.952710+0000","last_fullsized":"2024-09-16T00:11:13.952710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461247,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":176,"num_read_kb":1534,"num_write":179,"num_write_kb":1542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"123'400","reported_seq":710,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.942629+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:11:12.942629+0000","last_peered":"2024-09-16T00:11:12.942629+0000","last_clean":"2024-09-16T00:11:12.942629+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:11:12.942629+0000","last_undegraded":"2024-09-16T00:11:12.942629+0000","last_fullsized":"2024-09-16T00:11:12.942629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":28288,"num_write":378,"num_write_kb":66949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"119'400","reported_seq":662,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952094+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:11:13.952094+0000","last_peered":"2024-09-16T00:11:13.952094+0000","last_clean":"2024-09-16T00:11:13.952094+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:11:13.952094+0000","last_undegraded":"2024-09-16T00:11:13.952094+0000","last_fullsized":"2024-09-16T00:11:13.952094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":353,"num_write_kb":8381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"123'387","reported_seq":683,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951764+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:11:13.951764+0000","last_peered":"2024-09-16T00:11:13.951764+0000","last_clean":"2024-09-16T00:11:13.951764+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:11:13.951764+0000","last_undegraded":"2024-09-16T00:11:13.951764+0000","last_fullsized":"2024-09-16T00:11:13.951764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":22188,"num_write":360,"num_write_kb":71552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"122'57","reported_seq":316,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.942953+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:11:12.942953+0000","last_peered":"2024-09-16T00:11:12.942953+0000","last_clean":"2024-09-16T00:11:12.942953+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:11:12.942953+0000","last_undegraded":"2024-09-16T00:11:12.942953+0000","last_fullsized":"2024-09-16T00:11:12.942953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"123'426","reported_seq":725,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952294+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:11:13.952294+0000","last_peered":"2024-09-16T00:11:13.952294+0000","last_clean":"2024-09-16T00:11:13.952294+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:11:13.952294+0000","last_undegraded":"2024-09-16T00:11:13.952294+0000","last_fullsized":"2024-09-16T00:11:13.952294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":24752,"num_write":390,"num_write_kb":75609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"123'258","reported_seq":301,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137011+0000","last_change":"2024-09-16T00:10:58.839395+0000","last_active":"2024-09-16T00:11:13.137011+0000","last_peered":"2024-09-16T00:11:13.137011+0000","last_clean":"2024-09-16T00:11:13.137011+0000","last_became_active":"2024-09-16T00:10:58.839124+0000","last_became_peered":"2024-09-16T00:10:58.839124+0000","last_unstale":"2024-09-16T00:11:13.137011+0000","last_undegraded":"2024-09-16T00:11:13.137011+0000","last_fullsized":"2024-09-16T00:11:13.137011+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:29.136640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":202,"num_write_kb":4192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"123'398","reported_seq":713,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.942847+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:11:12.942847+0000","last_peered":"2024-09-16T00:11:12.942847+0000","last_clean":"2024-09-16T00:11:12.942847+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:11:12.942847+0000","last_undegraded":"2024-09-16T00:11:12.942847+0000","last_fullsized":"2024-09-16T00:11:12.942847+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":31221,"num_write":370,"num_write_kb":76143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"125'186","reported_seq":480,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.943411+0000","last_change":"2024-09-16T00:11:04.894640+0000","last_active":"2024-09-16T00:11:12.943411+0000","last_peered":"2024-09-16T00:11:12.943411+0000","last_clean":"2024-09-16T00:11:12.943411+0000","last_became_active":"2024-09-16T00:11:04.894163+0000","last_became_peered":"2024-09-16T00:11:04.894163+0000","last_unstale":"2024-09-16T00:11:12.943411+0000","last_undegraded":"2024-09-16T00:11:12.943411+0000","last_fullsized":"2024-09-16T00:11:12.943411+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:41.395352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":345,"num_write_kb":1559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":2964,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"123'344","reported_seq":784,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544293+0000","last_change":"2024-09-16T00:10:56.334688+0000","last_active":"2024-09-16T00:11:13.544293+0000","last_peered":"2024-09-16T00:11:13.544293+0000","last_clean":"2024-09-16T00:11:13.544293+0000","last_became_active":"2024-09-16T00:10:55.845451+0000","last_became_peered":"2024-09-16T00:10:55.845451+0000","last_unstale":"2024-09-16T00:11:13.544293+0000","last_undegraded":"2024-09-16T00:11:13.544293+0000","last_fullsized":"2024-09-16T00:11:13.544293+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:04.965027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":19274,"num_write":323,"num_write_kb":22071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"123'63","reported_seq":309,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011696+0000","last_change":"2024-09-16T00:10:55.858845+0000","last_active":"2024-09-16T00:11:14.011696+0000","last_peered":"2024-09-16T00:11:14.011696+0000","last_clean":"2024-09-16T00:11:14.011696+0000","last_became_active":"2024-09-16T00:10:55.845687+0000","last_became_peered":"2024-09-16T00:10:55.845687+0000","last_unstale":"2024-09-16T00:11:14.011696+0000","last_undegraded":"2024-09-16T00:11:14.011696+0000","last_fullsized":"2024-09-16T00:11:14.011696+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:53.322675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"123'447","reported_seq":750,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381154+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:11:14.381154+0000","last_peered":"2024-09-16T00:11:14.381154+0000","last_clean":"2024-09-16T00:11:14.381154+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:11:14.381154+0000","last_undegraded":"2024-09-16T00:11:14.381154+0000","last_fullsized":"2024-09-16T00:11:14.381154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":17894,"num_write":426,"num_write_kb":88843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"123'564","reported_seq":784,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136761+0000","last_change":"2024-09-16T00:10:55.852895+0000","last_active":"2024-09-16T00:11:13.136761+0000","last_peered":"2024-09-16T00:11:13.136761+0000","last_clean":"2024-09-16T00:11:13.136761+0000","last_became_active":"2024-09-16T00:10:55.828121+0000","last_became_peered":"2024-09-16T00:10:55.828121+0000","last_unstale":"2024-09-16T00:11:13.136761+0000","last_undegraded":"2024-09-16T00:11:13.136761+0000","last_fullsized":"2024-09-16T00:11:13.136761+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:57.000428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"123'454","reported_seq":732,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951802+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:11:13.951802+0000","last_peered":"2024-09-16T00:11:13.951802+0000","last_clean":"2024-09-16T00:11:13.951802+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:11:13.951802+0000","last_undegraded":"2024-09-16T00:11:13.951802+0000","last_fullsized":"2024-09-16T00:11:13.951802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":9206,"num_write":434,"num_write_kb":79714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"122'57","reported_seq":327,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.942786+0000","last_change":"2024-09-16T00:10:55.850694+0000","last_active":"2024-09-16T00:11:12.942786+0000","last_peered":"2024-09-16T00:11:12.942786+0000","last_clean":"2024-09-16T00:11:12.942786+0000","last_became_active":"2024-09-16T00:10:55.837749+0000","last_became_peered":"2024-09-16T00:10:55.837749+0000","last_unstale":"2024-09-16T00:11:12.942786+0000","last_undegraded":"2024-09-16T00:11:12.942786+0000","last_fullsized":"2024-09-16T00:11:12.942786+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:53.941848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"123'382","reported_seq":1525,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544264+0000","last_change":"2024-09-16T00:11:04.892185+0000","last_active":"2024-09-16T00:11:13.544264+0000","last_peered":"2024-09-16T00:11:13.544264+0000","last_clean":"2024-09-16T00:11:13.544264+0000","last_became_active":"2024-09-16T00:11:04.890897+0000","last_became_peered":"2024-09-16T00:11:04.890897+0000","last_unstale":"2024-09-16T00:11:13.544264+0000","last_undegraded":"2024-09-16T00:11:13.544264+0000","last_fullsized":"2024-09-16T00:11:13.544264+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:18.504239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":8384,"num_write":359,"num_write_kb":50710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":421,"num_bytes_recovered":615552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"122'180","reported_seq":467,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:12.942719+0000","last_change":"2024-09-16T00:11:04.891195+0000","last_active":"2024-09-16T00:11:12.942719+0000","last_peered":"2024-09-16T00:11:12.942719+0000","last_clean":"2024-09-16T00:11:12.942719+0000","last_became_active":"2024-09-16T00:11:04.890722+0000","last_became_peered":"2024-09-16T00:11:04.890722+0000","last_unstale":"2024-09-16T00:11:12.942719+0000","last_undegraded":"2024-09-16T00:11:12.942719+0000","last_fullsized":"2024-09-16T00:11:12.942719+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:36.844666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":4191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":5577662,"num_keys_recovered":51,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"123'420","reported_seq":748,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544746+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:11:13.544746+0000","last_peered":"2024-09-16T00:11:13.544746+0000","last_clean":"2024-09-16T00:11:13.544746+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:11:13.544746+0000","last_undegraded":"2024-09-16T00:11:13.544746+0000","last_fullsized":"2024-09-16T00:11:13.544746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":37740,"num_write":400,"num_write_kb":67449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"122'114","reported_seq":323,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136851+0000","last_change":"2024-09-16T00:10:55.854939+0000","last_active":"2024-09-16T00:11:13.136851+0000","last_peered":"2024-09-16T00:11:13.136851+0000","last_clean":"2024-09-16T00:11:13.136851+0000","last_became_active":"2024-09-16T00:10:55.832841+0000","last_became_peered":"2024-09-16T00:10:55.832841+0000","last_unstale":"2024-09-16T00:11:13.136851+0000","last_undegraded":"2024-09-16T00:11:13.136851+0000","last_fullsized":"2024-09-16T00:11:13.136851+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:06:48.016972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":8303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"123'452","reported_seq":608,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011438+0000","last_change":"2024-09-16T00:10:00.699942+0000","last_active":"2024-09-16T00:11:14.011438+0000","last_peered":"2024-09-16T00:11:14.011438+0000","last_clean":"2024-09-16T00:11:14.011438+0000","last_became_active":"2024-09-16T00:10:00.281972+0000","last_became_peered":"2024-09-16T00:10:00.281972+0000","last_unstale":"2024-09-16T00:11:14.011438+0000","last_undegraded":"2024-09-16T00:11:14.011438+0000","last_fullsized":"2024-09-16T00:11:14.011438+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:21.943709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":43345,"num_write":425,"num_write_kb":121191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":29546741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"123'396","reported_seq":725,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952254+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:11:13.952254+0000","last_peered":"2024-09-16T00:11:13.952254+0000","last_clean":"2024-09-16T00:11:13.952254+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:11:13.952254+0000","last_undegraded":"2024-09-16T00:11:13.952254+0000","last_fullsized":"2024-09-16T00:11:13.952254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":32882,"num_write":368,"num_write_kb":108251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"123'219","reported_seq":436,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136812+0000","last_change":"2024-09-16T00:10:55.835894+0000","last_active":"2024-09-16T00:11:13.136812+0000","last_peered":"2024-09-16T00:11:13.136812+0000","last_clean":"2024-09-16T00:11:13.136812+0000","last_became_active":"2024-09-16T00:10:55.835301+0000","last_became_peered":"2024-09-16T00:10:55.835301+0000","last_unstale":"2024-09-16T00:11:13.136812+0000","last_undegraded":"2024-09-16T00:11:13.136812+0000","last_fullsized":"2024-09-16T00:11:13.136812+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:32.623825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":4194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"123'385","reported_seq":714,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951587+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:11:13.951587+0000","last_peered":"2024-09-16T00:11:13.951587+0000","last_clean":"2024-09-16T00:11:13.951587+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:11:13.951587+0000","last_undegraded":"2024-09-16T00:11:13.951587+0000","last_fullsized":"2024-09-16T00:11:13.951587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":36278,"num_write":367,"num_write_kb":113361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"123'263","reported_seq":492,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136841+0000","last_change":"2024-09-16T00:10:55.856431+0000","last_active":"2024-09-16T00:11:13.136841+0000","last_peered":"2024-09-16T00:11:13.136841+0000","last_clean":"2024-09-16T00:11:13.136841+0000","last_became_active":"2024-09-16T00:10:55.843843+0000","last_became_peered":"2024-09-16T00:10:55.843843+0000","last_unstale":"2024-09-16T00:11:13.136841+0000","last_undegraded":"2024-09-16T00:11:13.136841+0000","last_fullsized":"2024-09-16T00:11:13.136841+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:11.989047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":197,"num_write_kb":12428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":30,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"123'348","reported_seq":601,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952826+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:11:13.952826+0000","last_peered":"2024-09-16T00:11:13.952826+0000","last_clean":"2024-09-16T00:11:13.952826+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:11:13.952826+0000","last_undegraded":"2024-09-16T00:11:13.952826+0000","last_fullsized":"2024-09-16T00:11:13.952826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":12352,"num_write":323,"num_write_kb":38097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"122'93","reported_seq":354,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544743+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:11:13.544743+0000","last_peered":"2024-09-16T00:11:13.544743+0000","last_clean":"2024-09-16T00:11:13.544743+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:11:13.544743+0000","last_undegraded":"2024-09-16T00:11:13.544743+0000","last_fullsized":"2024-09-16T00:11:13.544743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":4141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"123'414","reported_seq":793,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.544646+0000","last_change":"2024-09-16T00:10:00.556848+0000","last_active":"2024-09-16T00:11:13.544646+0000","last_peered":"2024-09-16T00:11:13.544646+0000","last_clean":"2024-09-16T00:11:13.544646+0000","last_became_active":"2024-09-16T00:10:00.283734+0000","last_became_peered":"2024-09-16T00:10:00.283734+0000","last_unstale":"2024-09-16T00:11:13.544646+0000","last_undegraded":"2024-09-16T00:11:13.544646+0000","last_fullsized":"2024-09-16T00:11:13.544646+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:55:01.843183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":40163,"num_write":392,"num_write_kb":81063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":21131701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"123'98","reported_seq":372,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381221+0000","last_change":"2024-09-16T00:10:55.856802+0000","last_active":"2024-09-16T00:11:14.381221+0000","last_peered":"2024-09-16T00:11:14.381221+0000","last_clean":"2024-09-16T00:11:14.381221+0000","last_became_active":"2024-09-16T00:10:55.844922+0000","last_became_peered":"2024-09-16T00:10:55.844922+0000","last_unstale":"2024-09-16T00:11:14.381221+0000","last_undegraded":"2024-09-16T00:11:14.381221+0000","last_fullsized":"2024-09-16T00:11:14.381221+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:19.134467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":4138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2561,"num_read_kb":1362012,"num_write":24417,"num_write_kb":4088607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3177,"num_bytes_recovered":294487447,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5234,"internal_metadata":0},"log_size":25917,"ondisk_log_size":25917,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":129048453,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":176,"num_read_kb":1534,"num_write":5618,"num_write_kb":131059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":26562382,"num_keys_recovered":3117,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86642688,"data_stored":258096906,"data_compressed":38739888,"data_compressed_allocated":49811456,"data_compressed_original":221323264,"omap_allocated":262411,"internal_metadata":0},"log_size":7144,"ondisk_log_size":7144,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":127,"seq":545460846598,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57992,"kb_used_data":16044,"kb_used_omap":42,"kb_used_meta":41877,"kb_avail":94313848,"statfs":{"total":96636764160,"available":96577380352,"internally_reserved":0,"allocated":16429056,"data_stored":51762927,"data_compressed":7683094,"data_compressed_allocated":9629696,"data_compressed_original":45862912,"omap_allocated":43696,"internal_metadata":42882384},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":94,"seq":403726925841,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56032,"kb_used_data":14280,"kb_used_omap":221,"kb_used_meta":41506,"kb_avail":94315808,"statfs":{"total":96636764160,"available":96579387392,"internally_reserved":0,"allocated":14622720,"data_stored":44546809,"data_compressed":6872875,"data_compressed_allocated":8704000,"data_compressed_original":39534592,"omap_allocated":227133,"internal_metadata":42502339},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509490,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73848,"kb_used_data":12128,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94297992,"statfs":{"total":96636764160,"available":96561143808,"internally_reserved":0,"allocated":12419072,"data_stored":26012395,"data_compressed":3584779,"data_compressed_allocated":4849664,"data_compressed_original":19345408,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509491,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63496,"kb_used_data":18412,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94308344,"statfs":{"total":96636764160,"available":96571744256,"internally_reserved":0,"allocated":18853888,"data_stored":46983951,"data_compressed":6735741,"data_compressed_allocated":8769536,"data_compressed_original":37810176,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51040,"kb_used_data":11528,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320800,"statfs":{"total":96636764160,"available":96584499200,"internally_reserved":0,"allocated":11804672,"data_stored":33418992,"data_compressed":5001493,"data_compressed_allocated":6361088,"data_compressed_original":28876800,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542194,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67128,"kb_used_data":9564,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94304712,"statfs":{"total":96636764160,"available":96568025088,"internally_reserved":0,"allocated":9793536,"data_stored":26006313,"data_compressed":3864818,"data_compressed_allocated":4833280,"data_compressed_original":21946368,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542194,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57372,"kb_used_data":12292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94314468,"statfs":{"total":96636764160,"available":96578015232,"internally_reserved":0,"allocated":12587008,"data_stored":30797615,"data_compressed":4284061,"data_compressed_allocated":5672960,"data_compressed_original":24797184,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35280,"kb_used_data":4088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336560,"statfs":{"total":96636764160,"available":96600637440,"internally_reserved":0,"allocated":4186112,"data_stored":6502032,"data_compressed":777781,"data_compressed_allocated":1081344,"data_compressed_original":4296704,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10776576,"data_stored":29360766,"data_compressed":4251684,"data_compressed_allocated":5627904,"data_compressed_original":24223744,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8060928,"data_stored":25165984,"data_compressed":3864818,"data_compressed_allocated":4833280,"data_compressed_original":21946368,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2441216,"data_stored":5655551,"data_compressed":777781,"data_compressed_allocated":1081344,"data_compressed_original":4296704,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10072064,"data_stored":32578663,"data_compressed":5001493,"data_compressed_allocated":6361088,"data_compressed_original":28876800,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17108992,"data_stored":46137470,"data_compressed":6735741,"data_compressed_allocated":8769536,"data_compressed_original":37810176,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10674176,"data_stored":25165914,"data_compressed":3584779,"data_compressed_allocated":4849664,"data_compressed_original":19345408,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12877824,"data_stored":43700328,"data_compressed":6872875,"data_compressed_allocated":8704000,"data_compressed_original":39534592,"omap_allocated":225539,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14630912,"data_stored":50332230,"data_compressed":7650717,"data_compressed_allocated":9584640,"data_compressed_original":45289472,"omap_allocated":36872,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5234,"internal_metadata":0}]}} 2024-09-16T00:11:17.832 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-16T00:11:17.833 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-16T00:11:17.971 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.967+0000 7f46f8dbd640 1 Processor -- start 2024-09-16T00:11:17.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.967+0000 7f46f8dbd640 1 -- start start 2024-09-16T00:11:17.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.967+0000 7f46f8dbd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f46f4154c20 0x7f46f415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.967+0000 7f46f8dbd640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f46f40ccc90 0x7f46f40cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.967+0000 7f46f8dbd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f40cc4b0 con 0x7f46f4068060 2024-09-16T00:11:17.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.967+0000 7f46f8dbd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f46f40c5830 con 0x7f46f4154c20 2024-09-16T00:11:17.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.967+0000 7f46f8dbd640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f46f415b7c0 con 0x7f46f40ccc90 2024-09-16T00:11:17.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.968+0000 7f46f259c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f46f4154c20 0x7f46f415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.968+0000 7f46ebfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f46f4068060 0x7f46f40c5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45688/0 (socket says 172.21.15.139:45688) 2024-09-16T00:11:17.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.968+0000 7f46ebfff640 1 -- 172.21.15.139:0/2099460764 learned_addr learned my addr 172.21.15.139:0/2099460764 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:17.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.968+0000 7f46f2d9d640 1 --2- 172.21.15.139:0/2099460764 >> v2:172.21.15.164:3300/0 conn(0x7f46f40ccc90 0x7f46f40cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.972 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.968+0000 7f46f259c640 1 -- 172.21.15.139:0/2099460764 >> v2:172.21.15.164:3300/0 conn(0x7f46f40ccc90 msgr2=0x7f46f40cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:17.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.968+0000 7f46f259c640 1 --2- 172.21.15.139:0/2099460764 >> v2:172.21.15.164:3300/0 conn(0x7f46f40ccc90 0x7f46f40cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.968+0000 7f46f259c640 1 -- 172.21.15.139:0/2099460764 >> v1:172.21.15.139:6789/0 conn(0x7f46f4068060 legacy=0x7f46f40c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.968+0000 7f46f259c640 1 -- 172.21.15.139:0/2099460764 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f40cc790 con 0x7f46f4154c20 2024-09-16T00:11:17.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.969+0000 7f46f259c640 1 --2- 172.21.15.139:0/2099460764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46f4154c20 0x7f46f415b280 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f46f40c7300 tx=0x7f46e402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5677433c092267 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:17.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.969+0000 7f46f1d9b640 1 -- 172.21.15.139:0/2099460764 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4037070 con 0x7f46f4154c20 2024-09-16T00:11:17.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.969+0000 7f46f1d9b640 1 -- 172.21.15.139:0/2099460764 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e4032440 con 0x7f46f4154c20 2024-09-16T00:11:17.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.969+0000 7f46f1d9b640 1 -- 172.21.15.139:0/2099460764 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4032810 con 0x7f46f4154c20 2024-09-16T00:11:17.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.969+0000 7f46f8dbd640 1 -- 172.21.15.139:0/2099460764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46f4154c20 msgr2=0x7f46f415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.969+0000 7f46f8dbd640 1 --2- 172.21.15.139:0/2099460764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46f4154c20 0x7f46f415b280 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f46f40c7300 tx=0x7f46e402a1a0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.969+0000 7f46f8dbd640 1 -- 172.21.15.139:0/2099460764 shutdown_connections 2024-09-16T00:11:17.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.969+0000 7f46f8dbd640 1 --2- 172.21.15.139:0/2099460764 >> v2:172.21.15.164:3300/0 conn(0x7f46f40ccc90 0x7f46f40cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.969+0000 7f46f8dbd640 1 --2- 172.21.15.139:0/2099460764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46f4154c20 0x7f46f415b280 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.973 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.969+0000 7f46f8dbd640 1 -- 172.21.15.139:0/2099460764 >> 172.21.15.139:0/2099460764 conn(0x7f46f406ca40 msgr2=0x7f46f4067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:17.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.969+0000 7f46f8dbd640 1 -- 172.21.15.139:0/2099460764 shutdown_connections 2024-09-16T00:11:17.974 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.969+0000 7f46f8dbd640 1 -- 172.21.15.139:0/2099460764 wait complete. 2024-09-16T00:11:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.971+0000 7f46f8dbd640 1 Processor -- start 2024-09-16T00:11:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.971+0000 7f46f8dbd640 1 -- start start 2024-09-16T00:11:17.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.971+0000 7f46f8dbd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46f4068060 0x7f46f4149a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.971+0000 7f46f8dbd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46f40ccc90 0x7f46f414a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.971+0000 7f46f8dbd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46f4154c20 0x7f46f4152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.971+0000 7f46f8dbd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f46f415e0d0 con 0x7f46f4154c20 2024-09-16T00:11:17.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.971+0000 7f46f8dbd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f46f40cc260 con 0x7f46f40ccc90 2024-09-16T00:11:17.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.971+0000 7f46f8dbd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f46f415e3b0 con 0x7f46f4068060 2024-09-16T00:11:17.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.972+0000 7f46f2d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46f4154c20 0x7f46f4152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.972+0000 7f46f2d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46f4154c20 0x7f46f4152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43334/0 (socket says 172.21.15.139:43334) 2024-09-16T00:11:17.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.972+0000 7f46f2d9d640 1 -- 172.21.15.139:0/1088430565 learned_addr learned my addr 172.21.15.139:0/1088430565 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:17.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.972+0000 7f46f259c640 1 --2- 172.21.15.139:0/1088430565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46f4068060 0x7f46f4149a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.972+0000 7f46ebfff640 1 --2- 172.21.15.139:0/1088430565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46f40ccc90 0x7f46f414a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.972+0000 7f46f2d9d640 1 -- 172.21.15.139:0/1088430565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46f4068060 msgr2=0x7f46f4149a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.972+0000 7f46f2d9d640 1 --2- 172.21.15.139:0/1088430565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46f4068060 0x7f46f4149a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.972+0000 7f46f2d9d640 1 -- 172.21.15.139:0/1088430565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46f40ccc90 msgr2=0x7f46f414a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.972+0000 7f46f2d9d640 1 --2- 172.21.15.139:0/1088430565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46f40ccc90 0x7f46f414a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.972+0000 7f46f2d9d640 1 -- 172.21.15.139:0/1088430565 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f415bd70 con 0x7f46f4154c20 2024-09-16T00:11:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.972+0000 7f46f2d9d640 1 --2- 172.21.15.139:0/1088430565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46f4154c20 0x7f46f4152540 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009d20 tx=0x7f46ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.972+0000 7f46eaffd640 1 -- 172.21.15.139:0/1088430565 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec018020 con 0x7f46f4154c20 2024-09-16T00:11:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.972+0000 7f46eaffd640 1 -- 172.21.15.139:0/1088430565 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec009b10 con 0x7f46f4154c20 2024-09-16T00:11:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.972+0000 7f46eaffd640 1 -- 172.21.15.139:0/1088430565 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0146c0 con 0x7f46f4154c20 2024-09-16T00:11:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.972+0000 7f46f8dbd640 1 -- 172.21.15.139:0/1088430565 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f415c050 con 0x7f46f4154c20 2024-09-16T00:11:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.972+0000 7f46f8dbd640 1 -- 172.21.15.139:0/1088430565 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f415c610 con 0x7f46f4154c20 2024-09-16T00:11:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.973+0000 7f46f8dbd640 1 -- 172.21.15.139:0/1088430565 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f415e690 con 0x7f46f4154c20 2024-09-16T00:11:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.974+0000 7f46eaffd640 1 -- 172.21.15.139:0/1088430565 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f46ec01b020 con 0x7f46f4154c20 2024-09-16T00:11:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.974+0000 7f46eaffd640 1 --2- 172.21.15.139:0/1088430565 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f46d406c2e0 0x7f46d406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.974+0000 7f46eaffd640 1 -- 172.21.15.139:0/1088430565 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f46ec08bfc0 con 0x7f46f4154c20 2024-09-16T00:11:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.974+0000 7f46f259c640 1 --2- 172.21.15.139:0/1088430565 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f46d406c2e0 0x7f46d406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:17.978 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.974+0000 7f46f259c640 1 --2- 172.21.15.139:0/1088430565 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f46d406c2e0 0x7f46d406e7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f46e4034000 tx=0x7f46e402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:17.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:17.978+0000 7f46eaffd640 1 -- 172.21.15.139:0/1088430565 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f46ec022030 con 0x7f46f4154c20 2024-09-16T00:11:18.147 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.145+0000 7f46f8dbd640 1 -- 172.21.15.139:0/1088430565 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} v 0) -- 0x7f46f40cca70 con 0x7f46f4154c20 2024-09-16T00:11:18.150 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.148+0000 7f46eaffd640 1 -- 172.21.15.139:0/1088430565 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]=0 v11) ==== 118+0+0 (secure 0 0 0) 0x7f46ec010040 con 0x7f46f4154c20 2024-09-16T00:11:18.152 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.150+0000 7f46f8dbd640 1 -- 172.21.15.139:0/1088430565 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f46d406c2e0 msgr2=0x7f46d406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:18.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.150+0000 7f46f8dbd640 1 --2- 172.21.15.139:0/1088430565 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f46d406c2e0 0x7f46d406e7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f46e4034000 tx=0x7f46e402e000 comp rx=0 tx=0).stop 2024-09-16T00:11:18.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.150+0000 7f46f8dbd640 1 -- 172.21.15.139:0/1088430565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46f4154c20 msgr2=0x7f46f4152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:18.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.150+0000 7f46f8dbd640 1 --2- 172.21.15.139:0/1088430565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46f4154c20 0x7f46f4152540 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009d20 tx=0x7f46ec00c490 comp rx=0 tx=0).stop 2024-09-16T00:11:18.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.150+0000 7f46f8dbd640 1 -- 172.21.15.139:0/1088430565 shutdown_connections 2024-09-16T00:11:18.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.150+0000 7f46f8dbd640 1 --2- 172.21.15.139:0/1088430565 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f46d406c2e0 0x7f46d406e7a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.150+0000 7f46f8dbd640 1 --2- 172.21.15.139:0/1088430565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46f4154c20 0x7f46f4152540 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.150+0000 7f46f8dbd640 1 --2- 172.21.15.139:0/1088430565 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f46f40ccc90 0x7f46f414a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.150+0000 7f46f8dbd640 1 --2- 172.21.15.139:0/1088430565 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f46f4068060 0x7f46f4149a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.150+0000 7f46f8dbd640 1 -- 172.21.15.139:0/1088430565 >> 172.21.15.139:0/1088430565 conn(0x7f46f406ca40 msgr2=0x7f46f41596d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:18.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.150+0000 7f46f8dbd640 1 -- 172.21.15.139:0/1088430565 shutdown_connections 2024-09-16T00:11:18.153 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.150+0000 7f46f8dbd640 1 -- 172.21.15.139:0/1088430565 wait complete. 2024-09-16T00:11:18.164 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-16T00:11:18.177 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-16T00:11:18.177 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:11:18.177 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:11:18.193 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:11:18.193 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:11:18.250 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T00:11:18.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.387+0000 7f83f3053640 1 Processor -- start 2024-09-16T00:11:18.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.387+0000 7f83f3053640 1 -- start start 2024-09-16T00:11:18.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.387+0000 7f83f3053640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f83ec15b000 0x7f83ec15b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:18.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.387+0000 7f83f3053640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83ec073740 0x7f83ec073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:18.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.387+0000 7f83f3053640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83ec0c7250 con 0x7f83ec154bf0 2024-09-16T00:11:18.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.387+0000 7f83f3053640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f83ec15ab70 con 0x7f83ec15b000 2024-09-16T00:11:18.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.387+0000 7f83f3053640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f83ec154fd0 con 0x7f83ec073740 2024-09-16T00:11:18.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.388+0000 7f83ebfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f83ec15b000 0x7f83ec15b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:18.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.388+0000 7f83f0def640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f83ec154bf0 0x7f83ec072e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45690/0 (socket says 172.21.15.139:45690) 2024-09-16T00:11:18.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.388+0000 7f83f0def640 1 -- 172.21.15.139:0/330686248 learned_addr learned my addr 172.21.15.139:0/330686248 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:18.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.388+0000 7f83f15f0640 1 --2- 172.21.15.139:0/330686248 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83ec073740 0x7f83ec073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:18.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.388+0000 7f83ebfff640 1 -- 172.21.15.139:0/330686248 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83ec073740 msgr2=0x7f83ec073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:18.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.388+0000 7f83ebfff640 1 --2- 172.21.15.139:0/330686248 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83ec073740 0x7f83ec073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.388+0000 7f83ebfff640 1 -- 172.21.15.139:0/330686248 >> v1:172.21.15.139:6789/0 conn(0x7f83ec154bf0 legacy=0x7f83ec072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:18.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.388+0000 7f83ebfff640 1 -- 172.21.15.139:0/330686248 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ec06c530 con 0x7f83ec15b000 2024-09-16T00:11:18.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.388+0000 7f83eb7fe640 1 -- 172.21.15.139:0/330686248 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3083180184 0 0) 0x7f83e0001110 con 0x7f83ec154bf0 2024-09-16T00:11:18.392 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.389+0000 7f83ebfff640 1 --2- 172.21.15.139:0/330686248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83ec15b000 0x7f83ec15b420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009790 tx=0x7f83d802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8320cb102f6634ac server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:18.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.389+0000 7f83eb7fe640 1 -- 172.21.15.139:0/330686248 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8037070 con 0x7f83ec15b000 2024-09-16T00:11:18.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.389+0000 7f83eb7fe640 1 -- 172.21.15.139:0/330686248 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83d8032440 con 0x7f83ec15b000 2024-09-16T00:11:18.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.389+0000 7f83eb7fe640 1 -- 172.21.15.139:0/330686248 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8032810 con 0x7f83ec15b000 2024-09-16T00:11:18.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.389+0000 7f83f3053640 1 -- 172.21.15.139:0/330686248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83ec15b000 msgr2=0x7f83ec15b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:18.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.389+0000 7f83f3053640 1 --2- 172.21.15.139:0/330686248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83ec15b000 0x7f83ec15b420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009790 tx=0x7f83d802a210 comp rx=0 tx=0).stop 2024-09-16T00:11:18.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.389+0000 7f83f3053640 1 -- 172.21.15.139:0/330686248 shutdown_connections 2024-09-16T00:11:18.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.389+0000 7f83f3053640 1 --2- 172.21.15.139:0/330686248 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83ec073740 0x7f83ec073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.389+0000 7f83f3053640 1 --2- 172.21.15.139:0/330686248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83ec15b000 0x7f83ec15b420 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.389+0000 7f83f3053640 1 -- 172.21.15.139:0/330686248 >> 172.21.15.139:0/330686248 conn(0x7f83ec0c9350 msgr2=0x7f83ec0c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:18.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.389+0000 7f83f3053640 1 -- 172.21.15.139:0/330686248 shutdown_connections 2024-09-16T00:11:18.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.389+0000 7f83f3053640 1 -- 172.21.15.139:0/330686248 wait complete. 2024-09-16T00:11:18.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.390+0000 7f83f3053640 1 Processor -- start 2024-09-16T00:11:18.393 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.391+0000 7f83f3053640 1 -- start start 2024-09-16T00:11:18.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.391+0000 7f83f3053640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83ec073740 0x7f83ec152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:18.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.391+0000 7f83f3053640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83ec154bf0 0x7f83ec1e3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:18.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.391+0000 7f83f3053640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83ec15b000 0x7f83ec1e4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:18.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.391+0000 7f83f3053640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec073560 con 0x7f83ec15b000 2024-09-16T00:11:18.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.391+0000 7f83f3053640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec06db30 con 0x7f83ec154bf0 2024-09-16T00:11:18.396 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.391+0000 7f83f3053640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f83ec06c7f0 con 0x7f83ec073740 2024-09-16T00:11:18.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.391+0000 7f83f15f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83ec15b000 0x7f83ec1e4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:18.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.392+0000 7f83f15f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83ec15b000 0x7f83ec1e4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43352/0 (socket says 172.21.15.139:43352) 2024-09-16T00:11:18.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.392+0000 7f83f15f0640 1 -- 172.21.15.139:0/3123076919 learned_addr learned my addr 172.21.15.139:0/3123076919 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:18.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.392+0000 7f83f0def640 1 --2- 172.21.15.139:0/3123076919 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83ec073740 0x7f83ec152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:18.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.392+0000 7f83ebfff640 1 --2- 172.21.15.139:0/3123076919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83ec154bf0 0x7f83ec1e3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:18.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.392+0000 7f83f15f0640 1 -- 172.21.15.139:0/3123076919 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83ec073740 msgr2=0x7f83ec152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:18.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.392+0000 7f83f15f0640 1 --2- 172.21.15.139:0/3123076919 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83ec073740 0x7f83ec152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.392+0000 7f83f15f0640 1 -- 172.21.15.139:0/3123076919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83ec154bf0 msgr2=0x7f83ec1e3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:18.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.392+0000 7f83f15f0640 1 --2- 172.21.15.139:0/3123076919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83ec154bf0 0x7f83ec1e3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.392+0000 7f83f15f0640 1 -- 172.21.15.139:0/3123076919 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ec0c6820 con 0x7f83ec15b000 2024-09-16T00:11:18.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.392+0000 7f83f0def640 1 --2- 172.21.15.139:0/3123076919 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83ec073740 0x7f83ec152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:18.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.392+0000 7f83f15f0640 1 --2- 172.21.15.139:0/3123076919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83ec15b000 0x7f83ec1e4050 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f83e4007e40 tx=0x7f83e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:18.397 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.392+0000 7f83e97fa640 1 -- 172.21.15.139:0/3123076919 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4014620 con 0x7f83ec15b000 2024-09-16T00:11:18.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.392+0000 7f83e97fa640 1 -- 172.21.15.139:0/3123076919 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e4019070 con 0x7f83ec15b000 2024-09-16T00:11:18.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.392+0000 7f83e97fa640 1 -- 172.21.15.139:0/3123076919 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4014a20 con 0x7f83ec15b000 2024-09-16T00:11:18.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.392+0000 7f83f3053640 1 -- 172.21.15.139:0/3123076919 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ec0c6ac0 con 0x7f83ec15b000 2024-09-16T00:11:18.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.392+0000 7f83f3053640 1 -- 172.21.15.139:0/3123076919 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ec1f3b60 con 0x7f83ec15b000 2024-09-16T00:11:18.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.393+0000 7f83f3053640 1 -- 172.21.15.139:0/3123076919 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83ec1e7fc0 con 0x7f83ec15b000 2024-09-16T00:11:18.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.393+0000 7f83e97fa640 1 -- 172.21.15.139:0/3123076919 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f83e4014bc0 con 0x7f83ec15b000 2024-09-16T00:11:18.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.394+0000 7f83e97fa640 1 --2- 172.21.15.139:0/3123076919 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f83d006c2e0 0x7f83d006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:18.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.394+0000 7f83e97fa640 1 -- 172.21.15.139:0/3123076919 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f83e408be70 con 0x7f83ec15b000 2024-09-16T00:11:18.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.394+0000 7f83f0def640 1 --2- 172.21.15.139:0/3123076919 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f83d006c2e0 0x7f83d006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:18.398 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.394+0000 7f83f0def640 1 --2- 172.21.15.139:0/3123076919 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f83d006c2e0 0x7f83d006e7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f83e0000f60 tx=0x7f83e0007a10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:18.400 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.398+0000 7f83e97fa640 1 -- 172.21.15.139:0/3123076919 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f83ec1e7fc0 con 0x7f83ec15b000 2024-09-16T00:11:18.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.572+0000 7f83f3053640 1 -- 172.21.15.139:0/3123076919 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f83ec06cad0 con 0x7f83ec15b000 2024-09-16T00:11:18.574 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.572+0000 7f83e97fa640 1 -- 172.21.15.139:0/3123076919 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v146) ==== 80+0+0 (secure 0 0 0) 0x7f83e405a460 con 0x7f83ec15b000 2024-09-16T00:11:18.574 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-09-16T00:11:18.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.574+0000 7f83f3053640 1 -- 172.21.15.139:0/3123076919 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f83d006c2e0 msgr2=0x7f83d006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:18.576 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.574+0000 7f83f3053640 1 --2- 172.21.15.139:0/3123076919 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f83d006c2e0 0x7f83d006e7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f83e0000f60 tx=0x7f83e0007a10 comp rx=0 tx=0).stop 2024-09-16T00:11:18.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.574+0000 7f83f3053640 1 -- 172.21.15.139:0/3123076919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83ec15b000 msgr2=0x7f83ec1e4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:18.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.574+0000 7f83f3053640 1 --2- 172.21.15.139:0/3123076919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83ec15b000 0x7f83ec1e4050 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f83e4007e40 tx=0x7f83e400e4b0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.574+0000 7f83f3053640 1 -- 172.21.15.139:0/3123076919 shutdown_connections 2024-09-16T00:11:18.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.574+0000 7f83f3053640 1 --2- 172.21.15.139:0/3123076919 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f83d006c2e0 0x7f83d006e7a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.574+0000 7f83f3053640 1 --2- 172.21.15.139:0/3123076919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f83ec15b000 0x7f83ec1e4050 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.574+0000 7f83f3053640 1 --2- 172.21.15.139:0/3123076919 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f83ec154bf0 0x7f83ec1e3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.574+0000 7f83f3053640 1 --2- 172.21.15.139:0/3123076919 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f83ec073740 0x7f83ec152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.574+0000 7f83f3053640 1 -- 172.21.15.139:0/3123076919 >> 172.21.15.139:0/3123076919 conn(0x7f83ec0c9350 msgr2=0x7f83ec072e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:18.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.574+0000 7f83f3053640 1 -- 172.21.15.139:0/3123076919 shutdown_connections 2024-09-16T00:11:18.577 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.574+0000 7f83f3053640 1 -- 172.21.15.139:0/3123076919 wait complete. 2024-09-16T00:11:18.587 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-16T00:11:18.587 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:11:18.603 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-16T00:11:18.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.741+0000 7fbbc49b7640 1 Processor -- start 2024-09-16T00:11:18.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.741+0000 7fbbc49b7640 1 -- start start 2024-09-16T00:11:18.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.742+0000 7fbbc49b7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc0068060 0x7fbbc015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:18.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.742+0000 7fbbc49b7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbbc00ccc90 0x7fbbc00cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:18.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.742+0000 7fbbc49b7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbc00cc4b0 con 0x7fbbc0154c20 2024-09-16T00:11:18.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.742+0000 7fbbc49b7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbbc00c5830 con 0x7fbbc00ccc90 2024-09-16T00:11:18.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.742+0000 7fbbc49b7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbbc015b7c0 con 0x7fbbc0068060 2024-09-16T00:11:18.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.742+0000 7fbbbed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbbc00ccc90 0x7fbbc00cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:18.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.742+0000 7fbbbe59c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fbbc0154c20 0x7fbbc00c5120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45694/0 (socket says 172.21.15.139:45694) 2024-09-16T00:11:18.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.742+0000 7fbbbe59c640 1 -- 172.21.15.139:0/1051640318 learned_addr learned my addr 172.21.15.139:0/1051640318 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:18.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.743+0000 7fbbbdd9b640 1 --2- 172.21.15.139:0/1051640318 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc0068060 0x7fbbc015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:18.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.743+0000 7fbbbed9d640 1 -- 172.21.15.139:0/1051640318 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc0068060 msgr2=0x7fbbc015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:18.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.743+0000 7fbbbed9d640 1 --2- 172.21.15.139:0/1051640318 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc0068060 0x7fbbc015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.743+0000 7fbbbed9d640 1 -- 172.21.15.139:0/1051640318 >> v1:172.21.15.139:6789/0 conn(0x7fbbc0154c20 legacy=0x7fbbc00c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:18.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.743+0000 7fbbbd59a640 1 -- 172.21.15.139:0/1051640318 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2110728188 0 0) 0x7fbbc00cc4b0 con 0x7fbbc0154c20 2024-09-16T00:11:18.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.743+0000 7fbbbed9d640 1 -- 172.21.15.139:0/1051640318 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc00cc790 con 0x7fbbc00ccc90 2024-09-16T00:11:18.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.743+0000 7fbbbdd9b640 1 --2- 172.21.15.139:0/1051640318 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc0068060 0x7fbbc015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:18.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.743+0000 7fbbbed9d640 1 --2- 172.21.15.139:0/1051640318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbbc00ccc90 0x7fbbc00cd0c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80097a0 tx=0x7fbbb802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc91bd3f162f937 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:18.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.743+0000 7fbbbd59a640 1 -- 172.21.15.139:0/1051640318 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8037070 con 0x7fbbc00ccc90 2024-09-16T00:11:18.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.743+0000 7fbbbd59a640 1 -- 172.21.15.139:0/1051640318 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbb8032440 con 0x7fbbc00ccc90 2024-09-16T00:11:18.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.744+0000 7fbbbd59a640 1 -- 172.21.15.139:0/1051640318 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8032810 con 0x7fbbc00ccc90 2024-09-16T00:11:18.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.744+0000 7fbbc49b7640 1 -- 172.21.15.139:0/1051640318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbbc00ccc90 msgr2=0x7fbbc00cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:18.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.744+0000 7fbbc49b7640 1 --2- 172.21.15.139:0/1051640318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbbc00ccc90 0x7fbbc00cd0c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80097a0 tx=0x7fbbb802a210 comp rx=0 tx=0).stop 2024-09-16T00:11:18.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.744+0000 7fbbc49b7640 1 -- 172.21.15.139:0/1051640318 shutdown_connections 2024-09-16T00:11:18.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.744+0000 7fbbc49b7640 1 --2- 172.21.15.139:0/1051640318 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbbc00ccc90 0x7fbbc00cd0c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.744+0000 7fbbc49b7640 1 --2- 172.21.15.139:0/1051640318 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc0068060 0x7fbbc015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.744+0000 7fbbc49b7640 1 -- 172.21.15.139:0/1051640318 >> 172.21.15.139:0/1051640318 conn(0x7fbbc006ca40 msgr2=0x7fbbc0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:18.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.744+0000 7fbbc49b7640 1 -- 172.21.15.139:0/1051640318 shutdown_connections 2024-09-16T00:11:18.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.744+0000 7fbbc49b7640 1 -- 172.21.15.139:0/1051640318 wait complete. 2024-09-16T00:11:18.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.745+0000 7fbbc49b7640 1 Processor -- start 2024-09-16T00:11:18.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.745+0000 7fbbc49b7640 1 -- start start 2024-09-16T00:11:18.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.746+0000 7fbbc49b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbbc0068060 0x7fbbc01f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:18.748 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.746+0000 7fbbc49b7640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbbc00ccc90 0x7fbbc00c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:18.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.746+0000 7fbbc49b7640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc0154c20 0x7fbbc00c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:18.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.746+0000 7fbbc49b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc015e0d0 con 0x7fbbc0068060 2024-09-16T00:11:18.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.746+0000 7fbbc49b7640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc00cc260 con 0x7fbbc00ccc90 2024-09-16T00:11:18.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.746+0000 7fbbc49b7640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbbc015e3b0 con 0x7fbbc0154c20 2024-09-16T00:11:18.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.746+0000 7fbbbe59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbbc0068060 0x7fbbc01f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:18.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.746+0000 7fbbbe59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbbc0068060 0x7fbbc01f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43378/0 (socket says 172.21.15.139:43378) 2024-09-16T00:11:18.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.746+0000 7fbbbe59c640 1 -- 172.21.15.139:0/3826913082 learned_addr learned my addr 172.21.15.139:0/3826913082 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:18.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.746+0000 7fbbbed9d640 1 --2- 172.21.15.139:0/3826913082 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc0154c20 0x7fbbc00c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:18.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.746+0000 7fbbbdd9b640 1 --2- 172.21.15.139:0/3826913082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbbc00ccc90 0x7fbbc00c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:18.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.746+0000 7fbbbe59c640 1 -- 172.21.15.139:0/3826913082 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc0154c20 msgr2=0x7fbbc00c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:18.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.746+0000 7fbbbe59c640 1 --2- 172.21.15.139:0/3826913082 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc0154c20 0x7fbbc00c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.746+0000 7fbbbe59c640 1 -- 172.21.15.139:0/3826913082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbbc00ccc90 msgr2=0x7fbbc00c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:18.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.746+0000 7fbbbe59c640 1 --2- 172.21.15.139:0/3826913082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbbc00ccc90 0x7fbbc00c61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.749 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.746+0000 7fbbbe59c640 1 -- 172.21.15.139:0/3826913082 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc01f0760 con 0x7fbbc0068060 2024-09-16T00:11:18.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.747+0000 7fbbbe59c640 1 --2- 172.21.15.139:0/3826913082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbbc0068060 0x7fbbc01f3d30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000c7f0 tx=0x7fbbb000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:18.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.747+0000 7fbbaeffd640 1 -- 172.21.15.139:0/3826913082 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0018070 con 0x7fbbc0068060 2024-09-16T00:11:18.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.747+0000 7fbbaeffd640 1 -- 172.21.15.139:0/3826913082 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbb000a640 con 0x7fbbc0068060 2024-09-16T00:11:18.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.747+0000 7fbbc49b7640 1 -- 172.21.15.139:0/3826913082 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc00c7940 con 0x7fbbc0068060 2024-09-16T00:11:18.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.747+0000 7fbbaeffd640 1 -- 172.21.15.139:0/3826913082 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb000aa30 con 0x7fbbc0068060 2024-09-16T00:11:18.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.747+0000 7fbbc49b7640 1 -- 172.21.15.139:0/3826913082 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc01f0d00 con 0x7fbbc0068060 2024-09-16T00:11:18.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.748+0000 7fbbc49b7640 1 -- 172.21.15.139:0/3826913082 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbc015e690 con 0x7fbbc0068060 2024-09-16T00:11:18.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.748+0000 7fbbaeffd640 1 -- 172.21.15.139:0/3826913082 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbbb001b070 con 0x7fbbc0068060 2024-09-16T00:11:18.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.749+0000 7fbbaeffd640 1 --2- 172.21.15.139:0/3826913082 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbba006c2b0 0x7fbba006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:18.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.749+0000 7fbbaeffd640 1 -- 172.21.15.139:0/3826913082 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7fbbb008ab80 con 0x7fbbc0068060 2024-09-16T00:11:18.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.749+0000 7fbbbdd9b640 1 --2- 172.21.15.139:0/3826913082 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbba006c2b0 0x7fbba006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:18.751 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.749+0000 7fbbbdd9b640 1 --2- 172.21.15.139:0/3826913082 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbba006c2b0 0x7fbba006e770 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4006fd0 tx=0x7fbbb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:18.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.753+0000 7fbbaeffd640 1 -- 172.21.15.139:0/3826913082 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbbc00c7940 con 0x7fbbc0068060 2024-09-16T00:11:18.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.929+0000 7fbbc49b7640 1 -- 172.21.15.139:0/3826913082 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbbc00b2b90 con 0x7fbbc0068060 2024-09-16T00:11:18.930 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.929+0000 7fbbaeffd640 1 -- 172.21.15.139:0/3826913082 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v146) ==== 80+0+0 (secure 0 0 0) 0x7fbbb0059170 con 0x7fbbc0068060 2024-09-16T00:11:18.930 INFO:teuthology.orchestra.run.smithi139.stderr:listed 0 entries 2024-09-16T00:11:18.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.932+0000 7fbbc49b7640 1 -- 172.21.15.139:0/3826913082 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbba006c2b0 msgr2=0x7fbba006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:18.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.932+0000 7fbbc49b7640 1 --2- 172.21.15.139:0/3826913082 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbba006c2b0 0x7fbba006e770 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4006fd0 tx=0x7fbbb4008040 comp rx=0 tx=0).stop 2024-09-16T00:11:18.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.932+0000 7fbbc49b7640 1 -- 172.21.15.139:0/3826913082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbbc0068060 msgr2=0x7fbbc01f3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:18.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.932+0000 7fbbc49b7640 1 --2- 172.21.15.139:0/3826913082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbbc0068060 0x7fbbc01f3d30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000c7f0 tx=0x7fbbb000ccb0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.932+0000 7fbbc49b7640 1 -- 172.21.15.139:0/3826913082 shutdown_connections 2024-09-16T00:11:18.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.932+0000 7fbbc49b7640 1 --2- 172.21.15.139:0/3826913082 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbba006c2b0 0x7fbba006e770 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.932+0000 7fbbc49b7640 1 --2- 172.21.15.139:0/3826913082 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbbc0154c20 0x7fbbc00c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.933 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.932+0000 7fbbc49b7640 1 --2- 172.21.15.139:0/3826913082 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbbc00ccc90 0x7fbbc00c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.932+0000 7fbbc49b7640 1 --2- 172.21.15.139:0/3826913082 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbbc0068060 0x7fbbc01f3d30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:18.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.932+0000 7fbbc49b7640 1 -- 172.21.15.139:0/3826913082 >> 172.21.15.139:0/3826913082 conn(0x7fbbc006ca40 msgr2=0x7fbbc014ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:18.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.932+0000 7fbbc49b7640 1 -- 172.21.15.139:0/3826913082 shutdown_connections 2024-09-16T00:11:18.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:18.932+0000 7fbbc49b7640 1 -- 172.21.15.139:0/3826913082 wait complete. 2024-09-16T00:11:18.944 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2024-09-16T00:11:18.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:18.945 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-16T00:11:18.970 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:18.971 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi139.stderr:ceph-fuse[43418]: fuse finished with error 0 and tester_r 0 2024-09-16T00:11:19.070 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T00:11:19.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:19.070 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T00:11:19.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:19.222 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-16T00:11:19.238 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T00:11:19.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:19.239 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:11:19.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:11:19.304 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link delete ceph-brx 2024-09-16T00:11:19.413 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T00:11:19.423 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T00:11:19.424 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:11:19.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.523+0000 7f457cbe8640 1 Processor -- start 2024-09-16T00:11:19.524 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.523+0000 7f457cbe8640 1 -- start start 2024-09-16T00:11:19.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.524+0000 7f457cbe8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f4578151230 0x7f45781585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:19.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.524+0000 7f457cbe8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f457806c170 0x7f4578158b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:19.525 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.524+0000 7f457cbe8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45780c5690 con 0x7f457806bd90 2024-09-16T00:11:19.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.524+0000 7f457cbe8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f45780c46b0 con 0x7f457806c170 2024-09-16T00:11:19.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.524+0000 7f457cbe8640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f45780c64f0 con 0x7f4578151230 2024-09-16T00:11:19.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.524+0000 7f4576d9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f457806c170 0x7f4578158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:19.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.524+0000 7f457659c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f457806bd90 0x7f4578150b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45702/0 (socket says 172.21.15.139:45702) 2024-09-16T00:11:19.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.524+0000 7f457659c640 1 -- 172.21.15.139:0/3414566047 learned_addr learned my addr 172.21.15.139:0/3414566047 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:19.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.524+0000 7f4575d9b640 1 --2- 172.21.15.139:0/3414566047 >> v2:172.21.15.164:3300/0 conn(0x7f4578151230 0x7f45781585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:19.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.524+0000 7f4576d9d640 1 -- 172.21.15.139:0/3414566047 >> v2:172.21.15.164:3300/0 conn(0x7f4578151230 msgr2=0x7f45781585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:19.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.524+0000 7f4576d9d640 1 --2- 172.21.15.139:0/3414566047 >> v2:172.21.15.164:3300/0 conn(0x7f4578151230 0x7f45781585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.524+0000 7f4576d9d640 1 -- 172.21.15.139:0/3414566047 >> v1:172.21.15.139:6789/0 conn(0x7f457806bd90 legacy=0x7f4578150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:19.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.525+0000 7f457559a640 1 -- 172.21.15.139:0/3414566047 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 990629394 0 0) 0x7f45780c5690 con 0x7f457806bd90 2024-09-16T00:11:19.526 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.525+0000 7f4576d9d640 1 -- 172.21.15.139:0/3414566047 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45780c5970 con 0x7f457806c170 2024-09-16T00:11:19.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.525+0000 7f4576d9d640 1 --2- 172.21.15.139:0/3414566047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f457806c170 0x7f4578158b00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f45780729e0 tx=0x7f456802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5215a44ecd82dee server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:19.527 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.525+0000 7f457559a640 1 -- 172.21.15.139:0/3414566047 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568037070 con 0x7f457806c170 2024-09-16T00:11:19.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.525+0000 7f457559a640 1 -- 172.21.15.139:0/3414566047 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4568032440 con 0x7f457806c170 2024-09-16T00:11:19.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.525+0000 7f457559a640 1 -- 172.21.15.139:0/3414566047 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568032810 con 0x7f457806c170 2024-09-16T00:11:19.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.526+0000 7f457cbe8640 1 -- 172.21.15.139:0/3414566047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f457806c170 msgr2=0x7f4578158b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:19.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.526+0000 7f457cbe8640 1 --2- 172.21.15.139:0/3414566047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f457806c170 0x7f4578158b00 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f45780729e0 tx=0x7f456802a1a0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.526+0000 7f457cbe8640 1 -- 172.21.15.139:0/3414566047 shutdown_connections 2024-09-16T00:11:19.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.526+0000 7f457cbe8640 1 --2- 172.21.15.139:0/3414566047 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f457806c170 0x7f4578158b00 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.526+0000 7f457cbe8640 1 --2- 172.21.15.139:0/3414566047 >> v2:172.21.15.164:3300/0 conn(0x7f4578151230 0x7f45781585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.526+0000 7f457cbe8640 1 -- 172.21.15.139:0/3414566047 >> 172.21.15.139:0/3414566047 conn(0x7f4578067950 msgr2=0x7f4578067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:19.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.526+0000 7f457cbe8640 1 -- 172.21.15.139:0/3414566047 shutdown_connections 2024-09-16T00:11:19.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.526+0000 7f457cbe8640 1 -- 172.21.15.139:0/3414566047 wait complete. 2024-09-16T00:11:19.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.527+0000 7f457cbe8640 1 Processor -- start 2024-09-16T00:11:19.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.527+0000 7f457cbe8640 1 -- start start 2024-09-16T00:11:19.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.528+0000 7f457cbe8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f457806bd90 0x7f4578153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:19.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.528+0000 7f457cbe8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f457806c170 0x7f45781541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:19.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.528+0000 7f457cbe8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4578151230 0x7f4578156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:19.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.528+0000 7f457cbe8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f45780c6210 con 0x7f457806c170 2024-09-16T00:11:19.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.528+0000 7f457cbe8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f45780c5f30 con 0x7f457806bd90 2024-09-16T00:11:19.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.528+0000 7f457cbe8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f4578073120 con 0x7f4578151230 2024-09-16T00:11:19.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.528+0000 7f4575d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f457806c170 0x7f45781541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:19.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.528+0000 7f4575d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f457806c170 0x7f45781541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43408/0 (socket says 172.21.15.139:43408) 2024-09-16T00:11:19.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.528+0000 7f4575d9b640 1 -- 172.21.15.139:0/21412174 learned_addr learned my addr 172.21.15.139:0/21412174 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:19.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.528+0000 7f457659c640 1 --2- 172.21.15.139:0/21412174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f457806bd90 0x7f4578153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:19.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.528+0000 7f4576d9d640 1 --2- 172.21.15.139:0/21412174 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4578151230 0x7f4578156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:19.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.528+0000 7f4575d9b640 1 -- 172.21.15.139:0/21412174 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4578151230 msgr2=0x7f4578156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:19.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.528+0000 7f4575d9b640 1 --2- 172.21.15.139:0/21412174 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4578151230 0x7f4578156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.528+0000 7f4575d9b640 1 -- 172.21.15.139:0/21412174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f457806bd90 msgr2=0x7f4578153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:19.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.528+0000 7f4575d9b640 1 --2- 172.21.15.139:0/21412174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f457806bd90 0x7f4578153c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.528+0000 7f4575d9b640 1 -- 172.21.15.139:0/21412174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4578151b60 con 0x7f457806c170 2024-09-16T00:11:19.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.529+0000 7f4575d9b640 1 --2- 172.21.15.139:0/21412174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f457806c170 0x7f45781541a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f456c00ccc0 tx=0x7f456c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:19.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.529+0000 7f4566ffd640 1 -- 172.21.15.139:0/21412174 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c003990 con 0x7f457806c170 2024-09-16T00:11:19.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.529+0000 7f4566ffd640 1 -- 172.21.15.139:0/21412174 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f456c003b30 con 0x7f457806c170 2024-09-16T00:11:19.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.529+0000 7f457cbe8640 1 -- 172.21.15.139:0/21412174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4578151e00 con 0x7f457806c170 2024-09-16T00:11:19.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.529+0000 7f4566ffd640 1 -- 172.21.15.139:0/21412174 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c004030 con 0x7f457806c170 2024-09-16T00:11:19.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.529+0000 7f457cbe8640 1 -- 172.21.15.139:0/21412174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4578153090 con 0x7f457806c170 2024-09-16T00:11:19.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.530+0000 7f457cbe8640 1 -- 172.21.15.139:0/21412174 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45780c5690 con 0x7f457806c170 2024-09-16T00:11:19.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.530+0000 7f4566ffd640 1 -- 172.21.15.139:0/21412174 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f456c0041d0 con 0x7f457806c170 2024-09-16T00:11:19.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.531+0000 7f4566ffd640 1 --2- 172.21.15.139:0/21412174 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f455806c2b0 0x7f455806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:19.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.531+0000 7f4566ffd640 1 -- 172.21.15.139:0/21412174 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f456c08b990 con 0x7f457806c170 2024-09-16T00:11:19.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.531+0000 7f457659c640 1 --2- 172.21.15.139:0/21412174 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f455806c2b0 0x7f455806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:19.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.531+0000 7f457659c640 1 --2- 172.21.15.139:0/21412174 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f455806c2b0 0x7f455806e770 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4570000f60 tx=0x7f4570007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:19.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.535+0000 7f4566ffd640 1 -- 172.21.15.139:0/21412174 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f45780c5690 con 0x7f457806c170 2024-09-16T00:11:19.697 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.697+0000 7f457cbe8640 1 -- 172.21.15.139:0/21412174 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4578150b20 con 0x7f455806c2b0 2024-09-16T00:11:19.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.702+0000 7f4566ffd640 1 -- 172.21.15.139:0/21412174 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254583 (secure 0 0 0) 0x7f4578150b20 con 0x7f455806c2b0 2024-09-16T00:11:19.706 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:11:19.707 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:11:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.705+0000 7f457cbe8640 1 -- 172.21.15.139:0/21412174 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f455806c2b0 msgr2=0x7f455806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.705+0000 7f457cbe8640 1 --2- 172.21.15.139:0/21412174 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f455806c2b0 0x7f455806e770 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4570000f60 tx=0x7f4570007d50 comp rx=0 tx=0).stop 2024-09-16T00:11:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.705+0000 7f457cbe8640 1 -- 172.21.15.139:0/21412174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f457806c170 msgr2=0x7f45781541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.705+0000 7f457cbe8640 1 --2- 172.21.15.139:0/21412174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f457806c170 0x7f45781541a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f456c00ccc0 tx=0x7f456c00a5c0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.705+0000 7f457cbe8640 1 -- 172.21.15.139:0/21412174 shutdown_connections 2024-09-16T00:11:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.705+0000 7f457cbe8640 1 --2- 172.21.15.139:0/21412174 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f455806c2b0 0x7f455806e770 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.705+0000 7f457cbe8640 1 --2- 172.21.15.139:0/21412174 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f4578151230 0x7f4578156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.705+0000 7f457cbe8640 1 --2- 172.21.15.139:0/21412174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f457806c170 0x7f45781541a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.705+0000 7f457cbe8640 1 --2- 172.21.15.139:0/21412174 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f457806bd90 0x7f4578153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.705+0000 7f457cbe8640 1 -- 172.21.15.139:0/21412174 >> 172.21.15.139:0/21412174 conn(0x7f4578067950 msgr2=0x7f4578159a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.705+0000 7f457cbe8640 1 -- 172.21.15.139:0/21412174 shutdown_connections 2024-09-16T00:11:19.709 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.706+0000 7f457cbe8640 1 -- 172.21.15.139:0/21412174 wait complete. 2024-09-16T00:11:19.718 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-09-16T00:11:18.066845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":129638821,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":2853,"num_read_kb":1363646,"num_write":30194,"num_write_kb":4222082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3232,"num_bytes_recovered":321049829,"num_keys_recovered":3117,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33178,"ondisk_log_size":33178,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":462196,"kb_used_data":98348,"kb_used_omap":273,"kb_used_meta":363374,"kb_avail":754512524,"statfs":{"total":773094113280,"available":772620824576,"internally_reserved":0,"allocated":100708352,"data_stored":266037186,"data_compressed":38804642,"data_compressed_allocated":49901568,"data_compressed_original":222470144,"omap_allocated":280397,"internal_metadata":372095155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.134232"},"pg_stats":[{"pgid":"3.3f","version":"123'434","reported_seq":867,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137191+0000","last_change":"2024-09-16T00:10:56.416283+0000","last_active":"2024-09-16T00:11:13.137191+0000","last_peered":"2024-09-16T00:11:13.137191+0000","last_clean":"2024-09-16T00:11:13.137191+0000","last_became_active":"2024-09-16T00:10:55.845182+0000","last_became_peered":"2024-09-16T00:10:55.845182+0000","last_unstale":"2024-09-16T00:11:13.137191+0000","last_undegraded":"2024-09-16T00:11:13.137191+0000","last_fullsized":"2024-09-16T00:11:13.137191+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:52.137388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":42086,"num_write":409,"num_write_kb":112716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":31107821,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"123'417","reported_seq":1093,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749522+0000","last_change":"2024-09-16T00:11:04.897161+0000","last_active":"2024-09-16T00:11:15.749522+0000","last_peered":"2024-09-16T00:11:15.749522+0000","last_clean":"2024-09-16T00:11:15.749522+0000","last_became_active":"2024-09-16T00:11:04.894999+0000","last_became_peered":"2024-09-16T00:11:04.894999+0000","last_unstale":"2024-09-16T00:11:15.749522+0000","last_undegraded":"2024-09-16T00:11:15.749522+0000","last_fullsized":"2024-09-16T00:11:15.749522+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:13.015652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":23064,"num_write":397,"num_write_kb":47220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":689591,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"123'367","reported_seq":657,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381045+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:11:14.381045+0000","last_peered":"2024-09-16T00:11:14.381045+0000","last_clean":"2024-09-16T00:11:14.381045+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:11:14.381045+0000","last_undegraded":"2024-09-16T00:11:14.381045+0000","last_fullsized":"2024-09-16T00:11:14.381045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15782,"num_write":349,"num_write_kb":54932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"123'389","reported_seq":679,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951475+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:11:13.951475+0000","last_peered":"2024-09-16T00:11:13.951475+0000","last_clean":"2024-09-16T00:11:13.951475+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:11:13.951475+0000","last_undegraded":"2024-09-16T00:11:13.951475+0000","last_fullsized":"2024-09-16T00:11:13.951475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":14080,"num_write":364,"num_write_kb":63191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"123'445","reported_seq":748,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749287+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:11:15.749287+0000","last_peered":"2024-09-16T00:11:15.749287+0000","last_clean":"2024-09-16T00:11:15.749287+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:11:15.749287+0000","last_undegraded":"2024-09-16T00:11:15.749287+0000","last_fullsized":"2024-09-16T00:11:15.749287+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":36096,"num_write":427,"num_write_kb":89616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"123'384","reported_seq":1352,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952787+0000","last_change":"2024-09-16T00:10:45.795247+0000","last_active":"2024-09-16T00:11:13.952787+0000","last_peered":"2024-09-16T00:11:13.952787+0000","last_clean":"2024-09-16T00:11:13.952787+0000","last_became_active":"2024-09-16T00:10:45.793434+0000","last_became_peered":"2024-09-16T00:10:45.793434+0000","last_unstale":"2024-09-16T00:11:13.952787+0000","last_undegraded":"2024-09-16T00:11:13.952787+0000","last_fullsized":"2024-09-16T00:11:13.952787+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:55.019829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":6174,"num_write":362,"num_write_kb":50827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":425,"num_bytes_recovered":23619955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"123'420","reported_seq":779,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136743+0000","last_change":"2024-09-16T00:10:56.030536+0000","last_active":"2024-09-16T00:11:13.136743+0000","last_peered":"2024-09-16T00:11:13.136743+0000","last_clean":"2024-09-16T00:11:13.136743+0000","last_became_active":"2024-09-16T00:10:55.835516+0000","last_became_peered":"2024-09-16T00:10:55.835516+0000","last_unstale":"2024-09-16T00:11:13.136743+0000","last_undegraded":"2024-09-16T00:11:13.136743+0000","last_fullsized":"2024-09-16T00:11:13.136743+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:43.152015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":16577,"num_write":401,"num_write_kb":64470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"123'374","reported_seq":641,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952597+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:11:13.952597+0000","last_peered":"2024-09-16T00:11:13.952597+0000","last_clean":"2024-09-16T00:11:13.952597+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:11:13.952597+0000","last_undegraded":"2024-09-16T00:11:13.952597+0000","last_fullsized":"2024-09-16T00:11:13.952597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":16776,"num_write":347,"num_write_kb":80164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"123'394","reported_seq":724,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137150+0000","last_change":"2024-09-16T00:10:56.261956+0000","last_active":"2024-09-16T00:11:13.137150+0000","last_peered":"2024-09-16T00:11:13.137150+0000","last_clean":"2024-09-16T00:11:13.137150+0000","last_became_active":"2024-09-16T00:10:55.840604+0000","last_became_peered":"2024-09-16T00:10:55.840604+0000","last_unstale":"2024-09-16T00:11:13.137150+0000","last_undegraded":"2024-09-16T00:11:13.137150+0000","last_fullsized":"2024-09-16T00:11:13.137150+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:39.279660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1432,"num_write":368,"num_write_kb":30311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"123'367","reported_seq":536,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011843+0000","last_change":"2024-09-16T00:10:00.734336+0000","last_active":"2024-09-16T00:11:14.011843+0000","last_peered":"2024-09-16T00:11:14.011843+0000","last_clean":"2024-09-16T00:11:14.011843+0000","last_became_active":"2024-09-16T00:10:00.284452+0000","last_became_peered":"2024-09-16T00:10:00.284452+0000","last_unstale":"2024-09-16T00:11:14.011843+0000","last_undegraded":"2024-09-16T00:11:14.011843+0000","last_fullsized":"2024-09-16T00:11:14.011843+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:56:50.530439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4096,"num_write":335,"num_write_kb":29952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":8736255,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"123'355","reported_seq":535,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011551+0000","last_change":"2024-09-16T00:10:00.587110+0000","last_active":"2024-09-16T00:11:14.011551+0000","last_peered":"2024-09-16T00:11:14.011551+0000","last_clean":"2024-09-16T00:11:14.011551+0000","last_became_active":"2024-09-16T00:10:00.280273+0000","last_became_peered":"2024-09-16T00:10:00.280273+0000","last_unstale":"2024-09-16T00:11:14.011551+0000","last_undegraded":"2024-09-16T00:11:14.011551+0000","last_fullsized":"2024-09-16T00:11:14.011551+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:01:57.902939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":975,"num_write":339,"num_write_kb":25518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":10753732,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"123'366","reported_seq":705,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952011+0000","last_change":"2024-09-16T00:10:00.797372+0000","last_active":"2024-09-16T00:11:13.952011+0000","last_peered":"2024-09-16T00:11:13.952011+0000","last_clean":"2024-09-16T00:11:13.952011+0000","last_became_active":"2024-09-16T00:10:00.278765+0000","last_became_peered":"2024-09-16T00:10:00.278765+0000","last_unstale":"2024-09-16T00:11:13.952011+0000","last_undegraded":"2024-09-16T00:11:13.952011+0000","last_fullsized":"2024-09-16T00:11:13.952011+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:03.318919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":11563,"num_write":348,"num_write_kb":58933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":29565877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"123'419","reported_seq":784,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749693+0000","last_change":"2024-09-16T00:10:00.715647+0000","last_active":"2024-09-16T00:11:15.749693+0000","last_peered":"2024-09-16T00:11:15.749693+0000","last_clean":"2024-09-16T00:11:15.749693+0000","last_became_active":"2024-09-16T00:10:00.286050+0000","last_became_peered":"2024-09-16T00:10:00.286050+0000","last_unstale":"2024-09-16T00:11:15.749693+0000","last_undegraded":"2024-09-16T00:11:15.749693+0000","last_fullsized":"2024-09-16T00:11:15.749693+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:07.462899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":35908,"num_write":393,"num_write_kb":104863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":30230864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"123'415","reported_seq":724,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.380880+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:11:14.380880+0000","last_peered":"2024-09-16T00:11:14.380880+0000","last_clean":"2024-09-16T00:11:14.380880+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:11:14.380880+0000","last_undegraded":"2024-09-16T00:11:14.380880+0000","last_fullsized":"2024-09-16T00:11:14.380880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":23364,"num_write":390,"num_write_kb":43321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"123'392","reported_seq":684,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749312+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:11:15.749312+0000","last_peered":"2024-09-16T00:11:15.749312+0000","last_clean":"2024-09-16T00:11:15.749312+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:11:15.749312+0000","last_undegraded":"2024-09-16T00:11:15.749312+0000","last_fullsized":"2024-09-16T00:11:15.749312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":14230,"num_write":362,"num_write_kb":33768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"123'439","reported_seq":756,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951004+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:11:13.951004+0000","last_peered":"2024-09-16T00:11:13.951004+0000","last_clean":"2024-09-16T00:11:13.951004+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:11:13.951004+0000","last_undegraded":"2024-09-16T00:11:13.951004+0000","last_fullsized":"2024-09-16T00:11:13.951004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":40234,"num_write":414,"num_write_kb":67785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"123'394","reported_seq":890,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952446+0000","last_change":"2024-09-16T00:10:56.577752+0000","last_active":"2024-09-16T00:11:13.952446+0000","last_peered":"2024-09-16T00:11:13.952446+0000","last_clean":"2024-09-16T00:11:13.952446+0000","last_became_active":"2024-09-16T00:10:55.845880+0000","last_became_peered":"2024-09-16T00:10:55.845880+0000","last_unstale":"2024-09-16T00:11:13.952446+0000","last_undegraded":"2024-09-16T00:11:13.952446+0000","last_fullsized":"2024-09-16T00:11:13.952446+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:32.037945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":22524,"num_write":374,"num_write_kb":46842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"123'391","reported_seq":720,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952778+0000","last_change":"2024-09-16T00:10:00.461437+0000","last_active":"2024-09-16T00:11:13.952778+0000","last_peered":"2024-09-16T00:11:13.952778+0000","last_clean":"2024-09-16T00:11:13.952778+0000","last_became_active":"2024-09-16T00:10:00.282592+0000","last_became_peered":"2024-09-16T00:10:00.282592+0000","last_unstale":"2024-09-16T00:11:13.952778+0000","last_undegraded":"2024-09-16T00:11:13.952778+0000","last_fullsized":"2024-09-16T00:11:13.952778+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:09.733353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8711,"num_write":366,"num_write_kb":46855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":21285626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"123'417","reported_seq":825,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137126+0000","last_change":"2024-09-16T00:10:56.142127+0000","last_active":"2024-09-16T00:11:13.137126+0000","last_peered":"2024-09-16T00:11:13.137126+0000","last_clean":"2024-09-16T00:11:13.137126+0000","last_became_active":"2024-09-16T00:10:55.832595+0000","last_became_peered":"2024-09-16T00:10:55.832595+0000","last_unstale":"2024-09-16T00:11:13.137126+0000","last_undegraded":"2024-09-16T00:11:13.137126+0000","last_fullsized":"2024-09-16T00:11:13.137126+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:44:56.072997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":14437,"num_write":396,"num_write_kb":83979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":103497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"123'423","reported_seq":602,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011546+0000","last_change":"2024-09-16T00:10:00.566870+0000","last_active":"2024-09-16T00:11:14.011546+0000","last_peered":"2024-09-16T00:11:14.011546+0000","last_clean":"2024-09-16T00:11:14.011546+0000","last_became_active":"2024-09-16T00:10:00.281258+0000","last_became_peered":"2024-09-16T00:10:00.281258+0000","last_unstale":"2024-09-16T00:11:14.011546+0000","last_undegraded":"2024-09-16T00:11:14.011546+0000","last_fullsized":"2024-09-16T00:11:14.011546+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:29.197743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":15502,"num_write":406,"num_write_kb":30222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":17199573,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"123'404","reported_seq":883,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381025+0000","last_change":"2024-09-16T00:10:56.104046+0000","last_active":"2024-09-16T00:11:14.381025+0000","last_peered":"2024-09-16T00:11:14.381025+0000","last_clean":"2024-09-16T00:11:14.381025+0000","last_became_active":"2024-09-16T00:10:55.835150+0000","last_became_peered":"2024-09-16T00:10:55.835150+0000","last_unstale":"2024-09-16T00:11:14.381025+0000","last_undegraded":"2024-09-16T00:11:14.381025+0000","last_fullsized":"2024-09-16T00:11:14.381025+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:08.552926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":34318,"num_write":385,"num_write_kb":91769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"123'444","reported_seq":918,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951970+0000","last_change":"2024-09-16T00:10:55.939360+0000","last_active":"2024-09-16T00:11:13.951970+0000","last_peered":"2024-09-16T00:11:13.951970+0000","last_clean":"2024-09-16T00:11:13.951970+0000","last_became_active":"2024-09-16T00:10:55.833335+0000","last_became_peered":"2024-09-16T00:10:55.833335+0000","last_unstale":"2024-09-16T00:11:13.951970+0000","last_undegraded":"2024-09-16T00:11:13.951970+0000","last_fullsized":"2024-09-16T00:11:13.951970+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:08:29.270004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":35528,"num_write":422,"num_write_kb":92852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"123'395","reported_seq":683,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749700+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:11:15.749700+0000","last_peered":"2024-09-16T00:11:15.749700+0000","last_clean":"2024-09-16T00:11:15.749700+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:11:15.749700+0000","last_undegraded":"2024-09-16T00:11:15.749700+0000","last_fullsized":"2024-09-16T00:11:15.749700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":13410,"num_write":369,"num_write_kb":58805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"123'421","reported_seq":703,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951092+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:11:13.951092+0000","last_peered":"2024-09-16T00:11:13.951092+0000","last_clean":"2024-09-16T00:11:13.951092+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:11:13.951092+0000","last_undegraded":"2024-09-16T00:11:13.951092+0000","last_fullsized":"2024-09-16T00:11:13.951092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":12682,"num_write":399,"num_write_kb":54869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"123'421","reported_seq":712,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749956+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:11:15.749956+0000","last_peered":"2024-09-16T00:11:15.749956+0000","last_clean":"2024-09-16T00:11:15.749956+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:11:15.749956+0000","last_undegraded":"2024-09-16T00:11:15.749956+0000","last_fullsized":"2024-09-16T00:11:15.749956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":14188,"num_write":408,"num_write_kb":59178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"123'448","reported_seq":862,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011709+0000","last_change":"2024-09-16T00:10:56.267499+0000","last_active":"2024-09-16T00:11:14.011709+0000","last_peered":"2024-09-16T00:11:14.011709+0000","last_clean":"2024-09-16T00:11:14.011709+0000","last_became_active":"2024-09-16T00:10:55.841967+0000","last_became_peered":"2024-09-16T00:10:55.841967+0000","last_unstale":"2024-09-16T00:11:14.011709+0000","last_undegraded":"2024-09-16T00:11:14.011709+0000","last_fullsized":"2024-09-16T00:11:14.011709+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:15.324718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":12330,"num_write":421,"num_write_kb":35146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":12903786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"123'402","reported_seq":1052,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749340+0000","last_change":"2024-09-16T00:11:04.895240+0000","last_active":"2024-09-16T00:11:15.749340+0000","last_peered":"2024-09-16T00:11:15.749340+0000","last_clean":"2024-09-16T00:11:15.749340+0000","last_became_active":"2024-09-16T00:11:04.892794+0000","last_became_peered":"2024-09-16T00:11:04.892794+0000","last_unstale":"2024-09-16T00:11:15.749340+0000","last_undegraded":"2024-09-16T00:11:15.749340+0000","last_fullsized":"2024-09-16T00:11:15.749340+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:44.365114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":21104,"num_write":383,"num_write_kb":75742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":628439,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"123'386","reported_seq":688,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749839+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:11:15.749839+0000","last_peered":"2024-09-16T00:11:15.749839+0000","last_clean":"2024-09-16T00:11:15.749839+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:11:15.749839+0000","last_undegraded":"2024-09-16T00:11:15.749839+0000","last_fullsized":"2024-09-16T00:11:15.749839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":16408,"num_write":366,"num_write_kb":51674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"123'398","reported_seq":717,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381121+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:11:14.381121+0000","last_peered":"2024-09-16T00:11:14.381121+0000","last_clean":"2024-09-16T00:11:14.381121+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:11:14.381121+0000","last_undegraded":"2024-09-16T00:11:14.381121+0000","last_fullsized":"2024-09-16T00:11:14.381121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":33174,"num_write":373,"num_write_kb":66849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"123'486","reported_seq":780,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952852+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:11:13.952852+0000","last_peered":"2024-09-16T00:11:13.952852+0000","last_clean":"2024-09-16T00:11:13.952852+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:11:13.952852+0000","last_undegraded":"2024-09-16T00:11:13.952852+0000","last_fullsized":"2024-09-16T00:11:13.952852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":27231,"num_write":453,"num_write_kb":104771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"123'451","reported_seq":789,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952668+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:11:13.952668+0000","last_peered":"2024-09-16T00:11:13.952668+0000","last_clean":"2024-09-16T00:11:13.952668+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:11:13.952668+0000","last_undegraded":"2024-09-16T00:11:13.952668+0000","last_fullsized":"2024-09-16T00:11:13.952668+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":35655,"num_write":431,"num_write_kb":97245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"123'421","reported_seq":602,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011470+0000","last_change":"2024-09-16T00:10:00.535553+0000","last_active":"2024-09-16T00:11:14.011470+0000","last_peered":"2024-09-16T00:11:14.011470+0000","last_clean":"2024-09-16T00:11:14.011470+0000","last_became_active":"2024-09-16T00:10:00.281378+0000","last_became_peered":"2024-09-16T00:10:00.281378+0000","last_unstale":"2024-09-16T00:11:14.011470+0000","last_undegraded":"2024-09-16T00:11:14.011470+0000","last_fullsized":"2024-09-16T00:11:14.011470+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:49.548709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20395,"num_write":401,"num_write_kb":32731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":22876917,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"123'401","reported_seq":741,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951565+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:11:13.951565+0000","last_peered":"2024-09-16T00:11:13.951565+0000","last_clean":"2024-09-16T00:11:13.951565+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:11:13.951565+0000","last_undegraded":"2024-09-16T00:11:13.951565+0000","last_fullsized":"2024-09-16T00:11:13.951565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":48141,"num_write":381,"num_write_kb":63611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"120'126","reported_seq":402,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749808+0000","last_change":"2024-09-16T00:10:55.854872+0000","last_active":"2024-09-16T00:11:15.749808+0000","last_peered":"2024-09-16T00:11:15.749808+0000","last_clean":"2024-09-16T00:11:15.749808+0000","last_became_active":"2024-09-16T00:10:55.843367+0000","last_became_peered":"2024-09-16T00:10:55.843367+0000","last_unstale":"2024-09-16T00:11:15.749808+0000","last_undegraded":"2024-09-16T00:11:15.749808+0000","last_fullsized":"2024-09-16T00:11:15.749808+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:07.797903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"123'346","reported_seq":633,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381167+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:11:14.381167+0000","last_peered":"2024-09-16T00:11:14.381167+0000","last_clean":"2024-09-16T00:11:14.381167+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:11:14.381167+0000","last_undegraded":"2024-09-16T00:11:14.381167+0000","last_fullsized":"2024-09-16T00:11:14.381167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":16420,"num_write":326,"num_write_kb":62710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"122'340","reported_seq":505,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011382+0000","last_change":"2024-09-16T00:10:00.282958+0000","last_active":"2024-09-16T00:11:14.011382+0000","last_peered":"2024-09-16T00:11:14.011382+0000","last_clean":"2024-09-16T00:11:14.011382+0000","last_became_active":"2024-09-16T00:10:00.282541+0000","last_became_peered":"2024-09-16T00:10:00.282541+0000","last_unstale":"2024-09-16T00:11:14.011382+0000","last_undegraded":"2024-09-16T00:11:14.011382+0000","last_fullsized":"2024-09-16T00:11:14.011382+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:51.198787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":285,"num_write_kb":12471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"123'226","reported_seq":510,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749827+0000","last_change":"2024-09-16T00:10:00.313589+0000","last_active":"2024-09-16T00:11:15.749827+0000","last_peered":"2024-09-16T00:11:15.749827+0000","last_clean":"2024-09-16T00:11:15.749827+0000","last_became_active":"2024-09-16T00:10:00.284602+0000","last_became_peered":"2024-09-16T00:10:00.284602+0000","last_unstale":"2024-09-16T00:11:15.749827+0000","last_undegraded":"2024-09-16T00:11:15.749827+0000","last_fullsized":"2024-09-16T00:11:15.749827+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:30.032698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":14,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"123'433","reported_seq":732,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749206+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:11:15.749206+0000","last_peered":"2024-09-16T00:11:15.749206+0000","last_clean":"2024-09-16T00:11:15.749206+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:11:15.749206+0000","last_undegraded":"2024-09-16T00:11:15.749206+0000","last_fullsized":"2024-09-16T00:11:15.749206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":18480,"num_write":411,"num_write_kb":71748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"123'833","reported_seq":1095,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951164+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:11:13.951164+0000","last_peered":"2024-09-16T00:11:13.951164+0000","last_clean":"2024-09-16T00:11:13.951164+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:11:13.951164+0000","last_undegraded":"2024-09-16T00:11:13.951164+0000","last_fullsized":"2024-09-16T00:11:13.951164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":8591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"123'425","reported_seq":774,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749285+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:11:15.749285+0000","last_peered":"2024-09-16T00:11:15.749285+0000","last_clean":"2024-09-16T00:11:15.749285+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:11:15.749285+0000","last_undegraded":"2024-09-16T00:11:15.749285+0000","last_fullsized":"2024-09-16T00:11:15.749285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":46871,"num_write":401,"num_write_kb":84187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"123'65","reported_seq":296,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011701+0000","last_change":"2024-09-16T00:10:00.283876+0000","last_active":"2024-09-16T00:11:14.011701+0000","last_peered":"2024-09-16T00:11:14.011701+0000","last_clean":"2024-09-16T00:11:14.011701+0000","last_became_active":"2024-09-16T00:10:00.283555+0000","last_became_peered":"2024-09-16T00:10:00.283555+0000","last_unstale":"2024-09-16T00:11:14.011701+0000","last_undegraded":"2024-09-16T00:11:14.011701+0000","last_fullsized":"2024-09-16T00:11:14.011701+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:48.543737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"123'440","reported_seq":904,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951434+0000","last_change":"2024-09-16T00:10:56.502658+0000","last_active":"2024-09-16T00:11:13.951434+0000","last_peered":"2024-09-16T00:11:13.951434+0000","last_clean":"2024-09-16T00:11:13.951434+0000","last_became_active":"2024-09-16T00:10:55.840027+0000","last_became_peered":"2024-09-16T00:10:55.840027+0000","last_unstale":"2024-09-16T00:11:13.951434+0000","last_undegraded":"2024-09-16T00:11:13.951434+0000","last_fullsized":"2024-09-16T00:11:13.951434+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:52.105868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":16686,"num_write":415,"num_write_kb":80112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"123'225","reported_seq":494,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951555+0000","last_change":"2024-09-16T00:10:00.285386+0000","last_active":"2024-09-16T00:11:13.951555+0000","last_peered":"2024-09-16T00:11:13.951555+0000","last_clean":"2024-09-16T00:11:13.951555+0000","last_became_active":"2024-09-16T00:10:00.284717+0000","last_became_peered":"2024-09-16T00:10:00.284717+0000","last_unstale":"2024-09-16T00:11:13.951555+0000","last_undegraded":"2024-09-16T00:11:13.951555+0000","last_fullsized":"2024-09-16T00:11:13.951555+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:07.794163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":4181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"123'378","reported_seq":681,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381204+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:11:14.381204+0000","last_peered":"2024-09-16T00:11:14.381204+0000","last_clean":"2024-09-16T00:11:14.381204+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:11:14.381204+0000","last_undegraded":"2024-09-16T00:11:14.381204+0000","last_fullsized":"2024-09-16T00:11:14.381204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":18030,"num_write":359,"num_write_kb":55428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"125'131","reported_seq":392,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749519+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:11:15.749519+0000","last_peered":"2024-09-16T00:11:15.749519+0000","last_clean":"2024-09-16T00:11:15.749519+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:11:15.749519+0000","last_undegraded":"2024-09-16T00:11:15.749519+0000","last_fullsized":"2024-09-16T00:11:15.749519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"123'373","reported_seq":818,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952022+0000","last_change":"2024-09-16T00:10:56.173134+0000","last_active":"2024-09-16T00:11:13.952022+0000","last_peered":"2024-09-16T00:11:13.952022+0000","last_clean":"2024-09-16T00:11:13.952022+0000","last_became_active":"2024-09-16T00:10:55.833273+0000","last_became_peered":"2024-09-16T00:10:55.833273+0000","last_unstale":"2024-09-16T00:11:13.952022+0000","last_undegraded":"2024-09-16T00:11:13.952022+0000","last_fullsized":"2024-09-16T00:11:13.952022+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:11.402988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":22960,"num_write":348,"num_write_kb":63161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"125'238","reported_seq":414,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011582+0000","last_change":"2024-09-16T00:10:00.300895+0000","last_active":"2024-09-16T00:11:14.011582+0000","last_peered":"2024-09-16T00:11:14.011582+0000","last_clean":"2024-09-16T00:11:14.011582+0000","last_became_active":"2024-09-16T00:10:00.283474+0000","last_became_peered":"2024-09-16T00:10:00.283474+0000","last_unstale":"2024-09-16T00:11:14.011582+0000","last_undegraded":"2024-09-16T00:11:14.011582+0000","last_fullsized":"2024-09-16T00:11:14.011582+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:28:02.563817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10145896,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":186,"num_write_kb":10014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":13200,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"123'403","reported_seq":774,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136869+0000","last_change":"2024-09-16T00:10:56.320793+0000","last_active":"2024-09-16T00:11:13.136869+0000","last_peered":"2024-09-16T00:11:13.136869+0000","last_clean":"2024-09-16T00:11:13.136869+0000","last_became_active":"2024-09-16T00:10:55.836462+0000","last_became_peered":"2024-09-16T00:10:55.836462+0000","last_unstale":"2024-09-16T00:11:13.136869+0000","last_undegraded":"2024-09-16T00:11:13.136869+0000","last_fullsized":"2024-09-16T00:11:13.136869+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:07:47.717822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":23285,"num_write":372,"num_write_kb":67093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"122'470","reported_seq":732,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381212+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:11:14.381212+0000","last_peered":"2024-09-16T00:11:14.381212+0000","last_clean":"2024-09-16T00:11:14.381212+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:11:14.381212+0000","last_undegraded":"2024-09-16T00:11:14.381212+0000","last_fullsized":"2024-09-16T00:11:14.381212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":419,"num_write_kb":8413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"123'404","reported_seq":872,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.380881+0000","last_change":"2024-09-16T00:10:56.427976+0000","last_active":"2024-09-16T00:11:14.380881+0000","last_peered":"2024-09-16T00:11:14.380881+0000","last_clean":"2024-09-16T00:11:14.380881+0000","last_became_active":"2024-09-16T00:10:55.839196+0000","last_became_peered":"2024-09-16T00:10:55.839196+0000","last_unstale":"2024-09-16T00:11:14.380881+0000","last_undegraded":"2024-09-16T00:11:14.380881+0000","last_fullsized":"2024-09-16T00:11:14.380881+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:37.852194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":13263,"num_write":381,"num_write_kb":51174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"123'59","reported_seq":321,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952149+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:11:13.952149+0000","last_peered":"2024-09-16T00:11:13.952149+0000","last_clean":"2024-09-16T00:11:13.952149+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:11:13.952149+0000","last_undegraded":"2024-09-16T00:11:13.952149+0000","last_fullsized":"2024-09-16T00:11:13.952149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"123'394","reported_seq":733,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136975+0000","last_change":"2024-09-16T00:10:56.085067+0000","last_active":"2024-09-16T00:11:13.136975+0000","last_peered":"2024-09-16T00:11:13.136975+0000","last_clean":"2024-09-16T00:11:13.136975+0000","last_became_active":"2024-09-16T00:10:55.834112+0000","last_became_peered":"2024-09-16T00:10:55.834112+0000","last_unstale":"2024-09-16T00:11:13.136975+0000","last_undegraded":"2024-09-16T00:11:13.136975+0000","last_fullsized":"2024-09-16T00:11:13.136975+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:35.278069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":13990,"num_write":368,"num_write_kb":63962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"120'70","reported_seq":218,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011398+0000","last_change":"2024-09-16T00:10:00.294519+0000","last_active":"2024-09-16T00:11:14.011398+0000","last_peered":"2024-09-16T00:11:14.011398+0000","last_clean":"2024-09-16T00:11:14.011398+0000","last_became_active":"2024-09-16T00:10:00.283066+0000","last_became_peered":"2024-09-16T00:10:00.283066+0000","last_unstale":"2024-09-16T00:11:14.011398+0000","last_undegraded":"2024-09-16T00:11:14.011398+0000","last_fullsized":"2024-09-16T00:11:14.011398+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:32.621176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"123'424","reported_seq":607,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011378+0000","last_change":"2024-09-16T00:10:00.460910+0000","last_active":"2024-09-16T00:11:14.011378+0000","last_peered":"2024-09-16T00:11:14.011378+0000","last_clean":"2024-09-16T00:11:14.011378+0000","last_became_active":"2024-09-16T00:10:00.280750+0000","last_became_peered":"2024-09-16T00:10:00.280750+0000","last_unstale":"2024-09-16T00:11:14.011378+0000","last_undegraded":"2024-09-16T00:11:14.011378+0000","last_fullsized":"2024-09-16T00:11:14.011378+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:47.634650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":19573,"num_write":402,"num_write_kb":50826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":33491520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"125'72","reported_seq":334,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952685+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:11:13.952685+0000","last_peered":"2024-09-16T00:11:13.952685+0000","last_clean":"2024-09-16T00:11:13.952685+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:11:13.952685+0000","last_undegraded":"2024-09-16T00:11:13.952685+0000","last_fullsized":"2024-09-16T00:11:13.952685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"123'457","reported_seq":814,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136679+0000","last_change":"2024-09-16T00:10:55.922694+0000","last_active":"2024-09-16T00:11:13.136679+0000","last_peered":"2024-09-16T00:11:13.136679+0000","last_clean":"2024-09-16T00:11:13.136679+0000","last_became_active":"2024-09-16T00:10:55.827989+0000","last_became_peered":"2024-09-16T00:10:55.827989+0000","last_unstale":"2024-09-16T00:11:13.136679+0000","last_undegraded":"2024-09-16T00:11:13.136679+0000","last_fullsized":"2024-09-16T00:11:13.136679+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:23.666346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20419,"num_write":427,"num_write_kb":43035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"123'145","reported_seq":407,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951453+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:11:13.951453+0000","last_peered":"2024-09-16T00:11:13.951453+0000","last_clean":"2024-09-16T00:11:13.951453+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:11:13.951453+0000","last_undegraded":"2024-09-16T00:11:13.951453+0000","last_fullsized":"2024-09-16T00:11:13.951453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91,"num_write_kb":4217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"123'358","reported_seq":694,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137061+0000","last_change":"2024-09-16T00:10:55.971163+0000","last_active":"2024-09-16T00:11:13.137061+0000","last_peered":"2024-09-16T00:11:13.137061+0000","last_clean":"2024-09-16T00:11:13.137061+0000","last_became_active":"2024-09-16T00:10:55.834401+0000","last_became_peered":"2024-09-16T00:10:55.834401+0000","last_unstale":"2024-09-16T00:11:13.137061+0000","last_undegraded":"2024-09-16T00:11:13.137061+0000","last_fullsized":"2024-09-16T00:11:13.137061+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:05.466545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":10790,"num_write":330,"num_write_kb":21585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"120'282","reported_seq":544,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952893+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:11:13.952893+0000","last_peered":"2024-09-16T00:11:13.952893+0000","last_clean":"2024-09-16T00:11:13.952893+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:11:13.952893+0000","last_undegraded":"2024-09-16T00:11:13.952893+0000","last_fullsized":"2024-09-16T00:11:13.952893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":238,"num_write_kb":4238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"123'372","reported_seq":696,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137259+0000","last_change":"2024-09-16T00:10:56.200838+0000","last_active":"2024-09-16T00:11:13.137259+0000","last_peered":"2024-09-16T00:11:13.137259+0000","last_clean":"2024-09-16T00:11:13.137259+0000","last_became_active":"2024-09-16T00:10:55.838254+0000","last_became_peered":"2024-09-16T00:10:55.838254+0000","last_unstale":"2024-09-16T00:11:13.137259+0000","last_undegraded":"2024-09-16T00:11:13.137259+0000","last_fullsized":"2024-09-16T00:11:13.137259+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:30.944542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":350,"num_write":351,"num_write_kb":38221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"122'534","reported_seq":795,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749610+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:11:15.749610+0000","last_peered":"2024-09-16T00:11:15.749610+0000","last_clean":"2024-09-16T00:11:15.749610+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:11:15.749610+0000","last_undegraded":"2024-09-16T00:11:15.749610+0000","last_fullsized":"2024-09-16T00:11:15.749610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":473,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"123'418","reported_seq":682,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951696+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:11:13.951696+0000","last_peered":"2024-09-16T00:11:13.951696+0000","last_clean":"2024-09-16T00:11:13.951696+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:11:13.951696+0000","last_undegraded":"2024-09-16T00:11:13.951696+0000","last_fullsized":"2024-09-16T00:11:13.951696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":675,"num_write":389,"num_write_kb":34696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"123'72","reported_seq":334,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952967+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:11:13.952967+0000","last_peered":"2024-09-16T00:11:13.952967+0000","last_clean":"2024-09-16T00:11:13.952967+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:11:13.952967+0000","last_undegraded":"2024-09-16T00:11:13.952967+0000","last_fullsized":"2024-09-16T00:11:13.952967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"122'397","reported_seq":761,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136957+0000","last_change":"2024-09-16T00:10:56.368023+0000","last_active":"2024-09-16T00:11:13.136957+0000","last_peered":"2024-09-16T00:11:13.136957+0000","last_clean":"2024-09-16T00:11:13.136957+0000","last_became_active":"2024-09-16T00:10:55.839714+0000","last_became_peered":"2024-09-16T00:10:55.839714+0000","last_unstale":"2024-09-16T00:11:13.136957+0000","last_undegraded":"2024-09-16T00:11:13.136957+0000","last_fullsized":"2024-09-16T00:11:13.136957+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:04.192344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":22590,"num_write":379,"num_write_kb":59088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":372,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136608+0000","last_change":"2024-09-16T00:10:55.832444+0000","last_active":"2024-09-16T00:11:13.136608+0000","last_peered":"2024-09-16T00:11:13.136608+0000","last_clean":"2024-09-16T00:11:13.136608+0000","last_became_active":"2024-09-16T00:10:55.831881+0000","last_became_peered":"2024-09-16T00:10:55.831881+0000","last_unstale":"2024-09-16T00:11:13.136608+0000","last_undegraded":"2024-09-16T00:11:13.136608+0000","last_fullsized":"2024-09-16T00:11:13.136608+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:48.243166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"123'63","reported_seq":325,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951198+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:11:13.951198+0000","last_peered":"2024-09-16T00:11:13.951198+0000","last_clean":"2024-09-16T00:11:13.951198+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:11:13.951198+0000","last_undegraded":"2024-09-16T00:11:13.951198+0000","last_fullsized":"2024-09-16T00:11:13.951198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"123'369","reported_seq":664,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749611+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:11:15.749611+0000","last_peered":"2024-09-16T00:11:15.749611+0000","last_clean":"2024-09-16T00:11:15.749611+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:11:15.749611+0000","last_undegraded":"2024-09-16T00:11:15.749611+0000","last_fullsized":"2024-09-16T00:11:15.749611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":16103,"num_write":342,"num_write_kb":46241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"123'399","reported_seq":643,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136617+0000","last_change":"2024-09-16T00:11:04.896119+0000","last_active":"2024-09-16T00:11:13.136617+0000","last_peered":"2024-09-16T00:11:13.136617+0000","last_clean":"2024-09-16T00:11:13.136617+0000","last_became_active":"2024-09-16T00:11:04.895672+0000","last_became_peered":"2024-09-16T00:11:04.895672+0000","last_unstale":"2024-09-16T00:11:13.136617+0000","last_undegraded":"2024-09-16T00:11:13.136617+0000","last_fullsized":"2024-09-16T00:11:13.136617+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:45.173943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":346,"num_write_kb":8359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":16777216,"num_keys_recovered":28,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"123'393","reported_seq":700,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.380785+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:11:14.380785+0000","last_peered":"2024-09-16T00:11:14.380785+0000","last_clean":"2024-09-16T00:11:14.380785+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:11:14.380785+0000","last_undegraded":"2024-09-16T00:11:14.380785+0000","last_fullsized":"2024-09-16T00:11:14.380785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":29385,"num_write":362,"num_write_kb":83610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"125'242","reported_seq":680,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952710+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:11:13.952710+0000","last_peered":"2024-09-16T00:11:13.952710+0000","last_clean":"2024-09-16T00:11:13.952710+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:11:13.952710+0000","last_undegraded":"2024-09-16T00:11:13.952710+0000","last_fullsized":"2024-09-16T00:11:13.952710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461247,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":176,"num_read_kb":1534,"num_write":179,"num_write_kb":1542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"123'400","reported_seq":712,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749491+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:11:15.749491+0000","last_peered":"2024-09-16T00:11:15.749491+0000","last_clean":"2024-09-16T00:11:15.749491+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:11:15.749491+0000","last_undegraded":"2024-09-16T00:11:15.749491+0000","last_fullsized":"2024-09-16T00:11:15.749491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":28288,"num_write":378,"num_write_kb":66949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"119'400","reported_seq":662,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952094+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:11:13.952094+0000","last_peered":"2024-09-16T00:11:13.952094+0000","last_clean":"2024-09-16T00:11:13.952094+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:11:13.952094+0000","last_undegraded":"2024-09-16T00:11:13.952094+0000","last_fullsized":"2024-09-16T00:11:13.952094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":353,"num_write_kb":8381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"123'387","reported_seq":683,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951764+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:11:13.951764+0000","last_peered":"2024-09-16T00:11:13.951764+0000","last_clean":"2024-09-16T00:11:13.951764+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:11:13.951764+0000","last_undegraded":"2024-09-16T00:11:13.951764+0000","last_fullsized":"2024-09-16T00:11:13.951764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":22188,"num_write":360,"num_write_kb":71552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"122'57","reported_seq":318,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749407+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:11:15.749407+0000","last_peered":"2024-09-16T00:11:15.749407+0000","last_clean":"2024-09-16T00:11:15.749407+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:11:15.749407+0000","last_undegraded":"2024-09-16T00:11:15.749407+0000","last_fullsized":"2024-09-16T00:11:15.749407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"123'426","reported_seq":725,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952294+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:11:13.952294+0000","last_peered":"2024-09-16T00:11:13.952294+0000","last_clean":"2024-09-16T00:11:13.952294+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:11:13.952294+0000","last_undegraded":"2024-09-16T00:11:13.952294+0000","last_fullsized":"2024-09-16T00:11:13.952294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":24752,"num_write":390,"num_write_kb":75609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"123'258","reported_seq":301,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137011+0000","last_change":"2024-09-16T00:10:58.839395+0000","last_active":"2024-09-16T00:11:13.137011+0000","last_peered":"2024-09-16T00:11:13.137011+0000","last_clean":"2024-09-16T00:11:13.137011+0000","last_became_active":"2024-09-16T00:10:58.839124+0000","last_became_peered":"2024-09-16T00:10:58.839124+0000","last_unstale":"2024-09-16T00:11:13.137011+0000","last_undegraded":"2024-09-16T00:11:13.137011+0000","last_fullsized":"2024-09-16T00:11:13.137011+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:29.136640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":202,"num_write_kb":4192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"123'398","reported_seq":715,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749363+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:11:15.749363+0000","last_peered":"2024-09-16T00:11:15.749363+0000","last_clean":"2024-09-16T00:11:15.749363+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:11:15.749363+0000","last_undegraded":"2024-09-16T00:11:15.749363+0000","last_fullsized":"2024-09-16T00:11:15.749363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":31221,"num_write":370,"num_write_kb":76143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"125'186","reported_seq":482,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749569+0000","last_change":"2024-09-16T00:11:04.894640+0000","last_active":"2024-09-16T00:11:15.749569+0000","last_peered":"2024-09-16T00:11:15.749569+0000","last_clean":"2024-09-16T00:11:15.749569+0000","last_became_active":"2024-09-16T00:11:04.894163+0000","last_became_peered":"2024-09-16T00:11:04.894163+0000","last_unstale":"2024-09-16T00:11:15.749569+0000","last_undegraded":"2024-09-16T00:11:15.749569+0000","last_fullsized":"2024-09-16T00:11:15.749569+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:41.395352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":345,"num_write_kb":1559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":2964,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"123'344","reported_seq":786,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.750028+0000","last_change":"2024-09-16T00:10:56.334688+0000","last_active":"2024-09-16T00:11:15.750028+0000","last_peered":"2024-09-16T00:11:15.750028+0000","last_clean":"2024-09-16T00:11:15.750028+0000","last_became_active":"2024-09-16T00:10:55.845451+0000","last_became_peered":"2024-09-16T00:10:55.845451+0000","last_unstale":"2024-09-16T00:11:15.750028+0000","last_undegraded":"2024-09-16T00:11:15.750028+0000","last_fullsized":"2024-09-16T00:11:15.750028+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:04.965027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":19274,"num_write":323,"num_write_kb":22071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"123'63","reported_seq":309,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011696+0000","last_change":"2024-09-16T00:10:55.858845+0000","last_active":"2024-09-16T00:11:14.011696+0000","last_peered":"2024-09-16T00:11:14.011696+0000","last_clean":"2024-09-16T00:11:14.011696+0000","last_became_active":"2024-09-16T00:10:55.845687+0000","last_became_peered":"2024-09-16T00:10:55.845687+0000","last_unstale":"2024-09-16T00:11:14.011696+0000","last_undegraded":"2024-09-16T00:11:14.011696+0000","last_fullsized":"2024-09-16T00:11:14.011696+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:53.322675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"123'447","reported_seq":750,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381154+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:11:14.381154+0000","last_peered":"2024-09-16T00:11:14.381154+0000","last_clean":"2024-09-16T00:11:14.381154+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:11:14.381154+0000","last_undegraded":"2024-09-16T00:11:14.381154+0000","last_fullsized":"2024-09-16T00:11:14.381154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":17894,"num_write":426,"num_write_kb":88843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"123'564","reported_seq":784,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136761+0000","last_change":"2024-09-16T00:10:55.852895+0000","last_active":"2024-09-16T00:11:13.136761+0000","last_peered":"2024-09-16T00:11:13.136761+0000","last_clean":"2024-09-16T00:11:13.136761+0000","last_became_active":"2024-09-16T00:10:55.828121+0000","last_became_peered":"2024-09-16T00:10:55.828121+0000","last_unstale":"2024-09-16T00:11:13.136761+0000","last_undegraded":"2024-09-16T00:11:13.136761+0000","last_fullsized":"2024-09-16T00:11:13.136761+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:57.000428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"123'454","reported_seq":732,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951802+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:11:13.951802+0000","last_peered":"2024-09-16T00:11:13.951802+0000","last_clean":"2024-09-16T00:11:13.951802+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:11:13.951802+0000","last_undegraded":"2024-09-16T00:11:13.951802+0000","last_fullsized":"2024-09-16T00:11:13.951802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":9206,"num_write":434,"num_write_kb":79714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"122'57","reported_seq":329,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749408+0000","last_change":"2024-09-16T00:10:55.850694+0000","last_active":"2024-09-16T00:11:15.749408+0000","last_peered":"2024-09-16T00:11:15.749408+0000","last_clean":"2024-09-16T00:11:15.749408+0000","last_became_active":"2024-09-16T00:10:55.837749+0000","last_became_peered":"2024-09-16T00:10:55.837749+0000","last_unstale":"2024-09-16T00:11:15.749408+0000","last_undegraded":"2024-09-16T00:11:15.749408+0000","last_fullsized":"2024-09-16T00:11:15.749408+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:53.941848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"123'382","reported_seq":1527,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749454+0000","last_change":"2024-09-16T00:11:04.892185+0000","last_active":"2024-09-16T00:11:15.749454+0000","last_peered":"2024-09-16T00:11:15.749454+0000","last_clean":"2024-09-16T00:11:15.749454+0000","last_became_active":"2024-09-16T00:11:04.890897+0000","last_became_peered":"2024-09-16T00:11:04.890897+0000","last_unstale":"2024-09-16T00:11:15.749454+0000","last_undegraded":"2024-09-16T00:11:15.749454+0000","last_fullsized":"2024-09-16T00:11:15.749454+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:18.504239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":8384,"num_write":359,"num_write_kb":50710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":421,"num_bytes_recovered":615552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"122'180","reported_seq":469,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749303+0000","last_change":"2024-09-16T00:11:04.891195+0000","last_active":"2024-09-16T00:11:15.749303+0000","last_peered":"2024-09-16T00:11:15.749303+0000","last_clean":"2024-09-16T00:11:15.749303+0000","last_became_active":"2024-09-16T00:11:04.890722+0000","last_became_peered":"2024-09-16T00:11:04.890722+0000","last_unstale":"2024-09-16T00:11:15.749303+0000","last_undegraded":"2024-09-16T00:11:15.749303+0000","last_fullsized":"2024-09-16T00:11:15.749303+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:36.844666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":4191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":5577662,"num_keys_recovered":51,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"123'420","reported_seq":750,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749807+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:11:15.749807+0000","last_peered":"2024-09-16T00:11:15.749807+0000","last_clean":"2024-09-16T00:11:15.749807+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:11:15.749807+0000","last_undegraded":"2024-09-16T00:11:15.749807+0000","last_fullsized":"2024-09-16T00:11:15.749807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":37740,"num_write":400,"num_write_kb":67449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"122'114","reported_seq":323,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136851+0000","last_change":"2024-09-16T00:10:55.854939+0000","last_active":"2024-09-16T00:11:13.136851+0000","last_peered":"2024-09-16T00:11:13.136851+0000","last_clean":"2024-09-16T00:11:13.136851+0000","last_became_active":"2024-09-16T00:10:55.832841+0000","last_became_peered":"2024-09-16T00:10:55.832841+0000","last_unstale":"2024-09-16T00:11:13.136851+0000","last_undegraded":"2024-09-16T00:11:13.136851+0000","last_fullsized":"2024-09-16T00:11:13.136851+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:06:48.016972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":8303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"123'452","reported_seq":608,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011438+0000","last_change":"2024-09-16T00:10:00.699942+0000","last_active":"2024-09-16T00:11:14.011438+0000","last_peered":"2024-09-16T00:11:14.011438+0000","last_clean":"2024-09-16T00:11:14.011438+0000","last_became_active":"2024-09-16T00:10:00.281972+0000","last_became_peered":"2024-09-16T00:10:00.281972+0000","last_unstale":"2024-09-16T00:11:14.011438+0000","last_undegraded":"2024-09-16T00:11:14.011438+0000","last_fullsized":"2024-09-16T00:11:14.011438+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:21.943709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":43345,"num_write":425,"num_write_kb":121191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":29546741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"123'396","reported_seq":725,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952254+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:11:13.952254+0000","last_peered":"2024-09-16T00:11:13.952254+0000","last_clean":"2024-09-16T00:11:13.952254+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:11:13.952254+0000","last_undegraded":"2024-09-16T00:11:13.952254+0000","last_fullsized":"2024-09-16T00:11:13.952254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":32882,"num_write":368,"num_write_kb":108251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"123'219","reported_seq":436,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136812+0000","last_change":"2024-09-16T00:10:55.835894+0000","last_active":"2024-09-16T00:11:13.136812+0000","last_peered":"2024-09-16T00:11:13.136812+0000","last_clean":"2024-09-16T00:11:13.136812+0000","last_became_active":"2024-09-16T00:10:55.835301+0000","last_became_peered":"2024-09-16T00:10:55.835301+0000","last_unstale":"2024-09-16T00:11:13.136812+0000","last_undegraded":"2024-09-16T00:11:13.136812+0000","last_fullsized":"2024-09-16T00:11:13.136812+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:32.623825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":4194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"123'385","reported_seq":714,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951587+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:11:13.951587+0000","last_peered":"2024-09-16T00:11:13.951587+0000","last_clean":"2024-09-16T00:11:13.951587+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:11:13.951587+0000","last_undegraded":"2024-09-16T00:11:13.951587+0000","last_fullsized":"2024-09-16T00:11:13.951587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":36278,"num_write":367,"num_write_kb":113361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"123'263","reported_seq":492,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136841+0000","last_change":"2024-09-16T00:10:55.856431+0000","last_active":"2024-09-16T00:11:13.136841+0000","last_peered":"2024-09-16T00:11:13.136841+0000","last_clean":"2024-09-16T00:11:13.136841+0000","last_became_active":"2024-09-16T00:10:55.843843+0000","last_became_peered":"2024-09-16T00:10:55.843843+0000","last_unstale":"2024-09-16T00:11:13.136841+0000","last_undegraded":"2024-09-16T00:11:13.136841+0000","last_fullsized":"2024-09-16T00:11:13.136841+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:11.989047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":197,"num_write_kb":12428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":30,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"123'348","reported_seq":601,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952826+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:11:13.952826+0000","last_peered":"2024-09-16T00:11:13.952826+0000","last_clean":"2024-09-16T00:11:13.952826+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:11:13.952826+0000","last_undegraded":"2024-09-16T00:11:13.952826+0000","last_fullsized":"2024-09-16T00:11:13.952826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":12352,"num_write":323,"num_write_kb":38097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"122'93","reported_seq":356,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.750053+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:11:15.750053+0000","last_peered":"2024-09-16T00:11:15.750053+0000","last_clean":"2024-09-16T00:11:15.750053+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:11:15.750053+0000","last_undegraded":"2024-09-16T00:11:15.750053+0000","last_fullsized":"2024-09-16T00:11:15.750053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":4141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"123'414","reported_seq":795,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749958+0000","last_change":"2024-09-16T00:10:00.556848+0000","last_active":"2024-09-16T00:11:15.749958+0000","last_peered":"2024-09-16T00:11:15.749958+0000","last_clean":"2024-09-16T00:11:15.749958+0000","last_became_active":"2024-09-16T00:10:00.283734+0000","last_became_peered":"2024-09-16T00:10:00.283734+0000","last_unstale":"2024-09-16T00:11:15.749958+0000","last_undegraded":"2024-09-16T00:11:15.749958+0000","last_fullsized":"2024-09-16T00:11:15.749958+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:55:01.843183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":40163,"num_write":392,"num_write_kb":81063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":21131701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"123'98","reported_seq":372,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381221+0000","last_change":"2024-09-16T00:10:55.856802+0000","last_active":"2024-09-16T00:11:14.381221+0000","last_peered":"2024-09-16T00:11:14.381221+0000","last_clean":"2024-09-16T00:11:14.381221+0000","last_became_active":"2024-09-16T00:10:55.844922+0000","last_became_peered":"2024-09-16T00:10:55.844922+0000","last_unstale":"2024-09-16T00:11:14.381221+0000","last_undegraded":"2024-09-16T00:11:14.381221+0000","last_fullsized":"2024-09-16T00:11:14.381221+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:19.134467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":4138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2561,"num_read_kb":1362012,"num_write":24417,"num_write_kb":4088607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3177,"num_bytes_recovered":294487447,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5234,"internal_metadata":0},"log_size":25917,"ondisk_log_size":25917,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":129048453,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":176,"num_read_kb":1534,"num_write":5618,"num_write_kb":131059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":26562382,"num_keys_recovered":3117,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86642688,"data_stored":258096906,"data_compressed":38739888,"data_compressed_allocated":49811456,"data_compressed_original":221323264,"omap_allocated":262411,"internal_metadata":0},"log_size":7144,"ondisk_log_size":7144,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":127,"seq":545460846598,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57992,"kb_used_data":16044,"kb_used_omap":42,"kb_used_meta":41877,"kb_avail":94313848,"statfs":{"total":96636764160,"available":96577380352,"internally_reserved":0,"allocated":16429056,"data_stored":51762927,"data_compressed":7683094,"data_compressed_allocated":9629696,"data_compressed_original":45862912,"omap_allocated":43696,"internal_metadata":42882384},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":94,"seq":403726925841,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56032,"kb_used_data":14280,"kb_used_omap":221,"kb_used_meta":41506,"kb_avail":94315808,"statfs":{"total":96636764160,"available":96579387392,"internally_reserved":0,"allocated":14622720,"data_stored":44546809,"data_compressed":6872875,"data_compressed_allocated":8704000,"data_compressed_original":39534592,"omap_allocated":227133,"internal_metadata":42502339},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73848,"kb_used_data":12128,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94297992,"statfs":{"total":96636764160,"available":96561143808,"internally_reserved":0,"allocated":12419072,"data_stored":26012395,"data_compressed":3584779,"data_compressed_allocated":4849664,"data_compressed_original":19345408,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509492,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63496,"kb_used_data":18412,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94308344,"statfs":{"total":96636764160,"available":96571744256,"internally_reserved":0,"allocated":18853888,"data_stored":46983951,"data_compressed":6735741,"data_compressed_allocated":8769536,"data_compressed_original":37810176,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51048,"kb_used_data":11540,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320792,"statfs":{"total":96636764160,"available":96584491008,"internally_reserved":0,"allocated":11816960,"data_stored":33425144,"data_compressed":5001493,"data_compressed_allocated":6361088,"data_compressed_original":28876800,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542195,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67128,"kb_used_data":9564,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94304712,"statfs":{"total":96636764160,"available":96568025088,"internally_reserved":0,"allocated":9793536,"data_stored":26006313,"data_compressed":3864818,"data_compressed_allocated":4833280,"data_compressed_original":21946368,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542195,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57372,"kb_used_data":12292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94314468,"statfs":{"total":96636764160,"available":96578015232,"internally_reserved":0,"allocated":12587008,"data_stored":30797615,"data_compressed":4284061,"data_compressed_allocated":5672960,"data_compressed_original":24797184,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35280,"kb_used_data":4088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336560,"statfs":{"total":96636764160,"available":96600637440,"internally_reserved":0,"allocated":4186112,"data_stored":6502032,"data_compressed":777781,"data_compressed_allocated":1081344,"data_compressed_original":4296704,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10776576,"data_stored":29360766,"data_compressed":4251684,"data_compressed_allocated":5627904,"data_compressed_original":24223744,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8060928,"data_stored":25165984,"data_compressed":3864818,"data_compressed_allocated":4833280,"data_compressed_original":21946368,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2441216,"data_stored":5655551,"data_compressed":777781,"data_compressed_allocated":1081344,"data_compressed_original":4296704,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10072064,"data_stored":32578663,"data_compressed":5001493,"data_compressed_allocated":6361088,"data_compressed_original":28876800,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17108992,"data_stored":46137470,"data_compressed":6735741,"data_compressed_allocated":8769536,"data_compressed_original":37810176,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10674176,"data_stored":25165914,"data_compressed":3584779,"data_compressed_allocated":4849664,"data_compressed_original":19345408,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12877824,"data_stored":43700328,"data_compressed":6872875,"data_compressed_allocated":8704000,"data_compressed_original":39534592,"omap_allocated":225539,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14630912,"data_stored":50332230,"data_compressed":7650717,"data_compressed_allocated":9584640,"data_compressed_original":45289472,"omap_allocated":36872,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5234,"internal_metadata":0}]}} 2024-09-16T00:11:19.722 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:11:19.858 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.857+0000 7febcc065640 1 Processor -- start 2024-09-16T00:11:19.859 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.858+0000 7febcc065640 1 -- start start 2024-09-16T00:11:19.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.859+0000 7febcc065640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febc415b1c0 0x7febc40c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:19.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.859+0000 7febcc065640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febc415b5c0 con 0x7febc40caf20 2024-09-16T00:11:19.860 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.859+0000 7febcc065640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febc40c98b0 con 0x7febc4154980 2024-09-16T00:11:19.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.859+0000 7febcc065640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7febc40c9b90 con 0x7febc415b1c0 2024-09-16T00:11:19.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.859+0000 7febca602640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7febc40caf20 0x7febc40cb3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45716/0 (socket says 172.21.15.139:45716) 2024-09-16T00:11:19.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.859+0000 7febca602640 1 -- 172.21.15.139:0/3501792670 learned_addr learned my addr 172.21.15.139:0/3501792670 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:19.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.859+0000 7febc9600640 1 --2- 172.21.15.139:0/3501792670 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febc415b1c0 0x7febc40c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:19.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.859+0000 7febc8dff640 1 -- 172.21.15.139:0/3501792670 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2776061984 0 0) 0x7febc415b5c0 con 0x7febc40caf20 2024-09-16T00:11:19.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.859+0000 7febc8dff640 1 -- 172.21.15.139:0/3501792670 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febb0003520 con 0x7febc40caf20 2024-09-16T00:11:19.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.859+0000 7febc8dff640 1 -- 172.21.15.139:0/3501792670 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2326216425 0 0) 0x7febc40c98b0 con 0x7febc4154980 2024-09-16T00:11:19.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.859+0000 7febc8dff640 1 -- 172.21.15.139:0/3501792670 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febc415b5c0 con 0x7febc4154980 2024-09-16T00:11:19.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.859+0000 7febc9600640 1 -- 172.21.15.139:0/3501792670 >> v1:172.21.15.164:6789/0 conn(0x7febc4154980 legacy=0x7febc40ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:19.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.859+0000 7febc8dff640 1 -- 172.21.15.139:0/3501792670 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 401384926 0 0) 0x7febb0003520 con 0x7febc40caf20 2024-09-16T00:11:19.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.859+0000 7febc9600640 1 -- 172.21.15.139:0/3501792670 >> v1:172.21.15.139:6789/0 conn(0x7febc40caf20 legacy=0x7febc40cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:19.861 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.859+0000 7febc9600640 1 -- 172.21.15.139:0/3501792670 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc4070c50 con 0x7febc415b1c0 2024-09-16T00:11:19.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.860+0000 7febc9600640 1 --2- 172.21.15.139:0/3501792670 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febc415b1c0 0x7febc40c8cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7febb40097d0 tx=0x7febb402a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96964ce270cd8da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.860+0000 7febc8dff640 1 -- 172.21.15.139:0/3501792670 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4037070 con 0x7febc415b1c0 2024-09-16T00:11:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.860+0000 7febc8dff640 1 -- 172.21.15.139:0/3501792670 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febb402acc0 con 0x7febc415b1c0 2024-09-16T00:11:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.860+0000 7febc8dff640 1 -- 172.21.15.139:0/3501792670 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc001670 con 0x7febc415b1c0 2024-09-16T00:11:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.861+0000 7febcc065640 1 -- 172.21.15.139:0/3501792670 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febc415b1c0 msgr2=0x7febc40c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.861+0000 7febcc065640 1 --2- 172.21.15.139:0/3501792670 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febc415b1c0 0x7febc40c8cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7febb40097d0 tx=0x7febb402a230 comp rx=0 tx=0).stop 2024-09-16T00:11:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.861+0000 7febcc065640 1 -- 172.21.15.139:0/3501792670 shutdown_connections 2024-09-16T00:11:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.861+0000 7febcc065640 1 --2- 172.21.15.139:0/3501792670 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febc415b1c0 0x7febc40c8cc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.861+0000 7febcc065640 1 -- 172.21.15.139:0/3501792670 >> 172.21.15.139:0/3501792670 conn(0x7febc4067d70 msgr2=0x7febc4068180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.861+0000 7febcc065640 1 -- 172.21.15.139:0/3501792670 shutdown_connections 2024-09-16T00:11:19.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.861+0000 7febcc065640 1 -- 172.21.15.139:0/3501792670 wait complete. 2024-09-16T00:11:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.862+0000 7febcc065640 1 Processor -- start 2024-09-16T00:11:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.862+0000 7febcc065640 1 -- start start 2024-09-16T00:11:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.863+0000 7febcc065640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febc40caf20 0x7febc4156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.863+0000 7febcc065640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febc4154980 0x7febc41572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.863+0000 7febcc065640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febc415b1c0 0x7febc4153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.863+0000 7febcc065640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7febc4154ee0 con 0x7febc40caf20 2024-09-16T00:11:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.863+0000 7febcc065640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7febc4154d60 con 0x7febc4154980 2024-09-16T00:11:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.863+0000 7febcc065640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7febc40ceeb0 con 0x7febc415b1c0 2024-09-16T00:11:19.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.863+0000 7febc9e01640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febc40caf20 0x7febc4156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.863+0000 7febc9e01640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febc40caf20 0x7febc4156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43416/0 (socket says 172.21.15.139:43416) 2024-09-16T00:11:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.863+0000 7febc9e01640 1 -- 172.21.15.139:0/877761830 learned_addr learned my addr 172.21.15.139:0/877761830 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.863+0000 7febc9600640 1 --2- 172.21.15.139:0/877761830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febc4154980 0x7febc41572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.863+0000 7febca602640 1 --2- 172.21.15.139:0/877761830 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febc415b1c0 0x7febc4153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.864+0000 7febc9e01640 1 -- 172.21.15.139:0/877761830 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febc415b1c0 msgr2=0x7febc4153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.864+0000 7febc9e01640 1 --2- 172.21.15.139:0/877761830 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febc415b1c0 0x7febc4153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.864+0000 7febc9e01640 1 -- 172.21.15.139:0/877761830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febc4154980 msgr2=0x7febc41572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.864+0000 7febc9e01640 1 --2- 172.21.15.139:0/877761830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febc4154980 0x7febc41572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.864+0000 7febc9e01640 1 -- 172.21.15.139:0/877761830 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc4155d10 con 0x7febc40caf20 2024-09-16T00:11:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.864+0000 7febc9e01640 1 --2- 172.21.15.139:0/877761830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febc40caf20 0x7febc4156d60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7febc000c800 tx=0x7febc000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.864+0000 7febba7fc640 1 -- 172.21.15.139:0/877761830 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0003b70 con 0x7febc40caf20 2024-09-16T00:11:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.864+0000 7febba7fc640 1 -- 172.21.15.139:0/877761830 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc0003d10 con 0x7febc40caf20 2024-09-16T00:11:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.864+0000 7febcc065640 1 -- 172.21.15.139:0/877761830 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc40af900 con 0x7febc40caf20 2024-09-16T00:11:19.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.864+0000 7febba7fc640 1 -- 172.21.15.139:0/877761830 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc000aa90 con 0x7febc40caf20 2024-09-16T00:11:19.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.864+0000 7febcc065640 1 -- 172.21.15.139:0/877761830 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc40afec0 con 0x7febc40caf20 2024-09-16T00:11:19.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.865+0000 7febcc065640 1 -- 172.21.15.139:0/877761830 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc40ca180 con 0x7febc40caf20 2024-09-16T00:11:19.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.865+0000 7febba7fc640 1 -- 172.21.15.139:0/877761830 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7febc000ac30 con 0x7febc40caf20 2024-09-16T00:11:19.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.866+0000 7febba7fc640 1 --2- 172.21.15.139:0/877761830 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7feba806c2b0 0x7feba806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:19.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.866+0000 7febba7fc640 1 -- 172.21.15.139:0/877761830 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7febc008b340 con 0x7febc40caf20 2024-09-16T00:11:19.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.866+0000 7febc9600640 1 --2- 172.21.15.139:0/877761830 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7feba806c2b0 0x7feba806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:19.869 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.866+0000 7febc9600640 1 --2- 172.21.15.139:0/877761830 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7feba806c2b0 0x7feba806e770 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7febb4034000 tx=0x7febb402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:19.871 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:19.870+0000 7febba7fc640 1 -- 172.21.15.139:0/877761830 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7febc0059400 con 0x7febc40caf20 2024-09-16T00:11:20.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.036+0000 7febcc065640 1 -- 172.21.15.139:0/877761830 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7febc40ced00 con 0x7feba806c2b0 2024-09-16T00:11:20.042 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.042+0000 7febba7fc640 1 -- 172.21.15.139:0/877761830 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254583 (secure 0 0 0) 0x7febc40ced00 con 0x7feba806c2b0 2024-09-16T00:11:20.045 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:11:20.047 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:11:20.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.045+0000 7febcc065640 1 -- 172.21.15.139:0/877761830 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7feba806c2b0 msgr2=0x7feba806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:20.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.045+0000 7febcc065640 1 --2- 172.21.15.139:0/877761830 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7feba806c2b0 0x7feba806e770 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7febb4034000 tx=0x7febb402e000 comp rx=0 tx=0).stop 2024-09-16T00:11:20.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.045+0000 7febcc065640 1 -- 172.21.15.139:0/877761830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febc40caf20 msgr2=0x7febc4156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:20.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.045+0000 7febcc065640 1 --2- 172.21.15.139:0/877761830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febc40caf20 0x7febc4156d60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7febc000c800 tx=0x7febc000ccc0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.045+0000 7febcc065640 1 -- 172.21.15.139:0/877761830 shutdown_connections 2024-09-16T00:11:20.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.045+0000 7febcc065640 1 --2- 172.21.15.139:0/877761830 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7feba806c2b0 0x7feba806e770 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.045+0000 7febcc065640 1 --2- 172.21.15.139:0/877761830 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7febc415b1c0 0x7febc4153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.045+0000 7febcc065640 1 --2- 172.21.15.139:0/877761830 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7febc4154980 0x7febc41572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.045+0000 7febcc065640 1 --2- 172.21.15.139:0/877761830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7febc40caf20 0x7febc4156d60 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.045+0000 7febcc065640 1 -- 172.21.15.139:0/877761830 >> 172.21.15.139:0/877761830 conn(0x7febc4067d70 msgr2=0x7febc40c89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:20.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.045+0000 7febcc065640 1 -- 172.21.15.139:0/877761830 shutdown_connections 2024-09-16T00:11:20.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.045+0000 7febcc065640 1 -- 172.21.15.139:0/877761830 wait complete. 2024-09-16T00:11:20.057 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-09-16T00:11:18.066845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":129638821,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":2853,"num_read_kb":1363646,"num_write":30194,"num_write_kb":4222082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3232,"num_bytes_recovered":321049829,"num_keys_recovered":3117,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33178,"ondisk_log_size":33178,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":462196,"kb_used_data":98348,"kb_used_omap":273,"kb_used_meta":363374,"kb_avail":754512524,"statfs":{"total":773094113280,"available":772620824576,"internally_reserved":0,"allocated":100708352,"data_stored":266037186,"data_compressed":38804642,"data_compressed_allocated":49901568,"data_compressed_original":222470144,"omap_allocated":280397,"internal_metadata":372095155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.134232"},"pg_stats":[{"pgid":"3.3f","version":"123'434","reported_seq":867,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137191+0000","last_change":"2024-09-16T00:10:56.416283+0000","last_active":"2024-09-16T00:11:13.137191+0000","last_peered":"2024-09-16T00:11:13.137191+0000","last_clean":"2024-09-16T00:11:13.137191+0000","last_became_active":"2024-09-16T00:10:55.845182+0000","last_became_peered":"2024-09-16T00:10:55.845182+0000","last_unstale":"2024-09-16T00:11:13.137191+0000","last_undegraded":"2024-09-16T00:11:13.137191+0000","last_fullsized":"2024-09-16T00:11:13.137191+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:52.137388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":42086,"num_write":409,"num_write_kb":112716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":31107821,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"123'417","reported_seq":1093,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749522+0000","last_change":"2024-09-16T00:11:04.897161+0000","last_active":"2024-09-16T00:11:15.749522+0000","last_peered":"2024-09-16T00:11:15.749522+0000","last_clean":"2024-09-16T00:11:15.749522+0000","last_became_active":"2024-09-16T00:11:04.894999+0000","last_became_peered":"2024-09-16T00:11:04.894999+0000","last_unstale":"2024-09-16T00:11:15.749522+0000","last_undegraded":"2024-09-16T00:11:15.749522+0000","last_fullsized":"2024-09-16T00:11:15.749522+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:13.015652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":23064,"num_write":397,"num_write_kb":47220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":689591,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"123'367","reported_seq":657,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381045+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:11:14.381045+0000","last_peered":"2024-09-16T00:11:14.381045+0000","last_clean":"2024-09-16T00:11:14.381045+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:11:14.381045+0000","last_undegraded":"2024-09-16T00:11:14.381045+0000","last_fullsized":"2024-09-16T00:11:14.381045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15782,"num_write":349,"num_write_kb":54932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"123'389","reported_seq":679,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951475+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:11:13.951475+0000","last_peered":"2024-09-16T00:11:13.951475+0000","last_clean":"2024-09-16T00:11:13.951475+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:11:13.951475+0000","last_undegraded":"2024-09-16T00:11:13.951475+0000","last_fullsized":"2024-09-16T00:11:13.951475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":14080,"num_write":364,"num_write_kb":63191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"123'445","reported_seq":748,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749287+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:11:15.749287+0000","last_peered":"2024-09-16T00:11:15.749287+0000","last_clean":"2024-09-16T00:11:15.749287+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:11:15.749287+0000","last_undegraded":"2024-09-16T00:11:15.749287+0000","last_fullsized":"2024-09-16T00:11:15.749287+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":36096,"num_write":427,"num_write_kb":89616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"123'384","reported_seq":1352,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952787+0000","last_change":"2024-09-16T00:10:45.795247+0000","last_active":"2024-09-16T00:11:13.952787+0000","last_peered":"2024-09-16T00:11:13.952787+0000","last_clean":"2024-09-16T00:11:13.952787+0000","last_became_active":"2024-09-16T00:10:45.793434+0000","last_became_peered":"2024-09-16T00:10:45.793434+0000","last_unstale":"2024-09-16T00:11:13.952787+0000","last_undegraded":"2024-09-16T00:11:13.952787+0000","last_fullsized":"2024-09-16T00:11:13.952787+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:55.019829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":6174,"num_write":362,"num_write_kb":50827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":425,"num_bytes_recovered":23619955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"123'420","reported_seq":779,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136743+0000","last_change":"2024-09-16T00:10:56.030536+0000","last_active":"2024-09-16T00:11:13.136743+0000","last_peered":"2024-09-16T00:11:13.136743+0000","last_clean":"2024-09-16T00:11:13.136743+0000","last_became_active":"2024-09-16T00:10:55.835516+0000","last_became_peered":"2024-09-16T00:10:55.835516+0000","last_unstale":"2024-09-16T00:11:13.136743+0000","last_undegraded":"2024-09-16T00:11:13.136743+0000","last_fullsized":"2024-09-16T00:11:13.136743+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:43.152015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":16577,"num_write":401,"num_write_kb":64470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"123'374","reported_seq":641,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952597+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:11:13.952597+0000","last_peered":"2024-09-16T00:11:13.952597+0000","last_clean":"2024-09-16T00:11:13.952597+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:11:13.952597+0000","last_undegraded":"2024-09-16T00:11:13.952597+0000","last_fullsized":"2024-09-16T00:11:13.952597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":16776,"num_write":347,"num_write_kb":80164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"123'394","reported_seq":724,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137150+0000","last_change":"2024-09-16T00:10:56.261956+0000","last_active":"2024-09-16T00:11:13.137150+0000","last_peered":"2024-09-16T00:11:13.137150+0000","last_clean":"2024-09-16T00:11:13.137150+0000","last_became_active":"2024-09-16T00:10:55.840604+0000","last_became_peered":"2024-09-16T00:10:55.840604+0000","last_unstale":"2024-09-16T00:11:13.137150+0000","last_undegraded":"2024-09-16T00:11:13.137150+0000","last_fullsized":"2024-09-16T00:11:13.137150+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:39.279660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1432,"num_write":368,"num_write_kb":30311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"123'367","reported_seq":536,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011843+0000","last_change":"2024-09-16T00:10:00.734336+0000","last_active":"2024-09-16T00:11:14.011843+0000","last_peered":"2024-09-16T00:11:14.011843+0000","last_clean":"2024-09-16T00:11:14.011843+0000","last_became_active":"2024-09-16T00:10:00.284452+0000","last_became_peered":"2024-09-16T00:10:00.284452+0000","last_unstale":"2024-09-16T00:11:14.011843+0000","last_undegraded":"2024-09-16T00:11:14.011843+0000","last_fullsized":"2024-09-16T00:11:14.011843+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:56:50.530439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4096,"num_write":335,"num_write_kb":29952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":8736255,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"123'355","reported_seq":535,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011551+0000","last_change":"2024-09-16T00:10:00.587110+0000","last_active":"2024-09-16T00:11:14.011551+0000","last_peered":"2024-09-16T00:11:14.011551+0000","last_clean":"2024-09-16T00:11:14.011551+0000","last_became_active":"2024-09-16T00:10:00.280273+0000","last_became_peered":"2024-09-16T00:10:00.280273+0000","last_unstale":"2024-09-16T00:11:14.011551+0000","last_undegraded":"2024-09-16T00:11:14.011551+0000","last_fullsized":"2024-09-16T00:11:14.011551+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:01:57.902939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":975,"num_write":339,"num_write_kb":25518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":10753732,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"123'366","reported_seq":705,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952011+0000","last_change":"2024-09-16T00:10:00.797372+0000","last_active":"2024-09-16T00:11:13.952011+0000","last_peered":"2024-09-16T00:11:13.952011+0000","last_clean":"2024-09-16T00:11:13.952011+0000","last_became_active":"2024-09-16T00:10:00.278765+0000","last_became_peered":"2024-09-16T00:10:00.278765+0000","last_unstale":"2024-09-16T00:11:13.952011+0000","last_undegraded":"2024-09-16T00:11:13.952011+0000","last_fullsized":"2024-09-16T00:11:13.952011+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:03.318919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":11563,"num_write":348,"num_write_kb":58933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":29565877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"123'419","reported_seq":784,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749693+0000","last_change":"2024-09-16T00:10:00.715647+0000","last_active":"2024-09-16T00:11:15.749693+0000","last_peered":"2024-09-16T00:11:15.749693+0000","last_clean":"2024-09-16T00:11:15.749693+0000","last_became_active":"2024-09-16T00:10:00.286050+0000","last_became_peered":"2024-09-16T00:10:00.286050+0000","last_unstale":"2024-09-16T00:11:15.749693+0000","last_undegraded":"2024-09-16T00:11:15.749693+0000","last_fullsized":"2024-09-16T00:11:15.749693+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:07.462899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":35908,"num_write":393,"num_write_kb":104863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":30230864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"123'415","reported_seq":724,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.380880+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:11:14.380880+0000","last_peered":"2024-09-16T00:11:14.380880+0000","last_clean":"2024-09-16T00:11:14.380880+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:11:14.380880+0000","last_undegraded":"2024-09-16T00:11:14.380880+0000","last_fullsized":"2024-09-16T00:11:14.380880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":23364,"num_write":390,"num_write_kb":43321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"123'392","reported_seq":684,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749312+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:11:15.749312+0000","last_peered":"2024-09-16T00:11:15.749312+0000","last_clean":"2024-09-16T00:11:15.749312+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:11:15.749312+0000","last_undegraded":"2024-09-16T00:11:15.749312+0000","last_fullsized":"2024-09-16T00:11:15.749312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":14230,"num_write":362,"num_write_kb":33768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"123'439","reported_seq":756,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951004+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:11:13.951004+0000","last_peered":"2024-09-16T00:11:13.951004+0000","last_clean":"2024-09-16T00:11:13.951004+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:11:13.951004+0000","last_undegraded":"2024-09-16T00:11:13.951004+0000","last_fullsized":"2024-09-16T00:11:13.951004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":40234,"num_write":414,"num_write_kb":67785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"123'394","reported_seq":890,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952446+0000","last_change":"2024-09-16T00:10:56.577752+0000","last_active":"2024-09-16T00:11:13.952446+0000","last_peered":"2024-09-16T00:11:13.952446+0000","last_clean":"2024-09-16T00:11:13.952446+0000","last_became_active":"2024-09-16T00:10:55.845880+0000","last_became_peered":"2024-09-16T00:10:55.845880+0000","last_unstale":"2024-09-16T00:11:13.952446+0000","last_undegraded":"2024-09-16T00:11:13.952446+0000","last_fullsized":"2024-09-16T00:11:13.952446+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:32.037945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":22524,"num_write":374,"num_write_kb":46842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"123'391","reported_seq":720,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952778+0000","last_change":"2024-09-16T00:10:00.461437+0000","last_active":"2024-09-16T00:11:13.952778+0000","last_peered":"2024-09-16T00:11:13.952778+0000","last_clean":"2024-09-16T00:11:13.952778+0000","last_became_active":"2024-09-16T00:10:00.282592+0000","last_became_peered":"2024-09-16T00:10:00.282592+0000","last_unstale":"2024-09-16T00:11:13.952778+0000","last_undegraded":"2024-09-16T00:11:13.952778+0000","last_fullsized":"2024-09-16T00:11:13.952778+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:09.733353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8711,"num_write":366,"num_write_kb":46855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":21285626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"123'417","reported_seq":825,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137126+0000","last_change":"2024-09-16T00:10:56.142127+0000","last_active":"2024-09-16T00:11:13.137126+0000","last_peered":"2024-09-16T00:11:13.137126+0000","last_clean":"2024-09-16T00:11:13.137126+0000","last_became_active":"2024-09-16T00:10:55.832595+0000","last_became_peered":"2024-09-16T00:10:55.832595+0000","last_unstale":"2024-09-16T00:11:13.137126+0000","last_undegraded":"2024-09-16T00:11:13.137126+0000","last_fullsized":"2024-09-16T00:11:13.137126+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:44:56.072997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":14437,"num_write":396,"num_write_kb":83979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":103497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"123'423","reported_seq":602,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011546+0000","last_change":"2024-09-16T00:10:00.566870+0000","last_active":"2024-09-16T00:11:14.011546+0000","last_peered":"2024-09-16T00:11:14.011546+0000","last_clean":"2024-09-16T00:11:14.011546+0000","last_became_active":"2024-09-16T00:10:00.281258+0000","last_became_peered":"2024-09-16T00:10:00.281258+0000","last_unstale":"2024-09-16T00:11:14.011546+0000","last_undegraded":"2024-09-16T00:11:14.011546+0000","last_fullsized":"2024-09-16T00:11:14.011546+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:29.197743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":15502,"num_write":406,"num_write_kb":30222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":17199573,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"123'404","reported_seq":883,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381025+0000","last_change":"2024-09-16T00:10:56.104046+0000","last_active":"2024-09-16T00:11:14.381025+0000","last_peered":"2024-09-16T00:11:14.381025+0000","last_clean":"2024-09-16T00:11:14.381025+0000","last_became_active":"2024-09-16T00:10:55.835150+0000","last_became_peered":"2024-09-16T00:10:55.835150+0000","last_unstale":"2024-09-16T00:11:14.381025+0000","last_undegraded":"2024-09-16T00:11:14.381025+0000","last_fullsized":"2024-09-16T00:11:14.381025+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:08.552926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":34318,"num_write":385,"num_write_kb":91769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"123'444","reported_seq":918,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951970+0000","last_change":"2024-09-16T00:10:55.939360+0000","last_active":"2024-09-16T00:11:13.951970+0000","last_peered":"2024-09-16T00:11:13.951970+0000","last_clean":"2024-09-16T00:11:13.951970+0000","last_became_active":"2024-09-16T00:10:55.833335+0000","last_became_peered":"2024-09-16T00:10:55.833335+0000","last_unstale":"2024-09-16T00:11:13.951970+0000","last_undegraded":"2024-09-16T00:11:13.951970+0000","last_fullsized":"2024-09-16T00:11:13.951970+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:08:29.270004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":35528,"num_write":422,"num_write_kb":92852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"123'395","reported_seq":683,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749700+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:11:15.749700+0000","last_peered":"2024-09-16T00:11:15.749700+0000","last_clean":"2024-09-16T00:11:15.749700+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:11:15.749700+0000","last_undegraded":"2024-09-16T00:11:15.749700+0000","last_fullsized":"2024-09-16T00:11:15.749700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":13410,"num_write":369,"num_write_kb":58805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"123'421","reported_seq":703,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951092+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:11:13.951092+0000","last_peered":"2024-09-16T00:11:13.951092+0000","last_clean":"2024-09-16T00:11:13.951092+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:11:13.951092+0000","last_undegraded":"2024-09-16T00:11:13.951092+0000","last_fullsized":"2024-09-16T00:11:13.951092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":12682,"num_write":399,"num_write_kb":54869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"123'421","reported_seq":712,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749956+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:11:15.749956+0000","last_peered":"2024-09-16T00:11:15.749956+0000","last_clean":"2024-09-16T00:11:15.749956+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:11:15.749956+0000","last_undegraded":"2024-09-16T00:11:15.749956+0000","last_fullsized":"2024-09-16T00:11:15.749956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":14188,"num_write":408,"num_write_kb":59178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"123'448","reported_seq":862,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011709+0000","last_change":"2024-09-16T00:10:56.267499+0000","last_active":"2024-09-16T00:11:14.011709+0000","last_peered":"2024-09-16T00:11:14.011709+0000","last_clean":"2024-09-16T00:11:14.011709+0000","last_became_active":"2024-09-16T00:10:55.841967+0000","last_became_peered":"2024-09-16T00:10:55.841967+0000","last_unstale":"2024-09-16T00:11:14.011709+0000","last_undegraded":"2024-09-16T00:11:14.011709+0000","last_fullsized":"2024-09-16T00:11:14.011709+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:15.324718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":12330,"num_write":421,"num_write_kb":35146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":12903786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"123'402","reported_seq":1052,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749340+0000","last_change":"2024-09-16T00:11:04.895240+0000","last_active":"2024-09-16T00:11:15.749340+0000","last_peered":"2024-09-16T00:11:15.749340+0000","last_clean":"2024-09-16T00:11:15.749340+0000","last_became_active":"2024-09-16T00:11:04.892794+0000","last_became_peered":"2024-09-16T00:11:04.892794+0000","last_unstale":"2024-09-16T00:11:15.749340+0000","last_undegraded":"2024-09-16T00:11:15.749340+0000","last_fullsized":"2024-09-16T00:11:15.749340+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:44.365114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":21104,"num_write":383,"num_write_kb":75742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":628439,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"123'386","reported_seq":688,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749839+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:11:15.749839+0000","last_peered":"2024-09-16T00:11:15.749839+0000","last_clean":"2024-09-16T00:11:15.749839+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:11:15.749839+0000","last_undegraded":"2024-09-16T00:11:15.749839+0000","last_fullsized":"2024-09-16T00:11:15.749839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":16408,"num_write":366,"num_write_kb":51674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"123'398","reported_seq":717,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381121+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:11:14.381121+0000","last_peered":"2024-09-16T00:11:14.381121+0000","last_clean":"2024-09-16T00:11:14.381121+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:11:14.381121+0000","last_undegraded":"2024-09-16T00:11:14.381121+0000","last_fullsized":"2024-09-16T00:11:14.381121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":33174,"num_write":373,"num_write_kb":66849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"123'486","reported_seq":780,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952852+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:11:13.952852+0000","last_peered":"2024-09-16T00:11:13.952852+0000","last_clean":"2024-09-16T00:11:13.952852+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:11:13.952852+0000","last_undegraded":"2024-09-16T00:11:13.952852+0000","last_fullsized":"2024-09-16T00:11:13.952852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":27231,"num_write":453,"num_write_kb":104771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"123'451","reported_seq":789,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952668+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:11:13.952668+0000","last_peered":"2024-09-16T00:11:13.952668+0000","last_clean":"2024-09-16T00:11:13.952668+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:11:13.952668+0000","last_undegraded":"2024-09-16T00:11:13.952668+0000","last_fullsized":"2024-09-16T00:11:13.952668+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":35655,"num_write":431,"num_write_kb":97245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"123'421","reported_seq":602,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011470+0000","last_change":"2024-09-16T00:10:00.535553+0000","last_active":"2024-09-16T00:11:14.011470+0000","last_peered":"2024-09-16T00:11:14.011470+0000","last_clean":"2024-09-16T00:11:14.011470+0000","last_became_active":"2024-09-16T00:10:00.281378+0000","last_became_peered":"2024-09-16T00:10:00.281378+0000","last_unstale":"2024-09-16T00:11:14.011470+0000","last_undegraded":"2024-09-16T00:11:14.011470+0000","last_fullsized":"2024-09-16T00:11:14.011470+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:49.548709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20395,"num_write":401,"num_write_kb":32731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":22876917,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"123'401","reported_seq":741,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951565+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:11:13.951565+0000","last_peered":"2024-09-16T00:11:13.951565+0000","last_clean":"2024-09-16T00:11:13.951565+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:11:13.951565+0000","last_undegraded":"2024-09-16T00:11:13.951565+0000","last_fullsized":"2024-09-16T00:11:13.951565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":48141,"num_write":381,"num_write_kb":63611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"120'126","reported_seq":402,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749808+0000","last_change":"2024-09-16T00:10:55.854872+0000","last_active":"2024-09-16T00:11:15.749808+0000","last_peered":"2024-09-16T00:11:15.749808+0000","last_clean":"2024-09-16T00:11:15.749808+0000","last_became_active":"2024-09-16T00:10:55.843367+0000","last_became_peered":"2024-09-16T00:10:55.843367+0000","last_unstale":"2024-09-16T00:11:15.749808+0000","last_undegraded":"2024-09-16T00:11:15.749808+0000","last_fullsized":"2024-09-16T00:11:15.749808+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:07.797903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"123'346","reported_seq":633,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381167+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:11:14.381167+0000","last_peered":"2024-09-16T00:11:14.381167+0000","last_clean":"2024-09-16T00:11:14.381167+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:11:14.381167+0000","last_undegraded":"2024-09-16T00:11:14.381167+0000","last_fullsized":"2024-09-16T00:11:14.381167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":16420,"num_write":326,"num_write_kb":62710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"122'340","reported_seq":505,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011382+0000","last_change":"2024-09-16T00:10:00.282958+0000","last_active":"2024-09-16T00:11:14.011382+0000","last_peered":"2024-09-16T00:11:14.011382+0000","last_clean":"2024-09-16T00:11:14.011382+0000","last_became_active":"2024-09-16T00:10:00.282541+0000","last_became_peered":"2024-09-16T00:10:00.282541+0000","last_unstale":"2024-09-16T00:11:14.011382+0000","last_undegraded":"2024-09-16T00:11:14.011382+0000","last_fullsized":"2024-09-16T00:11:14.011382+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:51.198787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":285,"num_write_kb":12471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"123'226","reported_seq":510,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749827+0000","last_change":"2024-09-16T00:10:00.313589+0000","last_active":"2024-09-16T00:11:15.749827+0000","last_peered":"2024-09-16T00:11:15.749827+0000","last_clean":"2024-09-16T00:11:15.749827+0000","last_became_active":"2024-09-16T00:10:00.284602+0000","last_became_peered":"2024-09-16T00:10:00.284602+0000","last_unstale":"2024-09-16T00:11:15.749827+0000","last_undegraded":"2024-09-16T00:11:15.749827+0000","last_fullsized":"2024-09-16T00:11:15.749827+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:30.032698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":14,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"123'433","reported_seq":732,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749206+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:11:15.749206+0000","last_peered":"2024-09-16T00:11:15.749206+0000","last_clean":"2024-09-16T00:11:15.749206+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:11:15.749206+0000","last_undegraded":"2024-09-16T00:11:15.749206+0000","last_fullsized":"2024-09-16T00:11:15.749206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":18480,"num_write":411,"num_write_kb":71748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"123'833","reported_seq":1095,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951164+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:11:13.951164+0000","last_peered":"2024-09-16T00:11:13.951164+0000","last_clean":"2024-09-16T00:11:13.951164+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:11:13.951164+0000","last_undegraded":"2024-09-16T00:11:13.951164+0000","last_fullsized":"2024-09-16T00:11:13.951164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":8591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"123'425","reported_seq":774,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749285+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:11:15.749285+0000","last_peered":"2024-09-16T00:11:15.749285+0000","last_clean":"2024-09-16T00:11:15.749285+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:11:15.749285+0000","last_undegraded":"2024-09-16T00:11:15.749285+0000","last_fullsized":"2024-09-16T00:11:15.749285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":46871,"num_write":401,"num_write_kb":84187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"123'65","reported_seq":296,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011701+0000","last_change":"2024-09-16T00:10:00.283876+0000","last_active":"2024-09-16T00:11:14.011701+0000","last_peered":"2024-09-16T00:11:14.011701+0000","last_clean":"2024-09-16T00:11:14.011701+0000","last_became_active":"2024-09-16T00:10:00.283555+0000","last_became_peered":"2024-09-16T00:10:00.283555+0000","last_unstale":"2024-09-16T00:11:14.011701+0000","last_undegraded":"2024-09-16T00:11:14.011701+0000","last_fullsized":"2024-09-16T00:11:14.011701+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:48.543737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"123'440","reported_seq":904,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951434+0000","last_change":"2024-09-16T00:10:56.502658+0000","last_active":"2024-09-16T00:11:13.951434+0000","last_peered":"2024-09-16T00:11:13.951434+0000","last_clean":"2024-09-16T00:11:13.951434+0000","last_became_active":"2024-09-16T00:10:55.840027+0000","last_became_peered":"2024-09-16T00:10:55.840027+0000","last_unstale":"2024-09-16T00:11:13.951434+0000","last_undegraded":"2024-09-16T00:11:13.951434+0000","last_fullsized":"2024-09-16T00:11:13.951434+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:52.105868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":16686,"num_write":415,"num_write_kb":80112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"123'225","reported_seq":494,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951555+0000","last_change":"2024-09-16T00:10:00.285386+0000","last_active":"2024-09-16T00:11:13.951555+0000","last_peered":"2024-09-16T00:11:13.951555+0000","last_clean":"2024-09-16T00:11:13.951555+0000","last_became_active":"2024-09-16T00:10:00.284717+0000","last_became_peered":"2024-09-16T00:10:00.284717+0000","last_unstale":"2024-09-16T00:11:13.951555+0000","last_undegraded":"2024-09-16T00:11:13.951555+0000","last_fullsized":"2024-09-16T00:11:13.951555+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:07.794163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":4181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"123'378","reported_seq":681,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381204+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:11:14.381204+0000","last_peered":"2024-09-16T00:11:14.381204+0000","last_clean":"2024-09-16T00:11:14.381204+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:11:14.381204+0000","last_undegraded":"2024-09-16T00:11:14.381204+0000","last_fullsized":"2024-09-16T00:11:14.381204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":18030,"num_write":359,"num_write_kb":55428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"125'131","reported_seq":392,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749519+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:11:15.749519+0000","last_peered":"2024-09-16T00:11:15.749519+0000","last_clean":"2024-09-16T00:11:15.749519+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:11:15.749519+0000","last_undegraded":"2024-09-16T00:11:15.749519+0000","last_fullsized":"2024-09-16T00:11:15.749519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"123'373","reported_seq":818,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952022+0000","last_change":"2024-09-16T00:10:56.173134+0000","last_active":"2024-09-16T00:11:13.952022+0000","last_peered":"2024-09-16T00:11:13.952022+0000","last_clean":"2024-09-16T00:11:13.952022+0000","last_became_active":"2024-09-16T00:10:55.833273+0000","last_became_peered":"2024-09-16T00:10:55.833273+0000","last_unstale":"2024-09-16T00:11:13.952022+0000","last_undegraded":"2024-09-16T00:11:13.952022+0000","last_fullsized":"2024-09-16T00:11:13.952022+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:11.402988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":22960,"num_write":348,"num_write_kb":63161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"125'238","reported_seq":414,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011582+0000","last_change":"2024-09-16T00:10:00.300895+0000","last_active":"2024-09-16T00:11:14.011582+0000","last_peered":"2024-09-16T00:11:14.011582+0000","last_clean":"2024-09-16T00:11:14.011582+0000","last_became_active":"2024-09-16T00:10:00.283474+0000","last_became_peered":"2024-09-16T00:10:00.283474+0000","last_unstale":"2024-09-16T00:11:14.011582+0000","last_undegraded":"2024-09-16T00:11:14.011582+0000","last_fullsized":"2024-09-16T00:11:14.011582+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:28:02.563817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10145896,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":186,"num_write_kb":10014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":13200,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"123'403","reported_seq":774,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136869+0000","last_change":"2024-09-16T00:10:56.320793+0000","last_active":"2024-09-16T00:11:13.136869+0000","last_peered":"2024-09-16T00:11:13.136869+0000","last_clean":"2024-09-16T00:11:13.136869+0000","last_became_active":"2024-09-16T00:10:55.836462+0000","last_became_peered":"2024-09-16T00:10:55.836462+0000","last_unstale":"2024-09-16T00:11:13.136869+0000","last_undegraded":"2024-09-16T00:11:13.136869+0000","last_fullsized":"2024-09-16T00:11:13.136869+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:07:47.717822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":23285,"num_write":372,"num_write_kb":67093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"122'470","reported_seq":732,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381212+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:11:14.381212+0000","last_peered":"2024-09-16T00:11:14.381212+0000","last_clean":"2024-09-16T00:11:14.381212+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:11:14.381212+0000","last_undegraded":"2024-09-16T00:11:14.381212+0000","last_fullsized":"2024-09-16T00:11:14.381212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":419,"num_write_kb":8413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"123'404","reported_seq":872,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.380881+0000","last_change":"2024-09-16T00:10:56.427976+0000","last_active":"2024-09-16T00:11:14.380881+0000","last_peered":"2024-09-16T00:11:14.380881+0000","last_clean":"2024-09-16T00:11:14.380881+0000","last_became_active":"2024-09-16T00:10:55.839196+0000","last_became_peered":"2024-09-16T00:10:55.839196+0000","last_unstale":"2024-09-16T00:11:14.380881+0000","last_undegraded":"2024-09-16T00:11:14.380881+0000","last_fullsized":"2024-09-16T00:11:14.380881+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:37.852194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":13263,"num_write":381,"num_write_kb":51174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"123'59","reported_seq":321,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952149+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:11:13.952149+0000","last_peered":"2024-09-16T00:11:13.952149+0000","last_clean":"2024-09-16T00:11:13.952149+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:11:13.952149+0000","last_undegraded":"2024-09-16T00:11:13.952149+0000","last_fullsized":"2024-09-16T00:11:13.952149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"123'394","reported_seq":733,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136975+0000","last_change":"2024-09-16T00:10:56.085067+0000","last_active":"2024-09-16T00:11:13.136975+0000","last_peered":"2024-09-16T00:11:13.136975+0000","last_clean":"2024-09-16T00:11:13.136975+0000","last_became_active":"2024-09-16T00:10:55.834112+0000","last_became_peered":"2024-09-16T00:10:55.834112+0000","last_unstale":"2024-09-16T00:11:13.136975+0000","last_undegraded":"2024-09-16T00:11:13.136975+0000","last_fullsized":"2024-09-16T00:11:13.136975+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:35.278069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":13990,"num_write":368,"num_write_kb":63962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"120'70","reported_seq":218,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011398+0000","last_change":"2024-09-16T00:10:00.294519+0000","last_active":"2024-09-16T00:11:14.011398+0000","last_peered":"2024-09-16T00:11:14.011398+0000","last_clean":"2024-09-16T00:11:14.011398+0000","last_became_active":"2024-09-16T00:10:00.283066+0000","last_became_peered":"2024-09-16T00:10:00.283066+0000","last_unstale":"2024-09-16T00:11:14.011398+0000","last_undegraded":"2024-09-16T00:11:14.011398+0000","last_fullsized":"2024-09-16T00:11:14.011398+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:32.621176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"123'424","reported_seq":607,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011378+0000","last_change":"2024-09-16T00:10:00.460910+0000","last_active":"2024-09-16T00:11:14.011378+0000","last_peered":"2024-09-16T00:11:14.011378+0000","last_clean":"2024-09-16T00:11:14.011378+0000","last_became_active":"2024-09-16T00:10:00.280750+0000","last_became_peered":"2024-09-16T00:10:00.280750+0000","last_unstale":"2024-09-16T00:11:14.011378+0000","last_undegraded":"2024-09-16T00:11:14.011378+0000","last_fullsized":"2024-09-16T00:11:14.011378+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:47.634650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":19573,"num_write":402,"num_write_kb":50826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":33491520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"125'72","reported_seq":334,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952685+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:11:13.952685+0000","last_peered":"2024-09-16T00:11:13.952685+0000","last_clean":"2024-09-16T00:11:13.952685+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:11:13.952685+0000","last_undegraded":"2024-09-16T00:11:13.952685+0000","last_fullsized":"2024-09-16T00:11:13.952685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"123'457","reported_seq":814,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136679+0000","last_change":"2024-09-16T00:10:55.922694+0000","last_active":"2024-09-16T00:11:13.136679+0000","last_peered":"2024-09-16T00:11:13.136679+0000","last_clean":"2024-09-16T00:11:13.136679+0000","last_became_active":"2024-09-16T00:10:55.827989+0000","last_became_peered":"2024-09-16T00:10:55.827989+0000","last_unstale":"2024-09-16T00:11:13.136679+0000","last_undegraded":"2024-09-16T00:11:13.136679+0000","last_fullsized":"2024-09-16T00:11:13.136679+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:23.666346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20419,"num_write":427,"num_write_kb":43035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"123'145","reported_seq":407,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951453+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:11:13.951453+0000","last_peered":"2024-09-16T00:11:13.951453+0000","last_clean":"2024-09-16T00:11:13.951453+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:11:13.951453+0000","last_undegraded":"2024-09-16T00:11:13.951453+0000","last_fullsized":"2024-09-16T00:11:13.951453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91,"num_write_kb":4217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"123'358","reported_seq":694,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137061+0000","last_change":"2024-09-16T00:10:55.971163+0000","last_active":"2024-09-16T00:11:13.137061+0000","last_peered":"2024-09-16T00:11:13.137061+0000","last_clean":"2024-09-16T00:11:13.137061+0000","last_became_active":"2024-09-16T00:10:55.834401+0000","last_became_peered":"2024-09-16T00:10:55.834401+0000","last_unstale":"2024-09-16T00:11:13.137061+0000","last_undegraded":"2024-09-16T00:11:13.137061+0000","last_fullsized":"2024-09-16T00:11:13.137061+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:05.466545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":10790,"num_write":330,"num_write_kb":21585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"120'282","reported_seq":544,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952893+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:11:13.952893+0000","last_peered":"2024-09-16T00:11:13.952893+0000","last_clean":"2024-09-16T00:11:13.952893+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:11:13.952893+0000","last_undegraded":"2024-09-16T00:11:13.952893+0000","last_fullsized":"2024-09-16T00:11:13.952893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":238,"num_write_kb":4238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"123'372","reported_seq":696,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137259+0000","last_change":"2024-09-16T00:10:56.200838+0000","last_active":"2024-09-16T00:11:13.137259+0000","last_peered":"2024-09-16T00:11:13.137259+0000","last_clean":"2024-09-16T00:11:13.137259+0000","last_became_active":"2024-09-16T00:10:55.838254+0000","last_became_peered":"2024-09-16T00:10:55.838254+0000","last_unstale":"2024-09-16T00:11:13.137259+0000","last_undegraded":"2024-09-16T00:11:13.137259+0000","last_fullsized":"2024-09-16T00:11:13.137259+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:30.944542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":350,"num_write":351,"num_write_kb":38221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"122'534","reported_seq":795,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749610+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:11:15.749610+0000","last_peered":"2024-09-16T00:11:15.749610+0000","last_clean":"2024-09-16T00:11:15.749610+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:11:15.749610+0000","last_undegraded":"2024-09-16T00:11:15.749610+0000","last_fullsized":"2024-09-16T00:11:15.749610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":473,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"123'418","reported_seq":682,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951696+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:11:13.951696+0000","last_peered":"2024-09-16T00:11:13.951696+0000","last_clean":"2024-09-16T00:11:13.951696+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:11:13.951696+0000","last_undegraded":"2024-09-16T00:11:13.951696+0000","last_fullsized":"2024-09-16T00:11:13.951696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":675,"num_write":389,"num_write_kb":34696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"123'72","reported_seq":334,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952967+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:11:13.952967+0000","last_peered":"2024-09-16T00:11:13.952967+0000","last_clean":"2024-09-16T00:11:13.952967+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:11:13.952967+0000","last_undegraded":"2024-09-16T00:11:13.952967+0000","last_fullsized":"2024-09-16T00:11:13.952967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"122'397","reported_seq":761,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136957+0000","last_change":"2024-09-16T00:10:56.368023+0000","last_active":"2024-09-16T00:11:13.136957+0000","last_peered":"2024-09-16T00:11:13.136957+0000","last_clean":"2024-09-16T00:11:13.136957+0000","last_became_active":"2024-09-16T00:10:55.839714+0000","last_became_peered":"2024-09-16T00:10:55.839714+0000","last_unstale":"2024-09-16T00:11:13.136957+0000","last_undegraded":"2024-09-16T00:11:13.136957+0000","last_fullsized":"2024-09-16T00:11:13.136957+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:04.192344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":22590,"num_write":379,"num_write_kb":59088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":372,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136608+0000","last_change":"2024-09-16T00:10:55.832444+0000","last_active":"2024-09-16T00:11:13.136608+0000","last_peered":"2024-09-16T00:11:13.136608+0000","last_clean":"2024-09-16T00:11:13.136608+0000","last_became_active":"2024-09-16T00:10:55.831881+0000","last_became_peered":"2024-09-16T00:10:55.831881+0000","last_unstale":"2024-09-16T00:11:13.136608+0000","last_undegraded":"2024-09-16T00:11:13.136608+0000","last_fullsized":"2024-09-16T00:11:13.136608+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:48.243166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"123'63","reported_seq":325,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951198+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:11:13.951198+0000","last_peered":"2024-09-16T00:11:13.951198+0000","last_clean":"2024-09-16T00:11:13.951198+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:11:13.951198+0000","last_undegraded":"2024-09-16T00:11:13.951198+0000","last_fullsized":"2024-09-16T00:11:13.951198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"123'369","reported_seq":664,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749611+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:11:15.749611+0000","last_peered":"2024-09-16T00:11:15.749611+0000","last_clean":"2024-09-16T00:11:15.749611+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:11:15.749611+0000","last_undegraded":"2024-09-16T00:11:15.749611+0000","last_fullsized":"2024-09-16T00:11:15.749611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":16103,"num_write":342,"num_write_kb":46241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"123'399","reported_seq":643,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136617+0000","last_change":"2024-09-16T00:11:04.896119+0000","last_active":"2024-09-16T00:11:13.136617+0000","last_peered":"2024-09-16T00:11:13.136617+0000","last_clean":"2024-09-16T00:11:13.136617+0000","last_became_active":"2024-09-16T00:11:04.895672+0000","last_became_peered":"2024-09-16T00:11:04.895672+0000","last_unstale":"2024-09-16T00:11:13.136617+0000","last_undegraded":"2024-09-16T00:11:13.136617+0000","last_fullsized":"2024-09-16T00:11:13.136617+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:45.173943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":346,"num_write_kb":8359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":16777216,"num_keys_recovered":28,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"123'393","reported_seq":700,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.380785+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:11:14.380785+0000","last_peered":"2024-09-16T00:11:14.380785+0000","last_clean":"2024-09-16T00:11:14.380785+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:11:14.380785+0000","last_undegraded":"2024-09-16T00:11:14.380785+0000","last_fullsized":"2024-09-16T00:11:14.380785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":29385,"num_write":362,"num_write_kb":83610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"125'242","reported_seq":680,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952710+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:11:13.952710+0000","last_peered":"2024-09-16T00:11:13.952710+0000","last_clean":"2024-09-16T00:11:13.952710+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:11:13.952710+0000","last_undegraded":"2024-09-16T00:11:13.952710+0000","last_fullsized":"2024-09-16T00:11:13.952710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461247,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":176,"num_read_kb":1534,"num_write":179,"num_write_kb":1542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"123'400","reported_seq":712,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749491+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:11:15.749491+0000","last_peered":"2024-09-16T00:11:15.749491+0000","last_clean":"2024-09-16T00:11:15.749491+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:11:15.749491+0000","last_undegraded":"2024-09-16T00:11:15.749491+0000","last_fullsized":"2024-09-16T00:11:15.749491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":28288,"num_write":378,"num_write_kb":66949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"119'400","reported_seq":662,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952094+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:11:13.952094+0000","last_peered":"2024-09-16T00:11:13.952094+0000","last_clean":"2024-09-16T00:11:13.952094+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:11:13.952094+0000","last_undegraded":"2024-09-16T00:11:13.952094+0000","last_fullsized":"2024-09-16T00:11:13.952094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":353,"num_write_kb":8381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"123'387","reported_seq":683,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951764+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:11:13.951764+0000","last_peered":"2024-09-16T00:11:13.951764+0000","last_clean":"2024-09-16T00:11:13.951764+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:11:13.951764+0000","last_undegraded":"2024-09-16T00:11:13.951764+0000","last_fullsized":"2024-09-16T00:11:13.951764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":22188,"num_write":360,"num_write_kb":71552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"122'57","reported_seq":318,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749407+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:11:15.749407+0000","last_peered":"2024-09-16T00:11:15.749407+0000","last_clean":"2024-09-16T00:11:15.749407+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:11:15.749407+0000","last_undegraded":"2024-09-16T00:11:15.749407+0000","last_fullsized":"2024-09-16T00:11:15.749407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"123'426","reported_seq":725,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952294+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:11:13.952294+0000","last_peered":"2024-09-16T00:11:13.952294+0000","last_clean":"2024-09-16T00:11:13.952294+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:11:13.952294+0000","last_undegraded":"2024-09-16T00:11:13.952294+0000","last_fullsized":"2024-09-16T00:11:13.952294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":24752,"num_write":390,"num_write_kb":75609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"123'258","reported_seq":301,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.137011+0000","last_change":"2024-09-16T00:10:58.839395+0000","last_active":"2024-09-16T00:11:13.137011+0000","last_peered":"2024-09-16T00:11:13.137011+0000","last_clean":"2024-09-16T00:11:13.137011+0000","last_became_active":"2024-09-16T00:10:58.839124+0000","last_became_peered":"2024-09-16T00:10:58.839124+0000","last_unstale":"2024-09-16T00:11:13.137011+0000","last_undegraded":"2024-09-16T00:11:13.137011+0000","last_fullsized":"2024-09-16T00:11:13.137011+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:29.136640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":202,"num_write_kb":4192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"123'398","reported_seq":715,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749363+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:11:15.749363+0000","last_peered":"2024-09-16T00:11:15.749363+0000","last_clean":"2024-09-16T00:11:15.749363+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:11:15.749363+0000","last_undegraded":"2024-09-16T00:11:15.749363+0000","last_fullsized":"2024-09-16T00:11:15.749363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":31221,"num_write":370,"num_write_kb":76143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"125'186","reported_seq":482,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749569+0000","last_change":"2024-09-16T00:11:04.894640+0000","last_active":"2024-09-16T00:11:15.749569+0000","last_peered":"2024-09-16T00:11:15.749569+0000","last_clean":"2024-09-16T00:11:15.749569+0000","last_became_active":"2024-09-16T00:11:04.894163+0000","last_became_peered":"2024-09-16T00:11:04.894163+0000","last_unstale":"2024-09-16T00:11:15.749569+0000","last_undegraded":"2024-09-16T00:11:15.749569+0000","last_fullsized":"2024-09-16T00:11:15.749569+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:41.395352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":345,"num_write_kb":1559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":2964,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"123'344","reported_seq":786,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.750028+0000","last_change":"2024-09-16T00:10:56.334688+0000","last_active":"2024-09-16T00:11:15.750028+0000","last_peered":"2024-09-16T00:11:15.750028+0000","last_clean":"2024-09-16T00:11:15.750028+0000","last_became_active":"2024-09-16T00:10:55.845451+0000","last_became_peered":"2024-09-16T00:10:55.845451+0000","last_unstale":"2024-09-16T00:11:15.750028+0000","last_undegraded":"2024-09-16T00:11:15.750028+0000","last_fullsized":"2024-09-16T00:11:15.750028+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:04.965027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":19274,"num_write":323,"num_write_kb":22071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"123'63","reported_seq":309,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011696+0000","last_change":"2024-09-16T00:10:55.858845+0000","last_active":"2024-09-16T00:11:14.011696+0000","last_peered":"2024-09-16T00:11:14.011696+0000","last_clean":"2024-09-16T00:11:14.011696+0000","last_became_active":"2024-09-16T00:10:55.845687+0000","last_became_peered":"2024-09-16T00:10:55.845687+0000","last_unstale":"2024-09-16T00:11:14.011696+0000","last_undegraded":"2024-09-16T00:11:14.011696+0000","last_fullsized":"2024-09-16T00:11:14.011696+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:53.322675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"123'447","reported_seq":750,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381154+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:11:14.381154+0000","last_peered":"2024-09-16T00:11:14.381154+0000","last_clean":"2024-09-16T00:11:14.381154+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:11:14.381154+0000","last_undegraded":"2024-09-16T00:11:14.381154+0000","last_fullsized":"2024-09-16T00:11:14.381154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":17894,"num_write":426,"num_write_kb":88843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"123'564","reported_seq":784,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136761+0000","last_change":"2024-09-16T00:10:55.852895+0000","last_active":"2024-09-16T00:11:13.136761+0000","last_peered":"2024-09-16T00:11:13.136761+0000","last_clean":"2024-09-16T00:11:13.136761+0000","last_became_active":"2024-09-16T00:10:55.828121+0000","last_became_peered":"2024-09-16T00:10:55.828121+0000","last_unstale":"2024-09-16T00:11:13.136761+0000","last_undegraded":"2024-09-16T00:11:13.136761+0000","last_fullsized":"2024-09-16T00:11:13.136761+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:57.000428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"123'454","reported_seq":732,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951802+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:11:13.951802+0000","last_peered":"2024-09-16T00:11:13.951802+0000","last_clean":"2024-09-16T00:11:13.951802+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:11:13.951802+0000","last_undegraded":"2024-09-16T00:11:13.951802+0000","last_fullsized":"2024-09-16T00:11:13.951802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":9206,"num_write":434,"num_write_kb":79714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"122'57","reported_seq":329,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749408+0000","last_change":"2024-09-16T00:10:55.850694+0000","last_active":"2024-09-16T00:11:15.749408+0000","last_peered":"2024-09-16T00:11:15.749408+0000","last_clean":"2024-09-16T00:11:15.749408+0000","last_became_active":"2024-09-16T00:10:55.837749+0000","last_became_peered":"2024-09-16T00:10:55.837749+0000","last_unstale":"2024-09-16T00:11:15.749408+0000","last_undegraded":"2024-09-16T00:11:15.749408+0000","last_fullsized":"2024-09-16T00:11:15.749408+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:53.941848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"123'382","reported_seq":1527,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749454+0000","last_change":"2024-09-16T00:11:04.892185+0000","last_active":"2024-09-16T00:11:15.749454+0000","last_peered":"2024-09-16T00:11:15.749454+0000","last_clean":"2024-09-16T00:11:15.749454+0000","last_became_active":"2024-09-16T00:11:04.890897+0000","last_became_peered":"2024-09-16T00:11:04.890897+0000","last_unstale":"2024-09-16T00:11:15.749454+0000","last_undegraded":"2024-09-16T00:11:15.749454+0000","last_fullsized":"2024-09-16T00:11:15.749454+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:18.504239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":8384,"num_write":359,"num_write_kb":50710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":421,"num_bytes_recovered":615552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"122'180","reported_seq":469,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749303+0000","last_change":"2024-09-16T00:11:04.891195+0000","last_active":"2024-09-16T00:11:15.749303+0000","last_peered":"2024-09-16T00:11:15.749303+0000","last_clean":"2024-09-16T00:11:15.749303+0000","last_became_active":"2024-09-16T00:11:04.890722+0000","last_became_peered":"2024-09-16T00:11:04.890722+0000","last_unstale":"2024-09-16T00:11:15.749303+0000","last_undegraded":"2024-09-16T00:11:15.749303+0000","last_fullsized":"2024-09-16T00:11:15.749303+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:36.844666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":4191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":5577662,"num_keys_recovered":51,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"123'420","reported_seq":750,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749807+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:11:15.749807+0000","last_peered":"2024-09-16T00:11:15.749807+0000","last_clean":"2024-09-16T00:11:15.749807+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:11:15.749807+0000","last_undegraded":"2024-09-16T00:11:15.749807+0000","last_fullsized":"2024-09-16T00:11:15.749807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":37740,"num_write":400,"num_write_kb":67449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"122'114","reported_seq":323,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136851+0000","last_change":"2024-09-16T00:10:55.854939+0000","last_active":"2024-09-16T00:11:13.136851+0000","last_peered":"2024-09-16T00:11:13.136851+0000","last_clean":"2024-09-16T00:11:13.136851+0000","last_became_active":"2024-09-16T00:10:55.832841+0000","last_became_peered":"2024-09-16T00:10:55.832841+0000","last_unstale":"2024-09-16T00:11:13.136851+0000","last_undegraded":"2024-09-16T00:11:13.136851+0000","last_fullsized":"2024-09-16T00:11:13.136851+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:06:48.016972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":8303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"123'452","reported_seq":608,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011438+0000","last_change":"2024-09-16T00:10:00.699942+0000","last_active":"2024-09-16T00:11:14.011438+0000","last_peered":"2024-09-16T00:11:14.011438+0000","last_clean":"2024-09-16T00:11:14.011438+0000","last_became_active":"2024-09-16T00:10:00.281972+0000","last_became_peered":"2024-09-16T00:10:00.281972+0000","last_unstale":"2024-09-16T00:11:14.011438+0000","last_undegraded":"2024-09-16T00:11:14.011438+0000","last_fullsized":"2024-09-16T00:11:14.011438+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:21.943709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":43345,"num_write":425,"num_write_kb":121191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":29546741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"123'396","reported_seq":725,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952254+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:11:13.952254+0000","last_peered":"2024-09-16T00:11:13.952254+0000","last_clean":"2024-09-16T00:11:13.952254+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:11:13.952254+0000","last_undegraded":"2024-09-16T00:11:13.952254+0000","last_fullsized":"2024-09-16T00:11:13.952254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":32882,"num_write":368,"num_write_kb":108251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"123'219","reported_seq":436,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136812+0000","last_change":"2024-09-16T00:10:55.835894+0000","last_active":"2024-09-16T00:11:13.136812+0000","last_peered":"2024-09-16T00:11:13.136812+0000","last_clean":"2024-09-16T00:11:13.136812+0000","last_became_active":"2024-09-16T00:10:55.835301+0000","last_became_peered":"2024-09-16T00:10:55.835301+0000","last_unstale":"2024-09-16T00:11:13.136812+0000","last_undegraded":"2024-09-16T00:11:13.136812+0000","last_fullsized":"2024-09-16T00:11:13.136812+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:32.623825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":4194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"123'385","reported_seq":714,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951587+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:11:13.951587+0000","last_peered":"2024-09-16T00:11:13.951587+0000","last_clean":"2024-09-16T00:11:13.951587+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:11:13.951587+0000","last_undegraded":"2024-09-16T00:11:13.951587+0000","last_fullsized":"2024-09-16T00:11:13.951587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":36278,"num_write":367,"num_write_kb":113361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"123'263","reported_seq":492,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.136841+0000","last_change":"2024-09-16T00:10:55.856431+0000","last_active":"2024-09-16T00:11:13.136841+0000","last_peered":"2024-09-16T00:11:13.136841+0000","last_clean":"2024-09-16T00:11:13.136841+0000","last_became_active":"2024-09-16T00:10:55.843843+0000","last_became_peered":"2024-09-16T00:10:55.843843+0000","last_unstale":"2024-09-16T00:11:13.136841+0000","last_undegraded":"2024-09-16T00:11:13.136841+0000","last_fullsized":"2024-09-16T00:11:13.136841+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:11.989047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":197,"num_write_kb":12428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":30,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"123'348","reported_seq":601,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952826+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:11:13.952826+0000","last_peered":"2024-09-16T00:11:13.952826+0000","last_clean":"2024-09-16T00:11:13.952826+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:11:13.952826+0000","last_undegraded":"2024-09-16T00:11:13.952826+0000","last_fullsized":"2024-09-16T00:11:13.952826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":12352,"num_write":323,"num_write_kb":38097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"122'93","reported_seq":356,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.750053+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:11:15.750053+0000","last_peered":"2024-09-16T00:11:15.750053+0000","last_clean":"2024-09-16T00:11:15.750053+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:11:15.750053+0000","last_undegraded":"2024-09-16T00:11:15.750053+0000","last_fullsized":"2024-09-16T00:11:15.750053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":4141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"123'414","reported_seq":795,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749958+0000","last_change":"2024-09-16T00:10:00.556848+0000","last_active":"2024-09-16T00:11:15.749958+0000","last_peered":"2024-09-16T00:11:15.749958+0000","last_clean":"2024-09-16T00:11:15.749958+0000","last_became_active":"2024-09-16T00:10:00.283734+0000","last_became_peered":"2024-09-16T00:10:00.283734+0000","last_unstale":"2024-09-16T00:11:15.749958+0000","last_undegraded":"2024-09-16T00:11:15.749958+0000","last_fullsized":"2024-09-16T00:11:15.749958+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:55:01.843183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":40163,"num_write":392,"num_write_kb":81063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":21131701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"123'98","reported_seq":372,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381221+0000","last_change":"2024-09-16T00:10:55.856802+0000","last_active":"2024-09-16T00:11:14.381221+0000","last_peered":"2024-09-16T00:11:14.381221+0000","last_clean":"2024-09-16T00:11:14.381221+0000","last_became_active":"2024-09-16T00:10:55.844922+0000","last_became_peered":"2024-09-16T00:10:55.844922+0000","last_unstale":"2024-09-16T00:11:14.381221+0000","last_undegraded":"2024-09-16T00:11:14.381221+0000","last_fullsized":"2024-09-16T00:11:14.381221+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:19.134467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":4138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2561,"num_read_kb":1362012,"num_write":24417,"num_write_kb":4088607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3177,"num_bytes_recovered":294487447,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5234,"internal_metadata":0},"log_size":25917,"ondisk_log_size":25917,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":129048453,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":176,"num_read_kb":1534,"num_write":5618,"num_write_kb":131059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":26562382,"num_keys_recovered":3117,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86642688,"data_stored":258096906,"data_compressed":38739888,"data_compressed_allocated":49811456,"data_compressed_original":221323264,"omap_allocated":262411,"internal_metadata":0},"log_size":7144,"ondisk_log_size":7144,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":127,"seq":545460846598,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57992,"kb_used_data":16044,"kb_used_omap":42,"kb_used_meta":41877,"kb_avail":94313848,"statfs":{"total":96636764160,"available":96577380352,"internally_reserved":0,"allocated":16429056,"data_stored":51762927,"data_compressed":7683094,"data_compressed_allocated":9629696,"data_compressed_original":45862912,"omap_allocated":43696,"internal_metadata":42882384},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":94,"seq":403726925841,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56032,"kb_used_data":14280,"kb_used_omap":221,"kb_used_meta":41506,"kb_avail":94315808,"statfs":{"total":96636764160,"available":96579387392,"internally_reserved":0,"allocated":14622720,"data_stored":44546809,"data_compressed":6872875,"data_compressed_allocated":8704000,"data_compressed_original":39534592,"omap_allocated":227133,"internal_metadata":42502339},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73848,"kb_used_data":12128,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94297992,"statfs":{"total":96636764160,"available":96561143808,"internally_reserved":0,"allocated":12419072,"data_stored":26012395,"data_compressed":3584779,"data_compressed_allocated":4849664,"data_compressed_original":19345408,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509492,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63496,"kb_used_data":18412,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94308344,"statfs":{"total":96636764160,"available":96571744256,"internally_reserved":0,"allocated":18853888,"data_stored":46983951,"data_compressed":6735741,"data_compressed_allocated":8769536,"data_compressed_original":37810176,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51048,"kb_used_data":11540,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320792,"statfs":{"total":96636764160,"available":96584491008,"internally_reserved":0,"allocated":11816960,"data_stored":33425144,"data_compressed":5001493,"data_compressed_allocated":6361088,"data_compressed_original":28876800,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542195,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67128,"kb_used_data":9564,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94304712,"statfs":{"total":96636764160,"available":96568025088,"internally_reserved":0,"allocated":9793536,"data_stored":26006313,"data_compressed":3864818,"data_compressed_allocated":4833280,"data_compressed_original":21946368,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542195,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57372,"kb_used_data":12292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94314468,"statfs":{"total":96636764160,"available":96578015232,"internally_reserved":0,"allocated":12587008,"data_stored":30797615,"data_compressed":4284061,"data_compressed_allocated":5672960,"data_compressed_original":24797184,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35280,"kb_used_data":4088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336560,"statfs":{"total":96636764160,"available":96600637440,"internally_reserved":0,"allocated":4186112,"data_stored":6502032,"data_compressed":777781,"data_compressed_allocated":1081344,"data_compressed_original":4296704,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10776576,"data_stored":29360766,"data_compressed":4251684,"data_compressed_allocated":5627904,"data_compressed_original":24223744,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8060928,"data_stored":25165984,"data_compressed":3864818,"data_compressed_allocated":4833280,"data_compressed_original":21946368,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2441216,"data_stored":5655551,"data_compressed":777781,"data_compressed_allocated":1081344,"data_compressed_original":4296704,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10072064,"data_stored":32578663,"data_compressed":5001493,"data_compressed_allocated":6361088,"data_compressed_original":28876800,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17108992,"data_stored":46137470,"data_compressed":6735741,"data_compressed_allocated":8769536,"data_compressed_original":37810176,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10674176,"data_stored":25165914,"data_compressed":3584779,"data_compressed_allocated":4849664,"data_compressed_original":19345408,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12877824,"data_stored":43700328,"data_compressed":6872875,"data_compressed_allocated":8704000,"data_compressed_original":39534592,"omap_allocated":225539,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14630912,"data_stored":50332230,"data_compressed":7650717,"data_compressed_allocated":9584640,"data_compressed_original":45289472,"omap_allocated":36872,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5234,"internal_metadata":0}]}} 2024-09-16T00:11:20.061 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T00:11:20.062 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:11:20.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.195+0000 7ff924526640 1 Processor -- start 2024-09-16T00:11:20.196 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.195+0000 7ff924526640 1 -- start start 2024-09-16T00:11:20.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.196+0000 7ff924526640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff91c1523f0 0x7ff91c1527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:20.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.196+0000 7ff924526640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff91c06d670 0x7ff91c158df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:20.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.196+0000 7ff924526640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff91c0c7f00 con 0x7ff91c06eb00 2024-09-16T00:11:20.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.196+0000 7ff924526640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff91c06d2c0 con 0x7ff91c1523f0 2024-09-16T00:11:20.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.196+0000 7ff924526640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7ff91c0c7cb0 con 0x7ff91c06d670 2024-09-16T00:11:20.197 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.196+0000 7ff9222c2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff91c1523f0 0x7ff91c1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:20.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.196+0000 7ff922ac3640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff91c06eb00 0x7ff91c15bb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45730/0 (socket says 172.21.15.139:45730) 2024-09-16T00:11:20.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.196+0000 7ff922ac3640 1 -- 172.21.15.139:0/3110748051 learned_addr learned my addr 172.21.15.139:0/3110748051 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:20.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.196+0000 7ff921ac1640 1 --2- 172.21.15.139:0/3110748051 >> v2:172.21.15.164:3300/0 conn(0x7ff91c06d670 0x7ff91c158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:20.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.196+0000 7ff9222c2640 1 -- 172.21.15.139:0/3110748051 >> v2:172.21.15.164:3300/0 conn(0x7ff91c06d670 msgr2=0x7ff91c158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:20.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.196+0000 7ff9222c2640 1 --2- 172.21.15.139:0/3110748051 >> v2:172.21.15.164:3300/0 conn(0x7ff91c06d670 0x7ff91c158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.196+0000 7ff9222c2640 1 -- 172.21.15.139:0/3110748051 >> v1:172.21.15.139:6789/0 conn(0x7ff91c06eb00 legacy=0x7ff91c15bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:20.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.196+0000 7ff9222c2640 1 -- 172.21.15.139:0/3110748051 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c06bcc0 con 0x7ff91c1523f0 2024-09-16T00:11:20.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.196+0000 7ff9212c0640 1 -- 172.21.15.139:0/3110748051 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1249822521 0 0) 0x7ff91c0c7f00 con 0x7ff91c06eb00 2024-09-16T00:11:20.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.197+0000 7ff9222c2640 1 --2- 172.21.15.139:0/3110748051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c1523f0 0x7ff91c1527d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0c7e30 tx=0x7ff91802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413f2b432f6713e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:20.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.197+0000 7ff9212c0640 1 -- 172.21.15.139:0/3110748051 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918037070 con 0x7ff91c1523f0 2024-09-16T00:11:20.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.197+0000 7ff9212c0640 1 -- 172.21.15.139:0/3110748051 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff918032440 con 0x7ff91c1523f0 2024-09-16T00:11:20.199 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.197+0000 7ff9212c0640 1 -- 172.21.15.139:0/3110748051 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918032810 con 0x7ff91c1523f0 2024-09-16T00:11:20.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.197+0000 7ff924526640 1 -- 172.21.15.139:0/3110748051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c1523f0 msgr2=0x7ff91c1527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:20.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.197+0000 7ff924526640 1 --2- 172.21.15.139:0/3110748051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c1523f0 0x7ff91c1527d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0c7e30 tx=0x7ff91802a230 comp rx=0 tx=0).stop 2024-09-16T00:11:20.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.198+0000 7ff924526640 1 -- 172.21.15.139:0/3110748051 shutdown_connections 2024-09-16T00:11:20.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.198+0000 7ff924526640 1 --2- 172.21.15.139:0/3110748051 >> v2:172.21.15.164:3300/0 conn(0x7ff91c06d670 0x7ff91c158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.198+0000 7ff924526640 1 --2- 172.21.15.139:0/3110748051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c1523f0 0x7ff91c1527d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.198+0000 7ff924526640 1 -- 172.21.15.139:0/3110748051 >> 172.21.15.139:0/3110748051 conn(0x7ff91c067950 msgr2=0x7ff91c067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:20.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.198+0000 7ff924526640 1 -- 172.21.15.139:0/3110748051 shutdown_connections 2024-09-16T00:11:20.200 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.198+0000 7ff924526640 1 -- 172.21.15.139:0/3110748051 wait complete. 2024-09-16T00:11:20.204 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.199+0000 7ff924526640 1 Processor -- start 2024-09-16T00:11:20.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.199+0000 7ff924526640 1 -- start start 2024-09-16T00:11:20.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff924526640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff91c06d670 0x7ff91c149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:20.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff924526640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c06eb00 0x7ff91c14efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:20.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff924526640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff91c1523f0 0x7ff91c14e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:20.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff924526640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c158580 con 0x7ff91c06eb00 2024-09-16T00:11:20.205 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff924526640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c0c76e0 con 0x7ff91c1523f0 2024-09-16T00:11:20.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff924526640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7ff91c0cd220 con 0x7ff91c06d670 2024-09-16T00:11:20.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff921ac1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c06eb00 0x7ff91c14efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:20.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff921ac1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c06eb00 0x7ff91c14efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43420/0 (socket says 172.21.15.139:43420) 2024-09-16T00:11:20.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff921ac1640 1 -- 172.21.15.139:0/2167555793 learned_addr learned my addr 172.21.15.139:0/2167555793 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:20.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff9222c2640 1 --2- 172.21.15.139:0/2167555793 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff91c06d670 0x7ff91c149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:20.206 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff922ac3640 1 --2- 172.21.15.139:0/2167555793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff91c1523f0 0x7ff91c14e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:20.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff921ac1640 1 -- 172.21.15.139:0/2167555793 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff91c06d670 msgr2=0x7ff91c149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:20.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff921ac1640 1 --2- 172.21.15.139:0/2167555793 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff91c06d670 0x7ff91c149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff921ac1640 1 -- 172.21.15.139:0/2167555793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff91c1523f0 msgr2=0x7ff91c14e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:20.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff921ac1640 1 --2- 172.21.15.139:0/2167555793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff91c1523f0 0x7ff91c14e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.207 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff921ac1640 1 -- 172.21.15.139:0/2167555793 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c0c9930 con 0x7ff91c06eb00 2024-09-16T00:11:20.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff922ac3640 1 --2- 172.21.15.139:0/2167555793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff91c1523f0 0x7ff91c14e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:20.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff921ac1640 1 --2- 172.21.15.139:0/2167555793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c06eb00 0x7ff91c14efd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00d800 tx=0x7ff90c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:20.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff912ffd640 1 -- 172.21.15.139:0/2167555793 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c00b6c0 con 0x7ff91c06eb00 2024-09-16T00:11:20.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff912ffd640 1 -- 172.21.15.139:0/2167555793 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff90c019070 con 0x7ff91c06eb00 2024-09-16T00:11:20.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.200+0000 7ff924526640 1 -- 172.21.15.139:0/2167555793 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91c1f0060 con 0x7ff91c06eb00 2024-09-16T00:11:20.208 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.201+0000 7ff912ffd640 1 -- 172.21.15.139:0/2167555793 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c00ba80 con 0x7ff91c06eb00 2024-09-16T00:11:20.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.201+0000 7ff924526640 1 -- 172.21.15.139:0/2167555793 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff91c1f0620 con 0x7ff91c06eb00 2024-09-16T00:11:20.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.201+0000 7ff924526640 1 -- 172.21.15.139:0/2167555793 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff91c0ca6e0 con 0x7ff91c06eb00 2024-09-16T00:11:20.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.202+0000 7ff912ffd640 1 -- 172.21.15.139:0/2167555793 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff90c00bc20 con 0x7ff91c06eb00 2024-09-16T00:11:20.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.202+0000 7ff912ffd640 1 --2- 172.21.15.139:0/2167555793 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff90006c2b0 0x7ff90006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:20.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.202+0000 7ff912ffd640 1 -- 172.21.15.139:0/2167555793 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7ff90c08b8c0 con 0x7ff91c06eb00 2024-09-16T00:11:20.209 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.202+0000 7ff9222c2640 1 --2- 172.21.15.139:0/2167555793 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff90006c2b0 0x7ff90006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:20.210 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.203+0000 7ff9222c2640 1 --2- 172.21.15.139:0/2167555793 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff90006c2b0 0x7ff90006e770 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff918034000 tx=0x7ff91802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:20.212 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.207+0000 7ff912ffd640 1 -- 172.21.15.139:0/2167555793 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff90c059fd0 con 0x7ff91c06eb00 2024-09-16T00:11:20.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.369+0000 7ff924526640 1 -- 172.21.15.139:0/2167555793 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff91c0c6ca0 con 0x7ff90006c2b0 2024-09-16T00:11:20.376 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.375+0000 7ff912ffd640 1 -- 172.21.15.139:0/2167555793 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254607 (secure 0 0 0) 0x7ff91c0c6ca0 con 0x7ff90006c2b0 2024-09-16T00:11:20.380 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:11:20.381 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:11:20.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.378+0000 7ff924526640 1 -- 172.21.15.139:0/2167555793 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff90006c2b0 msgr2=0x7ff90006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:20.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.378+0000 7ff924526640 1 --2- 172.21.15.139:0/2167555793 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff90006c2b0 0x7ff90006e770 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff918034000 tx=0x7ff91802e000 comp rx=0 tx=0).stop 2024-09-16T00:11:20.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.378+0000 7ff924526640 1 -- 172.21.15.139:0/2167555793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c06eb00 msgr2=0x7ff91c14efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:20.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.378+0000 7ff924526640 1 --2- 172.21.15.139:0/2167555793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c06eb00 0x7ff91c14efd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00d800 tx=0x7ff90c00dcc0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.378+0000 7ff924526640 1 -- 172.21.15.139:0/2167555793 shutdown_connections 2024-09-16T00:11:20.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.378+0000 7ff924526640 1 --2- 172.21.15.139:0/2167555793 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7ff90006c2b0 0x7ff90006e770 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.378+0000 7ff924526640 1 --2- 172.21.15.139:0/2167555793 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff91c1523f0 0x7ff91c14e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.378+0000 7ff924526640 1 --2- 172.21.15.139:0/2167555793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff91c06eb00 0x7ff91c14efd0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.378+0000 7ff924526640 1 --2- 172.21.15.139:0/2167555793 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7ff91c06d670 0x7ff91c149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.378+0000 7ff924526640 1 -- 172.21.15.139:0/2167555793 >> 172.21.15.139:0/2167555793 conn(0x7ff91c067950 msgr2=0x7ff91c1563f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:20.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.378+0000 7ff924526640 1 -- 172.21.15.139:0/2167555793 shutdown_connections 2024-09-16T00:11:20.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.378+0000 7ff924526640 1 -- 172.21.15.139:0/2167555793 wait complete. 2024-09-16T00:11:20.391 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-16T00:11:20.067332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":129638821,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":2853,"num_read_kb":1363646,"num_write":30194,"num_write_kb":4222082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3232,"num_bytes_recovered":321049829,"num_keys_recovered":3117,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33229,"ondisk_log_size":33229,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":462192,"kb_used_data":98360,"kb_used_omap":273,"kb_used_meta":363374,"kb_avail":754512528,"statfs":{"total":773094113280,"available":772620828672,"internally_reserved":0,"allocated":100720640,"data_stored":266043338,"data_compressed":38804642,"data_compressed_allocated":49901568,"data_compressed_original":222470144,"omap_allocated":280397,"internal_metadata":372095155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001521"},"pg_stats":[{"pgid":"3.3f","version":"123'434","reported_seq":869,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.904363+0000","last_change":"2024-09-16T00:10:56.416283+0000","last_active":"2024-09-16T00:11:14.904363+0000","last_peered":"2024-09-16T00:11:14.904363+0000","last_clean":"2024-09-16T00:11:14.904363+0000","last_became_active":"2024-09-16T00:10:55.845182+0000","last_became_peered":"2024-09-16T00:10:55.845182+0000","last_unstale":"2024-09-16T00:11:14.904363+0000","last_undegraded":"2024-09-16T00:11:14.904363+0000","last_fullsized":"2024-09-16T00:11:14.904363+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:52.137388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":42086,"num_write":409,"num_write_kb":112716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":31107821,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"123'417","reported_seq":1093,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749522+0000","last_change":"2024-09-16T00:11:04.897161+0000","last_active":"2024-09-16T00:11:15.749522+0000","last_peered":"2024-09-16T00:11:15.749522+0000","last_clean":"2024-09-16T00:11:15.749522+0000","last_became_active":"2024-09-16T00:11:04.894999+0000","last_became_peered":"2024-09-16T00:11:04.894999+0000","last_unstale":"2024-09-16T00:11:15.749522+0000","last_undegraded":"2024-09-16T00:11:15.749522+0000","last_fullsized":"2024-09-16T00:11:15.749522+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:13.015652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":23064,"num_write":397,"num_write_kb":47220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":689591,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"123'367","reported_seq":657,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381045+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:11:14.381045+0000","last_peered":"2024-09-16T00:11:14.381045+0000","last_clean":"2024-09-16T00:11:14.381045+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:11:14.381045+0000","last_undegraded":"2024-09-16T00:11:14.381045+0000","last_fullsized":"2024-09-16T00:11:14.381045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15782,"num_write":349,"num_write_kb":54932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"123'389","reported_seq":679,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951475+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:11:13.951475+0000","last_peered":"2024-09-16T00:11:13.951475+0000","last_clean":"2024-09-16T00:11:13.951475+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:11:13.951475+0000","last_undegraded":"2024-09-16T00:11:13.951475+0000","last_fullsized":"2024-09-16T00:11:13.951475+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":14080,"num_write":364,"num_write_kb":63191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"123'445","reported_seq":748,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749287+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:11:15.749287+0000","last_peered":"2024-09-16T00:11:15.749287+0000","last_clean":"2024-09-16T00:11:15.749287+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:11:15.749287+0000","last_undegraded":"2024-09-16T00:11:15.749287+0000","last_fullsized":"2024-09-16T00:11:15.749287+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":36096,"num_write":427,"num_write_kb":89616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"146'390","reported_seq":1358,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.012367+0000","last_change":"2024-09-16T00:10:45.795247+0000","last_active":"2024-09-16T00:11:19.012367+0000","last_peered":"2024-09-16T00:11:19.012367+0000","last_clean":"2024-09-16T00:11:19.012367+0000","last_became_active":"2024-09-16T00:10:45.793434+0000","last_became_peered":"2024-09-16T00:10:45.793434+0000","last_unstale":"2024-09-16T00:11:19.012367+0000","last_undegraded":"2024-09-16T00:11:19.012367+0000","last_fullsized":"2024-09-16T00:11:19.012367+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:55.019829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":6174,"num_write":362,"num_write_kb":50827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":425,"num_bytes_recovered":23619955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"123'420","reported_seq":781,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.903864+0000","last_change":"2024-09-16T00:10:56.030536+0000","last_active":"2024-09-16T00:11:14.903864+0000","last_peered":"2024-09-16T00:11:14.903864+0000","last_clean":"2024-09-16T00:11:14.903864+0000","last_became_active":"2024-09-16T00:10:55.835516+0000","last_became_peered":"2024-09-16T00:10:55.835516+0000","last_unstale":"2024-09-16T00:11:14.903864+0000","last_undegraded":"2024-09-16T00:11:14.903864+0000","last_fullsized":"2024-09-16T00:11:14.903864+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:43.152015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":16577,"num_write":401,"num_write_kb":64470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"146'383","reported_seq":650,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.028552+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:11:19.028552+0000","last_peered":"2024-09-16T00:11:19.028552+0000","last_clean":"2024-09-16T00:11:19.028552+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:11:19.028552+0000","last_undegraded":"2024-09-16T00:11:19.028552+0000","last_fullsized":"2024-09-16T00:11:19.028552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":16776,"num_write":347,"num_write_kb":80164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"123'394","reported_seq":726,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.904298+0000","last_change":"2024-09-16T00:10:56.261956+0000","last_active":"2024-09-16T00:11:14.904298+0000","last_peered":"2024-09-16T00:11:14.904298+0000","last_clean":"2024-09-16T00:11:14.904298+0000","last_became_active":"2024-09-16T00:10:55.840604+0000","last_became_peered":"2024-09-16T00:10:55.840604+0000","last_unstale":"2024-09-16T00:11:14.904298+0000","last_undegraded":"2024-09-16T00:11:14.904298+0000","last_fullsized":"2024-09-16T00:11:14.904298+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:39.279660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1432,"num_write":368,"num_write_kb":30311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"123'367","reported_seq":536,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011843+0000","last_change":"2024-09-16T00:10:00.734336+0000","last_active":"2024-09-16T00:11:14.011843+0000","last_peered":"2024-09-16T00:11:14.011843+0000","last_clean":"2024-09-16T00:11:14.011843+0000","last_became_active":"2024-09-16T00:10:00.284452+0000","last_became_peered":"2024-09-16T00:10:00.284452+0000","last_unstale":"2024-09-16T00:11:14.011843+0000","last_undegraded":"2024-09-16T00:11:14.011843+0000","last_fullsized":"2024-09-16T00:11:14.011843+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:56:50.530439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4096,"num_write":335,"num_write_kb":29952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":8736255,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"123'355","reported_seq":535,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011551+0000","last_change":"2024-09-16T00:10:00.587110+0000","last_active":"2024-09-16T00:11:14.011551+0000","last_peered":"2024-09-16T00:11:14.011551+0000","last_clean":"2024-09-16T00:11:14.011551+0000","last_became_active":"2024-09-16T00:10:00.280273+0000","last_became_peered":"2024-09-16T00:10:00.280273+0000","last_unstale":"2024-09-16T00:11:14.011551+0000","last_undegraded":"2024-09-16T00:11:14.011551+0000","last_fullsized":"2024-09-16T00:11:14.011551+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:01:57.902939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":975,"num_write":339,"num_write_kb":25518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":10753732,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"123'366","reported_seq":705,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952011+0000","last_change":"2024-09-16T00:10:00.797372+0000","last_active":"2024-09-16T00:11:13.952011+0000","last_peered":"2024-09-16T00:11:13.952011+0000","last_clean":"2024-09-16T00:11:13.952011+0000","last_became_active":"2024-09-16T00:10:00.278765+0000","last_became_peered":"2024-09-16T00:10:00.278765+0000","last_unstale":"2024-09-16T00:11:13.952011+0000","last_undegraded":"2024-09-16T00:11:13.952011+0000","last_fullsized":"2024-09-16T00:11:13.952011+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:03.318919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":11563,"num_write":348,"num_write_kb":58933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":29565877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"123'419","reported_seq":784,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749693+0000","last_change":"2024-09-16T00:10:00.715647+0000","last_active":"2024-09-16T00:11:15.749693+0000","last_peered":"2024-09-16T00:11:15.749693+0000","last_clean":"2024-09-16T00:11:15.749693+0000","last_became_active":"2024-09-16T00:10:00.286050+0000","last_became_peered":"2024-09-16T00:10:00.286050+0000","last_unstale":"2024-09-16T00:11:15.749693+0000","last_undegraded":"2024-09-16T00:11:15.749693+0000","last_fullsized":"2024-09-16T00:11:15.749693+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:07.462899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":35908,"num_write":393,"num_write_kb":104863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":30230864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"123'415","reported_seq":724,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.380880+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:11:14.380880+0000","last_peered":"2024-09-16T00:11:14.380880+0000","last_clean":"2024-09-16T00:11:14.380880+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:11:14.380880+0000","last_undegraded":"2024-09-16T00:11:14.380880+0000","last_fullsized":"2024-09-16T00:11:14.380880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":23364,"num_write":390,"num_write_kb":43321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"123'392","reported_seq":684,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749312+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:11:15.749312+0000","last_peered":"2024-09-16T00:11:15.749312+0000","last_clean":"2024-09-16T00:11:15.749312+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:11:15.749312+0000","last_undegraded":"2024-09-16T00:11:15.749312+0000","last_fullsized":"2024-09-16T00:11:15.749312+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":392,"log_dups_size":0,"ondisk_log_size":392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":14230,"num_write":362,"num_write_kb":33768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"123'439","reported_seq":756,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951004+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:11:13.951004+0000","last_peered":"2024-09-16T00:11:13.951004+0000","last_clean":"2024-09-16T00:11:13.951004+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:11:13.951004+0000","last_undegraded":"2024-09-16T00:11:13.951004+0000","last_fullsized":"2024-09-16T00:11:13.951004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":40234,"num_write":414,"num_write_kb":67785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"123'394","reported_seq":890,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952446+0000","last_change":"2024-09-16T00:10:56.577752+0000","last_active":"2024-09-16T00:11:13.952446+0000","last_peered":"2024-09-16T00:11:13.952446+0000","last_clean":"2024-09-16T00:11:13.952446+0000","last_became_active":"2024-09-16T00:10:55.845880+0000","last_became_peered":"2024-09-16T00:10:55.845880+0000","last_unstale":"2024-09-16T00:11:13.952446+0000","last_undegraded":"2024-09-16T00:11:13.952446+0000","last_fullsized":"2024-09-16T00:11:13.952446+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:32.037945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":22524,"num_write":374,"num_write_kb":46842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"146'401","reported_seq":730,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.008979+0000","last_change":"2024-09-16T00:10:00.461437+0000","last_active":"2024-09-16T00:11:19.008979+0000","last_peered":"2024-09-16T00:11:19.008979+0000","last_clean":"2024-09-16T00:11:19.008979+0000","last_became_active":"2024-09-16T00:10:00.282592+0000","last_became_peered":"2024-09-16T00:10:00.282592+0000","last_unstale":"2024-09-16T00:11:19.008979+0000","last_undegraded":"2024-09-16T00:11:19.008979+0000","last_fullsized":"2024-09-16T00:11:19.008979+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:09.733353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8711,"num_write":366,"num_write_kb":46855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":21285626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"123'417","reported_seq":827,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.904239+0000","last_change":"2024-09-16T00:10:56.142127+0000","last_active":"2024-09-16T00:11:14.904239+0000","last_peered":"2024-09-16T00:11:14.904239+0000","last_clean":"2024-09-16T00:11:14.904239+0000","last_became_active":"2024-09-16T00:10:55.832595+0000","last_became_peered":"2024-09-16T00:10:55.832595+0000","last_unstale":"2024-09-16T00:11:14.904239+0000","last_undegraded":"2024-09-16T00:11:14.904239+0000","last_fullsized":"2024-09-16T00:11:14.904239+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:44:56.072997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":14437,"num_write":396,"num_write_kb":83979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":103497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"123'423","reported_seq":602,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011546+0000","last_change":"2024-09-16T00:10:00.566870+0000","last_active":"2024-09-16T00:11:14.011546+0000","last_peered":"2024-09-16T00:11:14.011546+0000","last_clean":"2024-09-16T00:11:14.011546+0000","last_became_active":"2024-09-16T00:10:00.281258+0000","last_became_peered":"2024-09-16T00:10:00.281258+0000","last_unstale":"2024-09-16T00:11:14.011546+0000","last_undegraded":"2024-09-16T00:11:14.011546+0000","last_fullsized":"2024-09-16T00:11:14.011546+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:29.197743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":15502,"num_write":406,"num_write_kb":30222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":17199573,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"123'404","reported_seq":883,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381025+0000","last_change":"2024-09-16T00:10:56.104046+0000","last_active":"2024-09-16T00:11:14.381025+0000","last_peered":"2024-09-16T00:11:14.381025+0000","last_clean":"2024-09-16T00:11:14.381025+0000","last_became_active":"2024-09-16T00:10:55.835150+0000","last_became_peered":"2024-09-16T00:10:55.835150+0000","last_unstale":"2024-09-16T00:11:14.381025+0000","last_undegraded":"2024-09-16T00:11:14.381025+0000","last_fullsized":"2024-09-16T00:11:14.381025+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:08.552926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":34318,"num_write":385,"num_write_kb":91769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"123'444","reported_seq":918,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951970+0000","last_change":"2024-09-16T00:10:55.939360+0000","last_active":"2024-09-16T00:11:13.951970+0000","last_peered":"2024-09-16T00:11:13.951970+0000","last_clean":"2024-09-16T00:11:13.951970+0000","last_became_active":"2024-09-16T00:10:55.833335+0000","last_became_peered":"2024-09-16T00:10:55.833335+0000","last_unstale":"2024-09-16T00:11:13.951970+0000","last_undegraded":"2024-09-16T00:11:13.951970+0000","last_fullsized":"2024-09-16T00:11:13.951970+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:08:29.270004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":35528,"num_write":422,"num_write_kb":92852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"123'395","reported_seq":683,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749700+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:11:15.749700+0000","last_peered":"2024-09-16T00:11:15.749700+0000","last_clean":"2024-09-16T00:11:15.749700+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:11:15.749700+0000","last_undegraded":"2024-09-16T00:11:15.749700+0000","last_fullsized":"2024-09-16T00:11:15.749700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":395,"log_dups_size":0,"ondisk_log_size":395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":13410,"num_write":369,"num_write_kb":58805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"123'421","reported_seq":703,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951092+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:11:13.951092+0000","last_peered":"2024-09-16T00:11:13.951092+0000","last_clean":"2024-09-16T00:11:13.951092+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:11:13.951092+0000","last_undegraded":"2024-09-16T00:11:13.951092+0000","last_fullsized":"2024-09-16T00:11:13.951092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":12682,"num_write":399,"num_write_kb":54869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"123'421","reported_seq":712,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749956+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:11:15.749956+0000","last_peered":"2024-09-16T00:11:15.749956+0000","last_clean":"2024-09-16T00:11:15.749956+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:11:15.749956+0000","last_undegraded":"2024-09-16T00:11:15.749956+0000","last_fullsized":"2024-09-16T00:11:15.749956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":14188,"num_write":408,"num_write_kb":59178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"123'448","reported_seq":862,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011709+0000","last_change":"2024-09-16T00:10:56.267499+0000","last_active":"2024-09-16T00:11:14.011709+0000","last_peered":"2024-09-16T00:11:14.011709+0000","last_clean":"2024-09-16T00:11:14.011709+0000","last_became_active":"2024-09-16T00:10:55.841967+0000","last_became_peered":"2024-09-16T00:10:55.841967+0000","last_unstale":"2024-09-16T00:11:14.011709+0000","last_undegraded":"2024-09-16T00:11:14.011709+0000","last_fullsized":"2024-09-16T00:11:14.011709+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:15.324718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":12330,"num_write":421,"num_write_kb":35146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":12903786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"123'402","reported_seq":1052,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749340+0000","last_change":"2024-09-16T00:11:04.895240+0000","last_active":"2024-09-16T00:11:15.749340+0000","last_peered":"2024-09-16T00:11:15.749340+0000","last_clean":"2024-09-16T00:11:15.749340+0000","last_became_active":"2024-09-16T00:11:04.892794+0000","last_became_peered":"2024-09-16T00:11:04.892794+0000","last_unstale":"2024-09-16T00:11:15.749340+0000","last_undegraded":"2024-09-16T00:11:15.749340+0000","last_fullsized":"2024-09-16T00:11:15.749340+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:44.365114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":21104,"num_write":383,"num_write_kb":75742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":628439,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"123'386","reported_seq":688,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749839+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:11:15.749839+0000","last_peered":"2024-09-16T00:11:15.749839+0000","last_clean":"2024-09-16T00:11:15.749839+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:11:15.749839+0000","last_undegraded":"2024-09-16T00:11:15.749839+0000","last_fullsized":"2024-09-16T00:11:15.749839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":16408,"num_write":366,"num_write_kb":51674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"123'398","reported_seq":717,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381121+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:11:14.381121+0000","last_peered":"2024-09-16T00:11:14.381121+0000","last_clean":"2024-09-16T00:11:14.381121+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:11:14.381121+0000","last_undegraded":"2024-09-16T00:11:14.381121+0000","last_fullsized":"2024-09-16T00:11:14.381121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":33174,"num_write":373,"num_write_kb":66849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"146'494","reported_seq":788,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.011926+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:11:19.011926+0000","last_peered":"2024-09-16T00:11:19.011926+0000","last_clean":"2024-09-16T00:11:19.011926+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:11:19.011926+0000","last_undegraded":"2024-09-16T00:11:19.011926+0000","last_fullsized":"2024-09-16T00:11:19.011926+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":27231,"num_write":453,"num_write_kb":104771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"146'463","reported_seq":801,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.011767+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:11:19.011767+0000","last_peered":"2024-09-16T00:11:19.011767+0000","last_clean":"2024-09-16T00:11:19.011767+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:11:19.011767+0000","last_undegraded":"2024-09-16T00:11:19.011767+0000","last_fullsized":"2024-09-16T00:11:19.011767+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":35655,"num_write":431,"num_write_kb":97245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"123'421","reported_seq":602,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011470+0000","last_change":"2024-09-16T00:10:00.535553+0000","last_active":"2024-09-16T00:11:14.011470+0000","last_peered":"2024-09-16T00:11:14.011470+0000","last_clean":"2024-09-16T00:11:14.011470+0000","last_became_active":"2024-09-16T00:10:00.281378+0000","last_became_peered":"2024-09-16T00:10:00.281378+0000","last_unstale":"2024-09-16T00:11:14.011470+0000","last_undegraded":"2024-09-16T00:11:14.011470+0000","last_fullsized":"2024-09-16T00:11:14.011470+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:49.548709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20395,"num_write":401,"num_write_kb":32731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":22876917,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"123'401","reported_seq":741,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951565+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:11:13.951565+0000","last_peered":"2024-09-16T00:11:13.951565+0000","last_clean":"2024-09-16T00:11:13.951565+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:11:13.951565+0000","last_undegraded":"2024-09-16T00:11:13.951565+0000","last_fullsized":"2024-09-16T00:11:13.951565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":48141,"num_write":381,"num_write_kb":63611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"120'126","reported_seq":402,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749808+0000","last_change":"2024-09-16T00:10:55.854872+0000","last_active":"2024-09-16T00:11:15.749808+0000","last_peered":"2024-09-16T00:11:15.749808+0000","last_clean":"2024-09-16T00:11:15.749808+0000","last_became_active":"2024-09-16T00:10:55.843367+0000","last_became_peered":"2024-09-16T00:10:55.843367+0000","last_unstale":"2024-09-16T00:11:15.749808+0000","last_undegraded":"2024-09-16T00:11:15.749808+0000","last_fullsized":"2024-09-16T00:11:15.749808+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:07.797903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"123'346","reported_seq":633,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381167+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:11:14.381167+0000","last_peered":"2024-09-16T00:11:14.381167+0000","last_clean":"2024-09-16T00:11:14.381167+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:11:14.381167+0000","last_undegraded":"2024-09-16T00:11:14.381167+0000","last_fullsized":"2024-09-16T00:11:14.381167+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":16420,"num_write":326,"num_write_kb":62710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"122'340","reported_seq":505,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011382+0000","last_change":"2024-09-16T00:10:00.282958+0000","last_active":"2024-09-16T00:11:14.011382+0000","last_peered":"2024-09-16T00:11:14.011382+0000","last_clean":"2024-09-16T00:11:14.011382+0000","last_became_active":"2024-09-16T00:10:00.282541+0000","last_became_peered":"2024-09-16T00:10:00.282541+0000","last_unstale":"2024-09-16T00:11:14.011382+0000","last_undegraded":"2024-09-16T00:11:14.011382+0000","last_fullsized":"2024-09-16T00:11:14.011382+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:51.198787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":285,"num_write_kb":12471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"123'226","reported_seq":510,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749827+0000","last_change":"2024-09-16T00:10:00.313589+0000","last_active":"2024-09-16T00:11:15.749827+0000","last_peered":"2024-09-16T00:11:15.749827+0000","last_clean":"2024-09-16T00:11:15.749827+0000","last_became_active":"2024-09-16T00:10:00.284602+0000","last_became_peered":"2024-09-16T00:10:00.284602+0000","last_unstale":"2024-09-16T00:11:15.749827+0000","last_undegraded":"2024-09-16T00:11:15.749827+0000","last_fullsized":"2024-09-16T00:11:15.749827+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:30.032698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":14,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"123'433","reported_seq":732,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749206+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:11:15.749206+0000","last_peered":"2024-09-16T00:11:15.749206+0000","last_clean":"2024-09-16T00:11:15.749206+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:11:15.749206+0000","last_undegraded":"2024-09-16T00:11:15.749206+0000","last_fullsized":"2024-09-16T00:11:15.749206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":18480,"num_write":411,"num_write_kb":71748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"123'833","reported_seq":1095,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951164+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:11:13.951164+0000","last_peered":"2024-09-16T00:11:13.951164+0000","last_clean":"2024-09-16T00:11:13.951164+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:11:13.951164+0000","last_undegraded":"2024-09-16T00:11:13.951164+0000","last_fullsized":"2024-09-16T00:11:13.951164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":8591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"123'425","reported_seq":774,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749285+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:11:15.749285+0000","last_peered":"2024-09-16T00:11:15.749285+0000","last_clean":"2024-09-16T00:11:15.749285+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:11:15.749285+0000","last_undegraded":"2024-09-16T00:11:15.749285+0000","last_fullsized":"2024-09-16T00:11:15.749285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":46871,"num_write":401,"num_write_kb":84187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"123'65","reported_seq":296,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011701+0000","last_change":"2024-09-16T00:10:00.283876+0000","last_active":"2024-09-16T00:11:14.011701+0000","last_peered":"2024-09-16T00:11:14.011701+0000","last_clean":"2024-09-16T00:11:14.011701+0000","last_became_active":"2024-09-16T00:10:00.283555+0000","last_became_peered":"2024-09-16T00:10:00.283555+0000","last_unstale":"2024-09-16T00:11:14.011701+0000","last_undegraded":"2024-09-16T00:11:14.011701+0000","last_fullsized":"2024-09-16T00:11:14.011701+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:48.543737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"123'440","reported_seq":904,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951434+0000","last_change":"2024-09-16T00:10:56.502658+0000","last_active":"2024-09-16T00:11:13.951434+0000","last_peered":"2024-09-16T00:11:13.951434+0000","last_clean":"2024-09-16T00:11:13.951434+0000","last_became_active":"2024-09-16T00:10:55.840027+0000","last_became_peered":"2024-09-16T00:10:55.840027+0000","last_unstale":"2024-09-16T00:11:13.951434+0000","last_undegraded":"2024-09-16T00:11:13.951434+0000","last_fullsized":"2024-09-16T00:11:13.951434+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:52.105868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":16686,"num_write":415,"num_write_kb":80112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"123'225","reported_seq":494,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951555+0000","last_change":"2024-09-16T00:10:00.285386+0000","last_active":"2024-09-16T00:11:13.951555+0000","last_peered":"2024-09-16T00:11:13.951555+0000","last_clean":"2024-09-16T00:11:13.951555+0000","last_became_active":"2024-09-16T00:10:00.284717+0000","last_became_peered":"2024-09-16T00:10:00.284717+0000","last_unstale":"2024-09-16T00:11:13.951555+0000","last_undegraded":"2024-09-16T00:11:13.951555+0000","last_fullsized":"2024-09-16T00:11:13.951555+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:07.794163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":4181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"123'378","reported_seq":681,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381204+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:11:14.381204+0000","last_peered":"2024-09-16T00:11:14.381204+0000","last_clean":"2024-09-16T00:11:14.381204+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:11:14.381204+0000","last_undegraded":"2024-09-16T00:11:14.381204+0000","last_fullsized":"2024-09-16T00:11:14.381204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":18030,"num_write":359,"num_write_kb":55428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"125'131","reported_seq":392,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749519+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:11:15.749519+0000","last_peered":"2024-09-16T00:11:15.749519+0000","last_clean":"2024-09-16T00:11:15.749519+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:11:15.749519+0000","last_undegraded":"2024-09-16T00:11:15.749519+0000","last_fullsized":"2024-09-16T00:11:15.749519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"123'373","reported_seq":818,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952022+0000","last_change":"2024-09-16T00:10:56.173134+0000","last_active":"2024-09-16T00:11:13.952022+0000","last_peered":"2024-09-16T00:11:13.952022+0000","last_clean":"2024-09-16T00:11:13.952022+0000","last_became_active":"2024-09-16T00:10:55.833273+0000","last_became_peered":"2024-09-16T00:10:55.833273+0000","last_unstale":"2024-09-16T00:11:13.952022+0000","last_undegraded":"2024-09-16T00:11:13.952022+0000","last_fullsized":"2024-09-16T00:11:13.952022+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:11.402988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":22960,"num_write":348,"num_write_kb":63161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"125'238","reported_seq":414,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011582+0000","last_change":"2024-09-16T00:10:00.300895+0000","last_active":"2024-09-16T00:11:14.011582+0000","last_peered":"2024-09-16T00:11:14.011582+0000","last_clean":"2024-09-16T00:11:14.011582+0000","last_became_active":"2024-09-16T00:10:00.283474+0000","last_became_peered":"2024-09-16T00:10:00.283474+0000","last_unstale":"2024-09-16T00:11:14.011582+0000","last_undegraded":"2024-09-16T00:11:14.011582+0000","last_fullsized":"2024-09-16T00:11:14.011582+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:28:02.563817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10145896,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":186,"num_write_kb":10014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":13200,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"123'403","reported_seq":776,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.904032+0000","last_change":"2024-09-16T00:10:56.320793+0000","last_active":"2024-09-16T00:11:14.904032+0000","last_peered":"2024-09-16T00:11:14.904032+0000","last_clean":"2024-09-16T00:11:14.904032+0000","last_became_active":"2024-09-16T00:10:55.836462+0000","last_became_peered":"2024-09-16T00:10:55.836462+0000","last_unstale":"2024-09-16T00:11:14.904032+0000","last_undegraded":"2024-09-16T00:11:14.904032+0000","last_fullsized":"2024-09-16T00:11:14.904032+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:07:47.717822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":23285,"num_write":372,"num_write_kb":67093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"122'470","reported_seq":732,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381212+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:11:14.381212+0000","last_peered":"2024-09-16T00:11:14.381212+0000","last_clean":"2024-09-16T00:11:14.381212+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:11:14.381212+0000","last_undegraded":"2024-09-16T00:11:14.381212+0000","last_fullsized":"2024-09-16T00:11:14.381212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":419,"num_write_kb":8413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"123'404","reported_seq":872,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.380881+0000","last_change":"2024-09-16T00:10:56.427976+0000","last_active":"2024-09-16T00:11:14.380881+0000","last_peered":"2024-09-16T00:11:14.380881+0000","last_clean":"2024-09-16T00:11:14.380881+0000","last_became_active":"2024-09-16T00:10:55.839196+0000","last_became_peered":"2024-09-16T00:10:55.839196+0000","last_unstale":"2024-09-16T00:11:14.380881+0000","last_undegraded":"2024-09-16T00:11:14.380881+0000","last_fullsized":"2024-09-16T00:11:14.380881+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:37.852194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":13263,"num_write":381,"num_write_kb":51174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"123'59","reported_seq":321,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952149+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:11:13.952149+0000","last_peered":"2024-09-16T00:11:13.952149+0000","last_clean":"2024-09-16T00:11:13.952149+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:11:13.952149+0000","last_undegraded":"2024-09-16T00:11:13.952149+0000","last_fullsized":"2024-09-16T00:11:13.952149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"123'394","reported_seq":735,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.904018+0000","last_change":"2024-09-16T00:10:56.085067+0000","last_active":"2024-09-16T00:11:14.904018+0000","last_peered":"2024-09-16T00:11:14.904018+0000","last_clean":"2024-09-16T00:11:14.904018+0000","last_became_active":"2024-09-16T00:10:55.834112+0000","last_became_peered":"2024-09-16T00:10:55.834112+0000","last_unstale":"2024-09-16T00:11:14.904018+0000","last_undegraded":"2024-09-16T00:11:14.904018+0000","last_fullsized":"2024-09-16T00:11:14.904018+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:35.278069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":13990,"num_write":368,"num_write_kb":63962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"120'70","reported_seq":218,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011398+0000","last_change":"2024-09-16T00:10:00.294519+0000","last_active":"2024-09-16T00:11:14.011398+0000","last_peered":"2024-09-16T00:11:14.011398+0000","last_clean":"2024-09-16T00:11:14.011398+0000","last_became_active":"2024-09-16T00:10:00.283066+0000","last_became_peered":"2024-09-16T00:10:00.283066+0000","last_unstale":"2024-09-16T00:11:14.011398+0000","last_undegraded":"2024-09-16T00:11:14.011398+0000","last_fullsized":"2024-09-16T00:11:14.011398+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:32.621176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"123'424","reported_seq":607,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011378+0000","last_change":"2024-09-16T00:10:00.460910+0000","last_active":"2024-09-16T00:11:14.011378+0000","last_peered":"2024-09-16T00:11:14.011378+0000","last_clean":"2024-09-16T00:11:14.011378+0000","last_became_active":"2024-09-16T00:10:00.280750+0000","last_became_peered":"2024-09-16T00:10:00.280750+0000","last_unstale":"2024-09-16T00:11:14.011378+0000","last_undegraded":"2024-09-16T00:11:14.011378+0000","last_fullsized":"2024-09-16T00:11:14.011378+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:47.634650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":19573,"num_write":402,"num_write_kb":50826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":33491520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"125'72","reported_seq":334,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952685+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:11:13.952685+0000","last_peered":"2024-09-16T00:11:13.952685+0000","last_clean":"2024-09-16T00:11:13.952685+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:11:13.952685+0000","last_undegraded":"2024-09-16T00:11:13.952685+0000","last_fullsized":"2024-09-16T00:11:13.952685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"123'457","reported_seq":816,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.903784+0000","last_change":"2024-09-16T00:10:55.922694+0000","last_active":"2024-09-16T00:11:14.903784+0000","last_peered":"2024-09-16T00:11:14.903784+0000","last_clean":"2024-09-16T00:11:14.903784+0000","last_became_active":"2024-09-16T00:10:55.827989+0000","last_became_peered":"2024-09-16T00:10:55.827989+0000","last_unstale":"2024-09-16T00:11:14.903784+0000","last_undegraded":"2024-09-16T00:11:14.903784+0000","last_fullsized":"2024-09-16T00:11:14.903784+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:23.666346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20419,"num_write":427,"num_write_kb":43035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"123'145","reported_seq":407,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951453+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:11:13.951453+0000","last_peered":"2024-09-16T00:11:13.951453+0000","last_clean":"2024-09-16T00:11:13.951453+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:11:13.951453+0000","last_undegraded":"2024-09-16T00:11:13.951453+0000","last_fullsized":"2024-09-16T00:11:13.951453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91,"num_write_kb":4217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"123'358","reported_seq":696,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.904177+0000","last_change":"2024-09-16T00:10:55.971163+0000","last_active":"2024-09-16T00:11:14.904177+0000","last_peered":"2024-09-16T00:11:14.904177+0000","last_clean":"2024-09-16T00:11:14.904177+0000","last_became_active":"2024-09-16T00:10:55.834401+0000","last_became_peered":"2024-09-16T00:10:55.834401+0000","last_unstale":"2024-09-16T00:11:14.904177+0000","last_undegraded":"2024-09-16T00:11:14.904177+0000","last_fullsized":"2024-09-16T00:11:14.904177+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:05.466545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":10790,"num_write":330,"num_write_kb":21585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"120'282","reported_seq":544,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952893+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:11:13.952893+0000","last_peered":"2024-09-16T00:11:13.952893+0000","last_clean":"2024-09-16T00:11:13.952893+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:11:13.952893+0000","last_undegraded":"2024-09-16T00:11:13.952893+0000","last_fullsized":"2024-09-16T00:11:13.952893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":238,"num_write_kb":4238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"123'372","reported_seq":698,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.904474+0000","last_change":"2024-09-16T00:10:56.200838+0000","last_active":"2024-09-16T00:11:14.904474+0000","last_peered":"2024-09-16T00:11:14.904474+0000","last_clean":"2024-09-16T00:11:14.904474+0000","last_became_active":"2024-09-16T00:10:55.838254+0000","last_became_peered":"2024-09-16T00:10:55.838254+0000","last_unstale":"2024-09-16T00:11:14.904474+0000","last_undegraded":"2024-09-16T00:11:14.904474+0000","last_fullsized":"2024-09-16T00:11:14.904474+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:30.944542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":350,"num_write":351,"num_write_kb":38221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"122'534","reported_seq":795,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749610+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:11:15.749610+0000","last_peered":"2024-09-16T00:11:15.749610+0000","last_clean":"2024-09-16T00:11:15.749610+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:11:15.749610+0000","last_undegraded":"2024-09-16T00:11:15.749610+0000","last_fullsized":"2024-09-16T00:11:15.749610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":473,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"123'418","reported_seq":682,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951696+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:11:13.951696+0000","last_peered":"2024-09-16T00:11:13.951696+0000","last_clean":"2024-09-16T00:11:13.951696+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:11:13.951696+0000","last_undegraded":"2024-09-16T00:11:13.951696+0000","last_fullsized":"2024-09-16T00:11:13.951696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":675,"num_write":389,"num_write_kb":34696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"123'72","reported_seq":334,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952967+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:11:13.952967+0000","last_peered":"2024-09-16T00:11:13.952967+0000","last_clean":"2024-09-16T00:11:13.952967+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:11:13.952967+0000","last_undegraded":"2024-09-16T00:11:13.952967+0000","last_fullsized":"2024-09-16T00:11:13.952967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"122'397","reported_seq":763,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.904197+0000","last_change":"2024-09-16T00:10:56.368023+0000","last_active":"2024-09-16T00:11:14.904197+0000","last_peered":"2024-09-16T00:11:14.904197+0000","last_clean":"2024-09-16T00:11:14.904197+0000","last_became_active":"2024-09-16T00:10:55.839714+0000","last_became_peered":"2024-09-16T00:10:55.839714+0000","last_unstale":"2024-09-16T00:11:14.904197+0000","last_undegraded":"2024-09-16T00:11:14.904197+0000","last_fullsized":"2024-09-16T00:11:14.904197+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:04.192344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":22590,"num_write":379,"num_write_kb":59088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":374,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.903717+0000","last_change":"2024-09-16T00:10:55.832444+0000","last_active":"2024-09-16T00:11:14.903717+0000","last_peered":"2024-09-16T00:11:14.903717+0000","last_clean":"2024-09-16T00:11:14.903717+0000","last_became_active":"2024-09-16T00:10:55.831881+0000","last_became_peered":"2024-09-16T00:10:55.831881+0000","last_unstale":"2024-09-16T00:11:14.903717+0000","last_undegraded":"2024-09-16T00:11:14.903717+0000","last_fullsized":"2024-09-16T00:11:14.903717+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:48.243166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"123'63","reported_seq":325,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951198+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:11:13.951198+0000","last_peered":"2024-09-16T00:11:13.951198+0000","last_clean":"2024-09-16T00:11:13.951198+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:11:13.951198+0000","last_undegraded":"2024-09-16T00:11:13.951198+0000","last_fullsized":"2024-09-16T00:11:13.951198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"123'369","reported_seq":664,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749611+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:11:15.749611+0000","last_peered":"2024-09-16T00:11:15.749611+0000","last_clean":"2024-09-16T00:11:15.749611+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:11:15.749611+0000","last_undegraded":"2024-09-16T00:11:15.749611+0000","last_fullsized":"2024-09-16T00:11:15.749611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":16103,"num_write":342,"num_write_kb":46241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"123'399","reported_seq":645,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.903728+0000","last_change":"2024-09-16T00:11:04.896119+0000","last_active":"2024-09-16T00:11:14.903728+0000","last_peered":"2024-09-16T00:11:14.903728+0000","last_clean":"2024-09-16T00:11:14.903728+0000","last_became_active":"2024-09-16T00:11:04.895672+0000","last_became_peered":"2024-09-16T00:11:04.895672+0000","last_unstale":"2024-09-16T00:11:14.903728+0000","last_undegraded":"2024-09-16T00:11:14.903728+0000","last_fullsized":"2024-09-16T00:11:14.903728+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:45.173943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":346,"num_write_kb":8359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":16777216,"num_keys_recovered":28,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"123'393","reported_seq":700,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.380785+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:11:14.380785+0000","last_peered":"2024-09-16T00:11:14.380785+0000","last_clean":"2024-09-16T00:11:14.380785+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:11:14.380785+0000","last_undegraded":"2024-09-16T00:11:14.380785+0000","last_fullsized":"2024-09-16T00:11:14.380785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":29385,"num_write":362,"num_write_kb":83610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"125'242","reported_seq":680,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952710+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:11:13.952710+0000","last_peered":"2024-09-16T00:11:13.952710+0000","last_clean":"2024-09-16T00:11:13.952710+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:11:13.952710+0000","last_undegraded":"2024-09-16T00:11:13.952710+0000","last_fullsized":"2024-09-16T00:11:13.952710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461247,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":176,"num_read_kb":1534,"num_write":179,"num_write_kb":1542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"123'400","reported_seq":712,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749491+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:11:15.749491+0000","last_peered":"2024-09-16T00:11:15.749491+0000","last_clean":"2024-09-16T00:11:15.749491+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:11:15.749491+0000","last_undegraded":"2024-09-16T00:11:15.749491+0000","last_fullsized":"2024-09-16T00:11:15.749491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":28288,"num_write":378,"num_write_kb":66949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"119'400","reported_seq":662,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952094+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:11:13.952094+0000","last_peered":"2024-09-16T00:11:13.952094+0000","last_clean":"2024-09-16T00:11:13.952094+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:11:13.952094+0000","last_undegraded":"2024-09-16T00:11:13.952094+0000","last_fullsized":"2024-09-16T00:11:13.952094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":353,"num_write_kb":8381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"123'387","reported_seq":683,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951764+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:11:13.951764+0000","last_peered":"2024-09-16T00:11:13.951764+0000","last_clean":"2024-09-16T00:11:13.951764+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:11:13.951764+0000","last_undegraded":"2024-09-16T00:11:13.951764+0000","last_fullsized":"2024-09-16T00:11:13.951764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":22188,"num_write":360,"num_write_kb":71552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"122'57","reported_seq":318,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749407+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:11:15.749407+0000","last_peered":"2024-09-16T00:11:15.749407+0000","last_clean":"2024-09-16T00:11:15.749407+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:11:15.749407+0000","last_undegraded":"2024-09-16T00:11:15.749407+0000","last_fullsized":"2024-09-16T00:11:15.749407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"123'426","reported_seq":725,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952294+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:11:13.952294+0000","last_peered":"2024-09-16T00:11:13.952294+0000","last_clean":"2024-09-16T00:11:13.952294+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:11:13.952294+0000","last_undegraded":"2024-09-16T00:11:13.952294+0000","last_fullsized":"2024-09-16T00:11:13.952294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":24752,"num_write":390,"num_write_kb":75609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"123'258","reported_seq":303,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.903671+0000","last_change":"2024-09-16T00:10:58.839395+0000","last_active":"2024-09-16T00:11:14.903671+0000","last_peered":"2024-09-16T00:11:14.903671+0000","last_clean":"2024-09-16T00:11:14.903671+0000","last_became_active":"2024-09-16T00:10:58.839124+0000","last_became_peered":"2024-09-16T00:10:58.839124+0000","last_unstale":"2024-09-16T00:11:14.903671+0000","last_undegraded":"2024-09-16T00:11:14.903671+0000","last_fullsized":"2024-09-16T00:11:14.903671+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:29.136640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":202,"num_write_kb":4192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"123'398","reported_seq":715,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749363+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:11:15.749363+0000","last_peered":"2024-09-16T00:11:15.749363+0000","last_clean":"2024-09-16T00:11:15.749363+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:11:15.749363+0000","last_undegraded":"2024-09-16T00:11:15.749363+0000","last_fullsized":"2024-09-16T00:11:15.749363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":31221,"num_write":370,"num_write_kb":76143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"125'186","reported_seq":482,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749569+0000","last_change":"2024-09-16T00:11:04.894640+0000","last_active":"2024-09-16T00:11:15.749569+0000","last_peered":"2024-09-16T00:11:15.749569+0000","last_clean":"2024-09-16T00:11:15.749569+0000","last_became_active":"2024-09-16T00:11:04.894163+0000","last_became_peered":"2024-09-16T00:11:04.894163+0000","last_unstale":"2024-09-16T00:11:15.749569+0000","last_undegraded":"2024-09-16T00:11:15.749569+0000","last_fullsized":"2024-09-16T00:11:15.749569+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:41.395352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":345,"num_write_kb":1559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":2964,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"123'344","reported_seq":786,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.750028+0000","last_change":"2024-09-16T00:10:56.334688+0000","last_active":"2024-09-16T00:11:15.750028+0000","last_peered":"2024-09-16T00:11:15.750028+0000","last_clean":"2024-09-16T00:11:15.750028+0000","last_became_active":"2024-09-16T00:10:55.845451+0000","last_became_peered":"2024-09-16T00:10:55.845451+0000","last_unstale":"2024-09-16T00:11:15.750028+0000","last_undegraded":"2024-09-16T00:11:15.750028+0000","last_fullsized":"2024-09-16T00:11:15.750028+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":344,"log_dups_size":0,"ondisk_log_size":344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:04.965027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":19274,"num_write":323,"num_write_kb":22071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"123'63","reported_seq":309,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011696+0000","last_change":"2024-09-16T00:10:55.858845+0000","last_active":"2024-09-16T00:11:14.011696+0000","last_peered":"2024-09-16T00:11:14.011696+0000","last_clean":"2024-09-16T00:11:14.011696+0000","last_became_active":"2024-09-16T00:10:55.845687+0000","last_became_peered":"2024-09-16T00:10:55.845687+0000","last_unstale":"2024-09-16T00:11:14.011696+0000","last_undegraded":"2024-09-16T00:11:14.011696+0000","last_fullsized":"2024-09-16T00:11:14.011696+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:53.322675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"123'447","reported_seq":750,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381154+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:11:14.381154+0000","last_peered":"2024-09-16T00:11:14.381154+0000","last_clean":"2024-09-16T00:11:14.381154+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:11:14.381154+0000","last_undegraded":"2024-09-16T00:11:14.381154+0000","last_fullsized":"2024-09-16T00:11:14.381154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":17894,"num_write":426,"num_write_kb":88843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"123'564","reported_seq":786,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.903856+0000","last_change":"2024-09-16T00:10:55.852895+0000","last_active":"2024-09-16T00:11:14.903856+0000","last_peered":"2024-09-16T00:11:14.903856+0000","last_clean":"2024-09-16T00:11:14.903856+0000","last_became_active":"2024-09-16T00:10:55.828121+0000","last_became_peered":"2024-09-16T00:10:55.828121+0000","last_unstale":"2024-09-16T00:11:14.903856+0000","last_undegraded":"2024-09-16T00:11:14.903856+0000","last_fullsized":"2024-09-16T00:11:14.903856+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:57.000428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"123'454","reported_seq":732,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951802+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:11:13.951802+0000","last_peered":"2024-09-16T00:11:13.951802+0000","last_clean":"2024-09-16T00:11:13.951802+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:11:13.951802+0000","last_undegraded":"2024-09-16T00:11:13.951802+0000","last_fullsized":"2024-09-16T00:11:13.951802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":9206,"num_write":434,"num_write_kb":79714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"122'57","reported_seq":329,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749408+0000","last_change":"2024-09-16T00:10:55.850694+0000","last_active":"2024-09-16T00:11:15.749408+0000","last_peered":"2024-09-16T00:11:15.749408+0000","last_clean":"2024-09-16T00:11:15.749408+0000","last_became_active":"2024-09-16T00:10:55.837749+0000","last_became_peered":"2024-09-16T00:10:55.837749+0000","last_unstale":"2024-09-16T00:11:15.749408+0000","last_undegraded":"2024-09-16T00:11:15.749408+0000","last_fullsized":"2024-09-16T00:11:15.749408+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:53.941848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"123'382","reported_seq":1527,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749454+0000","last_change":"2024-09-16T00:11:04.892185+0000","last_active":"2024-09-16T00:11:15.749454+0000","last_peered":"2024-09-16T00:11:15.749454+0000","last_clean":"2024-09-16T00:11:15.749454+0000","last_became_active":"2024-09-16T00:11:04.890897+0000","last_became_peered":"2024-09-16T00:11:04.890897+0000","last_unstale":"2024-09-16T00:11:15.749454+0000","last_undegraded":"2024-09-16T00:11:15.749454+0000","last_fullsized":"2024-09-16T00:11:15.749454+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":382,"log_dups_size":0,"ondisk_log_size":382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:18.504239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":8384,"num_write":359,"num_write_kb":50710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":421,"num_bytes_recovered":615552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"122'180","reported_seq":469,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749303+0000","last_change":"2024-09-16T00:11:04.891195+0000","last_active":"2024-09-16T00:11:15.749303+0000","last_peered":"2024-09-16T00:11:15.749303+0000","last_clean":"2024-09-16T00:11:15.749303+0000","last_became_active":"2024-09-16T00:11:04.890722+0000","last_became_peered":"2024-09-16T00:11:04.890722+0000","last_unstale":"2024-09-16T00:11:15.749303+0000","last_undegraded":"2024-09-16T00:11:15.749303+0000","last_fullsized":"2024-09-16T00:11:15.749303+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:36.844666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":4191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":5577662,"num_keys_recovered":51,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"123'420","reported_seq":750,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749807+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:11:15.749807+0000","last_peered":"2024-09-16T00:11:15.749807+0000","last_clean":"2024-09-16T00:11:15.749807+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:11:15.749807+0000","last_undegraded":"2024-09-16T00:11:15.749807+0000","last_fullsized":"2024-09-16T00:11:15.749807+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":420,"log_dups_size":0,"ondisk_log_size":420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":37740,"num_write":400,"num_write_kb":67449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"122'114","reported_seq":325,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.903984+0000","last_change":"2024-09-16T00:10:55.854939+0000","last_active":"2024-09-16T00:11:14.903984+0000","last_peered":"2024-09-16T00:11:14.903984+0000","last_clean":"2024-09-16T00:11:14.903984+0000","last_became_active":"2024-09-16T00:10:55.832841+0000","last_became_peered":"2024-09-16T00:10:55.832841+0000","last_unstale":"2024-09-16T00:11:14.903984+0000","last_undegraded":"2024-09-16T00:11:14.903984+0000","last_fullsized":"2024-09-16T00:11:14.903984+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:06:48.016972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":8303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"123'452","reported_seq":608,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011438+0000","last_change":"2024-09-16T00:10:00.699942+0000","last_active":"2024-09-16T00:11:14.011438+0000","last_peered":"2024-09-16T00:11:14.011438+0000","last_clean":"2024-09-16T00:11:14.011438+0000","last_became_active":"2024-09-16T00:10:00.281972+0000","last_became_peered":"2024-09-16T00:10:00.281972+0000","last_unstale":"2024-09-16T00:11:14.011438+0000","last_undegraded":"2024-09-16T00:11:14.011438+0000","last_fullsized":"2024-09-16T00:11:14.011438+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:21.943709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":43345,"num_write":425,"num_write_kb":121191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":29546741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"123'396","reported_seq":725,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952254+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:11:13.952254+0000","last_peered":"2024-09-16T00:11:13.952254+0000","last_clean":"2024-09-16T00:11:13.952254+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:11:13.952254+0000","last_undegraded":"2024-09-16T00:11:13.952254+0000","last_fullsized":"2024-09-16T00:11:13.952254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":32882,"num_write":368,"num_write_kb":108251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"123'219","reported_seq":438,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.903959+0000","last_change":"2024-09-16T00:10:55.835894+0000","last_active":"2024-09-16T00:11:14.903959+0000","last_peered":"2024-09-16T00:11:14.903959+0000","last_clean":"2024-09-16T00:11:14.903959+0000","last_became_active":"2024-09-16T00:10:55.835301+0000","last_became_peered":"2024-09-16T00:10:55.835301+0000","last_unstale":"2024-09-16T00:11:14.903959+0000","last_undegraded":"2024-09-16T00:11:14.903959+0000","last_fullsized":"2024-09-16T00:11:14.903959+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:32.623825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":4194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"123'385","reported_seq":714,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951587+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:11:13.951587+0000","last_peered":"2024-09-16T00:11:13.951587+0000","last_clean":"2024-09-16T00:11:13.951587+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:11:13.951587+0000","last_undegraded":"2024-09-16T00:11:13.951587+0000","last_fullsized":"2024-09-16T00:11:13.951587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":36278,"num_write":367,"num_write_kb":113361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"123'263","reported_seq":494,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.904030+0000","last_change":"2024-09-16T00:10:55.856431+0000","last_active":"2024-09-16T00:11:14.904030+0000","last_peered":"2024-09-16T00:11:14.904030+0000","last_clean":"2024-09-16T00:11:14.904030+0000","last_became_active":"2024-09-16T00:10:55.843843+0000","last_became_peered":"2024-09-16T00:10:55.843843+0000","last_unstale":"2024-09-16T00:11:14.904030+0000","last_undegraded":"2024-09-16T00:11:14.904030+0000","last_fullsized":"2024-09-16T00:11:14.904030+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:11.989047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":197,"num_write_kb":12428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":30,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"146'354","reported_seq":607,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.032219+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:11:19.032219+0000","last_peered":"2024-09-16T00:11:19.032219+0000","last_clean":"2024-09-16T00:11:19.032219+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:11:19.032219+0000","last_undegraded":"2024-09-16T00:11:19.032219+0000","last_fullsized":"2024-09-16T00:11:19.032219+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":12352,"num_write":323,"num_write_kb":38097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"122'93","reported_seq":356,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.750053+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:11:15.750053+0000","last_peered":"2024-09-16T00:11:15.750053+0000","last_clean":"2024-09-16T00:11:15.750053+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:11:15.750053+0000","last_undegraded":"2024-09-16T00:11:15.750053+0000","last_fullsized":"2024-09-16T00:11:15.750053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":4141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"123'414","reported_seq":795,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749958+0000","last_change":"2024-09-16T00:10:00.556848+0000","last_active":"2024-09-16T00:11:15.749958+0000","last_peered":"2024-09-16T00:11:15.749958+0000","last_clean":"2024-09-16T00:11:15.749958+0000","last_became_active":"2024-09-16T00:10:00.283734+0000","last_became_peered":"2024-09-16T00:10:00.283734+0000","last_unstale":"2024-09-16T00:11:15.749958+0000","last_undegraded":"2024-09-16T00:11:15.749958+0000","last_fullsized":"2024-09-16T00:11:15.749958+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:55:01.843183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":40163,"num_write":392,"num_write_kb":81063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":21131701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"123'98","reported_seq":372,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381221+0000","last_change":"2024-09-16T00:10:55.856802+0000","last_active":"2024-09-16T00:11:14.381221+0000","last_peered":"2024-09-16T00:11:14.381221+0000","last_clean":"2024-09-16T00:11:14.381221+0000","last_became_active":"2024-09-16T00:10:55.844922+0000","last_became_peered":"2024-09-16T00:10:55.844922+0000","last_unstale":"2024-09-16T00:11:14.381221+0000","last_undegraded":"2024-09-16T00:11:14.381221+0000","last_fullsized":"2024-09-16T00:11:14.381221+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:19.134467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":4138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2561,"num_read_kb":1362012,"num_write":24417,"num_write_kb":4088607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3177,"num_bytes_recovered":294487447,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5234,"internal_metadata":0},"log_size":25968,"ondisk_log_size":25968,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":129048453,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":176,"num_read_kb":1534,"num_write":5618,"num_write_kb":131059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":26562382,"num_keys_recovered":3117,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86642688,"data_stored":258096906,"data_compressed":38739888,"data_compressed_allocated":49811456,"data_compressed_original":221323264,"omap_allocated":262411,"internal_metadata":0},"log_size":7144,"ondisk_log_size":7144,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":127,"seq":545460846599,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57996,"kb_used_data":16056,"kb_used_omap":42,"kb_used_meta":41877,"kb_avail":94313844,"statfs":{"total":96636764160,"available":96577376256,"internally_reserved":0,"allocated":16441344,"data_stored":51769079,"data_compressed":7683094,"data_compressed_allocated":9629696,"data_compressed_original":45862912,"omap_allocated":43696,"internal_metadata":42882384},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":94,"seq":403726925842,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56032,"kb_used_data":14280,"kb_used_omap":221,"kb_used_meta":41506,"kb_avail":94315808,"statfs":{"total":96636764160,"available":96579387392,"internally_reserved":0,"allocated":14622720,"data_stored":44546809,"data_compressed":6872875,"data_compressed_allocated":8704000,"data_compressed_original":39534592,"omap_allocated":227133,"internal_metadata":42502339},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73848,"kb_used_data":12128,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94297992,"statfs":{"total":96636764160,"available":96561143808,"internally_reserved":0,"allocated":12419072,"data_stored":26012395,"data_compressed":3584779,"data_compressed_allocated":4849664,"data_compressed_original":19345408,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509492,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63496,"kb_used_data":18412,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94308344,"statfs":{"total":96636764160,"available":96571744256,"internally_reserved":0,"allocated":18853888,"data_stored":46983951,"data_compressed":6735741,"data_compressed_allocated":8769536,"data_compressed_original":37810176,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509490,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51048,"kb_used_data":11540,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320792,"statfs":{"total":96636764160,"available":96584491008,"internally_reserved":0,"allocated":11816960,"data_stored":33425144,"data_compressed":5001493,"data_compressed_allocated":6361088,"data_compressed_original":28876800,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542195,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67128,"kb_used_data":9564,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94304712,"statfs":{"total":96636764160,"available":96568025088,"internally_reserved":0,"allocated":9793536,"data_stored":26006313,"data_compressed":3864818,"data_compressed_allocated":4833280,"data_compressed_original":21946368,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542195,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57372,"kb_used_data":12292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94314468,"statfs":{"total":96636764160,"available":96578015232,"internally_reserved":0,"allocated":12587008,"data_stored":30797615,"data_compressed":4284061,"data_compressed_allocated":5672960,"data_compressed_original":24797184,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35272,"kb_used_data":4088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336568,"statfs":{"total":96636764160,"available":96600645632,"internally_reserved":0,"allocated":4186112,"data_stored":6502032,"data_compressed":777781,"data_compressed_allocated":1081344,"data_compressed_original":4296704,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10776576,"data_stored":29360766,"data_compressed":4251684,"data_compressed_allocated":5627904,"data_compressed_original":24223744,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8060928,"data_stored":25165984,"data_compressed":3864818,"data_compressed_allocated":4833280,"data_compressed_original":21946368,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2441216,"data_stored":5655551,"data_compressed":777781,"data_compressed_allocated":1081344,"data_compressed_original":4296704,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10072064,"data_stored":32578663,"data_compressed":5001493,"data_compressed_allocated":6361088,"data_compressed_original":28876800,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17108992,"data_stored":46137470,"data_compressed":6735741,"data_compressed_allocated":8769536,"data_compressed_original":37810176,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10674176,"data_stored":25165914,"data_compressed":3584779,"data_compressed_allocated":4849664,"data_compressed_original":19345408,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12877824,"data_stored":43700328,"data_compressed":6872875,"data_compressed_allocated":8704000,"data_compressed_original":39534592,"omap_allocated":225539,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14630912,"data_stored":50332230,"data_compressed":7650717,"data_compressed_allocated":9584640,"data_compressed_original":45289472,"omap_allocated":36872,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5234,"internal_metadata":0}]}} 2024-09-16T00:11:20.397 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:11:20.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.527+0000 7fab24c27640 1 Processor -- start 2024-09-16T00:11:20.529 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.528+0000 7fab24c27640 1 -- start start 2024-09-16T00:11:20.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.528+0000 7fab24c27640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fab200c67e0 0x7fab200c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:20.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.528+0000 7fab24c27640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab2006c110 0x7fab2015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:20.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.528+0000 7fab24c27640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fab2006c530 0x7fab2015ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:20.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.528+0000 7fab24c27640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fab2006cd10 con 0x7fab2006c530 2024-09-16T00:11:20.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.528+0000 7fab24c27640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fab200c6bc0 con 0x7fab200c67e0 2024-09-16T00:11:20.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.528+0000 7fab24c27640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fab2015e850 con 0x7fab2006c110 2024-09-16T00:11:20.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.529+0000 7fab1ed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fab2006c530 0x7fab2015ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:20.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.529+0000 7fab1ed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fab2006c530 0x7fab2015ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43432/0 (socket says 172.21.15.139:43432) 2024-09-16T00:11:20.530 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.529+0000 7fab1ed9d640 1 -- 172.21.15.139:0/2603715317 learned_addr learned my addr 172.21.15.139:0/2603715317 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:20.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.529+0000 7fab1dd9b640 1 --2- 172.21.15.139:0/2603715317 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab2006c110 0x7fab2015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:20.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.529+0000 7fab1e59c640 1 --2- 172.21.15.139:0/2603715317 >> v2:172.21.15.164:3300/0 conn(0x7fab200c67e0 0x7fab200c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:20.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.529+0000 7fab1ed9d640 1 -- 172.21.15.139:0/2603715317 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab2006c110 msgr2=0x7fab2015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:20.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.529+0000 7fab1ed9d640 1 --2- 172.21.15.139:0/2603715317 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab2006c110 0x7fab2015a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.529+0000 7fab1ed9d640 1 -- 172.21.15.139:0/2603715317 >> v2:172.21.15.164:3300/0 conn(0x7fab200c67e0 msgr2=0x7fab200c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:20.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.529+0000 7fab1ed9d640 1 --2- 172.21.15.139:0/2603715317 >> v2:172.21.15.164:3300/0 conn(0x7fab200c67e0 0x7fab200c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.529+0000 7fab1ed9d640 1 -- 172.21.15.139:0/2603715317 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2006ca30 con 0x7fab2006c530 2024-09-16T00:11:20.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.530+0000 7fab1ed9d640 1 --2- 172.21.15.139:0/2603715317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab2006c530 0x7fab2015ad50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fab2008ded0 tx=0x7fab1802a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5130c6edede6a4ad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:20.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.530+0000 7fab1d59a640 1 -- 172.21.15.139:0/2603715317 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab18036070 con 0x7fab2006c530 2024-09-16T00:11:20.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.530+0000 7fab1d59a640 1 -- 172.21.15.139:0/2603715317 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab18031600 con 0x7fab2006c530 2024-09-16T00:11:20.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.530+0000 7fab1d59a640 1 -- 172.21.15.139:0/2603715317 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab180319d0 con 0x7fab2006c530 2024-09-16T00:11:20.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.530+0000 7fab24c27640 1 -- 172.21.15.139:0/2603715317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab2006c530 msgr2=0x7fab2015ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:20.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.530+0000 7fab24c27640 1 --2- 172.21.15.139:0/2603715317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab2006c530 0x7fab2015ad50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fab2008ded0 tx=0x7fab1802a750 comp rx=0 tx=0).stop 2024-09-16T00:11:20.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.530+0000 7fab24c27640 1 -- 172.21.15.139:0/2603715317 shutdown_connections 2024-09-16T00:11:20.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.530+0000 7fab24c27640 1 --2- 172.21.15.139:0/2603715317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab2006c530 0x7fab2015ad50 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.530+0000 7fab24c27640 1 --2- 172.21.15.139:0/2603715317 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab2006c110 0x7fab2015a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.530+0000 7fab24c27640 1 --2- 172.21.15.139:0/2603715317 >> v2:172.21.15.164:3300/0 conn(0x7fab200c67e0 0x7fab200c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.530+0000 7fab24c27640 1 -- 172.21.15.139:0/2603715317 >> 172.21.15.139:0/2603715317 conn(0x7fab200d05f0 msgr2=0x7fab200d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:20.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.530+0000 7fab24c27640 1 -- 172.21.15.139:0/2603715317 shutdown_connections 2024-09-16T00:11:20.533 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.530+0000 7fab24c27640 1 -- 172.21.15.139:0/2603715317 wait complete. 2024-09-16T00:11:20.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.532+0000 7fab24c27640 1 Processor -- start 2024-09-16T00:11:20.534 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.532+0000 7fab24c27640 1 -- start start 2024-09-16T00:11:20.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.532+0000 7fab24c27640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab2006c110 0x7fab201594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:20.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.532+0000 7fab24c27640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab2006c530 0x7fab200ce220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:20.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.532+0000 7fab24c27640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab200c67e0 0x7fab201f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:20.536 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.532+0000 7fab24c27640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fab2015e4e0 con 0x7fab2006c110 2024-09-16T00:11:20.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.532+0000 7fab24c27640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fab2015e200 con 0x7fab200c67e0 2024-09-16T00:11:20.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.532+0000 7fab24c27640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fab200cbe50 con 0x7fab2006c530 2024-09-16T00:11:20.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.532+0000 7fab1e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab2006c110 0x7fab201594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:20.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.532+0000 7fab1e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab2006c110 0x7fab201594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43436/0 (socket says 172.21.15.139:43436) 2024-09-16T00:11:20.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.532+0000 7fab1e59c640 1 -- 172.21.15.139:0/966098900 learned_addr learned my addr 172.21.15.139:0/966098900 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:20.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.533+0000 7fab1dd9b640 1 --2- 172.21.15.139:0/966098900 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab2006c530 0x7fab200ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:20.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.533+0000 7fab1ed9d640 1 --2- 172.21.15.139:0/966098900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab200c67e0 0x7fab201f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:20.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.533+0000 7fab1e59c640 1 -- 172.21.15.139:0/966098900 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab2006c530 msgr2=0x7fab200ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:20.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.533+0000 7fab1e59c640 1 --2- 172.21.15.139:0/966098900 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab2006c530 0x7fab200ce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.533+0000 7fab1e59c640 1 -- 172.21.15.139:0/966098900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab200c67e0 msgr2=0x7fab201f04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:20.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.533+0000 7fab1e59c640 1 --2- 172.21.15.139:0/966098900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab200c67e0 0x7fab201f04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.533+0000 7fab1e59c640 1 -- 172.21.15.139:0/966098900 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab201e62c0 con 0x7fab2006c110 2024-09-16T00:11:20.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.533+0000 7fab1e59c640 1 --2- 172.21.15.139:0/966098900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab2006c110 0x7fab201594a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fab1000e870 tx=0x7fab1000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:20.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.533+0000 7fab16ffd640 1 -- 172.21.15.139:0/966098900 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10017070 con 0x7fab2006c110 2024-09-16T00:11:20.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.533+0000 7fab16ffd640 1 -- 172.21.15.139:0/966098900 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab1000ee60 con 0x7fab2006c110 2024-09-16T00:11:20.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.533+0000 7fab24c27640 1 -- 172.21.15.139:0/966098900 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab201e65a0 con 0x7fab2006c110 2024-09-16T00:11:20.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.533+0000 7fab16ffd640 1 -- 172.21.15.139:0/966098900 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1000cb00 con 0x7fab2006c110 2024-09-16T00:11:20.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.533+0000 7fab24c27640 1 -- 172.21.15.139:0/966098900 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab20156b60 con 0x7fab2006c110 2024-09-16T00:11:20.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.534+0000 7fab24c27640 1 -- 172.21.15.139:0/966098900 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab200c9650 con 0x7fab2006c110 2024-09-16T00:11:20.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.535+0000 7fab16ffd640 1 -- 172.21.15.139:0/966098900 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fab1001a070 con 0x7fab2006c110 2024-09-16T00:11:20.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.535+0000 7fab16ffd640 1 --2- 172.21.15.139:0/966098900 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fab0006c2b0 0x7fab0006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:20.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.535+0000 7fab16ffd640 1 -- 172.21.15.139:0/966098900 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7fab1008b180 con 0x7fab2006c110 2024-09-16T00:11:20.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.535+0000 7fab1dd9b640 1 --2- 172.21.15.139:0/966098900 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fab0006c2b0 0x7fab0006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:20.538 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.535+0000 7fab1dd9b640 1 --2- 172.21.15.139:0/966098900 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fab0006c2b0 0x7fab0006e770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fab200c2c10 tx=0x7fab08009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:20.540 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.539+0000 7fab16ffd640 1 -- 172.21.15.139:0/966098900 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fab200c9650 con 0x7fab2006c110 2024-09-16T00:11:20.707 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.706+0000 7fab24c27640 1 -- 172.21.15.139:0/966098900 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab200cc220 con 0x7fab2006c110 2024-09-16T00:11:20.708 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.707+0000 7fab16ffd640 1 -- 172.21.15.139:0/966098900 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+17050 (secure 0 0 0) 0x7fab200cc220 con 0x7fab2006c110 2024-09-16T00:11:20.708 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:11:20.709 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":146,"fsid":"2d3366e5-6a59-4b59-93b2-615e72ac7015","created":"2024-09-16T00:07:09.967833+0000","modified":"2024-09-16T00:11:13.946870+0000","last_up_change":"2024-09-16T00:10:54.809507+0000","last_in_change":"2024-09-16T00:07:12.025620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:07:22.056303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:07:30.768649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:07:32.008209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83699a05-4937-43c3-aefc-b669bed9bb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6817","nonce":1974901438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6819","nonce":1974901438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6823","nonce":1974901438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6821","nonce":1974901438}]},"public_addr":"172.21.15.139:6817/1974901438","cluster_addr":"172.21.15.139:6819/1974901438","heartbeat_back_addr":"172.21.15.139:6823/1974901438","heartbeat_front_addr":"172.21.15.139:6821/1974901438","state":["exists","up"]},{"osd":1,"uuid":"9c04991e-9695-4bbb-8011-bf613bf404b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6809","nonce":2232380998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6811","nonce":2232380998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6815","nonce":2232380998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6813","nonce":2232380998}]},"public_addr":"172.21.15.139:6809/2232380998","cluster_addr":"172.21.15.139:6811/2232380998","heartbeat_back_addr":"172.21.15.139:6815/2232380998","heartbeat_front_addr":"172.21.15.139:6813/2232380998","state":["exists","up"]},{"osd":2,"uuid":"8048fc64-72b3-4866-ac62-6a8b2bf6f5c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6801","nonce":4180588905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6803","nonce":4180588905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6807","nonce":4180588905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6805","nonce":4180588905}]},"public_addr":"172.21.15.139:6801/4180588905","cluster_addr":"172.21.15.139:6803/4180588905","heartbeat_back_addr":"172.21.15.139:6807/4180588905","heartbeat_front_addr":"172.21.15.139:6805/4180588905","state":["exists","up"]},{"osd":3,"uuid":"c34ac77b-354c-4884-902d-3e2a84f5eb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6827","nonce":3947768492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6829","nonce":3947768492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6833","nonce":3947768492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6831","nonce":3947768492}]},"public_addr":"172.21.15.139:6827/3947768492","cluster_addr":"172.21.15.139:6829/3947768492","heartbeat_back_addr":"172.21.15.139:6833/3947768492","heartbeat_front_addr":"172.21.15.139:6831/3947768492","state":["exists","up"]},{"osd":4,"uuid":"ba9a7c93-9672-43e1-8c98-2878cdf7f548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6817","nonce":642632916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6819","nonce":642632916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6823","nonce":642632916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6821","nonce":642632916}]},"public_addr":"172.21.15.164:6817/642632916","cluster_addr":"172.21.15.164:6819/642632916","heartbeat_back_addr":"172.21.15.164:6823/642632916","heartbeat_front_addr":"172.21.15.164:6821/642632916","state":["exists","up"]},{"osd":5,"uuid":"46a0916b-474c-411b-84ad-86f27d4cbf53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6801","nonce":1051778383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6803","nonce":1051778383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6807","nonce":1051778383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6805","nonce":1051778383}]},"public_addr":"172.21.15.164:6801/1051778383","cluster_addr":"172.21.15.164:6803/1051778383","heartbeat_back_addr":"172.21.15.164:6807/1051778383","heartbeat_front_addr":"172.21.15.164:6805/1051778383","state":["exists","up"]},{"osd":6,"uuid":"9b6b33cc-d6d9-4a0a-8143-526073a58b93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":82,"up_from":94,"up_thru":127,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":164138206},{"type":"v1","addr":"172.21.15.164:6825","nonce":164138206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":164138206},{"type":"v1","addr":"172.21.15.164:6827","nonce":164138206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":164138206},{"type":"v1","addr":"172.21.15.164:6831","nonce":164138206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":164138206},{"type":"v1","addr":"172.21.15.164:6829","nonce":164138206}]},"public_addr":"172.21.15.164:6825/164138206","cluster_addr":"172.21.15.164:6827/164138206","heartbeat_back_addr":"172.21.15.164:6831/164138206","heartbeat_front_addr":"172.21.15.164:6829/164138206","state":["exists","up"]},{"osd":7,"uuid":"50bcd808-3d3e-4a72-88e4-30decb53cfea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":112,"up_from":127,"up_thru":136,"down_at":113,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2782017910},{"type":"v1","addr":"172.21.15.164:6809","nonce":2782017910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2782017910},{"type":"v1","addr":"172.21.15.164:6811","nonce":2782017910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2782017910},{"type":"v1","addr":"172.21.15.164:6815","nonce":2782017910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2782017910},{"type":"v1","addr":"172.21.15.164:6813","nonce":2782017910}]},"public_addr":"172.21.15.164:6809/2782017910","cluster_addr":"172.21.15.164:6811/2782017910","heartbeat_back_addr":"172.21.15.164:6815/2782017910","heartbeat_front_addr":"172.21.15.164:6813/2782017910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.843117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.622789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:17.919457+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:19.330341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.815640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.083046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.551499+0000","dead_epoch":84},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.377179+0000","dead_epoch":113}],"pg_upmap":[{"pgid":"3.2d","osds":[7,5]}],"pg_upmap_items":[{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:11:20.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.709+0000 7fab24c27640 1 -- 172.21.15.139:0/966098900 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fab0006c2b0 msgr2=0x7fab0006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:20.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.709+0000 7fab24c27640 1 --2- 172.21.15.139:0/966098900 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fab0006c2b0 0x7fab0006e770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fab200c2c10 tx=0x7fab08009190 comp rx=0 tx=0).stop 2024-09-16T00:11:20.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.709+0000 7fab24c27640 1 -- 172.21.15.139:0/966098900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab2006c110 msgr2=0x7fab201594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:20.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.709+0000 7fab24c27640 1 --2- 172.21.15.139:0/966098900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab2006c110 0x7fab201594a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fab1000e870 tx=0x7fab1000ed30 comp rx=0 tx=0).stop 2024-09-16T00:11:20.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.709+0000 7fab24c27640 1 -- 172.21.15.139:0/966098900 shutdown_connections 2024-09-16T00:11:20.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.709+0000 7fab24c27640 1 --2- 172.21.15.139:0/966098900 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fab0006c2b0 0x7fab0006e770 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.711 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.710+0000 7fab24c27640 1 --2- 172.21.15.139:0/966098900 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fab200c67e0 0x7fab201f04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.710+0000 7fab24c27640 1 --2- 172.21.15.139:0/966098900 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fab2006c530 0x7fab200ce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.710+0000 7fab24c27640 1 --2- 172.21.15.139:0/966098900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab2006c110 0x7fab201594a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:20.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.710+0000 7fab24c27640 1 -- 172.21.15.139:0/966098900 >> 172.21.15.139:0/966098900 conn(0x7fab200d05f0 msgr2=0x7fab200cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:20.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.710+0000 7fab24c27640 1 -- 172.21.15.139:0/966098900 shutdown_connections 2024-09-16T00:11:20.712 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:20.710+0000 7fab24c27640 1 -- 172.21.15.139:0/966098900 wait complete. 2024-09-16T00:11:21.723 INFO:tasks.ceph:Scrubbing osd.0 2024-09-16T00:11:21.723 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-16T00:11:21.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.840+0000 7fea82f49640 1 Processor -- start 2024-09-16T00:11:21.841 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.840+0000 7fea82f49640 1 -- start start 2024-09-16T00:11:21.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.840+0000 7fea82f49640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fea7c06f120 0x7fea7c1511d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.841+0000 7fea82f49640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fea7c06f500 0x7fea7c151710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.841+0000 7fea82f49640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea7c0c79c0 0x7fea7c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.841+0000 7fea82f49640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fea7c0c48e0 con 0x7fea7c06f120 2024-09-16T00:11:21.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.841+0000 7fea82f49640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fea7c1574d0 con 0x7fea7c06f500 2024-09-16T00:11:21.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.841+0000 7fea82f49640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fea7c1577b0 con 0x7fea7c0c79c0 2024-09-16T00:11:21.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.841+0000 7fea80ce5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fea7c06f120 0x7fea7c1511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.841+0000 7fea80ce5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fea7c06f120 0x7fea7c1511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43448/0 (socket says 172.21.15.139:43448) 2024-09-16T00:11:21.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.841+0000 7fea80ce5640 1 -- 172.21.15.139:0/1902106017 learned_addr learned my addr 172.21.15.139:0/1902106017 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:21.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.841+0000 7fea814e6640 1 --2- 172.21.15.139:0/1902106017 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea7c0c79c0 0x7fea7c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.841+0000 7fea7bfff640 1 --2- 172.21.15.139:0/1902106017 >> v2:172.21.15.164:3300/0 conn(0x7fea7c06f500 0x7fea7c151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.843 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.841+0000 7fea80ce5640 1 -- 172.21.15.139:0/1902106017 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea7c0c79c0 msgr2=0x7fea7c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:21.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.841+0000 7fea80ce5640 1 --2- 172.21.15.139:0/1902106017 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea7c0c79c0 0x7fea7c15a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.841+0000 7fea80ce5640 1 -- 172.21.15.139:0/1902106017 >> v2:172.21.15.164:3300/0 conn(0x7fea7c06f500 msgr2=0x7fea7c151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:21.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.841+0000 7fea80ce5640 1 --2- 172.21.15.139:0/1902106017 >> v2:172.21.15.164:3300/0 conn(0x7fea7c06f500 0x7fea7c151710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.841+0000 7fea80ce5640 1 -- 172.21.15.139:0/1902106017 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c157a90 con 0x7fea7c06f120 2024-09-16T00:11:21.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.841+0000 7fea814e6640 1 --2- 172.21.15.139:0/1902106017 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea7c0c79c0 0x7fea7c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:21.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.842+0000 7fea80ce5640 1 --2- 172.21.15.139:0/1902106017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c06f120 0x7fea7c1511d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0ca8b0 tx=0x7fea7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f5d0ef4afa569b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:21.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.842+0000 7fea7b7fe640 1 -- 172.21.15.139:0/1902106017 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70037070 con 0x7fea7c06f120 2024-09-16T00:11:21.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.842+0000 7fea7b7fe640 1 -- 172.21.15.139:0/1902106017 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea70032440 con 0x7fea7c06f120 2024-09-16T00:11:21.844 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.842+0000 7fea7b7fe640 1 -- 172.21.15.139:0/1902106017 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea70032810 con 0x7fea7c06f120 2024-09-16T00:11:21.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.843+0000 7fea82f49640 1 -- 172.21.15.139:0/1902106017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c06f120 msgr2=0x7fea7c1511d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:21.845 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.843+0000 7fea82f49640 1 --2- 172.21.15.139:0/1902106017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c06f120 0x7fea7c1511d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0ca8b0 tx=0x7fea7002a270 comp rx=0 tx=0).stop 2024-09-16T00:11:21.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.843+0000 7fea82f49640 1 -- 172.21.15.139:0/1902106017 shutdown_connections 2024-09-16T00:11:21.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.843+0000 7fea82f49640 1 --2- 172.21.15.139:0/1902106017 >> v2:172.21.15.164:3300/0 conn(0x7fea7c06f500 0x7fea7c151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.843+0000 7fea82f49640 1 --2- 172.21.15.139:0/1902106017 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea7c0c79c0 0x7fea7c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.843+0000 7fea82f49640 1 --2- 172.21.15.139:0/1902106017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c06f120 0x7fea7c1511d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.843+0000 7fea82f49640 1 -- 172.21.15.139:0/1902106017 >> 172.21.15.139:0/1902106017 conn(0x7fea7c070420 msgr2=0x7fea7c070830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:21.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.843+0000 7fea82f49640 1 -- 172.21.15.139:0/1902106017 shutdown_connections 2024-09-16T00:11:21.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.843+0000 7fea82f49640 1 -- 172.21.15.139:0/1902106017 wait complete. 2024-09-16T00:11:21.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.844+0000 7fea82f49640 1 Processor -- start 2024-09-16T00:11:21.846 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.844+0000 7fea82f49640 1 -- start start 2024-09-16T00:11:21.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.845+0000 7fea82f49640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea7c06f120 0x7fea7c06cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.848 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.845+0000 7fea82f49640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea7c06f500 0x7fea7c1e20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.845+0000 7fea82f49640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c0c79c0 0x7fea7c1e25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.845+0000 7fea82f49640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c0c4640 con 0x7fea7c0c79c0 2024-09-16T00:11:21.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.845+0000 7fea82f49640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c0c44c0 con 0x7fea7c06f120 2024-09-16T00:11:21.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.845+0000 7fea82f49640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fea7c06eaf0 con 0x7fea7c06f500 2024-09-16T00:11:21.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.845+0000 7fea814e6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c0c79c0 0x7fea7c1e25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.845+0000 7fea7bfff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea7c06f500 0x7fea7c1e20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.845+0000 7fea814e6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c0c79c0 0x7fea7c1e25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43452/0 (socket says 172.21.15.139:43452) 2024-09-16T00:11:21.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.845+0000 7fea80ce5640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea7c06f120 0x7fea7c06cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.845+0000 7fea814e6640 1 -- 172.21.15.139:0/3344878412 learned_addr learned my addr 172.21.15.139:0/3344878412 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:21.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.846+0000 7fea814e6640 1 -- 172.21.15.139:0/3344878412 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea7c06f500 msgr2=0x7fea7c1e20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:21.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.846+0000 7fea814e6640 1 --2- 172.21.15.139:0/3344878412 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fea7c06f500 0x7fea7c1e20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.846+0000 7fea814e6640 1 -- 172.21.15.139:0/3344878412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea7c06f120 msgr2=0x7fea7c06cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:21.849 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.846+0000 7fea814e6640 1 --2- 172.21.15.139:0/3344878412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea7c06f120 0x7fea7c06cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.846+0000 7fea814e6640 1 -- 172.21.15.139:0/3344878412 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c150200 con 0x7fea7c0c79c0 2024-09-16T00:11:21.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.846+0000 7fea80ce5640 1 --2- 172.21.15.139:0/3344878412 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fea7c06f120 0x7fea7c06cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:11:21.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.846+0000 7fea814e6640 1 --2- 172.21.15.139:0/3344878412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c0c79c0 0x7fea7c1e25f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fea74007ef0 tx=0x7fea7400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:21.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.846+0000 7fea797fa640 1 -- 172.21.15.139:0/3344878412 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7401a070 con 0x7fea7c0c79c0 2024-09-16T00:11:21.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.846+0000 7fea797fa640 1 -- 172.21.15.139:0/3344878412 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea74015600 con 0x7fea7c0c79c0 2024-09-16T00:11:21.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.846+0000 7fea82f49640 1 -- 172.21.15.139:0/3344878412 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7c14fd40 con 0x7fea7c0c79c0 2024-09-16T00:11:21.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.846+0000 7fea797fa640 1 -- 172.21.15.139:0/3344878412 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea740159f0 con 0x7fea7c0c79c0 2024-09-16T00:11:21.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.846+0000 7fea82f49640 1 -- 172.21.15.139:0/3344878412 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea7c1eff00 con 0x7fea7c0c79c0 2024-09-16T00:11:21.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.847+0000 7fea82f49640 1 -- 172.21.15.139:0/3344878412 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea7c0c7060 con 0x7fea7c0c79c0 2024-09-16T00:11:21.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.848+0000 7fea797fa640 1 -- 172.21.15.139:0/3344878412 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fea74005d10 con 0x7fea7c0c79c0 2024-09-16T00:11:21.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.848+0000 7fea797fa640 1 --2- 172.21.15.139:0/3344878412 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fea6006c2e0 0x7fea6006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.848+0000 7fea797fa640 1 -- 172.21.15.139:0/3344878412 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7fea7408bd90 con 0x7fea7c0c79c0 2024-09-16T00:11:21.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.848+0000 7fea80ce5640 1 --2- 172.21.15.139:0/3344878412 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fea6006c2e0 0x7fea6006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.848+0000 7fea797fa640 1 --2- 172.21.15.139:0/3344878412 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fea600747e0 0x7fea60076c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.848+0000 7fea797fa640 1 -- 172.21.15.139:0/3344878412 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea60077350 con 0x7fea600747e0 2024-09-16T00:11:21.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.848+0000 7fea797fa640 1 -- 172.21.15.139:0/3344878412 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7fea7408c220 con 0x7fea7c0c79c0 2024-09-16T00:11:21.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.849+0000 7fea7bfff640 1 --2- 172.21.15.139:0/3344878412 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fea600747e0 0x7fea60076c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.849+0000 7fea80ce5640 1 --2- 172.21.15.139:0/3344878412 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fea6006c2e0 0x7fea6006e7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fea70034000 tx=0x7fea7002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:21.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.849+0000 7fea7bfff640 1 --2- 172.21.15.139:0/3344878412 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fea600747e0 0x7fea60076c80 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:21.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.850+0000 7fea797fa640 1 -- 172.21.15.139:0/3344878412 <== osd.0 v2:172.21.15.139:6816/1974901438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea60077350 con 0x7fea600747e0 2024-09-16T00:11:21.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.862+0000 7fea82f49640 1 -- 172.21.15.139:0/3344878412 --> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fea7c156f20 con 0x7fea600747e0 2024-09-16T00:11:21.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.863+0000 7fea797fa640 1 -- 172.21.15.139:0/3344878412 <== osd.0 v2:172.21.15.139:6816/1974901438 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fea7c156f20 con 0x7fea600747e0 2024-09-16T00:11:21.864 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:11:21.864 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T00:11:21.864 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:11:21.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.863+0000 7fea82f49640 1 -- 172.21.15.139:0/3344878412 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fea600747e0 msgr2=0x7fea60076c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:21.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.863+0000 7fea82f49640 1 --2- 172.21.15.139:0/3344878412 >> [v2:172.21.15.139:6816/1974901438,v1:172.21.15.139:6817/1974901438] conn(0x7fea600747e0 0x7fea60076c80 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.863+0000 7fea82f49640 1 -- 172.21.15.139:0/3344878412 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fea6006c2e0 msgr2=0x7fea6006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:21.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.863+0000 7fea82f49640 1 --2- 172.21.15.139:0/3344878412 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fea6006c2e0 0x7fea6006e7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fea70034000 tx=0x7fea7002e000 comp rx=0 tx=0).stop 2024-09-16T00:11:21.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.863+0000 7fea82f49640 1 -- 172.21.15.139:0/3344878412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c0c79c0 msgr2=0x7fea7c1e25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:21.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.863+0000 7fea82f49640 1 --2- 172.21.15.139:0/3344878412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea7c0c79c0 0x7fea7c1e25f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fea74007ef0 tx=0x7fea7400d4a0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.865 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.863+0000 7fea814e6640 1 -- 172.21.15.139:0/3344878412 reap_dead start 2024-09-16T00:11:21.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.863+0000 7fea82f49640 1 -- 172.21.15.139:0/3344878412 shutdown_connections 2024-09-16T00:11:21.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.863+0000 7fea82f49640 1 -- 172.21.15.139:0/3344878412 >> 172.21.15.139:0/3344878412 conn(0x7fea7c070420 msgr2=0x7fea7c150fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:21.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.863+0000 7fea82f49640 1 -- 172.21.15.139:0/3344878412 shutdown_connections 2024-09-16T00:11:21.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.863+0000 7fea82f49640 1 -- 172.21.15.139:0/3344878412 wait complete. 2024-09-16T00:11:21.875 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-16T00:11:21.976 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.975+0000 7f104817f640 1 Processor -- start 2024-09-16T00:11:21.977 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.976+0000 7f104817f640 1 -- start start 2024-09-16T00:11:21.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.976+0000 7f104817f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1040154c20 0x7f104015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.977+0000 7f104817f640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f10400ccc90 0x7f10400cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.977+0000 7f104817f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10400cc4b0 con 0x7f1040068060 2024-09-16T00:11:21.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.977+0000 7f104817f640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f10400c5830 con 0x7f10400ccc90 2024-09-16T00:11:21.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.977+0000 7f104817f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f104015b7c0 con 0x7f1040154c20 2024-09-16T00:11:21.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.977+0000 7f1045f1b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1040154c20 0x7f104015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.977+0000 7f1045f1b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1040154c20 0x7f104015b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:44634/0 (socket says 172.21.15.139:44634) 2024-09-16T00:11:21.979 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.977+0000 7f1045f1b640 1 -- 172.21.15.139:0/2463108125 learned_addr learned my addr 172.21.15.139:0/2463108125 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:21.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.977+0000 7f104671c640 1 --2- 172.21.15.139:0/2463108125 >> v2:172.21.15.164:3300/0 conn(0x7f10400ccc90 0x7f10400cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.977+0000 7f1045f1b640 1 -- 172.21.15.139:0/2463108125 >> v2:172.21.15.164:3300/0 conn(0x7f10400ccc90 msgr2=0x7f10400cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:21.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.977+0000 7f1045f1b640 1 --2- 172.21.15.139:0/2463108125 >> v2:172.21.15.164:3300/0 conn(0x7f10400ccc90 0x7f10400cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.977+0000 7f1045f1b640 1 -- 172.21.15.139:0/2463108125 >> v1:172.21.15.164:6789/0 conn(0x7f1040068060 legacy=0x7f10400c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:21.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.977+0000 7f1045f1b640 1 -- 172.21.15.139:0/2463108125 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10400cc790 con 0x7f1040154c20 2024-09-16T00:11:21.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.978+0000 7f1045f1b640 1 --2- 172.21.15.139:0/2463108125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1040154c20 0x7f104015b280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1030009870 tx=0x7f103002a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb3f26fc713747bf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:21.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.978+0000 7f1044f19640 1 -- 172.21.15.139:0/2463108125 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030037070 con 0x7f1040154c20 2024-09-16T00:11:21.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.978+0000 7f1044f19640 1 -- 172.21.15.139:0/2463108125 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f103002ae50 con 0x7f1040154c20 2024-09-16T00:11:21.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.978+0000 7f1044f19640 1 -- 172.21.15.139:0/2463108125 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030032860 con 0x7f1040154c20 2024-09-16T00:11:21.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.979+0000 7f104817f640 1 -- 172.21.15.139:0/2463108125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1040154c20 msgr2=0x7f104015b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:21.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.979+0000 7f104817f640 1 --2- 172.21.15.139:0/2463108125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1040154c20 0x7f104015b280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1030009870 tx=0x7f103002a300 comp rx=0 tx=0).stop 2024-09-16T00:11:21.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.979+0000 7f104817f640 1 -- 172.21.15.139:0/2463108125 shutdown_connections 2024-09-16T00:11:21.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.979+0000 7f104817f640 1 --2- 172.21.15.139:0/2463108125 >> v2:172.21.15.164:3300/0 conn(0x7f10400ccc90 0x7f10400cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.979+0000 7f104817f640 1 --2- 172.21.15.139:0/2463108125 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f1040154c20 0x7f104015b280 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.979+0000 7f104817f640 1 -- 172.21.15.139:0/2463108125 >> 172.21.15.139:0/2463108125 conn(0x7f104006ca40 msgr2=0x7f1040067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.979+0000 7f104817f640 1 -- 172.21.15.139:0/2463108125 shutdown_connections 2024-09-16T00:11:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.979+0000 7f104817f640 1 -- 172.21.15.139:0/2463108125 wait complete. 2024-09-16T00:11:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.980+0000 7f104817f640 1 Processor -- start 2024-09-16T00:11:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.980+0000 7f104817f640 1 -- start start 2024-09-16T00:11:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f104817f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1040068060 0x7f10400b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f104817f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10400ccc90 0x7f10400b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f104817f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1040154c20 0x7f1040153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f104817f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f104015e0d0 con 0x7f1040154c20 2024-09-16T00:11:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f104817f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f10400cc260 con 0x7f1040068060 2024-09-16T00:11:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f104817f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f104015e3b0 con 0x7f10400ccc90 2024-09-16T00:11:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f104671c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1040154c20 0x7f1040153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f104671c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1040154c20 0x7f1040153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43458/0 (socket says 172.21.15.139:43458) 2024-09-16T00:11:21.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f1045f1b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1040068060 0x7f10400b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f104671c640 1 -- 172.21.15.139:0/1891113904 learned_addr learned my addr 172.21.15.139:0/1891113904 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:21.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f104571a640 1 --2- 172.21.15.139:0/1891113904 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10400ccc90 0x7f10400b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f104671c640 1 -- 172.21.15.139:0/1891113904 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10400ccc90 msgr2=0x7f10400b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:21.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f104671c640 1 --2- 172.21.15.139:0/1891113904 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10400ccc90 0x7f10400b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f104671c640 1 -- 172.21.15.139:0/1891113904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1040068060 msgr2=0x7f10400b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:21.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f104671c640 1 --2- 172.21.15.139:0/1891113904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1040068060 0x7f10400b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:21.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f104671c640 1 -- 172.21.15.139:0/1891113904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10400b48f0 con 0x7f1040154c20 2024-09-16T00:11:21.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.981+0000 7f104671c640 1 --2- 172.21.15.139:0/1891113904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1040154c20 0x7f1040153700 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f103800efc0 tx=0x7f103800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:21.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.983+0000 7f10367fc640 1 -- 172.21.15.139:0/1891113904 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038018020 con 0x7f1040154c20 2024-09-16T00:11:21.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.983+0000 7f10367fc640 1 -- 172.21.15.139:0/1891113904 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1038009b10 con 0x7f1040154c20 2024-09-16T00:11:21.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.983+0000 7f104817f640 1 -- 172.21.15.139:0/1891113904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1040148610 con 0x7f1040154c20 2024-09-16T00:11:21.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.983+0000 7f10367fc640 1 -- 172.21.15.139:0/1891113904 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10380146c0 con 0x7f1040154c20 2024-09-16T00:11:21.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.984+0000 7f104817f640 1 -- 172.21.15.139:0/1891113904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104014a560 con 0x7f1040154c20 2024-09-16T00:11:21.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.985+0000 7f104817f640 1 -- 172.21.15.139:0/1891113904 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f104015e690 con 0x7f1040154c20 2024-09-16T00:11:21.986 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.985+0000 7f10367fc640 1 -- 172.21.15.139:0/1891113904 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1038005d10 con 0x7f1040154c20 2024-09-16T00:11:21.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.986+0000 7f10367fc640 1 --2- 172.21.15.139:0/1891113904 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f102406c2e0 0x7f102406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:21.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.986+0000 7f10367fc640 1 -- 172.21.15.139:0/1891113904 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f103808bdb0 con 0x7f1040154c20 2024-09-16T00:11:21.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.986+0000 7f1045f1b640 1 --2- 172.21.15.139:0/1891113904 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f102406c2e0 0x7f102406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:21.987 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.986+0000 7f1045f1b640 1 --2- 172.21.15.139:0/1891113904 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f102406c2e0 0x7f102406e7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1030034000 tx=0x7f103002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:21.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:21.990+0000 7f10367fc640 1 -- 172.21.15.139:0/1891113904 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1038022070 con 0x7f1040154c20 2024-09-16T00:11:22.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.164+0000 7f104817f640 1 -- 172.21.15.139:0/1891113904 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f10400cca70 con 0x7f102406c2e0 2024-09-16T00:11:22.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.165+0000 7f10367fc640 1 -- 172.21.15.139:0/1891113904 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f10400cca70 con 0x7f102406c2e0 2024-09-16T00:11:22.166 INFO:teuthology.orchestra.run.smithi139.stderr:instructed osd(s) 0 to deep-scrub 2024-09-16T00:11:22.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.167+0000 7f104817f640 1 -- 172.21.15.139:0/1891113904 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f102406c2e0 msgr2=0x7f102406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.167+0000 7f104817f640 1 --2- 172.21.15.139:0/1891113904 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f102406c2e0 0x7f102406e7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1030034000 tx=0x7f103002e000 comp rx=0 tx=0).stop 2024-09-16T00:11:22.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.167+0000 7f104817f640 1 -- 172.21.15.139:0/1891113904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1040154c20 msgr2=0x7f1040153700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.167+0000 7f104817f640 1 --2- 172.21.15.139:0/1891113904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1040154c20 0x7f1040153700 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f103800efc0 tx=0x7f103800c490 comp rx=0 tx=0).stop 2024-09-16T00:11:22.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.167+0000 7f104817f640 1 -- 172.21.15.139:0/1891113904 shutdown_connections 2024-09-16T00:11:22.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.167+0000 7f104817f640 1 --2- 172.21.15.139:0/1891113904 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f102406c2e0 0x7f102406e7a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.167+0000 7f104817f640 1 --2- 172.21.15.139:0/1891113904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1040154c20 0x7f1040153700 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.167+0000 7f104817f640 1 --2- 172.21.15.139:0/1891113904 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f10400ccc90 0x7f10400b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.167+0000 7f104817f640 1 --2- 172.21.15.139:0/1891113904 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f1040068060 0x7f10400b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.167+0000 7f104817f640 1 -- 172.21.15.139:0/1891113904 >> 172.21.15.139:0/1891113904 conn(0x7f104006ca40 msgr2=0x7f10401509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:22.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.167+0000 7f104817f640 1 -- 172.21.15.139:0/1891113904 shutdown_connections 2024-09-16T00:11:22.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.167+0000 7f104817f640 1 -- 172.21.15.139:0/1891113904 wait complete. 2024-09-16T00:11:22.179 INFO:tasks.ceph:Scrubbing osd.1 2024-09-16T00:11:22.180 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-16T00:11:22.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.286+0000 7f437e1e8640 1 Processor -- start 2024-09-16T00:11:22.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.286+0000 7f437e1e8640 1 -- start start 2024-09-16T00:11:22.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.287+0000 7f437e1e8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f43780c6570 0x7f437806d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.287+0000 7f437e1e8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f437806bdf0 con 0x7f43780c6950 2024-09-16T00:11:22.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.287+0000 7f437e1e8640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f437806c0d0 con 0x7f437815de10 2024-09-16T00:11:22.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.287+0000 7f437e1e8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f437806c3b0 con 0x7f43780c6570 2024-09-16T00:11:22.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.287+0000 7f43777fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f43780c6570 0x7f437806d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.287+0000 7f43777fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f43780c6570 0x7f437806d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43470/0 (socket says 172.21.15.139:43470) 2024-09-16T00:11:22.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.287+0000 7f43777fe640 1 -- 172.21.15.139:0/2065616742 learned_addr learned my addr 172.21.15.139:0/2065616742 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:22.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.288+0000 7f43777fe640 1 -- 172.21.15.139:0/2065616742 >> v1:172.21.15.164:6789/0 conn(0x7f437815de10 legacy=0x7f437815e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:22.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.288+0000 7f43777fe640 1 -- 172.21.15.139:0/2065616742 >> v1:172.21.15.139:6789/0 conn(0x7f43780c6950 legacy=0x7f43780cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.288+0000 7f43777fe640 1 -- 172.21.15.139:0/2065616742 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437806c690 con 0x7f43780c6570 2024-09-16T00:11:22.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.288+0000 7f4376ffd640 1 -- 172.21.15.139:0/2065616742 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1062690184 0 0) 0x7f437806bdf0 con 0x7f43780c6950 2024-09-16T00:11:22.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.288+0000 7f43777fe640 1 --2- 172.21.15.139:0/2065616742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43780c6570 0x7f437806d170 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4368009870 tx=0x7f436802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cd1b04da202204 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:22.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.288+0000 7f4376ffd640 1 -- 172.21.15.139:0/2065616742 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368037070 con 0x7f43780c6570 2024-09-16T00:11:22.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.288+0000 7f4376ffd640 1 -- 172.21.15.139:0/2065616742 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4368032440 con 0x7f43780c6570 2024-09-16T00:11:22.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.288+0000 7f4376ffd640 1 -- 172.21.15.139:0/2065616742 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368032810 con 0x7f43780c6570 2024-09-16T00:11:22.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.289+0000 7f437e1e8640 1 -- 172.21.15.139:0/2065616742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43780c6570 msgr2=0x7f437806d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.289+0000 7f437e1e8640 1 --2- 172.21.15.139:0/2065616742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43780c6570 0x7f437806d170 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4368009870 tx=0x7f436802a2a0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.289+0000 7f437e1e8640 1 -- 172.21.15.139:0/2065616742 shutdown_connections 2024-09-16T00:11:22.291 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.289+0000 7f437e1e8640 1 --2- 172.21.15.139:0/2065616742 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43780c6570 0x7f437806d170 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.289+0000 7f437e1e8640 1 -- 172.21.15.139:0/2065616742 >> 172.21.15.139:0/2065616742 conn(0x7f43780d01f0 msgr2=0x7f43780d0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:22.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.289+0000 7f437e1e8640 1 -- 172.21.15.139:0/2065616742 shutdown_connections 2024-09-16T00:11:22.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.289+0000 7f437e1e8640 1 -- 172.21.15.139:0/2065616742 wait complete. 2024-09-16T00:11:22.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.290+0000 7f437e1e8640 1 Processor -- start 2024-09-16T00:11:22.292 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.290+0000 7f437e1e8640 1 -- start start 2024-09-16T00:11:22.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.291+0000 7f437e1e8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43780c6570 0x7f4378153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.294 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.291+0000 7f437e1e8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43780c6950 0x7f4378150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.291+0000 7f437e1e8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f437815de10 0x7f43781f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.291+0000 7f437e1e8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f43780cc5b0 con 0x7f43780c6570 2024-09-16T00:11:22.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.291+0000 7f437e1e8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f437806cc50 con 0x7f43780c6950 2024-09-16T00:11:22.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.291+0000 7f437e1e8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f437806d6b0 con 0x7f437815de10 2024-09-16T00:11:22.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.291+0000 7f43777fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43780c6570 0x7f4378153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.291+0000 7f43777fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43780c6570 0x7f4378153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43484/0 (socket says 172.21.15.139:43484) 2024-09-16T00:11:22.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.291+0000 7f43777fe640 1 -- 172.21.15.139:0/2918761376 learned_addr learned my addr 172.21.15.139:0/2918761376 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:22.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.291+0000 7f436ffff640 1 --2- 172.21.15.139:0/2918761376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43780c6950 0x7f4378150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.291+0000 7f4377fff640 1 --2- 172.21.15.139:0/2918761376 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f437815de10 0x7f43781f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.292+0000 7f43777fe640 1 -- 172.21.15.139:0/2918761376 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f437815de10 msgr2=0x7f43781f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.292+0000 7f43777fe640 1 --2- 172.21.15.139:0/2918761376 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f437815de10 0x7f43781f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.292+0000 7f43777fe640 1 -- 172.21.15.139:0/2918761376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43780c6950 msgr2=0x7f4378150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.292+0000 7f43777fe640 1 --2- 172.21.15.139:0/2918761376 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f43780c6950 0x7f4378150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.292+0000 7f43777fe640 1 -- 172.21.15.139:0/2918761376 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43780c92c0 con 0x7f43780c6570 2024-09-16T00:11:22.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.292+0000 7f43777fe640 1 --2- 172.21.15.139:0/2918761376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43780c6570 0x7f4378153bd0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4368009870 tx=0x7f43680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:22.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.292+0000 7f4374ff9640 1 -- 172.21.15.139:0/2918761376 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368037040 con 0x7f43780c6570 2024-09-16T00:11:22.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.292+0000 7f4374ff9640 1 -- 172.21.15.139:0/2918761376 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4368004250 con 0x7f43780c6570 2024-09-16T00:11:22.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.292+0000 7f4374ff9640 1 -- 172.21.15.139:0/2918761376 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436803c730 con 0x7f43780c6570 2024-09-16T00:11:22.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.292+0000 7f437e1e8640 1 -- 172.21.15.139:0/2918761376 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43781f0040 con 0x7f43780c6570 2024-09-16T00:11:22.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.292+0000 7f437e1e8640 1 -- 172.21.15.139:0/2918761376 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43781f0600 con 0x7f43780c6570 2024-09-16T00:11:22.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.293+0000 7f437e1e8640 1 -- 172.21.15.139:0/2918761376 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f43780690a0 con 0x7f43780c6570 2024-09-16T00:11:22.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.294+0000 7f4374ff9640 1 -- 172.21.15.139:0/2918761376 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f436803a070 con 0x7f43780c6570 2024-09-16T00:11:22.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.294+0000 7f4374ff9640 1 --2- 172.21.15.139:0/2918761376 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f435806c2b0 0x7f435806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.295+0000 7f4374ff9640 1 -- 172.21.15.139:0/2918761376 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f43680abf20 con 0x7f43780c6570 2024-09-16T00:11:22.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.295+0000 7f436ffff640 1 --2- 172.21.15.139:0/2918761376 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f435806c2b0 0x7f435806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.295+0000 7f4374ff9640 1 --2- 172.21.15.139:0/2918761376 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f43580747b0 0x7f4358076c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.295+0000 7f4374ff9640 1 -- 172.21.15.139:0/2918761376 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4368033e50 con 0x7f43580747b0 2024-09-16T00:11:22.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.295+0000 7f4374ff9640 1 -- 172.21.15.139:0/2918761376 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7f43680ae620 con 0x7f43780c6570 2024-09-16T00:11:22.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.295+0000 7f4377fff640 1 --2- 172.21.15.139:0/2918761376 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f43580747b0 0x7f4358076c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.295+0000 7f436ffff640 1 --2- 172.21.15.139:0/2918761376 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f435806c2b0 0x7f435806e770 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f43780d0b40 tx=0x7f4360009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:22.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.295+0000 7f4377fff640 1 --2- 172.21.15.139:0/2918761376 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f43580747b0 0x7f4358076c50 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:22.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.296+0000 7f4374ff9640 1 -- 172.21.15.139:0/2918761376 <== osd.1 v2:172.21.15.139:6808/2232380998 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4368033e50 con 0x7f43580747b0 2024-09-16T00:11:22.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.308+0000 7f437e1e8640 1 -- 172.21.15.139:0/2918761376 --> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f43781f3520 con 0x7f43580747b0 2024-09-16T00:11:22.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.308+0000 7f4374ff9640 1 -- 172.21.15.139:0/2918761376 <== osd.1 v2:172.21.15.139:6808/2232380998 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f43781f3520 con 0x7f43580747b0 2024-09-16T00:11:22.309 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:11:22.310 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T00:11:22.310 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:11:22.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.309+0000 7f437e1e8640 1 -- 172.21.15.139:0/2918761376 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f43580747b0 msgr2=0x7f4358076c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.309+0000 7f437e1e8640 1 --2- 172.21.15.139:0/2918761376 >> [v2:172.21.15.139:6808/2232380998,v1:172.21.15.139:6809/2232380998] conn(0x7f43580747b0 0x7f4358076c50 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.309+0000 7f437e1e8640 1 -- 172.21.15.139:0/2918761376 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f435806c2b0 msgr2=0x7f435806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.309+0000 7f437e1e8640 1 --2- 172.21.15.139:0/2918761376 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f435806c2b0 0x7f435806e770 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f43780d0b40 tx=0x7f4360009040 comp rx=0 tx=0).stop 2024-09-16T00:11:22.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.309+0000 7f437e1e8640 1 -- 172.21.15.139:0/2918761376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43780c6570 msgr2=0x7f4378153bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.309+0000 7f437e1e8640 1 --2- 172.21.15.139:0/2918761376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f43780c6570 0x7f4378153bd0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4368009870 tx=0x7f43680046c0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.309+0000 7f4377fff640 1 -- 172.21.15.139:0/2918761376 reap_dead start 2024-09-16T00:11:22.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.309+0000 7f437e1e8640 1 -- 172.21.15.139:0/2918761376 shutdown_connections 2024-09-16T00:11:22.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.309+0000 7f437e1e8640 1 -- 172.21.15.139:0/2918761376 >> 172.21.15.139:0/2918761376 conn(0x7f43780d01f0 msgr2=0x7f43780c5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:22.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.309+0000 7f437e1e8640 1 -- 172.21.15.139:0/2918761376 shutdown_connections 2024-09-16T00:11:22.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.309+0000 7f437e1e8640 1 -- 172.21.15.139:0/2918761376 wait complete. 2024-09-16T00:11:22.321 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-16T00:11:22.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.421+0000 7f42c4d78640 1 Processor -- start 2024-09-16T00:11:22.422 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.421+0000 7f42c4d78640 1 -- start start 2024-09-16T00:11:22.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.422+0000 7f42c4d78640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c01524f0 0x7f42c0158630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.422+0000 7f42c4d78640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f42c01528d0 0x7f42c00c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.422+0000 7f42c4d78640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42c00702c0 con 0x7f42c0072f00 2024-09-16T00:11:22.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.422+0000 7f42c4d78640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f42c00c9ae0 con 0x7f42c01528d0 2024-09-16T00:11:22.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.422+0000 7f42c4d78640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f42c00c6c00 con 0x7f42c01524f0 2024-09-16T00:11:22.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.422+0000 7f42bdd9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f42c01528d0 0x7f42c00c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.423 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.422+0000 7f42bed9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f42c0072f00 0x7f42c00c5910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45746/0 (socket says 172.21.15.139:45746) 2024-09-16T00:11:22.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.422+0000 7f42bdd9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f42c01528d0 0x7f42c00c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43488/0 (socket says 172.21.15.139:43488) 2024-09-16T00:11:22.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.422+0000 7f42bed9d640 1 -- 172.21.15.139:0/123243353 learned_addr learned my addr 172.21.15.139:0/123243353 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:22.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.422+0000 7f42be59c640 1 --2- 172.21.15.139:0/123243353 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c01524f0 0x7f42c0158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.422+0000 7f42bdd9b640 1 -- 172.21.15.139:0/123243353 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c01524f0 msgr2=0x7f42c0158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:22.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.422+0000 7f42bdd9b640 1 --2- 172.21.15.139:0/123243353 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c01524f0 0x7f42c0158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.422+0000 7f42bdd9b640 1 -- 172.21.15.139:0/123243353 >> v1:172.21.15.139:6789/0 conn(0x7f42c0072f00 legacy=0x7f42c00c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.422+0000 7f42bd59a640 1 -- 172.21.15.139:0/123243353 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 396591953 0 0) 0x7f42c00702c0 con 0x7f42c0072f00 2024-09-16T00:11:22.424 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.422+0000 7f42bdd9b640 1 -- 172.21.15.139:0/123243353 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c00c8320 con 0x7f42c01528d0 2024-09-16T00:11:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.423+0000 7f42bdd9b640 1 --2- 172.21.15.139:0/123243353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c01528d0 0x7f42c00c95a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f42b40097a0 tx=0x7f42b402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eda0d427fd13e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.423+0000 7f42bd59a640 1 -- 172.21.15.139:0/123243353 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4037070 con 0x7f42c01528d0 2024-09-16T00:11:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.423+0000 7f42bd59a640 1 -- 172.21.15.139:0/123243353 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b4032440 con 0x7f42c01528d0 2024-09-16T00:11:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.423+0000 7f42bd59a640 1 -- 172.21.15.139:0/123243353 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4032810 con 0x7f42c01528d0 2024-09-16T00:11:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.424+0000 7f42c4d78640 1 -- 172.21.15.139:0/123243353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c01528d0 msgr2=0x7f42c00c95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.424+0000 7f42c4d78640 1 --2- 172.21.15.139:0/123243353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c01528d0 0x7f42c00c95a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f42b40097a0 tx=0x7f42b402a180 comp rx=0 tx=0).stop 2024-09-16T00:11:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.424+0000 7f42c4d78640 1 -- 172.21.15.139:0/123243353 shutdown_connections 2024-09-16T00:11:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.424+0000 7f42c4d78640 1 --2- 172.21.15.139:0/123243353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c01528d0 0x7f42c00c95a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.426 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.424+0000 7f42c4d78640 1 --2- 172.21.15.139:0/123243353 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c01524f0 0x7f42c0158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.424+0000 7f42c4d78640 1 -- 172.21.15.139:0/123243353 >> 172.21.15.139:0/123243353 conn(0x7f42c006da40 msgr2=0x7f42c006f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:22.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.424+0000 7f42c4d78640 1 -- 172.21.15.139:0/123243353 shutdown_connections 2024-09-16T00:11:22.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.424+0000 7f42c4d78640 1 -- 172.21.15.139:0/123243353 wait complete. 2024-09-16T00:11:22.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.425+0000 7f42c4d78640 1 Processor -- start 2024-09-16T00:11:22.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.425+0000 7f42c4d78640 1 -- start start 2024-09-16T00:11:22.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.426+0000 7f42c4d78640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c0072f00 0x7f42c01478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.426+0000 7f42c4d78640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c01524f0 0x7f42c00cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.426+0000 7f42c4d78640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c01528d0 0x7f42c0155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.426+0000 7f42c4d78640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42c006ffe0 con 0x7f42c0072f00 2024-09-16T00:11:22.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.426+0000 7f42c4d78640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f42c006fd00 con 0x7f42c01528d0 2024-09-16T00:11:22.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.426+0000 7f42c4d78640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f42c00c8da0 con 0x7f42c01524f0 2024-09-16T00:11:22.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.426+0000 7f42be59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c0072f00 0x7f42c01478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.426+0000 7f42be59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c0072f00 0x7f42c01478b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43492/0 (socket says 172.21.15.139:43492) 2024-09-16T00:11:22.429 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.426+0000 7f42be59c640 1 -- 172.21.15.139:0/1972097130 learned_addr learned my addr 172.21.15.139:0/1972097130 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:22.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.426+0000 7f42bdd9b640 1 --2- 172.21.15.139:0/1972097130 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c01524f0 0x7f42c00cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.426+0000 7f42bed9d640 1 --2- 172.21.15.139:0/1972097130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c01528d0 0x7f42c0155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.426+0000 7f42be59c640 1 -- 172.21.15.139:0/1972097130 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c01524f0 msgr2=0x7f42c00cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.426+0000 7f42be59c640 1 --2- 172.21.15.139:0/1972097130 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c01524f0 0x7f42c00cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.426+0000 7f42be59c640 1 -- 172.21.15.139:0/1972097130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c01528d0 msgr2=0x7f42c0155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.426+0000 7f42be59c640 1 --2- 172.21.15.139:0/1972097130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c01528d0 0x7f42c0155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.426+0000 7f42be59c640 1 -- 172.21.15.139:0/1972097130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c00cb790 con 0x7f42c0072f00 2024-09-16T00:11:22.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.427+0000 7f42be59c640 1 --2- 172.21.15.139:0/1972097130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c0072f00 0x7f42c01478b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f42b000e9e0 tx=0x7f42b000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:22.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.427+0000 7f42aeffd640 1 -- 172.21.15.139:0/1972097130 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b0017070 con 0x7f42c0072f00 2024-09-16T00:11:22.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.427+0000 7f42aeffd640 1 -- 172.21.15.139:0/1972097130 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b000c9d0 con 0x7f42c0072f00 2024-09-16T00:11:22.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.427+0000 7f42aeffd640 1 -- 172.21.15.139:0/1972097130 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b000cdc0 con 0x7f42c0072f00 2024-09-16T00:11:22.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.427+0000 7f42c4d78640 1 -- 172.21.15.139:0/1972097130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c0153c70 con 0x7f42c0072f00 2024-09-16T00:11:22.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.427+0000 7f42c4d78640 1 -- 172.21.15.139:0/1972097130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c0155f40 con 0x7f42c0072f00 2024-09-16T00:11:22.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.428+0000 7f42c4d78640 1 -- 172.21.15.139:0/1972097130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c00718b0 con 0x7f42c0072f00 2024-09-16T00:11:22.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.428+0000 7f42aeffd640 1 -- 172.21.15.139:0/1972097130 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f42b001a070 con 0x7f42c0072f00 2024-09-16T00:11:22.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.429+0000 7f42aeffd640 1 --2- 172.21.15.139:0/1972097130 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f42a006c2b0 0x7f42a006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.429+0000 7f42aeffd640 1 -- 172.21.15.139:0/1972097130 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f42b008b510 con 0x7f42c0072f00 2024-09-16T00:11:22.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.429+0000 7f42bdd9b640 1 --2- 172.21.15.139:0/1972097130 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f42a006c2b0 0x7f42a006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.432 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.429+0000 7f42bdd9b640 1 --2- 172.21.15.139:0/1972097130 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f42a006c2b0 0x7f42a006e770 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f42b4034000 tx=0x7f42b402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:22.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.433+0000 7f42aeffd640 1 -- 172.21.15.139:0/1972097130 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f42b0059540 con 0x7f42c0072f00 2024-09-16T00:11:22.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.607+0000 7f42c4d78640 1 -- 172.21.15.139:0/1972097130 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f42c0072b30 con 0x7f42a006c2b0 2024-09-16T00:11:22.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.608+0000 7f42aeffd640 1 -- 172.21.15.139:0/1972097130 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f42c0072b30 con 0x7f42a006c2b0 2024-09-16T00:11:22.610 INFO:teuthology.orchestra.run.smithi139.stderr:instructed osd(s) 1 to deep-scrub 2024-09-16T00:11:22.611 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.610+0000 7f42c4d78640 1 -- 172.21.15.139:0/1972097130 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f42a006c2b0 msgr2=0x7f42a006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.610+0000 7f42c4d78640 1 --2- 172.21.15.139:0/1972097130 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f42a006c2b0 0x7f42a006e770 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f42b4034000 tx=0x7f42b402e000 comp rx=0 tx=0).stop 2024-09-16T00:11:22.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.610+0000 7f42c4d78640 1 -- 172.21.15.139:0/1972097130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c0072f00 msgr2=0x7f42c01478b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.610+0000 7f42c4d78640 1 --2- 172.21.15.139:0/1972097130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c0072f00 0x7f42c01478b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f42b000e9e0 tx=0x7f42b000eea0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.610+0000 7f42c4d78640 1 -- 172.21.15.139:0/1972097130 shutdown_connections 2024-09-16T00:11:22.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.610+0000 7f42c4d78640 1 --2- 172.21.15.139:0/1972097130 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f42a006c2b0 0x7f42a006e770 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.610+0000 7f42c4d78640 1 --2- 172.21.15.139:0/1972097130 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42c01528d0 0x7f42c0155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.610+0000 7f42c4d78640 1 --2- 172.21.15.139:0/1972097130 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f42c01524f0 0x7f42c00cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.610+0000 7f42c4d78640 1 --2- 172.21.15.139:0/1972097130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c0072f00 0x7f42c01478b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.610+0000 7f42c4d78640 1 -- 172.21.15.139:0/1972097130 >> 172.21.15.139:0/1972097130 conn(0x7f42c006da40 msgr2=0x7f42c014f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:22.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.610+0000 7f42c4d78640 1 -- 172.21.15.139:0/1972097130 shutdown_connections 2024-09-16T00:11:22.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.610+0000 7f42c4d78640 1 -- 172.21.15.139:0/1972097130 wait complete. 2024-09-16T00:11:22.623 INFO:tasks.ceph:Scrubbing osd.2 2024-09-16T00:11:22.623 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-16T00:11:22.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.727+0000 7f1581d6f640 1 Processor -- start 2024-09-16T00:11:22.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.727+0000 7f1581d6f640 1 -- start start 2024-09-16T00:11:22.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.727+0000 7f1581d6f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f157c06f500 0x7f157c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.728+0000 7f1581d6f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f157c0c48e0 con 0x7f157c06f120 2024-09-16T00:11:22.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.728+0000 7f1581d6f640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f157c1574d0 con 0x7f157c0c79c0 2024-09-16T00:11:22.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.728+0000 7f1581d6f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f157c1577b0 con 0x7f157c06f500 2024-09-16T00:11:22.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.728+0000 7f157b7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f157c06f120 0x7f157c1511d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45760/0 (socket says 172.21.15.139:45760) 2024-09-16T00:11:22.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.728+0000 7f157b7fe640 1 -- 172.21.15.139:0/2786743736 learned_addr learned my addr 172.21.15.139:0/2786743736 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:22.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.728+0000 7f157affd640 1 --2- 172.21.15.139:0/2786743736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f157c06f500 0x7f157c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.728+0000 7f157a7fc640 1 -- 172.21.15.139:0/2786743736 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 821224463 0 0) 0x7f157c0c48e0 con 0x7f157c06f120 2024-09-16T00:11:22.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.728+0000 7f157a7fc640 1 -- 172.21.15.139:0/2786743736 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1564003520 con 0x7f157c06f120 2024-09-16T00:11:22.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.728+0000 7f157a7fc640 1 -- 172.21.15.139:0/2786743736 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 597025723 0 0) 0x7f157c1574d0 con 0x7f157c0c79c0 2024-09-16T00:11:22.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.728+0000 7f157a7fc640 1 -- 172.21.15.139:0/2786743736 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f157c0c48e0 con 0x7f157c0c79c0 2024-09-16T00:11:22.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.728+0000 7f157affd640 1 -- 172.21.15.139:0/2786743736 >> v1:172.21.15.164:6789/0 conn(0x7f157c0c79c0 legacy=0x7f157c161d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.728+0000 7f157affd640 1 -- 172.21.15.139:0/2786743736 >> v1:172.21.15.139:6789/0 conn(0x7f157c06f120 legacy=0x7f157c1511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.728+0000 7f157a7fc640 1 -- 172.21.15.139:0/2786743736 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4004423891 0 0) 0x7f1564003520 con 0x7f157c06f120 2024-09-16T00:11:22.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.728+0000 7f157affd640 1 -- 172.21.15.139:0/2786743736 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157c157a90 con 0x7f157c06f500 2024-09-16T00:11:22.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.729+0000 7f157affd640 1 --2- 172.21.15.139:0/2786743736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f157c06f500 0x7f157c15a9b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f15700097d0 tx=0x7f157002a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3acb0d32e69cb12 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:22.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.729+0000 7f157a7fc640 1 -- 172.21.15.139:0/2786743736 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570037070 con 0x7f157c06f500 2024-09-16T00:11:22.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.729+0000 7f157a7fc640 1 -- 172.21.15.139:0/2786743736 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f157002acc0 con 0x7f157c06f500 2024-09-16T00:11:22.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.729+0000 7f157a7fc640 1 -- 172.21.15.139:0/2786743736 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c001670 con 0x7f157c06f500 2024-09-16T00:11:22.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.730+0000 7f1581d6f640 1 -- 172.21.15.139:0/2786743736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f157c06f500 msgr2=0x7f157c15a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.730+0000 7f1581d6f640 1 --2- 172.21.15.139:0/2786743736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f157c06f500 0x7f157c15a9b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f15700097d0 tx=0x7f157002a230 comp rx=0 tx=0).stop 2024-09-16T00:11:22.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.730+0000 7f1581d6f640 1 -- 172.21.15.139:0/2786743736 shutdown_connections 2024-09-16T00:11:22.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.730+0000 7f1581d6f640 1 --2- 172.21.15.139:0/2786743736 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f157c06f500 0x7f157c15a9b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.730+0000 7f1581d6f640 1 -- 172.21.15.139:0/2786743736 >> 172.21.15.139:0/2786743736 conn(0x7f157c070420 msgr2=0x7f157c070830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:22.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.730+0000 7f1581d6f640 1 -- 172.21.15.139:0/2786743736 shutdown_connections 2024-09-16T00:11:22.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.730+0000 7f1581d6f640 1 -- 172.21.15.139:0/2786743736 wait complete. 2024-09-16T00:11:22.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.731+0000 7f1581d6f640 1 Processor -- start 2024-09-16T00:11:22.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.731+0000 7f1581d6f640 1 -- start start 2024-09-16T00:11:22.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.732+0000 7f1581d6f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f157c06f120 0x7f157c153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.732+0000 7f1581d6f640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f157c06f500 0x7f157c14e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.732+0000 7f1581d6f640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f157c0c79c0 0x7f157c1519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.732+0000 7f1581d6f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f157c0c4640 con 0x7f157c06f120 2024-09-16T00:11:22.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.732+0000 7f1581d6f640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f157c0c44c0 con 0x7f157c0c79c0 2024-09-16T00:11:22.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.732+0000 7f1581d6f640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f157c06eaf0 con 0x7f157c06f500 2024-09-16T00:11:22.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.732+0000 7f157b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f157c06f120 0x7f157c153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.732+0000 7f157b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f157c06f120 0x7f157c153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43500/0 (socket says 172.21.15.139:43500) 2024-09-16T00:11:22.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.732+0000 7f157b7fe640 1 -- 172.21.15.139:0/437253600 learned_addr learned my addr 172.21.15.139:0/437253600 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:22.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.732+0000 7f157bfff640 1 --2- 172.21.15.139:0/437253600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f157c0c79c0 0x7f157c1519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.732+0000 7f157affd640 1 --2- 172.21.15.139:0/437253600 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f157c06f500 0x7f157c14e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.732+0000 7f157b7fe640 1 -- 172.21.15.139:0/437253600 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f157c06f500 msgr2=0x7f157c14e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.732+0000 7f157b7fe640 1 --2- 172.21.15.139:0/437253600 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f157c06f500 0x7f157c14e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.743 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.732+0000 7f157b7fe640 1 -- 172.21.15.139:0/437253600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f157c0c79c0 msgr2=0x7f157c1519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.732+0000 7f157b7fe640 1 --2- 172.21.15.139:0/437253600 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f157c0c79c0 0x7f157c1519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.732+0000 7f157b7fe640 1 -- 172.21.15.139:0/437253600 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157c1ee910 con 0x7f157c06f120 2024-09-16T00:11:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.733+0000 7f157b7fe640 1 --2- 172.21.15.139:0/437253600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f157c06f120 0x7f157c153a80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c5a0 tx=0x7f156c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.733+0000 7f1563fff640 1 -- 172.21.15.139:0/437253600 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c017070 con 0x7f157c06f120 2024-09-16T00:11:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.733+0000 7f1563fff640 1 -- 172.21.15.139:0/437253600 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f156c005ce0 con 0x7f157c06f120 2024-09-16T00:11:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.733+0000 7f1581d6f640 1 -- 172.21.15.139:0/437253600 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157c1e39a0 con 0x7f157c06f120 2024-09-16T00:11:22.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.733+0000 7f1563fff640 1 -- 172.21.15.139:0/437253600 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c0072b0 con 0x7f157c06f120 2024-09-16T00:11:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.733+0000 7f1581d6f640 1 -- 172.21.15.139:0/437253600 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f157c1eebb0 con 0x7f157c06f120 2024-09-16T00:11:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.734+0000 7f1581d6f640 1 -- 172.21.15.139:0/437253600 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f157c0c4c30 con 0x7f157c06f120 2024-09-16T00:11:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.734+0000 7f1563fff640 1 -- 172.21.15.139:0/437253600 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f156c01a070 con 0x7f157c06f120 2024-09-16T00:11:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.735+0000 7f1563fff640 1 --2- 172.21.15.139:0/437253600 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f156806c2b0 0x7f156806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.735+0000 7f157affd640 1 --2- 172.21.15.139:0/437253600 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f156806c2b0 0x7f156806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.735+0000 7f1563fff640 1 -- 172.21.15.139:0/437253600 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f156c08b140 con 0x7f157c06f120 2024-09-16T00:11:22.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.735+0000 7f1563fff640 1 --2- 172.21.15.139:0/437253600 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f15680747b0 0x7f1568076c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.735+0000 7f1563fff640 1 -- 172.21.15.139:0/437253600 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f156c009eb0 con 0x7f15680747b0 2024-09-16T00:11:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.735+0000 7f1563fff640 1 -- 172.21.15.139:0/437253600 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7f156c08b5d0 con 0x7f157c06f120 2024-09-16T00:11:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.735+0000 7f157bfff640 1 --2- 172.21.15.139:0/437253600 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f15680747b0 0x7f1568076c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.735+0000 7f157affd640 1 --2- 172.21.15.139:0/437253600 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f156806c2b0 0x7f156806e770 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1570034000 tx=0x7f157002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.736+0000 7f157bfff640 1 --2- 172.21.15.139:0/437253600 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f15680747b0 0x7f1568076c50 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:22.746 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.736+0000 7f1563fff640 1 -- 172.21.15.139:0/437253600 <== osd.2 v2:172.21.15.139:6800/4180588905 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f156c009eb0 con 0x7f15680747b0 2024-09-16T00:11:22.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.749+0000 7f1581d6f640 1 -- 172.21.15.139:0/437253600 --> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f157c158580 con 0x7f15680747b0 2024-09-16T00:11:22.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.749+0000 7f1563fff640 1 -- 172.21.15.139:0/437253600 <== osd.2 v2:172.21.15.139:6800/4180588905 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f157c158580 con 0x7f15680747b0 2024-09-16T00:11:22.750 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:11:22.750 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T00:11:22.751 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:11:22.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.749+0000 7f1581d6f640 1 -- 172.21.15.139:0/437253600 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f15680747b0 msgr2=0x7f1568076c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.749+0000 7f1581d6f640 1 --2- 172.21.15.139:0/437253600 >> [v2:172.21.15.139:6800/4180588905,v1:172.21.15.139:6801/4180588905] conn(0x7f15680747b0 0x7f1568076c50 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.752 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.749+0000 7f1581d6f640 1 -- 172.21.15.139:0/437253600 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f156806c2b0 msgr2=0x7f156806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.749+0000 7f1581d6f640 1 --2- 172.21.15.139:0/437253600 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f156806c2b0 0x7f156806e770 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1570034000 tx=0x7f157002e000 comp rx=0 tx=0).stop 2024-09-16T00:11:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.749+0000 7f1581d6f640 1 -- 172.21.15.139:0/437253600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f157c06f120 msgr2=0x7f157c153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.749+0000 7f1581d6f640 1 --2- 172.21.15.139:0/437253600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f157c06f120 0x7f157c153a80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f156c00c5a0 tx=0x7f156c00ca60 comp rx=0 tx=0).stop 2024-09-16T00:11:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.749+0000 7f157bfff640 1 -- 172.21.15.139:0/437253600 reap_dead start 2024-09-16T00:11:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.749+0000 7f1581d6f640 1 -- 172.21.15.139:0/437253600 shutdown_connections 2024-09-16T00:11:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.749+0000 7f1581d6f640 1 -- 172.21.15.139:0/437253600 >> 172.21.15.139:0/437253600 conn(0x7f157c070420 msgr2=0x7f157c144850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:22.753 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.749+0000 7f1581d6f640 1 -- 172.21.15.139:0/437253600 shutdown_connections 2024-09-16T00:11:22.754 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.749+0000 7f1581d6f640 1 -- 172.21.15.139:0/437253600 wait complete. 2024-09-16T00:11:22.761 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-16T00:11:22.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.862+0000 7fb4b6a5e640 1 Processor -- start 2024-09-16T00:11:22.863 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.862+0000 7fb4b6a5e640 1 -- start start 2024-09-16T00:11:22.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.863+0000 7fb4b6a5e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4b0154c20 0x7fb4b015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.863+0000 7fb4b6a5e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb4b0068060 0x7fb4b00c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.863+0000 7fb4b6a5e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4b00cc4b0 con 0x7fb4b00ccc90 2024-09-16T00:11:22.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.863+0000 7fb4b6a5e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb4b015b7c0 con 0x7fb4b0068060 2024-09-16T00:11:22.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.863+0000 7fb4b6a5e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b00cc260 con 0x7fb4b0154c20 2024-09-16T00:11:22.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.863+0000 7fb4affff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4b0154c20 0x7fb4b015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.866 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.863+0000 7fb4a7fff640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb4b0068060 0x7fb4b00c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.863+0000 7fb4affff640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4b0154c20 0x7fb4b015b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3301/0 says I am v2:172.21.15.139:44708/0 (socket says 172.21.15.139:44708) 2024-09-16T00:11:22.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.863+0000 7fb4affff640 1 -- 172.21.15.139:0/2450846112 learned_addr learned my addr 172.21.15.139:0/2450846112 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:22.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.864+0000 7fb4affff640 1 -- 172.21.15.139:0/2450846112 >> v2:172.21.15.164:3300/0 conn(0x7fb4b0068060 msgr2=0x7fb4b00c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:22.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.864+0000 7fb4affff640 1 --2- 172.21.15.139:0/2450846112 >> v2:172.21.15.164:3300/0 conn(0x7fb4b0068060 0x7fb4b00c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.864+0000 7fb4affff640 1 -- 172.21.15.139:0/2450846112 >> v1:172.21.15.164:6789/0 conn(0x7fb4b00ccc90 legacy=0x7fb4b00cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.864+0000 7fb4affff640 1 -- 172.21.15.139:0/2450846112 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b00cc790 con 0x7fb4b0154c20 2024-09-16T00:11:22.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.864+0000 7fb4a7fff640 1 --2- 172.21.15.139:0/2450846112 >> v2:172.21.15.164:3300/0 conn(0x7fb4b0068060 0x7fb4b00c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:11:22.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.864+0000 7fb4affff640 1 --2- 172.21.15.139:0/2450846112 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4b0154c20 0x7fb4b015b280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009870 tx=0x7fb4a002a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd9783d396450b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:22.867 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.865+0000 7fb4af7fe640 1 -- 172.21.15.139:0/2450846112 <== mon.2 v2:172.21.15.164:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0037070 con 0x7fb4b0154c20 2024-09-16T00:11:22.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.865+0000 7fb4af7fe640 1 -- 172.21.15.139:0/2450846112 <== mon.2 v2:172.21.15.164:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4a002ae50 con 0x7fb4b0154c20 2024-09-16T00:11:22.868 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.865+0000 7fb4af7fe640 1 -- 172.21.15.139:0/2450846112 <== mon.2 v2:172.21.15.164:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0032860 con 0x7fb4b0154c20 2024-09-16T00:11:22.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.865+0000 7fb4b6a5e640 1 -- 172.21.15.139:0/2450846112 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4b0154c20 msgr2=0x7fb4b015b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.865+0000 7fb4b6a5e640 1 --2- 172.21.15.139:0/2450846112 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4b0154c20 0x7fb4b015b280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009870 tx=0x7fb4a002a300 comp rx=0 tx=0).stop 2024-09-16T00:11:22.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.865+0000 7fb4b6a5e640 1 -- 172.21.15.139:0/2450846112 shutdown_connections 2024-09-16T00:11:22.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.865+0000 7fb4b6a5e640 1 --2- 172.21.15.139:0/2450846112 >> v2:172.21.15.164:3300/0 conn(0x7fb4b0068060 0x7fb4b00c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.873 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.865+0000 7fb4b6a5e640 1 --2- 172.21.15.139:0/2450846112 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4b0154c20 0x7fb4b015b280 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.874 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.865+0000 7fb4b6a5e640 1 -- 172.21.15.139:0/2450846112 >> 172.21.15.139:0/2450846112 conn(0x7fb4b006ca40 msgr2=0x7fb4b0067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:22.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.865+0000 7fb4b6a5e640 1 -- 172.21.15.139:0/2450846112 shutdown_connections 2024-09-16T00:11:22.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.865+0000 7fb4b6a5e640 1 -- 172.21.15.139:0/2450846112 wait complete. 2024-09-16T00:11:22.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.866+0000 7fb4b6a5e640 1 Processor -- start 2024-09-16T00:11:22.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.867+0000 7fb4b6a5e640 1 -- start start 2024-09-16T00:11:22.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.867+0000 7fb4b6a5e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4b0068060 0x7fb4b00b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.867+0000 7fb4b6a5e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4b00ccc90 0x7fb4b014dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.867+0000 7fb4b6a5e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4b0154c20 0x7fb4b014e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.867+0000 7fb4b6a5e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b015e3b0 con 0x7fb4b0154c20 2024-09-16T00:11:22.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.867+0000 7fb4b6a5e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b015e0d0 con 0x7fb4b00ccc90 2024-09-16T00:11:22.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.867+0000 7fb4b6a5e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b015e690 con 0x7fb4b0068060 2024-09-16T00:11:22.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.867+0000 7fb4b4ffb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4b0154c20 0x7fb4b014e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.877 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.867+0000 7fb4b4ffb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4b0154c20 0x7fb4b014e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43506/0 (socket says 172.21.15.139:43506) 2024-09-16T00:11:22.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.867+0000 7fb4b4ffb640 1 -- 172.21.15.139:0/1758718760 learned_addr learned my addr 172.21.15.139:0/1758718760 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:22.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.867+0000 7fb4affff640 1 --2- 172.21.15.139:0/1758718760 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4b0068060 0x7fb4b00b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.867+0000 7fb4a7fff640 1 --2- 172.21.15.139:0/1758718760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4b00ccc90 0x7fb4b014dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.868+0000 7fb4b4ffb640 1 -- 172.21.15.139:0/1758718760 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4b0068060 msgr2=0x7fb4b00b55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.868+0000 7fb4b4ffb640 1 --2- 172.21.15.139:0/1758718760 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4b0068060 0x7fb4b00b55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.868+0000 7fb4b4ffb640 1 -- 172.21.15.139:0/1758718760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4b00ccc90 msgr2=0x7fb4b014dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:22.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.868+0000 7fb4b4ffb640 1 --2- 172.21.15.139:0/1758718760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4b00ccc90 0x7fb4b014dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:22.878 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.868+0000 7fb4b4ffb640 1 -- 172.21.15.139:0/1758718760 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b0148670 con 0x7fb4b0154c20 2024-09-16T00:11:22.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.868+0000 7fb4b4ffb640 1 --2- 172.21.15.139:0/1758718760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4b0154c20 0x7fb4b014e520 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800c730 tx=0x7fb4a800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:22.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.868+0000 7fb4ad7fa640 1 -- 172.21.15.139:0/1758718760 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8003b70 con 0x7fb4b0154c20 2024-09-16T00:11:22.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.868+0000 7fb4ad7fa640 1 -- 172.21.15.139:0/1758718760 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4a8003d30 con 0x7fb4b0154c20 2024-09-16T00:11:22.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.868+0000 7fb4b6a5e640 1 -- 172.21.15.139:0/1758718760 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b015dab0 con 0x7fb4b0154c20 2024-09-16T00:11:22.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.868+0000 7fb4ad7fa640 1 -- 172.21.15.139:0/1758718760 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8005310 con 0x7fb4b0154c20 2024-09-16T00:11:22.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.868+0000 7fb4b6a5e640 1 -- 172.21.15.139:0/1758718760 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b015c690 con 0x7fb4b0154c20 2024-09-16T00:11:22.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.869+0000 7fb4b6a5e640 1 -- 172.21.15.139:0/1758718760 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b014fbb0 con 0x7fb4b0154c20 2024-09-16T00:11:22.879 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.870+0000 7fb4ad7fa640 1 -- 172.21.15.139:0/1758718760 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb4a80056c0 con 0x7fb4b0154c20 2024-09-16T00:11:22.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.870+0000 7fb4ad7fa640 1 --2- 172.21.15.139:0/1758718760 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb49006c2e0 0x7fb49006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:22.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.870+0000 7fb4affff640 1 --2- 172.21.15.139:0/1758718760 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb49006c2e0 0x7fb49006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:22.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.870+0000 7fb4ad7fa640 1 -- 172.21.15.139:0/1758718760 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7fb4a808b390 con 0x7fb4b0154c20 2024-09-16T00:11:22.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.871+0000 7fb4affff640 1 --2- 172.21.15.139:0/1758718760 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb49006c2e0 0x7fb49006e7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0034000 tx=0x7fb4a002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:22.880 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:22.875+0000 7fb4ad7fa640 1 -- 172.21.15.139:0/1758718760 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb4b014fbb0 con 0x7fb4b0154c20 2024-09-16T00:11:23.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.047+0000 7fb4b6a5e640 1 -- 172.21.15.139:0/1758718760 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fb4b00686b0 con 0x7fb49006c2e0 2024-09-16T00:11:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.048+0000 7fb4ad7fa640 1 -- 172.21.15.139:0/1758718760 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb4b00686b0 con 0x7fb49006c2e0 2024-09-16T00:11:23.049 INFO:teuthology.orchestra.run.smithi139.stderr:instructed osd(s) 2 to deep-scrub 2024-09-16T00:11:23.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.050+0000 7fb4b6a5e640 1 -- 172.21.15.139:0/1758718760 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb49006c2e0 msgr2=0x7fb49006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.050+0000 7fb4b6a5e640 1 --2- 172.21.15.139:0/1758718760 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb49006c2e0 0x7fb49006e7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0034000 tx=0x7fb4a002e000 comp rx=0 tx=0).stop 2024-09-16T00:11:23.051 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.050+0000 7fb4b6a5e640 1 -- 172.21.15.139:0/1758718760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4b0154c20 msgr2=0x7fb4b014e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.050+0000 7fb4b6a5e640 1 --2- 172.21.15.139:0/1758718760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4b0154c20 0x7fb4b014e520 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800c730 tx=0x7fb4a800cbf0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.050+0000 7fb4b6a5e640 1 -- 172.21.15.139:0/1758718760 shutdown_connections 2024-09-16T00:11:23.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.050+0000 7fb4b6a5e640 1 --2- 172.21.15.139:0/1758718760 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb49006c2e0 0x7fb49006e7a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.050+0000 7fb4b6a5e640 1 --2- 172.21.15.139:0/1758718760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4b0154c20 0x7fb4b014e520 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.050+0000 7fb4b6a5e640 1 --2- 172.21.15.139:0/1758718760 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb4b00ccc90 0x7fb4b014dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.050+0000 7fb4b6a5e640 1 --2- 172.21.15.139:0/1758718760 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb4b0068060 0x7fb4b00b55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.050+0000 7fb4b6a5e640 1 -- 172.21.15.139:0/1758718760 >> 172.21.15.139:0/1758718760 conn(0x7fb4b006ca40 msgr2=0x7fb4b014ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:23.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.050+0000 7fb4b6a5e640 1 -- 172.21.15.139:0/1758718760 shutdown_connections 2024-09-16T00:11:23.052 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.050+0000 7fb4b6a5e640 1 -- 172.21.15.139:0/1758718760 wait complete. 2024-09-16T00:11:23.063 INFO:tasks.ceph:Scrubbing osd.3 2024-09-16T00:11:23.063 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-16T00:11:23.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.165+0000 7fef70781640 1 Processor -- start 2024-09-16T00:11:23.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.165+0000 7fef70781640 1 -- start start 2024-09-16T00:11:23.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.166+0000 7fef70781640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fef68151690 0x7fef680c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.166+0000 7fef70781640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fef680c57d0 0x7fef680c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.166+0000 7fef70781640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef68072480 con 0x7fef68151a70 2024-09-16T00:11:23.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.166+0000 7fef70781640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fef680c6f80 con 0x7fef680c57d0 2024-09-16T00:11:23.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.166+0000 7fef70781640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fef680c5340 con 0x7fef68151690 2024-09-16T00:11:23.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.166+0000 7fef6ed1e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fef680c57d0 0x7fef680c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.166+0000 7fef6dd1c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fef68151a70 0x7fef680c3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45766/0 (socket says 172.21.15.139:45766) 2024-09-16T00:11:23.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.166+0000 7fef6dd1c640 1 -- 172.21.15.139:0/3663809179 learned_addr learned my addr 172.21.15.139:0/3663809179 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:23.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.166+0000 7fef6e51d640 1 --2- 172.21.15.139:0/3663809179 >> v2:172.21.15.164:3300/0 conn(0x7fef68151690 0x7fef680c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.166+0000 7fef6ed1e640 1 -- 172.21.15.139:0/3663809179 >> v2:172.21.15.164:3300/0 conn(0x7fef68151690 msgr2=0x7fef680c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:23.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.166+0000 7fef6ed1e640 1 --2- 172.21.15.139:0/3663809179 >> v2:172.21.15.164:3300/0 conn(0x7fef68151690 0x7fef680c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.166+0000 7fef6ed1e640 1 -- 172.21.15.139:0/3663809179 >> v1:172.21.15.139:6789/0 conn(0x7fef68151a70 legacy=0x7fef680c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.166+0000 7fef6ed1e640 1 -- 172.21.15.139:0/3663809179 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef68072a70 con 0x7fef680c57d0 2024-09-16T00:11:23.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.166+0000 7fef6d51b640 1 -- 172.21.15.139:0/3663809179 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2570597283 0 0) 0x7fef68072480 con 0x7fef68151a70 2024-09-16T00:11:23.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.166+0000 7fef6e51d640 1 --2- 172.21.15.139:0/3663809179 >> v2:172.21.15.164:3300/0 conn(0x7fef68151690 0x7fef680c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:23.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.167+0000 7fef6ed1e640 1 --2- 172.21.15.139:0/3663809179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef680c57d0 0x7fef680c71a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fef680c9d70 tx=0x7fef6002c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bedf9d119a2326d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:23.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.167+0000 7fef6d51b640 1 -- 172.21.15.139:0/3663809179 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60009540 con 0x7fef680c57d0 2024-09-16T00:11:23.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.167+0000 7fef6d51b640 1 -- 172.21.15.139:0/3663809179 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef60037070 con 0x7fef680c57d0 2024-09-16T00:11:23.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.167+0000 7fef6d51b640 1 -- 172.21.15.139:0/3663809179 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60009900 con 0x7fef680c57d0 2024-09-16T00:11:23.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.167+0000 7fef70781640 1 -- 172.21.15.139:0/3663809179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef680c57d0 msgr2=0x7fef680c71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.167+0000 7fef70781640 1 --2- 172.21.15.139:0/3663809179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef680c57d0 0x7fef680c71a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fef680c9d70 tx=0x7fef6002c910 comp rx=0 tx=0).stop 2024-09-16T00:11:23.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.167+0000 7fef70781640 1 -- 172.21.15.139:0/3663809179 shutdown_connections 2024-09-16T00:11:23.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.167+0000 7fef70781640 1 --2- 172.21.15.139:0/3663809179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef680c57d0 0x7fef680c71a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.167+0000 7fef70781640 1 --2- 172.21.15.139:0/3663809179 >> v2:172.21.15.164:3300/0 conn(0x7fef68151690 0x7fef680c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.167+0000 7fef70781640 1 -- 172.21.15.139:0/3663809179 >> 172.21.15.139:0/3663809179 conn(0x7fef6806f440 msgr2=0x7fef6806f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:23.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.168+0000 7fef70781640 1 -- 172.21.15.139:0/3663809179 shutdown_connections 2024-09-16T00:11:23.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.168+0000 7fef70781640 1 -- 172.21.15.139:0/3663809179 wait complete. 2024-09-16T00:11:23.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.169+0000 7fef70781640 1 Processor -- start 2024-09-16T00:11:23.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.169+0000 7fef70781640 1 -- start start 2024-09-16T00:11:23.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.169+0000 7fef70781640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef680c57d0 0x7fef681dd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.172 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.169+0000 7fef70781640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fef68151690 0x7fef681de500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.169+0000 7fef70781640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef68151a70 0x7fef6814cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.169+0000 7fef70781640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fef68072120 con 0x7fef68151a70 2024-09-16T00:11:23.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.169+0000 7fef70781640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fef68073030 con 0x7fef680c57d0 2024-09-16T00:11:23.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.169+0000 7fef70781640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fef680c7c90 con 0x7fef68151690 2024-09-16T00:11:23.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.169+0000 7fef6ed1e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef68151a70 0x7fef6814cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.169+0000 7fef6ed1e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef68151a70 0x7fef6814cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43528/0 (socket says 172.21.15.139:43528) 2024-09-16T00:11:23.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.169+0000 7fef6ed1e640 1 -- 172.21.15.139:0/3395044208 learned_addr learned my addr 172.21.15.139:0/3395044208 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:23.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.170+0000 7fef6e51d640 1 --2- 172.21.15.139:0/3395044208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef680c57d0 0x7fef681dd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.170+0000 7fef6dd1c640 1 --2- 172.21.15.139:0/3395044208 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fef68151690 0x7fef681de500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.170+0000 7fef6ed1e640 1 -- 172.21.15.139:0/3395044208 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fef68151690 msgr2=0x7fef681de500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.170+0000 7fef6ed1e640 1 --2- 172.21.15.139:0/3395044208 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fef68151690 0x7fef681de500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.170+0000 7fef6ed1e640 1 -- 172.21.15.139:0/3395044208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef680c57d0 msgr2=0x7fef681dd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.173 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.170+0000 7fef6ed1e640 1 --2- 172.21.15.139:0/3395044208 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fef680c57d0 0x7fef681dd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.170+0000 7fef6ed1e640 1 -- 172.21.15.139:0/3395044208 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6814e830 con 0x7fef68151a70 2024-09-16T00:11:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.170+0000 7fef6ed1e640 1 --2- 172.21.15.139:0/3395044208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef68151a70 0x7fef6814cb40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fef6000d140 tx=0x7fef60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.170+0000 7fef5effd640 1 -- 172.21.15.139:0/3395044208 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6003c810 con 0x7fef68151a70 2024-09-16T00:11:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.170+0000 7fef5effd640 1 -- 172.21.15.139:0/3395044208 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef6003c9b0 con 0x7fef68151a70 2024-09-16T00:11:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.170+0000 7fef5effd640 1 -- 172.21.15.139:0/3395044208 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6003cda0 con 0x7fef68151a70 2024-09-16T00:11:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.170+0000 7fef70781640 1 -- 172.21.15.139:0/3395044208 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6814b910 con 0x7fef68151a70 2024-09-16T00:11:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.170+0000 7fef70781640 1 -- 172.21.15.139:0/3395044208 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6814bed0 con 0x7fef68151a70 2024-09-16T00:11:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.171+0000 7fef70781640 1 -- 172.21.15.139:0/3395044208 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef6806fd50 con 0x7fef68151a70 2024-09-16T00:11:23.174 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.172+0000 7fef5effd640 1 -- 172.21.15.139:0/3395044208 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fef60031070 con 0x7fef68151a70 2024-09-16T00:11:23.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.172+0000 7fef5effd640 1 --2- 172.21.15.139:0/3395044208 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fef4c06c2b0 0x7fef4c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.172+0000 7fef5effd640 1 -- 172.21.15.139:0/3395044208 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7fef600ae8c0 con 0x7fef68151a70 2024-09-16T00:11:23.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.172+0000 7fef6e51d640 1 --2- 172.21.15.139:0/3395044208 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fef4c06c2b0 0x7fef4c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.172+0000 7fef5effd640 1 --2- 172.21.15.139:0/3395044208 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fef4c074810 0x7fef4c076cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.175 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.172+0000 7fef5effd640 1 -- 172.21.15.139:0/3395044208 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef6000ce60 con 0x7fef4c074810 2024-09-16T00:11:23.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.172+0000 7fef5effd640 1 -- 172.21.15.139:0/3395044208 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7fef600b32c0 con 0x7fef68151a70 2024-09-16T00:11:23.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.172+0000 7fef6dd1c640 1 --2- 172.21.15.139:0/3395044208 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fef4c074810 0x7fef4c076cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.173+0000 7fef6e51d640 1 --2- 172.21.15.139:0/3395044208 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fef4c06c2b0 0x7fef4c06e770 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fef6400a950 tx=0x7fef64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:23.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.173+0000 7fef6dd1c640 1 --2- 172.21.15.139:0/3395044208 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fef4c074810 0x7fef4c076cb0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:23.176 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.173+0000 7fef5effd640 1 -- 172.21.15.139:0/3395044208 <== osd.3 v2:172.21.15.139:6826/3947768492 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef6000ce60 con 0x7fef4c074810 2024-09-16T00:11:23.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.186+0000 7fef70781640 1 -- 172.21.15.139:0/3395044208 --> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fef680c5c20 con 0x7fef4c074810 2024-09-16T00:11:23.186 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.186+0000 7fef5effd640 1 -- 172.21.15.139:0/3395044208 <== osd.3 v2:172.21.15.139:6826/3947768492 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fef680c5c20 con 0x7fef4c074810 2024-09-16T00:11:23.187 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:11:23.187 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T00:11:23.187 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:11:23.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.186+0000 7fef70781640 1 -- 172.21.15.139:0/3395044208 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fef4c074810 msgr2=0x7fef4c076cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.186+0000 7fef70781640 1 --2- 172.21.15.139:0/3395044208 >> [v2:172.21.15.139:6826/3947768492,v1:172.21.15.139:6827/3947768492] conn(0x7fef4c074810 0x7fef4c076cb0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.186+0000 7fef70781640 1 -- 172.21.15.139:0/3395044208 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fef4c06c2b0 msgr2=0x7fef4c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.186+0000 7fef70781640 1 --2- 172.21.15.139:0/3395044208 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fef4c06c2b0 0x7fef4c06e770 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fef6400a950 tx=0x7fef64008040 comp rx=0 tx=0).stop 2024-09-16T00:11:23.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.186+0000 7fef70781640 1 -- 172.21.15.139:0/3395044208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef68151a70 msgr2=0x7fef6814cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.186+0000 7fef70781640 1 --2- 172.21.15.139:0/3395044208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef68151a70 0x7fef6814cb40 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fef6000d140 tx=0x7fef60004a40 comp rx=0 tx=0).stop 2024-09-16T00:11:23.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.186+0000 7fef6ed1e640 1 -- 172.21.15.139:0/3395044208 reap_dead start 2024-09-16T00:11:23.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.186+0000 7fef70781640 1 -- 172.21.15.139:0/3395044208 shutdown_connections 2024-09-16T00:11:23.188 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.186+0000 7fef70781640 1 -- 172.21.15.139:0/3395044208 >> 172.21.15.139:0/3395044208 conn(0x7fef6806f440 msgr2=0x7fef680c3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:23.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.186+0000 7fef70781640 1 -- 172.21.15.139:0/3395044208 shutdown_connections 2024-09-16T00:11:23.189 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.186+0000 7fef70781640 1 -- 172.21.15.139:0/3395044208 wait complete. 2024-09-16T00:11:23.198 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-16T00:11:23.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.297+0000 7fb04b60e640 1 Processor -- start 2024-09-16T00:11:23.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.297+0000 7fb04b60e640 1 -- start start 2024-09-16T00:11:23.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb04b60e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb044154bf0 0x7fb044072e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb04b60e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fb04415b000 0x7fb04415b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb04b60e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0440c7250 con 0x7fb044073740 2024-09-16T00:11:23.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb04b60e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7fb04415ab70 con 0x7fb04415b000 2024-09-16T00:11:23.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb04b60e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb044154fd0 con 0x7fb044154bf0 2024-09-16T00:11:23.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb049bab640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb044073740 0x7fb04415de10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45776/0 (socket says 172.21.15.139:45776) 2024-09-16T00:11:23.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb049bab640 1 -- 172.21.15.139:0/3442984091 learned_addr learned my addr 172.21.15.139:0/3442984091 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:23.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb048ba9640 1 --2- 172.21.15.139:0/3442984091 >> v2:172.21.15.164:3300/0 conn(0x7fb04415b000 0x7fb04415b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb0493aa640 1 --2- 172.21.15.139:0/3442984091 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb044154bf0 0x7fb044072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb03bfff640 1 -- 172.21.15.139:0/3442984091 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3093683737 0 0) 0x7fb03c001490 con 0x7fb044073740 2024-09-16T00:11:23.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb03bfff640 1 -- 172.21.15.139:0/3442984091 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb030003520 con 0x7fb044073740 2024-09-16T00:11:23.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb03bfff640 1 -- 172.21.15.139:0/3442984091 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 595344411 0 0) 0x7fb030003520 con 0x7fb044073740 2024-09-16T00:11:23.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb03bfff640 1 -- 172.21.15.139:0/3442984091 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb044154bf0 msgr2=0x7fb044072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:23.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb03bfff640 1 --2- 172.21.15.139:0/3442984091 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb044154bf0 0x7fb044072e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.306 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb03bfff640 1 -- 172.21.15.139:0/3442984091 >> v2:172.21.15.164:3300/0 conn(0x7fb04415b000 msgr2=0x7fb04415b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:23.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb03bfff640 1 --2- 172.21.15.139:0/3442984091 >> v2:172.21.15.164:3300/0 conn(0x7fb04415b000 0x7fb04415b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb03bfff640 1 -- 172.21.15.139:0/3442984091 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04406c530 con 0x7fb044073740 2024-09-16T00:11:23.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb03bfff640 1 -- 172.21.15.139:0/3442984091 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7fb03c001bc0 con 0x7fb044073740 2024-09-16T00:11:23.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb03bfff640 1 -- 172.21.15.139:0/3442984091 >> v1:172.21.15.139:6789/0 conn(0x7fb044073740 legacy=0x7fb04415de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.298+0000 7fb03bfff640 1 --2- 172.21.15.139:0/3442984091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0300039b0 0x7fb030003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.299+0000 7fb03bfff640 1 --2- 172.21.15.139:0/3442984091 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb044154bf0 0x7fb030004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.299+0000 7fb03bfff640 1 --2- 172.21.15.139:0/3442984091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb030004d40 0x7fb0300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.299+0000 7fb03bfff640 1 -- 172.21.15.139:0/3442984091 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb04415ab70 con 0x7fb030004d40 2024-09-16T00:11:23.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.299+0000 7fb03bfff640 1 -- 172.21.15.139:0/3442984091 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb044154fd0 con 0x7fb0300039b0 2024-09-16T00:11:23.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.299+0000 7fb03bfff640 1 -- 172.21.15.139:0/3442984091 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb03000b9c0 con 0x7fb044154bf0 2024-09-16T00:11:23.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.299+0000 7fb0493aa640 1 --2- 172.21.15.139:0/3442984091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0300039b0 0x7fb030003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.299+0000 7fb049bab640 1 --2- 172.21.15.139:0/3442984091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb030004d40 0x7fb0300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.299+0000 7fb048ba9640 1 --2- 172.21.15.139:0/3442984091 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb044154bf0 0x7fb030004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.299+0000 7fb049bab640 1 -- 172.21.15.139:0/3442984091 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb044154bf0 msgr2=0x7fb030004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.299+0000 7fb049bab640 1 --2- 172.21.15.139:0/3442984091 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb044154bf0 0x7fb030004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.299+0000 7fb049bab640 1 -- 172.21.15.139:0/3442984091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0300039b0 msgr2=0x7fb030003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.299+0000 7fb049bab640 1 --2- 172.21.15.139:0/3442984091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0300039b0 0x7fb030003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.299+0000 7fb049bab640 1 -- 172.21.15.139:0/3442984091 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb030003520 con 0x7fb030004d40 2024-09-16T00:11:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.299+0000 7fb0493aa640 1 --2- 172.21.15.139:0/3442984091 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0300039b0 0x7fb030003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:11:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.300+0000 7fb049bab640 1 --2- 172.21.15.139:0/3442984091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb030004d40 0x7fb0300051c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00aa50 tx=0x7fb030005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb72c3200909286d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.300+0000 7fb03bfff640 1 -- 172.21.15.139:0/3442984091 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00a670 con 0x7fb030004d40 2024-09-16T00:11:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.300+0000 7fb03bfff640 1 -- 172.21.15.139:0/3442984091 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb03c02f070 con 0x7fb030004d40 2024-09-16T00:11:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.300+0000 7fb03bfff640 1 -- 172.21.15.139:0/3442984091 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c034470 con 0x7fb030004d40 2024-09-16T00:11:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.300+0000 7fb04b60e640 1 -- 172.21.15.139:0/3442984091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb030004d40 msgr2=0x7fb0300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.300+0000 7fb04b60e640 1 --2- 172.21.15.139:0/3442984091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb030004d40 0x7fb0300051c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00aa50 tx=0x7fb030005830 comp rx=0 tx=0).stop 2024-09-16T00:11:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.300+0000 7fb049bab640 1 -- 172.21.15.139:0/3442984091 reap_dead start 2024-09-16T00:11:23.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.300+0000 7fb04b60e640 1 -- 172.21.15.139:0/3442984091 shutdown_connections 2024-09-16T00:11:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.300+0000 7fb04b60e640 1 -- 172.21.15.139:0/3442984091 >> 172.21.15.139:0/3442984091 conn(0x7fb0440c9350 msgr2=0x7fb0440c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.300+0000 7fb04b60e640 1 -- 172.21.15.139:0/3442984091 shutdown_connections 2024-09-16T00:11:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.300+0000 7fb04b60e640 1 -- 172.21.15.139:0/3442984091 wait complete. 2024-09-16T00:11:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.301+0000 7fb04b60e640 1 Processor -- start 2024-09-16T00:11:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.302+0000 7fb04b60e640 1 -- start start 2024-09-16T00:11:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.302+0000 7fb04b60e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb030004d40 0x7fb04414edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.302+0000 7fb04b60e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0441e31d0 0x7fb0441f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.302+0000 7fb04b60e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0441f4070 0x7fb0441f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.302+0000 7fb04b60e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb04406c7f0 con 0x7fb0441e31d0 2024-09-16T00:11:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.302+0000 7fb04b60e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb04406db30 con 0x7fb0441f4070 2024-09-16T00:11:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.302+0000 7fb04b60e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb04406cad0 con 0x7fb030004d40 2024-09-16T00:11:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.302+0000 7fb048ba9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0441e31d0 0x7fb0441f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.302+0000 7fb048ba9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0441e31d0 0x7fb0441f3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43540/0 (socket says 172.21.15.139:43540) 2024-09-16T00:11:23.309 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.302+0000 7fb048ba9640 1 -- 172.21.15.139:0/2320936037 learned_addr learned my addr 172.21.15.139:0/2320936037 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:23.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.302+0000 7fb0493aa640 1 --2- 172.21.15.139:0/2320936037 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb030004d40 0x7fb04414edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.302+0000 7fb049bab640 1 --2- 172.21.15.139:0/2320936037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0441f4070 0x7fb0441f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.303+0000 7fb048ba9640 1 -- 172.21.15.139:0/2320936037 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb030004d40 msgr2=0x7fb04414edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.303+0000 7fb048ba9640 1 --2- 172.21.15.139:0/2320936037 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb030004d40 0x7fb04414edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.303+0000 7fb048ba9640 1 -- 172.21.15.139:0/2320936037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0441f4070 msgr2=0x7fb0441f1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.303+0000 7fb048ba9640 1 --2- 172.21.15.139:0/2320936037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0441f4070 0x7fb0441f1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.303+0000 7fb048ba9640 1 -- 172.21.15.139:0/2320936037 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0441f47d0 con 0x7fb0441e31d0 2024-09-16T00:11:23.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.303+0000 7fb048ba9640 1 --2- 172.21.15.139:0/2320936037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0441e31d0 0x7fb0441f3b30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb034009fd0 tx=0x7fb034011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:23.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.303+0000 7fb039ffb640 1 -- 172.21.15.139:0/2320936037 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034005ba0 con 0x7fb0441e31d0 2024-09-16T00:11:23.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.303+0000 7fb039ffb640 1 -- 172.21.15.139:0/2320936037 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb034005d40 con 0x7fb0441e31d0 2024-09-16T00:11:23.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.303+0000 7fb04b60e640 1 -- 172.21.15.139:0/2320936037 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0440c6c10 con 0x7fb0441e31d0 2024-09-16T00:11:23.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.303+0000 7fb039ffb640 1 -- 172.21.15.139:0/2320936037 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03400f9e0 con 0x7fb0441e31d0 2024-09-16T00:11:23.310 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.303+0000 7fb04b60e640 1 -- 172.21.15.139:0/2320936037 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0441e2650 con 0x7fb0441e31d0 2024-09-16T00:11:23.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.304+0000 7fb04b60e640 1 -- 172.21.15.139:0/2320936037 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb04415a1a0 con 0x7fb0441e31d0 2024-09-16T00:11:23.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.305+0000 7fb039ffb640 1 -- 172.21.15.139:0/2320936037 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb034012040 con 0x7fb0441e31d0 2024-09-16T00:11:23.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.305+0000 7fb039ffb640 1 --2- 172.21.15.139:0/2320936037 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb02806c2e0 0x7fb02806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.305+0000 7fb039ffb640 1 -- 172.21.15.139:0/2320936037 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7fb03408b980 con 0x7fb0441e31d0 2024-09-16T00:11:23.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.305+0000 7fb0493aa640 1 --2- 172.21.15.139:0/2320936037 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb02806c2e0 0x7fb02806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.311 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.306+0000 7fb0493aa640 1 --2- 172.21.15.139:0/2320936037 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb02806c2e0 0x7fb02806e7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb040009ce0 tx=0x7fb040009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:23.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.310+0000 7fb039ffb640 1 -- 172.21.15.139:0/2320936037 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb04415a1a0 con 0x7fb0441e31d0 2024-09-16T00:11:23.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.478+0000 7fb04b60e640 1 -- 172.21.15.139:0/2320936037 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fb04406d090 con 0x7fb02806c2e0 2024-09-16T00:11:23.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.479+0000 7fb039ffb640 1 -- 172.21.15.139:0/2320936037 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb04406d090 con 0x7fb02806c2e0 2024-09-16T00:11:23.480 INFO:teuthology.orchestra.run.smithi139.stderr:instructed osd(s) 3 to deep-scrub 2024-09-16T00:11:23.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.481+0000 7fb04b60e640 1 -- 172.21.15.139:0/2320936037 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb02806c2e0 msgr2=0x7fb02806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.482 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.481+0000 7fb04b60e640 1 --2- 172.21.15.139:0/2320936037 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb02806c2e0 0x7fb02806e7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb040009ce0 tx=0x7fb040009190 comp rx=0 tx=0).stop 2024-09-16T00:11:23.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.481+0000 7fb04b60e640 1 -- 172.21.15.139:0/2320936037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0441e31d0 msgr2=0x7fb0441f3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.481+0000 7fb04b60e640 1 --2- 172.21.15.139:0/2320936037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0441e31d0 0x7fb0441f3b30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb034009fd0 tx=0x7fb034011d70 comp rx=0 tx=0).stop 2024-09-16T00:11:23.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.481+0000 7fb04b60e640 1 -- 172.21.15.139:0/2320936037 shutdown_connections 2024-09-16T00:11:23.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.481+0000 7fb04b60e640 1 --2- 172.21.15.139:0/2320936037 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb02806c2e0 0x7fb02806e7a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.481+0000 7fb04b60e640 1 --2- 172.21.15.139:0/2320936037 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb0441f4070 0x7fb0441f1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.481+0000 7fb04b60e640 1 --2- 172.21.15.139:0/2320936037 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0441e31d0 0x7fb0441f3b30 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.481+0000 7fb04b60e640 1 --2- 172.21.15.139:0/2320936037 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb030004d40 0x7fb04414edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.481+0000 7fb04b60e640 1 -- 172.21.15.139:0/2320936037 >> 172.21.15.139:0/2320936037 conn(0x7fb0440c9350 msgr2=0x7fb044152410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:23.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.481+0000 7fb04b60e640 1 -- 172.21.15.139:0/2320936037 shutdown_connections 2024-09-16T00:11:23.483 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.481+0000 7fb04b60e640 1 -- 172.21.15.139:0/2320936037 wait complete. 2024-09-16T00:11:23.493 INFO:tasks.ceph:Scrubbing osd.4 2024-09-16T00:11:23.494 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-16T00:11:23.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.595+0000 7f53c3af8640 1 Processor -- start 2024-09-16T00:11:23.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.595+0000 7f53c3af8640 1 -- start start 2024-09-16T00:11:23.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.596+0000 7f53c3af8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53bc1528d0 0x7f53bc0c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.596+0000 7f53c3af8640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f53bc0cb650 0x7f53bc0cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.596+0000 7f53c3af8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53bc152cb0 con 0x7f53bc0c7140 2024-09-16T00:11:23.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.596+0000 7f53c3af8640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f53bc0cc040 con 0x7f53bc0cb650 2024-09-16T00:11:23.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.596+0000 7f53c3af8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f53bc067fb0 con 0x7f53bc1528d0 2024-09-16T00:11:23.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.596+0000 7f53c1093640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f53bc0c7140 0x7f53bc0c5e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45782/0 (socket says 172.21.15.139:45782) 2024-09-16T00:11:23.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.596+0000 7f53c1093640 1 -- 172.21.15.139:0/127690198 learned_addr learned my addr 172.21.15.139:0/127690198 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:23.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.596+0000 7f53c2095640 1 --2- 172.21.15.139:0/127690198 >> v2:172.21.15.164:3300/0 conn(0x7f53bc0cb650 0x7f53bc0cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.596+0000 7f53c1894640 1 --2- 172.21.15.139:0/127690198 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53bc1528d0 0x7f53bc0c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.596+0000 7f53c0892640 1 -- 172.21.15.139:0/127690198 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1580055696 0 0) 0x7f53b0001110 con 0x7f53bc0c7140 2024-09-16T00:11:23.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.596+0000 7f53c0892640 1 -- 172.21.15.139:0/127690198 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53a4003520 con 0x7f53bc0c7140 2024-09-16T00:11:23.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c0892640 1 -- 172.21.15.139:0/127690198 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1197407222 0 0) 0x7f53a4003520 con 0x7f53bc0c7140 2024-09-16T00:11:23.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c0892640 1 -- 172.21.15.139:0/127690198 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53bc1528d0 msgr2=0x7f53bc0c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:23.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c0892640 1 --2- 172.21.15.139:0/127690198 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53bc1528d0 0x7f53bc0c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c0892640 1 -- 172.21.15.139:0/127690198 >> v2:172.21.15.164:3300/0 conn(0x7f53bc0cb650 msgr2=0x7f53bc0cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:23.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c0892640 1 --2- 172.21.15.139:0/127690198 >> v2:172.21.15.164:3300/0 conn(0x7f53bc0cb650 0x7f53bc0cbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c0892640 1 -- 172.21.15.139:0/127690198 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53bc159180 con 0x7f53bc0c7140 2024-09-16T00:11:23.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c0892640 1 -- 172.21.15.139:0/127690198 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7f53b0001840 con 0x7f53bc0c7140 2024-09-16T00:11:23.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c2095640 1 --2- 172.21.15.139:0/127690198 >> v2:172.21.15.164:3300/0 conn(0x7f53bc0cb650 0x7f53bc0cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:11:23.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c0892640 1 -- 172.21.15.139:0/127690198 >> v1:172.21.15.139:6789/0 conn(0x7f53bc0c7140 legacy=0x7f53bc0c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c0892640 1 --2- 172.21.15.139:0/127690198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53a40039b0 0x7f53a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c0892640 1 --2- 172.21.15.139:0/127690198 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53bc1528d0 0x7f53a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c0892640 1 --2- 172.21.15.139:0/127690198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53a4004d40 0x7f53a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c0892640 1 -- 172.21.15.139:0/127690198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc0cc040 con 0x7f53a4004d40 2024-09-16T00:11:23.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c0892640 1 -- 172.21.15.139:0/127690198 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc067fb0 con 0x7f53a40039b0 2024-09-16T00:11:23.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c0892640 1 -- 172.21.15.139:0/127690198 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f53a400b9c0 con 0x7f53bc1528d0 2024-09-16T00:11:23.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c2095640 1 --2- 172.21.15.139:0/127690198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53a4004d40 0x7f53a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c1894640 1 --2- 172.21.15.139:0/127690198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53a40039b0 0x7f53a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c1093640 1 --2- 172.21.15.139:0/127690198 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53bc1528d0 0x7f53a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c2095640 1 -- 172.21.15.139:0/127690198 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53bc1528d0 msgr2=0x7f53a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:23.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c2095640 1 --2- 172.21.15.139:0/127690198 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53bc1528d0 0x7f53a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c2095640 1 -- 172.21.15.139:0/127690198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53a40039b0 msgr2=0x7f53a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:23.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c2095640 1 --2- 172.21.15.139:0/127690198 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53a40039b0 0x7f53a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.597+0000 7f53c2095640 1 -- 172.21.15.139:0/127690198 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a4003520 con 0x7f53a4004d40 2024-09-16T00:11:23.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.598+0000 7f53c2095640 1 --2- 172.21.15.139:0/127690198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53a4004d40 0x7f53a40051c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009c90 tx=0x7f53a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb42e9469ef3a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:23.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.598+0000 7f53c0892640 1 -- 172.21.15.139:0/127690198 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4036070 con 0x7f53a4004d40 2024-09-16T00:11:23.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.598+0000 7f53c0892640 1 -- 172.21.15.139:0/127690198 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53b4031400 con 0x7f53a4004d40 2024-09-16T00:11:23.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.598+0000 7f53c0892640 1 -- 172.21.15.139:0/127690198 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4009860 con 0x7f53a4004d40 2024-09-16T00:11:23.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.599+0000 7f53c3af8640 1 -- 172.21.15.139:0/127690198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53a4004d40 msgr2=0x7f53a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.599+0000 7f53c3af8640 1 --2- 172.21.15.139:0/127690198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53a4004d40 0x7f53a40051c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f53b4009c90 tx=0x7f53a4005830 comp rx=0 tx=0).stop 2024-09-16T00:11:23.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.599+0000 7f53c2095640 1 -- 172.21.15.139:0/127690198 reap_dead start 2024-09-16T00:11:23.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.599+0000 7f53c3af8640 1 -- 172.21.15.139:0/127690198 shutdown_connections 2024-09-16T00:11:23.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.599+0000 7f53c3af8640 1 -- 172.21.15.139:0/127690198 >> 172.21.15.139:0/127690198 conn(0x7f53bc0c9240 msgr2=0x7f53bc0c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:23.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.599+0000 7f53c3af8640 1 -- 172.21.15.139:0/127690198 shutdown_connections 2024-09-16T00:11:23.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.599+0000 7f53c3af8640 1 -- 172.21.15.139:0/127690198 wait complete. 2024-09-16T00:11:23.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.600+0000 7f53c3af8640 1 Processor -- start 2024-09-16T00:11:23.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.600+0000 7f53c3af8640 1 -- start start 2024-09-16T00:11:23.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c3af8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53bc1e53b0 0x7f53bc148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c3af8640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53bc1e5790 0x7f53bc1edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c3af8640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53bc1563d0 0x7f53bc1ee510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c3af8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc159460 con 0x7f53bc1e53b0 2024-09-16T00:11:23.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c3af8640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f53bc067cd0 con 0x7f53bc1e5790 2024-09-16T00:11:23.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c3af8640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f53bc0c7560 con 0x7f53bc1563d0 2024-09-16T00:11:23.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c1894640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53bc1e53b0 0x7f53bc148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c1894640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53bc1e53b0 0x7f53bc148c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43554/0 (socket says 172.21.15.139:43554) 2024-09-16T00:11:23.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c1894640 1 -- 172.21.15.139:0/1481177337 learned_addr learned my addr 172.21.15.139:0/1481177337 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:23.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c2095640 1 --2- 172.21.15.139:0/1481177337 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53bc1563d0 0x7f53bc1ee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c1093640 1 --2- 172.21.15.139:0/1481177337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53bc1e5790 0x7f53bc1edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c1894640 1 -- 172.21.15.139:0/1481177337 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53bc1563d0 msgr2=0x7f53bc1ee510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c1894640 1 --2- 172.21.15.139:0/1481177337 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53bc1563d0 0x7f53bc1ee510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c1894640 1 -- 172.21.15.139:0/1481177337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53bc1e5790 msgr2=0x7f53bc1edfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c1894640 1 --2- 172.21.15.139:0/1481177337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53bc1e5790 0x7f53bc1edfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c1894640 1 -- 172.21.15.139:0/1481177337 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53bc149290 con 0x7f53bc1e53b0 2024-09-16T00:11:23.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c2095640 1 --2- 172.21.15.139:0/1481177337 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f53bc1563d0 0x7f53bc1ee510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:23.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c1093640 1 --2- 172.21.15.139:0/1481177337 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f53bc1e5790 0x7f53bc1edfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:23.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c1894640 1 --2- 172.21.15.139:0/1481177337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53bc1e53b0 0x7f53bc148c30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009f90 tx=0x7f53ac00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:23.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53ba7fc640 1 -- 172.21.15.139:0/1481177337 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac00fea0 con 0x7f53bc1e53b0 2024-09-16T00:11:23.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53ba7fc640 1 -- 172.21.15.139:0/1481177337 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ac015630 con 0x7f53bc1e53b0 2024-09-16T00:11:23.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c3af8640 1 -- 172.21.15.139:0/1481177337 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53bc1de640 con 0x7f53bc1e53b0 2024-09-16T00:11:23.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53ba7fc640 1 -- 172.21.15.139:0/1481177337 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac015a20 con 0x7f53bc1e53b0 2024-09-16T00:11:23.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.601+0000 7f53c3af8640 1 -- 172.21.15.139:0/1481177337 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53bc1dec00 con 0x7f53bc1e53b0 2024-09-16T00:11:23.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.602+0000 7f53c3af8640 1 -- 172.21.15.139:0/1481177337 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f53bc0ca6d0 con 0x7f53bc1e53b0 2024-09-16T00:11:23.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.603+0000 7f53ba7fc640 1 -- 172.21.15.139:0/1481177337 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f53ac015bc0 con 0x7f53bc1e53b0 2024-09-16T00:11:23.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.603+0000 7f53ba7fc640 1 --2- 172.21.15.139:0/1481177337 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f53a806c2e0 0x7f53a806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.603+0000 7f53c1093640 1 --2- 172.21.15.139:0/1481177337 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f53a806c2e0 0x7f53a806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.603+0000 7f53ba7fc640 1 -- 172.21.15.139:0/1481177337 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f53ac08c680 con 0x7f53bc1e53b0 2024-09-16T00:11:23.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.603+0000 7f53ba7fc640 1 --2- 172.21.15.139:0/1481177337 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f53a80747e0 0x7f53a8076c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.603+0000 7f53ba7fc640 1 -- 172.21.15.139:0/1481177337 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53ac025ec0 con 0x7f53a80747e0 2024-09-16T00:11:23.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.603+0000 7f53ba7fc640 1 -- 172.21.15.139:0/1481177337 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7f53ac093050 con 0x7f53bc1e53b0 2024-09-16T00:11:23.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.604+0000 7f53c1093640 1 --2- 172.21.15.139:0/1481177337 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f53a806c2e0 0x7f53a806e7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f53b00012f0 tx=0x7f53b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:23.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.604+0000 7f53c2095640 1 --2- 172.21.15.139:0/1481177337 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f53a80747e0 0x7f53a8076c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.604+0000 7f53c2095640 1 --2- 172.21.15.139:0/1481177337 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f53a80747e0 0x7f53a8076c80 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:23.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.605+0000 7f53ba7fc640 1 -- 172.21.15.139:0/1481177337 <== osd.4 v2:172.21.15.164:6816/642632916 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f53ac025ec0 con 0x7f53a80747e0 2024-09-16T00:11:23.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.617+0000 7f53c3af8640 1 -- 172.21.15.139:0/1481177337 --> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f53bc1ed9f0 con 0x7f53a80747e0 2024-09-16T00:11:23.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.618+0000 7f53ba7fc640 1 -- 172.21.15.139:0/1481177337 <== osd.4 v2:172.21.15.164:6816/642632916 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f53bc1ed9f0 con 0x7f53a80747e0 2024-09-16T00:11:23.619 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:11:23.619 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T00:11:23.619 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:11:23.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.618+0000 7f53c3af8640 1 -- 172.21.15.139:0/1481177337 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f53a80747e0 msgr2=0x7f53a8076c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.618+0000 7f53c3af8640 1 --2- 172.21.15.139:0/1481177337 >> [v2:172.21.15.164:6816/642632916,v1:172.21.15.164:6817/642632916] conn(0x7f53a80747e0 0x7f53a8076c80 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.618+0000 7f53c3af8640 1 -- 172.21.15.139:0/1481177337 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f53a806c2e0 msgr2=0x7f53a806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.618+0000 7f53c3af8640 1 --2- 172.21.15.139:0/1481177337 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f53a806c2e0 0x7f53a806e7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f53b00012f0 tx=0x7f53b0009040 comp rx=0 tx=0).stop 2024-09-16T00:11:23.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.618+0000 7f53c3af8640 1 -- 172.21.15.139:0/1481177337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53bc1e53b0 msgr2=0x7f53bc148c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.618+0000 7f53c3af8640 1 --2- 172.21.15.139:0/1481177337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53bc1e53b0 0x7f53bc148c30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009f90 tx=0x7f53ac00f580 comp rx=0 tx=0).stop 2024-09-16T00:11:23.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.618+0000 7f53c2095640 1 -- 172.21.15.139:0/1481177337 reap_dead start 2024-09-16T00:11:23.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.618+0000 7f53c3af8640 1 -- 172.21.15.139:0/1481177337 shutdown_connections 2024-09-16T00:11:23.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.618+0000 7f53c3af8640 1 -- 172.21.15.139:0/1481177337 >> 172.21.15.139:0/1481177337 conn(0x7f53bc0c9240 msgr2=0x7f53bc072a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:23.620 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.618+0000 7f53c3af8640 1 -- 172.21.15.139:0/1481177337 shutdown_connections 2024-09-16T00:11:23.621 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.618+0000 7f53c3af8640 1 -- 172.21.15.139:0/1481177337 wait complete. 2024-09-16T00:11:23.629 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-16T00:11:23.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.727+0000 7fe8a4b98640 1 Processor -- start 2024-09-16T00:11:23.728 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.727+0000 7fe8a4b98640 1 -- start start 2024-09-16T00:11:23.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.727+0000 7fe8a4b98640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a00c61d0 0x7fe8a0158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.728+0000 7fe8a4b98640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe8a006d780 0x7fe8a0073210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.728+0000 7fe8a4b98640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8a006bd70 con 0x7fe8a00c65b0 2024-09-16T00:11:23.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.728+0000 7fe8a4b98640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe8a006bb20 con 0x7fe8a006d780 2024-09-16T00:11:23.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.728+0000 7fe8a4b98640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a00c89e0 con 0x7fe8a00c61d0 2024-09-16T00:11:23.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.728+0000 7fe89ed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe8a006d780 0x7fe8a0073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.728+0000 7fe895d9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe8a00c65b0 0x7fe8a0152380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45792/0 (socket says 172.21.15.139:45792) 2024-09-16T00:11:23.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.728+0000 7fe895d9b640 1 -- 172.21.15.139:0/974253156 learned_addr learned my addr 172.21.15.139:0/974253156 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:23.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.728+0000 7fe89e59c640 1 --2- 172.21.15.139:0/974253156 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a00c61d0 0x7fe8a0158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.728+0000 7fe89ed9d640 1 -- 172.21.15.139:0/974253156 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a00c61d0 msgr2=0x7fe8a0158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:23.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.728+0000 7fe89ed9d640 1 --2- 172.21.15.139:0/974253156 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a00c61d0 0x7fe8a0158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.728+0000 7fe89ed9d640 1 -- 172.21.15.139:0/974253156 >> v1:172.21.15.139:6789/0 conn(0x7fe8a00c65b0 legacy=0x7fe8a0152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.728+0000 7fe89ed9d640 1 -- 172.21.15.139:0/974253156 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a006c050 con 0x7fe8a006d780 2024-09-16T00:11:23.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.728+0000 7fe89dd9b640 1 -- 172.21.15.139:0/974253156 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68486609 0 0) 0x7fe8a006bd70 con 0x7fe8a00c65b0 2024-09-16T00:11:23.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.728+0000 7fe89e59c640 1 --2- 172.21.15.139:0/974253156 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a00c61d0 0x7fe8a0158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:23.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.729+0000 7fe89ed9d640 1 --2- 172.21.15.139:0/974253156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a006d780 0x7fe8a0073210 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe898009d90 tx=0x7fe89802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa14fad8e6704961 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:23.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.729+0000 7fe89dd9b640 1 -- 172.21.15.139:0/974253156 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898007740 con 0x7fe8a006d780 2024-09-16T00:11:23.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.729+0000 7fe89dd9b640 1 -- 172.21.15.139:0/974253156 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8980078e0 con 0x7fe8a006d780 2024-09-16T00:11:23.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.729+0000 7fe89dd9b640 1 -- 172.21.15.139:0/974253156 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898007cb0 con 0x7fe8a006d780 2024-09-16T00:11:23.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.729+0000 7fe8a4b98640 1 -- 172.21.15.139:0/974253156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a006d780 msgr2=0x7fe8a0073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.729+0000 7fe8a4b98640 1 --2- 172.21.15.139:0/974253156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a006d780 0x7fe8a0073210 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe898009d90 tx=0x7fe89802a6f0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.729+0000 7fe8a4b98640 1 -- 172.21.15.139:0/974253156 shutdown_connections 2024-09-16T00:11:23.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.729+0000 7fe8a4b98640 1 --2- 172.21.15.139:0/974253156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a006d780 0x7fe8a0073210 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.729+0000 7fe8a4b98640 1 --2- 172.21.15.139:0/974253156 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a00c61d0 0x7fe8a0158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.729+0000 7fe8a4b98640 1 -- 172.21.15.139:0/974253156 >> 172.21.15.139:0/974253156 conn(0x7fe8a00c9350 msgr2=0x7fe8a00c9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:23.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.729+0000 7fe8a4b98640 1 -- 172.21.15.139:0/974253156 shutdown_connections 2024-09-16T00:11:23.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.729+0000 7fe8a4b98640 1 -- 172.21.15.139:0/974253156 wait complete. 2024-09-16T00:11:23.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.731+0000 7fe8a4b98640 1 Processor -- start 2024-09-16T00:11:23.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.731+0000 7fe8a4b98640 1 -- start start 2024-09-16T00:11:23.736 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.731+0000 7fe8a4b98640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a006d780 0x7fe8a01599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.731+0000 7fe8a4b98640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a00c61d0 0x7fe8a015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe8a4b98640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a00c65b0 0x7fe8a015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe8a4b98640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a0072930 con 0x7fe8a00c65b0 2024-09-16T00:11:23.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe8a4b98640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a00c8f10 con 0x7fe8a00c61d0 2024-09-16T00:11:23.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe8a4b98640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a0152a90 con 0x7fe8a006d780 2024-09-16T00:11:23.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe89ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a00c65b0 0x7fe8a015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe89ed9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a00c65b0 0x7fe8a015bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43580/0 (socket says 172.21.15.139:43580) 2024-09-16T00:11:23.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe89ed9d640 1 -- 172.21.15.139:0/607425950 learned_addr learned my addr 172.21.15.139:0/607425950 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:23.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe895d9b640 1 --2- 172.21.15.139:0/607425950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a00c61d0 0x7fe8a015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe89e59c640 1 --2- 172.21.15.139:0/607425950 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a006d780 0x7fe8a01599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe89ed9d640 1 -- 172.21.15.139:0/607425950 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a006d780 msgr2=0x7fe8a01599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe89ed9d640 1 --2- 172.21.15.139:0/607425950 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a006d780 0x7fe8a01599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe89ed9d640 1 -- 172.21.15.139:0/607425950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a00c61d0 msgr2=0x7fe8a015bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe89ed9d640 1 --2- 172.21.15.139:0/607425950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a00c61d0 0x7fe8a015bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe89ed9d640 1 -- 172.21.15.139:0/607425950 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a0155ed0 con 0x7fe8a00c65b0 2024-09-16T00:11:23.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe895d9b640 1 --2- 172.21.15.139:0/607425950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a00c61d0 0x7fe8a015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:23.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe89ed9d640 1 --2- 172.21.15.139:0/607425950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a00c65b0 0x7fe8a015bff0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe898009eb0 tx=0x7fe898004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:23.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe8977fe640 1 -- 172.21.15.139:0/607425950 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898040070 con 0x7fe8a00c65b0 2024-09-16T00:11:23.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe8977fe640 1 -- 172.21.15.139:0/607425950 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe898031c40 con 0x7fe8a00c65b0 2024-09-16T00:11:23.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe8a4b98640 1 -- 172.21.15.139:0/607425950 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a0151230 con 0x7fe8a00c65b0 2024-09-16T00:11:23.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe8977fe640 1 -- 172.21.15.139:0/607425950 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898003990 con 0x7fe8a00c65b0 2024-09-16T00:11:23.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.732+0000 7fe8a4b98640 1 -- 172.21.15.139:0/607425950 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a0155bf0 con 0x7fe8a00c65b0 2024-09-16T00:11:23.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.733+0000 7fe8a4b98640 1 -- 172.21.15.139:0/607425950 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a0158ff0 con 0x7fe8a00c65b0 2024-09-16T00:11:23.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.734+0000 7fe8977fe640 1 -- 172.21.15.139:0/607425950 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe898039040 con 0x7fe8a00c65b0 2024-09-16T00:11:23.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.734+0000 7fe8977fe640 1 --2- 172.21.15.139:0/607425950 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe88006c2b0 0x7fe88006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:23.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.734+0000 7fe8977fe640 1 -- 172.21.15.139:0/607425950 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7fe8980ac8c0 con 0x7fe8a00c65b0 2024-09-16T00:11:23.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.734+0000 7fe89e59c640 1 --2- 172.21.15.139:0/607425950 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe88006c2b0 0x7fe88006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:23.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.735+0000 7fe89e59c640 1 --2- 172.21.15.139:0/607425950 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe88006c2b0 0x7fe88006e770 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe890008850 tx=0x7fe890008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:23.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.739+0000 7fe8977fe640 1 -- 172.21.15.139:0/607425950 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe89807aec0 con 0x7fe8a00c65b0 2024-09-16T00:11:23.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.906+0000 7fe8a4b98640 1 -- 172.21.15.139:0/607425950 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fe8a0151510 con 0x7fe88006c2b0 2024-09-16T00:11:23.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.907+0000 7fe8977fe640 1 -- 172.21.15.139:0/607425950 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe890008540 con 0x7fe88006c2b0 2024-09-16T00:11:23.908 INFO:teuthology.orchestra.run.smithi139.stderr:instructed osd(s) 4 to deep-scrub 2024-09-16T00:11:23.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.909+0000 7fe8a4b98640 1 -- 172.21.15.139:0/607425950 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe88006c2b0 msgr2=0x7fe88006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.909+0000 7fe8a4b98640 1 --2- 172.21.15.139:0/607425950 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe88006c2b0 0x7fe88006e770 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe890008850 tx=0x7fe890008040 comp rx=0 tx=0).stop 2024-09-16T00:11:23.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.909+0000 7fe8a4b98640 1 -- 172.21.15.139:0/607425950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a00c65b0 msgr2=0x7fe8a015bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:23.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.909+0000 7fe8a4b98640 1 --2- 172.21.15.139:0/607425950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a00c65b0 0x7fe8a015bff0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe898009eb0 tx=0x7fe898004a40 comp rx=0 tx=0).stop 2024-09-16T00:11:23.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.909+0000 7fe8a4b98640 1 -- 172.21.15.139:0/607425950 shutdown_connections 2024-09-16T00:11:23.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.909+0000 7fe8a4b98640 1 --2- 172.21.15.139:0/607425950 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fe88006c2b0 0x7fe88006e770 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.909+0000 7fe8a4b98640 1 --2- 172.21.15.139:0/607425950 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8a00c65b0 0x7fe8a015bff0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.909+0000 7fe8a4b98640 1 --2- 172.21.15.139:0/607425950 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe8a00c61d0 0x7fe8a015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.909+0000 7fe8a4b98640 1 --2- 172.21.15.139:0/607425950 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fe8a006d780 0x7fe8a01599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:23.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.909+0000 7fe8a4b98640 1 -- 172.21.15.139:0/607425950 >> 172.21.15.139:0/607425950 conn(0x7fe8a00c9350 msgr2=0x7fe8a014e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:23.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.909+0000 7fe8a4b98640 1 -- 172.21.15.139:0/607425950 shutdown_connections 2024-09-16T00:11:23.911 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:23.909+0000 7fe8a4b98640 1 -- 172.21.15.139:0/607425950 wait complete. 2024-09-16T00:11:23.921 INFO:tasks.ceph:Scrubbing osd.5 2024-09-16T00:11:23.921 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-16T00:11:24.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.024+0000 7f091489b640 1 Processor -- start 2024-09-16T00:11:24.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.024+0000 7f091489b640 1 -- start start 2024-09-16T00:11:24.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.025+0000 7f091489b640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0910151a70 0x7f09100c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.025+0000 7f091489b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f09100c57d0 0x7f09100c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.025+0000 7f091489b640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0910072480 con 0x7f0910151690 2024-09-16T00:11:24.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.025+0000 7f091489b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f09100c6f80 con 0x7f09100c57d0 2024-09-16T00:11:24.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.025+0000 7f091489b640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f09100c5340 con 0x7f0910151a70 2024-09-16T00:11:24.026 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.025+0000 7f090ed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f09100c57d0 0x7f09100c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.025+0000 7f090ed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f09100c57d0 0x7f09100c71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43586/0 (socket says 172.21.15.139:43586) 2024-09-16T00:11:24.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.025+0000 7f090ed9d640 1 -- 172.21.15.139:0/2311607087 learned_addr learned my addr 172.21.15.139:0/2311607087 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:24.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.025+0000 7f090ed9d640 1 -- 172.21.15.139:0/2311607087 >> v2:172.21.15.164:3300/0 conn(0x7f0910151a70 msgr2=0x7f09100c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:24.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.025+0000 7f090ed9d640 1 --2- 172.21.15.139:0/2311607087 >> v2:172.21.15.164:3300/0 conn(0x7f0910151a70 0x7f09100c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.025+0000 7f090ed9d640 1 -- 172.21.15.139:0/2311607087 >> v1:172.21.15.164:6789/0 conn(0x7f0910151690 legacy=0x7f09100c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:24.027 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.025+0000 7f090ed9d640 1 -- 172.21.15.139:0/2311607087 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0910072a70 con 0x7f09100c57d0 2024-09-16T00:11:24.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.026+0000 7f090ed9d640 1 --2- 172.21.15.139:0/2311607087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09100c57d0 0x7f09100c71a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0910071690 tx=0x7f090802a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b00fce3cfe56d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.026+0000 7f090dd9b640 1 -- 172.21.15.139:0/2311607087 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908036070 con 0x7f09100c57d0 2024-09-16T00:11:24.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.026+0000 7f090dd9b640 1 -- 172.21.15.139:0/2311607087 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0908031600 con 0x7f09100c57d0 2024-09-16T00:11:24.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.026+0000 7f090dd9b640 1 -- 172.21.15.139:0/2311607087 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09080319d0 con 0x7f09100c57d0 2024-09-16T00:11:24.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.026+0000 7f091489b640 1 -- 172.21.15.139:0/2311607087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09100c57d0 msgr2=0x7f09100c71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.026+0000 7f091489b640 1 --2- 172.21.15.139:0/2311607087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09100c57d0 0x7f09100c71a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0910071690 tx=0x7f090802a780 comp rx=0 tx=0).stop 2024-09-16T00:11:24.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.026+0000 7f091489b640 1 -- 172.21.15.139:0/2311607087 shutdown_connections 2024-09-16T00:11:24.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.026+0000 7f091489b640 1 --2- 172.21.15.139:0/2311607087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09100c57d0 0x7f09100c71a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.026+0000 7f091489b640 1 --2- 172.21.15.139:0/2311607087 >> v2:172.21.15.164:3300/0 conn(0x7f0910151a70 0x7f09100c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.026+0000 7f091489b640 1 -- 172.21.15.139:0/2311607087 >> 172.21.15.139:0/2311607087 conn(0x7f091006f440 msgr2=0x7f091006f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:24.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.027+0000 7f091489b640 1 -- 172.21.15.139:0/2311607087 shutdown_connections 2024-09-16T00:11:24.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.027+0000 7f091489b640 1 -- 172.21.15.139:0/2311607087 wait complete. 2024-09-16T00:11:24.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.028+0000 7f091489b640 1 Processor -- start 2024-09-16T00:11:24.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.028+0000 7f091489b640 1 -- start start 2024-09-16T00:11:24.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.028+0000 7f091489b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09100c57d0 0x7f09101efdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.028+0000 7f091489b640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0910151690 0x7f09100c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.028+0000 7f091489b640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0910151a70 0x7f091014c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.028+0000 7f091489b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0910072120 con 0x7f09100c57d0 2024-09-16T00:11:24.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.028+0000 7f091489b640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0910073030 con 0x7f0910151690 2024-09-16T00:11:24.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.028+0000 7f091489b640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f09100c7c90 con 0x7f0910151a70 2024-09-16T00:11:24.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.029+0000 7f090e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09100c57d0 0x7f09101efdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.029+0000 7f090e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09100c57d0 0x7f09101efdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43592/0 (socket says 172.21.15.139:43592) 2024-09-16T00:11:24.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.029+0000 7f090e59c640 1 -- 172.21.15.139:0/3560200261 learned_addr learned my addr 172.21.15.139:0/3560200261 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:24.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.029+0000 7f090e59c640 1 -- 172.21.15.139:0/3560200261 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0910151a70 msgr2=0x7f091014c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.029+0000 7f090e59c640 1 --2- 172.21.15.139:0/3560200261 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0910151a70 0x7f091014c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.029+0000 7f090e59c640 1 -- 172.21.15.139:0/3560200261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0910151690 msgr2=0x7f09100c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.029+0000 7f090e59c640 1 --2- 172.21.15.139:0/3560200261 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0910151690 0x7f09100c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.029+0000 7f090e59c640 1 -- 172.21.15.139:0/3560200261 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09101f0310 con 0x7f09100c57d0 2024-09-16T00:11:24.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.029+0000 7f090e59c640 1 --2- 172.21.15.139:0/3560200261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09100c57d0 0x7f09101efdd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f090000b410 tx=0x7f090000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.029+0000 7f0906ffd640 1 -- 172.21.15.139:0/3560200261 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900015020 con 0x7f09100c57d0 2024-09-16T00:11:24.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.029+0000 7f0906ffd640 1 -- 172.21.15.139:0/3560200261 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0900011830 con 0x7f09100c57d0 2024-09-16T00:11:24.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.029+0000 7f0906ffd640 1 -- 172.21.15.139:0/3560200261 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900011c20 con 0x7f09100c57d0 2024-09-16T00:11:24.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.029+0000 7f091489b640 1 -- 172.21.15.139:0/3560200261 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09101e0d70 con 0x7f09100c57d0 2024-09-16T00:11:24.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.029+0000 7f091489b640 1 -- 172.21.15.139:0/3560200261 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09101e1010 con 0x7f09100c57d0 2024-09-16T00:11:24.033 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.030+0000 7f091489b640 1 -- 172.21.15.139:0/3560200261 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f09101e2240 con 0x7f09100c57d0 2024-09-16T00:11:24.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.031+0000 7f0906ffd640 1 -- 172.21.15.139:0/3560200261 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0900018020 con 0x7f09100c57d0 2024-09-16T00:11:24.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.031+0000 7f0906ffd640 1 --2- 172.21.15.139:0/3560200261 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08f006c2b0 0x7f08f006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.031+0000 7f0907fff640 1 --2- 172.21.15.139:0/3560200261 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08f006c2b0 0x7f08f006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.031+0000 7f0906ffd640 1 -- 172.21.15.139:0/3560200261 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f090008b4c0 con 0x7f09100c57d0 2024-09-16T00:11:24.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.031+0000 7f0906ffd640 1 --2- 172.21.15.139:0/3560200261 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f08f0074810 0x7f08f0076cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.031+0000 7f0906ffd640 1 -- 172.21.15.139:0/3560200261 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08f0077380 con 0x7f08f0074810 2024-09-16T00:11:24.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.031+0000 7f0906ffd640 1 -- 172.21.15.139:0/3560200261 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7f0900092050 con 0x7f09100c57d0 2024-09-16T00:11:24.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.032+0000 7f0907fff640 1 --2- 172.21.15.139:0/3560200261 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08f006c2b0 0x7f08f006e770 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f091014ee60 tx=0x7f08f8006c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.032+0000 7f090ed9d640 1 --2- 172.21.15.139:0/3560200261 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f08f0074810 0x7f08f0076cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.032+0000 7f090ed9d640 1 --2- 172.21.15.139:0/3560200261 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f08f0074810 0x7f08f0076cb0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.034 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.033+0000 7f0906ffd640 1 -- 172.21.15.139:0/3560200261 <== osd.5 v2:172.21.15.164:6800/1051778383 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f08f0077380 con 0x7f08f0074810 2024-09-16T00:11:24.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.045+0000 7f091489b640 1 -- 172.21.15.139:0/3560200261 --> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f09100c5c20 con 0x7f08f0074810 2024-09-16T00:11:24.046 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.046+0000 7f0906ffd640 1 -- 172.21.15.139:0/3560200261 <== osd.5 v2:172.21.15.164:6800/1051778383 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f09100c5c20 con 0x7f08f0074810 2024-09-16T00:11:24.047 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:11:24.047 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T00:11:24.047 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:11:24.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.046+0000 7f091489b640 1 -- 172.21.15.139:0/3560200261 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f08f0074810 msgr2=0x7f08f0076cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.046+0000 7f091489b640 1 --2- 172.21.15.139:0/3560200261 >> [v2:172.21.15.164:6800/1051778383,v1:172.21.15.164:6801/1051778383] conn(0x7f08f0074810 0x7f08f0076cb0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.046+0000 7f091489b640 1 -- 172.21.15.139:0/3560200261 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08f006c2b0 msgr2=0x7f08f006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.046+0000 7f091489b640 1 --2- 172.21.15.139:0/3560200261 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f08f006c2b0 0x7f08f006e770 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f091014ee60 tx=0x7f08f8006c90 comp rx=0 tx=0).stop 2024-09-16T00:11:24.048 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.046+0000 7f091489b640 1 -- 172.21.15.139:0/3560200261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09100c57d0 msgr2=0x7f09101efdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.046+0000 7f091489b640 1 --2- 172.21.15.139:0/3560200261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09100c57d0 0x7f09101efdd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f090000b410 tx=0x7f090000b8d0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.046+0000 7f090ed9d640 1 -- 172.21.15.139:0/3560200261 reap_dead start 2024-09-16T00:11:24.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.046+0000 7f091489b640 1 -- 172.21.15.139:0/3560200261 shutdown_connections 2024-09-16T00:11:24.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.046+0000 7f091489b640 1 -- 172.21.15.139:0/3560200261 >> 172.21.15.139:0/3560200261 conn(0x7f091006f440 msgr2=0x7f09100c7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:24.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.046+0000 7f091489b640 1 -- 172.21.15.139:0/3560200261 shutdown_connections 2024-09-16T00:11:24.049 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.046+0000 7f091489b640 1 -- 172.21.15.139:0/3560200261 wait complete. 2024-09-16T00:11:24.057 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-16T00:11:24.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.155+0000 7fb660427640 1 Processor -- start 2024-09-16T00:11:24.157 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.156+0000 7fb660427640 1 -- start start 2024-09-16T00:11:24.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.156+0000 7fb660427640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb6580ccff0 0x7fb65806bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.156+0000 7fb660427640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb65806c3f0 con 0x7fb65815b390 2024-09-16T00:11:24.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.156+0000 7fb660427640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb65806f510 con 0x7fb6580c8040 2024-09-16T00:11:24.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.156+0000 7fb660427640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb65815a780 con 0x7fb6580ccff0 2024-09-16T00:11:24.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.157+0000 7fb65e1c3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb6580ccff0 0x7fb65806bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.157+0000 7fb65e9c4640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fb65815b390 0x7fb658165100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45808/0 (socket says 172.21.15.139:45808) 2024-09-16T00:11:24.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.157+0000 7fb65e9c4640 1 -- 172.21.15.139:0/3062769555 learned_addr learned my addr 172.21.15.139:0/3062769555 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:24.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.157+0000 7fb65e1c3640 1 -- 172.21.15.139:0/3062769555 >> v1:172.21.15.164:6789/0 conn(0x7fb6580c8040 legacy=0x7fb65815ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.157+0000 7fb65e1c3640 1 -- 172.21.15.139:0/3062769555 >> v1:172.21.15.139:6789/0 conn(0x7fb65815b390 legacy=0x7fb658165100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.157+0000 7fb65d1c1640 1 -- 172.21.15.139:0/3062769555 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 205213562 0 0) 0x7fb65806c3f0 con 0x7fb65815b390 2024-09-16T00:11:24.158 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.157+0000 7fb65e1c3640 1 -- 172.21.15.139:0/3062769555 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65806f7b0 con 0x7fb6580ccff0 2024-09-16T00:11:24.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.158+0000 7fb65e1c3640 1 --2- 172.21.15.139:0/3062769555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6580ccff0 0x7fb65806bcb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0097a0 tx=0x7fb64c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b991fd9358f892ad server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.158+0000 7fb65d1c1640 1 -- 172.21.15.139:0/3062769555 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c037070 con 0x7fb6580ccff0 2024-09-16T00:11:24.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.158+0000 7fb65d1c1640 1 -- 172.21.15.139:0/3062769555 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb64c032440 con 0x7fb6580ccff0 2024-09-16T00:11:24.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.158+0000 7fb65d1c1640 1 -- 172.21.15.139:0/3062769555 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c032810 con 0x7fb6580ccff0 2024-09-16T00:11:24.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.158+0000 7fb660427640 1 -- 172.21.15.139:0/3062769555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6580ccff0 msgr2=0x7fb65806bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.158+0000 7fb660427640 1 --2- 172.21.15.139:0/3062769555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6580ccff0 0x7fb65806bcb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb64c0097a0 tx=0x7fb64c02a230 comp rx=0 tx=0).stop 2024-09-16T00:11:24.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.158+0000 7fb660427640 1 -- 172.21.15.139:0/3062769555 shutdown_connections 2024-09-16T00:11:24.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.158+0000 7fb660427640 1 --2- 172.21.15.139:0/3062769555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6580ccff0 0x7fb65806bcb0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.158+0000 7fb660427640 1 -- 172.21.15.139:0/3062769555 >> 172.21.15.139:0/3062769555 conn(0x7fb65806cc30 msgr2=0x7fb65806d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:24.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.158+0000 7fb660427640 1 -- 172.21.15.139:0/3062769555 shutdown_connections 2024-09-16T00:11:24.160 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.158+0000 7fb660427640 1 -- 172.21.15.139:0/3062769555 wait complete. 2024-09-16T00:11:24.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.159+0000 7fb660427640 1 Processor -- start 2024-09-16T00:11:24.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.160+0000 7fb660427640 1 -- start start 2024-09-16T00:11:24.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.160+0000 7fb660427640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6580c8040 0x7fb6581e4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.160+0000 7fb660427640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6580ccff0 0x7fb6581f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.160+0000 7fb660427640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb65815b390 0x7fb6580c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.160+0000 7fb660427640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb658154b30 con 0x7fb6580ccff0 2024-09-16T00:11:24.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.160+0000 7fb660427640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fb65806fd70 con 0x7fb6580c8040 2024-09-16T00:11:24.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.160+0000 7fb660427640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fb658154cb0 con 0x7fb65815b390 2024-09-16T00:11:24.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.160+0000 7fb65d9c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6580ccff0 0x7fb6581f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.160+0000 7fb65d9c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6580ccff0 0x7fb6581f47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43612/0 (socket says 172.21.15.139:43612) 2024-09-16T00:11:24.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.160+0000 7fb65d9c2640 1 -- 172.21.15.139:0/2465520776 learned_addr learned my addr 172.21.15.139:0/2465520776 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:24.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.160+0000 7fb65e1c3640 1 --2- 172.21.15.139:0/2465520776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6580c8040 0x7fb6581e4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.160+0000 7fb65e9c4640 1 --2- 172.21.15.139:0/2465520776 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb65815b390 0x7fb6580c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.161+0000 7fb65d9c2640 1 -- 172.21.15.139:0/2465520776 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb65815b390 msgr2=0x7fb6580c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.161+0000 7fb65d9c2640 1 --2- 172.21.15.139:0/2465520776 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb65815b390 0x7fb6580c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.161+0000 7fb65d9c2640 1 -- 172.21.15.139:0/2465520776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6580c8040 msgr2=0x7fb6581e4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.161+0000 7fb65d9c2640 1 --2- 172.21.15.139:0/2465520776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6580c8040 0x7fb6581e4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.161+0000 7fb65d9c2640 1 -- 172.21.15.139:0/2465520776 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6581e4e80 con 0x7fb6580ccff0 2024-09-16T00:11:24.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.161+0000 7fb65d9c2640 1 --2- 172.21.15.139:0/2465520776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6580ccff0 0x7fb6581f47e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb65400c450 tx=0x7fb65400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.161+0000 7fb64affd640 1 -- 172.21.15.139:0/2465520776 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6540052d0 con 0x7fb6580ccff0 2024-09-16T00:11:24.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.161+0000 7fb64affd640 1 -- 172.21.15.139:0/2465520776 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb654005470 con 0x7fb6580ccff0 2024-09-16T00:11:24.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.161+0000 7fb660427640 1 -- 172.21.15.139:0/2465520776 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6581508c0 con 0x7fb6580ccff0 2024-09-16T00:11:24.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.161+0000 7fb64affd640 1 -- 172.21.15.139:0/2465520776 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb654012870 con 0x7fb6580ccff0 2024-09-16T00:11:24.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.161+0000 7fb660427640 1 -- 172.21.15.139:0/2465520776 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6581f3fe0 con 0x7fb6580ccff0 2024-09-16T00:11:24.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.162+0000 7fb660427640 1 -- 172.21.15.139:0/2465520776 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb658150de0 con 0x7fb6580ccff0 2024-09-16T00:11:24.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.163+0000 7fb64affd640 1 -- 172.21.15.139:0/2465520776 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb654012a10 con 0x7fb6580ccff0 2024-09-16T00:11:24.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.163+0000 7fb64affd640 1 --2- 172.21.15.139:0/2465520776 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb63c06c2b0 0x7fb63c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.163+0000 7fb64affd640 1 -- 172.21.15.139:0/2465520776 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7fb65408b080 con 0x7fb6580ccff0 2024-09-16T00:11:24.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.163+0000 7fb65e1c3640 1 --2- 172.21.15.139:0/2465520776 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb63c06c2b0 0x7fb63c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.164+0000 7fb65e1c3640 1 --2- 172.21.15.139:0/2465520776 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb63c06c2b0 0x7fb63c06e770 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb64c034000 tx=0x7fb64c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.167+0000 7fb64affd640 1 -- 172.21.15.139:0/2465520776 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb6581e4e80 con 0x7fb6580ccff0 2024-09-16T00:11:24.338 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.337+0000 7fb660427640 1 -- 172.21.15.139:0/2465520776 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fb65806ea70 con 0x7fb63c06c2b0 2024-09-16T00:11:24.339 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.338+0000 7fb64affd640 1 -- 172.21.15.139:0/2465520776 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb65806ea70 con 0x7fb63c06c2b0 2024-09-16T00:11:24.339 INFO:teuthology.orchestra.run.smithi139.stderr:instructed osd(s) 5 to deep-scrub 2024-09-16T00:11:24.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.340+0000 7fb660427640 1 -- 172.21.15.139:0/2465520776 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb63c06c2b0 msgr2=0x7fb63c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.340+0000 7fb660427640 1 --2- 172.21.15.139:0/2465520776 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb63c06c2b0 0x7fb63c06e770 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb64c034000 tx=0x7fb64c02e000 comp rx=0 tx=0).stop 2024-09-16T00:11:24.341 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.340+0000 7fb660427640 1 -- 172.21.15.139:0/2465520776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6580ccff0 msgr2=0x7fb6581f47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.340+0000 7fb660427640 1 --2- 172.21.15.139:0/2465520776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6580ccff0 0x7fb6581f47e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb65400c450 tx=0x7fb65400c910 comp rx=0 tx=0).stop 2024-09-16T00:11:24.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.340+0000 7fb660427640 1 -- 172.21.15.139:0/2465520776 shutdown_connections 2024-09-16T00:11:24.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.340+0000 7fb660427640 1 --2- 172.21.15.139:0/2465520776 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fb63c06c2b0 0x7fb63c06e770 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.340+0000 7fb660427640 1 --2- 172.21.15.139:0/2465520776 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fb65815b390 0x7fb6580c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.340+0000 7fb660427640 1 --2- 172.21.15.139:0/2465520776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6580ccff0 0x7fb6581f47e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.340+0000 7fb660427640 1 --2- 172.21.15.139:0/2465520776 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fb6580c8040 0x7fb6581e4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.340+0000 7fb660427640 1 -- 172.21.15.139:0/2465520776 >> 172.21.15.139:0/2465520776 conn(0x7fb65806cc30 msgr2=0x7fb6580c8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:24.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.340+0000 7fb660427640 1 -- 172.21.15.139:0/2465520776 shutdown_connections 2024-09-16T00:11:24.342 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.340+0000 7fb660427640 1 -- 172.21.15.139:0/2465520776 wait complete. 2024-09-16T00:11:24.352 INFO:tasks.ceph:Scrubbing osd.6 2024-09-16T00:11:24.352 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-16T00:11:24.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.456+0000 7f47c5711640 1 Processor -- start 2024-09-16T00:11:24.458 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.456+0000 7f47c5711640 1 -- start start 2024-09-16T00:11:24.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.457+0000 7f47c5711640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f47c00c7c40 0x7f47c00c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.457+0000 7f47c5711640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c00cbb40 con 0x7f47c00c98f0 2024-09-16T00:11:24.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.457+0000 7f47c5711640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c0152fe0 con 0x7f47c0152c00 2024-09-16T00:11:24.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.457+0000 7f47c5711640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f47c00ca810 con 0x7f47c00c7c40 2024-09-16T00:11:24.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.457+0000 7f47bf7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f47c00c7c40 0x7f47c00c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.457+0000 7f47be7fc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f47c00c98f0 0x7f47c00c7530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45822/0 (socket says 172.21.15.139:45822) 2024-09-16T00:11:24.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.457+0000 7f47be7fc640 1 -- 172.21.15.139:0/2908190517 learned_addr learned my addr 172.21.15.139:0/2908190517 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:24.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.458+0000 7f47bf7fe640 1 -- 172.21.15.139:0/2908190517 >> v1:172.21.15.164:6789/0 conn(0x7f47c0152c00 legacy=0x7f47c00cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:24.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.458+0000 7f47bf7fe640 1 -- 172.21.15.139:0/2908190517 >> v1:172.21.15.139:6789/0 conn(0x7f47c00c98f0 legacy=0x7f47c00c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.458+0000 7f47bdffb640 1 -- 172.21.15.139:0/2908190517 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1457066986 0 0) 0x7f47c00cbb40 con 0x7f47c00c98f0 2024-09-16T00:11:24.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.458+0000 7f47bf7fe640 1 -- 172.21.15.139:0/2908190517 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c00c5a90 con 0x7f47c00c7c40 2024-09-16T00:11:24.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.458+0000 7f47bf7fe640 1 --2- 172.21.15.139:0/2908190517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c00c7c40 0x7f47c00c9340 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f47c00ca990 tx=0x7f47b802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21faf068bb8f9ddc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.459+0000 7f47bdffb640 1 -- 172.21.15.139:0/2908190517 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8007740 con 0x7f47c00c7c40 2024-09-16T00:11:24.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.459+0000 7f47bdffb640 1 -- 172.21.15.139:0/2908190517 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b80078e0 con 0x7f47c00c7c40 2024-09-16T00:11:24.460 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.459+0000 7f47bdffb640 1 -- 172.21.15.139:0/2908190517 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8007cb0 con 0x7f47c00c7c40 2024-09-16T00:11:24.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.459+0000 7f47c5711640 1 -- 172.21.15.139:0/2908190517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c00c7c40 msgr2=0x7f47c00c9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.459+0000 7f47c5711640 1 --2- 172.21.15.139:0/2908190517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c00c7c40 0x7f47c00c9340 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f47c00ca990 tx=0x7f47b802a730 comp rx=0 tx=0).stop 2024-09-16T00:11:24.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.459+0000 7f47c5711640 1 -- 172.21.15.139:0/2908190517 shutdown_connections 2024-09-16T00:11:24.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.459+0000 7f47c5711640 1 --2- 172.21.15.139:0/2908190517 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c00c7c40 0x7f47c00c9340 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.459+0000 7f47c5711640 1 -- 172.21.15.139:0/2908190517 >> 172.21.15.139:0/2908190517 conn(0x7f47c006bb20 msgr2=0x7f47c006fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:24.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.459+0000 7f47c5711640 1 -- 172.21.15.139:0/2908190517 shutdown_connections 2024-09-16T00:11:24.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.459+0000 7f47c5711640 1 -- 172.21.15.139:0/2908190517 wait complete. 2024-09-16T00:11:24.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.460+0000 7f47c5711640 1 Processor -- start 2024-09-16T00:11:24.462 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.460+0000 7f47c5711640 1 -- start start 2024-09-16T00:11:24.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.461+0000 7f47c5711640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47c00c7c40 0x7f47c01ee5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.461+0000 7f47c5711640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c00c98f0 0x7f47c0148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.461+0000 7f47c5711640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47c0152c00 0x7f47c01e01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.461+0000 7f47c5711640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f47c00ca1c0 con 0x7f47c00c98f0 2024-09-16T00:11:24.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.461+0000 7f47c5711640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f47c00c9ee0 con 0x7f47c0152c00 2024-09-16T00:11:24.464 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.461+0000 7f47c5711640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f47c00ca4a0 con 0x7f47c00c7c40 2024-09-16T00:11:24.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.461+0000 7f47be7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c00c98f0 0x7f47c0148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.461+0000 7f47be7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c00c98f0 0x7f47c0148740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43640/0 (socket says 172.21.15.139:43640) 2024-09-16T00:11:24.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.461+0000 7f47be7fc640 1 -- 172.21.15.139:0/2771685668 learned_addr learned my addr 172.21.15.139:0/2771685668 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:24.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.461+0000 7f47bf7fe640 1 --2- 172.21.15.139:0/2771685668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47c0152c00 0x7f47c01e01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.461+0000 7f47beffd640 1 --2- 172.21.15.139:0/2771685668 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47c00c7c40 0x7f47c01ee5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.461+0000 7f47be7fc640 1 -- 172.21.15.139:0/2771685668 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47c00c7c40 msgr2=0x7f47c01ee5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.461+0000 7f47be7fc640 1 --2- 172.21.15.139:0/2771685668 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47c00c7c40 0x7f47c01ee5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.461+0000 7f47be7fc640 1 -- 172.21.15.139:0/2771685668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47c0152c00 msgr2=0x7f47c01e01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.461+0000 7f47be7fc640 1 --2- 172.21.15.139:0/2771685668 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f47c0152c00 0x7f47c01e01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.462+0000 7f47be7fc640 1 -- 172.21.15.139:0/2771685668 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c0149a20 con 0x7f47c00c98f0 2024-09-16T00:11:24.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.462+0000 7f47beffd640 1 --2- 172.21.15.139:0/2771685668 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f47c00c7c40 0x7f47c01ee5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:24.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.462+0000 7f47be7fc640 1 --2- 172.21.15.139:0/2771685668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c00c98f0 0x7f47c0148740 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f47b400c500 tx=0x7f47b400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.462+0000 7f47a77fe640 1 -- 172.21.15.139:0/2771685668 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4007480 con 0x7f47c00c98f0 2024-09-16T00:11:24.465 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.462+0000 7f47a77fe640 1 -- 172.21.15.139:0/2771685668 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b40057a0 con 0x7f47c00c98f0 2024-09-16T00:11:24.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.462+0000 7f47c5711640 1 -- 172.21.15.139:0/2771685668 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c0149d00 con 0x7f47c00c98f0 2024-09-16T00:11:24.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.462+0000 7f47a77fe640 1 -- 172.21.15.139:0/2771685668 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4007620 con 0x7f47c00c98f0 2024-09-16T00:11:24.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.462+0000 7f47c5711640 1 -- 172.21.15.139:0/2771685668 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c01df9f0 con 0x7f47c00c98f0 2024-09-16T00:11:24.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.463+0000 7f47c5711640 1 -- 172.21.15.139:0/2771685668 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47c00c5650 con 0x7f47c00c98f0 2024-09-16T00:11:24.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.463+0000 7f47a77fe640 1 -- 172.21.15.139:0/2771685668 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f47b4005b50 con 0x7f47c00c98f0 2024-09-16T00:11:24.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.464+0000 7f47a77fe640 1 --2- 172.21.15.139:0/2771685668 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f47ac06c2b0 0x7f47ac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.464+0000 7f47a77fe640 1 -- 172.21.15.139:0/2771685668 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f47b408b1d0 con 0x7f47c00c98f0 2024-09-16T00:11:24.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.464+0000 7f47beffd640 1 --2- 172.21.15.139:0/2771685668 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f47ac06c2b0 0x7f47ac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.464+0000 7f47a77fe640 1 --2- 172.21.15.139:0/2771685668 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f47ac0747b0 0x7f47ac076c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.464+0000 7f47a77fe640 1 -- 172.21.15.139:0/2771685668 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47b4020e50 con 0x7f47ac0747b0 2024-09-16T00:11:24.467 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.464+0000 7f47a77fe640 1 -- 172.21.15.139:0/2771685668 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7f47b4092050 con 0x7f47c00c98f0 2024-09-16T00:11:24.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.464+0000 7f47bf7fe640 1 --2- 172.21.15.139:0/2771685668 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f47ac0747b0 0x7f47ac076c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.464+0000 7f47beffd640 1 --2- 172.21.15.139:0/2771685668 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f47ac06c2b0 0x7f47ac06e770 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f47b000a830 tx=0x7f47b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.465+0000 7f47bf7fe640 1 --2- 172.21.15.139:0/2771685668 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f47ac0747b0 0x7f47ac076c50 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.468 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.465+0000 7f47a77fe640 1 -- 172.21.15.139:0/2771685668 <== osd.6 v2:172.21.15.164:6824/164138206 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47b4020e50 con 0x7f47ac0747b0 2024-09-16T00:11:24.478 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.478+0000 7f47c5711640 1 -- 172.21.15.139:0/2771685668 --> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f47c00cb7d0 con 0x7f47ac0747b0 2024-09-16T00:11:24.479 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.478+0000 7f47a77fe640 1 -- 172.21.15.139:0/2771685668 <== osd.6 v2:172.21.15.164:6824/164138206 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f47c00cb7d0 con 0x7f47ac0747b0 2024-09-16T00:11:24.479 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:11:24.479 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T00:11:24.479 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:11:24.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.478+0000 7f47c5711640 1 -- 172.21.15.139:0/2771685668 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f47ac0747b0 msgr2=0x7f47ac076c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.478+0000 7f47c5711640 1 --2- 172.21.15.139:0/2771685668 >> [v2:172.21.15.164:6824/164138206,v1:172.21.15.164:6825/164138206] conn(0x7f47ac0747b0 0x7f47ac076c50 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.478+0000 7f47c5711640 1 -- 172.21.15.139:0/2771685668 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f47ac06c2b0 msgr2=0x7f47ac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.480 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.478+0000 7f47c5711640 1 --2- 172.21.15.139:0/2771685668 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f47ac06c2b0 0x7f47ac06e770 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f47b000a830 tx=0x7f47b0008040 comp rx=0 tx=0).stop 2024-09-16T00:11:24.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.478+0000 7f47c5711640 1 -- 172.21.15.139:0/2771685668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c00c98f0 msgr2=0x7f47c0148740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.479+0000 7f47c5711640 1 --2- 172.21.15.139:0/2771685668 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c00c98f0 0x7f47c0148740 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f47b400c500 tx=0x7f47b400c9c0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.479+0000 7f47bf7fe640 1 -- 172.21.15.139:0/2771685668 reap_dead start 2024-09-16T00:11:24.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.479+0000 7f47c5711640 1 -- 172.21.15.139:0/2771685668 shutdown_connections 2024-09-16T00:11:24.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.479+0000 7f47c5711640 1 -- 172.21.15.139:0/2771685668 >> 172.21.15.139:0/2771685668 conn(0x7f47c006bb20 msgr2=0x7f47c00ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:24.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.479+0000 7f47c5711640 1 -- 172.21.15.139:0/2771685668 shutdown_connections 2024-09-16T00:11:24.481 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.479+0000 7f47c5711640 1 -- 172.21.15.139:0/2771685668 wait complete. 2024-09-16T00:11:24.490 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-16T00:11:24.589 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.588+0000 7efd5eee4640 1 Processor -- start 2024-09-16T00:11:24.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.588+0000 7efd5eee4640 1 -- start start 2024-09-16T00:11:24.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.589+0000 7efd5eee4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efd580c71e0 0x7efd58157910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.589+0000 7efd5eee4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efd580c79c0 0x7efd58157e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.589+0000 7efd5eee4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd580c8170 con 0x7efd58151800 2024-09-16T00:11:24.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.589+0000 7efd5eee4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7efd580c7f20 con 0x7efd580c71e0 2024-09-16T00:11:24.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.589+0000 7efd5eee4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7efd580c8450 con 0x7efd580c79c0 2024-09-16T00:11:24.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.589+0000 7efd57fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efd580c71e0 0x7efd58157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.589+0000 7efd57fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efd580c71e0 0x7efd58157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43644/0 (socket says 172.21.15.139:43644) 2024-09-16T00:11:24.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.589+0000 7efd57fff640 1 -- 172.21.15.139:0/2156676882 learned_addr learned my addr 172.21.15.139:0/2156676882 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:24.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.590+0000 7efd5d481640 1 --2- 172.21.15.139:0/2156676882 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efd580c79c0 0x7efd58157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.590+0000 7efd57fff640 1 -- 172.21.15.139:0/2156676882 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efd580c79c0 msgr2=0x7efd58157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:24.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.590+0000 7efd57fff640 1 --2- 172.21.15.139:0/2156676882 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efd580c79c0 0x7efd58157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.591 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.590+0000 7efd57fff640 1 -- 172.21.15.139:0/2156676882 >> v1:172.21.15.164:6789/0 conn(0x7efd58151800 legacy=0x7efd5806eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.592 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.590+0000 7efd57fff640 1 -- 172.21.15.139:0/2156676882 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd580c8730 con 0x7efd580c71e0 2024-09-16T00:11:24.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.590+0000 7efd57fff640 1 --2- 172.21.15.139:0/2156676882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd580c71e0 0x7efd58157910 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7efd40009870 tx=0x7efd4002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee21c1841c134c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.591+0000 7efd577fe640 1 -- 172.21.15.139:0/2156676882 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40037070 con 0x7efd580c71e0 2024-09-16T00:11:24.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.591+0000 7efd577fe640 1 -- 172.21.15.139:0/2156676882 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd40032440 con 0x7efd580c71e0 2024-09-16T00:11:24.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.591+0000 7efd577fe640 1 -- 172.21.15.139:0/2156676882 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40032810 con 0x7efd580c71e0 2024-09-16T00:11:24.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.591+0000 7efd5eee4640 1 -- 172.21.15.139:0/2156676882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd580c71e0 msgr2=0x7efd58157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.591+0000 7efd5eee4640 1 --2- 172.21.15.139:0/2156676882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd580c71e0 0x7efd58157910 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7efd40009870 tx=0x7efd4002a130 comp rx=0 tx=0).stop 2024-09-16T00:11:24.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.591+0000 7efd5eee4640 1 -- 172.21.15.139:0/2156676882 shutdown_connections 2024-09-16T00:11:24.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.591+0000 7efd5eee4640 1 --2- 172.21.15.139:0/2156676882 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efd580c79c0 0x7efd58157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.593 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.591+0000 7efd5eee4640 1 --2- 172.21.15.139:0/2156676882 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd580c71e0 0x7efd58157910 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.591+0000 7efd5eee4640 1 -- 172.21.15.139:0/2156676882 >> 172.21.15.139:0/2156676882 conn(0x7efd58070460 msgr2=0x7efd58070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:24.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.591+0000 7efd5eee4640 1 -- 172.21.15.139:0/2156676882 shutdown_connections 2024-09-16T00:11:24.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.591+0000 7efd5eee4640 1 -- 172.21.15.139:0/2156676882 wait complete. 2024-09-16T00:11:24.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.592+0000 7efd5eee4640 1 Processor -- start 2024-09-16T00:11:24.594 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.592+0000 7efd5eee4640 1 -- start start 2024-09-16T00:11:24.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.593+0000 7efd5eee4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd580c71e0 0x7efd58147350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.593+0000 7efd5eee4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd580c79c0 0x7efd5806c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.593+0000 7efd5eee4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efd58151800 0x7efd5806c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.593+0000 7efd5eee4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efd580c44c0 con 0x7efd580c71e0 2024-09-16T00:11:24.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.593+0000 7efd5eee4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7efd58151be0 con 0x7efd580c79c0 2024-09-16T00:11:24.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.593+0000 7efd5eee4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7efd580c4640 con 0x7efd58151800 2024-09-16T00:11:24.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.593+0000 7efd5cc80640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd580c71e0 0x7efd58147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.593+0000 7efd5cc80640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd580c71e0 0x7efd58147350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43658/0 (socket says 172.21.15.139:43658) 2024-09-16T00:11:24.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.593+0000 7efd5d481640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efd58151800 0x7efd5806c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.596 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.593+0000 7efd5cc80640 1 -- 172.21.15.139:0/2158118861 learned_addr learned my addr 172.21.15.139:0/2158118861 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:24.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.593+0000 7efd57fff640 1 --2- 172.21.15.139:0/2158118861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd580c79c0 0x7efd5806c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.594+0000 7efd5cc80640 1 -- 172.21.15.139:0/2158118861 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efd58151800 msgr2=0x7efd5806c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.594+0000 7efd5cc80640 1 --2- 172.21.15.139:0/2158118861 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efd58151800 0x7efd5806c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.594+0000 7efd5cc80640 1 -- 172.21.15.139:0/2158118861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd580c79c0 msgr2=0x7efd5806c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.594+0000 7efd5cc80640 1 --2- 172.21.15.139:0/2158118861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd580c79c0 0x7efd5806c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.594+0000 7efd5cc80640 1 -- 172.21.15.139:0/2158118861 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5806ccb0 con 0x7efd580c71e0 2024-09-16T00:11:24.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.594+0000 7efd5cc80640 1 --2- 172.21.15.139:0/2158118861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd580c71e0 0x7efd58147350 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7efd4800c730 tx=0x7efd4800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.594+0000 7efd557fa640 1 -- 172.21.15.139:0/2158118861 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48016020 con 0x7efd580c71e0 2024-09-16T00:11:24.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.594+0000 7efd557fa640 1 -- 172.21.15.139:0/2158118861 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd48006d50 con 0x7efd580c71e0 2024-09-16T00:11:24.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.594+0000 7efd557fa640 1 -- 172.21.15.139:0/2158118861 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48003bb0 con 0x7efd580c71e0 2024-09-16T00:11:24.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.594+0000 7efd5eee4640 1 -- 172.21.15.139:0/2158118861 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5806cf50 con 0x7efd580c71e0 2024-09-16T00:11:24.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.594+0000 7efd5eee4640 1 -- 172.21.15.139:0/2158118861 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5806d9b0 con 0x7efd580c71e0 2024-09-16T00:11:24.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.595+0000 7efd5eee4640 1 -- 172.21.15.139:0/2158118861 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd5806f750 con 0x7efd580c71e0 2024-09-16T00:11:24.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.595+0000 7efd557fa640 1 -- 172.21.15.139:0/2158118861 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efd480050d0 con 0x7efd580c71e0 2024-09-16T00:11:24.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.596+0000 7efd557fa640 1 --2- 172.21.15.139:0/2158118861 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efd3806c2e0 0x7efd3806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.596+0000 7efd557fa640 1 -- 172.21.15.139:0/2158118861 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7efd4808af20 con 0x7efd580c71e0 2024-09-16T00:11:24.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.596+0000 7efd57fff640 1 --2- 172.21.15.139:0/2158118861 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efd3806c2e0 0x7efd3806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.596+0000 7efd57fff640 1 --2- 172.21.15.139:0/2158118861 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efd3806c2e0 0x7efd3806e7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efd40034000 tx=0x7efd4002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.600+0000 7efd557fa640 1 -- 172.21.15.139:0/2158118861 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efd48020020 con 0x7efd580c71e0 2024-09-16T00:11:24.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.769+0000 7efd5eee4640 1 -- 172.21.15.139:0/2158118861 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7efd581e4340 con 0x7efd3806c2e0 2024-09-16T00:11:24.770 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.769+0000 7efd557fa640 1 -- 172.21.15.139:0/2158118861 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efd581e4340 con 0x7efd3806c2e0 2024-09-16T00:11:24.770 INFO:teuthology.orchestra.run.smithi139.stderr:instructed osd(s) 6 to deep-scrub 2024-09-16T00:11:24.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.771+0000 7efd5eee4640 1 -- 172.21.15.139:0/2158118861 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efd3806c2e0 msgr2=0x7efd3806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.771+0000 7efd5eee4640 1 --2- 172.21.15.139:0/2158118861 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efd3806c2e0 0x7efd3806e7a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efd40034000 tx=0x7efd4002e000 comp rx=0 tx=0).stop 2024-09-16T00:11:24.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.771+0000 7efd5eee4640 1 -- 172.21.15.139:0/2158118861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd580c71e0 msgr2=0x7efd58147350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.771+0000 7efd5eee4640 1 --2- 172.21.15.139:0/2158118861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd580c71e0 0x7efd58147350 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7efd4800c730 tx=0x7efd4800cbf0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.771+0000 7efd5eee4640 1 -- 172.21.15.139:0/2158118861 shutdown_connections 2024-09-16T00:11:24.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.771+0000 7efd5eee4640 1 --2- 172.21.15.139:0/2158118861 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7efd3806c2e0 0x7efd3806e7a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.771+0000 7efd5eee4640 1 --2- 172.21.15.139:0/2158118861 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7efd58151800 0x7efd5806c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.771+0000 7efd5eee4640 1 --2- 172.21.15.139:0/2158118861 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7efd580c79c0 0x7efd5806c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.771+0000 7efd5eee4640 1 --2- 172.21.15.139:0/2158118861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd580c71e0 0x7efd58147350 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.771+0000 7efd5eee4640 1 -- 172.21.15.139:0/2158118861 >> 172.21.15.139:0/2158118861 conn(0x7efd58070460 msgr2=0x7efd581e09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:24.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.771+0000 7efd5eee4640 1 -- 172.21.15.139:0/2158118861 shutdown_connections 2024-09-16T00:11:24.773 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.771+0000 7efd5eee4640 1 -- 172.21.15.139:0/2158118861 wait complete. 2024-09-16T00:11:24.783 INFO:tasks.ceph:Scrubbing osd.7 2024-09-16T00:11:24.783 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-16T00:11:24.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.885+0000 7f74c3f03640 1 Processor -- start 2024-09-16T00:11:24.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.885+0000 7f74c3f03640 1 -- start start 2024-09-16T00:11:24.889 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c3f03640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f74bc151a70 0x7f74bc0c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c3f03640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74bc072480 con 0x7f74bc151690 2024-09-16T00:11:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c3f03640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74bc072a70 con 0x7f74bc0c57d0 2024-09-16T00:11:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c3f03640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f74bc0c6f80 con 0x7f74bc151a70 2024-09-16T00:11:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c1c9f640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f74bc151690 0x7f74bc0c3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45832/0 (socket says 172.21.15.139:45832) 2024-09-16T00:11:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c1c9f640 1 -- 172.21.15.139:0/2456481258 learned_addr learned my addr 172.21.15.139:0/2456481258 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c0c9d640 1 -- 172.21.15.139:0/2456481258 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2348606984 0 0) 0x7f74bc072480 con 0x7f74bc151690 2024-09-16T00:11:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c149e640 1 --2- 172.21.15.139:0/2456481258 >> v2:172.21.15.164:3300/0 conn(0x7f74bc151a70 0x7f74bc0c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c0c9d640 1 -- 172.21.15.139:0/2456481258 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74a8003520 con 0x7f74bc151690 2024-09-16T00:11:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c0c9d640 1 -- 172.21.15.139:0/2456481258 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1777138543 0 0) 0x7f74a8003520 con 0x7f74bc151690 2024-09-16T00:11:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c0c9d640 1 -- 172.21.15.139:0/2456481258 >> v2:172.21.15.164:3300/0 conn(0x7f74bc151a70 msgr2=0x7f74bc0c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c0c9d640 1 --2- 172.21.15.139:0/2456481258 >> v2:172.21.15.164:3300/0 conn(0x7f74bc151a70 0x7f74bc0c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c0c9d640 1 -- 172.21.15.139:0/2456481258 >> v1:172.21.15.164:6789/0 conn(0x7f74bc0c57d0 legacy=0x7f74bc0c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.890 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c0c9d640 1 -- 172.21.15.139:0/2456481258 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc072d50 con 0x7f74bc151690 2024-09-16T00:11:24.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c0c9d640 1 -- 172.21.15.139:0/2456481258 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7f74b80019f0 con 0x7f74bc151690 2024-09-16T00:11:24.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c0c9d640 1 -- 172.21.15.139:0/2456481258 >> v1:172.21.15.139:6789/0 conn(0x7f74bc151690 legacy=0x7f74bc0c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c0c9d640 1 --2- 172.21.15.139:0/2456481258 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74a80039b0 0x7f74a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c0c9d640 1 --2- 172.21.15.139:0/2456481258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a8004800 0x7f74a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c0c9d640 1 --2- 172.21.15.139:0/2456481258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a80051c0 0x7f74a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c0c9d640 1 -- 172.21.15.139:0/2456481258 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f74a800c040 con 0x7f74a80051c0 2024-09-16T00:11:24.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c0c9d640 1 -- 172.21.15.139:0/2456481258 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74a800c1f0 con 0x7f74a8004800 2024-09-16T00:11:24.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c0c9d640 1 -- 172.21.15.139:0/2456481258 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f74a800c3f0 con 0x7f74a80039b0 2024-09-16T00:11:24.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.886+0000 7f74c24a0640 1 --2- 172.21.15.139:0/2456481258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a80051c0 0x7f74a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.887+0000 7f74c149e640 1 --2- 172.21.15.139:0/2456481258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a8004800 0x7f74a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.887+0000 7f74c1c9f640 1 --2- 172.21.15.139:0/2456481258 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74a80039b0 0x7f74a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.887+0000 7f74c24a0640 1 -- 172.21.15.139:0/2456481258 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74a80039b0 msgr2=0x7f74a8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:24.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.887+0000 7f74c24a0640 1 --2- 172.21.15.139:0/2456481258 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74a80039b0 0x7f74a8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.891 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.887+0000 7f74c24a0640 1 -- 172.21.15.139:0/2456481258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a8004800 msgr2=0x7f74a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:24.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.887+0000 7f74c24a0640 1 --2- 172.21.15.139:0/2456481258 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74a8004800 0x7f74a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.887+0000 7f74c24a0640 1 -- 172.21.15.139:0/2456481258 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a8003520 con 0x7f74a80051c0 2024-09-16T00:11:24.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.887+0000 7f74c24a0640 1 -- 172.21.15.139:0/2456481258 reap_dead start 2024-09-16T00:11:24.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.887+0000 7f74c24a0640 1 --2- 172.21.15.139:0/2456481258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a80051c0 0x7f74a800b9c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f74b4001750 tx=0x7f74a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffa67554143f7da server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.888+0000 7f74c0c9d640 1 -- 172.21.15.139:0/2456481258 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b4001370 con 0x7f74a80051c0 2024-09-16T00:11:24.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.888+0000 7f74c0c9d640 1 -- 172.21.15.139:0/2456481258 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b4005310 con 0x7f74a80051c0 2024-09-16T00:11:24.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.888+0000 7f74c0c9d640 1 -- 172.21.15.139:0/2456481258 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b402abe0 con 0x7f74a80051c0 2024-09-16T00:11:24.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.888+0000 7f74c3f03640 1 -- 172.21.15.139:0/2456481258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a80051c0 msgr2=0x7f74a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.888+0000 7f74c3f03640 1 --2- 172.21.15.139:0/2456481258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a80051c0 0x7f74a800b9c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f74b4001750 tx=0x7f74a8005830 comp rx=0 tx=0).stop 2024-09-16T00:11:24.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.888+0000 7f74c3f03640 1 -- 172.21.15.139:0/2456481258 shutdown_connections 2024-09-16T00:11:24.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.888+0000 7f74c3f03640 1 --2- 172.21.15.139:0/2456481258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a80051c0 0x7f74a800b9c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.888+0000 7f74c3f03640 1 -- 172.21.15.139:0/2456481258 >> 172.21.15.139:0/2456481258 conn(0x7f74bc06f440 msgr2=0x7f74bc06f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:24.892 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.888+0000 7f74c3f03640 1 -- 172.21.15.139:0/2456481258 shutdown_connections 2024-09-16T00:11:24.893 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.888+0000 7f74c3f03640 1 -- 172.21.15.139:0/2456481258 wait complete. 2024-09-16T00:11:24.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.889+0000 7f74c3f03640 1 Processor -- start 2024-09-16T00:11:24.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.889+0000 7f74c3f03640 1 -- start start 2024-09-16T00:11:24.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.890+0000 7f74c3f03640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a80051c0 0x7f74bc0ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.890+0000 7f74c3f03640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74bc1eee80 0x7f74bc140db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.890+0000 7f74c3f03640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74bc1412f0 0x7f74bc147550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.890+0000 7f74c3f03640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc072120 con 0x7f74a80051c0 2024-09-16T00:11:24.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.890+0000 7f74c3f03640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc0c5340 con 0x7f74bc1412f0 2024-09-16T00:11:24.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.890+0000 7f74c3f03640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f74bc0c7c90 con 0x7f74bc1eee80 2024-09-16T00:11:24.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.890+0000 7f74c1c9f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a80051c0 0x7f74bc0ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.890+0000 7f74c1c9f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a80051c0 0x7f74bc0ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43666/0 (socket says 172.21.15.139:43666) 2024-09-16T00:11:24.896 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.890+0000 7f74c1c9f640 1 -- 172.21.15.139:0/4044825418 learned_addr learned my addr 172.21.15.139:0/4044825418 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:24.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.890+0000 7f74c24a0640 1 --2- 172.21.15.139:0/4044825418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74bc1412f0 0x7f74bc147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.890+0000 7f74c149e640 1 --2- 172.21.15.139:0/4044825418 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74bc1eee80 0x7f74bc140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.890+0000 7f74c1c9f640 1 -- 172.21.15.139:0/4044825418 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74bc1eee80 msgr2=0x7f74bc140db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.890+0000 7f74c1c9f640 1 --2- 172.21.15.139:0/4044825418 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f74bc1eee80 0x7f74bc140db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.890+0000 7f74c1c9f640 1 -- 172.21.15.139:0/4044825418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74bc1412f0 msgr2=0x7f74bc147550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.890+0000 7f74c1c9f640 1 --2- 172.21.15.139:0/4044825418 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f74bc1412f0 0x7f74bc147550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.890+0000 7f74c1c9f640 1 -- 172.21.15.139:0/4044825418 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc1435f0 con 0x7f74a80051c0 2024-09-16T00:11:24.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.891+0000 7f74c1c9f640 1 --2- 172.21.15.139:0/4044825418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a80051c0 0x7f74bc0ca000 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f74b80077f0 tx=0x7f74b800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.891+0000 7f74b27fc640 1 -- 172.21.15.139:0/4044825418 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8017020 con 0x7f74a80051c0 2024-09-16T00:11:24.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.891+0000 7f74b27fc640 1 -- 172.21.15.139:0/4044825418 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74b80133f0 con 0x7f74a80051c0 2024-09-16T00:11:24.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.891+0000 7f74c3f03640 1 -- 172.21.15.139:0/4044825418 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74bc0ca540 con 0x7f74a80051c0 2024-09-16T00:11:24.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.891+0000 7f74b27fc640 1 -- 172.21.15.139:0/4044825418 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b80137e0 con 0x7f74a80051c0 2024-09-16T00:11:24.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.891+0000 7f74c3f03640 1 -- 172.21.15.139:0/4044825418 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74bc1ef740 con 0x7f74a80051c0 2024-09-16T00:11:24.897 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.891+0000 7f74c3f03640 1 -- 172.21.15.139:0/4044825418 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f74bc0c5c20 con 0x7f74a80051c0 2024-09-16T00:11:24.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.892+0000 7f74b27fc640 1 -- 172.21.15.139:0/4044825418 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f74b801a020 con 0x7f74a80051c0 2024-09-16T00:11:24.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.892+0000 7f74b27fc640 1 --2- 172.21.15.139:0/4044825418 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f74a006c2e0 0x7f74a006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.893+0000 7f74b27fc640 1 -- 172.21.15.139:0/4044825418 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f74b808bf20 con 0x7f74a80051c0 2024-09-16T00:11:24.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.893+0000 7f74c149e640 1 --2- 172.21.15.139:0/4044825418 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f74a006c2e0 0x7f74a006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.893+0000 7f74b27fc640 1 --2- 172.21.15.139:0/4044825418 >> [v2:172.21.15.164:6808/2782017910,v1:172.21.15.164:6809/2782017910] conn(0x7f74a0074840 0x7f74a0076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:24.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.893+0000 7f74b27fc640 1 -- 172.21.15.139:0/4044825418 --> [v2:172.21.15.164:6808/2782017910,v1:172.21.15.164:6809/2782017910] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74b801aec0 con 0x7f74a0074840 2024-09-16T00:11:24.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.893+0000 7f74b27fc640 1 -- 172.21.15.139:0/4044825418 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=146) ==== 24+0+0 (secure 0 0 0) 0x7f74b808e620 con 0x7f74a80051c0 2024-09-16T00:11:24.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.893+0000 7f74c149e640 1 --2- 172.21.15.139:0/4044825418 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f74a006c2e0 0x7f74a006e7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f74bc156550 tx=0x7f74ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.893+0000 7f74c24a0640 1 --2- 172.21.15.139:0/4044825418 >> [v2:172.21.15.164:6808/2782017910,v1:172.21.15.164:6809/2782017910] conn(0x7f74a0074840 0x7f74a0076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:24.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.893+0000 7f74c24a0640 1 --2- 172.21.15.139:0/4044825418 >> [v2:172.21.15.164:6808/2782017910,v1:172.21.15.164:6809/2782017910] conn(0x7f74a0074840 0x7f74a0076ce0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:24.898 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.894+0000 7f74b27fc640 1 -- 172.21.15.139:0/4044825418 <== osd.7 v2:172.21.15.164:6808/2782017910 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f74b801aec0 con 0x7f74a0074840 2024-09-16T00:11:24.907 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.906+0000 7f74c3f03640 1 -- 172.21.15.139:0/4044825418 --> [v2:172.21.15.164:6808/2782017910,v1:172.21.15.164:6809/2782017910] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f74bc06fb60 con 0x7f74a0074840 2024-09-16T00:11:24.908 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.907+0000 7f74b27fc640 1 -- 172.21.15.139:0/4044825418 <== osd.7 v2:172.21.15.164:6808/2782017910 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f74bc06fb60 con 0x7f74a0074840 2024-09-16T00:11:24.908 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-16T00:11:24.908 INFO:teuthology.orchestra.run.smithi139.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-16T00:11:24.908 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-16T00:11:24.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.907+0000 7f74c3f03640 1 -- 172.21.15.139:0/4044825418 >> [v2:172.21.15.164:6808/2782017910,v1:172.21.15.164:6809/2782017910] conn(0x7f74a0074840 msgr2=0x7f74a0076ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.907+0000 7f74c3f03640 1 --2- 172.21.15.139:0/4044825418 >> [v2:172.21.15.164:6808/2782017910,v1:172.21.15.164:6809/2782017910] conn(0x7f74a0074840 0x7f74a0076ce0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:24.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.907+0000 7f74c3f03640 1 -- 172.21.15.139:0/4044825418 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f74a006c2e0 msgr2=0x7f74a006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.907+0000 7f74c3f03640 1 --2- 172.21.15.139:0/4044825418 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f74a006c2e0 0x7f74a006e7a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f74bc156550 tx=0x7f74ac008040 comp rx=0 tx=0).stop 2024-09-16T00:11:24.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.907+0000 7f74c3f03640 1 -- 172.21.15.139:0/4044825418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a80051c0 msgr2=0x7f74bc0ca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:24.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.907+0000 7f74c3f03640 1 --2- 172.21.15.139:0/4044825418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f74a80051c0 0x7f74bc0ca000 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f74b80077f0 tx=0x7f74b800a710 comp rx=0 tx=0).stop 2024-09-16T00:11:24.909 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.907+0000 7f74c24a0640 1 -- 172.21.15.139:0/4044825418 reap_dead start 2024-09-16T00:11:24.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.907+0000 7f74c3f03640 1 -- 172.21.15.139:0/4044825418 shutdown_connections 2024-09-16T00:11:24.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.907+0000 7f74c3f03640 1 -- 172.21.15.139:0/4044825418 >> 172.21.15.139:0/4044825418 conn(0x7f74bc06f440 msgr2=0x7f74bc0c8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:24.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.907+0000 7f74c3f03640 1 -- 172.21.15.139:0/4044825418 shutdown_connections 2024-09-16T00:11:24.910 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:24.907+0000 7f74c3f03640 1 -- 172.21.15.139:0/4044825418 wait complete. 2024-09-16T00:11:24.918 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-16T00:11:25.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.018+0000 7f0f3d83c640 1 Processor -- start 2024-09-16T00:11:25.019 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.018+0000 7f0f3d83c640 1 -- start start 2024-09-16T00:11:25.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.019+0000 7f0f3d83c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f3815d760 0x7f0f380ca390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.019+0000 7f0f3d83c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f380b50e0 con 0x7f0f380d0900 2024-09-16T00:11:25.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.019+0000 7f0f3d83c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f3815c290 con 0x7f0f380d0520 2024-09-16T00:11:25.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.019+0000 7f0f3d83c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0f380c77f0 con 0x7f0f3815d760 2024-09-16T00:11:25.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.019+0000 7f0f2e7fc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f0f380d0900 0x7f0f380c9c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:45848/0 (socket says 172.21.15.139:45848) 2024-09-16T00:11:25.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.019+0000 7f0f2e7fc640 1 -- 172.21.15.139:0/671398238 learned_addr learned my addr 172.21.15.139:0/671398238 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:25.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.019+0000 7f0f377fe640 1 --2- 172.21.15.139:0/671398238 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f3815d760 0x7f0f380ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:25.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.019+0000 7f0f367fc640 1 -- 172.21.15.139:0/671398238 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4115857023 0 0) 0x7f0f380b50e0 con 0x7f0f380d0900 2024-09-16T00:11:25.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.019+0000 7f0f367fc640 1 -- 172.21.15.139:0/671398238 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f24003520 con 0x7f0f380d0900 2024-09-16T00:11:25.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f367fc640 1 -- 172.21.15.139:0/671398238 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1167608342 0 0) 0x7f0f24003520 con 0x7f0f380d0900 2024-09-16T00:11:25.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f367fc640 1 -- 172.21.15.139:0/671398238 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f3815d760 msgr2=0x7f0f380ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:25.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f367fc640 1 --2- 172.21.15.139:0/671398238 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f3815d760 0x7f0f380ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f367fc640 1 -- 172.21.15.139:0/671398238 >> v1:172.21.15.164:6789/0 conn(0x7f0f380d0520 legacy=0x7f0f3815d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:25.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f367fc640 1 -- 172.21.15.139:0/671398238 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3815c570 con 0x7f0f380d0900 2024-09-16T00:11:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f367fc640 1 -- 172.21.15.139:0/671398238 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7f0f20001670 con 0x7f0f380d0900 2024-09-16T00:11:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f367fc640 1 -- 172.21.15.139:0/671398238 >> v1:172.21.15.139:6789/0 conn(0x7f0f380d0900 legacy=0x7f0f380c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f367fc640 1 --2- 172.21.15.139:0/671398238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240039e0 0x7f0f24003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f367fc640 1 --2- 172.21.15.139:0/671398238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f24004390 0x7f0f24004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f367fc640 1 --2- 172.21.15.139:0/671398238 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f3815d760 0x7f0f240050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f367fc640 1 -- 172.21.15.139:0/671398238 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0f380c77f0 con 0x7f0f240039e0 2024-09-16T00:11:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f367fc640 1 -- 172.21.15.139:0/671398238 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2400b9c0 con 0x7f0f24004390 2024-09-16T00:11:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f367fc640 1 -- 172.21.15.139:0/671398238 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2400bbc0 con 0x7f0f3815d760 2024-09-16T00:11:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f36ffd640 1 --2- 172.21.15.139:0/671398238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240039e0 0x7f0f24003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f2e7fc640 1 --2- 172.21.15.139:0/671398238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f24004390 0x7f0f24004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f377fe640 1 --2- 172.21.15.139:0/671398238 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f3815d760 0x7f0f240050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f36ffd640 1 -- 172.21.15.139:0/671398238 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f3815d760 msgr2=0x7f0f240050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f36ffd640 1 --2- 172.21.15.139:0/671398238 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f3815d760 0x7f0f240050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.024 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f36ffd640 1 -- 172.21.15.139:0/671398238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f24004390 msgr2=0x7f0f24004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f36ffd640 1 --2- 172.21.15.139:0/671398238 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f24004390 0x7f0f24004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.020+0000 7f0f36ffd640 1 -- 172.21.15.139:0/671398238 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f24003520 con 0x7f0f240039e0 2024-09-16T00:11:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.021+0000 7f0f36ffd640 1 --2- 172.21.15.139:0/671398238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240039e0 0x7f0f24003dc0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0f28001860 tx=0x7f0f24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0cc0f0ae6edaf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.021+0000 7f0f367fc640 1 -- 172.21.15.139:0/671398238 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28037070 con 0x7f0f240039e0 2024-09-16T00:11:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.021+0000 7f0f367fc640 1 -- 172.21.15.139:0/671398238 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f2802ac30 con 0x7f0f240039e0 2024-09-16T00:11:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.021+0000 7f0f367fc640 1 -- 172.21.15.139:0/671398238 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28001430 con 0x7f0f240039e0 2024-09-16T00:11:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.021+0000 7f0f3d83c640 1 -- 172.21.15.139:0/671398238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240039e0 msgr2=0x7f0f24003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.021+0000 7f0f3d83c640 1 --2- 172.21.15.139:0/671398238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240039e0 0x7f0f24003dc0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0f28001860 tx=0x7f0f24005830 comp rx=0 tx=0).stop 2024-09-16T00:11:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.021+0000 7f0f377fe640 1 -- 172.21.15.139:0/671398238 reap_dead start 2024-09-16T00:11:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.022+0000 7f0f3d83c640 1 -- 172.21.15.139:0/671398238 shutdown_connections 2024-09-16T00:11:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.022+0000 7f0f3d83c640 1 -- 172.21.15.139:0/671398238 >> 172.21.15.139:0/671398238 conn(0x7f0f3806e850 msgr2=0x7f0f3806c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.022+0000 7f0f3d83c640 1 -- 172.21.15.139:0/671398238 shutdown_connections 2024-09-16T00:11:25.025 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.022+0000 7f0f3d83c640 1 -- 172.21.15.139:0/671398238 wait complete. 2024-09-16T00:11:25.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.023+0000 7f0f3d83c640 1 Processor -- start 2024-09-16T00:11:25.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.023+0000 7f0f3d83c640 1 -- start start 2024-09-16T00:11:25.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.023+0000 7f0f3d83c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240039e0 0x7f0f3815aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.023+0000 7f0f3d83c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f381570f0 0x7f0f381f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.023+0000 7f0f3d83c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f381561a0 0x7f0f380c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.028 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.023+0000 7f0f3d83c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3815cbc0 con 0x7f0f240039e0 2024-09-16T00:11:25.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.023+0000 7f0f3d83c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3815c040 con 0x7f0f381561a0 2024-09-16T00:11:25.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.023+0000 7f0f3d83c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f0f380b35c0 con 0x7f0f381570f0 2024-09-16T00:11:25.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.024+0000 7f0f36ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240039e0 0x7f0f3815aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:25.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.024+0000 7f0f36ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240039e0 0x7f0f3815aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43682/0 (socket says 172.21.15.139:43682) 2024-09-16T00:11:25.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.024+0000 7f0f36ffd640 1 -- 172.21.15.139:0/1312300991 learned_addr learned my addr 172.21.15.139:0/1312300991 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:25.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.024+0000 7f0f377fe640 1 --2- 172.21.15.139:0/1312300991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f381561a0 0x7f0f380c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:25.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.024+0000 7f0f2e7fc640 1 --2- 172.21.15.139:0/1312300991 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f381570f0 0x7f0f381f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:25.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.024+0000 7f0f36ffd640 1 -- 172.21.15.139:0/1312300991 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f381570f0 msgr2=0x7f0f381f30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:25.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.024+0000 7f0f36ffd640 1 --2- 172.21.15.139:0/1312300991 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f381570f0 0x7f0f381f30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.024+0000 7f0f36ffd640 1 -- 172.21.15.139:0/1312300991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f381561a0 msgr2=0x7f0f380c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:25.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.024+0000 7f0f36ffd640 1 --2- 172.21.15.139:0/1312300991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f381561a0 0x7f0f380c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.024+0000 7f0f36ffd640 1 -- 172.21.15.139:0/1312300991 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f38157dd0 con 0x7f0f240039e0 2024-09-16T00:11:25.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.024+0000 7f0f36ffd640 1 --2- 172.21.15.139:0/1312300991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240039e0 0x7f0f3815aa90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0f2802a6c0 tx=0x7f0f28001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:25.029 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.024+0000 7f0f34ff9640 1 -- 172.21.15.139:0/1312300991 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28037040 con 0x7f0f240039e0 2024-09-16T00:11:25.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.024+0000 7f0f34ff9640 1 -- 172.21.15.139:0/1312300991 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f28041070 con 0x7f0f240039e0 2024-09-16T00:11:25.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.024+0000 7f0f3d83c640 1 -- 172.21.15.139:0/1312300991 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f381556c0 con 0x7f0f240039e0 2024-09-16T00:11:25.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.024+0000 7f0f34ff9640 1 -- 172.21.15.139:0/1312300991 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28001430 con 0x7f0f240039e0 2024-09-16T00:11:25.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.024+0000 7f0f3d83c640 1 -- 172.21.15.139:0/1312300991 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f380c7e20 con 0x7f0f240039e0 2024-09-16T00:11:25.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.025+0000 7f0f3d83c640 1 -- 172.21.15.139:0/1312300991 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f3815d050 con 0x7f0f240039e0 2024-09-16T00:11:25.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.026+0000 7f0f34ff9640 1 -- 172.21.15.139:0/1312300991 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0f28005310 con 0x7f0f240039e0 2024-09-16T00:11:25.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.026+0000 7f0f34ff9640 1 --2- 172.21.15.139:0/1312300991 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0f1806c2e0 0x7f0f1806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.026+0000 7f0f34ff9640 1 -- 172.21.15.139:0/1312300991 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f0f280ab410 con 0x7f0f240039e0 2024-09-16T00:11:25.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.026+0000 7f0f2e7fc640 1 --2- 172.21.15.139:0/1312300991 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0f1806c2e0 0x7f0f1806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:25.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.027+0000 7f0f2e7fc640 1 --2- 172.21.15.139:0/1312300991 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0f1806c2e0 0x7f0f1806e7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0f380cc510 tx=0x7f0f20000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:25.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.031+0000 7f0f34ff9640 1 -- 172.21.15.139:0/1312300991 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0f280793b0 con 0x7f0f240039e0 2024-09-16T00:11:25.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.197+0000 7f0f3d83c640 1 -- 172.21.15.139:0/1312300991 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f0f3815edb0 con 0x7f0f1806c2e0 2024-09-16T00:11:25.198 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.197+0000 7f0f34ff9640 1 -- 172.21.15.139:0/1312300991 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0f3815edb0 con 0x7f0f1806c2e0 2024-09-16T00:11:25.198 INFO:teuthology.orchestra.run.smithi139.stderr:instructed osd(s) 7 to deep-scrub 2024-09-16T00:11:25.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.199+0000 7f0f3d83c640 1 -- 172.21.15.139:0/1312300991 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0f1806c2e0 msgr2=0x7f0f1806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:25.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.199+0000 7f0f3d83c640 1 --2- 172.21.15.139:0/1312300991 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0f1806c2e0 0x7f0f1806e7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0f380cc510 tx=0x7f0f20000ed0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.199+0000 7f0f3d83c640 1 -- 172.21.15.139:0/1312300991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240039e0 msgr2=0x7f0f3815aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:25.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.199+0000 7f0f3d83c640 1 --2- 172.21.15.139:0/1312300991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240039e0 0x7f0f3815aa90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0f2802a6c0 tx=0x7f0f28001230 comp rx=0 tx=0).stop 2024-09-16T00:11:25.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.199+0000 7f0f3d83c640 1 -- 172.21.15.139:0/1312300991 shutdown_connections 2024-09-16T00:11:25.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.199+0000 7f0f3d83c640 1 --2- 172.21.15.139:0/1312300991 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f0f1806c2e0 0x7f0f1806e7a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.199+0000 7f0f3d83c640 1 --2- 172.21.15.139:0/1312300991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0f381561a0 0x7f0f380c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.199+0000 7f0f3d83c640 1 --2- 172.21.15.139:0/1312300991 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f0f381570f0 0x7f0f381f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.199+0000 7f0f3d83c640 1 --2- 172.21.15.139:0/1312300991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f240039e0 0x7f0f3815aa90 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.201 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.199+0000 7f0f3d83c640 1 -- 172.21.15.139:0/1312300991 >> 172.21.15.139:0/1312300991 conn(0x7f0f3806e850 msgr2=0x7f0f380cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:25.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.200+0000 7f0f3d83c640 1 -- 172.21.15.139:0/1312300991 shutdown_connections 2024-09-16T00:11:25.202 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.200+0000 7f0f3d83c640 1 -- 172.21.15.139:0/1312300991 wait complete. 2024-09-16T00:11:25.211 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:11:25.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.311+0000 7f140eec9640 1 Processor -- start 2024-09-16T00:11:25.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.311+0000 7f140eec9640 1 -- start start 2024-09-16T00:11:25.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.312+0000 7f140eec9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f14080c3c60 0x7f14080cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.312+0000 7f140eec9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14080c55f0 0x7f140815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.312+0000 7f140eec9640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14080c5a00 con 0x7f14081519d0 2024-09-16T00:11:25.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.312+0000 7f140eec9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f140815b920 con 0x7f14080c3c60 2024-09-16T00:11:25.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.312+0000 7f140eec9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f140815b640 con 0x7f14080c55f0 2024-09-16T00:11:25.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.312+0000 7f1407fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f14080c3c60 0x7f14080cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:25.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.312+0000 7f1407fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f14080c3c60 0x7f14080cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43694/0 (socket says 172.21.15.139:43694) 2024-09-16T00:11:25.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.312+0000 7f1407fff640 1 -- 172.21.15.139:0/2116135457 learned_addr learned my addr 172.21.15.139:0/2116135457 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:25.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.312+0000 7f1407fff640 1 -- 172.21.15.139:0/2116135457 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14080c55f0 msgr2=0x7f140815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:25.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.312+0000 7f1407fff640 1 --2- 172.21.15.139:0/2116135457 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14080c55f0 0x7f140815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.312+0000 7f1407fff640 1 -- 172.21.15.139:0/2116135457 >> v1:172.21.15.164:6789/0 conn(0x7f14081519d0 legacy=0x7f140806cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:25.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.312+0000 7f1407fff640 1 -- 172.21.15.139:0/2116135457 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14080ca600 con 0x7f14080c3c60 2024-09-16T00:11:25.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.313+0000 7f1407fff640 1 --2- 172.21.15.139:0/2116135457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14080c3c60 0x7f14080cb3c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009870 tx=0x7f13f402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d037eaa6cd49eab server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:25.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.313+0000 7f14077fe640 1 -- 172.21.15.139:0/2116135457 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4037070 con 0x7f14080c3c60 2024-09-16T00:11:25.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.313+0000 7f14077fe640 1 -- 172.21.15.139:0/2116135457 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f4032440 con 0x7f14080c3c60 2024-09-16T00:11:25.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.313+0000 7f14077fe640 1 -- 172.21.15.139:0/2116135457 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4032810 con 0x7f14080c3c60 2024-09-16T00:11:25.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.313+0000 7f140eec9640 1 -- 172.21.15.139:0/2116135457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14080c3c60 msgr2=0x7f14080cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:25.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.313+0000 7f140eec9640 1 --2- 172.21.15.139:0/2116135457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14080c3c60 0x7f14080cb3c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009870 tx=0x7f13f402a180 comp rx=0 tx=0).stop 2024-09-16T00:11:25.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.313+0000 7f140eec9640 1 -- 172.21.15.139:0/2116135457 shutdown_connections 2024-09-16T00:11:25.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.313+0000 7f140eec9640 1 --2- 172.21.15.139:0/2116135457 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14080c55f0 0x7f140815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.313+0000 7f140eec9640 1 --2- 172.21.15.139:0/2116135457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14080c3c60 0x7f14080cb3c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.313+0000 7f140eec9640 1 -- 172.21.15.139:0/2116135457 >> 172.21.15.139:0/2116135457 conn(0x7f14080c91d0 msgr2=0x7f14080c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:25.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.313+0000 7f140eec9640 1 -- 172.21.15.139:0/2116135457 shutdown_connections 2024-09-16T00:11:25.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.313+0000 7f140eec9640 1 -- 172.21.15.139:0/2116135457 wait complete. 2024-09-16T00:11:25.316 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.314+0000 7f140eec9640 1 Processor -- start 2024-09-16T00:11:25.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.315+0000 7f140eec9640 1 -- start start 2024-09-16T00:11:25.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.315+0000 7f140eec9640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14080c3c60 0x7f1408073070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.315+0000 7f140eec9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14080c55f0 0x7f140814ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.315+0000 7f140eec9640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14081519d0 0x7f1408143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.315+0000 7f140eec9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f14080caea0 con 0x7f14080c55f0 2024-09-16T00:11:25.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.315+0000 7f140eec9640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f14080cabc0 con 0x7f14080c3c60 2024-09-16T00:11:25.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.315+0000 7f140eec9640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f140806d340 con 0x7f14081519d0 2024-09-16T00:11:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.315+0000 7f1407fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14080c55f0 0x7f140814ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.316+0000 7f1407fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14080c55f0 0x7f140814ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:43702/0 (socket says 172.21.15.139:43702) 2024-09-16T00:11:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.316+0000 7f1407fff640 1 -- 172.21.15.139:0/512745350 learned_addr learned my addr 172.21.15.139:0/512745350 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.316+0000 7f140cc65640 1 --2- 172.21.15.139:0/512745350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14080c3c60 0x7f1408073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.316+0000 7f140d466640 1 --2- 172.21.15.139:0/512745350 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14081519d0 0x7f1408143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.316+0000 7f1407fff640 1 -- 172.21.15.139:0/512745350 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14081519d0 msgr2=0x7f1408143ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.316+0000 7f1407fff640 1 --2- 172.21.15.139:0/512745350 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14081519d0 0x7f1408143ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.316+0000 7f1407fff640 1 -- 172.21.15.139:0/512745350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14080c3c60 msgr2=0x7f1408073070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.316+0000 7f1407fff640 1 --2- 172.21.15.139:0/512745350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14080c3c60 0x7f1408073070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.316+0000 7f1407fff640 1 -- 172.21.15.139:0/512745350 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14081f4090 con 0x7f14080c55f0 2024-09-16T00:11:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.316+0000 7f1407fff640 1 --2- 172.21.15.139:0/512745350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14080c55f0 0x7f140814ce90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009870 tx=0x7f13f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.316+0000 7f14057fa640 1 -- 172.21.15.139:0/512745350 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4041070 con 0x7f14080c55f0 2024-09-16T00:11:25.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.316+0000 7f14057fa640 1 -- 172.21.15.139:0/512745350 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f403c870 con 0x7f14080c55f0 2024-09-16T00:11:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.316+0000 7f140eec9640 1 -- 172.21.15.139:0/512745350 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14081f1680 con 0x7f14080c55f0 2024-09-16T00:11:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.316+0000 7f14057fa640 1 -- 172.21.15.139:0/512745350 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4037040 con 0x7f14080c55f0 2024-09-16T00:11:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.316+0000 7f140eec9640 1 -- 172.21.15.139:0/512745350 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14081f4610 con 0x7f14080c55f0 2024-09-16T00:11:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.317+0000 7f140eec9640 1 -- 172.21.15.139:0/512745350 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14080cb3c0 con 0x7f14080c55f0 2024-09-16T00:11:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.318+0000 7f14057fa640 1 -- 172.21.15.139:0/512745350 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f13f4044070 con 0x7f14080c55f0 2024-09-16T00:11:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.318+0000 7f14057fa640 1 --2- 172.21.15.139:0/512745350 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f13ec06c2e0 0x7f13ec06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.318+0000 7f14057fa640 1 -- 172.21.15.139:0/512745350 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f13f40ac100 con 0x7f14080c55f0 2024-09-16T00:11:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.318+0000 7f140cc65640 1 --2- 172.21.15.139:0/512745350 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f13ec06c2e0 0x7f13ec06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:25.321 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.319+0000 7f140cc65640 1 --2- 172.21.15.139:0/512745350 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f13ec06c2e0 0x7f13ec06e7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f13fc006fd0 tx=0x7f13fc009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:25.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.322+0000 7f14057fa640 1 -- 172.21.15.139:0/512745350 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f14080c5a00 con 0x7f14080c55f0 2024-09-16T00:11:25.492 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.492+0000 7f140eec9640 1 -- 172.21.15.139:0/512745350 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f14081501c0 con 0x7f13ec06c2e0 2024-09-16T00:11:25.498 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.497+0000 7f14057fa640 1 -- 172.21.15.139:0/512745350 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+254702 (secure 0 0 0) 0x7f14081501c0 con 0x7f13ec06c2e0 2024-09-16T00:11:25.501 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:11:25.502 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:11:25.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.501+0000 7f140eec9640 1 -- 172.21.15.139:0/512745350 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f13ec06c2e0 msgr2=0x7f13ec06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:25.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.501+0000 7f140eec9640 1 --2- 172.21.15.139:0/512745350 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f13ec06c2e0 0x7f13ec06e7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f13fc006fd0 tx=0x7f13fc009060 comp rx=0 tx=0).stop 2024-09-16T00:11:25.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.501+0000 7f140eec9640 1 -- 172.21.15.139:0/512745350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14080c55f0 msgr2=0x7f140814ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:25.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.501+0000 7f140eec9640 1 --2- 172.21.15.139:0/512745350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14080c55f0 0x7f140814ce90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009870 tx=0x7f13f40046c0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.501+0000 7f140eec9640 1 -- 172.21.15.139:0/512745350 shutdown_connections 2024-09-16T00:11:25.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.501+0000 7f140eec9640 1 --2- 172.21.15.139:0/512745350 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f13ec06c2e0 0x7f13ec06e7a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.501+0000 7f140eec9640 1 --2- 172.21.15.139:0/512745350 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f14081519d0 0x7f1408143ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.501+0000 7f140eec9640 1 --2- 172.21.15.139:0/512745350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14080c55f0 0x7f140814ce90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.501+0000 7f140eec9640 1 --2- 172.21.15.139:0/512745350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f14080c3c60 0x7f1408073070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:25.504 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.501+0000 7f140eec9640 1 -- 172.21.15.139:0/512745350 >> 172.21.15.139:0/512745350 conn(0x7f14080c91d0 msgr2=0x7f14081e0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:25.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.501+0000 7f140eec9640 1 -- 172.21.15.139:0/512745350 shutdown_connections 2024-09-16T00:11:25.505 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:25.501+0000 7f140eec9640 1 -- 172.21.15.139:0/512745350 wait complete. 2024-09-16T00:11:25.513 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-09-16T00:11:24.068086+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":129639070,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":2853,"num_read_kb":1363646,"num_write":30197,"num_write_kb":4222085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3232,"num_bytes_recovered":321049829,"num_keys_recovered":3117,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33681,"ondisk_log_size":33681,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":462148,"kb_used_data":98372,"kb_used_omap":273,"kb_used_meta":363374,"kb_avail":754512572,"statfs":{"total":773094113280,"available":772620873728,"internally_reserved":0,"allocated":100732928,"data_stored":266049988,"data_compressed":38804642,"data_compressed_allocated":49901568,"data_compressed_original":222470144,"omap_allocated":280397,"internal_metadata":372095155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.116777"},"pg_stats":[{"pgid":"3.3f","version":"146'442","reported_seq":877,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.027770+0000","last_change":"2024-09-16T00:10:56.416283+0000","last_active":"2024-09-16T00:11:19.027770+0000","last_peered":"2024-09-16T00:11:19.027770+0000","last_clean":"2024-09-16T00:11:19.027770+0000","last_became_active":"2024-09-16T00:10:55.845182+0000","last_became_peered":"2024-09-16T00:10:55.845182+0000","last_unstale":"2024-09-16T00:11:19.027770+0000","last_undegraded":"2024-09-16T00:11:19.027770+0000","last_fullsized":"2024-09-16T00:11:19.027770+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:46:52.137388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":42086,"num_write":409,"num_write_kb":112716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":31107821,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"146'421","reported_seq":1097,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.017078+0000","last_change":"2024-09-16T00:11:04.897161+0000","last_active":"2024-09-16T00:11:19.017078+0000","last_peered":"2024-09-16T00:11:19.017078+0000","last_clean":"2024-09-16T00:11:19.017078+0000","last_became_active":"2024-09-16T00:11:04.894999+0000","last_became_peered":"2024-09-16T00:11:04.894999+0000","last_unstale":"2024-09-16T00:11:19.017078+0000","last_undegraded":"2024-09-16T00:11:19.017078+0000","last_fullsized":"2024-09-16T00:11:19.017078+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:13.015652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":23064,"num_write":397,"num_write_kb":47220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":689591,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"146'374","reported_seq":664,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.014217+0000","last_change":"2024-09-16T00:07:33.624946+0000","last_active":"2024-09-16T00:11:19.014217+0000","last_peered":"2024-09-16T00:11:19.014217+0000","last_clean":"2024-09-16T00:11:19.014217+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:11:19.014217+0000","last_undegraded":"2024-09-16T00:11:19.014217+0000","last_fullsized":"2024-09-16T00:11:19.014217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15782,"num_write":349,"num_write_kb":54932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"146'397","reported_seq":687,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.020049+0000","last_change":"2024-09-16T00:07:33.632196+0000","last_active":"2024-09-16T00:11:19.020049+0000","last_peered":"2024-09-16T00:11:19.020049+0000","last_clean":"2024-09-16T00:11:19.020049+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:11:19.020049+0000","last_undegraded":"2024-09-16T00:11:19.020049+0000","last_fullsized":"2024-09-16T00:11:19.020049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:18.573765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":14080,"num_write":364,"num_write_kb":63191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"146'449","reported_seq":752,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.009242+0000","last_change":"2024-09-16T00:08:15.791635+0000","last_active":"2024-09-16T00:11:19.009242+0000","last_peered":"2024-09-16T00:11:19.009242+0000","last_clean":"2024-09-16T00:11:19.009242+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:11:19.009242+0000","last_undegraded":"2024-09-16T00:11:19.009242+0000","last_fullsized":"2024-09-16T00:11:19.009242+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:24.147341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":36096,"num_write":427,"num_write_kb":89616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"146'390","reported_seq":1358,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.012367+0000","last_change":"2024-09-16T00:10:45.795247+0000","last_active":"2024-09-16T00:11:19.012367+0000","last_peered":"2024-09-16T00:11:19.012367+0000","last_clean":"2024-09-16T00:11:19.012367+0000","last_became_active":"2024-09-16T00:10:45.793434+0000","last_became_peered":"2024-09-16T00:10:45.793434+0000","last_unstale":"2024-09-16T00:11:19.012367+0000","last_undegraded":"2024-09-16T00:11:19.012367+0000","last_fullsized":"2024-09-16T00:11:19.012367+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:55.019829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":6174,"num_write":362,"num_write_kb":50827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":425,"num_bytes_recovered":23619955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"146'425","reported_seq":786,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.017326+0000","last_change":"2024-09-16T00:10:56.030536+0000","last_active":"2024-09-16T00:11:19.017326+0000","last_peered":"2024-09-16T00:11:19.017326+0000","last_clean":"2024-09-16T00:11:19.017326+0000","last_became_active":"2024-09-16T00:10:55.835516+0000","last_became_peered":"2024-09-16T00:10:55.835516+0000","last_unstale":"2024-09-16T00:11:19.017326+0000","last_undegraded":"2024-09-16T00:11:19.017326+0000","last_fullsized":"2024-09-16T00:11:19.017326+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:43.152015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":16577,"num_write":401,"num_write_kb":64470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"146'383","reported_seq":650,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.028552+0000","last_change":"2024-09-16T00:08:15.815020+0000","last_active":"2024-09-16T00:11:19.028552+0000","last_peered":"2024-09-16T00:11:19.028552+0000","last_clean":"2024-09-16T00:11:19.028552+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:11:19.028552+0000","last_undegraded":"2024-09-16T00:11:19.028552+0000","last_fullsized":"2024-09-16T00:11:19.028552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:29.096199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":16776,"num_write":347,"num_write_kb":80164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"146'405","reported_seq":737,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.033919+0000","last_change":"2024-09-16T00:10:56.261956+0000","last_active":"2024-09-16T00:11:19.033919+0000","last_peered":"2024-09-16T00:11:19.033919+0000","last_clean":"2024-09-16T00:11:19.033919+0000","last_became_active":"2024-09-16T00:10:55.840604+0000","last_became_peered":"2024-09-16T00:10:55.840604+0000","last_unstale":"2024-09-16T00:11:19.033919+0000","last_undegraded":"2024-09-16T00:11:19.033919+0000","last_fullsized":"2024-09-16T00:11:19.033919+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:39:39.279660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1432,"num_write":368,"num_write_kb":30311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"146'383","reported_seq":552,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.029601+0000","last_change":"2024-09-16T00:10:00.734336+0000","last_active":"2024-09-16T00:11:19.029601+0000","last_peered":"2024-09-16T00:11:19.029601+0000","last_clean":"2024-09-16T00:11:19.029601+0000","last_became_active":"2024-09-16T00:10:00.284452+0000","last_became_peered":"2024-09-16T00:10:00.284452+0000","last_unstale":"2024-09-16T00:11:19.029601+0000","last_undegraded":"2024-09-16T00:11:19.029601+0000","last_fullsized":"2024-09-16T00:11:19.029601+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:56:50.530439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4096,"num_write":335,"num_write_kb":29952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":8736255,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"146'361","reported_seq":541,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.021063+0000","last_change":"2024-09-16T00:10:00.587110+0000","last_active":"2024-09-16T00:11:19.021063+0000","last_peered":"2024-09-16T00:11:19.021063+0000","last_clean":"2024-09-16T00:11:19.021063+0000","last_became_active":"2024-09-16T00:10:00.280273+0000","last_became_peered":"2024-09-16T00:10:00.280273+0000","last_unstale":"2024-09-16T00:11:19.021063+0000","last_undegraded":"2024-09-16T00:11:19.021063+0000","last_fullsized":"2024-09-16T00:11:19.021063+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:01:57.902939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":975,"num_write":339,"num_write_kb":25518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":10753732,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"146'373","reported_seq":712,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.029789+0000","last_change":"2024-09-16T00:10:00.797372+0000","last_active":"2024-09-16T00:11:19.029789+0000","last_peered":"2024-09-16T00:11:19.029789+0000","last_clean":"2024-09-16T00:11:19.029789+0000","last_became_active":"2024-09-16T00:10:00.278765+0000","last_became_peered":"2024-09-16T00:10:00.278765+0000","last_unstale":"2024-09-16T00:11:19.029789+0000","last_undegraded":"2024-09-16T00:11:19.029789+0000","last_fullsized":"2024-09-16T00:11:19.029789+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:22:03.318919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":11563,"num_write":348,"num_write_kb":58933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":29565877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"146'429","reported_seq":794,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.010574+0000","last_change":"2024-09-16T00:10:00.715647+0000","last_active":"2024-09-16T00:11:19.010574+0000","last_peered":"2024-09-16T00:11:19.010574+0000","last_clean":"2024-09-16T00:11:19.010574+0000","last_became_active":"2024-09-16T00:10:00.286050+0000","last_became_peered":"2024-09-16T00:10:00.286050+0000","last_unstale":"2024-09-16T00:11:19.010574+0000","last_undegraded":"2024-09-16T00:11:19.010574+0000","last_fullsized":"2024-09-16T00:11:19.010574+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:07.462899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":35908,"num_write":393,"num_write_kb":104863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":30230864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"146'422","reported_seq":731,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.017405+0000","last_change":"2024-09-16T00:07:33.627315+0000","last_active":"2024-09-16T00:11:19.017405+0000","last_peered":"2024-09-16T00:11:19.017405+0000","last_clean":"2024-09-16T00:11:19.017405+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:11:19.017405+0000","last_undegraded":"2024-09-16T00:11:19.017405+0000","last_fullsized":"2024-09-16T00:11:19.017405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:29.000892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":23364,"num_write":390,"num_write_kb":43321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"146'399","reported_seq":691,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.012904+0000","last_change":"2024-09-16T00:07:33.620282+0000","last_active":"2024-09-16T00:11:19.012904+0000","last_peered":"2024-09-16T00:11:19.012904+0000","last_clean":"2024-09-16T00:11:19.012904+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:11:19.012904+0000","last_undegraded":"2024-09-16T00:11:19.012904+0000","last_fullsized":"2024-09-16T00:11:19.012904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":14230,"num_write":362,"num_write_kb":33768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"146'448","reported_seq":765,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.028937+0000","last_change":"2024-09-16T00:07:33.627908+0000","last_active":"2024-09-16T00:11:19.028937+0000","last_peered":"2024-09-16T00:11:19.028937+0000","last_clean":"2024-09-16T00:11:19.028937+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:11:19.028937+0000","last_undegraded":"2024-09-16T00:11:19.028937+0000","last_fullsized":"2024-09-16T00:11:19.028937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":40234,"num_write":414,"num_write_kb":67785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"146'402","reported_seq":898,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.033851+0000","last_change":"2024-09-16T00:10:56.577752+0000","last_active":"2024-09-16T00:11:19.033851+0000","last_peered":"2024-09-16T00:11:19.033851+0000","last_clean":"2024-09-16T00:11:19.033851+0000","last_became_active":"2024-09-16T00:10:55.845880+0000","last_became_peered":"2024-09-16T00:10:55.845880+0000","last_unstale":"2024-09-16T00:11:19.033851+0000","last_undegraded":"2024-09-16T00:11:19.033851+0000","last_fullsized":"2024-09-16T00:11:19.033851+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:32.037945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":22524,"num_write":374,"num_write_kb":46842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"146'401","reported_seq":730,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.008979+0000","last_change":"2024-09-16T00:10:00.461437+0000","last_active":"2024-09-16T00:11:19.008979+0000","last_peered":"2024-09-16T00:11:19.008979+0000","last_clean":"2024-09-16T00:11:19.008979+0000","last_became_active":"2024-09-16T00:10:00.282592+0000","last_became_peered":"2024-09-16T00:10:00.282592+0000","last_unstale":"2024-09-16T00:11:19.008979+0000","last_undegraded":"2024-09-16T00:11:19.008979+0000","last_fullsized":"2024-09-16T00:11:19.008979+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:53:09.733353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8711,"num_write":366,"num_write_kb":46855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":21285626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"146'424","reported_seq":834,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.029674+0000","last_change":"2024-09-16T00:10:56.142127+0000","last_active":"2024-09-16T00:11:19.029674+0000","last_peered":"2024-09-16T00:11:19.029674+0000","last_clean":"2024-09-16T00:11:19.029674+0000","last_became_active":"2024-09-16T00:10:55.832595+0000","last_became_peered":"2024-09-16T00:10:55.832595+0000","last_unstale":"2024-09-16T00:11:19.029674+0000","last_undegraded":"2024-09-16T00:11:19.029674+0000","last_fullsized":"2024-09-16T00:11:19.029674+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:44:56.072997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":14437,"num_write":396,"num_write_kb":83979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":103497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"146'431","reported_seq":610,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.021767+0000","last_change":"2024-09-16T00:10:00.566870+0000","last_active":"2024-09-16T00:11:19.021767+0000","last_peered":"2024-09-16T00:11:19.021767+0000","last_clean":"2024-09-16T00:11:19.021767+0000","last_became_active":"2024-09-16T00:10:00.281258+0000","last_became_peered":"2024-09-16T00:10:00.281258+0000","last_unstale":"2024-09-16T00:11:19.021767+0000","last_undegraded":"2024-09-16T00:11:19.021767+0000","last_fullsized":"2024-09-16T00:11:19.021767+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:29.197743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":15502,"num_write":406,"num_write_kb":30222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":17199573,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"146'415","reported_seq":894,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.024866+0000","last_change":"2024-09-16T00:10:56.104046+0000","last_active":"2024-09-16T00:11:19.024866+0000","last_peered":"2024-09-16T00:11:19.024866+0000","last_clean":"2024-09-16T00:11:19.024866+0000","last_became_active":"2024-09-16T00:10:55.835150+0000","last_became_peered":"2024-09-16T00:10:55.835150+0000","last_unstale":"2024-09-16T00:11:19.024866+0000","last_undegraded":"2024-09-16T00:11:19.024866+0000","last_fullsized":"2024-09-16T00:11:19.024866+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:08.552926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":34318,"num_write":385,"num_write_kb":91769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"146'454","reported_seq":928,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.031487+0000","last_change":"2024-09-16T00:10:55.939360+0000","last_active":"2024-09-16T00:11:19.031487+0000","last_peered":"2024-09-16T00:11:19.031487+0000","last_clean":"2024-09-16T00:11:19.031487+0000","last_became_active":"2024-09-16T00:10:55.833335+0000","last_became_peered":"2024-09-16T00:10:55.833335+0000","last_unstale":"2024-09-16T00:11:19.031487+0000","last_undegraded":"2024-09-16T00:11:19.031487+0000","last_fullsized":"2024-09-16T00:11:19.031487+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:08:29.270004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":35528,"num_write":422,"num_write_kb":92852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"146'403","reported_seq":691,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.027184+0000","last_change":"2024-09-16T00:07:33.632423+0000","last_active":"2024-09-16T00:11:19.027184+0000","last_peered":"2024-09-16T00:11:19.027184+0000","last_clean":"2024-09-16T00:11:19.027184+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:11:19.027184+0000","last_undegraded":"2024-09-16T00:11:19.027184+0000","last_fullsized":"2024-09-16T00:11:19.027184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:34.020897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":13410,"num_write":369,"num_write_kb":58805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"146'427","reported_seq":709,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.020420+0000","last_change":"2024-09-16T00:07:33.627233+0000","last_active":"2024-09-16T00:11:19.020420+0000","last_peered":"2024-09-16T00:11:19.020420+0000","last_clean":"2024-09-16T00:11:19.020420+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:11:19.020420+0000","last_undegraded":"2024-09-16T00:11:19.020420+0000","last_fullsized":"2024-09-16T00:11:19.020420+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":12682,"num_write":399,"num_write_kb":54869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"146'423","reported_seq":714,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.011812+0000","last_change":"2024-09-16T00:07:33.634956+0000","last_active":"2024-09-16T00:11:19.011812+0000","last_peered":"2024-09-16T00:11:19.011812+0000","last_clean":"2024-09-16T00:11:19.011812+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:11:19.011812+0000","last_undegraded":"2024-09-16T00:11:19.011812+0000","last_fullsized":"2024-09-16T00:11:19.011812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:30.406556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":14188,"num_write":408,"num_write_kb":59178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"146'463","reported_seq":877,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.027229+0000","last_change":"2024-09-16T00:10:56.267499+0000","last_active":"2024-09-16T00:11:19.027229+0000","last_peered":"2024-09-16T00:11:19.027229+0000","last_clean":"2024-09-16T00:11:19.027229+0000","last_became_active":"2024-09-16T00:10:55.841967+0000","last_became_peered":"2024-09-16T00:10:55.841967+0000","last_unstale":"2024-09-16T00:11:19.027229+0000","last_undegraded":"2024-09-16T00:11:19.027229+0000","last_fullsized":"2024-09-16T00:11:19.027229+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:15.324718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":12330,"num_write":421,"num_write_kb":35146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":12903786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"146'408","reported_seq":1058,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:18.998926+0000","last_change":"2024-09-16T00:11:04.895240+0000","last_active":"2024-09-16T00:11:18.998926+0000","last_peered":"2024-09-16T00:11:18.998926+0000","last_clean":"2024-09-16T00:11:18.998926+0000","last_became_active":"2024-09-16T00:11:04.892794+0000","last_became_peered":"2024-09-16T00:11:04.892794+0000","last_unstale":"2024-09-16T00:11:18.998926+0000","last_undegraded":"2024-09-16T00:11:18.998926+0000","last_fullsized":"2024-09-16T00:11:18.998926+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:26:44.365114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":21104,"num_write":383,"num_write_kb":75742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":628439,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"146'390","reported_seq":692,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.015372+0000","last_change":"2024-09-16T00:07:33.634960+0000","last_active":"2024-09-16T00:11:19.015372+0000","last_peered":"2024-09-16T00:11:19.015372+0000","last_clean":"2024-09-16T00:11:19.015372+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:11:19.015372+0000","last_undegraded":"2024-09-16T00:11:19.015372+0000","last_fullsized":"2024-09-16T00:11:19.015372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:13.948325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":16408,"num_write":366,"num_write_kb":51674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"146'405","reported_seq":724,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.008566+0000","last_change":"2024-09-16T00:07:33.625929+0000","last_active":"2024-09-16T00:11:19.008566+0000","last_peered":"2024-09-16T00:11:19.008566+0000","last_clean":"2024-09-16T00:11:19.008566+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:11:19.008566+0000","last_undegraded":"2024-09-16T00:11:19.008566+0000","last_fullsized":"2024-09-16T00:11:19.008566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:27.283713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":33174,"num_write":373,"num_write_kb":66849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"146'494","reported_seq":788,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.011926+0000","last_change":"2024-09-16T00:08:15.790616+0000","last_active":"2024-09-16T00:11:19.011926+0000","last_peered":"2024-09-16T00:11:19.011926+0000","last_clean":"2024-09-16T00:11:19.011926+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:11:19.011926+0000","last_undegraded":"2024-09-16T00:11:19.011926+0000","last_fullsized":"2024-09-16T00:11:19.011926+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:49.214388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":27231,"num_write":453,"num_write_kb":104771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"146'463","reported_seq":801,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.011767+0000","last_change":"2024-09-16T00:08:15.792552+0000","last_active":"2024-09-16T00:11:19.011767+0000","last_peered":"2024-09-16T00:11:19.011767+0000","last_clean":"2024-09-16T00:11:19.011767+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:11:19.011767+0000","last_undegraded":"2024-09-16T00:11:19.011767+0000","last_fullsized":"2024-09-16T00:11:19.011767+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:24:16.637050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":35655,"num_write":431,"num_write_kb":97245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"146'430","reported_seq":611,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.015178+0000","last_change":"2024-09-16T00:10:00.535553+0000","last_active":"2024-09-16T00:11:19.015178+0000","last_peered":"2024-09-16T00:11:19.015178+0000","last_clean":"2024-09-16T00:11:19.015178+0000","last_became_active":"2024-09-16T00:10:00.281378+0000","last_became_peered":"2024-09-16T00:10:00.281378+0000","last_unstale":"2024-09-16T00:11:19.015178+0000","last_undegraded":"2024-09-16T00:11:19.015178+0000","last_fullsized":"2024-09-16T00:11:19.015178+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:49.548709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20395,"num_write":401,"num_write_kb":32731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":22876917,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"146'408","reported_seq":748,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.009659+0000","last_change":"2024-09-16T00:07:33.632430+0000","last_active":"2024-09-16T00:11:19.009659+0000","last_peered":"2024-09-16T00:11:19.009659+0000","last_clean":"2024-09-16T00:11:19.009659+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:11:19.009659+0000","last_undegraded":"2024-09-16T00:11:19.009659+0000","last_fullsized":"2024-09-16T00:11:19.009659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:24.754189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":48141,"num_write":381,"num_write_kb":63611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"120'126","reported_seq":402,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749808+0000","last_change":"2024-09-16T00:10:55.854872+0000","last_active":"2024-09-16T00:11:15.749808+0000","last_peered":"2024-09-16T00:11:15.749808+0000","last_clean":"2024-09-16T00:11:15.749808+0000","last_became_active":"2024-09-16T00:10:55.843367+0000","last_became_peered":"2024-09-16T00:10:55.843367+0000","last_unstale":"2024-09-16T00:11:15.749808+0000","last_undegraded":"2024-09-16T00:11:15.749808+0000","last_fullsized":"2024-09-16T00:11:15.749808+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:07.797903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"146'354","reported_seq":641,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.027153+0000","last_change":"2024-09-16T00:07:33.635925+0000","last_active":"2024-09-16T00:11:19.027153+0000","last_peered":"2024-09-16T00:11:19.027153+0000","last_clean":"2024-09-16T00:11:19.027153+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:11:19.027153+0000","last_undegraded":"2024-09-16T00:11:19.027153+0000","last_fullsized":"2024-09-16T00:11:19.027153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":16420,"num_write":326,"num_write_kb":62710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"122'340","reported_seq":505,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011382+0000","last_change":"2024-09-16T00:10:00.282958+0000","last_active":"2024-09-16T00:11:14.011382+0000","last_peered":"2024-09-16T00:11:14.011382+0000","last_clean":"2024-09-16T00:11:14.011382+0000","last_became_active":"2024-09-16T00:10:00.282541+0000","last_became_peered":"2024-09-16T00:10:00.282541+0000","last_unstale":"2024-09-16T00:11:14.011382+0000","last_undegraded":"2024-09-16T00:11:14.011382+0000","last_fullsized":"2024-09-16T00:11:14.011382+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:51.198787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":285,"num_write_kb":12471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"123'226","reported_seq":510,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749827+0000","last_change":"2024-09-16T00:10:00.313589+0000","last_active":"2024-09-16T00:11:15.749827+0000","last_peered":"2024-09-16T00:11:15.749827+0000","last_clean":"2024-09-16T00:11:15.749827+0000","last_became_active":"2024-09-16T00:10:00.284602+0000","last_became_peered":"2024-09-16T00:10:00.284602+0000","last_unstale":"2024-09-16T00:11:15.749827+0000","last_undegraded":"2024-09-16T00:11:15.749827+0000","last_fullsized":"2024-09-16T00:11:15.749827+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:30.032698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":14,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"146'441","reported_seq":740,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.014677+0000","last_change":"2024-09-16T00:07:33.625895+0000","last_active":"2024-09-16T00:11:19.014677+0000","last_peered":"2024-09-16T00:11:19.014677+0000","last_clean":"2024-09-16T00:11:19.014677+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:11:19.014677+0000","last_undegraded":"2024-09-16T00:11:19.014677+0000","last_fullsized":"2024-09-16T00:11:19.014677+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:52.555484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":18480,"num_write":411,"num_write_kb":71748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"123'833","reported_seq":1095,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951164+0000","last_change":"2024-09-16T00:07:32.621012+0000","last_active":"2024-09-16T00:11:13.951164+0000","last_peered":"2024-09-16T00:11:13.951164+0000","last_clean":"2024-09-16T00:11:13.951164+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:11:13.951164+0000","last_undegraded":"2024-09-16T00:11:13.951164+0000","last_fullsized":"2024-09-16T00:11:13.951164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:02.583380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":8591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"146'435","reported_seq":784,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.007333+0000","last_change":"2024-09-16T00:07:33.625391+0000","last_active":"2024-09-16T00:11:19.007333+0000","last_peered":"2024-09-16T00:11:19.007333+0000","last_clean":"2024-09-16T00:11:19.007333+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:11:19.007333+0000","last_undegraded":"2024-09-16T00:11:19.007333+0000","last_fullsized":"2024-09-16T00:11:19.007333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:10.653965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":46871,"num_write":401,"num_write_kb":84187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"123'65","reported_seq":296,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011701+0000","last_change":"2024-09-16T00:10:00.283876+0000","last_active":"2024-09-16T00:11:14.011701+0000","last_peered":"2024-09-16T00:11:14.011701+0000","last_clean":"2024-09-16T00:11:14.011701+0000","last_became_active":"2024-09-16T00:10:00.283555+0000","last_became_peered":"2024-09-16T00:10:00.283555+0000","last_unstale":"2024-09-16T00:11:14.011701+0000","last_undegraded":"2024-09-16T00:11:14.011701+0000","last_fullsized":"2024-09-16T00:11:14.011701+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:54:48.543737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"146'450","reported_seq":914,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.024328+0000","last_change":"2024-09-16T00:10:56.502658+0000","last_active":"2024-09-16T00:11:19.024328+0000","last_peered":"2024-09-16T00:11:19.024328+0000","last_clean":"2024-09-16T00:11:19.024328+0000","last_became_active":"2024-09-16T00:10:55.840027+0000","last_became_peered":"2024-09-16T00:10:55.840027+0000","last_unstale":"2024-09-16T00:11:19.024328+0000","last_undegraded":"2024-09-16T00:11:19.024328+0000","last_fullsized":"2024-09-16T00:11:19.024328+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:52.105868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":16686,"num_write":415,"num_write_kb":80112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"123'225","reported_seq":494,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951555+0000","last_change":"2024-09-16T00:10:00.285386+0000","last_active":"2024-09-16T00:11:13.951555+0000","last_peered":"2024-09-16T00:11:13.951555+0000","last_clean":"2024-09-16T00:11:13.951555+0000","last_became_active":"2024-09-16T00:10:00.284717+0000","last_became_peered":"2024-09-16T00:10:00.284717+0000","last_unstale":"2024-09-16T00:11:13.951555+0000","last_undegraded":"2024-09-16T00:11:13.951555+0000","last_fullsized":"2024-09-16T00:11:13.951555+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:07.794163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":4181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"146'387","reported_seq":690,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.031903+0000","last_change":"2024-09-16T00:07:33.634086+0000","last_active":"2024-09-16T00:11:19.031903+0000","last_peered":"2024-09-16T00:11:19.031903+0000","last_clean":"2024-09-16T00:11:19.031903+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:11:19.031903+0000","last_undegraded":"2024-09-16T00:11:19.031903+0000","last_fullsized":"2024-09-16T00:11:19.031903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:00:31.134582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":18030,"num_write":359,"num_write_kb":55428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"125'131","reported_seq":392,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749519+0000","last_change":"2024-09-16T00:07:32.617007+0000","last_active":"2024-09-16T00:11:15.749519+0000","last_peered":"2024-09-16T00:11:15.749519+0000","last_clean":"2024-09-16T00:11:15.749519+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:11:15.749519+0000","last_undegraded":"2024-09-16T00:11:15.749519+0000","last_fullsized":"2024-09-16T00:11:15.749519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:19:45.756890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"146'384","reported_seq":829,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.030185+0000","last_change":"2024-09-16T00:10:56.173134+0000","last_active":"2024-09-16T00:11:19.030185+0000","last_peered":"2024-09-16T00:11:19.030185+0000","last_clean":"2024-09-16T00:11:19.030185+0000","last_became_active":"2024-09-16T00:10:55.833273+0000","last_became_peered":"2024-09-16T00:10:55.833273+0000","last_unstale":"2024-09-16T00:11:19.030185+0000","last_undegraded":"2024-09-16T00:11:19.030185+0000","last_fullsized":"2024-09-16T00:11:19.030185+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:11.402988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":22960,"num_write":348,"num_write_kb":63161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"146'241","reported_seq":417,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:18.974056+0000","last_change":"2024-09-16T00:10:00.300895+0000","last_active":"2024-09-16T00:11:18.974056+0000","last_peered":"2024-09-16T00:11:18.974056+0000","last_clean":"2024-09-16T00:11:18.974056+0000","last_became_active":"2024-09-16T00:10:00.283474+0000","last_became_peered":"2024-09-16T00:10:00.283474+0000","last_unstale":"2024-09-16T00:11:18.974056+0000","last_undegraded":"2024-09-16T00:11:18.974056+0000","last_fullsized":"2024-09-16T00:11:18.974056+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":241,"log_dups_size":0,"ondisk_log_size":241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:28:02.563817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10146145,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":10017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":13200,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"146'409","reported_seq":782,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.018778+0000","last_change":"2024-09-16T00:10:56.320793+0000","last_active":"2024-09-16T00:11:19.018778+0000","last_peered":"2024-09-16T00:11:19.018778+0000","last_clean":"2024-09-16T00:11:19.018778+0000","last_became_active":"2024-09-16T00:10:55.836462+0000","last_became_peered":"2024-09-16T00:10:55.836462+0000","last_unstale":"2024-09-16T00:11:19.018778+0000","last_undegraded":"2024-09-16T00:11:19.018778+0000","last_fullsized":"2024-09-16T00:11:19.018778+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:07:47.717822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":23285,"num_write":372,"num_write_kb":67093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"122'470","reported_seq":732,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381212+0000","last_change":"2024-09-16T00:07:32.629358+0000","last_active":"2024-09-16T00:11:14.381212+0000","last_peered":"2024-09-16T00:11:14.381212+0000","last_clean":"2024-09-16T00:11:14.381212+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:11:14.381212+0000","last_undegraded":"2024-09-16T00:11:14.381212+0000","last_fullsized":"2024-09-16T00:11:14.381212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":419,"num_write_kb":8413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"146'413","reported_seq":881,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.023743+0000","last_change":"2024-09-16T00:10:56.427976+0000","last_active":"2024-09-16T00:11:19.023743+0000","last_peered":"2024-09-16T00:11:19.023743+0000","last_clean":"2024-09-16T00:11:19.023743+0000","last_became_active":"2024-09-16T00:10:55.839196+0000","last_became_peered":"2024-09-16T00:10:55.839196+0000","last_unstale":"2024-09-16T00:11:19.023743+0000","last_undegraded":"2024-09-16T00:11:19.023743+0000","last_fullsized":"2024-09-16T00:11:19.023743+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:37.852194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":13263,"num_write":381,"num_write_kb":51174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"123'59","reported_seq":321,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952149+0000","last_change":"2024-09-16T00:07:32.625760+0000","last_active":"2024-09-16T00:11:13.952149+0000","last_peered":"2024-09-16T00:11:13.952149+0000","last_clean":"2024-09-16T00:11:13.952149+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:11:13.952149+0000","last_undegraded":"2024-09-16T00:11:13.952149+0000","last_fullsized":"2024-09-16T00:11:13.952149+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:23.268476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"146'409","reported_seq":750,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.030441+0000","last_change":"2024-09-16T00:10:56.085067+0000","last_active":"2024-09-16T00:11:19.030441+0000","last_peered":"2024-09-16T00:11:19.030441+0000","last_clean":"2024-09-16T00:11:19.030441+0000","last_became_active":"2024-09-16T00:10:55.834112+0000","last_became_peered":"2024-09-16T00:10:55.834112+0000","last_unstale":"2024-09-16T00:11:19.030441+0000","last_undegraded":"2024-09-16T00:11:19.030441+0000","last_fullsized":"2024-09-16T00:11:19.030441+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:35.278069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":13990,"num_write":368,"num_write_kb":63962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"120'70","reported_seq":218,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011398+0000","last_change":"2024-09-16T00:10:00.294519+0000","last_active":"2024-09-16T00:11:14.011398+0000","last_peered":"2024-09-16T00:11:14.011398+0000","last_clean":"2024-09-16T00:11:14.011398+0000","last_became_active":"2024-09-16T00:10:00.283066+0000","last_became_peered":"2024-09-16T00:10:00.283066+0000","last_unstale":"2024-09-16T00:11:14.011398+0000","last_undegraded":"2024-09-16T00:11:14.011398+0000","last_fullsized":"2024-09-16T00:11:14.011398+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:33:32.621176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"146'430","reported_seq":613,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:18.985627+0000","last_change":"2024-09-16T00:10:00.460910+0000","last_active":"2024-09-16T00:11:18.985627+0000","last_peered":"2024-09-16T00:11:18.985627+0000","last_clean":"2024-09-16T00:11:18.985627+0000","last_became_active":"2024-09-16T00:10:00.280750+0000","last_became_peered":"2024-09-16T00:10:00.280750+0000","last_unstale":"2024-09-16T00:11:18.985627+0000","last_undegraded":"2024-09-16T00:11:18.985627+0000","last_fullsized":"2024-09-16T00:11:18.985627+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:47.634650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":19573,"num_write":402,"num_write_kb":50826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":33491520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"125'72","reported_seq":334,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952685+0000","last_change":"2024-09-16T00:07:32.622989+0000","last_active":"2024-09-16T00:11:13.952685+0000","last_peered":"2024-09-16T00:11:13.952685+0000","last_clean":"2024-09-16T00:11:13.952685+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:11:13.952685+0000","last_undegraded":"2024-09-16T00:11:13.952685+0000","last_fullsized":"2024-09-16T00:11:13.952685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:10.846583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"146'463","reported_seq":822,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.029209+0000","last_change":"2024-09-16T00:10:55.922694+0000","last_active":"2024-09-16T00:11:19.029209+0000","last_peered":"2024-09-16T00:11:19.029209+0000","last_clean":"2024-09-16T00:11:19.029209+0000","last_became_active":"2024-09-16T00:10:55.827989+0000","last_became_peered":"2024-09-16T00:10:55.827989+0000","last_unstale":"2024-09-16T00:11:19.029209+0000","last_undegraded":"2024-09-16T00:11:19.029209+0000","last_fullsized":"2024-09-16T00:11:19.029209+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:01:23.666346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20419,"num_write":427,"num_write_kb":43035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"123'145","reported_seq":407,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951453+0000","last_change":"2024-09-16T00:07:32.628615+0000","last_active":"2024-09-16T00:11:13.951453+0000","last_peered":"2024-09-16T00:11:13.951453+0000","last_clean":"2024-09-16T00:11:13.951453+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:11:13.951453+0000","last_undegraded":"2024-09-16T00:11:13.951453+0000","last_fullsized":"2024-09-16T00:11:13.951453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:04:36.452327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91,"num_write_kb":4217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"146'363","reported_seq":701,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.028870+0000","last_change":"2024-09-16T00:10:55.971163+0000","last_active":"2024-09-16T00:11:19.028870+0000","last_peered":"2024-09-16T00:11:19.028870+0000","last_clean":"2024-09-16T00:11:19.028870+0000","last_became_active":"2024-09-16T00:10:55.834401+0000","last_became_peered":"2024-09-16T00:10:55.834401+0000","last_unstale":"2024-09-16T00:11:19.028870+0000","last_undegraded":"2024-09-16T00:11:19.028870+0000","last_fullsized":"2024-09-16T00:11:19.028870+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:05.466545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":10790,"num_write":330,"num_write_kb":21585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"120'282","reported_seq":544,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952893+0000","last_change":"2024-09-16T00:07:32.628773+0000","last_active":"2024-09-16T00:11:13.952893+0000","last_peered":"2024-09-16T00:11:13.952893+0000","last_clean":"2024-09-16T00:11:13.952893+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:11:13.952893+0000","last_undegraded":"2024-09-16T00:11:13.952893+0000","last_fullsized":"2024-09-16T00:11:13.952893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:38.568073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":238,"num_write_kb":4238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"146'381","reported_seq":707,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.030031+0000","last_change":"2024-09-16T00:10:56.200838+0000","last_active":"2024-09-16T00:11:19.030031+0000","last_peered":"2024-09-16T00:11:19.030031+0000","last_clean":"2024-09-16T00:11:19.030031+0000","last_became_active":"2024-09-16T00:10:55.838254+0000","last_became_peered":"2024-09-16T00:10:55.838254+0000","last_unstale":"2024-09-16T00:11:19.030031+0000","last_undegraded":"2024-09-16T00:11:19.030031+0000","last_fullsized":"2024-09-16T00:11:19.030031+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:30.944542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":350,"num_write":351,"num_write_kb":38221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"122'534","reported_seq":795,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749610+0000","last_change":"2024-09-16T00:07:32.624549+0000","last_active":"2024-09-16T00:11:15.749610+0000","last_peered":"2024-09-16T00:11:15.749610+0000","last_clean":"2024-09-16T00:11:15.749610+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:11:15.749610+0000","last_undegraded":"2024-09-16T00:11:15.749610+0000","last_fullsized":"2024-09-16T00:11:15.749610+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:35.320386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":473,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"146'424","reported_seq":688,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.019656+0000","last_change":"2024-09-16T00:07:33.627599+0000","last_active":"2024-09-16T00:11:19.019656+0000","last_peered":"2024-09-16T00:11:19.019656+0000","last_clean":"2024-09-16T00:11:19.019656+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:11:19.019656+0000","last_undegraded":"2024-09-16T00:11:19.019656+0000","last_fullsized":"2024-09-16T00:11:19.019656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:15:49.299736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":675,"num_write":389,"num_write_kb":34696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"123'72","reported_seq":334,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952967+0000","last_change":"2024-09-16T00:07:32.629066+0000","last_active":"2024-09-16T00:11:13.952967+0000","last_peered":"2024-09-16T00:11:13.952967+0000","last_clean":"2024-09-16T00:11:13.952967+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:11:13.952967+0000","last_undegraded":"2024-09-16T00:11:13.952967+0000","last_fullsized":"2024-09-16T00:11:13.952967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:39.444390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"146'404","reported_seq":770,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.030006+0000","last_change":"2024-09-16T00:10:56.368023+0000","last_active":"2024-09-16T00:11:19.030006+0000","last_peered":"2024-09-16T00:11:19.030006+0000","last_clean":"2024-09-16T00:11:19.030006+0000","last_became_active":"2024-09-16T00:10:55.839714+0000","last_became_peered":"2024-09-16T00:10:55.839714+0000","last_unstale":"2024-09-16T00:11:19.030006+0000","last_undegraded":"2024-09-16T00:11:19.030006+0000","last_fullsized":"2024-09-16T00:11:19.030006+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:04.192344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":22590,"num_write":379,"num_write_kb":59088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":374,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.903717+0000","last_change":"2024-09-16T00:10:55.832444+0000","last_active":"2024-09-16T00:11:14.903717+0000","last_peered":"2024-09-16T00:11:14.903717+0000","last_clean":"2024-09-16T00:11:14.903717+0000","last_became_active":"2024-09-16T00:10:55.831881+0000","last_became_peered":"2024-09-16T00:10:55.831881+0000","last_unstale":"2024-09-16T00:11:14.903717+0000","last_undegraded":"2024-09-16T00:11:14.903717+0000","last_fullsized":"2024-09-16T00:11:14.903717+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:22.067282+0000","last_clean_scrub_stamp":"2024-09-16T00:07:22.067282+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:00:48.243166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"123'63","reported_seq":325,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.951198+0000","last_change":"2024-09-16T00:07:32.622400+0000","last_active":"2024-09-16T00:11:13.951198+0000","last_peered":"2024-09-16T00:11:13.951198+0000","last_clean":"2024-09-16T00:11:13.951198+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:11:13.951198+0000","last_undegraded":"2024-09-16T00:11:13.951198+0000","last_fullsized":"2024-09-16T00:11:13.951198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:50.015190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"146'377","reported_seq":672,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.031318+0000","last_change":"2024-09-16T00:07:33.634090+0000","last_active":"2024-09-16T00:11:19.031318+0000","last_peered":"2024-09-16T00:11:19.031318+0000","last_clean":"2024-09-16T00:11:19.031318+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:11:19.031318+0000","last_undegraded":"2024-09-16T00:11:19.031318+0000","last_fullsized":"2024-09-16T00:11:19.031318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:07:44.752086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":16103,"num_write":342,"num_write_kb":46241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"123'399","reported_seq":645,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.903728+0000","last_change":"2024-09-16T00:11:04.896119+0000","last_active":"2024-09-16T00:11:14.903728+0000","last_peered":"2024-09-16T00:11:14.903728+0000","last_clean":"2024-09-16T00:11:14.903728+0000","last_became_active":"2024-09-16T00:11:04.895672+0000","last_became_peered":"2024-09-16T00:11:04.895672+0000","last_unstale":"2024-09-16T00:11:14.903728+0000","last_undegraded":"2024-09-16T00:11:14.903728+0000","last_fullsized":"2024-09-16T00:11:14.903728+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:45.173943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":346,"num_write_kb":8359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":16777216,"num_keys_recovered":28,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"146'398","reported_seq":705,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.027184+0000","last_change":"2024-09-16T00:07:33.622242+0000","last_active":"2024-09-16T00:11:19.027184+0000","last_peered":"2024-09-16T00:11:19.027184+0000","last_clean":"2024-09-16T00:11:19.027184+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:11:19.027184+0000","last_undegraded":"2024-09-16T00:11:19.027184+0000","last_fullsized":"2024-09-16T00:11:19.027184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:04.067803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":29385,"num_write":362,"num_write_kb":83610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"125'242","reported_seq":680,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952710+0000","last_change":"2024-09-16T00:07:32.628602+0000","last_active":"2024-09-16T00:11:13.952710+0000","last_peered":"2024-09-16T00:11:13.952710+0000","last_clean":"2024-09-16T00:11:13.952710+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:11:13.952710+0000","last_undegraded":"2024-09-16T00:11:13.952710+0000","last_fullsized":"2024-09-16T00:11:13.952710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:17:57.683749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461247,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":176,"num_read_kb":1534,"num_write":179,"num_write_kb":1542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"146'408","reported_seq":720,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.010372+0000","last_change":"2024-09-16T00:07:33.617833+0000","last_active":"2024-09-16T00:11:19.010372+0000","last_peered":"2024-09-16T00:11:19.010372+0000","last_clean":"2024-09-16T00:11:19.010372+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:11:19.010372+0000","last_undegraded":"2024-09-16T00:11:19.010372+0000","last_fullsized":"2024-09-16T00:11:19.010372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:29:04.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":28288,"num_write":378,"num_write_kb":66949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"119'400","reported_seq":662,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:13.952094+0000","last_change":"2024-09-16T00:07:32.623256+0000","last_active":"2024-09-16T00:11:13.952094+0000","last_peered":"2024-09-16T00:11:13.952094+0000","last_clean":"2024-09-16T00:11:13.952094+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:11:13.952094+0000","last_undegraded":"2024-09-16T00:11:13.952094+0000","last_fullsized":"2024-09-16T00:11:13.952094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":353,"num_write_kb":8381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"146'394","reported_seq":690,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.025407+0000","last_change":"2024-09-16T00:07:33.628364+0000","last_active":"2024-09-16T00:11:19.025407+0000","last_peered":"2024-09-16T00:11:19.025407+0000","last_clean":"2024-09-16T00:11:19.025407+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:11:19.025407+0000","last_undegraded":"2024-09-16T00:11:19.025407+0000","last_fullsized":"2024-09-16T00:11:19.025407+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:11:35.726011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":22188,"num_write":360,"num_write_kb":71552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"122'57","reported_seq":318,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749407+0000","last_change":"2024-09-16T00:07:32.620971+0000","last_active":"2024-09-16T00:11:15.749407+0000","last_peered":"2024-09-16T00:11:15.749407+0000","last_clean":"2024-09-16T00:11:15.749407+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:11:15.749407+0000","last_undegraded":"2024-09-16T00:11:15.749407+0000","last_fullsized":"2024-09-16T00:11:15.749407+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:08:23.746660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"146'431","reported_seq":730,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.019955+0000","last_change":"2024-09-16T00:07:33.627096+0000","last_active":"2024-09-16T00:11:19.019955+0000","last_peered":"2024-09-16T00:11:19.019955+0000","last_clean":"2024-09-16T00:11:19.019955+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:11:19.019955+0000","last_undegraded":"2024-09-16T00:11:19.019955+0000","last_fullsized":"2024-09-16T00:11:19.019955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:38:04.986700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":24752,"num_write":390,"num_write_kb":75609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"123'258","reported_seq":303,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.903671+0000","last_change":"2024-09-16T00:10:58.839395+0000","last_active":"2024-09-16T00:11:14.903671+0000","last_peered":"2024-09-16T00:11:14.903671+0000","last_clean":"2024-09-16T00:11:14.903671+0000","last_became_active":"2024-09-16T00:10:58.839124+0000","last_became_peered":"2024-09-16T00:10:58.839124+0000","last_unstale":"2024-09-16T00:11:14.903671+0000","last_undegraded":"2024-09-16T00:11:14.903671+0000","last_fullsized":"2024-09-16T00:11:14.903671+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:29.136640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":202,"num_write_kb":4192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"146'407","reported_seq":724,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.002015+0000","last_change":"2024-09-16T00:07:33.617696+0000","last_active":"2024-09-16T00:11:19.002015+0000","last_peered":"2024-09-16T00:11:19.002015+0000","last_clean":"2024-09-16T00:11:19.002015+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:11:19.002015+0000","last_undegraded":"2024-09-16T00:11:19.002015+0000","last_fullsized":"2024-09-16T00:11:19.002015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:18:27.507297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":31221,"num_write":370,"num_write_kb":76143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"125'186","reported_seq":482,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749569+0000","last_change":"2024-09-16T00:11:04.894640+0000","last_active":"2024-09-16T00:11:15.749569+0000","last_peered":"2024-09-16T00:11:15.749569+0000","last_clean":"2024-09-16T00:11:15.749569+0000","last_became_active":"2024-09-16T00:11:04.894163+0000","last_became_peered":"2024-09-16T00:11:04.894163+0000","last_unstale":"2024-09-16T00:11:15.749569+0000","last_undegraded":"2024-09-16T00:11:15.749569+0000","last_fullsized":"2024-09-16T00:11:15.749569+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:41.395352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":345,"num_write_kb":1559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":2964,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"146'349","reported_seq":791,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.010591+0000","last_change":"2024-09-16T00:10:56.334688+0000","last_active":"2024-09-16T00:11:19.010591+0000","last_peered":"2024-09-16T00:11:19.010591+0000","last_clean":"2024-09-16T00:11:19.010591+0000","last_became_active":"2024-09-16T00:10:55.845451+0000","last_became_peered":"2024-09-16T00:10:55.845451+0000","last_unstale":"2024-09-16T00:11:19.010591+0000","last_undegraded":"2024-09-16T00:11:19.010591+0000","last_fullsized":"2024-09-16T00:11:19.010591+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:04.965027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":19274,"num_write":323,"num_write_kb":22071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"123'63","reported_seq":309,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.011696+0000","last_change":"2024-09-16T00:10:55.858845+0000","last_active":"2024-09-16T00:11:14.011696+0000","last_peered":"2024-09-16T00:11:14.011696+0000","last_clean":"2024-09-16T00:11:14.011696+0000","last_became_active":"2024-09-16T00:10:55.845687+0000","last_became_peered":"2024-09-16T00:10:55.845687+0000","last_unstale":"2024-09-16T00:11:14.011696+0000","last_undegraded":"2024-09-16T00:11:14.011696+0000","last_fullsized":"2024-09-16T00:11:14.011696+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:53.322675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"146'454","reported_seq":757,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.017826+0000","last_change":"2024-09-16T00:07:33.626191+0000","last_active":"2024-09-16T00:11:19.017826+0000","last_peered":"2024-09-16T00:11:19.017826+0000","last_clean":"2024-09-16T00:11:19.017826+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:11:19.017826+0000","last_undegraded":"2024-09-16T00:11:19.017826+0000","last_fullsized":"2024-09-16T00:11:19.017826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:25:07.333833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":17894,"num_write":426,"num_write_kb":88843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"123'564","reported_seq":786,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.903856+0000","last_change":"2024-09-16T00:10:55.852895+0000","last_active":"2024-09-16T00:11:14.903856+0000","last_peered":"2024-09-16T00:11:14.903856+0000","last_clean":"2024-09-16T00:11:14.903856+0000","last_became_active":"2024-09-16T00:10:55.828121+0000","last_became_peered":"2024-09-16T00:10:55.828121+0000","last_unstale":"2024-09-16T00:11:14.903856+0000","last_undegraded":"2024-09-16T00:11:14.903856+0000","last_fullsized":"2024-09-16T00:11:14.903856+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:57.000428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"146'456","reported_seq":734,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:18.983750+0000","last_change":"2024-09-16T00:07:33.621993+0000","last_active":"2024-09-16T00:11:18.983750+0000","last_peered":"2024-09-16T00:11:18.983750+0000","last_clean":"2024-09-16T00:11:18.983750+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:11:18.983750+0000","last_undegraded":"2024-09-16T00:11:18.983750+0000","last_fullsized":"2024-09-16T00:11:18.983750+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:33:24.276005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":9206,"num_write":434,"num_write_kb":79714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"122'57","reported_seq":329,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749408+0000","last_change":"2024-09-16T00:10:55.850694+0000","last_active":"2024-09-16T00:11:15.749408+0000","last_peered":"2024-09-16T00:11:15.749408+0000","last_clean":"2024-09-16T00:11:15.749408+0000","last_became_active":"2024-09-16T00:10:55.837749+0000","last_became_peered":"2024-09-16T00:10:55.837749+0000","last_unstale":"2024-09-16T00:11:15.749408+0000","last_undegraded":"2024-09-16T00:11:15.749408+0000","last_fullsized":"2024-09-16T00:11:15.749408+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:21:53.941848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"146'391","reported_seq":1536,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.010242+0000","last_change":"2024-09-16T00:11:04.892185+0000","last_active":"2024-09-16T00:11:19.010242+0000","last_peered":"2024-09-16T00:11:19.010242+0000","last_clean":"2024-09-16T00:11:19.010242+0000","last_became_active":"2024-09-16T00:11:04.890897+0000","last_became_peered":"2024-09-16T00:11:04.890897+0000","last_unstale":"2024-09-16T00:11:19.010242+0000","last_undegraded":"2024-09-16T00:11:19.010242+0000","last_fullsized":"2024-09-16T00:11:19.010242+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:59:18.504239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":8384,"num_write":359,"num_write_kb":50710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":421,"num_bytes_recovered":615552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"122'180","reported_seq":469,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.749303+0000","last_change":"2024-09-16T00:11:04.891195+0000","last_active":"2024-09-16T00:11:15.749303+0000","last_peered":"2024-09-16T00:11:15.749303+0000","last_clean":"2024-09-16T00:11:15.749303+0000","last_became_active":"2024-09-16T00:11:04.890722+0000","last_became_peered":"2024-09-16T00:11:04.890722+0000","last_unstale":"2024-09-16T00:11:15.749303+0000","last_undegraded":"2024-09-16T00:11:15.749303+0000","last_fullsized":"2024-09-16T00:11:15.749303+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:36.844666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":4191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":5577662,"num_keys_recovered":51,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"146'428","reported_seq":758,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.004325+0000","last_change":"2024-09-16T00:07:33.630272+0000","last_active":"2024-09-16T00:11:19.004325+0000","last_peered":"2024-09-16T00:11:19.004325+0000","last_clean":"2024-09-16T00:11:19.004325+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:11:19.004325+0000","last_undegraded":"2024-09-16T00:11:19.004325+0000","last_fullsized":"2024-09-16T00:11:19.004325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:52:27.255471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":37740,"num_write":400,"num_write_kb":67449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"122'114","reported_seq":325,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.903984+0000","last_change":"2024-09-16T00:10:55.854939+0000","last_active":"2024-09-16T00:11:14.903984+0000","last_peered":"2024-09-16T00:11:14.903984+0000","last_clean":"2024-09-16T00:11:14.903984+0000","last_became_active":"2024-09-16T00:10:55.832841+0000","last_became_peered":"2024-09-16T00:10:55.832841+0000","last_unstale":"2024-09-16T00:11:14.903984+0000","last_undegraded":"2024-09-16T00:11:14.903984+0000","last_fullsized":"2024-09-16T00:11:14.903984+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:06:48.016972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":8303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"146'457","reported_seq":613,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.019527+0000","last_change":"2024-09-16T00:10:00.699942+0000","last_active":"2024-09-16T00:11:19.019527+0000","last_peered":"2024-09-16T00:11:19.019527+0000","last_clean":"2024-09-16T00:11:19.019527+0000","last_became_active":"2024-09-16T00:10:00.281972+0000","last_became_peered":"2024-09-16T00:10:00.281972+0000","last_unstale":"2024-09-16T00:11:19.019527+0000","last_undegraded":"2024-09-16T00:11:19.019527+0000","last_fullsized":"2024-09-16T00:11:19.019527+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:21.943709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":43345,"num_write":425,"num_write_kb":121191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":29546741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"146'411","reported_seq":740,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.033049+0000","last_change":"2024-09-16T00:07:33.620862+0000","last_active":"2024-09-16T00:11:19.033049+0000","last_peered":"2024-09-16T00:11:19.033049+0000","last_clean":"2024-09-16T00:11:19.033049+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:11:19.033049+0000","last_undegraded":"2024-09-16T00:11:19.033049+0000","last_fullsized":"2024-09-16T00:11:19.033049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:54:57.592761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":32882,"num_write":368,"num_write_kb":108251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"123'219","reported_seq":438,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.903959+0000","last_change":"2024-09-16T00:10:55.835894+0000","last_active":"2024-09-16T00:11:14.903959+0000","last_peered":"2024-09-16T00:11:14.903959+0000","last_clean":"2024-09-16T00:11:14.903959+0000","last_became_active":"2024-09-16T00:10:55.835301+0000","last_became_peered":"2024-09-16T00:10:55.835301+0000","last_unstale":"2024-09-16T00:11:14.903959+0000","last_undegraded":"2024-09-16T00:11:14.903959+0000","last_fullsized":"2024-09-16T00:11:14.903959+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:32.623825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":4194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"146'394","reported_seq":723,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.031731+0000","last_change":"2024-09-16T00:07:33.635628+0000","last_active":"2024-09-16T00:11:19.031731+0000","last_peered":"2024-09-16T00:11:19.031731+0000","last_clean":"2024-09-16T00:11:19.031731+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:11:19.031731+0000","last_undegraded":"2024-09-16T00:11:19.031731+0000","last_fullsized":"2024-09-16T00:11:19.031731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.535482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":36278,"num_write":367,"num_write_kb":113361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"123'263","reported_seq":494,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.904030+0000","last_change":"2024-09-16T00:10:55.856431+0000","last_active":"2024-09-16T00:11:14.904030+0000","last_peered":"2024-09-16T00:11:14.904030+0000","last_clean":"2024-09-16T00:11:14.904030+0000","last_became_active":"2024-09-16T00:10:55.843843+0000","last_became_peered":"2024-09-16T00:10:55.843843+0000","last_unstale":"2024-09-16T00:11:14.904030+0000","last_undegraded":"2024-09-16T00:11:14.904030+0000","last_fullsized":"2024-09-16T00:11:14.904030+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":263,"log_dups_size":0,"ondisk_log_size":263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:11.989047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":197,"num_write_kb":12428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":30,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"146'354","reported_seq":607,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.032219+0000","last_change":"2024-09-16T00:08:15.815358+0000","last_active":"2024-09-16T00:11:19.032219+0000","last_peered":"2024-09-16T00:11:19.032219+0000","last_clean":"2024-09-16T00:11:19.032219+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:11:19.032219+0000","last_undegraded":"2024-09-16T00:11:19.032219+0000","last_fullsized":"2024-09-16T00:11:19.032219+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:08.248193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":12352,"num_write":323,"num_write_kb":38097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"122'93","reported_seq":356,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:15.750053+0000","last_change":"2024-09-16T00:07:32.627916+0000","last_active":"2024-09-16T00:11:15.750053+0000","last_peered":"2024-09-16T00:11:15.750053+0000","last_clean":"2024-09-16T00:11:15.750053+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:11:15.750053+0000","last_undegraded":"2024-09-16T00:11:15.750053+0000","last_fullsized":"2024-09-16T00:11:15.750053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:12:36.617469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":4141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"146'419","reported_seq":800,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:19.007104+0000","last_change":"2024-09-16T00:10:00.556848+0000","last_active":"2024-09-16T00:11:19.007104+0000","last_peered":"2024-09-16T00:11:19.007104+0000","last_clean":"2024-09-16T00:11:19.007104+0000","last_became_active":"2024-09-16T00:10:00.283734+0000","last_became_peered":"2024-09-16T00:10:00.283734+0000","last_unstale":"2024-09-16T00:11:19.007104+0000","last_undegraded":"2024-09-16T00:11:19.007104+0000","last_fullsized":"2024-09-16T00:11:19.007104+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:32.602688+0000","last_clean_scrub_stamp":"2024-09-16T00:07:32.602688+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:55:01.843183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":40163,"num_write":392,"num_write_kb":81063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":21131701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"123'98","reported_seq":372,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:14.381221+0000","last_change":"2024-09-16T00:10:55.856802+0000","last_active":"2024-09-16T00:11:14.381221+0000","last_peered":"2024-09-16T00:11:14.381221+0000","last_clean":"2024-09-16T00:11:14.381221+0000","last_became_active":"2024-09-16T00:10:55.844922+0000","last_became_peered":"2024-09-16T00:10:55.844922+0000","last_unstale":"2024-09-16T00:11:14.381221+0000","last_undegraded":"2024-09-16T00:11:14.381221+0000","last_fullsized":"2024-09-16T00:11:14.381221+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:07:31.595159+0000","last_clean_scrub_stamp":"2024-09-16T00:07:31.595159+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:26:19.134467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":4138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2561,"num_read_kb":1362012,"num_write":24417,"num_write_kb":4088607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3177,"num_bytes_recovered":294487447,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5234,"internal_metadata":0},"log_size":26417,"ondisk_log_size":26417,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":129048702,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":176,"num_read_kb":1534,"num_write":5621,"num_write_kb":131062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":26562382,"num_keys_recovered":3117,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86642688,"data_stored":258097404,"data_compressed":38739888,"data_compressed_allocated":49811456,"data_compressed_original":221323264,"omap_allocated":262411,"internal_metadata":0},"log_size":7147,"ondisk_log_size":7147,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":127,"seq":545460846600,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57992,"kb_used_data":16056,"kb_used_omap":42,"kb_used_meta":41877,"kb_avail":94313848,"statfs":{"total":96636764160,"available":96577380352,"internally_reserved":0,"allocated":16441344,"data_stored":51769079,"data_compressed":7683094,"data_compressed_allocated":9629696,"data_compressed_original":45862912,"omap_allocated":43696,"internal_metadata":42882384},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":94,"seq":403726925843,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56024,"kb_used_data":14280,"kb_used_omap":221,"kb_used_meta":41506,"kb_avail":94315816,"statfs":{"total":96636764160,"available":96579395584,"internally_reserved":0,"allocated":14622720,"data_stored":44547058,"data_compressed":6872875,"data_compressed_allocated":8704000,"data_compressed_original":39534592,"omap_allocated":227133,"internal_metadata":42502339},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509492,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73840,"kb_used_data":12128,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94298000,"statfs":{"total":96636764160,"available":96561152000,"internally_reserved":0,"allocated":12419072,"data_stored":26012395,"data_compressed":3584779,"data_compressed_allocated":4849664,"data_compressed_original":19345408,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509493,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63484,"kb_used_data":18412,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94308356,"statfs":{"total":96636764160,"available":96571756544,"internally_reserved":0,"allocated":18853888,"data_stored":46983951,"data_compressed":6735741,"data_compressed_allocated":8769536,"data_compressed_original":37810176,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509491,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51044,"kb_used_data":11540,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320796,"statfs":{"total":96636764160,"available":96584495104,"internally_reserved":0,"allocated":11816960,"data_stored":33425393,"data_compressed":5001493,"data_compressed_allocated":6361088,"data_compressed_original":28876800,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542196,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67128,"kb_used_data":9576,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94304712,"statfs":{"total":96636764160,"available":96568025088,"internally_reserved":0,"allocated":9805824,"data_stored":26012465,"data_compressed":3864818,"data_compressed_allocated":4833280,"data_compressed_original":21946368,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542196,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57364,"kb_used_data":12292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94314476,"statfs":{"total":96636764160,"available":96578023424,"internally_reserved":0,"allocated":12587008,"data_stored":30797615,"data_compressed":4284061,"data_compressed_allocated":5672960,"data_compressed_original":24797184,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574900,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35272,"kb_used_data":4088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336568,"statfs":{"total":96636764160,"available":96600645632,"internally_reserved":0,"allocated":4186112,"data_stored":6502032,"data_compressed":777781,"data_compressed_allocated":1081344,"data_compressed_original":4296704,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10776576,"data_stored":29360766,"data_compressed":4251684,"data_compressed_allocated":5627904,"data_compressed_original":24223744,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8060928,"data_stored":25165984,"data_compressed":3864818,"data_compressed_allocated":4833280,"data_compressed_original":21946368,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2441216,"data_stored":5655551,"data_compressed":777781,"data_compressed_allocated":1081344,"data_compressed_original":4296704,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10072064,"data_stored":32578912,"data_compressed":5001493,"data_compressed_allocated":6361088,"data_compressed_original":28876800,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17108992,"data_stored":46137470,"data_compressed":6735741,"data_compressed_allocated":8769536,"data_compressed_original":37810176,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10674176,"data_stored":25165914,"data_compressed":3584779,"data_compressed_allocated":4849664,"data_compressed_original":19345408,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12877824,"data_stored":43700577,"data_compressed":6872875,"data_compressed_allocated":8704000,"data_compressed_original":39534592,"omap_allocated":225539,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14630912,"data_stored":50332230,"data_compressed":7650717,"data_compressed_allocated":9584640,"data_compressed_original":45289472,"omap_allocated":36872,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5234,"internal_metadata":0}]}} 2024-09-16T00:11:25.521 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.521 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.521 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.521 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.521 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.521 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.522 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.522 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.522 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.522 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.522 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.522 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.523 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.523 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.523 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.523 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.523 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.523 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.523 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.524 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.524 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.524 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.524 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.524 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.524 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.524 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.524 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.525 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.525 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.525 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.525 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.525 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.525 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.525 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.525 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.525 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.526 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.526 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.526 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.526 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.526 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.526 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.526 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.526 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.526 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.527 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.527 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.527 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.527 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.527 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.527 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.527 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.527 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.528 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.528 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.528 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.528 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.528 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.528 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.528 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.528 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.529 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.529 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.529 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.529 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-16T00:07:22.067282+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=22, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.529 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.529 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.529 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.529 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.529 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.530 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.530 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.530 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.530 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.530 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.530 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.530 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.530 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.530 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.531 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.531 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.531 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.531 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.531 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.531 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.531 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.531 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.532 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.532 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.532 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.532 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.532 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.532 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.532 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.532 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.532 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-16T00:07:32.602688+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=32, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.533 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-09-16T00:07:31.595159+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=7, tm_sec=31, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=0, tm_min=11, tm_sec=20, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T00:11:25.533 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T00:11:33.489 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:11:33.492 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:11:45.537 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:11:45.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.647+0000 7f99c23c4640 1 Processor -- start 2024-09-16T00:11:45.648 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.647+0000 7f99c23c4640 1 -- start start 2024-09-16T00:11:45.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.648+0000 7f99c23c4640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f99bc0c57a0 0x7f99bc0c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:45.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.648+0000 7f99c23c4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f99bc0c3020 0x7f99bc159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:45.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.648+0000 7f99c23c4640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99bc0c6e80 con 0x7f99bc15bc50 2024-09-16T00:11:45.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.648+0000 7f99c23c4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f99bc159660 con 0x7f99bc0c3020 2024-09-16T00:11:45.654 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.648+0000 7f99c23c4640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f99bc158660 con 0x7f99bc0c57a0 2024-09-16T00:11:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.648+0000 7f99bb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f99bc0c3020 0x7f99bc159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.648+0000 7f99bb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f99bc0c3020 0x7f99bc159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45264/0 (socket says 172.21.15.139:45264) 2024-09-16T00:11:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.648+0000 7f99bb7fe640 1 -- 172.21.15.139:0/3553953860 learned_addr learned my addr 172.21.15.139:0/3553953860 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.648+0000 7f99bbfff640 1 --2- 172.21.15.139:0/3553953860 >> v2:172.21.15.164:3300/0 conn(0x7f99bc0c57a0 0x7f99bc0c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.648+0000 7f99bb7fe640 1 -- 172.21.15.139:0/3553953860 >> v2:172.21.15.164:3300/0 conn(0x7f99bc0c57a0 msgr2=0x7f99bc0c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.648+0000 7f99bb7fe640 1 --2- 172.21.15.139:0/3553953860 >> v2:172.21.15.164:3300/0 conn(0x7f99bc0c57a0 0x7f99bc0c5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.648+0000 7f99bb7fe640 1 -- 172.21.15.139:0/3553953860 >> v1:172.21.15.164:6789/0 conn(0x7f99bc15bc50 legacy=0x7f99bc1524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.648+0000 7f99bb7fe640 1 -- 172.21.15.139:0/3553953860 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99bc0c8870 con 0x7f99bc0c3020 2024-09-16T00:11:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.649+0000 7f99bb7fe640 1 --2- 172.21.15.139:0/3553953860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99bc0c3020 0x7f99bc159120 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0cd130 tx=0x7f99b002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e691899f2023b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.649+0000 7f99baffd640 1 -- 172.21.15.139:0/3553953860 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0037070 con 0x7f99bc0c3020 2024-09-16T00:11:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.649+0000 7f99baffd640 1 -- 172.21.15.139:0/3553953860 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99b002ae50 con 0x7f99bc0c3020 2024-09-16T00:11:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.649+0000 7f99baffd640 1 -- 172.21.15.139:0/3553953860 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99b0032860 con 0x7f99bc0c3020 2024-09-16T00:11:45.655 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.650+0000 7f99c23c4640 1 -- 172.21.15.139:0/3553953860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99bc0c3020 msgr2=0x7f99bc159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:45.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.650+0000 7f99c23c4640 1 --2- 172.21.15.139:0/3553953860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99bc0c3020 0x7f99bc159120 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0cd130 tx=0x7f99b002a300 comp rx=0 tx=0).stop 2024-09-16T00:11:45.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.650+0000 7f99c23c4640 1 -- 172.21.15.139:0/3553953860 shutdown_connections 2024-09-16T00:11:45.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.650+0000 7f99c23c4640 1 --2- 172.21.15.139:0/3553953860 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99bc0c3020 0x7f99bc159120 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:45.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.650+0000 7f99c23c4640 1 --2- 172.21.15.139:0/3553953860 >> v2:172.21.15.164:3300/0 conn(0x7f99bc0c57a0 0x7f99bc0c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:45.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.650+0000 7f99c23c4640 1 -- 172.21.15.139:0/3553953860 >> 172.21.15.139:0/3553953860 conn(0x7f99bc067950 msgr2=0x7f99bc067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:45.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.650+0000 7f99c23c4640 1 -- 172.21.15.139:0/3553953860 shutdown_connections 2024-09-16T00:11:45.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.650+0000 7f99c23c4640 1 -- 172.21.15.139:0/3553953860 wait complete. 2024-09-16T00:11:45.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.651+0000 7f99c23c4640 1 Processor -- start 2024-09-16T00:11:45.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.651+0000 7f99c23c4640 1 -- start start 2024-09-16T00:11:45.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.652+0000 7f99c23c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99bc0c3020 0x7f99bc0ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:45.656 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.652+0000 7f99c23c4640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99bc0c57a0 0x7f99bc0cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:45.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.652+0000 7f99c23c4640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99bc15bc50 0x7f99bc14aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:45.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.652+0000 7f99c23c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc158c00 con 0x7f99bc0c3020 2024-09-16T00:11:45.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.652+0000 7f99c23c4640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc158920 con 0x7f99bc15bc50 2024-09-16T00:11:45.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.652+0000 7f99c23c4640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f99bc0c6a90 con 0x7f99bc0c57a0 2024-09-16T00:11:45.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.652+0000 7f99bbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99bc0c3020 0x7f99bc0ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:45.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.652+0000 7f99bbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99bc0c3020 0x7f99bc0ca8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45280/0 (socket says 172.21.15.139:45280) 2024-09-16T00:11:45.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.652+0000 7f99bbfff640 1 -- 172.21.15.139:0/396782073 learned_addr learned my addr 172.21.15.139:0/396782073 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:45.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.652+0000 7f99bb7fe640 1 --2- 172.21.15.139:0/396782073 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99bc0c57a0 0x7f99bc0cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:45.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.652+0000 7f99c0961640 1 --2- 172.21.15.139:0/396782073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99bc15bc50 0x7f99bc14aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:45.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.652+0000 7f99bbfff640 1 -- 172.21.15.139:0/396782073 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99bc0c57a0 msgr2=0x7f99bc0cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:45.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.652+0000 7f99bbfff640 1 --2- 172.21.15.139:0/396782073 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99bc0c57a0 0x7f99bc0cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:45.657 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.652+0000 7f99bbfff640 1 -- 172.21.15.139:0/396782073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99bc15bc50 msgr2=0x7f99bc14aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:45.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.653+0000 7f99bbfff640 1 --2- 172.21.15.139:0/396782073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99bc15bc50 0x7f99bc14aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:45.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.653+0000 7f99bbfff640 1 -- 172.21.15.139:0/396782073 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99bc1ee560 con 0x7f99bc0c3020 2024-09-16T00:11:45.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.653+0000 7f99c0961640 1 --2- 172.21.15.139:0/396782073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99bc15bc50 0x7f99bc14aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:11:45.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.653+0000 7f99bbfff640 1 --2- 172.21.15.139:0/396782073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99bc0c3020 0x7f99bc0ca8e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00d800 tx=0x7f99ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:45.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.653+0000 7f99b8ff9640 1 -- 172.21.15.139:0/396782073 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac00b6c0 con 0x7f99bc0c3020 2024-09-16T00:11:45.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.653+0000 7f99b8ff9640 1 -- 172.21.15.139:0/396782073 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99ac019070 con 0x7f99bc0c3020 2024-09-16T00:11:45.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.653+0000 7f99c23c4640 1 -- 172.21.15.139:0/396782073 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99bc072990 con 0x7f99bc0c3020 2024-09-16T00:11:45.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.653+0000 7f99b8ff9640 1 -- 172.21.15.139:0/396782073 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac00bac0 con 0x7f99bc0c3020 2024-09-16T00:11:45.658 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.653+0000 7f99c23c4640 1 -- 172.21.15.139:0/396782073 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99bc1ee800 con 0x7f99bc0c3020 2024-09-16T00:11:45.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.654+0000 7f99c23c4640 1 -- 172.21.15.139:0/396782073 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99bc06c520 con 0x7f99bc0c3020 2024-09-16T00:11:45.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.654+0000 7f99b8ff9640 1 -- 172.21.15.139:0/396782073 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f99ac00bc60 con 0x7f99bc0c3020 2024-09-16T00:11:45.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.655+0000 7f99b8ff9640 1 --2- 172.21.15.139:0/396782073 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f99a806c2b0 0x7f99a806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:45.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.655+0000 7f99b8ff9640 1 -- 172.21.15.139:0/396782073 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f99ac08b9c0 con 0x7f99bc0c3020 2024-09-16T00:11:45.659 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.655+0000 7f99bb7fe640 1 --2- 172.21.15.139:0/396782073 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f99a806c2b0 0x7f99a806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:45.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.655+0000 7f99bb7fe640 1 --2- 172.21.15.139:0/396782073 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f99a806c2b0 0x7f99a806e770 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f99b0034000 tx=0x7f99b002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:45.660 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.659+0000 7f99b8ff9640 1 -- 172.21.15.139:0/396782073 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f99ac08de20 con 0x7f99bc0c3020 2024-09-16T00:11:45.822 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.821+0000 7f99c23c4640 1 -- 172.21.15.139:0/396782073 --> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99bc151cf0 con 0x7f99a806c2b0 2024-09-16T00:11:45.827 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.827+0000 7f99b8ff9640 1 -- 172.21.15.139:0/396782073 <== mgr.4101 v2:172.21.15.139:6824/2998946291 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+255375 (secure 0 0 0) 0x7f99bc151cf0 con 0x7f99a806c2b0 2024-09-16T00:11:45.831 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:11:45.832 INFO:teuthology.orchestra.run.smithi139.stderr:dumped all 2024-09-16T00:11:45.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.830+0000 7f99c23c4640 1 -- 172.21.15.139:0/396782073 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f99a806c2b0 msgr2=0x7f99a806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:45.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.830+0000 7f99c23c4640 1 --2- 172.21.15.139:0/396782073 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f99a806c2b0 0x7f99a806e770 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f99b0034000 tx=0x7f99b002e000 comp rx=0 tx=0).stop 2024-09-16T00:11:45.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.830+0000 7f99c23c4640 1 -- 172.21.15.139:0/396782073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99bc0c3020 msgr2=0x7f99bc0ca8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:45.834 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.830+0000 7f99c23c4640 1 --2- 172.21.15.139:0/396782073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99bc0c3020 0x7f99bc0ca8e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00d800 tx=0x7f99ac00dcc0 comp rx=0 tx=0).stop 2024-09-16T00:11:45.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.830+0000 7f99c23c4640 1 -- 172.21.15.139:0/396782073 shutdown_connections 2024-09-16T00:11:45.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.830+0000 7f99c23c4640 1 --2- 172.21.15.139:0/396782073 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f99a806c2b0 0x7f99a806e770 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:45.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.830+0000 7f99c23c4640 1 --2- 172.21.15.139:0/396782073 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99bc15bc50 0x7f99bc14aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:45.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.830+0000 7f99c23c4640 1 --2- 172.21.15.139:0/396782073 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f99bc0c57a0 0x7f99bc0cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:45.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.830+0000 7f99c23c4640 1 --2- 172.21.15.139:0/396782073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99bc0c3020 0x7f99bc0ca8e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:45.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.830+0000 7f99c23c4640 1 -- 172.21.15.139:0/396782073 >> 172.21.15.139:0/396782073 conn(0x7f99bc067950 msgr2=0x7f99bc0c26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:45.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.830+0000 7f99c23c4640 1 -- 172.21.15.139:0/396782073 shutdown_connections 2024-09-16T00:11:45.835 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.830+0000 7f99c23c4640 1 -- 172.21.15.139:0/396782073 wait complete. 2024-09-16T00:11:45.842 INFO:teuthology.orchestra.run.smithi139.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-09-16T00:11:44.073167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":129639070,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":2853,"num_read_kb":1363646,"num_write":30197,"num_write_kb":4222085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3232,"num_bytes_recovered":321049829,"num_keys_recovered":3117,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33734,"ondisk_log_size":33734,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":445508,"kb_used_data":98372,"kb_used_omap":273,"kb_used_meta":346734,"kb_avail":754529212,"statfs":{"total":773094113280,"available":772637913088,"internally_reserved":0,"allocated":100732928,"data_stored":266049988,"data_compressed":38804642,"data_compressed_allocated":49901568,"data_compressed_original":222470144,"omap_allocated":280397,"internal_metadata":355055795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003075"},"pg_stats":[{"pgid":"3.3f","version":"146'442","reported_seq":886,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:40.058051+0000","last_change":"2024-09-16T00:11:40.058102+0000","last_active":"2024-09-16T00:11:40.058051+0000","last_peered":"2024-09-16T00:11:40.058051+0000","last_clean":"2024-09-16T00:11:40.058051+0000","last_became_active":"2024-09-16T00:10:55.845182+0000","last_became_peered":"2024-09-16T00:10:55.845182+0000","last_unstale":"2024-09-16T00:11:40.058051+0000","last_undegraded":"2024-09-16T00:11:40.058051+0000","last_fullsized":"2024-09-16T00:11:40.058051+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'442","last_scrub_stamp":"2024-09-16T00:11:40.057948+0000","last_deep_scrub":"146'442","last_deep_scrub_stamp":"2024-09-16T00:11:40.057948+0000","last_clean_scrub_stamp":"2024-09-16T00:11:40.057948+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:25:37.483858+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":64,"num_read_kb":42086,"num_write":409,"num_write_kb":112716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":31107821,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"146'421","reported_seq":1106,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:35.390945+0000","last_change":"2024-09-16T00:11:35.390972+0000","last_active":"2024-09-16T00:11:35.390945+0000","last_peered":"2024-09-16T00:11:35.390945+0000","last_clean":"2024-09-16T00:11:35.390945+0000","last_became_active":"2024-09-16T00:11:04.894999+0000","last_became_peered":"2024-09-16T00:11:04.894999+0000","last_unstale":"2024-09-16T00:11:35.390945+0000","last_undegraded":"2024-09-16T00:11:35.390945+0000","last_fullsized":"2024-09-16T00:11:35.390945+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'421","last_scrub_stamp":"2024-09-16T00:11:35.390861+0000","last_deep_scrub":"146'421","last_deep_scrub_stamp":"2024-09-16T00:11:35.390861+0000","last_clean_scrub_stamp":"2024-09-16T00:11:35.390861+0000","objects_scrubbed":0,"log_size":421,"log_dups_size":0,"ondisk_log_size":421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:56.815721+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":23064,"num_write":397,"num_write_kb":47220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":689591,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"146'374","reported_seq":673,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:24.748992+0000","last_change":"2024-09-16T00:11:24.749014+0000","last_active":"2024-09-16T00:11:24.748992+0000","last_peered":"2024-09-16T00:11:24.748992+0000","last_clean":"2024-09-16T00:11:24.748992+0000","last_became_active":"2024-09-16T00:07:33.624540+0000","last_became_peered":"2024-09-16T00:07:33.624540+0000","last_unstale":"2024-09-16T00:11:24.748992+0000","last_undegraded":"2024-09-16T00:11:24.748992+0000","last_fullsized":"2024-09-16T00:11:24.748992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'374","last_scrub_stamp":"2024-09-16T00:11:24.748908+0000","last_deep_scrub":"146'374","last_deep_scrub_stamp":"2024-09-16T00:11:24.748908+0000","last_clean_scrub_stamp":"2024-09-16T00:11:24.748908+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:44.252088+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15782,"num_write":349,"num_write_kb":54932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"146'397","reported_seq":697,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:33.694231+0000","last_change":"2024-09-16T00:11:33.694231+0000","last_active":"2024-09-16T00:11:33.694231+0000","last_peered":"2024-09-16T00:11:33.694231+0000","last_clean":"2024-09-16T00:11:33.694231+0000","last_became_active":"2024-09-16T00:07:33.627321+0000","last_became_peered":"2024-09-16T00:07:33.627321+0000","last_unstale":"2024-09-16T00:11:33.694231+0000","last_undegraded":"2024-09-16T00:11:33.694231+0000","last_fullsized":"2024-09-16T00:11:33.694231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'397","last_scrub_stamp":"2024-09-16T00:11:33.694128+0000","last_deep_scrub":"146'397","last_deep_scrub_stamp":"2024-09-16T00:11:33.694128+0000","last_clean_scrub_stamp":"2024-09-16T00:11:33.694128+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:24.810003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":14080,"num_write":364,"num_write_kb":63191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"146'449","reported_seq":761,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:30.387020+0000","last_change":"2024-09-16T00:11:30.387050+0000","last_active":"2024-09-16T00:11:30.387020+0000","last_peered":"2024-09-16T00:11:30.387020+0000","last_clean":"2024-09-16T00:11:30.387020+0000","last_became_active":"2024-09-16T00:08:15.791351+0000","last_became_peered":"2024-09-16T00:08:15.791351+0000","last_unstale":"2024-09-16T00:11:30.387020+0000","last_undegraded":"2024-09-16T00:11:30.387020+0000","last_fullsized":"2024-09-16T00:11:30.387020+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'449","last_scrub_stamp":"2024-09-16T00:11:30.386936+0000","last_deep_scrub":"146'449","last_deep_scrub_stamp":"2024-09-16T00:11:30.386936+0000","last_clean_scrub_stamp":"2024-09-16T00:11:30.386936+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:34.187428+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":36096,"num_write":427,"num_write_kb":89616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"146'390","reported_seq":1367,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:32.396897+0000","last_change":"2024-09-16T00:11:32.396924+0000","last_active":"2024-09-16T00:11:32.396897+0000","last_peered":"2024-09-16T00:11:32.396897+0000","last_clean":"2024-09-16T00:11:32.396897+0000","last_became_active":"2024-09-16T00:10:45.793434+0000","last_became_peered":"2024-09-16T00:10:45.793434+0000","last_unstale":"2024-09-16T00:11:32.396897+0000","last_undegraded":"2024-09-16T00:11:32.396897+0000","last_fullsized":"2024-09-16T00:11:32.396897+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'390","last_scrub_stamp":"2024-09-16T00:11:32.396811+0000","last_deep_scrub":"146'390","last_deep_scrub_stamp":"2024-09-16T00:11:32.396811+0000","last_clean_scrub_stamp":"2024-09-16T00:11:32.396811+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:38.689010+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":6174,"num_write":362,"num_write_kb":50827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":425,"num_bytes_recovered":23619955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"146'425","reported_seq":795,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:34.063475+0000","last_change":"2024-09-16T00:11:34.063520+0000","last_active":"2024-09-16T00:11:34.063475+0000","last_peered":"2024-09-16T00:11:34.063475+0000","last_clean":"2024-09-16T00:11:34.063475+0000","last_became_active":"2024-09-16T00:10:55.835516+0000","last_became_peered":"2024-09-16T00:10:55.835516+0000","last_unstale":"2024-09-16T00:11:34.063475+0000","last_undegraded":"2024-09-16T00:11:34.063475+0000","last_fullsized":"2024-09-16T00:11:34.063475+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'425","last_scrub_stamp":"2024-09-16T00:11:34.063354+0000","last_deep_scrub":"146'425","last_deep_scrub_stamp":"2024-09-16T00:11:34.063354+0000","last_clean_scrub_stamp":"2024-09-16T00:11:34.063354+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:00.916950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":16577,"num_write":401,"num_write_kb":64470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"146'383","reported_seq":659,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:29.330136+0000","last_change":"2024-09-16T00:11:29.330164+0000","last_active":"2024-09-16T00:11:29.330136+0000","last_peered":"2024-09-16T00:11:29.330136+0000","last_clean":"2024-09-16T00:11:29.330136+0000","last_became_active":"2024-09-16T00:08:15.814735+0000","last_became_peered":"2024-09-16T00:08:15.814735+0000","last_unstale":"2024-09-16T00:11:29.330136+0000","last_undegraded":"2024-09-16T00:11:29.330136+0000","last_fullsized":"2024-09-16T00:11:29.330136+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'383","last_scrub_stamp":"2024-09-16T00:11:29.330054+0000","last_deep_scrub":"146'383","last_deep_scrub_stamp":"2024-09-16T00:11:29.330054+0000","last_clean_scrub_stamp":"2024-09-16T00:11:29.330054+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:57:09.334596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":16776,"num_write":347,"num_write_kb":80164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"146'405","reported_seq":746,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:37.070082+0000","last_change":"2024-09-16T00:11:37.070108+0000","last_active":"2024-09-16T00:11:37.070082+0000","last_peered":"2024-09-16T00:11:37.070082+0000","last_clean":"2024-09-16T00:11:37.070082+0000","last_became_active":"2024-09-16T00:10:55.840604+0000","last_became_peered":"2024-09-16T00:10:55.840604+0000","last_unstale":"2024-09-16T00:11:37.070082+0000","last_undegraded":"2024-09-16T00:11:37.070082+0000","last_fullsized":"2024-09-16T00:11:37.070082+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'405","last_scrub_stamp":"2024-09-16T00:11:37.069990+0000","last_deep_scrub":"146'405","last_deep_scrub_stamp":"2024-09-16T00:11:37.069990+0000","last_clean_scrub_stamp":"2024-09-16T00:11:37.069990+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:28.681000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1432,"num_write":368,"num_write_kb":30311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"146'383","reported_seq":561,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:34.597058+0000","last_change":"2024-09-16T00:11:34.597084+0000","last_active":"2024-09-16T00:11:34.597058+0000","last_peered":"2024-09-16T00:11:34.597058+0000","last_clean":"2024-09-16T00:11:34.597058+0000","last_became_active":"2024-09-16T00:10:00.284452+0000","last_became_peered":"2024-09-16T00:10:00.284452+0000","last_unstale":"2024-09-16T00:11:34.597058+0000","last_undegraded":"2024-09-16T00:11:34.597058+0000","last_fullsized":"2024-09-16T00:11:34.597058+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'383","last_scrub_stamp":"2024-09-16T00:11:34.596973+0000","last_deep_scrub":"146'383","last_deep_scrub_stamp":"2024-09-16T00:11:34.596973+0000","last_clean_scrub_stamp":"2024-09-16T00:11:34.596973+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:02.751166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4096,"num_write":335,"num_write_kb":29952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":8736255,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"146'361","reported_seq":550,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:32.517863+0000","last_change":"2024-09-16T00:11:32.517911+0000","last_active":"2024-09-16T00:11:32.517863+0000","last_peered":"2024-09-16T00:11:32.517863+0000","last_clean":"2024-09-16T00:11:32.517863+0000","last_became_active":"2024-09-16T00:10:00.280273+0000","last_became_peered":"2024-09-16T00:10:00.280273+0000","last_unstale":"2024-09-16T00:11:32.517863+0000","last_undegraded":"2024-09-16T00:11:32.517863+0000","last_fullsized":"2024-09-16T00:11:32.517863+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'361","last_scrub_stamp":"2024-09-16T00:11:32.517726+0000","last_deep_scrub":"146'361","last_deep_scrub_stamp":"2024-09-16T00:11:32.517726+0000","last_clean_scrub_stamp":"2024-09-16T00:11:32.517726+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:06:30.798676+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":975,"num_write":339,"num_write_kb":25518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":10753732,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"146'373","reported_seq":721,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:28.231364+0000","last_change":"2024-09-16T00:11:28.231387+0000","last_active":"2024-09-16T00:11:28.231364+0000","last_peered":"2024-09-16T00:11:28.231364+0000","last_clean":"2024-09-16T00:11:28.231364+0000","last_became_active":"2024-09-16T00:10:00.278765+0000","last_became_peered":"2024-09-16T00:10:00.278765+0000","last_unstale":"2024-09-16T00:11:28.231364+0000","last_undegraded":"2024-09-16T00:11:28.231364+0000","last_fullsized":"2024-09-16T00:11:28.231364+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'373","last_scrub_stamp":"2024-09-16T00:11:28.231269+0000","last_deep_scrub":"146'373","last_deep_scrub_stamp":"2024-09-16T00:11:28.231269+0000","last_clean_scrub_stamp":"2024-09-16T00:11:28.231269+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:57.838613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":11563,"num_write":348,"num_write_kb":58933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":29565877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"146'429","reported_seq":803,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:32.686255+0000","last_change":"2024-09-16T00:11:32.686290+0000","last_active":"2024-09-16T00:11:32.686255+0000","last_peered":"2024-09-16T00:11:32.686255+0000","last_clean":"2024-09-16T00:11:32.686255+0000","last_became_active":"2024-09-16T00:10:00.286050+0000","last_became_peered":"2024-09-16T00:10:00.286050+0000","last_unstale":"2024-09-16T00:11:32.686255+0000","last_undegraded":"2024-09-16T00:11:32.686255+0000","last_fullsized":"2024-09-16T00:11:32.686255+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'429","last_scrub_stamp":"2024-09-16T00:11:32.686174+0000","last_deep_scrub":"146'429","last_deep_scrub_stamp":"2024-09-16T00:11:32.686174+0000","last_clean_scrub_stamp":"2024-09-16T00:11:32.686174+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:49.184718+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":35908,"num_write":393,"num_write_kb":104863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":30230864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"146'422","reported_seq":740,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:27.750182+0000","last_change":"2024-09-16T00:11:27.750211+0000","last_active":"2024-09-16T00:11:27.750182+0000","last_peered":"2024-09-16T00:11:27.750182+0000","last_clean":"2024-09-16T00:11:27.750182+0000","last_became_active":"2024-09-16T00:07:33.626696+0000","last_became_peered":"2024-09-16T00:07:33.626696+0000","last_unstale":"2024-09-16T00:11:27.750182+0000","last_undegraded":"2024-09-16T00:11:27.750182+0000","last_fullsized":"2024-09-16T00:11:27.750182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'422","last_scrub_stamp":"2024-09-16T00:11:27.750098+0000","last_deep_scrub":"146'422","last_deep_scrub_stamp":"2024-09-16T00:11:27.750098+0000","last_clean_scrub_stamp":"2024-09-16T00:11:27.750098+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:31:28.749269+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":23364,"num_write":390,"num_write_kb":43321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"146'399","reported_seq":700,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:28.368837+0000","last_change":"2024-09-16T00:11:28.368884+0000","last_active":"2024-09-16T00:11:28.368837+0000","last_peered":"2024-09-16T00:11:28.368837+0000","last_clean":"2024-09-16T00:11:28.368837+0000","last_became_active":"2024-09-16T00:07:33.619981+0000","last_became_peered":"2024-09-16T00:07:33.619981+0000","last_unstale":"2024-09-16T00:11:28.368837+0000","last_undegraded":"2024-09-16T00:11:28.368837+0000","last_fullsized":"2024-09-16T00:11:28.368837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'399","last_scrub_stamp":"2024-09-16T00:11:28.368683+0000","last_deep_scrub":"146'399","last_deep_scrub_stamp":"2024-09-16T00:11:28.368683+0000","last_clean_scrub_stamp":"2024-09-16T00:11:28.368683+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:48.308695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":14230,"num_write":362,"num_write_kb":33768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"146'448","reported_seq":774,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:32.727488+0000","last_change":"2024-09-16T00:11:32.727515+0000","last_active":"2024-09-16T00:11:32.727488+0000","last_peered":"2024-09-16T00:11:32.727488+0000","last_clean":"2024-09-16T00:11:32.727488+0000","last_became_active":"2024-09-16T00:07:33.627619+0000","last_became_peered":"2024-09-16T00:07:33.627619+0000","last_unstale":"2024-09-16T00:11:32.727488+0000","last_undegraded":"2024-09-16T00:11:32.727488+0000","last_fullsized":"2024-09-16T00:11:32.727488+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'448","last_scrub_stamp":"2024-09-16T00:11:32.727396+0000","last_deep_scrub":"146'448","last_deep_scrub_stamp":"2024-09-16T00:11:32.727396+0000","last_clean_scrub_stamp":"2024-09-16T00:11:32.727396+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:23:49.081392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":40234,"num_write":414,"num_write_kb":67785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"146'402","reported_seq":907,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:38.719436+0000","last_change":"2024-09-16T00:11:38.719470+0000","last_active":"2024-09-16T00:11:38.719436+0000","last_peered":"2024-09-16T00:11:38.719436+0000","last_clean":"2024-09-16T00:11:38.719436+0000","last_became_active":"2024-09-16T00:10:55.845880+0000","last_became_peered":"2024-09-16T00:10:55.845880+0000","last_unstale":"2024-09-16T00:11:38.719436+0000","last_undegraded":"2024-09-16T00:11:38.719436+0000","last_fullsized":"2024-09-16T00:11:38.719436+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'402","last_scrub_stamp":"2024-09-16T00:11:38.719338+0000","last_deep_scrub":"146'402","last_deep_scrub_stamp":"2024-09-16T00:11:38.719338+0000","last_clean_scrub_stamp":"2024-09-16T00:11:38.719338+0000","objects_scrubbed":0,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:59:39.735521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":22524,"num_write":374,"num_write_kb":46842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"146'401","reported_seq":739,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:31.353974+0000","last_change":"2024-09-16T00:11:31.353998+0000","last_active":"2024-09-16T00:11:31.353974+0000","last_peered":"2024-09-16T00:11:31.353974+0000","last_clean":"2024-09-16T00:11:31.353974+0000","last_became_active":"2024-09-16T00:10:00.282592+0000","last_became_peered":"2024-09-16T00:10:00.282592+0000","last_unstale":"2024-09-16T00:11:31.353974+0000","last_undegraded":"2024-09-16T00:11:31.353974+0000","last_fullsized":"2024-09-16T00:11:31.353974+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'401","last_scrub_stamp":"2024-09-16T00:11:31.353888+0000","last_deep_scrub":"146'401","last_deep_scrub_stamp":"2024-09-16T00:11:31.353888+0000","last_clean_scrub_stamp":"2024-09-16T00:11:31.353888+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:35.907404+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8711,"num_write":366,"num_write_kb":46855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":21285626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"146'424","reported_seq":843,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:35.064563+0000","last_change":"2024-09-16T00:11:35.064591+0000","last_active":"2024-09-16T00:11:35.064563+0000","last_peered":"2024-09-16T00:11:35.064563+0000","last_clean":"2024-09-16T00:11:35.064563+0000","last_became_active":"2024-09-16T00:10:55.832595+0000","last_became_peered":"2024-09-16T00:10:55.832595+0000","last_unstale":"2024-09-16T00:11:35.064563+0000","last_undegraded":"2024-09-16T00:11:35.064563+0000","last_fullsized":"2024-09-16T00:11:35.064563+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'424","last_scrub_stamp":"2024-09-16T00:11:35.064476+0000","last_deep_scrub":"146'424","last_deep_scrub_stamp":"2024-09-16T00:11:35.064476+0000","last_clean_scrub_stamp":"2024-09-16T00:11:35.064476+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:31:14.280270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":14437,"num_write":396,"num_write_kb":83979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":103497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"146'431","reported_seq":619,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:31.508535+0000","last_change":"2024-09-16T00:11:31.508571+0000","last_active":"2024-09-16T00:11:31.508535+0000","last_peered":"2024-09-16T00:11:31.508535+0000","last_clean":"2024-09-16T00:11:31.508535+0000","last_became_active":"2024-09-16T00:10:00.281258+0000","last_became_peered":"2024-09-16T00:10:00.281258+0000","last_unstale":"2024-09-16T00:11:31.508535+0000","last_undegraded":"2024-09-16T00:11:31.508535+0000","last_fullsized":"2024-09-16T00:11:31.508535+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'431","last_scrub_stamp":"2024-09-16T00:11:31.508451+0000","last_deep_scrub":"146'431","last_deep_scrub_stamp":"2024-09-16T00:11:31.508451+0000","last_clean_scrub_stamp":"2024-09-16T00:11:31.508451+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:09.852867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":15502,"num_write":406,"num_write_kb":30222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":17199573,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"146'415","reported_seq":903,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:32.743538+0000","last_change":"2024-09-16T00:11:32.743563+0000","last_active":"2024-09-16T00:11:32.743538+0000","last_peered":"2024-09-16T00:11:32.743538+0000","last_clean":"2024-09-16T00:11:32.743538+0000","last_became_active":"2024-09-16T00:10:55.835150+0000","last_became_peered":"2024-09-16T00:10:55.835150+0000","last_unstale":"2024-09-16T00:11:32.743538+0000","last_undegraded":"2024-09-16T00:11:32.743538+0000","last_fullsized":"2024-09-16T00:11:32.743538+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'415","last_scrub_stamp":"2024-09-16T00:11:32.743462+0000","last_deep_scrub":"146'415","last_deep_scrub_stamp":"2024-09-16T00:11:32.743462+0000","last_clean_scrub_stamp":"2024-09-16T00:11:32.743462+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:33.883475+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":34318,"num_write":385,"num_write_kb":91769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"146'454","reported_seq":937,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:36.768924+0000","last_change":"2024-09-16T00:11:36.768950+0000","last_active":"2024-09-16T00:11:36.768924+0000","last_peered":"2024-09-16T00:11:36.768924+0000","last_clean":"2024-09-16T00:11:36.768924+0000","last_became_active":"2024-09-16T00:10:55.833335+0000","last_became_peered":"2024-09-16T00:10:55.833335+0000","last_unstale":"2024-09-16T00:11:36.768924+0000","last_undegraded":"2024-09-16T00:11:36.768924+0000","last_fullsized":"2024-09-16T00:11:36.768924+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'454","last_scrub_stamp":"2024-09-16T00:11:36.768838+0000","last_deep_scrub":"146'454","last_deep_scrub_stamp":"2024-09-16T00:11:36.768838+0000","last_clean_scrub_stamp":"2024-09-16T00:11:36.768838+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:07.955079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":35528,"num_write":422,"num_write_kb":92852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"146'403","reported_seq":700,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:26.759837+0000","last_change":"2024-09-16T00:11:26.759863+0000","last_active":"2024-09-16T00:11:26.759837+0000","last_peered":"2024-09-16T00:11:26.759837+0000","last_clean":"2024-09-16T00:11:26.759837+0000","last_became_active":"2024-09-16T00:07:33.632129+0000","last_became_peered":"2024-09-16T00:07:33.632129+0000","last_unstale":"2024-09-16T00:11:26.759837+0000","last_undegraded":"2024-09-16T00:11:26.759837+0000","last_fullsized":"2024-09-16T00:11:26.759837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'403","last_scrub_stamp":"2024-09-16T00:11:26.759743+0000","last_deep_scrub":"146'403","last_deep_scrub_stamp":"2024-09-16T00:11:26.759743+0000","last_clean_scrub_stamp":"2024-09-16T00:11:26.759743+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:19:16.661570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":13410,"num_write":369,"num_write_kb":58805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"146'427","reported_seq":718,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:30.742086+0000","last_change":"2024-09-16T00:11:30.742109+0000","last_active":"2024-09-16T00:11:30.742086+0000","last_peered":"2024-09-16T00:11:30.742086+0000","last_clean":"2024-09-16T00:11:30.742086+0000","last_became_active":"2024-09-16T00:07:33.626748+0000","last_became_peered":"2024-09-16T00:07:33.626748+0000","last_unstale":"2024-09-16T00:11:30.742086+0000","last_undegraded":"2024-09-16T00:11:30.742086+0000","last_fullsized":"2024-09-16T00:11:30.742086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'427","last_scrub_stamp":"2024-09-16T00:11:30.742007+0000","last_deep_scrub":"146'427","last_deep_scrub_stamp":"2024-09-16T00:11:30.742007+0000","last_clean_scrub_stamp":"2024-09-16T00:11:30.742007+0000","objects_scrubbed":0,"log_size":427,"log_dups_size":0,"ondisk_log_size":427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:41:30.603248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":12682,"num_write":399,"num_write_kb":54869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"146'423","reported_seq":723,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:29.671214+0000","last_change":"2024-09-16T00:11:29.671237+0000","last_active":"2024-09-16T00:11:29.671214+0000","last_peered":"2024-09-16T00:11:29.671214+0000","last_clean":"2024-09-16T00:11:29.671214+0000","last_became_active":"2024-09-16T00:07:33.634703+0000","last_became_peered":"2024-09-16T00:07:33.634703+0000","last_unstale":"2024-09-16T00:11:29.671214+0000","last_undegraded":"2024-09-16T00:11:29.671214+0000","last_fullsized":"2024-09-16T00:11:29.671214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'423","last_scrub_stamp":"2024-09-16T00:11:29.671132+0000","last_deep_scrub":"146'423","last_deep_scrub_stamp":"2024-09-16T00:11:29.671132+0000","last_clean_scrub_stamp":"2024-09-16T00:11:29.671132+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:22.598678+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":14188,"num_write":408,"num_write_kb":59178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"146'463","reported_seq":886,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:36.603974+0000","last_change":"2024-09-16T00:11:36.604001+0000","last_active":"2024-09-16T00:11:36.603974+0000","last_peered":"2024-09-16T00:11:36.603974+0000","last_clean":"2024-09-16T00:11:36.603974+0000","last_became_active":"2024-09-16T00:10:55.841967+0000","last_became_peered":"2024-09-16T00:10:55.841967+0000","last_unstale":"2024-09-16T00:11:36.603974+0000","last_undegraded":"2024-09-16T00:11:36.603974+0000","last_fullsized":"2024-09-16T00:11:36.603974+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'463","last_scrub_stamp":"2024-09-16T00:11:36.603886+0000","last_deep_scrub":"146'463","last_deep_scrub_stamp":"2024-09-16T00:11:36.603886+0000","last_clean_scrub_stamp":"2024-09-16T00:11:36.603886+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:29:27.345718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":12330,"num_write":421,"num_write_kb":35146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":12903786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"146'408","reported_seq":1067,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:34.436108+0000","last_change":"2024-09-16T00:11:34.436150+0000","last_active":"2024-09-16T00:11:34.436108+0000","last_peered":"2024-09-16T00:11:34.436108+0000","last_clean":"2024-09-16T00:11:34.436108+0000","last_became_active":"2024-09-16T00:11:04.892794+0000","last_became_peered":"2024-09-16T00:11:04.892794+0000","last_unstale":"2024-09-16T00:11:34.436108+0000","last_undegraded":"2024-09-16T00:11:34.436108+0000","last_fullsized":"2024-09-16T00:11:34.436108+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'408","last_scrub_stamp":"2024-09-16T00:11:34.435972+0000","last_deep_scrub":"146'408","last_deep_scrub_stamp":"2024-09-16T00:11:34.435972+0000","last_clean_scrub_stamp":"2024-09-16T00:11:34.435972+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:13.343385+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":21104,"num_write":383,"num_write_kb":75742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":628439,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"146'390","reported_seq":701,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:28.696379+0000","last_change":"2024-09-16T00:11:28.696405+0000","last_active":"2024-09-16T00:11:28.696379+0000","last_peered":"2024-09-16T00:11:28.696379+0000","last_clean":"2024-09-16T00:11:28.696379+0000","last_became_active":"2024-09-16T00:07:33.634659+0000","last_became_peered":"2024-09-16T00:07:33.634659+0000","last_unstale":"2024-09-16T00:11:28.696379+0000","last_undegraded":"2024-09-16T00:11:28.696379+0000","last_fullsized":"2024-09-16T00:11:28.696379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'390","last_scrub_stamp":"2024-09-16T00:11:28.696296+0000","last_deep_scrub":"146'390","last_deep_scrub_stamp":"2024-09-16T00:11:28.696296+0000","last_clean_scrub_stamp":"2024-09-16T00:11:28.696296+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:11:42.242907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":16408,"num_write":366,"num_write_kb":51674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"146'405","reported_seq":733,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:25.762325+0000","last_change":"2024-09-16T00:11:25.762349+0000","last_active":"2024-09-16T00:11:25.762325+0000","last_peered":"2024-09-16T00:11:25.762325+0000","last_clean":"2024-09-16T00:11:25.762325+0000","last_became_active":"2024-09-16T00:07:33.625663+0000","last_became_peered":"2024-09-16T00:07:33.625663+0000","last_unstale":"2024-09-16T00:11:25.762325+0000","last_undegraded":"2024-09-16T00:11:25.762325+0000","last_fullsized":"2024-09-16T00:11:25.762325+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'405","last_scrub_stamp":"2024-09-16T00:11:25.762263+0000","last_deep_scrub":"146'405","last_deep_scrub_stamp":"2024-09-16T00:11:25.762263+0000","last_clean_scrub_stamp":"2024-09-16T00:11:25.762263+0000","objects_scrubbed":0,"log_size":405,"log_dups_size":0,"ondisk_log_size":405,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:09.468020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":59,"num_read_kb":33174,"num_write":373,"num_write_kb":66849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"146'494","reported_seq":797,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:27.372381+0000","last_change":"2024-09-16T00:11:27.372403+0000","last_active":"2024-09-16T00:11:27.372381+0000","last_peered":"2024-09-16T00:11:27.372381+0000","last_clean":"2024-09-16T00:11:27.372381+0000","last_became_active":"2024-09-16T00:08:15.790284+0000","last_became_peered":"2024-09-16T00:08:15.790284+0000","last_unstale":"2024-09-16T00:11:27.372381+0000","last_undegraded":"2024-09-16T00:11:27.372381+0000","last_fullsized":"2024-09-16T00:11:27.372381+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'494","last_scrub_stamp":"2024-09-16T00:11:27.372282+0000","last_deep_scrub":"146'494","last_deep_scrub_stamp":"2024-09-16T00:11:27.372282+0000","last_clean_scrub_stamp":"2024-09-16T00:11:27.372282+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:17:44.798062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":27231,"num_write":453,"num_write_kb":104771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"146'463","reported_seq":810,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:28.369119+0000","last_change":"2024-09-16T00:11:28.369152+0000","last_active":"2024-09-16T00:11:28.369119+0000","last_peered":"2024-09-16T00:11:28.369119+0000","last_clean":"2024-09-16T00:11:28.369119+0000","last_became_active":"2024-09-16T00:08:15.792302+0000","last_became_peered":"2024-09-16T00:08:15.792302+0000","last_unstale":"2024-09-16T00:11:28.369119+0000","last_undegraded":"2024-09-16T00:11:28.369119+0000","last_fullsized":"2024-09-16T00:11:28.369119+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'463","last_scrub_stamp":"2024-09-16T00:11:28.369001+0000","last_deep_scrub":"146'463","last_deep_scrub_stamp":"2024-09-16T00:11:28.369001+0000","last_clean_scrub_stamp":"2024-09-16T00:11:28.369001+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:50.559905+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":35655,"num_write":431,"num_write_kb":97245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"146'430","reported_seq":620,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:30.556725+0000","last_change":"2024-09-16T00:11:30.556752+0000","last_active":"2024-09-16T00:11:30.556725+0000","last_peered":"2024-09-16T00:11:30.556725+0000","last_clean":"2024-09-16T00:11:30.556725+0000","last_became_active":"2024-09-16T00:10:00.281378+0000","last_became_peered":"2024-09-16T00:10:00.281378+0000","last_unstale":"2024-09-16T00:11:30.556725+0000","last_undegraded":"2024-09-16T00:11:30.556725+0000","last_fullsized":"2024-09-16T00:11:30.556725+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'430","last_scrub_stamp":"2024-09-16T00:11:30.556631+0000","last_deep_scrub":"146'430","last_deep_scrub_stamp":"2024-09-16T00:11:30.556631+0000","last_clean_scrub_stamp":"2024-09-16T00:11:30.556631+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:40:02.011016+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20395,"num_write":401,"num_write_kb":32731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":22876917,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"146'408","reported_seq":757,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:27.271326+0000","last_change":"2024-09-16T00:11:27.271378+0000","last_active":"2024-09-16T00:11:27.271326+0000","last_peered":"2024-09-16T00:11:27.271326+0000","last_clean":"2024-09-16T00:11:27.271326+0000","last_became_active":"2024-09-16T00:07:33.631698+0000","last_became_peered":"2024-09-16T00:07:33.631698+0000","last_unstale":"2024-09-16T00:11:27.271326+0000","last_undegraded":"2024-09-16T00:11:27.271326+0000","last_fullsized":"2024-09-16T00:11:27.271326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'408","last_scrub_stamp":"2024-09-16T00:11:27.271213+0000","last_deep_scrub":"146'408","last_deep_scrub_stamp":"2024-09-16T00:11:27.271213+0000","last_clean_scrub_stamp":"2024-09-16T00:11:27.271213+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:55:12.896737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":48141,"num_write":381,"num_write_kb":63611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"146'127","reported_seq":412,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:33.718046+0000","last_change":"2024-09-16T00:11:33.718075+0000","last_active":"2024-09-16T00:11:33.718046+0000","last_peered":"2024-09-16T00:11:33.718046+0000","last_clean":"2024-09-16T00:11:33.718046+0000","last_became_active":"2024-09-16T00:10:55.843367+0000","last_became_peered":"2024-09-16T00:10:55.843367+0000","last_unstale":"2024-09-16T00:11:33.718046+0000","last_undegraded":"2024-09-16T00:11:33.718046+0000","last_fullsized":"2024-09-16T00:11:33.718046+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'127","last_scrub_stamp":"2024-09-16T00:11:33.717959+0000","last_deep_scrub":"146'127","last_deep_scrub_stamp":"2024-09-16T00:11:33.717959+0000","last_clean_scrub_stamp":"2024-09-16T00:11:33.717959+0000","objects_scrubbed":1,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:15.038001+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66,"num_write_kb":4147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"146'354","reported_seq":650,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:29.782687+0000","last_change":"2024-09-16T00:11:29.782714+0000","last_active":"2024-09-16T00:11:29.782687+0000","last_peered":"2024-09-16T00:11:29.782687+0000","last_clean":"2024-09-16T00:11:29.782687+0000","last_became_active":"2024-09-16T00:07:33.635626+0000","last_became_peered":"2024-09-16T00:07:33.635626+0000","last_unstale":"2024-09-16T00:11:29.782687+0000","last_undegraded":"2024-09-16T00:11:29.782687+0000","last_fullsized":"2024-09-16T00:11:29.782687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'354","last_scrub_stamp":"2024-09-16T00:11:29.782606+0000","last_deep_scrub":"146'354","last_deep_scrub_stamp":"2024-09-16T00:11:29.782606+0000","last_clean_scrub_stamp":"2024-09-16T00:11:29.782606+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:59:06.677976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":16420,"num_write":326,"num_write_kb":62710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"146'343","reported_seq":517,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:25.645782+0000","last_change":"2024-09-16T00:11:25.645813+0000","last_active":"2024-09-16T00:11:25.645782+0000","last_peered":"2024-09-16T00:11:25.645782+0000","last_clean":"2024-09-16T00:11:25.645782+0000","last_became_active":"2024-09-16T00:10:00.282541+0000","last_became_peered":"2024-09-16T00:10:00.282541+0000","last_unstale":"2024-09-16T00:11:25.645782+0000","last_undegraded":"2024-09-16T00:11:25.645782+0000","last_fullsized":"2024-09-16T00:11:25.645782+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'343","last_scrub_stamp":"2024-09-16T00:11:25.645694+0000","last_deep_scrub":"146'343","last_deep_scrub_stamp":"2024-09-16T00:11:25.645694+0000","last_clean_scrub_stamp":"2024-09-16T00:11:25.645694+0000","objects_scrubbed":3,"log_size":343,"log_dups_size":0,"ondisk_log_size":343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:41.964965+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":285,"num_write_kb":12471,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"146'227","reported_seq":520,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:30.712348+0000","last_change":"2024-09-16T00:11:30.712377+0000","last_active":"2024-09-16T00:11:30.712348+0000","last_peered":"2024-09-16T00:11:30.712348+0000","last_clean":"2024-09-16T00:11:30.712348+0000","last_became_active":"2024-09-16T00:10:00.284602+0000","last_became_peered":"2024-09-16T00:10:00.284602+0000","last_unstale":"2024-09-16T00:11:30.712348+0000","last_undegraded":"2024-09-16T00:11:30.712348+0000","last_fullsized":"2024-09-16T00:11:30.712348+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'227","last_scrub_stamp":"2024-09-16T00:11:30.712251+0000","last_deep_scrub":"146'227","last_deep_scrub_stamp":"2024-09-16T00:11:30.712251+0000","last_clean_scrub_stamp":"2024-09-16T00:11:30.712251+0000","objects_scrubbed":1,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:20.855656+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":181,"num_write_kb":4224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":14,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"146'441","reported_seq":749,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:24.764032+0000","last_change":"2024-09-16T00:11:24.764060+0000","last_active":"2024-09-16T00:11:24.764032+0000","last_peered":"2024-09-16T00:11:24.764032+0000","last_clean":"2024-09-16T00:11:24.764032+0000","last_became_active":"2024-09-16T00:07:33.625641+0000","last_became_peered":"2024-09-16T00:07:33.625641+0000","last_unstale":"2024-09-16T00:11:24.764032+0000","last_undegraded":"2024-09-16T00:11:24.764032+0000","last_fullsized":"2024-09-16T00:11:24.764032+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'441","last_scrub_stamp":"2024-09-16T00:11:24.763970+0000","last_deep_scrub":"146'441","last_deep_scrub_stamp":"2024-09-16T00:11:24.763970+0000","last_clean_scrub_stamp":"2024-09-16T00:11:24.763970+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:15:54.261841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":38,"num_read_kb":18480,"num_write":411,"num_write_kb":71748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"146'835","reported_seq":1106,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:24.839247+0000","last_change":"2024-09-16T00:11:24.839281+0000","last_active":"2024-09-16T00:11:24.839247+0000","last_peered":"2024-09-16T00:11:24.839247+0000","last_clean":"2024-09-16T00:11:24.839247+0000","last_became_active":"2024-09-16T00:07:32.620764+0000","last_became_peered":"2024-09-16T00:07:32.620764+0000","last_unstale":"2024-09-16T00:11:24.839247+0000","last_undegraded":"2024-09-16T00:11:24.839247+0000","last_fullsized":"2024-09-16T00:11:24.839247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'835","last_scrub_stamp":"2024-09-16T00:11:24.839144+0000","last_deep_scrub":"146'835","last_deep_scrub_stamp":"2024-09-16T00:11:24.839144+0000","last_clean_scrub_stamp":"2024-09-16T00:11:24.839144+0000","objects_scrubbed":2,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:44.779155+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":8591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"146'435","reported_seq":793,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:29.365687+0000","last_change":"2024-09-16T00:11:29.365711+0000","last_active":"2024-09-16T00:11:29.365687+0000","last_peered":"2024-09-16T00:11:29.365687+0000","last_clean":"2024-09-16T00:11:29.365687+0000","last_became_active":"2024-09-16T00:07:33.625119+0000","last_became_peered":"2024-09-16T00:07:33.625119+0000","last_unstale":"2024-09-16T00:11:29.365687+0000","last_undegraded":"2024-09-16T00:11:29.365687+0000","last_fullsized":"2024-09-16T00:11:29.365687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'435","last_scrub_stamp":"2024-09-16T00:11:29.365604+0000","last_deep_scrub":"146'435","last_deep_scrub_stamp":"2024-09-16T00:11:29.365604+0000","last_clean_scrub_stamp":"2024-09-16T00:11:29.365604+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:59:06.260974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":46871,"num_write":401,"num_write_kb":84187,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"123'65","reported_seq":305,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:26.582789+0000","last_change":"2024-09-16T00:11:26.582832+0000","last_active":"2024-09-16T00:11:26.582789+0000","last_peered":"2024-09-16T00:11:26.582789+0000","last_clean":"2024-09-16T00:11:26.582789+0000","last_became_active":"2024-09-16T00:10:00.283555+0000","last_became_peered":"2024-09-16T00:10:00.283555+0000","last_unstale":"2024-09-16T00:11:26.582789+0000","last_undegraded":"2024-09-16T00:11:26.582789+0000","last_fullsized":"2024-09-16T00:11:26.582789+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"123'65","last_scrub_stamp":"2024-09-16T00:11:26.582667+0000","last_deep_scrub":"123'65","last_deep_scrub_stamp":"2024-09-16T00:11:26.582667+0000","last_clean_scrub_stamp":"2024-09-16T00:11:26.582667+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:00.545904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"146'450","reported_seq":923,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:37.765343+0000","last_change":"2024-09-16T00:11:37.765371+0000","last_active":"2024-09-16T00:11:37.765343+0000","last_peered":"2024-09-16T00:11:37.765343+0000","last_clean":"2024-09-16T00:11:37.765343+0000","last_became_active":"2024-09-16T00:10:55.840027+0000","last_became_peered":"2024-09-16T00:10:55.840027+0000","last_unstale":"2024-09-16T00:11:37.765343+0000","last_undegraded":"2024-09-16T00:11:37.765343+0000","last_fullsized":"2024-09-16T00:11:37.765343+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'450","last_scrub_stamp":"2024-09-16T00:11:37.765247+0000","last_deep_scrub":"146'450","last_deep_scrub_stamp":"2024-09-16T00:11:37.765247+0000","last_clean_scrub_stamp":"2024-09-16T00:11:37.765247+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:12:01.886039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":16686,"num_write":415,"num_write_kb":80112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"146'226","reported_seq":504,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:35.759775+0000","last_change":"2024-09-16T00:11:35.759808+0000","last_active":"2024-09-16T00:11:35.759775+0000","last_peered":"2024-09-16T00:11:35.759775+0000","last_clean":"2024-09-16T00:11:35.759775+0000","last_became_active":"2024-09-16T00:10:00.284717+0000","last_became_peered":"2024-09-16T00:10:00.284717+0000","last_unstale":"2024-09-16T00:11:35.759775+0000","last_undegraded":"2024-09-16T00:11:35.759775+0000","last_fullsized":"2024-09-16T00:11:35.759775+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'226","last_scrub_stamp":"2024-09-16T00:11:35.759687+0000","last_deep_scrub":"146'226","last_deep_scrub_stamp":"2024-09-16T00:11:35.759687+0000","last_clean_scrub_stamp":"2024-09-16T00:11:35.759687+0000","objects_scrubbed":1,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:44:22.614836+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":168,"num_write_kb":4181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"146'387","reported_seq":699,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:28.769728+0000","last_change":"2024-09-16T00:11:28.769814+0000","last_active":"2024-09-16T00:11:28.769728+0000","last_peered":"2024-09-16T00:11:28.769728+0000","last_clean":"2024-09-16T00:11:28.769728+0000","last_became_active":"2024-09-16T00:07:33.633705+0000","last_became_peered":"2024-09-16T00:07:33.633705+0000","last_unstale":"2024-09-16T00:11:28.769728+0000","last_undegraded":"2024-09-16T00:11:28.769728+0000","last_fullsized":"2024-09-16T00:11:28.769728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'387","last_scrub_stamp":"2024-09-16T00:11:28.769598+0000","last_deep_scrub":"146'387","last_deep_scrub_stamp":"2024-09-16T00:11:28.769598+0000","last_clean_scrub_stamp":"2024-09-16T00:11:28.769598+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:48.709609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":18030,"num_write":359,"num_write_kb":55428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"146'133","reported_seq":403,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:23.307945+0000","last_change":"2024-09-16T00:11:23.307972+0000","last_active":"2024-09-16T00:11:23.307945+0000","last_peered":"2024-09-16T00:11:23.307945+0000","last_clean":"2024-09-16T00:11:23.307945+0000","last_became_active":"2024-09-16T00:07:32.616729+0000","last_became_peered":"2024-09-16T00:07:32.616729+0000","last_unstale":"2024-09-16T00:11:23.307945+0000","last_undegraded":"2024-09-16T00:11:23.307945+0000","last_fullsized":"2024-09-16T00:11:23.307945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'133","last_scrub_stamp":"2024-09-16T00:11:23.307873+0000","last_deep_scrub":"146'133","last_deep_scrub_stamp":"2024-09-16T00:11:23.307873+0000","last_clean_scrub_stamp":"2024-09-16T00:11:23.307873+0000","objects_scrubbed":2,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:02:52.915217+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":81,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"146'384","reported_seq":838,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:29.208959+0000","last_change":"2024-09-16T00:11:29.208985+0000","last_active":"2024-09-16T00:11:29.208959+0000","last_peered":"2024-09-16T00:11:29.208959+0000","last_clean":"2024-09-16T00:11:29.208959+0000","last_became_active":"2024-09-16T00:10:55.833273+0000","last_became_peered":"2024-09-16T00:10:55.833273+0000","last_unstale":"2024-09-16T00:11:29.208959+0000","last_undegraded":"2024-09-16T00:11:29.208959+0000","last_fullsized":"2024-09-16T00:11:29.208959+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'384","last_scrub_stamp":"2024-09-16T00:11:29.208870+0000","last_deep_scrub":"146'384","last_deep_scrub_stamp":"2024-09-16T00:11:29.208870+0000","last_clean_scrub_stamp":"2024-09-16T00:11:29.208870+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:48.712050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":22960,"num_write":348,"num_write_kb":63161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"146'245","reported_seq":430,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:28.606802+0000","last_change":"2024-09-16T00:11:28.606836+0000","last_active":"2024-09-16T00:11:28.606802+0000","last_peered":"2024-09-16T00:11:28.606802+0000","last_clean":"2024-09-16T00:11:28.606802+0000","last_became_active":"2024-09-16T00:10:00.283474+0000","last_became_peered":"2024-09-16T00:10:00.283474+0000","last_unstale":"2024-09-16T00:11:28.606802+0000","last_undegraded":"2024-09-16T00:11:28.606802+0000","last_fullsized":"2024-09-16T00:11:28.606802+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'245","last_scrub_stamp":"2024-09-16T00:11:28.606724+0000","last_deep_scrub":"146'245","last_deep_scrub_stamp":"2024-09-16T00:11:28.606724+0000","last_clean_scrub_stamp":"2024-09-16T00:11:28.606724+0000","objects_scrubbed":6,"log_size":245,"log_dups_size":0,"ondisk_log_size":245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:38:18.282645+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10146145,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":189,"num_write_kb":10017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":13200,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"146'409","reported_seq":791,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:38.092688+0000","last_change":"2024-09-16T00:11:38.092715+0000","last_active":"2024-09-16T00:11:38.092688+0000","last_peered":"2024-09-16T00:11:38.092688+0000","last_clean":"2024-09-16T00:11:38.092688+0000","last_became_active":"2024-09-16T00:10:55.836462+0000","last_became_peered":"2024-09-16T00:10:55.836462+0000","last_unstale":"2024-09-16T00:11:38.092688+0000","last_undegraded":"2024-09-16T00:11:38.092688+0000","last_fullsized":"2024-09-16T00:11:38.092688+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'409","last_scrub_stamp":"2024-09-16T00:11:38.092601+0000","last_deep_scrub":"146'409","last_deep_scrub_stamp":"2024-09-16T00:11:38.092601+0000","last_clean_scrub_stamp":"2024-09-16T00:11:38.092601+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:17:06.905525+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":23285,"num_write":372,"num_write_kb":67093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":88,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"146'474","reported_seq":745,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:22.726770+0000","last_change":"2024-09-16T00:11:22.726799+0000","last_active":"2024-09-16T00:11:22.726770+0000","last_peered":"2024-09-16T00:11:22.726770+0000","last_clean":"2024-09-16T00:11:22.726770+0000","last_became_active":"2024-09-16T00:07:32.629053+0000","last_became_peered":"2024-09-16T00:07:32.629053+0000","last_unstale":"2024-09-16T00:11:22.726770+0000","last_undegraded":"2024-09-16T00:11:22.726770+0000","last_fullsized":"2024-09-16T00:11:22.726770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'474","last_scrub_stamp":"2024-09-16T00:11:22.726696+0000","last_deep_scrub":"146'474","last_deep_scrub_stamp":"2024-09-16T00:11:22.726696+0000","last_clean_scrub_stamp":"2024-09-16T00:11:22.726696+0000","objects_scrubbed":4,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:55.750436+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":419,"num_write_kb":8413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"146'413","reported_seq":890,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:31.761797+0000","last_change":"2024-09-16T00:11:31.761827+0000","last_active":"2024-09-16T00:11:31.761797+0000","last_peered":"2024-09-16T00:11:31.761797+0000","last_clean":"2024-09-16T00:11:31.761797+0000","last_became_active":"2024-09-16T00:10:55.839196+0000","last_became_peered":"2024-09-16T00:10:55.839196+0000","last_unstale":"2024-09-16T00:11:31.761797+0000","last_undegraded":"2024-09-16T00:11:31.761797+0000","last_fullsized":"2024-09-16T00:11:31.761797+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'413","last_scrub_stamp":"2024-09-16T00:11:31.761695+0000","last_deep_scrub":"146'413","last_deep_scrub_stamp":"2024-09-16T00:11:31.761695+0000","last_clean_scrub_stamp":"2024-09-16T00:11:31.761695+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:58:23.101516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":13263,"num_write":381,"num_write_kb":51174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"123'59","reported_seq":330,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:27.755986+0000","last_change":"2024-09-16T00:11:27.756038+0000","last_active":"2024-09-16T00:11:27.755986+0000","last_peered":"2024-09-16T00:11:27.755986+0000","last_clean":"2024-09-16T00:11:27.755986+0000","last_became_active":"2024-09-16T00:07:32.625492+0000","last_became_peered":"2024-09-16T00:07:32.625492+0000","last_unstale":"2024-09-16T00:11:27.755986+0000","last_undegraded":"2024-09-16T00:11:27.755986+0000","last_fullsized":"2024-09-16T00:11:27.755986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"123'59","last_scrub_stamp":"2024-09-16T00:11:27.755884+0000","last_deep_scrub":"123'59","last_deep_scrub_stamp":"2024-09-16T00:11:27.755884+0000","last_clean_scrub_stamp":"2024-09-16T00:11:27.755884+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:02.948201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"146'409","reported_seq":759,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:26.037167+0000","last_change":"2024-09-16T00:11:26.037210+0000","last_active":"2024-09-16T00:11:26.037167+0000","last_peered":"2024-09-16T00:11:26.037167+0000","last_clean":"2024-09-16T00:11:26.037167+0000","last_became_active":"2024-09-16T00:10:55.834112+0000","last_became_peered":"2024-09-16T00:10:55.834112+0000","last_unstale":"2024-09-16T00:11:26.037167+0000","last_undegraded":"2024-09-16T00:11:26.037167+0000","last_fullsized":"2024-09-16T00:11:26.037167+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'409","last_scrub_stamp":"2024-09-16T00:11:26.037036+0000","last_deep_scrub":"146'409","last_deep_scrub_stamp":"2024-09-16T00:11:26.037036+0000","last_clean_scrub_stamp":"2024-09-16T00:11:26.037036+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:02:34.098111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":13990,"num_write":368,"num_write_kb":63962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"120'70","reported_seq":227,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:27.544444+0000","last_change":"2024-09-16T00:11:27.544512+0000","last_active":"2024-09-16T00:11:27.544444+0000","last_peered":"2024-09-16T00:11:27.544444+0000","last_clean":"2024-09-16T00:11:27.544444+0000","last_became_active":"2024-09-16T00:10:00.283066+0000","last_became_peered":"2024-09-16T00:10:00.283066+0000","last_unstale":"2024-09-16T00:11:27.544444+0000","last_undegraded":"2024-09-16T00:11:27.544444+0000","last_fullsized":"2024-09-16T00:11:27.544444+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"120'70","last_scrub_stamp":"2024-09-16T00:11:27.544334+0000","last_deep_scrub":"120'70","last_deep_scrub_stamp":"2024-09-16T00:11:27.544334+0000","last_clean_scrub_stamp":"2024-09-16T00:11:27.544334+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:19.167083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"146'430","reported_seq":622,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:29.573627+0000","last_change":"2024-09-16T00:11:29.573655+0000","last_active":"2024-09-16T00:11:29.573627+0000","last_peered":"2024-09-16T00:11:29.573627+0000","last_clean":"2024-09-16T00:11:29.573627+0000","last_became_active":"2024-09-16T00:10:00.280750+0000","last_became_peered":"2024-09-16T00:10:00.280750+0000","last_unstale":"2024-09-16T00:11:29.573627+0000","last_undegraded":"2024-09-16T00:11:29.573627+0000","last_fullsized":"2024-09-16T00:11:29.573627+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'430","last_scrub_stamp":"2024-09-16T00:11:29.573541+0000","last_deep_scrub":"146'430","last_deep_scrub_stamp":"2024-09-16T00:11:29.573541+0000","last_clean_scrub_stamp":"2024-09-16T00:11:29.573541+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:10:27.629256+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":19573,"num_write":402,"num_write_kb":50826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":33491520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"146'76","reported_seq":347,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:23.465739+0000","last_change":"2024-09-16T00:11:23.465779+0000","last_active":"2024-09-16T00:11:23.465739+0000","last_peered":"2024-09-16T00:11:23.465739+0000","last_clean":"2024-09-16T00:11:23.465739+0000","last_became_active":"2024-09-16T00:07:32.622724+0000","last_became_peered":"2024-09-16T00:07:32.622724+0000","last_unstale":"2024-09-16T00:11:23.465739+0000","last_undegraded":"2024-09-16T00:11:23.465739+0000","last_fullsized":"2024-09-16T00:11:23.465739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'76","last_scrub_stamp":"2024-09-16T00:11:23.465674+0000","last_deep_scrub":"146'76","last_deep_scrub_stamp":"2024-09-16T00:11:23.465674+0000","last_clean_scrub_stamp":"2024-09-16T00:11:23.465674+0000","objects_scrubbed":4,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:11.533426+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"146'463","reported_seq":831,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:32.006547+0000","last_change":"2024-09-16T00:11:32.006599+0000","last_active":"2024-09-16T00:11:32.006547+0000","last_peered":"2024-09-16T00:11:32.006547+0000","last_clean":"2024-09-16T00:11:32.006547+0000","last_became_active":"2024-09-16T00:10:55.827989+0000","last_became_peered":"2024-09-16T00:10:55.827989+0000","last_unstale":"2024-09-16T00:11:32.006547+0000","last_undegraded":"2024-09-16T00:11:32.006547+0000","last_fullsized":"2024-09-16T00:11:32.006547+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'463","last_scrub_stamp":"2024-09-16T00:11:32.006446+0000","last_deep_scrub":"146'463","last_deep_scrub_stamp":"2024-09-16T00:11:32.006446+0000","last_clean_scrub_stamp":"2024-09-16T00:11:32.006446+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:48.690715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":32,"num_read_kb":20419,"num_write":427,"num_write_kb":43035,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"146'147","reported_seq":418,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:24.318825+0000","last_change":"2024-09-16T00:11:24.318878+0000","last_active":"2024-09-16T00:11:24.318825+0000","last_peered":"2024-09-16T00:11:24.318825+0000","last_clean":"2024-09-16T00:11:24.318825+0000","last_became_active":"2024-09-16T00:07:32.628162+0000","last_became_peered":"2024-09-16T00:07:32.628162+0000","last_unstale":"2024-09-16T00:11:24.318825+0000","last_undegraded":"2024-09-16T00:11:24.318825+0000","last_fullsized":"2024-09-16T00:11:24.318825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'147","last_scrub_stamp":"2024-09-16T00:11:24.318710+0000","last_deep_scrub":"146'147","last_deep_scrub_stamp":"2024-09-16T00:11:24.318710+0000","last_clean_scrub_stamp":"2024-09-16T00:11:24.318710+0000","objects_scrubbed":2,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:34.118902+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":91,"num_write_kb":4217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"146'363","reported_seq":710,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:33.051067+0000","last_change":"2024-09-16T00:11:33.051093+0000","last_active":"2024-09-16T00:11:33.051067+0000","last_peered":"2024-09-16T00:11:33.051067+0000","last_clean":"2024-09-16T00:11:33.051067+0000","last_became_active":"2024-09-16T00:10:55.834401+0000","last_became_peered":"2024-09-16T00:10:55.834401+0000","last_unstale":"2024-09-16T00:11:33.051067+0000","last_undegraded":"2024-09-16T00:11:33.051067+0000","last_fullsized":"2024-09-16T00:11:33.051067+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'363","last_scrub_stamp":"2024-09-16T00:11:33.050983+0000","last_deep_scrub":"146'363","last_deep_scrub_stamp":"2024-09-16T00:11:33.050983+0000","last_clean_scrub_stamp":"2024-09-16T00:11:33.050983+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:47.276713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":28,"num_read_kb":10790,"num_write":330,"num_write_kb":21585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"146'284","reported_seq":555,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:25.447480+0000","last_change":"2024-09-16T00:11:25.447508+0000","last_active":"2024-09-16T00:11:25.447480+0000","last_peered":"2024-09-16T00:11:25.447480+0000","last_clean":"2024-09-16T00:11:25.447480+0000","last_became_active":"2024-09-16T00:07:32.628448+0000","last_became_peered":"2024-09-16T00:07:32.628448+0000","last_unstale":"2024-09-16T00:11:25.447480+0000","last_undegraded":"2024-09-16T00:11:25.447480+0000","last_fullsized":"2024-09-16T00:11:25.447480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'284","last_scrub_stamp":"2024-09-16T00:11:25.447407+0000","last_deep_scrub":"146'284","last_deep_scrub_stamp":"2024-09-16T00:11:25.447407+0000","last_clean_scrub_stamp":"2024-09-16T00:11:25.447407+0000","objects_scrubbed":2,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:09:11.660233+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":238,"num_write_kb":4238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"146'381","reported_seq":716,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:36.035779+0000","last_change":"2024-09-16T00:11:36.035806+0000","last_active":"2024-09-16T00:11:36.035779+0000","last_peered":"2024-09-16T00:11:36.035779+0000","last_clean":"2024-09-16T00:11:36.035779+0000","last_became_active":"2024-09-16T00:10:55.838254+0000","last_became_peered":"2024-09-16T00:10:55.838254+0000","last_unstale":"2024-09-16T00:11:36.035779+0000","last_undegraded":"2024-09-16T00:11:36.035779+0000","last_fullsized":"2024-09-16T00:11:36.035779+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'381","last_scrub_stamp":"2024-09-16T00:11:36.035687+0000","last_deep_scrub":"146'381","last_deep_scrub_stamp":"2024-09-16T00:11:36.035687+0000","last_clean_scrub_stamp":"2024-09-16T00:11:36.035687+0000","objects_scrubbed":0,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:47.668509+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":350,"num_write":351,"num_write_kb":38221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"146'536","reported_seq":806,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:25.299903+0000","last_change":"2024-09-16T00:11:25.299947+0000","last_active":"2024-09-16T00:11:25.299903+0000","last_peered":"2024-09-16T00:11:25.299903+0000","last_clean":"2024-09-16T00:11:25.299903+0000","last_became_active":"2024-09-16T00:07:32.624270+0000","last_became_peered":"2024-09-16T00:07:32.624270+0000","last_unstale":"2024-09-16T00:11:25.299903+0000","last_undegraded":"2024-09-16T00:11:25.299903+0000","last_fullsized":"2024-09-16T00:11:25.299903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'536","last_scrub_stamp":"2024-09-16T00:11:25.299818+0000","last_deep_scrub":"146'536","last_deep_scrub_stamp":"2024-09-16T00:11:25.299818+0000","last_clean_scrub_stamp":"2024-09-16T00:11:25.299818+0000","objects_scrubbed":2,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:09.005575+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":473,"num_write_kb":4335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"146'424","reported_seq":697,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:31.724052+0000","last_change":"2024-09-16T00:11:31.724102+0000","last_active":"2024-09-16T00:11:31.724052+0000","last_peered":"2024-09-16T00:11:31.724052+0000","last_clean":"2024-09-16T00:11:31.724052+0000","last_became_active":"2024-09-16T00:07:33.626923+0000","last_became_peered":"2024-09-16T00:07:33.626923+0000","last_unstale":"2024-09-16T00:11:31.724052+0000","last_undegraded":"2024-09-16T00:11:31.724052+0000","last_fullsized":"2024-09-16T00:11:31.724052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'424","last_scrub_stamp":"2024-09-16T00:11:31.723956+0000","last_deep_scrub":"146'424","last_deep_scrub_stamp":"2024-09-16T00:11:31.723956+0000","last_clean_scrub_stamp":"2024-09-16T00:11:31.723956+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:17.210721+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":675,"num_write":389,"num_write_kb":34696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"123'72","reported_seq":343,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:26.402655+0000","last_change":"2024-09-16T00:11:26.402677+0000","last_active":"2024-09-16T00:11:26.402655+0000","last_peered":"2024-09-16T00:11:26.402655+0000","last_clean":"2024-09-16T00:11:26.402655+0000","last_became_active":"2024-09-16T00:07:32.628807+0000","last_became_peered":"2024-09-16T00:07:32.628807+0000","last_unstale":"2024-09-16T00:11:26.402655+0000","last_undegraded":"2024-09-16T00:11:26.402655+0000","last_fullsized":"2024-09-16T00:11:26.402655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"123'72","last_scrub_stamp":"2024-09-16T00:11:26.402569+0000","last_deep_scrub":"123'72","last_deep_scrub_stamp":"2024-09-16T00:11:26.402569+0000","last_clean_scrub_stamp":"2024-09-16T00:11:26.402569+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:51:30.744670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"146'404","reported_seq":779,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:39.076277+0000","last_change":"2024-09-16T00:11:39.076319+0000","last_active":"2024-09-16T00:11:39.076277+0000","last_peered":"2024-09-16T00:11:39.076277+0000","last_clean":"2024-09-16T00:11:39.076277+0000","last_became_active":"2024-09-16T00:10:55.839714+0000","last_became_peered":"2024-09-16T00:10:55.839714+0000","last_unstale":"2024-09-16T00:11:39.076277+0000","last_undegraded":"2024-09-16T00:11:39.076277+0000","last_fullsized":"2024-09-16T00:11:39.076277+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'404","last_scrub_stamp":"2024-09-16T00:11:39.076155+0000","last_deep_scrub":"146'404","last_deep_scrub_stamp":"2024-09-16T00:11:39.076155+0000","last_clean_scrub_stamp":"2024-09-16T00:11:39.076155+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:05.735512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":22590,"num_write":379,"num_write_kb":59088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"146'119","reported_seq":385,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:27.033257+0000","last_change":"2024-09-16T00:11:27.033287+0000","last_active":"2024-09-16T00:11:27.033257+0000","last_peered":"2024-09-16T00:11:27.033257+0000","last_clean":"2024-09-16T00:11:27.033257+0000","last_became_active":"2024-09-16T00:10:55.831881+0000","last_became_peered":"2024-09-16T00:10:55.831881+0000","last_unstale":"2024-09-16T00:11:27.033257+0000","last_undegraded":"2024-09-16T00:11:27.033257+0000","last_fullsized":"2024-09-16T00:11:27.033257+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'119","last_scrub_stamp":"2024-09-16T00:11:27.033154+0000","last_deep_scrub":"146'119","last_deep_scrub_stamp":"2024-09-16T00:11:27.033154+0000","last_clean_scrub_stamp":"2024-09-16T00:11:27.033154+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:44:57.799706+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"123'63","reported_seq":335,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:25.830562+0000","last_change":"2024-09-16T00:11:25.830562+0000","last_active":"2024-09-16T00:11:25.830562+0000","last_peered":"2024-09-16T00:11:25.830562+0000","last_clean":"2024-09-16T00:11:25.830562+0000","last_became_active":"2024-09-16T00:07:32.622089+0000","last_became_peered":"2024-09-16T00:07:32.622089+0000","last_unstale":"2024-09-16T00:11:25.830562+0000","last_undegraded":"2024-09-16T00:11:25.830562+0000","last_fullsized":"2024-09-16T00:11:25.830562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"123'63","last_scrub_stamp":"2024-09-16T00:11:25.830420+0000","last_deep_scrub":"123'63","last_deep_scrub_stamp":"2024-09-16T00:11:25.830420+0000","last_clean_scrub_stamp":"2024-09-16T00:11:25.830420+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:57:34.620409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"146'377","reported_seq":681,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:27.711579+0000","last_change":"2024-09-16T00:11:27.711606+0000","last_active":"2024-09-16T00:11:27.711579+0000","last_peered":"2024-09-16T00:11:27.711579+0000","last_clean":"2024-09-16T00:11:27.711579+0000","last_became_active":"2024-09-16T00:07:33.633800+0000","last_became_peered":"2024-09-16T00:07:33.633800+0000","last_unstale":"2024-09-16T00:11:27.711579+0000","last_undegraded":"2024-09-16T00:11:27.711579+0000","last_fullsized":"2024-09-16T00:11:27.711579+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'377","last_scrub_stamp":"2024-09-16T00:11:27.711474+0000","last_deep_scrub":"146'377","last_deep_scrub_stamp":"2024-09-16T00:11:27.711474+0000","last_clean_scrub_stamp":"2024-09-16T00:11:27.711474+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:50:15.305604+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":16103,"num_write":342,"num_write_kb":46241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"146'401","reported_seq":656,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:42.060527+0000","last_change":"2024-09-16T00:11:42.060563+0000","last_active":"2024-09-16T00:11:42.060527+0000","last_peered":"2024-09-16T00:11:42.060527+0000","last_clean":"2024-09-16T00:11:42.060527+0000","last_became_active":"2024-09-16T00:11:04.895672+0000","last_became_peered":"2024-09-16T00:11:04.895672+0000","last_unstale":"2024-09-16T00:11:42.060527+0000","last_undegraded":"2024-09-16T00:11:42.060527+0000","last_fullsized":"2024-09-16T00:11:42.060527+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'401","last_scrub_stamp":"2024-09-16T00:11:42.060439+0000","last_deep_scrub":"146'401","last_deep_scrub_stamp":"2024-09-16T00:11:42.060439+0000","last_clean_scrub_stamp":"2024-09-16T00:11:42.060439+0000","objects_scrubbed":2,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:44:40.764009+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":346,"num_write_kb":8359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":16777216,"num_keys_recovered":28,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"146'398","reported_seq":714,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:23.753444+0000","last_change":"2024-09-16T00:11:23.753496+0000","last_active":"2024-09-16T00:11:23.753444+0000","last_peered":"2024-09-16T00:11:23.753444+0000","last_clean":"2024-09-16T00:11:23.753444+0000","last_became_active":"2024-09-16T00:07:33.621934+0000","last_became_peered":"2024-09-16T00:07:33.621934+0000","last_unstale":"2024-09-16T00:11:23.753444+0000","last_undegraded":"2024-09-16T00:11:23.753444+0000","last_fullsized":"2024-09-16T00:11:23.753444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'398","last_scrub_stamp":"2024-09-16T00:11:23.753325+0000","last_deep_scrub":"146'398","last_deep_scrub_stamp":"2024-09-16T00:11:23.753325+0000","last_clean_scrub_stamp":"2024-09-16T00:11:23.753325+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:49:12.204683+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":29385,"num_write":362,"num_write_kb":83610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"146'243","reported_seq":690,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:24.433303+0000","last_change":"2024-09-16T00:11:24.433327+0000","last_active":"2024-09-16T00:11:24.433303+0000","last_peered":"2024-09-16T00:11:24.433303+0000","last_clean":"2024-09-16T00:11:24.433303+0000","last_became_active":"2024-09-16T00:07:32.628307+0000","last_became_peered":"2024-09-16T00:07:32.628307+0000","last_unstale":"2024-09-16T00:11:24.433303+0000","last_undegraded":"2024-09-16T00:11:24.433303+0000","last_fullsized":"2024-09-16T00:11:24.433303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'243","last_scrub_stamp":"2024-09-16T00:11:24.433238+0000","last_deep_scrub":"146'243","last_deep_scrub_stamp":"2024-09-16T00:11:24.433238+0000","last_clean_scrub_stamp":"2024-09-16T00:11:24.433238+0000","objects_scrubbed":1,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:18.135714+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1461247,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":176,"num_read_kb":1534,"num_write":179,"num_write_kb":1542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"146'408","reported_seq":730,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:27.334072+0000","last_change":"2024-09-16T00:11:27.334072+0000","last_active":"2024-09-16T00:11:27.334072+0000","last_peered":"2024-09-16T00:11:27.334072+0000","last_clean":"2024-09-16T00:11:27.334072+0000","last_became_active":"2024-09-16T00:07:33.617586+0000","last_became_peered":"2024-09-16T00:07:33.617586+0000","last_unstale":"2024-09-16T00:11:27.334072+0000","last_undegraded":"2024-09-16T00:11:27.334072+0000","last_fullsized":"2024-09-16T00:11:27.334072+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'408","last_scrub_stamp":"2024-09-16T00:11:27.333947+0000","last_deep_scrub":"146'408","last_deep_scrub_stamp":"2024-09-16T00:11:27.333947+0000","last_clean_scrub_stamp":"2024-09-16T00:11:27.333947+0000","objects_scrubbed":0,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:14:13.097981+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":28288,"num_write":378,"num_write_kb":66949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"146'402","reported_seq":674,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:26.826323+0000","last_change":"2024-09-16T00:11:26.826323+0000","last_active":"2024-09-16T00:11:26.826323+0000","last_peered":"2024-09-16T00:11:26.826323+0000","last_clean":"2024-09-16T00:11:26.826323+0000","last_became_active":"2024-09-16T00:07:32.622961+0000","last_became_peered":"2024-09-16T00:07:32.622961+0000","last_unstale":"2024-09-16T00:11:26.826323+0000","last_undegraded":"2024-09-16T00:11:26.826323+0000","last_fullsized":"2024-09-16T00:11:26.826323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'402","last_scrub_stamp":"2024-09-16T00:11:26.826171+0000","last_deep_scrub":"146'402","last_deep_scrub_stamp":"2024-09-16T00:11:26.826171+0000","last_clean_scrub_stamp":"2024-09-16T00:11:26.826171+0000","objects_scrubbed":2,"log_size":402,"log_dups_size":0,"ondisk_log_size":402,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:13:26.068250+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":353,"num_write_kb":8381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"146'394","reported_seq":699,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:26.248661+0000","last_change":"2024-09-16T00:11:26.248688+0000","last_active":"2024-09-16T00:11:26.248661+0000","last_peered":"2024-09-16T00:11:26.248661+0000","last_clean":"2024-09-16T00:11:26.248661+0000","last_became_active":"2024-09-16T00:07:33.628069+0000","last_became_peered":"2024-09-16T00:07:33.628069+0000","last_unstale":"2024-09-16T00:11:26.248661+0000","last_undegraded":"2024-09-16T00:11:26.248661+0000","last_fullsized":"2024-09-16T00:11:26.248661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'394","last_scrub_stamp":"2024-09-16T00:11:26.248577+0000","last_deep_scrub":"146'394","last_deep_scrub_stamp":"2024-09-16T00:11:26.248577+0000","last_clean_scrub_stamp":"2024-09-16T00:11:26.248577+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:04.342557+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":22188,"num_write":360,"num_write_kb":71552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"122'57","reported_seq":327,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:24.334534+0000","last_change":"2024-09-16T00:11:24.334554+0000","last_active":"2024-09-16T00:11:24.334534+0000","last_peered":"2024-09-16T00:11:24.334534+0000","last_clean":"2024-09-16T00:11:24.334534+0000","last_became_active":"2024-09-16T00:07:32.620576+0000","last_became_peered":"2024-09-16T00:07:32.620576+0000","last_unstale":"2024-09-16T00:11:24.334534+0000","last_undegraded":"2024-09-16T00:11:24.334534+0000","last_fullsized":"2024-09-16T00:11:24.334534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"122'57","last_scrub_stamp":"2024-09-16T00:11:24.334465+0000","last_deep_scrub":"122'57","last_deep_scrub_stamp":"2024-09-16T00:11:24.334465+0000","last_clean_scrub_stamp":"2024-09-16T00:11:24.334465+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:21.001782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"146'431","reported_seq":739,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:29.717126+0000","last_change":"2024-09-16T00:11:29.717151+0000","last_active":"2024-09-16T00:11:29.717126+0000","last_peered":"2024-09-16T00:11:29.717126+0000","last_clean":"2024-09-16T00:11:29.717126+0000","last_became_active":"2024-09-16T00:07:33.626808+0000","last_became_peered":"2024-09-16T00:07:33.626808+0000","last_unstale":"2024-09-16T00:11:29.717126+0000","last_undegraded":"2024-09-16T00:11:29.717126+0000","last_fullsized":"2024-09-16T00:11:29.717126+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'431","last_scrub_stamp":"2024-09-16T00:11:29.717049+0000","last_deep_scrub":"146'431","last_deep_scrub_stamp":"2024-09-16T00:11:29.717049+0000","last_clean_scrub_stamp":"2024-09-16T00:11:29.717049+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:05:13.047929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":24752,"num_write":390,"num_write_kb":75609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"146'259","reported_seq":313,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:41.072661+0000","last_change":"2024-09-16T00:11:41.072694+0000","last_active":"2024-09-16T00:11:41.072661+0000","last_peered":"2024-09-16T00:11:41.072661+0000","last_clean":"2024-09-16T00:11:41.072661+0000","last_became_active":"2024-09-16T00:10:58.839124+0000","last_became_peered":"2024-09-16T00:10:58.839124+0000","last_unstale":"2024-09-16T00:11:41.072661+0000","last_undegraded":"2024-09-16T00:11:41.072661+0000","last_fullsized":"2024-09-16T00:11:41.072661+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'259","last_scrub_stamp":"2024-09-16T00:11:41.072551+0000","last_deep_scrub":"146'259","last_deep_scrub_stamp":"2024-09-16T00:11:41.072551+0000","last_clean_scrub_stamp":"2024-09-16T00:11:41.072551+0000","objects_scrubbed":1,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:57:59.477514+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":202,"num_write_kb":4192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4194304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"146'407","reported_seq":733,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:26.306732+0000","last_change":"2024-09-16T00:11:26.306760+0000","last_active":"2024-09-16T00:11:26.306732+0000","last_peered":"2024-09-16T00:11:26.306732+0000","last_clean":"2024-09-16T00:11:26.306732+0000","last_became_active":"2024-09-16T00:07:33.617431+0000","last_became_peered":"2024-09-16T00:07:33.617431+0000","last_unstale":"2024-09-16T00:11:26.306732+0000","last_undegraded":"2024-09-16T00:11:26.306732+0000","last_fullsized":"2024-09-16T00:11:26.306732+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'407","last_scrub_stamp":"2024-09-16T00:11:26.306651+0000","last_deep_scrub":"146'407","last_deep_scrub_stamp":"2024-09-16T00:11:26.306651+0000","last_clean_scrub_stamp":"2024-09-16T00:11:26.306651+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:57:39.788461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":31221,"num_write":370,"num_write_kb":76143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"146'187","reported_seq":492,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:33.393099+0000","last_change":"2024-09-16T00:11:33.393143+0000","last_active":"2024-09-16T00:11:33.393099+0000","last_peered":"2024-09-16T00:11:33.393099+0000","last_clean":"2024-09-16T00:11:33.393099+0000","last_became_active":"2024-09-16T00:11:04.894163+0000","last_became_peered":"2024-09-16T00:11:04.894163+0000","last_unstale":"2024-09-16T00:11:33.393099+0000","last_undegraded":"2024-09-16T00:11:33.393099+0000","last_fullsized":"2024-09-16T00:11:33.393099+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'187","last_scrub_stamp":"2024-09-16T00:11:33.392996+0000","last_deep_scrub":"146'187","last_deep_scrub_stamp":"2024-09-16T00:11:33.392996+0000","last_clean_scrub_stamp":"2024-09-16T00:11:33.392996+0000","objects_scrubbed":1,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:34.533009+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":345,"num_write_kb":1559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":2964,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"146'349","reported_seq":800,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:34.663912+0000","last_change":"2024-09-16T00:11:34.663936+0000","last_active":"2024-09-16T00:11:34.663912+0000","last_peered":"2024-09-16T00:11:34.663912+0000","last_clean":"2024-09-16T00:11:34.663912+0000","last_became_active":"2024-09-16T00:10:55.845451+0000","last_became_peered":"2024-09-16T00:10:55.845451+0000","last_unstale":"2024-09-16T00:11:34.663912+0000","last_undegraded":"2024-09-16T00:11:34.663912+0000","last_fullsized":"2024-09-16T00:11:34.663912+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'349","last_scrub_stamp":"2024-09-16T00:11:34.663824+0000","last_deep_scrub":"146'349","last_deep_scrub_stamp":"2024-09-16T00:11:34.663824+0000","last_clean_scrub_stamp":"2024-09-16T00:11:34.663824+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:06:57.482319+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":19274,"num_write":323,"num_write_kb":22071,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"146'64","reported_seq":319,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:35.633796+0000","last_change":"2024-09-16T00:11:35.633858+0000","last_active":"2024-09-16T00:11:35.633796+0000","last_peered":"2024-09-16T00:11:35.633796+0000","last_clean":"2024-09-16T00:11:35.633796+0000","last_became_active":"2024-09-16T00:10:55.845687+0000","last_became_peered":"2024-09-16T00:10:55.845687+0000","last_unstale":"2024-09-16T00:11:35.633796+0000","last_undegraded":"2024-09-16T00:11:35.633796+0000","last_fullsized":"2024-09-16T00:11:35.633796+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'64","last_scrub_stamp":"2024-09-16T00:11:35.633698+0000","last_deep_scrub":"146'64","last_deep_scrub_stamp":"2024-09-16T00:11:35.633698+0000","last_clean_scrub_stamp":"2024-09-16T00:11:35.633698+0000","objects_scrubbed":1,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:49.303952+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":18,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"146'454","reported_seq":766,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:26.752597+0000","last_change":"2024-09-16T00:11:26.752623+0000","last_active":"2024-09-16T00:11:26.752597+0000","last_peered":"2024-09-16T00:11:26.752597+0000","last_clean":"2024-09-16T00:11:26.752597+0000","last_became_active":"2024-09-16T00:07:33.625948+0000","last_became_peered":"2024-09-16T00:07:33.625948+0000","last_unstale":"2024-09-16T00:11:26.752597+0000","last_undegraded":"2024-09-16T00:11:26.752597+0000","last_fullsized":"2024-09-16T00:11:26.752597+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'454","last_scrub_stamp":"2024-09-16T00:11:26.752511+0000","last_deep_scrub":"146'454","last_deep_scrub_stamp":"2024-09-16T00:11:26.752511+0000","last_clean_scrub_stamp":"2024-09-16T00:11:26.752511+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:51:17.956581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":17894,"num_write":426,"num_write_kb":88843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"146'565","reported_seq":796,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:29.015728+0000","last_change":"2024-09-16T00:11:29.015757+0000","last_active":"2024-09-16T00:11:29.015728+0000","last_peered":"2024-09-16T00:11:29.015728+0000","last_clean":"2024-09-16T00:11:29.015728+0000","last_became_active":"2024-09-16T00:10:55.828121+0000","last_became_peered":"2024-09-16T00:10:55.828121+0000","last_unstale":"2024-09-16T00:11:29.015728+0000","last_undegraded":"2024-09-16T00:11:29.015728+0000","last_fullsized":"2024-09-16T00:11:29.015728+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'565","last_scrub_stamp":"2024-09-16T00:11:29.015652+0000","last_deep_scrub":"146'565","last_deep_scrub_stamp":"2024-09-16T00:11:29.015652+0000","last_clean_scrub_stamp":"2024-09-16T00:11:29.015652+0000","objects_scrubbed":1,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:26.362357+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":4358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"146'456","reported_seq":743,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:25.259014+0000","last_change":"2024-09-16T00:11:25.259041+0000","last_active":"2024-09-16T00:11:25.259014+0000","last_peered":"2024-09-16T00:11:25.259014+0000","last_clean":"2024-09-16T00:11:25.259014+0000","last_became_active":"2024-09-16T00:07:33.621713+0000","last_became_peered":"2024-09-16T00:07:33.621713+0000","last_unstale":"2024-09-16T00:11:25.259014+0000","last_undegraded":"2024-09-16T00:11:25.259014+0000","last_fullsized":"2024-09-16T00:11:25.259014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'456","last_scrub_stamp":"2024-09-16T00:11:25.258916+0000","last_deep_scrub":"146'456","last_deep_scrub_stamp":"2024-09-16T00:11:25.258916+0000","last_clean_scrub_stamp":"2024-09-16T00:11:25.258916+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:11:03.026337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":9206,"num_write":434,"num_write_kb":79714,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"122'57","reported_seq":338,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:31.344017+0000","last_change":"2024-09-16T00:11:31.344042+0000","last_active":"2024-09-16T00:11:31.344017+0000","last_peered":"2024-09-16T00:11:31.344017+0000","last_clean":"2024-09-16T00:11:31.344017+0000","last_became_active":"2024-09-16T00:10:55.837749+0000","last_became_peered":"2024-09-16T00:10:55.837749+0000","last_unstale":"2024-09-16T00:11:31.344017+0000","last_undegraded":"2024-09-16T00:11:31.344017+0000","last_fullsized":"2024-09-16T00:11:31.344017+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"122'57","last_scrub_stamp":"2024-09-16T00:11:31.343924+0000","last_deep_scrub":"122'57","last_deep_scrub_stamp":"2024-09-16T00:11:31.343924+0000","last_clean_scrub_stamp":"2024-09-16T00:11:31.343924+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:29.012334+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"146'391","reported_seq":1545,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:35.633854+0000","last_change":"2024-09-16T00:11:35.633878+0000","last_active":"2024-09-16T00:11:35.633854+0000","last_peered":"2024-09-16T00:11:35.633854+0000","last_clean":"2024-09-16T00:11:35.633854+0000","last_became_active":"2024-09-16T00:11:04.890897+0000","last_became_peered":"2024-09-16T00:11:04.890897+0000","last_unstale":"2024-09-16T00:11:35.633854+0000","last_undegraded":"2024-09-16T00:11:35.633854+0000","last_fullsized":"2024-09-16T00:11:35.633854+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'391","last_scrub_stamp":"2024-09-16T00:11:35.633747+0000","last_deep_scrub":"146'391","last_deep_scrub_stamp":"2024-09-16T00:11:35.633747+0000","last_clean_scrub_stamp":"2024-09-16T00:11:35.633747+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:13:52.614093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":8384,"num_write":359,"num_write_kb":50710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":421,"num_bytes_recovered":615552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"146'181","reported_seq":479,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:32.358058+0000","last_change":"2024-09-16T00:11:32.358120+0000","last_active":"2024-09-16T00:11:32.358058+0000","last_peered":"2024-09-16T00:11:32.358058+0000","last_clean":"2024-09-16T00:11:32.358058+0000","last_became_active":"2024-09-16T00:11:04.890722+0000","last_became_peered":"2024-09-16T00:11:04.890722+0000","last_unstale":"2024-09-16T00:11:32.358058+0000","last_undegraded":"2024-09-16T00:11:32.358058+0000","last_fullsized":"2024-09-16T00:11:32.358058+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'181","last_scrub_stamp":"2024-09-16T00:11:32.357952+0000","last_deep_scrub":"146'181","last_deep_scrub_stamp":"2024-09-16T00:11:32.357952+0000","last_clean_scrub_stamp":"2024-09-16T00:11:32.357952+0000","objects_scrubbed":1,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:47:07.550270+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120,"num_write_kb":4191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":5577662,"num_keys_recovered":51,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"146'428","reported_seq":767,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:25.734659+0000","last_change":"2024-09-16T00:11:25.734698+0000","last_active":"2024-09-16T00:11:25.734659+0000","last_peered":"2024-09-16T00:11:25.734659+0000","last_clean":"2024-09-16T00:11:25.734659+0000","last_became_active":"2024-09-16T00:07:33.629948+0000","last_became_peered":"2024-09-16T00:07:33.629948+0000","last_unstale":"2024-09-16T00:11:25.734659+0000","last_undegraded":"2024-09-16T00:11:25.734659+0000","last_fullsized":"2024-09-16T00:11:25.734659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'428","last_scrub_stamp":"2024-09-16T00:11:25.734582+0000","last_deep_scrub":"146'428","last_deep_scrub_stamp":"2024-09-16T00:11:25.734582+0000","last_clean_scrub_stamp":"2024-09-16T00:11:25.734582+0000","objects_scrubbed":0,"log_size":428,"log_dups_size":0,"ondisk_log_size":428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:00.796399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":37740,"num_write":400,"num_write_kb":67449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"146'117","reported_seq":337,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:30.006957+0000","last_change":"2024-09-16T00:11:30.006991+0000","last_active":"2024-09-16T00:11:30.006957+0000","last_peered":"2024-09-16T00:11:30.006957+0000","last_clean":"2024-09-16T00:11:30.006957+0000","last_became_active":"2024-09-16T00:10:55.832841+0000","last_became_peered":"2024-09-16T00:10:55.832841+0000","last_unstale":"2024-09-16T00:11:30.006957+0000","last_undegraded":"2024-09-16T00:11:30.006957+0000","last_fullsized":"2024-09-16T00:11:30.006957+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'117","last_scrub_stamp":"2024-09-16T00:11:30.006851+0000","last_deep_scrub":"146'117","last_deep_scrub_stamp":"2024-09-16T00:11:30.006851+0000","last_clean_scrub_stamp":"2024-09-16T00:11:30.006851+0000","objects_scrubbed":3,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:57:54.512442+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":8303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"146'457","reported_seq":622,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:33.564197+0000","last_change":"2024-09-16T00:11:33.564240+0000","last_active":"2024-09-16T00:11:33.564197+0000","last_peered":"2024-09-16T00:11:33.564197+0000","last_clean":"2024-09-16T00:11:33.564197+0000","last_became_active":"2024-09-16T00:10:00.281972+0000","last_became_peered":"2024-09-16T00:10:00.281972+0000","last_unstale":"2024-09-16T00:11:33.564197+0000","last_undegraded":"2024-09-16T00:11:33.564197+0000","last_fullsized":"2024-09-16T00:11:33.564197+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'457","last_scrub_stamp":"2024-09-16T00:11:33.564084+0000","last_deep_scrub":"146'457","last_deep_scrub_stamp":"2024-09-16T00:11:33.564084+0000","last_clean_scrub_stamp":"2024-09-16T00:11:33.564084+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:20:14.567084+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":43345,"num_write":425,"num_write_kb":121191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":29546741,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"146'411","reported_seq":749,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:28.752759+0000","last_change":"2024-09-16T00:11:28.752803+0000","last_active":"2024-09-16T00:11:28.752759+0000","last_peered":"2024-09-16T00:11:28.752759+0000","last_clean":"2024-09-16T00:11:28.752759+0000","last_became_active":"2024-09-16T00:07:33.620545+0000","last_became_peered":"2024-09-16T00:07:33.620545+0000","last_unstale":"2024-09-16T00:11:28.752759+0000","last_undegraded":"2024-09-16T00:11:28.752759+0000","last_fullsized":"2024-09-16T00:11:28.752759+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'411","last_scrub_stamp":"2024-09-16T00:11:28.752614+0000","last_deep_scrub":"146'411","last_deep_scrub_stamp":"2024-09-16T00:11:28.752614+0000","last_clean_scrub_stamp":"2024-09-16T00:11:28.752614+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:29.892627+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":32882,"num_write":368,"num_write_kb":108251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"146'221","reported_seq":449,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:27.994173+0000","last_change":"2024-09-16T00:11:27.994208+0000","last_active":"2024-09-16T00:11:27.994173+0000","last_peered":"2024-09-16T00:11:27.994173+0000","last_clean":"2024-09-16T00:11:27.994173+0000","last_became_active":"2024-09-16T00:10:55.835301+0000","last_became_peered":"2024-09-16T00:10:55.835301+0000","last_unstale":"2024-09-16T00:11:27.994173+0000","last_undegraded":"2024-09-16T00:11:27.994173+0000","last_fullsized":"2024-09-16T00:11:27.994173+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'221","last_scrub_stamp":"2024-09-16T00:11:27.994078+0000","last_deep_scrub":"146'221","last_deep_scrub_stamp":"2024-09-16T00:11:27.994078+0000","last_clean_scrub_stamp":"2024-09-16T00:11:27.994078+0000","objects_scrubbed":2,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:04.197549+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":169,"num_write_kb":4194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"146'394","reported_seq":732,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:34.699219+0000","last_change":"2024-09-16T00:11:34.699249+0000","last_active":"2024-09-16T00:11:34.699219+0000","last_peered":"2024-09-16T00:11:34.699219+0000","last_clean":"2024-09-16T00:11:34.699219+0000","last_became_active":"2024-09-16T00:07:33.635304+0000","last_became_peered":"2024-09-16T00:07:33.635304+0000","last_unstale":"2024-09-16T00:11:34.699219+0000","last_undegraded":"2024-09-16T00:11:34.699219+0000","last_fullsized":"2024-09-16T00:11:34.699219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'394","last_scrub_stamp":"2024-09-16T00:11:34.699132+0000","last_deep_scrub":"146'394","last_deep_scrub_stamp":"2024-09-16T00:11:34.699132+0000","last_clean_scrub_stamp":"2024-09-16T00:11:34.699132+0000","objects_scrubbed":0,"log_size":394,"log_dups_size":0,"ondisk_log_size":394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:08.072472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":36278,"num_write":367,"num_write_kb":113361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"146'267","reported_seq":507,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:31.044946+0000","last_change":"2024-09-16T00:11:31.045010+0000","last_active":"2024-09-16T00:11:31.044946+0000","last_peered":"2024-09-16T00:11:31.044946+0000","last_clean":"2024-09-16T00:11:31.044946+0000","last_became_active":"2024-09-16T00:10:55.843843+0000","last_became_peered":"2024-09-16T00:10:55.843843+0000","last_unstale":"2024-09-16T00:11:31.044946+0000","last_undegraded":"2024-09-16T00:11:31.044946+0000","last_fullsized":"2024-09-16T00:11:31.044946+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'267","last_scrub_stamp":"2024-09-16T00:11:31.044853+0000","last_deep_scrub":"146'267","last_deep_scrub_stamp":"2024-09-16T00:11:31.044853+0000","last_clean_scrub_stamp":"2024-09-16T00:11:31.044853+0000","objects_scrubbed":4,"log_size":267,"log_dups_size":0,"ondisk_log_size":267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:30:02.227397+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":197,"num_write_kb":12428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":30,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"146'354","reported_seq":616,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:30.378003+0000","last_change":"2024-09-16T00:11:30.378069+0000","last_active":"2024-09-16T00:11:30.378003+0000","last_peered":"2024-09-16T00:11:30.378003+0000","last_clean":"2024-09-16T00:11:30.378003+0000","last_became_active":"2024-09-16T00:08:15.815077+0000","last_became_peered":"2024-09-16T00:08:15.815077+0000","last_unstale":"2024-09-16T00:11:30.378003+0000","last_undegraded":"2024-09-16T00:11:30.378003+0000","last_fullsized":"2024-09-16T00:11:30.378003+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'354","last_scrub_stamp":"2024-09-16T00:11:30.377869+0000","last_deep_scrub":"146'354","last_deep_scrub_stamp":"2024-09-16T00:11:30.377869+0000","last_clean_scrub_stamp":"2024-09-16T00:11:30.377869+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:42:42.348218+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":12352,"num_write":323,"num_write_kb":38097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"146'94","reported_seq":367,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:23.758581+0000","last_change":"2024-09-16T00:11:23.758581+0000","last_active":"2024-09-16T00:11:23.758581+0000","last_peered":"2024-09-16T00:11:23.758581+0000","last_clean":"2024-09-16T00:11:23.758581+0000","last_became_active":"2024-09-16T00:07:32.627638+0000","last_became_peered":"2024-09-16T00:07:32.627638+0000","last_unstale":"2024-09-16T00:11:23.758581+0000","last_undegraded":"2024-09-16T00:11:23.758581+0000","last_fullsized":"2024-09-16T00:11:23.758581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'94","last_scrub_stamp":"2024-09-16T00:11:23.758415+0000","last_deep_scrub":"146'94","last_deep_scrub_stamp":"2024-09-16T00:11:23.758415+0000","last_clean_scrub_stamp":"2024-09-16T00:11:23.758415+0000","objects_scrubbed":1,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:01:39.271812+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":4141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"146'419","reported_seq":810,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:31.729240+0000","last_change":"2024-09-16T00:11:31.729240+0000","last_active":"2024-09-16T00:11:31.729240+0000","last_peered":"2024-09-16T00:11:31.729240+0000","last_clean":"2024-09-16T00:11:31.729240+0000","last_became_active":"2024-09-16T00:10:00.283734+0000","last_became_peered":"2024-09-16T00:10:00.283734+0000","last_unstale":"2024-09-16T00:11:31.729240+0000","last_undegraded":"2024-09-16T00:11:31.729240+0000","last_fullsized":"2024-09-16T00:11:31.729240+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'419","last_scrub_stamp":"2024-09-16T00:11:31.729088+0000","last_deep_scrub":"146'419","last_deep_scrub_stamp":"2024-09-16T00:11:31.729088+0000","last_clean_scrub_stamp":"2024-09-16T00:11:31.729088+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:08:58.075218+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":40163,"num_write":392,"num_write_kb":81063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":21131701,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"146'101","reported_seq":384,"reported_epoch":146,"state":"active+clean","last_fresh":"2024-09-16T00:11:30.768908+0000","last_change":"2024-09-16T00:11:30.768941+0000","last_active":"2024-09-16T00:11:30.768908+0000","last_peered":"2024-09-16T00:11:30.768908+0000","last_clean":"2024-09-16T00:11:30.768908+0000","last_became_active":"2024-09-16T00:10:55.844922+0000","last_became_peered":"2024-09-16T00:10:55.844922+0000","last_unstale":"2024-09-16T00:11:30.768908+0000","last_undegraded":"2024-09-16T00:11:30.768908+0000","last_fullsized":"2024-09-16T00:11:30.768908+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'101","last_scrub_stamp":"2024-09-16T00:11:30.768826+0000","last_deep_scrub":"146'101","last_deep_scrub_stamp":"2024-09-16T00:11:30.768826+0000","last_clean_scrub_stamp":"2024-09-16T00:11:30.768826+0000","objects_scrubbed":3,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:34.569318+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60,"num_write_kb":4138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2561,"num_read_kb":1362012,"num_write":24417,"num_write_kb":4088607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3177,"num_bytes_recovered":294487447,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5234,"internal_metadata":0},"log_size":26417,"ondisk_log_size":26417,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":129048702,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":176,"num_read_kb":1534,"num_write":5621,"num_write_kb":131062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":26562382,"num_keys_recovered":3117,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86642688,"data_stored":258097404,"data_compressed":38739888,"data_compressed_allocated":49811456,"data_compressed_original":221323264,"omap_allocated":262411,"internal_metadata":0},"log_size":7198,"ondisk_log_size":7198,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64754,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":127,"seq":545460846604,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57992,"kb_used_data":16056,"kb_used_omap":42,"kb_used_meta":41877,"kb_avail":94313848,"statfs":{"total":96636764160,"available":96577380352,"internally_reserved":0,"allocated":16441344,"data_stored":51769079,"data_compressed":7683094,"data_compressed_allocated":9629696,"data_compressed_original":45862912,"omap_allocated":43696,"internal_metadata":42882384},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":94,"seq":403726925847,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56024,"kb_used_data":14280,"kb_used_omap":221,"kb_used_meta":41506,"kb_avail":94315816,"statfs":{"total":96636764160,"available":96579395584,"internally_reserved":0,"allocated":14622720,"data_stored":44547058,"data_compressed":6872875,"data_compressed_allocated":8704000,"data_compressed_original":39534592,"omap_allocated":227133,"internal_metadata":42502339},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509496,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57200,"kb_used_data":12128,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94314640,"statfs":{"total":96636764160,"available":96578191360,"internally_reserved":0,"allocated":12419072,"data_stored":26012395,"data_compressed":3584779,"data_compressed_allocated":4849664,"data_compressed_original":19345408,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509497,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63484,"kb_used_data":18412,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94308356,"statfs":{"total":96636764160,"available":96571756544,"internally_reserved":0,"allocated":18853888,"data_stored":46983951,"data_compressed":6735741,"data_compressed_allocated":8769536,"data_compressed_original":37810176,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509495,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51044,"kb_used_data":11540,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94320796,"statfs":{"total":96636764160,"available":96584495104,"internally_reserved":0,"allocated":11816960,"data_stored":33425393,"data_compressed":5001493,"data_compressed_allocated":6361088,"data_compressed_original":28876800,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542200,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67128,"kb_used_data":9576,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94304712,"statfs":{"total":96636764160,"available":96568025088,"internally_reserved":0,"allocated":9805824,"data_stored":26012465,"data_compressed":3864818,"data_compressed_allocated":4833280,"data_compressed_original":21946368,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542200,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57364,"kb_used_data":12292,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94314476,"statfs":{"total":96636764160,"available":96578023424,"internally_reserved":0,"allocated":12587008,"data_stored":30797615,"data_compressed":4284061,"data_compressed_allocated":5672960,"data_compressed_original":24797184,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35272,"kb_used_data":4088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336568,"statfs":{"total":96636764160,"available":96600645632,"internally_reserved":0,"allocated":4186112,"data_stored":6502032,"data_compressed":777781,"data_compressed_allocated":1081344,"data_compressed_original":4296704,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32377,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10776576,"data_stored":29360766,"data_compressed":4251684,"data_compressed_allocated":5627904,"data_compressed_original":24223744,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8060928,"data_stored":25165984,"data_compressed":3864818,"data_compressed_allocated":4833280,"data_compressed_original":21946368,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2441216,"data_stored":5655551,"data_compressed":777781,"data_compressed_allocated":1081344,"data_compressed_original":4296704,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10072064,"data_stored":32578912,"data_compressed":5001493,"data_compressed_allocated":6361088,"data_compressed_original":28876800,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":17108992,"data_stored":46137470,"data_compressed":6735741,"data_compressed_allocated":8769536,"data_compressed_original":37810176,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10674176,"data_stored":25165914,"data_compressed":3584779,"data_compressed_allocated":4849664,"data_compressed_original":19345408,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12877824,"data_stored":43700577,"data_compressed":6872875,"data_compressed_allocated":8704000,"data_compressed_original":39534592,"omap_allocated":225539,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14630912,"data_stored":50332230,"data_compressed":7650717,"data_compressed_allocated":9584640,"data_compressed_original":45289472,"omap_allocated":36872,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5234,"internal_metadata":0}]}} 2024-09-16T00:11:45.849 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T00:11:45.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.980+0000 7f2e2f83c640 1 Processor -- start 2024-09-16T00:11:45.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.981+0000 7f2e2f83c640 1 -- start start 2024-09-16T00:11:45.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.981+0000 7f2e2f83c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2e280ce810 0x7f2e280c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:45.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.981+0000 7f2e2f83c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e280703e0 con 0x7f2e280ce430 2024-09-16T00:11:45.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.981+0000 7f2e2f83c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e28070680 con 0x7f2e280c9430 2024-09-16T00:11:45.990 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.981+0000 7f2e2f83c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2e280c5750 con 0x7f2e280ce810 2024-09-16T00:11:45.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.982+0000 7f2e2cdd7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2e280ce810 0x7f2e280c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:45.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.982+0000 7f2e2d5d8640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f2e280ce430 0x7f2e280b31c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56432/0 (socket says 172.21.15.139:56432) 2024-09-16T00:11:45.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.982+0000 7f2e2d5d8640 1 -- 172.21.15.139:0/2289847956 learned_addr learned my addr 172.21.15.139:0/2289847956 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:45.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.982+0000 7f2e2cdd7640 1 -- 172.21.15.139:0/2289847956 >> v1:172.21.15.164:6789/0 conn(0x7f2e280c9430 legacy=0x7f2e2815bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:45.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.982+0000 7f2e2cdd7640 1 -- 172.21.15.139:0/2289847956 >> v1:172.21.15.139:6789/0 conn(0x7f2e280ce430 legacy=0x7f2e280b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:45.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.982+0000 7f2e1ffff640 1 -- 172.21.15.139:0/2289847956 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 801089023 0 0) 0x7f2e280703e0 con 0x7f2e280ce430 2024-09-16T00:11:45.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.982+0000 7f2e2cdd7640 1 -- 172.21.15.139:0/2289847956 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e28070920 con 0x7f2e280ce810 2024-09-16T00:11:45.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.983+0000 7f2e2cdd7640 1 --2- 172.21.15.139:0/2289847956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e280ce810 0x7f2e280c5be0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2e180097a0 tx=0x7f2e1802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d26b0ebd9f187d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:45.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.983+0000 7f2e1ffff640 1 -- 172.21.15.139:0/2289847956 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18037070 con 0x7f2e280ce810 2024-09-16T00:11:45.991 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.983+0000 7f2e1ffff640 1 -- 172.21.15.139:0/2289847956 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e18032440 con 0x7f2e280ce810 2024-09-16T00:11:45.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.983+0000 7f2e1ffff640 1 -- 172.21.15.139:0/2289847956 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e18032810 con 0x7f2e280ce810 2024-09-16T00:11:45.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.983+0000 7f2e2f83c640 1 -- 172.21.15.139:0/2289847956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e280ce810 msgr2=0x7f2e280c5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:45.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.983+0000 7f2e2f83c640 1 --2- 172.21.15.139:0/2289847956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e280ce810 0x7f2e280c5be0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2e180097a0 tx=0x7f2e1802a230 comp rx=0 tx=0).stop 2024-09-16T00:11:45.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.983+0000 7f2e2f83c640 1 -- 172.21.15.139:0/2289847956 shutdown_connections 2024-09-16T00:11:45.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.983+0000 7f2e2f83c640 1 --2- 172.21.15.139:0/2289847956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e280ce810 0x7f2e280c5be0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:45.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.983+0000 7f2e2f83c640 1 -- 172.21.15.139:0/2289847956 >> 172.21.15.139:0/2289847956 conn(0x7f2e2806c150 msgr2=0x7f2e28067950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:45.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.983+0000 7f2e2f83c640 1 -- 172.21.15.139:0/2289847956 shutdown_connections 2024-09-16T00:11:45.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.983+0000 7f2e2f83c640 1 -- 172.21.15.139:0/2289847956 wait complete. 2024-09-16T00:11:45.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.984+0000 7f2e2f83c640 1 Processor -- start 2024-09-16T00:11:45.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.984+0000 7f2e2f83c640 1 -- start start 2024-09-16T00:11:45.992 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.985+0000 7f2e2f83c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e280c9430 0x7f2e280ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:45.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.985+0000 7f2e2f83c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e280ce430 0x7f2e28158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:45.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.985+0000 7f2e2f83c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e280ce810 0x7f2e280cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:45.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.985+0000 7f2e2f83c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2806ffc0 con 0x7f2e280ce810 2024-09-16T00:11:45.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.985+0000 7f2e2f83c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f2e28070ee0 con 0x7f2e280c9430 2024-09-16T00:11:45.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.985+0000 7f2e2f83c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f2e28070140 con 0x7f2e280ce430 2024-09-16T00:11:45.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.985+0000 7f2e2ddd9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e280ce810 0x7f2e280cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:45.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.985+0000 7f2e2ddd9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e280ce810 0x7f2e280cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45308/0 (socket says 172.21.15.139:45308) 2024-09-16T00:11:45.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.985+0000 7f2e2ddd9640 1 -- 172.21.15.139:0/3480027025 learned_addr learned my addr 172.21.15.139:0/3480027025 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:45.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.985+0000 7f2e2d5d8640 1 --2- 172.21.15.139:0/3480027025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e280c9430 0x7f2e280ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:45.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.985+0000 7f2e2cdd7640 1 --2- 172.21.15.139:0/3480027025 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e280ce430 0x7f2e28158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:45.993 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.986+0000 7f2e2ddd9640 1 -- 172.21.15.139:0/3480027025 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e280ce430 msgr2=0x7f2e28158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:45.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.986+0000 7f2e2ddd9640 1 --2- 172.21.15.139:0/3480027025 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e280ce430 0x7f2e28158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:45.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.986+0000 7f2e2ddd9640 1 -- 172.21.15.139:0/3480027025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e280c9430 msgr2=0x7f2e280ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:45.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.986+0000 7f2e2ddd9640 1 --2- 172.21.15.139:0/3480027025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e280c9430 0x7f2e280ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:45.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.986+0000 7f2e2ddd9640 1 -- 172.21.15.139:0/3480027025 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e281f0320 con 0x7f2e280ce810 2024-09-16T00:11:45.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.986+0000 7f2e2ddd9640 1 --2- 172.21.15.139:0/3480027025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e280ce810 0x7f2e280cd440 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000c760 tx=0x7f2e2000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:45.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.986+0000 7f2e1dffb640 1 -- 172.21.15.139:0/3480027025 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20003b90 con 0x7f2e280ce810 2024-09-16T00:11:45.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.986+0000 7f2e1dffb640 1 -- 172.21.15.139:0/3480027025 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e20003d30 con 0x7f2e280ce810 2024-09-16T00:11:45.994 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.986+0000 7f2e1dffb640 1 -- 172.21.15.139:0/3480027025 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2000aa90 con 0x7f2e280ce810 2024-09-16T00:11:45.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.986+0000 7f2e2f83c640 1 -- 172.21.15.139:0/3480027025 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e281f05a0 con 0x7f2e280ce810 2024-09-16T00:11:45.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.986+0000 7f2e2f83c640 1 -- 172.21.15.139:0/3480027025 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e281f0b60 con 0x7f2e280ce810 2024-09-16T00:11:45.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.987+0000 7f2e2f83c640 1 -- 172.21.15.139:0/3480027025 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e280c9ab0 con 0x7f2e280ce810 2024-09-16T00:11:45.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.987+0000 7f2e1dffb640 1 -- 172.21.15.139:0/3480027025 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2e2000ac30 con 0x7f2e280ce810 2024-09-16T00:11:45.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.988+0000 7f2e1dffb640 1 --2- 172.21.15.139:0/3480027025 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2e0c06c2b0 0x7f2e0c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:45.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.988+0000 7f2e1dffb640 1 -- 172.21.15.139:0/3480027025 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f2e2008b2a0 con 0x7f2e280ce810 2024-09-16T00:11:45.995 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.988+0000 7f2e2d5d8640 1 --2- 172.21.15.139:0/3480027025 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2e0c06c2b0 0x7f2e0c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:45.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.988+0000 7f2e2d5d8640 1 --2- 172.21.15.139:0/3480027025 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2e0c06c2b0 0x7f2e0c06e770 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2e24007f70 tx=0x7f2e24007f00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:45.996 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:45.992+0000 7f2e1dffb640 1 -- 172.21.15.139:0/3480027025 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2e280c9ab0 con 0x7f2e280ce810 2024-09-16T00:11:46.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.160+0000 7f2e2f83c640 1 -- 172.21.15.139:0/3480027025 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f2e280b31c0 con 0x7f2e280ce810 2024-09-16T00:11:46.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.162+0000 7f2e1dffb640 1 -- 172.21.15.139:0/3480027025 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f2e281f0320 con 0x7f2e280ce810 2024-09-16T00:11:46.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.163+0000 7f2e1dffb640 1 -- 172.21.15.139:0/3480027025 <== mon.0 v2:172.21.15.139:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2e2008b0d0 con 0x7f2e280ce810 2024-09-16T00:11:46.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.166+0000 7f2e2f83c640 1 -- 172.21.15.139:0/3480027025 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2e0c06c2b0 msgr2=0x7f2e0c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.166+0000 7f2e2f83c640 1 --2- 172.21.15.139:0/3480027025 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2e0c06c2b0 0x7f2e0c06e770 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2e24007f70 tx=0x7f2e24007f00 comp rx=0 tx=0).stop 2024-09-16T00:11:46.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.166+0000 7f2e2f83c640 1 -- 172.21.15.139:0/3480027025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e280ce810 msgr2=0x7f2e280cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.166+0000 7f2e2f83c640 1 --2- 172.21.15.139:0/3480027025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e280ce810 0x7f2e280cd440 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000c760 tx=0x7f2e2000cc20 comp rx=0 tx=0).stop 2024-09-16T00:11:46.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.166+0000 7f2e2f83c640 1 -- 172.21.15.139:0/3480027025 shutdown_connections 2024-09-16T00:11:46.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.166+0000 7f2e2f83c640 1 --2- 172.21.15.139:0/3480027025 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f2e0c06c2b0 0x7f2e0c06e770 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.166+0000 7f2e2f83c640 1 --2- 172.21.15.139:0/3480027025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e280ce810 0x7f2e280cd440 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.166+0000 7f2e2f83c640 1 --2- 172.21.15.139:0/3480027025 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f2e280ce430 0x7f2e28158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.166+0000 7f2e2f83c640 1 --2- 172.21.15.139:0/3480027025 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2e280c9430 0x7f2e280ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.166+0000 7f2e2f83c640 1 -- 172.21.15.139:0/3480027025 >> 172.21.15.139:0/3480027025 conn(0x7f2e2806c150 msgr2=0x7f2e280c8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:46.167 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.166+0000 7f2e2f83c640 1 -- 172.21.15.139:0/3480027025 shutdown_connections 2024-09-16T00:11:46.168 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.166+0000 7f2e2f83c640 1 -- 172.21.15.139:0/3480027025 wait complete. 2024-09-16T00:11:46.178 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T00:11:46.179 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T00:11:46.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.276+0000 7f590f265640 1 Processor -- start 2024-09-16T00:11:46.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.276+0000 7f590f265640 1 -- start start 2024-09-16T00:11:46.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.277+0000 7f590f265640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f59080c57a0 0x7f59080c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.277+0000 7f590f265640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f590815bc50 0x7f5908159120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.277+0000 7f590f265640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59080c6e80 con 0x7f59080c3020 2024-09-16T00:11:46.279 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.277+0000 7f590f265640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f5908159660 con 0x7f59080c57a0 2024-09-16T00:11:46.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.277+0000 7f590f265640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f5908158660 con 0x7f590815bc50 2024-09-16T00:11:46.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.277+0000 7f590c800640 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f59080c3020 0x7f59081524e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.139:55732/0 (socket says 172.21.15.139:55732) 2024-09-16T00:11:46.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.278+0000 7f590c800640 1 -- 172.21.15.139:0/4150556709 learned_addr learned my addr 172.21.15.139:0/4150556709 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:46.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.278+0000 7f590d802640 1 --2- 172.21.15.139:0/4150556709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f590815bc50 0x7f5908159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.278+0000 7f590d001640 1 --2- 172.21.15.139:0/4150556709 >> v2:172.21.15.164:3300/0 conn(0x7f59080c57a0 0x7f59080c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.278+0000 7f58fffff640 1 -- 172.21.15.139:0/4150556709 <== mon.1 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1450247925 0 0) 0x7f59080c6e80 con 0x7f59080c3020 2024-09-16T00:11:46.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.278+0000 7f58fffff640 1 -- 172.21.15.139:0/4150556709 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58f4003520 con 0x7f59080c3020 2024-09-16T00:11:46.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.278+0000 7f590d001640 1 -- 172.21.15.139:0/4150556709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f590815bc50 msgr2=0x7f5908159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:46.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.278+0000 7f590d001640 1 --2- 172.21.15.139:0/4150556709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f590815bc50 0x7f5908159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.278+0000 7f590d001640 1 -- 172.21.15.139:0/4150556709 >> v1:172.21.15.164:6789/0 conn(0x7f59080c3020 legacy=0x7f59081524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.280 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.278+0000 7f590d001640 1 -- 172.21.15.139:0/4150556709 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59080c8870 con 0x7f59080c57a0 2024-09-16T00:11:46.281 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.278+0000 7f590d802640 1 --2- 172.21.15.139:0/4150556709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f590815bc50 0x7f5908159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:11:46.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.279+0000 7f590d001640 1 --2- 172.21.15.139:0/4150556709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59080c57a0 0x7f59080c5b80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f59040097a0 tx=0x7f590402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9c8e2da802b3503 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:46.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.279+0000 7f58fffff640 1 -- 172.21.15.139:0/4150556709 <== mon.1 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5904037070 con 0x7f59080c57a0 2024-09-16T00:11:46.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.279+0000 7f58fffff640 1 -- 172.21.15.139:0/4150556709 <== mon.1 v2:172.21.15.164:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5904032440 con 0x7f59080c57a0 2024-09-16T00:11:46.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.279+0000 7f58fffff640 1 -- 172.21.15.139:0/4150556709 <== mon.1 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59040327f0 con 0x7f59080c57a0 2024-09-16T00:11:46.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.280+0000 7f590f265640 1 -- 172.21.15.139:0/4150556709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59080c57a0 msgr2=0x7f59080c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.280+0000 7f590f265640 1 --2- 172.21.15.139:0/4150556709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59080c57a0 0x7f59080c5b80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f59040097a0 tx=0x7f590402a210 comp rx=0 tx=0).stop 2024-09-16T00:11:46.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.280+0000 7f590f265640 1 -- 172.21.15.139:0/4150556709 shutdown_connections 2024-09-16T00:11:46.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.280+0000 7f590f265640 1 --2- 172.21.15.139:0/4150556709 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f590815bc50 0x7f5908159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.286 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.280+0000 7f590f265640 1 --2- 172.21.15.139:0/4150556709 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59080c57a0 0x7f59080c5b80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.280+0000 7f590f265640 1 -- 172.21.15.139:0/4150556709 >> 172.21.15.139:0/4150556709 conn(0x7f5908067950 msgr2=0x7f5908067d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:46.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.280+0000 7f590f265640 1 -- 172.21.15.139:0/4150556709 shutdown_connections 2024-09-16T00:11:46.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.280+0000 7f590f265640 1 -- 172.21.15.139:0/4150556709 wait complete. 2024-09-16T00:11:46.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.281+0000 7f590f265640 1 Processor -- start 2024-09-16T00:11:46.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.281+0000 7f590f265640 1 -- start start 2024-09-16T00:11:46.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.281+0000 7f590f265640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59080c3020 0x7f5908154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590f265640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080c57a0 0x7f5908155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590f265640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f590815bc50 0x7f59080cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.287 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590f265640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5908158c00 con 0x7f59080c57a0 2024-09-16T00:11:46.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590f265640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f5908158920 con 0x7f59080c3020 2024-09-16T00:11:46.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590f265640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f59080c6a90 con 0x7f590815bc50 2024-09-16T00:11:46.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590c800640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080c57a0 0x7f5908155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590c800640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080c57a0 0x7f5908155770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45324/0 (socket says 172.21.15.139:45324) 2024-09-16T00:11:46.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590c800640 1 -- 172.21.15.139:0/2100610628 learned_addr learned my addr 172.21.15.139:0/2100610628 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:46.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590d001640 1 --2- 172.21.15.139:0/2100610628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59080c3020 0x7f5908154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590d802640 1 --2- 172.21.15.139:0/2100610628 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f590815bc50 0x7f59080cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590c800640 1 -- 172.21.15.139:0/2100610628 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f590815bc50 msgr2=0x7f59080cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.288 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590c800640 1 --2- 172.21.15.139:0/2100610628 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f590815bc50 0x7f59080cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590c800640 1 -- 172.21.15.139:0/2100610628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59080c3020 msgr2=0x7f5908154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590c800640 1 --2- 172.21.15.139:0/2100610628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59080c3020 0x7f5908154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590c800640 1 -- 172.21.15.139:0/2100610628 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59081eed40 con 0x7f59080c57a0 2024-09-16T00:11:46.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590c800640 1 --2- 172.21.15.139:0/2100610628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080c57a0 0x7f5908155770 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f58f800c390 tx=0x7f58f800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:46.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f58fdffb640 1 -- 172.21.15.139:0/2100610628 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8017070 con 0x7f59080c57a0 2024-09-16T00:11:46.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f58fdffb640 1 -- 172.21.15.139:0/2100610628 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f58f8005ce0 con 0x7f59080c57a0 2024-09-16T00:11:46.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f58fdffb640 1 -- 172.21.15.139:0/2100610628 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8007310 con 0x7f59080c57a0 2024-09-16T00:11:46.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.282+0000 7f590f265640 1 -- 172.21.15.139:0/2100610628 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590814dcf0 con 0x7f59080c57a0 2024-09-16T00:11:46.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.283+0000 7f590f265640 1 -- 172.21.15.139:0/2100610628 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f590814e2b0 con 0x7f59080c57a0 2024-09-16T00:11:46.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.284+0000 7f590f265640 1 -- 172.21.15.139:0/2100610628 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f590806c520 con 0x7f59080c57a0 2024-09-16T00:11:46.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.284+0000 7f58fdffb640 1 -- 172.21.15.139:0/2100610628 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f58f801a070 con 0x7f59080c57a0 2024-09-16T00:11:46.289 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.284+0000 7f58fdffb640 1 --2- 172.21.15.139:0/2100610628 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f58ec06c2b0 0x7f58ec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.284+0000 7f58fdffb640 1 -- 172.21.15.139:0/2100610628 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f58f808a8a0 con 0x7f59080c57a0 2024-09-16T00:11:46.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.284+0000 7f590d001640 1 --2- 172.21.15.139:0/2100610628 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f58ec06c2b0 0x7f58ec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.285+0000 7f590d001640 1 --2- 172.21.15.139:0/2100610628 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f58ec06c2b0 0x7f58ec06e770 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5904034000 tx=0x7f590402e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:46.290 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.289+0000 7f58fdffb640 1 -- 172.21.15.139:0/2100610628 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f58f8058840 con 0x7f59080c57a0 2024-09-16T00:11:46.486 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.485+0000 7f590f265640 1 -- 172.21.15.139:0/2100610628 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59080c27c0 con 0x7f59080c57a0 2024-09-16T00:11:46.487 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.486+0000 7f58fdffb640 1 -- 172.21.15.139:0/2100610628 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6881 (secure 0 0 0) 0x7f58f805cb40 con 0x7f59080c57a0 2024-09-16T00:11:46.487 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:11:46.487 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":8,"btime":"2024-09-16T00:07:38:635384+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2773358731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2773358731},{"type":"v1","addr":"172.21.15.139:6835","nonce":2773358731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4268,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/2919602297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":2919602297},{"type":"v1","addr":"172.21.15.164:6837","nonce":2919602297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3575043661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":3575043661},{"type":"v1","addr":"172.21.15.139:6837","nonce":3575043661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/4113980181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":4113980181},{"type":"v1","addr":"172.21.15.139:6839","nonce":4113980181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/374774889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":374774889},{"type":"v1","addr":"172.21.15.164:6834","nonce":374774889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T00:07:32.949865+0000","modified":"2024-09-16T00:07:38.635380+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.164:6836/4279698936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":4279698936},{"type":"v1","addr":"172.21.15.164:6836","nonce":4279698936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-09-16T00:11:46.487 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 8 2024-09-16T00:11:46.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.488+0000 7f590f265640 1 -- 172.21.15.139:0/2100610628 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f58ec06c2b0 msgr2=0x7f58ec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.488+0000 7f590f265640 1 --2- 172.21.15.139:0/2100610628 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f58ec06c2b0 0x7f58ec06e770 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5904034000 tx=0x7f590402e000 comp rx=0 tx=0).stop 2024-09-16T00:11:46.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.488+0000 7f590f265640 1 -- 172.21.15.139:0/2100610628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080c57a0 msgr2=0x7f5908155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.488+0000 7f590f265640 1 --2- 172.21.15.139:0/2100610628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080c57a0 0x7f5908155770 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f58f800c390 tx=0x7f58f800c790 comp rx=0 tx=0).stop 2024-09-16T00:11:46.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.488+0000 7f590f265640 1 -- 172.21.15.139:0/2100610628 shutdown_connections 2024-09-16T00:11:46.489 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.488+0000 7f590f265640 1 --2- 172.21.15.139:0/2100610628 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f58ec06c2b0 0x7f58ec06e770 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.488+0000 7f590f265640 1 --2- 172.21.15.139:0/2100610628 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f590815bc50 0x7f59080cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.488+0000 7f590f265640 1 --2- 172.21.15.139:0/2100610628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59080c57a0 0x7f5908155770 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.488+0000 7f590f265640 1 --2- 172.21.15.139:0/2100610628 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f59080c3020 0x7f5908154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.488+0000 7f590f265640 1 -- 172.21.15.139:0/2100610628 >> 172.21.15.139:0/2100610628 conn(0x7f5908067950 msgr2=0x7f590815a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:46.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.488+0000 7f590f265640 1 -- 172.21.15.139:0/2100610628 shutdown_connections 2024-09-16T00:11:46.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.488+0000 7f590f265640 1 -- 172.21.15.139:0/2100610628 wait complete. 2024-09-16T00:11:46.500 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-16T00:11:46.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.598+0000 7faf9d52a640 1 Processor -- start 2024-09-16T00:11:46.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.598+0000 7faf9d52a640 1 -- start start 2024-09-16T00:11:46.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.599+0000 7faf9d52a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faf980cbe50 0x7faf980cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.599+0000 7faf9d52a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf9806ca30 con 0x7faf980c7b90 2024-09-16T00:11:46.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.599+0000 7faf9d52a640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf9815df20 con 0x7faf980c67e0 2024-09-16T00:11:46.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.599+0000 7faf9d52a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7faf9806cd10 con 0x7faf980cbe50 2024-09-16T00:11:46.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.599+0000 7faf977fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faf980cbe50 0x7faf980cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.599+0000 7faf8ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7faf980c7b90 0x7faf9815a810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56448/0 (socket says 172.21.15.139:56448) 2024-09-16T00:11:46.600 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.599+0000 7faf8ffff640 1 -- 172.21.15.139:0/1036557856 learned_addr learned my addr 172.21.15.139:0/1036557856 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:46.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.599+0000 7faf977fe640 1 -- 172.21.15.139:0/1036557856 >> v1:172.21.15.164:6789/0 conn(0x7faf980c67e0 legacy=0x7faf9806c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.599+0000 7faf977fe640 1 -- 172.21.15.139:0/1036557856 >> v1:172.21.15.139:6789/0 conn(0x7faf980c7b90 legacy=0x7faf9815a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.599+0000 7faf977fe640 1 -- 172.21.15.139:0/1036557856 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9815e200 con 0x7faf980cbe50 2024-09-16T00:11:46.601 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.599+0000 7faf967fc640 1 -- 172.21.15.139:0/1036557856 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2445723131 0 0) 0x7faf9806ca30 con 0x7faf980c7b90 2024-09-16T00:11:46.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.600+0000 7faf977fe640 1 --2- 172.21.15.139:0/1036557856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf980cbe50 0x7faf980cc2a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7faf90009d90 tx=0x7faf9002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a4b7ca9fad2db8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:46.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.600+0000 7faf967fc640 1 -- 172.21.15.139:0/1036557856 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf90037070 con 0x7faf980cbe50 2024-09-16T00:11:46.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.600+0000 7faf967fc640 1 -- 172.21.15.139:0/1036557856 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7faf90007740 con 0x7faf980cbe50 2024-09-16T00:11:46.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.600+0000 7faf967fc640 1 -- 172.21.15.139:0/1036557856 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf90007b30 con 0x7faf980cbe50 2024-09-16T00:11:46.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.601+0000 7faf9d52a640 1 -- 172.21.15.139:0/1036557856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf980cbe50 msgr2=0x7faf980cc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.601+0000 7faf9d52a640 1 --2- 172.21.15.139:0/1036557856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf980cbe50 0x7faf980cc2a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7faf90009d90 tx=0x7faf9002a780 comp rx=0 tx=0).stop 2024-09-16T00:11:46.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.601+0000 7faf9d52a640 1 -- 172.21.15.139:0/1036557856 shutdown_connections 2024-09-16T00:11:46.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.601+0000 7faf9d52a640 1 --2- 172.21.15.139:0/1036557856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf980cbe50 0x7faf980cc2a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.601+0000 7faf9d52a640 1 -- 172.21.15.139:0/1036557856 >> 172.21.15.139:0/1036557856 conn(0x7faf980d05f0 msgr2=0x7faf980d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:46.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.601+0000 7faf9d52a640 1 -- 172.21.15.139:0/1036557856 shutdown_connections 2024-09-16T00:11:46.603 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.601+0000 7faf9d52a640 1 -- 172.21.15.139:0/1036557856 wait complete. 2024-09-16T00:11:46.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.602+0000 7faf9d52a640 1 Processor -- start 2024-09-16T00:11:46.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.602+0000 7faf9d52a640 1 -- start start 2024-09-16T00:11:46.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.602+0000 7faf9d52a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf980c67e0 0x7faf9814d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.602+0000 7faf9d52a640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf980c7b90 0x7faf981528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.602+0000 7faf9d52a640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf980cbe50 0x7faf98152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.602+0000 7faf9d52a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faf980c7fe0 con 0x7faf980c67e0 2024-09-16T00:11:46.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.602+0000 7faf9d52a640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7faf9815e850 con 0x7faf980c7b90 2024-09-16T00:11:46.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.602+0000 7faf9d52a640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7faf9815af20 con 0x7faf980cbe50 2024-09-16T00:11:46.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.603+0000 7faf96ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf980c67e0 0x7faf9814d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.603+0000 7faf96ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf980c67e0 0x7faf9814d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45330/0 (socket says 172.21.15.139:45330) 2024-09-16T00:11:46.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.603+0000 7faf96ffd640 1 -- 172.21.15.139:0/1003810314 learned_addr learned my addr 172.21.15.139:0/1003810314 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:46.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.603+0000 7faf96ffd640 1 -- 172.21.15.139:0/1003810314 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf980cbe50 msgr2=0x7faf98152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.603+0000 7faf96ffd640 1 --2- 172.21.15.139:0/1003810314 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf980cbe50 0x7faf98152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.603+0000 7faf96ffd640 1 -- 172.21.15.139:0/1003810314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf980c7b90 msgr2=0x7faf981528c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.603+0000 7faf8ffff640 1 --2- 172.21.15.139:0/1003810314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf980c7b90 0x7faf981528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.603+0000 7faf96ffd640 1 --2- 172.21.15.139:0/1003810314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf980c7b90 0x7faf981528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.603+0000 7faf96ffd640 1 -- 172.21.15.139:0/1003810314 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf980c45a0 con 0x7faf980c67e0 2024-09-16T00:11:46.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.603+0000 7faf96ffd640 1 --2- 172.21.15.139:0/1003810314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf980c67e0 0x7faf9814d7e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7faf8800c480 tx=0x7faf8800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:46.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.603+0000 7faf8f7fe640 1 -- 172.21.15.139:0/1003810314 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf880052d0 con 0x7faf980c67e0 2024-09-16T00:11:46.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.603+0000 7faf8f7fe640 1 -- 172.21.15.139:0/1003810314 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7faf88005470 con 0x7faf980c67e0 2024-09-16T00:11:46.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.603+0000 7faf9d52a640 1 -- 172.21.15.139:0/1003810314 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf980c4880 con 0x7faf980c67e0 2024-09-16T00:11:46.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.603+0000 7faf8f7fe640 1 -- 172.21.15.139:0/1003810314 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88012890 con 0x7faf980c67e0 2024-09-16T00:11:46.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.603+0000 7faf9d52a640 1 -- 172.21.15.139:0/1003810314 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf981e3dd0 con 0x7faf980c67e0 2024-09-16T00:11:46.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.604+0000 7faf9d52a640 1 -- 172.21.15.139:0/1003810314 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf980cc2a0 con 0x7faf980c67e0 2024-09-16T00:11:46.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.605+0000 7faf8f7fe640 1 -- 172.21.15.139:0/1003810314 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7faf88012a30 con 0x7faf980c67e0 2024-09-16T00:11:46.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.605+0000 7faf8f7fe640 1 --2- 172.21.15.139:0/1003810314 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faf7806c2b0 0x7faf7806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.605+0000 7faf8f7fe640 1 -- 172.21.15.139:0/1003810314 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7faf8808b130 con 0x7faf980c67e0 2024-09-16T00:11:46.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.605+0000 7faf8ffff640 1 --2- 172.21.15.139:0/1003810314 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faf7806c2b0 0x7faf7806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.605+0000 7faf8ffff640 1 --2- 172.21.15.139:0/1003810314 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faf7806c2b0 0x7faf7806e770 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faf80007fd0 tx=0x7faf80009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:46.610 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.609+0000 7faf8f7fe640 1 -- 172.21.15.139:0/1003810314 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7faf88059840 con 0x7faf980c67e0 2024-09-16T00:11:46.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.807+0000 7faf9d52a640 1 -- 172.21.15.139:0/1003810314 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf980cbc30 con 0x7faf980c67e0 2024-09-16T00:11:46.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.807+0000 7faf8f7fe640 1 -- 172.21.15.139:0/1003810314 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6881 (secure 0 0 0) 0x7faf88059140 con 0x7faf980c67e0 2024-09-16T00:11:46.808 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:11:46.809 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":8,"btime":"2024-09-16T00:07:38:635384+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2773358731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2773358731},{"type":"v1","addr":"172.21.15.139:6835","nonce":2773358731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4268,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/2919602297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":2919602297},{"type":"v1","addr":"172.21.15.164:6837","nonce":2919602297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/3575043661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":3575043661},{"type":"v1","addr":"172.21.15.139:6837","nonce":3575043661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/4113980181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":4113980181},{"type":"v1","addr":"172.21.15.139:6839","nonce":4113980181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6834/374774889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6832","nonce":374774889},{"type":"v1","addr":"172.21.15.164:6834","nonce":374774889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T00:07:32.949865+0000","modified":"2024-09-16T00:07:38.635380+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.164:6836/4279698936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6833","nonce":4279698936},{"type":"v1","addr":"172.21.15.164:6836","nonce":4279698936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4300,"qdb_cluster":[4300]},"id":1}]} 2024-09-16T00:11:46.809 INFO:teuthology.orchestra.run.smithi139.stderr:dumped fsmap epoch 8 2024-09-16T00:11:46.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.809+0000 7faf9d52a640 1 -- 172.21.15.139:0/1003810314 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faf7806c2b0 msgr2=0x7faf7806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.809+0000 7faf9d52a640 1 --2- 172.21.15.139:0/1003810314 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faf7806c2b0 0x7faf7806e770 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faf80007fd0 tx=0x7faf80009040 comp rx=0 tx=0).stop 2024-09-16T00:11:46.810 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.809+0000 7faf9d52a640 1 -- 172.21.15.139:0/1003810314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf980c67e0 msgr2=0x7faf9814d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.809+0000 7faf9d52a640 1 --2- 172.21.15.139:0/1003810314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf980c67e0 0x7faf9814d7e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7faf8800c480 tx=0x7faf8800c940 comp rx=0 tx=0).stop 2024-09-16T00:11:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.809+0000 7faf9d52a640 1 -- 172.21.15.139:0/1003810314 shutdown_connections 2024-09-16T00:11:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.809+0000 7faf9d52a640 1 --2- 172.21.15.139:0/1003810314 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7faf7806c2b0 0x7faf7806e770 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.809+0000 7faf9d52a640 1 --2- 172.21.15.139:0/1003810314 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7faf980cbe50 0x7faf98152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.809+0000 7faf9d52a640 1 --2- 172.21.15.139:0/1003810314 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7faf980c7b90 0x7faf981528c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.809+0000 7faf9d52a640 1 --2- 172.21.15.139:0/1003810314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf980c67e0 0x7faf9814d7e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.809+0000 7faf9d52a640 1 -- 172.21.15.139:0/1003810314 >> 172.21.15.139:0/1003810314 conn(0x7faf980d05f0 msgr2=0x7faf98153a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.809+0000 7faf9d52a640 1 -- 172.21.15.139:0/1003810314 shutdown_connections 2024-09-16T00:11:46.811 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.809+0000 7faf9d52a640 1 -- 172.21.15.139:0/1003810314 wait complete. 2024-09-16T00:11:46.821 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:11:46.919 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.919+0000 7fbb89a4c640 1 Processor -- start 2024-09-16T00:11:46.920 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.919+0000 7fbb89a4c640 1 -- start start 2024-09-16T00:11:46.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.919+0000 7fbb89a4c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbb840c7b90 0x7fbb8415a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.919+0000 7fbb89a4c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb8406ca30 con 0x7fbb8415ad50 2024-09-16T00:11:46.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.919+0000 7fbb89a4c640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb8415df20 con 0x7fbb840c67e0 2024-09-16T00:11:46.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.919+0000 7fbb89a4c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbb8406cd10 con 0x7fbb840c7b90 2024-09-16T00:11:46.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.919+0000 7fbb827fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbb840c7b90 0x7fbb8415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.919+0000 7fbb837fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fbb8415ad50 0x7fbb840cbe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:56452/0 (socket says 172.21.15.139:56452) 2024-09-16T00:11:46.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.920+0000 7fbb837fe640 1 -- 172.21.15.139:0/4266870203 learned_addr learned my addr 172.21.15.139:0/4266870203 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:46.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.920+0000 7fbb827fc640 1 -- 172.21.15.139:0/4266870203 >> v1:172.21.15.164:6789/0 conn(0x7fbb840c67e0 legacy=0x7fbb8406c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.920+0000 7fbb827fc640 1 -- 172.21.15.139:0/4266870203 >> v1:172.21.15.139:6789/0 conn(0x7fbb8415ad50 legacy=0x7fbb840cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.920+0000 7fbb81ffb640 1 -- 172.21.15.139:0/4266870203 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2738400332 0 0) 0x7fbb8406ca30 con 0x7fbb8415ad50 2024-09-16T00:11:46.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.920+0000 7fbb827fc640 1 -- 172.21.15.139:0/4266870203 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8415e200 con 0x7fbb840c7b90 2024-09-16T00:11:46.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.920+0000 7fbb827fc640 1 --2- 172.21.15.139:0/4266870203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb840c7b90 0x7fbb8415a810 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0097a0 tx=0x7fbb6c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36a9463e6c1dcba server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.921+0000 7fbb81ffb640 1 -- 172.21.15.139:0/4266870203 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c037070 con 0x7fbb840c7b90 2024-09-16T00:11:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.921+0000 7fbb81ffb640 1 -- 172.21.15.139:0/4266870203 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbb6c032640 con 0x7fbb840c7b90 2024-09-16T00:11:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.921+0000 7fbb81ffb640 1 -- 172.21.15.139:0/4266870203 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c032a30 con 0x7fbb840c7b90 2024-09-16T00:11:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.921+0000 7fbb89a4c640 1 -- 172.21.15.139:0/4266870203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb840c7b90 msgr2=0x7fbb8415a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.921+0000 7fbb89a4c640 1 --2- 172.21.15.139:0/4266870203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb840c7b90 0x7fbb8415a810 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0097a0 tx=0x7fbb6c02a2c0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.921+0000 7fbb89a4c640 1 -- 172.21.15.139:0/4266870203 shutdown_connections 2024-09-16T00:11:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.921+0000 7fbb89a4c640 1 --2- 172.21.15.139:0/4266870203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb840c7b90 0x7fbb8415a810 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.921+0000 7fbb89a4c640 1 -- 172.21.15.139:0/4266870203 >> 172.21.15.139:0/4266870203 conn(0x7fbb840d05f0 msgr2=0x7fbb840d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.921+0000 7fbb89a4c640 1 -- 172.21.15.139:0/4266870203 shutdown_connections 2024-09-16T00:11:46.923 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.921+0000 7fbb89a4c640 1 -- 172.21.15.139:0/4266870203 wait complete. 2024-09-16T00:11:46.925 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.922+0000 7fbb89a4c640 1 Processor -- start 2024-09-16T00:11:46.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.922+0000 7fbb89a4c640 1 -- start start 2024-09-16T00:11:46.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.923+0000 7fbb89a4c640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb840c67e0 0x7fbb841e40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.923+0000 7fbb89a4c640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb840c7b90 0x7fbb841f1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.923+0000 7fbb89a4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb8415ad50 0x7fbb840c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.923+0000 7fbb89a4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbb840c7fe0 con 0x7fbb8415ad50 2024-09-16T00:11:46.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.923+0000 7fbb89a4c640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8415e850 con 0x7fbb840c7b90 2024-09-16T00:11:46.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.923+0000 7fbb89a4c640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7fbb840cc560 con 0x7fbb840c67e0 2024-09-16T00:11:46.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.923+0000 7fbb837fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb8415ad50 0x7fbb840c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.923+0000 7fbb837fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb8415ad50 0x7fbb840c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:45352/0 (socket says 172.21.15.139:45352) 2024-09-16T00:11:46.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.923+0000 7fbb837fe640 1 -- 172.21.15.139:0/934022170 learned_addr learned my addr 172.21.15.139:0/934022170 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:46.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.923+0000 7fbb827fc640 1 --2- 172.21.15.139:0/934022170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb840c7b90 0x7fbb841f1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.923+0000 7fbb82ffd640 1 --2- 172.21.15.139:0/934022170 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb840c67e0 0x7fbb841e40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.926 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.924+0000 7fbb837fe640 1 -- 172.21.15.139:0/934022170 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb840c67e0 msgr2=0x7fbb841e40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.924+0000 7fbb837fe640 1 --2- 172.21.15.139:0/934022170 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb840c67e0 0x7fbb841e40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.924+0000 7fbb837fe640 1 -- 172.21.15.139:0/934022170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb840c7b90 msgr2=0x7fbb841f1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:46.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.924+0000 7fbb837fe640 1 --2- 172.21.15.139:0/934022170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb840c7b90 0x7fbb841f1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:46.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.924+0000 7fbb837fe640 1 -- 172.21.15.139:0/934022170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb841e4620 con 0x7fbb8415ad50 2024-09-16T00:11:46.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.924+0000 7fbb837fe640 1 --2- 172.21.15.139:0/934022170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb8415ad50 0x7fbb840c83d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00c410 tx=0x7fbb7c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:46.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.924+0000 7fbb7bfff640 1 -- 172.21.15.139:0/934022170 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c0074e0 con 0x7fbb8415ad50 2024-09-16T00:11:46.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.924+0000 7fbb7bfff640 1 -- 172.21.15.139:0/934022170 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbb7c0047e0 con 0x7fbb8415ad50 2024-09-16T00:11:46.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.924+0000 7fbb7bfff640 1 -- 172.21.15.139:0/934022170 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c004bf0 con 0x7fbb8415ad50 2024-09-16T00:11:46.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.924+0000 7fbb89a4c640 1 -- 172.21.15.139:0/934022170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb841570c0 con 0x7fbb8415ad50 2024-09-16T00:11:46.927 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.924+0000 7fbb89a4c640 1 -- 172.21.15.139:0/934022170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb84157680 con 0x7fbb8415ad50 2024-09-16T00:11:46.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.925+0000 7fbb89a4c640 1 -- 172.21.15.139:0/934022170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb84159b50 con 0x7fbb8415ad50 2024-09-16T00:11:46.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.925+0000 7fbb7bfff640 1 -- 172.21.15.139:0/934022170 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbb7c011070 con 0x7fbb8415ad50 2024-09-16T00:11:46.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.926+0000 7fbb7bfff640 1 --2- 172.21.15.139:0/934022170 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb6406c2b0 0x7fbb6406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:46.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.926+0000 7fbb82ffd640 1 --2- 172.21.15.139:0/934022170 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb6406c2b0 0x7fbb6406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:46.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.926+0000 7fbb7bfff640 1 -- 172.21.15.139:0/934022170 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7fbb7c08afe0 con 0x7fbb8415ad50 2024-09-16T00:11:46.928 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.926+0000 7fbb82ffd640 1 --2- 172.21.15.139:0/934022170 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb6406c2b0 0x7fbb6406e770 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbb740012e0 tx=0x7fbb74009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:46.931 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:46.930+0000 7fbb7bfff640 1 -- 172.21.15.139:0/934022170 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbb84159b50 con 0x7fbb8415ad50 2024-09-16T00:11:47.099 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.098+0000 7fbb89a4c640 1 -- 172.21.15.139:0/934022170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb8415a810 con 0x7fbb8415ad50 2024-09-16T00:11:47.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.099+0000 7fbb7bfff640 1 -- 172.21.15.139:0/934022170 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+17050 (secure 0 0 0) 0x7fbb841e4620 con 0x7fbb8415ad50 2024-09-16T00:11:47.100 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-16T00:11:47.101 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":146,"fsid":"2d3366e5-6a59-4b59-93b2-615e72ac7015","created":"2024-09-16T00:07:09.967833+0000","modified":"2024-09-16T00:11:13.946870+0000","last_up_change":"2024-09-16T00:10:54.809507+0000","last_in_change":"2024-09-16T00:07:12.025620+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:07:22.056303+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:07:30.768649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:07:32.008209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83699a05-4937-43c3-aefc-b669bed9bb6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6817","nonce":1974901438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6819","nonce":1974901438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6823","nonce":1974901438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":1974901438},{"type":"v1","addr":"172.21.15.139:6821","nonce":1974901438}]},"public_addr":"172.21.15.139:6817/1974901438","cluster_addr":"172.21.15.139:6819/1974901438","heartbeat_back_addr":"172.21.15.139:6823/1974901438","heartbeat_front_addr":"172.21.15.139:6821/1974901438","state":["exists","up"]},{"osd":1,"uuid":"9c04991e-9695-4bbb-8011-bf613bf404b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6809","nonce":2232380998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6811","nonce":2232380998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6815","nonce":2232380998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2232380998},{"type":"v1","addr":"172.21.15.139:6813","nonce":2232380998}]},"public_addr":"172.21.15.139:6809/2232380998","cluster_addr":"172.21.15.139:6811/2232380998","heartbeat_back_addr":"172.21.15.139:6815/2232380998","heartbeat_front_addr":"172.21.15.139:6813/2232380998","state":["exists","up"]},{"osd":2,"uuid":"8048fc64-72b3-4866-ac62-6a8b2bf6f5c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6801","nonce":4180588905}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6803","nonce":4180588905}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6807","nonce":4180588905}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":4180588905},{"type":"v1","addr":"172.21.15.139:6805","nonce":4180588905}]},"public_addr":"172.21.15.139:6801/4180588905","cluster_addr":"172.21.15.139:6803/4180588905","heartbeat_back_addr":"172.21.15.139:6807/4180588905","heartbeat_front_addr":"172.21.15.139:6805/4180588905","state":["exists","up"]},{"osd":3,"uuid":"c34ac77b-354c-4884-902d-3e2a84f5eb27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6827","nonce":3947768492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6829","nonce":3947768492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6833","nonce":3947768492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":3947768492},{"type":"v1","addr":"172.21.15.139:6831","nonce":3947768492}]},"public_addr":"172.21.15.139:6827/3947768492","cluster_addr":"172.21.15.139:6829/3947768492","heartbeat_back_addr":"172.21.15.139:6833/3947768492","heartbeat_front_addr":"172.21.15.139:6831/3947768492","state":["exists","up"]},{"osd":4,"uuid":"ba9a7c93-9672-43e1-8c98-2878cdf7f548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6817","nonce":642632916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6818","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6819","nonce":642632916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6822","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6823","nonce":642632916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":642632916},{"type":"v1","addr":"172.21.15.164:6821","nonce":642632916}]},"public_addr":"172.21.15.164:6817/642632916","cluster_addr":"172.21.15.164:6819/642632916","heartbeat_back_addr":"172.21.15.164:6823/642632916","heartbeat_front_addr":"172.21.15.164:6821/642632916","state":["exists","up"]},{"osd":5,"uuid":"46a0916b-474c-411b-84ad-86f27d4cbf53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6801","nonce":1051778383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6803","nonce":1051778383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6807","nonce":1051778383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":1051778383},{"type":"v1","addr":"172.21.15.164:6805","nonce":1051778383}]},"public_addr":"172.21.15.164:6801/1051778383","cluster_addr":"172.21.15.164:6803/1051778383","heartbeat_back_addr":"172.21.15.164:6807/1051778383","heartbeat_front_addr":"172.21.15.164:6805/1051778383","state":["exists","up"]},{"osd":6,"uuid":"9b6b33cc-d6d9-4a0a-8143-526073a58b93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":82,"up_from":94,"up_thru":127,"down_at":84,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":164138206},{"type":"v1","addr":"172.21.15.164:6825","nonce":164138206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":164138206},{"type":"v1","addr":"172.21.15.164:6827","nonce":164138206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":164138206},{"type":"v1","addr":"172.21.15.164:6831","nonce":164138206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":164138206},{"type":"v1","addr":"172.21.15.164:6829","nonce":164138206}]},"public_addr":"172.21.15.164:6825/164138206","cluster_addr":"172.21.15.164:6827/164138206","heartbeat_back_addr":"172.21.15.164:6831/164138206","heartbeat_front_addr":"172.21.15.164:6829/164138206","state":["exists","up"]},{"osd":7,"uuid":"50bcd808-3d3e-4a72-88e4-30decb53cfea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":112,"up_from":127,"up_thru":136,"down_at":113,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":2782017910},{"type":"v1","addr":"172.21.15.164:6809","nonce":2782017910}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6810","nonce":2782017910},{"type":"v1","addr":"172.21.15.164:6811","nonce":2782017910}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6814","nonce":2782017910},{"type":"v1","addr":"172.21.15.164:6815","nonce":2782017910}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":2782017910},{"type":"v1","addr":"172.21.15.164:6813","nonce":2782017910}]},"public_addr":"172.21.15.164:6809/2782017910","cluster_addr":"172.21.15.164:6811/2782017910","heartbeat_back_addr":"172.21.15.164:6815/2782017910","heartbeat_front_addr":"172.21.15.164:6813/2782017910","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.843117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.622789+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:17.919457+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:19.330341+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.815640+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.083046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.551499+0000","dead_epoch":84},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:07:18.377179+0000","dead_epoch":113}],"pg_upmap":[{"pgid":"3.2d","osds":[7,5]}],"pg_upmap_items":[{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:11:47.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.101+0000 7fbb89a4c640 1 -- 172.21.15.139:0/934022170 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb6406c2b0 msgr2=0x7fbb6406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.101+0000 7fbb89a4c640 1 --2- 172.21.15.139:0/934022170 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb6406c2b0 0x7fbb6406e770 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbb740012e0 tx=0x7fbb74009040 comp rx=0 tx=0).stop 2024-09-16T00:11:47.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.101+0000 7fbb89a4c640 1 -- 172.21.15.139:0/934022170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb8415ad50 msgr2=0x7fbb840c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.101+0000 7fbb89a4c640 1 --2- 172.21.15.139:0/934022170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb8415ad50 0x7fbb840c83d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c00c410 tx=0x7fbb7c00c8d0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.101+0000 7fbb89a4c640 1 -- 172.21.15.139:0/934022170 shutdown_connections 2024-09-16T00:11:47.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.101+0000 7fbb89a4c640 1 --2- 172.21.15.139:0/934022170 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7fbb6406c2b0 0x7fbb6406e770 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.101+0000 7fbb89a4c640 1 --2- 172.21.15.139:0/934022170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb8415ad50 0x7fbb840c83d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.101+0000 7fbb89a4c640 1 --2- 172.21.15.139:0/934022170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbb840c7b90 0x7fbb841f1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.101+0000 7fbb89a4c640 1 --2- 172.21.15.139:0/934022170 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7fbb840c67e0 0x7fbb841e40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.102+0000 7fbb89a4c640 1 -- 172.21.15.139:0/934022170 >> 172.21.15.139:0/934022170 conn(0x7fbb840d05f0 msgr2=0x7fbb840d09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:47.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.102+0000 7fbb89a4c640 1 -- 172.21.15.139:0/934022170 shutdown_connections 2024-09-16T00:11:47.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.102+0000 7fbb89a4c640 1 -- 172.21.15.139:0/934022170 wait complete. 2024-09-16T00:11:47.114 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-16T00:11:47.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.213+0000 7f82519cd640 1 Processor -- start 2024-09-16T00:11:47.214 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.213+0000 7f82519cd640 1 -- start start 2024-09-16T00:11:47.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f82519cd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f824c0d0900 0x7f824c0c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f82519cd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f824c0b50e0 con 0x7f824c0d0520 2024-09-16T00:11:47.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f82519cd640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f824c15c290 con 0x7f824c15d760 2024-09-16T00:11:47.217 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f82519cd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f824c0c77f0 con 0x7f824c0d0900 2024-09-16T00:11:47.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824affd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f824c0d0520 0x7f824c15d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:37410/0 (socket says 172.21.15.139:37410) 2024-09-16T00:11:47.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824affd640 1 -- 172.21.15.139:0/1020641369 learned_addr learned my addr 172.21.15.139:0/1020641369 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-16T00:11:47.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824a7fc640 1 -- 172.21.15.139:0/1020641369 <== mon.0 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 86861081 0 0) 0x7f824c0b50e0 con 0x7f824c0d0520 2024-09-16T00:11:47.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f82427fc640 1 --2- 172.21.15.139:0/1020641369 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f824c0d0900 0x7f824c0c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824a7fc640 1 -- 172.21.15.139:0/1020641369 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8238003520 con 0x7f824c0d0520 2024-09-16T00:11:47.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824a7fc640 1 -- 172.21.15.139:0/1020641369 <== mon.0 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1579272785 0 0) 0x7f8238003520 con 0x7f824c0d0520 2024-09-16T00:11:47.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824a7fc640 1 -- 172.21.15.139:0/1020641369 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f824c0d0900 msgr2=0x7f824c0c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:47.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824a7fc640 1 --2- 172.21.15.139:0/1020641369 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f824c0d0900 0x7f824c0c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824a7fc640 1 -- 172.21.15.139:0/1020641369 >> v1:172.21.15.164:6789/0 conn(0x7f824c15d760 legacy=0x7f824c0ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824a7fc640 1 -- 172.21.15.139:0/1020641369 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c15c570 con 0x7f824c0d0520 2024-09-16T00:11:47.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824a7fc640 1 -- 172.21.15.139:0/1020641369 <== mon.0 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 110432025 0 0) 0x7f823c001670 con 0x7f824c0d0520 2024-09-16T00:11:47.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824a7fc640 1 -- 172.21.15.139:0/1020641369 >> v1:172.21.15.139:6789/0 conn(0x7f824c0d0520 legacy=0x7f824c15d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824a7fc640 1 --2- 172.21.15.139:0/1020641369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82380039e0 0x7f8238003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824a7fc640 1 --2- 172.21.15.139:0/1020641369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8238004800 0x7f8238004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824a7fc640 1 --2- 172.21.15.139:0/1020641369 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f824c0d0900 0x7f82380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824a7fc640 1 -- 172.21.15.139:0/1020641369 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f824c0c77f0 con 0x7f8238004800 2024-09-16T00:11:47.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824a7fc640 1 -- 172.21.15.139:0/1020641369 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f823800baa0 con 0x7f82380039e0 2024-09-16T00:11:47.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.214+0000 7f824a7fc640 1 -- 172.21.15.139:0/1020641369 --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f823800bca0 con 0x7f824c0d0900 2024-09-16T00:11:47.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.215+0000 7f82427fc640 1 --2- 172.21.15.139:0/1020641369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8238004800 0x7f8238004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.215+0000 7f824affd640 1 --2- 172.21.15.139:0/1020641369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82380039e0 0x7f8238003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.215+0000 7f824b7fe640 1 --2- 172.21.15.139:0/1020641369 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f824c0d0900 0x7f82380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.215+0000 7f82427fc640 1 -- 172.21.15.139:0/1020641369 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f824c0d0900 msgr2=0x7f82380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:47.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.215+0000 7f82427fc640 1 --2- 172.21.15.139:0/1020641369 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f824c0d0900 0x7f82380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.215+0000 7f82427fc640 1 -- 172.21.15.139:0/1020641369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82380039e0 msgr2=0x7f8238003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:47.219 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.215+0000 7f82427fc640 1 --2- 172.21.15.139:0/1020641369 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f82380039e0 0x7f8238003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.215+0000 7f82427fc640 1 -- 172.21.15.139:0/1020641369 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8238003520 con 0x7f8238004800 2024-09-16T00:11:47.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.215+0000 7f82427fc640 1 --2- 172.21.15.139:0/1020641369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8238004800 0x7f8238004c80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f824c06d230 tx=0x7f8238005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6352dd559d386957 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:47.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.216+0000 7f824a7fc640 1 -- 172.21.15.139:0/1020641369 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234036070 con 0x7f8238004800 2024-09-16T00:11:47.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.216+0000 7f824a7fc640 1 -- 172.21.15.139:0/1020641369 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8234031400 con 0x7f8238004800 2024-09-16T00:11:47.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.216+0000 7f824a7fc640 1 -- 172.21.15.139:0/1020641369 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8234009690 con 0x7f8238004800 2024-09-16T00:11:47.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.216+0000 7f82519cd640 1 -- 172.21.15.139:0/1020641369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8238004800 msgr2=0x7f8238004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.216+0000 7f82519cd640 1 --2- 172.21.15.139:0/1020641369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8238004800 0x7f8238004c80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f824c06d230 tx=0x7f8238005830 comp rx=0 tx=0).stop 2024-09-16T00:11:47.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.216+0000 7f824b7fe640 1 -- 172.21.15.139:0/1020641369 reap_dead start 2024-09-16T00:11:47.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.216+0000 7f82519cd640 1 -- 172.21.15.139:0/1020641369 shutdown_connections 2024-09-16T00:11:47.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.216+0000 7f82519cd640 1 -- 172.21.15.139:0/1020641369 >> 172.21.15.139:0/1020641369 conn(0x7f824c06e850 msgr2=0x7f824c06c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:47.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.216+0000 7f82519cd640 1 -- 172.21.15.139:0/1020641369 shutdown_connections 2024-09-16T00:11:47.220 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.216+0000 7f82519cd640 1 -- 172.21.15.139:0/1020641369 wait complete. 2024-09-16T00:11:47.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.217+0000 7f82519cd640 1 Processor -- start 2024-09-16T00:11:47.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.218+0000 7f82519cd640 1 -- start start 2024-09-16T00:11:47.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.218+0000 7f82519cd640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8238004800 0x7f824c0c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.218+0000 7f82519cd640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f824c0c9620 0x7f824c150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.218+0000 7f82519cd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f824c0c7990 0x7f824c0c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.218+0000 7f82519cd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f824c15cbc0 con 0x7f824c0c7990 2024-09-16T00:11:47.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.218+0000 7f82519cd640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f824c15c040 con 0x7f824c0c9620 2024-09-16T00:11:47.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.218+0000 7f82519cd640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f824c0b35c0 con 0x7f8238004800 2024-09-16T00:11:47.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.218+0000 7f824b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f824c0c7990 0x7f824c0c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.218+0000 7f824b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f824c0c7990 0x7f824c0c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42370/0 (socket says 172.21.15.139:42370) 2024-09-16T00:11:47.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.218+0000 7f82427fc640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f824c0c9620 0x7f824c150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.218+0000 7f824b7fe640 1 -- 172.21.15.139:0/4203944385 learned_addr learned my addr 172.21.15.139:0/4203944385 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:47.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.218+0000 7f824affd640 1 --2- 172.21.15.139:0/4203944385 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8238004800 0x7f824c0c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.219+0000 7f824b7fe640 1 -- 172.21.15.139:0/4203944385 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8238004800 msgr2=0x7f824c0c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.219+0000 7f824b7fe640 1 --2- 172.21.15.139:0/4203944385 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8238004800 0x7f824c0c4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.219+0000 7f824b7fe640 1 -- 172.21.15.139:0/4203944385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f824c0c9620 msgr2=0x7f824c150930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.219+0000 7f824b7fe640 1 --2- 172.21.15.139:0/4203944385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f824c0c9620 0x7f824c150930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.219+0000 7f824b7fe640 1 -- 172.21.15.139:0/4203944385 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c0c46b0 con 0x7f824c0c7990 2024-09-16T00:11:47.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.219+0000 7f824b7fe640 1 --2- 172.21.15.139:0/4203944385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f824c0c7990 0x7f824c0c7e10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f824c15db40 tx=0x7f824400fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:47.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.219+0000 7f8248ff9640 1 -- 172.21.15.139:0/4203944385 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824401a070 con 0x7f824c0c7990 2024-09-16T00:11:47.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.219+0000 7f8248ff9640 1 -- 172.21.15.139:0/4203944385 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8244006d60 con 0x7f824c0c7990 2024-09-16T00:11:47.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.219+0000 7f82519cd640 1 -- 172.21.15.139:0/4203944385 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824c0c6ca0 con 0x7f824c0c7990 2024-09-16T00:11:47.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.219+0000 7f8248ff9640 1 -- 172.21.15.139:0/4203944385 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8244003bb0 con 0x7f824c0c7990 2024-09-16T00:11:47.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.219+0000 7f82519cd640 1 -- 172.21.15.139:0/4203944385 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f824c1f05f0 con 0x7f824c0c7990 2024-09-16T00:11:47.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.220+0000 7f82519cd640 1 -- 172.21.15.139:0/4203944385 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f824c15d050 con 0x7f824c0c7990 2024-09-16T00:11:47.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.221+0000 7f8248ff9640 1 -- 172.21.15.139:0/4203944385 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f824401d070 con 0x7f824c0c7990 2024-09-16T00:11:47.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.221+0000 7f8248ff9640 1 --2- 172.21.15.139:0/4203944385 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f822c06c2e0 0x7f822c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.221+0000 7f8248ff9640 1 -- 172.21.15.139:0/4203944385 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5944+0+0 (secure 0 0 0) 0x7f824408aa60 con 0x7f824c0c7990 2024-09-16T00:11:47.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.221+0000 7f824affd640 1 --2- 172.21.15.139:0/4203944385 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f822c06c2e0 0x7f822c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.221+0000 7f824affd640 1 --2- 172.21.15.139:0/4203944385 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f822c06c2e0 0x7f822c06e7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f823c000c00 tx=0x7f823c001110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:47.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.225+0000 7f8248ff9640 1 -- 172.21.15.139:0/4203944385 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8244058a00 con 0x7f824c0c7990 2024-09-16T00:11:47.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.424+0000 7f82519cd640 1 -- 172.21.15.139:0/4203944385 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f824c15eca0 con 0x7f824c0c7990 2024-09-16T00:11:47.427 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.426+0000 7f8248ff9640 1 -- 172.21.15.139:0/4203944385 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f824405cd00 con 0x7f824c0c7990 2024-09-16T00:11:47.427 INFO:teuthology.orchestra.run.smithi139.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T00:11:47.430 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.429+0000 7f82519cd640 1 -- 172.21.15.139:0/4203944385 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f822c06c2e0 msgr2=0x7f822c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.429+0000 7f82519cd640 1 --2- 172.21.15.139:0/4203944385 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f822c06c2e0 0x7f822c06e7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f823c000c00 tx=0x7f823c001110 comp rx=0 tx=0).stop 2024-09-16T00:11:47.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.429+0000 7f82519cd640 1 -- 172.21.15.139:0/4203944385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f824c0c7990 msgr2=0x7f824c0c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.429+0000 7f82519cd640 1 --2- 172.21.15.139:0/4203944385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f824c0c7990 0x7f824c0c7e10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f824c15db40 tx=0x7f824400fc80 comp rx=0 tx=0).stop 2024-09-16T00:11:47.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.429+0000 7f82519cd640 1 -- 172.21.15.139:0/4203944385 shutdown_connections 2024-09-16T00:11:47.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.429+0000 7f82519cd640 1 --2- 172.21.15.139:0/4203944385 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f822c06c2e0 0x7f822c06e7a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.429+0000 7f82519cd640 1 --2- 172.21.15.139:0/4203944385 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f824c0c7990 0x7f824c0c7e10 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.429+0000 7f82519cd640 1 --2- 172.21.15.139:0/4203944385 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f824c0c9620 0x7f824c150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.429+0000 7f82519cd640 1 --2- 172.21.15.139:0/4203944385 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f8238004800 0x7f824c0c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.429+0000 7f82519cd640 1 -- 172.21.15.139:0/4203944385 >> 172.21.15.139:0/4203944385 conn(0x7f824c06e850 msgr2=0x7f824c0cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:47.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.429+0000 7f82519cd640 1 -- 172.21.15.139:0/4203944385 shutdown_connections 2024-09-16T00:11:47.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.430+0000 7f82519cd640 1 -- 172.21.15.139:0/4203944385 wait complete. 2024-09-16T00:11:47.444 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T00:11:47.479 INFO:tasks.ceph.mds.f.smithi164.stdout:starting mds.f at 2024-09-16T00:11:47.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.545+0000 7f0805053640 1 Processor -- start 2024-09-16T00:11:47.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.545+0000 7f0805053640 1 -- start start 2024-09-16T00:11:47.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.546+0000 7f0805053640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0800154cb0 0x7f080006cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.546+0000 7f0805053640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08000c9d20 0x7f080006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.546+0000 7f0805053640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08000c8bc0 con 0x7f080006ea10 2024-09-16T00:11:47.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.546+0000 7f0805053640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f08000ca7b0 con 0x7f0800154cb0 2024-09-16T00:11:47.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.546+0000 7f0805053640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f08000cb2b0 con 0x7f08000c9d20 2024-09-16T00:11:47.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.546+0000 7f07fed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0800154cb0 0x7f080006cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.546+0000 7f07fed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0800154cb0 0x7f080006cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42374/0 (socket says 172.21.15.139:42374) 2024-09-16T00:11:47.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.546+0000 7f07fed9d640 1 -- 172.21.15.139:0/4209462437 learned_addr learned my addr 172.21.15.139:0/4209462437 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:47.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.546+0000 7f07ff59e640 1 --2- 172.21.15.139:0/4209462437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08000c9d20 0x7f080006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.546+0000 7f07fed9d640 1 -- 172.21.15.139:0/4209462437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08000c9d20 msgr2=0x7f080006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:47.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.546+0000 7f07fed9d640 1 --2- 172.21.15.139:0/4209462437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08000c9d20 0x7f080006d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.546+0000 7f07fed9d640 1 -- 172.21.15.139:0/4209462437 >> v1:172.21.15.164:6789/0 conn(0x7f080006ea10 legacy=0x7f0800071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:47.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.546+0000 7f07fed9d640 1 -- 172.21.15.139:0/4209462437 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08000c8e80 con 0x7f0800154cb0 2024-09-16T00:11:47.548 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.547+0000 7f07fed9d640 1 --2- 172.21.15.139:0/4209462437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0800154cb0 0x7f080006cc30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f08000ca930 tx=0x7f07f002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa91f2551d4ec1f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:47.549 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.547+0000 7f07fdd9b640 1 -- 172.21.15.139:0/4209462437 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0037070 con 0x7f0800154cb0 2024-09-16T00:11:47.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.547+0000 7f07fdd9b640 1 -- 172.21.15.139:0/4209462437 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f07f0032440 con 0x7f0800154cb0 2024-09-16T00:11:47.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.547+0000 7f07fdd9b640 1 -- 172.21.15.139:0/4209462437 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0032830 con 0x7f0800154cb0 2024-09-16T00:11:47.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.547+0000 7f0805053640 1 -- 172.21.15.139:0/4209462437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0800154cb0 msgr2=0x7f080006cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.547+0000 7f0805053640 1 --2- 172.21.15.139:0/4209462437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0800154cb0 0x7f080006cc30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f08000ca930 tx=0x7f07f002a250 comp rx=0 tx=0).stop 2024-09-16T00:11:47.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.547+0000 7f0805053640 1 -- 172.21.15.139:0/4209462437 shutdown_connections 2024-09-16T00:11:47.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.547+0000 7f0805053640 1 --2- 172.21.15.139:0/4209462437 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08000c9d20 0x7f080006d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.547+0000 7f0805053640 1 --2- 172.21.15.139:0/4209462437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0800154cb0 0x7f080006cc30 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.547+0000 7f0805053640 1 -- 172.21.15.139:0/4209462437 >> 172.21.15.139:0/4209462437 conn(0x7f080006bb20 msgr2=0x7f080006f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:47.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.547+0000 7f0805053640 1 -- 172.21.15.139:0/4209462437 shutdown_connections 2024-09-16T00:11:47.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.547+0000 7f0805053640 1 -- 172.21.15.139:0/4209462437 wait complete. 2024-09-16T00:11:47.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.549+0000 7f0805053640 1 Processor -- start 2024-09-16T00:11:47.550 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.549+0000 7f0805053640 1 -- start start 2024-09-16T00:11:47.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f0805053640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f080006ea10 0x7f08000cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f0805053640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08000c9d20 0x7f08001505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f0805053640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0800154cb0 0x7f0800156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f0805053640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f08000ca440 con 0x7f0800154cb0 2024-09-16T00:11:47.552 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f0805053640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f08000cafd0 con 0x7f080006ea10 2024-09-16T00:11:47.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f0805053640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f080015a750 con 0x7f08000c9d20 2024-09-16T00:11:47.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f07ff59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0800154cb0 0x7f0800156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f07ff59e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0800154cb0 0x7f0800156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42380/0 (socket says 172.21.15.139:42380) 2024-09-16T00:11:47.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f07ff59e640 1 -- 172.21.15.139:0/4101717170 learned_addr learned my addr 172.21.15.139:0/4101717170 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:47.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f07fed9d640 1 --2- 172.21.15.139:0/4101717170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f080006ea10 0x7f08000cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f07fe59c640 1 --2- 172.21.15.139:0/4101717170 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08000c9d20 0x7f08001505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f07ff59e640 1 -- 172.21.15.139:0/4101717170 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08000c9d20 msgr2=0x7f08001505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f07ff59e640 1 --2- 172.21.15.139:0/4101717170 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08000c9d20 0x7f08001505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f07ff59e640 1 -- 172.21.15.139:0/4101717170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f080006ea10 msgr2=0x7f08000cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:47.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f07ff59e640 1 --2- 172.21.15.139:0/4101717170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f080006ea10 0x7f08000cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:47.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f07ff59e640 1 -- 172.21.15.139:0/4101717170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08001f3ea0 con 0x7f0800154cb0 2024-09-16T00:11:47.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f07ff59e640 1 --2- 172.21.15.139:0/4101717170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0800154cb0 0x7f0800156e20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f07f8007e70 tx=0x7f07f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:47.553 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f07e77fe640 1 -- 172.21.15.139:0/4101717170 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8014620 con 0x7f0800154cb0 2024-09-16T00:11:47.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f07e77fe640 1 -- 172.21.15.139:0/4101717170 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f07f8019070 con 0x7f0800154cb0 2024-09-16T00:11:47.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f07e77fe640 1 -- 172.21.15.139:0/4101717170 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f8014a40 con 0x7f0800154cb0 2024-09-16T00:11:47.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f0805053640 1 -- 172.21.15.139:0/4101717170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08001f4180 con 0x7f0800154cb0 2024-09-16T00:11:47.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.550+0000 7f0805053640 1 -- 172.21.15.139:0/4101717170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08001f4740 con 0x7f0800154cb0 2024-09-16T00:11:47.554 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.551+0000 7f0805053640 1 -- 172.21.15.139:0/4101717170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08000c5140 con 0x7f0800154cb0 2024-09-16T00:11:47.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.552+0000 7f07e77fe640 1 -- 172.21.15.139:0/4101717170 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f07f8014be0 con 0x7f0800154cb0 2024-09-16T00:11:47.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.552+0000 7f07e77fe640 1 --2- 172.21.15.139:0/4101717170 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07ec06c2b0 0x7f07ec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:47.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.552+0000 7f07e77fe640 1 -- 172.21.15.139:0/4101717170 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6030+0+0 (secure 0 0 0) 0x7f07f808bd10 con 0x7f0800154cb0 2024-09-16T00:11:47.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.552+0000 7f07fed9d640 1 --2- 172.21.15.139:0/4101717170 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07ec06c2b0 0x7f07ec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:47.555 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.553+0000 7f07fed9d640 1 --2- 172.21.15.139:0/4101717170 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07ec06c2b0 0x7f07ec06e770 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f07f0034000 tx=0x7f07f002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:47.557 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.557+0000 7f07e77fe640 1 -- 172.21.15.139:0/4101717170 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f07f8059cf0 con 0x7f0800154cb0 2024-09-16T00:11:47.758 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:47.757+0000 7f0805053640 1 -- 172.21.15.139:0/4101717170 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f08000cb870 con 0x7f0800154cb0 2024-09-16T00:11:48.434 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.433+0000 7f07e77fe640 1 -- 172.21.15.139:0/4101717170 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f07f805dff0 con 0x7f0800154cb0 2024-09-16T00:11:48.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.437+0000 7f0805053640 1 -- 172.21.15.139:0/4101717170 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07ec06c2b0 msgr2=0x7f07ec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:48.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.437+0000 7f0805053640 1 --2- 172.21.15.139:0/4101717170 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07ec06c2b0 0x7f07ec06e770 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f07f0034000 tx=0x7f07f002e000 comp rx=0 tx=0).stop 2024-09-16T00:11:48.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.437+0000 7f0805053640 1 -- 172.21.15.139:0/4101717170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0800154cb0 msgr2=0x7f0800156e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:48.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.437+0000 7f0805053640 1 --2- 172.21.15.139:0/4101717170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0800154cb0 0x7f0800156e20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f07f8007e70 tx=0x7f07f800e4b0 comp rx=0 tx=0).stop 2024-09-16T00:11:48.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.437+0000 7f0805053640 1 -- 172.21.15.139:0/4101717170 shutdown_connections 2024-09-16T00:11:48.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.437+0000 7f0805053640 1 --2- 172.21.15.139:0/4101717170 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f07ec06c2b0 0x7f07ec06e770 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:48.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.437+0000 7f0805053640 1 --2- 172.21.15.139:0/4101717170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0800154cb0 0x7f0800156e20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:48.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.437+0000 7f0805053640 1 --2- 172.21.15.139:0/4101717170 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f08000c9d20 0x7f08001505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:48.438 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.437+0000 7f0805053640 1 --2- 172.21.15.139:0/4101717170 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f080006ea10 0x7f08000cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:48.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.437+0000 7f0805053640 1 -- 172.21.15.139:0/4101717170 >> 172.21.15.139:0/4101717170 conn(0x7f080006bb20 msgr2=0x7f08001550c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:48.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.437+0000 7f0805053640 1 -- 172.21.15.139:0/4101717170 shutdown_connections 2024-09-16T00:11:48.439 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.437+0000 7f0805053640 1 -- 172.21.15.139:0/4101717170 wait complete. 2024-09-16T00:11:48.453 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T00:11:48.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.560+0000 7f70d6191640 1 Processor -- start 2024-09-16T00:11:48.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.560+0000 7f70d6191640 1 -- start start 2024-09-16T00:11:48.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.560+0000 7f70d6191640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f70d00c6570 0x7f70d006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:48.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.560+0000 7f70d6191640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70d015de10 0x7f70d015e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:48.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.560+0000 7f70d6191640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70d006bdf0 con 0x7f70d00c6950 2024-09-16T00:11:48.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.560+0000 7f70d6191640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f70d006c0d0 con 0x7f70d00c6570 2024-09-16T00:11:48.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.560+0000 7f70d6191640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f70d006c3b0 con 0x7f70d015de10 2024-09-16T00:11:48.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.561+0000 7f70cf7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f70d00c6570 0x7f70d006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:48.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.561+0000 7f70cf7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f70d00c6570 0x7f70d006d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42390/0 (socket says 172.21.15.139:42390) 2024-09-16T00:11:48.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.561+0000 7f70cf7fe640 1 -- 172.21.15.139:0/271365996 learned_addr learned my addr 172.21.15.139:0/271365996 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:48.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.561+0000 7f70cffff640 1 --2- 172.21.15.139:0/271365996 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70d015de10 0x7f70d015e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:48.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.561+0000 7f70cf7fe640 1 -- 172.21.15.139:0/271365996 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70d015de10 msgr2=0x7f70d015e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:48.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.561+0000 7f70cf7fe640 1 --2- 172.21.15.139:0/271365996 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70d015de10 0x7f70d015e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:48.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.561+0000 7f70cf7fe640 1 -- 172.21.15.139:0/271365996 >> v1:172.21.15.164:6789/0 conn(0x7f70d00c6950 legacy=0x7f70d00cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:48.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.561+0000 7f70cf7fe640 1 -- 172.21.15.139:0/271365996 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d006c690 con 0x7f70d00c6570 2024-09-16T00:11:48.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.561+0000 7f70cffff640 1 --2- 172.21.15.139:0/271365996 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70d015de10 0x7f70d015e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:11:48.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.562+0000 7f70cf7fe640 1 --2- 172.21.15.139:0/271365996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70d00c6570 0x7f70d006d170 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f70d008d740 tx=0x7f70c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85fc4642d30aace server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:48.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.562+0000 7f70ce7fc640 1 -- 172.21.15.139:0/271365996 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0037070 con 0x7f70d00c6570 2024-09-16T00:11:48.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.562+0000 7f70ce7fc640 1 -- 172.21.15.139:0/271365996 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f70c0032440 con 0x7f70d00c6570 2024-09-16T00:11:48.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.562+0000 7f70ce7fc640 1 -- 172.21.15.139:0/271365996 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0032830 con 0x7f70d00c6570 2024-09-16T00:11:48.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.562+0000 7f70d6191640 1 -- 172.21.15.139:0/271365996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70d00c6570 msgr2=0x7f70d006d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:48.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.562+0000 7f70d6191640 1 --2- 172.21.15.139:0/271365996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70d00c6570 0x7f70d006d170 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f70d008d740 tx=0x7f70c002a1e0 comp rx=0 tx=0).stop 2024-09-16T00:11:48.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.562+0000 7f70d6191640 1 -- 172.21.15.139:0/271365996 shutdown_connections 2024-09-16T00:11:48.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.562+0000 7f70d6191640 1 --2- 172.21.15.139:0/271365996 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70d015de10 0x7f70d015e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:48.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.562+0000 7f70d6191640 1 --2- 172.21.15.139:0/271365996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70d00c6570 0x7f70d006d170 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:48.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.562+0000 7f70d6191640 1 -- 172.21.15.139:0/271365996 >> 172.21.15.139:0/271365996 conn(0x7f70d00d01f0 msgr2=0x7f70d00d0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:48.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.562+0000 7f70d6191640 1 -- 172.21.15.139:0/271365996 shutdown_connections 2024-09-16T00:11:48.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.562+0000 7f70d6191640 1 -- 172.21.15.139:0/271365996 wait complete. 2024-09-16T00:11:48.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.563+0000 7f70d6191640 1 Processor -- start 2024-09-16T00:11:48.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.564+0000 7f70d6191640 1 -- start start 2024-09-16T00:11:48.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.564+0000 7f70d6191640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70d00c6570 0x7f70d00cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:48.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.564+0000 7f70d6191640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70d00c6950 0x7f70d01f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:48.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.564+0000 7f70d6191640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70d015de10 0x7f70d01f0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:48.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.564+0000 7f70d6191640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f70d00cc5b0 con 0x7f70d00c6570 2024-09-16T00:11:48.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.564+0000 7f70d6191640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f70d006cc50 con 0x7f70d00c6950 2024-09-16T00:11:48.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.564+0000 7f70d6191640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f70d006d6b0 con 0x7f70d015de10 2024-09-16T00:11:48.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.564+0000 7f70cf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70d00c6570 0x7f70d00cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:48.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.564+0000 7f70cf7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70d00c6570 0x7f70d00cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42406/0 (socket says 172.21.15.139:42406) 2024-09-16T00:11:48.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.564+0000 7f70cf7fe640 1 -- 172.21.15.139:0/2758019440 learned_addr learned my addr 172.21.15.139:0/2758019440 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:48.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.565+0000 7f70cffff640 1 --2- 172.21.15.139:0/2758019440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70d015de10 0x7f70d01f0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:48.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.565+0000 7f70ceffd640 1 --2- 172.21.15.139:0/2758019440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70d00c6950 0x7f70d01f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:48.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.565+0000 7f70cf7fe640 1 -- 172.21.15.139:0/2758019440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70d015de10 msgr2=0x7f70d01f0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:48.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.565+0000 7f70cf7fe640 1 --2- 172.21.15.139:0/2758019440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70d015de10 0x7f70d01f0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:48.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.565+0000 7f70cf7fe640 1 -- 172.21.15.139:0/2758019440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70d00c6950 msgr2=0x7f70d01f02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:48.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.565+0000 7f70cf7fe640 1 --2- 172.21.15.139:0/2758019440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70d00c6950 0x7f70d01f02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:48.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.565+0000 7f70cf7fe640 1 -- 172.21.15.139:0/2758019440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d0150930 con 0x7f70d00c6570 2024-09-16T00:11:48.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.565+0000 7f70cf7fe640 1 --2- 172.21.15.139:0/2758019440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70d00c6570 0x7f70d00cb810 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f70c0000c00 tx=0x7f70c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:48.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.565+0000 7f70b7fff640 1 -- 172.21.15.139:0/2758019440 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0037040 con 0x7f70d00c6570 2024-09-16T00:11:48.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.565+0000 7f70b7fff640 1 -- 172.21.15.139:0/2758019440 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f70c0042070 con 0x7f70d00c6570 2024-09-16T00:11:48.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.565+0000 7f70b7fff640 1 -- 172.21.15.139:0/2758019440 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c0030bb0 con 0x7f70d00c6570 2024-09-16T00:11:48.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.565+0000 7f70d6191640 1 -- 172.21.15.139:0/2758019440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d014fd70 con 0x7f70d00c6570 2024-09-16T00:11:48.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.565+0000 7f70d6191640 1 -- 172.21.15.139:0/2758019440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d0150bd0 con 0x7f70d00c6570 2024-09-16T00:11:48.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.566+0000 7f70d6191640 1 -- 172.21.15.139:0/2758019440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d01511e0 con 0x7f70d00c6570 2024-09-16T00:11:48.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.567+0000 7f70b7fff640 1 -- 172.21.15.139:0/2758019440 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f70c0047460 con 0x7f70d00c6570 2024-09-16T00:11:48.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.567+0000 7f70b7fff640 1 --2- 172.21.15.139:0/2758019440 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f70bc06c2e0 0x7f70bc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:48.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.567+0000 7f70b7fff640 1 -- 172.21.15.139:0/2758019440 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 6030+0+0 (secure 0 0 0) 0x7f70c00ac310 con 0x7f70d00c6570 2024-09-16T00:11:48.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.567+0000 7f70ceffd640 1 --2- 172.21.15.139:0/2758019440 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f70bc06c2e0 0x7f70bc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:48.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.567+0000 7f70ceffd640 1 --2- 172.21.15.139:0/2758019440 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f70bc06c2e0 0x7f70bc06e7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f70d0154cd0 tx=0x7f70c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:48.572 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.571+0000 7f70b7fff640 1 -- 172.21.15.139:0/2758019440 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f70c0045360 con 0x7f70d00c6570 2024-09-16T00:11:48.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:48.740+0000 7f70d6191640 1 -- 172.21.15.139:0/2758019440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f70d0073ce0 con 0x7f70d00c6570 2024-09-16T00:11:49.436 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.435+0000 7f70b7fff640 1 -- 172.21.15.139:0/2758019440 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v148) ==== 181+0+0 (secure 0 0 0) 0x7f70c007a8b0 con 0x7f70d00c6570 2024-09-16T00:11:49.436 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_metadata' removed 2024-09-16T00:11:49.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.439+0000 7f70d6191640 1 -- 172.21.15.139:0/2758019440 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f70bc06c2e0 msgr2=0x7f70bc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:49.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.439+0000 7f70d6191640 1 --2- 172.21.15.139:0/2758019440 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f70bc06c2e0 0x7f70bc06e7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f70d0154cd0 tx=0x7f70c4009040 comp rx=0 tx=0).stop 2024-09-16T00:11:49.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.439+0000 7f70d6191640 1 -- 172.21.15.139:0/2758019440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70d00c6570 msgr2=0x7f70d00cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:49.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.439+0000 7f70d6191640 1 --2- 172.21.15.139:0/2758019440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70d00c6570 0x7f70d00cb810 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f70c0000c00 tx=0x7f70c00046c0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.439+0000 7f70d6191640 1 -- 172.21.15.139:0/2758019440 shutdown_connections 2024-09-16T00:11:49.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.439+0000 7f70d6191640 1 --2- 172.21.15.139:0/2758019440 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f70bc06c2e0 0x7f70bc06e7a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.439+0000 7f70d6191640 1 --2- 172.21.15.139:0/2758019440 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f70d015de10 0x7f70d01f0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.439+0000 7f70d6191640 1 --2- 172.21.15.139:0/2758019440 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f70d00c6950 0x7f70d01f02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.440 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.439+0000 7f70d6191640 1 --2- 172.21.15.139:0/2758019440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70d00c6570 0x7f70d00cb810 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.439+0000 7f70d6191640 1 -- 172.21.15.139:0/2758019440 >> 172.21.15.139:0/2758019440 conn(0x7f70d00d01f0 msgr2=0x7f70d00c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:49.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.439+0000 7f70d6191640 1 -- 172.21.15.139:0/2758019440 shutdown_connections 2024-09-16T00:11:49.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.439+0000 7f70d6191640 1 -- 172.21.15.139:0/2758019440 wait complete. 2024-09-16T00:11:49.454 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T00:11:49.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.558+0000 7f86fea2e640 1 Processor -- start 2024-09-16T00:11:49.559 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.558+0000 7f86fea2e640 1 -- start start 2024-09-16T00:11:49.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.559+0000 7f86fea2e640 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f86f8151690 0x7f86f80c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:49.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.559+0000 7f86fea2e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f86f8151a70 0x7f86f80c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:49.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.559+0000 7f86fea2e640 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86f8072480 con 0x7f86f80c57d0 2024-09-16T00:11:49.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.559+0000 7f86fea2e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f86f80c6f80 con 0x7f86f8151a70 2024-09-16T00:11:49.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.559+0000 7f86fea2e640 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 -- 0x7f86f80c5340 con 0x7f86f8151690 2024-09-16T00:11:49.561 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.559+0000 7f86f77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f86f8151a70 0x7f86f80c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:49.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.559+0000 7f86f77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f86f8151a70 0x7f86f80c3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42408/0 (socket says 172.21.15.139:42408) 2024-09-16T00:11:49.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.559+0000 7f86f77fe640 1 -- 172.21.15.139:0/3508327232 learned_addr learned my addr 172.21.15.139:0/3508327232 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:49.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.559+0000 7f86f7fff640 1 --2- 172.21.15.139:0/3508327232 >> v2:172.21.15.164:3300/0 conn(0x7f86f8151690 0x7f86f80c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:49.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.559+0000 7f86f77fe640 1 -- 172.21.15.139:0/3508327232 >> v2:172.21.15.164:3300/0 conn(0x7f86f8151690 msgr2=0x7f86f80c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:49.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.559+0000 7f86f77fe640 1 --2- 172.21.15.139:0/3508327232 >> v2:172.21.15.164:3300/0 conn(0x7f86f8151690 0x7f86f80c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.560+0000 7f86f77fe640 1 -- 172.21.15.139:0/3508327232 >> v1:172.21.15.164:6789/0 conn(0x7f86f80c57d0 legacy=0x7f86f80c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:11:49.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.560+0000 7f86f77fe640 1 -- 172.21.15.139:0/3508327232 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f8072a70 con 0x7f86f8151a70 2024-09-16T00:11:49.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.560+0000 7f86f77fe640 1 --2- 172.21.15.139:0/3508327232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86f8151a70 0x7f86f80c3700 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f86f80c9d70 tx=0x7f86ec02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2cfeea81756577 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:49.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.560+0000 7f86f6ffd640 1 -- 172.21.15.139:0/3508327232 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec037070 con 0x7f86f8151a70 2024-09-16T00:11:49.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.560+0000 7f86f6ffd640 1 -- 172.21.15.139:0/3508327232 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f86ec032440 con 0x7f86f8151a70 2024-09-16T00:11:49.562 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.560+0000 7f86f6ffd640 1 -- 172.21.15.139:0/3508327232 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec032830 con 0x7f86f8151a70 2024-09-16T00:11:49.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.561+0000 7f86fea2e640 1 -- 172.21.15.139:0/3508327232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86f8151a70 msgr2=0x7f86f80c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:49.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.561+0000 7f86fea2e640 1 --2- 172.21.15.139:0/3508327232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86f8151a70 0x7f86f80c3700 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f86f80c9d70 tx=0x7f86ec02a2c0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.561+0000 7f86fea2e640 1 -- 172.21.15.139:0/3508327232 shutdown_connections 2024-09-16T00:11:49.563 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.561+0000 7f86fea2e640 1 --2- 172.21.15.139:0/3508327232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86f8151a70 0x7f86f80c3700 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.561+0000 7f86fea2e640 1 --2- 172.21.15.139:0/3508327232 >> v2:172.21.15.164:3300/0 conn(0x7f86f8151690 0x7f86f80c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.561+0000 7f86fea2e640 1 -- 172.21.15.139:0/3508327232 >> 172.21.15.139:0/3508327232 conn(0x7f86f806f440 msgr2=0x7f86f806f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:49.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.561+0000 7f86fea2e640 1 -- 172.21.15.139:0/3508327232 shutdown_connections 2024-09-16T00:11:49.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.561+0000 7f86fea2e640 1 -- 172.21.15.139:0/3508327232 wait complete. 2024-09-16T00:11:49.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.562+0000 7f86fea2e640 1 Processor -- start 2024-09-16T00:11:49.564 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.562+0000 7f86fea2e640 1 -- start start 2024-09-16T00:11:49.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.563+0000 7f86fea2e640 1 --2- >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86f80c57d0 0x7f86f8145360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:49.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.563+0000 7f86fea2e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86f8151690 0x7f86f8157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:49.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.563+0000 7f86fea2e640 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f8151a70 0x7f86f8152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:49.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.563+0000 7f86fea2e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f86f80c3c40 con 0x7f86f8151690 2024-09-16T00:11:49.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.563+0000 7f86fea2e640 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 -- 0x7f86f8073030 con 0x7f86f8151a70 2024-09-16T00:11:49.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.563+0000 7f86fea2e640 1 -- --> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] -- mon_getmap magic: 0 -- 0x7f86f8072120 con 0x7f86f80c57d0 2024-09-16T00:11:49.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.563+0000 7f86f77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86f8151690 0x7f86f8157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:49.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.563+0000 7f86f77fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86f8151690 0x7f86f8157320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:42422/0 (socket says 172.21.15.139:42422) 2024-09-16T00:11:49.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.563+0000 7f86f77fe640 1 -- 172.21.15.139:0/1836008586 learned_addr learned my addr 172.21.15.139:0/1836008586 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-16T00:11:49.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.563+0000 7f86fcfcb640 1 --2- 172.21.15.139:0/1836008586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f8151a70 0x7f86f8152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:49.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.563+0000 7f86f7fff640 1 --2- 172.21.15.139:0/1836008586 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86f80c57d0 0x7f86f8145360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:49.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.563+0000 7f86f77fe640 1 -- 172.21.15.139:0/1836008586 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86f80c57d0 msgr2=0x7f86f8145360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:49.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.563+0000 7f86f77fe640 1 --2- 172.21.15.139:0/1836008586 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86f80c57d0 0x7f86f8145360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.563+0000 7f86f77fe640 1 -- 172.21.15.139:0/1836008586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f8151a70 msgr2=0x7f86f8152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:49.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.563+0000 7f86f77fe640 1 --2- 172.21.15.139:0/1836008586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f8151a70 0x7f86f8152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:49.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.563+0000 7f86f77fe640 1 -- 172.21.15.139:0/1836008586 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f81583b0 con 0x7f86f8151690 2024-09-16T00:11:49.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.564+0000 7f86f77fe640 1 --2- 172.21.15.139:0/1836008586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86f8151690 0x7f86f8157320 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0090b0 tx=0x7f86ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:49.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.564+0000 7f86f4ff9640 1 -- 172.21.15.139:0/1836008586 <== mon.0 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec037040 con 0x7f86f8151690 2024-09-16T00:11:49.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.564+0000 7f86f4ff9640 1 -- 172.21.15.139:0/1836008586 <== mon.0 v2:172.21.15.139:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f86ec042070 con 0x7f86f8151690 2024-09-16T00:11:49.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.564+0000 7f86fea2e640 1 -- 172.21.15.139:0/1836008586 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f8153360 con 0x7f86f8151690 2024-09-16T00:11:49.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.564+0000 7f86f4ff9640 1 -- 172.21.15.139:0/1836008586 <== mon.0 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec030bf0 con 0x7f86f8151690 2024-09-16T00:11:49.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.564+0000 7f86fea2e640 1 -- 172.21.15.139:0/1836008586 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f8153900 con 0x7f86f8151690 2024-09-16T00:11:49.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.565+0000 7f86fea2e640 1 -- 172.21.15.139:0/1836008586 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f80c7c90 con 0x7f86f8151690 2024-09-16T00:11:49.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.565+0000 7f86f4ff9640 1 -- 172.21.15.139:0/1836008586 <== mon.0 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f86ec047460 con 0x7f86f8151690 2024-09-16T00:11:49.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.566+0000 7f86f4ff9640 1 --2- 172.21.15.139:0/1836008586 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f86e406c2b0 0x7f86e406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:11:49.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.566+0000 7f86f4ff9640 1 -- 172.21.15.139:0/1836008586 <== mon.0 v2:172.21.15.139:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5592+0+0 (secure 0 0 0) 0x7f86ec0abf60 con 0x7f86f8151690 2024-09-16T00:11:49.568 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.566+0000 7f86f7fff640 1 --2- 172.21.15.139:0/1836008586 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f86e406c2b0 0x7f86e406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:11:49.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.566+0000 7f86f7fff640 1 --2- 172.21.15.139:0/1836008586 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f86e406c2b0 0x7f86e406e770 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f86e800a970 tx=0x7f86e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:11:49.571 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.570+0000 7f86f4ff9640 1 -- 172.21.15.139:0/1836008586 <== mon.0 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f86ec045360 con 0x7f86f8151690 2024-09-16T00:11:49.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:49.737+0000 7f86fea2e640 1 -- 172.21.15.139:0/1836008586 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f86f80c3e90 con 0x7f86f8151690 2024-09-16T00:11:50.451 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:50.450+0000 7f86f4ff9640 1 -- 172.21.15.139:0/1836008586 <== mon.0 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v149) ==== 169+0+0 (secure 0 0 0) 0x7f86ec07a7d0 con 0x7f86f8151690 2024-09-16T00:11:50.451 INFO:teuthology.orchestra.run.smithi139.stderr:pool 'cephfs_data' removed 2024-09-16T00:11:50.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:50.455+0000 7f86fea2e640 1 -- 172.21.15.139:0/1836008586 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f86e406c2b0 msgr2=0x7f86e406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:50.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:50.455+0000 7f86fea2e640 1 --2- 172.21.15.139:0/1836008586 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f86e406c2b0 0x7f86e406e770 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f86e800a970 tx=0x7f86e8008040 comp rx=0 tx=0).stop 2024-09-16T00:11:50.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:50.455+0000 7f86fea2e640 1 -- 172.21.15.139:0/1836008586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86f8151690 msgr2=0x7f86f8157320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:11:50.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:50.455+0000 7f86fea2e640 1 --2- 172.21.15.139:0/1836008586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86f8151690 0x7f86f8157320 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0090b0 tx=0x7f86ec0046c0 comp rx=0 tx=0).stop 2024-09-16T00:11:50.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:50.455+0000 7f86fea2e640 1 -- 172.21.15.139:0/1836008586 shutdown_connections 2024-09-16T00:11:50.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:50.455+0000 7f86fea2e640 1 --2- 172.21.15.139:0/1836008586 >> [v2:172.21.15.139:6824/2998946291,v1:172.21.15.139:6825/2998946291] conn(0x7f86e406c2b0 0x7f86e406e770 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:50.456 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:50.455+0000 7f86fea2e640 1 --2- 172.21.15.139:0/1836008586 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f86f8151a70 0x7f86f8152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:50.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:50.455+0000 7f86fea2e640 1 --2- 172.21.15.139:0/1836008586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86f8151690 0x7f86f8157320 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:50.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:50.455+0000 7f86fea2e640 1 --2- 172.21.15.139:0/1836008586 >> [v2:172.21.15.164:3301/0,v1:172.21.15.164:6790/0] conn(0x7f86f80c57d0 0x7f86f8145360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:11:50.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:50.455+0000 7f86fea2e640 1 -- 172.21.15.139:0/1836008586 >> 172.21.15.139:0/1836008586 conn(0x7f86f806f440 msgr2=0x7f86f80c7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:11:50.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:50.455+0000 7f86fea2e640 1 -- 172.21.15.139:0/1836008586 shutdown_connections 2024-09-16T00:11:50.457 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-16T00:11:50.455+0000 7f86fea2e640 1 -- 172.21.15.139:0/1836008586 wait complete. 2024-09-16T00:11:50.474 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T00:11:50.474 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-16T00:11:50.475 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:50.485 INFO:tasks.ceph.mds.a:Stopped 2024-09-16T00:11:50.485 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-16T00:11:50.485 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:50.534 INFO:tasks.ceph.mds.c:Stopped 2024-09-16T00:11:50.534 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-09-16T00:11:50.534 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:50.584 INFO:tasks.ceph.mds.e:Stopped 2024-09-16T00:11:50.584 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-16T00:11:50.584 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:50.596 INFO:tasks.ceph.mds.b:Stopped 2024-09-16T00:11:50.596 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-16T00:11:50.596 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:50.647 INFO:tasks.ceph.mds.d:Stopped 2024-09-16T00:11:50.647 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-09-16T00:11:50.648 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:50.698 INFO:tasks.ceph.mds.f:Stopped 2024-09-16T00:11:50.698 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T00:11:50.698 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T00:11:50.698 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:50.731 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T00:11:50.731 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T00:11:50.731 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:50.800 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T00:11:50.800 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T00:11:50.800 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:50.866 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T00:11:50.867 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-16T00:11:50.867 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:50.939 INFO:tasks.ceph.osd.3:Stopped 2024-09-16T00:11:50.939 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-16T00:11:50.939 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:50.981 INFO:tasks.ceph.osd.4:Stopped 2024-09-16T00:11:50.982 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-16T00:11:50.982 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:51.035 INFO:tasks.ceph.osd.5:Stopped 2024-09-16T00:11:51.035 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-16T00:11:51.036 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:51.075 INFO:tasks.ceph.osd.6:Stopped 2024-09-16T00:11:51.076 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-16T00:11:51.076 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:51.153 INFO:tasks.ceph.osd.7:Stopped 2024-09-16T00:11:51.153 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T00:11:51.154 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-16T00:11:51.154 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:51.181 INFO:tasks.ceph.mgr.y:Stopped 2024-09-16T00:11:51.181 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T00:11:51.181 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:51.221 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T00:11:51.221 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T00:11:51.222 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T00:11:51.222 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:51.232 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T00:11:51.233 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-16T00:11:51.233 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:51.274 INFO:tasks.ceph.mon.b:Stopped 2024-09-16T00:11:51.274 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-16T00:11:51.274 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:11:51.326 INFO:tasks.ceph.mon.c:Stopped 2024-09-16T00:11:51.326 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T00:11:51.327 DEBUG:teuthology.orchestra.run.smithi139:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v MDS_SLOW_METADATA_IO | egrep -v MDS_TRIM | head -n 1 2024-09-16T00:11:51.368 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T00:11:51.368 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T00:11:51.685 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T00:11:51.685 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T00:11:51.783 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T00:11:51.783 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T00:11:51.827 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T00:11:51.827 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-16T00:11:51.921 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T00:11:51.921 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-16T00:11:52.406 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T00:11:52.406 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-16T00:11:52.544 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T00:11:52.545 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-16T00:11:52.606 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T00:11:52.607 DEBUG:teuthology.orchestra.run.smithi164:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-16T00:11:52.673 INFO:tasks.ceph:Archiving mon data... 2024-09-16T00:11:52.674 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906374/data/mon.a.tgz 2024-09-16T00:11:52.674 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2024-09-16T00:11:52.691 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.hRrjfpcR7H 2024-09-16T00:11:52.691 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.hRrjfpcR7H 2024-09-16T00:11:52.844 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.hRrjfpcR7H 2024-09-16T00:11:52.927 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.hRrjfpcR7H is 449KB 2024-09-16T00:11:52.996 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.hRrjfpcR7H 2024-09-16T00:11:53.012 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906374/data/mon.b.tgz 2024-09-16T00:11:53.012 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-09-16T00:11:53.029 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.Tvrh0QEf1y 2024-09-16T00:11:53.030 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Tvrh0QEf1y 2024-09-16T00:11:53.227 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.Tvrh0QEf1y 2024-09-16T00:11:53.310 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.Tvrh0QEf1y is 620KB 2024-09-16T00:11:53.381 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.Tvrh0QEf1y 2024-09-16T00:11:53.396 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906374/data/mon.c.tgz 2024-09-16T00:11:53.397 DEBUG:teuthology.orchestra.run.smithi164:> mktemp 2024-09-16T00:11:53.451 INFO:teuthology.orchestra.run.smithi164.stdout:/tmp/tmp.nhqS07h6P3 2024-09-16T00:11:53.451 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.nhqS07h6P3 2024-09-16T00:11:53.638 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0666 /tmp/tmp.nhqS07h6P3 2024-09-16T00:11:53.722 DEBUG:teuthology.orchestra.remote:smithi164:/tmp/tmp.nhqS07h6P3 is 620KB 2024-09-16T00:11:53.794 DEBUG:teuthology.orchestra.run.smithi164:> rm -fr /tmp/tmp.nhqS07h6P3 2024-09-16T00:11:53.809 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T00:11:53.810 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T00:11:53.812 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T00:11:53.894 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-16T00:11:53.930 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-16T00:11:53.930 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T00:11:53.932 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906374/remote/smithi139/crash 2024-09-16T00:11:53.933 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T00:11:53.961 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906374/remote/smithi164/crash 2024-09-16T00:11:53.967 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T00:11:53.994 INFO:tasks.ceph:Shutting down logrotate 2024-09-16T00:11:53.995 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:11:54.011 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-16T00:11:54.085 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-16T00:11:54.111 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-16T00:11:54.135 INFO:tasks.ceph:Compressing logs... 2024-09-16T00:11:54.136 DEBUG:teuthology.orchestra.run.smithi139:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:11:54.153 DEBUG:teuthology.orchestra.run.smithi164:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:11:54.215 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.46944.log 2024-09-16T00:11:54.216 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36486.log 2024-09-16T00:11:54.216 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/objectstore_tool.46944.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-16T00:11:54.216 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36486.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.49622.log 2024-09-16T00:11:54.216 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.54610.log 2024-09-16T00:11:54.216 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36399.log 2024-09-16T00:11:54.216 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/objectstore_tool.49622.log: gzip 87.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.36486.log.gz -- 2024-09-16T00:11:54.216 INFO:teuthology.orchestra.run.smithi164.stderr: /var/log/ceph/ceph-client.admin.36279.log 2024-09-16T00:11:54.216 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/objectstore_tool.54610.log: /var/log/ceph/ceph-client.admin.36399.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.53748.log 2024-09-16T00:11:54.216 INFO:teuthology.orchestra.run.smithi164.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36399.log.gz 2024-09-16T00:11:54.216 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36279.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-16T00:11:54.216 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/objectstore_tool.53748.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36279.log.gz 2024-09-16T00:11:54.217 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-16T00:11:54.217 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T00:11:54.217 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph.audit.log: 93.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-16T00:11:54.217 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36439.log 2024-09-16T00:11:54.217 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-16T00:11:54.217 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36439.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36439.log.gz 2024-09-16T00:11:54.217 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.56448.log 2024-09-16T00:11:54.217 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mon.c.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T00:11:54.228 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.48755.log 2024-09-16T00:11:54.240 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/objectstore_tool.56448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36319.log 2024-09-16T00:11:54.247 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/objectstore_tool.48755.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-16T00:11:54.253 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36319.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-16T00:11:54.253 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36319.log.gz 2024-09-16T00:11:54.260 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T00:11:54.261 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-osd.5.log: 94.1% -- replaced with /var/log/ceph/objectstore_tool.49622.log.gz 2024-09-16T00:11:54.273 INFO:teuthology.orchestra.run.smithi164.stderr: 93.9% -- replaced with /var/log/ceph/objectstore_tool.54610.log.gz 2024-09-16T00:11:54.278 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36530.log 2024-09-16T00:11:54.292 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph.log: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T00:11:54.292 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-16T00:11:54.292 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36530.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36530.log.gz 2024-09-16T00:11:54.304 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-16T00:11:54.308 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-16T00:11:54.308 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45122.log 2024-09-16T00:11:54.308 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50350.log 2024-09-16T00:11:54.308 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.45122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42677.log 2024-09-16T00:11:54.308 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.50350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55144.log 2024-09-16T00:11:54.308 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.45122.log.gz 2024-09-16T00:11:54.308 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50350.log.gz 2024-09-16T00:11:54.308 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66786.log 2024-09-16T00:11:54.308 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42677.log: /var/log/ceph/ceph-client.admin.55144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67226.log 2024-09-16T00:11:54.308 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42677.log.gz 2024-09-16T00:11:54.308 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54958.log 2024-09-16T00:11:54.309 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66786.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.55144.log.gz 2024-09-16T00:11:54.309 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67226.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66786.log.gz 2024-09-16T00:11:54.309 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45398.log 2024-09-16T00:11:54.309 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.67226.log.gz 2024-09-16T00:11:54.309 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42761.log 2024-09-16T00:11:54.309 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54958.log.gz 2024-09-16T00:11:54.309 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.45398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41333.log 2024-09-16T00:11:54.309 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42761.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45398.log.gz 2024-09-16T00:11:54.309 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59729.log 2024-09-16T00:11:54.309 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41333.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42761.log.gz 2024-09-16T00:11:54.309 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47438.log 2024-09-16T00:11:54.309 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41333.log.gz 2024-09-16T00:11:54.309 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68346.log 2024-09-16T00:11:54.310 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47438.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59729.log.gz 2024-09-16T00:11:54.310 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65046.log 2024-09-16T00:11:54.310 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47458.log 2024-09-16T00:11:54.310 INFO:teuthology.orchestra.run.smithi139.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.47438.log.gz 2024-09-16T00:11:54.310 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68346.log.gz 2024-09-16T00:11:54.310 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57219.log 2024-09-16T00:11:54.310 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65046.log.gz 2024-09-16T00:11:54.310 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67269.log 2024-09-16T00:11:54.310 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66524.log 2024-09-16T00:11:54.310 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47458.log.gz 2024-09-16T00:11:54.310 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57219.log.gz 2024-09-16T00:11:54.310 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63645.log 2024-09-16T00:11:54.310 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2%/var/log/ceph/ceph-client.admin.66524.log: -- replaced with /var/log/ceph/ceph-client.admin.67269.log.gz 2024-09-16T00:11:54.311 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49348.log 2024-09-16T00:11:54.311 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63645.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66524.log.gz 2024-09-16T00:11:54.311 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59688.log 2024-09-16T00:11:54.311 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48175.log 2024-09-16T00:11:54.311 INFO:teuthology.orchestra.run.smithi139.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.63645.log.gz 2024-09-16T00:11:54.311 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49306.log 2024-09-16T00:11:54.311 INFO:teuthology.orchestra.run.smithi139.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.49348.log.gz 2024-09-16T00:11:54.311 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67403.log 2024-09-16T00:11:54.311 INFO:teuthology.orchestra.run.smithi139.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59688.log.gz 2024-09-16T00:11:54.311 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48175.log.gz 2024-09-16T00:11:54.311 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66212.log 2024-09-16T00:11:54.311 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50392.log 2024-09-16T00:11:54.311 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67403.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49306.log.gz 2024-09-16T00:11:54.311 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52207.log 2024-09-16T00:11:54.312 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66212.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67403.log.gz 2024-09-16T00:11:54.312 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.50392.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66212.log.gz 2024-09-16T00:11:54.312 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45750.log 2024-09-16T00:11:54.312 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50392.log.gz 2024-09-16T00:11:54.312 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55688.log 2024-09-16T00:11:54.312 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.45750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47836.log 2024-09-16T00:11:54.312 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52207.log.gz 2024-09-16T00:11:54.312 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44787.log 2024-09-16T00:11:54.312 INFO:teuthology.orchestra.run.smithi139.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45750.log.gz 2024-09-16T00:11:54.312 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47836.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55688.log.gz 2024-09-16T00:11:54.312 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66211.log 2024-09-16T00:11:54.312 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47836.log.gz 2024-09-16T00:11:54.312 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5/var/log/ceph/ceph-client.admin.44787.log: --verbose -- /var/log/ceph/ceph-client.admin.65232.log 2024-09-16T00:11:54.313 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52345.log 2024-09-16T00:11:54.313 INFO:teuthology.orchestra.run.smithi139.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44787.log.gz 2024-09-16T00:11:54.313 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65232.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66211.log.gz 2024-09-16T00:11:54.313 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56064.log 2024-09-16T00:11:54.313 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65232.log.gz 2024-09-16T00:11:54.313 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67952.log 2024-09-16T00:11:54.313 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52345.log: /var/log/ceph/ceph-client.admin.56064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56935.log 2024-09-16T00:11:54.313 INFO:teuthology.orchestra.run.smithi139.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52345.log.gz 2024-09-16T00:11:54.313 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56064.log.gz 2024-09-16T00:11:54.313 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53075.log 2024-09-16T00:11:54.313 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67952.log: /var/log/ceph/ceph-client.admin.56935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49828.log 2024-09-16T00:11:54.313 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67952.log.gz 2024-09-16T00:11:54.313 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59367.log 2024-09-16T00:11:54.314 INFO:teuthology.orchestra.run.smithi139.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56935.log.gz 2024-09-16T00:11:54.314 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53075.log.gz 2024-09-16T00:11:54.314 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43556.log 2024-09-16T00:11:54.314 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49828.log.gz 2024-09-16T00:11:54.314 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65872.log 2024-09-16T00:11:54.314 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53653.log 2024-09-16T00:11:54.314 INFO:teuthology.orchestra.run.smithi139.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.59367.log.gz 2024-09-16T00:11:54.314 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43556.log.gz 2024-09-16T00:11:54.314 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59368.log 2024-09-16T00:11:54.314 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60732.log 2024-09-16T00:11:54.314 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65872.log.gz 2024-09-16T00:11:54.314 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59368.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-16T00:11:54.314 INFO:teuthology.orchestra.run.smithi139.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.53653.log.gz 2024-09-16T00:11:54.314 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59368.log.gz 2024-09-16T00:11:54.315 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42593.log 2024-09-16T00:11:54.315 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67868.log 2024-09-16T00:11:54.315 INFO:teuthology.orchestra.run.smithi139.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.60732.log.gz 2024-09-16T00:11:54.315 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43598.log 2024-09-16T00:11:54.315 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42593.log.gz 2024-09-16T00:11:54.315 INFO:teuthology.orchestra.run.smithi139.stderr:gzip/var/log/ceph/ceph-client.admin.67868.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54978.log 2024-09-16T00:11:54.315 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64458.log 2024-09-16T00:11:54.315 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67868.log.gz 2024-09-16T00:11:54.315 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54978.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43598.log.gz 2024-09-16T00:11:54.315 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47519.log 2024-09-16T00:11:54.315 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64458.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54978.log.gz 2024-09-16T00:11:54.315 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46759.log 2024-09-16T00:11:54.315 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.64458.log.gz 2024-09-16T00:11:54.316 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63248.log 2024-09-16T00:11:54.316 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47519.log: /var/log/ceph/ceph-client.admin.46759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45241.log 2024-09-16T00:11:54.316 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47519.log.gz 2024-09-16T00:11:54.316 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66210.log 2024-09-16T00:11:54.316 INFO:teuthology.orchestra.run.smithi139.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.46759.log.gz 2024-09-16T00:11:54.316 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63248.log.gz 2024-09-16T00:11:54.316 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.45241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48799.log 2024-09-16T00:11:54.316 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41501.log 2024-09-16T00:11:54.316 INFO:teuthology.orchestra.run.smithi139.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.45241.log.gz 2024-09-16T00:11:54.316 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66210.log.gz 2024-09-16T00:11:54.316 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57770.log 2024-09-16T00:11:54.316 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48279.log 2024-09-16T00:11:54.316 INFO:teuthology.orchestra.run.smithi139.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48799.log.gz 2024-09-16T00:11:54.316 INFO:teuthology.orchestra.run.smithi139.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41501.log.gz 2024-09-16T00:11:54.317 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40871.log 2024-09-16T00:11:54.317 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57770.log.gz 2024-09-16T00:11:54.317 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55987.log 2024-09-16T00:11:54.317 INFO:teuthology.orchestra.run.smithi139.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48279.log.gz 2024-09-16T00:11:54.317 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60998.log 2024-09-16T00:11:54.317 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40871.log: /var/log/ceph/ceph-client.admin.55987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50872.log 2024-09-16T00:11:54.317 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40871.log.gz 2024-09-16T00:11:54.317 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66528.log 2024-09-16T00:11:54.317 INFO:teuthology.orchestra.run.smithi139.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.55987.log.gz 2024-09-16T00:11:54.318 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.50872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66744.log 2024-09-16T00:11:54.318 INFO:teuthology.orchestra.run.smithi139.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.60998.log.gz 2024-09-16T00:11:54.318 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66528.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50872.log.gz 2024-09-16T00:11:54.318 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64076.log 2024-09-16T00:11:54.318 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66528.log.gz 2024-09-16T00:11:54.318 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.66744.log: -- /var/log/ceph/ceph-client.admin.61536.log 2024-09-16T00:11:54.318 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64334.log 2024-09-16T00:11:54.318 INFO:teuthology.orchestra.run.smithi139.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66744.log.gz 2024-09-16T00:11:54.318 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.61536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59806.log 2024-09-16T00:11:54.318 INFO:teuthology.orchestra.run.smithi139.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.64076.log.gz 2024-09-16T00:11:54.318 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64334.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.61536.log.gz 2024-09-16T00:11:54.318 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61775.log 2024-09-16T00:11:54.319 INFO:teuthology.orchestra.run.smithi139.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64334.log.gz 2024-09-16T00:11:54.319 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66197.log 2024-09-16T00:11:54.319 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59806.log: /var/log/ceph/ceph-client.admin.61775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56044.log 2024-09-16T00:11:54.319 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59806.log.gz 2024-09-16T00:11:54.319 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66197.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61775.log.gz 2024-09-16T00:11:54.319 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67100.log 2024-09-16T00:11:54.319 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66197.log.gz 2024-09-16T00:11:54.319 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48942.log 2024-09-16T00:11:54.319 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56044.log: /var/log/ceph/ceph-client.admin.67100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45807.log 2024-09-16T00:11:54.319 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56044.log.gz 2024-09-16T00:11:54.319 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67100.log.gz 2024-09-16T00:11:54.319 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43880.log 2024-09-16T00:11:54.319 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68094.log 2024-09-16T00:11:54.320 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.45807.log: /var/log/ceph/ceph-client.admin.43880.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.48942.log.gz 2024-09-16T00:11:54.320 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41720.log 2024-09-16T00:11:54.320 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45807.log.gz 2024-09-16T00:11:54.320 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51272.log 2024-09-16T00:11:54.320 INFO:teuthology.orchestra.run.smithi139.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.43880.log.gz 2024-09-16T00:11:54.320 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50815.log 88.3% 2024-09-16T00:11:54.320 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68094.log.gz 2024-09-16T00:11:54.320 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41720.log.gz 2024-09-16T00:11:54.320 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67447.log 2024-09-16T00:11:54.320 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51272.log.gz 2024-09-16T00:11:54.320 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53929.log 2024-09-16T00:11:54.320 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.50815.log: /var/log/ceph/ceph-client.admin.67447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67700.log 2024-09-16T00:11:54.320 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53929.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67447.log.gz 2024-09-16T00:11:54.321 INFO:teuthology.orchestra.run.smithi139.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50815.log.gz 2024-09-16T00:11:54.321 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42395.log 2024-09-16T00:11:54.321 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53929.log.gz 2024-09-16T00:11:54.321 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52325.log 2024-09-16T00:11:54.321 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67700.log.gz 2024-09-16T00:11:54.321 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51859.log 2024-09-16T00:11:54.321 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42395.log: /var/log/ceph/ceph-client.admin.52325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41039.log 2024-09-16T00:11:54.321 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42395.log.gz 2024-09-16T00:11:54.321 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51859.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52325.log.gz 2024-09-16T00:11:54.321 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59212.log 2024-09-16T00:11:54.321 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44725.log 2024-09-16T00:11:54.321 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51859.log.gz 2024-09-16T00:11:54.321 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41039.log: /var/log/ceph/ceph-client.admin.59212.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41039.log.gz 2024-09-16T00:11:54.322 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58141.log 2024-09-16T00:11:54.322 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46010.log 2024-09-16T00:11:54.322 INFO:teuthology.orchestra.run.smithi139.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59212.log.gz 2024-09-16T00:11:54.322 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58141.log: gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.65442.log 2024-09-16T00:11:54.322 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44725.log.gz 2024-09-16T00:11:54.322 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58141.log.gz 2024-09-16T00:11:54.322 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66960.log 2024-09-16T00:11:54.322 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54516.log 2024-09-16T00:11:54.322 INFO:teuthology.orchestra.run.smithi139.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.46010.log.gz 2024-09-16T00:11:54.322 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65442.log.gz 2024-09-16T00:11:54.322 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42061.log 2024-09-16T00:11:54.322 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60088.log 2024-09-16T00:11:54.322 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66960.log.gz 2024-09-16T00:11:54.323 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67910.log 2024-09-16T00:11:54.323 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.54516.log.gz 2024-09-16T00:11:54.323 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42061.log.gz 2024-09-16T00:11:54.323 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40723.log 2024-09-16T00:11:54.323 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60088.log.gz 2024-09-16T00:11:54.323 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46583.log 2024-09-16T00:11:54.323 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67910.log.gz 2024-09-16T00:11:54.323 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65358.log 2024-09-16T00:11:54.323 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40723.log: /var/log/ceph/ceph-client.admin.46583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40579.log 2024-09-16T00:11:54.323 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40723.log.gz 2024-09-16T00:11:54.323 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46583.log.gz 2024-09-16T00:11:54.323 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41724.log 2024-09-16T00:11:54.323 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53872.log 2024-09-16T00:11:54.323 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65358.log.gz 2024-09-16T00:11:54.324 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40579.log: /var/log/ceph/ceph-client.admin.41724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60894.log 2024-09-16T00:11:54.324 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40579.log.gz 2024-09-16T00:11:54.324 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42056.log 2024-09-16T00:11:54.324 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53872.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41724.log.gz 2024-09-16T00:11:54.324 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62091.log 2024-09-16T00:11:54.324 INFO:teuthology.orchestra.run.smithi139.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53872.log.gz 2024-09-16T00:11:54.324 INFO:teuthology.orchestra.run.smithi139.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60894.log.gz 2024-09-16T00:11:54.324 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66549.log 2024-09-16T00:11:54.324 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62091.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.42056.log.gz 2024-09-16T00:11:54.324 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41721.log 2024-09-16T00:11:54.324 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62091.log.gz 2024-09-16T00:11:54.324 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52106.log 2024-09-16T00:11:54.324 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66549.log: /var/log/ceph/ceph-client.admin.41721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42391.log 2024-09-16T00:11:54.325 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66549.log.gz 2024-09-16T00:11:54.325 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52106.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41721.log.gz 2024-09-16T00:11:54.325 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53454.log 2024-09-16T00:11:54.325 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52106.log.gz 2024-09-16T00:11:54.325 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43881.log 2024-09-16T00:11:54.325 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42391.log.gz 2024-09-16T00:11:54.325 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57880.log 2024-09-16T00:11:54.325 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53454.log: /var/log/ceph/ceph-client.admin.43881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42803.log 2024-09-16T00:11:54.325 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53454.log.gz 2024-09-16T00:11:54.325 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43881.log.gz 2024-09-16T00:11:54.325 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49671.log 2024-09-16T00:11:54.325 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59889.log 2024-09-16T00:11:54.325 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42803.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.57880.log.gz 2024-09-16T00:11:54.325 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49184.log 2024-09-16T00:11:54.326 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42803.log.gz 2024-09-16T00:11:54.326 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54302.log 2024-09-16T00:11:54.326 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59889.log: /var/log/ceph/ceph-client.admin.49184.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.49671.log.gz 2024-09-16T00:11:54.326 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67742.log 2024-09-16T00:11:54.326 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59889.log.gz 2024-09-16T00:11:54.326 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49184.log.gz 2024-09-16T00:11:54.326 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47195.log 2024-09-16T00:11:54.326 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67742.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T00:11:54.326 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54302.log.gz 2024-09-16T00:11:54.326 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47195.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67742.log.gz 2024-09-16T00:11:54.326 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40293.log 2024-09-16T00:11:54.326 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48841.log 2024-09-16T00:11:54.326 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3%/var/log/ceph/ceph.audit.log: -- replaced with /var/log/ceph/ceph-client.admin.47195.log.gz 2024-09-16T00:11:54.327 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41249.log 2024-09-16T00:11:54.327 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40293.log: /var/log/ceph/ceph-client.admin.48841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52813.log 2024-09-16T00:11:54.327 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40293.log.gz 2024-09-16T00:11:54.327 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48841.log.gz 2024-09-16T00:11:54.327 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65316.log 2024-09-16T00:11:54.327 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5/var/log/ceph/ceph-client.admin.52813.log: --verbose -- /var/log/ceph/ceph-client.admin.50575.log 2024-09-16T00:11:54.327 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41249.log.gz 2024-09-16T00:11:54.327 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58163.log 2024-09-16T00:11:54.327 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52813.log.gz 2024-09-16T00:11:54.327 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T00:11:54.327 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.50575.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65316.log.gz 2024-09-16T00:11:54.327 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64962.log 2024-09-16T00:11:54.327 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52534.log 2024-09-16T00:11:54.327 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58163.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.50575.log.gz 2024-09-16T00:11:54.328 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49487.log 2024-09-16T00:11:54.328 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52534.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.58163.log.gz 2024-09-16T00:11:54.328 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64962.log.gz 2024-09-16T00:11:54.328 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65874.log 2024-09-16T00:11:54.328 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52534.log.gz 2024-09-16T00:11:54.328 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58515.log 2024-09-16T00:11:54.328 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46916.log 2024-09-16T00:11:54.328 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65874.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49487.log.gz 2024-09-16T00:11:54.328 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58515.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65874.log.gz 2024-09-16T00:11:54.328 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63388.log 2024-09-16T00:11:54.328 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58515.log.gz 2024-09-16T00:11:54.328 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67184.log 2024-09-16T00:11:54.328 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44329.log 2024-09-16T00:11:54.329 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46916.log.gz 2024-09-16T00:11:54.329 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41725.log 2024-09-16T00:11:54.329 INFO:teuthology.orchestra.run.smithi139.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.63388.log.gz 2024-09-16T00:11:54.329 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67184.log.gz 2024-09-16T00:11:54.329 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65870.log 2024-09-16T00:11:54.329 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44329.log: /var/log/ceph/ceph-client.admin.41725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67532.log 2024-09-16T00:11:54.329 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65870.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41725.log.gz 2024-09-16T00:11:54.329 INFO:teuthology.orchestra.run.smithi139.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.44329.log.gz 2024-09-16T00:11:54.329 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56230.log 2024-09-16T00:11:54.329 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67532.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65870.log.gz 2024-09-16T00:11:54.329 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48656.log 2024-09-16T00:11:54.329 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67532.log.gz 2024-09-16T00:11:54.329 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51860.log 2024-09-16T00:11:54.330 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56230.log: /var/log/ceph/ceph-client.admin.48656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56569.log 2024-09-16T00:11:54.330 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56230.log.gz 2024-09-16T00:11:54.330 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51860.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48656.log.gz 2024-09-16T00:11:54.330 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60370.log 2024-09-16T00:11:54.330 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66525.log 2024-09-16T00:11:54.330 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51860.log.gz 2024-09-16T00:11:54.330 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56569.log: /var/log/ceph/ceph-client.admin.60370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48963.log 2024-09-16T00:11:54.330 INFO:teuthology.orchestra.run.smithi139.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.56569.log.gz 2024-09-16T00:11:54.330 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50211.log 2024-09-16T00:11:54.330 INFO:teuthology.orchestra.run.smithi139.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.60370.log.gz 2024-09-16T00:11:54.330 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36201.log 2024-09-16T00:11:54.330 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66525.log.gz 2024-09-16T00:11:54.330 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48963.log.gz 2024-09-16T00:11:54.331 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.50211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55084.log 2024-09-16T00:11:54.331 INFO:teuthology.orchestra.run.smithi139.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50211.log.gz 2024-09-16T00:11:54.331 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.36201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45061.log 2024-09-16T00:11:54.331 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41717.log 2024-09-16T00:11:54.331 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55084.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36201.log.gz 2024-09-16T00:11:54.331 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.45061.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55084.log.gzgzip 2024-09-16T00:11:54.331 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54900.log 2024-09-16T00:11:54.331 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45061.log.gz 2024-09-16T00:11:54.331 INFO:teuthology.orchestra.run.smithi139.stderr:gzip/var/log/ceph/ceph-client.admin.41717.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45480.log 2024-09-16T00:11:54.331 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51679.log 2024-09-16T00:11:54.331 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41717.log.gz 2024-09-16T00:11:54.331 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.45480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43682.log 2024-09-16T00:11:54.331 INFO:teuthology.orchestra.run.smithi139.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.54900.log.gz 2024-09-16T00:11:54.331 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45480.log.gz 2024-09-16T00:11:54.332 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65864.log 2024-09-16T00:11:54.332 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60936.log 2024-09-16T00:11:54.332 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51679.log.gz 2024-09-16T00:11:54.332 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65864.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43682.log.gz 2024-09-16T00:11:54.332 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60290.log 2024-09-16T00:11:54.332 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65864.log.gz 2024-09-16T00:11:54.332 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62670.log 2024-09-16T00:11:54.332 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60936.log: /var/log/ceph/ceph-client.admin.60290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65400.log 2024-09-16T00:11:54.332 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60936.log.gz 2024-09-16T00:11:54.332 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60290.log.gz 2024-09-16T00:11:54.332 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56527.log 2024-09-16T00:11:54.332 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65875.log 2024-09-16T00:11:54.332 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65400.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.62670.log.gz 2024-09-16T00:11:54.333 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42059.log 2024-09-16T00:11:54.333 INFO:teuthology.orchestra.run.smithi139.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65400.log.gz 2024-09-16T00:11:54.333 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56527.log.gz 2024-09-16T00:11:54.333 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46540.log 2024-09-16T00:11:54.333 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65875.log: /var/log/ceph/ceph-client.admin.42059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48018.log 2024-09-16T00:11:54.333 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65875.log.gz 2024-09-16T00:11:54.333 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42059.log.gz 2024-09-16T00:11:54.333 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46540.log: /var/log/ceph/ceph-client.admin.48018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52735.log 2024-09-16T00:11:54.333 INFO:teuthology.orchestra.run.smithi139.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.46540.log.gz 2024-09-16T00:11:54.333 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49868.log 2024-09-16T00:11:54.333 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58418.log 2024-09-16T00:11:54.333 INFO:teuthology.orchestra.run.smithi139.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48018.log.gz 2024-09-16T00:11:54.333 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49868.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52735.log.gz 2024-09-16T00:11:54.333 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62966.log 2024-09-16T00:11:54.334 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49868.log.gz 2024-09-16T00:11:54.334 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54455.log 2024-09-16T00:11:54.334 INFO:teuthology.orchestra.run.smithi139.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.58418.log.gz 2024-09-16T00:11:54.334 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42887.log 2024-09-16T00:11:54.334 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62966.log: /var/log/ceph/ceph-client.admin.54455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42389.log 2024-09-16T00:11:54.334 INFO:teuthology.orchestra.run.smithi139.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62966.log.gz 2024-09-16T00:11:54.334 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42887.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54455.log.gz 2024-09-16T00:11:54.334 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62233.log 2024-09-16T00:11:54.334 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50434.log 2024-09-16T00:11:54.334 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42389.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42887.log.gz 2024-09-16T00:11:54.334 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62233.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-16T00:11:54.334 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42389.log.gz 2024-09-16T00:11:54.334 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53136.log 2024-09-16T00:11:54.335 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.50434.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.62233.log.gz 2024-09-16T00:11:54.335 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67142.log 2024-09-16T00:11:54.335 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50434.log.gz 2024-09-16T00:11:54.335 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66828.log 2024-09-16T00:11:54.335 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53136.log: /var/log/ceph/ceph-client.admin.67142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60289.log 2024-09-16T00:11:54.335 INFO:teuthology.orchestra.run.smithi139.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-16T00:11:54.335 INFO:teuthology.orchestra.run.smithi139.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53136.log.gz 2024-09-16T00:11:54.335 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66828.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67142.log.gz 2024-09-16T00:11:54.335 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54394.log 2024-09-16T00:11:54.335 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60289.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66828.log.gz 2024-09-16T00:11:54.335 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43882.log 2024-09-16T00:11:54.335 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60289.log.gz 2024-09-16T00:11:54.335 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49911.log 2024-09-16T00:11:54.335 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54394.log: /var/log/ceph/ceph-client.admin.43882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46998.log 2024-09-16T00:11:54.336 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.54394.log.gz 2024-09-16T00:11:54.336 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43882.log.gz 2024-09-16T00:11:54.336 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53453.log 2024-09-16T00:11:54.336 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49911.log.gz 2024-09-16T00:11:54.336 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60491.log 2024-09-16T00:11:54.336 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.53453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41081.log 2024-09-16T00:11:54.336 INFO:teuthology.orchestra.run.smithi139.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.46998.log.gz 2024-09-16T00:11:54.336 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51984.log 2024-09-16T00:11:54.336 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60491.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.53453.log.gz 2024-09-16T00:11:54.336 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42058.log 2024-09-16T00:11:54.336 INFO:teuthology.orchestra.run.smithi139.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60491.log.gz 2024-09-16T00:11:54.336 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55407.log 2024-09-16T00:11:54.336 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41081.log.gz 2024-09-16T00:11:54.337 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51984.log: /var/log/ceph/ceph-client.admin.42058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65274.log 2024-09-16T00:11:54.337 INFO:teuthology.orchestra.run.smithi139.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51984.log.gz 2024-09-16T00:11:54.337 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63941.log 2024-09-16T00:11:54.337 INFO:teuthology.orchestra.run.smithi139.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42058.log.gz 2024-09-16T00:11:54.337 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55407.log: /var/log/ceph/ceph-client.admin.65274.log: gzip -5 87.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.55407.log.gz 2024-09-16T00:11:54.337 INFO:teuthology.orchestra.run.smithi139.stderr: /var/log/ceph/ceph-client.admin.54071.log 2024-09-16T00:11:54.337 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65274.log.gz 2024-09-16T00:11:54.337 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41543.log 2024-09-16T00:11:54.337 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63941.log: /var/log/ceph/ceph-client.admin.54071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52492.log 2024-09-16T00:11:54.337 INFO:teuthology.orchestra.run.smithi139.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54071.log.gz 2024-09-16T00:11:54.337 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41543.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.63941.log.gz 2024-09-16T00:11:54.337 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48653.log 2024-09-16T00:11:54.337 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68052.log 2024-09-16T00:11:54.337 INFO:teuthology.orchestra.run.smithi139.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41543.log.gz 2024-09-16T00:11:54.338 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52492.log: /var/log/ceph/ceph-client.admin.48653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65869.log 2024-09-16T00:11:54.338 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52492.log.gz 2024-09-16T00:11:54.338 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4%/var/log/ceph/ceph-client.admin.68052.log: -- replaced with /var/log/ceph/ceph-client.admin.48653.log.gz 2024-09-16T00:11:54.338 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41459.log 2024-09-16T00:11:54.338 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68052.log.gz 2024-09-16T00:11:54.338 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57708.log 2024-09-16T00:11:54.338 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65869.log.gz 2024-09-16T00:11:54.338 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65526.log 2024-09-16T00:11:54.338 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41459.log: /var/log/ceph/ceph-client.admin.57708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58651.log 2024-09-16T00:11:54.338 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41459.log.gz 2024-09-16T00:11:54.338 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65526.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57708.log.gz 2024-09-16T00:11:54.338 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68178.log 2024-09-16T00:11:54.338 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65526.log.gz 2024-09-16T00:11:54.339 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67826.log 2024-09-16T00:11:54.339 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41712.log 2024-09-16T00:11:54.339 INFO:teuthology.orchestra.run.smithi139.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.58651.log.gz 2024-09-16T00:11:54.339 INFO:teuthology.orchestra.run.smithi139.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68178.log.gz 2024-09-16T00:11:54.339 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66526.log 2024-09-16T00:11:54.339 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65652.log 2024-09-16T00:11:54.339 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67826.log.gz 2024-09-16T00:11:54.339 INFO:teuthology.orchestra.run.smithi139.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41712.log.gz 2024-09-16T00:11:54.339 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51114.log 2024-09-16T00:11:54.339 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66526.log.gz 2024-09-16T00:11:54.339 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65652.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-16T00:11:54.339 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48259.log 2024-09-16T00:11:54.339 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51114.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65652.log.gz 2024-09-16T00:11:54.339 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57477.log 2024-09-16T00:11:54.340 INFO:teuthology.orchestra.run.smithi139.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.51114.log.gz 2024-09-16T00:11:54.340 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44386.log 2024-09-16T00:11:54.340 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65694.log 2024-09-16T00:11:54.340 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48259.log.gz 2024-09-16T00:11:54.340 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57477.log: /var/log/ceph/ceph-client.admin.44386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66204.log 2024-09-16T00:11:54.340 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57477.log.gz 2024-09-16T00:11:54.340 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65694.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44386.log.gz 2024-09-16T00:11:54.340 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42055.log 2024-09-16T00:11:54.340 INFO:teuthology.orchestra.run.smithi139.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65694.log.gz 2024-09-16T00:11:54.340 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65172.log 2024-09-16T00:11:54.340 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52978.log 2024-09-16T00:11:54.340 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42055.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66204.log.gz 2024-09-16T00:11:54.340 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65172.log: gzip 88.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.42055.log.gz -- 2024-09-16T00:11:54.341 INFO:teuthology.orchestra.run.smithi139.stderr: /var/log/ceph/ceph-client.admin.59807.log 2024-09-16T00:11:54.341 INFO:teuthology.orchestra.run.smithi139.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65172.log.gz 2024-09-16T00:11:54.341 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45626.log 2024-09-16T00:11:54.341 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42393.log 2024-09-16T00:11:54.341 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59807.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.52978.log.gz 2024-09-16T00:11:54.341 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.45626.log: gzip -5 87.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.59807.log.gz -- 2024-09-16T00:11:54.341 INFO:teuthology.orchestra.run.smithi139.stderr: /var/log/ceph/ceph-client.admin.62629.log 2024-09-16T00:11:54.341 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45626.log.gz 2024-09-16T00:11:54.341 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36465.log 2024-09-16T00:11:54.341 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68220.log 2024-09-16T00:11:54.341 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42393.log.gz 2024-09-16T00:11:54.341 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62629.log: /var/log/ceph/ceph-client.admin.36465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61394.log 2024-09-16T00:11:54.341 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62629.log.gz 2024-09-16T00:11:54.341 INFO:teuthology.orchestra.run.smithi139.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36465.log.gz 2024-09-16T00:11:54.342 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42635.log 2024-09-16T00:11:54.342 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43640.log 2024-09-16T00:11:54.342 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.61394.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68220.log.gz 2024-09-16T00:11:54.342 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42635.log: gzip -5 --verbose -- 88.8% /var/log/ceph/ceph-client.admin.67784.log 2024-09-16T00:11:54.342 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61394.log.gz 2024-09-16T00:11:54.342 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42635.log.gz 2024-09-16T00:11:54.342 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42989.log 2024-09-16T00:11:54.342 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42257.log 2024-09-16T00:11:54.342 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67784.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43640.log.gz 2024-09-16T00:11:54.342 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42989.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67784.log.gz 2024-09-16T00:11:54.342 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44081.log 2024-09-16T00:11:54.342 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42257.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42989.log.gz 2024-09-16T00:11:54.342 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42257.log.gz 2024-09-16T00:11:54.343 INFO:teuthology.orchestra.run.smithi139.stderr: /var/log/ceph/ceph-osd.2.log 2024-09-16T00:11:54.343 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40765.log 2024-09-16T00:11:54.343 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.2.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44081.log.gz 2024-09-16T00:11:54.343 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64831.log 2024-09-16T00:11:54.343 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60653.log 2024-09-16T00:11:54.343 INFO:teuthology.orchestra.run.smithi139.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40765.log.gz/var/log/ceph/ceph-client.admin.64831.log: 2024-09-16T00:11:54.343 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45708.log 2024-09-16T00:11:54.343 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.64831.log.gz 2024-09-16T00:11:54.343 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57360.log 2024-09-16T00:11:54.343 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60653.log.gz 2024-09-16T00:11:54.343 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.45708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40955.log 2024-09-16T00:11:54.343 INFO:teuthology.orchestra.run.smithi139.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45708.log.gz 2024-09-16T00:11:54.343 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49954.log 2024-09-16T00:11:54.344 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40955.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57360.log.gz 2024-09-16T00:11:54.344 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54657.log 2024-09-16T00:11:54.344 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40955.log.gz 2024-09-16T00:11:54.344 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55591.log 2024-09-16T00:11:54.344 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54657.log: gzip -5 91.3% -- replaced with /var/log/ceph/ceph-client.admin.49954.log.gz 2024-09-16T00:11:54.344 INFO:teuthology.orchestra.run.smithi139.stderr: --verbose -- /var/log/ceph/ceph-client.admin.68012.log 2024-09-16T00:11:54.344 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54657.log.gz 2024-09-16T00:11:54.344 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63860.log 2024-09-16T00:11:54.344 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68012.log: gzip -5 90.6% -- replaced with /var/log/ceph/ceph-client.admin.55591.log.gz 2024-09-16T00:11:54.344 INFO:teuthology.orchestra.run.smithi139.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51273.log 2024-09-16T00:11:54.344 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63860.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68012.log.gz 2024-09-16T00:11:54.344 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65088.log 2024-09-16T00:11:54.344 INFO:teuthology.orchestra.run.smithi139.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63860.log.gz 2024-09-16T00:11:54.345 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36557.log 2024-09-16T00:11:54.345 INFO:teuthology.orchestra.run.smithi139.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51273.log.gz 2024-09-16T00:11:54.345 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42046.log 2024-09-16T00:11:54.345 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65088.log.gz 2024-09-16T00:11:54.345 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.36557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68262.log 2024-09-16T00:11:54.345 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42046.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36557.log.gz 2024-09-16T00:11:54.345 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41417.log 2024-09-16T00:11:54.345 INFO:teuthology.orchestra.run.smithi139.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42046.log.gz 2024-09-16T00:11:54.345 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58808.log 2024-09-16T00:11:54.345 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9%/var/log/ceph/ceph-client.admin.41417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64915.log 2024-09-16T00:11:54.345 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68262.log.gz 2024-09-16T00:11:54.345 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.58808.log: gzip -5 --verbose -- 88.0% /var/log/ceph/ceph-mgr.y.log 2024-09-16T00:11:54.345 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41417.log.gz 2024-09-16T00:11:54.345 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58808.log.gz 2024-09-16T00:11:54.346 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58954.log 2024-09-16T00:11:54.346 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64915.log: /var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41123.log 2024-09-16T00:11:54.346 INFO:teuthology.orchestra.run.smithi139.stderr:gzip/var/log/ceph/ceph-client.admin.58954.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41723.log 2024-09-16T00:11:54.346 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41123.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.64915.log.gz 2024-09-16T00:11:54.346 INFO:teuthology.orchestra.run.smithi139.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.58954.log.gz 2024-09-16T00:11:54.346 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61297.log 2024-09-16T00:11:54.346 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41123.log.gz 2024-09-16T00:11:54.346 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T00:11:54.346 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41723.log: /var/log/ceph/ceph-client.admin.61297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52653.log 2024-09-16T00:11:54.346 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41723.log.gz 2024-09-16T00:11:54.346 INFO:teuthology.orchestra.run.smithi139.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.61297.log.gz 2024-09-16T00:11:54.346 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55211.log 2024-09-16T00:11:54.346 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52653.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52653.log.gz 2024-09-16T00:11:54.347 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55325.log 2024-09-16T00:11:54.347 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55211.log: 85.4% -- replaced with /var/log/ceph/ceph.log.gzgzip 2024-09-16T00:11:54.347 INFO:teuthology.orchestra.run.smithi139.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42054.log 2024-09-16T00:11:54.347 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55211.log.gz 2024-09-16T00:11:54.347 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67357.log 2024-09-16T00:11:54.347 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55325.log: /var/log/ceph/ceph-client.admin.42054.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55325.log.gz 2024-09-16T00:11:54.347 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44166.log 2024-09-16T00:11:54.347 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42054.log.gz 2024-09-16T00:11:54.347 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41165.log 2024-09-16T00:11:54.347 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67357.log: /var/log/ceph/ceph-client.admin.44166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54456.log 2024-09-16T00:11:54.347 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41165.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67357.log.gz 2024-09-16T00:11:54.347 INFO:teuthology.orchestra.run.smithi139.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44166.log.gz 2024-09-16T00:11:54.347 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41165.log.gz 2024-09-16T00:11:54.347 INFO:teuthology.orchestra.run.smithi139.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49389.log 2024-09-16T00:11:54.348 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43418.log 2024-09-16T00:11:54.348 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54456.log.gz 2024-09-16T00:11:54.348 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52450.log 2024-09-16T00:11:54.348 INFO:teuthology.orchestra.run.smithi139.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49389.log.gz 2024-09-16T00:11:54.348 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.0.43418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40827.log 2024-09-16T00:11:54.348 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59970.log 2024-09-16T00:11:54.348 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.52450.log: /var/log/ceph/ceph-client.admin.40827.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52450.log.gz 2024-09-16T00:11:54.348 INFO:teuthology.orchestra.run.smithi139.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40827.log.gz 2024-09-16T00:11:54.348 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61994.log 2024-09-16T00:11:54.348 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55487.log 2024-09-16T00:11:54.348 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59970.log: /var/log/ceph/ceph-client.admin.61994.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.59970.log.gz 2024-09-16T00:11:54.348 INFO:teuthology.orchestra.run.smithi139.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.61994.log.gz 2024-09-16T00:11:54.348 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57155.log 2024-09-16T00:11:54.349 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55487.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55487.log.gz 2024-09-16T00:11:54.349 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42719.log 2024-09-16T00:11:54.349 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51427.log 2024-09-16T00:11:54.349 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.57155.log: /var/log/ceph/ceph-client.admin.42719.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.57155.log.gz 2024-09-16T00:11:54.349 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42719.log.gz 2024-09-16T00:11:54.349 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65484.log 2024-09-16T00:11:54.349 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64553.log 2024-09-16T00:11:54.349 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51427.log: /var/log/ceph/ceph-client.admin.65484.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51427.log.gz 2024-09-16T00:11:54.349 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65484.log.gz 2024-09-16T00:11:54.349 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51211.log 2024-09-16T00:11:54.349 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64553.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64553.log.gz 2024-09-16T00:11:54.349 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42397.log 2024-09-16T00:11:54.349 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66208.log 2024-09-16T00:11:54.350 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51211.log: /var/log/ceph/ceph-client.admin.42397.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51211.log.gz 2024-09-16T00:11:54.350 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42397.log.gz 2024-09-16T00:11:54.350 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45563.log 2024-09-16T00:11:54.350 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66208.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66208.log.gz 2024-09-16T00:11:54.350 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67616.log 2024-09-16T00:11:54.350 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41722.log 2024-09-16T00:11:54.350 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.45563.log: /var/log/ceph/ceph-client.admin.67616.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45563.log.gz 2024-09-16T00:11:54.350 INFO:teuthology.orchestra.run.smithi139.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49053.log 2024-09-16T00:11:54.350 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67616.log.gz 2024-09-16T00:11:54.350 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41722.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41722.log.gz 2024-09-16T00:11:54.350 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64172.log 2024-09-16T00:11:54.350 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60633.log 2024-09-16T00:11:54.350 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49053.log: /var/log/ceph/ceph-client.admin.64172.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49053.log.gz 2024-09-16T00:11:54.350 INFO:teuthology.orchestra.run.smithi139.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64172.log.gz 2024-09-16T00:11:54.351 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67574.log 2024-09-16T00:11:54.351 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60633.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60633.log.gz 2024-09-16T00:11:54.351 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50643.log 2024-09-16T00:11:54.351 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42060.log 2024-09-16T00:11:54.351 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67574.log: /var/log/ceph/ceph-client.admin.50643.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67574.log.gz 2024-09-16T00:11:54.351 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50643.log.gz 2024-09-16T00:11:54.351 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65004.log 2024-09-16T00:11:54.351 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61474.log 2024-09-16T00:11:54.351 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42060.log: /var/log/ceph/ceph-client.admin.65004.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42060.log.gz 2024-09-16T00:11:54.351 INFO:teuthology.orchestra.run.smithi139.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65004.log.gz 2024-09-16T00:11:54.351 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56829.log 2024-09-16T00:11:54.351 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.61474.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61474.log.gz 2024-09-16T00:11:54.351 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66523.log 2024-09-16T00:11:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65868.log 2024-09-16T00:11:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56829.log: /var/log/ceph/ceph-client.admin.66523.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56829.log.gz 2024-09-16T00:11:54.352 INFO:teuthology.orchestra.run.smithi139.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66523.log.gz 2024-09-16T00:11:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65568.log 2024-09-16T00:11:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56146.log 2024-09-16T00:11:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65868.log: /var/log/ceph/ceph-client.admin.65568.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65868.log.gz 2024-09-16T00:11:54.352 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65568.log.gz 2024-09-16T00:11:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42396.log 2024-09-16T00:11:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56146.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56146.log.gz 2024-09-16T00:11:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53790.log 2024-09-16T00:11:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55042.log 2024-09-16T00:11:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42396.log: /var/log/ceph/ceph-client.admin.53790.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42396.log.gz 2024-09-16T00:11:54.352 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53790.log.gz 2024-09-16T00:11:54.352 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45338.log 2024-09-16T00:11:54.353 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55768.log 2024-09-16T00:11:54.353 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55042.log: /var/log/ceph/ceph-client.admin.45338.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55042.log.gz 2024-09-16T00:11:54.353 INFO:teuthology.orchestra.run.smithi139.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45338.log.gz 2024-09-16T00:11:54.353 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50292.log 2024-09-16T00:11:54.353 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44548.log 2024-09-16T00:11:54.353 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.55768.log: /var/log/ceph/ceph-client.admin.50292.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.55768.log.gz 2024-09-16T00:11:54.353 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50292.log.gz 2024-09-16T00:11:54.353 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51882.log 2024-09-16T00:11:54.353 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.44548.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.44548.log.gz 2024-09-16T00:11:54.353 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46188.log 2024-09-16T00:11:54.353 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47295.log 2024-09-16T00:11:54.353 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51882.log: /var/log/ceph/ceph-client.admin.46188.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51882.log.gz 2024-09-16T00:11:54.353 INFO:teuthology.orchestra.run.smithi139.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46188.log.gz 2024-09-16T00:11:54.354 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65876.log 2024-09-16T00:11:54.354 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47295.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.47295.log.gz 2024-09-16T00:11:54.354 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54233.log 2024-09-16T00:11:54.354 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43744.log 2024-09-16T00:11:54.354 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65876.log: /var/log/ceph/ceph-client.admin.54233.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65876.log.gz 2024-09-16T00:11:54.354 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47779.log 2024-09-16T00:11:54.354 INFO:teuthology.orchestra.run.smithi139.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.54233.log.gz 2024-09-16T00:11:54.354 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.43744.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.43744.log.gz 2024-09-16T00:11:54.354 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54819.log 2024-09-16T00:11:54.354 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40997.log 2024-09-16T00:11:54.354 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.47779.log: /var/log/ceph/ceph-client.admin.54819.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47779.log.gz 2024-09-16T00:11:54.354 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54819.log.gz 2024-09-16T00:11:54.355 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42394.log 2024-09-16T00:11:54.355 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49447.log 2024-09-16T00:11:54.355 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40997.log: /var/log/ceph/ceph-client.admin.42394.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40997.log.gz 2024-09-16T00:11:54.355 INFO:teuthology.orchestra.run.smithi139.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42845.log 2024-09-16T00:11:54.355 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42394.log.gz 2024-09-16T00:11:54.355 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.49447.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49447.log.gz 2024-09-16T00:11:54.355 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64873.log 2024-09-16T00:11:54.355 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65610.log 2024-09-16T00:11:54.355 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.42845.log: /var/log/ceph/ceph-client.admin.64873.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42845.log.gz 2024-09-16T00:11:54.355 INFO:teuthology.orchestra.run.smithi139.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64873.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41291.log 2024-09-16T00:11:54.355 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-09-16T00:11:54.355 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65610.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65610.log.gz 2024-09-16T00:11:54.355 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67658.log 2024-09-16T00:11:54.355 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68136.log 2024-09-16T00:11:54.356 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41291.log: /var/log/ceph/ceph-client.admin.67658.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41291.log.gz 2024-09-16T00:11:54.356 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67658.log.gz 2024-09-16T00:11:54.356 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56371.log 2024-09-16T00:11:54.356 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68136.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68136.log.gz 2024-09-16T00:11:54.356 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61074.log 2024-09-16T00:11:54.356 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40464.log 2024-09-16T00:11:54.356 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.56371.log: /var/log/ceph/ceph-client.admin.61074.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61074.log.gz 2024-09-16T00:11:54.356 INFO:teuthology.orchestra.run.smithi139.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.56371.log.gz 2024-09-16T00:11:54.356 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52554.log 2024-09-16T00:11:54.356 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-09-16T00:11:54.356 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40464.log: /var/log/ceph/ceph-client.admin.52554.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40464.log.gz 2024-09-16T00:11:54.356 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51356.log 2024-09-16T00:11:54.356 INFO:teuthology.orchestra.run.smithi139.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.52554.log.gz 2024-09-16T00:11:54.357 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mds.e.log: 93.2% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-09-16T00:11:54.357 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61816.log 2024-09-16T00:11:54.357 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46147.log 2024-09-16T00:11:54.357 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.51356.log: /var/log/ceph/ceph-client.admin.61816.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51356.log.gz 2024-09-16T00:11:54.357 INFO:teuthology.orchestra.run.smithi139.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.61816.log.gz 2024-09-16T00:11:54.357 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48418.log 2024-09-16T00:11:54.357 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46147.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46147.log.gz 2024-09-16T00:11:54.357 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48739.log 2024-09-16T00:11:54.365 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mds.d.log: 93.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36359.log 2024-09-16T00:11:54.365 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-16T00:11:54.365 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mgr.x.log: 93.4% 94.1% -- replaced with /var/log/ceph/objectstore_tool.48755.log.gz 2024-09-16T00:11:54.365 INFO:teuthology.orchestra.run.smithi164.stderr: -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-16T00:11:54.365 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-16T00:11:54.365 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36359.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36359.log.gz 2024-09-16T00:11:54.365 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36570.log 2024-09-16T00:11:54.365 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.47827.log 2024-09-16T00:11:54.366 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/ceph-client.admin.36570.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36570.log.gz 2024-09-16T00:11:54.366 INFO:teuthology.orchestra.run.smithi164.stderr: 93.8% -- replaced with /var/log/ceph/objectstore_tool.56448.log.gz 2024-09-16T00:11:54.366 INFO:teuthology.orchestra.run.smithi164.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.55490.log 2024-09-16T00:11:54.369 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/objectstore_tool.47827.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-09-16T00:11:54.372 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66408.log 2024-09-16T00:11:54.372 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48418.log: /var/log/ceph/ceph-client.admin.48739.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48418.log.gz 2024-09-16T00:11:54.372 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30149.log 2024-09-16T00:11:54.372 INFO:teuthology.orchestra.run.smithi139.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.48739.log.gz 2024-09-16T00:11:54.372 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66408.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66408.log.gz 2024-09-16T00:11:54.372 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41375.log 2024-09-16T00:11:54.372 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63583.log 2024-09-16T00:11:54.372 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph.tmp-client.admin.30149.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30149.log.gz 2024-09-16T00:11:54.372 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65130.log 2024-09-16T00:11:54.373 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41375.log.gz 2024-09-16T00:11:54.373 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.63583.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63583.log.gz 2024-09-16T00:11:54.373 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62310.log 2024-09-16T00:11:54.373 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66207.log 2024-09-16T00:11:54.373 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.65130.log: /var/log/ceph/ceph-client.admin.62310.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65130.log.gz 2024-09-16T00:11:54.373 INFO:teuthology.orchestra.run.smithi139.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62310.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40681.log 2024-09-16T00:11:54.373 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-09-16T00:11:54.373 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66207.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66207.log.gz 2024-09-16T00:11:54.373 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-16T00:11:54.373 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59071.log 2024-09-16T00:11:54.373 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40681.log: /var/log/ceph/ceph-osd.3.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40681.log.gz 2024-09-16T00:11:54.373 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60425.log 2024-09-16T00:11:54.373 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.59071.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59071.log.gz 2024-09-16T00:11:54.374 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67489.log 2024-09-16T00:11:54.374 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.60425.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60425.log.gz 2024-09-16T00:11:54.374 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66902.log 2024-09-16T00:11:54.374 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67489.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67489.log.gz 2024-09-16T00:11:54.374 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-16T00:11:54.374 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66902.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66902.log.gz 2024-09-16T00:11:54.374 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66527.log 2024-09-16T00:11:54.374 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mds.c.log: 93.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-16T00:11:54.374 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54051.log 2024-09-16T00:11:54.374 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66527.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66527.log.gz 2024-09-16T00:11:54.374 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68304.log 2024-09-16T00:11:54.374 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.54051.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54051.log.gz 2024-09-16T00:11:54.374 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46265.log 2024-09-16T00:11:54.374 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.68304.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68304.log.gz 2024-09-16T00:11:54.375 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66205.log 2024-09-16T00:11:54.375 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.46265.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46265.log.gz 2024-09-16T00:11:54.375 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48213.log 2024-09-16T00:11:54.375 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.66205.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66205.log.gz 2024-09-16T00:11:54.375 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67311.log 2024-09-16T00:11:54.375 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.48213.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48213.log.gz 2024-09-16T00:11:54.375 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41207.log 2024-09-16T00:11:54.375 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.67311.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67311.log.gz 2024-09-16T00:11:54.375 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40913.log 2024-09-16T00:11:54.375 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.41207.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41207.log.gz 2024-09-16T00:11:54.375 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62492.log 2024-09-16T00:11:54.375 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.40913.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40913.log.gz 2024-09-16T00:11:54.375 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64232.log 2024-09-16T00:11:54.375 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.62492.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.62492.log.gz 2024-09-16T00:11:54.376 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.64232.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.64232.log.gz 2024-09-16T00:11:54.376 INFO:teuthology.orchestra.run.smithi139.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-16T00:11:54.388 INFO:teuthology.orchestra.run.smithi164.stderr:/var/log/ceph/objectstore_tool.55490.log: /var/log/ceph/ceph-mds.f.log: 91.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-16T00:11:54.402 INFO:teuthology.orchestra.run.smithi164.stderr: 94.7% -- replaced with /var/log/ceph/objectstore_tool.46944.log.gz 2024-09-16T00:11:54.420 INFO:teuthology.orchestra.run.smithi164.stderr: 94.1% -- replaced with /var/log/ceph/objectstore_tool.47827.log.gz 2024-09-16T00:11:54.433 INFO:teuthology.orchestra.run.smithi164.stderr: 94.7% -- replaced with /var/log/ceph/objectstore_tool.53748.log.gz 2024-09-16T00:11:54.458 INFO:teuthology.orchestra.run.smithi164.stderr: 93.8% -- replaced with /var/log/ceph/objectstore_tool.55490.log.gz 2024-09-16T00:11:54.553 INFO:teuthology.orchestra.run.smithi164.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-16T00:11:54.568 INFO:teuthology.orchestra.run.smithi139.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-16T00:11:57.452 INFO:teuthology.orchestra.run.smithi164.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-16T00:11:57.470 INFO:teuthology.orchestra.run.smithi139.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-16T00:11:58.030 INFO:teuthology.orchestra.run.smithi164.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-16T00:11:58.213 INFO:teuthology.orchestra.run.smithi139.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-16T00:11:58.407 INFO:teuthology.orchestra.run.smithi139.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-16T00:11:58.748 INFO:teuthology.orchestra.run.smithi164.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-16T00:11:59.597 INFO:teuthology.orchestra.run.smithi139.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-16T00:11:59.687 INFO:teuthology.orchestra.run.smithi164.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-16T00:12:08.216 INFO:teuthology.orchestra.run.smithi139.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.43418.log.gz 2024-09-16T00:12:08.217 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-09-16T00:12:08.217 INFO:teuthology.orchestra.run.smithi139.stderr:real 0m14.052s 2024-09-16T00:12:08.217 INFO:teuthology.orchestra.run.smithi139.stderr:user 0m30.191s 2024-09-16T00:12:08.218 INFO:teuthology.orchestra.run.smithi139.stderr:sys 0m1.269s 2024-09-16T00:12:40.546 INFO:teuthology.orchestra.run.smithi164.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-09-16T00:12:40.546 INFO:teuthology.orchestra.run.smithi164.stderr: 2024-09-16T00:12:40.547 INFO:teuthology.orchestra.run.smithi164.stderr:real 0m46.350s 2024-09-16T00:12:40.547 INFO:teuthology.orchestra.run.smithi164.stderr:user 1m2.101s 2024-09-16T00:12:40.547 INFO:teuthology.orchestra.run.smithi164.stderr:sys 0m2.024s 2024-09-16T00:12:40.547 INFO:tasks.ceph:Archiving logs... 2024-09-16T00:12:40.547 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906374/remote/smithi139/log 2024-09-16T00:12:40.549 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:12:45.541 DEBUG:teuthology.misc:Transferring archived files from smithi164:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906374/remote/smithi164/log 2024-09-16T00:12:45.541 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:12:54.954 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T00:12:54.969 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T00:12:54.970 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:12:54.972 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:12:55.049 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T00:12:55.058 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T00:12:55.058 DEBUG:teuthology.orchestra.run.smithi139:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:12:55.061 DEBUG:teuthology.orchestra.run.smithi164:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:12:55.077 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:12:55.078 INFO:teuthology.orchestra.run.smithi164.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:12:55.079 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:12:55.079 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2024-09-16T00:12:55.079 INFO:teuthology.orchestra.run.smithi164.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 +121us[ +120us] +/- 50ms 2024-09-16T00:12:55.080 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 20 -1199us[-1200us] +/- 63ms 2024-09-16T00:12:55.080 INFO:teuthology.orchestra.run.smithi164.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 +1171us[+1171us] +/- 72ms 2024-09-16T00:12:55.080 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:12:55.081 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:12:55.081 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-09-16T00:12:55.081 INFO:teuthology.orchestra.run.smithi139.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 +113us[ +99us] +/- 50ms 2024-09-16T00:12:55.081 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 21 -1213us[-1213us] +/- 63ms 2024-09-16T00:12:55.081 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 20 +1148us[+1148us] +/- 72ms 2024-09-16T00:12:55.082 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:12:55.082 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T00:12:55.094 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T00:12:55.094 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T00:12:55.200 DEBUG:teuthology.orchestra.run.smithi139:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:12:55.288 DEBUG:teuthology.orchestra.run.smithi164:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:12:55.360 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:12:55.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:12:55.392 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:12:55.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:12:55.421 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T00:12:55.430 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T00:12:55.449 INFO:teuthology.task.internal:Duration was 702.976866 seconds 2024-09-16T00:12:55.449 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T00:12:55.463 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T00:12:55.463 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:12:55.466 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:12:55.504 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:12:55.505 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:12:55.994 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T00:12:55.995 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi139.front.sepia.ceph.com 2024-09-16T00:12:55.995 DEBUG:teuthology.orchestra.run.smithi139:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:12:56.056 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi164.front.sepia.ceph.com 2024-09-16T00:12:56.056 DEBUG:teuthology.orchestra.run.smithi164:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:12:56.078 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T00:12:56.078 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:12:56.103 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:12:56.145 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T00:12:56.146 DEBUG:teuthology.orchestra.run.smithi139:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:12:56.171 DEBUG:teuthology.orchestra.run.smithi164:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:12:56.306 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T00:12:56.316 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T00:12:56.316 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:12:56.341 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:12:56.366 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T00:12:56.378 DEBUG:teuthology.orchestra.run.smithi139:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:12:56.384 DEBUG:teuthology.orchestra.run.smithi164:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:12:56.405 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = core 2024-09-16T00:12:56.429 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = core 2024-09-16T00:12:56.455 DEBUG:teuthology.orchestra.run.smithi139:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:12:56.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:12:56.473 DEBUG:teuthology.orchestra.run.smithi164:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:12:56.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:12:56.510 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T00:12:56.520 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T00:12:56.521 DEBUG:teuthology.misc:Transferring archived files from smithi139:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906374/remote/smithi139 2024-09-16T00:12:56.521 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:12:56.557 DEBUG:teuthology.misc:Transferring archived files from smithi164:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906374/remote/smithi164 2024-09-16T00:12:56.557 DEBUG:teuthology.orchestra.run.smithi164:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:12:56.590 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T00:12:56.591 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:12:56.593 DEBUG:teuthology.orchestra.run.smithi164:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:12:56.641 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T00:12:56.652 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T00:12:56.652 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T00:12:56.665 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T00:12:56.666 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:12:56.668 DEBUG:teuthology.orchestra.run.smithi164:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:12:56.680 INFO:teuthology.orchestra.run.smithi139.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:12 /home/ubuntu/cephtest 2024-09-16T00:12:56.696 INFO:teuthology.orchestra.run.smithi164.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:12 /home/ubuntu/cephtest 2024-09-16T00:12:56.697 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T00:12:56.711 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T00:12:56.743 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/osd 2-workunit/suites/fsstress}} duration: 702.9768664836884 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-16T00:12:56.743 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T00:12:56.821 INFO:teuthology.run:pass